From d0dac38ea8343b44708179bb4f55ac191f15542c Mon Sep 17 00:00:00 2001 From: Brad Chase Date: Sat, 2 Dec 2017 13:44:49 -0500 Subject: [PATCH] Squashed 'src/rocksdb2/' changes from 1fdd726a8..aead40417 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit aead40417 fix HISTORY.md typo a0cdc3cec Bump version to 5.8.7 7513f6350 Fix IOError on WAL write doesn't propagate to write group follower 9e47084ce Bump version to 5.8.6 36074ba5d Enable cacheline_aligned_alloc() to allocate from jemalloc if enabled. aa00523e0 Add -DPORTABLE=1 to MSVC CI build cf2b98237 Bump version to 5.8.5 e8c9350f2 Blob DB: not using PinnableSlice move assignment 4907d2463 Bump version to 5.8.4 5d928c795 Blob DB: Fix race condition between flush and write 725bb9d66 Blob DB: Fix release build b7367fe84 Bump version to 5.8.3 13b2a9b6f Blob DB: use compression in file header instead of global options 5dc70a15c Fix PinnableSlice move assignment 9019e9125 dynamically change current memtable size 7f1815c37 Bump version to 5.8.2 2584a18ef Blob DB: Fix BlobDBTest::SnapshotAndGarbageCollection asan failure 17f67b546 PinnableSlice move assignment 6fb56c582 Blob DB: Add compaction filter to remove expired blob index entries f90ced92f Blob DB: fix snapshot handling 632f36dcd Blob DB: option to enable garbage collection 11bacd578 Blob DB: Fix flaky BlobDBTest::GCExpiredKeyWhileOverwriting test f98efcb1e Blob DB: Evict oldest blob file when close to blob db size limit c1e99eddc Blob DB: cleanup unused options ffc3c62ca Blob DB: Initialize all fields in Blob Header, Footer and Record structs 9e8254090 Blob DB: update blob file format d66bb21e1 Blob DB: Inline small values in base DB 05d5c575a Return write error on reaching blob dir size limit 2b8893b9e Blob DB: Store blob index as kTypeBlobIndex in base db 419b93c56 Blob DB: not writing sequence number as blob record footer 8afb0036c fix lite build dded348dd Blob DB: Move BlobFile definition to a separate file 374736123 add GetLiveFiles and GetLiveFilesMetaData for BlobDB 8cff6e945 Enable WAL for blob index c29347290 Add ValueType::kTypeBlobIndex eae53de3b Make it explicit blob db doesn't support CF 65aec19df Fix memory leak on blob db open 30b38c98c TableProperty::oldest_key_time defaults to 0 2879f4beb Bump version to 5.8.1 88595c882 Add DB::Properties::kEstimateOldestKeyTime 266ac245a Bumping version to 5.8 64185c23a update HISTORY.md for DeleteRange bug fix e83d6a02e Not using aligned_alloc with gcc4 + asan 0980dc6c9 Fix wrong smallest key of delete range tombstones b76797231 avoid use-after-move error c41744270 CMake: Fix formatting c21ea8f7a CMake: Add support for CMake packages 544434558 add Erlang to the list of language bindings 2972a702a Minor updates to FlushWAL blog fbfa3e7a4 WriteAtPrepare: Efficient read from snapshot list b01f426f5 Blog post for FlushWAL 503db684f make blob file close synchronous 3c840d1a6 Allow DB reopen with reduced options.num_levels 92bfd6c50 Fix DropColumnFamily data race 7fdf735d5 Pinnableslice examples and blog post 7fbb9ecca support disabling checksum in block-based table 19cc66dc4 fix clang bug in block-based table reader 7eba54eb9 test compaction input-level split range tombstone assumption cd26af347 Add unit test for WritePrepared skeleton a12479819 Improved transactions support in C API c10b39131 LANGUAGE-BINDINGS.md: add another rust binding 90177432e Remove leftover references to phutil_module_cache 234f33a3f allow nullptr Slice only as sentinel ccf7f833e Use PinnableSlice in Transactions 1dfcdb15f Extend pin_l0 to filter partitions 39ef90055 stop calling memcmp with nullptrs 78cb6b611 Provide byte[] version of SstFileWriter.merge to reduce GC Stall 867fe92e5 Scale histogram bucket size by constant factor f004307e9 CMake improvements 09ac6206a Circumvent ASAN false positive 5b68b114f Blob db create a snapshot before every read 4624ae52c GC the oldest file when out of space 8ace1f79b add counter for deletion dropping optimization 0d8e992b4 Revert the mistake in version update 5358a8056 add VerifyChecksum to HISTORY.md ed0a4c93e perf_context measure user bytes read 1efc600dd Preload l0 index partitions bddd5d363 Added mechanism to track deadlock chain c1384a707 fix db_stress uint64_t to int32 cast 29877ec7b Fix blob db crash during calculating write amp 8f2598ac9 Enable Cassandra merge operator to be called with a single merge operand 9a44b4c32 Allow merge operator to be called even with a single operand ac8fb77af fix some misspellings 23593171c minor improvements to db_stress af012c0f8 fix deleterange with memtable prefix bloom 1c8dbe2aa update scores after picking universal compaction eb6425303 Update WritePrepared with the pseudo code 132306fbf Remove PartialMerge implementation from Cassandra merge operator 71598cdc7 Fix false removal of tombstone issue in FIFO and kCompactionStyleNone 3204a4f64 Fix missing stdlib include required for abort() 7aa96db7a db_stress rolling active window dfa6c23c4 Update RocksDBCommonHelper to use escapeshellarg e367774d1 Overload new[] to properly align LRUCacheShard ad42d2fcb Remove residual arcanist_util directory 279296f4d properly set C[XX]FLAGS during CMake configure-time checks c5f0c6cc6 compile with correct flags to determine SSE4.2 support 185ade4c0 cmake: support more compression type 5449c0990 rocksdb: make buildable on aarch64 a144a9782 Fix for CMakeLists.txt on Windows for RocksJava acf935e40 fix deletion dropping in intra-L0 8254e9b57 make sst_dump compression size command consistent 74f18c130 db_bench support for non-uniform column family ops 5de98f2d5 approximate histogram stats to save cpu 3f5888430 Fix c_test ASAN failure e5a1b727c Fix blob DB transaction usage while GC 6f051e0c7 fix corruption_test valgrind ac098a462 expose set_skip_stats_update_on_db_open to C bindings 666a005f9 Support prefetch last 512KB with direct I/O in block based file reader ad77ee0ea Revert "Makefile: correct faligned-new test" b87ee6f77 Use more keys per lock in daily TSAN crash test 25df24254 Add column families related functions (C API) 64f848435 block_cache_tier: fix gcc-7 warnings 0cecf8155 Write batch for `TransactionDB` in C API 6a9de4347 Windows.h macro call fix 23c7d1354 fix comment 1fbad84b6 Makefile: correct faligned-new test 7848f0b24 add VerifyChecksum() to db.h 47ed3bfc3 fix WinEnv assertions d97a72d63 Try to repair db with wal_dir option, avoid leak some WAL files 36375de76 gcc-7/i386: markup intentional fallthroughs bdc056f8a Refactor PessimisticTransaction a9a4e89c3 Fix valgrind complaint about initialization 4ca11b4b0 Update USERS.md c9804e007 Refactor TransactionDBImpl 20dc5e74f Optimize range-delete aggregator call in merge helper. 0d4a2b733 Avoid blob db call Sync() while writing 627c9f1ab Don't add -ljemalloc when DISABLE_JEMALLOC is set dce6d5a83 db_bench background work thread pool size arguments 4f81ab38b Makefile: fix for GCC 7+ and clang 4+ 92afe830f Update all blob db TTL and timestamps to uint64_t 5883a1ae2 Fix /bin/bash shebangs cc01985db Introduce bottom-pri thread pool for large universal compactions 0b814ba92 Allow concurrent writes to blob db 2c45ada4c Blob DB garbage collection should keep keys with newer version 58410aee4 Fix the overflow bug in AwaitState c3d5c4d38 Refactor TransactionImpl 060ccd4f8 support multiple CFs with OPTIONS file 345387067 Fix statistics in RocksJava sample 1900771bd Dump Blob DB options to info log 3218edc57 Fix universal compaction bug 6a36b3a7b fix db get/write stats a84cee812 Add a missing "once" in .h 21696ba50 Replace dynamic_cast<> e85f2c64c Prevent empty memtables from using a lot of memory ac748c57e Fix FIFO Compaction with TTL tests aaf42fe77 Move blob_db/ttl_extractor.h into blob_db/blob_db.h aace46516 Fix license headers in Cassandra related files 50a969131 CacheActivityLogger, component to log cache activity into a file 6083bc79f Blob DB TTL extractor 710411aea fix asan/valgrind for TableCache cleanup 3a3fb00b7 TARGETS file not setting sse explicitly fca4d6da1 Build fewer tests in Travis platform_dependent tests 8f553d3c5 remove unnecessary internal_comparator param in newIterator 7f6d012d7 "ccache -C" in Travis d12691b86 move TableCache::EraseHandle outside of db mutex f33f11368 fix db_bench argument type e7697b8ce Fix LITE unit tests 3ce20e985 Fix use of RocksDBCommonHelper in cont_integration.sh c281b4482 Revert "CRC32 Power Optimization Changes" 9980de262 Fix FIFO compaction picker test 2289d3811 CRC32 Power Optimization Changes 30b58cf71 Remove the orphan assert on !need_log_sync fe1a5559f Fix flaky write_callback_test addbd279c 5.6.1 release blog post 30edff308 buckification: remove explicit `-msse*` compiler flags 2b259c9d4 Lower num of iterations in DeadlockCycle test 277f6f23d Release note for partitioned index/filters 5e731a138 Remove unused rocksdb arcanist lib 9b11d4345 Fix broken links 06f191744 add vcpkg as an windows option ea8ad4f67 Fix compaction div by zero logging 34112aeff Added db paths to c 1d8aa2961 Gcc 7 ParsedInternalKey replace memset with clear function. a4c42e800 Fix UBSAN issue of passing nullptr to memcmp 16e038820 LRUCacheShard cache line size alignment 216644c61 enable UBSAN macro in TARGETS e67b35c07 Add Iterator::Refresh() a34b2e388 Fix caching of compaction picker's next index 72502cf22 Revert "comment out unused parameters" 1d7048c59 comment out unused parameters 534c255c7 Cassandra compaction filter for purge expired columns and rows 63163a8c6 Remove make_new_version.sh 0302da47a Reduce blob db noisy logging 3e5ea29a8 Fix Flaky DeleteSchedulerTest::ImmediateDeleteOn25PercDBSize a22b9cc6f overlapping endpoint fixes in level compaction picker ffd2a2eef delete ExpandInputsToCleanCut failure log 3e6e863b1 Remove arcanist_util directory 36651d14e Moving static AdaptationContext to outside function 6e3ee015f Update java/rocksjni.pom ecff9d5e3 Include write_buffer_manager in ImmutableDBOptions::Dump ae28634e9 Remove some left-over BSD headers 33b1de82a Remove format compatibility hack 2f375154e checkout local branch in check_format_compatible.sh ddb22ac59 avoid collision with master branch in check format 0c03a7f17 set the remote for git checkout 7ac184c6d Revert cmake -DNDEBUG for non-MSVC 0655b5858 enable PinnableSlice for RowCache 00464a314 Fix column_family_test with LITE build b2dd192fe tools/write_stress.cc: Correct "1204" typos. cbaab3044 table/block.h: change memset f1a056e00 CodeMod: Prefer ADD_FAILURE() over EXPECT_TRUE(false), et cetera 4a2e4891f Add back the LevelDB license file a7321fc97 Remove the licensing description in CONTRIBUTING.md 3c327ac2d Change RocksDB License 132013366 Make TARGETS file portable ccf5f08f8 Set CACHE_LINE_SIZE for s390, PPC, ARM64 67510eeff db_crashtest.py: remove need for shell 4267eb00d Remove punit tests 5bfb67d90 Enable write rate limit for updaterandom benchmark 20a691d98 Update HISTORY to release 5.7 98d1a5510 db_bench to by default verify checksum 26ce69b19 Update blob db to use ROCKS_LOG_* macro 43e4eef77 remove unnecessary fadvise 21b17d768 Fix BlobDB::Get which only get out the value offset 70440f7a6 Add virtual func IsDeleteRangeSupported 7550255c5 Add JAVAC_ARGS as Makefile variable 7a0b5de77 Gcc 7 ignored quantifiers 000bf0af3 Improve the design and native object management of Stats in RocksJava 269d383d5 Bump version to 5.7 c32f27223 Fixes db_bench with blob db fcd99d27c db_bench_tool: fix buffer size 87128bd5c fix regression test 8f927e5f7 Fix undefined behavior in Hash 643b787c7 Added a note about LZ4 compression dependency 56656e12d Temporarily disable FIFOCompactionWithTTLTest b5fb85ec5 fix valgrind init complaint 657df29ea Add max_background_jobs to db_bench a43c053ad remove duplicated utilities/merge_operators/cassandra/test_utils.cc in src.mk 7c4a9e6c9 Initialize a variable in ldb to make code analysis tool happy 98669b535 init filters_in_partition_ 0013bf14e fix asan and valgrind leak report in test 521b4c28b rocksdb 5.5.1 release post 33042573d Fix GetCurrentTime() initialization for valgrind f6b9d9355 Fix clang error in PartitionedFilterBlockBuilder 45b9bb033 Cut filter partition based on metadata_block_size f4ae1bab0 update history for OnBackgroundError and DeleteRange fix 1cb8c6de6 Add -enable_pipelined_write to db_bench and add two defaults 7604b463b Update the AddDBStats in LITE 1e34d07e1 Simplify and document sync rules for logs_ etc d310e0f33 Regression test for empty dedicated range deletion file e9f91a517 Add a fetch_add variation to AddDBStats c1b375e96 skip generating empty sst 67b417d62 fix format compatible test afbef6518 Bug fix: Fast CRC Support printing is not honest 397ab1115 Improve Status message for block checksum mismatches 18c63af6e Make "make analyze" happy 01534db24 Fix the reported asan issues 1cd45cd1b FIFO Compaction with TTL 982cec22a Fix TARGETS file tests list b49b37109 allow numa >= 2.0.8 e517bfa2c CLANG Tidy dc3d2e4d2 update compatible test 89468c01d Fix Windows build broken by 5c97a7c0664d4071768113814e9ba71fe87e18cf 51778612c Encryption at rest support 7061912c2 Trivial typo in HISTORY.md 2a9cd8799 Fix jni WriteBatchThreadedTest 0025a3640 revert perf_context and io_stats to __thread 5c97a7c06 Unit Tests for sync, range sync and file close failures 4cee11f4e Intra-L0 blog post 857e9960b Improve the error message for I/O related errors. d757355cb Fix bug that flush doesn't respond to fsync result 8e6345d2d Update rename of ParanoidCheck 499ebb3ab Optimize for serial commits in 2PC 0ac4afb97 Sanitize partitioning options 521724ba8 fixed wrong type for "allow_compaction" parameter 71f5bcb73 Introduce OnBackgroundError callback 88cd2d96e Downgrade option sanitiy check level for prefix_extractor 6837a1762 Fix Data Race Between CreateColumnFamily() and GetAggregatedIntProperty() af1746751 WriteBufferManager will not trigger flush if much data is already being flushed 9467eb614 Fix flush assertion with tsan 048446fc7 Fix cassandra ASAN use-after-free a21db161c Implement ReopenWritibaleFile on Windows and other fixes c430d69ee fix coredump for release nullptr 0d278456c default implementation for InRange cbd825dee Create a MergeOperator for Cassandra Row Value 2c98b06bf Remove pin_slice option by making it the default c80c6115d add db_bench options for partitioning 6a3377f45 Synchronize statistic enumeration values between statistics.h and java API 53dda8797 Do not run RateLimiterTest.Rate test on Travis+Mac OSX. ae8571f5c Fix blob db compression bug 7a380deff Update blob_db_test 89ad9f3ad Allow ignoring unknown options when loading options from a file 6b5a5dc5d fixed typo 0f228be3b fixed typo in util/dynamic_bloom.h c217e0b9c Call RateLimiter for compaction reads 91e2aa3ce write exact sequence number for each put in write batch 6f4154d69 record index partition properties 5d5a28a98 Fix Clang release build broken by 5582123dee8426a5191dfd5e846cea8c676c793c 0175d58c3 Make direct I/O write use incremental buffer 7a270069b GNU C library for struct tm has 2 additional fields. d713471da Limit trash directory to be 25% of total DB 9bb91e932 Dedup release 27b450165 Update HistogramTypes in the Java API e97304c68 update history for 5.6 5582123de Sample number of reads per SST file db818d2d1 Fix RocksDB Lite build with CLANG a472c4ae4 update 5.5 change log bc09c8a0d Fix crash in PosixWritableFile::Close() when fstat() fails 6d0f22e42 Fix mock_env.cc uninitialized variable c2012d488 Java APIs for put, merge and delete in file ingestion 85dace2af Disable DBRangeDelTest::TailingIteratorRangeTombstoneUnsupported for ubsan d4f7731b6 fix travis error with init time in mockenv 550a1df72 Fix clang errors by asserting the precondition cc5f9339e Fix concurrency issue with filter_block_set_ 2e64f450d bump version to 5.6 afbc2d0d2 Force travis to build with clang on MacOS b172a3f1a Fix warnings while generating RocksJava documentation 52a7f38b1 WriteOptions.low_pri which can throttle low pri writes if needed 26a8a8071 Switch from CentOS 5 to CentOS 6 for crossbuilding RocksJava dba9f3722 Fix db_write_test clang/windows build failure c7662a44a fixed typo 7e8d95cc0 Fix the Java build which was broken by a4d9c02 7e5fac2c3 remove test dir before exit when current regression is running 7f6c02dda using ThreadLocalPtr to hide ROCKSDB_SUPPORT_THREAD_LOCAL from public… 138b87eae Fix interaction between CompactionFilter::Decision::kRemoveAndSkipUnt… 95b0e89b5 Improve write buffer manager (and allow the size to be tracked in block cache) a4d9c0251 Pass CF ID to MemTableRepFactory f68d88be5 Fix DBWriteTest::ReturnSequenceNumberMultiThreaded data race 215076ef0 Fix TSAN: avoid arena mode with range deletions 3a8a848a5 account for L0 size in estimated compaction bytes 0fae3f5dd codemod: format TARGETS with buildifier [5/5] (D5092623) 872199606 add checkpoint support for single db in regression test 5a9b4d743 Retire memenv https://github.com/facebook/rocksdb/pull/2082 d6019651b sync internal/external TARGETS bbaba51bb Add missing index type to C-API 292edfd51 travis: test with xcode8.3 (OS X 10.12) 0dc3040d5 db: avoid `#include`ing malloc and jemalloc simultaneously 9b3ed8350 fix regression test 9c9909bf7 Support ingest file when range deletions exist ad19eb868 Fixing blob db sequence number handling 51ac91f58 Histogram of number of merge operands 345878a7f update blob_db_test cbc821c25 change regression rebuild to one level 103d0692e Avoid unsupported attributes when not building with UBSAN 5fd04566c travis: reduce the number of travis builders 2d05002b9 RocksDB 5.4.5 release blog post 7eca90f18 Update RocksDB blog authors d03c34497 update comment of GetNextFile f7bb1a006 support merge and delete in file ingestion c2c62ad4e Reorder variables of ReadOptions 7bb1f5d48 Increase of compaction threads should be logged at info level instead of a warning 6c456ecae Clean zstd files 347e16f83 codemod: replace `headers = AutoHeaders.*` with `auto_headers` 0be636bf7 Fix db_bench build break with blob db 135ee6a3f fix tsan crash data race a99fb9928 fix column_family_test asan f41bffb3d travis: clang-3.6 -> clang-4.0 e7612798b update buckifer/TARGETS bb01c1880 Introduce max_background_jobs mutable option 5a2530462 Fix the CMakeLists for RocksJava 41cbb7274 options.delayed_write_rate use the rate of rate_limiter by default. 506803466 range sync should be enabled 02594b5f1 Fix build errors in blob_dump_tool with GCC 4.8 52d9e5f7b Fix column family seconds_up accounting 7d8207f1f Fix errors in clang-analyzer builds 85b8569ae Fix release build on Linux 69ec8356b Allow SstFileWriter to use the rate limiter 6cc9aef16 New API for background work in single thread pool 9d0a07ed5 Fix rocksdb.estimate-num-keys DB property underflow 578fb0b1d Simple blob file dumper ac39d6bec Core-local statistics 3e86c0f07 disable direct reads for log and manifest and add direct io to tests 15ba4d6c4 Address MS Visual Studio 2017 issue with autovector 88c818e43 Replace deprecated RocksDB#addFile with RocksDB#ingestExternalFile 228f49d20 Fix data races caught by tsan 4c9d2b104 remove #include port/port.h in public header file 07bdcb91f New WriteImpl to pipeline WAL/memtable write d746aead1 Suppress clang-analyzer false positive 217b866f4 column_family_test: EnvCounter::num_new_writable_file_ to be atomic 9f839a7f6 keep util/build_version.cc when make clean 7eecd40a4 add emacs tags file - etags 9bbba4fec Remoe unused BlockBasedTable::compaction_optimized_ f5ba131bf Fixed some spelling mistakes 146b7718f Fix mingw compilation with -DNDEBUG a36220ccf fix unity test 0ebdd7057 fixed typo 8032f4cb3 Remove -pie in TSAN 362ba9b02 Release RocksDB 5.5.0 ba685a472 Support ingest_behind for IngestExternalFile 01ab7b528 Add ROCKSDB_LIBRARY_API macro to a few C APIs, to fix windows build cb9392a09 add Transactions and Checkpoint to C API 445f1235b s/std::snprintf/snprintf cd593c283 Fix travis java_test f720796e2 fixed typo a48a62d5b define NDEBUG in CMake non-debug builds 1ca723dbd C API: support pinnable get 2ef15b85d Core-local stats blog post 4f9e69ccf fix log err 11c5d4741 cross-platform compatibility improvements d00433302 Put lib files into suitable path in RPM package 86d549253 Fix build error with blob DB. 254c46800 Fix the RocksJava Release on Windows 7a47b431f Fix .gitignore pattern fa5a15ceb Make sure that zstd is statically linked correctly in the Java static build 3fa9a39c6 Add GetAllKeyVersions API 1a60982a5 Simplified instructions for CentOS a5cc7ecec Facility for cross-building RocksJava using Docker ccd3dddf5 Blog post for partitioned index/filters b145c34d7 Update blog authors bbe9ee7dd core-local array type conversions c2be43430 Build and link with ZStd when creating the static RocksJava build c61e72c42 Add missing files of blob_db to CMake file 3907c94ff Fix ColumnFamilyTest:BulkAddDrop cda5fde2d CoreLocalArray class 93949667c update TARGETS 4e83b8001 title: Bulkoading -> title: Bulkloading d85ff4953 Blob storage pr 492fc49a8 fix readampbitmap tests be421b0b1 portable sched_getcpu calls 0f559abdb Add NO_UPDATE_BUILD_VERSION option to makefile 3a04a254a Flink state 35df23fe8 Fix suite exclisions e7cea86f7 Fixes the CentOS 5 cross-building of RocksJava 259a00eac unbiase readamp bitmap a62096696 port: updated PhysicalCoreID() df035b682 Print compaction_options_universal.stop_style in LOG file 4897eb250 dont skip IO for filter blocks 3f73d54bb Add C API to set max_file_opening_threads option 0b69e5079 Define CACHE_LINE_SIZE only when it's not defined 2cd00773c Add bulk create/drop column family API 40af2381e Object lifetime in cache fdaefa030 travis: add Windows cross-compilation a30a69603 do not read next datablock if upperbound is reached 2d42cf5ea Roundup read bytes in ReadaheadRandomAccessFile 264d3f540 Allow IntraL0 compaction in FIFO Compaction 8c3a180e8 Set lower-bound on dynamic level sizes 7c1c8ce5a Avoid calling fallocate with UINT64_MAX a45e98a5b max_open_files dynamic set, follow up 6b99dbe04 fix memory alignment with logical sector size e7ae4a3a0 Max open files mutable 60847a3b0 port: updated PhysicalCoreID() b551104e0 support PopSavePoint for WriteBatch 498693cf3 Remove orphaned Java classes 5e2ebf2bd travis: add CMake compilation af6fe69e4 Fix an issue of manual / auto compaction data race 6798d1f3b Revert "Delete filter before closing the table" 89833577a Delete filter before closing the table 47a09b0a8 Avoid pinning when row cache is accessed aeaba07b2 Remove an assert that causes TSAN failure. 0b90aa951 Set VALGRIND_VER a2b05210e Make PosixLogger::flush_pending_ atomic da4b2070b Fix WriteBatchWithIndex address use after scope error d616ebea2 Add GPLv2 as an alternative license. 4439b4596 Add documentation license 0ca3ead0c add GetRootDB() in DeleteFilesInRange cdad04b05 Remove double buffering on RandomRead on Windows. e15382c09 Disable two flaky tests 2150cc1f3 fix WritableFile buffer size in direct IO efc361ef7 Add user stats Reset API 6616e4d62 add prefetch to PosixRandomAccessFile in buffered io f6a27d0bc Extract statistics tests into separate file 7eddecce1 support bulk loading with universal compaction 3b4d1b7a4 add to avoid warning with glibc 2.25 e5e545a02 Reunite checkpoint and backup core logic 72c21fb3f call GetRootDB() before cast to DBImpl* in CancelAllBackgroundWork 4c9447d88 Add erase option to release cache 04d58970c AIX and Solaris Sparc Support afff9951e Respect deprecated flag in table options 066cfbacc Adding -noprofile to CMakeLists for Windows cb885bccf set compaction_iterator earliest_snapshot to max if no snapshot 7534ba7bd StackableDB should pass ResetStats() c1fbf91b2 Fixing Solaris Sparc crash due to cached TLS 963eeba48 Revert how check_format_compatible.sh checkout release branches. 97005dbd5 tools/check_format_compatible.sh to cover option file loading too 8f6196788 Add cpu usage to regression benchmarks (4th attempt) df74b775e Limit backups opened 1dd776051 Change L0 compaction score using level size 927bbab25 Revert "Add cpu usage to regression benchmarks (3rd attempt)" 8e84a388e Re-add index_per_partition but as deprecated 1553659d6 Add more recent versions to tools/check_format_compatible.sh 966ebb02f Hide event listeners from lite build 476e80be8 Add cpu usage to regression benchmarks (3rd attempt) c49d70465 Add DB:ResetStats() 0fcdccc33 Blob storage helper methods a6439d797 CMake: compile with -O2 e67f0adf3 enable O2 optimization for lz4 bc3973259 CMake: add support for SSE4.2 7d5f5aa97 Separate compile and link for shared library 071652734 remove warning 6e8d6f429 readahead backwards from sst end ca96654d8 Change Build Env to gcc-5 e9e6e5324 Simplify write thread logic 6799c7e00 Pass in remote as a param to branch creation script 44fa8ece9 change use_direct_writes to use_direct_io_for_flush_and_compaction 13b50358f add space for buggy kernel warning b6b9359ec Fix BYTES_WRITTEN accounting 13369fbd4 Update ShipIt to honor TARGETS updates f2449ce92 Remove .deprecated_arcconfig 415be221c RocksDB Release 5.4 : Update HISTORY.md and build version. 3eab41d7c java dependencies test -s -> use test -d a22ed4eab internal_repo_rocksdb to build Java and RocksDB LITE 9f2cc59ec sync TARGETS file 10d754696 set readahead buffer size from roundup(user_size) + 4k to roundup(use… ba7da434a fix db_stress crash caused by buggy kernel warning 6257837d8 Add ROCKSDB_JAVA_NO_COMPRESSION flag 6a6723ee1 Move MergeOperatorPinning tests to be with other merge operator tests 6a8d5c015 Revert "Report cpu usage using time command" 8f47a9751 File level histogram should be printed per CF, not per DB 9300ef545 Fix shared lock upgrades 1f8b119ed Limit maximum memory used in the WriteBatch representation 97ec8a134 Report cpu usage using time command 20778f2f9 Adding comments to the write path 7124268a0 Reduce the number of params needed to construct DBIter 04abb2b2d FreeBSD only requires WITH_JEMALLOC, not the rest 61730186d dummy diff 360e9960f Summary: Remove .arcconfig 69a5e6461 Deprecate .arcconfig 9690653db Add a verify phase to benchmarks dd8f9e38e Fix compilation for GCC-5 c2954f9b6 Add Travis job to build examples 72fc1e9d0 avoid non-existent O_DIRECT on OpenBSD ff9728701 Refactor compaction picker code 9e7293902 only FALLOC_FL_PUNCH_HOLE when ftruncate is buggy 343b59d6e Move various string utility functions into string_util 1d068f606 Fix CompactRange incorrect buffer release df6f5a377 Move memtable related files into memtable directory 107c5f6a6 CMake: more MinGW fixes d2dce5611 Move some files under util/ to separate dirs c50e3750d Use a human readable size for level report ce64b8b71 Divide db/db_impl.cc 02799ad77 Revert "delete fallocate with punch_hole" e2a7b202c Release note for partition filters af256eb2b build db every monday e5a1372b2 Rework test running script. d659faad5 Level-based L0->L0 compaction a12306fab Add a notice on gflags installation in INSTALL.md 43010a929 Revert "[rocksdb][PR] CMake: more MinGW fixes" a30b75cdc Add buckifier script to github repo 3450ac8c1 CMake: more MinGW fixes 90cfd4645 update IterKey that can get user key and internal key explicitly e2c6c0636 add TimedEnv 9e4453180 Refactor WriteImpl (pipeline write part 1) 6ef8c620d Move auto_roll_logger and filename out of db/ a1c469d71 Add release notes for PinnableSlice 0537f515c fix run_remote with strong quoting 72e600094 fixed misses on Centos library installation instructions 88cc81df5 auto_roll_logger_test to move away from real sleep d25e28d58 replace sometimes-undefined uint type with unsigned int a1d7e487b Add L0 write-amp to compaction level stats b6d609063 CMake: support AVX2 in MinGW bd7d13835 test remote instead run remote in regression test c81a805fe test db existence on the remote host 5fc1e6765 add -rf when remove db in regression test 4ab4049f2 gflags has moved to GitHub 4e0065015 make all DB::Get overloads virtual 6401a8b76 Fix build with MinGW 80fe5b385 disable test: DeleteSchedulerTest.DynamicRateLimiting1 a9c86f51b backup garbage collect shared_checksum tmp files da175f7ec exit with code 2 when there is already a db_bench running in regression test 0ee7f0403 Added missing options to RocksJava c6d04f2ec Option to fail a request as incomplete when skipping too many internal keys 58179ec4a Cleanup of ThreadStatusUtil structures should use the DB's reference f3607640a add ldb build to regression test 8a8c96746 Enable Fast CRC32 for Win64 f9813b853 Added SstFileWriter construtor without explicit comparator to JNI api 8d3cb4f20 Added naming of backup engine threads 67d762379 Expose the stalling information through DB::GetProperty() 0fd574926 delete fallocate with punch_hole 41fe9ad75 Hide usage of compaction_options_fifo from lite build e7731d119 Configure index partition size 69c8d524a Fix jni library name for PowerPC Architecture 34a70859b Fix segmentation fault caused by #1961 8dee8cad9 Enable fifo compaction benchmark to db_bench a5c8b5434 fix a header include 91b5feb37 Fix Windows Build broken by a recent commit 41ccae6d2 Add C API functions (and tests) for WriteBatchWithIndex 88bb6f6bf non_shm CI should run tests on /tmp 8888de2b1 Update .gitignore file in examples 203136e79 Fix Compilation errors when using IBM Java f4fce4751 Fix clang compile error - [-Werror,-Wunused-lambda-capture] a084b26a5 Blog post for releasing 5.2.1 15950fe3a Remove ASSERT_EQ(boolean, ...) 3e56c7e0c make total_log_size_ atomic 909028e21 HISTORY.md for log_size_for_flush in CreateCheckpoint() be723a8d8 Optionally construct Post Processing Info map in MemTableInserter e474df947 db_bench: not need to check mmap for PlainTable 8b0097b49 Readers for partition filter 9ef3627fd Allow checkpointing without flushing 17866ecc3 Allow Users to change customized ldb tools' header in help printing a2a883318 remove deleted option from benchmark.sh 78cb19559 add checkpoint to ldb 4b04addfc updated solution if "make format" command fails 8f5bf0446 Flush triggered by DB write buffer size picks the oldest unflushed CF 6908e24b5 dynamic setting of stats_dump_period_sec through SetDBOption() 93c68b642 change regression bash file with debug mode 21d8c3179 remove LIKELY from public headers 36ad75778 INSTALL: document USE_SSE 9272e12f1 avoid ftruncate twice in buffered io d52f334cb Break stalls when no bg work is happening e66221add fix db_bench rate limiter callsites dbae438a0 Replace "DEPRECATED" comment to "not supported" 995618a82 Support SstFileManager::SetDeleteRateBytesPerSecond() e19163688 Add macros to include file name and line number during Logging d525718a9 cleanup direct io flag in WritableFileWriter 5fa927aa9 Add Xpress and ZSTD CompressionType values to C header 11526252c Pinnableslice (2nd attempt) e5bd8def1 update history.md for fixing the bug that skips keys 1ffbdfd9a Add a new SstFileWriter constructor without explicit comparator ebd5639b6 Add ability to search for key prefix in sst_dump tool e6725e8c8 Fix some bugs in MockEnv 900c62be6 fix compile for VS2015 fe1835617 release 5.3 5dae01947 Revert "Report cpu usage using time command" f2817fb7f avoid ASSERT_EQ(false, ...); 5b11124e3 add max to histogram stats d43adf21b Report cpu usage using time command 18fc1bc0e minor changes for rate limiter test flakiness 12ba00ea6 Reset DBIter::saved_key_ with proper user key anywhere before pass to DBIter::FindNextUserEntry c9df05d1e Fix random access alignment f64991537 Add Bulkoading IngestExternalFile blog post 54b434110 Builders for partition filter 97edc72d3 Add a memtable-only iterator 72202962f fix db_sst_test flakiness 5f65dc877 Expose DB::DeleteRange and WriteBath::DeleteRange in Java 58b12dfe3 Set logs as getting flushed before releasing lock, race condition fix f8a4ea020 Move db_test and external_sst_file_test out of Travis's MAC OS run 534581a35 Fix a bug in tests in options operator= a2f7a514d Refactoring 2a5daa06f Add stderr log level for ldb backup commands 4561275c2 fix rate limiter test flakiness 7c80a6d7d Statistic for how often rate limiter is drained 0ad5af42d Clarify VerifyBackup behavior 6fb901344 sanitize readahead when direct read enabled f89b3893c Remove skip_table_builder_flush and default it to true cc253982d Use more default options in db_bench 8432bcf55 Make compaction_pri settable through option string d5b607a43 Make db_wal_test slightly faster ba4c77bd6 Divide external_sst_file_test e877afa08 Remove bulk loading and auto_roll_logger in rocksdb_lite 90d835507 Fix the wrong address for PREFETCH in DynamicBloom::Prefetch 08864df21 Move advanced column family options to advanced_options.h 2ca2059f6 Get unique_ptr to use delete[] for char[] in DumpMallocStats 253799c06 Add missing include for `abort()` c6d464a9d Fixed various memory leaks and Java 8 JNI Compatibility be3e5568b Fix unaligned reads in read cache 8ad0fcdf9 Separate small subset tests in DBTest 6c951c43c Run fewer tests in OSX f7997f134 add direct I/O to version notes 5.2.0 3b8ba703c Fix flaky DBTestUniversalCompaction.UniversalCompactionTrivialMoveTest2 96c7e1504 Fix Java build e0b87afc7 Black list some slow valgrind tests e67232cff Handle failed Finish() in SST file writer 8efb5ffa2 [rocksdb][PR] Remove option min_partial_merge_operands and verify_checksums_in_comp… 1ba2804b7 Remove XFunc tests e7d902e69 add direct_io and compaction_readahead_size in db_stress 1ef5f50e8 detect logical sector size ed50308d2 check backup directory exists before listing children 4d7c06ced Make WriteBatchWithIndex moveble 5040414e6 Gracefully handle previous backup interrupted f206af56f add use_direct_io() to ReadaheadRandomAccessFile 082493442 truncate patch 286a36db7 posix writablefile truncate f0879e4c3 Page size isn't always 4k on linux 18eeb7b90 Fix interference between max_total_wal_size and db_write_buffer_size checks 1560b2f5f Temporarly return deprecated functions to fix MongoRocks build 2a0f3d0de level compaction expansion ebc8a7998 alignment is on in ReadaheadRandomAccessFile::Read() 381fd3224 Remove timeout_hint_us from WriteOptions fce7a6e19 Fail IngestExternalFile when bg_error_ exists a618a16f4 New subcode for IOError to detect the ESTALE errno 7ab005183 Remove deprecated DB::AddFile and DB::CompactRange 401667366 Adding Dlang to the list 756c5924e Allow adding external v1 sst file with no global seqno support aa0298fa9 Remove trailing whitespace from examples Makefile db2b4eb50 avoid direct io in rocksdb_lite 43e9f01c2 Fix repair_test on ROCKSDB_LITE 7106a994f Use monotonic time points in write_controller.cc and rate_limiter.cc c2247dc1c Make DBImpl::has_unpersisted_data_ atomic eb912a927 Remove disableDataSync option 0227c16d6 Update static library versions and add checksums b3aae4d07 Add repair_test to make check 421ce7c2b Add support for JNI Library on Linux on PowerPC. 9afa20cf2 Increase build version and HISTORY.md for releasing 5.2 a5adda064 Fix repair issues b48e4778b Consolidate file cutting logic in compaction loop ac2a77a74 Announce the experimetnal two-level index feature in HISTORY.md c4a37dcb4 Print the missed last layer in cfstats a12818afc Blog post for 5.1.2 release 3b4ac8076 Clarify ldb column family argument d70ce7ee0 Move db_bench flags out of unnamed namespace 186c7eedb Remove incorrect statistics warning 53bb01516 [rocksdb][PR] compaction_style and compaction_pri should output their value as a st… 69d5262c8 Two-level Indexes 0a4cdde50 Windows thread 1aaa898cf Adding GetApproximateMemTableStats method 9fc23c55f Use gcc-4.9-glibc-2.20-fb python in precommit_checker b797e4215 Dump compression dictionary meta-block 036d668b1 Fix wrong result in data race case related to Get() 574b543f8 Rename merger.h -> merging_iterator.h add8b50cc Move ThreadLocal implementation into .cc 71d2496af Fix arc setting for Facebook internal tools f289d9f4a Fix OSX build break after the fallocate change 4a3e7d320 Change the default of delayed slowdown value to 16MB/s 0513e21f9 RangeSync() should work with ROCKSDB_FALLOCATE_PRESENT not set 8b369ae5b Cleaner default options using C++11 in-class init ec79a7b53 Dedup code in option.cc and db_options.cc b96372dea improving the C wrapper 04c4ec41d Change corruption_test to use 4 bits. 2d75cd40d NewLRUCache() to pick number of shard bits based on capacity if not given f25f1ec60 Add test DBTest2.GetRaceFlush which can expose a data race bug 37d4a79e9 Deserialize custom Statistics object in db_bench 3b35134e4 Avoid cache lookups for range deletion meta-block 94a0c32e7 Fix LRU Ref() for handles with external references only 17c118060 Generalize Env registration framework 07dddd5f7 EnvPosixTestWithParam should wait for all threads to finish 5dad9d6d2 Avoid logs_ operation out of DB mutex a7b13919b Fix CompactFiles() bug when used with CompactionFilter using SuperVersion 616a1464e Fix DeleteRange including sentinels in output files c918c4b76 Update USERS.md add user Pika 03ca2ac8a Remove function from DBImpl that are not used anywhere b0029bc7f Test merge op covered by range deletion in memtable d438e1ec1 Test range deletion block outlives table reader fba726e55 Version librocksdb.so 9da4d542f Range deletions unsupported in tailing iterator f2b4939da fixed typo 973f1b78f memtable: delete merge value for range deleteion aebfd1703 fix non-portable behavior in encoder 753ff84a3 Fix get approx size d7ea44f2f Fixup a couple of builds errors on Linux. 537da370d c: allow set savepoint to writebatch af6ec4d78 fix batchresult handle leak e29bb934f Zlib 1.2.8 is no longer available, switched to 1.2.10 5ac97314e Fix std::out_of_range when DBOptions::keep_log_file_num is zero 4e35ffdfa cmake: check -momit-leaf-frame-pointer before using it 3c0852d1d Make ingest external file backward compatible 0e8dfd606 Fix OptimizeForPointLookup() e840213d6 Change DB::GetApproximateSizes for more flexibility needed for MyRocks 9239103cd Flush job should release reference current version if sync log failed da54d36a9 Disable IngestExternalFile in ReadOnly mode 5cf176ca1 Fix for 2PC causing WAL to grow too large 4a73bb0b4 Split travis jobs c70d3c7ad Enable DBTest.GroupCommit as it runs in a reasonlable time now. 602c13a96 Remove fadvise with direct IO read f9d18e22d Fix DeleteRange file boundary correctness issue with max_compaction_bytes 3ce091fd7 Add KEEP_DB env var option 77b480662 Fix 2PC with concurrent memtable insert e8a096000 util/thread_local.h: silence a clang-build warning 324a0f988 Follow up for DirectIO refactor bc5d7b702 travis: For linux, do all tests under gcc 3e6899d11 change UseDirectIO() to use_direct_io() d4e07a845 fix warning of unused direct io helper functions dc2584eea direct reads refactor d18dd2c41 Abort compactions more reliably when closing DB 62384ebe9 Guarding extra fallocate call with TRAVIS because its not working pro… 9f246298e Performance: Iterate vector by reference fe395fb63 Allow incrementing refcount on cache handles 2172b660e Fix build on FreeBSD 3c233ca4e Fix Windows environment issues 763173456 Fix the error in ColumnFamiliesTest 7a02ad070 Update travis to ubuntu trusty 60c509ff1 Fix valgrind failure in test CurrentFileModifiedWhileCheckpointing2PC d0ba8ec8f Revert "PinnableSlice" 54d94e9c2 PinnableSlice e04480fae Fix MS warnings. Use ROCKSDB_Prsz for size_t. c081f7215 5.0.1 release blog post ac73d7558 Add GetSupportedCompressions() convenience function b104b8781 Maintain position in range deletions map 640d72480 Update db_bench and sst_dump to test with block cache mid-point inser? 653ac1f9c C API: support total_order_mode 85ac1a320 Fix rocksdb::Status::getState 76711b6e7 Make ExternalSSTFileTest::CompactionDeadlock more deterministic c963460db Fix tests under GCC_481 33c86d677 Fix backupable db test e425ec116 utilities/backupable: backup should limit the copy size of wal. 0712d541d Delegate Cleanables d58ef52ba Allow SstFileWriter to Fadvise the file away from page cache 17a4b75cc Always fsync the file after file copying 2fb70dc79 examples: Add options_file_example to target all a738af8f8 db/pinned_iterators_manager.h: bugfix 906523d98 Add description to the 2PC checkpooint bug in HISTORY.md 438f22bc5 Fix bug of Checkpoint loses recent transactions with 2PC 335981d47 Fix the directory path for RocksDB repo 548b62805 Enable conditionally using adaptive mutexes 4e07b08ef include/rocksdb/utilities/env_librados: fix typo ab48c165a Print cache options to info log 972f96b3f direct io write support 989e644ed Remove sst_file_manager option from LITE 1beef6569 Fix c_test 3d692822f persistent_cache: fix two timer 046099c9b The array is malloced by backtrace_symbols(), and must be freed 6ff2c8d7f Remove gflags as travis build dependency 3cd9ed1c3 Show sandcastle URL in phabricator 50e305de9 Collapse range deletions 5d1457dbb Dump persistent cache options 7bd725e96 db_bench: introduce --benchmark_read_rate_limit 296691847 Update Netflix section of USERS.md 342370f1d Simplify MemTable::Update 1a136c1f1 Expose file size fbff4628a Reduce compaction iterator status checks bd6cf7b51 WritableFileWriter: default buffer size equal min(64k,options.writabl? fc0c6fd98 "make format" format diff since last commit from master 816c1e30c gcc-7 requires include for std::function c27073586 Iterator should be in corrupted status if merge operator return false a8bf4d63f Make format shows wrong curl command for retrieving clang-format-diff.py 8f5d24ae6 C API: support get usage and pinned_usage for cache 0ab6fc167 Gcc-7 buffer size insufficient b7239bf7e Gcc 7 fallthrough 477b6ea57 std::remove_if requires 83f9a6fd2 Fail BackupEngine::Open upon meta-file read error a79eae4b0 Add pcache documentation images d71e728c7 Print user collected properties in sst_dump 7004a6f7b Add missing copyright header 3cdfaeca3 Fixes for MSVC compilation e097222e6 util/logging.cc: buffer of insufficient size (gcc-7 -Werror=format-length) cfc34d7c4 Missing break in case in DBTestBase::CurrentOptions bfbcec233 Gcc 7 error expansion to defined 6653e32ac build: make it easier to pass PORTABLE 67adc937b intentional fallthough (prevents gcc-7/clang-4 error) 1a146f89c break Flush wait for dropped CF c3e5ee715 util/histogram.cc: HistogramStat::toString buffer insufficient 5334d8b44 table/block_based_table_builder.cc: intentional fallthrough - comment to match gcc pattern 36d42e65d Disable test to unblock travis build b57dd9262 C API: support writebatch delete range 2ba59b5a1 Disallow ingesting files into dropped CFs 1f6f7e3e8 cast to signed char in ldb_cmd_test for ppc64le 243975d5d More accurate error status for BackupEngine::Open f0c509e2c Return finer-granularity status from Env::GetChildren* dc64f46b1 Add db_bench option for stderr logging 2cabdb8f4 Increase buffer size 4a17b47bb Remove unnecessary header include 8c2b921fd Fixed a crash in debug build in flush_job.cc 20ce081fa Fix issue where IngestExternalFile insert blocks in block cache with g_seqno=0 5241e0dbf fix db_bench argument type c04f6a0b4 Specify shell in makefile 45c7ce137 CompactRangeOptions C API 2c2ba6824 db_stress support for range deletions b821984d3 DeleteRange read path end-to-end tests 2f4fc539c Compaction::IsTrivialMove relaxing 1dce75b2e Update USERS.md 304b3c706 Update USERS.md fa50fffaf Option to expand range tombstones in db_bench c26a4d8e8 Fix compile error in trasaction_lock_mgr.cc ed8fbdb56 Add EventListener::OnExternalFileIngested() event 2005c88a7 Implement non-exclusive locks 0b0f23572 Mention IngestExternalFile changes in HISTORY.md 23db48e8d Update HISTORY.md for 5.0 branch beb36d9c1 Fixed CompactionFilter::Decision::kRemoveAndSkipUntil 67f37cf19 Allow user to specify a CF for SST files generated by SstFileWriter 9053fe2a5 Made delete_obsolete_files_period_micros option dynamic edde954e7 fix clang build 56281f3a9 Add memtable_insert_with_hint_prefix_size option to db_bench 4a21b1402 Cache heap::downheap() root comparison (optimize heap cmp call) e39d08087 Fix travis (compile for clang < 3.9) 3f407b065 Kill flashcache code in RocksDB b77007df8 Bug: paralle_group status updated in WriteThread::CompleteParallelWorker 247d0979a Support for range skips in compaction filter 96fcefbf1 c api: expose option for dynamic level size target 00197cff3 Add C API to set base_backgroud_compactions 5b219eccb deleterange end-to-end test improvements for lite/robustness aad119176 pass rocksdb oncall to mysql_mtr_filter otherwise tasks get created w… e33352899 DeleteRange write path end-to-end tests 7784980fc Fix mis-reporting of compaction read bytes to the base level 3c6b49ed6 Fix implicit conversion between int64_t to int b3b875657 Remove unused assignment in db/db_iter.cc 4f6e89b1d Fix range deletion covering key in same SST file a2bf265a3 Avoid intentional overflow in GetL0ThresholdSpeedupCompaction 52fd1ff2c disable UBSAN for functions with intentional -ve shift / overflow 1886c435b Fix CompactionJob::Install division by zero 63c30de80 fix options_test ubsan 13e66a8f5 Fix compaction_job.cc division by zero 01eabf737 Fix double-counted deletion stat 7ffb10fc1 DeleteRange compaction statistics 236d4c67e Less linear search in DBIter::Seek() when keys are overwritten a lot cd7c4143d Improve Write Stalling System dfb6fe675 Unified InlineSkipList::Insert algorithm with hinting 3068870cc Making persistent cache more resilient to filesystem failures 734e4acaf Eliminate redundant cache lookup with range deletion 182b940e7 Add WriteOptions.no_slowdown 4118e1333 Persistent Cache: Expose stats to user via public API f2a8f92a1 rocks_lua_compaction_filter: add unused attribute to a variable 4444256ab Remove use of deprecated LZ4 function 548d7fb26 Fix fd leak when using direct IOs fd43ee09d Range deletion microoptimizations 23a18ca5a Reword support a little bit to more clear and concise 481856ac4 Update support to separate code issues with general questions a0deec960 Fix deadlock when calling getMergedHistogram fe349db57 Remove Arena in RangeDelAggregator e63350e72 Use more efficient hash map for deadlock detection a13bde39e Skip ldb test in Travis 73843aa63 Direct I/O Reads Handle the last sector correctly. 9d60151b0 Implement PositionedAppend for PosixWritableFile 3f6221521 Lazily initialize RangeDelAggregator's map and pinning manager 41e77b839 cmake: s/STEQUAL/STREQUAL/ c1038d283 Release RocksDB 5.0 635a7bd1a refactor TableCache Get/NewIterator for single exit points f39452e81 Fix heap use after free ASAN/Valgrind a4eb7387b Allow plain table to store index on file with bloom filter disabled 36e4762ce Remove Ticker::SEQUENCE_NUMBER 86eb2b9ad Fix src.mk 0765babe1 Remove LATEST_BACKUP file 647eafdc2 Introduce Lua Extension: RocksLuaCompactionFilter 760ef68a6 fix deleterange asan issue 327085b7b fix valgrind 715591bba Ask travis to use JDK 7 972e3ff29 Enable allow_concurrent_memtable_write and enable_write_thread_adaptive_yield by default 420bdb42e option_change_migration_test: force full compaction when needed 1543d5d92 Report memory usage by memtable insert hints map. 018bb2ebf DeleteRange support for db_bench dc51bd716 CMakeLists.txt: FreeBSD has jemalloc as default malloc 48e8baebc Decouple data iterator and range deletion iterator in TableCache 4b0aa3c4c Fix failed compaction_filter_example and add it into make all 53b693f5f ldb support for range delete 661e4c926 DeleteRange unsupported in non-block-based tables 489d14280 DeleteRange interface eba99c28e Fix min_write_buffer_number_to_merge = 0 bug 2ef92fea5 Remove all instances of relative_url until GitHub pages problem is fixed. 91300d01f Dynamic max_total_wal_size option ec2f64794 Consider subcompaction boundaries when updating file boundaries for range deletion 800e51553 Fix CSS issues again b952c898b Parallize persistent_cache_test and transaction_test 3b192f618 Handle full final subcompaction output file with range deletions 6c5795200 Make range deletion inclusive-exclusive 425210cc4 CSS issues are arising on the Github Pages side. Temp fix. 1ea79a78c Optimize sequential insert into memtable - Part 1: Interface df5eeb85c Optimize sequential insert into memtable - Part 2: Implementation 5ed650857 Fix SstFileWriter destructor adb665e0b Allowed delayed_write_rate option to be dynamically set. 307a4e80c sst_dump support for range deletion 361010d44 Exporting compaction stats in the form of a map 672300f47 Use relative Urls for stylesheets b39b2ee12 do not call get() in recovery mode 1ca5f6d13 Fix 2PC Recovery SeqId Miscount e095d0cbc Rocksdb contruns to new Sandcastle API 14c0380e7 Convenience option to parse an internal key on command line c90fef88b fix open failure with empty wal 4e20c5da2 Store internal keys in TombstoneMap a9fb346e4 Fix RocksDB Lite build failure in c_test.cc d133b08f6 Use correct sequence number when creating memtable 144cdb8f1 16384 as e.g .value for compression_max_dict_bytes 9bd191d2f Fix deadlock between (WriterThread/Compaction/IngestExternalFile) a9fae0a9d CSS problems again :( 193221e0a Fix Forward Iterator Seek()/SeekToFirst() e48f3f8b9 remove tabs and duplicate #include in c api 85bd8f518 Minor fix to GFLAGS usage in persistent cache a7875272d c: support seek_for_prev 0f17f9279 Make the header links a bit more flexible cf19f559d single quotes in feed 2dc019e09 Fix header links f1aedda06 More Jekyll 3.3 fixes c54cdc378 More Jekyll 3.3 updates 2bcaf8246 Update product and feature template for Jekyll 3.3 24bceb096 Java API - Implement GetFromBatch and GetFromBatchAndDB in WBWI 815f54afa Insert range deletion meta-block into block cache 9e7cf3469 DeleteRange user iterator support 5c5d01ae7 Fix wrong comment (Maximum supported block size) f998c9790 DeleteRange Get support 879f36636 Add C api for RateLimiter 557034f36 Remove all instances of baseurl 437942e48 Add avoid_flush_during_shutdown DB option 2b16d664c Change max_bytes_for_level_multiplier to double 16fb04434 expose IngestExternalFile to c abi ce22ea99a Fix casts for MSVC 196af035c Introduce FAIL_ON_WARNINGS CMake variable (default ON) 40a2e406f DeleteRange flush support d5555d95a Fix MSVC compile error in 32 bit compilation da61f348d Print compression and Fast CRC support info as Header level f9eb56791 db_bench: --dump_malloc_stats takes no effect 6a4faee5c fix freebsd build include path err and so & jar file name c90c48d3c Show More DB Stats in info logs 1b295ac8a DBTest.GetThreadStatus: Wait for test results for longer 25f5742f0 Update documentation to point at gcc 4.8 b50a81a2b Add a test for tailing_iterator 04751d534 L0 compression should follow options.compression_per_level if not empty 2946cadc4 Improve RangeDelAggregator documentation 0a9fd05c2 Update Vagrant file (test internal phabricator workflow) fcd1e0bf6 Make rocksdb work with internal repo 0aab5e55f FreeBSD: malloc_usable_size is in (#1428) 9c0bb7f17 cmake: drop "-march=native" from CXX_FLAGS (#1429) eeb27e1bb Add handy option to turn on direct I/O in db_bench (#1424) c6168d13a removed some declarations from c.h which resulted in undefined symbols (#1407) bc429de49 revert fractional cascading in farward iterator b9bc7a2aa Use skiplist rep for range tombstone memtable 60a2bbba9 Makefile: generate util/build_version.cc from .in file (#1384) 9ee84067f Disable DBTest.RepeatedWritesToSameKey (#1420) f41df3045 OptionChangeMigration() to support FIFO compaction 2e8004e60 Changing the legocastle run to use valgrind_test instead of _check 9de2f7521 revert Prev() in MergingIterator to use previous code in non-prefix-seek mode 24495186d DBSSTTest.RateLimitedDelete: not to use real clock 1168cb810 Fix a bug that may cause a deleted row to appear again 99c052a34 Fix integer overflow in GetL0ThresholdSpeedupCompaction (#1378) f83cd64c0 Fix a bug that mistakenly disable regression_test.sh to update commit (#1415) 0e926b84f Passing DISABLE_JEMALLOC=1 to valgrind_check if run locally 4dfaa6610 Make IsDeadlockDetect() virtual member of Transaction 59a7c0337 Change ioptions to store user_comparator, fix bug 869ae5d78 Support IngestExternalFile (remove AddFile restrictions) 1d9dbef64 Restrict running condition of UniversalCompactionTrivialMoveTest2 4edd39fda Implement deadlock detection 48fd619a4 Minor fixes to RocksJava Native Library initialization (#1287) 48e4e842b Disable auto compactions in memory_test and re-enable the test (#1408) fb2e41294 column_family_test: disable some tests in LITE 5af651db2 fix data race in compact_files_test a0ba0aa87 Fix uninitialized variable gcc error for MyRocks b88f8e87c Support SST files with Global sequence numbers [reland] 08616b493 [db_bench] add filldeterministic (Universal+level compaction) 52c9808c3 not split file in compaciton on level 0 5e0d6b4cc fix db_stress assertion failure ab5399837 Bump RocksDB version to 4.13 (#1405) b4d07123c SamePrefixTest.InDomainTest to clear the test directory before testing aa09d0338 Avoid calling GetDBOptions() inside GetFromBatchAndDB() 6fbe96baf Compaction Support for Range Deletion 257de78d9 Remove "-Xcheck:jni" from Java tests (#1402) d88dff4ef add seeforprev in history 5027dd17a Fix a minor bug in the ldb tool that was not selecting the specified (#1399) fea6fdd67 Fix @see in two Java functions (#1396) b1031d6c1 Remove function local statics that interfere with memory pooling (#1392) f47054015 Handle WAL deletion when using avoid_flush_during_recovery e29d3b67c Make max_background_compactions and base_background_compactions dynamic changeable 21e8daced fix assertion failure in Prev() b9311aa65 Implement WinRandomRW file and improve code reuse (#1388) a249a0b75 check_format_compatible.sh to use some branch which allows to run with GCC 4.8 (#1393) 040328a30 Remove an assertion for single-delete in MergeHelper::MergeUntil 8cbe3e10c Relax the acceptable bias RateLimiterTest::Rate test be 25% f26a139d8 Log successful AddFile 5691a1d8a Fix compaction conflict with running compaction 017de666c fixup commit 1b7af5fb1 Redo handling of recycled logs in full purge 27bfe327b Editorial change to README.md 89cc404de A bit of doc restructuring 9e7fda829 Fix arcanist 2e4b5cab0 Add missing RateLimiter class to the Windows build (#1382) ce4963fdf [doc] Document that Visual Studio 2015+ is now required for Windows builds (#1389) e48927098 Fix scoped arena iterator (#1387) f8d8cf53f Fix log_write_bench -bytes_per_sync option. (#1375) 02b3e3985 Make txn->GetState() const 447f17127 new Prev() prefix support using SeekForPrev() 991b585ee More block cache tickers d6ae6dec6 Add Statistics::getAndResetTickerCount(). aea3ce4c8 Avoid string CONCAT which is not supported in cmake 2.6 (#1383) 2ad68b971 Support running consistency checks in release mode 67501cfc9 Fix -ve std::string::resize 04b02dd12 Testing asset links after config change 8c55bb87c Make Lock Info test multiple column families d06232897 Revert "Support SST files with Global sequence numbers" 5cd28833a [RocksJava] Adjusted RateLimiter to 3.10.0 (#1368) 37737c3a6 Expose Transaction State Publicly 2c1f95291 Add facility to write only a portion of WriteBatch to WAL 043cb62d6 Fix record_size in log_write_bench, swap args to std::string::assign. (#1373) 4985f60fc env_mirror: fix a few leaks (#1363) 5aded67dd update of c.h (#1371) 912aec13c "Recent Posts" -> "All Posts" 7cbb298db Make sure that when contribtuing we call out creating appropriate directories a06ad4711 Add top level doc information to CONTRIBUTING.md 3fdd5b971 A little more generic CONTRIBUTING.md ed4fc31db Add link to CONTRIBUTING.md to main docs README.md e4922e181 Forgot to truncate one blog post 6d8cd7ede Add CONTRIBUTING.md for rocksdb.org contribution guidance bd55e5a1e Fix some formatting of compaction blog post 0f60358b0 CRLF -> LF mod (including removing trailing whitespace for those files) b90e29c90 Truncate posts on the main /blog/ page 0d7acadaf Add author fields to blog posts 01be44181 Add GitHub link to the landing page header 9d6c96138 Fix Mac build ab01da543 Support SST files with Global sequence numbers d346ba246 Minor fixes around Windows 64 Java Artifacts (#1366) e91b4d0cf Add factory method for creating persistent cache that is accessible from public be1f1092c Expose transaction id, lock state information and transaction wait information 6009c473c Store range tombstones in memtable 3c21c64c7 Use size hint for HashMap in multiGet. Similar to https://github.com/facebook/rocksdb/pull/1344 (#1367) 13f7a01f6 Fixing JNI release build for gcc (#975) 7260662b3 Add Java API for SstFileWriter 26388247a delete unused variable for PrevInterval() 87dfc1d23 Fix conflict between AddFile() and CompactRange() eb44ed655 Update 2016-09-28-rocksdb-4-11-2-released.markdown e4437610d Update 2016-09-28-rocksdb-4-11-2-released.markdown 501f05108 Update 2016-09-28-rocksdb-4-11-2-released.markdown dec9009f8 Update 2016-09-28-rocksdb-4-11-2-released.markdown 4ed69dd0b Create 2016-09-28-rocksdb-4-11-2-released.markdown 21f4bb5a8 cmake support for linux and osx (#1358) 4defe306f fix typo in comments (#1360) f517d9dd0 Add SeekForPrev() to Iterator eb3894cf4 Recompute compaction score on SetOptions (#1346) 5c64fb67d Fix AddFile() conflict with compaction output [WaitForAddFile()] 9e9f5a0b9 Fix CompactFilesTest.ObsoleteFiles timeout (#1353) c2a62a4cb not cut compaction output when compact to level 0 9ed928e7a Split DBOptions into ImmutableDBOptions and MutableDBOptions 4bc8c88e6 Recover same sequence id from WAL (#1350) 0a1bd9c50 add cfh deletion started listener da5a9a65c Fix mac build d45eb6c6d Fix typo (#1349) abc0ae462 Add AddFile() InternalStats for Total files/L0 files/total keys ingested 715256338 forbid merge during recovery 5735b3dc2 Fix compiling under -Werror=missing-field-initializers 654ed9a28 loose the assertion condition of rate_limiter_test e4d3f5d9b Fix DBImpl::GetWalPreallocateBlockSize Mac build error 7afbb7420 solve the problem of table_factory_to_write_=nullptr (#1342) d78a4401b DBImpl::GetWalPreallocateBlockSize() should return size_t 42ac9c5f1 Retry getting arcanist token on failure b666f8544 Consider more factors when determining preallocation size of WAL files 4c3f4496b Add TableBuilderOptions::level and relevant changes (#1335) 3edb9461b Avoid hard-coded sleep in EnvPosixTestWithParam.TwoPools 0a88f38b7 Remove ColumnFamilyData::options() 41a9070f8 Fix java makefile dependencies 8d9bf5c49 Fix DBOptionsTest.GetLatestOptions 40cfa3e02 Fix DBWALTest.RecoveryWithLogDataForSomeCFs with mac 06b4785fe Fix recovery for WALs without data for all CFs d7242ff4d Fix GetSortedWalFiles when log recycling enabled 17f76fc56 DB::GetOptions() reflect dynamic changed options 215d12826 Fix typo (#903) a958c2643 Rename jvalue to jval in rocksjni 0a165bd7d Have Facebook link point to RocksDB on FB 3639f3288 Fix bug in UnScSigned-off-by: xh931076284 <931076284@qq.com> (#1336) 8e061f974 Refactor GetMutableOptionsFromStrings 81747f1be Refactor MutableCFOptions ba65c816b Support POSIX RandomRWFile 1d980a8e3 Create CNAME 2adab1dde Add API links to the header bar a182b2981 Preserve blog comments in markdown f54de9230 Adding Dgraph to list of Users (#1291) 9e4aa798c Summary: (#1313) a10e8a056 Fix C api memtable rep bugs. (#1328) eb1d4d53c Release RocksDB 4.12 22d88e24d Allow an offset as well as a length to be specified for byte[] operations in RocksJava JNI (#1264) b06b19136 add C api for set wal_recovery_mode (#1327) 1cca09129 Temporarily revert Prev() prefix support de28a2553 Update HISTORY.md for thread-local stats 0fcb6dbed Remove extraneous function prototypes from c.h (#1326) 52ee07b02 Move AddFile() tests to external_sst_file_test.cc 66a91e260 Add NoSpace subcode to IOError (#1320) 67036c040 Fix Flaky ColumnFamilyTest.FlushCloseWALFiles 0e2da497c fix typo in option.h's comment (#1321) 6d61358a0 Add real Google Analytics ID 2d9d36ea4 Have "Edit on GitHub" point to master instead of gh-pages 937751898 Update landing page content 1ec75ee76 Add redirects from old blog posts link to new format 607628d34 Support ZSTD with finalized format ce1be2ce3 Fix build error on Windows (AppVeyor) (#1315) f7669b40b Fix Windows Build 22696b088 Fix uninitlized CompactionJob::SubcompactionState::current_output_file_size c1865e0f7 Trigger more tests per diff a88677d2c Remove ImmutableCFOptions from public API 80c75593e Fix data race in AddFile() with multiple files + custom comparator bug 5051755e3 Fix db_bench memory use after free (detected by clang_analyze) 4fd08f4b8 Ensure Correct Behavior of StatsLevel kExceptDetailedTimers and kExceptTimeForMutex (#1308) e14fbaae2 Add FAQ based on the front page of the current rocksdb.org 3c2262400 Migrate the RocksDB Worpdress blog over to Jekyll ee0e2201e Transfer the current Getting Started contents to GitHub Pages 5a0e9a4cf Initial Landing Page 9447a8540 Remove the `doc` directory 32149059f Merge options source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes 4590b53a4 add stats to Cache::LookUp() 85bb30825 Expose Utility function StringToMap() (#1306) 8ce1b8440 Fix Travis on Mac 380e651af Fix Mac build failure (#1309) 1613fa949 Thread-specific histogram statistics 6a14d55bd add prefix_seek_mode to db_iter_test de47e2bd4 Fix ClockCache memory leak f099af4c7 Fix travis db74b1a21 fix bug in merge_iterator when data race happens b18f9c9ea add nullptr check to internal_prefix_transform 4e395e875 Update docs README.md 2482d5fb4 support Prev() in prefix seek mode 7541c7a79 Fix cache_test valgrind_check failure c8513cde0 Update the download location of Snappy (#1304) b49b92cf2 Introduce Read amplification bitmap (read amp statistics) c7004840d store prefix_extractor_name in table 4ad928e17 add comment to SimCache to estimate actual capacity e9b2af87f Expose ThreadPool under include/rocksdb/threadpool.h 23a057007 Document memtable flush behavior in CancelAllBackgroundWork() dade61ac2 Mitigate regression bug of options.max_successive_merges hit during DB Recovery cce702a6e [db_bench] Support single benchmark arguments (Repeat for X times, Warm up for X times), Support CombinedStats (AVG / MEDIAN) 3586901f8 cat tests logs sorted by exit code b2ce59537 Persist data during user initiated shutdown 4b3438d2d Fix parallel valgrind (valgrind_check) a081f798b Relax consistency for thread-local ticker stats b10d65c2a Update and slightly clarify instructions in build_detect_platform (#1301) f85f99bf6 Fix the Windows build of RocksDB Java. Similar to https://github.com/facebook/rocksdb/issues/1220 (#1284) 7b8109517 Fix a crash when compaction fails to open a file 7c9586837 Thread-specific ticker statistics ea9e0757f Add initial GitHub pages infra for RocksDB documentation move and update. (#1294) 2a9c97108 [Flaky Test] Disable DBPropertiesTest.GetProperty d76ddf327 Disable ClockCache db_crashtest cec2c6436 fix data race in NewIndexIterator() in block_based_table_reader.cc badbff65b Not insert into block cache if cache is full and not holding handle 4a16c32ec Option to cache index/filter blocks with priority 99c4af716 Make ClockCache available with TSAN build f57bc1d03 Fix lambda expression for clang/windows 5440675c3 Fix lambda capture expression for windows 6584cec8f Fold function for thread-local data 817eeb29b Add singleDelete to RocksJava (#1275) ffdf6eee1 Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273) ecf900386 Fix bug in printing values for block-based table 72f8cc703 LRU cache mid-point insertion 6a17b07ca Add TablePropertiesCollector support in SstFileWriter 78837f5d6 TableBuilder / TableReader support for range deletion 4cc37f59e Introduce ClockCache ff17a2abf Adding TBB as dependency. 49d88be02 c abi: allow compaction filter ignore snapshot (#1268) 0b63f51fb fixes 1215: execute_process(COMMAND mkdir ${DIR}) fails to create a directory with cmake on Windows (#1219) 3981345be Small nits (#1280) 2a2ebb6f5 Move LRUCache structs to lru_cache.h header 2fc2fd92a Single Delete Mismatch and Fallthrough statistics 3771e3797 WriteBatch support for range deletion 236756f2c Make SyncPoint return immediately when disabled 64a0082c6 Fix DBSSTest::AddExternalSstFileSkipSnapshot valgrind fail dd7a748cf Fix java build 4fe12baa6 Make db_bench less space for --stats_per_interval 6525ce4ca Compaction stats printing: "batch" => "commit group" a117891b4 Fixed typo (#1279) b248e98cf Fix a destruction order issue in ThreadStatusUpdater deda159b5 Added min/max/avg data block size output to sst_dump e408e98c8 add Name() to Cache a297643f2 Fix valgrind memory leak d11c09d9e Eliminate memcpy from ForwardIterator d36755502 Added further Java API options for controlling concurrent writes ebdfe34cc Exposed further Java API options for controlling compaction d1be59463 Improve documentation of SliceTransform. 6056d6317 Improve comment and bug fix for GetOptionsFromMap functions in convenience.h 76a67cf74 support stackableDB as the baseDB of transactionDB 67c1ae883 Travis build break fix b693ba68b Minor PinnedIteratorsManager Refactoring db3dfb164 Fixes for arcanist config (#1271) 87c91bd87 Persistent Read Cache (8) Benchmark tooling 2914de64e add sim_cache stats to Statistics 8b79422b5 [Proof-Of-Concept] RocksDB Blob Storage with a blob log file. 4beffe001 Fix test data race in two FaultInjectionTest tests 821bcb0b3 util/arena.cc: FreeBSD: More portable use of mmap(MAP_ANON) (#1254) 5370f44a8 Increase RocksDB version 56dd03411 read_options.background_purge_on_iterator_cleanup to cover forward iterator and log file closing too. ccecf3f4f UniversalCompaction should ignore sorted runs being compacted (when compacting for file num) 1b0069ce2 Remove non-gtest from parallelized tests 638c49f24 Change HISTORY.md for release 4.11 6b8e9c68b fix vs generator (#1269) c38b075e7 Update HISTORY.md 8f399e3fe Update HISTORY.md 98d0b78ea Added check_snapshot option in the DB's AddFile function (#1261) 9fd68b7fb set travis open file descriptor limit 59ddb5059 Fix travis build break f4d986364 Added SetOptions support to RocksJava (#1243) 7882cb977 Make DBOptionsTest::EnableAutoCompactionAndTriggerStall less falky 44f5cc57a Add time series database (resubmitted) 7c4615cf1 A utility function to help users migrate DB after options change 5bb0a7f73 Update appveyor.yml 86396cc18 Update appveyor.yml c1db098dc Update appveyor.yml 7da2eaf0d Update appveyor.yml 34723b4c4 Cleanup unused variable pending_fsync_. 7cc0dbd66 cat all logs in sandcastle output 9253767a6 Correct geHistogramData() -> getHistogramData() (#1257) f35b16f24 db_bench add an option of --base_background_compactions c3a4bea5d Fix flaky test `ObsoleteFiles` 8234faabf Fix failed test 4990c0d1a Remove deprecated LEVELDB_PLATFORM_POSIX 7323e4c8a Fix clang on macOS ee027fc19 Ignore write stall triggers when auto-compaction is disabled e4609a749 Fix Windows build issues (#1253) 2306167d3 Fix clang build failure and refactor unit test 343304e1d Use StopWatch to do statistic job in db_impl_add_file.cc cdc4eb689 Add a GetComparator() function to the ColumnFamilyHandle base class so that the user's comparator can be retrieved. 712dd27e6 Build break fixes 0155c73de Fix parallel tests `make check -j` c49ea68c5 Fix to enable running CI jobs locally 726c2f7e5 Build break fix d51dc96a7 Experiments on column-aware encodings c116b4780 Persistent Read Cache (part 6) Block Cache Tier Implementation 64046e581 Write a benchmark to emulate time series data 9ae92f50b More granular steps in the Makefile, can help with running all or single Java tests (and with ASAN build - https://github.com/facebook/rocksdb/wiki/JNI-Debugging) (#1237) 7c01d6534 [Fix Java] Remove duplicate cases in LoggerJniCallback::Logv 8796934af Added missing Java ReadOptions settings (#1109) 5e2c79658 Make DBTest.CompressionStatsTest more deterministic 557748ff7 Fix db_stress failure (pass merge_operator even if not used) 811ee2111 Bugfix to ensure that logging can be achieved from threads that are not known to the JVM (#1106) afad5bd1c Simplify thread-local static initialization 6920cde89 Remove an extra apostrophe e72ea485e add InDomain regression test 9c8ac144b Avoid duplicate task creation for RocksDB contruns d4c45428a db_stress shouldn't assert file size 0 if file creation fails d3bfd3397 Testing out parallel sandcastle changes 7efd9c25c Increse timeout in some tests 50b8d29b9 fixes 1230: Error:string sub-command REGEX, mode REPLACE needs at least 6 arguments total to command (#1231) 5c858ddd2 fix errata in libnuma test (#1244) e5b5f12b8 Change options memtable_prefix_bloom_huge_page_tlb_size => memtable_huge_page_size and cover huge page to memtable too 0ce258f9b Compaction picker to expand output level files for keys cross files' boundary too. ac0d93b08 fixes 1217: rocksdbjni javac and javah execute_processes fail on windows (#1218) 1ae46094d Appveyor badge to show master branch 8745f013f [Fix java build] Stop using non standard std::make_unique e12270dfe fix previous typo bbd6a5a18 ldb restore subcommand 9498069fc Run error-filtering script on diff-triggered tests f8061a237 Fix Statistics TickersNameMap miss match with Tickers enum 16e225f70 Fix MergeContext::copied_operands_ strings moving a4955b39a Run sandcastle tests in /dev/shm ae0ad719d Fix flaky DBSSTTEST::DeleteObsoleteFilesPendingOutputs b2a8016df Update db_bench_tool.cc (#1239) c6654588b Disable two dynamic options tests under lite build 2a6d0cde7 Ignore stale logs while restarting DBs ee8bf2e41 fixes 1228: rockdbjni loadLibraryFromJarToTemp fails when file is already present (#1232) f85df120f Re-enable tsan crash white-box test with reduced killing odds 89e4c4882 Update README.md to include appveyor badge b06ca5f86 ldb load, prefer ifsteam(/dev/stdin) to std::cin (#1207) 4ea0ab3cc Revert "Remove bashism from `make check` (#1225)" 12767b313 fixes 1220: rocksjni build fails on Windows due to variable-size array declaration (#1223) a9d512a76 Update .gitignore for internal release d5a51d4de Need to make sure log file synced before flushing memtable of one column family 89f319c2d Fix unit test which breaks lite build b50632920 Add unit test not on /dev/shm as part of the pre-commit tests b9a97181a Bump next release version 663afef88 Add EnvLibrados - RocksDB Env of RADOS (#1222) 32604e660 Fix flush not being commit while writing manifest 9ab38c45a Remove %z Format Specifier and Fix Windows Build of sim_cache.cc (#1224) 08ab1d83a Remove bashism from `make check` (#1225) f9b14be49 Re-enable TSAN crash test but only with black box crash test 68f3eb746 Run release build for CLANG and GCC 4.8.1 in pre-commit tests too e70020e4f Only cache level 0 indexes and filter when opening table reader 7bedd9440 Build break fix 68a8e6b8f Introduce FullMergeV2 (eliminate memcpy from merge operators) e70ba4e40 MemTable::PostProcess() can skip updating num_deletes if the delta is 0 2a282e5f5 DBTablePropertiesTest.GetPropertiesOfTablesInRange: Fix Flaky d9cfaa2b1 Persistent Read Cache (6) Persistent cache tier implentation - File layout 9430333f8 New Statistics to track Compression/Decompression (#1197) 515b11ffa fixes #1210: rocksdb/java/CMakeLists.txt lacks cmake_minimum_required (#1214) 876cb8bfb fixes #1212: rocksdbjni maven build does not escape slashes in groovy script (#1213) 21c55bdb6 DBTest.DynamicLevelCompressionPerLevel: Tune Threshold 4b9525358 Refactor cache.cc c6a8665b3 Update LANGUAGE-BINDINGS.md 880ee363e ldb backup support 6797e6ffa Avoid updating memtable allocated bytes if write_buffer_size is not set dda6c72ac Add DestroyColumnFamilyHandle(ColumnFamilyHandle**) to db.h 56222f57d Avoid FileMetaData copy 15b7a4ab8 Fixed output size and removed unneeded loop 6ea41f852 Fix deadlock when trying update options when write stalls efd013d6d Miscellaneous performance improvements e6f68faf9 Update Makefile to fix dependency 816ae098e fix test failure e295da126 Fix Log() doc for default level 8e6b38d89 update DB::AddFile to ingest list of sst files 296545a2c Fix clang analyzer errors 61dbfbb6c Add release build to RocksDB per-diff/post-commit tests 907f24d0e Concurrent memtable inserter to update counters and flush state after all inserts 0f691c4b5 CLI option & Rename() allow overwrite 7c190070b delete unnessary pointer cast in beginInternalTransaction() function e1b3ee8a7 Cleanup auto-roll logger flush-while-rolling test cd4178a01 Add a new feature to enforce a sync point only active on a thread b954847fc Fix release build for MyRocks by using debug-only code only in debug builds a00bf1b3c Add More Logging to track total_log_size 01f77cb19 Update USER.md to include more services at Facebook. eb53c05a3 Add comment for GetBackupInfo about returned BackupInfos order 32df9733d Add options.write_buffer_manager: control total memtable size across DB instances 5aaef91d4 group multiple batch of flush into one manifest file (one call to LogAndApply) a45ee8318 Fix a bug that accesses invalid address in iterator cleanup function 38fae9e65 fix typos in HISTORY.md (#1192) 1a11c934d Disable some persistent cache tests on linux/clang 9b5adea97 Add More Logging to track total_log_size 95d96eeeb remove LockFile ff45d1b54 if read only backup engine can't find meta dirs, return NotFound() instead of IOError() cb2476a0c fix rate limiter to avoid starvation 6b7167651 Run env_basic_test on Env::Default 9eb0b5395 Move env_basic_test cleanup to TearDown 1fe3bf829 Re-enable linux on travis 43692793e Fixed Minor Bug on Windows Build and db_bench_tool.cc (#1189) 95c192475 writable file close before reset 197b832af Update USERS.md bdb1d19a6 Fix UBSan build break caused by variable not initialized b726bf596 FreeBSD does not have std::to_string (#1190) faa7eb3b9 Improve regression_test.sh c4cef07f1 Update DBTestUniversalCompaction.UniversalCompactionSingleSortedRun to use max_size_amplification_percent = 0 892e9d304 make transaction WriteOptions modifiable 4f2b0946d fix simple typos (#1183) 3b7ed677d ColumnFamilyOptions API [CF + RepairDB part 3/3] 56ac68629 Detect column family from properties [CF + RepairDB part 2/3] 3fc713ed9 delete 2nd level children for default env 343507afb Refactor to use VersionSet [CF + RepairDB part 1/3] aa432be4b Workarounds for continuous build implementation 8cd9f04fe Test change to verify new commit detection 8a4ee7e90 Trivial change to test cont. build af6248d8b Fix max_bytes_for_level_base comment 0d7b26123 add tests to env_basic_test.cc 6576fa05a Fix minor typos and PHP source file name used to trigger the builds c4e19b77e Add a read option to enable background purge when cleaning up iterators fa813f747 Update DB::AddFile() to ingest the file to the lowest possible level d6b79e2fd Remove filter_deletes from crash_test a52e4d7d0 Framework for enabling continuous RocksDB build and tests f9bd66779 Makefile warning for invalid paths in make_config.mk 88a2776db Update SstFileWriter to use bottommost_compression if avaliable e87d5df1a TiKV use-case (#1172) 7b79238b6 Deprectate filter_deletes 4939fc389 Bulk load mode shouldn't stop ingest 3a2bccc84 Fixed a crash bug that incorrectly parse deprecated options in options_helper cf8adc971 Allow arcanist_util to work with both new and old arc versions 30a24f2d3 Add InternalStats and logging for AddFile() d26a84807 Temporarily disable travis on linux 249e796df Fix Flaky DBCompactionTest.SkipStatsUpdateTest 162c9170d Make sandcastle access secure 8366e10ff Fix clang build 0babce57f Move away from enum char value -1 812dbfb48 Optimize BlockIter::Prev() by caching decoded entries 550bf895e Minor bug fix with log name 886af5910 Fix examples/Makefile jemalloc error e3b1e3dfa Expose save points in Java WriteBatch and WBWI (#1092) f5177c761 Remove wasteful instrumentation in FullMerge (stacked on D59577) 97fd2a638 Remove dead Jenkins code and support `arc diff --preview` in RocksDB 7c919decc Reuse TimedFullMerge instead of FullMerge + instrumentation 9a33a723b Remove the comments saying allow_concurrent_memtable_write and enable_write_thread_adaptive_yield are not stable 81f6b33d9 Fix tsan error bc8af90e8 add option to not flush memtable on open() 8100ec2cd Fix libgcc broken lib path 7360db39e Add a check mode to verify compressed block can be decompressed back 2a79af1c5 Fix Java Break Related to memtable bloom bits to size ratio change 6faddd7c5 Merge db/slice.cc into util/slice.cc 5009b5326 BlockBasedTable::FullFilterKeyMayMatch() Should skip prefix bloom if full key bloom exists 2d05eaeb2 Fix name conflict in delete_shceduler_test and db_sst_test bde7d1055 Fix clang_analyze path in fbcode_config.sh ca3db5478 Fetch branches from github for format compatibility test 20699df84 memtable_prefix_bloom_bits -> memtable_prefix_bloom_bits_ratio and deprecate memtable_prefix_bloom_probes e9c1face6 Minor fix to precommit-check.py fcc47fa5f New features to precommit check script 56887f6cb Backup Options a683d4aba URI-based Env selection for db_bench 53a4bd8a6 duplicate line 3e8686961 release 4.9 update version and history b2973eaae Remove options builder 5b197bff4 Enabled Windows build for volatile tier implementation 281fbdddc Temporarily remove PersistentCacheOptions from persistent_cache_tier.h to fix unity build 2ae15b2d5 Format compatible test should cover forward compatibility up to 4.8. 00a058725 netflix use-case 5091dfc1e use branch names in format compatibility test edc764e91 Use valgrind built with gcc-4.9-glibc-2.20 8ff59b2b4 Disable PersistentCacheTierTest.VolatileCacheInsertWithEviction test under TSAN temporarily 1ba452226 Fix for GCC 5.4 (#1157) 972c895c3 Previously WARN level logging became FATAL level logging in the Java API (#1089) a73b26f60 Adding test for contiguous WAL detection 098da8348 Fix CLANG build break caused by the recent Persistent Cache change 54db29b8f Use gvfs links in dependencies.sh d755c62f9 Persistent Read Cache (5) Volatile cache tier implementation fda098461 Allow regression_test.sh to specify OPTIONS_FILE. Add header comments. 0fee89684 Fix Windows build 10d46b9c8 Update tp2 clang path to fix clang build 774a6aa29 Java API - Rename geHistogramData -> getHistogramData (#1107) 3070ed902 Persistent Read Cache (4) Interface definitions e42dc9192 Update paths for fbcode dependencies 5647fa427 stack_trace,cc: The current Stacktrace code does not compile for FreeBSD (#1153) 0d65acec0 threadpool.cc: abort() lives in stdlib.h on FreeBSD (#1155) 19dd5a61c env_chroot.cc: FreeBSD likes stdlib.h for realpaht() and friends (#1154) 5aca977be env_basic_test library for testing new Envs [pluggable Env part 3] 1147e5b05 Adding support for sharing throttler between multiple backup and restores 6e6622abb Create env_basic_test [pluggable Env part 2] e53287794 Add statistics field to show total size of index and filter blocks in block cache a791a2cf2 Java API - Bugfix for native linking of Compaction Filter (#1099) af0c9ac01 Env registry for URI-based Env selection [pluggable Env part 1] 02ec8154e allow updating block cache capacity from C (#1149) 630b732cb fix flaky sim_cache_test 62d548098 Add persistent cache to Windows build system 842958651 Fix race condition in SwitchMemtable 88acd932f Allows db_bench to take an options file 3a276b0cb Add a callback for when memtable is moved to immutable (#1137) 8cf0f86d3 Allow regression test to run db_bench at a remost host 27ad17071 Fix Windows build break 936973d14 Small tweaks to logging to track the number of immutable memtables 21c047ab4 add readahead size option (#1146) 71c7eed91 Assert boundary checks for SetPerfLevel() 5d85fdb2c add missing lock c40c4cae1 LDBCommand::SelectCommand to use a struct as the parameter 590e2617e fix delete file bug when do checkpoint (#1138) 8dfa980cb Add statically-linked library for tools/benchmarks f62fbd2c8 Handle overflow case of rate limiter's paramters 57461fba8 In-memory environment read beyond EOF 0e2000017 LDBCommand::InitFromCmdLineArgs() to move from template to function wrapper 472c06e90 Add low and upper bound values for rocksdb::PerfLevel enum 157e0633e MutexLock -> ThreadPoolMutexLock in util/threadpool.cc 23d4cf483 include/rocksdb/sst_file_writer.h should not depend on util/mutable_cf_options.h 345fd73fa Fix flaky DBTestDynamicLevel.DynamicLevelMaxBytesBase2 8fc75de32 Minor fix to disable DynamicLevelMaxBytesBase2 9dd50d990 Fix db_bench 5d660258e add simulator Cache as class SimCache/SimLRUCache(with test) d379d110e Update CMakeLists.txt for added test 21f847eda Direct IO fix for Mac 99765ed85 Clean up the ComputeCompactionScore() API def2f7bd0 Expose report_bg_io_stats option in the C API. (#1131) f89caa127 Direct IO capability for RocksDB 8f1214531 C API: Expose DeleteFileInRange (#1132) 11f329bd4 db/db_impl: restrict WALRecoveryMode when using recycled log files 2b2a898e0 db/log_reader: combine kBadRecord{Len,Checksum} for readability 34df1c94d db/log_reader: treat bad record length or checksum as EOF 7947aba68 db/log_reader: move kBadRecord{Len,Checksum} handling into ReadRecord 847e471db db/log_test: add recycle log test 4e7e41ba7 Disable lite build/testing for persistent read cache 1d725ca51 Deprecate BlockBasedTableOptions.hash_index_allow_collision=false. 0e77246ba backupable_db.cc: lambada to explictly caputre "this" when escaping scope 2073cf377 Eliminate use of 'using namespace std'. Also remove a number of ADL references to std functions. 26adaad43 Split WinEnv into separate classes. (#1128) bb98ca3c8 Implement GetUniqueId for Mac 1f2dca0ea Add MaxOperator to utilities/merge_operators/ f6e404c20 Added "number of merge operands" to statistics in ssts. 7383b64b3 Fix formatting of HISTORY.md (#1126) 0e665c399 Disable long running GroupCommitTest (#1125) 3c69f77c6 Move IO failure test to separate file 533cda90c Add GetStringFromCompressionType to include/rocksdb/convenience.h c70a9335d Fix mutex unlock issue between scheduled compaction and ReleaseCompactionFiles() 05c5c39a7 Fix build a6254f2bd Long outstanding prepare test 2ead11511 Fix TransactionTest.TwoPhaseMultiThreadTest under TSAN 1f0142ce1 Persistent Read Cache (Part 2) Data structure for building persistent read cache index 43afd72be [rocksdb] make more options dynamic bac3be7c4 Fix build issue. (#1123) f6aedb62c Fix Transaction memory leak a08c8c851 Added PersistentCache abstraction 5c06e0814 [ldb] Templatize the Selector aab91b8d8 Use generic threadpool for Windows environment (#1120) a40033639 TransactionLogIterator sequence gap fix fa3536d20 Store SST file compression algorithm as a TableProperty 40123b380 signed vs unsigned comparison fix 49815e384 [ldb] Export LDBCommandRunner c1af07ce8 Disable backupable_db_test.cc on Windows e61ba052b Isolate db env and backup Env in unit tests 560358dc9 Fix data race in GetObsoleteFiles() 5c1c90487 ldb option for compression dictionary size c27061dae [rocksdb] 2PC double recovery bug fix a657ee9a9 [rocksdb] Recovery path sequence miscount fix 8a66c85e9 [rocksdb] Two Phase Transaction 1b8a2e8fd [rocksdb] Memtable Log Referencing and Prepared Batch Recovery 0460e9dcc Modification of WriteBatch to support two phase commit f548da33e Follow symlinks in chroot directory d86f9b9c3 Fix lite build 4b3172343 Add bottommost_compression option bfb6b1b8a Estimate pending compaction bytes more accurately 258459ed5 Properly destroy ChrootEnv in env_test fca5aa6fc Initial script for the new regression test e1951b6f2 Add --index_block_restart_interval option in db_bench 730f7e2e2 Fix win build a9b3c47c8 Fix includes for clang on OS X 3f16a836a Introduce chroot Env 269f6b2e2 Revert "Modification of WriteBatch to support two phase commit" 04dec2a35 [ldb] Export ldb_cmd*.h 72c73cdc8 Java API - Add missing HEADER_LEVEL logging (#1104) 4d02bfa3a Add support for PauseBackgroundWork and ContinueBackgroundWork to the Java API (#1087) 8f65feafc Have sandcastle run lite_test for every diff 0d590d999 Make max_dict_bytes optional in options string 7ccb8d6ef BlockBasedTable::Get() not to use prefix bloom if read_options.total_order_seek = true e3c6ba37d OptimizeForSmallDb(): revert some options whose defaults were just changed 967476eae Fix valgrind (DBIteratorTest.ReadAhead) 9790b94c9 Add optimize_filters_for_hits option to db_bench a4ea345b0 Fixing lite build 24a24f013 Enable configurable readahead for iterators ff4b3fb5b Fix Iterator::Prev memory pinning bug cba752d58 sst_dump won't print size for unsupported compression type 6e801b0bd Eliminate memcpy in Iterator::Prev() by pinning blocks for keys spanning multiple blocks 1b166928c Release RocksDB 4.8.0 b8cf9130f Fix #1110, 32-bit build failure on Mac OSX (#1112) 21441c09b Fix calling GetCurrentMutableCFOptions in CompactionJob::ProcessKeyValueCompaction() 4ea6e051e Fix multiple issues with WinMmapFile fo sequential writing (#1108) f3bb024fd Fix clang build 6e56a114b Modification of WriteBatch to support two phase commit 1d2e4ef74 ldb support new WAL records a92049e3e Added EventListener::OnTableFileCreationStarted() callback e8115cea4 Revert "Use async file handle for better parallelism (#1049)" (#1105) 6a14f7a97 Change several option defaults c6c770a1a Use prefix_same_as_start to avoid iteration in FindNextUserEntryInternal. (#1102) 992a8f83b Not enable jemalloc status printing if USE_CLANG=1 029022b0f Fix crash_test a06faa632 Skip PresetCompressionDict test for lite e7899c661 Fix build issue. (#1103) 0f428c561 Fix compression dictionary clang osx error 6d4832a99 Merge pull request #1101 from flyd1005/wip-fix-typo af70f9ac6 Fix typo in build_tools/fbcode_config.sh 54de13aba Fix compression dictionary clang errors 0850bc514 Fix build on machines without jemalloc 4032145ad Configurable compression in db_bench 843d2e313 Shared dictionary compression using reference block ad573b902 Temporarily disable CompactFiles in db_stress in its default setting 1c80dfab2 Print memory allocation counters eb7398085 Fix BackupableDBTest ac0e54b4c CompactedDB should not be used if there is outstanding WAL files d719b095d Introduce PinnedIteratorsManager (Reduce PinData() overhead / Refactor PinData) 1995e34d6 Retrieve file size from proper Env 7c14abf2c Improve BytewiseComparatorImpl::FindShortestSeparator f3eb0b5b8 Make EventListenerTest.CompactionReasonLevel more deterministic 7b78d623f Shouldn't report default column family's compaction stats as DB compaction stats 995353e46 Fix null-pointer-dereference detected by Infer (https://github.com/facebook/infer) 24110ce90 Correct Statistics FLUSH_WRITE_BYTES b71c4e613 Alpine Linux Build (#990) 90ffed1f9 Update USERS.md with link to LinkedIn blog post (#1088) 99a3bf8f6 Merge pull request #1068 from daaku/c-purge-old-backups b54c34742 Use async file handle for better parallelism (#1049) c146c9be1 rocksdb_create_mem_env to allow C libraries to create mem env (#1066) 6da70c581 expose more options in the c api (#1067) 6f01687aa C rocksdb_create_iterators to expose NewIterators (#1069) 644f978c1 Fix RocksDB Lite build in db_stress 5bd4022fe Add comparator, merge operator, property collectors to SST file properties (again) 7a6045a3c fix typo in HISTORY.md 73a847ef8 Add per-level compression ratio property ee221d2de Introduce XPRESS compresssion on Windows. (#1081) 874c96ac1 Merge pull request #1083 from flabby/master 6cbffd50d Enable testing CompactFiles in db_stress b95510ddf Fix DBTest.RateLimitedDelete flakiness 6356b4d51 Fix nullptr dereference in adaptive_table 9385fd72c Delete deprecated backup classes in Java a2466c885 [db_stress] Make subcompaction random in crash_test c3c389d54 Fix column label for L0 write sum ec84bef24 New legocastle output parsing 725184b04 Fix db_block_cache_test in lite build 290883d94 Fix lite build 23089fd28 write_callback_test: clean test directory before running tests 792762c42 Split db_test.cc 40b840f29 Delete deprecated *BackupableDB interface for backups 6affd45d8 Make more tests run in parallel 47833e0ab Merge branch 'master' of github.com:facebook/rocksdb e5c614e1d Fixing snapshot 0 assertion 1b1adebe8 Fixing snapshot 0 assertion 6d436a3f8 DBTest.HardLimit made more deterministic 994d9bc82 Make parallel valgrind watch-log more readable 9d35ae649 Make DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels more deterministic cea8ed970 Fix backupable_db_test test cases that can't run by itself 4b6833aec Rename options.compaction_measure_io_stats to options.report_bg_io_stats and include flush too. 3894603fe Allow valgrind_check to run in parallel c9d668c58 Fix unit tests issues on Windows (#1078) 083cadc7e Minor fix to Java sandcastle job definition 80b74a6c6 Include ldb tool in the windows build (#914) 7c14d11eb Minor fix to java build job 535af525d BlockBasedTable::PrefixMayMatch() to skip index checking if we can't find a filter block. 09be5cad5 Minor fix to sandcastle java job definition 1aeca9733 Release RocksDB 4.7 dfc3de8b7 Split Travis unittests Job c2c8fe47f Add Java job for sandcastle 19ef3de57 Fix ManualCompactionPartial test flakiness b345b3662 Add a minimum value for the refill bytes per period value dff4c48ed BlockBasedTable::PrefixMayMatch: no need to find data block after full bloom checking 0353b853c Propagate sandcastle run error to UI b885f33a5 Parallelize travis jobs 71303e04e Update db_bench_tool.cc (#1073) 63cf15bb9 Fix option settable tests e208575b2 using java7 in runtime for hdfs env (#1072) 13e6c8e97 Relax an assertion in Compaction::ShouldStopBefore ae21d71e9 Fixed a bug in RocksDB Statistics where flush is considered as compaction 8e0e22f76 Fix Windows build by replacing strings.h include 5675d5037 Revert travis commit 91f0f1f5e fix travis a23c6052c Don't run DBOptionsAllFieldsSettable under valgrind 30d72ee43 PrefixTest.PrefixAndWholeKeyTest should run against a different directory from prefix_test 0e3cc2cf1 Add column family info to TableProperties::ToString() 2448f8037 Make sure that if use_mmap_reads is on use_os_buffer is also on 114a1b879 Fix build errors for windows 052299035 Improve sst_dump help message 0930e5e99 Update comments on include/rocksdb/perf_context.h 3b977bcdc instructing people to use java7 for hdfs (#1063) 1518b733e Change default number of cache shard bit to be 6 and max_file_opening_threads to be 16. ada88b63f fix wrong assignment of level0_stop_writes_trigger in spatialdb (#1061) 2391ef721 Embed column family name in SST file ab4c62332 Don't use version in the error message d02eb8d00 Fix unused variable warning 9278097f8 Merge pull request #1056 from facebook/igorcanadi-patch-1 cc87075d6 No need to limit to 20 files in UpdateAccumulatedStats() if options.max_open_files=-1 8a1a603fd Eliminate std::deque initialization while iterating over merge operands f38540b12 WriteBatchWithIndex micro optimization 200654067 Merge pull request #1053 from adamretter/benchmark-java-comparator f2c43a4a2 Stderr info logger b55e2165b Rocksdb backup can store optional application specific metadata 9b5198752 Adding pin_l0_filter_and_index_blocks_in_cache feature and related fixes. 2feafa3db Change some RocksDB default options a558830f8 Fixed compile warnings in posix_logger.h and coding.h 51c9464df Merge pull request #980 from adamretter/java-arm 925b5d002 Merge pull request #1054 from DCEngines/magic12 63e8f1b55 Formatted lines to adhere to 80 char limit 994b3bd69 Add support for UBsan builds to RocksDB 21700a510 to/from hex refactor 24420947d Replace kHeader by WriteBatchInternal::kHeader in few more places kHeader was moved from write_batch.cc to header file because it is being used wherever the number "12" was being used to check for record size 3bdbe8961 Merge branch 'magic12' of https://github.com/dcengines/rocksdb into magic12 78711524b In all the places where log records are read, there was a check that record.size() should not be less than 12. e7c64fb11 Imporve sst_file_manager comment 99ffb3d53 Fix perf_context::merge_operator_time_nanos calculation 07bb12d97 Update internal jemalloc and other versions ad2fdaa82 Correct a typo in a comment be9816b3d Fix data race issue when sub-compaction is used in CompactionJob e3802531f Merge pull request #1050 from yuslepukhin/support_db_test2 e7cc49cbd Add support for db_test2 for dev and CI runs 3996770d0 Add comments to perf_context skip counters 4e85b7479 Make WritableFileWrapper not screw up preallocation ec458dcde Merge pull request #1047 from PraveenSinghRao/wal_filter_ex 583157f71 Avoid overloaded virtual function b9d4fa890 Options settable tests to use a different special charactor 60e34baef Merge pull request #1044 from PraveenSinghRao/wal_filter_ex 136b8e0ca Merge from master 2dcbb3b4f Addressed review comments b1fafcaca Revert "Adding pin_l0_filter_and_index_blocks_in_cache feature." 5f8741a69 Revert "Fix failing Java unit test." 43bbb5619 tools/check_format_compatible.sh to use consistent version when testing backward and forward compatibility d7ae42b0f Fix failing Java unit test. fbbb8a614 Add test for Snapshot 0 e182f03c1 Add unit tests for RepairDB 7d371863e travis build fixes fbea4dc66 Merge pull request #1042 from SherlockNoMad/HistFix 780d2b04c Update format compatible checking tool 4f1c74a46 merge from master f8c218930 Publish log numbers for column family to wal_filter, and provide log number in the record callback 44756260a Reset block cache in failing unit test. 522de4f59 Adding pin_l0_filter_and_index_blocks_in_cache feature. be2227126 Merge pull request #1041 from yuslepukhin/adjust_for_jemalloc 4ecc03c03 Fix in HistogramWindowingImpl 2ca0994cf Latest versions of Jemalloc library do not require je_init()/je_unint() calls. #ifdef in the source code and make this a default build option. 90aff0c44 Update --max_write_buffer_number for compaction benchmarks 72224104d Forge current file for checkpoint 33d568611 Merge pull request #1040 from bureau14/master 02e62ebbc Fixes warnings and ensure correct int behavior on 32-bit platforms. 9cad56861 Merge pull request #1039 from bureau14/master 3d29f9146 Improve documentation of the allow_os_buffer parameter. 3ff98bd20 Fix no compression test b9cc42a72 Merge pull request #1038 from SherlockNoMad/HistFix f76b260ef Fix FB internal CI build failure 774922c68 Merge pull request #1026 from SherlockNoMad/Hist 17b879b91 Merge pull request #1037 from SherlockNoMad/BuildFix f11b0df12 Fix AppVeyor build error 6b03f93d4 Fix the build break on Ubuntu 15.10 when gcc 5.2.1 is used 697fab820 Updates to RocksDB subcompaction benchmarking script 58379bfb5 remove division from histogramwidowing impl 1a2cc27e0 ColumnFamilyOptions SanitizeOptions is buggy on 32-bit platforms. e778c34e7 Merge pull request #1035 from bureau14/master 5bd3da1c5 Added quasardb to the USERS.md file b2ae5950b Index Reader should not be reused after DB restart 0267655da Update change log for 4.6 release 08304c086 Expose RepairDB as ldb command fd664a27b Fix Build Error 580fede34 Aggregate hot Iterator counters in LocalStatistics (DBIter::Next perf regression) 54f6b9e16 Histogram Concurrency Improvement and Time-Windowing Support 790252805 Add multithreaded transaction test e8e6cf017 fix: handle_fatal_signal (sig=6) in std::vector >::_M_range_check | c++/4.8.2/bits/stl_vector.h:794 #174 d9620239d Cleanup stale manifests outside of full purge f71fc77b7 Cache to have an option to fail Cache::Insert() when full ee8cc3520 Merge pull request #938 from alexander-fenster/master 765597fa7 Update compaction score right after CompactFiles forms a compaction f0161c37b formatting fix aa3f02d50 Improve comment in compaction.h and compaction_picker.h 2200295ee optimistic transactions support for reinitialization badd6b784 Ignore db_test2 200080ed7 Improve snapshot handling for Transaction reinitialization 171c8e80b Update dependencies / Fix Clang 294bdf9ee Change Property name from "rocksdb.current_version_number" to "rocksdb.current-super-version-number" bf1c4089d Use pure if-then check instead of assert in EraseColumnFamilyInfo a7d4eb2f3 Fix a bug where flush does not happen when a manual compaction is running 68189f7e1 Update benchmarks used to measure subcompaction performance dfe96c72c Fix WriteLevel0TableForRecovery file delete protection 451678c8c Merge pull request #1025 from SherlockNoMad/BuildFix 58ecd9132 Fix Windows build 501927ffc [backupable db] Remove file size embedded in name workaround ef204df7e Compaction always needs to be removed from level0_compactions_in_progress_ for universal compaction e79ad9e18 Add Iterator Property rocksdb.iterator.version_number 19ea40f8b Subcompaction boundary keys should not terminate after an empty level deb08b822 Add parsing of missing DB options f8e90e875 Get file attributes in bulk for VerifyBackup and CreateNewBackup 12fd9b186 Change BlockBasedTableOptions.format_version default to 2 4572a2d8c Update current version to 4.6 74b660702 Rename iterator property "rocksdb.iterator.is.key.pinned" => "rocksdb.iterator.is-key-pinned" 6743135ea Fix DB::AddFile() issue when PurgeObsoleteFiles() is called 432f3adf2 Add DB Property "rocksdb.current_version_number" 188bb2e7a Fix formatting identified by `arc lint` 0f2d2fcff Refactored tests to use try-with-resources f8e02c782 Deprecate org.rocksdb.AbstractNativeReference#dispose() and implement java.lang.AutoCloseable 0f2fdfe23 Fix the javadoc and the formatting of the base-classes for objects with native references b5b1db167 Recompute compaction score after scheduling manual compaction 5ea9aa3c1 TransactionDB:ReinitializeTransaction 1f5954147 Introduce Iterator::GetProperty() and replace Iterator::IsKeyPinned() 67789419f Merge pull request #1020 from gongsu832/master 69c471bd9 Handle concurrent manifest update and backup creation 3373c81fa Modify build_tools/build_detect_platform to detect and set -march=z10 on Linux s390x. 990509045 Merge branch 'master' of https://github.com/gongsu832/rocksdb 3492889ab Merge pull request #1019 from javacruft/wip-omit-leaf-frame-pointer-archs 7ca731b12 build: Improve -momit-leaf-frame-pointer usage 21f17aaa6 Modified Makefile and build_tools/build_detect_platform to compile on Linux s390x. 8800975fb Make DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels more robust cd3fe675a Remove stale TODO 69c98f043 Reorder instance variables in backup test for proper destruction order 82f15fb15 Add test to make sure DropColumnFamily doesn't impact existing iterators 38201b359 Fix assert failure when DBImpl::SyncWAL() conflicts with log rolling 2568985ab IOStatsContext::ToString() add option to exclude zero counters b04691665 Redo SyncPoints for flush while rolling test 291ae4c20 Revert "Revert "Fixed the bug when both whole_key_filtering and prefix_extractor are set."" eef63ef80 Fixed CompactFiles() spuriously failing or corrupting DB 79ca039eb Relax the check condition of prefix_extractor in CheckOptionsCompatibility 4b1b4b8ae Merge pull request #1004 from yuslepukhin/child_attr 9ea2968d2 Implement ConsistentChildrenAttribute by using default implementation for now as it works. c7f1a8a46 Fix LITE build thread_local_test 0914f0ca5 Merge pull request #1003 from yuslepukhin/fix_mutexlock_pthread_build d37d348da This addresses build issues on Windows https://github.com/facebook/rocksdb/issues/1002 d825fc70d Use condition variable in log roller test 6b2a047df Fix SstFileManager uninitialized data member a3db93c26 Remove the SyncPoint usage in the destructor of PosixEnv df9ba6df6 Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage 3943d1678 Fix race conditions in auto-rolling logger d733dd572 [build] Fix env_win.cc compiler errors cf38e56f2 Fix broken appveyor build caused by D53991 351252b49 Merge pull request #998 from fengjian0106/master 133605249 fix ios build error d08d50295 Fix transaction locking 730a422c3 Improve the documentation of LoadLatestOptions a7b6f0748 Improve write_with_callback_test to sync WAL 5bcf952a8 Fix WriteImpl empty batch hanging issue 871cc5f98 fix build without gflags c90d63a23 can_unlock set but not used 44371501f Fixed a segfault when compaction fails 2f084d39b Merge pull request #992 from jofusa/jdennison/options-typo-fix 7bd284c37 Separeate main from bench functionality to allow cusomizations bd5f842bb fixes typo in options logging 1c868d684 Fix includes for env_test 545a19395 Add J to commit_prereq so comilation/execution happens in parallel 5bb7371ca [build] Evaluate test names only when db_test exists 6a2b4fcb8 Add flag to forcibly disable fallocate 92a9ccf1a Add a new compaction priority that picks file whose overlapping ratio is smallest 3dc3d1c14 Merge pull request #984 from petermattis/pmattis/comparator-iterate-upper-bound 239aaf2fc Use user_comparator when comparing against iterate_upper_bound. 908100399 Fixed a dependency issue of ThreadLocalPtr 337671b68 Add universal compaction benchmarks to run_flash_bench.sh 3a67bffaa Fix an ASAN error in transaction_test.cc c5af85eca Fix a memory leak of Slice objects from org.rocksdb.WBWIRocksIterator#entry1 e84137c8a Remove unnessecary java.util.List expense in JNI 76e8beeeb Pass by pointer from/to Java from JNI not by object 0e7e6f6e4 Improve Javadoc 18eb56305 Improve the speed and synchronization around the construction of Java/JNI objects 2a04268be Temporarily disable unstable tests in memory_test.cc 08a78b6b4 Merge pull request #979 from facebook/update_licenses 21e95811d Updated all copyright headers to the new format. 59b3ee658 Env function for bulk metadata retrieval 4a8cbf4e3 Allows Get and MultiGet to read directly from SST files. fe93bf9b5 Transaction::UndoGetForUpdate 2608219cc crash_test: cover concurrent memtable insert in default crash test a76e9093f Fix LITE db_test build broken by previous commit b1887c5dd Explictly fail when memtable doesn't support concurrent insert 8ed343877 Add option to run fillseq with WAL enabled in addition to WAL disabled 73a9b0f4b Update version to 4.5 6f71d3b68 Improve perf of Pessimistic Transaction expirations (and optimistic transactions) 8e6172bc5 Add BlockBasedTableOptions::index_block_restart_interval 34a40bf91 Add --allow_concurrent_memtable_write in stress test and run it in crash_test 73bf330c7 Merge pull request #973 from yuslepukhin/moveout_testcode f7c0f4e3e perf_context.cc and iostats_context.cc use different output macro (fix unity build) 9656eab00 This partially addresses issue https://github.com/facebook/rocksdb/issues/935 testutil.cc and testharness.cc could not be moved out at this time as they are used by 4 benchmarks in release builds. 14a322033 Remove references to files deleted in commit abb405227848581d3e6d2ba40d94dbc0a5513902 8445e5380 Add a mechanism to run all tests in sandcastle 461cec4e8 Merge pull request #972 from adamretter/wb-threads 9ab269ab3 Threaded tests for WriteBatch bf767c641 Minor fix to makefile 2c1db5ea5 always invalidate sequential-insertion cache for concurrent skiplist adds c12ff20ab Merge pull request #965 from koldat/jni_for_windows a09ce4fcd Skip some of the non-critical tests in ./tools/run_flash_bench.sh 284aa613a Eliminate duplicated property constants 94be872ea Merge branch 'master' of github.com:facebook/rocksdb 0c2bd5cb4 Removing data race from expirable transactions 5fcd1ba30 disable kConcurrentSkipList multithreaded test 466c2c1bf Generate tags for *.c files 70c068c97 Merge pull request #960 from koldat/masterFixes2 a62c519bb RollLogFile tries to find non conflicting file until there is no conflict. 57a95a700 Making use of GetSystemTimePreciseAsFileTime dynamic - code review fixes 502d41f15 Making use of GetSystemTimePreciseAsFileTime dynamic to not break compatibility with Windows 7. The issue with rotated logs was fixed other way. 52153930d Adding support for Windows JNI build - fix Java unit test for release build of JNI DLL e2972803a Adding support for Windows JNI build 9c2cf9479 Fix for --allow_concurrent_memtable_write with batching ac3fa9a6f Travis CI to disable ROCKSDB_LITE tests 7b943da1b Merge pull request #967 from SherlockNoMad/ValueSize b5750790e Merge pull request #968 from yuslepukhin/one_shot_buffer 1ad818295 Fix WriteBatchTest.ManyUpdates, WriteBatchTest.LargeKeyValue under clang ad7ecca72 Add unit tests to verify large key/value fdd70d149 Skip filters for last L0 file if hit-optimized aa5e3b7c0 PerfContext::ToString() add option to exclude zero counters 36300fbbe Enable per-request buffer allocation in RandomAccessFile This change impacts only non-buffered I/O on Windows. Currently, there is a buffer per RandomAccessFile instance that is protected by a lock. The reason we maintain the buffer is non-buffered I/O requires an aligned buffer to work. XPerf traces demonstrate that we accumulate a considerable wait time while waiting for that lock. This change enables to set random access buffer size to zero which would indicate a per request allocation. We are expecting that allocation expense would be much less than I/O costs plus wait time due to the fact that the memory heap would tend to re-use page aligned allocations especially with the use of Jemalloc. This change does not affect buffer use as a read_ahead_buffer for compaction purposes. 1d854fa3d Fixed the asan error on column_family_test 37159a644 Add histogram for value size per operation 3b2a1ddd2 Add options.base_background_compactions as a number of compaction threads for low compaction debt 6ee38bb15 Slowdown of writing to the last memtable should not override stopping d6c838f1e Add SstFileManager (component tracking all SST file in DBs and control the deletion rate) 45768ade4 transaction allocation perf improvements 77926f93e Merge pull request #964 from benoitc/fix/pi2 03a5661a1 fix build for raspberry 2 4b50f1354 Should not skip bloom filter for L0 during the query. eadd221d3 Merge pull request #959 from koldat/master d209076fa Merge pull request #961 from wingify/master 26c618004 Add Wingify to USERS.md 4265f81e8 Remove util/auto_roll_logger.cc (it was moved to different directory) d7f22b6d2 Fixing generated GenerateBuildVersion.vcxproj when one builds on different locale than english. The problem is that date and time CLI utilities generates different format so that REGEX in CMake does not work. d20915d52 Disable stats about mutex duration by default 0c433cd1e Fix issue in Iterator::Seek when using Block based filter block with prefix_extractor 035857a31 Fix RocksDB lite build 77ef87ccb Update fbcode_config4.8.1.sh to use update_dependencies.sh 955ecf8b4 Fix an ASAN error in compact_files_test b0afcdeea Fix bug in block based tables with full filter block and prefix_extractor 167bd8856 [directory includes cleanup] Finish removing util->db dependencies acd7d5869 [directory includes cleanup] Remove util->db dependency for ThreadStatusUtil 46f9cd46a [directory includes cleanup] Move cross-function test points 22ecb752d Add valgrind to pre-commit sandcastle testing b7ecf3d21 Fix intermittent hang in ColumnFamilyTest.FlushAndDropRaceCondition 38e1d7fea ldb to support --column_family option da33dfe18 Parameterize DBTest.Randomized fb9811ee9 Add a perf context level that doesn't measure time for mutex operations f7ef1a613 Include rest of dependencies in dependencies.sh 3e9209a07 Updated GetProperty documentation 40911e0b3 Run unit tests in parallel to find failing tests 2fbc59a34 Disallow SstFileWriter from creating empty sst files f53c95f81 Cosmetic fixes and comments for the reader f1ed17010 Add tests to make sure new DB or ColumnFamily options are settable through string f57596b0c Improvements to pre-commit 538eec066 Update fbcode_config.sh to use latest versions automatically 8019aa9b5 improve test for manifest write failure bcd4ccbc3 Revert D7809 b0a15e7fb Mechanism to run CI jobs on local branch via commit_prereq bb2888738 Cleanup property-related variable names 29289333d Add named constants for remaining properties 2c2b72218 Disable OptionsParserTest.BlockBasedTableOptionsAllFieldsSettable under CLANG a300d9928 Added sandcastle pre-commit 202be23e4 Add test that verifies all options in BlockBasedTableOptions is settable through GetBlockBasedTableOptionsFromString() eceb5cb1b Split db_test.cc (part 1: properties) 94918ae84 db_bench: explicitly clear buffer in compress benchmark fdbff4239 Crash test to make kill decision for every kill point 39c3e94ff Merge pull request #954 from davidbernard/solaris_build df7c2f3b5 As per google coding standard replace "using" in option_builder.cc and geodb_impl.cc 12809b44b Revert "Change notification email for travis" 34704d5c7 [easy] Fixed a crash in LogAndApply() when CF creation failed 791dbafa9 Merge pull request #953 from sselva/master 594a5ef02 Merge pull request #955 from bcbrock/ppc64-build f423f05dc Simple changes to support builds for ppc64[le] consistent with X86 3f12e16f2 Make alloca.h optional eaa563756 Change notification email for travis d78c6b28c Changes for build on solaris 2e9fae3f2 Add Rakuten Marketing to USERS.md 83e1de92a move internal build to use zstd 0.4.5 aec10f734 Guard falloc.h inclusion to avoid build breaks f7ebc2f34 Update HISTORY.mc for 4.4.0 addd9545f Merge pull request #947 from yuslepukhin/align_and_yield ac50fd3a7 Align statistics Use Yield macro to make it a little more portable between platforms. b54d4dd43 tools/sst_dump_tool_imp.h not to depend on "util/testutil.h" 48a8667c3 Merge pull request #929 from warrenfalk/fix32 d9bca1e14 Reduce iterator deletion overhead 45d794068 Merge pull request #940 from yuslepukhin/fix_windows_build_signed_unsigned 20d7902df Fix compile error. Use constructor style initialization instead of a cast for simplicity. e16438bb8 fixing build warning b73fbbaf6 added --no_value option to ldb scan to dump key only 1477dcb37 Merge pull request #937 from petehunt/master c7cb1076a Add Smyte to USERS.md df7e3b622 Include in table/plain_table_key_coding.h 235b162be Not scheduling more L1->L2 compaction if L0->L1 is pending with higher priority 9a8e3f73e plain table reader: non-mmap mode to keep two recent buffers 7ece10ecb DeleteFilesInRange: Mark files to be deleted as being compacted before applying change 94d9df248 fix an unused function compiler warning in crc32c in 32-bit mode 2f01e10fa use static_cast in crc32c instead of c-style cast 601f1306a fix shorten-64-to-32 warning in crc32c f3fb39814 Fix BlockBasedTableTest.NoopTransformSeek failure 55b37efa1 fix a compile error on 32-bit 8c71eb5af Optimize DBIter::Prev() by reducing stack overhead 73c31377b Revert "Fixed the bug when both whole_key_filtering and prefix_extractor are set." 57605d7ef Fixed the bug when both whole_key_filtering and prefix_extractor are set. 6935eb24e Add ColumnFamilyHandle::GetDescriptor() 9760c842c fix valgrind failure in backupable_db_test b1a3b4c0d Make ldb automagically determine the file type and use the correct dumping function ba8344736 Merge pull request #923 from petermattis/pmattis/prefix-may-match da032495d Optimize GetLatestSequenceForKey 260c29762 Fix index seeking in BlockTableReader::PrefixMayMatch. e541dcc8f Fix issue #921 51adc5457 fix sporadic failure in fault_injection_test a2422f053 fix potential test SleepingTask race condition 1627c4b1b Merge pull request #918 from mkurdej/fix/assertion-on-no-disk-space 92d0850f1 Fix failing assertion in logger on Windows when the disk is full. 7699439b7 Prevent the user from setting block_restart_interval to less than 1 4041903ec Enhance db_bench write rate limit 399343205 Add Airbnb and Pinterest to USERS.md d74c9f0a5 DeleteFilesInRange: Clean job context if no files deleted 1dec5b8f5 Merge pull request #916 from warrenfalk/capi_huge_page_option 12fa27b4f Merge pull request #915 from warrenfalk/capi_full_bloom 0fde291ab expose memtable_prefix_bloom_huge_page_tlb_size option to C API 7e81dba5c Support creation of "full" format bloom filter from C API bae5b0a1d Fix clang build ac16663bd use -Werror=missing-field-initializers, to closer match MyRocks build ab5a9a66d Merge pull request #911 from shuzhang1989/fix_envhdfs_virtual_func eb5a13904 update posix env schedule call a41f68ac2 fix inconsistency between env_hdfs and env 7238be090 Fix clang build in db_compaction_test c9e2490bc Fix DynamicBloomTest.concurrent_with_perf to pass TSAN 63ddb783d Delete files in given key range d8677a8d2 Upgrade internal CLANG version for FB-internal gcc 4.8.1 edf1cd497 Not generating "__attribute__((__unused__))" for padding fields if it is not CLANG 9eb4f9596 Merge pull request #907 from siying/master 22c0ed8a5 Disable Visual Studio Warning C4351 fcafac053 Fix memory leak in ColumnFamilyTest.WriteStall* b99d4276f Fix java test buid broken by 7d87f02799bd0a8fd36df24fab5baa4968615c86 11672df19 Fix CLANG errors introduced by 7d87f02799bd0a8fd36df24fab5baa4968615c86 7fafd52dc Merge pull request #900 from shuzhang1989/hdfs_env_fix 2b7c810db more foramt b79ccbd57 indent 7d87f0279 support for concurrent adds to memtable 5b2587b5c DBTest.HardLimit use special memtable b4aa82366 format 4dfdd1d92 format 298ba27ae Merge pull request #846 from yuslepukhin/enble_c4244_lossofdata 7810aa802 Merge pull request #899 from zhipeng-jia/fix_clang_warning 4c5560d70 Merge pull request #895 from zhipeng-jia/develop d43da8ae0 DBTest.DelayedWriteRate: fix assert of sign and unsign comparison 3280ae9a2 Fix warning in release ec2664fef Fix clang compile error under Linux 4fd23fb13 add a factory method for creating hdfs env 9c176ef90 Update liblz4 to r131 15b890226 Change default options.delayed_write_rate 73b175a77 Fix clang warnings regarding unnecessary std::move b9f77ba12 When slowdown is triggered, reduce the write rate 445d5b8c5 Fix clang build e089db40f Skip bottom-level filter block caching when hit-optimized 06c05495e Merge pull request #898 from zhipeng-jia/fix_move_warning aa515823b Fix clang warning 2ba03196d Merge pull request #897 from yuslepukhin/enable_status_move dbb8260f7 Make Status moveable Status is a class which is frequently returned by value from functions. Making it movable avoids 99% of the copies automatically on return by value. 2bf9b968c Fix lite_build d005c66fa Report compaction reason in CompactionListener 728f944f0 Fix computation of size of last sub-compaction 8ac7fb837 Merge pull request #863 from zhangyybuaa/fix_hdfs_error e53e8219a Merge pull request #894 from zhipeng-jia/develop e0abec158 Sorting std::vector instead of using std::set 33e09c0e1 add call to install superversion and schedule work in enableautocompactions 22c6b50ee Merge pull request #893 from zhipeng-jia/develop 24c7dae13 Fix clang warning regarding implicit conversion eff309867 Do not use timed_mutex in TransactionDB 97ea8afaa compaction assertion triggering test fix for sequence zeroing assertion trip 521da3abb Fix BlockBasedTableTest.BlockCacheLeak valgrind failure a48382399 Fix use-after free in db_bench bf8ffc1d6 Merge pull request #890 from zhipeng-jia/develop 131f7ddf6 fix typo: sr to picking_sr c37729a6a db_bench: --soft_pending_compaction_bytes_limit should set options.soft_pending_compaction_bytes_limit 7b12ae97d Add signalall after removing item from manual_compaction deque d72b31774 Slowdown when writing to the last write buffer 6b2a3ac92 Add documentation for unschedFunction 167fb919a ZSTD to use CompressionOptions.level 32ff05e97 Bump version to 4.4 aececc209 Introduce ReadOptions::pin_data (support zero copy for keys) e6e505a4d Fix examples aa29cc128 Improve examples/README.md 97265f5f1 Fix minor bugs in delete operator, snprintf, and size_t usage b68dc0f83 Merge pull request #885 from yuslepukhin/fix_size_t_formatting b6d19adcf Use port size_t formatting 963660eb5 Merge pull request #883 from zhipeng-jia/master 99ae549d3 Fix typo 636cd3c71 Clean up listener_test (reuse db_test_util) 030215bf0 Running manual compactions in parallel with other automatic or manual compactions in restricted cases d26a4ea62 Merge pull request #882 from SherlockNoMad/BuildFix 768a61486 Fix appVeyor Build problem aca403d2b Fix another rebase problems. a6fbdd64e Fix rebase issues and new code warnings. 3fa68af31 Enable MS compiler warning c4244. Mostly due to the fact that there are differences in sizes of int,long on 64 bit systems vs GNU. 236fe21c9 Enable MS compiler warning c4244. Mostly due to the fact that there are differences in sizes of int,long on 64 bit systems vs GNU. 84f98792d Transaction::SetWriteOptions() 3bfd3d39a Use SST files for Transaction conflict detection 362d819a1 Improving parser 00d6edf6a Ensure the destruction order of PosixEnv and ThreadLocalPtr 64fa43843 Merge pull request #862 from ceph/wip-env 2074ddd62 env: add EnvMirror a3ba5915c Correct a comment in include/rocksdb/cache.h f0a8e5a2d Fixed the valgrind error in ColumnFamilyTest::CreateAndDropRace 9e4462906 Change SingleDelete to support conflict checking c5af8bffb Merge pull request #879 from charsyam/feature/typos c30b49954 fix typos in comments 56e77f096 Deprecate options.soft_rate_limit and add options.soft_pending_compaction_bytes_limit d6e1035a1 A new compaction picking priority that optimizes for write amplification for random updates. de6958b2e Merge pull request #877 from yuslepukhin/fix_unnecessary_type_truncation 0991cee6c Merge pull request #815 from SherlockNoMad/CounterFix 49957f9a9 Prefer integer arithmetics The code had conversion to double then casting to size_t and then casting uint32_t which caused compiler warning (VS15). 0836d265c Merge pull request #876 from warrenfalk/wf_win_master c6fedf2bf Add compaction_iterator and delete_scheduler tests to Windows build ac8e56f05 db_bench: in uncompress benchmark, get Snappy size from compressed stream 9c227923c Merge pull request #788 from OpenChannelSSD/to_fb_master2 fa3dbf203 Merge pull request #853 from Vaisman/enable_C4267_warning ad6aaf4fa Merge pull request #848 from SherlockNoMad/db_bench 56bbecc31 Merge pull request #867 from SherlockNoMad/CacheFix 188170fb4 Updating HISTORY.md 758dbec7f Fix fb-only build for gcc 4.8.1 774b80e99 Resubmit the fix for a race condition in persisting options afc84731f Include ldb_tools and sst_dump_tools libraries in shared library e5c5f2381 Support marking snapshots for write-conflict checking - Take 2 ea1192355 Upgrade to ZSTD 0.4.2 b60cb88c7 Update examples/rocksdb_option_file_example.ini 3d8bb2c89 Fix valgrind failure in IncreaseUniversalCompactionNumLevels 7af91d425 Merge pull request #873 from yuslepukhin/make_vs15_build 1d63c3d61 Revert "Support marking snapshots for write-conflict checking" 78de0c922 Fix up VS 15 build. Fix warnings Take advantage of native snprintf on VS 15 ec704aafd Support marking snapshots for write-conflict checking 770dea932 Fix occasional failure of DBTest.DynamicCompactionOptions ebc2d490d Split histogram per OperationType in db_bench f307036bd Revert "Fix a race condition in persisting options" 2fa3ed518 Fix a race condition in persisting options f276c3a82 Fix valgrind failures in 3 tests in db_compaction_test due to new skiplist changes 291088ae4 Fix undeterministic failure of ColumnFamilyTest.DifferentWriteBufferSizes 3c2b995fb Merge branch 'master' of https://github.com/facebook/rocksdb into CacheFix 355fa9436 EstimatedNumKeys Counter Inaccurate b2863017b Move posix threads into a library 3a98a7ae7 Replace malloc with new for LRU Cache Handle a9ca9107b Fix db_universal_compaction_test d3bb572da Build break fix. b28b7c6dd Added callback notification when a snapshot is created e8180f990 added public api to schedule flush/compaction, code to prevent race with db::open 19b1201b2 Merge pull request #865 from yuslepukhin/fix_db_table_properties_test e0de7ef87 Avoid empty ranges vector with subsequent zero element access a330f0b3b Fix incorrect merge in db/db_compaction_test.cc bd7a49d44 Make DBCompactionTestWithParam::CompactionTrigger more deterministic be006d288 fix LinkFile() undefined reference error 4687ced5d fix ToString() not declared error bcd7bd122 Relax verification condition of DBTest.SuggestCompactRangeTest f9103d9a3 DBTest.DynamicCompactionOptions: More deterministic and readable 0ad68518b Fix DBCompactionTestWithParam.CompactionTrigger in non-jemalloc build. 459c7fba3 Revert previous behavior of internal_key_skipped_count 481f9edb1 Fix CLANG build d7421c22f Fixed some typos in the comments of rocksdb options file example ef8ed3681 Fix DBTest.SuggestCompactRangeTest for disable jemalloc case db320b1b8 DB to only flush the column family with the largest memtable while option.db_write_buffer_size is hit 4a009f917 Merge pull request #860 from SherlockNoMad/BuildFix b4efaebff Fix ms version Appveyor build error d27ea4c9e Initialize options.row_cache 72930485b Fix clang build 6bbfa1874 BackupDB to have a mode to use file size in file name f3ea00bc8 Merge pull request #856 from ceph/wip-env 4cedd6b03 EnvWrapper: add ReuseWritableFile 33e0c9382 Reduce extra key comparision in DBIter::Next() 9a9d4759b InlineSkipList part 3/3 - new skiplist type that colocates key and node 520172954 InlineSkipList - part 2/3 78812ec6b InlineSkipList - part 1/3 ffb466da4 Merge pull request #855 from yuslepukhin/enable_3rdparty_override 10d257d64 Enable override to 3rd party linkage 41b32c605 Enable C4267 warning c5b467306 Fix race condition that causes valgrind failures efb01a055 Merge pull request #850 from yuslepukhin/enable_2015_build 81be49c75 Have a way for compaction filter to ignore snapshots 047bd22aa Build on Visual Studio 2015 Update 1 88e052772 Reduce moving memory in LDB::ScanCommand 890f44f46 Merge pull request #844 from yuslepukhin/enable_C4804_unsafe_bool 9d0b8f19d plain table reader: avoid re-read the same position for index and data in non-mmap mode 89bacb7e7 Enable MS Warning C4804 : unsafe use of type 'bool' in operation d5239f870 build_tools/fbcode_config4.8.1.sh: upgrade versions of some dependencies c4ebb66d6 Not to build forward_iterator_bench now c342549d0 Merge pull request #841 from yuslepukhin/fix_test_rerun_logic 7cb1293b6 Fix log names when scheduling runs and reruns 51fce92e1 "ldb compact" should force bottommost level compaction f83164120 Merge pull request #837 from yuslepukhin/rerun_concurrency_value 4159ab816 Merge pull request #839 from SherlockNoMad/memtableOption 6170fec25 Fix build broken by previous commit of "option helper refactor" 3a6643c2f Merge pull request #805 from SherlockNoMad/OptionHelperFix 189b3e03d Fix uninitilizeded SpecialEnv::time_elapse_only_sleep_ d5540e18e DBTest.MergeTestTime to only use fake time to be determinstic bd7be035e Support Memtable Factory Parse in option_helper.cc 94e39e236 Exclude DBTest.FileCreationRandomFailure as a long running test Increase concurrency to 18 Fix exclusion but in the ps script 4189c0f9a Fix Java Makefile 605a24d94 Block forward_iterator_bench under MAC and Windows 2a0510c9f Failed tests must be rerun with concurrency 1 9b8c9be0b Fix forward_iterator allocation of vector. 5cbb7e43e DBTest.MergeTestTime: relax counter upper bound verification 52e04b3d0 Merge pull request #833 from yuslepukhin/fix_win_build_after_lint 314f62194 Remove headers from the cc since they are in the module's header. 472c74006 Add necessary headers after cpplint rearranged includes 9bc9c93bd Move to version 4.3 3381e2c3e Handle multiple calls to DBImpl::PauseBackgroundWork() and DBImpl::ContinueBackgroundWork() 65a042921 Merge pull request #831 from yuslepukhin/remove_forward_iter_bench_win ca5566d20 Fix clang build 4175472ad Merge pull request #832 from yuslepukhin/fix_forward_iter_outofbounds cb9459f85 Fix empty vector write in ForwardIterator a163cc2d5 Lint everything 8f01f2541 Remove forward_iter_bench from Win build. dac5b248b UniversalCompactionPicker::PickCompaction(): avoid to form compactions if there is no file d06b63e99 Fix Rocksdb lite build failure in forward_iterator_bench 7824444bf Reuse file iterators in tailing iterator when memtable is flushed 2ae4d7d70 Make sure that CompactFiles does not run two parallel Level 0 compactions d781da816 Add CheckOptionsCompatibility() API to options_util 2391b459b Merge pull request #824 from yuslepukhin/try_ci_tests_on_daily 2ab3e2df2 Fix a build break so tests can run 247c49a40 Merge branch 'master' into try_ci_tests_on_daily 935d1495c Run tests imporvements Add sequential rerun for any failed tests. Add env_test case. Limit concurrency Allow to specify individual tests Take $Limit into account when displaying number of tests 5ac16300b Fixed valgrind error in options_util_test 6ce42dd07 Don't merge WriteBatch-es if WAL is disabled 56245ddcf Fixed DBCompactionTest.SkipStatsUpdateTest e11f676e3 Add OptionsUtil::LoadOptionsFromFile() API e78389b55 Fixed build failure of RocksDBLite test on options_file_test.cc e114f0abb Enable RocksDB to persist Options file. 7ed2c3e45 Merge pull request #823 from yuslepukhin/fix_off_t_type 7f59e33b1 Make CI build debug/optimized ae2dfe404 Try running db_test during integration build 720af2269 Merge branch 'fix_off_t_type' of https://github.com/yuslepukhin/rocksdb into fix_off_t_type 5270b33bd Make use of portable `uint64_t` type to make possible file access in 64-bit. 631863c63 track WriteBatch contents 505accda3 remove constexpr from util/random.h for MSVC compat 5421c9728 Make use of portable `uint64_t` type to make possible file access in 64-bit. b81b43098 Switch to thread-local random for skiplist 75a8bad2a Merge pull request #821 from yuslepukhin/continue_windows_warnings 986230b8c Revert "Fix TSAN build for fbcode" f3ca28ab0 Correct the comment of GetApproximateMemoryUsageByType 838676c17 Revert "Adding new table properties" 7c86d5049 Enable C4305 'identifier' : truncation from 'type1' to 'type2' 85a2ce9c1 Enable C4702 unreachable code 62aa1b1b7 Enable C4200 warning nonstandard extension used : zero-sized array in struct/union 5b9ce1a32 Merge pull request #820 from yuslepukhin/enable_compiler_warnings 20f57b171 Enable Windows warnings C4307 C4309 C4512 C4701 Enable C4307 'operator' : integral constant overflow Longs and ints on Windows are 32-bit hence the overflow Enable C4309 'conversion' : truncation of constant value Enable C4512 'class' : assignment operator could not be generated Enable C4701 Potentially uninitialized local variable 'name' used 8be568a9c Adding new table properties 2b42000f4 incorrect batch group size computation for write throttling c745f1d2c Fix TSAN build for fbcode fe789c5f2 Document SingleDelete e89e5b253 Merge pull request #818 from yuslepukhin/improve_test_concurrency ae7940b62 Fix regression failure in PrefixTest.PrefixValid 3277d172b Improve concurrency when running tests PowerShell seems to have a hard time when a flood of async tasks is scheduled at the same time. I speculated that WaitForMultipleObjects() in Windows can only take up to 64 process handles and if you want to handle more than you should write some additional code which can be sub-optimal. I.e to implement Wait-Job -Any. I decided to test that suggestion and introduced a $Concurrency parameter with a default value of 62. So in the new version the script fires up up to $Concurrency value and wait for anything to complete before starting any more processes. This improved matters greatly. Individual tests against ramdrive now run in 8 minutes and all of the 200+ db_tests run in 9 minutes with concurrency values of 8-16. About 48 is required to load a CPU on my box running against HD but that does not improve running times much. c8e01ef98 Delete test iterators 9d50afc3b Prefix-based iterating only shows keys in prefix 14c6e1a04 Add write_stress to RocksDB Legocastle runs db3f5e494 Update HISTORY.md 042fb053f Fix clang 2419f435a Merge pull request #816 from SherlockNoMad/GeoDBTestFix 2e4540991 Fix appveyor build failure 183cadfc8 Add OptionsSanityCheckLevel dba5e0074 Fixed the compile error in RocksDBLite in memory_test.cc 7d7ee2b65 Add Memory Insight support to utilities 3ecbab004 Add GetAggregatedIntProperty(): returns the aggregated value from all CFs 93a966722 Merge branch 'master' of github.com:facebook/rocksdb c9aef3c41 Add RocksDb/GeoDb Iterator interface f31442fb5 Merge pull request #803 from SherlockNoMad/SkipFlush dcc898b02 Merge pull request #812 from yuslepukhin/fix_windows_warnings df7ed91ef Fix white space at end of line a0163c068 Do not disable compiler warnings: C4101 'identifier' : unreferenced local variable C4189 'identifier' : local variable is initialized but not referenced C4100 'identifier' : unreferenced formal parameter C4296 'operator' : expression is always false 279c8e0cd Merge pull request #811 from OverlordQ/unused-variable-warning affd83369 Fix introduced in 2ab7065 was reverted by 18285c1. db68a2c09 Merge pull request #806 from yuslepukhin/signed_unsigned_warning ccc8c10c0 Move skip_table_builder_flush to BlockBasedTableOption eaaf081d1 Do not suppress C4018 'expression' : signed/unsigned mismatch The code compiles cleanly for the most part. Fix db_test. Move debug file to testutil library. ff4499e29 Update DB::AddFile() to have less restrictions 9ac88c855 Merge branch 'master' of https://github.com/facebook/rocksdb into OptionTestFix 84992d647 Option Helper Refactoring 11c71a365 db_bench: --compaction_pri default should be rocksdb::Options().compaction_pri 335e4ce8c options_test: fix a bug of assertion 66a3a87ab Merge pull request #797 from SherlockNoMad/optionHelper 550af4ee6 Fix Travis Build Error a6dd0831d Add Option to Skip Flushing in TableBuilder 2872e0c8c Clean and expose CreateLoggerFromOptions 296c3a1f9 "make format" in some recent commits 6388e7f4e Merge pull request #798 from yuslepukhin/readahead_buffermanagement f4cbb90c4 Merge pull request #799 from yuslepukhin/fix_random_generator_compile 1277a48f1 Fix 80 character limit issue. ee2c3236d Fix compilation problem on Windows. char is not a valid template parameter for std::uniform_int_distribution according to the standard. Replacing with int should be just fine. b69b9b624 Support PlainTableOption in option_helper c97667d9f Fix RocksDB lite build for write_stress 0d720dfc1 Use the correct variable when fetching table properties. 4b66d9534 Write stress test 47414c6cd Move include/posix/io_posix.h to util/io_posix.h 2889df84c Revert "Avoid to reply on ROCKSDB_FALLOCATE_PRESENT in include/posix/io_posix.h" 28c8758a3 Merge pull request #795 from yuslepukhin/fix_mocktable_id 5c8f2ee78 Fix MockTable ID storage On Windows two tests fail that use MockTable: flush_job_test and compaction_job_test with the following message: compaction_job_test_je.exe : Assertion failed: result.size() == 4, file c:\dev\rocksdb\rocksdb\table\mock_table.cc, line 110 72d6e758b Fix WritableFileWriter::Append() return d0a18c284 Merge pull request #786 from aloukissas/unused_param c37223c08 Avoid to reply on ROCKSDB_FALLOCATE_PRESENT in include/posix/io_posix.h 6fbc4f9f3 Implement smart buffer management. introduce a new DBOption random_access_max_buffer_size to limit the size of the random access buffer used for unbuffered access. Implement read ahead buffering when enabled. To that effect propagate compaction_readahead_size and the new option to the env options to make it available for the implementation. Add Hint() override so SetupForCompaction() call would call Hint() readahead can now be setup from both Hint() and EnableReadAhead() Add new option random_access_max_buffer_size support db_bench, options_helper to make it string parsable and the unit test. d6219e4d9 Mac build break caused by include/posix/io_posix.h not declearing errno, beb69d451 Merge pull request #765 from PraveenSinghRao/wal_filter ab0f3b964 crash_test to trigger some less frequent crash point more frequently 7beb743cf Merge pull request #778 from Vaisman/master 4ce117c4d Merge branch 'master' into wal_filter 32cdec634 Fail recovery if filter provides more records than original and corresponding unit-test, fix naming conventions 44d4057d7 Avoid some includes in io_posix.h 2adad23a1 Fix unused parameter warnings. b0980ff74 Fix unused parameter warnings. bc898c5f8 Fix unused parameter warnings. 138876a62 Merge pull request #746 from ceph/wip-recycle 581f20fd8 Add LITE tests to Legocastle 3d56d868c Merge remote-tracking branch 'upstream/master' d69111114 include/posix/io_posix.h should have a once declartion a6962edf8 Merge pull request #783 from yuslepukhin/remove_test_conditional_compilation 3c750b59a No need to #ifdef test only code on windows 8c11c5dee Merge pull request #768 from OpenChannelSSD/to_fb_master2 6e6dd5f6f Split posix storage backend into Env and library 980a82ee2 Fix a bug in GetApproximateSizes d0d13ebf6 fix bug in db_crashtest.py 01a41af0a Merge remote-tracking branch 'upstream/master' 5678c05d8 Use DEBUG_LEVEL=0 in make release and make clean ac25fe6b9 Merge pull request #779 from yuslepukhin/optimize_windows_build e154ee086 Do not build test only code and unit tests in Release builds Test code errors are currently blocking Windows Release builew We do not want spend time building in Release what we can not run We want to eliminate a source of most frequent errors when people check-in test only code which can not be built in Release. This feature will work only if you invoke msbuild against rocksdb.sln Invoking it against ALL_BUILD target will attempt to build everything. cd3286fae Error while cmake by building from zip-archive e3d4e1407 DBCompactionTestWithParam.ManualCompaction to verify block cache is not filled in manual compaction 033c6f1ad T7916298, bug fix 7717ad1af Adding artifacts to stress_crash CI job 0bf656b90 Don't spew warnings when flint doesn't exist 6d6776f6b Log more information for the add file with overlapping range failure 7951b9b07 make field order match initialization order 90228bb08 Merge pull request #771 from maximecaron/patch-1 2938c5c13 merge upstream changes e3b1d23d3 Bump version to 4.2 a7b2bedfb log_{reader,write}: recyclable record format 4e07c99a9 Fix iOS build 0c59691dd Handle multiple batches in single log record - allow app to return a new batch + allow app to return corrupted record status 32c291e3c Merge branch 'master' of github.com:facebook/rocksdb into T7916298 4575de5b9 #7916298: merge tools/db_crashtest2.py into tools/db_crashtest.py 5c727de6a Merge pull request #777 from yuslepukhin/fix_win_build_uint cfaa33f9a Update transaction iterator documentation 2f680ed09 Make index same type as auto deduced uint32_t 09f853550 uint is a not a datatype on windows. ec1f8354a Fix the default assignment of DEBUG_LEVEL in Makefile f18acd887 Fixed the clang compilation failure 4104e9bb6 log_reader: introduce kBadHeader; drop wal mode from ReadPhysicalRecord 9c33f64d1 log_reader: pass in WALRecoveryMode instead of bool report_eof_inconsistency 718805210 db_test_util: add recycle_log_files to set of tested options 3ac13c99d log_reader: pass log_number and optional info_log to ctor 5830c699f log_writer: pass log number and whether recycling is enabled to ctor 666376150 db_impl: recycle log files d666225a0 db_impl: disable recycle_log_files if WAL archive is enabled 543c12ab0 options: add recycle_log_file_num option 1bcafb62f env: add ReuseWritableFile e1a09a770 Implementation for GetPropertiesOfTablesInRange ad471453e Allow GetProperty to report the number of currently running flushes / compactions. 277dea78f Add more kill points a98fbacfa Moving memtable related files from util to a new directory memtable 8f143e03f Add ClearSnapshot() f9ba79ecd crash_test to trigger fail points other than file appending more frequently 2f2de338c Run ROCKSDB_LITE tests in travis 680156ca6 crash_test to run with data sync on e1a5ff857 Allow users to disable some kill points in db_stress d306a7ea8 Merge pull request #773 from yuslepukhin/update_requirements bb64d6da4 Disabling TSAN crash test 8c2fe68fd Update 4 is required for building with MS Visual Studio 13 952ad994a Fix db_test under ROCKSDB_LITE 6d730b4ae Block tests under ROCKSDB_LITE 5eee1ef2d Merge pull request #770 from Vaisman/master 63e507c59 Move ldb and sst_dump from utils to tools. dae49e829 Make DBTest.ReadLatencyHistogramByLevel more robust 92060b215 Fix build error using Visual Studio 12 9f7413502 Error while cmake by building from zip-archive b81b2ec25 Fix benchmarks under ROCKSDB_LITE e587dbe03 Move manual_compaction_test.cc from util to db 666fb5df4 Remove DefaultCompactionFilterFactory. d662b8dab Merge pull request #766 from PraveenSinghRao/lockfix f55d3009c Make db_test_util compile under ROCKSDB_LITE 29a47cd2b Include the time unit in the comment of perf_context timers 2b925ccb4 Correct the comments in db/internal_stats.h 35ad531be Seperate InternalIterator from Iterator 91c041e57 move debug variable under ifndef 198ed5898 Merge pull request #760 from jwlent55/use-static-library-header-files cc4d13e0a Put wal_filter under #ifndef ROCKSDB_LITE 385b41600 Merge pull request #764 from dmittendorf/fix-java-static-packaging 7062d0ea6 Make perf_context.db_mutex_lock_nanos and db_condition_wait_nanos only measures DB Mutex 1fe78a407 Fix tests failing in ROCKSDB_LITE a6efefef7 Fix format specifiers f7b2a7b40 Fix format specifiers 7f58ff7c3 Remove db_impl_debug from release build 1ddd91cd2 Fixed packaging of java crossbuild jar by forcing all compiled binaries to be output to the java/target directory. The uber crossbuild jar is then assembled within the java/target directory. eb2417855 merge from master c64ae05b1 Move TEST_NewInternalIterator to NewInternalIterator 59a0c219b Adding log filter to inspect and filter log records on recovery 0be50ed12 Merge pull request #763 from PraveenSinghRao/lockfix a1d37602a Fixing mutex to not use unique_lock 9e819d096 Modify the way java static builds are done so that: 1) There is no need to download and install the compression libraries twice just to get access to their header files during the compile phase. 2) Ensure that the compression library headers files used during the compile phase are the same ones used to build the static library that is linked into the library. f1fdf5205 Clean up dependency: Move db_test_util.* to db directory 237994409 Fixed an incorrect replace of const value in util/options_helper.cc 0bb8ea56b [RocksDB Options File] Add TableOptions section and support BlockBasedTable 4a7970d75 Modify the way java static builds are done so that: 1) There is no need to download and install the compression libraries twice just to get access to their header files during the compile phase. 2) Ensure that the compression library headers files used during the compile phase are the same ones used to build the static library that is linked into the library. c4366165e Merge pull request #759 from jwlent55/statically-load-compression-libraries fa4b5b3db Fix for the travis build caused by my previous commit 3d07b815f Passing table properties to compaction callback 64546af83 Adding parser to CI jobs def74f876 Deferred snapshot creation in transactions c5f3707d4 DisableIndexing() for Transactions 776bd8d5e Pass column family ID to table property collector 5a7222782 Ensure that the compression libraries are statically linked into dynamic libraries included in the Java jar. Also build the linux libraries using the portable flag to fix a problem with the linux32 build and improve the general portability of the RocksDB dynamic libraries. ==> linux32: util/crc32c.cc:318:39: error: ‘_mm_crc32_u64’ was not declared in this scope e61d9c148 Make DBTest.AggregatedTableProperties more deterministic b77eb16ab New Manifest format to allow customized fields in NewFile. 6732a5765 Merge pull request #756 from viveknelamangala/master a52888ed0 Install snappy headers to standard locations using yum, so that build_tools/build_detect_platform sets -DSNAPPY flag to g++ . Current jars of rocksdb do no have snappy compression avaliable . 831101b5f Make it harder for users to run debug builds in production 000836a88 CompactionFilter::Context to contain column family ID 3a0bf873b Change RocksDB version to 4.1 77318ee14 Enable crash CI jobs 9803e0d81 compaction_filter.h cleanup 51fa7ecec Bytes read/written from cache statistics f925208ff Create Makefile target unity_test a065cdb38 bloom hit/miss stats for SST and memtable 40cdf797d Fix compile error on platforms without fallocate() 77e4ad7ce Fix compile failure on Travis 4049bcde3 Added boolean variable to guard fallocate() calls aadf49fe6 Travis shouldn't fail when clang-format suggests improvements d80ce7f99 Compaction filter on merge operands 726d9ce1a Disabling unity 026750265 Support for LevelDB SST with .ldb suffix 5855cdb6d Merge pull request #750 from yuslepukhin/fixup_build_options 7bbe10c01 Merge pull request #751 from yuslepukhin/return_noerror e95b703b7 Mmap reads should not return error if reading past file 25c58a204 Add shared_linked DEBUG flag, remove port from among the include directories. 60b1c0a70 Fix to CI job definition 9babaeed1 Update dump_tool and undump_tool to accept Options eb5b637fb Fix condition for bottommost level 9eaff629e Make corruption_test more robust bf19dbff4 Fix valgrind - Initialize done variable 5c7bf56d3 [RocksDB Options] Support more options in RocksDBOptionParser for sanity check. 115427ef6 Add APIs PauseBackgroundWork() and ContinueBackgroundWork() a39897369 Adding features to CI job description a47bf325c Merge pull request #748 from yuslepukhin/improve_test_runs 65324a16d Improve test running scripts Introduce proper command line arguments so we can control the script Add appveyor support Add an ability to run all other (non db_tests) test executables in parallel Use .NET HashSet instead of empty valued hashtable to improve the looks TODO: Some of the tests do not use GTests and need to improve log parsing 7e4ee4231 Merge pull request #743 from edsrzf/amalgamation 7a23e4d8c New amalgamation target e9a6808c4 Merge pull request #745 from yuslepukhin/test_appveyor_baseline 9320ffd67 Improve CI build and fix Windows build breakage Is there a way to enforce CMake additions for internal changes that seem to come w/o a PR? 03b08ba9a Return MergeInProgress when fetching from transactions or WBWI with overwrite_key c29af48d3 Add max_file_opening_threads to db_bench da1cf8a9b Add a missing check for deprecated options in options_helper.cc 5a51fa907 Fix accidental object copy in transactions 1e73b11af Better handling of deprecated options in RocksDBOptionsParser a8b8295d1 Fixed a compile warning in options_test.cc under clang 74b100ac1 RocksDB Options file format and its serialization / deserialization. 75134f756 Merge pull request #741 from yuslepukhin/test_appveyor_baseline 2e7506d82 Improve CI build and build switches Add an optimized build config switch for faster test runs Change compiler options to introduce more opitmizations and be more inline with MS internal switches. Make appveyor build to utilize all the avaiable cores on the VM (parallel) Introduce new appveyor configuration for daily test runs as it would take too long to run db_test after each checkin even in paralell. With some exclusions we make it in 38 minutes. We currently fail to install ramdisk during the build. Add a powershell script to faicilitate paralell run for db_test cases. 1eff1834b Remove non-existing functions. Closes #680 16d1ba700 Clear SyncPoint Trace in DeleteSchedulerTests 30f74fa96 Make CompactionJobStatsTest.UniversalCompactionTest more robust afe0dc539 SingleDelete support for Transactions a263002a3 Fixed a tsan warning in db_stress.cc e4861e7d6 Fixed a compile error in util/arena.h 0fdb4f168 Fixed a compile warning in util/arena.cc when hugetlb is not supported. 94ac8826c Merge pull request #737 from mlin/readonly-syncwal 60fa9cf0b Override DBImplReadOnly::SyncWAL() to return NotSupported. Previously, calling it caused program abort. 7df348b40 Minor fix to CI job definition dac3f22b7 Fix the test failure 63e0f8679 Fixed a bug which causes rocksdb.flush.write.bytes stat is always zero 7ee445dd6 Fix the compile warning 174e2be5f Merge pull request #735 from jsteemann/fix-potential-leak-in-allocate aa58958d3 prevent potential memleaks in Arena::Allocate*() 25fd743d7 Fix SingleDelete support in WriteBatchWithIndex b6aa3f962 Fixed a memory leak issue in DBTest.UnremovableSingleDelete 7b7b5d9f1 [minor] Reuse SleepingBackgroundTask e01f32cc3 Parameterizing email id c58bac701 Fix valgrind failure due to memory leaks a70d08ec0 Fix the bug of using freed memory introduced by recent plain table reader patch 628216fc1 Simplifying valgrind testing 4805fa0ea Remove ldb HexToString method's usage of sscanf f03b5c987 Add experimental DB::AddFile() to plug sst files into empty DB 3fdb6e523 Fixed old lint errors in db/filename.cc b349d2278 Fixed old lint errors in db/filename.h df34aea33 PlainTableReader to support non-mmap mode d746eaad5 RandomAccessFileReader should not inherit RandomAccessFile 03dd8f3ca Fixing punit job description d0c31641d Internal stats WAL file synced to match meaning of the stats of the same name 48b4497f7 Merge pull request #730 from yuslepukhin/fix_write_batch_win_const_expr a6c22e3e4 Disabling parallel test CI job 489a3e95d Re-work to support size_t max constant for 32/64-bit. ff57c6511 [RocksJava] Fix test failure of InfoLogLevelTest f1b9f804e Add a mode to always pick the oldest file to compact for each level 5e8f0a66d Use port::constant for std::muneric_limtis<>::max() 2754ec999 Fix Windows constexpr issue and '#ifdef' column_family_test in Release. dd2e1eeb3 Disabling log running jobs 199744f4c Merge pull request #728 from jsteemann/fix-missing-include-header 0e65693f1 Merge pull request #727 from jsteemann/micro-optimization 4d6eb52d1 Fix to CI jobs. 3bcc072d2 Added more CI jobs 669b892f9 add missing header required for std::function 624ef456d fixed formatting. thanks @4tXJ7f for pointing me at `make format` bbb18c827 removed unused variable of type Status, fixed indentation 470483335 pass input string to WriteBatch() by const reference 5ec129971 key_ cannot become nullptr, so no check is needed for that 834b12a8d made Size() function const because it does not modify data 1fc16cb2e Fix clang-format on Travis e244bdf39 Merge pull request #725 from adamretter/fail-failed-java-test 7d937a090 Exit with non-zero status if one or more Java tests fail 1b598213a Check formatting in Travis c7fba8029 Fix non-deterministic failure in backupable_db_test 014fd55ad Support for SingleDelete() f35560d00 Merge pull request #723 from jsteemann/fix-typos f8b770a94 fixed typos 51e1c1125 Do not flag error if file to be deleted does not exist a5e312a7a Improving condition for bottommost level during compaction 9aca7cd6d DB::Open() to flush info log after printing DB pointer cecd903ad Fix gflags build in Travis script 16934d495 Fix wrong constants in db_test_util 2e8e7eb39 Fix the verbosity issue in Java makefile f21c7415a Change the log level of DB start-up log from Warn to Header. 9566342d2 Build gflags from source for Travis 3ebf11ed1 Adding the increment for a counter for a number of WAL syncs 81a61d75d Skipped tests shouldn't be failures [part 2] 1b7ea8ce8 Skipped tests shouldn't be failures 5ba3297d0 Add compaction time to log output 31a27a360 Callback for informing backup downloading added d93a9f2aa [travis CI] Run ulimit -n 2000 2b683d497 Add DBOption.max_subcompaction to option dump 0e50a3fcc Merge issue with D46773 a7e80379b LogAndApply() should fail if the column family has been dropped 2819a1db3 Minor fix to CI job definition df22e2fb7 Relax memory order for faster tickers 488607317 Adding Slice::difference_offset() function 5ce63e30e Merge pull request #720 from AMDmi3/fix-constant-overflow 925babc76 Merge pull request #721 from AMDmi3/printf-size_t f171faa24 Fix printf format for size_t 4b0b0201c Fix `integer overflow in expression' error f3170b6f6 DBImpl::FindObsoleteFiles() shouldn't release mutex between getting min_pending_output and scanning files e467bf0de Fix valgrind error 7cb314b9e Skip some tests in ROCKSD_LITE 0bfe0573e Add gflags dependency to Travis script 5de807ac1 Add options.hard_pending_compaction_bytes_limit to stop writes if compaction lagging behind 7143242d1 Fix compaction_job_stats under ROCKSDB_LITE 592f6bf78 Merge pull request #716 from yuslepukhin/refactor_file_reader_writer_win e2d6011f4 Minor fix to CI job definition ad0d70ca1 Relax asserts in arena_test 03ddce9a0 Add counters for L0 stall while L0-L1 compaction is taking place a3fc49bfd Transactions: Release Locks when rolling back to a savepoint ddc8b4499 Address code review comments both GH and internal Fix compilation issues on GCC/CLANG Address Windows Release test build issues due to Sync 9f3a66a93 Improvements to CI jobs 7db1471cc Minor fix to CI job c67d20689 Fixed arena_test failure due to malloc_usable_size() 34cedaff6 Initialize variable to avoid warning aeb461268 Add counters for seek/next/prev 45e9e4f0b Refactor NewTableReader to accept TableReaderOptions ddb950f83 Fixed bug in compaction iterator 30e82d5c4 Refactor to support file_reader_writer on Windows. Summary. A change https://reviews.facebook.net/differential/diff/224721/ Has attempted to move common functionality out of platform dependent code to a new facility called file_reader_writer. This includes: - perf counters - Buffering - RateLimiting af7cdbf64 Run full test suite in Travis c25f6a85b Removed __unused__ attribute 6db0a939d Fix DBCompactionTest failure with parallel L0-L1 compactions 8aa1f1519 Refactored common code of Builder/CompactionJob out into a CompactionIterator 41bce0586 CI job improvements 95ffc5d2b Correct ASSERT_OK() in ReadDroppedColumnFamily 3c37b3ccc Determine boundaries of subcompactions 112664408 Relaxing consistency detection to include errors while inserting to memtable as WAL recovery error. abc7f5fdb Make DBTest.ReadLatencyHistogramByLevel more robust ac9bcb55c Set max_open_files based on ulimit 4cbd2f9aa Merge pull request #714 from facebook/travisformac d0df54d1f Run travis tests on OS X a55e5a52a Merge pull request #711 from facebook/testtravis 2b676d5bb Upgrade travis to new architecture c66d53fee Fixed minor issue in CompressionTypeSupported() f3f2032c4 Release RocksDB 4.0.0 44b6e99e1 update max_write_buffer_number_to_maintain docblock aa6eed0c1 Transaction stats 25dbc579f Update HISTORY file for transactions 52386a1e7 Minor fix to sandcastle jobs commands b5b2b75e5 better tuning of arena block size 342ba8089 Make DBTest.OptimizeFiltersForHits more deterministic e17e92ea1 Relaxed assert in forward iterator 5e94f68f3 TransactionDB Custom Locking API 0ccf2db38 Fixed broken build due to format specifier 6bdc484fd Added Equal method to Comparator interface 7a31960ee Tests for ManifestDumpCommand and ListColumnFamiliesCommand 778cf4449 Adding email notification. 3a0df7f16 Fixed comparison in ForwardIterator when computing hint for GetNextLevelIndex() 91f3c9079 Fix case when forward iterator misses a new update ff1953c89 Merge pull request #707 from dkorolev/master 7b463e657 Fixed a typo in INSTALL.md d9f42aa60 Adding a verifyBackup method to BackupEngine 50dc5f0c5 Replace BackupRateLimiter with GenericRateLimiter 20ef64cae Moving jobs to use gcc-4.9 0f1aab6c1 Add SetLockTimeout for Transactions 14456aea5 Fix compile 76f286cc8 Optimize bloom filter cache misses 0e6e5472e Fixed a compile warning in rocksjni/loggerjnicallback.cc b8a962d4f Adding commands for few more CI jobs. 8a2d59a35 Add Cloudera's blog post to USERS.md 3c9cef1ee Unified maps with Comparator for sorting, other cleanup 3e0a672c5 Bug fix: table readers created by TableCache::Get() doesn't have latency histogram reported b42cd6bed Remove the need for LATEST_BACKUP in BackupEngine 0f763db20 Merge pull request #705 from yuslepukhin/rate_limiter_fix 20c44fefb t6913679: Use fallocate on LOG FILESS f14c3363e Make WinEnv::NowMicros return system time Previous change for the function https://github.com/facebook/rocksdb/commit/555ca3e7b7f06bd01dfd5e04dbb2cef5360f7917#diff-bdc04e0404c2db4fd3ac5118a63eaa4a made use of the QueryPerformanceCounter to return microseconds values that do not repeat as std::chrono::system_clock returned values that made auto_roll_logger_test fail. aad0572f8 Fixed the build issue of rocksdbjavastaticrelease 5508122ed Fix a perf regression in ForwardIterator b72200777 Fix listener_test when using ROCKSDB_MALLOC_USABLE_SIZE 40cd91b7f Fixed compile warning in rocksdbjava 90415cfeb Fixed a compile warning in linux32 environment. 9d6503f88 Fix arena_test test break using glibc-2.17 77a28615e Support static Status messages 18db1e469 better db_bench options for transactions 0be260523 Merge pull request #702 from PraveenSinghRao/remove_spurious 8b689546b Add Subcompactions to Universal Compaction Unit Tests c6d870ffb Merge branch 'arcpatch-D45741' 57b3a8773 Adding sandcastle determinator for RocksDB 3d78eb66b Arena usage to be calculated using malloc_usable_size() effd9dd1e Fix deadlock in WAL sync 64f07deb8 remove spurious compression definitions 72a9b73c9 Removed unnecessary checks in DBTest.ApproximateMemoryUsage cb164bfc4 Do not delete iterators for immutable memtables. 7a0dbdf3a Add ZSTD (not final format) compression type e2db15efd Merge pull request #701 from PraveenSinghRao/usewinapi_notcruntime e853191c1 Fix DBTest.ApproximateMemoryUsage 7c916a5d3 Merge pull request #699 from OpenChannelSSD/to_fb_master 0886f4f66 Helper functions to support direct IO 7e327980a Remove usage of C runtime API that has file handle limitation 8ef0144e2 Add argument --show_table_properties to db_bench 1fb2abae2 ColumnFamilyOptions serialization / deserialization. 5f4166c90 ReadaheadRandomAccessFile -- userspace readahead 16ebe3a2a Mmap reads should not return error if reading past file d286b5df9 DBIter to out extra keys with higher sequence numbers when changing direction from forward to backward 3795449c9 Fix DBTest.GetProperty a7834a129 Merge pull request #698 from yuslepukhin/address_noexcept_windows 9ccf1bd3e Correct the comment for GetProperty() API. fbe2c05f5 s/NOEXCEPT/ROCKSDB_NOEXCEPT 6924d7582 Address noexcept and const integer lambda capture VS 2013 does not support noexcept. Complains about usage of ineteger constant within lambda requiring explicit capture. 2f8d71ec0 Moving sequence number compaction variables from SubCompactionState to CompactionJob bab9934d9 Fix build failure caused by bad merge. 4d28a7d8a Add a whitebox test for deleted file iterators. 249fb4f88 Fix use of deleted file iterators with incomplete iterators 53b88784d Add throttling to multi-threaded backups 09d982f9e Fix compact_files_example 6996de87a Expose per-level aggregated table properties via GetProperty() 86d6c3cde Fix Windows build 20d1e547d Common base class for transactions 205083297 Fixing race condition in DBTest.DynamicMemtableOptions e46bcc08b Remove an extra 's' from cur-size-all-mem-tabless 4ab26c5ad Smarter purging during flush 4c81ac0c5 Fix benchmark report script b6def58f7 Changed 'num_subcompactions' to the more accurate 'max_subcompactions' c85296846 db_iter_test: add more test cases for the data race bug 9130873a1 Add options.new_table_reader_for_compaction_inputs 07d2d3416 Add a counter about estimated pending compaction bytes 41a0e2811 Improve defaults for benchmarks a203b913c Fixed a rare deadlock in DBTest.ThreadStatusFlush 962aa6429 Merge pull request #695 from yuslepukhin/address_windows_build 5bf890762 More indent adjustment. e2a9f43d6 Adjust indent 6e9a260b0 Merge branch 'address_windows_build' of https://github.com/yuslepukhin/rocksdb into address_windows_build 1cac89c9b Address windows build issues Intro SubCompactionState move functionality =delete copy functionality #ifdef SyncPoint in tests for Windows Release builds f25f06ddd Address windows build issues Intro SubCompactionState move functionality =delete copy functionality #ifdef SyncPoint in tests for Windows Release builds 027ca5b2c Total SST files size DB Property b604d2562 Removing unused variables to fix build 1b114eed4 Free file iterators for files which are above the iterate upper bound to Improve memory utilization 3fd70b05b Rate limit deletes issued by DestroyDB df79eafcb Introduce GetIntProperty("rocksdb.size-all-mem-tables") 888fbdc88 Remove the contstaint that iterator upper bound needs to be within a prefix 137c37667 Removing variables used only in assertions to prevent build error b47cc5851 Bounding Number of Subcompactions e58e1b18e Make tailing iterator show new entries in memtable. 9ec957159 DBOptions serialization and deserialization b2df20a89 Make HashCuckooRep::ApproximateMemoryUsage() return reasonable estimation. 601b1aaca Fixing Failed Assertion in Subcompaction State Diff f0da6977a [Parallel L0-L1 Compaction Prep]: Giving Subcompactions Their Own State f32a57209 Simplify querying of merge results 72613657f Measure file read latency histogram per level b7198c3af reduce db mutex contention for write batch groups 603b6da8b Add options.compaction_measure_io_stats to print write I/O stats in compactions dc9d5634f Change master to 3.14 b78c8e07d Merge pull request #689 from msb-at-yahoo/add-tools-target 9f0dd2229 Add a 'tools' target. 463720712 Add test case to repro the mispositional iterator in a low-chance data race case 3bd9db420 [Cleanup] Remove RandomRWFile c3466eab0 Have Transactions use WriteBatch::RollbackToSavePoint 0db807ec2 Transaction error statuses c2f2cb021 Pessimistic Transactions c2868cbc5 Use manual_compaction for compaction_job_test 6b2d57039 Fix Windows build by adding snapshot_impl to CMakeLists e61fafbe7 Fixed clang-build error in util/thread_local.cc cee1e8a08 Parallelize LoadTableHandlers 4249f159d Removing duplicate code in db_bench/db_stress, fixing typos a03085b55 Fix linters on non-fb machines 1ae27113c reduce comparisons by skiplist b47d65b31 Fixed Segmentation Fault in db_stress on OSX. a1581eca8 Modernize RocksDB linters a9dcc0a63 Fix clang build 2cf0f4f47 Adding wal_recovery_mode log message 68f934355 Better CompactionJob testing 22dcaaff3 More accurate time measurement for delete_scheduler_test 0a7ea582c Add auto-build manifest for appveyor 0093271ee Merge pull request #685 from flandr/fix-tls-build ac04a6cfb Fix OSX + Windows build 16ea1c7d1 simple ManagedSnapshot wrapper 257ee895f Fixed memory leaks 254c4fb88 In HISTORY.md Switch unreleased notes to 3.13 40f893f4a Fix delete_scheduler_test valgrind error 6a4aaadcd Avoid type unique_ptr in LogWriterNumber::writer for Windows build break d7314ba75 Fixing endless loop if seeking to end of key with seq num 0 48e6e9aa8 Add util/delete_scheduler_impl.cc to CMakeLists.txt c7742452e Add Statistics.getHistogramString() to print more detailed outputs of a histogram 29b028b0e Make DeleteScheduler tests more reliable fca88f8e1 valgrind_check to exit on test failures 7d364d0d9 Fix build failure 960d936e8 Add function 'GetInfoLogList()' 7ccd1c80a Add two unit tests for SyncWAL() 3ae386eaf Add statistic histogram "rocksdb.sst.read.micros" 8ecb51a7e "make commit-prereq" should clean up rocksjava properly 9aec75fbb Enable DBTest.FlushSchedule under TSAN bd2fc5f5f Fix TSAN for delete_scheduler_test 8e01bd114 Fix misplaced position for reversing iterator direction while current key is a merge c46507102 Removing duplicate code e06cf1a09 [wal changes 3/3] method in DB to sync WAL without blocking writers 5dc3e6881 Update Tests To Enable Subcompactions c45a57b41 Support delete rate limiting 102ac118b Update JAVA-HISTORY.md for v3.13 3a1d4e6c9 Merge pull request #670 from skunkwerks/fix_osx_shared_library_names f5d072ae6 Fixed RocksJava test failure of shouldSetTestCappedPrefixExtractor f39cbcb0a Merge pull request #654 from adamretter/remove-emptyvalue-compactionfilter f0b5bcc7b add support for capped prefix extractor in java 18ba58a94 Upgrading jemalloc from 3.6.0 to the latest for fbcode+gcc 4.8.1 ce21afd20 Expose the BackupEngine from the Java API b0d12a135 Merge pull request #569 from adamretter/travis-java-api 241bb2aef Make DBCompactionTest.SkipStatsUpdateTest more stable. 3424eeb1e Polish HISTORY.md cf3e05304 crash_test cleans up directory before testing if TEST_TMPDIR is set 24daff6d7 Fix a typo and update HISTORY.md for NewCompactOnDeletionCollectorFactory(). 14d0bfa42 Add DBOptions::skip_sats_update_on_db_open e2a3bfe74 First half of whitebox_crash_test to keep crashing the same DB 2e73bd4ff crash_test to put DB under TEST_TMPDIR 1205bdbce crash_test to cover simply cases d5c0a6da6 Merge branch 'master' of github.com:facebook/rocksdb 2d41403f4 Made change to fix the memory leak 92f7039ee fix memory corruption issue in sst_dump --show_compression_sizes be8621ffa Fix compile warning in compact_on_deletion_collector in some environment 26894303c Add CompactOnDeletionCollector in utilities/table_properties_collectors. 20b244fcc Fix CompactFiles by adding all necessary files 87df6295d Make SuggestCompactRangeNoTwoLevel0Compactions deterministic 40c64434d Parallelize L0-L1 Compaction: Restructure Compaction Job 47316c2d0 dump_manifest supports DB with more number of levels bd852bf11 Fixed typos in db_stress 544be638a Fixing fprintf of non string literal 193dc977e Fixing dead code in table_properties_collector_test 05d4265a2 Merge branch 'master' of github.com:facebook/rocksdb 4be6d4416 Compression sizes option for sst_dump_tool 8161bdb5a WriteBatch Save Points 7bfae3a72 tools/db_crashtest2.py should run on the same DB d06c82e47 Further cleanup of CompactionJob and MergeHelper e95c59cd2 Count number of corrupt keys during compaction 221a94a5f Another attempt at adding the Java API and tests to the travis build 1bdfcef7b Fix when output level is 0 of universal compaction with trivial move 6a82fba75 Add missing hashCode() implementation f73c80143 Fixing Java tests. 14f413760 Correct the comment of DB::GetApproximateSizes 8279d4197 Merge pull request #667 from yuslepukhin/fix_now_microsec_win eb8e3b4c7 Fix shared library names on OSX 555ca3e7b Fix WinEnv::NowMicrosec * std::chrono does not provide enough granularity for microsecs and periodically emits duplicates * the bug is manifested in log rotation logic where we get duplicate log file names and loose previous log content * msvc does not imlement COW on std::strings adjusted the test to use refs in the loops as auto does not retain ref info * adjust auto_log rotation test with Windows specific command to remove a folder. The test previously worked because we have unix utils installed in house but this may not be the case for everyone. 82f148ef9 Fix test DBCompactionTest.PartialCompactionFailure undeterministic failure 6002801e0 Abandon ROCKSDB_WARNING 4922af6f8 fixed DBTest.GetPropertiesOfAllTablesTest and DBTest.GetUserDefinedTablaProperties flakiness 3bf9f9a83 cleaned up PosixMmapFile a little fe09a6dae [wal changes 2/3] write with sync=true syncs previous unsynced wals to prevent illegal data loss 06aebca59 Report live data size estimate 85ac65536 Tests to avoid to use TMPDIR directly f09d45690 Merge pull request #664 from yuslepukhin/add_tests_fix_sanity 31b35c902 Add missing tests, fix db_sanity Add heap_test, merge_helper_test Fix uninitialized pointers in db_sanity_test that cause SIGSEV when DB::Open fails in case compression is not linked. 66a3cbc54 Merge pull request #663 from yuslepukhin/fix_windows_build_refactor ac5e441ad Fix windows build after refactoring Missing and duplicate files in CMake Missing definition of port::Crash 02b635fa3 Fix undeterministic failure of DBTest.GetPropertiesOfAllTablesTest 3dbf4ba22 RangeSync not to sync last 1MB of the file 7219088cd Move general compaction tests from db_test.cc to db_compaction_test.cc 0adecd9f4 Add db_inplace_update_test back to Makefile 064294081 Improved FileExists API 9f1de9518 Revert Makefile 6867fb19c Revert "Add missing db_log_iter_test in the test list" 443c6646b Move remaining universal compaction tests from db_test.cc to db_universal_compaction_test.cc 7462286d3 Move in-place-update related tests from db_test.cc to db_inplace_update_test.cc 03467bdd4 Add missing db_log_iter_test in the test list 331954ab8 Fixed DBTestUniversalManualCompactionOutputPathId test a75f23eb8 Relax assertions in unit DropWrites to be more permissible ee80432ff db_bench add an option of --universal_allow_trivial_move 58b4209e0 Trigger non-trivial compaction in fault_injection_test 59eca2cc9 Make memenv_test runnable in ROCKSDB_LITE aa8ac6445 Skip unsupported tests in ROCKSDB_LITE ce9712d34 Make mock_env_test runnable in ROCKSDB_LITE c06d1d839 Make merge_test runnable in ROCKSDB_LITE 144d2910d Block backupable_db_test_lite in ROCKSDB_LITE 0d1d9aeeb Block plain_table_db_test in ROCKSDB_LITE 4853e228e Make table_test runnable in ROCKSDB_LITE f0fe9126f Fix compile for write_callback_test in ROCKSDB_LITE cf6a7bebc Block cuckoo table tests in ROCKSDB_LITE 20922c4a5 Make compaction_picker_test runnable in ROCKSDB_LITE 6e9fbeb27 Move rate_limiter, write buffering, most perf context instrumentation and most random kill out of Env 5ec829bc4 Cleaning up CYGWIN define of fread_unlocked to port 26ca89319 Block document_db_test in ROCKSDB_LITE 35ca59364 Don't let flushes preempt compactions 79373c372 Fix ROCKSDB_WARNING 74c755c55 Added JSON manifest dump option to ldb command a96fcd09b Deprecate CompactionFilterV2 1d20fa9d0 Fixed and simplified merge_helper aede5cd8e Merge pull request #656 from qinzuoyan/fb-master d730c3677 Merge pull request #657 from yuslepukhin/ensure_clean_public_headers ac2b9367f Fix a typo in variable 415c47323 Merge after rebasing d1a457181 Ensure Windows build w/o port/port.h in public headers 6c0c8dee7 Fix data loss after DB recovery by not allowing flush/compaction to be scheduled until DB opened e4af3bfb2 Test for compaction of corrupted keys 91bf1b80e Java facility to use the RemoveEmptyValueCompactionFilter 3d00271e4 The ability to specify a compaction filter via the Java API 62dec0e2b RemoveEmptyValueCompactionFilter - A compaction filter which removes entries which have an empty value c5bca5319 Fix compile on Mac 487bba434 extend temp str buffer size 247690fe3 Ensure Windows build w/o port/port.h in public headers 84c3577af fix append bug in DumpDBFileSummary() 43e982562 Fix mongo build -take 2 d8263d958 Unbreak mongo build 12c5528a8 Bump to RocksDB 3.13 81d072623 move convenience.h out of utilities beb19ad0d Fixing delete files in Trivial move of universal compaction c61396069 Build fix. 2c8de0eca Update --help message in db_bench. 6b2d44b2f Refactoring of writing key/value pairs e1c99e10c Replace std::priority_queue in MergingIterator with custom heap, take 2 9a6a0bd8c Style fix in compaction_job.cc e94c510c3 Make ldb_test not depend on compression ddad40e93 Fixed nullptr deref and added assert 1bc8eb877 make coverage should execute sequentially 801df912a Move UniversalCompaction related db-tests to db_universal_compaction_test.cc 3ca6b2541 Move TailingIterator tests from db_test.cc to db_test_tailing_iterator.cc ce829c77e Make TransactionLogIterator related tests from db_test.cc to db_log_iter_test.cc c3f98bb89 Move CompactionFilter tests in db_test.cc to db_compaction_filter_test.cc 0936362a7 Block SyncPoint in util/db_test_util.h in released Windows mode. 05e194158 Merge pull request #639 from cleaton/setMaxTableFileSize fc2b71d9c Merge pull request #655 from adamretter/java-make-resolve-maven 8a9fca261 Better error handling in BackupEngine 18d5e1bf8 Remove db_impl_readonly dependency on utilities 9d22a9737 Resolve Java test dependencies from local maven repo if present 49640bd82 Allow write_batch_test to run with ROCKSDB_LITE a9c510951 Deprecate purge_redundant_kvs_while_flush 5aea98ddd Deprecate WriteOptions::timeout_hint_us ae29495e4 Avoid manipulating const char* arrays ab137af4b Partial cleanup of CompactionJob 1879d9370 Add ldb_test.py back to `make check` b10cf4e2e Move DynamicLevel related db-tests to db_dynamic_level_test.cc e290f5d3c Block reduce_levels_test in ROCKSDB_LITE 04d201fa0 Block spatial_db_test in ROCKSDB_LITE 49f42ad03 Move global static functions in db_test_util to DBTestBase 625467a08 Move reusable part of db_test.cc to util/db_test_util.h e8e8c9049 fix compile for optimistic_transaction_test under ROCKSDB_LITE 8bca83e5d Add tombstone information in CompactionJobStats f9728640f "make format" against last 10 commits 76d3cd328 Fix public API dependency on internal codes and dependency on MAX_INT32 5fd11853c Print Fast CRC32 support information in DB LOG a6e38fd17 Fix a uncleaned counter in PerfContext::Reset() e41cbd9c2 Merge pull request #646 from yuslepukhin/ms_win_port 4cab5ebec Merge branch 'ms_win_port' of https://github.com/yuslepukhin/rocksdb into ms_win_port 296de4ae6 Address review comments Rule of five: add destructor Add a note to COMMIT.md for 3rd party json. 041b6f95a perf_context: report time spent on reading index and bloom blocks d08ba9f0c Merge branch 'ms_win_port' of http://vstfbing:8080/tfs/Bing/_git/repo.RocksDB into ms_win_port 805fe84ba Conditional use of third-party libraries Committed by Alexander Zinoviev 7/9/2015 2:42:41 PM 5555cc500 Improve build system c903ccc4c Merge from github/master 54d124a38 Conditional use of third-party libraries 7189e90c2 Fix a noisy unit test. 1f4d56570 Add db_bench flag to set cache_index_and_filter_blocks 5c7913233 Revert the changes related to Options, as requested to seperate them into a different patch. d8586ab22 All of these are in the new code added past 3.10 1) Crash in env_win.cc that prevented db_test run to completion and some new tests 2) Fix new corruption tests in DBTest by allowing a shared trunction of files. Note that this is generally needed ONLY for tests. 3) Close database so WAL is closed prior to inducing corruption similar to what we did within Corruption tests. 4bed00a44 Fix function name format according to google style e2e3d84b2 Added multi WAL log testing to recovery tests. 5219226d3 Merge branch 'ms_win_port' of http://vstfbing:8080/tfs/Bing/_git/repo.RocksDB into ms_win_port ef4b87f1b Commit both PR and internal code review changes 95f4c2bcb Conditional use of 3rd-party libraries 4f56632b1 Fix occasional failure in compaction_job_test 411c8e3d1 Build fail fix b7a2369fb Revert "Replace std::priority_queue in MergingIterator with custom heap" c0b23dd5b Enabling trivial move in universal compaction d8e3e766f Fixed a bug in test ThreadStatusSingleCompaction 57d216ea6 Remove assert(current_ == CurrentReverse()) in MergingIterator::Prev() 59b50dcef Update HISTORY.md for Listener 4ce5be425 fixed leaking log::Writers 685582a0b Revert two diffs related to DBIter::FindPrevUserKey() e12b40399 Initialize threads later in constructor 58d7ab3c6 Added tests for ExpandWhileOverlapping() 155ce60da Fix compaction_job_test b6655a679 Replace std::priority_queue in MergingIterator with custom heap e25ee32e3 Arena needs mman header for mmap d2f0912bd Merge the latest changes from github/master 35cd75c37 Introduce InfoLogLevel::HEADER_LEVEL acee2b08a Fixed endless loop in DBIter::FindPrevUserKey() 218487d8d [wal changes 1/3] fixed unbounded wal growth in some workloads feb99c31a Merge remote-tracking branch 'origin' into ms_win_port e70115e71 Fix unity build by removing anonymous namespace 4159f5b87 Prepare 3.12 a69bc91e3 Multithreaded backup and restore in BackupEngineImpl 9dbde7277 Merge remote-tracking branch 'origin' into ms_win_port 03d433ee6 [RocksJava] Fixed test failures 326da912d Add string.h to Histogram as we init the array out of curly braces ca2fe2c1b Address GCC compilation issues 19e13a595 Fix header inclusion 18285c1e2 Windows Port from Microsoft c00948d5e [RocksJava] Fix test failure of compactRangeToLevel 05e283196 Allocate LevelFileIteratorState and LevelFileNumIterator from DB iterator's arena 436ed904d Add rpath option to production builds for 4.8.1 toolchain b0f1927db Increasing timeout for drop writes. ec70fea4c Fix a comparison in DBIter::FindPrevUserKey() 501591c42 Make column_family_test runnable in ROCKSDB_LITE 91cb82f34 Merge branch 'master' of github.com:facebook/rocksdb 09f5a4b48 set -e in fb_compile_mongo.sh 6199cba99 Fix race in unit test. 0a019d74a Use malloc_usable_size() for accounting block cache size 4cbc4e6f8 Call merge operators with empty values 619167ee6 Fix mac compile 472e64d39 Improve fb_compile_mongo.sh c9cd404bc Make flush check for shutdown 4fb09c687 Updating SeekToLast with upper bound dadc42976 Reproducible MongoRocks compile with FB toolchain 62a8fd154 Make stringappend_test runnable in ROCKSDB_LITE 48da7a9ca Improve the comment for BYTES_READ in statistics. 72cab8895 Block redis_test in ROCKSDB_LITE dec2c9f56 Make table_properties_collector_test runnable in ROCKSDB_LITE 0b1ffe2e1 Remove -Wl,--no-as-needed flag when making shared_lib in OSX and IOS 674b1181c Bottommost level compaction option 782a1590f Implement a table-level row cache de85e4cad Introduce WAL recovery consistency levels 530534fce Fix trivial move merge 7015fd81c Add read_nanos to IOStatsContext. 7160f5d80 Fix broken gflags link dda74111a add setMaxTableFilesSize Options unit test d62b6ed83 add setMaxTableFilesSize to JNI interface e1d3c7dbe Fixing valgrind error in checkpoint_test 3bdec09cb Remove ldb_tests.py from make check until it is working again. 15325bf55 First version of rocksdb_dump and rocksdb_undump. 04251e1e3 Add wal files to Checkpoint for multiple column families. 18cc5018b Fix memory leaks in PinnedUsageTest bf03f59c1 Disable CompressLevelCompaction() if Zlib is not supported df719d496 Make autovector_test runnable in ROCKSDB_LITE 4d6d47688 Block geodb_test in ROCKSDB_LITE 71b438c4a Remove unused target --- compactor_test eade498bd Block utilities/write_batch_with_index in ROCKSDB_LITE 760e9a94d Fail DB::Open() when the requested compression is not available 69bb210d5 Add Cache.GetPinnedUsageUsage() 4eabbdb7e Skip bottommost level compaction if possible 4b8bb62f0 Don't dump DBOptions for each column family 176f0bedc Merge branch 'master' of github.com:facebook/rocksdb bb1c74ce1 Fixed a bug of CompactionStats in multi-level universal compaction case a66b8157d Merge branch 'master' of github.com:facebook/rocksdb f06be62fd Replace %llu with format macros in ParsedInternalKey::DebugString()) 2dc3910b5 Add --benchmark_write_rate_limit option to db_bench 12e030a99 Use CompactRangeOptions for CompactRange c89369f57 Move dockerbuild.sh to build_tools/ 4716ab4d1 Merge pull request #638 from HolodovAlexander/master 25d600569 Clean up InstallSuperVersion 1369f015e Only initialize the ThreadStatusData when necessary. 1a08d0beb Block c_test in ROCKSDB_LITE 40f562e74 Allow GetApproximateSize() to include mem table size if it is skip list memtable d59d90bb1 db_bench periodically writes QPS to CSV file 46296cc86 Cygwin build not to use -fPIC bee8d033f Removed two unused macros in iostats_context 5fec96387 Fixed false alarm of size comparison in compaction_job_stats_test cccd2199a Revert skip bottommost compaction 20f2b5425 Skip bottom most level compaction if no compaction filter 7842920be Slow down writes by bytes written a84df655f Don't let two L0->L1 compactions run in parallel d6ce0f7c6 Add largest sequence to FlushJobInfo ab455ce49 fix clang build 3eddd1abe Add Env::GetThreadID(), which returns the ID of the current thread. 73faa3d41 Handling edge cases for ReFitLevel bffaf0a8b Merge pull request #631 from mkhq/patch-1 821cff114 Re-generate WriteEntry on WBWIIterator::Entry() 8b7be1808 Updated OS X instructions, replace homebrew/dupes with homebrew/versions d03f11090 Link all libraries when building shared libraries 75222d130 Revert "Fix compile" 47f1e7212 Merge pull request #630 from rdallman/c-wb-logdata 51440f83e Fix compile 4949ef08d Re-generate WriteEntry on WBWIIterator::Entry() 735df6655 C: add WriteBatch.PutLogData support e409d3d74 Make "make all" work for CYGWIN 62c3a9579 Add test for iteration+mutation of WBWI d9b3338eb Add Yahoo's blog post about Sherpa to USERS.md 75d7075a8 Print info message about files need compaction for debuging purpose 406a5682e Fix hang when closing a DB after doing loads with WAL disabled. d8c8f08c1 GetSnapshot() and ReleaseSnapshot() to move new and free out of DB mutex 643bbbf08 Use nullptr for default compaction_filter_factory 21f2492ac Fix CYGWin release build f02ce0c65 Fix ASAN errors in c_test 133130a4f Merge pull request #625 from rdallman/c-slice-parts-support de4d172d0 Merge pull request #622 from rdallman/c-multiget 6df589b44 Add TablePropertiesCollector::NeedCompact() to suggest DB to further compact output files 2e764f06e [API Change] Improve EventListener::OnFlushCompleted interface 7322c7401 Revert incorrect commit 31e60e2a7 Unlock mutex in ReFitLevel 7647df8f9 Fixed the tsan failure in util/compaction_job_stats_impl.cc b2785472c Fix compile 3ce3bb3da Allowing L0 -> L1 trivial move on sorted data bb808eadd Changed the CompactionJobStats::output_key_prefix type from char[] to string. 0b3172d07 Add EventListener::OnTableFileDeletion() 211a195d4 C: add MultiGet support 5dc174e11 C: add support for WriteBatch SliceParts params 2d0b9e5f0 Fix compile on darwin 3af668ed1 Fix DBTest.MigrateToDynamicLevelMaxBytesBase slowness with valgrind 408cc4b8e Revert "Merge pull request #621 from rdallman/c-slice-parts-support" 78382d4ba Merge pull request #621 from rdallman/c-slice-parts-support ca8b85ac0 better document max_write_buffer_number_to_maintain 0483dab2a Remove a TODO that has been done 8afafc278 Fix compile warning in db/db_impl fe5c6321c Allow EventListener::OnCompactionCompleted to return CompactionJobStats. 3083ed212 Fixed heap-use-after-free error in compaction_job_test.cc 8d8d4e45b Fixed ROCKSDB_LITE compile error due to the missing of TableFileCreationInfo ab946af08 Fix a compile warning in listener_test.cc fc8382127 Add EventListener::OnTableFileCreated() 898e803fc Add a stats counter for DB_WRITE back which was mistakenly removed. ac81130fa Fix Bug: CompactRange() doesn't change to correct level caused by using wrong level ec7a94436 more times in perf_context and iostats_context 4266d4fd9 Allow users to migrate to options.level_compaction_dynamic_level_bytes=true using CompactRange() d333820ba Removed DBImpl::notifying_events_ 495ce6018 Fixed compile warning in compact_files_example.cc 2ecac9f96 add rocksdb::WritableFileWrapper similar to rocksdb::EnvWrapper a187e66ad Merge pull request #617 from rdallman/wb-merge-sliceparts 16c197627 Fixed db_stress 4c181f08b Fix compile on darwin bc7a7a400 fix LITE build 832271f6b Fixed a compile warning in db_stress in NDEBUG mode. dc9d70de6 Optimistic Transactions d5a0c0e69 Fixed a compile warning in db_stress ebfdb3c7f Fixed a compile error in ROCKSDB_LITE 9ffc8ba02 Include EventListener in stress test. a3da59022 Decrease number of jobs in make release 21cd6b7ad C: add support for WriteBatch SliceParts params a0635ba3f WriteBatch.Merge w/ SliceParts support c81535103 Support saving history in memtable_list ec4ff4e99 Rename EventLoggerHelpers EventHelpers 672dda9b3 [API Change] Move listeners from ColumnFamilyOptions to DBOptions 3ab8ffd4d Compaction now conditionally boosts the size of deletion entries. a81ac2412 Merge pull request #615 from rdallman/master 6d299b70b Fixed a bug in EventLoggerHelpers::LogTableFileCreation a0580205c Removed an unused private variable in db_impl.h 328ad902a update an import path to fit in with the rest of the kids 9c38ce1d0 C: extra bbto / noop slice transform 8d26799fe Merge pull request #614 from arschles/docker 32198343f fix typo in c_simple_example 6116ccc23 moving dockerfile to root d90cee9fd adding docker build script and dockerfile ea6d3a8ac Don't skip last level when calculating compaction stats 5c224d1b7 Fixed two bugs on logging file deletion. dc81efe41 Change the log-level of DB summary and options from INFO_LEVEL to WARN_LEVEL 687214f87 Ensure ColumnFamilyOptions.num_levels >= 2 when level compaction is used. 2abb59268 Avoid logging under mutex in DBImpl::WriteLevel0TableForRecovery(). 309a9d076 Run tests sequentally if J=1 7fee8775a Allow EventLogger to directly log from a JSONWriter. 7a3577519 Don't artificially inflate L0 score 4cb4d546c Set stats_dump_period_sec to 600 by default e2c1d4b57 [Public API Change] Make DB::GetDbIdentity() be const function. eaf61ba9f Minor text correction f16c0b289 Merge pull request #613 from DerekSchenk/DerekSchenk-patch-issue-606 d1a978ae3 Rename JSONWritter to JSONWriter 3e0817541 Add LDFLAGS to Java static library 812c461c9 Dump db stats in WARN level b588505a7 Update HISTORY.md for GetThreadList() update. 944043d68 Add --wal_bytes_per_sync for db_bench and more IO stats d5de04d20 Update history for 3.11 08b6b3796 FORCE_GIT_SHA 04feaeebb Fix comparison between signed and usigned integers 4a855c079 Add an option wal_bytes_per_sync to control sync_file_range for WAL files b0fdda4ff Allow flushes to run in parallel with manual compaction 74f3832d8 Fixed compile errors due to some gcc does not have std::map::emplace 0c8017dba Remove duplicated code fb5bdbf98 DBTest.DynamicLevelMaxBytesCompactRange: make sure L0 is not empty before running compact range 6fa708512 CompactRange skips levels 1 to base_level -1 for dynamic level base size 84a9c6a53 add comment eeb44366b C api: human-readable statistics 3f0867c0f Allow GetThreadList to report Flush properties. a66f643e9 Use a better way to initialize ThreadStatus::kNumOperationProperties. 7413306d9 Take a chance on a random file when choosing compaction 8c52788f0 Use version defined in Makefile in rocksdb_build_git_sha 5aad88129 DBTest.DynamicLevelMaxBytesBase2: remove an unnecesary check ec43a8b9f Universal Compaction with multiple levels won't allocate up to output size 714fcc067 Make ThreadStatus::InterpretOperationProperties take const uint64_t* bc68bd5a1 db_bench to support rate limiter df1f87a88 Fixed compile error in db/column_family.cc 14431e971 Fixed a bug in EventListener::OnCompactionCompleted(). dbd95b753 Add more table properties to EventLogger b5881762b Reset parent_index and base_index when picking files marked for compaction 711465cce API to fetch from both a WriteBatchWithIndex and the db 3996fff8a Fix clang build - add override d97813906 SuggestCompactRange() is manual compaction 2fe24e4b5 Don't treat warnings as error when building rocksdbjavastatic beda81c53 rm -rf in make clean 50eab9cf3 Fix BackupEngine 962f8ba33 Bump to 3.11 37bbd3185 Merge pull request #601 from adamretter/jdb-bench 77a5a543a Allow GetThreadList() to report basic compaction operation properties. 65fe1cfbb Cleanup CompactionJob df4130ad8 fix crashes in stats and compaction filter for db_ttl_impl 7ea769487 Fix flakiness in column_family_test a2c4cc756 Don't treat warnings as error when building release 9aa011fa3 Optimize GetRange Function 36a740889 Fix UNLIKELY parenthesis 9bdbaab94 Update USERS.md 2ab7065af build: avoid unused-variable warning 88044340c Add Size-GB column to benchmark reports d2346c2cf Fix hang with large write batches and column families. b6b100fe0 Remove iter_refresh_interval_us 197f01b7b Bugfix remove deprecated option use which was removed in previous commit 019ecd19329ee895284e9b040df0ffe4c08b35d8 aa094e8ea Fix conversion from nano-seconds to milli-seconds and seconds dddceefe5 Fix clang build d4540654e Optimize GetApproximateSizes() to use lesser CPU cycles. fd96b5540 Making GetOptions() comment better (#597) 7246ad34d Don't compact bottommost level in SuggestCompactRange 7f47ba0e2 Fix possible SIGSEGV in CompactRange (github issue #596) aba005c44 Merge pull request #585 from fyrz/RocksJava-RemoveDeprecatedStuff d6f39c5ae Helper function to time Merges a087f80e9 Add scripts to run leveldb benchmark 1bb4928da Include bunch of more events into EventLogger 3db81d535 Fix memory leak in cache_test introduced in the previous commit 4961a9622 Fix build 93ab1473d Merge pull request #593 from charsyam/feature/type-1 6ede020dc fix typos 3d1af4ae6 Don't preinstall jemalloc in Travis 242f9b4c2 Fix CLANG build issue introduced in previous commit 794ccfde8 Task 6532943: Rocksdb - SetCapacity() can dynamically change cache capacity if feasible 98a44559d Build for CYGWIN d01bbb53a Fix CompactRange for universal compaction with num_levels > 1 e003d3864 Abstract out SetMaxPossibleForUserKey() and SetMinPossibleForUserKey fd7a35731 Enable open source users to use jemalloc (github issue #438) aa14670b2 Add an assertion in CompactionPicker 2dc421df4 Implement DB::PromoteL0 method 9bf40b64d Print max score in level summary 397b6588b options.paranoid_file_checks to read all rows after writing to a file. 283a04296 Set --seed per test 618d07b06 Making PreShutdown tests more reliable. 0a91bca5d test: avoid vuln-inducing use of temporary directory 6e359419f Add rpath for production builds 78dbd087d Improve benchmark scripts 6a5ffee0c Fix gflags Makefile 108a927f0 Merge pull request #589 from coderplay/patch-1 a58fd7427 Update USERS.md d85d08c7b One last fix to Makefile 2db96dca1 Fix make install when there is no shared lib 7d136994c Get rid of error output 79c1b021a Fix Makefile 742fa9e31 Fix compile with two gflags 79c21ec0c skip ioctl-using tests when not supported 6059bdf86 Add experimental API MarkForCompaction() acf8a4141 maint: use ASSERT_TRUE, not ASSERT_EQ(true; same for false b5400f90f Kill dead code 48b0a045d Speed up reduce_levels_test 00c2afcd3 Fix bug in ExpandWhileOverlapping() 019ecd193 [RocksJava] Remove deprecated methods 98ef21d2f Merge pull request #584 from pshareghi/rocksdb-3.10-falloch 5b7131c75 [RocksJava] Removed deprecated skipLogErrorOnRecovery methods. 566f65271 [RocksJava] Removed deprecated ColumnFamilyDescriptor methods 6997aa0b6 Merge pull request #582 from fyrz/RocksJava-Fix-RateLimiter 582c4b0f7 [RocksJava] Fix RateLimiter Tests in 3.10 6cfb2150d Merge pull request #581 from vladb38/patch-3 d71e8f7ad Update USERS.md debaf85ef Bug of trivial move of dynamic level 12d7d3d28 Fix and Improve DBTest.DynamicLevelCompressionPerLevel2 a1271c6c6 Fix build break introduced by new SyncPoint interface change fcb206b66 SyncPoint to allow a callback with an argument and use it to get DBTest.DynamicLevelCompressionPerLevel2 more straight-forward 281db8bb6 Temporarily disable test CompactFilesOnLevelCompaction e8808b912 Added falloc.h in build_detect_platform 1983fadcb assert(sorted) in vector rep 9da874801 Get benchmark.sh loads to run faster 9b983befa Fix flakiness of WalManagerTest d41a565a4 Don't do O(N^2) operations in debug mode for vector memtable 08be1803e Fix bad performance in debug mode 0a0501c8d Add Xfunc to makefile e7ad14926 Fix flakiness in FIFOCompaction test (github issue #573) abb405227 Kill benchharness 894e9f745 Update Patent Grant. 590fadc40 Fix compile warning on CLANG 47b874398 Make Compaction class easier to use 753dd1fdd Fix valgrind issues in memtable_list_test 697380f3d Repairer documentation improvement. 2f66d7f92 Add LinkedIn back to USERS.md 0feeee643 Fix memtable_list_test 7b9581bc3 Fixed xfunc related compile errors in ROCKSDB_LITE fabc11569 MemTableList tests 9741dec0e Fix a compile error in ROCKSDB_LITE in db/db_impl.cc 465b25ca9 "make commit-prereq" doesn't really build ROCKSDB_LITE d2a056241 Fix a compilation error in ROCKSDB_LITE in db/internal_stats.h 316ec80bf fault_injection_test: add a test case to cover log syncing after a log roll ed229a0de Fixes for readcache-flashcache 91df4e969 Remove use of whole-archive to include jemalloc 84c5bd7eb Add thread-safety documentation to MemTable and related classes ee9bdd38a Script to check whether RocksDB can read DB generated by previous releases and vice versa 2b019a151 Enabling checksum in repair db as it should have been. b1bbdd791 Create EnvOptions using sanitized DB Options edbb08b5d Fix Makefile 199313dc3 build: create .o files specifically for java-related targets b118238a5 Trivial move to cover multiple input levels e7adfe690 Fix formatting of USERS.md 4e7543dcf Add USERS.md 58346b9e2 Log writer record format doc. db6569cd4 Fix the compilation error in flashcache.cc on Mac cba592001 build: don't use a glob for java/rocksjni/* c66483c13 Fix github issue #563 de22c7bd1 Integrate Jenkins with Phabricator f12614070 Fix TSAN build error of D36447 824e64634 Adding another NewFlashcacheAwareEnv function to support pre-opened fd 5e067a7b1 Clean up compression logging e3ee98b38 run 'make check's rules (and even subtests) in parallel a45e7581b Avoid naming conflict of EntryType 3be82bc89 Add p99.9 and p99.99 response time to benchmark report, add new summary report 953a885eb A new call back to TablePropertiesCollector to allow users know the entry is add, delete or merge d2a92c13b avoid returning a number-of-active-keys estimate of nearly 2^64 a7ac6cef1 Fix level size overflow for options_.level_compaction_dynamic_level_bytes=true 089509b84 db_test: clean up sync points in test cleaning up afbafeaea Disallow trivial move if compression level is different d0695f3e2 Fix crash caused by opening an empty DB in readonly mode 51c8133a7 Fix make unity build compiler warning about "stats" shadowing global variable df71c6b9e Script to trigger jenkins test 38a01ed1b Update COMMIT.md 76d63b452 Fix one non-determinism of DBTest.DynamicCompactionOptions b23bbaa82 Universal Compactions with Small Files 2511b7d94 Makefile minor cleanup 1bd70fb54 Add --stats_interval_seconds to db_bench fd3dbef22 Clean up old log files in background threads 99ec2412e Make the benchmark scripts configurable and add tests 2158e0f83 Fix clang build d61cb0b9d db_bench can now disable flashcache for background threads 1c47c433b build: always attempt to update util/build_version.cc e018892bb Formalize the DB properties string definitions. cfa576402 Make auto_sanity_test always use the db_sanity_test.cc of the newer commit. e9fddb7a2 Merge pull request #560 from xiaoxichen/patch-1 bcd8a71a2 Fix interger overflow on i386 arch 030859eb5 Dump compression info on startup 3539e0644 Merge pull request #558 from aamihailov/master a3e4b3248 fix compilation error (same as fix #284) ff1ff7c62 TSAN: avoid new link failure with -pg 39d508e34 Add a missing section title in HISTORY.md 727684bf9 Fixed a typo in RocksDBSample.java ca25e86ef build: cause the "check" rule to depend on $(PROGRAMS) 986bdc680 Merge branch 'master' of github.com:facebook/rocksdb 1e57f2bf2 Fix build 2d417e52d Update HISTORY.md for 3.10.0 cbd6a2073 Merge branch 'master' of github.com:facebook/rocksdb 211ca26ae Fixing build issue 2495f9396 Merge pull request #555 from pshareghi/staticLZ4 f06de5f23 Merge pull request #556 from fyrz/RocksJava-Maven-Fix cd987c383 Fix compile error when NROCKSDB_THREAD_STATUS is not used. 8f104ae5e [RocksJava] Maven build fix 3d1a924ff Adding stats for the merge and filter operation afc51649e Merge pull request #546 from fyrz/RocksJava-MemEnv 4806cc126 Added static lz4 support for roccksjavastatic 6284eef4c Merge pull request #545 from fyrz/RocksJava-Level-Compression 248c063ba Report elapsed time in micros in ThreadStatus instead of start time. 315abac94 Undeprecate GetLiveFiles() a057bb2a8 Improve ThreadStatusSingleCompaction 868968c8e Merge branch 'master' of github.com:facebook/rocksdb 689391406 Make SSTDumpTest.GetProperties less noisy 8d8656243 Merge pull request #551 from fyrz/RocksJava-JavaDoc-Fix f8c505b23 Merge pull request #550 from fyrz/RocksJava-Sample-Fix 46443bfa9 [RocksJava] Add missing JavaDoc annotations 864b7e88f [RocksJava] Java sample bugfix 2b2394cbb [RocksJava] DBBenchmark option for RocksMemEnv fd8804f97 [RocksJava] Expose MemEnv in RocksJava Summary: In 3.10 the C++ code was extended with a MemEnv implementation. This is now also available in RocksJava. 004b89fba [RocksJava] Add compression per level to API 51da3aab4 Merge pull request #536 from fyrz/RocksJava-32-Bit-adjustment b7e88cfb5 Merge pull request #543 from fyrz/RocksJava-Logger-Comment 38d286f14 Clean-up WAL directory before running db_test 28bc6de98 rocksdb: print status error message when (ASSERT|EXPECT)_OK fails 9405b5ef8 rocksdb: Remove #include "util/string_util.h" from util/testharness.h 220d0dff7 rocksdb: Remove #include "util/random.h" from util/testharness.h b088c83e6 Don't delete files when column family is dropped 17ae3fcbc rocksdb: initial util/testharness clean up 39f4271be [RocksJava] Enhanced Logger comment 5615e23d8 [RocksJava] 32-Bit adjustments 836bcc2f0 Merge pull request #532 from fyrz/RocksJava-LevelCompactionDynamicLevelBytes 67d995808 rocksdb: fix make unity 52e0f3353 Clean up compactions_in_progress_ 6b626ff24 rocksdb: change db_test::MultiThreadedDBTest as value parameterized test. 9720ea4de A build option to run through all check-in requirements. 0831a3599 Add a DB Property For Number of Deletions in Memtables f7ed65464 Fix RocksJava test failure due to deprecation of table_cache_remove_scan_count_limit 51301b869 Enable dynamic changing of rate limiter's bytes_per_second 652db51a3 Fix compilation error in rocksjni/write_batch_test.cc dfccc7b4e Add readwhilemerging benchmark c345d1ee8 [RocksJava] Integrated changes for D34857 12350115d [RocksJava] Added LevelCompactionDynamicLevelBytes to Options 230e68727 Fix TSAN failue in env_test 155d468c5 Using chrono as a fallback 81345b90f Create an abstract interface for write batches 46214df4a Only run db_test in Travis c88ff4ca7 Deprecate removeScanCountLimit in NewLRUCache b4b69e4f7 rocksdb: switch to gtest 413e35273 Merge pull request #540 from dalgaaf/wip-da-fix-elif 969aa806b util/xfunc.h: fix #elif check for NDEBUG 87c7d49d6 util/env_posix.cc: fix #elif check for __MACH__ c86e5d7b9 stack_trace.cc: fix #elif check for OS_MACOSX 98c37fda5 Remove unused parameter in CancelAllBackgroundWork 9fd6edf81 rocksdb: Replace ASSERT* with EXPECT* in functions that does not return void value d4d42c02e Fixed clang build in env.h b2b308652 Speed up rocksDB close call. a7aba2ef6 rocksdb: Add gtest 95344346a rocksdb: Small refactoring before migrating to gtest bd4963e64 rocksdb: update reference to the latest version of clang dev in fbcode 56337faf3 Fix compaction IO stats to handle large file counts eafa1bfc3 Merge pull request #529 from fyrz/RocksJava-Logger ac03c4d51 Merge pull request #522 from fyrz/RocksJava-Optimize-Filters-For-Hits c6967a1a5 Make RecordIn/RecordOut human readable c8da67032 Stop printing per-level stall times. 04778a94c [RocksJava] OptimizeFiltersForHits 57f2a00c6 RocksJava - JNI Logger callback 814627af3 [RocksJava] Change log level at runtime in custom logger a3bd4142f [RocksJava] Logging JNI callback 58878f1c6 Switch to use_existing_db=1 for updaterandom and mergerandom 12134139e Fixed the unit-test issue in PreShutdownCompactionMiddle fd1b3f385 Fix the issue in PreShutdownMultipleCompaction 56c4a9c76 Fix compile warning in thread_status_util.h on Mac 417367c42 Fix SIGSEGV when not using cache e25ff039c Prevent slowdowns and stalls in PreShutdown tests f69071265 Speed up db_bench shutdown c1b3cde18 Improve the robustness of ThreadStatusSingleCompaction 8c12426c9 Fix the deadlock issue in ThreadStatusSingleCompaction. b16ead531 DBTest.DynamicLevelCompressionPerLevel should not run without snappy support a5e60bafc Fix a typo / test failure in ThreadStatusSingleCompaction cb2c91850 Don't run some tests is snappy is not present c594b0e89 Allow GetThreadList() to report operation stage. 2623b2cf6 Include chrono 52d8347a9 EventLogger 756532daf Merge pull request #524 from fyrz/RocksJava-Test-Fix 47a2b3a40 Merge pull request #534 from fyrz/RocksJava-Fix-BrokenJacocoReport 9d22a1f13 Allow negative Wnew 2c84303aa Merge pull request #528 from fyrz/RocksJava-NativeLibraryLoader 2dc636f62 [RocksJava] Fix broken jacoco report f210b0f6c [RocksJava] Fix NativeLibraryLoader 3ebebfccd Prevent xxhash symbols from polluting global namespace 53996149d Removing unnecessary kInlineSize b411d0603 Prevent stalls in preshutdown tests 1d43bc41f Fixing segmentation fault in db_bench e9de8b65a Change the way options.compression_per_level is used when options.level_compaction_dynamic_level_bytes=true 2b785d76b Fixed a bug where CompactFiles won't delete obsolete files until flush. 6f5579868 Fixed a compile error in db_bench in mac. 05d92efa7 Add convenience.cc to src.mk 2884b100b db_bench: Better way to randomize repeated read keys in -read_random_exp_range 284be570c Provide a mechanism to inform Rocksdb that it is shutting down 2ddf53b2c Get OptimizeFilterForHits work on Mac 480b28476 Fix make_new_version.sh 89597bb66 Allow GetThreadList() to report the start time of the current operation. 37921b499 db_bench: Add Option -read_random_exp_range to allow read skewness. 485ac0dbd Add rate_limiter to string options e126e0da5 Single threaded tests -> sync=0 Multi threaded tests -> sync=1 by default unless DB_BENCH_NO_SYNC is defined dc4532c49 Add --thread_status_per_interval to db_bench 34c75e984 fix-up patch: avoid new link error ebc647de8 build: fix missing dependency problems 492f6d27e Fix a segfault in fbson under Mac OS X compiler 0d13bbe27 RocksJava] Fix ColumnFamily tests 67533809f [RocksJava] Fixed CompactionTest 1b7b997b8 [RocksJava] Remove MaxValue from Statistics f862b3812 [RocksJava] Fix cleanup in tests a01b59259 [RocksJava] DefaultColumnFamily Memory Fix 22c73d15b [RocksJava] Fix ColumnFamily tests 694988b62 Fix a bug in stall time counter. Improve its output format. b8d23cdcb Revert chrono use db0373934 options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. f29b33c73 Add functionality to pre-fetch blocks specified by a key range to BlockBasedTable implementation. c4bd03a97 Fix typo in log message 3cf7f353d Instrument memtable seeks 216a9e16f Fix compile b9ff6b050 Fix a bug in ReadOnlyBackupEngine afa8156af adding stdlib to fbson f9c14a42e Fix compile on Mac a9f0e2b5b Fix compile e7c434c36 Add columnfamily option optimize_filters_for_hits to optimize for key hits only ba9d1737a RocksDB on FreeBSD support 4ba119df5 rocksdb: Update path to dev clang in fbcode 8984e5f84 Fix race in sync point. 03b432d4b rocksdb: Fix uninitialized use error ccef2a766 Merge pull request #518 from fyrz/RocksJava-Native-Library-Loader-Fix 9fcf1a7b0 [RocksJava] RocksJava Testcases 03bbf718c Return fbson 62247ffa3 rocksdb: Add missing override 1e06a4068 Support builds for MongoDB+RocksDB b74ad6632 Merge pull request #508 from fyrz/RocksJava-Final-Patch d9f4875e5 Disable pre-fetching of index and filter blocks for sst_dump_tool. 182b4ceac Limit key range to number of keys, not number of writes a360bb61b Merge pull request #516 from fyrz/RocksJava-Update-Statistics-To-3.10 8c7684474 [RocksJava] Updated TickerTypes and Histogram to 3.10 7f0c77cb3 [RocksJava] Integrated changes from D33417 819e787bb [RocksJava] Final usage correction 5139e678b Upgrade compiler in Travis 4ade89962 Fix compile error on MacOS. ace3d8506 Revert "Unused managed iterator" 7b8f348e5 Attempt at fixing travis issue d85993998 Merge pull request #506 from fyrz/RocksJava-Raw-Use 30e93c9b9 Merge pull request #505 from fyrz/RocksJava-Redundant-Modifier 217854dc4 Introduce DISABLE_WARNING_AS_ERROR in Makefile bd339a979 Unused managed iterator 174a79c99 LevelDb include guard replaced with #pragma once 6fdda8ac4 rocksdb: changes to support 'make analyze' in Jenkins 96d989f70 catch config errors with L0 file count triggers 62f7a1be4 rocksdb: Fixed 'Dead assignment' and 'Dead initialization' scan-build warnings 5636c4a80 Verbose build in travis a047409ae Fixed a bug in the test case 4f514a53d build: enable more compiler warnings a2b911b63 inputs: restore "const" attribute removed by D33759 1b4082581 mark as unused some variables with cpp-derived names c6d54b503 fix erroneous assert: cast kBlockSize (of type unsigned int) to "int" aa5d8e6d9 table_test.cc: add missing 5th arg in TestArgs initializer c37937a9c maint: remove extraneous "const" attribute from return type 9283c7afd build: remove always-true assertions 06a766de5 Adding Flush to AutoRollLogger 92416fa7f Fix mac build 96ab15d30 GetOptionsFromString + fixes to block_based_table_options 73711f956 rocksdb: Fix scan-build bug 'Memory leak' in db/db_bench.cc 98870c7b9 rocksdb: Fix scan-build memory warning in table/block_based_table_reader.cc a42324e37 build: do not relink every single binary just for a timestamp d45a6a400 Add rocksdb.num-live-versions: number of live versions 11581b741 build: abbreviate AR command, too b8ac71ba1 Revert "Fbson to Json" 7ce1b2c19 Fbson to Json 7d817268b Managed iterator b4b8c25a5 build: factor out AM_LINK command (trivial) dc885c6e9 build: make "make" output readable by default a37b46ae1 build: fix Makefile inconsistencies (trivial) 55277c328 build: remove unused rules: rocksdb_shell, DBClientProxy_test 3ad6b794c rocksdb: Fix 'Division by zero' scan-build warning 12753130e Remove ThreadStatusMultiCompaction test f0c36da6e Add thread_status_util_debug.cc back daebb1f91 build: running "make" with no arguments must not "uninstall" e60bc99fe Allow GetThreadList to reflect flush activity. b9a0213cd build: fix unportable Makefile syntax 4e4b85784 rocksdb: Fix scan-build 'Called C++ object pointer is null' and 'Dereference of null pointer' bugs b3fd16226 build: remove unportable use of sed in favor of $(CXX)'s -MT e7ea51a8e Introduce job_id for flush and compaction 6a0e737d9 [RocksJava] Raw use of parametrized class 439701270 [RocksJava] Redundant access-modifier in interfaces 2d62e8051 Merge pull request #504 from fyrz/RocksJava-Flush-Correction eaf39568e [RocksJava] FlushOptions Correction 5d1151deb Added simple monitoring script to monitor overusage of memory in db_bench 5f00af457 DBTest.DestroyDBMetaDatabase: create DB directories if not exists 68af7811e Remember whole key/prefix filtering on/off in SST file fd5970b45 Merge pull request #503 from weiweisd/master 513ad866b modify double type euqal compare in json_document.cc 933973dc2 Merge pull request #495 from fyrz/RocksJava-CF-Name-Byte-Handling 5e8e453d5 [RocksJava] Integrated changes from D33165 677d02427 [RocksJava] CF Name shall handle bytes correctly d1cafc089 Merge pull request #1 from facebook/master 6d6305dd7 Perf Context to report DB mutex waiting time 863009b5a Fix deleting obsolete files #2 1851f977c Added RocksDB stats GET_HIT_L0 and GET_HIT_L1 91ac3b206 Print DB pointer when opening a DB bee4e5124 Merge pull request #492 from fyrz/logger-logv-virtual cfe8837e4 Switch logv with loglevel to virtual aaceef363 Fix formatting ee4aa9a0e Merge pull request #481 from mkevac/backupable 82faa377a added simple example for db restore from backup d090330c8 fixed c_simple_example and added some comments 965130830 renamed backup to backup_and_restore in c_test for clarity bbb52b21f Merge pull request #483 from adamretter/restructure-java-build 7e50ed8c2 Added some more wrappers and wrote a test for backup in C 218c3ecea Fix std::cout data race 8f679c290 Merge branch 'master' of github.com:facebook/rocksdb da9cbce73 Add Header to logging to capture application level information 2a979822b Fix deleting obsolete files 8e83a9d31 Add a missing field for STATE_MUTEX_WAIT to global_state_table 6f1013035 Fix DestroyDB 7de4e99a8 Revert "Fix wal_dir not getting cleaned" 9a52e06a0 Add GetID to ColumnFamilyHandle 181191a1e Add a counter for collecting the wait time on db mutex. f36d394ae Fix wal_dir not getting cleaned 53ae09c39 db_test: fix a data race in SpecialEnv fe9f69119 Fix fault_injestion_test b37f5ffc7 Put db_test back to TESTS in Makefile 108470e96 Fix stack trace on mac 3e53760fc Fix compaction_picker_test e39f4f6cf Fix data race #3 e63140d52 Get() to use prefix bloom filter when filter is not block based 678503ebc Add utility functions for interpreting ThreadStatus 4d98e2935 rocksdb: Enable scan-build static analysis 756e1f151 Remove unused util/thread_event_info.h dad98dd4a Changes for supporting cross functional tests for inplace_update 9898f6398 Divide test DBIteratorTest.DBIterator to smaller tests 829363b44 Options::PrepareForBulkLoad() to increase parallelism of flushes b04408c47 Fix unity build 8d3819369 NewIteratorWithBase() for default column family 2c2d5ab7e Fix compile warning in util/xfunc.h 0b8dec717 Cross functional test infrastructure for RocksDB. 868bfa403 Merge pull request #488 from ekg/master 9900f3821 Merge pull request #484 from fyrz/RocksJava-Release-MD-change e6eaf938c remove old debugging message (#487) f33f3955e Moved Java Samples main classes into samples/src/main/java ad325517f Update test lib versions and maven plugin versions d6187d07b Maven can now build a standard project layout 157768890 Moved Java Benchmark main classes into benchmark/src/main/java dd8d5471e Adjustment to NativeLibraryLoader to allow native library to be loaded from either java.library.path or from extracting from the Jar. Means that the test in the build do not need to rely on the Jar, useful when creating similar builds (and executing tests) from Maven 353db6dae Moved Java main classes into src/main/java 98cb501bc Moved Java test classes into src/test/java 7479a62a7 Release.md - Remove version change instrcution 4a4e4279f Update HISTORY-JAVA.md 384cb6619 Merge pull request #480 from fyrz/RocksJava-Deprecate-SkipLogError ca52a67cf [RocksJava] Deprecate setSkipLogErrorOnRecovery 114d21878 Merge pull request #479 from fyrz/RocksJava-Snapshot-Sequence-Number cb5c3159f [RocksJava] Snapshot - GetSequenceNumber ea189b320 Merge pull request #474 from fyrz/RocksJava-GetUpdatesSince 391f85fc8 [RocksJava] Incorporated changes for D32151 68cd93b87 [RocksJava] GetUpdatesSince support caedd40dd [RocksJava] Adjusted auto pointer b39006e3d [RocksJava] enable/disable File deletions 9a456fba2 [RocksJava] GetUpdatesSince support d3a736761 Merge pull request #482 from fyrz/RocksJava-TTL-Fix 939bb3659 [RocksJava] Fix ColumnFamily name alloc in TTL DB 86e2a1eee Allow creating backups from C db9ed5fdb Unaddressed comment in previous diff. Change only in code comments. 5917de0ba CappedFixTransform: return fixed length prefix, or full key if key is shorter than the fixed length 6c6037f60 Expose Snapshot's SequenceNumber 2fd8f750a Compile MemEnv with standard RocksDB library 173c52a97 Fix build on older compilers -- emplace() is not available d07fec3bd make DBTest.SharedWriteBuffer to pass MockEnv 4bdf38b16 Disable FlushSchedule when running TSAN e84299c76 Fix bug recently introduced in MemFile::Lock() e5aab4c2b Fix data race in HashLinkList 2113ecd3c Merge pull request #472 from fyrz/RocksJava-Cleanup 10af17f3d fault_injection_test: add a unit test to allow parallel compactions and multiple levels 0c4d1053d Fix data race #5 cc0d8be01 [RocksJava] Integrated review comments (D32145) 5257c9c42 Merge pull request #452 from robertabcd/backupable-mem 560ed402b [minor] fprintf to stderr instead of stdout in test 551a41df3 Merge pull request #476 from alabid/alabid/add-to-simple-example d2a2b058f fault_injection_test: to support file closed after being deleted f8f040ccc Updated .gitignore to ignore *~ files and example object files e8bf2310a Remove blob store from the codebase ea7d0b943 Added WriteBatch block to simple_example.cc d6c7300cc Fixed a compile warning in clang in db/listener_test.cc f9758e012 Add compaction listener. e919ecedf SuperVersion::Unref() to use sequential consistency to decrease ref counting 4c49fedaf Use ustricter consistency in thread local operations 1b43ab58d fault_injection_test: add more logging and makes synchronization slightly stronger ca2b00277 [RocksJava] Cleanup portal.h & tests f8dc5c459 [RocksJava] Add missing test to Makefile b3c133148 [RocksJava] Removed todo comment in portal.h 7ffcc457f [RocksJava] Cleanup portal.h c4fb83441 Update the comment for the removal of mac-install-gflags.sh be8f0b12e Rename DBImpl::log_dir_unsynced_ to log_dir_synced_ c1de6c42a fault_injection_test: add a test case to drop random number of unsynced data d888c9574 Sync WAL Directory and DB Path if different from DB directory 58f34edfc Fix valgrind f1c886247 Fix data race #1 b08b2fe73 Merge pull request #471 from fyrz/RocksJava-Fix-NativeLibraryLoader e61f38e5a [RocksJava] Fix native library loader 26b50783d Fix assert in histogramData 42189612c Fix data race #2 f5a839835 Fix archive WAL race conditions 43ec4e68b fault_injection_test: bring back 3 iteration runs c2e8e8c1c Fix two namings in fault_injection_test.cc b4c13a868 fault_injection_test: improvements and add new tests a52dd0024 Fix ASAN failure with backupable DB 910186c27 Return the build with 4.8.1 401d4205e Add thread sanitizer b068f0a67 Upgrade our compilers a76d92862 Merge pull request #466 from fyrz/RocksJava-Support-ReadOptions-Iterator bef7821f0 [RocksJava] ReadOptions support in Iterators 3b494a610 Make options_test runnable on ROCKSDB_LITE 912c52e82 Merge pull request #465 from fyrz/RocksJava-BlockBasedTable-FormatVersion cd4c07197 Update HISTORY.md for GetThreadStatus() support on compaction. 46a7048dc Reduce false alarm in ThreadStatusMultipleCompaction test aed028698 Merge pull request #462 from fyrz/RocksJava-JNI-allocation-correction e5df90f5d Fix comment (minor) dd53428f8 Incorporated review comments 908258a4f [RocksJava] BlockBasedTableConfig 3.10 2efe22849 [RocksJava] Incorporated changes for D31809 4e48753b7 Sync manifest file when initializing it e204a5a16 [RocksJava] ColumnFamily name JNI correction 96264784d [RocksJava] ColumnFamily name JNI correction ae82849bc Fix build failure 423dee841 Abort db_bench if Get() returns error 206237d12 DBImpl::CheckConsistency() shouldn't create path name with double "/" 5e98e5324 Merge pull request #458 from fyrz/RocksJava-TTLDB-Support 4ffe0be41 [RocksJava] Integrated changes for D31449 e82856754 [RocksJava] Integrated changes from D31449 859c54a03 [RocksJava] TTL-Support 5ff8aec4d [RocksJava] TTL Support ca47da9e6 [RocksJava] TTL-Support 1190ebe5a Merge pull request #461 from fyrz/RocksJava-DirectSlice-Fix ea25ff715 [RocksJava] Integrated proposed simplificiation d68e83c35 [RocksJava] DirectSlice String termination fix 0ddf5f73e memenv: normalize file path 4d9d5955a Merge pull request #464 from fyrz/RocksJava-Various-Fixes ceaea2b72 Adding prefix_extractor string config parameter 3d628f8f2 Update format_version comment 155bec4cf fallocate also tests FALLOC_FL_KEEP_SIZE c75c02e7a [RocksJava] WriteBatchWithIndexTest fix c787fb50b [RocksJava] JavaDoc errors in Java8 b229f970d Remove Compaction::ReleaseInputs(). f2ddb8b45 Fix for bug where GeoDB accesses key after next modification of iterator d10f1de2b Ported LevelDB's fault_injection_test 2bb059007 Change db_stress to work with format_version == 2 9ab5adfc5 New BlockBasedTable version -- better compressed block format 2355931c6 Merge pull request #450 from adamretter/writebatch-with-index 3d246c89c Abstract duplicate code on key and value slice objects into generic methods 2d0dd8db3 Implement WBWIRocksIterator for WriteBatchWithIndex in the Java API de678b288 Abstractions for common iterator behaviour e01acb3a0 Test for WriteBatchWithIndex#newIterator() 56f24941a Simplify the Java API by permitting WriteBatchWithIndex to be provided straight to RocksDB#write 95d5f9848 Test for RocksDB#write(WriteBatchWithIndex) ef5b34dee Implement WriteBatchWithIndex in the Java API c6e554561 Abstractions for common write batch behaviour be905491b Test for WriteBatchWithIndex#newIteratorWithBase(org.rocksdb.RocksIterator) 2241e3f4d Extract the interface for a RocksIterator a8cfa7ace Extract the interface for a WriteBatch 45e43b81d Adds support for db->DefaultColumnFamily() to the Java API 516a04267 Add LZ4 compression to sanity test 2ccc54301 Merge pull request #460 from neutronsharc/master 2a7bd0ea4 Remove duplicated method declarations in C header. bb128bfec More accurate message for compaction applied to a different version 96b8240bc Support footer versions bigger than 1 53f615df6 Fix clang build 02b30202c Merge pull request #455 from Andersbakken/stdlib_fix 2159484dd Remove two unnecessary blank lines in db/db_test.cc e7dd88c57 Merge pull request #441 from fyrz/RocksJava-ColumnFamilyDescriptor-Alignment d2c018fd5 Make ThreadStatusMultipleCompaction more robust. bf9aa4dfc Improve GetThreadStatus to avoid false alarm in some case. c91cdd59c Allow GetThreadList() to indicate a thread is doing Compaction. 402c1152a Fix c_simple_example a9ea65d65 Build with clang 3.5 on Linux. 23ad5f401 [RocksJava] Incorporated changes for D30525 0aab1005f [RocksJava] ColumnFamilyDescriptor alignment with listColumnFamilies 15d2abbec Fix build issues abb9b95ff Move compression functions from port/ to util/ 9132e52ea DB Stats Dump to print total stall time 93b35c299 Merge pull request #453 from fyrz/SimpleCExampleSigSegv 628a67b00 Reduce memory footprint in backupable db. ef3901642 Fixed memory issue in c_simple_example b89d58dfa :%s/build_config/make_config 242b9769c Memtablerep Benchmark 73ee4feba Add comments about properties supported by DB::GetProperty() and DB::GetIntProperty() 2dca48f55 Merge pull request #451 from StanislavGlebik/document_db_improvement 4b57d9a82 Fixed negative numbers comparison in DocumentDB 9ef59a09a VersionSet::AddLiveFiles() to assert current version is included. 4d16a9a63 VersionBuilder to optimize for applying a later edit deleting files added by previous edits 7731d51c8 Simplify column family concurrency 07aa4e0e3 Fix compaction summary log for trivial move 9d5bd411b benchmark.sh won't run through all tests properly if one specifies wal_dir to be different than db directory. 62ad0a9b1 Deprecating skip_log_error_on_recovery fa0b126c0 Fix corruption_test -- if status is not OK, return status -- during recovery d7b4bb62a Fail DB::Open() on WAL corruption 9619081d9 Merge pull request #449 from robertabcd/improve-backupable 49376bfe8 Fix errors when using -Wshorten-64-to-32. a8c5564a9 Do not issue extra GetFileSize() calls when loading BackupMeta. caa1fd0e0 Improve performance when loading BackupMeta. e9ca35815 Fix CLANG build for db_bench bf287b76e Add structures for exposing thread events and operations. a801c1fb0 db_bench --num_hot_column_families to be default off 2067058a6 Dump routine to BlockBasedTableReader (valgrind) ddc81440d db_bench to add an option as number of hot column families to add to a944afd35 Fixed a compile error in db/db_impl.cc on ROCKSDB_LITE 7ea7bdf04 Dump routine to BlockBasedTableReader ae508df90 Clean up compile for c_simple_example b62300961 Fix compile of compact_file_example ded26605f Merge pull request #444 from adamretter/java-api-fix 98490bccf Fix the build on Mac OS X 4d9972974 Merge pull request #443 from behanna/master 5045c4394 add support for nested BlockBasedTableOptions in config string d232cb156 Fix the build with -DNDEBUG. 45bab305f Move GetThreadList() feature under Env. 4fd26f287 Only execute flush from compaction if max_background_flushes = 0 0acc73881 Speed up FindObsoleteFiles() d8c4ce6b5 Merge pull request #442 from alabid/alabid/fix-example-typo 949bd71fd fix really trivial typo f8999fcf3 Fix a SIGSEGV in BackgroundFlush ade4034a9 MultiGet for DBWithTTL fdb6be4e2 Rewritten system for scheduling background work a3001b1d3 Remove -mtune=native because it's redundant e27c84522 Merge pull request #437 from fyrz/RocksJava-SliceTests-Fixes 1fed1282a [RocksJava] Incorporated changes D30081 5b9ceef01 [RocksJava] JavaDoc correction 5fbba60b6 [RocksJava] Incorporated changes D30081 b0230d7e0 [RocksJava] Incorporate additions for D30081 b015ed0ca [RocksJava] Slice / DirectSlice improvements 4d422db01 Merge pull request #430 from adamretter/increase-parallelism 04c4e4969 Merge pull request #411 from fyrz/RocksJava-RangeCompaction 62d19b7b5 Merge pull request #427 from haneefmubarak/c-examples 28424d734 style fixes in c example 7198ed5a2 Handle errors during pthread calls 91c58752f error detection and memory leaks in c example 25f70a5ab Avoid unnecessary unlock and lock mutex when notifying events. 7661e5a76 Move the file copy out of the mutex. 17e84f215 Rudimentary test cases for setIncreaseParallelism eda0dcdd9 Exposed IncreasedParallelism option to Java API as setIncreasedParallelism efc94ceb2 [RocksJava] Incorporated changes for D29283 69188ff44 [RocksJava] CompactRange support 48adce77c [RocksJava] CompactRange support 153f4f071 RocksDB: Allow Level-Style Compaction to Place Files in Different Paths 06eed650a Optimize default compile to compilation platform by default cef6f8439 Added 'dump_live_files' command to ldb tool. 7ab1526c0 Add an assert and avoid std::sort(autovector) to investigate an ASAN issue 74b3fb6d9 Fix Mac compile errors on util/cache_test.cc d7a486668 Improve scalability of DB::GetSnapshot() ee95cae9a Modifed the LRU cache eviction code so that it doesn't evict blocks which have exteranl references 0ab0242f3 VersionBuilder to use unordered set and map to store added and deleted files e93f044d9 add range scan test to benchmark script cb82d7b08 Fix #434 046ba7d47 Fix calculation of max_total_wal_size in db_options_.max_total_wal_size == 0 case 1b7fbb9e8 Update HISTORY.md for release 3.9 635c61fd3 Fix problem with create_if_missing option when wal_dir is used 2871bc7bc Merge pull request #422 from fyrz/RocksJava-Quality-Improvements 8c5781666 Add -fno-exceptions flag to ROCKSDB_LITE. 1f04066ca Add DBProperty to return number of snapshots and time for oldest snapshot 6436ba6b0 Provide mechanism to restart tests from previous error d84b2bade Replace exception by abort() in dummy HdfsEnv implementation. 9260e1ad7 Bump version to 3.9 8f4e1c1c9 Remove the compability check on log2 OS_ANDROID as it's already blocked by ROCKSDB_LITE c4a7423c1 Replace runtime_error exception by abort() in thread_local a94d54aa4 Remove the use of exception in WriteBatch::Handler a5d4fc0a2 Fix compile warning in db_stress 1a8f4821a Replace exception by assertion in autovector 97c194088 Fix compile warning in db_stress.cc on Mac 5f719d720 Replace exception by setting valid_ = false in DBIter::MergeValuesNewToOld() c0dee851c Improve formatting, add missing newlines 815f638cd Fix java build 32a0a0384 Add Moved(GB) to Compaction IO stats a14b7873e Enforce write buffer memory limit across column families 3e684aa68 Integrated changes from D29571 37d73d597 Fix linters a15169f2e Fixed a Lint problem b7f9e644c [RocksJava] Quality improvements e002a6122 [RocksJava] Comparator tests for CF 335e6ad5c [RocksJava] Remove obsolete dbFolder cleanup b036804ac RocksJava - FindBugs issues 9a632b4a9 Merge pull request #429 from fyrz/RocksJava-MacOSX-strip-fix b42667506 [RocksJava] MacOSX strip support e463cb0bc Merge pull request #424 from eile/master 91d898163 Tweak Makefile for building on BG/Q c6f31a289 minor memory leak in C example 703ef66a8 Merge pull request #426 from fyrz/RocksJava-Restore-PrecisionFix ac4ed1e30 fix examples/makefile for C example d7f5ccb0c add c example to makefile and fix "make clean" 9c34d5e36 fix type in C simple example 0a9a7e753 added C version of simple_example bcf908689 Block Universal and FIFO compactions in ROCKSDB_LITE 67cb7ca75 [RocksJava] Fixed MacOS build of RocksJava b8136a7d2 Merge pull request #398 from fyrz/RocksJava-CreateCheckPoint 533592a27 Merge pull request #401 from fyrz/RocksJava-Sigsegv-MergeOperatorName 73d72ed5c Block ReadOnlyDB in ROCKSDB_LITE e47f0fa9e Merge pull request #425 from adamretter/macosx-clean-fix ff0cb90d1 Do not delete Java Fatal Error Log, developers may still want these for reference 2a792cd30 There will also be a librocksdbjni-osx.jnilib.dSYM folder on MacOSX builds to be deleted beb74c14c Fix travis-build error a486352e0 Merge pull request #423 from zerebubuth/c_iterate_upper_bound 26109d487 Store upper bound `Slice` with the same lifetime as the `ReadOptions` so that we can provide a pointer to it. a97314219 Fix compile error in ROCKSDB_LITE 9d5019327 Replace log2 by implementing Log2 in options_builder 805bac6d2 Add test for upper bounds on iterators using C interface. f193deea3 [RocksJava] Addressed comments in D28971 94f70a86b [RocksJava] Incoroporated changes for D29013 a280af2a5 [RocksJava] Sigsegv fix for MergerOperatorByName fcc2dfd9f [RocksJava] Support for stored snapshots 274ba6270 Block internal_stats in ROCKSDB_LITE 4f2e8bab5 Merge pull request #421 from fyrz/RocksJava-PrecisionFix c4765dc10 [RocksJava] Fix precision problem in rocksjni 14788e181 Merge pull request #420 from rdallman/add-wal 88dd8d889 c api: add max wal total to opts 7e608e2fe Block plain_table_index.cc in ROCKSDB_LITE 13de000f0 Add rocksdb::ToString() to address cases where std::to_string is not available. 90ee85f8e Improve listener_test to avoid possible false alarm 2946e37a0 remove unreliable test in db/cuckoo_table_db_test.cc 9c7ca65d2 free builders in VersionSet::DumpManifest 7530c75ab Merge pull request #413 from saghmrossi/master d699d7034 Make RocksDB compile without gflags 325722149 Fixes valgrind error in GetSnapshotLink. Free checkpoint now. ada3d7873 Merge pull request #415 from fyrz/RocksJava-Makefile 569853ed1 Fix leak when create_missing_column_families=true on ThreadStatus c4b65f70f [RocksJava] Makefile correction 141018016 Make arena use hugepage if possible 3a40c427b Fix db_bench on CLANG mode 9222a2d02 Fixed iOS build caused by GetThreadList feature. aa31fc506 Improve listener_test by ensuring flushes are completed before assert. 7ec71f101 Provide default implementation of LinkFile, don't break the build cd278584c Clean up StringSplit d84069995 Fix mac compile 4f882924d Merge pull request #404 from fyrz/RocksJava-Backup-Restore-3.8 4b63fcbff Add enable_thread_tracking to DBOptions bafce6197 first rdb commit 9e285d423 Added CompatibleOptions for compatibility with LevelDB Options 353307758 Add IOS_CROSS_COMPILE to macro guard for GetThreadList feature. eecdebe65 Fixed the destruction order of static variables in ThreadStatusImpl. 004f416b7 Moved checkpoint to utilities beabc6879 Fixed ~ThreadStatusImpl(). faa8d32be [RocksJava] Integrated changes from D29019. 3d78c7a8c [RocksJava] Lint adjustments d7529b2de [RocksJava] Cleanup Backupable implementations fa703efb2 [RocksJava] Improved BackupableDBTest 24fdc4741 [RocksJava] Backupable/Restorable DB update 3.8.0 9972f969e [RocksJava] BackupableDBOptions alginment + 3.8 fbc42a093 Fixed -Werror=unused-but-set-variable in thread_status_impl a564be715 Fix asan error in thread_status_impl.cc 7165d1886 Fix clang compile error d0c5f28a5 Introduce GetThreadList API 1fd1aecb3 Merge pull request #409 from fyrz/RocksJava-Make-cleanup 91c8dcefc [RocksJava] Strip library in publish e7fcaa4d9 [RocksJava] JavaDoc is executed too often 2cd1794e4 [RocksJava] Make cleanup - Clean Target be005e17b fix clang compilation 5e69f19c4 Merge pull request #405 from fyrz/RocksJava-Convenient-Options 55a344872 Merge pull request #408 from fyrz/Missing-include 9e9a83baf Missing header in build on CentOS 91ccc8ebe [RocksJava] Integrated changes in D29025 5249d0db5 [RocksJava] Convenience methods for Options 8d3f8f969 remove all remaining references to cfd->options() 1e4a45aac remove cfd->options() in DBImpl::NotifyOnFlushCompleted 517c28994 Options helper supports k, m, g, and t unit suffixes c46c2be8d Merge pull request #397 from fyrz/RocksJava-GetIntProperty 8efd4bb42 [RocksJava] Improved comments in RocksDB class 5529c1ad1 [RocksJava] GetIntProperty in RocksDB db59eeb61 Merge pull request #406 from fyrz/Build-Fix e97f014b9 [RocksJava] JavaDoc corrections - Java8 98e59f981 Fixed a bug which could hide non-ok status in CompactionJob::Run() ec24bd4e6 Merge pull request #402 from adamretter/bugfix-native-library-loader d3c4a0f4a Improve the comment in InfoLogLevelTest.java a77e97c53 Merge pull request #396 from fyrz/RocksJava-LogLevel 585c759cf Make sure to use the correct Java classloader for loading the RocksDB Native Library c3915abba Minor tidyup and use Java 7 for file copying a122a42bb Merge pull request #399 from fyrz/RocksJava-Version-to-3.8 b8d5e3f08 [RocksJava] MVN Build reads version from version.h 23295b74b Clean job_context 0ce38fe98 Fix signed/unsigned compile e7960c03a Don't parallelize the build in travis 84af2ff8d Clean job context in DeleteFile 8a1bcc39c [RocksJava] Bump version to 3.8 in rocksjni.pom 5c04acda0 Explicitly clean JobContext 4947a0674 [RocksJava] Incorporated review comments D28947 07cd3c42a [RocksJava] LogLevel support in Options 26dc5da96 Fix compaction_job_test 5f583d2a9 Merge pull request #394 from lalinsky/cuckoo-c 353303a76 Merge pull request #380 from fyrz/RocksJava-Junit-Framework 3f9c95a51 [RocksJava] Minor lint correction e46450da6 [RocksJava] Rebased + integrated CF tests cd82beb0c [RocksJava] Merged in latest changes. b6abab8b7 [RocksJava] Merged & rebased to HEAD 74057d6d2 [RocksJava] Improved tests within RocksJava 628e39e97 [RocksJava] Integrated review comments from D28209 a4b28c1ae [RocksJava] Extended Testcases 36f3a0bb8 [RocksJava] Integrated review comments from adamretter in D28209 b09268695 [RocksJava] Extended testcases 9bec23c41 [RocksJava] Test-framework integration f617135d5 [RocksJava] Testcase improvements 1fe7a4c62 [RocksJava] Test-framework integration 04ca7481d Fix build 6c1b040cc Provide openable snapshots 9be338cf9 CompactionJobTest c9fd03ec5 Update docs for NewAdaptiveTableFactory e6c3cc657 Add very basic tests to make sure the C cuckoo table options compile and run 7fe247080 Update HISTORY.md for RocksJava c44a29278 Add cuckoo table options to the C interface 136b8583b Merge pull request #395 from lalinsky/fix-env-test 94fa542f8 Update HISTROY.md for 3.8 release a177742a9 Make db_stress built for ROCKSDB_LITE 746cfaac5 Relax the block count check on deallocation in env_test f822129b3 Add a unit test for behavior when merge operator and compaction filter co-exist. 4161de92a Fix SIGSEGV 373c665ed Fix broken test in 31b02d. 772bc97f1 No CompactFiles in ROCKSDB_LITE 1d1a64f58 Move NeedsCompaction() from VersionStorageInfo to CompactionPicker cd0980150 Add concurrency to compacting SpatialDB 3c92e5233 Fix include 25f273027 Fix iOS compile with -Wshorten-64-to-32 fa50abb72 Fix bug of reading from empty DB. 31b02dc21 Improve Backup Engine. 1033db29f Merge pull request #390 from fyrz/RocksJava-Cleanup 9a03da773 Merge pull request #375 from fyrz/RocksJava-ColumnFamilyOptions-Extension-3.6 d50c68e3a [RocksJava] JavaDoc cleanup warnings with Java8 079d942ea [RocksJava] Code-cleanup + Java7 warnings removed 9a255b95f [RocksJava] Sample and Default value 9d2ba2136 [RocksJava] Incorporated review comments fa9cfc65f [RocksJava] Integrated Review comments from yhchiang in D28023 75010d208 [RocksJava] ColumnFamily custom Options API extension 0345c2156 [RocksJava] Extend Options with ColumnFamilyOptions implementation ColumnFamilyOptions implementation with tests [RocksJava] Extended ColumnFamilyTest 975949522 Fixed clang compile error in version_builder_test 581141935 Fixed GetEstimatedActiveKeys 1f621e6ab Fix additional -Wshorten-64-to-32 errros 767777c2b Turn on -Wshorten-64-to-32 and fix all the errors 113796c49 Fix NewFileNumber() 625e162c6 Merge pull request #393 from fyrz/RocksJava-Flush fc6fcbab9 [RocksJava] Flush functionality 8e5547f64 [RocksJava] Makefile restructured 26f0a78b0 Merge pull request #363 from adamretter/write_batch-iterate 35c8c814e Make ForwardIterator::status() more efficient d88568c68 Move -Wnon-virtual-dtor to c++ flags c7ee9c3ab Fix -Wnon-virtual-dtor errors 746252197 Merge pull request #384 from msb-at-yahoo/compaction-filter-2-empty-changed-values dd726a59e Bump Version Number to 3.8 4a3bd2bad Optimize usage of Status in CompactionJob bcdb9671c Fix build d904fbbb0 Addresed comments from code review https://reviews.facebook.net/D27567 eeb9cf6c4 Test for WriteBatchHandler 8fb4751d5 Iterator support for Write Batches 00211f9c5 Fix SIGSEGV in db_stresS a4a2bfd6b Merge pull request #391 from Liuchang0812/fixmake 856059094 Merge pull request #389 from Liuchang0812/master 01a770637 remove unused target dc3410463 Merge branch 'fix-example' e7620536c fix make static_lib error 543df158c Expose sst_dump functionality as library call. e3d3567b5 Get rid of mutex in CompactionJob's state 344edbb04 Fixed the shadowing in db/compaction.cc and include/rocksdb/db.h b8b390342 Fixed compile error in db/db_impl.cc b622ba5d6 Fixed compile error in db/flush_job.cc 642ac9d8a Fixed compile error in db/compaction.cc and db/compaction_picker.cc 68effa034 Fix -Wshadow for tools 844786189 Fixed -WShadow errors in db/db_test.cc and include/rocksdb/metadata.h 28c82ff1b CompactFiles, EventListener and GetDatabaseMetaData 5c9309053 Turn on -Wshadow 31342c400 Fix implicit compare a0f887c9e Fix compile 53af5d877 Redesign pending_outputs_ ec101cd49 Correctly test both compaction styles in CompactionDeletionTriggerReopen 8d87467bb Make PartialCompactionFailure Test more robust again. 64d302d30 make DropWritesFlush deterministic cd5c0925a Merge pull request #387 from fyrz/RocksJava-WShadow-Fix c4bf07c24 [RocksJava] -WShadow improvements e526b7140 Make PartialCompactionFailure Test more robust. 0c2be0de3 Turn on -Wshadow for travis 5fd33d26f Turn off -Wshadow 9f20395cd Turn -Wshadow back on c02338a69 update HISOTRY.md for new release 367a3f9cb Improve DBTest.GroupCommitTest: artificially slowdown log writing to trigger group commit b52b144c7 Merge pull request #386 from EugenePig/java8 ac95ae1b5 Make sure WAL is synced for DB::Write() if write batch is empty 59d549798 suppress JDK8 errors for #385 ea18b944a Add db_bench option --report_file_operations 2ea1219eb Fix RecordIn and RecordDrop stats e4211d10c Apply InfoLogLevel to the logs in util/env_hdfs.cc 76f6c7c7c CompactionFilterV2: eliminate an often unnecessary allocation. 29a9161f3 Note dynamic options in options.h fd24ae9d0 SetOptions() to return status and also add it to StackableDB b1267750f fix the asan check 83bf09144 Bump verison number to 3.7 da5daa061 Replace some ASSERT_TRUE() asserts in DBTest.DynamicMemtableOptions and DynamicCompactionOptions with more specific ones b0cda4a11 DBTest.DynamicMemtableOptions to use single background compaction 8810850dd Apply InfoLogLevel to the logs in db/compaction_job.cc 71783f652 Merge pull request #377 from fyrz/RocksJava-KeyMayExist 614bbcbe2 Merge pull request #374 from fyrz/RocksJava-DBOptions-Extension-3.6 d8e119663 Apply InfoLogLevel to the logs in db/version_set.cc 2a019f1d0 Apply InfoLogLevel to the logs in db/wal_manager.cc 469d474ba Apply InfoLogLevel to the logs in db/db_impl.cc ac6afaf9e Enforce naming convention of getters in version_set.h 09899f0b5 DB::Open() to automatically increase thread pool size if it is smaller than max number of parallel compactions or flushes 636e57b52 Fix coverage script 30ca3752b Revamp our build tools 051c67ff5 Merge pull request #378 from baotiao/master d0e7e49ae Merge pull request #379 from fyrz/RocksJavaBuildFix 94e31ac22 [RocksJava] Extend Options with DBOptions implementation [RocksJava] Included DBOptionsTest and refactored OptionsTest b060d3006 [RocksJava] Build fix after options refactoring 9fd65e566 add make clean in examples makefile 8e79ce68c Revert "Fix lint errors and coding style of ldb related codes." 45a612f99 Revert "Fix incorrect fixing of lint errors in ldb_cmd.cc" 27129c739 [RocksJava] KeyMayExist w/o ColumnFamilies 86905e3cb Move VersionBuilder logic to a separate .cc file 74eb4fbe9 CompactionJob 8ddddd62d Fix incorrect fixing of lint errors in ldb_cmd.cc 46c14c666 Fix #258. benchmarkharness -- make bm_min_usec uint 72cb7cf20 Add fsync / corrupt simulation to env_mem 0e526eb9d introduce TestMemEnv and use it in db_test 8db24f4b3 exclude mock test file from MOCK_SOURCES 5594d446f unfriend DBImpl and InternalStats from VersionStorageInfo 82e3ae540 fix c_test bc9f36fd5 Fix lint errors and coding style of ldb related codes. c645250ee CompactionStats to support larger value of RecordIn and RecordDrop f7e6c856a Fix BaseReferencedVersionBuilder's destructor order c76dcb44d fix b452dede5 fix 29d83cc33 temporarily remove -Wshadow c1a924b9f Move convenience.h to /include 7e01d1202 Add support for in place update for db_stress 9f7fc3ac4 Turn on -Wshadow 98849a35f Apply InfoLogLevel to the logs in table/block_based_table_reader.cc 4d2ba38b6 Make VersionBuilder unit testable 2b1f23dca Apply InfoLogLevel to the logs in db/db_iter.cc ccaf1aa7c Merge pull request #372 from fyrz/RocksJava-CF-Merge-Hardening a29118ffc Merge pull request #355 from fyrz/RocksJava-Options-Refactoring-3.6 85b04ca76 [RocksJava] Review comments - reformatted MergeTest df7abb4e8 [RocksJava] Integrated code review comments 171be0ed5 Merge with ColumnFamilies & Hardening CFHandle 39464a990 [RocksJava] Options Refactoring 3.6 0f7f3b860 Check InfoLogLevel earlier in Log functions. 73605d917 Apply InfoLogLevel to the logs in util/db_info_dumper.cc fda592d90 Merge pull request #356 from fyrz/RocksJava-TableOptions-3.6 c73d13bb8 [RocksJava] Integrate review comments from yhchiang b011e201f Integrated review comments by ankgup87 2c1bd8846 BlockBasedTableConfig & PlainTableConfig enhancements e770d6175 Merge pull request #371 from dlezama/master 41af0f56b Fix build break because of unsigned/signed mismatch c5db7f260 Fix CompactionPickerTest.Level1Trigger2 37e9b6370 Apply InfoLogLevel to the logs in utilities/ttl/db_ttl_impl.h 217cc217d Apply InfoLogLevel to the logs in table/meta_blocks.cc 635905481 WalManager fd95745a5 Fix compile error in table/plain_table_index.cc c3dd0f75d comparator_db_test to cover more irregular comparators 6afafa369 Apply InfoLogLevel to the logs in utilities/merge_operators/uint64add.cc e7ad69b9f Apply InfoLogLevel to the logs in table/plain_table_index.cc bbd9c5345 Apply InfoLogLevel to the logs in table/block_based_table_builder.cc 065766b8d DynamicCompactionOptions: relax the check bound a little 5c82a8837 Add a test in compaction_picker_test to test the max score 86de2007b Add ComparatorDBTest to test non-default comparators 17be187ff dummy var to suppress compiler warning/error c2999f54b Revert "tmp" 76d1c28e8 Make CompactionPicker more easily tested 01e6f8509 Apply InfoLogLevel to the logs in db/transaction_log_impl.h 082e49ba8 Apply InfoLogLevel to the logs in db/repair.cc c4b468000 Apply InfoLogLevel to the logs in db/flush_job.cc 34d436b7d Apply InfoLogLevel to the logs in db/column_family.cc cda9943f9 Apply InfoLogLevel to the logs in db/compaction_picker.cc 7b3a618f9 Apply InfoLogLevel to the logs in db/db_filesnapshot.cc 2d4fe048f remove dead code 9ab013236 tmp 76d54530d minor - remove default value for ChangeFilterOptions() and ChangeCompactionOptions() 44f0ff31c use fallocate(FALLOC_FL_PUNCH_HOLE) to release unused blocks at the end of file 97451f837 add an env var ROCKSDB_TESTS_FROM to control where to start from a list of tests e130e88bc DBTest: options clean up - part 4 34f3c5a20 DBTest: options clean up - part 3 cdc7230e4 DBTest: options clean up - part 2 5a921b895 DBTest: options clean up - part 1 c9c935923 Move the check to the beginning of the loop in VersionEdit::EncodeTo() 2110e43a5 Remove an unnecessary include file in version_edit.cc 412b7f85b Include atomic in mock_table.h c08285334 Include all the mocks abac3d647 TableMock + framework for mock classes fb3f8ffe5 Improve the robustness of PartialCompactionFailure test again. 60fa7d136 Improve the robustnesss of PartialCompactionFailure test. 3772a3d09 Fix the bug where compaction does not fail when RocksDB can't create a new file. c49dedbe0 Merge pull request #366 from fyrz/RocksJava-Backup-Restore-Improvements a39e931e5 FlushProcess efa2fb33b make LevelFileNumIterator and LevelFileIteratorState anonymous f7c973069 [RocksJava] Integrated review comments 7e12ae5a2 [RocksJava] - BackupInfos & Restore-/BackupableDB enhancements eb357af58 unfriend ForwardIterator from VersionSet f981e0813 unfriend ColumnFamilyData from VersionSet 834c67d77 rename FileLevel to LevelFilesBrief / unfriend CompactedDBImpl a28b3c438 unfriend UniversalCompactionPicker,LevelCompactionPicker and FIFOCompactionPicker from VersionSet 5187d896b unfriend Compaction and CompactionPicker from VersionSet 75d7e2c37 Merge pull request #369 from fyrz/Small-Fix db52419cf Merge pull request #290 from vladb38/master 45e756f04 [RocksJava] Minor correction to the previous pull request merge f94f1a97d Merge pull request #368 from fyrz/RocksJava-Hardening-RocksIterator b08c39e14 [RocksJava] RocksIterator: Assert for valid RocksDB instance & documentation b680033e6 Include atomic in env_test 56ef2caaa [RocksJava] - Hardening RocksIterator c1c68bce4 remove atomic_pointer.h references 7c303f0e7 Include atomic 179c23021 Merge pull request #351 from fyrz/RocksJava_Snapshot_Support 48842ab31 Deprecate AtomicPointer f37048ad1 Merge pull request #367 from fyrz/FixBrokenJavaBuild 679a9671f RocksJava Fix after MutableCFOptions change. 714c63c58 db_stress for dynamic options f1841985e dynamic inplace_update options a04929aa4 fixed conflict in java/Makefile a1bae76c8 Integrated changes due to review bei ankgup87 b8ce52648 [RocksJava] Support Snapshots bc3bc4bc2 Merge pull request #357 from fyrz/JavaTest-Fix 42f0eaceb Merge pull request #354 from fyrz/RocksJava-memtables-3.6 965d9d50b Fix timing 001ce64dc Use chrono for timing 240ed0cd7 Fix uninitialized parameter caused by D24513 724fba2b3 Improve the log in Universal Compaction to include more debug information. 720c1c056 fix erro during merge b794194ad Remove java build from travis 122f98e0b dynamic max_mem_compact_level 1fee591e7 comments for DynamicCompactionOptions test 574028679 dynamic max_sequential_skip_in_iterations bd4fbaee3 Fixed cross platform build after introducing Java-7 dependencies 1eb545721 Fix incorrectly merged Java - Makefile 9aa9668a8 [RocksJava] Memtables update to 3.6 4b1786e95 Fix SIGSEGV when declaring Arena after ScopedArenaIterator 05b2e60dd Merge pull request #362 from adamretter/travis-jcheck 9383922cc Added java tests to travis build 90f156402 Fix CompactBetweenSnapshots 2717422e2 Merge pull request #361 from adamretter/fix-yosemite-build 3b5fe3a1f Correct the log message in VersionEdit c584d2b53 Fix for building RocksDB Java on Mac OS X Yosemite 2a8e5203d db_bench: --batch_size used for write benchmarks too d755e53b8 Printing number of keys in DB Stats 839c376bd fix table_test 0fd985f42 Avoid reloading filter on Get() if cache_index_and_filter_blocks == false e11a5e776 Improve the comment of util/thread_local.h 6398e6a6a Fix DeleteFile() + enable deleting files oldest files in level 0 0d3145198 Merge pull request #231 from adamretter/master a6fb7f312 Fix code review comments raised in https://reviews.facebook.net/D22779 c63494fb6 Tests for ComparatorOptions, Comparator and DirectComparator, and by proxy we also exercise Slice and DirectSlice 5e2527411 Fix code style problems identified by lint 25641bfc9 Fix to memory dealocation when creating a slice from a byte buffer fc12cb83f Add locking to comparator jni callback methods which must be thread-safe d6fe8dacc Feature - Implement Java API for Comparator and Slice. Allows use of either byte[] or DirectByteBuffer for accessing underlying data. 700f6ec3f Ignore IntelliJ idea project files and ignore java/out folder 5bfb7f5d0 db_bench: seekrandom can specify --seek_nexts to read specific keys after seek. ff8f74c20 remove checking lower bound of level size 2dd9bfe3a Sanitize block-based table index type and check prefix_extractor dbcfe27d6 Merge branch 'master' of https://github.com/facebook/rocksdb 6c6691864 Speed up DB::Open() and Version creation by limiting the number of FileMetaData initialization. 5db9e7664 Fix Mac compile error: C++11 forbids default arguments for lambda expressions f4363fb81 Fix DynamicMemtableOptions test 8f01bf80c Merge pull request #353 from fyrz/FixBloomfilterRocksJava ee80fb4b4 Total memtables size counter c12f571d3 Fix mac compile, second try d2e60f5ce Fix mac compile 274dc81c9 fix build failure d6c8dba72 Log MutableCFOptions in SetOptions 4d5708aa5 dynamic soft_rate_limit and hard_rate_limit 065a67c4f dynamic disable_auto_compactions dc50a1a59 make max_write_buffer_number dynamic 6a150c011 ldb: support --fix_prefix_len bafbc23ba Filters getting disposed by System.gc before EOL ca250d71a Move logging out of mutex 5cc9adf5b WriteBatchWithIndex's Iterator bug of SeekToFirst() and SeekToLast() 7658bcc1e Merge pull request #352 from fyrz/OptionsTestMergeProblem 1b97934a2 Options correction 2ef3ed86f Integrated feedback from ankgup87 a40ce219b Adding merge functions to RocksDBJava cc6c883f5 Stop stopping writes on bg_error_ 4942f6efb Merge pull request #350 from fyrz/reenable_accidentally_disabled_test ee28f431d With the last commit a Test was accidentally disabled. This commit solves this. b5dd7eed6 Merge pull request #319 from fyrz/column_families_java 18004d2f2 [RocksJava] Column family support 5908d08a0 Merge pull request #336 from fyrz/32BitRocksJavaBug 9e5f2a952 Merge pull request #344 from fyrz/java-doc-enhancements 4f5a68725 32-Bit RocksJava resolution for jlong overflows 16d2ebdbc Minor adjustment to prevent two warnings 70294c911 JavaDoc improvements on RocksJava 833357402 WriteBatchWithIndex supports an iterator that merge its change with a base iterator. 4f65fbd19 WriteBatchWithIndex's iterator to support SeekToFirst(), SeekToLast() and Prev() f441b273a WriteBatchWithIndex to support an option to overwrite rows when operating the same key 3ead857a0 Fixed Mac compile error in util/options_test.cc 5a7618634 Fixed compile error on Mac: default arguments for lambda expressions cd0d581ff convert Options from string f18b4a484 minor update to benchmark script b7d3d6ebc db_bench: set thread pool size according to max_background_flushes c5f54a8f3 Merge pull request #339 from fyrz/bloomFilterSupportNewFormat ced612957 Improved JavaDoc 5e43155b3 RocksJava should support not only BlockBased Bloomfilter 1d525891b Update HISTORY for 3.6 88edfd90a SkipListRep::LookaheadIterator 6a443309d Merge pull request #342 from fyrz/java_makefile_fix 4f272408c RocksJava Makefile includes incorrect paths to version.h f78b832e5 Log RocksDB version 25f6a852e add db_test for changing memtable size daab6dc51 Merge pull request #318 from criccomini/master 63eade401 Fix error introduced by merge ba882972f Merge pull request #340 from nbougalis/nullderef 6bcff9dc2 Merge pull request #341 from fyrz/arc-lint d6169954b Removed code which prevents `arc lint` from working properly. b87db0715 Avoid dereferencing a null field 1a1b95347 Merge pull request #270 from tdfischer/check-with-unity e107b6b2a Merge pull request #337 from fyrz/cross-platform-jdb_bench 70e401a2e Merge pull request #338 from fyrz/rm_obs_code_write_batch_with_index_test 1e5a52815 update release readme d44871e80 fix java doc directory in git ignore 2a4d6e796 merge master to resolve merge conflicts 6b2c1d962 make publish jni jars depend on release jni jars 1c7c76476 Replaced obsolete comparator with builtin variant. 22c64be43 Cross platform fix for Java benchmark shell script. 81828592b Merge pull request #335 from fyrz/version-script-cross-platform 69d4c5123 Cross-platform fix version.sh fcd13a77a Merge pull request #334 from fyrz/JavaDoc-Cleanup bfb0246f5 Merge pull request #331 from fyrz/findbug_issues 05204bb11 Lint changes da8ff9ff8 Fixed Findbugs issues a5757ff3c Listing of changes df3373fbf [Java] Fix compile error on DbBenchmark.java 4eb5a40f7 [Java] Fixed link error on library loading on Mac. 56dfd363f Fix a check in database shutdown or Column family drop during flush. 0e516a75d Fix lint errors in java/rocksjni/options.cc 8ea232b9e Add number of records dropped in compaction summary f4086a88b perf_context.get_from_output_files_time is set for MultiGet() and ReadOnly DB too. e869fc6a8 remove proper javadoc directory c1273533b Merge pull request #333 from nbougalis/cleanups 99744e0c4 bump version to 3.6 45d526e22 singular javadoc directory 378f321da merge master to resolve merge conflicts a1d3f0d2b don't fail if javadocs diretory doesn't exist c832f1644 add not about updating pom version and rename pom to be unversioned a213971d8 Don't return (or dereference) dangling pointer 2a1add673 use proper major/minor/micro version rather than hard coding 3.5.0 2d72f7807 update release docs in java 8322cf000 use javadoc instead of javadocs 079a612b5 Fix build_tools/version.sh 2e8012498 add javadoc and sources targets for sonatype df08a2d03 add single rocksdbjni pom deefcf476 make fat jar unclassified to satisfy sonatype fd2545c80 add maven publication target and instructions 1e9af10ef Merge pull request #330 from fyrz/setFilterPolicyForBBTConfig 017354177 FilterTest d410b39d5 BlockBasedTableConfig Filter policy support RocksJava 0908ddcea Don't keep managing two rocksdb version d0916f452 add major minor micro version to java jars d6987216c Merge pull request #327 from dalgaaf/wip-da-SCA-20141001 25888ae00 Merge pull request #329 from fyrz/master 89833e5a8 Fixed signed-unsigned comparison warning in db_test.cc fcac705f9 Fixed compile warning on Mac caused by unused variables. b3343fdea resolution for java build problem introduced by 5ec53f3edf62bec1b690ce12fb21a6c52203f3c8 187b29938 ForwardIterator: update prev_key_ only if prefix hasn't changed 5ec53f3ed make compaction related options changeable d122e7bcf Update INSTALL.md 9d6f38086 backupable_db_test.cc: pass const string param by reference 8ff0b4095 document_db_test.cc: pass const string param by reference 177caca42 ttl/ttl_test.cc: pass const string param by reference 4a171882d db/version_set.cc: remove unnecessary checks bf3bfd044 util/cache_test.cc: use static_cast over C-Style cast 86e29f033 document_db.cc: remove unused variable 28a6e3158 table/block_based_table_builder.cc: remove unused variable 091153493 db/db_test.cc: remove unused variable 6b6cedbb1 table/format.cc: reduce scope of some variables 55652043c table/cuckoo_table_reader.cc: pass func parameter by reference 5abd8add7 db/deletefile_test.cc: remove unused variable d6483af87 db/db_test.cc: reduce scope of some variables 44cca0cd8 db/db_iter.cc: remove unused variable 986dad025 Merge pull request #324 from dalgaaf/wip-da-SCA-20140930 3a0d498a3 rebase master 8ee75dca2 db/memtable.cc: remove unused variable merge_result 0fd8bbca5 db/db_impl.cc: reduce scope of prefix_initialized 676ff7b1f compaction_picker.cc: remove check for >=0 for unsigned e55aea551 document_db.cc: fix assert d517c8364 in_table_factory.cc: use correct format specifier b14037556 ttl/ttl_test.cc: prefer prefix ++operator for non-primitive types 43c789c8f spatialdb/spatial_db.cc: use !empty() instead of 'size() > 0' 0de452ee9 document_db.cc: pass const parameter by reference 4cc8643ba util/ldb_cmd.cc: prefer prefix ++operator for non-primitive types af8c2b2d9 util/signal_test.cc: suppress intentional null pointer deref 33580fa39 db/db_impl.cc: fix object handling, remove double lines 873f1356a db_ttl_impl.h: pass func parameter by reference 855845714 ldb_cmd_execute_result.h: perform init in initialization list 063471bf7 table/table_test.cc: pass func parameter by reference 93548ce8f table/cuckoo_table_reader.cc: pass func parameter by ref b8b7117e9 db/version_set.cc: use !empty() instead of 'size() > 0' 8ce050b51 table/bloom_block.*: pass func parameter by reference 53910ddb1 db_test.cc: pass parameter by reference 68ca53416 corruption_test.cc: pass parameter by reference 726ac5bca shrink vagrant commands to single line a2f98ef61 fix tabs in Makefile 7506198da cuckoo_table_db_test.cc: add flush after delete 1f963305a Print MB per second compaction throughput separately for reads and writes ffe3d490d Add an instruction about SSE in INSTALL.md 0b923f0f9 add centos 5.6 build instead of ubuntu. ee1f3ccb0 Package generation for Ubuntu and CentOS f0f795549 Fixing comile errors on OS X 99fb613e5 remove 2 space linter b2d64a486 Fix linters, second try 747523d24 Print per column family metrics in db_bench 56ebd4087 Fix arc lint (should fix #238) 637f89179 Merge pull request #321 from eonnen/master 827e31c74 Make test use a compatible type in the size checks. fd5d80d55 CompactedDB: log using the correct info_log 2faf49d5f use GetContext to replace callback function pointer 6a64ea617 add note about java 7 983d2de2d Add AUTHORS file. Fix #203 c4519c777 fix mis-named jar in JNI loader abd70c5e1 Merge pull request #316 from fyrz/ReverseBytewiseComparator 2dc6f62bb handle kDelete type in cuckoo builder 8b8011a68 Changed name of ReverseBytewiseComparator based on review comment b8e26615a since we're not sharing folders with the vm, copy built .so files and jars back to host system. 389edb6b1 universal compaction picker: use double for potential overflow 4e735bb7f Rsync files to VM rather than sync folders, since sync folders was causing clock skew and confusig make. 82a8f43cc Document RELEASE.mdgit status 9db13987b Update RocksDB's Java bindings to support multiple native RocksDB builds in the same Jar file. Cross build RocksDB for linux32 and linux64 using Vagrant. Build a cross-platform fat jar that contains osx, linux32, and linux64 RocksDB static builds. 534048426 Built-in comparator(s) in RocksJava d439451fa delay initialization of cuckoo table iterator 94997eab5 reduce memory usage of cuckoo table builder c6275956e improve memory efficiency of cuckoo reader 581442d44 option to choose module when calculating CuckooTable hash fbd2dafc9 CompactedDBImpl::MultiGet() for better CuckooTable performance 3c6800610 CompactedDBImpl f7375f39f Fix double deletes 21ddcf6e4 Remove allow_thread_local fb4a492cc Merge pull request #311 from ankgup87/master 611e286b9 Merge branch 'master' of https://github.com/facebook/rocksdb 0103b4498 Merge branch 'master' of ssh://github.com/ankgup87/rocksdb 1dfb7bb98 Add block based table config options cdaf44f9a Enlarge log size cap when printing file summary 7cc1ed7f0 Merge pull request #309 from naveenatceg/staticbuild ba6d660f6 Resolving merge conflict 51eeaf65e Addressing review comments fd7d3fe60 Addressing review comments (adding a env variable to override temp directory) cf7ace886 Addressing review comments 0a29ce539 re-enable BlockBasedTable::SetupForCompaction() 55af37075 Remove TODO for checking index checksums 3d74f0997 Fix compile 53b003995 Fix release compile d0de413f4 WriteBatchWithIndex to allow different Comparators for different column families 57a32f147 change target_file_size_base to uint64_t 5e6aee432 dont create backup_input if compaction filter v2 is not used 49b5f94c5 Merge pull request #306 from Liuchang0812/fix_cast 787cb4db2 remove cast, replace %llu with % PRIu64 a7574d4fa Update logging.cc 7e0dcb953 Update logging.cc 57fa3cc5b Merge pull request #304 from Liuchang0812/fix-check cd44522a9 Merge pull request #305 from Liuchang0812/fix-logging 6a031b6a8 remove unused variable 4436f17bd fixed #303: replace %ld with % PRId64 7a1bd057f Merge pull request #302 from ankgup87/master 423e52cd4 Merge branch 'master' of https://github.com/facebook/rocksdb bfeef94d3 Add rate limiter ed9a2df8c fix unity build 32f2532a0 Print compression_size_percent as a signed int 976caca09 Skip AllocateTest if fallocate() is not supported in the file system 3b897cddd Enable no-fbcode RocksDB build f44594743 RocksDB: Format uint64 using PRIu64 in db_impl.cc e17bc65c7 Merge pull request #299 from ankgup87/master b93797abc Fix build adae3ca1f [Java] Fix JNI link error caused by the removal of options.db_stats_log_interval 90b8c07b4 Fix unit tests errors 51af7c326 CuckooTable: add one option to allow identity function for the first hash function 035043559 Fixed a signed-unsigned comparison in spatial_db.cc -- issue #293 2fb1fea30 Fix syncronization issues 6f0964e37 Only run make unity on travis instead of make check 9ed1b49a2 Build unity build on make check ff7689561 Remove some unnecessary constructors feadb9df5 fix cuckoo table builder test 3c232e164 Fix mac compile 54cada92b Run make format on PR #249 27b22f13a Merge pull request #249 from tdfischer/decompression-refactoring fb6456b00 Replace naked calls to operator new and delete (Fixes #222) 5600c8f6e cuckoo table: return estimated size - 1 a062e1f2c SetOptions() for memtable related options e4eca6a1e Options conversion function for convenience a7c209452 Merge pull request #292 from saghmrossi/master 4d0523452 Merge branch 'master' of github.com:saghmrossi/rocksdb 60a4aa175 Test use_mmap_reads 94e43a1df [Java] Fixed 32-bit overflowing issue when converting jlong to size_t f9eaaa66e added include for inttypes.h to fix nonworking printf statements f090575e4 Replaced "built on on earlier work" by "built on earlier work" in README.md faad439ac Fix #284 49aacd8d2 Fix make install acb9348ff [Java] Include WriteBatch into RocksDBSample.java, fix how DbBenchmark.java handles WriteBatch. 4a27a2f19 Don't sync manifest when disableDataSync = true 9b8480d93 Merge pull request #287 from yinqiwen/rate-limiter-crash-fix 28be16b1d fix rate limiter crash #286 04ce1b25f Fix #284 add22e351 standardize scripts to run RocksDB benchmarks dee91c259 WriteThread 540a257f2 Fix WAL synced 24f034bf4 Merge pull request #282 from Chilledheart/develop 49fe329e5 Fix build issue under macosx ebb5c65e6 Add make install 0352a9fa9 add_wrapped_bloom_test 9c0e66ce9 Don't run background jobs (flush, compactions) when bg_error_ is set a9639bda8 Fix valgrind test d1f24dc7e Relax FlushSchedule test 3d9e6f775 Push model for flushing memtables 059e584dd [unit test] CompactRange should fail if we don't have space dd641b211 fix RocksDB java build 53404d9fb add_qps_info_in cache bench a52cecb56 Fix Mac compile 092f97e21 Fix comments and typos 6cc12860f Added a few statistics for BackupableDB 0a42295a2 Fix SimpleWriteTimeoutTest 06d986252 Always pass MergeContext as pointer, not reference d343c3fe4 Improve db recovery 6bb7e3ef2 Merger test 88841bd00 Explicitly cast char to signed char in Hash() 52311463e MemTableOptions 1d284db21 Addressing review comments 55114e7f4 Some updates for SpatialDB 171d4ff4a remove TailingIterator reference in db_impl.h 9b0f7ffa1 rename version_set options_ to db_options_ to avoid confusion 2d57828d0 Check stop level trigger-0 before slowdown level-0 trigger 659d2d50c move compaction_filter to immutable_options 048560a64 reduce references to cfd->options() in DBImpl 011241bb9 DB::Flush() Do not wait for background threads when there is nothing in mem table a2bb7c3c3 Push- instead of pull-model for managing Write stalls 0af157f9b Implement full filter for block based table. 9360cc690 Fix valgrind issue 02d5bff39 Merge pull request #277 from wankai/master 88a2f44f9 fix comments 7c16e3922 Merge pull request #276 from wankai/master 823773837 replace hard-coded number with named variable db8ca520b Merge pull request #273 from nbougalis/static-analysis b7b031f42 Merge pull request #274 from wankai/master 4c2b1f097 Merge remote-tracking branch 'upstream/master' a5d286307 typo improvement 9f8aa0939 Don't leak data returned by opendir d1cfb71ec Remove unused member(s) bfee319fb sizeof(int*) where sizeof(int) was intended d40c1f742 Add missing break statement 2e97c3898 Avoid off-by-one error when using readlink 40ddc3d6c add cache bench 9f1c80b55 Drop column family from write thread 8de151bb9 Add db_bench with lots of column families to regression tests c9e419ccb rename options_ to db_options_ in DBImpl to avoid confusion 5cd0576ff Fix compaction bug in Cuckoo Table Builder. Use kvs_.size() instead of num_entries in FileSize() method. 0fbb3facc fixed memory leak in unit test DBIteratorBoundTest adcd2532c fix asan check 4092b7a0b Merge pull request #272 from project-zerus/patch-1 bb6ae0f80 fix more compile warnings 6d3144118 Merge pull request #271 from nbougalis/cleanups 0cd0ec4fe Plug memory leak during index creation 4329d74e0 Fix swapped variable names to accurately reflect usage 45a5e3ede Remove path with arena==nullptr from NewInternalIterator 5665e5e28 introduce ImmutableOptions e0b99d4f5 created a new ReadOptions parameter 'iterate_upper_bound' 51ea88900 Fix travis builds a4816269f Relax backupable rate limiting test f7f973d35 Merge pull request #269 from huahang/patch-2 ef5b38472 fix a few compile warnings 2fd3806c8 Merge pull request #263 from wankai/master 1785114a6 delete unused Comparator 1b1d9619f update HISTORY.md 703c3eacd comments about the BlockBasedTableOptions migration in Options 4b5ad8865 Merge pull request #260 from wankai/master 19cc588b7 change to filter_block std::unique_ptr support RAII 9b976e34f Merge pull request #259 from wankai/master 5d25a4693 Merge remote-tracking branch 'upstream/master' dff2b1a8f typo improvement 343e98a7d Reverting import change ddb8039e3 RocksDB static build Make file changes to download and build the dependencies .Load the shared library when RocksDB is initialized REVERT: 1fdd726a8 Hotfix RocksDB 3.5 REVERT: d67500a59 Add `make install` to Makefile in 3.5.fb. REVERT: 4cb631aa0 update HISTORY.md REVERT: cfd0946be comments about the BlockBasedTableOptions migration in Options git-subtree-dir: src/rocksdb2 git-subtree-split: aead4041720f40e305e7f9d432875749d91d5a2d --- .arcconfig | 10 - .gitignore | 45 +- .travis.yml | 88 +- AUTHORS | 11 + CMakeLists.txt | 927 + CONTRIBUTING.md | 5 - COPYING | 339 + DEFAULT_OPTIONS_HISTORY.md | 24 + DUMP_FORMAT.md | 16 + HISTORY.md | 454 +- INSTALL.md | 122 +- LANGUAGE-BINDINGS.md | 16 + LICENSE.Apache | 202 + LICENSE => LICENSE.leveldb | 8 +- Makefile | 1639 +- PATENTS | 23 - README.md | 4 +- ROCKSDB_LITE.md | 1 + TARGETS | 534 + USERS.md | 85 + Vagrantfile | 34 + WINDOWS_PORT.md | 228 + appveyor.yml | 15 + buckifier/buckify_rocksdb.py | 172 + buckifier/rocks_test_runner.sh | 4 + buckifier/targets_builder.py | 65 + buckifier/targets_cfg.py | 124 + buckifier/util.py | 107 + build_tools/RocksDBCommonHelper.php | 377 + build_tools/amalgamate.py | 110 + build_tools/build_detect_platform | 361 +- build_tools/build_detect_version | 22 - build_tools/cont_integration.sh | 135 + build_tools/dependencies.sh | 18 + build_tools/dependencies_4.8.1.sh | 18 + build_tools/dockerbuild.sh | 2 + build_tools/error_filter.py | 167 + build_tools/fb_compile_mongo.sh | 55 + build_tools/fbcode.clang31.sh | 74 - build_tools/fbcode.gcc471.sh | 70 - build_tools/fbcode.gcc481.sh | 86 - build_tools/fbcode_config.sh | 156 + build_tools/fbcode_config4.8.1.sh | 115 + build_tools/format-diff.sh | 31 +- build_tools/gnu_parallel | 7936 ++++++ build_tools/mac-install-gflags.sh | 25 - build_tools/make_new_version.sh | 46 - build_tools/make_package.sh | 128 + build_tools/precommit_checker.py | 208 + build_tools/regression_build_test.sh | 53 +- build_tools/rocksdb-lego-determinator | 782 + build_tools/run_ci_db_test.ps1 | 456 + build_tools/unity | 78 - build_tools/update_dependencies.sh | 131 + build_tools/valgrind_test.sh | 15 - build_tools/version.sh | 22 + cache/cache_bench.cc | 284 + cache/cache_test.cc | 703 + cache/clock_cache.cc | 729 + cache/clock_cache.h | 16 + cache/lru_cache.cc | 530 + cache/lru_cache.h | 302 + cache/lru_cache_test.cc | 172 + cache/sharded_cache.cc | 161 + cache/sharded_cache.h | 102 + cmake/RocksDBConfig.cmake.in | 3 + cmake/modules/FindJeMalloc.cmake | 21 + cmake/modules/Findbzip2.cmake | 21 + cmake/modules/Findlz4.cmake | 21 + cmake/modules/Findsnappy.cmake | 21 + cmake/modules/Findzlib.cmake | 21 + cmake/modules/Findzstd.cmake | 21 + coverage/coverage_test.sh | 6 +- db/builder.cc | 337 +- db/builder.h | 72 +- db/c.cc | 2314 +- db/c_test.c | 992 +- db/column_family.cc | 857 +- db/column_family.h | 365 +- db/column_family_test.cc | 2662 +- db/compact_files_test.cc | 328 + db/compacted_db_impl.cc | 166 + db/compacted_db_impl.h | 102 + db/compaction.cc | 489 +- db/compaction.h | 256 +- db/compaction_iteration_stats.h | 35 + db/compaction_iterator.cc | 588 + db/compaction_iterator.h | 197 + db/compaction_iterator_test.cc | 568 + db/compaction_job.cc | 1471 ++ db/compaction_job.h | 165 + db/compaction_job_stats_test.cc | 1047 + db/compaction_job_test.cc | 949 + db/compaction_picker.cc | 2064 +- db/compaction_picker.h | 326 +- db/compaction_picker_test.cc | 1441 ++ db/compaction_picker_universal.cc | 748 + db/compaction_picker_universal.h | 91 + db/comparator_db_test.cc | 441 + db/convenience.cc | 58 + db/corruption_test.cc | 172 +- db/cuckoo_table_db_test.cc | 84 +- db/db_basic_test.cc | 856 + db/db_bench.cc | 2803 --- db/db_blob_index_test.cc | 409 + db/db_block_cache_test.cc | 579 + db/db_bloom_filter_test.cc | 1108 + db/db_compaction_filter_test.cc | 845 + db/db_compaction_test.cc | 2835 +++ db/db_dynamic_level_test.cc | 506 + db/db_encryption_test.cc | 94 + db/db_filesnapshot.cc | 120 +- db/db_flush_test.cc | 162 + db/db_impl.cc | 6357 ++--- db/db_impl.h | 1316 +- db/db_impl_compaction_flush.cc | 1910 ++ db/db_impl_debug.cc | 164 +- db/db_impl_experimental.cc | 152 + db/db_impl_files.cc | 548 + db/db_impl_open.cc | 1129 + db/db_impl_readonly.cc | 136 +- db/db_impl_readonly.h | 66 +- db/db_impl_write.cc | 1263 + .../db_info_dumper.cc | 58 +- db/db_info_dumper.h | 14 + db/db_inplace_update_test.cc | 177 + db/db_io_failure_test.cc | 568 + db/db_iter.cc | 1040 +- db/db_iter.h | 70 +- db/db_iter_test.cc | 2788 ++- db/db_iterator_test.cc | 1988 ++ db/db_log_iter_test.cc | 294 + db/db_memtable_test.cc | 195 + db/db_merge_operator_test.cc | 367 + db/db_options_test.cc | 452 + db/db_properties_test.cc | 1393 ++ db/db_range_del_test.cc | 1007 + db/db_sst_test.cc | 849 + db/db_statistics_test.cc | 149 + db/db_table_properties_test.cc | 261 + db/db_tailing_iter_test.cc | 814 + db/db_test.cc | 11614 ++++----- db/db_test2.cc | 2340 ++ db/db_test_util.cc | 1395 ++ db/db_test_util.h | 970 + db/db_universal_compaction_test.cc | 1589 ++ db/db_wal_test.cc | 1229 + db/db_write_test.cc | 129 + db/dbformat.cc | 58 +- db/dbformat.h | 339 +- db/dbformat_test.cc | 82 +- db/deletefile_test.cc | 272 +- db/event_helpers.cc | 155 + db/event_helpers.h | 52 + db/experimental.cc | 49 + db/external_sst_file_basic_test.cc | 617 + db/external_sst_file_ingestion_job.cc | 665 + db/external_sst_file_ingestion_job.h | 171 + db/external_sst_file_test.cc | 1961 ++ db/fault_injection_test.cc | 547 + db/file_indexer.cc | 48 +- db/file_indexer.h | 32 +- db/file_indexer_test.cc | 50 +- db/filename_test.cc | 21 +- db/flush_job.cc | 362 + db/flush_job.h | 110 + db/flush_job_test.cc | 223 + db/flush_scheduler.cc | 88 + db/flush_scheduler.h | 48 + db/forward_iterator.cc | 721 +- db/forward_iterator.h | 85 +- db/forward_iterator_bench.cc | 375 + db/internal_stats.cc | 1422 +- db/internal_stats.h | 555 +- db/job_context.h | 129 + db/listener_test.cc | 895 + db/log_and_apply_bench.cc | 84 - db/log_format.h | 26 +- db/log_reader.cc | 219 +- db/log_reader.h | 54 +- db/log_test.cc | 394 +- db/log_writer.cc | 88 +- db/log_writer.h | 80 +- db/malloc_stats.cc | 52 + db/malloc_stats.h | 22 + db/managed_iterator.cc | 262 + db/managed_iterator.h | 85 + {util => db}/manual_compaction_test.cc | 31 +- db/memtable.cc | 631 +- db/memtable.h | 316 +- db/memtable_list.cc | 404 +- db/memtable_list.h | 181 +- db/memtable_list_test.cc | 615 + db/merge_context.h | 111 +- db/merge_helper.cc | 414 +- db/merge_helper.h | 167 +- db/merge_helper_test.cc | 290 + db/merge_operator.cc | 45 +- db/merge_test.cc | 210 +- db/options_file_test.cc | 119 + db/perf_context_test.cc | 577 +- db/pinned_iterators_manager.h | 87 + db/plain_table_db_test.cc | 289 +- db/prefix_test.cc | 507 +- db/range_del_aggregator.cc | 520 + db/range_del_aggregator.h | 161 + db/range_del_aggregator_test.cc | 157 + db/repair.cc | 524 +- db/repair_test.cc | 328 + db/simple_table_db_test.cc | 810 - db/snapshot.h | 86 - db/snapshot_impl.cc | 26 + db/snapshot_impl.h | 151 + db/table_cache.cc | 419 +- db/table_cache.h | 87 +- db/table_properties_collector.cc | 97 +- db/table_properties_collector.h | 68 +- db/table_properties_collector_test.cc | 509 +- db/transaction_log_impl.cc | 82 +- db/transaction_log_impl.h | 74 +- db/version_builder.cc | 467 + db/version_builder.h | 45 + db/version_builder_test.cc | 305 + db/version_edit.cc | 254 +- db/version_edit.h | 123 +- db/version_edit_test.cc | 145 +- db/version_set.cc | 3200 ++- db/version_set.h | 786 +- db/version_set_test.cc | 235 +- db/wal_manager.cc | 476 + db/wal_manager.h | 95 + db/wal_manager_test.cc | 310 + db/write_batch.cc | 1217 +- db/write_batch_base.cc | 94 + db/write_batch_internal.h | 174 +- db/write_batch_test.cc | 690 +- db/write_callback.h | 27 + db/write_callback_test.cc | 397 + db/write_controller.cc | 128 + db/write_controller.h | 144 + db/write_controller_test.cc | 135 + db/write_thread.cc | 661 + db/write_thread.h | 391 + doc/doc.css | 89 - doc/index.html | 827 - doc/log_format.txt | 75 - doc/rockslogo.jpg | Bin 137232 -> 0 bytes doc/rockslogo.png | Bin 61703 -> 0 bytes docs/.gitignore | 9 + docs/CNAME | 1 + docs/CONTRIBUTING.md | 115 + docs/Gemfile | 2 + docs/Gemfile.lock | 145 + docs/LICENSE-DOCUMENTATION | 385 + docs/README.md | 80 + docs/TEMPLATE-INFORMATION.md | 17 + docs/_config.yml | 85 + docs/_data/authors.yml | 62 + docs/_data/features.yml | 19 + docs/_data/nav.yml | 30 + docs/_data/nav_docs.yml | 3 + docs/_data/powered_by.yml | 1 + docs/_data/powered_by_highlight.yml | 1 + docs/_data/promo.yml | 6 + docs/_docs/faq.md | 48 + docs/_docs/getting-started.md | 78 + docs/_includes/blog_pagination.html | 28 + docs/_includes/content/gridblocks.html | 5 + docs/_includes/content/items/gridblock.html | 37 + docs/_includes/doc.html | 25 + docs/_includes/doc_paging.html | 0 docs/_includes/footer.html | 33 + docs/_includes/head.html | 23 + docs/_includes/header.html | 19 + docs/_includes/hero.html | 0 docs/_includes/home_header.html | 22 + docs/_includes/katex_import.html | 3 + docs/_includes/katex_render.html | 210 + docs/_includes/nav.html | 37 + docs/_includes/nav/collection_nav.html | 64 + docs/_includes/nav/collection_nav_group.html | 19 + .../nav/collection_nav_group_item.html | 1 + docs/_includes/nav/header_nav.html | 30 + docs/_includes/nav_search.html | 15 + docs/_includes/plugins/all_share.html | 3 + docs/_includes/plugins/ascii_cinema.html | 2 + docs/_includes/plugins/button.html | 6 + docs/_includes/plugins/github_star.html | 4 + docs/_includes/plugins/github_watch.html | 4 + docs/_includes/plugins/google_share.html | 5 + docs/_includes/plugins/iframe.html | 6 + docs/_includes/plugins/like_button.html | 18 + docs/_includes/plugins/plugin_row.html | 5 + .../plugins/post_social_plugins.html | 34 + docs/_includes/plugins/slideshow.html | 88 + docs/_includes/plugins/twitter_follow.html | 5 + docs/_includes/plugins/twitter_share.html | 4 + docs/_includes/post.html | 35 + docs/_includes/powered_by.html | 28 + docs/_includes/social_plugins.html | 24 + docs/_includes/ui/button.html | 1 + docs/_layouts/basic.html | 12 + docs/_layouts/blog.html | 11 + docs/_layouts/blog_default.html | 14 + docs/_layouts/default.html | 12 + docs/_layouts/doc_default.html | 14 + docs/_layouts/doc_page.html | 10 + docs/_layouts/docs.html | 5 + docs/_layouts/home.html | 17 + docs/_layouts/page.html | 3 + docs/_layouts/plain.html | 10 + docs/_layouts/post.html | 8 + docs/_layouts/redirect.html | 6 + docs/_layouts/top-level.html | 10 + .../2014-03-27-how-to-backup-rocksdb.markdown | 135 + ...ersist-in-memory-rocksdb-database.markdown | 54 + ...ocal-meetup-held-on-march-27-2014.markdown | 53 + .../2014-04-07-rocksdb-2-8-release.markdown | 40 + ...les-for-better-lookup-performance.markdown | 28 + docs/_posts/2014-05-14-lock.markdown | 88 + .../2014-05-19-rocksdb-3-0-release.markdown | 24 + .../2014-05-22-rocksdb-3-1-release.markdown | 20 + ...6-23-plaintable-a-new-file-format.markdown | 47 + ...6-27-avoid-expensive-locks-in-get.markdown | 89 + .../2014-06-27-rocksdb-3-2-release.markdown | 30 + .../2014-07-29-rocksdb-3-3-release.markdown | 34 + docs/_posts/2014-09-12-cuckoo.markdown | 74 + ...014-09-12-new-bloom-filter-format.markdown | 52 + .../2014-09-15-rocksdb-3-5-release.markdown | 38 + ...grating-from-leveldb-to-rocksdb-2.markdown | 112 + ...ading-rocksdb-options-from-a-file.markdown | 41 + ...2015-02-27-write-batch-with-index.markdown | 20 + ...ntegrating-rocksdb-with-mongodb-2.markdown | 16 + .../2015-06-12-rocksdb-in-osquery.markdown | 10 + ...015-07-15-rocksdb-2015-h2-roadmap.markdown | 92 + ...07-17-spatial-indexing-in-rocksdb.markdown | 78 + ...now-available-in-windows-platform.markdown | 30 + docs/_posts/2015-07-23-dynamic-level.markdown | 29 + docs/_posts/2015-10-27-getthreadlist.markdown | 193 + ...eckpoints-for-efficient-snapshots.markdown | 45 + ...alysis-file-read-latency-by-level.markdown | 244 + .../_posts/2016-01-29-compaction_pri.markdown | 51 + .../2016-02-24-rocksdb-4-2-release.markdown | 41 + docs/_posts/2016-02-25-rocksdb-ama.markdown | 20 + .../2016-03-07-rocksdb-options-file.markdown | 24 + ...2016-04-26-rocksdb-4-5-1-released.markdown | 60 + .../2016-07-26-rocksdb-4-8-released.markdown | 48 + ...016-09-28-rocksdb-4-11-2-released.markdown | 49 + ...2017-01-06-rocksdb-5-0-1-released.markdown | 26 + ...2017-02-07-rocksdb-5-1-2-released.markdown | 15 + ...017-02-17-bulkoad-ingest-sst-file.markdown | 50 + ...2017-03-02-rocksdb-5-2-1-released.markdown | 22 + ...17-05-12-partitioned-index-filter.markdown | 34 + .../2017-05-14-core-local-stats.markdown | 106 + ...2017-05-26-rocksdb-5-4-5-released.markdown | 39 + ...2017-06-26-17-level-based-changes.markdown | 60 + ...2017-06-29-rocksdb-5-5-1-released.markdown | 22 + ...2017-07-25-rocksdb-5-6-1-released.markdown | 22 + docs/_posts/2017-08-24-pinnableslice.markdown | 37 + docs/_posts/2017-08-25-flushwal.markdown | 26 + docs/_sass/_base.scss | 492 + docs/_sass/_blog.scss | 45 + docs/_sass/_buttons.scss | 47 + docs/_sass/_footer.scss | 82 + docs/_sass/_gridBlock.scss | 115 + docs/_sass/_header.scss | 138 + docs/_sass/_poweredby.scss | 69 + docs/_sass/_promo.scss | 55 + docs/_sass/_react_docs_nav.scss | 332 + docs/_sass/_react_header_nav.scss | 141 + docs/_sass/_reset.scss | 43 + docs/_sass/_search.scss | 142 + docs/_sass/_slideshow.scss | 48 + docs/_sass/_syntax-highlighting.scss | 129 + docs/_sass/_tables.scss | 47 + docs/_top-level/support.md | 22 + docs/blog/all.html | 20 + docs/blog/index.html | 12 + docs/css/main.scss | 149 + .../2016-04-07-blog-post-example.md | 21 + docs/doc-type-examples/docs-hello-world.md | 12 + docs/doc-type-examples/top-level-example.md | 8 + docs/docs/index.html | 6 + docs/feed.xml | 30 + docs/index.md | 9 + docs/static/favicon.png | Bin 0 -> 3927 bytes docs/static/fonts/LatoLatin-Black.woff | Bin 0 -> 70460 bytes docs/static/fonts/LatoLatin-Black.woff2 | Bin 0 -> 43456 bytes docs/static/fonts/LatoLatin-BlackItalic.woff | Bin 0 -> 72372 bytes docs/static/fonts/LatoLatin-BlackItalic.woff2 | Bin 0 -> 44316 bytes docs/static/fonts/LatoLatin-Italic.woff | Bin 0 -> 74708 bytes docs/static/fonts/LatoLatin-Italic.woff2 | Bin 0 -> 45388 bytes docs/static/fonts/LatoLatin-Light.woff | Bin 0 -> 72604 bytes docs/static/fonts/LatoLatin-Light.woff2 | Bin 0 -> 43468 bytes docs/static/fonts/LatoLatin-Regular.woff | Bin 0 -> 72456 bytes docs/static/fonts/LatoLatin-Regular.woff2 | Bin 0 -> 43760 bytes .../Resize-of-20140327_200754-300x225.jpg | Bin 0 -> 26670 bytes docs/static/images/compaction/full-range.png | Bin 0 -> 193353 bytes .../images/compaction/l0-l1-contend.png | Bin 0 -> 203828 bytes .../images/compaction/l1-l2-contend.png | Bin 0 -> 230195 bytes .../images/compaction/part-range-old.png | Bin 0 -> 165547 bytes docs/static/images/pcache-blockindex.jpg | Bin 0 -> 55324 bytes docs/static/images/pcache-fileindex.jpg | Bin 0 -> 54922 bytes docs/static/images/pcache-filelayout.jpg | Bin 0 -> 47197 bytes docs/static/images/pcache-readiopath.jpg | Bin 0 -> 16381 bytes docs/static/images/pcache-tieredstorage.jpg | Bin 0 -> 78208 bytes docs/static/images/pcache-writeiopath.jpg | Bin 0 -> 22616 bytes docs/static/images/promo-adapt.svg | 8 + docs/static/images/promo-flash.svg | 28 + docs/static/images/promo-operations.svg | 6 + docs/static/images/promo-performance.svg | 134 + docs/static/images/tree_example1.png | Bin 0 -> 17804 bytes docs/static/logo.svg | 76 + docs/static/og_image.png | Bin 0 -> 17639 bytes {util => env}/env.cc | 159 +- env/env_basic_test.cc | 356 + env/env_chroot.cc | 324 + env/env_chroot.h | 22 + env/env_encryption.cc | 909 + {util => env}/env_hdfs.cc | 147 +- env/env_posix.cc | 976 + env/env_test.cc | 1498 ++ env/io_posix.cc | 1028 + env/io_posix.h | 248 + env/mock_env.cc | 799 + env/mock_env.h | 115 + env/mock_env_test.cc | 85 + {util => env}/posix_logger.h | 54 +- examples/.gitignore | 6 + examples/Makefile | 49 +- examples/README.md | 3 +- examples/c_simple_example.c | 79 + examples/column_families_example.cc | 10 +- examples/compact_files_example.cc | 171 + examples/compaction_filter_example.cc | 87 + examples/optimistic_transaction_example.cc | 142 + examples/options_file_example.cc | 113 + examples/rocksdb_option_file_example.ini | 143 + examples/simple_example.cc | 54 +- examples/transaction_example.cc | 144 + hdfs/README | 6 +- hdfs/env_hdfs.h | 139 +- helpers/memenv/memenv.cc | 395 - include/rocksdb/advanced_options.h | 578 + include/rocksdb/c.h | 1718 +- include/rocksdb/cache.h | 156 +- include/rocksdb/cleanable.h | 82 + include/rocksdb/compaction_filter.h | 233 +- include/rocksdb/compaction_job_stats.h | 91 + include/rocksdb/comparator.h | 25 +- include/rocksdb/convenience.h | 339 + include/rocksdb/db.h | 837 +- include/rocksdb/db_bench_tool.h | 9 + include/rocksdb/db_dump_tool.h | 45 + include/rocksdb/env.h | 635 +- include/rocksdb/env_encryption.h | 196 + include/rocksdb/experimental.h | 29 + include/rocksdb/filter_policy.h | 85 +- include/rocksdb/flush_block_policy.h | 18 +- include/rocksdb/iostats_context.h | 42 +- include/rocksdb/iterator.h | 54 +- include/rocksdb/ldb_tool.h | 18 +- include/rocksdb/listener.h | 391 + include/rocksdb/memtablerep.h | 106 +- include/rocksdb/merge_operator.h | 100 +- include/rocksdb/metadata.h | 94 + include/rocksdb/options.h | 1246 +- include/rocksdb/perf_context.h | 166 +- include/rocksdb/perf_level.h | 33 + include/rocksdb/persistent_cache.h | 67 + include/rocksdb/rate_limiter.h | 101 +- include/rocksdb/slice.h | 141 +- include/rocksdb/slice_transform.h | 72 +- include/rocksdb/snapshot.h | 48 + include/rocksdb/sst_dump_tool.h | 17 + include/rocksdb/sst_file_manager.h | 85 + include/rocksdb/sst_file_writer.h | 115 + include/rocksdb/statistics.h | 275 +- include/rocksdb/status.h | 230 +- include/rocksdb/table.h | 248 +- include/rocksdb/table_properties.h | 186 +- include/rocksdb/thread_status.h | 193 + include/rocksdb/threadpool.h | 57 + include/rocksdb/transaction_log.h | 29 +- include/rocksdb/types.h | 8 +- include/rocksdb/universal_compaction.h | 18 +- include/rocksdb/utilities/backupable_db.h | 232 +- include/rocksdb/utilities/checkpoint.h | 43 + include/rocksdb/utilities/convenience.h | 10 + include/rocksdb/utilities/date_tiered_db.h | 108 + include/rocksdb/utilities/db_ttl.h | 8 +- include/rocksdb/utilities/debug.h | 41 + include/rocksdb/utilities/document_db.h | 8 +- include/rocksdb/utilities/env_librados.h | 176 + include/rocksdb/utilities/env_mirror.h | 176 + include/rocksdb/utilities/geo_db.h | 29 +- include/rocksdb/utilities/info_log_finder.h | 19 + include/rocksdb/utilities/json_document.h | 155 +- include/rocksdb/utilities/ldb_cmd.h | 260 + .../utilities}/ldb_cmd_execute_result.h | 27 +- include/rocksdb/utilities/leveldb_options.h | 144 + .../lua/rocks_lua_compaction_filter.h | 189 + .../utilities/lua/rocks_lua_custom_library.h | 43 + .../rocksdb/utilities/lua/rocks_lua_util.h | 55 + include/rocksdb/utilities/memory_util.h | 50 + include/rocksdb/utilities/object_registry.h | 90 + .../utilities/optimistic_transaction_db.h | 76 + .../utilities/option_change_migration.h | 19 + include/rocksdb/utilities/options_util.h | 93 + include/rocksdb/utilities/sim_cache.h | 89 + include/rocksdb/utilities/spatial_db.h | 61 +- include/rocksdb/utilities/stackable_db.h | 184 +- .../utilities/table_properties_collectors.h | 29 + include/rocksdb/utilities/transaction.h | 479 + include/rocksdb/utilities/transaction_db.h | 226 + .../rocksdb/utilities/transaction_db_mutex.h | 92 + include/rocksdb/utilities/utility_db.h | 7 +- .../utilities/write_batch_with_index.h | 205 +- include/rocksdb/version.h | 15 +- include/rocksdb/wal_filter.h | 101 + include/rocksdb/write_batch.h | 271 +- include/rocksdb/write_batch_base.h | 125 + include/rocksdb/write_buffer_manager.h | 100 + include/utilities/backupable_db.h | 12 - include/utilities/db_ttl.h | 8 - include/utilities/document_db.h | 8 - include/utilities/geo_db.h | 8 - include/utilities/json_document.h | 7 - include/utilities/stackable_db.h | 7 - include/utilities/utility_db.h | 7 - java/CMakeLists.txt | 207 + java/HISTORY-JAVA.md | 32 + java/Makefile | 245 +- java/RELEASE.md | 54 + java/RocksDBSample.java | 259 - .../org/rocksdb/benchmark/DbBenchmark.java | 364 +- java/crossbuild/Vagrantfile | 27 + java/crossbuild/build-linux-centos.sh | 32 + java/crossbuild/build-linux.sh | 14 + java/crossbuild/docker-build-linux-centos.sh | 11 + java/jdb_bench.sh | 11 +- java/org/rocksdb/BackupableDB.java | 90 - java/org/rocksdb/BackupableDBOptions.java | 73 - java/org/rocksdb/BlockBasedTableConfig.java | 210 - java/org/rocksdb/BloomFilter.java | 37 - java/org/rocksdb/CompactionStyle.java | 22 - java/org/rocksdb/CompressionType.java | 25 - .../rocksdb/HashLinkedListMemTableConfig.java | 52 - java/org/rocksdb/HistogramType.java | 39 - java/org/rocksdb/Options.java | 2240 -- java/org/rocksdb/PlainTableConfig.java | 123 - java/org/rocksdb/ReadOptions.java | 125 - java/org/rocksdb/RestoreBackupableDB.java | 84 - java/org/rocksdb/RestoreOptions.java | 37 - java/org/rocksdb/RocksDB.java | 370 - java/org/rocksdb/RocksDBException.java | 23 - java/org/rocksdb/RocksEnv.java | 102 - java/org/rocksdb/RocksIterator.java | 136 - java/org/rocksdb/RocksObject.java | 117 - java/org/rocksdb/SkipListMemTableConfig.java | 15 - java/org/rocksdb/Statistics.java | 38 - java/org/rocksdb/TickerType.java | 123 - java/org/rocksdb/WriteBatch.java | 112 - java/org/rocksdb/WriteBatchTest.java | 124 - java/org/rocksdb/WriteOptions.java | 99 - java/org/rocksdb/test/BackupableDBTest.java | 63 - java/org/rocksdb/test/OptionsTest.java | 388 - java/org/rocksdb/test/ReadOptionsTest.java | 40 - .../rocksdb/test/StatisticsCollectorTest.java | 43 - java/org/rocksdb/util/Environment.java | 37 - java/org/rocksdb/util/SizeUnit.java | 16 - java/rocksjni.pom | 150 + java/rocksjni/backupablejni.cc | 323 +- java/rocksjni/backupenginejni.cc | 236 + .../rocksjni/cassandra_compactionfilterjni.cc | 22 + java/rocksjni/cassandra_value_operator.cc | 45 + java/rocksjni/checkpoint.cc | 68 + java/rocksjni/clock_cache.cc | 40 + java/rocksjni/columnfamilyhandle.cc | 26 + java/rocksjni/compaction_filter.cc | 27 + java/rocksjni/compaction_options_fifo.cc | 55 + java/rocksjni/compaction_options_universal.cc | 194 + java/rocksjni/comparator.cc | 68 + java/rocksjni/comparatorjnicallback.cc | 366 + java/rocksjni/comparatorjnicallback.h | 94 + java/rocksjni/compression_options.cc | 121 + java/rocksjni/env.cc | 47 +- java/rocksjni/env_options.cc | 297 + java/rocksjni/filter.cc | 29 +- java/rocksjni/ingest_external_file_options.cc | 149 + java/rocksjni/iterator.cc | 119 +- java/rocksjni/loggerjnicallback.cc | 313 + java/rocksjni/loggerjnicallback.h | 50 + java/rocksjni/lru_cache.cc | 41 + java/rocksjni/memtablejni.cc | 66 +- java/rocksjni/merge_operator.cc | 48 + java/rocksjni/options.cc | 5513 +++- java/rocksjni/portal.h | 3440 ++- java/rocksjni/ratelimiterjni.cc | 96 + .../remove_emptyvalue_compactionfilterjni.cc | 24 + java/rocksjni/restorejni.cc | 126 +- java/rocksjni/rocksdb_exception_test.cc | 78 + java/rocksjni/rocksjni.cc | 2185 +- java/rocksjni/slice.cc | 349 + java/rocksjni/snapshot.cc | 26 + java/rocksjni/sst_file_writerjni.cc | 248 + java/rocksjni/statistics.cc | 248 +- java/rocksjni/statisticsjni.cc | 33 + java/rocksjni/statisticsjni.h | 33 + java/rocksjni/table.cc | 58 +- java/rocksjni/transaction_log.cc | 71 + java/rocksjni/ttl.cc | 183 + java/rocksjni/write_batch.cc | 394 +- java/rocksjni/write_batch_test.cc | 161 + java/rocksjni/write_batch_with_index.cc | 582 + java/rocksjni/writebatchhandlerjnicallback.cc | 306 + java/rocksjni/writebatchhandlerjnicallback.h | 48 + .../main/java/RocksDBColumnFamilySample.java | 78 + java/samples/src/main/java/RocksDBSample.java | 303 + .../org/rocksdb/AbstractCompactionFilter.java | 30 + .../java/org/rocksdb/AbstractComparator.java | 106 + .../AbstractImmutableNativeReference.java | 66 + .../org/rocksdb/AbstractNativeReference.java | 76 + .../org/rocksdb/AbstractRocksIterator.java | 101 + .../main/java/org/rocksdb/AbstractSlice.java | 191 + .../java/org/rocksdb/AbstractWriteBatch.java | 119 + .../src/main/java/org/rocksdb/AccessHint.java | 53 + .../AdvancedColumnFamilyOptionsInterface.java | 465 + ...edMutableColumnFamilyOptionsInterface.java | 437 + .../main/java/org/rocksdb/BackupEngine.java | 221 + .../src/main/java/org/rocksdb/BackupInfo.java | 66 + .../java/org/rocksdb/BackupableDBOptions.java | 465 + .../org/rocksdb/BlockBasedTableConfig.java | 452 + .../main/java/org/rocksdb/BloomFilter.java | 79 + .../java/org/rocksdb/BuiltinComparator.java | 20 + java/src/main/java/org/rocksdb/Cache.java | 13 + .../rocksdb/CassandraCompactionFilter.java | 18 + .../rocksdb/CassandraValueMergeOperator.java | 20 + .../src/main/java/org/rocksdb/Checkpoint.java | 66 + .../main/java/org/rocksdb/ChecksumType.java | 39 + .../src/main/java/org/rocksdb/ClockCache.java | 59 + .../org/rocksdb/ColumnFamilyDescriptor.java | 61 + .../java/org/rocksdb/ColumnFamilyHandle.java | 42 + .../java/org/rocksdb/ColumnFamilyOptions.java | 909 + .../rocksdb/ColumnFamilyOptionsInterface.java | 375 + .../org/rocksdb/CompactionOptionsFIFO.java | 50 + .../rocksdb/CompactionOptionsUniversal.java | 273 + .../java/org/rocksdb/CompactionPriority.java | 73 + .../java/org/rocksdb/CompactionStopStyle.java | 54 + .../java/org/rocksdb/CompactionStyle.java | 52 + .../src/main/java/org/rocksdb/Comparator.java | 32 + .../java/org/rocksdb/ComparatorOptions.java | 51 + .../java/org/rocksdb/CompressionOptions.java | 85 + .../java/org/rocksdb/CompressionType.java | 99 + java/src/main/java/org/rocksdb/DBOptions.java | 1120 + .../java/org/rocksdb/DBOptionsInterface.java | 1549 ++ java/src/main/java/org/rocksdb/DbPath.java | 47 + .../java/org/rocksdb/DirectComparator.java | 33 + .../main/java/org/rocksdb/DirectSlice.java | 132 + .../main/java/org/rocksdb/EncodingType.java | 55 + java/src/main/java/org/rocksdb/Env.java | 92 + .../src/main/java/org/rocksdb/EnvOptions.java | 207 + .../main/java/org/rocksdb/Experimental.java | 23 + .../main/java}/org/rocksdb/Filter.java | 20 +- .../main/java/org/rocksdb/FlushOptions.java | 49 + .../rocksdb/HashLinkedListMemTableConfig.java | 173 + .../rocksdb/HashSkipListMemTableConfig.java | 16 +- .../main/java}/org/rocksdb/HistogramData.java | 13 +- .../main/java/org/rocksdb/HistogramType.java | 98 + java/src/main/java/org/rocksdb/IndexType.java | 41 + .../main/java/org/rocksdb/InfoLogLevel.java | 48 + .../rocksdb/IngestExternalFileOptions.java | 125 + java/src/main/java/org/rocksdb/LRUCache.java | 82 + java/src/main/java/org/rocksdb/Logger.java | 111 + .../java}/org/rocksdb/MemTableConfig.java | 12 +- .../main/java/org/rocksdb/MergeOperator.java | 17 + .../rocksdb/MutableColumnFamilyOptions.java | 997 + .../MutableColumnFamilyOptionsInterface.java | 159 + .../java/org/rocksdb/NativeLibraryLoader.java | 124 + java/src/main/java/org/rocksdb/Options.java | 1864 ++ .../java/org/rocksdb/PlainTableConfig.java | 251 + .../main/java/org/rocksdb/RateLimiter.java | 119 + .../main/java/org/rocksdb/ReadOptions.java | 397 + java/src/main/java/org/rocksdb/ReadTier.java | 48 + .../RemoveEmptyValueCompactionFilter.java | 18 + .../main/java/org/rocksdb/RestoreOptions.java | 32 + java/src/main/java/org/rocksdb/RocksDB.java | 2384 ++ .../java/org/rocksdb/RocksDBException.java | 44 + java/src/main/java/org/rocksdb/RocksEnv.java | 43 + .../main/java/org/rocksdb/RocksIterator.java | 64 + .../org/rocksdb/RocksIteratorInterface.java | 80 + .../main/java/org/rocksdb/RocksMemEnv.java | 27 + .../java/org/rocksdb/RocksMutableObject.java | 87 + .../main/java/org/rocksdb/RocksObject.java | 41 + .../org/rocksdb/SkipListMemTableConfig.java | 50 + java/src/main/java/org/rocksdb/Slice.java | 112 + java/src/main/java/org/rocksdb/Snapshot.java | 37 + .../main/java/org/rocksdb/SstFileWriter.java | 256 + .../src/main/java/org/rocksdb/Statistics.java | 149 + .../org/rocksdb/StatisticsCollector.java | 62 +- .../rocksdb/StatisticsCollectorCallback.java | 20 +- .../org/rocksdb/StatsCollectorInput.java | 20 +- .../src/main/java/org/rocksdb/StatsLevel.java | 65 + java/src/main/java/org/rocksdb/Status.java | 113 + .../org/rocksdb/StringAppendOperator.java | 19 + .../java}/org/rocksdb/TableFormatConfig.java | 14 +- .../src/main/java/org/rocksdb/TickerType.java | 480 + .../org/rocksdb/TransactionLogIterator.java | 111 + java/src/main/java/org/rocksdb/TtlDB.java | 211 + .../org/rocksdb/VectorMemTableConfig.java | 9 +- .../java/org/rocksdb/WALRecoveryMode.java | 83 + .../java/org/rocksdb/WBWIRocksIterator.java | 184 + .../src/main/java/org/rocksdb/WriteBatch.java | 153 + .../java/org/rocksdb/WriteBatchInterface.java | 146 + .../java/org/rocksdb/WriteBatchWithIndex.java | 282 + .../main/java/org/rocksdb/WriteOptions.java | 159 + .../org/rocksdb/util/BytewiseComparator.java | 91 + .../util/DirectBytewiseComparator.java | 88 + .../java/org/rocksdb/util/Environment.java | 90 + .../util/ReverseBytewiseComparator.java | 37 + .../main/java/org/rocksdb/util/SizeUnit.java | 16 + .../org/rocksdb/AbstractComparatorTest.java | 199 + .../java/org/rocksdb/BackupEngineTest.java | 240 + .../org/rocksdb/BackupableDBOptionsTest.java | 351 + .../rocksdb/BlockBasedTableConfigTest.java | 171 + .../test/java/org/rocksdb/CheckPointTest.java | 82 + .../test/java/org/rocksdb/ClockCacheTest.java | 26 + .../org/rocksdb/ColumnFamilyOptionsTest.java | 567 + .../java/org/rocksdb/ColumnFamilyTest.java | 606 + .../rocksdb/CompactionOptionsFIFOTest.java | 26 + .../CompactionOptionsUniversalTest.java | 80 + .../org/rocksdb/CompactionPriorityTest.java | 31 + .../org/rocksdb/CompactionStopStyleTest.java | 31 + .../org/rocksdb/ComparatorOptionsTest.java | 32 + .../test/java/org/rocksdb/ComparatorTest.java | 200 + .../org/rocksdb/CompressionOptionsTest.java | 53 + .../org/rocksdb/CompressionTypesTest.java | 20 + .../test/java/org/rocksdb/DBOptionsTest.java | 637 + .../org/rocksdb/DirectComparatorTest.java | 52 + .../java/org/rocksdb/DirectSliceTest.java | 93 + .../test/java/org/rocksdb/EnvOptionsTest.java | 133 + .../src/test/java/org/rocksdb/FilterTest.java | 39 + java/src/test/java/org/rocksdb/FlushTest.java | 49 + .../java/org/rocksdb/InfoLogLevelTest.java | 107 + .../IngestExternalFileOptionsTest.java | 87 + .../java/org/rocksdb/KeyMayExistTest.java | 87 + .../test/java/org/rocksdb/LRUCacheTest.java | 27 + .../src/test/java/org/rocksdb/LoggerTest.java | 238 + .../test/java/org/rocksdb/MemTableTest.java | 111 + java/src/test/java/org/rocksdb/MergeTest.java | 240 + .../java/org/rocksdb/MixedOptionsTest.java | 55 + .../MutableColumnFamilyOptionsTest.java | 88 + .../org/rocksdb/NativeLibraryLoaderTest.java | 41 + .../test/java/org/rocksdb/OptionsTest.java | 1095 + .../org/rocksdb/PlainTableConfigTest.java | 89 + .../org/rocksdb/PlatformRandomHelper.java | 58 + .../java/org/rocksdb/RateLimiterTest.java | 49 + .../test/java/org/rocksdb/ReadOnlyTest.java | 305 + .../java/org/rocksdb/ReadOptionsTest.java | 201 + .../org/rocksdb/RocksDBExceptionTest.java | 115 + .../test/java/org/rocksdb/RocksDBTest.java | 766 + .../test/java/org/rocksdb/RocksEnvTest.java | 39 + .../java/org/rocksdb/RocksIteratorTest.java | 58 + .../java/org/rocksdb/RocksMemEnvTest.java | 148 + .../java/org/rocksdb/RocksMemoryResource.java | 24 + java/src/test/java/org/rocksdb/SliceTest.java | 80 + .../test/java/org/rocksdb/SnapshotTest.java | 169 + .../java/org/rocksdb/SstFileWriterTest.java | 226 + .../org/rocksdb/StatisticsCollectorTest.java | 55 + .../test/java/org/rocksdb/StatisticsTest.java | 160 + .../java/org/rocksdb}/StatsCallbackMock.java | 12 +- .../rocksdb/TransactionLogIteratorTest.java | 138 + java/src/test/java/org/rocksdb/TtlDBTest.java | 112 + java/src/test/java/org/rocksdb/Types.java | 43 + .../java/org/rocksdb/WALRecoveryModeTest.java | 22 + .../org/rocksdb/WriteBatchHandlerTest.java | 169 + .../test/java/org/rocksdb/WriteBatchTest.java | 296 + .../org/rocksdb/WriteBatchThreadedTest.java | 104 + .../org/rocksdb/WriteBatchWithIndexTest.java | 410 + .../java/org/rocksdb/WriteOptionsTest.java | 45 + .../org/rocksdb/test/RocksJunitRunner.java | 70 + .../rocksdb/util/BytewiseComparatorTest.java | 480 + .../org/rocksdb/util/EnvironmentTest.java | 172 + .../java/org/rocksdb/util/SizeUnitTest.java | 27 + linters/__phutil_library_init__.php | 3 - linters/__phutil_library_map__.php | 27 - linters/cpp_linter/ArcanistCpplintLinter.php | 88 - linters/cpp_linter/FbcodeCppLinter.php | 99 - linters/cpp_linter/PfffCppLinter.php | 68 - linters/cpp_linter/cpplint.py | 4767 ---- .../lint_engine/FacebookFbcodeLintEngine.php | 147 - memtable/alloc_tracker.cc | 59 + {util => memtable}/hash_cuckoo_rep.cc | 107 +- {util => memtable}/hash_cuckoo_rep.h | 14 +- {util => memtable}/hash_linklist_rep.cc | 280 +- {util => memtable}/hash_linklist_rep.h | 13 +- {util => memtable}/hash_skiplist_rep.cc | 105 +- {util => memtable}/hash_skiplist_rep.h | 13 +- memtable/inlineskiplist.h | 899 + memtable/inlineskiplist_test.cc | 626 + memtable/memtablerep_bench.cc | 698 + {db => memtable}/skiplist.h | 230 +- {db => memtable}/skiplist_test.cc | 97 +- memtable/skiplistrep.cc | 277 + memtable/stl_wrappers.h | 34 + {util => memtable}/vectorrep.cc | 49 +- memtable/write_buffer_manager.cc | 124 + memtable/write_buffer_manager_test.cc | 151 + monitoring/file_read_sample.h | 23 + monitoring/histogram.cc | 287 + monitoring/histogram.h | 149 + monitoring/histogram_test.cc | 208 + monitoring/histogram_windowing.cc | 197 + monitoring/histogram_windowing.h | 80 + monitoring/instrumented_mutex.cc | 91 + monitoring/instrumented_mutex.h | 98 + monitoring/iostats_context.cc | 61 + monitoring/iostats_context_imp.h | 54 + monitoring/iostats_context_test.cc | 29 + monitoring/perf_context.cc | 185 + monitoring/perf_context_imp.h | 52 + monitoring/perf_level.cc | 28 + monitoring/perf_level_imp.h | 18 + monitoring/perf_step_timer.h | 54 + monitoring/statistics.cc | 191 + monitoring/statistics.h | 115 + monitoring/statistics_test.cc | 35 + monitoring/thread_status_impl.cc | 167 + monitoring/thread_status_updater.cc | 349 + monitoring/thread_status_updater.h | 234 + monitoring/thread_status_updater_debug.cc | 46 + monitoring/thread_status_util.cc | 222 + monitoring/thread_status_util.h | 134 + monitoring/thread_status_util_debug.cc | 32 + options/cf_options.cc | 179 + options/cf_options.h | 239 + options/db_options.cc | 275 + options/db_options.h | 104 + options/options.cc | 636 + options/options_helper.cc | 1133 + options/options_helper.h | 650 + options/options_parser.cc | 792 + options/options_parser.h | 146 + options/options_sanity_check.cc | 38 + options/options_sanity_check.h | 48 + options/options_settable_test.cc | 454 + options/options_test.cc | 1655 ++ port/atomic_pointer.h | 157 - port/dirent.h | 47 + port/likely.h | 8 +- port/port.h | 19 +- port/port_example.h | 37 +- port/port_posix.cc | 92 +- port/port_posix.h | 433 +- port/stack_trace.cc | 30 +- port/stack_trace.h | 8 +- port/sys_time.h | 48 + port/util_logger.h | 23 + port/win/env_default.cc | 42 + port/win/env_win.cc | 1127 + port/win/env_win.h | 310 + port/win/io_win.cc | 1029 + port/win/io_win.h | 441 + port/win/port_win.cc | 230 + port/win/port_win.h | 347 + port/win/stdint.h | 24 - port/win/win_jemalloc.cc | 47 + port/win/win_logger.cc | 160 + port/win/win_logger.h | 63 + port/win/win_thread.cc | 166 + port/win/win_thread.h | 121 + port/win/xpress_win.cc | 226 + port/win/xpress_win.h | 26 + port/xpress.h | 17 + src.mk | 400 + table/adaptive_table_factory.cc | 55 +- table/adaptive_table_factory.h | 28 +- table/block.cc | 264 +- table/block.h | 257 +- table/block_based_filter_block.cc | 255 + table/block_based_filter_block.h | 112 + table/block_based_filter_block_test.cc | 248 + table/block_based_table_builder.cc | 861 +- table/block_based_table_builder.h | 58 +- table/block_based_table_factory.cc | 345 +- table/block_based_table_factory.h | 132 +- table/block_based_table_reader.cc | 2281 +- table/block_based_table_reader.h | 374 +- table/block_builder.cc | 62 +- table/block_builder.h | 23 +- table/block_hash_index.cc | 157 - table/block_hash_index.h | 85 - table/block_hash_index_test.cc | 117 - table/block_prefix_index.cc | 19 +- table/block_prefix_index.h | 11 +- table/block_test.cc | 330 +- table/bloom_block.cc | 10 +- table/bloom_block.h | 17 +- table/cleanable_test.cc | 277 + table/cuckoo_table_builder.cc | 237 +- table/cuckoo_table_builder.h | 59 +- table/cuckoo_table_builder_test.cc | 401 +- table/cuckoo_table_factory.cc | 55 +- table/cuckoo_table_factory.h | 70 +- table/cuckoo_table_reader.cc | 258 +- table/cuckoo_table_reader.h | 42 +- table/cuckoo_table_reader_test.cc | 271 +- table/filter_block.cc | 186 - table/filter_block.h | 131 +- table/filter_block_test.cc | 139 - table/flush_block_policy.cc | 36 +- table/format.cc | 601 +- table/format.h | 158 +- table/full_filter_bits_builder.h | 73 + table/full_filter_block.cc | 113 + table/full_filter_block.h | 117 + table/full_filter_block_test.cc | 189 + table/get_context.cc | 213 + table/get_context.h | 95 + table/index_builder.cc | 187 + table/index_builder.h | 342 + table/internal_iterator.h | 121 + table/iter_heap.h | 24 +- table/iterator.cc | 165 +- table/iterator_wrapper.h | 71 +- table/merger.cc | 353 - table/merger_test.cc | 169 + table/merging_iterator.cc | 404 + table/{merger.h => merging_iterator.h} | 26 +- table/meta_blocks.cc | 207 +- table/meta_blocks.h | 72 +- table/mock_table.cc | 142 + table/mock_table.h | 194 + table/partitioned_filter_block.cc | 311 + table/partitioned_filter_block.h | 102 + table/partitioned_filter_block_test.cc | 299 + table/persistent_cache_helper.cc | 114 + table/persistent_cache_helper.h | 44 + table/persistent_cache_options.h | 34 + table/plain_table_builder.cc | 123 +- table/plain_table_builder.h | 39 +- table/plain_table_factory.cc | 212 +- table/plain_table_factory.h | 88 +- table/plain_table_index.cc | 56 +- table/plain_table_index.h | 43 +- table/plain_table_key_coding.cc | 381 +- table/plain_table_key_coding.h | 118 +- table/plain_table_reader.cc | 329 +- table/plain_table_reader.h | 67 +- table/scoped_arena_iterator.h | 61 + table/sst_file_writer.cc | 262 + table/sst_file_writer_collectors.h | 84 + table/table_builder.h | 83 +- table/table_properties.cc | 146 +- table/table_properties_internal.h | 30 + table/table_reader.h | 68 +- table/table_reader_bench.cc | 110 +- table/table_test.cc | 2131 +- table/two_level_iterator.cc | 135 +- table/two_level_iterator.h | 23 +- third-party/fbson/COMMIT.md | 5 + third-party/fbson/FbsonDocument.h | 893 + third-party/fbson/FbsonJsonParser.h | 741 + third-party/fbson/FbsonStream.h | 182 + third-party/fbson/FbsonUtil.h | 163 + third-party/fbson/FbsonWriter.h | 430 + .../fused-src/gtest/CMakeLists.txt | 1 + .../gtest-1.7.0/fused-src/gtest/gtest-all.cc | 10261 ++++++++ .../gtest-1.7.0/fused-src/gtest/gtest.h | 20725 ++++++++++++++++ third-party/rapidjson/document.h | 821 - third-party/rapidjson/filestream.h | 46 - third-party/rapidjson/internal/pow10.h | 54 - third-party/rapidjson/internal/stack.h | 82 - third-party/rapidjson/internal/strfunc.h | 24 - third-party/rapidjson/license.txt | 19 - third-party/rapidjson/prettywriter.h | 156 - third-party/rapidjson/rapidjson.h | 525 - third-party/rapidjson/reader.h | 683 - third-party/rapidjson/stringbuffer.h | 49 - third-party/rapidjson/writer.h | 241 - thirdparty.inc | 254 + tools/CMakeLists.txt | 21 + tools/Dockerfile | 5 + tools/auto_sanity_test.sh | 34 +- tools/benchmark.sh | 511 + tools/benchmark_leveldb.sh | 185 + tools/blob_dump.cc | 89 + tools/blob_store_bench.cc | 280 - tools/check_format_compatible.sh | 111 + tools/db_bench.cc | 23 + tools/db_bench_tool.cc | 5328 ++++ tools/db_bench_tool_test.cc | 318 + tools/db_crashtest.py | 429 +- tools/db_crashtest2.py | 172 - tools/db_repl_stress.cc | 35 +- tools/db_sanity_test.cc | 161 +- tools/db_stress.cc | 1084 +- tools/dbench_monitor | 102 + tools/dump/db_dump_tool.cc | 261 + tools/dump/rocksdb_dump.cc | 63 + tools/dump/rocksdb_undump.cc | 62 + tools/generate_random_db.sh | 30 + tools/ldb.cc | 16 +- tools/ldb_cmd.cc | 2898 +++ tools/ldb_cmd_impl.h | 523 + tools/ldb_cmd_test.cc | 62 + tools/ldb_test.py | 186 +- tools/ldb_tool.cc | 131 + tools/pflag | 217 + tools/rdb/.gitignore | 1 + tools/rdb/API.md | 178 + tools/rdb/README.md | 40 + tools/rdb/binding.gyp | 25 + tools/rdb/db_wrapper.cc | 525 + tools/rdb/db_wrapper.h | 58 + tools/rdb/rdb | 3 + tools/rdb/rdb.cc | 15 + tools/rdb/unit_test.js | 124 + tools/reduce_levels_test.cc | 120 +- tools/regression_test.sh | 460 + tools/rocksdb_dump_test.sh | 7 + tools/run_flash_bench.sh | 358 + tools/run_leveldb.sh | 174 + tools/sample-dump.dmp | Bin 0 -> 100 bytes tools/sst_dump.cc | 429 +- tools/sst_dump_test.cc | 223 + tools/sst_dump_tool.cc | 689 + tools/sst_dump_tool_imp.h | 83 + tools/verify_random_db.sh | 33 + tools/write_stress.cc | 307 + tools/write_stress_runner.py | 73 + util/aligned_buffer.h | 179 + util/allocator.h | 57 + util/arena.cc | 134 +- util/arena.h | 47 +- util/arena_test.cc | 109 +- util/auto_roll_logger.cc | 117 +- util/auto_roll_logger.h | 76 +- util/auto_roll_logger_test.cc | 414 +- util/autovector.h | 65 +- util/autovector_test.cc | 115 +- util/benchharness.cc | 398 - util/benchharness.h | 357 - util/benchharness_test.cc | 67 - util/blob_store.cc | 270 - util/blob_store.h | 163 - util/blob_store_test.cc | 200 - util/bloom.cc | 301 +- util/bloom_test.cc | 195 +- util/build_version.cc.in | 4 + util/build_version.h | 15 +- util/cache.cc | 481 - util/cache_test.cc | 449 - util/cast_util.h | 21 + util/channel.h | 67 + util/coding.cc | 8 +- util/coding.h | 198 +- util/coding_test.cc | 11 +- util/compaction_job_stats_impl.cc | 86 + util/comparator.cc | 88 +- util/compression.h | 791 + util/concurrent_arena.cc | 39 + util/concurrent_arena.h | 215 + util/core_local.h | 83 + util/crc32c.cc | 39 +- util/crc32c.h | 10 +- util/crc32c_test.cc | 11 +- util/delete_scheduler.cc | 234 + util/delete_scheduler.h | 111 + util/delete_scheduler_test.cc | 563 + util/dynamic_bloom.cc | 36 +- util/dynamic_bloom.h | 76 +- util/dynamic_bloom_test.cc | 199 +- util/env_posix.cc | 1811 -- util/env_test.cc | 741 - util/event_logger.cc | 67 + util/event_logger.h | 196 + util/event_logger_test.cc | 43 + util/fault_injection_test_env.cc | 313 + util/fault_injection_test_env.h | 159 + util/file_reader_writer.cc | 649 + util/file_reader_writer.h | 213 + util/file_reader_writer_test.cc | 325 + util/file_util.cc | 102 + util/file_util.h | 27 + util/filelock_test.cc | 15 +- {db => util}/filename.cc | 119 +- {db => util}/filename.h | 56 +- util/filter_policy.cc | 8 +- util/hash.cc | 27 +- util/hash.h | 18 +- util/hash_map.h | 67 + util/hash_test.cc | 73 + util/heap.h | 166 + util/heap_test.cc | 139 + util/histogram.cc | 198 - util/histogram.h | 79 - util/histogram_test.cc | 62 - util/iostats_context.cc | 32 - util/iostats_context_imp.h | 46 - util/kv_map.h | 34 + util/ldb_cmd.cc | 1853 -- util/ldb_cmd.h | 730 - util/ldb_tool.cc | 121 - util/log_buffer.cc | 52 +- util/log_buffer.h | 18 +- util/log_write_bench.cc | 24 +- util/logging.cc | 99 - util/logging.h | 59 +- util/memory_usage.h | 25 + util/mpsc.h | 158 + util/murmurhash.cc | 32 +- util/murmurhash.h | 14 +- util/mutexlock.h | 61 +- util/options.cc | 530 - util/options_builder.cc | 196 - util/options_test.cc | 80 - util/perf_context.cc | 90 - util/perf_context_imp.h | 88 - util/random.cc | 38 + util/random.h | 35 +- util/rate_limiter.cc | 139 +- util/rate_limiter.h | 44 +- util/rate_limiter_test.cc | 182 +- util/signal_test.cc | 34 - util/skiplistrep.cc | 126 - util/slice.cc | 164 +- util/slice_transform_test.cc | 153 + util/sst_file_manager_impl.cc | 183 + util/sst_file_manager_impl.h | 105 + util/statistics.cc | 134 - util/statistics.h | 90 - util/status.cc | 65 +- util/status_message.cc | 21 + util/stderr_logger.h | 31 + util/stl_wrappers.h | 32 - util/stop_watch.h | 48 +- util/string_util.cc | 357 +- util/string_util.h | 125 +- util/sync_point.cc | 122 +- util/sync_point.h | 78 +- util/testharness.cc | 68 +- util/testharness.h | 138 +- util/testutil.cc | 319 +- util/testutil.h | 671 +- util/thread_list_test.cc | 352 + util/thread_local.cc | 387 +- util/thread_local.h | 151 +- util/thread_local_test.cc | 198 +- util/thread_operation.h | 121 + util/threadpool_imp.cc | 460 + util/threadpool_imp.h | 109 + util/timer_queue.h | 220 + util/timer_queue_test.cc | 72 + util/transaction_test_util.cc | 246 + util/transaction_test_util.h | 114 + util/xxhash.cc | 6 +- util/xxhash.h | 4 +- utilities/backupable/backupable_db.cc | 1723 +- utilities/backupable/backupable_db_test.cc | 1277 +- utilities/blob_db/blob_compaction_filter.h | 78 + utilities/blob_db/blob_db.cc | 235 + utilities/blob_db/blob_db.h | 263 + utilities/blob_db/blob_db_impl.cc | 2223 ++ utilities/blob_db/blob_db_impl.h | 546 + utilities/blob_db/blob_db_iterator.h | 104 + utilities/blob_db/blob_db_test.cc | 1279 + utilities/blob_db/blob_dump_tool.cc | 222 + utilities/blob_db/blob_dump_tool.h | 52 + utilities/blob_db/blob_file.cc | 242 + utilities/blob_db/blob_file.h | 216 + utilities/blob_db/blob_index.h | 161 + utilities/blob_db/blob_log_format.cc | 153 + utilities/blob_db/blob_log_format.h | 123 + utilities/blob_db/blob_log_reader.cc | 96 + utilities/blob_db/blob_log_reader.h | 92 + utilities/blob_db/blob_log_writer.cc | 121 + utilities/blob_db/blob_log_writer.h | 91 + utilities/blob_db/ttl_extractor.cc | 34 + .../cassandra/cassandra_compaction_filter.cc | 47 + .../cassandra/cassandra_compaction_filter.h | 39 + utilities/cassandra/cassandra_format_test.cc | 353 + .../cassandra/cassandra_functional_test.cc | 251 + .../cassandra/cassandra_row_merge_test.cc | 112 + .../cassandra/cassandra_serialize_test.cc | 188 + utilities/cassandra/format.cc | 362 + utilities/cassandra/format.h | 194 + utilities/cassandra/merge_operator.cc | 80 + utilities/cassandra/merge_operator.h | 33 + utilities/cassandra/serialize.h | 75 + utilities/cassandra/test_utils.cc | 67 + utilities/cassandra/test_utils.h | 43 + utilities/checkpoint/checkpoint_impl.cc | 304 + utilities/checkpoint/checkpoint_impl.h | 55 + utilities/checkpoint/checkpoint_test.cc | 595 + utilities/col_buf_decoder.cc | 240 + utilities/col_buf_decoder.h | 116 + utilities/col_buf_encoder.cc | 217 + utilities/col_buf_encoder.h | 219 + utilities/column_aware_encoding_exp.cc | 176 + utilities/column_aware_encoding_test.cc | 254 + utilities/column_aware_encoding_util.cc | 490 + utilities/column_aware_encoding_util.h | 80 + .../remove_emptyvalue_compactionfilter.cc | 30 + .../remove_emptyvalue_compactionfilter.h | 27 + utilities/convenience/info_log_finder.cc | 48 + utilities/date_tiered/date_tiered_db_impl.cc | 396 + utilities/date_tiered/date_tiered_db_impl.h | 89 + utilities/date_tiered/date_tiered_test.cc | 468 + utilities/debug.cc | 55 + utilities/document/document_db.cc | 218 +- utilities/document/document_db_test.cc | 101 +- utilities/document/json_document.cc | 946 +- utilities/document/json_document_builder.cc | 120 + utilities/document/json_document_test.cc | 245 +- utilities/env_librados.cc | 1488 ++ utilities/env_librados.md | 122 + utilities/env_librados_test.cc | 1146 + utilities/env_mirror.cc | 259 + .../env_mirror_test.cc | 147 +- utilities/env_timed.cc | 148 + utilities/env_timed_test.cc | 44 + utilities/geodb/geodb_impl.cc | 132 +- utilities/geodb/geodb_impl.h | 44 +- utilities/geodb/geodb_test.cc | 51 +- utilities/leveldb_options/leveldb_options.cc | 56 + utilities/lua/rocks_lua_compaction_filter.cc | 242 + utilities/lua/rocks_lua_test.cc | 498 + utilities/memory/memory_test.cc | 276 + utilities/memory/memory_util.cc | 52 + utilities/merge_operators.h | 17 +- utilities/merge_operators/max.cc | 77 + utilities/merge_operators/put.cc | 31 +- .../string_append/stringappend.cc | 3 - .../string_append/stringappend.h | 4 +- .../string_append/stringappend2.cc | 42 +- .../string_append/stringappend2.h | 7 +- .../string_append/stringappend_test.cc | 49 +- utilities/merge_operators/uint64add.cc | 12 +- utilities/object_registry_test.cc | 72 + .../option_change_migration.cc | 165 + .../option_change_migration_test.cc | 425 + utilities/options/options_util.cc | 102 + utilities/options/options_util_test.cc | 317 + .../persistent_cache/block_cache_tier.cc | 425 + utilities/persistent_cache/block_cache_tier.h | 156 + .../persistent_cache/block_cache_tier_file.cc | 593 + .../persistent_cache/block_cache_tier_file.h | 293 + .../block_cache_tier_file_buffer.h | 127 + .../block_cache_tier_metadata.cc | 85 + .../block_cache_tier_metadata.h | 125 + utilities/persistent_cache/hash_table.h | 238 + .../persistent_cache/hash_table_bench.cc | 303 + .../persistent_cache/hash_table_evictable.h | 168 + utilities/persistent_cache/hash_table_test.cc | 158 + utilities/persistent_cache/lrulist.h | 174 + .../persistent_cache_bench.cc | 360 + .../persistent_cache/persistent_cache_test.cc | 471 + .../persistent_cache/persistent_cache_test.h | 285 + .../persistent_cache/persistent_cache_tier.cc | 168 + .../persistent_cache/persistent_cache_tier.h | 336 + .../persistent_cache/persistent_cache_util.h | 67 + .../persistent_cache/volatile_tier_impl.cc | 138 + .../persistent_cache/volatile_tier_impl.h | 142 + utilities/redis/redis_list_exception.h | 2 +- utilities/redis/redis_list_iterator.h | 21 +- utilities/redis/redis_lists_test.cc | 90 +- utilities/simulator_cache/sim_cache.cc | 341 + utilities/simulator_cache/sim_cache_test.cc | 218 + utilities/spatialdb/spatial_db.cc | 144 +- utilities/spatialdb/spatial_db_test.cc | 65 +- utilities/spatialdb/utils.h | 14 +- .../compact_on_deletion_collector.cc | 94 + .../compact_on_deletion_collector.h | 103 + .../compact_on_deletion_collector_test.cc | 178 + .../transactions/optimistic_transaction.cc | 131 + .../transactions/optimistic_transaction.h | 98 + .../optimistic_transaction_db_impl.cc | 91 + .../optimistic_transaction_db_impl.h | 43 + .../optimistic_transaction_test.cc | 1401 ++ .../transactions/pessimistic_transaction.cc | 595 + .../transactions/pessimistic_transaction.h | 230 + .../pessimistic_transaction_db.cc | 806 + .../transactions/pessimistic_transaction_db.h | 316 + utilities/transactions/transaction_base.cc | 704 + utilities/transactions/transaction_base.h | 338 + .../transactions/transaction_db_mutex_impl.cc | 135 + .../transactions/transaction_db_mutex_impl.h | 26 + .../transactions/transaction_lock_mgr.cc | 742 + utilities/transactions/transaction_lock_mgr.h | 158 + utilities/transactions/transaction_test.cc | 4884 ++++ utilities/transactions/transaction_util.cc | 162 + utilities/transactions/transaction_util.h | 78 + utilities/transactions/write_prepared_txn.cc | 88 + utilities/transactions/write_prepared_txn.h | 76 + utilities/ttl/db_ttl_impl.cc | 51 +- utilities/ttl/db_ttl_impl.h | 105 +- utilities/ttl/ttl_test.cc | 176 +- utilities/util_merge_operators_test.cc | 99 + .../write_batch_with_index.cc | 995 +- .../write_batch_with_index_internal.cc | 269 + .../write_batch_with_index_internal.h | 114 + .../write_batch_with_index_test.cc | 1682 +- 1303 files changed, 308200 insertions(+), 62614 deletions(-) delete mode 100644 .arcconfig create mode 100644 AUTHORS create mode 100644 CMakeLists.txt create mode 100644 COPYING create mode 100644 DEFAULT_OPTIONS_HISTORY.md create mode 100644 DUMP_FORMAT.md create mode 100644 LANGUAGE-BINDINGS.md create mode 100644 LICENSE.Apache rename LICENSE => LICENSE.leveldb (90%) delete mode 100644 PATENTS create mode 100644 TARGETS create mode 100644 USERS.md create mode 100644 Vagrantfile create mode 100644 WINDOWS_PORT.md create mode 100644 appveyor.yml create mode 100644 buckifier/buckify_rocksdb.py create mode 100755 buckifier/rocks_test_runner.sh create mode 100644 buckifier/targets_builder.py create mode 100644 buckifier/targets_cfg.py create mode 100644 buckifier/util.py create mode 100644 build_tools/RocksDBCommonHelper.php create mode 100755 build_tools/amalgamate.py delete mode 100755 build_tools/build_detect_version create mode 100755 build_tools/cont_integration.sh create mode 100644 build_tools/dependencies.sh create mode 100644 build_tools/dependencies_4.8.1.sh create mode 100755 build_tools/dockerbuild.sh create mode 100644 build_tools/error_filter.py create mode 100755 build_tools/fb_compile_mongo.sh delete mode 100644 build_tools/fbcode.clang31.sh delete mode 100644 build_tools/fbcode.gcc471.sh delete mode 100644 build_tools/fbcode.gcc481.sh create mode 100644 build_tools/fbcode_config.sh create mode 100644 build_tools/fbcode_config4.8.1.sh create mode 100755 build_tools/gnu_parallel delete mode 100755 build_tools/mac-install-gflags.sh delete mode 100755 build_tools/make_new_version.sh create mode 100755 build_tools/make_package.sh create mode 100755 build_tools/precommit_checker.py create mode 100755 build_tools/rocksdb-lego-determinator create mode 100644 build_tools/run_ci_db_test.ps1 delete mode 100755 build_tools/unity create mode 100755 build_tools/update_dependencies.sh delete mode 100755 build_tools/valgrind_test.sh create mode 100755 build_tools/version.sh create mode 100644 cache/cache_bench.cc create mode 100644 cache/cache_test.cc create mode 100644 cache/clock_cache.cc create mode 100644 cache/clock_cache.h create mode 100644 cache/lru_cache.cc create mode 100644 cache/lru_cache.h create mode 100644 cache/lru_cache_test.cc create mode 100644 cache/sharded_cache.cc create mode 100644 cache/sharded_cache.h create mode 100644 cmake/RocksDBConfig.cmake.in create mode 100644 cmake/modules/FindJeMalloc.cmake create mode 100644 cmake/modules/Findbzip2.cmake create mode 100644 cmake/modules/Findlz4.cmake create mode 100644 cmake/modules/Findsnappy.cmake create mode 100644 cmake/modules/Findzlib.cmake create mode 100644 cmake/modules/Findzstd.cmake create mode 100644 db/compact_files_test.cc create mode 100644 db/compacted_db_impl.cc create mode 100644 db/compacted_db_impl.h create mode 100644 db/compaction_iteration_stats.h create mode 100644 db/compaction_iterator.cc create mode 100644 db/compaction_iterator.h create mode 100644 db/compaction_iterator_test.cc create mode 100644 db/compaction_job.cc create mode 100644 db/compaction_job.h create mode 100644 db/compaction_job_stats_test.cc create mode 100644 db/compaction_job_test.cc create mode 100644 db/compaction_picker_test.cc create mode 100644 db/compaction_picker_universal.cc create mode 100644 db/compaction_picker_universal.h create mode 100644 db/comparator_db_test.cc create mode 100644 db/convenience.cc create mode 100644 db/db_basic_test.cc delete mode 100644 db/db_bench.cc create mode 100644 db/db_blob_index_test.cc create mode 100644 db/db_block_cache_test.cc create mode 100644 db/db_bloom_filter_test.cc create mode 100644 db/db_compaction_filter_test.cc create mode 100644 db/db_compaction_test.cc create mode 100644 db/db_dynamic_level_test.cc create mode 100644 db/db_encryption_test.cc create mode 100644 db/db_flush_test.cc create mode 100644 db/db_impl_compaction_flush.cc create mode 100644 db/db_impl_experimental.cc create mode 100644 db/db_impl_files.cc create mode 100644 db/db_impl_open.cc create mode 100644 db/db_impl_write.cc rename util/db_info_dummper.cc => db/db_info_dumper.cc (62%) create mode 100644 db/db_info_dumper.h create mode 100644 db/db_inplace_update_test.cc create mode 100644 db/db_io_failure_test.cc create mode 100644 db/db_iterator_test.cc create mode 100644 db/db_log_iter_test.cc create mode 100644 db/db_memtable_test.cc create mode 100644 db/db_merge_operator_test.cc create mode 100644 db/db_options_test.cc create mode 100644 db/db_properties_test.cc create mode 100644 db/db_range_del_test.cc create mode 100644 db/db_sst_test.cc create mode 100644 db/db_statistics_test.cc create mode 100644 db/db_table_properties_test.cc create mode 100644 db/db_tailing_iter_test.cc create mode 100644 db/db_test2.cc create mode 100644 db/db_test_util.cc create mode 100644 db/db_test_util.h create mode 100644 db/db_universal_compaction_test.cc create mode 100644 db/db_wal_test.cc create mode 100644 db/db_write_test.cc create mode 100644 db/event_helpers.cc create mode 100644 db/event_helpers.h create mode 100644 db/experimental.cc create mode 100644 db/external_sst_file_basic_test.cc create mode 100644 db/external_sst_file_ingestion_job.cc create mode 100644 db/external_sst_file_ingestion_job.h create mode 100644 db/external_sst_file_test.cc create mode 100644 db/fault_injection_test.cc create mode 100644 db/flush_job.cc create mode 100644 db/flush_job.h create mode 100644 db/flush_job_test.cc create mode 100644 db/flush_scheduler.cc create mode 100644 db/flush_scheduler.h create mode 100644 db/forward_iterator_bench.cc create mode 100644 db/job_context.h create mode 100644 db/listener_test.cc delete mode 100644 db/log_and_apply_bench.cc create mode 100644 db/malloc_stats.cc create mode 100644 db/malloc_stats.h create mode 100644 db/managed_iterator.cc create mode 100644 db/managed_iterator.h rename {util => db}/manual_compaction_test.cc (82%) create mode 100644 db/memtable_list_test.cc create mode 100644 db/merge_helper_test.cc create mode 100644 db/options_file_test.cc create mode 100644 db/pinned_iterators_manager.h create mode 100644 db/range_del_aggregator.cc create mode 100644 db/range_del_aggregator.h create mode 100644 db/range_del_aggregator_test.cc create mode 100644 db/repair_test.cc delete mode 100644 db/simple_table_db_test.cc delete mode 100644 db/snapshot.h create mode 100644 db/snapshot_impl.cc create mode 100644 db/snapshot_impl.h create mode 100644 db/version_builder.cc create mode 100644 db/version_builder.h create mode 100644 db/version_builder_test.cc create mode 100644 db/wal_manager.cc create mode 100644 db/wal_manager.h create mode 100644 db/wal_manager_test.cc create mode 100644 db/write_batch_base.cc create mode 100644 db/write_callback.h create mode 100644 db/write_callback_test.cc create mode 100644 db/write_controller.cc create mode 100644 db/write_controller.h create mode 100644 db/write_controller_test.cc create mode 100644 db/write_thread.cc create mode 100644 db/write_thread.h delete mode 100644 doc/doc.css delete mode 100644 doc/index.html delete mode 100644 doc/log_format.txt delete mode 100644 doc/rockslogo.jpg delete mode 100644 doc/rockslogo.png create mode 100644 docs/.gitignore create mode 100644 docs/CNAME create mode 100644 docs/CONTRIBUTING.md create mode 100644 docs/Gemfile create mode 100644 docs/Gemfile.lock create mode 100644 docs/LICENSE-DOCUMENTATION create mode 100644 docs/README.md create mode 100644 docs/TEMPLATE-INFORMATION.md create mode 100644 docs/_config.yml create mode 100644 docs/_data/authors.yml create mode 100644 docs/_data/features.yml create mode 100644 docs/_data/nav.yml create mode 100644 docs/_data/nav_docs.yml create mode 100644 docs/_data/powered_by.yml create mode 100644 docs/_data/powered_by_highlight.yml create mode 100644 docs/_data/promo.yml create mode 100644 docs/_docs/faq.md create mode 100644 docs/_docs/getting-started.md create mode 100644 docs/_includes/blog_pagination.html create mode 100644 docs/_includes/content/gridblocks.html create mode 100644 docs/_includes/content/items/gridblock.html create mode 100644 docs/_includes/doc.html create mode 100644 docs/_includes/doc_paging.html create mode 100644 docs/_includes/footer.html create mode 100644 docs/_includes/head.html create mode 100644 docs/_includes/header.html create mode 100644 docs/_includes/hero.html create mode 100644 docs/_includes/home_header.html create mode 100644 docs/_includes/katex_import.html create mode 100644 docs/_includes/katex_render.html create mode 100644 docs/_includes/nav.html create mode 100644 docs/_includes/nav/collection_nav.html create mode 100644 docs/_includes/nav/collection_nav_group.html create mode 100644 docs/_includes/nav/collection_nav_group_item.html create mode 100644 docs/_includes/nav/header_nav.html create mode 100644 docs/_includes/nav_search.html create mode 100644 docs/_includes/plugins/all_share.html create mode 100644 docs/_includes/plugins/ascii_cinema.html create mode 100644 docs/_includes/plugins/button.html create mode 100644 docs/_includes/plugins/github_star.html create mode 100644 docs/_includes/plugins/github_watch.html create mode 100644 docs/_includes/plugins/google_share.html create mode 100644 docs/_includes/plugins/iframe.html create mode 100644 docs/_includes/plugins/like_button.html create mode 100644 docs/_includes/plugins/plugin_row.html create mode 100644 docs/_includes/plugins/post_social_plugins.html create mode 100644 docs/_includes/plugins/slideshow.html create mode 100644 docs/_includes/plugins/twitter_follow.html create mode 100644 docs/_includes/plugins/twitter_share.html create mode 100644 docs/_includes/post.html create mode 100644 docs/_includes/powered_by.html create mode 100644 docs/_includes/social_plugins.html create mode 100644 docs/_includes/ui/button.html create mode 100644 docs/_layouts/basic.html create mode 100644 docs/_layouts/blog.html create mode 100644 docs/_layouts/blog_default.html create mode 100644 docs/_layouts/default.html create mode 100644 docs/_layouts/doc_default.html create mode 100644 docs/_layouts/doc_page.html create mode 100644 docs/_layouts/docs.html create mode 100644 docs/_layouts/home.html create mode 100644 docs/_layouts/page.html create mode 100644 docs/_layouts/plain.html create mode 100644 docs/_layouts/post.html create mode 100644 docs/_layouts/redirect.html create mode 100644 docs/_layouts/top-level.html create mode 100644 docs/_posts/2014-03-27-how-to-backup-rocksdb.markdown create mode 100644 docs/_posts/2014-03-27-how-to-persist-in-memory-rocksdb-database.markdown create mode 100644 docs/_posts/2014-04-02-the-1st-rocksdb-local-meetup-held-on-march-27-2014.markdown create mode 100644 docs/_posts/2014-04-07-rocksdb-2-8-release.markdown create mode 100644 docs/_posts/2014-04-21-indexing-sst-files-for-better-lookup-performance.markdown create mode 100644 docs/_posts/2014-05-14-lock.markdown create mode 100644 docs/_posts/2014-05-19-rocksdb-3-0-release.markdown create mode 100644 docs/_posts/2014-05-22-rocksdb-3-1-release.markdown create mode 100644 docs/_posts/2014-06-23-plaintable-a-new-file-format.markdown create mode 100644 docs/_posts/2014-06-27-avoid-expensive-locks-in-get.markdown create mode 100644 docs/_posts/2014-06-27-rocksdb-3-2-release.markdown create mode 100644 docs/_posts/2014-07-29-rocksdb-3-3-release.markdown create mode 100644 docs/_posts/2014-09-12-cuckoo.markdown create mode 100644 docs/_posts/2014-09-12-new-bloom-filter-format.markdown create mode 100644 docs/_posts/2014-09-15-rocksdb-3-5-release.markdown create mode 100644 docs/_posts/2015-01-16-migrating-from-leveldb-to-rocksdb-2.markdown create mode 100644 docs/_posts/2015-02-24-reading-rocksdb-options-from-a-file.markdown create mode 100644 docs/_posts/2015-02-27-write-batch-with-index.markdown create mode 100644 docs/_posts/2015-04-22-integrating-rocksdb-with-mongodb-2.markdown create mode 100644 docs/_posts/2015-06-12-rocksdb-in-osquery.markdown create mode 100644 docs/_posts/2015-07-15-rocksdb-2015-h2-roadmap.markdown create mode 100644 docs/_posts/2015-07-17-spatial-indexing-in-rocksdb.markdown create mode 100644 docs/_posts/2015-07-22-rocksdb-is-now-available-in-windows-platform.markdown create mode 100644 docs/_posts/2015-07-23-dynamic-level.markdown create mode 100644 docs/_posts/2015-10-27-getthreadlist.markdown create mode 100644 docs/_posts/2015-11-10-use-checkpoints-for-efficient-snapshots.markdown create mode 100644 docs/_posts/2015-11-16-analysis-file-read-latency-by-level.markdown create mode 100644 docs/_posts/2016-01-29-compaction_pri.markdown create mode 100644 docs/_posts/2016-02-24-rocksdb-4-2-release.markdown create mode 100644 docs/_posts/2016-02-25-rocksdb-ama.markdown create mode 100644 docs/_posts/2016-03-07-rocksdb-options-file.markdown create mode 100644 docs/_posts/2016-04-26-rocksdb-4-5-1-released.markdown create mode 100644 docs/_posts/2016-07-26-rocksdb-4-8-released.markdown create mode 100644 docs/_posts/2016-09-28-rocksdb-4-11-2-released.markdown create mode 100644 docs/_posts/2017-01-06-rocksdb-5-0-1-released.markdown create mode 100644 docs/_posts/2017-02-07-rocksdb-5-1-2-released.markdown create mode 100644 docs/_posts/2017-02-17-bulkoad-ingest-sst-file.markdown create mode 100644 docs/_posts/2017-03-02-rocksdb-5-2-1-released.markdown create mode 100644 docs/_posts/2017-05-12-partitioned-index-filter.markdown create mode 100644 docs/_posts/2017-05-14-core-local-stats.markdown create mode 100644 docs/_posts/2017-05-26-rocksdb-5-4-5-released.markdown create mode 100644 docs/_posts/2017-06-26-17-level-based-changes.markdown create mode 100644 docs/_posts/2017-06-29-rocksdb-5-5-1-released.markdown create mode 100644 docs/_posts/2017-07-25-rocksdb-5-6-1-released.markdown create mode 100644 docs/_posts/2017-08-24-pinnableslice.markdown create mode 100644 docs/_posts/2017-08-25-flushwal.markdown create mode 100644 docs/_sass/_base.scss create mode 100644 docs/_sass/_blog.scss create mode 100644 docs/_sass/_buttons.scss create mode 100644 docs/_sass/_footer.scss create mode 100644 docs/_sass/_gridBlock.scss create mode 100644 docs/_sass/_header.scss create mode 100644 docs/_sass/_poweredby.scss create mode 100644 docs/_sass/_promo.scss create mode 100644 docs/_sass/_react_docs_nav.scss create mode 100644 docs/_sass/_react_header_nav.scss create mode 100644 docs/_sass/_reset.scss create mode 100644 docs/_sass/_search.scss create mode 100644 docs/_sass/_slideshow.scss create mode 100644 docs/_sass/_syntax-highlighting.scss create mode 100644 docs/_sass/_tables.scss create mode 100644 docs/_top-level/support.md create mode 100644 docs/blog/all.html create mode 100644 docs/blog/index.html create mode 100644 docs/css/main.scss create mode 100644 docs/doc-type-examples/2016-04-07-blog-post-example.md create mode 100644 docs/doc-type-examples/docs-hello-world.md create mode 100644 docs/doc-type-examples/top-level-example.md create mode 100644 docs/docs/index.html create mode 100644 docs/feed.xml create mode 100644 docs/index.md create mode 100644 docs/static/favicon.png create mode 100644 docs/static/fonts/LatoLatin-Black.woff create mode 100644 docs/static/fonts/LatoLatin-Black.woff2 create mode 100644 docs/static/fonts/LatoLatin-BlackItalic.woff create mode 100644 docs/static/fonts/LatoLatin-BlackItalic.woff2 create mode 100644 docs/static/fonts/LatoLatin-Italic.woff create mode 100644 docs/static/fonts/LatoLatin-Italic.woff2 create mode 100644 docs/static/fonts/LatoLatin-Light.woff create mode 100644 docs/static/fonts/LatoLatin-Light.woff2 create mode 100644 docs/static/fonts/LatoLatin-Regular.woff create mode 100644 docs/static/fonts/LatoLatin-Regular.woff2 create mode 100644 docs/static/images/Resize-of-20140327_200754-300x225.jpg create mode 100644 docs/static/images/compaction/full-range.png create mode 100644 docs/static/images/compaction/l0-l1-contend.png create mode 100644 docs/static/images/compaction/l1-l2-contend.png create mode 100644 docs/static/images/compaction/part-range-old.png create mode 100644 docs/static/images/pcache-blockindex.jpg create mode 100644 docs/static/images/pcache-fileindex.jpg create mode 100644 docs/static/images/pcache-filelayout.jpg create mode 100644 docs/static/images/pcache-readiopath.jpg create mode 100644 docs/static/images/pcache-tieredstorage.jpg create mode 100644 docs/static/images/pcache-writeiopath.jpg create mode 100644 docs/static/images/promo-adapt.svg create mode 100644 docs/static/images/promo-flash.svg create mode 100644 docs/static/images/promo-operations.svg create mode 100644 docs/static/images/promo-performance.svg create mode 100644 docs/static/images/tree_example1.png create mode 100644 docs/static/logo.svg create mode 100644 docs/static/og_image.png rename {util => env}/env.cc (50%) create mode 100644 env/env_basic_test.cc create mode 100644 env/env_chroot.cc create mode 100644 env/env_chroot.h create mode 100644 env/env_encryption.cc rename {util => env}/env_hdfs.cc (77%) create mode 100644 env/env_posix.cc create mode 100644 env/env_test.cc create mode 100644 env/io_posix.cc create mode 100644 env/io_posix.h create mode 100644 env/mock_env.cc create mode 100644 env/mock_env.h create mode 100644 env/mock_env_test.cc rename {util => env}/posix_logger.h (78%) create mode 100644 examples/c_simple_example.c create mode 100644 examples/compact_files_example.cc create mode 100644 examples/compaction_filter_example.cc create mode 100644 examples/optimistic_transaction_example.cc create mode 100644 examples/options_file_example.cc create mode 100644 examples/rocksdb_option_file_example.ini create mode 100644 examples/transaction_example.cc delete mode 100644 helpers/memenv/memenv.cc create mode 100644 include/rocksdb/advanced_options.h create mode 100644 include/rocksdb/cleanable.h create mode 100644 include/rocksdb/compaction_job_stats.h create mode 100644 include/rocksdb/convenience.h create mode 100644 include/rocksdb/db_bench_tool.h create mode 100644 include/rocksdb/db_dump_tool.h create mode 100644 include/rocksdb/env_encryption.h create mode 100644 include/rocksdb/experimental.h create mode 100644 include/rocksdb/listener.h create mode 100644 include/rocksdb/metadata.h create mode 100644 include/rocksdb/perf_level.h create mode 100644 include/rocksdb/persistent_cache.h create mode 100644 include/rocksdb/snapshot.h create mode 100644 include/rocksdb/sst_dump_tool.h create mode 100644 include/rocksdb/sst_file_manager.h create mode 100644 include/rocksdb/sst_file_writer.h create mode 100644 include/rocksdb/thread_status.h create mode 100644 include/rocksdb/threadpool.h create mode 100644 include/rocksdb/utilities/checkpoint.h create mode 100644 include/rocksdb/utilities/convenience.h create mode 100644 include/rocksdb/utilities/date_tiered_db.h create mode 100644 include/rocksdb/utilities/debug.h create mode 100644 include/rocksdb/utilities/env_librados.h create mode 100644 include/rocksdb/utilities/env_mirror.h create mode 100644 include/rocksdb/utilities/info_log_finder.h create mode 100644 include/rocksdb/utilities/ldb_cmd.h rename {util => include/rocksdb/utilities}/ldb_cmd_execute_result.h (63%) create mode 100644 include/rocksdb/utilities/leveldb_options.h create mode 100644 include/rocksdb/utilities/lua/rocks_lua_compaction_filter.h create mode 100644 include/rocksdb/utilities/lua/rocks_lua_custom_library.h create mode 100644 include/rocksdb/utilities/lua/rocks_lua_util.h create mode 100644 include/rocksdb/utilities/memory_util.h create mode 100644 include/rocksdb/utilities/object_registry.h create mode 100644 include/rocksdb/utilities/optimistic_transaction_db.h create mode 100644 include/rocksdb/utilities/option_change_migration.h create mode 100644 include/rocksdb/utilities/options_util.h create mode 100644 include/rocksdb/utilities/sim_cache.h create mode 100644 include/rocksdb/utilities/table_properties_collectors.h create mode 100644 include/rocksdb/utilities/transaction.h create mode 100644 include/rocksdb/utilities/transaction_db.h create mode 100644 include/rocksdb/utilities/transaction_db_mutex.h create mode 100644 include/rocksdb/wal_filter.h create mode 100644 include/rocksdb/write_batch_base.h create mode 100644 include/rocksdb/write_buffer_manager.h delete mode 100644 include/utilities/backupable_db.h delete mode 100644 include/utilities/db_ttl.h delete mode 100644 include/utilities/document_db.h delete mode 100644 include/utilities/geo_db.h delete mode 100644 include/utilities/json_document.h delete mode 100644 include/utilities/stackable_db.h delete mode 100644 include/utilities/utility_db.h create mode 100644 java/CMakeLists.txt create mode 100644 java/RELEASE.md delete mode 100644 java/RocksDBSample.java rename java/{ => benchmark/src/main/java}/org/rocksdb/benchmark/DbBenchmark.java (86%) create mode 100644 java/crossbuild/Vagrantfile create mode 100755 java/crossbuild/build-linux-centos.sh create mode 100755 java/crossbuild/build-linux.sh create mode 100755 java/crossbuild/docker-build-linux-centos.sh delete mode 100644 java/org/rocksdb/BackupableDB.java delete mode 100644 java/org/rocksdb/BackupableDBOptions.java delete mode 100644 java/org/rocksdb/BlockBasedTableConfig.java delete mode 100644 java/org/rocksdb/BloomFilter.java delete mode 100644 java/org/rocksdb/CompactionStyle.java delete mode 100644 java/org/rocksdb/CompressionType.java delete mode 100644 java/org/rocksdb/HashLinkedListMemTableConfig.java delete mode 100644 java/org/rocksdb/HistogramType.java delete mode 100644 java/org/rocksdb/Options.java delete mode 100644 java/org/rocksdb/PlainTableConfig.java delete mode 100644 java/org/rocksdb/ReadOptions.java delete mode 100644 java/org/rocksdb/RestoreBackupableDB.java delete mode 100644 java/org/rocksdb/RestoreOptions.java delete mode 100644 java/org/rocksdb/RocksDB.java delete mode 100644 java/org/rocksdb/RocksDBException.java delete mode 100644 java/org/rocksdb/RocksEnv.java delete mode 100644 java/org/rocksdb/RocksIterator.java delete mode 100644 java/org/rocksdb/RocksObject.java delete mode 100644 java/org/rocksdb/SkipListMemTableConfig.java delete mode 100644 java/org/rocksdb/Statistics.java delete mode 100644 java/org/rocksdb/TickerType.java delete mode 100644 java/org/rocksdb/WriteBatch.java delete mode 100644 java/org/rocksdb/WriteBatchTest.java delete mode 100644 java/org/rocksdb/WriteOptions.java delete mode 100644 java/org/rocksdb/test/BackupableDBTest.java delete mode 100644 java/org/rocksdb/test/OptionsTest.java delete mode 100644 java/org/rocksdb/test/ReadOptionsTest.java delete mode 100644 java/org/rocksdb/test/StatisticsCollectorTest.java delete mode 100644 java/org/rocksdb/util/Environment.java delete mode 100644 java/org/rocksdb/util/SizeUnit.java create mode 100644 java/rocksjni.pom create mode 100644 java/rocksjni/backupenginejni.cc create mode 100644 java/rocksjni/cassandra_compactionfilterjni.cc create mode 100644 java/rocksjni/cassandra_value_operator.cc create mode 100644 java/rocksjni/checkpoint.cc create mode 100644 java/rocksjni/clock_cache.cc create mode 100644 java/rocksjni/columnfamilyhandle.cc create mode 100644 java/rocksjni/compaction_filter.cc create mode 100644 java/rocksjni/compaction_options_fifo.cc create mode 100644 java/rocksjni/compaction_options_universal.cc create mode 100644 java/rocksjni/comparator.cc create mode 100644 java/rocksjni/comparatorjnicallback.cc create mode 100644 java/rocksjni/comparatorjnicallback.h create mode 100644 java/rocksjni/compression_options.cc create mode 100644 java/rocksjni/env_options.cc create mode 100644 java/rocksjni/ingest_external_file_options.cc create mode 100644 java/rocksjni/loggerjnicallback.cc create mode 100644 java/rocksjni/loggerjnicallback.h create mode 100644 java/rocksjni/lru_cache.cc create mode 100644 java/rocksjni/merge_operator.cc create mode 100644 java/rocksjni/ratelimiterjni.cc create mode 100644 java/rocksjni/remove_emptyvalue_compactionfilterjni.cc create mode 100644 java/rocksjni/rocksdb_exception_test.cc create mode 100644 java/rocksjni/slice.cc create mode 100644 java/rocksjni/snapshot.cc create mode 100644 java/rocksjni/sst_file_writerjni.cc create mode 100644 java/rocksjni/statisticsjni.cc create mode 100644 java/rocksjni/statisticsjni.h create mode 100644 java/rocksjni/transaction_log.cc create mode 100644 java/rocksjni/ttl.cc create mode 100644 java/rocksjni/write_batch_test.cc create mode 100644 java/rocksjni/write_batch_with_index.cc create mode 100644 java/rocksjni/writebatchhandlerjnicallback.cc create mode 100644 java/rocksjni/writebatchhandlerjnicallback.h create mode 100644 java/samples/src/main/java/RocksDBColumnFamilySample.java create mode 100644 java/samples/src/main/java/RocksDBSample.java create mode 100644 java/src/main/java/org/rocksdb/AbstractCompactionFilter.java create mode 100644 java/src/main/java/org/rocksdb/AbstractComparator.java create mode 100644 java/src/main/java/org/rocksdb/AbstractImmutableNativeReference.java create mode 100644 java/src/main/java/org/rocksdb/AbstractNativeReference.java create mode 100644 java/src/main/java/org/rocksdb/AbstractRocksIterator.java create mode 100644 java/src/main/java/org/rocksdb/AbstractSlice.java create mode 100644 java/src/main/java/org/rocksdb/AbstractWriteBatch.java create mode 100644 java/src/main/java/org/rocksdb/AccessHint.java create mode 100644 java/src/main/java/org/rocksdb/AdvancedColumnFamilyOptionsInterface.java create mode 100644 java/src/main/java/org/rocksdb/AdvancedMutableColumnFamilyOptionsInterface.java create mode 100644 java/src/main/java/org/rocksdb/BackupEngine.java create mode 100644 java/src/main/java/org/rocksdb/BackupInfo.java create mode 100644 java/src/main/java/org/rocksdb/BackupableDBOptions.java create mode 100644 java/src/main/java/org/rocksdb/BlockBasedTableConfig.java create mode 100644 java/src/main/java/org/rocksdb/BloomFilter.java create mode 100644 java/src/main/java/org/rocksdb/BuiltinComparator.java create mode 100644 java/src/main/java/org/rocksdb/Cache.java create mode 100644 java/src/main/java/org/rocksdb/CassandraCompactionFilter.java create mode 100644 java/src/main/java/org/rocksdb/CassandraValueMergeOperator.java create mode 100644 java/src/main/java/org/rocksdb/Checkpoint.java create mode 100644 java/src/main/java/org/rocksdb/ChecksumType.java create mode 100644 java/src/main/java/org/rocksdb/ClockCache.java create mode 100644 java/src/main/java/org/rocksdb/ColumnFamilyDescriptor.java create mode 100644 java/src/main/java/org/rocksdb/ColumnFamilyHandle.java create mode 100644 java/src/main/java/org/rocksdb/ColumnFamilyOptions.java create mode 100644 java/src/main/java/org/rocksdb/ColumnFamilyOptionsInterface.java create mode 100644 java/src/main/java/org/rocksdb/CompactionOptionsFIFO.java create mode 100644 java/src/main/java/org/rocksdb/CompactionOptionsUniversal.java create mode 100644 java/src/main/java/org/rocksdb/CompactionPriority.java create mode 100644 java/src/main/java/org/rocksdb/CompactionStopStyle.java create mode 100644 java/src/main/java/org/rocksdb/CompactionStyle.java create mode 100644 java/src/main/java/org/rocksdb/Comparator.java create mode 100644 java/src/main/java/org/rocksdb/ComparatorOptions.java create mode 100644 java/src/main/java/org/rocksdb/CompressionOptions.java create mode 100644 java/src/main/java/org/rocksdb/CompressionType.java create mode 100644 java/src/main/java/org/rocksdb/DBOptions.java create mode 100644 java/src/main/java/org/rocksdb/DBOptionsInterface.java create mode 100644 java/src/main/java/org/rocksdb/DbPath.java create mode 100644 java/src/main/java/org/rocksdb/DirectComparator.java create mode 100644 java/src/main/java/org/rocksdb/DirectSlice.java create mode 100644 java/src/main/java/org/rocksdb/EncodingType.java create mode 100644 java/src/main/java/org/rocksdb/Env.java create mode 100644 java/src/main/java/org/rocksdb/EnvOptions.java create mode 100644 java/src/main/java/org/rocksdb/Experimental.java rename java/{ => src/main/java}/org/rocksdb/Filter.java (57%) create mode 100644 java/src/main/java/org/rocksdb/FlushOptions.java create mode 100644 java/src/main/java/org/rocksdb/HashLinkedListMemTableConfig.java rename java/{ => src/main/java}/org/rocksdb/HashSkipListMemTableConfig.java (86%) rename java/{ => src/main/java}/org/rocksdb/HistogramData.java (62%) create mode 100644 java/src/main/java/org/rocksdb/HistogramType.java create mode 100644 java/src/main/java/org/rocksdb/IndexType.java create mode 100644 java/src/main/java/org/rocksdb/InfoLogLevel.java create mode 100644 java/src/main/java/org/rocksdb/IngestExternalFileOptions.java create mode 100644 java/src/main/java/org/rocksdb/LRUCache.java create mode 100644 java/src/main/java/org/rocksdb/Logger.java rename java/{ => src/main/java}/org/rocksdb/MemTableConfig.java (65%) create mode 100644 java/src/main/java/org/rocksdb/MergeOperator.java create mode 100644 java/src/main/java/org/rocksdb/MutableColumnFamilyOptions.java create mode 100644 java/src/main/java/org/rocksdb/MutableColumnFamilyOptionsInterface.java create mode 100644 java/src/main/java/org/rocksdb/NativeLibraryLoader.java create mode 100644 java/src/main/java/org/rocksdb/Options.java create mode 100644 java/src/main/java/org/rocksdb/PlainTableConfig.java create mode 100644 java/src/main/java/org/rocksdb/RateLimiter.java create mode 100644 java/src/main/java/org/rocksdb/ReadOptions.java create mode 100644 java/src/main/java/org/rocksdb/ReadTier.java create mode 100644 java/src/main/java/org/rocksdb/RemoveEmptyValueCompactionFilter.java create mode 100644 java/src/main/java/org/rocksdb/RestoreOptions.java create mode 100644 java/src/main/java/org/rocksdb/RocksDB.java create mode 100644 java/src/main/java/org/rocksdb/RocksDBException.java create mode 100644 java/src/main/java/org/rocksdb/RocksEnv.java create mode 100644 java/src/main/java/org/rocksdb/RocksIterator.java create mode 100644 java/src/main/java/org/rocksdb/RocksIteratorInterface.java create mode 100644 java/src/main/java/org/rocksdb/RocksMemEnv.java create mode 100644 java/src/main/java/org/rocksdb/RocksMutableObject.java create mode 100644 java/src/main/java/org/rocksdb/RocksObject.java create mode 100644 java/src/main/java/org/rocksdb/SkipListMemTableConfig.java create mode 100644 java/src/main/java/org/rocksdb/Slice.java create mode 100644 java/src/main/java/org/rocksdb/Snapshot.java create mode 100644 java/src/main/java/org/rocksdb/SstFileWriter.java create mode 100644 java/src/main/java/org/rocksdb/Statistics.java rename java/{ => src/main/java}/org/rocksdb/StatisticsCollector.java (57%) rename java/{ => src/main/java}/org/rocksdb/StatisticsCollectorCallback.java (70%) rename java/{ => src/main/java}/org/rocksdb/StatsCollectorInput.java (61%) create mode 100644 java/src/main/java/org/rocksdb/StatsLevel.java create mode 100644 java/src/main/java/org/rocksdb/Status.java create mode 100644 java/src/main/java/org/rocksdb/StringAppendOperator.java rename java/{ => src/main/java}/org/rocksdb/TableFormatConfig.java (51%) create mode 100644 java/src/main/java/org/rocksdb/TickerType.java create mode 100644 java/src/main/java/org/rocksdb/TransactionLogIterator.java create mode 100644 java/src/main/java/org/rocksdb/TtlDB.java rename java/{ => src/main/java}/org/rocksdb/VectorMemTableConfig.java (86%) create mode 100644 java/src/main/java/org/rocksdb/WALRecoveryMode.java create mode 100644 java/src/main/java/org/rocksdb/WBWIRocksIterator.java create mode 100644 java/src/main/java/org/rocksdb/WriteBatch.java create mode 100644 java/src/main/java/org/rocksdb/WriteBatchInterface.java create mode 100644 java/src/main/java/org/rocksdb/WriteBatchWithIndex.java create mode 100644 java/src/main/java/org/rocksdb/WriteOptions.java create mode 100644 java/src/main/java/org/rocksdb/util/BytewiseComparator.java create mode 100644 java/src/main/java/org/rocksdb/util/DirectBytewiseComparator.java create mode 100644 java/src/main/java/org/rocksdb/util/Environment.java create mode 100644 java/src/main/java/org/rocksdb/util/ReverseBytewiseComparator.java create mode 100644 java/src/main/java/org/rocksdb/util/SizeUnit.java create mode 100644 java/src/test/java/org/rocksdb/AbstractComparatorTest.java create mode 100644 java/src/test/java/org/rocksdb/BackupEngineTest.java create mode 100644 java/src/test/java/org/rocksdb/BackupableDBOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/BlockBasedTableConfigTest.java create mode 100644 java/src/test/java/org/rocksdb/CheckPointTest.java create mode 100644 java/src/test/java/org/rocksdb/ClockCacheTest.java create mode 100644 java/src/test/java/org/rocksdb/ColumnFamilyOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/ColumnFamilyTest.java create mode 100644 java/src/test/java/org/rocksdb/CompactionOptionsFIFOTest.java create mode 100644 java/src/test/java/org/rocksdb/CompactionOptionsUniversalTest.java create mode 100644 java/src/test/java/org/rocksdb/CompactionPriorityTest.java create mode 100644 java/src/test/java/org/rocksdb/CompactionStopStyleTest.java create mode 100644 java/src/test/java/org/rocksdb/ComparatorOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/ComparatorTest.java create mode 100644 java/src/test/java/org/rocksdb/CompressionOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/CompressionTypesTest.java create mode 100644 java/src/test/java/org/rocksdb/DBOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/DirectComparatorTest.java create mode 100644 java/src/test/java/org/rocksdb/DirectSliceTest.java create mode 100644 java/src/test/java/org/rocksdb/EnvOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/FilterTest.java create mode 100644 java/src/test/java/org/rocksdb/FlushTest.java create mode 100644 java/src/test/java/org/rocksdb/InfoLogLevelTest.java create mode 100644 java/src/test/java/org/rocksdb/IngestExternalFileOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/KeyMayExistTest.java create mode 100644 java/src/test/java/org/rocksdb/LRUCacheTest.java create mode 100644 java/src/test/java/org/rocksdb/LoggerTest.java create mode 100644 java/src/test/java/org/rocksdb/MemTableTest.java create mode 100644 java/src/test/java/org/rocksdb/MergeTest.java create mode 100644 java/src/test/java/org/rocksdb/MixedOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/MutableColumnFamilyOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/NativeLibraryLoaderTest.java create mode 100644 java/src/test/java/org/rocksdb/OptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/PlainTableConfigTest.java create mode 100644 java/src/test/java/org/rocksdb/PlatformRandomHelper.java create mode 100644 java/src/test/java/org/rocksdb/RateLimiterTest.java create mode 100644 java/src/test/java/org/rocksdb/ReadOnlyTest.java create mode 100644 java/src/test/java/org/rocksdb/ReadOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/RocksDBExceptionTest.java create mode 100644 java/src/test/java/org/rocksdb/RocksDBTest.java create mode 100644 java/src/test/java/org/rocksdb/RocksEnvTest.java create mode 100644 java/src/test/java/org/rocksdb/RocksIteratorTest.java create mode 100644 java/src/test/java/org/rocksdb/RocksMemEnvTest.java create mode 100644 java/src/test/java/org/rocksdb/RocksMemoryResource.java create mode 100644 java/src/test/java/org/rocksdb/SliceTest.java create mode 100644 java/src/test/java/org/rocksdb/SnapshotTest.java create mode 100644 java/src/test/java/org/rocksdb/SstFileWriterTest.java create mode 100644 java/src/test/java/org/rocksdb/StatisticsCollectorTest.java create mode 100644 java/src/test/java/org/rocksdb/StatisticsTest.java rename java/{org/rocksdb/test => src/test/java/org/rocksdb}/StatsCallbackMock.java (52%) create mode 100644 java/src/test/java/org/rocksdb/TransactionLogIteratorTest.java create mode 100644 java/src/test/java/org/rocksdb/TtlDBTest.java create mode 100644 java/src/test/java/org/rocksdb/Types.java create mode 100644 java/src/test/java/org/rocksdb/WALRecoveryModeTest.java create mode 100644 java/src/test/java/org/rocksdb/WriteBatchHandlerTest.java create mode 100644 java/src/test/java/org/rocksdb/WriteBatchTest.java create mode 100644 java/src/test/java/org/rocksdb/WriteBatchThreadedTest.java create mode 100644 java/src/test/java/org/rocksdb/WriteBatchWithIndexTest.java create mode 100644 java/src/test/java/org/rocksdb/WriteOptionsTest.java create mode 100644 java/src/test/java/org/rocksdb/test/RocksJunitRunner.java create mode 100644 java/src/test/java/org/rocksdb/util/BytewiseComparatorTest.java create mode 100644 java/src/test/java/org/rocksdb/util/EnvironmentTest.java create mode 100644 java/src/test/java/org/rocksdb/util/SizeUnitTest.java delete mode 100644 linters/__phutil_library_init__.php delete mode 100644 linters/__phutil_library_map__.php delete mode 100644 linters/cpp_linter/ArcanistCpplintLinter.php delete mode 100644 linters/cpp_linter/FbcodeCppLinter.php delete mode 100644 linters/cpp_linter/PfffCppLinter.php delete mode 100755 linters/cpp_linter/cpplint.py delete mode 100644 linters/lint_engine/FacebookFbcodeLintEngine.php create mode 100644 memtable/alloc_tracker.cc rename {util => memtable}/hash_cuckoo_rep.cc (88%) rename {util => memtable}/hash_cuckoo_rep.h (75%) rename {util => memtable}/hash_linklist_rep.cc (74%) rename {util => memtable}/hash_linklist_rep.h (79%) rename {util => memtable}/hash_skiplist_rep.cc (76%) rename {util => memtable}/hash_skiplist_rep.h (73%) create mode 100644 memtable/inlineskiplist.h create mode 100644 memtable/inlineskiplist_test.cc create mode 100644 memtable/memtablerep_bench.cc rename {db => memtable}/skiplist.h (61%) rename {db => memtable}/skiplist_test.cc (79%) create mode 100644 memtable/skiplistrep.cc create mode 100644 memtable/stl_wrappers.h rename {util => memtable}/vectorrep.cc (86%) create mode 100644 memtable/write_buffer_manager.cc create mode 100644 memtable/write_buffer_manager_test.cc create mode 100644 monitoring/file_read_sample.h create mode 100644 monitoring/histogram.cc create mode 100644 monitoring/histogram.h create mode 100644 monitoring/histogram_test.cc create mode 100644 monitoring/histogram_windowing.cc create mode 100644 monitoring/histogram_windowing.h create mode 100644 monitoring/instrumented_mutex.cc create mode 100644 monitoring/instrumented_mutex.h create mode 100644 monitoring/iostats_context.cc create mode 100644 monitoring/iostats_context_imp.h create mode 100644 monitoring/iostats_context_test.cc create mode 100644 monitoring/perf_context.cc create mode 100644 monitoring/perf_context_imp.h create mode 100644 monitoring/perf_level.cc create mode 100644 monitoring/perf_level_imp.h create mode 100644 monitoring/perf_step_timer.h create mode 100644 monitoring/statistics.cc create mode 100644 monitoring/statistics.h create mode 100644 monitoring/statistics_test.cc create mode 100644 monitoring/thread_status_impl.cc create mode 100644 monitoring/thread_status_updater.cc create mode 100644 monitoring/thread_status_updater.h create mode 100644 monitoring/thread_status_updater_debug.cc create mode 100644 monitoring/thread_status_util.cc create mode 100644 monitoring/thread_status_util.h create mode 100644 monitoring/thread_status_util_debug.cc create mode 100644 options/cf_options.cc create mode 100644 options/cf_options.h create mode 100644 options/db_options.cc create mode 100644 options/db_options.h create mode 100644 options/options.cc create mode 100644 options/options_helper.cc create mode 100644 options/options_helper.h create mode 100644 options/options_parser.cc create mode 100644 options/options_parser.h create mode 100644 options/options_sanity_check.cc create mode 100644 options/options_sanity_check.h create mode 100644 options/options_settable_test.cc create mode 100644 options/options_test.cc delete mode 100644 port/atomic_pointer.h create mode 100644 port/dirent.h create mode 100644 port/sys_time.h create mode 100644 port/util_logger.h create mode 100644 port/win/env_default.cc create mode 100644 port/win/env_win.cc create mode 100644 port/win/env_win.h create mode 100644 port/win/io_win.cc create mode 100644 port/win/io_win.h create mode 100644 port/win/port_win.cc create mode 100644 port/win/port_win.h delete mode 100644 port/win/stdint.h create mode 100644 port/win/win_jemalloc.cc create mode 100644 port/win/win_logger.cc create mode 100644 port/win/win_logger.h create mode 100644 port/win/win_thread.cc create mode 100644 port/win/win_thread.h create mode 100644 port/win/xpress_win.cc create mode 100644 port/win/xpress_win.h create mode 100644 port/xpress.h create mode 100644 src.mk create mode 100644 table/block_based_filter_block.cc create mode 100644 table/block_based_filter_block.h create mode 100644 table/block_based_filter_block_test.cc delete mode 100644 table/block_hash_index.cc delete mode 100644 table/block_hash_index.h delete mode 100644 table/block_hash_index_test.cc create mode 100644 table/cleanable_test.cc delete mode 100644 table/filter_block.cc delete mode 100644 table/filter_block_test.cc create mode 100644 table/full_filter_bits_builder.h create mode 100644 table/full_filter_block.cc create mode 100644 table/full_filter_block.h create mode 100644 table/full_filter_block_test.cc create mode 100644 table/get_context.cc create mode 100644 table/get_context.h create mode 100644 table/index_builder.cc create mode 100644 table/index_builder.h create mode 100644 table/internal_iterator.h delete mode 100644 table/merger.cc create mode 100644 table/merger_test.cc create mode 100644 table/merging_iterator.cc rename table/{merger.h => merging_iterator.h} (64%) create mode 100644 table/mock_table.cc create mode 100644 table/mock_table.h create mode 100644 table/partitioned_filter_block.cc create mode 100644 table/partitioned_filter_block.h create mode 100644 table/partitioned_filter_block_test.cc create mode 100644 table/persistent_cache_helper.cc create mode 100644 table/persistent_cache_helper.h create mode 100644 table/persistent_cache_options.h create mode 100644 table/scoped_arena_iterator.h create mode 100644 table/sst_file_writer.cc create mode 100644 table/sst_file_writer_collectors.h create mode 100644 table/table_properties_internal.h create mode 100644 third-party/fbson/COMMIT.md create mode 100644 third-party/fbson/FbsonDocument.h create mode 100644 third-party/fbson/FbsonJsonParser.h create mode 100644 third-party/fbson/FbsonStream.h create mode 100644 third-party/fbson/FbsonUtil.h create mode 100644 third-party/fbson/FbsonWriter.h create mode 100644 third-party/gtest-1.7.0/fused-src/gtest/CMakeLists.txt create mode 100644 third-party/gtest-1.7.0/fused-src/gtest/gtest-all.cc create mode 100644 third-party/gtest-1.7.0/fused-src/gtest/gtest.h delete mode 100644 third-party/rapidjson/document.h delete mode 100644 third-party/rapidjson/filestream.h delete mode 100644 third-party/rapidjson/internal/pow10.h delete mode 100644 third-party/rapidjson/internal/stack.h delete mode 100644 third-party/rapidjson/internal/strfunc.h delete mode 100644 third-party/rapidjson/license.txt delete mode 100644 third-party/rapidjson/prettywriter.h delete mode 100644 third-party/rapidjson/rapidjson.h delete mode 100644 third-party/rapidjson/reader.h delete mode 100644 third-party/rapidjson/stringbuffer.h delete mode 100644 third-party/rapidjson/writer.h create mode 100644 thirdparty.inc create mode 100644 tools/CMakeLists.txt create mode 100644 tools/Dockerfile create mode 100755 tools/benchmark.sh create mode 100755 tools/benchmark_leveldb.sh create mode 100644 tools/blob_dump.cc delete mode 100644 tools/blob_store_bench.cc create mode 100755 tools/check_format_compatible.sh create mode 100644 tools/db_bench.cc create mode 100644 tools/db_bench_tool.cc create mode 100644 tools/db_bench_tool_test.cc delete mode 100644 tools/db_crashtest2.py create mode 100755 tools/dbench_monitor create mode 100644 tools/dump/db_dump_tool.cc create mode 100644 tools/dump/rocksdb_dump.cc create mode 100644 tools/dump/rocksdb_undump.cc create mode 100755 tools/generate_random_db.sh create mode 100644 tools/ldb_cmd.cc create mode 100644 tools/ldb_cmd_impl.h create mode 100644 tools/ldb_cmd_test.cc create mode 100644 tools/ldb_tool.cc create mode 100755 tools/pflag create mode 100644 tools/rdb/.gitignore create mode 100644 tools/rdb/API.md create mode 100644 tools/rdb/README.md create mode 100644 tools/rdb/binding.gyp create mode 100644 tools/rdb/db_wrapper.cc create mode 100644 tools/rdb/db_wrapper.h create mode 100755 tools/rdb/rdb create mode 100644 tools/rdb/rdb.cc create mode 100644 tools/rdb/unit_test.js create mode 100755 tools/regression_test.sh create mode 100755 tools/rocksdb_dump_test.sh create mode 100755 tools/run_flash_bench.sh create mode 100755 tools/run_leveldb.sh create mode 100644 tools/sample-dump.dmp create mode 100644 tools/sst_dump_test.cc create mode 100644 tools/sst_dump_tool.cc create mode 100644 tools/sst_dump_tool_imp.h create mode 100755 tools/verify_random_db.sh create mode 100644 tools/write_stress.cc create mode 100644 tools/write_stress_runner.py create mode 100644 util/aligned_buffer.h create mode 100644 util/allocator.h mode change 100755 => 100644 util/auto_roll_logger_test.cc delete mode 100644 util/benchharness.cc delete mode 100644 util/benchharness.h delete mode 100644 util/benchharness_test.cc delete mode 100644 util/blob_store.cc delete mode 100644 util/blob_store.h delete mode 100644 util/blob_store_test.cc create mode 100644 util/build_version.cc.in delete mode 100644 util/cache.cc delete mode 100644 util/cache_test.cc create mode 100644 util/cast_util.h create mode 100644 util/channel.h create mode 100644 util/compaction_job_stats_impl.cc create mode 100644 util/compression.h create mode 100644 util/concurrent_arena.cc create mode 100644 util/concurrent_arena.h create mode 100644 util/core_local.h create mode 100644 util/delete_scheduler.cc create mode 100644 util/delete_scheduler.h create mode 100644 util/delete_scheduler_test.cc delete mode 100644 util/env_posix.cc delete mode 100644 util/env_test.cc create mode 100644 util/event_logger.cc create mode 100644 util/event_logger.h create mode 100644 util/event_logger_test.cc create mode 100644 util/fault_injection_test_env.cc create mode 100644 util/fault_injection_test_env.h create mode 100644 util/file_reader_writer.cc create mode 100644 util/file_reader_writer.h create mode 100644 util/file_reader_writer_test.cc create mode 100644 util/file_util.cc create mode 100644 util/file_util.h rename {db => util}/filename.cc (71%) rename {db => util}/filename.h (71%) create mode 100644 util/hash_map.h create mode 100644 util/hash_test.cc create mode 100644 util/heap.h create mode 100644 util/heap_test.cc delete mode 100644 util/histogram.cc delete mode 100644 util/histogram.h delete mode 100644 util/histogram_test.cc delete mode 100644 util/iostats_context.cc delete mode 100644 util/iostats_context_imp.h create mode 100644 util/kv_map.h delete mode 100644 util/ldb_cmd.cc delete mode 100644 util/ldb_cmd.h delete mode 100644 util/ldb_tool.cc delete mode 100644 util/logging.cc create mode 100644 util/memory_usage.h create mode 100644 util/mpsc.h delete mode 100644 util/options.cc delete mode 100644 util/options_builder.cc delete mode 100644 util/options_test.cc delete mode 100644 util/perf_context.cc delete mode 100644 util/perf_context_imp.h create mode 100644 util/random.cc delete mode 100644 util/signal_test.cc delete mode 100644 util/skiplistrep.cc create mode 100644 util/slice_transform_test.cc create mode 100644 util/sst_file_manager_impl.cc create mode 100644 util/sst_file_manager_impl.h delete mode 100644 util/statistics.cc delete mode 100644 util/statistics.h create mode 100644 util/status_message.cc create mode 100644 util/stderr_logger.h delete mode 100644 util/stl_wrappers.h create mode 100644 util/thread_list_test.cc create mode 100644 util/thread_operation.h create mode 100644 util/threadpool_imp.cc create mode 100644 util/threadpool_imp.h create mode 100644 util/timer_queue.h create mode 100644 util/timer_queue_test.cc create mode 100644 util/transaction_test_util.cc create mode 100644 util/transaction_test_util.h create mode 100644 utilities/blob_db/blob_compaction_filter.h create mode 100644 utilities/blob_db/blob_db.cc create mode 100644 utilities/blob_db/blob_db.h create mode 100644 utilities/blob_db/blob_db_impl.cc create mode 100644 utilities/blob_db/blob_db_impl.h create mode 100644 utilities/blob_db/blob_db_iterator.h create mode 100644 utilities/blob_db/blob_db_test.cc create mode 100644 utilities/blob_db/blob_dump_tool.cc create mode 100644 utilities/blob_db/blob_dump_tool.h create mode 100644 utilities/blob_db/blob_file.cc create mode 100644 utilities/blob_db/blob_file.h create mode 100644 utilities/blob_db/blob_index.h create mode 100644 utilities/blob_db/blob_log_format.cc create mode 100644 utilities/blob_db/blob_log_format.h create mode 100644 utilities/blob_db/blob_log_reader.cc create mode 100644 utilities/blob_db/blob_log_reader.h create mode 100644 utilities/blob_db/blob_log_writer.cc create mode 100644 utilities/blob_db/blob_log_writer.h create mode 100644 utilities/blob_db/ttl_extractor.cc create mode 100644 utilities/cassandra/cassandra_compaction_filter.cc create mode 100644 utilities/cassandra/cassandra_compaction_filter.h create mode 100644 utilities/cassandra/cassandra_format_test.cc create mode 100644 utilities/cassandra/cassandra_functional_test.cc create mode 100644 utilities/cassandra/cassandra_row_merge_test.cc create mode 100644 utilities/cassandra/cassandra_serialize_test.cc create mode 100644 utilities/cassandra/format.cc create mode 100644 utilities/cassandra/format.h create mode 100644 utilities/cassandra/merge_operator.cc create mode 100644 utilities/cassandra/merge_operator.h create mode 100644 utilities/cassandra/serialize.h create mode 100644 utilities/cassandra/test_utils.cc create mode 100644 utilities/cassandra/test_utils.h create mode 100644 utilities/checkpoint/checkpoint_impl.cc create mode 100644 utilities/checkpoint/checkpoint_impl.h create mode 100644 utilities/checkpoint/checkpoint_test.cc create mode 100644 utilities/col_buf_decoder.cc create mode 100644 utilities/col_buf_decoder.h create mode 100644 utilities/col_buf_encoder.cc create mode 100644 utilities/col_buf_encoder.h create mode 100644 utilities/column_aware_encoding_exp.cc create mode 100644 utilities/column_aware_encoding_test.cc create mode 100644 utilities/column_aware_encoding_util.cc create mode 100644 utilities/column_aware_encoding_util.h create mode 100644 utilities/compaction_filters/remove_emptyvalue_compactionfilter.cc create mode 100644 utilities/compaction_filters/remove_emptyvalue_compactionfilter.h create mode 100644 utilities/convenience/info_log_finder.cc create mode 100644 utilities/date_tiered/date_tiered_db_impl.cc create mode 100644 utilities/date_tiered/date_tiered_db_impl.h create mode 100644 utilities/date_tiered/date_tiered_test.cc create mode 100644 utilities/debug.cc create mode 100644 utilities/document/json_document_builder.cc create mode 100644 utilities/env_librados.cc create mode 100644 utilities/env_librados.md create mode 100644 utilities/env_librados_test.cc create mode 100644 utilities/env_mirror.cc rename helpers/memenv/memenv_test.cc => utilities/env_mirror_test.cc (62%) create mode 100644 utilities/env_timed.cc create mode 100644 utilities/env_timed_test.cc create mode 100644 utilities/leveldb_options/leveldb_options.cc create mode 100644 utilities/lua/rocks_lua_compaction_filter.cc create mode 100644 utilities/lua/rocks_lua_test.cc create mode 100644 utilities/memory/memory_test.cc create mode 100644 utilities/memory/memory_util.cc create mode 100644 utilities/merge_operators/max.cc create mode 100644 utilities/object_registry_test.cc create mode 100644 utilities/option_change_migration/option_change_migration.cc create mode 100644 utilities/option_change_migration/option_change_migration_test.cc create mode 100644 utilities/options/options_util.cc create mode 100644 utilities/options/options_util_test.cc create mode 100644 utilities/persistent_cache/block_cache_tier.cc create mode 100644 utilities/persistent_cache/block_cache_tier.h create mode 100644 utilities/persistent_cache/block_cache_tier_file.cc create mode 100644 utilities/persistent_cache/block_cache_tier_file.h create mode 100644 utilities/persistent_cache/block_cache_tier_file_buffer.h create mode 100644 utilities/persistent_cache/block_cache_tier_metadata.cc create mode 100644 utilities/persistent_cache/block_cache_tier_metadata.h create mode 100644 utilities/persistent_cache/hash_table.h create mode 100644 utilities/persistent_cache/hash_table_bench.cc create mode 100644 utilities/persistent_cache/hash_table_evictable.h create mode 100644 utilities/persistent_cache/hash_table_test.cc create mode 100644 utilities/persistent_cache/lrulist.h create mode 100644 utilities/persistent_cache/persistent_cache_bench.cc create mode 100644 utilities/persistent_cache/persistent_cache_test.cc create mode 100644 utilities/persistent_cache/persistent_cache_test.h create mode 100644 utilities/persistent_cache/persistent_cache_tier.cc create mode 100644 utilities/persistent_cache/persistent_cache_tier.h create mode 100644 utilities/persistent_cache/persistent_cache_util.h create mode 100644 utilities/persistent_cache/volatile_tier_impl.cc create mode 100644 utilities/persistent_cache/volatile_tier_impl.h create mode 100644 utilities/simulator_cache/sim_cache.cc create mode 100644 utilities/simulator_cache/sim_cache_test.cc create mode 100644 utilities/table_properties_collectors/compact_on_deletion_collector.cc create mode 100644 utilities/table_properties_collectors/compact_on_deletion_collector.h create mode 100644 utilities/table_properties_collectors/compact_on_deletion_collector_test.cc create mode 100644 utilities/transactions/optimistic_transaction.cc create mode 100644 utilities/transactions/optimistic_transaction.h create mode 100644 utilities/transactions/optimistic_transaction_db_impl.cc create mode 100644 utilities/transactions/optimistic_transaction_db_impl.h create mode 100644 utilities/transactions/optimistic_transaction_test.cc create mode 100644 utilities/transactions/pessimistic_transaction.cc create mode 100644 utilities/transactions/pessimistic_transaction.h create mode 100644 utilities/transactions/pessimistic_transaction_db.cc create mode 100644 utilities/transactions/pessimistic_transaction_db.h create mode 100644 utilities/transactions/transaction_base.cc create mode 100644 utilities/transactions/transaction_base.h create mode 100644 utilities/transactions/transaction_db_mutex_impl.cc create mode 100644 utilities/transactions/transaction_db_mutex_impl.h create mode 100644 utilities/transactions/transaction_lock_mgr.cc create mode 100644 utilities/transactions/transaction_lock_mgr.h create mode 100644 utilities/transactions/transaction_test.cc create mode 100644 utilities/transactions/transaction_util.cc create mode 100644 utilities/transactions/transaction_util.h create mode 100644 utilities/transactions/write_prepared_txn.cc create mode 100644 utilities/transactions/write_prepared_txn.h create mode 100644 utilities/util_merge_operators_test.cc create mode 100644 utilities/write_batch_with_index/write_batch_with_index_internal.cc create mode 100644 utilities/write_batch_with_index/write_batch_with_index_internal.h diff --git a/.arcconfig b/.arcconfig deleted file mode 100644 index 85ca38f2535..00000000000 --- a/.arcconfig +++ /dev/null @@ -1,10 +0,0 @@ -{ - "project_id" : "rocksdb", - "conduit_uri" : "https://reviews.facebook.net/", - "copyright_holder" : "Facebook", - "load" : [ - "linters" - ], - "lint.engine" : "FacebookFbcodeLintEngine", - "lint.engine.single.linter" : "FbcodeCppLinter" -} diff --git a/.gitignore b/.gitignore index 99a7d61d619..03b805983ad 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,4 @@ -TARGETS -build_config.mk +make_config.mk *.a *.arc @@ -20,16 +19,56 @@ build_config.mk *.d-e *.o-* *.swp +*~ +*.vcxproj +*.vcxproj.filters +*.sln +*.cmake +CMakeCache.txt +CMakeFiles/ +build/ ldb manifest_dump sst_dump +blob_dump +column_aware_encoding_exp util/build_version.cc build_tools/VALGRIND_LOGS/ coverage/COVERAGE_REPORT .gdbhistory -.phutil_module_cache +.gdb_history +package/ +unity.a tags +etags +rocksdb_dump +rocksdb_undump +db_test2 + +java/out +java/target +java/test-libs java/*.log java/include/org_rocksdb_*.h + +.idea/ +*.iml + +rocksdb.cc +rocksdb.h unity.cc +java/crossbuild/.vagrant +.vagrant/ +java/**/*.asc +java/javadoc + +scan_build_report/ +t +LOG + +db_logs/ +tp2/ +fbcode/ +fbcode +buckifier/*.pyc diff --git a/.travis.yml b/.travis.yml index 66f37a5d284..b76973d4e8e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,20 +1,72 @@ +sudo: false +dist: trusty language: cpp -compiler: gcc -before_install: -# As of this writing (10 May 2014) the Travis build environment is Ubuntu 12.04, -# which needs the following ugly dependency incantations to build RocksDB: - - sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test - - sudo apt-get update -qq - - sudo apt-get install -y -qq gcc-4.8 g++-4.8 zlib1g-dev libbz2-dev libsnappy-dev libjemalloc-dev - - sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 50 - - sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 50 - - wget https://gflags.googlecode.com/files/libgflags0_2.0-1_amd64.deb - - sudo dpkg -i libgflags0_2.0-1_amd64.deb - - wget https://gflags.googlecode.com/files/libgflags-dev_2.0-1_amd64.deb - - sudo dpkg -i libgflags-dev_2.0-1_amd64.deb -# Lousy hack to disable use and testing of fallocate, which doesn't behave quite -# as EnvPosixTest::AllocateTest expects within the Travis OpenVZ environment. - - sed -i "s/fallocate(/HACK_NO_fallocate(/" build_tools/build_detect_platform -script: make check -j8 +os: + - linux + - osx +compiler: + - clang + - gcc +osx_image: xcode8.3 +jdk: + - oraclejdk7 +cache: + - ccache + - apt + +addons: + apt: + packages: ['zlib1g-dev', 'libbz2-dev', 'libsnappy-dev', 'curl', 'libgflags-dev', 'mingw-w64'] +env: + - TEST_GROUP=platform_dependent # 16-18 minutes + - TEST_GROUP=1 # 33-35 minutes + - TEST_GROUP=2 # 30-32 minutes + # Run java tests + - JOB_NAME=java_test # 4-11 minutes + # Build ROCKSDB_LITE + - JOB_NAME=lite_build # 3-4 minutes + # Build examples + - JOB_NAME=examples # 5-7 minutes + - JOB_NAME=cmake # 3-5 minutes + - JOB_NAME=cmake-mingw # 3 minutes + +matrix: + exclude: + - os: osx + env: TEST_GROUP=1 + - os: osx + env: TEST_GROUP=2 + - os : osx + env: JOB_NAME=cmake-mingw + - os : linux + compiler: clang + - os : osx + compiler: gcc + +# https://docs.travis-ci.com/user/caching/#ccache-cache +install: + - if [ "${TRAVIS_OS_NAME}" == osx ]; then + brew install ccache; + PATH=$PATH:/usr/local/opt/ccache/libexec; + fi + +before_script: + # Increase the maximum number of open file descriptors, since some tests use + # more FDs than the default limit. + - ulimit -n 8192 + +script: + - ${CXX} --version + - if [ "${TEST_GROUP}" == 'platform_dependent' ]; then ccache -C && OPT=-DTRAVIS V=1 ROCKSDBTESTS_END=db_block_cache_test make -j4 all_but_some_tests check_some; fi + - if [ "${TEST_GROUP}" == '1' ]; then OPT=-DTRAVIS V=1 ROCKSDBTESTS_START=db_block_cache_test ROCKSDBTESTS_END=comparator_db_test make -j4 check_some; fi + - if [ "${TEST_GROUP}" == '2' ]; then OPT=-DTRAVIS V=1 ROCKSDBTESTS_START=comparator_db_test make -j4 check_some; fi + - if [ "${JOB_NAME}" == 'java_test' ]; then OPT=-DTRAVIS V=1 make clean jclean && make rocksdbjava jtest; fi + - if [ "${JOB_NAME}" == 'lite_build' ]; then OPT="-DTRAVIS -DROCKSDB_LITE" V=1 make -j4 static_lib tools; fi + - if [ "${JOB_NAME}" == 'examples' ]; then OPT=-DTRAVIS V=1 make -j4 static_lib; cd examples; make -j4; fi + - if [ "${JOB_NAME}" == 'cmake' ]; then mkdir build && cd build && cmake .. && make -j4 rocksdb; fi + - if [ "${JOB_NAME}" == 'cmake-mingw' ]; then mkdir build && cd build && cmake .. -DCMAKE_C_COMPILER=x86_64-w64-mingw32-gcc -DCMAKE_CXX_COMPILER=x86_64-w64-mingw32-g++ -DCMAKE_SYSTEM_NAME=Windows && make -j4 rocksdb; fi notifications: - email: false + email: + - leveldb@fb.com + webhooks: + - https://buildtimetrend.herokuapp.com/travis diff --git a/AUTHORS b/AUTHORS new file mode 100644 index 00000000000..e644f5530f5 --- /dev/null +++ b/AUTHORS @@ -0,0 +1,11 @@ +Facebook Inc. +Facebook Engineering Team + +Google Inc. +# Initial version authors: +Jeffrey Dean +Sanjay Ghemawat + +# Partial list of contributors: +Kevin Regan +Johan Bilien diff --git a/CMakeLists.txt b/CMakeLists.txt new file mode 100644 index 00000000000..45bb105a2ee --- /dev/null +++ b/CMakeLists.txt @@ -0,0 +1,927 @@ +# Prerequisites for Windows: +# This cmake build is for Windows 64-bit only. +# +# Prerequisites: +# You must have at least Visual Studio 2015 Update 3. Start the Developer Command Prompt window that is a part of Visual Studio installation. +# Run the build commands from within the Developer Command Prompt window to have paths to the compiler and runtime libraries set. +# You must have git.exe in your %PATH% environment variable. +# +# To build Rocksdb for Windows is as easy as 1-2-3-4-5: +# +# 1. Update paths to third-party libraries in thirdparty.inc file +# 2. Create a new directory for build artifacts +# mkdir build +# cd build +# 3. Run cmake to generate project files for Windows, add more options to enable required third-party libraries. +# See thirdparty.inc for more information. +# sample command: cmake -G "Visual Studio 14 Win64" -DGFLAGS=1 -DSNAPPY=1 -DJEMALLOC=1 -DJNI=1 .. +# 4. Then build the project in debug mode (you may want to add /m[:] flag to run msbuild in parallel threads +# or simply /m ot use all avail cores) +# msbuild rocksdb.sln +# +# rocksdb.sln build features exclusions of test only code in Release. If you build ALL_BUILD then everything +# will be attempted but test only code does not build in Release mode. +# +# 5. And release mode (/m[:] is also supported) +# msbuild rocksdb.sln /p:Configuration=Release +# +# Linux: +# +# 1. Install a recent toolchain such as devtoolset-3 if you're on a older distro. C++11 required. +# 2. mkdir build; cd build +# 3. cmake .. +# 4. make -j + +cmake_minimum_required(VERSION 2.6) +project(rocksdb) + +if(POLICY CMP0042) + cmake_policy(SET CMP0042 NEW) +endif() + +list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake/modules/") + +option(WITH_JEMALLOC "build with JeMalloc" OFF) +if(MSVC) + include(${CMAKE_CURRENT_SOURCE_DIR}/thirdparty.inc) +else() + if(CMAKE_SYSTEM_NAME MATCHES "FreeBSD") + # FreeBSD has jemaloc as default malloc + # but it does not have all the jemalloc files in include/... + set(WITH_JEMALLOC ON) + else() + if(WITH_JEMALLOC) + find_package(JeMalloc REQUIRED) + add_definitions(-DROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE) + include_directories(${JEMALLOC_INCLUDE_DIR}) + endif() + endif() + + option(WITH_SNAPPY "build with SNAPPY" OFF) + if(WITH_SNAPPY) + find_package(snappy REQUIRED) + add_definitions(-DSNAPPY) + include_directories(${SNAPPY_INCLUDE_DIR}) + list(APPEND THIRDPARTY_LIBS ${SNAPPY_LIBRARIES}) + endif() + + option(WITH_ZLIB "build with zlib" OFF) + if(WITH_ZLIB) + find_package(zlib REQUIRED) + add_definitions(-DZLIB) + include_directories(${ZLIB_INCLUDE_DIR}) + list(APPEND THIRDPARTY_LIBS ${ZLIB_LIBRARIES}) + endif() + + option(WITH_BZ2 "build with bzip2" OFF) + if(WITH_BZ2) + find_package(bzip2 REQUIRED) + add_definitions(-DBZIP2) + include_directories(${BZIP2_INCLUDE_DIR}) + list(APPEND THIRDPARTY_LIBS ${BZIP2_LIBRARIES}) + endif() + + option(WITH_LZ4 "build with lz4" OFF) + if(WITH_LZ4) + find_package(lz4 REQUIRED) + add_definitions(-DLZ4) + include_directories(${LZ4_INCLUDE_DIR}) + list(APPEND THIRDPARTY_LIBS ${LZ4_LIBRARIES}) + endif() + + option(WITH_ZSTD "build with zstd" OFF) + if(WITH_ZSTD) + find_package(zstd REQUIRED) + add_definitions(-DZSTD) + include_directories(${ZSTD_INCLUDE_DIR}) + list(APPEND THIRDPARTY_LIBS ${ZSTD_LIBRARIES}) + endif() +endif() + +if(WIN32) + execute_process(COMMAND powershell -noprofile -Command "Get-Date -format MM_dd_yyyy" OUTPUT_VARIABLE DATE) + execute_process(COMMAND powershell -noprofile -Command "Get-Date -format HH:mm:ss" OUTPUT_VARIABLE TIME) + string(REGEX REPLACE "(..)_(..)_..(..).*" "\\1/\\2/\\3" DATE "${DATE}") + string(REGEX REPLACE "(..):(.....).*" " \\1:\\2" TIME "${TIME}") + set(GIT_DATE_TIME "${DATE} ${TIME}") +else() + execute_process(COMMAND date "+%Y/%m/%d %H:%M:%S" OUTPUT_VARIABLE DATETIME) + string(REGEX REPLACE "\n" "" DATETIME ${DATETIME}) + set(GIT_DATE_TIME "${DATETIME}") +endif() + +find_package(Git) + +if(GIT_FOUND AND EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/.git") + if(WIN32) + execute_process(COMMAND $ENV{COMSPEC} /C ${GIT_EXECUTABLE} -C ${CMAKE_CURRENT_SOURCE_DIR} rev-parse HEAD OUTPUT_VARIABLE GIT_SHA) + else() + execute_process(COMMAND ${GIT_EXECUTABLE} -C ${CMAKE_CURRENT_SOURCE_DIR} rev-parse HEAD OUTPUT_VARIABLE GIT_SHA) + endif() +else() + set(GIT_SHA 0) +endif() + +string(REGEX REPLACE "[^0-9a-f]+" "" GIT_SHA "${GIT_SHA}") + +if(NOT WIN32) + execute_process(COMMAND + "./build_tools/version.sh" "full" + WORKING_DIRECTORY ${PROJECT_SOURCE_DIR} + OUTPUT_VARIABLE ROCKSDB_VERSION + ) + string(STRIP "${ROCKSDB_VERSION}" ROCKSDB_VERSION) + execute_process(COMMAND + "./build_tools/version.sh" "major" + WORKING_DIRECTORY ${PROJECT_SOURCE_DIR} + OUTPUT_VARIABLE ROCKSDB_VERSION_MAJOR + ) + string(STRIP "${ROCKSDB_VERSION_MAJOR}" ROCKSDB_VERSION_MAJOR) +endif() + +option(WITH_MD_LIBRARY "build with MD" ON) +if(WIN32 AND MSVC) + if(WITH_MD_LIBRARY) + set(RUNTIME_LIBRARY "MD") + else() + set(RUNTIME_LIBRARY "MT") + endif() +endif() + +set(BUILD_VERSION_CC ${CMAKE_BINARY_DIR}/build_version.cc) +configure_file(util/build_version.cc.in ${BUILD_VERSION_CC} @ONLY) +add_library(build_version OBJECT ${BUILD_VERSION_CC}) +target_include_directories(build_version PRIVATE + ${CMAKE_CURRENT_SOURCE_DIR}/util) +if(MSVC) + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /Zi /nologo /EHsc /GS /Gd /GR /GF /fp:precise /Zc:wchar_t /Zc:forScope /errorReport:queue") + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /FC /d2Zi+ /W3 /wd4127 /wd4800 /wd4996 /wd4351") +else() + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -W -Wextra -Wall") + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wsign-compare -Wshadow -Wno-unused-parameter -Wno-unused-variable -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers") + if(MINGW) + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-format") + endif() + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11") + if(NOT CMAKE_BUILD_TYPE STREQUAL "Debug") + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O2 -fno-omit-frame-pointer") + include(CheckCXXCompilerFlag) + CHECK_CXX_COMPILER_FLAG("-momit-leaf-frame-pointer" HAVE_OMIT_LEAF_FRAME_POINTER) + if(HAVE_OMIT_LEAF_FRAME_POINTER) + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -momit-leaf-frame-pointer") + endif() + endif() +endif() + +option(PORTABLE "build a portable binary" OFF) +option(FORCE_SSE42 "force building with SSE4.2, even when PORTABLE=ON" OFF) +if(PORTABLE) + # MSVC does not need a separate compiler flag to enable SSE4.2; if nmmintrin.h + # is available, it is available by default. + if(FORCE_SSE42 AND NOT MSVC) + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -msse4.2") + endif() +else() + if(MSVC) + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /arch:AVX2") + else() + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -march=native") + endif() +endif() + +set(CMAKE_REQUIRED_FLAGS ${CMAKE_CXX_FLAGS}) +include(CheckCXXSourceCompiles) +CHECK_CXX_SOURCE_COMPILES(" +#include +#include +int main() { + volatile uint32_t x = _mm_crc32_u32(0, 0); +} +" HAVE_SSE42) +if(HAVE_SSE42) + add_definitions(-DHAVE_SSE42) +elseif(FORCE_SSE42) + message(FATAL_ERROR "FORCE_SSE42=ON but unable to compile with SSE4.2 enabled") +endif() + +CHECK_CXX_SOURCE_COMPILES(" +#if defined(_MSC_VER) && !defined(__thread) +#define __thread __declspec(thread) +#endif +int main() { + static __thread int tls; +} +" HAVE_THREAD_LOCAL) +if(HAVE_THREAD_LOCAL) + add_definitions(-DROCKSDB_SUPPORT_THREAD_LOCAL) +endif() + +option(FAIL_ON_WARNINGS "Treat compile warnings as errors" ON) +if(FAIL_ON_WARNINGS) + if(MSVC) + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /WX") + else() # assume GCC + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror") + endif() +endif() + +option(WITH_ASAN "build with ASAN" OFF) +if(WITH_ASAN) + set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fsanitize=address") + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address") + set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsanitize=address") + if(WITH_JEMALLOC) + message(FATAL "ASAN does not work well with JeMalloc") + endif() +endif() + +option(WITH_TSAN "build with TSAN" OFF) +if(WITH_TSAN) + set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fsanitize=thread -pie") + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=thread -fPIC") + set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsanitize=thread -fPIC") + if(WITH_JEMALLOC) + message(FATAL "TSAN does not work well with JeMalloc") + endif() +endif() + +option(WITH_UBSAN "build with UBSAN" OFF) +if(WITH_UBSAN) + add_definitions(-DROCKSDB_UBSAN_RUN) + set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fsanitize=undefined") + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=undefined") + set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsanitize=undefined") + if(WITH_JEMALLOC) + message(FATAL "UBSAN does not work well with JeMalloc") + endif() +endif() + +# Used to run CI build and tests so we can run faster +set(OPTIMIZE_DEBUG_DEFAULT 0) # Debug build is unoptimized by default use -DOPTDBG=1 to optimize + +if(DEFINED OPTDBG) + set(OPTIMIZE_DEBUG ${OPTDBG}) +else() + set(OPTIMIZE_DEBUG ${OPTIMIZE_DEBUG_DEFAULT}) +endif() + +if(MSVC) + if((${OPTIMIZE_DEBUG} EQUAL 1)) + message(STATUS "Debug optimization is enabled") + set(CMAKE_CXX_FLAGS_DEBUG "/Oxt /${RUNTIME_LIBRARY}d") + else() + set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /Od /RTC1 /Gm /${RUNTIME_LIBRARY}d") + endif() + set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /Oxt /Zp8 /Gm- /Gy /${RUNTIME_LIBRARY}") + + set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} /DEBUG") + set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /DEBUG") +endif() + +if(CMAKE_COMPILER_IS_GNUCXX) + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-builtin-memcmp") +endif() + +option(ROCKSDB_LITE "Build RocksDBLite version" OFF) +if(ROCKSDB_LITE) + add_definitions(-DROCKSDB_LITE) + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-exceptions") +endif() + +if(CMAKE_SYSTEM_NAME MATCHES "Cygwin") + add_definitions(-fno-builtin-memcmp -DCYGWIN) +elseif(CMAKE_SYSTEM_NAME MATCHES "Darwin") + add_definitions(-DOS_MACOSX) + if(CMAKE_SYSTEM_PROCESSOR MATCHES arm) + add_definitions(-DIOS_CROSS_COMPILE -DROCKSDB_LITE) + # no debug info for IOS, that will make our library big + add_definitions(-DNDEBUG) + endif() +elseif(CMAKE_SYSTEM_NAME MATCHES "Linux") + add_definitions(-DOS_LINUX) +elseif(CMAKE_SYSTEM_NAME MATCHES "SunOS") + add_definitions(-DOS_SOLARIS) +elseif(CMAKE_SYSTEM_NAME MATCHES "FreeBSD") + add_definitions(-DOS_FREEBSD) +elseif(CMAKE_SYSTEM_NAME MATCHES "NetBSD") + add_definitions(-DOS_NETBSD) +elseif(CMAKE_SYSTEM_NAME MATCHES "OpenBSD") + add_definitions(-DOS_OPENBSD) +elseif(CMAKE_SYSTEM_NAME MATCHES "DragonFly") + add_definitions(-DOS_DRAGONFLYBSD) +elseif(CMAKE_SYSTEM_NAME MATCHES "Android") + add_definitions(-DOS_ANDROID) +elseif(CMAKE_SYSTEM_NAME MATCHES "Windows") + add_definitions(-DWIN32 -DOS_WIN -D_MBCS -DWIN64 -DNOMINMAX) + if(MINGW) + add_definitions(-D_WIN32_WINNT=_WIN32_WINNT_VISTA) + endif() +endif() + +if(NOT WIN32) + add_definitions(-DROCKSDB_PLATFORM_POSIX -DROCKSDB_LIB_IO_POSIX) +endif() + +option(WITH_FALLOCATE "build with fallocate" ON) + +if(WITH_FALLOCATE) + set(CMAKE_REQUIRED_FLAGS ${CMAKE_C_FLAGS}) + include(CheckCSourceCompiles) + CHECK_C_SOURCE_COMPILES(" +#include +#include +int main() { + int fd = open(\"/dev/null\", 0); + fallocate(fd, FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE, 0, 1024); +} +" HAVE_FALLOCATE) + if(HAVE_FALLOCATE) + add_definitions(-DROCKSDB_FALLOCATE_PRESENT) + endif() +endif() + +include(CheckFunctionExists) +CHECK_FUNCTION_EXISTS(malloc_usable_size HAVE_MALLOC_USABLE_SIZE) +if(HAVE_MALLOC_USABLE_SIZE) + add_definitions(-DROCKSDB_MALLOC_USABLE_SIZE) +endif() + +include_directories(${PROJECT_SOURCE_DIR}) +include_directories(${PROJECT_SOURCE_DIR}/include) +include_directories(SYSTEM ${PROJECT_SOURCE_DIR}/third-party/gtest-1.7.0/fused-src) +find_package(Threads REQUIRED) + +add_subdirectory(third-party/gtest-1.7.0/fused-src/gtest) + +# Main library source code + +set(SOURCES + cache/clock_cache.cc + cache/lru_cache.cc + cache/sharded_cache.cc + db/builder.cc + db/c.cc + db/column_family.cc + db/compacted_db_impl.cc + db/compaction.cc + db/compaction_iterator.cc + db/compaction_job.cc + db/compaction_picker.cc + db/compaction_picker_universal.cc + db/convenience.cc + db/db_filesnapshot.cc + db/db_impl.cc + db/db_impl_write.cc + db/db_impl_compaction_flush.cc + db/db_impl_files.cc + db/db_impl_open.cc + db/db_impl_debug.cc + db/db_impl_experimental.cc + db/db_impl_readonly.cc + db/db_info_dumper.cc + db/db_iter.cc + db/dbformat.cc + db/event_helpers.cc + db/experimental.cc + db/external_sst_file_ingestion_job.cc + db/file_indexer.cc + db/flush_job.cc + db/flush_scheduler.cc + db/forward_iterator.cc + db/internal_stats.cc + db/log_reader.cc + db/log_writer.cc + db/malloc_stats.cc + db/managed_iterator.cc + db/memtable.cc + db/memtable_list.cc + db/merge_helper.cc + db/merge_operator.cc + db/range_del_aggregator.cc + db/repair.cc + db/snapshot_impl.cc + db/table_cache.cc + db/table_properties_collector.cc + db/transaction_log_impl.cc + db/version_builder.cc + db/version_edit.cc + db/version_set.cc + db/wal_manager.cc + db/write_batch.cc + db/write_batch_base.cc + db/write_controller.cc + db/write_thread.cc + env/env.cc + env/env_chroot.cc + env/env_encryption.cc + env/env_hdfs.cc + env/mock_env.cc + memtable/alloc_tracker.cc + memtable/hash_cuckoo_rep.cc + memtable/hash_linklist_rep.cc + memtable/hash_skiplist_rep.cc + memtable/skiplistrep.cc + memtable/vectorrep.cc + memtable/write_buffer_manager.cc + monitoring/histogram.cc + monitoring/histogram_windowing.cc + monitoring/instrumented_mutex.cc + monitoring/iostats_context.cc + monitoring/perf_context.cc + monitoring/perf_level.cc + monitoring/statistics.cc + monitoring/thread_status_impl.cc + monitoring/thread_status_updater.cc + monitoring/thread_status_util.cc + monitoring/thread_status_util_debug.cc + options/cf_options.cc + options/db_options.cc + options/options.cc + options/options_helper.cc + options/options_parser.cc + options/options_sanity_check.cc + port/stack_trace.cc + table/adaptive_table_factory.cc + table/block.cc + table/block_based_filter_block.cc + table/block_based_table_builder.cc + table/block_based_table_factory.cc + table/block_based_table_reader.cc + table/block_builder.cc + table/block_prefix_index.cc + table/bloom_block.cc + table/cuckoo_table_builder.cc + table/cuckoo_table_factory.cc + table/cuckoo_table_reader.cc + table/flush_block_policy.cc + table/format.cc + table/full_filter_block.cc + table/get_context.cc + table/index_builder.cc + table/iterator.cc + table/merging_iterator.cc + table/meta_blocks.cc + table/partitioned_filter_block.cc + table/persistent_cache_helper.cc + table/plain_table_builder.cc + table/plain_table_factory.cc + table/plain_table_index.cc + table/plain_table_key_coding.cc + table/plain_table_reader.cc + table/sst_file_writer.cc + table/table_properties.cc + table/two_level_iterator.cc + tools/db_bench_tool.cc + tools/dump/db_dump_tool.cc + tools/ldb_cmd.cc + tools/ldb_tool.cc + tools/sst_dump_tool.cc + util/arena.cc + util/auto_roll_logger.cc + util/bloom.cc + util/coding.cc + util/compaction_job_stats_impl.cc + util/comparator.cc + util/concurrent_arena.cc + util/crc32c.cc + util/delete_scheduler.cc + util/dynamic_bloom.cc + util/event_logger.cc + util/file_reader_writer.cc + util/file_util.cc + util/filename.cc + util/filter_policy.cc + util/hash.cc + util/log_buffer.cc + util/murmurhash.cc + util/random.cc + util/rate_limiter.cc + util/slice.cc + util/sst_file_manager_impl.cc + util/status.cc + util/status_message.cc + util/string_util.cc + util/sync_point.cc + util/testutil.cc + util/thread_local.cc + util/threadpool_imp.cc + util/transaction_test_util.cc + util/xxhash.cc + utilities/backupable/backupable_db.cc + utilities/blob_db/blob_db.cc + utilities/blob_db/blob_db_impl.cc + utilities/blob_db/blob_dump_tool.cc + utilities/blob_db/blob_file.cc + utilities/blob_db/blob_log_reader.cc + utilities/blob_db/blob_log_writer.cc + utilities/blob_db/blob_log_format.cc + utilities/blob_db/ttl_extractor.cc + utilities/cassandra/cassandra_compaction_filter.cc + utilities/cassandra/format.cc + utilities/cassandra/merge_operator.cc + utilities/checkpoint/checkpoint_impl.cc + utilities/col_buf_decoder.cc + utilities/col_buf_encoder.cc + utilities/column_aware_encoding_util.cc + utilities/compaction_filters/remove_emptyvalue_compactionfilter.cc + utilities/date_tiered/date_tiered_db_impl.cc + utilities/debug.cc + utilities/document/document_db.cc + utilities/document/json_document.cc + utilities/document/json_document_builder.cc + utilities/env_mirror.cc + utilities/env_timed.cc + utilities/geodb/geodb_impl.cc + utilities/leveldb_options/leveldb_options.cc + utilities/lua/rocks_lua_compaction_filter.cc + utilities/memory/memory_util.cc + utilities/merge_operators/max.cc + utilities/merge_operators/put.cc + utilities/merge_operators/string_append/stringappend.cc + utilities/merge_operators/string_append/stringappend2.cc + utilities/merge_operators/uint64add.cc + utilities/option_change_migration/option_change_migration.cc + utilities/options/options_util.cc + utilities/persistent_cache/block_cache_tier.cc + utilities/persistent_cache/block_cache_tier_file.cc + utilities/persistent_cache/block_cache_tier_metadata.cc + utilities/persistent_cache/persistent_cache_tier.cc + utilities/persistent_cache/volatile_tier_impl.cc + utilities/redis/redis_lists.cc + utilities/simulator_cache/sim_cache.cc + utilities/spatialdb/spatial_db.cc + utilities/table_properties_collectors/compact_on_deletion_collector.cc + utilities/transactions/optimistic_transaction_db_impl.cc + utilities/transactions/optimistic_transaction.cc + utilities/transactions/transaction_base.cc + utilities/transactions/pessimistic_transaction_db.cc + utilities/transactions/transaction_db_mutex_impl.cc + utilities/transactions/pessimistic_transaction.cc + utilities/transactions/transaction_lock_mgr.cc + utilities/transactions/transaction_util.cc + utilities/transactions/write_prepared_txn.cc + utilities/ttl/db_ttl_impl.cc + utilities/write_batch_with_index/write_batch_with_index.cc + utilities/write_batch_with_index/write_batch_with_index_internal.cc + $) + +if(WIN32) + list(APPEND SOURCES + port/win/io_win.cc + port/win/env_win.cc + port/win/env_default.cc + port/win/port_win.cc + port/win/win_logger.cc + port/win/win_thread.cc + port/win/xpress_win.cc) + +if(WITH_JEMALLOC) + list(APPEND SOURCES + port/win/win_jemalloc.cc) +endif() + +else() + list(APPEND SOURCES + port/port_posix.cc + env/env_posix.cc + env/io_posix.cc) +endif() + +set(ROCKSDB_STATIC_LIB rocksdb${ARTIFACT_SUFFIX}) +set(ROCKSDB_SHARED_LIB rocksdb-shared${ARTIFACT_SUFFIX}) +set(ROCKSDB_IMPORT_LIB ${ROCKSDB_SHARED_LIB}) +if(WIN32) + set(SYSTEM_LIBS ${SYSTEM_LIBS} Shlwapi.lib Rpcrt4.lib) + set(LIBS ${ROCKSDB_STATIC_LIB} ${THIRDPARTY_LIBS} ${SYSTEM_LIBS}) +else() + set(SYSTEM_LIBS ${CMAKE_THREAD_LIBS_INIT}) + set(LIBS ${ROCKSDB_SHARED_LIB} ${THIRDPARTY_LIBS} ${SYSTEM_LIBS}) + + add_library(${ROCKSDB_SHARED_LIB} SHARED ${SOURCES}) + target_link_libraries(${ROCKSDB_SHARED_LIB} + ${THIRDPARTY_LIBS} ${SYSTEM_LIBS}) + set_target_properties(${ROCKSDB_SHARED_LIB} PROPERTIES + LINKER_LANGUAGE CXX + VERSION ${ROCKSDB_VERSION} + SOVERSION ${ROCKSDB_VERSION_MAJOR} + CXX_STANDARD 11 + OUTPUT_NAME "rocksdb") +endif() + +option(WITH_LIBRADOS "Build with librados" OFF) +if(WITH_LIBRADOS) + list(APPEND SOURCES + utilities/env_librados.cc) + list(APPEND THIRDPARTY_LIBS rados) +endif() + +add_library(${ROCKSDB_STATIC_LIB} STATIC ${SOURCES}) +target_link_libraries(${ROCKSDB_STATIC_LIB} + ${THIRDPARTY_LIBS} ${SYSTEM_LIBS}) + +if(WIN32) + add_library(${ROCKSDB_IMPORT_LIB} SHARED ${SOURCES}) + target_link_libraries(${ROCKSDB_IMPORT_LIB} + ${THIRDPARTY_LIBS} ${SYSTEM_LIBS}) + set_target_properties(${ROCKSDB_IMPORT_LIB} PROPERTIES + COMPILE_DEFINITIONS "ROCKSDB_DLL;ROCKSDB_LIBRARY_EXPORTS") + if(MSVC) + set_target_properties(${ROCKSDB_STATIC_LIB} PROPERTIES + COMPILE_FLAGS "/Fd${CMAKE_CFG_INTDIR}/${ROCKSDB_STATIC_LIB}.pdb") + set_target_properties(${ROCKSDB_IMPORT_LIB} PROPERTIES + COMPILE_FLAGS "/Fd${CMAKE_CFG_INTDIR}/${ROCKSDB_IMPORT_LIB}.pdb") + endif() +endif() + +option(WITH_JNI "build with JNI" OFF) +if(WITH_JNI OR JNI) + message(STATUS "JNI library is enabled") + add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/java) +else() + message(STATUS "JNI library is disabled") +endif() + +# Installation and packaging +if(WIN32) + option(ROCKSDB_INSTALL_ON_WINDOWS "Enable install target on Windows" OFF) +endif() +if(NOT WIN32 OR ROCKSDB_INSTALL_ON_WINDOWS) + if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT) + if(${CMAKE_SYSTEM_NAME} STREQUAL "Linux") + # Change default installation prefix on Linux to /usr + set(CMAKE_INSTALL_PREFIX /usr CACHE PATH "Install path prefix, prepended onto install directories." FORCE) + endif() + endif() + + include(GNUInstallDirs) + include(CMakePackageConfigHelpers) + + set(package_config_destination ${CMAKE_INSTALL_LIBDIR}/cmake/rocksdb) + + configure_package_config_file( + ${CMAKE_SOURCE_DIR}/cmake/RocksDBConfig.cmake.in RocksDBConfig.cmake + INSTALL_DESTINATION ${package_config_destination} + ) + + write_basic_package_version_file( + RocksDBConfigVersion.cmake + VERSION ${ROCKSDB_VERSION} + COMPATIBILITY SameMajorVersion + ) + + install(DIRECTORY include/rocksdb COMPONENT devel DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}") + + install( + TARGETS ${ROCKSDB_STATIC_LIB} + EXPORT RocksDBTargets + COMPONENT devel + ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}" + INCLUDES DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}" + ) + + install( + TARGETS ${ROCKSDB_SHARED_LIB} + EXPORT RocksDBTargets + COMPONENT runtime + RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}" + LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}" + INCLUDES DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}" + ) + + install( + EXPORT RocksDBTargets + COMPONENT devel + DESTINATION ${package_config_destination} + NAMESPACE RocksDB:: + ) + + install( + FILES + ${CMAKE_CURRENT_BINARY_DIR}/RocksDBConfig.cmake + ${CMAKE_CURRENT_BINARY_DIR}/RocksDBConfigVersion.cmake + COMPONENT devel + DESTINATION ${package_config_destination} + ) +endif() + +option(WITH_TESTS "build with tests" ON) +if(WITH_TESTS) + set(TESTS + cache/cache_test.cc + cache/lru_cache_test.cc + db/column_family_test.cc + db/compact_files_test.cc + db/compaction_iterator_test.cc + db/compaction_job_stats_test.cc + db/compaction_job_test.cc + db/compaction_picker_test.cc + db/comparator_db_test.cc + db/corruption_test.cc + db/cuckoo_table_db_test.cc + db/db_basic_test.cc + db/db_blob_index_test.cc + db/db_block_cache_test.cc + db/db_bloom_filter_test.cc + db/db_compaction_filter_test.cc + db/db_compaction_test.cc + db/db_dynamic_level_test.cc + db/db_flush_test.cc + db/db_inplace_update_test.cc + db/db_io_failure_test.cc + db/db_iter_test.cc + db/db_iterator_test.cc + db/db_log_iter_test.cc + db/db_memtable_test.cc + db/db_merge_operator_test.cc + db/db_options_test.cc + db/db_properties_test.cc + db/db_range_del_test.cc + db/db_sst_test.cc + db/db_statistics_test.cc + db/db_table_properties_test.cc + db/db_tailing_iter_test.cc + db/db_test.cc + db/db_test2.cc + db/db_universal_compaction_test.cc + db/db_wal_test.cc + db/db_write_test.cc + db/dbformat_test.cc + db/deletefile_test.cc + db/external_sst_file_basic_test.cc + db/external_sst_file_test.cc + db/fault_injection_test.cc + db/file_indexer_test.cc + db/filename_test.cc + db/flush_job_test.cc + db/listener_test.cc + db/log_test.cc + db/manual_compaction_test.cc + db/memtable_list_test.cc + db/merge_helper_test.cc + db/merge_test.cc + db/options_file_test.cc + db/perf_context_test.cc + db/plain_table_db_test.cc + db/prefix_test.cc + db/repair_test.cc + db/table_properties_collector_test.cc + db/version_builder_test.cc + db/version_edit_test.cc + db/version_set_test.cc + db/wal_manager_test.cc + db/write_batch_test.cc + db/write_callback_test.cc + db/write_controller_test.cc + env/env_basic_test.cc + env/env_test.cc + env/mock_env_test.cc + memtable/inlineskiplist_test.cc + memtable/skiplist_test.cc + memtable/write_buffer_manager_test.cc + monitoring/histogram_test.cc + monitoring/iostats_context_test.cc + monitoring/statistics_test.cc + options/options_settable_test.cc + options/options_test.cc + table/block_based_filter_block_test.cc + table/block_test.cc + table/cleanable_test.cc + table/cuckoo_table_builder_test.cc + table/cuckoo_table_reader_test.cc + table/full_filter_block_test.cc + table/merger_test.cc + table/table_test.cc + tools/ldb_cmd_test.cc + tools/reduce_levels_test.cc + tools/sst_dump_test.cc + util/arena_test.cc + util/auto_roll_logger_test.cc + util/autovector_test.cc + util/bloom_test.cc + util/coding_test.cc + util/crc32c_test.cc + util/delete_scheduler_test.cc + util/dynamic_bloom_test.cc + util/event_logger_test.cc + util/file_reader_writer_test.cc + util/filelock_test.cc + util/hash_test.cc + util/heap_test.cc + util/rate_limiter_test.cc + util/slice_transform_test.cc + util/timer_queue_test.cc + util/thread_list_test.cc + util/thread_local_test.cc + utilities/backupable/backupable_db_test.cc + utilities/blob_db/blob_db_test.cc + utilities/cassandra/cassandra_functional_test.cc + utilities/cassandra/cassandra_format_test.cc + utilities/cassandra/cassandra_row_merge_test.cc + utilities/cassandra/cassandra_serialize_test.cc + utilities/checkpoint/checkpoint_test.cc + utilities/column_aware_encoding_test.cc + utilities/date_tiered/date_tiered_test.cc + utilities/document/document_db_test.cc + utilities/document/json_document_test.cc + utilities/geodb/geodb_test.cc + utilities/lua/rocks_lua_test.cc + utilities/memory/memory_test.cc + utilities/merge_operators/string_append/stringappend_test.cc + utilities/object_registry_test.cc + utilities/option_change_migration/option_change_migration_test.cc + utilities/options/options_util_test.cc + utilities/persistent_cache/hash_table_test.cc + utilities/persistent_cache/persistent_cache_test.cc + utilities/redis/redis_lists_test.cc + utilities/spatialdb/spatial_db_test.cc + utilities/simulator_cache/sim_cache_test.cc + utilities/table_properties_collectors/compact_on_deletion_collector_test.cc + utilities/transactions/optimistic_transaction_test.cc + utilities/transactions/transaction_test.cc + utilities/ttl/ttl_test.cc + utilities/write_batch_with_index/write_batch_with_index_test.cc + ) + if(WITH_LIBRADOS) + list(APPEND TESTS utilities/env_librados_test.cc) + endif() + + set(BENCHMARKS + cache/cache_bench.cc + memtable/memtablerep_bench.cc + tools/db_bench.cc + table/table_reader_bench.cc + utilities/column_aware_encoding_exp.cc + utilities/persistent_cache/hash_table_bench.cc) + add_library(testharness OBJECT util/testharness.cc) + foreach(sourcefile ${BENCHMARKS}) + get_filename_component(exename ${sourcefile} NAME_WE) + add_executable(${exename}${ARTIFACT_SUFFIX} ${sourcefile} + $) + target_link_libraries(${exename}${ARTIFACT_SUFFIX} gtest ${LIBS}) + endforeach(sourcefile ${BENCHMARKS}) + + # For test util library that is build only in DEBUG mode + # and linked to tests. Add test only code that is not #ifdefed for Release here. + set(TESTUTIL_SOURCE + db/db_test_util.cc + monitoring/thread_status_updater_debug.cc + table/mock_table.cc + util/fault_injection_test_env.cc + utilities/cassandra/test_utils.cc + ) + # test utilities are only build in debug + enable_testing() + add_custom_target(check COMMAND ${CMAKE_CTEST_COMMAND}) + set(TESTUTILLIB testutillib${ARTIFACT_SUFFIX}) + add_library(${TESTUTILLIB} STATIC ${TESTUTIL_SOURCE}) + if(MSVC) + set_target_properties(${TESTUTILLIB} PROPERTIES COMPILE_FLAGS "/Fd${CMAKE_CFG_INTDIR}/testutillib${ARTIFACT_SUFFIX}.pdb") + endif() + set_target_properties(${TESTUTILLIB} + PROPERTIES EXCLUDE_FROM_DEFAULT_BUILD_RELEASE 1 + EXCLUDE_FROM_DEFAULT_BUILD_MINRELEASE 1 + EXCLUDE_FROM_DEFAULT_BUILD_RELWITHDEBINFO 1 + ) + + # Tests are excluded from Release builds + set(TEST_EXES ${TESTS}) + + foreach(sourcefile ${TEST_EXES}) + get_filename_component(exename ${sourcefile} NAME_WE) + add_executable(${exename}${ARTIFACT_SUFFIX} ${sourcefile} + $) + set_target_properties(${exename}${ARTIFACT_SUFFIX} + PROPERTIES EXCLUDE_FROM_DEFAULT_BUILD_RELEASE 1 + EXCLUDE_FROM_DEFAULT_BUILD_MINRELEASE 1 + EXCLUDE_FROM_DEFAULT_BUILD_RELWITHDEBINFO 1 + ) + target_link_libraries(${exename}${ARTIFACT_SUFFIX} testutillib${ARTIFACT_SUFFIX} gtest ${LIBS}) + if(NOT "${exename}" MATCHES "db_sanity_test") + add_test(NAME ${exename} COMMAND ${exename}${ARTIFACT_SUFFIX}) + add_dependencies(check ${exename}${ARTIFACT_SUFFIX}) + endif() + endforeach(sourcefile ${TEST_EXES}) + + # C executables must link to a shared object + set(C_TESTS db/c_test.c) + set(C_TEST_EXES ${C_TESTS}) + + foreach(sourcefile ${C_TEST_EXES}) + string(REPLACE ".c" "" exename ${sourcefile}) + string(REGEX REPLACE "^((.+)/)+" "" exename ${exename}) + add_executable(${exename}${ARTIFACT_SUFFIX} ${sourcefile}) + set_target_properties(${exename}${ARTIFACT_SUFFIX} + PROPERTIES EXCLUDE_FROM_DEFAULT_BUILD_RELEASE 1 + EXCLUDE_FROM_DEFAULT_BUILD_MINRELEASE 1 + EXCLUDE_FROM_DEFAULT_BUILD_RELWITHDEBINFO 1 + ) + target_link_libraries(${exename}${ARTIFACT_SUFFIX} ${ROCKSDB_IMPORT_LIB} testutillib${ARTIFACT_SUFFIX}) + add_test(NAME ${exename} COMMAND ${exename}${ARTIFACT_SUFFIX}) + add_dependencies(check ${exename}${ARTIFACT_SUFFIX}) + endforeach(sourcefile ${C_TEST_EXES}) +endif() + +option(WITH_TOOLS "build with tools" ON) +if(WITH_TOOLS) + add_subdirectory(tools) +endif() diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d6467fe07b1..b8b1a412e30 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -12,8 +12,3 @@ Complete your CLA here: If you prefer to sign a paper copy, we can send you a PDF. Send us an e-mail or create a new github issue to request the CLA in PDF format. - -## License - -By contributing to RocksDB, you agree that your contributions will be -licensed under the [BSD License](LICENSE). diff --git a/COPYING b/COPYING new file mode 100644 index 00000000000..d159169d105 --- /dev/null +++ b/COPYING @@ -0,0 +1,339 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Lesser General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License along + with this program; if not, write to the Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. diff --git a/DEFAULT_OPTIONS_HISTORY.md b/DEFAULT_OPTIONS_HISTORY.md new file mode 100644 index 00000000000..26280ee34de --- /dev/null +++ b/DEFAULT_OPTIONS_HISTORY.md @@ -0,0 +1,24 @@ +# RocksDB default options change log +## Unreleased +* delayed_write_rate takes the rate given by rate_limiter if not specified. + +## 5.2 +* Change the default of delayed slowdown value to 16MB/s and further increase the L0 stop condition to 36 files. + +## 5.0 (11/17/2016) +* Options::allow_concurrent_memtable_write and Options::enable_write_thread_adaptive_yield are now true by default +* Options.level0_stop_writes_trigger default value changes from 24 to 32. + +## 4.8.0 (5/2/2016) +* options.max_open_files changes from 5000 to -1. It improves performance, but users need to set file descriptor limit to be large enough and watch memory usage for index and bloom filters. +* options.base_background_compactions changes from max_background_compactions to 1. When users set higher max_background_compactions but the write throughput is not high, the writes are less spiky to disks. +* options.wal_recovery_mode changes from kTolerateCorruptedTailRecords to kPointInTimeRecovery. Avoid some false positive when file system or hardware reorder the writes for file data and metadata. + +## 4.7.0 (4/8/2016) +* options.write_buffer_size changes from 4MB to 64MB. +* options.target_file_size_base changes from 2MB to 64MB. +* options.max_bytes_for_level_base changes from 10MB to 256MB. +* options.soft_pending_compaction_bytes_limit changes from 0 (disabled) to 64GB. +* options.hard_pending_compaction_bytes_limit changes from 0 (disabled) to 256GB. +* table_cache_numshardbits changes from 4 to 6. +* max_file_opening_threads changes from 1 to 16. diff --git a/DUMP_FORMAT.md b/DUMP_FORMAT.md new file mode 100644 index 00000000000..009dabad529 --- /dev/null +++ b/DUMP_FORMAT.md @@ -0,0 +1,16 @@ +## RocksDB dump format + +The version 1 RocksDB dump format is fairly simple: + +1) The dump starts with the magic 8 byte identifier "ROCKDUMP" + +2) The magic is followed by an 8 byte big-endian version which is 0x00000001. + +3) Next are arbitrarily sized chunks of bytes prepended by 4 byte little endian number indicating how large each chunk is. + +4) The first chunk is special and is a json string indicating some things about the creation of this dump. It contains the following keys: +* database-path: The path of the database this dump was created from. +* hostname: The hostname of the machine where the dump was created. +* creation-time: Unix seconds since epoc when this dump was created. + +5) Following the info dump the slices paired into are key/value pairs. diff --git a/HISTORY.md b/HISTORY.md index c6c566ede22..e528440eeaf 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -1,12 +1,453 @@ # Rocksdb Change Log +## 5.8.7 (11/28/2017) +### Bug Fixes +* Fix IOError on WAL write doesn't propagate to write group follower -### Unreleased +## 5.8.6 (11/20/2017) +### Bug Fixes +* Fixed aligned_alloc issues with Windows. ------ Past Releases ----- +## 5.8.1 (10/23/2017) +### New Features +* Add a new db property "rocksdb.estimate-oldest-key-time" to return oldest data timestamp. The property is available only for FIFO compaction with compaction_options_fifo.allow_compaction = false. + +## 5.8.0 (08/30/2017) +### Public API Change +* Users of `Statistics::getHistogramString()` will see fewer histogram buckets and different bucket endpoints. +* `Slice::compare` and BytewiseComparator `Compare` no longer accept `Slice`s containing nullptr. +* `Transaction::Get` and `Transaction::GetForUpdate` variants with `PinnableSlice` added. + +### New Features +* Add Iterator::Refresh(), which allows users to update the iterator state so that they can avoid some initialization costs of recreating iterators. +* Replace dynamic_cast<> (except unit test) so people can choose to build with RTTI off. With make, release mode is by default built with -fno-rtti and debug mode is built without it. Users can override it by setting USE_RTTI=0 or 1. +* Universal compactions including the bottom level can be executed in a dedicated thread pool. This alleviates head-of-line blocking in the compaction queue, which cause write stalling, particularly in multi-instance use cases. Users can enable this feature via `Env::SetBackgroundThreads(N, Env::Priority::BOTTOM)`, where `N > 0`. +* Allow merge operator to be called even with a single merge operand during compactions, by appropriately overriding `MergeOperator::AllowSingleOperand`. +* Add `DB::VerifyChecksum()`, which verifies the checksums in all SST files in a running DB. +* Block-based table support for disabling checksums by setting `BlockBasedTableOptions::checksum = kNoChecksum`. + +### Bug Fixes +* Fix wrong latencies in `rocksdb.db.get.micros`, `rocksdb.db.write.micros`, and `rocksdb.sst.read.micros`. +* Fix incorrect dropping of deletions during intra-L0 compaction. +* Fix transient reappearance of keys covered by range deletions when memtable prefix bloom filter is enabled. +* Fix potentially wrong file smallest key when range deletions separated by snapshot are written together. + +## 5.7.0 (07/13/2017) +### Public API Change +* DB property "rocksdb.sstables" now prints keys in hex form. + +### New Features +* Measure estimated number of reads per file. The information can be accessed through DB::GetColumnFamilyMetaData or "rocksdb.sstables" DB property. +* RateLimiter support for throttling background reads, or throttling the sum of background reads and writes. This can give more predictable I/O usage when compaction reads more data than it writes, e.g., due to lots of deletions. +* [Experimental] FIFO compaction with TTL support. It can be enabled by setting CompactionOptionsFIFO.ttl > 0. +* Introduce `EventListener::OnBackgroundError()` callback. Users can implement it to be notified of errors causing the DB to enter read-only mode, and optionally override them. +* Partitioned Index/Filters exiting the experimental mode. To enable partitioned indexes set index_type to kTwoLevelIndexSearch and to further enable partitioned filters set partition_filters to true. To configure the partition size set metadata_block_size. + + +### Bug Fixes +* Fix discarding empty compaction output files when `DeleteRange()` is used together with subcompactions. + +## 5.6.0 (06/06/2017) +### Public API Change +* Scheduling flushes and compactions in the same thread pool is no longer supported by setting `max_background_flushes=0`. Instead, users can achieve this by configuring their high-pri thread pool to have zero threads. +* Replace `Options::max_background_flushes`, `Options::max_background_compactions`, and `Options::base_background_compactions` all with `Options::max_background_jobs`, which automatically decides how many threads to allocate towards flush/compaction. +* options.delayed_write_rate by default take the value of options.rate_limiter rate. +* Replace global variable `IOStatsContext iostats_context` with `IOStatsContext* get_iostats_context()`; replace global variable `PerfContext perf_context` with `PerfContext* get_perf_context()`. + +### New Features +* Change ticker/histogram statistics implementations to use core-local storage. This improves aggregation speed compared to our previous thread-local approach, particularly for applications with many threads. +* Users can pass a cache object to write buffer manager, so that they can cap memory usage for memtable and block cache using one single limit. +* Flush will be triggered when 7/8 of the limit introduced by write_buffer_manager or db_write_buffer_size is triggered, so that the hard threshold is hard to hit. +* Introduce WriteOptions.low_pri. If it is true, low priority writes will be throttled if the compaction is behind. +* `DB::IngestExternalFile()` now supports ingesting files into a database containing range deletions. + +### Bug Fixes +* Shouldn't ignore return value of fsync() in flush. + +## 5.5.0 (05/17/2017) +### New Features +* FIFO compaction to support Intra L0 compaction too with CompactionOptionsFIFO.allow_compaction=true. +* DB::ResetStats() to reset internal stats. +* Statistics::Reset() to reset user stats. +* ldb add option --try_load_options, which will open DB with its own option file. +* Introduce WriteBatch::PopSavePoint to pop the most recent save point explicitly. +* Support dynamically change `max_open_files` option via SetDBOptions() +* Added DB::CreateColumnFamilie() and DB::DropColumnFamilies() to bulk create/drop column families. +* Add debugging function `GetAllKeyVersions` to see internal versions of a range of keys. +* Support file ingestion with universal compaction style +* Support file ingestion behind with option `allow_ingest_behind` +* New option enable_pipelined_write which may improve write throughput in case writing from multiple threads and WAL enabled. + +### Bug Fixes +* Fix the bug that Direct I/O uses direct reads for non-SST file + +## 5.4.0 (04/11/2017) +### Public API Change +* random_access_max_buffer_size no longer has any effect +* Removed Env::EnableReadAhead(), Env::ShouldForwardRawRequest() +* Support dynamically change `stats_dump_period_sec` option via SetDBOptions(). +* Added ReadOptions::max_skippable_internal_keys to set a threshold to fail a request as incomplete when too many keys are being skipped when using iterators. +* DB::Get in place of std::string accepts PinnableSlice, which avoids the extra memcpy of value to std::string in most of cases. + * PinnableSlice releases the pinned resources that contain the value when it is destructed or when ::Reset() is called on it. + * The old API that accepts std::string, although discouraged, is still supported. +* Replace Options::use_direct_writes with Options::use_direct_io_for_flush_and_compaction. Read Direct IO wiki for details. +* Added CompactionEventListener and EventListener::OnFlushBegin interfaces. + +### New Features +* Memtable flush can be avoided during checkpoint creation if total log file size is smaller than a threshold specified by the user. +* Introduce level-based L0->L0 compactions to reduce file count, so write delays are incurred less often. +* (Experimental) Partitioning filters which creates an index on the partitions. The feature can be enabled by setting partition_filters when using kFullFilter. Currently the feature also requires two-level indexing to be enabled. Number of partitions is the same as the number of partitions for indexes, which is controlled by metadata_block_size. + +## 5.3.0 (03/08/2017) +### Public API Change +* Remove disableDataSync option. +* Remove timeout_hint_us option from WriteOptions. The option has been deprecated and has no effect since 3.13.0. +* Remove option min_partial_merge_operands. Partial merge operands will always be merged in flush or compaction if there are more than one. +* Remove option verify_checksums_in_compaction. Compaction will always verify checksum. + +### Bug Fixes +* Fix the bug that iterator may skip keys + +## 5.2.0 (02/08/2017) +### Public API Change +* NewLRUCache() will determine number of shard bits automatically based on capacity, if the user doesn't pass one. This also impacts the default block cache when the user doesn't explict provide one. +* Change the default of delayed slowdown value to 16MB/s and further increase the L0 stop condition to 36 files. +* Options::use_direct_writes and Options::use_direct_reads are now ready to use. +* (Experimental) Two-level indexing that partition the index and creates a 2nd level index on the partitions. The feature can be enabled by setting kTwoLevelIndexSearch as IndexType and configuring index_per_partition. + +### New Features +* Added new overloaded function GetApproximateSizes that allows to specify if memtable stats should be computed only without computing SST files' stats approximations. +* Added new function GetApproximateMemTableStats that approximates both number of records and size of memtables. +* Add Direct I/O mode for SST file I/O + +### Bug Fixes +* RangeSync() should work if ROCKSDB_FALLOCATE_PRESENT is not set +* Fix wrong results in a data race case in Get() +* Some fixes related to 2PC. +* Fix bugs of data corruption in direct I/O + +## 5.1.0 (01/13/2017) +* Support dynamically change `delete_obsolete_files_period_micros` option via SetDBOptions(). +* Added EventListener::OnExternalFileIngested which will be called when IngestExternalFile() add a file successfully. +* BackupEngine::Open and BackupEngineReadOnly::Open now always return error statuses matching those of the backup Env. + +### Bug Fixes +* Fix the bug that if 2PC is enabled, checkpoints may loss some recent transactions. +* When file copying is needed when creating checkpoints or bulk loading files, fsync the file after the file copying. + +## 5.0.0 (11/17/2016) +### Public API Change +* Options::max_bytes_for_level_multiplier is now a double along with all getters and setters. +* Support dynamically change `delayed_write_rate` and `max_total_wal_size` options via SetDBOptions(). +* Introduce DB::DeleteRange for optimized deletion of large ranges of contiguous keys. +* Support dynamically change `delayed_write_rate` option via SetDBOptions(). +* Options::allow_concurrent_memtable_write and Options::enable_write_thread_adaptive_yield are now true by default. +* Remove Tickers::SEQUENCE_NUMBER to avoid confusion if statistics object is shared among RocksDB instance. Alternatively DB::GetLatestSequenceNumber() can be used to get the same value. +* Options.level0_stop_writes_trigger default value changes from 24 to 32. +* New compaction filter API: CompactionFilter::FilterV2(). Allows to drop ranges of keys. +* Removed flashcache support. +* DB::AddFile() is deprecated and is replaced with DB::IngestExternalFile(). DB::IngestExternalFile() remove all the restrictions that existed for DB::AddFile. + +### New Features +* Add avoid_flush_during_shutdown option, which speeds up DB shutdown by not flushing unpersisted data (i.e. with disableWAL = true). Unpersisted data will be lost. The options is dynamically changeable via SetDBOptions(). +* Add memtable_insert_with_hint_prefix_extractor option. The option is mean to reduce CPU usage for inserting keys into memtable, if keys can be group by prefix and insert for each prefix are sequential or almost sequential. See include/rocksdb/options.h for more details. +* Add LuaCompactionFilter in utilities. This allows developers to write compaction filters in Lua. To use this feature, LUA_PATH needs to be set to the root directory of Lua. +* No longer populate "LATEST_BACKUP" file in backup directory, which formerly contained the number of the latest backup. The latest backup can be determined by finding the highest numbered file in the "meta/" subdirectory. + +## 4.13.0 (10/18/2016) +### Public API Change +* DB::GetOptions() reflect dynamic changed options (i.e. through DB::SetOptions()) and return copy of options instead of reference. +* Added Statistics::getAndResetTickerCount(). + +### New Features +* Add DB::SetDBOptions() to dynamic change base_background_compactions and max_background_compactions. +* Added Iterator::SeekForPrev(). This new API will seek to the last key that less than or equal to the target key. + +## 4.12.0 (9/12/2016) +### Public API Change +* CancelAllBackgroundWork() flushes all memtables for databases containing writes that have bypassed the WAL (writes issued with WriteOptions::disableWAL=true) before shutting down background threads. +* Merge options source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes. +* Remove ImmutableCFOptions. +* Add a compression type ZSTD, which can work with ZSTD 0.8.0 or up. Still keep ZSTDNotFinal for compatibility reasons. + +### New Features +* Introduce NewClockCache, which is based on CLOCK algorithm with better concurrent performance in some cases. It can be used to replace the default LRU-based block cache and table cache. To use it, RocksDB need to be linked with TBB lib. +* Change ticker/histogram statistics implementations to accumulate data in thread-local storage, which improves CPU performance by reducing cache coherency costs. Callers of CreateDBStatistics do not need to change anything to use this feature. +* Block cache mid-point insertion, where index and filter block are inserted into LRU block cache with higher priority. The feature can be enabled by setting BlockBasedTableOptions::cache_index_and_filter_blocks_with_high_priority to true and high_pri_pool_ratio > 0 when creating NewLRUCache. + +## 4.11.0 (8/1/2016) +### Public API Change +* options.memtable_prefix_bloom_huge_page_tlb_size => memtable_huge_page_size. When it is set, RocksDB will try to allocate memory from huge page for memtable too, rather than just memtable bloom filter. + +### New Features +* A tool to migrate DB after options change. See include/rocksdb/utilities/option_change_migration.h. +* Add ReadOptions.background_purge_on_iterator_cleanup. If true, we avoid file deletion when destorying iterators. + +## 4.10.0 (7/5/2016) +### Public API Change +* options.memtable_prefix_bloom_bits changes to options.memtable_prefix_bloom_bits_ratio and deprecate options.memtable_prefix_bloom_probes +* enum type CompressionType and PerfLevel changes from char to unsigned char. Value of all PerfLevel shift by one. +* Deprecate options.filter_deletes. + +### New Features +* Add avoid_flush_during_recovery option. +* Add a read option background_purge_on_iterator_cleanup to avoid deleting files in foreground when destroying iterators. Instead, a job is scheduled in high priority queue and would be executed in a separate background thread. +* RepairDB support for column families. RepairDB now associates data with non-default column families using information embedded in the SST/WAL files (4.7 or later). For data written by 4.6 or earlier, RepairDB associates it with the default column family. +* Add options.write_buffer_manager which allows users to control total memtable sizes across multiple DB instances. + +## 4.9.0 (6/9/2016) +### Public API changes +* Add bottommost_compression option, This option can be used to set a specific compression algorithm for the bottommost level (Last level containing files in the DB). +* Introduce CompactionJobInfo::compression, This field state the compression algorithm used to generate the output files of the compaction. +* Deprecate BlockBaseTableOptions.hash_index_allow_collision=false +* Deprecate options builder (GetOptions()). + +### New Features +* Introduce NewSimCache() in rocksdb/utilities/sim_cache.h. This function creates a block cache that is able to give simulation results (mainly hit rate) of simulating block behavior with a configurable cache size. + +## 4.8.0 (5/2/2016) +### Public API Change +* Allow preset compression dictionary for improved compression of block-based tables. This is supported for zlib, zstd, and lz4. The compression dictionary's size is configurable via CompressionOptions::max_dict_bytes. +* Delete deprecated classes for creating backups (BackupableDB) and restoring from backups (RestoreBackupableDB). Now, BackupEngine should be used for creating backups, and BackupEngineReadOnly should be used for restorations. For more details, see https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB%3F +* Expose estimate of per-level compression ratio via DB property: "rocksdb.compression-ratio-at-levelN". +* Added EventListener::OnTableFileCreationStarted. EventListener::OnTableFileCreated will be called on failure case. User can check creation status via TableFileCreationInfo::status. + +### New Features +* Add ReadOptions::readahead_size. If non-zero, NewIterator will create a new table reader which performs reads of the given size. + +## 4.7.0 (4/8/2016) +### Public API Change +* rename options compaction_measure_io_stats to report_bg_io_stats and include flush too. +* Change some default options. Now default options will optimize for server-workloads. Also enable slowdown and full stop triggers for pending compaction bytes. These changes may cause sub-optimal performance or significant increase of resource usage. To avoid these risks, users can open existing RocksDB with options extracted from RocksDB option files. See https://github.com/facebook/rocksdb/wiki/RocksDB-Options-File for how to use RocksDB option files. Or you can call Options.OldDefaults() to recover old defaults. DEFAULT_OPTIONS_HISTORY.md will track change history of default options. + +## 4.6.0 (3/10/2016) +### Public API Changes +* Change default of BlockBasedTableOptions.format_version to 2. It means default DB created by 4.6 or up cannot be opened by RocksDB version 3.9 or earlier. +* Added strict_capacity_limit option to NewLRUCache. If the flag is set to true, insert to cache will fail if no enough capacity can be free. Signature of Cache::Insert() is updated accordingly. +* Tickers [NUMBER_DB_NEXT, NUMBER_DB_PREV, NUMBER_DB_NEXT_FOUND, NUMBER_DB_PREV_FOUND, ITER_BYTES_READ] are not updated immediately. The are updated when the Iterator is deleted. +* Add monotonically increasing counter (DB property "rocksdb.current-super-version-number") that increments upon any change to the LSM tree. + +### New Features +* Add CompactionPri::kMinOverlappingRatio, a compaction picking mode friendly to write amplification. +* Deprecate Iterator::IsKeyPinned() and replace it with Iterator::GetProperty() with prop_name="rocksdb.iterator.is.key.pinned" + +## 4.5.0 (2/5/2016) +### Public API Changes +* Add a new perf context level between kEnableCount and kEnableTime. Level 2 now does not include timers for mutexes. +* Statistics of mutex operation durations will not be measured by default. If you want to have them enabled, you need to set Statistics::stats_level_ to kAll. +* DBOptions::delete_scheduler and NewDeleteScheduler() are removed, please use DBOptions::sst_file_manager and NewSstFileManager() instead + +### New Features +* ldb tool now supports operations to non-default column families. +* Add kPersistedTier to ReadTier. This option allows Get and MultiGet to read only the persited data and skip mem-tables if writes were done with disableWAL = true. +* Add DBOptions::sst_file_manager. Use NewSstFileManager() in include/rocksdb/sst_file_manager.h to create a SstFileManager that can be used to track the total size of SST files and control the SST files deletion rate. + +## 4.4.0 (1/14/2016) +### Public API Changes +* Change names in CompactionPri and add a new one. +* Deprecate options.soft_rate_limit and add options.soft_pending_compaction_bytes_limit. +* If options.max_write_buffer_number > 3, writes will be slowed down when writing to the last write buffer to delay a full stop. +* Introduce CompactionJobInfo::compaction_reason, this field include the reason to trigger the compaction. +* After slow down is triggered, if estimated pending compaction bytes keep increasing, slowdown more. +* Increase default options.delayed_write_rate to 2MB/s. +* Added a new parameter --path to ldb tool. --path accepts the name of either MANIFEST, SST or a WAL file. Either --db or --path can be used when calling ldb. + +## 4.3.0 (12/8/2015) +### New Features +* CompactionFilter has new member function called IgnoreSnapshots which allows CompactionFilter to be called even if there are snapshots later than the key. +* RocksDB will now persist options under the same directory as the RocksDB database on successful DB::Open, CreateColumnFamily, DropColumnFamily, and SetOptions. +* Introduce LoadLatestOptions() in rocksdb/utilities/options_util.h. This function can construct the latest DBOptions / ColumnFamilyOptions used by the specified RocksDB intance. +* Introduce CheckOptionsCompatibility() in rocksdb/utilities/options_util.h. This function checks whether the input set of options is able to open the specified DB successfully. + +### Public API Changes +* When options.db_write_buffer_size triggers, only the column family with the largest column family size will be flushed, not all the column families. + +## 4.2.0 (11/9/2015) +### New Features +* Introduce CreateLoggerFromOptions(), this function create a Logger for provided DBOptions. +* Add GetAggregatedIntProperty(), which returns the sum of the GetIntProperty of all the column families. +* Add MemoryUtil in rocksdb/utilities/memory.h. It currently offers a way to get the memory usage by type from a list rocksdb instances. + +### Public API Changes +* CompactionFilter::Context includes information of Column Family ID +* The need-compaction hint given by TablePropertiesCollector::NeedCompact() will be persistent and recoverable after DB recovery. This introduces a breaking format change. If you use this experimental feature, including NewCompactOnDeletionCollectorFactory() in the new version, you may not be able to directly downgrade the DB back to version 4.0 or lower. +* TablePropertiesCollectorFactory::CreateTablePropertiesCollector() now takes an option Context, containing the information of column family ID for the file being written. +* Remove DefaultCompactionFilterFactory. + + +## 4.1.0 (10/8/2015) +### New Features +* Added single delete operation as a more efficient way to delete keys that have not been overwritten. +* Added experimental AddFile() to DB interface that allow users to add files created by SstFileWriter into an empty Database, see include/rocksdb/sst_file_writer.h and DB::AddFile() for more info. +* Added support for opening SST files with .ldb suffix which enables opening LevelDB databases. +* CompactionFilter now supports filtering of merge operands and merge results. + +### Public API Changes +* Added SingleDelete() to the DB interface. +* Added AddFile() to DB interface. +* Added SstFileWriter class. +* CompactionFilter has a new method FilterMergeOperand() that RocksDB applies to every merge operand during compaction to decide whether to filter the operand. +* We removed CompactionFilterV2 interfaces from include/rocksdb/compaction_filter.h. The functionality was deprecated already in version 3.13. + +## 4.0.0 (9/9/2015) +### New Features +* Added support for transactions. See include/rocksdb/utilities/transaction.h for more info. +* DB::GetProperty() now accepts "rocksdb.aggregated-table-properties" and "rocksdb.aggregated-table-properties-at-levelN", in which case it returns aggregated table properties of the target column family, or the aggregated table properties of the specified level N if the "at-level" version is used. +* Add compression option kZSTDNotFinalCompression for people to experiment ZSTD although its format is not finalized. +* We removed the need for LATEST_BACKUP file in BackupEngine. We still keep writing it when we create new backups (because of backward compatibility), but we don't read it anymore. + +### Public API Changes +* Removed class Env::RandomRWFile and Env::NewRandomRWFile(). +* Renamed DBOptions.num_subcompactions to DBOptions.max_subcompactions to make the name better match the actual functionality of the option. +* Added Equal() method to the Comparator interface that can optionally be overwritten in cases where equality comparisons can be done more efficiently than three-way comparisons. +* Previous 'experimental' OptimisticTransaction class has been replaced by Transaction class. + +## 3.13.0 (8/6/2015) +### New Features +* RollbackToSavePoint() in WriteBatch/WriteBatchWithIndex +* Add NewCompactOnDeletionCollectorFactory() in utilities/table_properties_collectors, which allows rocksdb to mark a SST file as need-compaction when it observes at least D deletion entries in any N consecutive entries in that SST file. Note that this feature depends on an experimental NeedCompact() API --- the result of this API will not persist after DB restart. +* Add DBOptions::delete_scheduler. Use NewDeleteScheduler() in include/rocksdb/delete_scheduler.h to create a DeleteScheduler that can be shared among multiple RocksDB instances to control the file deletion rate of SST files that exist in the first db_path. + +### Public API Changes +* Deprecated WriteOptions::timeout_hint_us. We no longer support write timeout. If you really need this option, talk to us and we might consider returning it. +* Deprecated purge_redundant_kvs_while_flush option. +* Removed BackupEngine::NewBackupEngine() and NewReadOnlyBackupEngine() that were deprecated in RocksDB 3.8. Please use BackupEngine::Open() instead. +* Deprecated Compaction Filter V2. We are not aware of any existing use-cases. If you use this filter, your compile will break with RocksDB 3.13. Please let us know if you use it and we'll put it back in RocksDB 3.14. +* Env::FileExists now returns a Status instead of a boolean +* Add statistics::getHistogramString() to print detailed distribution of a histogram metric. +* Add DBOptions::skip_stats_update_on_db_open. When it is on, DB::Open() will run faster as it skips the random reads required for loading necessary stats from SST files to optimize compaction. + +## 3.12.0 (7/2/2015) +### New Features +* Added experimental support for optimistic transactions. See include/rocksdb/utilities/optimistic_transaction.h for more info. +* Added a new way to report QPS from db_bench (check out --report_file and --report_interval_seconds) +* Added a cache for individual rows. See DBOptions::row_cache for more info. +* Several new features on EventListener (see include/rocksdb/listener.h): + - OnCompationCompleted() now returns per-compaction job statistics, defined in include/rocksdb/compaction_job_stats.h. + - Added OnTableFileCreated() and OnTableFileDeleted(). +* Add compaction_options_universal.enable_trivial_move to true, to allow trivial move while performing universal compaction. Trivial move will happen only when all the input files are non overlapping. + +### Public API changes +* EventListener::OnFlushCompleted() now passes FlushJobInfo instead of a list of parameters. +* DB::GetDbIdentity() is now a const function. If this function is overridden in your application, be sure to also make GetDbIdentity() const to avoid compile error. +* Move listeners from ColumnFamilyOptions to DBOptions. +* Add max_write_buffer_number_to_maintain option +* DB::CompactRange()'s parameter reduce_level is changed to change_level, to allow users to move levels to lower levels if allowed. It can be used to migrate a DB from options.level_compaction_dynamic_level_bytes=false to options.level_compaction_dynamic_level_bytes.true. +* Change default value for options.compaction_filter_factory and options.compaction_filter_factory_v2 to nullptr instead of DefaultCompactionFilterFactory and DefaultCompactionFilterFactoryV2. +* If CancelAllBackgroundWork is called without doing a flush after doing loads with WAL disabled, the changes which haven't been flushed before the call to CancelAllBackgroundWork will be lost. +* WBWIIterator::Entry() now returns WriteEntry instead of `const WriteEntry&` +* options.hard_rate_limit is deprecated. +* When options.soft_rate_limit or options.level0_slowdown_writes_trigger is triggered, the way to slow down writes is changed to: write rate to DB is limited to to options.delayed_write_rate. +* DB::GetApproximateSizes() adds a parameter to allow the estimation to include data in mem table, with default to be not to include. It is now only supported in skip list mem table. +* DB::CompactRange() now accept CompactRangeOptions instead of multiple parameters. CompactRangeOptions is defined in include/rocksdb/options.h. +* CompactRange() will now skip bottommost level compaction for level based compaction if there is no compaction filter, bottommost_level_compaction is introduced in CompactRangeOptions to control when it's possible to skip bottommost level compaction. This mean that if you want the compaction to produce a single file you need to set bottommost_level_compaction to BottommostLevelCompaction::kForce. +* Add Cache.GetPinnedUsage() to get the size of memory occupied by entries that are in use by the system. +* DB:Open() will fail if the compression specified in Options is not linked with the binary. If you see this failure, recompile RocksDB with compression libraries present on your system. Also, previously our default compression was snappy. This behavior is now changed. Now, the default compression is snappy only if it's available on the system. If it isn't we change the default to kNoCompression. +* We changed how we account for memory used in block cache. Previously, we only counted the sum of block sizes currently present in block cache. Now, we count the actual memory usage of the blocks. For example, a block of size 4.5KB will use 8KB memory with jemalloc. This might decrease your memory usage and possibly decrease performance. Increase block cache size if you see this happening after an upgrade. +* Add BackupEngineImpl.options_.max_background_operations to specify the maximum number of operations that may be performed in parallel. Add support for parallelized backup and restore. +* Add DB::SyncWAL() that does a WAL sync without blocking writers. + +## 3.11.0 (5/19/2015) +### New Features +* Added a new API Cache::SetCapacity(size_t capacity) to dynamically change the maximum configured capacity of the cache. If the new capacity is less than the existing cache usage, the implementation will try to lower the usage by evicting the necessary number of elements following a strict LRU policy. +* Added an experimental API for handling flashcache devices (blacklists background threads from caching their reads) -- NewFlashcacheAwareEnv +* If universal compaction is used and options.num_levels > 1, compact files are tried to be stored in none-L0 with smaller files based on options.target_file_size_base. The limitation of DB size when using universal compaction is greatly mitigated by using more levels. You can set num_levels = 1 to make universal compaction behave as before. If you set num_levels > 1 and want to roll back to a previous version, you need to compact all files to a big file in level 0 (by setting target_file_size_base to be large and CompactRange(, nullptr, nullptr, true, 0) and reopen the DB with the same version to rewrite the manifest, and then you can open it using previous releases. +* More information about rocksdb background threads are available in Env::GetThreadList(), including the number of bytes read / written by a compaction job, mem-table size and current number of bytes written by a flush job and many more. Check include/rocksdb/thread_status.h for more detail. + +### Public API changes +* TablePropertiesCollector::AddUserKey() is added to replace TablePropertiesCollector::Add(). AddUserKey() exposes key type, sequence number and file size up to now to users. +* DBOptions::bytes_per_sync used to apply to both WAL and table files. As of 3.11 it applies only to table files. If you want to use this option to sync WAL in the background, please use wal_bytes_per_sync + +## 3.10.0 (3/24/2015) +### New Features +* GetThreadStatus() is now able to report detailed thread status, including: + - Thread Operation including flush and compaction. + - The stage of the current thread operation. + - The elapsed time in micros since the current thread operation started. + More information can be found in include/rocksdb/thread_status.h. In addition, when running db_bench with --thread_status_per_interval, db_bench will also report thread status periodically. +* Changed the LRU caching algorithm so that referenced blocks (by iterators) are never evicted. This change made parameter removeScanCountLimit obsolete. Because of that NewLRUCache doesn't take three arguments anymore. table_cache_remove_scan_limit option is also removed +* By default we now optimize the compilation for the compilation platform (using -march=native). If you want to build portable binary, use 'PORTABLE=1' before the make command. +* We now allow level-compaction to place files in different paths by + specifying them in db_paths along with the target_size. + Lower numbered levels will be placed earlier in the db_paths and higher + numbered levels will be placed later in the db_paths vector. +* Potentially big performance improvements if you're using RocksDB with lots of column families (100-1000) +* Added BlockBasedTableOptions.format_version option, which allows user to specify which version of block based table he wants. As a general guideline, newer versions have more features, but might not be readable by older versions of RocksDB. +* Added new block based table format (version 2), which you can enable by setting BlockBasedTableOptions.format_version = 2. This format changes how we encode size information in compressed blocks and should help with memory allocations if you're using Zlib or BZip2 compressions. +* MemEnv (env that stores data in memory) is now available in default library build. You can create it by calling NewMemEnv(). +* Add SliceTransform.SameResultWhenAppended() to help users determine it is safe to apply prefix bloom/hash. +* Block based table now makes use of prefix bloom filter if it is a full fulter. +* Block based table remembers whether a whole key or prefix based bloom filter is supported in SST files. Do a sanity check when reading the file with users' configuration. +* Fixed a bug in ReadOnlyBackupEngine that deleted corrupted backups in some cases, even though the engine was ReadOnly +* options.level_compaction_dynamic_level_bytes, a feature to allow RocksDB to pick dynamic base of bytes for levels. With this feature turned on, we will automatically adjust max bytes for each level. The goal of this feature is to have lower bound on size amplification. For more details, see comments in options.h. +* Added an abstract base class WriteBatchBase for write batches +* Fixed a bug where we start deleting files of a dropped column families even if there are still live references to it + +### Public API changes +* Deprecated skip_log_error_on_recovery and table_cache_remove_scan_count_limit options. +* Logger method logv with log level parameter is now virtual + +### RocksJava +* Added compression per level API. +* MemEnv is now available in RocksJava via RocksMemEnv class. +* lz4 compression is now included in rocksjava static library when running `make rocksdbjavastatic`. +* Overflowing a size_t when setting rocksdb options now throws an IllegalArgumentException, which removes the necessity for a developer to catch these Exceptions explicitly. + +## 3.9.0 (12/8/2014) + +### New Features +* Add rocksdb::GetThreadList(), which in the future will return the current status of all + rocksdb-related threads. We will have more code instruments in the following RocksDB + releases. +* Change convert function in rocksdb/utilities/convenience.h to return Status instead of boolean. + Also add support for nested options in convert function + +### Public API changes +* New API to create a checkpoint added. Given a directory name, creates a new + database which is an image of the existing database. +* New API LinkFile added to Env. If you implement your own Env class, an + implementation of the API LinkFile will have to be provided. +* MemTableRep takes MemTableAllocator instead of Arena + +### Improvements +* RocksDBLite library now becomes smaller and will be compiled with -fno-exceptions flag. + +## 3.8.0 (11/14/2014) + +### Public API changes +* BackupEngine::NewBackupEngine() was deprecated; please use BackupEngine::Open() from now on. +* BackupableDB/RestoreBackupableDB have new GarbageCollect() methods, which will clean up files from corrupt and obsolete backups. +* BackupableDB/RestoreBackupableDB have new GetCorruptedBackups() methods which list corrupt backups. + +### Cleanup +* Bunch of code cleanup, some extra warnings turned on (-Wshadow, -Wshorten-64-to-32, -Wnon-virtual-dtor) + +### New features +* CompactFiles and EventListener, although they are still in experimental state +* Full ColumnFamily support in RocksJava. + +## 3.7.0 (11/6/2014) +### Public API changes +* Introduce SetOptions() API to allow adjusting a subset of options dynamically online +* Introduce 4 new convenient functions for converting Options from string: GetColumnFamilyOptionsFromMap(), GetColumnFamilyOptionsFromString(), GetDBOptionsFromMap(), GetDBOptionsFromString() +* Remove WriteBatchWithIndex.Delete() overloads using SliceParts +* When opening a DB, if options.max_background_compactions is larger than the existing low pri pool of options.env, it will enlarge it. Similarly, options.max_background_flushes is larger than the existing high pri pool of options.env, it will enlarge it. + +## 3.6.0 (10/7/2014) +### Disk format changes +* If you're using RocksDB on ARM platforms and you're using default bloom filter, there is a disk format change you need to be aware of. There are three steps you need to do when you convert to new release: 1. turn off filter policy, 2. compact the whole database, 3. turn on filter policy + +### Behavior changes +* We have refactored our system of stalling writes. Any stall-related statistics' meanings are changed. Instead of per-write stall counts, we now count stalls per-epoch, where epochs are periods between flushes and compactions. You'll find more information in our Tuning Perf Guide once we release RocksDB 3.6. +* When disableDataSync=true, we no longer sync the MANIFEST file. +* Add identity_as_first_hash property to CuckooTable. SST file needs to be rebuilt to be opened by reader properly. + +### Public API changes +* Change target_file_size_base type to uint64_t from int. +* Remove allow_thread_local. This feature was proved to be stable, so we are turning it always-on. ## 3.5.0 (9/3/2014) ### New Features -* Add include/utilities/write_batch_with_index.h, providing a utilitiy class to query data out of WriteBatch when building it. +* Add include/utilities/write_batch_with_index.h, providing a utility class to query data out of WriteBatch when building it. * Move BlockBasedTable related options to BlockBasedTableOptions from Options. Change corresponding JNI interface. Options affected include: no_block_cache, block_cache, block_cache_compressed, block_size, block_size_deviation, block_restart_interval, filter_policy, whole_key_filtering. filter_policy is changed to shared_ptr from a raw pointer. * Remove deprecated options: disable_seek_compaction and db_stats_log_interval @@ -20,6 +461,7 @@ * Support Multiple DB paths in universal style compactions * Add feature of storing plain table index and bloom filter in SST file. * CompactRange() will never output compacted files to level 0. This used to be the case when all the compaction input files were at level 0. +* Added iterate_upper_bound to define the extent upto which the forward iterator will return entries. This will prevent iterating over delete markers and overwritten entries for edge cases where you want to break out the iterator anyways. This may improve performance in case there are a large number of delete markers or overwritten entries. ### Public API changes * DBOptions.db_paths now is a vector of a DBPath structure which indicates both of path and target size @@ -34,7 +476,7 @@ ### New Features * Added JSON API prototype. * HashLinklist reduces performance outlier caused by skewed bucket by switching data in the bucket from linked list to skip list. Add parameter threshold_use_skiplist in NewHashLinkListRepFactory(). -* RocksDB is now able to reclaim storage space more effectively during the compaction process. This is done by compensating the size of each deletion entry by the 2X average value size, which makes compaction to be triggerred by deletion entries more easily. +* RocksDB is now able to reclaim storage space more effectively during the compaction process. This is done by compensating the size of each deletion entry by the 2X average value size, which makes compaction to be triggered by deletion entries more easily. * Add TimeOut API to write. Now WriteOptions have a variable called timeout_hint_us. With timeout_hint_us set to non-zero, any write associated with this timeout_hint_us may be aborted when it runs longer than the specified timeout_hint_us, and it is guaranteed that any write completes earlier than the specified time-out will not be aborted due to the time-out condition. * Add a rate_limiter option, which controls total throughput of flush and compaction. The throughput is specified in bytes/sec. Flush always has precedence over compaction when available bandwidth is constrained. @@ -49,11 +491,11 @@ 2) It added some complexity to the important code-paths, 3) None of our internal customers were really using it. Because of that, Options::disable_seek_compaction is now obsolete. It is still a parameter in Options, so it does not break the build, but it does not have any effect. We plan to completely remove it at some point, so we ask users to please remove this option from your code base. -* Add two paramters to NewHashLinkListRepFactory() for logging on too many entries in a hash bucket when flushing. +* Add two parameters to NewHashLinkListRepFactory() for logging on too many entries in a hash bucket when flushing. * Added new option BlockBasedTableOptions::hash_index_allow_collision. When enabled, prefix hash index for block-based table will not store prefix and allow hash collision, reducing memory consumption. ### New Features -* PlainTable now supports a new key encoding: for keys of the same prefix, the prefix is only written once. It can be enabled through encoding_type paramter of NewPlainTableFactory() +* PlainTable now supports a new key encoding: for keys of the same prefix, the prefix is only written once. It can be enabled through encoding_type parameter of NewPlainTableFactory() * Add AdaptiveTableFactory, which is used to convert from a DB of PlainTable to BlockBasedTabe, or vise versa. It can be created using NewAdaptiveTableFactory() ### Performance Improvements diff --git a/INSTALL.md b/INSTALL.md index 8cf66e6ab24..04f0eb27976 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,37 +1,52 @@ ## Compilation +**Important**: If you plan to run RocksDB in production, don't compile using default +`make` or `make all`. That will compile RocksDB in debug mode, which is much slower +than release mode. + RocksDB's library should be able to compile without any dependency installed, although we recommend installing some compression libraries (see below). -We do depend on newer gcc with C++11 support. +We do depend on newer gcc/clang with C++11 support. There are few options when compiling RocksDB: -* [recommended] `make static_lib` will compile librocksdb.a, RocksDB static library. +* [recommended] `make static_lib` will compile librocksdb.a, RocksDB static library. Compiles static library in release mode. -* `make shared_lib` will compile librocksdb.so, RocksDB shared library. +* `make shared_lib` will compile librocksdb.so, RocksDB shared library. Compiles shared library in release mode. -* `make check` will compile and run all the unit tests +* `make check` will compile and run all the unit tests. `make check` will compile RocksDB in debug mode. * `make all` will compile our static library, and all our tools and unit tests. Our tools -depend on gflags. You will need to have gflags installed to run `make all`. +depend on gflags. You will need to have gflags installed to run `make all`. This will compile RocksDB in debug mode. Don't +use binaries compiled by `make all` in production. + +* By default the binary we produce is optimized for the platform you're compiling on +(`-march=native` or the equivalent). SSE4.2 will thus be enabled automatically if your +CPU supports it. To print a warning if your CPU does not support SSE4.2, build with +`USE_SSE=1 make static_lib` or, if using CMake, `cmake -DFORCE_SSE42=ON`. If you want +to build a portable binary, add `PORTABLE=1` before your make commands, like this: +`PORTABLE=1 make static_lib`. ## Dependencies * You can link RocksDB with following compression libraries: - [zlib](http://www.zlib.net/) - a library for data compression. - [bzip2](http://www.bzip.org/) - a library for data compression. - - [snappy](https://code.google.com/p/snappy/) - a library for fast + - [lz4](https://github.com/lz4/lz4) - a library for extremely fast data compression. + - [snappy](http://google.github.io/snappy/) - a library for fast data compression. + - [zstandard](http://www.zstd.net) - Fast real-time compression + algorithm. * All our tools depend on: - - [gflags](https://code.google.com/p/gflags/) - a library that handles + - [gflags](https://gflags.github.io/gflags/) - a library that handles command line flags processing. You can compile rocksdb library even if you don't have gflags installed. ## Supported platforms * **Linux - Ubuntu** - * Upgrade your gcc to version at least 4.7 to get C++11 support. + * Upgrade your gcc to version at least 4.8 to get C++11 support. * Install gflags. First, try: `sudo apt-get install libgflags-dev` If this doesn't work and you're using Ubuntu, here's a nice tutorial: (http://askubuntu.com/questions/312173/installing-gflags-12-04) @@ -39,46 +54,91 @@ depend on gflags. You will need to have gflags installed to run `make all`. `sudo apt-get install libsnappy-dev`. * Install zlib. Try: `sudo apt-get install zlib1g-dev`. * Install bzip2: `sudo apt-get install libbz2-dev`. -* **Linux - CentOS** - * Upgrade your gcc to version at least 4.7 to get C++11 support: - `yum install gcc47-c++` + * Install lz4: `sudo apt-get install liblz4-dev`. + * Install zstandard: `sudo apt-get install libzstd-dev`. + +* **Linux - CentOS / RHEL** + * Upgrade your gcc to version at least 4.8 to get C++11 support: + `yum install gcc48-c++` * Install gflags: - wget https://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz - tar -xzvf gflags-2.0-no-svn-files.tar.gz - cd gflags-2.0 + git clone https://github.com/gflags/gflags.git + cd gflags + git checkout v2.0 ./configure && make && sudo make install + **Notice**: Once installed, please add the include path for gflags to your `CPATH` environment variable and the + lib path to `LIBRARY_PATH`. If installed with default settings, the include path will be `/usr/local/include` + and the lib path will be `/usr/local/lib`. + * Install snappy: - wget https://snappy.googlecode.com/files/snappy-1.1.1.tar.gz - tar -xzvf snappy-1.1.1.tar.gz - cd snappy-1.1.1 - ./configure && make && sudo make install + sudo yum install snappy snappy-devel * Install zlib: - sudo yum install zlib - sudo yum install zlib-devel + sudo yum install zlib zlib-devel * Install bzip2: - sudo yum install bzip2 - sudo yum install bzip2-devel + sudo yum install bzip2 bzip2-devel + + * Install lz4: + + sudo yum install lz4-devel + + * Install ASAN (optional for debugging): + + sudo yum install libasan + + * Install zstandard: + + wget https://github.com/facebook/zstd/archive/v1.1.3.tar.gz + mv v1.1.3.tar.gz zstd-1.1.3.tar.gz + tar zxvf zstd-1.1.3.tar.gz + cd zstd-1.1.3 + make && sudo make install * **OS X**: * Install latest C++ compiler that supports C++ 11: * Update XCode: run `xcode-select --install` (or install it from XCode App's settting). * Install via [homebrew](http://brew.sh/). * If you're first time developer in MacOS, you still need to run: `xcode-select --install` in your command line. - * run `brew tap homebrew/dupes; brew install gcc47 --use-llvm` to install gcc 4.7 (or higher). - * Install zlib, bzip2 and snappy libraries for compression. - * Install gflags. We have included a script - `build_tools/mac-install-gflags.sh`, which should automatically install it (execute this file instead of runing using "source" command). - If you installed gflags by other means (for example, `brew install gflags`), - please set `LIBRARY_PATH` and `CPATH` accordingly. - * Please note that some of the optimizations/features are disabled in OSX. - We did not run any production workloads on it. + * run `brew tap homebrew/versions; brew install gcc48 --use-llvm` to install gcc 4.8 (or higher). + * run `brew install rocksdb` * **iOS**: - * Run: `TARGET_OS=IOS make static_lib` + * Run: `TARGET_OS=IOS make static_lib`. When building the project which uses rocksdb iOS library, make sure to define two important pre-processing macros: `ROCKSDB_LITE` and `IOS_CROSS_COMPILE`. + +* **Windows**: + * For building with MS Visual Studio 13 you will need Update 4 installed. + * Read and follow the instructions at CMakeLists.txt + * Or install via [vcpkg](https://github.com/microsoft/vcpkg) + * run `vcpkg install rocksdb` + +* **AIX 6.1** + * Install AIX Toolbox rpms with gcc + * Use these environment variables: + + export PORTABLE=1 + export CC=gcc + export AR="ar -X64" + export EXTRA_ARFLAGS=-X64 + export EXTRA_CFLAGS=-maix64 + export EXTRA_CXXFLAGS=-maix64 + export PLATFORM_LDFLAGS="-static-libstdc++ -static-libgcc" + export LIBPATH=/opt/freeware/lib + export JAVA_HOME=/usr/java8_64 + export PATH=/opt/freeware/bin:$PATH + +* **Solaris Sparc** + * Install GCC 4.8.2 and higher. + * Use these environment variables: + + export CC=gcc + export EXTRA_CFLAGS=-m64 + export EXTRA_CXXFLAGS=-m64 + export EXTRA_LDFLAGS=-m64 + export PORTABLE=1 + export PLATFORM_LDFLAGS="-static-libstdc++ -static-libgcc" + diff --git a/LANGUAGE-BINDINGS.md b/LANGUAGE-BINDINGS.md new file mode 100644 index 00000000000..ffeed98f28d --- /dev/null +++ b/LANGUAGE-BINDINGS.md @@ -0,0 +1,16 @@ +This is the list of all known third-party language bindings for RocksDB. If something is missing, please open a pull request to add it. + +* Java - https://github.com/facebook/rocksdb/tree/master/java +* Python - http://pyrocksdb.readthedocs.org/en/latest/ +* Perl - https://metacpan.org/pod/RocksDB +* Node.js - https://npmjs.org/package/rocksdb +* Go - https://github.com/tecbot/gorocksdb +* Ruby - http://rubygems.org/gems/rocksdb-ruby +* Haskell - https://hackage.haskell.org/package/rocksdb-haskell +* PHP - https://github.com/Photonios/rocksdb-php +* C# - https://github.com/warrenfalk/rocksdb-sharp +* Rust + * https://github.com/spacejam/rust-rocksdb + * https://github.com/bh1xuw/rust-rocks +* D programming language - https://github.com/b1naryth1ef/rocksdb +* Erlang - https://gitlab.com/barrel-db/erlang-rocksdb diff --git a/LICENSE.Apache b/LICENSE.Apache new file mode 100644 index 00000000000..d6456956733 --- /dev/null +++ b/LICENSE.Apache @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/LICENSE b/LICENSE.leveldb similarity index 90% rename from LICENSE rename to LICENSE.leveldb index b1329018690..7108b0bfba7 100644 --- a/LICENSE +++ b/LICENSE.leveldb @@ -1,10 +1,4 @@ -BSD License - -For rocksdb software - -Copyright (c) 2014, Facebook, Inc. -All rights reserved. ---------------------------------------------------------------------- +This contains code that is from LevelDB, and that code is under the following license: Copyright (c) 2011 The LevelDB Authors. All rights reserved. diff --git a/Makefile b/Makefile index 0e969e0fdd0..5a89f6bf79d 100644 --- a/Makefile +++ b/Makefile @@ -6,28 +6,170 @@ #----------------------------------------------- -ifneq ($(MAKECMDGOALS),dbg) -OPT += -O2 -fno-omit-frame-pointer -momit-leaf-frame-pointer -else -# intentionally left blank +BASH_EXISTS := $(shell which bash) +SHELL := $(shell which bash) + +CLEAN_FILES = # deliberately empty, so we can append below. +CFLAGS += ${EXTRA_CFLAGS} +CXXFLAGS += ${EXTRA_CXXFLAGS} +LDFLAGS += $(EXTRA_LDFLAGS) +MACHINE ?= $(shell uname -m) +ARFLAGS = ${EXTRA_ARFLAGS} rs +STRIPFLAGS = -S -x + +# Transform parallel LOG output into something more readable. +perl_command = perl -n \ + -e '@a=split("\t",$$_,-1); $$t=$$a[8];' \ + -e '$$t =~ /.*if\s\[\[\s"(.*?\.[\w\/]+)/ and $$t=$$1;' \ + -e '$$t =~ s,^\./,,;' \ + -e '$$t =~ s, >.*,,; chomp $$t;' \ + -e '$$t =~ /.*--gtest_filter=(.*?\.[\w\/]+)/ and $$t=$$1;' \ + -e 'printf "%7.3f %s %s\n", $$a[3], $$a[6] == 0 ? "PASS" : "FAIL", $$t' +quoted_perl_command = $(subst ','\'',$(perl_command)) + +# DEBUG_LEVEL can have three values: +# * DEBUG_LEVEL=2; this is the ultimate debug mode. It will compile rocksdb +# without any optimizations. To compile with level 2, issue `make dbg` +# * DEBUG_LEVEL=1; debug level 1 enables all assertions and debug code, but +# compiles rocksdb with -O2 optimizations. this is the default debug level. +# `make all` or `make ` compile RocksDB with debug level 1. +# We use this debug level when developing RocksDB. +# * DEBUG_LEVEL=0; this is the debug level we use for release. If you're +# running rocksdb in production you most definitely want to compile RocksDB +# with debug level 0. To compile with level 0, run `make shared_lib`, +# `make install-shared`, `make static_lib`, `make install-static` or +# `make install` + +# Set the default DEBUG_LEVEL to 1 +DEBUG_LEVEL?=1 + +ifeq ($(MAKECMDGOALS),dbg) + DEBUG_LEVEL=2 +endif + +ifeq ($(MAKECMDGOALS),clean) + DEBUG_LEVEL=0 +endif + +ifeq ($(MAKECMDGOALS),release) + DEBUG_LEVEL=0 endif ifeq ($(MAKECMDGOALS),shared_lib) -OPT += -DNDEBUG + DEBUG_LEVEL=0 +endif + +ifeq ($(MAKECMDGOALS),install-shared) + DEBUG_LEVEL=0 endif ifeq ($(MAKECMDGOALS),static_lib) + DEBUG_LEVEL=0 +endif + +ifeq ($(MAKECMDGOALS),install-static) + DEBUG_LEVEL=0 +endif + +ifeq ($(MAKECMDGOALS),install) + DEBUG_LEVEL=0 +endif + +ifeq ($(MAKECMDGOALS),rocksdbjavastatic) + DEBUG_LEVEL=0 +endif + +ifeq ($(MAKECMDGOALS),rocksdbjavastaticrelease) + DEBUG_LEVEL=0 +endif + +ifeq ($(MAKECMDGOALS),rocksdbjavastaticpublish) + DEBUG_LEVEL=0 +endif + +# compile with -O2 if debug level is not 2 +ifneq ($(DEBUG_LEVEL), 2) +OPT += -O2 -fno-omit-frame-pointer +# Skip for archs that don't support -momit-leaf-frame-pointer +ifeq (,$(shell $(CXX) -fsyntax-only -momit-leaf-frame-pointer -xc /dev/null 2>&1)) +OPT += -momit-leaf-frame-pointer +endif +endif + +# if we're compiling for release, compile without debug code (-DNDEBUG) and +# don't treat warnings as errors +ifeq ($(DEBUG_LEVEL),0) OPT += -DNDEBUG +DISABLE_WARNING_AS_ERROR=1 + +ifneq ($(USE_RTTI), 1) + CXXFLAGS += -fno-rtti +else + CXXFLAGS += -DROCKSDB_USE_RTTI +endif +else +ifneq ($(USE_RTTI), 0) + CXXFLAGS += -DROCKSDB_USE_RTTI +else + CXXFLAGS += -fno-rtti +endif + +$(warning Warning: Compiling in debug mode. Don't use the resulting binary in production) endif #----------------------------------------------- +include src.mk + +AM_DEFAULT_VERBOSITY = 0 + +AM_V_GEN = $(am__v_GEN_$(V)) +am__v_GEN_ = $(am__v_GEN_$(AM_DEFAULT_VERBOSITY)) +am__v_GEN_0 = @echo " GEN " $@; +am__v_GEN_1 = +AM_V_at = $(am__v_at_$(V)) +am__v_at_ = $(am__v_at_$(AM_DEFAULT_VERBOSITY)) +am__v_at_0 = @ +am__v_at_1 = + +AM_V_CC = $(am__v_CC_$(V)) +am__v_CC_ = $(am__v_CC_$(AM_DEFAULT_VERBOSITY)) +am__v_CC_0 = @echo " CC " $@; +am__v_CC_1 = +CCLD = $(CC) +LINK = $(CCLD) $(AM_CFLAGS) $(CFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ +AM_V_CCLD = $(am__v_CCLD_$(V)) +am__v_CCLD_ = $(am__v_CCLD_$(AM_DEFAULT_VERBOSITY)) +am__v_CCLD_0 = @echo " CCLD " $@; +am__v_CCLD_1 = +AM_V_AR = $(am__v_AR_$(V)) +am__v_AR_ = $(am__v_AR_$(AM_DEFAULT_VERBOSITY)) +am__v_AR_0 = @echo " AR " $@; +am__v_AR_1 = + +ifdef ROCKSDB_USE_LIBRADOS +LIB_SOURCES += utilities/env_librados.cc +LDFLAGS += -lrados +endif +AM_LINK = $(AM_V_CCLD)$(CXX) $^ $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) # detect what platform we're building on -$(shell (export ROCKSDB_ROOT="$(CURDIR)"; "$(CURDIR)/build_tools/build_detect_platform" "$(CURDIR)/build_config.mk")) +dummy := $(shell (export ROCKSDB_ROOT="$(CURDIR)"; export PORTABLE="$(PORTABLE)"; "$(CURDIR)/build_tools/build_detect_platform" "$(CURDIR)/make_config.mk")) # this file is generated by the previous line to set build flags and sources -include build_config.mk +include make_config.mk +CLEAN_FILES += make_config.mk -ifneq ($(PLATFORM), IOS) +missing_make_config_paths := $(shell \ + grep "\/\S*" -o $(CURDIR)/make_config.mk | \ + while read path; \ + do [ -e $$path ] || echo $$path; \ + done | sort | uniq) + +$(foreach path, $(missing_make_config_paths), \ + $(warning Warning: $(path) dont exist)) + +ifeq ($(PLATFORM), OS_AIX) +# no debug info +else ifneq ($(PLATFORM), IOS) CFLAGS += -g CXXFLAGS += -g else @@ -35,132 +177,364 @@ else OPT += -DNDEBUG endif +ifeq ($(PLATFORM), OS_AIX) +ARFLAGS = -X64 rs +STRIPFLAGS = -X64 -x +endif + +ifeq ($(PLATFORM), OS_SOLARIS) + PLATFORM_CXXFLAGS += -D _GLIBCXX_USE_C99 +endif +ifneq ($(filter -DROCKSDB_LITE,$(OPT)),) + # found + CFLAGS += -fno-exceptions + CXXFLAGS += -fno-exceptions + # LUA is not supported under ROCKSDB_LITE + LUA_PATH = +endif + # ASAN doesn't work well with jemalloc. If we're compiling with ASAN, we should use regular malloc. ifdef COMPILE_WITH_ASAN - # ASAN compile flags + DISABLE_JEMALLOC=1 EXEC_LDFLAGS += -fsanitize=address PLATFORM_CCFLAGS += -fsanitize=address PLATFORM_CXXFLAGS += -fsanitize=address -else - # if we're not compiling with ASAN, use jemalloc +endif + +# TSAN doesn't work well with jemalloc. If we're compiling with TSAN, we should use regular malloc. +ifdef COMPILE_WITH_TSAN + DISABLE_JEMALLOC=1 + EXEC_LDFLAGS += -fsanitize=thread + PLATFORM_CCFLAGS += -fsanitize=thread -fPIC + PLATFORM_CXXFLAGS += -fsanitize=thread -fPIC + # Turn off -pg when enabling TSAN testing, because that induces + # a link failure. TODO: find the root cause + PROFILING_FLAGS = + # LUA is not supported under TSAN + LUA_PATH = +endif + +# AIX doesn't work with -pg +ifeq ($(PLATFORM), OS_AIX) + PROFILING_FLAGS = +endif + +# USAN doesn't work well with jemalloc. If we're compiling with USAN, we should use regular malloc. +ifdef COMPILE_WITH_UBSAN + DISABLE_JEMALLOC=1 + EXEC_LDFLAGS += -fsanitize=undefined + PLATFORM_CCFLAGS += -fsanitize=undefined -DROCKSDB_UBSAN_RUN + PLATFORM_CXXFLAGS += -fsanitize=undefined -DROCKSDB_UBSAN_RUN +endif + +ifndef DISABLE_JEMALLOC + ifdef JEMALLOC + PLATFORM_CXXFLAGS += -DROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE + PLATFORM_CCFLAGS += -DROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE + endif + ifdef WITH_JEMALLOC_FLAG + PLATFORM_LDFLAGS += -ljemalloc + JAVA_LDFLAGS += -ljemalloc + endif EXEC_LDFLAGS := $(JEMALLOC_LIB) $(EXEC_LDFLAGS) - PLATFORM_CXXFLAGS += $(JEMALLOC_INCLUDE) -DHAVE_JEMALLOC - PLATFORM_CCFLAGS += $(JEMALLOC_INCLUDE) -DHAVE_JEMALLOC + PLATFORM_CXXFLAGS += $(JEMALLOC_INCLUDE) + PLATFORM_CCFLAGS += $(JEMALLOC_INCLUDE) endif -#------------------------------------------------- -# make install related stuff -INSTALL_PATH ?= /usr/local +export GTEST_THROW_ON_FAILURE=1 +export GTEST_HAS_EXCEPTIONS=1 +GTEST_DIR = ./third-party/gtest-1.7.0/fused-src +# AIX: pre-defined system headers are surrounded by an extern "C" block +ifeq ($(PLATFORM), OS_AIX) + PLATFORM_CCFLAGS += -I$(GTEST_DIR) + PLATFORM_CXXFLAGS += -I$(GTEST_DIR) +else + PLATFORM_CCFLAGS += -isystem $(GTEST_DIR) + PLATFORM_CXXFLAGS += -isystem $(GTEST_DIR) +endif -uninstall: - @rm -rf $(INSTALL_PATH)/include/rocksdb - @rm -rf $(INSTALL_PATH)/lib/$(LIBRARY) - @rm -rf $(INSTALL_PATH)/lib/$(SHARED) +# This (the first rule) must depend on "all". +default: all + +WARNING_FLAGS = -W -Wextra -Wall -Wsign-compare -Wshadow \ + -Wno-unused-parameter + +ifndef DISABLE_WARNING_AS_ERROR + WARNING_FLAGS += -Werror +endif + + +ifdef LUA_PATH + +ifndef LUA_INCLUDE +LUA_INCLUDE=$(LUA_PATH)/include +endif + +LUA_INCLUDE_FILE=$(LUA_INCLUDE)/lualib.h + +ifeq ("$(wildcard $(LUA_INCLUDE_FILE))", "") +# LUA_INCLUDE_FILE does not exist +$(error Cannot find lualib.h under $(LUA_INCLUDE). Try to specify both LUA_PATH and LUA_INCLUDE manually) +endif +LUA_FLAGS = -I$(LUA_INCLUDE) -DLUA -DLUA_COMPAT_ALL +CFLAGS += $(LUA_FLAGS) +CXXFLAGS += $(LUA_FLAGS) + +ifndef LUA_LIB +LUA_LIB = $(LUA_PATH)/lib/liblua.a +endif +ifeq ("$(wildcard $(LUA_LIB))", "") # LUA_LIB does not exist +$(error $(LUA_LIB) does not exist. Try to specify both LUA_PATH and LUA_LIB manually) +endif +LDFLAGS += $(LUA_LIB) + +endif -install: - @install -d $(INSTALL_PATH)/lib - @for header_dir in `find "include/rocksdb" -type d`; do \ - install -d $(INSTALL_PATH)/$$header_dir; \ - done - @for header in `find "include/rocksdb" -type f -name *.h`; do \ - install -C -m 644 $$header $(INSTALL_PATH)/$$header; \ - done - @[ ! -e $(LIBRARY) ] || install -C -m 644 $(LIBRARY) $(INSTALL_PATH)/lib - @[ ! -e $(SHARED) ] || install -C -m 644 $(SHARED) $(INSTALL_PATH)/lib -#------------------------------------------------- -WARNING_FLAGS = -Wall -Werror -Wsign-compare CFLAGS += $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CCFLAGS) $(OPT) -CXXFLAGS += $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CXXFLAGS) $(OPT) -Woverloaded-virtual +CXXFLAGS += $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CXXFLAGS) $(OPT) -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers LDFLAGS += $(PLATFORM_LDFLAGS) -LIBOBJECTS = $(SOURCES:.cc=.o) -LIBOBJECTS += $(SOURCESCPP:.cpp=.o) -MEMENVOBJECTS = $(MEMENV_SOURCES:.cc=.o) +# If NO_UPDATE_BUILD_VERSION is set we don't update util/build_version.cc, but +# the file needs to already exist or else the build will fail +ifndef NO_UPDATE_BUILD_VERSION +date := $(shell date +%F) +ifdef FORCE_GIT_SHA + git_sha := $(FORCE_GIT_SHA) +else + git_sha := $(shell git rev-parse HEAD 2>/dev/null) +endif +gen_build_version = sed -e s/@@GIT_SHA@@/$(git_sha)/ -e s/@@GIT_DATE_TIME@@/$(date)/ util/build_version.cc.in + +# Record the version of the source that we are compiling. +# We keep a record of the git revision in this file. It is then built +# as a regular source file as part of the compilation process. +# One can run "strings executable_filename | grep _build_" to find +# the version of the source that we used to build the executable file. +FORCE: +util/build_version.cc: FORCE + $(AM_V_GEN)rm -f $@-t + $(AM_V_at)$(gen_build_version) > $@-t + $(AM_V_at)if test -f $@; then \ + cmp -s $@-t $@ && rm -f $@-t || mv -f $@-t $@; \ + else mv -f $@-t $@; fi +endif +LIBOBJECTS = $(LIB_SOURCES:.cc=.o) +LIBOBJECTS += $(TOOL_LIB_SOURCES:.cc=.o) +MOCKOBJECTS = $(MOCK_LIB_SOURCES:.cc=.o) + +GTEST = $(GTEST_DIR)/gtest/gtest-all.o TESTUTIL = ./util/testutil.o -TESTHARNESS = ./util/testharness.o $(TESTUTIL) -BENCHHARNESS = ./util/benchharness.o +TESTHARNESS = ./util/testharness.o $(TESTUTIL) $(MOCKOBJECTS) $(GTEST) VALGRIND_ERROR = 2 -VALGRIND_DIR = build_tools/VALGRIND_LOGS VALGRIND_VER := $(join $(VALGRIND_VER),valgrind) + VALGRIND_OPTS = --error-exitcode=$(VALGRIND_ERROR) --leak-check=full +BENCHTOOLOBJECTS = $(BENCH_LIB_SOURCES:.cc=.o) $(LIBOBJECTS) $(TESTUTIL) + +EXPOBJECTS = $(EXP_LIB_SOURCES:.cc=.o) $(LIBOBJECTS) $(TESTUTIL) + TESTS = \ + db_basic_test \ + db_encryption_test \ + db_test2 \ + external_sst_file_basic_test \ + auto_roll_logger_test \ + bloom_test \ + dynamic_bloom_test \ + c_test \ + checkpoint_test \ + crc32c_test \ + coding_test \ + inlineskiplist_test \ + env_basic_test \ + env_test \ + hash_test \ + thread_local_test \ + rate_limiter_test \ + perf_context_test \ + iostats_context_test \ + db_wal_test \ + db_block_cache_test \ db_test \ + db_blob_index_test \ + db_bloom_filter_test \ db_iter_test \ - block_hash_index_test \ + db_log_iter_test \ + db_compaction_filter_test \ + db_compaction_test \ + db_dynamic_level_test \ + db_flush_test \ + db_inplace_update_test \ + db_iterator_test \ + db_memtable_test \ + db_merge_operator_test \ + db_options_test \ + db_range_del_test \ + db_sst_test \ + db_tailing_iter_test \ + db_universal_compaction_test \ + db_io_failure_test \ + db_properties_test \ + db_table_properties_test \ + db_statistics_test \ + db_write_test \ autovector_test \ + blob_db_test \ + cleanable_test \ column_family_test \ table_properties_collector_test \ arena_test \ - auto_roll_logger_test \ - benchharness_test \ block_test \ - bloom_test \ - dynamic_bloom_test \ - c_test \ cache_test \ - coding_test \ corruption_test \ - crc32c_test \ + slice_transform_test \ dbformat_test \ - env_test \ - blob_store_test \ + fault_injection_test \ filelock_test \ filename_test \ - filter_block_test \ + file_reader_writer_test \ + block_based_filter_block_test \ + full_filter_block_test \ + partitioned_filter_block_test \ + hash_table_test \ histogram_test \ log_test \ manual_compaction_test \ - memenv_test \ + mock_env_test \ + memtable_list_test \ + merge_helper_test \ + memory_test \ merge_test \ + merger_test \ + util_merge_operators_test \ + options_file_test \ redis_test \ reduce_levels_test \ plain_table_db_test \ + comparator_db_test \ + external_sst_file_test \ prefix_test \ - simple_table_db_test \ skiplist_test \ + write_buffer_manager_test \ stringappend_test \ + cassandra_format_test \ + cassandra_functional_test \ + cassandra_row_merge_test \ + cassandra_serialize_test \ ttl_test \ + date_tiered_test \ backupable_db_test \ document_db_test \ json_document_test \ + sim_cache_test \ spatial_db_test \ version_edit_test \ version_set_test \ + compaction_picker_test \ + version_builder_test \ file_indexer_test \ - write_batch_test\ + write_batch_test \ + write_batch_with_index_test \ + write_controller_test\ deletefile_test \ table_test \ - thread_local_test \ geodb_test \ - rate_limiter_test \ + delete_scheduler_test \ options_test \ + options_settable_test \ + options_util_test \ + event_logger_test \ + timer_queue_test \ cuckoo_table_builder_test \ cuckoo_table_reader_test \ cuckoo_table_db_test \ - write_batch_with_index_test + flush_job_test \ + wal_manager_test \ + listener_test \ + compaction_iterator_test \ + compaction_job_test \ + thread_list_test \ + sst_dump_test \ + column_aware_encoding_test \ + compact_files_test \ + optimistic_transaction_test \ + write_callback_test \ + heap_test \ + compact_on_deletion_collector_test \ + compaction_job_stats_test \ + option_change_migration_test \ + transaction_test \ + ldb_cmd_test \ + persistent_cache_test \ + statistics_test \ + lua_test \ + range_del_aggregator_test \ + lru_cache_test \ + object_registry_test \ + repair_test \ + env_timed_test \ + +PARALLEL_TEST = \ + backupable_db_test \ + db_compaction_filter_test \ + db_compaction_test \ + db_sst_test \ + db_test \ + db_universal_compaction_test \ + db_wal_test \ + external_sst_file_test \ + fault_injection_test \ + inlineskiplist_test \ + manual_compaction_test \ + persistent_cache_test \ + table_test \ + transaction_test + +SUBSET := $(TESTS) +ifdef ROCKSDBTESTS_START + SUBSET := $(shell echo $(SUBSET) | sed 's/^.*$(ROCKSDBTESTS_START)/$(ROCKSDBTESTS_START)/') +endif + +ifdef ROCKSDBTESTS_END + SUBSET := $(shell echo $(SUBSET) | sed 's/$(ROCKSDBTESTS_END).*//') +endif TOOLS = \ - sst_dump \ + sst_dump \ db_sanity_test \ - db_stress \ - ldb \ + db_stress \ + write_stress \ + ldb \ db_repl_stress \ - options_test \ - blob_store_bench + rocksdb_dump \ + rocksdb_undump \ + blob_dump \ + +TEST_LIBS = \ + librocksdb_env_basic_test.a -PROGRAMS = db_bench signal_test table_reader_bench log_and_apply_bench $(TOOLS) +# TODO: add back forward_iterator_bench, after making it build in all environemnts. +BENCHMARKS = db_bench table_reader_bench cache_bench memtablerep_bench column_aware_encoding_exp persistent_cache_bench -# The library name is configurable since we are maintaining libraries of both -# debug/release mode. +# if user didn't config LIBNAME, set the default ifeq ($(LIBNAME),) +# we should only run rocksdb in production with DEBUG_LEVEL 0 +ifeq ($(DEBUG_LEVEL),0) LIBNAME=librocksdb +else + LIBNAME=librocksdb_debug +endif endif LIBRARY = ${LIBNAME}.a -MEMENVLIBRARY = libmemenv.a +TOOLS_LIBRARY = ${LIBNAME}_tools.a + +ROCKSDB_MAJOR = $(shell egrep "ROCKSDB_MAJOR.[0-9]" include/rocksdb/version.h | cut -d ' ' -f 3) +ROCKSDB_MINOR = $(shell egrep "ROCKSDB_MINOR.[0-9]" include/rocksdb/version.h | cut -d ' ' -f 3) +ROCKSDB_PATCH = $(shell egrep "ROCKSDB_PATCH.[0-9]" include/rocksdb/version.h | cut -d ' ' -f 3) default: all @@ -173,64 +547,268 @@ ifneq ($(PLATFORM_SHARED_VERSIONED),true) SHARED1 = ${LIBNAME}.$(PLATFORM_SHARED_EXT) SHARED2 = $(SHARED1) SHARED3 = $(SHARED1) +SHARED4 = $(SHARED1) SHARED = $(SHARED1) else -# Update db.h if you change these. -SHARED_MAJOR = 3 -SHARED_MINOR = 5 +SHARED_MAJOR = $(ROCKSDB_MAJOR) +SHARED_MINOR = $(ROCKSDB_MINOR) +SHARED_PATCH = $(ROCKSDB_PATCH) SHARED1 = ${LIBNAME}.$(PLATFORM_SHARED_EXT) +ifeq ($(PLATFORM), OS_MACOSX) +SHARED_OSX = $(LIBNAME).$(SHARED_MAJOR) +SHARED2 = $(SHARED_OSX).$(PLATFORM_SHARED_EXT) +SHARED3 = $(SHARED_OSX).$(SHARED_MINOR).$(PLATFORM_SHARED_EXT) +SHARED4 = $(SHARED_OSX).$(SHARED_MINOR).$(SHARED_PATCH).$(PLATFORM_SHARED_EXT) +else SHARED2 = $(SHARED1).$(SHARED_MAJOR) SHARED3 = $(SHARED1).$(SHARED_MAJOR).$(SHARED_MINOR) -SHARED = $(SHARED1) $(SHARED2) $(SHARED3) -$(SHARED1): $(SHARED3) - ln -fs $(SHARED3) $(SHARED1) -$(SHARED2): $(SHARED3) - ln -fs $(SHARED3) $(SHARED2) +SHARED4 = $(SHARED1).$(SHARED_MAJOR).$(SHARED_MINOR).$(SHARED_PATCH) +endif +SHARED = $(SHARED1) $(SHARED2) $(SHARED3) $(SHARED4) +$(SHARED1): $(SHARED4) + ln -fs $(SHARED4) $(SHARED1) +$(SHARED2): $(SHARED4) + ln -fs $(SHARED4) $(SHARED2) +$(SHARED3): $(SHARED4) + ln -fs $(SHARED4) $(SHARED3) endif -$(SHARED3): - $(CXX) $(PLATFORM_SHARED_LDFLAGS)$(SHARED2) $(CXXFLAGS) $(PLATFORM_SHARED_CFLAGS) $(SOURCES) $(LDFLAGS) -o $@ +shared_libobjects = $(patsubst %,shared-objects/%,$(LIBOBJECTS)) +CLEAN_FILES += shared-objects + +$(shared_libobjects): shared-objects/%.o: %.cc + $(AM_V_CC)mkdir -p $(@D) && $(CXX) $(CXXFLAGS) $(PLATFORM_SHARED_CFLAGS) -c $< -o $@ + +$(SHARED4): $(shared_libobjects) + $(CXX) $(PLATFORM_SHARED_LDFLAGS)$(SHARED3) $(CXXFLAGS) $(PLATFORM_SHARED_CFLAGS) $(shared_libobjects) $(LDFLAGS) -o $@ endif # PLATFORM_SHARED_EXT -.PHONY: blackbox_crash_test check clean coverage crash_test ldb_tests \ +.PHONY: blackbox_crash_test check clean coverage crash_test ldb_tests package \ release tags valgrind_check whitebox_crash_test format static_lib shared_lib all \ - dbg + dbg rocksdbjavastatic rocksdbjava install install-static install-shared uninstall \ + analyze tools tools_lib + -all: $(LIBRARY) $(PROGRAMS) $(TESTS) +all: $(LIBRARY) $(BENCHMARKS) tools tools_lib test_libs $(TESTS) + +all_but_some_tests: $(LIBRARY) $(BENCHMARKS) tools tools_lib test_libs $(SUBSET) static_lib: $(LIBRARY) shared_lib: $(SHARED) -dbg: $(LIBRARY) $(PROGRAMS) $(TESTS) +tools: $(TOOLS) + +tools_lib: $(TOOLS_LIBRARY) + +test_libs: $(TEST_LIBS) + +dbg: $(LIBRARY) $(BENCHMARKS) tools $(TESTS) # creates static library and programs release: $(MAKE) clean - OPT="-DNDEBUG -O2" $(MAKE) static_lib $(PROGRAMS) -j32 + DEBUG_LEVEL=0 $(MAKE) static_lib tools db_bench coverage: $(MAKE) clean - COVERAGEFLAGS="-fprofile-arcs -ftest-coverage" LDFLAGS+="-lgcov" $(MAKE) all check -j32 - (cd coverage; ./coverage_test.sh) - # Delete intermediate files + COVERAGEFLAGS="-fprofile-arcs -ftest-coverage" LDFLAGS+="-lgcov" $(MAKE) J=1 all check + cd coverage && ./coverage_test.sh + # Delete intermediate files find . -type f -regex ".*\.\(\(gcda\)\|\(gcno\)\)" -exec rm {} \; -check: $(TESTS) ldb - for t in $(TESTS); do echo "***** Running $$t"; ./$$t || exit 1; done +ifneq (,$(filter check parallel_check,$(MAKECMDGOALS)),) +# Use /dev/shm if it has the sticky bit set (otherwise, /tmp), +# and create a randomly-named rocksdb.XXXX directory therein. +# We'll use that directory in the "make check" rules. +ifeq ($(TMPD),) +TMPDIR := $(shell echo $${TMPDIR:-/tmp}) +TMPD := $(shell f=/dev/shm; test -k $$f || f=$(TMPDIR); \ + perl -le 'use File::Temp "tempdir";' \ + -e 'print tempdir("'$$f'/rocksdb.XXXX", CLEANUP => 0)') +endif +endif + +# Run all tests in parallel, accumulating per-test logs in t/log-*. +# +# Each t/run-* file is a tiny generated bourne shell script that invokes one of +# sub-tests. Why use a file for this? Because that makes the invocation of +# parallel below simpler, which in turn makes the parsing of parallel's +# LOG simpler (the latter is for live monitoring as parallel +# tests run). +# +# Test names are extracted by running tests with --gtest_list_tests. +# This filter removes the "#"-introduced comments, and expands to +# fully-qualified names by changing input like this: +# +# DBTest. +# Empty +# WriteEmptyBatch +# MultiThreaded/MultiThreadedDBTest. +# MultiThreaded/0 # GetParam() = 0 +# MultiThreaded/1 # GetParam() = 1 +# +# into this: +# +# DBTest.Empty +# DBTest.WriteEmptyBatch +# MultiThreaded/MultiThreadedDBTest.MultiThreaded/0 +# MultiThreaded/MultiThreadedDBTest.MultiThreaded/1 +# + +parallel_tests = $(patsubst %,parallel_%,$(PARALLEL_TEST)) +.PHONY: gen_parallel_tests $(parallel_tests) +$(parallel_tests): $(PARALLEL_TEST) + $(AM_V_at)TEST_BINARY=$(patsubst parallel_%,%,$@); \ + TEST_NAMES=` \ + ./$$TEST_BINARY --gtest_list_tests \ + | perl -n \ + -e 's/ *\#.*//;' \ + -e '/^(\s*)(\S+)/; !$$1 and do {$$p=$$2; break};' \ + -e 'print qq! $$p$$2!'`; \ + for TEST_NAME in $$TEST_NAMES; do \ + TEST_SCRIPT=t/run-$$TEST_BINARY-$${TEST_NAME//\//-}; \ + echo " GEN " $$TEST_SCRIPT; \ + printf '%s\n' \ + '#!/bin/sh' \ + "d=\$(TMPD)$$TEST_SCRIPT" \ + 'mkdir -p $$d' \ + "TEST_TMPDIR=\$$d $(DRIVER) ./$$TEST_BINARY --gtest_filter=$$TEST_NAME" \ + > $$TEST_SCRIPT; \ + chmod a=rx $$TEST_SCRIPT; \ + done + +gen_parallel_tests: + $(AM_V_at)mkdir -p t + $(AM_V_at)rm -f t/run-* + $(MAKE) $(parallel_tests) + +# Reorder input lines (which are one per test) so that the +# longest-running tests appear first in the output. +# Do this by prefixing each selected name with its duration, +# sort the resulting names, and remove the leading numbers. +# FIXME: the "100" we prepend is a fake time, for now. +# FIXME: squirrel away timings from each run and use them +# (when present) on subsequent runs to order these tests. +# +# Without this reordering, these two tests would happen to start only +# after almost all other tests had completed, thus adding 100 seconds +# to the duration of parallel "make check". That's the difference +# between 4 minutes (old) and 2m20s (new). +# +# 152.120 PASS t/DBTest.FileCreationRandomFailure +# 107.816 PASS t/DBTest.EncodeDecompressedBlockSizeTest +# +slow_test_regexp = \ + ^t/run-table_test-HarnessTest.Randomized$$|^t/run-db_test-.*(?:FileCreationRandomFailure|EncodeDecompressedBlockSizeTest)$$|^.*RecoverFromCorruptedWALWithoutFlush$$ +prioritize_long_running_tests = \ + perl -pe 's,($(slow_test_regexp)),100 $$1,' \ + | sort -k1,1gr \ + | sed 's/^[.0-9]* //' + +# "make check" uses +# Run with "make J=1 check" to disable parallelism in "make check". +# Run with "make J=200% check" to run two parallel jobs per core. +# The default is to run one job per core (J=100%). +# See "man parallel" for its "-j ..." option. +J ?= 100% + +# Use this regexp to select the subset of tests whose names match. +tests-regexp = . + +t_run = $(wildcard t/run-*) +.PHONY: check_0 +check_0: + $(AM_V_GEN)export TEST_TMPDIR=$(TMPD); \ + printf '%s\n' '' \ + 'To monitor subtest ,' \ + ' run "make watch-log" in a separate window' ''; \ + test -t 1 && eta=--eta || eta=; \ + { \ + printf './%s\n' $(filter-out $(PARALLEL_TEST),$(TESTS)); \ + printf '%s\n' $(t_run); \ + } \ + | $(prioritize_long_running_tests) \ + | grep -E '$(tests-regexp)' \ + | build_tools/gnu_parallel -j$(J) --plain --joblog=LOG $$eta --gnu '{} >& t/log-{/}' + +valgrind-blacklist-regexp = InlineSkipTest.ConcurrentInsert|TransactionTest.DeadlockStress|DBCompactionTest.SuggestCompactRangeNoTwoLevel0Compactions|BackupableDBTest.RateLimiting|DBTest.CloseSpeedup|DBTest.ThreadStatusFlush|DBTest.RateLimitingTest|DBTest.EncodeDecompressedBlockSizeTest|FaultInjectionTest.UninstalledCompaction|HarnessTest.Randomized|ExternalSSTFileTest.CompactDuringAddFileRandom|ExternalSSTFileTest.IngestFileWithGlobalSeqnoRandomized + +.PHONY: valgrind_check_0 +valgrind_check_0: + $(AM_V_GEN)export TEST_TMPDIR=$(TMPD); \ + printf '%s\n' '' \ + 'To monitor subtest ,' \ + ' run "make watch-log" in a separate window' ''; \ + test -t 1 && eta=--eta || eta=; \ + { \ + printf './%s\n' $(filter-out $(PARALLEL_TEST) %skiplist_test options_settable_test, $(TESTS)); \ + printf '%s\n' $(t_run); \ + } \ + | $(prioritize_long_running_tests) \ + | grep -E '$(tests-regexp)' \ + | grep -E -v '$(valgrind-blacklist-regexp)' \ + | build_tools/gnu_parallel -j$(J) --plain --joblog=LOG $$eta --gnu \ + '(if [[ "{}" == "./"* ]] ; then $(DRIVER) {}; else {}; fi) ' \ + '>& t/valgrind_log-{/}' + +CLEAN_FILES += t LOG $(TMPD) + +# When running parallel "make check", you can monitor its progress +# from another window. +# Run "make watch_LOG" to show the duration,PASS/FAIL,name of parallel +# tests as they are being run. We sort them so that longer-running ones +# appear at the top of the list and any failing tests remain at the top +# regardless of their duration. As with any use of "watch", hit ^C to +# interrupt. +watch-log: + watch --interval=0 'sort -k7,7nr -k4,4gr LOG|$(quoted_perl_command)' + +# If J != 1 and GNU parallel is installed, run the tests in parallel, +# via the check_0 rule above. Otherwise, run them sequentially. +check: all + $(MAKE) gen_parallel_tests + $(AM_V_GEN)if test "$(J)" != 1 \ + && (build_tools/gnu_parallel --gnu --help 2>/dev/null) | \ + grep -q 'GNU Parallel'; \ + then \ + $(MAKE) T="$$t" TMPD=$(TMPD) check_0; \ + else \ + for t in $(TESTS); do \ + echo "===== Running $$t"; ./$$t || exit 1; done; \ + fi + rm -rf $(TMPD) +ifneq ($(PLATFORM), OS_AIX) +ifeq ($(filter -DROCKSDB_LITE,$(OPT)),) python tools/ldb_test.py + sh tools/rocksdb_dump_test.sh +endif +endif +# TODO add ldb_tests +check_some: $(SUBSET) + for t in $(SUBSET); do echo "===== Running $$t"; ./$$t || exit 1; done + +.PHONY: ldb_tests ldb_tests: ldb python tools/ldb_test.py crash_test: whitebox_crash_test blackbox_crash_test blackbox_crash_test: db_stress - python -u tools/db_crashtest.py + python -u tools/db_crashtest.py --simple blackbox $(CRASH_TEST_EXT_ARGS) + python -u tools/db_crashtest.py blackbox $(CRASH_TEST_EXT_ARGS) + +ifeq ($(CRASH_TEST_KILL_ODD),) + CRASH_TEST_KILL_ODD=888887 +endif whitebox_crash_test: db_stress - python -u tools/db_crashtest2.py + python -u tools/db_crashtest.py --simple whitebox --random_kill_odd \ + $(CRASH_TEST_KILL_ODD) $(CRASH_TEST_EXT_ARGS) + python -u tools/db_crashtest.py whitebox --random_kill_odd \ + $(CRASH_TEST_KILL_ODD) $(CRASH_TEST_EXT_ARGS) asan_check: $(MAKE) clean @@ -242,284 +820,884 @@ asan_crash_test: COMPILE_WITH_ASAN=1 $(MAKE) crash_test $(MAKE) clean -valgrind_check: all $(PROGRAMS) $(TESTS) - mkdir -p $(VALGRIND_DIR) - echo TESTS THAT HAVE VALGRIND ERRORS > $(VALGRIND_DIR)/valgrind_failed_tests; \ - echo TIMES in seconds TAKEN BY TESTS ON VALGRIND > $(VALGRIND_DIR)/valgrind_tests_times; \ - for t in $(filter-out skiplist_test,$(TESTS)); do \ - stime=`date '+%s'`; \ - $(VALGRIND_VER) $(VALGRIND_OPTS) ./$$t; \ - if [ $$? -eq $(VALGRIND_ERROR) ] ; then \ - echo $$t >> $(VALGRIND_DIR)/valgrind_failed_tests; \ +ubsan_check: + $(MAKE) clean + COMPILE_WITH_UBSAN=1 $(MAKE) check -j32 + $(MAKE) clean + +ubsan_crash_test: + $(MAKE) clean + COMPILE_WITH_UBSAN=1 $(MAKE) crash_test + $(MAKE) clean + +valgrind_test: + DISABLE_JEMALLOC=1 $(MAKE) valgrind_check + +valgrind_check: $(TESTS) + $(MAKE) DRIVER="$(VALGRIND_VER) $(VALGRIND_OPTS)" gen_parallel_tests + $(AM_V_GEN)if test "$(J)" != 1 \ + && (build_tools/gnu_parallel --gnu --help 2>/dev/null) | \ + grep -q 'GNU Parallel'; \ + then \ + $(MAKE) TMPD=$(TMPD) \ + DRIVER="$(VALGRIND_VER) $(VALGRIND_OPTS)" valgrind_check_0; \ + else \ + for t in $(filter-out %skiplist_test options_settable_test,$(TESTS)); do \ + $(VALGRIND_VER) $(VALGRIND_OPTS) ./$$t; \ + ret_code=$$?; \ + if [ $$ret_code -ne 0 ]; then \ + exit $$ret_code; \ + fi; \ + done; \ + fi + + +ifneq ($(PAR_TEST),) +parloop: + ret_bad=0; \ + for t in $(PAR_TEST); do \ + echo "===== Running $$t in parallel $(NUM_PAR)";\ + if [ $(db_test) -eq 1 ]; then \ + seq $(J) | v="$$t" build_tools/gnu_parallel --gnu --plain 's=$(TMPD)/rdb-{}; export TEST_TMPDIR=$$s;' \ + 'timeout 2m ./db_test --gtest_filter=$$v >> $$s/log-{} 2>1'; \ + else\ + seq $(J) | v="./$$t" build_tools/gnu_parallel --gnu --plain 's=$(TMPD)/rdb-{};' \ + 'export TEST_TMPDIR=$$s; timeout 10m $$v >> $$s/log-{} 2>1'; \ + fi; \ + ret_code=$$?; \ + if [ $$ret_code -ne 0 ]; then \ + ret_bad=$$ret_code; \ + echo $$t exited with $$ret_code; \ fi; \ - etime=`date '+%s'`; \ - echo $$t $$((etime - stime)) >> $(VALGRIND_DIR)/valgrind_tests_times; \ + done; \ + exit $$ret_bad; +endif + +test_names = \ + ./db_test --gtest_list_tests \ + | perl -n \ + -e 's/ *\#.*//;' \ + -e '/^(\s*)(\S+)/; !$$1 and do {$$p=$$2; break};' \ + -e 'print qq! $$p$$2!' + +parallel_check: $(TESTS) + $(AM_V_GEN)if test "$(J)" > 1 \ + && (build_tools/gnu_parallel --gnu --help 2>/dev/null) | \ + grep -q 'GNU Parallel'; \ + then \ + echo Running in parallel $(J); \ + else \ + echo "Need to have GNU Parallel and J > 1"; exit 1; \ + fi; \ + ret_bad=0; \ + echo $(J);\ + echo Test Dir: $(TMPD); \ + seq $(J) | build_tools/gnu_parallel --gnu --plain 's=$(TMPD)/rdb-{}; rm -rf $$s; mkdir $$s'; \ + $(MAKE) PAR_TEST="$(shell $(test_names))" TMPD=$(TMPD) \ + J=$(J) db_test=1 parloop; \ + $(MAKE) PAR_TEST="$(filter-out db_test, $(TESTS))" \ + TMPD=$(TMPD) J=$(J) db_test=0 parloop; + +analyze: clean + $(CLANG_SCAN_BUILD) --use-analyzer=$(CLANG_ANALYZER) \ + --use-c++=$(CXX) --use-cc=$(CC) --status-bugs \ + -o $(CURDIR)/scan_build_report \ + $(MAKE) dbg + +CLEAN_FILES += unity.cc +unity.cc: Makefile + rm -f $@ $@-t + for source_file in $(LIB_SOURCES); do \ + echo "#include \"$$source_file\"" >> $@-t; \ done + chmod a=r $@-t + mv $@-t $@ -unity.cc: - $(shell (export ROCKSDB_ROOT="$(CURDIR)"; "$(CURDIR)/build_tools/unity" "$(CURDIR)/unity.cc")) +unity.a: unity.o + $(AM_V_AR)rm -f $@ + $(AM_V_at)$(AR) $(ARFLAGS) $@ unity.o -unity: unity.cc unity.o - $(CXX) unity.o $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +# try compiling db_test with unity +unity_test: db/db_test.o db/db_test_util.o $(TESTHARNESS) unity.a + $(AM_LINK) + ./unity_test + +rocksdb.h rocksdb.cc: build_tools/amalgamate.py Makefile $(LIB_SOURCES) unity.cc + build_tools/amalgamate.py -I. -i./include unity.cc -x include/rocksdb/c.h -H rocksdb.h -o rocksdb.cc clean: - -rm -f $(PROGRAMS) $(TESTS) $(LIBRARY) $(SHARED) $(MEMENVLIBRARY) build_config.mk unity.cc - -rm -rf ios-x86/* ios-arm/* - -find . -name "*.[od]" -exec rm {} \; - -find . -type f -regex ".*\.\(\(gcda\)\|\(gcno\)\)" -exec rm {} \; + rm -f $(BENCHMARKS) $(TOOLS) $(TESTS) $(LIBRARY) $(SHARED) + rm -rf $(CLEAN_FILES) ios-x86 ios-arm scan_build_report + find . -name "*.[oda]" -exec rm -f {} \; + find . -type f -regex ".*\.\(\(gcda\)\|\(gcno\)\)" -exec rm {} \; + rm -rf bzip2* snappy* zlib* lz4* zstd* + cd java; $(MAKE) clean + tags: ctags * -R - cscope -b `find . -name '*.cc'` `find . -name '*.h'` + cscope -b `find . -name '*.cc'` `find . -name '*.h'` `find . -name '*.c'` + ctags -e -R -o etags * format: build_tools/format-diff.sh +package: + bash build_tools/make_package.sh $(SHARED_MAJOR).$(SHARED_MINOR) + # --------------------------------------------------------------------------- # Unit tests and tools # --------------------------------------------------------------------------- $(LIBRARY): $(LIBOBJECTS) - rm -f $@ - $(AR) -rs $@ $(LIBOBJECTS) + $(AM_V_AR)rm -f $@ + $(AM_V_at)$(AR) $(ARFLAGS) $@ $(LIBOBJECTS) + +$(TOOLS_LIBRARY): $(BENCH_LIB_SOURCES:.cc=.o) $(TOOL_LIB_SOURCES:.cc=.o) $(LIB_SOURCES:.cc=.o) $(TESTUTIL) + $(AM_V_AR)rm -f $@ + $(AM_V_at)$(AR) $(ARFLAGS) $@ $^ -db_bench: db/db_bench.o $(LIBOBJECTS) $(TESTUTIL) - $(CXX) db/db_bench.o $(LIBOBJECTS) $(TESTUTIL) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +librocksdb_env_basic_test.a: env/env_basic_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_V_AR)rm -f $@ + $(AM_V_at)$(AR) $(ARFLAGS) $@ $^ -block_hash_index_test: table/block_hash_index_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) table/block_hash_index_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +db_bench: tools/db_bench.o $(BENCHTOOLOBJECTS) + $(AM_LINK) + +cache_bench: cache/cache_bench.o $(LIBOBJECTS) $(TESTUTIL) + $(AM_LINK) + +persistent_cache_bench: utilities/persistent_cache/persistent_cache_bench.o $(LIBOBJECTS) $(TESTUTIL) + $(AM_LINK) + +memtablerep_bench: memtable/memtablerep_bench.o $(LIBOBJECTS) $(TESTUTIL) + $(AM_LINK) db_stress: tools/db_stress.o $(LIBOBJECTS) $(TESTUTIL) - $(CXX) tools/db_stress.o $(LIBOBJECTS) $(TESTUTIL) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +write_stress: tools/write_stress.o $(LIBOBJECTS) $(TESTUTIL) + $(AM_LINK) db_sanity_test: tools/db_sanity_test.o $(LIBOBJECTS) $(TESTUTIL) - $(CXX) tools/db_sanity_test.o $(LIBOBJECTS) $(TESTUTIL) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) db_repl_stress: tools/db_repl_stress.o $(LIBOBJECTS) $(TESTUTIL) - $(CXX) tools/db_repl_stress.o $(LIBOBJECTS) $(TESTUTIL) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) - -blob_store_bench: tools/blob_store_bench.o $(LIBOBJECTS) $(TESTUTIL) - $(CXX) tools/blob_store_bench.o $(LIBOBJECTS) $(TESTUTIL) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) - -signal_test: util/signal_test.o $(LIBOBJECTS) - $(CXX) util/signal_test.o $(LIBOBJECTS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) arena_test: util/arena_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/arena_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) autovector_test: util/autovector_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/autovector_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) -column_family_test: db/column_family_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/column_family_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +column_family_test: db/column_family_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) table_properties_collector_test: db/table_properties_collector_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/table_properties_collector_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) bloom_test: util/bloom_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/bloom_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) dynamic_bloom_test: util/dynamic_bloom_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/dynamic_bloom_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) c_test: db/c_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/c_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) -cache_test: util/cache_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/cache_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +cache_test: cache/cache_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) coding_test: util/coding_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/coding_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +hash_test: util/hash_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -blob_store_test: util/blob_store_test.o $(LIBOBJECTS) $(TESTHARNESS) $(TESTUTIL) - $(CXX) util/blob_store_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o$@ $(LDFLAGS) $(COVERAGEFLAGS) +option_change_migration_test: utilities/option_change_migration/option_change_migration_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) stringappend_test: utilities/merge_operators/string_append/stringappend_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) utilities/merge_operators/string_append/stringappend_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +cassandra_format_test: utilities/cassandra/cassandra_format_test.o utilities/cassandra/test_utils.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +cassandra_functional_test: utilities/cassandra/cassandra_functional_test.o utilities/cassandra/test_utils.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +cassandra_row_merge_test: utilities/cassandra/cassandra_row_merge_test.o utilities/cassandra/test_utils.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +cassandra_serialize_test: utilities/cassandra/cassandra_serialize_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) redis_test: utilities/redis/redis_lists_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) utilities/redis/redis_lists_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) -benchharness_test: util/benchharness_test.o $(LIBOBJECTS) $(TESTHARNESS) $(BENCHHARNESS) - $(CXX) util/benchharness_test.o $(LIBOBJECTS) $(TESTHARNESS) $(BENCHHARNESS) $(EXEC_LDFLAGS) -o$@ $(LDFLAGS) $(COVERAGEFLAGS) +hash_table_test: utilities/persistent_cache/hash_table_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -histogram_test: util/histogram_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/histogram_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o$@ $(LDFLAGS) $(COVERAGEFLAGS) +histogram_test: monitoring/histogram_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) thread_local_test: util/thread_local_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/thread_local_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) corruption_test: db/corruption_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/corruption_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) crc32c_test: util/crc32c_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/crc32c_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +slice_transform_test: util/slice_transform_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_basic_test: db/db_basic_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_encryption_test: db/db_encryption_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_test: db/db_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_test2: db/db_test2.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_blob_index_test: db/db_blob_index_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_block_cache_test: db/db_block_cache_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_bloom_filter_test: db/db_bloom_filter_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_log_iter_test: db/db_log_iter_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_compaction_filter_test: db/db_compaction_filter_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_compaction_test: db/db_compaction_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_dynamic_level_test: db/db_dynamic_level_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_flush_test: db/db_flush_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_inplace_update_test: db/db_inplace_update_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_iterator_test: db/db_iterator_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_memtable_test: db/db_memtable_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_merge_operator_test: db/db_merge_operator_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_options_test: db/db_options_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_range_del_test: db/db_range_del_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -db_test: db/db_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +db_sst_test: db/db_sst_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_statistics_test: db/db_statistics_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_write_test: db/db_write_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +external_sst_file_basic_test: db/external_sst_file_basic_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +external_sst_file_test: db/external_sst_file_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_tailing_iter_test: db/db_tailing_iter_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) db_iter_test: db/db_iter_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/db_iter_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +db_universal_compaction_test: db/db_universal_compaction_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_wal_test: db/db_wal_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_io_failure_test: db/db_io_failure_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_properties_test: db/db_properties_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_table_properties_test: db/db_table_properties_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) log_write_bench: util/log_write_bench.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/log_write_bench.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) -pg + $(AM_LINK) $(PROFILING_FLAGS) plain_table_db_test: db/plain_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/plain_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) -simple_table_db_test: db/simple_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/simple_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +comparator_db_test: db/comparator_db_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) table_reader_bench: table/table_reader_bench.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) table/table_reader_bench.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) -pg - -log_and_apply_bench: db/log_and_apply_bench.o $(LIBOBJECTS) $(TESTHARNESS) $(BENCHHARNESS) - $(CXX) db/log_and_apply_bench.o $(LIBOBJECTS) $(TESTHARNESS) $(BENCHHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) -pg + $(AM_LINK) $(PROFILING_FLAGS) perf_context_test: db/perf_context_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/perf_context_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) + $(AM_V_CCLD)$(CXX) $^ $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) prefix_test: db/prefix_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/prefix_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) + $(AM_V_CCLD)$(CXX) $^ $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) backupable_db_test: utilities/backupable/backupable_db_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) utilities/backupable/backupable_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +checkpoint_test: utilities/checkpoint/checkpoint_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) document_db_test: utilities/document/document_db_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) utilities/document/document_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) json_document_test: utilities/document/json_document_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) utilities/document/json_document_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +sim_cache_test: utilities/simulator_cache/sim_cache_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) spatial_db_test: utilities/spatialdb/spatial_db_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) utilities/spatialdb/spatial_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +env_mirror_test: utilities/env_mirror_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +env_timed_test: utilities/env_timed_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +ifdef ROCKSDB_USE_LIBRADOS +env_librados_test: utilities/env_librados_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_V_CCLD)$(CXX) $^ $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +endif + +object_registry_test: utilities/object_registry_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) ttl_test: utilities/ttl/ttl_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) utilities/ttl/ttl_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +date_tiered_test: utilities/date_tiered/date_tiered_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) write_batch_with_index_test: utilities/write_batch_with_index/write_batch_with_index_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) utilities/write_batch_with_index/write_batch_with_index_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +flush_job_test: db/flush_job_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +compaction_iterator_test: db/compaction_iterator_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +compaction_job_test: db/compaction_job_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +compaction_job_stats_test: db/compaction_job_stats_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +compact_on_deletion_collector_test: utilities/table_properties_collectors/compact_on_deletion_collector_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +wal_manager_test: db/wal_manager_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) dbformat_test: db/dbformat_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/dbformat_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +env_basic_test: env/env_basic_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +env_test: env/env_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -env_test: util/env_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/env_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +fault_injection_test: db/fault_injection_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) rate_limiter_test: util/rate_limiter_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/rate_limiter_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +delete_scheduler_test: util/delete_scheduler_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) filename_test: db/filename_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/filename_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +file_reader_writer_test: util/file_reader_writer_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +block_based_filter_block_test: table/block_based_filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +full_filter_block_test: table/full_filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -filter_block_test: table/filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) table/filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +partitioned_filter_block_test: table/partitioned_filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) log_test: db/log_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/log_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +cleanable_test: table/cleanable_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) table_test: table/table_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) table/table_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) block_test: table/block_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) table/block_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +inlineskiplist_test: memtable/inlineskiplist_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -skiplist_test: db/skiplist_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/skiplist_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +skiplist_test: memtable/skiplist_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +write_buffer_manager_test: memtable/write_buffer_manager_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) version_edit_test: db/version_edit_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/version_edit_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) version_set_test: db/version_set_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/version_set_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +compaction_picker_test: db/compaction_picker_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -file_indexer_test : db/file_indexer_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/file_indexer_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +version_builder_test: db/version_builder_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +file_indexer_test: db/file_indexer_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) reduce_levels_test: tools/reduce_levels_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) tools/reduce_levels_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) write_batch_test: db/write_batch_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/write_batch_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +write_controller_test: db/write_controller_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +merge_helper_test: db/merge_helper_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +memory_test: utilities/memory/memory_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) merge_test: db/merge_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/merge_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +merger_test: table/merger_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +util_merge_operators_test: utilities/util_merge_operators_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +options_file_test: db/options_file_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) deletefile_test: db/deletefile_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/deletefile_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) + $(AM_LINK) geodb_test: utilities/geodb/geodb_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) utilities/geodb/geodb_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +rocksdb_dump: tools/dump/rocksdb_dump.o $(LIBOBJECTS) + $(AM_LINK) + +rocksdb_undump: tools/dump/rocksdb_undump.o $(LIBOBJECTS) + $(AM_LINK) cuckoo_table_builder_test: table/cuckoo_table_builder_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) table/cuckoo_table_builder_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) -cuckoo_table_reader_test: table/cuckoo_table_reader_test.o $(LIBOBJECTS) $(TESTHARNESS) $(BENCHHARNESS) - $(CXX) table/cuckoo_table_reader_test.o $(LIBOBJECTS) $(TESTHARNESS) $(BENCHHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +cuckoo_table_reader_test: table/cuckoo_table_reader_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) cuckoo_table_db_test: db/cuckoo_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) db/cuckoo_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) -options_test: util/options_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/options_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +listener_test: db/listener_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -$(MEMENVLIBRARY) : $(MEMENVOBJECTS) - rm -f $@ - $(AR) -rs $@ $(MEMENVOBJECTS) +thread_list_test: util/thread_list_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -memenv_test : helpers/memenv/memenv_test.o $(MEMENVOBJECTS) $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) helpers/memenv/memenv_test.o $(MEMENVOBJECTS) $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +compact_files_test: db/compact_files_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -manual_compaction_test: util/manual_compaction_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/manual_compaction_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +options_test: options/options_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -rocksdb_shell: tools/shell/ShellContext.o tools/shell/ShellState.o tools/shell/LeveldbShell.o tools/shell/DBClientProxy.o tools/shell/ShellContext.h tools/shell/ShellState.h tools/shell/DBClientProxy.h $(LIBOBJECTS) - $(CXX) tools/shell/ShellContext.o tools/shell/ShellState.o tools/shell/LeveldbShell.o tools/shell/DBClientProxy.o $(LIBOBJECTS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +options_settable_test: options/options_settable_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) -DBClientProxy_test: tools/shell/test/DBClientProxyTest.o tools/shell/DBClientProxy.o $(LIBRARY) - $(CXX) tools/shell/test/DBClientProxyTest.o tools/shell/DBClientProxy.o $(LIBRARY) $(EXEC_LDFLAGS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) +options_util_test: utilities/options/options_util_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +db_bench_tool_test: tools/db_bench_tool_test.o $(BENCHTOOLOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +event_logger_test: util/event_logger_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +timer_queue_test: util/timer_queue_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +sst_dump_test: tools/sst_dump_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +column_aware_encoding_test: utilities/column_aware_encoding_test.o $(TESTHARNESS) $(EXPOBJECTS) + $(AM_LINK) + +optimistic_transaction_test: utilities/transactions/optimistic_transaction_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +mock_env_test : env/mock_env_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +manual_compaction_test: db/manual_compaction_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) filelock_test: util/filelock_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/filelock_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) auto_roll_logger_test: util/auto_roll_logger_test.o $(LIBOBJECTS) $(TESTHARNESS) - $(CXX) util/auto_roll_logger_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +memtable_list_test: db/memtable_list_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +write_callback_test: db/write_callback_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +heap_test: util/heap_test.o $(GTEST) + $(AM_LINK) + +transaction_test: utilities/transactions/transaction_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) sst_dump: tools/sst_dump.o $(LIBOBJECTS) - $(CXX) tools/sst_dump.o $(LIBOBJECTS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +blob_dump: tools/blob_dump.o $(LIBOBJECTS) + $(AM_LINK) + +column_aware_encoding_exp: utilities/column_aware_encoding_exp.o $(EXPOBJECTS) + $(AM_LINK) + +repair_test: db/repair_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +ldb_cmd_test: tools/ldb_cmd_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) ldb: tools/ldb.o $(LIBOBJECTS) - $(CXX) tools/ldb.o $(LIBOBJECTS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) + $(AM_LINK) + +iostats_context_test: monitoring/iostats_context_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_V_CCLD)$(CXX) $^ $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) + +persistent_cache_test: utilities/persistent_cache/persistent_cache_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +statistics_test: monitoring/statistics_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +lru_cache_test: cache/lru_cache_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +lua_test: utilities/lua/rocks_lua_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +range_del_aggregator_test: db/range_del_aggregator_test.o db/db_test_util.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +blob_db_test: utilities/blob_db/blob_db_test.o $(LIBOBJECTS) $(TESTHARNESS) + $(AM_LINK) + +#------------------------------------------------- +# make install related stuff +INSTALL_PATH ?= /usr/local + +uninstall: + rm -rf $(INSTALL_PATH)/include/rocksdb \ + $(INSTALL_PATH)/lib/$(LIBRARY) \ + $(INSTALL_PATH)/lib/$(SHARED4) \ + $(INSTALL_PATH)/lib/$(SHARED3) \ + $(INSTALL_PATH)/lib/$(SHARED2) \ + $(INSTALL_PATH)/lib/$(SHARED1) + +install-headers: + install -d $(INSTALL_PATH)/lib + for header_dir in `find "include/rocksdb" -type d`; do \ + install -d $(INSTALL_PATH)/$$header_dir; \ + done + for header in `find "include/rocksdb" -type f -name *.h`; do \ + install -C -m 644 $$header $(INSTALL_PATH)/$$header; \ + done + +install-static: install-headers $(LIBRARY) + install -C -m 755 $(LIBRARY) $(INSTALL_PATH)/lib + +install-shared: install-headers $(SHARED4) + install -C -m 755 $(SHARED4) $(INSTALL_PATH)/lib && \ + ln -fs $(SHARED4) $(INSTALL_PATH)/lib/$(SHARED3) && \ + ln -fs $(SHARED4) $(INSTALL_PATH)/lib/$(SHARED2) && \ + ln -fs $(SHARED4) $(INSTALL_PATH)/lib/$(SHARED1) + +# install static by default + install shared if it exists +install: install-static + [ -e $(SHARED4) ] && $(MAKE) install-shared || : + +#------------------------------------------------- + # --------------------------------------------------------------------------- # Jni stuff # --------------------------------------------------------------------------- -JNI_NATIVE_SOURCES = ./java/rocksjni/*.cc JAVA_INCLUDE = -I$(JAVA_HOME)/include/ -I$(JAVA_HOME)/include/linux -ROCKSDBJNILIB = librocksdbjni.so -ROCKSDB_JAR = rocksdbjni.jar +ifeq ($(PLATFORM), OS_SOLARIS) + ARCH := $(shell isainfo -b) +else + ARCH := $(shell getconf LONG_BIT) +endif + +ifeq (,$(findstring ppc,$(MACHINE))) + ROCKSDBJNILIB = librocksdbjni-linux$(ARCH).so +else + ROCKSDBJNILIB = librocksdbjni-linux-$(MACHINE).so +endif +ROCKSDB_JAR = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-linux$(ARCH).jar +ROCKSDB_JAR_ALL = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH).jar +ROCKSDB_JAVADOCS_JAR = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-javadoc.jar +ROCKSDB_SOURCES_JAR = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-sources.jar +SHA256_CMD = sha256sum + +ZLIB_VER ?= 1.2.11 +ZLIB_SHA256 ?= c3e5e9fdd5004dcb542feda5ee4f0ff0744628baf8ed2dd5d66f8ca1197cb1a1 +ZLIB_DOWNLOAD_BASE ?= http://zlib.net +BZIP2_VER ?= 1.0.6 +BZIP2_SHA256 ?= a2848f34fcd5d6cf47def00461fcb528a0484d8edef8208d6d2e2909dc61d9cd +BZIP2_DOWNLOAD_BASE ?= http://www.bzip.org +SNAPPY_VER ?= 1.1.4 +SNAPPY_SHA256 ?= 134bfe122fd25599bb807bb8130e7ba6d9bdb851e0b16efcb83ac4f5d0b70057 +SNAPPY_DOWNLOAD_BASE ?= https://github.com/google/snappy/releases/download +LZ4_VER ?= 1.7.5 +LZ4_SHA256 ?= 0190cacd63022ccb86f44fa5041dc6c3804407ad61550ca21c382827319e7e7e +LZ4_DOWNLOAD_BASE ?= https://github.com/lz4/lz4/archive +ZSTD_VER ?= 1.2.0 +ZSTD_SHA256 ?= 4a7e4593a3638276ca7f2a09dc4f38e674d8317bbea51626393ca73fc047cbfb +ZSTD_DOWNLOAD_BASE ?= https://github.com/facebook/zstd/archive ifeq ($(PLATFORM), OS_MACOSX) -ROCKSDBJNILIB = librocksdbjni.jnilib -JAVA_INCLUDE = -I/System/Library/Frameworks/JavaVM.framework/Headers/ + ROCKSDBJNILIB = librocksdbjni-osx.jnilib + ROCKSDB_JAR = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-osx.jar + SHA256_CMD = openssl sha256 -r +ifneq ("$(wildcard $(JAVA_HOME)/include/darwin)","") + JAVA_INCLUDE = -I$(JAVA_HOME)/include -I $(JAVA_HOME)/include/darwin +else + JAVA_INCLUDE = -I/System/Library/Frameworks/JavaVM.framework/Headers/ +endif +endif +ifeq ($(PLATFORM), OS_FREEBSD) + JAVA_INCLUDE += -I$(JAVA_HOME)/include/freebsd + ROCKSDBJNILIB = librocksdbjni-freebsd$(ARCH).so + ROCKSDB_JAR = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-freebsd$(ARCH).jar +endif +ifeq ($(PLATFORM), OS_SOLARIS) + ROCKSDBJNILIB = librocksdbjni-solaris$(ARCH).so + ROCKSDB_JAR = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-solaris$(ARCH).jar + JAVA_INCLUDE = -I$(JAVA_HOME)/include/ -I$(JAVA_HOME)/include/solaris + SHA256_CMD = digest -a sha256 +endif +ifeq ($(PLATFORM), OS_AIX) + JAVA_INCLUDE = -I$(JAVA_HOME)/include/ -I$(JAVA_HOME)/include/aix + ROCKSDBJNILIB = librocksdbjni-aix.so + EXTRACT_SOURCES = gunzip < TAR_GZ | tar xvf - + SNAPPY_MAKE_TARGET = libsnappy.la endif -rocksdbjava: - OPT="-fPIC -DNDEBUG -O2" $(MAKE) $(LIBRARY) -j32 - cd java;$(MAKE) java; - rm -f ./java/$(ROCKSDBJNILIB) - $(CXX) $(CXXFLAGS) -I./java/. $(JAVA_INCLUDE) -shared -fPIC -o ./java/$(ROCKSDBJNILIB) $(JNI_NATIVE_SOURCES) $(LIBOBJECTS) $(JAVA_LDFLAGS) $(COVERAGEFLAGS) - cd java;jar -cf $(ROCKSDB_JAR) org/rocksdb/*.class org/rocksdb/util/*.class HISTORY*.md $(ROCKSDBJNILIB) +libz.a: + -rm -rf zlib-$(ZLIB_VER) + curl -O -L ${ZLIB_DOWNLOAD_BASE}/zlib-$(ZLIB_VER).tar.gz + ZLIB_SHA256_ACTUAL=`$(SHA256_CMD) zlib-$(ZLIB_VER).tar.gz | cut -d ' ' -f 1`; \ + if [ "$(ZLIB_SHA256)" != "$$ZLIB_SHA256_ACTUAL" ]; then \ + echo zlib-$(ZLIB_VER).tar.gz checksum mismatch, expected=\"$(ZLIB_SHA256)\" actual=\"$$ZLIB_SHA256_ACTUAL\"; \ + exit 1; \ + fi + tar xvzf zlib-$(ZLIB_VER).tar.gz + cd zlib-$(ZLIB_VER) && CFLAGS='-fPIC ${EXTRA_CFLAGS}' LDFLAGS='${EXTRA_LDFLAGS}' ./configure --static && make + cp zlib-$(ZLIB_VER)/libz.a . + +libbz2.a: + -rm -rf bzip2-$(BZIP2_VER) + curl -O -L ${BZIP2_DOWNLOAD_BASE}/$(BZIP2_VER)/bzip2-$(BZIP2_VER).tar.gz + BZIP2_SHA256_ACTUAL=`$(SHA256_CMD) bzip2-$(BZIP2_VER).tar.gz | cut -d ' ' -f 1`; \ + if [ "$(BZIP2_SHA256)" != "$$BZIP2_SHA256_ACTUAL" ]; then \ + echo bzip2-$(BZIP2_VER).tar.gz checksum mismatch, expected=\"$(BZIP2_SHA256)\" actual=\"$$BZIP2_SHA256_ACTUAL\"; \ + exit 1; \ + fi + tar xvzf bzip2-$(BZIP2_VER).tar.gz + cd bzip2-$(BZIP2_VER) && make CFLAGS='-fPIC -O2 -g -D_FILE_OFFSET_BITS=64 ${EXTRA_CFLAGS}' AR='ar ${EXTRA_ARFLAGS}' + cp bzip2-$(BZIP2_VER)/libbz2.a . + +libsnappy.a: + -rm -rf snappy-$(SNAPPY_VER) + curl -O -L ${SNAPPY_DOWNLOAD_BASE}/$(SNAPPY_VER)/snappy-$(SNAPPY_VER).tar.gz + SNAPPY_SHA256_ACTUAL=`$(SHA256_CMD) snappy-$(SNAPPY_VER).tar.gz | cut -d ' ' -f 1`; \ + if [ "$(SNAPPY_SHA256)" != "$$SNAPPY_SHA256_ACTUAL" ]; then \ + echo snappy-$(SNAPPY_VER).tar.gz checksum mismatch, expected=\"$(SNAPPY_SHA256)\" actual=\"$$SNAPPY_SHA256_ACTUAL\"; \ + exit 1; \ + fi + tar xvzf snappy-$(SNAPPY_VER).tar.gz + cd snappy-$(SNAPPY_VER) && CFLAGS='${EXTRA_CFLAGS}' CXXFLAGS='${EXTRA_CXXFLAGS}' LDFLAGS='${EXTRA_LDFLAGS}' ./configure --with-pic --enable-static --disable-shared + cd snappy-$(SNAPPY_VER) && make ${SNAPPY_MAKE_TARGET} + cp snappy-$(SNAPPY_VER)/.libs/libsnappy.a . + +liblz4.a: + -rm -rf lz4-$(LZ4_VER) + curl -O -L ${LZ4_DOWNLOAD_BASE}/v$(LZ4_VER).tar.gz + mv v$(LZ4_VER).tar.gz lz4-$(LZ4_VER).tar.gz + LZ4_SHA256_ACTUAL=`$(SHA256_CMD) lz4-$(LZ4_VER).tar.gz | cut -d ' ' -f 1`; \ + if [ "$(LZ4_SHA256)" != "$$LZ4_SHA256_ACTUAL" ]; then \ + echo lz4-$(LZ4_VER).tar.gz checksum mismatch, expected=\"$(LZ4_SHA256)\" actual=\"$$LZ4_SHA256_ACTUAL\"; \ + exit 1; \ + fi + tar xvzf lz4-$(LZ4_VER).tar.gz + cd lz4-$(LZ4_VER)/lib && make CFLAGS='-fPIC -O2 ${EXTRA_CFLAGS}' all + cp lz4-$(LZ4_VER)/lib/liblz4.a . + +libzstd.a: + -rm -rf zstd-$(ZSTD_VER) + curl -O -L ${ZSTD_DOWNLOAD_BASE}/v$(ZSTD_VER).tar.gz + mv v$(ZSTD_VER).tar.gz zstd-$(ZSTD_VER).tar.gz + ZSTD_SHA256_ACTUAL=`$(SHA256_CMD) zstd-$(ZSTD_VER).tar.gz | cut -d ' ' -f 1`; \ + if [ "$(ZSTD_SHA256)" != "$$ZSTD_SHA256_ACTUAL" ]; then \ + echo zstd-$(ZSTD_VER).tar.gz checksum mismatch, expected=\"$(ZSTD_SHA256)\" actual=\"$$ZSTD_SHA256_ACTUAL\"; \ + exit 1; \ + fi + tar xvzf zstd-$(ZSTD_VER).tar.gz + cd zstd-$(ZSTD_VER)/lib && make CFLAGS='-fPIC -O2 ${EXTRA_CFLAGS}' all + cp zstd-$(ZSTD_VER)/lib/libzstd.a . + +# A version of each $(LIBOBJECTS) compiled with -fPIC and a fixed set of static compression libraries +java_static_libobjects = $(patsubst %,jls/%,$(LIBOBJECTS)) +CLEAN_FILES += jls + +ifneq ($(ROCKSDB_JAVA_NO_COMPRESSION), 1) +JAVA_COMPRESSIONS = libz.a libbz2.a libsnappy.a liblz4.a libzstd.a +endif + +JAVA_STATIC_FLAGS = -DZLIB -DBZIP2 -DSNAPPY -DLZ4 -DZSTD +JAVA_STATIC_INCLUDES = -I./zlib-$(ZLIB_VER) -I./bzip2-$(BZIP2_VER) -I./snappy-$(SNAPPY_VER) -I./lz4-$(LZ4_VER)/lib -I./zstd-$(ZSTD_VER)/lib + +$(java_static_libobjects): jls/%.o: %.cc $(JAVA_COMPRESSIONS) + $(AM_V_CC)mkdir -p $(@D) && $(CXX) $(CXXFLAGS) $(JAVA_STATIC_FLAGS) $(JAVA_STATIC_INCLUDES) -fPIC -c $< -o $@ $(COVERAGEFLAGS) + +rocksdbjavastatic: $(java_static_libobjects) + cd java;$(MAKE) javalib; + rm -f ./java/target/$(ROCKSDBJNILIB) + $(CXX) $(CXXFLAGS) -I./java/. $(JAVA_INCLUDE) -shared -fPIC \ + -o ./java/target/$(ROCKSDBJNILIB) $(JNI_NATIVE_SOURCES) \ + $(java_static_libobjects) $(COVERAGEFLAGS) \ + $(JAVA_COMPRESSIONS) $(JAVA_STATIC_LDFLAGS) + cd java/target;strip $(STRIPFLAGS) $(ROCKSDBJNILIB) + cd java;jar -cf target/$(ROCKSDB_JAR) HISTORY*.md + cd java/target;jar -uf $(ROCKSDB_JAR) $(ROCKSDBJNILIB) + cd java/target/classes;jar -uf ../$(ROCKSDB_JAR) org/rocksdb/*.class org/rocksdb/util/*.class + cd java/target/apidocs;jar -cf ../$(ROCKSDB_JAVADOCS_JAR) * + cd java/src/main/java;jar -cf ../../../target/$(ROCKSDB_SOURCES_JAR) org + +rocksdbjavastaticrelease: rocksdbjavastatic + cd java/crossbuild && vagrant destroy -f && vagrant up linux32 && vagrant halt linux32 && vagrant up linux64 && vagrant halt linux64 + cd java;jar -cf target/$(ROCKSDB_JAR_ALL) HISTORY*.md + cd java/target;jar -uf $(ROCKSDB_JAR_ALL) librocksdbjni-*.so librocksdbjni-*.jnilib + cd java/target/classes;jar -uf ../$(ROCKSDB_JAR_ALL) org/rocksdb/*.class org/rocksdb/util/*.class + +rocksdbjavastaticreleasedocker: rocksdbjavastatic + DOCKER_LINUX_X64_CONTAINER=`docker ps -aqf name=rocksdb_linux_x64-be`; \ + if [ -z "$$DOCKER_LINUX_X64_CONTAINER" ]; then \ + docker container create --attach stdin --attach stdout --attach stderr --volume `pwd`:/rocksdb-host --name rocksdb_linux_x64-be evolvedbinary/rocksjava:centos6_x64-be /rocksdb-host/java/crossbuild/docker-build-linux-centos.sh; \ + fi + docker start -a rocksdb_linux_x64-be + DOCKER_LINUX_X86_CONTAINER=`docker ps -aqf name=rocksdb_linux_x86-be`; \ + if [ -z "$$DOCKER_LINUX_X86_CONTAINER" ]; then \ + docker container create --attach stdin --attach stdout --attach stderr --volume `pwd`:/rocksdb-host --name rocksdb_linux_x86-be evolvedbinary/rocksjava:centos6_x86-be /rocksdb-host/java/crossbuild/docker-build-linux-centos.sh; \ + fi + docker start -a rocksdb_linux_x86-be + cd java;jar -cf target/$(ROCKSDB_JAR_ALL) HISTORY*.md + cd java/target;jar -uf $(ROCKSDB_JAR_ALL) librocksdbjni-*.so librocksdbjni-*.jnilib + cd java/target/classes;jar -uf ../$(ROCKSDB_JAR_ALL) org/rocksdb/*.class org/rocksdb/util/*.class + +rocksdbjavastaticpublish: rocksdbjavastaticrelease rocksdbjavastaticpublishcentral + +rocksdbjavastaticpublishdocker: rocksdbjavastaticreleasedocker rocksdbjavastaticpublishcentral + +rocksdbjavastaticpublishcentral: + mvn gpg:sign-and-deploy-file -Durl=https://oss.sonatype.org/service/local/staging/deploy/maven2/ -DrepositoryId=sonatype-nexus-staging -DpomFile=java/rocksjni.pom -Dfile=java/target/rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-javadoc.jar -Dclassifier=javadoc + mvn gpg:sign-and-deploy-file -Durl=https://oss.sonatype.org/service/local/staging/deploy/maven2/ -DrepositoryId=sonatype-nexus-staging -DpomFile=java/rocksjni.pom -Dfile=java/target/rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-sources.jar -Dclassifier=sources + mvn gpg:sign-and-deploy-file -Durl=https://oss.sonatype.org/service/local/staging/deploy/maven2/ -DrepositoryId=sonatype-nexus-staging -DpomFile=java/rocksjni.pom -Dfile=java/target/rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-linux64.jar -Dclassifier=linux64 + mvn gpg:sign-and-deploy-file -Durl=https://oss.sonatype.org/service/local/staging/deploy/maven2/ -DrepositoryId=sonatype-nexus-staging -DpomFile=java/rocksjni.pom -Dfile=java/target/rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-linux32.jar -Dclassifier=linux32 + mvn gpg:sign-and-deploy-file -Durl=https://oss.sonatype.org/service/local/staging/deploy/maven2/ -DrepositoryId=sonatype-nexus-staging -DpomFile=java/rocksjni.pom -Dfile=java/target/rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-osx.jar -Dclassifier=osx + mvn gpg:sign-and-deploy-file -Durl=https://oss.sonatype.org/service/local/staging/deploy/maven2/ -DrepositoryId=sonatype-nexus-staging -DpomFile=java/rocksjni.pom -Dfile=java/target/rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-win64.jar -Dclassifier=win64 + mvn gpg:sign-and-deploy-file -Durl=https://oss.sonatype.org/service/local/staging/deploy/maven2/ -DrepositoryId=sonatype-nexus-staging -DpomFile=java/rocksjni.pom -Dfile=java/target/rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH).jar + +# A version of each $(LIBOBJECTS) compiled with -fPIC +java_libobjects = $(patsubst %,jl/%,$(LIBOBJECTS)) +CLEAN_FILES += jl + +$(java_libobjects): jl/%.o: %.cc + $(AM_V_CC)mkdir -p $(@D) && $(CXX) $(CXXFLAGS) -fPIC -c $< -o $@ $(COVERAGEFLAGS) + +rocksdbjava: $(java_libobjects) + $(AM_V_GEN)cd java;$(MAKE) javalib; + $(AM_V_at)rm -f ./java/target/$(ROCKSDBJNILIB) + $(AM_V_at)$(CXX) $(CXXFLAGS) -I./java/. $(JAVA_INCLUDE) -shared -fPIC -o ./java/target/$(ROCKSDBJNILIB) $(JNI_NATIVE_SOURCES) $(java_libobjects) $(JAVA_LDFLAGS) $(COVERAGEFLAGS) + $(AM_V_at)cd java;jar -cf target/$(ROCKSDB_JAR) HISTORY*.md + $(AM_V_at)cd java/target;jar -uf $(ROCKSDB_JAR) $(ROCKSDBJNILIB) + $(AM_V_at)cd java/target/classes;jar -uf ../$(ROCKSDB_JAR) org/rocksdb/*.class org/rocksdb/util/*.class jclean: cd java;$(MAKE) clean; - rm -f $(ROCKSDBJNILIB) -jtest: +jtest_compile: rocksdbjava + cd java;$(MAKE) java_test + +jtest_run: + cd java;$(MAKE) run_test + +jtest: rocksdbjava cd java;$(MAKE) sample;$(MAKE) test; jdb_bench: cd java;$(MAKE) db_bench; +commit_prereq: build_tools/rocksdb-lego-determinator \ + build_tools/precommit_checker.py + J=$(J) build_tools/precommit_checker.py unit unit_481 clang_unit release release_481 clang_release tsan asan ubsan lite unit_non_shm + $(MAKE) clean && $(MAKE) jclean && $(MAKE) rocksdbjava; + # --------------------------------------------------------------------------- # Platform-specific compilation # --------------------------------------------------------------------------- @@ -548,32 +1726,27 @@ IOSVERSION=$(shell defaults read $(PLATFORMSROOT)/iPhoneOS.platform/version CFBu else .cc.o: - $(CXX) $(CXXFLAGS) -c $< -o $@ $(COVERAGEFLAGS) + $(AM_V_CC)$(CXX) $(CXXFLAGS) -c $< -o $@ $(COVERAGEFLAGS) .c.o: - $(CC) $(CFLAGS) -c $< -o $@ + $(AM_V_CC)$(CC) $(CFLAGS) -c $< -o $@ endif # --------------------------------------------------------------------------- # Source files dependencies detection # --------------------------------------------------------------------------- +all_sources = $(LIB_SOURCES) $(MAIN_SOURCES) $(MOCK_LIB_SOURCES) $(TOOL_LIB_SOURCES) $(BENCH_LIB_SOURCES) $(TEST_LIB_SOURCES) $(EXP_LIB_SOURCES) +DEPFILES = $(all_sources:.cc=.d) + # Add proper dependency support so changing a .h file forces a .cc file to # rebuild. # The .d file indicates .cc file's dependencies on .h files. We generate such # dependency by g++'s -MM option, whose output is a make dependency rule. -# The sed command makes sure the "target" file in the generated .d file has -# the correct path prefix. -%.d: %.cc - $(CXX) $(CXXFLAGS) $(PLATFORM_SHARED_CFLAGS) -MM $< -o $@ -ifeq ($(PLATFORM), OS_MACOSX) - @sed -i '' -e 's,.*:,$*.o:,' $@ -else - @sed -i -e 's,.*:,$*.o:,' $@ -endif - -DEPFILES = $(filter-out util/build_version.d,$(SOURCES:.cc=.d)) +$(DEPFILES): %.d: %.cc + @$(CXX) $(CXXFLAGS) $(PLATFORM_SHARED_CFLAGS) \ + -MM -MT'$@' -MT'$(<:.cc=.o)' "$<" -o '$@' depend: $(DEPFILES) @@ -585,8 +1758,12 @@ ifneq ($(MAKECMDGOALS),clean) ifneq ($(MAKECMDGOALS),format) ifneq ($(MAKECMDGOALS),jclean) ifneq ($(MAKECMDGOALS),jtest) +ifneq ($(MAKECMDGOALS),package) +ifneq ($(MAKECMDGOALS),analyze) -include $(DEPFILES) endif endif endif endif +endif +endif diff --git a/PATENTS b/PATENTS deleted file mode 100644 index 8a6fca4d2ba..00000000000 --- a/PATENTS +++ /dev/null @@ -1,23 +0,0 @@ -Additional Grant of Patent Rights - -“Software” means the rocksdb software distributed by Facebook, Inc. - -Facebook hereby grants you a perpetual, worldwide, royalty-free, -non-exclusive, irrevocable (subject to the termination provision below) -license under any rights in any patent claims owned by Facebook, to make, -have made, use, sell, offer to sell, import, and otherwise transfer the -Software. For avoidance of doubt, no license is granted under Facebook’s -rights in any patent claims that are infringed by (i) modifications to the -Software made by you or a third party, or (ii) the Software in combination -with any software or other technology provided by you or a third party. - -The license granted hereunder will terminate, automatically and without -notice, for anyone that makes any claim (including by filing any lawsuit, -assertion or other action) alleging (a) direct, indirect, or contributory -infringement or inducement to infringe any patent: (i) by Facebook or any -of its subsidiaries or affiliates, whether or not such claim is related -to the Software, (ii) by any party if such claim arises in whole or in -part from any software, product or service of Facebook or any of its -subsidiaries or affiliates, whether or not such claim is related to the -Software, or (iii) by any party relating to the Software; or (b) that -any right in any patent claim of Facebook is invalid or unenforceable. diff --git a/README.md b/README.md index bda801fd77b..550c352b882 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,11 @@ ## RocksDB: A Persistent Key-Value Store for Flash and RAM Storage [![Build Status](https://travis-ci.org/facebook/rocksdb.svg?branch=master)](https://travis-ci.org/facebook/rocksdb) +[![Build status](https://ci.appveyor.com/api/projects/status/fbgfu0so3afcno78/branch/master?svg=true)](https://ci.appveyor.com/project/Facebook/rocksdb/branch/master) + RocksDB is developed and maintained by Facebook Database Engineering Team. -It is built on on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) +It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com) This code is a library that forms the core building block for a fast diff --git a/ROCKSDB_LITE.md b/ROCKSDB_LITE.md index e7e3752c8c5..41cfbecc2cc 100644 --- a/ROCKSDB_LITE.md +++ b/ROCKSDB_LITE.md @@ -8,6 +8,7 @@ Some examples of the features disabled by ROCKSDB_LITE: * No support for replication (which we provide in form of TrasactionalIterator) * No advanced monitoring tools * No special-purpose memtables that are highly optimized for specific use cases +* No Transactions When adding a new big feature to RocksDB, please add ROCKSDB_LITE compile guard if: * Nobody from mobile really needs your feature, diff --git a/TARGETS b/TARGETS new file mode 100644 index 00000000000..ac85eab93c7 --- /dev/null +++ b/TARGETS @@ -0,0 +1,534 @@ + +import os + +TARGETS_PATH = os.path.dirname(__file__) +REPO_PATH = TARGETS_PATH[(TARGETS_PATH.find('fbcode/') + len('fbcode/')):] + "/" +BUCK_BINS = "buck-out/gen/" + REPO_PATH +TEST_RUNNER = REPO_PATH + "buckifier/rocks_test_runner.sh" +rocksdb_compiler_flags = [ + "-fno-builtin-memcmp", + "-DROCKSDB_PLATFORM_POSIX", + "-DROCKSDB_LIB_IO_POSIX", + "-DROCKSDB_FALLOCATE_PRESENT", + "-DROCKSDB_MALLOC_USABLE_SIZE", + "-DROCKSDB_RANGESYNC_PRESENT", + "-DROCKSDB_SCHED_GETCPU_PRESENT", + "-DROCKSDB_SUPPORT_THREAD_LOCAL", + "-DOS_LINUX", + "-DROCKSDB_UBSAN_RUN", + # Flags to enable libs we include + "-DSNAPPY", + "-DZLIB", + "-DBZIP2", + "-DLZ4", + "-DZSTD", + "-DGFLAGS=gflags", + "-DNUMA", + "-DTBB", + # Needed to compile in fbcode + "-Wno-expansion-to-defined", +] + +rocksdb_external_deps = [ + ('bzip2', None, 'bz2'), + ('snappy', None, "snappy"), + ('zlib', None, 'z'), + ('gflags', None, 'gflags'), + ('lz4', None, 'lz4'), + ('zstd', None), + ('tbb', None), + ("numa", None, "numa"), + ("googletest", None, "gtest"), +] + +rocksdb_preprocessor_flags = [ + # Directories with files for #include + "-I" + REPO_PATH + "include/", + "-I" + REPO_PATH, +] + +rocksdb_arch_preprocessor_flags = { + "x86_64": ["-DHAVE_SSE42"], +} + +cpp_library( + name = "rocksdb_lib", + headers = AutoHeaders.RECURSIVE_GLOB, + srcs = [ + "cache/clock_cache.cc", + "cache/lru_cache.cc", + "cache/sharded_cache.cc", + "db/builder.cc", + "db/c.cc", + "db/column_family.cc", + "db/compacted_db_impl.cc", + "db/compaction.cc", + "db/compaction_iterator.cc", + "db/compaction_job.cc", + "db/compaction_picker.cc", + "db/compaction_picker_universal.cc", + "db/convenience.cc", + "db/db_filesnapshot.cc", + "db/db_impl.cc", + "db/db_impl_write.cc", + "db/db_impl_compaction_flush.cc", + "db/db_impl_files.cc", + "db/db_impl_open.cc", + "db/db_impl_debug.cc", + "db/db_impl_experimental.cc", + "db/db_impl_readonly.cc", + "db/db_info_dumper.cc", + "db/db_iter.cc", + "db/dbformat.cc", + "db/event_helpers.cc", + "db/experimental.cc", + "db/external_sst_file_ingestion_job.cc", + "db/file_indexer.cc", + "db/flush_job.cc", + "db/flush_scheduler.cc", + "db/forward_iterator.cc", + "db/internal_stats.cc", + "db/log_reader.cc", + "db/log_writer.cc", + "db/malloc_stats.cc", + "db/managed_iterator.cc", + "db/memtable.cc", + "db/memtable_list.cc", + "db/merge_helper.cc", + "db/merge_operator.cc", + "db/range_del_aggregator.cc", + "db/repair.cc", + "db/snapshot_impl.cc", + "db/table_cache.cc", + "db/table_properties_collector.cc", + "db/transaction_log_impl.cc", + "db/version_builder.cc", + "db/version_edit.cc", + "db/version_set.cc", + "db/wal_manager.cc", + "db/write_batch.cc", + "db/write_batch_base.cc", + "db/write_controller.cc", + "db/write_thread.cc", + "env/env.cc", + "env/env_chroot.cc", + "env/env_encryption.cc", + "env/env_hdfs.cc", + "env/env_posix.cc", + "env/io_posix.cc", + "env/mock_env.cc", + "memtable/alloc_tracker.cc", + "memtable/hash_cuckoo_rep.cc", + "memtable/hash_linklist_rep.cc", + "memtable/hash_skiplist_rep.cc", + "memtable/skiplistrep.cc", + "memtable/vectorrep.cc", + "memtable/write_buffer_manager.cc", + "monitoring/histogram.cc", + "monitoring/histogram_windowing.cc", + "monitoring/instrumented_mutex.cc", + "monitoring/iostats_context.cc", + "monitoring/perf_context.cc", + "monitoring/perf_level.cc", + "monitoring/statistics.cc", + "monitoring/thread_status_impl.cc", + "monitoring/thread_status_updater.cc", + "monitoring/thread_status_updater_debug.cc", + "monitoring/thread_status_util.cc", + "monitoring/thread_status_util_debug.cc", + "options/cf_options.cc", + "options/db_options.cc", + "options/options.cc", + "options/options_helper.cc", + "options/options_parser.cc", + "options/options_sanity_check.cc", + "port/port_posix.cc", + "port/stack_trace.cc", + "table/adaptive_table_factory.cc", + "table/block.cc", + "table/block_based_filter_block.cc", + "table/block_based_table_builder.cc", + "table/block_based_table_factory.cc", + "table/block_based_table_reader.cc", + "table/block_builder.cc", + "table/block_prefix_index.cc", + "table/bloom_block.cc", + "table/cuckoo_table_builder.cc", + "table/cuckoo_table_factory.cc", + "table/cuckoo_table_reader.cc", + "table/flush_block_policy.cc", + "table/format.cc", + "table/full_filter_block.cc", + "table/get_context.cc", + "table/index_builder.cc", + "table/iterator.cc", + "table/merging_iterator.cc", + "table/meta_blocks.cc", + "table/partitioned_filter_block.cc", + "table/persistent_cache_helper.cc", + "table/plain_table_builder.cc", + "table/plain_table_factory.cc", + "table/plain_table_index.cc", + "table/plain_table_key_coding.cc", + "table/plain_table_reader.cc", + "table/sst_file_writer.cc", + "table/table_properties.cc", + "table/two_level_iterator.cc", + "tools/dump/db_dump_tool.cc", + "util/arena.cc", + "util/auto_roll_logger.cc", + "util/bloom.cc", + "util/build_version.cc", + "util/coding.cc", + "util/compaction_job_stats_impl.cc", + "util/comparator.cc", + "util/concurrent_arena.cc", + "util/crc32c.cc", + "util/delete_scheduler.cc", + "util/dynamic_bloom.cc", + "util/event_logger.cc", + "util/file_reader_writer.cc", + "util/file_util.cc", + "util/filename.cc", + "util/filter_policy.cc", + "util/hash.cc", + "util/log_buffer.cc", + "util/murmurhash.cc", + "util/random.cc", + "util/rate_limiter.cc", + "util/slice.cc", + "util/sst_file_manager_impl.cc", + "util/status.cc", + "util/status_message.cc", + "util/string_util.cc", + "util/sync_point.cc", + "util/thread_local.cc", + "util/threadpool_imp.cc", + "util/transaction_test_util.cc", + "util/xxhash.cc", + "utilities/backupable/backupable_db.cc", + "utilities/blob_db/blob_db.cc", + "utilities/blob_db/blob_db_impl.cc", + "utilities/blob_db/blob_file.cc", + "utilities/blob_db/blob_log_reader.cc", + "utilities/blob_db/blob_log_writer.cc", + "utilities/blob_db/blob_log_format.cc", + "utilities/blob_db/ttl_extractor.cc", + "utilities/cassandra/cassandra_compaction_filter.cc", + "utilities/cassandra/format.cc", + "utilities/cassandra/merge_operator.cc", + "utilities/checkpoint/checkpoint_impl.cc", + "utilities/compaction_filters/remove_emptyvalue_compactionfilter.cc", + "utilities/convenience/info_log_finder.cc", + "utilities/date_tiered/date_tiered_db_impl.cc", + "utilities/debug.cc", + "utilities/document/document_db.cc", + "utilities/document/json_document.cc", + "utilities/document/json_document_builder.cc", + "utilities/env_mirror.cc", + "utilities/env_timed.cc", + "utilities/geodb/geodb_impl.cc", + "utilities/leveldb_options/leveldb_options.cc", + "utilities/lua/rocks_lua_compaction_filter.cc", + "utilities/memory/memory_util.cc", + "utilities/merge_operators/max.cc", + "utilities/merge_operators/put.cc", + "utilities/merge_operators/string_append/stringappend.cc", + "utilities/merge_operators/string_append/stringappend2.cc", + "utilities/merge_operators/uint64add.cc", + "utilities/option_change_migration/option_change_migration.cc", + "utilities/options/options_util.cc", + "utilities/persistent_cache/block_cache_tier.cc", + "utilities/persistent_cache/block_cache_tier_file.cc", + "utilities/persistent_cache/block_cache_tier_metadata.cc", + "utilities/persistent_cache/persistent_cache_tier.cc", + "utilities/persistent_cache/volatile_tier_impl.cc", + "utilities/redis/redis_lists.cc", + "utilities/simulator_cache/sim_cache.cc", + "utilities/spatialdb/spatial_db.cc", + "utilities/table_properties_collectors/compact_on_deletion_collector.cc", + "utilities/transactions/optimistic_transaction_db_impl.cc", + "utilities/transactions/optimistic_transaction.cc", + "utilities/transactions/transaction_base.cc", + "utilities/transactions/pessimistic_transaction_db.cc", + "utilities/transactions/transaction_db_mutex_impl.cc", + "utilities/transactions/pessimistic_transaction.cc", + "utilities/transactions/transaction_lock_mgr.cc", + "utilities/transactions/transaction_util.cc", + "utilities/transactions/write_prepared_txn.cc", + "utilities/ttl/db_ttl_impl.cc", + "utilities/write_batch_with_index/write_batch_with_index.cc", + "utilities/write_batch_with_index/write_batch_with_index_internal.cc", + "tools/ldb_cmd.cc", + "tools/ldb_tool.cc", + "tools/sst_dump_tool.cc", + "utilities/blob_db/blob_dump_tool.cc", + ], + deps = [], + preprocessor_flags = rocksdb_preprocessor_flags, + arch_preprocessor_flags = rocksdb_arch_preprocessor_flags, + compiler_flags = rocksdb_compiler_flags, + external_deps = rocksdb_external_deps, +) + +cpp_library( + name = "rocksdb_test_lib", + headers = AutoHeaders.RECURSIVE_GLOB, + srcs = [ + "table/mock_table.cc", + "util/fault_injection_test_env.cc", + "util/testharness.cc", + "util/testutil.cc", + "db/db_test_util.cc", + "utilities/cassandra/test_utils.cc", + "utilities/col_buf_encoder.cc", + "utilities/col_buf_decoder.cc", + "utilities/column_aware_encoding_util.cc", + ], + deps = [":rocksdb_lib"], + preprocessor_flags = rocksdb_preprocessor_flags, + arch_preprocessor_flags = rocksdb_arch_preprocessor_flags, + compiler_flags = rocksdb_compiler_flags, + external_deps = rocksdb_external_deps, +) + +cpp_library( + name = "rocksdb_tools_lib", + headers = AutoHeaders.RECURSIVE_GLOB, + srcs = [ + "tools/db_bench_tool.cc", + "util/testutil.cc", + ], + deps = [":rocksdb_lib"], + preprocessor_flags = rocksdb_preprocessor_flags, + arch_preprocessor_flags = rocksdb_arch_preprocessor_flags, + compiler_flags = rocksdb_compiler_flags, + external_deps = rocksdb_external_deps, +) + +cpp_library( + name = "env_basic_test_lib", + headers = AutoHeaders.RECURSIVE_GLOB, + srcs = ["env/env_basic_test.cc"], + deps = [":rocksdb_test_lib"], + preprocessor_flags = rocksdb_preprocessor_flags, + arch_preprocessor_flags = rocksdb_arch_preprocessor_flags, + compiler_flags = rocksdb_compiler_flags, + external_deps = rocksdb_external_deps, +) + +# [test_name, test_src, test_type] +ROCKS_TESTS = [['arena_test', 'util/arena_test.cc', 'serial'], + ['auto_roll_logger_test', 'util/auto_roll_logger_test.cc', 'serial'], + ['autovector_test', 'util/autovector_test.cc', 'serial'], + ['backupable_db_test', + 'utilities/backupable/backupable_db_test.cc', + 'parallel'], + ['blob_db_test', 'utilities/blob_db/blob_db_test.cc', 'serial'], + ['block_based_filter_block_test', + 'table/block_based_filter_block_test.cc', + 'serial'], + ['block_test', 'table/block_test.cc', 'serial'], + ['bloom_test', 'util/bloom_test.cc', 'serial'], + ['c_test', 'db/c_test.c', 'serial'], + ['cache_test', 'cache/cache_test.cc', 'serial'], + ['cassandra_format_test', + 'utilities/cassandra/cassandra_format_test.cc', + 'serial'], + ['cassandra_functional_test', + 'utilities/cassandra/cassandra_functional_test.cc', + 'serial'], + ['cassandra_row_merge_test', + 'utilities/cassandra/cassandra_row_merge_test.cc', + 'serial'], + ['cassandra_serialize_test', + 'utilities/cassandra/cassandra_serialize_test.cc', + 'serial'], + ['checkpoint_test', 'utilities/checkpoint/checkpoint_test.cc', 'serial'], + ['cleanable_test', 'table/cleanable_test.cc', 'serial'], + ['coding_test', 'util/coding_test.cc', 'serial'], + ['column_aware_encoding_test', + 'utilities/column_aware_encoding_test.cc', + 'serial'], + ['column_family_test', 'db/column_family_test.cc', 'serial'], + ['compact_files_test', 'db/compact_files_test.cc', 'serial'], + ['compact_on_deletion_collector_test', + 'utilities/table_properties_collectors/compact_on_deletion_collector_test.cc', + 'serial'], + ['compaction_iterator_test', 'db/compaction_iterator_test.cc', 'serial'], + ['compaction_job_stats_test', 'db/compaction_job_stats_test.cc', 'serial'], + ['compaction_job_test', 'db/compaction_job_test.cc', 'serial'], + ['compaction_picker_test', 'db/compaction_picker_test.cc', 'serial'], + ['comparator_db_test', 'db/comparator_db_test.cc', 'serial'], + ['corruption_test', 'db/corruption_test.cc', 'serial'], + ['crc32c_test', 'util/crc32c_test.cc', 'serial'], + ['cuckoo_table_builder_test', 'table/cuckoo_table_builder_test.cc', 'serial'], + ['cuckoo_table_db_test', 'db/cuckoo_table_db_test.cc', 'serial'], + ['cuckoo_table_reader_test', 'table/cuckoo_table_reader_test.cc', 'serial'], + ['date_tiered_test', 'utilities/date_tiered/date_tiered_test.cc', 'serial'], + ['db_basic_test', 'db/db_basic_test.cc', 'serial'], + ['db_blob_index_test', 'db/db_blob_index_test.cc', 'serial'], + ['db_block_cache_test', 'db/db_block_cache_test.cc', 'serial'], + ['db_bloom_filter_test', 'db/db_bloom_filter_test.cc', 'serial'], + ['db_compaction_filter_test', 'db/db_compaction_filter_test.cc', 'parallel'], + ['db_compaction_test', 'db/db_compaction_test.cc', 'parallel'], + ['db_dynamic_level_test', 'db/db_dynamic_level_test.cc', 'serial'], + ['db_encryption_test', 'db/db_encryption_test.cc', 'serial'], + ['db_flush_test', 'db/db_flush_test.cc', 'serial'], + ['db_inplace_update_test', 'db/db_inplace_update_test.cc', 'serial'], + ['db_io_failure_test', 'db/db_io_failure_test.cc', 'serial'], + ['db_iter_test', 'db/db_iter_test.cc', 'serial'], + ['db_iterator_test', 'db/db_iterator_test.cc', 'serial'], + ['db_log_iter_test', 'db/db_log_iter_test.cc', 'serial'], + ['db_memtable_test', 'db/db_memtable_test.cc', 'serial'], + ['db_merge_operator_test', 'db/db_merge_operator_test.cc', 'serial'], + ['db_options_test', 'db/db_options_test.cc', 'serial'], + ['db_properties_test', 'db/db_properties_test.cc', 'serial'], + ['db_range_del_test', 'db/db_range_del_test.cc', 'serial'], + ['db_sst_test', 'db/db_sst_test.cc', 'parallel'], + ['db_statistics_test', 'db/db_statistics_test.cc', 'serial'], + ['db_table_properties_test', 'db/db_table_properties_test.cc', 'serial'], + ['db_tailing_iter_test', 'db/db_tailing_iter_test.cc', 'serial'], + ['db_test', 'db/db_test.cc', 'parallel'], + ['db_test2', 'db/db_test2.cc', 'serial'], + ['db_universal_compaction_test', + 'db/db_universal_compaction_test.cc', + 'parallel'], + ['db_wal_test', 'db/db_wal_test.cc', 'parallel'], + ['db_write_test', 'db/db_write_test.cc', 'serial'], + ['dbformat_test', 'db/dbformat_test.cc', 'serial'], + ['delete_scheduler_test', 'util/delete_scheduler_test.cc', 'serial'], + ['deletefile_test', 'db/deletefile_test.cc', 'serial'], + ['document_db_test', 'utilities/document/document_db_test.cc', 'serial'], + ['dynamic_bloom_test', 'util/dynamic_bloom_test.cc', 'serial'], + ['env_basic_test', 'env/env_basic_test.cc', 'serial'], + ['env_test', 'env/env_test.cc', 'serial'], + ['env_timed_test', 'utilities/env_timed_test.cc', 'serial'], + ['event_logger_test', 'util/event_logger_test.cc', 'serial'], + ['external_sst_file_basic_test', + 'db/external_sst_file_basic_test.cc', + 'serial'], + ['external_sst_file_test', 'db/external_sst_file_test.cc', 'parallel'], + ['fault_injection_test', 'db/fault_injection_test.cc', 'parallel'], + ['file_indexer_test', 'db/file_indexer_test.cc', 'serial'], + ['file_reader_writer_test', 'util/file_reader_writer_test.cc', 'serial'], + ['filelock_test', 'util/filelock_test.cc', 'serial'], + ['filename_test', 'db/filename_test.cc', 'serial'], + ['flush_job_test', 'db/flush_job_test.cc', 'serial'], + ['full_filter_block_test', 'table/full_filter_block_test.cc', 'serial'], + ['geodb_test', 'utilities/geodb/geodb_test.cc', 'serial'], + ['hash_table_test', + 'utilities/persistent_cache/hash_table_test.cc', + 'serial'], + ['hash_test', 'util/hash_test.cc', 'serial'], + ['heap_test', 'util/heap_test.cc', 'serial'], + ['histogram_test', 'monitoring/histogram_test.cc', 'serial'], + ['inlineskiplist_test', 'memtable/inlineskiplist_test.cc', 'parallel'], + ['iostats_context_test', 'monitoring/iostats_context_test.cc', 'serial'], + ['json_document_test', 'utilities/document/json_document_test.cc', 'serial'], + ['ldb_cmd_test', 'tools/ldb_cmd_test.cc', 'serial'], + ['listener_test', 'db/listener_test.cc', 'serial'], + ['log_test', 'db/log_test.cc', 'serial'], + ['lru_cache_test', 'cache/lru_cache_test.cc', 'serial'], + ['manual_compaction_test', 'db/manual_compaction_test.cc', 'parallel'], + ['memory_test', 'utilities/memory/memory_test.cc', 'serial'], + ['memtable_list_test', 'db/memtable_list_test.cc', 'serial'], + ['merge_helper_test', 'db/merge_helper_test.cc', 'serial'], + ['merge_test', 'db/merge_test.cc', 'serial'], + ['merger_test', 'table/merger_test.cc', 'serial'], + ['mock_env_test', 'env/mock_env_test.cc', 'serial'], + ['object_registry_test', 'utilities/object_registry_test.cc', 'serial'], + ['optimistic_transaction_test', + 'utilities/transactions/optimistic_transaction_test.cc', + 'serial'], + ['option_change_migration_test', + 'utilities/option_change_migration/option_change_migration_test.cc', + 'serial'], + ['options_file_test', 'db/options_file_test.cc', 'serial'], + ['options_settable_test', 'options/options_settable_test.cc', 'serial'], + ['options_test', 'options/options_test.cc', 'serial'], + ['options_util_test', 'utilities/options/options_util_test.cc', 'serial'], + ['partitioned_filter_block_test', + 'table/partitioned_filter_block_test.cc', + 'serial'], + ['perf_context_test', 'db/perf_context_test.cc', 'serial'], + ['persistent_cache_test', + 'utilities/persistent_cache/persistent_cache_test.cc', + 'parallel'], + ['plain_table_db_test', 'db/plain_table_db_test.cc', 'serial'], + ['prefix_test', 'db/prefix_test.cc', 'serial'], + ['range_del_aggregator_test', 'db/range_del_aggregator_test.cc', 'serial'], + ['rate_limiter_test', 'util/rate_limiter_test.cc', 'serial'], + ['reduce_levels_test', 'tools/reduce_levels_test.cc', 'serial'], + ['repair_test', 'db/repair_test.cc', 'serial'], + ['sim_cache_test', 'utilities/simulator_cache/sim_cache_test.cc', 'serial'], + ['skiplist_test', 'memtable/skiplist_test.cc', 'serial'], + ['slice_transform_test', 'util/slice_transform_test.cc', 'serial'], + ['spatial_db_test', 'utilities/spatialdb/spatial_db_test.cc', 'serial'], + ['sst_dump_test', 'tools/sst_dump_test.cc', 'serial'], + ['statistics_test', 'monitoring/statistics_test.cc', 'serial'], + ['stringappend_test', + 'utilities/merge_operators/string_append/stringappend_test.cc', + 'serial'], + ['table_properties_collector_test', + 'db/table_properties_collector_test.cc', + 'serial'], + ['table_test', 'table/table_test.cc', 'parallel'], + ['thread_list_test', 'util/thread_list_test.cc', 'serial'], + ['thread_local_test', 'util/thread_local_test.cc', 'serial'], + ['timer_queue_test', 'util/timer_queue_test.cc', 'serial'], + ['transaction_test', 'utilities/transactions/transaction_test.cc', 'serial'], + ['ttl_test', 'utilities/ttl/ttl_test.cc', 'serial'], + ['util_merge_operators_test', + 'utilities/util_merge_operators_test.cc', + 'serial'], + ['version_builder_test', 'db/version_builder_test.cc', 'serial'], + ['version_edit_test', 'db/version_edit_test.cc', 'serial'], + ['version_set_test', 'db/version_set_test.cc', 'serial'], + ['wal_manager_test', 'db/wal_manager_test.cc', 'serial'], + ['write_batch_test', 'db/write_batch_test.cc', 'serial'], + ['write_batch_with_index_test', + 'utilities/write_batch_with_index/write_batch_with_index_test.cc', + 'serial'], + ['write_buffer_manager_test', + 'memtable/write_buffer_manager_test.cc', + 'serial'], + ['write_callback_test', 'db/write_callback_test.cc', 'serial'], + ['write_controller_test', 'db/write_controller_test.cc', 'serial']] + + +# Generate a test rule for each entry in ROCKS_TESTS +for test_cfg in ROCKS_TESTS: + test_name = test_cfg[0] + test_cc = test_cfg[1] + ttype = "gtest" if test_cfg[2] == "parallel" else "simple" + test_bin = test_name + "_bin" + + cpp_binary ( + name = test_bin, + srcs = [test_cc], + deps = [":rocksdb_test_lib"], + preprocessor_flags = rocksdb_preprocessor_flags, + arch_preprocessor_flags = rocksdb_arch_preprocessor_flags, + compiler_flags = rocksdb_compiler_flags, + external_deps = rocksdb_external_deps, + ) + + custom_unittest( + name = test_name, + type = ttype, + deps = [":" + test_bin], + command = [TEST_RUNNER, BUCK_BINS + test_bin] + ) + +custom_unittest( + name = "make_rocksdbjavastatic", + type = "simple", + command = ["internal_repo_rocksdb/make_rocksdbjavastatic.sh"], +) + +custom_unittest( + name = "make_rocksdb_lite_release", + type = "simple", + command = ["internal_repo_rocksdb/make_rocksdb_lite_release.sh"], +) diff --git a/USERS.md b/USERS.md new file mode 100644 index 00000000000..7be093f9582 --- /dev/null +++ b/USERS.md @@ -0,0 +1,85 @@ +This document lists users of RocksDB and their use cases. If you are using RocksDB, please open a pull request and add yourself to the list. + +## Facebook +At Facebook, we use RocksDB as storage engines in multiple data management services and a backend for many different stateful services, including: + +1. MyRocks -- https://github.com/MySQLOnRocksDB/mysql-5.6 +2. MongoRocks -- https://github.com/mongodb-partners/mongo-rocks +3. ZippyDB -- Facebook's distributed key-value store with Paxos-style replication, built on top of RocksDB.[*] https://www.youtube.com/watch?v=DfiN7pG0D0khtt +4. Laser -- Laser is a high query throughput, low (millisecond) latency, key-value storage service built on top of RocksDB.[*] +4. Dragon -- a distributed graph query engine. https://code.facebook.com/posts/1737605303120405/dragon-a-distributed-graph-query-engine/ +5. Stylus -- a low-level stream processing framework writtenin C++.[*] + +[*] https://research.facebook.com/publications/realtime-data-processing-at-facebook/ + +## LinkedIn +Two different use cases at Linkedin are using RocksDB as a storage engine: + +1. LinkedIn's follow feed for storing user's activities. Check out the blog post: https://engineering.linkedin.com/blog/2016/03/followfeed--linkedin-s-feed-made-faster-and-smarter +2. Apache Samza, open source framework for stream processing + +Learn more about those use cases in a Tech Talk by Ankit Gupta and Naveen Somasundaram: http://www.youtube.com/watch?v=plqVp_OnSzg + +## Yahoo +Yahoo is using RocksDB as a storage engine for their biggest distributed data store Sherpa. Learn more about it here: http://yahooeng.tumblr.com/post/120730204806/sherpa-scales-new-heights + +## CockroachDB +CockroachDB is an open-source geo-replicated transactional database (still in development). They are using RocksDB as their storage engine. Check out their github: https://github.com/cockroachdb/cockroach + +## DNANexus +DNANexus is using RocksDB to speed up processing of genomics data. +You can learn more from this great blog post by Mike Lin: http://devblog.dnanexus.com/faster-bam-sorting-with-samtools-and-rocksdb/ + +## Iron.io +Iron.io is using RocksDB as a storage engine for their distributed queueing system. +Learn more from Tech Talk by Reed Allman: http://www.youtube.com/watch?v=HTjt6oj-RL4 + +## Tango Me +Tango is using RocksDB as a graph storage to store all users' connection data and other social activity data. + +## Turn +Turn is using RocksDB as a storage layer for their key/value store, serving at peak 2.4MM QPS out of different datacenters. +Check out our RocksDB Protobuf merge operator at: https://github.com/vladb38/rocksdb_protobuf + +## Santanader UK/Cloudera Profession Services +Check out their blog post: http://blog.cloudera.com/blog/2015/08/inside-santanders-near-real-time-data-ingest-architecture/ + +## Airbnb +Airbnb is using RocksDB as a storage engine for their personalized search service. You can learn more about it here: https://www.youtube.com/watch?v=ASQ6XMtogMs + +## Pinterest +Pinterest's Object Retrieval System uses RocksDB for storage: https://www.youtube.com/watch?v=MtFEVEs_2Vo + +## Smyte +[Smyte](https://www.smyte.com/) uses RocksDB as the storage layer for their core key-value storage, high-performance counters and time-windowed HyperLogLog services. + +## Rakuten Marketing +[Rakuten Marketing](https://marketing.rakuten.com/) uses RocksDB as the disk cache layer for the real-time bidding service in their Performance DSP. + +## VWO, Wingify +[VWO's](https://vwo.com/) Smart Code checker and URL helper uses RocksDB to store all the URLs where VWO's Smart Code is installed. + +## quasardb +[quasardb](https://www.quasardb.net) is a high-performance, distributed, transactional key-value database that integrates well with in-memory analytics engines such as Apache Spark. +quasardb uses a heavily tuned RocksDB as its persistence layer. + +## Netflix +[Netflix](http://techblog.netflix.com/2016/05/application-data-caching-using-ssds.html) Netflix uses RocksDB on AWS EC2 instances with local SSD drives to cache application data. + +## TiKV +[TiKV](https://github.com/pingcap/tikv) is a GEO-replicated, high-performance, distributed, transactional key-value database. TiKV is powered by Rust and Raft. TiKV uses RocksDB as its persistence layer. + +## Apache Flink +[Apache Flink](https://flink.apache.org/news/2016/03/08/release-1.0.0.html) uses RocksDB to store state locally on a machine. + +## Dgraph +[Dgraph](https://github.com/dgraph-io/dgraph) is an open-source, scalable, distributed, low latency, high throughput Graph database .They use RocksDB to store state locally on a machine. + +## Uber +[Uber](http://eng.uber.com/cherami/) uses RocksDB as a durable and scalable task queue. + +## 360 Pika +[360](http://www.360.cn/) [Pika](https://github.com/Qihoo360/pika) is a nosql compatible with redis. With the huge amount of data stored, redis may suffer for a capacity bottleneck, and pika was born for solving it. It has widely been widely used in many company + +## LzLabs +LzLabs is using RocksDB as a storage engine in their multi-database distributed framework to store application configuration and user data. diff --git a/Vagrantfile b/Vagrantfile new file mode 100644 index 00000000000..d7c2991d799 --- /dev/null +++ b/Vagrantfile @@ -0,0 +1,34 @@ +# Vagrant file +Vagrant.configure("2") do |config| + + config.vm.provider "virtualbox" do |v| + v.memory = 4096 + v.cpus = 2 + end + + config.vm.define "ubuntu14" do |box| + box.vm.box = "ubuntu/trusty64" + end + + config.vm.define "centos65" do |box| + box.vm.box = "chef/centos-6.5" + end + + config.vm.define "FreeBSD10" do |box| + box.vm.guest = :freebsd + box.vm.box = "robin/freebsd-10" + # FreeBSD does not support 'mount_virtualbox_shared_folder', use NFS + box.vm.synced_folder ".", "/vagrant", :nfs => true, id: "vagrant-root" + box.vm.network "private_network", ip: "10.0.1.10" + + # build everything after creating VM, skip using --no-provision + box.vm.provision "shell", inline: <<-SCRIPT + pkg install -y gmake clang35 + export CXX=/usr/local/bin/clang++35 + cd /vagrant + gmake clean + gmake all OPT=-g + SCRIPT + end + +end diff --git a/WINDOWS_PORT.md b/WINDOWS_PORT.md new file mode 100644 index 00000000000..a0fe1fe11f3 --- /dev/null +++ b/WINDOWS_PORT.md @@ -0,0 +1,228 @@ +# Microsoft Contribution Notes + +## Contributors +* Alexander Zinoviev https://github.com/zinoale +* Dmitri Smirnov https://github.com/yuslepukhin +* Praveen Rao https://github.com/PraveenSinghRao +* Sherlock Huang https://github.com/SherlockNoMad + +## Introduction +RocksDB is a well proven open source key-value persistent store, optimized for fast storage. It provides scalability with number of CPUs and storage IOPS, to support IO-bound, in-memory and write-once workloads, most importantly, to be flexible to allow for innovation. + +As Microsoft Bing team we have been continuously pushing hard to improve the scalability, efficiency of platform and eventually benefit Bing end-user satisfaction. We would like to explore the opportunity to embrace open source, RocksDB here, to use, enhance and customize for our usage, and also contribute back to the RocksDB community. Herein, we are pleased to offer this RocksDB port for Windows platform. + +These notes describe some decisions and changes we had to make with regards to porting RocksDB on Windows. We hope this will help both reviewers and users of the Windows port. +We are open for comments and improvements. + +## OS specifics +All of the porting, testing and benchmarking was done on Windows Server 2012 R2 Datacenter 64-bit but to the best of our knowledge there is not a specific API we used during porting that is unsupported on other Windows OS after Vista. + +## Porting goals +We strive to achieve the following goals: +* make use of the existing porting interface of RocksDB +* make minimum [WY2]modifications within platform independent code. +* make all unit test pass both in debug and release builds. + * Note: latest introduction of SyncPoint seems to disable running db_test in Release. +* make performance on par with published benchmarks accounting for HW differences +* we would like to keep the port code inline with the master branch with no forking + +## Build system +We have chosen CMake as a widely accepted build system to build the Windows port. It is very fast and convenient. + +At the same time it generates Visual Studio projects that are both usable from a command line and IDE. + +The top-level CMakeLists.txt file contains description of all targets and build rules. It also provides brief instructions on how to build the software for Windows. One more build related file is thirdparty.inc that also resides on the top level. This file must be edited to point to actual third party libraries location. +We think that it would be beneficial to merge the existing make-based build system and the new cmake-based build system into a single one to use on all platforms. + +All building and testing was done for 64-bit. We have not conducted any testing for 32-bit and early reports indicate that it will not run on 32-bit. + +## C++ and STL notes +We had to make some minimum changes within the portable files that either account for OS differences or the shortcomings of C++11 support in the current version of the MS compiler. Most or all of them are expected to be fixed in the upcoming compiler releases. + +We plan to use this port for our business purposes here at Bing and this provided business justification for this port. This also means, we do not have at present to choose the compiler version at will. + +* Certain headers that are not present and not necessary on Windows were simply `#ifndef OS_WIN` in a few places (`unistd.h`) +* All posix specific headers were replaced to port/port.h which worked well +* Replaced `dirent.h` for `port/dirent.h` (very few places) with the implementation of the relevant interfaces within `rocksdb::port` namespace +* Replaced `sys/time.h` to `port/sys_time.h` (few places) implemented equivalents within `rocksdb::port` +* `printf %z` specification is not supported on Windows. To imitate existing standards we came up with a string macro `ROCKSDB_PRIszt` which expands to `%z` on posix systems and to Iu on windows. +* in class member initialization were moved to a __ctors in some cases +* `constexpr` is not supported. We had to replace `std::numeric_limits<>::max/min()` to its C macros for constants. Sometimes we had to make class members `static const` and place a definition within a .cc file. +* `constexpr` for functions was replaced to a template specialization (1 place) +* Union members that have non-trivial constructors were replaced to `char[]` in one place along with bug fixes (spatial experimental feature) +* Zero-sized arrays are deemed a non-standard extension which we converted to 1 size array and that should work well for the purposes of these classes. +* `std::chrono` lacks nanoseconds support (fixed in the upcoming release of the STL) and we had to use `QueryPerfCounter()` within env_win.cc +* Function local statics initialization is still not safe. Used `std::once` to mitigate within WinEnv. + +## Windows Environments notes +We endeavored to make it functionally on par with posix_env. This means we replicated the functionality of the thread pool and other things as precise as possible, including: +* Replicate posix logic using std:thread primitives. +* Implement all posix_env disk access functionality. +* Set `use_os_buffer=false` to disable OS disk buffering for WinWritableFile and WinRandomAccessFile. +* Replace `pread/pwrite` with `WriteFile/ReadFile` with `OVERLAPPED` structure. +* Use `SetFileInformationByHandle` to compensate absence of `fallocate`. + +### In detail +Even though Windows provides its own efficient thread-pool implementation we chose to replicate posix logic using `std::thread` primitives. This allows anyone to quickly detect any changes within the posix source code and replicate them within windows env. This has proven to work very well. At the same time for anyone who wishes to replace the built-in thread-pool can do so using RocksDB stackable environments. + +For disk access we implemented all of the functionality present within the posix_env which includes memory mapped files, random access, rate-limiter support etc. +The `use_os_buffer` flag on Posix platforms currently denotes disabling read-ahead log via `fadvise` mechanism. Windows does not have `fadvise` system call. What is more, it implements disk cache in a way that differs from Linux greatly. Its not an uncommon practice on Windows to perform un-buffered disk access to gain control of the memory consumption. We think that in our use case this may also be a good configuration option at the expense of disk throughput. To compensate one may increase the configured in-memory cache size instead. Thus we have chosen `use_os_buffer=false` to disable OS disk buffering for `WinWritableFile` and `WinRandomAccessFile`. The OS imposes restrictions on the alignment of the disk offsets, buffers used and the amount of data that is read/written when accessing files in un-buffered mode. When the option is true, the classes behave in a standard way. This allows to perform writes and reads in cases when un-buffered access does not make sense such as WAL and MANIFEST. + +We have replaced `pread/pwrite` with `WriteFile/ReadFile` with `OVERLAPPED` structure so we can atomically seek to the position of the disk operation but still perform the operation synchronously. Thus we able to emulate that functionality of `pread/pwrite` reasonably well. The only difference is that the file pointer is not returned to its original position but that hardly matters given the random nature of access. + +We used `SetFileInformationByHandle` both to truncate files after writing a full final page to disk and to pre-allocate disk space for faster I/O thus compensating for the absence of `fallocate` although some differences remain. For example, the pre-allocated space is not filled with zeros like on Linux, however, on a positive note, the end of file position is also not modified after pre-allocation. + +RocksDB renames, copies and deletes files at will even though they may be opened with another handle at the same time. We had to relax and allow nearly all the concurrent access permissions possible. + +## Thread-Local Storage +Thread-Local storage plays a significant role for RocksDB performance. Rather than creating a separate implementation we chose to create inline wrappers that forward `pthread_specific` calls to Windows `Tls` interfaces within `rocksdb::port` namespace. This leaves the existing meat of the logic in tact and unchanged and just as maintainable. + +To mitigate the lack of thread local storage cleanup on thread-exit we added a limited amount of windows specific code within the same thread_local.cc file that injects a cleanup callback into a `"__tls"` structure within `".CRT$XLB"` data segment. This approach guarantees that the callback is invoked regardless of whether RocksDB used within an executable, standalone DLL or within another DLL. + +## Jemalloc usage + +When RocksDB is used with Jemalloc the latter needs to be initialized before any of the C++ globals or statics. To accomplish that we injected an initialization routine into `".CRT$XCT"` that is automatically invoked by the runtime before initializing static objects. je-uninit is queued to `atexit()`. + +The jemalloc redirecting `new/delete` global operators are used by the linker providing certain conditions are met. See build section in these notes. + +## Stack Trace and Unhandled Exception Handler + +We decided not to implement these two features because the hosting program as a rule has these two things in it. +We experienced no inconveniences debugging issues in the debugger or analyzing process dumps if need be and thus we did not +see this as a priority. + +## Performance results +### Setup +All of the benchmarks are run on the same set of machines. Here are the details of the test setup: +* 2 Intel(R) Xeon(R) E5 2450 0 @ 2.10 GHz (total 16 cores) +* 2 XK0480GDQPH SSD Device, total 894GB free disk +* Machine has 128 GB of RAM +* Operating System: Windows Server 2012 R2 Datacenter +* 100 Million keys; each key is of size 10 bytes, each value is of size 800 bytes +* total database size is ~76GB +* The performance result is based on RocksDB 3.11. +* The parameters used, unless specified, were exactly the same as published in the GitHub Wiki page. + +### RocksDB on flash storage + +#### Test 1. Bulk Load of keys in Random Order + +Version 3.11 + +* Total Run Time: 17.6 min +* Fillrandom: 5.480 micros/op 182465 ops/sec; 142.0 MB/s +* Compact: 486056544.000 micros/op 0 ops/sec + +Version 3.10 + +* Total Run Time: 16.2 min +* Fillrandom: 5.018 micros/op 199269 ops/sec; 155.1 MB/s +* Compact: 441313173.000 micros/op 0 ops/sec; + + +#### Test 2. Bulk Load of keys in Sequential Order + +Version 3.11 + +* Fillseq: 4.944 micros/op 202k ops/sec; 157.4 MB/s + +Version 3.10 + +* Fillseq: 4.105 micros/op 243.6k ops/sec; 189.6 MB/s + + +#### Test 3. Random Write + +Version 3.11 + +* Unbuffered I/O enabled +* Overwrite: 52.661 micros/op 18.9k ops/sec; 14.8 MB/s + +Version 3.10 + +* Unbuffered I/O enabled +* Overwrite: 52.661 micros/op 18.9k ops/sec; + + +#### Test 4. Random Read + +Version 3.11 + +* Unbuffered I/O enabled +* Readrandom: 15.716 micros/op 63.6k ops/sec; 49.5 MB/s + +Version 3.10 + +* Unbuffered I/O enabled +* Readrandom: 15.548 micros/op 64.3k ops/sec; + + +#### Test 5. Multi-threaded read and single-threaded write + +Version 3.11 + +* Unbuffered I/O enabled +* Readwhilewriting: 25.128 micros/op 39.7k ops/sec; + +Version 3.10 + +* Unbuffered I/O enabled +* Readwhilewriting: 24.854 micros/op 40.2k ops/sec; + + +### RocksDB In Memory + +#### Test 1. Point Lookup + +Version 3.11 + +80K writes/sec +* Write Rate Achieved: 40.5k write/sec; +* Readwhilewriting: 0.314 micros/op 3187455 ops/sec; 364.8 MB/s (715454999 of 715454999 found) + +Version 3.10 + +* Write Rate Achieved: 50.6k write/sec +* Readwhilewriting: 0.316 micros/op 3162028 ops/sec; (719576999 of 719576999 found) + + +*10K writes/sec* + +Version 3.11 + +* Write Rate Achieved: 5.8k/s write/sec +* Readwhilewriting: 0.246 micros/op 4062669 ops/sec; 464.9 MB/s (915481999 of 915481999 found) + +Version 3.10 + +* Write Rate Achieved: 5.8k/s write/sec +* Readwhilewriting: 0.244 micros/op 4106253 ops/sec; (927986999 of 927986999 found) + + +#### Test 2. Prefix Range Query + +Version 3.11 + +80K writes/sec +* Write Rate Achieved: 46.3k/s write/sec +* Readwhilewriting: 0.362 micros/op 2765052 ops/sec; 316.4 MB/s (611549999 of 611549999 found) + +Version 3.10 + +* Write Rate Achieved: 45.8k/s write/sec +* Readwhilewriting: 0.317 micros/op 3154941 ops/sec; (708158999 of 708158999 found) + +Version 3.11 + +10K writes/sec +* Write Rate Achieved: 5.78k write/sec +* Readwhilewriting: 0.269 micros/op 3716692 ops/sec; 425.3 MB/s (837401999 of 837401999 found) + +Version 3.10 + +* Write Rate Achieved: 5.7k write/sec +* Readwhilewriting: 0.261 micros/op 3830152 ops/sec; (863482999 of 863482999 found) + + +We think that there is still big room to improve the performance, which will be an ongoing effort for us. + diff --git a/appveyor.yml b/appveyor.yml new file mode 100644 index 00000000000..be9b66b45c9 --- /dev/null +++ b/appveyor.yml @@ -0,0 +1,15 @@ +version: 1.0.{build} +image: Visual Studio 2015 +before_build: +- md %APPVEYOR_BUILD_FOLDER%\build +- cd %APPVEYOR_BUILD_FOLDER%\build +- cmake -G "Visual Studio 14 2015 Win64" -DOPTDBG=1 -DXPRESS=1 -DPORTABLE=1 .. +- cd .. +build: + project: build\rocksdb.sln + parallel: true + verbosity: minimal +test: +test_script: +- ps: build_tools\run_ci_db_test.ps1 -SuiteRun db_basic_test,db_test2,db_test,env_basic_test,env_test -Concurrency 8 + diff --git a/buckifier/buckify_rocksdb.py b/buckifier/buckify_rocksdb.py new file mode 100644 index 00000000000..a3c8be3b17c --- /dev/null +++ b/buckifier/buckify_rocksdb.py @@ -0,0 +1,172 @@ +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function +from __future__ import unicode_literals +from targets_builder import TARGETSBuilder +from optparse import OptionParser +import os +import fnmatch +import sys +import tempfile + +from util import ColorString +import util + +# tests to export as libraries for inclusion in other projects +_EXPORTED_TEST_LIBS = ["env_basic_test"] + +# Parse src.mk files as a Dictionary of +# VAR_NAME => list of files +def parse_src_mk(repo_path): + src_mk = repo_path + "/src.mk" + src_files = {} + for line in open(src_mk): + line = line.strip() + if len(line) == 0 or line[0] == '#': + continue + if '=' in line: + current_src = line.split('=')[0].strip() + src_files[current_src] = [] + elif '.cc' in line: + src_path = line.split('.cc')[0].strip() + '.cc' + src_files[current_src].append(src_path) + return src_files + + +# get all .cc / .c files +def get_cc_files(repo_path): + cc_files = [] + for root, dirnames, filenames in os.walk(repo_path): + root = root[(len(repo_path) + 1):] + if "java" in root: + # Skip java + continue + for filename in fnmatch.filter(filenames, '*.cc'): + cc_files.append(os.path.join(root, filename)) + for filename in fnmatch.filter(filenames, '*.c'): + cc_files.append(os.path.join(root, filename)) + return cc_files + + +# Get tests from Makefile +def get_tests(repo_path): + Makefile = repo_path + "/Makefile" + + # Dictionary TEST_NAME => IS_PARALLEL + tests = {} + + found_tests = False + for line in open(Makefile): + line = line.strip() + if line.startswith("TESTS ="): + found_tests = True + elif found_tests: + if line.endswith("\\"): + # remove the trailing \ + line = line[:-1] + line = line.strip() + tests[line] = False + else: + # we consumed all the tests + break + + found_parallel_tests = False + for line in open(Makefile): + line = line.strip() + if line.startswith("PARALLEL_TEST ="): + found_parallel_tests = True + elif found_parallel_tests: + if line.endswith("\\"): + # remove the trailing \ + line = line[:-1] + line = line.strip() + tests[line] = True + else: + # we consumed all the parallel tests + break + + return tests + + +# Prepare TARGETS file for buck +def generate_targets(repo_path): + print(ColorString.info("Generating TARGETS")) + # parsed src.mk file + src_mk = parse_src_mk(repo_path) + # get all .cc files + cc_files = get_cc_files(repo_path) + # get tests from Makefile + tests = get_tests(repo_path) + + if src_mk is None or cc_files is None or tests is None: + return False + + TARGETS = TARGETSBuilder("%s/TARGETS" % repo_path) + # rocksdb_lib + TARGETS.add_library( + "rocksdb_lib", + src_mk["LIB_SOURCES"] + + src_mk["TOOL_LIB_SOURCES"]) + # rocksdb_test_lib + TARGETS.add_library( + "rocksdb_test_lib", + src_mk.get("MOCK_LIB_SOURCES", []) + + src_mk.get("TEST_LIB_SOURCES", []) + + src_mk.get("EXP_LIB_SOURCES", []), + [":rocksdb_lib"]) + # rocksdb_tools_lib + TARGETS.add_library( + "rocksdb_tools_lib", + src_mk.get("BENCH_LIB_SOURCES", []) + + ["util/testutil.cc"], + [":rocksdb_lib"]) + + # test for every test we found in the Makefile + for test in sorted(tests): + match_src = [src for src in cc_files if ("/%s.c" % test) in src] + if len(match_src) == 0: + print(ColorString.warning("Cannot find .cc file for %s" % test)) + continue + elif len(match_src) > 1: + print(ColorString.warning("Found more than one .cc for %s" % test)) + print(match_src) + continue + + assert(len(match_src) == 1) + is_parallel = tests[test] + TARGETS.register_test(test, match_src[0], is_parallel) + + if test in _EXPORTED_TEST_LIBS: + test_library = "%s_lib" % test + TARGETS.add_library(test_library, match_src, [":rocksdb_test_lib"]) + TARGETS.flush_tests() + + print(ColorString.info("Generated TARGETS Summary:")) + print(ColorString.info("- %d libs" % TARGETS.total_lib)) + print(ColorString.info("- %d binarys" % TARGETS.total_bin)) + print(ColorString.info("- %d tests" % TARGETS.total_test)) + return True + + +def get_rocksdb_path(): + # rocksdb = {script_dir}/.. + script_dir = os.path.dirname(sys.argv[0]) + script_dir = os.path.abspath(script_dir) + rocksdb_path = os.path.abspath( + os.path.join(script_dir, "../")) + + return rocksdb_path + +def exit_with_error(msg): + print(ColorString.error(msg)) + sys.exit(1) + + +def main(): + # Generate TARGETS file for buck + ok = generate_targets(get_rocksdb_path()) + if not ok: + exit_with_error("Failed to generate TARGETS files") + +if __name__ == "__main__": + main() diff --git a/buckifier/rocks_test_runner.sh b/buckifier/rocks_test_runner.sh new file mode 100755 index 00000000000..e1f48a760d3 --- /dev/null +++ b/buckifier/rocks_test_runner.sh @@ -0,0 +1,4 @@ +#!/usr/bin/env bash +# Create a tmp directory for the test to use +TEST_DIR=$(mktemp -d /dev/shm/fbcode_rocksdb_XXXXXXX) +TEST_TMPDIR="$TEST_DIR" $@ && rm -rf "$TEST_DIR" diff --git a/buckifier/targets_builder.py b/buckifier/targets_builder.py new file mode 100644 index 00000000000..7d47d2d1f92 --- /dev/null +++ b/buckifier/targets_builder.py @@ -0,0 +1,65 @@ +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function +from __future__ import unicode_literals +import targets_cfg +import pprint + +# TODO(tec): replace this with PrettyPrinter +def pretty_list(lst, indent=6): + if lst is None or len(lst) == 0: + return "" + + if len(lst) == 1: + return "\"%s\"" % lst[0] + + separator = "\",\n%s\"" % (" " * indent) + res = separator.join(lst) + res = "\n" + (" " * indent) + "\"" + res + "\",\n" + (" " * (indent - 2)) + return res + + +class TARGETSBuilder: + def __init__(self, path): + self.path = path + self.targets_file = open(path, 'w') + self.targets_file.write(targets_cfg.rocksdb_target_header) + self.total_lib = 0 + self.total_bin = 0 + self.total_test = 0 + self.tests_cfg = [] + + def __del__(self): + self.targets_file.close() + + def add_library(self, name, srcs, deps=None, headers=None): + if headers is None: + headers = "AutoHeaders.RECURSIVE_GLOB" + self.targets_file.write(targets_cfg.library_template % ( + name, + headers, + pretty_list(srcs), + pretty_list(deps))) + self.total_lib = self.total_lib + 1 + + def add_binary(self, name, srcs, deps=None): + self.targets_file.write(targets_cfg.binary_template % ( + name, + pretty_list(srcs), + pretty_list(deps))) + self.total_bin = self.total_bin + 1 + + def register_test(self, test_name, src, is_parallel): + exec_mode = "serial" + if is_parallel: + exec_mode = "parallel" + self.tests_cfg.append([test_name, str(src), str(exec_mode)]) + + self.total_test = self.total_test + 1 + + def flush_tests(self): + self.targets_file.write(targets_cfg.unittests_template % ( + pprint.PrettyPrinter().pformat(self.tests_cfg) + )) + + self.tests_cfg = [] diff --git a/buckifier/targets_cfg.py b/buckifier/targets_cfg.py new file mode 100644 index 00000000000..33023a589f4 --- /dev/null +++ b/buckifier/targets_cfg.py @@ -0,0 +1,124 @@ +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function +from __future__ import unicode_literals +rocksdb_target_header = """ +import os + +TARGETS_PATH = os.path.dirname(__file__) +REPO_PATH = "rocksdb/src/" +BUCK_BINS = "buck-out/gen/" + REPO_PATH +TEST_RUNNER = REPO_PATH + "buckifier/rocks_test_runner.sh" +rocksdb_compiler_flags = [ + "-fno-builtin-memcmp", + "-DROCKSDB_PLATFORM_POSIX", + "-DROCKSDB_LIB_IO_POSIX", + "-DROCKSDB_FALLOCATE_PRESENT", + "-DROCKSDB_MALLOC_USABLE_SIZE", + "-DROCKSDB_RANGESYNC_PRESENT", + "-DROCKSDB_SCHED_GETCPU_PRESENT", + "-DROCKSDB_SUPPORT_THREAD_LOCAL", + "-DOS_LINUX", + # Flags to enable libs we include + "-DSNAPPY", + "-DZLIB", + "-DBZIP2", + "-DLZ4", + "-DZSTD", + "-DGFLAGS=gflags", + "-DNUMA", + "-DTBB", + # Needed to compile in fbcode + "-Wno-expansion-to-defined", +] + +rocksdb_external_deps = [ + ('bzip2', None, 'bz2'), + ('snappy', None, "snappy"), + ('zlib', None, 'z'), + ('gflags', None, 'gflags'), + ('lz4', None, 'lz4'), + ('zstd', None), + ('tbb', None), + ("numa", None, "numa"), + ("googletest", None, "gtest"), +] + +rocksdb_preprocessor_flags = [ + # Directories with files for #include + "-I" + REPO_PATH + "include/", + "-I" + REPO_PATH, +] + +rocksdb_arch_preprocessor_flags = { + "x86_64": ["-DHAVE_SSE42"], +} +""" + + +library_template = """ +cpp_library( + name = "%s", + headers = %s, + srcs = [%s], + deps = [%s], + preprocessor_flags = rocksdb_preprocessor_flags, + arch_preprocessor_flags = rocksdb_arch_preprocessor_flags, + compiler_flags = rocksdb_compiler_flags, + external_deps = rocksdb_external_deps, +) +""" + +binary_template = """ +cpp_binary( + name = "%s", + srcs = [%s], + deps = [%s], + preprocessor_flags = rocksdb_preprocessor_flags, + arch_preprocessor_flags = rocksdb_arch_preprocessor_flags, + compiler_flags = rocksdb_compiler_flags, + external_deps = rocksdb_external_deps, +) +""" + +unittests_template = """ +# [test_name, test_src, test_type] +ROCKS_TESTS = %s + + +# Generate a test rule for each entry in ROCKS_TESTS +for test_cfg in ROCKS_TESTS: + test_name = test_cfg[0] + test_cc = test_cfg[1] + ttype = "gtest" if test_cfg[2] == "parallel" else "simple" + test_bin = test_name + "_bin" + + cpp_binary ( + name = test_bin, + srcs = [test_cc], + deps = [":rocksdb_test_lib"], + preprocessor_flags = rocksdb_preprocessor_flags, + arch_preprocessor_flags = rocksdb_arch_preprocessor_flags, + compiler_flags = rocksdb_compiler_flags, + external_deps = rocksdb_external_deps, + ) + + custom_unittest( + name = test_name, + type = ttype, + deps = [":" + test_bin], + command = [TEST_RUNNER, BUCK_BINS + test_bin] + ) + +custom_unittest( + name = "make_rocksdbjavastatic", + type = "simple", + command = ["internal_repo_rocksdb/make_rocksdbjavastatic.sh"], +) + +custom_unittest( + name = "make_rocksdb_lite_release", + type = "simple", + command = ["internal_repo_rocksdb/make_rocksdb_lite_release.sh"], +) +""" diff --git a/buckifier/util.py b/buckifier/util.py new file mode 100644 index 00000000000..350b7335c33 --- /dev/null +++ b/buckifier/util.py @@ -0,0 +1,107 @@ +""" +This module keeps commonly used components. +""" +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function +from __future__ import unicode_literals +import subprocess +import os +import time + +class ColorString: + """ Generate colorful strings on terminal """ + HEADER = '\033[95m' + BLUE = '\033[94m' + GREEN = '\033[92m' + WARNING = '\033[93m' + FAIL = '\033[91m' + ENDC = '\033[0m' + + @staticmethod + def _make_color_str(text, color): + return "".join([color, text.encode('utf-8'), ColorString.ENDC]) + + @staticmethod + def ok(text): + if ColorString.is_disabled: + return text + return ColorString._make_color_str(text, ColorString.GREEN) + + @staticmethod + def info(text): + if ColorString.is_disabled: + return text + return ColorString._make_color_str(text, ColorString.BLUE) + + @staticmethod + def header(text): + if ColorString.is_disabled: + return text + return ColorString._make_color_str(text, ColorString.HEADER) + + @staticmethod + def error(text): + if ColorString.is_disabled: + return text + return ColorString._make_color_str(text, ColorString.FAIL) + + @staticmethod + def warning(text): + if ColorString.is_disabled: + return text + return ColorString._make_color_str(text, ColorString.WARNING) + + is_disabled = False + + +def run_shell_command(shell_cmd, cmd_dir=None): + """ Run a single shell command. + @returns a tuple of shell command return code, stdout, stderr """ + + if cmd_dir is not None and not os.path.exists(cmd_dir): + run_shell_command("mkdir -p %s" % cmd_dir) + + start = time.time() + print("\t>>> Running: " + shell_cmd) + p = subprocess.Popen(shell_cmd, + shell=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + cwd=cmd_dir) + stdout, stderr = p.communicate() + end = time.time() + + # Report time if we spent more than 5 minutes executing a command + execution_time = end - start + if execution_time > (60 * 5): + mins = (execution_time / 60) + secs = (execution_time % 60) + print("\t>time spent: %d minutes %d seconds" % (mins, secs)) + + + return p.returncode, stdout, stderr + + +def run_shell_commands(shell_cmds, cmd_dir=None, verbose=False): + """ Execute a sequence of shell commands, which is equivalent to + running `cmd1 && cmd2 && cmd3` + @returns boolean indication if all commands succeeds. + """ + + if cmd_dir: + print("\t=== Set current working directory => %s" % cmd_dir) + + for shell_cmd in shell_cmds: + ret_code, stdout, stderr = run_shell_command(shell_cmd, cmd_dir) + if stdout: + if verbose or ret_code != 0: + print(ColorString.info("stdout: \n"), stdout) + if stderr: + # contents in stderr is not necessarily to be error messages. + if verbose or ret_code != 0: + print(ColorString.error("stderr: \n"), stderr) + if ret_code != 0: + return False + + return True diff --git a/build_tools/RocksDBCommonHelper.php b/build_tools/RocksDBCommonHelper.php new file mode 100644 index 00000000000..9fe770fe956 --- /dev/null +++ b/build_tools/RocksDBCommonHelper.php @@ -0,0 +1,377 @@ + 0); + assert(is_numeric($diffID)); + assert(strlen($url) > 0); + + $cmd_args = array( + 'diff_id' => (int)$diffID, + 'name' => sprintf( + 'click here for sandcastle tests for D%d', + (int)$diffID + ), + 'link' => $url + ); + $cmd = 'echo ' . escapeshellarg(json_encode($cmd_args)) + . ' | arc call-conduit differential.updateunitresults'; + + shell_exec($cmd); +} + +function buildUpdateTestStatusCmd($diffID, $test, $status) { + assert(strlen($diffID) > 0); + assert(is_numeric($diffID)); + assert(strlen($test) > 0); + assert(strlen($status) > 0); + + $cmd_args = array( + 'diff_id' => (int)$diffID, + 'name' => $test, + 'result' => $status + ); + + $cmd = 'echo ' . escapeshellarg(json_encode($cmd_args)) + . ' | arc call-conduit differential.updateunitresults'; + + return $cmd; +} + +function updateTestStatus($diffID, $test) { + assert(strlen($diffID) > 0); + assert(is_numeric($diffID)); + assert(strlen($test) > 0); + + shell_exec(buildUpdateTestStatusCmd($diffID, $test, "waiting")); +} + +function getSteps($applyDiff, $diffID, $username, $test) { + assert(strlen($username) > 0); + assert(strlen($test) > 0); + + if ($applyDiff) { + assert(strlen($diffID) > 0); + assert(is_numeric($diffID)); + + $arcrc_content = (PHP_OS == "Darwin" ? + exec("cat ~/.arcrc | gzip -f | base64") : + exec("cat ~/.arcrc | gzip -f | base64 -w0")); + assert(strlen($arcrc_content) > 0); + + // Sandcastle machines don't have arc setup. We copy the user certificate + // and authenticate using that in Sandcastle. + $setup = array( + "name" => "Setup arcrc", + "shell" => "echo " . escapeshellarg($arcrc_content) . " | base64 --decode" + . " | gzip -d > ~/.arcrc", + "user" => "root" + ); + + // arc demands certain permission on its config. + // also fix the sticky bit issue in sandcastle + $fix_permission = array( + "name" => "Fix environment", + "shell" => "chmod 600 ~/.arcrc && chmod +t /dev/shm", + "user" => "root" + ); + + // Construct the steps in the order of execution. + $steps[] = $setup; + $steps[] = $fix_permission; + } + + // fbcode is a sub-repo. We cannot patch until we add it to ignore otherwise + // Git thinks it is an uncommited change. + $fix_git_ignore = array( + "name" => "Fix git ignore", + "shell" => "echo fbcode >> .git/info/exclude", + "user" => "root" + ); + + // This fixes "FATAL: ThreadSanitizer can not mmap the shadow memory" + // Source: + // https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual#FAQ + $fix_kernel_issue = array( + "name" => "Fix kernel issue with tsan", + "shell" => "echo 2 >/proc/sys/kernel/randomize_va_space", + "user" => "root" + ); + + $steps[] = $fix_git_ignore; + $steps[] = $fix_kernel_issue; + + // This will be the command used to execute particular type of tests. + $cmd = ""; + + if ($applyDiff) { + // Patch the code (keep your fingures crossed). + $patch = array( + "name" => "Patch " . $diffID, + "shell" => "arc --arcrc-file ~/.arcrc " + . "patch --nocommit --diff " . escapeshellarg($diffID), + "user" => "root" + ); + + $steps[] = $patch; + + updateTestStatus($diffID, $test); + $cmd = buildUpdateTestStatusCmd($diffID, $test, "running") . "; "; + } + + // Run the actual command. + $cmd = $cmd . "J=$(nproc) ./build_tools/precommit_checker.py " . + escapeshellarg($test) . "; exit_code=$?; "; + + if ($applyDiff) { + $cmd = $cmd . "([[ \$exit_code -eq 0 ]] &&" + . buildUpdateTestStatusCmd($diffID, $test, "pass") . ")" + . "||" . buildUpdateTestStatusCmd($diffID, $test, "fail") + . "; "; + } + + // shell command to sort the tests based on exit code and print + // the output of the log files. + $cat_sorted_logs = " + while read code log_file; + do echo \"################ cat \$log_file [exit_code : \$code] ################\"; + cat \$log_file; + done < <(tail -n +2 LOG | sort -k7,7n -k4,4gr | awk '{print \$7,\$NF}')"; + + // Shell command to cat all log files + $cat_all_logs = "for f in `ls t/!(run-*)`; do echo \$f;cat \$f; done"; + + // If LOG file exist use it to cat log files sorted by exit code, otherwise + // cat everything + $logs_cmd = "if [ -f LOG ]; then {$cat_sorted_logs}; else {$cat_all_logs}; fi"; + + $cmd = $cmd . " cat /tmp/precommit-check.log" + . "; shopt -s extglob; {$logs_cmd}" + . "; shopt -u extglob; [[ \$exit_code -eq 0 ]]"; + assert(strlen($cmd) > 0); + + $run_test = array( + "name" => "Run " . $test, + "shell" => $cmd, + "user" => "root", + "parser" => "python build_tools/error_filter.py " . escapeshellarg($test), + ); + + $steps[] = $run_test; + + if ($applyDiff) { + // Clean up the user arc config we are using. + $cleanup = array( + "name" => "Arc cleanup", + "shell" => "rm -f ~/.arcrc", + "user" => "root" + ); + + $steps[] = $cleanup; + } + + assert(count($steps) > 0); + return $steps; +} + +function getSandcastleConfig() { + $sandcastle_config = array(); + + $cwd = getcwd(); + $cwd_token_file = "{$cwd}/.sandcastle"; + // This is a case when we're executed from a continuous run. Fetch the values + // from the environment. + if (getenv(ENV_POST_RECEIVE_HOOK)) { + $sandcastle_config[0] = getenv(ENV_HTTPS_APP_VALUE); + $sandcastle_config[1] = getenv(ENV_HTTPS_TOKEN_VALUE); + } else { + // This is a typical `[p]arc diff` case. Fetch the values from the specific + // configuration files. + for ($i = 0; $i < 50; $i++) { + if (file_exists(PRIMARY_TOKEN_FILE) || + file_exists($cwd_token_file)) { + break; + } + // If we failed to fetch the tokens, sleep for 0.2 second and try again + usleep(200000); + } + assert(file_exists(PRIMARY_TOKEN_FILE) || + file_exists($cwd_token_file)); + + // Try the primary location first, followed by a secondary. + if (file_exists(PRIMARY_TOKEN_FILE)) { + $cmd = 'cat ' . PRIMARY_TOKEN_FILE; + } else { + $cmd = 'cat ' . escapeshellarg($cwd_token_file); + } + + assert(strlen($cmd) > 0); + $sandcastle_config = explode(':', rtrim(shell_exec($cmd))); + } + + // In this case be very explicit about the implications. + if (count($sandcastle_config) != 2) { + echo "Sandcastle configuration files don't contain valid information " . + "or the necessary environment variables aren't defined. Unable " . + "to validate the code changes."; + exit(1); + } + + assert(strlen($sandcastle_config[0]) > 0); + assert(strlen($sandcastle_config[1]) > 0); + assert(count($sandcastle_config) > 0); + + return $sandcastle_config; +} + +// This function can be called either from `[p]arc diff` command or during +// the Git post-receive hook. + function startTestsInSandcastle($applyDiff, $workflow, $diffID) { + // Default options don't terminate on failure, but that's what we want. In + // the current case we use assertions intentionally as "terminate on failure + // invariants". + assert_options(ASSERT_BAIL, true); + + // In case of a diff we'll send notificatios to the author. Else it'll go to + // the entire team because failures indicate that build quality has regressed. + $username = $applyDiff ? exec("whoami") : CONT_RUN_ALIAS; + assert(strlen($username) > 0); + + if ($applyDiff) { + assert($workflow); + assert(strlen($diffID) > 0); + assert(is_numeric($diffID)); + } + + // List of tests we want to run in Sandcastle. + $tests = array("unit", "unit_non_shm", "unit_481", "clang_unit", "tsan", + "asan", "lite_test", "valgrind", "release", "release_481", + "clang_release", "clang_analyze", "code_cov", + "java_build", "no_compression", "unity", "ubsan"); + + $send_email_template = array( + 'type' => 'email', + 'triggers' => array('fail'), + 'emails' => array($username . '@fb.com'), + ); + + // Construct a job definition for each test and add it to the master plan. + foreach ($tests as $test) { + $stepName = "RocksDB diff " . $diffID . " test " . $test; + + if (!$applyDiff) { + $stepName = "RocksDB continuous integration test " . $test; + } + + $arg[] = array( + "name" => $stepName, + "report" => array($send_email_template), + "steps" => getSteps($applyDiff, $diffID, $username, $test) + ); + } + + // We cannot submit the parallel execution master plan to Sandcastle and + // need supply the job plan as a determinator. So we construct a small job + // that will spit out the master job plan which Sandcastle will parse and + // execute. Why compress the job definitions? Otherwise we run over the max + // string size. + $cmd = "echo " . base64_encode(json_encode($arg)) + . (PHP_OS == "Darwin" ? + " | gzip -f | base64" : + " | gzip -f | base64 -w0"); + assert(strlen($cmd) > 0); + + $arg_encoded = shell_exec($cmd); + assert(strlen($arg_encoded) > 0); + + $runName = "Run diff " . $diffID . "for user " . $username; + + if (!$applyDiff) { + $runName = "RocksDB continuous integration build and test run"; + } + + $command = array( + "name" => $runName, + "steps" => array() + ); + + $command["steps"][] = array( + "name" => "Generate determinator", + "shell" => "echo " . $arg_encoded . " | base64 --decode | gzip -d" + . " | base64 --decode", + "determinator" => true, + "user" => "root" + ); + + // Submit to Sandcastle. + $url = 'https://interngraph.intern.facebook.com/sandcastle/create'; + + $job = array( + 'command' => 'SandcastleUniversalCommand', + 'args' => $command, + 'capabilities' => array( + 'vcs' => 'rocksdb-int-git', + 'type' => 'lego', + ), + 'hash' => 'origin/master', + 'user' => $username, + 'alias' => 'rocksdb-precommit', + 'tags' => array('rocksdb'), + 'description' => 'Rocksdb precommit job', + ); + + // Fetch the configuration necessary to submit a successful HTTPS request. + $sandcastle_config = getSandcastleConfig(); + + $app = $sandcastle_config[0]; + $token = $sandcastle_config[1]; + + $cmd = 'curl -s -k ' + . ' -F app=' . escapeshellarg($app) + . ' -F token=' . escapeshellarg($token) + . ' -F job=' . escapeshellarg(json_encode($job)) + .' ' . escapeshellarg($url); + + $output = shell_exec($cmd); + assert(strlen($output) > 0); + + // Extract Sandcastle URL from the response. + preg_match('/url": "(.+)"/', $output, $sandcastle_url); + + assert(count($sandcastle_url) > 0, "Unable to submit Sandcastle request."); + assert(strlen($sandcastle_url[1]) > 0, "Unable to extract Sandcastle URL."); + + if ($applyDiff) { + echo "\nSandcastle URL: " . $sandcastle_url[1] . "\n"; + // Ask Phabricator to display it on the diff UI. + postURL($diffID, $sandcastle_url[1]); + } else { + echo "Continuous integration started Sandcastle tests. You can look at "; + echo "the progress at:\n" . $sandcastle_url[1] . "\n"; + } +} + +// Continuous run cript will set the environment variable and based on that +// we'll trigger the execution of tests in Sandcastle. In that case we don't +// need to apply any diffs and there's no associated workflow either. +if (getenv(ENV_POST_RECEIVE_HOOK)) { + startTestsInSandcastle( + false /* $applyDiff */, + NULL /* $workflow */, + NULL /* $diffID */); +} diff --git a/build_tools/amalgamate.py b/build_tools/amalgamate.py new file mode 100755 index 00000000000..548b1e8cec0 --- /dev/null +++ b/build_tools/amalgamate.py @@ -0,0 +1,110 @@ +#!/usr/bin/python + +# amalgamate.py creates an amalgamation from a unity build. +# It can be run with either Python 2 or 3. +# An amalgamation consists of a header that includes the contents of all public +# headers and a source file that includes the contents of all source files and +# private headers. +# +# This script works by starting with the unity build file and recursively expanding +# #include directives. If the #include is found in a public include directory, +# that header is expanded into the amalgamation header. +# +# A particular header is only expanded once, so this script will +# break if there are multiple inclusions of the same header that are expected to +# expand differently. Similarly, this type of code causes issues: +# +# #ifdef FOO +# #include "bar.h" +# // code here +# #else +# #include "bar.h" // oops, doesn't get expanded +# // different code here +# #endif +# +# The solution is to move the include out of the #ifdef. + +from __future__ import print_function + +import argparse +from os import path +import re +import sys + +include_re = re.compile('^[ \t]*#include[ \t]+"(.*)"[ \t]*$') +included = set() +excluded = set() + +def find_header(name, abs_path, include_paths): + samedir = path.join(path.dirname(abs_path), name) + if path.exists(samedir): + return samedir + for include_path in include_paths: + include_path = path.join(include_path, name) + if path.exists(include_path): + return include_path + return None + +def expand_include(include_path, f, abs_path, source_out, header_out, include_paths, public_include_paths): + if include_path in included: + return False + + included.add(include_path) + with open(include_path) as f: + print('#line 1 "{}"'.format(include_path), file=source_out) + process_file(f, include_path, source_out, header_out, include_paths, public_include_paths) + return True + +def process_file(f, abs_path, source_out, header_out, include_paths, public_include_paths): + for (line, text) in enumerate(f): + m = include_re.match(text) + if m: + filename = m.groups()[0] + # first check private headers + include_path = find_header(filename, abs_path, include_paths) + if include_path: + if include_path in excluded: + source_out.write(text) + expanded = False + else: + expanded = expand_include(include_path, f, abs_path, source_out, header_out, include_paths, public_include_paths) + else: + # now try public headers + include_path = find_header(filename, abs_path, public_include_paths) + if include_path: + # found public header + expanded = False + if include_path in excluded: + source_out.write(text) + else: + expand_include(include_path, f, abs_path, header_out, None, public_include_paths, []) + else: + sys.exit("unable to find {}, included in {} on line {}".format(filename, abs_path, line)) + + if expanded: + print('#line {} "{}"'.format(line+1, abs_path), file=source_out) + elif text != "#pragma once\n": + source_out.write(text) + +def main(): + parser = argparse.ArgumentParser(description="Transform a unity build into an amalgamation") + parser.add_argument("source", help="source file") + parser.add_argument("-I", action="append", dest="include_paths", help="include paths for private headers") + parser.add_argument("-i", action="append", dest="public_include_paths", help="include paths for public headers") + parser.add_argument("-x", action="append", dest="excluded", help="excluded header files") + parser.add_argument("-o", dest="source_out", help="output C++ file", required=True) + parser.add_argument("-H", dest="header_out", help="output C++ header file", required=True) + args = parser.parse_args() + + include_paths = list(map(path.abspath, args.include_paths or [])) + public_include_paths = list(map(path.abspath, args.public_include_paths or [])) + excluded.update(map(path.abspath, args.excluded or [])) + filename = args.source + abs_path = path.abspath(filename) + with open(filename) as f, open(args.source_out, 'w') as source_out, open(args.header_out, 'w') as header_out: + print('#line 1 "{}"'.format(filename), file=source_out) + print('#include "{}"'.format(header_out.name), file=source_out) + process_file(f, abs_path, source_out, header_out, include_paths, public_include_paths) + +if __name__ == "__main__": + main() diff --git a/build_tools/build_detect_platform b/build_tools/build_detect_platform index 3389d2851b0..c7ddb7cceec 100755 --- a/build_tools/build_detect_platform +++ b/build_tools/build_detect_platform @@ -8,6 +8,7 @@ # CXX C++ Compiler path # PLATFORM_LDFLAGS Linker flags # JAVA_LDFLAGS Linker flags for RocksDBJava +# JAVA_STATIC_LDFLAGS Linker flags for RocksDBJava static build # PLATFORM_SHARED_EXT Extension for shared libraries # PLATFORM_SHARED_LDFLAGS Flags for building shared library # PLATFORM_SHARED_CFLAGS Flags for compiling objects for shared library @@ -18,21 +19,21 @@ # # The PLATFORM_CCFLAGS and PLATFORM_CXXFLAGS might include the following: # -# -DLEVELDB_PLATFORM_POSIX if cstdatomic is present -# -DLEVELDB_PLATFORM_NOATOMIC if it is not +# -DROCKSDB_PLATFORM_POSIX if posix-platform based # -DSNAPPY if the Snappy library is present # -DLZ4 if the LZ4 library is present +# -DZSTD if the ZSTD library is present # -DNUMA if the NUMA library is present +# -DTBB if the TBB library is present # # Using gflags in rocksdb: # Our project depends on gflags, which requires users to take some extra steps # before they can compile the whole repository: # 1. Install gflags. You may download it from here: -# https://code.google.com/p/gflags/ -# 2. Once install, add the include path/lib path for gflags to CPATH and -# LIBRARY_PATH respectively. If installed with default mode, the -# lib and include path will be /usr/local/lib and /usr/local/include -# Mac user can do this by running build_tools/mac-install-gflags.sh +# https://gflags.github.io/gflags/ (Mac users can `brew install gflags`) +# 2. Once installed, add the include path for gflags to your CPATH env var and +# the lib path to LIBRARY_PATH. If installed with default settings, the lib +# will be /usr/local/lib and the include path will be /usr/local/include OUTPUT=$1 if test -z "$OUTPUT"; then @@ -43,21 +44,18 @@ fi # we depend on C++11 PLATFORM_CXXFLAGS="-std=c++11" # we currently depend on POSIX platform -COMMON_FLAGS="-DROCKSDB_PLATFORM_POSIX" +COMMON_FLAGS="-DROCKSDB_PLATFORM_POSIX -DROCKSDB_LIB_IO_POSIX" # Default to fbcode gcc on internal fb machines -if [ -d /mnt/gvfs/third-party -a -z "$CXX" ]; then +if [ -z "$ROCKSDB_NO_FBCODE" -a -d /mnt/gvfs/third-party ]; then FBCODE_BUILD="true" - if [ -z "$USE_CLANG" ]; then - CENTOS_VERSION=`rpm -q --qf "%{VERSION}" \ - $(rpm -q --whatprovides redhat-release)` - if [ "$CENTOS_VERSION" = "6" ]; then - source "$PWD/build_tools/fbcode.gcc481.sh" - else - source "$PWD/build_tools/fbcode.gcc471.sh" - fi + # If we're compiling with TSAN we need pic build + PIC_BUILD=$COMPILE_WITH_TSAN + if [ -z "$ROCKSDB_FBCODE_BUILD_WITH_481" ]; then + source "$PWD/build_tools/fbcode_config.sh" else - source "$PWD/build_tools/fbcode.clang31.sh" + # we need this to build with MySQL. Don't use for other purposes. + source "$PWD/build_tools/fbcode_config4.8.1.sh" fi fi @@ -78,15 +76,26 @@ if test -z "$TARGET_OS"; then TARGET_OS=`uname -s` fi +if test -z "$TARGET_ARCHITECTURE"; then + TARGET_ARCHITECTURE=`uname -m` +fi + +if test -z "$CLANG_SCAN_BUILD"; then + CLANG_SCAN_BUILD=scan-build +fi + +if test -z "$CLANG_ANALYZER"; then + CLANG_ANALYZER=$(which clang++ 2> /dev/null) +fi + COMMON_FLAGS="$COMMON_FLAGS ${CFLAGS}" CROSS_COMPILE= PLATFORM_CCFLAGS= -PLATFORM_CXXFLAGS="$PLATFORM_CXXFLAGS ${CXXFLAGS}" PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS" PLATFORM_SHARED_EXT="so" -PLATFORM_SHARED_LDFLAGS="-shared -Wl,-soname -Wl," +PLATFORM_SHARED_LDFLAGS="-Wl,--no-as-needed -shared -Wl,-soname -Wl," PLATFORM_SHARED_CFLAGS="-fPIC" -PLATFORM_SHARED_VERSIONED=false +PLATFORM_SHARED_VERSIONED=true # generic port files (working on all platform by #ifdef) go directly in /port GENERIC_PORT_FILES=`cd "$ROCKSDB_ROOT"; find port -name '*.cc' | tr "\n" " "` @@ -106,6 +115,7 @@ case "$TARGET_OS" in PLATFORM_SHARED_EXT=dylib PLATFORM_SHARED_LDFLAGS="-dynamiclib -install_name " CROSS_COMPILE=true + PLATFORM_SHARED_VERSIONED= ;; Linux) PLATFORM=OS_LINUX @@ -118,10 +128,17 @@ case "$TARGET_OS" in ;; SunOS) PLATFORM=OS_SOLARIS - COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_SOLARIS" - PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lpthread -lrt" + COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_SOLARIS -m64" + PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lpthread -lrt -static-libstdc++ -static-libgcc -m64" # PORT_FILES=port/sunos/sunos_specific.cc ;; + AIX) + PLATFORM=OS_AIX + CC=gcc + COMMON_FLAGS="$COMMON_FLAGS -maix64 -pthread -fno-builtin-memcmp -D_REENTRANT -DOS_AIX -D__STDC_FORMAT_MACROS" + PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -pthread -lpthread -lrt -maix64 -static-libstdc++ -static-libgcc" + # PORT_FILES=port/aix/aix_specific.cc + ;; FreeBSD) PLATFORM=OS_FREEBSD COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_FREEBSD" @@ -146,9 +163,20 @@ case "$TARGET_OS" in PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lpthread" # PORT_FILES=port/dragonfly/dragonfly_specific.cc ;; + Cygwin) + PLATFORM=CYGWIN + PLATFORM_SHARED_CFLAGS="" + PLATFORM_CXXFLAGS="-std=gnu++11" + COMMON_FLAGS="$COMMON_FLAGS -DCYGWIN" + if [ -z "$USE_CLANG" ]; then + COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp" + fi + PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lpthread -lrt" + # PORT_FILES=port/linux/linux_specific.cc + ;; OS_ANDROID_CROSSCOMPILE) PLATFORM=OS_ANDROID - COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_ANDROID -DLEVELDB_PLATFORM_POSIX" + COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_ANDROID -DROCKSDB_PLATFORM_POSIX" PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS " # All pthread features are in the Android C library # PORT_FILES=port/android/android.cc CROSS_COMPILE=true @@ -158,55 +186,28 @@ case "$TARGET_OS" in exit 1 esac +PLATFORM_CXXFLAGS="$PLATFORM_CXXFLAGS ${CXXFLAGS}" JAVA_LDFLAGS="$PLATFORM_LDFLAGS" - -if test -z "$DO_NOT_RUN_BUILD_DETECT_VERSION"; then - "$PWD/build_tools/build_detect_version" -fi - -# We want to make a list of all cc files within util, db, table, and helpers -# except for the test and benchmark files. By default, find will output a list -# of all files matching either rule, so we need to append -print to make the -# prune take effect. -DIRS="util db table utilities" - -set -f # temporarily disable globbing so that our patterns arent expanded -PRUNE_TEST="-name *test*.cc -prune" -PRUNE_BENCH="-name *bench*.cc -prune" -PORTABLE_FILES=`cd "$ROCKSDB_ROOT"; find $DIRS $PRUNE_TEST -o $PRUNE_BENCH -o -name '*.cc' -print | sort | tr "\n" " "` -PORTABLE_CPP=`cd "$ROCKSDB_ROOT"; find $DIRS $PRUNE_TEST -o $PRUNE_BENCH -o -name '*.cpp' -print | sort | tr "\n" " "` -set +f # re-enable globbing - -# The sources consist of the portable files, plus the platform-specific port -# file. -echo "SOURCES=$PORTABLE_FILES $GENERIC_PORT_FILES $PORT_FILES" >> "$OUTPUT" -echo "SOURCESCPP=$PORTABLE_CPP" >> "$OUTPUT" -echo "MEMENV_SOURCES=helpers/memenv/memenv.cc" >> "$OUTPUT" +JAVA_STATIC_LDFLAGS="$PLATFORM_LDFLAGS" if [ "$CROSS_COMPILE" = "true" -o "$FBCODE_BUILD" = "true" ]; then # Cross-compiling; do not try any compilation tests. # Also don't need any compilation tests if compiling on fbcode true else - # If -std=c++0x works, use . Otherwise use port_posix.h. - $CXX $CFLAGS -std=c++0x -x c++ - -o /dev/null 2>/dev/null < - int main() {} -EOF - if [ "$?" = 0 ]; then - COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_ATOMIC_PRESENT" - fi - - # Test whether fallocate is available - $CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null < - int main() { - int fd = open("/dev/null", 0); - fallocate(fd, 0, 0, 1024); - } + if ! test $ROCKSDB_DISABLE_FALLOCATE; then + # Test whether fallocate is available + $CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null < + #include + int main() { + int fd = open("/dev/null", 0); + fallocate(fd, FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE, 0, 1024); + } EOF - if [ "$?" = 0 ]; then - COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_FALLOCATE_PRESENT" + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_FALLOCATE_PRESENT" + fi fi # Test whether Snappy library is installed @@ -221,11 +222,10 @@ EOF JAVA_LDFLAGS="$JAVA_LDFLAGS -lsnappy" fi - # Test whether gflags library is installed - # http://code.google.com/p/gflags/ + # http://gflags.github.io/gflags/ # check if the namespace is gflags - $CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null </dev/null << EOF #include using namespace gflags; int main() {} @@ -233,17 +233,17 @@ EOF if [ "$?" = 0 ]; then COMMON_FLAGS="$COMMON_FLAGS -DGFLAGS=gflags" PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lgflags" - fi - - # check if namespace is google - $CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null < - using namespace google; - int main() {} + else + # check if namespace is google + $CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null << EOF + #include + using namespace google; + int main() {} EOF - if [ "$?" = 0 ]; then - COMMON_FLAGS="$COMMON_FLAGS -DGFLAGS=google" - PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lgflags" + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DGFLAGS=google" + PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lgflags" + fi fi # Test whether zlib library is installed @@ -280,10 +280,21 @@ EOF JAVA_LDFLAGS="$JAVA_LDFLAGS -llz4" fi + # Test whether zstd library is installed + $CXX $CFLAGS $COMMON_FLAGS -x c++ - -o /dev/null 2>/dev/null < + int main() {} +EOF + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DZSTD" + PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lzstd" + JAVA_LDFLAGS="$JAVA_LDFLAGS -lzstd" + fi + # Test whether numa is available $CXX $CFLAGS -x c++ - -o /dev/null -lnuma 2>/dev/null < - #inlcude + #include int main() {} EOF if [ "$?" = 0 ]; then @@ -292,16 +303,131 @@ EOF JAVA_LDFLAGS="$JAVA_LDFLAGS -lnuma" fi - # Test whether tcmalloc is available - $CXX $CFLAGS -x c++ - -o /dev/null -ltcmalloc 2>/dev/null </dev/null < int main() {} EOF if [ "$?" = 0 ]; then - PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -ltcmalloc" - JAVA_LDFLAGS="$JAVA_LDFLAGS -ltcmalloc" + COMMON_FLAGS="$COMMON_FLAGS -DTBB" + PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -ltbb" + JAVA_LDFLAGS="$JAVA_LDFLAGS -ltbb" + fi + + # Test whether jemalloc is available + if echo 'int main() {}' | $CXX $CFLAGS -x c++ - -o /dev/null -ljemalloc \ + 2>/dev/null; then + # This will enable some preprocessor identifiers in the Makefile + JEMALLOC=1 + # JEMALLOC can be enabled either using the flag (like here) or by + # providing direct link to the jemalloc library + WITH_JEMALLOC_FLAG=1 + else + # jemalloc is not available. Let's try tcmalloc + if echo 'int main() {}' | $CXX $CFLAGS -x c++ - -o /dev/null \ + -ltcmalloc 2>/dev/null; then + PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -ltcmalloc" + JAVA_LDFLAGS="$JAVA_LDFLAGS -ltcmalloc" + fi + fi + + # Test whether malloc_usable_size is available + $CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null < + int main() { + size_t res = malloc_usable_size(0); + return 0; + } +EOF + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_MALLOC_USABLE_SIZE" + fi + + # Test whether PTHREAD_MUTEX_ADAPTIVE_NP mutex type is available + $CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null < + int main() { + int x = PTHREAD_MUTEX_ADAPTIVE_NP; + return 0; + } +EOF + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_PTHREAD_ADAPTIVE_MUTEX" + fi + + # Test whether backtrace is available + $CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null <> + int main() { + void* frames[1]; + backtrace_symbols(frames, backtrace(frames, 1)); + return 0; + } +EOF + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_BACKTRACE" + else + # Test whether execinfo library is installed + $CXX $CFLAGS -lexecinfo -x c++ - -o /dev/null 2>/dev/null < + int main() { + void* frames[1]; + backtrace_symbols(frames, backtrace(frames, 1)); + } +EOF + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_BACKTRACE" + PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lexecinfo" + JAVA_LDFLAGS="$JAVA_LDFLAGS -lexecinfo" + fi + fi + + # Test if -pg is supported + $CXX $CFLAGS -pg -x c++ - -o /dev/null 2>/dev/null </dev/null < + int main() { + int fd = open("/dev/null", 0); + sync_file_range(fd, 0, 1024, SYNC_FILE_RANGE_WRITE); + } +EOF + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_RANGESYNC_PRESENT" + fi + + # Test whether sched_getcpu is supported + $CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null < + int main() { + int cpuid = sched_getcpu(); + } +EOF + if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_SCHED_GETCPU_PRESENT" fi fi +# TODO(tec): Fix -Wshorten-64-to-32 errors on FreeBSD and enable the warning. +# -Wshorten-64-to-32 breaks compilation on FreeBSD i386 +if ! [ "$TARGET_OS" = FreeBSD -a "$TARGET_ARCHITECTURE" = i386 ]; then + # Test whether -Wshorten-64-to-32 is available + $CXX $CFLAGS -x c++ - -o /dev/null -Wshorten-64-to-32 2>/dev/null </dev/null < + #include + int main() { + volatile uint32_t x = _mm_crc32_u32(0, 0); + } +EOF +if [ "$?" = 0 ]; then + COMMON_FLAGS="$COMMON_FLAGS -DHAVE_SSE42" +elif test "$USE_SSE"; then + echo "warning: USE_SSE specified but compiler could not use SSE intrinsics, disabling" +fi + +# iOS doesn't support thread-local storage, but this check would erroneously +# succeed because the cross-compiler flags are added by the Makefile, not this +# script. +if [ "$PLATFORM" != IOS ]; then + $CXX $COMMON_FLAGS -x c++ - -o /dev/null 2>/dev/null <> "$OUTPUT" echo "CXX=$CXX" >> "$OUTPUT" echo "PLATFORM=$PLATFORM" >> "$OUTPUT" echo "PLATFORM_LDFLAGS=$PLATFORM_LDFLAGS" >> "$OUTPUT" echo "JAVA_LDFLAGS=$JAVA_LDFLAGS" >> "$OUTPUT" +echo "JAVA_STATIC_LDFLAGS=$JAVA_STATIC_LDFLAGS" >> "$OUTPUT" echo "VALGRIND_VER=$VALGRIND_VER" >> "$OUTPUT" echo "PLATFORM_CCFLAGS=$PLATFORM_CCFLAGS" >> "$OUTPUT" echo "PLATFORM_CXXFLAGS=$PLATFORM_CXXFLAGS" >> "$OUTPUT" @@ -341,3 +513,20 @@ echo "PLATFORM_SHARED_VERSIONED=$PLATFORM_SHARED_VERSIONED" >> "$OUTPUT" echo "EXEC_LDFLAGS=$EXEC_LDFLAGS" >> "$OUTPUT" echo "JEMALLOC_INCLUDE=$JEMALLOC_INCLUDE" >> "$OUTPUT" echo "JEMALLOC_LIB=$JEMALLOC_LIB" >> "$OUTPUT" +echo "ROCKSDB_MAJOR=$ROCKSDB_MAJOR" >> "$OUTPUT" +echo "ROCKSDB_MINOR=$ROCKSDB_MINOR" >> "$OUTPUT" +echo "ROCKSDB_PATCH=$ROCKSDB_PATCH" >> "$OUTPUT" +echo "CLANG_SCAN_BUILD=$CLANG_SCAN_BUILD" >> "$OUTPUT" +echo "CLANG_ANALYZER=$CLANG_ANALYZER" >> "$OUTPUT" +echo "PROFILING_FLAGS=$PROFILING_FLAGS" >> "$OUTPUT" +# This will enable some related identifiers for the preprocessor +if test -n "$JEMALLOC"; then + echo "JEMALLOC=1" >> "$OUTPUT" +fi +# Indicates that jemalloc should be enabled using -ljemalloc flag +# The alternative is to porvide a direct link to the library via JEMALLOC_LIB +# and JEMALLOC_INCLUDE +if test -n "$WITH_JEMALLOC_FLAG"; then + echo "WITH_JEMALLOC_FLAG=$WITH_JEMALLOC_FLAG" >> "$OUTPUT" +fi +echo "LUA_PATH=$LUA_PATH" >> "$OUTPUT" diff --git a/build_tools/build_detect_version b/build_tools/build_detect_version deleted file mode 100755 index f7d711f0ddd..00000000000 --- a/build_tools/build_detect_version +++ /dev/null @@ -1,22 +0,0 @@ -#!/bin/sh -# -# Record the version of the source that we are compiling. -# We keep a record of the git revision in util/version.cc. This source file -# is then built as a regular source file as part of the compilation process. -# One can run "strings executable_filename | grep _build_" to find the version of -# the source that we used to build the executable file. - -OUTFILE="$PWD/util/build_version.cc" - -GIT_SHA="" -if command -v git >/dev/null 2>&1; then - GIT_SHA=$(git rev-parse HEAD 2>/dev/null) -fi - -cat > "${OUTFILE}" <:: Failure' + _GTEST_FAIL_PATTERN = re.compile(r'(unknown file|\S+:\d+): Failure$') + + def __init__(self): + self._last_gtest_name = 'Unknown test' + + def parse_error(self, line): + gtest_name_match = self._GTEST_NAME_PATTERN.match(line) + if gtest_name_match: + self._last_gtest_name = gtest_name_match.group(1) + return None + gtest_fail_match = self._GTEST_FAIL_PATTERN.match(line) + if gtest_fail_match: + return '%s failed: %s' % ( + self._last_gtest_name, gtest_fail_match.group(1)) + return None + + +class MatchErrorParser(ErrorParserBase): + '''A simple parser that returns the whole line if it matches the pattern. + ''' + def __init__(self, pattern): + self._pattern = re.compile(pattern) + + def parse_error(self, line): + if self._pattern.match(line): + return line + return None + + +class CompilerErrorParser(MatchErrorParser): + def __init__(self): + # format: '::: error: ' + super(CompilerErrorParser, self).__init__(r'\S+:\d+:\d+: error:') + + +class ScanBuildErrorParser(MatchErrorParser): + def __init__(self): + super(ScanBuildErrorParser, self).__init__( + r'scan-build: \d+ bugs found.$') + + +class DbCrashErrorParser(MatchErrorParser): + def __init__(self): + super(DbCrashErrorParser, self).__init__(r'\*\*\*.*\^$|TEST FAILED.') + + +class WriteStressErrorParser(MatchErrorParser): + def __init__(self): + super(WriteStressErrorParser, self).__init__( + r'ERROR: write_stress died with exitcode=\d+') + + +class AsanErrorParser(MatchErrorParser): + def __init__(self): + super(AsanErrorParser, self).__init__( + r'==\d+==ERROR: AddressSanitizer:') + + +class UbsanErrorParser(MatchErrorParser): + def __init__(self): + # format: '::: runtime error: ' + super(UbsanErrorParser, self).__init__(r'\S+:\d+:\d+: runtime error:') + + +class ValgrindErrorParser(MatchErrorParser): + def __init__(self): + # just grab the summary, valgrind doesn't clearly distinguish errors + # from other log messages. + super(ValgrindErrorParser, self).__init__(r'==\d+== ERROR SUMMARY:') + + +class CompatErrorParser(MatchErrorParser): + def __init__(self): + super(CompatErrorParser, self).__init__(r'==== .*[Ee]rror.* ====$') + + +class TsanErrorParser(MatchErrorParser): + def __init__(self): + super(TsanErrorParser, self).__init__(r'WARNING: ThreadSanitizer:') + + +_TEST_NAME_TO_PARSERS = { + 'punit': [CompilerErrorParser, GTestErrorParser], + 'unit': [CompilerErrorParser, GTestErrorParser], + 'release': [CompilerErrorParser, GTestErrorParser], + 'unit_481': [CompilerErrorParser, GTestErrorParser], + 'release_481': [CompilerErrorParser, GTestErrorParser], + 'clang_unit': [CompilerErrorParser, GTestErrorParser], + 'clang_release': [CompilerErrorParser, GTestErrorParser], + 'clang_analyze': [CompilerErrorParser, ScanBuildErrorParser], + 'code_cov': [CompilerErrorParser, GTestErrorParser], + 'unity': [CompilerErrorParser, GTestErrorParser], + 'lite': [CompilerErrorParser], + 'lite_test': [CompilerErrorParser, GTestErrorParser], + 'stress_crash': [CompilerErrorParser, DbCrashErrorParser], + 'write_stress': [CompilerErrorParser, WriteStressErrorParser], + 'asan': [CompilerErrorParser, GTestErrorParser, AsanErrorParser], + 'asan_crash': [CompilerErrorParser, AsanErrorParser, DbCrashErrorParser], + 'ubsan': [CompilerErrorParser, GTestErrorParser, UbsanErrorParser], + 'ubsan_crash': [CompilerErrorParser, UbsanErrorParser, DbCrashErrorParser], + 'valgrind': [CompilerErrorParser, GTestErrorParser, ValgrindErrorParser], + 'tsan': [CompilerErrorParser, GTestErrorParser, TsanErrorParser], + 'format_compatible': [CompilerErrorParser, CompatErrorParser], + 'run_format_compatible': [CompilerErrorParser, CompatErrorParser], + 'no_compression': [CompilerErrorParser, GTestErrorParser], + 'run_no_compression': [CompilerErrorParser, GTestErrorParser], + 'regression': [CompilerErrorParser], + 'run_regression': [CompilerErrorParser], +} + + +def main(): + if len(sys.argv) != 2: + return 'Usage: %s ' % sys.argv[0] + test_name = sys.argv[1] + if test_name not in _TEST_NAME_TO_PARSERS: + return 'Unknown test name: %s' % test_name + + error_parsers = [] + for parser_cls in _TEST_NAME_TO_PARSERS[test_name]: + error_parsers.append(parser_cls()) + + for line in sys.stdin: + line = line.strip() + for error_parser in error_parsers: + error_msg = error_parser.parse_error(line) + if error_msg is not None: + print(error_msg) + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/build_tools/fb_compile_mongo.sh b/build_tools/fb_compile_mongo.sh new file mode 100755 index 00000000000..c087f81611f --- /dev/null +++ b/build_tools/fb_compile_mongo.sh @@ -0,0 +1,55 @@ +#!/bin/sh + +# fail early +set -e + +if test -z $ROCKSDB_PATH; then + ROCKSDB_PATH=~/rocksdb +fi +source $ROCKSDB_PATH/build_tools/fbcode_config4.8.1.sh + +EXTRA_LDFLAGS="" + +if test -z $ALLOC; then + # default + ALLOC=tcmalloc +elif [[ $ALLOC == "jemalloc" ]]; then + ALLOC=system + EXTRA_LDFLAGS+=" -Wl,--whole-archive $JEMALLOC_LIB -Wl,--no-whole-archive" +fi + +# we need to force mongo to use static library, not shared +STATIC_LIB_DEP_DIR='build/static_library_dependencies' +test -d $STATIC_LIB_DEP_DIR || mkdir $STATIC_LIB_DEP_DIR +test -h $STATIC_LIB_DEP_DIR/`basename $SNAPPY_LIBS` || ln -s $SNAPPY_LIBS $STATIC_LIB_DEP_DIR +test -h $STATIC_LIB_DEP_DIR/`basename $LZ4_LIBS` || ln -s $LZ4_LIBS $STATIC_LIB_DEP_DIR + +EXTRA_LDFLAGS+=" -L $STATIC_LIB_DEP_DIR" + +set -x + +EXTRA_CMD="" +if ! test -e version.json; then + # this is Mongo 3.0 + EXTRA_CMD="--rocksdb \ + --variant-dir=linux2/norm + --cxx=${CXX} \ + --cc=${CC} \ + --use-system-zlib" # add this line back to normal code path + # when https://jira.mongodb.org/browse/SERVER-19123 is resolved +fi + +scons \ + LINKFLAGS="$EXTRA_LDFLAGS $EXEC_LDFLAGS $PLATFORM_LDFLAGS" \ + CCFLAGS="$CXXFLAGS -L $STATIC_LIB_DEP_DIR" \ + LIBS="lz4 gcc stdc++" \ + LIBPATH="$ROCKSDB_PATH" \ + CPPPATH="$ROCKSDB_PATH/include" \ + -j32 \ + --allocator=$ALLOC \ + --nostrip \ + --opt=on \ + --disable-minimum-compiler-version-enforcement \ + --use-system-snappy \ + --disable-warnings-as-errors \ + $EXTRA_CMD $* diff --git a/build_tools/fbcode.clang31.sh b/build_tools/fbcode.clang31.sh deleted file mode 100644 index 25a2ca72fbc..00000000000 --- a/build_tools/fbcode.clang31.sh +++ /dev/null @@ -1,74 +0,0 @@ -#!/bin/sh -# -# Set environment variables so that we can compile leveldb using -# fbcode settings. It uses the latest g++ compiler and also -# uses jemalloc - -TOOLCHAIN_REV=fbe3b095a4cc4a3713730050d182b7b4a80c342f -TOOLCHAIN_EXECUTABLES="/mnt/gvfs/third-party/$TOOLCHAIN_REV/centos5.2-native" -TOOLCHAIN_LIB_BASE="/mnt/gvfs/third-party/$TOOLCHAIN_REV/gcc-4.7.1-glibc-2.14.1" -TOOL_JEMALLOC=jemalloc-3.3.1/9202ce3 -GLIBC_RUNTIME_PATH=/usr/local/fbcode/gcc-4.7.1-glibc-2.14.1 - -# location of libgcc -LIBGCC_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.7.1/afc21dc/include" -LIBGCC_LIBS=" -L $TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.7.1/afc21dc/libs" - -# location of glibc -GLIBC_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/glibc/glibc-2.14.1/99df8fc/include" -GLIBC_LIBS=" -L $TOOLCHAIN_LIB_BASE/glibc/glibc-2.14.1/99df8fc/lib" - -# location of snappy headers and libraries -SNAPPY_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/snappy/snappy-1.0.3/7518bbe/include" -SNAPPY_LIBS=" $TOOLCHAIN_LIB_BASE/snappy/snappy-1.0.3/7518bbe/lib/libsnappy.a" - -# location of zlib headers and libraries -ZLIB_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/zlib/zlib-1.2.5/91ddd43/include" -ZLIB_LIBS=" $TOOLCHAIN_LIB_BASE/zlib/zlib-1.2.5/91ddd43/lib/libz.a" - -# location of gflags headers and libraries -GFLAGS_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/gflags/gflags-1.6/91ddd43/include" -GFLAGS_LIBS=" $TOOLCHAIN_LIB_BASE/gflags/gflags-1.6/91ddd43/lib/libgflags.a" - -# location of bzip headers and libraries -BZIP_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/bzip2/bzip2-1.0.6/91ddd43/include" -BZIP_LIBS=" $TOOLCHAIN_LIB_BASE/bzip2/bzip2-1.0.6/91ddd43/lib/libbz2.a" - -# location of gflags headers and libraries -GFLAGS_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/gflags/gflags-1.6/91ddd43/include" -GFLAGS_LIBS=" $TOOLCHAIN_LIB_BASE/gflags/gflags-1.6/91ddd43/lib/libgflags.a" - -# use Intel SSE support for checksum calculations -export USE_SSE=" -msse -msse4.2 " - -CC="$TOOLCHAIN_EXECUTABLES/clang/clang-3.2/0b7c69d/bin/clang $CLANG_INCLUDES" -CXX="$TOOLCHAIN_EXECUTABLES/clang/clang-3.2/0b7c69d/bin/clang++ $CLANG_INCLUDES $JINCLUDE $SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $GFLAGS_INCLUDE" -AR=$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/ar -RANLIB=$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/ranlib - -CFLAGS="-B$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin -nostdlib " -CFLAGS+=" -nostdinc -isystem $TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.7.1/afc21dc/include/c++/4.7.1 " -CFLAGS+=" -isystem $TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.7.1/afc21dc/include/c++/4.7.1/x86_64-facebook-linux " -CFLAGS+=" -isystem $TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.7.1/afc21dc/include/c++/4.7.1/backward " -CFLAGS+=" -isystem $TOOLCHAIN_LIB_BASE/glibc/glibc-2.14.1/99df8fc/include " -CFLAGS+=" -isystem $TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.7.1/afc21dc/include " -CFLAGS+=" -isystem $TOOLCHAIN_LIB_BASE/clang/clang-3.2/0b7c69d/lib/clang/3.2/include " -CFLAGS+=" -isystem $TOOLCHAIN_LIB_BASE/kernel-headers/kernel-headers-3.2.18_70_fbk11_00129_gc8882d0/da39a3e/include/linux " -CFLAGS+=" -isystem $TOOLCHAIN_LIB_BASE/kernel-headers/kernel-headers-3.2.18_70_fbk11_00129_gc8882d0/da39a3e/include " -CFLAGS+=" -Wall -Wno-sign-compare -Wno-unused-variable -Winvalid-pch -Wno-deprecated -Woverloaded-virtual" -CFLAGS+=" $LIBGCC_INCLUDE $GLIBC_INCLUDE" -CXXFLAGS="$CFLAGS -nostdinc++" - -CFLAGS+=" -I $TOOLCHAIN_LIB_BASE/jemalloc/$TOOL_JEMALLOC/include -DHAVE_JEMALLOC" - -EXEC_LDFLAGS=" -Wl,--whole-archive $TOOLCHAIN_LIB_BASE/jemalloc/$TOOL_JEMALLOC/lib/libjemalloc.a" -EXEC_LDFLAGS+=" -Wl,--no-whole-archive $TOOLCHAIN_LIB_BASE/libunwind/libunwind-1.0.1/350336c/lib/libunwind.a" -EXEC_LDFLAGS+=" $HDFSLIB $SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $GFLAGS_LIBS" -EXEC_LDFLAGS+=" -Wl,--dynamic-linker,$GLIBC_RUNTIME_PATH/lib/ld-linux-x86-64.so.2" -EXEC_LDFLAGS+=" -B$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin" - -PLATFORM_LDFLAGS="$LIBGCC_LIBS $GLIBC_LIBS " - -EXEC_LDFLAGS_SHARED="$SNAPPY_LIBS $ZLIB_LIBS $GFLAGS_LIBS" - -export CC CXX AR RANLIB CFLAGS EXEC_LDFLAGS EXEC_LDFLAGS_SHARED diff --git a/build_tools/fbcode.gcc471.sh b/build_tools/fbcode.gcc471.sh deleted file mode 100644 index c971cda5b1b..00000000000 --- a/build_tools/fbcode.gcc471.sh +++ /dev/null @@ -1,70 +0,0 @@ -#!/bin/sh -# -# Set environment variables so that we can compile leveldb using -# fbcode settings. It uses the latest g++ compiler and also -# uses jemalloc - -TOOLCHAIN_REV=fbe3b095a4cc4a3713730050d182b7b4a80c342f -TOOLCHAIN_EXECUTABLES="/mnt/gvfs/third-party/$TOOLCHAIN_REV/centos5.2-native" -TOOLCHAIN_LIB_BASE="/mnt/gvfs/third-party/$TOOLCHAIN_REV/gcc-4.7.1-glibc-2.14.1" -TOOL_JEMALLOC=jemalloc-3.3.1/9202ce3 - -# location of libhdfs libraries -if test "$USE_HDFS"; then - JAVA_HOME="/usr/local/jdk-6u22-64" - JINCLUDE="-I$JAVA_HOME/include -I$JAVA_HOME/include/linux" - GLIBC_RUNTIME_PATH="/usr/local/fbcode/gcc-4.7.1-glibc-2.14.1" - HDFSLIB=" -Wl,--no-whole-archive hdfs/libhdfs.a -L$JAVA_HOME/jre/lib/amd64 " - HDFSLIB+=" -L$JAVA_HOME/jre/lib/amd64/server -L$GLIBC_RUNTIME_PATH/lib " - HDFSLIB+=" -ldl -lverify -ljava -ljvm " -fi - -# location of libgcc -LIBGCC_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.7.1/afc21dc/include" -LIBGCC_LIBS=" -L $TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.7.1/afc21dc/libs" - -# location of glibc -GLIBC_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/glibc/glibc-2.14.1/99df8fc/include" -GLIBC_LIBS=" -L $TOOLCHAIN_LIB_BASE/glibc/glibc-2.14.1/99df8fc/lib" - -# location of snappy headers and libraries -SNAPPY_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/snappy/snappy-1.0.3/7518bbe/include" -SNAPPY_LIBS=" $TOOLCHAIN_LIB_BASE/snappy/snappy-1.0.3/7518bbe/lib/libsnappy.a" - -# location of zlib headers and libraries -ZLIB_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/zlib/zlib-1.2.5/91ddd43/include" -ZLIB_LIBS=" $TOOLCHAIN_LIB_BASE/zlib/zlib-1.2.5/91ddd43/lib/libz.a" - -# location of bzip headers and libraries -BZIP_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/bzip2/bzip2-1.0.6/91ddd43/include" -BZIP_LIBS=" $TOOLCHAIN_LIB_BASE/bzip2/bzip2-1.0.6/91ddd43/lib/libbz2.a" - -# location of gflags headers and libraries -GFLAGS_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/gflags/gflags-1.6/91ddd43/include" -GFLAGS_LIBS=" $TOOLCHAIN_LIB_BASE/gflags/gflags-1.6/91ddd43/lib/libgflags.a" - -# use Intel SSE support for checksum calculations -export USE_SSE=" -msse -msse4.2 " - -CC="$TOOLCHAIN_EXECUTABLES/gcc/gcc-4.7.1-glibc-2.14.1/bin/gcc" -CXX="$TOOLCHAIN_EXECUTABLES/gcc/gcc-4.7.1-glibc-2.14.1/bin/g++ $JINCLUDE $SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $GFLAGS_INCLUDE" -AR=$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/ar -RANLIB=$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/ranlib - -CFLAGS="-B$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/gold -m64 -mtune=generic" -CFLAGS+=" -I $TOOLCHAIN_LIB_BASE/jemalloc/$TOOL_JEMALLOC/include -DHAVE_JEMALLOC" -CFLAGS+=" $LIBGCC_INCLUDE $GLIBC_INCLUDE" -CFLAGS+=" -DROCKSDB_PLATFORM_POSIX -DROCKSDB_ATOMIC_PRESENT -DROCKSDB_FALLOCATE_PRESENT" -CFLAGS+=" -DSNAPPY -DGFLAGS=google -DZLIB -DBZIP2" - -EXEC_LDFLAGS=" -Wl,--whole-archive $TOOLCHAIN_LIB_BASE/jemalloc/$TOOL_JEMALLOC/lib/libjemalloc.a" -EXEC_LDFLAGS+=" -Wl,--no-whole-archive $TOOLCHAIN_LIB_BASE/libunwind/libunwind-1.0.1/350336c/lib/libunwind.a" -EXEC_LDFLAGS+=" $HDFSLIB $SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $GFLAGS_LIBS" - -PLATFORM_LDFLAGS="$LIBGCC_LIBS $GLIBC_LIBS " - -EXEC_LDFLAGS_SHARED="$SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $GFLAGS_LIBS" - -VALGRIND_VER="$TOOLCHAIN_LIB_BASE/valgrind/valgrind-3.8.1/91ddd43/bin/" - -export CC CXX AR RANLIB CFLAGS EXEC_LDFLAGS EXEC_LDFLAGS_SHARED VALGRIND_VER diff --git a/build_tools/fbcode.gcc481.sh b/build_tools/fbcode.gcc481.sh deleted file mode 100644 index 5426e3f9ac2..00000000000 --- a/build_tools/fbcode.gcc481.sh +++ /dev/null @@ -1,86 +0,0 @@ -#!/bin/sh -# -# Set environment variables so that we can compile rocksdb using -# fbcode settings. It uses the latest g++ compiler and also -# uses jemalloc - -TOOLCHAIN_REV=53dc1fe83f84e9145b9ffb81b81aa7f6a49c87cc -CENTOS_VERSION=`rpm -q --qf "%{VERSION}" $(rpm -q --whatprovides redhat-release)` -if [ "$CENTOS_VERSION" = "6" ]; then - TOOLCHAIN_EXECUTABLES="/mnt/gvfs/third-party/$TOOLCHAIN_REV/centos6-native" -else - TOOLCHAIN_EXECUTABLES="/mnt/gvfs/third-party/$TOOLCHAIN_REV/centos5.2-native" -fi -TOOLCHAIN_LIB_BASE="/mnt/gvfs/third-party/$TOOLCHAIN_REV/gcc-4.8.1-glibc-2.17" - -# location of libhdfs libraries -if test "$USE_HDFS"; then - JAVA_HOME="/usr/local/jdk-6u22-64" - JINCLUDE="-I$JAVA_HOME/include -I$JAVA_HOME/include/linux" - GLIBC_RUNTIME_PATH="/usr/local/fbcode/gcc-4.8.1-glibc-2.17" - HDFSLIB=" -Wl,--no-whole-archive hdfs/libhdfs.a -L$JAVA_HOME/jre/lib/amd64 " - HDFSLIB+=" -L$JAVA_HOME/jre/lib/amd64/server -L$GLIBC_RUNTIME_PATH/lib " - HDFSLIB+=" -ldl -lverify -ljava -ljvm " -fi - -# location of libgcc -LIBGCC_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.8.1/8aac7fc/include" -LIBGCC_LIBS=" -L $TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.8.1/8aac7fc/libs" - -# location of glibc -GLIBC_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/glibc/glibc-2.17/99df8fc/include" -GLIBC_LIBS=" -L $TOOLCHAIN_LIB_BASE/glibc/glibc-2.17/99df8fc/lib" - -# location of snappy headers and libraries -SNAPPY_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/snappy/snappy-1.0.3/43d84e2/include" -SNAPPY_LIBS=" $TOOLCHAIN_LIB_BASE/snappy/snappy-1.0.3/43d84e2/lib/libsnappy.a" - -# location of zlib headers and libraries -ZLIB_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/zlib/zlib-1.2.5/c3f970a/include" -ZLIB_LIBS=" $TOOLCHAIN_LIB_BASE/zlib/zlib-1.2.5/c3f970a/lib/libz.a" - -# location of bzip headers and libraries -BZIP_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/bzip2/bzip2-1.0.6/c3f970a/include" -BZIP_LIBS=" $TOOLCHAIN_LIB_BASE/bzip2/bzip2-1.0.6/c3f970a/lib/libbz2.a" - -LZ4_REV=065ec7e38fe83329031f6668c43bef83eff5808b -LZ4_INCLUDE=" -I /mnt/gvfs/third-party2/lz4/$LZ4_REV/r108/gcc-4.8.1-glibc-2.17/c3f970a/include" -LZ4_LIBS=" /mnt/gvfs/third-party2/lz4/$LZ4_REV/r108/gcc-4.8.1-glibc-2.17/c3f970a/lib/liblz4.a" - -# location of gflags headers and libraries -GFLAGS_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/gflags/gflags-1.6/c3f970a/include" -GFLAGS_LIBS=" $TOOLCHAIN_LIB_BASE/gflags/gflags-1.6/c3f970a/lib/libgflags.a" - -# location of jemalloc -JEMALLOC_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/jemalloc/jemalloc-3.4.1/4d53c6f/include/" -JEMALLOC_LIB=" -Wl,--whole-archive $TOOLCHAIN_LIB_BASE/jemalloc/jemalloc-3.4.1/4d53c6f/lib/libjemalloc.a" - -# location of numa -NUMA_REV=829d10dac0230f99cd7e1778869d2adf3da24b65 -NUMA_INCLUDE=" -I /mnt/gvfs/third-party2/numa/$NUMA_REV/2.0.8/gcc-4.8.1-glibc-2.17/c3f970a/include/" -NUMA_LIB=" /mnt/gvfs/third-party2/numa/$NUMA_REV/2.0.8/gcc-4.8.1-glibc-2.17/c3f970a/lib/libnuma.a" - -# use Intel SSE support for checksum calculations -export USE_SSE=" -msse -msse4.2 " - -CC="$TOOLCHAIN_EXECUTABLES/gcc/gcc-4.8.1/cc6c9dc/bin/gcc" -CXX="$TOOLCHAIN_EXECUTABLES/gcc/gcc-4.8.1/cc6c9dc/bin/g++ $JINCLUDE $SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $LZ4_INCLUDE $GFLAGS_INCLUDE $NUMA_INCLUDE" -AR=$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/ar -RANLIB=$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/ranlib - -CFLAGS="-B$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/gold -m64 -mtune=generic" -CFLAGS+=" $LIBGCC_INCLUDE $GLIBC_INCLUDE" -CFLAGS+=" -DROCKSDB_PLATFORM_POSIX -DROCKSDB_ATOMIC_PRESENT -DROCKSDB_FALLOCATE_PRESENT" -CFLAGS+=" -DSNAPPY -DGFLAGS=google -DZLIB -DBZIP2 -DLZ4 -DNUMA" - -EXEC_LDFLAGS="-Wl,--dynamic-linker,/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/ld.so" -EXEC_LDFLAGS+=" -Wl,--no-whole-archive $TOOLCHAIN_LIB_BASE/libunwind/libunwind-1.0.1/675d945/lib/libunwind.a" -EXEC_LDFLAGS+=" $HDFSLIB $SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $GFLAGS_LIBS $NUMA_LIB" - -PLATFORM_LDFLAGS="$LIBGCC_LIBS $GLIBC_LIBS " - -EXEC_LDFLAGS_SHARED="$SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $GFLAGS_LIBS" - -VALGRIND_VER="$TOOLCHAIN_LIB_BASE/valgrind/valgrind-3.8.1/c3f970a/bin/" - -export CC CXX AR RANLIB CFLAGS EXEC_LDFLAGS EXEC_LDFLAGS_SHARED VALGRIND_VER JEMALLOC_LIB JEMALLOC_INCLUDE diff --git a/build_tools/fbcode_config.sh b/build_tools/fbcode_config.sh new file mode 100644 index 00000000000..b8609a11c69 --- /dev/null +++ b/build_tools/fbcode_config.sh @@ -0,0 +1,156 @@ +#!/bin/sh +# +# Set environment variables so that we can compile rocksdb using +# fbcode settings. It uses the latest g++ and clang compilers and also +# uses jemalloc +# Environment variables that change the behavior of this script: +# PIC_BUILD -- if true, it will only take pic versions of libraries from fbcode. libraries that don't have pic variant will not be included + + +BASEDIR=`dirname $BASH_SOURCE` +source "$BASEDIR/dependencies.sh" + +CFLAGS="" + +# libgcc +LIBGCC_INCLUDE="$LIBGCC_BASE/include" +LIBGCC_LIBS=" -L $LIBGCC_BASE/lib" + +# glibc +GLIBC_INCLUDE="$GLIBC_BASE/include" +GLIBC_LIBS=" -L $GLIBC_BASE/lib" + +# snappy +SNAPPY_INCLUDE=" -I $SNAPPY_BASE/include/" +if test -z $PIC_BUILD; then + SNAPPY_LIBS=" $SNAPPY_BASE/lib/libsnappy.a" +else + SNAPPY_LIBS=" $SNAPPY_BASE/lib/libsnappy_pic.a" +fi +CFLAGS+=" -DSNAPPY" + +if test -z $PIC_BUILD; then + # location of zlib headers and libraries + ZLIB_INCLUDE=" -I $ZLIB_BASE/include/" + ZLIB_LIBS=" $ZLIB_BASE/lib/libz.a" + CFLAGS+=" -DZLIB" + + # location of bzip headers and libraries + BZIP_INCLUDE=" -I $BZIP2_BASE/include/" + BZIP_LIBS=" $BZIP2_BASE/lib/libbz2.a" + CFLAGS+=" -DBZIP2" + + LZ4_INCLUDE=" -I $LZ4_BASE/include/" + LZ4_LIBS=" $LZ4_BASE/lib/liblz4.a" + CFLAGS+=" -DLZ4" + + ZSTD_INCLUDE=" -I $ZSTD_BASE/include/" + ZSTD_LIBS=" $ZSTD_BASE/lib/libzstd.a" + CFLAGS+=" -DZSTD" +fi + +# location of gflags headers and libraries +GFLAGS_INCLUDE=" -I $GFLAGS_BASE/include/" +if test -z $PIC_BUILD; then + GFLAGS_LIBS=" $GFLAGS_BASE/lib/libgflags.a" +else + GFLAGS_LIBS=" $GFLAGS_BASE/lib/libgflags_pic.a" +fi +CFLAGS+=" -DGFLAGS=gflags" + +# location of jemalloc +JEMALLOC_INCLUDE=" -I $JEMALLOC_BASE/include/" +JEMALLOC_LIB=" $JEMALLOC_BASE/lib/libjemalloc.a" + +if test -z $PIC_BUILD; then + # location of numa + NUMA_INCLUDE=" -I $NUMA_BASE/include/" + NUMA_LIB=" $NUMA_BASE/lib/libnuma.a" + CFLAGS+=" -DNUMA" + + # location of libunwind + LIBUNWIND="$LIBUNWIND_BASE/lib/libunwind.a" +fi + +# location of TBB +TBB_INCLUDE=" -isystem $TBB_BASE/include/" +if test -z $PIC_BUILD; then + TBB_LIBS="$TBB_BASE/lib/libtbb.a" +else + TBB_LIBS="$TBB_BASE/lib/libtbb_pic.a" +fi +CFLAGS+=" -DTBB" + +# use Intel SSE support for checksum calculations +export USE_SSE=1 + +BINUTILS="$BINUTILS_BASE/bin" +AR="$BINUTILS/ar" + +DEPS_INCLUDE="$SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $LZ4_INCLUDE $ZSTD_INCLUDE $GFLAGS_INCLUDE $NUMA_INCLUDE $TBB_INCLUDE" + +STDLIBS="-L $GCC_BASE/lib64" + +CLANG_BIN="$CLANG_BASE/bin" +CLANG_LIB="$CLANG_BASE/lib" +CLANG_SRC="$CLANG_BASE/../../src" + +CLANG_ANALYZER="$CLANG_BIN/clang++" +CLANG_SCAN_BUILD="$CLANG_SRC/llvm/tools/clang/tools/scan-build/bin/scan-build" + +if [ -z "$USE_CLANG" ]; then + # gcc + CC="$GCC_BASE/bin/gcc" + CXX="$GCC_BASE/bin/g++" + + CFLAGS+=" -B$BINUTILS/gold" + CFLAGS+=" -isystem $GLIBC_INCLUDE" + CFLAGS+=" -isystem $LIBGCC_INCLUDE" + JEMALLOC=1 +else + # clang + CLANG_INCLUDE="$CLANG_LIB/clang/stable/include" + CC="$CLANG_BIN/clang" + CXX="$CLANG_BIN/clang++" + + KERNEL_HEADERS_INCLUDE="$KERNEL_HEADERS_BASE/include" + + CFLAGS+=" -B$BINUTILS/gold -nostdinc -nostdlib" + CFLAGS+=" -isystem $LIBGCC_BASE/include/c++/5.x " + CFLAGS+=" -isystem $LIBGCC_BASE/include/c++/5.x/x86_64-facebook-linux " + CFLAGS+=" -isystem $GLIBC_INCLUDE" + CFLAGS+=" -isystem $LIBGCC_INCLUDE" + CFLAGS+=" -isystem $CLANG_INCLUDE" + CFLAGS+=" -isystem $KERNEL_HEADERS_INCLUDE/linux " + CFLAGS+=" -isystem $KERNEL_HEADERS_INCLUDE " + CFLAGS+=" -Wno-expansion-to-defined " + CXXFLAGS="-nostdinc++" +fi + +CFLAGS+=" $DEPS_INCLUDE" +CFLAGS+=" -DROCKSDB_PLATFORM_POSIX -DROCKSDB_LIB_IO_POSIX -DROCKSDB_FALLOCATE_PRESENT -DROCKSDB_MALLOC_USABLE_SIZE -DROCKSDB_RANGESYNC_PRESENT -DROCKSDB_SCHED_GETCPU_PRESENT -DROCKSDB_SUPPORT_THREAD_LOCAL -DHAVE_SSE42" +CXXFLAGS+=" $CFLAGS" + +EXEC_LDFLAGS=" $SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $ZSTD_LIBS $GFLAGS_LIBS $NUMA_LIB $TBB_LIBS" +EXEC_LDFLAGS+=" -B$BINUTILS/gold" +EXEC_LDFLAGS+=" -Wl,--dynamic-linker,/usr/local/fbcode/gcc-5-glibc-2.23/lib/ld.so" +EXEC_LDFLAGS+=" $LIBUNWIND" +EXEC_LDFLAGS+=" -Wl,-rpath=/usr/local/fbcode/gcc-5-glibc-2.23/lib" +# required by libtbb +EXEC_LDFLAGS+=" -ldl" + +PLATFORM_LDFLAGS="$LIBGCC_LIBS $GLIBC_LIBS $STDLIBS -lgcc -lstdc++" + +EXEC_LDFLAGS_SHARED="$SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $ZSTD_LIBS $GFLAGS_LIBS $TBB_LIBS" + +VALGRIND_VER="$VALGRIND_BASE/bin/" + +LUA_PATH="$LUA_BASE" + +if test -z $PIC_BUILD; then + LUA_LIB=" $LUA_PATH/lib/liblua.a" +else + LUA_LIB=" $LUA_PATH/lib/liblua_pic.a" +fi + +export CC CXX AR CFLAGS CXXFLAGS EXEC_LDFLAGS EXEC_LDFLAGS_SHARED VALGRIND_VER JEMALLOC_LIB JEMALLOC_INCLUDE CLANG_ANALYZER CLANG_SCAN_BUILD LUA_PATH LUA_LIB diff --git a/build_tools/fbcode_config4.8.1.sh b/build_tools/fbcode_config4.8.1.sh new file mode 100644 index 00000000000..f5b8334db21 --- /dev/null +++ b/build_tools/fbcode_config4.8.1.sh @@ -0,0 +1,115 @@ +#!/bin/sh +# +# Set environment variables so that we can compile rocksdb using +# fbcode settings. It uses the latest g++ compiler and also +# uses jemalloc + +BASEDIR=`dirname $BASH_SOURCE` +source "$BASEDIR/dependencies_4.8.1.sh" + +# location of libgcc +LIBGCC_INCLUDE="$LIBGCC_BASE/include" +LIBGCC_LIBS=" -L $LIBGCC_BASE/lib" + +# location of glibc +GLIBC_INCLUDE="$GLIBC_BASE/include" +GLIBC_LIBS=" -L $GLIBC_BASE/lib" + +# location of snappy headers and libraries +SNAPPY_INCLUDE=" -I $SNAPPY_BASE/include" +SNAPPY_LIBS=" $SNAPPY_BASE/lib/libsnappy.a" + +# location of zlib headers and libraries +ZLIB_INCLUDE=" -I $ZLIB_BASE/include" +ZLIB_LIBS=" $ZLIB_BASE/lib/libz.a" + +# location of bzip headers and libraries +BZIP2_INCLUDE=" -I $BZIP2_BASE/include/" +BZIP2_LIBS=" $BZIP2_BASE/lib/libbz2.a" + +LZ4_INCLUDE=" -I $LZ4_BASE/include" +LZ4_LIBS=" $LZ4_BASE/lib/liblz4.a" + +ZSTD_INCLUDE=" -I $ZSTD_BASE/include" +ZSTD_LIBS=" $ZSTD_BASE/lib/libzstd.a" + +# location of gflags headers and libraries +GFLAGS_INCLUDE=" -I $GFLAGS_BASE/include/" +GFLAGS_LIBS=" $GFLAGS_BASE/lib/libgflags.a" + +# location of jemalloc +JEMALLOC_INCLUDE=" -I $JEMALLOC_BASE/include" +JEMALLOC_LIB="$JEMALLOC_BASE/lib/libjemalloc.a" + +# location of numa +NUMA_INCLUDE=" -I $NUMA_BASE/include/" +NUMA_LIB=" $NUMA_BASE/lib/libnuma.a" + +# location of libunwind +LIBUNWIND="$LIBUNWIND_BASE/lib/libunwind.a" + +# location of tbb +TBB_INCLUDE=" -isystem $TBB_BASE/include/" +TBB_LIBS="$TBB_BASE/lib/libtbb.a" + +# use Intel SSE support for checksum calculations +export USE_SSE=1 + +BINUTILS="$BINUTILS_BASE/bin" +AR="$BINUTILS/ar" + +DEPS_INCLUDE="$SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP2_INCLUDE $LZ4_INCLUDE $ZSTD_INCLUDE $GFLAGS_INCLUDE $NUMA_INCLUDE $TBB_INCLUDE" + +STDLIBS="-L $GCC_BASE/lib64" + +if [ -z "$USE_CLANG" ]; then + # gcc + CC="$GCC_BASE/bin/gcc" + CXX="$GCC_BASE/bin/g++" + + CFLAGS="-B$BINUTILS/gold -m64 -mtune=generic" + CFLAGS+=" -isystem $GLIBC_INCLUDE" + CFLAGS+=" -isystem $LIBGCC_INCLUDE" + JEMALLOC=1 +else + # clang + CLANG_BIN="$CLANG_BASE/bin" + CLANG_LIB="$CLANG_BASE/lib" + CLANG_INCLUDE="$CLANG_LIB/clang/*/include" + CC="$CLANG_BIN/clang" + CXX="$CLANG_BIN/clang++" + + KERNEL_HEADERS_INCLUDE="$KERNEL_HEADERS_BASE/include/" + + CFLAGS="-B$BINUTILS/gold -nostdinc -nostdlib" + CFLAGS+=" -isystem $LIBGCC_BASE/include/c++/4.8.1 " + CFLAGS+=" -isystem $LIBGCC_BASE/include/c++/4.8.1/x86_64-facebook-linux " + CFLAGS+=" -isystem $GLIBC_INCLUDE" + CFLAGS+=" -isystem $LIBGCC_INCLUDE" + CFLAGS+=" -isystem $CLANG_INCLUDE" + CFLAGS+=" -isystem $KERNEL_HEADERS_INCLUDE/linux " + CFLAGS+=" -isystem $KERNEL_HEADERS_INCLUDE " + CXXFLAGS="-nostdinc++" +fi + +CFLAGS+=" $DEPS_INCLUDE" +CFLAGS+=" -DROCKSDB_PLATFORM_POSIX -DROCKSDB_LIB_IO_POSIX -DROCKSDB_FALLOCATE_PRESENT -DROCKSDB_MALLOC_USABLE_SIZE -DROCKSDB_RANGESYNC_PRESENT -DROCKSDB_SCHED_GETCPU_PRESENT -DROCKSDB_SUPPORT_THREAD_LOCAL -DHAVE_SSE42" +CFLAGS+=" -DSNAPPY -DGFLAGS=google -DZLIB -DBZIP2 -DLZ4 -DZSTD -DNUMA -DTBB" +CXXFLAGS+=" $CFLAGS" + +EXEC_LDFLAGS=" $SNAPPY_LIBS $ZLIB_LIBS $BZIP2_LIBS $LZ4_LIBS $ZSTD_LIBS $GFLAGS_LIBS $NUMA_LIB $TBB_LIBS" +EXEC_LDFLAGS+=" -Wl,--dynamic-linker,/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib/ld.so" +EXEC_LDFLAGS+=" $LIBUNWIND" +EXEC_LDFLAGS+=" -Wl,-rpath=/usr/local/fbcode/gcc-4.8.1-glibc-2.17/lib" +# required by libtbb +EXEC_LDFLAGS+=" -ldl" + +PLATFORM_LDFLAGS="$LIBGCC_LIBS $GLIBC_LIBS $STDLIBS -lgcc -lstdc++" + +EXEC_LDFLAGS_SHARED="$SNAPPY_LIBS $ZLIB_LIBS $BZIP2_LIBS $LZ4_LIBS $ZSTD_LIBS $GFLAGS_LIBS" + +VALGRIND_VER="$VALGRIND_BASE/bin/" + +LUA_PATH="$LUA_BASE" + +export CC CXX AR CFLAGS CXXFLAGS EXEC_LDFLAGS EXEC_LDFLAGS_SHARED VALGRIND_VER JEMALLOC_LIB JEMALLOC_INCLUDE LUA_PATH diff --git a/build_tools/format-diff.sh b/build_tools/format-diff.sh index 2d60620091f..81221ed9a49 100755 --- a/build_tools/format-diff.sh +++ b/build_tools/format-diff.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # If clang_format_diff.py command is not specfied, we assume we are able to # access directly without any path. if [ -z $CLANG_FORMAT_DIFF ] @@ -9,9 +9,12 @@ fi # Check clang-format-diff.py if ! which $CLANG_FORMAT_DIFF &> /dev/null then - echo "You didn't have clang-format-diff.py available in your computer!" - echo "You can download it by running: " - echo " curl http://goo.gl/iUW1u2" + echo "You didn't have clang-format-diff.py and/or clang-format available in your computer!" + echo "You can download clang-format-diff.py by running: " + echo " curl --location http://goo.gl/iUW1u2 -o ${CLANG_FORMAT_DIFF}" + echo "You can download clang-format by running: " + echo " brew install clang-format" + echo "Then, move both files (i.e. ${CLANG_FORMAT_DIFF} and clang-format) to some directory within PATH=${PATH}" exit 128 fi @@ -50,14 +53,15 @@ fi set -e uncommitted_code=`git diff HEAD` +LAST_MASTER=`git merge-base master HEAD` # If there's no uncommitted changes, we assume user are doing post-commit -# format check, in which case we'll check the modified lines from latest commit. -# Otherwise, we'll check format of the uncommitted code only. +# format check, in which case we'll check the modified lines since last commit +# from master. Otherwise, we'll check format of the uncommitted code only. if [ -z "$uncommitted_code" ] then # Check the format of last commit - diffs=$(git diff -U0 HEAD^ | $CLANG_FORMAT_DIFF -p 1) + diffs=$(git diff -U0 $LAST_MASTER^ | $CLANG_FORMAT_DIFF -p 1) else # Check the format of uncommitted lines, diffs=$(git diff -U0 HEAD | $CLANG_FORMAT_DIFF -p 1) @@ -79,6 +83,12 @@ echo -e "Detect lines that doesn't follow the format rules:\r" echo "$diffs" | sed -e "s/\(^-.*$\)/`echo -e \"$COLOR_RED\1$COLOR_END\"`/" | sed -e "s/\(^+.*$\)/`echo -e \"$COLOR_GREEN\1$COLOR_END\"`/" + +if [[ "$OPT" == *"-DTRAVIS"* ]] +then + exit 1 +fi + echo -e "Would you like to fix the format automatically (y/n): \c" # Make sure under any mode, we can read user input. @@ -91,7 +101,12 @@ then fi # Do in-place format adjustment. -git diff -U0 HEAD^ | $CLANG_FORMAT_DIFF -i -p 1 +if [ -z "$uncommitted_code" ] +then + git diff -U0 $LAST_MASTER^ | $CLANG_FORMAT_DIFF -i -p 1 +else + git diff -U0 HEAD^ | $CLANG_FORMAT_DIFF -i -p 1 +fi echo "Files reformatted!" # Amend to last commit if user do the post-commit format check diff --git a/build_tools/gnu_parallel b/build_tools/gnu_parallel new file mode 100755 index 00000000000..abbf8f1008a --- /dev/null +++ b/build_tools/gnu_parallel @@ -0,0 +1,7936 @@ +#!/usr/bin/env perl + +# Copyright (C) 2007,2008,2009,2010,2011,2012,2013,2014 Ole Tange and +# Free Software Foundation, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, see +# or write to the Free Software Foundation, Inc., 51 Franklin St, +# Fifth Floor, Boston, MA 02110-1301 USA + +# open3 used in Job::start +use IPC::Open3; +# &WNOHANG used in reaper +use POSIX qw(:sys_wait_h setsid ceil :errno_h); +# gensym used in Job::start +use Symbol qw(gensym); +# tempfile used in Job::start +use File::Temp qw(tempfile tempdir); +# mkpath used in openresultsfile +use File::Path; +# GetOptions used in get_options_from_array +use Getopt::Long; +# Used to ensure code quality +use strict; +use File::Basename; + +if(not $ENV{HOME}) { + # $ENV{HOME} is sometimes not set if called from PHP + ::warning("\$HOME not set. Using /tmp\n"); + $ENV{HOME} = "/tmp"; +} + +save_stdin_stdout_stderr(); +save_original_signal_handler(); +parse_options(); +::debug("init", "Open file descriptors: ", join(" ",keys %Global::fd), "\n"); +my $number_of_args; +if($Global::max_number_of_args) { + $number_of_args=$Global::max_number_of_args; +} elsif ($opt::X or $opt::m or $opt::xargs) { + $number_of_args = undef; +} else { + $number_of_args = 1; +} + +my @command; +@command = @ARGV; + +my @fhlist; +if($opt::pipepart) { + @fhlist = map { open_or_exit($_) } "/dev/null"; +} else { + @fhlist = map { open_or_exit($_) } @opt::a; + if(not @fhlist and not $opt::pipe) { + @fhlist = (*STDIN); + } +} + +if($opt::skip_first_line) { + # Skip the first line for the first file handle + my $fh = $fhlist[0]; + <$fh>; +} +if($opt::header and not $opt::pipe) { + my $fh = $fhlist[0]; + # split with colsep or \t + # $header force $colsep = \t if undef? + my $delimiter = $opt::colsep; + $delimiter ||= "\$"; + my $id = 1; + for my $fh (@fhlist) { + my $line = <$fh>; + chomp($line); + ::debug("init", "Delimiter: '$delimiter'"); + for my $s (split /$delimiter/o, $line) { + ::debug("init", "Colname: '$s'"); + # Replace {colname} with {2} + # TODO accept configurable short hands + # TODO how to deal with headers in {=...=} + for(@command) { + s:\{$s(|/|//|\.|/\.)\}:\{$id$1\}:g; + } + $Global::input_source_header{$id} = $s; + $id++; + } + } +} else { + my $id = 1; + for my $fh (@fhlist) { + $Global::input_source_header{$id} = $id; + $id++; + } +} + +if($opt::filter_hosts and (@opt::sshlogin or @opt::sshloginfile)) { + # Parallel check all hosts are up. Remove hosts that are down + filter_hosts(); +} + +if($opt::nonall or $opt::onall) { + onall(@command); + wait_and_exit(min(undef_as_zero($Global::exitstatus),254)); +} + +# TODO --transfer foo/./bar --cleanup +# multiple --transfer and --basefile with different /./ + +$Global::JobQueue = JobQueue->new( + \@command,\@fhlist,$Global::ContextReplace,$number_of_args,\@Global::ret_files); + +if($opt::eta or $opt::bar) { + # Count the number of jobs before starting any + $Global::JobQueue->total_jobs(); +} +if($opt::pipepart) { + @Global::cat_partials = map { pipe_part_files($_) } @opt::a; + # Unget the command as many times as there are parts + $Global::JobQueue->{'commandlinequeue'}->unget( + map { $Global::JobQueue->{'commandlinequeue'}->get() } @Global::cat_partials + ); +} +for my $sshlogin (values %Global::host) { + $sshlogin->max_jobs_running(); +} + +init_run_jobs(); +my $sem; +if($Global::semaphore) { + $sem = acquire_semaphore(); +} +$SIG{TERM} = \&start_no_new_jobs; + +start_more_jobs(); +if(not $opt::pipepart) { + if($opt::pipe) { + spreadstdin(); + } +} +::debug("init", "Start draining\n"); +drain_job_queue(); +::debug("init", "Done draining\n"); +reaper(); +::debug("init", "Done reaping\n"); +if($opt::pipe and @opt::a) { + for my $job (@Global::tee_jobs) { + unlink $job->fh(2,"name"); + $job->set_fh(2,"name",""); + $job->print(); + unlink $job->fh(1,"name"); + } +} +::debug("init", "Cleaning\n"); +cleanup(); +if($Global::semaphore) { + $sem->release(); +} +for(keys %Global::sshmaster) { + kill "TERM", $_; +} +::debug("init", "Halt\n"); +if($opt::halt_on_error) { + wait_and_exit($Global::halt_on_error_exitstatus); +} else { + wait_and_exit(min(undef_as_zero($Global::exitstatus),254)); +} + +sub __PIPE_MODE__ {} + +sub pipe_part_files { + # Input: + # $file = the file to read + # Returns: + # @commands that will cat_partial each part + my ($file) = @_; + my $buf = ""; + my $header = find_header(\$buf,open_or_exit($file)); + # find positions + my @pos = find_split_positions($file,$opt::blocksize,length $header); + # Make @cat_partials + my @cat_partials = (); + for(my $i=0; $i<$#pos; $i++) { + push @cat_partials, cat_partial($file, 0, length($header), $pos[$i], $pos[$i+1]); + } + # Remote exec should look like: + # ssh -oLogLevel=quiet lo 'eval `echo $SHELL | grep "/t\{0,1\}csh" > /dev/null && echo setenv PARALLEL_SEQ '$PARALLEL_SEQ'\; setenv PARALLEL_PID '$PARALLEL_PID' || echo PARALLEL_SEQ='$PARALLEL_SEQ'\;export PARALLEL_SEQ\; PARALLEL_PID='$PARALLEL_PID'\;export PARALLEL_PID` ;' tty\ \>/dev/null\ \&\&\ stty\ isig\ -onlcr\ -echo\;echo\ \$SHELL\ \|\ grep\ \"/t\\\{0,1\\\}csh\"\ \>\ /dev/null\ \&\&\ setenv\ FOO\ /tmp/foo\ \|\|\ export\ FOO=/tmp/foo\; \(wc\ -\ \$FOO\) + # ssh -tt not allowed. Remote will die due to broken pipe anyway. + # TODO test remote with --fifo / --cat + return @cat_partials; +} + +sub find_header { + # Input: + # $buf_ref = reference to read-in buffer + # $fh = filehandle to read from + # Uses: + # $opt::header + # $opt::blocksize + # Returns: + # $header string + my ($buf_ref, $fh) = @_; + my $header = ""; + if($opt::header) { + if($opt::header eq ":") { $opt::header = "(.*\n)"; } + # Number = number of lines + $opt::header =~ s/^(\d+)$/"(.*\n)"x$1/e; + while(read($fh,substr($$buf_ref,length $$buf_ref,0),$opt::blocksize)) { + if($$buf_ref=~s/^($opt::header)//) { + $header = $1; + last; + } + } + } + return $header; +} + +sub find_split_positions { + # Input: + # $file = the file to read + # $block = (minimal) --block-size of each chunk + # $headerlen = length of header to be skipped + # Uses: + # $opt::recstart + # $opt::recend + # Returns: + # @positions of block start/end + my($file, $block, $headerlen) = @_; + my $size = -s $file; + $block = int $block; + # The optimal dd blocksize for mint, redhat, solaris, openbsd = 2^17..2^20 + # The optimal dd blocksize for freebsd = 2^15..2^17 + my $dd_block_size = 131072; # 2^17 + my @pos; + my ($recstart,$recend) = recstartrecend(); + my $recendrecstart = $recend.$recstart; + my $fh = ::open_or_exit($file); + push(@pos,$headerlen); + for(my $pos = $block+$headerlen; $pos < $size; $pos += $block) { + my $buf; + seek($fh, $pos, 0) || die; + while(read($fh,substr($buf,length $buf,0),$dd_block_size)) { + if($opt::regexp) { + # If match /$recend$recstart/ => Record position + if($buf =~ /(.*$recend)$recstart/os) { + my $i = length($1); + push(@pos,$pos+$i); + # Start looking for next record _after_ this match + $pos += $i; + last; + } + } else { + # If match $recend$recstart => Record position + my $i = index($buf,$recendrecstart); + if($i != -1) { + push(@pos,$pos+$i); + # Start looking for next record _after_ this match + $pos += $i; + last; + } + } + } + } + push(@pos,$size); + close $fh; + return @pos; +} + +sub cat_partial { + # Input: + # $file = the file to read + # ($start, $end, [$start2, $end2, ...]) = start byte, end byte + # Returns: + # Efficient perl command to copy $start..$end, $start2..$end2, ... to stdout + my($file, @start_end) = @_; + my($start, $i); + # Convert start_end to start_len + my @start_len = map { if(++$i % 2) { $start = $_; } else { $_-$start } } @start_end; + return "<". shell_quote_scalar($file) . + q{ perl -e 'while(@ARGV) { sysseek(STDIN,shift,0) || die; $left = shift; while($read = sysread(STDIN,$buf, ($left > 32768 ? 32768 : $left))){ $left -= $read; syswrite(STDOUT,$buf); } }' } . + " @start_len"; +} + +sub spreadstdin { + # read a record + # Spawn a job and print the record to it. + # Uses: + # $opt::blocksize + # STDIN + # $opr::r + # $Global::max_lines + # $Global::max_number_of_args + # $opt::regexp + # $Global::start_no_new_jobs + # $opt::roundrobin + # %Global::running + + my $buf = ""; + my ($recstart,$recend) = recstartrecend(); + my $recendrecstart = $recend.$recstart; + my $chunk_number = 1; + my $one_time_through; + my $blocksize = $opt::blocksize; + my $in = *STDIN; + my $header = find_header(\$buf,$in); + while(1) { + my $anything_written = 0; + if(not read($in,substr($buf,length $buf,0),$blocksize)) { + # End-of-file + $chunk_number != 1 and last; + # Force the while-loop once if everything was read by header reading + $one_time_through++ and last; + } + if($opt::r) { + # Remove empty lines + $buf =~ s/^\s*\n//gm; + if(length $buf == 0) { + next; + } + } + if($Global::max_lines and not $Global::max_number_of_args) { + # Read n-line records + my $n_lines = $buf =~ tr/\n/\n/; + my $last_newline_pos = rindex($buf,"\n"); + while($n_lines % $Global::max_lines) { + $n_lines--; + $last_newline_pos = rindex($buf,"\n",$last_newline_pos-1); + } + # Chop at $last_newline_pos as that is where n-line record ends + $anything_written += + write_record_to_pipe($chunk_number++,\$header,\$buf, + $recstart,$recend,$last_newline_pos+1); + substr($buf,0,$last_newline_pos+1) = ""; + } elsif($opt::regexp) { + if($Global::max_number_of_args) { + # -N => (start..*?end){n} + # -L -N => (start..*?end){n*l} + my $read_n_lines = $Global::max_number_of_args * ($Global::max_lines || 1); + while($buf =~ s/((?:$recstart.*?$recend){$read_n_lines})($recstart.*)$/$2/os) { + # Copy to modifiable variable + my $b = $1; + $anything_written += + write_record_to_pipe($chunk_number++,\$header,\$b, + $recstart,$recend,length $1); + } + } else { + # Find the last recend-recstart in $buf + if($buf =~ s/(.*$recend)($recstart.*?)$/$2/os) { + # Copy to modifiable variable + my $b = $1; + $anything_written += + write_record_to_pipe($chunk_number++,\$header,\$b, + $recstart,$recend,length $1); + } + } + } else { + if($Global::max_number_of_args) { + # -N => (start..*?end){n} + my $i = 0; + my $read_n_lines = $Global::max_number_of_args * ($Global::max_lines || 1); + while(($i = nindex(\$buf,$recendrecstart,$read_n_lines)) != -1) { + $i += length $recend; # find the actual splitting location + $anything_written += + write_record_to_pipe($chunk_number++,\$header,\$buf, + $recstart,$recend,$i); + substr($buf,0,$i) = ""; + } + } else { + # Find the last recend-recstart in $buf + my $i = rindex($buf,$recendrecstart); + if($i != -1) { + $i += length $recend; # find the actual splitting location + $anything_written += + write_record_to_pipe($chunk_number++,\$header,\$buf, + $recstart,$recend,$i); + substr($buf,0,$i) = ""; + } + } + } + if(not $anything_written and not eof($in)) { + # Nothing was written - maybe the block size < record size? + # Increase blocksize exponentially + my $old_blocksize = $blocksize; + $blocksize = ceil($blocksize * 1.3 + 1); + ::warning("A record was longer than $old_blocksize. " . + "Increasing to --blocksize $blocksize\n"); + } + } + ::debug("init", "Done reading input\n"); + + # If there is anything left in the buffer write it + substr($buf,0,0) = ""; + write_record_to_pipe($chunk_number++,\$header,\$buf,$recstart,$recend,length $buf); + + $Global::start_no_new_jobs ||= 1; + if($opt::roundrobin) { + for my $job (values %Global::running) { + close $job->fh(0,"w"); + } + my %incomplete_jobs = %Global::running; + my $sleep = 1; + while(keys %incomplete_jobs) { + my $something_written = 0; + for my $pid (keys %incomplete_jobs) { + my $job = $incomplete_jobs{$pid}; + if($job->stdin_buffer_length()) { + $something_written += $job->non_block_write(); + } else { + delete $incomplete_jobs{$pid} + } + } + if($something_written) { + $sleep = $sleep/2+0.001; + } + $sleep = ::reap_usleep($sleep); + } + } +} + +sub recstartrecend { + # Uses: + # $opt::recstart + # $opt::recend + # Returns: + # $recstart,$recend with default values and regexp conversion + my($recstart,$recend); + if(defined($opt::recstart) and defined($opt::recend)) { + # If both --recstart and --recend is given then both must match + $recstart = $opt::recstart; + $recend = $opt::recend; + } elsif(defined($opt::recstart)) { + # If --recstart is given it must match start of record + $recstart = $opt::recstart; + $recend = ""; + } elsif(defined($opt::recend)) { + # If --recend is given then it must match end of record + $recstart = ""; + $recend = $opt::recend; + } + + if($opt::regexp) { + # If $recstart/$recend contains '|' this should only apply to the regexp + $recstart = "(?:".$recstart.")"; + $recend = "(?:".$recend.")"; + } else { + # $recstart/$recend = printf strings (\n) + $recstart =~ s/\\([0rnt\'\"\\])/"qq|\\$1|"/gee; + $recend =~ s/\\([0rnt\'\"\\])/"qq|\\$1|"/gee; + } + return ($recstart,$recend); +} + +sub nindex { + # See if string is in buffer N times + # Returns: + # the position where the Nth copy is found + my ($buf_ref, $str, $n) = @_; + my $i = 0; + for(1..$n) { + $i = index($$buf_ref,$str,$i+1); + if($i == -1) { last } + } + return $i; +} + +{ + my @robin_queue; + + sub round_robin_write { + # Input: + # $header_ref = ref to $header string + # $block_ref = ref to $block to be written + # $recstart = record start string + # $recend = record end string + # $endpos = end position of $block + # Uses: + # %Global::running + my ($header_ref,$block_ref,$recstart,$recend,$endpos) = @_; + my $something_written = 0; + my $block_passed = 0; + my $sleep = 1; + while(not $block_passed) { + # Continue flushing existing buffers + # until one is empty and a new block is passed + # Make a queue to spread the blocks evenly + if(not @robin_queue) { + push @robin_queue, values %Global::running; + } + while(my $job = shift @robin_queue) { + if($job->stdin_buffer_length() > 0) { + $something_written += $job->non_block_write(); + } else { + $job->set_stdin_buffer($header_ref,$block_ref,$endpos,$recstart,$recend); + $block_passed = 1; + $job->set_virgin(0); + $something_written += $job->non_block_write(); + last; + } + } + $sleep = ::reap_usleep($sleep); + } + return $something_written; + } +} + +sub write_record_to_pipe { + # Fork then + # Write record from pos 0 .. $endpos to pipe + # Input: + # $chunk_number = sequence number - to see if already run + # $header_ref = reference to header string to prepend + # $record_ref = reference to record to write + # $recstart = start string of record + # $recend = end string of record + # $endpos = position in $record_ref where record ends + # Uses: + # $Global::job_already_run + # $opt::roundrobin + # @Global::virgin_jobs + # Returns: + # Number of chunks written (0 or 1) + my ($chunk_number,$header_ref,$record_ref,$recstart,$recend,$endpos) = @_; + if($endpos == 0) { return 0; } + if(vec($Global::job_already_run,$chunk_number,1)) { return 1; } + if($opt::roundrobin) { + return round_robin_write($header_ref,$record_ref,$recstart,$recend,$endpos); + } + # If no virgin found, backoff + my $sleep = 0.0001; # 0.01 ms - better performance on highend + while(not @Global::virgin_jobs) { + ::debug("pipe", "No virgin jobs"); + $sleep = ::reap_usleep($sleep); + # Jobs may not be started because of loadavg + # or too little time between each ssh login. + start_more_jobs(); + } + my $job = shift @Global::virgin_jobs; + # Job is no longer virgin + $job->set_virgin(0); + if(fork()) { + # Skip + } else { + # Chop of at $endpos as we do not know how many rec_sep will + # be removed. + substr($$record_ref,$endpos,length $$record_ref) = ""; + # Remove rec_sep + if($opt::remove_rec_sep) { + Job::remove_rec_sep($record_ref,$recstart,$recend); + } + $job->write($header_ref); + $job->write($record_ref); + close $job->fh(0,"w"); + exit(0); + } + close $job->fh(0,"w"); + return 1; +} + +sub __SEM_MODE__ {} + +sub acquire_semaphore { + # Acquires semaphore. If needed: spawns to the background + # Uses: + # @Global::host + # Returns: + # The semaphore to be released when jobs is complete + $Global::host{':'} = SSHLogin->new(":"); + my $sem = Semaphore->new($Semaphore::name,$Global::host{':'}->max_jobs_running()); + $sem->acquire(); + if($Semaphore::fg) { + # skip + } else { + # If run in the background, the PID will change + # therefore release and re-acquire the semaphore + $sem->release(); + if(fork()) { + exit(0); + } else { + # child + # Get a semaphore for this pid + ::die_bug("Can't start a new session: $!") if setsid() == -1; + $sem = Semaphore->new($Semaphore::name,$Global::host{':'}->max_jobs_running()); + $sem->acquire(); + } + } + return $sem; +} + +sub __PARSE_OPTIONS__ {} + +sub options_hash { + # Returns: + # %hash = the GetOptions config + return + ("debug|D=s" => \$opt::D, + "xargs" => \$opt::xargs, + "m" => \$opt::m, + "X" => \$opt::X, + "v" => \@opt::v, + "joblog=s" => \$opt::joblog, + "results|result|res=s" => \$opt::results, + "resume" => \$opt::resume, + "resume-failed|resumefailed" => \$opt::resume_failed, + "silent" => \$opt::silent, + #"silent-error|silenterror" => \$opt::silent_error, + "keep-order|keeporder|k" => \$opt::keeporder, + "group" => \$opt::group, + "g" => \$opt::retired, + "ungroup|u" => \$opt::ungroup, + "linebuffer|linebuffered|line-buffer|line-buffered" => \$opt::linebuffer, + "tmux" => \$opt::tmux, + "null|0" => \$opt::0, + "quote|q" => \$opt::q, + # Replacement strings + "parens=s" => \$opt::parens, + "rpl=s" => \@opt::rpl, + "plus" => \$opt::plus, + "I=s" => \$opt::I, + "extensionreplace|er=s" => \$opt::U, + "U=s" => \$opt::retired, + "basenamereplace|bnr=s" => \$opt::basenamereplace, + "dirnamereplace|dnr=s" => \$opt::dirnamereplace, + "basenameextensionreplace|bner=s" => \$opt::basenameextensionreplace, + "seqreplace=s" => \$opt::seqreplace, + "slotreplace=s" => \$opt::slotreplace, + "jobs|j=s" => \$opt::jobs, + "delay=f" => \$opt::delay, + "sshdelay=f" => \$opt::sshdelay, + "load=s" => \$opt::load, + "noswap" => \$opt::noswap, + "max-line-length-allowed" => \$opt::max_line_length_allowed, + "number-of-cpus" => \$opt::number_of_cpus, + "number-of-cores" => \$opt::number_of_cores, + "use-cpus-instead-of-cores" => \$opt::use_cpus_instead_of_cores, + "shellquote|shell_quote|shell-quote" => \$opt::shellquote, + "nice=i" => \$opt::nice, + "timeout=s" => \$opt::timeout, + "tag" => \$opt::tag, + "tagstring|tag-string=s" => \$opt::tagstring, + "onall" => \$opt::onall, + "nonall" => \$opt::nonall, + "filter-hosts|filterhosts|filter-host" => \$opt::filter_hosts, + "sshlogin|S=s" => \@opt::sshlogin, + "sshloginfile|slf=s" => \@opt::sshloginfile, + "controlmaster|M" => \$opt::controlmaster, + "return=s" => \@opt::return, + "trc=s" => \@opt::trc, + "transfer" => \$opt::transfer, + "cleanup" => \$opt::cleanup, + "basefile|bf=s" => \@opt::basefile, + "B=s" => \$opt::retired, + "ctrlc|ctrl-c" => \$opt::ctrlc, + "noctrlc|no-ctrlc|no-ctrl-c" => \$opt::noctrlc, + "workdir|work-dir|wd=s" => \$opt::workdir, + "W=s" => \$opt::retired, + "tmpdir=s" => \$opt::tmpdir, + "tempdir=s" => \$opt::tmpdir, + "use-compress-program|compress-program=s" => \$opt::compress_program, + "use-decompress-program|decompress-program=s" => \$opt::decompress_program, + "compress" => \$opt::compress, + "tty" => \$opt::tty, + "T" => \$opt::retired, + "halt-on-error|halt=s" => \$opt::halt_on_error, + "H=i" => \$opt::retired, + "retries=i" => \$opt::retries, + "dry-run|dryrun" => \$opt::dryrun, + "progress" => \$opt::progress, + "eta" => \$opt::eta, + "bar" => \$opt::bar, + "arg-sep|argsep=s" => \$opt::arg_sep, + "arg-file-sep|argfilesep=s" => \$opt::arg_file_sep, + "trim=s" => \$opt::trim, + "env=s" => \@opt::env, + "recordenv|record-env" => \$opt::record_env, + "plain" => \$opt::plain, + "profile|J=s" => \@opt::profile, + "pipe|spreadstdin" => \$opt::pipe, + "robin|round-robin|roundrobin" => \$opt::roundrobin, + "recstart=s" => \$opt::recstart, + "recend=s" => \$opt::recend, + "regexp|regex" => \$opt::regexp, + "remove-rec-sep|removerecsep|rrs" => \$opt::remove_rec_sep, + "files|output-as-files|outputasfiles" => \$opt::files, + "block|block-size|blocksize=s" => \$opt::blocksize, + "tollef" => \$opt::retired, + "gnu" => \$opt::gnu, + "xapply" => \$opt::xapply, + "bibtex" => \$opt::bibtex, + "nn|nonotice|no-notice" => \$opt::no_notice, + # xargs-compatibility - implemented, man, testsuite + "max-procs|P=s" => \$opt::jobs, + "delimiter|d=s" => \$opt::d, + "max-chars|s=i" => \$opt::max_chars, + "arg-file|a=s" => \@opt::a, + "no-run-if-empty|r" => \$opt::r, + "replace|i:s" => \$opt::i, + "E=s" => \$opt::eof, + "eof|e:s" => \$opt::eof, + "max-args|n=i" => \$opt::max_args, + "max-replace-args|N=i" => \$opt::max_replace_args, + "colsep|col-sep|C=s" => \$opt::colsep, + "help|h" => \$opt::help, + "L=f" => \$opt::L, + "max-lines|l:f" => \$opt::max_lines, + "interactive|p" => \$opt::p, + "verbose|t" => \$opt::verbose, + "version|V" => \$opt::version, + "minversion|min-version=i" => \$opt::minversion, + "show-limits|showlimits" => \$opt::show_limits, + "exit|x" => \$opt::x, + # Semaphore + "semaphore" => \$opt::semaphore, + "semaphoretimeout=i" => \$opt::semaphoretimeout, + "semaphorename|id=s" => \$opt::semaphorename, + "fg" => \$opt::fg, + "bg" => \$opt::bg, + "wait" => \$opt::wait, + # Shebang #!/usr/bin/parallel --shebang + "shebang|hashbang" => \$opt::shebang, + "internal-pipe-means-argfiles" => \$opt::internal_pipe_means_argfiles, + "Y" => \$opt::retired, + "skip-first-line" => \$opt::skip_first_line, + "header=s" => \$opt::header, + "cat" => \$opt::cat, + "fifo" => \$opt::fifo, + "pipepart|pipe-part" => \$opt::pipepart, + "hgrp|hostgroup|hostgroups" => \$opt::hostgroups, + ); +} + +sub get_options_from_array { + # Run GetOptions on @array + # Input: + # $array_ref = ref to @ARGV to parse + # @keep_only = Keep only these options + # Uses: + # @ARGV + # Returns: + # true if parsing worked + # false if parsing failed + # @$array_ref is changed + my ($array_ref, @keep_only) = @_; + if(not @$array_ref) { + # Empty array: No need to look more at that + return 1; + } + # A bit of shuffling of @ARGV needed as GetOptionsFromArray is not + # supported everywhere + my @save_argv; + my $this_is_ARGV = (\@::ARGV == $array_ref); + if(not $this_is_ARGV) { + @save_argv = @::ARGV; + @::ARGV = @{$array_ref}; + } + # If @keep_only set: Ignore all values except @keep_only + my %options = options_hash(); + if(@keep_only) { + my (%keep,@dummy); + @keep{@keep_only} = @keep_only; + for my $k (grep { not $keep{$_} } keys %options) { + # Store the value of the option in @dummy + $options{$k} = \@dummy; + } + } + my $retval = GetOptions(%options); + if(not $this_is_ARGV) { + @{$array_ref} = @::ARGV; + @::ARGV = @save_argv; + } + return $retval; +} + +sub parse_options { + # Returns: N/A + # Defaults: + $Global::version = 20141122; + $Global::progname = 'parallel'; + $Global::infinity = 2**31; + $Global::debug = 0; + $Global::verbose = 0; + $Global::quoting = 0; + # Read only table with default --rpl values + %Global::replace = + ( + '{}' => '', + '{#}' => '1 $_=$job->seq()', + '{%}' => '1 $_=$job->slot()', + '{/}' => 's:.*/::', + '{//}' => '$Global::use{"File::Basename"} ||= eval "use File::Basename; 1;"; $_ = dirname($_);', + '{/.}' => 's:.*/::; s:\.[^/.]+$::;', + '{.}' => 's:\.[^/.]+$::', + ); + %Global::plus = + ( + # {} = {+/}/{/} + # = {.}.{+.} = {+/}/{/.}.{+.} + # = {..}.{+..} = {+/}/{/..}.{+..} + # = {...}.{+...} = {+/}/{/...}.{+...} + '{+/}' => 's:/[^/]*$::', + '{+.}' => 's:.*\.::', + '{+..}' => 's:.*\.([^.]*\.):$1:', + '{+...}' => 's:.*\.([^.]*\.[^.]*\.):$1:', + '{..}' => 's:\.[^/.]+$::; s:\.[^/.]+$::', + '{...}' => 's:\.[^/.]+$::; s:\.[^/.]+$::; s:\.[^/.]+$::', + '{/..}' => 's:.*/::; s:\.[^/.]+$::; s:\.[^/.]+$::', + '{/...}' => 's:.*/::; s:\.[^/.]+$::; s:\.[^/.]+$::; s:\.[^/.]+$::', + ); + # Modifiable copy of %Global::replace + %Global::rpl = %Global::replace; + $Global::parens = "{==}"; + $/="\n"; + $Global::ignore_empty = 0; + $Global::interactive = 0; + $Global::stderr_verbose = 0; + $Global::default_simultaneous_sshlogins = 9; + $Global::exitstatus = 0; + $Global::halt_on_error_exitstatus = 0; + $Global::arg_sep = ":::"; + $Global::arg_file_sep = "::::"; + $Global::trim = 'n'; + $Global::max_jobs_running = 0; + $Global::job_already_run = ''; + $ENV{'TMPDIR'} ||= "/tmp"; + + @ARGV=read_options(); + + if(@opt::v) { $Global::verbose = $#opt::v+1; } # Convert -v -v to v=2 + $Global::debug = $opt::D; + $Global::shell = $ENV{'PARALLEL_SHELL'} || parent_shell($$) || $ENV{'SHELL'} || "/bin/sh"; + if(defined $opt::X) { $Global::ContextReplace = 1; } + if(defined $opt::silent) { $Global::verbose = 0; } + if(defined $opt::0) { $/ = "\0"; } + if(defined $opt::d) { my $e="sprintf \"$opt::d\""; $/ = eval $e; } + if(defined $opt::p) { $Global::interactive = $opt::p; } + if(defined $opt::q) { $Global::quoting = 1; } + if(defined $opt::r) { $Global::ignore_empty = 1; } + if(defined $opt::verbose) { $Global::stderr_verbose = 1; } + # Deal with --rpl + sub rpl { + # Modify %Global::rpl + # Replace $old with $new + my ($old,$new) = @_; + if($old ne $new) { + $Global::rpl{$new} = $Global::rpl{$old}; + delete $Global::rpl{$old}; + } + } + if(defined $opt::parens) { $Global::parens = $opt::parens; } + my $parenslen = 0.5*length $Global::parens; + $Global::parensleft = substr($Global::parens,0,$parenslen); + $Global::parensright = substr($Global::parens,$parenslen); + if(defined $opt::plus) { %Global::rpl = (%Global::plus,%Global::rpl); } + if(defined $opt::I) { rpl('{}',$opt::I); } + if(defined $opt::U) { rpl('{.}',$opt::U); } + if(defined $opt::i and $opt::i) { rpl('{}',$opt::i); } + if(defined $opt::basenamereplace) { rpl('{/}',$opt::basenamereplace); } + if(defined $opt::dirnamereplace) { rpl('{//}',$opt::dirnamereplace); } + if(defined $opt::seqreplace) { rpl('{#}',$opt::seqreplace); } + if(defined $opt::slotreplace) { rpl('{%}',$opt::slotreplace); } + if(defined $opt::basenameextensionreplace) { + rpl('{/.}',$opt::basenameextensionreplace); + } + for(@opt::rpl) { + # Create $Global::rpl entries for --rpl options + # E.g: "{..} s:\.[^.]+$:;s:\.[^.]+$:;" + my ($shorthand,$long) = split/ /,$_,2; + $Global::rpl{$shorthand} = $long; + } + if(defined $opt::eof) { $Global::end_of_file_string = $opt::eof; } + if(defined $opt::max_args) { $Global::max_number_of_args = $opt::max_args; } + if(defined $opt::timeout) { $Global::timeoutq = TimeoutQueue->new($opt::timeout); } + if(defined $opt::tmpdir) { $ENV{'TMPDIR'} = $opt::tmpdir; } + if(defined $opt::help) { die_usage(); } + if(defined $opt::colsep) { $Global::trim = 'lr'; } + if(defined $opt::header) { $opt::colsep = defined $opt::colsep ? $opt::colsep : "\t"; } + if(defined $opt::trim) { $Global::trim = $opt::trim; } + if(defined $opt::arg_sep) { $Global::arg_sep = $opt::arg_sep; } + if(defined $opt::arg_file_sep) { $Global::arg_file_sep = $opt::arg_file_sep; } + if(defined $opt::number_of_cpus) { print SSHLogin::no_of_cpus(),"\n"; wait_and_exit(0); } + if(defined $opt::number_of_cores) { + print SSHLogin::no_of_cores(),"\n"; wait_and_exit(0); + } + if(defined $opt::max_line_length_allowed) { + print Limits::Command::real_max_length(),"\n"; wait_and_exit(0); + } + if(defined $opt::version) { version(); wait_and_exit(0); } + if(defined $opt::bibtex) { bibtex(); wait_and_exit(0); } + if(defined $opt::record_env) { record_env(); wait_and_exit(0); } + if(defined $opt::show_limits) { show_limits(); } + if(@opt::sshlogin) { @Global::sshlogin = @opt::sshlogin; } + if(@opt::sshloginfile) { read_sshloginfiles(@opt::sshloginfile); } + if(@opt::return) { push @Global::ret_files, @opt::return; } + if(not defined $opt::recstart and + not defined $opt::recend) { $opt::recend = "\n"; } + if(not defined $opt::blocksize) { $opt::blocksize = "1M"; } + $opt::blocksize = multiply_binary_prefix($opt::blocksize); + if(defined $opt::controlmaster) { $opt::noctrlc = 1; } + if(defined $opt::semaphore) { $Global::semaphore = 1; } + if(defined $opt::semaphoretimeout) { $Global::semaphore = 1; } + if(defined $opt::semaphorename) { $Global::semaphore = 1; } + if(defined $opt::fg) { $Global::semaphore = 1; } + if(defined $opt::bg) { $Global::semaphore = 1; } + if(defined $opt::wait) { $Global::semaphore = 1; } + if(defined $opt::halt_on_error and + $opt::halt_on_error=~/%/) { $opt::halt_on_error /= 100; } + if(defined $opt::timeout and $opt::timeout !~ /^\d+(\.\d+)?%?$/) { + ::error("--timeout must be seconds or percentage\n"); + wait_and_exit(255); + } + if(defined $opt::minversion) { + print $Global::version,"\n"; + if($Global::version < $opt::minversion) { + wait_and_exit(255); + } else { + wait_and_exit(0); + } + } + if(not defined $opt::delay) { + # Set --delay to --sshdelay if not set + $opt::delay = $opt::sshdelay; + } + if($opt::compress_program) { + $opt::compress = 1; + $opt::decompress_program ||= $opt::compress_program." -dc"; + } + if($opt::compress) { + my ($compress, $decompress) = find_compression_program(); + $opt::compress_program ||= $compress; + $opt::decompress_program ||= $decompress; + } + if(defined $opt::nonall) { + # Append a dummy empty argument + push @ARGV, $Global::arg_sep, ""; + } + if(defined $opt::tty) { + # Defaults for --tty: -j1 -u + # Can be overridden with -jXXX -g + if(not defined $opt::jobs) { + $opt::jobs = 1; + } + if(not defined $opt::group) { + $opt::ungroup = 0; + } + } + if(@opt::trc) { + push @Global::ret_files, @opt::trc; + $opt::transfer = 1; + $opt::cleanup = 1; + } + if(defined $opt::max_lines) { + if($opt::max_lines eq "-0") { + # -l -0 (swallowed -0) + $opt::max_lines = 1; + $opt::0 = 1; + $/ = "\0"; + } elsif ($opt::max_lines == 0) { + # If not given (or if 0 is given) => 1 + $opt::max_lines = 1; + } + $Global::max_lines = $opt::max_lines; + if(not $opt::pipe) { + # --pipe -L means length of record - not max_number_of_args + $Global::max_number_of_args ||= $Global::max_lines; + } + } + + # Read more than one arg at a time (-L, -N) + if(defined $opt::L) { + $Global::max_lines = $opt::L; + if(not $opt::pipe) { + # --pipe -L means length of record - not max_number_of_args + $Global::max_number_of_args ||= $Global::max_lines; + } + } + if(defined $opt::max_replace_args) { + $Global::max_number_of_args = $opt::max_replace_args; + $Global::ContextReplace = 1; + } + if((defined $opt::L or defined $opt::max_replace_args) + and + not ($opt::xargs or $opt::m)) { + $Global::ContextReplace = 1; + } + if(defined $opt::tag and not defined $opt::tagstring) { + $opt::tagstring = "\257<\257>"; # Default = {} + } + if(defined $opt::pipepart and + (defined $opt::L or defined $opt::max_lines + or defined $opt::max_replace_args)) { + ::error("--pipepart is incompatible with --max-replace-args, ", + "--max-lines, and -L.\n"); + wait_and_exit(255); + } + if(grep /^$Global::arg_sep$|^$Global::arg_file_sep$/o, @ARGV) { + # Deal with ::: and :::: + @ARGV=read_args_from_command_line(); + } + + # Semaphore defaults + # Must be done before computing number of processes and max_line_length + # because when running as a semaphore GNU Parallel does not read args + $Global::semaphore ||= ($0 =~ m:(^|/)sem$:); # called as 'sem' + if($Global::semaphore) { + # A semaphore does not take input from neither stdin nor file + @opt::a = ("/dev/null"); + push(@Global::unget_argv, [Arg->new("")]); + $Semaphore::timeout = $opt::semaphoretimeout || 0; + if(defined $opt::semaphorename) { + $Semaphore::name = $opt::semaphorename; + } else { + $Semaphore::name = `tty`; + chomp $Semaphore::name; + } + $Semaphore::fg = $opt::fg; + $Semaphore::wait = $opt::wait; + $Global::default_simultaneous_sshlogins = 1; + if(not defined $opt::jobs) { + $opt::jobs = 1; + } + if($Global::interactive and $opt::bg) { + ::error("Jobs running in the ". + "background cannot be interactive.\n"); + ::wait_and_exit(255); + } + } + if(defined $opt::eta) { + $opt::progress = $opt::eta; + } + if(defined $opt::bar) { + $opt::progress = $opt::bar; + } + if(defined $opt::retired) { + ::error("-g has been retired. Use --group.\n"); + ::error("-B has been retired. Use --bf.\n"); + ::error("-T has been retired. Use --tty.\n"); + ::error("-U has been retired. Use --er.\n"); + ::error("-W has been retired. Use --wd.\n"); + ::error("-Y has been retired. Use --shebang.\n"); + ::error("-H has been retired. Use --halt.\n"); + ::error("--tollef has been retired. Use -u -q --arg-sep -- and --load for -l.\n"); + ::wait_and_exit(255); + } + citation_notice(); + + parse_sshlogin(); + parse_env_var(); + + if(remote_hosts() and ($opt::X or $opt::m or $opt::xargs)) { + # As we do not know the max line length on the remote machine + # long commands generated by xargs may fail + # If opt_N is set, it is probably safe + ::warning("Using -X or -m with --sshlogin may fail.\n"); + } + + if(not defined $opt::jobs) { + $opt::jobs = "100%"; + } + open_joblog(); +} + +sub env_quote { + # Input: + # $v = value to quote + # Returns: + # $v = value quoted as environment variable + my $v = $_[0]; + $v =~ s/([\\])/\\$1/g; + $v =~ s/([\[\] \#\'\&\<\>\(\)\;\{\}\t\"\$\`\*\174\!\?\~])/\\$1/g; + $v =~ s/\n/"\n"/g; + return $v; +} + +sub record_env { + # Record current %ENV-keys in ~/.parallel/ignored_vars + # Returns: N/A + my $ignore_filename = $ENV{'HOME'} . "/.parallel/ignored_vars"; + if(open(my $vars_fh, ">", $ignore_filename)) { + print $vars_fh map { $_,"\n" } keys %ENV; + } else { + ::error("Cannot write to $ignore_filename\n"); + ::wait_and_exit(255); + } +} + +sub parse_env_var { + # Parse --env and set $Global::envvar, $Global::envwarn and $Global::envvarlen + # + # Bash functions must be parsed to export them remotely + # Pre-shellshock style bash function: + # myfunc=() {... + # Post-shellshock style bash function: + # BASH_FUNC_myfunc()=() {... + # + # Uses: + # $Global::envvar = eval string that will set variables in both bash and csh + # $Global::envwarn = If functions are used: Give warning in csh + # $Global::envvarlen = length of $Global::envvar + # @opt::env + # $Global::shell + # %ENV + # Returns: N/A + $Global::envvar = ""; + $Global::envwarn = ""; + my @vars = ('parallel_bash_environment'); + for my $varstring (@opt::env) { + # Split up --env VAR1,VAR2 + push @vars, split /,/, $varstring; + } + if(grep { /^_$/ } @vars) { + # --env _ + # Include all vars that are not in a clean environment + if(open(my $vars_fh, "<", $ENV{'HOME'} . "/.parallel/ignored_vars")) { + my @ignore = <$vars_fh>; + chomp @ignore; + my %ignore; + @ignore{@ignore} = @ignore; + close $vars_fh; + push @vars, grep { not defined $ignore{$_} } keys %ENV; + @vars = grep { not /^_$/ } @vars; + } else { + ::error("Run '$Global::progname --record-env' in a clean environment first.\n"); + ::wait_and_exit(255); + } + } + # Duplicate vars as BASH functions to include post-shellshock functions. + # So --env myfunc should also look for BASH_FUNC_myfunc() + @vars = map { $_, "BASH_FUNC_$_()" } @vars; + # Keep only defined variables + @vars = grep { defined($ENV{$_}) } @vars; + # Pre-shellshock style bash function: + # myfunc=() { echo myfunc + # } + # Post-shellshock style bash function: + # BASH_FUNC_myfunc()=() { echo myfunc + # } + my @bash_functions = grep { substr($ENV{$_},0,4) eq "() {" } @vars; + my @non_functions = grep { substr($ENV{$_},0,4) ne "() {" } @vars; + if(@bash_functions) { + # Functions are not supported for all shells + if($Global::shell !~ m:/(bash|rbash|zsh|rzsh|dash|ksh):) { + ::warning("Shell functions may not be supported in $Global::shell\n"); + } + } + + # Pre-shellschock names are without () + my @bash_pre_shellshock = grep { not /\(\)/ } @bash_functions; + # Post-shellschock names are with () + my @bash_post_shellshock = grep { /\(\)/ } @bash_functions; + + my @qcsh = (map { my $a=$_; "setenv $a " . env_quote($ENV{$a}) } + grep { not /^parallel_bash_environment$/ } @non_functions); + my @qbash = (map { my $a=$_; "export $a=" . env_quote($ENV{$a}) } + @non_functions, @bash_pre_shellshock); + + push @qbash, map { my $a=$_; "eval $a\"\$$a\"" } @bash_pre_shellshock; + push @qbash, map { /BASH_FUNC_(.*)\(\)/; "$1 $ENV{$_}" } @bash_post_shellshock; + + #ssh -tt -oLogLevel=quiet lo 'eval `echo PARALLEL_SEQ='$PARALLEL_SEQ'\;export PARALLEL_SEQ\; PARALLEL_PID='$PARALLEL_PID'\;export PARALLEL_PID` ;' tty\ \>/dev/null\ \&\&\ stty\ isig\ -onlcr\ -echo\;echo\ \$SHELL\ \|\ grep\ \"/t\\\{0,1\\\}csh\"\ \>\ /dev/null\ \&\&\ setenv\ BASH_FUNC_myfunc\ \\\(\\\)\\\ \\\{\\\ \\\ echo\\\ a\"' + #'\"\\\}\ \|\|\ myfunc\(\)\ \{\ \ echo\ a' + #'\}\ \;myfunc\ 1; + + # Check if any variables contain \n + if(my @v = map { s/BASH_FUNC_(.*)\(\)/$1/; $_ } grep { $ENV{$_}=~/\n/ } @vars) { + # \n is bad for csh and will cause it to fail. + $Global::envwarn = ::shell_quote_scalar(q{echo $SHELL | egrep "/t?csh" > /dev/null && echo CSH/TCSH DO NOT SUPPORT newlines IN VARIABLES/FUNCTIONS. Unset }."@v".q{ && exec false;}."\n\n") . $Global::envwarn; + } + + if(not @qcsh) { push @qcsh, "true"; } + if(not @qbash) { push @qbash, "true"; } + # Create lines like: + # echo $SHELL | grep "/t\\{0,1\\}csh" >/dev/null && setenv V1 val1 && setenv V2 val2 || export V1=val1 && export V2=val2 ; echo "$V1$V2" + if(@vars) { + $Global::envvar .= + join"", + (q{echo $SHELL | grep "/t\\{0,1\\}csh" > /dev/null && } + . join(" && ", @qcsh) + . q{ || } + . join(" && ", @qbash) + .q{;}); + if($ENV{'parallel_bash_environment'}) { + $Global::envvar .= 'eval "$parallel_bash_environment";'."\n"; + } + } + $Global::envvarlen = length $Global::envvar; +} + +sub open_joblog { + # Open joblog as specified by --joblog + # Uses: + # $opt::resume + # $opt::resume_failed + # $opt::joblog + # $opt::results + # $Global::job_already_run + # %Global::fd + my $append = 0; + if(($opt::resume or $opt::resume_failed) + and + not ($opt::joblog or $opt::results)) { + ::error("--resume and --resume-failed require --joblog or --results.\n"); + ::wait_and_exit(255); + } + if($opt::joblog) { + if($opt::resume || $opt::resume_failed) { + if(open(my $joblog_fh, "<", $opt::joblog)) { + # Read the joblog + $append = <$joblog_fh>; # If there is a header: Open as append later + my $joblog_regexp; + if($opt::resume_failed) { + # Make a regexp that only matches commands with exit+signal=0 + # 4 host 1360490623.067 3.445 1023 1222 0 0 command + $joblog_regexp='^(\d+)(?:\t[^\t]+){5}\t0\t0\t'; + } else { + # Just match the job number + $joblog_regexp='^(\d+)'; + } + while(<$joblog_fh>) { + if(/$joblog_regexp/o) { + # This is 30% faster than set_job_already_run($1); + vec($Global::job_already_run,($1||0),1) = 1; + } elsif(not /\d+\s+[^\s]+\s+([0-9.]+\s+){6}/) { + ::error("Format of '$opt::joblog' is wrong: $_"); + ::wait_and_exit(255); + } + } + close $joblog_fh; + } + } + if($append) { + # Append to joblog + if(not open($Global::joblog, ">>", $opt::joblog)) { + ::error("Cannot append to --joblog $opt::joblog.\n"); + ::wait_and_exit(255); + } + } else { + if($opt::joblog eq "-") { + # Use STDOUT as joblog + $Global::joblog = $Global::fd{1}; + } elsif(not open($Global::joblog, ">", $opt::joblog)) { + # Overwrite the joblog + ::error("Cannot write to --joblog $opt::joblog.\n"); + ::wait_and_exit(255); + } + print $Global::joblog + join("\t", "Seq", "Host", "Starttime", "JobRuntime", + "Send", "Receive", "Exitval", "Signal", "Command" + ). "\n"; + } + } +} + +sub find_compression_program { + # Find a fast compression program + # Returns: + # $compress_program = compress program with options + # $decompress_program = decompress program with options + + # Search for these. Sorted by speed + my @prg = qw(lzop pigz pxz gzip plzip pbzip2 lzma xz lzip bzip2); + for my $p (@prg) { + if(which($p)) { + return ("$p -c -1","$p -dc"); + } + } + # Fall back to cat + return ("cat","cat"); +} + + +sub read_options { + # Read options from command line, profile and $PARALLEL + # Uses: + # $opt::shebang_wrap + # $opt::shebang + # @ARGV + # $opt::plain + # @opt::profile + # $ENV{'HOME'} + # $ENV{'PARALLEL'} + # Returns: + # @ARGV_no_opt = @ARGV without --options + + # This must be done first as this may exec myself + if(defined $ARGV[0] and ($ARGV[0] =~ /^--shebang/ or + $ARGV[0] =~ /^--shebang-?wrap/ or + $ARGV[0] =~ /^--hashbang/)) { + # Program is called from #! line in script + # remove --shebang-wrap if it is set + $opt::shebang_wrap = ($ARGV[0] =~ s/^--shebang-?wrap *//); + # remove --shebang if it is set + $opt::shebang = ($ARGV[0] =~ s/^--shebang *//); + # remove --hashbang if it is set + $opt::shebang .= ($ARGV[0] =~ s/^--hashbang *//); + if($opt::shebang) { + my $argfile = shell_quote_scalar(pop @ARGV); + # exec myself to split $ARGV[0] into separate fields + exec "$0 --skip-first-line -a $argfile @ARGV"; + } + if($opt::shebang_wrap) { + my @options; + my @parser; + if ($^O eq 'freebsd') { + # FreeBSD's #! puts different values in @ARGV than Linux' does. + my @nooptions = @ARGV; + get_options_from_array(\@nooptions); + while($#ARGV > $#nooptions) { + push @options, shift @ARGV; + } + while(@ARGV and $ARGV[0] ne ":::") { + push @parser, shift @ARGV; + } + if(@ARGV and $ARGV[0] eq ":::") { + shift @ARGV; + } + } else { + @options = shift @ARGV; + } + my $script = shell_quote_scalar(shift @ARGV); + # exec myself to split $ARGV[0] into separate fields + exec "$0 --internal-pipe-means-argfiles @options @parser $script ::: @ARGV"; + } + } + + Getopt::Long::Configure("bundling","require_order"); + my @ARGV_copy = @ARGV; + # Check if there is a --profile to set @opt::profile + get_options_from_array(\@ARGV_copy,"profile|J=s","plain") || die_usage(); + my @ARGV_profile = (); + my @ARGV_env = (); + if(not $opt::plain) { + # Add options from .parallel/config and other profiles + my @config_profiles = ( + "/etc/parallel/config", + $ENV{'HOME'}."/.parallel/config", + $ENV{'HOME'}."/.parallelrc"); + my @profiles = @config_profiles; + if(@opt::profile) { + # --profile overrides default profiles + @profiles = (); + for my $profile (@opt::profile) { + if(-r $profile) { + push @profiles, $profile; + } else { + push @profiles, $ENV{'HOME'}."/.parallel/".$profile; + } + } + } + for my $profile (@profiles) { + if(-r $profile) { + open (my $in_fh, "<", $profile) || ::die_bug("read-profile: $profile"); + while(<$in_fh>) { + /^\s*\#/ and next; + chomp; + push @ARGV_profile, shellwords($_); + } + close $in_fh; + } else { + if(grep /^$profile$/, @config_profiles) { + # config file is not required to exist + } else { + ::error("$profile not readable.\n"); + wait_and_exit(255); + } + } + } + # Add options from shell variable $PARALLEL + if($ENV{'PARALLEL'}) { + @ARGV_env = shellwords($ENV{'PARALLEL'}); + } + } + Getopt::Long::Configure("bundling","require_order"); + get_options_from_array(\@ARGV_profile) || die_usage(); + get_options_from_array(\@ARGV_env) || die_usage(); + get_options_from_array(\@ARGV) || die_usage(); + + # Prepend non-options to @ARGV (such as commands like 'nice') + unshift @ARGV, @ARGV_profile, @ARGV_env; + return @ARGV; +} + +sub read_args_from_command_line { + # Arguments given on the command line after: + # ::: ($Global::arg_sep) + # :::: ($Global::arg_file_sep) + # Removes the arguments from @ARGV and: + # - puts filenames into -a + # - puts arguments into files and add the files to -a + # Input: + # @::ARGV = command option ::: arg arg arg :::: argfiles + # Uses: + # $Global::arg_sep + # $Global::arg_file_sep + # $opt::internal_pipe_means_argfiles + # $opt::pipe + # @opt::a + # Returns: + # @argv_no_argsep = @::ARGV without ::: and :::: and following args + my @new_argv = (); + for(my $arg = shift @ARGV; @ARGV; $arg = shift @ARGV) { + if($arg eq $Global::arg_sep + or + $arg eq $Global::arg_file_sep) { + my $group = $arg; # This group of arguments is args or argfiles + my @group; + while(defined ($arg = shift @ARGV)) { + if($arg eq $Global::arg_sep + or + $arg eq $Global::arg_file_sep) { + # exit while loop if finding new separator + last; + } else { + # If not hitting ::: or :::: + # Append it to the group + push @group, $arg; + } + } + + if($group eq $Global::arg_file_sep + or ($opt::internal_pipe_means_argfiles and $opt::pipe) + ) { + # Group of file names on the command line. + # Append args into -a + push @opt::a, @group; + } elsif($group eq $Global::arg_sep) { + # Group of arguments on the command line. + # Put them into a file. + # Create argfile + my ($outfh,$name) = ::tmpfile(SUFFIX => ".arg"); + unlink($name); + # Put args into argfile + print $outfh map { $_,$/ } @group; + seek $outfh, 0, 0; + # Append filehandle to -a + push @opt::a, $outfh; + } else { + ::die_bug("Unknown command line group: $group"); + } + if(defined($arg)) { + # $arg is ::: or :::: + redo; + } else { + # $arg is undef -> @ARGV empty + last; + } + } + push @new_argv, $arg; + } + # Output: @ARGV = command to run with options + return @new_argv; +} + +sub cleanup { + # Returns: N/A + if(@opt::basefile) { cleanup_basefile(); } +} + +sub __QUOTING_ARGUMENTS_FOR_SHELL__ {} + +sub shell_quote { + # Input: + # @strings = strings to be quoted + # Output: + # @shell_quoted_strings = string quoted with \ as needed by the shell + my @strings = (@_); + for my $a (@strings) { + $a =~ s/([\002-\011\013-\032\\\#\?\`\(\)\{\}\[\]\*\>\<\~\|\; \"\!\$\&\'\202-\377])/\\$1/g; + $a =~ s/[\n]/'\n'/g; # filenames with '\n' is quoted using \' + } + return wantarray ? @strings : "@strings"; +} + +sub shell_quote_empty { + # Inputs: + # @strings = strings to be quoted + # Returns: + # @quoted_strings = empty strings quoted as ''. + my @strings = shell_quote(@_); + for my $a (@strings) { + if($a eq "") { + $a = "''"; + } + } + return wantarray ? @strings : "@strings"; +} + +sub shell_quote_scalar { + # Quote the string so shell will not expand any special chars + # Inputs: + # $string = string to be quoted + # Returns: + # $shell_quoted = string quoted with \ as needed by the shell + my $a = $_[0]; + if(defined $a) { + # $a =~ s/([\002-\011\013-\032\\\#\?\`\(\)\{\}\[\]\*\>\<\~\|\; \"\!\$\&\'\202-\377])/\\$1/g; + # This is 1% faster than the above + $a =~ s/[\002-\011\013-\032\\\#\?\`\(\)\{\}\[\]\*\>\<\~\|\; \"\!\$\&\'\202-\377]/\\$&/go; + $a =~ s/[\n]/'\n'/go; # filenames with '\n' is quoted using \' + } + return $a; +} + +sub shell_quote_file { + # Quote the string so shell will not expand any special chars and prepend ./ if needed + # Input: + # $filename = filename to be shell quoted + # Returns: + # $quoted_filename = filename quoted with \ as needed by the shell and ./ if needed + my $a = shell_quote_scalar(shift); + if(defined $a) { + if($a =~ m:^/: or $a =~ m:^\./:) { + # /abs/path or ./rel/path => skip + } else { + # rel/path => ./rel/path + $a = "./".$a; + } + } + return $a; +} + +sub shellwords { + # Input: + # $string = shell line + # Returns: + # @shell_words = $string split into words as shell would do + $Global::use{"Text::ParseWords"} ||= eval "use Text::ParseWords; 1;"; + return Text::ParseWords::shellwords(@_); +} + + +sub __FILEHANDLES__ {} + + +sub save_stdin_stdout_stderr { + # Remember the original STDIN, STDOUT and STDERR + # and file descriptors opened by the shell (e.g. 3>/tmp/foo) + # Uses: + # %Global::fd + # $Global::original_stderr + # $Global::original_stdin + # Returns: N/A + + # Find file descriptors that are already opened (by the shell) + for my $fdno (1..61) { + # /dev/fd/62 and above are used by bash for <(cmd) + my $fh; + # 2-argument-open is used to be compatible with old perl 5.8.0 + # bug #43570: Perl 5.8.0 creates 61 files + if(open($fh,">&=$fdno")) { + $Global::fd{$fdno}=$fh; + } + } + open $Global::original_stderr, ">&", "STDERR" or + ::die_bug("Can't dup STDERR: $!"); + open $Global::original_stdin, "<&", "STDIN" or + ::die_bug("Can't dup STDIN: $!"); +} + +sub enough_file_handles { + # Check that we have enough filehandles available for starting + # another job + # Uses: + # $opt::ungroup + # %Global::fd + # Returns: + # 1 if ungrouped (thus not needing extra filehandles) + # 0 if too few filehandles + # 1 if enough filehandles + if(not $opt::ungroup) { + my %fh; + my $enough_filehandles = 1; + # perl uses 7 filehandles for something? + # open3 uses 2 extra filehandles temporarily + # We need a filehandle for each redirected file descriptor + # (normally just STDOUT and STDERR) + for my $i (1..(7+2+keys %Global::fd)) { + $enough_filehandles &&= open($fh{$i}, "<", "/dev/null"); + } + for (values %fh) { close $_; } + return $enough_filehandles; + } else { + # Ungrouped does not need extra file handles + return 1; + } +} + +sub open_or_exit { + # Open a file name or exit if the file cannot be opened + # Inputs: + # $file = filehandle or filename to open + # Uses: + # $Global::stdin_in_opt_a + # $Global::original_stdin + # Returns: + # $fh = file handle to read-opened file + my $file = shift; + if($file eq "-") { + $Global::stdin_in_opt_a = 1; + return ($Global::original_stdin || *STDIN); + } + if(ref $file eq "GLOB") { + # This is an open filehandle + return $file; + } + my $fh = gensym; + if(not open($fh, "<", $file)) { + ::error("Cannot open input file `$file': No such file or directory.\n"); + wait_and_exit(255); + } + return $fh; +} + +sub __RUNNING_THE_JOBS_AND_PRINTING_PROGRESS__ {} + +# Variable structure: +# +# $Global::running{$pid} = Pointer to Job-object +# @Global::virgin_jobs = Pointer to Job-object that have received no input +# $Global::host{$sshlogin} = Pointer to SSHLogin-object +# $Global::total_running = total number of running jobs +# $Global::total_started = total jobs started + +sub init_run_jobs { + $Global::total_running = 0; + $Global::total_started = 0; + $Global::tty_taken = 0; + $SIG{USR1} = \&list_running_jobs; + $SIG{USR2} = \&toggle_progress; + if(@opt::basefile) { setup_basefile(); } +} + +{ + my $last_time; + my %last_mtime; + +sub start_more_jobs { + # Run start_another_job() but only if: + # * not $Global::start_no_new_jobs set + # * not JobQueue is empty + # * not load on server is too high + # * not server swapping + # * not too short time since last remote login + # Uses: + # $Global::max_procs_file + # $Global::max_procs_file_last_mod + # %Global::host + # @opt::sshloginfile + # $Global::start_no_new_jobs + # $opt::filter_hosts + # $Global::JobQueue + # $opt::pipe + # $opt::load + # $opt::noswap + # $opt::delay + # $Global::newest_starttime + # Returns: + # $jobs_started = number of jobs started + my $jobs_started = 0; + my $jobs_started_this_round = 0; + if($Global::start_no_new_jobs) { + return $jobs_started; + } + if(time - ($last_time||0) > 1) { + # At most do this every second + $last_time = time; + if($Global::max_procs_file) { + # --jobs filename + my $mtime = (stat($Global::max_procs_file))[9]; + if($mtime > $Global::max_procs_file_last_mod) { + # file changed: Force re-computing max_jobs_running + $Global::max_procs_file_last_mod = $mtime; + for my $sshlogin (values %Global::host) { + $sshlogin->set_max_jobs_running(undef); + } + } + } + if(@opt::sshloginfile) { + # Is --sshloginfile changed? + for my $slf (@opt::sshloginfile) { + my $actual_file = expand_slf_shorthand($slf); + my $mtime = (stat($actual_file))[9]; + $last_mtime{$actual_file} ||= $mtime; + if($mtime - $last_mtime{$actual_file} > 1) { + ::debug("run","--sshloginfile $actual_file changed. reload\n"); + $last_mtime{$actual_file} = $mtime; + # Reload $slf + # Empty sshlogins + @Global::sshlogin = (); + for (values %Global::host) { + # Don't start new jobs on any host + # except the ones added back later + $_->set_max_jobs_running(0); + } + # This will set max_jobs_running on the SSHlogins + read_sshloginfile($actual_file); + parse_sshlogin(); + $opt::filter_hosts and filter_hosts(); + setup_basefile(); + } + } + } + } + do { + $jobs_started_this_round = 0; + # This will start 1 job on each --sshlogin (if possible) + # thus distribute the jobs on the --sshlogins round robin + + for my $sshlogin (values %Global::host) { + if($Global::JobQueue->empty() and not $opt::pipe) { + # No more jobs in the queue + last; + } + debug("run", "Running jobs before on ", $sshlogin->string(), ": ", + $sshlogin->jobs_running(), "\n"); + if ($sshlogin->jobs_running() < $sshlogin->max_jobs_running()) { + if($opt::load and $sshlogin->loadavg_too_high()) { + # The load is too high or unknown + next; + } + if($opt::noswap and $sshlogin->swapping()) { + # The server is swapping + next; + } + if($sshlogin->too_fast_remote_login()) { + # It has been too short since + next; + } + if($opt::delay and $opt::delay > ::now() - $Global::newest_starttime) { + # It has been too short since last start + next; + } + debug("run", $sshlogin->string(), " has ", $sshlogin->jobs_running(), + " out of ", $sshlogin->max_jobs_running(), + " jobs running. Start another.\n"); + if(start_another_job($sshlogin) == 0) { + # No more jobs to start on this $sshlogin + debug("run","No jobs started on ", $sshlogin->string(), "\n"); + next; + } + $sshlogin->inc_jobs_running(); + $sshlogin->set_last_login_at(::now()); + $jobs_started++; + $jobs_started_this_round++; + } + debug("run","Running jobs after on ", $sshlogin->string(), ": ", + $sshlogin->jobs_running(), " of ", + $sshlogin->max_jobs_running(), "\n"); + } + } while($jobs_started_this_round); + + return $jobs_started; +} +} + +{ + my $no_more_file_handles_warned; + +sub start_another_job { + # If there are enough filehandles + # and JobQueue not empty + # and not $job is in joblog + # Then grab a job from Global::JobQueue, + # start it at sshlogin + # mark it as virgin_job + # Inputs: + # $sshlogin = the SSHLogin to start the job on + # Uses: + # $Global::JobQueue + # $opt::pipe + # $opt::results + # $opt::resume + # @Global::virgin_jobs + # Returns: + # 1 if another jobs was started + # 0 otherwise + my $sshlogin = shift; + # Do we have enough file handles to start another job? + if(enough_file_handles()) { + if($Global::JobQueue->empty() and not $opt::pipe) { + # No more commands to run + debug("start", "Not starting: JobQueue empty\n"); + return 0; + } else { + my $job; + # Skip jobs already in job log + # Skip jobs already in results + do { + $job = get_job_with_sshlogin($sshlogin); + if(not defined $job) { + # No command available for that sshlogin + debug("start", "Not starting: no jobs available for ", + $sshlogin->string(), "\n"); + return 0; + } + } while ($job->is_already_in_joblog() + or + ($opt::results and $opt::resume and $job->is_already_in_results())); + debug("start", "Command to run on '", $job->sshlogin()->string(), "': '", + $job->replaced(),"'\n"); + if($job->start()) { + if($opt::pipe) { + push(@Global::virgin_jobs,$job); + } + debug("start", "Started as seq ", $job->seq(), + " pid:", $job->pid(), "\n"); + return 1; + } else { + # Not enough processes to run the job. + # Put it back on the queue. + $Global::JobQueue->unget($job); + # Count down the number of jobs to run for this SSHLogin. + my $max = $sshlogin->max_jobs_running(); + if($max > 1) { $max--; } else { + ::error("No more processes: cannot run a single job. Something is wrong.\n"); + ::wait_and_exit(255); + } + $sshlogin->set_max_jobs_running($max); + # Sleep up to 300 ms to give other processes time to die + ::usleep(rand()*300); + ::warning("No more processes: ", + "Decreasing number of running jobs to $max. ", + "Raising ulimit -u or /etc/security/limits.conf may help.\n"); + return 0; + } + } + } else { + # No more file handles + $no_more_file_handles_warned++ or + ::warning("No more file handles. ", + "Raising ulimit -n or /etc/security/limits.conf may help.\n"); + return 0; + } +} +} + +sub init_progress { + # Uses: + # $opt::bar + # Returns: + # list of computers for progress output + $|=1; + if($opt::bar) { + return("",""); + } + my %progress = progress(); + return ("\nComputers / CPU cores / Max jobs to run\n", + $progress{'workerlist'}); +} + +sub drain_job_queue { + # Uses: + # $opt::progress + # $Global::original_stderr + # $Global::total_running + # $Global::max_jobs_running + # %Global::running + # $Global::JobQueue + # %Global::host + # $Global::start_no_new_jobs + # Returns: N/A + if($opt::progress) { + print $Global::original_stderr init_progress(); + } + my $last_header=""; + my $sleep = 0.2; + do { + while($Global::total_running > 0) { + debug($Global::total_running, "==", scalar + keys %Global::running," slots: ", $Global::max_jobs_running); + if($opt::pipe) { + # When using --pipe sometimes file handles are not closed properly + for my $job (values %Global::running) { + close $job->fh(0,"w"); + } + } + if($opt::progress) { + my %progress = progress(); + if($last_header ne $progress{'header'}) { + print $Global::original_stderr "\n", $progress{'header'}, "\n"; + $last_header = $progress{'header'}; + } + print $Global::original_stderr "\r",$progress{'status'}; + flush $Global::original_stderr; + } + if($Global::total_running < $Global::max_jobs_running + and not $Global::JobQueue->empty()) { + # These jobs may not be started because of loadavg + # or too little time between each ssh login. + if(start_more_jobs() > 0) { + # Exponential back-on if jobs were started + $sleep = $sleep/2+0.001; + } + } + # Sometimes SIGCHLD is not registered, so force reaper + $sleep = ::reap_usleep($sleep); + } + if(not $Global::JobQueue->empty()) { + # These jobs may not be started: + # * because there the --filter-hosts has removed all + if(not %Global::host) { + ::error("There are no hosts left to run on.\n"); + ::wait_and_exit(255); + } + # * because of loadavg + # * because of too little time between each ssh login. + start_more_jobs(); + $sleep = ::reap_usleep($sleep); + if($Global::max_jobs_running == 0) { + ::warning("There are no job slots available. Increase --jobs.\n"); + } + } + } while ($Global::total_running > 0 + or + not $Global::start_no_new_jobs and not $Global::JobQueue->empty()); + if($opt::progress) { + my %progress = progress(); + print $Global::original_stderr "\r", $progress{'status'}, "\n"; + flush $Global::original_stderr; + } +} + +sub toggle_progress { + # Turn on/off progress view + # Uses: + # $opt::progress + # $Global::original_stderr + # Returns: N/A + $opt::progress = not $opt::progress; + if($opt::progress) { + print $Global::original_stderr init_progress(); + } +} + +sub progress { + # Uses: + # $opt::bar + # $opt::eta + # %Global::host + # $Global::total_started + # Returns: + # $workerlist = list of workers + # $header = that will fit on the screen + # $status = message that will fit on the screen + if($opt::bar) { + return ("workerlist" => "", "header" => "", "status" => bar()); + } + my $eta = ""; + my ($status,$header)=("",""); + if($opt::eta) { + my($total, $completed, $left, $pctcomplete, $avgtime, $this_eta) = + compute_eta(); + $eta = sprintf("ETA: %ds Left: %d AVG: %.2fs ", + $this_eta, $left, $avgtime); + } + my $termcols = terminal_columns(); + my @workers = sort keys %Global::host; + my %sshlogin = map { $_ eq ":" ? ($_=>"local") : ($_=>$_) } @workers; + my $workerno = 1; + my %workerno = map { ($_=>$workerno++) } @workers; + my $workerlist = ""; + for my $w (@workers) { + $workerlist .= + $workerno{$w}.":".$sshlogin{$w} ." / ". + ($Global::host{$w}->ncpus() || "-")." / ". + $Global::host{$w}->max_jobs_running()."\n"; + } + $status = "x"x($termcols+1); + if(length $status > $termcols) { + # sshlogin1:XX/XX/XX%/XX.Xs sshlogin2:XX/XX/XX%/XX.Xs sshlogin3:XX/XX/XX%/XX.Xs + $header = "Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete"; + $status = $eta . + join(" ",map + { + if($Global::total_started) { + my $completed = ($Global::host{$_}->jobs_completed()||0); + my $running = $Global::host{$_}->jobs_running(); + my $time = $completed ? (time-$^T)/($completed) : "0"; + sprintf("%s:%d/%d/%d%%/%.1fs ", + $sshlogin{$_}, $running, $completed, + ($running+$completed)*100 + / $Global::total_started, $time); + } + } @workers); + } + if(length $status > $termcols) { + # 1:XX/XX/XX%/XX.Xs 2:XX/XX/XX%/XX.Xs 3:XX/XX/XX%/XX.Xs 4:XX/XX/XX%/XX.Xs + $header = "Computer:jobs running/jobs completed/%of started jobs"; + $status = $eta . + join(" ",map + { + my $completed = ($Global::host{$_}->jobs_completed()||0); + my $running = $Global::host{$_}->jobs_running(); + my $time = $completed ? (time-$^T)/($completed) : "0"; + sprintf("%s:%d/%d/%d%%/%.1fs ", + $workerno{$_}, $running, $completed, + ($running+$completed)*100 + / $Global::total_started, $time); + } @workers); + } + if(length $status > $termcols) { + # sshlogin1:XX/XX/XX% sshlogin2:XX/XX/XX% sshlogin3:XX/XX/XX% + $header = "Computer:jobs running/jobs completed/%of started jobs"; + $status = $eta . + join(" ",map + { sprintf("%s:%d/%d/%d%%", + $sshlogin{$_}, + $Global::host{$_}->jobs_running(), + ($Global::host{$_}->jobs_completed()||0), + ($Global::host{$_}->jobs_running()+ + ($Global::host{$_}->jobs_completed()||0))*100 + / $Global::total_started) } + @workers); + } + if(length $status > $termcols) { + # 1:XX/XX/XX% 2:XX/XX/XX% 3:XX/XX/XX% 4:XX/XX/XX% 5:XX/XX/XX% 6:XX/XX/XX% + $header = "Computer:jobs running/jobs completed/%of started jobs"; + $status = $eta . + join(" ",map + { sprintf("%s:%d/%d/%d%%", + $workerno{$_}, + $Global::host{$_}->jobs_running(), + ($Global::host{$_}->jobs_completed()||0), + ($Global::host{$_}->jobs_running()+ + ($Global::host{$_}->jobs_completed()||0))*100 + / $Global::total_started) } + @workers); + } + if(length $status > $termcols) { + # sshlogin1:XX/XX/XX% sshlogin2:XX/XX/XX% sshlogin3:XX/XX sshlogin4:XX/XX + $header = "Computer:jobs running/jobs completed"; + $status = $eta . + join(" ",map + { sprintf("%s:%d/%d", + $sshlogin{$_}, $Global::host{$_}->jobs_running(), + ($Global::host{$_}->jobs_completed()||0)) } + @workers); + } + if(length $status > $termcols) { + # sshlogin1:XX/XX sshlogin2:XX/XX sshlogin3:XX/XX sshlogin4:XX/XX + $header = "Computer:jobs running/jobs completed"; + $status = $eta . + join(" ",map + { sprintf("%s:%d/%d", + $sshlogin{$_}, $Global::host{$_}->jobs_running(), + ($Global::host{$_}->jobs_completed()||0)) } + @workers); + } + if(length $status > $termcols) { + # 1:XX/XX 2:XX/XX 3:XX/XX 4:XX/XX 5:XX/XX 6:XX/XX + $header = "Computer:jobs running/jobs completed"; + $status = $eta . + join(" ",map + { sprintf("%s:%d/%d", + $workerno{$_}, $Global::host{$_}->jobs_running(), + ($Global::host{$_}->jobs_completed()||0)) } + @workers); + } + if(length $status > $termcols) { + # sshlogin1:XX sshlogin2:XX sshlogin3:XX sshlogin4:XX sshlogin5:XX + $header = "Computer:jobs completed"; + $status = $eta . + join(" ",map + { sprintf("%s:%d", + $sshlogin{$_}, + ($Global::host{$_}->jobs_completed()||0)) } + @workers); + } + if(length $status > $termcols) { + # 1:XX 2:XX 3:XX 4:XX 5:XX 6:XX + $header = "Computer:jobs completed"; + $status = $eta . + join(" ",map + { sprintf("%s:%d", + $workerno{$_}, + ($Global::host{$_}->jobs_completed()||0)) } + @workers); + } + return ("workerlist" => $workerlist, "header" => $header, "status" => $status); +} + +{ + my ($total, $first_completed, $smoothed_avg_time); + + sub compute_eta { + # Calculate important numbers for ETA + # Returns: + # $total = number of jobs in total + # $completed = number of jobs completed + # $left = number of jobs left + # $pctcomplete = percent of jobs completed + # $avgtime = averaged time + # $eta = smoothed eta + $total ||= $Global::JobQueue->total_jobs(); + my $completed = 0; + for(values %Global::host) { $completed += $_->jobs_completed() } + my $left = $total - $completed; + if(not $completed) { + return($total, $completed, $left, 0, 0, 0); + } + my $pctcomplete = $completed / $total; + $first_completed ||= time; + my $timepassed = (time - $first_completed); + my $avgtime = $timepassed / $completed; + $smoothed_avg_time ||= $avgtime; + # Smooth the eta so it does not jump wildly + $smoothed_avg_time = (1 - $pctcomplete) * $smoothed_avg_time + + $pctcomplete * $avgtime; + my $eta = int($left * $smoothed_avg_time); + return($total, $completed, $left, $pctcomplete, $avgtime, $eta); + } +} + +{ + my ($rev,$reset); + + sub bar { + # Return: + # $status = bar with eta, completed jobs, arg and pct + $rev ||= "\033[7m"; + $reset ||= "\033[0m"; + my($total, $completed, $left, $pctcomplete, $avgtime, $eta) = + compute_eta(); + my $arg = $Global::newest_job ? + $Global::newest_job->{'commandline'}->replace_placeholders(["\257<\257>"],0,0) : ""; + # These chars mess up display in the terminal + $arg =~ tr/[\011-\016\033\302-\365]//d; + my $bar_text = + sprintf("%d%% %d:%d=%ds %s", + $pctcomplete*100, $completed, $left, $eta, $arg); + my $terminal_width = terminal_columns(); + my $s = sprintf("%-${terminal_width}s", + substr($bar_text." "x$terminal_width, + 0,$terminal_width)); + my $width = int($terminal_width * $pctcomplete); + substr($s,$width,0) = $reset; + my $zenity = sprintf("%-${terminal_width}s", + substr("# $eta sec $arg", + 0,$terminal_width)); + $s = "\r" . $zenity . "\r" . $pctcomplete*100 . # Prefix with zenity header + "\r" . $rev . $s . $reset; + return $s; + } +} + +{ + my ($columns,$last_column_time); + + sub terminal_columns { + # Get the number of columns of the display + # Returns: + # number of columns of the screen + if(not $columns or $last_column_time < time) { + $last_column_time = time; + $columns = $ENV{'COLUMNS'}; + if(not $columns) { + my $resize = qx{ resize 2>/dev/null }; + $resize =~ /COLUMNS=(\d+);/ and do { $columns = $1; }; + } + $columns ||= 80; + } + return $columns; + } +} + +sub get_job_with_sshlogin { + # Returns: + # next job object for $sshlogin if any available + my $sshlogin = shift; + my $job = undef; + + if ($opt::hostgroups) { + my @other_hostgroup_jobs = (); + + while($job = $Global::JobQueue->get()) { + if($sshlogin->in_hostgroups($job->hostgroups())) { + # Found a job for this hostgroup + last; + } else { + # This job was not in the hostgroups of $sshlogin + push @other_hostgroup_jobs, $job; + } + } + $Global::JobQueue->unget(@other_hostgroup_jobs); + if(not defined $job) { + # No more jobs + return undef; + } + } else { + $job = $Global::JobQueue->get(); + if(not defined $job) { + # No more jobs + ::debug("start", "No more jobs: JobQueue empty\n"); + return undef; + } + } + + my $clean_command = $job->replaced(); + if($clean_command =~ /^\s*$/) { + # Do not run empty lines + if(not $Global::JobQueue->empty()) { + return get_job_with_sshlogin($sshlogin); + } else { + return undef; + } + } + $job->set_sshlogin($sshlogin); + if($opt::retries and $clean_command and + $job->failed_here()) { + # This command with these args failed for this sshlogin + my ($no_of_failed_sshlogins,$min_failures) = $job->min_failed(); + # Only look at the Global::host that have > 0 jobslots + if($no_of_failed_sshlogins == grep { $_->max_jobs_running() > 0 } values %Global::host + and $job->failed_here() == $min_failures) { + # It failed the same or more times on another host: + # run it on this host + } else { + # If it failed fewer times on another host: + # Find another job to run + my $nextjob; + if(not $Global::JobQueue->empty()) { + # This can potentially recurse for all args + no warnings 'recursion'; + $nextjob = get_job_with_sshlogin($sshlogin); + } + # Push the command back on the queue + $Global::JobQueue->unget($job); + return $nextjob; + } + } + return $job; +} + +sub __REMOTE_SSH__ {} + +sub read_sshloginfiles { + # Returns: N/A + for my $s (@_) { + read_sshloginfile(expand_slf_shorthand($s)); + } +} + +sub expand_slf_shorthand { + my $file = shift; + if($file eq "-") { + # skip: It is stdin + } elsif($file eq "..") { + $file = $ENV{'HOME'}."/.parallel/sshloginfile"; + } elsif($file eq ".") { + $file = "/etc/parallel/sshloginfile"; + } elsif(not -r $file) { + if(not -r $ENV{'HOME'}."/.parallel/".$file) { + # Try prepending ~/.parallel + ::error("Cannot open $file.\n"); + ::wait_and_exit(255); + } else { + $file = $ENV{'HOME'}."/.parallel/".$file; + } + } + return $file; +} + +sub read_sshloginfile { + # Returns: N/A + my $file = shift; + my $close = 1; + my $in_fh; + ::debug("init","--slf ",$file); + if($file eq "-") { + $in_fh = *STDIN; + $close = 0; + } else { + if(not open($in_fh, "<", $file)) { + # Try the filename + ::error("Cannot open $file.\n"); + ::wait_and_exit(255); + } + } + while(<$in_fh>) { + chomp; + /^\s*#/ and next; + /^\s*$/ and next; + push @Global::sshlogin, $_; + } + if($close) { + close $in_fh; + } +} + +sub parse_sshlogin { + # Returns: N/A + my @login; + if(not @Global::sshlogin) { @Global::sshlogin = (":"); } + for my $sshlogin (@Global::sshlogin) { + # Split up -S sshlogin,sshlogin + for my $s (split /,/, $sshlogin) { + if ($s eq ".." or $s eq "-") { + # This may add to @Global::sshlogin - possibly bug + read_sshloginfile(expand_slf_shorthand($s)); + } else { + push (@login, $s); + } + } + } + $Global::minimal_command_line_length = 8_000_000; + my @allowed_hostgroups; + for my $ncpu_sshlogin_string (::uniq(@login)) { + my $sshlogin = SSHLogin->new($ncpu_sshlogin_string); + my $sshlogin_string = $sshlogin->string(); + if($sshlogin_string eq "") { + # This is an ssh group: -S @webservers + push @allowed_hostgroups, $sshlogin->hostgroups(); + next; + } + if($Global::host{$sshlogin_string}) { + # This sshlogin has already been added: + # It is probably a host that has come back + # Set the max_jobs_running back to the original + debug("run","Already seen $sshlogin_string\n"); + if($sshlogin->{'ncpus'}) { + # If ncpus set by '#/' of the sshlogin, overwrite it: + $Global::host{$sshlogin_string}->set_ncpus($sshlogin->ncpus()); + } + $Global::host{$sshlogin_string}->set_max_jobs_running(undef); + next; + } + if($sshlogin_string eq ":") { + $sshlogin->set_maxlength(Limits::Command::max_length()); + } else { + # If all chars needs to be quoted, every other character will be \ + $sshlogin->set_maxlength(int(Limits::Command::max_length()/2)); + } + $Global::minimal_command_line_length = + ::min($Global::minimal_command_line_length, $sshlogin->maxlength()); + $Global::host{$sshlogin_string} = $sshlogin; + } + if(@allowed_hostgroups) { + # Remove hosts that are not in these groups + while (my ($string, $sshlogin) = each %Global::host) { + if(not $sshlogin->in_hostgroups(@allowed_hostgroups)) { + delete $Global::host{$string}; + } + } + } + + # debug("start", "sshlogin: ", my_dump(%Global::host),"\n"); + if($opt::transfer or @opt::return or $opt::cleanup or @opt::basefile) { + if(not remote_hosts()) { + # There are no remote hosts + if(@opt::trc) { + ::warning("--trc ignored as there are no remote --sshlogin.\n"); + } elsif (defined $opt::transfer) { + ::warning("--transfer ignored as there are no remote --sshlogin.\n"); + } elsif (@opt::return) { + ::warning("--return ignored as there are no remote --sshlogin.\n"); + } elsif (defined $opt::cleanup) { + ::warning("--cleanup ignored as there are no remote --sshlogin.\n"); + } elsif (@opt::basefile) { + ::warning("--basefile ignored as there are no remote --sshlogin.\n"); + } + } + } +} + +sub remote_hosts { + # Return sshlogins that are not ':' + # Returns: + # list of sshlogins with ':' removed + return grep !/^:$/, keys %Global::host; +} + +sub setup_basefile { + # Transfer basefiles to each $sshlogin + # This needs to be done before first jobs on $sshlogin is run + # Returns: N/A + my $cmd = ""; + my $rsync_destdir; + my $workdir; + for my $sshlogin (values %Global::host) { + if($sshlogin->string() eq ":") { next } + for my $file (@opt::basefile) { + if($file !~ m:^/: and $opt::workdir eq "...") { + ::error("Work dir '...' will not work with relative basefiles\n"); + ::wait_and_exit(255); + } + $workdir ||= Job->new("")->workdir(); + $cmd .= $sshlogin->rsync_transfer_cmd($file,$workdir) . "&"; + } + } + $cmd .= "wait;"; + debug("init", "basesetup: $cmd\n"); + print `$cmd`; +} + +sub cleanup_basefile { + # Remove the basefiles transferred + # Returns: N/A + my $cmd=""; + my $workdir = Job->new("")->workdir(); + for my $sshlogin (values %Global::host) { + if($sshlogin->string() eq ":") { next } + for my $file (@opt::basefile) { + $cmd .= $sshlogin->cleanup_cmd($file,$workdir)."&"; + } + } + $cmd .= "wait;"; + debug("init", "basecleanup: $cmd\n"); + print `$cmd`; +} + +sub filter_hosts { + my(@cores, @cpus, @maxline, @echo); + my $envvar = ::shell_quote_scalar($Global::envvar); + while (my ($host, $sshlogin) = each %Global::host) { + if($host eq ":") { next } + # The 'true' is used to get the $host out later + my $sshcmd = "true $host;" . $sshlogin->sshcommand()." ".$sshlogin->serverlogin(); + push(@cores, $host."\t".$sshcmd." ".$envvar." parallel --number-of-cores\n\0"); + push(@cpus, $host."\t".$sshcmd." ".$envvar." parallel --number-of-cpus\n\0"); + push(@maxline, $host."\t".$sshcmd." ".$envvar." parallel --max-line-length-allowed\n\0"); + # 'echo' is used to get the best possible value for an ssh login time + push(@echo, $host."\t".$sshcmd." echo\n\0"); + } + my ($fh, $tmpfile) = ::tmpfile(SUFFIX => ".ssh"); + print $fh @cores, @cpus, @maxline, @echo; + close $fh; + # --timeout 5: Setting up an SSH connection and running a simple + # command should never take > 5 sec. + # --delay 0.1: If multiple sshlogins use the same proxy the delay + # will make it less likely to overload the ssh daemon. + # --retries 3: If the ssh daemon it overloaded, try 3 times + # -s 16000: Half of the max line on UnixWare + my $cmd = "cat $tmpfile | $0 -j0 --timeout 5 -s 16000 --joblog - --plain --delay 0.1 --retries 3 --tag --tagstring {1} -0 --colsep '\t' -k eval {2} 2>/dev/null"; + ::debug("init", $cmd, "\n"); + open(my $host_fh, "-|", $cmd) || ::die_bug("parallel host check: $cmd"); + my (%ncores, %ncpus, %time_to_login, %maxlen, %echo, @down_hosts); + my $prepend = ""; + while(<$host_fh>) { + if(/\'$/) { + # if last char = ' then append next line + # This may be due to quoting of $Global::envvar + $prepend .= $_; + next; + } + $_ = $prepend . $_; + $prepend = ""; + chomp; + my @col = split /\t/, $_; + if(defined $col[6]) { + # This is a line from --joblog + # seq host time spent sent received exit signal command + # 2 : 1372607672.654 0.675 0 0 0 0 eval true\ m\;ssh\ m\ parallel\ --number-of-cores + if($col[0] eq "Seq" and $col[1] eq "Host" and + $col[2] eq "Starttime") { + # Header => skip + next; + } + # Get server from: eval true server\; + $col[8] =~ /eval true..([^;]+).;/ or ::die_bug("col8 does not contain host: $col[8]"); + my $host = $1; + $host =~ tr/\\//d; + $Global::host{$host} or next; + if($col[6] eq "255" or $col[7] eq "15") { + # exit == 255 or signal == 15: ssh failed + # Remove sshlogin + ::debug("init", "--filtered $host\n"); + push(@down_hosts, $host); + @down_hosts = uniq(@down_hosts); + } elsif($col[6] eq "127") { + # signal == 127: parallel not installed remote + # Set ncpus and ncores = 1 + ::warning("Could not figure out ", + "number of cpus on $host. Using 1.\n"); + $ncores{$host} = 1; + $ncpus{$host} = 1; + $maxlen{$host} = Limits::Command::max_length(); + } elsif($col[0] =~ /^\d+$/ and $Global::host{$host}) { + # Remember how log it took to log in + # 2 : 1372607672.654 0.675 0 0 0 0 eval true\ m\;ssh\ m\ echo + $time_to_login{$host} = ::min($time_to_login{$host},$col[3]); + } else { + ::die_bug("host check unmatched long jobline: $_"); + } + } elsif($Global::host{$col[0]}) { + # This output from --number-of-cores, --number-of-cpus, + # --max-line-length-allowed + # ncores: server 8 + # ncpus: server 2 + # maxlen: server 131071 + if(not $ncores{$col[0]}) { + $ncores{$col[0]} = $col[1]; + } elsif(not $ncpus{$col[0]}) { + $ncpus{$col[0]} = $col[1]; + } elsif(not $maxlen{$col[0]}) { + $maxlen{$col[0]} = $col[1]; + } elsif(not $echo{$col[0]}) { + $echo{$col[0]} = $col[1]; + } elsif(m/perl: warning:|LANGUAGE =|LC_ALL =|LANG =|are supported and installed/) { + # Skip these: + # perl: warning: Setting locale failed. + # perl: warning: Please check that your locale settings: + # LANGUAGE = (unset), + # LC_ALL = (unset), + # LANG = "en_US.UTF-8" + # are supported and installed on your system. + # perl: warning: Falling back to the standard locale ("C"). + } else { + ::die_bug("host check too many col0: $_"); + } + } else { + ::die_bug("host check unmatched short jobline ($col[0]): $_"); + } + } + close $host_fh; + $Global::debug or unlink $tmpfile; + delete @Global::host{@down_hosts}; + @down_hosts and ::warning("Removed @down_hosts\n"); + $Global::minimal_command_line_length = 8_000_000; + while (my ($sshlogin, $obj) = each %Global::host) { + if($sshlogin eq ":") { next } + $ncpus{$sshlogin} or ::die_bug("ncpus missing: ".$obj->serverlogin()); + $ncores{$sshlogin} or ::die_bug("ncores missing: ".$obj->serverlogin()); + $time_to_login{$sshlogin} or ::die_bug("time_to_login missing: ".$obj->serverlogin()); + $maxlen{$sshlogin} or ::die_bug("maxlen missing: ".$obj->serverlogin()); + if($opt::use_cpus_instead_of_cores) { + $obj->set_ncpus($ncpus{$sshlogin}); + } else { + $obj->set_ncpus($ncores{$sshlogin}); + } + $obj->set_time_to_login($time_to_login{$sshlogin}); + $obj->set_maxlength($maxlen{$sshlogin}); + $Global::minimal_command_line_length = + ::min($Global::minimal_command_line_length, + int($maxlen{$sshlogin}/2)); + ::debug("init", "Timing from -S:$sshlogin ncpus:",$ncpus{$sshlogin}, + " ncores:", $ncores{$sshlogin}, + " time_to_login:", $time_to_login{$sshlogin}, + " maxlen:", $maxlen{$sshlogin}, + " min_max_len:", $Global::minimal_command_line_length,"\n"); + } +} + +sub onall { + sub tmp_joblog { + my $joblog = shift; + if(not defined $joblog) { + return undef; + } + my ($fh, $tmpfile) = ::tmpfile(SUFFIX => ".log"); + close $fh; + return $tmpfile; + } + my @command = @_; + if($Global::quoting) { + @command = shell_quote_empty(@command); + } + + # Copy all @fhlist into tempfiles + my @argfiles = (); + for my $fh (@fhlist) { + my ($outfh, $name) = ::tmpfile(SUFFIX => ".all", UNLINK => 1); + print $outfh (<$fh>); + close $outfh; + push @argfiles, $name; + } + if(@opt::basefile) { setup_basefile(); } + # for each sshlogin do: + # parallel -S $sshlogin $command :::: @argfiles + # + # Pass some of the options to the sub-parallels, not all of them as + # -P should only go to the first, and -S should not be copied at all. + my $options = + join(" ", + ((defined $opt::jobs) ? "-P $opt::jobs" : ""), + ((defined $opt::linebuffer) ? "--linebuffer" : ""), + ((defined $opt::ungroup) ? "-u" : ""), + ((defined $opt::group) ? "-g" : ""), + ((defined $opt::keeporder) ? "--keeporder" : ""), + ((defined $opt::D) ? "-D $opt::D" : ""), + ((defined $opt::plain) ? "--plain" : ""), + ((defined $opt::max_chars) ? "--max-chars ".$opt::max_chars : ""), + ); + my $suboptions = + join(" ", + ((defined $opt::ungroup) ? "-u" : ""), + ((defined $opt::linebuffer) ? "--linebuffer" : ""), + ((defined $opt::group) ? "-g" : ""), + ((defined $opt::files) ? "--files" : ""), + ((defined $opt::keeporder) ? "--keeporder" : ""), + ((defined $opt::colsep) ? "--colsep ".shell_quote($opt::colsep) : ""), + ((@opt::v) ? "-vv" : ""), + ((defined $opt::D) ? "-D $opt::D" : ""), + ((defined $opt::timeout) ? "--timeout ".$opt::timeout : ""), + ((defined $opt::plain) ? "--plain" : ""), + ((defined $opt::retries) ? "--retries ".$opt::retries : ""), + ((defined $opt::max_chars) ? "--max-chars ".$opt::max_chars : ""), + ((defined $opt::arg_sep) ? "--arg-sep ".$opt::arg_sep : ""), + ((defined $opt::arg_file_sep) ? "--arg-file-sep ".$opt::arg_file_sep : ""), + (@opt::env ? map { "--env ".::shell_quote_scalar($_) } @opt::env : ""), + ); + ::debug("init", "| $0 $options\n"); + open(my $parallel_fh, "|-", "$0 --no-notice -j0 $options") || + ::die_bug("This does not run GNU Parallel: $0 $options"); + my @joblogs; + for my $host (sort keys %Global::host) { + my $sshlogin = $Global::host{$host}; + my $joblog = tmp_joblog($opt::joblog); + if($joblog) { + push @joblogs, $joblog; + $joblog = "--joblog $joblog"; + } + my $quad = $opt::arg_file_sep || "::::"; + ::debug("init", "$0 $suboptions -j1 $joblog ", + ((defined $opt::tag) ? + "--tagstring ".shell_quote_scalar($sshlogin->string()) : ""), + " -S ", shell_quote_scalar($sshlogin->string())," ", + join(" ",shell_quote(@command))," $quad @argfiles\n"); + print $parallel_fh "$0 $suboptions -j1 $joblog ", + ((defined $opt::tag) ? + "--tagstring ".shell_quote_scalar($sshlogin->string()) : ""), + " -S ", shell_quote_scalar($sshlogin->string())," ", + join(" ",shell_quote(@command))," $quad @argfiles\n"; + } + close $parallel_fh; + $Global::exitstatus = $? >> 8; + debug("init", "--onall exitvalue ", $?); + if(@opt::basefile) { cleanup_basefile(); } + $Global::debug or unlink(@argfiles); + my %seen; + for my $joblog (@joblogs) { + # Append to $joblog + open(my $fh, "<", $joblog) || ::die_bug("Cannot open tmp joblog $joblog"); + # Skip first line (header); + <$fh>; + print $Global::joblog (<$fh>); + close $fh; + unlink($joblog); + } +} + +sub __SIGNAL_HANDLING__ {} + +sub save_original_signal_handler { + # Remember the original signal handler + # Returns: N/A + $SIG{TERM} ||= sub { exit 0; }; # $SIG{TERM} is not set on Mac OS X + $SIG{INT} = sub { if($opt::tmux) { qx { tmux kill-session -t p$$ }; } + unlink keys %Global::unlink; exit -1 }; + $SIG{TERM} = sub { if($opt::tmux) { qx { tmux kill-session -t p$$ }; } + unlink keys %Global::unlink; exit -1 }; + %Global::original_sig = %SIG; + $SIG{TERM} = sub {}; # Dummy until jobs really start +} + +sub list_running_jobs { + # Returns: N/A + for my $v (values %Global::running) { + print $Global::original_stderr "$Global::progname: ",$v->replaced(),"\n"; + } +} + +sub start_no_new_jobs { + # Returns: N/A + $SIG{TERM} = $Global::original_sig{TERM}; + print $Global::original_stderr + ("$Global::progname: SIGTERM received. No new jobs will be started.\n", + "$Global::progname: Waiting for these ", scalar(keys %Global::running), + " jobs to finish. Send SIGTERM again to stop now.\n"); + list_running_jobs(); + $Global::start_no_new_jobs ||= 1; +} + +sub reaper { + # A job finished. + # Print the output. + # Start another job + # Returns: N/A + my $stiff; + my $children_reaped = 0; + debug("run", "Reaper "); + while (($stiff = waitpid(-1, &WNOHANG)) > 0) { + $children_reaped++; + if($Global::sshmaster{$stiff}) { + # This is one of the ssh -M: ignore + next; + } + my $job = $Global::running{$stiff}; + # '-a <(seq 10)' will give us a pid not in %Global::running + $job or next; + $job->set_exitstatus($? >> 8); + $job->set_exitsignal($? & 127); + debug("run", "died (", $job->exitstatus(), "): ", $job->seq()); + $job->set_endtime(::now()); + if($stiff == $Global::tty_taken) { + # The process that died had the tty => release it + $Global::tty_taken = 0; + } + + if(not $job->should_be_retried()) { + # The job is done + # Free the jobslot + push @Global::slots, $job->slot(); + if($opt::timeout) { + # Update average runtime for timeout + $Global::timeoutq->update_delta_time($job->runtime()); + } + # Force printing now if the job failed and we are going to exit + my $print_now = ($opt::halt_on_error and $opt::halt_on_error == 2 + and $job->exitstatus()); + if($opt::keeporder and not $print_now) { + print_earlier_jobs($job); + } else { + $job->print(); + } + if($job->exitstatus()) { + process_failed_job($job); + } + + } + my $sshlogin = $job->sshlogin(); + $sshlogin->dec_jobs_running(); + $sshlogin->inc_jobs_completed(); + $Global::total_running--; + delete $Global::running{$stiff}; + start_more_jobs(); + } + debug("run", "done "); + return $children_reaped; +} + +sub process_failed_job { + # The jobs had a exit status <> 0, so error + # Returns: N/A + my $job = shift; + $Global::exitstatus++; + $Global::total_failed++; + if($opt::halt_on_error) { + if($opt::halt_on_error == 1 + or + ($opt::halt_on_error < 1 and $Global::total_failed > 3 + and + $Global::total_failed / $Global::total_started > $opt::halt_on_error)) { + # If halt on error == 1 or --halt 10% + # we should gracefully exit + print $Global::original_stderr + ("$Global::progname: Starting no more jobs. ", + "Waiting for ", scalar(keys %Global::running), + " jobs to finish. This job failed:\n", + $job->replaced(),"\n"); + $Global::start_no_new_jobs ||= 1; + $Global::halt_on_error_exitstatus = $job->exitstatus(); + } elsif($opt::halt_on_error == 2) { + # If halt on error == 2 we should exit immediately + print $Global::original_stderr + ("$Global::progname: This job failed:\n", + $job->replaced(),"\n"); + exit ($job->exitstatus()); + } + } +} + +{ + my (%print_later,$job_end_sequence); + + sub print_earlier_jobs { + # Print jobs completed earlier + # Returns: N/A + my $job = shift; + $print_later{$job->seq()} = $job; + $job_end_sequence ||= 1; + debug("run", "Looking for: $job_end_sequence ", + "Current: ", $job->seq(), "\n"); + for(my $j = $print_later{$job_end_sequence}; + $j or vec($Global::job_already_run,$job_end_sequence,1); + $job_end_sequence++, + $j = $print_later{$job_end_sequence}) { + debug("run", "Found job end $job_end_sequence"); + if($j) { + $j->print(); + delete $print_later{$job_end_sequence}; + } + } + } +} + +sub __USAGE__ {} + +sub wait_and_exit { + # If we do not wait, we sometimes get segfault + # Returns: N/A + my $error = shift; + if($error) { + # Kill all without printing + for my $job (values %Global::running) { + $job->kill("TERM"); + $job->kill("TERM"); + } + } + for (keys %Global::unkilled_children) { + kill 9, $_; + waitpid($_,0); + delete $Global::unkilled_children{$_}; + } + wait(); + exit($error); +} + +sub die_usage { + # Returns: N/A + usage(); + wait_and_exit(255); +} + +sub usage { + # Returns: N/A + print join + ("\n", + "Usage:", + "", + "$Global::progname [options] [command [arguments]] < list_of_arguments", + "$Global::progname [options] [command [arguments]] (::: arguments|:::: argfile(s))...", + "cat ... | $Global::progname --pipe [options] [command [arguments]]", + "", + "-j n Run n jobs in parallel", + "-k Keep same order", + "-X Multiple arguments with context replace", + "--colsep regexp Split input on regexp for positional replacements", + "{} {.} {/} {/.} {#} {%} {= perl code =} Replacement strings", + "{3} {3.} {3/} {3/.} {=3 perl code =} Positional replacement strings", + "With --plus: {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} =", + " {+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...}", + "", + "-S sshlogin Example: foo\@server.example.com", + "--slf .. Use ~/.parallel/sshloginfile as the list of sshlogins", + "--trc {}.bar Shorthand for --transfer --return {}.bar --cleanup", + "--onall Run the given command with argument on all sshlogins", + "--nonall Run the given command with no arguments on all sshlogins", + "", + "--pipe Split stdin (standard input) to multiple jobs.", + "--recend str Record end separator for --pipe.", + "--recstart str Record start separator for --pipe.", + "", + "See 'man $Global::progname' for details", + "", + "When using programs that use GNU Parallel to process data for publication please cite:", + "", + "O. Tange (2011): GNU Parallel - The Command-Line Power Tool,", + ";login: The USENIX Magazine, February 2011:42-47.", + "", + "Or you can get GNU Parallel without this requirement by paying 10000 EUR.", + ""); +} + + +sub citation_notice { + # if --no-notice or --plain: do nothing + # if stderr redirected: do nothing + # if ~/.parallel/will-cite: do nothing + # else: print citation notice to stderr + if($opt::no_notice + or + $opt::plain + or + not -t $Global::original_stderr + or + -e $ENV{'HOME'}."/.parallel/will-cite") { + # skip + } else { + print $Global::original_stderr + ("When using programs that use GNU Parallel to process data for publication please cite:\n", + "\n", + " O. Tange (2011): GNU Parallel - The Command-Line Power Tool,\n", + " ;login: The USENIX Magazine, February 2011:42-47.\n", + "\n", + "This helps funding further development; and it won't cost you a cent.\n", + "Or you can get GNU Parallel without this requirement by paying 10000 EUR.\n", + "\n", + "To silence this citation notice run 'parallel --bibtex' once or use '--no-notice'.\n\n", + ); + flush $Global::original_stderr; + } +} + + +sub warning { + my @w = @_; + my $fh = $Global::original_stderr || *STDERR; + my $prog = $Global::progname || "parallel"; + print $fh $prog, ": Warning: ", @w; +} + + +sub error { + my @w = @_; + my $fh = $Global::original_stderr || *STDERR; + my $prog = $Global::progname || "parallel"; + print $fh $prog, ": Error: ", @w; +} + + +sub die_bug { + my $bugid = shift; + print STDERR + ("$Global::progname: This should not happen. You have found a bug.\n", + "Please contact and include:\n", + "* The version number: $Global::version\n", + "* The bugid: $bugid\n", + "* The command line being run\n", + "* The files being read (put the files on a webserver if they are big)\n", + "\n", + "If you get the error on smaller/fewer files, please include those instead.\n"); + ::wait_and_exit(255); +} + +sub version { + # Returns: N/A + if($opt::tollef and not $opt::gnu) { + print "WARNING: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu.\n"; + } + print join("\n", + "GNU $Global::progname $Global::version", + "Copyright (C) 2007,2008,2009,2010,2011,2012,2013,2014 Ole Tange and Free Software Foundation, Inc.", + "License GPLv3+: GNU GPL version 3 or later ", + "This is free software: you are free to change and redistribute it.", + "GNU $Global::progname comes with no warranty.", + "", + "Web site: http://www.gnu.org/software/${Global::progname}\n", + "When using programs that use GNU Parallel to process data for publication please cite:\n", + "O. Tange (2011): GNU Parallel - The Command-Line Power Tool, ", + ";login: The USENIX Magazine, February 2011:42-47.\n", + "Or you can get GNU Parallel without this requirement by paying 10000 EUR.\n", + ); +} + +sub bibtex { + # Returns: N/A + if($opt::tollef and not $opt::gnu) { + print "WARNING: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu.\n"; + } + print join("\n", + "When using programs that use GNU Parallel to process data for publication please cite:", + "", + "\@article{Tange2011a,", + " title = {GNU Parallel - The Command-Line Power Tool},", + " author = {O. Tange},", + " address = {Frederiksberg, Denmark},", + " journal = {;login: The USENIX Magazine},", + " month = {Feb},", + " number = {1},", + " volume = {36},", + " url = {http://www.gnu.org/s/parallel},", + " year = {2011},", + " pages = {42-47}", + "}", + "", + "(Feel free to use \\nocite{Tange2011a})", + "", + "This helps funding further development.", + "", + "Or you can get GNU Parallel without this requirement by paying 10000 EUR.", + "" + ); + while(not -e $ENV{'HOME'}."/.parallel/will-cite") { + print "\nType: 'will cite' and press enter.\n> "; + my $input = ; + if($input =~ /will cite/i) { + mkdir $ENV{'HOME'}."/.parallel"; + open (my $fh, ">", $ENV{'HOME'}."/.parallel/will-cite") + || ::die_bug("Cannot write: ".$ENV{'HOME'}."/.parallel/will-cite"); + close $fh; + print "\nThank you for your support. It is much appreciated. The citation\n", + "notice is now silenced.\n"; + } + } +} + +sub show_limits { + # Returns: N/A + print("Maximal size of command: ",Limits::Command::real_max_length(),"\n", + "Maximal used size of command: ",Limits::Command::max_length(),"\n", + "\n", + "Execution of will continue now, and it will try to read its input\n", + "and run commands; if this is not what you wanted to happen, please\n", + "press CTRL-D or CTRL-C\n"); +} + +sub __GENERIC_COMMON_FUNCTION__ {} + +sub uniq { + # Remove duplicates and return unique values + return keys %{{ map { $_ => 1 } @_ }}; +} + +sub min { + # Returns: + # Minimum value of array + my $min; + for (@_) { + # Skip undefs + defined $_ or next; + defined $min or do { $min = $_; next; }; # Set $_ to the first non-undef + $min = ($min < $_) ? $min : $_; + } + return $min; +} + +sub max { + # Returns: + # Maximum value of array + my $max; + for (@_) { + # Skip undefs + defined $_ or next; + defined $max or do { $max = $_; next; }; # Set $_ to the first non-undef + $max = ($max > $_) ? $max : $_; + } + return $max; +} + +sub sum { + # Returns: + # Sum of values of array + my @args = @_; + my $sum = 0; + for (@args) { + # Skip undefs + $_ and do { $sum += $_; } + } + return $sum; +} + +sub undef_as_zero { + my $a = shift; + return $a ? $a : 0; +} + +sub undef_as_empty { + my $a = shift; + return $a ? $a : ""; +} + +{ + my $hostname; + sub hostname { + if(not $hostname) { + $hostname = `hostname`; + chomp($hostname); + $hostname ||= "nohostname"; + } + return $hostname; + } +} + +sub which { + # Input: + # @programs = programs to find the path to + # Returns: + # @full_path = full paths to @programs. Nothing if not found + my @which; + for my $prg (@_) { + push @which, map { $_."/".$prg } grep { -x $_."/".$prg } split(":",$ENV{'PATH'}); + } + return @which; +} + +{ + my ($regexp,%fakename); + + sub parent_shell { + # Input: + # $pid = pid to see if (grand)*parent is a shell + # Returns: + # $shellpath = path to shell - undef if no shell found + my $pid = shift; + if(not $regexp) { + # All shells known to mankind + # + # ash bash csh dash fdsh fish fizsh ksh ksh93 mksh pdksh + # posh rbash rush rzsh sash sh static-sh tcsh yash zsh + my @shells = qw(ash bash csh dash fdsh fish fizsh ksh + ksh93 mksh pdksh posh rbash rush rzsh + sash sh static-sh tcsh yash zsh -sh -csh); + # Can be formatted as: + # [sh] -sh sh busybox sh + # /bin/sh /sbin/sh /opt/csw/sh + # NOT: foo.sh sshd crash flush pdflush scosh fsflush ssh + my $shell = "(?:".join("|",@shells).")"; + $regexp = '^((\[)('. $shell. ')(\])|(|\S+/|busybox )('. $shell. '))($| )'; + %fakename = ( + # csh and tcsh disguise themselves as -sh/-csh + "-sh" => ["csh", "tcsh"], + "-csh" => ["tcsh", "csh"], + ); + } + my ($children_of_ref, $parent_of_ref, $name_of_ref) = pid_table(); + my $shellpath; + my $testpid = $pid; + while($testpid) { + ::debug("init", "shell? ". $name_of_ref->{$testpid}."\n"); + if($name_of_ref->{$testpid} =~ /$regexp/o) { + ::debug("init", "which ".($3||$6)." => "); + $shellpath = (which($3 || $6,@{$fakename{$3 || $6}}))[0]; + ::debug("init", "shell path $shellpath\n"); + $shellpath and last; + } + $testpid = $parent_of_ref->{$testpid}; + } + return $shellpath; + } +} + +{ + my %pid_parentpid_cmd; + + sub pid_table { + # Returns: + # %children_of = { pid -> children of pid } + # %parent_of = { pid -> pid of parent } + # %name_of = { pid -> commandname } + + if(not %pid_parentpid_cmd) { + # Filter for SysV-style `ps` + my $sysv = q( ps -ef | perl -ane '1..1 and /^(.*)CO?MM?A?N?D/ and $s=length $1;). + q(s/^.{$s}//; print "@F[1,2] $_"' ); + # BSD-style `ps` + my $bsd = q(ps -o pid,ppid,command -ax); + %pid_parentpid_cmd = + ( + 'aix' => $sysv, + 'cygwin' => $sysv, + 'msys' => $sysv, + 'dec_osf' => $sysv, + 'darwin' => $bsd, + 'dragonfly' => $bsd, + 'freebsd' => $bsd, + 'gnu' => $sysv, + 'hpux' => $sysv, + 'linux' => $sysv, + 'mirbsd' => $bsd, + 'netbsd' => $bsd, + 'nto' => $sysv, + 'openbsd' => $bsd, + 'solaris' => $sysv, + 'svr5' => $sysv, + ); + } + $pid_parentpid_cmd{$^O} or ::die_bug("pid_parentpid_cmd for $^O missing"); + + my (@pidtable,%parent_of,%children_of,%name_of); + # Table with pid -> children of pid + @pidtable = `$pid_parentpid_cmd{$^O}`; + my $p=$$; + for (@pidtable) { + # must match: 24436 21224 busybox ash + /(\S+)\s+(\S+)\s+(\S+.*)/ or ::die_bug("pidtable format: $_"); + $parent_of{$1} = $2; + push @{$children_of{$2}}, $1; + $name_of{$1} = $3; + } + return(\%children_of, \%parent_of, \%name_of); + } +} + +sub reap_usleep { + # Reap dead children. + # If no dead children: Sleep specified amount with exponential backoff + # Input: + # $ms = milliseconds to sleep + # Returns: + # $ms/2+0.001 if children reaped + # $ms*1.1 if no children reaped + my $ms = shift; + if(reaper()) { + # Sleep exponentially shorter (1/2^n) if a job finished + return $ms/2+0.001; + } else { + if($opt::timeout) { + $Global::timeoutq->process_timeouts(); + } + usleep($ms); + Job::exit_if_disk_full(); + if($opt::linebuffer) { + for my $job (values %Global::running) { + $job->print(); + } + } + # Sleep exponentially longer (1.1^n) if a job did not finish + # though at most 1000 ms. + return (($ms < 1000) ? ($ms * 1.1) : ($ms)); + } +} + +sub usleep { + # Sleep this many milliseconds. + # Input: + # $ms = milliseconds to sleep + my $ms = shift; + ::debug(int($ms),"ms "); + select(undef, undef, undef, $ms/1000); +} + +sub now { + # Returns time since epoch as in seconds with 3 decimals + # Uses: + # @Global::use + # Returns: + # $time = time now with millisecond accuracy + if(not $Global::use{"Time::HiRes"}) { + if(eval "use Time::HiRes qw ( time );") { + eval "sub TimeHiRestime { return Time::HiRes::time };"; + } else { + eval "sub TimeHiRestime { return time() };"; + } + $Global::use{"Time::HiRes"} = 1; + } + + return (int(TimeHiRestime()*1000))/1000; +} + +sub multiply_binary_prefix { + # Evalualte numbers with binary prefix + # Ki=2^10, Mi=2^20, Gi=2^30, Ti=2^40, Pi=2^50, Ei=2^70, Zi=2^80, Yi=2^80 + # ki=2^10, mi=2^20, gi=2^30, ti=2^40, pi=2^50, ei=2^70, zi=2^80, yi=2^80 + # K =2^10, M =2^20, G =2^30, T =2^40, P =2^50, E =2^70, Z =2^80, Y =2^80 + # k =10^3, m =10^6, g =10^9, t=10^12, p=10^15, e=10^18, z=10^21, y=10^24 + # 13G = 13*1024*1024*1024 = 13958643712 + # Input: + # $s = string with prefixes + # Returns: + # $value = int with prefixes multiplied + my $s = shift; + $s =~ s/ki/*1024/gi; + $s =~ s/mi/*1024*1024/gi; + $s =~ s/gi/*1024*1024*1024/gi; + $s =~ s/ti/*1024*1024*1024*1024/gi; + $s =~ s/pi/*1024*1024*1024*1024*1024/gi; + $s =~ s/ei/*1024*1024*1024*1024*1024*1024/gi; + $s =~ s/zi/*1024*1024*1024*1024*1024*1024*1024/gi; + $s =~ s/yi/*1024*1024*1024*1024*1024*1024*1024*1024/gi; + $s =~ s/xi/*1024*1024*1024*1024*1024*1024*1024*1024*1024/gi; + + $s =~ s/K/*1024/g; + $s =~ s/M/*1024*1024/g; + $s =~ s/G/*1024*1024*1024/g; + $s =~ s/T/*1024*1024*1024*1024/g; + $s =~ s/P/*1024*1024*1024*1024*1024/g; + $s =~ s/E/*1024*1024*1024*1024*1024*1024/g; + $s =~ s/Z/*1024*1024*1024*1024*1024*1024*1024/g; + $s =~ s/Y/*1024*1024*1024*1024*1024*1024*1024*1024/g; + $s =~ s/X/*1024*1024*1024*1024*1024*1024*1024*1024*1024/g; + + $s =~ s/k/*1000/g; + $s =~ s/m/*1000*1000/g; + $s =~ s/g/*1000*1000*1000/g; + $s =~ s/t/*1000*1000*1000*1000/g; + $s =~ s/p/*1000*1000*1000*1000*1000/g; + $s =~ s/e/*1000*1000*1000*1000*1000*1000/g; + $s =~ s/z/*1000*1000*1000*1000*1000*1000*1000/g; + $s =~ s/y/*1000*1000*1000*1000*1000*1000*1000*1000/g; + $s =~ s/x/*1000*1000*1000*1000*1000*1000*1000*1000*1000/g; + + $s = eval $s; + ::debug($s); + return $s; +} + +sub tmpfile { + # Create tempfile as $TMPDIR/parXXXXX + # Returns: + # $filename = file name created + return ::tempfile(DIR=>$ENV{'TMPDIR'}, TEMPLATE => 'parXXXXX', @_); +} + +sub __DEBUGGING__ {} + +sub debug { + # Uses: + # $Global::debug + # %Global::fd + # Returns: N/A + $Global::debug or return; + @_ = grep { defined $_ ? $_ : "" } @_; + if($Global::debug eq "all" or $Global::debug eq $_[0]) { + if($Global::fd{1}) { + # Original stdout was saved + my $stdout = $Global::fd{1}; + print $stdout @_[1..$#_]; + } else { + print @_[1..$#_]; + } + } +} + +sub my_memory_usage { + # Returns: + # memory usage if found + # 0 otherwise + use strict; + use FileHandle; + + my $pid = $$; + if(-e "/proc/$pid/stat") { + my $fh = FileHandle->new("; + chomp $data; + $fh->close; + + my @procinfo = split(/\s+/,$data); + + return undef_as_zero($procinfo[22]); + } else { + return 0; + } +} + +sub my_size { + # Returns: + # $size = size of object if Devel::Size is installed + # -1 otherwise + my @size_this = (@_); + eval "use Devel::Size qw(size total_size)"; + if ($@) { + return -1; + } else { + return total_size(@_); + } +} + +sub my_dump { + # Returns: + # ascii expression of object if Data::Dump(er) is installed + # error code otherwise + my @dump_this = (@_); + eval "use Data::Dump qw(dump);"; + if ($@) { + # Data::Dump not installed + eval "use Data::Dumper;"; + if ($@) { + my $err = "Neither Data::Dump nor Data::Dumper is installed\n". + "Not dumping output\n"; + print $Global::original_stderr $err; + return $err; + } else { + return Dumper(@dump_this); + } + } else { + # Create a dummy Data::Dump:dump as Hans Schou sometimes has + # it undefined + eval "sub Data::Dump:dump {}"; + eval "use Data::Dump qw(dump);"; + return (Data::Dump::dump(@dump_this)); + } +} + +sub my_croak { + eval "use Carp; 1"; + $Carp::Verbose = 1; + croak(@_); +} + +sub my_carp { + eval "use Carp; 1"; + $Carp::Verbose = 1; + carp(@_); +} + +sub __OBJECT_ORIENTED_PARTS__ {} + +package SSHLogin; + +sub new { + my $class = shift; + my $sshlogin_string = shift; + my $ncpus; + my %hostgroups; + # SSHLogins can have these formats: + # @grp+grp/ncpu//usr/bin/ssh user@server + # ncpu//usr/bin/ssh user@server + # /usr/bin/ssh user@server + # user@server + # ncpu/user@server + # @grp+grp/user@server + if($sshlogin_string =~ s:^\@([^/]+)/?::) { + # Look for SSHLogin hostgroups + %hostgroups = map { $_ => 1 } split(/\+/, $1); + } + if ($sshlogin_string =~ s:^(\d+)/::) { + # Override default autodetected ncpus unless missing + $ncpus = $1; + } + my $string = $sshlogin_string; + # An SSHLogin is always in the hostgroup of its $string-name + $hostgroups{$string} = 1; + @Global::hostgroups{keys %hostgroups} = values %hostgroups; + my @unget = (); + my $no_slash_string = $string; + $no_slash_string =~ s/[^-a-z0-9:]/_/gi; + return bless { + 'string' => $string, + 'jobs_running' => 0, + 'jobs_completed' => 0, + 'maxlength' => undef, + 'max_jobs_running' => undef, + 'orig_max_jobs_running' => undef, + 'ncpus' => $ncpus, + 'hostgroups' => \%hostgroups, + 'sshcommand' => undef, + 'serverlogin' => undef, + 'control_path_dir' => undef, + 'control_path' => undef, + 'time_to_login' => undef, + 'last_login_at' => undef, + 'loadavg_file' => $ENV{'HOME'} . "/.parallel/tmp/loadavg-" . + $no_slash_string, + 'loadavg' => undef, + 'last_loadavg_update' => 0, + 'swap_activity_file' => $ENV{'HOME'} . "/.parallel/tmp/swap_activity-" . + $no_slash_string, + 'swap_activity' => undef, + }, ref($class) || $class; +} + +sub DESTROY { + my $self = shift; + # Remove temporary files if they are created. + unlink $self->{'loadavg_file'}; + unlink $self->{'swap_activity_file'}; +} + +sub string { + my $self = shift; + return $self->{'string'}; +} + +sub jobs_running { + my $self = shift; + + return ($self->{'jobs_running'} || "0"); +} + +sub inc_jobs_running { + my $self = shift; + $self->{'jobs_running'}++; +} + +sub dec_jobs_running { + my $self = shift; + $self->{'jobs_running'}--; +} + +sub set_maxlength { + my $self = shift; + $self->{'maxlength'} = shift; +} + +sub maxlength { + my $self = shift; + return $self->{'maxlength'}; +} + +sub jobs_completed { + my $self = shift; + return $self->{'jobs_completed'}; +} + +sub in_hostgroups { + # Input: + # @hostgroups = the hostgroups to look for + # Returns: + # true if intersection of @hostgroups and the hostgroups of this + # SSHLogin is non-empty + my $self = shift; + return grep { defined $self->{'hostgroups'}{$_} } @_; +} + +sub hostgroups { + my $self = shift; + return keys %{$self->{'hostgroups'}}; +} + +sub inc_jobs_completed { + my $self = shift; + $self->{'jobs_completed'}++; +} + +sub set_max_jobs_running { + my $self = shift; + if(defined $self->{'max_jobs_running'}) { + $Global::max_jobs_running -= $self->{'max_jobs_running'}; + } + $self->{'max_jobs_running'} = shift; + if(defined $self->{'max_jobs_running'}) { + # max_jobs_running could be resat if -j is a changed file + $Global::max_jobs_running += $self->{'max_jobs_running'}; + } + # Initialize orig to the first non-zero value that comes around + $self->{'orig_max_jobs_running'} ||= $self->{'max_jobs_running'}; +} + +sub swapping { + my $self = shift; + my $swapping = $self->swap_activity(); + return (not defined $swapping or $swapping) +} + +sub swap_activity { + # If the currently known swap activity is too old: + # Recompute a new one in the background + # Returns: + # last swap activity computed + my $self = shift; + # Should we update the swap_activity file? + my $update_swap_activity_file = 0; + if(-r $self->{'swap_activity_file'}) { + open(my $swap_fh, "<", $self->{'swap_activity_file'}) || ::die_bug("swap_activity_file-r"); + my $swap_out = <$swap_fh>; + close $swap_fh; + if($swap_out =~ /^(\d+)$/) { + $self->{'swap_activity'} = $1; + ::debug("swap", "New swap_activity: ", $self->{'swap_activity'}); + } + ::debug("swap", "Last update: ", $self->{'last_swap_activity_update'}); + if(time - $self->{'last_swap_activity_update'} > 10) { + # last swap activity update was started 10 seconds ago + ::debug("swap", "Older than 10 sec: ", $self->{'swap_activity_file'}); + $update_swap_activity_file = 1; + } + } else { + ::debug("swap", "No swap_activity file: ", $self->{'swap_activity_file'}); + $self->{'swap_activity'} = undef; + $update_swap_activity_file = 1; + } + if($update_swap_activity_file) { + ::debug("swap", "Updating swap_activity file ", $self->{'swap_activity_file'}); + $self->{'last_swap_activity_update'} = time; + -e $ENV{'HOME'}."/.parallel" or mkdir $ENV{'HOME'}."/.parallel"; + -e $ENV{'HOME'}."/.parallel/tmp" or mkdir $ENV{'HOME'}."/.parallel/tmp"; + my $swap_activity; + $swap_activity = swapactivityscript(); + if($self->{'string'} ne ":") { + $swap_activity = $self->sshcommand() . " " . $self->serverlogin() . " " . + ::shell_quote_scalar($swap_activity); + } + # Run swap_activity measuring. + # As the command can take long to run if run remote + # save it to a tmp file before moving it to the correct file + my $file = $self->{'swap_activity_file'}; + my ($dummy_fh, $tmpfile) = ::tmpfile(SUFFIX => ".swp"); + ::debug("swap", "\n", $swap_activity, "\n"); + qx{ ($swap_activity > $tmpfile && mv $tmpfile $file || rm $tmpfile) & }; + } + return $self->{'swap_activity'}; +} + +{ + my $script; + + sub swapactivityscript { + # Returns: + # shellscript for detecting swap activity + # + # arguments for vmstat are OS dependant + # swap_in and swap_out are in different columns depending on OS + # + if(not $script) { + my %vmstat = ( + # linux: $7*$8 + # $ vmstat 1 2 + # procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- + # r b swpd free buff cache si so bi bo in cs us sy id wa + # 5 0 51208 1701096 198012 18857888 0 0 37 153 28 19 56 11 33 1 + # 3 0 51208 1701288 198012 18857972 0 0 0 0 3638 10412 15 3 82 0 + 'linux' => ['vmstat 1 2 | tail -n1', '$7*$8'], + + # solaris: $6*$7 + # $ vmstat -S 1 2 + # kthr memory page disk faults cpu + # r b w swap free si so pi po fr de sr s3 s4 -- -- in sy cs us sy id + # 0 0 0 4628952 3208408 0 0 3 1 1 0 0 -0 2 0 0 263 613 246 1 2 97 + # 0 0 0 4552504 3166360 0 0 0 0 0 0 0 0 0 0 0 246 213 240 1 1 98 + 'solaris' => ['vmstat -S 1 2 | tail -1', '$6*$7'], + + # darwin (macosx): $21*$22 + # $ vm_stat -c 2 1 + # Mach Virtual Memory Statistics: (page size of 4096 bytes) + # free active specul inactive throttle wired prgable faults copy 0fill reactive purged file-backed anonymous cmprssed cmprssor dcomprs comprs pageins pageout swapins swapouts + # 346306 829050 74871 606027 0 240231 90367 544858K 62343596 270837K 14178 415070 570102 939846 356 370 116 922 4019813 4 0 0 + # 345740 830383 74875 606031 0 239234 90369 2696 359 553 0 0 570110 941179 356 370 0 0 0 0 0 0 + 'darwin' => ['vm_stat -c 2 1 | tail -n1', '$21*$22'], + + # ultrix: $12*$13 + # $ vmstat -S 1 2 + # procs faults cpu memory page disk + # r b w in sy cs us sy id avm fre si so pi po fr de sr s0 + # 1 0 0 4 23 2 3 0 97 7743 217k 0 0 0 0 0 0 0 0 + # 1 0 0 6 40 8 0 1 99 7743 217k 0 0 3 0 0 0 0 0 + 'ultrix' => ['vmstat -S 1 2 | tail -1', '$12*$13'], + + # aix: $6*$7 + # $ vmstat 1 2 + # System configuration: lcpu=1 mem=2048MB + # + # kthr memory page faults cpu + # ----- ----------- ------------------------ ------------ ----------- + # r b avm fre re pi po fr sr cy in sy cs us sy id wa + # 0 0 333933 241803 0 0 0 0 0 0 10 143 90 0 0 99 0 + # 0 0 334125 241569 0 0 0 0 0 0 37 5368 184 0 9 86 5 + 'aix' => ['vmstat 1 2 | tail -n1', '$6*$7'], + + # freebsd: $8*$9 + # $ vmstat -H 1 2 + # procs memory page disks faults cpu + # r b w avm fre flt re pi po fr sr ad0 ad1 in sy cs us sy id + # 1 0 0 596716 19560 32 0 0 0 33 8 0 0 11 220 277 0 0 99 + # 0 0 0 596716 19560 2 0 0 0 0 0 0 0 11 144 263 0 1 99 + 'freebsd' => ['vmstat -H 1 2 | tail -n1', '$8*$9'], + + # mirbsd: $8*$9 + # $ vmstat 1 2 + # procs memory page disks traps cpu + # r b w avm fre flt re pi po fr sr wd0 cd0 int sys cs us sy id + # 0 0 0 25776 164968 34 0 0 0 0 0 0 0 230 259 38 4 0 96 + # 0 0 0 25776 164968 24 0 0 0 0 0 0 0 237 275 37 0 0 100 + 'mirbsd' => ['vmstat 1 2 | tail -n1', '$8*$9'], + + # netbsd: $7*$8 + # $ vmstat 1 2 + # procs memory page disks faults cpu + # r b avm fre flt re pi po fr sr w0 w1 in sy cs us sy id + # 0 0 138452 6012 54 0 0 0 1 2 3 0 4 100 23 0 0 100 + # 0 0 138456 6008 1 0 0 0 0 0 0 0 7 26 19 0 0 100 + 'netbsd' => ['vmstat 1 2 | tail -n1', '$7*$8'], + + # openbsd: $8*$9 + # $ vmstat 1 2 + # procs memory page disks traps cpu + # r b w avm fre flt re pi po fr sr wd0 wd1 int sys cs us sy id + # 0 0 0 76596 109944 73 0 0 0 0 0 0 1 5 259 22 0 1 99 + # 0 0 0 76604 109936 24 0 0 0 0 0 0 0 7 114 20 0 1 99 + 'openbsd' => ['vmstat 1 2 | tail -n1', '$8*$9'], + + # hpux: $8*$9 + # $ vmstat 1 2 + # procs memory page faults cpu + # r b w avm free re at pi po fr de sr in sy cs us sy id + # 1 0 0 247211 216476 4 1 0 0 0 0 0 102 73005 54 6 11 83 + # 1 0 0 247211 216421 43 9 0 0 0 0 0 144 1675 96 25269512791222387000 25269512791222387000 105 + 'hpux' => ['vmstat 1 2 | tail -n1', '$8*$9'], + + # dec_osf (tru64): $11*$12 + # $ vmstat 1 2 + # Virtual Memory Statistics: (pagesize = 8192) + # procs memory pages intr cpu + # r w u act free wire fault cow zero react pin pout in sy cs us sy id + # 3 181 36 51K 1895 8696 348M 59M 122M 259 79M 0 5 218 302 4 1 94 + # 3 181 36 51K 1893 8696 3 15 21 0 28 0 4 81 321 1 1 98 + 'dec_osf' => ['vmstat 1 2 | tail -n1', '$11*$12'], + + # gnu (hurd): $7*$8 + # $ vmstat -k 1 2 + # (pagesize: 4, size: 512288, swap size: 894972) + # free actv inact wired zeroed react pgins pgouts pfaults cowpfs hrat caobj cache swfree + # 371940 30844 89228 20276 298348 0 48192 19016 756105 99808 98% 876 20628 894972 + # 371940 30844 89228 20276 +0 +0 +0 +0 +42 +2 98% 876 20628 894972 + 'gnu' => ['vmstat -k 1 2 | tail -n1', '$7*$8'], + + # -nto (qnx has no swap) + #-irix + #-svr5 (scosysv) + ); + my $perlscript = ""; + for my $os (keys %vmstat) { + #q[ { vmstat 1 2 2> /dev/null || vmstat -c 1 2; } | ]. + # q[ awk 'NR!=4{next} NF==17||NF==16{print $7*$8} NF==22{print $21*$22} {exit}' ]; + $vmstat{$os}[1] =~ s/\$/\\\\\\\$/g; # $ => \\\$ + $perlscript .= 'if($^O eq "'.$os.'") { print `'.$vmstat{$os}[0].' | awk "{print ' . + $vmstat{$os}[1] . '}"` }'; + } + $perlscript = "perl -e " . ::shell_quote_scalar($perlscript); + $script = $Global::envvar. " " .$perlscript; + } + return $script; + } +} + +sub too_fast_remote_login { + my $self = shift; + if($self->{'last_login_at'} and $self->{'time_to_login'}) { + # sshd normally allows 10 simultaneous logins + # A login takes time_to_login + # So time_to_login/5 should be safe + # If now <= last_login + time_to_login/5: Then it is too soon. + my $too_fast = (::now() <= $self->{'last_login_at'} + + $self->{'time_to_login'}/5); + ::debug("run", "Too fast? $too_fast "); + return $too_fast; + } else { + # No logins so far (or time_to_login not computed): it is not too fast + return 0; + } +} + +sub last_login_at { + my $self = shift; + return $self->{'last_login_at'}; +} + +sub set_last_login_at { + my $self = shift; + $self->{'last_login_at'} = shift; +} + +sub loadavg_too_high { + my $self = shift; + my $loadavg = $self->loadavg(); + return (not defined $loadavg or + $loadavg > $self->max_loadavg()); +} + +sub loadavg { + # If the currently know loadavg is too old: + # Recompute a new one in the background + # The load average is computed as the number of processes waiting for disk + # or CPU right now. So it is the server load this instant and not averaged over + # several minutes. This is needed so GNU Parallel will at most start one job + # that will push the load over the limit. + # + # Returns: + # $last_loadavg = last load average computed (undef if none) + my $self = shift; + # Should we update the loadavg file? + my $update_loadavg_file = 0; + if(open(my $load_fh, "<", $self->{'loadavg_file'})) { + local $/ = undef; + my $load_out = <$load_fh>; + close $load_fh; + my $load =()= ($load_out=~/(^[DR]....[^\[])/gm); + if($load > 0) { + # load is overestimated by 1 + $self->{'loadavg'} = $load - 1; + ::debug("load", "New loadavg: ", $self->{'loadavg'}); + } else { + ::die_bug("loadavg_invalid_content: $load_out"); + } + ::debug("load", "Last update: ", $self->{'last_loadavg_update'}); + if(time - $self->{'last_loadavg_update'} > 10) { + # last loadavg was started 10 seconds ago + ::debug("load", time - $self->{'last_loadavg_update'}, " secs old: ", + $self->{'loadavg_file'}); + $update_loadavg_file = 1; + } + } else { + ::debug("load", "No loadavg file: ", $self->{'loadavg_file'}); + $self->{'loadavg'} = undef; + $update_loadavg_file = 1; + } + if($update_loadavg_file) { + ::debug("load", "Updating loadavg file", $self->{'loadavg_file'}, "\n"); + $self->{'last_loadavg_update'} = time; + -e $ENV{'HOME'}."/.parallel" or mkdir $ENV{'HOME'}."/.parallel"; + -e $ENV{'HOME'}."/.parallel/tmp" or mkdir $ENV{'HOME'}."/.parallel/tmp"; + my $cmd = ""; + if($self->{'string'} ne ":") { + $cmd = $self->sshcommand() . " " . $self->serverlogin() . " "; + } + # TODO Is is called 'ps ax -o state,command' on other platforms? + $cmd .= "ps ax -o state,command"; + # As the command can take long to run if run remote + # save it to a tmp file before moving it to the correct file + my $file = $self->{'loadavg_file'}; + my ($dummy_fh, $tmpfile) = ::tmpfile(SUFFIX => ".loa"); + qx{ ($cmd > $tmpfile && mv $tmpfile $file || rm $tmpfile) & }; + } + return $self->{'loadavg'}; +} + +sub max_loadavg { + my $self = shift; + # If --load is a file it might be changed + if($Global::max_load_file) { + my $mtime = (stat($Global::max_load_file))[9]; + if($mtime > $Global::max_load_file_last_mod) { + $Global::max_load_file_last_mod = $mtime; + for my $sshlogin (values %Global::host) { + $sshlogin->set_max_loadavg(undef); + } + } + } + if(not defined $self->{'max_loadavg'}) { + $self->{'max_loadavg'} = + $self->compute_max_loadavg($opt::load); + } + ::debug("load", "max_loadavg: ", $self->string(), " ", $self->{'max_loadavg'}); + return $self->{'max_loadavg'}; +} + +sub set_max_loadavg { + my $self = shift; + $self->{'max_loadavg'} = shift; +} + +sub compute_max_loadavg { + # Parse the max loadaverage that the user asked for using --load + # Returns: + # max loadaverage + my $self = shift; + my $loadspec = shift; + my $load; + if(defined $loadspec) { + if($loadspec =~ /^\+(\d+)$/) { + # E.g. --load +2 + my $j = $1; + $load = + $self->ncpus() + $j; + } elsif ($loadspec =~ /^-(\d+)$/) { + # E.g. --load -2 + my $j = $1; + $load = + $self->ncpus() - $j; + } elsif ($loadspec =~ /^(\d+)\%$/) { + my $j = $1; + $load = + $self->ncpus() * $j / 100; + } elsif ($loadspec =~ /^(\d+(\.\d+)?)$/) { + $load = $1; + } elsif (-f $loadspec) { + $Global::max_load_file = $loadspec; + $Global::max_load_file_last_mod = (stat($Global::max_load_file))[9]; + if(open(my $in_fh, "<", $Global::max_load_file)) { + my $opt_load_file = join("",<$in_fh>); + close $in_fh; + $load = $self->compute_max_loadavg($opt_load_file); + } else { + print $Global::original_stderr "Cannot open $loadspec\n"; + ::wait_and_exit(255); + } + } else { + print $Global::original_stderr "Parsing of --load failed\n"; + ::die_usage(); + } + if($load < 0.01) { + $load = 0.01; + } + } + return $load; +} + +sub time_to_login { + my $self = shift; + return $self->{'time_to_login'}; +} + +sub set_time_to_login { + my $self = shift; + $self->{'time_to_login'} = shift; +} + +sub max_jobs_running { + my $self = shift; + if(not defined $self->{'max_jobs_running'}) { + my $nproc = $self->compute_number_of_processes($opt::jobs); + $self->set_max_jobs_running($nproc); + } + return $self->{'max_jobs_running'}; +} + +sub orig_max_jobs_running { + my $self = shift; + return $self->{'orig_max_jobs_running'}; +} + +sub compute_number_of_processes { + # Number of processes wanted and limited by system resources + # Returns: + # Number of processes + my $self = shift; + my $opt_P = shift; + my $wanted_processes = $self->user_requested_processes($opt_P); + if(not defined $wanted_processes) { + $wanted_processes = $Global::default_simultaneous_sshlogins; + } + ::debug("load", "Wanted procs: $wanted_processes\n"); + my $system_limit = + $self->processes_available_by_system_limit($wanted_processes); + ::debug("load", "Limited to procs: $system_limit\n"); + return $system_limit; +} + +sub processes_available_by_system_limit { + # If the wanted number of processes is bigger than the system limits: + # Limit them to the system limits + # Limits are: File handles, number of input lines, processes, + # and taking > 1 second to spawn 10 extra processes + # Returns: + # Number of processes + my $self = shift; + my $wanted_processes = shift; + + my $system_limit = 0; + my @jobs = (); + my $job; + my @args = (); + my $arg; + my $more_filehandles = 1; + my $max_system_proc_reached = 0; + my $slow_spawining_warning_printed = 0; + my $time = time; + my %fh; + my @children; + + # Reserve filehandles + # perl uses 7 filehandles for something? + # parallel uses 1 for memory_usage + # parallel uses 4 for ? + for my $i (1..12) { + open($fh{"init-$i"}, "<", "/dev/null"); + } + + for(1..2) { + # System process limit + my $child; + if($child = fork()) { + push (@children,$child); + $Global::unkilled_children{$child} = 1; + } elsif(defined $child) { + # The child takes one process slot + # It will be killed later + $SIG{TERM} = $Global::original_sig{TERM}; + sleep 10000000; + exit(0); + } else { + $max_system_proc_reached = 1; + } + } + my $count_jobs_already_read = $Global::JobQueue->next_seq(); + my $wait_time_for_getting_args = 0; + my $start_time = time; + while(1) { + $system_limit >= $wanted_processes and last; + not $more_filehandles and last; + $max_system_proc_reached and last; + my $before_getting_arg = time; + if($Global::semaphore or $opt::pipe) { + # Skip: No need to get args + } elsif(defined $opt::retries and $count_jobs_already_read) { + # For retries we may need to run all jobs on this sshlogin + # so include the already read jobs for this sshlogin + $count_jobs_already_read--; + } else { + if($opt::X or $opt::m) { + # The arguments may have to be re-spread over several jobslots + # So pessimistically only read one arg per jobslot + # instead of a full commandline + if($Global::JobQueue->{'commandlinequeue'}->{'arg_queue'}->empty()) { + if($Global::JobQueue->empty()) { + last; + } else { + ($job) = $Global::JobQueue->get(); + push(@jobs, $job); + } + } else { + ($arg) = $Global::JobQueue->{'commandlinequeue'}->{'arg_queue'}->get(); + push(@args, $arg); + } + } else { + # If there are no more command lines, then we have a process + # per command line, so no need to go further + $Global::JobQueue->empty() and last; + ($job) = $Global::JobQueue->get(); + push(@jobs, $job); + } + } + $wait_time_for_getting_args += time - $before_getting_arg; + $system_limit++; + + # Every simultaneous process uses 2 filehandles when grouping + # Every simultaneous process uses 2 filehandles when compressing + $more_filehandles = open($fh{$system_limit*10}, "<", "/dev/null") + && open($fh{$system_limit*10+2}, "<", "/dev/null") + && open($fh{$system_limit*10+3}, "<", "/dev/null") + && open($fh{$system_limit*10+4}, "<", "/dev/null"); + + # System process limit + my $child; + if($child = fork()) { + push (@children,$child); + $Global::unkilled_children{$child} = 1; + } elsif(defined $child) { + # The child takes one process slot + # It will be killed later + $SIG{TERM} = $Global::original_sig{TERM}; + sleep 10000000; + exit(0); + } else { + $max_system_proc_reached = 1; + } + my $forktime = time - $time - $wait_time_for_getting_args; + ::debug("run", "Time to fork $system_limit procs: $wait_time_for_getting_args ", + $forktime, + " (processes so far: ", $system_limit,")\n"); + if($system_limit > 10 and + $forktime > 1 and + $forktime > $system_limit * 0.01 + and not $slow_spawining_warning_printed) { + # It took more than 0.01 second to fork a processes on avg. + # Give the user a warning. He can press Ctrl-C if this + # sucks. + print $Global::original_stderr + ("parallel: Warning: Starting $system_limit processes took > $forktime sec.\n", + "Consider adjusting -j. Press CTRL-C to stop.\n"); + $slow_spawining_warning_printed = 1; + } + } + # Cleanup: Close the files + for (values %fh) { close $_ } + # Cleanup: Kill the children + for my $pid (@children) { + kill 9, $pid; + waitpid($pid,0); + delete $Global::unkilled_children{$pid}; + } + # Cleanup: Unget the command_lines or the @args + $Global::JobQueue->{'commandlinequeue'}->{'arg_queue'}->unget(@args); + $Global::JobQueue->unget(@jobs); + if($system_limit < $wanted_processes) { + # The system_limit is less than the wanted_processes + if($system_limit < 1 and not $Global::JobQueue->empty()) { + ::warning("Cannot spawn any jobs. Raising ulimit -u or /etc/security/limits.conf\n", + "or /proc/sys/kernel/pid_max may help.\n"); + ::wait_and_exit(255); + } + if(not $more_filehandles) { + ::warning("Only enough file handles to run ", $system_limit, " jobs in parallel.\n", + "Running 'parallel -j0 -N", $system_limit, " --pipe parallel -j0' or ", + "raising ulimit -n or /etc/security/limits.conf may help.\n"); + } + if($max_system_proc_reached) { + ::warning("Only enough available processes to run ", $system_limit, + " jobs in parallel. Raising ulimit -u or /etc/security/limits.conf\n", + "or /proc/sys/kernel/pid_max may help.\n"); + } + } + if($] == 5.008008 and $system_limit > 1000) { + # https://savannah.gnu.org/bugs/?36942 + $system_limit = 1000; + } + if($Global::JobQueue->empty()) { + $system_limit ||= 1; + } + if($self->string() ne ":" and + $system_limit > $Global::default_simultaneous_sshlogins) { + $system_limit = + $self->simultaneous_sshlogin_limit($system_limit); + } + return $system_limit; +} + +sub simultaneous_sshlogin_limit { + # Test by logging in wanted number of times simultaneously + # Returns: + # min($wanted_processes,$working_simultaneous_ssh_logins-1) + my $self = shift; + my $wanted_processes = shift; + if($self->{'time_to_login'}) { + return $wanted_processes; + } + + # Try twice because it guesses wrong sometimes + # Choose the minimal + my $ssh_limit = + ::min($self->simultaneous_sshlogin($wanted_processes), + $self->simultaneous_sshlogin($wanted_processes)); + if($ssh_limit < $wanted_processes) { + my $serverlogin = $self->serverlogin(); + ::warning("ssh to $serverlogin only allows ", + "for $ssh_limit simultaneous logins.\n", + "You may raise this by changing ", + "/etc/ssh/sshd_config:MaxStartups and MaxSessions on $serverlogin.\n", + "Using only ",$ssh_limit-1," connections ", + "to avoid race conditions.\n"); + } + # Race condition can cause problem if using all sshs. + if($ssh_limit > 1) { $ssh_limit -= 1; } + return $ssh_limit; +} + +sub simultaneous_sshlogin { + # Using $sshlogin try to see if we can do $wanted_processes + # simultaneous logins + # (ssh host echo simultaneouslogin & ssh host echo simultaneouslogin & ...)|grep simul|wc -l + # Returns: + # Number of succesful logins + my $self = shift; + my $wanted_processes = shift; + my $sshcmd = $self->sshcommand(); + my $serverlogin = $self->serverlogin(); + my $sshdelay = $opt::sshdelay ? "sleep $opt::sshdelay;" : ""; + my $cmd = "$sshdelay$sshcmd $serverlogin echo simultaneouslogin &1 &"x$wanted_processes; + ::debug("init", "Trying $wanted_processes logins at $serverlogin\n"); + open (my $simul_fh, "-|", "($cmd)|grep simultaneouslogin | wc -l") or + ::die_bug("simultaneouslogin"); + my $ssh_limit = <$simul_fh>; + close $simul_fh; + chomp $ssh_limit; + return $ssh_limit; +} + +sub set_ncpus { + my $self = shift; + $self->{'ncpus'} = shift; +} + +sub user_requested_processes { + # Parse the number of processes that the user asked for using -j + # Returns: + # the number of processes to run on this sshlogin + my $self = shift; + my $opt_P = shift; + my $processes; + if(defined $opt_P) { + if($opt_P =~ /^\+(\d+)$/) { + # E.g. -P +2 + my $j = $1; + $processes = + $self->ncpus() + $j; + } elsif ($opt_P =~ /^-(\d+)$/) { + # E.g. -P -2 + my $j = $1; + $processes = + $self->ncpus() - $j; + } elsif ($opt_P =~ /^(\d+(\.\d+)?)\%$/) { + # E.g. -P 10.5% + my $j = $1; + $processes = + $self->ncpus() * $j / 100; + } elsif ($opt_P =~ /^(\d+)$/) { + $processes = $1; + if($processes == 0) { + # -P 0 = infinity (or at least close) + $processes = $Global::infinity; + } + } elsif (-f $opt_P) { + $Global::max_procs_file = $opt_P; + $Global::max_procs_file_last_mod = (stat($Global::max_procs_file))[9]; + if(open(my $in_fh, "<", $Global::max_procs_file)) { + my $opt_P_file = join("",<$in_fh>); + close $in_fh; + $processes = $self->user_requested_processes($opt_P_file); + } else { + ::error("Cannot open $opt_P.\n"); + ::wait_and_exit(255); + } + } else { + ::error("Parsing of --jobs/-j/--max-procs/-P failed.\n"); + ::die_usage(); + } + $processes = ::ceil($processes); + } + return $processes; +} + +sub ncpus { + my $self = shift; + if(not defined $self->{'ncpus'}) { + my $sshcmd = $self->sshcommand(); + my $serverlogin = $self->serverlogin(); + if($serverlogin eq ":") { + if($opt::use_cpus_instead_of_cores) { + $self->{'ncpus'} = no_of_cpus(); + } else { + $self->{'ncpus'} = no_of_cores(); + } + } else { + my $ncpu; + my $sqe = ::shell_quote_scalar($Global::envvar); + if($opt::use_cpus_instead_of_cores) { + $ncpu = qx(echo|$sshcmd $serverlogin $sqe parallel --number-of-cpus); + } else { + ::debug("init",qq(echo|$sshcmd $serverlogin $sqe parallel --number-of-cores\n)); + $ncpu = qx(echo|$sshcmd $serverlogin $sqe parallel --number-of-cores); + } + chomp $ncpu; + if($ncpu =~ /^\s*[0-9]+\s*$/s) { + $self->{'ncpus'} = $ncpu; + } else { + ::warning("Could not figure out ", + "number of cpus on $serverlogin ($ncpu). Using 1.\n"); + $self->{'ncpus'} = 1; + } + } + } + return $self->{'ncpus'}; +} + +sub no_of_cpus { + # Returns: + # Number of physical CPUs + local $/="\n"; # If delimiter is set, then $/ will be wrong + my $no_of_cpus; + if ($^O eq 'linux') { + $no_of_cpus = no_of_cpus_gnu_linux() || no_of_cores_gnu_linux(); + } elsif ($^O eq 'freebsd') { + $no_of_cpus = no_of_cpus_freebsd(); + } elsif ($^O eq 'netbsd') { + $no_of_cpus = no_of_cpus_netbsd(); + } elsif ($^O eq 'openbsd') { + $no_of_cpus = no_of_cpus_openbsd(); + } elsif ($^O eq 'gnu') { + $no_of_cpus = no_of_cpus_hurd(); + } elsif ($^O eq 'darwin') { + $no_of_cpus = no_of_cpus_darwin(); + } elsif ($^O eq 'solaris') { + $no_of_cpus = no_of_cpus_solaris(); + } elsif ($^O eq 'aix') { + $no_of_cpus = no_of_cpus_aix(); + } elsif ($^O eq 'hpux') { + $no_of_cpus = no_of_cpus_hpux(); + } elsif ($^O eq 'nto') { + $no_of_cpus = no_of_cpus_qnx(); + } elsif ($^O eq 'svr5') { + $no_of_cpus = no_of_cpus_openserver(); + } elsif ($^O eq 'irix') { + $no_of_cpus = no_of_cpus_irix(); + } elsif ($^O eq 'dec_osf') { + $no_of_cpus = no_of_cpus_tru64(); + } else { + $no_of_cpus = (no_of_cpus_gnu_linux() + || no_of_cpus_freebsd() + || no_of_cpus_netbsd() + || no_of_cpus_openbsd() + || no_of_cpus_hurd() + || no_of_cpus_darwin() + || no_of_cpus_solaris() + || no_of_cpus_aix() + || no_of_cpus_hpux() + || no_of_cpus_qnx() + || no_of_cpus_openserver() + || no_of_cpus_irix() + || no_of_cpus_tru64() + # Number of cores is better than no guess for #CPUs + || nproc() + ); + } + if($no_of_cpus) { + chomp $no_of_cpus; + return $no_of_cpus; + } else { + ::warning("Cannot figure out number of cpus. Using 1.\n"); + return 1; + } +} + +sub no_of_cores { + # Returns: + # Number of CPU cores + local $/="\n"; # If delimiter is set, then $/ will be wrong + my $no_of_cores; + if ($^O eq 'linux') { + $no_of_cores = no_of_cores_gnu_linux(); + } elsif ($^O eq 'freebsd') { + $no_of_cores = no_of_cores_freebsd(); + } elsif ($^O eq 'netbsd') { + $no_of_cores = no_of_cores_netbsd(); + } elsif ($^O eq 'openbsd') { + $no_of_cores = no_of_cores_openbsd(); + } elsif ($^O eq 'gnu') { + $no_of_cores = no_of_cores_hurd(); + } elsif ($^O eq 'darwin') { + $no_of_cores = no_of_cores_darwin(); + } elsif ($^O eq 'solaris') { + $no_of_cores = no_of_cores_solaris(); + } elsif ($^O eq 'aix') { + $no_of_cores = no_of_cores_aix(); + } elsif ($^O eq 'hpux') { + $no_of_cores = no_of_cores_hpux(); + } elsif ($^O eq 'nto') { + $no_of_cores = no_of_cores_qnx(); + } elsif ($^O eq 'svr5') { + $no_of_cores = no_of_cores_openserver(); + } elsif ($^O eq 'irix') { + $no_of_cores = no_of_cores_irix(); + } elsif ($^O eq 'dec_osf') { + $no_of_cores = no_of_cores_tru64(); + } else { + $no_of_cores = (no_of_cores_gnu_linux() + || no_of_cores_freebsd() + || no_of_cores_netbsd() + || no_of_cores_openbsd() + || no_of_cores_hurd() + || no_of_cores_darwin() + || no_of_cores_solaris() + || no_of_cores_aix() + || no_of_cores_hpux() + || no_of_cores_qnx() + || no_of_cores_openserver() + || no_of_cores_irix() + || no_of_cores_tru64() + || nproc() + ); + } + if($no_of_cores) { + chomp $no_of_cores; + return $no_of_cores; + } else { + ::warning("Cannot figure out number of CPU cores. Using 1.\n"); + return 1; + } +} + +sub nproc { + # Returns: + # Number of cores using `nproc` + my $no_of_cores = `nproc 2>/dev/null`; + return $no_of_cores; +} + +sub no_of_cpus_gnu_linux { + # Returns: + # Number of physical CPUs on GNU/Linux + # undef if not GNU/Linux + my $no_of_cpus; + my $no_of_cores; + if(-e "/proc/cpuinfo") { + $no_of_cpus = 0; + $no_of_cores = 0; + my %seen; + open(my $in_fh, "<", "/proc/cpuinfo") || return undef; + while(<$in_fh>) { + if(/^physical id.*[:](.*)/ and not $seen{$1}++) { + $no_of_cpus++; + } + /^processor.*[:]/i and $no_of_cores++; + } + close $in_fh; + } + return ($no_of_cpus||$no_of_cores); +} + +sub no_of_cores_gnu_linux { + # Returns: + # Number of CPU cores on GNU/Linux + # undef if not GNU/Linux + my $no_of_cores; + if(-e "/proc/cpuinfo") { + $no_of_cores = 0; + open(my $in_fh, "<", "/proc/cpuinfo") || return undef; + while(<$in_fh>) { + /^processor.*[:]/i and $no_of_cores++; + } + close $in_fh; + } + return $no_of_cores; +} + +sub no_of_cpus_freebsd { + # Returns: + # Number of physical CPUs on FreeBSD + # undef if not FreeBSD + my $no_of_cpus = + (`sysctl -a dev.cpu 2>/dev/null | grep \%parent | awk '{ print \$2 }' | uniq | wc -l | awk '{ print \$1 }'` + or + `sysctl hw.ncpu 2>/dev/null | awk '{ print \$2 }'`); + chomp $no_of_cpus; + return $no_of_cpus; +} + +sub no_of_cores_freebsd { + # Returns: + # Number of CPU cores on FreeBSD + # undef if not FreeBSD + my $no_of_cores = + (`sysctl hw.ncpu 2>/dev/null | awk '{ print \$2 }'` + or + `sysctl -a hw 2>/dev/null | grep [^a-z]logicalcpu[^a-z] | awk '{ print \$2 }'`); + chomp $no_of_cores; + return $no_of_cores; +} + +sub no_of_cpus_netbsd { + # Returns: + # Number of physical CPUs on NetBSD + # undef if not NetBSD + my $no_of_cpus = `sysctl -n hw.ncpu 2>/dev/null`; + chomp $no_of_cpus; + return $no_of_cpus; +} + +sub no_of_cores_netbsd { + # Returns: + # Number of CPU cores on NetBSD + # undef if not NetBSD + my $no_of_cores = `sysctl -n hw.ncpu 2>/dev/null`; + chomp $no_of_cores; + return $no_of_cores; +} + +sub no_of_cpus_openbsd { + # Returns: + # Number of physical CPUs on OpenBSD + # undef if not OpenBSD + my $no_of_cpus = `sysctl -n hw.ncpu 2>/dev/null`; + chomp $no_of_cpus; + return $no_of_cpus; +} + +sub no_of_cores_openbsd { + # Returns: + # Number of CPU cores on OpenBSD + # undef if not OpenBSD + my $no_of_cores = `sysctl -n hw.ncpu 2>/dev/null`; + chomp $no_of_cores; + return $no_of_cores; +} + +sub no_of_cpus_hurd { + # Returns: + # Number of physical CPUs on HURD + # undef if not HURD + my $no_of_cpus = `nproc`; + chomp $no_of_cpus; + return $no_of_cpus; +} + +sub no_of_cores_hurd { + # Returns: + # Number of physical CPUs on HURD + # undef if not HURD + my $no_of_cores = `nproc`; + chomp $no_of_cores; + return $no_of_cores; +} + +sub no_of_cpus_darwin { + # Returns: + # Number of physical CPUs on Mac Darwin + # undef if not Mac Darwin + my $no_of_cpus = + (`sysctl -n hw.physicalcpu 2>/dev/null` + or + `sysctl -a hw 2>/dev/null | grep [^a-z]physicalcpu[^a-z] | awk '{ print \$2 }'`); + return $no_of_cpus; +} + +sub no_of_cores_darwin { + # Returns: + # Number of CPU cores on Mac Darwin + # undef if not Mac Darwin + my $no_of_cores = + (`sysctl -n hw.logicalcpu 2>/dev/null` + or + `sysctl -a hw 2>/dev/null | grep [^a-z]logicalcpu[^a-z] | awk '{ print \$2 }'`); + return $no_of_cores; +} + +sub no_of_cpus_solaris { + # Returns: + # Number of physical CPUs on Solaris + # undef if not Solaris + if(-x "/usr/sbin/psrinfo") { + my @psrinfo = `/usr/sbin/psrinfo`; + if($#psrinfo >= 0) { + return $#psrinfo +1; + } + } + if(-x "/usr/sbin/prtconf") { + my @prtconf = `/usr/sbin/prtconf | grep cpu..instance`; + if($#prtconf >= 0) { + return $#prtconf +1; + } + } + return undef; +} + +sub no_of_cores_solaris { + # Returns: + # Number of CPU cores on Solaris + # undef if not Solaris + if(-x "/usr/sbin/psrinfo") { + my @psrinfo = `/usr/sbin/psrinfo`; + if($#psrinfo >= 0) { + return $#psrinfo +1; + } + } + if(-x "/usr/sbin/prtconf") { + my @prtconf = `/usr/sbin/prtconf | grep cpu..instance`; + if($#prtconf >= 0) { + return $#prtconf +1; + } + } + return undef; +} + +sub no_of_cpus_aix { + # Returns: + # Number of physical CPUs on AIX + # undef if not AIX + my $no_of_cpus = 0; + if(-x "/usr/sbin/lscfg") { + open(my $in_fh, "-|", "/usr/sbin/lscfg -vs |grep proc | wc -l|tr -d ' '") + || return undef; + $no_of_cpus = <$in_fh>; + chomp ($no_of_cpus); + close $in_fh; + } + return $no_of_cpus; +} + +sub no_of_cores_aix { + # Returns: + # Number of CPU cores on AIX + # undef if not AIX + my $no_of_cores; + if(-x "/usr/bin/vmstat") { + open(my $in_fh, "-|", "/usr/bin/vmstat 1 1") || return undef; + while(<$in_fh>) { + /lcpu=([0-9]*) / and $no_of_cores = $1; + } + close $in_fh; + } + return $no_of_cores; +} + +sub no_of_cpus_hpux { + # Returns: + # Number of physical CPUs on HP-UX + # undef if not HP-UX + my $no_of_cpus = + (`/usr/bin/mpsched -s 2>&1 | grep 'Locality Domain Count' | awk '{ print \$4 }'`); + return $no_of_cpus; +} + +sub no_of_cores_hpux { + # Returns: + # Number of CPU cores on HP-UX + # undef if not HP-UX + my $no_of_cores = + (`/usr/bin/mpsched -s 2>&1 | grep 'Processor Count' | awk '{ print \$3 }'`); + return $no_of_cores; +} + +sub no_of_cpus_qnx { + # Returns: + # Number of physical CPUs on QNX + # undef if not QNX + # BUG: It is now known how to calculate this. + my $no_of_cpus = 0; + return $no_of_cpus; +} + +sub no_of_cores_qnx { + # Returns: + # Number of CPU cores on QNX + # undef if not QNX + # BUG: It is now known how to calculate this. + my $no_of_cores = 0; + return $no_of_cores; +} + +sub no_of_cpus_openserver { + # Returns: + # Number of physical CPUs on SCO OpenServer + # undef if not SCO OpenServer + my $no_of_cpus = 0; + if(-x "/usr/sbin/psrinfo") { + my @psrinfo = `/usr/sbin/psrinfo`; + if($#psrinfo >= 0) { + return $#psrinfo +1; + } + } + return $no_of_cpus; +} + +sub no_of_cores_openserver { + # Returns: + # Number of CPU cores on SCO OpenServer + # undef if not SCO OpenServer + my $no_of_cores = 0; + if(-x "/usr/sbin/psrinfo") { + my @psrinfo = `/usr/sbin/psrinfo`; + if($#psrinfo >= 0) { + return $#psrinfo +1; + } + } + return $no_of_cores; +} + +sub no_of_cpus_irix { + # Returns: + # Number of physical CPUs on IRIX + # undef if not IRIX + my $no_of_cpus = `hinv | grep HZ | grep Processor | awk '{print \$1}'`; + return $no_of_cpus; +} + +sub no_of_cores_irix { + # Returns: + # Number of CPU cores on IRIX + # undef if not IRIX + my $no_of_cores = `hinv | grep HZ | grep Processor | awk '{print \$1}'`; + return $no_of_cores; +} + +sub no_of_cpus_tru64 { + # Returns: + # Number of physical CPUs on Tru64 + # undef if not Tru64 + my $no_of_cpus = `sizer -pr`; + return $no_of_cpus; +} + +sub no_of_cores_tru64 { + # Returns: + # Number of CPU cores on Tru64 + # undef if not Tru64 + my $no_of_cores = `sizer -pr`; + return $no_of_cores; +} + +sub sshcommand { + my $self = shift; + if (not defined $self->{'sshcommand'}) { + $self->sshcommand_of_sshlogin(); + } + return $self->{'sshcommand'}; +} + +sub serverlogin { + my $self = shift; + if (not defined $self->{'serverlogin'}) { + $self->sshcommand_of_sshlogin(); + } + return $self->{'serverlogin'}; +} + +sub sshcommand_of_sshlogin { + # 'server' -> ('ssh -S /tmp/parallel-ssh-RANDOM/host-','server') + # 'user@server' -> ('ssh','user@server') + # 'myssh user@server' -> ('myssh','user@server') + # 'myssh -l user server' -> ('myssh -l user','server') + # '/usr/bin/myssh -l user server' -> ('/usr/bin/myssh -l user','server') + # Returns: + # sshcommand - defaults to 'ssh' + # login@host + my $self = shift; + my ($sshcmd, $serverlogin); + if($self->{'string'} =~ /(.+) (\S+)$/) { + # Own ssh command + $sshcmd = $1; $serverlogin = $2; + } else { + # Normal ssh + if($opt::controlmaster) { + # Use control_path to make ssh faster + my $control_path = $self->control_path_dir()."/ssh-%r@%h:%p"; + $sshcmd = "ssh -S ".$control_path; + $serverlogin = $self->{'string'}; + if(not $self->{'control_path'}{$control_path}++) { + # Master is not running for this control_path + # Start it + my $pid = fork(); + if($pid) { + $Global::sshmaster{$pid} ||= 1; + } else { + $SIG{'TERM'} = undef; + # Ignore the 'foo' being printed + open(STDOUT,">","/dev/null"); + # OpenSSH_3.6.1p2 gives 'tcgetattr: Invalid argument' with -tt + # STDERR >/dev/null to ignore "process_mux_new_session: tcgetattr: Invalid argument" + open(STDERR,">","/dev/null"); + open(STDIN,"<","/dev/null"); + # Run a sleep that outputs data, so it will discover if the ssh connection closes. + my $sleep = ::shell_quote_scalar('$|=1;while(1){sleep 1;print "foo\n"}'); + my @master = ("ssh", "-tt", "-MTS", $control_path, $serverlogin, "perl", "-e", $sleep); + exec(@master); + } + } + } else { + $sshcmd = "ssh"; $serverlogin = $self->{'string'}; + } + } + $self->{'sshcommand'} = $sshcmd; + $self->{'serverlogin'} = $serverlogin; +} + +sub control_path_dir { + # Returns: + # path to directory + my $self = shift; + if(not defined $self->{'control_path_dir'}) { + -e $ENV{'HOME'}."/.parallel" or mkdir $ENV{'HOME'}."/.parallel"; + -e $ENV{'HOME'}."/.parallel/tmp" or mkdir $ENV{'HOME'}."/.parallel/tmp"; + $self->{'control_path_dir'} = + File::Temp::tempdir($ENV{'HOME'} + . "/.parallel/tmp/control_path_dir-XXXX", + CLEANUP => 1); + } + return $self->{'control_path_dir'}; +} + +sub rsync_transfer_cmd { + # Command to run to transfer a file + # Input: + # $file = filename of file to transfer + # $workdir = destination dir + # Returns: + # $cmd = rsync command to run to transfer $file ("" if unreadable) + my $self = shift; + my $file = shift; + my $workdir = shift; + if(not -r $file) { + ::warning($file, " is not readable and will not be transferred.\n"); + return "true"; + } + my $rsync_destdir; + if($file =~ m:^/:) { + # rsync /foo/bar / + $rsync_destdir = "/"; + } else { + $rsync_destdir = ::shell_quote_file($workdir); + } + $file = ::shell_quote_file($file); + my $sshcmd = $self->sshcommand(); + my $rsync_opt = "-rlDzR -e" . ::shell_quote_scalar($sshcmd); + my $serverlogin = $self->serverlogin(); + # Make dir if it does not exist + return "( $sshcmd $serverlogin mkdir -p $rsync_destdir;" . + rsync()." $rsync_opt $file $serverlogin:$rsync_destdir )"; +} + +sub cleanup_cmd { + # Command to run to remove the remote file + # Input: + # $file = filename to remove + # $workdir = destination dir + # Returns: + # $cmd = ssh command to run to remove $file and empty parent dirs + my $self = shift; + my $file = shift; + my $workdir = shift; + my $f = $file; + if($f =~ m:/\./:) { + # foo/bar/./baz/quux => workdir/baz/quux + # /foo/bar/./baz/quux => workdir/baz/quux + $f =~ s:.*/\./:$workdir/:; + } elsif($f =~ m:^[^/]:) { + # foo/bar => workdir/foo/bar + $f = $workdir."/".$f; + } + my @subdirs = split m:/:, ::dirname($f); + my @rmdir; + my $dir = ""; + for(@subdirs) { + $dir .= $_."/"; + unshift @rmdir, ::shell_quote_file($dir); + } + my $rmdir = @rmdir ? "rmdir @rmdir 2>/dev/null;" : ""; + if(defined $opt::workdir and $opt::workdir eq "...") { + $rmdir .= "rm -rf " . ::shell_quote_file($workdir).';'; + } + + $f = ::shell_quote_file($f); + my $sshcmd = $self->sshcommand(); + my $serverlogin = $self->serverlogin(); + return "$sshcmd $serverlogin ".::shell_quote_scalar("(rm -f $f; $rmdir)"); +} + +{ + my $rsync; + + sub rsync { + # rsync 3.1.x uses protocol 31 which is unsupported by 2.5.7. + # If the version >= 3.1.0: downgrade to protocol 30 + if(not $rsync) { + my @out = `rsync --version`; + for (@out) { + if(/version (\d+.\d+)(.\d+)?/) { + if($1 >= 3.1) { + # Version 3.1.0 or later: Downgrade to protocol 30 + $rsync = "rsync --protocol 30"; + } else { + $rsync = "rsync"; + } + } + } + $rsync or ::die_bug("Cannot figure out version of rsync: @out"); + } + return $rsync; + } +} + + +package JobQueue; + +sub new { + my $class = shift; + my $commandref = shift; + my $read_from = shift; + my $context_replace = shift; + my $max_number_of_args = shift; + my $return_files = shift; + my $commandlinequeue = CommandLineQueue->new + ($commandref, $read_from, $context_replace, $max_number_of_args, + $return_files); + my @unget = (); + return bless { + 'unget' => \@unget, + 'commandlinequeue' => $commandlinequeue, + 'total_jobs' => undef, + }, ref($class) || $class; +} + +sub get { + my $self = shift; + + if(@{$self->{'unget'}}) { + my $job = shift @{$self->{'unget'}}; + return ($job); + } else { + my $commandline = $self->{'commandlinequeue'}->get(); + if(defined $commandline) { + my $job = Job->new($commandline); + return $job; + } else { + return undef; + } + } +} + +sub unget { + my $self = shift; + unshift @{$self->{'unget'}}, @_; +} + +sub empty { + my $self = shift; + my $empty = (not @{$self->{'unget'}}) + && $self->{'commandlinequeue'}->empty(); + ::debug("run", "JobQueue->empty $empty "); + return $empty; +} + +sub total_jobs { + my $self = shift; + if(not defined $self->{'total_jobs'}) { + my $job; + my @queue; + my $start = time; + while($job = $self->get()) { + if(time - $start > 10) { + ::warning("Reading all arguments takes longer than 10 seconds.\n"); + $opt::eta && ::warning("Consider removing --eta.\n"); + $opt::bar && ::warning("Consider removing --bar.\n"); + last; + } + push @queue, $job; + } + while($job = $self->get()) { + push @queue, $job; + } + + $self->unget(@queue); + $self->{'total_jobs'} = $#queue+1; + } + return $self->{'total_jobs'}; +} + +sub next_seq { + my $self = shift; + + return $self->{'commandlinequeue'}->seq(); +} + +sub quote_args { + my $self = shift; + return $self->{'commandlinequeue'}->quote_args(); +} + + +package Job; + +sub new { + my $class = shift; + my $commandlineref = shift; + return bless { + 'commandline' => $commandlineref, # CommandLine object + 'workdir' => undef, # --workdir + 'stdin' => undef, # filehandle for stdin (used for --pipe) + # filename for writing stdout to (used for --files) + 'remaining' => "", # remaining data not sent to stdin (used for --pipe) + 'datawritten' => 0, # amount of data sent via stdin (used for --pipe) + 'transfersize' => 0, # size of files using --transfer + 'returnsize' => 0, # size of files using --return + 'pid' => undef, + # hash of { SSHLogins => number of times the command failed there } + 'failed' => undef, + 'sshlogin' => undef, + # The commandline wrapped with rsync and ssh + 'sshlogin_wrap' => undef, + 'exitstatus' => undef, + 'exitsignal' => undef, + # Timestamp for timeout if any + 'timeout' => undef, + 'virgin' => 1, + }, ref($class) || $class; +} + +sub replaced { + my $self = shift; + $self->{'commandline'} or ::die_bug("commandline empty"); + return $self->{'commandline'}->replaced(); +} + +sub seq { + my $self = shift; + return $self->{'commandline'}->seq(); +} + +sub slot { + my $self = shift; + return $self->{'commandline'}->slot(); +} + +{ + my($cattail); + + sub cattail { + # Returns: + # $cattail = perl program for: cattail "decompress program" writerpid [file_to_decompress or stdin] [file_to_unlink] + if(not $cattail) { + $cattail = q{ + # cat followed by tail. + # If $writerpid dead: finish after this round + use Fcntl; + + $|=1; + + my ($cmd, $writerpid, $read_file, $unlink_file) = @ARGV; + if($read_file) { + open(IN,"<",$read_file) || die("cattail: Cannot open $read_file"); + } else { + *IN = *STDIN; + } + + my $flags; + fcntl(IN, F_GETFL, $flags) || die $!; # Get the current flags on the filehandle + $flags |= O_NONBLOCK; # Add non-blocking to the flags + fcntl(IN, F_SETFL, $flags) || die $!; # Set the flags on the filehandle + open(OUT,"|-",$cmd) || die("cattail: Cannot run $cmd"); + + while(1) { + # clear EOF + seek(IN,0,1); + my $writer_running = kill 0, $writerpid; + $read = sysread(IN,$buf,32768); + if($read) { + # We can unlink the file now: The writer has written something + -e $unlink_file and unlink $unlink_file; + # Blocking print + while($buf) { + my $bytes_written = syswrite(OUT,$buf); + # syswrite may be interrupted by SIGHUP + substr($buf,0,$bytes_written) = ""; + } + # Something printed: Wait less next time + $sleep /= 2; + } else { + if(eof(IN) and not $writer_running) { + # Writer dead: There will never be more to read => exit + exit; + } + # TODO This could probably be done more efficiently using select(2) + # Nothing read: Wait longer before next read + # Up to 30 milliseconds + $sleep = ($sleep < 30) ? ($sleep * 1.001 + 0.01) : ($sleep); + usleep($sleep); + } + } + + sub usleep { + # Sleep this many milliseconds. + my $secs = shift; + select(undef, undef, undef, $secs/1000); + } + }; + $cattail =~ s/#.*//mg; + $cattail =~ s/\s+/ /g; + } + return $cattail; + } +} + +sub openoutputfiles { + # Open files for STDOUT and STDERR + # Set file handles in $self->fh + my $self = shift; + my ($outfhw, $errfhw, $outname, $errname); + if($opt::results) { + my $args_as_dirname = $self->{'commandline'}->args_as_dirname(); + # Output in: prefix/name1/val1/name2/val2/stdout + my $dir = $opt::results."/".$args_as_dirname; + if(eval{ File::Path::mkpath($dir); }) { + # OK + } else { + # mkpath failed: Argument probably too long. + # Set $Global::max_file_length, which will keep the individual + # dir names shorter than the max length + max_file_name_length($opt::results); + $args_as_dirname = $self->{'commandline'}->args_as_dirname(); + # prefix/name1/val1/name2/val2/ + $dir = $opt::results."/".$args_as_dirname; + File::Path::mkpath($dir); + } + # prefix/name1/val1/name2/val2/stdout + $outname = "$dir/stdout"; + if(not open($outfhw, "+>", $outname)) { + ::error("Cannot write to `$outname'.\n"); + ::wait_and_exit(255); + } + # prefix/name1/val1/name2/val2/stderr + $errname = "$dir/stderr"; + if(not open($errfhw, "+>", $errname)) { + ::error("Cannot write to `$errname'.\n"); + ::wait_and_exit(255); + } + $self->set_fh(1,"unlink",""); + $self->set_fh(2,"unlink",""); + } elsif(not $opt::ungroup) { + # To group we create temporary files for STDOUT and STDERR + # To avoid the cleanup unlink the files immediately (but keep them open) + if(@Global::tee_jobs) { + # files must be removed when the tee is done + } elsif($opt::files) { + ($outfhw, $outname) = ::tmpfile(SUFFIX => ".par"); + ($errfhw, $errname) = ::tmpfile(SUFFIX => ".par"); + # --files => only remove stderr + $self->set_fh(1,"unlink",""); + $self->set_fh(2,"unlink",$errname); + } else { + ($outfhw, $outname) = ::tmpfile(SUFFIX => ".par"); + ($errfhw, $errname) = ::tmpfile(SUFFIX => ".par"); + $self->set_fh(1,"unlink",$outname); + $self->set_fh(2,"unlink",$errname); + } + } else { + # --ungroup + open($outfhw,">&",$Global::fd{1}) || die; + open($errfhw,">&",$Global::fd{2}) || die; + # File name must be empty as it will otherwise be printed + $outname = ""; + $errname = ""; + $self->set_fh(1,"unlink",$outname); + $self->set_fh(2,"unlink",$errname); + } + # Set writing FD + $self->set_fh(1,'w',$outfhw); + $self->set_fh(2,'w',$errfhw); + $self->set_fh(1,'name',$outname); + $self->set_fh(2,'name',$errname); + if($opt::compress) { + # Send stdout to stdin for $opt::compress_program(1) + # Send stderr to stdin for $opt::compress_program(2) + # cattail get pid: $pid = $self->fh($fdno,'rpid'); + my $cattail = cattail(); + for my $fdno (1,2) { + my $wpid = open(my $fdw,"|-","$opt::compress_program >>". + $self->fh($fdno,'name')) || die $?; + $self->set_fh($fdno,'w',$fdw); + $self->set_fh($fdno,'wpid',$wpid); + my $rpid = open(my $fdr, "-|", "perl", "-e", $cattail, + $opt::decompress_program, $wpid, + $self->fh($fdno,'name'),$self->fh($fdno,'unlink')) || die $?; + $self->set_fh($fdno,'r',$fdr); + $self->set_fh($fdno,'rpid',$rpid); + } + } elsif(not $opt::ungroup) { + # Set reading FD if using --group (--ungroup does not need) + for my $fdno (1,2) { + # Re-open the file for reading + # so fdw can be closed seperately + # and fdr can be seeked seperately (for --line-buffer) + open(my $fdr,"<", $self->fh($fdno,'name')) || + ::die_bug("fdr: Cannot open ".$self->fh($fdno,'name')); + $self->set_fh($fdno,'r',$fdr); + # Unlink if required + $Global::debug or unlink $self->fh($fdno,"unlink"); + } + } + if($opt::linebuffer) { + # Set non-blocking when using --linebuffer + $Global::use{"Fcntl"} ||= eval "use Fcntl qw(:DEFAULT :flock); 1;"; + for my $fdno (1,2) { + my $fdr = $self->fh($fdno,'r'); + my $flags; + fcntl($fdr, &F_GETFL, $flags) || die $!; # Get the current flags on the filehandle + $flags |= &O_NONBLOCK; # Add non-blocking to the flags + fcntl($fdr, &F_SETFL, $flags) || die $!; # Set the flags on the filehandle + } + } +} + +sub max_file_name_length { + # Figure out the max length of a subdir + # TODO and the max total length + # Ext4 = 255,130816 + my $testdir = shift; + + my $upper = 8_000_000; + my $len = 8; + my $dir="x"x$len; + do { + rmdir($testdir."/".$dir); + $len *= 16; + $dir="x"x$len; + } while (mkdir $testdir."/".$dir); + # Then search for the actual max length between $len/16 and $len + my $min = $len/16; + my $max = $len; + while($max-$min > 5) { + # If we are within 5 chars of the exact value: + # it is not worth the extra time to find the exact value + my $test = int(($min+$max)/2); + $dir="x"x$test; + if(mkdir $testdir."/".$dir) { + rmdir($testdir."/".$dir); + $min = $test; + } else { + $max = $test; + } + } + $Global::max_file_length = $min; + return $min; +} + +sub set_fh { + # Set file handle + my ($self, $fd_no, $key, $fh) = @_; + $self->{'fd'}{$fd_no,$key} = $fh; +} + +sub fh { + # Get file handle + my ($self, $fd_no, $key) = @_; + return $self->{'fd'}{$fd_no,$key}; +} + +sub write { + my $self = shift; + my $remaining_ref = shift; + my $stdin_fh = $self->fh(0,"w"); + syswrite($stdin_fh,$$remaining_ref); +} + +sub set_stdin_buffer { + # Copy stdin buffer from $block_ref up to $endpos + # Prepend with $header_ref + # Remove $recstart and $recend if needed + # Input: + # $header_ref = ref to $header to prepend + # $block_ref = ref to $block to pass on + # $endpos = length of $block to pass on + # $recstart = --recstart regexp + # $recend = --recend regexp + # Returns: + # N/A + my $self = shift; + my ($header_ref,$block_ref,$endpos,$recstart,$recend) = @_; + $self->{'stdin_buffer'} = ($self->virgin() ? $$header_ref : "").substr($$block_ref,0,$endpos); + if($opt::remove_rec_sep) { + remove_rec_sep(\$self->{'stdin_buffer'},$recstart,$recend); + } + $self->{'stdin_buffer_length'} = length $self->{'stdin_buffer'}; + $self->{'stdin_buffer_pos'} = 0; +} + +sub stdin_buffer_length { + my $self = shift; + return $self->{'stdin_buffer_length'}; +} + +sub remove_rec_sep { + my ($block_ref,$recstart,$recend) = @_; + # Remove record separator + $$block_ref =~ s/$recend$recstart//gos; + $$block_ref =~ s/^$recstart//os; + $$block_ref =~ s/$recend$//os; +} + +sub non_block_write { + my $self = shift; + my $something_written = 0; + use POSIX qw(:errno_h); +# use Fcntl; +# my $flags = ''; + for my $buf (substr($self->{'stdin_buffer'},$self->{'stdin_buffer_pos'})) { + my $in = $self->fh(0,"w"); +# fcntl($in, F_GETFL, $flags) +# or die "Couldn't get flags for HANDLE : $!\n"; +# $flags |= O_NONBLOCK; +# fcntl($in, F_SETFL, $flags) +# or die "Couldn't set flags for HANDLE: $!\n"; + my $rv = syswrite($in, $buf); + if (!defined($rv) && $! == EAGAIN) { + # would block + $something_written = 0; + } elsif ($self->{'stdin_buffer_pos'}+$rv != $self->{'stdin_buffer_length'}) { + # incomplete write + # Remove the written part + $self->{'stdin_buffer_pos'} += $rv; + $something_written = $rv; + } else { + # successfully wrote everything + my $a=""; + $self->set_stdin_buffer(\$a,\$a,"",""); + $something_written = $rv; + } + } + + ::debug("pipe", "Non-block: ", $something_written); + return $something_written; +} + + +sub virgin { + my $self = shift; + return $self->{'virgin'}; +} + +sub set_virgin { + my $self = shift; + $self->{'virgin'} = shift; +} + +sub pid { + my $self = shift; + return $self->{'pid'}; +} + +sub set_pid { + my $self = shift; + $self->{'pid'} = shift; +} + +sub starttime { + # Returns: + # UNIX-timestamp this job started + my $self = shift; + return sprintf("%.3f",$self->{'starttime'}); +} + +sub set_starttime { + my $self = shift; + my $starttime = shift || ::now(); + $self->{'starttime'} = $starttime; +} + +sub runtime { + # Returns: + # Run time in seconds + my $self = shift; + return sprintf("%.3f",int(($self->endtime() - $self->starttime())*1000)/1000); +} + +sub endtime { + # Returns: + # UNIX-timestamp this job ended + # 0 if not ended yet + my $self = shift; + return ($self->{'endtime'} || 0); +} + +sub set_endtime { + my $self = shift; + my $endtime = shift; + $self->{'endtime'} = $endtime; +} + +sub timedout { + # Is the job timedout? + # Input: + # $delta_time = time that the job may run + # Returns: + # True or false + my $self = shift; + my $delta_time = shift; + return time > $self->{'starttime'} + $delta_time; +} + +sub kill { + # Kill the job. + # Send the signals to (grand)*children and pid. + # If no signals: TERM TERM KILL + # Wait 200 ms after each TERM. + # Input: + # @signals = signals to send + my $self = shift; + my @signals = @_; + my @family_pids = $self->family_pids(); + # Record this jobs as failed + $self->set_exitstatus(-1); + # Send two TERMs to give time to clean up + ::debug("run", "Kill seq ", $self->seq(), "\n"); + my @send_signals = @signals || ("TERM", "TERM", "KILL"); + for my $signal (@send_signals) { + my $alive = 0; + for my $pid (@family_pids) { + if(kill 0, $pid) { + # The job still running + kill $signal, $pid; + $alive = 1; + } + } + # If a signal was given as input, do not do the sleep below + @signals and next; + + if($signal eq "TERM" and $alive) { + # Wait up to 200 ms between TERMs - but only if any pids are alive + my $sleep = 1; + for (my $sleepsum = 0; kill 0, $family_pids[0] and $sleepsum < 200; + $sleepsum += $sleep) { + $sleep = ::reap_usleep($sleep); + } + } + } +} + +sub family_pids { + # Find the pids with this->pid as (grand)*parent + # Returns: + # @pids = pids of (grand)*children + my $self = shift; + my $pid = $self->pid(); + my @pids; + + my ($children_of_ref, $parent_of_ref, $name_of_ref) = ::pid_table(); + + my @more = ($pid); + # While more (grand)*children + while(@more) { + my @m; + push @pids, @more; + for my $parent (@more) { + if($children_of_ref->{$parent}) { + # add the children of this parent + push @m, @{$children_of_ref->{$parent}}; + } + } + @more = @m; + } + return (@pids); +} + +sub failed { + # return number of times failed for this $sshlogin + # Input: + # $sshlogin + # Returns: + # Number of times failed for $sshlogin + my $self = shift; + my $sshlogin = shift; + return $self->{'failed'}{$sshlogin}; +} + +sub failed_here { + # return number of times failed for the current $sshlogin + # Returns: + # Number of times failed for this sshlogin + my $self = shift; + return $self->{'failed'}{$self->sshlogin()}; +} + +sub add_failed { + # increase the number of times failed for this $sshlogin + my $self = shift; + my $sshlogin = shift; + $self->{'failed'}{$sshlogin}++; +} + +sub add_failed_here { + # increase the number of times failed for the current $sshlogin + my $self = shift; + $self->{'failed'}{$self->sshlogin()}++; +} + +sub reset_failed { + # increase the number of times failed for this $sshlogin + my $self = shift; + my $sshlogin = shift; + delete $self->{'failed'}{$sshlogin}; +} + +sub reset_failed_here { + # increase the number of times failed for this $sshlogin + my $self = shift; + delete $self->{'failed'}{$self->sshlogin()}; +} + +sub min_failed { + # Returns: + # the number of sshlogins this command has failed on + # the minimal number of times this command has failed + my $self = shift; + my $min_failures = + ::min(map { $self->{'failed'}{$_} } keys %{$self->{'failed'}}); + my $number_of_sshlogins_failed_on = scalar keys %{$self->{'failed'}}; + return ($number_of_sshlogins_failed_on,$min_failures); +} + +sub total_failed { + # Returns: + # $total_failures = the number of times this command has failed + my $self = shift; + my $total_failures = 0; + for (values %{$self->{'failed'}}) { + $total_failures += $_; + } + return $total_failures; +} + +sub wrapped { + # Wrap command with: + # * --shellquote + # * --nice + # * --cat + # * --fifo + # * --sshlogin + # * --pipepart (@Global::cat_partials) + # * --pipe + # * --tmux + # The ordering of the wrapping is important: + # * --nice/--cat/--fifo should be done on the remote machine + # * --pipepart/--pipe should be done on the local machine inside --tmux + # Uses: + # $Global::envvar + # $opt::shellquote + # $opt::nice + # $Global::shell + # $opt::cat + # $opt::fifo + # @Global::cat_partials + # $opt::pipe + # $opt::tmux + # Returns: + # $self->{'wrapped'} = the command wrapped with the above + my $self = shift; + if(not defined $self->{'wrapped'}) { + my $command = $Global::envvar.$self->replaced(); + if($opt::shellquote) { + # Prepend echo + # and quote twice + $command = "echo " . + ::shell_quote_scalar(::shell_quote_scalar($command)); + } + if($opt::nice) { + # Prepend \nice -n19 $SHELL -c + # and quote. + # The '\' before nice is needed to avoid tcsh's built-in + $command = '\nice'. " -n". $opt::nice. " ". + $Global::shell. " -c ". + ::shell_quote_scalar($command); + } + if($opt::cat) { + # Prepend 'cat > {};' + # Append '_EXIT=$?;(rm {};exit $_EXIT)' + $command = + $self->{'commandline'}->replace_placeholders(["cat > \257<\257>; "], 0, 0). + $command. + $self->{'commandline'}->replace_placeholders( + ["; _EXIT=\$?; rm \257<\257>; exit \$_EXIT"], 0, 0); + } elsif($opt::fifo) { + # Prepend 'mkfifo {}; (' + # Append ') & _PID=$!; cat > {}; wait $_PID; _EXIT=$?;(rm {};exit $_EXIT)' + $command = + $self->{'commandline'}->replace_placeholders(["mkfifo \257<\257>; ("], 0, 0). + $command. + $self->{'commandline'}->replace_placeholders([") & _PID=\$!; cat > \257<\257>; ", + "wait \$_PID; _EXIT=\$?; ", + "rm \257<\257>; exit \$_EXIT"], + 0,0); + } + # Wrap with ssh + tranferring of files + $command = $self->sshlogin_wrap($command); + if(@Global::cat_partials) { + # Prepend: + # < /tmp/foo perl -e 'while(@ARGV) { sysseek(STDIN,shift,0) || die; $left = shift; while($read = sysread(STDIN,$buf, ($left > 32768 ? 32768 : $left))){ $left -= $read; syswrite(STDOUT,$buf); } }' 0 0 0 11 | + $command = (shift @Global::cat_partials). "|". "(". $command. ")"; + } elsif($opt::pipe) { + # Prepend EOF-detector to avoid starting $command if EOF. + # The $tmpfile might exist if run on a remote system - we accept that risk + my ($dummy_fh, $tmpfile) = ::tmpfile(SUFFIX => ".chr"); + # Unlink to avoid leaving files if --dry-run or --sshlogin + unlink $tmpfile; + $command = + # Exit value: + # empty input = true + # some input = exit val from command + qq{ sh -c 'dd bs=1 count=1 of=$tmpfile 2>/dev/null'; }. + qq{ test \! -s "$tmpfile" && rm -f "$tmpfile" && exec true; }. + qq{ (cat $tmpfile; rm $tmpfile; cat - ) | }. + "($command);"; + } + if($opt::tmux) { + # Wrap command with 'tmux' + $command = $self->tmux_wrap($command); + } + $self->{'wrapped'} = $command; + } + return $self->{'wrapped'}; +} + +sub set_sshlogin { + my $self = shift; + my $sshlogin = shift; + $self->{'sshlogin'} = $sshlogin; + delete $self->{'sshlogin_wrap'}; # If sshlogin is changed the wrap is wrong + delete $self->{'wrapped'}; +} + +sub sshlogin { + my $self = shift; + return $self->{'sshlogin'}; +} + +sub sshlogin_wrap { + # Wrap the command with the commands needed to run remotely + # Returns: + # $self->{'sshlogin_wrap'} = command wrapped with ssh+transfer commands + my $self = shift; + my $command = shift; + if(not defined $self->{'sshlogin_wrap'}) { + my $sshlogin = $self->sshlogin(); + my $sshcmd = $sshlogin->sshcommand(); + my $serverlogin = $sshlogin->serverlogin(); + my ($pre,$post,$cleanup)=("","",""); + + if($serverlogin eq ":") { + # No transfer neeeded + $self->{'sshlogin_wrap'} = $command; + } else { + # --transfer + $pre .= $self->sshtransfer(); + # --return + $post .= $self->sshreturn(); + # --cleanup + $post .= $self->sshcleanup(); + if($post) { + # We need to save the exit status of the job + $post = '_EXIT_status=$?; ' . $post . ' exit $_EXIT_status;'; + } + # If the remote login shell is (t)csh then use 'setenv' + # otherwise use 'export' + # We cannot use parse_env_var(), as PARALLEL_SEQ changes + # for each command + my $parallel_env = + ($Global::envwarn + . q{ 'eval `echo $SHELL | grep "/t\\{0,1\\}csh" > /dev/null } + . q{ && echo setenv PARALLEL_SEQ '$PARALLEL_SEQ'\; } + . q{ setenv PARALLEL_PID '$PARALLEL_PID' } + . q{ || echo PARALLEL_SEQ='$PARALLEL_SEQ'\;export PARALLEL_SEQ\; } + . q{ PARALLEL_PID='$PARALLEL_PID'\;export PARALLEL_PID` ;' }); + my $remote_pre = ""; + my $ssh_options = ""; + if(($opt::pipe or $opt::pipepart) and $opt::ctrlc + or + not ($opt::pipe or $opt::pipepart) and not $opt::noctrlc) { + # TODO Determine if this is needed + # Propagating CTRL-C to kill remote jobs requires + # remote jobs to be run with a terminal. + $ssh_options = "-tt -oLogLevel=quiet"; +# $ssh_options = ""; + # tty - check if we have a tty. + # stty: + # -onlcr - make output 8-bit clean + # isig - pass CTRL-C as signal + # -echo - do not echo input + $remote_pre .= ::shell_quote_scalar('tty >/dev/null && stty isig -onlcr -echo;'); + } + if($opt::workdir) { + my $wd = ::shell_quote_file($self->workdir()); + $remote_pre .= ::shell_quote_scalar("mkdir -p ") . $wd . + ::shell_quote_scalar("; cd ") . $wd . + # exit 255 (instead of exec false) would be the correct thing, + # but that fails on tcsh + ::shell_quote_scalar(qq{ || exec false;}); + } + # This script is to solve the problem of + # * not mixing STDERR and STDOUT + # * terminating with ctrl-c + # It works on Linux but not Solaris + # Finishes on Solaris, but wrong exit code: + # $SIG{CHLD} = sub {exit ($?&127 ? 128+($?&127) : 1+$?>>8)}; + # Hangs on Solaris, but correct exit code on Linux: + # $SIG{CHLD} = sub { $done = 1 }; + # $p->poll; + my $signal_script = "perl -e '". + q{ + use IO::Poll; + $SIG{CHLD} = sub { $done = 1 }; + $p = IO::Poll->new; + $p->mask(STDOUT, POLLHUP); + $pid=fork; unless($pid) {setpgrp; exec $ENV{SHELL}, "-c", @ARGV; die "exec: $!\n"} + $p->poll; + kill SIGHUP, -${pid} unless $done; + wait; exit ($?&127 ? 128+($?&127) : 1+$?>>8) + } . "' "; + $signal_script =~ s/\s+/ /g; + + $self->{'sshlogin_wrap'} = + ($pre + . "$sshcmd $ssh_options $serverlogin $parallel_env " + . $remote_pre +# . ::shell_quote_scalar($signal_script . ::shell_quote_scalar($command)) + . ::shell_quote_scalar($command) + . ";" + . $post); + } + } + return $self->{'sshlogin_wrap'}; +} + +sub transfer { + # Files to transfer + # Returns: + # @transfer - File names of files to transfer + my $self = shift; + my @transfer = (); + $self->{'transfersize'} = 0; + if($opt::transfer) { + for my $record (@{$self->{'commandline'}{'arg_list'}}) { + # Merge arguments from records into args + for my $arg (@$record) { + CORE::push @transfer, $arg->orig(); + # filesize + if(-e $arg->orig()) { + $self->{'transfersize'} += (stat($arg->orig()))[7]; + } + } + } + } + return @transfer; +} + +sub transfersize { + my $self = shift; + return $self->{'transfersize'}; +} + +sub sshtransfer { + # Returns for each transfer file: + # rsync $file remote:$workdir + my $self = shift; + my @pre; + my $sshlogin = $self->sshlogin(); + my $workdir = $self->workdir(); + for my $file ($self->transfer()) { + push @pre, $sshlogin->rsync_transfer_cmd($file,$workdir).";"; + } + return join("",@pre); +} + +sub return { + # Files to return + # Non-quoted and with {...} substituted + # Returns: + # @non_quoted_filenames + my $self = shift; + return $self->{'commandline'}-> + replace_placeholders($self->{'commandline'}{'return_files'},0,0); +} + +sub returnsize { + # This is called after the job has finished + # Returns: + # $number_of_bytes transferred in return + my $self = shift; + for my $file ($self->return()) { + if(-e $file) { + $self->{'returnsize'} += (stat($file))[7]; + } + } + return $self->{'returnsize'}; +} + +sub sshreturn { + # Returns for each return-file: + # rsync remote:$workdir/$file . + my $self = shift; + my $sshlogin = $self->sshlogin(); + my $sshcmd = $sshlogin->sshcommand(); + my $serverlogin = $sshlogin->serverlogin(); + my $rsync_opt = "-rlDzR -e".::shell_quote_scalar($sshcmd); + my $pre = ""; + for my $file ($self->return()) { + $file =~ s:^\./::g; # Remove ./ if any + my $relpath = ($file !~ m:^/:); # Is the path relative? + my $cd = ""; + my $wd = ""; + if($relpath) { + # rsync -avR /foo/./bar/baz.c remote:/tmp/ + # == (on old systems) + # rsync -avR --rsync-path="cd /foo; rsync" remote:bar/baz.c /tmp/ + $wd = ::shell_quote_file($self->workdir()."/"); + } + # Only load File::Basename if actually needed + $Global::use{"File::Basename"} ||= eval "use File::Basename; 1;"; + # dir/./file means relative to dir, so remove dir on remote + $file =~ m:(.*)/\./:; + my $basedir = $1 ? ::shell_quote_file($1."/") : ""; + my $nobasedir = $file; + $nobasedir =~ s:.*/\./::; + $cd = ::shell_quote_file(::dirname($nobasedir)); + my $rsync_cd = '--rsync-path='.::shell_quote_scalar("cd $wd$cd; rsync"); + my $basename = ::shell_quote_scalar(::shell_quote_file(basename($file))); + # --return + # mkdir -p /home/tange/dir/subdir/; + # rsync (--protocol 30) -rlDzR --rsync-path="cd /home/tange/dir/subdir/; rsync" + # server:file.gz /home/tange/dir/subdir/ + $pre .= "mkdir -p $basedir$cd; ".$sshlogin->rsync()." $rsync_cd $rsync_opt $serverlogin:". + $basename . " ".$basedir.$cd.";"; + } + return $pre; +} + +sub sshcleanup { + # Return the sshcommand needed to remove the file + # Returns: + # ssh command needed to remove files from sshlogin + my $self = shift; + my $sshlogin = $self->sshlogin(); + my $sshcmd = $sshlogin->sshcommand(); + my $serverlogin = $sshlogin->serverlogin(); + my $workdir = $self->workdir(); + my $cleancmd = ""; + + for my $file ($self->cleanup()) { + my @subworkdirs = parentdirs_of($file); + $cleancmd .= $sshlogin->cleanup_cmd($file,$workdir).";"; + } + if(defined $opt::workdir and $opt::workdir eq "...") { + $cleancmd .= "$sshcmd $serverlogin rm -rf " . ::shell_quote_scalar($workdir).';'; + } + return $cleancmd; +} + +sub cleanup { + # Returns: + # Files to remove at cleanup + my $self = shift; + if($opt::cleanup) { + my @transfer = $self->transfer(); + my @return = $self->return(); + return (@transfer,@return); + } else { + return (); + } +} + +sub workdir { + # Returns: + # the workdir on a remote machine + my $self = shift; + if(not defined $self->{'workdir'}) { + my $workdir; + if(defined $opt::workdir) { + if($opt::workdir eq ".") { + # . means current dir + my $home = $ENV{'HOME'}; + eval 'use Cwd'; + my $cwd = cwd(); + $workdir = $cwd; + if($home) { + # If homedir exists: remove the homedir from + # workdir if cwd starts with homedir + # E.g. /home/foo/my/dir => my/dir + # E.g. /tmp/my/dir => /tmp/my/dir + my ($home_dev, $home_ino) = (stat($home))[0,1]; + my $parent = ""; + my @dir_parts = split(m:/:,$cwd); + my $part; + while(defined ($part = shift @dir_parts)) { + $part eq "" and next; + $parent .= "/".$part; + my ($parent_dev, $parent_ino) = (stat($parent))[0,1]; + if($parent_dev == $home_dev and $parent_ino == $home_ino) { + # dev and ino is the same: We found the homedir. + $workdir = join("/",@dir_parts); + last; + } + } + } + if($workdir eq "") { + $workdir = "."; + } + } elsif($opt::workdir eq "...") { + $workdir = ".parallel/tmp/" . ::hostname() . "-" . $$ + . "-" . $self->seq(); + } else { + $workdir = $opt::workdir; + # Rsync treats /./ special. We dont want that + $workdir =~ s:/\./:/:g; # Remove /./ + $workdir =~ s:/+$::; # Remove ending / if any + $workdir =~ s:^\./::g; # Remove starting ./ if any + } + } else { + $workdir = "."; + } + $self->{'workdir'} = ::shell_quote_scalar($workdir); + } + return $self->{'workdir'}; +} + +sub parentdirs_of { + # Return: + # all parentdirs except . of this dir or file - sorted desc by length + my $d = shift; + my @parents = (); + while($d =~ s:/[^/]+$::) { + if($d ne ".") { + push @parents, $d; + } + } + return @parents; +} + +sub start { + # Setup STDOUT and STDERR for a job and start it. + # Returns: + # job-object or undef if job not to run + my $job = shift; + # Get the shell command to be executed (possibly with ssh infront). + my $command = $job->wrapped(); + + if($Global::interactive or $Global::stderr_verbose) { + if($Global::interactive) { + print $Global::original_stderr "$command ?..."; + open(my $tty_fh, "<", "/dev/tty") || ::die_bug("interactive-tty"); + my $answer = <$tty_fh>; + close $tty_fh; + my $run_yes = ($answer =~ /^\s*y/i); + if (not $run_yes) { + $command = "true"; # Run the command 'true' + } + } else { + print $Global::original_stderr "$command\n"; + } + } + + my $pid; + $job->openoutputfiles(); + my($stdout_fh,$stderr_fh) = ($job->fh(1,"w"),$job->fh(2,"w")); + local (*IN,*OUT,*ERR); + open OUT, '>&', $stdout_fh or ::die_bug("Can't redirect STDOUT: $!"); + open ERR, '>&', $stderr_fh or ::die_bug("Can't dup STDOUT: $!"); + + if(($opt::dryrun or $Global::verbose) and $opt::ungroup) { + if($Global::verbose <= 1) { + print $stdout_fh $job->replaced(),"\n"; + } else { + # Verbose level > 1: Print the rsync and stuff + print $stdout_fh $command,"\n"; + } + } + if($opt::dryrun) { + $command = "true"; + } + $ENV{'PARALLEL_SEQ'} = $job->seq(); + $ENV{'PARALLEL_PID'} = $$; + ::debug("run", $Global::total_running, " processes . Starting (", + $job->seq(), "): $command\n"); + if($opt::pipe) { + my ($stdin_fh); + # The eval is needed to catch exception from open3 + eval { + $pid = ::open3($stdin_fh, ">&OUT", ">&ERR", $Global::shell, "-c", $command) || + ::die_bug("open3-pipe"); + 1; + }; + $job->set_fh(0,"w",$stdin_fh); + } elsif(@opt::a and not $Global::stdin_in_opt_a and $job->seq() == 1 + and $job->sshlogin()->string() eq ":") { + # Give STDIN to the first job if using -a (but only if running + # locally - otherwise CTRL-C does not work for other jobs Bug#36585) + *IN = *STDIN; + # The eval is needed to catch exception from open3 + eval { + $pid = ::open3("<&IN", ">&OUT", ">&ERR", $Global::shell, "-c", $command) || + ::die_bug("open3-a"); + 1; + }; + # Re-open to avoid complaining + open(STDIN, "<&", $Global::original_stdin) + or ::die_bug("dup-\$Global::original_stdin: $!"); + } elsif ($opt::tty and not $Global::tty_taken and -c "/dev/tty" and + open(my $devtty_fh, "<", "/dev/tty")) { + # Give /dev/tty to the command if no one else is using it + *IN = $devtty_fh; + # The eval is needed to catch exception from open3 + eval { + $pid = ::open3("<&IN", ">&OUT", ">&ERR", $Global::shell, "-c", $command) || + ::die_bug("open3-/dev/tty"); + $Global::tty_taken = $pid; + close $devtty_fh; + 1; + }; + } else { + # The eval is needed to catch exception from open3 + eval { + $pid = ::open3(::gensym, ">&OUT", ">&ERR", $Global::shell, "-c", $command) || + ::die_bug("open3-gensym"); + 1; + }; + } + if($pid) { + # A job was started + $Global::total_running++; + $Global::total_started++; + $job->set_pid($pid); + $job->set_starttime(); + $Global::running{$job->pid()} = $job; + if($opt::timeout) { + $Global::timeoutq->insert($job); + } + $Global::newest_job = $job; + $Global::newest_starttime = ::now(); + return $job; + } else { + # No more processes + ::debug("run", "Cannot spawn more jobs.\n"); + return undef; + } +} + +sub tmux_wrap { + # Wrap command with tmux for session pPID + # Input: + # $actual_command = the actual command being run (incl ssh wrap) + my $self = shift; + my $actual_command = shift; + # Temporary file name. Used for fifo to communicate exit val + my ($fh, $tmpfile) = ::tmpfile(SUFFIX => ".tmx"); + $Global::unlink{$tmpfile}=1; + close $fh; + unlink $tmpfile; + my $visual_command = $self->replaced(); + my $title = $visual_command; + # ; causes problems + # ascii 194-245 annoys tmux + $title =~ tr/[\011-\016;\302-\365]//d; + + my $tmux; + if($Global::total_running == 0) { + $tmux = "tmux new-session -s p$$ -d -n ". + ::shell_quote_scalar($title); + print $Global::original_stderr "See output with: tmux attach -t p$$\n"; + } else { + $tmux = "tmux new-window -t p$$ -n ".::shell_quote_scalar($title); + } + return "mkfifo $tmpfile; $tmux ". + # Run in tmux + ::shell_quote_scalar( + "(".$actual_command.');(echo $?$status;echo 255) >'.$tmpfile."&". + "echo ".::shell_quote_scalar($visual_command).";". + "echo \007Job finished at: `date`;sleep 10"). + # Run outside tmux + # Read the first line from the fifo and use that as status code + "; exit `perl -ne 'unlink \$ARGV; 1..1 and print' $tmpfile` "; +} + +sub is_already_in_results { + # Do we already have results for this job? + # Returns: + # $job_already_run = bool whether there is output for this or not + my $job = $_[0]; + my $args_as_dirname = $job->{'commandline'}->args_as_dirname(); + # prefix/name1/val1/name2/val2/ + my $dir = $opt::results."/".$args_as_dirname; + ::debug("run", "Test $dir/stdout", -e "$dir/stdout", "\n"); + return -e "$dir/stdout"; +} + +sub is_already_in_joblog { + my $job = shift; + return vec($Global::job_already_run,$job->seq(),1); +} + +sub set_job_in_joblog { + my $job = shift; + vec($Global::job_already_run,$job->seq(),1) = 1; +} + +sub should_be_retried { + # Should this job be retried? + # Returns + # 0 - do not retry + # 1 - job queued for retry + my $self = shift; + if (not $opt::retries) { + return 0; + } + if(not $self->exitstatus()) { + # Completed with success. If there is a recorded failure: forget it + $self->reset_failed_here(); + return 0 + } else { + # The job failed. Should it be retried? + $self->add_failed_here(); + if($self->total_failed() == $opt::retries) { + # This has been retried enough + return 0; + } else { + # This command should be retried + $self->set_endtime(undef); + $Global::JobQueue->unget($self); + ::debug("run", "Retry ", $self->seq(), "\n"); + return 1; + } + } +} + +sub print { + # Print the output of the jobs + # Returns: N/A + + my $self = shift; + ::debug("print", ">>joboutput ", $self->replaced(), "\n"); + if($opt::dryrun) { + # Nothing was printed to this job: + # cleanup tmp files if --files was set + unlink $self->fh(1,"name"); + } + if($opt::pipe and $self->virgin()) { + # Skip --joblog, --dryrun, --verbose + } else { + if($Global::joblog and defined $self->{'exitstatus'}) { + # Add to joblog when finished + $self->print_joblog(); + } + + # Printing is only relevant for grouped/--line-buffer output. + $opt::ungroup and return; + # Check for disk full + exit_if_disk_full(); + + if(($opt::dryrun or $Global::verbose) + and + not $self->{'verbose_printed'}) { + $self->{'verbose_printed'}++; + if($Global::verbose <= 1) { + print STDOUT $self->replaced(),"\n"; + } else { + # Verbose level > 1: Print the rsync and stuff + print STDOUT $self->wrapped(),"\n"; + } + # If STDOUT and STDERR are merged, + # we want the command to be printed first + # so flush to avoid STDOUT being buffered + flush STDOUT; + } + } + for my $fdno (sort { $a <=> $b } keys %Global::fd) { + # Sort by file descriptor numerically: 1,2,3,..,9,10,11 + $fdno == 0 and next; + my $out_fd = $Global::fd{$fdno}; + my $in_fh = $self->fh($fdno,"r"); + if(not $in_fh) { + if(not $Job::file_descriptor_warning_printed{$fdno}++) { + # ::warning("File descriptor $fdno not defined\n"); + } + next; + } + ::debug("print", "File descriptor $fdno (", $self->fh($fdno,"name"), "):"); + if($opt::files) { + # If --compress: $in_fh must be closed first. + close $self->fh($fdno,"w"); + close $in_fh; + if($opt::pipe and $self->virgin()) { + # Nothing was printed to this job: + # cleanup unused tmp files if --files was set + for my $fdno (1,2) { + unlink $self->fh($fdno,"name"); + unlink $self->fh($fdno,"unlink"); + } + } elsif($fdno == 1 and $self->fh($fdno,"name")) { + print $out_fd $self->fh($fdno,"name"),"\n"; + } + } elsif($opt::linebuffer) { + # Line buffered print out + $self->linebuffer_print($fdno,$in_fh,$out_fd); + } else { + my $buf; + close $self->fh($fdno,"w"); + seek $in_fh, 0, 0; + # $in_fh is now ready for reading at position 0 + if($opt::tag or defined $opt::tagstring) { + my $tag = $self->tag(); + if($fdno == 2) { + # OpenSSH_3.6.1p2 gives 'tcgetattr: Invalid argument' with -tt + # This is a crappy way of ignoring it. + while(<$in_fh>) { + if(/^(client_process_control: )?tcgetattr: Invalid argument\n/) { + # Skip + } else { + print $out_fd $tag,$_; + } + # At most run the loop once + last; + } + } + while(<$in_fh>) { + print $out_fd $tag,$_; + } + } else { + my $buf; + if($fdno == 2) { + # OpenSSH_3.6.1p2 gives 'tcgetattr: Invalid argument' with -tt + # This is a crappy way of ignoring it. + sysread($in_fh,$buf,1_000); + $buf =~ s/^(client_process_control: )?tcgetattr: Invalid argument\n//; + print $out_fd $buf; + } + while(sysread($in_fh,$buf,32768)) { + print $out_fd $buf; + } + } + close $in_fh; + } + flush $out_fd; + } + ::debug("print", "<{'partial_line',$fdno}; + + if(defined $self->{'exitstatus'}) { + # If the job is dead: close printing fh. Needed for --compress + close $self->fh($fdno,"w"); + if($opt::compress) { + # Blocked reading in final round + $Global::use{"Fcntl"} ||= eval "use Fcntl qw(:DEFAULT :flock); 1;"; + for my $fdno (1,2) { + my $fdr = $self->fh($fdno,'r'); + my $flags; + fcntl($fdr, &F_GETFL, $flags) || die $!; # Get the current flags on the filehandle + $flags &= ~&O_NONBLOCK; # Remove non-blocking to the flags + fcntl($fdr, &F_SETFL, $flags) || die $!; # Set the flags on the filehandle + } + } + } + # This seek will clear EOF + seek $in_fh, tell($in_fh), 0; + # The read is non-blocking: The $in_fh is set to non-blocking. + # 32768 --tag = 5.1s + # 327680 --tag = 4.4s + # 1024000 --tag = 4.4s + # 3276800 --tag = 4.3s + # 32768000 --tag = 4.7s + # 10240000 --tag = 4.3s + while(read($in_fh,substr($$partial,length $$partial),3276800)) { + # Append to $$partial + # Find the last \n + my $i = rindex($$partial,"\n"); + if($i != -1) { + # One or more complete lines were found + if($fdno == 2 and not $self->{'printed_first_line',$fdno}++) { + # OpenSSH_3.6.1p2 gives 'tcgetattr: Invalid argument' with -tt + # This is a crappy way of ignoring it. + $$partial =~ s/^(client_process_control: )?tcgetattr: Invalid argument\n//; + # Length of partial line has changed: Find the last \n again + $i = rindex($$partial,"\n"); + } + if($opt::tag or defined $opt::tagstring) { + # Replace ^ with $tag within the full line + my $tag = $self->tag(); + substr($$partial,0,$i+1) =~ s/^/$tag/gm; + # Length of partial line has changed: Find the last \n again + $i = rindex($$partial,"\n"); + } + # Print up to and including the last \n + print $out_fd substr($$partial,0,$i+1); + # Remove the printed part + substr($$partial,0,$i+1)=""; + } + } + if(defined $self->{'exitstatus'}) { + # If the job is dead: print the remaining partial line + # read remaining + if($$partial and ($opt::tag or defined $opt::tagstring)) { + my $tag = $self->tag(); + $$partial =~ s/^/$tag/gm; + } + print $out_fd $$partial; + # Release the memory + $$partial = undef; + if($self->fh($fdno,"rpid") and CORE::kill 0, $self->fh($fdno,"rpid")) { + # decompress still running + } else { + # decompress done: close fh + close $in_fh; + } + } +} + +sub print_joblog { + my $self = shift; + my $cmd; + if($Global::verbose <= 1) { + $cmd = $self->replaced(); + } else { + # Verbose level > 1: Print the rsync and stuff + $cmd = "@command"; + } + print $Global::joblog + join("\t", $self->seq(), $self->sshlogin()->string(), + $self->starttime(), sprintf("%10.3f",$self->runtime()), + $self->transfersize(), $self->returnsize(), + $self->exitstatus(), $self->exitsignal(), $cmd + ). "\n"; + flush $Global::joblog; + $self->set_job_in_joblog(); +} + +sub tag { + my $self = shift; + if(not defined $self->{'tag'}) { + $self->{'tag'} = $self->{'commandline'}-> + replace_placeholders([$opt::tagstring],0,0)."\t"; + } + return $self->{'tag'}; +} + +sub hostgroups { + my $self = shift; + if(not defined $self->{'hostgroups'}) { + $self->{'hostgroups'} = $self->{'commandline'}->{'arg_list'}[0][0]->{'hostgroups'}; + } + return @{$self->{'hostgroups'}}; +} + +sub exitstatus { + my $self = shift; + return $self->{'exitstatus'}; +} + +sub set_exitstatus { + my $self = shift; + my $exitstatus = shift; + if($exitstatus) { + # Overwrite status if non-zero + $self->{'exitstatus'} = $exitstatus; + } else { + # Set status but do not overwrite + # Status may have been set by --timeout + $self->{'exitstatus'} ||= $exitstatus; + } +} + +sub exitsignal { + my $self = shift; + return $self->{'exitsignal'}; +} + +sub set_exitsignal { + my $self = shift; + my $exitsignal = shift; + $self->{'exitsignal'} = $exitsignal; +} + +{ + my ($disk_full_fh, $b8193, $name); + sub exit_if_disk_full { + # Checks if $TMPDIR is full by writing 8kb to a tmpfile + # If the disk is full: Exit immediately. + # Returns: + # N/A + if(not $disk_full_fh) { + ($disk_full_fh, $name) = ::tmpfile(SUFFIX => ".df"); + unlink $name; + $b8193 = "x"x8193; + } + # Linux does not discover if a disk is full if writing <= 8192 + # Tested on: + # bfs btrfs cramfs ext2 ext3 ext4 ext4dev jffs2 jfs minix msdos + # ntfs reiserfs tmpfs ubifs vfat xfs + # TODO this should be tested on different OS similar to this: + # + # doit() { + # sudo mount /dev/ram0 /mnt/loop; sudo chmod 1777 /mnt/loop + # seq 100000 | parallel --tmpdir /mnt/loop/ true & + # seq 6900000 > /mnt/loop/i && echo seq OK + # seq 6980868 > /mnt/loop/i + # seq 10000 > /mnt/loop/ii + # sleep 3 + # sudo umount /mnt/loop/ || sudo umount -l /mnt/loop/ + # echo >&2 + # } + print $disk_full_fh $b8193; + if(not $disk_full_fh + or + tell $disk_full_fh == 0) { + ::error("Output is incomplete. Cannot append to buffer file in $ENV{'TMPDIR'}. Is the disk full?\n"); + ::error("Change \$TMPDIR with --tmpdir or use --compress.\n"); + ::wait_and_exit(255); + } + truncate $disk_full_fh, 0; + seek($disk_full_fh, 0, 0) || die; + } +} + + +package CommandLine; + +sub new { + my $class = shift; + my $seq = shift; + my $commandref = shift; + $commandref || die; + my $arg_queue = shift; + my $context_replace = shift; + my $max_number_of_args = shift; # for -N and normal (-n1) + my $return_files = shift; + my $replacecount_ref = shift; + my $len_ref = shift; + my %replacecount = %$replacecount_ref; + my %len = %$len_ref; + for (keys %$replacecount_ref) { + # Total length of this replacement string {} replaced with all args + $len{$_} = 0; + } + return bless { + 'command' => $commandref, + 'seq' => $seq, + 'len' => \%len, + 'arg_list' => [], + 'arg_queue' => $arg_queue, + 'max_number_of_args' => $max_number_of_args, + 'replacecount' => \%replacecount, + 'context_replace' => $context_replace, + 'return_files' => $return_files, + 'replaced' => undef, + }, ref($class) || $class; +} + +sub seq { + my $self = shift; + return $self->{'seq'}; +} + +{ + my $max_slot_number; + + sub slot { + # Find the number of a free job slot and return it + # Uses: + # @Global::slots + # Returns: + # $jobslot = number of jobslot + my $self = shift; + if(not $self->{'slot'}) { + if(not @Global::slots) { + # $Global::max_slot_number will typically be $Global::max_jobs_running + push @Global::slots, ++$max_slot_number; + } + $self->{'slot'} = shift @Global::slots; + } + return $self->{'slot'}; + } +} + +sub populate { + # Add arguments from arg_queue until the number of arguments or + # max line length is reached + # Uses: + # $Global::minimal_command_line_length + # $opt::cat + # $opt::fifo + # $Global::JobQueue + # $opt::m + # $opt::X + # $CommandLine::already_spread + # $Global::max_jobs_running + # Returns: N/A + my $self = shift; + my $next_arg; + my $max_len = $Global::minimal_command_line_length || Limits::Command::max_length(); + + if($opt::cat or $opt::fifo) { + # Generate a tempfile name that will be used as {} + my($outfh,$name) = ::tmpfile(SUFFIX => ".pip"); + close $outfh; + # Unlink is needed if: ssh otheruser@localhost + unlink $name; + $Global::JobQueue->{'commandlinequeue'}->{'arg_queue'}->unget([Arg->new($name)]); + } + + while (not $self->{'arg_queue'}->empty()) { + $next_arg = $self->{'arg_queue'}->get(); + if(not defined $next_arg) { + next; + } + $self->push($next_arg); + if($self->len() >= $max_len) { + # Command length is now > max_length + # If there are arguments: remove the last + # If there are no arguments: Error + # TODO stuff about -x opt_x + if($self->number_of_args() > 1) { + # There is something to work on + $self->{'arg_queue'}->unget($self->pop()); + last; + } else { + my $args = join(" ", map { $_->orig() } @$next_arg); + ::error("Command line too long (", + $self->len(), " >= ", + $max_len, + ") at number ", + $self->{'arg_queue'}->arg_number(), + ": ". + (substr($args,0,50))."...\n"); + $self->{'arg_queue'}->unget($self->pop()); + ::wait_and_exit(255); + } + } + + if(defined $self->{'max_number_of_args'}) { + if($self->number_of_args() >= $self->{'max_number_of_args'}) { + last; + } + } + } + if(($opt::m or $opt::X) and not $CommandLine::already_spread + and $self->{'arg_queue'}->empty() and $Global::max_jobs_running) { + # -m or -X and EOF => Spread the arguments over all jobslots + # (unless they are already spread) + $CommandLine::already_spread ||= 1; + if($self->number_of_args() > 1) { + $self->{'max_number_of_args'} = + ::ceil($self->number_of_args()/$Global::max_jobs_running); + $Global::JobQueue->{'commandlinequeue'}->{'max_number_of_args'} = + $self->{'max_number_of_args'}; + $self->{'arg_queue'}->unget($self->pop_all()); + while($self->number_of_args() < $self->{'max_number_of_args'}) { + $self->push($self->{'arg_queue'}->get()); + } + } + } +} + +sub push { + # Add one or more records as arguments + # Returns: N/A + my $self = shift; + my $record = shift; + push @{$self->{'arg_list'}}, $record; + + my $quote_arg = $Global::noquote ? 0 : not $Global::quoting; + my $rep; + for my $arg (@$record) { + if(defined $arg) { + for my $perlexpr (keys %{$self->{'replacecount'}}) { + # 50% faster than below + $self->{'len'}{$perlexpr} += length $arg->replace($perlexpr,$quote_arg,$self); + # $rep = $arg->replace($perlexpr,$quote_arg,$self); + # $self->{'len'}{$perlexpr} += length $rep; + # ::debug("length", "Length: ", length $rep, + # "(", $perlexpr, "=>", $rep, ")\n"); + } + } + } +} + +sub pop { + # Remove last argument + # Returns: + # the last record + my $self = shift; + my $record = pop @{$self->{'arg_list'}}; + my $quote_arg = $Global::noquote ? 0 : not $Global::quoting; + for my $arg (@$record) { + if(defined $arg) { + for my $perlexpr (keys %{$self->{'replacecount'}}) { + $self->{'len'}{$perlexpr} -= + length $arg->replace($perlexpr,$quote_arg,$self); + } + } + } + return $record; +} + +sub pop_all { + # Remove all arguments and zeros the length of replacement strings + # Returns: + # all records + my $self = shift; + my @popped = @{$self->{'arg_list'}}; + for my $replacement_string (keys %{$self->{'replacecount'}}) { + $self->{'len'}{$replacement_string} = 0; + } + $self->{'arg_list'} = []; + return @popped; +} + +sub number_of_args { + # The number of records + # Returns: + # number of records + my $self = shift; + # Ftq rudef oaawuq ime dqxqmeqp az 2011-01-24 mzp ime iaz nk MQhmd + # Mdzrvadp Nvmdymeaz az 2011-04-10. Ftue oaawuq dqxqmeqp az + # 2013-08-18 ue m nuf tmdpqd me kag tmhq fa geq daf14. Bxqmeq + # qymux oaawuq@fmzsq.pw itqz kag dqmp ftue. + # + # U my ftq ymuzfmuzqd ar m buqoq ar rdqq earfimdq omxxqp SZG + # Bmdmxxqx. Rdqq earfimdq sgmdmzfqqe kag mooqee fa ftq eagdoq + # oapq, ngf U tmhq nqqz iazpqduzs tai ymzk mofgmxxk _dqmp_ ftq + # eagdoq oapq. + # + # Fa fqef ftue U bgf uz m oayyqzf fqxxuzs bqabxq fa qymux yq itqz + # ftqk dqmp ftue. Ftq oayyqzf ime bgf uz m eqofuaz ar ftq oapq + # ftmf za azq iagxp xaaw fa ruj ad uybdahq ftq earfimdq - ea ftq + # eagdoq oapq qcguhmxqzf fa m pgefk oadzqd. Fa ymwq egdq ftq + # oayyqzf iagxp zaf etai gb ur eayq azq vgef sdqbbqp ftdagst ftq + # eagdoq oapq U daf13'qp ftq eagdoq oapq + # tffb://qz.iuwubqpum.ads/iuwu/DAF13 + # + # 2.5 yazfte xmfqd U dqoquhqp mz qymux rday eayqazq ita zaf azxk + # ymzmsqp fa ruzp ftq oayyqzf, ngf mxea ymzmsqp fa sgqee ftq oapq + # tmp fa nq daf13'qp. + # + # Ftue nduzse yq fa ftq oazoxgeuaz ftmf ftqdq _mdq_ bqabxq, ita + # mdq zaf mrruxumfqp iuft ftq bdavqof, ftmf iuxx dqmp ftq eagdoq + # oapq - ftagst uf ymk zaf tmbbqz hqdk arfqz. + # + # This is really the number of records + return $#{$self->{'arg_list'}}+1; +} + +sub number_of_recargs { + # The number of args in records + # Returns: + # number of args records + my $self = shift; + my $sum = 0; + my $nrec = scalar @{$self->{'arg_list'}}; + if($nrec) { + $sum = $nrec * (scalar @{$self->{'arg_list'}[0]}); + } + return $sum; +} + +sub args_as_string { + # Returns: + # all unmodified arguments joined with ' ' (similar to {}) + my $self = shift; + return (join " ", map { $_->orig() } + map { @$_ } @{$self->{'arg_list'}}); +} + +sub args_as_dirname { + # Returns: + # all unmodified arguments joined with '/' (similar to {}) + # \t \0 \\ and / are quoted as: \t \0 \\ \_ + # If $Global::max_file_length: Keep subdirs < $Global::max_file_length + my $self = shift; + my @res = (); + + for my $rec_ref (@{$self->{'arg_list'}}) { + # If headers are used, sort by them. + # Otherwise keep the order from the command line. + my @header_indexes_sorted = header_indexes_sorted($#$rec_ref+1); + for my $n (@header_indexes_sorted) { + CORE::push(@res, + $Global::input_source_header{$n}, + map { my $s = $_; + # \t \0 \\ and / are quoted as: \t \0 \\ \_ + $s =~ s/\\/\\\\/g; + $s =~ s/\t/\\t/g; + $s =~ s/\0/\\0/g; + $s =~ s:/:\\_:g; + if($Global::max_file_length) { + # Keep each subdir shorter than the longest + # allowed file name + $s = substr($s,0,$Global::max_file_length); + } + $s; } + $rec_ref->[$n-1]->orig()); + } + } + return join "/", @res; +} + +sub header_indexes_sorted { + # Sort headers first by number then by name. + # E.g.: 1a 1b 11a 11b + # Returns: + # Indexes of %Global::input_source_header sorted + my $max_col = shift; + + no warnings 'numeric'; + for my $col (1 .. $max_col) { + # Make sure the header is defined. If it is not: use column number + if(not defined $Global::input_source_header{$col}) { + $Global::input_source_header{$col} = $col; + } + } + my @header_indexes_sorted = sort { + # Sort headers numerically then asciibetically + $Global::input_source_header{$a} <=> $Global::input_source_header{$b} + or + $Global::input_source_header{$a} cmp $Global::input_source_header{$b} + } 1 .. $max_col; + return @header_indexes_sorted; +} + +sub len { + # Uses: + # $opt::shellquote + # The length of the command line with args substituted + my $self = shift; + my $len = 0; + # Add length of the original command with no args + # Length of command w/ all replacement args removed + $len += $self->{'len'}{'noncontext'} + @{$self->{'command'}} -1; + ::debug("length", "noncontext + command: $len\n"); + my $recargs = $self->number_of_recargs(); + if($self->{'context_replace'}) { + # Context is duplicated for each arg + $len += $recargs * $self->{'len'}{'context'}; + for my $replstring (keys %{$self->{'replacecount'}}) { + # If the replacements string is more than once: mulitply its length + $len += $self->{'len'}{$replstring} * + $self->{'replacecount'}{$replstring}; + ::debug("length", $replstring, " ", $self->{'len'}{$replstring}, "*", + $self->{'replacecount'}{$replstring}, "\n"); + } + # echo 11 22 33 44 55 66 77 88 99 1010 + # echo 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 + # 5 + ctxgrp*arg + ::debug("length", "Ctxgrp: ", $self->{'len'}{'contextgroups'}, + " Groups: ", $self->{'len'}{'noncontextgroups'}, "\n"); + # Add space between context groups + $len += ($recargs-1) * ($self->{'len'}{'contextgroups'}); + } else { + # Each replacement string may occur several times + # Add the length for each time + $len += 1*$self->{'len'}{'context'}; + ::debug("length", "context+noncontext + command: $len\n"); + for my $replstring (keys %{$self->{'replacecount'}}) { + # (space between regargs + length of replacement) + # * number this replacement is used + $len += ($recargs -1 + $self->{'len'}{$replstring}) * + $self->{'replacecount'}{$replstring}; + } + } + if($opt::nice) { + # Pessimistic length if --nice is set + # Worse than worst case: every char needs to be quoted with \ + $len *= 2; + } + if($Global::quoting) { + # Pessimistic length if -q is set + # Worse than worst case: every char needs to be quoted with \ + $len *= 2; + } + if($opt::shellquote) { + # Pessimistic length if --shellquote is set + # Worse than worst case: every char needs to be quoted with \ twice + $len *= 4; + } + # If we are using --env, add the prefix for that, too. + $len += $Global::envvarlen; + + return $len; +} + +sub replaced { + # Uses: + # $Global::noquote + # $Global::quoting + # Returns: + # $replaced = command with place holders replaced and prepended + my $self = shift; + if(not defined $self->{'replaced'}) { + # Don't quote arguments if the input is the full command line + my $quote_arg = $Global::noquote ? 0 : not $Global::quoting; + $self->{'replaced'} = $self->replace_placeholders($self->{'command'},$Global::quoting,$quote_arg); + my $len = length $self->{'replaced'}; + if ($len != $self->len()) { + ::debug("length", $len, " != ", $self->len(), " ", $self->{'replaced'}, "\n"); + } else { + ::debug("length", $len, " == ", $self->len(), " ", $self->{'replaced'}, "\n"); + } + } + return $self->{'replaced'}; +} + +sub replace_placeholders { + # Replace foo{}bar with fooargbar + # Input: + # $targetref = command as shell words + # $quote = should everything be quoted? + # $quote_arg = should replaced arguments be quoted? + # Returns: + # @target with placeholders replaced + my $self = shift; + my $targetref = shift; + my $quote = shift; + my $quote_arg = shift; + my $context_replace = $self->{'context_replace'}; + my @target = @$targetref; + ::debug("replace", "Replace @target\n"); + # -X = context replace + # maybe multiple input sources + # maybe --xapply + if(not @target) { + # @target is empty: Return empty array + return @target; + } + # Fish out the words that have replacement strings in them + my %word; + for (@target) { + my $tt = $_; + ::debug("replace", "Target: $tt"); + # a{1}b{}c{}d + # a{=1 $_=$_ =}b{= $_=$_ =}c{= $_=$_ =}d + # a\257<1 $_=$_ \257>b\257< $_=$_ \257>c\257< $_=$_ \257>d + # A B C => aAbA B CcA B Cd + # -X A B C => aAbAcAd aAbBcBd aAbCcCd + + if($context_replace) { + while($tt =~ s/([^\s\257]* # before {= + (?: + \257< # {= + [^\257]*? # The perl expression + \257> # =} + [^\s\257]* # after =} + )+)/ /x) { + # $1 = pre \257 perlexpr \257 post + $word{"$1"} ||= 1; + } + } else { + while($tt =~ s/( (?: \257<([^\257]*?)\257>) )//x) { + # $f = \257 perlexpr \257 + $word{$1} ||= 1; + } + } + } + my @word = keys %word; + + my %replace; + my @arg; + for my $record (@{$self->{'arg_list'}}) { + # $self->{'arg_list'} = [ [Arg11, Arg12], [Arg21, Arg22], [Arg31, Arg32] ] + # Merge arg-objects from records into @arg for easy access + CORE::push @arg, @$record; + } + # Add one arg if empty to allow {#} and {%} to be computed only once + if(not @arg) { @arg = (Arg->new("")); } + # Number of arguments - used for positional arguments + my $n = $#_+1; + + # This is actually a CommandLine-object, + # but it looks nice to be able to say {= $job->slot() =} + my $job = $self; + for my $word (@word) { + # word = AB \257< perlexpr \257> CD \257< perlexpr \257> EF + my $w = $word; + ::debug("replace", "Replacing in $w\n"); + + # Replace positional arguments + $w =~ s< ([^\s\257]*) # before {= + \257< # {= + (-?\d+) # Position (eg. -2 or 3) + ([^\257]*?) # The perl expression + \257> # =} + ([^\s\257]*) # after =} + > + { $1. # Context (pre) + ( + $arg[$2 > 0 ? $2-1 : $n+$2] ? # If defined: replace + $arg[$2 > 0 ? $2-1 : $n+$2]->replace($3,$quote_arg,$self) + : "") + .$4 }egx;# Context (post) + ::debug("replace", "Positional replaced $word with: $w\n"); + + if($w !~ /\257/) { + # No more replacement strings in $w: No need to do more + if($quote) { + CORE::push(@{$replace{::shell_quote($word)}}, $w); + } else { + CORE::push(@{$replace{$word}}, $w); + } + next; + } + # for each arg: + # compute replacement for each string + # replace replacement strings with replacement in the word value + # push to replace word value + ::debug("replace", "Positional done: $w\n"); + for my $arg (@arg) { + my $val = $w; + my $number_of_replacements = 0; + for my $perlexpr (keys %{$self->{'replacecount'}}) { + # Replace {= perl expr =} with value for each arg + $number_of_replacements += + $val =~ s{\257<\Q$perlexpr\E\257>} + {$arg ? $arg->replace($perlexpr,$quote_arg,$self) : ""}eg; + } + my $ww = $word; + if($quote) { + $ww = ::shell_quote_scalar($word); + $val = ::shell_quote_scalar($val); + } + if($number_of_replacements) { + CORE::push(@{$replace{$ww}}, $val); + } + } + } + + if($quote) { + @target = ::shell_quote(@target); + } + # ::debug("replace", "%replace=",::my_dump(%replace),"\n"); + if(%replace) { + # Substitute the replace strings with the replacement values + # Must be sorted by length if a short word is a substring of a long word + my $regexp = join('|', map { my $s = $_; $s =~ s/(\W)/\\$1/g; $s } + sort { length $b <=> length $a } keys %replace); + for(@target) { + s/($regexp)/join(" ",@{$replace{$1}})/ge; + } + } + ::debug("replace", "Return @target\n"); + return wantarray ? @target : "@target"; +} + + +package CommandLineQueue; + +sub new { + my $class = shift; + my $commandref = shift; + my $read_from = shift; + my $context_replace = shift; + my $max_number_of_args = shift; + my $return_files = shift; + my @unget = (); + my ($count,%replacecount,$posrpl,$perlexpr,%len); + my @command = @$commandref; + # If the first command start with '-' it is probably an option + if($command[0] =~ /^\s*(-\S+)/) { + # Is this really a command in $PATH starting with '-'? + my $cmd = $1; + if(not ::which($cmd)) { + ::error("Command ($cmd) starts with '-'. Is this a wrong option?\n"); + ::wait_and_exit(255); + } + } + # Replace replacement strings with {= perl expr =} + # Protect matching inside {= perl expr =} + # by replacing {= and =} with \257< and \257> + for(@command) { + if(/\257/) { + ::error("Command cannot contain the character \257. Use a function for that.\n"); + ::wait_and_exit(255); + } + s/\Q$Global::parensleft\E(.*?)\Q$Global::parensright\E/\257<$1\257>/gx; + } + for my $rpl (keys %Global::rpl) { + # Replace the short hand string with the {= perl expr =} in $command and $opt::tagstring + # Avoid replacing inside existing {= perl expr =} + for(@command,@Global::ret_files) { + while(s/((^|\257>)[^\257]*?) # Don't replace after \257 unless \257> + \Q$rpl\E/$1\257<$Global::rpl{$rpl}\257>/xg) { + } + } + if(defined $opt::tagstring) { + for($opt::tagstring) { + while(s/((^|\257>)[^\257]*?) # Don't replace after \257 unless \257> + \Q$rpl\E/$1\257<$Global::rpl{$rpl}\257>/x) {} + } + } + # Do the same for the positional replacement strings + # A bit harder as we have to put in the position number + $posrpl = $rpl; + if($posrpl =~ s/^\{//) { + # Only do this if the shorthand start with { + for(@command,@Global::ret_files) { + s/\{(-?\d+)\Q$posrpl\E/\257<$1 $Global::rpl{$rpl}\257>/g; + } + if(defined $opt::tagstring) { + $opt::tagstring =~ s/\{(-?\d+)\Q$posrpl\E/\257<$1 $perlexpr\257>/g; + } + } + } + my $sum = 0; + while($sum == 0) { + # Count how many times each replacement string is used + my @cmd = @command; + my $contextlen = 0; + my $noncontextlen = 0; + my $contextgroups = 0; + for my $c (@cmd) { + while($c =~ s/ \257<([^\257]*?)\257> /\000/x) { + # %replacecount = { "perlexpr" => number of times seen } + # e.g { "$_++" => 2 } + $replacecount{$1} ++; + $sum++; + } + # Measure the length of the context around the {= perl expr =} + # Use that {=...=} has been replaced with \000 above + # So there is no need to deal with \257< + while($c =~ s/ (\S*\000\S*) //x) { + my $w = $1; + $w =~ tr/\000//d; # Remove all \000's + $contextlen += length($w); + $contextgroups++; + } + # All {= perl expr =} have been removed: The rest is non-context + $noncontextlen += length $c; + } + if($opt::tagstring) { + my $t = $opt::tagstring; + while($t =~ s/ \257<([^\257]*)\257> //x) { + # %replacecount = { "perlexpr" => number of times seen } + # e.g { "$_++" => 2 } + # But for tagstring we just need to mark it as seen + $replacecount{$1}||=1; + } + } + + $len{'context'} = 0+$contextlen; + $len{'noncontext'} = $noncontextlen; + $len{'contextgroups'} = $contextgroups; + $len{'noncontextgroups'} = @cmd-$contextgroups; + ::debug("length", "@command Context: ", $len{'context'}, + " Non: ", $len{'noncontext'}, " Ctxgrp: ", $len{'contextgroups'}, + " NonCtxGrp: ", $len{'noncontextgroups'}, "\n"); + if($sum == 0) { + # Default command = {} + # If not replacement string: append {} + if(not @command) { + @command = ("\257<\257>"); + $Global::noquote = 1; + } elsif(($opt::pipe or $opt::pipepart) + and not $opt::fifo and not $opt::cat) { + # With --pipe / --pipe-part you can have no replacement + last; + } else { + # Append {} to the command if there are no {...}'s and no {=...=} + push @command, ("\257<\257>"); + } + } + } + + return bless { + 'unget' => \@unget, + 'command' => \@command, + 'replacecount' => \%replacecount, + 'arg_queue' => RecordQueue->new($read_from,$opt::colsep), + 'context_replace' => $context_replace, + 'len' => \%len, + 'max_number_of_args' => $max_number_of_args, + 'size' => undef, + 'return_files' => $return_files, + 'seq' => 1, + }, ref($class) || $class; +} + +sub get { + my $self = shift; + if(@{$self->{'unget'}}) { + my $cmd_line = shift @{$self->{'unget'}}; + return ($cmd_line); + } else { + my $cmd_line; + $cmd_line = CommandLine->new($self->seq(), + $self->{'command'}, + $self->{'arg_queue'}, + $self->{'context_replace'}, + $self->{'max_number_of_args'}, + $self->{'return_files'}, + $self->{'replacecount'}, + $self->{'len'}, + ); + $cmd_line->populate(); + ::debug("init","cmd_line->number_of_args ", + $cmd_line->number_of_args(), "\n"); + if($opt::pipe or $opt::pipepart) { + if($cmd_line->replaced() eq "") { + # Empty command - pipe requires a command + ::error("--pipe must have a command to pipe into (e.g. 'cat').\n"); + ::wait_and_exit(255); + } + } else { + if($cmd_line->number_of_args() == 0) { + # We did not get more args - maybe at EOF string? + return undef; + } elsif($cmd_line->replaced() eq "") { + # Empty command - get the next instead + return $self->get(); + } + } + $self->set_seq($self->seq()+1); + return $cmd_line; + } +} + +sub unget { + my $self = shift; + unshift @{$self->{'unget'}}, @_; +} + +sub empty { + my $self = shift; + my $empty = (not @{$self->{'unget'}}) && $self->{'arg_queue'}->empty(); + ::debug("run", "CommandLineQueue->empty $empty"); + return $empty; +} + +sub seq { + my $self = shift; + return $self->{'seq'}; +} + +sub set_seq { + my $self = shift; + $self->{'seq'} = shift; +} + +sub quote_args { + my $self = shift; + # If there is not command emulate |bash + return $self->{'command'}; +} + +sub size { + my $self = shift; + if(not $self->{'size'}) { + my @all_lines = (); + while(not $self->{'arg_queue'}->empty()) { + push @all_lines, CommandLine->new($self->{'command'}, + $self->{'arg_queue'}, + $self->{'context_replace'}, + $self->{'max_number_of_args'}); + } + $self->{'size'} = @all_lines; + $self->unget(@all_lines); + } + return $self->{'size'}; +} + + +package Limits::Command; + +# Maximal command line length (for -m and -X) +sub max_length { + # Find the max_length of a command line and cache it + # Returns: + # number of chars on the longest command line allowed + if(not $Limits::Command::line_max_len) { + # Disk cache of max command line length + my $len_cache = $ENV{'HOME'} . "/.parallel/tmp/linelen-" . ::hostname(); + my $cached_limit; + if(-e $len_cache) { + open(my $fh, "<", $len_cache) || ::die_bug("Cannot read $len_cache"); + $cached_limit = <$fh>; + close $fh; + } else { + $cached_limit = real_max_length(); + # If $HOME is write protected: Do not fail + mkdir($ENV{'HOME'} . "/.parallel"); + mkdir($ENV{'HOME'} . "/.parallel/tmp"); + open(my $fh, ">", $len_cache); + print $fh $cached_limit; + close $fh; + } + $Limits::Command::line_max_len = $cached_limit; + if($opt::max_chars) { + if($opt::max_chars <= $cached_limit) { + $Limits::Command::line_max_len = $opt::max_chars; + } else { + ::warning("Value for -s option ", + "should be < $cached_limit.\n"); + } + } + } + return $Limits::Command::line_max_len; +} + +sub real_max_length { + # Find the max_length of a command line + # Returns: + # The maximal command line length + # Use an upper bound of 8 MB if the shell allows for for infinite long lengths + my $upper = 8_000_000; + my $len = 8; + do { + if($len > $upper) { return $len }; + $len *= 16; + } while (is_acceptable_command_line_length($len)); + # Then search for the actual max length between 0 and upper bound + return binary_find_max_length(int($len/16),$len); +} + +sub binary_find_max_length { + # Given a lower and upper bound find the max_length of a command line + # Returns: + # number of chars on the longest command line allowed + my ($lower, $upper) = (@_); + if($lower == $upper or $lower == $upper-1) { return $lower; } + my $middle = int (($upper-$lower)/2 + $lower); + ::debug("init", "Maxlen: $lower,$upper,$middle : "); + if (is_acceptable_command_line_length($middle)) { + return binary_find_max_length($middle,$upper); + } else { + return binary_find_max_length($lower,$middle); + } +} + +sub is_acceptable_command_line_length { + # Test if a command line of this length can run + # Returns: + # 0 if the command line length is too long + # 1 otherwise + my $len = shift; + + local *STDERR; + open (STDERR, ">", "/dev/null"); + system "true "."x"x$len; + close STDERR; + ::debug("init", "$len=$? "); + return not $?; +} + + +package RecordQueue; + +sub new { + my $class = shift; + my $fhs = shift; + my $colsep = shift; + my @unget = (); + my $arg_sub_queue; + if($colsep) { + # Open one file with colsep + $arg_sub_queue = RecordColQueue->new($fhs); + } else { + # Open one or more files if multiple -a + $arg_sub_queue = MultifileQueue->new($fhs); + } + return bless { + 'unget' => \@unget, + 'arg_number' => 0, + 'arg_sub_queue' => $arg_sub_queue, + }, ref($class) || $class; +} + +sub get { + # Returns: + # reference to array of Arg-objects + my $self = shift; + if(@{$self->{'unget'}}) { + $self->{'arg_number'}++; + return shift @{$self->{'unget'}}; + } + my $ret = $self->{'arg_sub_queue'}->get(); + if(defined $Global::max_number_of_args + and $Global::max_number_of_args == 0) { + ::debug("run", "Read 1 but return 0 args\n"); + return [Arg->new("")]; + } else { + return $ret; + } +} + +sub unget { + my $self = shift; + ::debug("run", "RecordQueue-unget '@_'\n"); + $self->{'arg_number'} -= @_; + unshift @{$self->{'unget'}}, @_; +} + +sub empty { + my $self = shift; + my $empty = not @{$self->{'unget'}}; + $empty &&= $self->{'arg_sub_queue'}->empty(); + ::debug("run", "RecordQueue->empty $empty"); + return $empty; +} + +sub arg_number { + my $self = shift; + return $self->{'arg_number'}; +} + + +package RecordColQueue; + +sub new { + my $class = shift; + my $fhs = shift; + my @unget = (); + my $arg_sub_queue = MultifileQueue->new($fhs); + return bless { + 'unget' => \@unget, + 'arg_sub_queue' => $arg_sub_queue, + }, ref($class) || $class; +} + +sub get { + # Returns: + # reference to array of Arg-objects + my $self = shift; + if(@{$self->{'unget'}}) { + return shift @{$self->{'unget'}}; + } + my $unget_ref=$self->{'unget'}; + if($self->{'arg_sub_queue'}->empty()) { + return undef; + } + my $in_record = $self->{'arg_sub_queue'}->get(); + if(defined $in_record) { + my @out_record = (); + for my $arg (@$in_record) { + ::debug("run", "RecordColQueue::arg $arg\n"); + my $line = $arg->orig(); + ::debug("run", "line='$line'\n"); + if($line ne "") { + for my $s (split /$opt::colsep/o, $line, -1) { + push @out_record, Arg->new($s); + } + } else { + push @out_record, Arg->new(""); + } + } + return \@out_record; + } else { + return undef; + } +} + +sub unget { + my $self = shift; + ::debug("run", "RecordColQueue-unget '@_'\n"); + unshift @{$self->{'unget'}}, @_; +} + +sub empty { + my $self = shift; + my $empty = (not @{$self->{'unget'}} and $self->{'arg_sub_queue'}->empty()); + ::debug("run", "RecordColQueue->empty $empty"); + return $empty; +} + + +package MultifileQueue; + +@Global::unget_argv=(); + +sub new { + my $class = shift; + my $fhs = shift; + for my $fh (@$fhs) { + if(-t $fh) { + ::warning("Input is read from the terminal. ". + "Only experts do this on purpose. ". + "Press CTRL-D to exit.\n"); + } + } + return bless { + 'unget' => \@Global::unget_argv, + 'fhs' => $fhs, + 'arg_matrix' => undef, + }, ref($class) || $class; +} + +sub get { + my $self = shift; + if($opt::xapply) { + return $self->xapply_get(); + } else { + return $self->nest_get(); + } +} + +sub unget { + my $self = shift; + ::debug("run", "MultifileQueue-unget '@_'\n"); + unshift @{$self->{'unget'}}, @_; +} + +sub empty { + my $self = shift; + my $empty = (not @Global::unget_argv + and not @{$self->{'unget'}}); + for my $fh (@{$self->{'fhs'}}) { + $empty &&= eof($fh); + } + ::debug("run", "MultifileQueue->empty $empty "); + return $empty; +} + +sub xapply_get { + my $self = shift; + if(@{$self->{'unget'}}) { + return shift @{$self->{'unget'}}; + } + my @record = (); + my $prepend = undef; + my $empty = 1; + for my $fh (@{$self->{'fhs'}}) { + my $arg = read_arg_from_fh($fh); + if(defined $arg) { + # Record $arg for recycling at end of file + push @{$self->{'arg_matrix'}{$fh}}, $arg; + push @record, $arg; + $empty = 0; + } else { + ::debug("run", "EOA "); + # End of file: Recycle arguments + push @{$self->{'arg_matrix'}{$fh}}, shift @{$self->{'arg_matrix'}{$fh}}; + # return last @{$args->{'args'}{$fh}}; + push @record, @{$self->{'arg_matrix'}{$fh}}[-1]; + } + } + if($empty) { + return undef; + } else { + return \@record; + } +} + +sub nest_get { + my $self = shift; + if(@{$self->{'unget'}}) { + return shift @{$self->{'unget'}}; + } + my @record = (); + my $prepend = undef; + my $empty = 1; + my $no_of_inputsources = $#{$self->{'fhs'}} + 1; + if(not $self->{'arg_matrix'}) { + # Initialize @arg_matrix with one arg from each file + # read one line from each file + my @first_arg_set; + my $all_empty = 1; + for (my $fhno = 0; $fhno < $no_of_inputsources ; $fhno++) { + my $arg = read_arg_from_fh($self->{'fhs'}[$fhno]); + if(defined $arg) { + $all_empty = 0; + } + $self->{'arg_matrix'}[$fhno][0] = $arg || Arg->new(""); + push @first_arg_set, $self->{'arg_matrix'}[$fhno][0]; + } + if($all_empty) { + # All filehandles were at eof or eof-string + return undef; + } + return [@first_arg_set]; + } + + # Treat the case with one input source special. For multiple + # input sources we need to remember all previously read values to + # generate all combinations. But for one input source we can + # forget the value after first use. + if($no_of_inputsources == 1) { + my $arg = read_arg_from_fh($self->{'fhs'}[0]); + if(defined($arg)) { + return [$arg]; + } + return undef; + } + for (my $fhno = $no_of_inputsources - 1; $fhno >= 0; $fhno--) { + if(eof($self->{'fhs'}[$fhno])) { + next; + } else { + # read one + my $arg = read_arg_from_fh($self->{'fhs'}[$fhno]); + defined($arg) || next; # If we just read an EOF string: Treat this as EOF + my $len = $#{$self->{'arg_matrix'}[$fhno]} + 1; + $self->{'arg_matrix'}[$fhno][$len] = $arg; + # make all new combinations + my @combarg = (); + for (my $fhn = 0; $fhn < $no_of_inputsources; $fhn++) { + push @combarg, [0, $#{$self->{'arg_matrix'}[$fhn]}]; + } + $combarg[$fhno] = [$len,$len]; # Find only combinations with this new entry + # map combinations + # [ 1, 3, 7 ], [ 2, 4, 1 ] + # => + # [ m[0][1], m[1][3], m[3][7] ], [ m[0][2], m[1][4], m[2][1] ] + my @mapped; + for my $c (expand_combinations(@combarg)) { + my @a; + for my $n (0 .. $no_of_inputsources - 1 ) { + push @a, $self->{'arg_matrix'}[$n][$$c[$n]]; + } + push @mapped, \@a; + } + # append the mapped to the ungotten arguments + push @{$self->{'unget'}}, @mapped; + # get the first + return shift @{$self->{'unget'}}; + } + } + # all are eof or at EOF string; return from the unget queue + return shift @{$self->{'unget'}}; +} + +sub read_arg_from_fh { + # Read one Arg from filehandle + # Returns: + # Arg-object with one read line + # undef if end of file + my $fh = shift; + my $prepend = undef; + my $arg; + do {{ + # This makes 10% faster + if(not ($arg = <$fh>)) { + if(defined $prepend) { + return Arg->new($prepend); + } else { + return undef; + } + } +# ::debug("run", "read $arg\n"); + # Remove delimiter + $arg =~ s:$/$::; + if($Global::end_of_file_string and + $arg eq $Global::end_of_file_string) { + # Ignore the rest of input file + close $fh; + ::debug("run", "EOF-string ($arg) met\n"); + if(defined $prepend) { + return Arg->new($prepend); + } else { + return undef; + } + } + if(defined $prepend) { + $arg = $prepend.$arg; # For line continuation + $prepend = undef; #undef; + } + if($Global::ignore_empty) { + if($arg =~ /^\s*$/) { + redo; # Try the next line + } + } + if($Global::max_lines) { + if($arg =~ /\s$/) { + # Trailing space => continued on next line + $prepend = $arg; + redo; + } + } + }} while (1 == 0); # Dummy loop {{}} for redo + if(defined $arg) { + return Arg->new($arg); + } else { + ::die_bug("multiread arg undefined"); + } +} + +sub expand_combinations { + # Input: + # ([xmin,xmax], [ymin,ymax], ...) + # Returns: ([x,y,...],[x,y,...]) + # where xmin <= x <= xmax and ymin <= y <= ymax + my $minmax_ref = shift; + my $xmin = $$minmax_ref[0]; + my $xmax = $$minmax_ref[1]; + my @p; + if(@_) { + # If there are more columns: Compute those recursively + my @rest = expand_combinations(@_); + for(my $x = $xmin; $x <= $xmax; $x++) { + push @p, map { [$x, @$_] } @rest; + } + } else { + for(my $x = $xmin; $x <= $xmax; $x++) { + push @p, [$x]; + } + } + return @p; +} + + +package Arg; + +sub new { + my $class = shift; + my $orig = shift; + my @hostgroups; + if($opt::hostgroups) { + if($orig =~ s:@(.+)::) { + # We found hostgroups on the arg + @hostgroups = split(/\+/, $1); + if(not grep { defined $Global::hostgroups{$_} } @hostgroups) { + ::warning("No such hostgroup (@hostgroups)\n"); + @hostgroups = (keys %Global::hostgroups); + } + } else { + @hostgroups = (keys %Global::hostgroups); + } + } + return bless { + 'orig' => $orig, + 'hostgroups' => \@hostgroups, + }, ref($class) || $class; +} + +sub replace { + # Calculates the corresponding value for a given perl expression + # Returns: + # The calculated string (quoted if asked for) + my $self = shift; + my $perlexpr = shift; # E.g. $_=$_ or s/.gz// + my $quote = (shift) ? 1 : 0; # should the string be quoted? + # This is actually a CommandLine-object, + # but it looks nice to be able to say {= $job->slot() =} + my $job = shift; + $perlexpr =~ s/^-?\d+ //; # Positional replace treated as normal replace + if(not defined $self->{"rpl",0,$perlexpr}) { + local $_; + if($Global::trim eq "n") { + $_ = $self->{'orig'}; + } else { + $_ = trim_of($self->{'orig'}); + } + ::debug("replace", "eval ", $perlexpr, " ", $_, "\n"); + if(not $Global::perleval{$perlexpr}) { + # Make an anonymous function of the $perlexpr + # And more importantly: Compile it only once + if($Global::perleval{$perlexpr} = + eval('sub { no strict; no warnings; my $job = shift; '. + $perlexpr.' }')) { + # All is good + } else { + # The eval failed. Maybe $perlexpr is invalid perl? + ::error("Cannot use $perlexpr: $@\n"); + ::wait_and_exit(255); + } + } + # Execute the function + $Global::perleval{$perlexpr}->($job); + $self->{"rpl",0,$perlexpr} = $_; + } + if(not defined $self->{"rpl",$quote,$perlexpr}) { + $self->{"rpl",1,$perlexpr} = + ::shell_quote_scalar($self->{"rpl",0,$perlexpr}); + } + return $self->{"rpl",$quote,$perlexpr}; +} + +sub orig { + my $self = shift; + return $self->{'orig'}; +} + +sub trim_of { + # Removes white space as specifed by --trim: + # n = nothing + # l = start + # r = end + # lr|rl = both + # Returns: + # string with white space removed as needed + my @strings = map { defined $_ ? $_ : "" } (@_); + my $arg; + if($Global::trim eq "n") { + # skip + } elsif($Global::trim eq "l") { + for my $arg (@strings) { $arg =~ s/^\s+//; } + } elsif($Global::trim eq "r") { + for my $arg (@strings) { $arg =~ s/\s+$//; } + } elsif($Global::trim eq "rl" or $Global::trim eq "lr") { + for my $arg (@strings) { $arg =~ s/^\s+//; $arg =~ s/\s+$//; } + } else { + ::error("--trim must be one of: r l rl lr.\n"); + ::wait_and_exit(255); + } + return wantarray ? @strings : "@strings"; +} + + +package TimeoutQueue; + +sub new { + my $class = shift; + my $delta_time = shift; + my ($pct); + if($delta_time =~ /(\d+(\.\d+)?)%/) { + # Timeout in percent + $pct = $1/100; + $delta_time = 1_000_000; + } + return bless { + 'queue' => [], + 'delta_time' => $delta_time, + 'pct' => $pct, + 'remedian_idx' => 0, + 'remedian_arr' => [], + 'remedian' => undef, + }, ref($class) || $class; +} + +sub delta_time { + my $self = shift; + return $self->{'delta_time'}; +} + +sub set_delta_time { + my $self = shift; + $self->{'delta_time'} = shift; +} + +sub remedian { + my $self = shift; + return $self->{'remedian'}; +} + +sub set_remedian { + # Set median of the last 999^3 (=997002999) values using Remedian + # + # Rousseeuw, Peter J., and Gilbert W. Bassett Jr. "The remedian: A + # robust averaging method for large data sets." Journal of the + # American Statistical Association 85.409 (1990): 97-104. + my $self = shift; + my $val = shift; + my $i = $self->{'remedian_idx'}++; + my $rref = $self->{'remedian_arr'}; + $rref->[0][$i%999] = $val; + $rref->[1][$i/999%999] = (sort @{$rref->[0]})[$#{$rref->[0]}/2]; + $rref->[2][$i/999/999%999] = (sort @{$rref->[1]})[$#{$rref->[1]}/2]; + $self->{'remedian'} = (sort @{$rref->[2]})[$#{$rref->[2]}/2]; +} + +sub update_delta_time { + # Update delta_time based on runtime of finished job if timeout is + # a percentage + my $self = shift; + my $runtime = shift; + if($self->{'pct'}) { + $self->set_remedian($runtime); + $self->{'delta_time'} = $self->{'pct'} * $self->remedian(); + ::debug("run", "Timeout: $self->{'delta_time'}s "); + } +} + +sub process_timeouts { + # Check if there was a timeout + my $self = shift; + # $self->{'queue'} is sorted by start time + while (@{$self->{'queue'}}) { + my $job = $self->{'queue'}[0]; + if($job->endtime()) { + # Job already finished. No need to timeout the job + # This could be because of --keep-order + shift @{$self->{'queue'}}; + } elsif($job->timedout($self->{'delta_time'})) { + # Need to shift off queue before kill + # because kill calls usleep that calls process_timeouts + shift @{$self->{'queue'}}; + $job->kill(); + } else { + # Because they are sorted by start time the rest are later + last; + } + } +} + +sub insert { + my $self = shift; + my $in = shift; + push @{$self->{'queue'}}, $in; +} + + +package Semaphore; + +# This package provides a counting semaphore +# +# If a process dies without releasing the semaphore the next process +# that needs that entry will clean up dead semaphores +# +# The semaphores are stored in ~/.parallel/semaphores/id- Each +# file in ~/.parallel/semaphores/id-/ is the process ID of the +# process holding the entry. If the process dies, the entry can be +# taken by another process. + +sub new { + my $class = shift; + my $id = shift; + my $count = shift; + $id=~s/([^-_a-z0-9])/unpack("H*",$1)/ige; # Convert non-word chars to hex + $id="id-".$id; # To distinguish it from a process id + my $parallel_dir = $ENV{'HOME'}."/.parallel"; + -d $parallel_dir or mkdir_or_die($parallel_dir); + my $parallel_locks = $parallel_dir."/semaphores"; + -d $parallel_locks or mkdir_or_die($parallel_locks); + my $lockdir = "$parallel_locks/$id"; + my $lockfile = $lockdir.".lock"; + if($count < 1) { ::die_bug("semaphore-count: $count"); } + return bless { + 'lockfile' => $lockfile, + 'lockfh' => Symbol::gensym(), + 'lockdir' => $lockdir, + 'id' => $id, + 'idfile' => $lockdir."/".$id, + 'pid' => $$, + 'pidfile' => $lockdir."/".$$.'@'.::hostname(), + 'count' => $count + 1 # nlinks returns a link for the 'id-' as well + }, ref($class) || $class; +} + +sub acquire { + my $self = shift; + my $sleep = 1; # 1 ms + my $start_time = time; + while(1) { + $self->atomic_link_if_count_less_than() and last; + ::debug("sem", "Remove dead locks"); + my $lockdir = $self->{'lockdir'}; + for my $d (glob "$lockdir/*") { + ::debug("sem", "Lock $d $lockdir\n"); + $d =~ m:$lockdir/([0-9]+)\@([-\._a-z0-9]+)$:o or next; + my ($pid, $host) = ($1, $2); + if($host eq ::hostname()) { + if(not kill 0, $1) { + ::debug("sem", "Dead: $d"); + unlink $d; + } else { + ::debug("sem", "Alive: $d"); + } + } + } + # try again + $self->atomic_link_if_count_less_than() and last; + # Retry slower and slower up to 1 second + $sleep = ($sleep < 1000) ? ($sleep * 1.1) : ($sleep); + # Random to avoid every sleeping job waking up at the same time + ::usleep(rand()*$sleep); + if(defined($opt::timeout) and + $start_time + $opt::timeout > time) { + # Acquire the lock anyway + if(not -e $self->{'idfile'}) { + open (my $fh, ">", $self->{'idfile'}) or + ::die_bug("timeout_write_idfile: $self->{'idfile'}"); + close $fh; + } + link $self->{'idfile'}, $self->{'pidfile'}; + last; + } + } + ::debug("sem", "acquired $self->{'pid'}\n"); +} + +sub release { + my $self = shift; + unlink $self->{'pidfile'}; + if($self->nlinks() == 1) { + # This is the last link, so atomic cleanup + $self->lock(); + if($self->nlinks() == 1) { + unlink $self->{'idfile'}; + rmdir $self->{'lockdir'}; + } + $self->unlock(); + } + ::debug("run", "released $self->{'pid'}\n"); +} + +sub _release { + my $self = shift; + + unlink $self->{'pidfile'}; + $self->lock(); + my $nlinks = $self->nlinks(); + ::debug("sem", $nlinks, "<", $self->{'count'}); + if($nlinks-- > 1) { + unlink $self->{'idfile'}; + open (my $fh, ">", $self->{'idfile'}) or + ::die_bug("write_idfile: $self->{'idfile'}"); + print $fh "#"x$nlinks; + close $fh; + } else { + unlink $self->{'idfile'}; + rmdir $self->{'lockdir'}; + } + $self->unlock(); + ::debug("sem", "released $self->{'pid'}\n"); +} + +sub atomic_link_if_count_less_than { + # Link $file1 to $file2 if nlinks to $file1 < $count + my $self = shift; + my $retval = 0; + $self->lock(); + ::debug($self->nlinks(), "<", $self->{'count'}); + if($self->nlinks() < $self->{'count'}) { + -d $self->{'lockdir'} or mkdir_or_die($self->{'lockdir'}); + if(not -e $self->{'idfile'}) { + open (my $fh, ">", $self->{'idfile'}) or + ::die_bug("write_idfile: $self->{'idfile'}"); + close $fh; + } + $retval = link $self->{'idfile'}, $self->{'pidfile'}; + } + $self->unlock(); + ::debug("run", "atomic $retval"); + return $retval; +} + +sub _atomic_link_if_count_less_than { + # Link $file1 to $file2 if nlinks to $file1 < $count + my $self = shift; + my $retval = 0; + $self->lock(); + my $nlinks = $self->nlinks(); + ::debug("sem", $nlinks, "<", $self->{'count'}); + if($nlinks++ < $self->{'count'}) { + -d $self->{'lockdir'} or mkdir_or_die($self->{'lockdir'}); + if(not -e $self->{'idfile'}) { + open (my $fh, ">", $self->{'idfile'}) or + ::die_bug("write_idfile: $self->{'idfile'}"); + close $fh; + } + open (my $fh, ">", $self->{'idfile'}) or + ::die_bug("write_idfile: $self->{'idfile'}"); + print $fh "#"x$nlinks; + close $fh; + $retval = link $self->{'idfile'}, $self->{'pidfile'}; + } + $self->unlock(); + ::debug("sem", "atomic $retval"); + return $retval; +} + +sub nlinks { + my $self = shift; + if(-e $self->{'idfile'}) { + ::debug("sem", "nlinks", (stat(_))[3], "size", (stat(_))[7], "\n"); + return (stat(_))[3]; + } else { + return 0; + } +} + +sub lock { + my $self = shift; + my $sleep = 100; # 100 ms + my $total_sleep = 0; + $Global::use{"Fcntl"} ||= eval "use Fcntl qw(:DEFAULT :flock); 1;"; + my $locked = 0; + while(not $locked) { + if(tell($self->{'lockfh'}) == -1) { + # File not open + open($self->{'lockfh'}, ">", $self->{'lockfile'}) + or ::debug("run", "Cannot open $self->{'lockfile'}"); + } + if($self->{'lockfh'}) { + # File is open + chmod 0666, $self->{'lockfile'}; # assuming you want it a+rw + if(flock($self->{'lockfh'}, LOCK_EX()|LOCK_NB())) { + # The file is locked: No need to retry + $locked = 1; + last; + } else { + if ($! =~ m/Function not implemented/) { + ::warning("flock: $!"); + ::warning("Will wait for a random while\n"); + ::usleep(rand(5000)); + # File cannot be locked: No need to retry + $locked = 2; + last; + } + } + } + # Locking failed in first round + # Sleep and try again + $sleep = ($sleep < 1000) ? ($sleep * 1.1) : ($sleep); + # Random to avoid every sleeping job waking up at the same time + ::usleep(rand()*$sleep); + $total_sleep += $sleep; + if($opt::semaphoretimeout) { + if($total_sleep/1000 > $opt::semaphoretimeout) { + # Timeout: bail out + ::warning("Semaphore timed out. Ignoring timeout."); + $locked = 3; + last; + } + } else { + if($total_sleep/1000 > 30) { + ::warning("Semaphore stuck for 30 seconds. Consider using --semaphoretimeout."); + } + } + } + ::debug("run", "locked $self->{'lockfile'}"); +} + +sub unlock { + my $self = shift; + unlink $self->{'lockfile'}; + close $self->{'lockfh'}; + ::debug("run", "unlocked\n"); +} + +sub mkdir_or_die { + # If dir is not writable: die + my $dir = shift; + my @dir_parts = split(m:/:,$dir); + my ($ddir,$part); + while(defined ($part = shift @dir_parts)) { + $part eq "" and next; + $ddir .= "/".$part; + -d $ddir and next; + mkdir $ddir; + } + if(not -w $dir) { + ::error("Cannot write to $dir: $!\n"); + ::wait_and_exit(255); + } +} + +# Keep perl -w happy +$opt::x = $Semaphore::timeout = $Semaphore::wait = +$Job::file_descriptor_warning_printed = 0; diff --git a/build_tools/mac-install-gflags.sh b/build_tools/mac-install-gflags.sh deleted file mode 100755 index a245a26a894..00000000000 --- a/build_tools/mac-install-gflags.sh +++ /dev/null @@ -1,25 +0,0 @@ -#!/bin/sh -# Install gflags for mac developers. - -set -e - -DIR=`mktemp -d /tmp/rocksdb_gflags_XXXX` - -cd $DIR -wget https://gflags.googlecode.com/files/gflags-2.0.tar.gz -tar xvfz gflags-2.0.tar.gz -cd gflags-2.0 - -./configure -make -make install - -# Add include/lib path for g++ -echo 'export LIBRARY_PATH+=":/usr/local/lib"' >> ~/.bash_profile -echo 'export CPATH+=":/usr/local/include"' >> ~/.bash_profile - -echo "" -echo "-----------------------------------------------------------------------------" -echo "| Installation Completed |" -echo "-----------------------------------------------------------------------------" -echo "Please run \`. ~/.bash_profile\` to be able to compile with gflags" diff --git a/build_tools/make_new_version.sh b/build_tools/make_new_version.sh deleted file mode 100755 index a8d524fcc36..00000000000 --- a/build_tools/make_new_version.sh +++ /dev/null @@ -1,46 +0,0 @@ -#!/bin/bash -# Copyright (c) 2013, Facebook, Inc. All rights reserved. -# This source code is licensed under the BSD-style license found in the -# LICENSE file in the root directory of this source tree. An additional grant -# of patent rights can be found in the PATENTS file in the same directory. - -set -e -if [ -z "$GIT" ] -then - GIT="git" -fi - -# Print out the colored progress info so that it can be brainlessly -# distinguished by users. -function title() { - echo -e "\033[1;32m$*\033[0m" -} - -usage="Create new RocksDB version and prepare it for the release process\n" -usage+="USAGE: ./make_new_version.sh " - -# -- Pre-check -if [[ $# < 1 ]]; then - echo -e $usage - exit 1 -fi - -ROCKSDB_VERSION=$1 - -GIT_BRANCH=`git rev-parse --abbrev-ref HEAD` -echo $GIT_BRANCH - -if [ $GIT_BRANCH != "master" ]; then - echo "Error: Current branch is '$GIT_BRANCH', Please switch to master branch." - exit 1 -fi - -title "Adding new tag for this release ..." -BRANCH="$ROCKSDB_VERSION.fb" -$GIT co -b $BRANCH - -# Setting up the proxy for remote repo access -title "Pushing new branch to remote repo ..." -git push origin --set-upstream $BRANCH - -title "Branch $BRANCH is pushed to github;" diff --git a/build_tools/make_package.sh b/build_tools/make_package.sh new file mode 100755 index 00000000000..58bac447392 --- /dev/null +++ b/build_tools/make_package.sh @@ -0,0 +1,128 @@ +#/usr/bin/env bash + +set -e + +function log() { + echo "[+] $1" +} + +function fatal() { + echo "[!] $1" + exit 1 +} + +function platform() { + local __resultvar=$1 + if [[ -f "/etc/yum.conf" ]]; then + eval $__resultvar="centos" + elif [[ -f "/etc/dpkg/dpkg.cfg" ]]; then + eval $__resultvar="ubuntu" + else + fatal "Unknwon operating system" + fi +} +platform OS + +function package() { + if [[ $OS = "ubuntu" ]]; then + if dpkg --get-selections | grep --quiet $1; then + log "$1 is already installed. skipping." + else + apt-get install $@ -y + fi + elif [[ $OS = "centos" ]]; then + if rpm -qa | grep --quiet $1; then + log "$1 is already installed. skipping." + else + yum install $@ -y + fi + fi +} + +function detect_fpm_output() { + if [[ $OS = "ubuntu" ]]; then + export FPM_OUTPUT=deb + elif [[ $OS = "centos" ]]; then + export FPM_OUTPUT=rpm + fi +} +detect_fpm_output + +function gem_install() { + if gem list | grep --quiet $1; then + log "$1 is already installed. skipping." + else + gem install $@ + fi +} + +function main() { + if [[ $# -ne 1 ]]; then + fatal "Usage: $0 " + else + log "using rocksdb version: $1" + fi + + if [[ -d /vagrant ]]; then + if [[ $OS = "ubuntu" ]]; then + package g++-4.8 + export CXX=g++-4.8 + + # the deb would depend on libgflags2, but the static lib is the only thing + # installed by make install + package libgflags-dev + + package ruby-all-dev + elif [[ $OS = "centos" ]]; then + pushd /etc/yum.repos.d + if [[ ! -f /etc/yum.repos.d/devtools-1.1.repo ]]; then + wget http://people.centos.org/tru/devtools-1.1/devtools-1.1.repo + fi + package devtoolset-1.1-gcc --enablerepo=testing-1.1-devtools-6 + package devtoolset-1.1-gcc-c++ --enablerepo=testing-1.1-devtools-6 + export CC=/opt/centos/devtoolset-1.1/root/usr/bin/gcc + export CPP=/opt/centos/devtoolset-1.1/root/usr/bin/cpp + export CXX=/opt/centos/devtoolset-1.1/root/usr/bin/c++ + export PATH=$PATH:/opt/centos/devtoolset-1.1/root/usr/bin + popd + if ! rpm -qa | grep --quiet gflags; then + rpm -i https://github.com/schuhschuh/gflags/releases/download/v2.1.0/gflags-devel-2.1.0-1.amd64.rpm + fi + + package ruby + package ruby-devel + package rubygems + package rpm-build + fi + fi + gem_install fpm + + make static_lib + make install INSTALL_PATH=package + + cd package + + LIB_DIR=lib + if [[ -z "$ARCH" ]]; then + ARCH=$(getconf LONG_BIT) + fi + if [[ ("$FPM_OUTPUT" = "rpm") && ($ARCH -eq 64) ]]; then + mv lib lib64 + LIB_DIR=lib64 + fi + + fpm \ + -s dir \ + -t $FPM_OUTPUT \ + -n rocksdb \ + -v $1 \ + --prefix /usr \ + --url http://rocksdb.org/ \ + -m rocksdb@fb.com \ + --license BSD \ + --vendor Facebook \ + --description "RocksDB is an embeddable persistent key-value store for fast storage." \ + include $LIB_DIR +} + +main $@ diff --git a/build_tools/precommit_checker.py b/build_tools/precommit_checker.py new file mode 100755 index 00000000000..0f8884dfda3 --- /dev/null +++ b/build_tools/precommit_checker.py @@ -0,0 +1,208 @@ +#!/usr/local/fbcode/gcc-4.9-glibc-2.20-fb/bin/python2.7 + +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function +from __future__ import unicode_literals +import argparse +import commands +import subprocess +import sys +import re +import os +import time + + +# +# Simple logger +# + +class Log: + + def __init__(self, filename): + self.filename = filename + self.f = open(self.filename, 'w+', 0) + + def caption(self, str): + line = "\n##### %s #####\n" % str + if self.f: + self.f.write("%s \n" % line) + else: + print(line) + + def error(self, str): + data = "\n\n##### ERROR ##### %s" % str + if self.f: + self.f.write("%s \n" % data) + else: + print(data) + + def log(self, str): + if self.f: + self.f.write("%s \n" % str) + else: + print(str) + +# +# Shell Environment +# + + +class Env(object): + + def __init__(self, logfile, tests): + self.tests = tests + self.log = Log(logfile) + + def shell(self, cmd, path=os.getcwd()): + if path: + os.chdir(path) + + self.log.log("==== shell session ===========================") + self.log.log("%s> %s" % (path, cmd)) + status = subprocess.call("cd %s; %s" % (path, cmd), shell=True, + stdout=self.log.f, stderr=self.log.f) + self.log.log("status = %s" % status) + self.log.log("============================================== \n\n") + return status + + def GetOutput(self, cmd, path=os.getcwd()): + if path: + os.chdir(path) + + self.log.log("==== shell session ===========================") + self.log.log("%s> %s" % (path, cmd)) + status, out = commands.getstatusoutput(cmd) + self.log.log("status = %s" % status) + self.log.log("out = %s" % out) + self.log.log("============================================== \n\n") + return status, out + +# +# Pre-commit checker +# + + +class PreCommitChecker(Env): + + def __init__(self, args): + Env.__init__(self, args.logfile, args.tests) + self.ignore_failure = args.ignore_failure + + # + # Get commands for a given job from the determinator file + # + def get_commands(self, test): + status, out = self.GetOutput( + "RATIO=1 build_tools/rocksdb-lego-determinator %s" % test, ".") + return status, out + + # + # Run a specific CI job + # + def run_test(self, test): + self.log.caption("Running test %s locally" % test) + + # get commands for the CI job determinator + status, cmds = self.get_commands(test) + if status != 0: + self.log.error("Error getting commands for test %s" % test) + return False + + # Parse the JSON to extract the commands to run + cmds = re.findall("'shell':'([^\']*)'", cmds) + + if len(cmds) == 0: + self.log.log("No commands found") + return False + + # Run commands + for cmd in cmds: + # Replace J=<..> with the local environment variable + if "J" in os.environ: + cmd = cmd.replace("J=1", "J=%s" % os.environ["J"]) + cmd = cmd.replace("make ", "make -j%s " % os.environ["J"]) + # Run the command + status = self.shell(cmd, ".") + if status != 0: + self.log.error("Error running command %s for test %s" + % (cmd, test)) + return False + + return True + + # + # Run specified CI jobs + # + def run_tests(self): + if not self.tests: + self.log.error("Invalid args. Please provide tests") + return False + + self.print_separator() + self.print_row("TEST", "RESULT") + self.print_separator() + + result = True + for test in self.tests: + start_time = time.time() + self.print_test(test) + result = self.run_test(test) + elapsed_min = (time.time() - start_time) / 60 + if not result: + self.log.error("Error running test %s" % test) + self.print_result("FAIL (%dm)" % elapsed_min) + if not self.ignore_failure: + return False + result = False + else: + self.print_result("PASS (%dm)" % elapsed_min) + + self.print_separator() + return result + + # + # Print a line + # + def print_separator(self): + print("".ljust(60, "-")) + + # + # Print two colums + # + def print_row(self, c0, c1): + print("%s%s" % (c0.ljust(40), c1.ljust(20))) + + def print_test(self, test): + print(test.ljust(40), end="") + sys.stdout.flush() + + def print_result(self, result): + print(result.ljust(20)) + +# +# Main +# +parser = argparse.ArgumentParser(description='RocksDB pre-commit checker.') + +# --log +parser.add_argument('--logfile', default='/tmp/precommit-check.log', + help='Log file. Default is /tmp/precommit-check.log') +# --ignore_failure +parser.add_argument('--ignore_failure', action='store_true', default=False, + help='Stop when an error occurs') +# +parser.add_argument('tests', nargs='+', + help='CI test(s) to run. e.g: unit punit asan tsan ubsan') + +args = parser.parse_args() +checker = PreCommitChecker(args) + +print("Please follow log %s" % checker.log.filename) + +if not checker.run_tests(): + print("Error running tests. Please check log file %s" + % checker.log.filename) + sys.exit(1) + +sys.exit(0) diff --git a/build_tools/regression_build_test.sh b/build_tools/regression_build_test.sh index 5e335afde21..6980633287c 100755 --- a/build_tools/regression_build_test.sh +++ b/build_tools/regression_build_test.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -e @@ -55,7 +55,6 @@ make release --open_files=55000 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 > ${STAT_FILE}.fillseq @@ -73,7 +72,6 @@ make release --open_files=55000 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --threads=8 > ${STAT_FILE}.overwrite @@ -92,7 +90,6 @@ make release --open_files=55000 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --threads=1 > /dev/null @@ -111,7 +108,6 @@ make release --open_files=55000 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --threads=16 > ${STAT_FILE}.readrandom @@ -131,7 +127,6 @@ make release --use_tailing_iterator=1 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --threads=16 > ${STAT_FILE}.readrandomtailing @@ -150,7 +145,6 @@ make release --open_files=55000 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --threads=16 > ${STAT_FILE}.readrandomsmallblockcache @@ -171,7 +165,6 @@ make release --open_files=55000 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --threads=16 > ${STAT_FILE}.readrandom_mem_sst @@ -191,7 +184,6 @@ make release --open_files=55000 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --threads=1 > /dev/null @@ -210,7 +202,6 @@ make release --open_files=55000 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --threads=16 > /dev/null @@ -230,7 +221,6 @@ make release --disable_auto_compactions=1 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --threads=16 > ${STAT_FILE}.readrandom_filluniquerandom @@ -243,7 +233,7 @@ make release --bloom_bits=10 \ --num=$((NUM / 4)) \ --reads=$((NUM / 4)) \ - --writes_per_second=1000 \ + --benchmark_write_rate_limit=$(( 110 * 1024 )) \ --write_buffer_size=100000000 \ --cache_size=6442450944 \ --cache_numshardbits=6 \ @@ -251,7 +241,6 @@ make release --open_files=55000 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --threads=16 > ${STAT_FILE}.readwhilewriting @@ -270,7 +259,6 @@ make release --open_files=55000 \ --statistics=1 \ --histogram=1 \ - --disable_data_sync=1 \ --disable_wal=1 \ --sync=0 \ --value_size=10 \ @@ -295,7 +283,6 @@ common_in_mem_args="--db=/dev/shm/rocksdb \ --disable_wal=0 \ --wal_dir=/dev/shm/rocksdb \ --sync=0 \ - --disable_data_sync=1 \ --verify_checksum=1 \ --delete_obsolete_files_period_micros=314572800 \ --max_grandparent_overlap_factor=10 \ @@ -329,7 +316,7 @@ common_in_mem_args="--db=/dev/shm/rocksdb \ --use_existing_db=1 \ --duration=600 \ --threads=32 \ - --writes_per_second=81920 > ${STAT_FILE}.readwhilewriting_in_ram + --benchmark_write_rate_limit=9502720 > ${STAT_FILE}.readwhilewriting_in_ram # Seekrandomwhilewriting ./db_bench \ @@ -342,8 +329,38 @@ common_in_mem_args="--db=/dev/shm/rocksdb \ --use_tailing_iterator=1 \ --duration=600 \ --threads=32 \ - --writes_per_second=81920 > ${STAT_FILE}.seekwhilewriting_in_ram + --benchmark_write_rate_limit=9502720 > ${STAT_FILE}.seekwhilewriting_in_ram +# measure fillseq with bunch of column families +./db_bench \ + --benchmarks=fillseq \ + --num_column_families=500 \ + --write_buffer_size=1048576 \ + --db=$DATA_DIR \ + --use_existing_db=0 \ + --num=$NUM \ + --writes=$NUM \ + --open_files=55000 \ + --statistics=1 \ + --histogram=1 \ + --disable_wal=1 \ + --sync=0 > ${STAT_FILE}.fillseq_lots_column_families + +# measure overwrite performance with bunch of column families +./db_bench \ + --benchmarks=overwrite \ + --num_column_families=500 \ + --write_buffer_size=1048576 \ + --db=$DATA_DIR \ + --use_existing_db=1 \ + --num=$NUM \ + --writes=$((NUM / 10)) \ + --open_files=55000 \ + --statistics=1 \ + --histogram=1 \ + --disable_wal=1 \ + --sync=0 \ + --threads=8 > ${STAT_FILE}.overwrite_lots_column_families # send data to ods function send_to_ods { @@ -392,3 +409,5 @@ send_benchmark_to_ods readrandom memtablereadrandom $STAT_FILE.memtablefillreadr send_benchmark_to_ods readwhilewriting readwhilewriting $STAT_FILE.readwhilewriting send_benchmark_to_ods readwhilewriting readwhilewriting_in_ram ${STAT_FILE}.readwhilewriting_in_ram send_benchmark_to_ods seekrandomwhilewriting seekwhilewriting_in_ram ${STAT_FILE}.seekwhilewriting_in_ram +send_benchmark_to_ods fillseq fillseq_lots_column_families ${STAT_FILE}.fillseq_lots_column_families +send_benchmark_to_ods overwrite overwrite_lots_column_families ${STAT_FILE}.overwrite_lots_column_families diff --git a/build_tools/rocksdb-lego-determinator b/build_tools/rocksdb-lego-determinator new file mode 100755 index 00000000000..6e8ae9cd733 --- /dev/null +++ b/build_tools/rocksdb-lego-determinator @@ -0,0 +1,782 @@ +#!/usr/bin/env bash +# This script is executed by Sandcastle +# to determine next steps to run + +# Usage: +# EMAIL= ONCALL= TRIGGER= SUBSCRIBER= rocks_ci.py +# +# Input Value +# ------------------------------------------------------------------------- +# EMAIL Email address to report on trigger conditions +# ONCALL Email address to raise a task on failure +# TRIGGER Trigger conditions for email. Valid values are fail, warn, all +# SUBSCRIBER Email addresss to add as subscriber for task +# + +# +# Report configuration +# +REPORT_EMAIL= +if [ ! -z $EMAIL ]; then + if [ -z $TRIGGER ]; then + TRIGGER="fail" + fi + + REPORT_EMAIL=" + { + 'type':'email', + 'triggers': [ '$TRIGGER' ], + 'emails':['$EMAIL'] + }," +fi + +CREATE_TASK= +if [ ! -z $ONCALL ]; then + CREATE_TASK=" + { + 'type':'task', + 'triggers':[ 'fail' ], + 'priority':0, + 'subscribers':[ '$SUBSCRIBER' ], + 'tags':[ 'rocksdb', 'ci' ], + }," +fi + +# For now, create the tasks using only the dedicated task creation tool. +CREATE_TASK= + +REPORT= +if [[ ! -z $REPORT_EMAIL || ! -z $CREATE_TASK ]]; then + REPORT="'report': [ + $REPORT_EMAIL + $CREATE_TASK + ]" +fi + +# +# Helper variables +# +CLEANUP_ENV=" +{ + 'name':'Cleanup environment', + 'shell':'rm -rf /dev/shm/rocksdb && mkdir /dev/shm/rocksdb && (chmod +t /dev/shm || true) && make clean', + 'user':'root' +}" + +# We will eventually set the RATIO to 1, but we want do this +# in steps. RATIO=$(nproc) will make it work as J=1 +if [ -z $RATIO ]; then + RATIO=$(nproc) +fi + +if [ -z $PARALLEL_J ]; then + PARALLEL_J="J=$(expr $(nproc) / ${RATIO})" +fi + +if [ -z $PARALLEL_j ]; then + PARALLEL_j="-j$(expr $(nproc) / ${RATIO})" +fi + +PARALLELISM="$PARALLEL_J $PARALLEL_j" + +DEBUG="OPT=-g" +SHM="TEST_TMPDIR=/dev/shm/rocksdb" +NON_SHM="TMPD=/tmp/rocksdb_test_tmp" +GCC_481="ROCKSDB_FBCODE_BUILD_WITH_481=1" +ASAN="COMPILE_WITH_ASAN=1" +CLANG="USE_CLANG=1" +LITE="OPT=\"-DROCKSDB_LITE -g\"" +TSAN="COMPILE_WITH_TSAN=1" +UBSAN="COMPILE_WITH_UBSAN=1" +DISABLE_JEMALLOC="DISABLE_JEMALLOC=1" +HTTP_PROXY="https_proxy=http://fwdproxy.29.prn1:8080 http_proxy=http://fwdproxy.29.prn1:8080 ftp_proxy=http://fwdproxy.29.prn1:8080" +SETUP_JAVA_ENV="export $HTTP_PROXY; export JAVA_HOME=/usr/local/jdk-8u60-64/; export PATH=\$JAVA_HOME/bin:\$PATH" +PARSER="'parser':'python build_tools/error_filter.py $1'" + +CONTRUN_NAME="ROCKSDB_CONTRUN_NAME" + +# This code is getting called under various scenarios. What we care about is to +# understand when it's called from nightly contruns because in that case we'll +# create tasks for any failures. To follow the existing pattern, we'll check +# the value of $ONCALL. If it's a diff then just call `false` to make sure +# that errors will be properly propagated to the caller. +if [ ! -z $ONCALL ]; then + TASK_CREATION_TOOL="/usr/local/bin/mysql_mtr_filter --rocksdb --oncall $ONCALL" +else + TASK_CREATION_TOOL="false" +fi + +ARTIFACTS=" 'artifacts': [ + { + 'name':'database', + 'paths':[ '/dev/shm/rocksdb' ], + } +]" + +# +# A mechanism to disable tests temporarily +# +DISABLE_COMMANDS="[ + { + 'name':'Disable test', + 'oncall':'$ONCALL', + 'steps': [ + { + 'name':'Job disabled. Please contact test owner', + 'shell':'exit 1', + 'user':'root' + }, + ], + } +]" + +# +# RocksDB unit test +# +UNIT_TEST_COMMANDS="[ + { + 'name':'Rocksdb Unit Test', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build and test RocksDB debug version', + 'shell':'$SHM $DEBUG make $PARALLELISM check || $CONTRUN_NAME=check $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB unit test not under /dev/shm +# +UNIT_TEST_NON_SHM_COMMANDS="[ + { + 'name':'Rocksdb Unit Test', + 'oncall':'$ONCALL', + 'timeout': 86400, + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build and test RocksDB debug version', + 'timeout': 86400, + 'shell':'$NON_SHM $DEBUG make $PARALLELISM check || $CONTRUN_NAME=non_shm_check $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB release build and unit tests +# +RELEASE_BUILD_COMMANDS="[ + { + 'name':'Rocksdb Release Build', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build RocksDB release', + 'shell':'make $PARALLEL_j release || $CONTRUN_NAME=release $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB unit test on gcc-4.8.1 +# +UNIT_TEST_COMMANDS_481="[ + { + 'name':'Rocksdb Unit Test on GCC 4.8.1', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build and test RocksDB debug version', + 'shell':'$SHM $GCC_481 $DEBUG make $PARALLELISM check || $CONTRUN_NAME=unit_gcc_481_check $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB release build and unit tests +# +RELEASE_BUILD_COMMANDS_481="[ + { + 'name':'Rocksdb Release on GCC 4.8.1', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build RocksDB release on GCC 4.8.1', + 'shell':'$GCC_481 make $PARALLEL_j release || $CONTRUN_NAME=release_gcc481 $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB unit test with CLANG +# +CLANG_UNIT_TEST_COMMANDS="[ + { + 'name':'Rocksdb Unit Test', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build and test RocksDB debug', + 'shell':'$CLANG $SHM $DEBUG make $PARALLELISM check || $CONTRUN_NAME=clang_check $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB release build with CLANG +# +CLANG_RELEASE_BUILD_COMMANDS="[ + { + 'name':'Rocksdb CLANG Release Build', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build RocksDB release', + 'shell':'$CLANG make $PARALLEL_j release|| $CONTRUN_NAME=clang_release $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB analyze +# +CLANG_ANALYZE_COMMANDS="[ + { + 'name':'Rocksdb analyze', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'RocksDB build and analyze', + 'shell':'$CLANG $SHM $DEBUG make $PARALLEL_j analyze || $CONTRUN_NAME=clang_analyze $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB code coverage +# +CODE_COV_COMMANDS="[ + { + 'name':'Rocksdb Unit Test Code Coverage', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build, test and collect code coverage info', + 'shell':'$SHM $DEBUG make $PARALLELISM coverage || $CONTRUN_NAME=coverage $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB unity +# +UNITY_COMMANDS="[ + { + 'name':'Rocksdb Unity', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build, test unity test', + 'shell':'$SHM $DEBUG V=1 make J=1 unity_test || $CONTRUN_NAME=unity_test $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# Build RocksDB lite +# +LITE_BUILD_COMMANDS="[ + { + 'name':'Rocksdb Lite build', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build RocksDB debug version', + 'shell':'$LITE make J=1 all check || $CONTRUN_NAME=lite $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB stress/crash test +# +STRESS_CRASH_TEST_COMMANDS="[ + { + 'name':'Rocksdb Stress/Crash Test', + 'oncall':'$ONCALL', + 'timeout': 86400, + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build and run RocksDB debug stress tests', + 'shell':'$SHM $DEBUG make J=1 db_stress || $CONTRUN_NAME=db_stress $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + { + 'name':'Build and run RocksDB debug crash tests', + 'timeout': 86400, + 'shell':'$SHM $DEBUG make J=1 crash_test || $CONTRUN_NAME=crash_test $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + } + ], + $ARTIFACTS, + $REPORT + } +]" + +# RocksDB write stress test. +# We run on disk device on purpose (i.e. no $SHM) +# because we want to add some randomness to fsync commands +WRITE_STRESS_COMMANDS="[ + { + 'name':'Rocksdb Write Stress Test', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build and run RocksDB write stress tests', + 'shell':'make write_stress && python tools/write_stress_runner.py --runtime_sec=3600 --db=/tmp/rocksdb_write_stress || $CONTRUN_NAME=write_stress $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + } + ], + 'artifacts': [{'name': 'database', 'paths': ['/tmp/rocksdb_write_stress']}], + $REPORT + } +]" + + +# +# RocksDB test under address sanitizer +# +ASAN_TEST_COMMANDS="[ + { + 'name':'Rocksdb Unit Test under ASAN', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Test RocksDB debug under ASAN', +'shell':'set -o pipefail && ($SHM $ASAN $DEBUG make $PARALLELISM asan_check || $CONTRUN_NAME=asan_check $TASK_CREATION_TOOL) |& /usr/facebook/ops/scripts/asan_symbolize.py -d', + 'user':'root', + $PARSER + } + ], + $REPORT + } +]" + +# +# RocksDB crash testing under address sanitizer +# +ASAN_CRASH_TEST_COMMANDS="[ + { + 'name':'Rocksdb crash test under ASAN', + 'oncall':'$ONCALL', + 'timeout': 86400, + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build and run RocksDB debug asan_crash_test', + 'timeout': 86400, + 'shell':'$SHM $DEBUG make J=1 asan_crash_test || $CONTRUN_NAME=asan_crash_test $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB test under undefined behavior sanitizer +# +UBSAN_TEST_COMMANDS="[ + { + 'name':'Rocksdb Unit Test under UBSAN', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Test RocksDB debug under UBSAN', + 'shell':'set -o pipefail && $SHM $UBSAN $DEBUG make $PARALLELISM ubsan_check || $CONTRUN_NAME=ubsan_check $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + } + ], + $REPORT + } +]" + +# +# RocksDB crash testing under udnefined behavior sanitizer +# +UBSAN_CRASH_TEST_COMMANDS="[ + { + 'name':'Rocksdb crash test under UBSAN', + 'oncall':'$ONCALL', + 'timeout': 86400, + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build and run RocksDB debug ubsan_crash_test', + 'timeout': 86400, + 'shell':'$SHM $DEBUG make J=1 ubsan_crash_test || $CONTRUN_NAME=ubsan_crash_test $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB unit test under valgrind +# +VALGRIND_TEST_COMMANDS="[ + { + 'name':'Rocksdb Unit Test under valgrind', + 'oncall':'$ONCALL', + 'timeout': 86400, + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Run RocksDB debug unit tests', + 'timeout': 86400, + 'shell':'$SHM $DEBUG make $PARALLELISM valgrind_test || $CONTRUN_NAME=valgrind_check $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB test under TSAN +# +TSAN_UNIT_TEST_COMMANDS="[ + { + 'name':'Rocksdb Unit Test under TSAN', + 'oncall':'$ONCALL', + 'timeout': 86400, + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Run RocksDB debug unit test', + 'timeout': 86400, + 'shell':'set -o pipefail && $SHM $DEBUG $TSAN make $PARALLELISM check || $CONTRUN_NAME=tsan_check $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB crash test under TSAN +# +TSAN_CRASH_TEST_COMMANDS="[ + { + 'name':'Rocksdb Crash Test under TSAN', + 'oncall':'$ONCALL', + 'timeout': 86400, + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Compile and run', + 'timeout': 86400, + 'shell':'set -o pipefail && $SHM $DEBUG $TSAN CRASH_TEST_KILL_ODD=1887 CRASH_TEST_EXT_ARGS=--log2_keys_per_lock=22 make J=1 crash_test || $CONTRUN_NAME=tsan_crash_test $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB format compatible +# + +run_format_compatible() +{ + export TEST_TMPDIR=/dev/shm/rocksdb + rm -rf /dev/shm/rocksdb + mkdir /dev/shm/rocksdb + + tools/check_format_compatible.sh +} + +FORMAT_COMPATIBLE_COMMANDS="[ + { + 'name':'Rocksdb Format Compatible tests', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Run RocksDB debug unit test', + 'shell':'build_tools/rocksdb-lego-determinator run_format_compatible || $CONTRUN_NAME=run_format_compatible $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB no compression +# +run_no_compression() +{ + export TEST_TMPDIR=/dev/shm/rocksdb + rm -rf /dev/shm/rocksdb + mkdir /dev/shm/rocksdb + make clean + cat build_tools/fbcode_config.sh | grep -iv dzlib | grep -iv dlz4 | grep -iv dsnappy | grep -iv dbzip2 > .tmp.fbcode_config.sh + mv .tmp.fbcode_config.sh build_tools/fbcode_config.sh + cat Makefile | grep -v tools/ldb_test.py > .tmp.Makefile + mv .tmp.Makefile Makefile + make $DEBUG J=1 check +} + +NO_COMPRESSION_COMMANDS="[ + { + 'name':'Rocksdb No Compression tests', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Run RocksDB debug unit test', + 'shell':'build_tools/rocksdb-lego-determinator run_no_compression || $CONTRUN_NAME=run_no_compression $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB regression +# +run_regression() +{ + time -v bash -vx ./build_tools/regression_build_test.sh $(mktemp -d $WORKSPACE/leveldb.XXXX) $(mktemp leveldb_test_stats.XXXX) + + # ======= report size to ODS ======== + + # parameters: $1 -- key, $2 -- value + function send_size_to_ods { + curl -s "https://www.intern.facebook.com/intern/agent/ods_set.php?entity=rocksdb_build&key=rocksdb.build_size.$1&value=$2" \ + --connect-timeout 60 + } + + # === normal build === + make clean + make -j$(nproc) static_lib + send_size_to_ods static_lib $(stat --printf="%s" librocksdb.a) + strip librocksdb.a + send_size_to_ods static_lib_stripped $(stat --printf="%s" librocksdb.a) + + make -j$(nproc) shared_lib + send_size_to_ods shared_lib $(stat --printf="%s" `readlink -f librocksdb.so`) + strip `readlink -f librocksdb.so` + send_size_to_ods shared_lib_stripped $(stat --printf="%s" `readlink -f librocksdb.so`) + + # === lite build === + make clean + OPT=-DROCKSDB_LITE make -j$(nproc) static_lib + send_size_to_ods static_lib_lite $(stat --printf="%s" librocksdb.a) + strip librocksdb.a + send_size_to_ods static_lib_lite_stripped $(stat --printf="%s" librocksdb.a) + + OPT=-DROCKSDB_LITE make -j$(nproc) shared_lib + send_size_to_ods shared_lib_lite $(stat --printf="%s" `readlink -f librocksdb.so`) + strip `readlink -f librocksdb.so` + send_size_to_ods shared_lib_lite_stripped $(stat --printf="%s" `readlink -f librocksdb.so`) +} + +REGRESSION_COMMANDS="[ + { + 'name':'Rocksdb regression commands', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Make and run script', + 'shell':'build_tools/rocksdb-lego-determinator run_regression || $CONTRUN_NAME=run_regression $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + +# +# RocksDB Java build +# +JAVA_BUILD_TEST_COMMANDS="[ + { + 'name':'Rocksdb Java Build', + 'oncall':'$ONCALL', + 'steps': [ + $CLEANUP_ENV, + { + 'name':'Build RocksDB for Java', + 'shell':'$SETUP_JAVA_ENV; $SHM make rocksdbjava || $CONTRUN_NAME=rocksdbjava $TASK_CREATION_TOOL', + 'user':'root', + $PARSER + }, + ], + $REPORT + } +]" + + +case $1 in + unit) + echo $UNIT_TEST_COMMANDS + ;; + unit_non_shm) + echo $UNIT_TEST_NON_SHM_COMMANDS + ;; + release) + echo $RELEASE_BUILD_COMMANDS + ;; + unit_481) + echo $UNIT_TEST_COMMANDS_481 + ;; + release_481) + echo $RELEASE_BUILD_COMMANDS_481 + ;; + clang_unit) + echo $CLANG_UNIT_TEST_COMMANDS + ;; + clang_release) + echo $CLANG_RELEASE_BUILD_COMMANDS + ;; + clang_analyze) + echo $CLANG_ANALYZE_COMMANDS + ;; + code_cov) + echo $CODE_COV_COMMANDS + ;; + unity) + echo $UNITY_COMMANDS + ;; + lite) + echo $LITE_BUILD_COMMANDS + ;; + stress_crash) + echo $STRESS_CRASH_TEST_COMMANDS + ;; + write_stress) + echo $WRITE_STRESS_COMMANDS + ;; + asan) + echo $ASAN_TEST_COMMANDS + ;; + asan_crash) + echo $ASAN_CRASH_TEST_COMMANDS + ;; + ubsan) + echo $UBSAN_TEST_COMMANDS + ;; + ubsan_crash) + echo $UBSAN_CRASH_TEST_COMMANDS + ;; + valgrind) + echo $VALGRIND_TEST_COMMANDS + ;; + tsan) + echo $TSAN_UNIT_TEST_COMMANDS + ;; + tsan_crash) + echo $TSAN_CRASH_TEST_COMMANDS + ;; + format_compatible) + echo $FORMAT_COMPATIBLE_COMMANDS + ;; + run_format_compatible) + run_format_compatible + ;; + no_compression) + echo $NO_COMPRESSION_COMMANDS + ;; + run_no_compression) + run_no_compression + ;; + regression) + echo $REGRESSION_COMMANDS + ;; + run_regression) + run_regression + ;; + java_build) + echo $JAVA_BUILD_TEST_COMMANDS + ;; + *) + echo "Invalid determinator command" + ;; +esac diff --git a/build_tools/run_ci_db_test.ps1 b/build_tools/run_ci_db_test.ps1 new file mode 100644 index 00000000000..c8167ed9571 --- /dev/null +++ b/build_tools/run_ci_db_test.ps1 @@ -0,0 +1,456 @@ +# This script enables you running RocksDB tests by running +# All the tests concurrently and utilizing all the cores +Param( + [switch]$EnableJE = $false, # Look for and use _je executable, append _je to listed exclusions + [switch]$RunAll = $false, # Will attempt discover all *_test[_je].exe binaries and run all + # of them as Google suites. I.e. It will run test cases concurrently + # except those mentioned as $Run, those will run as individual test cases + # And any execlued with $ExcludeExes or $ExcludeCases + # It will also not run any individual test cases + # excluded but $ExcludeCasese + [string]$SuiteRun = "", # Split test suites in test cases and run in parallel, not compatible with $RunAll + [string]$Run = "", # Run specified executables in parallel but do not split to test cases + [string]$ExcludeCases = "", # Exclude test cases, expects a comma separated list, no spaces + # Takes effect when $RunAll or $SuiteRun is specified. Must have full + # Test cases name including a group and a parameter if any + [string]$ExcludeExes = "", # Exclude exes from consideration, expects a comma separated list, + # no spaces. Takes effect only when $RunAll is specified + [string]$WorkFolder = "", # Direct tests to use that folder. SSD or Ram drive are better options. + # Number of async tasks that would run concurrently. Recommend a number below 64. + # However, CPU utlization really depends on the storage media. Recommend ram based disk. + # a value of 1 will run everything serially + [int]$Concurrency = 8, + [int]$Limit = -1 # -1 means do not limit for test purposes +) + +# Folders and commands must be fullpath to run assuming +# the current folder is at the root of the git enlistment +$StartDate = (Get-Date) +$StartDate + + +$DebugPreference = "Continue" + +# These tests are not google test suites and we should guard +# Against running them as suites +$RunOnly = New-Object System.Collections.Generic.HashSet[string] +$RunOnly.Add("c_test") | Out-Null +$RunOnly.Add("compact_on_deletion_collector_test") | Out-Null +$RunOnly.Add("merge_test") | Out-Null +$RunOnly.Add("stringappend_test") | Out-Null # Apparently incorrectly written +$RunOnly.Add("backupable_db_test") | Out-Null # Disabled + + +if($RunAll -and $SuiteRun -ne "") { + Write-Error "$RunAll and $SuiteRun are not compatible" + exit 1 +} + +# If running under Appveyor assume that root +[string]$Appveyor = $Env:APPVEYOR_BUILD_FOLDER +if($Appveyor -ne "") { + $RootFolder = $Appveyor +} else { + $RootFolder = $PSScriptRoot -replace '\\build_tools', '' +} + +$LogFolder = -Join($RootFolder, "\db_logs\") +$BinariesFolder = -Join($RootFolder, "\build\Debug\") + +if($WorkFolder -eq "") { + + # If TEST_TMPDIR is set use it + [string]$var = $Env:TEST_TMPDIR + if($var -eq "") { + $WorkFolder = -Join($RootFolder, "\db_tests\") + $Env:TEST_TMPDIR = $WorkFolder + } else { + $WorkFolder = $var + } +} else { +# Override from a command line + $Env:TEST_TMPDIR = $WorkFolder +} + +Write-Output "Root: $RootFolder, WorkFolder: $WorkFolder" +Write-Output "BinariesFolder: $BinariesFolder, LogFolder: $LogFolder" + +# Create test directories in the current folder +md -Path $WorkFolder -ErrorAction Ignore | Out-Null +md -Path $LogFolder -ErrorAction Ignore | Out-Null + + +$ExcludeCasesSet = New-Object System.Collections.Generic.HashSet[string] +if($ExcludeCases -ne "") { + Write-Host "ExcludeCases: $ExcludeCases" + $l = $ExcludeCases -split ' ' + ForEach($t in $l) { + $ExcludeCasesSet.Add($t) | Out-Null + } +} + +$ExcludeExesSet = New-Object System.Collections.Generic.HashSet[string] +if($ExcludeExes -ne "") { + Write-Host "ExcludeExe: $ExcludeExes" + $l = $ExcludeExes -split ' ' + ForEach($t in $l) { + $ExcludeExesSet.Add($t) | Out-Null + } +} + + +# Extract the names of its tests by running db_test with --gtest_list_tests. +# This filter removes the "#"-introduced comments, and expands to +# fully-qualified names by changing input like this: +# +# DBTest. +# Empty +# WriteEmptyBatch +# MultiThreaded/MultiThreadedDBTest. +# MultiThreaded/0 # GetParam() = 0 +# MultiThreaded/1 # GetParam() = 1 +# +# into this: +# +# DBTest.Empty +# DBTest.WriteEmptyBatch +# MultiThreaded/MultiThreadedDBTest.MultiThreaded/0 +# MultiThreaded/MultiThreadedDBTest.MultiThreaded/1 +# +# Output into the parameter in a form TestName -> Log File Name +function ExtractTestCases([string]$GTestExe, $HashTable) { + + $Tests = @() +# Run db_test to get a list of tests and store it into $a array + &$GTestExe --gtest_list_tests | tee -Variable Tests | Out-Null + + # Current group + $Group="" + + ForEach( $l in $Tests) { + + # Leading whitespace is fine + $l = $l -replace '^\s+','' + # but no whitespace any other place + if($l -match "\s+") { + continue + } + # Trailing dot is a test group but no whitespace + elseif ( $l -match "\.$" ) { + $Group = $l + } else { + # Otherwise it is a test name, remove leading space + $test = $l + # remove trailing comment if any and create a log name + $test = $test -replace '\s+\#.*','' + $test = "$Group$test" + + if($ExcludeCasesSet.Contains($test)) { + Write-Warning "$test case is excluded" + continue + } + + $test_log = $test -replace '[\./]','_' + $test_log += ".log" + $log_path = -join ($LogFolder, $test_log) + + # Add to a hashtable + $HashTable.Add($test, $log_path); + } + } +} + +# The function removes trailing .exe siffix if any, +# creates a name for the log file +# Then adds the test name if it was not excluded into +# a HashTable in a form of test_name -> log_path +function MakeAndAdd([string]$token, $HashTable) { + + $test_name = $token -replace '.exe$', '' + $log_name = -join ($test_name, ".log") + $log_path = -join ($LogFolder, $log_name) + $HashTable.Add($test_name, $log_path) +} + +# This function takes a list of Suites to run +# Lists all the test cases in each of the suite +# and populates HashOfHashes +# Ordered by suite(exe) @{ Exe = @{ TestCase = LogName }} +function ProcessSuites($ListOfSuites, $HashOfHashes) { + + $suite_list = $ListOfSuites + # Problem: if you run --gtest_list_tests on + # a non Google Test executable then it will start executing + # and we will get nowhere + ForEach($suite in $suite_list) { + + if($RunOnly.Contains($suite)) { + Write-Warning "$suite is excluded from running as Google test suite" + continue + } + + if($EnableJE) { + $suite += "_je" + } + + $Cases = [ordered]@{} + $Cases.Clear() + $suite_exe = -Join ($BinariesFolder, $suite) + ExtractTestCases -GTestExe $suite_exe -HashTable $Cases + if($Cases.Count -gt 0) { + $HashOfHashes.Add($suite, $Cases); + } + } + + # Make logs and run + if($CasesToRun.Count -lt 1) { + Write-Error "Failed to extract tests from $SuiteRun" + exit 1 + } + +} + +# This will contain all test executables to run + +# Hash table that contains all non suite +# Test executable to run +$TestExes = [ordered]@{} + +# Check for test exe that are not +# Google Test Suites +# Since this is explicitely mentioned it is not subject +# for exclusions +if($Run -ne "") { + + $test_list = $Run -split ' ' + + ForEach($t in $test_list) { + + if($EnableJE) { + $t += "_je" + } + + MakeAndAdd -token $t -HashTable $TestExes + } + + if($TestExes.Count -lt 1) { + Write-Error "Failed to extract tests from $Run" + exit 1 + } +} + +# Ordered by exe @{ Exe = @{ TestCase = LogName }} +$CasesToRun = [ordered]@{} + +if($SuiteRun -ne "") { + $suite_list = $SuiteRun -split ' ' + ProcessSuites -ListOfSuites $suite_list -HashOfHashes $CasesToRun +} + +if($RunAll) { +# Discover all the test binaries + if($EnableJE) { + $pattern = "*_test_je.exe" + } else { + $pattern = "*_test.exe" + } + + + $search_path = -join ($BinariesFolder, $pattern) + Write-Host "Binaries Search Path: $search_path" + + $ListOfExe = @() + dir -Path $search_path | ForEach-Object { + $ListOfExe += ($_.Name) + } + + # Exclude those in RunOnly from running as suites + $ListOfSuites = @() + ForEach($e in $ListOfExe) { + + $e = $e -replace '.exe$', '' + $bare_name = $e -replace '_je$', '' + + if($ExcludeExesSet.Contains($bare_name)) { + Write-Warning "Test $e is excluded" + continue + } + + if($RunOnly.Contains($bare_name)) { + MakeAndAdd -token $e -HashTable $TestExes + } else { + $ListOfSuites += $bare_name + } + } + + ProcessSuites -ListOfSuites $ListOfSuites -HashOfHashes $CasesToRun +} + + +Write-Host "Attempting to start: $NumTestsToStart tests" + +# Invoke a test with a filter and redirect all output +$InvokeTestCase = { + param($exe, $test, $log); + &$exe --gtest_filter=$test > $log 2>&1 +} + +# Invoke all tests and redirect output +$InvokeTestAsync = { + param($exe, $log) + &$exe > $log 2>&1 +} + +# Hash that contains tests to rerun if any failed +# Those tests will be rerun sequentially +# $Rerun = [ordered]@{} +# Test limiting factor here +[int]$count = 0 +# Overall status +[bool]$success = $true; + +function RunJobs($Suites, $TestCmds, [int]$ConcurrencyVal) +{ + # Array to wait for any of the running jobs + $jobs = @() + # Hash JobToLog + $JobToLog = @{} + + # Wait for all to finish and get the results + while(($JobToLog.Count -gt 0) -or + ($TestCmds.Count -gt 0) -or + ($Suites.Count -gt 0)) { + + # Make sure we have maximum concurrent jobs running if anything + # and the $Limit either not set or allows to proceed + while(($JobToLog.Count -lt $ConcurrencyVal) -and + ((($TestCmds.Count -gt 0) -or ($Suites.Count -gt 0)) -and + (($Limit -lt 0) -or ($count -lt $Limit)))) { + + # We always favore suites to run if available + [string]$exe_name = "" + [string]$log_path = "" + $Cases = @{} + + if($Suites.Count -gt 0) { + # Will the first one + ForEach($e in $Suites.Keys) { + $exe_name = $e + $Cases = $Suites[$e] + break + } + [string]$test_case = "" + [string]$log_path = "" + ForEach($c in $Cases.Keys) { + $test_case = $c + $log_path = $Cases[$c] + break + } + + Write-Host "Starting $exe_name::$test_case" + [string]$Exe = -Join ($BinariesFolder, $exe_name) + $job = Start-Job -Name "$exe_name::$test_case" -ArgumentList @($Exe,$test_case,$log_path) -ScriptBlock $InvokeTestCase + $JobToLog.Add($job, $log_path) + + $Cases.Remove($test_case) + if($Cases.Count -lt 1) { + $Suites.Remove($exe_name) + } + + } elseif ($TestCmds.Count -gt 0) { + + ForEach($e in $TestCmds.Keys) { + $exe_name = $e + $log_path = $TestCmds[$e] + break + } + + [string]$Exe = -Join ($BinariesFolder, $exe_name) + $job = Start-Job -Name $exe_name -ScriptBlock $InvokeTestAsync -ArgumentList @($Exe,$log_path) + $JobToLog.Add($job, $log_path) + + $TestCmds.Remove($exe_name) + + } else { + Write-Error "In the job loop but nothing to run" + exit 1 + } + + ++$count + } # End of Job starting loop + + if($JobToLog.Count -lt 1) { + break + } + + $jobs = @() + foreach($k in $JobToLog.Keys) { $jobs += $k } + + $completed = Wait-Job -Job $jobs -Any + $log = $JobToLog[$completed] + $JobToLog.Remove($completed) + + $message = -join @($completed.Name, " State: ", ($completed.State)) + + $log_content = @(Get-Content $log) + + if($completed.State -ne "Completed") { + $success = $false + Write-Warning $message + $log_content | Write-Warning + } else { + # Scan the log. If we find PASSED and no occurrence of FAILED + # then it is a success + [bool]$pass_found = $false + ForEach($l in $log_content) { + + if(($l -match "^\[\s+FAILED") -or + ($l -match "Assertion failed:")) { + $pass_found = $false + break + } + + if(($l -match "^\[\s+PASSED") -or + ($l -match " : PASSED$") -or + ($l -match "^PASS$") -or # Special c_test case + ($l -match "Passed all tests!") ) { + $pass_found = $true + } + } + + if(!$pass_found) { + $success = $false; + Write-Warning $message + $log_content | Write-Warning + } else { + Write-Host $message + } + } + + # Remove cached job info from the system + # Should be no output + Receive-Job -Job $completed | Out-Null + } +} + +RunJobs -Suites $CasesToRun -TestCmds $TestExes -ConcurrencyVal $Concurrency + +$EndDate = (Get-Date) + +New-TimeSpan -Start $StartDate -End $EndDate | + ForEach-Object { + "Elapsed time: {0:g}" -f $_ + } + + +if(!$success) { +# This does not succeed killing off jobs quick +# So we simply exit +# Remove-Job -Job $jobs -Force +# indicate failure using this exit code + exit 1 + } + + exit 0 + + diff --git a/build_tools/unity b/build_tools/unity deleted file mode 100755 index 477b8f7fb04..00000000000 --- a/build_tools/unity +++ /dev/null @@ -1,78 +0,0 @@ -#!/bin/sh -# -# Create the unity file -# - -OUTPUT=$1 -if test -z "$OUTPUT"; then - echo "usage: $0 " >&2 - exit 1 -fi - -# Delete existing file, if it exists -rm -f "$OUTPUT" -touch "$OUTPUT" - -# Detect OS -if test -z "$TARGET_OS"; then - TARGET_OS=`uname -s` -fi - -# generic port files (working on all platform by #ifdef) go directly in /port -GENERIC_PORT_FILES=`cd "$ROCKSDB_ROOT"; find port -name '*.cc' | tr "\n" " "` - -# On GCC, we pick libc's memcmp over GCC's memcmp via -fno-builtin-memcmp -case "$TARGET_OS" in - Darwin) - # PORT_FILES=port/darwin/darwin_specific.cc - ;; - IOS) - ;; - Linux) - # PORT_FILES=port/linux/linux_specific.cc - ;; - SunOS) - # PORT_FILES=port/sunos/sunos_specific.cc - ;; - FreeBSD) - # PORT_FILES=port/freebsd/freebsd_specific.cc - ;; - NetBSD) - # PORT_FILES=port/netbsd/netbsd_specific.cc - ;; - OpenBSD) - # PORT_FILES=port/openbsd/openbsd_specific.cc - ;; - DragonFly) - # PORT_FILES=port/dragonfly/dragonfly_specific.cc - ;; - OS_ANDROID_CROSSCOMPILE) - # PORT_FILES=port/android/android.cc - ;; - *) - echo "Unknown platform!" >&2 - exit 1 -esac - -# We want to make a list of all cc files within util, db, table, and helpers -# except for the test and benchmark files. By default, find will output a list -# of all files matching either rule, so we need to append -print to make the -# prune take effect. -DIRS="util db table utilities" - -set -f # temporarily disable globbing so that our patterns arent expanded -PRUNE_TEST="-name *test*.cc -prune" -PRUNE_BENCH="-name *bench*.cc -prune" -PORTABLE_FILES=`cd "$ROCKSDB_ROOT"; find $DIRS $PRUNE_TEST -o $PRUNE_BENCH -o -name '*.cc' -print | sort` -PORTABLE_CPP=`cd "$ROCKSDB_ROOT"; find $DIRS $PRUNE_TEST -o $PRUNE_BENCH -o -name '*.cpp' -print | sort` -set +f # re-enable globbing - -# The sources consist of the portable files, plus the platform-specific port -# file. -for SOURCE_FILE in $PORTABLE_FILES $GENERIC_PORT_FILES $PORT_FILES $PORTABLE_CPP -do - echo "#include <$SOURCE_FILE>" >> "$OUTPUT" -done - -echo "int main(int argc, char** argv){ return 0; }" >> "$OUTPUT" - diff --git a/build_tools/update_dependencies.sh b/build_tools/update_dependencies.sh new file mode 100755 index 00000000000..c7b99326463 --- /dev/null +++ b/build_tools/update_dependencies.sh @@ -0,0 +1,131 @@ +#!/bin/sh +# +# Update dependencies.sh file with the latest avaliable versions + +BASEDIR=$(dirname $0) +OUTPUT="" + +function log_variable() +{ + echo "$1=${!1}" >> "$OUTPUT" +} + + +TP2_LATEST="/mnt/vol/engshare/fbcode/third-party2" +## $1 => lib name +## $2 => lib version (if not provided, will try to pick latest) +## $3 => platform (if not provided, will try to pick latest gcc) +## +## get_lib_base will set a variable named ${LIB_NAME}_BASE to the lib location +function get_lib_base() +{ + local lib_name=$1 + local lib_version=$2 + local lib_platform=$3 + + local result="$TP2_LATEST/$lib_name/" + + # Lib Version + if [ -z "$lib_version" ] || [ "$lib_version" = "LATEST" ]; then + # version is not provided, use latest + result=`ls -dr1v $result/*/ | head -n1` + else + result="$result/$lib_version/" + fi + + # Lib Platform + if [ -z "$lib_platform" ]; then + # platform is not provided, use latest gcc + result=`ls -dr1v $result/gcc-*[^fb]/ | head -n1` + else + result="$result/$lib_platform/" + fi + + result=`ls -1d $result/*/ | head -n1` + + # lib_name => LIB_NAME_BASE + local __res_var=${lib_name^^}"_BASE" + __res_var=`echo $__res_var | tr - _` + # LIB_NAME_BASE=$result + eval $__res_var=`readlink -f $result` + + log_variable $__res_var +} + +########################################################### +# 5.x dependencies # +########################################################### + +OUTPUT="$BASEDIR/dependencies.sh" + +rm -f "$OUTPUT" +touch "$OUTPUT" + +echo "Writing dependencies to $OUTPUT" + +# Compilers locations +GCC_BASE=`readlink -f $TP2_LATEST/gcc/5.x/centos6-native/*/` +CLANG_BASE=`readlink -f $TP2_LATEST/llvm-fb/stable/centos6-native/*/` + +log_variable GCC_BASE +log_variable CLANG_BASE + +# Libraries locations +get_lib_base libgcc 5.x +get_lib_base glibc 2.23 +get_lib_base snappy LATEST gcc-5-glibc-2.23 +get_lib_base zlib LATEST +get_lib_base bzip2 LATEST +get_lib_base lz4 LATEST +get_lib_base zstd LATEST +get_lib_base gflags LATEST +get_lib_base jemalloc LATEST +get_lib_base numa LATEST +get_lib_base libunwind LATEST +get_lib_base tbb 4.0_update2 gcc-5-glibc-2.23 + +get_lib_base kernel-headers LATEST +get_lib_base binutils LATEST centos6-native +get_lib_base valgrind 3.10.0 gcc-5-glibc-2.23 +get_lib_base lua 5.2.3 gcc-5-glibc-2.23 + +git diff $OUTPUT + +########################################################### +# 4.8.1 dependencies # +########################################################### + +OUTPUT="$BASEDIR/dependencies_4.8.1.sh" + +rm -f "$OUTPUT" +touch "$OUTPUT" + +echo "Writing 4.8.1 dependencies to $OUTPUT" + +# Compilers locations +GCC_BASE=`readlink -f $TP2_LATEST/gcc/4.8.1/centos6-native/*/` +CLANG_BASE=`readlink -f $TP2_LATEST/llvm-fb/stable/centos6-native/*/` + +log_variable GCC_BASE +log_variable CLANG_BASE + +# Libraries locations +get_lib_base libgcc 4.8.1 gcc-4.8.1-glibc-2.17 +get_lib_base glibc 2.17 gcc-4.8.1-glibc-2.17 +get_lib_base snappy LATEST gcc-4.8.1-glibc-2.17 +get_lib_base zlib LATEST gcc-4.8.1-glibc-2.17 +get_lib_base bzip2 LATEST gcc-4.8.1-glibc-2.17 +get_lib_base lz4 LATEST gcc-4.8.1-glibc-2.17 +get_lib_base zstd LATEST gcc-4.8.1-glibc-2.17 +get_lib_base gflags LATEST gcc-4.8.1-glibc-2.17 +get_lib_base jemalloc LATEST gcc-4.8.1-glibc-2.17 +get_lib_base numa LATEST gcc-4.8.1-glibc-2.17 +get_lib_base libunwind LATEST gcc-4.8.1-glibc-2.17 +get_lib_base tbb 4.0_update2 gcc-4.8.1-glibc-2.17 + +get_lib_base kernel-headers LATEST gcc-4.8.1-glibc-2.17 +get_lib_base binutils LATEST centos6-native +get_lib_base valgrind 3.8.1 gcc-4.8.1-glibc-2.17 +get_lib_base lua 5.2.3 centos6-native + +git diff $OUTPUT diff --git a/build_tools/valgrind_test.sh b/build_tools/valgrind_test.sh deleted file mode 100755 index 8c7e5213457..00000000000 --- a/build_tools/valgrind_test.sh +++ /dev/null @@ -1,15 +0,0 @@ -#!/bin/bash -#A shell script for Jenknis to run valgrind on rocksdb tests -#Returns 0 on success when there are no failed tests - -VALGRIND_DIR=build_tools/VALGRIND_LOGS -make clean -make -j$(nproc) valgrind_check -NUM_FAILED_TESTS=$((`wc -l $VALGRIND_DIR/valgrind_failed_tests | awk '{print $1}'` - 1)) -if [ $NUM_FAILED_TESTS -lt 1 ]; then - echo No tests have valgrind errors - exit 0 -else - cat $VALGRIND_DIR/valgrind_failed_tests - exit 1 -fi diff --git a/build_tools/version.sh b/build_tools/version.sh new file mode 100755 index 00000000000..f3ca98cf61e --- /dev/null +++ b/build_tools/version.sh @@ -0,0 +1,22 @@ +#!/bin/sh +if [ "$#" = "0" ]; then + echo "Usage: $0 major|minor|patch|full" + exit 1 +fi + +if [ "$1" = "major" ]; then + cat include/rocksdb/version.h | grep MAJOR | head -n1 | awk '{print $3}' +fi +if [ "$1" = "minor" ]; then + cat include/rocksdb/version.h | grep MINOR | head -n1 | awk '{print $3}' +fi +if [ "$1" = "patch" ]; then + cat include/rocksdb/version.h | grep PATCH | head -n1 | awk '{print $3}' +fi +if [ "$1" = "full" ]; then + awk '/#define ROCKSDB/ { env[$2] = $3 } + END { printf "%s.%s.%s\n", env["ROCKSDB_MAJOR"], + env["ROCKSDB_MINOR"], + env["ROCKSDB_PATCH"] }' \ + include/rocksdb/version.h +fi diff --git a/cache/cache_bench.cc b/cache/cache_bench.cc new file mode 100644 index 00000000000..16c2ced1dde --- /dev/null +++ b/cache/cache_bench.cc @@ -0,0 +1,284 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif +#ifndef GFLAGS +#include +int main() { + fprintf(stderr, "Please install gflags to run rocksdb tools\n"); + return 1; +} +#else + +#include +#include +#include +#include + +#include "rocksdb/db.h" +#include "rocksdb/cache.h" +#include "rocksdb/env.h" +#include "port/port.h" +#include "util/mutexlock.h" +#include "util/random.h" + +using GFLAGS::ParseCommandLineFlags; + +static const uint32_t KB = 1024; + +DEFINE_int32(threads, 16, "Number of concurrent threads to run."); +DEFINE_int64(cache_size, 8 * KB * KB, + "Number of bytes to use as a cache of uncompressed data."); +DEFINE_int32(num_shard_bits, 4, "shard_bits."); + +DEFINE_int64(max_key, 1 * KB * KB * KB, "Max number of key to place in cache"); +DEFINE_uint64(ops_per_thread, 1200000, "Number of operations per thread."); + +DEFINE_bool(populate_cache, false, "Populate cache before operations"); +DEFINE_int32(insert_percent, 40, + "Ratio of insert to total workload (expressed as a percentage)"); +DEFINE_int32(lookup_percent, 50, + "Ratio of lookup to total workload (expressed as a percentage)"); +DEFINE_int32(erase_percent, 10, + "Ratio of erase to total workload (expressed as a percentage)"); + +DEFINE_bool(use_clock_cache, false, ""); + +namespace rocksdb { + +class CacheBench; +namespace { +void deleter(const Slice& key, void* value) { + delete reinterpret_cast(value); +} + +// State shared by all concurrent executions of the same benchmark. +class SharedState { + public: + explicit SharedState(CacheBench* cache_bench) + : cv_(&mu_), + num_threads_(FLAGS_threads), + num_initialized_(0), + start_(false), + num_done_(0), + cache_bench_(cache_bench) { + } + + ~SharedState() {} + + port::Mutex* GetMutex() { + return &mu_; + } + + port::CondVar* GetCondVar() { + return &cv_; + } + + CacheBench* GetCacheBench() const { + return cache_bench_; + } + + void IncInitialized() { + num_initialized_++; + } + + void IncDone() { + num_done_++; + } + + bool AllInitialized() const { + return num_initialized_ >= num_threads_; + } + + bool AllDone() const { + return num_done_ >= num_threads_; + } + + void SetStart() { + start_ = true; + } + + bool Started() const { + return start_; + } + + private: + port::Mutex mu_; + port::CondVar cv_; + + const uint64_t num_threads_; + uint64_t num_initialized_; + bool start_; + uint64_t num_done_; + + CacheBench* cache_bench_; +}; + +// Per-thread state for concurrent executions of the same benchmark. +struct ThreadState { + uint32_t tid; + Random rnd; + SharedState* shared; + + ThreadState(uint32_t index, SharedState* _shared) + : tid(index), rnd(1000 + index), shared(_shared) {} +}; +} // namespace + +class CacheBench { + public: + CacheBench() : num_threads_(FLAGS_threads) { + if (FLAGS_use_clock_cache) { + cache_ = NewClockCache(FLAGS_cache_size, FLAGS_num_shard_bits); + if (!cache_) { + fprintf(stderr, "Clock cache not supported.\n"); + exit(1); + } + } else { + cache_ = NewLRUCache(FLAGS_cache_size, FLAGS_num_shard_bits); + } + } + + ~CacheBench() {} + + void PopulateCache() { + Random rnd(1); + for (int64_t i = 0; i < FLAGS_cache_size; i++) { + uint64_t rand_key = rnd.Next() % FLAGS_max_key; + // Cast uint64* to be char*, data would be copied to cache + Slice key(reinterpret_cast(&rand_key), 8); + // do insert + cache_->Insert(key, new char[10], 1, &deleter); + } + } + + bool Run() { + rocksdb::Env* env = rocksdb::Env::Default(); + + PrintEnv(); + SharedState shared(this); + std::vector threads(num_threads_); + for (uint32_t i = 0; i < num_threads_; i++) { + threads[i] = new ThreadState(i, &shared); + env->StartThread(ThreadBody, threads[i]); + } + { + MutexLock l(shared.GetMutex()); + while (!shared.AllInitialized()) { + shared.GetCondVar()->Wait(); + } + // Record start time + uint64_t start_time = env->NowMicros(); + + // Start all threads + shared.SetStart(); + shared.GetCondVar()->SignalAll(); + + // Wait threads to complete + while (!shared.AllDone()) { + shared.GetCondVar()->Wait(); + } + + // Record end time + uint64_t end_time = env->NowMicros(); + double elapsed = static_cast(end_time - start_time) * 1e-6; + uint32_t qps = static_cast( + static_cast(FLAGS_threads * FLAGS_ops_per_thread) / elapsed); + fprintf(stdout, "Complete in %.3f s; QPS = %u\n", elapsed, qps); + } + return true; + } + + private: + std::shared_ptr cache_; + uint32_t num_threads_; + + static void ThreadBody(void* v) { + ThreadState* thread = reinterpret_cast(v); + SharedState* shared = thread->shared; + + { + MutexLock l(shared->GetMutex()); + shared->IncInitialized(); + if (shared->AllInitialized()) { + shared->GetCondVar()->SignalAll(); + } + while (!shared->Started()) { + shared->GetCondVar()->Wait(); + } + } + thread->shared->GetCacheBench()->OperateCache(thread); + + { + MutexLock l(shared->GetMutex()); + shared->IncDone(); + if (shared->AllDone()) { + shared->GetCondVar()->SignalAll(); + } + } + } + + void OperateCache(ThreadState* thread) { + for (uint64_t i = 0; i < FLAGS_ops_per_thread; i++) { + uint64_t rand_key = thread->rnd.Next() % FLAGS_max_key; + // Cast uint64* to be char*, data would be copied to cache + Slice key(reinterpret_cast(&rand_key), 8); + int32_t prob_op = thread->rnd.Uniform(100); + if (prob_op >= 0 && prob_op < FLAGS_insert_percent) { + // do insert + cache_->Insert(key, new char[10], 1, &deleter); + } else if (prob_op -= FLAGS_insert_percent && + prob_op < FLAGS_lookup_percent) { + // do lookup + auto handle = cache_->Lookup(key); + if (handle) { + cache_->Release(handle); + } + } else if (prob_op -= FLAGS_lookup_percent && + prob_op < FLAGS_erase_percent) { + // do erase + cache_->Erase(key); + } + } + } + + void PrintEnv() const { + printf("RocksDB version : %d.%d\n", kMajorVersion, kMinorVersion); + printf("Number of threads : %d\n", FLAGS_threads); + printf("Ops per thread : %" PRIu64 "\n", FLAGS_ops_per_thread); + printf("Cache size : %" PRIu64 "\n", FLAGS_cache_size); + printf("Num shard bits : %d\n", FLAGS_num_shard_bits); + printf("Max key : %" PRIu64 "\n", FLAGS_max_key); + printf("Populate cache : %d\n", FLAGS_populate_cache); + printf("Insert percentage : %d%%\n", FLAGS_insert_percent); + printf("Lookup percentage : %d%%\n", FLAGS_lookup_percent); + printf("Erase percentage : %d%%\n", FLAGS_erase_percent); + printf("----------------------------\n"); + } +}; +} // namespace rocksdb + +int main(int argc, char** argv) { + ParseCommandLineFlags(&argc, &argv, true); + + if (FLAGS_threads <= 0) { + fprintf(stderr, "threads number <= 0\n"); + exit(1); + } + + rocksdb::CacheBench bench; + if (FLAGS_populate_cache) { + bench.PopulateCache(); + } + if (bench.Run()) { + return 0; + } else { + return 1; + } +} + +#endif // GFLAGS diff --git a/cache/cache_test.cc b/cache/cache_test.cc new file mode 100644 index 00000000000..8e241226d9c --- /dev/null +++ b/cache/cache_test.cc @@ -0,0 +1,703 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "rocksdb/cache.h" + +#include +#include +#include +#include +#include +#include "cache/clock_cache.h" +#include "cache/lru_cache.h" +#include "util/coding.h" +#include "util/string_util.h" +#include "util/testharness.h" + +namespace rocksdb { + +// Conversions between numeric keys/values and the types expected by Cache. +static std::string EncodeKey(int k) { + std::string result; + PutFixed32(&result, k); + return result; +} +static int DecodeKey(const Slice& k) { + assert(k.size() == 4); + return DecodeFixed32(k.data()); +} +static void* EncodeValue(uintptr_t v) { return reinterpret_cast(v); } +static int DecodeValue(void* v) { + return static_cast(reinterpret_cast(v)); +} + +const std::string kLRU = "lru"; +const std::string kClock = "clock"; + +void dumbDeleter(const Slice& key, void* value) {} + +void eraseDeleter(const Slice& key, void* value) { + Cache* cache = reinterpret_cast(value); + cache->Erase("foo"); +} + +class CacheTest : public testing::TestWithParam { + public: + static CacheTest* current_; + + static void Deleter(const Slice& key, void* v) { + current_->deleted_keys_.push_back(DecodeKey(key)); + current_->deleted_values_.push_back(DecodeValue(v)); + } + + static const int kCacheSize = 1000; + static const int kNumShardBits = 4; + + static const int kCacheSize2 = 100; + static const int kNumShardBits2 = 2; + + std::vector deleted_keys_; + std::vector deleted_values_; + shared_ptr cache_; + shared_ptr cache2_; + + CacheTest() + : cache_(NewCache(kCacheSize, kNumShardBits, false)), + cache2_(NewCache(kCacheSize2, kNumShardBits2, false)) { + current_ = this; + } + + ~CacheTest() { + } + + std::shared_ptr NewCache(size_t capacity) { + auto type = GetParam(); + if (type == kLRU) { + return NewLRUCache(capacity); + } + if (type == kClock) { + return NewClockCache(capacity); + } + return nullptr; + } + + std::shared_ptr NewCache(size_t capacity, int num_shard_bits, + bool strict_capacity_limit) { + auto type = GetParam(); + if (type == kLRU) { + return NewLRUCache(capacity, num_shard_bits, strict_capacity_limit); + } + if (type == kClock) { + return NewClockCache(capacity, num_shard_bits, strict_capacity_limit); + } + return nullptr; + } + + int Lookup(shared_ptr cache, int key) { + Cache::Handle* handle = cache->Lookup(EncodeKey(key)); + const int r = (handle == nullptr) ? -1 : DecodeValue(cache->Value(handle)); + if (handle != nullptr) { + cache->Release(handle); + } + return r; + } + + void Insert(shared_ptr cache, int key, int value, int charge = 1) { + cache->Insert(EncodeKey(key), EncodeValue(value), charge, + &CacheTest::Deleter); + } + + void Erase(shared_ptr cache, int key) { + cache->Erase(EncodeKey(key)); + } + + + int Lookup(int key) { + return Lookup(cache_, key); + } + + void Insert(int key, int value, int charge = 1) { + Insert(cache_, key, value, charge); + } + + void Erase(int key) { + Erase(cache_, key); + } + + int Lookup2(int key) { + return Lookup(cache2_, key); + } + + void Insert2(int key, int value, int charge = 1) { + Insert(cache2_, key, value, charge); + } + + void Erase2(int key) { + Erase(cache2_, key); + } +}; +CacheTest* CacheTest::current_; + +TEST_P(CacheTest, UsageTest) { + // cache is shared_ptr and will be automatically cleaned up. + const uint64_t kCapacity = 100000; + auto cache = NewCache(kCapacity, 8, false); + + size_t usage = 0; + char value[10] = "abcdef"; + // make sure everything will be cached + for (int i = 1; i < 100; ++i) { + std::string key(i, 'a'); + auto kv_size = key.size() + 5; + cache->Insert(key, reinterpret_cast(value), kv_size, dumbDeleter); + usage += kv_size; + ASSERT_EQ(usage, cache->GetUsage()); + } + + // make sure the cache will be overloaded + for (uint64_t i = 1; i < kCapacity; ++i) { + auto key = ToString(i); + cache->Insert(key, reinterpret_cast(value), key.size() + 5, + dumbDeleter); + } + + // the usage should be close to the capacity + ASSERT_GT(kCapacity, cache->GetUsage()); + ASSERT_LT(kCapacity * 0.95, cache->GetUsage()); +} + +TEST_P(CacheTest, PinnedUsageTest) { + // cache is shared_ptr and will be automatically cleaned up. + const uint64_t kCapacity = 100000; + auto cache = NewCache(kCapacity, 8, false); + + size_t pinned_usage = 0; + char value[10] = "abcdef"; + + std::forward_list unreleased_handles; + + // Add entries. Unpin some of them after insertion. Then, pin some of them + // again. Check GetPinnedUsage(). + for (int i = 1; i < 100; ++i) { + std::string key(i, 'a'); + auto kv_size = key.size() + 5; + Cache::Handle* handle; + cache->Insert(key, reinterpret_cast(value), kv_size, dumbDeleter, + &handle); + pinned_usage += kv_size; + ASSERT_EQ(pinned_usage, cache->GetPinnedUsage()); + if (i % 2 == 0) { + cache->Release(handle); + pinned_usage -= kv_size; + ASSERT_EQ(pinned_usage, cache->GetPinnedUsage()); + } else { + unreleased_handles.push_front(handle); + } + if (i % 3 == 0) { + unreleased_handles.push_front(cache->Lookup(key)); + // If i % 2 == 0, then the entry was unpinned before Lookup, so pinned + // usage increased + if (i % 2 == 0) { + pinned_usage += kv_size; + } + ASSERT_EQ(pinned_usage, cache->GetPinnedUsage()); + } + } + + // check that overloading the cache does not change the pinned usage + for (uint64_t i = 1; i < 2 * kCapacity; ++i) { + auto key = ToString(i); + cache->Insert(key, reinterpret_cast(value), key.size() + 5, + dumbDeleter); + } + ASSERT_EQ(pinned_usage, cache->GetPinnedUsage()); + + // release handles for pinned entries to prevent memory leaks + for (auto handle : unreleased_handles) { + cache->Release(handle); + } +} + +TEST_P(CacheTest, HitAndMiss) { + ASSERT_EQ(-1, Lookup(100)); + + Insert(100, 101); + ASSERT_EQ(101, Lookup(100)); + ASSERT_EQ(-1, Lookup(200)); + ASSERT_EQ(-1, Lookup(300)); + + Insert(200, 201); + ASSERT_EQ(101, Lookup(100)); + ASSERT_EQ(201, Lookup(200)); + ASSERT_EQ(-1, Lookup(300)); + + Insert(100, 102); + ASSERT_EQ(102, Lookup(100)); + ASSERT_EQ(201, Lookup(200)); + ASSERT_EQ(-1, Lookup(300)); + + ASSERT_EQ(1U, deleted_keys_.size()); + ASSERT_EQ(100, deleted_keys_[0]); + ASSERT_EQ(101, deleted_values_[0]); +} + +TEST_P(CacheTest, InsertSameKey) { + Insert(1, 1); + Insert(1, 2); + ASSERT_EQ(2, Lookup(1)); +} + +TEST_P(CacheTest, Erase) { + Erase(200); + ASSERT_EQ(0U, deleted_keys_.size()); + + Insert(100, 101); + Insert(200, 201); + Erase(100); + ASSERT_EQ(-1, Lookup(100)); + ASSERT_EQ(201, Lookup(200)); + ASSERT_EQ(1U, deleted_keys_.size()); + ASSERT_EQ(100, deleted_keys_[0]); + ASSERT_EQ(101, deleted_values_[0]); + + Erase(100); + ASSERT_EQ(-1, Lookup(100)); + ASSERT_EQ(201, Lookup(200)); + ASSERT_EQ(1U, deleted_keys_.size()); +} + +TEST_P(CacheTest, EntriesArePinned) { + Insert(100, 101); + Cache::Handle* h1 = cache_->Lookup(EncodeKey(100)); + ASSERT_EQ(101, DecodeValue(cache_->Value(h1))); + ASSERT_EQ(1U, cache_->GetUsage()); + + Insert(100, 102); + Cache::Handle* h2 = cache_->Lookup(EncodeKey(100)); + ASSERT_EQ(102, DecodeValue(cache_->Value(h2))); + ASSERT_EQ(0U, deleted_keys_.size()); + ASSERT_EQ(2U, cache_->GetUsage()); + + cache_->Release(h1); + ASSERT_EQ(1U, deleted_keys_.size()); + ASSERT_EQ(100, deleted_keys_[0]); + ASSERT_EQ(101, deleted_values_[0]); + ASSERT_EQ(1U, cache_->GetUsage()); + + Erase(100); + ASSERT_EQ(-1, Lookup(100)); + ASSERT_EQ(1U, deleted_keys_.size()); + ASSERT_EQ(1U, cache_->GetUsage()); + + cache_->Release(h2); + ASSERT_EQ(2U, deleted_keys_.size()); + ASSERT_EQ(100, deleted_keys_[1]); + ASSERT_EQ(102, deleted_values_[1]); + ASSERT_EQ(0U, cache_->GetUsage()); +} + +TEST_P(CacheTest, EvictionPolicy) { + Insert(100, 101); + Insert(200, 201); + + // Frequently used entry must be kept around + for (int i = 0; i < kCacheSize + 100; i++) { + Insert(1000+i, 2000+i); + ASSERT_EQ(101, Lookup(100)); + } + ASSERT_EQ(101, Lookup(100)); + ASSERT_EQ(-1, Lookup(200)); +} + +TEST_P(CacheTest, ExternalRefPinsEntries) { + Insert(100, 101); + Cache::Handle* h = cache_->Lookup(EncodeKey(100)); + ASSERT_TRUE(cache_->Ref(h)); + ASSERT_EQ(101, DecodeValue(cache_->Value(h))); + ASSERT_EQ(1U, cache_->GetUsage()); + + for (int i = 0; i < 3; ++i) { + if (i > 0) { + // First release (i == 1) corresponds to Ref(), second release (i == 2) + // corresponds to Lookup(). Then, since all external refs are released, + // the below insertions should push out the cache entry. + cache_->Release(h); + } + // double cache size because the usage bit in block cache prevents 100 from + // being evicted in the first kCacheSize iterations + for (int j = 0; j < 2 * kCacheSize + 100; j++) { + Insert(1000 + j, 2000 + j); + } + if (i < 2) { + ASSERT_EQ(101, Lookup(100)); + } + } + ASSERT_EQ(-1, Lookup(100)); +} + +TEST_P(CacheTest, EvictionPolicyRef) { + Insert(100, 101); + Insert(101, 102); + Insert(102, 103); + Insert(103, 104); + Insert(200, 101); + Insert(201, 102); + Insert(202, 103); + Insert(203, 104); + Cache::Handle* h201 = cache_->Lookup(EncodeKey(200)); + Cache::Handle* h202 = cache_->Lookup(EncodeKey(201)); + Cache::Handle* h203 = cache_->Lookup(EncodeKey(202)); + Cache::Handle* h204 = cache_->Lookup(EncodeKey(203)); + Insert(300, 101); + Insert(301, 102); + Insert(302, 103); + Insert(303, 104); + + // Insert entries much more than Cache capacity + for (int i = 0; i < kCacheSize + 100; i++) { + Insert(1000 + i, 2000 + i); + } + + // Check whether the entries inserted in the beginning + // are evicted. Ones without extra ref are evicted and + // those with are not. + ASSERT_EQ(-1, Lookup(100)); + ASSERT_EQ(-1, Lookup(101)); + ASSERT_EQ(-1, Lookup(102)); + ASSERT_EQ(-1, Lookup(103)); + + ASSERT_EQ(-1, Lookup(300)); + ASSERT_EQ(-1, Lookup(301)); + ASSERT_EQ(-1, Lookup(302)); + ASSERT_EQ(-1, Lookup(303)); + + ASSERT_EQ(101, Lookup(200)); + ASSERT_EQ(102, Lookup(201)); + ASSERT_EQ(103, Lookup(202)); + ASSERT_EQ(104, Lookup(203)); + + // Cleaning up all the handles + cache_->Release(h201); + cache_->Release(h202); + cache_->Release(h203); + cache_->Release(h204); +} + +TEST_P(CacheTest, EvictEmptyCache) { + // Insert item large than capacity to trigger eviction on empty cache. + auto cache = NewCache(1, 0, false); + ASSERT_OK(cache->Insert("foo", nullptr, 10, dumbDeleter)); +} + +TEST_P(CacheTest, EraseFromDeleter) { + // Have deleter which will erase item from cache, which will re-enter + // the cache at that point. + std::shared_ptr cache = NewCache(10, 0, false); + ASSERT_OK(cache->Insert("foo", nullptr, 1, dumbDeleter)); + ASSERT_OK(cache->Insert("bar", cache.get(), 1, eraseDeleter)); + cache->Erase("bar"); + ASSERT_EQ(nullptr, cache->Lookup("foo")); + ASSERT_EQ(nullptr, cache->Lookup("bar")); +} + +TEST_P(CacheTest, ErasedHandleState) { + // insert a key and get two handles + Insert(100, 1000); + Cache::Handle* h1 = cache_->Lookup(EncodeKey(100)); + Cache::Handle* h2 = cache_->Lookup(EncodeKey(100)); + ASSERT_EQ(h1, h2); + ASSERT_EQ(DecodeValue(cache_->Value(h1)), 1000); + ASSERT_EQ(DecodeValue(cache_->Value(h2)), 1000); + + // delete the key from the cache + Erase(100); + // can no longer find in the cache + ASSERT_EQ(-1, Lookup(100)); + + // release one handle + cache_->Release(h1); + // still can't find in cache + ASSERT_EQ(-1, Lookup(100)); + + cache_->Release(h2); +} + +TEST_P(CacheTest, HeavyEntries) { + // Add a bunch of light and heavy entries and then count the combined + // size of items still in the cache, which must be approximately the + // same as the total capacity. + const int kLight = 1; + const int kHeavy = 10; + int added = 0; + int index = 0; + while (added < 2*kCacheSize) { + const int weight = (index & 1) ? kLight : kHeavy; + Insert(index, 1000+index, weight); + added += weight; + index++; + } + + int cached_weight = 0; + for (int i = 0; i < index; i++) { + const int weight = (i & 1 ? kLight : kHeavy); + int r = Lookup(i); + if (r >= 0) { + cached_weight += weight; + ASSERT_EQ(1000+i, r); + } + } + ASSERT_LE(cached_weight, kCacheSize + kCacheSize/10); +} + +TEST_P(CacheTest, NewId) { + uint64_t a = cache_->NewId(); + uint64_t b = cache_->NewId(); + ASSERT_NE(a, b); +} + + +class Value { + public: + explicit Value(size_t v) : v_(v) { } + + size_t v_; +}; + +namespace { +void deleter(const Slice& key, void* value) { + delete static_cast(value); +} +} // namespace + +TEST_P(CacheTest, ReleaseAndErase) { + std::shared_ptr cache = NewCache(5, 0, false); + Cache::Handle* handle; + Status s = cache->Insert(EncodeKey(100), EncodeValue(100), 1, + &CacheTest::Deleter, &handle); + ASSERT_TRUE(s.ok()); + ASSERT_EQ(5U, cache->GetCapacity()); + ASSERT_EQ(1U, cache->GetUsage()); + ASSERT_EQ(0U, deleted_keys_.size()); + auto erased = cache->Release(handle, true); + ASSERT_TRUE(erased); + // This tests that deleter has been called + ASSERT_EQ(1U, deleted_keys_.size()); +} + +TEST_P(CacheTest, ReleaseWithoutErase) { + std::shared_ptr cache = NewCache(5, 0, false); + Cache::Handle* handle; + Status s = cache->Insert(EncodeKey(100), EncodeValue(100), 1, + &CacheTest::Deleter, &handle); + ASSERT_TRUE(s.ok()); + ASSERT_EQ(5U, cache->GetCapacity()); + ASSERT_EQ(1U, cache->GetUsage()); + ASSERT_EQ(0U, deleted_keys_.size()); + auto erased = cache->Release(handle); + ASSERT_FALSE(erased); + // This tests that deleter is not called. When cache has free capacity it is + // not expected to immediately erase the released items. + ASSERT_EQ(0U, deleted_keys_.size()); +} + +TEST_P(CacheTest, SetCapacity) { + // test1: increase capacity + // lets create a cache with capacity 5, + // then, insert 5 elements, then increase capacity + // to 10, returned capacity should be 10, usage=5 + std::shared_ptr cache = NewCache(5, 0, false); + std::vector handles(10); + // Insert 5 entries, but not releasing. + for (size_t i = 0; i < 5; i++) { + std::string key = ToString(i+1); + Status s = cache->Insert(key, new Value(i + 1), 1, &deleter, &handles[i]); + ASSERT_TRUE(s.ok()); + } + ASSERT_EQ(5U, cache->GetCapacity()); + ASSERT_EQ(5U, cache->GetUsage()); + cache->SetCapacity(10); + ASSERT_EQ(10U, cache->GetCapacity()); + ASSERT_EQ(5U, cache->GetUsage()); + + // test2: decrease capacity + // insert 5 more elements to cache, then release 5, + // then decrease capacity to 7, final capacity should be 7 + // and usage should be 7 + for (size_t i = 5; i < 10; i++) { + std::string key = ToString(i+1); + Status s = cache->Insert(key, new Value(i + 1), 1, &deleter, &handles[i]); + ASSERT_TRUE(s.ok()); + } + ASSERT_EQ(10U, cache->GetCapacity()); + ASSERT_EQ(10U, cache->GetUsage()); + for (size_t i = 0; i < 5; i++) { + cache->Release(handles[i]); + } + ASSERT_EQ(10U, cache->GetCapacity()); + ASSERT_EQ(10U, cache->GetUsage()); + cache->SetCapacity(7); + ASSERT_EQ(7, cache->GetCapacity()); + ASSERT_EQ(7, cache->GetUsage()); + + // release remaining 5 to keep valgrind happy + for (size_t i = 5; i < 10; i++) { + cache->Release(handles[i]); + } +} + +TEST_P(CacheTest, SetStrictCapacityLimit) { + // test1: set the flag to false. Insert more keys than capacity. See if they + // all go through. + std::shared_ptr cache = NewLRUCache(5, 0, false); + std::vector handles(10); + Status s; + for (size_t i = 0; i < 10; i++) { + std::string key = ToString(i + 1); + s = cache->Insert(key, new Value(i + 1), 1, &deleter, &handles[i]); + ASSERT_OK(s); + ASSERT_NE(nullptr, handles[i]); + } + + // test2: set the flag to true. Insert and check if it fails. + std::string extra_key = "extra"; + Value* extra_value = new Value(0); + cache->SetStrictCapacityLimit(true); + Cache::Handle* handle; + s = cache->Insert(extra_key, extra_value, 1, &deleter, &handle); + ASSERT_TRUE(s.IsIncomplete()); + ASSERT_EQ(nullptr, handle); + + for (size_t i = 0; i < 10; i++) { + cache->Release(handles[i]); + } + + // test3: init with flag being true. + std::shared_ptr cache2 = NewLRUCache(5, 0, true); + for (size_t i = 0; i < 5; i++) { + std::string key = ToString(i + 1); + s = cache2->Insert(key, new Value(i + 1), 1, &deleter, &handles[i]); + ASSERT_OK(s); + ASSERT_NE(nullptr, handles[i]); + } + s = cache2->Insert(extra_key, extra_value, 1, &deleter, &handle); + ASSERT_TRUE(s.IsIncomplete()); + ASSERT_EQ(nullptr, handle); + // test insert without handle + s = cache2->Insert(extra_key, extra_value, 1, &deleter); + // AS if the key have been inserted into cache but get evicted immediately. + ASSERT_OK(s); + ASSERT_EQ(5, cache->GetUsage()); + ASSERT_EQ(nullptr, cache2->Lookup(extra_key)); + + for (size_t i = 0; i < 5; i++) { + cache2->Release(handles[i]); + } +} + +TEST_P(CacheTest, OverCapacity) { + size_t n = 10; + + // a LRUCache with n entries and one shard only + std::shared_ptr cache = NewCache(n, 0, false); + + std::vector handles(n+1); + + // Insert n+1 entries, but not releasing. + for (size_t i = 0; i < n + 1; i++) { + std::string key = ToString(i+1); + Status s = cache->Insert(key, new Value(i + 1), 1, &deleter, &handles[i]); + ASSERT_TRUE(s.ok()); + } + + // Guess what's in the cache now? + for (size_t i = 0; i < n + 1; i++) { + std::string key = ToString(i+1); + auto h = cache->Lookup(key); + ASSERT_TRUE(h != nullptr); + if (h) cache->Release(h); + } + + // the cache is over capacity since nothing could be evicted + ASSERT_EQ(n + 1U, cache->GetUsage()); + for (size_t i = 0; i < n + 1; i++) { + cache->Release(handles[i]); + } + // Make sure eviction is triggered. + cache->SetCapacity(n); + + // cache is under capacity now since elements were released + ASSERT_EQ(n, cache->GetUsage()); + + // element 0 is evicted and the rest is there + // This is consistent with the LRU policy since the element 0 + // was released first + for (size_t i = 0; i < n + 1; i++) { + std::string key = ToString(i+1); + auto h = cache->Lookup(key); + if (h) { + ASSERT_NE(i, 0U); + cache->Release(h); + } else { + ASSERT_EQ(i, 0U); + } + } +} + +namespace { +std::vector> callback_state; +void callback(void* entry, size_t charge) { + callback_state.push_back({DecodeValue(entry), static_cast(charge)}); +} +}; + +TEST_P(CacheTest, ApplyToAllCacheEntiresTest) { + std::vector> inserted; + callback_state.clear(); + + for (int i = 0; i < 10; ++i) { + Insert(i, i * 2, i + 1); + inserted.push_back({i * 2, i + 1}); + } + cache_->ApplyToAllCacheEntries(callback, true); + + std::sort(inserted.begin(), inserted.end()); + std::sort(callback_state.begin(), callback_state.end()); + ASSERT_TRUE(inserted == callback_state); +} + +TEST_P(CacheTest, DefaultShardBits) { + // test1: set the flag to false. Insert more keys than capacity. See if they + // all go through. + std::shared_ptr cache = NewCache(16 * 1024L * 1024L); + ShardedCache* sc = dynamic_cast(cache.get()); + ASSERT_EQ(5, sc->GetNumShardBits()); + + cache = NewLRUCache(511 * 1024L, -1, true); + sc = dynamic_cast(cache.get()); + ASSERT_EQ(0, sc->GetNumShardBits()); + + cache = NewLRUCache(1024L * 1024L * 1024L, -1, true); + sc = dynamic_cast(cache.get()); + ASSERT_EQ(6, sc->GetNumShardBits()); +} + +#ifdef SUPPORT_CLOCK_CACHE +shared_ptr (*new_clock_cache_func)(size_t, int, bool) = NewClockCache; +INSTANTIATE_TEST_CASE_P(CacheTestInstance, CacheTest, + testing::Values(kLRU, kClock)); +#else +INSTANTIATE_TEST_CASE_P(CacheTestInstance, CacheTest, testing::Values(kLRU)); +#endif // SUPPORT_CLOCK_CACHE + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/cache/clock_cache.cc b/cache/clock_cache.cc new file mode 100644 index 00000000000..7e42714ef14 --- /dev/null +++ b/cache/clock_cache.cc @@ -0,0 +1,729 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "cache/clock_cache.h" + +#ifndef SUPPORT_CLOCK_CACHE + +namespace rocksdb { + +std::shared_ptr NewClockCache(size_t capacity, int num_shard_bits, + bool strict_capacity_limit) { + // Clock cache not supported. + return nullptr; +} + +} // namespace rocksdb + +#else + +#include +#include +#include + +// "tbb/concurrent_hash_map.h" requires RTTI if exception is enabled. +// Disable it so users can chooose to disable RTTI. +#ifndef ROCKSDB_USE_RTTI +#define TBB_USE_EXCEPTIONS 0 +#endif +#include "tbb/concurrent_hash_map.h" + +#include "cache/sharded_cache.h" +#include "port/port.h" +#include "util/autovector.h" +#include "util/mutexlock.h" + +namespace rocksdb { + +namespace { + +// An implementation of the Cache interface based on CLOCK algorithm, with +// better concurrent performance than LRUCache. The idea of CLOCK algorithm +// is to maintain all cache entries in a circular list, and an iterator +// (the "head") pointing to the last examined entry. Eviction starts from the +// current head. Each entry is given a second chance before eviction, if it +// has been access since last examine. In contrast to LRU, no modification +// to the internal data-structure (except for flipping the usage bit) needs +// to be done upon lookup. This gives us oppertunity to implement a cache +// with better concurrency. +// +// Each cache entry is represented by a cache handle, and all the handles +// are arranged in a circular list, as describe above. Upon erase of an entry, +// we never remove the handle. Instead, the handle is put into a recycle bin +// to be re-use. This is to avoid memory dealocation, which is hard to deal +// with in concurrent environment. +// +// The cache also maintains a concurrent hash map for lookup. Any concurrent +// hash map implementation should do the work. We currently use +// tbb::concurrent_hash_map because it supports concurrent erase. +// +// Each cache handle has the following flags and counters, which are squeeze +// in an atomic interger, to make sure the handle always be in a consistent +// state: +// +// * In-cache bit: whether the entry is reference by the cache itself. If +// an entry is in cache, its key would also be available in the hash map. +// * Usage bit: whether the entry has been access by user since last +// examine for eviction. Can be reset by eviction. +// * Reference count: reference count by user. +// +// An entry can be reference only when it's in cache. An entry can be evicted +// only when it is in cache, has no usage since last examine, and reference +// count is zero. +// +// The follow figure shows a possible layout of the cache. Boxes represents +// cache handles and numbers in each box being in-cache bit, usage bit and +// reference count respectively. +// +// hash map: +// +-------+--------+ +// | key | handle | +// +-------+--------+ +// | "foo" | 5 |-------------------------------------+ +// +-------+--------+ | +// | "bar" | 2 |--+ | +// +-------+--------+ | | +// | | +// head | | +// | | | +// circular list: | | | +// +-------+ +-------+ +-------+ +-------+ +-------+ +------- +// |(0,0,0)|---|(1,1,0)|---|(0,0,0)|---|(0,1,3)|---|(1,0,0)|---| ... +// +-------+ +-------+ +-------+ +-------+ +-------+ +------- +// | | +// +-------+ +-----------+ +// | | +// +---+---+ +// recycle bin: | 1 | 3 | +// +---+---+ +// +// Suppose we try to insert "baz" into the cache at this point and the cache is +// full. The cache will first look for entries to evict, starting from where +// head points to (the second entry). It resets usage bit of the second entry, +// skips the third and fourth entry since they are not in cache, and finally +// evict the fifth entry ("foo"). It looks at recycle bin for available handle, +// grabs handle 3, and insert the key into the handle. The following figure +// shows the resulting layout. +// +// hash map: +// +-------+--------+ +// | key | handle | +// +-------+--------+ +// | "baz" | 3 |-------------+ +// +-------+--------+ | +// | "bar" | 2 |--+ | +// +-------+--------+ | | +// | | +// | | head +// | | | +// circular list: | | | +// +-------+ +-------+ +-------+ +-------+ +-------+ +------- +// |(0,0,0)|---|(1,0,0)|---|(1,0,0)|---|(0,1,3)|---|(0,0,0)|---| ... +// +-------+ +-------+ +-------+ +-------+ +-------+ +------- +// | | +// +-------+ +-----------------------------------+ +// | | +// +---+---+ +// recycle bin: | 1 | 5 | +// +---+---+ +// +// A global mutex guards the circular list, the head, and the recycle bin. +// We additionally require that modifying the hash map needs to hold the mutex. +// As such, Modifying the cache (such as Insert() and Erase()) require to +// hold the mutex. Lookup() only access the hash map and the flags associated +// with each handle, and don't require explicit locking. Release() has to +// acquire the mutex only when it releases the last reference to the entry and +// the entry has been erased from cache explicitly. A future improvement could +// be to remove the mutex completely. +// +// Benchmark: +// We run readrandom db_bench on a test DB of size 13GB, with size of each +// level: +// +// Level Files Size(MB) +// ------------------------- +// L0 1 0.01 +// L1 18 17.32 +// L2 230 182.94 +// L3 1186 1833.63 +// L4 4602 8140.30 +// +// We test with both 32 and 16 read threads, with 2GB cache size (the whole DB +// doesn't fits in) and 64GB cache size (the whole DB can fit in cache), and +// whether to put index and filter blocks in block cache. The benchmark runs +// with +// with RocksDB 4.10. We got the following result: +// +// Threads Cache Cache ClockCache LRUCache +// Size Index/Filter Throughput(MB/s) Hit Throughput(MB/s) Hit +// 32 2GB yes 466.7 85.9% 433.7 86.5% +// 32 2GB no 529.9 72.7% 532.7 73.9% +// 32 64GB yes 649.9 99.9% 507.9 99.9% +// 32 64GB no 740.4 99.9% 662.8 99.9% +// 16 2GB yes 278.4 85.9% 283.4 86.5% +// 16 2GB no 318.6 72.7% 335.8 73.9% +// 16 64GB yes 391.9 99.9% 353.3 99.9% +// 16 64GB no 433.8 99.8% 419.4 99.8% + +// Cache entry meta data. +struct CacheHandle { + Slice key; + uint32_t hash; + void* value; + size_t charge; + void (*deleter)(const Slice&, void* value); + + // Flags and counters associated with the cache handle: + // lowest bit: n-cache bit + // second lowest bit: usage bit + // the rest bits: reference count + // The handle is unused when flags equals to 0. The thread decreases the count + // to 0 is responsible to put the handle back to recycle_ and cleanup memory. + std::atomic flags; + + CacheHandle() = default; + + CacheHandle(const CacheHandle& a) { *this = a; } + + CacheHandle(const Slice& k, void* v, + void (*del)(const Slice& key, void* value)) + : key(k), value(v), deleter(del) {} + + CacheHandle& operator=(const CacheHandle& a) { + // Only copy members needed for deletion. + key = a.key; + value = a.value; + deleter = a.deleter; + return *this; + } +}; + +// Key of hash map. We store hash value with the key for convenience. +struct CacheKey { + Slice key; + uint32_t hash_value; + + CacheKey() = default; + + CacheKey(const Slice& k, uint32_t h) { + key = k; + hash_value = h; + } + + static bool equal(const CacheKey& a, const CacheKey& b) { + return a.hash_value == b.hash_value && a.key == b.key; + } + + static size_t hash(const CacheKey& a) { + return static_cast(a.hash_value); + } +}; + +struct CleanupContext { + // List of values to be deleted, along with the key and deleter. + autovector to_delete_value; + + // List of keys to be deleted. + autovector to_delete_key; +}; + +// A cache shard which maintains its own CLOCK cache. +class ClockCacheShard : public CacheShard { + public: + // Hash map type. + typedef tbb::concurrent_hash_map HashTable; + + ClockCacheShard(); + ~ClockCacheShard(); + + // Interfaces + virtual void SetCapacity(size_t capacity) override; + virtual void SetStrictCapacityLimit(bool strict_capacity_limit) override; + virtual Status Insert(const Slice& key, uint32_t hash, void* value, + size_t charge, + void (*deleter)(const Slice& key, void* value), + Cache::Handle** handle, + Cache::Priority priority) override; + virtual Cache::Handle* Lookup(const Slice& key, uint32_t hash) override; + // If the entry in in cache, increase reference count and return true. + // Return false otherwise. + // + // Not necessary to hold mutex_ before being called. + virtual bool Ref(Cache::Handle* handle) override; + virtual bool Release(Cache::Handle* handle, + bool force_erase = false) override; + virtual void Erase(const Slice& key, uint32_t hash) override; + bool EraseAndConfirm(const Slice& key, uint32_t hash, + CleanupContext* context); + virtual size_t GetUsage() const override; + virtual size_t GetPinnedUsage() const override; + virtual void EraseUnRefEntries() override; + virtual void ApplyToAllCacheEntries(void (*callback)(void*, size_t), + bool thread_safe) override; + + private: + static const uint32_t kInCacheBit = 1; + static const uint32_t kUsageBit = 2; + static const uint32_t kRefsOffset = 2; + static const uint32_t kOneRef = 1 << kRefsOffset; + + // Helper functions to extract cache handle flags and counters. + static bool InCache(uint32_t flags) { return flags & kInCacheBit; } + static bool HasUsage(uint32_t flags) { return flags & kUsageBit; } + static uint32_t CountRefs(uint32_t flags) { return flags >> kRefsOffset; } + + // Decrease reference count of the entry. If this decreases the count to 0, + // recycle the entry. If set_usage is true, also set the usage bit. + // + // returns true if a value is erased. + // + // Not necessary to hold mutex_ before being called. + bool Unref(CacheHandle* handle, bool set_usage, CleanupContext* context); + + // Unset in-cache bit of the entry. Recycle the handle if necessary. + // + // returns true if a value is erased. + // + // Has to hold mutex_ before being called. + bool UnsetInCache(CacheHandle* handle, CleanupContext* context); + + // Put the handle back to recycle_ list, and put the value associated with + // it into to-be-deleted list. It doesn't cleanup the key as it might be + // reused by another handle. + // + // Has to hold mutex_ before being called. + void RecycleHandle(CacheHandle* handle, CleanupContext* context); + + // Delete keys and values in to-be-deleted list. Call the method without + // holding mutex, as destructors can be expensive. + void Cleanup(const CleanupContext& context); + + // Examine the handle for eviction. If the handle is in cache, usage bit is + // not set, and referece count is 0, evict it from cache. Otherwise unset + // the usage bit. + // + // Has to hold mutex_ before being called. + bool TryEvict(CacheHandle* value, CleanupContext* context); + + // Scan through the circular list, evict entries until we get enough capacity + // for new cache entry of specific size. Return true if success, false + // otherwise. + // + // Has to hold mutex_ before being called. + bool EvictFromCache(size_t charge, CleanupContext* context); + + CacheHandle* Insert(const Slice& key, uint32_t hash, void* value, + size_t change, + void (*deleter)(const Slice& key, void* value), + bool hold_reference, CleanupContext* context); + + // Guards list_, head_, and recycle_. In addition, updating table_ also has + // to hold the mutex, to avoid the cache being in inconsistent state. + mutable port::Mutex mutex_; + + // The circular list of cache handles. Initially the list is empty. Once a + // handle is needed by insertion, and no more handles are available in + // recycle bin, one more handle is appended to the end. + // + // We use std::deque for the circular list because we want to make sure + // pointers to handles are valid through out the life-cycle of the cache + // (in contrast to std::vector), and be able to grow the list (in contrast + // to statically allocated arrays). + std::deque list_; + + // Pointer to the next handle in the circular list to be examine for + // eviction. + size_t head_; + + // Recycle bin of cache handles. + autovector recycle_; + + // Maximum cache size. + std::atomic capacity_; + + // Current total size of the cache. + std::atomic usage_; + + // Total un-released cache size. + std::atomic pinned_usage_; + + // Whether allow insert into cache if cache is full. + std::atomic strict_capacity_limit_; + + // Hash table (tbb::concurrent_hash_map) for lookup. + HashTable table_; +}; + +ClockCacheShard::ClockCacheShard() + : head_(0), usage_(0), pinned_usage_(0), strict_capacity_limit_(false) {} + +ClockCacheShard::~ClockCacheShard() { + for (auto& handle : list_) { + uint32_t flags = handle.flags.load(std::memory_order_relaxed); + if (InCache(flags) || CountRefs(flags) > 0) { + (*handle.deleter)(handle.key, handle.value); + delete[] handle.key.data(); + } + } +} + +size_t ClockCacheShard::GetUsage() const { + return usage_.load(std::memory_order_relaxed); +} + +size_t ClockCacheShard::GetPinnedUsage() const { + return pinned_usage_.load(std::memory_order_relaxed); +} + +void ClockCacheShard::ApplyToAllCacheEntries(void (*callback)(void*, size_t), + bool thread_safe) { + if (thread_safe) { + mutex_.Lock(); + } + for (auto& handle : list_) { + // Use relaxed semantics instead of acquire semantics since we are either + // holding mutex, or don't have thread safe requirement. + uint32_t flags = handle.flags.load(std::memory_order_relaxed); + if (InCache(flags)) { + callback(handle.value, handle.charge); + } + } + if (thread_safe) { + mutex_.Unlock(); + } +} + +void ClockCacheShard::RecycleHandle(CacheHandle* handle, + CleanupContext* context) { + mutex_.AssertHeld(); + assert(!InCache(handle->flags) && CountRefs(handle->flags) == 0); + context->to_delete_key.push_back(handle->key.data()); + context->to_delete_value.emplace_back(*handle); + handle->key.clear(); + handle->value = nullptr; + handle->deleter = nullptr; + recycle_.push_back(handle); + usage_.fetch_sub(handle->charge, std::memory_order_relaxed); +} + +void ClockCacheShard::Cleanup(const CleanupContext& context) { + for (const CacheHandle& handle : context.to_delete_value) { + if (handle.deleter) { + (*handle.deleter)(handle.key, handle.value); + } + } + for (const char* key : context.to_delete_key) { + delete[] key; + } +} + +bool ClockCacheShard::Ref(Cache::Handle* h) { + auto handle = reinterpret_cast(h); + // CAS loop to increase reference count. + uint32_t flags = handle->flags.load(std::memory_order_relaxed); + while (InCache(flags)) { + // Use acquire semantics on success, as further operations on the cache + // entry has to be order after reference count is increased. + if (handle->flags.compare_exchange_weak(flags, flags + kOneRef, + std::memory_order_acquire, + std::memory_order_relaxed)) { + if (CountRefs(flags) == 0) { + // No reference count before the operation. + pinned_usage_.fetch_add(handle->charge, std::memory_order_relaxed); + } + return true; + } + } + return false; +} + +bool ClockCacheShard::Unref(CacheHandle* handle, bool set_usage, + CleanupContext* context) { + if (set_usage) { + handle->flags.fetch_or(kUsageBit, std::memory_order_relaxed); + } + // Use acquire-release semantics as previous operations on the cache entry + // has to be order before reference count is decreased, and potential cleanup + // of the entry has to be order after. + uint32_t flags = handle->flags.fetch_sub(kOneRef, std::memory_order_acq_rel); + assert(CountRefs(flags) > 0); + if (CountRefs(flags) == 1) { + // this is the last reference. + pinned_usage_.fetch_sub(handle->charge, std::memory_order_relaxed); + // Cleanup if it is the last reference. + if (!InCache(flags)) { + MutexLock l(&mutex_); + RecycleHandle(handle, context); + } + } + return context->to_delete_value.size(); +} + +bool ClockCacheShard::UnsetInCache(CacheHandle* handle, + CleanupContext* context) { + mutex_.AssertHeld(); + // Use acquire-release semantics as previous operations on the cache entry + // has to be order before reference count is decreased, and potential cleanup + // of the entry has to be order after. + uint32_t flags = + handle->flags.fetch_and(~kInCacheBit, std::memory_order_acq_rel); + // Cleanup if it is the last reference. + if (InCache(flags) && CountRefs(flags) == 0) { + RecycleHandle(handle, context); + } + return context->to_delete_value.size(); +} + +bool ClockCacheShard::TryEvict(CacheHandle* handle, CleanupContext* context) { + mutex_.AssertHeld(); + uint32_t flags = kInCacheBit; + if (handle->flags.compare_exchange_strong(flags, 0, std::memory_order_acquire, + std::memory_order_relaxed)) { + bool erased __attribute__((__unused__)) = + table_.erase(CacheKey(handle->key, handle->hash)); + assert(erased); + RecycleHandle(handle, context); + return true; + } + handle->flags.fetch_and(~kUsageBit, std::memory_order_relaxed); + return false; +} + +bool ClockCacheShard::EvictFromCache(size_t charge, CleanupContext* context) { + size_t usage = usage_.load(std::memory_order_relaxed); + size_t capacity = capacity_.load(std::memory_order_relaxed); + if (usage == 0) { + return charge <= capacity; + } + size_t new_head = head_; + bool second_iteration = false; + while (usage + charge > capacity) { + assert(new_head < list_.size()); + if (TryEvict(&list_[new_head], context)) { + usage = usage_.load(std::memory_order_relaxed); + } + new_head = (new_head + 1 >= list_.size()) ? 0 : new_head + 1; + if (new_head == head_) { + if (second_iteration) { + return false; + } else { + second_iteration = true; + } + } + } + head_ = new_head; + return true; +} + +void ClockCacheShard::SetCapacity(size_t capacity) { + CleanupContext context; + { + MutexLock l(&mutex_); + capacity_.store(capacity, std::memory_order_relaxed); + EvictFromCache(0, &context); + } + Cleanup(context); +} + +void ClockCacheShard::SetStrictCapacityLimit(bool strict_capacity_limit) { + strict_capacity_limit_.store(strict_capacity_limit, + std::memory_order_relaxed); +} + +CacheHandle* ClockCacheShard::Insert( + const Slice& key, uint32_t hash, void* value, size_t charge, + void (*deleter)(const Slice& key, void* value), bool hold_reference, + CleanupContext* context) { + MutexLock l(&mutex_); + bool success = EvictFromCache(charge, context); + bool strict = strict_capacity_limit_.load(std::memory_order_relaxed); + if (!success && (strict || !hold_reference)) { + context->to_delete_key.push_back(key.data()); + if (!hold_reference) { + context->to_delete_value.emplace_back(key, value, deleter); + } + return nullptr; + } + // Grab available handle from recycle bin. If recycle bin is empty, create + // and append new handle to end of circular list. + CacheHandle* handle = nullptr; + if (!recycle_.empty()) { + handle = recycle_.back(); + recycle_.pop_back(); + } else { + list_.emplace_back(); + handle = &list_.back(); + } + // Fill handle. + handle->key = key; + handle->hash = hash; + handle->value = value; + handle->charge = charge; + handle->deleter = deleter; + uint32_t flags = hold_reference ? kInCacheBit + kOneRef : kInCacheBit; + handle->flags.store(flags, std::memory_order_relaxed); + HashTable::accessor accessor; + if (table_.find(accessor, CacheKey(key, hash))) { + CacheHandle* existing_handle = accessor->second; + table_.erase(accessor); + UnsetInCache(existing_handle, context); + } + table_.insert(HashTable::value_type(CacheKey(key, hash), handle)); + if (hold_reference) { + pinned_usage_.fetch_add(charge, std::memory_order_relaxed); + } + usage_.fetch_add(charge, std::memory_order_relaxed); + return handle; +} + +Status ClockCacheShard::Insert(const Slice& key, uint32_t hash, void* value, + size_t charge, + void (*deleter)(const Slice& key, void* value), + Cache::Handle** out_handle, + Cache::Priority priority) { + CleanupContext context; + HashTable::accessor accessor; + char* key_data = new char[key.size()]; + memcpy(key_data, key.data(), key.size()); + Slice key_copy(key_data, key.size()); + CacheHandle* handle = Insert(key_copy, hash, value, charge, deleter, + out_handle != nullptr, &context); + Status s; + if (out_handle != nullptr) { + if (handle == nullptr) { + s = Status::Incomplete("Insert failed due to LRU cache being full."); + } else { + *out_handle = reinterpret_cast(handle); + } + } + Cleanup(context); + return s; +} + +Cache::Handle* ClockCacheShard::Lookup(const Slice& key, uint32_t hash) { + HashTable::const_accessor accessor; + if (!table_.find(accessor, CacheKey(key, hash))) { + return nullptr; + } + CacheHandle* handle = accessor->second; + accessor.release(); + // Ref() could fail if another thread sneak in and evict/erase the cache + // entry before we are able to hold reference. + if (!Ref(reinterpret_cast(handle))) { + return nullptr; + } + // Double check the key since the handle may now representing another key + // if other threads sneak in, evict/erase the entry and re-used the handle + // for another cache entry. + if (hash != handle->hash || key != handle->key) { + CleanupContext context; + Unref(handle, false, &context); + // It is possible Unref() delete the entry, so we need to cleanup. + Cleanup(context); + return nullptr; + } + return reinterpret_cast(handle); +} + +bool ClockCacheShard::Release(Cache::Handle* h, bool force_erase) { + CleanupContext context; + CacheHandle* handle = reinterpret_cast(h); + bool erased = Unref(handle, true, &context); + if (force_erase && !erased) { + erased = EraseAndConfirm(handle->key, handle->hash, &context); + } + Cleanup(context); + return erased; +} + +void ClockCacheShard::Erase(const Slice& key, uint32_t hash) { + CleanupContext context; + EraseAndConfirm(key, hash, &context); + Cleanup(context); +} + +bool ClockCacheShard::EraseAndConfirm(const Slice& key, uint32_t hash, + CleanupContext* context) { + MutexLock l(&mutex_); + HashTable::accessor accessor; + bool erased = false; + if (table_.find(accessor, CacheKey(key, hash))) { + CacheHandle* handle = accessor->second; + table_.erase(accessor); + erased = UnsetInCache(handle, context); + } + return erased; +} + +void ClockCacheShard::EraseUnRefEntries() { + CleanupContext context; + { + MutexLock l(&mutex_); + table_.clear(); + for (auto& handle : list_) { + UnsetInCache(&handle, &context); + } + } + Cleanup(context); +} + +class ClockCache : public ShardedCache { + public: + ClockCache(size_t capacity, int num_shard_bits, bool strict_capacity_limit) + : ShardedCache(capacity, num_shard_bits, strict_capacity_limit) { + int num_shards = 1 << num_shard_bits; + shards_ = new ClockCacheShard[num_shards]; + SetCapacity(capacity); + SetStrictCapacityLimit(strict_capacity_limit); + } + + virtual ~ClockCache() { delete[] shards_; } + + virtual const char* Name() const override { return "ClockCache"; } + + virtual CacheShard* GetShard(int shard) override { + return reinterpret_cast(&shards_[shard]); + } + + virtual const CacheShard* GetShard(int shard) const override { + return reinterpret_cast(&shards_[shard]); + } + + virtual void* Value(Handle* handle) override { + return reinterpret_cast(handle)->value; + } + + virtual size_t GetCharge(Handle* handle) const override { + return reinterpret_cast(handle)->charge; + } + + virtual uint32_t GetHash(Handle* handle) const override { + return reinterpret_cast(handle)->hash; + } + + virtual void DisownData() override { shards_ = nullptr; } + + private: + ClockCacheShard* shards_; +}; + +} // end anonymous namespace + +std::shared_ptr NewClockCache(size_t capacity, int num_shard_bits, + bool strict_capacity_limit) { + if (num_shard_bits < 0) { + num_shard_bits = GetDefaultCacheShardBits(capacity); + } + return std::make_shared(capacity, num_shard_bits, + strict_capacity_limit); +} + +} // namespace rocksdb + +#endif // SUPPORT_CLOCK_CACHE diff --git a/cache/clock_cache.h b/cache/clock_cache.h new file mode 100644 index 00000000000..1614c0ed454 --- /dev/null +++ b/cache/clock_cache.h @@ -0,0 +1,16 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include "rocksdb/cache.h" + +#if defined(TBB) && !defined(ROCKSDB_LITE) +#define SUPPORT_CLOCK_CACHE +#endif diff --git a/cache/lru_cache.cc b/cache/lru_cache.cc new file mode 100644 index 00000000000..d29e7093427 --- /dev/null +++ b/cache/lru_cache.cc @@ -0,0 +1,530 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include "cache/lru_cache.h" + +#include +#include +#include +#include + +#include "util/mutexlock.h" + +namespace rocksdb { + +LRUHandleTable::LRUHandleTable() : list_(nullptr), length_(0), elems_(0) { + Resize(); +} + +LRUHandleTable::~LRUHandleTable() { + ApplyToAllCacheEntries([](LRUHandle* h) { + if (h->refs == 1) { + h->Free(); + } + }); + delete[] list_; +} + +LRUHandle* LRUHandleTable::Lookup(const Slice& key, uint32_t hash) { + return *FindPointer(key, hash); +} + +LRUHandle* LRUHandleTable::Insert(LRUHandle* h) { + LRUHandle** ptr = FindPointer(h->key(), h->hash); + LRUHandle* old = *ptr; + h->next_hash = (old == nullptr ? nullptr : old->next_hash); + *ptr = h; + if (old == nullptr) { + ++elems_; + if (elems_ > length_) { + // Since each cache entry is fairly large, we aim for a small + // average linked list length (<= 1). + Resize(); + } + } + return old; +} + +LRUHandle* LRUHandleTable::Remove(const Slice& key, uint32_t hash) { + LRUHandle** ptr = FindPointer(key, hash); + LRUHandle* result = *ptr; + if (result != nullptr) { + *ptr = result->next_hash; + --elems_; + } + return result; +} + +LRUHandle** LRUHandleTable::FindPointer(const Slice& key, uint32_t hash) { + LRUHandle** ptr = &list_[hash & (length_ - 1)]; + while (*ptr != nullptr && ((*ptr)->hash != hash || key != (*ptr)->key())) { + ptr = &(*ptr)->next_hash; + } + return ptr; +} + +void LRUHandleTable::Resize() { + uint32_t new_length = 16; + while (new_length < elems_ * 1.5) { + new_length *= 2; + } + LRUHandle** new_list = new LRUHandle*[new_length]; + memset(new_list, 0, sizeof(new_list[0]) * new_length); + uint32_t count = 0; + for (uint32_t i = 0; i < length_; i++) { + LRUHandle* h = list_[i]; + while (h != nullptr) { + LRUHandle* next = h->next_hash; + uint32_t hash = h->hash; + LRUHandle** ptr = &new_list[hash & (new_length - 1)]; + h->next_hash = *ptr; + *ptr = h; + h = next; + count++; + } + } + assert(elems_ == count); + delete[] list_; + list_ = new_list; + length_ = new_length; +} + +LRUCacheShard::LRUCacheShard() + : high_pri_pool_usage_(0), usage_(0), lru_usage_(0) { + // Make empty circular linked list + lru_.next = &lru_; + lru_.prev = &lru_; + lru_low_pri_ = &lru_; +} + +LRUCacheShard::~LRUCacheShard() {} + +bool LRUCacheShard::Unref(LRUHandle* e) { + assert(e->refs > 0); + e->refs--; + return e->refs == 0; +} + +// Call deleter and free + +void LRUCacheShard::EraseUnRefEntries() { + autovector last_reference_list; + { + MutexLock l(&mutex_); + while (lru_.next != &lru_) { + LRUHandle* old = lru_.next; + assert(old->InCache()); + assert(old->refs == + 1); // LRU list contains elements which may be evicted + LRU_Remove(old); + table_.Remove(old->key(), old->hash); + old->SetInCache(false); + Unref(old); + usage_ -= old->charge; + last_reference_list.push_back(old); + } + } + + for (auto entry : last_reference_list) { + entry->Free(); + } +} + +void LRUCacheShard::ApplyToAllCacheEntries(void (*callback)(void*, size_t), + bool thread_safe) { + if (thread_safe) { + mutex_.Lock(); + } + table_.ApplyToAllCacheEntries( + [callback](LRUHandle* h) { callback(h->value, h->charge); }); + if (thread_safe) { + mutex_.Unlock(); + } +} + +void LRUCacheShard::TEST_GetLRUList(LRUHandle** lru, LRUHandle** lru_low_pri) { + *lru = &lru_; + *lru_low_pri = lru_low_pri_; +} + +size_t LRUCacheShard::TEST_GetLRUSize() { + LRUHandle* lru_handle = lru_.next; + size_t lru_size = 0; + while (lru_handle != &lru_) { + lru_size++; + lru_handle = lru_handle->next; + } + return lru_size; +} + +void LRUCacheShard::LRU_Remove(LRUHandle* e) { + assert(e->next != nullptr); + assert(e->prev != nullptr); + if (lru_low_pri_ == e) { + lru_low_pri_ = e->prev; + } + e->next->prev = e->prev; + e->prev->next = e->next; + e->prev = e->next = nullptr; + lru_usage_ -= e->charge; + if (e->InHighPriPool()) { + assert(high_pri_pool_usage_ >= e->charge); + high_pri_pool_usage_ -= e->charge; + } +} + +void LRUCacheShard::LRU_Insert(LRUHandle* e) { + assert(e->next == nullptr); + assert(e->prev == nullptr); + if (high_pri_pool_ratio_ > 0 && e->IsHighPri()) { + // Inset "e" to head of LRU list. + e->next = &lru_; + e->prev = lru_.prev; + e->prev->next = e; + e->next->prev = e; + e->SetInHighPriPool(true); + high_pri_pool_usage_ += e->charge; + MaintainPoolSize(); + } else { + // Insert "e" to the head of low-pri pool. Note that when + // high_pri_pool_ratio is 0, head of low-pri pool is also head of LRU list. + e->next = lru_low_pri_->next; + e->prev = lru_low_pri_; + e->prev->next = e; + e->next->prev = e; + e->SetInHighPriPool(false); + lru_low_pri_ = e; + } + lru_usage_ += e->charge; +} + +void LRUCacheShard::MaintainPoolSize() { + while (high_pri_pool_usage_ > high_pri_pool_capacity_) { + // Overflow last entry in high-pri pool to low-pri pool. + lru_low_pri_ = lru_low_pri_->next; + assert(lru_low_pri_ != &lru_); + lru_low_pri_->SetInHighPriPool(false); + high_pri_pool_usage_ -= lru_low_pri_->charge; + } +} + +void LRUCacheShard::EvictFromLRU(size_t charge, + autovector* deleted) { + while (usage_ + charge > capacity_ && lru_.next != &lru_) { + LRUHandle* old = lru_.next; + assert(old->InCache()); + assert(old->refs == 1); // LRU list contains elements which may be evicted + LRU_Remove(old); + table_.Remove(old->key(), old->hash); + old->SetInCache(false); + Unref(old); + usage_ -= old->charge; + deleted->push_back(old); + } +} + +void* LRUCacheShard::operator new(size_t size) { + return port::cacheline_aligned_alloc(size); +} + +void* LRUCacheShard::operator new[](size_t size) { + return port::cacheline_aligned_alloc(size); +} + +void LRUCacheShard::operator delete(void *memblock) { + port::cacheline_aligned_free(memblock); +} + +void LRUCacheShard::operator delete[](void* memblock) { + port::cacheline_aligned_free(memblock); +} + +void LRUCacheShard::SetCapacity(size_t capacity) { + autovector last_reference_list; + { + MutexLock l(&mutex_); + capacity_ = capacity; + high_pri_pool_capacity_ = capacity_ * high_pri_pool_ratio_; + EvictFromLRU(0, &last_reference_list); + } + // we free the entries here outside of mutex for + // performance reasons + for (auto entry : last_reference_list) { + entry->Free(); + } +} + +void LRUCacheShard::SetStrictCapacityLimit(bool strict_capacity_limit) { + MutexLock l(&mutex_); + strict_capacity_limit_ = strict_capacity_limit; +} + +Cache::Handle* LRUCacheShard::Lookup(const Slice& key, uint32_t hash) { + MutexLock l(&mutex_); + LRUHandle* e = table_.Lookup(key, hash); + if (e != nullptr) { + assert(e->InCache()); + if (e->refs == 1) { + LRU_Remove(e); + } + e->refs++; + } + return reinterpret_cast(e); +} + +bool LRUCacheShard::Ref(Cache::Handle* h) { + LRUHandle* handle = reinterpret_cast(h); + MutexLock l(&mutex_); + if (handle->InCache() && handle->refs == 1) { + LRU_Remove(handle); + } + handle->refs++; + return true; +} + +void LRUCacheShard::SetHighPriorityPoolRatio(double high_pri_pool_ratio) { + MutexLock l(&mutex_); + high_pri_pool_ratio_ = high_pri_pool_ratio; + high_pri_pool_capacity_ = capacity_ * high_pri_pool_ratio_; + MaintainPoolSize(); +} + +bool LRUCacheShard::Release(Cache::Handle* handle, bool force_erase) { + if (handle == nullptr) { + return false; + } + LRUHandle* e = reinterpret_cast(handle); + bool last_reference = false; + { + MutexLock l(&mutex_); + last_reference = Unref(e); + if (last_reference) { + usage_ -= e->charge; + } + if (e->refs == 1 && e->InCache()) { + // The item is still in cache, and nobody else holds a reference to it + if (usage_ > capacity_ || force_erase) { + // the cache is full + // The LRU list must be empty since the cache is full + assert(!(usage_ > capacity_) || lru_.next == &lru_); + // take this opportunity and remove the item + table_.Remove(e->key(), e->hash); + e->SetInCache(false); + Unref(e); + usage_ -= e->charge; + last_reference = true; + } else { + // put the item on the list to be potentially freed + LRU_Insert(e); + } + } + } + + // free outside of mutex + if (last_reference) { + e->Free(); + } + return last_reference; +} + +Status LRUCacheShard::Insert(const Slice& key, uint32_t hash, void* value, + size_t charge, + void (*deleter)(const Slice& key, void* value), + Cache::Handle** handle, Cache::Priority priority) { + // Allocate the memory here outside of the mutex + // If the cache is full, we'll have to release it + // It shouldn't happen very often though. + LRUHandle* e = reinterpret_cast( + new char[sizeof(LRUHandle) - 1 + key.size()]); + Status s; + autovector last_reference_list; + + e->value = value; + e->deleter = deleter; + e->charge = charge; + e->key_length = key.size(); + e->hash = hash; + e->refs = (handle == nullptr + ? 1 + : 2); // One from LRUCache, one for the returned handle + e->next = e->prev = nullptr; + e->SetInCache(true); + e->SetPriority(priority); + memcpy(e->key_data, key.data(), key.size()); + + { + MutexLock l(&mutex_); + + // Free the space following strict LRU policy until enough space + // is freed or the lru list is empty + EvictFromLRU(charge, &last_reference_list); + + if (usage_ - lru_usage_ + charge > capacity_ && + (strict_capacity_limit_ || handle == nullptr)) { + if (handle == nullptr) { + // Don't insert the entry but still return ok, as if the entry inserted + // into cache and get evicted immediately. + last_reference_list.push_back(e); + } else { + delete[] reinterpret_cast(e); + *handle = nullptr; + s = Status::Incomplete("Insert failed due to LRU cache being full."); + } + } else { + // insert into the cache + // note that the cache might get larger than its capacity if not enough + // space was freed + LRUHandle* old = table_.Insert(e); + usage_ += e->charge; + if (old != nullptr) { + old->SetInCache(false); + if (Unref(old)) { + usage_ -= old->charge; + // old is on LRU because it's in cache and its reference count + // was just 1 (Unref returned 0) + LRU_Remove(old); + last_reference_list.push_back(old); + } + } + if (handle == nullptr) { + LRU_Insert(e); + } else { + *handle = reinterpret_cast(e); + } + s = Status::OK(); + } + } + + // we free the entries here outside of mutex for + // performance reasons + for (auto entry : last_reference_list) { + entry->Free(); + } + + return s; +} + +void LRUCacheShard::Erase(const Slice& key, uint32_t hash) { + LRUHandle* e; + bool last_reference = false; + { + MutexLock l(&mutex_); + e = table_.Remove(key, hash); + if (e != nullptr) { + last_reference = Unref(e); + if (last_reference) { + usage_ -= e->charge; + } + if (last_reference && e->InCache()) { + LRU_Remove(e); + } + e->SetInCache(false); + } + } + + // mutex not held here + // last_reference will only be true if e != nullptr + if (last_reference) { + e->Free(); + } +} + +size_t LRUCacheShard::GetUsage() const { + MutexLock l(&mutex_); + return usage_; +} + +size_t LRUCacheShard::GetPinnedUsage() const { + MutexLock l(&mutex_); + assert(usage_ >= lru_usage_); + return usage_ - lru_usage_; +} + +std::string LRUCacheShard::GetPrintableOptions() const { + const int kBufferSize = 200; + char buffer[kBufferSize]; + { + MutexLock l(&mutex_); + snprintf(buffer, kBufferSize, " high_pri_pool_ratio: %.3lf\n", + high_pri_pool_ratio_); + } + return std::string(buffer); +} + +LRUCache::LRUCache(size_t capacity, int num_shard_bits, + bool strict_capacity_limit, double high_pri_pool_ratio) + : ShardedCache(capacity, num_shard_bits, strict_capacity_limit) { + num_shards_ = 1 << num_shard_bits; + shards_ = new LRUCacheShard[num_shards_]; + SetCapacity(capacity); + SetStrictCapacityLimit(strict_capacity_limit); + for (int i = 0; i < num_shards_; i++) { + shards_[i].SetHighPriorityPoolRatio(high_pri_pool_ratio); + } +} + +LRUCache::~LRUCache() { delete[] shards_; } + +CacheShard* LRUCache::GetShard(int shard) { + return reinterpret_cast(&shards_[shard]); +} + +const CacheShard* LRUCache::GetShard(int shard) const { + return reinterpret_cast(&shards_[shard]); +} + +void* LRUCache::Value(Handle* handle) { + return reinterpret_cast(handle)->value; +} + +size_t LRUCache::GetCharge(Handle* handle) const { + return reinterpret_cast(handle)->charge; +} + +uint32_t LRUCache::GetHash(Handle* handle) const { + return reinterpret_cast(handle)->hash; +} + +void LRUCache::DisownData() { +// Do not drop data if compile with ASAN to suppress leak warning. +#ifndef __SANITIZE_ADDRESS__ + shards_ = nullptr; +#endif // !__SANITIZE_ADDRESS__ +} + +size_t LRUCache::TEST_GetLRUSize() { + size_t lru_size_of_all_shards = 0; + for (int i = 0; i < num_shards_; i++) { + lru_size_of_all_shards += shards_[i].TEST_GetLRUSize(); + } + return lru_size_of_all_shards; +} + +std::shared_ptr NewLRUCache(size_t capacity, int num_shard_bits, + bool strict_capacity_limit, + double high_pri_pool_ratio) { + if (num_shard_bits >= 20) { + return nullptr; // the cache cannot be sharded into too many fine pieces + } + if (high_pri_pool_ratio < 0.0 || high_pri_pool_ratio > 1.0) { + // invalid high_pri_pool_ratio + return nullptr; + } + if (num_shard_bits < 0) { + num_shard_bits = GetDefaultCacheShardBits(capacity); + } + return std::make_shared(capacity, num_shard_bits, + strict_capacity_limit, high_pri_pool_ratio); +} + +} // namespace rocksdb diff --git a/cache/lru_cache.h b/cache/lru_cache.h new file mode 100644 index 00000000000..abe78fd0c78 --- /dev/null +++ b/cache/lru_cache.h @@ -0,0 +1,302 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#pragma once + +#include + +#include "cache/sharded_cache.h" + +#include "port/port.h" +#include "util/autovector.h" + +namespace rocksdb { + +// LRU cache implementation + +// An entry is a variable length heap-allocated structure. +// Entries are referenced by cache and/or by any external entity. +// The cache keeps all its entries in table. Some elements +// are also stored on LRU list. +// +// LRUHandle can be in these states: +// 1. Referenced externally AND in hash table. +// In that case the entry is *not* in the LRU. (refs > 1 && in_cache == true) +// 2. Not referenced externally and in hash table. In that case the entry is +// in the LRU and can be freed. (refs == 1 && in_cache == true) +// 3. Referenced externally and not in hash table. In that case the entry is +// in not on LRU and not in table. (refs >= 1 && in_cache == false) +// +// All newly created LRUHandles are in state 1. If you call +// LRUCacheShard::Release +// on entry in state 1, it will go into state 2. To move from state 1 to +// state 3, either call LRUCacheShard::Erase or LRUCacheShard::Insert with the +// same key. +// To move from state 2 to state 1, use LRUCacheShard::Lookup. +// Before destruction, make sure that no handles are in state 1. This means +// that any successful LRUCacheShard::Lookup/LRUCacheShard::Insert have a +// matching +// RUCache::Release (to move into state 2) or LRUCacheShard::Erase (for state 3) + +struct LRUHandle { + void* value; + void (*deleter)(const Slice&, void* value); + LRUHandle* next_hash; + LRUHandle* next; + LRUHandle* prev; + size_t charge; // TODO(opt): Only allow uint32_t? + size_t key_length; + uint32_t refs; // a number of refs to this entry + // cache itself is counted as 1 + + // Include the following flags: + // in_cache: whether this entry is referenced by the hash table. + // is_high_pri: whether this entry is high priority entry. + // in_high_pro_pool: whether this entry is in high-pri pool. + char flags; + + uint32_t hash; // Hash of key(); used for fast sharding and comparisons + + char key_data[1]; // Beginning of key + + Slice key() const { + // For cheaper lookups, we allow a temporary Handle object + // to store a pointer to a key in "value". + if (next == this) { + return *(reinterpret_cast(value)); + } else { + return Slice(key_data, key_length); + } + } + + bool InCache() { return flags & 1; } + bool IsHighPri() { return flags & 2; } + bool InHighPriPool() { return flags & 4; } + + void SetInCache(bool in_cache) { + if (in_cache) { + flags |= 1; + } else { + flags &= ~1; + } + } + + void SetPriority(Cache::Priority priority) { + if (priority == Cache::Priority::HIGH) { + flags |= 2; + } else { + flags &= ~2; + } + } + + void SetInHighPriPool(bool in_high_pri_pool) { + if (in_high_pri_pool) { + flags |= 4; + } else { + flags &= ~4; + } + } + + void Free() { + assert((refs == 1 && InCache()) || (refs == 0 && !InCache())); + if (deleter) { + (*deleter)(key(), value); + } + delete[] reinterpret_cast(this); + } +}; + +// We provide our own simple hash table since it removes a whole bunch +// of porting hacks and is also faster than some of the built-in hash +// table implementations in some of the compiler/runtime combinations +// we have tested. E.g., readrandom speeds up by ~5% over the g++ +// 4.4.3's builtin hashtable. +class LRUHandleTable { + public: + LRUHandleTable(); + ~LRUHandleTable(); + + LRUHandle* Lookup(const Slice& key, uint32_t hash); + LRUHandle* Insert(LRUHandle* h); + LRUHandle* Remove(const Slice& key, uint32_t hash); + + template + void ApplyToAllCacheEntries(T func) { + for (uint32_t i = 0; i < length_; i++) { + LRUHandle* h = list_[i]; + while (h != nullptr) { + auto n = h->next_hash; + assert(h->InCache()); + func(h); + h = n; + } + } + } + + private: + // Return a pointer to slot that points to a cache entry that + // matches key/hash. If there is no such cache entry, return a + // pointer to the trailing slot in the corresponding linked list. + LRUHandle** FindPointer(const Slice& key, uint32_t hash); + + void Resize(); + + // The table consists of an array of buckets where each bucket is + // a linked list of cache entries that hash into the bucket. + LRUHandle** list_; + uint32_t length_; + uint32_t elems_; +}; + +// A single shard of sharded cache. +class ALIGN_AS(CACHE_LINE_SIZE) LRUCacheShard : public CacheShard { + public: + LRUCacheShard(); + virtual ~LRUCacheShard(); + + // Separate from constructor so caller can easily make an array of LRUCache + // if current usage is more than new capacity, the function will attempt to + // free the needed space + virtual void SetCapacity(size_t capacity) override; + + // Set the flag to reject insertion if cache if full. + virtual void SetStrictCapacityLimit(bool strict_capacity_limit) override; + + // Set percentage of capacity reserved for high-pri cache entries. + void SetHighPriorityPoolRatio(double high_pri_pool_ratio); + + // Like Cache methods, but with an extra "hash" parameter. + virtual Status Insert(const Slice& key, uint32_t hash, void* value, + size_t charge, + void (*deleter)(const Slice& key, void* value), + Cache::Handle** handle, + Cache::Priority priority) override; + virtual Cache::Handle* Lookup(const Slice& key, uint32_t hash) override; + virtual bool Ref(Cache::Handle* handle) override; + virtual bool Release(Cache::Handle* handle, + bool force_erase = false) override; + virtual void Erase(const Slice& key, uint32_t hash) override; + + // Although in some platforms the update of size_t is atomic, to make sure + // GetUsage() and GetPinnedUsage() work correctly under any platform, we'll + // protect them with mutex_. + + virtual size_t GetUsage() const override; + virtual size_t GetPinnedUsage() const override; + + virtual void ApplyToAllCacheEntries(void (*callback)(void*, size_t), + bool thread_safe) override; + + virtual void EraseUnRefEntries() override; + + virtual std::string GetPrintableOptions() const override; + + void TEST_GetLRUList(LRUHandle** lru, LRUHandle** lru_low_pri); + + // Retrieves number of elements in LRU, for unit test purpose only + // not threadsafe + size_t TEST_GetLRUSize(); + + // Overloading to aligned it to cache line size + void* operator new(size_t); + + void* operator new[](size_t); + + void operator delete(void *); + + void operator delete[](void*); + + private: + void LRU_Remove(LRUHandle* e); + void LRU_Insert(LRUHandle* e); + + // Overflow the last entry in high-pri pool to low-pri pool until size of + // high-pri pool is no larger than the size specify by high_pri_pool_pct. + void MaintainPoolSize(); + + // Just reduce the reference count by 1. + // Return true if last reference + bool Unref(LRUHandle* e); + + // Free some space following strict LRU policy until enough space + // to hold (usage_ + charge) is freed or the lru list is empty + // This function is not thread safe - it needs to be executed while + // holding the mutex_ + void EvictFromLRU(size_t charge, autovector* deleted); + + // Initialized before use. + size_t capacity_; + + // Memory size for entries in high-pri pool. + size_t high_pri_pool_usage_; + + // Whether to reject insertion if cache reaches its full capacity. + bool strict_capacity_limit_; + + // Ratio of capacity reserved for high priority cache entries. + double high_pri_pool_ratio_; + + // High-pri pool size, equals to capacity * high_pri_pool_ratio. + // Remember the value to avoid recomputing each time. + double high_pri_pool_capacity_; + + // Dummy head of LRU list. + // lru.prev is newest entry, lru.next is oldest entry. + // LRU contains items which can be evicted, ie reference only by cache + LRUHandle lru_; + + // Pointer to head of low-pri pool in LRU list. + LRUHandle* lru_low_pri_; + + // ------------^^^^^^^^^^^^^----------- + // Not frequently modified data members + // ------------------------------------ + // + // We separate data members that are updated frequently from the ones that + // are not frequently updated so that they don't share the same cache line + // which will lead into false cache sharing + // + // ------------------------------------ + // Frequently modified data members + // ------------vvvvvvvvvvvvv----------- + LRUHandleTable table_; + + // Memory size for entries residing in the cache + size_t usage_; + + // Memory size for entries residing only in the LRU list + size_t lru_usage_; + + // mutex_ protects the following state. + // We don't count mutex_ as the cache's internal state so semantically we + // don't mind mutex_ invoking the non-const actions. + mutable port::Mutex mutex_; +}; + +class LRUCache : public ShardedCache { + public: + LRUCache(size_t capacity, int num_shard_bits, bool strict_capacity_limit, + double high_pri_pool_ratio); + virtual ~LRUCache(); + virtual const char* Name() const override { return "LRUCache"; } + virtual CacheShard* GetShard(int shard) override; + virtual const CacheShard* GetShard(int shard) const override; + virtual void* Value(Handle* handle) override; + virtual size_t GetCharge(Handle* handle) const override; + virtual uint32_t GetHash(Handle* handle) const override; + virtual void DisownData() override; + + // Retrieves number of elements in LRU, for unit test purpose only + size_t TEST_GetLRUSize(); + + private: + LRUCacheShard* shards_; + int num_shards_ = 0; +}; + +} // namespace rocksdb diff --git a/cache/lru_cache_test.cc b/cache/lru_cache_test.cc new file mode 100644 index 00000000000..1b83033c36c --- /dev/null +++ b/cache/lru_cache_test.cc @@ -0,0 +1,172 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "cache/lru_cache.h" + +#include +#include +#include "util/testharness.h" + +namespace rocksdb { + +class LRUCacheTest : public testing::Test { + public: + LRUCacheTest() {} + ~LRUCacheTest() {} + + void NewCache(size_t capacity, double high_pri_pool_ratio = 0.0) { + cache_.reset( +#if defined(_MSC_VER) +#pragma warning(push) +#pragma warning(disable: 4316) // We've validated the alignment with the new operators +#endif + new LRUCacheShard() +#if defined(_MSC_VER) +#pragma warning(pop) +#endif + ); + cache_->SetCapacity(capacity); + cache_->SetStrictCapacityLimit(false); + cache_->SetHighPriorityPoolRatio(high_pri_pool_ratio); + } + + void Insert(const std::string& key, + Cache::Priority priority = Cache::Priority::LOW) { + cache_->Insert(key, 0 /*hash*/, nullptr /*value*/, 1 /*charge*/, + nullptr /*deleter*/, nullptr /*handle*/, priority); + } + + void Insert(char key, Cache::Priority priority = Cache::Priority::LOW) { + Insert(std::string(1, key), priority); + } + + bool Lookup(const std::string& key) { + auto handle = cache_->Lookup(key, 0 /*hash*/); + if (handle) { + cache_->Release(handle); + return true; + } + return false; + } + + bool Lookup(char key) { return Lookup(std::string(1, key)); } + + void Erase(const std::string& key) { cache_->Erase(key, 0 /*hash*/); } + + void ValidateLRUList(std::vector keys, + size_t num_high_pri_pool_keys = 0) { + LRUHandle* lru; + LRUHandle* lru_low_pri; + cache_->TEST_GetLRUList(&lru, &lru_low_pri); + LRUHandle* iter = lru; + bool in_high_pri_pool = false; + size_t high_pri_pool_keys = 0; + if (iter == lru_low_pri) { + in_high_pri_pool = true; + } + for (const auto& key : keys) { + iter = iter->next; + ASSERT_NE(lru, iter); + ASSERT_EQ(key, iter->key().ToString()); + ASSERT_EQ(in_high_pri_pool, iter->InHighPriPool()); + if (in_high_pri_pool) { + high_pri_pool_keys++; + } + if (iter == lru_low_pri) { + ASSERT_FALSE(in_high_pri_pool); + in_high_pri_pool = true; + } + } + ASSERT_EQ(lru, iter->next); + ASSERT_TRUE(in_high_pri_pool); + ASSERT_EQ(num_high_pri_pool_keys, high_pri_pool_keys); + } + + private: + std::unique_ptr cache_; +}; + +TEST_F(LRUCacheTest, BasicLRU) { + NewCache(5); + for (char ch = 'a'; ch <= 'e'; ch++) { + Insert(ch); + } + ValidateLRUList({"a", "b", "c", "d", "e"}); + for (char ch = 'x'; ch <= 'z'; ch++) { + Insert(ch); + } + ValidateLRUList({"d", "e", "x", "y", "z"}); + ASSERT_FALSE(Lookup("b")); + ValidateLRUList({"d", "e", "x", "y", "z"}); + ASSERT_TRUE(Lookup("e")); + ValidateLRUList({"d", "x", "y", "z", "e"}); + ASSERT_TRUE(Lookup("z")); + ValidateLRUList({"d", "x", "y", "e", "z"}); + Erase("x"); + ValidateLRUList({"d", "y", "e", "z"}); + ASSERT_TRUE(Lookup("d")); + ValidateLRUList({"y", "e", "z", "d"}); + Insert("u"); + ValidateLRUList({"y", "e", "z", "d", "u"}); + Insert("v"); + ValidateLRUList({"e", "z", "d", "u", "v"}); +} + +TEST_F(LRUCacheTest, MidPointInsertion) { + // Allocate 2 cache entries to high-pri pool. + NewCache(5, 0.45); + + Insert("a", Cache::Priority::LOW); + Insert("b", Cache::Priority::LOW); + Insert("c", Cache::Priority::LOW); + ValidateLRUList({"a", "b", "c"}, 0); + + // Low-pri entries can take high-pri pool capacity if available + Insert("u", Cache::Priority::LOW); + Insert("v", Cache::Priority::LOW); + ValidateLRUList({"a", "b", "c", "u", "v"}, 0); + + Insert("X", Cache::Priority::HIGH); + Insert("Y", Cache::Priority::HIGH); + ValidateLRUList({"c", "u", "v", "X", "Y"}, 2); + + // High-pri entries can overflow to low-pri pool. + Insert("Z", Cache::Priority::HIGH); + ValidateLRUList({"u", "v", "X", "Y", "Z"}, 2); + + // Low-pri entries will be inserted to head of low-pri pool. + Insert("a", Cache::Priority::LOW); + ValidateLRUList({"v", "X", "a", "Y", "Z"}, 2); + + // Low-pri entries will be inserted to head of low-pri pool after lookup. + ASSERT_TRUE(Lookup("v")); + ValidateLRUList({"X", "a", "v", "Y", "Z"}, 2); + + // High-pri entries will be inserted to the head of the list after lookup. + ASSERT_TRUE(Lookup("X")); + ValidateLRUList({"a", "v", "Y", "Z", "X"}, 2); + ASSERT_TRUE(Lookup("Z")); + ValidateLRUList({"a", "v", "Y", "X", "Z"}, 2); + + Erase("Y"); + ValidateLRUList({"a", "v", "X", "Z"}, 2); + Erase("X"); + ValidateLRUList({"a", "v", "Z"}, 1); + Insert("d", Cache::Priority::LOW); + Insert("e", Cache::Priority::LOW); + ValidateLRUList({"a", "v", "d", "e", "Z"}, 1); + Insert("f", Cache::Priority::LOW); + Insert("g", Cache::Priority::LOW); + ValidateLRUList({"d", "e", "f", "g", "Z"}, 1); + ASSERT_TRUE(Lookup("d")); + ValidateLRUList({"e", "f", "g", "d", "Z"}, 1); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/cache/sharded_cache.cc b/cache/sharded_cache.cc new file mode 100644 index 00000000000..9bdea3a08e1 --- /dev/null +++ b/cache/sharded_cache.cc @@ -0,0 +1,161 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include "cache/sharded_cache.h" + +#include + +#include "util/mutexlock.h" + +namespace rocksdb { + +ShardedCache::ShardedCache(size_t capacity, int num_shard_bits, + bool strict_capacity_limit) + : num_shard_bits_(num_shard_bits), + capacity_(capacity), + strict_capacity_limit_(strict_capacity_limit), + last_id_(1) {} + +void ShardedCache::SetCapacity(size_t capacity) { + int num_shards = 1 << num_shard_bits_; + const size_t per_shard = (capacity + (num_shards - 1)) / num_shards; + MutexLock l(&capacity_mutex_); + for (int s = 0; s < num_shards; s++) { + GetShard(s)->SetCapacity(per_shard); + } + capacity_ = capacity; +} + +void ShardedCache::SetStrictCapacityLimit(bool strict_capacity_limit) { + int num_shards = 1 << num_shard_bits_; + MutexLock l(&capacity_mutex_); + for (int s = 0; s < num_shards; s++) { + GetShard(s)->SetStrictCapacityLimit(strict_capacity_limit); + } + strict_capacity_limit_ = strict_capacity_limit; +} + +Status ShardedCache::Insert(const Slice& key, void* value, size_t charge, + void (*deleter)(const Slice& key, void* value), + Handle** handle, Priority priority) { + uint32_t hash = HashSlice(key); + return GetShard(Shard(hash)) + ->Insert(key, hash, value, charge, deleter, handle, priority); +} + +Cache::Handle* ShardedCache::Lookup(const Slice& key, Statistics* stats) { + uint32_t hash = HashSlice(key); + return GetShard(Shard(hash))->Lookup(key, hash); +} + +bool ShardedCache::Ref(Handle* handle) { + uint32_t hash = GetHash(handle); + return GetShard(Shard(hash))->Ref(handle); +} + +bool ShardedCache::Release(Handle* handle, bool force_erase) { + uint32_t hash = GetHash(handle); + return GetShard(Shard(hash))->Release(handle, force_erase); +} + +void ShardedCache::Erase(const Slice& key) { + uint32_t hash = HashSlice(key); + GetShard(Shard(hash))->Erase(key, hash); +} + +uint64_t ShardedCache::NewId() { + return last_id_.fetch_add(1, std::memory_order_relaxed); +} + +size_t ShardedCache::GetCapacity() const { + MutexLock l(&capacity_mutex_); + return capacity_; +} + +bool ShardedCache::HasStrictCapacityLimit() const { + MutexLock l(&capacity_mutex_); + return strict_capacity_limit_; +} + +size_t ShardedCache::GetUsage() const { + // We will not lock the cache when getting the usage from shards. + int num_shards = 1 << num_shard_bits_; + size_t usage = 0; + for (int s = 0; s < num_shards; s++) { + usage += GetShard(s)->GetUsage(); + } + return usage; +} + +size_t ShardedCache::GetUsage(Handle* handle) const { + return GetCharge(handle); +} + +size_t ShardedCache::GetPinnedUsage() const { + // We will not lock the cache when getting the usage from shards. + int num_shards = 1 << num_shard_bits_; + size_t usage = 0; + for (int s = 0; s < num_shards; s++) { + usage += GetShard(s)->GetPinnedUsage(); + } + return usage; +} + +void ShardedCache::ApplyToAllCacheEntries(void (*callback)(void*, size_t), + bool thread_safe) { + int num_shards = 1 << num_shard_bits_; + for (int s = 0; s < num_shards; s++) { + GetShard(s)->ApplyToAllCacheEntries(callback, thread_safe); + } +} + +void ShardedCache::EraseUnRefEntries() { + int num_shards = 1 << num_shard_bits_; + for (int s = 0; s < num_shards; s++) { + GetShard(s)->EraseUnRefEntries(); + } +} + +std::string ShardedCache::GetPrintableOptions() const { + std::string ret; + ret.reserve(20000); + const int kBufferSize = 200; + char buffer[kBufferSize]; + { + MutexLock l(&capacity_mutex_); + snprintf(buffer, kBufferSize, " capacity : %" ROCKSDB_PRIszt "\n", + capacity_); + ret.append(buffer); + snprintf(buffer, kBufferSize, " num_shard_bits : %d\n", num_shard_bits_); + ret.append(buffer); + snprintf(buffer, kBufferSize, " strict_capacity_limit : %d\n", + strict_capacity_limit_); + ret.append(buffer); + } + ret.append(GetShard(0)->GetPrintableOptions()); + return ret; +} +int GetDefaultCacheShardBits(size_t capacity) { + int num_shard_bits = 0; + size_t min_shard_size = 512L * 1024L; // Every shard is at least 512KB. + size_t num_shards = capacity / min_shard_size; + while (num_shards >>= 1) { + if (++num_shard_bits >= 6) { + // No more than 6. + return num_shard_bits; + } + } + return num_shard_bits; +} + +} // namespace rocksdb diff --git a/cache/sharded_cache.h b/cache/sharded_cache.h new file mode 100644 index 00000000000..4f9dea2ad0f --- /dev/null +++ b/cache/sharded_cache.h @@ -0,0 +1,102 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include +#include + +#include "port/port.h" +#include "rocksdb/cache.h" +#include "util/hash.h" + +namespace rocksdb { + +// Single cache shard interface. +class CacheShard { + public: + CacheShard() = default; + virtual ~CacheShard() = default; + + virtual Status Insert(const Slice& key, uint32_t hash, void* value, + size_t charge, + void (*deleter)(const Slice& key, void* value), + Cache::Handle** handle, Cache::Priority priority) = 0; + virtual Cache::Handle* Lookup(const Slice& key, uint32_t hash) = 0; + virtual bool Ref(Cache::Handle* handle) = 0; + virtual bool Release(Cache::Handle* handle, bool force_erase = false) = 0; + virtual void Erase(const Slice& key, uint32_t hash) = 0; + virtual void SetCapacity(size_t capacity) = 0; + virtual void SetStrictCapacityLimit(bool strict_capacity_limit) = 0; + virtual size_t GetUsage() const = 0; + virtual size_t GetPinnedUsage() const = 0; + virtual void ApplyToAllCacheEntries(void (*callback)(void*, size_t), + bool thread_safe) = 0; + virtual void EraseUnRefEntries() = 0; + virtual std::string GetPrintableOptions() const { return ""; } +}; + +// Generic cache interface which shards cache by hash of keys. 2^num_shard_bits +// shards will be created, with capacity split evenly to each of the shards. +// Keys are sharded by the highest num_shard_bits bits of hash value. +class ShardedCache : public Cache { + public: + ShardedCache(size_t capacity, int num_shard_bits, bool strict_capacity_limit); + virtual ~ShardedCache() = default; + virtual const char* Name() const override = 0; + virtual CacheShard* GetShard(int shard) = 0; + virtual const CacheShard* GetShard(int shard) const = 0; + virtual void* Value(Handle* handle) override = 0; + virtual size_t GetCharge(Handle* handle) const = 0; + virtual uint32_t GetHash(Handle* handle) const = 0; + virtual void DisownData() override = 0; + + virtual void SetCapacity(size_t capacity) override; + virtual void SetStrictCapacityLimit(bool strict_capacity_limit) override; + + virtual Status Insert(const Slice& key, void* value, size_t charge, + void (*deleter)(const Slice& key, void* value), + Handle** handle, Priority priority) override; + virtual Handle* Lookup(const Slice& key, Statistics* stats) override; + virtual bool Ref(Handle* handle) override; + virtual bool Release(Handle* handle, bool force_erase = false) override; + virtual void Erase(const Slice& key) override; + virtual uint64_t NewId() override; + virtual size_t GetCapacity() const override; + virtual bool HasStrictCapacityLimit() const override; + virtual size_t GetUsage() const override; + virtual size_t GetUsage(Handle* handle) const override; + virtual size_t GetPinnedUsage() const override; + virtual void ApplyToAllCacheEntries(void (*callback)(void*, size_t), + bool thread_safe) override; + virtual void EraseUnRefEntries() override; + virtual std::string GetPrintableOptions() const override; + + int GetNumShardBits() const { return num_shard_bits_; } + + private: + static inline uint32_t HashSlice(const Slice& s) { + return Hash(s.data(), s.size(), 0); + } + + uint32_t Shard(uint32_t hash) { + // Note, hash >> 32 yields hash in gcc, not the zero we expect! + return (num_shard_bits_ > 0) ? (hash >> (32 - num_shard_bits_)) : 0; + } + + int num_shard_bits_; + mutable port::Mutex capacity_mutex_; + size_t capacity_; + bool strict_capacity_limit_; + std::atomic last_id_; +}; + +extern int GetDefaultCacheShardBits(size_t capacity); + +} // namespace rocksdb diff --git a/cmake/RocksDBConfig.cmake.in b/cmake/RocksDBConfig.cmake.in new file mode 100644 index 00000000000..b3cb2b27adf --- /dev/null +++ b/cmake/RocksDBConfig.cmake.in @@ -0,0 +1,3 @@ +@PACKAGE_INIT@ +include("${CMAKE_CURRENT_LIST_DIR}/RocksDBTargets.cmake") +check_required_components(RocksDB) diff --git a/cmake/modules/FindJeMalloc.cmake b/cmake/modules/FindJeMalloc.cmake new file mode 100644 index 00000000000..7911f77c4c3 --- /dev/null +++ b/cmake/modules/FindJeMalloc.cmake @@ -0,0 +1,21 @@ +# - Find JeMalloc library +# Find the native JeMalloc includes and library +# +# JEMALLOC_INCLUDE_DIR - where to find jemalloc.h, etc. +# JEMALLOC_LIBRARIES - List of libraries when using jemalloc. +# JEMALLOC_FOUND - True if jemalloc found. + +find_path(JEMALLOC_INCLUDE_DIR + NAMES jemalloc/jemalloc.h + HINTS ${JEMALLOC_ROOT_DIR}/include) + +find_library(JEMALLOC_LIBRARIES + NAMES jemalloc + HINTS ${JEMALLOC_ROOT_DIR}/lib) + +include(FindPackageHandleStandardArgs) +find_package_handle_standard_args(jemalloc DEFAULT_MSG JEMALLOC_LIBRARIES JEMALLOC_INCLUDE_DIR) + +mark_as_advanced( + JEMALLOC_LIBRARIES + JEMALLOC_INCLUDE_DIR) diff --git a/cmake/modules/Findbzip2.cmake b/cmake/modules/Findbzip2.cmake new file mode 100644 index 00000000000..87abbe941e0 --- /dev/null +++ b/cmake/modules/Findbzip2.cmake @@ -0,0 +1,21 @@ +# - Find Bzip2 +# Find the bzip2 compression library and includes +# +# BZIP2_INCLUDE_DIR - where to find bzlib.h, etc. +# BZIP2_LIBRARIES - List of libraries when using bzip2. +# BZIP2_FOUND - True if bzip2 found. + +find_path(BZIP2_INCLUDE_DIR + NAMES bzlib.h + HINTS ${BZIP2_ROOT_DIR}/include) + +find_library(BZIP2_LIBRARIES + NAMES bz2 + HINTS ${BZIP2_ROOT_DIR}/lib) + +include(FindPackageHandleStandardArgs) +find_package_handle_standard_args(bzip2 DEFAULT_MSG BZIP2_LIBRARIES BZIP2_INCLUDE_DIR) + +mark_as_advanced( + BZIP2_LIBRARIES + BZIP2_INCLUDE_DIR) diff --git a/cmake/modules/Findlz4.cmake b/cmake/modules/Findlz4.cmake new file mode 100644 index 00000000000..c34acef5e39 --- /dev/null +++ b/cmake/modules/Findlz4.cmake @@ -0,0 +1,21 @@ +# - Find Lz4 +# Find the lz4 compression library and includes +# +# LZ4_INCLUDE_DIR - where to find lz4.h, etc. +# LZ4_LIBRARIES - List of libraries when using lz4. +# LZ4_FOUND - True if lz4 found. + +find_path(LZ4_INCLUDE_DIR + NAMES lz4.h + HINTS ${LZ4_ROOT_DIR}/include) + +find_library(LZ4_LIBRARIES + NAMES lz4 + HINTS ${LZ4_ROOT_DIR}/lib) + +include(FindPackageHandleStandardArgs) +find_package_handle_standard_args(lz4 DEFAULT_MSG LZ4_LIBRARIES LZ4_INCLUDE_DIR) + +mark_as_advanced( + LZ4_LIBRARIES + LZ4_INCLUDE_DIR) diff --git a/cmake/modules/Findsnappy.cmake b/cmake/modules/Findsnappy.cmake new file mode 100644 index 00000000000..6ed5fda3d57 --- /dev/null +++ b/cmake/modules/Findsnappy.cmake @@ -0,0 +1,21 @@ +# - Find Snappy +# Find the snappy compression library and includes +# +# SNAPPY_INCLUDE_DIR - where to find snappy.h, etc. +# SNAPPY_LIBRARIES - List of libraries when using snappy. +# SNAPPY_FOUND - True if snappy found. + +find_path(SNAPPY_INCLUDE_DIR + NAMES snappy.h + HINTS ${SNAPPY_ROOT_DIR}/include) + +find_library(SNAPPY_LIBRARIES + NAMES snappy + HINTS ${SNAPPY_ROOT_DIR}/lib) + +include(FindPackageHandleStandardArgs) +find_package_handle_standard_args(snappy DEFAULT_MSG SNAPPY_LIBRARIES SNAPPY_INCLUDE_DIR) + +mark_as_advanced( + SNAPPY_LIBRARIES + SNAPPY_INCLUDE_DIR) diff --git a/cmake/modules/Findzlib.cmake b/cmake/modules/Findzlib.cmake new file mode 100644 index 00000000000..fb5aee9b5aa --- /dev/null +++ b/cmake/modules/Findzlib.cmake @@ -0,0 +1,21 @@ +# - Find zlib +# Find the zlib compression library and includes +# +# ZLIB_INCLUDE_DIR - where to find zlib.h, etc. +# ZLIB_LIBRARIES - List of libraries when using zlib. +# ZLIB_FOUND - True if zlib found. + +find_path(ZLIB_INCLUDE_DIR + NAMES zlib.h + HINTS ${ZLIB_ROOT_DIR}/include) + +find_library(ZLIB_LIBRARIES + NAMES z + HINTS ${ZLIB_ROOT_DIR}/lib) + +include(FindPackageHandleStandardArgs) +find_package_handle_standard_args(zlib DEFAULT_MSG ZLIB_LIBRARIES ZLIB_INCLUDE_DIR) + +mark_as_advanced( + ZLIB_LIBRARIES + ZLIB_INCLUDE_DIR) diff --git a/cmake/modules/Findzstd.cmake b/cmake/modules/Findzstd.cmake new file mode 100644 index 00000000000..a2964aa9f80 --- /dev/null +++ b/cmake/modules/Findzstd.cmake @@ -0,0 +1,21 @@ +# - Find zstd +# Find the zstd compression library and includes +# +# ZSTD_INCLUDE_DIR - where to find zstd.h, etc. +# ZSTD_LIBRARIES - List of libraries when using zstd. +# ZSTD_FOUND - True if zstd found. + +find_path(ZSTD_INCLUDE_DIR + NAMES zstd.h + HINTS ${ZSTD_ROOT_DIR}/include) + +find_library(ZSTD_LIBRARIES + NAMES zstd + HINTS ${ZSTD_ROOT_DIR}/lib) + +include(FindPackageHandleStandardArgs) +find_package_handle_standard_args(zstd DEFAULT_MSG ZSTD_LIBRARIES ZSTD_INCLUDE_DIR) + +mark_as_advanced( + ZSTD_LIBRARIES + ZSTD_INCLUDE_DIR) diff --git a/coverage/coverage_test.sh b/coverage/coverage_test.sh index 08dbd05a594..6d87ae90867 100755 --- a/coverage/coverage_test.sh +++ b/coverage/coverage_test.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # Exit on error. set -e @@ -11,8 +11,8 @@ fi ROOT=".." # Fetch right version of gcov if [ -d /mnt/gvfs/third-party -a -z "$CXX" ]; then - source $ROOT/build_tools/fbcode.gcc471.sh - GCOV=$TOOLCHAIN_EXECUTABLES/gcc/gcc-4.7.1/cc6c9dc/bin/gcov + source $ROOT/build_tools/fbcode_config.sh + GCOV=$GCC_BASE/bin/gcov else GCOV=$(which gcov) fi diff --git a/db/builder.cc b/db/builder.cc index 1084f04138e..7cfa7800cce 100644 --- a/db/builder.cc +++ b/db/builder.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,212 +9,201 @@ #include "db/builder.h" +#include +#include +#include + +#include "db/compaction_iterator.h" #include "db/dbformat.h" -#include "db/filename.h" +#include "db/event_helpers.h" +#include "db/internal_stats.h" #include "db/merge_helper.h" #include "db/table_cache.h" #include "db/version_edit.h" +#include "monitoring/iostats_context_imp.h" +#include "monitoring/thread_status_util.h" #include "rocksdb/db.h" #include "rocksdb/env.h" #include "rocksdb/iterator.h" #include "rocksdb/options.h" #include "rocksdb/table.h" #include "table/block_based_table_builder.h" +#include "table/internal_iterator.h" +#include "util/file_reader_writer.h" +#include "util/filename.h" #include "util/stop_watch.h" +#include "util/sync_point.h" namespace rocksdb { class TableFactory; -TableBuilder* NewTableBuilder(const Options& options, - const InternalKeyComparator& internal_comparator, - WritableFile* file, - CompressionType compression_type) { - return options.table_factory->NewTableBuilder(options, internal_comparator, - file, compression_type); +TableBuilder* NewTableBuilder( + const ImmutableCFOptions& ioptions, + const InternalKeyComparator& internal_comparator, + const std::vector>* + int_tbl_prop_collector_factories, + uint32_t column_family_id, const std::string& column_family_name, + WritableFileWriter* file, const CompressionType compression_type, + const CompressionOptions& compression_opts, int level, + const std::string* compression_dict, const bool skip_filters, + const uint64_t creation_time, const uint64_t oldest_key_time) { + assert((column_family_id == + TablePropertiesCollectorFactory::Context::kUnknownColumnFamily) == + column_family_name.empty()); + return ioptions.table_factory->NewTableBuilder( + TableBuilderOptions( + ioptions, internal_comparator, int_tbl_prop_collector_factories, + compression_type, compression_opts, compression_dict, skip_filters, + column_family_name, level, creation_time, oldest_key_time), + column_family_id, file); } -Status BuildTable(const std::string& dbname, Env* env, const Options& options, - const EnvOptions& soptions, TableCache* table_cache, - Iterator* iter, FileMetaData* meta, - const InternalKeyComparator& internal_comparator, - const SequenceNumber newest_snapshot, - const SequenceNumber earliest_seqno_in_memtable, - const CompressionType compression, - const Env::IOPriority io_priority) { +Status BuildTable( + const std::string& dbname, Env* env, const ImmutableCFOptions& ioptions, + const MutableCFOptions& mutable_cf_options, const EnvOptions& env_options, + TableCache* table_cache, InternalIterator* iter, + std::unique_ptr range_del_iter, FileMetaData* meta, + const InternalKeyComparator& internal_comparator, + const std::vector>* + int_tbl_prop_collector_factories, + uint32_t column_family_id, const std::string& column_family_name, + std::vector snapshots, + SequenceNumber earliest_write_conflict_snapshot, + const CompressionType compression, + const CompressionOptions& compression_opts, bool paranoid_file_checks, + InternalStats* internal_stats, TableFileCreationReason reason, + EventLogger* event_logger, int job_id, const Env::IOPriority io_priority, + TableProperties* table_properties, int level, const uint64_t creation_time, + const uint64_t oldest_key_time) { + assert((column_family_id == + TablePropertiesCollectorFactory::Context::kUnknownColumnFamily) == + column_family_name.empty()); + // Reports the IOStats for flush for every following bytes. + const size_t kReportFlushIOStatsEvery = 1048576; Status s; meta->fd.file_size = 0; - meta->smallest_seqno = meta->largest_seqno = 0; iter->SeekToFirst(); - - // If the sequence number of the smallest entry in the memtable is - // smaller than the most recent snapshot, then we do not trigger - // removal of duplicate/deleted keys as part of this builder. - bool purge = options.purge_redundant_kvs_while_flush; - if (earliest_seqno_in_memtable <= newest_snapshot) { - purge = false; + std::unique_ptr range_del_agg( + new RangeDelAggregator(internal_comparator, snapshots)); + s = range_del_agg->AddTombstones(std::move(range_del_iter)); + if (!s.ok()) { + // may be non-ok if a range tombstone key is unparsable + return s; } - std::string fname = TableFileName(options.db_paths, meta->fd.GetNumber(), + std::string fname = TableFileName(ioptions.db_paths, meta->fd.GetNumber(), meta->fd.GetPathId()); - if (iter->Valid()) { - unique_ptr file; - s = env->NewWritableFile(fname, &file, soptions); - if (!s.ok()) { - return s; - } - file->SetIOPriority(io_priority); - - TableBuilder* builder = - NewTableBuilder(options, internal_comparator, file.get(), compression); - - // the first key is the smallest key - Slice key = iter->key(); - meta->smallest.DecodeFrom(key); - meta->smallest_seqno = GetInternalKeySeqno(key); - meta->largest_seqno = meta->smallest_seqno; - - MergeHelper merge(internal_comparator.user_comparator(), - options.merge_operator.get(), options.info_log.get(), - options.min_partial_merge_operands, - true /* internal key corruption is not ok */); - - if (purge) { - // Ugly walkaround to avoid compiler error for release build - bool ok __attribute__((unused)) = true; - - // Will write to builder if current key != prev key - ParsedInternalKey prev_ikey; - std::string prev_key; - bool is_first_key = true; // Also write if this is the very first key - - while (iter->Valid()) { - bool iterator_at_next = false; - - // Get current key - ParsedInternalKey this_ikey; - Slice key = iter->key(); - Slice value = iter->value(); - - // In-memory key corruption is not ok; - // TODO: find a clean way to treat in memory key corruption - ok = ParseInternalKey(key, &this_ikey); - assert(ok); - assert(this_ikey.sequence >= earliest_seqno_in_memtable); - - // If the key is the same as the previous key (and it is not the - // first key), then we skip it, since it is an older version. - // Otherwise we output the key and mark it as the "new" previous key. - if (!is_first_key && !internal_comparator.user_comparator()->Compare( - prev_ikey.user_key, this_ikey.user_key)) { - // seqno within the same key are in decreasing order - assert(this_ikey.sequence < prev_ikey.sequence); - } else { - is_first_key = false; - - if (this_ikey.type == kTypeMerge) { - // TODO(tbd): Add a check here to prevent RocksDB from crash when - // reopening a DB w/o properly specifying the merge operator. But - // currently we observed a memory leak on failing in RocksDB - // recovery, so we decide to let it crash instead of causing - // memory leak for now before we have identified the real cause - // of the memory leak. - - // Handle merge-type keys using the MergeHelper - // TODO: pass statistics to MergeUntil - merge.MergeUntil(iter, 0 /* don't worry about snapshot */); - iterator_at_next = true; - if (merge.IsSuccess()) { - // Merge completed correctly. - // Add the resulting merge key/value and continue to next - builder->Add(merge.key(), merge.value()); - prev_key.assign(merge.key().data(), merge.key().size()); - ok = ParseInternalKey(Slice(prev_key), &prev_ikey); - assert(ok); - } else { - // Merge did not find a Put/Delete. - // Can not compact these merges into a kValueType. - // Write them out one-by-one. (Proceed back() to front()) - const std::deque& keys = merge.keys(); - const std::deque& values = merge.values(); - assert(keys.size() == values.size() && keys.size() >= 1); - std::deque::const_reverse_iterator key_iter; - std::deque::const_reverse_iterator value_iter; - for (key_iter=keys.rbegin(), value_iter = values.rbegin(); - key_iter != keys.rend() && value_iter != values.rend(); - ++key_iter, ++value_iter) { - - builder->Add(Slice(*key_iter), Slice(*value_iter)); - } - - // Sanity check. Both iterators should end at the same time - assert(key_iter == keys.rend() && value_iter == values.rend()); - - prev_key.assign(keys.front()); - ok = ParseInternalKey(Slice(prev_key), &prev_ikey); - assert(ok); - } - } else { - // Handle Put/Delete-type keys by simply writing them - builder->Add(key, value); - prev_key.assign(key.data(), key.size()); - ok = ParseInternalKey(Slice(prev_key), &prev_ikey); - assert(ok); - } - } - - if (!iterator_at_next) iter->Next(); +#ifndef ROCKSDB_LITE + EventHelpers::NotifyTableFileCreationStarted( + ioptions.listeners, dbname, column_family_name, fname, job_id, reason); +#endif // !ROCKSDB_LITE + TableProperties tp; + + if (iter->Valid() || range_del_agg->ShouldAddTombstones()) { + TableBuilder* builder; + unique_ptr file_writer; + { + unique_ptr file; +#ifndef NDEBUG + bool use_direct_writes = env_options.use_direct_writes; + TEST_SYNC_POINT_CALLBACK("BuildTable:create_file", &use_direct_writes); +#endif // !NDEBUG + s = NewWritableFile(env, fname, &file, env_options); + if (!s.ok()) { + EventHelpers::LogAndNotifyTableFileCreationFinished( + event_logger, ioptions.listeners, dbname, column_family_name, fname, + job_id, meta->fd, tp, reason, s); + return s; } + file->SetIOPriority(io_priority); + + file_writer.reset(new WritableFileWriter(std::move(file), env_options, + ioptions.statistics)); + builder = NewTableBuilder( + ioptions, internal_comparator, int_tbl_prop_collector_factories, + column_family_id, column_family_name, file_writer.get(), compression, + compression_opts, level, nullptr /* compression_dict */, + false /* skip_filters */, creation_time, oldest_key_time); + } - // The last key is the largest key - meta->largest.DecodeFrom(Slice(prev_key)); - SequenceNumber seqno = GetInternalKeySeqno(Slice(prev_key)); - meta->smallest_seqno = std::min(meta->smallest_seqno, seqno); - meta->largest_seqno = std::max(meta->largest_seqno, seqno); - - } else { - for (; iter->Valid(); iter->Next()) { - Slice key = iter->key(); - meta->largest.DecodeFrom(key); - builder->Add(key, iter->value()); - SequenceNumber seqno = GetInternalKeySeqno(key); - meta->smallest_seqno = std::min(meta->smallest_seqno, seqno); - meta->largest_seqno = std::max(meta->largest_seqno, seqno); + MergeHelper merge(env, internal_comparator.user_comparator(), + ioptions.merge_operator, nullptr, ioptions.info_log, + true /* internal key corruption is not ok */, + snapshots.empty() ? 0 : snapshots.back()); + + CompactionIterator c_iter( + iter, internal_comparator.user_comparator(), &merge, kMaxSequenceNumber, + &snapshots, earliest_write_conflict_snapshot, env, + true /* internal key corruption is not ok */, range_del_agg.get()); + c_iter.SeekToFirst(); + for (; c_iter.Valid(); c_iter.Next()) { + const Slice& key = c_iter.key(); + const Slice& value = c_iter.value(); + builder->Add(key, value); + meta->UpdateBoundaries(key, c_iter.ikey().sequence); + + // TODO(noetzli): Update stats after flush, too. + if (io_priority == Env::IO_HIGH && + IOSTATS(bytes_written) >= kReportFlushIOStatsEvery) { + ThreadStatusUtil::SetThreadOperationProperty( + ThreadStatus::FLUSH_BYTES_WRITTEN, IOSTATS(bytes_written)); } } + // nullptr for table_{min,max} so all range tombstones will be flushed + range_del_agg->AddToBuilder(builder, nullptr /* lower_bound */, + nullptr /* upper_bound */, meta); // Finish and check for builder errors - if (s.ok()) { + bool empty = builder->NumEntries() == 0; + s = c_iter.status(); + if (!s.ok() || empty) { + builder->Abandon(); + } else { s = builder->Finish(); - if (s.ok()) { - meta->fd.file_size = builder->FileSize(); - assert(meta->fd.GetFileSize() > 0); + } + + if (s.ok() && !empty) { + uint64_t file_size = builder->FileSize(); + meta->fd.file_size = file_size; + meta->marked_for_compaction = builder->NeedCompact(); + assert(meta->fd.GetFileSize() > 0); + tp = builder->GetTableProperties(); + if (table_properties) { + *table_properties = tp; } - } else { - builder->Abandon(); } delete builder; // Finish and check for file errors - if (s.ok() && !options.disableDataSync) { - if (options.use_fsync) { - StopWatch sw(env, options.statistics.get(), TABLE_SYNC_MICROS); - s = file->Fsync(); - } else { - StopWatch sw(env, options.statistics.get(), TABLE_SYNC_MICROS); - s = file->Sync(); - } + if (s.ok() && !empty) { + StopWatch sw(env, ioptions.statistics, TABLE_SYNC_MICROS); + s = file_writer->Sync(ioptions.use_fsync); } - if (s.ok()) { - s = file->Close(); + if (s.ok() && !empty) { + s = file_writer->Close(); } - if (s.ok()) { + if (s.ok() && !empty) { // Verify that the table is usable - Iterator* it = table_cache->NewIterator(ReadOptions(), soptions, - internal_comparator, meta->fd); + // We set for_compaction to false and don't OptimizeForCompactionTableRead + // here because this is a special case after we finish the table building + // No matter whether use_direct_io_for_flush_and_compaction is true, + // we will regrad this verification as user reads since the goal is + // to cache it here for further user reads + std::unique_ptr it(table_cache->NewIterator( + ReadOptions(), env_options, internal_comparator, meta->fd, + nullptr /* range_del_agg */, nullptr, + (internal_stats == nullptr) ? nullptr + : internal_stats->GetFileReadHist(0), + false /* for_compaction */, nullptr /* arena */, + false /* skip_filter */, level)); s = it->status(); - delete it; + if (s.ok() && paranoid_file_checks) { + for (it->SeekToFirst(); it->Valid(); it->Next()) { + } + s = it->status(); + } } } @@ -223,11 +212,15 @@ Status BuildTable(const std::string& dbname, Env* env, const Options& options, s = iter->status(); } - if (s.ok() && meta->fd.GetFileSize() > 0) { - // Keep it - } else { + if (!s.ok() || meta->fd.GetFileSize() == 0) { env->DeleteFile(fname); } + + // Output to event logger and fire events. + EventHelpers::LogAndNotifyTableFileCreationFinished( + event_logger, ioptions.listeners, dbname, column_family_name, fname, + job_id, meta->fd, tp, reason, s); + return s; } diff --git a/db/builder.h b/db/builder.h index f57501abd13..5a5081c647e 100644 --- a/db/builder.h +++ b/db/builder.h @@ -1,16 +1,25 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #pragma once +#include +#include +#include +#include "db/table_properties_collector.h" +#include "options/cf_options.h" #include "rocksdb/comparator.h" #include "rocksdb/env.h" +#include "rocksdb/listener.h" +#include "rocksdb/options.h" #include "rocksdb/status.h" +#include "rocksdb/table_properties.h" #include "rocksdb/types.h" -#include "rocksdb/options.h" +#include "table/scoped_arena_iterator.h" +#include "util/event_logger.h" namespace rocksdb { @@ -23,25 +32,52 @@ class Iterator; class TableCache; class VersionEdit; class TableBuilder; -class WritableFile; +class WritableFileWriter; +class InternalStats; +class InternalIterator; -extern TableBuilder* NewTableBuilder( - const Options& options, const InternalKeyComparator& internal_comparator, - WritableFile* file, CompressionType compression_type); +// @param column_family_name Name of the column family that is also identified +// by column_family_id, or empty string if unknown. It must outlive the +// TableBuilder returned by this function. +// @param compression_dict Data for presetting the compression library's +// dictionary, or nullptr. +TableBuilder* NewTableBuilder( + const ImmutableCFOptions& options, + const InternalKeyComparator& internal_comparator, + const std::vector>* + int_tbl_prop_collector_factories, + uint32_t column_family_id, const std::string& column_family_name, + WritableFileWriter* file, const CompressionType compression_type, + const CompressionOptions& compression_opts, int level, + const std::string* compression_dict = nullptr, + const bool skip_filters = false, const uint64_t creation_time = 0, + const uint64_t oldest_key_time = 0); // Build a Table file from the contents of *iter. The generated file // will be named according to number specified in meta. On success, the rest of // *meta will be filled with metadata about the generated table. // If no data is present in *iter, meta->file_size will be set to // zero, and no Table file will be produced. -extern Status BuildTable(const std::string& dbname, Env* env, - const Options& options, const EnvOptions& soptions, - TableCache* table_cache, Iterator* iter, - FileMetaData* meta, - const InternalKeyComparator& internal_comparator, - const SequenceNumber newest_snapshot, - const SequenceNumber earliest_seqno_in_memtable, - const CompressionType compression, - const Env::IOPriority io_priority = Env::IO_HIGH); +// +// @param column_family_name Name of the column family that is also identified +// by column_family_id, or empty string if unknown. +extern Status BuildTable( + const std::string& dbname, Env* env, const ImmutableCFOptions& options, + const MutableCFOptions& mutable_cf_options, const EnvOptions& env_options, + TableCache* table_cache, InternalIterator* iter, + std::unique_ptr range_del_iter, FileMetaData* meta, + const InternalKeyComparator& internal_comparator, + const std::vector>* + int_tbl_prop_collector_factories, + uint32_t column_family_id, const std::string& column_family_name, + std::vector snapshots, + SequenceNumber earliest_write_conflict_snapshot, + const CompressionType compression, + const CompressionOptions& compression_opts, bool paranoid_file_checks, + InternalStats* internal_stats, TableFileCreationReason reason, + EventLogger* event_logger = nullptr, int job_id = 0, + const Env::IOPriority io_priority = Env::IO_HIGH, + TableProperties* table_properties = nullptr, int level = -1, + const uint64_t creation_time = 0, const uint64_t oldest_key_time = 0); } // namespace rocksdb diff --git a/db/c.cc b/db/c.cc index 3114f350042..cbfb8557d0d 100644 --- a/db/c.cc +++ b/db/c.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -12,10 +12,11 @@ #include "rocksdb/c.h" #include -#include +#include "port/port.h" #include "rocksdb/cache.h" #include "rocksdb/compaction_filter.h" #include "rocksdb/comparator.h" +#include "rocksdb/convenience.h" #include "rocksdb/db.h" #include "rocksdb/env.h" #include "rocksdb/filter_policy.h" @@ -29,60 +30,102 @@ #include "rocksdb/statistics.h" #include "rocksdb/slice_transform.h" #include "rocksdb/table.h" - +#include "rocksdb/rate_limiter.h" +#include "rocksdb/utilities/backupable_db.h" +#include "rocksdb/utilities/write_batch_with_index.h" +#include "utilities/merge_operators.h" +#include "rocksdb/utilities/transaction.h" +#include "rocksdb/utilities/transaction_db.h" +#include "rocksdb/utilities/optimistic_transaction_db.h" +#include "rocksdb/utilities/checkpoint.h" + +using rocksdb::BytewiseComparator; using rocksdb::Cache; using rocksdb::ColumnFamilyDescriptor; using rocksdb::ColumnFamilyHandle; using rocksdb::ColumnFamilyOptions; using rocksdb::CompactionFilter; using rocksdb::CompactionFilterFactory; -using rocksdb::CompactionFilterV2; -using rocksdb::CompactionFilterFactoryV2; using rocksdb::CompactionFilterContext; using rocksdb::CompactionOptionsFIFO; using rocksdb::Comparator; using rocksdb::CompressionType; +using rocksdb::WALRecoveryMode; using rocksdb::DB; using rocksdb::DBOptions; +using rocksdb::DbPath; using rocksdb::Env; +using rocksdb::EnvOptions; using rocksdb::InfoLogLevel; using rocksdb::FileLock; using rocksdb::FilterPolicy; using rocksdb::FlushOptions; +using rocksdb::IngestExternalFileOptions; using rocksdb::Iterator; using rocksdb::Logger; using rocksdb::MergeOperator; +using rocksdb::MergeOperators; using rocksdb::NewBloomFilterPolicy; using rocksdb::NewLRUCache; using rocksdb::Options; using rocksdb::BlockBasedTableOptions; +using rocksdb::CuckooTableOptions; using rocksdb::RandomAccessFile; using rocksdb::Range; using rocksdb::ReadOptions; using rocksdb::SequentialFile; using rocksdb::Slice; +using rocksdb::SliceParts; using rocksdb::SliceTransform; using rocksdb::Snapshot; +using rocksdb::SstFileWriter; using rocksdb::Status; using rocksdb::WritableFile; using rocksdb::WriteBatch; +using rocksdb::WriteBatchWithIndex; using rocksdb::WriteOptions; using rocksdb::LiveFileMetaData; +using rocksdb::BackupEngine; +using rocksdb::BackupableDBOptions; +using rocksdb::BackupInfo; +using rocksdb::RestoreOptions; +using rocksdb::CompactRangeOptions; +using rocksdb::RateLimiter; +using rocksdb::NewGenericRateLimiter; +using rocksdb::PinnableSlice; +using rocksdb::TransactionDBOptions; +using rocksdb::TransactionDB; +using rocksdb::TransactionOptions; +using rocksdb::OptimisticTransactionDB; +using rocksdb::OptimisticTransactionOptions; +using rocksdb::Transaction; +using rocksdb::Checkpoint; using std::shared_ptr; extern "C" { struct rocksdb_t { DB* rep; }; +struct rocksdb_backup_engine_t { BackupEngine* rep; }; +struct rocksdb_backup_engine_info_t { std::vector rep; }; +struct rocksdb_restore_options_t { RestoreOptions rep; }; struct rocksdb_iterator_t { Iterator* rep; }; struct rocksdb_writebatch_t { WriteBatch rep; }; +struct rocksdb_writebatch_wi_t { WriteBatchWithIndex* rep; }; struct rocksdb_snapshot_t { const Snapshot* rep; }; struct rocksdb_flushoptions_t { FlushOptions rep; }; struct rocksdb_fifo_compaction_options_t { CompactionOptionsFIFO rep; }; -struct rocksdb_readoptions_t { ReadOptions rep; }; +struct rocksdb_readoptions_t { + ReadOptions rep; + Slice upper_bound; // stack variable to set pointer to in ReadOptions +}; struct rocksdb_writeoptions_t { WriteOptions rep; }; struct rocksdb_options_t { Options rep; }; +struct rocksdb_compactoptions_t { + CompactRangeOptions rep; +}; struct rocksdb_block_based_table_options_t { BlockBasedTableOptions rep; }; +struct rocksdb_cuckoo_table_options_t { CuckooTableOptions rep; }; struct rocksdb_seqfile_t { SequentialFile* rep; }; struct rocksdb_randomfile_t { RandomAccessFile* rep; }; struct rocksdb_writablefile_t { WritableFile* rep; }; @@ -91,6 +134,34 @@ struct rocksdb_logger_t { shared_ptr rep; }; struct rocksdb_cache_t { shared_ptr rep; }; struct rocksdb_livefiles_t { std::vector rep; }; struct rocksdb_column_family_handle_t { ColumnFamilyHandle* rep; }; +struct rocksdb_envoptions_t { EnvOptions rep; }; +struct rocksdb_ingestexternalfileoptions_t { IngestExternalFileOptions rep; }; +struct rocksdb_sstfilewriter_t { SstFileWriter* rep; }; +struct rocksdb_ratelimiter_t { RateLimiter* rep; }; +struct rocksdb_pinnableslice_t { + PinnableSlice rep; +}; +struct rocksdb_transactiondb_options_t { + TransactionDBOptions rep; +}; +struct rocksdb_transactiondb_t { + TransactionDB* rep; +}; +struct rocksdb_transaction_options_t { + TransactionOptions rep; +}; +struct rocksdb_transaction_t { + Transaction* rep; +}; +struct rocksdb_checkpoint_t { + Checkpoint* rep; +}; +struct rocksdb_optimistictransactiondb_t { + OptimisticTransactionDB* rep; +}; +struct rocksdb_optimistictransaction_options_t { + OptimisticTransactionOptions rep; +}; struct rocksdb_compactionfiltercontext_t { CompactionFilter::Context rep; @@ -107,18 +178,16 @@ struct rocksdb_compactionfilter_t : public CompactionFilter { char** new_value, size_t *new_value_length, unsigned char* value_changed); const char* (*name_)(void*); + unsigned char ignore_snapshots_; virtual ~rocksdb_compactionfilter_t() { (*destructor_)(state_); } - virtual bool Filter( - int level, - const Slice& key, - const Slice& existing_value, - std::string* new_value, - bool* value_changed) const { - char* c_new_value = NULL; + virtual bool Filter(int level, const Slice& key, const Slice& existing_value, + std::string* new_value, + bool* value_changed) const override { + char* c_new_value = nullptr; size_t new_value_length = 0; unsigned char c_value_changed = 0; unsigned char result = (*filter_)( @@ -134,9 +203,9 @@ struct rocksdb_compactionfilter_t : public CompactionFilter { return result; } - virtual const char* Name() const { - return (*name_)(state_); - } + virtual const char* Name() const override { return (*name_)(state_); } + + virtual bool IgnoreSnapshots() const override { return ignore_snapshots_; } }; struct rocksdb_compactionfilterfactory_t : public CompactionFilterFactory { @@ -149,112 +218,14 @@ struct rocksdb_compactionfilterfactory_t : public CompactionFilterFactory { virtual ~rocksdb_compactionfilterfactory_t() { (*destructor_)(state_); } virtual std::unique_ptr CreateCompactionFilter( - const CompactionFilter::Context& context) { + const CompactionFilter::Context& context) override { rocksdb_compactionfiltercontext_t ccontext; ccontext.rep = context; CompactionFilter* cf = (*create_compaction_filter_)(state_, &ccontext); return std::unique_ptr(cf); } - virtual const char* Name() const { return (*name_)(state_); } -}; - -struct rocksdb_compactionfilterv2_t : public CompactionFilterV2 { - void* state_; - void (*destructor_)(void*); - const char* (*name_)(void*); - void (*filter_)(void*, int level, size_t num_keys, - const char* const* keys_list, const size_t* keys_list_sizes, - const char* const* existing_values_list, const size_t* existing_values_list_sizes, - char** new_values_list, size_t* new_values_list_sizes, - unsigned char* to_delete_list); - - virtual ~rocksdb_compactionfilterv2_t() { - (*destructor_)(state_); - } - - virtual const char* Name() const { - return (*name_)(state_); - } - - virtual std::vector Filter(int level, - const SliceVector& keys, - const SliceVector& existing_values, - std::vector* new_values, - std::vector* values_changed) const { - // Make a vector pointing to the underlying key data. - size_t num_keys = keys.size(); - std::vector keys_list(num_keys); - std::vector keys_list_sizes(num_keys); - for (size_t i = 0; i < num_keys; ++i) { - keys_list[i] = keys[i].data(); - keys_list_sizes[i] = keys[i].size(); - } - // Make a vector pointing to the underlying value data. - std::vector existing_values_list(num_keys); - std::vector existing_values_list_sizes(num_keys); - for (size_t i = 0; i < num_keys; ++i) { - existing_values_list[i] = existing_values[i].data(); - existing_values_list_sizes[i] = existing_values[i].size(); - } - // Make a vector which will accept newly-allocated char* arrays - // which we will take ownership of and assign to strings in new_values. - new_values->clear(); - std::vector new_values_list(num_keys); - std::vector new_values_list_sizes(num_keys); - // Resize values_changed to hold all keys. - values_changed->resize(num_keys); - // Make a vector for bools indicating a value should be deleted - // on compaction (true) or maintained (false). - std::vector to_delete_list(num_keys); - - (*filter_)( - state_, level, num_keys, &keys_list[0], &keys_list_sizes[0], - &existing_values_list[0], &existing_values_list_sizes[0], - &new_values_list[0], &new_values_list_sizes[0], &to_delete_list[0]); - - // Now, we transfer any changed values, setting values_changed and - // initializing new_values in the event a value changed. - std::vector to_delete(num_keys); - for (size_t i = 0; i < num_keys; ++i) { - to_delete[i] = to_delete_list[i]; - (*values_changed)[i] = new_values_list[i] != nullptr; - if ((*values_changed)[i]) { - new_values->push_back(std::string(new_values_list[i], new_values_list_sizes[i])); - free(new_values_list[i]); - } - } - return to_delete; - } -}; - -struct rocksdb_compactionfilterfactoryv2_t : public CompactionFilterFactoryV2 { - void* state_; - void (*destructor_)(void*); - const char* (*name_)(void*); - rocksdb_compactionfilterv2_t* (*create_compaction_filter_v2_)( - void* state, const rocksdb_compactionfiltercontext_t* context); - - rocksdb_compactionfilterfactoryv2_t(const SliceTransform* prefix_extractor) - : CompactionFilterFactoryV2(prefix_extractor) { - } - - virtual ~rocksdb_compactionfilterfactoryv2_t() { - (*destructor_)(state_); - } - - virtual const char* Name() const { - return (*name_)(state_); - } - - virtual std::unique_ptr CreateCompactionFilterV2( - const CompactionFilterContext& context) { - struct rocksdb_compactionfiltercontext_t c_context; - c_context.rep.is_full_compaction = context.is_full_compaction; - c_context.rep.is_manual_compaction = context.is_manual_compaction; - return std::unique_ptr( - (*create_compaction_filter_v2_)(state_, &c_context)); - } + virtual const char* Name() const override { return (*name_)(state_); } }; struct rocksdb_comparator_t : public Comparator { @@ -270,17 +241,16 @@ struct rocksdb_comparator_t : public Comparator { (*destructor_)(state_); } - virtual int Compare(const Slice& a, const Slice& b) const { + virtual int Compare(const Slice& a, const Slice& b) const override { return (*compare_)(state_, a.data(), a.size(), b.data(), b.size()); } - virtual const char* Name() const { - return (*name_)(state_); - } + virtual const char* Name() const override { return (*name_)(state_); } // No-ops since the C binding does not support key shortening methods. - virtual void FindShortestSeparator(std::string*, const Slice&) const { } - virtual void FindShortSuccessor(std::string* key) const { } + virtual void FindShortestSeparator(std::string*, + const Slice&) const override {} + virtual void FindShortSuccessor(std::string* key) const override {} }; struct rocksdb_filterpolicy_t : public FilterPolicy { @@ -304,11 +274,10 @@ struct rocksdb_filterpolicy_t : public FilterPolicy { (*destructor_)(state_); } - virtual const char* Name() const { - return (*name_)(state_); - } + virtual const char* Name() const override { return (*name_)(state_); } - virtual void CreateFilter(const Slice* keys, int n, std::string* dst) const { + virtual void CreateFilter(const Slice* keys, int n, + std::string* dst) const override { std::vector key_pointers(n); std::vector key_sizes(n); for (int i = 0; i < n; i++) { @@ -326,7 +295,8 @@ struct rocksdb_filterpolicy_t : public FilterPolicy { } } - virtual bool KeyMayMatch(const Slice& key, const Slice& filter) const { + virtual bool KeyMayMatch(const Slice& key, + const Slice& filter) const override { return (*key_match_)(state_, key.data(), key.size(), filter.data(), filter.size()); } @@ -355,42 +325,33 @@ struct rocksdb_mergeoperator_t : public MergeOperator { (*destructor_)(state_); } - virtual const char* Name() const { - return (*name_)(state_); - } - - virtual bool FullMerge( - const Slice& key, - const Slice* existing_value, - const std::deque& operand_list, - std::string* new_value, - Logger* logger) const { + virtual const char* Name() const override { return (*name_)(state_); } - size_t n = operand_list.size(); + virtual bool FullMergeV2(const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const override { + size_t n = merge_in.operand_list.size(); std::vector operand_pointers(n); std::vector operand_sizes(n); for (size_t i = 0; i < n; i++) { - Slice operand(operand_list[i]); + Slice operand(merge_in.operand_list[i]); operand_pointers[i] = operand.data(); operand_sizes[i] = operand.size(); } const char* existing_value_data = nullptr; size_t existing_value_len = 0; - if (existing_value != nullptr) { - existing_value_data = existing_value->data(); - existing_value_len = existing_value->size(); + if (merge_in.existing_value != nullptr) { + existing_value_data = merge_in.existing_value->data(); + existing_value_len = merge_in.existing_value->size(); } unsigned char success; size_t new_value_len; char* tmp_new_value = (*full_merge_)( - state_, - key.data(), key.size(), - existing_value_data, existing_value_len, - &operand_pointers[0], &operand_sizes[0], n, - &success, &new_value_len); - new_value->assign(tmp_new_value, new_value_len); + state_, merge_in.key.data(), merge_in.key.size(), existing_value_data, + existing_value_len, &operand_pointers[0], &operand_sizes[0], + static_cast(n), &success, &new_value_len); + merge_out->new_value.assign(tmp_new_value, new_value_len); if (delete_value_ != nullptr) { (*delete_value_)(state_, tmp_new_value, new_value_len); @@ -403,7 +364,8 @@ struct rocksdb_mergeoperator_t : public MergeOperator { virtual bool PartialMergeMulti(const Slice& key, const std::deque& operand_list, - std::string* new_value, Logger* logger) const { + std::string* new_value, + Logger* logger) const override { size_t operand_count = operand_list.size(); std::vector operand_pointers(operand_count); std::vector operand_sizes(operand_count); @@ -417,7 +379,7 @@ struct rocksdb_mergeoperator_t : public MergeOperator { size_t new_value_len; char* tmp_new_value = (*partial_merge_)( state_, key.data(), key.size(), &operand_pointers[0], &operand_sizes[0], - operand_count, &success, &new_value_len); + static_cast(operand_count), &success, &new_value_len); new_value->assign(tmp_new_value, new_value_len); if (delete_value_ != nullptr) { @@ -430,6 +392,10 @@ struct rocksdb_mergeoperator_t : public MergeOperator { } }; +struct rocksdb_dbpath_t { + DbPath rep; +}; + struct rocksdb_env_t { Env* rep; bool is_default; @@ -454,21 +420,19 @@ struct rocksdb_slicetransform_t : public SliceTransform { (*destructor_)(state_); } - virtual const char* Name() const { - return (*name_)(state_); - } + virtual const char* Name() const override { return (*name_)(state_); } - virtual Slice Transform(const Slice& src) const { + virtual Slice Transform(const Slice& src) const override { size_t len; char* dst = (*transform_)(state_, src.data(), src.size(), &len); return Slice(dst, len); } - virtual bool InDomain(const Slice& src) const { + virtual bool InDomain(const Slice& src) const override { return (*in_domain_)(state_, src.data(), src.size()); } - virtual bool InRange(const Slice& src) const { + virtual bool InRange(const Slice& src) const override { return (*in_range_)(state_, src.data(), src.size()); } }; @@ -485,6 +449,7 @@ static bool SaveError(char** errptr, const Status& s) { *errptr = strdup(s.ToString().c_str()); } else { // TODO(sanjay): Merge with existing error? + // This is a bug if *errptr is not created by malloc() free(*errptr); *errptr = strdup(s.ToString().c_str()); } @@ -524,11 +489,127 @@ rocksdb_t* rocksdb_open_for_read_only( return result; } +rocksdb_backup_engine_t* rocksdb_backup_engine_open( + const rocksdb_options_t* options, const char* path, char** errptr) { + BackupEngine* be; + if (SaveError(errptr, BackupEngine::Open(options->rep.env, + BackupableDBOptions(path, + nullptr, + true, + options->rep.info_log.get()), + &be))) { + return nullptr; + } + rocksdb_backup_engine_t* result = new rocksdb_backup_engine_t; + result->rep = be; + return result; +} + +void rocksdb_backup_engine_create_new_backup(rocksdb_backup_engine_t* be, + rocksdb_t* db, char** errptr) { + SaveError(errptr, be->rep->CreateNewBackup(db->rep)); +} + +void rocksdb_backup_engine_purge_old_backups(rocksdb_backup_engine_t* be, + uint32_t num_backups_to_keep, + char** errptr) { + SaveError(errptr, be->rep->PurgeOldBackups(num_backups_to_keep)); +} + +rocksdb_restore_options_t* rocksdb_restore_options_create() { + return new rocksdb_restore_options_t; +} + +void rocksdb_restore_options_destroy(rocksdb_restore_options_t* opt) { + delete opt; +} + +void rocksdb_restore_options_set_keep_log_files(rocksdb_restore_options_t* opt, + int v) { + opt->rep.keep_log_files = v; +} + +void rocksdb_backup_engine_restore_db_from_latest_backup( + rocksdb_backup_engine_t* be, const char* db_dir, const char* wal_dir, + const rocksdb_restore_options_t* restore_options, char** errptr) { + SaveError(errptr, be->rep->RestoreDBFromLatestBackup(std::string(db_dir), + std::string(wal_dir), + restore_options->rep)); +} + +const rocksdb_backup_engine_info_t* rocksdb_backup_engine_get_backup_info( + rocksdb_backup_engine_t* be) { + rocksdb_backup_engine_info_t* result = new rocksdb_backup_engine_info_t; + be->rep->GetBackupInfo(&result->rep); + return result; +} + +int rocksdb_backup_engine_info_count(const rocksdb_backup_engine_info_t* info) { + return static_cast(info->rep.size()); +} + +int64_t rocksdb_backup_engine_info_timestamp( + const rocksdb_backup_engine_info_t* info, int index) { + return info->rep[index].timestamp; +} + +uint32_t rocksdb_backup_engine_info_backup_id( + const rocksdb_backup_engine_info_t* info, int index) { + return info->rep[index].backup_id; +} + +uint64_t rocksdb_backup_engine_info_size( + const rocksdb_backup_engine_info_t* info, int index) { + return info->rep[index].size; +} + +uint32_t rocksdb_backup_engine_info_number_files( + const rocksdb_backup_engine_info_t* info, int index) { + return info->rep[index].number_files; +} + +void rocksdb_backup_engine_info_destroy( + const rocksdb_backup_engine_info_t* info) { + delete info; +} + +void rocksdb_backup_engine_close(rocksdb_backup_engine_t* be) { + delete be->rep; + delete be; +} + +rocksdb_checkpoint_t* rocksdb_checkpoint_object_create(rocksdb_t* db, + char** errptr) { + Checkpoint* checkpoint; + if (SaveError(errptr, Checkpoint::Create(db->rep, &checkpoint))) { + return nullptr; + } + rocksdb_checkpoint_t* result = new rocksdb_checkpoint_t; + result->rep = checkpoint; + return result; +} + +void rocksdb_checkpoint_create(rocksdb_checkpoint_t* checkpoint, + const char* checkpoint_dir, + uint64_t log_size_for_flush, char** errptr) { + SaveError(errptr, checkpoint->rep->CreateCheckpoint( + std::string(checkpoint_dir), log_size_for_flush)); +} + +void rocksdb_checkpoint_object_destroy(rocksdb_checkpoint_t* checkpoint) { + delete checkpoint->rep; + delete checkpoint; +} + void rocksdb_close(rocksdb_t* db) { delete db->rep; delete db; } +void rocksdb_options_set_uint64add_merge_operator(rocksdb_options_t* opt) { + opt->rep.merge_operator = rocksdb::MergeOperators::CreateUInt64AddOperator(); +} + rocksdb_t* rocksdb_open_column_families( const rocksdb_options_t* db_options, const char* name, @@ -757,6 +838,69 @@ char* rocksdb_get_cf( return result; } +void rocksdb_multi_get( + rocksdb_t* db, + const rocksdb_readoptions_t* options, + size_t num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, + char** values_list, size_t* values_list_sizes, + char** errs) { + std::vector keys(num_keys); + for (size_t i = 0; i < num_keys; i++) { + keys[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + std::vector values(num_keys); + std::vector statuses = db->rep->MultiGet(options->rep, keys, &values); + for (size_t i = 0; i < num_keys; i++) { + if (statuses[i].ok()) { + values_list[i] = CopyString(values[i]); + values_list_sizes[i] = values[i].size(); + errs[i] = nullptr; + } else { + values_list[i] = nullptr; + values_list_sizes[i] = 0; + if (!statuses[i].IsNotFound()) { + errs[i] = strdup(statuses[i].ToString().c_str()); + } else { + errs[i] = nullptr; + } + } + } +} + +void rocksdb_multi_get_cf( + rocksdb_t* db, + const rocksdb_readoptions_t* options, + const rocksdb_column_family_handle_t* const* column_families, + size_t num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, + char** values_list, size_t* values_list_sizes, + char** errs) { + std::vector keys(num_keys); + std::vector cfs(num_keys); + for (size_t i = 0; i < num_keys; i++) { + keys[i] = Slice(keys_list[i], keys_list_sizes[i]); + cfs[i] = column_families[i]->rep; + } + std::vector values(num_keys); + std::vector statuses = db->rep->MultiGet(options->rep, cfs, keys, &values); + for (size_t i = 0; i < num_keys; i++) { + if (statuses[i].ok()) { + values_list[i] = CopyString(values[i]); + values_list_sizes[i] = values[i].size(); + errs[i] = nullptr; + } else { + values_list[i] = nullptr; + values_list_sizes[i] = 0; + if (!statuses[i].IsNotFound()) { + errs[i] = strdup(statuses[i].ToString().c_str()); + } else { + errs[i] = nullptr; + } + } + } +} + rocksdb_iterator_t* rocksdb_create_iterator( rocksdb_t* db, const rocksdb_readoptions_t* options) { @@ -774,6 +918,31 @@ rocksdb_iterator_t* rocksdb_create_iterator_cf( return result; } +void rocksdb_create_iterators( + rocksdb_t *db, + rocksdb_readoptions_t* opts, + rocksdb_column_family_handle_t** column_families, + rocksdb_iterator_t** iterators, + size_t size, + char** errptr) { + std::vector column_families_vec; + for (size_t i = 0; i < size; i++) { + column_families_vec.push_back(column_families[i]->rep); + } + + std::vector res; + Status status = db->rep->NewIterators(opts->rep, column_families_vec, &res); + assert(res.size() == size); + if (SaveError(errptr, status)) { + return; + } + + for (size_t i = 0; i < size; i++) { + iterators[i] = new rocksdb_iterator_t; + iterators[i]->rep = res[i]; + } +} + const rocksdb_snapshot_t* rocksdb_create_snapshot( rocksdb_t* db) { rocksdb_snapshot_t* result = new rocksdb_snapshot_t; @@ -800,6 +969,17 @@ char* rocksdb_property_value( } } +int rocksdb_property_int( + rocksdb_t* db, + const char* propname, + uint64_t *out_val) { + if (db->rep->GetIntProperty(Slice(propname), out_val)) { + return 0; + } else { + return -1; + } +} + char* rocksdb_property_value_cf( rocksdb_t* db, rocksdb_column_family_handle_t* column_family, @@ -863,6 +1043,7 @@ void rocksdb_compact_range( const char* limit_key, size_t limit_key_len) { Slice a, b; db->rep->CompactRange( + CompactRangeOptions(), // Pass nullptr Slice if corresponding "const char*" is nullptr (start_key ? (a = Slice(start_key, start_key_len), &a) : nullptr), (limit_key ? (b = Slice(limit_key, limit_key_len), &b) : nullptr)); @@ -875,7 +1056,31 @@ void rocksdb_compact_range_cf( const char* limit_key, size_t limit_key_len) { Slice a, b; db->rep->CompactRange( - column_family->rep, + CompactRangeOptions(), column_family->rep, + // Pass nullptr Slice if corresponding "const char*" is nullptr + (start_key ? (a = Slice(start_key, start_key_len), &a) : nullptr), + (limit_key ? (b = Slice(limit_key, limit_key_len), &b) : nullptr)); +} + +void rocksdb_compact_range_opt(rocksdb_t* db, rocksdb_compactoptions_t* opt, + const char* start_key, size_t start_key_len, + const char* limit_key, size_t limit_key_len) { + Slice a, b; + db->rep->CompactRange( + opt->rep, + // Pass nullptr Slice if corresponding "const char*" is nullptr + (start_key ? (a = Slice(start_key, start_key_len), &a) : nullptr), + (limit_key ? (b = Slice(limit_key, limit_key_len), &b) : nullptr)); +} + +void rocksdb_compact_range_cf_opt(rocksdb_t* db, + rocksdb_column_family_handle_t* column_family, + rocksdb_compactoptions_t* opt, + const char* start_key, size_t start_key_len, + const char* limit_key, size_t limit_key_len) { + Slice a, b; + db->rep->CompactRange( + opt->rep, column_family->rep, // Pass nullptr Slice if corresponding "const char*" is nullptr (start_key ? (a = Slice(start_key, start_key_len), &a) : nullptr), (limit_key ? (b = Slice(limit_key, limit_key_len), &b) : nullptr)); @@ -936,6 +1141,11 @@ void rocksdb_iter_seek(rocksdb_iterator_t* iter, const char* k, size_t klen) { iter->rep->Seek(Slice(k, klen)); } +void rocksdb_iter_seek_for_prev(rocksdb_iterator_t* iter, const char* k, + size_t klen) { + iter->rep->SeekForPrev(Slice(k, klen)); +} + void rocksdb_iter_next(rocksdb_iterator_t* iter) { iter->rep->Next(); } @@ -998,6 +1208,43 @@ void rocksdb_writebatch_put_cf( b->rep.Put(column_family->rep, Slice(key, klen), Slice(val, vlen)); } +void rocksdb_writebatch_putv( + rocksdb_writebatch_t* b, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + std::vector value_slices(num_values); + for (int i = 0; i < num_values; i++) { + value_slices[i] = Slice(values_list[i], values_list_sizes[i]); + } + b->rep.Put(SliceParts(key_slices.data(), num_keys), + SliceParts(value_slices.data(), num_values)); +} + +void rocksdb_writebatch_putv_cf( + rocksdb_writebatch_t* b, + rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + std::vector value_slices(num_values); + for (int i = 0; i < num_values; i++) { + value_slices[i] = Slice(values_list[i], values_list_sizes[i]); + } + b->rep.Put(column_family->rep, SliceParts(key_slices.data(), num_keys), + SliceParts(value_slices.data(), num_values)); +} + void rocksdb_writebatch_merge( rocksdb_writebatch_t* b, const char* key, size_t klen, @@ -1013,6 +1260,43 @@ void rocksdb_writebatch_merge_cf( b->rep.Merge(column_family->rep, Slice(key, klen), Slice(val, vlen)); } +void rocksdb_writebatch_mergev( + rocksdb_writebatch_t* b, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + std::vector value_slices(num_values); + for (int i = 0; i < num_values; i++) { + value_slices[i] = Slice(values_list[i], values_list_sizes[i]); + } + b->rep.Merge(SliceParts(key_slices.data(), num_keys), + SliceParts(value_slices.data(), num_values)); +} + +void rocksdb_writebatch_mergev_cf( + rocksdb_writebatch_t* b, + rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + std::vector value_slices(num_values); + for (int i = 0; i < num_values; i++) { + value_slices[i] = Slice(values_list[i], values_list_sizes[i]); + } + b->rep.Merge(column_family->rep, SliceParts(key_slices.data(), num_keys), + SliceParts(value_slices.data(), num_values)); +} + void rocksdb_writebatch_delete( rocksdb_writebatch_t* b, const char* key, size_t klen) { @@ -1026,6 +1310,82 @@ void rocksdb_writebatch_delete_cf( b->rep.Delete(column_family->rep, Slice(key, klen)); } +void rocksdb_writebatch_deletev( + rocksdb_writebatch_t* b, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + b->rep.Delete(SliceParts(key_slices.data(), num_keys)); +} + +void rocksdb_writebatch_deletev_cf( + rocksdb_writebatch_t* b, + rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + b->rep.Delete(column_family->rep, SliceParts(key_slices.data(), num_keys)); +} + +void rocksdb_writebatch_delete_range(rocksdb_writebatch_t* b, + const char* start_key, + size_t start_key_len, const char* end_key, + size_t end_key_len) { + b->rep.DeleteRange(Slice(start_key, start_key_len), + Slice(end_key, end_key_len)); +} + +void rocksdb_writebatch_delete_range_cf( + rocksdb_writebatch_t* b, rocksdb_column_family_handle_t* column_family, + const char* start_key, size_t start_key_len, const char* end_key, + size_t end_key_len) { + b->rep.DeleteRange(column_family->rep, Slice(start_key, start_key_len), + Slice(end_key, end_key_len)); +} + +void rocksdb_writebatch_delete_rangev(rocksdb_writebatch_t* b, int num_keys, + const char* const* start_keys_list, + const size_t* start_keys_list_sizes, + const char* const* end_keys_list, + const size_t* end_keys_list_sizes) { + std::vector start_key_slices(num_keys); + std::vector end_key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + start_key_slices[i] = Slice(start_keys_list[i], start_keys_list_sizes[i]); + end_key_slices[i] = Slice(end_keys_list[i], end_keys_list_sizes[i]); + } + b->rep.DeleteRange(SliceParts(start_key_slices.data(), num_keys), + SliceParts(end_key_slices.data(), num_keys)); +} + +void rocksdb_writebatch_delete_rangev_cf( + rocksdb_writebatch_t* b, rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* start_keys_list, + const size_t* start_keys_list_sizes, const char* const* end_keys_list, + const size_t* end_keys_list_sizes) { + std::vector start_key_slices(num_keys); + std::vector end_key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + start_key_slices[i] = Slice(start_keys_list[i], start_keys_list_sizes[i]); + end_key_slices[i] = Slice(end_keys_list[i], end_keys_list_sizes[i]); + } + b->rep.DeleteRange(column_family->rep, + SliceParts(start_key_slices.data(), num_keys), + SliceParts(end_key_slices.data(), num_keys)); +} + +void rocksdb_writebatch_put_log_data( + rocksdb_writebatch_t* b, + const char* blob, size_t len) { + b->rep.PutLogData(Slice(blob, len)); +} + void rocksdb_writebatch_iterate( rocksdb_writebatch_t* b, void* state, @@ -1036,10 +1396,10 @@ void rocksdb_writebatch_iterate( void* state_; void (*put_)(void*, const char* k, size_t klen, const char* v, size_t vlen); void (*deleted_)(void*, const char* k, size_t klen); - virtual void Put(const Slice& key, const Slice& value) { + virtual void Put(const Slice& key, const Slice& value) override { (*put_)(state_, key.data(), key.size(), value.data(), value.size()); } - virtual void Delete(const Slice& key) { + virtual void Delete(const Slice& key) override { (*deleted_)(state_, key.data(), key.size()); } }; @@ -1055,80 +1415,542 @@ const char* rocksdb_writebatch_data(rocksdb_writebatch_t* b, size_t* size) { return b->rep.Data().c_str(); } -rocksdb_block_based_table_options_t* -rocksdb_block_based_options_create() { - return new rocksdb_block_based_table_options_t; +void rocksdb_writebatch_set_save_point(rocksdb_writebatch_t* b) { + b->rep.SetSavePoint(); } -void rocksdb_block_based_options_destroy( - rocksdb_block_based_table_options_t* options) { - delete options; +void rocksdb_writebatch_rollback_to_save_point(rocksdb_writebatch_t* b, + char** errptr) { + SaveError(errptr, b->rep.RollbackToSavePoint()); } -void rocksdb_block_based_options_set_block_size( - rocksdb_block_based_table_options_t* options, size_t block_size) { - options->rep.block_size = block_size; +void rocksdb_writebatch_pop_save_point(rocksdb_writebatch_t* b, char** errptr) { + SaveError(errptr, b->rep.PopSavePoint()); } -void rocksdb_block_based_options_set_block_size_deviation( - rocksdb_block_based_table_options_t* options, int block_size_deviation) { - options->rep.block_size_deviation = block_size_deviation; +rocksdb_writebatch_wi_t* rocksdb_writebatch_wi_create(size_t reserved_bytes, unsigned char overwrite_key) { + rocksdb_writebatch_wi_t* b = new rocksdb_writebatch_wi_t; + b->rep = new WriteBatchWithIndex(BytewiseComparator(), reserved_bytes, overwrite_key); + return b; } -void rocksdb_block_based_options_set_block_restart_interval( - rocksdb_block_based_table_options_t* options, int block_restart_interval) { - options->rep.block_restart_interval = block_restart_interval; +void rocksdb_writebatch_wi_destroy(rocksdb_writebatch_wi_t* b) { + if (b->rep) { + delete b->rep; + } + delete b; } -void rocksdb_block_based_options_set_filter_policy( - rocksdb_block_based_table_options_t* options, - rocksdb_filterpolicy_t* filter_policy) { - options->rep.filter_policy.reset(filter_policy); +void rocksdb_writebatch_wi_clear(rocksdb_writebatch_wi_t* b) { + b->rep->Clear(); } -void rocksdb_block_based_options_set_no_block_cache( - rocksdb_block_based_table_options_t* options, - unsigned char no_block_cache) { - options->rep.no_block_cache = no_block_cache; +int rocksdb_writebatch_wi_count(rocksdb_writebatch_wi_t* b) { + return b->rep->GetWriteBatch()->Count(); } -void rocksdb_block_based_options_set_block_cache( - rocksdb_block_based_table_options_t* options, - rocksdb_cache_t* block_cache) { - if (block_cache) { - options->rep.block_cache = block_cache->rep; +void rocksdb_writebatch_wi_put( + rocksdb_writebatch_wi_t* b, + const char* key, size_t klen, + const char* val, size_t vlen) { + b->rep->Put(Slice(key, klen), Slice(val, vlen)); +} + +void rocksdb_writebatch_wi_put_cf( + rocksdb_writebatch_wi_t* b, + rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, + const char* val, size_t vlen) { + b->rep->Put(column_family->rep, Slice(key, klen), Slice(val, vlen)); +} + +void rocksdb_writebatch_wi_putv( + rocksdb_writebatch_wi_t* b, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); } + std::vector value_slices(num_values); + for (int i = 0; i < num_values; i++) { + value_slices[i] = Slice(values_list[i], values_list_sizes[i]); + } + b->rep->Put(SliceParts(key_slices.data(), num_keys), + SliceParts(value_slices.data(), num_values)); } -void rocksdb_block_based_options_set_block_cache_compressed( - rocksdb_block_based_table_options_t* options, - rocksdb_cache_t* block_cache_compressed) { - if (block_cache_compressed) { - options->rep.block_cache_compressed = block_cache_compressed->rep; +void rocksdb_writebatch_wi_putv_cf( + rocksdb_writebatch_wi_t* b, + rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + std::vector value_slices(num_values); + for (int i = 0; i < num_values; i++) { + value_slices[i] = Slice(values_list[i], values_list_sizes[i]); } + b->rep->Put(column_family->rep, SliceParts(key_slices.data(), num_keys), + SliceParts(value_slices.data(), num_values)); } -void rocksdb_block_based_options_set_whole_key_filtering( - rocksdb_block_based_table_options_t* options, unsigned char v) { - options->rep.whole_key_filtering = v; +void rocksdb_writebatch_wi_merge( + rocksdb_writebatch_wi_t* b, + const char* key, size_t klen, + const char* val, size_t vlen) { + b->rep->Merge(Slice(key, klen), Slice(val, vlen)); } -void rocksdb_options_set_block_based_table_factory( - rocksdb_options_t *opt, - rocksdb_block_based_table_options_t* table_options) { - if (table_options) { - opt->rep.table_factory.reset( - rocksdb::NewBlockBasedTableFactory(table_options->rep)); +void rocksdb_writebatch_wi_merge_cf( + rocksdb_writebatch_wi_t* b, + rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, + const char* val, size_t vlen) { + b->rep->Merge(column_family->rep, Slice(key, klen), Slice(val, vlen)); +} + +void rocksdb_writebatch_wi_mergev( + rocksdb_writebatch_wi_t* b, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); } + std::vector value_slices(num_values); + for (int i = 0; i < num_values; i++) { + value_slices[i] = Slice(values_list[i], values_list_sizes[i]); + } + b->rep->Merge(SliceParts(key_slices.data(), num_keys), + SliceParts(value_slices.data(), num_values)); } +void rocksdb_writebatch_wi_mergev_cf( + rocksdb_writebatch_wi_t* b, + rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + std::vector value_slices(num_values); + for (int i = 0; i < num_values; i++) { + value_slices[i] = Slice(values_list[i], values_list_sizes[i]); + } + b->rep->Merge(column_family->rep, SliceParts(key_slices.data(), num_keys), + SliceParts(value_slices.data(), num_values)); +} -rocksdb_options_t* rocksdb_options_create() { - return new rocksdb_options_t; +void rocksdb_writebatch_wi_delete( + rocksdb_writebatch_wi_t* b, + const char* key, size_t klen) { + b->rep->Delete(Slice(key, klen)); } -void rocksdb_options_destroy(rocksdb_options_t* options) { - delete options; +void rocksdb_writebatch_wi_delete_cf( + rocksdb_writebatch_wi_t* b, + rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen) { + b->rep->Delete(column_family->rep, Slice(key, klen)); +} + +void rocksdb_writebatch_wi_deletev( + rocksdb_writebatch_wi_t* b, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + b->rep->Delete(SliceParts(key_slices.data(), num_keys)); +} + +void rocksdb_writebatch_wi_deletev_cf( + rocksdb_writebatch_wi_t* b, + rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes) { + std::vector key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + key_slices[i] = Slice(keys_list[i], keys_list_sizes[i]); + } + b->rep->Delete(column_family->rep, SliceParts(key_slices.data(), num_keys)); +} + +void rocksdb_writebatch_wi_delete_range(rocksdb_writebatch_wi_t* b, + const char* start_key, + size_t start_key_len, const char* end_key, + size_t end_key_len) { + b->rep->DeleteRange(Slice(start_key, start_key_len), + Slice(end_key, end_key_len)); +} + +void rocksdb_writebatch_wi_delete_range_cf( + rocksdb_writebatch_wi_t* b, rocksdb_column_family_handle_t* column_family, + const char* start_key, size_t start_key_len, const char* end_key, + size_t end_key_len) { + b->rep->DeleteRange(column_family->rep, Slice(start_key, start_key_len), + Slice(end_key, end_key_len)); +} + +void rocksdb_writebatch_wi_delete_rangev(rocksdb_writebatch_wi_t* b, int num_keys, + const char* const* start_keys_list, + const size_t* start_keys_list_sizes, + const char* const* end_keys_list, + const size_t* end_keys_list_sizes) { + std::vector start_key_slices(num_keys); + std::vector end_key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + start_key_slices[i] = Slice(start_keys_list[i], start_keys_list_sizes[i]); + end_key_slices[i] = Slice(end_keys_list[i], end_keys_list_sizes[i]); + } + b->rep->DeleteRange(SliceParts(start_key_slices.data(), num_keys), + SliceParts(end_key_slices.data(), num_keys)); +} + +void rocksdb_writebatch_wi_delete_rangev_cf( + rocksdb_writebatch_wi_t* b, rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* start_keys_list, + const size_t* start_keys_list_sizes, const char* const* end_keys_list, + const size_t* end_keys_list_sizes) { + std::vector start_key_slices(num_keys); + std::vector end_key_slices(num_keys); + for (int i = 0; i < num_keys; i++) { + start_key_slices[i] = Slice(start_keys_list[i], start_keys_list_sizes[i]); + end_key_slices[i] = Slice(end_keys_list[i], end_keys_list_sizes[i]); + } + b->rep->DeleteRange(column_family->rep, + SliceParts(start_key_slices.data(), num_keys), + SliceParts(end_key_slices.data(), num_keys)); +} + +void rocksdb_writebatch_wi_put_log_data( + rocksdb_writebatch_wi_t* b, + const char* blob, size_t len) { + b->rep->PutLogData(Slice(blob, len)); +} + +void rocksdb_writebatch_wi_iterate( + rocksdb_writebatch_wi_t* b, + void* state, + void (*put)(void*, const char* k, size_t klen, const char* v, size_t vlen), + void (*deleted)(void*, const char* k, size_t klen)) { + class H : public WriteBatch::Handler { + public: + void* state_; + void (*put_)(void*, const char* k, size_t klen, const char* v, size_t vlen); + void (*deleted_)(void*, const char* k, size_t klen); + virtual void Put(const Slice& key, const Slice& value) override { + (*put_)(state_, key.data(), key.size(), value.data(), value.size()); + } + virtual void Delete(const Slice& key) override { + (*deleted_)(state_, key.data(), key.size()); + } + }; + H handler; + handler.state_ = state; + handler.put_ = put; + handler.deleted_ = deleted; + b->rep->GetWriteBatch()->Iterate(&handler); +} + +const char* rocksdb_writebatch_wi_data(rocksdb_writebatch_wi_t* b, size_t* size) { + WriteBatch* wb = b->rep->GetWriteBatch(); + *size = wb->GetDataSize(); + return wb->Data().c_str(); +} + +void rocksdb_writebatch_wi_set_save_point(rocksdb_writebatch_wi_t* b) { + b->rep->SetSavePoint(); +} + +void rocksdb_writebatch_wi_rollback_to_save_point(rocksdb_writebatch_wi_t* b, + char** errptr) { + SaveError(errptr, b->rep->RollbackToSavePoint()); +} + +rocksdb_iterator_t* rocksdb_writebatch_wi_create_iterator_with_base( + rocksdb_writebatch_wi_t* wbwi, + rocksdb_iterator_t* base_iterator) { + rocksdb_iterator_t* result = new rocksdb_iterator_t; + result->rep = wbwi->rep->NewIteratorWithBase(base_iterator->rep); + delete base_iterator; + return result; +} + +rocksdb_iterator_t* rocksdb_writebatch_wi_create_iterator_with_base_cf( + rocksdb_writebatch_wi_t* wbwi, + rocksdb_iterator_t* base_iterator, + rocksdb_column_family_handle_t* column_family) { + rocksdb_iterator_t* result = new rocksdb_iterator_t; + result->rep = wbwi->rep->NewIteratorWithBase(column_family->rep, base_iterator->rep); + delete base_iterator; + return result; +} + +char* rocksdb_writebatch_wi_get_from_batch( + rocksdb_writebatch_wi_t* wbwi, + const rocksdb_options_t* options, + const char* key, size_t keylen, + size_t* vallen, + char** errptr) { + char* result = nullptr; + std::string tmp; + Status s = wbwi->rep->GetFromBatch(options->rep, Slice(key, keylen), &tmp); + if (s.ok()) { + *vallen = tmp.size(); + result = CopyString(tmp); + } else { + *vallen = 0; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + } + return result; +} + +char* rocksdb_writebatch_wi_get_from_batch_cf( + rocksdb_writebatch_wi_t* wbwi, + const rocksdb_options_t* options, + rocksdb_column_family_handle_t* column_family, + const char* key, size_t keylen, + size_t* vallen, + char** errptr) { + char* result = nullptr; + std::string tmp; + Status s = wbwi->rep->GetFromBatch(column_family->rep, options->rep, + Slice(key, keylen), &tmp); + if (s.ok()) { + *vallen = tmp.size(); + result = CopyString(tmp); + } else { + *vallen = 0; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + } + return result; +} + +char* rocksdb_writebatch_wi_get_from_batch_and_db( + rocksdb_writebatch_wi_t* wbwi, + rocksdb_t* db, + const rocksdb_readoptions_t* options, + const char* key, size_t keylen, + size_t* vallen, + char** errptr) { + char* result = nullptr; + std::string tmp; + Status s = wbwi->rep->GetFromBatchAndDB(db->rep, options->rep, Slice(key, keylen), &tmp); + if (s.ok()) { + *vallen = tmp.size(); + result = CopyString(tmp); + } else { + *vallen = 0; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + } + return result; +} + +char* rocksdb_writebatch_wi_get_from_batch_and_db_cf( + rocksdb_writebatch_wi_t* wbwi, + rocksdb_t* db, + const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* column_family, + const char* key, size_t keylen, + size_t* vallen, + char** errptr) { + char* result = nullptr; + std::string tmp; + Status s = wbwi->rep->GetFromBatchAndDB(db->rep, options->rep, column_family->rep, + Slice(key, keylen), &tmp); + if (s.ok()) { + *vallen = tmp.size(); + result = CopyString(tmp); + } else { + *vallen = 0; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + } + return result; +} + +void rocksdb_write_writebatch_wi( + rocksdb_t* db, + const rocksdb_writeoptions_t* options, + rocksdb_writebatch_wi_t* wbwi, + char** errptr) { + WriteBatch* wb = wbwi->rep->GetWriteBatch(); + SaveError(errptr, db->rep->Write(options->rep, wb)); +} + +rocksdb_block_based_table_options_t* +rocksdb_block_based_options_create() { + return new rocksdb_block_based_table_options_t; +} + +void rocksdb_block_based_options_destroy( + rocksdb_block_based_table_options_t* options) { + delete options; +} + +void rocksdb_block_based_options_set_block_size( + rocksdb_block_based_table_options_t* options, size_t block_size) { + options->rep.block_size = block_size; +} + +void rocksdb_block_based_options_set_block_size_deviation( + rocksdb_block_based_table_options_t* options, int block_size_deviation) { + options->rep.block_size_deviation = block_size_deviation; +} + +void rocksdb_block_based_options_set_block_restart_interval( + rocksdb_block_based_table_options_t* options, int block_restart_interval) { + options->rep.block_restart_interval = block_restart_interval; +} + +void rocksdb_block_based_options_set_filter_policy( + rocksdb_block_based_table_options_t* options, + rocksdb_filterpolicy_t* filter_policy) { + options->rep.filter_policy.reset(filter_policy); +} + +void rocksdb_block_based_options_set_no_block_cache( + rocksdb_block_based_table_options_t* options, + unsigned char no_block_cache) { + options->rep.no_block_cache = no_block_cache; +} + +void rocksdb_block_based_options_set_block_cache( + rocksdb_block_based_table_options_t* options, + rocksdb_cache_t* block_cache) { + if (block_cache) { + options->rep.block_cache = block_cache->rep; + } +} + +void rocksdb_block_based_options_set_block_cache_compressed( + rocksdb_block_based_table_options_t* options, + rocksdb_cache_t* block_cache_compressed) { + if (block_cache_compressed) { + options->rep.block_cache_compressed = block_cache_compressed->rep; + } +} + +void rocksdb_block_based_options_set_whole_key_filtering( + rocksdb_block_based_table_options_t* options, unsigned char v) { + options->rep.whole_key_filtering = v; +} + +void rocksdb_block_based_options_set_format_version( + rocksdb_block_based_table_options_t* options, int v) { + options->rep.format_version = v; +} + +void rocksdb_block_based_options_set_index_type( + rocksdb_block_based_table_options_t* options, int v) { + options->rep.index_type = static_cast(v); +} + +void rocksdb_block_based_options_set_hash_index_allow_collision( + rocksdb_block_based_table_options_t* options, unsigned char v) { + options->rep.hash_index_allow_collision = v; +} + +void rocksdb_block_based_options_set_cache_index_and_filter_blocks( + rocksdb_block_based_table_options_t* options, unsigned char v) { + options->rep.cache_index_and_filter_blocks = v; +} + +void rocksdb_block_based_options_set_pin_l0_filter_and_index_blocks_in_cache( + rocksdb_block_based_table_options_t* options, unsigned char v) { + options->rep.pin_l0_filter_and_index_blocks_in_cache = v; +} + +void rocksdb_options_set_block_based_table_factory( + rocksdb_options_t *opt, + rocksdb_block_based_table_options_t* table_options) { + if (table_options) { + opt->rep.table_factory.reset( + rocksdb::NewBlockBasedTableFactory(table_options->rep)); + } +} + + +rocksdb_cuckoo_table_options_t* +rocksdb_cuckoo_options_create() { + return new rocksdb_cuckoo_table_options_t; +} + +void rocksdb_cuckoo_options_destroy( + rocksdb_cuckoo_table_options_t* options) { + delete options; +} + +void rocksdb_cuckoo_options_set_hash_ratio( + rocksdb_cuckoo_table_options_t* options, double v) { + options->rep.hash_table_ratio = v; +} + +void rocksdb_cuckoo_options_set_max_search_depth( + rocksdb_cuckoo_table_options_t* options, uint32_t v) { + options->rep.max_search_depth = v; +} + +void rocksdb_cuckoo_options_set_cuckoo_block_size( + rocksdb_cuckoo_table_options_t* options, uint32_t v) { + options->rep.cuckoo_block_size = v; +} + +void rocksdb_cuckoo_options_set_identity_as_first_hash( + rocksdb_cuckoo_table_options_t* options, unsigned char v) { + options->rep.identity_as_first_hash = v; +} + +void rocksdb_cuckoo_options_set_use_module_hash( + rocksdb_cuckoo_table_options_t* options, unsigned char v) { + options->rep.use_module_hash = v; +} + +void rocksdb_options_set_cuckoo_table_factory( + rocksdb_options_t *opt, + rocksdb_cuckoo_table_options_t* table_options) { + if (table_options) { + opt->rep.table_factory.reset( + rocksdb::NewCuckooTableFactory(table_options->rep)); + } +} + +void rocksdb_set_options( + rocksdb_t* db, int count, const char* const keys[], const char* const values[], char** errptr) { + std::unordered_map options_map; + for (int i=0; irep->SetOptions(options_map)); + } + +rocksdb_options_t* rocksdb_options_create() { + return new rocksdb_options_t; +} + +void rocksdb_options_destroy(rocksdb_options_t* options) { + delete options; } void rocksdb_options_increase_parallelism( @@ -1163,6 +1985,11 @@ void rocksdb_options_set_compaction_filter_factory( std::shared_ptr(factory); } +void rocksdb_options_compaction_readahead_size( + rocksdb_options_t* opt, size_t s) { + opt->rep.compaction_readahead_size = s; +} + void rocksdb_options_set_comparator( rocksdb_options_t* opt, rocksdb_comparator_t* cmp) { @@ -1175,11 +2002,6 @@ void rocksdb_options_set_merge_operator( opt->rep.merge_operator = std::shared_ptr(merge_operator); } -void rocksdb_options_set_compaction_filter_factory_v2( - rocksdb_options_t* opt, - rocksdb_compactionfilterfactoryv2_t* compaction_filter_factory_v2) { - opt->rep.compaction_filter_factory_v2 = std::shared_ptr(compaction_filter_factory_v2); -} void rocksdb_options_set_create_if_missing( rocksdb_options_t* opt, unsigned char v) { @@ -1201,6 +2023,16 @@ void rocksdb_options_set_paranoid_checks( opt->rep.paranoid_checks = v; } +void rocksdb_options_set_db_paths(rocksdb_options_t* opt, + const rocksdb_dbpath_t** dbpath_values, + size_t num_paths) { + std::vector db_paths(num_paths); + for (size_t i = 0; i < num_paths; ++i) { + db_paths[i] = dbpath_values[i]->rep; + } + opt->rep.db_paths = db_paths; +} + void rocksdb_options_set_env(rocksdb_options_t* opt, rocksdb_env_t* env) { opt->rep.env = (env ? env->rep : nullptr); } @@ -1216,6 +2048,11 @@ void rocksdb_options_set_info_log_level( opt->rep.info_log_level = static_cast(v); } +void rocksdb_options_set_db_write_buffer_size(rocksdb_options_t* opt, + size_t s) { + opt->rep.db_write_buffer_size = s; +} + void rocksdb_options_set_write_buffer_size(rocksdb_options_t* opt, size_t s) { opt->rep.write_buffer_size = s; } @@ -1224,6 +2061,14 @@ void rocksdb_options_set_max_open_files(rocksdb_options_t* opt, int n) { opt->rep.max_open_files = n; } +void rocksdb_options_set_max_file_opening_threads(rocksdb_options_t* opt, int n) { + opt->rep.max_file_opening_threads = n; +} + +void rocksdb_options_set_max_total_wal_size(rocksdb_options_t* opt, uint64_t n) { + opt->rep.max_total_wal_size = n; +} + void rocksdb_options_set_target_file_size_base( rocksdb_options_t* opt, uint64_t n) { opt->rep.target_file_size_base = n; @@ -1239,19 +2084,19 @@ void rocksdb_options_set_max_bytes_for_level_base( opt->rep.max_bytes_for_level_base = n; } -void rocksdb_options_set_max_bytes_for_level_multiplier( - rocksdb_options_t* opt, int n) { - opt->rep.max_bytes_for_level_multiplier = n; +void rocksdb_options_set_level_compaction_dynamic_level_bytes( + rocksdb_options_t* opt, unsigned char v) { + opt->rep.level_compaction_dynamic_level_bytes = v; } -void rocksdb_options_set_expanded_compaction_factor( - rocksdb_options_t* opt, int n) { - opt->rep.expanded_compaction_factor = n; +void rocksdb_options_set_max_bytes_for_level_multiplier(rocksdb_options_t* opt, + double n) { + opt->rep.max_bytes_for_level_multiplier = n; } -void rocksdb_options_set_max_grandparent_overlap_factor( - rocksdb_options_t* opt, int n) { - opt->rep.max_grandparent_overlap_factor = n; +void rocksdb_options_set_max_compaction_bytes(rocksdb_options_t* opt, + uint64_t n) { + opt->rep.max_compaction_bytes = n; } void rocksdb_options_set_max_bytes_for_level_multiplier_additional( @@ -1266,6 +2111,10 @@ void rocksdb_options_enable_statistics(rocksdb_options_t* opt) { opt->rep.statistics = rocksdb::CreateDBStatistics(); } +void rocksdb_options_set_skip_stats_update_on_db_open(rocksdb_options_t* opt, unsigned char val) { + opt->rep.skip_stats_update_on_db_open = val; +} + void rocksdb_options_set_num_levels(rocksdb_options_t* opt, int n) { opt->rep.num_levels = n; } @@ -1285,9 +2134,11 @@ void rocksdb_options_set_level0_stop_writes_trigger( opt->rep.level0_stop_writes_trigger = n; } -void rocksdb_options_set_max_mem_compaction_level( - rocksdb_options_t* opt, int n) { - opt->rep.max_mem_compaction_level = n; +void rocksdb_options_set_max_mem_compaction_level(rocksdb_options_t* opt, + int n) {} + +void rocksdb_options_set_wal_recovery_mode(rocksdb_options_t* opt,int mode) { + opt->rep.wal_recovery_mode = static_cast(mode); } void rocksdb_options_set_compression(rocksdb_options_t* opt, int t) { @@ -1304,11 +2155,13 @@ void rocksdb_options_set_compression_per_level(rocksdb_options_t* opt, } } -void rocksdb_options_set_compression_options( - rocksdb_options_t* opt, int w_bits, int level, int strategy) { +void rocksdb_options_set_compression_options(rocksdb_options_t* opt, int w_bits, + int level, int strategy, + int max_dict_bytes) { opt->rep.compression_opts.window_bits = w_bits; opt->rep.compression_opts.level = level; opt->rep.compression_opts.strategy = strategy; + opt->rep.compression_opts.max_dict_bytes = max_dict_bytes; } void rocksdb_options_set_prefix_extractor( @@ -1316,11 +2169,6 @@ void rocksdb_options_set_prefix_extractor( opt->rep.prefix_extractor.reset(prefix_extractor); } -void rocksdb_options_set_disable_data_sync( - rocksdb_options_t* opt, int disable_data_sync) { - opt->rep.disableDataSync = disable_data_sync; -} - void rocksdb_options_set_use_fsync( rocksdb_options_t* opt, int use_fsync) { opt->rep.use_fsync = use_fsync; @@ -1350,14 +2198,18 @@ void rocksdb_options_set_manifest_preallocation_size( opt->rep.manifest_preallocation_size = v; } -void rocksdb_options_set_purge_redundant_kvs_while_flush( - rocksdb_options_t* opt, unsigned char v) { - opt->rep.purge_redundant_kvs_while_flush = v; +// noop +void rocksdb_options_set_purge_redundant_kvs_while_flush(rocksdb_options_t* opt, + unsigned char v) {} + +void rocksdb_options_set_use_direct_reads(rocksdb_options_t* opt, + unsigned char v) { + opt->rep.use_direct_reads = v; } -void rocksdb_options_set_allow_os_buffer( +void rocksdb_options_set_use_direct_io_for_flush_and_compaction( rocksdb_options_t* opt, unsigned char v) { - opt->rep.allow_os_buffer = v; + opt->rep.use_direct_io_for_flush_and_compaction = v; } void rocksdb_options_set_allow_mmap_reads( @@ -1418,14 +2270,14 @@ void rocksdb_options_set_bytes_per_sync( opt->rep.bytes_per_sync = v; } -void rocksdb_options_set_verify_checksums_in_compaction( - rocksdb_options_t* opt, unsigned char v) { - opt->rep.verify_checksums_in_compaction = v; +void rocksdb_options_set_allow_concurrent_memtable_write(rocksdb_options_t* opt, + unsigned char v) { + opt->rep.allow_concurrent_memtable_write = v; } -void rocksdb_options_set_filter_deletes( +void rocksdb_options_set_enable_write_thread_adaptive_yield( rocksdb_options_t* opt, unsigned char v) { - opt->rep.filter_deletes = v; + opt->rep.enable_write_thread_adaptive_yield = v; } void rocksdb_options_set_max_sequential_skip_in_iterations( @@ -1441,10 +2293,20 @@ void rocksdb_options_set_min_write_buffer_number_to_merge(rocksdb_options_t* opt opt->rep.min_write_buffer_number_to_merge = n; } +void rocksdb_options_set_max_write_buffer_number_to_maintain( + rocksdb_options_t* opt, int n) { + opt->rep.max_write_buffer_number_to_maintain = n; +} + void rocksdb_options_set_max_background_compactions(rocksdb_options_t* opt, int n) { opt->rep.max_background_compactions = n; } +void rocksdb_options_set_base_background_compactions(rocksdb_options_t* opt, + int n) { + opt->rep.base_background_compactions = n; +} + void rocksdb_options_set_max_background_flushes(rocksdb_options_t* opt, int n) { opt->rep.max_background_flushes = n; } @@ -1461,6 +2323,11 @@ void rocksdb_options_set_keep_log_file_num(rocksdb_options_t* opt, size_t v) { opt->rep.keep_log_file_num = v; } +void rocksdb_options_set_recycle_log_file_num(rocksdb_options_t* opt, + size_t v) { + opt->rep.recycle_log_file_num = v; +} + void rocksdb_options_set_soft_rate_limit(rocksdb_options_t* opt, double v) { opt->rep.soft_rate_limit = v; } @@ -1469,6 +2336,14 @@ void rocksdb_options_set_hard_rate_limit(rocksdb_options_t* opt, double v) { opt->rep.hard_rate_limit = v; } +void rocksdb_options_set_soft_pending_compaction_bytes_limit(rocksdb_options_t* opt, size_t v) { + opt->rep.soft_pending_compaction_bytes_limit = v; +} + +void rocksdb_options_set_hard_pending_compaction_bytes_limit(rocksdb_options_t* opt, size_t v) { + opt->rep.hard_pending_compaction_bytes_limit = v; +} + void rocksdb_options_set_rate_limit_delay_max_milliseconds( rocksdb_options_t* opt, unsigned int v) { opt->rep.rate_limit_delay_max_milliseconds = v; @@ -1486,7 +2361,7 @@ void rocksdb_options_set_table_cache_numshardbits( void rocksdb_options_set_table_cache_remove_scan_count_limit( rocksdb_options_t* opt, int v) { - opt->rep.table_cache_remove_scan_count_limit = v; + // this option is deprecated } void rocksdb_options_set_arena_block_size( @@ -1498,71 +2373,56 @@ void rocksdb_options_set_disable_auto_compactions(rocksdb_options_t* opt, int di opt->rep.disable_auto_compactions = disable; } +void rocksdb_options_set_optimize_filters_for_hits(rocksdb_options_t* opt, int v) { + opt->rep.optimize_filters_for_hits = v; +} + void rocksdb_options_set_delete_obsolete_files_period_micros( rocksdb_options_t* opt, uint64_t v) { opt->rep.delete_obsolete_files_period_micros = v; } -void rocksdb_options_set_source_compaction_factor( - rocksdb_options_t* opt, int n) { - opt->rep.expanded_compaction_factor = n; -} - void rocksdb_options_prepare_for_bulk_load(rocksdb_options_t* opt) { opt->rep.PrepareForBulkLoad(); } void rocksdb_options_set_memtable_vector_rep(rocksdb_options_t *opt) { - static rocksdb::VectorRepFactory* factory = 0; - if (!factory) { - factory = new rocksdb::VectorRepFactory; - } - opt->rep.memtable_factory.reset(factory); + opt->rep.memtable_factory.reset(new rocksdb::VectorRepFactory); } -void rocksdb_options_set_memtable_prefix_bloom_bits( - rocksdb_options_t* opt, uint32_t v) { - opt->rep.memtable_prefix_bloom_bits = v; +void rocksdb_options_set_memtable_prefix_bloom_size_ratio( + rocksdb_options_t* opt, double v) { + opt->rep.memtable_prefix_bloom_size_ratio = v; } -void rocksdb_options_set_memtable_prefix_bloom_probes( - rocksdb_options_t* opt, uint32_t v) { - opt->rep.memtable_prefix_bloom_probes = v; +void rocksdb_options_set_memtable_huge_page_size(rocksdb_options_t* opt, + size_t v) { + opt->rep.memtable_huge_page_size = v; } void rocksdb_options_set_hash_skip_list_rep( rocksdb_options_t *opt, size_t bucket_count, int32_t skiplist_height, int32_t skiplist_branching_factor) { - static rocksdb::MemTableRepFactory* factory = 0; - if (!factory) { - factory = rocksdb::NewHashSkipListRepFactory( - bucket_count, skiplist_height, skiplist_branching_factor); - } + rocksdb::MemTableRepFactory* factory = rocksdb::NewHashSkipListRepFactory( + bucket_count, skiplist_height, skiplist_branching_factor); opt->rep.memtable_factory.reset(factory); } void rocksdb_options_set_hash_link_list_rep( rocksdb_options_t *opt, size_t bucket_count) { - static rocksdb::MemTableRepFactory* factory = 0; - if (!factory) { - factory = rocksdb::NewHashLinkListRepFactory(bucket_count); - } - opt->rep.memtable_factory.reset(factory); + opt->rep.memtable_factory.reset(rocksdb::NewHashLinkListRepFactory(bucket_count)); } void rocksdb_options_set_plain_table_factory( rocksdb_options_t *opt, uint32_t user_key_len, int bloom_bits_per_key, double hash_table_ratio, size_t index_sparseness) { - static rocksdb::TableFactory* factory = 0; - if (!factory) { - rocksdb::PlainTableOptions options; - options.user_key_len = user_key_len; - options.bloom_bits_per_key = bloom_bits_per_key; - options.hash_table_ratio = hash_table_ratio; - options.index_sparseness = index_sparseness; + rocksdb::PlainTableOptions options; + options.user_key_len = user_key_len; + options.bloom_bits_per_key = bloom_bits_per_key; + options.hash_table_ratio = hash_table_ratio; + options.index_sparseness = index_sparseness; - factory = rocksdb::NewPlainTableFactory(options); - } + rocksdb::TableFactory* factory = rocksdb::NewPlainTableFactory(options); opt->rep.table_factory.reset(factory); } @@ -1571,21 +2431,11 @@ void rocksdb_options_set_max_successive_merges( opt->rep.max_successive_merges = v; } -void rocksdb_options_set_min_partial_merge_operands( - rocksdb_options_t* opt, uint32_t v) { - opt->rep.min_partial_merge_operands = v; -} - void rocksdb_options_set_bloom_locality( rocksdb_options_t* opt, uint32_t v) { opt->rep.bloom_locality = v; } -void rocksdb_options_set_allow_thread_local( - rocksdb_options_t* opt, unsigned char v) { - opt->rep.allow_thread_local = v; -} - void rocksdb_options_set_inplace_update_support( rocksdb_options_t* opt, unsigned char v) { opt->rep.inplace_update_support = v; @@ -1596,6 +2446,11 @@ void rocksdb_options_set_inplace_update_num_locks( opt->rep.inplace_update_num_locks = v; } +void rocksdb_options_set_report_bg_io_stats( + rocksdb_options_t* opt, int v) { + opt->rep.report_bg_io_stats = v; +} + void rocksdb_options_set_compaction_style(rocksdb_options_t *opt, int style) { opt->rep.compaction_style = static_cast(style); } @@ -1610,10 +2465,39 @@ void rocksdb_options_set_fifo_compaction_options( opt->rep.compaction_options_fifo = fifo->rep; } +char *rocksdb_options_statistics_get_string(rocksdb_options_t *opt) { + rocksdb::Statistics *statistics = opt->rep.statistics.get(); + if (statistics) { + return strdup(statistics->ToString().c_str()); + } + return nullptr; +} + +void rocksdb_options_set_ratelimiter(rocksdb_options_t *opt, rocksdb_ratelimiter_t *limiter) { + opt->rep.rate_limiter.reset(limiter->rep); + limiter->rep = nullptr; +} + +rocksdb_ratelimiter_t* rocksdb_ratelimiter_create( + int64_t rate_bytes_per_sec, + int64_t refill_period_us, + int32_t fairness) { + rocksdb_ratelimiter_t* rate_limiter = new rocksdb_ratelimiter_t; + rate_limiter->rep = NewGenericRateLimiter(rate_bytes_per_sec, + refill_period_us, fairness); + return rate_limiter; +} + +void rocksdb_ratelimiter_destroy(rocksdb_ratelimiter_t *limiter) { + if (limiter->rep) { + delete limiter->rep; + } + delete limiter; +} + /* TODO: DB::OpenForReadOnly -DB::MultiGet DB::KeyMayExist DB::GetOptions DB::GetSortedWalFiles @@ -1640,10 +2524,17 @@ rocksdb_compactionfilter_t* rocksdb_compactionfilter_create( result->state_ = state; result->destructor_ = destructor; result->filter_ = filter; + result->ignore_snapshots_ = false; result->name_ = name; return result; } +void rocksdb_compactionfilter_set_ignore_snapshots( + rocksdb_compactionfilter_t* filter, + unsigned char whether_ignore) { + filter->ignore_snapshots_ = whether_ignore; +} + void rocksdb_compactionfilter_destroy(rocksdb_compactionfilter_t* filter) { delete filter; } @@ -1677,46 +2568,6 @@ void rocksdb_compactionfilterfactory_destroy( delete factory; } -rocksdb_compactionfilterv2_t* rocksdb_compactionfilterv2_create( - void* state, - void (*destructor)(void*), - void (*filter)(void*, int level, size_t num_keys, - const char* const* keys_list, const size_t* keys_list_sizes, - const char* const* existing_values_list, const size_t* existing_values_list_sizes, - char** new_values_list, size_t* new_values_list_sizes, - unsigned char* to_delete_list), - const char* (*name)(void*)) { - rocksdb_compactionfilterv2_t* result = new rocksdb_compactionfilterv2_t; - result->state_ = state; - result->destructor_ = destructor; - result->filter_ = filter; - result->name_ = name; - return result; -} - -void rocksdb_compactionfilterv2_destroy(rocksdb_compactionfilterv2_t* filter) { - delete filter; -} - -rocksdb_compactionfilterfactoryv2_t* rocksdb_compactionfilterfactoryv2_create( - void* state, - rocksdb_slicetransform_t* prefix_extractor, - void (*destructor)(void*), - rocksdb_compactionfilterv2_t* (*create_compaction_filter_v2)( - void* state, const rocksdb_compactionfiltercontext_t* context), - const char* (*name)(void*)) { - rocksdb_compactionfilterfactoryv2_t* result = new rocksdb_compactionfilterfactoryv2_t(prefix_extractor); - result->state_ = state; - result->destructor_ = destructor; - result->create_compaction_filter_v2_ = create_compaction_filter_v2; - result->name_ = name; - return result; -} - -void rocksdb_compactionfilterfactoryv2_destroy(rocksdb_compactionfilterfactoryv2_t* factory) { - delete factory; -} - rocksdb_comparator_t* rocksdb_comparator_create( void* state, void (*destructor)(void*), @@ -1767,30 +2618,39 @@ void rocksdb_filterpolicy_destroy(rocksdb_filterpolicy_t* filter) { delete filter; } -rocksdb_filterpolicy_t* rocksdb_filterpolicy_create_bloom(int bits_per_key) { +rocksdb_filterpolicy_t* rocksdb_filterpolicy_create_bloom_format(int bits_per_key, bool original_format) { // Make a rocksdb_filterpolicy_t, but override all of its methods so // they delegate to a NewBloomFilterPolicy() instead of user // supplied C functions. struct Wrapper : public rocksdb_filterpolicy_t { const FilterPolicy* rep_; ~Wrapper() { delete rep_; } - const char* Name() const { return rep_->Name(); } - void CreateFilter(const Slice* keys, int n, std::string* dst) const { + const char* Name() const override { return rep_->Name(); } + void CreateFilter(const Slice* keys, int n, + std::string* dst) const override { return rep_->CreateFilter(keys, n, dst); } - bool KeyMayMatch(const Slice& key, const Slice& filter) const { + bool KeyMayMatch(const Slice& key, const Slice& filter) const override { return rep_->KeyMayMatch(key, filter); } static void DoNothing(void*) { } }; Wrapper* wrapper = new Wrapper; - wrapper->rep_ = NewBloomFilterPolicy(bits_per_key); + wrapper->rep_ = NewBloomFilterPolicy(bits_per_key, original_format); wrapper->state_ = nullptr; wrapper->delete_filter_ = nullptr; wrapper->destructor_ = &Wrapper::DoNothing; return wrapper; } +rocksdb_filterpolicy_t* rocksdb_filterpolicy_create_bloom_full(int bits_per_key) { + return rocksdb_filterpolicy_create_bloom_format(bits_per_key, false); +} + +rocksdb_filterpolicy_t* rocksdb_filterpolicy_create_bloom(int bits_per_key) { + return rocksdb_filterpolicy_create_bloom_format(bits_per_key, true); +} + rocksdb_mergeoperator_t* rocksdb_mergeoperator_create( void* state, void (*destructor)(void*), char* (*full_merge)(void*, const char* key, size_t key_length, @@ -1844,6 +2704,19 @@ void rocksdb_readoptions_set_snapshot( opt->rep.snapshot = (snap ? snap->rep : nullptr); } +void rocksdb_readoptions_set_iterate_upper_bound( + rocksdb_readoptions_t* opt, + const char* key, size_t keylen) { + if (key == nullptr) { + opt->upper_bound = Slice(); + opt->rep.iterate_upper_bound = nullptr; + + } else { + opt->upper_bound = Slice(key, keylen); + opt->rep.iterate_upper_bound = &opt->upper_bound; + } +} + void rocksdb_readoptions_set_read_tier( rocksdb_readoptions_t* opt, int v) { opt->rep.read_tier = static_cast(v); @@ -1854,6 +2727,21 @@ void rocksdb_readoptions_set_tailing( opt->rep.tailing = v; } +void rocksdb_readoptions_set_readahead_size( + rocksdb_readoptions_t* opt, size_t v) { + opt->rep.readahead_size = v; +} + +void rocksdb_readoptions_set_pin_data(rocksdb_readoptions_t* opt, + unsigned char v) { + opt->rep.pin_data = v; +} + +void rocksdb_readoptions_set_total_order_seek(rocksdb_readoptions_t* opt, + unsigned char v) { + opt->rep.total_order_seek = v; +} + rocksdb_writeoptions_t* rocksdb_writeoptions_create() { return new rocksdb_writeoptions_t; } @@ -1867,52 +2755,226 @@ void rocksdb_writeoptions_set_sync( opt->rep.sync = v; } -void rocksdb_writeoptions_disable_WAL(rocksdb_writeoptions_t* opt, int disable) { - opt->rep.disableWAL = disable; +void rocksdb_writeoptions_disable_WAL(rocksdb_writeoptions_t* opt, int disable) { + opt->rep.disableWAL = disable; +} + +rocksdb_compactoptions_t* rocksdb_compactoptions_create() { + return new rocksdb_compactoptions_t; +} + +void rocksdb_compactoptions_destroy(rocksdb_compactoptions_t* opt) { + delete opt; +} + +void rocksdb_compactoptions_set_exclusive_manual_compaction( + rocksdb_compactoptions_t* opt, unsigned char v) { + opt->rep.exclusive_manual_compaction = v; +} + +void rocksdb_compactoptions_set_change_level(rocksdb_compactoptions_t* opt, + unsigned char v) { + opt->rep.change_level = v; +} + +void rocksdb_compactoptions_set_target_level(rocksdb_compactoptions_t* opt, + int n) { + opt->rep.target_level = n; +} + +rocksdb_flushoptions_t* rocksdb_flushoptions_create() { + return new rocksdb_flushoptions_t; +} + +void rocksdb_flushoptions_destroy(rocksdb_flushoptions_t* opt) { + delete opt; +} + +void rocksdb_flushoptions_set_wait( + rocksdb_flushoptions_t* opt, unsigned char v) { + opt->rep.wait = v; +} + +rocksdb_cache_t* rocksdb_cache_create_lru(size_t capacity) { + rocksdb_cache_t* c = new rocksdb_cache_t; + c->rep = NewLRUCache(capacity); + return c; +} + +void rocksdb_cache_destroy(rocksdb_cache_t* cache) { + delete cache; +} + +void rocksdb_cache_set_capacity(rocksdb_cache_t* cache, size_t capacity) { + cache->rep->SetCapacity(capacity); +} + +size_t rocksdb_cache_get_usage(rocksdb_cache_t* cache) { + return cache->rep->GetUsage(); +} + +size_t rocksdb_cache_get_pinned_usage(rocksdb_cache_t* cache) { + return cache->rep->GetPinnedUsage(); +} + +rocksdb_dbpath_t* rocksdb_dbpath_create(const char* path, uint64_t target_size) { + rocksdb_dbpath_t* result = new rocksdb_dbpath_t; + result->rep.path = std::string(path); + result->rep.target_size = target_size; + return result; +} + +void rocksdb_dbpath_destroy(rocksdb_dbpath_t* dbpath) { + delete dbpath; +} + +rocksdb_env_t* rocksdb_create_default_env() { + rocksdb_env_t* result = new rocksdb_env_t; + result->rep = Env::Default(); + result->is_default = true; + return result; +} + +rocksdb_env_t* rocksdb_create_mem_env() { + rocksdb_env_t* result = new rocksdb_env_t; + result->rep = rocksdb::NewMemEnv(Env::Default()); + result->is_default = false; + return result; +} + +void rocksdb_env_set_background_threads(rocksdb_env_t* env, int n) { + env->rep->SetBackgroundThreads(n); +} + +void rocksdb_env_set_high_priority_background_threads(rocksdb_env_t* env, int n) { + env->rep->SetBackgroundThreads(n, Env::HIGH); +} + +void rocksdb_env_join_all_threads(rocksdb_env_t* env) { + env->rep->WaitForJoin(); +} + +void rocksdb_env_destroy(rocksdb_env_t* env) { + if (!env->is_default) delete env->rep; + delete env; +} + +rocksdb_envoptions_t* rocksdb_envoptions_create() { + rocksdb_envoptions_t* opt = new rocksdb_envoptions_t; + return opt; +} + +void rocksdb_envoptions_destroy(rocksdb_envoptions_t* opt) { delete opt; } + +rocksdb_sstfilewriter_t* rocksdb_sstfilewriter_create( + const rocksdb_envoptions_t* env, const rocksdb_options_t* io_options) { + rocksdb_sstfilewriter_t* writer = new rocksdb_sstfilewriter_t; + writer->rep = new SstFileWriter(env->rep, io_options->rep); + return writer; +} + +rocksdb_sstfilewriter_t* rocksdb_sstfilewriter_create_with_comparator( + const rocksdb_envoptions_t* env, const rocksdb_options_t* io_options, + const rocksdb_comparator_t* comparator) { + rocksdb_sstfilewriter_t* writer = new rocksdb_sstfilewriter_t; + writer->rep = new SstFileWriter(env->rep, io_options->rep); + return writer; +} + +void rocksdb_sstfilewriter_open(rocksdb_sstfilewriter_t* writer, + const char* name, char** errptr) { + SaveError(errptr, writer->rep->Open(std::string(name))); +} + +void rocksdb_sstfilewriter_add(rocksdb_sstfilewriter_t* writer, const char* key, + size_t keylen, const char* val, size_t vallen, + char** errptr) { + SaveError(errptr, writer->rep->Put(Slice(key, keylen), Slice(val, vallen))); +} + +void rocksdb_sstfilewriter_put(rocksdb_sstfilewriter_t* writer, const char* key, + size_t keylen, const char* val, size_t vallen, + char** errptr) { + SaveError(errptr, writer->rep->Put(Slice(key, keylen), Slice(val, vallen))); +} + +void rocksdb_sstfilewriter_merge(rocksdb_sstfilewriter_t* writer, + const char* key, size_t keylen, + const char* val, size_t vallen, + char** errptr) { + SaveError(errptr, writer->rep->Merge(Slice(key, keylen), Slice(val, vallen))); } +void rocksdb_sstfilewriter_delete(rocksdb_sstfilewriter_t* writer, + const char* key, size_t keylen, + char** errptr) { + SaveError(errptr, writer->rep->Delete(Slice(key, keylen))); +} -rocksdb_flushoptions_t* rocksdb_flushoptions_create() { - return new rocksdb_flushoptions_t; +void rocksdb_sstfilewriter_finish(rocksdb_sstfilewriter_t* writer, + char** errptr) { + SaveError(errptr, writer->rep->Finish(NULL)); } -void rocksdb_flushoptions_destroy(rocksdb_flushoptions_t* opt) { - delete opt; +void rocksdb_sstfilewriter_destroy(rocksdb_sstfilewriter_t* writer) { + delete writer->rep; + delete writer; } -void rocksdb_flushoptions_set_wait( - rocksdb_flushoptions_t* opt, unsigned char v) { - opt->rep.wait = v; +rocksdb_ingestexternalfileoptions_t* +rocksdb_ingestexternalfileoptions_create() { + rocksdb_ingestexternalfileoptions_t* opt = + new rocksdb_ingestexternalfileoptions_t; + return opt; } -rocksdb_cache_t* rocksdb_cache_create_lru(size_t capacity) { - rocksdb_cache_t* c = new rocksdb_cache_t; - c->rep = NewLRUCache(capacity); - return c; +void rocksdb_ingestexternalfileoptions_set_move_files( + rocksdb_ingestexternalfileoptions_t* opt, unsigned char move_files) { + opt->rep.move_files = move_files; } -void rocksdb_cache_destroy(rocksdb_cache_t* cache) { - delete cache; +void rocksdb_ingestexternalfileoptions_set_snapshot_consistency( + rocksdb_ingestexternalfileoptions_t* opt, + unsigned char snapshot_consistency) { + opt->rep.snapshot_consistency = snapshot_consistency; } -rocksdb_env_t* rocksdb_create_default_env() { - rocksdb_env_t* result = new rocksdb_env_t; - result->rep = Env::Default(); - result->is_default = true; - return result; +void rocksdb_ingestexternalfileoptions_set_allow_global_seqno( + rocksdb_ingestexternalfileoptions_t* opt, + unsigned char allow_global_seqno) { + opt->rep.allow_global_seqno = allow_global_seqno; } -void rocksdb_env_set_background_threads(rocksdb_env_t* env, int n) { - env->rep->SetBackgroundThreads(n); +void rocksdb_ingestexternalfileoptions_set_allow_blocking_flush( + rocksdb_ingestexternalfileoptions_t* opt, + unsigned char allow_blocking_flush) { + opt->rep.allow_blocking_flush = allow_blocking_flush; } -void rocksdb_env_set_high_priority_background_threads(rocksdb_env_t* env, int n) { - env->rep->SetBackgroundThreads(n, Env::HIGH); +void rocksdb_ingestexternalfileoptions_destroy( + rocksdb_ingestexternalfileoptions_t* opt) { + delete opt; } -void rocksdb_env_destroy(rocksdb_env_t* env) { - if (!env->is_default) delete env->rep; - delete env; +void rocksdb_ingest_external_file( + rocksdb_t* db, const char* const* file_list, const size_t list_len, + const rocksdb_ingestexternalfileoptions_t* opt, char** errptr) { + std::vector files(list_len); + for (size_t i = 0; i < list_len; ++i) { + files[i] = std::string(file_list[i]); + } + SaveError(errptr, db->rep->IngestExternalFile(files, opt->rep)); +} + +void rocksdb_ingest_external_file_cf( + rocksdb_t* db, rocksdb_column_family_handle_t* handle, + const char* const* file_list, const size_t list_len, + const rocksdb_ingestexternalfileoptions_t* opt, char** errptr) { + std::vector files(list_len); + for (size_t i = 0; i < list_len; ++i) { + files[i] = std::string(file_list[i]); + } + SaveError(errptr, db->rep->IngestExternalFile(handle->rep, files, opt->rep)); } rocksdb_slicetransform_t* rocksdb_slicetransform_create( @@ -1947,16 +3009,14 @@ rocksdb_slicetransform_t* rocksdb_slicetransform_create_fixed_prefix(size_t pref struct Wrapper : public rocksdb_slicetransform_t { const SliceTransform* rep_; ~Wrapper() { delete rep_; } - const char* Name() const { return rep_->Name(); } - Slice Transform(const Slice& src) const { + const char* Name() const override { return rep_->Name(); } + Slice Transform(const Slice& src) const override { return rep_->Transform(src); } - bool InDomain(const Slice& src) const { + bool InDomain(const Slice& src) const override { return rep_->InDomain(src); } - bool InRange(const Slice& src) const { - return rep_->InRange(src); - } + bool InRange(const Slice& src) const override { return rep_->InRange(src); } static void DoNothing(void*) { } }; Wrapper* wrapper = new Wrapper; @@ -1966,6 +3026,27 @@ rocksdb_slicetransform_t* rocksdb_slicetransform_create_fixed_prefix(size_t pref return wrapper; } +rocksdb_slicetransform_t* rocksdb_slicetransform_create_noop() { + struct Wrapper : public rocksdb_slicetransform_t { + const SliceTransform* rep_; + ~Wrapper() { delete rep_; } + const char* Name() const override { return rep_->Name(); } + Slice Transform(const Slice& src) const override { + return rep_->Transform(src); + } + bool InDomain(const Slice& src) const override { + return rep_->InDomain(src); + } + bool InRange(const Slice& src) const override { return rep_->InRange(src); } + static void DoNothing(void*) { } + }; + Wrapper* wrapper = new Wrapper; + wrapper->rep_ = rocksdb::NewNoopTransform(); + wrapper->state_ = nullptr; + wrapper->destructor_ = &Wrapper::DoNothing; + return wrapper; +} + rocksdb_universal_compaction_options_t* rocksdb_universal_compaction_options_create() { rocksdb_universal_compaction_options_t* result = new rocksdb_universal_compaction_options_t; result->rep = new rocksdb::CompactionOptionsUniversal; @@ -2039,7 +3120,7 @@ void rocksdb_options_set_min_level_to_compress(rocksdb_options_t* opt, int level int rocksdb_livefiles_count( const rocksdb_livefiles_t* lf) { - return lf->rep.size(); + return static_cast(lf->rep.size()); } const char* rocksdb_livefiles_name( @@ -2081,6 +3162,501 @@ extern void rocksdb_livefiles_destroy( delete lf; } +void rocksdb_get_options_from_string(const rocksdb_options_t* base_options, + const char* opts_str, + rocksdb_options_t* new_options, + char** errptr) { + SaveError(errptr, + GetOptionsFromString(base_options->rep, std::string(opts_str), + &new_options->rep)); +} + +void rocksdb_delete_file_in_range(rocksdb_t* db, const char* start_key, + size_t start_key_len, const char* limit_key, + size_t limit_key_len, char** errptr) { + Slice a, b; + SaveError( + errptr, + DeleteFilesInRange( + db->rep, db->rep->DefaultColumnFamily(), + (start_key ? (a = Slice(start_key, start_key_len), &a) : nullptr), + (limit_key ? (b = Slice(limit_key, limit_key_len), &b) : nullptr))); +} + +void rocksdb_delete_file_in_range_cf( + rocksdb_t* db, rocksdb_column_family_handle_t* column_family, + const char* start_key, size_t start_key_len, const char* limit_key, + size_t limit_key_len, char** errptr) { + Slice a, b; + SaveError( + errptr, + DeleteFilesInRange( + db->rep, column_family->rep, + (start_key ? (a = Slice(start_key, start_key_len), &a) : nullptr), + (limit_key ? (b = Slice(limit_key, limit_key_len), &b) : nullptr))); +} + +rocksdb_transactiondb_options_t* rocksdb_transactiondb_options_create() { + return new rocksdb_transactiondb_options_t; +} + +void rocksdb_transactiondb_options_destroy(rocksdb_transactiondb_options_t* opt){ + delete opt; +} + +void rocksdb_transactiondb_options_set_max_num_locks( + rocksdb_transactiondb_options_t* opt, int64_t max_num_locks) { + opt->rep.max_num_locks = max_num_locks; +} + +void rocksdb_transactiondb_options_set_num_stripes( + rocksdb_transactiondb_options_t* opt, size_t num_stripes) { + opt->rep.num_stripes = num_stripes; +} + +void rocksdb_transactiondb_options_set_transaction_lock_timeout( + rocksdb_transactiondb_options_t* opt, int64_t txn_lock_timeout) { + opt->rep.transaction_lock_timeout = txn_lock_timeout; +} + +void rocksdb_transactiondb_options_set_default_lock_timeout( + rocksdb_transactiondb_options_t* opt, int64_t default_lock_timeout) { + opt->rep.default_lock_timeout = default_lock_timeout; +} + +rocksdb_transaction_options_t* rocksdb_transaction_options_create() { + return new rocksdb_transaction_options_t; +} + +void rocksdb_transaction_options_destroy(rocksdb_transaction_options_t* opt) { + delete opt; +} + +void rocksdb_transaction_options_set_set_snapshot( + rocksdb_transaction_options_t* opt, unsigned char v) { + opt->rep.set_snapshot = v; +} + +void rocksdb_transaction_options_set_deadlock_detect( + rocksdb_transaction_options_t* opt, unsigned char v) { + opt->rep.deadlock_detect = v; +} + +void rocksdb_transaction_options_set_lock_timeout( + rocksdb_transaction_options_t* opt, int64_t lock_timeout) { + opt->rep.lock_timeout = lock_timeout; +} + +void rocksdb_transaction_options_set_expiration( + rocksdb_transaction_options_t* opt, int64_t expiration) { + opt->rep.expiration = expiration; +} + +void rocksdb_transaction_options_set_deadlock_detect_depth( + rocksdb_transaction_options_t* opt, int64_t depth) { + opt->rep.deadlock_detect_depth = depth; +} + +void rocksdb_transaction_options_set_max_write_batch_size( + rocksdb_transaction_options_t* opt, size_t size) { + opt->rep.max_write_batch_size = size; +} + +rocksdb_optimistictransaction_options_t* +rocksdb_optimistictransaction_options_create() { + return new rocksdb_optimistictransaction_options_t; +} + +void rocksdb_optimistictransaction_options_destroy( + rocksdb_optimistictransaction_options_t* opt) { + delete opt; +} + +void rocksdb_optimistictransaction_options_set_set_snapshot( + rocksdb_optimistictransaction_options_t* opt, unsigned char v) { + opt->rep.set_snapshot = v; +} + +rocksdb_column_family_handle_t* rocksdb_transactiondb_create_column_family( + rocksdb_transactiondb_t* txn_db, + const rocksdb_options_t* column_family_options, + const char* column_family_name, char** errptr) { + rocksdb_column_family_handle_t* handle = new rocksdb_column_family_handle_t; + SaveError(errptr, txn_db->rep->CreateColumnFamily( + ColumnFamilyOptions(column_family_options->rep), + std::string(column_family_name), &(handle->rep))); + return handle; +} + +rocksdb_transactiondb_t* rocksdb_transactiondb_open( + const rocksdb_options_t* options, + const rocksdb_transactiondb_options_t* txn_db_options, const char* name, + char** errptr) { + TransactionDB* txn_db; + if (SaveError(errptr, TransactionDB::Open(options->rep, txn_db_options->rep, + std::string(name), &txn_db))) { + return nullptr; + } + rocksdb_transactiondb_t* result = new rocksdb_transactiondb_t; + result->rep = txn_db; + return result; +} + +const rocksdb_snapshot_t* rocksdb_transactiondb_create_snapshot( + rocksdb_transactiondb_t* txn_db) { + rocksdb_snapshot_t* result = new rocksdb_snapshot_t; + result->rep = txn_db->rep->GetSnapshot(); + return result; +} + +void rocksdb_transactiondb_release_snapshot( + rocksdb_transactiondb_t* txn_db, const rocksdb_snapshot_t* snapshot) { + txn_db->rep->ReleaseSnapshot(snapshot->rep); + delete snapshot; +} + +rocksdb_transaction_t* rocksdb_transaction_begin( + rocksdb_transactiondb_t* txn_db, + const rocksdb_writeoptions_t* write_options, + const rocksdb_transaction_options_t* txn_options, + rocksdb_transaction_t* old_txn) { + if (old_txn == nullptr) { + rocksdb_transaction_t* result = new rocksdb_transaction_t; + result->rep = txn_db->rep->BeginTransaction(write_options->rep, + txn_options->rep, nullptr); + return result; + } + old_txn->rep = txn_db->rep->BeginTransaction(write_options->rep, + txn_options->rep, old_txn->rep); + return old_txn; +} + +void rocksdb_transaction_commit(rocksdb_transaction_t* txn, char** errptr) { + SaveError(errptr, txn->rep->Commit()); +} + +void rocksdb_transaction_rollback(rocksdb_transaction_t* txn, char** errptr) { + SaveError(errptr, txn->rep->Rollback()); +} + +void rocksdb_transaction_destroy(rocksdb_transaction_t* txn) { + delete txn->rep; + delete txn; +} + +const rocksdb_snapshot_t* rocksdb_transaction_get_snapshot( + rocksdb_transaction_t* txn) { + rocksdb_snapshot_t* result = new rocksdb_snapshot_t; + result->rep = txn->rep->GetSnapshot(); + return result; +} + +// Read a key inside a transaction +char* rocksdb_transaction_get(rocksdb_transaction_t* txn, + const rocksdb_readoptions_t* options, + const char* key, size_t klen, size_t* vlen, + char** errptr) { + char* result = nullptr; + std::string tmp; + Status s = txn->rep->Get(options->rep, Slice(key, klen), &tmp); + if (s.ok()) { + *vlen = tmp.size(); + result = CopyString(tmp); + } else { + *vlen = 0; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + } + return result; +} + +char* rocksdb_transaction_get_cf(rocksdb_transaction_t* txn, + const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, size_t* vlen, + char** errptr) { + char* result = nullptr; + std::string tmp; + Status s = + txn->rep->Get(options->rep, column_family->rep, Slice(key, klen), &tmp); + if (s.ok()) { + *vlen = tmp.size(); + result = CopyString(tmp); + } else { + *vlen = 0; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + } + return result; +} + +// Read a key inside a transaction +char* rocksdb_transaction_get_for_update(rocksdb_transaction_t* txn, + const rocksdb_readoptions_t* options, + const char* key, size_t klen, + size_t* vlen, unsigned char exclusive, + char** errptr) { + char* result = nullptr; + std::string tmp; + Status s = + txn->rep->GetForUpdate(options->rep, Slice(key, klen), &tmp, exclusive); + if (s.ok()) { + *vlen = tmp.size(); + result = CopyString(tmp); + } else { + *vlen = 0; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + } + return result; +} + +// Read a key outside a transaction +char* rocksdb_transactiondb_get( + rocksdb_transactiondb_t* txn_db, + const rocksdb_readoptions_t* options, + const char* key, size_t klen, + size_t* vlen, + char** errptr){ + char* result = nullptr; + std::string tmp; + Status s = txn_db->rep->Get(options->rep, Slice(key, klen), &tmp); + if (s.ok()) { + *vlen = tmp.size(); + result = CopyString(tmp); + } else { + *vlen = 0; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + } + return result; +} + +char* rocksdb_transactiondb_get_cf( + rocksdb_transactiondb_t* txn_db, const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, size_t* vallen, char** errptr) { + char* result = nullptr; + std::string tmp; + Status s = txn_db->rep->Get(options->rep, column_family->rep, + Slice(key, keylen), &tmp); + if (s.ok()) { + *vallen = tmp.size(); + result = CopyString(tmp); + } else { + *vallen = 0; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + } + return result; +} + +// Put a key inside a transaction +void rocksdb_transaction_put(rocksdb_transaction_t* txn, const char* key, + size_t klen, const char* val, size_t vlen, + char** errptr) { + SaveError(errptr, txn->rep->Put(Slice(key, klen), Slice(val, vlen))); +} + +void rocksdb_transaction_put_cf(rocksdb_transaction_t* txn, + rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, const char* val, + size_t vlen, char** errptr) { + SaveError(errptr, txn->rep->Put(column_family->rep, Slice(key, klen), + Slice(val, vlen))); +} + +// Put a key outside a transaction +void rocksdb_transactiondb_put(rocksdb_transactiondb_t* txn_db, + const rocksdb_writeoptions_t* options, + const char* key, size_t klen, const char* val, + size_t vlen, char** errptr) { + SaveError(errptr, txn_db->rep->Put(options->rep, Slice(key, klen), + Slice(val, vlen))); +} + +void rocksdb_transactiondb_put_cf(rocksdb_transactiondb_t* txn_db, + const rocksdb_writeoptions_t* options, + rocksdb_column_family_handle_t* column_family, + const char* key, size_t keylen, + const char* val, size_t vallen, + char** errptr) { + SaveError(errptr, txn_db->rep->Put(options->rep, column_family->rep, + Slice(key, keylen), Slice(val, vallen))); +} + +// Write batch into transaction db +void rocksdb_transactiondb_write( + rocksdb_transactiondb_t* db, + const rocksdb_writeoptions_t* options, + rocksdb_writebatch_t* batch, + char** errptr) { + SaveError(errptr, db->rep->Write(options->rep, &batch->rep)); +} + +// Merge a key inside a transaction +void rocksdb_transaction_merge(rocksdb_transaction_t* txn, const char* key, + size_t klen, const char* val, size_t vlen, + char** errptr) { + SaveError(errptr, txn->rep->Merge(Slice(key, klen), Slice(val, vlen))); +} + +// Merge a key outside a transaction +void rocksdb_transactiondb_merge(rocksdb_transactiondb_t* txn_db, + const rocksdb_writeoptions_t* options, + const char* key, size_t klen, const char* val, + size_t vlen, char** errptr) { + SaveError(errptr, + txn_db->rep->Merge(options->rep, Slice(key, klen), Slice(val, vlen))); +} + +// Delete a key inside a transaction +void rocksdb_transaction_delete(rocksdb_transaction_t* txn, const char* key, + size_t klen, char** errptr) { + SaveError(errptr, txn->rep->Delete(Slice(key, klen))); +} + +void rocksdb_transaction_delete_cf( + rocksdb_transaction_t* txn, rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, char** errptr) { + SaveError(errptr, txn->rep->Delete(column_family->rep, Slice(key, klen))); +} + +// Delete a key outside a transaction +void rocksdb_transactiondb_delete(rocksdb_transactiondb_t* txn_db, + const rocksdb_writeoptions_t* options, + const char* key, size_t klen, char** errptr) { + SaveError(errptr, txn_db->rep->Delete(options->rep, Slice(key, klen))); +} + +void rocksdb_transactiondb_delete_cf( + rocksdb_transactiondb_t* txn_db, const rocksdb_writeoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, char** errptr) { + SaveError(errptr, txn_db->rep->Delete(options->rep, column_family->rep, + Slice(key, keylen))); +} + +// Create an iterator inside a transaction +rocksdb_iterator_t* rocksdb_transaction_create_iterator( + rocksdb_transaction_t* txn, const rocksdb_readoptions_t* options) { + rocksdb_iterator_t* result = new rocksdb_iterator_t; + result->rep = txn->rep->GetIterator(options->rep); + return result; +} + +// Create an iterator outside a transaction +rocksdb_iterator_t* rocksdb_transactiondb_create_iterator( + rocksdb_transactiondb_t* txn_db, const rocksdb_readoptions_t* options) { + rocksdb_iterator_t* result = new rocksdb_iterator_t; + result->rep = txn_db->rep->NewIterator(options->rep); + return result; +} + +void rocksdb_transactiondb_close(rocksdb_transactiondb_t* txn_db) { + delete txn_db->rep; + delete txn_db; +} + +rocksdb_checkpoint_t* rocksdb_transactiondb_checkpoint_object_create( + rocksdb_transactiondb_t* txn_db, char** errptr) { + Checkpoint* checkpoint; + if (SaveError(errptr, Checkpoint::Create(txn_db->rep, &checkpoint))) { + return nullptr; + } + rocksdb_checkpoint_t* result = new rocksdb_checkpoint_t; + result->rep = checkpoint; + return result; +} + +rocksdb_optimistictransactiondb_t* rocksdb_optimistictransactiondb_open( + const rocksdb_options_t* options, const char* name, + char** errptr) { + OptimisticTransactionDB* otxn_db; + if (SaveError(errptr, OptimisticTransactionDB::Open( + options->rep, std::string(name), &otxn_db))) { + return nullptr; + } + rocksdb_optimistictransactiondb_t* result = + new rocksdb_optimistictransactiondb_t; + result->rep = otxn_db; + return result; +} + +rocksdb_transaction_t* rocksdb_optimistictransaction_begin( + rocksdb_optimistictransactiondb_t* otxn_db, + const rocksdb_writeoptions_t* write_options, + const rocksdb_optimistictransaction_options_t* otxn_options, + rocksdb_transaction_t* old_txn) { + if (old_txn == nullptr) { + rocksdb_transaction_t* result = new rocksdb_transaction_t; + result->rep = otxn_db->rep->BeginTransaction(write_options->rep, + otxn_options->rep, nullptr); + return result; + } + old_txn->rep = otxn_db->rep->BeginTransaction( + write_options->rep, otxn_options->rep, old_txn->rep); + return old_txn; +} + +void rocksdb_optimistictransactiondb_close( + rocksdb_optimistictransactiondb_t* otxn_db) { + delete otxn_db->rep; + delete otxn_db; +} + +void rocksdb_free(void* ptr) { free(ptr); } + +rocksdb_pinnableslice_t* rocksdb_get_pinned( + rocksdb_t* db, const rocksdb_readoptions_t* options, const char* key, + size_t keylen, char** errptr) { + rocksdb_pinnableslice_t* v = new (rocksdb_pinnableslice_t); + Status s = db->rep->Get(options->rep, db->rep->DefaultColumnFamily(), + Slice(key, keylen), &v->rep); + if (!s.ok()) { + delete (v); + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + return NULL; + } + return v; +} + +rocksdb_pinnableslice_t* rocksdb_get_pinned_cf( + rocksdb_t* db, const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, char** errptr) { + rocksdb_pinnableslice_t* v = new (rocksdb_pinnableslice_t); + Status s = db->rep->Get(options->rep, column_family->rep, Slice(key, keylen), + &v->rep); + if (!s.ok()) { + delete v; + if (!s.IsNotFound()) { + SaveError(errptr, s); + } + return NULL; + } + return v; +} + +void rocksdb_pinnableslice_destroy(rocksdb_pinnableslice_t* v) { delete v; } + +const char* rocksdb_pinnableslice_value(const rocksdb_pinnableslice_t* v, + size_t* vlen) { + if (!v) { + *vlen = 0; + return NULL; + } + + *vlen = v->rep.size(); + return v->rep.data(); +} } // end extern "C" -#endif // ROCKSDB_LITE +#endif // !ROCKSDB_LITE diff --git a/db/c_test.c b/db/c_test.c index 171fd6d5c88..7b76badf1ce 100644 --- a/db/c_test.c +++ b/db/c_test.c @@ -2,23 +2,51 @@ Use of this source code is governed by a BSD-style license that can be found in the LICENSE file. See the AUTHORS file for names of contributors. */ +#include + +#ifndef ROCKSDB_LITE // Lite does not support C API + #include "rocksdb/c.h" #include -#include #include #include #include +#ifndef OS_WIN #include +#endif +#include + +// Can not use port/port.h macros as this is a c file +#ifdef OS_WIN + +#include + +#define snprintf _snprintf + +// Ok for uniqueness +int geteuid() { + int result = 0; + + result = ((int)GetCurrentProcessId() << 16); + result |= (int)GetCurrentThreadId(); + + return result; +} + +#endif const char* phase = ""; static char dbname[200]; +static char sstfilename[200]; +static char dbbackupname[200]; +static char dbcheckpointname[200]; +static char dbpathname[200]; static void StartPhase(const char* name) { fprintf(stderr, "=== Test %s\n", name); phase = name; } - static const char* GetTempDir(void) { const char* ret = getenv("TEST_TMPDIR"); if (ret == NULL || ret[0] == '\0') @@ -61,6 +89,16 @@ static void Free(char** ptr) { } } +static void CheckValue( + char* err, + const char* expected, + char** actual, + size_t actual_length) { + CheckNoError(err); + CheckEqual(expected, *actual, actual_length); + Free(actual); +} + static void CheckGet( rocksdb_t* db, const rocksdb_readoptions_t* options, @@ -90,6 +128,32 @@ static void CheckGetCF( Free(&val); } +static void CheckPinGet(rocksdb_t* db, const rocksdb_readoptions_t* options, + const char* key, const char* expected) { + char* err = NULL; + size_t val_len; + const char* val; + rocksdb_pinnableslice_t* p; + p = rocksdb_get_pinned(db, options, key, strlen(key), &err); + CheckNoError(err); + val = rocksdb_pinnableslice_value(p, &val_len); + CheckEqual(expected, val, val_len); + rocksdb_pinnableslice_destroy(p); +} + +static void CheckPinGetCF(rocksdb_t* db, const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* handle, + const char* key, const char* expected) { + char* err = NULL; + size_t val_len; + const char* val; + rocksdb_pinnableslice_t* p; + p = rocksdb_get_pinned_cf(db, options, handle, key, strlen(key), &err); + CheckNoError(err); + val = rocksdb_pinnableslice_value(p, &val_len); + CheckEqual(expected, val, val_len); + rocksdb_pinnableslice_destroy(p); +} static void CheckIter(rocksdb_iterator_t* iter, const char* key, const char* val) { @@ -132,7 +196,7 @@ static void CmpDestroy(void* arg) { } static int CmpCompare(void* arg, const char* a, size_t alen, const char* b, size_t blen) { - int n = (alen < blen) ? alen : blen; + size_t n = (alen < blen) ? alen : blen; int r = memcmp(a, b, n); if (r == 0) { if (alen < blen) r = -1; @@ -225,79 +289,6 @@ static rocksdb_t* CheckCompaction(rocksdb_t* db, rocksdb_options_t* options, return db; } -// Custom compaction filter V2. -static void CompactionFilterV2Destroy(void* arg) { } -static const char* CompactionFilterV2Name(void* arg) { - return "TestCompactionFilterV2"; -} -static void CompactionFilterV2Filter( - void* arg, int level, size_t num_keys, - const char* const* keys_list, const size_t* keys_list_sizes, - const char* const* existing_values_list, const size_t* existing_values_list_sizes, - char** new_values_list, size_t* new_values_list_sizes, - unsigned char* to_delete_list) { - size_t i; - for (i = 0; i < num_keys; i++) { - // If any value is "gc", it's removed. - if (existing_values_list_sizes[i] == 2 && memcmp(existing_values_list[i], "gc", 2) == 0) { - to_delete_list[i] = 1; - } else if (existing_values_list_sizes[i] == 6 && memcmp(existing_values_list[i], "gc all", 6) == 0) { - // If any value is "gc all", all keys are removed. - size_t j; - for (j = 0; j < num_keys; j++) { - to_delete_list[j] = 1; - } - return; - } else if (existing_values_list_sizes[i] == 6 && memcmp(existing_values_list[i], "change", 6) == 0) { - // If value is "change", set changed value to "changed". - size_t len; - len = strlen("changed"); - new_values_list[i] = malloc(len); - memcpy(new_values_list[i], "changed", len); - new_values_list_sizes[i] = len; - } else { - // Otherwise, no keys are removed. - } - } -} - -// Custom prefix extractor for compaction filter V2 which extracts first 3 characters. -static void CFV2PrefixExtractorDestroy(void* arg) { } -static char* CFV2PrefixExtractorTransform(void* arg, const char* key, size_t length, size_t* dst_length) { - // Verify keys are maximum length 4; this verifies fix for a - // prior bug which was passing the RocksDB-encoded key with - // logical timestamp suffix instead of parsed user key. - if (length > 4) { - fprintf(stderr, "%s:%d: %s: key %s is not user key\n", __FILE__, __LINE__, phase, key); - abort(); - } - *dst_length = length < 3 ? length : 3; - return (char*)key; -} -static unsigned char CFV2PrefixExtractorInDomain(void* state, const char* key, size_t length) { - return 1; -} -static unsigned char CFV2PrefixExtractorInRange(void* state, const char* key, size_t length) { - return 1; -} -static const char* CFV2PrefixExtractorName(void* state) { - return "TestCFV2PrefixExtractor"; -} - -// Custom compaction filter factory V2. -static void CompactionFilterFactoryV2Destroy(void* arg) { - rocksdb_slicetransform_destroy((rocksdb_slicetransform_t*)arg); -} -static const char* CompactionFilterFactoryV2Name(void* arg) { - return "TestCompactionFilterFactoryV2"; -} -static rocksdb_compactionfilterv2_t* CompactionFilterFactoryV2Create( - void* state, const rocksdb_compactionfiltercontext_t* context) { - return rocksdb_compactionfilterv2_create(state, CompactionFilterV2Destroy, - CompactionFilterV2Filter, - CompactionFilterV2Name); -} - // Custom merge operator static void MergeOperatorDestroy(void* arg) { } static const char* MergeOperatorName(void* arg) { @@ -329,15 +320,64 @@ static char* MergeOperatorPartialMerge( return result; } +static void CheckTxnGet( + rocksdb_transaction_t* txn, + const rocksdb_readoptions_t* options, + const char* key, + const char* expected) { + char* err = NULL; + size_t val_len; + char* val; + val = rocksdb_transaction_get(txn, options, key, strlen(key), &val_len, &err); + CheckNoError(err); + CheckEqual(expected, val, val_len); + Free(&val); +} + +static void CheckTxnDBGet( + rocksdb_transactiondb_t* txn_db, + const rocksdb_readoptions_t* options, + const char* key, + const char* expected) { + char* err = NULL; + size_t val_len; + char* val; + val = rocksdb_transactiondb_get(txn_db, options, key, strlen(key), &val_len, &err); + CheckNoError(err); + CheckEqual(expected, val, val_len); + Free(&val); +} + +static void CheckTxnDBGetCF(rocksdb_transactiondb_t* txn_db, + const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* column_family, + const char* key, const char* expected) { + char* err = NULL; + size_t val_len; + char* val; + val = rocksdb_transactiondb_get_cf(txn_db, options, column_family, key, + strlen(key), &val_len, &err); + CheckNoError(err); + CheckEqual(expected, val, val_len); + Free(&val); +} + int main(int argc, char** argv) { rocksdb_t* db; rocksdb_comparator_t* cmp; rocksdb_cache_t* cache; + rocksdb_dbpath_t *dbpath; rocksdb_env_t* env; rocksdb_options_t* options; + rocksdb_compactoptions_t* coptions; rocksdb_block_based_table_options_t* table_options; rocksdb_readoptions_t* roptions; rocksdb_writeoptions_t* woptions; + rocksdb_ratelimiter_t* rate_limiter; + rocksdb_transactiondb_t* txn_db; + rocksdb_transactiondb_options_t* txn_db_options; + rocksdb_transaction_t* txn; + rocksdb_transaction_options_t* txn_options; char* err = NULL; int run = -1; @@ -346,8 +386,29 @@ int main(int argc, char** argv) { GetTempDir(), ((int) geteuid())); + snprintf(dbbackupname, sizeof(dbbackupname), + "%s/rocksdb_c_test-%d-backup", + GetTempDir(), + ((int) geteuid())); + + snprintf(dbcheckpointname, sizeof(dbcheckpointname), + "%s/rocksdb_c_test-%d-checkpoint", + GetTempDir(), + ((int) geteuid())); + + snprintf(sstfilename, sizeof(sstfilename), + "%s/rocksdb_c_test-%d-sst", + GetTempDir(), + ((int)geteuid())); + + snprintf(dbpathname, sizeof(dbpathname), + "%s/rocksdb_c_test-%d-dbpath", + GetTempDir(), + ((int) geteuid())); + StartPhase("create_objects"); cmp = rocksdb_comparator_create(NULL, CmpDestroy, CmpCompare, CmpName); + dbpath = rocksdb_dbpath_create(dbpathname, 1024 * 1024); env = rocksdb_create_default_env(); cache = rocksdb_cache_create_lru(100000); @@ -359,29 +420,36 @@ int main(int argc, char** argv) { rocksdb_options_set_write_buffer_size(options, 100000); rocksdb_options_set_paranoid_checks(options, 1); rocksdb_options_set_max_open_files(options, 10); + rocksdb_options_set_base_background_compactions(options, 1); table_options = rocksdb_block_based_options_create(); rocksdb_block_based_options_set_block_cache(table_options, cache); rocksdb_options_set_block_based_table_factory(options, table_options); rocksdb_options_set_compression(options, rocksdb_no_compression); - rocksdb_options_set_compression_options(options, -14, -1, 0); + rocksdb_options_set_compression_options(options, -14, -1, 0, 0); int compression_levels[] = {rocksdb_no_compression, rocksdb_no_compression, rocksdb_no_compression, rocksdb_no_compression}; rocksdb_options_set_compression_per_level(options, compression_levels, 4); + rate_limiter = rocksdb_ratelimiter_create(1000 * 1024 * 1024, 100 * 1000, 10); + rocksdb_options_set_ratelimiter(options, rate_limiter); + rocksdb_ratelimiter_destroy(rate_limiter); roptions = rocksdb_readoptions_create(); rocksdb_readoptions_set_verify_checksums(roptions, 1); - rocksdb_readoptions_set_fill_cache(roptions, 0); + rocksdb_readoptions_set_fill_cache(roptions, 1); woptions = rocksdb_writeoptions_create(); rocksdb_writeoptions_set_sync(woptions, 1); + coptions = rocksdb_compactoptions_create(); + rocksdb_compactoptions_set_exclusive_manual_compaction(coptions, 1); + StartPhase("destroy"); rocksdb_destroy_db(options, dbname, &err); Free(&err); StartPhase("open_error"); - db = rocksdb_open(options, dbname, &err); + rocksdb_open(options, dbname, &err); CheckCondition(err != NULL); Free(&err); @@ -396,6 +464,89 @@ int main(int argc, char** argv) { CheckNoError(err); CheckGet(db, roptions, "foo", "hello"); + StartPhase("backup_and_restore"); + { + rocksdb_destroy_db(options, dbbackupname, &err); + CheckNoError(err); + + rocksdb_backup_engine_t *be = rocksdb_backup_engine_open(options, dbbackupname, &err); + CheckNoError(err); + + rocksdb_backup_engine_create_new_backup(be, db, &err); + CheckNoError(err); + + // need a change to trigger a new backup + rocksdb_delete(db, woptions, "does-not-exist", 14, &err); + CheckNoError(err); + + rocksdb_backup_engine_create_new_backup(be, db, &err); + CheckNoError(err); + + const rocksdb_backup_engine_info_t* bei = rocksdb_backup_engine_get_backup_info(be); + CheckCondition(rocksdb_backup_engine_info_count(bei) > 1); + rocksdb_backup_engine_info_destroy(bei); + + rocksdb_backup_engine_purge_old_backups(be, 1, &err); + CheckNoError(err); + + bei = rocksdb_backup_engine_get_backup_info(be); + CheckCondition(rocksdb_backup_engine_info_count(bei) == 1); + rocksdb_backup_engine_info_destroy(bei); + + rocksdb_delete(db, woptions, "foo", 3, &err); + CheckNoError(err); + + rocksdb_close(db); + + rocksdb_destroy_db(options, dbname, &err); + CheckNoError(err); + + rocksdb_restore_options_t *restore_options = rocksdb_restore_options_create(); + rocksdb_restore_options_set_keep_log_files(restore_options, 0); + rocksdb_backup_engine_restore_db_from_latest_backup(be, dbname, dbname, restore_options, &err); + CheckNoError(err); + rocksdb_restore_options_destroy(restore_options); + + rocksdb_options_set_error_if_exists(options, 0); + db = rocksdb_open(options, dbname, &err); + CheckNoError(err); + rocksdb_options_set_error_if_exists(options, 1); + + CheckGet(db, roptions, "foo", "hello"); + + rocksdb_backup_engine_close(be); + } + + StartPhase("checkpoint"); + { + rocksdb_destroy_db(options, dbcheckpointname, &err); + CheckNoError(err); + + rocksdb_checkpoint_t* checkpoint = rocksdb_checkpoint_object_create(db, &err); + CheckNoError(err); + + rocksdb_checkpoint_create(checkpoint, dbcheckpointname, 0, &err); + CheckNoError(err); + + // start a new database from the checkpoint + rocksdb_close(db); + rocksdb_options_set_error_if_exists(options, 0); + db = rocksdb_open(options, dbcheckpointname, &err); + CheckNoError(err); + + CheckGet(db, roptions, "foo", "hello"); + + rocksdb_checkpoint_object_destroy(checkpoint); + + rocksdb_close(db); + rocksdb_destroy_db(options, dbcheckpointname, &err); + CheckNoError(err); + + db = rocksdb_open(options, dbname, &err); + CheckNoError(err); + rocksdb_options_set_error_if_exists(options, 1); + } + StartPhase("compactall"); rocksdb_compact_range(db, NULL, 0, NULL, 0); CheckGet(db, roptions, "foo", "hello"); @@ -404,6 +555,95 @@ int main(int argc, char** argv) { rocksdb_compact_range(db, "a", 1, "z", 1); CheckGet(db, roptions, "foo", "hello"); + StartPhase("compactallopt"); + rocksdb_compact_range_opt(db, coptions, NULL, 0, NULL, 0); + CheckGet(db, roptions, "foo", "hello"); + + StartPhase("compactrangeopt"); + rocksdb_compact_range_opt(db, coptions, "a", 1, "z", 1); + CheckGet(db, roptions, "foo", "hello"); + + // Simple check cache usage + StartPhase("cache_usage"); + { + rocksdb_readoptions_set_pin_data(roptions, 1); + rocksdb_iterator_t* iter = rocksdb_create_iterator(db, roptions); + rocksdb_iter_seek(iter, "foo", 3); + + size_t usage = rocksdb_cache_get_usage(cache); + CheckCondition(usage > 0); + + size_t pin_usage = rocksdb_cache_get_pinned_usage(cache); + CheckCondition(pin_usage > 0); + + rocksdb_iter_next(iter); + rocksdb_iter_destroy(iter); + rocksdb_readoptions_set_pin_data(roptions, 0); + } + + StartPhase("addfile"); + { + rocksdb_envoptions_t* env_opt = rocksdb_envoptions_create(); + rocksdb_options_t* io_options = rocksdb_options_create(); + rocksdb_sstfilewriter_t* writer = + rocksdb_sstfilewriter_create(env_opt, io_options); + + unlink(sstfilename); + rocksdb_sstfilewriter_open(writer, sstfilename, &err); + CheckNoError(err); + rocksdb_sstfilewriter_put(writer, "sstk1", 5, "v1", 2, &err); + CheckNoError(err); + rocksdb_sstfilewriter_put(writer, "sstk2", 5, "v2", 2, &err); + CheckNoError(err); + rocksdb_sstfilewriter_put(writer, "sstk3", 5, "v3", 2, &err); + CheckNoError(err); + rocksdb_sstfilewriter_finish(writer, &err); + CheckNoError(err); + + rocksdb_ingestexternalfileoptions_t* ing_opt = + rocksdb_ingestexternalfileoptions_create(); + const char* file_list[1] = {sstfilename}; + rocksdb_ingest_external_file(db, file_list, 1, ing_opt, &err); + CheckNoError(err); + CheckGet(db, roptions, "sstk1", "v1"); + CheckGet(db, roptions, "sstk2", "v2"); + CheckGet(db, roptions, "sstk3", "v3"); + + unlink(sstfilename); + rocksdb_sstfilewriter_open(writer, sstfilename, &err); + CheckNoError(err); + rocksdb_sstfilewriter_put(writer, "sstk2", 5, "v4", 2, &err); + CheckNoError(err); + rocksdb_sstfilewriter_put(writer, "sstk22", 6, "v5", 2, &err); + CheckNoError(err); + rocksdb_sstfilewriter_put(writer, "sstk3", 5, "v6", 2, &err); + CheckNoError(err); + rocksdb_sstfilewriter_finish(writer, &err); + CheckNoError(err); + + rocksdb_ingest_external_file(db, file_list, 1, ing_opt, &err); + CheckNoError(err); + CheckGet(db, roptions, "sstk1", "v1"); + CheckGet(db, roptions, "sstk2", "v4"); + CheckGet(db, roptions, "sstk22", "v5"); + CheckGet(db, roptions, "sstk3", "v6"); + + rocksdb_ingestexternalfileoptions_destroy(ing_opt); + rocksdb_sstfilewriter_destroy(writer); + rocksdb_options_destroy(io_options); + rocksdb_envoptions_destroy(env_opt); + + // Delete all keys we just ingested + rocksdb_delete(db, woptions, "sstk1", 5, &err); + CheckNoError(err); + rocksdb_delete(db, woptions, "sstk2", 5, &err); + CheckNoError(err); + rocksdb_delete(db, woptions, "sstk22", 6, &err); + CheckNoError(err); + rocksdb_delete(db, woptions, "sstk3", 5, &err); + CheckNoError(err); + } + StartPhase("writebatch"); { rocksdb_writebatch_t* wb = rocksdb_writebatch_create(); @@ -420,6 +660,62 @@ int main(int argc, char** argv) { int pos = 0; rocksdb_writebatch_iterate(wb, &pos, CheckPut, CheckDel); CheckCondition(pos == 3); + rocksdb_writebatch_clear(wb); + rocksdb_writebatch_put(wb, "bar", 3, "b", 1); + rocksdb_writebatch_put(wb, "bay", 3, "d", 1); + rocksdb_writebatch_delete_range(wb, "bar", 3, "bay", 3); + rocksdb_write(db, woptions, wb, &err); + CheckNoError(err); + CheckGet(db, roptions, "bar", NULL); + CheckGet(db, roptions, "bay", "d"); + rocksdb_writebatch_clear(wb); + const char* start_list[1] = {"bay"}; + const size_t start_sizes[1] = {3}; + const char* end_list[1] = {"baz"}; + const size_t end_sizes[1] = {3}; + rocksdb_writebatch_delete_rangev(wb, 1, start_list, start_sizes, end_list, + end_sizes); + rocksdb_write(db, woptions, wb, &err); + CheckNoError(err); + CheckGet(db, roptions, "bay", NULL); + rocksdb_writebatch_destroy(wb); + } + + StartPhase("writebatch_vectors"); + { + rocksdb_writebatch_t* wb = rocksdb_writebatch_create(); + const char* k_list[2] = { "z", "ap" }; + const size_t k_sizes[2] = { 1, 2 }; + const char* v_list[3] = { "x", "y", "z" }; + const size_t v_sizes[3] = { 1, 1, 1 }; + rocksdb_writebatch_putv(wb, 2, k_list, k_sizes, 3, v_list, v_sizes); + rocksdb_write(db, woptions, wb, &err); + CheckNoError(err); + CheckGet(db, roptions, "zap", "xyz"); + rocksdb_writebatch_delete(wb, "zap", 3); + rocksdb_write(db, woptions, wb, &err); + CheckNoError(err); + CheckGet(db, roptions, "zap", NULL); + rocksdb_writebatch_destroy(wb); + } + + StartPhase("writebatch_savepoint"); + { + rocksdb_writebatch_t* wb = rocksdb_writebatch_create(); + rocksdb_writebatch_set_save_point(wb); + rocksdb_writebatch_set_save_point(wb); + const char* k_list[2] = {"z", "ap"}; + const size_t k_sizes[2] = {1, 2}; + const char* v_list[3] = {"x", "y", "z"}; + const size_t v_sizes[3] = {1, 1, 1}; + rocksdb_writebatch_pop_save_point(wb, &err); + CheckNoError(err); + rocksdb_writebatch_putv(wb, 2, k_list, k_sizes, 3, v_list, v_sizes); + rocksdb_writebatch_rollback_to_save_point(wb, &err); + CheckNoError(err); + rocksdb_write(db, woptions, wb, &err); + CheckNoError(err); + CheckGet(db, roptions, "zap", NULL); rocksdb_writebatch_destroy(wb); } @@ -441,6 +737,90 @@ int main(int argc, char** argv) { rocksdb_writebatch_destroy(wb2); } + StartPhase("writebatch_wi"); + { + rocksdb_writebatch_wi_t* wbi = rocksdb_writebatch_wi_create(0, 1); + rocksdb_writebatch_wi_put(wbi, "foo", 3, "a", 1); + rocksdb_writebatch_wi_clear(wbi); + rocksdb_writebatch_wi_put(wbi, "bar", 3, "b", 1); + rocksdb_writebatch_wi_put(wbi, "box", 3, "c", 1); + rocksdb_writebatch_wi_delete(wbi, "bar", 3); + int count = rocksdb_writebatch_wi_count(wbi); + CheckCondition(count == 3); + size_t size; + char* value; + value = rocksdb_writebatch_wi_get_from_batch(wbi, options, "box", 3, &size, &err); + CheckValue(err, "c", &value, size); + value = rocksdb_writebatch_wi_get_from_batch(wbi, options, "bar", 3, &size, &err); + CheckValue(err, NULL, &value, size); + value = rocksdb_writebatch_wi_get_from_batch_and_db(wbi, db, roptions, "foo", 3, &size, &err); + CheckValue(err, "hello", &value, size); + value = rocksdb_writebatch_wi_get_from_batch_and_db(wbi, db, roptions, "box", 3, &size, &err); + CheckValue(err, "c", &value, size); + rocksdb_write_writebatch_wi(db, woptions, wbi, &err); + CheckNoError(err); + CheckGet(db, roptions, "foo", "hello"); + CheckGet(db, roptions, "bar", NULL); + CheckGet(db, roptions, "box", "c"); + int pos = 0; + rocksdb_writebatch_wi_iterate(wbi, &pos, CheckPut, CheckDel); + CheckCondition(pos == 3); + rocksdb_writebatch_wi_clear(wbi); + rocksdb_writebatch_wi_put(wbi, "bar", 3, "b", 1); + rocksdb_writebatch_wi_put(wbi, "bay", 3, "d", 1); + rocksdb_writebatch_wi_delete_range(wbi, "bar", 3, "bay", 3); + rocksdb_write_writebatch_wi(db, woptions, wbi, &err); + CheckNoError(err); + CheckGet(db, roptions, "bar", NULL); + CheckGet(db, roptions, "bay", "d"); + rocksdb_writebatch_wi_clear(wbi); + const char* start_list[1] = {"bay"}; + const size_t start_sizes[1] = {3}; + const char* end_list[1] = {"baz"}; + const size_t end_sizes[1] = {3}; + rocksdb_writebatch_wi_delete_rangev(wbi, 1, start_list, start_sizes, end_list, + end_sizes); + rocksdb_write_writebatch_wi(db, woptions, wbi, &err); + CheckNoError(err); + CheckGet(db, roptions, "bay", NULL); + rocksdb_writebatch_wi_destroy(wbi); + } + + StartPhase("writebatch_wi_vectors"); + { + rocksdb_writebatch_wi_t* wb = rocksdb_writebatch_wi_create(0, 1); + const char* k_list[2] = { "z", "ap" }; + const size_t k_sizes[2] = { 1, 2 }; + const char* v_list[3] = { "x", "y", "z" }; + const size_t v_sizes[3] = { 1, 1, 1 }; + rocksdb_writebatch_wi_putv(wb, 2, k_list, k_sizes, 3, v_list, v_sizes); + rocksdb_write_writebatch_wi(db, woptions, wb, &err); + CheckNoError(err); + CheckGet(db, roptions, "zap", "xyz"); + rocksdb_writebatch_wi_delete(wb, "zap", 3); + rocksdb_write_writebatch_wi(db, woptions, wb, &err); + CheckNoError(err); + CheckGet(db, roptions, "zap", NULL); + rocksdb_writebatch_wi_destroy(wb); + } + + StartPhase("writebatch_wi_savepoint"); + { + rocksdb_writebatch_wi_t* wb = rocksdb_writebatch_wi_create(0, 1); + rocksdb_writebatch_wi_set_save_point(wb); + const char* k_list[2] = {"z", "ap"}; + const size_t k_sizes[2] = {1, 2}; + const char* v_list[3] = {"x", "y", "z"}; + const size_t v_sizes[3] = {1, 1, 1}; + rocksdb_writebatch_wi_putv(wb, 2, k_list, k_sizes, 3, v_list, v_sizes); + rocksdb_writebatch_wi_rollback_to_save_point(wb, &err); + CheckNoError(err); + rocksdb_write_writebatch_wi(db, woptions, wb, &err); + CheckNoError(err); + CheckGet(db, roptions, "zap", NULL); + rocksdb_writebatch_wi_destroy(wb); + } + StartPhase("iter"); { rocksdb_iterator_t* iter = rocksdb_create_iterator(db, roptions); @@ -458,9 +838,78 @@ int main(int argc, char** argv) { CheckIter(iter, "foo", "hello"); rocksdb_iter_seek(iter, "b", 1); CheckIter(iter, "box", "c"); + rocksdb_iter_seek_for_prev(iter, "g", 1); + CheckIter(iter, "foo", "hello"); + rocksdb_iter_seek_for_prev(iter, "box", 3); + CheckIter(iter, "box", "c"); + rocksdb_iter_get_error(iter, &err); + CheckNoError(err); + rocksdb_iter_destroy(iter); + } + + StartPhase("wbwi_iter"); + { + rocksdb_iterator_t* base_iter = rocksdb_create_iterator(db, roptions); + rocksdb_writebatch_wi_t* wbi = rocksdb_writebatch_wi_create(0, 1); + rocksdb_writebatch_wi_put(wbi, "bar", 3, "b", 1); + rocksdb_writebatch_wi_delete(wbi, "foo", 3); + rocksdb_iterator_t* iter = rocksdb_writebatch_wi_create_iterator_with_base(wbi, base_iter); + CheckCondition(!rocksdb_iter_valid(iter)); + rocksdb_iter_seek_to_first(iter); + CheckCondition(rocksdb_iter_valid(iter)); + CheckIter(iter, "bar", "b"); + rocksdb_iter_next(iter); + CheckIter(iter, "box", "c"); + rocksdb_iter_prev(iter); + CheckIter(iter, "bar", "b"); + rocksdb_iter_prev(iter); + CheckCondition(!rocksdb_iter_valid(iter)); + rocksdb_iter_seek_to_last(iter); + CheckIter(iter, "box", "c"); + rocksdb_iter_seek(iter, "b", 1); + CheckIter(iter, "bar", "b"); + rocksdb_iter_seek_for_prev(iter, "c", 1); + CheckIter(iter, "box", "c"); + rocksdb_iter_seek_for_prev(iter, "box", 3); + CheckIter(iter, "box", "c"); rocksdb_iter_get_error(iter, &err); CheckNoError(err); rocksdb_iter_destroy(iter); + rocksdb_writebatch_wi_destroy(wbi); + } + + StartPhase("multiget"); + { + const char* keys[3] = { "box", "foo", "notfound" }; + const size_t keys_sizes[3] = { 3, 3, 8 }; + char* vals[3]; + size_t vals_sizes[3]; + char* errs[3]; + rocksdb_multi_get(db, roptions, 3, keys, keys_sizes, vals, vals_sizes, errs); + + int i; + for (i = 0; i < 3; i++) { + CheckEqual(NULL, errs[i], 0); + switch (i) { + case 0: + CheckEqual("c", vals[i], vals_sizes[i]); + break; + case 1: + CheckEqual("hello", vals[i], vals_sizes[i]); + break; + case 2: + CheckEqual(NULL, vals[i], vals_sizes[i]); + break; + } + Free(&vals[i]); + } + } + + StartPhase("pin_get"); + { + CheckPinGet(db, roptions, "box", "c"); + CheckPinGet(db, roptions, "foo", "hello"); + CheckPinGet(db, roptions, "notfound", NULL); } StartPhase("approximate_sizes"); @@ -519,6 +968,7 @@ int main(int argc, char** argv) { rocksdb_close(db); rocksdb_options_set_create_if_missing(options, 0); rocksdb_options_set_error_if_exists(options, 0); + rocksdb_options_set_wal_recovery_mode(options, 2); rocksdb_repair_db(options, dbname, &err); CheckNoError(err); db = rocksdb_open(options, dbname, &err); @@ -576,81 +1026,39 @@ int main(int argc, char** argv) { StartPhase("compaction_filter"); { - rocksdb_options_t* options = rocksdb_options_create(); - rocksdb_options_set_create_if_missing(options, 1); + rocksdb_options_t* options_with_filter = rocksdb_options_create(); + rocksdb_options_set_create_if_missing(options_with_filter, 1); rocksdb_compactionfilter_t* cfilter; cfilter = rocksdb_compactionfilter_create(NULL, CFilterDestroy, CFilterFilter, CFilterName); // Create new database rocksdb_close(db); - rocksdb_destroy_db(options, dbname, &err); - rocksdb_options_set_compaction_filter(options, cfilter); - db = CheckCompaction(db, options, roptions, woptions); + rocksdb_destroy_db(options_with_filter, dbname, &err); + rocksdb_options_set_compaction_filter(options_with_filter, cfilter); + db = CheckCompaction(db, options_with_filter, roptions, woptions); - rocksdb_options_set_compaction_filter(options, NULL); + rocksdb_options_set_compaction_filter(options_with_filter, NULL); rocksdb_compactionfilter_destroy(cfilter); - rocksdb_options_destroy(options); + rocksdb_options_destroy(options_with_filter); } StartPhase("compaction_filter_factory"); { - rocksdb_options_t* options = rocksdb_options_create(); - rocksdb_options_set_create_if_missing(options, 1); + rocksdb_options_t* options_with_filter_factory = rocksdb_options_create(); + rocksdb_options_set_create_if_missing(options_with_filter_factory, 1); rocksdb_compactionfilterfactory_t* factory; factory = rocksdb_compactionfilterfactory_create( NULL, CFilterFactoryDestroy, CFilterCreate, CFilterFactoryName); // Create new database rocksdb_close(db); - rocksdb_destroy_db(options, dbname, &err); - rocksdb_options_set_compaction_filter_factory(options, factory); - db = CheckCompaction(db, options, roptions, woptions); - - rocksdb_options_set_compaction_filter_factory(options, NULL); - rocksdb_options_destroy(options); - } - - StartPhase("compaction_filter_v2"); - { - rocksdb_compactionfilterfactoryv2_t* factory; - rocksdb_slicetransform_t* prefix_extractor; - prefix_extractor = rocksdb_slicetransform_create( - NULL, CFV2PrefixExtractorDestroy, CFV2PrefixExtractorTransform, - CFV2PrefixExtractorInDomain, CFV2PrefixExtractorInRange, - CFV2PrefixExtractorName); - factory = rocksdb_compactionfilterfactoryv2_create( - prefix_extractor, prefix_extractor, CompactionFilterFactoryV2Destroy, - CompactionFilterFactoryV2Create, CompactionFilterFactoryV2Name); - // Create new database - rocksdb_close(db); - rocksdb_destroy_db(options, dbname, &err); - rocksdb_options_set_compaction_filter_factory_v2(options, factory); - db = rocksdb_open(options, dbname, &err); - CheckNoError(err); - // Only foo2 is GC'd, foo3 is changed. - rocksdb_put(db, woptions, "foo1", 4, "no gc", 5, &err); - CheckNoError(err); - rocksdb_put(db, woptions, "foo2", 4, "gc", 2, &err); - CheckNoError(err); - rocksdb_put(db, woptions, "foo3", 4, "change", 6, &err); - CheckNoError(err); - // All bars are GC'd. - rocksdb_put(db, woptions, "bar1", 4, "no gc", 5, &err); - CheckNoError(err); - rocksdb_put(db, woptions, "bar2", 4, "gc all", 6, &err); - CheckNoError(err); - rocksdb_put(db, woptions, "bar3", 4, "no gc", 5, &err); - CheckNoError(err); - // Compact the DB to garbage collect. - rocksdb_compact_range(db, NULL, 0, NULL, 0); - - // Verify foo entries. - CheckGet(db, roptions, "foo1", "no gc"); - CheckGet(db, roptions, "foo2", NULL); - CheckGet(db, roptions, "foo3", "changed"); - // Verify bar entries were all deleted. - CheckGet(db, roptions, "bar1", NULL); - CheckGet(db, roptions, "bar2", NULL); - CheckGet(db, roptions, "bar3", NULL); + rocksdb_destroy_db(options_with_filter_factory, dbname, &err); + rocksdb_options_set_compaction_filter_factory(options_with_filter_factory, + factory); + db = CheckCompaction(db, options_with_filter_factory, roptions, woptions); + + rocksdb_options_set_compaction_filter_factory( + options_with_filter_factory, NULL); + rocksdb_options_destroy(options_with_filter_factory); } StartPhase("merge_operator"); @@ -683,7 +1091,7 @@ int main(int argc, char** argv) { { rocksdb_close(db); rocksdb_destroy_db(options, dbname, &err); - CheckNoError(err) + CheckNoError(err); rocksdb_options_t* db_options = rocksdb_options_create(); rocksdb_options_set_create_if_missing(db_options, 1); @@ -715,11 +1123,13 @@ int main(int argc, char** argv) { CheckNoError(err); CheckGetCF(db, roptions, handles[1], "foo", "hello"); + CheckPinGetCF(db, roptions, handles[1], "foo", "hello"); rocksdb_delete_cf(db, woptions, handles[1], "foo", 3, &err); CheckNoError(err); CheckGetCF(db, roptions, handles[1], "foo", NULL); + CheckPinGetCF(db, roptions, handles[1], "foo", NULL); rocksdb_writebatch_t* wb = rocksdb_writebatch_create(); rocksdb_writebatch_put_cf(wb, handles[1], "baz", 3, "a", 1); @@ -732,14 +1142,65 @@ int main(int argc, char** argv) { CheckGetCF(db, roptions, handles[1], "baz", NULL); CheckGetCF(db, roptions, handles[1], "bar", NULL); CheckGetCF(db, roptions, handles[1], "box", "c"); + CheckPinGetCF(db, roptions, handles[1], "baz", NULL); + CheckPinGetCF(db, roptions, handles[1], "bar", NULL); + CheckPinGetCF(db, roptions, handles[1], "box", "c"); rocksdb_writebatch_destroy(wb); + const char* keys[3] = { "box", "box", "barfooxx" }; + const rocksdb_column_family_handle_t* get_handles[3] = { handles[0], handles[1], handles[1] }; + const size_t keys_sizes[3] = { 3, 3, 8 }; + char* vals[3]; + size_t vals_sizes[3]; + char* errs[3]; + rocksdb_multi_get_cf(db, roptions, get_handles, 3, keys, keys_sizes, vals, vals_sizes, errs); + + int i; + for (i = 0; i < 3; i++) { + CheckEqual(NULL, errs[i], 0); + switch (i) { + case 0: + CheckEqual(NULL, vals[i], vals_sizes[i]); // wrong cf + break; + case 1: + CheckEqual("c", vals[i], vals_sizes[i]); // bingo + break; + case 2: + CheckEqual(NULL, vals[i], vals_sizes[i]); // normal not found + break; + } + Free(&vals[i]); + } + rocksdb_iterator_t* iter = rocksdb_create_iterator_cf(db, roptions, handles[1]); CheckCondition(!rocksdb_iter_valid(iter)); rocksdb_iter_seek_to_first(iter); CheckCondition(rocksdb_iter_valid(iter)); - int i; + for (i = 0; rocksdb_iter_valid(iter) != 0; rocksdb_iter_next(iter)) { + i++; + } + CheckCondition(i == 1); + rocksdb_iter_get_error(iter, &err); + CheckNoError(err); + rocksdb_iter_destroy(iter); + + rocksdb_column_family_handle_t* iters_cf_handles[2] = { handles[0], handles[1] }; + rocksdb_iterator_t* iters_handles[2]; + rocksdb_create_iterators(db, roptions, iters_cf_handles, iters_handles, 2, &err); + CheckNoError(err); + + iter = iters_handles[0]; + CheckCondition(!rocksdb_iter_valid(iter)); + rocksdb_iter_seek_to_first(iter); + CheckCondition(!rocksdb_iter_valid(iter)); + rocksdb_iter_destroy(iter); + + iter = iters_handles[1]; + CheckCondition(!rocksdb_iter_valid(iter)); + rocksdb_iter_seek_to_first(iter); + CheckCondition(rocksdb_iter_valid(iter)); + for (i = 0; rocksdb_iter_valid(iter) != 0; rocksdb_iter_next(iter)) { i++; } @@ -766,6 +1227,7 @@ int main(int argc, char** argv) { rocksdb_options_set_prefix_extractor(options, rocksdb_slicetransform_create_fixed_prefix(3)); rocksdb_options_set_hash_skip_list_rep(options, 5000, 4, 4); rocksdb_options_set_plain_table_factory(options, 4, 10, 0.75, 16); + rocksdb_options_set_allow_concurrent_memtable_write(options, 0); db = rocksdb_open(options, dbname, &err); CheckNoError(err); @@ -799,19 +1261,269 @@ int main(int argc, char** argv) { rocksdb_iter_get_error(iter, &err); CheckNoError(err); rocksdb_iter_destroy(iter); + + rocksdb_readoptions_set_total_order_seek(roptions, 1); + iter = rocksdb_create_iterator(db, roptions); + CheckCondition(!rocksdb_iter_valid(iter)); + + rocksdb_iter_seek(iter, "ba", 2); + rocksdb_iter_get_error(iter, &err); + CheckNoError(err); + CheckCondition(rocksdb_iter_valid(iter)); + CheckIter(iter, "bar1", "bar"); + + rocksdb_iter_destroy(iter); + rocksdb_readoptions_set_total_order_seek(roptions, 0); + + rocksdb_close(db); + rocksdb_destroy_db(options, dbname, &err); + } + + StartPhase("cuckoo_options"); + { + rocksdb_cuckoo_table_options_t* cuckoo_options; + cuckoo_options = rocksdb_cuckoo_options_create(); + rocksdb_cuckoo_options_set_hash_ratio(cuckoo_options, 0.5); + rocksdb_cuckoo_options_set_max_search_depth(cuckoo_options, 200); + rocksdb_cuckoo_options_set_cuckoo_block_size(cuckoo_options, 10); + rocksdb_cuckoo_options_set_identity_as_first_hash(cuckoo_options, 1); + rocksdb_cuckoo_options_set_use_module_hash(cuckoo_options, 0); + rocksdb_options_set_cuckoo_table_factory(options, cuckoo_options); + + db = rocksdb_open(options, dbname, &err); + CheckNoError(err); + + rocksdb_cuckoo_options_destroy(cuckoo_options); + } + + StartPhase("iterate_upper_bound"); + { + // Create new empty database + rocksdb_close(db); + rocksdb_destroy_db(options, dbname, &err); + CheckNoError(err); + + rocksdb_options_set_prefix_extractor(options, NULL); + db = rocksdb_open(options, dbname, &err); + CheckNoError(err); + + rocksdb_put(db, woptions, "a", 1, "0", 1, &err); CheckNoError(err); + rocksdb_put(db, woptions, "foo", 3, "bar", 3, &err); CheckNoError(err); + rocksdb_put(db, woptions, "foo1", 4, "bar1", 4, &err); CheckNoError(err); + rocksdb_put(db, woptions, "g1", 2, "0", 1, &err); CheckNoError(err); + + // testing basic case with no iterate_upper_bound and no prefix_extractor + { + rocksdb_readoptions_set_iterate_upper_bound(roptions, NULL, 0); + rocksdb_iterator_t* iter = rocksdb_create_iterator(db, roptions); + + rocksdb_iter_seek(iter, "foo", 3); + CheckCondition(rocksdb_iter_valid(iter)); + CheckIter(iter, "foo", "bar"); + + rocksdb_iter_next(iter); + CheckCondition(rocksdb_iter_valid(iter)); + CheckIter(iter, "foo1", "bar1"); + + rocksdb_iter_next(iter); + CheckCondition(rocksdb_iter_valid(iter)); + CheckIter(iter, "g1", "0"); + + rocksdb_iter_destroy(iter); + } + + // testing iterate_upper_bound and forward iterator + // to make sure it stops at bound + { + // iterate_upper_bound points beyond the last expected entry + rocksdb_readoptions_set_iterate_upper_bound(roptions, "foo2", 4); + + rocksdb_iterator_t* iter = rocksdb_create_iterator(db, roptions); + + rocksdb_iter_seek(iter, "foo", 3); + CheckCondition(rocksdb_iter_valid(iter)); + CheckIter(iter, "foo", "bar"); + + rocksdb_iter_next(iter); + CheckCondition(rocksdb_iter_valid(iter)); + CheckIter(iter, "foo1", "bar1"); + + rocksdb_iter_next(iter); + // should stop here... + CheckCondition(!rocksdb_iter_valid(iter)); + + rocksdb_iter_destroy(iter); + } + } + + StartPhase("transactions"); + { + rocksdb_close(db); + rocksdb_destroy_db(options, dbname, &err); + CheckNoError(err); + + // open a TransactionDB + txn_db_options = rocksdb_transactiondb_options_create(); + txn_options = rocksdb_transaction_options_create(); + rocksdb_options_set_create_if_missing(options, 1); + txn_db = rocksdb_transactiondb_open(options, txn_db_options, dbname, &err); + CheckNoError(err); + + // put outside a transaction + rocksdb_transactiondb_put(txn_db, woptions, "foo", 3, "hello", 5, &err); + CheckNoError(err); + CheckTxnDBGet(txn_db, roptions, "foo", "hello"); + + // delete from outside transaction + rocksdb_transactiondb_delete(txn_db, woptions, "foo", 3, &err); + CheckNoError(err); + CheckTxnDBGet(txn_db, roptions, "foo", NULL); + + // write batch into TransactionDB + rocksdb_writebatch_t* wb = rocksdb_writebatch_create(); + rocksdb_writebatch_put(wb, "foo", 3, "a", 1); + rocksdb_writebatch_clear(wb); + rocksdb_writebatch_put(wb, "bar", 3, "b", 1); + rocksdb_writebatch_put(wb, "box", 3, "c", 1); + rocksdb_writebatch_delete(wb, "bar", 3); + rocksdb_transactiondb_write(txn_db, woptions, wb, &err); + rocksdb_writebatch_destroy(wb); + CheckTxnDBGet(txn_db, roptions, "box", "c"); + CheckNoError(err); + + // begin a transaction + txn = rocksdb_transaction_begin(txn_db, woptions, txn_options, NULL); + // put + rocksdb_transaction_put(txn, "foo", 3, "hello", 5, &err); + CheckNoError(err); + CheckTxnGet(txn, roptions, "foo", "hello"); + // delete + rocksdb_transaction_delete(txn, "foo", 3, &err); + CheckNoError(err); + CheckTxnGet(txn, roptions, "foo", NULL); + + rocksdb_transaction_put(txn, "foo", 3, "hello", 5, &err); + CheckNoError(err); + + // read from outside transaction, before commit + CheckTxnDBGet(txn_db, roptions, "foo", NULL); + + // commit + rocksdb_transaction_commit(txn, &err); + CheckNoError(err); + + // read from outside transaction, after commit + CheckTxnDBGet(txn_db, roptions, "foo", "hello"); + + // reuse old transaction + txn = rocksdb_transaction_begin(txn_db, woptions, txn_options, txn); + + // snapshot + const rocksdb_snapshot_t* snapshot; + snapshot = rocksdb_transactiondb_create_snapshot(txn_db); + rocksdb_readoptions_set_snapshot(roptions, snapshot); + + rocksdb_transactiondb_put(txn_db, woptions, "foo", 3, "hey", 3, &err); + CheckNoError(err); + + CheckTxnDBGet(txn_db, roptions, "foo", "hello"); + rocksdb_readoptions_set_snapshot(roptions, NULL); + rocksdb_transactiondb_release_snapshot(txn_db, snapshot); + CheckTxnDBGet(txn_db, roptions, "foo", "hey"); + + // iterate + rocksdb_transaction_put(txn, "bar", 3, "hi", 2, &err); + rocksdb_iterator_t* iter = rocksdb_transaction_create_iterator(txn, roptions); + CheckCondition(!rocksdb_iter_valid(iter)); + rocksdb_iter_seek_to_first(iter); + CheckCondition(rocksdb_iter_valid(iter)); + CheckIter(iter, "bar", "hi"); + rocksdb_iter_get_error(iter, &err); + CheckNoError(err); + rocksdb_iter_destroy(iter); + + // rollback + rocksdb_transaction_rollback(txn, &err); + CheckNoError(err); + CheckTxnDBGet(txn_db, roptions, "bar", NULL); + + // Column families. + rocksdb_column_family_handle_t* cfh; + cfh = rocksdb_transactiondb_create_column_family(txn_db, options, + "txn_db_cf", &err); + CheckNoError(err); + + rocksdb_transactiondb_put_cf(txn_db, woptions, cfh, "cf_foo", 6, "cf_hello", + 8, &err); + CheckNoError(err); + CheckTxnDBGetCF(txn_db, roptions, cfh, "cf_foo", "cf_hello"); + + rocksdb_transactiondb_delete_cf(txn_db, woptions, cfh, "cf_foo", 6, &err); + CheckNoError(err); + CheckTxnDBGetCF(txn_db, roptions, cfh, "cf_foo", NULL); + + rocksdb_column_family_handle_destroy(cfh); + + // close and destroy + rocksdb_transaction_destroy(txn); + rocksdb_transactiondb_close(txn_db); + rocksdb_destroy_db(options, dbname, &err); + CheckNoError(err); + rocksdb_transaction_options_destroy(txn_options); + rocksdb_transactiondb_options_destroy(txn_db_options); } + // Simple sanity check that setting memtable rep works. + StartPhase("memtable_reps"); + { + // Create database with vector memtable. + rocksdb_options_set_memtable_vector_rep(options); + db = rocksdb_open(options, dbname, &err); + CheckNoError(err); + + // Create database with hash skiplist memtable. + rocksdb_close(db); + rocksdb_destroy_db(options, dbname, &err); + CheckNoError(err); + + rocksdb_options_set_hash_skip_list_rep(options, 5000, 4, 4); + db = rocksdb_open(options, dbname, &err); + CheckNoError(err); + } + // Simple sanity check that options setting db_paths work. + StartPhase("open_db_paths"); + { + rocksdb_close(db); + rocksdb_destroy_db(options, dbname, &err); + + const rocksdb_dbpath_t* paths[1] = {dbpath}; + rocksdb_options_set_db_paths(options, paths, 1); + db = rocksdb_open(options, dbname, &err); + CheckNoError(err); + } + StartPhase("cleanup"); rocksdb_close(db); rocksdb_options_destroy(options); rocksdb_block_based_options_destroy(table_options); rocksdb_readoptions_destroy(roptions); rocksdb_writeoptions_destroy(woptions); + rocksdb_compactoptions_destroy(coptions); rocksdb_cache_destroy(cache); rocksdb_comparator_destroy(cmp); + rocksdb_dbpath_destroy(dbpath); rocksdb_env_destroy(env); fprintf(stderr, "PASS\n"); return 0; } + +#else + +int main() { + fprintf(stderr, "SKIPPED\n"); + return 0; +} + +#endif // !ROCKSDB_LITE diff --git a/db/column_family.cc b/db/column_family.cc index b1c9ba7e83c..6fd07878470 100644 --- a/db/column_family.cc +++ b/db/column_family.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,24 +9,36 @@ #include "db/column_family.h" +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include #include #include #include #include +#include "db/compaction_picker.h" +#include "db/compaction_picker_universal.h" #include "db/db_impl.h" -#include "db/version_set.h" #include "db/internal_stats.h" -#include "db/compaction_picker.h" +#include "db/job_context.h" #include "db/table_properties_collector.h" +#include "db/version_set.h" +#include "db/write_controller.h" +#include "memtable/hash_skiplist_rep.h" +#include "monitoring/thread_status_util.h" +#include "options/options_helper.h" +#include "table/block_based_table_factory.h" #include "util/autovector.h" -#include "util/hash_skiplist_rep.h" +#include "util/compression.h" namespace rocksdb { -ColumnFamilyHandleImpl::ColumnFamilyHandleImpl(ColumnFamilyData* cfd, - DBImpl* db, port::Mutex* mutex) - : cfd_(cfd), db_(db), mutex_(mutex) { +ColumnFamilyHandleImpl::ColumnFamilyHandleImpl( + ColumnFamilyData* column_family_data, DBImpl* db, InstrumentedMutex* mutex) + : cfd_(column_family_data), db_(db), mutex_(mutex) { if (cfd_ != nullptr) { cfd_->Ref(); } @@ -34,50 +46,149 @@ ColumnFamilyHandleImpl::ColumnFamilyHandleImpl(ColumnFamilyData* cfd, ColumnFamilyHandleImpl::~ColumnFamilyHandleImpl() { if (cfd_ != nullptr) { - DBImpl::DeletionState deletion_state; +#ifndef ROCKSDB_LITE + for (auto& listener : cfd_->ioptions()->listeners) { + listener->OnColumnFamilyHandleDeletionStarted(this); + } +#endif // ROCKSDB_LITE + // Job id == 0 means that this is not our background process, but rather + // user thread + JobContext job_context(0); mutex_->Lock(); if (cfd_->Unref()) { delete cfd_; } - db_->FindObsoleteFiles(deletion_state, false, true); + db_->FindObsoleteFiles(&job_context, false, true); mutex_->Unlock(); - if (deletion_state.HaveSomethingToDelete()) { - db_->PurgeObsoleteFiles(deletion_state); + if (job_context.HaveSomethingToDelete()) { + db_->PurgeObsoleteFiles(job_context); } + job_context.Clean(); } } uint32_t ColumnFamilyHandleImpl::GetID() const { return cfd()->GetID(); } -ColumnFamilyOptions SanitizeOptions(const InternalKeyComparator* icmp, +const std::string& ColumnFamilyHandleImpl::GetName() const { + return cfd()->GetName(); +} + +Status ColumnFamilyHandleImpl::GetDescriptor(ColumnFamilyDescriptor* desc) { +#ifndef ROCKSDB_LITE + // accessing mutable cf-options requires db mutex. + InstrumentedMutexLock l(mutex_); + *desc = ColumnFamilyDescriptor(cfd()->GetName(), cfd()->GetLatestCFOptions()); + return Status::OK(); +#else + return Status::NotSupported(); +#endif // !ROCKSDB_LITE +} + +const Comparator* ColumnFamilyHandleImpl::GetComparator() const { + return cfd()->user_comparator(); +} + +void GetIntTblPropCollectorFactory( + const ImmutableCFOptions& ioptions, + std::vector>* + int_tbl_prop_collector_factories) { + auto& collector_factories = ioptions.table_properties_collector_factories; + for (size_t i = 0; i < ioptions.table_properties_collector_factories.size(); + ++i) { + assert(collector_factories[i]); + int_tbl_prop_collector_factories->emplace_back( + new UserKeyTablePropertiesCollectorFactory(collector_factories[i])); + } + // Add collector to collect internal key statistics + int_tbl_prop_collector_factories->emplace_back( + new InternalKeyPropertiesCollectorFactory); +} + +Status CheckCompressionSupported(const ColumnFamilyOptions& cf_options) { + if (!cf_options.compression_per_level.empty()) { + for (size_t level = 0; level < cf_options.compression_per_level.size(); + ++level) { + if (!CompressionTypeSupported(cf_options.compression_per_level[level])) { + return Status::InvalidArgument( + "Compression type " + + CompressionTypeToString(cf_options.compression_per_level[level]) + + " is not linked with the binary."); + } + } + } else { + if (!CompressionTypeSupported(cf_options.compression)) { + return Status::InvalidArgument( + "Compression type " + + CompressionTypeToString(cf_options.compression) + + " is not linked with the binary."); + } + } + return Status::OK(); +} + +Status CheckConcurrentWritesSupported(const ColumnFamilyOptions& cf_options) { + if (cf_options.inplace_update_support) { + return Status::InvalidArgument( + "In-place memtable updates (inplace_update_support) is not compatible " + "with concurrent writes (allow_concurrent_memtable_write)"); + } + if (!cf_options.memtable_factory->IsInsertConcurrentlySupported()) { + return Status::InvalidArgument( + "Memtable doesn't concurrent writes (allow_concurrent_memtable_write)"); + } + return Status::OK(); +} + +ColumnFamilyOptions SanitizeOptions(const ImmutableDBOptions& db_options, const ColumnFamilyOptions& src) { ColumnFamilyOptions result = src; - result.comparator = icmp; -#ifdef OS_MACOSX - // TODO(icanadi) make write_buffer_size uint64_t instead of size_t - ClipToRange(&result.write_buffer_size, ((size_t)64) << 10, ((size_t)1) << 30); -#else - ClipToRange(&result.write_buffer_size, - ((size_t)64) << 10, ((size_t)64) << 30); -#endif + size_t clamp_max = std::conditional< + sizeof(size_t) == 4, std::integral_constant, + std::integral_constant>::type::value; + ClipToRange(&result.write_buffer_size, ((size_t)64) << 10, clamp_max); // if user sets arena_block_size, we trust user to use this value. Otherwise, // calculate a proper value from writer_buffer_size; if (result.arena_block_size <= 0) { - result.arena_block_size = result.write_buffer_size / 10; + result.arena_block_size = result.write_buffer_size / 8; + + // Align up to 4k + const size_t align = 4 * 1024; + result.arena_block_size = + ((result.arena_block_size + align - 1) / align) * align; } result.min_write_buffer_number_to_merge = std::min(result.min_write_buffer_number_to_merge, result.max_write_buffer_number - 1); - result.compression_per_level = src.compression_per_level; - if (result.max_mem_compaction_level >= result.num_levels) { - result.max_mem_compaction_level = result.num_levels - 1; + if (result.min_write_buffer_number_to_merge < 1) { + result.min_write_buffer_number_to_merge = 1; + } + + if (result.num_levels < 1) { + result.num_levels = 1; } - if (result.soft_rate_limit > result.hard_rate_limit) { - result.soft_rate_limit = result.hard_rate_limit; + if (result.compaction_style == kCompactionStyleLevel && + result.num_levels < 2) { + result.num_levels = 2; } + + if (result.compaction_style == kCompactionStyleUniversal && + db_options.allow_ingest_behind && result.num_levels < 3) { + result.num_levels = 3; + } + if (result.max_write_buffer_number < 2) { result.max_write_buffer_number = 2; } + if (result.max_write_buffer_number_to_maintain < 0) { + result.max_write_buffer_number_to_maintain = result.max_write_buffer_number; + } + // bloom filter size shouldn't exceed 1/4 of memtable size. + if (result.memtable_prefix_bloom_size_ratio > 0.25) { + result.memtable_prefix_bloom_size_ratio = 0.25; + } else if (result.memtable_prefix_bloom_size_ratio < 0) { + result.memtable_prefix_bloom_size_ratio = 0; + } + if (!result.prefix_extractor) { assert(result.memtable_factory); Slice name = result.memtable_factory->Name(); @@ -87,31 +198,80 @@ ColumnFamilyOptions SanitizeOptions(const InternalKeyComparator* icmp, } } - // -- Sanitize the table properties collector - // All user defined properties collectors will be wrapped by - // UserKeyTablePropertiesCollector since for them they only have the - // knowledge of the user keys; internal keys are invisible to them. - auto& collector_factories = result.table_properties_collector_factories; - for (size_t i = 0; i < result.table_properties_collector_factories.size(); - ++i) { - assert(collector_factories[i]); - collector_factories[i] = - std::make_shared( - collector_factories[i]); - } - // Add collector to collect internal key statistics - collector_factories.push_back( - std::make_shared()); - if (result.compaction_style == kCompactionStyleFIFO) { result.num_levels = 1; // since we delete level0 files in FIFO compaction when there are too many // of them, these options don't really mean anything - result.level0_file_num_compaction_trigger = std::numeric_limits::max(); result.level0_slowdown_writes_trigger = std::numeric_limits::max(); result.level0_stop_writes_trigger = std::numeric_limits::max(); } + if (result.max_bytes_for_level_multiplier <= 0) { + result.max_bytes_for_level_multiplier = 1; + } + + if (result.level0_file_num_compaction_trigger == 0) { + ROCKS_LOG_WARN(db_options.info_log.get(), + "level0_file_num_compaction_trigger cannot be 0"); + result.level0_file_num_compaction_trigger = 1; + } + + if (result.level0_stop_writes_trigger < + result.level0_slowdown_writes_trigger || + result.level0_slowdown_writes_trigger < + result.level0_file_num_compaction_trigger) { + ROCKS_LOG_WARN(db_options.info_log.get(), + "This condition must be satisfied: " + "level0_stop_writes_trigger(%d) >= " + "level0_slowdown_writes_trigger(%d) >= " + "level0_file_num_compaction_trigger(%d)", + result.level0_stop_writes_trigger, + result.level0_slowdown_writes_trigger, + result.level0_file_num_compaction_trigger); + if (result.level0_slowdown_writes_trigger < + result.level0_file_num_compaction_trigger) { + result.level0_slowdown_writes_trigger = + result.level0_file_num_compaction_trigger; + } + if (result.level0_stop_writes_trigger < + result.level0_slowdown_writes_trigger) { + result.level0_stop_writes_trigger = result.level0_slowdown_writes_trigger; + } + ROCKS_LOG_WARN(db_options.info_log.get(), + "Adjust the value to " + "level0_stop_writes_trigger(%d)" + "level0_slowdown_writes_trigger(%d)" + "level0_file_num_compaction_trigger(%d)", + result.level0_stop_writes_trigger, + result.level0_slowdown_writes_trigger, + result.level0_file_num_compaction_trigger); + } + + if (result.soft_pending_compaction_bytes_limit == 0) { + result.soft_pending_compaction_bytes_limit = + result.hard_pending_compaction_bytes_limit; + } else if (result.hard_pending_compaction_bytes_limit > 0 && + result.soft_pending_compaction_bytes_limit > + result.hard_pending_compaction_bytes_limit) { + result.soft_pending_compaction_bytes_limit = + result.hard_pending_compaction_bytes_limit; + } + + if (result.level_compaction_dynamic_level_bytes) { + if (result.compaction_style != kCompactionStyleLevel || + db_options.db_paths.size() > 1U) { + // 1. level_compaction_dynamic_level_bytes only makes sense for + // level-based compaction. + // 2. we don't yet know how to make both of this feature and multiple + // DB path work. + result.level_compaction_dynamic_level_bytes = false; + } + } + + if (result.max_compaction_bytes == 0) { + result.max_compaction_bytes = result.target_file_size_base * 25; + } + return result; } @@ -132,7 +292,7 @@ SuperVersion* SuperVersion::Ref() { bool SuperVersion::Unref() { // fetch_sub returns the previous value of ref - uint32_t previous_refs = refs.fetch_sub(1, std::memory_order_relaxed); + uint32_t previous_refs = refs.fetch_sub(1); assert(previous_refs > 0); return previous_refs == 1; } @@ -142,6 +302,9 @@ void SuperVersion::Cleanup() { imm->Unref(&to_delete); MemTable* m = mem->Unref(); if (m != nullptr) { + auto* memory_usage = current->cfd()->imm()->current_memory_usage(); + assert(*memory_usage >= m->ApproximateMemoryUsage()); + *memory_usage -= m->ApproximateMemoryUsage(); to_delete.push_back(m); } current->Unref(); @@ -174,70 +337,102 @@ void SuperVersionUnrefHandle(void* ptr) { } } // anonymous namespace -ColumnFamilyData::ColumnFamilyData(uint32_t id, const std::string& name, - Version* dummy_versions, Cache* table_cache, - const ColumnFamilyOptions& options, - const DBOptions* db_options, - const EnvOptions& storage_options, - ColumnFamilySet* column_family_set) +ColumnFamilyData::ColumnFamilyData( + uint32_t id, const std::string& name, Version* _dummy_versions, + Cache* _table_cache, WriteBufferManager* write_buffer_manager, + const ColumnFamilyOptions& cf_options, const ImmutableDBOptions& db_options, + const EnvOptions& env_options, ColumnFamilySet* column_family_set) : id_(id), name_(name), - dummy_versions_(dummy_versions), + dummy_versions_(_dummy_versions), current_(nullptr), refs_(0), + initialized_(false), dropped_(false), - internal_comparator_(options.comparator), - options_(*db_options, SanitizeOptions(&internal_comparator_, options)), + internal_comparator_(cf_options.comparator), + initial_cf_options_(SanitizeOptions(db_options, cf_options)), + ioptions_(db_options, initial_cf_options_), + mutable_cf_options_(initial_cf_options_), + is_delete_range_supported_( + cf_options.table_factory->IsDeleteRangeSupported()), + write_buffer_manager_(write_buffer_manager), mem_(nullptr), - imm_(options_.min_write_buffer_number_to_merge), + imm_(ioptions_.min_write_buffer_number_to_merge, + ioptions_.max_write_buffer_number_to_maintain), super_version_(nullptr), super_version_number_(0), local_sv_(new ThreadLocalPtr(&SuperVersionUnrefHandle)), next_(nullptr), prev_(nullptr), log_number_(0), - need_slowdown_for_num_level0_files_(false), - column_family_set_(column_family_set) { + column_family_set_(column_family_set), + pending_flush_(false), + pending_compaction_(false), + prev_compaction_needed_bytes_(0), + allow_2pc_(db_options.allow_2pc) { Ref(); - // if dummy_versions is nullptr, then this is a dummy column family. - if (dummy_versions != nullptr) { + // Convert user defined table properties collector factories to internal ones. + GetIntTblPropCollectorFactory(ioptions_, &int_tbl_prop_collector_factories_); + + // if _dummy_versions is nullptr, then this is a dummy column family. + if (_dummy_versions != nullptr) { internal_stats_.reset( - new InternalStats(options_.num_levels, db_options->env, this)); - table_cache_.reset(new TableCache(&options_, storage_options, table_cache)); - if (options_.compaction_style == kCompactionStyleUniversal) { + new InternalStats(ioptions_.num_levels, db_options.env, this)); + table_cache_.reset(new TableCache(ioptions_, env_options, _table_cache)); + if (ioptions_.compaction_style == kCompactionStyleLevel) { + compaction_picker_.reset( + new LevelCompactionPicker(ioptions_, &internal_comparator_)); +#ifndef ROCKSDB_LITE + } else if (ioptions_.compaction_style == kCompactionStyleUniversal) { compaction_picker_.reset( - new UniversalCompactionPicker(&options_, &internal_comparator_)); - } else if (options_.compaction_style == kCompactionStyleLevel) { + new UniversalCompactionPicker(ioptions_, &internal_comparator_)); + } else if (ioptions_.compaction_style == kCompactionStyleFIFO) { compaction_picker_.reset( - new LevelCompactionPicker(&options_, &internal_comparator_)); + new FIFOCompactionPicker(ioptions_, &internal_comparator_)); + } else if (ioptions_.compaction_style == kCompactionStyleNone) { + compaction_picker_.reset(new NullCompactionPicker( + ioptions_, &internal_comparator_)); + ROCKS_LOG_WARN(ioptions_.info_log, + "Column family %s does not use any background compaction. " + "Compactions can only be done via CompactFiles\n", + GetName().c_str()); +#endif // !ROCKSDB_LITE } else { - assert(options_.compaction_style == kCompactionStyleFIFO); + ROCKS_LOG_ERROR(ioptions_.info_log, + "Unable to recognize the specified compaction style %d. " + "Column family %s will use kCompactionStyleLevel.\n", + ioptions_.compaction_style, GetName().c_str()); compaction_picker_.reset( - new FIFOCompactionPicker(&options_, &internal_comparator_)); + new LevelCompactionPicker(ioptions_, &internal_comparator_)); } - Log(options_.info_log, "Options for column family \"%s\":\n", - name.c_str()); - const ColumnFamilyOptions* cf_options = &options_; - cf_options->Dump(options_.info_log.get()); + if (column_family_set_->NumberOfColumnFamilies() < 10) { + ROCKS_LOG_INFO(ioptions_.info_log, + "--------------- Options for column family [%s]:\n", + name.c_str()); + initial_cf_options_.Dump(ioptions_.info_log); + } else { + ROCKS_LOG_INFO(ioptions_.info_log, "\t(skipping printing options)\n"); + } } - RecalculateWriteStallConditions(); + RecalculateWriteStallConditions(mutable_cf_options_); } // DB mutex held ColumnFamilyData::~ColumnFamilyData() { - assert(refs_ == 0); + assert(refs_.load(std::memory_order_relaxed) == 0); // remove from linked list auto prev = prev_; auto next = next_; prev->next_ = next; next->prev_ = prev; - // it's nullptr for dummy CFD - if (column_family_set_ != nullptr) { - // remove from column_family_set + if (!dropped_ && column_family_set_ != nullptr) { + // If it's dropped, it's already removed from column family set + // If column_family_set_ == nullptr, this is dummy CFD and not in + // ColumnFamilySet column_family_set_->RemoveColumnFamily(this); } @@ -245,6 +440,11 @@ ColumnFamilyData::~ColumnFamilyData() { current_->Unref(); } + // It would be wrong if this ColumnFamilyData is in flush_queue_ or + // compaction_queue_ and we destroyed it + assert(!pending_flush_); + assert(!pending_compaction_); + if (super_version_ != nullptr) { // Release SuperVersion reference kept in ThreadLocalPtr. // This must be done outside of mutex_ since unref handler can lock mutex. @@ -262,8 +462,9 @@ ColumnFamilyData::~ColumnFamilyData() { if (dummy_versions_ != nullptr) { // List must be empty - assert(dummy_versions_->next_ == dummy_versions_); - delete dummy_versions_; + assert(dummy_versions_->TEST_Next() == dummy_versions_); + bool deleted __attribute__((unused)) = dummy_versions_->Unref(); + assert(deleted); } if (mem_ != nullptr) { @@ -276,90 +477,381 @@ ColumnFamilyData::~ColumnFamilyData() { } } -void ColumnFamilyData::RecalculateWriteStallConditions() { - need_wait_for_num_memtables_ = - (imm()->size() == options()->max_write_buffer_number - 1); +void ColumnFamilyData::SetDropped() { + // can't drop default CF + assert(id_ != 0); + dropped_ = true; + write_controller_token_.reset(); - if (current_ != nullptr) { - need_wait_for_num_level0_files_ = - (current_->NumLevelFiles(0) >= options()->level0_stop_writes_trigger); - } else { - need_wait_for_num_level0_files_ = false; + // remove from column_family_set + column_family_set_->RemoveColumnFamily(this); +} + +ColumnFamilyOptions ColumnFamilyData::GetLatestCFOptions() const { + return BuildColumnFamilyOptions(initial_cf_options_, mutable_cf_options_); +} + +uint64_t ColumnFamilyData::OldestLogToKeep() { + auto current_log = GetLogNumber(); + + if (allow_2pc_) { + auto imm_prep_log = imm()->GetMinLogContainingPrepSection(); + auto mem_prep_log = mem()->GetMinLogContainingPrepSection(); + + if (imm_prep_log > 0 && imm_prep_log < current_log) { + current_log = imm_prep_log; + } + + if (mem_prep_log > 0 && mem_prep_log < current_log) { + current_log = mem_prep_log; + } } - RecalculateWriteStallRateLimitsConditions(); + return current_log; } -void ColumnFamilyData::RecalculateWriteStallRateLimitsConditions() { - if (current_ != nullptr) { - exceeds_hard_rate_limit_ = - (options()->hard_rate_limit > 1.0 && - current_->MaxCompactionScore() > options()->hard_rate_limit); +const double kIncSlowdownRatio = 0.8; +const double kDecSlowdownRatio = 1 / kIncSlowdownRatio; +const double kNearStopSlowdownRatio = 0.6; +const double kDelayRecoverSlowdownRatio = 1.4; + +namespace { +// If penalize_stop is true, we further reduce slowdown rate. +std::unique_ptr SetupDelay( + WriteController* write_controller, uint64_t compaction_needed_bytes, + uint64_t prev_compaction_need_bytes, bool penalize_stop, + bool auto_comapctions_disabled) { + const uint64_t kMinWriteRate = 16 * 1024u; // Minimum write rate 16KB/s. + + uint64_t max_write_rate = write_controller->max_delayed_write_rate(); + uint64_t write_rate = write_controller->delayed_write_rate(); + + if (auto_comapctions_disabled) { + // When auto compaction is disabled, always use the value user gave. + write_rate = max_write_rate; + } else if (write_controller->NeedsDelay() && max_write_rate > kMinWriteRate) { + // If user gives rate less than kMinWriteRate, don't adjust it. + // + // If already delayed, need to adjust based on previous compaction debt. + // When there are two or more column families require delay, we always + // increase or reduce write rate based on information for one single + // column family. It is likely to be OK but we can improve if there is a + // problem. + // Ignore compaction_needed_bytes = 0 case because compaction_needed_bytes + // is only available in level-based compaction + // + // If the compaction debt stays the same as previously, we also further slow + // down. It usually means a mem table is full. It's mainly for the case + // where both of flush and compaction are much slower than the speed we + // insert to mem tables, so we need to actively slow down before we get + // feedback signal from compaction and flushes to avoid the full stop + // because of hitting the max write buffer number. + // + // If DB just falled into the stop condition, we need to further reduce + // the write rate to avoid the stop condition. + if (penalize_stop) { + // Penalize the near stop or stop condition by more aggressive slowdown. + // This is to provide the long term slowdown increase signal. + // The penalty is more than the reward of recovering to the normal + // condition. + write_rate = static_cast(static_cast(write_rate) * + kNearStopSlowdownRatio); + if (write_rate < kMinWriteRate) { + write_rate = kMinWriteRate; + } + } else if (prev_compaction_need_bytes > 0 && + prev_compaction_need_bytes <= compaction_needed_bytes) { + write_rate = static_cast(static_cast(write_rate) * + kIncSlowdownRatio); + if (write_rate < kMinWriteRate) { + write_rate = kMinWriteRate; + } + } else if (prev_compaction_need_bytes > compaction_needed_bytes) { + // We are speeding up by ratio of kSlowdownRatio when we have paid + // compaction debt. But we'll never speed up to faster than the write rate + // given by users. + write_rate = static_cast(static_cast(write_rate) * + kDecSlowdownRatio); + if (write_rate > max_write_rate) { + write_rate = max_write_rate; + } + } + } + return write_controller->GetDelayToken(write_rate); +} + +int GetL0ThresholdSpeedupCompaction(int level0_file_num_compaction_trigger, + int level0_slowdown_writes_trigger) { + // SanitizeOptions() ensures it. + assert(level0_file_num_compaction_trigger <= level0_slowdown_writes_trigger); - exceeds_soft_rate_limit_ = - (options()->soft_rate_limit > 0.0 && - current_->MaxCompactionScore() > options()->soft_rate_limit); + if (level0_file_num_compaction_trigger < 0) { + return std::numeric_limits::max(); + } + + const int64_t twice_level0_trigger = + static_cast(level0_file_num_compaction_trigger) * 2; + + const int64_t one_fourth_trigger_slowdown = + static_cast(level0_file_num_compaction_trigger) + + ((level0_slowdown_writes_trigger - level0_file_num_compaction_trigger) / + 4); + + assert(twice_level0_trigger >= 0); + assert(one_fourth_trigger_slowdown >= 0); + + // 1/4 of the way between L0 compaction trigger threshold and slowdown + // condition. + // Or twice as compaction trigger, if it is smaller. + int64_t res = std::min(twice_level0_trigger, one_fourth_trigger_slowdown); + if (res >= port::kMaxInt32) { + return port::kMaxInt32; } else { - exceeds_hard_rate_limit_ = false; - exceeds_soft_rate_limit_ = false; + // res fits in int + return static_cast(res); + } +} +} // namespace + +void ColumnFamilyData::RecalculateWriteStallConditions( + const MutableCFOptions& mutable_cf_options) { + if (current_ != nullptr) { + auto* vstorage = current_->storage_info(); + auto write_controller = column_family_set_->write_controller_; + uint64_t compaction_needed_bytes = + vstorage->estimated_compaction_needed_bytes(); + + bool was_stopped = write_controller->IsStopped(); + bool needed_delay = write_controller->NeedsDelay(); + + if (imm()->NumNotFlushed() >= mutable_cf_options.max_write_buffer_number) { + write_controller_token_ = write_controller->GetStopToken(); + internal_stats_->AddCFStats(InternalStats::MEMTABLE_COMPACTION, 1); + ROCKS_LOG_WARN( + ioptions_.info_log, + "[%s] Stopping writes because we have %d immutable memtables " + "(waiting for flush), max_write_buffer_number is set to %d", + name_.c_str(), imm()->NumNotFlushed(), + mutable_cf_options.max_write_buffer_number); + } else if (!mutable_cf_options.disable_auto_compactions && + vstorage->l0_delay_trigger_count() >= + mutable_cf_options.level0_stop_writes_trigger) { + write_controller_token_ = write_controller->GetStopToken(); + internal_stats_->AddCFStats(InternalStats::LEVEL0_NUM_FILES_TOTAL, 1); + if (compaction_picker_->IsLevel0CompactionInProgress()) { + internal_stats_->AddCFStats( + InternalStats::LEVEL0_NUM_FILES_WITH_COMPACTION, 1); + } + ROCKS_LOG_WARN(ioptions_.info_log, + "[%s] Stopping writes because we have %d level-0 files", + name_.c_str(), vstorage->l0_delay_trigger_count()); + } else if (!mutable_cf_options.disable_auto_compactions && + mutable_cf_options.hard_pending_compaction_bytes_limit > 0 && + compaction_needed_bytes >= + mutable_cf_options.hard_pending_compaction_bytes_limit) { + write_controller_token_ = write_controller->GetStopToken(); + internal_stats_->AddCFStats( + InternalStats::HARD_PENDING_COMPACTION_BYTES_LIMIT, 1); + ROCKS_LOG_WARN( + ioptions_.info_log, + "[%s] Stopping writes because of estimated pending compaction " + "bytes %" PRIu64, + name_.c_str(), compaction_needed_bytes); + } else if (mutable_cf_options.max_write_buffer_number > 3 && + imm()->NumNotFlushed() >= + mutable_cf_options.max_write_buffer_number - 1) { + write_controller_token_ = + SetupDelay(write_controller, compaction_needed_bytes, + prev_compaction_needed_bytes_, was_stopped, + mutable_cf_options.disable_auto_compactions); + internal_stats_->AddCFStats(InternalStats::MEMTABLE_SLOWDOWN, 1); + ROCKS_LOG_WARN( + ioptions_.info_log, + "[%s] Stalling writes because we have %d immutable memtables " + "(waiting for flush), max_write_buffer_number is set to %d " + "rate %" PRIu64, + name_.c_str(), imm()->NumNotFlushed(), + mutable_cf_options.max_write_buffer_number, + write_controller->delayed_write_rate()); + } else if (!mutable_cf_options.disable_auto_compactions && + mutable_cf_options.level0_slowdown_writes_trigger >= 0 && + vstorage->l0_delay_trigger_count() >= + mutable_cf_options.level0_slowdown_writes_trigger) { + // L0 is the last two files from stopping. + bool near_stop = vstorage->l0_delay_trigger_count() >= + mutable_cf_options.level0_stop_writes_trigger - 2; + write_controller_token_ = + SetupDelay(write_controller, compaction_needed_bytes, + prev_compaction_needed_bytes_, was_stopped || near_stop, + mutable_cf_options.disable_auto_compactions); + internal_stats_->AddCFStats(InternalStats::LEVEL0_SLOWDOWN_TOTAL, 1); + if (compaction_picker_->IsLevel0CompactionInProgress()) { + internal_stats_->AddCFStats( + InternalStats::LEVEL0_SLOWDOWN_WITH_COMPACTION, 1); + } + ROCKS_LOG_WARN(ioptions_.info_log, + "[%s] Stalling writes because we have %d level-0 files " + "rate %" PRIu64, + name_.c_str(), vstorage->l0_delay_trigger_count(), + write_controller->delayed_write_rate()); + } else if (!mutable_cf_options.disable_auto_compactions && + mutable_cf_options.soft_pending_compaction_bytes_limit > 0 && + vstorage->estimated_compaction_needed_bytes() >= + mutable_cf_options.soft_pending_compaction_bytes_limit) { + // If the distance to hard limit is less than 1/4 of the gap between soft + // and + // hard bytes limit, we think it is near stop and speed up the slowdown. + bool near_stop = + mutable_cf_options.hard_pending_compaction_bytes_limit > 0 && + (compaction_needed_bytes - + mutable_cf_options.soft_pending_compaction_bytes_limit) > + 3 * (mutable_cf_options.hard_pending_compaction_bytes_limit - + mutable_cf_options.soft_pending_compaction_bytes_limit) / + 4; + + write_controller_token_ = + SetupDelay(write_controller, compaction_needed_bytes, + prev_compaction_needed_bytes_, was_stopped || near_stop, + mutable_cf_options.disable_auto_compactions); + internal_stats_->AddCFStats( + InternalStats::SOFT_PENDING_COMPACTION_BYTES_LIMIT, 1); + ROCKS_LOG_WARN( + ioptions_.info_log, + "[%s] Stalling writes because of estimated pending compaction " + "bytes %" PRIu64 " rate %" PRIu64, + name_.c_str(), vstorage->estimated_compaction_needed_bytes(), + write_controller->delayed_write_rate()); + } else { + if (vstorage->l0_delay_trigger_count() >= + GetL0ThresholdSpeedupCompaction( + mutable_cf_options.level0_file_num_compaction_trigger, + mutable_cf_options.level0_slowdown_writes_trigger)) { + write_controller_token_ = + write_controller->GetCompactionPressureToken(); + ROCKS_LOG_INFO( + ioptions_.info_log, + "[%s] Increasing compaction threads because we have %d level-0 " + "files ", + name_.c_str(), vstorage->l0_delay_trigger_count()); + } else if (vstorage->estimated_compaction_needed_bytes() >= + mutable_cf_options.soft_pending_compaction_bytes_limit / 4) { + // Increase compaction threads if bytes needed for compaction exceeds + // 1/4 of threshold for slowing down. + // If soft pending compaction byte limit is not set, always speed up + // compaction. + write_controller_token_ = + write_controller->GetCompactionPressureToken(); + if (mutable_cf_options.soft_pending_compaction_bytes_limit > 0) { + ROCKS_LOG_INFO( + ioptions_.info_log, + "[%s] Increasing compaction threads because of estimated pending " + "compaction " + "bytes %" PRIu64, + name_.c_str(), vstorage->estimated_compaction_needed_bytes()); + } + } else { + write_controller_token_.reset(); + } + // If the DB recovers from delay conditions, we reward with reducing + // double the slowdown ratio. This is to balance the long term slowdown + // increase signal. + if (needed_delay) { + uint64_t write_rate = write_controller->delayed_write_rate(); + write_controller->set_delayed_write_rate(static_cast( + static_cast(write_rate) * kDelayRecoverSlowdownRatio)); + // Set the low pri limit to be 1/4 the delayed write rate. + // Note we don't reset this value even after delay condition is relased. + // Low-pri rate will continue to apply if there is a compaction + // pressure. + write_controller->low_pri_rate_limiter()->SetBytesPerSecond(write_rate / + 4); + } + } + prev_compaction_needed_bytes_ = compaction_needed_bytes; } } const EnvOptions* ColumnFamilyData::soptions() const { - return &(column_family_set_->storage_options_); + return &(column_family_set_->env_options_); } -void ColumnFamilyData::SetCurrent(Version* current) { - current_ = current; - need_slowdown_for_num_level0_files_ = - (options_.level0_slowdown_writes_trigger >= 0 && - current_->NumLevelFiles(0) >= options_.level0_slowdown_writes_trigger); +void ColumnFamilyData::SetCurrent(Version* current_version) { + current_ = current_version; } -void ColumnFamilyData::CreateNewMemtable() { - assert(current_ != nullptr); +uint64_t ColumnFamilyData::GetNumLiveVersions() const { + return VersionSet::GetNumLiveVersions(dummy_versions_); +} + +uint64_t ColumnFamilyData::GetTotalSstFilesSize() const { + return VersionSet::GetTotalSstFilesSize(dummy_versions_); +} + +MemTable* ColumnFamilyData::ConstructNewMemtable( + const MutableCFOptions& mutable_cf_options, SequenceNumber earliest_seq) { + return new MemTable(internal_comparator_, ioptions_, mutable_cf_options, + write_buffer_manager_, earliest_seq, id_); +} + +void ColumnFamilyData::CreateNewMemtable( + const MutableCFOptions& mutable_cf_options, SequenceNumber earliest_seq) { if (mem_ != nullptr) { delete mem_->Unref(); } - mem_ = new MemTable(internal_comparator_, options_); + SetMemtable(ConstructNewMemtable(mutable_cf_options, earliest_seq)); mem_->Ref(); } -Compaction* ColumnFamilyData::PickCompaction(LogBuffer* log_buffer) { - auto result = compaction_picker_->PickCompaction(current_, log_buffer); - RecalculateWriteStallRateLimitsConditions(); +bool ColumnFamilyData::NeedsCompaction() const { + return compaction_picker_->NeedsCompaction(current_->storage_info()); +} + +Compaction* ColumnFamilyData::PickCompaction( + const MutableCFOptions& mutable_options, LogBuffer* log_buffer) { + auto* result = compaction_picker_->PickCompaction( + GetName(), mutable_options, current_->storage_info(), log_buffer); + if (result != nullptr) { + result->SetInputVersion(current_); + } return result; } -Compaction* ColumnFamilyData::CompactRange(int input_level, int output_level, - uint32_t output_path_id, - const InternalKey* begin, - const InternalKey* end, - InternalKey** compaction_end) { - return compaction_picker_->CompactRange(current_, input_level, output_level, - output_path_id, begin, end, - compaction_end); +bool ColumnFamilyData::RangeOverlapWithCompaction( + const Slice& smallest_user_key, const Slice& largest_user_key, + int level) const { + return compaction_picker_->RangeOverlapWithCompaction( + smallest_user_key, largest_user_key, level); +} + +const int ColumnFamilyData::kCompactAllLevels = -1; +const int ColumnFamilyData::kCompactToBaseLevel = -2; + +Compaction* ColumnFamilyData::CompactRange( + const MutableCFOptions& mutable_cf_options, int input_level, + int output_level, uint32_t output_path_id, const InternalKey* begin, + const InternalKey* end, InternalKey** compaction_end, bool* conflict) { + auto* result = compaction_picker_->CompactRange( + GetName(), mutable_cf_options, current_->storage_info(), input_level, + output_level, output_path_id, begin, end, compaction_end, conflict); + if (result != nullptr) { + result->SetInputVersion(current_); + } + return result; } SuperVersion* ColumnFamilyData::GetReferencedSuperVersion( - port::Mutex* db_mutex) { + InstrumentedMutex* db_mutex) { SuperVersion* sv = nullptr; - if (LIKELY(column_family_set_->db_options_->allow_thread_local)) { - sv = GetThreadLocalSuperVersion(db_mutex); - sv->Ref(); - if (!ReturnThreadLocalSuperVersion(sv)) { - sv->Unref(); - } - } else { - db_mutex->Lock(); - sv = super_version_->Ref(); - db_mutex->Unlock(); + sv = GetThreadLocalSuperVersion(db_mutex); + sv->Ref(); + if (!ReturnThreadLocalSuperVersion(sv)) { + sv->Unref(); } return sv; } SuperVersion* ColumnFamilyData::GetThreadLocalSuperVersion( - port::Mutex* db_mutex) { + InstrumentedMutex* db_mutex) { SuperVersion* sv = nullptr; // The SuperVersion is cached in thread local storage to avoid acquiring // mutex when SuperVersion does not change since the last use. When a new @@ -382,11 +874,11 @@ SuperVersion* ColumnFamilyData::GetThreadLocalSuperVersion( sv = static_cast(ptr); if (sv == SuperVersion::kSVObsolete || sv->version_number != super_version_number_.load()) { - RecordTick(options_.statistics.get(), NUMBER_SUPERVERSION_ACQUIRES); + RecordTick(ioptions_.statistics, NUMBER_SUPERVERSION_ACQUIRES); SuperVersion* sv_to_delete = nullptr; if (sv && sv->Unref()) { - RecordTick(options_.statistics.get(), NUMBER_SUPERVERSION_CLEANUPS); + RecordTick(ioptions_.statistics, NUMBER_SUPERVERSION_CLEANUPS); db_mutex->Lock(); // NOTE: underlying resources held by superversion (sst files) might // not be released until the next background job. @@ -410,7 +902,7 @@ bool ColumnFamilyData::ReturnThreadLocalSuperVersion(SuperVersion* sv) { void* expected = SuperVersion::kSVInUse; if (local_sv_->CompareAndSwap(static_cast(sv), expected)) { // When we see kSVInUse in the ThreadLocal, we are sure ThreadLocal - // storage has not been altered and no Scrape has happend. The + // storage has not been altered and no Scrape has happened. The // SuperVersion is still current. return true; } else { @@ -423,19 +915,32 @@ bool ColumnFamilyData::ReturnThreadLocalSuperVersion(SuperVersion* sv) { } SuperVersion* ColumnFamilyData::InstallSuperVersion( - SuperVersion* new_superversion, port::Mutex* db_mutex) { + SuperVersion* new_superversion, InstrumentedMutex* db_mutex) { + db_mutex->AssertHeld(); + return InstallSuperVersion(new_superversion, db_mutex, mutable_cf_options_); +} + +SuperVersion* ColumnFamilyData::InstallSuperVersion( + SuperVersion* new_superversion, InstrumentedMutex* db_mutex, + const MutableCFOptions& mutable_cf_options) { new_superversion->db_mutex = db_mutex; + new_superversion->mutable_cf_options = mutable_cf_options; new_superversion->Init(mem_, imm_.current(), current_); SuperVersion* old_superversion = super_version_; super_version_ = new_superversion; ++super_version_number_; super_version_->version_number = super_version_number_; - // Reset SuperVersions cached in thread local storage - if (column_family_set_->db_options_->allow_thread_local) { - ResetThreadLocalSuperVersions(); + if (old_superversion != nullptr) { + if (old_superversion->mutable_cf_options.write_buffer_size != + mutable_cf_options.write_buffer_size) { + mem_->UpdateWriteBufferSize(mutable_cf_options.write_buffer_size); + } } - RecalculateWriteStallConditions(); + // Reset SuperVersions cached in thread local storage + ResetThreadLocalSuperVersions(); + + RecalculateWriteStallConditions(mutable_cf_options); if (old_superversion != nullptr && old_superversion->Unref()) { old_superversion->Cleanup(); @@ -460,20 +965,37 @@ void ColumnFamilyData::ResetThreadLocalSuperVersions() { } } +#ifndef ROCKSDB_LITE +Status ColumnFamilyData::SetOptions( + const std::unordered_map& options_map) { + MutableCFOptions new_mutable_cf_options; + Status s = GetMutableOptionsFromStrings(mutable_cf_options_, options_map, + &new_mutable_cf_options); + if (s.ok()) { + mutable_cf_options_ = new_mutable_cf_options; + mutable_cf_options_.RefreshDerivedOptions(ioptions_); + } + return s; +} +#endif // ROCKSDB_LITE + ColumnFamilySet::ColumnFamilySet(const std::string& dbname, - const DBOptions* db_options, - const EnvOptions& storage_options, - Cache* table_cache) + const ImmutableDBOptions* db_options, + const EnvOptions& env_options, + Cache* table_cache, + WriteBufferManager* write_buffer_manager, + WriteController* write_controller) : max_column_family_(0), - dummy_cfd_(new ColumnFamilyData(0, "", nullptr, nullptr, - ColumnFamilyOptions(), db_options, - storage_options_, nullptr)), + dummy_cfd_(new ColumnFamilyData(0, "", nullptr, nullptr, nullptr, + ColumnFamilyOptions(), *db_options, + env_options, nullptr)), default_cfd_cache_(nullptr), db_name_(dbname), db_options_(db_options), - storage_options_(storage_options), + env_options_(env_options), table_cache_(table_cache), - spin_lock_(ATOMIC_FLAG_INIT) { + write_buffer_manager_(write_buffer_manager), + write_controller_(write_controller) { // initialize linked list dummy_cfd_->prev_ = dummy_cfd_; dummy_cfd_->next_ = dummy_cfd_; @@ -530,18 +1052,16 @@ size_t ColumnFamilySet::NumberOfColumnFamilies() const { return column_families_.size(); } -// under a DB mutex +// under a DB mutex AND write thread ColumnFamilyData* ColumnFamilySet::CreateColumnFamily( const std::string& name, uint32_t id, Version* dummy_versions, const ColumnFamilyOptions& options) { assert(column_families_.find(name) == column_families_.end()); - ColumnFamilyData* new_cfd = - new ColumnFamilyData(id, name, dummy_versions, table_cache_, options, - db_options_, storage_options_, this); - Lock(); + ColumnFamilyData* new_cfd = new ColumnFamilyData( + id, name, dummy_versions, table_cache_, write_buffer_manager_, options, + *db_options_, env_options_, this); column_families_.insert({name, id}); column_family_data_.insert({id, new_cfd}); - Unlock(); max_column_family_ = std::max(max_column_family_, id); // add to linked list new_cfd->next_ = dummy_cfd_; @@ -555,19 +1075,11 @@ ColumnFamilyData* ColumnFamilySet::CreateColumnFamily( return new_cfd; } -void ColumnFamilySet::Lock() { - // spin lock - while (spin_lock_.test_and_set(std::memory_order_acquire)) { - } -} - -void ColumnFamilySet::Unlock() { spin_lock_.clear(std::memory_order_release); } - // REQUIRES: DB mutex held void ColumnFamilySet::FreeDeadColumnFamilies() { autovector to_delete; for (auto cfd = dummy_cfd_->next_; cfd != dummy_cfd_; cfd = cfd->next_) { - if (cfd->refs_ == 0) { + if (cfd->refs_.load(std::memory_order_relaxed) == 0) { to_delete.push_back(cfd); } } @@ -577,25 +1089,21 @@ void ColumnFamilySet::FreeDeadColumnFamilies() { } } -// under a DB mutex +// under a DB mutex AND from a write thread void ColumnFamilySet::RemoveColumnFamily(ColumnFamilyData* cfd) { auto cfd_iter = column_family_data_.find(cfd->GetID()); assert(cfd_iter != column_family_data_.end()); - Lock(); column_family_data_.erase(cfd_iter); column_families_.erase(cfd->GetName()); - Unlock(); } +// under a DB mutex OR from a write thread bool ColumnFamilyMemTablesImpl::Seek(uint32_t column_family_id) { if (column_family_id == 0) { // optimization for common case current_ = column_family_set_->GetDefault(); } else { - // maybe outside of db mutex, should lock - column_family_set_->Lock(); current_ = column_family_set_->GetColumnFamily(column_family_id); - column_family_set_->Unlock(); } handle_.SetCFD(current_); return current_ != nullptr; @@ -611,11 +1119,6 @@ MemTable* ColumnFamilyMemTablesImpl::GetMemTable() const { return current_->mem(); } -const Options* ColumnFamilyMemTablesImpl::GetOptions() const { - assert(current_ != nullptr); - return current_->options(); -} - ColumnFamilyHandle* ColumnFamilyMemTablesImpl::GetColumnFamilyHandle() { assert(current_ != nullptr); return &handle_; @@ -630,4 +1133,12 @@ uint32_t GetColumnFamilyID(ColumnFamilyHandle* column_family) { return column_family_id; } +const Comparator* GetColumnFamilyUserComparator( + ColumnFamilyHandle* column_family) { + if (column_family != nullptr) { + return column_family->GetComparator(); + } + return nullptr; +} + } // namespace rocksdb diff --git a/db/column_family.h b/db/column_family.h index 33bceadc62c..3a807d22b91 100644 --- a/db/column_family.h +++ b/db/column_family.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -14,12 +14,16 @@ #include #include -#include "rocksdb/options.h" -#include "rocksdb/db.h" -#include "rocksdb/env.h" #include "db/memtable_list.h" -#include "db/write_batch_internal.h" #include "db/table_cache.h" +#include "db/table_properties_collector.h" +#include "db/write_batch_internal.h" +#include "db/write_controller.h" +#include "options/cf_options.h" +#include "rocksdb/compaction_job_stats.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/options.h" #include "util/thread_local.h" namespace rocksdb { @@ -35,6 +39,10 @@ class InternalStats; class ColumnFamilyData; class DBImpl; class LogBuffer; +class InstrumentedMutex; +class InstrumentedMutexLock; + +extern const double kIncSlowdownRatio; // ColumnFamilyHandleImpl is the class that clients use to access different // column families. It has non-trivial destructor, which gets called when client @@ -42,17 +50,21 @@ class LogBuffer; class ColumnFamilyHandleImpl : public ColumnFamilyHandle { public: // create while holding the mutex - ColumnFamilyHandleImpl(ColumnFamilyData* cfd, DBImpl* db, port::Mutex* mutex); + ColumnFamilyHandleImpl( + ColumnFamilyData* cfd, DBImpl* db, InstrumentedMutex* mutex); // destroy without mutex virtual ~ColumnFamilyHandleImpl(); virtual ColumnFamilyData* cfd() const { return cfd_; } - virtual uint32_t GetID() const; + virtual uint32_t GetID() const override; + virtual const std::string& GetName() const override; + virtual Status GetDescriptor(ColumnFamilyDescriptor* desc) override; + virtual const Comparator* GetComparator() const override; private: ColumnFamilyData* cfd_; DBImpl* db_; - port::Mutex* mutex_; + InstrumentedMutex* mutex_; }; // Does not ref-count ColumnFamilyData @@ -66,7 +78,7 @@ class ColumnFamilyHandleInternal : public ColumnFamilyHandleImpl { ColumnFamilyHandleInternal() : ColumnFamilyHandleImpl(nullptr, nullptr, nullptr) {} - void SetCFD(ColumnFamilyData* cfd) { internal_cfd_ = cfd; } + void SetCFD(ColumnFamilyData* _cfd) { internal_cfd_ = _cfd; } virtual ColumnFamilyData* cfd() const override { return internal_cfd_; } private: @@ -75,23 +87,23 @@ class ColumnFamilyHandleInternal : public ColumnFamilyHandleImpl { // holds references to memtable, all immutable memtables and version struct SuperVersion { + // Accessing members of this class is not thread-safe and requires external + // synchronization (ie db mutex held or on write thread). MemTable* mem; MemTableListVersion* imm; Version* current; - std::atomic refs; - // We need to_delete because during Cleanup(), imm->Unref() returns - // all memtables that we need to free through this vector. We then - // delete all those memtables outside of mutex, during destruction - autovector to_delete; + MutableCFOptions mutable_cf_options; // Version number of the current SuperVersion uint64_t version_number; - port::Mutex* db_mutex; + + InstrumentedMutex* db_mutex; // should be called outside the mutex SuperVersion() = default; ~SuperVersion(); SuperVersion* Ref(); - + // If Unref() returns true, Cleanup() should be called with mutex held + // before deleting this SuperVersion. bool Unref(); // call these two methods with db mutex held @@ -110,15 +122,33 @@ struct SuperVersion { static int dummy; static void* const kSVInUse; static void* const kSVObsolete; + + private: + std::atomic refs; + // We need to_delete because during Cleanup(), imm->Unref() returns + // all memtables that we need to free through this vector. We then + // delete all those memtables outside of mutex, during destruction + autovector to_delete; }; -extern ColumnFamilyOptions SanitizeOptions(const InternalKeyComparator* icmp, +extern Status CheckCompressionSupported(const ColumnFamilyOptions& cf_options); + +extern Status CheckConcurrentWritesSupported( + const ColumnFamilyOptions& cf_options); + +extern ColumnFamilyOptions SanitizeOptions(const ImmutableDBOptions& db_options, const ColumnFamilyOptions& src); +// Wrap user defined table proproties collector factories `from cf_options` +// into internal ones in int_tbl_prop_collector_factories. Add a system internal +// one too. +extern void GetIntTblPropCollectorFactory( + const ImmutableCFOptions& ioptions, + std::vector>* + int_tbl_prop_collector_factories); class ColumnFamilySet; -// This class keeps all the data that a column family needs. It's mosly dumb and -// used just to provide access to metadata. +// This class keeps all the data that a column family needs. // Most methods require DB mutex held, unless otherwise noted class ColumnFamilyData { public: @@ -129,45 +159,74 @@ class ColumnFamilyData { // thread-safe const std::string& GetName() const { return name_; } - void Ref() { ++refs_; } - // will just decrease reference count to 0, but will not delete it. returns - // true if the ref count was decreased to zero. in that case, it can be - // deleted by the caller immediatelly, or later, by calling - // FreeDeadColumnFamilies() + // Ref() can only be called from a context where the caller can guarantee + // that ColumnFamilyData is alive (while holding a non-zero ref already, + // holding a DB mutex, or as the leader in a write batch group). + void Ref() { refs_.fetch_add(1, std::memory_order_relaxed); } + + // Unref decreases the reference count, but does not handle deletion + // when the count goes to 0. If this method returns true then the + // caller should delete the instance immediately, or later, by calling + // FreeDeadColumnFamilies(). Unref() can only be called while holding + // a DB mutex, or during single-threaded recovery. bool Unref() { - assert(refs_ > 0); - return --refs_ == 0; + int old_refs = refs_.fetch_sub(1, std::memory_order_relaxed); + assert(old_refs > 0); + return old_refs == 1; } - // This can only be called from single-threaded VersionSet::LogAndApply() + // SetDropped() can only be called under following conditions: + // 1) Holding a DB mutex, + // 2) from single-threaded write thread, AND + // 3) from single-threaded VersionSet::LogAndApply() // After dropping column family no other operation on that column family // will be executed. All the files and memory will be, however, kept around // until client drops the column family handle. That way, client can still // access data from dropped column family. // Column family can be dropped and still alive. In that state: - // *) Column family is not included in the iteration. // *) Compaction and flush is not executed on the dropped column family. - // *) Client can continue writing and reading from column family. However, all - // writes stay in the current memtable. + // *) Client can continue reading from column family. Writes will fail unless + // WriteOptions::ignore_missing_column_families is true // When the dropped column family is unreferenced, then we: + // *) Remove column family from the linked list maintained by ColumnFamilySet // *) delete all memory associated with that column family // *) delete all the files associated with that column family - void SetDropped() { - // can't drop default CF - assert(id_ != 0); - dropped_ = true; - } + void SetDropped(); bool IsDropped() const { return dropped_; } // thread-safe - int NumberLevels() const { return options_.num_levels; } + int NumberLevels() const { return ioptions_.num_levels; } void SetLogNumber(uint64_t log_number) { log_number_ = log_number; } uint64_t GetLogNumber() const { return log_number_; } // thread-safe - const Options* options() const { return &options_; } const EnvOptions* soptions() const; + const ImmutableCFOptions* ioptions() const { return &ioptions_; } + // REQUIRES: DB mutex held + // This returns the MutableCFOptions used by current SuperVersion + // You should use this API to reference MutableCFOptions most of the time. + const MutableCFOptions* GetCurrentMutableCFOptions() const { + return &(super_version_->mutable_cf_options); + } + // REQUIRES: DB mutex held + // This returns the latest MutableCFOptions, which may be not in effect yet. + const MutableCFOptions* GetLatestMutableCFOptions() const { + return &mutable_cf_options_; + } + + // REQUIRES: DB mutex held + // Build ColumnFamiliesOptions with immutable options and latest mutable + // options. + ColumnFamilyOptions GetLatestCFOptions() const; + + bool is_delete_range_supported() { return is_delete_range_supported_; } + +#ifndef ROCKSDB_LITE + // REQUIRES: DB mutex held + Status SetOptions( + const std::unordered_map& options_map); +#endif // ROCKSDB_LITE InternalStats* internal_stats() { return internal_stats_.get(); } @@ -175,18 +234,46 @@ class ColumnFamilyData { MemTable* mem() { return mem_; } Version* current() { return current_; } Version* dummy_versions() { return dummy_versions_; } + void SetCurrent(Version* _current); + uint64_t GetNumLiveVersions() const; // REQUIRE: DB mutex held + uint64_t GetTotalSstFilesSize() const; // REQUIRE: DB mutex held void SetMemtable(MemTable* new_mem) { mem_ = new_mem; } - void SetCurrent(Version* current); - void CreateNewMemtable(); + + // calculate the oldest log needed for the durability of this column family + uint64_t OldestLogToKeep(); + + // See Memtable constructor for explanation of earliest_seq param. + MemTable* ConstructNewMemtable(const MutableCFOptions& mutable_cf_options, + SequenceNumber earliest_seq); + void CreateNewMemtable(const MutableCFOptions& mutable_cf_options, + SequenceNumber earliest_seq); TableCache* table_cache() const { return table_cache_.get(); } // See documentation in compaction_picker.h - Compaction* PickCompaction(LogBuffer* log_buffer); - Compaction* CompactRange(int input_level, int output_level, + // REQUIRES: DB mutex held + bool NeedsCompaction() const; + // REQUIRES: DB mutex held + Compaction* PickCompaction(const MutableCFOptions& mutable_options, + LogBuffer* log_buffer); + + // Check if the passed range overlap with any running compactions. + // REQUIRES: DB mutex held + bool RangeOverlapWithCompaction(const Slice& smallest_user_key, + const Slice& largest_user_key, + int level) const; + + // A flag to tell a manual compaction is to compact all levels together + // instad of for specific level. + static const int kCompactAllLevels; + // A flag to tell a manual compaction's output is base level. + static const int kCompactToBaseLevel; + // REQUIRES: DB mutex held + Compaction* CompactRange(const MutableCFOptions& mutable_cf_options, + int input_level, int output_level, uint32_t output_path_id, const InternalKey* begin, - const InternalKey* end, - InternalKey** compaction_end); + const InternalKey* end, InternalKey** compaction_end, + bool* manual_conflict); CompactionPicker* compaction_picker() { return compaction_picker_.get(); } // thread-safe @@ -198,14 +285,19 @@ class ColumnFamilyData { return internal_comparator_; } + const std::vector>* + int_tbl_prop_collector_factories() const { + return &int_tbl_prop_collector_factories_; + } + SuperVersion* GetSuperVersion() { return super_version_; } // thread-safe // Return a already referenced SuperVersion to be used safely. - SuperVersion* GetReferencedSuperVersion(port::Mutex* db_mutex); + SuperVersion* GetReferencedSuperVersion(InstrumentedMutex* db_mutex); // thread-safe // Get SuperVersion stored in thread local storage. If it does not exist, // get a reference from a current SuperVersion. - SuperVersion* GetThreadLocalSuperVersion(port::Mutex* db_mutex); + SuperVersion* GetThreadLocalSuperVersion(InstrumentedMutex* db_mutex); // Try to return SuperVersion back to thread local storage. Retrun true on // success and false on failure. It fails when the thread local storage // contains anything other than SuperVersion::kSVInUse flag. @@ -218,66 +310,68 @@ class ColumnFamilyData { // if its reference count is zero and needs deletion or nullptr if not // As argument takes a pointer to allocated SuperVersion to enable // the clients to allocate SuperVersion outside of mutex. + // IMPORTANT: Only call this from DBImpl::InstallSuperVersion() SuperVersion* InstallSuperVersion(SuperVersion* new_superversion, - port::Mutex* db_mutex); + InstrumentedMutex* db_mutex, + const MutableCFOptions& mutable_cf_options); + SuperVersion* InstallSuperVersion(SuperVersion* new_superversion, + InstrumentedMutex* db_mutex); void ResetThreadLocalSuperVersions(); - // A Flag indicating whether write needs to slowdown because of there are - // too many number of level0 files. - bool NeedSlowdownForNumLevel0Files() const { - return need_slowdown_for_num_level0_files_; - } + // Protected by DB mutex + void set_pending_flush(bool value) { pending_flush_ = value; } + void set_pending_compaction(bool value) { pending_compaction_ = value; } + bool pending_flush() { return pending_flush_; } + bool pending_compaction() { return pending_compaction_; } - bool NeedWaitForNumLevel0Files() const { - return need_wait_for_num_level0_files_; - } - - bool NeedWaitForNumMemtables() const { - return need_wait_for_num_memtables_; - } + // Recalculate some small conditions, which are changed only during + // compaction, adding new memtable and/or + // recalculation of compaction score. These values are used in + // DBImpl::MakeRoomForWrite function to decide, if it need to make + // a write stall + void RecalculateWriteStallConditions( + const MutableCFOptions& mutable_cf_options); - bool ExceedsSoftRateLimit() const { - return exceeds_soft_rate_limit_; - } + void set_initialized() { initialized_.store(true); } - bool ExceedsHardRateLimit() const { - return exceeds_hard_rate_limit_; - } + bool initialized() const { return initialized_.load(); } private: friend class ColumnFamilySet; ColumnFamilyData(uint32_t id, const std::string& name, Version* dummy_versions, Cache* table_cache, + WriteBufferManager* write_buffer_manager, const ColumnFamilyOptions& options, - const DBOptions* db_options, - const EnvOptions& storage_options, + const ImmutableDBOptions& db_options, + const EnvOptions& env_options, ColumnFamilySet* column_family_set); - // Recalculate some small conditions, which are changed only during - // compaction, adding new memtable and/or - // recalculation of compaction score. These values are used in - // DBImpl::MakeRoomForWrite function to decide, if it need to make - // a write stall - void RecalculateWriteStallConditions(); - void RecalculateWriteStallRateLimitsConditions(); - uint32_t id_; const std::string name_; Version* dummy_versions_; // Head of circular doubly-linked list of versions. Version* current_; // == dummy_versions->prev_ - int refs_; // outstanding references to ColumnFamilyData + std::atomic refs_; // outstanding references to ColumnFamilyData + std::atomic initialized_; bool dropped_; // true if client dropped it const InternalKeyComparator internal_comparator_; + std::vector> + int_tbl_prop_collector_factories_; - Options const options_; + const ColumnFamilyOptions initial_cf_options_; + const ImmutableCFOptions ioptions_; + MutableCFOptions mutable_cf_options_; + + const bool is_delete_range_supported_; std::unique_ptr table_cache_; std::unique_ptr internal_stats_; + WriteBufferManager* write_buffer_manager_; + MemTable* mem_; MemTableList imm_; SuperVersion* super_version_; @@ -291,8 +385,9 @@ class ColumnFamilyData { // This needs to be destructed before mutex_ std::unique_ptr local_sv_; - // pointers for a circular linked list. we use it to support iterations - // that can be concurrent with writes + // pointers for a circular linked list. we use it to support iterations over + // all column families that are alive (note: dropped column families can also + // be alive as long as client holds a reference) ColumnFamilyData* next_; ColumnFamilyData* prev_; @@ -301,46 +396,43 @@ class ColumnFamilyData { // recovered from uint64_t log_number_; - // A flag indicating whether we should delay writes because - // we have too many level 0 files - bool need_slowdown_for_num_level0_files_; + // An object that keeps all the compaction stats + // and picks the next compaction + std::unique_ptr compaction_picker_; - // These 4 variables are updated only after compaction, - // adding new memtable, flushing memtables to files - // and/or add recalculation of compaction score. - // That's why theirs values are cached in ColumnFamilyData. - // Recalculation is made by RecalculateWriteStallConditions and - // RecalculateWriteStallRateLimitsConditions function. They are used - // in DBImpl::MakeRoomForWrite function to decide, if it need - // to sleep during write operation - bool need_wait_for_num_memtables_; + ColumnFamilySet* column_family_set_; - bool need_wait_for_num_level0_files_; + std::unique_ptr write_controller_token_; - bool exceeds_hard_rate_limit_; + // If true --> this ColumnFamily is currently present in DBImpl::flush_queue_ + bool pending_flush_; - bool exceeds_soft_rate_limit_; + // If true --> this ColumnFamily is currently present in + // DBImpl::compaction_queue_ + bool pending_compaction_; - // An object that keeps all the compaction stats - // and picks the next compaction - std::unique_ptr compaction_picker_; + uint64_t prev_compaction_needed_bytes_; - ColumnFamilySet* column_family_set_; + // if the database was opened with 2pc enabled + bool allow_2pc_; }; // ColumnFamilySet has interesting thread-safety requirements -// * CreateColumnFamily() or RemoveColumnFamily() -- need to protect by DB -// mutex. Inside, column_family_data_ and column_families_ will be protected -// by Lock() and Unlock(). CreateColumnFamily() should ONLY be called from -// VersionSet::LogAndApply() in the normal runtime. It is also called -// during Recovery and in DumpManifest(). RemoveColumnFamily() is called -// from ColumnFamilyData destructor +// * CreateColumnFamily() or RemoveColumnFamily() -- need to be protected by DB +// mutex AND executed in the write thread. +// CreateColumnFamily() should ONLY be called from VersionSet::LogAndApply() AND +// single-threaded write thread. It is also called during Recovery and in +// DumpManifest(). +// RemoveColumnFamily() is only called from SetDropped(). DB mutex needs to be +// held and it needs to be executed from the write thread. SetDropped() also +// guarantees that it will be called only from single-threaded LogAndApply(), +// but this condition is not that important. // * Iteration -- hold DB mutex, but you can release it in the body of // iteration. If you release DB mutex in body, reference the column // family before the mutex and unreference after you unlock, since the column // family might get dropped when the DB mutex is released // * GetDefault() -- thread safe -// * GetColumnFamily() -- either inside of DB mutex or call Lock() <-> Unlock() +// * GetColumnFamily() -- either inside of DB mutex or from a write thread // * GetNextColumnFamilyID(), GetMaxColumnFamily(), UpdateMaxColumnFamily(), // NumberOfColumnFamilies -- inside of DB mutex class ColumnFamilySet { @@ -351,10 +443,13 @@ class ColumnFamilySet { explicit iterator(ColumnFamilyData* cfd) : current_(cfd) {} iterator& operator++() { - // dummy is never dead or dropped, so this will never be infinite + // dropped column families might still be included in this iteration + // (we're only removing them when client drops the last reference to the + // column family). + // dummy is never dead, so this will never be infinite do { current_ = current_->next_; - } while (current_->refs_ == 0 || current_->IsDropped()); + } while (current_->refs_.load(std::memory_order_relaxed) == 0); return *this; } bool operator!=(const iterator& other) { @@ -366,8 +461,11 @@ class ColumnFamilySet { ColumnFamilyData* current_; }; - ColumnFamilySet(const std::string& dbname, const DBOptions* db_options, - const EnvOptions& storage_options, Cache* table_cache); + ColumnFamilySet(const std::string& dbname, + const ImmutableDBOptions* db_options, + const EnvOptions& env_options, Cache* table_cache, + WriteBufferManager* write_buffer_manager, + WriteController* write_controller); ~ColumnFamilySet(); ColumnFamilyData* GetDefault() const; @@ -390,13 +488,12 @@ class ColumnFamilySet { iterator begin() { return iterator(dummy_cfd_->next_); } iterator end() { return iterator(dummy_cfd_); } - void Lock(); - void Unlock(); - // REQUIRES: DB mutex held // Don't call while iterating over ColumnFamilySet void FreeDeadColumnFamilies(); + Cache* get_table_cache() { return table_cache_; } + private: friend class ColumnFamilyData; // helper function that gets called from cfd destructor @@ -404,9 +501,12 @@ class ColumnFamilySet { void RemoveColumnFamily(ColumnFamilyData* cfd); // column_families_ and column_family_data_ need to be protected: - // * when mutating: 1. DB mutex locked first, 2. spinlock locked second - // * when reading, either: 1. lock DB mutex, or 2. lock spinlock - // (if both, respect the ordering to avoid deadlock!) + // * when mutating both conditions have to be satisfied: + // 1. DB mutex locked + // 2. thread currently in single-threaded write thread + // * when reading, at least one condition needs to be satisfied: + // 1. DB mutex locked + // 2. accessed from a single-threaded write thread std::unordered_map column_families_; std::unordered_map column_family_data_; @@ -419,10 +519,11 @@ class ColumnFamilySet { ColumnFamilyData* default_cfd_cache_; const std::string db_name_; - const DBOptions* const db_options_; - const EnvOptions storage_options_; + const ImmutableDBOptions* const db_options_; + const EnvOptions env_options_; Cache* table_cache_; - std::atomic_flag spin_lock_; + WriteBufferManager* write_buffer_manager_; + WriteController* write_controller_; }; // We use ColumnFamilyMemTablesImpl to provide WriteBatch a way to access @@ -432,23 +533,36 @@ class ColumnFamilyMemTablesImpl : public ColumnFamilyMemTables { explicit ColumnFamilyMemTablesImpl(ColumnFamilySet* column_family_set) : column_family_set_(column_family_set), current_(nullptr) {} + // Constructs a ColumnFamilyMemTablesImpl equivalent to one constructed + // with the arguments used to construct *orig. + explicit ColumnFamilyMemTablesImpl(ColumnFamilyMemTablesImpl* orig) + : column_family_set_(orig->column_family_set_), current_(nullptr) {} + // sets current_ to ColumnFamilyData with column_family_id // returns false if column family doesn't exist + // REQUIRES: use this function of DBImpl::column_family_memtables_ should be + // under a DB mutex OR from a write thread bool Seek(uint32_t column_family_id) override; // Returns log number of the selected column family + // REQUIRES: under a DB mutex OR from a write thread uint64_t GetLogNumber() const override; // REQUIRES: Seek() called first + // REQUIRES: use this function of DBImpl::column_family_memtables_ should be + // under a DB mutex OR from a write thread virtual MemTable* GetMemTable() const override; - // Returns options for selected column family - // REQUIRES: Seek() called first - virtual const Options* GetOptions() const override; - // Returns column family handle for the selected column family + // REQUIRES: use this function of DBImpl::column_family_memtables_ should be + // under a DB mutex OR from a write thread virtual ColumnFamilyHandle* GetColumnFamilyHandle() override; + // Cannot be called while another thread is calling Seek(). + // REQUIRES: use this function of DBImpl::column_family_memtables_ should be + // under a DB mutex OR from a write thread + virtual ColumnFamilyData* current() override { return current_; } + private: ColumnFamilySet* column_family_set_; ColumnFamilyData* current_; @@ -457,4 +571,7 @@ class ColumnFamilyMemTablesImpl : public ColumnFamilyMemTables { extern uint32_t GetColumnFamilyID(ColumnFamilyHandle* column_family); +extern const Comparator* GetColumnFamilyUserComparator( + ColumnFamilyHandle* column_family); + } // namespace rocksdb diff --git a/db/column_family_test.cc b/db/column_family_test.cc index b96e66829a5..88786d469d5 100644 --- a/db/column_family_test.cc +++ b/db/column_family_test.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,17 +10,27 @@ #include #include #include +#include #include "db/db_impl.h" -#include "rocksdb/env.h" +#include "db/db_test_util.h" +#include "options/options_parser.h" +#include "port/port.h" #include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/iterator.h" +#include "util/coding.h" +#include "util/fault_injection_test_env.h" +#include "util/string_util.h" +#include "util/sync_point.h" #include "util/testharness.h" #include "util/testutil.h" -#include "util/coding.h" #include "utilities/merge_operators.h" namespace rocksdb { +static const int kValueSize = 1000; + namespace { std::string RandomString(Random* rnd, int len) { std::string r; @@ -38,32 +48,109 @@ class EnvCounter : public EnvWrapper { return num_new_writable_file_; } Status NewWritableFile(const std::string& f, unique_ptr* r, - const EnvOptions& soptions) { + const EnvOptions& soptions) override { ++num_new_writable_file_; return EnvWrapper::NewWritableFile(f, r, soptions); } private: - int num_new_writable_file_; + std::atomic num_new_writable_file_; }; -class ColumnFamilyTest { +class ColumnFamilyTest : public testing::Test { public: ColumnFamilyTest() : rnd_(139) { env_ = new EnvCounter(Env::Default()); dbname_ = test::TmpDir() + "/column_family_test"; db_options_.create_if_missing = true; + db_options_.fail_if_options_file_error = true; db_options_.env = env_; DestroyDB(dbname_, Options(db_options_, column_family_options_)); } ~ColumnFamilyTest() { + Close(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + Destroy(); delete env_; } + // Return the value to associate with the specified key + Slice Value(int k, std::string* storage) { + if (k == 0) { + // Ugh. Random seed of 0 used to produce no entropy. This code + // preserves the implementation that was in place when all of the + // magic values in this file were picked. + *storage = std::string(kValueSize, ' '); + return Slice(*storage); + } else { + Random r(k); + return test::RandomString(&r, kValueSize, storage); + } + } + + void Build(int base, int n, int flush_every = 0) { + std::string key_space, value_space; + WriteBatch batch; + + for (int i = 0; i < n; i++) { + if (flush_every != 0 && i != 0 && i % flush_every == 0) { + DBImpl* dbi = reinterpret_cast(db_); + dbi->TEST_FlushMemTable(); + } + + int keyi = base + i; + Slice key(DBTestBase::Key(keyi)); + + batch.Clear(); + batch.Put(handles_[0], key, Value(keyi, &value_space)); + batch.Put(handles_[1], key, Value(keyi, &value_space)); + batch.Put(handles_[2], key, Value(keyi, &value_space)); + ASSERT_OK(db_->Write(WriteOptions(), &batch)); + } + } + + void CheckMissed() { + uint64_t next_expected = 0; + uint64_t missed = 0; + int bad_keys = 0; + int bad_values = 0; + int correct = 0; + std::string value_space; + for (int cf = 0; cf < 3; cf++) { + next_expected = 0; + Iterator* iter = db_->NewIterator(ReadOptions(false, true), handles_[cf]); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + uint64_t key; + Slice in(iter->key()); + in.remove_prefix(3); + if (!ConsumeDecimalNumber(&in, &key) || !in.empty() || + key < next_expected) { + bad_keys++; + continue; + } + missed += (key - next_expected); + next_expected = key + 1; + if (iter->value() != Value(static_cast(key), &value_space)) { + bad_values++; + } else { + correct++; + } + } + delete iter; + } + + ASSERT_EQ(0, bad_keys); + ASSERT_EQ(0, bad_values); + ASSERT_EQ(0, missed); + (void)correct; + } + void Close() { for (auto h : handles_) { - delete h; + if (h) { + db_->DestroyColumnFamilyHandle(h); + } } handles_.clear(); names_.clear(); @@ -96,10 +183,12 @@ class ColumnFamilyTest { &db_); } +#ifndef ROCKSDB_LITE // ReadOnlyDB is not supported void AssertOpenReadOnly(std::vector cf, std::vector options = {}) { ASSERT_OK(OpenReadOnly(cf, options)); } +#endif // !ROCKSDB_LITE void Open(std::vector cf, @@ -115,32 +204,62 @@ class ColumnFamilyTest { int GetProperty(int cf, std::string property) { std::string value; - ASSERT_TRUE(dbfull()->GetProperty(handles_[cf], property, &value)); + EXPECT_TRUE(dbfull()->GetProperty(handles_[cf], property, &value)); +#ifndef CYGWIN return std::stoi(value); +#else + return std::strtol(value.c_str(), 0 /* off */, 10 /* base */); +#endif } - void Destroy() { - for (auto h : handles_) { - delete h; + bool IsDbWriteStopped() { +#ifndef ROCKSDB_LITE + uint64_t v; + EXPECT_TRUE(dbfull()->GetIntProperty("rocksdb.is-write-stopped", &v)); + return (v == 1); +#else + return dbfull()->TEST_write_controler().IsStopped(); +#endif // !ROCKSDB_LITE + } + + uint64_t GetDbDelayedWriteRate() { +#ifndef ROCKSDB_LITE + uint64_t v; + EXPECT_TRUE( + dbfull()->GetIntProperty("rocksdb.actual-delayed-write-rate", &v)); + return v; +#else + if (!dbfull()->TEST_write_controler().NeedsDelay()) { + return 0; } - handles_.clear(); - names_.clear(); - delete db_; - db_ = nullptr; + return dbfull()->TEST_write_controler().delayed_write_rate(); +#endif // !ROCKSDB_LITE + } + + void Destroy() { + Close(); ASSERT_OK(DestroyDB(dbname_, Options(db_options_, column_family_options_))); } void CreateColumnFamilies( const std::vector& cfs, const std::vector options = {}) { - int cfi = handles_.size(); + int cfi = static_cast(handles_.size()); handles_.resize(cfi + cfs.size()); names_.resize(cfi + cfs.size()); for (size_t i = 0; i < cfs.size(); ++i) { - ASSERT_OK(db_->CreateColumnFamily( - options.size() == 0 ? column_family_options_ : options[i], cfs[i], - &handles_[cfi])); + const auto& current_cf_opt = + options.size() == 0 ? column_family_options_ : options[i]; + ASSERT_OK( + db_->CreateColumnFamily(current_cf_opt, cfs[i], &handles_[cfi])); names_[cfi] = cfs[i]; + +#ifndef ROCKSDB_LITE // RocksDBLite does not support GetDescriptor + // Verify the CF options of the returned CF handle. + ColumnFamilyDescriptor desc; + ASSERT_OK(handles_[cfi]->GetDescriptor(&desc)); + RocksDBOptionsParser::VerifyCFOptions(desc.options, current_cf_opt); +#endif // !ROCKSDB_LITE cfi++; } } @@ -165,25 +284,44 @@ class ColumnFamilyTest { void DropColumnFamilies(const std::vector& cfs) { for (auto cf : cfs) { ASSERT_OK(db_->DropColumnFamily(handles_[cf])); - delete handles_[cf]; + db_->DestroyColumnFamilyHandle(handles_[cf]); handles_[cf] = nullptr; names_[cf] = ""; } } - void PutRandomData(int cf, int num, int key_value_size) { + void PutRandomData(int cf, int num, int key_value_size, bool save = false) { for (int i = 0; i < num; ++i) { // 10 bytes for key, rest is value - ASSERT_OK(Put(cf, test::RandomKey(&rnd_, 10), - RandomString(&rnd_, key_value_size - 10))); + if (!save) { + ASSERT_OK(Put(cf, test::RandomKey(&rnd_, 11), + RandomString(&rnd_, key_value_size - 10))); + } else { + std::string key = test::RandomKey(&rnd_, 11); + keys_.insert(key); + ASSERT_OK(Put(cf, key, RandomString(&rnd_, key_value_size - 10))); + } } + db_->FlushWAL(false); } +#ifndef ROCKSDB_LITE // TEST functions in DB are not supported in lite void WaitForFlush(int cf) { ASSERT_OK(dbfull()->TEST_WaitForFlushMemTable(handles_[cf])); } - void WaitForCompaction() { ASSERT_OK(dbfull()->TEST_WaitForCompact()); } + void WaitForCompaction() { + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + } + + uint64_t MaxTotalInMemoryState() { + return dbfull()->TEST_MaxTotalInMemoryState(); + } + + void AssertMaxTotalInMemoryState(uint64_t value) { + ASSERT_EQ(value, MaxTotalInMemoryState()); + } +#endif // !ROCKSDB_LITE Status Put(int cf, const std::string& key, const std::string& value) { return db_->Put(WriteOptions(), handles_[cf], Slice(key), Slice(value)); @@ -209,18 +347,21 @@ class ColumnFamilyTest { } void CompactAll(int cf) { - ASSERT_OK(db_->CompactRange(handles_[cf], nullptr, nullptr)); + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), handles_[cf], nullptr, + nullptr)); } void Compact(int cf, const Slice& start, const Slice& limit) { - ASSERT_OK(db_->CompactRange(handles_[cf], &start, &limit)); + ASSERT_OK( + db_->CompactRange(CompactRangeOptions(), handles_[cf], &start, &limit)); } int NumTableFilesAtLevel(int level, int cf) { return GetProperty(cf, - "rocksdb.num-files-at-level" + std::to_string(level)); + "rocksdb.num-files-at-level" + ToString(level)); } +#ifndef ROCKSDB_LITE // Return spread of files per level std::string FilesPerLevel(int cf) { std::string result; @@ -231,18 +372,33 @@ class ColumnFamilyTest { snprintf(buf, sizeof(buf), "%s%d", (level ? "," : ""), f); result += buf; if (f > 0) { - last_non_zero_offset = result.size(); + last_non_zero_offset = static_cast(result.size()); } } result.resize(last_non_zero_offset); return result; } +#endif + + void AssertFilesPerLevel(const std::string& value, int cf) { +#ifndef ROCKSDB_LITE + ASSERT_EQ(value, FilesPerLevel(cf)); +#endif + } +#ifndef ROCKSDB_LITE // GetLiveFilesMetaData is not supported int CountLiveFiles() { std::vector metadata; db_->GetLiveFilesMetaData(&metadata); return static_cast(metadata.size()); } +#endif // !ROCKSDB_LITE + + void AssertCountLiveFiles(int expected_value) { +#ifndef ROCKSDB_LITE + ASSERT_EQ(expected_value, CountLiveFiles()); +#endif + } // Do n memtable flushes, each of which produces an sstable // covering the range [small,large]. @@ -255,6 +411,7 @@ class ColumnFamilyTest { } } +#ifndef ROCKSDB_LITE // GetSortedWalFiles is not supported int CountLiveLogFiles() { int micros_wait_for_log_deletion = 20000; env_->SleepForMicroseconds(micros_wait_for_log_deletion); @@ -262,7 +419,7 @@ class ColumnFamilyTest { VectorLogPtr wal_files; Status s; // GetSortedWalFiles is a flakey function -- it gets all the wal_dir - // children files and then later checks for their existance. if some of the + // children files and then later checks for their existence. if some of the // log files doesn't exist anymore, it reports an error. it does all of this // without DB mutex held, so if a background process deletes the log file // while the function is being executed, it returns an error. We retry the @@ -274,22 +431,32 @@ class ColumnFamilyTest { break; } } - ASSERT_OK(s); + EXPECT_OK(s); for (const auto& wal : wal_files) { if (wal->Type() == kAliveLogFile) { ++ret; } } return ret; + return 0; + } +#endif // !ROCKSDB_LITE + + void AssertCountLiveLogFiles(int value) { +#ifndef ROCKSDB_LITE // GetSortedWalFiles is not supported + ASSERT_EQ(value, CountLiveLogFiles()); +#endif // !ROCKSDB_LITE } void AssertNumberOfImmutableMemtables(std::vector num_per_cf) { assert(num_per_cf.size() == handles_.size()); +#ifndef ROCKSDB_LITE // GetProperty is not supported in lite for (size_t i = 0; i < num_per_cf.size(); ++i) { - ASSERT_EQ(num_per_cf[i], - GetProperty(i, "rocksdb.num-immutable-mem-table")); + ASSERT_EQ(num_per_cf[i], GetProperty(static_cast(i), + "rocksdb.num-immutable-mem-table")); } +#endif // !ROCKSDB_LITE } void CopyFile(const std::string& source, const std::string& destination, @@ -318,6 +485,7 @@ class ColumnFamilyTest { std::vector handles_; std::vector names_; + std::set keys_; ColumnFamilyOptions column_family_options_; DBOptions db_options_; std::string dbname_; @@ -326,7 +494,7 @@ class ColumnFamilyTest { Random rnd_; }; -TEST(ColumnFamilyTest, DontReuseColumnFamilyID) { +TEST_F(ColumnFamilyTest, DontReuseColumnFamilyID) { for (int iter = 0; iter < 3; ++iter) { Open(); CreateColumnFamilies({"one", "two", "three"}); @@ -353,8 +521,162 @@ TEST(ColumnFamilyTest, DontReuseColumnFamilyID) { } } +#ifndef ROCKSDB_LITE +TEST_F(ColumnFamilyTest, CreateCFRaceWithGetAggProperty) { + Open(); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::WriteOptionsFile:1", + "ColumnFamilyTest.CreateCFRaceWithGetAggProperty:1"}, + {"ColumnFamilyTest.CreateCFRaceWithGetAggProperty:2", + "DBImpl::WriteOptionsFile:2"}}); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + rocksdb::port::Thread thread([&] { CreateColumnFamilies({"one"}); }); + + TEST_SYNC_POINT("ColumnFamilyTest.CreateCFRaceWithGetAggProperty:1"); + uint64_t pv; + db_->GetAggregatedIntProperty(DB::Properties::kEstimateTableReadersMem, &pv); + TEST_SYNC_POINT("ColumnFamilyTest.CreateCFRaceWithGetAggProperty:2"); + + thread.join(); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} +#endif // !ROCKSDB_LITE + +class FlushEmptyCFTestWithParam : public ColumnFamilyTest, + public testing::WithParamInterface { + public: + FlushEmptyCFTestWithParam() { allow_2pc_ = GetParam(); } + + // Required if inheriting from testing::WithParamInterface<> + static void SetUpTestCase() {} + static void TearDownTestCase() {} + + bool allow_2pc_; +}; + +TEST_P(FlushEmptyCFTestWithParam, FlushEmptyCFTest) { + std::unique_ptr fault_env( + new FaultInjectionTestEnv(env_)); + db_options_.env = fault_env.get(); + db_options_.allow_2pc = allow_2pc_; + Open(); + CreateColumnFamilies({"one", "two"}); + // Generate log file A. + ASSERT_OK(Put(1, "foo", "v1")); // seqID 1 + + Reopen(); + // Log file A is not dropped after reopening because default column family's + // min log number is 0. + // It flushes to SST file X + ASSERT_OK(Put(1, "foo", "v1")); // seqID 2 + ASSERT_OK(Put(1, "bar", "v2")); // seqID 3 + // Current log file is file B now. While flushing, a new log file C is created + // and is set to current. Boths' min log number is set to file C in memory, so + // after flushing file B is deleted. At the same time, the min log number of + // default CF is not written to manifest. Log file A still remains. + // Flushed to SST file Y. + Flush(1); + Flush(0); + ASSERT_OK(Put(1, "bar", "v3")); // seqID 4 + ASSERT_OK(Put(1, "foo", "v4")); // seqID 5 + db_->FlushWAL(false); + + // Preserve file system state up to here to simulate a crash condition. + fault_env->SetFilesystemActive(false); + std::vector names; + for (auto name : names_) { + if (name != "") { + names.push_back(name); + } + } + + Close(); + fault_env->ResetState(); + + // Before opening, there are four files: + // Log file A contains seqID 1 + // Log file C contains seqID 4, 5 + // SST file X contains seqID 1 + // SST file Y contains seqID 2, 3 + // Min log number: + // default CF: 0 + // CF one, two: C + // When opening the DB, all the seqID should be preserved. + Open(names, {}); + ASSERT_EQ("v4", Get(1, "foo")); + ASSERT_EQ("v3", Get(1, "bar")); + Close(); + + db_options_.env = env_; +} + +TEST_P(FlushEmptyCFTestWithParam, FlushEmptyCFTest2) { + std::unique_ptr fault_env( + new FaultInjectionTestEnv(env_)); + db_options_.env = fault_env.get(); + db_options_.allow_2pc = allow_2pc_; + Open(); + CreateColumnFamilies({"one", "two"}); + // Generate log file A. + ASSERT_OK(Put(1, "foo", "v1")); // seqID 1 + + Reopen(); + // Log file A is not dropped after reopening because default column family's + // min log number is 0. + // It flushes to SST file X + ASSERT_OK(Put(1, "foo", "v1")); // seqID 2 + ASSERT_OK(Put(1, "bar", "v2")); // seqID 3 + // Current log file is file B now. While flushing, a new log file C is created + // and is set to current. Both CFs' min log number is set to file C so after + // flushing file B is deleted. Log file A still remains. + // Flushed to SST file Y. + Flush(1); + ASSERT_OK(Put(0, "bar", "v2")); // seqID 4 + ASSERT_OK(Put(2, "bar", "v2")); // seqID 5 + ASSERT_OK(Put(1, "bar", "v3")); // seqID 6 + // Flushing all column families. This forces all CFs' min log to current. This + // is written to the manifest file. Log file C is cleared. + Flush(0); + Flush(1); + Flush(2); + // Write to log file D + ASSERT_OK(Put(1, "bar", "v4")); // seqID 7 + ASSERT_OK(Put(1, "bar", "v5")); // seqID 8 + db_->FlushWAL(false); + // Preserve file system state up to here to simulate a crash condition. + fault_env->SetFilesystemActive(false); + std::vector names; + for (auto name : names_) { + if (name != "") { + names.push_back(name); + } + } + + Close(); + fault_env->ResetState(); + // Before opening, there are two logfiles: + // Log file A contains seqID 1 + // Log file D contains seqID 7, 8 + // Min log number: + // default CF: D + // CF one, two: D + // When opening the DB, log file D should be replayed using the seqID + // specified in the file. + Open(names, {}); + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_EQ("v5", Get(1, "bar")); + Close(); + + db_options_.env = env_; +} + +INSTANTIATE_TEST_CASE_P(FlushEmptyCFTestWithParam, FlushEmptyCFTestWithParam, + ::testing::Bool()); -TEST(ColumnFamilyTest, AddDrop) { +TEST_F(ColumnFamilyTest, AddDrop) { Open(); CreateColumnFamilies({"one", "two", "three"}); ASSERT_EQ("NOT_FOUND", Get(1, "fodor")); @@ -375,19 +697,57 @@ TEST(ColumnFamilyTest, AddDrop) { std::vector families; ASSERT_OK(DB::ListColumnFamilies(db_options_, dbname_, &families)); - sort(families.begin(), families.end()); + std::sort(families.begin(), families.end()); ASSERT_TRUE(families == std::vector({"default", "four", "three"})); } -TEST(ColumnFamilyTest, DropTest) { +TEST_F(ColumnFamilyTest, BulkAddDrop) { + constexpr int kNumCF = 1000; + ColumnFamilyOptions cf_options; + WriteOptions write_options; + Open(); + std::vector cf_names; + std::vector cf_handles; + for (int i = 1; i <= kNumCF; i++) { + cf_names.push_back("cf1-" + ToString(i)); + } + ASSERT_OK(db_->CreateColumnFamilies(cf_options, cf_names, &cf_handles)); + for (int i = 1; i <= kNumCF; i++) { + ASSERT_OK(db_->Put(write_options, cf_handles[i - 1], "foo", "bar")); + } + ASSERT_OK(db_->DropColumnFamilies(cf_handles)); + std::vector cf_descriptors; + for (auto* handle : cf_handles) { + delete handle; + } + cf_handles.clear(); + for (int i = 1; i <= kNumCF; i++) { + cf_descriptors.emplace_back("cf2-" + ToString(i), ColumnFamilyOptions()); + } + ASSERT_OK(db_->CreateColumnFamilies(cf_descriptors, &cf_handles)); + for (int i = 1; i <= kNumCF; i++) { + ASSERT_OK(db_->Put(write_options, cf_handles[i - 1], "foo", "bar")); + } + ASSERT_OK(db_->DropColumnFamilies(cf_handles)); + for (auto* handle : cf_handles) { + delete handle; + } + Close(); + std::vector families; + ASSERT_OK(DB::ListColumnFamilies(db_options_, dbname_, &families)); + std::sort(families.begin(), families.end()); + ASSERT_TRUE(families == std::vector({"default"})); +} + +TEST_F(ColumnFamilyTest, DropTest) { // first iteration - dont reopen DB before dropping // second iteration - reopen DB before dropping for (int iter = 0; iter < 2; ++iter) { Open({"default"}); CreateColumnFamiliesAndReopen({"pikachu"}); for (int i = 0; i < 100; ++i) { - ASSERT_OK(Put(1, std::to_string(i), "bar" + std::to_string(i))); + ASSERT_OK(Put(1, ToString(i), "bar" + ToString(i))); } ASSERT_OK(Flush(1)); @@ -396,15 +756,15 @@ TEST(ColumnFamilyTest, DropTest) { } ASSERT_EQ("bar1", Get(1, "1")); - ASSERT_EQ(CountLiveFiles(), 1); + AssertCountLiveFiles(1); DropColumnFamilies({1}); // make sure that all files are deleted when we drop the column family - ASSERT_EQ(CountLiveFiles(), 0); + AssertCountLiveFiles(0); Destroy(); } } -TEST(ColumnFamilyTest, WriteBatchFailure) { +TEST_F(ColumnFamilyTest, WriteBatchFailure) { Open(); CreateColumnFamiliesAndReopen({"one", "two"}); WriteBatch batch; @@ -422,7 +782,7 @@ TEST(ColumnFamilyTest, WriteBatchFailure) { Close(); } -TEST(ColumnFamilyTest, ReadWrite) { +TEST_F(ColumnFamilyTest, ReadWrite) { Open(); CreateColumnFamiliesAndReopen({"one", "two"}); ASSERT_OK(Put(0, "foo", "v1")); @@ -446,7 +806,7 @@ TEST(ColumnFamilyTest, ReadWrite) { Close(); } -TEST(ColumnFamilyTest, IgnoreRecoveredLog) { +TEST_F(ColumnFamilyTest, IgnoreRecoveredLog) { std::string backup_logs = dbname_ + "/backup_logs"; // delete old files in backup_logs directory @@ -521,7 +881,8 @@ TEST(ColumnFamilyTest, IgnoreRecoveredLog) { } } -TEST(ColumnFamilyTest, FlushTest) { +#ifndef ROCKSDB_LITE // TEST functions used are not supported +TEST_F(ColumnFamilyTest, FlushTest) { Open(); CreateColumnFamiliesAndReopen({"one", "two"}); ASSERT_OK(Put(0, "foo", "v1")); @@ -529,8 +890,27 @@ TEST(ColumnFamilyTest, FlushTest) { ASSERT_OK(Put(1, "mirko", "v3")); ASSERT_OK(Put(0, "foo", "v2")); ASSERT_OK(Put(2, "fodor", "v5")); - for (int i = 0; i < 3; ++i) { - Flush(i); + + for (int j = 0; j < 2; j++) { + ReadOptions ro; + std::vector iterators; + // Hold super version. + if (j == 0) { + ASSERT_OK(db_->NewIterators(ro, handles_, &iterators)); + } + + for (int i = 0; i < 3; ++i) { + uint64_t max_total_in_memory_state = + MaxTotalInMemoryState(); + Flush(i); + AssertMaxTotalInMemoryState(max_total_in_memory_state); + } + ASSERT_OK(Put(1, "foofoo", "bar")); + ASSERT_OK(Put(0, "foofoo", "bar")); + + for (auto* it : iterators) { + delete it; + } } Reopen(); @@ -550,15 +930,16 @@ TEST(ColumnFamilyTest, FlushTest) { } // Makes sure that obsolete log files get deleted -TEST(ColumnFamilyTest, LogDeletionTest) { +TEST_F(ColumnFamilyTest, LogDeletionTest) { db_options_.max_total_wal_size = std::numeric_limits::max(); + column_family_options_.arena_block_size = 4 * 1024; column_family_options_.write_buffer_size = 100000; // 100KB Open(); CreateColumnFamilies({"one", "two", "three", "four"}); // Each bracket is one log file. if number is in (), it means // we don't need it anymore (it's been flushed) // [] - ASSERT_EQ(CountLiveLogFiles(), 0); + AssertCountLiveLogFiles(0); PutRandomData(0, 1, 100); // [0] PutRandomData(1, 1, 100); @@ -566,58 +947,98 @@ TEST(ColumnFamilyTest, LogDeletionTest) { PutRandomData(1, 1000, 100); WaitForFlush(1); // [0, (1)] [1] - ASSERT_EQ(CountLiveLogFiles(), 2); + AssertCountLiveLogFiles(2); PutRandomData(0, 1, 100); // [0, (1)] [0, 1] - ASSERT_EQ(CountLiveLogFiles(), 2); + AssertCountLiveLogFiles(2); PutRandomData(2, 1, 100); // [0, (1)] [0, 1, 2] PutRandomData(2, 1000, 100); WaitForFlush(2); // [0, (1)] [0, 1, (2)] [2] - ASSERT_EQ(CountLiveLogFiles(), 3); + AssertCountLiveLogFiles(3); PutRandomData(2, 1000, 100); WaitForFlush(2); // [0, (1)] [0, 1, (2)] [(2)] [2] - ASSERT_EQ(CountLiveLogFiles(), 4); + AssertCountLiveLogFiles(4); PutRandomData(3, 1, 100); // [0, (1)] [0, 1, (2)] [(2)] [2, 3] PutRandomData(1, 1, 100); // [0, (1)] [0, 1, (2)] [(2)] [1, 2, 3] - ASSERT_EQ(CountLiveLogFiles(), 4); + AssertCountLiveLogFiles(4); PutRandomData(1, 1000, 100); WaitForFlush(1); // [0, (1)] [0, (1), (2)] [(2)] [(1), 2, 3] [1] - ASSERT_EQ(CountLiveLogFiles(), 5); + AssertCountLiveLogFiles(5); PutRandomData(0, 1000, 100); WaitForFlush(0); // [(0), (1)] [(0), (1), (2)] [(2)] [(1), 2, 3] [1, (0)] [0] // delete obsolete logs --> // [(1), 2, 3] [1, (0)] [0] - ASSERT_EQ(CountLiveLogFiles(), 3); + AssertCountLiveLogFiles(3); PutRandomData(0, 1000, 100); WaitForFlush(0); // [(1), 2, 3] [1, (0)], [(0)] [0] - ASSERT_EQ(CountLiveLogFiles(), 4); + AssertCountLiveLogFiles(4); PutRandomData(1, 1000, 100); WaitForFlush(1); // [(1), 2, 3] [(1), (0)] [(0)] [0, (1)] [1] - ASSERT_EQ(CountLiveLogFiles(), 5); + AssertCountLiveLogFiles(5); PutRandomData(2, 1000, 100); WaitForFlush(2); // [(1), (2), 3] [(1), (0)] [(0)] [0, (1)] [1, (2)], [2] - ASSERT_EQ(CountLiveLogFiles(), 6); + AssertCountLiveLogFiles(6); PutRandomData(3, 1000, 100); WaitForFlush(3); // [(1), (2), (3)] [(1), (0)] [(0)] [0, (1)] [1, (2)], [2, (3)] [3] // delete obsolete logs --> // [0, (1)] [1, (2)], [2, (3)] [3] - ASSERT_EQ(CountLiveLogFiles(), 4); + AssertCountLiveLogFiles(4); + Close(); +} +#endif // !ROCKSDB_LITE + +TEST_F(ColumnFamilyTest, CrashAfterFlush) { + std::unique_ptr fault_env( + new FaultInjectionTestEnv(env_)); + db_options_.env = fault_env.get(); + Open(); + CreateColumnFamilies({"one"}); + + WriteBatch batch; + batch.Put(handles_[0], Slice("foo"), Slice("bar")); + batch.Put(handles_[1], Slice("foo"), Slice("bar")); + ASSERT_OK(db_->Write(WriteOptions(), &batch)); + Flush(0); + fault_env->SetFilesystemActive(false); + + std::vector names; + for (auto name : names_) { + if (name != "") { + names.push_back(name); + } + } + Close(); + fault_env->DropUnsyncedFileData(); + fault_env->ResetState(); + Open(names, {}); + + // Write batch should be atomic. + ASSERT_EQ(Get(0, "foo"), Get(1, "foo")); + + Close(); + db_options_.env = env_; +} + +TEST_F(ColumnFamilyTest, OpenNonexistentColumnFamily) { + ASSERT_OK(TryOpen({"default"})); Close(); + ASSERT_TRUE(TryOpen({"default", "dne"}).IsInvalidArgument()); } +#ifndef ROCKSDB_LITE // WaitForFlush() is not supported // Makes sure that obsolete log files get deleted -TEST(ColumnFamilyTest, DifferentWriteBufferSizes) { +TEST_F(ColumnFamilyTest, DifferentWriteBufferSizes) { // disable flushing stale column families db_options_.max_total_wal_size = std::numeric_limits::max(); Open(); @@ -629,17 +1050,25 @@ TEST(ColumnFamilyTest, DifferentWriteBufferSizes) { // "two" -> 1MB memtable, start flushing with three immutable memtables // "three" -> 90KB memtable, start flushing with four immutable memtables default_cf.write_buffer_size = 100000; + default_cf.arena_block_size = 4 * 4096; default_cf.max_write_buffer_number = 10; default_cf.min_write_buffer_number_to_merge = 1; + default_cf.max_write_buffer_number_to_maintain = 0; one.write_buffer_size = 200000; + one.arena_block_size = 4 * 4096; one.max_write_buffer_number = 10; one.min_write_buffer_number_to_merge = 2; + one.max_write_buffer_number_to_maintain = 1; two.write_buffer_size = 1000000; + two.arena_block_size = 4 * 4096; two.max_write_buffer_number = 10; two.min_write_buffer_number_to_merge = 3; - three.write_buffer_size = 90000; + two.max_write_buffer_number_to_maintain = 2; + three.write_buffer_size = 4096 * 22; + three.arena_block_size = 4096; three.max_write_buffer_number = 10; three.min_write_buffer_number_to_merge = 4; + three.max_write_buffer_number_to_maintain = -1; Reopen({default_cf, one, two, three}); @@ -647,71 +1076,131 @@ TEST(ColumnFamilyTest, DifferentWriteBufferSizes) { PutRandomData(0, 100, 1000); WaitForFlush(0); AssertNumberOfImmutableMemtables({0, 0, 0, 0}); - ASSERT_EQ(CountLiveLogFiles(), 1); + AssertCountLiveLogFiles(1); PutRandomData(1, 200, 1000); env_->SleepForMicroseconds(micros_wait_for_flush); AssertNumberOfImmutableMemtables({0, 1, 0, 0}); - ASSERT_EQ(CountLiveLogFiles(), 2); + AssertCountLiveLogFiles(2); PutRandomData(2, 1000, 1000); env_->SleepForMicroseconds(micros_wait_for_flush); AssertNumberOfImmutableMemtables({0, 1, 1, 0}); - ASSERT_EQ(CountLiveLogFiles(), 3); + AssertCountLiveLogFiles(3); PutRandomData(2, 1000, 1000); env_->SleepForMicroseconds(micros_wait_for_flush); AssertNumberOfImmutableMemtables({0, 1, 2, 0}); - ASSERT_EQ(CountLiveLogFiles(), 4); - PutRandomData(3, 90, 1000); + AssertCountLiveLogFiles(4); + PutRandomData(3, 93, 990); env_->SleepForMicroseconds(micros_wait_for_flush); AssertNumberOfImmutableMemtables({0, 1, 2, 1}); - ASSERT_EQ(CountLiveLogFiles(), 5); - PutRandomData(3, 90, 1000); + AssertCountLiveLogFiles(5); + PutRandomData(3, 88, 990); env_->SleepForMicroseconds(micros_wait_for_flush); AssertNumberOfImmutableMemtables({0, 1, 2, 2}); - ASSERT_EQ(CountLiveLogFiles(), 6); - PutRandomData(3, 90, 1000); + AssertCountLiveLogFiles(6); + PutRandomData(3, 88, 990); env_->SleepForMicroseconds(micros_wait_for_flush); AssertNumberOfImmutableMemtables({0, 1, 2, 3}); - ASSERT_EQ(CountLiveLogFiles(), 7); + AssertCountLiveLogFiles(7); PutRandomData(0, 100, 1000); WaitForFlush(0); AssertNumberOfImmutableMemtables({0, 1, 2, 3}); - ASSERT_EQ(CountLiveLogFiles(), 8); + AssertCountLiveLogFiles(8); PutRandomData(2, 100, 10000); WaitForFlush(2); AssertNumberOfImmutableMemtables({0, 1, 0, 3}); - ASSERT_EQ(CountLiveLogFiles(), 9); - PutRandomData(3, 90, 1000); + AssertCountLiveLogFiles(9); + PutRandomData(3, 88, 990); WaitForFlush(3); AssertNumberOfImmutableMemtables({0, 1, 0, 0}); - ASSERT_EQ(CountLiveLogFiles(), 10); - PutRandomData(3, 90, 1000); + AssertCountLiveLogFiles(10); + PutRandomData(3, 88, 990); env_->SleepForMicroseconds(micros_wait_for_flush); AssertNumberOfImmutableMemtables({0, 1, 0, 1}); - ASSERT_EQ(CountLiveLogFiles(), 11); + AssertCountLiveLogFiles(11); PutRandomData(1, 200, 1000); WaitForFlush(1); AssertNumberOfImmutableMemtables({0, 0, 0, 1}); - ASSERT_EQ(CountLiveLogFiles(), 5); - PutRandomData(3, 90*6, 1000); + AssertCountLiveLogFiles(5); + PutRandomData(3, 88 * 3, 990); + WaitForFlush(3); + PutRandomData(3, 88 * 4, 990); WaitForFlush(3); AssertNumberOfImmutableMemtables({0, 0, 0, 0}); - ASSERT_EQ(CountLiveLogFiles(), 12); + AssertCountLiveLogFiles(12); PutRandomData(0, 100, 1000); WaitForFlush(0); AssertNumberOfImmutableMemtables({0, 0, 0, 0}); - ASSERT_EQ(CountLiveLogFiles(), 12); - PutRandomData(2, 3*100, 10000); + AssertCountLiveLogFiles(12); + PutRandomData(2, 3 * 1000, 1000); WaitForFlush(2); AssertNumberOfImmutableMemtables({0, 0, 0, 0}); - ASSERT_EQ(CountLiveLogFiles(), 12); + AssertCountLiveLogFiles(12); PutRandomData(1, 2*200, 1000); WaitForFlush(1); AssertNumberOfImmutableMemtables({0, 0, 0, 0}); - ASSERT_EQ(CountLiveLogFiles(), 7); + AssertCountLiveLogFiles(7); + Close(); +} +#endif // !ROCKSDB_LITE + +#ifndef ROCKSDB_LITE // Cuckoo is not supported in lite +TEST_F(ColumnFamilyTest, MemtableNotSupportSnapshot) { + db_options_.allow_concurrent_memtable_write = false; + Open(); + auto* s1 = dbfull()->GetSnapshot(); + ASSERT_TRUE(s1 != nullptr); + dbfull()->ReleaseSnapshot(s1); + + // Add a column family that doesn't support snapshot + ColumnFamilyOptions first; + first.memtable_factory.reset(NewHashCuckooRepFactory(1024 * 1024)); + CreateColumnFamilies({"first"}, {first}); + auto* s2 = dbfull()->GetSnapshot(); + ASSERT_TRUE(s2 == nullptr); + + // Add a column family that supports snapshot. Snapshot stays not supported. + ColumnFamilyOptions second; + CreateColumnFamilies({"second"}, {second}); + auto* s3 = dbfull()->GetSnapshot(); + ASSERT_TRUE(s3 == nullptr); + Close(); +} +#endif // !ROCKSDB_LITE + +class TestComparator : public Comparator { + int Compare(const rocksdb::Slice& a, const rocksdb::Slice& b) const override { + return 0; + } + const char* Name() const override { return "Test"; } + void FindShortestSeparator(std::string* start, + const rocksdb::Slice& limit) const override {} + void FindShortSuccessor(std::string* key) const override {} +}; + +static TestComparator third_comparator; +static TestComparator fourth_comparator; + +// Test that we can retrieve the comparator from a created CF +TEST_F(ColumnFamilyTest, GetComparator) { + Open(); + // Add a column family with no comparator specified + CreateColumnFamilies({"first"}); + const Comparator* comp = handles_[0]->GetComparator(); + ASSERT_EQ(comp, BytewiseComparator()); + + // Add three column families - one with no comparator and two + // with comparators specified + ColumnFamilyOptions second, third, fourth; + second.comparator = &third_comparator; + third.comparator = &fourth_comparator; + CreateColumnFamilies({"second", "third", "fourth"}, {second, third, fourth}); + ASSERT_EQ(handles_[1]->GetComparator(), BytewiseComparator()); + ASSERT_EQ(handles_[2]->GetComparator(), &third_comparator); + ASSERT_EQ(handles_[3]->GetComparator(), &fourth_comparator); Close(); } -TEST(ColumnFamilyTest, DifferentMergeOperators) { +TEST_F(ColumnFamilyTest, DifferentMergeOperators) { Open(); CreateColumnFamilies({"first", "second"}); ColumnFamilyOptions default_cf, first, second; @@ -741,30 +1230,32 @@ TEST(ColumnFamilyTest, DifferentMergeOperators) { Close(); } -TEST(ColumnFamilyTest, DifferentCompactionStyles) { +#ifndef ROCKSDB_LITE // WaitForFlush() is not supported +TEST_F(ColumnFamilyTest, DifferentCompactionStyles) { Open(); CreateColumnFamilies({"one", "two"}); ColumnFamilyOptions default_cf, one, two; db_options_.max_open_files = 20; // only 10 files in file cache - db_options_.disableDataSync = true; default_cf.compaction_style = kCompactionStyleLevel; default_cf.num_levels = 3; default_cf.write_buffer_size = 64 << 10; // 64KB default_cf.target_file_size_base = 30 << 10; - default_cf.source_compaction_factor = 100; + default_cf.max_compaction_bytes = static_cast(1) << 60; + BlockBasedTableOptions table_options; table_options.no_block_cache = true; default_cf.table_factory.reset(NewBlockBasedTableFactory(table_options)); one.compaction_style = kCompactionStyleUniversal; + + one.num_levels = 1; // trigger compaction if there are >= 4 files one.level0_file_num_compaction_trigger = 4; - one.write_buffer_size = 100000; + one.write_buffer_size = 120000; two.compaction_style = kCompactionStyleLevel; two.num_levels = 4; - two.max_mem_compaction_level = 0; two.level0_file_num_compaction_trigger = 3; two.write_buffer_size = 100000; @@ -772,172 +1263,963 @@ TEST(ColumnFamilyTest, DifferentCompactionStyles) { // SETUP column family "one" -- universal style for (int i = 0; i < one.level0_file_num_compaction_trigger - 1; ++i) { - PutRandomData(1, 11, 10000); + PutRandomData(1, 10, 12000); + PutRandomData(1, 1, 10); WaitForFlush(1); - ASSERT_EQ(std::to_string(i + 1), FilesPerLevel(1)); + AssertFilesPerLevel(ToString(i + 1), 1); } // SETUP column family "two" -- level style with 4 levels for (int i = 0; i < two.level0_file_num_compaction_trigger - 1; ++i) { - PutRandomData(2, 15, 10000); + PutRandomData(2, 10, 12000); + PutRandomData(2, 1, 10); WaitForFlush(2); - ASSERT_EQ(std::to_string(i + 1), FilesPerLevel(2)); + AssertFilesPerLevel(ToString(i + 1), 2); } // TRIGGER compaction "one" - PutRandomData(1, 12, 10000); + PutRandomData(1, 10, 12000); + PutRandomData(1, 1, 10); // TRIGGER compaction "two" - PutRandomData(2, 10, 10000); + PutRandomData(2, 10, 12000); + PutRandomData(2, 1, 10); // WAIT for compactions WaitForCompaction(); // VERIFY compaction "one" - ASSERT_EQ("1", FilesPerLevel(1)); + AssertFilesPerLevel("1", 1); // VERIFY compaction "two" - ASSERT_EQ("0,1", FilesPerLevel(2)); + AssertFilesPerLevel("0,1", 2); CompactAll(2); - ASSERT_EQ("0,1", FilesPerLevel(2)); + AssertFilesPerLevel("0,1", 2); Close(); } +#endif // !ROCKSDB_LITE -namespace { -std::string IterStatus(Iterator* iter) { - std::string result; - if (iter->Valid()) { - result = iter->key().ToString() + "->" + iter->value().ToString(); - } else { - result = "(invalid)"; - } - return result; -} -} // anonymous namespace +#ifndef ROCKSDB_LITE +// Sync points not supported in RocksDB Lite -TEST(ColumnFamilyTest, NewIteratorsTest) { - // iter == 0 -- no tailing - // iter == 2 -- tailing - for (int iter = 0; iter < 2; ++iter) { - Open(); - CreateColumnFamiliesAndReopen({"one", "two"}); - ASSERT_OK(Put(0, "a", "b")); - ASSERT_OK(Put(1, "b", "a")); - ASSERT_OK(Put(2, "c", "m")); - ASSERT_OK(Put(2, "v", "t")); - std::vector iterators; - ReadOptions options; - options.tailing = (iter == 1); - ASSERT_OK(db_->NewIterators(options, handles_, &iterators)); +TEST_F(ColumnFamilyTest, MultipleManualCompactions) { + Open(); + CreateColumnFamilies({"one", "two"}); + ColumnFamilyOptions default_cf, one, two; + db_options_.max_open_files = 20; // only 10 files in file cache + db_options_.max_background_compactions = 3; - for (auto it : iterators) { - it->SeekToFirst(); - } - ASSERT_EQ(IterStatus(iterators[0]), "a->b"); - ASSERT_EQ(IterStatus(iterators[1]), "b->a"); - ASSERT_EQ(IterStatus(iterators[2]), "c->m"); + default_cf.compaction_style = kCompactionStyleLevel; + default_cf.num_levels = 3; + default_cf.write_buffer_size = 64 << 10; // 64KB + default_cf.target_file_size_base = 30 << 10; + default_cf.max_compaction_bytes = default_cf.target_file_size_base * 1100; + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + default_cf.table_factory.reset(NewBlockBasedTableFactory(table_options)); - ASSERT_OK(Put(1, "x", "x")); + one.compaction_style = kCompactionStyleUniversal; - for (auto it : iterators) { - it->Next(); - } + one.num_levels = 1; + // trigger compaction if there are >= 4 files + one.level0_file_num_compaction_trigger = 4; + one.write_buffer_size = 120000; - ASSERT_EQ(IterStatus(iterators[0]), "(invalid)"); - if (iter == 0) { - // no tailing - ASSERT_EQ(IterStatus(iterators[1]), "(invalid)"); - } else { - // tailing - ASSERT_EQ(IterStatus(iterators[1]), "x->x"); - } - ASSERT_EQ(IterStatus(iterators[2]), "v->t"); + two.compaction_style = kCompactionStyleLevel; + two.num_levels = 4; + two.level0_file_num_compaction_trigger = 3; + two.write_buffer_size = 100000; - for (auto it : iterators) { - delete it; - } - Destroy(); - } -} + Reopen({default_cf, one, two}); -TEST(ColumnFamilyTest, ReadOnlyDBTest) { - Open(); - CreateColumnFamiliesAndReopen({"one", "two", "three", "four"}); - ASSERT_OK(Put(0, "a", "b")); - ASSERT_OK(Put(1, "foo", "bla")); - ASSERT_OK(Put(2, "foo", "blabla")); - ASSERT_OK(Put(3, "foo", "blablabla")); - ASSERT_OK(Put(4, "foo", "blablablabla")); + // SETUP column family "one" -- universal style + for (int i = 0; i < one.level0_file_num_compaction_trigger - 2; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(i + 1), 1); + } + bool cf_1_1 = true; + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"ColumnFamilyTest::MultiManual:4", "ColumnFamilyTest::MultiManual:1"}, + {"ColumnFamilyTest::MultiManual:2", "ColumnFamilyTest::MultiManual:5"}, + {"ColumnFamilyTest::MultiManual:2", "ColumnFamilyTest::MultiManual:3"}}); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) { + if (cf_1_1) { + TEST_SYNC_POINT("ColumnFamilyTest::MultiManual:4"); + cf_1_1 = false; + TEST_SYNC_POINT("ColumnFamilyTest::MultiManual:3"); + } + }); - DropColumnFamilies({2}); - Close(); - // open only a subset of column families - AssertOpenReadOnly({"default", "one", "four"}); - ASSERT_EQ("NOT_FOUND", Get(0, "foo")); - ASSERT_EQ("bla", Get(1, "foo")); - ASSERT_EQ("blablablabla", Get(2, "foo")); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + std::vector threads; + threads.emplace_back([&] { + CompactRangeOptions compact_options; + compact_options.exclusive_manual_compaction = false; + ASSERT_OK( + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr)); + }); + // SETUP column family "two" -- level style with 4 levels + for (int i = 0; i < two.level0_file_num_compaction_trigger - 2; ++i) { + PutRandomData(2, 10, 12000); + PutRandomData(2, 1, 10); + WaitForFlush(2); + AssertFilesPerLevel(ToString(i + 1), 2); + } + threads.emplace_back([&] { + TEST_SYNC_POINT("ColumnFamilyTest::MultiManual:1"); + CompactRangeOptions compact_options; + compact_options.exclusive_manual_compaction = false; + ASSERT_OK( + db_->CompactRange(compact_options, handles_[2], nullptr, nullptr)); + TEST_SYNC_POINT("ColumnFamilyTest::MultiManual:2"); + }); + + TEST_SYNC_POINT("ColumnFamilyTest::MultiManual:5"); + for (auto& t : threads) { + t.join(); + } - // test newiterators - { - std::vector iterators; - ASSERT_OK(db_->NewIterators(ReadOptions(), handles_, &iterators)); - for (auto it : iterators) { - it->SeekToFirst(); - } - ASSERT_EQ(IterStatus(iterators[0]), "a->b"); - ASSERT_EQ(IterStatus(iterators[1]), "foo->bla"); - ASSERT_EQ(IterStatus(iterators[2]), "foo->blablablabla"); - for (auto it : iterators) { - it->Next(); - } - ASSERT_EQ(IterStatus(iterators[0]), "(invalid)"); - ASSERT_EQ(IterStatus(iterators[1]), "(invalid)"); - ASSERT_EQ(IterStatus(iterators[2]), "(invalid)"); + // VERIFY compaction "one" + AssertFilesPerLevel("1", 1); - for (auto it : iterators) { - delete it; - } + // VERIFY compaction "two" + AssertFilesPerLevel("0,1", 2); + CompactAll(2); + AssertFilesPerLevel("0,1", 2); + // Compare against saved keys + std::set::iterator key_iter = keys_.begin(); + while (key_iter != keys_.end()) { + ASSERT_NE("NOT_FOUND", Get(1, *key_iter)); + key_iter++; } - Close(); - // can't open dropped column family - Status s = OpenReadOnly({"default", "one", "two"}); - ASSERT_TRUE(!s.ok()); +} + +TEST_F(ColumnFamilyTest, AutomaticAndManualCompactions) { + Open(); + CreateColumnFamilies({"one", "two"}); + ColumnFamilyOptions default_cf, one, two; + db_options_.max_open_files = 20; // only 10 files in file cache + db_options_.max_background_compactions = 3; + + default_cf.compaction_style = kCompactionStyleLevel; + default_cf.num_levels = 3; + default_cf.write_buffer_size = 64 << 10; // 64KB + default_cf.target_file_size_base = 30 << 10; + default_cf.max_compaction_bytes = default_cf.target_file_size_base * 1100; + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + default_cf.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + one.compaction_style = kCompactionStyleUniversal; + + one.num_levels = 1; + // trigger compaction if there are >= 4 files + one.level0_file_num_compaction_trigger = 4; + one.write_buffer_size = 120000; + + two.compaction_style = kCompactionStyleLevel; + two.num_levels = 4; + two.level0_file_num_compaction_trigger = 3; + two.write_buffer_size = 100000; + + Reopen({default_cf, one, two}); + // make sure all background compaction jobs can be scheduled + auto stop_token = + dbfull()->TEST_write_controler().GetCompactionPressureToken(); + + bool cf_1_1 = true; + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"ColumnFamilyTest::AutoManual:4", "ColumnFamilyTest::AutoManual:1"}, + {"ColumnFamilyTest::AutoManual:2", "ColumnFamilyTest::AutoManual:5"}, + {"ColumnFamilyTest::AutoManual:2", "ColumnFamilyTest::AutoManual:3"}}); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) { + if (cf_1_1) { + cf_1_1 = false; + TEST_SYNC_POINT("ColumnFamilyTest::AutoManual:4"); + TEST_SYNC_POINT("ColumnFamilyTest::AutoManual:3"); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + // SETUP column family "one" -- universal style + for (int i = 0; i < one.level0_file_num_compaction_trigger; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(i + 1), 1); + } + + TEST_SYNC_POINT("ColumnFamilyTest::AutoManual:1"); + + // SETUP column family "two" -- level style with 4 levels + for (int i = 0; i < two.level0_file_num_compaction_trigger - 2; ++i) { + PutRandomData(2, 10, 12000); + PutRandomData(2, 1, 10); + WaitForFlush(2); + AssertFilesPerLevel(ToString(i + 1), 2); + } + rocksdb::port::Thread threads([&] { + CompactRangeOptions compact_options; + compact_options.exclusive_manual_compaction = false; + ASSERT_OK( + db_->CompactRange(compact_options, handles_[2], nullptr, nullptr)); + TEST_SYNC_POINT("ColumnFamilyTest::AutoManual:2"); + }); + + TEST_SYNC_POINT("ColumnFamilyTest::AutoManual:5"); + threads.join(); + + // WAIT for compactions + WaitForCompaction(); + + // VERIFY compaction "one" + AssertFilesPerLevel("1", 1); + + // VERIFY compaction "two" + AssertFilesPerLevel("0,1", 2); + CompactAll(2); + AssertFilesPerLevel("0,1", 2); + // Compare against saved keys + std::set::iterator key_iter = keys_.begin(); + while (key_iter != keys_.end()) { + ASSERT_NE("NOT_FOUND", Get(1, *key_iter)); + key_iter++; + } +} + +TEST_F(ColumnFamilyTest, ManualAndAutomaticCompactions) { + Open(); + CreateColumnFamilies({"one", "two"}); + ColumnFamilyOptions default_cf, one, two; + db_options_.max_open_files = 20; // only 10 files in file cache + db_options_.max_background_compactions = 3; + + default_cf.compaction_style = kCompactionStyleLevel; + default_cf.num_levels = 3; + default_cf.write_buffer_size = 64 << 10; // 64KB + default_cf.target_file_size_base = 30 << 10; + default_cf.max_compaction_bytes = default_cf.target_file_size_base * 1100; + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + default_cf.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + one.compaction_style = kCompactionStyleUniversal; + + one.num_levels = 1; + // trigger compaction if there are >= 4 files + one.level0_file_num_compaction_trigger = 4; + one.write_buffer_size = 120000; + + two.compaction_style = kCompactionStyleLevel; + two.num_levels = 4; + two.level0_file_num_compaction_trigger = 3; + two.write_buffer_size = 100000; + + Reopen({default_cf, one, two}); + // make sure all background compaction jobs can be scheduled + auto stop_token = + dbfull()->TEST_write_controler().GetCompactionPressureToken(); + + // SETUP column family "one" -- universal style + for (int i = 0; i < one.level0_file_num_compaction_trigger - 2; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(i + 1), 1); + } + bool cf_1_1 = true; + bool cf_1_2 = true; + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"ColumnFamilyTest::ManualAuto:4", "ColumnFamilyTest::ManualAuto:1"}, + {"ColumnFamilyTest::ManualAuto:5", "ColumnFamilyTest::ManualAuto:2"}, + {"ColumnFamilyTest::ManualAuto:2", "ColumnFamilyTest::ManualAuto:3"}}); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) { + if (cf_1_1) { + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:4"); + cf_1_1 = false; + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:3"); + } else if (cf_1_2) { + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:2"); + cf_1_2 = false; + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + rocksdb::port::Thread threads([&] { + CompactRangeOptions compact_options; + compact_options.exclusive_manual_compaction = false; + ASSERT_OK( + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr)); + }); + + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:1"); + + // SETUP column family "two" -- level style with 4 levels + for (int i = 0; i < two.level0_file_num_compaction_trigger; ++i) { + PutRandomData(2, 10, 12000); + PutRandomData(2, 1, 10); + WaitForFlush(2); + AssertFilesPerLevel(ToString(i + 1), 2); + } + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:5"); + threads.join(); + + // WAIT for compactions + WaitForCompaction(); + + // VERIFY compaction "one" + AssertFilesPerLevel("1", 1); + + // VERIFY compaction "two" + AssertFilesPerLevel("0,1", 2); + CompactAll(2); + AssertFilesPerLevel("0,1", 2); + // Compare against saved keys + std::set::iterator key_iter = keys_.begin(); + while (key_iter != keys_.end()) { + ASSERT_NE("NOT_FOUND", Get(1, *key_iter)); + key_iter++; + } +} + +TEST_F(ColumnFamilyTest, SameCFManualManualCompactions) { + Open(); + CreateColumnFamilies({"one"}); + ColumnFamilyOptions default_cf, one; + db_options_.max_open_files = 20; // only 10 files in file cache + db_options_.max_background_compactions = 3; + + default_cf.compaction_style = kCompactionStyleLevel; + default_cf.num_levels = 3; + default_cf.write_buffer_size = 64 << 10; // 64KB + default_cf.target_file_size_base = 30 << 10; + default_cf.max_compaction_bytes = default_cf.target_file_size_base * 1100; + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + default_cf.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + one.compaction_style = kCompactionStyleUniversal; + + one.num_levels = 1; + // trigger compaction if there are >= 4 files + one.level0_file_num_compaction_trigger = 4; + one.write_buffer_size = 120000; + + Reopen({default_cf, one}); + // make sure all background compaction jobs can be scheduled + auto stop_token = + dbfull()->TEST_write_controler().GetCompactionPressureToken(); + + // SETUP column family "one" -- universal style + for (int i = 0; i < one.level0_file_num_compaction_trigger - 2; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(i + 1), 1); + } + bool cf_1_1 = true; + bool cf_1_2 = true; + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"ColumnFamilyTest::ManualManual:4", "ColumnFamilyTest::ManualManual:2"}, + {"ColumnFamilyTest::ManualManual:4", "ColumnFamilyTest::ManualManual:5"}, + {"ColumnFamilyTest::ManualManual:1", "ColumnFamilyTest::ManualManual:2"}, + {"ColumnFamilyTest::ManualManual:1", + "ColumnFamilyTest::ManualManual:3"}}); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) { + if (cf_1_1) { + TEST_SYNC_POINT("ColumnFamilyTest::ManualManual:4"); + cf_1_1 = false; + TEST_SYNC_POINT("ColumnFamilyTest::ManualManual:3"); + } else if (cf_1_2) { + TEST_SYNC_POINT("ColumnFamilyTest::ManualManual:2"); + cf_1_2 = false; + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + rocksdb::port::Thread threads([&] { + CompactRangeOptions compact_options; + compact_options.exclusive_manual_compaction = true; + ASSERT_OK( + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr)); + }); + + TEST_SYNC_POINT("ColumnFamilyTest::ManualManual:5"); + + WaitForFlush(1); + + // Add more L0 files and force another manual compaction + for (int i = 0; i < one.level0_file_num_compaction_trigger - 2; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(one.level0_file_num_compaction_trigger + i), + 1); + } + + rocksdb::port::Thread threads1([&] { + CompactRangeOptions compact_options; + compact_options.exclusive_manual_compaction = false; + ASSERT_OK( + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr)); + }); + + TEST_SYNC_POINT("ColumnFamilyTest::ManualManual:1"); + + threads.join(); + threads1.join(); + WaitForCompaction(); + // VERIFY compaction "one" + ASSERT_LE(NumTableFilesAtLevel(0, 1), 2); + + // Compare against saved keys + std::set::iterator key_iter = keys_.begin(); + while (key_iter != keys_.end()) { + ASSERT_NE("NOT_FOUND", Get(1, *key_iter)); + key_iter++; + } +} + +TEST_F(ColumnFamilyTest, SameCFManualAutomaticCompactions) { + Open(); + CreateColumnFamilies({"one"}); + ColumnFamilyOptions default_cf, one; + db_options_.max_open_files = 20; // only 10 files in file cache + db_options_.max_background_compactions = 3; + + default_cf.compaction_style = kCompactionStyleLevel; + default_cf.num_levels = 3; + default_cf.write_buffer_size = 64 << 10; // 64KB + default_cf.target_file_size_base = 30 << 10; + default_cf.max_compaction_bytes = default_cf.target_file_size_base * 1100; + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + default_cf.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + one.compaction_style = kCompactionStyleUniversal; + + one.num_levels = 1; + // trigger compaction if there are >= 4 files + one.level0_file_num_compaction_trigger = 4; + one.write_buffer_size = 120000; + + Reopen({default_cf, one}); + // make sure all background compaction jobs can be scheduled + auto stop_token = + dbfull()->TEST_write_controler().GetCompactionPressureToken(); + + // SETUP column family "one" -- universal style + for (int i = 0; i < one.level0_file_num_compaction_trigger - 2; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(i + 1), 1); + } + bool cf_1_1 = true; + bool cf_1_2 = true; + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"ColumnFamilyTest::ManualAuto:4", "ColumnFamilyTest::ManualAuto:2"}, + {"ColumnFamilyTest::ManualAuto:4", "ColumnFamilyTest::ManualAuto:5"}, + {"ColumnFamilyTest::ManualAuto:1", "ColumnFamilyTest::ManualAuto:2"}, + {"ColumnFamilyTest::ManualAuto:1", "ColumnFamilyTest::ManualAuto:3"}}); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) { + if (cf_1_1) { + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:4"); + cf_1_1 = false; + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:3"); + } else if (cf_1_2) { + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:2"); + cf_1_2 = false; + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + rocksdb::port::Thread threads([&] { + CompactRangeOptions compact_options; + compact_options.exclusive_manual_compaction = false; + ASSERT_OK( + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr)); + }); + + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:5"); + + WaitForFlush(1); + + // Add more L0 files and force automatic compaction + for (int i = 0; i < one.level0_file_num_compaction_trigger; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(one.level0_file_num_compaction_trigger + i), + 1); + } + + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:1"); + + threads.join(); + WaitForCompaction(); + // VERIFY compaction "one" + ASSERT_LE(NumTableFilesAtLevel(0, 1), 2); + + // Compare against saved keys + std::set::iterator key_iter = keys_.begin(); + while (key_iter != keys_.end()) { + ASSERT_NE("NOT_FOUND", Get(1, *key_iter)); + key_iter++; + } +} + +TEST_F(ColumnFamilyTest, SameCFManualAutomaticCompactionsLevel) { + Open(); + CreateColumnFamilies({"one"}); + ColumnFamilyOptions default_cf, one; + db_options_.max_open_files = 20; // only 10 files in file cache + db_options_.max_background_compactions = 3; + + default_cf.compaction_style = kCompactionStyleLevel; + default_cf.num_levels = 3; + default_cf.write_buffer_size = 64 << 10; // 64KB + default_cf.target_file_size_base = 30 << 10; + default_cf.max_compaction_bytes = default_cf.target_file_size_base * 1100; + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + default_cf.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + one.compaction_style = kCompactionStyleLevel; + + one.num_levels = 1; + // trigger compaction if there are >= 4 files + one.level0_file_num_compaction_trigger = 3; + one.write_buffer_size = 120000; + + Reopen({default_cf, one}); + // make sure all background compaction jobs can be scheduled + auto stop_token = + dbfull()->TEST_write_controler().GetCompactionPressureToken(); + + // SETUP column family "one" -- level style + for (int i = 0; i < one.level0_file_num_compaction_trigger - 2; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(i + 1), 1); + } + bool cf_1_1 = true; + bool cf_1_2 = true; + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"ColumnFamilyTest::ManualAuto:4", "ColumnFamilyTest::ManualAuto:2"}, + {"ColumnFamilyTest::ManualAuto:4", "ColumnFamilyTest::ManualAuto:5"}, + {"ColumnFamilyTest::ManualAuto:3", "ColumnFamilyTest::ManualAuto:2"}, + {"LevelCompactionPicker::PickCompactionBySize:0", + "ColumnFamilyTest::ManualAuto:3"}, + {"ColumnFamilyTest::ManualAuto:1", "ColumnFamilyTest::ManualAuto:3"}}); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) { + if (cf_1_1) { + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:4"); + cf_1_1 = false; + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:3"); + } else if (cf_1_2) { + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:2"); + cf_1_2 = false; + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + rocksdb::port::Thread threads([&] { + CompactRangeOptions compact_options; + compact_options.exclusive_manual_compaction = false; + ASSERT_OK( + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr)); + }); + + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:5"); + + // Add more L0 files and force automatic compaction + for (int i = 0; i < one.level0_file_num_compaction_trigger; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(one.level0_file_num_compaction_trigger + i), + 1); + } + + TEST_SYNC_POINT("ColumnFamilyTest::ManualAuto:1"); + + threads.join(); + WaitForCompaction(); + // VERIFY compaction "one" + AssertFilesPerLevel("0,1", 1); + + // Compare against saved keys + std::set::iterator key_iter = keys_.begin(); + while (key_iter != keys_.end()) { + ASSERT_NE("NOT_FOUND", Get(1, *key_iter)); + key_iter++; + } +} + +// This test checks for automatic getting a conflict if there is a +// manual which has not yet been scheduled. +// The manual compaction waits in NotScheduled +// We generate more files and then trigger an automatic compaction +// This will wait because there is an unscheduled manual compaction. +// Once the conflict is hit, the manual compaction starts and ends +// Then another automatic will start and end. +TEST_F(ColumnFamilyTest, SameCFManualAutomaticConflict) { + Open(); + CreateColumnFamilies({"one"}); + ColumnFamilyOptions default_cf, one; + db_options_.max_open_files = 20; // only 10 files in file cache + db_options_.max_background_compactions = 3; + + default_cf.compaction_style = kCompactionStyleLevel; + default_cf.num_levels = 3; + default_cf.write_buffer_size = 64 << 10; // 64KB + default_cf.target_file_size_base = 30 << 10; + default_cf.max_compaction_bytes = default_cf.target_file_size_base * 1100; + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + default_cf.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + one.compaction_style = kCompactionStyleUniversal; + + one.num_levels = 1; + // trigger compaction if there are >= 4 files + one.level0_file_num_compaction_trigger = 4; + one.write_buffer_size = 120000; + + Reopen({default_cf, one}); + // make sure all background compaction jobs can be scheduled + auto stop_token = + dbfull()->TEST_write_controler().GetCompactionPressureToken(); + + // SETUP column family "one" -- universal style + for (int i = 0; i < one.level0_file_num_compaction_trigger - 2; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(i + 1), 1); + } + bool cf_1_1 = true; + bool cf_1_2 = true; + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::BackgroundCompaction()::Conflict", + "ColumnFamilyTest::ManualAutoCon:7"}, + {"ColumnFamilyTest::ManualAutoCon:9", + "ColumnFamilyTest::ManualAutoCon:8"}, + {"ColumnFamilyTest::ManualAutoCon:2", + "ColumnFamilyTest::ManualAutoCon:6"}, + {"ColumnFamilyTest::ManualAutoCon:4", + "ColumnFamilyTest::ManualAutoCon:5"}, + {"ColumnFamilyTest::ManualAutoCon:1", + "ColumnFamilyTest::ManualAutoCon:2"}, + {"ColumnFamilyTest::ManualAutoCon:1", + "ColumnFamilyTest::ManualAutoCon:3"}}); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) { + if (cf_1_1) { + TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:4"); + cf_1_1 = false; + TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:3"); + } else if (cf_1_2) { + cf_1_2 = false; + TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:2"); + } + }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::RunManualCompaction:NotScheduled", [&](void* arg) { + InstrumentedMutex* mutex = static_cast(arg); + mutex->Unlock(); + TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:9"); + TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:7"); + mutex->Lock(); + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + rocksdb::port::Thread threads([&] { + CompactRangeOptions compact_options; + compact_options.exclusive_manual_compaction = false; + ASSERT_OK( + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr)); + TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:6"); + }); + + TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:8"); + WaitForFlush(1); + + // Add more L0 files and force automatic compaction + for (int i = 0; i < one.level0_file_num_compaction_trigger; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(one.level0_file_num_compaction_trigger + i), + 1); + } + + TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:5"); + // Add more L0 files and force automatic compaction + for (int i = 0; i < one.level0_file_num_compaction_trigger; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + } + TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:1"); + + threads.join(); + WaitForCompaction(); + // VERIFY compaction "one" + ASSERT_LE(NumTableFilesAtLevel(0, 1), 3); + + // Compare against saved keys + std::set::iterator key_iter = keys_.begin(); + while (key_iter != keys_.end()) { + ASSERT_NE("NOT_FOUND", Get(1, *key_iter)); + key_iter++; + } +} + +// In this test, we generate enough files to trigger automatic compactions. +// The automatic compaction waits in NonTrivial:AfterRun +// We generate more files and then trigger an automatic compaction +// This will wait because the automatic compaction has files it needs. +// Once the conflict is hit, the automatic compaction starts and ends +// Then the manual will run and end. +TEST_F(ColumnFamilyTest, SameCFAutomaticManualCompactions) { + Open(); + CreateColumnFamilies({"one"}); + ColumnFamilyOptions default_cf, one; + db_options_.max_open_files = 20; // only 10 files in file cache + db_options_.max_background_compactions = 3; + + default_cf.compaction_style = kCompactionStyleLevel; + default_cf.num_levels = 3; + default_cf.write_buffer_size = 64 << 10; // 64KB + default_cf.target_file_size_base = 30 << 10; + default_cf.max_compaction_bytes = default_cf.target_file_size_base * 1100; + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + default_cf.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + one.compaction_style = kCompactionStyleUniversal; + + one.num_levels = 1; + // trigger compaction if there are >= 4 files + one.level0_file_num_compaction_trigger = 4; + one.write_buffer_size = 120000; + + Reopen({default_cf, one}); + // make sure all background compaction jobs can be scheduled + auto stop_token = + dbfull()->TEST_write_controler().GetCompactionPressureToken(); + + bool cf_1_1 = true; + bool cf_1_2 = true; + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"ColumnFamilyTest::AutoManual:4", "ColumnFamilyTest::AutoManual:2"}, + {"ColumnFamilyTest::AutoManual:4", "ColumnFamilyTest::AutoManual:5"}, + {"CompactionPicker::CompactRange:Conflict", + "ColumnFamilyTest::AutoManual:3"}}); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) { + if (cf_1_1) { + TEST_SYNC_POINT("ColumnFamilyTest::AutoManual:4"); + cf_1_1 = false; + TEST_SYNC_POINT("ColumnFamilyTest::AutoManual:3"); + } else if (cf_1_2) { + TEST_SYNC_POINT("ColumnFamilyTest::AutoManual:2"); + cf_1_2 = false; + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // SETUP column family "one" -- universal style + for (int i = 0; i < one.level0_file_num_compaction_trigger; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + AssertFilesPerLevel(ToString(i + 1), 1); + } + + TEST_SYNC_POINT("ColumnFamilyTest::AutoManual:5"); + + // Add another L0 file and force automatic compaction + for (int i = 0; i < one.level0_file_num_compaction_trigger - 2; ++i) { + PutRandomData(1, 10, 12000, true); + PutRandomData(1, 1, 10, true); + WaitForFlush(1); + } + + CompactRangeOptions compact_options; + compact_options.exclusive_manual_compaction = false; + ASSERT_OK(db_->CompactRange(compact_options, handles_[1], nullptr, nullptr)); + + TEST_SYNC_POINT("ColumnFamilyTest::AutoManual:1"); + + WaitForCompaction(); + // VERIFY compaction "one" + AssertFilesPerLevel("1", 1); + // Compare against saved keys + std::set::iterator key_iter = keys_.begin(); + while (key_iter != keys_.end()) { + ASSERT_NE("NOT_FOUND", Get(1, *key_iter)); + key_iter++; + } +} +#endif // !ROCKSDB_LITE + +#ifndef ROCKSDB_LITE // Tailing interator not supported +namespace { +std::string IterStatus(Iterator* iter) { + std::string result; + if (iter->Valid()) { + result = iter->key().ToString() + "->" + iter->value().ToString(); + } else { + result = "(invalid)"; + } + return result; +} +} // anonymous namespace + +TEST_F(ColumnFamilyTest, NewIteratorsTest) { + // iter == 0 -- no tailing + // iter == 2 -- tailing + for (int iter = 0; iter < 2; ++iter) { + Open(); + CreateColumnFamiliesAndReopen({"one", "two"}); + ASSERT_OK(Put(0, "a", "b")); + ASSERT_OK(Put(1, "b", "a")); + ASSERT_OK(Put(2, "c", "m")); + ASSERT_OK(Put(2, "v", "t")); + std::vector iterators; + ReadOptions options; + options.tailing = (iter == 1); + ASSERT_OK(db_->NewIterators(options, handles_, &iterators)); + + for (auto it : iterators) { + it->SeekToFirst(); + } + ASSERT_EQ(IterStatus(iterators[0]), "a->b"); + ASSERT_EQ(IterStatus(iterators[1]), "b->a"); + ASSERT_EQ(IterStatus(iterators[2]), "c->m"); + + ASSERT_OK(Put(1, "x", "x")); + + for (auto it : iterators) { + it->Next(); + } + + ASSERT_EQ(IterStatus(iterators[0]), "(invalid)"); + if (iter == 0) { + // no tailing + ASSERT_EQ(IterStatus(iterators[1]), "(invalid)"); + } else { + // tailing + ASSERT_EQ(IterStatus(iterators[1]), "x->x"); + } + ASSERT_EQ(IterStatus(iterators[2]), "v->t"); + + for (auto it : iterators) { + delete it; + } + Destroy(); + } +} +#endif // !ROCKSDB_LITE + +#ifndef ROCKSDB_LITE // ReadOnlyDB is not supported +TEST_F(ColumnFamilyTest, ReadOnlyDBTest) { + Open(); + CreateColumnFamiliesAndReopen({"one", "two", "three", "four"}); + ASSERT_OK(Put(0, "a", "b")); + ASSERT_OK(Put(1, "foo", "bla")); + ASSERT_OK(Put(2, "foo", "blabla")); + ASSERT_OK(Put(3, "foo", "blablabla")); + ASSERT_OK(Put(4, "foo", "blablablabla")); + + DropColumnFamilies({2}); + Close(); + // open only a subset of column families + AssertOpenReadOnly({"default", "one", "four"}); + ASSERT_EQ("NOT_FOUND", Get(0, "foo")); + ASSERT_EQ("bla", Get(1, "foo")); + ASSERT_EQ("blablablabla", Get(2, "foo")); + + + // test newiterators + { + std::vector iterators; + ASSERT_OK(db_->NewIterators(ReadOptions(), handles_, &iterators)); + for (auto it : iterators) { + it->SeekToFirst(); + } + ASSERT_EQ(IterStatus(iterators[0]), "a->b"); + ASSERT_EQ(IterStatus(iterators[1]), "foo->bla"); + ASSERT_EQ(IterStatus(iterators[2]), "foo->blablablabla"); + for (auto it : iterators) { + it->Next(); + } + ASSERT_EQ(IterStatus(iterators[0]), "(invalid)"); + ASSERT_EQ(IterStatus(iterators[1]), "(invalid)"); + ASSERT_EQ(IterStatus(iterators[2]), "(invalid)"); + + for (auto it : iterators) { + delete it; + } + } + + Close(); + // can't open dropped column family + Status s = OpenReadOnly({"default", "one", "two"}); + ASSERT_TRUE(!s.ok()); // Can't open without specifying default column family s = OpenReadOnly({"one", "four"}); ASSERT_TRUE(!s.ok()); } +#endif // !ROCKSDB_LITE -TEST(ColumnFamilyTest, DontRollEmptyLogs) { +#ifndef ROCKSDB_LITE // WaitForFlush() is not supported in lite +TEST_F(ColumnFamilyTest, DontRollEmptyLogs) { Open(); CreateColumnFamiliesAndReopen({"one", "two", "three", "four"}); for (size_t i = 0; i < handles_.size(); ++i) { - PutRandomData(i, 10, 100); + PutRandomData(static_cast(i), 10, 100); } int num_writable_file_start = env_->GetNumberOfNewWritableFileCalls(); // this will trigger the flushes - for (size_t i = 0; i <= 4; ++i) { + for (int i = 0; i <= 4; ++i) { ASSERT_OK(Flush(i)); } for (int i = 0; i < 4; ++i) { - dbfull()->TEST_WaitForFlushMemTable(handles_[i]); + WaitForFlush(i); } int total_new_writable_files = env_->GetNumberOfNewWritableFileCalls() - num_writable_file_start; ASSERT_EQ(static_cast(total_new_writable_files), handles_.size() + 1); Close(); } +#endif // !ROCKSDB_LITE -TEST(ColumnFamilyTest, FlushStaleColumnFamilies) { +#ifndef ROCKSDB_LITE // WaitForCompaction() is not supported in lite +TEST_F(ColumnFamilyTest, FlushStaleColumnFamilies) { Open(); CreateColumnFamilies({"one", "two"}); ColumnFamilyOptions default_cf, one, two; default_cf.write_buffer_size = 100000; // small write buffer size + default_cf.arena_block_size = 4096; default_cf.disable_auto_compactions = true; one.disable_auto_compactions = true; two.disable_auto_compactions = true; @@ -949,7 +2231,8 @@ TEST(ColumnFamilyTest, FlushStaleColumnFamilies) { for (int i = 0; i < 2; ++i) { PutRandomData(0, 100, 1000); // flush WaitForFlush(0); - ASSERT_EQ(i + 1, CountLiveFiles()); + + AssertCountLiveFiles(i + 1); } // third flush. now, CF [two] should be detected as stale and flushed // column family 1 should not be flushed since it's empty @@ -958,11 +2241,15 @@ TEST(ColumnFamilyTest, FlushStaleColumnFamilies) { WaitForFlush(2); // 3 files for default column families, 1 file for column family [two], zero // files for column family [one], because it's empty - ASSERT_EQ(4, CountLiveFiles()); + AssertCountLiveFiles(4); + + Flush(0); + ASSERT_EQ(0, dbfull()->TEST_total_log_size()); Close(); } +#endif // !ROCKSDB_LITE -TEST(ColumnFamilyTest, CreateMissingColumnFamilies) { +TEST_F(ColumnFamilyTest, CreateMissingColumnFamilies) { Status s = TryOpen({"one", "two"}); ASSERT_TRUE(!s.ok()); db_options_.create_missing_column_families = true; @@ -971,8 +2258,987 @@ TEST(ColumnFamilyTest, CreateMissingColumnFamilies) { Close(); } +TEST_F(ColumnFamilyTest, SanitizeOptions) { + DBOptions db_options; + for (int s = kCompactionStyleLevel; s <= kCompactionStyleUniversal; ++s) { + for (int l = 0; l <= 2; l++) { + for (int i = 1; i <= 3; i++) { + for (int j = 1; j <= 3; j++) { + for (int k = 1; k <= 3; k++) { + ColumnFamilyOptions original; + original.compaction_style = static_cast(s); + original.num_levels = l; + original.level0_stop_writes_trigger = i; + original.level0_slowdown_writes_trigger = j; + original.level0_file_num_compaction_trigger = k; + original.write_buffer_size = + l * 4 * 1024 * 1024 + i * 1024 * 1024 + j * 1024 + k; + + ColumnFamilyOptions result = + SanitizeOptions(ImmutableDBOptions(db_options), original); + ASSERT_TRUE(result.level0_stop_writes_trigger >= + result.level0_slowdown_writes_trigger); + ASSERT_TRUE(result.level0_slowdown_writes_trigger >= + result.level0_file_num_compaction_trigger); + ASSERT_TRUE(result.level0_file_num_compaction_trigger == + original.level0_file_num_compaction_trigger); + if (s == kCompactionStyleLevel) { + ASSERT_GE(result.num_levels, 2); + } else { + ASSERT_GE(result.num_levels, 1); + if (original.num_levels >= 1) { + ASSERT_EQ(result.num_levels, original.num_levels); + } + } + + // Make sure Sanitize options sets arena_block_size to 1/8 of + // the write_buffer_size, rounded up to a multiple of 4k. + size_t expected_arena_block_size = + l * 4 * 1024 * 1024 / 8 + i * 1024 * 1024 / 8; + if (j + k != 0) { + // not a multiple of 4k, round up 4k + expected_arena_block_size += 4 * 1024; + } + ASSERT_EQ(expected_arena_block_size, result.arena_block_size); + } + } + } + } + } +} + +TEST_F(ColumnFamilyTest, ReadDroppedColumnFamily) { + // iter 0 -- drop CF, don't reopen + // iter 1 -- delete CF, reopen + for (int iter = 0; iter < 2; ++iter) { + db_options_.create_missing_column_families = true; + db_options_.max_open_files = 20; + // delete obsolete files always + db_options_.delete_obsolete_files_period_micros = 0; + Open({"default", "one", "two"}); + ColumnFamilyOptions options; + options.level0_file_num_compaction_trigger = 100; + options.level0_slowdown_writes_trigger = 200; + options.level0_stop_writes_trigger = 200; + options.write_buffer_size = 100000; // small write buffer size + Reopen({options, options, options}); + + // 1MB should create ~10 files for each CF + int kKeysNum = 10000; + PutRandomData(0, kKeysNum, 100); + PutRandomData(1, kKeysNum, 100); + PutRandomData(2, kKeysNum, 100); + + { + std::unique_ptr iterator( + db_->NewIterator(ReadOptions(), handles_[2])); + iterator->SeekToFirst(); + + if (iter == 0) { + // Drop CF two + ASSERT_OK(db_->DropColumnFamily(handles_[2])); + } else { + // delete CF two + db_->DestroyColumnFamilyHandle(handles_[2]); + handles_[2] = nullptr; + } + // Make sure iterator created can still be used. + int count = 0; + for (; iterator->Valid(); iterator->Next()) { + ASSERT_OK(iterator->status()); + ++count; + } + ASSERT_OK(iterator->status()); + ASSERT_EQ(count, kKeysNum); + } + + // Add bunch more data to other CFs + PutRandomData(0, kKeysNum, 100); + PutRandomData(1, kKeysNum, 100); + + if (iter == 1) { + Reopen(); + } + + // Since we didn't delete CF handle, RocksDB's contract guarantees that + // we're still able to read dropped CF + for (int i = 0; i < 3; ++i) { + std::unique_ptr iterator( + db_->NewIterator(ReadOptions(), handles_[i])); + int count = 0; + for (iterator->SeekToFirst(); iterator->Valid(); iterator->Next()) { + ASSERT_OK(iterator->status()); + ++count; + } + ASSERT_OK(iterator->status()); + ASSERT_EQ(count, kKeysNum * ((i == 2) ? 1 : 2)); + } + + Close(); + Destroy(); + } +} + +TEST_F(ColumnFamilyTest, FlushAndDropRaceCondition) { + db_options_.create_missing_column_families = true; + Open({"default", "one"}); + ColumnFamilyOptions options; + options.level0_file_num_compaction_trigger = 100; + options.level0_slowdown_writes_trigger = 200; + options.level0_stop_writes_trigger = 200; + options.max_write_buffer_number = 20; + options.write_buffer_size = 100000; // small write buffer size + Reopen({options, options}); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"VersionSet::LogAndApply::ColumnFamilyDrop:0", + "FlushJob::WriteLevel0Table"}, + {"VersionSet::LogAndApply::ColumnFamilyDrop:1", + "FlushJob::InstallResults"}, + {"FlushJob::InstallResults", + "VersionSet::LogAndApply::ColumnFamilyDrop:2"}}); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + test::SleepingBackgroundTask sleeping_task; + + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task, + Env::Priority::HIGH); + + // 1MB should create ~10 files for each CF + int kKeysNum = 10000; + PutRandomData(1, kKeysNum, 100); + + std::vector threads; + threads.emplace_back([&] { ASSERT_OK(db_->DropColumnFamily(handles_[1])); }); + + sleeping_task.WakeUp(); + sleeping_task.WaitUntilDone(); + sleeping_task.Reset(); + // now we sleep again. this is just so we're certain that flush job finished + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task, + Env::Priority::HIGH); + sleeping_task.WakeUp(); + sleeping_task.WaitUntilDone(); + + { + // Since we didn't delete CF handle, RocksDB's contract guarantees that + // we're still able to read dropped CF + std::unique_ptr iterator( + db_->NewIterator(ReadOptions(), handles_[1])); + int count = 0; + for (iterator->SeekToFirst(); iterator->Valid(); iterator->Next()) { + ASSERT_OK(iterator->status()); + ++count; + } + ASSERT_OK(iterator->status()); + ASSERT_EQ(count, kKeysNum); + } + for (auto& t : threads) { + t.join(); + } + + Close(); + Destroy(); +} + +#ifndef ROCKSDB_LITE +// skipped as persisting options is not supported in ROCKSDB_LITE +namespace { +std::atomic test_stage(0); +const int kMainThreadStartPersistingOptionsFile = 1; +const int kChildThreadFinishDroppingColumnFamily = 2; +const int kChildThreadWaitingMainThreadPersistOptions = 3; +void DropSingleColumnFamily(ColumnFamilyTest* cf_test, int cf_id, + std::vector* comparators) { + while (test_stage < kMainThreadStartPersistingOptionsFile) { + Env::Default()->SleepForMicroseconds(100); + } + cf_test->DropColumnFamilies({cf_id}); + if ((*comparators)[cf_id]) { + delete (*comparators)[cf_id]; + (*comparators)[cf_id] = nullptr; + } + test_stage = kChildThreadFinishDroppingColumnFamily; +} +} // namespace + +TEST_F(ColumnFamilyTest, CreateAndDropRace) { + const int kCfCount = 5; + std::vector cf_opts; + std::vector comparators; + for (int i = 0; i < kCfCount; ++i) { + cf_opts.emplace_back(); + comparators.push_back(new test::SimpleSuffixReverseComparator()); + cf_opts.back().comparator = comparators.back(); + } + db_options_.create_if_missing = true; + db_options_.create_missing_column_families = true; + + auto main_thread_id = std::this_thread::get_id(); + + rocksdb::SyncPoint::GetInstance()->SetCallBack("PersistRocksDBOptions:start", + [&](void* arg) { + auto current_thread_id = std::this_thread::get_id(); + // If it's the main thread hitting this sync-point, then it + // will be blocked until some other thread update the test_stage. + if (main_thread_id == current_thread_id) { + test_stage = kMainThreadStartPersistingOptionsFile; + while (test_stage < kChildThreadFinishDroppingColumnFamily) { + Env::Default()->SleepForMicroseconds(100); + } + } + }); + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "WriteThread::EnterUnbatched:Wait", [&](void* arg) { + // This means a thread doing DropColumnFamily() is waiting for + // other thread to finish persisting options. + // In such case, we update the test_stage to unblock the main thread. + test_stage = kChildThreadWaitingMainThreadPersistOptions; + + // Note that based on the test setting, this must not be the + // main thread. + ASSERT_NE(main_thread_id, std::this_thread::get_id()); + }); + + // Create a database with four column families + Open({"default", "one", "two", "three"}, + {cf_opts[0], cf_opts[1], cf_opts[2], cf_opts[3]}); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Start a thread that will drop the first column family + // and its comparator + rocksdb::port::Thread drop_cf_thread(DropSingleColumnFamily, this, 1, &comparators); + + DropColumnFamilies({2}); + + drop_cf_thread.join(); + Close(); + Destroy(); + for (auto* comparator : comparators) { + if (comparator) { + delete comparator; + } + } +} +#endif // !ROCKSDB_LITE + +TEST_F(ColumnFamilyTest, WriteStallSingleColumnFamily) { + const uint64_t kBaseRate = 800000u; + db_options_.delayed_write_rate = kBaseRate; + db_options_.max_background_compactions = 6; + + Open({"default"}); + ColumnFamilyData* cfd = + static_cast(db_->DefaultColumnFamily())->cfd(); + + VersionStorageInfo* vstorage = cfd->current()->storage_info(); + + MutableCFOptions mutable_cf_options(column_family_options_); + + mutable_cf_options.level0_slowdown_writes_trigger = 20; + mutable_cf_options.level0_stop_writes_trigger = 10000; + mutable_cf_options.soft_pending_compaction_bytes_limit = 200; + mutable_cf_options.hard_pending_compaction_bytes_limit = 2000; + mutable_cf_options.disable_auto_compactions = false; + + vstorage->TEST_set_estimated_compaction_needed_bytes(50); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(201); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate, GetDbDelayedWriteRate()); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(400); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25, GetDbDelayedWriteRate()); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(500); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25 / 1.25, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(450); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(205); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(202); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(201); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(198); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(399); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(599); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(2001); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(3001); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(390); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(100); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + + vstorage->set_l0_delay_trigger_count(100); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate, GetDbDelayedWriteRate()); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->set_l0_delay_trigger_count(101); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25, GetDbDelayedWriteRate()); + + vstorage->set_l0_delay_trigger_count(0); + vstorage->TEST_set_estimated_compaction_needed_bytes(300); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25 / 1.25, GetDbDelayedWriteRate()); + + vstorage->set_l0_delay_trigger_count(101); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25 / 1.25 / 1.25, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(200); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25 / 1.25, GetDbDelayedWriteRate()); + + vstorage->set_l0_delay_trigger_count(0); + vstorage->TEST_set_estimated_compaction_needed_bytes(0); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + + mutable_cf_options.disable_auto_compactions = true; + dbfull()->TEST_write_controler().set_delayed_write_rate(kBaseRate); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + + vstorage->set_l0_delay_trigger_count(50); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(0, GetDbDelayedWriteRate()); + ASSERT_EQ(kBaseRate, dbfull()->TEST_write_controler().delayed_write_rate()); + + vstorage->set_l0_delay_trigger_count(60); + vstorage->TEST_set_estimated_compaction_needed_bytes(300); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(0, GetDbDelayedWriteRate()); + ASSERT_EQ(kBaseRate, dbfull()->TEST_write_controler().delayed_write_rate()); + + mutable_cf_options.disable_auto_compactions = false; + vstorage->set_l0_delay_trigger_count(70); + vstorage->TEST_set_estimated_compaction_needed_bytes(500); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate, GetDbDelayedWriteRate()); + + vstorage->set_l0_delay_trigger_count(71); + vstorage->TEST_set_estimated_compaction_needed_bytes(501); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25, GetDbDelayedWriteRate()); +} + +TEST_F(ColumnFamilyTest, CompactionSpeedupSingleColumnFamily) { + db_options_.max_background_compactions = 6; + Open({"default"}); + ColumnFamilyData* cfd = + static_cast(db_->DefaultColumnFamily())->cfd(); + + VersionStorageInfo* vstorage = cfd->current()->storage_info(); + + MutableCFOptions mutable_cf_options(column_family_options_); + + // Speed up threshold = min(4 * 2, 4 + (36 - 4)/4) = 8 + mutable_cf_options.level0_file_num_compaction_trigger = 4; + mutable_cf_options.level0_slowdown_writes_trigger = 36; + mutable_cf_options.level0_stop_writes_trigger = 50; + // Speedup threshold = 200 / 4 = 50 + mutable_cf_options.soft_pending_compaction_bytes_limit = 200; + mutable_cf_options.hard_pending_compaction_bytes_limit = 2000; + + vstorage->TEST_set_estimated_compaction_needed_bytes(40); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(50); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(300); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(45); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->set_l0_delay_trigger_count(7); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->set_l0_delay_trigger_count(9); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->set_l0_delay_trigger_count(6); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + + // Speed up threshold = min(4 * 2, 4 + (12 - 4)/4) = 6 + mutable_cf_options.level0_file_num_compaction_trigger = 4; + mutable_cf_options.level0_slowdown_writes_trigger = 16; + mutable_cf_options.level0_stop_writes_trigger = 30; + + vstorage->set_l0_delay_trigger_count(5); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->set_l0_delay_trigger_count(7); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->set_l0_delay_trigger_count(3); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); +} + +TEST_F(ColumnFamilyTest, WriteStallTwoColumnFamilies) { + const uint64_t kBaseRate = 810000u; + db_options_.delayed_write_rate = kBaseRate; + Open(); + CreateColumnFamilies({"one"}); + ColumnFamilyData* cfd = + static_cast(db_->DefaultColumnFamily())->cfd(); + VersionStorageInfo* vstorage = cfd->current()->storage_info(); + + ColumnFamilyData* cfd1 = + static_cast(handles_[1])->cfd(); + VersionStorageInfo* vstorage1 = cfd1->current()->storage_info(); + + MutableCFOptions mutable_cf_options(column_family_options_); + mutable_cf_options.level0_slowdown_writes_trigger = 20; + mutable_cf_options.level0_stop_writes_trigger = 10000; + mutable_cf_options.soft_pending_compaction_bytes_limit = 200; + mutable_cf_options.hard_pending_compaction_bytes_limit = 2000; + + MutableCFOptions mutable_cf_options1 = mutable_cf_options; + mutable_cf_options1.soft_pending_compaction_bytes_limit = 500; + + vstorage->TEST_set_estimated_compaction_needed_bytes(50); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + + vstorage1->TEST_set_estimated_compaction_needed_bytes(201); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + + vstorage1->TEST_set_estimated_compaction_needed_bytes(600); + cfd1->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(70); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate, GetDbDelayedWriteRate()); + + vstorage1->TEST_set_estimated_compaction_needed_bytes(800); + cfd1->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(300); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25 / 1.25, GetDbDelayedWriteRate()); + + vstorage1->TEST_set_estimated_compaction_needed_bytes(700); + cfd1->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25, GetDbDelayedWriteRate()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(500); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25 / 1.25, GetDbDelayedWriteRate()); + + vstorage1->TEST_set_estimated_compaction_needed_bytes(600); + cfd1->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_TRUE(!IsDbWriteStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + ASSERT_EQ(kBaseRate / 1.25, GetDbDelayedWriteRate()); +} + +TEST_F(ColumnFamilyTest, CompactionSpeedupTwoColumnFamilies) { + db_options_.max_background_compactions = 6; + column_family_options_.soft_pending_compaction_bytes_limit = 200; + column_family_options_.hard_pending_compaction_bytes_limit = 2000; + Open(); + CreateColumnFamilies({"one"}); + ColumnFamilyData* cfd = + static_cast(db_->DefaultColumnFamily())->cfd(); + VersionStorageInfo* vstorage = cfd->current()->storage_info(); + + ColumnFamilyData* cfd1 = + static_cast(handles_[1])->cfd(); + VersionStorageInfo* vstorage1 = cfd1->current()->storage_info(); + + MutableCFOptions mutable_cf_options(column_family_options_); + // Speed up threshold = min(4 * 2, 4 + (36 - 4)/4) = 8 + mutable_cf_options.level0_file_num_compaction_trigger = 4; + mutable_cf_options.level0_slowdown_writes_trigger = 36; + mutable_cf_options.level0_stop_writes_trigger = 30; + // Speedup threshold = 200 / 4 = 50 + mutable_cf_options.soft_pending_compaction_bytes_limit = 200; + mutable_cf_options.hard_pending_compaction_bytes_limit = 2000; + + MutableCFOptions mutable_cf_options1 = mutable_cf_options; + mutable_cf_options1.level0_slowdown_writes_trigger = 16; + + vstorage->TEST_set_estimated_compaction_needed_bytes(40); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(60); + cfd1->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage1->TEST_set_estimated_compaction_needed_bytes(30); + cfd1->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage1->TEST_set_estimated_compaction_needed_bytes(70); + cfd1->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->TEST_set_estimated_compaction_needed_bytes(20); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage1->TEST_set_estimated_compaction_needed_bytes(3); + cfd1->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->set_l0_delay_trigger_count(9); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage1->set_l0_delay_trigger_count(2); + cfd1->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(6, dbfull()->TEST_BGCompactionsAllowed()); + + vstorage->set_l0_delay_trigger_count(0); + cfd->RecalculateWriteStallConditions(mutable_cf_options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); +} + +#ifndef ROCKSDB_LITE +TEST_F(ColumnFamilyTest, FlushCloseWALFiles) { + SpecialEnv env(Env::Default()); + db_options_.env = &env; + db_options_.max_background_flushes = 1; + column_family_options_.memtable_factory.reset(new SpecialSkipListFactory(2)); + Open(); + CreateColumnFamilies({"one"}); + ASSERT_OK(Put(1, "fodor", "mirko")); + ASSERT_OK(Put(0, "fodor", "mirko")); + ASSERT_OK(Put(1, "fodor", "mirko")); + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"DBImpl::BGWorkFlush:done", "FlushCloseWALFiles:0"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Block flush jobs from running + test::SleepingBackgroundTask sleeping_task; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task, + Env::Priority::HIGH); + + WriteOptions wo; + wo.sync = true; + ASSERT_OK(db_->Put(wo, handles_[1], "fodor", "mirko")); + + ASSERT_EQ(2, env.num_open_wal_file_.load()); + + sleeping_task.WakeUp(); + sleeping_task.WaitUntilDone(); + TEST_SYNC_POINT("FlushCloseWALFiles:0"); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + ASSERT_EQ(1, env.num_open_wal_file_.load()); + + Reopen(); + ASSERT_EQ("mirko", Get(0, "fodor")); + ASSERT_EQ("mirko", Get(1, "fodor")); + db_options_.env = env_; + Close(); +} +#endif // !ROCKSDB_LITE + +#ifndef ROCKSDB_LITE // WaitForFlush() is not supported +TEST_F(ColumnFamilyTest, IteratorCloseWALFile1) { + SpecialEnv env(Env::Default()); + db_options_.env = &env; + db_options_.max_background_flushes = 1; + column_family_options_.memtable_factory.reset(new SpecialSkipListFactory(2)); + Open(); + CreateColumnFamilies({"one"}); + ASSERT_OK(Put(1, "fodor", "mirko")); + // Create an iterator holding the current super version. + Iterator* it = db_->NewIterator(ReadOptions(), handles_[1]); + // A flush will make `it` hold the last reference of its super version. + Flush(1); + + ASSERT_OK(Put(1, "fodor", "mirko")); + ASSERT_OK(Put(0, "fodor", "mirko")); + ASSERT_OK(Put(1, "fodor", "mirko")); + + // Flush jobs will close previous WAL files after finishing. By + // block flush jobs from running, we trigger a condition where + // the iterator destructor should close the WAL files. + test::SleepingBackgroundTask sleeping_task; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task, + Env::Priority::HIGH); + + WriteOptions wo; + wo.sync = true; + ASSERT_OK(db_->Put(wo, handles_[1], "fodor", "mirko")); + + ASSERT_EQ(2, env.num_open_wal_file_.load()); + // Deleting the iterator will clear its super version, triggering + // closing all files + delete it; + ASSERT_EQ(1, env.num_open_wal_file_.load()); + + sleeping_task.WakeUp(); + sleeping_task.WaitUntilDone(); + WaitForFlush(1); + + Reopen(); + ASSERT_EQ("mirko", Get(0, "fodor")); + ASSERT_EQ("mirko", Get(1, "fodor")); + db_options_.env = env_; + Close(); +} + +TEST_F(ColumnFamilyTest, IteratorCloseWALFile2) { + SpecialEnv env(Env::Default()); + // Allow both of flush and purge job to schedule. + env.SetBackgroundThreads(2, Env::HIGH); + db_options_.env = &env; + db_options_.max_background_flushes = 1; + column_family_options_.memtable_factory.reset(new SpecialSkipListFactory(2)); + Open(); + CreateColumnFamilies({"one"}); + ASSERT_OK(Put(1, "fodor", "mirko")); + // Create an iterator holding the current super version. + ReadOptions ro; + ro.background_purge_on_iterator_cleanup = true; + Iterator* it = db_->NewIterator(ro, handles_[1]); + // A flush will make `it` hold the last reference of its super version. + Flush(1); + + ASSERT_OK(Put(1, "fodor", "mirko")); + ASSERT_OK(Put(0, "fodor", "mirko")); + ASSERT_OK(Put(1, "fodor", "mirko")); + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"ColumnFamilyTest::IteratorCloseWALFile2:0", + "DBImpl::BGWorkPurge:start"}, + {"ColumnFamilyTest::IteratorCloseWALFile2:2", + "DBImpl::BackgroundCallFlush:start"}, + {"DBImpl::BGWorkPurge:end", "ColumnFamilyTest::IteratorCloseWALFile2:1"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + WriteOptions wo; + wo.sync = true; + ASSERT_OK(db_->Put(wo, handles_[1], "fodor", "mirko")); + + ASSERT_EQ(2, env.num_open_wal_file_.load()); + // Deleting the iterator will clear its super version, triggering + // closing all files + delete it; + ASSERT_EQ(2, env.num_open_wal_file_.load()); + + TEST_SYNC_POINT("ColumnFamilyTest::IteratorCloseWALFile2:0"); + TEST_SYNC_POINT("ColumnFamilyTest::IteratorCloseWALFile2:1"); + ASSERT_EQ(1, env.num_open_wal_file_.load()); + TEST_SYNC_POINT("ColumnFamilyTest::IteratorCloseWALFile2:2"); + WaitForFlush(1); + ASSERT_EQ(1, env.num_open_wal_file_.load()); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + Reopen(); + ASSERT_EQ("mirko", Get(0, "fodor")); + ASSERT_EQ("mirko", Get(1, "fodor")); + db_options_.env = env_; + Close(); +} +#endif // !ROCKSDB_LITE + +#ifndef ROCKSDB_LITE // TEST functions are not supported in lite +TEST_F(ColumnFamilyTest, ForwardIteratorCloseWALFile) { + SpecialEnv env(Env::Default()); + // Allow both of flush and purge job to schedule. + env.SetBackgroundThreads(2, Env::HIGH); + db_options_.env = &env; + db_options_.max_background_flushes = 1; + column_family_options_.memtable_factory.reset(new SpecialSkipListFactory(3)); + column_family_options_.level0_file_num_compaction_trigger = 2; + Open(); + CreateColumnFamilies({"one"}); + ASSERT_OK(Put(1, "fodor", "mirko")); + ASSERT_OK(Put(1, "fodar2", "mirko")); + Flush(1); + + // Create an iterator holding the current super version, as well as + // the SST file just flushed. + ReadOptions ro; + ro.tailing = true; + ro.background_purge_on_iterator_cleanup = true; + Iterator* it = db_->NewIterator(ro, handles_[1]); + // A flush will make `it` hold the last reference of its super version. + + ASSERT_OK(Put(1, "fodor", "mirko")); + ASSERT_OK(Put(1, "fodar2", "mirko")); + Flush(1); + + WaitForCompaction(); + + ASSERT_OK(Put(1, "fodor", "mirko")); + ASSERT_OK(Put(1, "fodor", "mirko")); + ASSERT_OK(Put(0, "fodor", "mirko")); + ASSERT_OK(Put(1, "fodor", "mirko")); + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"ColumnFamilyTest::IteratorCloseWALFile2:0", + "DBImpl::BGWorkPurge:start"}, + {"ColumnFamilyTest::IteratorCloseWALFile2:2", + "DBImpl::BackgroundCallFlush:start"}, + {"DBImpl::BGWorkPurge:end", "ColumnFamilyTest::IteratorCloseWALFile2:1"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + WriteOptions wo; + wo.sync = true; + ASSERT_OK(db_->Put(wo, handles_[1], "fodor", "mirko")); + + env.delete_count_.store(0); + ASSERT_EQ(2, env.num_open_wal_file_.load()); + // Deleting the iterator will clear its super version, triggering + // closing all files + it->Seek(""); + ASSERT_EQ(2, env.num_open_wal_file_.load()); + ASSERT_EQ(0, env.delete_count_.load()); + + TEST_SYNC_POINT("ColumnFamilyTest::IteratorCloseWALFile2:0"); + TEST_SYNC_POINT("ColumnFamilyTest::IteratorCloseWALFile2:1"); + ASSERT_EQ(1, env.num_open_wal_file_.load()); + ASSERT_EQ(1, env.delete_count_.load()); + TEST_SYNC_POINT("ColumnFamilyTest::IteratorCloseWALFile2:2"); + WaitForFlush(1); + ASSERT_EQ(1, env.num_open_wal_file_.load()); + ASSERT_EQ(1, env.delete_count_.load()); + + delete it; + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + Reopen(); + ASSERT_EQ("mirko", Get(0, "fodor")); + ASSERT_EQ("mirko", Get(1, "fodor")); + db_options_.env = env_; + Close(); +} +#endif // !ROCKSDB_LITE + +// Disable on windows because SyncWAL requires env->IsSyncThreadSafe() +// to return true which is not so in unbuffered mode. +#ifndef OS_WIN +TEST_F(ColumnFamilyTest, LogSyncConflictFlush) { + Open(); + CreateColumnFamiliesAndReopen({"one", "two"}); + + Put(0, "", ""); + Put(1, "foo", "bar"); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::SyncWAL:BeforeMarkLogsSynced:1", + "ColumnFamilyTest::LogSyncConflictFlush:1"}, + {"ColumnFamilyTest::LogSyncConflictFlush:2", + "DBImpl::SyncWAL:BeforeMarkLogsSynced:2"}}); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + rocksdb::port::Thread thread([&] { db_->SyncWAL(); }); + + TEST_SYNC_POINT("ColumnFamilyTest::LogSyncConflictFlush:1"); + Flush(1); + Put(1, "foo", "bar"); + Flush(1); + + TEST_SYNC_POINT("ColumnFamilyTest::LogSyncConflictFlush:2"); + + thread.join(); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + Close(); +} +#endif + +// this test is placed here, because the infrastructure for Column Family +// test is being used to ensure a roll of wal files. +// Basic idea is to test that WAL truncation is being detected and not +// ignored +TEST_F(ColumnFamilyTest, DISABLED_LogTruncationTest) { + Open(); + CreateColumnFamiliesAndReopen({"one", "two"}); + + Build(0, 100); + + // Flush the 0th column family to force a roll of the wal log + Flush(0); + + // Add some more entries + Build(100, 100); + + std::vector filenames; + ASSERT_OK(env_->GetChildren(dbname_, &filenames)); + + // collect wal files + std::vector logfs; + for (size_t i = 0; i < filenames.size(); i++) { + uint64_t number; + FileType type; + if (!(ParseFileName(filenames[i], &number, &type))) continue; + + if (type != kLogFile) continue; + + logfs.push_back(filenames[i]); + } + + std::sort(logfs.begin(), logfs.end()); + ASSERT_GE(logfs.size(), 2); + + // Take the last but one file, and truncate it + std::string fpath = dbname_ + "/" + logfs[logfs.size() - 2]; + std::vector names_save = names_; + + uint64_t fsize; + ASSERT_OK(env_->GetFileSize(fpath, &fsize)); + ASSERT_GT(fsize, 0); + + Close(); + + std::string backup_logs = dbname_ + "/backup_logs"; + std::string t_fpath = backup_logs + "/" + logfs[logfs.size() - 2]; + + ASSERT_OK(env_->CreateDirIfMissing(backup_logs)); + // Not sure how easy it is to make this data driven. + // need to read back the WAL file and truncate last 10 + // entries + CopyFile(fpath, t_fpath, fsize - 9180); + + ASSERT_OK(env_->DeleteFile(fpath)); + ASSERT_OK(env_->RenameFile(t_fpath, fpath)); + + db_options_.wal_recovery_mode = WALRecoveryMode::kPointInTimeRecovery; + + OpenReadOnly(names_save); + + CheckMissed(); + + Close(); + + Open(names_save); + + CheckMissed(); + + Close(); + + // cleanup + env_->DeleteDir(backup_logs); +} } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/compact_files_test.cc b/db/compact_files_test.cc new file mode 100644 index 00000000000..5aad6114f5e --- /dev/null +++ b/db/compact_files_test.cc @@ -0,0 +1,328 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include +#include +#include +#include + +#include "db/db_impl.h" +#include "port/port.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "util/string_util.h" +#include "util/sync_point.h" +#include "util/testharness.h" + +namespace rocksdb { + +class CompactFilesTest : public testing::Test { + public: + CompactFilesTest() { + env_ = Env::Default(); + db_name_ = test::TmpDir(env_) + "/compact_files_test"; + } + + std::string db_name_; + Env* env_; +}; + +// A class which remembers the name of each flushed file. +class FlushedFileCollector : public EventListener { + public: + FlushedFileCollector() {} + ~FlushedFileCollector() {} + + virtual void OnFlushCompleted( + DB* db, const FlushJobInfo& info) override { + std::lock_guard lock(mutex_); + flushed_files_.push_back(info.file_path); + } + + std::vector GetFlushedFiles() { + std::lock_guard lock(mutex_); + std::vector result; + for (auto fname : flushed_files_) { + result.push_back(fname); + } + return result; + } + void ClearFlushedFiles() { + std::lock_guard lock(mutex_); + flushed_files_.clear(); + } + + private: + std::vector flushed_files_; + std::mutex mutex_; +}; + +TEST_F(CompactFilesTest, L0ConflictsFiles) { + Options options; + // to trigger compaction more easily + const int kWriteBufferSize = 10000; + const int kLevel0Trigger = 2; + options.create_if_missing = true; + options.compaction_style = kCompactionStyleLevel; + // Small slowdown and stop trigger for experimental purpose. + options.level0_slowdown_writes_trigger = 20; + options.level0_stop_writes_trigger = 20; + options.level0_stop_writes_trigger = 20; + options.write_buffer_size = kWriteBufferSize; + options.level0_file_num_compaction_trigger = kLevel0Trigger; + options.compression = kNoCompression; + + DB* db = nullptr; + DestroyDB(db_name_, options); + Status s = DB::Open(options, db_name_, &db); + assert(s.ok()); + assert(db); + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"CompactFilesImpl:0", "BackgroundCallCompaction:0"}, + {"BackgroundCallCompaction:1", "CompactFilesImpl:1"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // create couple files + // Background compaction starts and waits in BackgroundCallCompaction:0 + for (int i = 0; i < kLevel0Trigger * 4; ++i) { + db->Put(WriteOptions(), ToString(i), ""); + db->Put(WriteOptions(), ToString(100 - i), ""); + db->Flush(FlushOptions()); + } + + rocksdb::ColumnFamilyMetaData meta; + db->GetColumnFamilyMetaData(&meta); + std::string file1; + for (auto& file : meta.levels[0].files) { + ASSERT_EQ(0, meta.levels[0].level); + if (file1 == "") { + file1 = file.db_path + "/" + file.name; + } else { + std::string file2 = file.db_path + "/" + file.name; + // Another thread starts a compact files and creates an L0 compaction + // The background compaction then notices that there is an L0 compaction + // already in progress and doesn't do an L0 compaction + // Once the background compaction finishes, the compact files finishes + ASSERT_OK( + db->CompactFiles(rocksdb::CompactionOptions(), {file1, file2}, 0)); + break; + } + } + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + delete db; +} + +TEST_F(CompactFilesTest, ObsoleteFiles) { + Options options; + // to trigger compaction more easily + const int kWriteBufferSize = 65536; + options.create_if_missing = true; + // Disable RocksDB background compaction. + options.compaction_style = kCompactionStyleNone; + options.level0_slowdown_writes_trigger = (1 << 30); + options.level0_stop_writes_trigger = (1 << 30); + options.write_buffer_size = kWriteBufferSize; + options.max_write_buffer_number = 2; + options.compression = kNoCompression; + + // Add listener + FlushedFileCollector* collector = new FlushedFileCollector(); + options.listeners.emplace_back(collector); + + DB* db = nullptr; + DestroyDB(db_name_, options); + Status s = DB::Open(options, db_name_, &db); + assert(s.ok()); + assert(db); + + // create couple files + for (int i = 1000; i < 2000; ++i) { + db->Put(WriteOptions(), ToString(i), + std::string(kWriteBufferSize / 10, 'a' + (i % 26))); + } + + auto l0_files = collector->GetFlushedFiles(); + ASSERT_OK(db->CompactFiles(CompactionOptions(), l0_files, 1)); + reinterpret_cast(db)->TEST_WaitForCompact(); + + // verify all compaction input files are deleted + for (auto fname : l0_files) { + ASSERT_EQ(Status::NotFound(), env_->FileExists(fname)); + } + delete db; +} + +TEST_F(CompactFilesTest, NotCutOutputOnLevel0) { + Options options; + options.create_if_missing = true; + // Disable RocksDB background compaction. + options.compaction_style = kCompactionStyleNone; + options.level0_slowdown_writes_trigger = 1000; + options.level0_stop_writes_trigger = 1000; + options.write_buffer_size = 65536; + options.max_write_buffer_number = 2; + options.compression = kNoCompression; + options.max_compaction_bytes = 5000; + + // Add listener + FlushedFileCollector* collector = new FlushedFileCollector(); + options.listeners.emplace_back(collector); + + DB* db = nullptr; + DestroyDB(db_name_, options); + Status s = DB::Open(options, db_name_, &db); + assert(s.ok()); + assert(db); + + // create couple files + for (int i = 0; i < 500; ++i) { + db->Put(WriteOptions(), ToString(i), std::string(1000, 'a' + (i % 26))); + } + reinterpret_cast(db)->TEST_WaitForFlushMemTable(); + auto l0_files_1 = collector->GetFlushedFiles(); + collector->ClearFlushedFiles(); + for (int i = 0; i < 500; ++i) { + db->Put(WriteOptions(), ToString(i), std::string(1000, 'a' + (i % 26))); + } + reinterpret_cast(db)->TEST_WaitForFlushMemTable(); + auto l0_files_2 = collector->GetFlushedFiles(); + ASSERT_OK(db->CompactFiles(CompactionOptions(), l0_files_1, 0)); + ASSERT_OK(db->CompactFiles(CompactionOptions(), l0_files_2, 0)); + // no assertion failure + delete db; +} + +TEST_F(CompactFilesTest, CapturingPendingFiles) { + Options options; + options.create_if_missing = true; + // Disable RocksDB background compaction. + options.compaction_style = kCompactionStyleNone; + // Always do full scans for obsolete files (needed to reproduce the issue). + options.delete_obsolete_files_period_micros = 0; + + // Add listener. + FlushedFileCollector* collector = new FlushedFileCollector(); + options.listeners.emplace_back(collector); + + DB* db = nullptr; + DestroyDB(db_name_, options); + Status s = DB::Open(options, db_name_, &db); + assert(s.ok()); + assert(db); + + // Create 5 files. + for (int i = 0; i < 5; ++i) { + db->Put(WriteOptions(), "key" + ToString(i), "value"); + db->Flush(FlushOptions()); + } + + auto l0_files = collector->GetFlushedFiles(); + EXPECT_EQ(5, l0_files.size()); + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"CompactFilesImpl:2", "CompactFilesTest.CapturingPendingFiles:0"}, + {"CompactFilesTest.CapturingPendingFiles:1", "CompactFilesImpl:3"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Start compacting files. + rocksdb::port::Thread compaction_thread( + [&] { EXPECT_OK(db->CompactFiles(CompactionOptions(), l0_files, 1)); }); + + // In the meantime flush another file. + TEST_SYNC_POINT("CompactFilesTest.CapturingPendingFiles:0"); + db->Put(WriteOptions(), "key5", "value"); + db->Flush(FlushOptions()); + TEST_SYNC_POINT("CompactFilesTest.CapturingPendingFiles:1"); + + compaction_thread.join(); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + delete db; + + // Make sure we can reopen the DB. + s = DB::Open(options, db_name_, &db); + ASSERT_TRUE(s.ok()); + assert(db); + delete db; +} + +TEST_F(CompactFilesTest, CompactionFilterWithGetSv) { + class FilterWithGet : public CompactionFilter { + public: + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, + bool* value_changed) const override { + if (db_ == nullptr) { + return true; + } + std::string res; + db_->Get(ReadOptions(), "", &res); + return true; + } + + void SetDB(DB* db) { + db_ = db; + } + + virtual const char* Name() const override { return "FilterWithGet"; } + + private: + DB* db_; + }; + + + std::shared_ptr cf(new FilterWithGet()); + + Options options; + options.create_if_missing = true; + options.compaction_filter = cf.get(); + + DB* db = nullptr; + DestroyDB(db_name_, options); + Status s = DB::Open(options, db_name_, &db); + ASSERT_OK(s); + + cf->SetDB(db); + + // Write one L0 file + db->Put(WriteOptions(), "K1", "V1"); + db->Flush(FlushOptions()); + + // Compact all L0 files using CompactFiles + rocksdb::ColumnFamilyMetaData meta; + db->GetColumnFamilyMetaData(&meta); + for (auto& file : meta.levels[0].files) { + std::string fname = file.db_path + "/" + file.name; + ASSERT_OK( + db->CompactFiles(rocksdb::CompactionOptions(), {fname}, 0)); + } + + + delete db; +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, + "SKIPPED as DBImpl::CompactFiles is not supported in ROCKSDB_LITE\n"); + return 0; +} + +#endif // !ROCKSDB_LITE diff --git a/db/compacted_db_impl.cc b/db/compacted_db_impl.cc new file mode 100644 index 00000000000..d1007d972a1 --- /dev/null +++ b/db/compacted_db_impl.cc @@ -0,0 +1,166 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE +#include "db/compacted_db_impl.h" +#include "db/db_impl.h" +#include "db/version_set.h" +#include "table/get_context.h" + +namespace rocksdb { + +extern void MarkKeyMayExist(void* arg); +extern bool SaveValue(void* arg, const ParsedInternalKey& parsed_key, + const Slice& v, bool hit_and_return); + +CompactedDBImpl::CompactedDBImpl( + const DBOptions& options, const std::string& dbname) + : DBImpl(options, dbname) { +} + +CompactedDBImpl::~CompactedDBImpl() { +} + +size_t CompactedDBImpl::FindFile(const Slice& key) { + size_t left = 0; + size_t right = files_.num_files - 1; + while (left < right) { + size_t mid = (left + right) >> 1; + const FdWithKeyRange& f = files_.files[mid]; + if (user_comparator_->Compare(ExtractUserKey(f.largest_key), key) < 0) { + // Key at "mid.largest" is < "target". Therefore all + // files at or before "mid" are uninteresting. + left = mid + 1; + } else { + // Key at "mid.largest" is >= "target". Therefore all files + // after "mid" are uninteresting. + right = mid; + } + } + return right; +} + +Status CompactedDBImpl::Get(const ReadOptions& options, ColumnFamilyHandle*, + const Slice& key, PinnableSlice* value) { + GetContext get_context(user_comparator_, nullptr, nullptr, nullptr, + GetContext::kNotFound, key, value, nullptr, nullptr, + nullptr, nullptr); + LookupKey lkey(key, kMaxSequenceNumber); + files_.files[FindFile(key)].fd.table_reader->Get( + options, lkey.internal_key(), &get_context); + if (get_context.State() == GetContext::kFound) { + return Status::OK(); + } + return Status::NotFound(); +} + +std::vector CompactedDBImpl::MultiGet(const ReadOptions& options, + const std::vector&, + const std::vector& keys, std::vector* values) { + autovector reader_list; + for (const auto& key : keys) { + const FdWithKeyRange& f = files_.files[FindFile(key)]; + if (user_comparator_->Compare(key, ExtractUserKey(f.smallest_key)) < 0) { + reader_list.push_back(nullptr); + } else { + LookupKey lkey(key, kMaxSequenceNumber); + f.fd.table_reader->Prepare(lkey.internal_key()); + reader_list.push_back(f.fd.table_reader); + } + } + std::vector statuses(keys.size(), Status::NotFound()); + values->resize(keys.size()); + int idx = 0; + for (auto* r : reader_list) { + if (r != nullptr) { + PinnableSlice pinnable_val; + std::string& value = (*values)[idx]; + GetContext get_context(user_comparator_, nullptr, nullptr, nullptr, + GetContext::kNotFound, keys[idx], &pinnable_val, + nullptr, nullptr, nullptr, nullptr); + LookupKey lkey(keys[idx], kMaxSequenceNumber); + r->Get(options, lkey.internal_key(), &get_context); + value.assign(pinnable_val.data(), pinnable_val.size()); + if (get_context.State() == GetContext::kFound) { + statuses[idx] = Status::OK(); + } + } + ++idx; + } + return statuses; +} + +Status CompactedDBImpl::Init(const Options& options) { + mutex_.Lock(); + ColumnFamilyDescriptor cf(kDefaultColumnFamilyName, + ColumnFamilyOptions(options)); + Status s = Recover({cf}, true /* read only */, false, true); + if (s.ok()) { + cfd_ = reinterpret_cast( + DefaultColumnFamily())->cfd(); + delete cfd_->InstallSuperVersion(new SuperVersion(), &mutex_); + } + mutex_.Unlock(); + if (!s.ok()) { + return s; + } + NewThreadStatusCfInfo(cfd_); + version_ = cfd_->GetSuperVersion()->current; + user_comparator_ = cfd_->user_comparator(); + auto* vstorage = version_->storage_info(); + if (vstorage->num_non_empty_levels() == 0) { + return Status::NotSupported("no file exists"); + } + const LevelFilesBrief& l0 = vstorage->LevelFilesBrief(0); + // L0 should not have files + if (l0.num_files > 1) { + return Status::NotSupported("L0 contain more than 1 file"); + } + if (l0.num_files == 1) { + if (vstorage->num_non_empty_levels() > 1) { + return Status::NotSupported("Both L0 and other level contain files"); + } + files_ = l0; + return Status::OK(); + } + + for (int i = 1; i < vstorage->num_non_empty_levels() - 1; ++i) { + if (vstorage->LevelFilesBrief(i).num_files > 0) { + return Status::NotSupported("Other levels also contain files"); + } + } + + int level = vstorage->num_non_empty_levels() - 1; + if (vstorage->LevelFilesBrief(level).num_files > 0) { + files_ = vstorage->LevelFilesBrief(level); + return Status::OK(); + } + return Status::NotSupported("no file exists"); +} + +Status CompactedDBImpl::Open(const Options& options, + const std::string& dbname, DB** dbptr) { + *dbptr = nullptr; + + if (options.max_open_files != -1) { + return Status::InvalidArgument("require max_open_files = -1"); + } + if (options.merge_operator.get() != nullptr) { + return Status::InvalidArgument("merge operator is not supported"); + } + DBOptions db_options(options); + std::unique_ptr db(new CompactedDBImpl(db_options, dbname)); + Status s = db->Init(options); + if (s.ok()) { + ROCKS_LOG_INFO(db->immutable_db_options_.info_log, + "Opened the db as fully compacted mode"); + LogFlush(db->immutable_db_options_.info_log); + *dbptr = db.release(); + } + return s; +} + +} // namespace rocksdb +#endif // ROCKSDB_LITE diff --git a/db/compacted_db_impl.h b/db/compacted_db_impl.h new file mode 100644 index 00000000000..de32f21e681 --- /dev/null +++ b/db/compacted_db_impl.h @@ -0,0 +1,102 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#ifndef ROCKSDB_LITE +#include "db/db_impl.h" +#include +#include + +namespace rocksdb { + +class CompactedDBImpl : public DBImpl { + public: + CompactedDBImpl(const DBOptions& options, const std::string& dbname); + virtual ~CompactedDBImpl(); + + static Status Open(const Options& options, const std::string& dbname, + DB** dbptr); + + // Implementations of the DB interface + using DB::Get; + virtual Status Get(const ReadOptions& options, + ColumnFamilyHandle* column_family, const Slice& key, + PinnableSlice* value) override; + using DB::MultiGet; + virtual std::vector MultiGet( + const ReadOptions& options, + const std::vector&, + const std::vector& keys, std::vector* values) + override; + + using DBImpl::Put; + virtual Status Put(const WriteOptions& options, + ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) override { + return Status::NotSupported("Not supported in compacted db mode."); + } + using DBImpl::Merge; + virtual Status Merge(const WriteOptions& options, + ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) override { + return Status::NotSupported("Not supported in compacted db mode."); + } + using DBImpl::Delete; + virtual Status Delete(const WriteOptions& options, + ColumnFamilyHandle* column_family, + const Slice& key) override { + return Status::NotSupported("Not supported in compacted db mode."); + } + virtual Status Write(const WriteOptions& options, + WriteBatch* updates) override { + return Status::NotSupported("Not supported in compacted db mode."); + } + using DBImpl::CompactRange; + virtual Status CompactRange(const CompactRangeOptions& options, + ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) override { + return Status::NotSupported("Not supported in compacted db mode."); + } + + virtual Status DisableFileDeletions() override { + return Status::NotSupported("Not supported in compacted db mode."); + } + virtual Status EnableFileDeletions(bool force) override { + return Status::NotSupported("Not supported in compacted db mode."); + } + virtual Status GetLiveFiles(std::vector&, + uint64_t* manifest_file_size, + bool flush_memtable = true) override { + return Status::NotSupported("Not supported in compacted db mode."); + } + using DBImpl::Flush; + virtual Status Flush(const FlushOptions& options, + ColumnFamilyHandle* column_family) override { + return Status::NotSupported("Not supported in compacted db mode."); + } + using DB::IngestExternalFile; + virtual Status IngestExternalFile( + ColumnFamilyHandle* column_family, + const std::vector& external_files, + const IngestExternalFileOptions& ingestion_options) override { + return Status::NotSupported("Not supported in compacted db mode."); + } + + private: + friend class DB; + inline size_t FindFile(const Slice& key); + Status Init(const Options& options); + + ColumnFamilyData* cfd_; + Version* version_; + const Comparator* user_comparator_; + LevelFilesBrief files_; + + // No copying allowed + CompactedDBImpl(const CompactedDBImpl&); + void operator=(const CompactedDBImpl&); +}; +} +#endif // ROCKSDB_LITE diff --git a/db/compaction.cc b/db/compaction.cc index 0bffa0162fa..706eb3be031 100644 --- a/db/compaction.cc +++ b/db/compaction.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,12 +9,17 @@ #include "db/compaction.h" +#ifndef __STDC_FORMAT_MACROS #define __STDC_FORMAT_MACROS +#endif + #include #include #include "db/column_family.h" -#include "util/logging.h" +#include "rocksdb/compaction_filter.h" +#include "util/string_util.h" +#include "util/sync_point.h" namespace rocksdb { @@ -26,51 +31,170 @@ uint64_t TotalFileSize(const std::vector& files) { return sum; } -Compaction::Compaction(Version* input_version, int start_level, int out_level, - uint64_t target_file_size, - uint64_t max_grandparent_overlap_bytes, - uint32_t output_path_id, - CompressionType output_compression, bool seek_compaction, - bool deletion_compaction) - : start_level_(start_level), - output_level_(out_level), - max_output_file_size_(target_file_size), - max_grandparent_overlap_bytes_(max_grandparent_overlap_bytes), - input_version_(input_version), - number_levels_(input_version_->NumberLevels()), - cfd_(input_version_->cfd_), - output_path_id_(output_path_id), - output_compression_(output_compression), - seek_compaction_(seek_compaction), - deletion_compaction_(deletion_compaction), - grandparent_index_(0), - seen_key_(false), - overlapped_bytes_(0), - base_index_(-1), - parent_index_(-1), - score_(0), - bottommost_level_(false), - is_full_compaction_(false), - is_manual_compaction_(false), - level_ptrs_(std::vector(number_levels_)) { +void Compaction::SetInputVersion(Version* _input_version) { + input_version_ = _input_version; + cfd_ = input_version_->cfd(); cfd_->Ref(); input_version_->Ref(); - edit_ = new VersionEdit(); - edit_->SetColumnFamily(cfd_->GetID()); - for (int i = 0; i < number_levels_; i++) { - level_ptrs_[i] = 0; + edit_.SetColumnFamily(cfd_->GetID()); +} + +void Compaction::GetBoundaryKeys( + VersionStorageInfo* vstorage, + const std::vector& inputs, Slice* smallest_user_key, + Slice* largest_user_key) { + bool initialized = false; + const Comparator* ucmp = vstorage->InternalComparator()->user_comparator(); + for (size_t i = 0; i < inputs.size(); ++i) { + if (inputs[i].files.empty()) { + continue; + } + if (inputs[i].level == 0) { + // we need to consider all files on level 0 + for (const auto* f : inputs[i].files) { + const Slice& start_user_key = f->smallest.user_key(); + if (!initialized || + ucmp->Compare(start_user_key, *smallest_user_key) < 0) { + *smallest_user_key = start_user_key; + } + const Slice& end_user_key = f->largest.user_key(); + if (!initialized || + ucmp->Compare(end_user_key, *largest_user_key) > 0) { + *largest_user_key = end_user_key; + } + initialized = true; + } + } else { + // we only need to consider the first and last file + const Slice& start_user_key = inputs[i].files[0]->smallest.user_key(); + if (!initialized || + ucmp->Compare(start_user_key, *smallest_user_key) < 0) { + *smallest_user_key = start_user_key; + } + const Slice& end_user_key = inputs[i].files.back()->largest.user_key(); + if (!initialized || ucmp->Compare(end_user_key, *largest_user_key) > 0) { + *largest_user_key = end_user_key; + } + initialized = true; + } + } +} + +// helper function to determine if compaction is creating files at the +// bottommost level +bool Compaction::IsBottommostLevel( + int output_level, VersionStorageInfo* vstorage, + const std::vector& inputs) { + if (inputs[0].level == 0 && + inputs[0].files.back() != vstorage->LevelFiles(0).back()) { + return false; } - int num_levels = output_level_ - start_level_ + 1; - input_levels_.resize(num_levels); - inputs_.resize(num_levels); - for (int i = 0; i < num_levels; ++i) { - inputs_[i].level = start_level_ + i; + + Slice smallest_key, largest_key; + GetBoundaryKeys(vstorage, inputs, &smallest_key, &largest_key); + + // Checks whether there are files living beyond the output_level. + // If lower levels have files, it checks for overlap between files + // if the compaction process and those files. + // Bottomlevel optimizations can be made if there are no files in + // lower levels or if there is no overlap with the files in + // the lower levels. + for (int i = output_level + 1; i < vstorage->num_levels(); i++) { + // It is not the bottommost level if there are files in higher + // levels when the output level is 0 or if there are files in + // higher levels which overlap with files to be compacted. + // output_level == 0 means that we want it to be considered + // s the bottommost level only if the last file on the level + // is a part of the files to be compacted - this is verified by + // the first if condition in this function + if (vstorage->NumLevelFiles(i) > 0 && + (output_level == 0 || + vstorage->OverlapInLevel(i, &smallest_key, &largest_key))) { + return false; + } } + return true; +} + +// test function to validate the functionality of IsBottommostLevel() +// function -- determines if compaction with inputs and storage is bottommost +bool Compaction::TEST_IsBottommostLevel( + int output_level, VersionStorageInfo* vstorage, + const std::vector& inputs) { + return IsBottommostLevel(output_level, vstorage, inputs); +} + +bool Compaction::IsFullCompaction( + VersionStorageInfo* vstorage, + const std::vector& inputs) { + size_t num_files_in_compaction = 0; + size_t total_num_files = 0; + for (int l = 0; l < vstorage->num_levels(); l++) { + total_num_files += vstorage->NumLevelFiles(l); + } + for (size_t i = 0; i < inputs.size(); i++) { + num_files_in_compaction += inputs[i].size(); + } + return num_files_in_compaction == total_num_files; +} + +Compaction::Compaction(VersionStorageInfo* vstorage, + const ImmutableCFOptions& _immutable_cf_options, + const MutableCFOptions& _mutable_cf_options, + std::vector _inputs, + int _output_level, uint64_t _target_file_size, + uint64_t _max_compaction_bytes, uint32_t _output_path_id, + CompressionType _compression, + std::vector _grandparents, + bool _manual_compaction, double _score, + bool _deletion_compaction, + CompactionReason _compaction_reason) + : input_vstorage_(vstorage), + start_level_(_inputs[0].level), + output_level_(_output_level), + max_output_file_size_(_target_file_size), + max_compaction_bytes_(_max_compaction_bytes), + immutable_cf_options_(_immutable_cf_options), + mutable_cf_options_(_mutable_cf_options), + input_version_(nullptr), + number_levels_(vstorage->num_levels()), + cfd_(nullptr), + output_path_id_(_output_path_id), + output_compression_(_compression), + deletion_compaction_(_deletion_compaction), + inputs_(std::move(_inputs)), + grandparents_(std::move(_grandparents)), + score_(_score), + bottommost_level_(IsBottommostLevel(output_level_, vstorage, inputs_)), + is_full_compaction_(IsFullCompaction(vstorage, inputs_)), + is_manual_compaction_(_manual_compaction), + is_trivial_move_(false), + compaction_reason_(_compaction_reason) { + MarkFilesBeingCompacted(true); + if (is_manual_compaction_) { + compaction_reason_ = CompactionReason::kManualCompaction; + } + +#ifndef NDEBUG + for (size_t i = 1; i < inputs_.size(); ++i) { + assert(inputs_[i].level > inputs_[i - 1].level); + } +#endif + + // setup input_levels_ + { + input_levels_.resize(num_input_levels()); + for (size_t which = 0; which < num_input_levels(); which++) { + DoGenerateLevelFilesBrief(&input_levels_[which], inputs_[which].files, + &arena_); + } + } + + GetBoundaryKeys(vstorage, inputs_, &smallest_user_key_, &largest_user_key_); } Compaction::~Compaction() { - delete edit_; if (input_version_ != nullptr) { input_version_->Unref(); } @@ -81,11 +205,17 @@ Compaction::~Compaction() { } } -void Compaction::GenerateFileLevels() { - input_levels_.resize(num_input_levels()); - for (int which = 0; which < num_input_levels(); which++) { - DoGenerateFileLevel(&input_levels_[which], inputs_[which].files, &arena_); +bool Compaction::InputCompressionMatchesOutput() const { + int base_level = input_vstorage_->base_level(); + bool matches = (GetCompressionType(immutable_cf_options_, input_vstorage_, + mutable_cf_options_, start_level_, + base_level) == output_compression_); + if (matches) { + TEST_SYNC_POINT("Compaction::InputCompressionMatchesOutput:Matches"); + return true; } + TEST_SYNC_POINT("Compaction::InputCompressionMatchesOutput:DidntMatch"); + return matches; } bool Compaction::IsTrivialMove() const { @@ -93,129 +223,151 @@ bool Compaction::IsTrivialMove() const { // Otherwise, the move could create a parent file that will require // a very expensive merge later on. // If start_level_== output_level_, the purpose is to force compaction - // filter to be applied to that level, and thus cannot be a trivia move. - return (start_level_ != output_level_ && - num_input_levels() == 2 && - num_input_files(0) == 1 && - num_input_files(1) == 0 && - TotalFileSize(grandparents_) <= max_grandparent_overlap_bytes_); -} + // filter to be applied to that level, and thus cannot be a trivial move. -void Compaction::AddInputDeletions(VersionEdit* edit) { - for (int which = 0; which < num_input_levels(); which++) { - for (size_t i = 0; i < inputs_[which].size(); i++) { - edit->DeleteFile(level(which), inputs_[which][i]->fd.GetNumber()); - } + // Check if start level have files with overlapping ranges + if (start_level_ == 0 && input_vstorage_->level0_non_overlapping() == false) { + // We cannot move files from L0 to L1 if the files are overlapping + return false; } -} -bool Compaction::KeyNotExistsBeyondOutputLevel(const Slice& user_key) { - assert(cfd_->options()->compaction_style != kCompactionStyleFIFO); - if (cfd_->options()->compaction_style == kCompactionStyleUniversal) { - return bottommost_level_; + if (is_manual_compaction_ && + (immutable_cf_options_.compaction_filter != nullptr || + immutable_cf_options_.compaction_filter_factory != nullptr)) { + // This is a manual compaction and we have a compaction filter that should + // be executed, we cannot do a trivial move + return false; } - // Maybe use binary search to find right entry instead of linear search? - const Comparator* user_cmp = cfd_->user_comparator(); - for (int lvl = output_level_ + 1; lvl < number_levels_; lvl++) { - const std::vector& files = input_version_->files_[lvl]; - for (; level_ptrs_[lvl] < files.size(); ) { - FileMetaData* f = files[level_ptrs_[lvl]]; - if (user_cmp->Compare(user_key, f->largest.user_key()) <= 0) { - // We've advanced far enough - if (user_cmp->Compare(user_key, f->smallest.user_key()) >= 0) { - // Key falls in this file's range, so definitely - // exists beyond output level - return false; - } - break; - } - level_ptrs_[lvl]++; + + // Used in universal compaction, where trivial move can be done if the + // input files are non overlapping + if ((immutable_cf_options_.compaction_options_universal.allow_trivial_move) && + (output_level_ != 0)) { + return is_trivial_move_; + } + + if (!(start_level_ != output_level_ && num_input_levels() == 1 && + input(0, 0)->fd.GetPathId() == output_path_id() && + InputCompressionMatchesOutput())) { + return false; + } + + // assert inputs_.size() == 1 + + for (const auto& file : inputs_.front().files) { + std::vector file_grand_parents; + if (output_level_ + 1 >= number_levels_) { + continue; + } + input_vstorage_->GetOverlappingInputs(output_level_ + 1, &file->smallest, + &file->largest, &file_grand_parents); + const auto compaction_size = + file->fd.GetFileSize() + TotalFileSize(file_grand_parents); + if (compaction_size > max_compaction_bytes_) { + return false; } } + return true; } -bool Compaction::ShouldStopBefore(const Slice& internal_key) { - // Scan to find earliest grandparent file that contains key. - const InternalKeyComparator* icmp = &cfd_->internal_comparator(); - while (grandparent_index_ < grandparents_.size() && - icmp->Compare(internal_key, - grandparents_[grandparent_index_]->largest.Encode()) > 0) { - if (seen_key_) { - overlapped_bytes_ += grandparents_[grandparent_index_]->fd.GetFileSize(); +void Compaction::AddInputDeletions(VersionEdit* out_edit) { + for (size_t which = 0; which < num_input_levels(); which++) { + for (size_t i = 0; i < inputs_[which].size(); i++) { + out_edit->DeleteFile(level(which), inputs_[which][i]->fd.GetNumber()); } - assert(grandparent_index_ + 1 >= grandparents_.size() || - icmp->Compare(grandparents_[grandparent_index_]->largest.Encode(), - grandparents_[grandparent_index_+1]->smallest.Encode()) - < 0); - grandparent_index_++; } - seen_key_ = true; +} - if (overlapped_bytes_ > max_grandparent_overlap_bytes_) { - // Too much overlap for current output; start new output - overlapped_bytes_ = 0; +bool Compaction::KeyNotExistsBeyondOutputLevel( + const Slice& user_key, std::vector* level_ptrs) const { + assert(input_version_ != nullptr); + assert(level_ptrs != nullptr); + assert(level_ptrs->size() == static_cast(number_levels_)); + if (cfd_->ioptions()->compaction_style == kCompactionStyleLevel) { + if (output_level_ == 0) { + return false; + } + // Maybe use binary search to find right entry instead of linear search? + const Comparator* user_cmp = cfd_->user_comparator(); + for (int lvl = output_level_ + 1; lvl < number_levels_; lvl++) { + const std::vector& files = + input_vstorage_->LevelFiles(lvl); + for (; level_ptrs->at(lvl) < files.size(); level_ptrs->at(lvl)++) { + auto* f = files[level_ptrs->at(lvl)]; + if (user_cmp->Compare(user_key, f->largest.user_key()) <= 0) { + // We've advanced far enough + if (user_cmp->Compare(user_key, f->smallest.user_key()) >= 0) { + // Key falls in this file's range, so definitely + // exists beyond output level + return false; + } + break; + } + } + } return true; } else { - return false; + return bottommost_level_; } } // Mark (or clear) each file that is being compacted void Compaction::MarkFilesBeingCompacted(bool mark_as_compacted) { - for (int i = 0; i < num_input_levels(); i++) { - for (unsigned int j = 0; j < inputs_[i].size(); j++) { - assert(mark_as_compacted ? !inputs_[i][j]->being_compacted : - inputs_[i][j]->being_compacted); + for (size_t i = 0; i < num_input_levels(); i++) { + for (size_t j = 0; j < inputs_[i].size(); j++) { + assert(mark_as_compacted ? !inputs_[i][j]->being_compacted + : inputs_[i][j]->being_compacted); inputs_[i][j]->being_compacted = mark_as_compacted; } } } -// Is this compaction producing files at the bottommost level? -void Compaction::SetupBottomMostLevel(bool is_manual) { - assert(cfd_->options()->compaction_style != kCompactionStyleFIFO); - if (cfd_->options()->compaction_style == kCompactionStyleUniversal) { - // If universal compaction style is used and manual - // compaction is occuring, then we are guaranteed that - // all files will be picked in a single compaction - // run. We can safely set bottommost_level_ = true. - // If it is not manual compaction, then bottommost_level_ - // is already set when the Compaction was created. - if (is_manual) { - bottommost_level_ = true; +// Sample output: +// If compacting 3 L0 files, 2 L3 files and 1 L4 file, and outputting to L5, +// print: "3@0 + 2@3 + 1@4 files to L5" +const char* Compaction::InputLevelSummary( + InputLevelSummaryBuffer* scratch) const { + int len = 0; + bool is_first = true; + for (auto& input_level : inputs_) { + if (input_level.empty()) { + continue; } - return; - } - bottommost_level_ = true; - // checks whether there are files living beyond the output_level. - for (int i = output_level_ + 1; i < number_levels_; i++) { - if (input_version_->NumLevelFiles(i) > 0) { - bottommost_level_ = false; - break; + if (!is_first) { + len += + snprintf(scratch->buffer + len, sizeof(scratch->buffer) - len, " + "); + } else { + is_first = false; } + len += snprintf(scratch->buffer + len, sizeof(scratch->buffer) - len, + "%" ROCKSDB_PRIszt "@%d", input_level.size(), + input_level.level); } + snprintf(scratch->buffer + len, sizeof(scratch->buffer) - len, + " files to L%d", output_level()); + + return scratch->buffer; } -void Compaction::ReleaseInputs() { - if (input_version_ != nullptr) { - input_version_->Unref(); - input_version_ = nullptr; - } - if (cfd_ != nullptr) { - if (cfd_->Unref()) { - delete cfd_; +uint64_t Compaction::CalculateTotalInputSize() const { + uint64_t size = 0; + for (auto& input_level : inputs_) { + for (auto f : input_level.files) { + size += f->fd.GetFileSize(); } - cfd_ = nullptr; } + return size; } void Compaction::ReleaseCompactionFiles(Status status) { + MarkFilesBeingCompacted(false); cfd_->compaction_picker()->ReleaseCompactionFiles(this, status); } void Compaction::ResetNextCompactionIndex() { - input_version_->ResetNextCompactionIndex(start_level_); + assert(input_version_ != nullptr); + input_vstorage_->ResetNextCompactionIndex(start_level_); } namespace { @@ -223,7 +375,7 @@ int InputSummary(const std::vector& files, char* output, int len) { *output = '\0'; int write = 0; - for (unsigned int i = 0; i < files.size(); i++) { + for (size_t i = 0; i < files.size(); i++) { int sz = len - write; int ret; char sztxt[16]; @@ -240,22 +392,21 @@ int InputSummary(const std::vector& files, char* output, void Compaction::Summary(char* output, int len) { int write = - snprintf(output, len, "Base version %" PRIu64 - " Base level %d, seek compaction:%d, inputs: [", - input_version_->GetVersionNumber(), - start_level_, seek_compaction_); + snprintf(output, len, "Base version %" PRIu64 " Base level %d, inputs: [", + input_version_->GetVersionNumber(), start_level_); if (write < 0 || write >= len) { return; } - for (int level = 0; level < num_input_levels(); ++level) { - if (level > 0) { + for (size_t level_iter = 0; level_iter < num_input_levels(); ++level_iter) { + if (level_iter > 0) { write += snprintf(output + write, len - write, "], ["); if (write < 0 || write >= len) { return; } } - write += InputSummary(inputs_[level].files, output + write, len - write); + write += + InputSummary(inputs_[level_iter].files, output + write, len - write); if (write < 0 || write >= len) { return; } @@ -264,22 +415,66 @@ void Compaction::Summary(char* output, int len) { snprintf(output + write, len - write, "]"); } -uint64_t Compaction::OutputFilePreallocationSize() { +uint64_t Compaction::OutputFilePreallocationSize() const { uint64_t preallocation_size = 0; - if (cfd_->options()->compaction_style == kCompactionStyleLevel) { - preallocation_size = - cfd_->compaction_picker()->MaxFileSizeForLevel(output_level()); + if (max_output_file_size_ != port::kMaxUint64 && + (cfd_->ioptions()->compaction_style == kCompactionStyleLevel || + output_level() > 0)) { + preallocation_size = max_output_file_size_; } else { - for (int level = 0; level < num_input_levels(); ++level) { - for (const auto& f : inputs_[level].files) { - preallocation_size += f->fd.GetFileSize(); + for (const auto& level_files : inputs_) { + for (const auto& file : level_files.files) { + preallocation_size += file->fd.GetFileSize(); } } } // Over-estimate slightly so we don't end up just barely crossing // the threshold - return preallocation_size * 1.1; + return preallocation_size + (preallocation_size / 10); +} + +std::unique_ptr Compaction::CreateCompactionFilter() const { + if (!cfd_->ioptions()->compaction_filter_factory) { + return nullptr; + } + + CompactionFilter::Context context; + context.is_full_compaction = is_full_compaction_; + context.is_manual_compaction = is_manual_compaction_; + context.column_family_id = cfd_->GetID(); + return cfd_->ioptions()->compaction_filter_factory->CreateCompactionFilter( + context); +} + +bool Compaction::IsOutputLevelEmpty() const { + return inputs_.back().level != output_level_ || inputs_.back().empty(); +} + +bool Compaction::ShouldFormSubcompactions() const { + if (immutable_cf_options_.max_subcompactions <= 1 || cfd_ == nullptr) { + return false; + } + if (cfd_->ioptions()->compaction_style == kCompactionStyleLevel) { + return start_level_ == 0 && output_level_ > 0 && !IsOutputLevelEmpty(); + } else if (cfd_->ioptions()->compaction_style == kCompactionStyleUniversal) { + return number_levels_ > 1 && output_level_ > 0; + } else { + return false; + } +} + +uint64_t Compaction::MaxInputFileCreationTime() const { + uint64_t max_creation_time = 0; + for (const auto& file : inputs_[0].files) { + if (file->fd.table_reader != nullptr && + file->fd.table_reader->GetTableProperties() != nullptr) { + uint64_t creation_time = + file->fd.table_reader->GetTableProperties()->creation_time; + max_creation_time = std::max(max_creation_time, creation_time); + } + } + return max_creation_time; } } // namespace rocksdb diff --git a/db/compaction.h b/db/compaction.h index 6000f636be4..7be6df2c1e8 100644 --- a/db/compaction.h +++ b/db/compaction.h @@ -1,16 +1,17 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #pragma once +#include "db/version_set.h" +#include "options/cf_options.h" #include "util/arena.h" #include "util/autovector.h" -#include "db/version_set.h" namespace rocksdb { @@ -22,15 +23,28 @@ struct CompactionInputFiles { inline bool empty() const { return files.empty(); } inline size_t size() const { return files.size(); } inline void clear() { files.clear(); } - inline FileMetaData* operator[](int i) const { return files[i]; } + inline FileMetaData* operator[](size_t i) const { return files[i]; } }; class Version; class ColumnFamilyData; +class VersionStorageInfo; +class CompactionFilter; // A Compaction encapsulates information about a compaction. class Compaction { public: + Compaction(VersionStorageInfo* input_version, + const ImmutableCFOptions& immutable_cf_options, + const MutableCFOptions& mutable_cf_options, + std::vector inputs, int output_level, + uint64_t target_file_size, uint64_t max_compaction_bytes, + uint32_t output_path_id, CompressionType compression, + std::vector grandparents, + bool manual_compaction = false, double score = -1, + bool deletion_compaction = false, + CompactionReason compaction_reason = CompactionReason::kUnknown); + // No copying allowed Compaction(const Compaction&) = delete; void operator=(const Compaction&) = delete; @@ -39,25 +53,27 @@ class Compaction { // Returns the level associated to the specified compaction input level. // If compaction_input_level is not specified, then input_level is set to 0. - int level(int compaction_input_level = 0) const { + int level(size_t compaction_input_level = 0) const { return inputs_[compaction_input_level].level; } + int start_level() const { return start_level_; } + // Outputs will go to this level int output_level() const { return output_level_; } // Returns the number of input levels in this compaction. - int num_input_levels() const { return inputs_.size(); } + size_t num_input_levels() const { return inputs_.size(); } // Return the object that holds the edits to the descriptor done // by this compaction. - VersionEdit* edit() const { return edit_; } + VersionEdit* edit() { return &edit_; } // Returns the number of input files associated to the specified // compaction input level. // The function will return 0 if when "compaction_input_level" < 0 // or "compaction_input_level" >= "num_input_levels()". - int num_input_files(size_t compaction_input_level) const { + size_t num_input_files(size_t compaction_input_level) const { if (compaction_input_level < inputs_.size()) { return inputs_[compaction_input_level].size(); } @@ -74,7 +90,7 @@ class Compaction { // specified compaction input level. // REQUIREMENT: "compaction_input_level" must be >= 0 and // < "input_levels()" - FileMetaData* input(size_t compaction_input_level, int i) const { + FileMetaData* input(size_t compaction_input_level, size_t i) const { assert(compaction_input_level < inputs_.size()); return inputs_[compaction_input_level][i]; } @@ -83,55 +99,47 @@ class Compaction { // input level. // REQUIREMENT: "compaction_input_level" must be >= 0 and // < "input_levels()" - std::vector* const inputs(size_t compaction_input_level) { + const std::vector* inputs( + size_t compaction_input_level) const { assert(compaction_input_level < inputs_.size()); return &inputs_[compaction_input_level].files; } - // Returns the FileLevel of the specified compaction input level. - FileLevel* input_levels(int compaction_input_level) { + const std::vector* inputs() { return &inputs_; } + + // Returns the LevelFilesBrief of the specified compaction input level. + const LevelFilesBrief* input_levels(size_t compaction_input_level) const { return &input_levels_[compaction_input_level]; } // Maximum size of files to build during this compaction. - uint64_t MaxOutputFileSize() const { return max_output_file_size_; } + uint64_t max_output_file_size() const { return max_output_file_size_; } // What compression for output - CompressionType OutputCompressionType() const { return output_compression_; } + CompressionType output_compression() const { return output_compression_; } // Whether need to write output file to second DB path. - uint32_t GetOutputPathId() const { return output_path_id_; } - - // Generate input_levels_ from inputs_ - // Should be called when inputs_ is stable - void GenerateFileLevels(); + uint32_t output_path_id() const { return output_path_id_; } // Is this a trivial compaction that can be implemented by just // moving a single input file to the next level (no merging or splitting) bool IsTrivialMove() const; - // If true, then the comaction can be done by simply deleting input files. - bool IsDeletionCompaction() const { - return deletion_compaction_; - } + // If true, then the compaction can be done by simply deleting input files. + bool deletion_compaction() const { return deletion_compaction_; } // Add all inputs to this compaction as delete operations to *edit. void AddInputDeletions(VersionEdit* edit); // Returns true if the available information we have guarantees that // the input "user_key" does not exist in any level beyond "output_level()". - bool KeyNotExistsBeyondOutputLevel(const Slice& user_key); - - // Returns true iff we should stop building the current output - // before processing "internal_key". - bool ShouldStopBefore(const Slice& internal_key); - - // Release the input version for the compaction, once the compaction - // is successful. - void ReleaseInputs(); + bool KeyNotExistsBeyondOutputLevel(const Slice& user_key, + std::vector* level_ptrs) const; // Clear all files to indicate that they are not being compacted // Delete this compaction from the list of running compactions. + // + // Requirement: DB mutex held void ReleaseCompactionFiles(Status status); // Returns the summary of the compaction in "output" with maximum "len" @@ -143,92 +151,172 @@ class Compaction { double score() const { return score_; } // Is this compaction creating a file in the bottom most level? - bool BottomMostLevel() { return bottommost_level_; } + bool bottommost_level() const { return bottommost_level_; } // Does this compaction include all sst files? - bool IsFullCompaction() { return is_full_compaction_; } + bool is_full_compaction() const { return is_full_compaction_; } // Was this compaction triggered manually by the client? - bool IsManualCompaction() { return is_manual_compaction_; } + bool is_manual_compaction() const { return is_manual_compaction_; } + + // Used when allow_trivial_move option is set in + // Universal compaction. If all the input files are + // non overlapping, then is_trivial_move_ variable + // will be set true, else false + void set_is_trivial_move(bool trivial_move) { + is_trivial_move_ = trivial_move; + } + + // Used when allow_trivial_move option is set in + // Universal compaction. Returns true, if the input files + // are non-overlapping and can be trivially moved. + bool is_trivial_move() const { return is_trivial_move_; } + + // How many total levels are there? + int number_levels() const { return number_levels_; } + + // Return the ImmutableCFOptions that should be used throughout the compaction + // procedure + const ImmutableCFOptions* immutable_cf_options() const { + return &immutable_cf_options_; + } + + // Return the MutableCFOptions that should be used throughout the compaction + // procedure + const MutableCFOptions* mutable_cf_options() const { + return &mutable_cf_options_; + } // Returns the size in bytes that the output file should be preallocated to. // In level compaction, that is max_file_size_. In universal compaction, that // is the sum of all input file sizes. - uint64_t OutputFilePreallocationSize(); + uint64_t OutputFilePreallocationSize() const; + + void SetInputVersion(Version* input_version); + + struct InputLevelSummaryBuffer { + char buffer[128]; + }; + + const char* InputLevelSummary(InputLevelSummaryBuffer* scratch) const; + + uint64_t CalculateTotalInputSize() const; + + // In case of compaction error, reset the nextIndex that is used + // to pick up the next file to be compacted from files_by_size_ + void ResetNextCompactionIndex(); + + // Create a CompactionFilter from compaction_filter_factory + std::unique_ptr CreateCompactionFilter() const; + + // Is the input level corresponding to output_level_ empty? + bool IsOutputLevelEmpty() const; + + // Should this compaction be broken up into smaller ones run in parallel? + bool ShouldFormSubcompactions() const; + + // test function to validate the functionality of IsBottommostLevel() + // function -- determines if compaction with inputs and storage is bottommost + static bool TEST_IsBottommostLevel( + int output_level, VersionStorageInfo* vstorage, + const std::vector& inputs); + + TablePropertiesCollection GetOutputTableProperties() const { + return output_table_properties_; + } + + void SetOutputTableProperties(TablePropertiesCollection tp) { + output_table_properties_ = std::move(tp); + } + + Slice GetSmallestUserKey() const { return smallest_user_key_; } + + Slice GetLargestUserKey() const { return largest_user_key_; } + + CompactionReason compaction_reason() { return compaction_reason_; } + + const std::vector& grandparents() const { + return grandparents_; + } + + uint64_t max_compaction_bytes() const { return max_compaction_bytes_; } + + uint64_t MaxInputFileCreationTime() const; private: - friend class CompactionPicker; - friend class UniversalCompactionPicker; - friend class FIFOCompactionPicker; - friend class LevelCompactionPicker; + // mark (or clear) all files that are being compacted + void MarkFilesBeingCompacted(bool mark_as_compacted); + + // get the smallest and largest key present in files to be compacted + static void GetBoundaryKeys(VersionStorageInfo* vstorage, + const std::vector& inputs, + Slice* smallest_key, Slice* largest_key); + + // helper function to determine if compaction with inputs and storage is + // bottommost + static bool IsBottommostLevel( + int output_level, VersionStorageInfo* vstorage, + const std::vector& inputs); - Compaction(Version* input_version, int start_level, int out_level, - uint64_t target_file_size, uint64_t max_grandparent_overlap_bytes, - uint32_t output_path_id, CompressionType output_compression, - bool seek_compaction = false, bool deletion_compaction = false); + static bool IsFullCompaction(VersionStorageInfo* vstorage, + const std::vector& inputs); + + VersionStorageInfo* input_vstorage_; const int start_level_; // the lowest level to be compacted const int output_level_; // levels to which output files are stored uint64_t max_output_file_size_; - uint64_t max_grandparent_overlap_bytes_; + uint64_t max_compaction_bytes_; + const ImmutableCFOptions immutable_cf_options_; + const MutableCFOptions mutable_cf_options_; Version* input_version_; - VersionEdit* edit_; - int number_levels_; + VersionEdit edit_; + const int number_levels_; ColumnFamilyData* cfd_; Arena arena_; // Arena used to allocate space for file_levels_ - uint32_t output_path_id_; + const uint32_t output_path_id_; CompressionType output_compression_; - bool seek_compaction_; // If true, then the comaction can be done by simply deleting input files. - bool deletion_compaction_; + const bool deletion_compaction_; - // Compaction input files organized by level. - autovector inputs_; + // Compaction input files organized by level. Constant after construction + const std::vector inputs_; // A copy of inputs_, organized more closely in memory - autovector input_levels_; + autovector input_levels_; - // State used to check for number of of overlapping grandparent files + // State used to check for number of overlapping grandparent files // (grandparent == "output_level_ + 1") - // This vector is updated by Version::GetOverlappingInputs(). std::vector grandparents_; - size_t grandparent_index_; // Index in grandparent_starts_ - bool seen_key_; // Some output key has been seen - uint64_t overlapped_bytes_; // Bytes of overlap between current output - // and grandparent files - int base_index_; // index of the file in files_[start_level_] - int parent_index_; // index of some file with same range in - // files_[start_level_+1] - double score_; // score that was used to pick this compaction. + const double score_; // score that was used to pick this compaction. // Is this compaction creating a file in the bottom most level? - bool bottommost_level_; + const bool bottommost_level_; // Does this compaction include all sst files? - bool is_full_compaction_; + const bool is_full_compaction_; // Is this compaction requested by the client? - bool is_manual_compaction_; + const bool is_manual_compaction_; - // "level_ptrs_" holds indices into "input_version_->levels_", where each - // index remembers which file of an associated level we are currently used - // to check KeyNotExistsBeyondOutputLevel() for deletion operation. - // As it is for checking KeyNotExistsBeyondOutputLevel(), it only - // records indices for all levels beyond "output_level_". - std::vector level_ptrs_; + // True if we can do trivial move in Universal multi level + // compaction + bool is_trivial_move_; - // mark (or clear) all files that are being compacted - void MarkFilesBeingCompacted(bool mark_as_compacted); + // Does input compression match the output compression? + bool InputCompressionMatchesOutput() const; - // Initialize whether the compaction is producing files at the - // bottommost level. - // - // @see BottomMostLevel() - void SetupBottomMostLevel(bool is_manual); + // table properties of output files + TablePropertiesCollection output_table_properties_; - // In case of compaction error, reset the nextIndex that is used - // to pick up the next file to be compacted from files_by_size_ - void ResetNextCompactionIndex(); + // smallest user keys in compaction + Slice smallest_user_key_; + + // largest user keys in compaction + Slice largest_user_key_; + + // Reason for compaction + CompactionReason compaction_reason_; }; // Utility function diff --git a/db/compaction_iteration_stats.h b/db/compaction_iteration_stats.h new file mode 100644 index 00000000000..ddb534622a5 --- /dev/null +++ b/db/compaction_iteration_stats.h @@ -0,0 +1,35 @@ +// Copyright (c) 2016-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +struct CompactionIterationStats { + // Compaction statistics + + // Doesn't include records skipped because of + // CompactionFilter::Decision::kRemoveAndSkipUntil. + int64_t num_record_drop_user = 0; + + int64_t num_record_drop_hidden = 0; + int64_t num_record_drop_obsolete = 0; + int64_t num_record_drop_range_del = 0; + int64_t num_range_del_drop_obsolete = 0; + // Deletions obsoleted before bottom level due to file gap optimization. + int64_t num_optimized_del_drop_obsolete = 0; + uint64_t total_filter_time = 0; + + // Input statistics + // TODO(noetzli): The stats are incomplete. They are lacking everything + // consumed by MergeHelper. + uint64_t num_input_records = 0; + uint64_t num_input_deletion_records = 0; + uint64_t num_input_corrupt_records = 0; + uint64_t total_input_raw_key_bytes = 0; + uint64_t total_input_raw_value_bytes = 0; + + // Single-Delete diagnostics for exceptional situations + uint64_t num_single_del_fallthru = 0; + uint64_t num_single_del_mismatch = 0; +}; diff --git a/db/compaction_iterator.cc b/db/compaction_iterator.cc new file mode 100644 index 00000000000..ae63f04d83c --- /dev/null +++ b/db/compaction_iterator.cc @@ -0,0 +1,588 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/compaction_iterator.h" +#include "rocksdb/listener.h" +#include "table/internal_iterator.h" + +namespace rocksdb { + +#ifndef ROCKSDB_LITE +CompactionEventListener::CompactionListenerValueType fromInternalValueType( + ValueType vt) { + switch (vt) { + case kTypeDeletion: + return CompactionEventListener::CompactionListenerValueType::kDelete; + case kTypeValue: + return CompactionEventListener::CompactionListenerValueType::kValue; + case kTypeMerge: + return CompactionEventListener::CompactionListenerValueType:: + kMergeOperand; + case kTypeSingleDeletion: + return CompactionEventListener::CompactionListenerValueType:: + kSingleDelete; + case kTypeRangeDeletion: + return CompactionEventListener::CompactionListenerValueType::kRangeDelete; + case kTypeBlobIndex: + return CompactionEventListener::CompactionListenerValueType::kBlobIndex; + default: + assert(false); + return CompactionEventListener::CompactionListenerValueType::kInvalid; + } +} +#endif // ROCKSDB_LITE + +CompactionIterator::CompactionIterator( + InternalIterator* input, const Comparator* cmp, MergeHelper* merge_helper, + SequenceNumber last_sequence, std::vector* snapshots, + SequenceNumber earliest_write_conflict_snapshot, Env* env, + bool expect_valid_internal_key, RangeDelAggregator* range_del_agg, + const Compaction* compaction, const CompactionFilter* compaction_filter, + CompactionEventListener* compaction_listener, + const std::atomic* shutting_down) + : CompactionIterator( + input, cmp, merge_helper, last_sequence, snapshots, + earliest_write_conflict_snapshot, env, expect_valid_internal_key, + range_del_agg, + std::unique_ptr( + compaction ? new CompactionProxy(compaction) : nullptr), + compaction_filter, compaction_listener, shutting_down) {} + +CompactionIterator::CompactionIterator( + InternalIterator* input, const Comparator* cmp, MergeHelper* merge_helper, + SequenceNumber last_sequence, std::vector* snapshots, + SequenceNumber earliest_write_conflict_snapshot, Env* env, + bool expect_valid_internal_key, RangeDelAggregator* range_del_agg, + std::unique_ptr compaction, + const CompactionFilter* compaction_filter, + CompactionEventListener* compaction_listener, + const std::atomic* shutting_down) + : input_(input), + cmp_(cmp), + merge_helper_(merge_helper), + snapshots_(snapshots), + earliest_write_conflict_snapshot_(earliest_write_conflict_snapshot), + env_(env), + expect_valid_internal_key_(expect_valid_internal_key), + range_del_agg_(range_del_agg), + compaction_(std::move(compaction)), + compaction_filter_(compaction_filter), +#ifndef ROCKSDB_LITE + compaction_listener_(compaction_listener), +#endif // ROCKSDB_LITE + shutting_down_(shutting_down), + ignore_snapshots_(false), + merge_out_iter_(merge_helper_) { + assert(compaction_filter_ == nullptr || compaction_ != nullptr); + bottommost_level_ = + compaction_ == nullptr ? false : compaction_->bottommost_level(); + if (compaction_ != nullptr) { + level_ptrs_ = std::vector(compaction_->number_levels(), 0); + } + + if (snapshots_->size() == 0) { + // optimize for fast path if there are no snapshots + visible_at_tip_ = true; + earliest_snapshot_ = kMaxSequenceNumber; + latest_snapshot_ = 0; + } else { + visible_at_tip_ = false; + earliest_snapshot_ = snapshots_->at(0); + latest_snapshot_ = snapshots_->back(); + } + if (compaction_filter_ != nullptr) { + if (compaction_filter_->IgnoreSnapshots()) { + ignore_snapshots_ = true; + } + } else { + ignore_snapshots_ = false; + } + input_->SetPinnedItersMgr(&pinned_iters_mgr_); +} + +CompactionIterator::~CompactionIterator() { + // input_ Iteartor lifetime is longer than pinned_iters_mgr_ lifetime + input_->SetPinnedItersMgr(nullptr); +} + +void CompactionIterator::ResetRecordCounts() { + iter_stats_.num_record_drop_user = 0; + iter_stats_.num_record_drop_hidden = 0; + iter_stats_.num_record_drop_obsolete = 0; + iter_stats_.num_record_drop_range_del = 0; + iter_stats_.num_range_del_drop_obsolete = 0; + iter_stats_.num_optimized_del_drop_obsolete = 0; +} + +void CompactionIterator::SeekToFirst() { + NextFromInput(); + PrepareOutput(); +} + +void CompactionIterator::Next() { + // If there is a merge output, return it before continuing to process the + // input. + if (merge_out_iter_.Valid()) { + merge_out_iter_.Next(); + + // Check if we returned all records of the merge output. + if (merge_out_iter_.Valid()) { + key_ = merge_out_iter_.key(); + value_ = merge_out_iter_.value(); + bool valid_key __attribute__((__unused__)) = + ParseInternalKey(key_, &ikey_); + // MergeUntil stops when it encounters a corrupt key and does not + // include them in the result, so we expect the keys here to be valid. + assert(valid_key); + // Keep current_key_ in sync. + current_key_.UpdateInternalKey(ikey_.sequence, ikey_.type); + key_ = current_key_.GetInternalKey(); + ikey_.user_key = current_key_.GetUserKey(); + valid_ = true; + } else { + // We consumed all pinned merge operands, release pinned iterators + pinned_iters_mgr_.ReleasePinnedData(); + // MergeHelper moves the iterator to the first record after the merged + // records, so even though we reached the end of the merge output, we do + // not want to advance the iterator. + NextFromInput(); + } + } else { + // Only advance the input iterator if there is no merge output and the + // iterator is not already at the next record. + if (!at_next_) { + input_->Next(); + } + NextFromInput(); + } + + if (valid_) { + // Record that we've outputted a record for the current key. + has_outputted_key_ = true; + } + + PrepareOutput(); +} + +void CompactionIterator::NextFromInput() { + at_next_ = false; + valid_ = false; + + while (!valid_ && input_->Valid() && !IsShuttingDown()) { + key_ = input_->key(); + value_ = input_->value(); + iter_stats_.num_input_records++; + + if (!ParseInternalKey(key_, &ikey_)) { + // If `expect_valid_internal_key_` is false, return the corrupted key + // and let the caller decide what to do with it. + // TODO(noetzli): We should have a more elegant solution for this. + if (expect_valid_internal_key_) { + assert(!"Corrupted internal key not expected."); + status_ = Status::Corruption("Corrupted internal key not expected."); + break; + } + key_ = current_key_.SetInternalKey(key_); + has_current_user_key_ = false; + current_user_key_sequence_ = kMaxSequenceNumber; + current_user_key_snapshot_ = 0; + iter_stats_.num_input_corrupt_records++; + valid_ = true; + break; + } + + // Update input statistics + if (ikey_.type == kTypeDeletion || ikey_.type == kTypeSingleDeletion) { + iter_stats_.num_input_deletion_records++; + } + iter_stats_.total_input_raw_key_bytes += key_.size(); + iter_stats_.total_input_raw_value_bytes += value_.size(); + + // If need_skip is true, we should seek the input iterator + // to internal key skip_until and continue from there. + bool need_skip = false; + // Points either into compaction_filter_skip_until_ or into + // merge_helper_->compaction_filter_skip_until_. + Slice skip_until; + + // Check whether the user key changed. After this if statement current_key_ + // is a copy of the current input key (maybe converted to a delete by the + // compaction filter). ikey_.user_key is pointing to the copy. + if (!has_current_user_key_ || + !cmp_->Equal(ikey_.user_key, current_user_key_)) { + // First occurrence of this user key + // Copy key for output + key_ = current_key_.SetInternalKey(key_, &ikey_); + current_user_key_ = ikey_.user_key; + has_current_user_key_ = true; + has_outputted_key_ = false; + current_user_key_sequence_ = kMaxSequenceNumber; + current_user_key_snapshot_ = 0; + +#ifndef ROCKSDB_LITE + if (compaction_listener_) { + compaction_listener_->OnCompaction(compaction_->level(), ikey_.user_key, + fromInternalValueType(ikey_.type), + value_, ikey_.sequence, true); + } +#endif // ROCKSDB_LITE + + // apply the compaction filter to the first occurrence of the user key + if (compaction_filter_ != nullptr && + (ikey_.type == kTypeValue || ikey_.type == kTypeBlobIndex) && + (visible_at_tip_ || ikey_.sequence > latest_snapshot_ || + ignore_snapshots_)) { + // If the user has specified a compaction filter and the sequence + // number is greater than any external snapshot, then invoke the + // filter. If the return value of the compaction filter is true, + // replace the entry with a deletion marker. + CompactionFilter::Decision filter; + compaction_filter_value_.clear(); + compaction_filter_skip_until_.Clear(); + CompactionFilter::ValueType value_type = + ikey_.type == kTypeValue ? CompactionFilter::ValueType::kValue + : CompactionFilter::ValueType::kBlobIndex; + { + StopWatchNano timer(env_, true); + filter = compaction_filter_->FilterV2( + compaction_->level(), ikey_.user_key, value_type, value_, + &compaction_filter_value_, compaction_filter_skip_until_.rep()); + iter_stats_.total_filter_time += + env_ != nullptr ? timer.ElapsedNanos() : 0; + } + + if (filter == CompactionFilter::Decision::kRemoveAndSkipUntil && + cmp_->Compare(*compaction_filter_skip_until_.rep(), + ikey_.user_key) <= 0) { + // Can't skip to a key smaller than the current one. + // Keep the key as per FilterV2 documentation. + filter = CompactionFilter::Decision::kKeep; + } + + if (filter == CompactionFilter::Decision::kRemove) { + // convert the current key to a delete; key_ is pointing into + // current_key_ at this point, so updating current_key_ updates key() + ikey_.type = kTypeDeletion; + current_key_.UpdateInternalKey(ikey_.sequence, kTypeDeletion); + // no value associated with delete + value_.clear(); + iter_stats_.num_record_drop_user++; + } else if (filter == CompactionFilter::Decision::kChangeValue) { + value_ = compaction_filter_value_; + } else if (filter == CompactionFilter::Decision::kRemoveAndSkipUntil) { + need_skip = true; + compaction_filter_skip_until_.ConvertFromUserKey(kMaxSequenceNumber, + kValueTypeForSeek); + skip_until = compaction_filter_skip_until_.Encode(); + } + } + } else { +#ifndef ROCKSDB_LITE + if (compaction_listener_) { + compaction_listener_->OnCompaction(compaction_->level(), ikey_.user_key, + fromInternalValueType(ikey_.type), + value_, ikey_.sequence, false); + } +#endif // ROCKSDB_LITE + + // Update the current key to reflect the new sequence number/type without + // copying the user key. + // TODO(rven): Compaction filter does not process keys in this path + // Need to have the compaction filter process multiple versions + // if we have versions on both sides of a snapshot + current_key_.UpdateInternalKey(ikey_.sequence, ikey_.type); + key_ = current_key_.GetInternalKey(); + ikey_.user_key = current_key_.GetUserKey(); + } + + // If there are no snapshots, then this kv affect visibility at tip. + // Otherwise, search though all existing snapshots to find the earliest + // snapshot that is affected by this kv. + SequenceNumber last_sequence __attribute__((__unused__)) = + current_user_key_sequence_; + current_user_key_sequence_ = ikey_.sequence; + SequenceNumber last_snapshot = current_user_key_snapshot_; + SequenceNumber prev_snapshot = 0; // 0 means no previous snapshot + current_user_key_snapshot_ = + visible_at_tip_ + ? earliest_snapshot_ + : findEarliestVisibleSnapshot(ikey_.sequence, &prev_snapshot); + + if (need_skip) { + // This case is handled below. + } else if (clear_and_output_next_key_) { + // In the previous iteration we encountered a single delete that we could + // not compact out. We will keep this Put, but can drop it's data. + // (See Optimization 3, below.) + assert(ikey_.type == kTypeValue); + assert(current_user_key_snapshot_ == last_snapshot); + + value_.clear(); + valid_ = true; + clear_and_output_next_key_ = false; + } else if (ikey_.type == kTypeSingleDeletion) { + // We can compact out a SingleDelete if: + // 1) We encounter the corresponding PUT -OR- we know that this key + // doesn't appear past this output level + // =AND= + // 2) We've already returned a record in this snapshot -OR- + // there are no earlier earliest_write_conflict_snapshot. + // + // Rule 1 is needed for SingleDelete correctness. Rule 2 is needed to + // allow Transactions to do write-conflict checking (if we compacted away + // all keys, then we wouldn't know that a write happened in this + // snapshot). If there is no earlier snapshot, then we know that there + // are no active transactions that need to know about any writes. + // + // Optimization 3: + // If we encounter a SingleDelete followed by a PUT and Rule 2 is NOT + // true, then we must output a SingleDelete. In this case, we will decide + // to also output the PUT. While we are compacting less by outputting the + // PUT now, hopefully this will lead to better compaction in the future + // when Rule 2 is later true (Ie, We are hoping we can later compact out + // both the SingleDelete and the Put, while we couldn't if we only + // outputted the SingleDelete now). + // In this case, we can save space by removing the PUT's value as it will + // never be read. + // + // Deletes and Merges are not supported on the same key that has a + // SingleDelete as it is not possible to correctly do any partial + // compaction of such a combination of operations. The result of mixing + // those operations for a given key is documented as being undefined. So + // we can choose how to handle such a combinations of operations. We will + // try to compact out as much as we can in these cases. + // We will report counts on these anomalous cases. + + // The easiest way to process a SingleDelete during iteration is to peek + // ahead at the next key. + ParsedInternalKey next_ikey; + input_->Next(); + + // Check whether the next key exists, is not corrupt, and is the same key + // as the single delete. + if (input_->Valid() && ParseInternalKey(input_->key(), &next_ikey) && + cmp_->Equal(ikey_.user_key, next_ikey.user_key)) { + // Check whether the next key belongs to the same snapshot as the + // SingleDelete. + if (prev_snapshot == 0 || next_ikey.sequence > prev_snapshot) { + if (next_ikey.type == kTypeSingleDeletion) { + // We encountered two SingleDeletes in a row. This could be due to + // unexpected user input. + // Skip the first SingleDelete and let the next iteration decide how + // to handle the second SingleDelete + + // First SingleDelete has been skipped since we already called + // input_->Next(). + ++iter_stats_.num_record_drop_obsolete; + ++iter_stats_.num_single_del_mismatch; + } else if ((ikey_.sequence <= earliest_write_conflict_snapshot_) || + has_outputted_key_) { + // Found a matching value, we can drop the single delete and the + // value. It is safe to drop both records since we've already + // outputted a key in this snapshot, or there is no earlier + // snapshot (Rule 2 above). + + // Note: it doesn't matter whether the second key is a Put or if it + // is an unexpected Merge or Delete. We will compact it out + // either way. We will maintain counts of how many mismatches + // happened + if (next_ikey.type != kTypeValue) { + ++iter_stats_.num_single_del_mismatch; + } + + ++iter_stats_.num_record_drop_hidden; + ++iter_stats_.num_record_drop_obsolete; + // Already called input_->Next() once. Call it a second time to + // skip past the second key. + input_->Next(); + } else { + // Found a matching value, but we cannot drop both keys since + // there is an earlier snapshot and we need to leave behind a record + // to know that a write happened in this snapshot (Rule 2 above). + // Clear the value and output the SingleDelete. (The value will be + // outputted on the next iteration.) + + // Setting valid_ to true will output the current SingleDelete + valid_ = true; + + // Set up the Put to be outputted in the next iteration. + // (Optimization 3). + clear_and_output_next_key_ = true; + } + } else { + // We hit the next snapshot without hitting a put, so the iterator + // returns the single delete. + valid_ = true; + } + } else { + // We are at the end of the input, could not parse the next key, or hit + // a different key. The iterator returns the single delete if the key + // possibly exists beyond the current output level. We set + // has_current_user_key to false so that if the iterator is at the next + // key, we do not compare it again against the previous key at the next + // iteration. If the next key is corrupt, we return before the + // comparison, so the value of has_current_user_key does not matter. + has_current_user_key_ = false; + if (compaction_ != nullptr && ikey_.sequence <= earliest_snapshot_ && + compaction_->KeyNotExistsBeyondOutputLevel(ikey_.user_key, + &level_ptrs_)) { + // Key doesn't exist outside of this range. + // Can compact out this SingleDelete. + ++iter_stats_.num_record_drop_obsolete; + ++iter_stats_.num_single_del_fallthru; + if (!bottommost_level_) { + ++iter_stats_.num_optimized_del_drop_obsolete; + } + } else { + // Output SingleDelete + valid_ = true; + } + } + + if (valid_) { + at_next_ = true; + } + } else if (last_snapshot == current_user_key_snapshot_) { + // If the earliest snapshot is which this key is visible in + // is the same as the visibility of a previous instance of the + // same key, then this kv is not visible in any snapshot. + // Hidden by an newer entry for same user key + // TODO(noetzli): why not > ? + // + // Note: Dropping this key will not affect TransactionDB write-conflict + // checking since there has already been a record returned for this key + // in this snapshot. + assert(last_sequence >= current_user_key_sequence_); + ++iter_stats_.num_record_drop_hidden; // (A) + input_->Next(); + } else if (compaction_ != nullptr && ikey_.type == kTypeDeletion && + ikey_.sequence <= earliest_snapshot_ && + compaction_->KeyNotExistsBeyondOutputLevel(ikey_.user_key, + &level_ptrs_)) { + // TODO(noetzli): This is the only place where we use compaction_ + // (besides the constructor). We should probably get rid of this + // dependency and find a way to do similar filtering during flushes. + // + // For this user key: + // (1) there is no data in higher levels + // (2) data in lower levels will have larger sequence numbers + // (3) data in layers that are being compacted here and have + // smaller sequence numbers will be dropped in the next + // few iterations of this loop (by rule (A) above). + // Therefore this deletion marker is obsolete and can be dropped. + // + // Note: Dropping this Delete will not affect TransactionDB + // write-conflict checking since it is earlier than any snapshot. + ++iter_stats_.num_record_drop_obsolete; + if (!bottommost_level_) { + ++iter_stats_.num_optimized_del_drop_obsolete; + } + input_->Next(); + } else if (ikey_.type == kTypeMerge) { + if (!merge_helper_->HasOperator()) { + status_ = Status::InvalidArgument( + "merge_operator is not properly initialized."); + return; + } + + pinned_iters_mgr_.StartPinning(); + // We know the merge type entry is not hidden, otherwise we would + // have hit (A) + // We encapsulate the merge related state machine in a different + // object to minimize change to the existing flow. + Status s = merge_helper_->MergeUntil(input_, range_del_agg_, + prev_snapshot, bottommost_level_); + merge_out_iter_.SeekToFirst(); + + if (!s.ok() && !s.IsMergeInProgress()) { + status_ = s; + return; + } else if (merge_out_iter_.Valid()) { + // NOTE: key, value, and ikey_ refer to old entries. + // These will be correctly set below. + key_ = merge_out_iter_.key(); + value_ = merge_out_iter_.value(); + bool valid_key __attribute__((__unused__)) = + ParseInternalKey(key_, &ikey_); + // MergeUntil stops when it encounters a corrupt key and does not + // include them in the result, so we expect the keys here to valid. + assert(valid_key); + // Keep current_key_ in sync. + current_key_.UpdateInternalKey(ikey_.sequence, ikey_.type); + key_ = current_key_.GetInternalKey(); + ikey_.user_key = current_key_.GetUserKey(); + valid_ = true; + } else { + // all merge operands were filtered out. reset the user key, since the + // batch consumed by the merge operator should not shadow any keys + // coming after the merges + has_current_user_key_ = false; + pinned_iters_mgr_.ReleasePinnedData(); + + if (merge_helper_->FilteredUntil(&skip_until)) { + need_skip = true; + } + } + } else { + // 1. new user key -OR- + // 2. different snapshot stripe + bool should_delete = range_del_agg_->ShouldDelete( + key_, RangeDelAggregator::RangePositioningMode::kForwardTraversal); + if (should_delete) { + ++iter_stats_.num_record_drop_hidden; + ++iter_stats_.num_record_drop_range_del; + input_->Next(); + } else { + valid_ = true; + } + } + + if (need_skip) { + input_->Seek(skip_until); + } + } + + if (!valid_ && IsShuttingDown()) { + status_ = Status::ShutdownInProgress(); + } +} + +void CompactionIterator::PrepareOutput() { + // Zeroing out the sequence number leads to better compression. + // If this is the bottommost level (no files in lower levels) + // and the earliest snapshot is larger than this seqno + // and the userkey differs from the last userkey in compaction + // then we can squash the seqno to zero. + + // This is safe for TransactionDB write-conflict checking since transactions + // only care about sequence number larger than any active snapshots. + if ((compaction_ != nullptr && !compaction_->allow_ingest_behind()) && + bottommost_level_ && valid_ && ikey_.sequence <= earliest_snapshot_ && + ikey_.type != kTypeMerge && + !cmp_->Equal(compaction_->GetLargestUserKey(), ikey_.user_key)) { + assert(ikey_.type != kTypeDeletion && ikey_.type != kTypeSingleDeletion); + ikey_.sequence = 0; + current_key_.UpdateInternalKey(0, ikey_.type); + } +} + +inline SequenceNumber CompactionIterator::findEarliestVisibleSnapshot( + SequenceNumber in, SequenceNumber* prev_snapshot) { + assert(snapshots_->size()); + SequenceNumber prev __attribute__((__unused__)) = kMaxSequenceNumber; + for (const auto cur : *snapshots_) { + assert(prev == kMaxSequenceNumber || prev <= cur); + if (cur >= in) { + *prev_snapshot = prev == kMaxSequenceNumber ? 0 : prev; + return cur; + } + prev = cur; + assert(prev < kMaxSequenceNumber); + } + *prev_snapshot = prev; + return kMaxSequenceNumber; +} + +} // namespace rocksdb diff --git a/db/compaction_iterator.h b/db/compaction_iterator.h new file mode 100644 index 00000000000..cad23866699 --- /dev/null +++ b/db/compaction_iterator.h @@ -0,0 +1,197 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once + +#include +#include +#include +#include + +#include "db/compaction.h" +#include "db/compaction_iteration_stats.h" +#include "db/merge_helper.h" +#include "db/pinned_iterators_manager.h" +#include "db/range_del_aggregator.h" +#include "options/cf_options.h" +#include "rocksdb/compaction_filter.h" + +namespace rocksdb { + +class CompactionEventListener; + +class CompactionIterator { + public: + // A wrapper around Compaction. Has a much smaller interface, only what + // CompactionIterator uses. Tests can override it. + class CompactionProxy { + public: + explicit CompactionProxy(const Compaction* compaction) + : compaction_(compaction) {} + + virtual ~CompactionProxy() = default; + virtual int level(size_t compaction_input_level = 0) const { + return compaction_->level(); + } + virtual bool KeyNotExistsBeyondOutputLevel( + const Slice& user_key, std::vector* level_ptrs) const { + return compaction_->KeyNotExistsBeyondOutputLevel(user_key, level_ptrs); + } + virtual bool bottommost_level() const { + return compaction_->bottommost_level(); + } + virtual int number_levels() const { return compaction_->number_levels(); } + virtual Slice GetLargestUserKey() const { + return compaction_->GetLargestUserKey(); + } + virtual bool allow_ingest_behind() const { + return compaction_->immutable_cf_options()->allow_ingest_behind; + } + + protected: + CompactionProxy() = default; + + private: + const Compaction* compaction_; + }; + + CompactionIterator(InternalIterator* input, const Comparator* cmp, + MergeHelper* merge_helper, SequenceNumber last_sequence, + std::vector* snapshots, + SequenceNumber earliest_write_conflict_snapshot, Env* env, + bool expect_valid_internal_key, + RangeDelAggregator* range_del_agg, + const Compaction* compaction = nullptr, + const CompactionFilter* compaction_filter = nullptr, + CompactionEventListener* compaction_listener = nullptr, + const std::atomic* shutting_down = nullptr); + + // Constructor with custom CompactionProxy, used for tests. + CompactionIterator(InternalIterator* input, const Comparator* cmp, + MergeHelper* merge_helper, SequenceNumber last_sequence, + std::vector* snapshots, + SequenceNumber earliest_write_conflict_snapshot, Env* env, + bool expect_valid_internal_key, + RangeDelAggregator* range_del_agg, + std::unique_ptr compaction, + const CompactionFilter* compaction_filter = nullptr, + CompactionEventListener* compaction_listener = nullptr, + const std::atomic* shutting_down = nullptr); + + ~CompactionIterator(); + + void ResetRecordCounts(); + + // Seek to the beginning of the compaction iterator output. + // + // REQUIRED: Call only once. + void SeekToFirst(); + + // Produces the next record in the compaction. + // + // REQUIRED: SeekToFirst() has been called. + void Next(); + + // Getters + const Slice& key() const { return key_; } + const Slice& value() const { return value_; } + const Status& status() const { return status_; } + const ParsedInternalKey& ikey() const { return ikey_; } + bool Valid() const { return valid_; } + const Slice& user_key() const { return current_user_key_; } + const CompactionIterationStats& iter_stats() const { return iter_stats_; } + + private: + // Processes the input stream to find the next output + void NextFromInput(); + + // Do last preparations before presenting the output to the callee. At this + // point this only zeroes out the sequence number if possible for better + // compression. + void PrepareOutput(); + + // Given a sequence number, return the sequence number of the + // earliest snapshot that this sequence number is visible in. + // The snapshots themselves are arranged in ascending order of + // sequence numbers. + // Employ a sequential search because the total number of + // snapshots are typically small. + inline SequenceNumber findEarliestVisibleSnapshot( + SequenceNumber in, SequenceNumber* prev_snapshot); + + InternalIterator* input_; + const Comparator* cmp_; + MergeHelper* merge_helper_; + const std::vector* snapshots_; + const SequenceNumber earliest_write_conflict_snapshot_; + Env* env_; + bool expect_valid_internal_key_; + RangeDelAggregator* range_del_agg_; + std::unique_ptr compaction_; + const CompactionFilter* compaction_filter_; +#ifndef ROCKSDB_LITE + CompactionEventListener* compaction_listener_; +#endif // ROCKSDB_LITE + const std::atomic* shutting_down_; + bool bottommost_level_; + bool valid_ = false; + bool visible_at_tip_; + SequenceNumber earliest_snapshot_; + SequenceNumber latest_snapshot_; + bool ignore_snapshots_; + + // State + // + // Points to a copy of the current compaction iterator output (current_key_) + // if valid_. + Slice key_; + // Points to the value in the underlying iterator that corresponds to the + // current output. + Slice value_; + // The status is OK unless compaction iterator encounters a merge operand + // while not having a merge operator defined. + Status status_; + // Stores the user key, sequence number and type of the current compaction + // iterator output (or current key in the underlying iterator during + // NextFromInput()). + ParsedInternalKey ikey_; + // Stores whether ikey_.user_key is valid. If set to false, the user key is + // not compared against the current key in the underlying iterator. + bool has_current_user_key_ = false; + bool at_next_ = false; // If false, the iterator + // Holds a copy of the current compaction iterator output (or current key in + // the underlying iterator during NextFromInput()). + IterKey current_key_; + Slice current_user_key_; + SequenceNumber current_user_key_sequence_; + SequenceNumber current_user_key_snapshot_; + + // True if the iterator has already returned a record for the current key. + bool has_outputted_key_ = false; + + // truncated the value of the next key and output it without applying any + // compaction rules. This is used for outputting a put after a single delete. + bool clear_and_output_next_key_ = false; + + MergeOutputIterator merge_out_iter_; + // PinnedIteratorsManager used to pin input_ Iterator blocks while reading + // merge operands and then releasing them after consuming them. + PinnedIteratorsManager pinned_iters_mgr_; + std::string compaction_filter_value_; + InternalKey compaction_filter_skip_until_; + // "level_ptrs" holds indices that remember which file of an associated + // level we were last checking during the last call to compaction-> + // KeyNotExistsBeyondOutputLevel(). This allows future calls to the function + // to pick off where it left off since each subcompaction's key range is + // increasing so a later call to the function must be looking for a key that + // is in or beyond the last file checked during the previous call + std::vector level_ptrs_; + CompactionIterationStats iter_stats_; + + bool IsShuttingDown() { + // This is a best-effort facility, so memory_order_relaxed is sufficient. + return shutting_down_ && shutting_down_->load(std::memory_order_relaxed); + } +}; +} // namespace rocksdb diff --git a/db/compaction_iterator_test.cc b/db/compaction_iterator_test.cc new file mode 100644 index 00000000000..dfc4139363b --- /dev/null +++ b/db/compaction_iterator_test.cc @@ -0,0 +1,568 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/compaction_iterator.h" + +#include +#include + +#include "port/port.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +// Expects no merging attempts. +class NoMergingMergeOp : public MergeOperator { + public: + bool FullMergeV2(const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const override { + ADD_FAILURE(); + return false; + } + bool PartialMergeMulti(const Slice& key, + const std::deque& operand_list, + std::string* new_value, + Logger* logger) const override { + ADD_FAILURE(); + return false; + } + const char* Name() const override { + return "CompactionIteratorTest NoMergingMergeOp"; + } +}; + +// Compaction filter that gets stuck when it sees a particular key, +// then gets unstuck when told to. +// Always returns Decition::kRemove. +class StallingFilter : public CompactionFilter { + public: + virtual Decision FilterV2(int level, const Slice& key, ValueType t, + const Slice& existing_value, std::string* new_value, + std::string* skip_until) const override { + int k = std::atoi(key.ToString().c_str()); + last_seen.store(k); + while (k >= stall_at.load()) { + std::this_thread::yield(); + } + return Decision::kRemove; + } + + const char* Name() const override { + return "CompactionIteratorTest StallingFilter"; + } + + // Wait until the filter sees a key >= k and stalls at that key. + // If `exact`, asserts that the seen key is equal to k. + void WaitForStall(int k, bool exact = true) { + stall_at.store(k); + while (last_seen.load() < k) { + std::this_thread::yield(); + } + if (exact) { + EXPECT_EQ(k, last_seen.load()); + } + } + + // Filter will stall on key >= stall_at. Advance stall_at to unstall. + mutable std::atomic stall_at{0}; + // Last key the filter was called with. + mutable std::atomic last_seen{0}; +}; + +class LoggingForwardVectorIterator : public InternalIterator { + public: + struct Action { + enum class Type { + SEEK_TO_FIRST, + SEEK, + NEXT, + }; + + Type type; + std::string arg; + + explicit Action(Type _type, std::string _arg = "") + : type(_type), arg(_arg) {} + + bool operator==(const Action& rhs) const { + return std::tie(type, arg) == std::tie(rhs.type, rhs.arg); + } + }; + + LoggingForwardVectorIterator(const std::vector& keys, + const std::vector& values) + : keys_(keys), values_(values), current_(keys.size()) { + assert(keys_.size() == values_.size()); + } + + virtual bool Valid() const override { return current_ < keys_.size(); } + + virtual void SeekToFirst() override { + log.emplace_back(Action::Type::SEEK_TO_FIRST); + current_ = 0; + } + virtual void SeekToLast() override { assert(false); } + + virtual void Seek(const Slice& target) override { + log.emplace_back(Action::Type::SEEK, target.ToString()); + current_ = std::lower_bound(keys_.begin(), keys_.end(), target.ToString()) - + keys_.begin(); + } + + virtual void SeekForPrev(const Slice& target) override { assert(false); } + + virtual void Next() override { + assert(Valid()); + log.emplace_back(Action::Type::NEXT); + current_++; + } + virtual void Prev() override { assert(false); } + + virtual Slice key() const override { + assert(Valid()); + return Slice(keys_[current_]); + } + virtual Slice value() const override { + assert(Valid()); + return Slice(values_[current_]); + } + + virtual Status status() const override { return Status::OK(); } + + std::vector log; + + private: + std::vector keys_; + std::vector values_; + size_t current_; +}; + +class FakeCompaction : public CompactionIterator::CompactionProxy { + public: + FakeCompaction() = default; + + virtual int level(size_t compaction_input_level) const { return 0; } + virtual bool KeyNotExistsBeyondOutputLevel( + const Slice& user_key, std::vector* level_ptrs) const { + return key_not_exists_beyond_output_level; + } + virtual bool bottommost_level() const { return false; } + virtual int number_levels() const { return 1; } + virtual Slice GetLargestUserKey() const { + return "\xff\xff\xff\xff\xff\xff\xff\xff\xff"; + } + virtual bool allow_ingest_behind() const { return false; } + + bool key_not_exists_beyond_output_level = false; +}; + +class CompactionIteratorTest : public testing::Test { + public: + CompactionIteratorTest() + : cmp_(BytewiseComparator()), icmp_(cmp_), snapshots_({}) {} + + void InitIterators(const std::vector& ks, + const std::vector& vs, + const std::vector& range_del_ks, + const std::vector& range_del_vs, + SequenceNumber last_sequence, + MergeOperator* merge_op = nullptr, + CompactionFilter* filter = nullptr) { + std::unique_ptr range_del_iter( + new test::VectorIterator(range_del_ks, range_del_vs)); + range_del_agg_.reset(new RangeDelAggregator(icmp_, snapshots_)); + ASSERT_OK(range_del_agg_->AddTombstones(std::move(range_del_iter))); + + std::unique_ptr compaction; + if (filter) { + compaction_proxy_ = new FakeCompaction(); + compaction.reset(compaction_proxy_); + } + + merge_helper_.reset(new MergeHelper(Env::Default(), cmp_, merge_op, filter, + nullptr, false, 0, 0, nullptr, + &shutting_down_)); + iter_.reset(new LoggingForwardVectorIterator(ks, vs)); + iter_->SeekToFirst(); + c_iter_.reset(new CompactionIterator( + iter_.get(), cmp_, merge_helper_.get(), last_sequence, &snapshots_, + kMaxSequenceNumber, Env::Default(), false, range_del_agg_.get(), + std::move(compaction), filter, nullptr, &shutting_down_)); + } + + void AddSnapshot(SequenceNumber snapshot) { snapshots_.push_back(snapshot); } + + const Comparator* cmp_; + const InternalKeyComparator icmp_; + std::vector snapshots_; + std::unique_ptr merge_helper_; + std::unique_ptr iter_; + std::unique_ptr c_iter_; + std::unique_ptr range_del_agg_; + std::atomic shutting_down_{false}; + FakeCompaction* compaction_proxy_; +}; + +// It is possible that the output of the compaction iterator is empty even if +// the input is not. +TEST_F(CompactionIteratorTest, EmptyResult) { + InitIterators({test::KeyStr("a", 5, kTypeSingleDeletion), + test::KeyStr("a", 3, kTypeValue)}, + {"", "val"}, {}, {}, 5); + c_iter_->SeekToFirst(); + ASSERT_FALSE(c_iter_->Valid()); +} + +// If there is a corruption after a single deletion, the corrupted key should +// be preserved. +TEST_F(CompactionIteratorTest, CorruptionAfterSingleDeletion) { + InitIterators({test::KeyStr("a", 5, kTypeSingleDeletion), + test::KeyStr("a", 3, kTypeValue, true), + test::KeyStr("b", 10, kTypeValue)}, + {"", "val", "val2"}, {}, {}, 10); + c_iter_->SeekToFirst(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("a", 5, kTypeSingleDeletion), + c_iter_->key().ToString()); + c_iter_->Next(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("a", 3, kTypeValue, true), c_iter_->key().ToString()); + c_iter_->Next(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("b", 10, kTypeValue), c_iter_->key().ToString()); + c_iter_->Next(); + ASSERT_FALSE(c_iter_->Valid()); +} + +TEST_F(CompactionIteratorTest, SimpleRangeDeletion) { + InitIterators({test::KeyStr("morning", 5, kTypeValue), + test::KeyStr("morning", 2, kTypeValue), + test::KeyStr("night", 3, kTypeValue)}, + {"zao", "zao", "wan"}, + {test::KeyStr("ma", 4, kTypeRangeDeletion)}, {"mz"}, 5); + c_iter_->SeekToFirst(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("morning", 5, kTypeValue), c_iter_->key().ToString()); + c_iter_->Next(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("night", 3, kTypeValue), c_iter_->key().ToString()); + c_iter_->Next(); + ASSERT_FALSE(c_iter_->Valid()); +} + +TEST_F(CompactionIteratorTest, RangeDeletionWithSnapshots) { + AddSnapshot(10); + std::vector ks1; + ks1.push_back(test::KeyStr("ma", 28, kTypeRangeDeletion)); + std::vector vs1{"mz"}; + std::vector ks2{test::KeyStr("morning", 15, kTypeValue), + test::KeyStr("morning", 5, kTypeValue), + test::KeyStr("night", 40, kTypeValue), + test::KeyStr("night", 20, kTypeValue)}; + std::vector vs2{"zao 15", "zao 5", "wan 40", "wan 20"}; + InitIterators(ks2, vs2, ks1, vs1, 40); + c_iter_->SeekToFirst(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("morning", 5, kTypeValue), c_iter_->key().ToString()); + c_iter_->Next(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("night", 40, kTypeValue), c_iter_->key().ToString()); + c_iter_->Next(); + ASSERT_FALSE(c_iter_->Valid()); +} + +TEST_F(CompactionIteratorTest, CompactionFilterSkipUntil) { + class Filter : public CompactionFilter { + virtual Decision FilterV2(int level, const Slice& key, ValueType t, + const Slice& existing_value, + std::string* new_value, + std::string* skip_until) const override { + std::string k = key.ToString(); + std::string v = existing_value.ToString(); + // See InitIterators() call below for the sequence of keys and their + // filtering decisions. Here we closely assert that compaction filter is + // called with the expected keys and only them, and with the right values. + if (k == "a") { + EXPECT_EQ(ValueType::kValue, t); + EXPECT_EQ("av50", v); + return Decision::kKeep; + } + if (k == "b") { + EXPECT_EQ(ValueType::kValue, t); + EXPECT_EQ("bv60", v); + *skip_until = "d+"; + return Decision::kRemoveAndSkipUntil; + } + if (k == "e") { + EXPECT_EQ(ValueType::kMergeOperand, t); + EXPECT_EQ("em71", v); + return Decision::kKeep; + } + if (k == "f") { + if (v == "fm65") { + EXPECT_EQ(ValueType::kMergeOperand, t); + *skip_until = "f"; + } else { + EXPECT_EQ("fm30", v); + EXPECT_EQ(ValueType::kMergeOperand, t); + *skip_until = "g+"; + } + return Decision::kRemoveAndSkipUntil; + } + if (k == "h") { + EXPECT_EQ(ValueType::kValue, t); + EXPECT_EQ("hv91", v); + return Decision::kKeep; + } + if (k == "i") { + EXPECT_EQ(ValueType::kMergeOperand, t); + EXPECT_EQ("im95", v); + *skip_until = "z"; + return Decision::kRemoveAndSkipUntil; + } + ADD_FAILURE(); + return Decision::kKeep; + } + + const char* Name() const override { + return "CompactionIteratorTest.CompactionFilterSkipUntil::Filter"; + } + }; + + NoMergingMergeOp merge_op; + Filter filter; + InitIterators( + {test::KeyStr("a", 50, kTypeValue), // keep + test::KeyStr("a", 45, kTypeMerge), + test::KeyStr("b", 60, kTypeValue), // skip to "d+" + test::KeyStr("b", 40, kTypeValue), test::KeyStr("c", 35, kTypeValue), + test::KeyStr("d", 70, kTypeMerge), + test::KeyStr("e", 71, kTypeMerge), // keep + test::KeyStr("f", 65, kTypeMerge), // skip to "f", aka keep + test::KeyStr("f", 30, kTypeMerge), // skip to "g+" + test::KeyStr("f", 25, kTypeValue), test::KeyStr("g", 90, kTypeValue), + test::KeyStr("h", 91, kTypeValue), // keep + test::KeyStr("i", 95, kTypeMerge), // skip to "z" + test::KeyStr("j", 99, kTypeValue)}, + {"av50", "am45", "bv60", "bv40", "cv35", "dm70", "em71", "fm65", "fm30", + "fv25", "gv90", "hv91", "im95", "jv99"}, + {}, {}, kMaxSequenceNumber, &merge_op, &filter); + + // Compaction should output just "a", "e" and "h" keys. + c_iter_->SeekToFirst(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("a", 50, kTypeValue), c_iter_->key().ToString()); + ASSERT_EQ("av50", c_iter_->value().ToString()); + c_iter_->Next(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("e", 71, kTypeMerge), c_iter_->key().ToString()); + ASSERT_EQ("em71", c_iter_->value().ToString()); + c_iter_->Next(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("h", 91, kTypeValue), c_iter_->key().ToString()); + ASSERT_EQ("hv91", c_iter_->value().ToString()); + c_iter_->Next(); + ASSERT_FALSE(c_iter_->Valid()); + + // Check that the compaction iterator did the correct sequence of calls on + // the underlying iterator. + using A = LoggingForwardVectorIterator::Action; + using T = A::Type; + std::vector expected_actions = { + A(T::SEEK_TO_FIRST), + A(T::NEXT), + A(T::NEXT), + A(T::SEEK, test::KeyStr("d+", kMaxSequenceNumber, kValueTypeForSeek)), + A(T::NEXT), + A(T::NEXT), + A(T::SEEK, test::KeyStr("g+", kMaxSequenceNumber, kValueTypeForSeek)), + A(T::NEXT), + A(T::SEEK, test::KeyStr("z", kMaxSequenceNumber, kValueTypeForSeek))}; + ASSERT_EQ(expected_actions, iter_->log); +} + +TEST_F(CompactionIteratorTest, ShuttingDownInFilter) { + NoMergingMergeOp merge_op; + StallingFilter filter; + InitIterators( + {test::KeyStr("1", 1, kTypeValue), test::KeyStr("2", 2, kTypeValue), + test::KeyStr("3", 3, kTypeValue), test::KeyStr("4", 4, kTypeValue)}, + {"v1", "v2", "v3", "v4"}, {}, {}, kMaxSequenceNumber, &merge_op, &filter); + // Don't leave tombstones (kTypeDeletion) for filtered keys. + compaction_proxy_->key_not_exists_beyond_output_level = true; + + std::atomic seek_done{false}; + rocksdb::port::Thread compaction_thread([&] { + c_iter_->SeekToFirst(); + EXPECT_FALSE(c_iter_->Valid()); + EXPECT_TRUE(c_iter_->status().IsShutdownInProgress()); + seek_done.store(true); + }); + + // Let key 1 through. + filter.WaitForStall(1); + + // Shutdown during compaction filter call for key 2. + filter.WaitForStall(2); + shutting_down_.store(true); + EXPECT_FALSE(seek_done.load()); + + // Unstall filter and wait for SeekToFirst() to return. + filter.stall_at.store(3); + compaction_thread.join(); + assert(seek_done.load()); + + // Check that filter was never called again. + EXPECT_EQ(2, filter.last_seen.load()); +} + +// Same as ShuttingDownInFilter, but shutdown happens during filter call for +// a merge operand, not for a value. +TEST_F(CompactionIteratorTest, ShuttingDownInMerge) { + NoMergingMergeOp merge_op; + StallingFilter filter; + InitIterators( + {test::KeyStr("1", 1, kTypeValue), test::KeyStr("2", 2, kTypeMerge), + test::KeyStr("3", 3, kTypeMerge), test::KeyStr("4", 4, kTypeValue)}, + {"v1", "v2", "v3", "v4"}, {}, {}, kMaxSequenceNumber, &merge_op, &filter); + compaction_proxy_->key_not_exists_beyond_output_level = true; + + std::atomic seek_done{false}; + rocksdb::port::Thread compaction_thread([&] { + c_iter_->SeekToFirst(); + ASSERT_FALSE(c_iter_->Valid()); + ASSERT_TRUE(c_iter_->status().IsShutdownInProgress()); + seek_done.store(true); + }); + + // Let key 1 through. + filter.WaitForStall(1); + + // Shutdown during compaction filter call for key 2. + filter.WaitForStall(2); + shutting_down_.store(true); + EXPECT_FALSE(seek_done.load()); + + // Unstall filter and wait for SeekToFirst() to return. + filter.stall_at.store(3); + compaction_thread.join(); + assert(seek_done.load()); + + // Check that filter was never called again. + EXPECT_EQ(2, filter.last_seen.load()); +} + +TEST_F(CompactionIteratorTest, SingleMergeOperand) { + class Filter : public CompactionFilter { + virtual Decision FilterV2(int level, const Slice& key, ValueType t, + const Slice& existing_value, + std::string* new_value, + std::string* skip_until) const override { + std::string k = key.ToString(); + std::string v = existing_value.ToString(); + + // See InitIterators() call below for the sequence of keys and their + // filtering decisions. Here we closely assert that compaction filter is + // called with the expected keys and only them, and with the right values. + if (k == "a") { + EXPECT_EQ(ValueType::kMergeOperand, t); + EXPECT_EQ("av1", v); + return Decision::kKeep; + } else if (k == "b") { + EXPECT_EQ(ValueType::kMergeOperand, t); + return Decision::kKeep; + } else if (k == "c") { + return Decision::kKeep; + } + + ADD_FAILURE(); + return Decision::kKeep; + } + + const char* Name() const override { + return "CompactionIteratorTest.SingleMergeOperand::Filter"; + } + }; + + class SingleMergeOp : public MergeOperator { + public: + bool FullMergeV2(const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const override { + // See InitIterators() call below for why "c" is the only key for which + // FullMergeV2 should be called. + EXPECT_EQ("c", merge_in.key.ToString()); + + std::string temp_value; + if (merge_in.existing_value != nullptr) { + temp_value = merge_in.existing_value->ToString(); + } + + for (auto& operand : merge_in.operand_list) { + temp_value.append(operand.ToString()); + } + merge_out->new_value = temp_value; + + return true; + } + + bool PartialMergeMulti(const Slice& key, + const std::deque& operand_list, + std::string* new_value, + Logger* logger) const override { + std::string string_key = key.ToString(); + EXPECT_TRUE(string_key == "a" || string_key == "b"); + + if (string_key == "a") { + EXPECT_EQ(1, operand_list.size()); + } else if (string_key == "b") { + EXPECT_EQ(2, operand_list.size()); + } + + std::string temp_value; + for (auto& operand : operand_list) { + temp_value.append(operand.ToString()); + } + swap(temp_value, *new_value); + + return true; + } + + const char* Name() const override { + return "CompactionIteratorTest SingleMergeOp"; + } + + bool AllowSingleOperand() const override { return true; } + }; + + SingleMergeOp merge_op; + Filter filter; + InitIterators( + // a should invoke PartialMergeMulti with a single merge operand. + {test::KeyStr("a", 50, kTypeMerge), + // b should invoke PartialMergeMulti with two operands. + test::KeyStr("b", 70, kTypeMerge), test::KeyStr("b", 60, kTypeMerge), + // c should invoke FullMerge due to kTypeValue at the beginning. + test::KeyStr("c", 90, kTypeMerge), test::KeyStr("c", 80, kTypeValue)}, + {"av1", "bv2", "bv1", "cv2", "cv1"}, {}, {}, kMaxSequenceNumber, + &merge_op, &filter); + + c_iter_->SeekToFirst(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ(test::KeyStr("a", 50, kTypeMerge), c_iter_->key().ToString()); + ASSERT_EQ("av1", c_iter_->value().ToString()); + c_iter_->Next(); + ASSERT_TRUE(c_iter_->Valid()); + ASSERT_EQ("bv1bv2", c_iter_->value().ToString()); + c_iter_->Next(); + ASSERT_EQ("cv1cv2", c_iter_->value().ToString()); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/compaction_job.cc b/db/compaction_job.cc new file mode 100644 index 00000000000..1d023ca4563 --- /dev/null +++ b/db/compaction_job.cc @@ -0,0 +1,1471 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/compaction_job.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "db/builder.h" +#include "db/db_iter.h" +#include "db/dbformat.h" +#include "db/event_helpers.h" +#include "db/log_reader.h" +#include "db/log_writer.h" +#include "db/memtable.h" +#include "db/memtable_list.h" +#include "db/merge_context.h" +#include "db/merge_helper.h" +#include "db/version_set.h" +#include "monitoring/iostats_context_imp.h" +#include "monitoring/perf_context_imp.h" +#include "monitoring/thread_status_util.h" +#include "port/likely.h" +#include "port/port.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/statistics.h" +#include "rocksdb/status.h" +#include "rocksdb/table.h" +#include "table/block.h" +#include "table/block_based_table_factory.h" +#include "table/merging_iterator.h" +#include "table/table_builder.h" +#include "util/coding.h" +#include "util/file_reader_writer.h" +#include "util/filename.h" +#include "util/log_buffer.h" +#include "util/logging.h" +#include "util/mutexlock.h" +#include "util/random.h" +#include "util/sst_file_manager_impl.h" +#include "util/stop_watch.h" +#include "util/string_util.h" +#include "util/sync_point.h" + +namespace rocksdb { + +// Maintains state for each sub-compaction +struct CompactionJob::SubcompactionState { + const Compaction* compaction; + std::unique_ptr c_iter; + + // The boundaries of the key-range this compaction is interested in. No two + // subcompactions may have overlapping key-ranges. + // 'start' is inclusive, 'end' is exclusive, and nullptr means unbounded + Slice *start, *end; + + // The return status of this subcompaction + Status status; + + // Files produced by this subcompaction + struct Output { + FileMetaData meta; + bool finished; + std::shared_ptr table_properties; + }; + + // State kept for output being generated + std::vector outputs; + std::unique_ptr outfile; + std::unique_ptr builder; + Output* current_output() { + if (outputs.empty()) { + // This subcompaction's outptut could be empty if compaction was aborted + // before this subcompaction had a chance to generate any output files. + // When subcompactions are executed sequentially this is more likely and + // will be particulalry likely for the later subcompactions to be empty. + // Once they are run in parallel however it should be much rarer. + return nullptr; + } else { + return &outputs.back(); + } + } + + uint64_t current_output_file_size; + + // State during the subcompaction + uint64_t total_bytes; + uint64_t num_input_records; + uint64_t num_output_records; + CompactionJobStats compaction_job_stats; + uint64_t approx_size; + // An index that used to speed up ShouldStopBefore(). + size_t grandparent_index = 0; + // The number of bytes overlapping between the current output and + // grandparent files used in ShouldStopBefore(). + uint64_t overlapped_bytes = 0; + // A flag determine whether the key has been seen in ShouldStopBefore() + bool seen_key = false; + std::string compression_dict; + + SubcompactionState(Compaction* c, Slice* _start, Slice* _end, + uint64_t size = 0) + : compaction(c), + start(_start), + end(_end), + outfile(nullptr), + builder(nullptr), + current_output_file_size(0), + total_bytes(0), + num_input_records(0), + num_output_records(0), + approx_size(size), + grandparent_index(0), + overlapped_bytes(0), + seen_key(false), + compression_dict() { + assert(compaction != nullptr); + } + + SubcompactionState(SubcompactionState&& o) { *this = std::move(o); } + + SubcompactionState& operator=(SubcompactionState&& o) { + compaction = std::move(o.compaction); + start = std::move(o.start); + end = std::move(o.end); + status = std::move(o.status); + outputs = std::move(o.outputs); + outfile = std::move(o.outfile); + builder = std::move(o.builder); + current_output_file_size = std::move(o.current_output_file_size); + total_bytes = std::move(o.total_bytes); + num_input_records = std::move(o.num_input_records); + num_output_records = std::move(o.num_output_records); + compaction_job_stats = std::move(o.compaction_job_stats); + approx_size = std::move(o.approx_size); + grandparent_index = std::move(o.grandparent_index); + overlapped_bytes = std::move(o.overlapped_bytes); + seen_key = std::move(o.seen_key); + compression_dict = std::move(o.compression_dict); + return *this; + } + + // Because member unique_ptrs do not have these. + SubcompactionState(const SubcompactionState&) = delete; + + SubcompactionState& operator=(const SubcompactionState&) = delete; + + // Returns true iff we should stop building the current output + // before processing "internal_key". + bool ShouldStopBefore(const Slice& internal_key, uint64_t curr_file_size) { + const InternalKeyComparator* icmp = + &compaction->column_family_data()->internal_comparator(); + const std::vector& grandparents = compaction->grandparents(); + + // Scan to find earliest grandparent file that contains key. + while (grandparent_index < grandparents.size() && + icmp->Compare(internal_key, + grandparents[grandparent_index]->largest.Encode()) > + 0) { + if (seen_key) { + overlapped_bytes += grandparents[grandparent_index]->fd.GetFileSize(); + } + assert(grandparent_index + 1 >= grandparents.size() || + icmp->Compare( + grandparents[grandparent_index]->largest.Encode(), + grandparents[grandparent_index + 1]->smallest.Encode()) <= 0); + grandparent_index++; + } + seen_key = true; + + if (overlapped_bytes + curr_file_size > + compaction->max_compaction_bytes()) { + // Too much overlap for current output; start new output + overlapped_bytes = 0; + return true; + } + + return false; + } +}; + +// Maintains state for the entire compaction +struct CompactionJob::CompactionState { + Compaction* const compaction; + + // REQUIRED: subcompaction states are stored in order of increasing + // key-range + std::vector sub_compact_states; + Status status; + + uint64_t total_bytes; + uint64_t num_input_records; + uint64_t num_output_records; + + explicit CompactionState(Compaction* c) + : compaction(c), + total_bytes(0), + num_input_records(0), + num_output_records(0) {} + + size_t NumOutputFiles() { + size_t total = 0; + for (auto& s : sub_compact_states) { + total += s.outputs.size(); + } + return total; + } + + Slice SmallestUserKey() { + for (const auto& sub_compact_state : sub_compact_states) { + if (!sub_compact_state.outputs.empty() && + sub_compact_state.outputs[0].finished) { + return sub_compact_state.outputs[0].meta.smallest.user_key(); + } + } + // If there is no finished output, return an empty slice. + return Slice(nullptr, 0); + } + + Slice LargestUserKey() { + for (auto it = sub_compact_states.rbegin(); it < sub_compact_states.rend(); + ++it) { + if (!it->outputs.empty() && it->current_output()->finished) { + assert(it->current_output() != nullptr); + return it->current_output()->meta.largest.user_key(); + } + } + // If there is no finished output, return an empty slice. + return Slice(nullptr, 0); + } +}; + +void CompactionJob::AggregateStatistics() { + for (SubcompactionState& sc : compact_->sub_compact_states) { + compact_->total_bytes += sc.total_bytes; + compact_->num_input_records += sc.num_input_records; + compact_->num_output_records += sc.num_output_records; + } + if (compaction_job_stats_) { + for (SubcompactionState& sc : compact_->sub_compact_states) { + compaction_job_stats_->Add(sc.compaction_job_stats); + } + } +} + +CompactionJob::CompactionJob( + int job_id, Compaction* compaction, const ImmutableDBOptions& db_options, + const EnvOptions& env_options, VersionSet* versions, + const std::atomic* shutting_down, LogBuffer* log_buffer, + Directory* db_directory, Directory* output_directory, Statistics* stats, + InstrumentedMutex* db_mutex, Status* db_bg_error, + std::vector existing_snapshots, + SequenceNumber earliest_write_conflict_snapshot, + std::shared_ptr table_cache, EventLogger* event_logger, + bool paranoid_file_checks, bool measure_io_stats, const std::string& dbname, + CompactionJobStats* compaction_job_stats) + : job_id_(job_id), + compact_(new CompactionState(compaction)), + compaction_job_stats_(compaction_job_stats), + compaction_stats_(1), + dbname_(dbname), + db_options_(db_options), + env_options_(env_options), + env_(db_options.env), + versions_(versions), + shutting_down_(shutting_down), + log_buffer_(log_buffer), + db_directory_(db_directory), + output_directory_(output_directory), + stats_(stats), + db_mutex_(db_mutex), + db_bg_error_(db_bg_error), + existing_snapshots_(std::move(existing_snapshots)), + earliest_write_conflict_snapshot_(earliest_write_conflict_snapshot), + table_cache_(std::move(table_cache)), + event_logger_(event_logger), + paranoid_file_checks_(paranoid_file_checks), + measure_io_stats_(measure_io_stats) { + assert(log_buffer_ != nullptr); + const auto* cfd = compact_->compaction->column_family_data(); + ThreadStatusUtil::SetColumnFamily(cfd, cfd->ioptions()->env, + db_options_.enable_thread_tracking); + ThreadStatusUtil::SetThreadOperation(ThreadStatus::OP_COMPACTION); + ReportStartedCompaction(compaction); +} + +CompactionJob::~CompactionJob() { + assert(compact_ == nullptr); + ThreadStatusUtil::ResetThreadStatus(); +} + +void CompactionJob::ReportStartedCompaction( + Compaction* compaction) { + const auto* cfd = compact_->compaction->column_family_data(); + ThreadStatusUtil::SetColumnFamily(cfd, cfd->ioptions()->env, + db_options_.enable_thread_tracking); + + ThreadStatusUtil::SetThreadOperationProperty( + ThreadStatus::COMPACTION_JOB_ID, + job_id_); + + ThreadStatusUtil::SetThreadOperationProperty( + ThreadStatus::COMPACTION_INPUT_OUTPUT_LEVEL, + (static_cast(compact_->compaction->start_level()) << 32) + + compact_->compaction->output_level()); + + // In the current design, a CompactionJob is always created + // for non-trivial compaction. + assert(compaction->IsTrivialMove() == false || + compaction->is_manual_compaction() == true); + + ThreadStatusUtil::SetThreadOperationProperty( + ThreadStatus::COMPACTION_PROP_FLAGS, + compaction->is_manual_compaction() + + (compaction->deletion_compaction() << 1)); + + ThreadStatusUtil::SetThreadOperationProperty( + ThreadStatus::COMPACTION_TOTAL_INPUT_BYTES, + compaction->CalculateTotalInputSize()); + + IOSTATS_RESET(bytes_written); + IOSTATS_RESET(bytes_read); + ThreadStatusUtil::SetThreadOperationProperty( + ThreadStatus::COMPACTION_BYTES_WRITTEN, 0); + ThreadStatusUtil::SetThreadOperationProperty( + ThreadStatus::COMPACTION_BYTES_READ, 0); + + // Set the thread operation after operation properties + // to ensure GetThreadList() can always show them all together. + ThreadStatusUtil::SetThreadOperation( + ThreadStatus::OP_COMPACTION); + + if (compaction_job_stats_) { + compaction_job_stats_->is_manual_compaction = + compaction->is_manual_compaction(); + } +} + +void CompactionJob::Prepare() { + AutoThreadOperationStageUpdater stage_updater( + ThreadStatus::STAGE_COMPACTION_PREPARE); + + // Generate file_levels_ for compaction berfore making Iterator + auto* c = compact_->compaction; + assert(c->column_family_data() != nullptr); + assert(c->column_family_data()->current()->storage_info() + ->NumLevelFiles(compact_->compaction->level()) > 0); + + // Is this compaction producing files at the bottommost level? + bottommost_level_ = c->bottommost_level(); + + if (c->ShouldFormSubcompactions()) { + const uint64_t start_micros = env_->NowMicros(); + GenSubcompactionBoundaries(); + MeasureTime(stats_, SUBCOMPACTION_SETUP_TIME, + env_->NowMicros() - start_micros); + + assert(sizes_.size() == boundaries_.size() + 1); + + for (size_t i = 0; i <= boundaries_.size(); i++) { + Slice* start = i == 0 ? nullptr : &boundaries_[i - 1]; + Slice* end = i == boundaries_.size() ? nullptr : &boundaries_[i]; + compact_->sub_compact_states.emplace_back(c, start, end, sizes_[i]); + } + MeasureTime(stats_, NUM_SUBCOMPACTIONS_SCHEDULED, + compact_->sub_compact_states.size()); + } else { + compact_->sub_compact_states.emplace_back(c, nullptr, nullptr); + } +} + +struct RangeWithSize { + Range range; + uint64_t size; + + RangeWithSize(const Slice& a, const Slice& b, uint64_t s = 0) + : range(a, b), size(s) {} +}; + +// Generates a histogram representing potential divisions of key ranges from +// the input. It adds the starting and/or ending keys of certain input files +// to the working set and then finds the approximate size of data in between +// each consecutive pair of slices. Then it divides these ranges into +// consecutive groups such that each group has a similar size. +void CompactionJob::GenSubcompactionBoundaries() { + auto* c = compact_->compaction; + auto* cfd = c->column_family_data(); + const Comparator* cfd_comparator = cfd->user_comparator(); + std::vector bounds; + int start_lvl = c->start_level(); + int out_lvl = c->output_level(); + + // Add the starting and/or ending key of certain input files as a potential + // boundary + for (size_t lvl_idx = 0; lvl_idx < c->num_input_levels(); lvl_idx++) { + int lvl = c->level(lvl_idx); + if (lvl >= start_lvl && lvl <= out_lvl) { + const LevelFilesBrief* flevel = c->input_levels(lvl_idx); + size_t num_files = flevel->num_files; + + if (num_files == 0) { + continue; + } + + if (lvl == 0) { + // For level 0 add the starting and ending key of each file since the + // files may have greatly differing key ranges (not range-partitioned) + for (size_t i = 0; i < num_files; i++) { + bounds.emplace_back(flevel->files[i].smallest_key); + bounds.emplace_back(flevel->files[i].largest_key); + } + } else { + // For all other levels add the smallest/largest key in the level to + // encompass the range covered by that level + bounds.emplace_back(flevel->files[0].smallest_key); + bounds.emplace_back(flevel->files[num_files - 1].largest_key); + if (lvl == out_lvl) { + // For the last level include the starting keys of all files since + // the last level is the largest and probably has the widest key + // range. Since it's range partitioned, the ending key of one file + // and the starting key of the next are very close (or identical). + for (size_t i = 1; i < num_files; i++) { + bounds.emplace_back(flevel->files[i].smallest_key); + } + } + } + } + } + + std::sort(bounds.begin(), bounds.end(), + [cfd_comparator] (const Slice& a, const Slice& b) -> bool { + return cfd_comparator->Compare(ExtractUserKey(a), ExtractUserKey(b)) < 0; + }); + // Remove duplicated entries from bounds + bounds.erase(std::unique(bounds.begin(), bounds.end(), + [cfd_comparator] (const Slice& a, const Slice& b) -> bool { + return cfd_comparator->Compare(ExtractUserKey(a), ExtractUserKey(b)) == 0; + }), bounds.end()); + + // Combine consecutive pairs of boundaries into ranges with an approximate + // size of data covered by keys in that range + uint64_t sum = 0; + std::vector ranges; + auto* v = cfd->current(); + for (auto it = bounds.begin();;) { + const Slice a = *it; + it++; + + if (it == bounds.end()) { + break; + } + + const Slice b = *it; + uint64_t size = versions_->ApproximateSize(v, a, b, start_lvl, out_lvl + 1); + ranges.emplace_back(a, b, size); + sum += size; + } + + // Group the ranges into subcompactions + const double min_file_fill_percent = 4.0 / 5; + uint64_t max_output_files = static_cast( + std::ceil(sum / min_file_fill_percent / + c->mutable_cf_options()->MaxFileSizeForLevel(out_lvl))); + uint64_t subcompactions = + std::min({static_cast(ranges.size()), + static_cast(db_options_.max_subcompactions), + max_output_files}); + + if (subcompactions > 1) { + double mean = sum * 1.0 / subcompactions; + // Greedily add ranges to the subcompaction until the sum of the ranges' + // sizes becomes >= the expected mean size of a subcompaction + sum = 0; + for (size_t i = 0; i < ranges.size() - 1; i++) { + sum += ranges[i].size; + if (subcompactions == 1) { + // If there's only one left to schedule then it goes to the end so no + // need to put an end boundary + continue; + } + if (sum >= mean) { + boundaries_.emplace_back(ExtractUserKey(ranges[i].range.limit)); + sizes_.emplace_back(sum); + subcompactions--; + sum = 0; + } + } + sizes_.emplace_back(sum + ranges.back().size); + } else { + // Only one range so its size is the total sum of sizes computed above + sizes_.emplace_back(sum); + } +} + +Status CompactionJob::Run() { + AutoThreadOperationStageUpdater stage_updater( + ThreadStatus::STAGE_COMPACTION_RUN); + TEST_SYNC_POINT("CompactionJob::Run():Start"); + log_buffer_->FlushBufferToLog(); + LogCompaction(); + + const size_t num_threads = compact_->sub_compact_states.size(); + assert(num_threads > 0); + const uint64_t start_micros = env_->NowMicros(); + + // Launch a thread for each of subcompactions 1...num_threads-1 + std::vector thread_pool; + thread_pool.reserve(num_threads - 1); + for (size_t i = 1; i < compact_->sub_compact_states.size(); i++) { + thread_pool.emplace_back(&CompactionJob::ProcessKeyValueCompaction, this, + &compact_->sub_compact_states[i]); + } + + // Always schedule the first subcompaction (whether or not there are also + // others) in the current thread to be efficient with resources + ProcessKeyValueCompaction(&compact_->sub_compact_states[0]); + + // Wait for all other threads (if there are any) to finish execution + for (auto& thread : thread_pool) { + thread.join(); + } + + if (output_directory_) { + output_directory_->Fsync(); + } + + compaction_stats_.micros = env_->NowMicros() - start_micros; + MeasureTime(stats_, COMPACTION_TIME, compaction_stats_.micros); + + // Check if any thread encountered an error during execution + Status status; + for (const auto& state : compact_->sub_compact_states) { + if (!state.status.ok()) { + status = state.status; + break; + } + } + + TablePropertiesCollection tp; + for (const auto& state : compact_->sub_compact_states) { + for (const auto& output : state.outputs) { + auto fn = TableFileName(db_options_.db_paths, output.meta.fd.GetNumber(), + output.meta.fd.GetPathId()); + tp[fn] = output.table_properties; + } + } + compact_->compaction->SetOutputTableProperties(std::move(tp)); + + // Finish up all book-keeping to unify the subcompaction results + AggregateStatistics(); + UpdateCompactionStats(); + RecordCompactionIOStats(); + LogFlush(db_options_.info_log); + TEST_SYNC_POINT("CompactionJob::Run():End"); + + compact_->status = status; + return status; +} + +Status CompactionJob::Install(const MutableCFOptions& mutable_cf_options) { + AutoThreadOperationStageUpdater stage_updater( + ThreadStatus::STAGE_COMPACTION_INSTALL); + db_mutex_->AssertHeld(); + Status status = compact_->status; + ColumnFamilyData* cfd = compact_->compaction->column_family_data(); + cfd->internal_stats()->AddCompactionStats( + compact_->compaction->output_level(), compaction_stats_); + + if (status.ok()) { + status = InstallCompactionResults(mutable_cf_options); + } + VersionStorageInfo::LevelSummaryStorage tmp; + auto vstorage = cfd->current()->storage_info(); + const auto& stats = compaction_stats_; + + double read_write_amp = 0.0; + double write_amp = 0.0; + double bytes_read_per_sec = 0; + double bytes_written_per_sec = 0; + + if (stats.bytes_read_non_output_levels > 0) { + read_write_amp = (stats.bytes_written + stats.bytes_read_output_level + + stats.bytes_read_non_output_levels) / + static_cast(stats.bytes_read_non_output_levels); + write_amp = stats.bytes_written / + static_cast(stats.bytes_read_non_output_levels); + } + if (stats.micros > 0) { + bytes_read_per_sec = + (stats.bytes_read_non_output_levels + stats.bytes_read_output_level) / + static_cast(stats.micros); + bytes_written_per_sec = + stats.bytes_written / static_cast(stats.micros); + } + + ROCKS_LOG_BUFFER( + log_buffer_, + "[%s] compacted to: %s, MB/sec: %.1f rd, %.1f wr, level %d, " + "files in(%d, %d) out(%d) " + "MB in(%.1f, %.1f) out(%.1f), read-write-amplify(%.1f) " + "write-amplify(%.1f) %s, records in: %d, records dropped: %d\n", + cfd->GetName().c_str(), vstorage->LevelSummary(&tmp), bytes_read_per_sec, + bytes_written_per_sec, compact_->compaction->output_level(), + stats.num_input_files_in_non_output_levels, + stats.num_input_files_in_output_level, stats.num_output_files, + stats.bytes_read_non_output_levels / 1048576.0, + stats.bytes_read_output_level / 1048576.0, + stats.bytes_written / 1048576.0, read_write_amp, write_amp, + status.ToString().c_str(), stats.num_input_records, + stats.num_dropped_records); + + UpdateCompactionJobStats(stats); + + auto stream = event_logger_->LogToBuffer(log_buffer_); + stream << "job" << job_id_ + << "event" << "compaction_finished" + << "compaction_time_micros" << compaction_stats_.micros + << "output_level" << compact_->compaction->output_level() + << "num_output_files" << compact_->NumOutputFiles() + << "total_output_size" << compact_->total_bytes + << "num_input_records" << compact_->num_input_records + << "num_output_records" << compact_->num_output_records + << "num_subcompactions" << compact_->sub_compact_states.size(); + + if (compaction_job_stats_ != nullptr) { + stream << "num_single_delete_mismatches" + << compaction_job_stats_->num_single_del_mismatch; + stream << "num_single_delete_fallthrough" + << compaction_job_stats_->num_single_del_fallthru; + } + + if (measure_io_stats_ && compaction_job_stats_ != nullptr) { + stream << "file_write_nanos" << compaction_job_stats_->file_write_nanos; + stream << "file_range_sync_nanos" + << compaction_job_stats_->file_range_sync_nanos; + stream << "file_fsync_nanos" << compaction_job_stats_->file_fsync_nanos; + stream << "file_prepare_write_nanos" + << compaction_job_stats_->file_prepare_write_nanos; + } + + stream << "lsm_state"; + stream.StartArray(); + for (int level = 0; level < vstorage->num_levels(); ++level) { + stream << vstorage->NumLevelFiles(level); + } + stream.EndArray(); + + CleanupCompaction(); + return status; +} + +void CompactionJob::ProcessKeyValueCompaction(SubcompactionState* sub_compact) { + assert(sub_compact != nullptr); + ColumnFamilyData* cfd = sub_compact->compaction->column_family_data(); + std::unique_ptr range_del_agg( + new RangeDelAggregator(cfd->internal_comparator(), existing_snapshots_)); + std::unique_ptr input(versions_->MakeInputIterator( + sub_compact->compaction, range_del_agg.get())); + + AutoThreadOperationStageUpdater stage_updater( + ThreadStatus::STAGE_COMPACTION_PROCESS_KV); + + // I/O measurement variables + PerfLevel prev_perf_level = PerfLevel::kEnableTime; + const uint64_t kRecordStatsEvery = 1000; + uint64_t prev_write_nanos = 0; + uint64_t prev_fsync_nanos = 0; + uint64_t prev_range_sync_nanos = 0; + uint64_t prev_prepare_write_nanos = 0; + if (measure_io_stats_) { + prev_perf_level = GetPerfLevel(); + SetPerfLevel(PerfLevel::kEnableTime); + prev_write_nanos = IOSTATS(write_nanos); + prev_fsync_nanos = IOSTATS(fsync_nanos); + prev_range_sync_nanos = IOSTATS(range_sync_nanos); + prev_prepare_write_nanos = IOSTATS(prepare_write_nanos); + } + + const MutableCFOptions* mutable_cf_options = + sub_compact->compaction->mutable_cf_options(); + + // To build compression dictionary, we sample the first output file, assuming + // it'll reach the maximum length, and then use the dictionary for compressing + // subsequent output files. The dictionary may be less than max_dict_bytes if + // the first output file's length is less than the maximum. + const int kSampleLenShift = 6; // 2^6 = 64-byte samples + std::set sample_begin_offsets; + if (bottommost_level_ && + cfd->ioptions()->compression_opts.max_dict_bytes > 0) { + const size_t kMaxSamples = + cfd->ioptions()->compression_opts.max_dict_bytes >> kSampleLenShift; + const size_t kOutFileLen = mutable_cf_options->MaxFileSizeForLevel( + compact_->compaction->output_level()); + if (kOutFileLen != port::kMaxSizet) { + const size_t kOutFileNumSamples = kOutFileLen >> kSampleLenShift; + Random64 generator{versions_->NewFileNumber()}; + for (size_t i = 0; i < kMaxSamples; ++i) { + sample_begin_offsets.insert(generator.Uniform(kOutFileNumSamples) + << kSampleLenShift); + } + } + } + + auto compaction_filter = cfd->ioptions()->compaction_filter; + std::unique_ptr compaction_filter_from_factory = nullptr; + if (compaction_filter == nullptr) { + compaction_filter_from_factory = + sub_compact->compaction->CreateCompactionFilter(); + compaction_filter = compaction_filter_from_factory.get(); + } + MergeHelper merge( + env_, cfd->user_comparator(), cfd->ioptions()->merge_operator, + compaction_filter, db_options_.info_log.get(), + false /* internal key corruption is expected */, + existing_snapshots_.empty() ? 0 : existing_snapshots_.back(), + compact_->compaction->level(), db_options_.statistics.get(), + shutting_down_); + + TEST_SYNC_POINT("CompactionJob::Run():Inprogress"); + + Slice* start = sub_compact->start; + Slice* end = sub_compact->end; + if (start != nullptr) { + IterKey start_iter; + start_iter.SetInternalKey(*start, kMaxSequenceNumber, kValueTypeForSeek); + input->Seek(start_iter.GetInternalKey()); + } else { + input->SeekToFirst(); + } + + // we allow only 1 compaction event listener. Used by blob storage + CompactionEventListener* comp_event_listener = nullptr; +#ifndef ROCKSDB_LITE + for (auto& celitr : cfd->ioptions()->listeners) { + comp_event_listener = celitr->GetCompactionEventListener(); + if (comp_event_listener != nullptr) { + break; + } + } +#endif // ROCKSDB_LITE + + Status status; + sub_compact->c_iter.reset(new CompactionIterator( + input.get(), cfd->user_comparator(), &merge, versions_->LastSequence(), + &existing_snapshots_, earliest_write_conflict_snapshot_, env_, false, + range_del_agg.get(), sub_compact->compaction, compaction_filter, + comp_event_listener, shutting_down_)); + auto c_iter = sub_compact->c_iter.get(); + c_iter->SeekToFirst(); + if (c_iter->Valid() && + sub_compact->compaction->output_level() != 0) { + // ShouldStopBefore() maintains state based on keys processed so far. The + // compaction loop always calls it on the "next" key, thus won't tell it the + // first key. So we do that here. + sub_compact->ShouldStopBefore( + c_iter->key(), sub_compact->current_output_file_size); + } + const auto& c_iter_stats = c_iter->iter_stats(); + auto sample_begin_offset_iter = sample_begin_offsets.cbegin(); + // data_begin_offset and compression_dict are only valid while generating + // dictionary from the first output file. + size_t data_begin_offset = 0; + std::string compression_dict; + compression_dict.reserve(cfd->ioptions()->compression_opts.max_dict_bytes); + + while (status.ok() && !cfd->IsDropped() && c_iter->Valid()) { + // Invariant: c_iter.status() is guaranteed to be OK if c_iter->Valid() + // returns true. + const Slice& key = c_iter->key(); + const Slice& value = c_iter->value(); + + // If an end key (exclusive) is specified, check if the current key is + // >= than it and exit if it is because the iterator is out of its range + if (end != nullptr && + cfd->user_comparator()->Compare(c_iter->user_key(), *end) >= 0) { + break; + } + if (c_iter_stats.num_input_records % kRecordStatsEvery == + kRecordStatsEvery - 1) { + RecordDroppedKeys(c_iter_stats, &sub_compact->compaction_job_stats); + c_iter->ResetRecordCounts(); + RecordCompactionIOStats(); + } + + // Open output file if necessary + if (sub_compact->builder == nullptr) { + status = OpenCompactionOutputFile(sub_compact); + if (!status.ok()) { + break; + } + } + assert(sub_compact->builder != nullptr); + assert(sub_compact->current_output() != nullptr); + sub_compact->builder->Add(key, value); + sub_compact->current_output_file_size = sub_compact->builder->FileSize(); + sub_compact->current_output()->meta.UpdateBoundaries( + key, c_iter->ikey().sequence); + sub_compact->num_output_records++; + + if (sub_compact->outputs.size() == 1) { // first output file + // Check if this key/value overlaps any sample intervals; if so, appends + // overlapping portions to the dictionary. + for (const auto& data_elmt : {key, value}) { + size_t data_end_offset = data_begin_offset + data_elmt.size(); + while (sample_begin_offset_iter != sample_begin_offsets.cend() && + *sample_begin_offset_iter < data_end_offset) { + size_t sample_end_offset = + *sample_begin_offset_iter + (1 << kSampleLenShift); + // Invariant: Because we advance sample iterator while processing the + // data_elmt containing the sample's last byte, the current sample + // cannot end before the current data_elmt. + assert(data_begin_offset < sample_end_offset); + + size_t data_elmt_copy_offset, data_elmt_copy_len; + if (*sample_begin_offset_iter <= data_begin_offset) { + // The sample starts before data_elmt starts, so take bytes starting + // at the beginning of data_elmt. + data_elmt_copy_offset = 0; + } else { + // data_elmt starts before the sample starts, so take bytes starting + // at the below offset into data_elmt. + data_elmt_copy_offset = + *sample_begin_offset_iter - data_begin_offset; + } + if (sample_end_offset <= data_end_offset) { + // The sample ends before data_elmt ends, so take as many bytes as + // needed. + data_elmt_copy_len = + sample_end_offset - (data_begin_offset + data_elmt_copy_offset); + } else { + // data_elmt ends before the sample ends, so take all remaining + // bytes in data_elmt. + data_elmt_copy_len = + data_end_offset - (data_begin_offset + data_elmt_copy_offset); + } + compression_dict.append(&data_elmt.data()[data_elmt_copy_offset], + data_elmt_copy_len); + if (sample_end_offset > data_end_offset) { + // Didn't finish sample. Try to finish it with the next data_elmt. + break; + } + // Next sample may require bytes from same data_elmt. + sample_begin_offset_iter++; + } + data_begin_offset = data_end_offset; + } + } + + // Close output file if it is big enough. Two possibilities determine it's + // time to close it: (1) the current key should be this file's last key, (2) + // the next key should not be in this file. + // + // TODO(aekmekji): determine if file should be closed earlier than this + // during subcompactions (i.e. if output size, estimated by input size, is + // going to be 1.2MB and max_output_file_size = 1MB, prefer to have 0.6MB + // and 0.6MB instead of 1MB and 0.2MB) + bool output_file_ended = false; + Status input_status; + if (sub_compact->compaction->output_level() != 0 && + sub_compact->current_output_file_size >= + sub_compact->compaction->max_output_file_size()) { + // (1) this key terminates the file. For historical reasons, the iterator + // status before advancing will be given to FinishCompactionOutputFile(). + input_status = input->status(); + output_file_ended = true; + } + c_iter->Next(); + if (!output_file_ended && c_iter->Valid() && + sub_compact->compaction->output_level() != 0 && + sub_compact->ShouldStopBefore( + c_iter->key(), sub_compact->current_output_file_size) && + sub_compact->builder != nullptr) { + // (2) this key belongs to the next file. For historical reasons, the + // iterator status after advancing will be given to + // FinishCompactionOutputFile(). + input_status = input->status(); + output_file_ended = true; + } + if (output_file_ended) { + const Slice* next_key = nullptr; + if (c_iter->Valid()) { + next_key = &c_iter->key(); + } + CompactionIterationStats range_del_out_stats; + status = FinishCompactionOutputFile(input_status, sub_compact, + range_del_agg.get(), + &range_del_out_stats, next_key); + RecordDroppedKeys(range_del_out_stats, + &sub_compact->compaction_job_stats); + if (sub_compact->outputs.size() == 1) { + // Use dictionary from first output file for compression of subsequent + // files. + sub_compact->compression_dict = std::move(compression_dict); + } + } + } + + sub_compact->num_input_records = c_iter_stats.num_input_records; + sub_compact->compaction_job_stats.num_input_deletion_records = + c_iter_stats.num_input_deletion_records; + sub_compact->compaction_job_stats.num_corrupt_keys = + c_iter_stats.num_input_corrupt_records; + sub_compact->compaction_job_stats.num_single_del_fallthru = + c_iter_stats.num_single_del_fallthru; + sub_compact->compaction_job_stats.num_single_del_mismatch = + c_iter_stats.num_single_del_mismatch; + sub_compact->compaction_job_stats.total_input_raw_key_bytes += + c_iter_stats.total_input_raw_key_bytes; + sub_compact->compaction_job_stats.total_input_raw_value_bytes += + c_iter_stats.total_input_raw_value_bytes; + + RecordTick(stats_, FILTER_OPERATION_TOTAL_TIME, + c_iter_stats.total_filter_time); + RecordDroppedKeys(c_iter_stats, &sub_compact->compaction_job_stats); + RecordCompactionIOStats(); + + if (status.ok() && (shutting_down_->load(std::memory_order_relaxed) || + cfd->IsDropped())) { + status = Status::ShutdownInProgress( + "Database shutdown or Column family drop during compaction"); + } + if (status.ok()) { + status = input->status(); + } + if (status.ok()) { + status = c_iter->status(); + } + + if (status.ok() && sub_compact->builder == nullptr && + sub_compact->outputs.size() == 0 && + range_del_agg->ShouldAddTombstones(bottommost_level_)) { + // handle subcompaction containing only range deletions + status = OpenCompactionOutputFile(sub_compact); + } + + // Call FinishCompactionOutputFile() even if status is not ok: it needs to + // close the output file. + if (sub_compact->builder != nullptr) { + CompactionIterationStats range_del_out_stats; + Status s = FinishCompactionOutputFile( + status, sub_compact, range_del_agg.get(), &range_del_out_stats); + if (status.ok()) { + status = s; + } + RecordDroppedKeys(range_del_out_stats, &sub_compact->compaction_job_stats); + } + + if (measure_io_stats_) { + sub_compact->compaction_job_stats.file_write_nanos += + IOSTATS(write_nanos) - prev_write_nanos; + sub_compact->compaction_job_stats.file_fsync_nanos += + IOSTATS(fsync_nanos) - prev_fsync_nanos; + sub_compact->compaction_job_stats.file_range_sync_nanos += + IOSTATS(range_sync_nanos) - prev_range_sync_nanos; + sub_compact->compaction_job_stats.file_prepare_write_nanos += + IOSTATS(prepare_write_nanos) - prev_prepare_write_nanos; + if (prev_perf_level != PerfLevel::kEnableTime) { + SetPerfLevel(prev_perf_level); + } + } + + sub_compact->c_iter.reset(); + input.reset(); + sub_compact->status = status; +} + +void CompactionJob::RecordDroppedKeys( + const CompactionIterationStats& c_iter_stats, + CompactionJobStats* compaction_job_stats) { + if (c_iter_stats.num_record_drop_user > 0) { + RecordTick(stats_, COMPACTION_KEY_DROP_USER, + c_iter_stats.num_record_drop_user); + } + if (c_iter_stats.num_record_drop_hidden > 0) { + RecordTick(stats_, COMPACTION_KEY_DROP_NEWER_ENTRY, + c_iter_stats.num_record_drop_hidden); + if (compaction_job_stats) { + compaction_job_stats->num_records_replaced += + c_iter_stats.num_record_drop_hidden; + } + } + if (c_iter_stats.num_record_drop_obsolete > 0) { + RecordTick(stats_, COMPACTION_KEY_DROP_OBSOLETE, + c_iter_stats.num_record_drop_obsolete); + if (compaction_job_stats) { + compaction_job_stats->num_expired_deletion_records += + c_iter_stats.num_record_drop_obsolete; + } + } + if (c_iter_stats.num_record_drop_range_del > 0) { + RecordTick(stats_, COMPACTION_KEY_DROP_RANGE_DEL, + c_iter_stats.num_record_drop_range_del); + } + if (c_iter_stats.num_range_del_drop_obsolete > 0) { + RecordTick(stats_, COMPACTION_RANGE_DEL_DROP_OBSOLETE, + c_iter_stats.num_range_del_drop_obsolete); + } + if (c_iter_stats.num_optimized_del_drop_obsolete > 0) { + RecordTick(stats_, COMPACTION_OPTIMIZED_DEL_DROP_OBSOLETE, + c_iter_stats.num_optimized_del_drop_obsolete); + } +} + +Status CompactionJob::FinishCompactionOutputFile( + const Status& input_status, SubcompactionState* sub_compact, + RangeDelAggregator* range_del_agg, + CompactionIterationStats* range_del_out_stats, + const Slice* next_table_min_key /* = nullptr */) { + AutoThreadOperationStageUpdater stage_updater( + ThreadStatus::STAGE_COMPACTION_SYNC_FILE); + assert(sub_compact != nullptr); + assert(sub_compact->outfile); + assert(sub_compact->builder != nullptr); + assert(sub_compact->current_output() != nullptr); + + uint64_t output_number = sub_compact->current_output()->meta.fd.GetNumber(); + assert(output_number != 0); + + // Check for iterator errors + Status s = input_status; + auto meta = &sub_compact->current_output()->meta; + if (s.ok()) { + Slice lower_bound_guard, upper_bound_guard; + const Slice *lower_bound, *upper_bound; + if (sub_compact->outputs.size() == 1) { + // For the first output table, include range tombstones before the min key + // but after the subcompaction boundary. + lower_bound = sub_compact->start; + } else if (meta->smallest.size() > 0) { + // For subsequent output tables, only include range tombstones from min + // key onwards since the previous file was extended to contain range + // tombstones falling before min key. + lower_bound_guard = meta->smallest.user_key(); + lower_bound = &lower_bound_guard; + } else { + lower_bound = nullptr; + } + if (next_table_min_key != nullptr) { + // This isn't the last file in the subcompaction, so extend until the next + // file starts. + upper_bound_guard = ExtractUserKey(*next_table_min_key); + upper_bound = &upper_bound_guard; + } else { + // This is the last file in the subcompaction, so extend until the + // subcompaction ends. + upper_bound = sub_compact->end; + } + range_del_agg->AddToBuilder(sub_compact->builder.get(), lower_bound, + upper_bound, meta, range_del_out_stats, + bottommost_level_); + } + const uint64_t current_entries = sub_compact->builder->NumEntries(); + meta->marked_for_compaction = sub_compact->builder->NeedCompact(); + if (s.ok()) { + s = sub_compact->builder->Finish(); + } else { + sub_compact->builder->Abandon(); + } + const uint64_t current_bytes = sub_compact->builder->FileSize(); + meta->fd.file_size = current_bytes; + sub_compact->current_output()->finished = true; + sub_compact->total_bytes += current_bytes; + + // Finish and check for file errors + if (s.ok()) { + StopWatch sw(env_, stats_, COMPACTION_OUTFILE_SYNC_MICROS); + s = sub_compact->outfile->Sync(db_options_.use_fsync); + } + if (s.ok()) { + s = sub_compact->outfile->Close(); + } + sub_compact->outfile.reset(); + + if (s.ok() && current_entries == 0) { + // If there is nothing to output, no necessary to generate a sst file. + // This happens when the output level is bottom level, at the same time + // the sub_compact output nothing. + std::string fname = TableFileName( + db_options_.db_paths, meta->fd.GetNumber(), meta->fd.GetPathId()); + env_->DeleteFile(fname); + + // Also need to remove the file from outputs, or it will be added to the + // VersionEdit. + assert(!sub_compact->outputs.empty()); + sub_compact->outputs.pop_back(); + sub_compact->builder.reset(); + sub_compact->current_output_file_size = 0; + return s; + } + + ColumnFamilyData* cfd = sub_compact->compaction->column_family_data(); + TableProperties tp; + if (s.ok() && current_entries > 0) { + // Verify that the table is usable + // We set for_compaction to false and don't OptimizeForCompactionTableRead + // here because this is a special case after we finish the table building + // No matter whether use_direct_io_for_flush_and_compaction is true, + // we will regrad this verification as user reads since the goal is + // to cache it here for further user reads + InternalIterator* iter = cfd->table_cache()->NewIterator( + ReadOptions(), env_options_, cfd->internal_comparator(), meta->fd, + nullptr /* range_del_agg */, nullptr, + cfd->internal_stats()->GetFileReadHist( + compact_->compaction->output_level()), + false); + s = iter->status(); + + if (s.ok() && paranoid_file_checks_) { + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) {} + s = iter->status(); + } + + delete iter; + + // Output to event logger and fire events. + if (s.ok()) { + tp = sub_compact->builder->GetTableProperties(); + sub_compact->current_output()->table_properties = + std::make_shared(tp); + ROCKS_LOG_INFO(db_options_.info_log, + "[%s] [JOB %d] Generated table #%" PRIu64 ": %" PRIu64 + " keys, %" PRIu64 " bytes%s", + cfd->GetName().c_str(), job_id_, output_number, + current_entries, current_bytes, + meta->marked_for_compaction ? " (need compaction)" : ""); + } + } + std::string fname = TableFileName(db_options_.db_paths, meta->fd.GetNumber(), + meta->fd.GetPathId()); + EventHelpers::LogAndNotifyTableFileCreationFinished( + event_logger_, cfd->ioptions()->listeners, dbname_, cfd->GetName(), fname, + job_id_, meta->fd, tp, TableFileCreationReason::kCompaction, s); + +#ifndef ROCKSDB_LITE + // Report new file to SstFileManagerImpl + auto sfm = + static_cast(db_options_.sst_file_manager.get()); + if (sfm && meta->fd.GetPathId() == 0) { + auto fn = TableFileName(cfd->ioptions()->db_paths, meta->fd.GetNumber(), + meta->fd.GetPathId()); + sfm->OnAddFile(fn); + if (sfm->IsMaxAllowedSpaceReached()) { + // TODO(ajkr): should we return OK() if max space was reached by the final + // compaction output file (similarly to how flush works when full)? + s = Status::IOError("Max allowed space was reached"); + TEST_SYNC_POINT( + "CompactionJob::FinishCompactionOutputFile:" + "MaxAllowedSpaceReached"); + InstrumentedMutexLock l(db_mutex_); + if (db_bg_error_->ok()) { + Status new_bg_error = s; + // may temporarily unlock and lock the mutex. + EventHelpers::NotifyOnBackgroundError( + cfd->ioptions()->listeners, BackgroundErrorReason::kCompaction, + &new_bg_error, db_mutex_); + if (!new_bg_error.ok()) { + *db_bg_error_ = new_bg_error; + } + } + } + } +#endif + + sub_compact->builder.reset(); + sub_compact->current_output_file_size = 0; + return s; +} + +Status CompactionJob::InstallCompactionResults( + const MutableCFOptions& mutable_cf_options) { + db_mutex_->AssertHeld(); + + auto* compaction = compact_->compaction; + // paranoia: verify that the files that we started with + // still exist in the current version and in the same original level. + // This ensures that a concurrent compaction did not erroneously + // pick the same files to compact_. + if (!versions_->VerifyCompactionFileConsistency(compaction)) { + Compaction::InputLevelSummaryBuffer inputs_summary; + + ROCKS_LOG_ERROR(db_options_.info_log, "[%s] [JOB %d] Compaction %s aborted", + compaction->column_family_data()->GetName().c_str(), + job_id_, compaction->InputLevelSummary(&inputs_summary)); + return Status::Corruption("Compaction input files inconsistent"); + } + + { + Compaction::InputLevelSummaryBuffer inputs_summary; + ROCKS_LOG_INFO( + db_options_.info_log, "[%s] [JOB %d] Compacted %s => %" PRIu64 " bytes", + compaction->column_family_data()->GetName().c_str(), job_id_, + compaction->InputLevelSummary(&inputs_summary), compact_->total_bytes); + } + + // Add compaction outputs + compaction->AddInputDeletions(compact_->compaction->edit()); + + for (const auto& sub_compact : compact_->sub_compact_states) { + for (const auto& out : sub_compact.outputs) { + compaction->edit()->AddFile(compaction->output_level(), out.meta); + } + } + return versions_->LogAndApply(compaction->column_family_data(), + mutable_cf_options, compaction->edit(), + db_mutex_, db_directory_); +} + +void CompactionJob::RecordCompactionIOStats() { + RecordTick(stats_, COMPACT_READ_BYTES, IOSTATS(bytes_read)); + ThreadStatusUtil::IncreaseThreadOperationProperty( + ThreadStatus::COMPACTION_BYTES_READ, IOSTATS(bytes_read)); + IOSTATS_RESET(bytes_read); + RecordTick(stats_, COMPACT_WRITE_BYTES, IOSTATS(bytes_written)); + ThreadStatusUtil::IncreaseThreadOperationProperty( + ThreadStatus::COMPACTION_BYTES_WRITTEN, IOSTATS(bytes_written)); + IOSTATS_RESET(bytes_written); +} + +Status CompactionJob::OpenCompactionOutputFile( + SubcompactionState* sub_compact) { + assert(sub_compact != nullptr); + assert(sub_compact->builder == nullptr); + // no need to lock because VersionSet::next_file_number_ is atomic + uint64_t file_number = versions_->NewFileNumber(); + std::string fname = TableFileName(db_options_.db_paths, file_number, + sub_compact->compaction->output_path_id()); + // Fire events. + ColumnFamilyData* cfd = sub_compact->compaction->column_family_data(); +#ifndef ROCKSDB_LITE + EventHelpers::NotifyTableFileCreationStarted( + cfd->ioptions()->listeners, dbname_, cfd->GetName(), fname, job_id_, + TableFileCreationReason::kCompaction); +#endif // !ROCKSDB_LITE + // Make the output file + unique_ptr writable_file; + EnvOptions opt_env_opts = + env_->OptimizeForCompactionTableWrite(env_options_, db_options_); + TEST_SYNC_POINT_CALLBACK("CompactionJob::OpenCompactionOutputFile", + &opt_env_opts.use_direct_writes); + Status s = NewWritableFile(env_, fname, &writable_file, opt_env_opts); + if (!s.ok()) { + ROCKS_LOG_ERROR( + db_options_.info_log, + "[%s] [JOB %d] OpenCompactionOutputFiles for table #%" PRIu64 + " fails at NewWritableFile with status %s", + sub_compact->compaction->column_family_data()->GetName().c_str(), + job_id_, file_number, s.ToString().c_str()); + LogFlush(db_options_.info_log); + EventHelpers::LogAndNotifyTableFileCreationFinished( + event_logger_, cfd->ioptions()->listeners, dbname_, cfd->GetName(), + fname, job_id_, FileDescriptor(), TableProperties(), + TableFileCreationReason::kCompaction, s); + return s; + } + + SubcompactionState::Output out; + out.meta.fd = + FileDescriptor(file_number, sub_compact->compaction->output_path_id(), 0); + out.finished = false; + + sub_compact->outputs.push_back(out); + writable_file->SetIOPriority(Env::IO_LOW); + writable_file->SetPreallocationBlockSize(static_cast( + sub_compact->compaction->OutputFilePreallocationSize())); + sub_compact->outfile.reset(new WritableFileWriter( + std::move(writable_file), env_options_, db_options_.statistics.get())); + + // If the Column family flag is to only optimize filters for hits, + // we can skip creating filters if this is the bottommost_level where + // data is going to be found + bool skip_filters = + cfd->ioptions()->optimize_filters_for_hits && bottommost_level_; + + uint64_t output_file_creation_time = + sub_compact->compaction->MaxInputFileCreationTime(); + if (output_file_creation_time == 0) { + int64_t _current_time = 0; + db_options_.env->GetCurrentTime(&_current_time); // ignore error + output_file_creation_time = static_cast(_current_time); + } + + sub_compact->builder.reset(NewTableBuilder( + *cfd->ioptions(), cfd->internal_comparator(), + cfd->int_tbl_prop_collector_factories(), cfd->GetID(), cfd->GetName(), + sub_compact->outfile.get(), sub_compact->compaction->output_compression(), + cfd->ioptions()->compression_opts, + sub_compact->compaction->output_level(), &sub_compact->compression_dict, + skip_filters, output_file_creation_time)); + LogFlush(db_options_.info_log); + return s; +} + +void CompactionJob::CleanupCompaction() { + for (SubcompactionState& sub_compact : compact_->sub_compact_states) { + const auto& sub_status = sub_compact.status; + + if (sub_compact.builder != nullptr) { + // May happen if we get a shutdown call in the middle of compaction + sub_compact.builder->Abandon(); + sub_compact.builder.reset(); + } else { + assert(!sub_status.ok() || sub_compact.outfile == nullptr); + } + for (const auto& out : sub_compact.outputs) { + // If this file was inserted into the table cache then remove + // them here because this compaction was not committed. + if (!sub_status.ok()) { + TableCache::Evict(table_cache_.get(), out.meta.fd.GetNumber()); + } + } + } + delete compact_; + compact_ = nullptr; +} + +#ifndef ROCKSDB_LITE +namespace { +void CopyPrefix( + const Slice& src, size_t prefix_length, std::string* dst) { + assert(prefix_length > 0); + size_t length = src.size() > prefix_length ? prefix_length : src.size(); + dst->assign(src.data(), length); +} +} // namespace + +#endif // !ROCKSDB_LITE + +void CompactionJob::UpdateCompactionStats() { + Compaction* compaction = compact_->compaction; + compaction_stats_.num_input_files_in_non_output_levels = 0; + compaction_stats_.num_input_files_in_output_level = 0; + for (int input_level = 0; + input_level < static_cast(compaction->num_input_levels()); + ++input_level) { + if (compaction->level(input_level) != compaction->output_level()) { + UpdateCompactionInputStatsHelper( + &compaction_stats_.num_input_files_in_non_output_levels, + &compaction_stats_.bytes_read_non_output_levels, + input_level); + } else { + UpdateCompactionInputStatsHelper( + &compaction_stats_.num_input_files_in_output_level, + &compaction_stats_.bytes_read_output_level, + input_level); + } + } + + for (const auto& sub_compact : compact_->sub_compact_states) { + size_t num_output_files = sub_compact.outputs.size(); + if (sub_compact.builder != nullptr) { + // An error occurred so ignore the last output. + assert(num_output_files > 0); + --num_output_files; + } + compaction_stats_.num_output_files += static_cast(num_output_files); + + for (const auto& out : sub_compact.outputs) { + compaction_stats_.bytes_written += out.meta.fd.file_size; + } + if (sub_compact.num_input_records > sub_compact.num_output_records) { + compaction_stats_.num_dropped_records += + sub_compact.num_input_records - sub_compact.num_output_records; + } + } +} + +void CompactionJob::UpdateCompactionInputStatsHelper( + int* num_files, uint64_t* bytes_read, int input_level) { + const Compaction* compaction = compact_->compaction; + auto num_input_files = compaction->num_input_files(input_level); + *num_files += static_cast(num_input_files); + + for (size_t i = 0; i < num_input_files; ++i) { + const auto* file_meta = compaction->input(input_level, i); + *bytes_read += file_meta->fd.GetFileSize(); + compaction_stats_.num_input_records += + static_cast(file_meta->num_entries); + } +} + +void CompactionJob::UpdateCompactionJobStats( + const InternalStats::CompactionStats& stats) const { +#ifndef ROCKSDB_LITE + if (compaction_job_stats_) { + compaction_job_stats_->elapsed_micros = stats.micros; + + // input information + compaction_job_stats_->total_input_bytes = + stats.bytes_read_non_output_levels + + stats.bytes_read_output_level; + compaction_job_stats_->num_input_records = + compact_->num_input_records; + compaction_job_stats_->num_input_files = + stats.num_input_files_in_non_output_levels + + stats.num_input_files_in_output_level; + compaction_job_stats_->num_input_files_at_output_level = + stats.num_input_files_in_output_level; + + // output information + compaction_job_stats_->total_output_bytes = stats.bytes_written; + compaction_job_stats_->num_output_records = + compact_->num_output_records; + compaction_job_stats_->num_output_files = stats.num_output_files; + + if (compact_->NumOutputFiles() > 0U) { + CopyPrefix( + compact_->SmallestUserKey(), + CompactionJobStats::kMaxPrefixLength, + &compaction_job_stats_->smallest_output_key_prefix); + CopyPrefix( + compact_->LargestUserKey(), + CompactionJobStats::kMaxPrefixLength, + &compaction_job_stats_->largest_output_key_prefix); + } + } +#endif // !ROCKSDB_LITE +} + +void CompactionJob::LogCompaction() { + Compaction* compaction = compact_->compaction; + ColumnFamilyData* cfd = compaction->column_family_data(); + + // Let's check if anything will get logged. Don't prepare all the info if + // we're not logging + if (db_options_.info_log_level <= InfoLogLevel::INFO_LEVEL) { + Compaction::InputLevelSummaryBuffer inputs_summary; + ROCKS_LOG_INFO( + db_options_.info_log, "[%s] [JOB %d] Compacting %s, score %.2f", + cfd->GetName().c_str(), job_id_, + compaction->InputLevelSummary(&inputs_summary), compaction->score()); + char scratch[2345]; + compaction->Summary(scratch, sizeof(scratch)); + ROCKS_LOG_INFO(db_options_.info_log, "[%s] Compaction start summary: %s\n", + cfd->GetName().c_str(), scratch); + // build event logger report + auto stream = event_logger_->Log(); + stream << "job" << job_id_ << "event" + << "compaction_started"; + for (size_t i = 0; i < compaction->num_input_levels(); ++i) { + stream << ("files_L" + ToString(compaction->level(i))); + stream.StartArray(); + for (auto f : *compaction->inputs(i)) { + stream << f->fd.GetNumber(); + } + stream.EndArray(); + } + stream << "score" << compaction->score() << "input_data_size" + << compaction->CalculateTotalInputSize(); + } +} + +} // namespace rocksdb diff --git a/db/compaction_job.h b/db/compaction_job.h new file mode 100644 index 00000000000..6ca5d627a75 --- /dev/null +++ b/db/compaction_job.h @@ -0,0 +1,165 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "db/column_family.h" +#include "db/compaction_iterator.h" +#include "db/dbformat.h" +#include "db/flush_scheduler.h" +#include "db/internal_stats.h" +#include "db/job_context.h" +#include "db/log_writer.h" +#include "db/memtable_list.h" +#include "db/range_del_aggregator.h" +#include "db/version_edit.h" +#include "db/write_controller.h" +#include "db/write_thread.h" +#include "options/db_options.h" +#include "port/port.h" +#include "rocksdb/compaction_filter.h" +#include "rocksdb/compaction_job_stats.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/transaction_log.h" +#include "table/scoped_arena_iterator.h" +#include "util/autovector.h" +#include "util/event_logger.h" +#include "util/stop_watch.h" +#include "util/thread_local.h" + +namespace rocksdb { + +class MemTable; +class TableCache; +class Version; +class VersionEdit; +class VersionSet; +class Arena; + +class CompactionJob { + public: + CompactionJob(int job_id, Compaction* compaction, + const ImmutableDBOptions& db_options, + const EnvOptions& env_options, VersionSet* versions, + const std::atomic* shutting_down, LogBuffer* log_buffer, + Directory* db_directory, Directory* output_directory, + Statistics* stats, InstrumentedMutex* db_mutex, + Status* db_bg_error, + std::vector existing_snapshots, + SequenceNumber earliest_write_conflict_snapshot, + std::shared_ptr table_cache, EventLogger* event_logger, + bool paranoid_file_checks, bool measure_io_stats, + const std::string& dbname, + CompactionJobStats* compaction_job_stats); + + ~CompactionJob(); + + // no copy/move + CompactionJob(CompactionJob&& job) = delete; + CompactionJob(const CompactionJob& job) = delete; + CompactionJob& operator=(const CompactionJob& job) = delete; + + // REQUIRED: mutex held + void Prepare(); + // REQUIRED mutex not held + Status Run(); + + // REQUIRED: mutex held + Status Install(const MutableCFOptions& mutable_cf_options); + + private: + struct SubcompactionState; + + void AggregateStatistics(); + void GenSubcompactionBoundaries(); + + // update the thread status for starting a compaction. + void ReportStartedCompaction(Compaction* compaction); + void AllocateCompactionOutputFileNumbers(); + // Call compaction filter. Then iterate through input and compact the + // kv-pairs + void ProcessKeyValueCompaction(SubcompactionState* sub_compact); + + Status FinishCompactionOutputFile( + const Status& input_status, SubcompactionState* sub_compact, + RangeDelAggregator* range_del_agg, + CompactionIterationStats* range_del_out_stats, + const Slice* next_table_min_key = nullptr); + Status InstallCompactionResults(const MutableCFOptions& mutable_cf_options); + void RecordCompactionIOStats(); + Status OpenCompactionOutputFile(SubcompactionState* sub_compact); + void CleanupCompaction(); + void UpdateCompactionJobStats( + const InternalStats::CompactionStats& stats) const; + void RecordDroppedKeys(const CompactionIterationStats& c_iter_stats, + CompactionJobStats* compaction_job_stats = nullptr); + + void UpdateCompactionStats(); + void UpdateCompactionInputStatsHelper( + int* num_files, uint64_t* bytes_read, int input_level); + + void LogCompaction(); + + int job_id_; + + // CompactionJob state + struct CompactionState; + CompactionState* compact_; + CompactionJobStats* compaction_job_stats_; + InternalStats::CompactionStats compaction_stats_; + + // DBImpl state + const std::string& dbname_; + const ImmutableDBOptions& db_options_; + const EnvOptions& env_options_; + + Env* env_; + VersionSet* versions_; + const std::atomic* shutting_down_; + LogBuffer* log_buffer_; + Directory* db_directory_; + Directory* output_directory_; + Statistics* stats_; + InstrumentedMutex* db_mutex_; + Status* db_bg_error_; + // If there were two snapshots with seq numbers s1 and + // s2 and s1 < s2, and if we find two instances of a key k1 then lies + // entirely within s1 and s2, then the earlier version of k1 can be safely + // deleted because that version is not visible in any snapshot. + std::vector existing_snapshots_; + + // This is the earliest snapshot that could be used for write-conflict + // checking by a transaction. For any user-key newer than this snapshot, we + // should make sure not to remove evidence that a write occurred. + SequenceNumber earliest_write_conflict_snapshot_; + + std::shared_ptr table_cache_; + + EventLogger* event_logger_; + + bool bottommost_level_; + bool paranoid_file_checks_; + bool measure_io_stats_; + // Stores the Slices that designate the boundaries for each subcompaction + std::vector boundaries_; + // Stores the approx size of keys covered in the range of each subcompaction + std::vector sizes_; +}; + +} // namespace rocksdb diff --git a/db/compaction_job_stats_test.cc b/db/compaction_job_stats_test.cc new file mode 100644 index 00000000000..9a8372f5785 --- /dev/null +++ b/db/compaction_job_stats_test.cc @@ -0,0 +1,1047 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "db/db_impl.h" +#include "db/dbformat.h" +#include "db/job_context.h" +#include "db/version_set.h" +#include "db/write_batch_internal.h" +#include "env/mock_env.h" +#include "memtable/hash_linklist_rep.h" +#include "monitoring/statistics.h" +#include "monitoring/thread_status_util.h" +#include "port/stack_trace.h" +#include "rocksdb/cache.h" +#include "rocksdb/compaction_filter.h" +#include "rocksdb/convenience.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/experimental.h" +#include "rocksdb/filter_policy.h" +#include "rocksdb/options.h" +#include "rocksdb/perf_context.h" +#include "rocksdb/slice.h" +#include "rocksdb/slice_transform.h" +#include "rocksdb/table.h" +#include "rocksdb/table_properties.h" +#include "rocksdb/thread_status.h" +#include "rocksdb/utilities/checkpoint.h" +#include "rocksdb/utilities/write_batch_with_index.h" +#include "table/block_based_table_factory.h" +#include "table/mock_table.h" +#include "table/plain_table_factory.h" +#include "table/scoped_arena_iterator.h" +#include "util/compression.h" +#include "util/filename.h" +#include "util/hash.h" +#include "util/logging.h" +#include "util/mutexlock.h" +#include "util/rate_limiter.h" +#include "util/string_util.h" +#include "util/sync_point.h" +#include "util/testharness.h" +#include "util/testutil.h" +#include "utilities/merge_operators.h" + +#if !defined(IOS_CROSS_COMPILE) +#ifndef ROCKSDB_LITE +namespace rocksdb { + +static std::string RandomString(Random* rnd, int len, double ratio) { + std::string r; + test::CompressibleString(rnd, ratio, len, &r); + return r; +} + +std::string Key(uint64_t key, int length) { + const int kBufSize = 1000; + char buf[kBufSize]; + if (length > kBufSize) { + length = kBufSize; + } + snprintf(buf, kBufSize, "%0*" PRIu64, length, key); + return std::string(buf); +} + +class CompactionJobStatsTest : public testing::Test, + public testing::WithParamInterface { + public: + std::string dbname_; + std::string alternative_wal_dir_; + Env* env_; + DB* db_; + std::vector handles_; + uint32_t max_subcompactions_; + + Options last_options_; + + CompactionJobStatsTest() : env_(Env::Default()) { + env_->SetBackgroundThreads(1, Env::LOW); + env_->SetBackgroundThreads(1, Env::HIGH); + dbname_ = test::TmpDir(env_) + "/compaction_job_stats_test"; + alternative_wal_dir_ = dbname_ + "/wal"; + Options options; + options.create_if_missing = true; + max_subcompactions_ = GetParam(); + options.max_subcompactions = max_subcompactions_; + auto delete_options = options; + delete_options.wal_dir = alternative_wal_dir_; + EXPECT_OK(DestroyDB(dbname_, delete_options)); + // Destroy it for not alternative WAL dir is used. + EXPECT_OK(DestroyDB(dbname_, options)); + db_ = nullptr; + Reopen(options); + } + + ~CompactionJobStatsTest() { + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + rocksdb::SyncPoint::GetInstance()->LoadDependency({}); + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); + Close(); + Options options; + options.db_paths.emplace_back(dbname_, 0); + options.db_paths.emplace_back(dbname_ + "_2", 0); + options.db_paths.emplace_back(dbname_ + "_3", 0); + options.db_paths.emplace_back(dbname_ + "_4", 0); + EXPECT_OK(DestroyDB(dbname_, options)); + } + + // Required if inheriting from testing::WithParamInterface<> + static void SetUpTestCase() {} + static void TearDownTestCase() {} + + DBImpl* dbfull() { + return reinterpret_cast(db_); + } + + void CreateColumnFamilies(const std::vector& cfs, + const Options& options) { + ColumnFamilyOptions cf_opts(options); + size_t cfi = handles_.size(); + handles_.resize(cfi + cfs.size()); + for (auto cf : cfs) { + ASSERT_OK(db_->CreateColumnFamily(cf_opts, cf, &handles_[cfi++])); + } + } + + void CreateAndReopenWithCF(const std::vector& cfs, + const Options& options) { + CreateColumnFamilies(cfs, options); + std::vector cfs_plus_default = cfs; + cfs_plus_default.insert(cfs_plus_default.begin(), kDefaultColumnFamilyName); + ReopenWithColumnFamilies(cfs_plus_default, options); + } + + void ReopenWithColumnFamilies(const std::vector& cfs, + const std::vector& options) { + ASSERT_OK(TryReopenWithColumnFamilies(cfs, options)); + } + + void ReopenWithColumnFamilies(const std::vector& cfs, + const Options& options) { + ASSERT_OK(TryReopenWithColumnFamilies(cfs, options)); + } + + Status TryReopenWithColumnFamilies( + const std::vector& cfs, + const std::vector& options) { + Close(); + EXPECT_EQ(cfs.size(), options.size()); + std::vector column_families; + for (size_t i = 0; i < cfs.size(); ++i) { + column_families.push_back(ColumnFamilyDescriptor(cfs[i], options[i])); + } + DBOptions db_opts = DBOptions(options[0]); + return DB::Open(db_opts, dbname_, column_families, &handles_, &db_); + } + + Status TryReopenWithColumnFamilies(const std::vector& cfs, + const Options& options) { + Close(); + std::vector v_opts(cfs.size(), options); + return TryReopenWithColumnFamilies(cfs, v_opts); + } + + void Reopen(const Options& options) { + ASSERT_OK(TryReopen(options)); + } + + void Close() { + for (auto h : handles_) { + delete h; + } + handles_.clear(); + delete db_; + db_ = nullptr; + } + + void DestroyAndReopen(const Options& options) { + // Destroy using last options + Destroy(last_options_); + ASSERT_OK(TryReopen(options)); + } + + void Destroy(const Options& options) { + Close(); + ASSERT_OK(DestroyDB(dbname_, options)); + } + + Status ReadOnlyReopen(const Options& options) { + return DB::OpenForReadOnly(options, dbname_, &db_); + } + + Status TryReopen(const Options& options) { + Close(); + last_options_ = options; + return DB::Open(options, dbname_, &db_); + } + + Status Flush(int cf = 0) { + if (cf == 0) { + return db_->Flush(FlushOptions()); + } else { + return db_->Flush(FlushOptions(), handles_[cf]); + } + } + + Status Put(const Slice& k, const Slice& v, WriteOptions wo = WriteOptions()) { + return db_->Put(wo, k, v); + } + + Status Put(int cf, const Slice& k, const Slice& v, + WriteOptions wo = WriteOptions()) { + return db_->Put(wo, handles_[cf], k, v); + } + + Status Delete(const std::string& k) { + return db_->Delete(WriteOptions(), k); + } + + Status Delete(int cf, const std::string& k) { + return db_->Delete(WriteOptions(), handles_[cf], k); + } + + std::string Get(const std::string& k, const Snapshot* snapshot = nullptr) { + ReadOptions options; + options.verify_checksums = true; + options.snapshot = snapshot; + std::string result; + Status s = db_->Get(options, k, &result); + if (s.IsNotFound()) { + result = "NOT_FOUND"; + } else if (!s.ok()) { + result = s.ToString(); + } + return result; + } + + std::string Get(int cf, const std::string& k, + const Snapshot* snapshot = nullptr) { + ReadOptions options; + options.verify_checksums = true; + options.snapshot = snapshot; + std::string result; + Status s = db_->Get(options, handles_[cf], k, &result); + if (s.IsNotFound()) { + result = "NOT_FOUND"; + } else if (!s.ok()) { + result = s.ToString(); + } + return result; + } + + int NumTableFilesAtLevel(int level, int cf = 0) { + std::string property; + if (cf == 0) { + // default cfd + EXPECT_TRUE(db_->GetProperty( + "rocksdb.num-files-at-level" + NumberToString(level), &property)); + } else { + EXPECT_TRUE(db_->GetProperty( + handles_[cf], "rocksdb.num-files-at-level" + NumberToString(level), + &property)); + } + return atoi(property.c_str()); + } + + // Return spread of files per level + std::string FilesPerLevel(int cf = 0) { + int num_levels = + (cf == 0) ? db_->NumberLevels() : db_->NumberLevels(handles_[1]); + std::string result; + size_t last_non_zero_offset = 0; + for (int level = 0; level < num_levels; level++) { + int f = NumTableFilesAtLevel(level, cf); + char buf[100]; + snprintf(buf, sizeof(buf), "%s%d", (level ? "," : ""), f); + result += buf; + if (f > 0) { + last_non_zero_offset = result.size(); + } + } + result.resize(last_non_zero_offset); + return result; + } + + uint64_t Size(const Slice& start, const Slice& limit, int cf = 0) { + Range r(start, limit); + uint64_t size; + if (cf == 0) { + db_->GetApproximateSizes(&r, 1, &size); + } else { + db_->GetApproximateSizes(handles_[1], &r, 1, &size); + } + return size; + } + + void Compact(int cf, const Slice& start, const Slice& limit, + uint32_t target_path_id) { + CompactRangeOptions compact_options; + compact_options.target_path_id = target_path_id; + ASSERT_OK(db_->CompactRange(compact_options, handles_[cf], &start, &limit)); + } + + void Compact(int cf, const Slice& start, const Slice& limit) { + ASSERT_OK( + db_->CompactRange(CompactRangeOptions(), handles_[cf], &start, &limit)); + } + + void Compact(const Slice& start, const Slice& limit) { + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), &start, &limit)); + } + + void TEST_Compact(int level, int cf, const Slice& start, const Slice& limit) { + ASSERT_OK(dbfull()->TEST_CompactRange(level, &start, &limit, handles_[cf], + true /* disallow trivial move */)); + } + + // Do n memtable compactions, each of which produces an sstable + // covering the range [small,large]. + void MakeTables(int n, const std::string& small, const std::string& large, + int cf = 0) { + for (int i = 0; i < n; i++) { + ASSERT_OK(Put(cf, small, "begin")); + ASSERT_OK(Put(cf, large, "end")); + ASSERT_OK(Flush(cf)); + } + } + + static void SetDeletionCompactionStats( + CompactionJobStats *stats, uint64_t input_deletions, + uint64_t expired_deletions, uint64_t records_replaced) { + stats->num_input_deletion_records = input_deletions; + stats->num_expired_deletion_records = expired_deletions; + stats->num_records_replaced = records_replaced; + } + + void MakeTableWithKeyValues( + Random* rnd, uint64_t smallest, uint64_t largest, + int key_size, int value_size, uint64_t interval, + double ratio, int cf = 0) { + for (auto key = smallest; key < largest; key += interval) { + ASSERT_OK(Put(cf, Slice(Key(key, key_size)), + Slice(RandomString(rnd, value_size, ratio)))); + } + ASSERT_OK(Flush(cf)); + } + + // This function behaves with the implicit understanding that two + // rounds of keys are inserted into the database, as per the behavior + // of the DeletionStatsTest. + void SelectivelyDeleteKeys(uint64_t smallest, uint64_t largest, + uint64_t interval, int deletion_interval, int key_size, + uint64_t cutoff_key_num, CompactionJobStats* stats, int cf = 0) { + + // interval needs to be >= 2 so that deletion entries can be inserted + // that are intended to not result in an actual key deletion by using + // an offset of 1 from another existing key + ASSERT_GE(interval, 2); + + uint64_t ctr = 1; + uint32_t deletions_made = 0; + uint32_t num_deleted = 0; + uint32_t num_expired = 0; + for (auto key = smallest; key <= largest; key += interval, ctr++) { + if (ctr % deletion_interval == 0) { + ASSERT_OK(Delete(cf, Key(key, key_size))); + deletions_made++; + num_deleted++; + + if (key > cutoff_key_num) { + num_expired++; + } + } + } + + // Insert some deletions for keys that don't exist that + // are both in and out of the key range + ASSERT_OK(Delete(cf, Key(smallest+1, key_size))); + deletions_made++; + + ASSERT_OK(Delete(cf, Key(smallest-1, key_size))); + deletions_made++; + num_expired++; + + ASSERT_OK(Delete(cf, Key(smallest-9, key_size))); + deletions_made++; + num_expired++; + + ASSERT_OK(Flush(cf)); + SetDeletionCompactionStats(stats, deletions_made, num_expired, + num_deleted); + } +}; + +// An EventListener which helps verify the compaction results in +// test CompactionJobStatsTest. +class CompactionJobStatsChecker : public EventListener { + public: + CompactionJobStatsChecker() + : compression_enabled_(false), verify_next_comp_io_stats_(false) {} + + size_t NumberOfUnverifiedStats() { return expected_stats_.size(); } + + void set_verify_next_comp_io_stats(bool v) { verify_next_comp_io_stats_ = v; } + + // Once a compaction completed, this function will verify the returned + // CompactionJobInfo with the oldest CompactionJobInfo added earlier + // in "expected_stats_" which has not yet being used for verification. + virtual void OnCompactionCompleted(DB *db, const CompactionJobInfo& ci) { + if (verify_next_comp_io_stats_) { + ASSERT_GT(ci.stats.file_write_nanos, 0); + ASSERT_GT(ci.stats.file_range_sync_nanos, 0); + ASSERT_GT(ci.stats.file_fsync_nanos, 0); + ASSERT_GT(ci.stats.file_prepare_write_nanos, 0); + verify_next_comp_io_stats_ = false; + } + + std::lock_guard lock(mutex_); + if (expected_stats_.size()) { + Verify(ci.stats, expected_stats_.front()); + expected_stats_.pop(); + } + } + + // A helper function which verifies whether two CompactionJobStats + // match. The verification of all compaction stats are done by + // ASSERT_EQ except for the total input / output bytes, which we + // use ASSERT_GE and ASSERT_LE with a reasonable bias --- + // 10% in uncompressed case and 20% when compression is used. + virtual void Verify(const CompactionJobStats& current_stats, + const CompactionJobStats& stats) { + // time + ASSERT_GT(current_stats.elapsed_micros, 0U); + + ASSERT_EQ(current_stats.num_input_records, + stats.num_input_records); + ASSERT_EQ(current_stats.num_input_files, + stats.num_input_files); + ASSERT_EQ(current_stats.num_input_files_at_output_level, + stats.num_input_files_at_output_level); + + ASSERT_EQ(current_stats.num_output_records, + stats.num_output_records); + ASSERT_EQ(current_stats.num_output_files, + stats.num_output_files); + + ASSERT_EQ(current_stats.is_manual_compaction, + stats.is_manual_compaction); + + // file size + double kFileSizeBias = compression_enabled_ ? 0.20 : 0.10; + ASSERT_GE(current_stats.total_input_bytes * (1.00 + kFileSizeBias), + stats.total_input_bytes); + ASSERT_LE(current_stats.total_input_bytes, + stats.total_input_bytes * (1.00 + kFileSizeBias)); + ASSERT_GE(current_stats.total_output_bytes * (1.00 + kFileSizeBias), + stats.total_output_bytes); + ASSERT_LE(current_stats.total_output_bytes, + stats.total_output_bytes * (1.00 + kFileSizeBias)); + ASSERT_EQ(current_stats.total_input_raw_key_bytes, + stats.total_input_raw_key_bytes); + ASSERT_EQ(current_stats.total_input_raw_value_bytes, + stats.total_input_raw_value_bytes); + + ASSERT_EQ(current_stats.num_records_replaced, + stats.num_records_replaced); + + ASSERT_EQ(current_stats.num_corrupt_keys, + stats.num_corrupt_keys); + + ASSERT_EQ( + std::string(current_stats.smallest_output_key_prefix), + std::string(stats.smallest_output_key_prefix)); + ASSERT_EQ( + std::string(current_stats.largest_output_key_prefix), + std::string(stats.largest_output_key_prefix)); + } + + // Add an expected compaction stats, which will be used to + // verify the CompactionJobStats returned by the OnCompactionCompleted() + // callback. + void AddExpectedStats(const CompactionJobStats& stats) { + std::lock_guard lock(mutex_); + expected_stats_.push(stats); + } + + void EnableCompression(bool flag) { + compression_enabled_ = flag; + } + + bool verify_next_comp_io_stats() const { return verify_next_comp_io_stats_; } + + private: + std::mutex mutex_; + std::queue expected_stats_; + bool compression_enabled_; + bool verify_next_comp_io_stats_; +}; + +// An EventListener which helps verify the compaction statistics in +// the test DeletionStatsTest. +class CompactionJobDeletionStatsChecker : public CompactionJobStatsChecker { + public: + // Verifies whether two CompactionJobStats match. + void Verify(const CompactionJobStats& current_stats, + const CompactionJobStats& stats) { + ASSERT_EQ( + current_stats.num_input_deletion_records, + stats.num_input_deletion_records); + ASSERT_EQ( + current_stats.num_expired_deletion_records, + stats.num_expired_deletion_records); + ASSERT_EQ( + current_stats.num_records_replaced, + stats.num_records_replaced); + + ASSERT_EQ(current_stats.num_corrupt_keys, + stats.num_corrupt_keys); + } +}; + +namespace { + +uint64_t EstimatedFileSize( + uint64_t num_records, size_t key_size, size_t value_size, + double compression_ratio = 1.0, + size_t block_size = 4096, + int bloom_bits_per_key = 10) { + const size_t kPerKeyOverhead = 8; + const size_t kFooterSize = 512; + + uint64_t data_size = + static_cast( + num_records * (key_size + value_size * compression_ratio + + kPerKeyOverhead)); + + return data_size + kFooterSize + + num_records * bloom_bits_per_key / 8 // filter block + + data_size * (key_size + 8) / block_size; // index block +} + +namespace { + +void CopyPrefix( + const Slice& src, size_t prefix_length, std::string* dst) { + assert(prefix_length > 0); + size_t length = src.size() > prefix_length ? prefix_length : src.size(); + dst->assign(src.data(), length); +} + +} // namespace + +CompactionJobStats NewManualCompactionJobStats( + const std::string& smallest_key, const std::string& largest_key, + size_t num_input_files, size_t num_input_files_at_output_level, + uint64_t num_input_records, size_t key_size, size_t value_size, + size_t num_output_files, uint64_t num_output_records, + double compression_ratio, uint64_t num_records_replaced, + bool is_manual = true) { + CompactionJobStats stats; + stats.Reset(); + + stats.num_input_records = num_input_records; + stats.num_input_files = num_input_files; + stats.num_input_files_at_output_level = num_input_files_at_output_level; + + stats.num_output_records = num_output_records; + stats.num_output_files = num_output_files; + + stats.total_input_bytes = + EstimatedFileSize( + num_input_records / num_input_files, + key_size, value_size, compression_ratio) * num_input_files; + stats.total_output_bytes = + EstimatedFileSize( + num_output_records / num_output_files, + key_size, value_size, compression_ratio) * num_output_files; + stats.total_input_raw_key_bytes = + num_input_records * (key_size + 8); + stats.total_input_raw_value_bytes = + num_input_records * value_size; + + stats.is_manual_compaction = is_manual; + + stats.num_records_replaced = num_records_replaced; + + CopyPrefix(smallest_key, + CompactionJobStats::kMaxPrefixLength, + &stats.smallest_output_key_prefix); + CopyPrefix(largest_key, + CompactionJobStats::kMaxPrefixLength, + &stats.largest_output_key_prefix); + + return stats; +} + +CompressionType GetAnyCompression() { + if (Snappy_Supported()) { + return kSnappyCompression; + } else if (Zlib_Supported()) { + return kZlibCompression; + } else if (BZip2_Supported()) { + return kBZip2Compression; + } else if (LZ4_Supported()) { + return kLZ4Compression; + } else if (XPRESS_Supported()) { + return kXpressCompression; + } + + return kNoCompression; +} + +} // namespace + +TEST_P(CompactionJobStatsTest, CompactionJobStatsTest) { + Random rnd(301); + const int kBufSize = 100; + char buf[kBufSize]; + uint64_t key_base = 100000000l; + // Note: key_base must be multiple of num_keys_per_L0_file + int num_keys_per_L0_file = 100; + const int kTestScale = 8; + const int kKeySize = 10; + const int kValueSize = 1000; + const double kCompressionRatio = 0.5; + double compression_ratio = 1.0; + uint64_t key_interval = key_base / num_keys_per_L0_file; + + // Whenever a compaction completes, this listener will try to + // verify whether the returned CompactionJobStats matches + // what we expect. The expected CompactionJobStats is added + // via AddExpectedStats(). + auto* stats_checker = new CompactionJobStatsChecker(); + Options options; + options.listeners.emplace_back(stats_checker); + options.create_if_missing = true; + // just enough setting to hold off auto-compaction. + options.level0_file_num_compaction_trigger = kTestScale + 1; + options.num_levels = 3; + options.compression = kNoCompression; + options.max_subcompactions = max_subcompactions_; + options.bytes_per_sync = 512 * 1024; + + options.report_bg_io_stats = true; + for (int test = 0; test < 2; ++test) { + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // 1st Phase: generate "num_L0_files" L0 files. + int num_L0_files = 0; + for (uint64_t start_key = key_base; + start_key <= key_base * kTestScale; + start_key += key_base) { + MakeTableWithKeyValues( + &rnd, start_key, start_key + key_base - 1, + kKeySize, kValueSize, key_interval, + compression_ratio, 1); + snprintf(buf, kBufSize, "%d", ++num_L0_files); + ASSERT_EQ(std::string(buf), FilesPerLevel(1)); + } + ASSERT_EQ(ToString(num_L0_files), FilesPerLevel(1)); + + // 2nd Phase: perform L0 -> L1 compaction. + int L0_compaction_count = 6; + int count = 1; + std::string smallest_key; + std::string largest_key; + for (uint64_t start_key = key_base; + start_key <= key_base * L0_compaction_count; + start_key += key_base, count++) { + smallest_key = Key(start_key, 10); + largest_key = Key(start_key + key_base - key_interval, 10); + stats_checker->AddExpectedStats( + NewManualCompactionJobStats( + smallest_key, largest_key, + 1, 0, num_keys_per_L0_file, + kKeySize, kValueSize, + 1, num_keys_per_L0_file, + compression_ratio, 0)); + ASSERT_EQ(stats_checker->NumberOfUnverifiedStats(), 1U); + TEST_Compact(0, 1, smallest_key, largest_key); + snprintf(buf, kBufSize, "%d,%d", num_L0_files - count, count); + ASSERT_EQ(std::string(buf), FilesPerLevel(1)); + } + + // compact two files into one in the last L0 -> L1 compaction + int num_remaining_L0 = num_L0_files - L0_compaction_count; + smallest_key = Key(key_base * (L0_compaction_count + 1), 10); + largest_key = Key(key_base * (kTestScale + 1) - key_interval, 10); + stats_checker->AddExpectedStats( + NewManualCompactionJobStats( + smallest_key, largest_key, + num_remaining_L0, + 0, num_keys_per_L0_file * num_remaining_L0, + kKeySize, kValueSize, + 1, num_keys_per_L0_file * num_remaining_L0, + compression_ratio, 0)); + ASSERT_EQ(stats_checker->NumberOfUnverifiedStats(), 1U); + TEST_Compact(0, 1, smallest_key, largest_key); + + int num_L1_files = num_L0_files - num_remaining_L0 + 1; + num_L0_files = 0; + snprintf(buf, kBufSize, "%d,%d", num_L0_files, num_L1_files); + ASSERT_EQ(std::string(buf), FilesPerLevel(1)); + + // 3rd Phase: generate sparse L0 files (wider key-range, same num of keys) + int sparseness = 2; + for (uint64_t start_key = key_base; + start_key <= key_base * kTestScale; + start_key += key_base * sparseness) { + MakeTableWithKeyValues( + &rnd, start_key, start_key + key_base * sparseness - 1, + kKeySize, kValueSize, + key_base * sparseness / num_keys_per_L0_file, + compression_ratio, 1); + snprintf(buf, kBufSize, "%d,%d", ++num_L0_files, num_L1_files); + ASSERT_EQ(std::string(buf), FilesPerLevel(1)); + } + + // 4th Phase: perform L0 -> L1 compaction again, expect higher write amp + // When subcompactions are enabled, the number of output files increases + // by 1 because multiple threads are consuming the input and generating + // output files without coordinating to see if the output could fit into + // a smaller number of files like it does when it runs sequentially + int num_output_files = options.max_subcompactions > 1 ? 2 : 1; + for (uint64_t start_key = key_base; + num_L0_files > 1; + start_key += key_base * sparseness) { + smallest_key = Key(start_key, 10); + largest_key = + Key(start_key + key_base * sparseness - key_interval, 10); + stats_checker->AddExpectedStats( + NewManualCompactionJobStats( + smallest_key, largest_key, + 3, 2, num_keys_per_L0_file * 3, + kKeySize, kValueSize, + num_output_files, + num_keys_per_L0_file * 2, // 1/3 of the data will be updated. + compression_ratio, + num_keys_per_L0_file)); + ASSERT_EQ(stats_checker->NumberOfUnverifiedStats(), 1U); + Compact(1, smallest_key, largest_key); + if (options.max_subcompactions == 1) { + --num_L1_files; + } + snprintf(buf, kBufSize, "%d,%d", --num_L0_files, num_L1_files); + ASSERT_EQ(std::string(buf), FilesPerLevel(1)); + } + + // 5th Phase: Do a full compaction, which involves in two sub-compactions. + // Here we expect to have 1 L0 files and 4 L1 files + // In the first sub-compaction, we expect L0 compaction. + smallest_key = Key(key_base, 10); + largest_key = Key(key_base * (kTestScale + 1) - key_interval, 10); + stats_checker->AddExpectedStats( + NewManualCompactionJobStats( + Key(key_base * (kTestScale + 1 - sparseness), 10), largest_key, + 2, 1, num_keys_per_L0_file * 3, + kKeySize, kValueSize, + 1, num_keys_per_L0_file * 2, + compression_ratio, + num_keys_per_L0_file)); + ASSERT_EQ(stats_checker->NumberOfUnverifiedStats(), 1U); + Compact(1, smallest_key, largest_key); + + num_L1_files = options.max_subcompactions > 1 ? 7 : 4; + char L1_buf[4]; + snprintf(L1_buf, sizeof(L1_buf), "0,%d", num_L1_files); + std::string L1_files(L1_buf); + ASSERT_EQ(L1_files, FilesPerLevel(1)); + options.compression = GetAnyCompression(); + if (options.compression == kNoCompression) { + break; + } + stats_checker->EnableCompression(true); + compression_ratio = kCompressionRatio; + + for (int i = 0; i < 5; i++) { + ASSERT_OK(Put(1, Slice(Key(key_base + i, 10)), + Slice(RandomString(&rnd, 512 * 1024, 1)))); + } + + ASSERT_OK(Flush(1)); + reinterpret_cast(db_)->TEST_WaitForCompact(); + + stats_checker->set_verify_next_comp_io_stats(true); + std::atomic first_prepare_write(true); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "WritableFileWriter::Append:BeforePrepareWrite", [&](void* arg) { + if (first_prepare_write.load()) { + options.env->SleepForMicroseconds(3); + first_prepare_write.store(false); + } + }); + + std::atomic first_flush(true); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "WritableFileWriter::Flush:BeforeAppend", [&](void* arg) { + if (first_flush.load()) { + options.env->SleepForMicroseconds(3); + first_flush.store(false); + } + }); + + std::atomic first_sync(true); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "WritableFileWriter::SyncInternal:0", [&](void* arg) { + if (first_sync.load()) { + options.env->SleepForMicroseconds(3); + first_sync.store(false); + } + }); + + std::atomic first_range_sync(true); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "WritableFileWriter::RangeSync:0", [&](void* arg) { + if (first_range_sync.load()) { + options.env->SleepForMicroseconds(3); + first_range_sync.store(false); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Compact(1, smallest_key, largest_key); + + ASSERT_TRUE(!stats_checker->verify_next_comp_io_stats()); + ASSERT_TRUE(!first_prepare_write.load()); + ASSERT_TRUE(!first_flush.load()); + ASSERT_TRUE(!first_sync.load()); + ASSERT_TRUE(!first_range_sync.load()); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + } + ASSERT_EQ(stats_checker->NumberOfUnverifiedStats(), 0U); +} + +TEST_P(CompactionJobStatsTest, DeletionStatsTest) { + Random rnd(301); + uint64_t key_base = 100000l; + // Note: key_base must be multiple of num_keys_per_L0_file + int num_keys_per_L0_file = 20; + const int kTestScale = 8; // make sure this is even + const int kKeySize = 10; + const int kValueSize = 100; + double compression_ratio = 1.0; + uint64_t key_interval = key_base / num_keys_per_L0_file; + uint64_t largest_key_num = key_base * (kTestScale + 1) - key_interval; + uint64_t cutoff_key_num = key_base * (kTestScale / 2 + 1) - key_interval; + const std::string smallest_key = Key(key_base - 10, kKeySize); + const std::string largest_key = Key(largest_key_num + 10, kKeySize); + + // Whenever a compaction completes, this listener will try to + // verify whether the returned CompactionJobStats matches + // what we expect. + auto* stats_checker = new CompactionJobDeletionStatsChecker(); + Options options; + options.listeners.emplace_back(stats_checker); + options.create_if_missing = true; + options.level0_file_num_compaction_trigger = kTestScale+1; + options.num_levels = 3; + options.compression = kNoCompression; + options.max_bytes_for_level_multiplier = 2; + options.max_subcompactions = max_subcompactions_; + + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Stage 1: Generate several L0 files and then send them to L2 by + // using CompactRangeOptions and CompactRange(). These files will + // have a strict subset of the keys from the full key-range + for (uint64_t start_key = key_base; + start_key <= key_base * kTestScale / 2; + start_key += key_base) { + MakeTableWithKeyValues( + &rnd, start_key, start_key + key_base - 1, + kKeySize, kValueSize, key_interval, + compression_ratio, 1); + } + + CompactRangeOptions cr_options; + cr_options.change_level = true; + cr_options.target_level = 2; + db_->CompactRange(cr_options, handles_[1], nullptr, nullptr); + ASSERT_GT(NumTableFilesAtLevel(2, 1), 0); + + // Stage 2: Generate files including keys from the entire key range + for (uint64_t start_key = key_base; + start_key <= key_base * kTestScale; + start_key += key_base) { + MakeTableWithKeyValues( + &rnd, start_key, start_key + key_base - 1, + kKeySize, kValueSize, key_interval, + compression_ratio, 1); + } + + // Send these L0 files to L1 + TEST_Compact(0, 1, smallest_key, largest_key); + ASSERT_GT(NumTableFilesAtLevel(1, 1), 0); + + // Add a new record and flush so now there is a L0 file + // with a value too (not just deletions from the next step) + ASSERT_OK(Put(1, Key(key_base-6, kKeySize), "test")); + ASSERT_OK(Flush(1)); + + // Stage 3: Generate L0 files with some deletions so now + // there are files with the same key range in L0, L1, and L2 + int deletion_interval = 3; + CompactionJobStats first_compaction_stats; + SelectivelyDeleteKeys(key_base, largest_key_num, + key_interval, deletion_interval, kKeySize, cutoff_key_num, + &first_compaction_stats, 1); + + stats_checker->AddExpectedStats(first_compaction_stats); + + // Stage 4: Trigger compaction and verify the stats + TEST_Compact(0, 1, smallest_key, largest_key); +} + +namespace { +int GetUniversalCompactionInputUnits(uint32_t num_flushes) { + uint32_t compaction_input_units; + for (compaction_input_units = 1; + num_flushes >= compaction_input_units; + compaction_input_units *= 2) { + if ((num_flushes & compaction_input_units) != 0) { + return compaction_input_units > 1 ? compaction_input_units : 0; + } + } + return 0; +} +} // namespace + +TEST_P(CompactionJobStatsTest, UniversalCompactionTest) { + Random rnd(301); + uint64_t key_base = 100000000l; + // Note: key_base must be multiple of num_keys_per_L0_file + int num_keys_per_table = 100; + const uint32_t kTestScale = 6; + const int kKeySize = 10; + const int kValueSize = 900; + double compression_ratio = 1.0; + uint64_t key_interval = key_base / num_keys_per_table; + + auto* stats_checker = new CompactionJobStatsChecker(); + Options options; + options.listeners.emplace_back(stats_checker); + options.create_if_missing = true; + options.num_levels = 3; + options.compression = kNoCompression; + options.level0_file_num_compaction_trigger = 2; + options.target_file_size_base = num_keys_per_table * 1000; + options.compaction_style = kCompactionStyleUniversal; + options.compaction_options_universal.size_ratio = 1; + options.compaction_options_universal.max_size_amplification_percent = 1000; + options.max_subcompactions = max_subcompactions_; + + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Generates the expected CompactionJobStats for each compaction + for (uint32_t num_flushes = 2; num_flushes <= kTestScale; num_flushes++) { + // Here we treat one newly flushed file as an unit. + // + // For example, if a newly flushed file is 100k, and a compaction has + // 4 input units, then this compaction inputs 400k. + uint32_t num_input_units = GetUniversalCompactionInputUnits(num_flushes); + if (num_input_units == 0) { + continue; + } + // The following statement determines the expected smallest key + // based on whether it is a full compaction. A full compaction only + // happens when the number of flushes equals to the number of compaction + // input runs. + uint64_t smallest_key = + (num_flushes == num_input_units) ? + key_base : key_base * (num_flushes - 1); + + stats_checker->AddExpectedStats( + NewManualCompactionJobStats( + Key(smallest_key, 10), + Key(smallest_key + key_base * num_input_units - key_interval, 10), + num_input_units, + num_input_units > 2 ? num_input_units / 2 : 0, + num_keys_per_table * num_input_units, + kKeySize, kValueSize, + num_input_units, + num_keys_per_table * num_input_units, + 1.0, 0, false)); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_EQ(stats_checker->NumberOfUnverifiedStats(), 3U); + + for (uint64_t start_key = key_base; + start_key <= key_base * kTestScale; + start_key += key_base) { + MakeTableWithKeyValues( + &rnd, start_key, start_key + key_base - 1, + kKeySize, kValueSize, key_interval, + compression_ratio, 1); + reinterpret_cast(db_)->TEST_WaitForCompact(); + } + ASSERT_EQ(stats_checker->NumberOfUnverifiedStats(), 0U); +} + +INSTANTIATE_TEST_CASE_P(CompactionJobStatsTest, CompactionJobStatsTest, + ::testing::Values(1, 4)); +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, "SKIPPED, not supported in ROCKSDB_LITE\n"); + return 0; +} + +#endif // !ROCKSDB_LITE + +#else + +int main(int argc, char** argv) { return 0; } +#endif // !defined(IOS_CROSS_COMPILE) diff --git a/db/compaction_job_test.cc b/db/compaction_job_test.cc new file mode 100644 index 00000000000..cace1814ad8 --- /dev/null +++ b/db/compaction_job_test.cc @@ -0,0 +1,949 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include +#include +#include +#include + +#include "db/column_family.h" +#include "db/compaction_job.h" +#include "db/version_set.h" +#include "rocksdb/cache.h" +#include "rocksdb/db.h" +#include "rocksdb/options.h" +#include "rocksdb/write_buffer_manager.h" +#include "table/mock_table.h" +#include "util/file_reader_writer.h" +#include "util/string_util.h" +#include "util/testharness.h" +#include "util/testutil.h" +#include "utilities/merge_operators.h" + +namespace rocksdb { + +namespace { + +void VerifyInitializationOfCompactionJobStats( + const CompactionJobStats& compaction_job_stats) { +#if !defined(IOS_CROSS_COMPILE) + ASSERT_EQ(compaction_job_stats.elapsed_micros, 0U); + + ASSERT_EQ(compaction_job_stats.num_input_records, 0U); + ASSERT_EQ(compaction_job_stats.num_input_files, 0U); + ASSERT_EQ(compaction_job_stats.num_input_files_at_output_level, 0U); + + ASSERT_EQ(compaction_job_stats.num_output_records, 0U); + ASSERT_EQ(compaction_job_stats.num_output_files, 0U); + + ASSERT_EQ(compaction_job_stats.is_manual_compaction, true); + + ASSERT_EQ(compaction_job_stats.total_input_bytes, 0U); + ASSERT_EQ(compaction_job_stats.total_output_bytes, 0U); + + ASSERT_EQ(compaction_job_stats.total_input_raw_key_bytes, 0U); + ASSERT_EQ(compaction_job_stats.total_input_raw_value_bytes, 0U); + + ASSERT_EQ(compaction_job_stats.smallest_output_key_prefix[0], 0); + ASSERT_EQ(compaction_job_stats.largest_output_key_prefix[0], 0); + + ASSERT_EQ(compaction_job_stats.num_records_replaced, 0U); + + ASSERT_EQ(compaction_job_stats.num_input_deletion_records, 0U); + ASSERT_EQ(compaction_job_stats.num_expired_deletion_records, 0U); + + ASSERT_EQ(compaction_job_stats.num_corrupt_keys, 0U); +#endif // !defined(IOS_CROSS_COMPILE) +} + +} // namespace + +// TODO(icanadi) Make it simpler once we mock out VersionSet +class CompactionJobTest : public testing::Test { + public: + CompactionJobTest() + : env_(Env::Default()), + dbname_(test::TmpDir() + "/compaction_job_test"), + db_options_(), + mutable_cf_options_(cf_options_), + table_cache_(NewLRUCache(50000, 16)), + write_buffer_manager_(db_options_.db_write_buffer_size), + versions_(new VersionSet(dbname_, &db_options_, env_options_, + table_cache_.get(), &write_buffer_manager_, + &write_controller_)), + shutting_down_(false), + mock_table_factory_(new mock::MockTableFactory()) { + EXPECT_OK(env_->CreateDirIfMissing(dbname_)); + db_options_.db_paths.emplace_back(dbname_, + std::numeric_limits::max()); + } + + std::string GenerateFileName(uint64_t file_number) { + FileMetaData meta; + std::vector db_paths; + db_paths.emplace_back(dbname_, std::numeric_limits::max()); + meta.fd = FileDescriptor(file_number, 0, 0); + return TableFileName(db_paths, meta.fd.GetNumber(), meta.fd.GetPathId()); + } + + std::string KeyStr(const std::string& user_key, const SequenceNumber seq_num, + const ValueType t) { + return InternalKey(user_key, seq_num, t).Encode().ToString(); + } + + void AddMockFile(const stl_wrappers::KVMap& contents, int level = 0) { + assert(contents.size() > 0); + + bool first_key = true; + std::string smallest, largest; + InternalKey smallest_key, largest_key; + SequenceNumber smallest_seqno = kMaxSequenceNumber; + SequenceNumber largest_seqno = 0; + for (auto kv : contents) { + ParsedInternalKey key; + std::string skey; + std::string value; + std::tie(skey, value) = kv; + ParseInternalKey(skey, &key); + + smallest_seqno = std::min(smallest_seqno, key.sequence); + largest_seqno = std::max(largest_seqno, key.sequence); + + if (first_key || + cfd_->user_comparator()->Compare(key.user_key, smallest) < 0) { + smallest.assign(key.user_key.data(), key.user_key.size()); + smallest_key.DecodeFrom(skey); + } + if (first_key || + cfd_->user_comparator()->Compare(key.user_key, largest) > 0) { + largest.assign(key.user_key.data(), key.user_key.size()); + largest_key.DecodeFrom(skey); + } + + first_key = false; + } + + uint64_t file_number = versions_->NewFileNumber(); + EXPECT_OK(mock_table_factory_->CreateMockTable( + env_, GenerateFileName(file_number), std::move(contents))); + + VersionEdit edit; + edit.AddFile(level, file_number, 0, 10, smallest_key, largest_key, + smallest_seqno, largest_seqno, false); + + mutex_.Lock(); + versions_->LogAndApply(versions_->GetColumnFamilySet()->GetDefault(), + mutable_cf_options_, &edit, &mutex_); + mutex_.Unlock(); + } + + void SetLastSequence(const SequenceNumber sequence_number) { + versions_->SetLastToBeWrittenSequence(sequence_number + 1); + versions_->SetLastSequence(sequence_number + 1); + } + + // returns expected result after compaction + stl_wrappers::KVMap CreateTwoFiles(bool gen_corrupted_keys) { + auto expected_results = mock::MakeMockFile(); + const int kKeysPerFile = 10000; + const int kCorruptKeysPerFile = 200; + const int kMatchingKeys = kKeysPerFile / 2; + SequenceNumber sequence_number = 0; + + auto corrupt_id = [&](int id) { + return gen_corrupted_keys && id > 0 && id <= kCorruptKeysPerFile; + }; + + for (int i = 0; i < 2; ++i) { + auto contents = mock::MakeMockFile(); + for (int k = 0; k < kKeysPerFile; ++k) { + auto key = ToString(i * kMatchingKeys + k); + auto value = ToString(i * kKeysPerFile + k); + InternalKey internal_key(key, ++sequence_number, kTypeValue); + + // This is how the key will look like once it's written in bottommost + // file + InternalKey bottommost_internal_key( + key, (key == "9999") ? sequence_number : 0, kTypeValue); + + if (corrupt_id(k)) { + test::CorruptKeyType(&internal_key); + test::CorruptKeyType(&bottommost_internal_key); + } + contents.insert({ internal_key.Encode().ToString(), value }); + if (i == 1 || k < kMatchingKeys || corrupt_id(k - kMatchingKeys)) { + expected_results.insert( + { bottommost_internal_key.Encode().ToString(), value }); + } + } + + AddMockFile(contents); + } + + SetLastSequence(sequence_number); + + return expected_results; + } + + void NewDB() { + VersionEdit new_db; + new_db.SetLogNumber(0); + new_db.SetNextFile(2); + new_db.SetLastSequence(0); + + const std::string manifest = DescriptorFileName(dbname_, 1); + unique_ptr file; + Status s = env_->NewWritableFile( + manifest, &file, env_->OptimizeForManifestWrite(env_options_)); + ASSERT_OK(s); + unique_ptr file_writer( + new WritableFileWriter(std::move(file), env_options_)); + { + log::Writer log(std::move(file_writer), 0, false); + std::string record; + new_db.EncodeTo(&record); + s = log.AddRecord(record); + } + ASSERT_OK(s); + // Make "CURRENT" file that points to the new manifest file. + s = SetCurrentFile(env_, dbname_, 1, nullptr); + + std::vector column_families; + cf_options_.table_factory = mock_table_factory_; + cf_options_.merge_operator = merge_op_; + cf_options_.compaction_filter = compaction_filter_.get(); + column_families.emplace_back(kDefaultColumnFamilyName, cf_options_); + + EXPECT_OK(versions_->Recover(column_families, false)); + cfd_ = versions_->GetColumnFamilySet()->GetDefault(); + } + + void RunCompaction( + const std::vector>& input_files, + const stl_wrappers::KVMap& expected_results, + const std::vector& snapshots = {}, + SequenceNumber earliest_write_conflict_snapshot = kMaxSequenceNumber) { + auto cfd = versions_->GetColumnFamilySet()->GetDefault(); + + size_t num_input_files = 0; + std::vector compaction_input_files; + for (size_t level = 0; level < input_files.size(); level++) { + auto level_files = input_files[level]; + CompactionInputFiles compaction_level; + compaction_level.level = static_cast(level); + compaction_level.files.insert(compaction_level.files.end(), + level_files.begin(), level_files.end()); + compaction_input_files.push_back(compaction_level); + num_input_files += level_files.size(); + } + + Compaction compaction(cfd->current()->storage_info(), *cfd->ioptions(), + *cfd->GetLatestMutableCFOptions(), + compaction_input_files, 1, 1024 * 1024, + 10 * 1024 * 1024, 0, kNoCompression, {}, true); + compaction.SetInputVersion(cfd->current()); + + LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, db_options_.info_log.get()); + mutex_.Lock(); + EventLogger event_logger(db_options_.info_log.get()); + CompactionJob compaction_job( + 0, &compaction, db_options_, env_options_, versions_.get(), + &shutting_down_, &log_buffer, nullptr, nullptr, nullptr, &mutex_, + &bg_error_, snapshots, earliest_write_conflict_snapshot, table_cache_, + &event_logger, false, false, dbname_, &compaction_job_stats_); + + VerifyInitializationOfCompactionJobStats(compaction_job_stats_); + + compaction_job.Prepare(); + mutex_.Unlock(); + Status s; + s = compaction_job.Run(); + ASSERT_OK(s); + mutex_.Lock(); + ASSERT_OK(compaction_job.Install(*cfd->GetLatestMutableCFOptions())); + mutex_.Unlock(); + + if (expected_results.size() == 0) { + ASSERT_GE(compaction_job_stats_.elapsed_micros, 0U); + ASSERT_EQ(compaction_job_stats_.num_input_files, num_input_files); + ASSERT_EQ(compaction_job_stats_.num_output_files, 0U); + } else { + ASSERT_GE(compaction_job_stats_.elapsed_micros, 0U); + ASSERT_EQ(compaction_job_stats_.num_input_files, num_input_files); + ASSERT_EQ(compaction_job_stats_.num_output_files, 1U); + mock_table_factory_->AssertLatestFile(expected_results); + } + } + + Env* env_; + std::string dbname_; + EnvOptions env_options_; + ImmutableDBOptions db_options_; + ColumnFamilyOptions cf_options_; + MutableCFOptions mutable_cf_options_; + std::shared_ptr table_cache_; + WriteController write_controller_; + WriteBufferManager write_buffer_manager_; + std::unique_ptr versions_; + InstrumentedMutex mutex_; + std::atomic shutting_down_; + std::shared_ptr mock_table_factory_; + CompactionJobStats compaction_job_stats_; + ColumnFamilyData* cfd_; + std::unique_ptr compaction_filter_; + std::shared_ptr merge_op_; + Status bg_error_; +}; + +TEST_F(CompactionJobTest, Simple) { + NewDB(); + + auto expected_results = CreateTwoFiles(false); + auto cfd = versions_->GetColumnFamilySet()->GetDefault(); + auto files = cfd->current()->storage_info()->LevelFiles(0); + ASSERT_EQ(2U, files.size()); + RunCompaction({ files }, expected_results); +} + +TEST_F(CompactionJobTest, SimpleCorrupted) { + NewDB(); + + auto expected_results = CreateTwoFiles(true); + auto cfd = versions_->GetColumnFamilySet()->GetDefault(); + auto files = cfd->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results); + ASSERT_EQ(compaction_job_stats_.num_corrupt_keys, 400U); +} + +TEST_F(CompactionJobTest, SimpleDeletion) { + NewDB(); + + auto file1 = mock::MakeMockFile({{KeyStr("c", 4U, kTypeDeletion), ""}, + {KeyStr("c", 3U, kTypeValue), "val"}}); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile({{KeyStr("b", 2U, kTypeValue), "val"}, + {KeyStr("b", 1U, kTypeValue), "val"}}); + AddMockFile(file2); + + auto expected_results = + mock::MakeMockFile({{KeyStr("b", 0U, kTypeValue), "val"}}); + + SetLastSequence(4U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results); +} + +TEST_F(CompactionJobTest, OutputNothing) { + NewDB(); + + auto file1 = mock::MakeMockFile({{KeyStr("a", 1U, kTypeValue), "val"}}); + + AddMockFile(file1); + + auto file2 = mock::MakeMockFile({{KeyStr("a", 2U, kTypeDeletion), ""}}); + + AddMockFile(file2); + + auto expected_results = mock::MakeMockFile(); + + SetLastSequence(4U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results); +} + +TEST_F(CompactionJobTest, SimpleOverwrite) { + NewDB(); + + auto file1 = mock::MakeMockFile({ + {KeyStr("a", 3U, kTypeValue), "val2"}, + {KeyStr("b", 4U, kTypeValue), "val3"}, + }); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile({{KeyStr("a", 1U, kTypeValue), "val"}, + {KeyStr("b", 2U, kTypeValue), "val"}}); + AddMockFile(file2); + + auto expected_results = + mock::MakeMockFile({{KeyStr("a", 0U, kTypeValue), "val2"}, + {KeyStr("b", 4U, kTypeValue), "val3"}}); + + SetLastSequence(4U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results); +} + +TEST_F(CompactionJobTest, SimpleNonLastLevel) { + NewDB(); + + auto file1 = mock::MakeMockFile({ + {KeyStr("a", 5U, kTypeValue), "val2"}, + {KeyStr("b", 6U, kTypeValue), "val3"}, + }); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile({{KeyStr("a", 3U, kTypeValue), "val"}, + {KeyStr("b", 4U, kTypeValue), "val"}}); + AddMockFile(file2, 1); + + auto file3 = mock::MakeMockFile({{KeyStr("a", 1U, kTypeValue), "val"}, + {KeyStr("b", 2U, kTypeValue), "val"}}); + AddMockFile(file3, 2); + + // Because level 1 is not the last level, the sequence numbers of a and b + // cannot be set to 0 + auto expected_results = + mock::MakeMockFile({{KeyStr("a", 5U, kTypeValue), "val2"}, + {KeyStr("b", 6U, kTypeValue), "val3"}}); + + SetLastSequence(6U); + auto lvl0_files = cfd_->current()->storage_info()->LevelFiles(0); + auto lvl1_files = cfd_->current()->storage_info()->LevelFiles(1); + RunCompaction({lvl0_files, lvl1_files}, expected_results); +} + +TEST_F(CompactionJobTest, SimpleMerge) { + merge_op_ = MergeOperators::CreateStringAppendOperator(); + NewDB(); + + auto file1 = mock::MakeMockFile({ + {KeyStr("a", 5U, kTypeMerge), "5"}, + {KeyStr("a", 4U, kTypeMerge), "4"}, + {KeyStr("a", 3U, kTypeValue), "3"}, + }); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile( + {{KeyStr("b", 2U, kTypeMerge), "2"}, {KeyStr("b", 1U, kTypeValue), "1"}}); + AddMockFile(file2); + + auto expected_results = + mock::MakeMockFile({{KeyStr("a", 0U, kTypeValue), "3,4,5"}, + {KeyStr("b", 2U, kTypeValue), "1,2"}}); + + SetLastSequence(5U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results); +} + +TEST_F(CompactionJobTest, NonAssocMerge) { + merge_op_ = MergeOperators::CreateStringAppendTESTOperator(); + NewDB(); + + auto file1 = mock::MakeMockFile({ + {KeyStr("a", 5U, kTypeMerge), "5"}, + {KeyStr("a", 4U, kTypeMerge), "4"}, + {KeyStr("a", 3U, kTypeMerge), "3"}, + }); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile( + {{KeyStr("b", 2U, kTypeMerge), "2"}, {KeyStr("b", 1U, kTypeMerge), "1"}}); + AddMockFile(file2); + + auto expected_results = + mock::MakeMockFile({{KeyStr("a", 0U, kTypeValue), "3,4,5"}, + {KeyStr("b", 2U, kTypeMerge), "2"}, + {KeyStr("b", 1U, kTypeMerge), "1"}}); + + SetLastSequence(5U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results); +} + +// Filters merge operands with value 10. +TEST_F(CompactionJobTest, MergeOperandFilter) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + compaction_filter_.reset(new test::FilterNumber(10U)); + NewDB(); + + auto file1 = mock::MakeMockFile( + {{KeyStr("a", 5U, kTypeMerge), test::EncodeInt(5U)}, + {KeyStr("a", 4U, kTypeMerge), test::EncodeInt(10U)}, // Filtered + {KeyStr("a", 3U, kTypeMerge), test::EncodeInt(3U)}}); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile({ + {KeyStr("b", 2U, kTypeMerge), test::EncodeInt(2U)}, + {KeyStr("b", 1U, kTypeMerge), test::EncodeInt(10U)} // Filtered + }); + AddMockFile(file2); + + auto expected_results = + mock::MakeMockFile({{KeyStr("a", 0U, kTypeValue), test::EncodeInt(8U)}, + {KeyStr("b", 2U, kTypeMerge), test::EncodeInt(2U)}}); + + SetLastSequence(5U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results); +} + +TEST_F(CompactionJobTest, FilterSomeMergeOperands) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + compaction_filter_.reset(new test::FilterNumber(10U)); + NewDB(); + + auto file1 = mock::MakeMockFile( + {{KeyStr("a", 5U, kTypeMerge), test::EncodeInt(5U)}, + {KeyStr("a", 4U, kTypeMerge), test::EncodeInt(10U)}, // Filtered + {KeyStr("a", 3U, kTypeValue), test::EncodeInt(5U)}, + {KeyStr("d", 8U, kTypeMerge), test::EncodeInt(10U)}}); + AddMockFile(file1); + + auto file2 = + mock::MakeMockFile({{KeyStr("b", 2U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("b", 1U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("c", 2U, kTypeMerge), test::EncodeInt(3U)}, + {KeyStr("c", 1U, kTypeValue), test::EncodeInt(7U)}, + {KeyStr("d", 1U, kTypeValue), test::EncodeInt(6U)}}); + AddMockFile(file2); + + auto file3 = + mock::MakeMockFile({{KeyStr("a", 1U, kTypeMerge), test::EncodeInt(3U)}}); + AddMockFile(file3, 2); + + auto expected_results = mock::MakeMockFile({ + {KeyStr("a", 5U, kTypeValue), test::EncodeInt(10U)}, + {KeyStr("c", 2U, kTypeValue), test::EncodeInt(10U)}, + {KeyStr("d", 1U, kTypeValue), test::EncodeInt(6U)} + // b does not appear because the operands are filtered + }); + + SetLastSequence(5U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results); +} + +// Test where all operands/merge results are filtered out. +TEST_F(CompactionJobTest, FilterAllMergeOperands) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + compaction_filter_.reset(new test::FilterNumber(10U)); + NewDB(); + + auto file1 = + mock::MakeMockFile({{KeyStr("a", 11U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("a", 10U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("a", 9U, kTypeMerge), test::EncodeInt(10U)}}); + AddMockFile(file1); + + auto file2 = + mock::MakeMockFile({{KeyStr("b", 8U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("b", 7U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("b", 6U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("b", 5U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("b", 4U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("b", 3U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("b", 2U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("c", 2U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("c", 1U, kTypeMerge), test::EncodeInt(10U)}}); + AddMockFile(file2); + + auto file3 = + mock::MakeMockFile({{KeyStr("a", 2U, kTypeMerge), test::EncodeInt(10U)}, + {KeyStr("b", 1U, kTypeMerge), test::EncodeInt(10U)}}); + AddMockFile(file3, 2); + + SetLastSequence(11U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + + stl_wrappers::KVMap empty_map; + RunCompaction({files}, empty_map); +} + +TEST_F(CompactionJobTest, SimpleSingleDelete) { + NewDB(); + + auto file1 = mock::MakeMockFile({ + {KeyStr("a", 5U, kTypeDeletion), ""}, + {KeyStr("b", 6U, kTypeSingleDeletion), ""}, + }); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile({{KeyStr("a", 3U, kTypeValue), "val"}, + {KeyStr("b", 4U, kTypeValue), "val"}}); + AddMockFile(file2); + + auto file3 = mock::MakeMockFile({ + {KeyStr("a", 1U, kTypeValue), "val"}, + }); + AddMockFile(file3, 2); + + auto expected_results = + mock::MakeMockFile({{KeyStr("a", 5U, kTypeDeletion), ""}}); + + SetLastSequence(6U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results); +} + +TEST_F(CompactionJobTest, SingleDeleteSnapshots) { + NewDB(); + + auto file1 = mock::MakeMockFile({ + {KeyStr("A", 12U, kTypeSingleDeletion), ""}, + {KeyStr("a", 12U, kTypeSingleDeletion), ""}, + {KeyStr("b", 21U, kTypeSingleDeletion), ""}, + {KeyStr("c", 22U, kTypeSingleDeletion), ""}, + {KeyStr("d", 9U, kTypeSingleDeletion), ""}, + {KeyStr("f", 21U, kTypeSingleDeletion), ""}, + {KeyStr("j", 11U, kTypeSingleDeletion), ""}, + {KeyStr("j", 9U, kTypeSingleDeletion), ""}, + {KeyStr("k", 12U, kTypeSingleDeletion), ""}, + {KeyStr("k", 11U, kTypeSingleDeletion), ""}, + {KeyStr("l", 3U, kTypeSingleDeletion), ""}, + {KeyStr("l", 2U, kTypeSingleDeletion), ""}, + }); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile({ + {KeyStr("0", 2U, kTypeSingleDeletion), ""}, + {KeyStr("a", 11U, kTypeValue), "val1"}, + {KeyStr("b", 11U, kTypeValue), "val2"}, + {KeyStr("c", 21U, kTypeValue), "val3"}, + {KeyStr("d", 8U, kTypeValue), "val4"}, + {KeyStr("e", 2U, kTypeSingleDeletion), ""}, + {KeyStr("f", 1U, kTypeValue), "val1"}, + {KeyStr("g", 11U, kTypeSingleDeletion), ""}, + {KeyStr("h", 2U, kTypeSingleDeletion), ""}, + {KeyStr("m", 12U, kTypeValue), "val1"}, + {KeyStr("m", 11U, kTypeSingleDeletion), ""}, + {KeyStr("m", 8U, kTypeValue), "val2"}, + }); + AddMockFile(file2); + + auto file3 = mock::MakeMockFile({ + {KeyStr("A", 1U, kTypeValue), "val"}, + {KeyStr("e", 1U, kTypeValue), "val"}, + }); + AddMockFile(file3, 2); + + auto expected_results = mock::MakeMockFile({ + {KeyStr("A", 12U, kTypeSingleDeletion), ""}, + {KeyStr("a", 12U, kTypeSingleDeletion), ""}, + {KeyStr("a", 11U, kTypeValue), ""}, + {KeyStr("b", 21U, kTypeSingleDeletion), ""}, + {KeyStr("b", 11U, kTypeValue), "val2"}, + {KeyStr("c", 22U, kTypeSingleDeletion), ""}, + {KeyStr("c", 21U, kTypeValue), ""}, + {KeyStr("e", 2U, kTypeSingleDeletion), ""}, + {KeyStr("f", 21U, kTypeSingleDeletion), ""}, + {KeyStr("f", 1U, kTypeValue), "val1"}, + {KeyStr("g", 11U, kTypeSingleDeletion), ""}, + {KeyStr("j", 11U, kTypeSingleDeletion), ""}, + {KeyStr("k", 11U, kTypeSingleDeletion), ""}, + {KeyStr("m", 12U, kTypeValue), "val1"}, + {KeyStr("m", 11U, kTypeSingleDeletion), ""}, + {KeyStr("m", 8U, kTypeValue), "val2"}, + }); + + SetLastSequence(22U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results, {10U, 20U}, 10U); +} + +TEST_F(CompactionJobTest, EarliestWriteConflictSnapshot) { + NewDB(); + + // Test multiple snapshots where the earliest snapshot is not a + // write-conflic-snapshot. + + auto file1 = mock::MakeMockFile({ + {KeyStr("A", 24U, kTypeSingleDeletion), ""}, + {KeyStr("A", 23U, kTypeValue), "val"}, + {KeyStr("B", 24U, kTypeSingleDeletion), ""}, + {KeyStr("B", 23U, kTypeValue), "val"}, + {KeyStr("D", 24U, kTypeSingleDeletion), ""}, + {KeyStr("G", 32U, kTypeSingleDeletion), ""}, + {KeyStr("G", 31U, kTypeValue), "val"}, + {KeyStr("G", 24U, kTypeSingleDeletion), ""}, + {KeyStr("G", 23U, kTypeValue), "val2"}, + {KeyStr("H", 31U, kTypeValue), "val"}, + {KeyStr("H", 24U, kTypeSingleDeletion), ""}, + {KeyStr("H", 23U, kTypeValue), "val"}, + {KeyStr("I", 35U, kTypeSingleDeletion), ""}, + {KeyStr("I", 34U, kTypeValue), "val2"}, + {KeyStr("I", 33U, kTypeSingleDeletion), ""}, + {KeyStr("I", 32U, kTypeValue), "val3"}, + {KeyStr("I", 31U, kTypeSingleDeletion), ""}, + {KeyStr("J", 34U, kTypeValue), "val"}, + {KeyStr("J", 33U, kTypeSingleDeletion), ""}, + {KeyStr("J", 25U, kTypeValue), "val2"}, + {KeyStr("J", 24U, kTypeSingleDeletion), ""}, + }); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile({ + {KeyStr("A", 14U, kTypeSingleDeletion), ""}, + {KeyStr("A", 13U, kTypeValue), "val2"}, + {KeyStr("C", 14U, kTypeSingleDeletion), ""}, + {KeyStr("C", 13U, kTypeValue), "val"}, + {KeyStr("E", 12U, kTypeSingleDeletion), ""}, + {KeyStr("F", 4U, kTypeSingleDeletion), ""}, + {KeyStr("F", 3U, kTypeValue), "val"}, + {KeyStr("G", 14U, kTypeSingleDeletion), ""}, + {KeyStr("G", 13U, kTypeValue), "val3"}, + {KeyStr("H", 14U, kTypeSingleDeletion), ""}, + {KeyStr("H", 13U, kTypeValue), "val2"}, + {KeyStr("I", 13U, kTypeValue), "val4"}, + {KeyStr("I", 12U, kTypeSingleDeletion), ""}, + {KeyStr("I", 11U, kTypeValue), "val5"}, + {KeyStr("J", 15U, kTypeValue), "val3"}, + {KeyStr("J", 14U, kTypeSingleDeletion), ""}, + }); + AddMockFile(file2); + + auto expected_results = mock::MakeMockFile({ + {KeyStr("A", 24U, kTypeSingleDeletion), ""}, + {KeyStr("A", 23U, kTypeValue), ""}, + {KeyStr("B", 24U, kTypeSingleDeletion), ""}, + {KeyStr("B", 23U, kTypeValue), ""}, + {KeyStr("D", 24U, kTypeSingleDeletion), ""}, + {KeyStr("E", 12U, kTypeSingleDeletion), ""}, + {KeyStr("G", 32U, kTypeSingleDeletion), ""}, + {KeyStr("G", 31U, kTypeValue), ""}, + {KeyStr("H", 31U, kTypeValue), "val"}, + {KeyStr("I", 35U, kTypeSingleDeletion), ""}, + {KeyStr("I", 34U, kTypeValue), ""}, + {KeyStr("I", 31U, kTypeSingleDeletion), ""}, + {KeyStr("I", 13U, kTypeValue), "val4"}, + {KeyStr("J", 34U, kTypeValue), "val"}, + {KeyStr("J", 33U, kTypeSingleDeletion), ""}, + {KeyStr("J", 25U, kTypeValue), "val2"}, + {KeyStr("J", 24U, kTypeSingleDeletion), ""}, + {KeyStr("J", 15U, kTypeValue), "val3"}, + {KeyStr("J", 14U, kTypeSingleDeletion), ""}, + }); + + SetLastSequence(24U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results, {10U, 20U, 30U}, 20U); +} + +TEST_F(CompactionJobTest, SingleDeleteZeroSeq) { + NewDB(); + + auto file1 = mock::MakeMockFile({ + {KeyStr("A", 10U, kTypeSingleDeletion), ""}, + {KeyStr("dummy", 5U, kTypeValue), "val2"}, + }); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile({ + {KeyStr("A", 0U, kTypeValue), "val"}, + }); + AddMockFile(file2); + + auto expected_results = mock::MakeMockFile({ + {KeyStr("dummy", 5U, kTypeValue), "val2"}, + }); + + SetLastSequence(22U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results, {}); +} + +TEST_F(CompactionJobTest, MultiSingleDelete) { + // Tests three scenarios involving multiple single delete/put pairs: + // + // A: Put Snapshot SDel Put SDel -> Put Snapshot SDel + // B: Snapshot Put SDel Put SDel Snapshot -> Snapshot SDel Snapshot + // C: SDel Put SDel Snapshot Put -> Snapshot Put + // D: (Put) SDel Snapshot Put SDel -> (Put) SDel Snapshot SDel + // E: Put SDel Snapshot Put SDel -> Snapshot SDel + // F: Put SDel Put Sdel Snapshot -> removed + // G: Snapshot SDel Put SDel Put -> Snapshot Put SDel + // H: (Put) Put SDel Put Sdel Snapshot -> Removed + // I: (Put) Snapshot Put SDel Put SDel -> SDel + // J: Put Put SDel Put SDel SDel Snapshot Put Put SDel SDel Put + // -> Snapshot Put + // K: SDel SDel Put SDel Put Put Snapshot SDel Put SDel SDel Put SDel + // -> Snapshot Put Snapshot SDel + // L: SDel Put Del Put SDel Snapshot Del Put Del SDel Put SDel + // -> Snapshot SDel + // M: (Put) SDel Put Del Put SDel Snapshot Put Del SDel Put SDel Del + // -> SDel Snapshot Del + NewDB(); + + auto file1 = mock::MakeMockFile({ + {KeyStr("A", 14U, kTypeSingleDeletion), ""}, + {KeyStr("A", 13U, kTypeValue), "val5"}, + {KeyStr("A", 12U, kTypeSingleDeletion), ""}, + {KeyStr("B", 14U, kTypeSingleDeletion), ""}, + {KeyStr("B", 13U, kTypeValue), "val2"}, + {KeyStr("C", 14U, kTypeValue), "val3"}, + {KeyStr("D", 12U, kTypeSingleDeletion), ""}, + {KeyStr("D", 11U, kTypeValue), "val4"}, + {KeyStr("G", 15U, kTypeValue), "val"}, + {KeyStr("G", 14U, kTypeSingleDeletion), ""}, + {KeyStr("G", 13U, kTypeValue), "val"}, + {KeyStr("I", 14U, kTypeSingleDeletion), ""}, + {KeyStr("I", 13U, kTypeValue), "val"}, + {KeyStr("J", 15U, kTypeValue), "val"}, + {KeyStr("J", 14U, kTypeSingleDeletion), ""}, + {KeyStr("J", 13U, kTypeSingleDeletion), ""}, + {KeyStr("J", 12U, kTypeValue), "val"}, + {KeyStr("J", 11U, kTypeValue), "val"}, + {KeyStr("K", 16U, kTypeSingleDeletion), ""}, + {KeyStr("K", 15U, kTypeValue), "val1"}, + {KeyStr("K", 14U, kTypeSingleDeletion), ""}, + {KeyStr("K", 13U, kTypeSingleDeletion), ""}, + {KeyStr("K", 12U, kTypeValue), "val2"}, + {KeyStr("K", 11U, kTypeSingleDeletion), ""}, + {KeyStr("L", 16U, kTypeSingleDeletion), ""}, + {KeyStr("L", 15U, kTypeValue), "val"}, + {KeyStr("L", 14U, kTypeSingleDeletion), ""}, + {KeyStr("L", 13U, kTypeDeletion), ""}, + {KeyStr("L", 12U, kTypeValue), "val"}, + {KeyStr("L", 11U, kTypeDeletion), ""}, + {KeyStr("M", 16U, kTypeDeletion), ""}, + {KeyStr("M", 15U, kTypeSingleDeletion), ""}, + {KeyStr("M", 14U, kTypeValue), "val"}, + {KeyStr("M", 13U, kTypeSingleDeletion), ""}, + {KeyStr("M", 12U, kTypeDeletion), ""}, + {KeyStr("M", 11U, kTypeValue), "val"}, + }); + AddMockFile(file1); + + auto file2 = mock::MakeMockFile({ + {KeyStr("A", 10U, kTypeValue), "val"}, + {KeyStr("B", 12U, kTypeSingleDeletion), ""}, + {KeyStr("B", 11U, kTypeValue), "val2"}, + {KeyStr("C", 10U, kTypeSingleDeletion), ""}, + {KeyStr("C", 9U, kTypeValue), "val6"}, + {KeyStr("C", 8U, kTypeSingleDeletion), ""}, + {KeyStr("D", 10U, kTypeSingleDeletion), ""}, + {KeyStr("E", 12U, kTypeSingleDeletion), ""}, + {KeyStr("E", 11U, kTypeValue), "val"}, + {KeyStr("E", 5U, kTypeSingleDeletion), ""}, + {KeyStr("E", 4U, kTypeValue), "val"}, + {KeyStr("F", 6U, kTypeSingleDeletion), ""}, + {KeyStr("F", 5U, kTypeValue), "val"}, + {KeyStr("F", 4U, kTypeSingleDeletion), ""}, + {KeyStr("F", 3U, kTypeValue), "val"}, + {KeyStr("G", 12U, kTypeSingleDeletion), ""}, + {KeyStr("H", 6U, kTypeSingleDeletion), ""}, + {KeyStr("H", 5U, kTypeValue), "val"}, + {KeyStr("H", 4U, kTypeSingleDeletion), ""}, + {KeyStr("H", 3U, kTypeValue), "val"}, + {KeyStr("I", 12U, kTypeSingleDeletion), ""}, + {KeyStr("I", 11U, kTypeValue), "val"}, + {KeyStr("J", 6U, kTypeSingleDeletion), ""}, + {KeyStr("J", 5U, kTypeSingleDeletion), ""}, + {KeyStr("J", 4U, kTypeValue), "val"}, + {KeyStr("J", 3U, kTypeSingleDeletion), ""}, + {KeyStr("J", 2U, kTypeValue), "val"}, + {KeyStr("K", 8U, kTypeValue), "val3"}, + {KeyStr("K", 7U, kTypeValue), "val4"}, + {KeyStr("K", 6U, kTypeSingleDeletion), ""}, + {KeyStr("K", 5U, kTypeValue), "val5"}, + {KeyStr("K", 2U, kTypeSingleDeletion), ""}, + {KeyStr("K", 1U, kTypeSingleDeletion), ""}, + {KeyStr("L", 5U, kTypeSingleDeletion), ""}, + {KeyStr("L", 4U, kTypeValue), "val"}, + {KeyStr("L", 3U, kTypeDeletion), ""}, + {KeyStr("L", 2U, kTypeValue), "val"}, + {KeyStr("L", 1U, kTypeSingleDeletion), ""}, + {KeyStr("M", 10U, kTypeSingleDeletion), ""}, + {KeyStr("M", 7U, kTypeValue), "val"}, + {KeyStr("M", 5U, kTypeDeletion), ""}, + {KeyStr("M", 4U, kTypeValue), "val"}, + {KeyStr("M", 3U, kTypeSingleDeletion), ""}, + }); + AddMockFile(file2); + + auto file3 = mock::MakeMockFile({ + {KeyStr("D", 1U, kTypeValue), "val"}, + {KeyStr("H", 1U, kTypeValue), "val"}, + {KeyStr("I", 2U, kTypeValue), "val"}, + }); + AddMockFile(file3, 2); + + auto file4 = mock::MakeMockFile({ + {KeyStr("M", 1U, kTypeValue), "val"}, + }); + AddMockFile(file4, 2); + + auto expected_results = + mock::MakeMockFile({{KeyStr("A", 14U, kTypeSingleDeletion), ""}, + {KeyStr("A", 13U, kTypeValue), ""}, + {KeyStr("A", 12U, kTypeSingleDeletion), ""}, + {KeyStr("A", 10U, kTypeValue), "val"}, + {KeyStr("B", 14U, kTypeSingleDeletion), ""}, + {KeyStr("B", 13U, kTypeValue), ""}, + {KeyStr("C", 14U, kTypeValue), "val3"}, + {KeyStr("D", 12U, kTypeSingleDeletion), ""}, + {KeyStr("D", 11U, kTypeValue), ""}, + {KeyStr("D", 10U, kTypeSingleDeletion), ""}, + {KeyStr("E", 12U, kTypeSingleDeletion), ""}, + {KeyStr("E", 11U, kTypeValue), ""}, + {KeyStr("G", 15U, kTypeValue), "val"}, + {KeyStr("G", 12U, kTypeSingleDeletion), ""}, + {KeyStr("I", 14U, kTypeSingleDeletion), ""}, + {KeyStr("I", 13U, kTypeValue), ""}, + {KeyStr("J", 15U, kTypeValue), "val"}, + {KeyStr("K", 16U, kTypeSingleDeletion), ""}, + {KeyStr("K", 15U, kTypeValue), ""}, + {KeyStr("K", 11U, kTypeSingleDeletion), ""}, + {KeyStr("K", 8U, kTypeValue), "val3"}, + {KeyStr("L", 16U, kTypeSingleDeletion), ""}, + {KeyStr("L", 15U, kTypeValue), ""}, + {KeyStr("M", 16U, kTypeDeletion), ""}, + {KeyStr("M", 3U, kTypeSingleDeletion), ""}}); + + SetLastSequence(22U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results, {10U}, 10U); +} + +// This test documents the behavior where a corrupt key follows a deletion or a +// single deletion and the (single) deletion gets removed while the corrupt key +// gets written out. TODO(noetzli): We probably want a better way to treat +// corrupt keys. +TEST_F(CompactionJobTest, CorruptionAfterDeletion) { + NewDB(); + + auto file1 = + mock::MakeMockFile({{test::KeyStr("A", 6U, kTypeValue), "val3"}, + {test::KeyStr("a", 5U, kTypeDeletion), ""}, + {test::KeyStr("a", 4U, kTypeValue, true), "val"}}); + AddMockFile(file1); + + auto file2 = + mock::MakeMockFile({{test::KeyStr("b", 3U, kTypeSingleDeletion), ""}, + {test::KeyStr("b", 2U, kTypeValue, true), "val"}, + {test::KeyStr("c", 1U, kTypeValue), "val2"}}); + AddMockFile(file2); + + auto expected_results = + mock::MakeMockFile({{test::KeyStr("A", 0U, kTypeValue), "val3"}, + {test::KeyStr("a", 0U, kTypeValue, true), "val"}, + {test::KeyStr("b", 0U, kTypeValue, true), "val"}, + {test::KeyStr("c", 1U, kTypeValue), "val2"}}); + + SetLastSequence(6U); + auto files = cfd_->current()->storage_info()->LevelFiles(0); + RunCompaction({files}, expected_results); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, + "SKIPPED as CompactionJobStats is not supported in ROCKSDB_LITE\n"); + return 0; +} + +#endif // ROCKSDB_LITE diff --git a/db/compaction_picker.cc b/db/compaction_picker.cc index e05d07776e5..79af3ed9fe0 100644 --- a/db/compaction_picker.cc +++ b/db/compaction_picker.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,15 +9,26 @@ #include "db/compaction_picker.h" +#ifndef __STDC_FORMAT_MACROS #define __STDC_FORMAT_MACROS +#endif + #include #include -#include "db/filename.h" +#include +#include +#include +#include "db/column_family.h" +#include "monitoring/statistics.h" +#include "util/filename.h" #include "util/log_buffer.h" -#include "util/statistics.h" +#include "util/random.h" +#include "util/string_util.h" +#include "util/sync_point.h" namespace rocksdb { +namespace { uint64_t TotalCompensatedFileSize(const std::vector& files) { uint64_t sum = 0; for (size_t i = 0; i < files.size() && files[i]; i++) { @@ -26,215 +37,248 @@ uint64_t TotalCompensatedFileSize(const std::vector& files) { return sum; } -namespace { +bool FindIntraL0Compaction(const std::vector& level_files, + size_t min_files_to_compact, + uint64_t max_compact_bytes_per_del_file, + CompactionInputFiles* comp_inputs) { + size_t compact_bytes = level_files[0]->fd.file_size; + size_t compact_bytes_per_del_file = port::kMaxSizet; + // compaction range will be [0, span_len). + size_t span_len; + // pull in files until the amount of compaction work per deleted file begins + // increasing. + size_t new_compact_bytes_per_del_file = 0; + for (span_len = 1; span_len < level_files.size(); ++span_len) { + compact_bytes += level_files[span_len]->fd.file_size; + new_compact_bytes_per_del_file = compact_bytes / span_len; + if (level_files[span_len]->being_compacted || + new_compact_bytes_per_del_file > compact_bytes_per_del_file) { + break; + } + compact_bytes_per_del_file = new_compact_bytes_per_del_file; + } + + if (span_len >= min_files_to_compact && + new_compact_bytes_per_del_file < max_compact_bytes_per_del_file) { + assert(comp_inputs != nullptr); + comp_inputs->level = 0; + for (size_t i = 0; i < span_len; ++i) { + comp_inputs->files.push_back(level_files[i]); + } + return true; + } + return false; +} +} // anonymous namespace + // Determine compression type, based on user options, level of the output // file and whether compression is disabled. // If enable_compression is false, then compression is always disabled no // matter what the values of the other two parameters are. // Otherwise, the compression type is determined based on options and level. -CompressionType GetCompressionType(const Options& options, int level, - const bool enable_compression = true) { +CompressionType GetCompressionType(const ImmutableCFOptions& ioptions, + const VersionStorageInfo* vstorage, + const MutableCFOptions& mutable_cf_options, + int level, int base_level, + const bool enable_compression) { if (!enable_compression) { // disable compression return kNoCompression; } - // If the use has specified a different compression level for each level, - // then pick the compresison for that level. - if (!options.compression_per_level.empty()) { - const int n = options.compression_per_level.size() - 1; + + // If bottommost_compression is set and we are compacting to the + // bottommost level then we should use it. + if (ioptions.bottommost_compression != kDisableCompressionOption && + level > base_level && level >= (vstorage->num_non_empty_levels() - 1)) { + return ioptions.bottommost_compression; + } + // If the user has specified a different compression level for each level, + // then pick the compression for that level. + if (!ioptions.compression_per_level.empty()) { + assert(level == 0 || level >= base_level); + int idx = (level == 0) ? 0 : level - base_level + 1; + + const int n = static_cast(ioptions.compression_per_level.size()) - 1; // It is possible for level_ to be -1; in that case, we use level // 0's compression. This occurs mostly in backwards compatibility // situations when the builder doesn't know what level the file - // belongs to. Likewise, if level_ is beyond the end of the + // belongs to. Likewise, if level is beyond the end of the // specified compression levels, use the last value. - return options.compression_per_level[std::max(0, std::min(level, n))]; + return ioptions.compression_per_level[std::max(0, std::min(idx, n))]; } else { - return options.compression; + return mutable_cf_options.compression; } } -// Multiple two operands. If they overflow, return op1. -uint64_t MultiplyCheckOverflow(uint64_t op1, int op2) { - if (op1 == 0) { - return 0; - } - if (op2 <= 0) { - return op1; - } - uint64_t casted_op2 = (uint64_t) op2; - if (std::numeric_limits::max() / op1 < casted_op2) { - return op1; - } - return op1 * casted_op2; -} - -} // anonymous namespace - -CompactionPicker::CompactionPicker(const Options* options, +CompactionPicker::CompactionPicker(const ImmutableCFOptions& ioptions, const InternalKeyComparator* icmp) - : compactions_in_progress_(options->num_levels), - options_(options), - num_levels_(options->num_levels), - icmp_(icmp) { - - max_file_size_.reset(new uint64_t[NumberLevels()]); - level_max_bytes_.reset(new uint64_t[NumberLevels()]); - int target_file_size_multiplier = options_->target_file_size_multiplier; - int max_bytes_multiplier = options_->max_bytes_for_level_multiplier; - for (int i = 0; i < NumberLevels(); i++) { - if (i == 0 && options_->compaction_style == kCompactionStyleUniversal) { - max_file_size_[i] = ULLONG_MAX; - level_max_bytes_[i] = options_->max_bytes_for_level_base; - } else if (i > 1) { - max_file_size_[i] = MultiplyCheckOverflow(max_file_size_[i - 1], - target_file_size_multiplier); - level_max_bytes_[i] = MultiplyCheckOverflow( - MultiplyCheckOverflow(level_max_bytes_[i - 1], max_bytes_multiplier), - options_->max_bytes_for_level_multiplier_additional[i - 1]); - } else { - max_file_size_[i] = options_->target_file_size_base; - level_max_bytes_[i] = options_->max_bytes_for_level_base; - } - } -} + : ioptions_(ioptions), icmp_(icmp) {} CompactionPicker::~CompactionPicker() {} -void CompactionPicker::SizeBeingCompacted(std::vector& sizes) { - for (int level = 0; level < NumberLevels() - 1; level++) { - uint64_t total = 0; - for (auto c : compactions_in_progress_[level]) { - assert(c->level() == level); - for (int i = 0; i < c->num_input_files(0); i++) { - total += c->input(0, i)->compensated_file_size; - } - } - sizes[level] = total; - } -} - -// Clear all files to indicate that they are not being compacted // Delete this compaction from the list of running compactions. void CompactionPicker::ReleaseCompactionFiles(Compaction* c, Status status) { - c->MarkFilesBeingCompacted(false); - compactions_in_progress_[c->level()].erase(c); + UnregisterCompaction(c); if (!status.ok()) { c->ResetNextCompactionIndex(); } } -uint64_t CompactionPicker::MaxFileSizeForLevel(int level) const { - assert(level >= 0); - assert(level < NumberLevels()); - return max_file_size_[level]; -} - -uint64_t CompactionPicker::MaxGrandParentOverlapBytes(int level) { - uint64_t result = MaxFileSizeForLevel(level); - result *= options_->max_grandparent_overlap_factor; - return result; -} - -double CompactionPicker::MaxBytesForLevel(int level) { - // Note: the result for level zero is not really used since we set - // the level-0 compaction threshold based on number of files. - assert(level >= 0); - assert(level < NumberLevels()); - return level_max_bytes_[level]; -} - -void CompactionPicker::GetRange(const std::vector& inputs, - InternalKey* smallest, InternalKey* largest) { +void CompactionPicker::GetRange(const CompactionInputFiles& inputs, + InternalKey* smallest, + InternalKey* largest) const { + const int level = inputs.level; assert(!inputs.empty()); smallest->Clear(); largest->Clear(); - for (size_t i = 0; i < inputs.size(); i++) { - FileMetaData* f = inputs[i]; - if (i == 0) { - *smallest = f->smallest; - *largest = f->largest; - } else { - if (icmp_->Compare(f->smallest, *smallest) < 0) { + + if (level == 0) { + for (size_t i = 0; i < inputs.size(); i++) { + FileMetaData* f = inputs[i]; + if (i == 0) { *smallest = f->smallest; - } - if (icmp_->Compare(f->largest, *largest) > 0) { *largest = f->largest; + } else { + if (icmp_->Compare(f->smallest, *smallest) < 0) { + *smallest = f->smallest; + } + if (icmp_->Compare(f->largest, *largest) > 0) { + *largest = f->largest; + } } } + } else { + *smallest = inputs[0]->smallest; + *largest = inputs[inputs.size() - 1]->largest; } } -void CompactionPicker::GetRange(const std::vector& inputs1, - const std::vector& inputs2, - InternalKey* smallest, InternalKey* largest) { - std::vector all = inputs1; - all.insert(all.end(), inputs2.begin(), inputs2.end()); - GetRange(all, smallest, largest); +void CompactionPicker::GetRange(const CompactionInputFiles& inputs1, + const CompactionInputFiles& inputs2, + InternalKey* smallest, + InternalKey* largest) const { + assert(!inputs1.empty() || !inputs2.empty()); + if (inputs1.empty()) { + GetRange(inputs2, smallest, largest); + } else if (inputs2.empty()) { + GetRange(inputs1, smallest, largest); + } else { + InternalKey smallest1, smallest2, largest1, largest2; + GetRange(inputs1, &smallest1, &largest1); + GetRange(inputs2, &smallest2, &largest2); + *smallest = + icmp_->Compare(smallest1, smallest2) < 0 ? smallest1 : smallest2; + *largest = icmp_->Compare(largest1, largest2) < 0 ? largest2 : largest1; + } } -bool CompactionPicker::ExpandWhileOverlapping(Compaction* c) { - assert(c != nullptr); - // If inputs are empty then there is nothing to expand. - if (c->inputs_[0].empty()) { - assert(c->inputs_[1].empty()); - // This isn't good compaction - return false; +void CompactionPicker::GetRange(const std::vector& inputs, + InternalKey* smallest, + InternalKey* largest) const { + InternalKey current_smallest; + InternalKey current_largest; + bool initialized = false; + for (const auto& in : inputs) { + if (in.empty()) { + continue; + } + GetRange(in, ¤t_smallest, ¤t_largest); + if (!initialized) { + *smallest = current_smallest; + *largest = current_largest; + initialized = true; + } else { + if (icmp_->Compare(current_smallest, *smallest) < 0) { + *smallest = current_smallest; + } + if (icmp_->Compare(current_largest, *largest) > 0) { + *largest = current_largest; + } + } } + assert(initialized); +} + +bool CompactionPicker::ExpandInputsToCleanCut(const std::string& cf_name, + VersionStorageInfo* vstorage, + CompactionInputFiles* inputs) { + // This isn't good compaction + assert(!inputs->empty()); + const int level = inputs->level; // GetOverlappingInputs will always do the right thing for level-0. // So we don't need to do any expansion if level == 0. - if (c->level() == 0) { + if (level == 0) { return true; } - const int level = c->level(); InternalKey smallest, largest; - // Keep expanding c->inputs_[0] until we are sure that there is a - // "clean cut" boundary between the files in input and the surrounding files. + // Keep expanding inputs until we are sure that there is a "clean cut" + // boundary between the files in input and the surrounding files. // This will ensure that no parts of a key are lost during compaction. int hint_index = -1; size_t old_size; do { - old_size = c->inputs_[0].size(); - GetRange(c->inputs_[0].files, &smallest, &largest); - c->inputs_[0].clear(); - c->input_version_->GetOverlappingInputs( - level, &smallest, &largest, &c->inputs_[0].files, - hint_index, &hint_index); - } while(c->inputs_[0].size() > old_size); + old_size = inputs->size(); + GetRange(*inputs, &smallest, &largest); + inputs->clear(); + vstorage->GetOverlappingInputs(level, &smallest, &largest, &inputs->files, + hint_index, &hint_index); + } while (inputs->size() > old_size); - // Get the new range - GetRange(c->inputs_[0].files, &smallest, &largest); + // we started off with inputs non-empty and the previous loop only grew + // inputs. thus, inputs should be non-empty here + assert(!inputs->empty()); // If, after the expansion, there are files that are already under // compaction, then we must drop/cancel this compaction. - int parent_index = -1; - if (c->inputs_[0].empty()) { - Log(options_->info_log, - "[%s] ExpandWhileOverlapping() failure because zero input files", - c->column_family_data()->GetName().c_str()); - } - if (c->inputs_[0].empty() || FilesInCompaction(c->inputs_[0].files) || - (c->level() != c->output_level() && - ParentRangeInCompaction(c->input_version_, &smallest, &largest, level, - &parent_index))) { - c->inputs_[0].clear(); - c->inputs_[1].clear(); + if (AreFilesInCompaction(inputs->files)) { return false; } return true; } -uint64_t CompactionPicker::ExpandedCompactionByteSizeLimit(int level) { - uint64_t result = MaxFileSizeForLevel(level); - result *= options_->expanded_compaction_factor; - return result; +bool CompactionPicker::RangeOverlapWithCompaction( + const Slice& smallest_user_key, const Slice& largest_user_key, + int level) const { + const Comparator* ucmp = icmp_->user_comparator(); + for (Compaction* c : compactions_in_progress_) { + if (c->output_level() == level && + ucmp->Compare(smallest_user_key, c->GetLargestUserKey()) <= 0 && + ucmp->Compare(largest_user_key, c->GetSmallestUserKey()) >= 0) { + // Overlap + return true; + } + } + // Did not overlap with any running compaction in level `level` + return false; +} + +bool CompactionPicker::FilesRangeOverlapWithCompaction( + const std::vector& inputs, int level) const { + bool is_empty = true; + for (auto& in : inputs) { + if (!in.empty()) { + is_empty = false; + break; + } + } + if (is_empty) { + // No files in inputs + return false; + } + + InternalKey smallest, largest; + GetRange(inputs, &smallest, &largest); + return RangeOverlapWithCompaction(smallest.user_key(), largest.user_key(), + level); } // Returns true if any one of specified files are being compacted -bool CompactionPicker::FilesInCompaction(std::vector& files) { - for (unsigned int i = 0; i < files.size(); i++) { +bool CompactionPicker::AreFilesInCompaction( + const std::vector& files) { + for (size_t i = 0; i < files.size(); i++) { if (files[i]->being_compacted) { return true; } @@ -242,127 +286,330 @@ bool CompactionPicker::FilesInCompaction(std::vector& files) { return false; } +Compaction* CompactionPicker::CompactFiles( + const CompactionOptions& compact_options, + const std::vector& input_files, int output_level, + VersionStorageInfo* vstorage, const MutableCFOptions& mutable_cf_options, + uint32_t output_path_id) { + assert(input_files.size()); + + // TODO(rven ): we might be able to run concurrent level 0 compaction + // if the key ranges of the two compactions do not overlap, but for now + // we do not allow it. + if ((input_files[0].level == 0) && !level0_compactions_in_progress_.empty()) { + return nullptr; + } + // This compaction output could overlap with a running compaction + if (FilesRangeOverlapWithCompaction(input_files, output_level)) { + return nullptr; + } + auto c = + new Compaction(vstorage, ioptions_, mutable_cf_options, input_files, + output_level, compact_options.output_file_size_limit, + mutable_cf_options.max_compaction_bytes, output_path_id, + compact_options.compression, /* grandparents */ {}, true); + + // If it's level 0 compaction, make sure we don't execute any other level 0 + // compactions in parallel + RegisterCompaction(c); + return c; +} + +Status CompactionPicker::GetCompactionInputsFromFileNumbers( + std::vector* input_files, + std::unordered_set* input_set, const VersionStorageInfo* vstorage, + const CompactionOptions& compact_options) const { + if (input_set->size() == 0U) { + return Status::InvalidArgument( + "Compaction must include at least one file."); + } + assert(input_files); + + std::vector matched_input_files; + matched_input_files.resize(vstorage->num_levels()); + int first_non_empty_level = -1; + int last_non_empty_level = -1; + // TODO(yhchiang): use a lazy-initialized mapping from + // file_number to FileMetaData in Version. + for (int level = 0; level < vstorage->num_levels(); ++level) { + for (auto file : vstorage->LevelFiles(level)) { + auto iter = input_set->find(file->fd.GetNumber()); + if (iter != input_set->end()) { + matched_input_files[level].files.push_back(file); + input_set->erase(iter); + last_non_empty_level = level; + if (first_non_empty_level == -1) { + first_non_empty_level = level; + } + } + } + } + + if (!input_set->empty()) { + std::string message( + "Cannot find matched SST files for the following file numbers:"); + for (auto fn : *input_set) { + message += " "; + message += ToString(fn); + } + return Status::InvalidArgument(message); + } + + for (int level = first_non_empty_level; level <= last_non_empty_level; + ++level) { + matched_input_files[level].level = level; + input_files->emplace_back(std::move(matched_input_files[level])); + } + + return Status::OK(); +} + // Returns true if any one of the parent files are being compacted -bool CompactionPicker::ParentRangeInCompaction(Version* version, - const InternalKey* smallest, - const InternalKey* largest, - int level, int* parent_index) { +bool CompactionPicker::IsRangeInCompaction(VersionStorageInfo* vstorage, + const InternalKey* smallest, + const InternalKey* largest, + int level, int* level_index) { std::vector inputs; - assert(level + 1 < NumberLevels()); - - version->GetOverlappingInputs(level + 1, smallest, largest, &inputs, - *parent_index, parent_index); - return FilesInCompaction(inputs); -} - -// Populates the set of inputs from "level+1" that overlap with "level". -// Will also attempt to expand "level" if that doesn't expand "level+1" -// or cause "level" to include a file for compaction that has an overlapping -// user-key with another file. -void CompactionPicker::SetupOtherInputs(Compaction* c) { - // If inputs are empty, then there is nothing to expand. - // If both input and output levels are the same, no need to consider - // files at level "level+1" - if (c->inputs_[0].empty() || c->level() == c->output_level()) { - return; + assert(level < NumberLevels()); + + vstorage->GetOverlappingInputs(level, smallest, largest, &inputs, + *level_index, level_index); + return AreFilesInCompaction(inputs); +} + +// Populates the set of inputs of all other levels that overlap with the +// start level. +// Now we assume all levels except start level and output level are empty. +// Will also attempt to expand "start level" if that doesn't expand +// "output level" or cause "level" to include a file for compaction that has an +// overlapping user-key with another file. +// REQUIRES: input_level and output_level are different +// REQUIRES: inputs->empty() == false +// Returns false if files on parent level are currently in compaction, which +// means that we can't compact them +bool CompactionPicker::SetupOtherInputs( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, CompactionInputFiles* inputs, + CompactionInputFiles* output_level_inputs, int* parent_index, + int base_index) { + assert(!inputs->empty()); + assert(output_level_inputs->empty()); + const int input_level = inputs->level; + const int output_level = output_level_inputs->level; + assert(input_level != output_level); + + // For now, we only support merging two levels, start level and output level. + // We need to assert other levels are empty. + for (int l = input_level + 1; l < output_level; l++) { + assert(vstorage->NumLevelFiles(l) == 0); } - const int level = c->level(); InternalKey smallest, largest; // Get the range one last time. - GetRange(c->inputs_[0].files, &smallest, &largest); - - // Populate the set of next-level files (inputs_[1]) to include in compaction - c->input_version_->GetOverlappingInputs( - level + 1, &smallest, &largest, - &c->inputs_[1].files, c->parent_index_, - &c->parent_index_); - - // Get entire range covered by compaction - InternalKey all_start, all_limit; - GetRange(c->inputs_[0].files, c->inputs_[1].files, &all_start, &all_limit); + GetRange(*inputs, &smallest, &largest); + + // Populate the set of next-level files (inputs_GetOutputLevelInputs()) to + // include in compaction + vstorage->GetOverlappingInputs(output_level, &smallest, &largest, + &output_level_inputs->files, *parent_index, + parent_index); + if (AreFilesInCompaction(output_level_inputs->files)) { + return false; + } + if (!output_level_inputs->empty()) { + if (!ExpandInputsToCleanCut(cf_name, vstorage, output_level_inputs)) { + return false; + } + } // See if we can further grow the number of inputs in "level" without // changing the number of "level+1" files we pick up. We also choose NOT // to expand if this would cause "level" to include some entries for some // user key, while excluding other entries for the same user key. This // can happen when one user key spans multiple files. - if (!c->inputs_[1].empty()) { - std::vector expanded0; - c->input_version_->GetOverlappingInputs( - level, &all_start, &all_limit, &expanded0, c->base_index_, nullptr); - const uint64_t inputs0_size = TotalCompensatedFileSize(c->inputs_[0].files); - const uint64_t inputs1_size = TotalCompensatedFileSize(c->inputs_[1].files); - const uint64_t expanded0_size = TotalCompensatedFileSize(expanded0); - uint64_t limit = ExpandedCompactionByteSizeLimit(level); - if (expanded0.size() > c->inputs_[0].size() && - inputs1_size + expanded0_size < limit && - !FilesInCompaction(expanded0) && - !c->input_version_->HasOverlappingUserKey(&expanded0, level)) { + if (!output_level_inputs->empty()) { + const uint64_t limit = mutable_cf_options.max_compaction_bytes; + const uint64_t output_level_inputs_size = + TotalCompensatedFileSize(output_level_inputs->files); + const uint64_t inputs_size = TotalCompensatedFileSize(inputs->files); + bool expand_inputs = false; + + CompactionInputFiles expanded_inputs; + expanded_inputs.level = input_level; + // Get closed interval of output level + InternalKey all_start, all_limit; + GetRange(*inputs, *output_level_inputs, &all_start, &all_limit); + bool try_overlapping_inputs = true; + vstorage->GetOverlappingInputs(input_level, &all_start, &all_limit, + &expanded_inputs.files, base_index, nullptr); + uint64_t expanded_inputs_size = + TotalCompensatedFileSize(expanded_inputs.files); + if (!ExpandInputsToCleanCut(cf_name, vstorage, &expanded_inputs)) { + try_overlapping_inputs = false; + } + if (try_overlapping_inputs && expanded_inputs.size() > inputs->size() && + output_level_inputs_size + expanded_inputs_size < limit && + !AreFilesInCompaction(expanded_inputs.files)) { InternalKey new_start, new_limit; - GetRange(expanded0, &new_start, &new_limit); - std::vector expanded1; - c->input_version_->GetOverlappingInputs(level + 1, &new_start, &new_limit, - &expanded1, c->parent_index_, - &c->parent_index_); - if (expanded1.size() == c->inputs_[1].size() && - !FilesInCompaction(expanded1)) { - Log(options_->info_log, - "[%s] Expanding@%d %zu+%zu (%" PRIu64 "+%" PRIu64 - " bytes) to %zu+%zu (%" PRIu64 "+%" PRIu64 "bytes)\n", - c->column_family_data()->GetName().c_str(), level, - c->inputs_[0].size(), c->inputs_[1].size(), inputs0_size, - inputs1_size, expanded0.size(), expanded1.size(), expanded0_size, - inputs1_size); - smallest = new_start; - largest = new_limit; - c->inputs_[0].files = expanded0; - c->inputs_[1].files = expanded1; - GetRange(c->inputs_[0].files, c->inputs_[1].files, - &all_start, &all_limit); + GetRange(expanded_inputs, &new_start, &new_limit); + CompactionInputFiles expanded_output_level_inputs; + expanded_output_level_inputs.level = output_level; + vstorage->GetOverlappingInputs(output_level, &new_start, &new_limit, + &expanded_output_level_inputs.files, + *parent_index, parent_index); + assert(!expanded_output_level_inputs.empty()); + if (!AreFilesInCompaction(expanded_output_level_inputs.files) && + ExpandInputsToCleanCut(cf_name, vstorage, + &expanded_output_level_inputs) && + expanded_output_level_inputs.size() == output_level_inputs->size()) { + expand_inputs = true; + } + } + if (!expand_inputs) { + vstorage->GetCleanInputsWithinInterval(input_level, &all_start, + &all_limit, &expanded_inputs.files, + base_index, nullptr); + expanded_inputs_size = TotalCompensatedFileSize(expanded_inputs.files); + if (expanded_inputs.size() > inputs->size() && + output_level_inputs_size + expanded_inputs_size < limit && + !AreFilesInCompaction(expanded_inputs.files)) { + expand_inputs = true; } } + if (expand_inputs) { + ROCKS_LOG_INFO(ioptions_.info_log, + "[%s] Expanding@%d %" ROCKSDB_PRIszt "+%" ROCKSDB_PRIszt + "(%" PRIu64 "+%" PRIu64 " bytes) to %" ROCKSDB_PRIszt + "+%" ROCKSDB_PRIszt " (%" PRIu64 "+%" PRIu64 "bytes)\n", + cf_name.c_str(), input_level, inputs->size(), + output_level_inputs->size(), inputs_size, + output_level_inputs_size, expanded_inputs.size(), + output_level_inputs->size(), expanded_inputs_size, + output_level_inputs_size); + inputs->files = expanded_inputs.files; + } } + return true; +} +void CompactionPicker::GetGrandparents( + VersionStorageInfo* vstorage, const CompactionInputFiles& inputs, + const CompactionInputFiles& output_level_inputs, + std::vector* grandparents) { + InternalKey start, limit; + GetRange(inputs, output_level_inputs, &start, &limit); // Compute the set of grandparent files that overlap this compaction // (parent == level+1; grandparent == level+2) - if (level + 2 < NumberLevels()) { - c->input_version_->GetOverlappingInputs(level + 2, &all_start, &all_limit, - &c->grandparents_); + if (output_level_inputs.level + 1 < NumberLevels()) { + vstorage->GetOverlappingInputs(output_level_inputs.level + 1, &start, + &limit, grandparents); } } -Compaction* CompactionPicker::CompactRange(Version* version, int input_level, - int output_level, - uint32_t output_path_id, - const InternalKey* begin, - const InternalKey* end, - InternalKey** compaction_end) { +Compaction* CompactionPicker::CompactRange( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, int input_level, int output_level, + uint32_t output_path_id, const InternalKey* begin, const InternalKey* end, + InternalKey** compaction_end, bool* manual_conflict) { // CompactionPickerFIFO has its own implementation of compact range - assert(options_->compaction_style != kCompactionStyleFIFO); + assert(ioptions_.compaction_style != kCompactionStyleFIFO); - std::vector inputs; + if (input_level == ColumnFamilyData::kCompactAllLevels) { + assert(ioptions_.compaction_style == kCompactionStyleUniversal); + + // Universal compaction with more than one level always compacts all the + // files together to the last level. + assert(vstorage->num_levels() > 1); + // DBImpl::CompactRange() set output level to be the last level + if (ioptions_.allow_ingest_behind) { + assert(output_level == vstorage->num_levels() - 2); + } else { + assert(output_level == vstorage->num_levels() - 1); + } + // DBImpl::RunManualCompaction will make full range for universal compaction + assert(begin == nullptr); + assert(end == nullptr); + *compaction_end = nullptr; + + int start_level = 0; + for (; start_level < vstorage->num_levels() && + vstorage->NumLevelFiles(start_level) == 0; + start_level++) { + } + if (start_level == vstorage->num_levels()) { + return nullptr; + } + + if ((start_level == 0) && (!level0_compactions_in_progress_.empty())) { + *manual_conflict = true; + // Only one level 0 compaction allowed + return nullptr; + } + + std::vector inputs(vstorage->num_levels() - + start_level); + for (int level = start_level; level < vstorage->num_levels(); level++) { + inputs[level - start_level].level = level; + auto& files = inputs[level - start_level].files; + for (FileMetaData* f : vstorage->LevelFiles(level)) { + files.push_back(f); + } + if (AreFilesInCompaction(files)) { + *manual_conflict = true; + return nullptr; + } + } + + // 2 non-exclusive manual compactions could run at the same time producing + // overlaping outputs in the same level. + if (FilesRangeOverlapWithCompaction(inputs, output_level)) { + // This compaction output could potentially conflict with the output + // of a currently running compaction, we cannot run it. + *manual_conflict = true; + return nullptr; + } + + Compaction* c = new Compaction( + vstorage, ioptions_, mutable_cf_options, std::move(inputs), + output_level, mutable_cf_options.MaxFileSizeForLevel(output_level), + /* max_compaction_bytes */ LLONG_MAX, output_path_id, + GetCompressionType(ioptions_, vstorage, mutable_cf_options, + output_level, 1), + /* grandparents */ {}, /* is manual */ true); + RegisterCompaction(c); + return c; + } + + CompactionInputFiles inputs; + inputs.level = input_level; bool covering_the_whole_range = true; // All files are 'overlapping' in universal style compaction. // We have to compact the entire range in one shot. - if (options_->compaction_style == kCompactionStyleUniversal) { + if (ioptions_.compaction_style == kCompactionStyleUniversal) { begin = nullptr; end = nullptr; } - version->GetOverlappingInputs(input_level, begin, end, &inputs); + + vstorage->GetOverlappingInputs(input_level, begin, end, &inputs.files); if (inputs.empty()) { return nullptr; } + if ((input_level == 0) && (!level0_compactions_in_progress_.empty())) { + // Only one level 0 compaction allowed + TEST_SYNC_POINT("CompactionPicker::CompactRange:Conflict"); + *manual_conflict = true; + return nullptr; + } + // Avoid compacting too much in one shot in case the range is large. // But we cannot do this for level-0 since level-0 files can overlap // and we must not pick one file and drop another older file if the // two files overlap. if (input_level > 0) { - const uint64_t limit = - MaxFileSizeForLevel(input_level) * options_->source_compaction_factor; + const uint64_t limit = mutable_cf_options.max_compaction_bytes; uint64_t total = 0; for (size_t i = 0; i + 1 < inputs.size(); ++i) { uint64_t s = inputs[i]->compensated_file_size; @@ -370,647 +617,978 @@ Compaction* CompactionPicker::CompactRange(Version* version, int input_level, if (total >= limit) { **compaction_end = inputs[i + 1]->smallest; covering_the_whole_range = false; - inputs.resize(i + 1); + inputs.files.resize(i + 1); break; } } } - assert(output_path_id < static_cast(options_->db_paths.size())); - Compaction* c = new Compaction( - version, input_level, output_level, MaxFileSizeForLevel(output_level), - MaxGrandParentOverlapBytes(input_level), output_path_id, - GetCompressionType(*options_, output_level)); - - c->inputs_[0].files = inputs; - if (ExpandWhileOverlapping(c) == false) { - delete c; - Log(options_->info_log, - "[%s] Could not compact due to expansion failure.\n", - version->cfd_->GetName().c_str()); + assert(output_path_id < static_cast(ioptions_.db_paths.size())); + + if (ExpandInputsToCleanCut(cf_name, vstorage, &inputs) == false) { + // manual compaction is now multi-threaded, so it can + // happen that ExpandWhileOverlapping fails + // we handle it higher in RunManualCompaction + *manual_conflict = true; return nullptr; } - SetupOtherInputs(c); - if (covering_the_whole_range) { *compaction_end = nullptr; } - // These files that are to be manaully compacted do not trample - // upon other files because manual compactions are processed when - // the system has a max of 1 background compaction thread. - c->MarkFilesBeingCompacted(true); + CompactionInputFiles output_level_inputs; + if (output_level == ColumnFamilyData::kCompactToBaseLevel) { + assert(input_level == 0); + output_level = vstorage->base_level(); + assert(output_level > 0); + } + output_level_inputs.level = output_level; + if (input_level != output_level) { + int parent_index = -1; + if (!SetupOtherInputs(cf_name, mutable_cf_options, vstorage, &inputs, + &output_level_inputs, &parent_index, -1)) { + // manual compaction is now multi-threaded, so it can + // happen that SetupOtherInputs fails + // we handle it higher in RunManualCompaction + *manual_conflict = true; + return nullptr; + } + } - // Is this compaction creating a file at the bottommost level - c->SetupBottomMostLevel(true); + std::vector compaction_inputs({inputs}); + if (!output_level_inputs.empty()) { + compaction_inputs.push_back(output_level_inputs); + } + for (size_t i = 0; i < compaction_inputs.size(); i++) { + if (AreFilesInCompaction(compaction_inputs[i].files)) { + *manual_conflict = true; + return nullptr; + } + } - c->is_manual_compaction_ = true; + // 2 non-exclusive manual compactions could run at the same time producing + // overlaping outputs in the same level. + if (FilesRangeOverlapWithCompaction(compaction_inputs, output_level)) { + // This compaction output could potentially conflict with the output + // of a currently running compaction, we cannot run it. + *manual_conflict = true; + return nullptr; + } - return c; + std::vector grandparents; + GetGrandparents(vstorage, inputs, output_level_inputs, &grandparents); + Compaction* compaction = new Compaction( + vstorage, ioptions_, mutable_cf_options, std::move(compaction_inputs), + output_level, mutable_cf_options.MaxFileSizeForLevel(output_level), + mutable_cf_options.max_compaction_bytes, output_path_id, + GetCompressionType(ioptions_, vstorage, mutable_cf_options, output_level, + vstorage->base_level()), + std::move(grandparents), /* is manual compaction */ true); + + TEST_SYNC_POINT_CALLBACK("CompactionPicker::CompactRange:Return", compaction); + RegisterCompaction(compaction); + + // Creating a compaction influences the compaction score because the score + // takes running compactions into account (by skipping files that are already + // being compacted). Since we just changed compaction score, we recalculate it + // here + vstorage->ComputeCompactionScore(ioptions_, mutable_cf_options); + + return compaction; } -Compaction* LevelCompactionPicker::PickCompaction(Version* version, - LogBuffer* log_buffer) { - Compaction* c = nullptr; - int level = -1; +#ifndef ROCKSDB_LITE +namespace { +// Test whether two files have overlapping key-ranges. +bool HaveOverlappingKeyRanges(const Comparator* c, const SstFileMetaData& a, + const SstFileMetaData& b) { + if (c->Compare(a.smallestkey, b.smallestkey) >= 0) { + if (c->Compare(a.smallestkey, b.largestkey) <= 0) { + // b.smallestkey <= a.smallestkey <= b.largestkey + return true; + } + } else if (c->Compare(a.largestkey, b.smallestkey) >= 0) { + // a.smallestkey < b.smallestkey <= a.largestkey + return true; + } + if (c->Compare(a.largestkey, b.largestkey) <= 0) { + if (c->Compare(a.largestkey, b.smallestkey) >= 0) { + // b.smallestkey <= a.largestkey <= b.largestkey + return true; + } + } else if (c->Compare(a.smallestkey, b.largestkey) <= 0) { + // a.smallestkey <= b.largestkey < a.largestkey + return true; + } + return false; +} +} // namespace + +Status CompactionPicker::SanitizeCompactionInputFilesForAllLevels( + std::unordered_set* input_files, + const ColumnFamilyMetaData& cf_meta, const int output_level) const { + auto& levels = cf_meta.levels; + auto comparator = icmp_->user_comparator(); + + // TODO(yhchiang): If there is any input files of L1 or up and there + // is at least one L0 files. All L0 files older than the L0 file needs + // to be included. Otherwise, it is a false conditoin + + // TODO(yhchiang): add is_adjustable to CompactionOptions + + // the smallest and largest key of the current compaction input + std::string smallestkey; + std::string largestkey; + // a flag for initializing smallest and largest key + bool is_first = false; + const int kNotFound = -1; + + // For each level, it does the following things: + // 1. Find the first and the last compaction input files + // in the current level. + // 2. Include all files between the first and the last + // compaction input files. + // 3. Update the compaction key-range. + // 4. For all remaining levels, include files that have + // overlapping key-range with the compaction key-range. + for (int l = 0; l <= output_level; ++l) { + auto& current_files = levels[l].files; + int first_included = static_cast(current_files.size()); + int last_included = kNotFound; + + // identify the first and the last compaction input files + // in the current level. + for (size_t f = 0; f < current_files.size(); ++f) { + if (input_files->find(TableFileNameToNumber(current_files[f].name)) != + input_files->end()) { + first_included = std::min(first_included, static_cast(f)); + last_included = std::max(last_included, static_cast(f)); + if (is_first == false) { + smallestkey = current_files[f].smallestkey; + largestkey = current_files[f].largestkey; + is_first = true; + } + } + } + if (last_included == kNotFound) { + continue; + } - // Compute the compactions needed. It is better to do it here - // and also in LogAndApply(), otherwise the values could be stale. - std::vector size_being_compacted(NumberLevels() - 1); - SizeBeingCompacted(size_being_compacted); - version->ComputeCompactionScore(size_being_compacted); + if (l != 0) { + // expend the compaction input of the current level if it + // has overlapping key-range with other non-compaction input + // files in the same level. + while (first_included > 0) { + if (comparator->Compare(current_files[first_included - 1].largestkey, + current_files[first_included].smallestkey) < + 0) { + break; + } + first_included--; + } - // We prefer compactions triggered by too much data in a level over - // the compactions triggered by seeks. - // - // Find the compactions by size on all levels. - for (int i = 0; i < NumberLevels() - 1; i++) { - assert(i == 0 || - version->compaction_score_[i] <= version->compaction_score_[i - 1]); - level = version->compaction_level_[i]; - if ((version->compaction_score_[i] >= 1)) { - c = PickCompactionBySize(version, level, version->compaction_score_[i]); - if (c == nullptr || ExpandWhileOverlapping(c) == false) { - delete c; - c = nullptr; - } else { - break; + while (last_included < static_cast(current_files.size()) - 1) { + if (comparator->Compare(current_files[last_included + 1].smallestkey, + current_files[last_included].largestkey) > 0) { + break; + } + last_included++; + } + } + + // include all files between the first and the last compaction input files. + for (int f = first_included; f <= last_included; ++f) { + if (current_files[f].being_compacted) { + return Status::Aborted("Necessary compaction input file " + + current_files[f].name + + " is currently being compacted."); + } + input_files->insert(TableFileNameToNumber(current_files[f].name)); + } + + // update smallest and largest key + if (l == 0) { + for (int f = first_included; f <= last_included; ++f) { + if (comparator->Compare(smallestkey, current_files[f].smallestkey) > + 0) { + smallestkey = current_files[f].smallestkey; + } + if (comparator->Compare(largestkey, current_files[f].largestkey) < 0) { + largestkey = current_files[f].largestkey; + } + } + } else { + if (comparator->Compare(smallestkey, + current_files[first_included].smallestkey) > 0) { + smallestkey = current_files[first_included].smallestkey; + } + if (comparator->Compare(largestkey, + current_files[last_included].largestkey) < 0) { + largestkey = current_files[last_included].largestkey; + } + } + + SstFileMetaData aggregated_file_meta; + aggregated_file_meta.smallestkey = smallestkey; + aggregated_file_meta.largestkey = largestkey; + + // For all lower levels, include all overlapping files. + // We need to add overlapping files from the current level too because even + // if there no input_files in level l, we would still need to add files + // which overlap with the range containing the input_files in levels 0 to l + // Level 0 doesn't need to be handled this way because files are sorted by + // time and not by key + for (int m = std::max(l, 1); m <= output_level; ++m) { + for (auto& next_lv_file : levels[m].files) { + if (HaveOverlappingKeyRanges(comparator, aggregated_file_meta, + next_lv_file)) { + if (next_lv_file.being_compacted) { + return Status::Aborted( + "File " + next_lv_file.name + + " that has overlapping key range with one of the compaction " + " input file is currently being compacted."); + } + input_files->insert(TableFileNameToNumber(next_lv_file.name)); + } } } } + return Status::OK(); +} - if (c == nullptr) { - return nullptr; +Status CompactionPicker::SanitizeCompactionInputFiles( + std::unordered_set* input_files, + const ColumnFamilyMetaData& cf_meta, const int output_level) const { + assert(static_cast(cf_meta.levels.size()) - 1 == + cf_meta.levels[cf_meta.levels.size() - 1].level); + if (output_level >= static_cast(cf_meta.levels.size())) { + return Status::InvalidArgument( + "Output level for column family " + cf_meta.name + + " must between [0, " + + ToString(cf_meta.levels[cf_meta.levels.size() - 1].level) + "]."); } - // Two level 0 compaction won't run at the same time, so don't need to worry - // about files on level 0 being compacted. - if (level == 0) { - assert(compactions_in_progress_[0].empty()); - InternalKey smallest, largest; - GetRange(c->inputs_[0].files, &smallest, &largest); - // Note that the next call will discard the file we placed in - // c->inputs_[0] earlier and replace it with an overlapping set - // which will include the picked file. - c->inputs_[0].clear(); - c->input_version_->GetOverlappingInputs(0, &smallest, &largest, - &c->inputs_[0].files); + if (output_level > MaxOutputLevel()) { + return Status::InvalidArgument( + "Exceed the maximum output level defined by " + "the current compaction algorithm --- " + + ToString(MaxOutputLevel())); + } - // If we include more L0 files in the same compaction run it can - // cause the 'smallest' and 'largest' key to get extended to a - // larger range. So, re-invoke GetRange to get the new key range - GetRange(c->inputs_[0].files, &smallest, &largest); - if (ParentRangeInCompaction(c->input_version_, &smallest, &largest, level, - &c->parent_index_)) { - delete c; - return nullptr; - } - assert(!c->inputs_[0].empty()); + if (output_level < 0) { + return Status::InvalidArgument("Output level cannot be negative."); } - // Setup "level+1" files (inputs_[1]) - SetupOtherInputs(c); + if (input_files->size() == 0) { + return Status::InvalidArgument( + "A compaction must contain at least one file."); + } - // mark all the files that are being compacted - c->MarkFilesBeingCompacted(true); + Status s = SanitizeCompactionInputFilesForAllLevels(input_files, cf_meta, + output_level); - // Is this compaction creating a file at the bottommost level - c->SetupBottomMostLevel(false); + if (!s.ok()) { + return s; + } - // remember this currently undergoing compaction - compactions_in_progress_[level].insert(c); + // for all input files, check whether the file number matches + // any currently-existing files. + for (auto file_num : *input_files) { + bool found = false; + for (auto level_meta : cf_meta.levels) { + for (auto file_meta : level_meta.files) { + if (file_num == TableFileNameToNumber(file_meta.name)) { + if (file_meta.being_compacted) { + return Status::Aborted("Specified compaction input file " + + MakeTableFileName("", file_num) + + " is already being compacted."); + } + found = true; + break; + } + } + if (found) { + break; + } + } + if (!found) { + return Status::InvalidArgument( + "Specified compaction input file " + MakeTableFileName("", file_num) + + " does not exist in column family " + cf_meta.name + "."); + } + } - return c; + return Status::OK(); } +#endif // !ROCKSDB_LITE -Compaction* LevelCompactionPicker::PickCompactionBySize(Version* version, - int level, - double score) { - Compaction* c = nullptr; +void CompactionPicker::RegisterCompaction(Compaction* c) { + if (c == nullptr) { + return; + } + assert(ioptions_.compaction_style != kCompactionStyleLevel || + c->output_level() == 0 || + !FilesRangeOverlapWithCompaction(*c->inputs(), c->output_level())); + if (c->start_level() == 0 || + ioptions_.compaction_style == kCompactionStyleUniversal) { + level0_compactions_in_progress_.insert(c); + } + compactions_in_progress_.insert(c); +} - // level 0 files are overlapping. So we cannot pick more - // than one concurrent compactions at this level. This - // could be made better by looking at key-ranges that are - // being compacted at level 0. - if (level == 0 && compactions_in_progress_[level].size() == 1) { - return nullptr; +void CompactionPicker::UnregisterCompaction(Compaction* c) { + if (c == nullptr) { + return; } + if (c->start_level() == 0 || + ioptions_.compaction_style == kCompactionStyleUniversal) { + level0_compactions_in_progress_.erase(c); + } + compactions_in_progress_.erase(c); +} - assert(level >= 0); - assert(level + 1 < NumberLevels()); - c = new Compaction(version, level, level + 1, MaxFileSizeForLevel(level + 1), - MaxGrandParentOverlapBytes(level), 0, - GetCompressionType(*options_, level + 1)); - c->score_ = score; +bool LevelCompactionPicker::NeedsCompaction( + const VersionStorageInfo* vstorage) const { + if (!vstorage->FilesMarkedForCompaction().empty()) { + return true; + } + for (int i = 0; i <= vstorage->MaxInputLevel(); i++) { + if (vstorage->CompactionScore(i) >= 1) { + return true; + } + } + return false; +} - // Pick the largest file in this level that is not already - // being compacted - std::vector& file_size = c->input_version_->files_by_size_[level]; +namespace { +// A class to build a leveled compaction step-by-step. +class LevelCompactionBuilder { + public: + LevelCompactionBuilder(const std::string& cf_name, + VersionStorageInfo* vstorage, + CompactionPicker* compaction_picker, + LogBuffer* log_buffer, + const MutableCFOptions& mutable_cf_options, + const ImmutableCFOptions& ioptions) + : cf_name_(cf_name), + vstorage_(vstorage), + compaction_picker_(compaction_picker), + log_buffer_(log_buffer), + mutable_cf_options_(mutable_cf_options), + ioptions_(ioptions) {} + + // Pick and return a compaction. + Compaction* PickCompaction(); + + // Pick the initial files to compact to the next level. (or together + // in Intra-L0 compactions) + void SetupInitialFiles(); + + // If the initial files are from L0 level, pick other L0 + // files if needed. + bool SetupOtherL0FilesIfNeeded(); + + // Based on initial files, setup other files need to be compacted + // in this compaction, accordingly. + bool SetupOtherInputsIfNeeded(); + + Compaction* GetCompaction(); + + // For the specfied level, pick a file that we want to compact. + // Returns false if there is no file to compact. + // If it returns true, inputs->files.size() will be exactly one. + // If level is 0 and there is already a compaction on that level, this + // function will return false. + bool PickFileToCompact(); + + // For L0->L0, picks the longest span of files that aren't currently + // undergoing compaction for which work-per-deleted-file decreases. The span + // always starts from the newest L0 file. + // + // Intra-L0 compaction is independent of all other files, so it can be + // performed even when L0->base_level compactions are blocked. + // + // Returns true if `inputs` is populated with a span of files to be compacted; + // otherwise, returns false. + bool PickIntraL0Compaction(); + + // If there is any file marked for compaction, put put it into inputs. + void PickFilesMarkedForCompaction(); + + const std::string& cf_name_; + VersionStorageInfo* vstorage_; + CompactionPicker* compaction_picker_; + LogBuffer* log_buffer_; + int start_level_ = -1; + int output_level_ = -1; + int parent_index_ = -1; + int base_index_ = -1; + double start_level_score_ = 0; + bool is_manual_ = false; + CompactionInputFiles start_level_inputs_; + std::vector compaction_inputs_; + CompactionInputFiles output_level_inputs_; + std::vector grandparents_; + CompactionReason compaction_reason_ = CompactionReason::kUnknown; + + const MutableCFOptions& mutable_cf_options_; + const ImmutableCFOptions& ioptions_; + // Pick a path ID to place a newly generated file, with its level + static uint32_t GetPathId(const ImmutableCFOptions& ioptions, + const MutableCFOptions& mutable_cf_options, + int level); + + static const int kMinFilesForIntraL0Compaction = 4; +}; + +void LevelCompactionBuilder::PickFilesMarkedForCompaction() { + if (vstorage_->FilesMarkedForCompaction().empty()) { + return; + } - // record the first file that is not yet compacted - int nextIndex = -1; + auto continuation = [&](std::pair level_file) { + // If it's being compacted it has nothing to do here. + // If this assert() fails that means that some function marked some + // files as being_compacted, but didn't call ComputeCompactionScore() + assert(!level_file.second->being_compacted); + start_level_ = level_file.first; + output_level_ = + (start_level_ == 0) ? vstorage_->base_level() : start_level_ + 1; + + if (start_level_ == 0 && + !compaction_picker_->level0_compactions_in_progress()->empty()) { + return false; + } - for (unsigned int i = c->input_version_->next_file_to_compact_by_size_[level]; - i < file_size.size(); i++) { - int index = file_size[i]; - FileMetaData* f = c->input_version_->files_[level][index]; + start_level_inputs_.files = {level_file.second}; + start_level_inputs_.level = start_level_; + return compaction_picker_->ExpandInputsToCleanCut(cf_name_, vstorage_, + &start_level_inputs_); + }; - // Check to verify files are arranged in descending compensated size. - assert((i == file_size.size() - 1) || - (i >= Version::number_of_files_to_sort_ - 1) || - (f->compensated_file_size >= - c->input_version_->files_[level][file_size[i + 1]]-> - compensated_file_size)); + // take a chance on a random file first + Random64 rnd(/* seed */ reinterpret_cast(vstorage_)); + size_t random_file_index = static_cast(rnd.Uniform( + static_cast(vstorage_->FilesMarkedForCompaction().size()))); - // do not pick a file to compact if it is being compacted - // from n-1 level. - if (f->being_compacted) { - continue; - } + if (continuation(vstorage_->FilesMarkedForCompaction()[random_file_index])) { + // found the compaction! + return; + } - // remember the startIndex for the next call to PickCompaction - if (nextIndex == -1) { - nextIndex = i; + for (auto& level_file : vstorage_->FilesMarkedForCompaction()) { + if (continuation(level_file)) { + // found the compaction! + return; } + } + start_level_inputs_.files.clear(); +} - // Do not pick this file if its parents at level+1 are being compacted. - // Maybe we can avoid redoing this work in SetupOtherInputs - int parent_index = -1; - if (ParentRangeInCompaction(c->input_version_, &f->smallest, &f->largest, - level, &parent_index)) { - continue; +void LevelCompactionBuilder::SetupInitialFiles() { + // Find the compactions by size on all levels. + bool skipped_l0_to_base = false; + for (int i = 0; i < compaction_picker_->NumberLevels() - 1; i++) { + start_level_score_ = vstorage_->CompactionScore(i); + start_level_ = vstorage_->CompactionScoreLevel(i); + assert(i == 0 || start_level_score_ <= vstorage_->CompactionScore(i - 1)); + if (start_level_score_ >= 1) { + if (skipped_l0_to_base && start_level_ == vstorage_->base_level()) { + // If L0->base_level compaction is pending, don't schedule further + // compaction from base level. Otherwise L0->base_level compaction + // may starve. + continue; + } + output_level_ = + (start_level_ == 0) ? vstorage_->base_level() : start_level_ + 1; + if (PickFileToCompact()) { + // found the compaction! + if (start_level_ == 0) { + // L0 score = `num L0 files` / `level0_file_num_compaction_trigger` + compaction_reason_ = CompactionReason::kLevelL0FilesNum; + } else { + // L1+ score = `Level files size` / `MaxBytesForLevel` + compaction_reason_ = CompactionReason::kLevelMaxLevelSize; + } + break; + } else { + // didn't find the compaction, clear the inputs + start_level_inputs_.clear(); + if (start_level_ == 0) { + skipped_l0_to_base = true; + // L0->base_level may be blocked due to ongoing L0->base_level + // compactions. It may also be blocked by an ongoing compaction from + // base_level downwards. + // + // In these cases, to reduce L0 file count and thus reduce likelihood + // of write stalls, we can attempt compacting a span of files within + // L0. + if (PickIntraL0Compaction()) { + output_level_ = 0; + compaction_reason_ = CompactionReason::kLevelL0FilesNum; + break; + } + } + } } - c->inputs_[0].files.push_back(f); - c->base_index_ = index; - c->parent_index_ = parent_index; - break; } - if (c->inputs_[0].empty()) { - delete c; - c = nullptr; + // if we didn't find a compaction, check if there are any files marked for + // compaction + if (start_level_inputs_.empty()) { + is_manual_ = true; + parent_index_ = base_index_ = -1; + PickFilesMarkedForCompaction(); + if (!start_level_inputs_.empty()) { + compaction_reason_ = CompactionReason::kFilesMarkedForCompaction; + } } +} - // store where to start the iteration in the next call to PickCompaction - version->next_file_to_compact_by_size_[level] = nextIndex; +bool LevelCompactionBuilder::SetupOtherL0FilesIfNeeded() { + if (start_level_ == 0 && output_level_ != 0) { + // Two level 0 compaction won't run at the same time, so don't need to worry + // about files on level 0 being compacted. + assert(compaction_picker_->level0_compactions_in_progress()->empty()); + InternalKey smallest, largest; + compaction_picker_->GetRange(start_level_inputs_, &smallest, &largest); + // Note that the next call will discard the file we placed in + // c->inputs_[0] earlier and replace it with an overlapping set + // which will include the picked file. + start_level_inputs_.files.clear(); + vstorage_->GetOverlappingInputs(0, &smallest, &largest, + &start_level_inputs_.files); - return c; + // If we include more L0 files in the same compaction run it can + // cause the 'smallest' and 'largest' key to get extended to a + // larger range. So, re-invoke GetRange to get the new key range + compaction_picker_->GetRange(start_level_inputs_, &smallest, &largest); + if (compaction_picker_->IsRangeInCompaction( + vstorage_, &smallest, &largest, output_level_, &parent_index_)) { + return false; + } + } + assert(!start_level_inputs_.files.empty()); + + return true; } -// Universal style of compaction. Pick files that are contiguous in -// time-range to compact. -// -Compaction* UniversalCompactionPicker::PickCompaction(Version* version, - LogBuffer* log_buffer) { - int level = 0; - double score = version->compaction_score_[0]; - - if ((version->files_[level].size() < - (unsigned int)options_->level0_file_num_compaction_trigger)) { - LogToBuffer(log_buffer, "[%s] Universal: nothing to do\n", - version->cfd_->GetName().c_str()); - return nullptr; - } - Version::FileSummaryStorage tmp; - LogToBuffer(log_buffer, "[%s] Universal: candidate files(%zu): %s\n", - version->cfd_->GetName().c_str(), version->files_[level].size(), - version->LevelFileSummary(&tmp, 0)); - - // Check for size amplification first. - Compaction* c; - if ((c = PickCompactionUniversalSizeAmp(version, score, log_buffer)) != - nullptr) { - LogToBuffer(log_buffer, "[%s] Universal: compacting for size amp\n", - version->cfd_->GetName().c_str()); - } else { - // Size amplification is within limits. Try reducing read - // amplification while maintaining file size ratios. - unsigned int ratio = options_->compaction_options_universal.size_ratio; - - if ((c = PickCompactionUniversalReadAmp(version, score, ratio, UINT_MAX, - log_buffer)) != nullptr) { - LogToBuffer(log_buffer, "[%s] Universal: compacting for size ratio\n", - version->cfd_->GetName().c_str()); - } else { - // Size amplification and file size ratios are within configured limits. - // If max read amplification is exceeding configured limits, then force - // compaction without looking at filesize ratios and try to reduce - // the number of files to fewer than level0_file_num_compaction_trigger. - unsigned int num_files = version->files_[level].size() - - options_->level0_file_num_compaction_trigger; - if ((c = PickCompactionUniversalReadAmp( - version, score, UINT_MAX, num_files, log_buffer)) != nullptr) { - LogToBuffer(log_buffer, "[%s] Universal: compacting for file num\n", - version->cfd_->GetName().c_str()); - } +bool LevelCompactionBuilder::SetupOtherInputsIfNeeded() { + // Setup input files from output level. For output to L0, we only compact + // spans of files that do not interact with any pending compactions, so don't + // need to consider other levels. + if (output_level_ != 0) { + output_level_inputs_.level = output_level_; + if (!compaction_picker_->SetupOtherInputs( + cf_name_, mutable_cf_options_, vstorage_, &start_level_inputs_, + &output_level_inputs_, &parent_index_, base_index_)) { + return false; + } + + compaction_inputs_.push_back(start_level_inputs_); + if (!output_level_inputs_.empty()) { + compaction_inputs_.push_back(output_level_inputs_); } + + // In some edge cases we could pick a compaction that will be compacting + // a key range that overlap with another running compaction, and both + // of them have the same output level. This could happen if + // (1) we are running a non-exclusive manual compaction + // (2) AddFile ingest a new file into the LSM tree + // We need to disallow this from happening. + if (compaction_picker_->FilesRangeOverlapWithCompaction(compaction_inputs_, + output_level_)) { + // This compaction output could potentially conflict with the output + // of a currently running compaction, we cannot run it. + return false; + } + compaction_picker_->GetGrandparents(vstorage_, start_level_inputs_, + output_level_inputs_, &grandparents_); + } else { + compaction_inputs_.push_back(start_level_inputs_); } - if (c == nullptr) { + return true; +} + +Compaction* LevelCompactionBuilder::PickCompaction() { + // Pick up the first file to start compaction. It may have been extended + // to a clean cut. + SetupInitialFiles(); + if (start_level_inputs_.empty()) { return nullptr; } - assert(c->inputs_[0].size() > 1); + assert(start_level_ >= 0 && output_level_ >= 0); - // validate that all the chosen files are non overlapping in time - FileMetaData* newerfile __attribute__((unused)) = nullptr; - for (unsigned int i = 0; i < c->inputs_[0].size(); i++) { - FileMetaData* f = c->inputs_[0][i]; - assert (f->smallest_seqno <= f->largest_seqno); - assert(newerfile == nullptr || - newerfile->smallest_seqno > f->largest_seqno); - newerfile = f; + // If it is a L0 -> base level compaction, we need to set up other L0 + // files if needed. + if (!SetupOtherL0FilesIfNeeded()) { + return nullptr; } - // Is the earliest file part of this compaction? - FileMetaData* last_file = c->input_version_->files_[level].back(); - c->bottommost_level_ = c->inputs_[0].files.back() == last_file; - - // update statistics - MeasureTime(options_->statistics.get(), - NUM_FILES_IN_SINGLE_COMPACTION, c->inputs_[0].size()); + // Pick files in the output level and expand more files in the start level + // if needed. + if (!SetupOtherInputsIfNeeded()) { + return nullptr; + } - // mark all the files that are being compacted - c->MarkFilesBeingCompacted(true); + // Form a compaction object containing the files we picked. + Compaction* c = GetCompaction(); - // remember this currently undergoing compaction - compactions_in_progress_[level].insert(c); + TEST_SYNC_POINT_CALLBACK("LevelCompactionPicker::PickCompaction:Return", c); - // Record whether this compaction includes all sst files. - // For now, it is only relevant in universal compaction mode. - c->is_full_compaction_ = - (c->inputs_[0].size() == c->input_version_->files_[0].size()); + return c; +} +Compaction* LevelCompactionBuilder::GetCompaction() { + auto c = new Compaction( + vstorage_, ioptions_, mutable_cf_options_, std::move(compaction_inputs_), + output_level_, mutable_cf_options_.MaxFileSizeForLevel(output_level_), + mutable_cf_options_.max_compaction_bytes, + GetPathId(ioptions_, mutable_cf_options_, output_level_), + GetCompressionType(ioptions_, vstorage_, mutable_cf_options_, + output_level_, vstorage_->base_level()), + std::move(grandparents_), is_manual_, start_level_score_, + false /* deletion_compaction */, compaction_reason_); + + // If it's level 0 compaction, make sure we don't execute any other level 0 + // compactions in parallel + compaction_picker_->RegisterCompaction(c); + + // Creating a compaction influences the compaction score because the score + // takes running compactions into account (by skipping files that are already + // being compacted). Since we just changed compaction score, we recalculate it + // here + vstorage_->ComputeCompactionScore(ioptions_, mutable_cf_options_); return c; } -uint32_t UniversalCompactionPicker::GetPathId(const Options& options, - uint64_t file_size) { - // Two conditions need to be satisfied: - // (1) the target path needs to be able to hold the file's size - // (2) Total size left in this and previous paths need to be not - // smaller than expected future file size before this new file is - // compacted, which is estimated based on size_ratio. - // For example, if now we are compacting files of size (1, 1, 2, 4, 8), - // we will make sure the target file, probably with size of 16, will be - // placed in a path so that eventually when new files are generated and - // compacted to (1, 1, 2, 4, 8, 16), all those files can be stored in or - // before the path we chose. - // - // TODO(sdong): now the case of multiple column families is not - // considered in this algorithm. So the target size can be violated in - // that case. We need to improve it. - uint64_t accumulated_size = 0; - uint64_t future_size = - file_size * (100 - options.compaction_options_universal.size_ratio) / 100; +/* + * Find the optimal path to place a file + * Given a level, finds the path where levels up to it will fit in levels + * up to and including this path + */ +uint32_t LevelCompactionBuilder::GetPathId( + const ImmutableCFOptions& ioptions, + const MutableCFOptions& mutable_cf_options, int level) { uint32_t p = 0; - for (; p < options.db_paths.size() - 1; p++) { - uint64_t target_size = options.db_paths[p].target_size; - if (target_size > file_size && - accumulated_size + (target_size - file_size) > future_size) { - return p; + assert(!ioptions.db_paths.empty()); + + // size remaining in the most recent path + uint64_t current_path_size = ioptions.db_paths[0].target_size; + + uint64_t level_size; + int cur_level = 0; + + level_size = mutable_cf_options.max_bytes_for_level_base; + + // Last path is the fallback + while (p < ioptions.db_paths.size() - 1) { + if (level_size <= current_path_size) { + if (cur_level == level) { + // Does desired level fit in this path? + return p; + } else { + current_path_size -= level_size; + level_size = static_cast( + level_size * mutable_cf_options.max_bytes_for_level_multiplier); + cur_level++; + continue; + } } - accumulated_size += target_size; + p++; + current_path_size = ioptions.db_paths[p].target_size; } return p; } -// -// Consider compaction files based on their size differences with -// the next file in time order. -// -Compaction* UniversalCompactionPicker::PickCompactionUniversalReadAmp( - Version* version, double score, unsigned int ratio, - unsigned int max_number_of_files_to_compact, LogBuffer* log_buffer) { - int level = 0; - - unsigned int min_merge_width = - options_->compaction_options_universal.min_merge_width; - unsigned int max_merge_width = - options_->compaction_options_universal.max_merge_width; - - // The files are sorted from newest first to oldest last. - const auto& files = version->files_[level]; - - FileMetaData* f = nullptr; - bool done = false; - int start_index = 0; - unsigned int candidate_count = 0; +bool LevelCompactionBuilder::PickFileToCompact() { + // level 0 files are overlapping. So we cannot pick more + // than one concurrent compactions at this level. This + // could be made better by looking at key-ranges that are + // being compacted at level 0. + if (start_level_ == 0 && + !compaction_picker_->level0_compactions_in_progress()->empty()) { + TEST_SYNC_POINT("LevelCompactionPicker::PickCompactionBySize:0"); + return false; + } - unsigned int max_files_to_compact = std::min(max_merge_width, - max_number_of_files_to_compact); - min_merge_width = std::max(min_merge_width, 2U); + start_level_inputs_.clear(); - // Considers a candidate file only if it is smaller than the - // total size accumulated so far. - for (unsigned int loop = 0; loop < files.size(); loop++) { + assert(start_level_ >= 0); - candidate_count = 0; + // Pick the largest file in this level that is not already + // being compacted + const std::vector& file_size = + vstorage_->FilesByCompactionPri(start_level_); + const std::vector& level_files = + vstorage_->LevelFiles(start_level_); - // Skip files that are already being compacted - for (f = nullptr; loop < files.size(); loop++) { - f = files[loop]; + unsigned int cmp_idx; + for (cmp_idx = vstorage_->NextCompactionIndex(start_level_); + cmp_idx < file_size.size(); cmp_idx++) { + int index = file_size[cmp_idx]; + auto* f = level_files[index]; - if (!f->being_compacted) { - candidate_count = 1; - break; - } - LogToBuffer(log_buffer, "[%s] Universal: file %" PRIu64 - "[%d] being compacted, skipping", - version->cfd_->GetName().c_str(), f->fd.GetNumber(), loop); - f = nullptr; - } - - // This file is not being compacted. Consider it as the - // first candidate to be compacted. - uint64_t candidate_size = f != nullptr? f->compensated_file_size : 0; - if (f != nullptr) { - char file_num_buf[kFormatFileNumberBufSize]; - FormatFileNumber(f->fd.GetNumber(), f->fd.GetPathId(), file_num_buf, - sizeof(file_num_buf)); - LogToBuffer(log_buffer, "[%s] Universal: Possible candidate file %s[%d].", - version->cfd_->GetName().c_str(), file_num_buf, loop); - } - - // Check if the suceeding files need compaction. - for (unsigned int i = loop + 1; - candidate_count < max_files_to_compact && i < files.size(); i++) { - FileMetaData* f = files[i]; - if (f->being_compacted) { - break; - } - // Pick files if the total/last candidate file size (increased by the - // specified ratio) is still larger than the next candidate file. - // candidate_size is the total size of files picked so far with the - // default kCompactionStopStyleTotalSize; with - // kCompactionStopStyleSimilarSize, it's simply the size of the last - // picked file. - uint64_t sz = (candidate_size * (100L + ratio)) /100; - if (sz < f->fd.GetFileSize()) { - break; - } - if (options_->compaction_options_universal.stop_style == kCompactionStopStyleSimilarSize) { - // Similar-size stopping rule: also check the last picked file isn't - // far larger than the next candidate file. - sz = (f->fd.GetFileSize() * (100L + ratio)) / 100; - if (sz < candidate_size) { - // If the small file we've encountered begins a run of similar-size - // files, we'll pick them up on a future iteration of the outer - // loop. If it's some lonely straggler, it'll eventually get picked - // by the last-resort read amp strategy which disregards size ratios. - break; - } - candidate_size = f->compensated_file_size; - } else { // default kCompactionStopStyleTotalSize - candidate_size += f->compensated_file_size; - } - candidate_count++; + // do not pick a file to compact if it is being compacted + // from n-1 level. + if (f->being_compacted) { + continue; } - // Found a series of consecutive files that need compaction. - if (candidate_count >= (unsigned int)min_merge_width) { - start_index = loop; - done = true; - break; - } else { - for (unsigned int i = loop; - i < loop + candidate_count && i < files.size(); i++) { - FileMetaData* f = files[i]; - LogToBuffer(log_buffer, "[%s] Universal: Skipping file %" PRIu64 - "[%d] with size %" PRIu64 - " (compensated size %" PRIu64 ") %d\n", - version->cfd_->GetName().c_str(), f->fd.GetNumber(), i, - f->fd.GetFileSize(), f->compensated_file_size, - f->being_compacted); - } + start_level_inputs_.files.push_back(f); + start_level_inputs_.level = start_level_; + if (!compaction_picker_->ExpandInputsToCleanCut(cf_name_, vstorage_, + &start_level_inputs_) || + compaction_picker_->FilesRangeOverlapWithCompaction( + {start_level_inputs_}, output_level_)) { + // A locked (pending compaction) input-level file was pulled in due to + // user-key overlap. + start_level_inputs_.clear(); + continue; } - } - if (!done || candidate_count <= 1) { - return nullptr; - } - unsigned int first_index_after = start_index + candidate_count; - // Compression is enabled if files compacted earlier already reached - // size ratio of compression. - bool enable_compression = true; - int ratio_to_compress = - options_->compaction_options_universal.compression_size_percent; - if (ratio_to_compress >= 0) { - uint64_t total_size = version->NumLevelBytes(level); - uint64_t older_file_size = 0; - for (unsigned int i = files.size() - 1; - i >= first_index_after; i--) { - older_file_size += files[i]->fd.GetFileSize(); - if (older_file_size * 100L >= total_size * (long) ratio_to_compress) { - enable_compression = false; - break; - } + + // Now that input level is fully expanded, we check whether any output files + // are locked due to pending compaction. + // + // Note we rely on ExpandInputsToCleanCut() to tell us whether any output- + // level files are locked, not just the extra ones pulled in for user-key + // overlap. + InternalKey smallest, largest; + compaction_picker_->GetRange(start_level_inputs_, &smallest, &largest); + CompactionInputFiles output_level_inputs; + output_level_inputs.level = output_level_; + vstorage_->GetOverlappingInputs(output_level_, &smallest, &largest, + &output_level_inputs.files); + if (!output_level_inputs.empty() && + !compaction_picker_->ExpandInputsToCleanCut(cf_name_, vstorage_, + &output_level_inputs)) { + start_level_inputs_.clear(); + continue; } + base_index_ = index; + break; } - uint64_t estimated_total_size = 0; - for (unsigned int i = 0; i < first_index_after; i++) { - estimated_total_size += files[i]->fd.GetFileSize(); - } - uint32_t path_id = GetPathId(*options_, estimated_total_size); + // store where to start the iteration in the next call to PickCompaction + vstorage_->SetNextCompactionIndex(start_level_, cmp_idx); - Compaction* c = new Compaction( - version, level, level, MaxFileSizeForLevel(level), LLONG_MAX, path_id, - GetCompressionType(*options_, level, enable_compression)); - c->score_ = score; - - for (unsigned int i = start_index; i < first_index_after; i++) { - FileMetaData* f = c->input_version_->files_[level][i]; - c->inputs_[0].files.push_back(f); - char file_num_buf[kFormatFileNumberBufSize]; - FormatFileNumber(f->fd.GetNumber(), f->fd.GetPathId(), file_num_buf, - sizeof(file_num_buf)); - LogToBuffer(log_buffer, - "[%s] Universal: Picking file %s[%d] " - "with size %" PRIu64 " (compensated size %" PRIu64 ")\n", - version->cfd_->GetName().c_str(), file_num_buf, i, - f->fd.GetFileSize(), f->compensated_file_size); - } - return c; + return start_level_inputs_.size() > 0; } -// Look at overall size amplification. If size amplification -// exceeeds the configured value, then do a compaction -// of the candidate files all the way upto the earliest -// base file (overrides configured values of file-size ratios, -// min_merge_width and max_merge_width). -// -Compaction* UniversalCompactionPicker::PickCompactionUniversalSizeAmp( - Version* version, double score, LogBuffer* log_buffer) { - int level = 0; - - // percentage flexibilty while reducing size amplification - uint64_t ratio = options_->compaction_options_universal. - max_size_amplification_percent; - - // The files are sorted from newest first to oldest last. - const auto& files = version->files_[level]; - - unsigned int candidate_count = 0; - uint64_t candidate_size = 0; - unsigned int start_index = 0; - FileMetaData* f = nullptr; - - // Skip files that are already being compacted - for (unsigned int loop = 0; loop < files.size() - 1; loop++) { - f = files[loop]; - if (!f->being_compacted) { - start_index = loop; // Consider this as the first candidate. - break; - } - char file_num_buf[kFormatFileNumberBufSize]; - FormatFileNumber(f->fd.GetNumber(), f->fd.GetPathId(), file_num_buf, - sizeof(file_num_buf)); - LogToBuffer(log_buffer, "[%s] Universal: skipping file %s[%d] compacted %s", - version->cfd_->GetName().c_str(), file_num_buf, loop, - " cannot be a candidate to reduce size amp.\n"); - f = nullptr; - } - if (f == nullptr) { - return nullptr; // no candidate files +bool LevelCompactionBuilder::PickIntraL0Compaction() { + start_level_inputs_.clear(); + const std::vector& level_files = + vstorage_->LevelFiles(0 /* level */); + if (level_files.size() < + static_cast( + mutable_cf_options_.level0_file_num_compaction_trigger + 2) || + level_files[0]->being_compacted) { + // If L0 isn't accumulating much files beyond the regular trigger, don't + // resort to L0->L0 compaction yet. + return false; } + return FindIntraL0Compaction(level_files, kMinFilesForIntraL0Compaction, + port::kMaxUint64, &start_level_inputs_); +} +} // namespace + +Compaction* LevelCompactionPicker::PickCompaction( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, LogBuffer* log_buffer) { + LevelCompactionBuilder builder(cf_name, vstorage, this, log_buffer, + mutable_cf_options, ioptions_); + return builder.PickCompaction(); +} - char file_num_buf[kFormatFileNumberBufSize]; - FormatFileNumber(f->fd.GetNumber(), f->fd.GetPathId(), file_num_buf, - sizeof(file_num_buf)); - LogToBuffer(log_buffer, "[%s] Universal: First candidate file %s[%d] %s", - version->cfd_->GetName().c_str(), file_num_buf, start_index, - " to reduce size amp.\n"); +#ifndef ROCKSDB_LITE +bool FIFOCompactionPicker::NeedsCompaction( + const VersionStorageInfo* vstorage) const { + const int kLevel0 = 0; + return vstorage->CompactionScore(kLevel0) >= 1; +} - // keep adding up all the remaining files - for (unsigned int loop = start_index; loop < files.size() - 1; loop++) { - f = files[loop]; - if (f->being_compacted) { - char file_num_buf[kFormatFileNumberBufSize]; - FormatFileNumber(f->fd.GetNumber(), f->fd.GetPathId(), file_num_buf, - sizeof(file_num_buf)); - LogToBuffer( - log_buffer, "[%s] Universal: Possible candidate file %s[%d] %s.", - version->cfd_->GetName().c_str(), file_num_buf, loop, - " is already being compacted. No size amp reduction possible.\n"); - return nullptr; - } - candidate_size += f->compensated_file_size; - candidate_count++; - } - if (candidate_count == 0) { - return nullptr; +namespace { +uint64_t GetTotalFilesSize( + const std::vector& files) { + uint64_t total_size = 0; + for (const auto& f : files) { + total_size += f->fd.file_size; } + return total_size; +} +} // anonymous namespace - // size of earliest file - uint64_t earliest_file_size = files.back()->fd.GetFileSize(); +Compaction* FIFOCompactionPicker::PickTTLCompaction( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, LogBuffer* log_buffer) { + assert(ioptions_.compaction_options_fifo.ttl > 0); - // size amplification = percentage of additional size - if (candidate_size * 100 < ratio * earliest_file_size) { - LogToBuffer( - log_buffer, - "[%s] Universal: size amp not needed. newer-files-total-size %" PRIu64 - "earliest-file-size %" PRIu64, - version->cfd_->GetName().c_str(), candidate_size, earliest_file_size); + const int kLevel0 = 0; + const std::vector& level_files = vstorage->LevelFiles(kLevel0); + uint64_t total_size = GetTotalFilesSize(level_files); + + int64_t _current_time; + auto status = ioptions_.env->GetCurrentTime(&_current_time); + if (!status.ok()) { + ROCKS_LOG_BUFFER(log_buffer, + "[%s] FIFO compaction: Couldn't get current time: %s. " + "Not doing compactions based on TTL. ", + cf_name.c_str(), status.ToString().c_str()); return nullptr; - } else { - LogToBuffer( - log_buffer, - "[%s] Universal: size amp needed. newer-files-total-size %" PRIu64 - "earliest-file-size %" PRIu64, - version->cfd_->GetName().c_str(), candidate_size, earliest_file_size); } - assert(start_index >= 0 && start_index < files.size() - 1); + const uint64_t current_time = static_cast(_current_time); + + std::vector inputs; + inputs.emplace_back(); + inputs[0].level = 0; + + // avoid underflow + if (current_time > ioptions_.compaction_options_fifo.ttl) { + for (auto ritr = level_files.rbegin(); ritr != level_files.rend(); ++ritr) { + auto f = *ritr; + if (f->fd.table_reader != nullptr && + f->fd.table_reader->GetTableProperties() != nullptr) { + auto creation_time = + f->fd.table_reader->GetTableProperties()->creation_time; + if (creation_time == 0 || + creation_time >= + (current_time - ioptions_.compaction_options_fifo.ttl)) { + break; + } + total_size -= f->compensated_file_size; + inputs[0].files.push_back(f); + } + } + } - // Estimate total file size - uint64_t estimated_total_size = 0; - for (unsigned int loop = start_index; loop < files.size(); loop++) { - estimated_total_size += files[loop]->fd.GetFileSize(); + // Return a nullptr and proceed to size-based FIFO compaction if: + // 1. there are no files older than ttl OR + // 2. there are a few files older than ttl, but deleting them will not bring + // the total size to be less than max_table_files_size threshold. + if (inputs[0].files.empty() || + total_size > ioptions_.compaction_options_fifo.max_table_files_size) { + return nullptr; } - uint32_t path_id = GetPathId(*options_, estimated_total_size); - // create a compaction request - // We always compact all the files, so always compress. - Compaction* c = - new Compaction(version, level, level, MaxFileSizeForLevel(level), - LLONG_MAX, path_id, GetCompressionType(*options_, level)); - c->score_ = score; - for (unsigned int loop = start_index; loop < files.size(); loop++) { - f = c->input_version_->files_[level][loop]; - c->inputs_[0].files.push_back(f); - LogToBuffer(log_buffer, - "[%s] Universal: size amp picking file %" PRIu64 "[%d] " - "with size %" PRIu64 " (compensated size %" PRIu64 ")", - version->cfd_->GetName().c_str(), - f->fd.GetNumber(), loop, - f->fd.GetFileSize(), f->compensated_file_size); + for (const auto& f : inputs[0].files) { + ROCKS_LOG_BUFFER(log_buffer, + "[%s] FIFO compaction: picking file %" PRIu64 + " with creation time %" PRIu64 " for deletion", + cf_name.c_str(), f->fd.GetNumber(), + f->fd.table_reader->GetTableProperties()->creation_time); } + + Compaction* c = new Compaction( + vstorage, ioptions_, mutable_cf_options, std::move(inputs), 0, 0, 0, 0, + kNoCompression, {}, /* is manual */ false, vstorage->CompactionScore(0), + /* is deletion compaction */ true, CompactionReason::kFIFOTtl); return c; } -Compaction* FIFOCompactionPicker::PickCompaction(Version* version, - LogBuffer* log_buffer) { - assert(version->NumberLevels() == 1); - uint64_t total_size = 0; - for (const auto& file : version->files_[0]) { - total_size += file->compensated_file_size; - } +Compaction* FIFOCompactionPicker::PickSizeCompaction( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, LogBuffer* log_buffer) { + const int kLevel0 = 0; + const std::vector& level_files = vstorage->LevelFiles(kLevel0); + uint64_t total_size = GetTotalFilesSize(level_files); - if (total_size <= options_->compaction_options_fifo.max_table_files_size || - version->files_[0].size() == 0) { + if (total_size <= ioptions_.compaction_options_fifo.max_table_files_size || + level_files.size() == 0) { // total size not exceeded - LogToBuffer(log_buffer, - "[%s] FIFO compaction: nothing to do. Total size %" PRIu64 - ", max size %" PRIu64 "\n", - version->cfd_->GetName().c_str(), total_size, - options_->compaction_options_fifo.max_table_files_size); + if (ioptions_.compaction_options_fifo.allow_compaction && + level_files.size() > 0) { + CompactionInputFiles comp_inputs; + if (FindIntraL0Compaction( + level_files, + mutable_cf_options + .level0_file_num_compaction_trigger /* min_files_to_compact */, + mutable_cf_options.write_buffer_size, &comp_inputs)) { + Compaction* c = new Compaction( + vstorage, ioptions_, mutable_cf_options, {comp_inputs}, 0, + 16 * 1024 * 1024 /* output file size limit */, + 0 /* max compaction bytes, not applicable */, + 0 /* output path ID */, mutable_cf_options.compression, {}, + /* is manual */ false, vstorage->CompactionScore(0), + /* is deletion compaction */ false, + CompactionReason::kFIFOReduceNumFiles); + return c; + } + } + + ROCKS_LOG_BUFFER(log_buffer, + "[%s] FIFO compaction: nothing to do. Total size %" PRIu64 + ", max size %" PRIu64 "\n", + cf_name.c_str(), total_size, + ioptions_.compaction_options_fifo.max_table_files_size); return nullptr; } - if (compactions_in_progress_[0].size() > 0) { - LogToBuffer(log_buffer, - "[%s] FIFO compaction: Already executing compaction. No need " - "to run parallel compactions since compactions are very fast", - version->cfd_->GetName().c_str()); + if (!level0_compactions_in_progress_.empty()) { + ROCKS_LOG_BUFFER( + log_buffer, + "[%s] FIFO compaction: Already executing compaction. No need " + "to run parallel compactions since compactions are very fast", + cf_name.c_str()); return nullptr; } - Compaction* c = new Compaction(version, 0, 0, 0, 0, 0, kNoCompression, false, - true /* is deletion compaction */); - // delete old files (FIFO) - for (auto ritr = version->files_[0].rbegin(); - ritr != version->files_[0].rend(); ++ritr) { + std::vector inputs; + inputs.emplace_back(); + inputs[0].level = 0; + + for (auto ritr = level_files.rbegin(); ritr != level_files.rend(); ++ritr) { auto f = *ritr; total_size -= f->compensated_file_size; - c->inputs_[0].files.push_back(f); + inputs[0].files.push_back(f); char tmp_fsize[16]; AppendHumanBytes(f->fd.GetFileSize(), tmp_fsize, sizeof(tmp_fsize)); - LogToBuffer(log_buffer, "[%s] FIFO compaction: picking file %" PRIu64 - " with size %s for deletion", - version->cfd_->GetName().c_str(), f->fd.GetNumber(), tmp_fsize); - if (total_size <= options_->compaction_options_fifo.max_table_files_size) { + ROCKS_LOG_BUFFER(log_buffer, + "[%s] FIFO compaction: picking file %" PRIu64 + " with size %s for deletion", + cf_name.c_str(), f->fd.GetNumber(), tmp_fsize); + if (total_size <= ioptions_.compaction_options_fifo.max_table_files_size) { break; } } - c->MarkFilesBeingCompacted(true); - compactions_in_progress_[0].insert(c); + Compaction* c = new Compaction( + vstorage, ioptions_, mutable_cf_options, std::move(inputs), 0, 0, 0, 0, + kNoCompression, {}, /* is manual */ false, vstorage->CompactionScore(0), + /* is deletion compaction */ true, CompactionReason::kFIFOMaxSize); + return c; +} + +Compaction* FIFOCompactionPicker::PickCompaction( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, LogBuffer* log_buffer) { + assert(vstorage->num_levels() == 1); + Compaction* c = nullptr; + if (ioptions_.compaction_options_fifo.ttl > 0) { + c = PickTTLCompaction(cf_name, mutable_cf_options, vstorage, log_buffer); + } + if (c == nullptr) { + c = PickSizeCompaction(cf_name, mutable_cf_options, vstorage, log_buffer); + } + RegisterCompaction(c); return c; } Compaction* FIFOCompactionPicker::CompactRange( - Version* version, int input_level, int output_level, + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, int input_level, int output_level, uint32_t output_path_id, const InternalKey* begin, const InternalKey* end, - InternalKey** compaction_end) { + InternalKey** compaction_end, bool* manual_conflict) { assert(input_level == 0); assert(output_level == 0); *compaction_end = nullptr; - LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, options_->info_log.get()); - Compaction* c = PickCompaction(version, &log_buffer); - if (c != nullptr) { - assert(output_path_id < static_cast(options_->db_paths.size())); - c->output_path_id_ = output_path_id; - } + LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, ioptions_.info_log); + Compaction* c = + PickCompaction(cf_name, mutable_cf_options, vstorage, &log_buffer); log_buffer.FlushBufferToLog(); return c; } +#endif // !ROCKSDB_LITE + } // namespace rocksdb diff --git a/db/compaction_picker.h b/db/compaction_picker.h index c1e27c4718c..f44139c2dd9 100644 --- a/db/compaction_picker.h +++ b/db/compaction_picker.h @@ -1,39 +1,47 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #pragma once -#include "db/version_set.h" -#include "db/compaction.h" -#include "rocksdb/status.h" -#include "rocksdb/options.h" -#include "rocksdb/env.h" -#include #include #include +#include +#include +#include + +#include "db/compaction.h" +#include "db/version_set.h" +#include "options/cf_options.h" +#include "rocksdb/env.h" +#include "rocksdb/options.h" +#include "rocksdb/status.h" namespace rocksdb { class LogBuffer; class Compaction; -class Version; +class VersionStorageInfo; +struct CompactionInputFiles; class CompactionPicker { public: - CompactionPicker(const Options* options, const InternalKeyComparator* icmp); + CompactionPicker(const ImmutableCFOptions& ioptions, + const InternalKeyComparator* icmp); virtual ~CompactionPicker(); // Pick level and inputs for a new compaction. // Returns nullptr if there is no compaction to be done. // Otherwise returns a pointer to a heap-allocated object that // describes the compaction. Caller should delete the result. - virtual Compaction* PickCompaction(Version* version, + virtual Compaction* PickCompaction(const std::string& cf_name, + const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, LogBuffer* log_buffer) = 0; // Return a compaction object for compacting the range [begin,end] in @@ -47,49 +55,84 @@ class CompactionPicker { // compaction_end will be set to nullptr. // Client is responsible for compaction_end storage -- when called, // *compaction_end should point to valid InternalKey! - virtual Compaction* CompactRange(Version* version, int input_level, - int output_level, uint32_t output_path_id, - const InternalKey* begin, - const InternalKey* end, - InternalKey** compaction_end); - - // Given the current number of levels, returns the lowest allowed level - // for compaction input. - virtual int MaxInputLevel(int current_num_levels) const = 0; + virtual Compaction* CompactRange( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, int input_level, int output_level, + uint32_t output_path_id, const InternalKey* begin, const InternalKey* end, + InternalKey** compaction_end, bool* manual_conflict); + + // The maximum allowed output level. Default value is NumberLevels() - 1. + virtual int MaxOutputLevel() const { return NumberLevels() - 1; } + + virtual bool NeedsCompaction(const VersionStorageInfo* vstorage) const = 0; + +// Sanitize the input set of compaction input files. +// When the input parameters do not describe a valid compaction, the +// function will try to fix the input_files by adding necessary +// files. If it's not possible to conver an invalid input_files +// into a valid one by adding more files, the function will return a +// non-ok status with specific reason. +#ifndef ROCKSDB_LITE + Status SanitizeCompactionInputFiles(std::unordered_set* input_files, + const ColumnFamilyMetaData& cf_meta, + const int output_level) const; +#endif // ROCKSDB_LITE // Free up the files that participated in a compaction + // + // Requirement: DB mutex held void ReleaseCompactionFiles(Compaction* c, Status status); - // Return the total amount of data that is undergoing - // compactions per level - void SizeBeingCompacted(std::vector& sizes); - - // Returns maximum total overlap bytes with grandparent - // level (i.e., level+2) before we stop building a single - // file in level->level+1 compaction. - uint64_t MaxGrandParentOverlapBytes(int level); - - // Returns maximum total bytes of data on a given level. - double MaxBytesForLevel(int level); + // Returns true if any one of the specified files are being compacted + bool AreFilesInCompaction(const std::vector& files); + + // Takes a list of CompactionInputFiles and returns a (manual) Compaction + // object. + Compaction* CompactFiles(const CompactionOptions& compact_options, + const std::vector& input_files, + int output_level, VersionStorageInfo* vstorage, + const MutableCFOptions& mutable_cf_options, + uint32_t output_path_id); + + // Converts a set of compaction input file numbers into + // a list of CompactionInputFiles. + Status GetCompactionInputsFromFileNumbers( + std::vector* input_files, + std::unordered_set* input_set, + const VersionStorageInfo* vstorage, + const CompactionOptions& compact_options) const; + + // Is there currently a compaction involving level 0 taking place + bool IsLevel0CompactionInProgress() const { + return !level0_compactions_in_progress_.empty(); + } - // Get the max file size in a given level. - uint64_t MaxFileSizeForLevel(int level) const; - - protected: - int NumberLevels() const { return num_levels_; } + // Return true if the passed key range overlap with a compaction output + // that is currently running. + bool RangeOverlapWithCompaction(const Slice& smallest_user_key, + const Slice& largest_user_key, + int level) const; // Stores the minimal range that covers all entries in inputs in // *smallest, *largest. // REQUIRES: inputs is not empty - void GetRange(const std::vector& inputs, InternalKey* smallest, - InternalKey* largest); + void GetRange(const CompactionInputFiles& inputs, InternalKey* smallest, + InternalKey* largest) const; // Stores the minimal range that covers all entries in inputs1 and inputs2 // in *smallest, *largest. // REQUIRES: inputs is not empty - void GetRange(const std::vector& inputs1, - const std::vector& inputs2, - InternalKey* smallest, InternalKey* largest); + void GetRange(const CompactionInputFiles& inputs1, + const CompactionInputFiles& inputs2, InternalKey* smallest, + InternalKey* largest) const; + + // Stores the minimal range that covers all entries in inputs + // in *smallest, *largest. + // REQUIRES: inputs is not empty (at least on entry have one file) + void GetRange(const std::vector& inputs, + InternalKey* smallest, InternalKey* largest) const; + + int NumberLevels() const { return ioptions_.num_levels; } // Add more files to the inputs on "level" to make sure that // no newer version of a key is compacted to "level+1" while leaving an older @@ -101,110 +144,155 @@ class CompactionPicker { // populated. // // Will return false if it is impossible to apply this compaction. - bool ExpandWhileOverlapping(Compaction* c); - - uint64_t ExpandedCompactionByteSizeLimit(int level); - - // Returns true if any one of the specified files are being compacted - bool FilesInCompaction(std::vector& files); + bool ExpandInputsToCleanCut(const std::string& cf_name, + VersionStorageInfo* vstorage, + CompactionInputFiles* inputs); // Returns true if any one of the parent files are being compacted - bool ParentRangeInCompaction(Version* version, const InternalKey* smallest, - const InternalKey* largest, int level, - int* index); - - void SetupOtherInputs(Compaction* c); - - // record all the ongoing compactions for all levels - std::vector> compactions_in_progress_; + bool IsRangeInCompaction(VersionStorageInfo* vstorage, + const InternalKey* smallest, + const InternalKey* largest, int level, int* index); + + // Returns true if the key range that `inputs` files cover overlap with the + // key range of a currently running compaction. + bool FilesRangeOverlapWithCompaction( + const std::vector& inputs, int level) const; + + bool SetupOtherInputs(const std::string& cf_name, + const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, + CompactionInputFiles* inputs, + CompactionInputFiles* output_level_inputs, + int* parent_index, int base_index); + + void GetGrandparents(VersionStorageInfo* vstorage, + const CompactionInputFiles& inputs, + const CompactionInputFiles& output_level_inputs, + std::vector* grandparents); + + // Register this compaction in the set of running compactions + void RegisterCompaction(Compaction* c); + + // Remove this compaction from the set of running compactions + void UnregisterCompaction(Compaction* c); + + std::set* level0_compactions_in_progress() { + return &level0_compactions_in_progress_; + } + std::unordered_set* compactions_in_progress() { + return &compactions_in_progress_; + } - // Per-level target file size. - std::unique_ptr max_file_size_; + protected: + const ImmutableCFOptions& ioptions_; - // Per-level max bytes - std::unique_ptr level_max_bytes_; +// A helper function to SanitizeCompactionInputFiles() that +// sanitizes "input_files" by adding necessary files. +#ifndef ROCKSDB_LITE + virtual Status SanitizeCompactionInputFilesForAllLevels( + std::unordered_set* input_files, + const ColumnFamilyMetaData& cf_meta, const int output_level) const; +#endif // ROCKSDB_LITE - const Options* const options_; + // Keeps track of all compactions that are running on Level0. + // Protected by DB mutex + std::set level0_compactions_in_progress_; - private: - int num_levels_; + // Keeps track of all compactions that are running. + // Protected by DB mutex + std::unordered_set compactions_in_progress_; const InternalKeyComparator* const icmp_; }; -class UniversalCompactionPicker : public CompactionPicker { +class LevelCompactionPicker : public CompactionPicker { public: - UniversalCompactionPicker(const Options* options, - const InternalKeyComparator* icmp) - : CompactionPicker(options, icmp) {} - virtual Compaction* PickCompaction(Version* version, + LevelCompactionPicker(const ImmutableCFOptions& ioptions, + const InternalKeyComparator* icmp) + : CompactionPicker(ioptions, icmp) {} + virtual Compaction* PickCompaction(const std::string& cf_name, + const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, LogBuffer* log_buffer) override; - // The maxinum allowed input level. Always return 0. - virtual int MaxInputLevel(int current_num_levels) const override { - return 0; - } - - private: - // Pick Universal compaction to limit read amplification - Compaction* PickCompactionUniversalReadAmp(Version* version, double score, - unsigned int ratio, - unsigned int num_files, - LogBuffer* log_buffer); - - // Pick Universal compaction to limit space amplification. - Compaction* PickCompactionUniversalSizeAmp(Version* version, double score, - LogBuffer* log_buffer); - - // Pick a path ID to place a newly generated file, with its estimated file - // size. - static uint32_t GetPathId(const Options& options, uint64_t file_size); + virtual bool NeedsCompaction( + const VersionStorageInfo* vstorage) const override; }; -class LevelCompactionPicker : public CompactionPicker { +#ifndef ROCKSDB_LITE +class FIFOCompactionPicker : public CompactionPicker { public: - LevelCompactionPicker(const Options* options, - const InternalKeyComparator* icmp) - : CompactionPicker(options, icmp) {} - virtual Compaction* PickCompaction(Version* version, + FIFOCompactionPicker(const ImmutableCFOptions& ioptions, + const InternalKeyComparator* icmp) + : CompactionPicker(ioptions, icmp) {} + + virtual Compaction* PickCompaction(const std::string& cf_name, + const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* version, LogBuffer* log_buffer) override; - // Returns current_num_levels - 2, meaning the last level cannot be - // compaction input level. - virtual int MaxInputLevel(int current_num_levels) const override { - return current_num_levels - 2; - } + virtual Compaction* CompactRange( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, int input_level, int output_level, + uint32_t output_path_id, const InternalKey* begin, const InternalKey* end, + InternalKey** compaction_end, bool* manual_conflict) override; + + // The maximum allowed output level. Always returns 0. + virtual int MaxOutputLevel() const override { return 0; } + + virtual bool NeedsCompaction( + const VersionStorageInfo* vstorage) const override; private: - // For the specfied level, pick a compaction. - // Returns nullptr if there is no compaction to be done. - // If level is 0 and there is already a compaction on that level, this - // function will return nullptr. - Compaction* PickCompactionBySize(Version* version, int level, double score); + Compaction* PickTTLCompaction(const std::string& cf_name, + const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* version, + LogBuffer* log_buffer); + + Compaction* PickSizeCompaction(const std::string& cf_name, + const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* version, + LogBuffer* log_buffer); }; -class FIFOCompactionPicker : public CompactionPicker { +class NullCompactionPicker : public CompactionPicker { public: - FIFOCompactionPicker(const Options* options, + NullCompactionPicker(const ImmutableCFOptions& ioptions, const InternalKeyComparator* icmp) - : CompactionPicker(options, icmp) {} - - virtual Compaction* PickCompaction(Version* version, - LogBuffer* log_buffer) override; + : CompactionPicker(ioptions, icmp) {} + virtual ~NullCompactionPicker() {} + + // Always return "nullptr" + Compaction* PickCompaction(const std::string& cf_name, + const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, + LogBuffer* log_buffer) override { + return nullptr; + } - virtual Compaction* CompactRange(Version* version, int input_level, - int output_level, uint32_t output_path_id, - const InternalKey* begin, - const InternalKey* end, - InternalKey** compaction_end) override; + // Always return "nullptr" + Compaction* CompactRange(const std::string& cf_name, + const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, int input_level, + int output_level, uint32_t output_path_id, + const InternalKey* begin, const InternalKey* end, + InternalKey** compaction_end, + bool* manual_conflict) override { + return nullptr; + } - // The maxinum allowed input level. Always return 0. - virtual int MaxInputLevel(int current_num_levels) const override { - return 0; + // Always returns false. + virtual bool NeedsCompaction( + const VersionStorageInfo* vstorage) const override { + return false; } }; +#endif // !ROCKSDB_LITE -// Utility function -extern uint64_t TotalCompensatedFileSize(const std::vector& files); +CompressionType GetCompressionType(const ImmutableCFOptions& ioptions, + const VersionStorageInfo* vstorage, + const MutableCFOptions& mutable_cf_options, + int level, int base_level, + const bool enable_compression = true); } // namespace rocksdb diff --git a/db/compaction_picker_test.cc b/db/compaction_picker_test.cc new file mode 100644 index 00000000000..bba2d073d88 --- /dev/null +++ b/db/compaction_picker_test.cc @@ -0,0 +1,1441 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/compaction_picker.h" +#include +#include +#include +#include "db/compaction.h" +#include "db/compaction_picker_universal.h" + +#include "util/logging.h" +#include "util/string_util.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +class CountingLogger : public Logger { + public: + using Logger::Logv; + virtual void Logv(const char* format, va_list ap) override { log_count++; } + size_t log_count; +}; + +class CompactionPickerTest : public testing::Test { + public: + const Comparator* ucmp_; + InternalKeyComparator icmp_; + Options options_; + ImmutableCFOptions ioptions_; + MutableCFOptions mutable_cf_options_; + LevelCompactionPicker level_compaction_picker; + std::string cf_name_; + CountingLogger logger_; + LogBuffer log_buffer_; + uint32_t file_num_; + CompactionOptionsFIFO fifo_options_; + std::unique_ptr vstorage_; + std::vector> files_; + // does not own FileMetaData + std::unordered_map> file_map_; + // input files to compaction process. + std::vector input_files_; + int compaction_level_start_; + + CompactionPickerTest() + : ucmp_(BytewiseComparator()), + icmp_(ucmp_), + ioptions_(options_), + mutable_cf_options_(options_), + level_compaction_picker(ioptions_, &icmp_), + cf_name_("dummy"), + log_buffer_(InfoLogLevel::INFO_LEVEL, &logger_), + file_num_(1), + vstorage_(nullptr) { + fifo_options_.max_table_files_size = 1; + mutable_cf_options_.RefreshDerivedOptions(ioptions_); + ioptions_.db_paths.emplace_back("dummy", + std::numeric_limits::max()); + } + + ~CompactionPickerTest() { + } + + void NewVersionStorage(int num_levels, CompactionStyle style) { + DeleteVersionStorage(); + options_.num_levels = num_levels; + vstorage_.reset(new VersionStorageInfo(&icmp_, ucmp_, options_.num_levels, + style, nullptr, false)); + vstorage_->CalculateBaseBytes(ioptions_, mutable_cf_options_); + } + + void DeleteVersionStorage() { + vstorage_.reset(); + files_.clear(); + file_map_.clear(); + input_files_.clear(); + } + + void Add(int level, uint32_t file_number, const char* smallest, + const char* largest, uint64_t file_size = 1, uint32_t path_id = 0, + SequenceNumber smallest_seq = 100, + SequenceNumber largest_seq = 100) { + assert(level < vstorage_->num_levels()); + FileMetaData* f = new FileMetaData; + f->fd = FileDescriptor(file_number, path_id, file_size); + f->smallest = InternalKey(smallest, smallest_seq, kTypeValue); + f->largest = InternalKey(largest, largest_seq, kTypeValue); + f->smallest_seqno = smallest_seq; + f->largest_seqno = largest_seq; + f->compensated_file_size = file_size; + f->refs = 0; + vstorage_->AddFile(level, f); + files_.emplace_back(f); + file_map_.insert({file_number, {f, level}}); + } + + void SetCompactionInputFilesLevels(int level_count, int start_level) { + input_files_.resize(level_count); + for (int i = 0; i < level_count; ++i) { + input_files_[i].level = start_level + i; + } + compaction_level_start_ = start_level; + } + + void AddToCompactionFiles(uint32_t file_number) { + auto iter = file_map_.find(file_number); + assert(iter != file_map_.end()); + int level = iter->second.second; + assert(level < vstorage_->num_levels()); + input_files_[level - compaction_level_start_].files.emplace_back( + iter->second.first); + } + + void UpdateVersionStorageInfo() { + vstorage_->CalculateBaseBytes(ioptions_, mutable_cf_options_); + vstorage_->UpdateFilesByCompactionPri(ioptions_.compaction_pri); + vstorage_->UpdateNumNonEmptyLevels(); + vstorage_->GenerateFileIndexer(); + vstorage_->GenerateLevelFilesBrief(); + vstorage_->ComputeCompactionScore(ioptions_, mutable_cf_options_); + vstorage_->GenerateLevel0NonOverlapping(); + vstorage_->ComputeFilesMarkedForCompaction(); + vstorage_->SetFinalized(); + } +}; + +TEST_F(CompactionPickerTest, Empty) { + NewVersionStorage(6, kCompactionStyleLevel); + UpdateVersionStorageInfo(); + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() == nullptr); +} + +TEST_F(CompactionPickerTest, Single) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + Add(0, 1U, "p", "q"); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() == nullptr); +} + +TEST_F(CompactionPickerTest, Level0Trigger) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + Add(0, 1U, "150", "200"); + Add(0, 2U, "200", "250"); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_files(0)); + ASSERT_EQ(1U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(2U, compaction->input(0, 1)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, Level1Trigger) { + NewVersionStorage(6, kCompactionStyleLevel); + Add(1, 66U, "150", "200", 1000000000U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(66U, compaction->input(0, 0)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, Level1Trigger2) { + NewVersionStorage(6, kCompactionStyleLevel); + Add(1, 66U, "150", "200", 1000000001U); + Add(1, 88U, "201", "300", 1000000000U); + Add(2, 6U, "150", "179", 1000000000U); + Add(2, 7U, "180", "220", 1000000000U); + Add(2, 8U, "221", "300", 1000000000U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(2U, compaction->num_input_files(1)); + ASSERT_EQ(66U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(6U, compaction->input(1, 0)->fd.GetNumber()); + ASSERT_EQ(7U, compaction->input(1, 1)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, LevelMaxScore) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.target_file_size_base = 10000000; + mutable_cf_options_.target_file_size_multiplier = 10; + mutable_cf_options_.max_bytes_for_level_base = 10 * 1024 * 1024; + Add(0, 1U, "150", "200", 1000000U); + // Level 1 score 1.2 + Add(1, 66U, "150", "200", 6000000U); + Add(1, 88U, "201", "300", 6000000U); + // Level 2 score 1.8. File 7 is the largest. Should be picked + Add(2, 6U, "150", "179", 60000000U); + Add(2, 7U, "180", "220", 60000001U); + Add(2, 8U, "221", "300", 60000000U); + // Level 3 score slightly larger than 1 + Add(3, 26U, "150", "170", 260000000U); + Add(3, 27U, "171", "179", 260000000U); + Add(3, 28U, "191", "220", 260000000U); + Add(3, 29U, "221", "300", 260000000U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(7U, compaction->input(0, 0)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, NeedsCompactionLevel) { + const int kLevels = 6; + const int kFileCount = 20; + + for (int level = 0; level < kLevels - 1; ++level) { + NewVersionStorage(kLevels, kCompactionStyleLevel); + uint64_t file_size = vstorage_->MaxBytesForLevel(level) * 2 / kFileCount; + for (int file_count = 1; file_count <= kFileCount; ++file_count) { + // start a brand new version in each test. + NewVersionStorage(kLevels, kCompactionStyleLevel); + for (int i = 0; i < file_count; ++i) { + Add(level, i, ToString((i + 100) * 1000).c_str(), + ToString((i + 100) * 1000 + 999).c_str(), + file_size, 0, i * 100, i * 100 + 99); + } + UpdateVersionStorageInfo(); + ASSERT_EQ(vstorage_->CompactionScoreLevel(0), level); + ASSERT_EQ(level_compaction_picker.NeedsCompaction(vstorage_.get()), + vstorage_->CompactionScore(0) >= 1); + // release the version storage + DeleteVersionStorage(); + } + } +} + +TEST_F(CompactionPickerTest, Level0TriggerDynamic) { + int num_levels = ioptions_.num_levels; + ioptions_.level_compaction_dynamic_level_bytes = true; + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + mutable_cf_options_.max_bytes_for_level_base = 200; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + NewVersionStorage(num_levels, kCompactionStyleLevel); + Add(0, 1U, "150", "200"); + Add(0, 2U, "200", "250"); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_files(0)); + ASSERT_EQ(1U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(2U, compaction->input(0, 1)->fd.GetNumber()); + ASSERT_EQ(1, static_cast(compaction->num_input_levels())); + ASSERT_EQ(num_levels - 1, compaction->output_level()); +} + +TEST_F(CompactionPickerTest, Level0TriggerDynamic2) { + int num_levels = ioptions_.num_levels; + ioptions_.level_compaction_dynamic_level_bytes = true; + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + mutable_cf_options_.max_bytes_for_level_base = 200; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + NewVersionStorage(num_levels, kCompactionStyleLevel); + Add(0, 1U, "150", "200"); + Add(0, 2U, "200", "250"); + Add(num_levels - 1, 3U, "200", "250", 300U); + + UpdateVersionStorageInfo(); + ASSERT_EQ(vstorage_->base_level(), num_levels - 2); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_files(0)); + ASSERT_EQ(1U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(2U, compaction->input(0, 1)->fd.GetNumber()); + ASSERT_EQ(1, static_cast(compaction->num_input_levels())); + ASSERT_EQ(num_levels - 2, compaction->output_level()); +} + +TEST_F(CompactionPickerTest, Level0TriggerDynamic3) { + int num_levels = ioptions_.num_levels; + ioptions_.level_compaction_dynamic_level_bytes = true; + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + mutable_cf_options_.max_bytes_for_level_base = 200; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + NewVersionStorage(num_levels, kCompactionStyleLevel); + Add(0, 1U, "150", "200"); + Add(0, 2U, "200", "250"); + Add(num_levels - 1, 3U, "200", "250", 300U); + Add(num_levels - 1, 4U, "300", "350", 3000U); + + UpdateVersionStorageInfo(); + ASSERT_EQ(vstorage_->base_level(), num_levels - 3); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_files(0)); + ASSERT_EQ(1U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(2U, compaction->input(0, 1)->fd.GetNumber()); + ASSERT_EQ(1, static_cast(compaction->num_input_levels())); + ASSERT_EQ(num_levels - 3, compaction->output_level()); +} + +TEST_F(CompactionPickerTest, Level0TriggerDynamic4) { + int num_levels = ioptions_.num_levels; + ioptions_.level_compaction_dynamic_level_bytes = true; + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + mutable_cf_options_.max_bytes_for_level_base = 200; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + + NewVersionStorage(num_levels, kCompactionStyleLevel); + Add(0, 1U, "150", "200"); + Add(0, 2U, "200", "250"); + Add(num_levels - 1, 3U, "200", "250", 300U); + Add(num_levels - 1, 4U, "300", "350", 3000U); + Add(num_levels - 3, 5U, "150", "180", 3U); + Add(num_levels - 3, 6U, "181", "300", 3U); + Add(num_levels - 3, 7U, "400", "450", 3U); + + UpdateVersionStorageInfo(); + ASSERT_EQ(vstorage_->base_level(), num_levels - 3); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_files(0)); + ASSERT_EQ(1U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(2U, compaction->input(0, 1)->fd.GetNumber()); + ASSERT_EQ(2U, compaction->num_input_files(1)); + ASSERT_EQ(num_levels - 3, compaction->level(1)); + ASSERT_EQ(5U, compaction->input(1, 0)->fd.GetNumber()); + ASSERT_EQ(6U, compaction->input(1, 1)->fd.GetNumber()); + ASSERT_EQ(2, static_cast(compaction->num_input_levels())); + ASSERT_EQ(num_levels - 3, compaction->output_level()); +} + +TEST_F(CompactionPickerTest, LevelTriggerDynamic4) { + int num_levels = ioptions_.num_levels; + ioptions_.level_compaction_dynamic_level_bytes = true; + ioptions_.compaction_pri = kMinOverlappingRatio; + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + mutable_cf_options_.max_bytes_for_level_base = 200; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + NewVersionStorage(num_levels, kCompactionStyleLevel); + Add(0, 1U, "150", "200"); + Add(num_levels - 1, 3U, "200", "250", 300U); + Add(num_levels - 1, 4U, "300", "350", 3000U); + Add(num_levels - 1, 4U, "400", "450", 3U); + Add(num_levels - 2, 5U, "150", "180", 300U); + Add(num_levels - 2, 6U, "181", "350", 500U); + Add(num_levels - 2, 7U, "400", "450", 200U); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(5U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(0, compaction->num_input_files(1)); + ASSERT_EQ(1U, compaction->num_input_levels()); + ASSERT_EQ(num_levels - 1, compaction->output_level()); +} + +// Universal and FIFO Compactions are not supported in ROCKSDB_LITE +#ifndef ROCKSDB_LITE +TEST_F(CompactionPickerTest, NeedsCompactionUniversal) { + NewVersionStorage(1, kCompactionStyleUniversal); + UniversalCompactionPicker universal_compaction_picker( + ioptions_, &icmp_); + // must return false when there's no files. + ASSERT_EQ(universal_compaction_picker.NeedsCompaction(vstorage_.get()), + false); + UpdateVersionStorageInfo(); + + // verify the trigger given different number of L0 files. + for (int i = 1; + i <= mutable_cf_options_.level0_file_num_compaction_trigger * 2; ++i) { + NewVersionStorage(1, kCompactionStyleUniversal); + Add(0, i, ToString((i + 100) * 1000).c_str(), + ToString((i + 100) * 1000 + 999).c_str(), 1000000, 0, i * 100, + i * 100 + 99); + UpdateVersionStorageInfo(); + ASSERT_EQ(level_compaction_picker.NeedsCompaction(vstorage_.get()), + vstorage_->CompactionScore(0) >= 1); + } +} + +TEST_F(CompactionPickerTest, CompactionUniversalIngestBehindReservedLevel) { + const uint64_t kFileSize = 100000; + NewVersionStorage(1, kCompactionStyleUniversal); + ioptions_.allow_ingest_behind = true; + ioptions_.num_levels = 3; + UniversalCompactionPicker universal_compaction_picker(ioptions_, &icmp_); + // must return false when there's no files. + ASSERT_EQ(universal_compaction_picker.NeedsCompaction(vstorage_.get()), + false); + + NewVersionStorage(3, kCompactionStyleUniversal); + + Add(0, 1U, "150", "200", kFileSize, 0, 500, 550); + Add(0, 2U, "201", "250", kFileSize, 0, 401, 450); + Add(0, 4U, "260", "300", kFileSize, 0, 260, 300); + Add(1, 5U, "100", "151", kFileSize, 0, 200, 251); + Add(1, 3U, "301", "350", kFileSize, 0, 101, 150); + Add(2, 6U, "120", "200", kFileSize, 0, 20, 100); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction( + universal_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + + // output level should be the one above the bottom-most + ASSERT_EQ(1, compaction->output_level()); +} +// Tests if the files can be trivially moved in multi level +// universal compaction when allow_trivial_move option is set +// In this test as the input files overlaps, they cannot +// be trivially moved. + +TEST_F(CompactionPickerTest, CannotTrivialMoveUniversal) { + const uint64_t kFileSize = 100000; + + ioptions_.compaction_options_universal.allow_trivial_move = true; + NewVersionStorage(1, kCompactionStyleUniversal); + UniversalCompactionPicker universal_compaction_picker(ioptions_, &icmp_); + // must return false when there's no files. + ASSERT_EQ(universal_compaction_picker.NeedsCompaction(vstorage_.get()), + false); + + NewVersionStorage(3, kCompactionStyleUniversal); + + Add(0, 1U, "150", "200", kFileSize, 0, 500, 550); + Add(0, 2U, "201", "250", kFileSize, 0, 401, 450); + Add(0, 4U, "260", "300", kFileSize, 0, 260, 300); + Add(1, 5U, "100", "151", kFileSize, 0, 200, 251); + Add(1, 3U, "301", "350", kFileSize, 0, 101, 150); + Add(2, 6U, "120", "200", kFileSize, 0, 20, 100); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction( + universal_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + + ASSERT_TRUE(!compaction->is_trivial_move()); +} +// Tests if the files can be trivially moved in multi level +// universal compaction when allow_trivial_move option is set +// In this test as the input files doesn't overlaps, they should +// be trivially moved. +TEST_F(CompactionPickerTest, AllowsTrivialMoveUniversal) { + const uint64_t kFileSize = 100000; + + ioptions_.compaction_options_universal.allow_trivial_move = true; + UniversalCompactionPicker universal_compaction_picker(ioptions_, &icmp_); + + NewVersionStorage(3, kCompactionStyleUniversal); + + Add(0, 1U, "150", "200", kFileSize, 0, 500, 550); + Add(0, 2U, "201", "250", kFileSize, 0, 401, 450); + Add(0, 4U, "260", "300", kFileSize, 0, 260, 300); + Add(1, 5U, "010", "080", kFileSize, 0, 200, 251); + Add(2, 3U, "301", "350", kFileSize, 0, 101, 150); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction( + universal_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + + ASSERT_TRUE(compaction->is_trivial_move()); +} + +TEST_F(CompactionPickerTest, NeedsCompactionFIFO) { + NewVersionStorage(1, kCompactionStyleFIFO); + const int kFileCount = + mutable_cf_options_.level0_file_num_compaction_trigger * 3; + const uint64_t kFileSize = 100000; + const uint64_t kMaxSize = kFileSize * kFileCount / 2; + + fifo_options_.max_table_files_size = kMaxSize; + ioptions_.compaction_options_fifo = fifo_options_; + FIFOCompactionPicker fifo_compaction_picker(ioptions_, &icmp_); + UpdateVersionStorageInfo(); + // must return false when there's no files. + ASSERT_EQ(fifo_compaction_picker.NeedsCompaction(vstorage_.get()), false); + + // verify whether compaction is needed based on the current + // size of L0 files. + uint64_t current_size = 0; + for (int i = 1; i <= kFileCount; ++i) { + NewVersionStorage(1, kCompactionStyleFIFO); + Add(0, i, ToString((i + 100) * 1000).c_str(), + ToString((i + 100) * 1000 + 999).c_str(), + kFileSize, 0, i * 100, i * 100 + 99); + current_size += kFileSize; + UpdateVersionStorageInfo(); + ASSERT_EQ(fifo_compaction_picker.NeedsCompaction(vstorage_.get()), + vstorage_->CompactionScore(0) >= 1); + } +} +#endif // ROCKSDB_LITE + +TEST_F(CompactionPickerTest, CompactionPriMinOverlapping1) { + NewVersionStorage(6, kCompactionStyleLevel); + ioptions_.compaction_pri = kMinOverlappingRatio; + mutable_cf_options_.target_file_size_base = 10000000; + mutable_cf_options_.target_file_size_multiplier = 10; + mutable_cf_options_.max_bytes_for_level_base = 10 * 1024 * 1024; + + Add(2, 6U, "150", "179", 50000000U); + Add(2, 7U, "180", "220", 50000000U); + Add(2, 8U, "321", "400", 50000000U); // File not overlapping + Add(2, 9U, "721", "800", 50000000U); + + Add(3, 26U, "150", "170", 260000000U); + Add(3, 27U, "171", "179", 260000000U); + Add(3, 28U, "191", "220", 260000000U); + Add(3, 29U, "221", "300", 260000000U); + Add(3, 30U, "750", "900", 260000000U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(1U, compaction->num_input_files(0)); + // Pick file 8 because it overlaps with 0 files on level 3. + ASSERT_EQ(8U, compaction->input(0, 0)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, CompactionPriMinOverlapping2) { + NewVersionStorage(6, kCompactionStyleLevel); + ioptions_.compaction_pri = kMinOverlappingRatio; + mutable_cf_options_.target_file_size_base = 10000000; + mutable_cf_options_.target_file_size_multiplier = 10; + mutable_cf_options_.max_bytes_for_level_base = 10 * 1024 * 1024; + + Add(2, 6U, "150", "175", + 60000000U); // Overlaps with file 26, 27, total size 521M + Add(2, 7U, "176", "200", 60000000U); // Overlaps with file 27, 28, total size + // 520M, the smalelst overlapping + Add(2, 8U, "201", "300", + 60000000U); // Overlaps with file 28, 29, total size 521M + + Add(3, 26U, "100", "110", 261000000U); + Add(3, 26U, "150", "170", 261000000U); + Add(3, 27U, "171", "179", 260000000U); + Add(3, 28U, "191", "220", 260000000U); + Add(3, 29U, "221", "300", 261000000U); + Add(3, 30U, "321", "400", 261000000U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(1U, compaction->num_input_files(0)); + // Picking file 7 because overlapping ratio is the biggest. + ASSERT_EQ(7U, compaction->input(0, 0)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, CompactionPriMinOverlapping3) { + NewVersionStorage(6, kCompactionStyleLevel); + ioptions_.compaction_pri = kMinOverlappingRatio; + mutable_cf_options_.max_bytes_for_level_base = 10000000; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + + // file 7 and 8 over lap with the same file, but file 8 is smaller so + // it will be picked. + Add(2, 6U, "150", "167", 60000000U); // Overlaps with file 26, 27 + Add(2, 7U, "168", "169", 60000000U); // Overlaps with file 27 + Add(2, 8U, "201", "300", 61000000U); // Overlaps with file 28, but the file + // itself is larger. Should be picked. + + Add(3, 26U, "160", "165", 260000000U); + Add(3, 27U, "166", "170", 260000000U); + Add(3, 28U, "180", "400", 260000000U); + Add(3, 29U, "401", "500", 260000000U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(1U, compaction->num_input_files(0)); + // Picking file 8 because overlapping ratio is the biggest. + ASSERT_EQ(8U, compaction->input(0, 0)->fd.GetNumber()); +} + +// This test exhibits the bug where we don't properly reset parent_index in +// PickCompaction() +TEST_F(CompactionPickerTest, ParentIndexResetBug) { + int num_levels = ioptions_.num_levels; + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + mutable_cf_options_.max_bytes_for_level_base = 200; + NewVersionStorage(num_levels, kCompactionStyleLevel); + Add(0, 1U, "150", "200"); // <- marked for compaction + Add(1, 3U, "400", "500", 600); // <- this one needs compacting + Add(2, 4U, "150", "200"); + Add(2, 5U, "201", "210"); + Add(2, 6U, "300", "310"); + Add(2, 7U, "400", "500"); // <- being compacted + + vstorage_->LevelFiles(2)[3]->being_compacted = true; + vstorage_->LevelFiles(0)[0]->marked_for_compaction = true; + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); +} + +// This test checks ExpandWhileOverlapping() by having overlapping user keys +// ranges (with different sequence numbers) in the input files. +TEST_F(CompactionPickerTest, OverlappingUserKeys) { + NewVersionStorage(6, kCompactionStyleLevel); + ioptions_.compaction_pri = kByCompensatedSize; + + Add(1, 1U, "100", "150", 1U); + // Overlapping user keys + Add(1, 2U, "200", "400", 1U); + Add(1, 3U, "400", "500", 1000000000U, 0, 0); + Add(2, 4U, "600", "700", 1U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(1U, compaction->num_input_levels()); + ASSERT_EQ(2U, compaction->num_input_files(0)); + ASSERT_EQ(2U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(3U, compaction->input(0, 1)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, OverlappingUserKeys2) { + NewVersionStorage(6, kCompactionStyleLevel); + // Overlapping user keys on same level and output level + Add(1, 1U, "200", "400", 1000000000U); + Add(1, 2U, "400", "500", 1U, 0, 0); + Add(2, 3U, "000", "100", 1U); + Add(2, 4U, "100", "600", 1U, 0, 0); + Add(2, 5U, "600", "700", 1U, 0, 0); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_EQ(2U, compaction->num_input_files(0)); + ASSERT_EQ(3U, compaction->num_input_files(1)); + ASSERT_EQ(1U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(2U, compaction->input(0, 1)->fd.GetNumber()); + ASSERT_EQ(3U, compaction->input(1, 0)->fd.GetNumber()); + ASSERT_EQ(4U, compaction->input(1, 1)->fd.GetNumber()); + ASSERT_EQ(5U, compaction->input(1, 2)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, OverlappingUserKeys3) { + NewVersionStorage(6, kCompactionStyleLevel); + // Chain of overlapping user key ranges (forces ExpandWhileOverlapping() to + // expand multiple times) + Add(1, 1U, "100", "150", 1U); + Add(1, 2U, "150", "200", 1U, 0, 0); + Add(1, 3U, "200", "250", 1000000000U, 0, 0); + Add(1, 4U, "250", "300", 1U, 0, 0); + Add(1, 5U, "300", "350", 1U, 0, 0); + // Output level overlaps with the beginning and the end of the chain + Add(2, 6U, "050", "100", 1U); + Add(2, 7U, "350", "400", 1U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_EQ(5U, compaction->num_input_files(0)); + ASSERT_EQ(2U, compaction->num_input_files(1)); + ASSERT_EQ(1U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(2U, compaction->input(0, 1)->fd.GetNumber()); + ASSERT_EQ(3U, compaction->input(0, 2)->fd.GetNumber()); + ASSERT_EQ(4U, compaction->input(0, 3)->fd.GetNumber()); + ASSERT_EQ(5U, compaction->input(0, 4)->fd.GetNumber()); + ASSERT_EQ(6U, compaction->input(1, 0)->fd.GetNumber()); + ASSERT_EQ(7U, compaction->input(1, 1)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, OverlappingUserKeys4) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.max_bytes_for_level_base = 1000000; + + Add(1, 1U, "100", "150", 1U); + Add(1, 2U, "150", "199", 1U, 0, 0); + Add(1, 3U, "200", "250", 1100000U, 0, 0); + Add(1, 4U, "251", "300", 1U, 0, 0); + Add(1, 5U, "300", "350", 1U, 0, 0); + + Add(2, 6U, "100", "115", 1U); + Add(2, 7U, "125", "325", 1U); + Add(2, 8U, "350", "400", 1U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(1U, compaction->num_input_files(1)); + ASSERT_EQ(3U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(7U, compaction->input(1, 0)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, OverlappingUserKeys5) { + NewVersionStorage(6, kCompactionStyleLevel); + // Overlapping user keys on same level and output level + Add(1, 1U, "200", "400", 1000000000U); + Add(1, 2U, "400", "500", 1U, 0, 0); + Add(2, 3U, "000", "100", 1U); + Add(2, 4U, "100", "600", 1U, 0, 0); + Add(2, 5U, "600", "700", 1U, 0, 0); + + vstorage_->LevelFiles(2)[2]->being_compacted = true; + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() == nullptr); +} + +TEST_F(CompactionPickerTest, OverlappingUserKeys6) { + NewVersionStorage(6, kCompactionStyleLevel); + // Overlapping user keys on same level and output level + Add(1, 1U, "200", "400", 1U, 0, 0); + Add(1, 2U, "401", "500", 1U, 0, 0); + Add(2, 3U, "000", "100", 1U); + Add(2, 4U, "100", "300", 1U, 0, 0); + Add(2, 5U, "305", "450", 1U, 0, 0); + Add(2, 6U, "460", "600", 1U, 0, 0); + Add(2, 7U, "600", "700", 1U, 0, 0); + + vstorage_->LevelFiles(1)[0]->marked_for_compaction = true; + vstorage_->LevelFiles(1)[1]->marked_for_compaction = true; + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(3U, compaction->num_input_files(1)); +} + +TEST_F(CompactionPickerTest, OverlappingUserKeys7) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.max_compaction_bytes = 100000000000u; + // Overlapping user keys on same level and output level + Add(1, 1U, "200", "400", 1U, 0, 0); + Add(1, 2U, "401", "500", 1000000000U, 0, 0); + Add(2, 3U, "100", "250", 1U); + Add(2, 4U, "300", "600", 1U, 0, 0); + Add(2, 5U, "600", "800", 1U, 0, 0); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_GE(1U, compaction->num_input_files(0)); + ASSERT_GE(2U, compaction->num_input_files(1)); + // File 5 has to be included in the compaction + ASSERT_EQ(5U, compaction->inputs(1)->back()->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, OverlappingUserKeys8) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.max_compaction_bytes = 100000000000u; + // grow the number of inputs in "level" without + // changing the number of "level+1" files we pick up + // Expand input level as much as possible + // no overlapping case + Add(1, 1U, "101", "150", 1U); + Add(1, 2U, "151", "200", 1U); + Add(1, 3U, "201", "300", 1000000000U); + Add(1, 4U, "301", "400", 1U); + Add(1, 5U, "401", "500", 1U); + Add(2, 6U, "150", "200", 1U); + Add(2, 7U, "200", "450", 1U, 0, 0); + Add(2, 8U, "500", "600", 1U); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_EQ(3U, compaction->num_input_files(0)); + ASSERT_EQ(2U, compaction->num_input_files(1)); + ASSERT_EQ(2U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(3U, compaction->input(0, 1)->fd.GetNumber()); + ASSERT_EQ(4U, compaction->input(0, 2)->fd.GetNumber()); + ASSERT_EQ(6U, compaction->input(1, 0)->fd.GetNumber()); + ASSERT_EQ(7U, compaction->input(1, 1)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, OverlappingUserKeys9) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.max_compaction_bytes = 100000000000u; + // grow the number of inputs in "level" without + // changing the number of "level+1" files we pick up + // Expand input level as much as possible + // overlapping case + Add(1, 1U, "121", "150", 1U); + Add(1, 2U, "151", "200", 1U); + Add(1, 3U, "201", "300", 1000000000U); + Add(1, 4U, "301", "400", 1U); + Add(1, 5U, "401", "500", 1U); + Add(2, 6U, "100", "120", 1U); + Add(2, 7U, "150", "200", 1U); + Add(2, 8U, "200", "450", 1U, 0, 0); + Add(2, 9U, "501", "600", 1U); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_EQ(5U, compaction->num_input_files(0)); + ASSERT_EQ(2U, compaction->num_input_files(1)); + ASSERT_EQ(1U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(2U, compaction->input(0, 1)->fd.GetNumber()); + ASSERT_EQ(3U, compaction->input(0, 2)->fd.GetNumber()); + ASSERT_EQ(4U, compaction->input(0, 3)->fd.GetNumber()); + ASSERT_EQ(7U, compaction->input(1, 0)->fd.GetNumber()); + ASSERT_EQ(8U, compaction->input(1, 1)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, OverlappingUserKeys10) { + // Locked file encountered when pulling in extra input-level files with same + // user keys. Verify we pick the next-best file from the same input level. + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.max_compaction_bytes = 100000000000u; + + // file_number 2U is largest and thus first choice. But it overlaps with + // file_number 1U which is being compacted. So instead we pick the next- + // biggest file, 3U, which is eligible for compaction. + Add(1 /* level */, 1U /* file_number */, "100" /* smallest */, + "150" /* largest */, 1U /* file_size */); + file_map_[1U].first->being_compacted = true; + Add(1 /* level */, 2U /* file_number */, "150" /* smallest */, + "200" /* largest */, 1000000000U /* file_size */, 0 /* smallest_seq */, + 0 /* largest_seq */); + Add(1 /* level */, 3U /* file_number */, "201" /* smallest */, + "250" /* largest */, 900000000U /* file_size */); + Add(2 /* level */, 4U /* file_number */, "100" /* smallest */, + "150" /* largest */, 1U /* file_size */); + Add(2 /* level */, 5U /* file_number */, "151" /* smallest */, + "200" /* largest */, 1U /* file_size */); + Add(2 /* level */, 6U /* file_number */, "201" /* smallest */, + "250" /* largest */, 1U /* file_size */); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(1U, compaction->num_input_files(1)); + ASSERT_EQ(3U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(6U, compaction->input(1, 0)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, OverlappingUserKeys11) { + // Locked file encountered when pulling in extra output-level files with same + // user keys. Expected to skip that compaction and pick the next-best choice. + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.max_compaction_bytes = 100000000000u; + + // score(L1) = 3.7 + // score(L2) = 1.85 + // There is no eligible file in L1 to compact since both candidates pull in + // file_number 5U, which overlaps with a file pending compaction (6U). The + // first eligible compaction is from L2->L3. + Add(1 /* level */, 2U /* file_number */, "151" /* smallest */, + "200" /* largest */, 1000000000U /* file_size */); + Add(1 /* level */, 3U /* file_number */, "201" /* smallest */, + "250" /* largest */, 1U /* file_size */); + Add(2 /* level */, 4U /* file_number */, "100" /* smallest */, + "149" /* largest */, 5000000000U /* file_size */); + Add(2 /* level */, 5U /* file_number */, "150" /* smallest */, + "201" /* largest */, 1U /* file_size */); + Add(2 /* level */, 6U /* file_number */, "201" /* smallest */, + "249" /* largest */, 1U /* file_size */, 0 /* smallest_seq */, + 0 /* largest_seq */); + file_map_[6U].first->being_compacted = true; + Add(3 /* level */, 7U /* file_number */, "100" /* smallest */, + "149" /* largest */, 1U /* file_size */); + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(1U, compaction->num_input_files(1)); + ASSERT_EQ(4U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(7U, compaction->input(1, 0)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, NotScheduleL1IfL0WithHigherPri1) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + mutable_cf_options_.max_bytes_for_level_base = 900000000U; + + // 6 L0 files, score 3. + Add(0, 1U, "000", "400", 1U); + Add(0, 2U, "001", "400", 1U, 0, 0); + Add(0, 3U, "001", "400", 1000000000U, 0, 0); + Add(0, 31U, "001", "400", 1000000000U, 0, 0); + Add(0, 32U, "001", "400", 1000000000U, 0, 0); + Add(0, 33U, "001", "400", 1000000000U, 0, 0); + + // L1 total size 2GB, score 2.2. If one file being comapcted, score 1.1. + Add(1, 4U, "050", "300", 1000000000U, 0, 0); + file_map_[4u].first->being_compacted = true; + Add(1, 5U, "301", "350", 1000000000U, 0, 0); + + // Output level overlaps with the beginning and the end of the chain + Add(2, 6U, "050", "100", 1U); + Add(2, 7U, "300", "400", 1U); + + // No compaction should be scheduled, if L0 has higher priority than L1 + // but L0->L1 compaction is blocked by a file in L1 being compacted. + UpdateVersionStorageInfo(); + ASSERT_EQ(0, vstorage_->CompactionScoreLevel(0)); + ASSERT_EQ(1, vstorage_->CompactionScoreLevel(1)); + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() == nullptr); +} + +TEST_F(CompactionPickerTest, NotScheduleL1IfL0WithHigherPri2) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + mutable_cf_options_.max_bytes_for_level_base = 900000000U; + + // 6 L0 files, score 3. + Add(0, 1U, "000", "400", 1U); + Add(0, 2U, "001", "400", 1U, 0, 0); + Add(0, 3U, "001", "400", 1000000000U, 0, 0); + Add(0, 31U, "001", "400", 1000000000U, 0, 0); + Add(0, 32U, "001", "400", 1000000000U, 0, 0); + Add(0, 33U, "001", "400", 1000000000U, 0, 0); + + // L1 total size 2GB, score 2.2. If one file being comapcted, score 1.1. + Add(1, 4U, "050", "300", 1000000000U, 0, 0); + Add(1, 5U, "301", "350", 1000000000U, 0, 0); + + // Output level overlaps with the beginning and the end of the chain + Add(2, 6U, "050", "100", 1U); + Add(2, 7U, "300", "400", 1U); + + // If no file in L1 being compacted, L0->L1 compaction will be scheduled. + UpdateVersionStorageInfo(); // being_compacted flag is cleared here. + ASSERT_EQ(0, vstorage_->CompactionScoreLevel(0)); + ASSERT_EQ(1, vstorage_->CompactionScoreLevel(1)); + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); +} + +TEST_F(CompactionPickerTest, NotScheduleL1IfL0WithHigherPri3) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.level0_file_num_compaction_trigger = 2; + mutable_cf_options_.max_bytes_for_level_base = 900000000U; + + // 6 L0 files, score 3. + Add(0, 1U, "000", "400", 1U); + Add(0, 2U, "001", "400", 1U, 0, 0); + Add(0, 3U, "001", "400", 1000000000U, 0, 0); + Add(0, 31U, "001", "400", 1000000000U, 0, 0); + Add(0, 32U, "001", "400", 1000000000U, 0, 0); + Add(0, 33U, "001", "400", 1000000000U, 0, 0); + + // L1 score more than 6. + Add(1, 4U, "050", "300", 1000000000U, 0, 0); + file_map_[4u].first->being_compacted = true; + Add(1, 5U, "301", "350", 1000000000U, 0, 0); + Add(1, 51U, "351", "400", 6000000000U, 0, 0); + + // Output level overlaps with the beginning and the end of the chain + Add(2, 6U, "050", "100", 1U); + Add(2, 7U, "300", "400", 1U); + + // If score in L1 is larger than L0, L1 compaction goes through despite + // there is pending L0 compaction. + UpdateVersionStorageInfo(); + ASSERT_EQ(1, vstorage_->CompactionScoreLevel(0)); + ASSERT_EQ(0, vstorage_->CompactionScoreLevel(1)); + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); +} + +TEST_F(CompactionPickerTest, EstimateCompactionBytesNeeded1) { + int num_levels = ioptions_.num_levels; + ioptions_.level_compaction_dynamic_level_bytes = false; + mutable_cf_options_.level0_file_num_compaction_trigger = 4; + mutable_cf_options_.max_bytes_for_level_base = 1000; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + NewVersionStorage(num_levels, kCompactionStyleLevel); + Add(0, 1U, "150", "200", 200); + Add(0, 2U, "150", "200", 200); + Add(0, 3U, "150", "200", 200); + // Level 1 is over target by 200 + Add(1, 4U, "400", "500", 600); + Add(1, 5U, "600", "700", 600); + // Level 2 is less than target 10000 even added size of level 1 + // Size ratio of L2/L1 is 9600 / 1200 = 8 + Add(2, 6U, "150", "200", 2500); + Add(2, 7U, "201", "210", 2000); + Add(2, 8U, "300", "310", 2600); + Add(2, 9U, "400", "500", 2500); + // Level 3 exceeds target 100,000 of 1000 + Add(3, 10U, "400", "500", 101000); + // Level 4 exceeds target 1,000,000 by 900 after adding size from level 3 + // Size ratio L4/L3 is 9.9 + // After merge from L3, L4 size is 1000900 + Add(4, 11U, "400", "500", 999900); + Add(5, 11U, "400", "500", 8007200); + + UpdateVersionStorageInfo(); + + ASSERT_EQ(200u * 9u + 10900u + 900u * 9, + vstorage_->estimated_compaction_needed_bytes()); +} + +TEST_F(CompactionPickerTest, EstimateCompactionBytesNeeded2) { + int num_levels = ioptions_.num_levels; + ioptions_.level_compaction_dynamic_level_bytes = false; + mutable_cf_options_.level0_file_num_compaction_trigger = 3; + mutable_cf_options_.max_bytes_for_level_base = 1000; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + NewVersionStorage(num_levels, kCompactionStyleLevel); + Add(0, 1U, "150", "200", 200); + Add(0, 2U, "150", "200", 200); + Add(0, 4U, "150", "200", 200); + Add(0, 5U, "150", "200", 200); + Add(0, 6U, "150", "200", 200); + // Level 1 size will be 1400 after merging with L0 + Add(1, 7U, "400", "500", 200); + Add(1, 8U, "600", "700", 200); + // Level 2 is less than target 10000 even added size of level 1 + Add(2, 9U, "150", "200", 9100); + // Level 3 over the target, but since level 4 is empty, we assume it will be + // a trivial move. + Add(3, 10U, "400", "500", 101000); + + UpdateVersionStorageInfo(); + + // estimated L1->L2 merge: 400 * (9100.0 / 1400.0 + 1.0) + ASSERT_EQ(1400u + 3000u, vstorage_->estimated_compaction_needed_bytes()); +} + +TEST_F(CompactionPickerTest, EstimateCompactionBytesNeeded3) { + int num_levels = ioptions_.num_levels; + ioptions_.level_compaction_dynamic_level_bytes = false; + mutable_cf_options_.level0_file_num_compaction_trigger = 3; + mutable_cf_options_.max_bytes_for_level_base = 1000; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + NewVersionStorage(num_levels, kCompactionStyleLevel); + Add(0, 1U, "150", "200", 2000); + Add(0, 2U, "150", "200", 2000); + Add(0, 4U, "150", "200", 2000); + Add(0, 5U, "150", "200", 2000); + Add(0, 6U, "150", "200", 1000); + // Level 1 size will be 10000 after merging with L0 + Add(1, 7U, "400", "500", 500); + Add(1, 8U, "600", "700", 500); + + Add(2, 9U, "150", "200", 10000); + + UpdateVersionStorageInfo(); + + ASSERT_EQ(10000u + 18000u, vstorage_->estimated_compaction_needed_bytes()); +} + +TEST_F(CompactionPickerTest, EstimateCompactionBytesNeededDynamicLevel) { + int num_levels = ioptions_.num_levels; + ioptions_.level_compaction_dynamic_level_bytes = true; + mutable_cf_options_.level0_file_num_compaction_trigger = 3; + mutable_cf_options_.max_bytes_for_level_base = 1000; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + NewVersionStorage(num_levels, kCompactionStyleLevel); + + // Set Last level size 50000 + // num_levels - 1 target 5000 + // num_levels - 2 is base level with target 1000 (rounded up to + // max_bytes_for_level_base). + Add(num_levels - 1, 10U, "400", "500", 50000); + + Add(0, 1U, "150", "200", 200); + Add(0, 2U, "150", "200", 200); + Add(0, 4U, "150", "200", 200); + Add(0, 5U, "150", "200", 200); + Add(0, 6U, "150", "200", 200); + // num_levels - 3 is over target by 100 + 1000 + Add(num_levels - 3, 7U, "400", "500", 550); + Add(num_levels - 3, 8U, "600", "700", 550); + // num_levels - 2 is over target by 1100 + 200 + Add(num_levels - 2, 9U, "150", "200", 5200); + + UpdateVersionStorageInfo(); + + // Merging to the second last level: (5200 / 2100 + 1) * 1100 + // Merging to the last level: (50000 / 6300 + 1) * 1300 + ASSERT_EQ(2100u + 3823u + 11617u, + vstorage_->estimated_compaction_needed_bytes()); +} + +TEST_F(CompactionPickerTest, IsBottommostLevelTest) { + // case 1: Higher levels are empty + NewVersionStorage(6, kCompactionStyleLevel); + Add(0, 1U, "a", "m"); + Add(0, 2U, "c", "z"); + Add(1, 3U, "d", "e"); + Add(1, 4U, "l", "p"); + Add(2, 5U, "g", "i"); + Add(2, 6U, "x", "z"); + UpdateVersionStorageInfo(); + SetCompactionInputFilesLevels(2, 1); + AddToCompactionFiles(3U); + AddToCompactionFiles(5U); + bool result = + Compaction::TEST_IsBottommostLevel(2, vstorage_.get(), input_files_); + ASSERT_TRUE(result); + + // case 2: Higher levels have no overlap + NewVersionStorage(6, kCompactionStyleLevel); + Add(0, 1U, "a", "m"); + Add(0, 2U, "c", "z"); + Add(1, 3U, "d", "e"); + Add(1, 4U, "l", "p"); + Add(2, 5U, "g", "i"); + Add(2, 6U, "x", "z"); + Add(3, 7U, "k", "p"); + Add(3, 8U, "t", "w"); + Add(4, 9U, "a", "b"); + Add(5, 10U, "c", "cc"); + UpdateVersionStorageInfo(); + SetCompactionInputFilesLevels(2, 1); + AddToCompactionFiles(3U); + AddToCompactionFiles(5U); + result = Compaction::TEST_IsBottommostLevel(2, vstorage_.get(), input_files_); + ASSERT_TRUE(result); + + // case 3.1: Higher levels (level 3) have overlap + NewVersionStorage(6, kCompactionStyleLevel); + Add(0, 1U, "a", "m"); + Add(0, 2U, "c", "z"); + Add(1, 3U, "d", "e"); + Add(1, 4U, "l", "p"); + Add(2, 5U, "g", "i"); + Add(2, 6U, "x", "z"); + Add(3, 7U, "e", "g"); + Add(3, 8U, "h", "k"); + Add(4, 9U, "a", "b"); + Add(5, 10U, "c", "cc"); + UpdateVersionStorageInfo(); + SetCompactionInputFilesLevels(2, 1); + AddToCompactionFiles(3U); + AddToCompactionFiles(5U); + result = Compaction::TEST_IsBottommostLevel(2, vstorage_.get(), input_files_); + ASSERT_FALSE(result); + + // case 3.2: Higher levels (level 5) have overlap + DeleteVersionStorage(); + NewVersionStorage(6, kCompactionStyleLevel); + Add(0, 1U, "a", "m"); + Add(0, 2U, "c", "z"); + Add(1, 3U, "d", "e"); + Add(1, 4U, "l", "p"); + Add(2, 5U, "g", "i"); + Add(2, 6U, "x", "z"); + Add(3, 7U, "j", "k"); + Add(3, 8U, "l", "m"); + Add(4, 9U, "a", "b"); + Add(5, 10U, "c", "cc"); + Add(5, 11U, "h", "k"); + Add(5, 12U, "y", "yy"); + Add(5, 13U, "z", "zz"); + UpdateVersionStorageInfo(); + SetCompactionInputFilesLevels(2, 1); + AddToCompactionFiles(3U); + AddToCompactionFiles(5U); + result = Compaction::TEST_IsBottommostLevel(2, vstorage_.get(), input_files_); + ASSERT_FALSE(result); + + // case 3.3: Higher levels (level 5) have overlap, but it's only overlapping + // one key ("d") + NewVersionStorage(6, kCompactionStyleLevel); + Add(0, 1U, "a", "m"); + Add(0, 2U, "c", "z"); + Add(1, 3U, "d", "e"); + Add(1, 4U, "l", "p"); + Add(2, 5U, "g", "i"); + Add(2, 6U, "x", "z"); + Add(3, 7U, "j", "k"); + Add(3, 8U, "l", "m"); + Add(4, 9U, "a", "b"); + Add(5, 10U, "c", "cc"); + Add(5, 11U, "ccc", "d"); + Add(5, 12U, "y", "yy"); + Add(5, 13U, "z", "zz"); + UpdateVersionStorageInfo(); + SetCompactionInputFilesLevels(2, 1); + AddToCompactionFiles(3U); + AddToCompactionFiles(5U); + result = Compaction::TEST_IsBottommostLevel(2, vstorage_.get(), input_files_); + ASSERT_FALSE(result); + + // Level 0 files overlap + NewVersionStorage(6, kCompactionStyleLevel); + Add(0, 1U, "s", "t"); + Add(0, 2U, "a", "m"); + Add(0, 3U, "b", "z"); + Add(0, 4U, "e", "f"); + Add(5, 10U, "y", "z"); + UpdateVersionStorageInfo(); + SetCompactionInputFilesLevels(1, 0); + AddToCompactionFiles(1U); + AddToCompactionFiles(2U); + AddToCompactionFiles(3U); + AddToCompactionFiles(4U); + result = Compaction::TEST_IsBottommostLevel(2, vstorage_.get(), input_files_); + ASSERT_FALSE(result); + + // Level 0 files don't overlap + NewVersionStorage(6, kCompactionStyleLevel); + Add(0, 1U, "s", "t"); + Add(0, 2U, "a", "m"); + Add(0, 3U, "b", "k"); + Add(0, 4U, "e", "f"); + Add(5, 10U, "y", "z"); + UpdateVersionStorageInfo(); + SetCompactionInputFilesLevels(1, 0); + AddToCompactionFiles(1U); + AddToCompactionFiles(2U); + AddToCompactionFiles(3U); + AddToCompactionFiles(4U); + result = Compaction::TEST_IsBottommostLevel(2, vstorage_.get(), input_files_); + ASSERT_TRUE(result); + + // Level 1 files overlap + NewVersionStorage(6, kCompactionStyleLevel); + Add(0, 1U, "s", "t"); + Add(0, 2U, "a", "m"); + Add(0, 3U, "b", "k"); + Add(0, 4U, "e", "f"); + Add(1, 5U, "a", "m"); + Add(1, 6U, "n", "o"); + Add(1, 7U, "w", "y"); + Add(5, 10U, "y", "z"); + UpdateVersionStorageInfo(); + SetCompactionInputFilesLevels(2, 0); + AddToCompactionFiles(1U); + AddToCompactionFiles(2U); + AddToCompactionFiles(3U); + AddToCompactionFiles(4U); + AddToCompactionFiles(5U); + AddToCompactionFiles(6U); + AddToCompactionFiles(7U); + result = Compaction::TEST_IsBottommostLevel(2, vstorage_.get(), input_files_); + ASSERT_FALSE(result); + + DeleteVersionStorage(); +} + +TEST_F(CompactionPickerTest, MaxCompactionBytesHit) { + mutable_cf_options_.max_bytes_for_level_base = 1000000u; + mutable_cf_options_.max_compaction_bytes = 800000u; + ioptions_.level_compaction_dynamic_level_bytes = false; + NewVersionStorage(6, kCompactionStyleLevel); + // A compaction should be triggered and pick file 2 and 5. + // It can expand because adding file 1 and 3, the compaction size will + // exceed mutable_cf_options_.max_bytes_for_level_base. + Add(1, 1U, "100", "150", 300000U); + Add(1, 2U, "151", "200", 300001U, 0, 0); + Add(1, 3U, "201", "250", 300000U, 0, 0); + Add(1, 4U, "251", "300", 300000U, 0, 0); + Add(2, 5U, "100", "256", 1U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(1U, compaction->num_input_files(1)); + ASSERT_EQ(2U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(5U, compaction->input(1, 0)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, MaxCompactionBytesNotHit) { + mutable_cf_options_.max_bytes_for_level_base = 800000u; + mutable_cf_options_.max_compaction_bytes = 1000000u; + ioptions_.level_compaction_dynamic_level_bytes = false; + NewVersionStorage(6, kCompactionStyleLevel); + // A compaction should be triggered and pick file 2 and 5. + // and it expands to file 1 and 3 too. + Add(1, 1U, "100", "150", 300000U); + Add(1, 2U, "151", "200", 300001U, 0, 0); + Add(1, 3U, "201", "250", 300000U, 0, 0); + Add(1, 4U, "251", "300", 300000U, 0, 0); + Add(2, 5U, "000", "251", 1U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(2U, compaction->num_input_levels()); + ASSERT_EQ(3U, compaction->num_input_files(0)); + ASSERT_EQ(1U, compaction->num_input_files(1)); + ASSERT_EQ(1U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(2U, compaction->input(0, 1)->fd.GetNumber()); + ASSERT_EQ(3U, compaction->input(0, 2)->fd.GetNumber()); + ASSERT_EQ(5U, compaction->input(1, 0)->fd.GetNumber()); +} + +TEST_F(CompactionPickerTest, IsTrivialMoveOn) { + mutable_cf_options_.max_bytes_for_level_base = 10000u; + mutable_cf_options_.max_compaction_bytes = 10001u; + ioptions_.level_compaction_dynamic_level_bytes = false; + NewVersionStorage(6, kCompactionStyleLevel); + // A compaction should be triggered and pick file 2 + Add(1, 1U, "100", "150", 3000U); + Add(1, 2U, "151", "200", 3001U); + Add(1, 3U, "201", "250", 3000U); + Add(1, 4U, "251", "300", 3000U); + + Add(3, 5U, "120", "130", 7000U); + Add(3, 6U, "170", "180", 7000U); + Add(3, 5U, "220", "230", 7000U); + Add(3, 5U, "270", "280", 7000U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_TRUE(compaction->IsTrivialMove()); +} + +TEST_F(CompactionPickerTest, IsTrivialMoveOff) { + mutable_cf_options_.max_bytes_for_level_base = 1000000u; + mutable_cf_options_.max_compaction_bytes = 10000u; + ioptions_.level_compaction_dynamic_level_bytes = false; + NewVersionStorage(6, kCompactionStyleLevel); + // A compaction should be triggered and pick all files from level 1 + Add(1, 1U, "100", "150", 300000U, 0, 0); + Add(1, 2U, "150", "200", 300000U, 0, 0); + Add(1, 3U, "200", "250", 300000U, 0, 0); + Add(1, 4U, "250", "300", 300000U, 0, 0); + + Add(3, 5U, "120", "130", 6000U); + Add(3, 6U, "140", "150", 6000U); + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_FALSE(compaction->IsTrivialMove()); +} + +TEST_F(CompactionPickerTest, CacheNextCompactionIndex) { + NewVersionStorage(6, kCompactionStyleLevel); + mutable_cf_options_.max_compaction_bytes = 100000000000u; + + Add(1 /* level */, 1U /* file_number */, "100" /* smallest */, + "149" /* largest */, 1000000000U /* file_size */); + file_map_[1U].first->being_compacted = true; + Add(1 /* level */, 2U /* file_number */, "150" /* smallest */, + "199" /* largest */, 900000000U /* file_size */); + Add(1 /* level */, 3U /* file_number */, "200" /* smallest */, + "249" /* largest */, 800000000U /* file_size */); + Add(1 /* level */, 4U /* file_number */, "250" /* smallest */, + "299" /* largest */, 700000000U /* file_size */); + Add(2 /* level */, 5U /* file_number */, "150" /* smallest */, + "199" /* largest */, 1U /* file_size */); + file_map_[5U].first->being_compacted = true; + + UpdateVersionStorageInfo(); + + std::unique_ptr compaction(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(1U, compaction->num_input_levels()); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(0U, compaction->num_input_files(1)); + ASSERT_EQ(3U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(2, vstorage_->NextCompactionIndex(1 /* level */)); + + compaction.reset(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() != nullptr); + ASSERT_EQ(1U, compaction->num_input_levels()); + ASSERT_EQ(1U, compaction->num_input_files(0)); + ASSERT_EQ(0U, compaction->num_input_files(1)); + ASSERT_EQ(4U, compaction->input(0, 0)->fd.GetNumber()); + ASSERT_EQ(3, vstorage_->NextCompactionIndex(1 /* level */)); + + compaction.reset(level_compaction_picker.PickCompaction( + cf_name_, mutable_cf_options_, vstorage_.get(), &log_buffer_)); + ASSERT_TRUE(compaction.get() == nullptr); + ASSERT_EQ(4, vstorage_->NextCompactionIndex(1 /* level */)); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/compaction_picker_universal.cc b/db/compaction_picker_universal.cc new file mode 100644 index 00000000000..14533fbcdd2 --- /dev/null +++ b/db/compaction_picker_universal.cc @@ -0,0 +1,748 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/compaction_picker_universal.h" +#ifndef ROCKSDB_LITE + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include +#include +#include +#include +#include "db/column_family.h" +#include "monitoring/statistics.h" +#include "util/filename.h" +#include "util/log_buffer.h" +#include "util/random.h" +#include "util/string_util.h" +#include "util/sync_point.h" + +namespace rocksdb { +namespace { +// Used in universal compaction when trivial move is enabled. +// This structure is used for the construction of min heap +// that contains the file meta data, the level of the file +// and the index of the file in that level + +struct InputFileInfo { + InputFileInfo() : f(nullptr) {} + + FileMetaData* f; + size_t level; + size_t index; +}; + +// Used in universal compaction when trivial move is enabled. +// This comparator is used for the construction of min heap +// based on the smallest key of the file. +struct SmallestKeyHeapComparator { + explicit SmallestKeyHeapComparator(const Comparator* ucmp) { ucmp_ = ucmp; } + + bool operator()(InputFileInfo i1, InputFileInfo i2) const { + return (ucmp_->Compare(i1.f->smallest.user_key(), + i2.f->smallest.user_key()) > 0); + } + + private: + const Comparator* ucmp_; +}; + +typedef std::priority_queue, + SmallestKeyHeapComparator> + SmallestKeyHeap; + +// This function creates the heap that is used to find if the files are +// overlapping during universal compaction when the allow_trivial_move +// is set. +SmallestKeyHeap create_level_heap(Compaction* c, const Comparator* ucmp) { + SmallestKeyHeap smallest_key_priority_q = + SmallestKeyHeap(SmallestKeyHeapComparator(ucmp)); + + InputFileInfo input_file; + + for (size_t l = 0; l < c->num_input_levels(); l++) { + if (c->num_input_files(l) != 0) { + if (l == 0 && c->start_level() == 0) { + for (size_t i = 0; i < c->num_input_files(0); i++) { + input_file.f = c->input(0, i); + input_file.level = 0; + input_file.index = i; + smallest_key_priority_q.push(std::move(input_file)); + } + } else { + input_file.f = c->input(l, 0); + input_file.level = l; + input_file.index = 0; + smallest_key_priority_q.push(std::move(input_file)); + } + } + } + return smallest_key_priority_q; +} + +#ifndef NDEBUG +// smallest_seqno and largest_seqno are set iff. `files` is not empty. +void GetSmallestLargestSeqno(const std::vector& files, + SequenceNumber* smallest_seqno, + SequenceNumber* largest_seqno) { + bool is_first = true; + for (FileMetaData* f : files) { + assert(f->smallest_seqno <= f->largest_seqno); + if (is_first) { + is_first = false; + *smallest_seqno = f->smallest_seqno; + *largest_seqno = f->largest_seqno; + } else { + if (f->smallest_seqno < *smallest_seqno) { + *smallest_seqno = f->smallest_seqno; + } + if (f->largest_seqno > *largest_seqno) { + *largest_seqno = f->largest_seqno; + } + } + } +} +#endif +} // namespace + +// Algorithm that checks to see if there are any overlapping +// files in the input +bool UniversalCompactionPicker::IsInputFilesNonOverlapping(Compaction* c) { + auto comparator = icmp_->user_comparator(); + int first_iter = 1; + + InputFileInfo prev, curr, next; + + SmallestKeyHeap smallest_key_priority_q = + create_level_heap(c, icmp_->user_comparator()); + + while (!smallest_key_priority_q.empty()) { + curr = smallest_key_priority_q.top(); + smallest_key_priority_q.pop(); + + if (first_iter) { + prev = curr; + first_iter = 0; + } else { + if (comparator->Compare(prev.f->largest.user_key(), + curr.f->smallest.user_key()) >= 0) { + // found overlapping files, return false + return false; + } + assert(comparator->Compare(curr.f->largest.user_key(), + prev.f->largest.user_key()) > 0); + prev = curr; + } + + next.f = nullptr; + + if (curr.level != 0 && curr.index < c->num_input_files(curr.level) - 1) { + next.f = c->input(curr.level, curr.index + 1); + next.level = curr.level; + next.index = curr.index + 1; + } + + if (next.f) { + smallest_key_priority_q.push(std::move(next)); + } + } + return true; +} + +bool UniversalCompactionPicker::NeedsCompaction( + const VersionStorageInfo* vstorage) const { + const int kLevel0 = 0; + return vstorage->CompactionScore(kLevel0) >= 1; +} + +void UniversalCompactionPicker::SortedRun::Dump(char* out_buf, + size_t out_buf_size, + bool print_path) const { + if (level == 0) { + assert(file != nullptr); + if (file->fd.GetPathId() == 0 || !print_path) { + snprintf(out_buf, out_buf_size, "file %" PRIu64, file->fd.GetNumber()); + } else { + snprintf(out_buf, out_buf_size, "file %" PRIu64 + "(path " + "%" PRIu32 ")", + file->fd.GetNumber(), file->fd.GetPathId()); + } + } else { + snprintf(out_buf, out_buf_size, "level %d", level); + } +} + +void UniversalCompactionPicker::SortedRun::DumpSizeInfo( + char* out_buf, size_t out_buf_size, size_t sorted_run_count) const { + if (level == 0) { + assert(file != nullptr); + snprintf(out_buf, out_buf_size, + "file %" PRIu64 "[%" ROCKSDB_PRIszt + "] " + "with size %" PRIu64 " (compensated size %" PRIu64 ")", + file->fd.GetNumber(), sorted_run_count, file->fd.GetFileSize(), + file->compensated_file_size); + } else { + snprintf(out_buf, out_buf_size, + "level %d[%" ROCKSDB_PRIszt + "] " + "with size %" PRIu64 " (compensated size %" PRIu64 ")", + level, sorted_run_count, size, compensated_file_size); + } +} + +std::vector +UniversalCompactionPicker::CalculateSortedRuns( + const VersionStorageInfo& vstorage, const ImmutableCFOptions& ioptions) { + std::vector ret; + for (FileMetaData* f : vstorage.LevelFiles(0)) { + ret.emplace_back(0, f, f->fd.GetFileSize(), f->compensated_file_size, + f->being_compacted); + } + for (int level = 1; level < vstorage.num_levels(); level++) { + uint64_t total_compensated_size = 0U; + uint64_t total_size = 0U; + bool being_compacted = false; + bool is_first = true; + for (FileMetaData* f : vstorage.LevelFiles(level)) { + total_compensated_size += f->compensated_file_size; + total_size += f->fd.GetFileSize(); + if (ioptions.compaction_options_universal.allow_trivial_move == true) { + if (f->being_compacted) { + being_compacted = f->being_compacted; + } + } else { + // Compaction always includes all files for a non-zero level, so for a + // non-zero level, all the files should share the same being_compacted + // value. + // This assumption is only valid when + // ioptions.compaction_options_universal.allow_trivial_move is false + assert(is_first || f->being_compacted == being_compacted); + } + if (is_first) { + being_compacted = f->being_compacted; + is_first = false; + } + } + if (total_compensated_size > 0) { + ret.emplace_back(level, nullptr, total_size, total_compensated_size, + being_compacted); + } + } + return ret; +} + +// Universal style of compaction. Pick files that are contiguous in +// time-range to compact. +// +Compaction* UniversalCompactionPicker::PickCompaction( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, LogBuffer* log_buffer) { + const int kLevel0 = 0; + double score = vstorage->CompactionScore(kLevel0); + std::vector sorted_runs = + CalculateSortedRuns(*vstorage, ioptions_); + + if (sorted_runs.size() == 0 || + sorted_runs.size() < + (unsigned int)mutable_cf_options.level0_file_num_compaction_trigger) { + ROCKS_LOG_BUFFER(log_buffer, "[%s] Universal: nothing to do\n", + cf_name.c_str()); + TEST_SYNC_POINT_CALLBACK("UniversalCompactionPicker::PickCompaction:Return", + nullptr); + return nullptr; + } + VersionStorageInfo::LevelSummaryStorage tmp; + ROCKS_LOG_BUFFER_MAX_SZ( + log_buffer, 3072, + "[%s] Universal: sorted runs files(%" ROCKSDB_PRIszt "): %s\n", + cf_name.c_str(), sorted_runs.size(), vstorage->LevelSummary(&tmp)); + + // Check for size amplification first. + Compaction* c; + if ((c = PickCompactionToReduceSizeAmp(cf_name, mutable_cf_options, vstorage, + score, sorted_runs, log_buffer)) != + nullptr) { + ROCKS_LOG_BUFFER(log_buffer, "[%s] Universal: compacting for size amp\n", + cf_name.c_str()); + } else { + // Size amplification is within limits. Try reducing read + // amplification while maintaining file size ratios. + unsigned int ratio = ioptions_.compaction_options_universal.size_ratio; + + if ((c = PickCompactionToReduceSortedRuns( + cf_name, mutable_cf_options, vstorage, score, ratio, UINT_MAX, + sorted_runs, log_buffer)) != nullptr) { + ROCKS_LOG_BUFFER(log_buffer, + "[%s] Universal: compacting for size ratio\n", + cf_name.c_str()); + } else { + // Size amplification and file size ratios are within configured limits. + // If max read amplification is exceeding configured limits, then force + // compaction without looking at filesize ratios and try to reduce + // the number of files to fewer than level0_file_num_compaction_trigger. + // This is guaranteed by NeedsCompaction() + assert(sorted_runs.size() >= + static_cast( + mutable_cf_options.level0_file_num_compaction_trigger)); + // Get the total number of sorted runs that are not being compacted + int num_sr_not_compacted = 0; + for (size_t i = 0; i < sorted_runs.size(); i++) { + if (sorted_runs[i].being_compacted == false) { + num_sr_not_compacted++; + } + } + + // The number of sorted runs that are not being compacted is greater than + // the maximum allowed number of sorted runs + if (num_sr_not_compacted > + mutable_cf_options.level0_file_num_compaction_trigger) { + unsigned int num_files = + num_sr_not_compacted - + mutable_cf_options.level0_file_num_compaction_trigger + 1; + if ((c = PickCompactionToReduceSortedRuns( + cf_name, mutable_cf_options, vstorage, score, UINT_MAX, + num_files, sorted_runs, log_buffer)) != nullptr) { + ROCKS_LOG_BUFFER(log_buffer, + "[%s] Universal: compacting for file num -- %u\n", + cf_name.c_str(), num_files); + } + } + } + } + if (c == nullptr) { + TEST_SYNC_POINT_CALLBACK("UniversalCompactionPicker::PickCompaction:Return", + nullptr); + return nullptr; + } + + if (ioptions_.compaction_options_universal.allow_trivial_move == true) { + c->set_is_trivial_move(IsInputFilesNonOverlapping(c)); + } + +// validate that all the chosen files of L0 are non overlapping in time +#ifndef NDEBUG + SequenceNumber prev_smallest_seqno = 0U; + bool is_first = true; + + size_t level_index = 0U; + if (c->start_level() == 0) { + for (auto f : *c->inputs(0)) { + assert(f->smallest_seqno <= f->largest_seqno); + if (is_first) { + is_first = false; + } + prev_smallest_seqno = f->smallest_seqno; + } + level_index = 1U; + } + for (; level_index < c->num_input_levels(); level_index++) { + if (c->num_input_files(level_index) != 0) { + SequenceNumber smallest_seqno = 0U; + SequenceNumber largest_seqno = 0U; + GetSmallestLargestSeqno(*(c->inputs(level_index)), &smallest_seqno, + &largest_seqno); + if (is_first) { + is_first = false; + } else if (prev_smallest_seqno > 0) { + // A level is considered as the bottommost level if there are + // no files in higher levels or if files in higher levels do + // not overlap with the files being compacted. Sequence numbers + // of files in bottommost level can be set to 0 to help + // compression. As a result, the following assert may not hold + // if the prev_smallest_seqno is 0. + assert(prev_smallest_seqno > largest_seqno); + } + prev_smallest_seqno = smallest_seqno; + } + } +#endif + // update statistics + MeasureTime(ioptions_.statistics, NUM_FILES_IN_SINGLE_COMPACTION, + c->inputs(0)->size()); + + RegisterCompaction(c); + vstorage->ComputeCompactionScore(ioptions_, mutable_cf_options); + + TEST_SYNC_POINT_CALLBACK("UniversalCompactionPicker::PickCompaction:Return", + c); + return c; +} + +uint32_t UniversalCompactionPicker::GetPathId( + const ImmutableCFOptions& ioptions, uint64_t file_size) { + // Two conditions need to be satisfied: + // (1) the target path needs to be able to hold the file's size + // (2) Total size left in this and previous paths need to be not + // smaller than expected future file size before this new file is + // compacted, which is estimated based on size_ratio. + // For example, if now we are compacting files of size (1, 1, 2, 4, 8), + // we will make sure the target file, probably with size of 16, will be + // placed in a path so that eventually when new files are generated and + // compacted to (1, 1, 2, 4, 8, 16), all those files can be stored in or + // before the path we chose. + // + // TODO(sdong): now the case of multiple column families is not + // considered in this algorithm. So the target size can be violated in + // that case. We need to improve it. + uint64_t accumulated_size = 0; + uint64_t future_size = + file_size * (100 - ioptions.compaction_options_universal.size_ratio) / + 100; + uint32_t p = 0; + assert(!ioptions.db_paths.empty()); + for (; p < ioptions.db_paths.size() - 1; p++) { + uint64_t target_size = ioptions.db_paths[p].target_size; + if (target_size > file_size && + accumulated_size + (target_size - file_size) > future_size) { + return p; + } + accumulated_size += target_size; + } + return p; +} + +// +// Consider compaction files based on their size differences with +// the next file in time order. +// +Compaction* UniversalCompactionPicker::PickCompactionToReduceSortedRuns( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, double score, unsigned int ratio, + unsigned int max_number_of_files_to_compact, + const std::vector& sorted_runs, LogBuffer* log_buffer) { + unsigned int min_merge_width = + ioptions_.compaction_options_universal.min_merge_width; + unsigned int max_merge_width = + ioptions_.compaction_options_universal.max_merge_width; + + const SortedRun* sr = nullptr; + bool done = false; + size_t start_index = 0; + unsigned int candidate_count = 0; + + unsigned int max_files_to_compact = + std::min(max_merge_width, max_number_of_files_to_compact); + min_merge_width = std::max(min_merge_width, 2U); + + // Caller checks the size before executing this function. This invariant is + // important because otherwise we may have a possible integer underflow when + // dealing with unsigned types. + assert(sorted_runs.size() > 0); + + // Considers a candidate file only if it is smaller than the + // total size accumulated so far. + for (size_t loop = 0; loop < sorted_runs.size(); loop++) { + candidate_count = 0; + + // Skip files that are already being compacted + for (sr = nullptr; loop < sorted_runs.size(); loop++) { + sr = &sorted_runs[loop]; + + if (!sr->being_compacted) { + candidate_count = 1; + break; + } + char file_num_buf[kFormatFileNumberBufSize]; + sr->Dump(file_num_buf, sizeof(file_num_buf)); + ROCKS_LOG_BUFFER(log_buffer, + "[%s] Universal: %s" + "[%d] being compacted, skipping", + cf_name.c_str(), file_num_buf, loop); + + sr = nullptr; + } + + // This file is not being compacted. Consider it as the + // first candidate to be compacted. + uint64_t candidate_size = sr != nullptr ? sr->compensated_file_size : 0; + if (sr != nullptr) { + char file_num_buf[kFormatFileNumberBufSize]; + sr->Dump(file_num_buf, sizeof(file_num_buf), true); + ROCKS_LOG_BUFFER(log_buffer, "[%s] Universal: Possible candidate %s[%d].", + cf_name.c_str(), file_num_buf, loop); + } + + // Check if the succeeding files need compaction. + for (size_t i = loop + 1; + candidate_count < max_files_to_compact && i < sorted_runs.size(); + i++) { + const SortedRun* succeeding_sr = &sorted_runs[i]; + if (succeeding_sr->being_compacted) { + break; + } + // Pick files if the total/last candidate file size (increased by the + // specified ratio) is still larger than the next candidate file. + // candidate_size is the total size of files picked so far with the + // default kCompactionStopStyleTotalSize; with + // kCompactionStopStyleSimilarSize, it's simply the size of the last + // picked file. + double sz = candidate_size * (100.0 + ratio) / 100.0; + if (sz < static_cast(succeeding_sr->size)) { + break; + } + if (ioptions_.compaction_options_universal.stop_style == + kCompactionStopStyleSimilarSize) { + // Similar-size stopping rule: also check the last picked file isn't + // far larger than the next candidate file. + sz = (succeeding_sr->size * (100.0 + ratio)) / 100.0; + if (sz < static_cast(candidate_size)) { + // If the small file we've encountered begins a run of similar-size + // files, we'll pick them up on a future iteration of the outer + // loop. If it's some lonely straggler, it'll eventually get picked + // by the last-resort read amp strategy which disregards size ratios. + break; + } + candidate_size = succeeding_sr->compensated_file_size; + } else { // default kCompactionStopStyleTotalSize + candidate_size += succeeding_sr->compensated_file_size; + } + candidate_count++; + } + + // Found a series of consecutive files that need compaction. + if (candidate_count >= (unsigned int)min_merge_width) { + start_index = loop; + done = true; + break; + } else { + for (size_t i = loop; + i < loop + candidate_count && i < sorted_runs.size(); i++) { + const SortedRun* skipping_sr = &sorted_runs[i]; + char file_num_buf[256]; + skipping_sr->DumpSizeInfo(file_num_buf, sizeof(file_num_buf), loop); + ROCKS_LOG_BUFFER(log_buffer, "[%s] Universal: Skipping %s", + cf_name.c_str(), file_num_buf); + } + } + } + if (!done || candidate_count <= 1) { + return nullptr; + } + size_t first_index_after = start_index + candidate_count; + // Compression is enabled if files compacted earlier already reached + // size ratio of compression. + bool enable_compression = true; + int ratio_to_compress = + ioptions_.compaction_options_universal.compression_size_percent; + if (ratio_to_compress >= 0) { + uint64_t total_size = 0; + for (auto& sorted_run : sorted_runs) { + total_size += sorted_run.compensated_file_size; + } + + uint64_t older_file_size = 0; + for (size_t i = sorted_runs.size() - 1; i >= first_index_after; i--) { + older_file_size += sorted_runs[i].size; + if (older_file_size * 100L >= total_size * (long)ratio_to_compress) { + enable_compression = false; + break; + } + } + } + + uint64_t estimated_total_size = 0; + for (unsigned int i = 0; i < first_index_after; i++) { + estimated_total_size += sorted_runs[i].size; + } + uint32_t path_id = GetPathId(ioptions_, estimated_total_size); + int start_level = sorted_runs[start_index].level; + int output_level; + if (first_index_after == sorted_runs.size()) { + output_level = vstorage->num_levels() - 1; + } else if (sorted_runs[first_index_after].level == 0) { + output_level = 0; + } else { + output_level = sorted_runs[first_index_after].level - 1; + } + + // last level is reserved for the files ingested behind + if (ioptions_.allow_ingest_behind && + (output_level == vstorage->num_levels() - 1)) { + assert(output_level > 1); + output_level--; + } + + std::vector inputs(vstorage->num_levels()); + for (size_t i = 0; i < inputs.size(); ++i) { + inputs[i].level = start_level + static_cast(i); + } + for (size_t i = start_index; i < first_index_after; i++) { + auto& picking_sr = sorted_runs[i]; + if (picking_sr.level == 0) { + FileMetaData* picking_file = picking_sr.file; + inputs[0].files.push_back(picking_file); + } else { + auto& files = inputs[picking_sr.level - start_level].files; + for (auto* f : vstorage->LevelFiles(picking_sr.level)) { + files.push_back(f); + } + } + char file_num_buf[256]; + picking_sr.DumpSizeInfo(file_num_buf, sizeof(file_num_buf), i); + ROCKS_LOG_BUFFER(log_buffer, "[%s] Universal: Picking %s", cf_name.c_str(), + file_num_buf); + } + + CompactionReason compaction_reason; + if (max_number_of_files_to_compact == UINT_MAX) { + compaction_reason = CompactionReason::kUniversalSortedRunNum; + } else { + compaction_reason = CompactionReason::kUniversalSizeRatio; + } + return new Compaction( + vstorage, ioptions_, mutable_cf_options, std::move(inputs), output_level, + mutable_cf_options.MaxFileSizeForLevel(output_level), LLONG_MAX, path_id, + GetCompressionType(ioptions_, vstorage, mutable_cf_options, start_level, + 1, enable_compression), + /* grandparents */ {}, /* is manual */ false, score, + false /* deletion_compaction */, compaction_reason); +} + +// Look at overall size amplification. If size amplification +// exceeeds the configured value, then do a compaction +// of the candidate files all the way upto the earliest +// base file (overrides configured values of file-size ratios, +// min_merge_width and max_merge_width). +// +Compaction* UniversalCompactionPicker::PickCompactionToReduceSizeAmp( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, double score, + const std::vector& sorted_runs, LogBuffer* log_buffer) { + // percentage flexibility while reducing size amplification + uint64_t ratio = + ioptions_.compaction_options_universal.max_size_amplification_percent; + + unsigned int candidate_count = 0; + uint64_t candidate_size = 0; + size_t start_index = 0; + const SortedRun* sr = nullptr; + + // Skip files that are already being compacted + for (size_t loop = 0; loop < sorted_runs.size() - 1; loop++) { + sr = &sorted_runs[loop]; + if (!sr->being_compacted) { + start_index = loop; // Consider this as the first candidate. + break; + } + char file_num_buf[kFormatFileNumberBufSize]; + sr->Dump(file_num_buf, sizeof(file_num_buf), true); + ROCKS_LOG_BUFFER(log_buffer, "[%s] Universal: skipping %s[%d] compacted %s", + cf_name.c_str(), file_num_buf, loop, + " cannot be a candidate to reduce size amp.\n"); + sr = nullptr; + } + + if (sr == nullptr) { + return nullptr; // no candidate files + } + { + char file_num_buf[kFormatFileNumberBufSize]; + sr->Dump(file_num_buf, sizeof(file_num_buf), true); + ROCKS_LOG_BUFFER( + log_buffer, + "[%s] Universal: First candidate %s[%" ROCKSDB_PRIszt "] %s", + cf_name.c_str(), file_num_buf, start_index, " to reduce size amp.\n"); + } + + // keep adding up all the remaining files + for (size_t loop = start_index; loop < sorted_runs.size() - 1; loop++) { + sr = &sorted_runs[loop]; + if (sr->being_compacted) { + char file_num_buf[kFormatFileNumberBufSize]; + sr->Dump(file_num_buf, sizeof(file_num_buf), true); + ROCKS_LOG_BUFFER( + log_buffer, "[%s] Universal: Possible candidate %s[%d] %s", + cf_name.c_str(), file_num_buf, start_index, + " is already being compacted. No size amp reduction possible.\n"); + return nullptr; + } + candidate_size += sr->compensated_file_size; + candidate_count++; + } + if (candidate_count == 0) { + return nullptr; + } + + // size of earliest file + uint64_t earliest_file_size = sorted_runs.back().size; + + // size amplification = percentage of additional size + if (candidate_size * 100 < ratio * earliest_file_size) { + ROCKS_LOG_BUFFER( + log_buffer, + "[%s] Universal: size amp not needed. newer-files-total-size %" PRIu64 + " earliest-file-size %" PRIu64, + cf_name.c_str(), candidate_size, earliest_file_size); + return nullptr; + } else { + ROCKS_LOG_BUFFER( + log_buffer, + "[%s] Universal: size amp needed. newer-files-total-size %" PRIu64 + " earliest-file-size %" PRIu64, + cf_name.c_str(), candidate_size, earliest_file_size); + } + assert(start_index < sorted_runs.size() - 1); + + // Estimate total file size + uint64_t estimated_total_size = 0; + for (size_t loop = start_index; loop < sorted_runs.size(); loop++) { + estimated_total_size += sorted_runs[loop].size; + } + uint32_t path_id = GetPathId(ioptions_, estimated_total_size); + int start_level = sorted_runs[start_index].level; + + std::vector inputs(vstorage->num_levels()); + for (size_t i = 0; i < inputs.size(); ++i) { + inputs[i].level = start_level + static_cast(i); + } + // We always compact all the files, so always compress. + for (size_t loop = start_index; loop < sorted_runs.size(); loop++) { + auto& picking_sr = sorted_runs[loop]; + if (picking_sr.level == 0) { + FileMetaData* f = picking_sr.file; + inputs[0].files.push_back(f); + } else { + auto& files = inputs[picking_sr.level - start_level].files; + for (auto* f : vstorage->LevelFiles(picking_sr.level)) { + files.push_back(f); + } + } + char file_num_buf[256]; + picking_sr.DumpSizeInfo(file_num_buf, sizeof(file_num_buf), loop); + ROCKS_LOG_BUFFER(log_buffer, "[%s] Universal: size amp picking %s", + cf_name.c_str(), file_num_buf); + } + + // output files at the bottom most level, unless it's reserved + int output_level = vstorage->num_levels() - 1; + // last level is reserved for the files ingested behind + if (ioptions_.allow_ingest_behind) { + assert(output_level > 1); + output_level--; + } + + return new Compaction( + vstorage, ioptions_, mutable_cf_options, std::move(inputs), + output_level, mutable_cf_options.MaxFileSizeForLevel(output_level), + /* max_grandparent_overlap_bytes */ LLONG_MAX, path_id, + GetCompressionType(ioptions_, vstorage, mutable_cf_options, + output_level, 1), + /* grandparents */ {}, /* is manual */ false, score, + false /* deletion_compaction */, + CompactionReason::kUniversalSizeAmplification); +} +} // namespace rocksdb + +#endif // !ROCKSDB_LITE diff --git a/db/compaction_picker_universal.h b/db/compaction_picker_universal.h new file mode 100644 index 00000000000..3f2bed3e621 --- /dev/null +++ b/db/compaction_picker_universal.h @@ -0,0 +1,91 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once +#ifndef ROCKSDB_LITE + +#include "db/compaction_picker.h" + +namespace rocksdb { +class UniversalCompactionPicker : public CompactionPicker { + public: + UniversalCompactionPicker(const ImmutableCFOptions& ioptions, + const InternalKeyComparator* icmp) + : CompactionPicker(ioptions, icmp) {} + virtual Compaction* PickCompaction(const std::string& cf_name, + const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, + LogBuffer* log_buffer) override; + + virtual int MaxOutputLevel() const override { return NumberLevels() - 1; } + + virtual bool NeedsCompaction( + const VersionStorageInfo* vstorage) const override; + + private: + struct SortedRun { + SortedRun(int _level, FileMetaData* _file, uint64_t _size, + uint64_t _compensated_file_size, bool _being_compacted) + : level(_level), + file(_file), + size(_size), + compensated_file_size(_compensated_file_size), + being_compacted(_being_compacted) { + assert(compensated_file_size > 0); + assert(level != 0 || file != nullptr); + } + + void Dump(char* out_buf, size_t out_buf_size, + bool print_path = false) const; + + // sorted_run_count is added into the string to print + void DumpSizeInfo(char* out_buf, size_t out_buf_size, + size_t sorted_run_count) const; + + int level; + // `file` Will be null for level > 0. For level = 0, the sorted run is + // for this file. + FileMetaData* file; + // For level > 0, `size` and `compensated_file_size` are sum of sizes all + // files in the level. `being_compacted` should be the same for all files + // in a non-zero level. Use the value here. + uint64_t size; + uint64_t compensated_file_size; + bool being_compacted; + }; + + // Pick Universal compaction to limit read amplification + Compaction* PickCompactionToReduceSortedRuns( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, double score, unsigned int ratio, + unsigned int num_files, const std::vector& sorted_runs, + LogBuffer* log_buffer); + + // Pick Universal compaction to limit space amplification. + Compaction* PickCompactionToReduceSizeAmp( + const std::string& cf_name, const MutableCFOptions& mutable_cf_options, + VersionStorageInfo* vstorage, double score, + const std::vector& sorted_runs, LogBuffer* log_buffer); + + // Used in universal compaction when the enabled_trivial_move + // option is set. Checks whether there are any overlapping files + // in the input. Returns true if the input files are non + // overlapping. + bool IsInputFilesNonOverlapping(Compaction* c); + + static std::vector CalculateSortedRuns( + const VersionStorageInfo& vstorage, const ImmutableCFOptions& ioptions); + + // Pick a path ID to place a newly generated file, with its estimated file + // size. + static uint32_t GetPathId(const ImmutableCFOptions& ioptions, + uint64_t file_size); +}; +} // namespace rocksdb +#endif // !ROCKSDB_LITE diff --git a/db/comparator_db_test.cc b/db/comparator_db_test.cc new file mode 100644 index 00000000000..28a2a5658e7 --- /dev/null +++ b/db/comparator_db_test.cc @@ -0,0 +1,441 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#include +#include + +#include "memtable/stl_wrappers.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "util/hash.h" +#include "util/kv_map.h" +#include "util/string_util.h" +#include "util/testharness.h" +#include "util/testutil.h" +#include "utilities/merge_operators.h" + +using std::unique_ptr; + +namespace rocksdb { +namespace { + +static const Comparator* comparator; + +class KVIter : public Iterator { + public: + explicit KVIter(const stl_wrappers::KVMap* map) + : map_(map), iter_(map_->end()) {} + virtual bool Valid() const override { return iter_ != map_->end(); } + virtual void SeekToFirst() override { iter_ = map_->begin(); } + virtual void SeekToLast() override { + if (map_->empty()) { + iter_ = map_->end(); + } else { + iter_ = map_->find(map_->rbegin()->first); + } + } + virtual void Seek(const Slice& k) override { + iter_ = map_->lower_bound(k.ToString()); + } + virtual void SeekForPrev(const Slice& k) override { + iter_ = map_->upper_bound(k.ToString()); + Prev(); + } + virtual void Next() override { ++iter_; } + virtual void Prev() override { + if (iter_ == map_->begin()) { + iter_ = map_->end(); + return; + } + --iter_; + } + + virtual Slice key() const override { return iter_->first; } + virtual Slice value() const override { return iter_->second; } + virtual Status status() const override { return Status::OK(); } + + private: + const stl_wrappers::KVMap* const map_; + stl_wrappers::KVMap::const_iterator iter_; +}; + +void AssertItersEqual(Iterator* iter1, Iterator* iter2) { + ASSERT_EQ(iter1->Valid(), iter2->Valid()); + if (iter1->Valid()) { + ASSERT_EQ(iter1->key().ToString(), iter2->key().ToString()); + ASSERT_EQ(iter1->value().ToString(), iter2->value().ToString()); + } +} + +// Measuring operations on DB (expect to be empty). +// source_strings are candidate keys +void DoRandomIteraratorTest(DB* db, std::vector source_strings, + Random* rnd, int num_writes, int num_iter_ops, + int num_trigger_flush) { + stl_wrappers::KVMap map((stl_wrappers::LessOfComparator(comparator))); + + for (int i = 0; i < num_writes; i++) { + if (num_trigger_flush > 0 && i != 0 && i % num_trigger_flush == 0) { + db->Flush(FlushOptions()); + } + + int type = rnd->Uniform(2); + int index = rnd->Uniform(static_cast(source_strings.size())); + auto& key = source_strings[index]; + switch (type) { + case 0: + // put + map[key] = key; + ASSERT_OK(db->Put(WriteOptions(), key, key)); + break; + case 1: + // delete + if (map.find(key) != map.end()) { + map.erase(key); + } + ASSERT_OK(db->Delete(WriteOptions(), key)); + break; + default: + assert(false); + } + } + + std::unique_ptr iter(db->NewIterator(ReadOptions())); + std::unique_ptr result_iter(new KVIter(&map)); + + bool is_valid = false; + for (int i = 0; i < num_iter_ops; i++) { + // Random walk and make sure iter and result_iter returns the + // same key and value + int type = rnd->Uniform(6); + ASSERT_OK(iter->status()); + switch (type) { + case 0: + // Seek to First + iter->SeekToFirst(); + result_iter->SeekToFirst(); + break; + case 1: + // Seek to last + iter->SeekToLast(); + result_iter->SeekToLast(); + break; + case 2: { + // Seek to random key + auto key_idx = rnd->Uniform(static_cast(source_strings.size())); + auto key = source_strings[key_idx]; + iter->Seek(key); + result_iter->Seek(key); + break; + } + case 3: + // Next + if (is_valid) { + iter->Next(); + result_iter->Next(); + } else { + continue; + } + break; + case 4: + // Prev + if (is_valid) { + iter->Prev(); + result_iter->Prev(); + } else { + continue; + } + break; + default: { + assert(type == 5); + auto key_idx = rnd->Uniform(static_cast(source_strings.size())); + auto key = source_strings[key_idx]; + std::string result; + auto status = db->Get(ReadOptions(), key, &result); + if (map.find(key) == map.end()) { + ASSERT_TRUE(status.IsNotFound()); + } else { + ASSERT_EQ(map[key], result); + } + break; + } + } + AssertItersEqual(iter.get(), result_iter.get()); + is_valid = iter->Valid(); + } +} + +class DoubleComparator : public Comparator { + public: + DoubleComparator() {} + + virtual const char* Name() const override { return "DoubleComparator"; } + + virtual int Compare(const Slice& a, const Slice& b) const override { +#ifndef CYGWIN + double da = std::stod(a.ToString()); + double db = std::stod(b.ToString()); +#else + double da = std::strtod(a.ToString().c_str(), 0 /* endptr */); + double db = std::strtod(a.ToString().c_str(), 0 /* endptr */); +#endif + if (da == db) { + return a.compare(b); + } else if (da > db) { + return 1; + } else { + return -1; + } + } + virtual void FindShortestSeparator(std::string* start, + const Slice& limit) const override {} + + virtual void FindShortSuccessor(std::string* key) const override {} +}; + +class HashComparator : public Comparator { + public: + HashComparator() {} + + virtual const char* Name() const override { return "HashComparator"; } + + virtual int Compare(const Slice& a, const Slice& b) const override { + uint32_t ha = Hash(a.data(), a.size(), 66); + uint32_t hb = Hash(b.data(), b.size(), 66); + if (ha == hb) { + return a.compare(b); + } else if (ha > hb) { + return 1; + } else { + return -1; + } + } + virtual void FindShortestSeparator(std::string* start, + const Slice& limit) const override {} + + virtual void FindShortSuccessor(std::string* key) const override {} +}; + +class TwoStrComparator : public Comparator { + public: + TwoStrComparator() {} + + virtual const char* Name() const override { return "TwoStrComparator"; } + + virtual int Compare(const Slice& a, const Slice& b) const override { + assert(a.size() >= 2); + assert(b.size() >= 2); + size_t size_a1 = static_cast(a[0]); + size_t size_b1 = static_cast(b[0]); + size_t size_a2 = static_cast(a[1]); + size_t size_b2 = static_cast(b[1]); + assert(size_a1 + size_a2 + 2 == a.size()); + assert(size_b1 + size_b2 + 2 == b.size()); + + Slice a1 = Slice(a.data() + 2, size_a1); + Slice b1 = Slice(b.data() + 2, size_b1); + Slice a2 = Slice(a.data() + 2 + size_a1, size_a2); + Slice b2 = Slice(b.data() + 2 + size_b1, size_b2); + + if (a1 != b1) { + return a1.compare(b1); + } + return a2.compare(b2); + } + virtual void FindShortestSeparator(std::string* start, + const Slice& limit) const override {} + + virtual void FindShortSuccessor(std::string* key) const override {} +}; +} // namespace + +class ComparatorDBTest : public testing::Test { + private: + std::string dbname_; + Env* env_; + DB* db_; + Options last_options_; + std::unique_ptr comparator_guard; + + public: + ComparatorDBTest() : env_(Env::Default()), db_(nullptr) { + comparator = BytewiseComparator(); + dbname_ = test::TmpDir() + "/comparator_db_test"; + EXPECT_OK(DestroyDB(dbname_, last_options_)); + } + + ~ComparatorDBTest() { + delete db_; + EXPECT_OK(DestroyDB(dbname_, last_options_)); + comparator = BytewiseComparator(); + } + + DB* GetDB() { return db_; } + + void SetOwnedComparator(const Comparator* cmp) { + comparator_guard.reset(cmp); + comparator = cmp; + last_options_.comparator = cmp; + } + + // Return the current option configuration. + Options* GetOptions() { return &last_options_; } + + void DestroyAndReopen() { + // Destroy using last options + Destroy(); + ASSERT_OK(TryReopen()); + } + + void Destroy() { + delete db_; + db_ = nullptr; + ASSERT_OK(DestroyDB(dbname_, last_options_)); + } + + Status TryReopen() { + delete db_; + db_ = nullptr; + last_options_.create_if_missing = true; + + return DB::Open(last_options_, dbname_, &db_); + } +}; + +TEST_F(ComparatorDBTest, Bytewise) { + for (int rand_seed = 301; rand_seed < 306; rand_seed++) { + DestroyAndReopen(); + Random rnd(rand_seed); + DoRandomIteraratorTest(GetDB(), + {"a", "b", "c", "d", "e", "f", "g", "h", "i"}, &rnd, + 8, 100, 3); + } +} + +TEST_F(ComparatorDBTest, SimpleSuffixReverseComparator) { + SetOwnedComparator(new test::SimpleSuffixReverseComparator()); + + for (int rnd_seed = 301; rnd_seed < 316; rnd_seed++) { + Options* opt = GetOptions(); + opt->comparator = comparator; + DestroyAndReopen(); + Random rnd(rnd_seed); + + std::vector source_strings; + std::vector source_prefixes; + // Randomly generate 5 prefixes + for (int i = 0; i < 5; i++) { + source_prefixes.push_back(test::RandomHumanReadableString(&rnd, 8)); + } + for (int j = 0; j < 20; j++) { + int prefix_index = rnd.Uniform(static_cast(source_prefixes.size())); + std::string key = source_prefixes[prefix_index] + + test::RandomHumanReadableString(&rnd, rnd.Uniform(8)); + source_strings.push_back(key); + } + + DoRandomIteraratorTest(GetDB(), source_strings, &rnd, 30, 600, 66); + } +} + +TEST_F(ComparatorDBTest, Uint64Comparator) { + SetOwnedComparator(test::Uint64Comparator()); + + for (int rnd_seed = 301; rnd_seed < 316; rnd_seed++) { + Options* opt = GetOptions(); + opt->comparator = comparator; + DestroyAndReopen(); + Random rnd(rnd_seed); + Random64 rnd64(rnd_seed); + + std::vector source_strings; + // Randomly generate source keys + for (int i = 0; i < 100; i++) { + uint64_t r = rnd64.Next(); + std::string str; + str.resize(8); + memcpy(&str[0], static_cast(&r), 8); + source_strings.push_back(str); + } + + DoRandomIteraratorTest(GetDB(), source_strings, &rnd, 200, 1000, 66); + } +} + +TEST_F(ComparatorDBTest, DoubleComparator) { + SetOwnedComparator(new DoubleComparator()); + + for (int rnd_seed = 301; rnd_seed < 316; rnd_seed++) { + Options* opt = GetOptions(); + opt->comparator = comparator; + DestroyAndReopen(); + Random rnd(rnd_seed); + + std::vector source_strings; + // Randomly generate source keys + for (int i = 0; i < 100; i++) { + uint32_t r = rnd.Next(); + uint32_t divide_order = rnd.Uniform(8); + double to_divide = 1.0; + for (uint32_t j = 0; j < divide_order; j++) { + to_divide *= 10.0; + } + source_strings.push_back(ToString(r / to_divide)); + } + + DoRandomIteraratorTest(GetDB(), source_strings, &rnd, 200, 1000, 66); + } +} + +TEST_F(ComparatorDBTest, HashComparator) { + SetOwnedComparator(new HashComparator()); + + for (int rnd_seed = 301; rnd_seed < 316; rnd_seed++) { + Options* opt = GetOptions(); + opt->comparator = comparator; + DestroyAndReopen(); + Random rnd(rnd_seed); + + std::vector source_strings; + // Randomly generate source keys + for (int i = 0; i < 100; i++) { + source_strings.push_back(test::RandomKey(&rnd, 8)); + } + + DoRandomIteraratorTest(GetDB(), source_strings, &rnd, 200, 1000, 66); + } +} + +TEST_F(ComparatorDBTest, TwoStrComparator) { + SetOwnedComparator(new TwoStrComparator()); + + for (int rnd_seed = 301; rnd_seed < 316; rnd_seed++) { + Options* opt = GetOptions(); + opt->comparator = comparator; + DestroyAndReopen(); + Random rnd(rnd_seed); + + std::vector source_strings; + // Randomly generate source keys + for (int i = 0; i < 100; i++) { + std::string str; + uint32_t size1 = rnd.Uniform(8); + uint32_t size2 = rnd.Uniform(8); + str.append(1, static_cast(size1)); + str.append(1, static_cast(size2)); + str.append(test::RandomKey(&rnd, size1)); + str.append(test::RandomKey(&rnd, size2)); + source_strings.push_back(str); + } + + DoRandomIteraratorTest(GetDB(), source_strings, &rnd, 200, 1000, 66); + } +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/convenience.cc b/db/convenience.cc new file mode 100644 index 00000000000..8ee31cacab5 --- /dev/null +++ b/db/convenience.cc @@ -0,0 +1,58 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// + +#ifndef ROCKSDB_LITE + +#include "rocksdb/convenience.h" + +#include "db/db_impl.h" +#include "util/cast_util.h" + +namespace rocksdb { + +void CancelAllBackgroundWork(DB* db, bool wait) { + (static_cast_with_check(db->GetRootDB())) + ->CancelAllBackgroundWork(wait); +} + +Status DeleteFilesInRange(DB* db, ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) { + return (static_cast_with_check(db->GetRootDB())) + ->DeleteFilesInRange(column_family, begin, end); +} + +Status VerifySstFileChecksum(const Options& options, + const EnvOptions& env_options, + const std::string& file_path) { + unique_ptr file; + uint64_t file_size; + InternalKeyComparator internal_comparator(options.comparator); + ImmutableCFOptions ioptions(options); + + Status s = ioptions.env->NewRandomAccessFile(file_path, &file, env_options); + if (s.ok()) { + s = ioptions.env->GetFileSize(file_path, &file_size); + } else { + return s; + } + unique_ptr table_reader; + std::unique_ptr file_reader( + new RandomAccessFileReader(std::move(file), file_path)); + s = ioptions.table_factory->NewTableReader( + TableReaderOptions(ioptions, env_options, internal_comparator, + false /* skip_filters */, -1 /* level */), + std::move(file_reader), file_size, &table_reader, + false /* prefetch_index_and_filter_in_cache */); + if (!s.ok()) { + return s; + } + s = table_reader->VerifyChecksum(); + return s; +} + +} // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/db/corruption_test.cc b/db/corruption_test.cc index 7a1a5221b03..56e157832c2 100644 --- a/db/corruption_test.cc +++ b/db/corruption_test.cc @@ -1,27 +1,31 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. +#ifndef ROCKSDB_LITE + #include "rocksdb/db.h" #include #include +#include #include #include +#include "db/db_impl.h" +#include "db/log_format.h" +#include "db/version_set.h" #include "rocksdb/cache.h" +#include "rocksdb/convenience.h" #include "rocksdb/env.h" #include "rocksdb/table.h" #include "rocksdb/write_batch.h" -#include "db/db_impl.h" -#include "db/filename.h" -#include "db/log_format.h" -#include "db/version_set.h" -#include "util/logging.h" +#include "util/filename.h" +#include "util/string_util.h" #include "util/testharness.h" #include "util/testutil.h" @@ -29,7 +33,7 @@ namespace rocksdb { static const int kValueSize = 1000; -class CorruptionTest { +class CorruptionTest : public testing::Test { public: test::ErrorEnv env_; std::string dbname_; @@ -38,7 +42,11 @@ class CorruptionTest { DB* db_; CorruptionTest() { - tiny_cache_ = NewLRUCache(100); + // If LRU cache shard bit is smaller than 2 (or -1 which will automatically + // set it to 0), test SequenceNumberRecovery will fail, likely because of a + // bug in recovery code. Keep it 4 for now to make the test passes. + tiny_cache_ = NewLRUCache(100, 4); + options_.wal_recovery_mode = WALRecoveryMode::kTolerateCorruptedTailRecords; options_.env = &env_; dbname_ = test::TmpDir() + "/corruption_test"; DestroyDB(dbname_, options_); @@ -57,6 +65,11 @@ class CorruptionTest { DestroyDB(dbname_, Options()); } + void CloseDb() { + delete db_; + db_ = nullptr; + } + Status TryReopen(Options* options = nullptr) { delete db_; db_ = nullptr; @@ -80,10 +93,14 @@ class CorruptionTest { ASSERT_OK(::rocksdb::RepairDB(dbname_, options_)); } - void Build(int n) { + void Build(int n, int flush_every = 0) { std::string key_space, value_space; WriteBatch batch; for (int i = 0; i < n; i++) { + if (flush_every != 0 && i != 0 && i % flush_every == 0) { + DBImpl* dbi = reinterpret_cast(db_); + dbi->TEST_FlushMemTable(); + } //if ((i % 100) == 0) fprintf(stderr, "@ %d of %d\n", i, n); Slice key = Key(i, &key_space); batch.Clear(); @@ -93,8 +110,8 @@ class CorruptionTest { } void Check(int min_expected, int max_expected) { - unsigned int next_expected = 0; - int missed = 0; + uint64_t next_expected = 0; + uint64_t missed = 0; int bad_keys = 0; int bad_values = 0; int correct = 0; @@ -103,7 +120,7 @@ class CorruptionTest { // db itself will raise errors because data is corrupted. // Instead, we want the reads to be successful and this test // will detect whether the appropriate corruptions have - // occured. + // occurred. Iterator* iter = db_->NewIterator(ReadOptions(false, true)); for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { uint64_t key; @@ -116,7 +133,7 @@ class CorruptionTest { } missed += (key - next_expected); next_expected = key + 1; - if (iter->value() != Value(key, &value_space)) { + if (iter->value() != Value(static_cast(key), &value_space)) { bad_values++; } else { correct++; @@ -125,17 +142,18 @@ class CorruptionTest { delete iter; fprintf(stderr, - "expected=%d..%d; got=%d; bad_keys=%d; bad_values=%d; missed=%d\n", - min_expected, max_expected, correct, bad_keys, bad_values, missed); + "expected=%d..%d; got=%d; bad_keys=%d; bad_values=%d; missed=%llu\n", + min_expected, max_expected, correct, bad_keys, bad_values, + static_cast(missed)); ASSERT_LE(min_expected, correct); ASSERT_GE(max_expected, correct); } - void CorruptFile(const std::string fname, int offset, int bytes_to_corrupt) { + void CorruptFile(const std::string& fname, int offset, int bytes_to_corrupt) { struct stat sbuf; if (stat(fname.c_str(), &sbuf) != 0) { const char* msg = strerror(errno); - ASSERT_TRUE(false) << fname << ": " << msg; + FAIL() << fname << ": " << msg; } if (offset < 0) { @@ -143,14 +161,14 @@ class CorruptionTest { if (-offset > sbuf.st_size) { offset = 0; } else { - offset = sbuf.st_size + offset; + offset = static_cast(sbuf.st_size + offset); } } if (offset > sbuf.st_size) { - offset = sbuf.st_size; + offset = static_cast(sbuf.st_size); } if (offset + bytes_to_corrupt > sbuf.st_size) { - bytes_to_corrupt = sbuf.st_size - offset; + bytes_to_corrupt = static_cast(sbuf.st_size - offset); } // Do it @@ -162,6 +180,9 @@ class CorruptionTest { } s = WriteStringToFile(Env::Default(), contents, fname); ASSERT_TRUE(s.ok()) << s.ToString(); + Options options; + EnvOptions env_options; + ASSERT_NOK(VerifySstFileChecksum(options, env_options, fname)); } void Corrupt(FileType filetype, int offset, int bytes_to_corrupt) { @@ -172,12 +193,12 @@ class CorruptionTest { FileType type; std::string fname; int picked_number = -1; - for (unsigned int i = 0; i < filenames.size(); i++) { + for (size_t i = 0; i < filenames.size(); i++) { if (ParseFileName(filenames[i], &number, &type) && type == filetype && static_cast(number) > picked_number) { // Pick latest file fname = dbname_ + "/" + filenames[i]; - picked_number = number; + picked_number = static_cast(number); } } ASSERT_TRUE(!fname.empty()) << filetype; @@ -196,7 +217,7 @@ class CorruptionTest { return; } } - ASSERT_TRUE(false) << "no file found at level"; + FAIL() << "no file found at level"; } @@ -221,32 +242,53 @@ class CorruptionTest { // Return the value to associate with the specified key Slice Value(int k, std::string* storage) { - Random r(k); - return test::RandomString(&r, kValueSize, storage); + if (k == 0) { + // Ugh. Random seed of 0 used to produce no entropy. This code + // preserves the implementation that was in place when all of the + // magic values in this file were picked. + *storage = std::string(kValueSize, ' '); + return Slice(*storage); + } else { + Random r(k); + return test::RandomString(&r, kValueSize, storage); + } } }; -TEST(CorruptionTest, Recovery) { +TEST_F(CorruptionTest, Recovery) { Build(100); Check(100, 100); +#ifdef OS_WIN + // On Wndows OS Disk cache does not behave properly + // We do not call FlushBuffers on every Flush. If we do not close + // the log file prior to the corruption we end up with the first + // block not corrupted but only the second. However, under the debugger + // things work just fine but never pass when running normally + // For that reason people may want to run with unbuffered I/O. That option + // is not available for WAL though. + CloseDb(); +#endif Corrupt(kLogFile, 19, 1); // WriteBatch tag for first record Corrupt(kLogFile, log::kBlockSize + 1000, 1); // Somewhere in second block - Reopen(); + ASSERT_TRUE(!TryReopen().ok()); + options_.paranoid_checks = false; + Reopen(&options_); // The 64 records in the first two log blocks are completely lost. Check(36, 36); } -TEST(CorruptionTest, RecoverWriteError) { +TEST_F(CorruptionTest, RecoverWriteError) { env_.writable_file_error_ = true; Status s = TryReopen(); ASSERT_TRUE(!s.ok()); } -TEST(CorruptionTest, NewFileErrorDuringWrite) { +TEST_F(CorruptionTest, NewFileErrorDuringWrite) { // Do enough writing to force minor compaction env_.writable_file_error_ = true; - const int num = 3 + (Options().write_buffer_size / kValueSize); + const int num = + static_cast(3 + (Options().write_buffer_size / kValueSize)); std::string value_storage; Status s; bool failed = false; @@ -265,7 +307,7 @@ TEST(CorruptionTest, NewFileErrorDuringWrite) { Reopen(); } -TEST(CorruptionTest, TableFile) { +TEST_F(CorruptionTest, TableFile) { Build(100); DBImpl* dbi = reinterpret_cast(db_); dbi->TEST_FlushMemTable(); @@ -274,26 +316,37 @@ TEST(CorruptionTest, TableFile) { Corrupt(kTableFile, 100, 1); Check(99, 99); + ASSERT_NOK(dbi->VerifyChecksum()); } -TEST(CorruptionTest, TableFileIndexData) { - Build(10000); // Enough to build multiple Tables +TEST_F(CorruptionTest, TableFileIndexData) { + Options options; + // very big, we'll trigger flushes manually + options.write_buffer_size = 100 * 1024 * 1024; + Reopen(&options); + // build 2 tables, flush at 5000 + Build(10000, 5000); DBImpl* dbi = reinterpret_cast(db_); dbi->TEST_FlushMemTable(); + // corrupt an index block of an entire file Corrupt(kTableFile, -2000, 500); Reopen(); - Check(5000, 9999); + dbi = reinterpret_cast(db_); + // one full file should be readable, since only one was corrupted + // the other file should be fully non-readable, since index was corrupted + Check(5000, 5000); + ASSERT_NOK(dbi->VerifyChecksum()); } -TEST(CorruptionTest, MissingDescriptor) { +TEST_F(CorruptionTest, MissingDescriptor) { Build(1000); RepairDB(); Reopen(); Check(1000, 1000); } -TEST(CorruptionTest, SequenceNumberRecovery) { +TEST_F(CorruptionTest, SequenceNumberRecovery) { ASSERT_OK(db_->Put(WriteOptions(), "foo", "v1")); ASSERT_OK(db_->Put(WriteOptions(), "foo", "v2")); ASSERT_OK(db_->Put(WriteOptions(), "foo", "v3")); @@ -314,7 +367,7 @@ TEST(CorruptionTest, SequenceNumberRecovery) { ASSERT_EQ("v6", v); } -TEST(CorruptionTest, CorruptedDescriptor) { +TEST_F(CorruptionTest, CorruptedDescriptor) { ASSERT_OK(db_->Put(WriteOptions(), "foo", "hello")); DBImpl* dbi = reinterpret_cast(db_); dbi->TEST_FlushMemTable(); @@ -331,22 +384,27 @@ TEST(CorruptionTest, CorruptedDescriptor) { ASSERT_EQ("hello", v); } -TEST(CorruptionTest, CompactionInputError) { +TEST_F(CorruptionTest, CompactionInputError) { + Options options; + Reopen(&options); Build(10); DBImpl* dbi = reinterpret_cast(db_); dbi->TEST_FlushMemTable(); - const int last = dbi->MaxMemCompactionLevel(); - ASSERT_EQ(1, Property("rocksdb.num-files-at-level" + NumberToString(last))); + dbi->TEST_CompactRange(0, nullptr, nullptr); + dbi->TEST_CompactRange(1, nullptr, nullptr); + ASSERT_EQ(1, Property("rocksdb.num-files-at-level2")); Corrupt(kTableFile, 100, 1); Check(9, 9); + ASSERT_NOK(dbi->VerifyChecksum()); // Force compactions by writing lots of values Build(10000); Check(10000, 10000); + ASSERT_NOK(dbi->VerifyChecksum()); } -TEST(CorruptionTest, CompactionInputErrorParanoid) { +TEST_F(CorruptionTest, CompactionInputErrorParanoid) { Options options; options.paranoid_checks = true; options.write_buffer_size = 131072; @@ -354,14 +412,17 @@ TEST(CorruptionTest, CompactionInputErrorParanoid) { Reopen(&options); DBImpl* dbi = reinterpret_cast(db_); - // Fill levels >= 1 so memtable flush outputs to level 0 + // Fill levels >= 1 for (int level = 1; level < dbi->NumberLevels(); level++) { dbi->Put(WriteOptions(), "", "begin"); dbi->Put(WriteOptions(), "~", "end"); dbi->TEST_FlushMemTable(); + for (int comp_level = 0; comp_level < dbi->NumberLevels() - level; + ++comp_level) { + dbi->TEST_CompactRange(comp_level, nullptr, nullptr); + } } - options.max_mem_compaction_level = 0; Reopen(&options); dbi = reinterpret_cast(db_); @@ -372,6 +433,7 @@ TEST(CorruptionTest, CompactionInputErrorParanoid) { CorruptTableFileAtLevel(0, 100, 1); Check(9, 9); + ASSERT_NOK(dbi->VerifyChecksum()); // Write must eventually fail because of corrupted table Status s; @@ -388,11 +450,12 @@ TEST(CorruptionTest, CompactionInputErrorParanoid) { ASSERT_TRUE(!s.ok()) << "write did not fail in corrupted paranoid db"; } -TEST(CorruptionTest, UnrelatedKeys) { +TEST_F(CorruptionTest, UnrelatedKeys) { Build(10); DBImpl* dbi = reinterpret_cast(db_); dbi->TEST_FlushMemTable(); Corrupt(kTableFile, 100, 1); + ASSERT_NOK(dbi->VerifyChecksum()); std::string tmp1, tmp2; ASSERT_OK(db_->Put(WriteOptions(), Key(1000, &tmp1), Value(1000, &tmp2))); @@ -404,7 +467,7 @@ TEST(CorruptionTest, UnrelatedKeys) { ASSERT_EQ(Value(1000, &tmp2).ToString(), v); } -TEST(CorruptionTest, FileSystemStateCorrupted) { +TEST_F(CorruptionTest, FileSystemStateCorrupted) { for (int iter = 0; iter < 2; ++iter) { Options options; options.paranoid_checks = true; @@ -440,5 +503,16 @@ TEST(CorruptionTest, FileSystemStateCorrupted) { } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, "SKIPPED as RepairDB() is not supported in ROCKSDB_LITE\n"); + return 0; +} + +#endif // !ROCKSDB_LITE diff --git a/db/cuckoo_table_db_test.cc b/db/cuckoo_table_db_test.cc index c1e59b1b568..e7c2d279a4d 100644 --- a/db/cuckoo_table_db_test.cc +++ b/db/cuckoo_table_db_test.cc @@ -1,20 +1,23 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE #include "db/db_impl.h" #include "rocksdb/db.h" #include "rocksdb/env.h" -#include "table/meta_blocks.h" #include "table/cuckoo_table_factory.h" #include "table/cuckoo_table_reader.h" +#include "table/meta_blocks.h" +#include "util/string_util.h" #include "util/testharness.h" #include "util/testutil.h" namespace rocksdb { -class CuckooTableDBTest { +class CuckooTableDBTest : public testing::Test { private: std::string dbname_; Env* env_; @@ -23,14 +26,14 @@ class CuckooTableDBTest { public: CuckooTableDBTest() : env_(Env::Default()) { dbname_ = test::TmpDir() + "/cuckoo_table_db_test"; - ASSERT_OK(DestroyDB(dbname_, Options())); + EXPECT_OK(DestroyDB(dbname_, Options())); db_ = nullptr; Reopen(); } ~CuckooTableDBTest() { delete db_; - ASSERT_OK(DestroyDB(dbname_, Options())); + EXPECT_OK(DestroyDB(dbname_, Options())); } Options CurrentOptions() { @@ -39,7 +42,7 @@ class CuckooTableDBTest { options.memtable_factory.reset(NewHashLinkListRepFactory(4, 0, 3, true)); options.allow_mmap_reads = true; options.create_if_missing = true; - options.max_mem_compaction_level = 0; + options.allow_concurrent_memtable_write = false; return options; } @@ -83,16 +86,15 @@ class CuckooTableDBTest { int NumTableFilesAtLevel(int level) { std::string property; - ASSERT_TRUE( - db_->GetProperty("rocksdb.num-files-at-level" + NumberToString(level), - &property)); + EXPECT_TRUE(db_->GetProperty( + "rocksdb.num-files-at-level" + NumberToString(level), &property)); return atoi(property.c_str()); } // Return spread of files per level std::string FilesPerLevel() { std::string result; - int last_non_zero_offset = 0; + size_t last_non_zero_offset = 0; for (int level = 0; level < db_->NumberLevels(); level++) { int f = NumTableFilesAtLevel(level); char buf[100]; @@ -107,7 +109,7 @@ class CuckooTableDBTest { } }; -TEST(CuckooTableDBTest, Flush) { +TEST_F(CuckooTableDBTest, Flush) { // Try with empty DB first. ASSERT_TRUE(dbfull() != nullptr); ASSERT_EQ("NOT_FOUND", Get("key2")); @@ -170,7 +172,7 @@ TEST(CuckooTableDBTest, Flush) { ASSERT_EQ("NOT_FOUND", Get("key6")); } -TEST(CuckooTableDBTest, FlushWithDuplicateKeys) { +TEST_F(CuckooTableDBTest, FlushWithDuplicateKeys) { Options options = CurrentOptions(); Reopen(&options); ASSERT_OK(Put("key1", "v1")); @@ -201,7 +203,7 @@ static std::string Uint64Key(uint64_t i) { } } // namespace. -TEST(CuckooTableDBTest, Uint64Comparator) { +TEST_F(CuckooTableDBTest, Uint64Comparator) { Options options = CurrentOptions(); options.comparator = test::Uint64Comparator(); Reopen(&options); @@ -218,6 +220,7 @@ TEST(CuckooTableDBTest, Uint64Comparator) { // Add more keys. ASSERT_OK(Delete(Uint64Key(2))); // Delete. + dbfull()->TEST_FlushMemTable(); ASSERT_OK(Put(Uint64Key(3), "v0")); // Update. ASSERT_OK(Put(Uint64Key(4), "v4")); dbfull()->TEST_FlushMemTable(); @@ -227,33 +230,31 @@ TEST(CuckooTableDBTest, Uint64Comparator) { ASSERT_EQ("v4", Get(Uint64Key(4))); } -TEST(CuckooTableDBTest, CompactionTrigger) { +TEST_F(CuckooTableDBTest, CompactionIntoMultipleFiles) { + // Create a big L0 file and check it compacts into multiple files in L1. Options options = CurrentOptions(); - options.write_buffer_size = 100 << 10; // 100KB - options.level0_file_num_compaction_trigger = 2; + options.write_buffer_size = 270 << 10; + // Two SST files should be created, each containing 14 keys. + // Number of buckets will be 16. Total size ~156 KB. + options.target_file_size_base = 160 << 10; Reopen(&options); - // Write 11 values, each 10016 B - for (int idx = 0; idx < 11; ++idx) { + // Write 28 values, each 10016 B ~ 10KB + for (int idx = 0; idx < 28; ++idx) { ASSERT_OK(Put(Key(idx), std::string(10000, 'a' + idx))); } dbfull()->TEST_WaitForFlushMemTable(); ASSERT_EQ("1", FilesPerLevel()); - // Generate one more file in level-0, and should trigger level-0 compaction - for (int idx = 11; idx < 22; ++idx) { - ASSERT_OK(Put(Key(idx), std::string(10000, 'a' + idx))); - } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_CompactRange(0, nullptr, nullptr); - + dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, + true /* disallow trivial move */); ASSERT_EQ("0,2", FilesPerLevel()); - for (int idx = 0; idx < 22; ++idx) { + for (int idx = 0; idx < 28; ++idx) { ASSERT_EQ(std::string(10000, 'a' + idx), Get(Key(idx))); } } -TEST(CuckooTableDBTest, SameKeyInsertedInTwoDifferentFilesAndCompacted) { +TEST_F(CuckooTableDBTest, SameKeyInsertedInTwoDifferentFilesAndCompacted) { // Insert same key twice so that they go to different SST files. Then wait for // compaction and check if the latest value is stored and old value removed. Options options = CurrentOptions(); @@ -281,7 +282,7 @@ TEST(CuckooTableDBTest, SameKeyInsertedInTwoDifferentFilesAndCompacted) { } } -TEST(CuckooTableDBTest, AdaptiveTable) { +TEST_F(CuckooTableDBTest, AdaptiveTable) { Options options = CurrentOptions(); // Write some keys using cuckoo table. @@ -318,4 +319,23 @@ TEST(CuckooTableDBTest, AdaptiveTable) { } } // namespace rocksdb -int main(int argc, char** argv) { return rocksdb::test::RunAllTests(); } +int main(int argc, char** argv) { + if (rocksdb::port::kLittleEndian) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); + } + else { + fprintf(stderr, "SKIPPED as Cuckoo table doesn't support Big Endian\n"); + return 0; + } +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, "SKIPPED as Cuckoo table is not supported in ROCKSDB_LITE\n"); + return 0; +} + +#endif // ROCKSDB_LITE diff --git a/db/db_basic_test.cc b/db/db_basic_test.cc new file mode 100644 index 00000000000..654a457ef5d --- /dev/null +++ b/db/db_basic_test.cc @@ -0,0 +1,856 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#include "db/db_test_util.h" +#include "port/stack_trace.h" +#include "rocksdb/perf_context.h" +#if !defined(ROCKSDB_LITE) +#include "util/sync_point.h" +#endif + +namespace rocksdb { + +class DBBasicTest : public DBTestBase { + public: + DBBasicTest() : DBTestBase("/db_basic_test") {} +}; + +TEST_F(DBBasicTest, OpenWhenOpen) { + Options options = CurrentOptions(); + options.env = env_; + rocksdb::DB* db2 = nullptr; + rocksdb::Status s = DB::Open(options, dbname_, &db2); + + ASSERT_EQ(Status::Code::kIOError, s.code()); + ASSERT_EQ(Status::SubCode::kNone, s.subcode()); + ASSERT_TRUE(strstr(s.getState(), "lock ") != nullptr); + + delete db2; +} + +#ifndef ROCKSDB_LITE +TEST_F(DBBasicTest, ReadOnlyDB) { + ASSERT_OK(Put("foo", "v1")); + ASSERT_OK(Put("bar", "v2")); + ASSERT_OK(Put("foo", "v3")); + Close(); + + auto options = CurrentOptions(); + assert(options.env = env_); + ASSERT_OK(ReadOnlyReopen(options)); + ASSERT_EQ("v3", Get("foo")); + ASSERT_EQ("v2", Get("bar")); + Iterator* iter = db_->NewIterator(ReadOptions()); + int count = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_OK(iter->status()); + ++count; + } + ASSERT_EQ(count, 2); + delete iter; + Close(); + + // Reopen and flush memtable. + Reopen(options); + Flush(); + Close(); + // Now check keys in read only mode. + ASSERT_OK(ReadOnlyReopen(options)); + ASSERT_EQ("v3", Get("foo")); + ASSERT_EQ("v2", Get("bar")); + ASSERT_TRUE(db_->SyncWAL().IsNotSupported()); +} + +TEST_F(DBBasicTest, CompactedDB) { + const uint64_t kFileSize = 1 << 20; + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.write_buffer_size = kFileSize; + options.target_file_size_base = kFileSize; + options.max_bytes_for_level_base = 1 << 30; + options.compression = kNoCompression; + Reopen(options); + // 1 L0 file, use CompactedDB if max_open_files = -1 + ASSERT_OK(Put("aaa", DummyString(kFileSize / 2, '1'))); + Flush(); + Close(); + ASSERT_OK(ReadOnlyReopen(options)); + Status s = Put("new", "value"); + ASSERT_EQ(s.ToString(), + "Not implemented: Not supported operation in read only mode."); + ASSERT_EQ(DummyString(kFileSize / 2, '1'), Get("aaa")); + Close(); + options.max_open_files = -1; + ASSERT_OK(ReadOnlyReopen(options)); + s = Put("new", "value"); + ASSERT_EQ(s.ToString(), + "Not implemented: Not supported in compacted db mode."); + ASSERT_EQ(DummyString(kFileSize / 2, '1'), Get("aaa")); + Close(); + Reopen(options); + // Add more L0 files + ASSERT_OK(Put("bbb", DummyString(kFileSize / 2, '2'))); + Flush(); + ASSERT_OK(Put("aaa", DummyString(kFileSize / 2, 'a'))); + Flush(); + ASSERT_OK(Put("bbb", DummyString(kFileSize / 2, 'b'))); + ASSERT_OK(Put("eee", DummyString(kFileSize / 2, 'e'))); + Flush(); + Close(); + + ASSERT_OK(ReadOnlyReopen(options)); + // Fallback to read-only DB + s = Put("new", "value"); + ASSERT_EQ(s.ToString(), + "Not implemented: Not supported operation in read only mode."); + Close(); + + // Full compaction + Reopen(options); + // Add more keys + ASSERT_OK(Put("fff", DummyString(kFileSize / 2, 'f'))); + ASSERT_OK(Put("hhh", DummyString(kFileSize / 2, 'h'))); + ASSERT_OK(Put("iii", DummyString(kFileSize / 2, 'i'))); + ASSERT_OK(Put("jjj", DummyString(kFileSize / 2, 'j'))); + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ(3, NumTableFilesAtLevel(1)); + Close(); + + // CompactedDB + ASSERT_OK(ReadOnlyReopen(options)); + s = Put("new", "value"); + ASSERT_EQ(s.ToString(), + "Not implemented: Not supported in compacted db mode."); + ASSERT_EQ("NOT_FOUND", Get("abc")); + ASSERT_EQ(DummyString(kFileSize / 2, 'a'), Get("aaa")); + ASSERT_EQ(DummyString(kFileSize / 2, 'b'), Get("bbb")); + ASSERT_EQ("NOT_FOUND", Get("ccc")); + ASSERT_EQ(DummyString(kFileSize / 2, 'e'), Get("eee")); + ASSERT_EQ(DummyString(kFileSize / 2, 'f'), Get("fff")); + ASSERT_EQ("NOT_FOUND", Get("ggg")); + ASSERT_EQ(DummyString(kFileSize / 2, 'h'), Get("hhh")); + ASSERT_EQ(DummyString(kFileSize / 2, 'i'), Get("iii")); + ASSERT_EQ(DummyString(kFileSize / 2, 'j'), Get("jjj")); + ASSERT_EQ("NOT_FOUND", Get("kkk")); + + // MultiGet + std::vector values; + std::vector status_list = dbfull()->MultiGet( + ReadOptions(), + std::vector({Slice("aaa"), Slice("ccc"), Slice("eee"), + Slice("ggg"), Slice("iii"), Slice("kkk")}), + &values); + ASSERT_EQ(status_list.size(), static_cast(6)); + ASSERT_EQ(values.size(), static_cast(6)); + ASSERT_OK(status_list[0]); + ASSERT_EQ(DummyString(kFileSize / 2, 'a'), values[0]); + ASSERT_TRUE(status_list[1].IsNotFound()); + ASSERT_OK(status_list[2]); + ASSERT_EQ(DummyString(kFileSize / 2, 'e'), values[2]); + ASSERT_TRUE(status_list[3].IsNotFound()); + ASSERT_OK(status_list[4]); + ASSERT_EQ(DummyString(kFileSize / 2, 'i'), values[4]); + ASSERT_TRUE(status_list[5].IsNotFound()); + + Reopen(options); + // Add a key + ASSERT_OK(Put("fff", DummyString(kFileSize / 2, 'f'))); + Close(); + ASSERT_OK(ReadOnlyReopen(options)); + s = Put("new", "value"); + ASSERT_EQ(s.ToString(), + "Not implemented: Not supported operation in read only mode."); +} + +TEST_F(DBBasicTest, LevelLimitReopen) { + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); + + const std::string value(1024 * 1024, ' '); + int i = 0; + while (NumTableFilesAtLevel(2, 1) == 0) { + ASSERT_OK(Put(1, Key(i++), value)); + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } + + options.num_levels = 1; + options.max_bytes_for_level_multiplier_additional.resize(1, 1); + Status s = TryReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_EQ(s.IsInvalidArgument(), true); + ASSERT_EQ(s.ToString(), + "Invalid argument: db has more levels than options.num_levels"); + + options.num_levels = 10; + options.max_bytes_for_level_multiplier_additional.resize(10, 1); + ASSERT_OK(TryReopenWithColumnFamilies({"default", "pikachu"}, options)); +} +#endif // ROCKSDB_LITE + +TEST_F(DBBasicTest, PutDeleteGet) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_OK(Put(1, "foo", "v2")); + ASSERT_EQ("v2", Get(1, "foo")); + ASSERT_OK(Delete(1, "foo")); + ASSERT_EQ("NOT_FOUND", Get(1, "foo")); + } while (ChangeOptions()); +} + +TEST_F(DBBasicTest, PutSingleDeleteGet) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_OK(Put(1, "foo2", "v2")); + ASSERT_EQ("v2", Get(1, "foo2")); + ASSERT_OK(SingleDelete(1, "foo")); + ASSERT_EQ("NOT_FOUND", Get(1, "foo")); + // Skip HashCuckooRep as it does not support single delete. FIFO and + // universal compaction do not apply to the test case. Skip MergePut + // because single delete does not get removed when it encounters a merge. + } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction | + kSkipUniversalCompaction | kSkipMergePut)); +} + +TEST_F(DBBasicTest, EmptyFlush) { + // It is possible to produce empty flushes when using single deletes. Tests + // whether empty flushes cause issues. + do { + Random rnd(301); + + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + CreateAndReopenWithCF({"pikachu"}, options); + + Put(1, "a", Slice()); + SingleDelete(1, "a"); + ASSERT_OK(Flush(1)); + + ASSERT_EQ("[ ]", AllEntriesFor("a", 1)); + // Skip HashCuckooRep as it does not support single delete. FIFO and + // universal compaction do not apply to the test case. Skip MergePut + // because merges cannot be combined with single deletions. + } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction | + kSkipUniversalCompaction | kSkipMergePut)); +} + +TEST_F(DBBasicTest, GetFromVersions) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_OK(Flush(1)); + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_EQ("NOT_FOUND", Get(0, "foo")); + } while (ChangeOptions()); +} + +#ifndef ROCKSDB_LITE +TEST_F(DBBasicTest, GetSnapshot) { + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions(options_override)); + // Try with both a short key and a long key + for (int i = 0; i < 2; i++) { + std::string key = (i == 0) ? std::string("foo") : std::string(200, 'x'); + ASSERT_OK(Put(1, key, "v1")); + const Snapshot* s1 = db_->GetSnapshot(); + if (option_config_ == kHashCuckoo) { + // Unsupported case. + ASSERT_TRUE(s1 == nullptr); + break; + } + ASSERT_OK(Put(1, key, "v2")); + ASSERT_EQ("v2", Get(1, key)); + ASSERT_EQ("v1", Get(1, key, s1)); + ASSERT_OK(Flush(1)); + ASSERT_EQ("v2", Get(1, key)); + ASSERT_EQ("v1", Get(1, key, s1)); + db_->ReleaseSnapshot(s1); + } + } while (ChangeOptions()); +} +#endif // ROCKSDB_LITE + +TEST_F(DBBasicTest, CheckLock) { + do { + DB* localdb; + Options options = CurrentOptions(); + ASSERT_OK(TryReopen(options)); + + // second open should fail + ASSERT_TRUE(!(DB::Open(options, dbname_, &localdb)).ok()); + } while (ChangeCompactOptions()); +} + +TEST_F(DBBasicTest, FlushMultipleMemtable) { + do { + Options options = CurrentOptions(); + WriteOptions writeOpt = WriteOptions(); + writeOpt.disableWAL = true; + options.max_write_buffer_number = 4; + options.min_write_buffer_number_to_merge = 3; + options.max_write_buffer_number_to_maintain = -1; + CreateAndReopenWithCF({"pikachu"}, options); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v1")); + ASSERT_OK(Flush(1)); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v1")); + + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_EQ("v1", Get(1, "bar")); + ASSERT_OK(Flush(1)); + } while (ChangeCompactOptions()); +} + +TEST_F(DBBasicTest, FlushEmptyColumnFamily) { + // Block flush thread and disable compaction thread + env_->SetBackgroundThreads(1, Env::HIGH); + env_->SetBackgroundThreads(1, Env::LOW); + test::SleepingBackgroundTask sleeping_task_low; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + test::SleepingBackgroundTask sleeping_task_high; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, + &sleeping_task_high, Env::Priority::HIGH); + + Options options = CurrentOptions(); + // disable compaction + options.disable_auto_compactions = true; + WriteOptions writeOpt = WriteOptions(); + writeOpt.disableWAL = true; + options.max_write_buffer_number = 2; + options.min_write_buffer_number_to_merge = 1; + options.max_write_buffer_number_to_maintain = 1; + CreateAndReopenWithCF({"pikachu"}, options); + + // Compaction can still go through even if no thread can flush the + // mem table. + ASSERT_OK(Flush(0)); + ASSERT_OK(Flush(1)); + + // Insert can go through + ASSERT_OK(dbfull()->Put(writeOpt, handles_[0], "foo", "v1")); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v1")); + + ASSERT_EQ("v1", Get(0, "foo")); + ASSERT_EQ("v1", Get(1, "bar")); + + sleeping_task_high.WakeUp(); + sleeping_task_high.WaitUntilDone(); + + // Flush can still go through. + ASSERT_OK(Flush(0)); + ASSERT_OK(Flush(1)); + + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilDone(); +} + +TEST_F(DBBasicTest, FLUSH) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + WriteOptions writeOpt = WriteOptions(); + writeOpt.disableWAL = true; + SetPerfLevel(kEnableTime); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v1")); + // this will now also flush the last 2 writes + ASSERT_OK(Flush(1)); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v1")); + + get_perf_context()->Reset(); + Get(1, "foo"); + ASSERT_TRUE((int)get_perf_context()->get_from_output_files_time > 0); + ASSERT_EQ(2, (int)get_perf_context()->get_read_bytes); + + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_EQ("v1", Get(1, "bar")); + + writeOpt.disableWAL = true; + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v2")); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v2")); + ASSERT_OK(Flush(1)); + + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_EQ("v2", Get(1, "bar")); + get_perf_context()->Reset(); + ASSERT_EQ("v2", Get(1, "foo")); + ASSERT_TRUE((int)get_perf_context()->get_from_output_files_time > 0); + + writeOpt.disableWAL = false; + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v3")); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v3")); + ASSERT_OK(Flush(1)); + + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + // 'foo' should be there because its put + // has WAL enabled. + ASSERT_EQ("v3", Get(1, "foo")); + ASSERT_EQ("v3", Get(1, "bar")); + + SetPerfLevel(kDisable); + } while (ChangeCompactOptions()); +} + +TEST_F(DBBasicTest, ManifestRollOver) { + do { + Options options; + options.max_manifest_file_size = 10; // 10 bytes + options = CurrentOptions(options); + CreateAndReopenWithCF({"pikachu"}, options); + { + ASSERT_OK(Put(1, "manifest_key1", std::string(1000, '1'))); + ASSERT_OK(Put(1, "manifest_key2", std::string(1000, '2'))); + ASSERT_OK(Put(1, "manifest_key3", std::string(1000, '3'))); + uint64_t manifest_before_flush = dbfull()->TEST_Current_Manifest_FileNo(); + ASSERT_OK(Flush(1)); // This should trigger LogAndApply. + uint64_t manifest_after_flush = dbfull()->TEST_Current_Manifest_FileNo(); + ASSERT_GT(manifest_after_flush, manifest_before_flush); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_GT(dbfull()->TEST_Current_Manifest_FileNo(), manifest_after_flush); + // check if a new manifest file got inserted or not. + ASSERT_EQ(std::string(1000, '1'), Get(1, "manifest_key1")); + ASSERT_EQ(std::string(1000, '2'), Get(1, "manifest_key2")); + ASSERT_EQ(std::string(1000, '3'), Get(1, "manifest_key3")); + } + } while (ChangeCompactOptions()); +} + +TEST_F(DBBasicTest, IdentityAcrossRestarts) { + do { + std::string id1; + ASSERT_OK(db_->GetDbIdentity(id1)); + + Options options = CurrentOptions(); + Reopen(options); + std::string id2; + ASSERT_OK(db_->GetDbIdentity(id2)); + // id1 should match id2 because identity was not regenerated + ASSERT_EQ(id1.compare(id2), 0); + + std::string idfilename = IdentityFileName(dbname_); + ASSERT_OK(env_->DeleteFile(idfilename)); + Reopen(options); + std::string id3; + ASSERT_OK(db_->GetDbIdentity(id3)); + // id1 should NOT match id3 because identity was regenerated + ASSERT_NE(id1.compare(id3), 0); + } while (ChangeCompactOptions()); +} + +#ifndef ROCKSDB_LITE +TEST_F(DBBasicTest, Snapshot) { + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions(options_override)); + Put(0, "foo", "0v1"); + Put(1, "foo", "1v1"); + + const Snapshot* s1 = db_->GetSnapshot(); + ASSERT_EQ(1U, GetNumSnapshots()); + uint64_t time_snap1 = GetTimeOldestSnapshots(); + ASSERT_GT(time_snap1, 0U); + Put(0, "foo", "0v2"); + Put(1, "foo", "1v2"); + + env_->addon_time_.fetch_add(1); + + const Snapshot* s2 = db_->GetSnapshot(); + ASSERT_EQ(2U, GetNumSnapshots()); + ASSERT_EQ(time_snap1, GetTimeOldestSnapshots()); + Put(0, "foo", "0v3"); + Put(1, "foo", "1v3"); + + { + ManagedSnapshot s3(db_); + ASSERT_EQ(3U, GetNumSnapshots()); + ASSERT_EQ(time_snap1, GetTimeOldestSnapshots()); + + Put(0, "foo", "0v4"); + Put(1, "foo", "1v4"); + ASSERT_EQ("0v1", Get(0, "foo", s1)); + ASSERT_EQ("1v1", Get(1, "foo", s1)); + ASSERT_EQ("0v2", Get(0, "foo", s2)); + ASSERT_EQ("1v2", Get(1, "foo", s2)); + ASSERT_EQ("0v3", Get(0, "foo", s3.snapshot())); + ASSERT_EQ("1v3", Get(1, "foo", s3.snapshot())); + ASSERT_EQ("0v4", Get(0, "foo")); + ASSERT_EQ("1v4", Get(1, "foo")); + } + + ASSERT_EQ(2U, GetNumSnapshots()); + ASSERT_EQ(time_snap1, GetTimeOldestSnapshots()); + ASSERT_EQ("0v1", Get(0, "foo", s1)); + ASSERT_EQ("1v1", Get(1, "foo", s1)); + ASSERT_EQ("0v2", Get(0, "foo", s2)); + ASSERT_EQ("1v2", Get(1, "foo", s2)); + ASSERT_EQ("0v4", Get(0, "foo")); + ASSERT_EQ("1v4", Get(1, "foo")); + + db_->ReleaseSnapshot(s1); + ASSERT_EQ("0v2", Get(0, "foo", s2)); + ASSERT_EQ("1v2", Get(1, "foo", s2)); + ASSERT_EQ("0v4", Get(0, "foo")); + ASSERT_EQ("1v4", Get(1, "foo")); + ASSERT_EQ(1U, GetNumSnapshots()); + ASSERT_LT(time_snap1, GetTimeOldestSnapshots()); + + db_->ReleaseSnapshot(s2); + ASSERT_EQ(0U, GetNumSnapshots()); + ASSERT_EQ("0v4", Get(0, "foo")); + ASSERT_EQ("1v4", Get(1, "foo")); + } while (ChangeOptions(kSkipHashCuckoo)); +} + +#endif // ROCKSDB_LITE + +TEST_F(DBBasicTest, CompactBetweenSnapshots) { + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; + do { + Options options = CurrentOptions(options_override); + options.disable_auto_compactions = true; + CreateAndReopenWithCF({"pikachu"}, options); + Random rnd(301); + FillLevels("a", "z", 1); + + Put(1, "foo", "first"); + const Snapshot* snapshot1 = db_->GetSnapshot(); + Put(1, "foo", "second"); + Put(1, "foo", "third"); + Put(1, "foo", "fourth"); + const Snapshot* snapshot2 = db_->GetSnapshot(); + Put(1, "foo", "fifth"); + Put(1, "foo", "sixth"); + + // All entries (including duplicates) exist + // before any compaction or flush is triggered. + ASSERT_EQ(AllEntriesFor("foo", 1), + "[ sixth, fifth, fourth, third, second, first ]"); + ASSERT_EQ("sixth", Get(1, "foo")); + ASSERT_EQ("fourth", Get(1, "foo", snapshot2)); + ASSERT_EQ("first", Get(1, "foo", snapshot1)); + + // After a flush, "second", "third" and "fifth" should + // be removed + ASSERT_OK(Flush(1)); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ sixth, fourth, first ]"); + + // after we release the snapshot1, only two values left + db_->ReleaseSnapshot(snapshot1); + FillLevels("a", "z", 1); + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + + // We have only one valid snapshot snapshot2. Since snapshot1 is + // not valid anymore, "first" should be removed by a compaction. + ASSERT_EQ("sixth", Get(1, "foo")); + ASSERT_EQ("fourth", Get(1, "foo", snapshot2)); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ sixth, fourth ]"); + + // after we release the snapshot2, only one value should be left + db_->ReleaseSnapshot(snapshot2); + FillLevels("a", "z", 1); + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + ASSERT_EQ("sixth", Get(1, "foo")); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ sixth ]"); + // skip HashCuckooRep as it does not support snapshot + } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction)); +} + +TEST_F(DBBasicTest, DBOpen_Options) { + Options options = CurrentOptions(); + std::string dbname = test::TmpDir(env_) + "/db_options_test"; + ASSERT_OK(DestroyDB(dbname, options)); + + // Does not exist, and create_if_missing == false: error + DB* db = nullptr; + options.create_if_missing = false; + Status s = DB::Open(options, dbname, &db); + ASSERT_TRUE(strstr(s.ToString().c_str(), "does not exist") != nullptr); + ASSERT_TRUE(db == nullptr); + + // Does not exist, and create_if_missing == true: OK + options.create_if_missing = true; + s = DB::Open(options, dbname, &db); + ASSERT_OK(s); + ASSERT_TRUE(db != nullptr); + + delete db; + db = nullptr; + + // Does exist, and error_if_exists == true: error + options.create_if_missing = false; + options.error_if_exists = true; + s = DB::Open(options, dbname, &db); + ASSERT_TRUE(strstr(s.ToString().c_str(), "exists") != nullptr); + ASSERT_TRUE(db == nullptr); + + // Does exist, and error_if_exists == false: OK + options.create_if_missing = true; + options.error_if_exists = false; + s = DB::Open(options, dbname, &db); + ASSERT_OK(s); + ASSERT_TRUE(db != nullptr); + + delete db; + db = nullptr; +} + +TEST_F(DBBasicTest, CompactOnFlush) { + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; + do { + Options options = CurrentOptions(options_override); + options.disable_auto_compactions = true; + CreateAndReopenWithCF({"pikachu"}, options); + + Put(1, "foo", "v1"); + ASSERT_OK(Flush(1)); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v1 ]"); + + // Write two new keys + Put(1, "a", "begin"); + Put(1, "z", "end"); + Flush(1); + + // Case1: Delete followed by a put + Delete(1, "foo"); + Put(1, "foo", "v2"); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, DEL, v1 ]"); + + // After the current memtable is flushed, the DEL should + // have been removed + ASSERT_OK(Flush(1)); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, v1 ]"); + + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2 ]"); + + // Case 2: Delete followed by another delete + Delete(1, "foo"); + Delete(1, "foo"); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, DEL, v2 ]"); + ASSERT_OK(Flush(1)); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, v2 ]"); + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ ]"); + + // Case 3: Put followed by a delete + Put(1, "foo", "v3"); + Delete(1, "foo"); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, v3 ]"); + ASSERT_OK(Flush(1)); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL ]"); + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ ]"); + + // Case 4: Put followed by another Put + Put(1, "foo", "v4"); + Put(1, "foo", "v5"); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v5, v4 ]"); + ASSERT_OK(Flush(1)); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v5 ]"); + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v5 ]"); + + // clear database + Delete(1, "foo"); + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ ]"); + + // Case 5: Put followed by snapshot followed by another Put + // Both puts should remain. + Put(1, "foo", "v6"); + const Snapshot* snapshot = db_->GetSnapshot(); + Put(1, "foo", "v7"); + ASSERT_OK(Flush(1)); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v7, v6 ]"); + db_->ReleaseSnapshot(snapshot); + + // clear database + Delete(1, "foo"); + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ ]"); + + // Case 5: snapshot followed by a put followed by another Put + // Only the last put should remain. + const Snapshot* snapshot1 = db_->GetSnapshot(); + Put(1, "foo", "v8"); + Put(1, "foo", "v9"); + ASSERT_OK(Flush(1)); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v9 ]"); + db_->ReleaseSnapshot(snapshot1); + } while (ChangeCompactOptions()); +} + +TEST_F(DBBasicTest, FlushOneColumnFamily) { + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu", "ilya", "muromec", "dobrynia", "nikitich", + "alyosha", "popovich"}, + options); + + ASSERT_OK(Put(0, "Default", "Default")); + ASSERT_OK(Put(1, "pikachu", "pikachu")); + ASSERT_OK(Put(2, "ilya", "ilya")); + ASSERT_OK(Put(3, "muromec", "muromec")); + ASSERT_OK(Put(4, "dobrynia", "dobrynia")); + ASSERT_OK(Put(5, "nikitich", "nikitich")); + ASSERT_OK(Put(6, "alyosha", "alyosha")); + ASSERT_OK(Put(7, "popovich", "popovich")); + + for (int i = 0; i < 8; ++i) { + Flush(i); + auto tables = ListTableFiles(env_, dbname_); + ASSERT_EQ(tables.size(), i + 1U); + } +} + +TEST_F(DBBasicTest, MultiGetSimple) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + SetPerfLevel(kEnableCount); + ASSERT_OK(Put(1, "k1", "v1")); + ASSERT_OK(Put(1, "k2", "v2")); + ASSERT_OK(Put(1, "k3", "v3")); + ASSERT_OK(Put(1, "k4", "v4")); + ASSERT_OK(Delete(1, "k4")); + ASSERT_OK(Put(1, "k5", "v5")); + ASSERT_OK(Delete(1, "no_key")); + + std::vector keys({"k1", "k2", "k3", "k4", "k5", "no_key"}); + + std::vector values(20, "Temporary data to be overwritten"); + std::vector cfs(keys.size(), handles_[1]); + + get_perf_context()->Reset(); + std::vector s = db_->MultiGet(ReadOptions(), cfs, keys, &values); + ASSERT_EQ(values.size(), keys.size()); + ASSERT_EQ(values[0], "v1"); + ASSERT_EQ(values[1], "v2"); + ASSERT_EQ(values[2], "v3"); + ASSERT_EQ(values[4], "v5"); + // four kv pairs * two bytes per value + ASSERT_EQ(8, (int)get_perf_context()->multiget_read_bytes); + + ASSERT_OK(s[0]); + ASSERT_OK(s[1]); + ASSERT_OK(s[2]); + ASSERT_TRUE(s[3].IsNotFound()); + ASSERT_OK(s[4]); + ASSERT_TRUE(s[5].IsNotFound()); + SetPerfLevel(kDisable); + } while (ChangeCompactOptions()); +} + +TEST_F(DBBasicTest, MultiGetEmpty) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + // Empty Key Set + std::vector keys; + std::vector values; + std::vector cfs; + std::vector s = db_->MultiGet(ReadOptions(), cfs, keys, &values); + ASSERT_EQ(s.size(), 0U); + + // Empty Database, Empty Key Set + Options options = CurrentOptions(); + options.create_if_missing = true; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + s = db_->MultiGet(ReadOptions(), cfs, keys, &values); + ASSERT_EQ(s.size(), 0U); + + // Empty Database, Search for Keys + keys.resize(2); + keys[0] = "a"; + keys[1] = "b"; + cfs.push_back(handles_[0]); + cfs.push_back(handles_[1]); + s = db_->MultiGet(ReadOptions(), cfs, keys, &values); + ASSERT_EQ(static_cast(s.size()), 2); + ASSERT_TRUE(s[0].IsNotFound() && s[1].IsNotFound()); + } while (ChangeCompactOptions()); +} + +TEST_F(DBBasicTest, ChecksumTest) { + BlockBasedTableOptions table_options; + Options options = CurrentOptions(); + // change when new checksum type added + int max_checksum = static_cast(kxxHash); + const int kNumPerFile = 2; + + // generate one table with each type of checksum + for (int i = 0; i <= max_checksum; ++i) { + table_options.checksum = static_cast(i); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + Reopen(options); + for (int j = 0; j < kNumPerFile; ++j) { + ASSERT_OK(Put(Key(i * kNumPerFile + j), Key(i * kNumPerFile + j))); + } + ASSERT_OK(Flush()); + } + + // verify data with each type of checksum + for (int i = 0; i <= kxxHash; ++i) { + table_options.checksum = static_cast(i); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + Reopen(options); + for (int j = 0; j < (max_checksum + 1) * kNumPerFile; ++j) { + ASSERT_EQ(Key(j), Get(Key(j))); + } + } +} + +// On Windows you can have either memory mapped file or a file +// with unbuffered access. So this asserts and does not make +// sense to run +#ifndef OS_WIN +TEST_F(DBBasicTest, MmapAndBufferOptions) { + if (!IsMemoryMappedAccessSupported()) { + return; + } + Options options = CurrentOptions(); + + options.use_direct_reads = true; + options.allow_mmap_reads = true; + ASSERT_NOK(TryReopen(options)); + + // All other combinations are acceptable + options.use_direct_reads = false; + ASSERT_OK(TryReopen(options)); + + if (IsDirectIOSupported()) { + options.use_direct_reads = true; + options.allow_mmap_reads = false; + ASSERT_OK(TryReopen(options)); + } + + options.use_direct_reads = false; + ASSERT_OK(TryReopen(options)); +} +#endif + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_bench.cc b/db/db_bench.cc deleted file mode 100644 index 2f88e81ffaa..00000000000 --- a/db/db_bench.cc +++ /dev/null @@ -1,2803 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#define __STDC_FORMAT_MACROS - -#ifndef GFLAGS -#include -int main() { - fprintf(stderr, "Please install gflags to run rocksdb tools\n"); - return 1; -} -#else - -#ifdef NUMA -#include -#include -#endif - -#include -#include -#include -#include -#include -#include -#include "db/db_impl.h" -#include "db/version_set.h" -#include "rocksdb/options.h" -#include "rocksdb/cache.h" -#include "rocksdb/db.h" -#include "rocksdb/env.h" -#include "rocksdb/memtablerep.h" -#include "rocksdb/write_batch.h" -#include "rocksdb/slice.h" -#include "rocksdb/slice_transform.h" -#include "rocksdb/statistics.h" -#include "rocksdb/perf_context.h" -#include "port/port.h" -#include "port/stack_trace.h" -#include "util/crc32c.h" -#include "util/histogram.h" -#include "util/mutexlock.h" -#include "util/random.h" -#include "util/string_util.h" -#include "util/statistics.h" -#include "util/testutil.h" -#include "util/xxhash.h" -#include "hdfs/env_hdfs.h" -#include "utilities/merge_operators.h" - -using GFLAGS::ParseCommandLineFlags; -using GFLAGS::RegisterFlagValidator; -using GFLAGS::SetUsageMessage; - -DEFINE_string(benchmarks, - "fillseq," - "fillsync," - "fillrandom," - "overwrite," - "readrandom," - "newiterator," - "newiteratorwhilewriting," - "seekrandom," - "seekrandomwhilewriting," - "readseq," - "readreverse," - "compact," - "readrandom," - "multireadrandom," - "readseq," - "readtocache," - "readreverse," - "readwhilewriting," - "readrandomwriterandom," - "updaterandom," - "randomwithverify," - "fill100K," - "crc32c," - "xxhash," - "compress," - "uncompress," - "acquireload,", - - "Comma-separated list of operations to run in the specified order" - "Actual benchmarks:\n" - "\tfillseq -- write N values in sequential key" - " order in async mode\n" - "\tfillrandom -- write N values in random key order in async" - " mode\n" - "\toverwrite -- overwrite N values in random key order in" - " async mode\n" - "\tfillsync -- write N/100 values in random key order in " - "sync mode\n" - "\tfill100K -- write N/1000 100K values in random order in" - " async mode\n" - "\tdeleteseq -- delete N keys in sequential order\n" - "\tdeleterandom -- delete N keys in random order\n" - "\treadseq -- read N times sequentially\n" - "\treadtocache -- 1 thread reading database sequentially\n" - "\treadreverse -- read N times in reverse order\n" - "\treadrandom -- read N times in random order\n" - "\treadmissing -- read N missing keys in random order\n" - "\treadhot -- read N times in random order from 1% section " - "of DB\n" - "\treadwhilewriting -- 1 writer, N threads doing random " - "reads\n" - "\treadrandomwriterandom -- N threads doing random-read, " - "random-write\n" - "\tprefixscanrandom -- prefix scan N times in random order\n" - "\tupdaterandom -- N threads doing read-modify-write for random " - "keys\n" - "\tappendrandom -- N threads doing read-modify-write with " - "growing values\n" - "\tmergerandom -- same as updaterandom/appendrandom using merge" - " operator. " - "Must be used with merge_operator\n" - "\treadrandommergerandom -- perform N random read-or-merge " - "operations. Must be used with merge_operator\n" - "\tnewiterator -- repeated iterator creation\n" - "\tseekrandom -- N random seeks\n" - "\tseekrandom -- 1 writer, N threads doing random seeks\n" - "\tcrc32c -- repeated crc32c of 4K of data\n" - "\txxhash -- repeated xxHash of 4K of data\n" - "\tacquireload -- load N*1000 times\n" - "Meta operations:\n" - "\tcompact -- Compact the entire DB\n" - "\tstats -- Print DB stats\n" - "\tlevelstats -- Print the number of files and bytes per level\n" - "\tsstables -- Print sstable info\n" - "\theapprofile -- Dump a heap profile (if supported by this" - " port)\n"); - -DEFINE_int64(num, 1000000, "Number of key/values to place in database"); - -DEFINE_int64(numdistinct, 1000, - "Number of distinct keys to use. Used in RandomWithVerify to " - "read/write on fewer keys so that gets are more likely to find the" - " key and puts are more likely to update the same key"); - -DEFINE_int64(merge_keys, -1, - "Number of distinct keys to use for MergeRandom and " - "ReadRandomMergeRandom. " - "If negative, there will be FLAGS_num keys."); -DEFINE_int32(num_column_families, 1, "Number of Column Families to use."); - -DEFINE_int64(reads, -1, "Number of read operations to do. " - "If negative, do FLAGS_num reads."); - -DEFINE_int32(bloom_locality, 0, "Control bloom filter probes locality"); - -DEFINE_int64(seed, 0, "Seed base for random number generators. " - "When 0 it is deterministic."); - -DEFINE_int32(threads, 1, "Number of concurrent threads to run."); - -DEFINE_int32(duration, 0, "Time in seconds for the random-ops tests to run." - " When 0 then num & reads determine the test duration"); - -DEFINE_int32(value_size, 100, "Size of each value"); - -DEFINE_bool(use_uint64_comparator, false, "use Uint64 user comparator"); - -static bool ValidateKeySize(const char* flagname, int32_t value) { - return true; -} - -DEFINE_int32(key_size, 16, "size of each key"); - -DEFINE_int32(num_multi_db, 0, - "Number of DBs used in the benchmark. 0 means single DB."); - -DEFINE_double(compression_ratio, 0.5, "Arrange to generate values that shrink" - " to this fraction of their original size after compression"); - -DEFINE_bool(histogram, false, "Print histogram of operation timings"); - -DEFINE_bool(enable_numa, false, - "Make operations aware of NUMA architecture and bind memory " - "and cpus corresponding to nodes together. In NUMA, memory " - "in same node as CPUs are closer when compared to memory in " - "other nodes. Reads can be faster when the process is bound to " - "CPU and memory of same node. Use \"$numactl --hardware\" command " - "to see NUMA memory architecture."); - -DEFINE_int64(write_buffer_size, rocksdb::Options().write_buffer_size, - "Number of bytes to buffer in memtable before compacting"); - -DEFINE_int32(max_write_buffer_number, - rocksdb::Options().max_write_buffer_number, - "The number of in-memory memtables. Each memtable is of size" - "write_buffer_size."); - -DEFINE_int32(min_write_buffer_number_to_merge, - rocksdb::Options().min_write_buffer_number_to_merge, - "The minimum number of write buffers that will be merged together" - "before writing to storage. This is cheap because it is an" - "in-memory merge. If this feature is not enabled, then all these" - "write buffers are flushed to L0 as separate files and this " - "increases read amplification because a get request has to check" - " in all of these files. Also, an in-memory merge may result in" - " writing less data to storage if there are duplicate records " - " in each of these individual write buffers."); - -DEFINE_int32(max_background_compactions, - rocksdb::Options().max_background_compactions, - "The maximum number of concurrent background compactions" - " that can occur in parallel."); - -DEFINE_int32(max_background_flushes, - rocksdb::Options().max_background_flushes, - "The maximum number of concurrent background flushes" - " that can occur in parallel."); - -static rocksdb::CompactionStyle FLAGS_compaction_style_e; -DEFINE_int32(compaction_style, (int32_t) rocksdb::Options().compaction_style, - "style of compaction: level-based vs universal"); - -DEFINE_int32(universal_size_ratio, 0, - "Percentage flexibility while comparing file size" - " (for universal compaction only)."); - -DEFINE_int32(universal_min_merge_width, 0, "The minimum number of files in a" - " single compaction run (for universal compaction only)."); - -DEFINE_int32(universal_max_merge_width, 0, "The max number of files to compact" - " in universal style compaction"); - -DEFINE_int32(universal_max_size_amplification_percent, 0, - "The max size amplification for universal style compaction"); - -DEFINE_int32(universal_compression_size_percent, -1, - "The percentage of the database to compress for universal " - "compaction. -1 means compress everything."); - -DEFINE_int64(cache_size, -1, "Number of bytes to use as a cache of uncompressed" - "data. Negative means use default settings."); - -DEFINE_int32(block_size, rocksdb::BlockBasedTableOptions().block_size, - "Number of bytes in a block."); - -DEFINE_int32(block_restart_interval, - rocksdb::BlockBasedTableOptions().block_restart_interval, - "Number of keys between restart points " - "for delta encoding of keys."); - -DEFINE_int64(compressed_cache_size, -1, - "Number of bytes to use as a cache of compressed data."); - -DEFINE_int32(open_files, rocksdb::Options().max_open_files, - "Maximum number of files to keep open at the same time" - " (use default if == 0)"); - -DEFINE_int32(bloom_bits, -1, "Bloom filter bits per key. Negative means" - " use default settings."); -DEFINE_int32(memtable_bloom_bits, 0, "Bloom filter bits per key for memtable. " - "Negative means no bloom filter."); - -DEFINE_bool(use_existing_db, false, "If true, do not destroy the existing" - " database. If you set this flag and also specify a benchmark that" - " wants a fresh database, that benchmark will fail."); - -DEFINE_string(db, "", "Use the db with the following name."); - -static bool ValidateCacheNumshardbits(const char* flagname, int32_t value) { - if (value >= 20) { - fprintf(stderr, "Invalid value for --%s: %d, must be < 20\n", - flagname, value); - return false; - } - return true; -} -DEFINE_int32(cache_numshardbits, -1, "Number of shards for the block cache" - " is 2 ** cache_numshardbits. Negative means use default settings." - " This is applied only if FLAGS_cache_size is non-negative."); - -DEFINE_int32(cache_remove_scan_count_limit, 32, ""); - -DEFINE_bool(verify_checksum, false, "Verify checksum for every block read" - " from storage"); - -DEFINE_bool(statistics, false, "Database statistics"); -static class std::shared_ptr dbstats; - -DEFINE_int64(writes, -1, "Number of write operations to do. If negative, do" - " --num reads."); - -DEFINE_int32(writes_per_second, 0, "Per-thread rate limit on writes per second." - " No limit when <= 0. Only for the readwhilewriting test."); - -DEFINE_bool(sync, false, "Sync all writes to disk"); - -DEFINE_bool(disable_data_sync, false, "If true, do not wait until data is" - " synced to disk."); - -DEFINE_bool(use_fsync, false, "If true, issue fsync instead of fdatasync"); - -DEFINE_bool(disable_wal, false, "If true, do not write WAL for write."); - -DEFINE_string(wal_dir, "", "If not empty, use the given dir for WAL"); - -DEFINE_int32(num_levels, 7, "The total number of levels"); - -DEFINE_int32(target_file_size_base, 2 * 1048576, "Target file size at level-1"); - -DEFINE_int32(target_file_size_multiplier, 1, - "A multiplier to compute target level-N file size (N >= 2)"); - -DEFINE_uint64(max_bytes_for_level_base, 10 * 1048576, "Max bytes for level-1"); - -DEFINE_int32(max_bytes_for_level_multiplier, 10, - "A multiplier to compute max bytes for level-N (N >= 2)"); - -static std::vector FLAGS_max_bytes_for_level_multiplier_additional_v; -DEFINE_string(max_bytes_for_level_multiplier_additional, "", - "A vector that specifies additional fanout per level"); - -DEFINE_int32(level0_stop_writes_trigger, 12, "Number of files in level-0" - " that will trigger put stop."); - -DEFINE_int32(level0_slowdown_writes_trigger, 8, "Number of files in level-0" - " that will slow down writes."); - -DEFINE_int32(level0_file_num_compaction_trigger, 4, "Number of files in level-0" - " when compactions start"); - -static bool ValidateInt32Percent(const char* flagname, int32_t value) { - if (value <= 0 || value>=100) { - fprintf(stderr, "Invalid value for --%s: %d, 0< pct <100 \n", - flagname, value); - return false; - } - return true; -} -DEFINE_int32(readwritepercent, 90, "Ratio of reads to reads/writes (expressed" - " as percentage) for the ReadRandomWriteRandom workload. The " - "default value 90 means 90% operations out of all reads and writes" - " operations are reads. In other words, 9 gets for every 1 put."); - -DEFINE_int32(mergereadpercent, 70, "Ratio of merges to merges&reads (expressed" - " as percentage) for the ReadRandomMergeRandom workload. The" - " default value 70 means 70% out of all read and merge operations" - " are merges. In other words, 7 merges for every 3 gets."); - -DEFINE_int32(deletepercent, 2, "Percentage of deletes out of reads/writes/" - "deletes (used in RandomWithVerify only). RandomWithVerify " - "calculates writepercent as (100 - FLAGS_readwritepercent - " - "deletepercent), so deletepercent must be smaller than (100 - " - "FLAGS_readwritepercent)"); - -DEFINE_uint64(delete_obsolete_files_period_micros, 0, "Option to delete " - "obsolete files periodically. 0 means that obsolete files are" - " deleted after every compaction run."); - -namespace { -enum rocksdb::CompressionType StringToCompressionType(const char* ctype) { - assert(ctype); - - if (!strcasecmp(ctype, "none")) - return rocksdb::kNoCompression; - else if (!strcasecmp(ctype, "snappy")) - return rocksdb::kSnappyCompression; - else if (!strcasecmp(ctype, "zlib")) - return rocksdb::kZlibCompression; - else if (!strcasecmp(ctype, "bzip2")) - return rocksdb::kBZip2Compression; - else if (!strcasecmp(ctype, "lz4")) - return rocksdb::kLZ4Compression; - else if (!strcasecmp(ctype, "lz4hc")) - return rocksdb::kLZ4HCCompression; - - fprintf(stdout, "Cannot parse compression type '%s'\n", ctype); - return rocksdb::kSnappyCompression; //default value -} -} // namespace - -DEFINE_string(compression_type, "snappy", - "Algorithm to use to compress the database"); -static enum rocksdb::CompressionType FLAGS_compression_type_e = - rocksdb::kSnappyCompression; - -DEFINE_int32(compression_level, -1, - "Compression level. For zlib this should be -1 for the " - "default level, or between 0 and 9."); - -static bool ValidateCompressionLevel(const char* flagname, int32_t value) { - if (value < -1 || value > 9) { - fprintf(stderr, "Invalid value for --%s: %d, must be between -1 and 9\n", - flagname, value); - return false; - } - return true; -} - -static const bool FLAGS_compression_level_dummy __attribute__((unused)) = - RegisterFlagValidator(&FLAGS_compression_level, &ValidateCompressionLevel); - -DEFINE_int32(min_level_to_compress, -1, "If non-negative, compression starts" - " from this level. Levels with number < min_level_to_compress are" - " not compressed. Otherwise, apply compression_type to " - "all levels."); - -static bool ValidateTableCacheNumshardbits(const char* flagname, - int32_t value) { - if (0 >= value || value > 20) { - fprintf(stderr, "Invalid value for --%s: %d, must be 0 < val <= 20\n", - flagname, value); - return false; - } - return true; -} -DEFINE_int32(table_cache_numshardbits, 4, ""); - -DEFINE_string(hdfs, "", "Name of hdfs environment"); -// posix or hdfs environment -static rocksdb::Env* FLAGS_env = rocksdb::Env::Default(); - -DEFINE_int64(stats_interval, 0, "Stats are reported every N operations when " - "this is greater than zero. When 0 the interval grows over time."); - -DEFINE_int32(stats_per_interval, 0, "Reports additional stats per interval when" - " this is greater than 0."); - -DEFINE_int32(perf_level, 0, "Level of perf collection"); - -static bool ValidateRateLimit(const char* flagname, double value) { - static constexpr double EPSILON = 1e-10; - if ( value < -EPSILON ) { - fprintf(stderr, "Invalid value for --%s: %12.6f, must be >= 0.0\n", - flagname, value); - return false; - } - return true; -} -DEFINE_double(soft_rate_limit, 0.0, ""); - -DEFINE_double(hard_rate_limit, 0.0, "When not equal to 0 this make threads " - "sleep at each stats reporting interval until the compaction" - " score for all levels is less than or equal to this value."); - -DEFINE_int32(rate_limit_delay_max_milliseconds, 1000, - "When hard_rate_limit is set then this is the max time a put will" - " be stalled."); - -DEFINE_int32(max_grandparent_overlap_factor, 10, "Control maximum bytes of " - "overlaps in grandparent (i.e., level+2) before we stop building a" - " single file in a level->level+1 compaction."); - -DEFINE_bool(readonly, false, "Run read only benchmarks."); - -DEFINE_bool(disable_auto_compactions, false, "Do not auto trigger compactions"); - -DEFINE_int32(source_compaction_factor, 1, "Cap the size of data in level-K for" - " a compaction run that compacts Level-K with Level-(K+1) (for" - " K >= 1)"); - -DEFINE_uint64(wal_ttl_seconds, 0, "Set the TTL for the WAL Files in seconds."); -DEFINE_uint64(wal_size_limit_MB, 0, "Set the size limit for the WAL Files" - " in MB."); - -DEFINE_bool(bufferedio, rocksdb::EnvOptions().use_os_buffer, - "Allow buffered io using OS buffers"); - -DEFINE_bool(mmap_read, rocksdb::EnvOptions().use_mmap_reads, - "Allow reads to occur via mmap-ing files"); - -DEFINE_bool(mmap_write, rocksdb::EnvOptions().use_mmap_writes, - "Allow writes to occur via mmap-ing files"); - -DEFINE_bool(advise_random_on_open, rocksdb::Options().advise_random_on_open, - "Advise random access on table file open"); - -DEFINE_string(compaction_fadvice, "NORMAL", - "Access pattern advice when a file is compacted"); -static auto FLAGS_compaction_fadvice_e = - rocksdb::Options().access_hint_on_compaction_start; - -DEFINE_bool(use_tailing_iterator, false, - "Use tailing iterator to access a series of keys instead of get"); -DEFINE_int64(iter_refresh_interval_us, -1, - "How often to refresh iterators. Disable refresh when -1"); - -DEFINE_bool(use_adaptive_mutex, rocksdb::Options().use_adaptive_mutex, - "Use adaptive mutex"); - -DEFINE_uint64(bytes_per_sync, rocksdb::Options().bytes_per_sync, - "Allows OS to incrementally sync files to disk while they are" - " being written, in the background. Issue one request for every" - " bytes_per_sync written. 0 turns it off."); -DEFINE_bool(filter_deletes, false, " On true, deletes use bloom-filter and drop" - " the delete if key not present"); - -DEFINE_int32(max_successive_merges, 0, "Maximum number of successive merge" - " operations on a key in the memtable"); - -static bool ValidatePrefixSize(const char* flagname, int32_t value) { - if (value < 0 || value>=2000000000) { - fprintf(stderr, "Invalid value for --%s: %d. 0<= PrefixSize <=2000000000\n", - flagname, value); - return false; - } - return true; -} -DEFINE_int32(prefix_size, 0, "control the prefix size for HashSkipList and " - "plain table"); -DEFINE_int64(keys_per_prefix, 0, "control average number of keys generated " - "per prefix, 0 means no special handling of the prefix, " - "i.e. use the prefix comes with the generated random number."); -DEFINE_bool(enable_io_prio, false, "Lower the background flush/compaction " - "threads' IO priority"); - -enum RepFactory { - kSkipList, - kPrefixHash, - kVectorRep, - kHashLinkedList, - kCuckoo -}; - -namespace { -enum RepFactory StringToRepFactory(const char* ctype) { - assert(ctype); - - if (!strcasecmp(ctype, "skip_list")) - return kSkipList; - else if (!strcasecmp(ctype, "prefix_hash")) - return kPrefixHash; - else if (!strcasecmp(ctype, "vector")) - return kVectorRep; - else if (!strcasecmp(ctype, "hash_linkedlist")) - return kHashLinkedList; - else if (!strcasecmp(ctype, "cuckoo")) - return kCuckoo; - - fprintf(stdout, "Cannot parse memreptable %s\n", ctype); - return kSkipList; -} -} // namespace - -static enum RepFactory FLAGS_rep_factory; -DEFINE_string(memtablerep, "skip_list", ""); -DEFINE_int64(hash_bucket_count, 1024 * 1024, "hash bucket count"); -DEFINE_bool(use_plain_table, false, "if use plain table " - "instead of block-based table format"); -DEFINE_bool(use_cuckoo_table, false, "if use cuckoo table format"); -DEFINE_double(cuckoo_hash_ratio, 0.9, "Hash ratio for Cuckoo SST table."); -DEFINE_bool(use_hash_search, false, "if use kHashSearch " - "instead of kBinarySearch. " - "This is valid if only we use BlockTable"); - -DEFINE_string(merge_operator, "", "The merge operator to use with the database." - "If a new merge operator is specified, be sure to use fresh" - " database The possible merge operators are defined in" - " utilities/merge_operators.h"); - -static const bool FLAGS_soft_rate_limit_dummy __attribute__((unused)) = - RegisterFlagValidator(&FLAGS_soft_rate_limit, &ValidateRateLimit); - -static const bool FLAGS_hard_rate_limit_dummy __attribute__((unused)) = - RegisterFlagValidator(&FLAGS_hard_rate_limit, &ValidateRateLimit); - -static const bool FLAGS_prefix_size_dummy __attribute__((unused)) = - RegisterFlagValidator(&FLAGS_prefix_size, &ValidatePrefixSize); - -static const bool FLAGS_key_size_dummy __attribute__((unused)) = - RegisterFlagValidator(&FLAGS_key_size, &ValidateKeySize); - -static const bool FLAGS_cache_numshardbits_dummy __attribute__((unused)) = - RegisterFlagValidator(&FLAGS_cache_numshardbits, - &ValidateCacheNumshardbits); - -static const bool FLAGS_readwritepercent_dummy __attribute__((unused)) = - RegisterFlagValidator(&FLAGS_readwritepercent, &ValidateInt32Percent); - -DEFINE_int32(disable_seek_compaction, false, - "Not used, left here for backwards compatibility"); - -static const bool FLAGS_deletepercent_dummy __attribute__((unused)) = - RegisterFlagValidator(&FLAGS_deletepercent, &ValidateInt32Percent); -static const bool FLAGS_table_cache_numshardbits_dummy __attribute__((unused)) = - RegisterFlagValidator(&FLAGS_table_cache_numshardbits, - &ValidateTableCacheNumshardbits); - -namespace rocksdb { - -// Helper for quickly generating random data. -class RandomGenerator { - private: - std::string data_; - unsigned int pos_; - - public: - RandomGenerator() { - // We use a limited amount of data over and over again and ensure - // that it is larger than the compression window (32KB), and also - // large enough to serve all typical value sizes we want to write. - Random rnd(301); - std::string piece; - while (data_.size() < (unsigned)std::max(1048576, FLAGS_value_size)) { - // Add a short fragment that is as compressible as specified - // by FLAGS_compression_ratio. - test::CompressibleString(&rnd, FLAGS_compression_ratio, 100, &piece); - data_.append(piece); - } - pos_ = 0; - } - - Slice Generate(unsigned int len) { - assert(len <= data_.size()); - if (pos_ + len > data_.size()) { - pos_ = 0; - } - pos_ += len; - return Slice(data_.data() + pos_ - len, len); - } -}; - -static void AppendWithSpace(std::string* str, Slice msg) { - if (msg.empty()) return; - if (!str->empty()) { - str->push_back(' '); - } - str->append(msg.data(), msg.size()); -} - -class Stats { - private: - int id_; - double start_; - double finish_; - double seconds_; - int64_t done_; - int64_t last_report_done_; - int64_t next_report_; - int64_t bytes_; - double last_op_finish_; - double last_report_finish_; - HistogramImpl hist_; - std::string message_; - bool exclude_from_merge_; - - public: - Stats() { Start(-1); } - - void Start(int id) { - id_ = id; - next_report_ = FLAGS_stats_interval ? FLAGS_stats_interval : 100; - last_op_finish_ = start_; - hist_.Clear(); - done_ = 0; - last_report_done_ = 0; - bytes_ = 0; - seconds_ = 0; - start_ = FLAGS_env->NowMicros(); - finish_ = start_; - last_report_finish_ = start_; - message_.clear(); - // When set, stats from this thread won't be merged with others. - exclude_from_merge_ = false; - } - - void Merge(const Stats& other) { - if (other.exclude_from_merge_) - return; - - hist_.Merge(other.hist_); - done_ += other.done_; - bytes_ += other.bytes_; - seconds_ += other.seconds_; - if (other.start_ < start_) start_ = other.start_; - if (other.finish_ > finish_) finish_ = other.finish_; - - // Just keep the messages from one thread - if (message_.empty()) message_ = other.message_; - } - - void Stop() { - finish_ = FLAGS_env->NowMicros(); - seconds_ = (finish_ - start_) * 1e-6; - } - - void AddMessage(Slice msg) { - AppendWithSpace(&message_, msg); - } - - void SetId(int id) { id_ = id; } - void SetExcludeFromMerge() { exclude_from_merge_ = true; } - - void FinishedOps(DB* db, int64_t num_ops) { - if (FLAGS_histogram) { - double now = FLAGS_env->NowMicros(); - double micros = now - last_op_finish_; - hist_.Add(micros); - if (micros > 20000 && !FLAGS_stats_interval) { - fprintf(stderr, "long op: %.1f micros%30s\r", micros, ""); - fflush(stderr); - } - last_op_finish_ = now; - } - - done_ += num_ops; - if (done_ >= next_report_) { - if (!FLAGS_stats_interval) { - if (next_report_ < 1000) next_report_ += 100; - else if (next_report_ < 5000) next_report_ += 500; - else if (next_report_ < 10000) next_report_ += 1000; - else if (next_report_ < 50000) next_report_ += 5000; - else if (next_report_ < 100000) next_report_ += 10000; - else if (next_report_ < 500000) next_report_ += 50000; - else next_report_ += 100000; - fprintf(stderr, "... finished %" PRIu64 " ops%30s\r", done_, ""); - fflush(stderr); - } else { - double now = FLAGS_env->NowMicros(); - fprintf(stderr, - "%s ... thread %d: (%" PRIu64 ",%" PRIu64 ") ops and " - "(%.1f,%.1f) ops/second in (%.6f,%.6f) seconds\n", - FLAGS_env->TimeToString((uint64_t) now/1000000).c_str(), - id_, - done_ - last_report_done_, done_, - (done_ - last_report_done_) / - ((now - last_report_finish_) / 1000000.0), - done_ / ((now - start_) / 1000000.0), - (now - last_report_finish_) / 1000000.0, - (now - start_) / 1000000.0); - - if (FLAGS_stats_per_interval) { - std::string stats; - if (db && db->GetProperty("rocksdb.stats", &stats)) - fprintf(stderr, "%s\n", stats.c_str()); - } - - fflush(stderr); - next_report_ += FLAGS_stats_interval; - last_report_finish_ = now; - last_report_done_ = done_; - } - } - } - - void AddBytes(int64_t n) { - bytes_ += n; - } - - void Report(const Slice& name) { - // Pretend at least one op was done in case we are running a benchmark - // that does not call FinishedOps(). - if (done_ < 1) done_ = 1; - - std::string extra; - if (bytes_ > 0) { - // Rate is computed on actual elapsed time, not the sum of per-thread - // elapsed times. - double elapsed = (finish_ - start_) * 1e-6; - char rate[100]; - snprintf(rate, sizeof(rate), "%6.1f MB/s", - (bytes_ / 1048576.0) / elapsed); - extra = rate; - } - AppendWithSpace(&extra, message_); - double elapsed = (finish_ - start_) * 1e-6; - double throughput = (double)done_/elapsed; - - fprintf(stdout, "%-12s : %11.3f micros/op %ld ops/sec;%s%s\n", - name.ToString().c_str(), - elapsed * 1e6 / done_, - (long)throughput, - (extra.empty() ? "" : " "), - extra.c_str()); - if (FLAGS_histogram) { - fprintf(stdout, "Microseconds per op:\n%s\n", hist_.ToString().c_str()); - } - fflush(stdout); - } -}; - -// State shared by all concurrent executions of the same benchmark. -struct SharedState { - port::Mutex mu; - port::CondVar cv; - int total; - int perf_level; - - // Each thread goes through the following states: - // (1) initializing - // (2) waiting for others to be initialized - // (3) running - // (4) done - - long num_initialized; - long num_done; - bool start; - - SharedState() : cv(&mu), perf_level(FLAGS_perf_level) { } -}; - -// Per-thread state for concurrent executions of the same benchmark. -struct ThreadState { - int tid; // 0..n-1 when running in n threads - Random64 rand; // Has different seeds for different threads - Stats stats; - SharedState* shared; - - /* implicit */ ThreadState(int index) - : tid(index), - rand((FLAGS_seed ? FLAGS_seed : 1000) + index) { - } -}; - -class Duration { - public: - Duration(int max_seconds, int64_t max_ops) { - max_seconds_ = max_seconds; - max_ops_= max_ops; - ops_ = 0; - start_at_ = FLAGS_env->NowMicros(); - } - - bool Done(int64_t increment) { - if (increment <= 0) increment = 1; // avoid Done(0) and infinite loops - ops_ += increment; - - if (max_seconds_) { - // Recheck every appx 1000 ops (exact iff increment is factor of 1000) - if ((ops_/1000) != ((ops_-increment)/1000)) { - double now = FLAGS_env->NowMicros(); - return ((now - start_at_) / 1000000.0) >= max_seconds_; - } else { - return false; - } - } else { - return ops_ > max_ops_; - } - } - - private: - int max_seconds_; - int64_t max_ops_; - int64_t ops_; - double start_at_; -}; - -class Benchmark { - private: - std::shared_ptr cache_; - std::shared_ptr compressed_cache_; - std::shared_ptr filter_policy_; - const SliceTransform* prefix_extractor_; - struct DBWithColumnFamilies { - std::vector cfh; - DB* db; - DBWithColumnFamilies() : db(nullptr) { - cfh.clear(); - } - }; - DBWithColumnFamilies db_; - std::vector multi_dbs_; - int64_t num_; - int value_size_; - int key_size_; - int prefix_size_; - int64_t keys_per_prefix_; - int64_t entries_per_batch_; - WriteOptions write_options_; - int64_t reads_; - int64_t writes_; - int64_t readwrites_; - int64_t merge_keys_; - - bool SanityCheck() { - if (FLAGS_compression_ratio > 1) { - fprintf(stderr, "compression_ratio should be between 0 and 1\n"); - return false; - } - return true; - } - - void PrintHeader() { - PrintEnvironment(); - fprintf(stdout, "Keys: %d bytes each\n", FLAGS_key_size); - fprintf(stdout, "Values: %d bytes each (%d bytes after compression)\n", - FLAGS_value_size, - static_cast(FLAGS_value_size * FLAGS_compression_ratio + 0.5)); - fprintf(stdout, "Entries: %" PRIu64 "\n", num_); - fprintf(stdout, "Prefix: %d bytes\n", FLAGS_prefix_size); - fprintf(stdout, "Keys per prefix: %" PRIu64 "\n", keys_per_prefix_); - fprintf(stdout, "RawSize: %.1f MB (estimated)\n", - ((static_cast(FLAGS_key_size + FLAGS_value_size) * num_) - / 1048576.0)); - fprintf(stdout, "FileSize: %.1f MB (estimated)\n", - (((FLAGS_key_size + FLAGS_value_size * FLAGS_compression_ratio) - * num_) - / 1048576.0)); - fprintf(stdout, "Write rate limit: %d\n", FLAGS_writes_per_second); - if (FLAGS_enable_numa) { - fprintf(stderr, "Running in NUMA enabled mode.\n"); -#ifndef NUMA - fprintf(stderr, "NUMA is not defined in the system.\n"); - exit(1); -#else - if (numa_available() == -1) { - fprintf(stderr, "NUMA is not supported by the system.\n"); - exit(1); - } -#endif - } - switch (FLAGS_compression_type_e) { - case rocksdb::kNoCompression: - fprintf(stdout, "Compression: none\n"); - break; - case rocksdb::kSnappyCompression: - fprintf(stdout, "Compression: snappy\n"); - break; - case rocksdb::kZlibCompression: - fprintf(stdout, "Compression: zlib\n"); - break; - case rocksdb::kBZip2Compression: - fprintf(stdout, "Compression: bzip2\n"); - break; - case rocksdb::kLZ4Compression: - fprintf(stdout, "Compression: lz4\n"); - break; - case rocksdb::kLZ4HCCompression: - fprintf(stdout, "Compression: lz4hc\n"); - break; - } - - switch (FLAGS_rep_factory) { - case kPrefixHash: - fprintf(stdout, "Memtablerep: prefix_hash\n"); - break; - case kSkipList: - fprintf(stdout, "Memtablerep: skip_list\n"); - break; - case kVectorRep: - fprintf(stdout, "Memtablerep: vector\n"); - break; - case kHashLinkedList: - fprintf(stdout, "Memtablerep: hash_linkedlist\n"); - break; - case kCuckoo: - fprintf(stdout, "Memtablerep: cuckoo\n"); - break; - } - fprintf(stdout, "Perf Level: %d\n", FLAGS_perf_level); - - PrintWarnings(); - fprintf(stdout, "------------------------------------------------\n"); - } - - void PrintWarnings() { -#if defined(__GNUC__) && !defined(__OPTIMIZE__) - fprintf(stdout, - "WARNING: Optimization is disabled: benchmarks unnecessarily slow\n" - ); -#endif -#ifndef NDEBUG - fprintf(stdout, - "WARNING: Assertions are enabled; benchmarks unnecessarily slow\n"); -#endif - if (FLAGS_compression_type_e != rocksdb::kNoCompression) { - // The test string should not be too small. - const int len = FLAGS_block_size; - char* text = (char*) malloc(len+1); - bool result = true; - const char* name = nullptr; - std::string compressed; - - memset(text, (int) 'y', len); - text[len] = '\0'; - switch (FLAGS_compression_type_e) { - case kSnappyCompression: - result = port::Snappy_Compress(Options().compression_opts, text, - strlen(text), &compressed); - name = "Snappy"; - break; - case kZlibCompression: - result = port::Zlib_Compress(Options().compression_opts, text, - strlen(text), &compressed); - name = "Zlib"; - break; - case kBZip2Compression: - result = port::BZip2_Compress(Options().compression_opts, text, - strlen(text), &compressed); - name = "BZip2"; - break; - case kLZ4Compression: - result = port::LZ4_Compress(Options().compression_opts, text, - strlen(text), &compressed); - name = "LZ4"; - break; - case kLZ4HCCompression: - result = port::LZ4HC_Compress(Options().compression_opts, text, - strlen(text), &compressed); - name = "LZ4HC"; - break; - case kNoCompression: - assert(false); // cannot happen - break; - } - - if (!result) { - fprintf(stdout, "WARNING: %s compression is not enabled\n", name); - } else if (name && compressed.size() >= strlen(text)) { - fprintf(stdout, "WARNING: %s compression is not effective\n", name); - } - - free(text); - } - } - -// Current the following isn't equivalent to OS_LINUX. -#if defined(__linux) - static Slice TrimSpace(Slice s) { - unsigned int start = 0; - while (start < s.size() && isspace(s[start])) { - start++; - } - unsigned int limit = s.size(); - while (limit > start && isspace(s[limit-1])) { - limit--; - } - return Slice(s.data() + start, limit - start); - } -#endif - - void PrintEnvironment() { - fprintf(stderr, "LevelDB: version %d.%d\n", - kMajorVersion, kMinorVersion); - -#if defined(__linux) - time_t now = time(nullptr); - fprintf(stderr, "Date: %s", ctime(&now)); // ctime() adds newline - - FILE* cpuinfo = fopen("/proc/cpuinfo", "r"); - if (cpuinfo != nullptr) { - char line[1000]; - int num_cpus = 0; - std::string cpu_type; - std::string cache_size; - while (fgets(line, sizeof(line), cpuinfo) != nullptr) { - const char* sep = strchr(line, ':'); - if (sep == nullptr) { - continue; - } - Slice key = TrimSpace(Slice(line, sep - 1 - line)); - Slice val = TrimSpace(Slice(sep + 1)); - if (key == "model name") { - ++num_cpus; - cpu_type = val.ToString(); - } else if (key == "cache size") { - cache_size = val.ToString(); - } - } - fclose(cpuinfo); - fprintf(stderr, "CPU: %d * %s\n", num_cpus, cpu_type.c_str()); - fprintf(stderr, "CPUCache: %s\n", cache_size.c_str()); - } -#endif - } - - public: - Benchmark() - : cache_(FLAGS_cache_size >= 0 ? - (FLAGS_cache_numshardbits >= 1 ? - NewLRUCache(FLAGS_cache_size, FLAGS_cache_numshardbits, - FLAGS_cache_remove_scan_count_limit) : - NewLRUCache(FLAGS_cache_size)) : nullptr), - compressed_cache_(FLAGS_compressed_cache_size >= 0 ? - (FLAGS_cache_numshardbits >= 1 ? - NewLRUCache(FLAGS_compressed_cache_size, FLAGS_cache_numshardbits) : - NewLRUCache(FLAGS_compressed_cache_size)) : nullptr), - filter_policy_(FLAGS_bloom_bits >= 0 - ? NewBloomFilterPolicy(FLAGS_bloom_bits) - : nullptr), - prefix_extractor_(NewFixedPrefixTransform(FLAGS_prefix_size)), - num_(FLAGS_num), - value_size_(FLAGS_value_size), - key_size_(FLAGS_key_size), - prefix_size_(FLAGS_prefix_size), - keys_per_prefix_(FLAGS_keys_per_prefix), - entries_per_batch_(1), - reads_(FLAGS_reads < 0 ? FLAGS_num : FLAGS_reads), - writes_(FLAGS_writes < 0 ? FLAGS_num : FLAGS_writes), - readwrites_((FLAGS_writes < 0 && FLAGS_reads < 0)? FLAGS_num : - ((FLAGS_writes > FLAGS_reads) ? FLAGS_writes : FLAGS_reads) - ), - merge_keys_(FLAGS_merge_keys < 0 ? FLAGS_num : FLAGS_merge_keys) { - if (FLAGS_prefix_size > FLAGS_key_size) { - fprintf(stderr, "prefix size is larger than key size"); - exit(1); - } - - std::vector files; - FLAGS_env->GetChildren(FLAGS_db, &files); - for (unsigned int i = 0; i < files.size(); i++) { - if (Slice(files[i]).starts_with("heap-")) { - FLAGS_env->DeleteFile(FLAGS_db + "/" + files[i]); - } - } - if (!FLAGS_use_existing_db) { - DestroyDB(FLAGS_db, Options()); - } - } - - ~Benchmark() { - delete db_.db; - delete prefix_extractor_; - } - - Slice AllocateKey() { - return Slice(new char[key_size_], key_size_); - } - - // Generate key according to the given specification and random number. - // The resulting key will have the following format (if keys_per_prefix_ - // is positive), extra trailing bytes are either cut off or paddd with '0'. - // The prefix value is derived from key value. - // ---------------------------- - // | prefix 00000 | key 00000 | - // ---------------------------- - // If keys_per_prefix_ is 0, the key is simply a binary representation of - // random number followed by trailing '0's - // ---------------------------- - // | key 00000 | - // ---------------------------- - void GenerateKeyFromInt(uint64_t v, int64_t num_keys, Slice* key) { - char* start = const_cast(key->data()); - char* pos = start; - if (keys_per_prefix_ > 0) { - int64_t num_prefix = num_keys / keys_per_prefix_; - int64_t prefix = v % num_prefix; - int bytes_to_fill = std::min(prefix_size_, 8); - if (port::kLittleEndian) { - for (int i = 0; i < bytes_to_fill; ++i) { - pos[i] = (prefix >> ((bytes_to_fill - i - 1) << 3)) & 0xFF; - } - } else { - memcpy(pos, static_cast(&prefix), bytes_to_fill); - } - if (prefix_size_ > 8) { - // fill the rest with 0s - memset(pos + 8, '0', prefix_size_ - 8); - } - pos += prefix_size_; - } - - int bytes_to_fill = std::min(key_size_ - static_cast(pos - start), 8); - if (port::kLittleEndian) { - for (int i = 0; i < bytes_to_fill; ++i) { - pos[i] = (v >> ((bytes_to_fill - i - 1) << 3)) & 0xFF; - } - } else { - memcpy(pos, static_cast(&v), bytes_to_fill); - } - pos += bytes_to_fill; - if (key_size_ > pos - start) { - memset(pos, '0', key_size_ - (pos - start)); - } - } - - std::string GetDbNameForMultiple(std::string base_name, size_t id) { - return base_name + std::to_string(id); - } - - std::string ColumnFamilyName(int i) { - if (i == 0) { - return kDefaultColumnFamilyName; - } else { - char name[100]; - snprintf(name, sizeof(name), "column_family_name_%06d", i); - return std::string(name); - } - } - - void Run() { - if (!SanityCheck()) { - exit(1); - } - PrintHeader(); - Open(); - const char* benchmarks = FLAGS_benchmarks.c_str(); - while (benchmarks != nullptr) { - const char* sep = strchr(benchmarks, ','); - Slice name; - if (sep == nullptr) { - name = benchmarks; - benchmarks = nullptr; - } else { - name = Slice(benchmarks, sep - benchmarks); - benchmarks = sep + 1; - } - - // Sanitize parameters - num_ = FLAGS_num; - reads_ = (FLAGS_reads < 0 ? FLAGS_num : FLAGS_reads); - writes_ = (FLAGS_writes < 0 ? FLAGS_num : FLAGS_writes); - value_size_ = FLAGS_value_size; - key_size_ = FLAGS_key_size; - entries_per_batch_ = 1; - write_options_ = WriteOptions(); - if (FLAGS_sync) { - write_options_.sync = true; - } - write_options_.disableWAL = FLAGS_disable_wal; - - void (Benchmark::*method)(ThreadState*) = nullptr; - bool fresh_db = false; - int num_threads = FLAGS_threads; - - if (name == Slice("fillseq")) { - fresh_db = true; - method = &Benchmark::WriteSeq; - } else if (name == Slice("fillbatch")) { - fresh_db = true; - entries_per_batch_ = 1000; - method = &Benchmark::WriteSeq; - } else if (name == Slice("fillrandom")) { - fresh_db = true; - method = &Benchmark::WriteRandom; - } else if (name == Slice("filluniquerandom")) { - fresh_db = true; - if (num_threads > 1) { - fprintf(stderr, "filluniquerandom multithreaded not supported" - ", use 1 thread"); - num_threads = 1; - } - method = &Benchmark::WriteUniqueRandom; - } else if (name == Slice("overwrite")) { - fresh_db = false; - method = &Benchmark::WriteRandom; - } else if (name == Slice("fillsync")) { - fresh_db = true; - num_ /= 1000; - write_options_.sync = true; - method = &Benchmark::WriteRandom; - } else if (name == Slice("fill100K")) { - fresh_db = true; - num_ /= 1000; - value_size_ = 100 * 1000; - method = &Benchmark::WriteRandom; - } else if (name == Slice("readseq")) { - method = &Benchmark::ReadSequential; - } else if (name == Slice("readtocache")) { - method = &Benchmark::ReadSequential; - num_threads = 1; - reads_ = num_; - } else if (name == Slice("readreverse")) { - method = &Benchmark::ReadReverse; - } else if (name == Slice("readrandom")) { - method = &Benchmark::ReadRandom; - } else if (name == Slice("multireadrandom")) { - method = &Benchmark::MultiReadRandom; - } else if (name == Slice("readmissing")) { - ++key_size_; - method = &Benchmark::ReadRandom; - } else if (name == Slice("newiterator")) { - method = &Benchmark::IteratorCreation; - } else if (name == Slice("newiteratorwhilewriting")) { - num_threads++; // Add extra thread for writing - method = &Benchmark::IteratorCreationWhileWriting; - } else if (name == Slice("seekrandom")) { - method = &Benchmark::SeekRandom; - } else if (name == Slice("seekrandomwhilewriting")) { - num_threads++; // Add extra thread for writing - method = &Benchmark::SeekRandomWhileWriting; - } else if (name == Slice("readrandomsmall")) { - reads_ /= 1000; - method = &Benchmark::ReadRandom; - } else if (name == Slice("deleteseq")) { - method = &Benchmark::DeleteSeq; - } else if (name == Slice("deleterandom")) { - method = &Benchmark::DeleteRandom; - } else if (name == Slice("readwhilewriting")) { - num_threads++; // Add extra thread for writing - method = &Benchmark::ReadWhileWriting; - } else if (name == Slice("readrandomwriterandom")) { - method = &Benchmark::ReadRandomWriteRandom; - } else if (name == Slice("readrandommergerandom")) { - if (FLAGS_merge_operator.empty()) { - fprintf(stdout, "%-12s : skipped (--merge_operator is unknown)\n", - name.ToString().c_str()); - exit(1); - } - method = &Benchmark::ReadRandomMergeRandom; - } else if (name == Slice("updaterandom")) { - method = &Benchmark::UpdateRandom; - } else if (name == Slice("appendrandom")) { - method = &Benchmark::AppendRandom; - } else if (name == Slice("mergerandom")) { - if (FLAGS_merge_operator.empty()) { - fprintf(stdout, "%-12s : skipped (--merge_operator is unknown)\n", - name.ToString().c_str()); - exit(1); - } - method = &Benchmark::MergeRandom; - } else if (name == Slice("randomwithverify")) { - method = &Benchmark::RandomWithVerify; - } else if (name == Slice("compact")) { - method = &Benchmark::Compact; - } else if (name == Slice("crc32c")) { - method = &Benchmark::Crc32c; - } else if (name == Slice("xxhash")) { - method = &Benchmark::xxHash; - } else if (name == Slice("acquireload")) { - method = &Benchmark::AcquireLoad; - } else if (name == Slice("compress")) { - method = &Benchmark::Compress; - } else if (name == Slice("uncompress")) { - method = &Benchmark::Uncompress; - } else if (name == Slice("stats")) { - PrintStats("rocksdb.stats"); - } else if (name == Slice("levelstats")) { - PrintStats("rocksdb.levelstats"); - } else if (name == Slice("sstables")) { - PrintStats("rocksdb.sstables"); - } else { - if (name != Slice()) { // No error message for empty name - fprintf(stderr, "unknown benchmark '%s'\n", name.ToString().c_str()); - exit(1); - } - } - - if (fresh_db) { - if (FLAGS_use_existing_db) { - fprintf(stdout, "%-12s : skipped (--use_existing_db is true)\n", - name.ToString().c_str()); - method = nullptr; - } else { - if (db_.db != nullptr) { - delete db_.db; - db_.db = nullptr; - db_.cfh.clear(); - DestroyDB(FLAGS_db, Options()); - } - for (size_t i = 0; i < multi_dbs_.size(); i++) { - delete multi_dbs_[i].db; - DestroyDB(GetDbNameForMultiple(FLAGS_db, i), Options()); - } - multi_dbs_.clear(); - } - Open(); - } - - if (method != nullptr) { - fprintf(stdout, "DB path: [%s]\n", FLAGS_db.c_str()); - RunBenchmark(num_threads, name, method); - } - } - if (FLAGS_statistics) { - fprintf(stdout, "STATISTICS:\n%s\n", dbstats->ToString().c_str()); - } - } - - private: - struct ThreadArg { - Benchmark* bm; - SharedState* shared; - ThreadState* thread; - void (Benchmark::*method)(ThreadState*); - }; - - static void ThreadBody(void* v) { - ThreadArg* arg = reinterpret_cast(v); - SharedState* shared = arg->shared; - ThreadState* thread = arg->thread; - { - MutexLock l(&shared->mu); - shared->num_initialized++; - if (shared->num_initialized >= shared->total) { - shared->cv.SignalAll(); - } - while (!shared->start) { - shared->cv.Wait(); - } - } - - SetPerfLevel(static_cast (shared->perf_level)); - thread->stats.Start(thread->tid); - (arg->bm->*(arg->method))(thread); - thread->stats.Stop(); - - { - MutexLock l(&shared->mu); - shared->num_done++; - if (shared->num_done >= shared->total) { - shared->cv.SignalAll(); - } - } - } - - void RunBenchmark(int n, Slice name, - void (Benchmark::*method)(ThreadState*)) { - SharedState shared; - shared.total = n; - shared.num_initialized = 0; - shared.num_done = 0; - shared.start = false; - - ThreadArg* arg = new ThreadArg[n]; - - for (int i = 0; i < n; i++) { -#ifdef NUMA - if (FLAGS_enable_numa) { - // Performs a local allocation of memory to threads in numa node. - int n_nodes = numa_num_task_nodes(); // Number of nodes in NUMA. - numa_exit_on_error = 1; - int numa_node = i % n_nodes; - bitmask* nodes = numa_allocate_nodemask(); - numa_bitmask_clearall(nodes); - numa_bitmask_setbit(nodes, numa_node); - // numa_bind() call binds the process to the node and these - // properties are passed on to the thread that is created in - // StartThread method called later in the loop. - numa_bind(nodes); - numa_set_strict(1); - numa_free_nodemask(nodes); - } -#endif - arg[i].bm = this; - arg[i].method = method; - arg[i].shared = &shared; - arg[i].thread = new ThreadState(i); - arg[i].thread->shared = &shared; - FLAGS_env->StartThread(ThreadBody, &arg[i]); - } - - shared.mu.Lock(); - while (shared.num_initialized < n) { - shared.cv.Wait(); - } - - shared.start = true; - shared.cv.SignalAll(); - while (shared.num_done < n) { - shared.cv.Wait(); - } - shared.mu.Unlock(); - - // Stats for some threads can be excluded. - Stats merge_stats; - for (int i = 0; i < n; i++) { - merge_stats.Merge(arg[i].thread->stats); - } - merge_stats.Report(name); - - for (int i = 0; i < n; i++) { - delete arg[i].thread; - } - delete[] arg; - } - - void Crc32c(ThreadState* thread) { - // Checksum about 500MB of data total - const int size = 4096; - const char* label = "(4K per op)"; - std::string data(size, 'x'); - int64_t bytes = 0; - uint32_t crc = 0; - while (bytes < 500 * 1048576) { - crc = crc32c::Value(data.data(), size); - thread->stats.FinishedOps(nullptr, 1); - bytes += size; - } - // Print so result is not dead - fprintf(stderr, "... crc=0x%x\r", static_cast(crc)); - - thread->stats.AddBytes(bytes); - thread->stats.AddMessage(label); - } - - void xxHash(ThreadState* thread) { - // Checksum about 500MB of data total - const int size = 4096; - const char* label = "(4K per op)"; - std::string data(size, 'x'); - int64_t bytes = 0; - unsigned int xxh32 = 0; - while (bytes < 500 * 1048576) { - xxh32 = XXH32(data.data(), size, 0); - thread->stats.FinishedOps(nullptr, 1); - bytes += size; - } - // Print so result is not dead - fprintf(stderr, "... xxh32=0x%x\r", static_cast(xxh32)); - - thread->stats.AddBytes(bytes); - thread->stats.AddMessage(label); - } - - void AcquireLoad(ThreadState* thread) { - int dummy; - port::AtomicPointer ap(&dummy); - int count = 0; - void *ptr = nullptr; - thread->stats.AddMessage("(each op is 1000 loads)"); - while (count < 100000) { - for (int i = 0; i < 1000; i++) { - ptr = ap.Acquire_Load(); - } - count++; - thread->stats.FinishedOps(nullptr, 1); - } - if (ptr == nullptr) exit(1); // Disable unused variable warning. - } - - void Compress(ThreadState *thread) { - RandomGenerator gen; - Slice input = gen.Generate(FLAGS_block_size); - int64_t bytes = 0; - int64_t produced = 0; - bool ok = true; - std::string compressed; - - // Compress 1G - while (ok && bytes < int64_t(1) << 30) { - switch (FLAGS_compression_type_e) { - case rocksdb::kSnappyCompression: - ok = port::Snappy_Compress(Options().compression_opts, input.data(), - input.size(), &compressed); - break; - case rocksdb::kZlibCompression: - ok = port::Zlib_Compress(Options().compression_opts, input.data(), - input.size(), &compressed); - break; - case rocksdb::kBZip2Compression: - ok = port::BZip2_Compress(Options().compression_opts, input.data(), - input.size(), &compressed); - break; - case rocksdb::kLZ4Compression: - ok = port::LZ4_Compress(Options().compression_opts, input.data(), - input.size(), &compressed); - break; - case rocksdb::kLZ4HCCompression: - ok = port::LZ4HC_Compress(Options().compression_opts, input.data(), - input.size(), &compressed); - break; - default: - ok = false; - } - produced += compressed.size(); - bytes += input.size(); - thread->stats.FinishedOps(nullptr, 1); - } - - if (!ok) { - thread->stats.AddMessage("(compression failure)"); - } else { - char buf[100]; - snprintf(buf, sizeof(buf), "(output: %.1f%%)", - (produced * 100.0) / bytes); - thread->stats.AddMessage(buf); - thread->stats.AddBytes(bytes); - } - } - - void Uncompress(ThreadState *thread) { - RandomGenerator gen; - Slice input = gen.Generate(FLAGS_block_size); - std::string compressed; - - bool ok; - switch (FLAGS_compression_type_e) { - case rocksdb::kSnappyCompression: - ok = port::Snappy_Compress(Options().compression_opts, input.data(), - input.size(), &compressed); - break; - case rocksdb::kZlibCompression: - ok = port::Zlib_Compress(Options().compression_opts, input.data(), - input.size(), &compressed); - break; - case rocksdb::kBZip2Compression: - ok = port::BZip2_Compress(Options().compression_opts, input.data(), - input.size(), &compressed); - break; - case rocksdb::kLZ4Compression: - ok = port::LZ4_Compress(Options().compression_opts, input.data(), - input.size(), &compressed); - break; - case rocksdb::kLZ4HCCompression: - ok = port::LZ4HC_Compress(Options().compression_opts, input.data(), - input.size(), &compressed); - break; - default: - ok = false; - } - - int64_t bytes = 0; - int decompress_size; - while (ok && bytes < 1024 * 1048576) { - char *uncompressed = nullptr; - switch (FLAGS_compression_type_e) { - case rocksdb::kSnappyCompression: - // allocate here to make comparison fair - uncompressed = new char[input.size()]; - ok = port::Snappy_Uncompress(compressed.data(), compressed.size(), - uncompressed); - break; - case rocksdb::kZlibCompression: - uncompressed = port::Zlib_Uncompress( - compressed.data(), compressed.size(), &decompress_size); - ok = uncompressed != nullptr; - break; - case rocksdb::kBZip2Compression: - uncompressed = port::BZip2_Uncompress( - compressed.data(), compressed.size(), &decompress_size); - ok = uncompressed != nullptr; - break; - case rocksdb::kLZ4Compression: - uncompressed = port::LZ4_Uncompress( - compressed.data(), compressed.size(), &decompress_size); - ok = uncompressed != nullptr; - break; - case rocksdb::kLZ4HCCompression: - uncompressed = port::LZ4_Uncompress( - compressed.data(), compressed.size(), &decompress_size); - ok = uncompressed != nullptr; - break; - default: - ok = false; - } - delete[] uncompressed; - bytes += input.size(); - thread->stats.FinishedOps(nullptr, 1); - } - - if (!ok) { - thread->stats.AddMessage("(compression failure)"); - } else { - thread->stats.AddBytes(bytes); - } - } - - void Open() { - assert(db_.db == nullptr); - Options options; - options.create_if_missing = !FLAGS_use_existing_db; - options.create_missing_column_families = FLAGS_num_column_families > 1; - options.write_buffer_size = FLAGS_write_buffer_size; - options.max_write_buffer_number = FLAGS_max_write_buffer_number; - options.min_write_buffer_number_to_merge = - FLAGS_min_write_buffer_number_to_merge; - options.max_background_compactions = FLAGS_max_background_compactions; - options.max_background_flushes = FLAGS_max_background_flushes; - options.compaction_style = FLAGS_compaction_style_e; - if (FLAGS_prefix_size != 0) { - options.prefix_extractor.reset( - NewFixedPrefixTransform(FLAGS_prefix_size)); - } - if (FLAGS_use_uint64_comparator) { - options.comparator = test::Uint64Comparator(); - if (FLAGS_key_size != 8) { - fprintf(stderr, "Using Uint64 comparator but key size is not 8.\n"); - exit(1); - } - } - options.memtable_prefix_bloom_bits = FLAGS_memtable_bloom_bits; - options.bloom_locality = FLAGS_bloom_locality; - options.max_open_files = FLAGS_open_files; - options.statistics = dbstats; - if (FLAGS_enable_io_prio) { - FLAGS_env->LowerThreadPoolIOPriority(Env::LOW); - FLAGS_env->LowerThreadPoolIOPriority(Env::HIGH); - } - options.env = FLAGS_env; - options.disableDataSync = FLAGS_disable_data_sync; - options.use_fsync = FLAGS_use_fsync; - options.wal_dir = FLAGS_wal_dir; - options.num_levels = FLAGS_num_levels; - options.target_file_size_base = FLAGS_target_file_size_base; - options.target_file_size_multiplier = FLAGS_target_file_size_multiplier; - options.max_bytes_for_level_base = FLAGS_max_bytes_for_level_base; - options.max_bytes_for_level_multiplier = - FLAGS_max_bytes_for_level_multiplier; - options.filter_deletes = FLAGS_filter_deletes; - if ((FLAGS_prefix_size == 0) && (FLAGS_rep_factory == kPrefixHash || - FLAGS_rep_factory == kHashLinkedList)) { - fprintf(stderr, "prefix_size should be non-zero if PrefixHash or " - "HashLinkedList memtablerep is used\n"); - exit(1); - } - switch (FLAGS_rep_factory) { - case kPrefixHash: - options.memtable_factory.reset(NewHashSkipListRepFactory( - FLAGS_hash_bucket_count)); - break; - case kSkipList: - // no need to do anything - break; - case kHashLinkedList: - options.memtable_factory.reset(NewHashLinkListRepFactory( - FLAGS_hash_bucket_count)); - break; - case kVectorRep: - options.memtable_factory.reset( - new VectorRepFactory - ); - break; - case kCuckoo: - options.memtable_factory.reset(NewHashCuckooRepFactory( - options.write_buffer_size, FLAGS_key_size + FLAGS_value_size)); - break; - } - if (FLAGS_use_plain_table) { - if (FLAGS_rep_factory != kPrefixHash && - FLAGS_rep_factory != kHashLinkedList) { - fprintf(stderr, "Waring: plain table is used with skipList\n"); - } - if (!FLAGS_mmap_read && !FLAGS_mmap_write) { - fprintf(stderr, "plain table format requires mmap to operate\n"); - exit(1); - } - - int bloom_bits_per_key = FLAGS_bloom_bits; - if (bloom_bits_per_key < 0) { - bloom_bits_per_key = 0; - } - - PlainTableOptions plain_table_options; - plain_table_options.user_key_len = FLAGS_key_size; - plain_table_options.bloom_bits_per_key = bloom_bits_per_key; - plain_table_options.hash_table_ratio = 0.75; - options.table_factory = std::shared_ptr( - NewPlainTableFactory(plain_table_options)); - } else if (FLAGS_use_cuckoo_table) { - if (FLAGS_cuckoo_hash_ratio > 1 || FLAGS_cuckoo_hash_ratio < 0) { - fprintf(stderr, "Invalid cuckoo_hash_ratio\n"); - exit(1); - } - options.table_factory = std::shared_ptr( - NewCuckooTableFactory(FLAGS_cuckoo_hash_ratio)); - } else { - BlockBasedTableOptions block_based_options; - if (FLAGS_use_hash_search) { - if (FLAGS_prefix_size == 0) { - fprintf(stderr, - "prefix_size not assigned when enable use_hash_search \n"); - exit(1); - } - block_based_options.index_type = BlockBasedTableOptions::kHashSearch; - } else { - block_based_options.index_type = BlockBasedTableOptions::kBinarySearch; - } - if (cache_ == nullptr) { - block_based_options.no_block_cache = true; - } - block_based_options.block_cache = cache_; - block_based_options.block_cache_compressed = compressed_cache_; - block_based_options.block_size = FLAGS_block_size; - block_based_options.block_restart_interval = FLAGS_block_restart_interval; - block_based_options.filter_policy = filter_policy_; - options.table_factory.reset( - NewBlockBasedTableFactory(block_based_options)); - } - if (FLAGS_max_bytes_for_level_multiplier_additional_v.size() > 0) { - if (FLAGS_max_bytes_for_level_multiplier_additional_v.size() != - (unsigned int)FLAGS_num_levels) { - fprintf(stderr, "Insufficient number of fanouts specified %d\n", - (int)FLAGS_max_bytes_for_level_multiplier_additional_v.size()); - exit(1); - } - options.max_bytes_for_level_multiplier_additional = - FLAGS_max_bytes_for_level_multiplier_additional_v; - } - options.level0_stop_writes_trigger = FLAGS_level0_stop_writes_trigger; - options.level0_file_num_compaction_trigger = - FLAGS_level0_file_num_compaction_trigger; - options.level0_slowdown_writes_trigger = - FLAGS_level0_slowdown_writes_trigger; - options.compression = FLAGS_compression_type_e; - options.compression_opts.level = FLAGS_compression_level; - options.WAL_ttl_seconds = FLAGS_wal_ttl_seconds; - options.WAL_size_limit_MB = FLAGS_wal_size_limit_MB; - if (FLAGS_min_level_to_compress >= 0) { - assert(FLAGS_min_level_to_compress <= FLAGS_num_levels); - options.compression_per_level.resize(FLAGS_num_levels); - for (int i = 0; i < FLAGS_min_level_to_compress; i++) { - options.compression_per_level[i] = kNoCompression; - } - for (int i = FLAGS_min_level_to_compress; - i < FLAGS_num_levels; i++) { - options.compression_per_level[i] = FLAGS_compression_type_e; - } - } - options.delete_obsolete_files_period_micros = - FLAGS_delete_obsolete_files_period_micros; - options.soft_rate_limit = FLAGS_soft_rate_limit; - options.hard_rate_limit = FLAGS_hard_rate_limit; - options.rate_limit_delay_max_milliseconds = - FLAGS_rate_limit_delay_max_milliseconds; - options.table_cache_numshardbits = FLAGS_table_cache_numshardbits; - options.max_grandparent_overlap_factor = - FLAGS_max_grandparent_overlap_factor; - options.disable_auto_compactions = FLAGS_disable_auto_compactions; - options.source_compaction_factor = FLAGS_source_compaction_factor; - - // fill storage options - options.allow_os_buffer = FLAGS_bufferedio; - options.allow_mmap_reads = FLAGS_mmap_read; - options.allow_mmap_writes = FLAGS_mmap_write; - options.advise_random_on_open = FLAGS_advise_random_on_open; - options.access_hint_on_compaction_start = FLAGS_compaction_fadvice_e; - options.use_adaptive_mutex = FLAGS_use_adaptive_mutex; - options.bytes_per_sync = FLAGS_bytes_per_sync; - - // merge operator options - options.merge_operator = MergeOperators::CreateFromStringId( - FLAGS_merge_operator); - if (options.merge_operator == nullptr && !FLAGS_merge_operator.empty()) { - fprintf(stderr, "invalid merge operator: %s\n", - FLAGS_merge_operator.c_str()); - exit(1); - } - options.max_successive_merges = FLAGS_max_successive_merges; - - // set universal style compaction configurations, if applicable - if (FLAGS_universal_size_ratio != 0) { - options.compaction_options_universal.size_ratio = - FLAGS_universal_size_ratio; - } - if (FLAGS_universal_min_merge_width != 0) { - options.compaction_options_universal.min_merge_width = - FLAGS_universal_min_merge_width; - } - if (FLAGS_universal_max_merge_width != 0) { - options.compaction_options_universal.max_merge_width = - FLAGS_universal_max_merge_width; - } - if (FLAGS_universal_max_size_amplification_percent != 0) { - options.compaction_options_universal.max_size_amplification_percent = - FLAGS_universal_max_size_amplification_percent; - } - if (FLAGS_universal_compression_size_percent != -1) { - options.compaction_options_universal.compression_size_percent = - FLAGS_universal_compression_size_percent; - } - - if (FLAGS_num_multi_db <= 1) { - OpenDb(options, FLAGS_db, &db_); - } else { - multi_dbs_.clear(); - multi_dbs_.resize(FLAGS_num_multi_db); - for (int i = 0; i < FLAGS_num_multi_db; i++) { - OpenDb(options, GetDbNameForMultiple(FLAGS_db, i), &multi_dbs_[i]); - } - } - if (FLAGS_min_level_to_compress >= 0) { - options.compression_per_level.clear(); - } - } - - void OpenDb(const Options& options, const std::string& db_name, - DBWithColumnFamilies* db) { - Status s; - // Open with column families if necessary. - if (FLAGS_num_column_families > 1) { - db->cfh.resize(FLAGS_num_column_families); - std::vector column_families; - for (int i = 0; i < FLAGS_num_column_families; i++) { - column_families.push_back(ColumnFamilyDescriptor( - ColumnFamilyName(i), ColumnFamilyOptions(options))); - } - if (FLAGS_readonly) { - s = DB::OpenForReadOnly(options, db_name, column_families, - &db->cfh, &db->db); - } else { - s = DB::Open(options, db_name, column_families, &db->cfh, &db->db); - } - } else if (FLAGS_readonly) { - s = DB::OpenForReadOnly(options, db_name, &db->db); - } else { - s = DB::Open(options, db_name, &db->db); - } - if (!s.ok()) { - fprintf(stderr, "open error: %s\n", s.ToString().c_str()); - exit(1); - } - } - - enum WriteMode { - RANDOM, SEQUENTIAL, UNIQUE_RANDOM - }; - - void WriteSeq(ThreadState* thread) { - DoWrite(thread, SEQUENTIAL); - } - - void WriteRandom(ThreadState* thread) { - DoWrite(thread, RANDOM); - } - - void WriteUniqueRandom(ThreadState* thread) { - DoWrite(thread, UNIQUE_RANDOM); - } - - class KeyGenerator { - public: - KeyGenerator(Random64* rand, WriteMode mode, - uint64_t num, uint64_t num_per_set = 64 * 1024) - : rand_(rand), - mode_(mode), - num_(num), - next_(0) { - if (mode_ == UNIQUE_RANDOM) { - // NOTE: if memory consumption of this approach becomes a concern, - // we can either break it into pieces and only random shuffle a section - // each time. Alternatively, use a bit map implementation - // (https://reviews.facebook.net/differential/diff/54627/) - values_.resize(num_); - for (uint64_t i = 0; i < num_; ++i) { - values_[i] = i; - } - std::shuffle(values_.begin(), values_.end(), - std::default_random_engine(FLAGS_seed)); - } - } - - uint64_t Next() { - switch (mode_) { - case SEQUENTIAL: - return next_++; - case RANDOM: - return rand_->Next() % num_; - case UNIQUE_RANDOM: - return values_[next_++]; - } - assert(false); - return std::numeric_limits::max(); - } - - private: - Random64* rand_; - WriteMode mode_; - const uint64_t num_; - uint64_t next_; - std::vector values_; - }; - - DB* SelectDB(ThreadState* thread) { - return SelectDBWithCfh(thread)->db; - } - - DBWithColumnFamilies* SelectDBWithCfh(ThreadState* thread) { - return SelectDBWithCfh(thread->rand.Next()); - } - - DBWithColumnFamilies* SelectDBWithCfh(uint64_t rand_int) { - if (db_.db != nullptr) { - return &db_; - } else { - return &multi_dbs_[rand_int % multi_dbs_.size()]; - } - } - - void DoWrite(ThreadState* thread, WriteMode write_mode) { - const int test_duration = write_mode == RANDOM ? FLAGS_duration : 0; - const int64_t num_ops = writes_ == 0 ? num_ : writes_; - - size_t num_key_gens = 1; - if (db_.db == nullptr) { - num_key_gens = multi_dbs_.size(); - } - std::vector> key_gens(num_key_gens); - Duration duration(test_duration, num_ops * num_key_gens); - for (size_t i = 0; i < num_key_gens; i++) { - key_gens[i].reset(new KeyGenerator(&(thread->rand), write_mode, num_ops)); - } - - if (num_ != FLAGS_num) { - char msg[100]; - snprintf(msg, sizeof(msg), "(%" PRIu64 " ops)", num_); - thread->stats.AddMessage(msg); - } - - RandomGenerator gen; - WriteBatch batch; - Status s; - int64_t bytes = 0; - - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - while (!duration.Done(entries_per_batch_)) { - size_t id = thread->rand.Next() % num_key_gens; - DBWithColumnFamilies* db_with_cfh = SelectDBWithCfh(id); - batch.Clear(); - for (int64_t j = 0; j < entries_per_batch_; j++) { - int64_t rand_num = key_gens[id]->Next(); - GenerateKeyFromInt(rand_num, FLAGS_num, &key); - if (FLAGS_num_column_families <= 1) { - batch.Put(key, gen.Generate(value_size_)); - } else { - // We use same rand_num as seed for key and column family so that we - // can deterministically find the cfh corresponding to a particular - // key while reading the key. - batch.Put(db_with_cfh->cfh[rand_num % db_with_cfh->cfh.size()], - key, gen.Generate(value_size_)); - } - bytes += value_size_ + key_size_; - } - s = db_with_cfh->db->Write(write_options_, &batch); - thread->stats.FinishedOps(db_with_cfh->db, entries_per_batch_); - if (!s.ok()) { - fprintf(stderr, "put error: %s\n", s.ToString().c_str()); - exit(1); - } - } - thread->stats.AddBytes(bytes); - } - - void ReadSequential(ThreadState* thread) { - if (db_.db != nullptr) { - ReadSequential(thread, db_.db); - } else { - for (const auto& db_with_cfh : multi_dbs_) { - ReadSequential(thread, db_with_cfh.db); - } - } - } - - void ReadSequential(ThreadState* thread, DB* db) { - Iterator* iter = db->NewIterator(ReadOptions(FLAGS_verify_checksum, true)); - int64_t i = 0; - int64_t bytes = 0; - for (iter->SeekToFirst(); i < reads_ && iter->Valid(); iter->Next()) { - bytes += iter->key().size() + iter->value().size(); - thread->stats.FinishedOps(db, 1); - ++i; - } - delete iter; - thread->stats.AddBytes(bytes); - } - - void ReadReverse(ThreadState* thread) { - if (db_.db != nullptr) { - ReadReverse(thread, db_.db); - } else { - for (const auto& db_with_cfh : multi_dbs_) { - ReadReverse(thread, db_with_cfh.db); - } - } - } - - void ReadReverse(ThreadState* thread, DB* db) { - Iterator* iter = db->NewIterator(ReadOptions(FLAGS_verify_checksum, true)); - int64_t i = 0; - int64_t bytes = 0; - for (iter->SeekToLast(); i < reads_ && iter->Valid(); iter->Prev()) { - bytes += iter->key().size() + iter->value().size(); - thread->stats.FinishedOps(db, 1); - ++i; - } - delete iter; - thread->stats.AddBytes(bytes); - } - - void ReadRandom(ThreadState* thread) { - int64_t read = 0; - int64_t found = 0; - ReadOptions options(FLAGS_verify_checksum, true); - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - std::string value; - - Duration duration(FLAGS_duration, reads_); - while (!duration.Done(1)) { - DBWithColumnFamilies* db_with_cfh = SelectDBWithCfh(thread); - // We use same key_rand as seed for key and column family so that we can - // deterministically find the cfh corresponding to a particular key, as it - // is done in DoWrite method. - int64_t key_rand = thread->rand.Next() % FLAGS_num; - GenerateKeyFromInt(key_rand, FLAGS_num, &key); - read++; - Status s; - if (FLAGS_num_column_families > 1) { - s = db_with_cfh->db->Get(options, - db_with_cfh->cfh[key_rand % db_with_cfh->cfh.size()], key, &value); - } else { - s = db_with_cfh->db->Get(options, key, &value); - } - if (s.ok()) { - found++; - } - thread->stats.FinishedOps(db_with_cfh->db, 1); - } - - char msg[100]; - snprintf(msg, sizeof(msg), "(%" PRIu64 " of %" PRIu64 " found)\n", - found, read); - - thread->stats.AddMessage(msg); - - if (FLAGS_perf_level > 0) { - thread->stats.AddMessage(perf_context.ToString()); - } - } - - // Calls MultiGet over a list of keys from a random distribution. - // Returns the total number of keys found. - void MultiReadRandom(ThreadState* thread) { - int64_t read = 0; - int64_t found = 0; - ReadOptions options(FLAGS_verify_checksum, true); - std::vector keys; - std::vector values(entries_per_batch_); - while (static_cast(keys.size()) < entries_per_batch_) { - keys.push_back(AllocateKey()); - } - - Duration duration(FLAGS_duration, reads_); - while (!duration.Done(1)) { - DB* db = SelectDB(thread); - for (int64_t i = 0; i < entries_per_batch_; ++i) { - GenerateKeyFromInt(thread->rand.Next() % FLAGS_num, - FLAGS_num, &keys[i]); - } - std::vector statuses = db->MultiGet(options, keys, &values); - assert(static_cast(statuses.size()) == entries_per_batch_); - - read += entries_per_batch_; - for (int64_t i = 0; i < entries_per_batch_; ++i) { - if (statuses[i].ok()) { - ++found; - } - } - thread->stats.FinishedOps(db, entries_per_batch_); - } - for (auto& k : keys) { - delete k.data(); - } - - char msg[100]; - snprintf(msg, sizeof(msg), "(%" PRIu64 " of %" PRIu64 " found)", - found, read); - thread->stats.AddMessage(msg); - } - - void IteratorCreation(ThreadState* thread) { - Duration duration(FLAGS_duration, reads_); - ReadOptions options(FLAGS_verify_checksum, true); - while (!duration.Done(1)) { - DB* db = SelectDB(thread); - Iterator* iter = db->NewIterator(options); - delete iter; - thread->stats.FinishedOps(db, 1); - } - } - - void IteratorCreationWhileWriting(ThreadState* thread) { - if (thread->tid > 0) { - IteratorCreation(thread); - } else { - BGWriter(thread); - } - } - - void SeekRandom(ThreadState* thread) { - int64_t read = 0; - int64_t found = 0; - ReadOptions options(FLAGS_verify_checksum, true); - options.tailing = FLAGS_use_tailing_iterator; - - Iterator* single_iter = nullptr; - std::vector multi_iters; - if (db_.db != nullptr) { - single_iter = db_.db->NewIterator(options); - } else { - for (const auto& db_with_cfh : multi_dbs_) { - multi_iters.push_back(db_with_cfh.db->NewIterator(options)); - } - } - uint64_t last_refresh = FLAGS_env->NowMicros(); - - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - - Duration duration(FLAGS_duration, reads_); - while (!duration.Done(1)) { - if (!FLAGS_use_tailing_iterator && FLAGS_iter_refresh_interval_us >= 0) { - uint64_t now = FLAGS_env->NowMicros(); - if (now - last_refresh > (uint64_t)FLAGS_iter_refresh_interval_us) { - if (db_.db != nullptr) { - delete single_iter; - single_iter = db_.db->NewIterator(options); - } else { - for (auto iter : multi_iters) { - delete iter; - } - multi_iters.clear(); - for (const auto& db_with_cfh : multi_dbs_) { - multi_iters.push_back(db_with_cfh.db->NewIterator(options)); - } - } - } - last_refresh = now; - } - // Pick a Iterator to use - Iterator* iter_to_use = single_iter; - if (single_iter == nullptr) { - iter_to_use = multi_iters[thread->rand.Next() % multi_iters.size()]; - } - - GenerateKeyFromInt(thread->rand.Next() % FLAGS_num, FLAGS_num, &key); - iter_to_use->Seek(key); - read++; - if (iter_to_use->Valid() && iter_to_use->key().compare(key) == 0) { - found++; - } - thread->stats.FinishedOps(db_.db, 1); - } - delete single_iter; - for (auto iter : multi_iters) { - delete iter; - } - - char msg[100]; - snprintf(msg, sizeof(msg), "(%" PRIu64 " of %" PRIu64 " found)\n", - found, read); - thread->stats.AddMessage(msg); - if (FLAGS_perf_level > 0) { - thread->stats.AddMessage(perf_context.ToString()); - } - } - - void SeekRandomWhileWriting(ThreadState* thread) { - if (thread->tid > 0) { - SeekRandom(thread); - } else { - BGWriter(thread); - } - } - - void DoDelete(ThreadState* thread, bool seq) { - WriteBatch batch; - Duration duration(seq ? 0 : FLAGS_duration, num_); - int64_t i = 0; - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - - while (!duration.Done(entries_per_batch_)) { - DB* db = SelectDB(thread); - batch.Clear(); - for (int64_t j = 0; j < entries_per_batch_; ++j) { - const int64_t k = seq ? i + j : (thread->rand.Next() % FLAGS_num); - GenerateKeyFromInt(k, FLAGS_num, &key); - batch.Delete(key); - } - auto s = db->Write(write_options_, &batch); - thread->stats.FinishedOps(db, entries_per_batch_); - if (!s.ok()) { - fprintf(stderr, "del error: %s\n", s.ToString().c_str()); - exit(1); - } - i += entries_per_batch_; - } - } - - void DeleteSeq(ThreadState* thread) { - DoDelete(thread, true); - } - - void DeleteRandom(ThreadState* thread) { - DoDelete(thread, false); - } - - void ReadWhileWriting(ThreadState* thread) { - if (thread->tid > 0) { - ReadRandom(thread); - } else { - BGWriter(thread); - } - } - - void BGWriter(ThreadState* thread) { - // Special thread that keeps writing until other threads are done. - RandomGenerator gen; - double last = FLAGS_env->NowMicros(); - int writes_per_second_by_10 = 0; - int num_writes = 0; - - // --writes_per_second rate limit is enforced per 100 milliseconds - // intervals to avoid a burst of writes at the start of each second. - - if (FLAGS_writes_per_second > 0) - writes_per_second_by_10 = FLAGS_writes_per_second / 10; - - // Don't merge stats from this thread with the readers. - thread->stats.SetExcludeFromMerge(); - - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - - while (true) { - DB* db = SelectDB(thread); - { - MutexLock l(&thread->shared->mu); - if (thread->shared->num_done + 1 >= thread->shared->num_initialized) { - // Other threads have finished - break; - } - } - - GenerateKeyFromInt(thread->rand.Next() % FLAGS_num, FLAGS_num, &key); - Status s = db->Put(write_options_, key, gen.Generate(value_size_)); - if (!s.ok()) { - fprintf(stderr, "put error: %s\n", s.ToString().c_str()); - exit(1); - } - thread->stats.FinishedOps(db_.db, 1); - - ++num_writes; - if (writes_per_second_by_10 && num_writes >= writes_per_second_by_10) { - double now = FLAGS_env->NowMicros(); - double usecs_since_last = now - last; - - num_writes = 0; - last = now; - - if (usecs_since_last < 100000.0) { - FLAGS_env->SleepForMicroseconds(100000.0 - usecs_since_last); - last = FLAGS_env->NowMicros(); - } - } - } - } - - // Given a key K and value V, this puts (K+"0", V), (K+"1", V), (K+"2", V) - // in DB atomically i.e in a single batch. Also refer GetMany. - Status PutMany(DB* db, const WriteOptions& writeoptions, const Slice& key, - const Slice& value) { - std::string suffixes[3] = {"2", "1", "0"}; - std::string keys[3]; - - WriteBatch batch; - Status s; - for (int i = 0; i < 3; i++) { - keys[i] = key.ToString() + suffixes[i]; - batch.Put(keys[i], value); - } - - s = db->Write(writeoptions, &batch); - return s; - } - - - // Given a key K, this deletes (K+"0", V), (K+"1", V), (K+"2", V) - // in DB atomically i.e in a single batch. Also refer GetMany. - Status DeleteMany(DB* db, const WriteOptions& writeoptions, - const Slice& key) { - std::string suffixes[3] = {"1", "2", "0"}; - std::string keys[3]; - - WriteBatch batch; - Status s; - for (int i = 0; i < 3; i++) { - keys[i] = key.ToString() + suffixes[i]; - batch.Delete(keys[i]); - } - - s = db->Write(writeoptions, &batch); - return s; - } - - // Given a key K and value V, this gets values for K+"0", K+"1" and K+"2" - // in the same snapshot, and verifies that all the values are identical. - // ASSUMES that PutMany was used to put (K, V) into the DB. - Status GetMany(DB* db, const ReadOptions& readoptions, const Slice& key, - std::string* value) { - std::string suffixes[3] = {"0", "1", "2"}; - std::string keys[3]; - Slice key_slices[3]; - std::string values[3]; - ReadOptions readoptionscopy = readoptions; - readoptionscopy.snapshot = db->GetSnapshot(); - Status s; - for (int i = 0; i < 3; i++) { - keys[i] = key.ToString() + suffixes[i]; - key_slices[i] = keys[i]; - s = db->Get(readoptionscopy, key_slices[i], value); - if (!s.ok() && !s.IsNotFound()) { - fprintf(stderr, "get error: %s\n", s.ToString().c_str()); - values[i] = ""; - // we continue after error rather than exiting so that we can - // find more errors if any - } else if (s.IsNotFound()) { - values[i] = ""; - } else { - values[i] = *value; - } - } - db->ReleaseSnapshot(readoptionscopy.snapshot); - - if ((values[0] != values[1]) || (values[1] != values[2])) { - fprintf(stderr, "inconsistent values for key %s: %s, %s, %s\n", - key.ToString().c_str(), values[0].c_str(), values[1].c_str(), - values[2].c_str()); - // we continue after error rather than exiting so that we can - // find more errors if any - } - - return s; - } - - // Differs from readrandomwriterandom in the following ways: - // (a) Uses GetMany/PutMany to read/write key values. Refer to those funcs. - // (b) Does deletes as well (per FLAGS_deletepercent) - // (c) In order to achieve high % of 'found' during lookups, and to do - // multiple writes (including puts and deletes) it uses upto - // FLAGS_numdistinct distinct keys instead of FLAGS_num distinct keys. - // (d) Does not have a MultiGet option. - void RandomWithVerify(ThreadState* thread) { - ReadOptions options(FLAGS_verify_checksum, true); - RandomGenerator gen; - std::string value; - int64_t found = 0; - int get_weight = 0; - int put_weight = 0; - int delete_weight = 0; - int64_t gets_done = 0; - int64_t puts_done = 0; - int64_t deletes_done = 0; - - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - - // the number of iterations is the larger of read_ or write_ - for (int64_t i = 0; i < readwrites_; i++) { - DB* db = SelectDB(thread); - if (get_weight == 0 && put_weight == 0 && delete_weight == 0) { - // one batch completed, reinitialize for next batch - get_weight = FLAGS_readwritepercent; - delete_weight = FLAGS_deletepercent; - put_weight = 100 - get_weight - delete_weight; - } - GenerateKeyFromInt(thread->rand.Next() % FLAGS_numdistinct, - FLAGS_numdistinct, &key); - if (get_weight > 0) { - // do all the gets first - Status s = GetMany(db, options, key, &value); - if (!s.ok() && !s.IsNotFound()) { - fprintf(stderr, "getmany error: %s\n", s.ToString().c_str()); - // we continue after error rather than exiting so that we can - // find more errors if any - } else if (!s.IsNotFound()) { - found++; - } - get_weight--; - gets_done++; - } else if (put_weight > 0) { - // then do all the corresponding number of puts - // for all the gets we have done earlier - Status s = PutMany(db, write_options_, key, gen.Generate(value_size_)); - if (!s.ok()) { - fprintf(stderr, "putmany error: %s\n", s.ToString().c_str()); - exit(1); - } - put_weight--; - puts_done++; - } else if (delete_weight > 0) { - Status s = DeleteMany(db, write_options_, key); - if (!s.ok()) { - fprintf(stderr, "deletemany error: %s\n", s.ToString().c_str()); - exit(1); - } - delete_weight--; - deletes_done++; - } - - thread->stats.FinishedOps(db_.db, 1); - } - char msg[100]; - snprintf(msg, sizeof(msg), - "( get:%" PRIu64 " put:%" PRIu64 " del:%" PRIu64 " total:%" \ - PRIu64 " found:%" PRIu64 ")", - gets_done, puts_done, deletes_done, readwrites_, found); - thread->stats.AddMessage(msg); - } - - // This is different from ReadWhileWriting because it does not use - // an extra thread. - void ReadRandomWriteRandom(ThreadState* thread) { - ReadOptions options(FLAGS_verify_checksum, true); - RandomGenerator gen; - std::string value; - int64_t found = 0; - int get_weight = 0; - int put_weight = 0; - int64_t reads_done = 0; - int64_t writes_done = 0; - Duration duration(FLAGS_duration, readwrites_); - - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - - // the number of iterations is the larger of read_ or write_ - while (!duration.Done(1)) { - DB* db = SelectDB(thread); - GenerateKeyFromInt(thread->rand.Next() % FLAGS_num, FLAGS_num, &key); - if (get_weight == 0 && put_weight == 0) { - // one batch completed, reinitialize for next batch - get_weight = FLAGS_readwritepercent; - put_weight = 100 - get_weight; - } - if (get_weight > 0) { - // do all the gets first - Status s = db->Get(options, key, &value); - if (!s.ok() && !s.IsNotFound()) { - fprintf(stderr, "get error: %s\n", s.ToString().c_str()); - // we continue after error rather than exiting so that we can - // find more errors if any - } else if (!s.IsNotFound()) { - found++; - } - get_weight--; - reads_done++; - } else if (put_weight > 0) { - // then do all the corresponding number of puts - // for all the gets we have done earlier - Status s = db->Put(write_options_, key, gen.Generate(value_size_)); - if (!s.ok()) { - fprintf(stderr, "put error: %s\n", s.ToString().c_str()); - exit(1); - } - put_weight--; - writes_done++; - } - thread->stats.FinishedOps(db, 1); - } - char msg[100]; - snprintf(msg, sizeof(msg), "( reads:%" PRIu64 " writes:%" PRIu64 \ - " total:%" PRIu64 " found:%" PRIu64 ")", - reads_done, writes_done, readwrites_, found); - thread->stats.AddMessage(msg); - } - - // - // Read-modify-write for random keys - void UpdateRandom(ThreadState* thread) { - ReadOptions options(FLAGS_verify_checksum, true); - RandomGenerator gen; - std::string value; - int64_t found = 0; - Duration duration(FLAGS_duration, readwrites_); - - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - // the number of iterations is the larger of read_ or write_ - while (!duration.Done(1)) { - DB* db = SelectDB(thread); - GenerateKeyFromInt(thread->rand.Next() % FLAGS_num, FLAGS_num, &key); - - if (db->Get(options, key, &value).ok()) { - found++; - } - - Status s = db->Put(write_options_, key, gen.Generate(value_size_)); - if (!s.ok()) { - fprintf(stderr, "put error: %s\n", s.ToString().c_str()); - exit(1); - } - thread->stats.FinishedOps(db, 1); - } - char msg[100]; - snprintf(msg, sizeof(msg), - "( updates:%" PRIu64 " found:%" PRIu64 ")", readwrites_, found); - thread->stats.AddMessage(msg); - } - - // Read-modify-write for random keys. - // Each operation causes the key grow by value_size (simulating an append). - // Generally used for benchmarking against merges of similar type - void AppendRandom(ThreadState* thread) { - ReadOptions options(FLAGS_verify_checksum, true); - RandomGenerator gen; - std::string value; - int64_t found = 0; - - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - // The number of iterations is the larger of read_ or write_ - Duration duration(FLAGS_duration, readwrites_); - while (!duration.Done(1)) { - DB* db = SelectDB(thread); - GenerateKeyFromInt(thread->rand.Next() % FLAGS_num, FLAGS_num, &key); - - // Get the existing value - if (db->Get(options, key, &value).ok()) { - found++; - } else { - // If not existing, then just assume an empty string of data - value.clear(); - } - - // Update the value (by appending data) - Slice operand = gen.Generate(value_size_); - if (value.size() > 0) { - // Use a delimeter to match the semantics for StringAppendOperator - value.append(1,','); - } - value.append(operand.data(), operand.size()); - - // Write back to the database - Status s = db->Put(write_options_, key, value); - if (!s.ok()) { - fprintf(stderr, "put error: %s\n", s.ToString().c_str()); - exit(1); - } - thread->stats.FinishedOps(db, 1); - } - - char msg[100]; - snprintf(msg, sizeof(msg), "( updates:%" PRIu64 " found:%" PRIu64 ")", - readwrites_, found); - thread->stats.AddMessage(msg); - } - - // Read-modify-write for random keys (using MergeOperator) - // The merge operator to use should be defined by FLAGS_merge_operator - // Adjust FLAGS_value_size so that the keys are reasonable for this operator - // Assumes that the merge operator is non-null (i.e.: is well-defined) - // - // For example, use FLAGS_merge_operator="uint64add" and FLAGS_value_size=8 - // to simulate random additions over 64-bit integers using merge. - // - // The number of merges on the same key can be controlled by adjusting - // FLAGS_merge_keys. - void MergeRandom(ThreadState* thread) { - RandomGenerator gen; - - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - // The number of iterations is the larger of read_ or write_ - Duration duration(FLAGS_duration, readwrites_); - while (!duration.Done(1)) { - DB* db = SelectDB(thread); - GenerateKeyFromInt(thread->rand.Next() % merge_keys_, merge_keys_, &key); - - Status s = db->Merge(write_options_, key, gen.Generate(value_size_)); - - if (!s.ok()) { - fprintf(stderr, "merge error: %s\n", s.ToString().c_str()); - exit(1); - } - thread->stats.FinishedOps(db, 1); - } - - // Print some statistics - char msg[100]; - snprintf(msg, sizeof(msg), "( updates:%" PRIu64 ")", readwrites_); - thread->stats.AddMessage(msg); - } - - // Read and merge random keys. The amount of reads and merges are controlled - // by adjusting FLAGS_num and FLAGS_mergereadpercent. The number of distinct - // keys (and thus also the number of reads and merges on the same key) can be - // adjusted with FLAGS_merge_keys. - // - // As with MergeRandom, the merge operator to use should be defined by - // FLAGS_merge_operator. - void ReadRandomMergeRandom(ThreadState* thread) { - ReadOptions options(FLAGS_verify_checksum, true); - RandomGenerator gen; - std::string value; - int64_t num_hits = 0; - int64_t num_gets = 0; - int64_t num_merges = 0; - size_t max_length = 0; - - Slice key = AllocateKey(); - std::unique_ptr key_guard(key.data()); - // the number of iterations is the larger of read_ or write_ - Duration duration(FLAGS_duration, readwrites_); - while (!duration.Done(1)) { - DB* db = SelectDB(thread); - GenerateKeyFromInt(thread->rand.Next() % merge_keys_, merge_keys_, &key); - - bool do_merge = int(thread->rand.Next() % 100) < FLAGS_mergereadpercent; - - if (do_merge) { - Status s = db->Merge(write_options_, key, gen.Generate(value_size_)); - if (!s.ok()) { - fprintf(stderr, "merge error: %s\n", s.ToString().c_str()); - exit(1); - } - - num_merges++; - - } else { - Status s = db->Get(options, key, &value); - if (value.length() > max_length) - max_length = value.length(); - - if (!s.ok() && !s.IsNotFound()) { - fprintf(stderr, "get error: %s\n", s.ToString().c_str()); - // we continue after error rather than exiting so that we can - // find more errors if any - } else if (!s.IsNotFound()) { - num_hits++; - } - - num_gets++; - - } - - thread->stats.FinishedOps(db, 1); - } - - char msg[100]; - snprintf(msg, sizeof(msg), - "(reads:%" PRIu64 " merges:%" PRIu64 " total:%" PRIu64 " hits:%" \ - PRIu64 " maxlength:%zu)", - num_gets, num_merges, readwrites_, num_hits, max_length); - thread->stats.AddMessage(msg); - } - - void Compact(ThreadState* thread) { - DB* db = SelectDB(thread); - db->CompactRange(nullptr, nullptr); - } - - void PrintStats(const char* key) { - if (db_.db != nullptr) { - PrintStats(db_.db, key, false); - } - for (const auto& db_with_cfh : multi_dbs_) { - PrintStats(db_with_cfh.db, key, true); - } - } - - void PrintStats(DB* db, const char* key, bool print_header = false) { - if (print_header) { - fprintf(stdout, "\n==== DB: %s ===\n", db->GetName().c_str()); - } - std::string stats; - if (!db->GetProperty(key, &stats)) { - stats = "(failed)"; - } - fprintf(stdout, "\n%s\n", stats.c_str()); - } -}; - -} // namespace rocksdb - -int main(int argc, char** argv) { - rocksdb::port::InstallStackTraceHandler(); - SetUsageMessage(std::string("\nUSAGE:\n") + std::string(argv[0]) + - " [OPTIONS]..."); - ParseCommandLineFlags(&argc, &argv, true); - - FLAGS_compaction_style_e = (rocksdb::CompactionStyle) FLAGS_compaction_style; - if (FLAGS_statistics) { - dbstats = rocksdb::CreateDBStatistics(); - } - - std::vector fanout = - rocksdb::stringSplit(FLAGS_max_bytes_for_level_multiplier_additional, ','); - for (unsigned int j= 0; j < fanout.size(); j++) { - FLAGS_max_bytes_for_level_multiplier_additional_v.push_back( - std::stoi(fanout[j])); - } - - FLAGS_compression_type_e = - StringToCompressionType(FLAGS_compression_type.c_str()); - - if (!FLAGS_hdfs.empty()) { - FLAGS_env = new rocksdb::HdfsEnv(FLAGS_hdfs); - } - - if (!strcasecmp(FLAGS_compaction_fadvice.c_str(), "NONE")) - FLAGS_compaction_fadvice_e = rocksdb::Options::NONE; - else if (!strcasecmp(FLAGS_compaction_fadvice.c_str(), "NORMAL")) - FLAGS_compaction_fadvice_e = rocksdb::Options::NORMAL; - else if (!strcasecmp(FLAGS_compaction_fadvice.c_str(), "SEQUENTIAL")) - FLAGS_compaction_fadvice_e = rocksdb::Options::SEQUENTIAL; - else if (!strcasecmp(FLAGS_compaction_fadvice.c_str(), "WILLNEED")) - FLAGS_compaction_fadvice_e = rocksdb::Options::WILLNEED; - else { - fprintf(stdout, "Unknown compaction fadvice:%s\n", - FLAGS_compaction_fadvice.c_str()); - } - - FLAGS_rep_factory = StringToRepFactory(FLAGS_memtablerep.c_str()); - - // The number of background threads should be at least as much the - // max number of concurrent compactions. - FLAGS_env->SetBackgroundThreads(FLAGS_max_background_compactions); - // Choose a location for the test database if none given with --db= - if (FLAGS_db.empty()) { - std::string default_db_path; - rocksdb::Env::Default()->GetTestDirectory(&default_db_path); - default_db_path += "/dbbench"; - FLAGS_db = default_db_path; - } - - rocksdb::Benchmark benchmark; - benchmark.Run(); - return 0; -} - -#endif // GFLAGS diff --git a/db/db_blob_index_test.cc b/db/db_blob_index_test.cc new file mode 100644 index 00000000000..e71b511df5b --- /dev/null +++ b/db/db_blob_index_test.cc @@ -0,0 +1,409 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include +#include +#include + +#include "db/column_family.h" +#include "db/db_iter.h" +#include "db/db_test_util.h" +#include "db/dbformat.h" +#include "db/write_batch_internal.h" +#include "port/port.h" +#include "port/stack_trace.h" +#include "util/string_util.h" +#include "utilities/merge_operators.h" + +namespace rocksdb { + +// kTypeBlobIndex is a value type used by BlobDB only. The base rocksdb +// should accept the value type on write, and report not supported value +// for reads, unless caller request for it explicitly. The base rocksdb +// doesn't understand format of actual blob index (the value). +class DBBlobIndexTest : public DBTestBase { + public: + enum Tier { + kMemtable = 0, + kImmutableMemtables = 1, + kL0SstFile = 2, + kLnSstFile = 3, + }; + const std::vector kAllTiers = {Tier::kMemtable, + Tier::kImmutableMemtables, + Tier::kL0SstFile, Tier::kLnSstFile}; + + DBBlobIndexTest() : DBTestBase("/db_blob_index_test") {} + + ColumnFamilyHandle* cfh() { return dbfull()->DefaultColumnFamily(); } + + ColumnFamilyData* cfd() { + return reinterpret_cast(cfh())->cfd(); + } + + Status PutBlobIndex(WriteBatch* batch, const Slice& key, + const Slice& blob_index) { + return WriteBatchInternal::PutBlobIndex(batch, cfd()->GetID(), key, + blob_index); + } + + Status Write(WriteBatch* batch) { + return dbfull()->Write(WriteOptions(), batch); + } + + std::string GetImpl(const Slice& key, bool* is_blob_index = nullptr, + const Snapshot* snapshot = nullptr) { + ReadOptions read_options; + read_options.snapshot = snapshot; + PinnableSlice value; + auto s = dbfull()->GetImpl(read_options, cfh(), key, &value, + nullptr /*value_found*/, is_blob_index); + if (s.IsNotFound()) { + return "NOT_FOUND"; + } + if (s.IsNotSupported()) { + return "NOT_SUPPORTED"; + } + if (!s.ok()) { + return s.ToString(); + } + return value.ToString(); + } + + std::string GetBlobIndex(const Slice& key, + const Snapshot* snapshot = nullptr) { + bool is_blob_index = false; + std::string value = GetImpl(key, &is_blob_index, snapshot); + if (!is_blob_index) { + return "NOT_BLOB"; + } + return value; + } + + ArenaWrappedDBIter* GetBlobIterator() { + return dbfull()->NewIteratorImpl(ReadOptions(), cfd(), + dbfull()->GetLatestSequenceNumber(), + true /*allow_blob*/); + } + + Options GetTestOptions() { + Options options; + options.create_if_missing = true; + options.num_levels = 2; + options.disable_auto_compactions = true; + // Disable auto flushes. + options.max_write_buffer_number = 10; + options.min_write_buffer_number_to_merge = 10; + options.merge_operator = MergeOperators::CreateStringAppendOperator(); + return options; + } + + void MoveDataTo(Tier tier) { + switch (tier) { + case Tier::kMemtable: + break; + case Tier::kImmutableMemtables: + ASSERT_OK(dbfull()->TEST_SwitchMemtable()); + break; + case Tier::kL0SstFile: + ASSERT_OK(Flush()); + break; + case Tier::kLnSstFile: + ASSERT_OK(Flush()); + ASSERT_OK(Put("a", "dummy")); + ASSERT_OK(Put("z", "dummy")); + ASSERT_OK(Flush()); + ASSERT_OK( + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr)); +#ifndef ROCKSDB_LITE + ASSERT_EQ("0,1", FilesPerLevel()); +#endif // !ROCKSDB_LITE + break; + } + } +}; + +// Should be able to write kTypeBlobIndex to memtables and SST files. +TEST_F(DBBlobIndexTest, Write) { + for (auto tier : kAllTiers) { + DestroyAndReopen(GetTestOptions()); + for (int i = 1; i <= 5; i++) { + std::string index = ToString(i); + WriteBatch batch; + ASSERT_OK(PutBlobIndex(&batch, "key" + index, "blob" + index)); + ASSERT_OK(Write(&batch)); + } + MoveDataTo(tier); + for (int i = 1; i <= 5; i++) { + std::string index = ToString(i); + ASSERT_EQ("blob" + index, GetBlobIndex("key" + index)); + } + } +} + +// Get should be able to return blob index if is_blob_index is provided, +// otherwise return Status::NotSupported status. +TEST_F(DBBlobIndexTest, Get) { + for (auto tier : kAllTiers) { + DestroyAndReopen(GetTestOptions()); + WriteBatch batch; + ASSERT_OK(batch.Put("key", "value")); + ASSERT_OK(PutBlobIndex(&batch, "blob_key", "blob_index")); + ASSERT_OK(Write(&batch)); + MoveDataTo(tier); + // Verify normal value + bool is_blob_index = false; + PinnableSlice value; + ASSERT_EQ("value", Get("key")); + ASSERT_EQ("value", GetImpl("key")); + ASSERT_EQ("value", GetImpl("key", &is_blob_index)); + ASSERT_FALSE(is_blob_index); + // Verify blob index + ASSERT_TRUE(Get("blob_key", &value).IsNotSupported()); + ASSERT_EQ("NOT_SUPPORTED", GetImpl("blob_key")); + ASSERT_EQ("blob_index", GetImpl("blob_key", &is_blob_index)); + ASSERT_TRUE(is_blob_index); + } +} + +// Get should NOT return Status::NotSupported if blob index is updated with +// a normal value. +TEST_F(DBBlobIndexTest, Updated) { + for (auto tier : kAllTiers) { + DestroyAndReopen(GetTestOptions()); + WriteBatch batch; + for (int i = 0; i < 10; i++) { + ASSERT_OK(PutBlobIndex(&batch, "key" + ToString(i), "blob_index")); + } + ASSERT_OK(Write(&batch)); + // Avoid blob values from being purged. + const Snapshot* snapshot = dbfull()->GetSnapshot(); + ASSERT_OK(Put("key1", "new_value")); + ASSERT_OK(Merge("key2", "a")); + ASSERT_OK(Merge("key2", "b")); + ASSERT_OK(Merge("key2", "c")); + ASSERT_OK(Delete("key3")); + ASSERT_OK(SingleDelete("key4")); + ASSERT_OK(Delete("key5")); + ASSERT_OK(Merge("key5", "a")); + ASSERT_OK(Merge("key5", "b")); + ASSERT_OK(Merge("key5", "c")); + ASSERT_OK(dbfull()->DeleteRange(WriteOptions(), cfh(), "key6", "key9")); + MoveDataTo(tier); + for (int i = 0; i < 10; i++) { + ASSERT_EQ("blob_index", GetBlobIndex("key" + ToString(i), snapshot)); + } + ASSERT_EQ("new_value", Get("key1")); + ASSERT_EQ("NOT_SUPPORTED", GetImpl("key2")); + ASSERT_EQ("NOT_FOUND", Get("key3")); + ASSERT_EQ("NOT_FOUND", Get("key4")); + ASSERT_EQ("a,b,c", GetImpl("key5")); + for (int i = 6; i < 9; i++) { + ASSERT_EQ("NOT_FOUND", Get("key" + ToString(i))); + } + ASSERT_EQ("blob_index", GetBlobIndex("key9")); + dbfull()->ReleaseSnapshot(snapshot); + } +} + +// Iterator should get blob value if allow_blob flag is set, +// otherwise return Status::NotSupported status. +TEST_F(DBBlobIndexTest, Iterate) { + const std::vector> data = { + /*00*/ {kTypeValue}, + /*01*/ {kTypeBlobIndex}, + /*02*/ {kTypeValue}, + /*03*/ {kTypeBlobIndex, kTypeValue}, + /*04*/ {kTypeValue}, + /*05*/ {kTypeValue, kTypeBlobIndex}, + /*06*/ {kTypeValue}, + /*07*/ {kTypeDeletion, kTypeBlobIndex}, + /*08*/ {kTypeValue}, + /*09*/ {kTypeSingleDeletion, kTypeBlobIndex}, + /*10*/ {kTypeValue}, + /*11*/ {kTypeMerge, kTypeMerge, kTypeMerge, kTypeBlobIndex}, + /*12*/ {kTypeValue}, + /*13*/ + {kTypeMerge, kTypeMerge, kTypeMerge, kTypeDeletion, kTypeBlobIndex}, + /*14*/ {kTypeValue}, + /*15*/ {kTypeBlobIndex}, + /*16*/ {kTypeValue}, + }; + + auto get_key = [](int index) { + char buf[20]; + snprintf(buf, sizeof(buf), "%02d", index); + return "key" + std::string(buf); + }; + + auto get_value = [&](int index, int version) { + return get_key(index) + "_value" + ToString(version); + }; + + auto check_iterator = [&](Iterator* iterator, Status::Code expected_status, + const Slice& expected_value) { + ASSERT_EQ(expected_status, iterator->status().code()); + if (expected_status == Status::kOk) { + ASSERT_TRUE(iterator->Valid()); + ASSERT_EQ(expected_value, iterator->value()); + } else { + ASSERT_FALSE(iterator->Valid()); + } + }; + + auto create_normal_iterator = [&]() -> Iterator* { + return dbfull()->NewIterator(ReadOptions()); + }; + + auto create_blob_iterator = [&]() -> Iterator* { return GetBlobIterator(); }; + + auto check_is_blob = [&](bool is_blob) { + return [is_blob](Iterator* iterator) { + ASSERT_EQ(is_blob, + reinterpret_cast(iterator)->IsBlob()); + }; + }; + + auto verify = [&](int index, Status::Code expected_status, + const Slice& forward_value, const Slice& backward_value, + std::function create_iterator, + std::function extra_check = nullptr) { + // Seek + auto* iterator = create_iterator(); + ASSERT_OK(iterator->Refresh()); + iterator->Seek(get_key(index)); + check_iterator(iterator, expected_status, forward_value); + if (extra_check) { + extra_check(iterator); + } + delete iterator; + + // Next + iterator = create_iterator(); + ASSERT_OK(iterator->Refresh()); + iterator->Seek(get_key(index - 1)); + ASSERT_TRUE(iterator->Valid()); + iterator->Next(); + check_iterator(iterator, expected_status, forward_value); + if (extra_check) { + extra_check(iterator); + } + delete iterator; + + // SeekForPrev + iterator = create_iterator(); + ASSERT_OK(iterator->Refresh()); + iterator->SeekForPrev(get_key(index)); + check_iterator(iterator, expected_status, backward_value); + if (extra_check) { + extra_check(iterator); + } + delete iterator; + + // Prev + iterator = create_iterator(); + iterator->Seek(get_key(index + 1)); + ASSERT_TRUE(iterator->Valid()); + iterator->Prev(); + check_iterator(iterator, expected_status, backward_value); + if (extra_check) { + extra_check(iterator); + } + delete iterator; + }; + + for (auto tier : {Tier::kMemtable} /*kAllTiers*/) { + // Avoid values from being purged. + std::vector snapshots; + DestroyAndReopen(GetTestOptions()); + + // fill data + for (int i = 0; i < static_cast(data.size()); i++) { + for (int j = static_cast(data[i].size()) - 1; j >= 0; j--) { + std::string key = get_key(i); + std::string value = get_value(i, j); + WriteBatch batch; + switch (data[i][j]) { + case kTypeValue: + ASSERT_OK(Put(key, value)); + break; + case kTypeDeletion: + ASSERT_OK(Delete(key)); + break; + case kTypeSingleDeletion: + ASSERT_OK(SingleDelete(key)); + break; + case kTypeMerge: + ASSERT_OK(Merge(key, value)); + break; + case kTypeBlobIndex: + ASSERT_OK(PutBlobIndex(&batch, key, value)); + ASSERT_OK(Write(&batch)); + break; + default: + assert(false); + }; + } + snapshots.push_back(dbfull()->GetSnapshot()); + } + ASSERT_OK( + dbfull()->DeleteRange(WriteOptions(), cfh(), get_key(15), get_key(16))); + snapshots.push_back(dbfull()->GetSnapshot()); + MoveDataTo(tier); + + // Normal iterator + verify(1, Status::kNotSupported, "", "", create_normal_iterator); + verify(3, Status::kNotSupported, "", "", create_normal_iterator); + verify(5, Status::kOk, get_value(5, 0), get_value(5, 0), + create_normal_iterator); + verify(7, Status::kOk, get_value(8, 0), get_value(6, 0), + create_normal_iterator); + verify(9, Status::kOk, get_value(10, 0), get_value(8, 0), + create_normal_iterator); + verify(11, Status::kNotSupported, "", "", create_normal_iterator); + verify(13, Status::kOk, + get_value(13, 2) + "," + get_value(13, 1) + "," + get_value(13, 0), + get_value(13, 2) + "," + get_value(13, 1) + "," + get_value(13, 0), + create_normal_iterator); + verify(15, Status::kOk, get_value(16, 0), get_value(14, 0), + create_normal_iterator); + + // Iterator with blob support + verify(1, Status::kOk, get_value(1, 0), get_value(1, 0), + create_blob_iterator, check_is_blob(true)); + verify(3, Status::kOk, get_value(3, 0), get_value(3, 0), + create_blob_iterator, check_is_blob(true)); + verify(5, Status::kOk, get_value(5, 0), get_value(5, 0), + create_blob_iterator, check_is_blob(false)); + verify(7, Status::kOk, get_value(8, 0), get_value(6, 0), + create_blob_iterator, check_is_blob(false)); + verify(9, Status::kOk, get_value(10, 0), get_value(8, 0), + create_blob_iterator, check_is_blob(false)); + verify(11, Status::kNotSupported, "", "", create_blob_iterator); + verify(13, Status::kOk, + get_value(13, 2) + "," + get_value(13, 1) + "," + get_value(13, 0), + get_value(13, 2) + "," + get_value(13, 1) + "," + get_value(13, 0), + create_blob_iterator, check_is_blob(false)); + verify(15, Status::kOk, get_value(16, 0), get_value(14, 0), + create_blob_iterator, check_is_blob(false)); + + for (auto* snapshot : snapshots) { + dbfull()->ReleaseSnapshot(snapshot); + } + } +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_block_cache_test.cc b/db/db_block_cache_test.cc new file mode 100644 index 00000000000..169cadc85c3 --- /dev/null +++ b/db/db_block_cache_test.cc @@ -0,0 +1,579 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#include +#include "cache/lru_cache.h" +#include "db/db_test_util.h" +#include "port/stack_trace.h" + +namespace rocksdb { + +class DBBlockCacheTest : public DBTestBase { + private: + size_t miss_count_ = 0; + size_t hit_count_ = 0; + size_t insert_count_ = 0; + size_t failure_count_ = 0; + size_t compressed_miss_count_ = 0; + size_t compressed_hit_count_ = 0; + size_t compressed_insert_count_ = 0; + size_t compressed_failure_count_ = 0; + + public: + const size_t kNumBlocks = 10; + const size_t kValueSize = 100; + + DBBlockCacheTest() : DBTestBase("/db_block_cache_test") {} + + BlockBasedTableOptions GetTableOptions() { + BlockBasedTableOptions table_options; + // Set a small enough block size so that each key-value get its own block. + table_options.block_size = 1; + return table_options; + } + + Options GetOptions(const BlockBasedTableOptions& table_options) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.avoid_flush_during_recovery = false; + // options.compression = kNoCompression; + options.statistics = rocksdb::CreateDBStatistics(); + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + return options; + } + + void InitTable(const Options& options) { + std::string value(kValueSize, 'a'); + for (size_t i = 0; i < kNumBlocks; i++) { + ASSERT_OK(Put(ToString(i), value.c_str())); + } + } + + void RecordCacheCounters(const Options& options) { + miss_count_ = TestGetTickerCount(options, BLOCK_CACHE_MISS); + hit_count_ = TestGetTickerCount(options, BLOCK_CACHE_HIT); + insert_count_ = TestGetTickerCount(options, BLOCK_CACHE_ADD); + failure_count_ = TestGetTickerCount(options, BLOCK_CACHE_ADD_FAILURES); + compressed_miss_count_ = + TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_MISS); + compressed_hit_count_ = + TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_HIT); + compressed_insert_count_ = + TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_ADD); + compressed_failure_count_ = + TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_ADD_FAILURES); + } + + void CheckCacheCounters(const Options& options, size_t expected_misses, + size_t expected_hits, size_t expected_inserts, + size_t expected_failures) { + size_t new_miss_count = TestGetTickerCount(options, BLOCK_CACHE_MISS); + size_t new_hit_count = TestGetTickerCount(options, BLOCK_CACHE_HIT); + size_t new_insert_count = TestGetTickerCount(options, BLOCK_CACHE_ADD); + size_t new_failure_count = + TestGetTickerCount(options, BLOCK_CACHE_ADD_FAILURES); + ASSERT_EQ(miss_count_ + expected_misses, new_miss_count); + ASSERT_EQ(hit_count_ + expected_hits, new_hit_count); + ASSERT_EQ(insert_count_ + expected_inserts, new_insert_count); + ASSERT_EQ(failure_count_ + expected_failures, new_failure_count); + miss_count_ = new_miss_count; + hit_count_ = new_hit_count; + insert_count_ = new_insert_count; + failure_count_ = new_failure_count; + } + + void CheckCompressedCacheCounters(const Options& options, + size_t expected_misses, + size_t expected_hits, + size_t expected_inserts, + size_t expected_failures) { + size_t new_miss_count = + TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_MISS); + size_t new_hit_count = + TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_HIT); + size_t new_insert_count = + TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_ADD); + size_t new_failure_count = + TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_ADD_FAILURES); + ASSERT_EQ(compressed_miss_count_ + expected_misses, new_miss_count); + ASSERT_EQ(compressed_hit_count_ + expected_hits, new_hit_count); + ASSERT_EQ(compressed_insert_count_ + expected_inserts, new_insert_count); + ASSERT_EQ(compressed_failure_count_ + expected_failures, new_failure_count); + compressed_miss_count_ = new_miss_count; + compressed_hit_count_ = new_hit_count; + compressed_insert_count_ = new_insert_count; + compressed_failure_count_ = new_failure_count; + } +}; + +TEST_F(DBBlockCacheTest, TestWithoutCompressedBlockCache) { + ReadOptions read_options; + auto table_options = GetTableOptions(); + auto options = GetOptions(table_options); + InitTable(options); + + std::shared_ptr cache = NewLRUCache(0, 0, false); + table_options.block_cache = cache; + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + Reopen(options); + RecordCacheCounters(options); + + std::vector> iterators(kNumBlocks - 1); + Iterator* iter = nullptr; + + // Load blocks into cache. + for (size_t i = 0; i < kNumBlocks - 1; i++) { + iter = db_->NewIterator(read_options); + iter->Seek(ToString(i)); + ASSERT_OK(iter->status()); + CheckCacheCounters(options, 1, 0, 1, 0); + iterators[i].reset(iter); + } + size_t usage = cache->GetUsage(); + ASSERT_LT(0, usage); + cache->SetCapacity(usage); + ASSERT_EQ(usage, cache->GetPinnedUsage()); + + // Test with strict capacity limit. + cache->SetStrictCapacityLimit(true); + iter = db_->NewIterator(read_options); + iter->Seek(ToString(kNumBlocks - 1)); + ASSERT_TRUE(iter->status().IsIncomplete()); + CheckCacheCounters(options, 1, 0, 0, 1); + delete iter; + iter = nullptr; + + // Release interators and access cache again. + for (size_t i = 0; i < kNumBlocks - 1; i++) { + iterators[i].reset(); + CheckCacheCounters(options, 0, 0, 0, 0); + } + ASSERT_EQ(0, cache->GetPinnedUsage()); + for (size_t i = 0; i < kNumBlocks - 1; i++) { + iter = db_->NewIterator(read_options); + iter->Seek(ToString(i)); + ASSERT_OK(iter->status()); + CheckCacheCounters(options, 0, 1, 0, 0); + iterators[i].reset(iter); + } +} + +#ifdef SNAPPY +TEST_F(DBBlockCacheTest, TestWithCompressedBlockCache) { + ReadOptions read_options; + auto table_options = GetTableOptions(); + auto options = GetOptions(table_options); + options.compression = CompressionType::kSnappyCompression; + InitTable(options); + + std::shared_ptr cache = NewLRUCache(0, 0, false); + std::shared_ptr compressed_cache = NewLRUCache(1 << 25, 0, false); + table_options.block_cache = cache; + table_options.block_cache_compressed = compressed_cache; + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + Reopen(options); + RecordCacheCounters(options); + + std::vector> iterators(kNumBlocks - 1); + Iterator* iter = nullptr; + + // Load blocks into cache. + for (size_t i = 0; i < kNumBlocks - 1; i++) { + iter = db_->NewIterator(read_options); + iter->Seek(ToString(i)); + ASSERT_OK(iter->status()); + CheckCacheCounters(options, 1, 0, 1, 0); + CheckCompressedCacheCounters(options, 1, 0, 1, 0); + iterators[i].reset(iter); + } + size_t usage = cache->GetUsage(); + ASSERT_LT(0, usage); + ASSERT_EQ(usage, cache->GetPinnedUsage()); + size_t compressed_usage = compressed_cache->GetUsage(); + ASSERT_LT(0, compressed_usage); + // Compressed block cache cannot be pinned. + ASSERT_EQ(0, compressed_cache->GetPinnedUsage()); + + // Set strict capacity limit flag. Now block will only load into compressed + // block cache. + cache->SetCapacity(usage); + cache->SetStrictCapacityLimit(true); + ASSERT_EQ(usage, cache->GetPinnedUsage()); + iter = db_->NewIterator(read_options); + iter->Seek(ToString(kNumBlocks - 1)); + ASSERT_TRUE(iter->status().IsIncomplete()); + CheckCacheCounters(options, 1, 0, 0, 1); + CheckCompressedCacheCounters(options, 1, 0, 1, 0); + delete iter; + iter = nullptr; + + // Clear strict capacity limit flag. This time we shall hit compressed block + // cache. + cache->SetStrictCapacityLimit(false); + iter = db_->NewIterator(read_options); + iter->Seek(ToString(kNumBlocks - 1)); + ASSERT_OK(iter->status()); + CheckCacheCounters(options, 1, 0, 1, 0); + CheckCompressedCacheCounters(options, 0, 1, 0, 0); + delete iter; + iter = nullptr; +} +#endif // SNAPPY + +#ifndef ROCKSDB_LITE + +// Make sure that when options.block_cache is set, after a new table is +// created its index/filter blocks are added to block cache. +TEST_F(DBBlockCacheTest, IndexAndFilterBlocksOfNewTableAddedToCache) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions table_options; + table_options.cache_index_and_filter_blocks = true; + table_options.filter_policy.reset(NewBloomFilterPolicy(20)); + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + CreateAndReopenWithCF({"pikachu"}, options); + + ASSERT_OK(Put(1, "key", "val")); + // Create a new table. + ASSERT_OK(Flush(1)); + + // index/filter blocks added to block cache right after table creation. + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(2, /* only index/filter were added */ + TestGetTickerCount(options, BLOCK_CACHE_ADD)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_DATA_MISS)); + uint64_t int_num; + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); + ASSERT_EQ(int_num, 0U); + + // Make sure filter block is in cache. + std::string value; + ReadOptions ropt; + db_->KeyMayExist(ReadOptions(), handles_[1], "key", &value); + + // Miss count should remain the same. + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + + db_->KeyMayExist(ReadOptions(), handles_[1], "key", &value); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(2, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + + // Make sure index block is in cache. + auto index_block_hit = TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT); + value = Get(1, "key"); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(index_block_hit + 1, + TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); + + value = Get(1, "key"); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(index_block_hit + 2, + TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); +} + +TEST_F(DBBlockCacheTest, IndexAndFilterBlocksStats) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions table_options; + table_options.cache_index_and_filter_blocks = true; + // 200 bytes are enough to hold the first two blocks + std::shared_ptr cache = NewLRUCache(200, 0, false); + table_options.block_cache = cache; + table_options.filter_policy.reset(NewBloomFilterPolicy(20)); + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + CreateAndReopenWithCF({"pikachu"}, options); + + ASSERT_OK(Put(1, "key", "val")); + // Create a new table + ASSERT_OK(Flush(1)); + size_t index_bytes_insert = + TestGetTickerCount(options, BLOCK_CACHE_INDEX_BYTES_INSERT); + size_t filter_bytes_insert = + TestGetTickerCount(options, BLOCK_CACHE_FILTER_BYTES_INSERT); + ASSERT_GT(index_bytes_insert, 0); + ASSERT_GT(filter_bytes_insert, 0); + ASSERT_EQ(cache->GetUsage(), index_bytes_insert + filter_bytes_insert); + // set the cache capacity to the current usage + cache->SetCapacity(index_bytes_insert + filter_bytes_insert); + ASSERT_EQ(TestGetTickerCount(options, BLOCK_CACHE_INDEX_BYTES_EVICT), 0); + ASSERT_EQ(TestGetTickerCount(options, BLOCK_CACHE_FILTER_BYTES_EVICT), 0); + ASSERT_OK(Put(1, "key2", "val")); + // Create a new table + ASSERT_OK(Flush(1)); + // cache evicted old index and block entries + ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_INDEX_BYTES_INSERT), + index_bytes_insert); + ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_FILTER_BYTES_INSERT), + filter_bytes_insert); + ASSERT_EQ(TestGetTickerCount(options, BLOCK_CACHE_INDEX_BYTES_EVICT), + index_bytes_insert); + ASSERT_EQ(TestGetTickerCount(options, BLOCK_CACHE_FILTER_BYTES_EVICT), + filter_bytes_insert); +} + +namespace { + +// A mock cache wraps LRUCache, and record how many entries have been +// inserted for each priority. +class MockCache : public LRUCache { + public: + static uint32_t high_pri_insert_count; + static uint32_t low_pri_insert_count; + + MockCache() : LRUCache(1 << 25, 0, false, 0.0) {} + + virtual Status Insert(const Slice& key, void* value, size_t charge, + void (*deleter)(const Slice& key, void* value), + Handle** handle, Priority priority) override { + if (priority == Priority::LOW) { + low_pri_insert_count++; + } else { + high_pri_insert_count++; + } + return LRUCache::Insert(key, value, charge, deleter, handle, priority); + } +}; + +uint32_t MockCache::high_pri_insert_count = 0; +uint32_t MockCache::low_pri_insert_count = 0; + +} // anonymous namespace + +TEST_F(DBBlockCacheTest, IndexAndFilterBlocksCachePriority) { + for (auto priority : {Cache::Priority::LOW, Cache::Priority::HIGH}) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions table_options; + table_options.cache_index_and_filter_blocks = true; + table_options.block_cache.reset(new MockCache()); + table_options.filter_policy.reset(NewBloomFilterPolicy(20)); + table_options.cache_index_and_filter_blocks_with_high_priority = + priority == Cache::Priority::HIGH ? true : false; + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + DestroyAndReopen(options); + + MockCache::high_pri_insert_count = 0; + MockCache::low_pri_insert_count = 0; + + // Create a new table. + ASSERT_OK(Put("foo", "value")); + ASSERT_OK(Put("bar", "value")); + ASSERT_OK(Flush()); + ASSERT_EQ(1, NumTableFilesAtLevel(0)); + + // index/filter blocks added to block cache right after table creation. + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(2, /* only index/filter were added */ + TestGetTickerCount(options, BLOCK_CACHE_ADD)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_DATA_MISS)); + if (priority == Cache::Priority::LOW) { + ASSERT_EQ(0, MockCache::high_pri_insert_count); + ASSERT_EQ(2, MockCache::low_pri_insert_count); + } else { + ASSERT_EQ(2, MockCache::high_pri_insert_count); + ASSERT_EQ(0, MockCache::low_pri_insert_count); + } + + // Access data block. + ASSERT_EQ("value", Get("foo")); + + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(3, /*adding data block*/ + TestGetTickerCount(options, BLOCK_CACHE_ADD)); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_DATA_MISS)); + + // Data block should be inserted with low priority. + if (priority == Cache::Priority::LOW) { + ASSERT_EQ(0, MockCache::high_pri_insert_count); + ASSERT_EQ(3, MockCache::low_pri_insert_count); + } else { + ASSERT_EQ(2, MockCache::high_pri_insert_count); + ASSERT_EQ(1, MockCache::low_pri_insert_count); + } + } +} + +TEST_F(DBBlockCacheTest, ParanoidFileChecks) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + options.level0_file_num_compaction_trigger = 2; + options.paranoid_file_checks = true; + BlockBasedTableOptions table_options; + table_options.cache_index_and_filter_blocks = false; + table_options.filter_policy.reset(NewBloomFilterPolicy(20)); + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + CreateAndReopenWithCF({"pikachu"}, options); + + ASSERT_OK(Put(1, "1_key", "val")); + ASSERT_OK(Put(1, "9_key", "val")); + // Create a new table. + ASSERT_OK(Flush(1)); + ASSERT_EQ(1, /* read and cache data block */ + TestGetTickerCount(options, BLOCK_CACHE_ADD)); + + ASSERT_OK(Put(1, "1_key2", "val2")); + ASSERT_OK(Put(1, "9_key2", "val2")); + // Create a new SST file. This will further trigger a compaction + // and generate another file. + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(3, /* Totally 3 files created up to now */ + TestGetTickerCount(options, BLOCK_CACHE_ADD)); + + // After disabling options.paranoid_file_checks. NO further block + // is added after generating a new file. + ASSERT_OK( + dbfull()->SetOptions(handles_[1], {{"paranoid_file_checks", "false"}})); + + ASSERT_OK(Put(1, "1_key3", "val3")); + ASSERT_OK(Put(1, "9_key3", "val3")); + ASSERT_OK(Flush(1)); + ASSERT_OK(Put(1, "1_key4", "val4")); + ASSERT_OK(Put(1, "9_key4", "val4")); + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(3, /* Totally 3 files created up to now */ + TestGetTickerCount(options, BLOCK_CACHE_ADD)); +} + +TEST_F(DBBlockCacheTest, CompressedCache) { + if (!Snappy_Supported()) { + return; + } + int num_iter = 80; + + // Run this test three iterations. + // Iteration 1: only a uncompressed block cache + // Iteration 2: only a compressed block cache + // Iteration 3: both block cache and compressed cache + // Iteration 4: both block cache and compressed cache, but DB is not + // compressed + for (int iter = 0; iter < 4; iter++) { + Options options = CurrentOptions(); + options.write_buffer_size = 64 * 1024; // small write buffer + options.statistics = rocksdb::CreateDBStatistics(); + + BlockBasedTableOptions table_options; + switch (iter) { + case 0: + // only uncompressed block cache + table_options.block_cache = NewLRUCache(8 * 1024); + table_options.block_cache_compressed = nullptr; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + break; + case 1: + // no block cache, only compressed cache + table_options.no_block_cache = true; + table_options.block_cache = nullptr; + table_options.block_cache_compressed = NewLRUCache(8 * 1024); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + break; + case 2: + // both compressed and uncompressed block cache + table_options.block_cache = NewLRUCache(1024); + table_options.block_cache_compressed = NewLRUCache(8 * 1024); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + break; + case 3: + // both block cache and compressed cache, but DB is not compressed + // also, make block cache sizes bigger, to trigger block cache hits + table_options.block_cache = NewLRUCache(1024 * 1024); + table_options.block_cache_compressed = NewLRUCache(8 * 1024 * 1024); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.compression = kNoCompression; + break; + default: + FAIL(); + } + CreateAndReopenWithCF({"pikachu"}, options); + // default column family doesn't have block cache + Options no_block_cache_opts; + no_block_cache_opts.statistics = options.statistics; + no_block_cache_opts = CurrentOptions(no_block_cache_opts); + BlockBasedTableOptions table_options_no_bc; + table_options_no_bc.no_block_cache = true; + no_block_cache_opts.table_factory.reset( + NewBlockBasedTableFactory(table_options_no_bc)); + ReopenWithColumnFamilies( + {"default", "pikachu"}, + std::vector({no_block_cache_opts, options})); + + Random rnd(301); + + // Write 8MB (80 values, each 100K) + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + std::vector values; + std::string str; + for (int i = 0; i < num_iter; i++) { + if (i % 4 == 0) { // high compression ratio + str = RandomString(&rnd, 1000); + } + values.push_back(str); + ASSERT_OK(Put(1, Key(i), values[i])); + } + + // flush all data from memtable so that reads are from block cache + ASSERT_OK(Flush(1)); + + for (int i = 0; i < num_iter; i++) { + ASSERT_EQ(Get(1, Key(i)), values[i]); + } + + // check that we triggered the appropriate code paths in the cache + switch (iter) { + case 0: + // only uncompressed block cache + ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_MISS), 0); + ASSERT_EQ(TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_MISS), 0); + break; + case 1: + // no block cache, only compressed cache + ASSERT_EQ(TestGetTickerCount(options, BLOCK_CACHE_MISS), 0); + ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_MISS), 0); + break; + case 2: + // both compressed and uncompressed block cache + ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_MISS), 0); + ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_MISS), 0); + break; + case 3: + // both compressed and uncompressed block cache + ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_MISS), 0); + ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_HIT), 0); + ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_MISS), 0); + // compressed doesn't have any hits since blocks are not compressed on + // storage + ASSERT_EQ(TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_HIT), 0); + break; + default: + FAIL(); + } + + options.create_if_missing = true; + DestroyAndReopen(options); + } +} + +#endif // ROCKSDB_LITE + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_bloom_filter_test.cc b/db/db_bloom_filter_test.cc new file mode 100644 index 00000000000..e6248a04014 --- /dev/null +++ b/db/db_bloom_filter_test.cc @@ -0,0 +1,1108 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_test_util.h" +#include "port/stack_trace.h" +#include "rocksdb/perf_context.h" + +namespace rocksdb { + +// DB tests related to bloom filter. + +class DBBloomFilterTest : public DBTestBase { + public: + DBBloomFilterTest() : DBTestBase("/db_bloom_filter_test") {} +}; + +class DBBloomFilterTestWithParam + : public DBTestBase, + public testing::WithParamInterface> { + // public testing::WithParamInterface { + protected: + bool use_block_based_filter_; + bool partition_filters_; + + public: + DBBloomFilterTestWithParam() : DBTestBase("/db_bloom_filter_tests") {} + + ~DBBloomFilterTestWithParam() {} + + void SetUp() override { + use_block_based_filter_ = std::get<0>(GetParam()); + partition_filters_ = std::get<1>(GetParam()); + } +}; + +// KeyMayExist can lead to a few false positives, but not false negatives. +// To make test deterministic, use a much larger number of bits per key-20 than +// bits in the key, so that false positives are eliminated +TEST_P(DBBloomFilterTestWithParam, KeyMayExist) { + do { + ReadOptions ropts; + std::string value; + anon::OptionsOverride options_override; + options_override.filter_policy.reset( + NewBloomFilterPolicy(20, use_block_based_filter_)); + options_override.partition_filters = partition_filters_; + options_override.metadata_block_size = 32; + Options options = CurrentOptions(options_override); + if (partition_filters_ && + static_cast( + options.table_factory->GetOptions()) + ->index_type != BlockBasedTableOptions::kTwoLevelIndexSearch) { + // In the current implementation partitioned filters depend on partitioned + // indexes + continue; + } + options.statistics = rocksdb::CreateDBStatistics(); + CreateAndReopenWithCF({"pikachu"}, options); + + ASSERT_TRUE(!db_->KeyMayExist(ropts, handles_[1], "a", &value)); + + ASSERT_OK(Put(1, "a", "b")); + bool value_found = false; + ASSERT_TRUE( + db_->KeyMayExist(ropts, handles_[1], "a", &value, &value_found)); + ASSERT_TRUE(value_found); + ASSERT_EQ("b", value); + + ASSERT_OK(Flush(1)); + value.clear(); + + uint64_t numopen = TestGetTickerCount(options, NO_FILE_OPENS); + uint64_t cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); + ASSERT_TRUE( + db_->KeyMayExist(ropts, handles_[1], "a", &value, &value_found)); + ASSERT_TRUE(!value_found); + // assert that no new files were opened and no new blocks were + // read into block cache. + ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); + ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); + + ASSERT_OK(Delete(1, "a")); + + numopen = TestGetTickerCount(options, NO_FILE_OPENS); + cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); + ASSERT_TRUE(!db_->KeyMayExist(ropts, handles_[1], "a", &value)); + ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); + ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); + + ASSERT_OK(Flush(1)); + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1], + true /* disallow trivial move */); + + numopen = TestGetTickerCount(options, NO_FILE_OPENS); + cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); + ASSERT_TRUE(!db_->KeyMayExist(ropts, handles_[1], "a", &value)); + ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); + ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); + + ASSERT_OK(Delete(1, "c")); + + numopen = TestGetTickerCount(options, NO_FILE_OPENS); + cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); + ASSERT_TRUE(!db_->KeyMayExist(ropts, handles_[1], "c", &value)); + ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); + ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); + + // KeyMayExist function only checks data in block caches, which is not used + // by plain table format. + } while ( + ChangeOptions(kSkipPlainTable | kSkipHashIndex | kSkipFIFOCompaction)); +} + +TEST_F(DBBloomFilterTest, GetFilterByPrefixBloom) { + for (bool partition_filters : {true, false}) { + Options options = last_options_; + options.prefix_extractor.reset(NewFixedPrefixTransform(8)); + options.statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions bbto; + bbto.filter_policy.reset(NewBloomFilterPolicy(10, false)); + if (partition_filters) { + bbto.partition_filters = true; + bbto.index_type = BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch; + } + bbto.whole_key_filtering = false; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + DestroyAndReopen(options); + + WriteOptions wo; + ReadOptions ro; + FlushOptions fo; + fo.wait = true; + std::string value; + + ASSERT_OK(dbfull()->Put(wo, "barbarbar", "foo")); + ASSERT_OK(dbfull()->Put(wo, "barbarbar2", "foo2")); + ASSERT_OK(dbfull()->Put(wo, "foofoofoo", "bar")); + + dbfull()->Flush(fo); + + ASSERT_EQ("foo", Get("barbarbar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 0); + ASSERT_EQ("foo2", Get("barbarbar2")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 0); + ASSERT_EQ("NOT_FOUND", Get("barbarbar3")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 0); + + ASSERT_EQ("NOT_FOUND", Get("barfoofoo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + + ASSERT_EQ("NOT_FOUND", Get("foobarbar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 2); + + ro.total_order_seek = true; + ASSERT_TRUE(db_->Get(ro, "foobarbar", &value).IsNotFound()); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 2); + } +} + +TEST_F(DBBloomFilterTest, WholeKeyFilterProp) { + for (bool partition_filters : {true, false}) { + Options options = last_options_; + options.prefix_extractor.reset(NewFixedPrefixTransform(3)); + options.statistics = rocksdb::CreateDBStatistics(); + + BlockBasedTableOptions bbto; + bbto.filter_policy.reset(NewBloomFilterPolicy(10, false)); + bbto.whole_key_filtering = false; + if (partition_filters) { + bbto.partition_filters = true; + bbto.index_type = BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch; + } + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + DestroyAndReopen(options); + + WriteOptions wo; + ReadOptions ro; + FlushOptions fo; + fo.wait = true; + std::string value; + + ASSERT_OK(dbfull()->Put(wo, "foobar", "foo")); + // Needs insert some keys to make sure files are not filtered out by key + // ranges. + ASSERT_OK(dbfull()->Put(wo, "aaa", "")); + ASSERT_OK(dbfull()->Put(wo, "zzz", "")); + dbfull()->Flush(fo); + + Reopen(options); + ASSERT_EQ("NOT_FOUND", Get("foo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 0); + ASSERT_EQ("NOT_FOUND", Get("bar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + ASSERT_EQ("foo", Get("foobar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + + // Reopen with whole key filtering enabled and prefix extractor + // NULL. Bloom filter should be off for both of whole key and + // prefix bloom. + bbto.whole_key_filtering = true; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + options.prefix_extractor.reset(); + Reopen(options); + + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + ASSERT_EQ("NOT_FOUND", Get("foo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + ASSERT_EQ("NOT_FOUND", Get("bar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + ASSERT_EQ("foo", Get("foobar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + // Write DB with only full key filtering. + ASSERT_OK(dbfull()->Put(wo, "foobar", "foo")); + // Needs insert some keys to make sure files are not filtered out by key + // ranges. + ASSERT_OK(dbfull()->Put(wo, "aaa", "")); + ASSERT_OK(dbfull()->Put(wo, "zzz", "")); + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + + // Reopen with both of whole key off and prefix extractor enabled. + // Still no bloom filter should be used. + options.prefix_extractor.reset(NewFixedPrefixTransform(3)); + bbto.whole_key_filtering = false; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + Reopen(options); + + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + ASSERT_EQ("NOT_FOUND", Get("foo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + ASSERT_EQ("NOT_FOUND", Get("bar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + ASSERT_EQ("foo", Get("foobar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + + // Try to create a DB with mixed files: + ASSERT_OK(dbfull()->Put(wo, "foobar", "foo")); + // Needs insert some keys to make sure files are not filtered out by key + // ranges. + ASSERT_OK(dbfull()->Put(wo, "aaa", "")); + ASSERT_OK(dbfull()->Put(wo, "zzz", "")); + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + + options.prefix_extractor.reset(); + bbto.whole_key_filtering = true; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + Reopen(options); + + // Try to create a DB with mixed files. + ASSERT_OK(dbfull()->Put(wo, "barfoo", "bar")); + // In this case needs insert some keys to make sure files are + // not filtered out by key ranges. + ASSERT_OK(dbfull()->Put(wo, "aaa", "")); + ASSERT_OK(dbfull()->Put(wo, "zzz", "")); + Flush(); + + // Now we have two files: + // File 1: An older file with prefix bloom. + // File 2: A newer file with whole bloom filter. + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 1); + ASSERT_EQ("NOT_FOUND", Get("foo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 2); + ASSERT_EQ("NOT_FOUND", Get("bar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 3); + ASSERT_EQ("foo", Get("foobar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 4); + ASSERT_EQ("bar", Get("barfoo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 4); + + // Reopen with the same setting: only whole key is used + Reopen(options); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 4); + ASSERT_EQ("NOT_FOUND", Get("foo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 5); + ASSERT_EQ("NOT_FOUND", Get("bar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 6); + ASSERT_EQ("foo", Get("foobar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 7); + ASSERT_EQ("bar", Get("barfoo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 7); + + // Restart with both filters are allowed + options.prefix_extractor.reset(NewFixedPrefixTransform(3)); + bbto.whole_key_filtering = true; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + Reopen(options); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 7); + // File 1 will has it filtered out. + // File 2 will not, as prefix `foo` exists in the file. + ASSERT_EQ("NOT_FOUND", Get("foo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 8); + ASSERT_EQ("NOT_FOUND", Get("bar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 10); + ASSERT_EQ("foo", Get("foobar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 11); + ASSERT_EQ("bar", Get("barfoo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 11); + + // Restart with only prefix bloom is allowed. + options.prefix_extractor.reset(NewFixedPrefixTransform(3)); + bbto.whole_key_filtering = false; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + Reopen(options); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 11); + ASSERT_EQ("NOT_FOUND", Get("foo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 11); + ASSERT_EQ("NOT_FOUND", Get("bar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 12); + ASSERT_EQ("foo", Get("foobar")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 12); + ASSERT_EQ("bar", Get("barfoo")); + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 12); + } +} + +TEST_P(DBBloomFilterTestWithParam, BloomFilter) { + do { + Options options = CurrentOptions(); + env_->count_random_reads_ = true; + options.env = env_; + // ChangeCompactOptions() only changes compaction style, which does not + // trigger reset of table_factory + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + table_options.filter_policy.reset( + NewBloomFilterPolicy(10, use_block_based_filter_)); + table_options.partition_filters = partition_filters_; + if (partition_filters_) { + table_options.index_type = + BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch; + } + table_options.metadata_block_size = 32; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + CreateAndReopenWithCF({"pikachu"}, options); + + // Populate multiple layers + const int N = 10000; + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(1, Key(i), Key(i))); + } + Compact(1, "a", "z"); + for (int i = 0; i < N; i += 100) { + ASSERT_OK(Put(1, Key(i), Key(i))); + } + Flush(1); + + // Prevent auto compactions triggered by seeks + env_->delay_sstable_sync_.store(true, std::memory_order_release); + + // Lookup present keys. Should rarely read from small sstable. + env_->random_read_counter_.Reset(); + for (int i = 0; i < N; i++) { + ASSERT_EQ(Key(i), Get(1, Key(i))); + } + int reads = env_->random_read_counter_.Read(); + fprintf(stderr, "%d present => %d reads\n", N, reads); + ASSERT_GE(reads, N); + if (partition_filters_) { + // Without block cache, we read an extra partition filter per each + // level*read and a partition index per each read + ASSERT_LE(reads, 4 * N + 2 * N / 100); + } else { + ASSERT_LE(reads, N + 2 * N / 100); + } + + // Lookup present keys. Should rarely read from either sstable. + env_->random_read_counter_.Reset(); + for (int i = 0; i < N; i++) { + ASSERT_EQ("NOT_FOUND", Get(1, Key(i) + ".missing")); + } + reads = env_->random_read_counter_.Read(); + fprintf(stderr, "%d missing => %d reads\n", N, reads); + if (partition_filters_) { + // With partitioned filter we read one extra filter per level per each + // missed read. + ASSERT_LE(reads, 2 * N + 3 * N / 100); + } else { + ASSERT_LE(reads, 3 * N / 100); + } + + env_->delay_sstable_sync_.store(false, std::memory_order_release); + Close(); + } while (ChangeCompactOptions()); +} + +INSTANTIATE_TEST_CASE_P(DBBloomFilterTestWithParam, DBBloomFilterTestWithParam, + ::testing::Values(std::make_tuple(true, false), + std::make_tuple(false, true), + std::make_tuple(false, false))); + +TEST_F(DBBloomFilterTest, BloomFilterRate) { + while (ChangeFilterOptions()) { + Options options = CurrentOptions(); + options.statistics = rocksdb::CreateDBStatistics(); + CreateAndReopenWithCF({"pikachu"}, options); + + const int maxKey = 10000; + for (int i = 0; i < maxKey; i++) { + ASSERT_OK(Put(1, Key(i), Key(i))); + } + // Add a large key to make the file contain wide range + ASSERT_OK(Put(1, Key(maxKey + 55555), Key(maxKey + 55555))); + Flush(1); + + // Check if they can be found + for (int i = 0; i < maxKey; i++) { + ASSERT_EQ(Key(i), Get(1, Key(i))); + } + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 0); + + // Check if filter is useful + for (int i = 0; i < maxKey; i++) { + ASSERT_EQ("NOT_FOUND", Get(1, Key(i + 33333))); + } + ASSERT_GE(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), maxKey * 0.98); + } +} + +TEST_F(DBBloomFilterTest, BloomFilterCompatibility) { + Options options = CurrentOptions(); + options.statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions table_options; + table_options.filter_policy.reset(NewBloomFilterPolicy(10, true)); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + // Create with block based filter + CreateAndReopenWithCF({"pikachu"}, options); + + const int maxKey = 10000; + for (int i = 0; i < maxKey; i++) { + ASSERT_OK(Put(1, Key(i), Key(i))); + } + ASSERT_OK(Put(1, Key(maxKey + 55555), Key(maxKey + 55555))); + Flush(1); + + // Check db with full filter + table_options.filter_policy.reset(NewBloomFilterPolicy(10, false)); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + // Check if they can be found + for (int i = 0; i < maxKey; i++) { + ASSERT_EQ(Key(i), Get(1, Key(i))); + } + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 0); + + // Check db with partitioned full filter + table_options.partition_filters = true; + table_options.index_type = + BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch; + table_options.filter_policy.reset(NewBloomFilterPolicy(10, false)); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + // Check if they can be found + for (int i = 0; i < maxKey; i++) { + ASSERT_EQ(Key(i), Get(1, Key(i))); + } + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 0); +} + +TEST_F(DBBloomFilterTest, BloomFilterReverseCompatibility) { + for (bool partition_filters : {true, false}) { + Options options = CurrentOptions(); + options.statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions table_options; + if (partition_filters) { + table_options.partition_filters = true; + table_options.index_type = + BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch; + } + table_options.filter_policy.reset(NewBloomFilterPolicy(10, false)); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + DestroyAndReopen(options); + + // Create with full filter + CreateAndReopenWithCF({"pikachu"}, options); + + const int maxKey = 10000; + for (int i = 0; i < maxKey; i++) { + ASSERT_OK(Put(1, Key(i), Key(i))); + } + ASSERT_OK(Put(1, Key(maxKey + 55555), Key(maxKey + 55555))); + Flush(1); + + // Check db with block_based filter + table_options.filter_policy.reset(NewBloomFilterPolicy(10, true)); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + // Check if they can be found + for (int i = 0; i < maxKey; i++) { + ASSERT_EQ(Key(i), Get(1, Key(i))); + } + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 0); + } +} + +namespace { +// A wrapped bloom over default FilterPolicy +class WrappedBloom : public FilterPolicy { + public: + explicit WrappedBloom(int bits_per_key) + : filter_(NewBloomFilterPolicy(bits_per_key)), counter_(0) {} + + ~WrappedBloom() { delete filter_; } + + const char* Name() const override { return "WrappedRocksDbFilterPolicy"; } + + void CreateFilter(const rocksdb::Slice* keys, int n, + std::string* dst) const override { + std::unique_ptr user_keys(new rocksdb::Slice[n]); + for (int i = 0; i < n; ++i) { + user_keys[i] = convertKey(keys[i]); + } + return filter_->CreateFilter(user_keys.get(), n, dst); + } + + bool KeyMayMatch(const rocksdb::Slice& key, + const rocksdb::Slice& filter) const override { + counter_++; + return filter_->KeyMayMatch(convertKey(key), filter); + } + + uint32_t GetCounter() { return counter_; } + + private: + const FilterPolicy* filter_; + mutable uint32_t counter_; + + rocksdb::Slice convertKey(const rocksdb::Slice& key) const { return key; } +}; +} // namespace + +TEST_F(DBBloomFilterTest, BloomFilterWrapper) { + Options options = CurrentOptions(); + options.statistics = rocksdb::CreateDBStatistics(); + + BlockBasedTableOptions table_options; + WrappedBloom* policy = new WrappedBloom(10); + table_options.filter_policy.reset(policy); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + CreateAndReopenWithCF({"pikachu"}, options); + + const int maxKey = 10000; + for (int i = 0; i < maxKey; i++) { + ASSERT_OK(Put(1, Key(i), Key(i))); + } + // Add a large key to make the file contain wide range + ASSERT_OK(Put(1, Key(maxKey + 55555), Key(maxKey + 55555))); + ASSERT_EQ(0U, policy->GetCounter()); + Flush(1); + + // Check if they can be found + for (int i = 0; i < maxKey; i++) { + ASSERT_EQ(Key(i), Get(1, Key(i))); + } + ASSERT_EQ(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 0); + ASSERT_EQ(1U * maxKey, policy->GetCounter()); + + // Check if filter is useful + for (int i = 0; i < maxKey; i++) { + ASSERT_EQ("NOT_FOUND", Get(1, Key(i + 33333))); + } + ASSERT_GE(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), maxKey * 0.98); + ASSERT_EQ(2U * maxKey, policy->GetCounter()); +} + +class SliceTransformLimitedDomain : public SliceTransform { + const char* Name() const override { return "SliceTransformLimitedDomain"; } + + Slice Transform(const Slice& src) const override { + return Slice(src.data(), 5); + } + + bool InDomain(const Slice& src) const override { + // prefix will be x???? + return src.size() >= 5 && src[0] == 'x'; + } + + bool InRange(const Slice& dst) const override { + // prefix will be x???? + return dst.size() == 5 && dst[0] == 'x'; + } +}; + +TEST_F(DBBloomFilterTest, PrefixExtractorFullFilter) { + BlockBasedTableOptions bbto; + // Full Filter Block + bbto.filter_policy.reset(rocksdb::NewBloomFilterPolicy(10, false)); + bbto.whole_key_filtering = false; + + Options options = CurrentOptions(); + options.prefix_extractor = std::make_shared(); + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + + DestroyAndReopen(options); + + ASSERT_OK(Put("x1111_AAAA", "val1")); + ASSERT_OK(Put("x1112_AAAA", "val2")); + ASSERT_OK(Put("x1113_AAAA", "val3")); + ASSERT_OK(Put("x1114_AAAA", "val4")); + // Not in domain, wont be added to filter + ASSERT_OK(Put("zzzzz_AAAA", "val5")); + + ASSERT_OK(Flush()); + + ASSERT_EQ(Get("x1111_AAAA"), "val1"); + ASSERT_EQ(Get("x1112_AAAA"), "val2"); + ASSERT_EQ(Get("x1113_AAAA"), "val3"); + ASSERT_EQ(Get("x1114_AAAA"), "val4"); + // Was not added to filter but rocksdb will try to read it from the filter + ASSERT_EQ(Get("zzzzz_AAAA"), "val5"); +} + +TEST_F(DBBloomFilterTest, PrefixExtractorBlockFilter) { + BlockBasedTableOptions bbto; + // Block Filter Block + bbto.filter_policy.reset(rocksdb::NewBloomFilterPolicy(10, true)); + + Options options = CurrentOptions(); + options.prefix_extractor = std::make_shared(); + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + + DestroyAndReopen(options); + + ASSERT_OK(Put("x1113_AAAA", "val3")); + ASSERT_OK(Put("x1114_AAAA", "val4")); + // Not in domain, wont be added to filter + ASSERT_OK(Put("zzzzz_AAAA", "val1")); + ASSERT_OK(Put("zzzzz_AAAB", "val2")); + ASSERT_OK(Put("zzzzz_AAAC", "val3")); + ASSERT_OK(Put("zzzzz_AAAD", "val4")); + + ASSERT_OK(Flush()); + + std::vector iter_res; + auto iter = db_->NewIterator(ReadOptions()); + // Seek to a key that was not in Domain + for (iter->Seek("zzzzz_AAAA"); iter->Valid(); iter->Next()) { + iter_res.emplace_back(iter->value().ToString()); + } + + std::vector expected_res = {"val1", "val2", "val3", "val4"}; + ASSERT_EQ(iter_res, expected_res); + delete iter; +} + +#ifndef ROCKSDB_LITE +class BloomStatsTestWithParam + : public DBBloomFilterTest, + public testing::WithParamInterface> { + public: + BloomStatsTestWithParam() { + use_block_table_ = std::get<0>(GetParam()); + use_block_based_builder_ = std::get<1>(GetParam()); + partition_filters_ = std::get<2>(GetParam()); + + options_.create_if_missing = true; + options_.prefix_extractor.reset(rocksdb::NewFixedPrefixTransform(4)); + options_.memtable_prefix_bloom_size_ratio = + 8.0 * 1024.0 / static_cast(options_.write_buffer_size); + if (use_block_table_) { + BlockBasedTableOptions table_options; + table_options.hash_index_allow_collision = false; + if (partition_filters_) { + assert(!use_block_based_builder_); + table_options.partition_filters = partition_filters_; + table_options.index_type = + BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch; + } + table_options.filter_policy.reset( + NewBloomFilterPolicy(10, use_block_based_builder_)); + options_.table_factory.reset(NewBlockBasedTableFactory(table_options)); + } else { + assert(!partition_filters_); // not supported in plain table + PlainTableOptions table_options; + options_.table_factory.reset(NewPlainTableFactory(table_options)); + } + options_.env = env_; + + get_perf_context()->Reset(); + DestroyAndReopen(options_); + } + + ~BloomStatsTestWithParam() { + get_perf_context()->Reset(); + Destroy(options_); + } + + // Required if inheriting from testing::WithParamInterface<> + static void SetUpTestCase() {} + static void TearDownTestCase() {} + + bool use_block_table_; + bool use_block_based_builder_; + bool partition_filters_; + Options options_; +}; + +// 1 Insert 2 K-V pairs into DB +// 2 Call Get() for both keys - expext memtable bloom hit stat to be 2 +// 3 Call Get() for nonexisting key - expect memtable bloom miss stat to be 1 +// 4 Call Flush() to create SST +// 5 Call Get() for both keys - expext SST bloom hit stat to be 2 +// 6 Call Get() for nonexisting key - expect SST bloom miss stat to be 1 +// Test both: block and plain SST +TEST_P(BloomStatsTestWithParam, BloomStatsTest) { + std::string key1("AAAA"); + std::string key2("RXDB"); // not in DB + std::string key3("ZBRA"); + std::string value1("Value1"); + std::string value3("Value3"); + + ASSERT_OK(Put(key1, value1, WriteOptions())); + ASSERT_OK(Put(key3, value3, WriteOptions())); + + // check memtable bloom stats + ASSERT_EQ(value1, Get(key1)); + ASSERT_EQ(1, get_perf_context()->bloom_memtable_hit_count); + ASSERT_EQ(value3, Get(key3)); + ASSERT_EQ(2, get_perf_context()->bloom_memtable_hit_count); + ASSERT_EQ(0, get_perf_context()->bloom_memtable_miss_count); + + ASSERT_EQ("NOT_FOUND", Get(key2)); + ASSERT_EQ(1, get_perf_context()->bloom_memtable_miss_count); + ASSERT_EQ(2, get_perf_context()->bloom_memtable_hit_count); + + // sanity checks + ASSERT_EQ(0, get_perf_context()->bloom_sst_hit_count); + ASSERT_EQ(0, get_perf_context()->bloom_sst_miss_count); + + Flush(); + + // sanity checks + ASSERT_EQ(0, get_perf_context()->bloom_sst_hit_count); + ASSERT_EQ(0, get_perf_context()->bloom_sst_miss_count); + + // check SST bloom stats + ASSERT_EQ(value1, Get(key1)); + ASSERT_EQ(1, get_perf_context()->bloom_sst_hit_count); + ASSERT_EQ(value3, Get(key3)); + ASSERT_EQ(2, get_perf_context()->bloom_sst_hit_count); + + ASSERT_EQ("NOT_FOUND", Get(key2)); + ASSERT_EQ(1, get_perf_context()->bloom_sst_miss_count); +} + +// Same scenario as in BloomStatsTest but using an iterator +TEST_P(BloomStatsTestWithParam, BloomStatsTestWithIter) { + std::string key1("AAAA"); + std::string key2("RXDB"); // not in DB + std::string key3("ZBRA"); + std::string value1("Value1"); + std::string value3("Value3"); + + ASSERT_OK(Put(key1, value1, WriteOptions())); + ASSERT_OK(Put(key3, value3, WriteOptions())); + + unique_ptr iter(dbfull()->NewIterator(ReadOptions())); + + // check memtable bloom stats + iter->Seek(key1); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(value1, iter->value().ToString()); + ASSERT_EQ(1, get_perf_context()->bloom_memtable_hit_count); + ASSERT_EQ(0, get_perf_context()->bloom_memtable_miss_count); + + iter->Seek(key3); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(value3, iter->value().ToString()); + ASSERT_EQ(2, get_perf_context()->bloom_memtable_hit_count); + ASSERT_EQ(0, get_perf_context()->bloom_memtable_miss_count); + + iter->Seek(key2); + ASSERT_OK(iter->status()); + ASSERT_TRUE(!iter->Valid()); + ASSERT_EQ(1, get_perf_context()->bloom_memtable_miss_count); + ASSERT_EQ(2, get_perf_context()->bloom_memtable_hit_count); + + Flush(); + + iter.reset(dbfull()->NewIterator(ReadOptions())); + + // Check SST bloom stats + iter->Seek(key1); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(value1, iter->value().ToString()); + ASSERT_EQ(1, get_perf_context()->bloom_sst_hit_count); + + iter->Seek(key3); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(value3, iter->value().ToString()); + ASSERT_EQ(2, get_perf_context()->bloom_sst_hit_count); + + iter->Seek(key2); + ASSERT_OK(iter->status()); + ASSERT_TRUE(!iter->Valid()); + ASSERT_EQ(1, get_perf_context()->bloom_sst_miss_count); + ASSERT_EQ(2, get_perf_context()->bloom_sst_hit_count); +} + +INSTANTIATE_TEST_CASE_P(BloomStatsTestWithParam, BloomStatsTestWithParam, + ::testing::Values(std::make_tuple(true, true, false), + std::make_tuple(true, false, false), + std::make_tuple(true, false, true), + std::make_tuple(false, false, + false))); + +namespace { +void PrefixScanInit(DBBloomFilterTest* dbtest) { + char buf[100]; + std::string keystr; + const int small_range_sstfiles = 5; + const int big_range_sstfiles = 5; + + // Generate 11 sst files with the following prefix ranges. + // GROUP 0: [0,10] (level 1) + // GROUP 1: [1,2], [2,3], [3,4], [4,5], [5, 6] (level 0) + // GROUP 2: [0,6], [0,7], [0,8], [0,9], [0,10] (level 0) + // + // A seek with the previous API would do 11 random I/Os (to all the + // files). With the new API and a prefix filter enabled, we should + // only do 2 random I/O, to the 2 files containing the key. + + // GROUP 0 + snprintf(buf, sizeof(buf), "%02d______:start", 0); + keystr = std::string(buf); + ASSERT_OK(dbtest->Put(keystr, keystr)); + snprintf(buf, sizeof(buf), "%02d______:end", 10); + keystr = std::string(buf); + ASSERT_OK(dbtest->Put(keystr, keystr)); + dbtest->Flush(); + dbtest->dbfull()->CompactRange(CompactRangeOptions(), nullptr, + nullptr); // move to level 1 + + // GROUP 1 + for (int i = 1; i <= small_range_sstfiles; i++) { + snprintf(buf, sizeof(buf), "%02d______:start", i); + keystr = std::string(buf); + ASSERT_OK(dbtest->Put(keystr, keystr)); + snprintf(buf, sizeof(buf), "%02d______:end", i + 1); + keystr = std::string(buf); + ASSERT_OK(dbtest->Put(keystr, keystr)); + dbtest->Flush(); + } + + // GROUP 2 + for (int i = 1; i <= big_range_sstfiles; i++) { + snprintf(buf, sizeof(buf), "%02d______:start", 0); + keystr = std::string(buf); + ASSERT_OK(dbtest->Put(keystr, keystr)); + snprintf(buf, sizeof(buf), "%02d______:end", small_range_sstfiles + i + 1); + keystr = std::string(buf); + ASSERT_OK(dbtest->Put(keystr, keystr)); + dbtest->Flush(); + } +} +} // namespace + +TEST_F(DBBloomFilterTest, PrefixScan) { + while (ChangeFilterOptions()) { + int count; + Slice prefix; + Slice key; + char buf[100]; + Iterator* iter; + snprintf(buf, sizeof(buf), "03______:"); + prefix = Slice(buf, 8); + key = Slice(buf, 9); + ASSERT_EQ(key.difference_offset(prefix), 8); + ASSERT_EQ(prefix.difference_offset(key), 8); + // db configs + env_->count_random_reads_ = true; + Options options = CurrentOptions(); + options.env = env_; + options.prefix_extractor.reset(NewFixedPrefixTransform(8)); + options.disable_auto_compactions = true; + options.max_background_compactions = 2; + options.create_if_missing = true; + options.memtable_factory.reset(NewHashSkipListRepFactory(16)); + options.allow_concurrent_memtable_write = false; + + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + table_options.filter_policy.reset(NewBloomFilterPolicy(10)); + table_options.whole_key_filtering = false; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + // 11 RAND I/Os + DestroyAndReopen(options); + PrefixScanInit(this); + count = 0; + env_->random_read_counter_.Reset(); + iter = db_->NewIterator(ReadOptions()); + for (iter->Seek(prefix); iter->Valid(); iter->Next()) { + if (!iter->key().starts_with(prefix)) { + break; + } + count++; + } + ASSERT_OK(iter->status()); + delete iter; + ASSERT_EQ(count, 2); + ASSERT_EQ(env_->random_read_counter_.Read(), 2); + Close(); + } // end of while +} + +TEST_F(DBBloomFilterTest, OptimizeFiltersForHits) { + Options options = CurrentOptions(); + options.write_buffer_size = 64 * 1024; + options.arena_block_size = 4 * 1024; + options.target_file_size_base = 64 * 1024; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 2; + options.level0_stop_writes_trigger = 4; + options.max_bytes_for_level_base = 256 * 1024; + options.max_write_buffer_number = 2; + options.max_background_compactions = 8; + options.max_background_flushes = 8; + options.compression = kNoCompression; + options.compaction_style = kCompactionStyleLevel; + options.level_compaction_dynamic_level_bytes = true; + BlockBasedTableOptions bbto; + bbto.cache_index_and_filter_blocks = true; + bbto.filter_policy.reset(NewBloomFilterPolicy(10, true)); + bbto.whole_key_filtering = true; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + options.optimize_filters_for_hits = true; + options.statistics = rocksdb::CreateDBStatistics(); + CreateAndReopenWithCF({"mypikachu"}, options); + + int numkeys = 200000; + + // Generate randomly shuffled keys, so the updates are almost + // random. + std::vector keys; + keys.reserve(numkeys); + for (int i = 0; i < numkeys; i += 2) { + keys.push_back(i); + } + std::random_shuffle(std::begin(keys), std::end(keys)); + + int num_inserted = 0; + for (int key : keys) { + ASSERT_OK(Put(1, Key(key), "val")); + if (++num_inserted % 1000 == 0) { + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } + } + ASSERT_OK(Put(1, Key(0), "val")); + ASSERT_OK(Put(1, Key(numkeys), "val")); + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + + if (NumTableFilesAtLevel(0, 1) == 0) { + // No Level 0 file. Create one. + ASSERT_OK(Put(1, Key(0), "val")); + ASSERT_OK(Put(1, Key(numkeys), "val")); + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + } + + for (int i = 1; i < numkeys; i += 2) { + ASSERT_EQ(Get(1, Key(i)), "NOT_FOUND"); + } + + ASSERT_EQ(0, TestGetTickerCount(options, GET_HIT_L0)); + ASSERT_EQ(0, TestGetTickerCount(options, GET_HIT_L1)); + ASSERT_EQ(0, TestGetTickerCount(options, GET_HIT_L2_AND_UP)); + + // Now we have three sorted run, L0, L5 and L6 with most files in L6 have + // no bloom filter. Most keys be checked bloom filters twice. + ASSERT_GT(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 65000 * 2); + ASSERT_LT(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 120000 * 2); + + for (int i = 0; i < numkeys; i += 2) { + ASSERT_EQ(Get(1, Key(i)), "val"); + } + + // Part 2 (read path): rewrite last level with blooms, then verify they get + // cached only if !optimize_filters_for_hits + options.disable_auto_compactions = true; + options.num_levels = 9; + options.optimize_filters_for_hits = false; + options.statistics = CreateDBStatistics(); + bbto.block_cache.reset(); + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + + ReopenWithColumnFamilies({"default", "mypikachu"}, options); + MoveFilesToLevel(7 /* level */, 1 /* column family index */); + + std::string value = Get(1, Key(0)); + uint64_t prev_cache_filter_hits = + TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT); + value = Get(1, Key(0)); + ASSERT_EQ(prev_cache_filter_hits + 1, + TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + + // Now that we know the filter blocks exist in the last level files, see if + // filter caching is skipped for this optimization + options.optimize_filters_for_hits = true; + options.statistics = CreateDBStatistics(); + bbto.block_cache.reset(); + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + + ReopenWithColumnFamilies({"default", "mypikachu"}, options); + + value = Get(1, Key(0)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(2 /* index and data block */, + TestGetTickerCount(options, BLOCK_CACHE_ADD)); + + // Check filter block ignored for files preloaded during DB::Open() + options.max_open_files = -1; + options.statistics = CreateDBStatistics(); + bbto.block_cache.reset(); + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + + ReopenWithColumnFamilies({"default", "mypikachu"}, options); + + uint64_t prev_cache_filter_misses = + TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS); + prev_cache_filter_hits = TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT); + Get(1, Key(0)); + ASSERT_EQ(prev_cache_filter_misses, + TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(prev_cache_filter_hits, + TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + + // Check filter block ignored for file trivially-moved to bottom level + bbto.block_cache.reset(); + options.max_open_files = 100; // setting > -1 makes it not preload all files + options.statistics = CreateDBStatistics(); + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + + ReopenWithColumnFamilies({"default", "mypikachu"}, options); + + ASSERT_OK(Put(1, Key(numkeys + 1), "val")); + ASSERT_OK(Flush(1)); + + int32_t trivial_move = 0; + int32_t non_trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", + [&](void* arg) { non_trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + CompactRangeOptions compact_options; + compact_options.bottommost_level_compaction = + BottommostLevelCompaction::kSkip; + compact_options.change_level = true; + compact_options.target_level = 7; + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr); + + ASSERT_EQ(trivial_move, 1); + ASSERT_EQ(non_trivial_move, 0); + + prev_cache_filter_hits = TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT); + prev_cache_filter_misses = + TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS); + value = Get(1, Key(numkeys + 1)); + ASSERT_EQ(prev_cache_filter_hits, + TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(prev_cache_filter_misses, + TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + + // Check filter block not cached for iterator + bbto.block_cache.reset(); + options.statistics = CreateDBStatistics(); + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + + ReopenWithColumnFamilies({"default", "mypikachu"}, options); + + std::unique_ptr iter(db_->NewIterator(ReadOptions(), handles_[1])); + iter->SeekToFirst(); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(2 /* index and data block */, + TestGetTickerCount(options, BLOCK_CACHE_ADD)); +} + +#endif // ROCKSDB_LITE + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_compaction_filter_test.cc b/db/db_compaction_filter_test.cc new file mode 100644 index 00000000000..9f751f059fa --- /dev/null +++ b/db/db_compaction_filter_test.cc @@ -0,0 +1,845 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_test_util.h" +#include "port/stack_trace.h" + +namespace rocksdb { + +static int cfilter_count = 0; +static int cfilter_skips = 0; + +// This is a static filter used for filtering +// kvs during the compaction process. +static std::string NEW_VALUE = "NewValue"; + +class DBTestCompactionFilter : public DBTestBase { + public: + DBTestCompactionFilter() : DBTestBase("/db_compaction_filter_test") {} +}; + +class KeepFilter : public CompactionFilter { + public: + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, bool* value_changed) const + override { + cfilter_count++; + return false; + } + + virtual const char* Name() const override { return "KeepFilter"; } +}; + +class DeleteFilter : public CompactionFilter { + public: + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, bool* value_changed) const + override { + cfilter_count++; + return true; + } + + virtual const char* Name() const override { return "DeleteFilter"; } +}; + +class DeleteISFilter : public CompactionFilter { + public: + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, + bool* value_changed) const override { + cfilter_count++; + int i = std::stoi(key.ToString()); + if (i > 5 && i <= 105) { + return true; + } + return false; + } + + virtual bool IgnoreSnapshots() const override { return true; } + + virtual const char* Name() const override { return "DeleteFilter"; } +}; + +// Skip x if floor(x/10) is even, use range skips. Requires that keys are +// zero-padded to length 10. +class SkipEvenFilter : public CompactionFilter { + public: + virtual Decision FilterV2(int level, const Slice& key, ValueType value_type, + const Slice& existing_value, std::string* new_value, + std::string* skip_until) const override { + cfilter_count++; + int i = std::stoi(key.ToString()); + if (i / 10 % 2 == 0) { + char key_str[100]; + snprintf(key_str, sizeof(key), "%010d", i / 10 * 10 + 10); + *skip_until = key_str; + ++cfilter_skips; + return Decision::kRemoveAndSkipUntil; + } + return Decision::kKeep; + } + + virtual bool IgnoreSnapshots() const override { return true; } + + virtual const char* Name() const override { return "DeleteFilter"; } +}; + +class DelayFilter : public CompactionFilter { + public: + explicit DelayFilter(DBTestBase* d) : db_test(d) {} + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, + bool* value_changed) const override { + db_test->env_->addon_time_.fetch_add(1000); + return true; + } + + virtual const char* Name() const override { return "DelayFilter"; } + + private: + DBTestBase* db_test; +}; + +class ConditionalFilter : public CompactionFilter { + public: + explicit ConditionalFilter(const std::string* filtered_value) + : filtered_value_(filtered_value) {} + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, + bool* value_changed) const override { + return value.ToString() == *filtered_value_; + } + + virtual const char* Name() const override { return "ConditionalFilter"; } + + private: + const std::string* filtered_value_; +}; + +class ChangeFilter : public CompactionFilter { + public: + explicit ChangeFilter() {} + + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, bool* value_changed) const + override { + assert(new_value != nullptr); + *new_value = NEW_VALUE; + *value_changed = true; + return false; + } + + virtual const char* Name() const override { return "ChangeFilter"; } +}; + +class KeepFilterFactory : public CompactionFilterFactory { + public: + explicit KeepFilterFactory(bool check_context = false, + bool check_context_cf_id = false) + : check_context_(check_context), + check_context_cf_id_(check_context_cf_id), + compaction_filter_created_(false) {} + + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + if (check_context_) { + EXPECT_EQ(expect_full_compaction_.load(), context.is_full_compaction); + EXPECT_EQ(expect_manual_compaction_.load(), context.is_manual_compaction); + } + if (check_context_cf_id_) { + EXPECT_EQ(expect_cf_id_.load(), context.column_family_id); + } + compaction_filter_created_ = true; + return std::unique_ptr(new KeepFilter()); + } + + bool compaction_filter_created() const { return compaction_filter_created_; } + + virtual const char* Name() const override { return "KeepFilterFactory"; } + bool check_context_; + bool check_context_cf_id_; + std::atomic_bool expect_full_compaction_; + std::atomic_bool expect_manual_compaction_; + std::atomic expect_cf_id_; + bool compaction_filter_created_; +}; + +class DeleteFilterFactory : public CompactionFilterFactory { + public: + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + if (context.is_manual_compaction) { + return std::unique_ptr(new DeleteFilter()); + } else { + return std::unique_ptr(nullptr); + } + } + + virtual const char* Name() const override { return "DeleteFilterFactory"; } +}; + +// Delete Filter Factory which ignores snapshots +class DeleteISFilterFactory : public CompactionFilterFactory { + public: + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + if (context.is_manual_compaction) { + return std::unique_ptr(new DeleteISFilter()); + } else { + return std::unique_ptr(nullptr); + } + } + + virtual const char* Name() const override { return "DeleteFilterFactory"; } +}; + +class SkipEvenFilterFactory : public CompactionFilterFactory { + public: + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + if (context.is_manual_compaction) { + return std::unique_ptr(new SkipEvenFilter()); + } else { + return std::unique_ptr(nullptr); + } + } + + virtual const char* Name() const override { return "SkipEvenFilterFactory"; } +}; + +class DelayFilterFactory : public CompactionFilterFactory { + public: + explicit DelayFilterFactory(DBTestBase* d) : db_test(d) {} + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + return std::unique_ptr(new DelayFilter(db_test)); + } + + virtual const char* Name() const override { return "DelayFilterFactory"; } + + private: + DBTestBase* db_test; +}; + +class ConditionalFilterFactory : public CompactionFilterFactory { + public: + explicit ConditionalFilterFactory(const Slice& filtered_value) + : filtered_value_(filtered_value.ToString()) {} + + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + return std::unique_ptr( + new ConditionalFilter(&filtered_value_)); + } + + virtual const char* Name() const override { + return "ConditionalFilterFactory"; + } + + private: + std::string filtered_value_; +}; + +class ChangeFilterFactory : public CompactionFilterFactory { + public: + explicit ChangeFilterFactory() {} + + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + return std::unique_ptr(new ChangeFilter()); + } + + virtual const char* Name() const override { return "ChangeFilterFactory"; } +}; + +#ifndef ROCKSDB_LITE +TEST_F(DBTestCompactionFilter, CompactionFilter) { + Options options = CurrentOptions(); + options.max_open_files = -1; + options.num_levels = 3; + options.compaction_filter_factory = std::make_shared(); + options = CurrentOptions(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Write 100K keys, these are written to a few files in L0. + const std::string value(10, 'x'); + for (int i = 0; i < 100000; i++) { + char key[100]; + snprintf(key, sizeof(key), "B%010d", i); + Put(1, key, value); + } + ASSERT_OK(Flush(1)); + + // Push all files to the highest level L2. Verify that + // the compaction is each level invokes the filter for + // all the keys in that level. + cfilter_count = 0; + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); + ASSERT_EQ(cfilter_count, 100000); + cfilter_count = 0; + dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); + ASSERT_EQ(cfilter_count, 100000); + + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + ASSERT_EQ(NumTableFilesAtLevel(1, 1), 0); + ASSERT_NE(NumTableFilesAtLevel(2, 1), 0); + cfilter_count = 0; + + // All the files are in the lowest level. + // Verify that all but the 100001st record + // has sequence number zero. The 100001st record + // is at the tip of this snapshot and cannot + // be zeroed out. + int count = 0; + int total = 0; + Arena arena; + { + InternalKeyComparator icmp(options.comparator); + RangeDelAggregator range_del_agg(icmp, {} /* snapshots */); + ScopedArenaIterator iter( + dbfull()->NewInternalIterator(&arena, &range_del_agg, handles_[1])); + iter->SeekToFirst(); + ASSERT_OK(iter->status()); + while (iter->Valid()) { + ParsedInternalKey ikey(Slice(), 0, kTypeValue); + ikey.sequence = -1; + ASSERT_EQ(ParseInternalKey(iter->key(), &ikey), true); + total++; + if (ikey.sequence != 0) { + count++; + } + iter->Next(); + } + } + ASSERT_EQ(total, 100000); + ASSERT_EQ(count, 1); + + // overwrite all the 100K keys once again. + for (int i = 0; i < 100000; i++) { + char key[100]; + snprintf(key, sizeof(key), "B%010d", i); + ASSERT_OK(Put(1, key, value)); + } + ASSERT_OK(Flush(1)); + + // push all files to the highest level L2. This + // means that all keys should pass at least once + // via the compaction filter + cfilter_count = 0; + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); + ASSERT_EQ(cfilter_count, 100000); + cfilter_count = 0; + dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); + ASSERT_EQ(cfilter_count, 100000); + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + ASSERT_EQ(NumTableFilesAtLevel(1, 1), 0); + ASSERT_NE(NumTableFilesAtLevel(2, 1), 0); + + // create a new database with the compaction + // filter in such a way that it deletes all keys + options.compaction_filter_factory = std::make_shared(); + options.create_if_missing = true; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // write all the keys once again. + for (int i = 0; i < 100000; i++) { + char key[100]; + snprintf(key, sizeof(key), "B%010d", i); + ASSERT_OK(Put(1, key, value)); + } + ASSERT_OK(Flush(1)); + ASSERT_NE(NumTableFilesAtLevel(0, 1), 0); + ASSERT_EQ(NumTableFilesAtLevel(1, 1), 0); + ASSERT_EQ(NumTableFilesAtLevel(2, 1), 0); + + // Push all files to the highest level L2. This + // triggers the compaction filter to delete all keys, + // verify that at the end of the compaction process, + // nothing is left. + cfilter_count = 0; + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); + ASSERT_EQ(cfilter_count, 100000); + cfilter_count = 0; + dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); + ASSERT_EQ(cfilter_count, 0); + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + ASSERT_EQ(NumTableFilesAtLevel(1, 1), 0); + + { + // Scan the entire database to ensure that nothing is left + std::unique_ptr iter( + db_->NewIterator(ReadOptions(), handles_[1])); + iter->SeekToFirst(); + count = 0; + while (iter->Valid()) { + count++; + iter->Next(); + } + ASSERT_EQ(count, 0); + } + + // The sequence number of the remaining record + // is not zeroed out even though it is at the + // level Lmax because this record is at the tip + count = 0; + { + InternalKeyComparator icmp(options.comparator); + RangeDelAggregator range_del_agg(icmp, {} /* snapshots */); + ScopedArenaIterator iter( + dbfull()->NewInternalIterator(&arena, &range_del_agg, handles_[1])); + iter->SeekToFirst(); + ASSERT_OK(iter->status()); + while (iter->Valid()) { + ParsedInternalKey ikey(Slice(), 0, kTypeValue); + ASSERT_EQ(ParseInternalKey(iter->key(), &ikey), true); + ASSERT_NE(ikey.sequence, (unsigned)0); + count++; + iter->Next(); + } + ASSERT_EQ(count, 0); + } +} + +// Tests the edge case where compaction does not produce any output -- all +// entries are deleted. The compaction should create bunch of 'DeleteFile' +// entries in VersionEdit, but none of the 'AddFile's. +TEST_F(DBTestCompactionFilter, CompactionFilterDeletesAll) { + Options options = CurrentOptions(); + options.compaction_filter_factory = std::make_shared(); + options.disable_auto_compactions = true; + options.create_if_missing = true; + DestroyAndReopen(options); + + // put some data + for (int table = 0; table < 4; ++table) { + for (int i = 0; i < 10 + table; ++i) { + Put(ToString(table * 100 + i), "val"); + } + Flush(); + } + + // this will produce empty file (delete compaction filter) + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ(0U, CountLiveFiles()); + + Reopen(options); + + Iterator* itr = db_->NewIterator(ReadOptions()); + itr->SeekToFirst(); + // empty db + ASSERT_TRUE(!itr->Valid()); + + delete itr; +} +#endif // ROCKSDB_LITE + +TEST_F(DBTestCompactionFilter, CompactionFilterWithValueChange) { + do { + Options options = CurrentOptions(); + options.num_levels = 3; + options.compaction_filter_factory = + std::make_shared(); + CreateAndReopenWithCF({"pikachu"}, options); + + // Write 100K+1 keys, these are written to a few files + // in L0. We do this so that the current snapshot points + // to the 100001 key.The compaction filter is not invoked + // on keys that are visible via a snapshot because we + // anyways cannot delete it. + const std::string value(10, 'x'); + for (int i = 0; i < 100001; i++) { + char key[100]; + snprintf(key, sizeof(key), "B%010d", i); + Put(1, key, value); + } + + // push all files to lower levels + ASSERT_OK(Flush(1)); + if (option_config_ != kUniversalCompactionMultiLevel && + option_config_ != kUniversalSubcompactions) { + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); + dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); + } else { + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + } + + // re-write all data again + for (int i = 0; i < 100001; i++) { + char key[100]; + snprintf(key, sizeof(key), "B%010d", i); + Put(1, key, value); + } + + // push all files to lower levels. This should + // invoke the compaction filter for all 100000 keys. + ASSERT_OK(Flush(1)); + if (option_config_ != kUniversalCompactionMultiLevel && + option_config_ != kUniversalSubcompactions) { + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); + dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); + } else { + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + } + + // verify that all keys now have the new value that + // was set by the compaction process. + for (int i = 0; i < 100001; i++) { + char key[100]; + snprintf(key, sizeof(key), "B%010d", i); + std::string newvalue = Get(1, key); + ASSERT_EQ(newvalue.compare(NEW_VALUE), 0); + } + } while (ChangeCompactOptions()); +} + +TEST_F(DBTestCompactionFilter, CompactionFilterWithMergeOperator) { + std::string one, two, three, four; + PutFixed64(&one, 1); + PutFixed64(&two, 2); + PutFixed64(&three, 3); + PutFixed64(&four, 4); + + Options options = CurrentOptions(); + options.create_if_missing = true; + options.merge_operator = MergeOperators::CreateUInt64AddOperator(); + options.num_levels = 3; + // Filter out keys with value is 2. + options.compaction_filter_factory = + std::make_shared(two); + DestroyAndReopen(options); + + // In the same compaction, a value type needs to be deleted based on + // compaction filter, and there is a merge type for the key. compaction + // filter result is ignored. + ASSERT_OK(db_->Put(WriteOptions(), "foo", two)); + ASSERT_OK(Flush()); + ASSERT_OK(db_->Merge(WriteOptions(), "foo", one)); + ASSERT_OK(Flush()); + std::string newvalue = Get("foo"); + ASSERT_EQ(newvalue, three); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + newvalue = Get("foo"); + ASSERT_EQ(newvalue, three); + + // value key can be deleted based on compaction filter, leaving only + // merge keys. + ASSERT_OK(db_->Put(WriteOptions(), "bar", two)); + ASSERT_OK(Flush()); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + newvalue = Get("bar"); + ASSERT_EQ("NOT_FOUND", newvalue); + ASSERT_OK(db_->Merge(WriteOptions(), "bar", two)); + ASSERT_OK(Flush()); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + newvalue = Get("bar"); + ASSERT_EQ(two, two); + + // Compaction filter never applies to merge keys. + ASSERT_OK(db_->Put(WriteOptions(), "foobar", one)); + ASSERT_OK(Flush()); + ASSERT_OK(db_->Merge(WriteOptions(), "foobar", two)); + ASSERT_OK(Flush()); + newvalue = Get("foobar"); + ASSERT_EQ(newvalue, three); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + newvalue = Get("foobar"); + ASSERT_EQ(newvalue, three); + + // In the same compaction, both of value type and merge type keys need to be + // deleted based on compaction filter, and there is a merge type for the key. + // For both keys, compaction filter results are ignored. + ASSERT_OK(db_->Put(WriteOptions(), "barfoo", two)); + ASSERT_OK(Flush()); + ASSERT_OK(db_->Merge(WriteOptions(), "barfoo", two)); + ASSERT_OK(Flush()); + newvalue = Get("barfoo"); + ASSERT_EQ(newvalue, four); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + newvalue = Get("barfoo"); + ASSERT_EQ(newvalue, four); +} + +#ifndef ROCKSDB_LITE +TEST_F(DBTestCompactionFilter, CompactionFilterContextManual) { + KeepFilterFactory* filter = new KeepFilterFactory(true, true); + + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.compaction_filter_factory.reset(filter); + options.compression = kNoCompression; + options.level0_file_num_compaction_trigger = 8; + Reopen(options); + int num_keys_per_file = 400; + for (int j = 0; j < 3; j++) { + // Write several keys. + const std::string value(10, 'x'); + for (int i = 0; i < num_keys_per_file; i++) { + char key[100]; + snprintf(key, sizeof(key), "B%08d%02d", i, j); + Put(key, value); + } + dbfull()->TEST_FlushMemTable(); + // Make sure next file is much smaller so automatic compaction will not + // be triggered. + num_keys_per_file /= 2; + } + dbfull()->TEST_WaitForCompact(); + + // Force a manual compaction + cfilter_count = 0; + filter->expect_manual_compaction_.store(true); + filter->expect_full_compaction_.store(true); + filter->expect_cf_id_.store(0); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ(cfilter_count, 700); + ASSERT_EQ(NumSortedRuns(0), 1); + ASSERT_TRUE(filter->compaction_filter_created()); + + // Verify total number of keys is correct after manual compaction. + { + int count = 0; + int total = 0; + Arena arena; + InternalKeyComparator icmp(options.comparator); + RangeDelAggregator range_del_agg(icmp, {} /* snapshots */); + ScopedArenaIterator iter( + dbfull()->NewInternalIterator(&arena, &range_del_agg)); + iter->SeekToFirst(); + ASSERT_OK(iter->status()); + while (iter->Valid()) { + ParsedInternalKey ikey(Slice(), 0, kTypeValue); + ikey.sequence = -1; + ASSERT_EQ(ParseInternalKey(iter->key(), &ikey), true); + total++; + if (ikey.sequence != 0) { + count++; + } + iter->Next(); + } + ASSERT_EQ(total, 700); + ASSERT_EQ(count, 1); + } +} +#endif // ROCKSDB_LITE + +TEST_F(DBTestCompactionFilter, CompactionFilterContextCfId) { + KeepFilterFactory* filter = new KeepFilterFactory(false, true); + filter->expect_cf_id_.store(1); + + Options options = CurrentOptions(); + options.compaction_filter_factory.reset(filter); + options.compression = kNoCompression; + options.level0_file_num_compaction_trigger = 2; + CreateAndReopenWithCF({"pikachu"}, options); + + int num_keys_per_file = 400; + for (int j = 0; j < 3; j++) { + // Write several keys. + const std::string value(10, 'x'); + for (int i = 0; i < num_keys_per_file; i++) { + char key[100]; + snprintf(key, sizeof(key), "B%08d%02d", i, j); + Put(1, key, value); + } + Flush(1); + // Make sure next file is much smaller so automatic compaction will not + // be triggered. + num_keys_per_file /= 2; + } + dbfull()->TEST_WaitForCompact(); + + ASSERT_TRUE(filter->compaction_filter_created()); +} + +#ifndef ROCKSDB_LITE +// Compaction filters should only be applied to records that are newer than the +// latest snapshot. This test inserts records and applies a delete filter. +TEST_F(DBTestCompactionFilter, CompactionFilterSnapshot) { + Options options = CurrentOptions(); + options.compaction_filter_factory = std::make_shared(); + options.disable_auto_compactions = true; + options.create_if_missing = true; + DestroyAndReopen(options); + + // Put some data. + const Snapshot* snapshot = nullptr; + for (int table = 0; table < 4; ++table) { + for (int i = 0; i < 10; ++i) { + Put(ToString(table * 100 + i), "val"); + } + Flush(); + + if (table == 0) { + snapshot = db_->GetSnapshot(); + } + } + assert(snapshot != nullptr); + + cfilter_count = 0; + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + // The filter should delete 10 records. + ASSERT_EQ(30U, cfilter_count); + + // Release the snapshot and compact again -> now all records should be + // removed. + db_->ReleaseSnapshot(snapshot); + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ(0U, CountLiveFiles()); +} + +// Compaction filters should only be applied to records that are newer than the +// latest snapshot. However, if the compaction filter asks to ignore snapshots +// records newer than the snapshot will also be processed +TEST_F(DBTestCompactionFilter, CompactionFilterIgnoreSnapshot) { + std::string five = ToString(5); + Options options = CurrentOptions(); + options.compaction_filter_factory = std::make_shared(); + options.disable_auto_compactions = true; + options.create_if_missing = true; + DestroyAndReopen(options); + + // Put some data. + const Snapshot* snapshot = nullptr; + for (int table = 0; table < 4; ++table) { + for (int i = 0; i < 10; ++i) { + Put(ToString(table * 100 + i), "val"); + } + Flush(); + + if (table == 0) { + snapshot = db_->GetSnapshot(); + } + } + assert(snapshot != nullptr); + + cfilter_count = 0; + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + // The filter should delete 40 records. + ASSERT_EQ(40U, cfilter_count); + + { + // Scan the entire database as of the snapshot to ensure + // that nothing is left + ReadOptions read_options; + read_options.snapshot = snapshot; + std::unique_ptr iter(db_->NewIterator(read_options)); + iter->SeekToFirst(); + int count = 0; + while (iter->Valid()) { + count++; + iter->Next(); + } + ASSERT_EQ(count, 6); + read_options.snapshot = 0; + std::unique_ptr iter1(db_->NewIterator(read_options)); + iter1->SeekToFirst(); + count = 0; + while (iter1->Valid()) { + count++; + iter1->Next(); + } + // We have deleted 10 keys from 40 using the compaction filter + // Keys 6-9 before the snapshot and 100-105 after the snapshot + ASSERT_EQ(count, 30); + } + + // Release the snapshot and compact again -> now all records should be + // removed. + db_->ReleaseSnapshot(snapshot); +} +#endif // ROCKSDB_LITE + +TEST_F(DBTestCompactionFilter, SkipUntil) { + Options options = CurrentOptions(); + options.compaction_filter_factory = std::make_shared(); + options.disable_auto_compactions = true; + options.create_if_missing = true; + DestroyAndReopen(options); + + // Write 100K keys, these are written to a few files in L0. + for (int table = 0; table < 4; ++table) { + // Key ranges in tables are [0, 38], [106, 149], [212, 260], [318, 371]. + for (int i = table * 6; i < 39 + table * 11; ++i) { + char key[100]; + snprintf(key, sizeof(key), "%010d", table * 100 + i); + Put(key, std::to_string(table * 1000 + i)); + } + Flush(); + } + + cfilter_skips = 0; + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + // Number of skips in tables: 2, 3, 3, 3. + ASSERT_EQ(11, cfilter_skips); + + for (int table = 0; table < 4; ++table) { + for (int i = table * 6; i < 39 + table * 11; ++i) { + int k = table * 100 + i; + char key[100]; + snprintf(key, sizeof(key), "%010d", table * 100 + i); + auto expected = std::to_string(table * 1000 + i); + std::string val; + Status s = db_->Get(ReadOptions(), key, &val); + if (k / 10 % 2 == 0) { + ASSERT_TRUE(s.IsNotFound()); + } else { + ASSERT_OK(s); + ASSERT_EQ(expected, val); + } + } + } +} + +TEST_F(DBTestCompactionFilter, SkipUntilWithBloomFilter) { + BlockBasedTableOptions table_options; + table_options.whole_key_filtering = false; + table_options.filter_policy.reset(NewBloomFilterPolicy(100, false)); + + Options options = CurrentOptions(); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.prefix_extractor.reset(NewCappedPrefixTransform(9)); + options.compaction_filter_factory = std::make_shared(); + options.disable_auto_compactions = true; + options.create_if_missing = true; + DestroyAndReopen(options); + + Put("0000000010", "v10"); + Put("0000000020", "v20"); // skipped + Put("0000000050", "v50"); + Flush(); + + cfilter_skips = 0; + EXPECT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + EXPECT_EQ(1, cfilter_skips); + + Status s; + std::string val; + + s = db_->Get(ReadOptions(), "0000000010", &val); + ASSERT_OK(s); + EXPECT_EQ("v10", val); + + s = db_->Get(ReadOptions(), "0000000020", &val); + EXPECT_TRUE(s.IsNotFound()); + + s = db_->Get(ReadOptions(), "0000000050", &val); + ASSERT_OK(s); + EXPECT_EQ("v50", val); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_compaction_test.cc b/db/db_compaction_test.cc new file mode 100644 index 00000000000..ca77d5b939f --- /dev/null +++ b/db/db_compaction_test.cc @@ -0,0 +1,2835 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_test_util.h" +#include "port/stack_trace.h" +#include "port/port.h" +#include "rocksdb/experimental.h" +#include "rocksdb/utilities/convenience.h" +#include "util/sync_point.h" +namespace rocksdb { + +// SYNC_POINT is not supported in released Windows mode. +#if !defined(ROCKSDB_LITE) + +class DBCompactionTest : public DBTestBase { + public: + DBCompactionTest() : DBTestBase("/db_compaction_test") {} +}; + +class DBCompactionTestWithParam + : public DBTestBase, + public testing::WithParamInterface> { + public: + DBCompactionTestWithParam() : DBTestBase("/db_compaction_test") { + max_subcompactions_ = std::get<0>(GetParam()); + exclusive_manual_compaction_ = std::get<1>(GetParam()); + } + + // Required if inheriting from testing::WithParamInterface<> + static void SetUpTestCase() {} + static void TearDownTestCase() {} + + uint32_t max_subcompactions_; + bool exclusive_manual_compaction_; +}; + +class DBCompactionDirectIOTest : public DBCompactionTest, + public ::testing::WithParamInterface { + public: + DBCompactionDirectIOTest() : DBCompactionTest() {} +}; + +namespace { + +class FlushedFileCollector : public EventListener { + public: + FlushedFileCollector() {} + ~FlushedFileCollector() {} + + virtual void OnFlushCompleted(DB* db, const FlushJobInfo& info) override { + std::lock_guard lock(mutex_); + flushed_files_.push_back(info.file_path); + } + + std::vector GetFlushedFiles() { + std::lock_guard lock(mutex_); + std::vector result; + for (auto fname : flushed_files_) { + result.push_back(fname); + } + return result; + } + + void ClearFlushedFiles() { flushed_files_.clear(); } + + private: + std::vector flushed_files_; + std::mutex mutex_; +}; + +static const int kCDTValueSize = 1000; +static const int kCDTKeysPerBuffer = 4; +static const int kCDTNumLevels = 8; +Options DeletionTriggerOptions(Options options) { + options.compression = kNoCompression; + options.write_buffer_size = kCDTKeysPerBuffer * (kCDTValueSize + 24); + options.min_write_buffer_number_to_merge = 1; + options.max_write_buffer_number_to_maintain = 0; + options.num_levels = kCDTNumLevels; + options.level0_file_num_compaction_trigger = 1; + options.target_file_size_base = options.write_buffer_size * 2; + options.target_file_size_multiplier = 2; + options.max_bytes_for_level_base = + options.target_file_size_base * options.target_file_size_multiplier; + options.max_bytes_for_level_multiplier = 2; + options.disable_auto_compactions = false; + return options; +} + +bool HaveOverlappingKeyRanges( + const Comparator* c, + const SstFileMetaData& a, const SstFileMetaData& b) { + if (c->Compare(a.smallestkey, b.smallestkey) >= 0) { + if (c->Compare(a.smallestkey, b.largestkey) <= 0) { + // b.smallestkey <= a.smallestkey <= b.largestkey + return true; + } + } else if (c->Compare(a.largestkey, b.smallestkey) >= 0) { + // a.smallestkey < b.smallestkey <= a.largestkey + return true; + } + if (c->Compare(a.largestkey, b.largestkey) <= 0) { + if (c->Compare(a.largestkey, b.smallestkey) >= 0) { + // b.smallestkey <= a.largestkey <= b.largestkey + return true; + } + } else if (c->Compare(a.smallestkey, b.largestkey) <= 0) { + // a.smallestkey <= b.largestkey < a.largestkey + return true; + } + return false; +} + +// Identifies all files between level "min_level" and "max_level" +// which has overlapping key range with "input_file_meta". +void GetOverlappingFileNumbersForLevelCompaction( + const ColumnFamilyMetaData& cf_meta, + const Comparator* comparator, + int min_level, int max_level, + const SstFileMetaData* input_file_meta, + std::set* overlapping_file_names) { + std::set overlapping_files; + overlapping_files.insert(input_file_meta); + for (int m = min_level; m <= max_level; ++m) { + for (auto& file : cf_meta.levels[m].files) { + for (auto* included_file : overlapping_files) { + if (HaveOverlappingKeyRanges( + comparator, *included_file, file)) { + overlapping_files.insert(&file); + overlapping_file_names->insert(file.name); + break; + } + } + } + } +} + +void VerifyCompactionResult( + const ColumnFamilyMetaData& cf_meta, + const std::set& overlapping_file_numbers) { +#ifndef NDEBUG + for (auto& level : cf_meta.levels) { + for (auto& file : level.files) { + assert(overlapping_file_numbers.find(file.name) == + overlapping_file_numbers.end()); + } + } +#endif +} + +const SstFileMetaData* PickFileRandomly( + const ColumnFamilyMetaData& cf_meta, + Random* rand, + int* level = nullptr) { + auto file_id = rand->Uniform(static_cast( + cf_meta.file_count)) + 1; + for (auto& level_meta : cf_meta.levels) { + if (file_id <= level_meta.files.size()) { + if (level != nullptr) { + *level = level_meta.level; + } + auto result = rand->Uniform(file_id); + return &(level_meta.files[result]); + } + file_id -= static_cast(level_meta.files.size()); + } + assert(false); + return nullptr; +} +} // anonymous namespace + +// All the TEST_P tests run once with sub_compactions disabled (i.e. +// options.max_subcompactions = 1) and once with it enabled +TEST_P(DBCompactionTestWithParam, CompactionDeletionTrigger) { + for (int tid = 0; tid < 3; ++tid) { + uint64_t db_size[2]; + Options options = DeletionTriggerOptions(CurrentOptions()); + options.max_subcompactions = max_subcompactions_; + + if (tid == 1) { + // the following only disable stats update in DB::Open() + // and should not affect the result of this test. + options.skip_stats_update_on_db_open = true; + } else if (tid == 2) { + // third pass with universal compaction + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 1; + } + + DestroyAndReopen(options); + Random rnd(301); + + const int kTestSize = kCDTKeysPerBuffer * 1024; + std::vector values; + for (int k = 0; k < kTestSize; ++k) { + values.push_back(RandomString(&rnd, kCDTValueSize)); + ASSERT_OK(Put(Key(k), values[k])); + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + db_size[0] = Size(Key(0), Key(kTestSize - 1)); + + for (int k = 0; k < kTestSize; ++k) { + ASSERT_OK(Delete(Key(k))); + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + db_size[1] = Size(Key(0), Key(kTestSize - 1)); + + // must have much smaller db size. + ASSERT_GT(db_size[0] / 3, db_size[1]); + } +} + +TEST_F(DBCompactionTest, SkipStatsUpdateTest) { + // This test verify UpdateAccumulatedStats is not on + // if options.skip_stats_update_on_db_open = true + // The test will need to be updated if the internal behavior changes. + + Options options = DeletionTriggerOptions(CurrentOptions()); + options.env = env_; + DestroyAndReopen(options); + Random rnd(301); + + const int kTestSize = kCDTKeysPerBuffer * 512; + std::vector values; + for (int k = 0; k < kTestSize; ++k) { + values.push_back(RandomString(&rnd, kCDTValueSize)); + ASSERT_OK(Put(Key(k), values[k])); + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + + // Reopen the DB with stats-update disabled + options.skip_stats_update_on_db_open = true; + env_->random_file_open_counter_.store(0); + Reopen(options); + + // As stats-update is disabled, we expect a very low number of + // random file open. + // Note that this number must be changed accordingly if we change + // the number of files needed to be opened in the DB::Open process. + const int kMaxFileOpenCount = 10; + ASSERT_LT(env_->random_file_open_counter_.load(), kMaxFileOpenCount); + + // Repeat the reopen process, but this time we enable + // stats-update. + options.skip_stats_update_on_db_open = false; + env_->random_file_open_counter_.store(0); + Reopen(options); + + // Since we do a normal stats update on db-open, there + // will be more random open files. + ASSERT_GT(env_->random_file_open_counter_.load(), kMaxFileOpenCount); +} + +TEST_F(DBCompactionTest, TestTableReaderForCompaction) { + Options options = CurrentOptions(); + options.env = env_; + options.new_table_reader_for_compaction_inputs = true; + options.max_open_files = 100; + options.level0_file_num_compaction_trigger = 3; + DestroyAndReopen(options); + Random rnd(301); + + int num_table_cache_lookup = 0; + int num_new_table_reader = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "TableCache::FindTable:0", [&](void* arg) { + assert(arg != nullptr); + bool no_io = *(reinterpret_cast(arg)); + if (!no_io) { + // filter out cases for table properties queries. + num_table_cache_lookup++; + } + }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "TableCache::GetTableReader:0", + [&](void* arg) { num_new_table_reader++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + for (int k = 0; k < options.level0_file_num_compaction_trigger; ++k) { + ASSERT_OK(Put(Key(k), Key(k))); + ASSERT_OK(Put(Key(10 - k), "bar")); + if (k < options.level0_file_num_compaction_trigger - 1) { + num_table_cache_lookup = 0; + Flush(); + dbfull()->TEST_WaitForCompact(); + // preloading iterator issues one table cache lookup and create + // a new table reader. + ASSERT_EQ(num_table_cache_lookup, 1); + ASSERT_EQ(num_new_table_reader, 1); + + num_table_cache_lookup = 0; + num_new_table_reader = 0; + ASSERT_EQ(Key(k), Get(Key(k))); + // lookup iterator from table cache and no need to create a new one. + ASSERT_EQ(num_table_cache_lookup, 1); + ASSERT_EQ(num_new_table_reader, 0); + } + } + + num_table_cache_lookup = 0; + num_new_table_reader = 0; + Flush(); + dbfull()->TEST_WaitForCompact(); + // Preloading iterator issues one table cache lookup and creates + // a new table reader. One file is created for flush and one for compaction. + // Compaction inputs make no table cache look-up for data/range deletion + // iterators + ASSERT_EQ(num_table_cache_lookup, 2); + // Create new iterator for: + // (1) 1 for verifying flush results + // (2) 3 for compaction input files + // (3) 1 for verifying compaction results. + ASSERT_EQ(num_new_table_reader, 5); + + num_table_cache_lookup = 0; + num_new_table_reader = 0; + ASSERT_EQ(Key(1), Get(Key(1))); + ASSERT_EQ(num_table_cache_lookup, 1); + ASSERT_EQ(num_new_table_reader, 0); + + num_table_cache_lookup = 0; + num_new_table_reader = 0; + CompactRangeOptions cro; + cro.change_level = true; + cro.target_level = 2; + cro.bottommost_level_compaction = BottommostLevelCompaction::kForce; + db_->CompactRange(cro, nullptr, nullptr); + // Only verifying compaction outputs issues one table cache lookup + // for both data block and range deletion block). + ASSERT_EQ(num_table_cache_lookup, 1); + // One for compaction input, one for verifying compaction results. + ASSERT_EQ(num_new_table_reader, 2); + + num_table_cache_lookup = 0; + num_new_table_reader = 0; + ASSERT_EQ(Key(1), Get(Key(1))); + ASSERT_EQ(num_table_cache_lookup, 1); + ASSERT_EQ(num_new_table_reader, 0); + + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); +} + +TEST_P(DBCompactionTestWithParam, CompactionDeletionTriggerReopen) { + for (int tid = 0; tid < 2; ++tid) { + uint64_t db_size[3]; + Options options = DeletionTriggerOptions(CurrentOptions()); + options.max_subcompactions = max_subcompactions_; + + if (tid == 1) { + // second pass with universal compaction + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 1; + } + + DestroyAndReopen(options); + Random rnd(301); + + // round 1 --- insert key/value pairs. + const int kTestSize = kCDTKeysPerBuffer * 512; + std::vector values; + for (int k = 0; k < kTestSize; ++k) { + values.push_back(RandomString(&rnd, kCDTValueSize)); + ASSERT_OK(Put(Key(k), values[k])); + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + db_size[0] = Size(Key(0), Key(kTestSize - 1)); + Close(); + + // round 2 --- disable auto-compactions and issue deletions. + options.create_if_missing = false; + options.disable_auto_compactions = true; + Reopen(options); + + for (int k = 0; k < kTestSize; ++k) { + ASSERT_OK(Delete(Key(k))); + } + db_size[1] = Size(Key(0), Key(kTestSize - 1)); + Close(); + // as auto_compaction is off, we shouldn't see too much reduce + // in db size. + ASSERT_LT(db_size[0] / 3, db_size[1]); + + // round 3 --- reopen db with auto_compaction on and see if + // deletion compensation still work. + options.disable_auto_compactions = false; + Reopen(options); + // insert relatively small amount of data to trigger auto compaction. + for (int k = 0; k < kTestSize / 10; ++k) { + ASSERT_OK(Put(Key(k), values[k])); + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + db_size[2] = Size(Key(0), Key(kTestSize - 1)); + // this time we're expecting significant drop in size. + ASSERT_GT(db_size[0] / 3, db_size[2]); + } +} + +TEST_F(DBCompactionTest, DisableStatsUpdateReopen) { + uint64_t db_size[3]; + for (int test = 0; test < 2; ++test) { + Options options = DeletionTriggerOptions(CurrentOptions()); + options.skip_stats_update_on_db_open = (test == 0); + + env_->random_read_counter_.Reset(); + DestroyAndReopen(options); + Random rnd(301); + + // round 1 --- insert key/value pairs. + const int kTestSize = kCDTKeysPerBuffer * 512; + std::vector values; + for (int k = 0; k < kTestSize; ++k) { + values.push_back(RandomString(&rnd, kCDTValueSize)); + ASSERT_OK(Put(Key(k), values[k])); + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + db_size[0] = Size(Key(0), Key(kTestSize - 1)); + Close(); + + // round 2 --- disable auto-compactions and issue deletions. + options.create_if_missing = false; + options.disable_auto_compactions = true; + + env_->random_read_counter_.Reset(); + Reopen(options); + + for (int k = 0; k < kTestSize; ++k) { + ASSERT_OK(Delete(Key(k))); + } + db_size[1] = Size(Key(0), Key(kTestSize - 1)); + Close(); + // as auto_compaction is off, we shouldn't see too much reduce + // in db size. + ASSERT_LT(db_size[0] / 3, db_size[1]); + + // round 3 --- reopen db with auto_compaction on and see if + // deletion compensation still work. + options.disable_auto_compactions = false; + Reopen(options); + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + db_size[2] = Size(Key(0), Key(kTestSize - 1)); + + if (options.skip_stats_update_on_db_open) { + // If update stats on DB::Open is disable, we don't expect + // deletion entries taking effect. + ASSERT_LT(db_size[0] / 3, db_size[2]); + } else { + // Otherwise, we should see a significant drop in db size. + ASSERT_GT(db_size[0] / 3, db_size[2]); + } + } +} + + +TEST_P(DBCompactionTestWithParam, CompactionTrigger) { + const int kNumKeysPerFile = 100; + + Options options = CurrentOptions(); + options.write_buffer_size = 110 << 10; // 110KB + options.arena_block_size = 4 << 10; + options.num_levels = 3; + options.level0_file_num_compaction_trigger = 3; + options.max_subcompactions = max_subcompactions_; + options.memtable_factory.reset(new SpecialSkipListFactory(kNumKeysPerFile)); + CreateAndReopenWithCF({"pikachu"}, options); + + Random rnd(301); + + for (int num = 0; num < options.level0_file_num_compaction_trigger - 1; + num++) { + std::vector values; + // Write 100KB (100 values, each 1K) + for (int i = 0; i < kNumKeysPerFile; i++) { + values.push_back(RandomString(&rnd, 990)); + ASSERT_OK(Put(1, Key(i), values[i])); + } + // put extra key to trigger flush + ASSERT_OK(Put(1, "", "")); + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + ASSERT_EQ(NumTableFilesAtLevel(0, 1), num + 1); + } + + // generate one more file in level-0, and should trigger level-0 compaction + std::vector values; + for (int i = 0; i < kNumKeysPerFile; i++) { + values.push_back(RandomString(&rnd, 990)); + ASSERT_OK(Put(1, Key(i), values[i])); + } + // put extra key to trigger flush + ASSERT_OK(Put(1, "", "")); + dbfull()->TEST_WaitForCompact(); + + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + ASSERT_EQ(NumTableFilesAtLevel(1, 1), 1); +} + +TEST_F(DBCompactionTest, BGCompactionsAllowed) { + // Create several column families. Make compaction triggers in all of them + // and see number of compactions scheduled to be less than allowed. + const int kNumKeysPerFile = 100; + + Options options = CurrentOptions(); + options.write_buffer_size = 110 << 10; // 110KB + options.arena_block_size = 4 << 10; + options.num_levels = 3; + // Should speed up compaction when there are 4 files. + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 20; + options.soft_pending_compaction_bytes_limit = 1 << 30; // Infinitely large + options.max_background_compactions = 3; + options.memtable_factory.reset(new SpecialSkipListFactory(kNumKeysPerFile)); + + // Block all threads in thread pool. + const size_t kTotalTasks = 4; + env_->SetBackgroundThreads(4, Env::LOW); + test::SleepingBackgroundTask sleeping_tasks[kTotalTasks]; + for (size_t i = 0; i < kTotalTasks; i++) { + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, + &sleeping_tasks[i], Env::Priority::LOW); + sleeping_tasks[i].WaitUntilSleeping(); + } + + CreateAndReopenWithCF({"one", "two", "three"}, options); + + Random rnd(301); + for (int cf = 0; cf < 4; cf++) { + for (int num = 0; num < options.level0_file_num_compaction_trigger; num++) { + for (int i = 0; i < kNumKeysPerFile; i++) { + ASSERT_OK(Put(cf, Key(i), "")); + } + // put extra key to trigger flush + ASSERT_OK(Put(cf, "", "")); + dbfull()->TEST_WaitForFlushMemTable(handles_[cf]); + ASSERT_EQ(NumTableFilesAtLevel(0, cf), num + 1); + } + } + + // Now all column families qualify compaction but only one should be + // scheduled, because no column family hits speed up condition. + ASSERT_EQ(1, env_->GetThreadPoolQueueLen(Env::Priority::LOW)); + + // Create two more files for one column family, which triggers speed up + // condition, three compactions will be scheduled. + for (int num = 0; num < options.level0_file_num_compaction_trigger; num++) { + for (int i = 0; i < kNumKeysPerFile; i++) { + ASSERT_OK(Put(2, Key(i), "")); + } + // put extra key to trigger flush + ASSERT_OK(Put(2, "", "")); + dbfull()->TEST_WaitForFlushMemTable(handles_[2]); + ASSERT_EQ(options.level0_file_num_compaction_trigger + num + 1, + NumTableFilesAtLevel(0, 2)); + } + ASSERT_EQ(3, env_->GetThreadPoolQueueLen(Env::Priority::LOW)); + + // Unblock all threads to unblock all compactions. + for (size_t i = 0; i < kTotalTasks; i++) { + sleeping_tasks[i].WakeUp(); + sleeping_tasks[i].WaitUntilDone(); + } + dbfull()->TEST_WaitForCompact(); + + // Verify number of compactions allowed will come back to 1. + + for (size_t i = 0; i < kTotalTasks; i++) { + sleeping_tasks[i].Reset(); + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, + &sleeping_tasks[i], Env::Priority::LOW); + sleeping_tasks[i].WaitUntilSleeping(); + } + for (int cf = 0; cf < 4; cf++) { + for (int num = 0; num < options.level0_file_num_compaction_trigger; num++) { + for (int i = 0; i < kNumKeysPerFile; i++) { + ASSERT_OK(Put(cf, Key(i), "")); + } + // put extra key to trigger flush + ASSERT_OK(Put(cf, "", "")); + dbfull()->TEST_WaitForFlushMemTable(handles_[cf]); + ASSERT_EQ(NumTableFilesAtLevel(0, cf), num + 1); + } + } + + // Now all column families qualify compaction but only one should be + // scheduled, because no column family hits speed up condition. + ASSERT_EQ(1, env_->GetThreadPoolQueueLen(Env::Priority::LOW)); + + for (size_t i = 0; i < kTotalTasks; i++) { + sleeping_tasks[i].WakeUp(); + sleeping_tasks[i].WaitUntilDone(); + } +} + +TEST_P(DBCompactionTestWithParam, CompactionsGenerateMultipleFiles) { + Options options = CurrentOptions(); + options.write_buffer_size = 100000000; // Large write buffer + options.max_subcompactions = max_subcompactions_; + CreateAndReopenWithCF({"pikachu"}, options); + + Random rnd(301); + + // Write 8MB (80 values, each 100K) + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + std::vector values; + for (int i = 0; i < 80; i++) { + values.push_back(RandomString(&rnd, 100000)); + ASSERT_OK(Put(1, Key(i), values[i])); + } + + // Reopening moves updates to level-0 + ReopenWithColumnFamilies({"default", "pikachu"}, options); + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1], + true /* disallow trivial move */); + + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + ASSERT_GT(NumTableFilesAtLevel(1, 1), 1); + for (int i = 0; i < 80; i++) { + ASSERT_EQ(Get(1, Key(i)), values[i]); + } +} + +TEST_F(DBCompactionTest, MinorCompactionsHappen) { + do { + Options options = CurrentOptions(); + options.write_buffer_size = 10000; + CreateAndReopenWithCF({"pikachu"}, options); + + const int N = 500; + + int starting_num_tables = TotalTableFiles(1); + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(1, Key(i), Key(i) + std::string(1000, 'v'))); + } + int ending_num_tables = TotalTableFiles(1); + ASSERT_GT(ending_num_tables, starting_num_tables); + + for (int i = 0; i < N; i++) { + ASSERT_EQ(Key(i) + std::string(1000, 'v'), Get(1, Key(i))); + } + + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + for (int i = 0; i < N; i++) { + ASSERT_EQ(Key(i) + std::string(1000, 'v'), Get(1, Key(i))); + } + } while (ChangeCompactOptions()); +} + +TEST_F(DBCompactionTest, UserKeyCrossFile1) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleLevel; + options.level0_file_num_compaction_trigger = 3; + + DestroyAndReopen(options); + + // create first file and flush to l0 + Put("4", "A"); + Put("3", "A"); + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + + Put("2", "A"); + Delete("3"); + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ("NOT_FOUND", Get("3")); + + // move both files down to l1 + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ("NOT_FOUND", Get("3")); + + for (int i = 0; i < 3; i++) { + Put("2", "B"); + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + } + dbfull()->TEST_WaitForCompact(); + + ASSERT_EQ("NOT_FOUND", Get("3")); +} + +TEST_F(DBCompactionTest, UserKeyCrossFile2) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleLevel; + options.level0_file_num_compaction_trigger = 3; + + DestroyAndReopen(options); + + // create first file and flush to l0 + Put("4", "A"); + Put("3", "A"); + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + + Put("2", "A"); + SingleDelete("3"); + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ("NOT_FOUND", Get("3")); + + // move both files down to l1 + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ("NOT_FOUND", Get("3")); + + for (int i = 0; i < 3; i++) { + Put("2", "B"); + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + } + dbfull()->TEST_WaitForCompact(); + + ASSERT_EQ("NOT_FOUND", Get("3")); +} + +TEST_F(DBCompactionTest, ZeroSeqIdCompaction) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleLevel; + options.level0_file_num_compaction_trigger = 3; + + FlushedFileCollector* collector = new FlushedFileCollector(); + options.listeners.emplace_back(collector); + + // compaction options + CompactionOptions compact_opt; + compact_opt.compression = kNoCompression; + compact_opt.output_file_size_limit = 4096; + const size_t key_len = + static_cast(compact_opt.output_file_size_limit) / 5; + + DestroyAndReopen(options); + + std::vector snaps; + + // create first file and flush to l0 + for (auto& key : {"1", "2", "3", "3", "3", "3"}) { + Put(key, std::string(key_len, 'A')); + snaps.push_back(dbfull()->GetSnapshot()); + } + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + + // create second file and flush to l0 + for (auto& key : {"3", "4", "5", "6", "7", "8"}) { + Put(key, std::string(key_len, 'A')); + snaps.push_back(dbfull()->GetSnapshot()); + } + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + + // move both files down to l1 + dbfull()->CompactFiles(compact_opt, collector->GetFlushedFiles(), 1); + + // release snap so that first instance of key(3) can have seqId=0 + for (auto snap : snaps) { + dbfull()->ReleaseSnapshot(snap); + } + + // create 3 files in l0 so to trigger compaction + for (int i = 0; i < options.level0_file_num_compaction_trigger; i++) { + Put("2", std::string(1, 'A')); + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + } + + dbfull()->TEST_WaitForCompact(); + ASSERT_OK(Put("", "")); +} + +TEST_F(DBCompactionTest, ManualCompactionUnknownOutputSize) { + // github issue #2249 + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleLevel; + options.level0_file_num_compaction_trigger = 3; + DestroyAndReopen(options); + + // create two files in l1 that we can compact + for (int i = 0; i < 2; ++i) { + for (int j = 0; j < options.level0_file_num_compaction_trigger; j++) { + // make l0 files' ranges overlap to avoid trivial move + Put(std::to_string(2 * i), std::string(1, 'A')); + Put(std::to_string(2 * i + 1), std::string(1, 'A')); + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + } + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(NumTableFilesAtLevel(0, 0), 0); + ASSERT_EQ(NumTableFilesAtLevel(1, 0), i + 1); + } + + ColumnFamilyMetaData cf_meta; + dbfull()->GetColumnFamilyMetaData(dbfull()->DefaultColumnFamily(), &cf_meta); + ASSERT_EQ(2, cf_meta.levels[1].files.size()); + std::vector input_filenames; + for (const auto& sst_file : cf_meta.levels[1].files) { + input_filenames.push_back(sst_file.name); + } + + // note CompactionOptions::output_file_size_limit is unset. + CompactionOptions compact_opt; + compact_opt.compression = kNoCompression; + dbfull()->CompactFiles(compact_opt, input_filenames, 1); +} + +// Check that writes done during a memtable compaction are recovered +// if the database is shutdown during the memtable compaction. +TEST_F(DBCompactionTest, RecoverDuringMemtableCompaction) { + do { + Options options = CurrentOptions(); + options.env = env_; + CreateAndReopenWithCF({"pikachu"}, options); + + // Trigger a long memtable compaction and reopen the database during it + ASSERT_OK(Put(1, "foo", "v1")); // Goes to 1st log file + ASSERT_OK(Put(1, "big1", std::string(10000000, 'x'))); // Fills memtable + ASSERT_OK(Put(1, "big2", std::string(1000, 'y'))); // Triggers compaction + ASSERT_OK(Put(1, "bar", "v2")); // Goes to new log file + + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_EQ("v2", Get(1, "bar")); + ASSERT_EQ(std::string(10000000, 'x'), Get(1, "big1")); + ASSERT_EQ(std::string(1000, 'y'), Get(1, "big2")); + } while (ChangeOptions()); +} + +TEST_P(DBCompactionTestWithParam, TrivialMoveOneFile) { + int32_t trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.write_buffer_size = 100000000; + options.max_subcompactions = max_subcompactions_; + DestroyAndReopen(options); + + int32_t num_keys = 80; + int32_t value_size = 100 * 1024; // 100 KB + + Random rnd(301); + std::vector values; + for (int i = 0; i < num_keys; i++) { + values.push_back(RandomString(&rnd, value_size)); + ASSERT_OK(Put(Key(i), values[i])); + } + + // Reopening moves updates to L0 + Reopen(options); + ASSERT_EQ(NumTableFilesAtLevel(0, 0), 1); // 1 file in L0 + ASSERT_EQ(NumTableFilesAtLevel(1, 0), 0); // 0 files in L1 + + std::vector metadata; + db_->GetLiveFilesMetaData(&metadata); + ASSERT_EQ(metadata.size(), 1U); + LiveFileMetaData level0_file = metadata[0]; // L0 file meta + + CompactRangeOptions cro; + cro.exclusive_manual_compaction = exclusive_manual_compaction_; + + // Compaction will initiate a trivial move from L0 to L1 + dbfull()->CompactRange(cro, nullptr, nullptr); + + // File moved From L0 to L1 + ASSERT_EQ(NumTableFilesAtLevel(0, 0), 0); // 0 files in L0 + ASSERT_EQ(NumTableFilesAtLevel(1, 0), 1); // 1 file in L1 + + metadata.clear(); + db_->GetLiveFilesMetaData(&metadata); + ASSERT_EQ(metadata.size(), 1U); + ASSERT_EQ(metadata[0].name /* level1_file.name */, level0_file.name); + ASSERT_EQ(metadata[0].size /* level1_file.size */, level0_file.size); + + for (int i = 0; i < num_keys; i++) { + ASSERT_EQ(Get(Key(i)), values[i]); + } + + ASSERT_EQ(trivial_move, 1); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_P(DBCompactionTestWithParam, TrivialMoveNonOverlappingFiles) { + int32_t trivial_move = 0; + int32_t non_trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", + [&](void* arg) { non_trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.write_buffer_size = 10 * 1024 * 1024; + options.max_subcompactions = max_subcompactions_; + + DestroyAndReopen(options); + // non overlapping ranges + std::vector> ranges = { + {100, 199}, + {300, 399}, + {0, 99}, + {200, 299}, + {600, 699}, + {400, 499}, + {500, 550}, + {551, 599}, + }; + int32_t value_size = 10 * 1024; // 10 KB + + Random rnd(301); + std::map values; + for (size_t i = 0; i < ranges.size(); i++) { + for (int32_t j = ranges[i].first; j <= ranges[i].second; j++) { + values[j] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(j), values[j])); + } + ASSERT_OK(Flush()); + } + + int32_t level0_files = NumTableFilesAtLevel(0, 0); + ASSERT_EQ(level0_files, ranges.size()); // Multiple files in L0 + ASSERT_EQ(NumTableFilesAtLevel(1, 0), 0); // No files in L1 + + CompactRangeOptions cro; + cro.exclusive_manual_compaction = exclusive_manual_compaction_; + + // Since data is non-overlapping we expect compaction to initiate + // a trivial move + db_->CompactRange(cro, nullptr, nullptr); + // We expect that all the files were trivially moved from L0 to L1 + ASSERT_EQ(NumTableFilesAtLevel(0, 0), 0); + ASSERT_EQ(NumTableFilesAtLevel(1, 0) /* level1_files */, level0_files); + + for (size_t i = 0; i < ranges.size(); i++) { + for (int32_t j = ranges[i].first; j <= ranges[i].second; j++) { + ASSERT_EQ(Get(Key(j)), values[j]); + } + } + + ASSERT_EQ(trivial_move, 1); + ASSERT_EQ(non_trivial_move, 0); + + trivial_move = 0; + non_trivial_move = 0; + values.clear(); + DestroyAndReopen(options); + // Same ranges as above but overlapping + ranges = { + {100, 199}, + {300, 399}, + {0, 99}, + {200, 299}, + {600, 699}, + {400, 499}, + {500, 560}, // this range overlap with the next one + {551, 599}, + }; + for (size_t i = 0; i < ranges.size(); i++) { + for (int32_t j = ranges[i].first; j <= ranges[i].second; j++) { + values[j] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(j), values[j])); + } + ASSERT_OK(Flush()); + } + + db_->CompactRange(cro, nullptr, nullptr); + + for (size_t i = 0; i < ranges.size(); i++) { + for (int32_t j = ranges[i].first; j <= ranges[i].second; j++) { + ASSERT_EQ(Get(Key(j)), values[j]); + } + } + ASSERT_EQ(trivial_move, 0); + ASSERT_EQ(non_trivial_move, 1); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_P(DBCompactionTestWithParam, TrivialMoveTargetLevel) { + int32_t trivial_move = 0; + int32_t non_trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", + [&](void* arg) { non_trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.write_buffer_size = 10 * 1024 * 1024; + options.num_levels = 7; + options.max_subcompactions = max_subcompactions_; + + DestroyAndReopen(options); + int32_t value_size = 10 * 1024; // 10 KB + + // Add 2 non-overlapping files + Random rnd(301); + std::map values; + + // file 1 [0 => 300] + for (int32_t i = 0; i <= 300; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // file 2 [600 => 700] + for (int32_t i = 600; i <= 700; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // 2 files in L0 + ASSERT_EQ("2", FilesPerLevel(0)); + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 6; + compact_options.exclusive_manual_compaction = exclusive_manual_compaction_; + ASSERT_OK(db_->CompactRange(compact_options, nullptr, nullptr)); + // 2 files in L6 + ASSERT_EQ("0,0,0,0,0,0,2", FilesPerLevel(0)); + + ASSERT_EQ(trivial_move, 1); + ASSERT_EQ(non_trivial_move, 0); + + for (int32_t i = 0; i <= 300; i++) { + ASSERT_EQ(Get(Key(i)), values[i]); + } + for (int32_t i = 600; i <= 700; i++) { + ASSERT_EQ(Get(Key(i)), values[i]); + } +} + +TEST_P(DBCompactionTestWithParam, ManualCompactionPartial) { + int32_t trivial_move = 0; + int32_t non_trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", + [&](void* arg) { non_trivial_move++; }); + bool first = true; + // Purpose of dependencies: + // 4 -> 1: ensure the order of two non-trivial compactions + // 5 -> 2 and 5 -> 3: ensure we do a check before two non-trivial compactions + // are installed + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBCompaction::ManualPartial:4", "DBCompaction::ManualPartial:1"}, + {"DBCompaction::ManualPartial:5", "DBCompaction::ManualPartial:2"}, + {"DBCompaction::ManualPartial:5", "DBCompaction::ManualPartial:3"}}); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) { + if (first) { + first = false; + TEST_SYNC_POINT("DBCompaction::ManualPartial:4"); + TEST_SYNC_POINT("DBCompaction::ManualPartial:3"); + } else { // second non-trivial compaction + TEST_SYNC_POINT("DBCompaction::ManualPartial:2"); + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.write_buffer_size = 10 * 1024 * 1024; + options.num_levels = 7; + options.max_subcompactions = max_subcompactions_; + options.level0_file_num_compaction_trigger = 3; + options.max_background_compactions = 3; + options.target_file_size_base = 1 << 23; // 8 MB + + DestroyAndReopen(options); + int32_t value_size = 10 * 1024; // 10 KB + + // Add 2 non-overlapping files + Random rnd(301); + std::map values; + + // file 1 [0 => 100] + for (int32_t i = 0; i < 100; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // file 2 [100 => 300] + for (int32_t i = 100; i < 300; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // 2 files in L0 + ASSERT_EQ("2", FilesPerLevel(0)); + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 6; + compact_options.exclusive_manual_compaction = exclusive_manual_compaction_; + // Trivial move the two non-overlapping files to level 6 + ASSERT_OK(db_->CompactRange(compact_options, nullptr, nullptr)); + // 2 files in L6 + ASSERT_EQ("0,0,0,0,0,0,2", FilesPerLevel(0)); + + ASSERT_EQ(trivial_move, 1); + ASSERT_EQ(non_trivial_move, 0); + + // file 3 [ 0 => 200] + for (int32_t i = 0; i < 200; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // 1 files in L0 + ASSERT_EQ("1,0,0,0,0,0,2", FilesPerLevel(0)); + ASSERT_OK(dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, false)); + ASSERT_OK(dbfull()->TEST_CompactRange(1, nullptr, nullptr, nullptr, false)); + ASSERT_OK(dbfull()->TEST_CompactRange(2, nullptr, nullptr, nullptr, false)); + ASSERT_OK(dbfull()->TEST_CompactRange(3, nullptr, nullptr, nullptr, false)); + ASSERT_OK(dbfull()->TEST_CompactRange(4, nullptr, nullptr, nullptr, false)); + // 2 files in L6, 1 file in L5 + ASSERT_EQ("0,0,0,0,0,1,2", FilesPerLevel(0)); + + ASSERT_EQ(trivial_move, 6); + ASSERT_EQ(non_trivial_move, 0); + + rocksdb::port::Thread threads([&] { + compact_options.change_level = false; + compact_options.exclusive_manual_compaction = false; + std::string begin_string = Key(0); + std::string end_string = Key(199); + Slice begin(begin_string); + Slice end(end_string); + // First non-trivial compaction is triggered + ASSERT_OK(db_->CompactRange(compact_options, &begin, &end)); + }); + + TEST_SYNC_POINT("DBCompaction::ManualPartial:1"); + // file 4 [300 => 400) + for (int32_t i = 300; i <= 400; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // file 5 [400 => 500) + for (int32_t i = 400; i <= 500; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // file 6 [500 => 600) + for (int32_t i = 500; i <= 600; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + // Second non-trivial compaction is triggered + ASSERT_OK(Flush()); + + // Before two non-trivial compactions are installed, there are 3 files in L0 + ASSERT_EQ("3,0,0,0,0,1,2", FilesPerLevel(0)); + TEST_SYNC_POINT("DBCompaction::ManualPartial:5"); + + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + // After two non-trivial compactions are installed, there is 1 file in L6, and + // 1 file in L1 + ASSERT_EQ("0,1,0,0,0,0,1", FilesPerLevel(0)); + threads.join(); + + for (int32_t i = 0; i < 600; i++) { + ASSERT_EQ(Get(Key(i)), values[i]); + } +} + +// Disable as the test is flaky. +TEST_F(DBCompactionTest, DISABLED_ManualPartialFill) { + int32_t trivial_move = 0; + int32_t non_trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", + [&](void* arg) { non_trivial_move++; }); + bool first = true; + bool second = true; + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBCompaction::PartialFill:4", "DBCompaction::PartialFill:1"}, + {"DBCompaction::PartialFill:2", "DBCompaction::PartialFill:3"}}); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) { + if (first) { + TEST_SYNC_POINT("DBCompaction::PartialFill:4"); + first = false; + TEST_SYNC_POINT("DBCompaction::PartialFill:3"); + } else if (second) { + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.write_buffer_size = 10 * 1024 * 1024; + options.max_bytes_for_level_multiplier = 2; + options.num_levels = 4; + options.level0_file_num_compaction_trigger = 3; + options.max_background_compactions = 3; + + DestroyAndReopen(options); + // make sure all background compaction jobs can be scheduled + auto stop_token = + dbfull()->TEST_write_controler().GetCompactionPressureToken(); + int32_t value_size = 10 * 1024; // 10 KB + + // Add 2 non-overlapping files + Random rnd(301); + std::map values; + + // file 1 [0 => 100] + for (int32_t i = 0; i < 100; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // file 2 [100 => 300] + for (int32_t i = 100; i < 300; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // 2 files in L0 + ASSERT_EQ("2", FilesPerLevel(0)); + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 2; + ASSERT_OK(db_->CompactRange(compact_options, nullptr, nullptr)); + // 2 files in L2 + ASSERT_EQ("0,0,2", FilesPerLevel(0)); + + ASSERT_EQ(trivial_move, 1); + ASSERT_EQ(non_trivial_move, 0); + + // file 3 [ 0 => 200] + for (int32_t i = 0; i < 200; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // 2 files in L2, 1 in L0 + ASSERT_EQ("1,0,2", FilesPerLevel(0)); + ASSERT_OK(dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, false)); + // 2 files in L2, 1 in L1 + ASSERT_EQ("0,1,2", FilesPerLevel(0)); + + ASSERT_EQ(trivial_move, 2); + ASSERT_EQ(non_trivial_move, 0); + + rocksdb::port::Thread threads([&] { + compact_options.change_level = false; + compact_options.exclusive_manual_compaction = false; + std::string begin_string = Key(0); + std::string end_string = Key(199); + Slice begin(begin_string); + Slice end(end_string); + ASSERT_OK(db_->CompactRange(compact_options, &begin, &end)); + }); + + TEST_SYNC_POINT("DBCompaction::PartialFill:1"); + // Many files 4 [300 => 4300) + for (int32_t i = 0; i <= 5; i++) { + for (int32_t j = 300; j < 4300; j++) { + if (j == 2300) { + ASSERT_OK(Flush()); + dbfull()->TEST_WaitForFlushMemTable(); + } + values[j] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(j), values[j])); + } + } + + // Verify level sizes + uint64_t target_size = 4 * options.max_bytes_for_level_base; + for (int32_t i = 1; i < options.num_levels; i++) { + ASSERT_LE(SizeAtLevel(i), target_size); + target_size = static_cast(target_size * + options.max_bytes_for_level_multiplier); + } + + TEST_SYNC_POINT("DBCompaction::PartialFill:2"); + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + threads.join(); + + for (int32_t i = 0; i < 4300; i++) { + ASSERT_EQ(Get(Key(i)), values[i]); + } +} + +TEST_F(DBCompactionTest, DeleteFileRange) { + Options options = CurrentOptions(); + options.write_buffer_size = 10 * 1024 * 1024; + options.max_bytes_for_level_multiplier = 2; + options.num_levels = 4; + options.level0_file_num_compaction_trigger = 3; + options.max_background_compactions = 3; + + DestroyAndReopen(options); + int32_t value_size = 10 * 1024; // 10 KB + + // Add 2 non-overlapping files + Random rnd(301); + std::map values; + + // file 1 [0 => 100] + for (int32_t i = 0; i < 100; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // file 2 [100 => 300] + for (int32_t i = 100; i < 300; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // 2 files in L0 + ASSERT_EQ("2", FilesPerLevel(0)); + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 2; + ASSERT_OK(db_->CompactRange(compact_options, nullptr, nullptr)); + // 2 files in L2 + ASSERT_EQ("0,0,2", FilesPerLevel(0)); + + // file 3 [ 0 => 200] + for (int32_t i = 0; i < 200; i++) { + values[i] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + // Many files 4 [300 => 4300) + for (int32_t i = 0; i <= 5; i++) { + for (int32_t j = 300; j < 4300; j++) { + if (j == 2300) { + ASSERT_OK(Flush()); + dbfull()->TEST_WaitForFlushMemTable(); + } + values[j] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(j), values[j])); + } + } + ASSERT_OK(Flush()); + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + + // Verify level sizes + uint64_t target_size = 4 * options.max_bytes_for_level_base; + for (int32_t i = 1; i < options.num_levels; i++) { + ASSERT_LE(SizeAtLevel(i), target_size); + target_size = static_cast(target_size * + options.max_bytes_for_level_multiplier); + } + + size_t old_num_files = CountFiles(); + std::string begin_string = Key(1000); + std::string end_string = Key(2000); + Slice begin(begin_string); + Slice end(end_string); + ASSERT_OK(DeleteFilesInRange(db_, db_->DefaultColumnFamily(), &begin, &end)); + + int32_t deleted_count = 0; + for (int32_t i = 0; i < 4300; i++) { + if (i < 1000 || i > 2000) { + ASSERT_EQ(Get(Key(i)), values[i]); + } else { + ReadOptions roptions; + std::string result; + Status s = db_->Get(roptions, Key(i), &result); + ASSERT_TRUE(s.IsNotFound() || s.ok()); + if (s.IsNotFound()) { + deleted_count++; + } + } + } + ASSERT_GT(deleted_count, 0); + begin_string = Key(5000); + end_string = Key(6000); + Slice begin1(begin_string); + Slice end1(end_string); + // Try deleting files in range which contain no keys + ASSERT_OK( + DeleteFilesInRange(db_, db_->DefaultColumnFamily(), &begin1, &end1)); + + // Push data from level 0 to level 1 to force all data to be deleted + // Note that we don't delete level 0 files + compact_options.change_level = true; + compact_options.target_level = 1; + ASSERT_OK(dbfull()->TEST_CompactRange(0, nullptr, nullptr)); + + ASSERT_OK( + DeleteFilesInRange(db_, db_->DefaultColumnFamily(), nullptr, nullptr)); + + int32_t deleted_count2 = 0; + for (int32_t i = 0; i < 4300; i++) { + ReadOptions roptions; + std::string result; + Status s = db_->Get(roptions, Key(i), &result); + ASSERT_TRUE(s.IsNotFound()); + deleted_count2++; + } + ASSERT_GT(deleted_count2, deleted_count); + size_t new_num_files = CountFiles(); + ASSERT_GT(old_num_files, new_num_files); +} + +TEST_P(DBCompactionTestWithParam, TrivialMoveToLastLevelWithFiles) { + int32_t trivial_move = 0; + int32_t non_trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", + [&](void* arg) { non_trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.write_buffer_size = 100000000; + options.max_subcompactions = max_subcompactions_; + DestroyAndReopen(options); + + int32_t value_size = 10 * 1024; // 10 KB + + Random rnd(301); + std::vector values; + // File with keys [ 0 => 99 ] + for (int i = 0; i < 100; i++) { + values.push_back(RandomString(&rnd, value_size)); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + ASSERT_EQ("1", FilesPerLevel(0)); + // Compaction will do L0=>L1 (trivial move) then move L1 files to L3 + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 3; + compact_options.exclusive_manual_compaction = exclusive_manual_compaction_; + ASSERT_OK(db_->CompactRange(compact_options, nullptr, nullptr)); + ASSERT_EQ("0,0,0,1", FilesPerLevel(0)); + ASSERT_EQ(trivial_move, 1); + ASSERT_EQ(non_trivial_move, 0); + + // File with keys [ 100 => 199 ] + for (int i = 100; i < 200; i++) { + values.push_back(RandomString(&rnd, value_size)); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + ASSERT_EQ("1,0,0,1", FilesPerLevel(0)); + CompactRangeOptions cro; + cro.exclusive_manual_compaction = exclusive_manual_compaction_; + // Compaction will do L0=>L1 L1=>L2 L2=>L3 (3 trivial moves) + ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr)); + ASSERT_EQ("0,0,0,2", FilesPerLevel(0)); + ASSERT_EQ(trivial_move, 4); + ASSERT_EQ(non_trivial_move, 0); + + for (int i = 0; i < 200; i++) { + ASSERT_EQ(Get(Key(i)), values[i]); + } + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_P(DBCompactionTestWithParam, LevelCompactionThirdPath) { + Options options = CurrentOptions(); + options.db_paths.emplace_back(dbname_, 500 * 1024); + options.db_paths.emplace_back(dbname_ + "_2", 4 * 1024 * 1024); + options.db_paths.emplace_back(dbname_ + "_3", 1024 * 1024 * 1024); + options.memtable_factory.reset( + new SpecialSkipListFactory(KNumKeysByGenerateNewFile - 1)); + options.compaction_style = kCompactionStyleLevel; + options.write_buffer_size = 110 << 10; // 110KB + options.arena_block_size = 4 << 10; + options.level0_file_num_compaction_trigger = 2; + options.num_levels = 4; + options.max_bytes_for_level_base = 400 * 1024; + options.max_subcompactions = max_subcompactions_; + // options = CurrentOptions(options); + + std::vector filenames; + env_->GetChildren(options.db_paths[1].path, &filenames); + // Delete archival files. + for (size_t i = 0; i < filenames.size(); ++i) { + env_->DeleteFile(options.db_paths[1].path + "/" + filenames[i]); + } + env_->DeleteDir(options.db_paths[1].path); + Reopen(options); + + Random rnd(301); + int key_idx = 0; + + // First three 110KB files are not going to second path. + // After that, (100K, 200K) + for (int num = 0; num < 3; num++) { + GenerateNewFile(&rnd, &key_idx); + } + + // Another 110KB triggers a compaction to 400K file to fill up first path + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(3, GetSstFileCount(options.db_paths[1].path)); + + // (1, 4) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4", FilesPerLevel(0)); + ASSERT_EQ(4, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 4, 1) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,1", FilesPerLevel(0)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(4, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 4, 2) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,2", FilesPerLevel(0)); + ASSERT_EQ(2, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(4, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 4, 3) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,3", FilesPerLevel(0)); + ASSERT_EQ(3, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(4, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 4, 4) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,4", FilesPerLevel(0)); + ASSERT_EQ(4, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(4, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 4, 5) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,5", FilesPerLevel(0)); + ASSERT_EQ(5, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(4, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 4, 6) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,6", FilesPerLevel(0)); + ASSERT_EQ(6, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(4, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 4, 7) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,7", FilesPerLevel(0)); + ASSERT_EQ(7, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(4, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 4, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,8", FilesPerLevel(0)); + ASSERT_EQ(8, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(4, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + for (int i = 0; i < key_idx; i++) { + auto v = Get(Key(i)); + ASSERT_NE(v, "NOT_FOUND"); + ASSERT_TRUE(v.size() == 1 || v.size() == 990); + } + + Reopen(options); + + for (int i = 0; i < key_idx; i++) { + auto v = Get(Key(i)); + ASSERT_NE(v, "NOT_FOUND"); + ASSERT_TRUE(v.size() == 1 || v.size() == 990); + } + + Destroy(options); +} + +TEST_P(DBCompactionTestWithParam, LevelCompactionPathUse) { + Options options = CurrentOptions(); + options.db_paths.emplace_back(dbname_, 500 * 1024); + options.db_paths.emplace_back(dbname_ + "_2", 4 * 1024 * 1024); + options.db_paths.emplace_back(dbname_ + "_3", 1024 * 1024 * 1024); + options.memtable_factory.reset( + new SpecialSkipListFactory(KNumKeysByGenerateNewFile - 1)); + options.compaction_style = kCompactionStyleLevel; + options.write_buffer_size = 110 << 10; // 110KB + options.arena_block_size = 4 << 10; + options.level0_file_num_compaction_trigger = 2; + options.num_levels = 4; + options.max_bytes_for_level_base = 400 * 1024; + options.max_subcompactions = max_subcompactions_; + // options = CurrentOptions(options); + + std::vector filenames; + env_->GetChildren(options.db_paths[1].path, &filenames); + // Delete archival files. + for (size_t i = 0; i < filenames.size(); ++i) { + env_->DeleteFile(options.db_paths[1].path + "/" + filenames[i]); + } + env_->DeleteDir(options.db_paths[1].path); + Reopen(options); + + Random rnd(301); + int key_idx = 0; + + // Always gets compacted into 1 Level1 file, + // 0/1 Level 0 file + for (int num = 0; num < 3; num++) { + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + } + + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,1", FilesPerLevel(0)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("0,1", FilesPerLevel(0)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,1", FilesPerLevel(0)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("0,1", FilesPerLevel(0)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,1", FilesPerLevel(0)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("0,1", FilesPerLevel(0)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,1", FilesPerLevel(0)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("0,1", FilesPerLevel(0)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,1", FilesPerLevel(0)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + for (int i = 0; i < key_idx; i++) { + auto v = Get(Key(i)); + ASSERT_NE(v, "NOT_FOUND"); + ASSERT_TRUE(v.size() == 1 || v.size() == 990); + } + + Reopen(options); + + for (int i = 0; i < key_idx; i++) { + auto v = Get(Key(i)); + ASSERT_NE(v, "NOT_FOUND"); + ASSERT_TRUE(v.size() == 1 || v.size() == 990); + } + + Destroy(options); +} + +TEST_P(DBCompactionTestWithParam, ConvertCompactionStyle) { + Random rnd(301); + int max_key_level_insert = 200; + int max_key_universal_insert = 600; + + // Stage 1: generate a db with level compaction + Options options = CurrentOptions(); + options.write_buffer_size = 110 << 10; // 110KB + options.arena_block_size = 4 << 10; + options.num_levels = 4; + options.level0_file_num_compaction_trigger = 3; + options.max_bytes_for_level_base = 500 << 10; // 500KB + options.max_bytes_for_level_multiplier = 1; + options.target_file_size_base = 200 << 10; // 200KB + options.target_file_size_multiplier = 1; + options.max_subcompactions = max_subcompactions_; + CreateAndReopenWithCF({"pikachu"}, options); + + for (int i = 0; i <= max_key_level_insert; i++) { + // each value is 10K + ASSERT_OK(Put(1, Key(i), RandomString(&rnd, 10000))); + } + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + + ASSERT_GT(TotalTableFiles(1, 4), 1); + int non_level0_num_files = 0; + for (int i = 1; i < options.num_levels; i++) { + non_level0_num_files += NumTableFilesAtLevel(i, 1); + } + ASSERT_GT(non_level0_num_files, 0); + + // Stage 2: reopen with universal compaction - should fail + options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 1; + options = CurrentOptions(options); + Status s = TryReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_TRUE(s.IsInvalidArgument()); + + // Stage 3: compact into a single file and move the file to level 0 + options = CurrentOptions(); + options.disable_auto_compactions = true; + options.target_file_size_base = INT_MAX; + options.target_file_size_multiplier = 1; + options.max_bytes_for_level_base = INT_MAX; + options.max_bytes_for_level_multiplier = 1; + options.num_levels = 4; + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 0; + compact_options.bottommost_level_compaction = + BottommostLevelCompaction::kForce; + compact_options.exclusive_manual_compaction = exclusive_manual_compaction_; + dbfull()->CompactRange(compact_options, handles_[1], nullptr, nullptr); + + // Only 1 file in L0 + ASSERT_EQ("1", FilesPerLevel(1)); + + // Stage 4: re-open in universal compaction style and do some db operations + options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 4; + options.write_buffer_size = 110 << 10; // 110KB + options.arena_block_size = 4 << 10; + options.level0_file_num_compaction_trigger = 3; + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + options.num_levels = 1; + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + for (int i = max_key_level_insert / 2; i <= max_key_universal_insert; i++) { + ASSERT_OK(Put(1, Key(i), RandomString(&rnd, 10000))); + } + dbfull()->Flush(FlushOptions()); + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + + for (int i = 1; i < options.num_levels; i++) { + ASSERT_EQ(NumTableFilesAtLevel(i, 1), 0); + } + + // verify keys inserted in both level compaction style and universal + // compaction style + std::string keys_in_db; + Iterator* iter = dbfull()->NewIterator(ReadOptions(), handles_[1]); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + keys_in_db.append(iter->key().ToString()); + keys_in_db.push_back(','); + } + delete iter; + + std::string expected_keys; + for (int i = 0; i <= max_key_universal_insert; i++) { + expected_keys.append(Key(i)); + expected_keys.push_back(','); + } + + ASSERT_EQ(keys_in_db, expected_keys); +} + +TEST_F(DBCompactionTest, L0_CompactionBug_Issue44_a) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "b", "v")); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_OK(Delete(1, "b")); + ASSERT_OK(Delete(1, "a")); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_OK(Delete(1, "a")); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "a", "v")); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_EQ("(a->v)", Contents(1)); + env_->SleepForMicroseconds(1000000); // Wait for compaction to finish + ASSERT_EQ("(a->v)", Contents(1)); + } while (ChangeCompactOptions()); +} + +TEST_F(DBCompactionTest, L0_CompactionBug_Issue44_b) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + Put(1, "", ""); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + Delete(1, "e"); + Put(1, "", ""); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + Put(1, "c", "cv"); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + Put(1, "", ""); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + Put(1, "", ""); + env_->SleepForMicroseconds(1000000); // Wait for compaction to finish + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + Put(1, "d", "dv"); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + Put(1, "", ""); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + Delete(1, "d"); + Delete(1, "b"); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_EQ("(->)(c->cv)", Contents(1)); + env_->SleepForMicroseconds(1000000); // Wait for compaction to finish + ASSERT_EQ("(->)(c->cv)", Contents(1)); + } while (ChangeCompactOptions()); +} + +TEST_F(DBCompactionTest, ManualAutoRace) { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::BGWorkCompaction", "DBCompactionTest::ManualAutoRace:1"}, + {"DBImpl::RunManualCompaction:WaitScheduled", + "BackgroundCallCompaction:0"}}); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Put(1, "foo", ""); + Put(1, "bar", ""); + Flush(1); + Put(1, "foo", ""); + Put(1, "bar", ""); + // Generate four files in CF 0, which should trigger an auto compaction + Put("foo", ""); + Put("bar", ""); + Flush(); + Put("foo", ""); + Put("bar", ""); + Flush(); + Put("foo", ""); + Put("bar", ""); + Flush(); + Put("foo", ""); + Put("bar", ""); + Flush(); + + // The auto compaction is scheduled but waited until here + TEST_SYNC_POINT("DBCompactionTest::ManualAutoRace:1"); + // The auto compaction will wait until the manual compaction is registerd + // before processing so that it will be cancelled. + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, nullptr); + ASSERT_EQ("0,1", FilesPerLevel(1)); + + // Eventually the cancelled compaction will be rescheduled and executed. + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ("0,1", FilesPerLevel(0)); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_P(DBCompactionTestWithParam, ManualCompaction) { + Options options = CurrentOptions(); + options.max_subcompactions = max_subcompactions_; + options.statistics = rocksdb::CreateDBStatistics(); + CreateAndReopenWithCF({"pikachu"}, options); + + // iter - 0 with 7 levels + // iter - 1 with 3 levels + for (int iter = 0; iter < 2; ++iter) { + MakeTables(3, "p", "q", 1); + ASSERT_EQ("1,1,1", FilesPerLevel(1)); + + // Compaction range falls before files + Compact(1, "", "c"); + ASSERT_EQ("1,1,1", FilesPerLevel(1)); + + // Compaction range falls after files + Compact(1, "r", "z"); + ASSERT_EQ("1,1,1", FilesPerLevel(1)); + + // Compaction range overlaps files + Compact(1, "p1", "p9"); + ASSERT_EQ("0,0,1", FilesPerLevel(1)); + + // Populate a different range + MakeTables(3, "c", "e", 1); + ASSERT_EQ("1,1,2", FilesPerLevel(1)); + + // Compact just the new range + Compact(1, "b", "f"); + ASSERT_EQ("0,0,2", FilesPerLevel(1)); + + // Compact all + MakeTables(1, "a", "z", 1); + ASSERT_EQ("1,0,2", FilesPerLevel(1)); + + uint64_t prev_block_cache_add = + options.statistics->getTickerCount(BLOCK_CACHE_ADD); + CompactRangeOptions cro; + cro.exclusive_manual_compaction = exclusive_manual_compaction_; + db_->CompactRange(cro, handles_[1], nullptr, nullptr); + // Verify manual compaction doesn't fill block cache + ASSERT_EQ(prev_block_cache_add, + options.statistics->getTickerCount(BLOCK_CACHE_ADD)); + + ASSERT_EQ("0,0,1", FilesPerLevel(1)); + + if (iter == 0) { + options = CurrentOptions(); + options.num_levels = 3; + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + } + } +} + + +TEST_P(DBCompactionTestWithParam, ManualLevelCompactionOutputPathId) { + Options options = CurrentOptions(); + options.db_paths.emplace_back(dbname_ + "_2", 2 * 10485760); + options.db_paths.emplace_back(dbname_ + "_3", 100 * 10485760); + options.db_paths.emplace_back(dbname_ + "_4", 120 * 10485760); + options.max_subcompactions = max_subcompactions_; + CreateAndReopenWithCF({"pikachu"}, options); + + // iter - 0 with 7 levels + // iter - 1 with 3 levels + for (int iter = 0; iter < 2; ++iter) { + for (int i = 0; i < 3; ++i) { + ASSERT_OK(Put(1, "p", "begin")); + ASSERT_OK(Put(1, "q", "end")); + ASSERT_OK(Flush(1)); + } + ASSERT_EQ("3", FilesPerLevel(1)); + ASSERT_EQ(3, GetSstFileCount(options.db_paths[0].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + // Compaction range falls before files + Compact(1, "", "c"); + ASSERT_EQ("3", FilesPerLevel(1)); + + // Compaction range falls after files + Compact(1, "r", "z"); + ASSERT_EQ("3", FilesPerLevel(1)); + + // Compaction range overlaps files + Compact(1, "p1", "p9", 1); + ASSERT_EQ("0,1", FilesPerLevel(1)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[0].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + // Populate a different range + for (int i = 0; i < 3; ++i) { + ASSERT_OK(Put(1, "c", "begin")); + ASSERT_OK(Put(1, "e", "end")); + ASSERT_OK(Flush(1)); + } + ASSERT_EQ("3,1", FilesPerLevel(1)); + + // Compact just the new range + Compact(1, "b", "f", 1); + ASSERT_EQ("0,2", FilesPerLevel(1)); + ASSERT_EQ(2, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[0].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + // Compact all + ASSERT_OK(Put(1, "a", "begin")); + ASSERT_OK(Put(1, "z", "end")); + ASSERT_OK(Flush(1)); + ASSERT_EQ("1,2", FilesPerLevel(1)); + ASSERT_EQ(2, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[0].path)); + CompactRangeOptions compact_options; + compact_options.target_path_id = 1; + compact_options.exclusive_manual_compaction = exclusive_manual_compaction_; + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr); + + ASSERT_EQ("0,1", FilesPerLevel(1)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[0].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + if (iter == 0) { + DestroyAndReopen(options); + options = CurrentOptions(); + options.db_paths.emplace_back(dbname_ + "_2", 2 * 10485760); + options.db_paths.emplace_back(dbname_ + "_3", 100 * 10485760); + options.db_paths.emplace_back(dbname_ + "_4", 120 * 10485760); + options.max_background_flushes = 1; + options.num_levels = 3; + options.create_if_missing = true; + CreateAndReopenWithCF({"pikachu"}, options); + } + } +} + +TEST_F(DBCompactionTest, FilesDeletedAfterCompaction) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "foo", "v2")); + Compact(1, "a", "z"); + const size_t num_files = CountLiveFiles(); + for (int i = 0; i < 10; i++) { + ASSERT_OK(Put(1, "foo", "v2")); + Compact(1, "a", "z"); + } + ASSERT_EQ(CountLiveFiles(), num_files); + } while (ChangeCompactOptions()); +} + +// Check level comapction with compact files +TEST_P(DBCompactionTestWithParam, DISABLED_CompactFilesOnLevelCompaction) { + const int kTestKeySize = 16; + const int kTestValueSize = 984; + const int kEntrySize = kTestKeySize + kTestValueSize; + const int kEntriesPerBuffer = 100; + Options options; + options.create_if_missing = true; + options.write_buffer_size = kEntrySize * kEntriesPerBuffer; + options.compaction_style = kCompactionStyleLevel; + options.target_file_size_base = options.write_buffer_size; + options.max_bytes_for_level_base = options.target_file_size_base * 2; + options.level0_stop_writes_trigger = 2; + options.max_bytes_for_level_multiplier = 2; + options.compression = kNoCompression; + options.max_subcompactions = max_subcompactions_; + options = CurrentOptions(options); + CreateAndReopenWithCF({"pikachu"}, options); + + Random rnd(301); + for (int key = 64 * kEntriesPerBuffer; key >= 0; --key) { + ASSERT_OK(Put(1, ToString(key), RandomString(&rnd, kTestValueSize))); + } + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + dbfull()->TEST_WaitForCompact(); + + ColumnFamilyMetaData cf_meta; + dbfull()->GetColumnFamilyMetaData(handles_[1], &cf_meta); + int output_level = static_cast(cf_meta.levels.size()) - 1; + for (int file_picked = 5; file_picked > 0; --file_picked) { + std::set overlapping_file_names; + std::vector compaction_input_file_names; + for (int f = 0; f < file_picked; ++f) { + int level = 0; + auto file_meta = PickFileRandomly(cf_meta, &rnd, &level); + compaction_input_file_names.push_back(file_meta->name); + GetOverlappingFileNumbersForLevelCompaction( + cf_meta, options.comparator, level, output_level, + file_meta, &overlapping_file_names); + } + + ASSERT_OK(dbfull()->CompactFiles( + CompactionOptions(), handles_[1], + compaction_input_file_names, + output_level)); + + // Make sure all overlapping files do not exist after compaction + dbfull()->GetColumnFamilyMetaData(handles_[1], &cf_meta); + VerifyCompactionResult(cf_meta, overlapping_file_names); + } + + // make sure all key-values are still there. + for (int key = 64 * kEntriesPerBuffer; key >= 0; --key) { + ASSERT_NE(Get(1, ToString(key)), "NOT_FOUND"); + } +} + +TEST_P(DBCompactionTestWithParam, PartialCompactionFailure) { + Options options; + const int kKeySize = 16; + const int kKvSize = 1000; + const int kKeysPerBuffer = 100; + const int kNumL1Files = 5; + options.create_if_missing = true; + options.write_buffer_size = kKeysPerBuffer * kKvSize; + options.max_write_buffer_number = 2; + options.target_file_size_base = + options.write_buffer_size * + (options.max_write_buffer_number - 1); + options.level0_file_num_compaction_trigger = kNumL1Files; + options.max_bytes_for_level_base = + options.level0_file_num_compaction_trigger * + options.target_file_size_base; + options.max_bytes_for_level_multiplier = 2; + options.compression = kNoCompression; + options.max_subcompactions = max_subcompactions_; + + env_->SetBackgroundThreads(1, Env::HIGH); + env_->SetBackgroundThreads(1, Env::LOW); + // stop the compaction thread until we simulate the file creation failure. + test::SleepingBackgroundTask sleeping_task_low; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + + options.env = env_; + + DestroyAndReopen(options); + + const int kNumInsertedKeys = + options.level0_file_num_compaction_trigger * + (options.max_write_buffer_number - 1) * + kKeysPerBuffer; + + Random rnd(301); + std::vector keys; + std::vector values; + for (int k = 0; k < kNumInsertedKeys; ++k) { + keys.emplace_back(RandomString(&rnd, kKeySize)); + values.emplace_back(RandomString(&rnd, kKvSize - kKeySize)); + ASSERT_OK(Put(Slice(keys[k]), Slice(values[k]))); + dbfull()->TEST_WaitForFlushMemTable(); + } + + dbfull()->TEST_FlushMemTable(true); + // Make sure the number of L0 files can trigger compaction. + ASSERT_GE(NumTableFilesAtLevel(0), + options.level0_file_num_compaction_trigger); + + auto previous_num_level0_files = NumTableFilesAtLevel(0); + + // Fail the first file creation. + env_->non_writable_count_ = 1; + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilDone(); + + // Expect compaction to fail here as one file will fail its + // creation. + ASSERT_TRUE(!dbfull()->TEST_WaitForCompact().ok()); + + // Verify L0 -> L1 compaction does fail. + ASSERT_EQ(NumTableFilesAtLevel(1), 0); + + // Verify all L0 files are still there. + ASSERT_EQ(NumTableFilesAtLevel(0), previous_num_level0_files); + + // All key-values must exist after compaction fails. + for (int k = 0; k < kNumInsertedKeys; ++k) { + ASSERT_EQ(values[k], Get(keys[k])); + } + + env_->non_writable_count_ = 0; + + // Make sure RocksDB will not get into corrupted state. + Reopen(options); + + // Verify again after reopen. + for (int k = 0; k < kNumInsertedKeys; ++k) { + ASSERT_EQ(values[k], Get(keys[k])); + } +} + +TEST_P(DBCompactionTestWithParam, DeleteMovedFileAfterCompaction) { + // iter 1 -- delete_obsolete_files_period_micros == 0 + for (int iter = 0; iter < 2; ++iter) { + // This test triggers move compaction and verifies that the file is not + // deleted when it's part of move compaction + Options options = CurrentOptions(); + options.env = env_; + if (iter == 1) { + options.delete_obsolete_files_period_micros = 0; + } + options.create_if_missing = true; + options.level0_file_num_compaction_trigger = + 2; // trigger compaction when we have 2 files + OnFileDeletionListener* listener = new OnFileDeletionListener(); + options.listeners.emplace_back(listener); + options.max_subcompactions = max_subcompactions_; + DestroyAndReopen(options); + + Random rnd(301); + // Create two 1MB sst files + for (int i = 0; i < 2; ++i) { + // Create 1MB sst file + for (int j = 0; j < 100; ++j) { + ASSERT_OK(Put(Key(i * 50 + j), RandomString(&rnd, 10 * 1024))); + } + ASSERT_OK(Flush()); + } + // this should execute L0->L1 + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ("0,1", FilesPerLevel(0)); + + // block compactions + test::SleepingBackgroundTask sleeping_task; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task, + Env::Priority::LOW); + + options.max_bytes_for_level_base = 1024 * 1024; // 1 MB + Reopen(options); + std::unique_ptr iterator(db_->NewIterator(ReadOptions())); + ASSERT_EQ("0,1", FilesPerLevel(0)); + // let compactions go + sleeping_task.WakeUp(); + sleeping_task.WaitUntilDone(); + + // this should execute L1->L2 (move) + dbfull()->TEST_WaitForCompact(); + + ASSERT_EQ("0,0,1", FilesPerLevel(0)); + + std::vector metadata; + db_->GetLiveFilesMetaData(&metadata); + ASSERT_EQ(metadata.size(), 1U); + auto moved_file_name = metadata[0].name; + + // Create two more 1MB sst files + for (int i = 0; i < 2; ++i) { + // Create 1MB sst file + for (int j = 0; j < 100; ++j) { + ASSERT_OK(Put(Key(i * 50 + j + 100), RandomString(&rnd, 10 * 1024))); + } + ASSERT_OK(Flush()); + } + // this should execute both L0->L1 and L1->L2 (merge with previous file) + dbfull()->TEST_WaitForCompact(); + + ASSERT_EQ("0,0,2", FilesPerLevel(0)); + + // iterator is holding the file + ASSERT_OK(env_->FileExists(dbname_ + moved_file_name)); + + listener->SetExpectedFileName(dbname_ + moved_file_name); + iterator.reset(); + + // this file should have been compacted away + ASSERT_NOK(env_->FileExists(dbname_ + moved_file_name)); + listener->VerifyMatchedCount(1); + } +} + +TEST_P(DBCompactionTestWithParam, CompressLevelCompaction) { + if (!Zlib_Supported()) { + return; + } + Options options = CurrentOptions(); + options.memtable_factory.reset( + new SpecialSkipListFactory(KNumKeysByGenerateNewFile - 1)); + options.compaction_style = kCompactionStyleLevel; + options.write_buffer_size = 110 << 10; // 110KB + options.arena_block_size = 4 << 10; + options.level0_file_num_compaction_trigger = 2; + options.num_levels = 4; + options.max_bytes_for_level_base = 400 * 1024; + options.max_subcompactions = max_subcompactions_; + // First two levels have no compression, so that a trivial move between + // them will be allowed. Level 2 has Zlib compression so that a trivial + // move to level 3 will not be allowed + options.compression_per_level = {kNoCompression, kNoCompression, + kZlibCompression}; + int matches = 0, didnt_match = 0, trivial_move = 0, non_trivial = 0; + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "Compaction::InputCompressionMatchesOutput:Matches", + [&](void* arg) { matches++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "Compaction::InputCompressionMatchesOutput:DidntMatch", + [&](void* arg) { didnt_match++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", + [&](void* arg) { non_trivial++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Reopen(options); + + Random rnd(301); + int key_idx = 0; + + // First three 110KB files are going to level 0 + // After that, (100K, 200K) + for (int num = 0; num < 3; num++) { + GenerateNewFile(&rnd, &key_idx); + } + + // Another 110KB triggers a compaction to 400K file to fill up level 0 + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(4, GetSstFileCount(dbname_)); + + // (1, 4) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4", FilesPerLevel(0)); + + // (1, 4, 1) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,1", FilesPerLevel(0)); + + // (1, 4, 2) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,2", FilesPerLevel(0)); + + // (1, 4, 3) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,3", FilesPerLevel(0)); + + // (1, 4, 4) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,4", FilesPerLevel(0)); + + // (1, 4, 5) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,5", FilesPerLevel(0)); + + // (1, 4, 6) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,6", FilesPerLevel(0)); + + // (1, 4, 7) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,7", FilesPerLevel(0)); + + // (1, 4, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ("1,4,8", FilesPerLevel(0)); + + ASSERT_EQ(matches, 12); + // Currently, the test relies on the number of calls to + // InputCompressionMatchesOutput() per compaction. + const int kCallsToInputCompressionMatch = 2; + ASSERT_EQ(didnt_match, 8 * kCallsToInputCompressionMatch); + ASSERT_EQ(trivial_move, 12); + ASSERT_EQ(non_trivial, 8); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + for (int i = 0; i < key_idx; i++) { + auto v = Get(Key(i)); + ASSERT_NE(v, "NOT_FOUND"); + ASSERT_TRUE(v.size() == 1 || v.size() == 990); + } + + Reopen(options); + + for (int i = 0; i < key_idx; i++) { + auto v = Get(Key(i)); + ASSERT_NE(v, "NOT_FOUND"); + ASSERT_TRUE(v.size() == 1 || v.size() == 990); + } + + Destroy(options); +} + +TEST_F(DBCompactionTest, SanitizeCompactionOptionsTest) { + Options options = CurrentOptions(); + options.max_background_compactions = 5; + options.soft_pending_compaction_bytes_limit = 0; + options.hard_pending_compaction_bytes_limit = 100; + options.create_if_missing = true; + DestroyAndReopen(options); + ASSERT_EQ(100, db_->GetOptions().soft_pending_compaction_bytes_limit); + + options.max_background_compactions = 3; + options.soft_pending_compaction_bytes_limit = 200; + options.hard_pending_compaction_bytes_limit = 150; + DestroyAndReopen(options); + ASSERT_EQ(150, db_->GetOptions().soft_pending_compaction_bytes_limit); +} + +// This tests for a bug that could cause two level0 compactions running +// concurrently +// TODO(aekmekji): Make sure that the reason this fails when run with +// max_subcompactions > 1 is not a correctness issue but just inherent to +// running parallel L0-L1 compactions +TEST_F(DBCompactionTest, SuggestCompactRangeNoTwoLevel0Compactions) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleLevel; + options.write_buffer_size = 110 << 10; + options.arena_block_size = 4 << 10; + options.level0_file_num_compaction_trigger = 4; + options.num_levels = 4; + options.compression = kNoCompression; + options.max_bytes_for_level_base = 450 << 10; + options.target_file_size_base = 98 << 10; + options.max_write_buffer_number = 2; + options.max_background_compactions = 2; + + DestroyAndReopen(options); + + // fill up the DB + Random rnd(301); + for (int num = 0; num < 10; num++) { + GenerateNewRandomFile(&rnd); + } + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"CompactionJob::Run():Start", + "DBCompactionTest::SuggestCompactRangeNoTwoLevel0Compactions:1"}, + {"DBCompactionTest::SuggestCompactRangeNoTwoLevel0Compactions:2", + "CompactionJob::Run():End"}}); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // trigger L0 compaction + for (int num = 0; num < options.level0_file_num_compaction_trigger + 1; + num++) { + GenerateNewRandomFile(&rnd, /* nowait */ true); + ASSERT_OK(Flush()); + } + + TEST_SYNC_POINT( + "DBCompactionTest::SuggestCompactRangeNoTwoLevel0Compactions:1"); + + GenerateNewRandomFile(&rnd, /* nowait */ true); + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_OK(experimental::SuggestCompactRange(db_, nullptr, nullptr)); + for (int num = 0; num < options.level0_file_num_compaction_trigger + 1; + num++) { + GenerateNewRandomFile(&rnd, /* nowait */ true); + ASSERT_OK(Flush()); + } + + TEST_SYNC_POINT( + "DBCompactionTest::SuggestCompactRangeNoTwoLevel0Compactions:2"); + dbfull()->TEST_WaitForCompact(); +} + + +TEST_P(DBCompactionTestWithParam, ForceBottommostLevelCompaction) { + int32_t trivial_move = 0; + int32_t non_trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", + [&](void* arg) { non_trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.write_buffer_size = 100000000; + options.max_subcompactions = max_subcompactions_; + DestroyAndReopen(options); + + int32_t value_size = 10 * 1024; // 10 KB + + Random rnd(301); + std::vector values; + // File with keys [ 0 => 99 ] + for (int i = 0; i < 100; i++) { + values.push_back(RandomString(&rnd, value_size)); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + ASSERT_EQ("1", FilesPerLevel(0)); + // Compaction will do L0=>L1 (trivial move) then move L1 files to L3 + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 3; + ASSERT_OK(db_->CompactRange(compact_options, nullptr, nullptr)); + ASSERT_EQ("0,0,0,1", FilesPerLevel(0)); + ASSERT_EQ(trivial_move, 1); + ASSERT_EQ(non_trivial_move, 0); + + // File with keys [ 100 => 199 ] + for (int i = 100; i < 200; i++) { + values.push_back(RandomString(&rnd, value_size)); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + ASSERT_EQ("1,0,0,1", FilesPerLevel(0)); + // Compaction will do L0=>L1 L1=>L2 L2=>L3 (3 trivial moves) + // then compacte the bottommost level L3=>L3 (non trivial move) + compact_options = CompactRangeOptions(); + compact_options.bottommost_level_compaction = + BottommostLevelCompaction::kForce; + ASSERT_OK(db_->CompactRange(compact_options, nullptr, nullptr)); + ASSERT_EQ("0,0,0,1", FilesPerLevel(0)); + ASSERT_EQ(trivial_move, 4); + ASSERT_EQ(non_trivial_move, 1); + + // File with keys [ 200 => 299 ] + for (int i = 200; i < 300; i++) { + values.push_back(RandomString(&rnd, value_size)); + ASSERT_OK(Put(Key(i), values[i])); + } + ASSERT_OK(Flush()); + + ASSERT_EQ("1,0,0,1", FilesPerLevel(0)); + trivial_move = 0; + non_trivial_move = 0; + compact_options = CompactRangeOptions(); + compact_options.bottommost_level_compaction = + BottommostLevelCompaction::kSkip; + // Compaction will do L0=>L1 L1=>L2 L2=>L3 (3 trivial moves) + // and will skip bottommost level compaction + ASSERT_OK(db_->CompactRange(compact_options, nullptr, nullptr)); + ASSERT_EQ("0,0,0,2", FilesPerLevel(0)); + ASSERT_EQ(trivial_move, 3); + ASSERT_EQ(non_trivial_move, 0); + + for (int i = 0; i < 300; i++) { + ASSERT_EQ(Get(Key(i)), values[i]); + } + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_P(DBCompactionTestWithParam, IntraL0Compaction) { + Options options = CurrentOptions(); + options.compression = kNoCompression; + options.level0_file_num_compaction_trigger = 5; + options.max_background_compactions = 2; + options.max_subcompactions = max_subcompactions_; + DestroyAndReopen(options); + + const size_t kValueSize = 1 << 20; + Random rnd(301); + std::string value(RandomString(&rnd, kValueSize)); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"LevelCompactionPicker::PickCompactionBySize:0", + "CompactionJob::Run():Start"}}); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // index: 0 1 2 3 4 5 6 7 8 9 + // size: 1MB 1MB 1MB 1MB 1MB 2MB 1MB 1MB 1MB 1MB + // score: 1.5 1.3 1.5 2.0 inf + // + // Files 0-4 will be included in an L0->L1 compaction. + // + // L0->L0 will be triggered since the sync points guarantee compaction to base + // level is still blocked when files 5-9 trigger another compaction. + // + // Files 6-9 are the longest span of available files for which + // work-per-deleted-file decreases (see "score" row above). + for (int i = 0; i < 10; ++i) { + ASSERT_OK(Put(Key(0), "")); // prevents trivial move + if (i == 5) { + ASSERT_OK(Put(Key(i + 1), value + value)); + } else { + ASSERT_OK(Put(Key(i + 1), value)); + } + ASSERT_OK(Flush()); + } + dbfull()->TEST_WaitForCompact(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + std::vector> level_to_files; + dbfull()->TEST_GetFilesMetaData(dbfull()->DefaultColumnFamily(), + &level_to_files); + ASSERT_GE(level_to_files.size(), 2); // at least L0 and L1 + // L0 has the 2MB file (not compacted) and 4MB file (output of L0->L0) + ASSERT_EQ(2, level_to_files[0].size()); + ASSERT_GT(level_to_files[1].size(), 0); + for (int i = 0; i < 2; ++i) { + ASSERT_GE(level_to_files[0][i].fd.file_size, 1 << 21); + } +} + +TEST_P(DBCompactionTestWithParam, IntraL0CompactionDoesNotObsoleteDeletions) { + // regression test for issue #2722: L0->L0 compaction can resurrect deleted + // keys from older L0 files if L1+ files' key-ranges do not include the key. + Options options = CurrentOptions(); + options.compression = kNoCompression; + options.level0_file_num_compaction_trigger = 5; + options.max_background_compactions = 2; + options.max_subcompactions = max_subcompactions_; + DestroyAndReopen(options); + + const size_t kValueSize = 1 << 20; + Random rnd(301); + std::string value(RandomString(&rnd, kValueSize)); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"LevelCompactionPicker::PickCompactionBySize:0", + "CompactionJob::Run():Start"}}); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // index: 0 1 2 3 4 5 6 7 8 9 + // size: 1MB 1MB 1MB 1MB 1MB 1MB 1MB 1MB 1MB 1MB + // score: 1.25 1.33 1.5 2.0 inf + // + // Files 0-4 will be included in an L0->L1 compaction. + // + // L0->L0 will be triggered since the sync points guarantee compaction to base + // level is still blocked when files 5-9 trigger another compaction. All files + // 5-9 are included in the L0->L0 due to work-per-deleted file decreasing. + // + // Put a key-value in files 0-4. Delete that key in files 5-9. Verify the + // L0->L0 preserves the deletion such that the key remains deleted. + for (int i = 0; i < 10; ++i) { + // key 0 serves both to prevent trivial move and as the key we want to + // verify is not resurrected by L0->L0 compaction. + if (i < 5) { + ASSERT_OK(Put(Key(0), "")); + } else { + ASSERT_OK(Delete(Key(0))); + } + ASSERT_OK(Put(Key(i + 1), value)); + ASSERT_OK(Flush()); + } + dbfull()->TEST_WaitForCompact(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + std::vector> level_to_files; + dbfull()->TEST_GetFilesMetaData(dbfull()->DefaultColumnFamily(), + &level_to_files); + ASSERT_GE(level_to_files.size(), 2); // at least L0 and L1 + // L0 has a single output file from L0->L0 + ASSERT_EQ(1, level_to_files[0].size()); + ASSERT_GT(level_to_files[1].size(), 0); + ASSERT_GE(level_to_files[0][0].fd.file_size, 1 << 22); + + ReadOptions roptions; + std::string result; + ASSERT_TRUE(db_->Get(roptions, Key(0), &result).IsNotFound()); +} + +TEST_F(DBCompactionTest, OptimizedDeletionObsoleting) { + // Deletions can be dropped when compacted to non-last level if they fall + // outside the lower-level files' key-ranges. + const int kNumL0Files = 4; + Options options = CurrentOptions(); + options.level0_file_num_compaction_trigger = kNumL0Files; + options.statistics = rocksdb::CreateDBStatistics(); + DestroyAndReopen(options); + + // put key 1 and 3 in separate L1, L2 files. + // So key 0, 2, and 4+ fall outside these levels' key-ranges. + for (int level = 2; level >= 1; --level) { + for (int i = 0; i < 2; ++i) { + Put(Key(2 * i + 1), "val"); + Flush(); + } + MoveFilesToLevel(level); + ASSERT_EQ(2, NumTableFilesAtLevel(level)); + } + + // Delete keys in range [1, 4]. These L0 files will be compacted with L1: + // - Tombstones for keys 2 and 4 can be dropped early. + // - Tombstones for keys 1 and 3 must be kept due to L2 files' key-ranges. + for (int i = 0; i < kNumL0Files; ++i) { + Put(Key(0), "val"); // sentinel to prevent trivial move + Delete(Key(i + 1)); + Flush(); + } + dbfull()->TEST_WaitForCompact(); + + for (int i = 0; i < kNumL0Files; ++i) { + std::string value; + ASSERT_TRUE(db_->Get(ReadOptions(), Key(i + 1), &value).IsNotFound()); + } + ASSERT_EQ(2, options.statistics->getTickerCount( + COMPACTION_OPTIMIZED_DEL_DROP_OBSOLETE)); + ASSERT_EQ(2, + options.statistics->getTickerCount(COMPACTION_KEY_DROP_OBSOLETE)); +} + +INSTANTIATE_TEST_CASE_P(DBCompactionTestWithParam, DBCompactionTestWithParam, + ::testing::Values(std::make_tuple(1, true), + std::make_tuple(1, false), + std::make_tuple(4, true), + std::make_tuple(4, false))); + +TEST_P(DBCompactionDirectIOTest, DirectIO) { + Options options = CurrentOptions(); + Destroy(options); + options.create_if_missing = true; + options.disable_auto_compactions = true; + options.use_direct_io_for_flush_and_compaction = GetParam(); + options.env = new MockEnv(Env::Default()); + Reopen(options); + bool readahead = false; + SyncPoint::GetInstance()->SetCallBack( + "TableCache::NewIterator:for_compaction", [&](void* arg) { + bool* use_direct_reads = static_cast(arg); + ASSERT_EQ(*use_direct_reads, + options.use_direct_io_for_flush_and_compaction); + }); + SyncPoint::GetInstance()->SetCallBack( + "CompactionJob::OpenCompactionOutputFile", [&](void* arg) { + bool* use_direct_writes = static_cast(arg); + ASSERT_EQ(*use_direct_writes, + options.use_direct_io_for_flush_and_compaction); + }); + if (options.use_direct_io_for_flush_and_compaction) { + SyncPoint::GetInstance()->SetCallBack( + "SanitizeOptions:direct_io", [&](void* arg) { + readahead = true; + }); + } + SyncPoint::GetInstance()->EnableProcessing(); + CreateAndReopenWithCF({"pikachu"}, options); + MakeTables(3, "p", "q", 1); + ASSERT_EQ("1,1,1", FilesPerLevel(1)); + Compact(1, "p1", "p9"); + ASSERT_FALSE(readahead ^ options.use_direct_io_for_flush_and_compaction); + ASSERT_EQ("0,0,1", FilesPerLevel(1)); + Destroy(options); + delete options.env; +} + +INSTANTIATE_TEST_CASE_P(DBCompactionDirectIOTest, DBCompactionDirectIOTest, + testing::Bool()); + +class CompactionPriTest : public DBTestBase, + public testing::WithParamInterface { + public: + CompactionPriTest() : DBTestBase("/compaction_pri_test") { + compaction_pri_ = GetParam(); + } + + // Required if inheriting from testing::WithParamInterface<> + static void SetUpTestCase() {} + static void TearDownTestCase() {} + + uint32_t compaction_pri_; +}; + +TEST_P(CompactionPriTest, Test) { + Options options = CurrentOptions(); + options.write_buffer_size = 16 * 1024; + options.compaction_pri = static_cast(compaction_pri_); + options.hard_pending_compaction_bytes_limit = 256 * 1024; + options.max_bytes_for_level_base = 64 * 1024; + options.max_bytes_for_level_multiplier = 4; + options.compression = kNoCompression; + + DestroyAndReopen(options); + + Random rnd(301); + const int kNKeys = 5000; + int keys[kNKeys]; + for (int i = 0; i < kNKeys; i++) { + keys[i] = i; + } + std::random_shuffle(std::begin(keys), std::end(keys)); + + for (int i = 0; i < kNKeys; i++) { + ASSERT_OK(Put(Key(keys[i]), RandomString(&rnd, 102))); + } + + dbfull()->TEST_WaitForCompact(); + for (int i = 0; i < kNKeys; i++) { + ASSERT_NE("NOT_FOUND", Get(Key(i))); + } +} + +INSTANTIATE_TEST_CASE_P( + CompactionPriTest, CompactionPriTest, + ::testing::Values(CompactionPri::kByCompensatedSize, + CompactionPri::kOldestLargestSeqFirst, + CompactionPri::kOldestSmallestSeqFirst, + CompactionPri::kMinOverlappingRatio)); + +#endif // !defined(ROCKSDB_LITE) +} // namespace rocksdb + +int main(int argc, char** argv) { +#if !defined(ROCKSDB_LITE) + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +#else + return 0; +#endif +} diff --git a/db/db_dynamic_level_test.cc b/db/db_dynamic_level_test.cc new file mode 100644 index 00000000000..f968e7fc057 --- /dev/null +++ b/db/db_dynamic_level_test.cc @@ -0,0 +1,506 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +// Introduction of SyncPoint effectively disabled building and running this test +// in Release build. +// which is a pity, it is a good test +#if !defined(ROCKSDB_LITE) + +#include "db/db_test_util.h" +#include "port/port.h" +#include "port/stack_trace.h" + +namespace rocksdb { +class DBTestDynamicLevel : public DBTestBase { + public: + DBTestDynamicLevel() : DBTestBase("/db_dynamic_level_test") {} +}; + +TEST_F(DBTestDynamicLevel, DynamicLevelMaxBytesBase) { + if (!Snappy_Supported() || !LZ4_Supported()) { + return; + } + // Use InMemoryEnv, or it would be too slow. + unique_ptr env(new MockEnv(env_)); + + const int kNKeys = 1000; + int keys[kNKeys]; + + auto verify_func = [&]() { + for (int i = 0; i < kNKeys; i++) { + ASSERT_NE("NOT_FOUND", Get(Key(i))); + ASSERT_NE("NOT_FOUND", Get(Key(kNKeys * 2 + i))); + if (i < kNKeys / 10) { + ASSERT_EQ("NOT_FOUND", Get(Key(kNKeys + keys[i]))); + } else { + ASSERT_NE("NOT_FOUND", Get(Key(kNKeys + keys[i]))); + } + } + }; + + Random rnd(301); + for (int ordered_insert = 0; ordered_insert <= 1; ordered_insert++) { + for (int i = 0; i < kNKeys; i++) { + keys[i] = i; + } + if (ordered_insert == 0) { + std::random_shuffle(std::begin(keys), std::end(keys)); + } + for (int max_background_compactions = 1; max_background_compactions < 4; + max_background_compactions += 2) { + Options options; + options.env = env.get(); + options.create_if_missing = true; + options.write_buffer_size = 2048; + options.max_write_buffer_number = 2; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 2; + options.level0_stop_writes_trigger = 2; + options.target_file_size_base = 2048; + options.level_compaction_dynamic_level_bytes = true; + options.max_bytes_for_level_base = 10240; + options.max_bytes_for_level_multiplier = 4; + options.soft_rate_limit = 1.1; + options.max_background_compactions = max_background_compactions; + options.num_levels = 5; + + options.compression_per_level.resize(3); + options.compression_per_level[0] = kNoCompression; + options.compression_per_level[1] = kLZ4Compression; + options.compression_per_level[2] = kSnappyCompression; + options.env = env_; + + DestroyAndReopen(options); + + for (int i = 0; i < kNKeys; i++) { + int key = keys[i]; + ASSERT_OK(Put(Key(kNKeys + key), RandomString(&rnd, 102))); + ASSERT_OK(Put(Key(key), RandomString(&rnd, 102))); + ASSERT_OK(Put(Key(kNKeys * 2 + key), RandomString(&rnd, 102))); + ASSERT_OK(Delete(Key(kNKeys + keys[i / 10]))); + env_->SleepForMicroseconds(5000); + } + + uint64_t int_prop; + ASSERT_TRUE(db_->GetIntProperty("rocksdb.background-errors", &int_prop)); + ASSERT_EQ(0U, int_prop); + + // Verify DB + for (int j = 0; j < 2; j++) { + verify_func(); + if (j == 0) { + Reopen(options); + } + } + + // Test compact range works + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + // All data should be in the last level. + ColumnFamilyMetaData cf_meta; + db_->GetColumnFamilyMetaData(&cf_meta); + ASSERT_EQ(5U, cf_meta.levels.size()); + for (int i = 0; i < 4; i++) { + ASSERT_EQ(0U, cf_meta.levels[i].files.size()); + } + ASSERT_GT(cf_meta.levels[4U].files.size(), 0U); + verify_func(); + + Close(); + } + } + + env_->SetBackgroundThreads(1, Env::LOW); + env_->SetBackgroundThreads(1, Env::HIGH); +} + +// Test specific cases in dynamic max bytes +TEST_F(DBTestDynamicLevel, DynamicLevelMaxBytesBase2) { + Random rnd(301); + int kMaxKey = 1000000; + + Options options = CurrentOptions(); + options.create_if_missing = true; + options.write_buffer_size = 20480; + options.max_write_buffer_number = 2; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 9999; + options.level0_stop_writes_trigger = 9999; + options.target_file_size_base = 9102; + options.level_compaction_dynamic_level_bytes = true; + options.max_bytes_for_level_base = 40960; + options.max_bytes_for_level_multiplier = 4; + options.max_background_compactions = 2; + options.num_levels = 5; + options.max_compaction_bytes = 0; // Force not expanding in compactions + BlockBasedTableOptions table_options; + table_options.block_size = 1024; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + DestroyAndReopen(options); + ASSERT_OK(dbfull()->SetOptions({ + {"disable_auto_compactions", "true"}, + })); + + uint64_t int_prop; + std::string str_prop; + + // Initial base level is the last level + ASSERT_TRUE(db_->GetIntProperty("rocksdb.base-level", &int_prop)); + ASSERT_EQ(4U, int_prop); + + // Put about 28K to L0 + for (int i = 0; i < 70; i++) { + ASSERT_OK(Put(Key(static_cast(rnd.Uniform(kMaxKey))), + RandomString(&rnd, 380))); + } + ASSERT_OK(dbfull()->SetOptions({ + {"disable_auto_compactions", "false"}, + })); + Flush(); + dbfull()->TEST_WaitForCompact(); + ASSERT_TRUE(db_->GetIntProperty("rocksdb.base-level", &int_prop)); + ASSERT_EQ(4U, int_prop); + + // Insert extra about 28K to L0. After they are compacted to L4, base level + // should be changed to L3. + ASSERT_OK(dbfull()->SetOptions({ + {"disable_auto_compactions", "true"}, + })); + for (int i = 0; i < 70; i++) { + ASSERT_OK(Put(Key(static_cast(rnd.Uniform(kMaxKey))), + RandomString(&rnd, 380))); + } + + ASSERT_OK(dbfull()->SetOptions({ + {"disable_auto_compactions", "false"}, + })); + Flush(); + dbfull()->TEST_WaitForCompact(); + ASSERT_TRUE(db_->GetIntProperty("rocksdb.base-level", &int_prop)); + ASSERT_EQ(3U, int_prop); + ASSERT_TRUE(db_->GetProperty("rocksdb.num-files-at-level1", &str_prop)); + ASSERT_EQ("0", str_prop); + ASSERT_TRUE(db_->GetProperty("rocksdb.num-files-at-level2", &str_prop)); + ASSERT_EQ("0", str_prop); + + // Trigger parallel compaction, and the first one would change the base + // level. + // Hold compaction jobs to make sure + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "CompactionJob::Run():Start", + [&](void* arg) { env_->SleepForMicroseconds(100000); }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + ASSERT_OK(dbfull()->SetOptions({ + {"disable_auto_compactions", "true"}, + })); + // Write about 40K more + for (int i = 0; i < 100; i++) { + ASSERT_OK(Put(Key(static_cast(rnd.Uniform(kMaxKey))), + RandomString(&rnd, 380))); + } + ASSERT_OK(dbfull()->SetOptions({ + {"disable_auto_compactions", "false"}, + })); + Flush(); + // Wait for 200 milliseconds before proceeding compactions to make sure two + // parallel ones are executed. + env_->SleepForMicroseconds(200000); + dbfull()->TEST_WaitForCompact(); + ASSERT_TRUE(db_->GetIntProperty("rocksdb.base-level", &int_prop)); + ASSERT_EQ(3U, int_prop); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + // Trigger a condition that the compaction changes base level and L0->Lbase + // happens at the same time. + // We try to make last levels' targets to be 40K, 160K, 640K, add triggers + // another compaction from 40K->160K. + ASSERT_OK(dbfull()->SetOptions({ + {"disable_auto_compactions", "true"}, + })); + // Write about 650K more. + // Each file is about 11KB, with 9KB of data. + for (int i = 0; i < 1300; i++) { + ASSERT_OK(Put(Key(static_cast(rnd.Uniform(kMaxKey))), + RandomString(&rnd, 380))); + } + ASSERT_OK(dbfull()->SetOptions({ + {"disable_auto_compactions", "false"}, + })); + Flush(); + dbfull()->TEST_WaitForCompact(); + ASSERT_TRUE(db_->GetIntProperty("rocksdb.base-level", &int_prop)); + ASSERT_EQ(2U, int_prop); + + // A manual compaction will trigger the base level to become L2 + // Keep Writing data until base level changed 2->1. There will be L0->L2 + // compaction going on at the same time. + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"CompactionJob::Run():Start", "DynamicLevelMaxBytesBase2:0"}, + {"DynamicLevelMaxBytesBase2:1", "CompactionJob::Run():End"}, + {"DynamicLevelMaxBytesBase2:compact_range_finish", + "FlushJob::WriteLevel0Table"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + rocksdb::port::Thread thread([this] { + TEST_SYNC_POINT("DynamicLevelMaxBytesBase2:compact_range_start"); + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + TEST_SYNC_POINT("DynamicLevelMaxBytesBase2:compact_range_finish"); + }); + + TEST_SYNC_POINT("DynamicLevelMaxBytesBase2:0"); + for (int i = 0; i < 2; i++) { + ASSERT_OK(Put(Key(static_cast(rnd.Uniform(kMaxKey))), + RandomString(&rnd, 380))); + } + TEST_SYNC_POINT("DynamicLevelMaxBytesBase2:1"); + + Flush(); + + thread.join(); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); + + ASSERT_TRUE(db_->GetIntProperty("rocksdb.base-level", &int_prop)); + ASSERT_EQ(1U, int_prop); +} + +// Test specific cases in dynamic max bytes +TEST_F(DBTestDynamicLevel, DynamicLevelMaxBytesCompactRange) { + Random rnd(301); + int kMaxKey = 1000000; + + Options options = CurrentOptions(); + options.create_if_missing = true; + options.write_buffer_size = 2048; + options.max_write_buffer_number = 2; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 9999; + options.level0_stop_writes_trigger = 9999; + options.target_file_size_base = 2; + options.level_compaction_dynamic_level_bytes = true; + options.max_bytes_for_level_base = 10240; + options.max_bytes_for_level_multiplier = 4; + options.max_background_compactions = 1; + const int kNumLevels = 5; + options.num_levels = kNumLevels; + options.max_compaction_bytes = 1; // Force not expanding in compactions + BlockBasedTableOptions table_options; + table_options.block_size = 1024; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + DestroyAndReopen(options); + + // Compact against empty DB + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + + uint64_t int_prop; + std::string str_prop; + + // Initial base level is the last level + ASSERT_TRUE(db_->GetIntProperty("rocksdb.base-level", &int_prop)); + ASSERT_EQ(4U, int_prop); + + // Put about 7K to L0 + for (int i = 0; i < 140; i++) { + ASSERT_OK(Put(Key(static_cast(rnd.Uniform(kMaxKey))), + RandomString(&rnd, 80))); + } + Flush(); + dbfull()->TEST_WaitForCompact(); + if (NumTableFilesAtLevel(0) == 0) { + // Make sure level 0 is not empty + ASSERT_OK(Put(Key(static_cast(rnd.Uniform(kMaxKey))), + RandomString(&rnd, 80))); + Flush(); + } + + ASSERT_TRUE(db_->GetIntProperty("rocksdb.base-level", &int_prop)); + ASSERT_EQ(3U, int_prop); + ASSERT_TRUE(db_->GetProperty("rocksdb.num-files-at-level1", &str_prop)); + ASSERT_EQ("0", str_prop); + ASSERT_TRUE(db_->GetProperty("rocksdb.num-files-at-level2", &str_prop)); + ASSERT_EQ("0", str_prop); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); + + std::set output_levels; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "CompactionPicker::CompactRange:Return", [&](void* arg) { + Compaction* compaction = reinterpret_cast(arg); + output_levels.insert(compaction->output_level()); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ(output_levels.size(), 2); + ASSERT_TRUE(output_levels.find(3) != output_levels.end()); + ASSERT_TRUE(output_levels.find(4) != output_levels.end()); + ASSERT_TRUE(db_->GetProperty("rocksdb.num-files-at-level0", &str_prop)); + ASSERT_EQ("0", str_prop); + ASSERT_TRUE(db_->GetProperty("rocksdb.num-files-at-level3", &str_prop)); + ASSERT_EQ("0", str_prop); + // Base level is still level 3. + ASSERT_TRUE(db_->GetIntProperty("rocksdb.base-level", &int_prop)); + ASSERT_EQ(3U, int_prop); +} + +TEST_F(DBTestDynamicLevel, DynamicLevelMaxBytesBaseInc) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.write_buffer_size = 2048; + options.max_write_buffer_number = 2; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 2; + options.level0_stop_writes_trigger = 2; + options.target_file_size_base = 2048; + options.level_compaction_dynamic_level_bytes = true; + options.max_bytes_for_level_base = 10240; + options.max_bytes_for_level_multiplier = 4; + options.soft_rate_limit = 1.1; + options.max_background_compactions = 2; + options.num_levels = 5; + options.max_compaction_bytes = 100000000; + + DestroyAndReopen(options); + + int non_trivial = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", + [&](void* arg) { non_trivial++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Random rnd(301); + const int total_keys = 3000; + const int random_part_size = 100; + for (int i = 0; i < total_keys; i++) { + std::string value = RandomString(&rnd, random_part_size); + PutFixed32(&value, static_cast(i)); + ASSERT_OK(Put(Key(i), value)); + } + Flush(); + dbfull()->TEST_WaitForCompact(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + ASSERT_EQ(non_trivial, 0); + + for (int i = 0; i < total_keys; i++) { + std::string value = Get(Key(i)); + ASSERT_EQ(DecodeFixed32(value.c_str() + random_part_size), + static_cast(i)); + } + + env_->SetBackgroundThreads(1, Env::LOW); + env_->SetBackgroundThreads(1, Env::HIGH); +} + +TEST_F(DBTestDynamicLevel, DISABLED_MigrateToDynamicLevelMaxBytesBase) { + Random rnd(301); + const int kMaxKey = 2000; + + Options options; + options.create_if_missing = true; + options.write_buffer_size = 2048; + options.max_write_buffer_number = 8; + options.level0_file_num_compaction_trigger = 4; + options.level0_slowdown_writes_trigger = 4; + options.level0_stop_writes_trigger = 8; + options.target_file_size_base = 2048; + options.level_compaction_dynamic_level_bytes = false; + options.max_bytes_for_level_base = 10240; + options.max_bytes_for_level_multiplier = 4; + options.soft_rate_limit = 1.1; + options.num_levels = 8; + + DestroyAndReopen(options); + + auto verify_func = [&](int num_keys, bool if_sleep) { + for (int i = 0; i < num_keys; i++) { + ASSERT_NE("NOT_FOUND", Get(Key(kMaxKey + i))); + if (i < num_keys / 10) { + ASSERT_EQ("NOT_FOUND", Get(Key(i))); + } else { + ASSERT_NE("NOT_FOUND", Get(Key(i))); + } + if (if_sleep && i % 1000 == 0) { + // Without it, valgrind may choose not to give another + // thread a chance to run before finishing the function, + // causing the test to be extremely slow. + env_->SleepForMicroseconds(1); + } + } + }; + + int total_keys = 1000; + for (int i = 0; i < total_keys; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 102))); + ASSERT_OK(Put(Key(kMaxKey + i), RandomString(&rnd, 102))); + ASSERT_OK(Delete(Key(i / 10))); + } + verify_func(total_keys, false); + dbfull()->TEST_WaitForCompact(); + + options.level_compaction_dynamic_level_bytes = true; + options.disable_auto_compactions = true; + Reopen(options); + verify_func(total_keys, false); + + std::atomic_bool compaction_finished; + compaction_finished = false; + // Issue manual compaction in one thread and still verify DB state + // in main thread. + rocksdb::port::Thread t([&]() { + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = options.num_levels - 1; + dbfull()->CompactRange(compact_options, nullptr, nullptr); + compaction_finished.store(true); + }); + do { + verify_func(total_keys, true); + } while (!compaction_finished.load()); + t.join(); + + ASSERT_OK(dbfull()->SetOptions({ + {"disable_auto_compactions", "false"}, + })); + + int total_keys2 = 2000; + for (int i = total_keys; i < total_keys2; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 102))); + ASSERT_OK(Put(Key(kMaxKey + i), RandomString(&rnd, 102))); + ASSERT_OK(Delete(Key(i / 10))); + } + + verify_func(total_keys2, false); + dbfull()->TEST_WaitForCompact(); + verify_func(total_keys2, false); + + // Base level is not level 1 + ASSERT_EQ(NumTableFilesAtLevel(1), 0); + ASSERT_EQ(NumTableFilesAtLevel(2), 0); +} +} // namespace rocksdb + +#endif // !defined(ROCKSDB_LITE) + +int main(int argc, char** argv) { +#if !defined(ROCKSDB_LITE) + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +#else + return 0; +#endif +} diff --git a/db/db_encryption_test.cc b/db/db_encryption_test.cc new file mode 100644 index 00000000000..38eee56459e --- /dev/null +++ b/db/db_encryption_test.cc @@ -0,0 +1,94 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#include "db/db_test_util.h" +#include "port/stack_trace.h" +#include "rocksdb/perf_context.h" +#if !defined(ROCKSDB_LITE) +#include "util/sync_point.h" +#endif +#include +#include + +namespace rocksdb { + +class DBEncryptionTest : public DBTestBase { + public: + DBEncryptionTest() : DBTestBase("/db_encryption_test") {} +}; + +#ifndef ROCKSDB_LITE + +TEST_F(DBEncryptionTest, CheckEncrypted) { + ASSERT_OK(Put("foo567", "v1.fetdq")); + ASSERT_OK(Put("bar123", "v2.dfgkjdfghsd")); + Close(); + + // Open all files and look for the values we've put in there. + // They should not be found if encrypted, otherwise + // they should be found. + std::vector fileNames; + auto status = env_->GetChildren(dbname_, &fileNames); + ASSERT_OK(status); + + auto defaultEnv = Env::Default(); + int hits = 0; + for (auto it = fileNames.begin() ; it != fileNames.end(); ++it) { + if ((*it == "..") || (*it == ".")) { + continue; + } + auto filePath = dbname_ + "/" + *it; + unique_ptr seqFile; + auto envOptions = EnvOptions(CurrentOptions()); + status = defaultEnv->NewSequentialFile(filePath, &seqFile, envOptions); + ASSERT_OK(status); + + uint64_t fileSize; + status = defaultEnv->GetFileSize(filePath, &fileSize); + ASSERT_OK(status); + + std::string scratch; + scratch.reserve(fileSize); + Slice data; + status = seqFile->Read(fileSize, &data, (char*)scratch.data()); + ASSERT_OK(status); + + if (data.ToString().find("foo567") != std::string::npos) { + hits++; + //std::cout << "Hit in " << filePath << "\n"; + } + if (data.ToString().find("v1.fetdq") != std::string::npos) { + hits++; + //std::cout << "Hit in " << filePath << "\n"; + } + if (data.ToString().find("bar123") != std::string::npos) { + hits++; + //std::cout << "Hit in " << filePath << "\n"; + } + if (data.ToString().find("v2.dfgkjdfghsd") != std::string::npos) { + hits++; + //std::cout << "Hit in " << filePath << "\n"; + } + if (data.ToString().find("dfgk") != std::string::npos) { + hits++; + //std::cout << "Hit in " << filePath << "\n"; + } + } + if (encrypted_env_) { + ASSERT_EQ(hits, 0); + } else { + ASSERT_GE(hits, 4); + } +} + +#endif // ROCKSDB_LITE + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_filesnapshot.cc b/db/db_filesnapshot.cc index 4185a40cab9..e266bf1ae1f 100644 --- a/db/db_filesnapshot.cc +++ b/db/db_filesnapshot.cc @@ -1,48 +1,52 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // -// Copyright (c) 2012 Facebook. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. #ifndef ROCKSDB_LITE +#ifndef __STDC_FORMAT_MACROS #define __STDC_FORMAT_MACROS +#endif + #include +#include #include #include -#include #include "db/db_impl.h" -#include "db/filename.h" +#include "db/job_context.h" #include "db/version_set.h" +#include "port/port.h" #include "rocksdb/db.h" #include "rocksdb/env.h" -#include "port/port.h" +#include "util/file_util.h" +#include "util/filename.h" #include "util/mutexlock.h" #include "util/sync_point.h" namespace rocksdb { Status DBImpl::DisableFileDeletions() { - MutexLock l(&mutex_); + InstrumentedMutexLock l(&mutex_); ++disable_delete_obsolete_files_; if (disable_delete_obsolete_files_ == 1) { - Log(options_.info_log, "File Deletions Disabled"); + ROCKS_LOG_INFO(immutable_db_options_.info_log, "File Deletions Disabled"); } else { - Log(options_.info_log, - "File Deletions Disabled, but already disabled. Counter: %d", - disable_delete_obsolete_files_); + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "File Deletions Disabled, but already disabled. Counter: %d", + disable_delete_obsolete_files_); } return Status::OK(); } Status DBImpl::EnableFileDeletions(bool force) { - DeletionState deletion_state; + // Job id == 0 means that this is not our background process, but rather + // user thread + JobContext job_context(0); bool should_purge_files = false; { - MutexLock l(&mutex_); + InstrumentedMutexLock l(&mutex_); if (force) { // if force, we need to enable file deletions right away disable_delete_obsolete_files_ = 0; @@ -50,19 +54,21 @@ Status DBImpl::EnableFileDeletions(bool force) { --disable_delete_obsolete_files_; } if (disable_delete_obsolete_files_ == 0) { - Log(options_.info_log, "File Deletions Enabled"); + ROCKS_LOG_INFO(immutable_db_options_.info_log, "File Deletions Enabled"); should_purge_files = true; - FindObsoleteFiles(deletion_state, true); + FindObsoleteFiles(&job_context, true); } else { - Log(options_.info_log, + ROCKS_LOG_WARN( + immutable_db_options_.info_log, "File Deletions Enable, but not really enabled. Counter: %d", disable_delete_obsolete_files_); } } if (should_purge_files) { - PurgeObsoleteFiles(deletion_state); + PurgeObsoleteFiles(job_context); } - LogFlush(options_.info_log); + job_context.Clean(); + LogFlush(immutable_db_options_.info_log); return Status::OK(); } @@ -73,7 +79,6 @@ int DBImpl::IsFileDeletionsEnabled() const { Status DBImpl::GetLiveFiles(std::vector& ret, uint64_t* manifest_file_size, bool flush_memtable) { - *manifest_file_size = 0; mutex_.Lock(); @@ -82,9 +87,14 @@ Status DBImpl::GetLiveFiles(std::vector& ret, // flush all dirty data to disk. Status status; for (auto cfd : *versions_->GetColumnFamilySet()) { + if (cfd->IsDropped()) { + continue; + } cfd->Ref(); mutex_.Unlock(); status = FlushMemTable(cfd, FlushOptions()); + TEST_SYNC_POINT("DBImpl::GetLiveFiles:1"); + TEST_SYNC_POINT("DBImpl::GetLiveFiles:2"); mutex_.Lock(); cfd->Unref(); if (!status.ok()) { @@ -95,8 +105,8 @@ Status DBImpl::GetLiveFiles(std::vector& ret, if (!status.ok()) { mutex_.Unlock(); - Log(options_.info_log, "Cannot Flush data %s\n", - status.ToString().c_str()); + ROCKS_LOG_ERROR(immutable_db_options_.info_log, "Cannot Flush data %s\n", + status.ToString().c_str()); return status; } } @@ -104,11 +114,14 @@ Status DBImpl::GetLiveFiles(std::vector& ret, // Make a set of all of the live *.sst files std::vector live; for (auto cfd : *versions_->GetColumnFamilySet()) { + if (cfd->IsDropped()) { + continue; + } cfd->current()->AddLiveFiles(&live); } ret.clear(); - ret.reserve(live.size() + 2); //*.sst + CURRENT + MANIFEST + ret.reserve(live.size() + 3); // *.sst + CURRENT + MANIFEST + OPTIONS // create names of the live files. The names are not absolute // paths, instead they are relative to dbname_; @@ -117,65 +130,18 @@ Status DBImpl::GetLiveFiles(std::vector& ret, } ret.push_back(CurrentFileName("")); - ret.push_back(DescriptorFileName("", versions_->ManifestFileNumber())); + ret.push_back(DescriptorFileName("", versions_->manifest_file_number())); + ret.push_back(OptionsFileName("", versions_->options_file_number())); // find length of manifest file while holding the mutex lock - *manifest_file_size = versions_->ManifestFileSize(); + *manifest_file_size = versions_->manifest_file_size(); mutex_.Unlock(); return Status::OK(); } Status DBImpl::GetSortedWalFiles(VectorLogPtr& files) { - // First get sorted files in db dir, then get sorted files from archived - // dir, to avoid a race condition where a log file is moved to archived - // dir in between. - Status s; - // list wal files in main db dir. - VectorLogPtr logs; - s = GetSortedWalsOfType(options_.wal_dir, logs, kAliveLogFile); - if (!s.ok()) { - return s; - } - - // Reproduce the race condition where a log file is moved - // to archived dir, between these two sync points, used in - // (DBTest,TransactionLogIteratorRace) - TEST_SYNC_POINT("DBImpl::GetSortedWalFiles:1"); - TEST_SYNC_POINT("DBImpl::GetSortedWalFiles:2"); - - files.clear(); - // list wal files in archive dir. - std::string archivedir = ArchivalDirectory(options_.wal_dir); - if (env_->FileExists(archivedir)) { - s = GetSortedWalsOfType(archivedir, files, kArchivedLogFile); - if (!s.ok()) { - return s; - } - } - - uint64_t latest_archived_log_number = 0; - if (!files.empty()) { - latest_archived_log_number = files.back()->LogNumber(); - Log(options_.info_log, "Latest Archived log: %" PRIu64, - latest_archived_log_number); - } - - files.reserve(files.size() + logs.size()); - for (auto& log : logs) { - if (log->LogNumber() > latest_archived_log_number) { - files.push_back(std::move(log)); - } else { - // When the race condition happens, we could see the - // same log in both db dir and archived dir. Simply - // ignore the one in db dir. Note that, if we read - // archived dir first, we would have missed the log file. - Log(options_.info_log, "%s already moved to archive", - log->PathName().c_str()); - } - } - - return s; + return wal_manager_.GetSortedWalFiles(files); } } diff --git a/db/db_flush_test.cc b/db/db_flush_test.cc new file mode 100644 index 00000000000..107e82467cb --- /dev/null +++ b/db/db_flush_test.cc @@ -0,0 +1,162 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_test_util.h" +#include "port/stack_trace.h" +#include "util/fault_injection_test_env.h" +#include "util/sync_point.h" + +namespace rocksdb { + +class DBFlushTest : public DBTestBase { + public: + DBFlushTest() : DBTestBase("/db_flush_test") {} +}; + +class DBFlushDirectIOTest : public DBFlushTest, + public ::testing::WithParamInterface { + public: + DBFlushDirectIOTest() : DBFlushTest() {} +}; + +// We had issue when two background threads trying to flush at the same time, +// only one of them get committed. The test verifies the issue is fixed. +TEST_F(DBFlushTest, FlushWhileWritingManifest) { + Options options; + options.disable_auto_compactions = true; + options.max_background_flushes = 2; + options.env = env_; + Reopen(options); + FlushOptions no_wait; + no_wait.wait = false; + + SyncPoint::GetInstance()->LoadDependency( + {{"VersionSet::LogAndApply:WriteManifest", + "DBFlushTest::FlushWhileWritingManifest:1"}, + {"MemTableList::InstallMemtableFlushResults:InProgress", + "VersionSet::LogAndApply:WriteManifestDone"}}); + SyncPoint::GetInstance()->EnableProcessing(); + + ASSERT_OK(Put("foo", "v")); + ASSERT_OK(dbfull()->Flush(no_wait)); + TEST_SYNC_POINT("DBFlushTest::FlushWhileWritingManifest:1"); + ASSERT_OK(Put("bar", "v")); + ASSERT_OK(dbfull()->Flush(no_wait)); + // If the issue is hit we will wait here forever. + dbfull()->TEST_WaitForFlushMemTable(); +#ifndef ROCKSDB_LITE + ASSERT_EQ(2, TotalTableFiles()); +#endif // ROCKSDB_LITE +} + +TEST_F(DBFlushTest, SyncFail) { + std::unique_ptr fault_injection_env( + new FaultInjectionTestEnv(env_)); + Options options; + options.disable_auto_compactions = true; + options.env = fault_injection_env.get(); + + SyncPoint::GetInstance()->LoadDependency( + {{"DBFlushTest::SyncFail:1", "DBImpl::SyncClosedLogs:Start"}, + {"DBImpl::SyncClosedLogs:Failed", "DBFlushTest::SyncFail:2"}}); + SyncPoint::GetInstance()->EnableProcessing(); + + Reopen(options); + Put("key", "value"); + auto* cfd = + reinterpret_cast(db_->DefaultColumnFamily()) + ->cfd(); + int refs_before = cfd->current()->TEST_refs(); + FlushOptions flush_options; + flush_options.wait = false; + ASSERT_OK(dbfull()->Flush(flush_options)); + fault_injection_env->SetFilesystemActive(false); + TEST_SYNC_POINT("DBFlushTest::SyncFail:1"); + TEST_SYNC_POINT("DBFlushTest::SyncFail:2"); + fault_injection_env->SetFilesystemActive(true); + dbfull()->TEST_WaitForFlushMemTable(); +#ifndef ROCKSDB_LITE + ASSERT_EQ("", FilesPerLevel()); // flush failed. +#endif // ROCKSDB_LITE + // Flush job should release ref count to current version. + ASSERT_EQ(refs_before, cfd->current()->TEST_refs()); + Destroy(options); +} + +TEST_F(DBFlushTest, FlushInLowPriThreadPool) { + // Verify setting an empty high-pri (flush) thread pool causes flushes to be + // scheduled in the low-pri (compaction) thread pool. + Options options = CurrentOptions(); + options.level0_file_num_compaction_trigger = 4; + options.memtable_factory.reset(new SpecialSkipListFactory(1)); + Reopen(options); + env_->SetBackgroundThreads(0, Env::HIGH); + + std::thread::id tid; + int num_flushes = 0, num_compactions = 0; + SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BGWorkFlush", [&](void* arg) { + if (tid == std::thread::id()) { + tid = std::this_thread::get_id(); + } else { + ASSERT_EQ(tid, std::this_thread::get_id()); + } + ++num_flushes; + }); + SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BGWorkCompaction", [&](void* arg) { + ASSERT_EQ(tid, std::this_thread::get_id()); + ++num_compactions; + }); + SyncPoint::GetInstance()->EnableProcessing(); + + ASSERT_OK(Put("key", "val")); + for (int i = 0; i < 4; ++i) { + ASSERT_OK(Put("key", "val")); + dbfull()->TEST_WaitForFlushMemTable(); + } + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(4, num_flushes); + ASSERT_EQ(1, num_compactions); +} + +TEST_P(DBFlushDirectIOTest, DirectIO) { + Options options; + options.create_if_missing = true; + options.disable_auto_compactions = true; + options.max_background_flushes = 2; + options.use_direct_io_for_flush_and_compaction = GetParam(); + options.env = new MockEnv(Env::Default()); + SyncPoint::GetInstance()->SetCallBack( + "BuildTable:create_file", [&](void* arg) { + bool* use_direct_writes = static_cast(arg); + ASSERT_EQ(*use_direct_writes, + options.use_direct_io_for_flush_and_compaction); + }); + + SyncPoint::GetInstance()->EnableProcessing(); + Reopen(options); + ASSERT_OK(Put("foo", "v")); + FlushOptions flush_options; + flush_options.wait = true; + ASSERT_OK(dbfull()->Flush(flush_options)); + Destroy(options); + delete options.env; +} + +INSTANTIATE_TEST_CASE_P(DBFlushDirectIOTest, DBFlushDirectIOTest, + testing::Bool()); + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_impl.cc b/db/db_impl.cc index 54faef26304..d1bfe41e8ca 100644 --- a/db/db_impl.cc +++ b/db/db_impl.cc @@ -1,393 +1,234 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. - #include "db/db_impl.h" +#ifndef __STDC_FORMAT_MACROS #define __STDC_FORMAT_MACROS +#endif #include +#include +#ifdef OS_SOLARIS +#include +#endif + #include #include #include +#include #include #include -#include #include -#include #include +#include #include #include #include "db/builder.h" +#include "db/compaction_job.h" +#include "db/db_info_dumper.h" #include "db/db_iter.h" #include "db/dbformat.h" -#include "db/filename.h" +#include "db/event_helpers.h" +#include "db/external_sst_file_ingestion_job.h" +#include "db/flush_job.h" +#include "db/forward_iterator.h" +#include "db/job_context.h" #include "db/log_reader.h" #include "db/log_writer.h" +#include "db/malloc_stats.h" +#include "db/managed_iterator.h" #include "db/memtable.h" #include "db/memtable_list.h" #include "db/merge_context.h" #include "db/merge_helper.h" +#include "db/range_del_aggregator.h" #include "db/table_cache.h" #include "db/table_properties_collector.h" -#include "db/forward_iterator.h" #include "db/transaction_log_impl.h" #include "db/version_set.h" #include "db/write_batch_internal.h" +#include "db/write_callback.h" +#include "memtable/hash_linklist_rep.h" +#include "memtable/hash_skiplist_rep.h" +#include "monitoring/iostats_context_imp.h" +#include "monitoring/perf_context_imp.h" +#include "monitoring/thread_status_updater.h" +#include "monitoring/thread_status_util.h" +#include "options/cf_options.h" +#include "options/options_helper.h" +#include "options/options_parser.h" +#include "port/likely.h" #include "port/port.h" #include "rocksdb/cache.h" -#include "port/likely.h" #include "rocksdb/compaction_filter.h" +#include "rocksdb/convenience.h" #include "rocksdb/db.h" #include "rocksdb/env.h" #include "rocksdb/merge_operator.h" #include "rocksdb/statistics.h" #include "rocksdb/status.h" #include "rocksdb/table.h" +#include "rocksdb/version.h" +#include "rocksdb/write_buffer_manager.h" #include "table/block.h" #include "table/block_based_table_factory.h" -#include "table/merger.h" +#include "table/merging_iterator.h" #include "table/table_builder.h" #include "table/two_level_iterator.h" +#include "tools/sst_dump_tool_imp.h" #include "util/auto_roll_logger.h" #include "util/autovector.h" #include "util/build_version.h" #include "util/coding.h" -#include "util/hash_skiplist_rep.h" -#include "util/hash_linklist_rep.h" -#include "util/logging.h" +#include "util/compression.h" +#include "util/crc32c.h" +#include "util/file_reader_writer.h" +#include "util/file_util.h" +#include "util/filename.h" #include "util/log_buffer.h" +#include "util/logging.h" #include "util/mutexlock.h" -#include "util/perf_context_imp.h" -#include "util/iostats_context_imp.h" +#include "util/sst_file_manager_impl.h" #include "util/stop_watch.h" +#include "util/string_util.h" #include "util/sync_point.h" namespace rocksdb { - const std::string kDefaultColumnFamilyName("default"); +void DumpRocksDBBuildVersion(Logger * log); -void DumpLeveldbBuildVersion(Logger * log); - -// Information kept for every waiting writer -struct DBImpl::Writer { - Status status; - WriteBatch* batch; - bool sync; - bool disableWAL; - bool in_batch_group; - bool done; - uint64_t timeout_hint_us; - port::CondVar cv; - - explicit Writer(port::Mutex* mu) : cv(mu) { } -}; - -struct DBImpl::WriteContext { - autovector superversions_to_free_; - autovector logs_to_free_; - - ~WriteContext() { - for (auto& sv : superversions_to_free_) { - delete sv; - } - for (auto& log : logs_to_free_) { - delete log; - } - } -}; - -struct DBImpl::CompactionState { - Compaction* const compaction; - - // If there were two snapshots with seq numbers s1 and - // s2 and s1 < s2, and if we find two instances of a key k1 then lies - // entirely within s1 and s2, then the earlier version of k1 can be safely - // deleted because that version is not visible in any snapshot. - std::vector existing_snapshots; - - // Files produced by compaction - struct Output { - uint64_t number; - uint32_t path_id; - uint64_t file_size; - InternalKey smallest, largest; - SequenceNumber smallest_seqno, largest_seqno; - }; - std::vector outputs; - std::list allocated_file_numbers; - - // State kept for output being generated - unique_ptr outfile; - unique_ptr builder; - - uint64_t total_bytes; - - Output* current_output() { return &outputs[outputs.size()-1]; } - - explicit CompactionState(Compaction* c) - : compaction(c), - total_bytes(0) { - } - - // Create a client visible context of this compaction - CompactionFilter::Context GetFilterContextV1() { - CompactionFilter::Context context; - context.is_full_compaction = compaction->IsFullCompaction(); - context.is_manual_compaction = compaction->IsManualCompaction(); - return context; - } - - // Create a client visible context of this compaction - CompactionFilterContext GetFilterContext() { - CompactionFilterContext context; - context.is_full_compaction = compaction->IsFullCompaction(); - context.is_manual_compaction = compaction->IsManualCompaction(); - return context; - } - - std::vector key_str_buf_; - std::vector existing_value_str_buf_; - // new_value_buf_ will only be appended if a value changes - std::vector new_value_buf_; - // if values_changed_buf_[i] is true - // new_value_buf_ will add a new entry with the changed value - std::vector value_changed_buf_; - // to_delete_buf_[i] is true iff key_buf_[i] is deleted - std::vector to_delete_buf_; - - std::vector other_key_str_buf_; - std::vector other_value_str_buf_; - - std::vector combined_key_buf_; - std::vector combined_value_buf_; - - std::string cur_prefix_; - - // Buffers the kv-pair that will be run through compaction filter V2 - // in the future. - void BufferKeyValueSlices(const Slice& key, const Slice& value) { - key_str_buf_.emplace_back(key.ToString()); - existing_value_str_buf_.emplace_back(value.ToString()); - } - - // Buffers the kv-pair that will not be run through compaction filter V2 - // in the future. - void BufferOtherKeyValueSlices(const Slice& key, const Slice& value) { - other_key_str_buf_.emplace_back(key.ToString()); - other_value_str_buf_.emplace_back(value.ToString()); - } - - // Add a kv-pair to the combined buffer - void AddToCombinedKeyValueSlices(const Slice& key, const Slice& value) { - // The real strings are stored in the batch buffers - combined_key_buf_.emplace_back(key); - combined_value_buf_.emplace_back(value); - } - - // Merging the two buffers - void MergeKeyValueSliceBuffer(const InternalKeyComparator* comparator) { - size_t i = 0; - size_t j = 0; - size_t total_size = key_str_buf_.size() + other_key_str_buf_.size(); - combined_key_buf_.reserve(total_size); - combined_value_buf_.reserve(total_size); - - while (i + j < total_size) { - int comp_res = 0; - if (i < key_str_buf_.size() && j < other_key_str_buf_.size()) { - comp_res = comparator->Compare(key_str_buf_[i], other_key_str_buf_[j]); - } else if (i >= key_str_buf_.size() && j < other_key_str_buf_.size()) { - comp_res = 1; - } else if (j >= other_key_str_buf_.size() && i < key_str_buf_.size()) { - comp_res = -1; - } - if (comp_res > 0) { - AddToCombinedKeyValueSlices(other_key_str_buf_[j], other_value_str_buf_[j]); - j++; - } else if (comp_res < 0) { - AddToCombinedKeyValueSlices(key_str_buf_[i], existing_value_str_buf_[i]); - i++; - } - } - } - - void CleanupBatchBuffer() { - to_delete_buf_.clear(); - key_str_buf_.clear(); - existing_value_str_buf_.clear(); - new_value_buf_.clear(); - value_changed_buf_.clear(); - - to_delete_buf_.shrink_to_fit(); - key_str_buf_.shrink_to_fit(); - existing_value_str_buf_.shrink_to_fit(); - new_value_buf_.shrink_to_fit(); - value_changed_buf_.shrink_to_fit(); - - other_key_str_buf_.clear(); - other_value_str_buf_.clear(); - other_key_str_buf_.shrink_to_fit(); - other_value_str_buf_.shrink_to_fit(); - } - - void CleanupMergedBuffer() { - combined_key_buf_.clear(); - combined_value_buf_.clear(); - combined_key_buf_.shrink_to_fit(); - combined_value_buf_.shrink_to_fit(); - } -}; - -Options SanitizeOptions(const std::string& dbname, - const InternalKeyComparator* icmp, - const Options& src) { - auto db_options = SanitizeOptions(dbname, DBOptions(src)); - auto cf_options = SanitizeOptions(icmp, ColumnFamilyOptions(src)); - return Options(db_options, cf_options); -} - -DBOptions SanitizeOptions(const std::string& dbname, const DBOptions& src) { - DBOptions result = src; - - // result.max_open_files means an "infinite" open files. - if (result.max_open_files != -1) { - ClipToRange(&result.max_open_files, 20, 1000000); - } - - if (result.info_log == nullptr) { - Status s = CreateLoggerFromOptions(dbname, result.db_log_dir, src.env, - result, &result.info_log); - if (!s.ok()) { - // No place suitable for logging - result.info_log = nullptr; - } - } - - if (!result.rate_limiter) { - if (result.bytes_per_sync == 0) { - result.bytes_per_sync = 1024 * 1024; - } - } - - if (result.wal_dir.empty()) { - // Use dbname as default - result.wal_dir = dbname; - } - if (result.wal_dir.back() == '/') { - result.wal_dir = result.wal_dir.substr(0, result.wal_dir.size() - 1); - } - - if (result.db_paths.size() == 0) { - result.db_paths.emplace_back(dbname, std::numeric_limits::max()); - } - - return result; -} - -namespace { - -Status SanitizeDBOptionsByCFOptions( - const DBOptions* db_opts, - const std::vector& column_families) { - Status s; - for (auto cf : column_families) { - s = cf.options.table_factory->SanitizeDBOptions(db_opts); - if (!s.ok()) { - return s; - } - } - return Status::OK(); -} - -CompressionType GetCompressionFlush(const Options& options) { +CompressionType GetCompressionFlush( + const ImmutableCFOptions& ioptions, + const MutableCFOptions& mutable_cf_options) { // Compressing memtable flushes might not help unless the sequential load // optimization is used for leveled compaction. Otherwise the CPU and // latency overhead is not offset by saving much space. - - bool can_compress; - - if (options.compaction_style == kCompactionStyleUniversal) { - can_compress = - (options.compaction_options_universal.compression_size_percent < 0); + if (ioptions.compaction_style == kCompactionStyleUniversal) { + if (ioptions.compaction_options_universal.compression_size_percent < 0) { + return mutable_cf_options.compression; + } else { + return kNoCompression; + } + } else if (!ioptions.compression_per_level.empty()) { + // For leveled compress when min_level_to_compress != 0. + return ioptions.compression_per_level[0]; } else { - // For leveled compress when min_level_to_compress == 0. - can_compress = options.compression_per_level.empty() || - options.compression_per_level[0] != kNoCompression; + return mutable_cf_options.compression; } +} - if (can_compress) { - return options.compression; - } else { - return kNoCompression; - } +namespace { +void DumpSupportInfo(Logger* logger) { + ROCKS_LOG_HEADER(logger, "Compression algorithms supported:"); + ROCKS_LOG_HEADER(logger, "\tSnappy supported: %d", Snappy_Supported()); + ROCKS_LOG_HEADER(logger, "\tZlib supported: %d", Zlib_Supported()); + ROCKS_LOG_HEADER(logger, "\tBzip supported: %d", BZip2_Supported()); + ROCKS_LOG_HEADER(logger, "\tLZ4 supported: %d", LZ4_Supported()); + ROCKS_LOG_HEADER(logger, "\tZSTD supported: %d", ZSTD_Supported()); + ROCKS_LOG_HEADER(logger, "Fast CRC32 supported: %d", + crc32c::IsFastCrc32Supported()); } -} // namespace + +int64_t kDefaultLowPriThrottledRate = 2 * 1024 * 1024; +} // namespace DBImpl::DBImpl(const DBOptions& options, const std::string& dbname) : env_(options.env), dbname_(dbname), - options_(SanitizeOptions(dbname, options)), - stats_(options_.statistics.get()), + initial_db_options_(SanitizeOptions(dbname, options)), + immutable_db_options_(initial_db_options_), + mutable_db_options_(initial_db_options_), + stats_(immutable_db_options_.statistics.get()), db_lock_(nullptr), - mutex_(options.use_adaptive_mutex), - shutting_down_(nullptr), + mutex_(stats_, env_, DB_MUTEX_WAIT_MICROS, + immutable_db_options_.use_adaptive_mutex), + shutting_down_(false), bg_cv_(&mutex_), logfile_number_(0), + log_dir_synced_(false), log_empty_(true), default_cf_handle_(nullptr), + log_sync_cv_(&mutex_), total_log_size_(0), max_total_in_memory_state_(0), - tmp_batch_(), - bg_schedule_needed_(false), + is_snapshot_supported_(true), + write_buffer_manager_(immutable_db_options_.write_buffer_manager.get()), + write_thread_(immutable_db_options_), + nonmem_write_thread_(immutable_db_options_), + write_controller_(mutable_db_options_.delayed_write_rate), + // Use delayed_write_rate as a base line to determine the initial + // low pri write rate limit. It may be adjusted later. + low_pri_write_rate_limiter_(NewGenericRateLimiter(std::min( + static_cast(mutable_db_options_.delayed_write_rate / 8), + kDefaultLowPriThrottledRate))), + last_batch_group_size_(0), + unscheduled_flushes_(0), + unscheduled_compactions_(0), + bg_bottom_compaction_scheduled_(0), bg_compaction_scheduled_(0), - bg_manual_only_(0), + num_running_compactions_(0), bg_flush_scheduled_(0), - manual_compaction_(nullptr), + num_running_flushes_(0), + bg_purge_scheduled_(0), disable_delete_obsolete_files_(0), - delete_obsolete_files_last_run_(options.env->NowMicros()), - purge_wal_files_last_run_(0), + delete_obsolete_files_last_run_(env_->NowMicros()), last_stats_dump_time_microsec_(0), - default_interval_to_delete_obsolete_WAL_(600), - flush_on_destroy_(false), - delayed_writes_(0), - storage_options_(options), - bg_work_gate_closed_(false), + next_job_id_(1), + has_unpersisted_data_(false), + unable_to_flush_oldest_log_(false), + env_options_(BuildDBOptions(immutable_db_options_, mutable_db_options_)), + num_running_ingest_file_(0), +#ifndef ROCKSDB_LITE + wal_manager_(immutable_db_options_, env_options_), +#endif // ROCKSDB_LITE + event_logger_(immutable_db_options_.info_log.get()), + bg_work_paused_(0), + bg_compaction_paused_(0), refitting_level_(false), - opened_successfully_(false) { + opened_successfully_(false), + concurrent_prepare_(options.concurrent_prepare), + manual_wal_flush_(options.manual_wal_flush) { env_->GetAbsolutePath(dbname, &db_absolute_path_); // Reserve ten files or so for other uses and give the rest to TableCache. // Give a large number for setting of "infinite" open files. - const int table_cache_size = - (options_.max_open_files == -1) ? 4194304 : options_.max_open_files - 10; - // Reserve ten files or so for other uses and give the rest to TableCache. - table_cache_ = - NewLRUCache(table_cache_size, options_.table_cache_numshardbits, - options_.table_cache_remove_scan_count_limit); - - versions_.reset( - new VersionSet(dbname_, &options_, storage_options_, table_cache_.get())); + const int table_cache_size = (mutable_db_options_.max_open_files == -1) + ? TableCache::kInfiniteCapacity + : mutable_db_options_.max_open_files - 10; + table_cache_ = NewLRUCache(table_cache_size, + immutable_db_options_.table_cache_numshardbits); + + versions_.reset(new VersionSet(dbname_, &immutable_db_options_, env_options_, + table_cache_.get(), write_buffer_manager_, + &write_controller_)); column_family_memtables_.reset( new ColumnFamilyMemTablesImpl(versions_->GetColumnFamilySet())); - DumpLeveldbBuildVersion(options_.info_log.get()); - DumpDBFileSummary(options_, dbname_); - options_.Dump(options_.info_log.get()); - - LogFlush(options_.info_log); + DumpRocksDBBuildVersion(immutable_db_options_.info_log.get()); + DumpDBFileSummary(immutable_db_options_, dbname_); + immutable_db_options_.Dump(immutable_db_options_.info_log.get()); + mutable_db_options_.Dump(immutable_db_options_.info_log.get()); + DumpSupportInfo(immutable_db_options_.info_log.get()); } -DBImpl::~DBImpl() { - mutex_.Lock(); - if (flush_on_destroy_) { +// Will lock the mutex_, will wait for completion if wait is true +void DBImpl::CancelAllBackgroundWork(bool wait) { + InstrumentedMutexLock l(&mutex_); + + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "Shutdown: canceling all background work"); + + if (!shutting_down_.load(std::memory_order_acquire) && + has_unpersisted_data_.load(std::memory_order_relaxed) && + !mutable_db_options_.avoid_flush_during_shutdown) { for (auto cfd : *versions_->GetColumnFamilySet()) { - if (cfd->mem()->GetFirstSequenceNumber() != 0) { + if (!cfd->IsDropped() && cfd->initialized() && !cfd->mem()->IsEmpty()) { cfd->Ref(); mutex_.Unlock(); FlushMemTable(cfd, FlushOptions()); @@ -398,11 +239,53 @@ DBImpl::~DBImpl() { versions_->GetColumnFamilySet()->FreeDeadColumnFamilies(); } + shutting_down_.store(true, std::memory_order_release); + bg_cv_.SignalAll(); + if (!wait) { + return; + } + // Wait for background work to finish + while (bg_bottom_compaction_scheduled_ || bg_compaction_scheduled_ || + bg_flush_scheduled_) { + bg_cv_.Wait(); + } +} + +DBImpl::~DBImpl() { + // CancelAllBackgroundWork called with false means we just set the shutdown + // marker. After this we do a variant of the waiting and unschedule work + // (to consider: moving all the waiting into CancelAllBackgroundWork(true)) + CancelAllBackgroundWork(false); + int bottom_compactions_unscheduled = + env_->UnSchedule(this, Env::Priority::BOTTOM); + int compactions_unscheduled = env_->UnSchedule(this, Env::Priority::LOW); + int flushes_unscheduled = env_->UnSchedule(this, Env::Priority::HIGH); + mutex_.Lock(); + bg_bottom_compaction_scheduled_ -= bottom_compactions_unscheduled; + bg_compaction_scheduled_ -= compactions_unscheduled; + bg_flush_scheduled_ -= flushes_unscheduled; + // Wait for background work to finish - shutting_down_.Release_Store(this); // Any non-nullptr value is ok - while (bg_compaction_scheduled_ || bg_flush_scheduled_) { + while (bg_bottom_compaction_scheduled_ || bg_compaction_scheduled_ || + bg_flush_scheduled_ || bg_purge_scheduled_) { + TEST_SYNC_POINT("DBImpl::~DBImpl:WaitJob"); bg_cv_.Wait(); } + EraseThreadStatusDbInfo(); + flush_scheduler_.Clear(); + + while (!flush_queue_.empty()) { + auto cfd = PopFirstFromFlushQueue(); + if (cfd->Unref()) { + delete cfd; + } + } + while (!compaction_queue_.empty()) { + auto cfd = PopFirstFromCompactionQueue(); + if (cfd->Unref()) { + delete cfd; + } + } if (default_cf_handle_ != nullptr) { // we need to delete handle outside of lock because it does its own locking @@ -411,25 +294,54 @@ DBImpl::~DBImpl() { mutex_.Lock(); } - if (options_.allow_thread_local) { - // Clean up obsolete files due to SuperVersion release. - // (1) Need to delete to obsolete files before closing because RepairDB() - // scans all existing files in the file system and builds manifest file. - // Keeping obsolete files confuses the repair process. - // (2) Need to check if we Open()/Recover() the DB successfully before - // deleting because if VersionSet recover fails (may be due to corrupted - // manifest file), it is not able to identify live files correctly. As a - // result, all "live" files can get deleted by accident. However, corrupted - // manifest is recoverable by RepairDB(). - if (opened_successfully_) { - DeletionState deletion_state; - FindObsoleteFiles(deletion_state, true); - // manifest number starting from 2 - deletion_state.manifest_file_number = 1; - if (deletion_state.HaveSomethingToDelete()) { - PurgeObsoleteFiles(deletion_state); - } + // Clean up obsolete files due to SuperVersion release. + // (1) Need to delete to obsolete files before closing because RepairDB() + // scans all existing files in the file system and builds manifest file. + // Keeping obsolete files confuses the repair process. + // (2) Need to check if we Open()/Recover() the DB successfully before + // deleting because if VersionSet recover fails (may be due to corrupted + // manifest file), it is not able to identify live files correctly. As a + // result, all "live" files can get deleted by accident. However, corrupted + // manifest is recoverable by RepairDB(). + if (opened_successfully_) { + JobContext job_context(next_job_id_.fetch_add(1)); + FindObsoleteFiles(&job_context, true); + + mutex_.Unlock(); + // manifest number starting from 2 + job_context.manifest_file_number = 1; + if (job_context.HaveSomethingToDelete()) { + PurgeObsoleteFiles(job_context); } + job_context.Clean(); + mutex_.Lock(); + } + + for (auto l : logs_to_free_) { + delete l; + } + for (auto& log : logs_) { + log.ClearWriter(); + } + logs_.clear(); + + // Table cache may have table handles holding blocks from the block cache. + // We need to release them before the block cache is destroyed. The block + // cache may be destroyed inside versions_.reset(), when column family data + // list is destroyed, so leaving handles in table cache after + // versions_.reset() may cause issues. + // Here we clean all unreferenced handles in table cache. + // Now we assume all user queries have finished, so only version set itself + // can possibly hold the blocks from block cache. After releasing unreferenced + // handles here, only handles held by version set left and inside + // versions_.reset(), we will release them. There, we need to make sure every + // time a handle is released, we erase it from the cache too. By doing that, + // we can guarantee that after versions_.reset(), table cache is empty + // so the cache can be safely destroyed. + table_cache_->EraseUnRefEntries(); + + for (auto& txn_entry : recovered_transactions_) { + delete txn_entry.second; } // versions need to be destroyed before table_cache since it can hold @@ -440,2997 +352,601 @@ DBImpl::~DBImpl() { env_->UnlockFile(db_lock_); } - LogFlush(options_.info_log); -} - -Status DBImpl::NewDB() { - VersionEdit new_db; - new_db.SetLogNumber(0); - new_db.SetNextFile(2); - new_db.SetLastSequence(0); - - Log(options_.info_log, "Creating manifest 1 \n"); - const std::string manifest = DescriptorFileName(dbname_, 1); - unique_ptr file; - Status s = env_->NewWritableFile( - manifest, &file, env_->OptimizeForManifestWrite(storage_options_)); - if (!s.ok()) { - return s; - } - file->SetPreallocationBlockSize(options_.manifest_preallocation_size); - { - log::Writer log(std::move(file)); - std::string record; - new_db.EncodeTo(&record); - s = log.AddRecord(record); - } - if (s.ok()) { - // Make "CURRENT" file that points to the new manifest file. - s = SetCurrentFile(env_, dbname_, 1, db_directory_.get()); - } else { - env_->DeleteFile(manifest); - } - return s; + ROCKS_LOG_INFO(immutable_db_options_.info_log, "Shutdown complete"); + LogFlush(immutable_db_options_.info_log); } void DBImpl::MaybeIgnoreError(Status* s) const { - if (s->ok() || options_.paranoid_checks) { + if (s->ok() || immutable_db_options_.paranoid_checks) { // No change needed } else { - Log(options_.info_log, "Ignoring error %s", s->ToString().c_str()); + ROCKS_LOG_WARN(immutable_db_options_.info_log, "Ignoring error %s", + s->ToString().c_str()); *s = Status::OK(); } } const Status DBImpl::CreateArchivalDirectory() { - if (options_.WAL_ttl_seconds > 0 || options_.WAL_size_limit_MB > 0) { - std::string archivalPath = ArchivalDirectory(options_.wal_dir); + if (immutable_db_options_.wal_ttl_seconds > 0 || + immutable_db_options_.wal_size_limit_mb > 0) { + std::string archivalPath = ArchivalDirectory(immutable_db_options_.wal_dir); return env_->CreateDirIfMissing(archivalPath); } return Status::OK(); } void DBImpl::PrintStatistics() { - auto dbstats = options_.statistics.get(); + auto dbstats = immutable_db_options_.statistics.get(); if (dbstats) { - Log(options_.info_log, - "STATISTCS:\n %s", - dbstats->ToString().c_str()); + ROCKS_LOG_WARN(immutable_db_options_.info_log, "STATISTICS:\n %s", + dbstats->ToString().c_str()); } } void DBImpl::MaybeDumpStats() { - if (options_.stats_dump_period_sec == 0) return; + mutex_.Lock(); + unsigned int stats_dump_period_sec = + mutable_db_options_.stats_dump_period_sec; + mutex_.Unlock(); + if (stats_dump_period_sec == 0) return; const uint64_t now_micros = env_->NowMicros(); - if (last_stats_dump_time_microsec_ + - options_.stats_dump_period_sec * 1000000 - <= now_micros) { + if (last_stats_dump_time_microsec_ + stats_dump_period_sec * 1000000 <= + now_micros) { // Multiple threads could race in here simultaneously. // However, the last one will update last_stats_dump_time_microsec_ // atomically. We could see more than one dump during one dump // period in rare cases. last_stats_dump_time_microsec_ = now_micros; - bool tmp1 = false; - bool tmp2 = false; - DBPropertyType cf_property_type = - GetPropertyType("rocksdb.cfstats", &tmp1, &tmp2); - DBPropertyType db_property_type = - GetPropertyType("rocksdb.dbstats", &tmp1, &tmp2); +#ifndef ROCKSDB_LITE + const DBPropertyInfo* cf_property_info = + GetPropertyInfo(DB::Properties::kCFStats); + assert(cf_property_info != nullptr); + const DBPropertyInfo* db_property_info = + GetPropertyInfo(DB::Properties::kDBStats); + assert(db_property_info != nullptr); + std::string stats; { - MutexLock l(&mutex_); + InstrumentedMutexLock l(&mutex_); + default_cf_internal_stats_->GetStringProperty( + *db_property_info, DB::Properties::kDBStats, &stats); for (auto cfd : *versions_->GetColumnFamilySet()) { - cfd->internal_stats()->GetStringProperty(cf_property_type, - "rocksdb.cfstats", &stats); + if (cfd->initialized()) { + cfd->internal_stats()->GetStringProperty( + *cf_property_info, DB::Properties::kCFStatsNoFileHistogram, + &stats); + } + } + for (auto cfd : *versions_->GetColumnFamilySet()) { + if (cfd->initialized()) { + cfd->internal_stats()->GetStringProperty( + *cf_property_info, DB::Properties::kCFFileHistogram, &stats); + } } - default_cf_internal_stats_->GetStringProperty(db_property_type, - "rocksdb.dbstats", &stats); } - Log(options_.info_log, "------- DUMPING STATS -------"); - Log(options_.info_log, "%s", stats.c_str()); + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "------- DUMPING STATS -------"); + ROCKS_LOG_WARN(immutable_db_options_.info_log, "%s", stats.c_str()); + if (immutable_db_options_.dump_malloc_stats) { + stats.clear(); + DumpMallocStats(&stats); + if (!stats.empty()) { + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "------- Malloc STATS -------"); + ROCKS_LOG_WARN(immutable_db_options_.info_log, "%s", stats.c_str()); + } + } +#endif // !ROCKSDB_LITE PrintStatistics(); } } -// Returns the list of live files in 'sst_live' and the list -// of all files in the filesystem in 'candidate_files'. -// no_full_scan = true -- never do the full scan using GetChildren() -// force = false -- don't force the full scan, except every -// options_.delete_obsolete_files_period_micros -// force = true -- force the full scan -void DBImpl::FindObsoleteFiles(DeletionState& deletion_state, - bool force, - bool no_full_scan) { - mutex_.AssertHeld(); - - // if deletion is disabled, do nothing - if (disable_delete_obsolete_files_ > 0) { - return; - } - - bool doing_the_full_scan = false; - - // logic for figurint out if we're doing the full scan - if (no_full_scan) { - doing_the_full_scan = false; - } else if (force || options_.delete_obsolete_files_period_micros == 0) { - doing_the_full_scan = true; - } else { - const uint64_t now_micros = env_->NowMicros(); - if (delete_obsolete_files_last_run_ + - options_.delete_obsolete_files_period_micros < now_micros) { - doing_the_full_scan = true; - delete_obsolete_files_last_run_ = now_micros; +void DBImpl::ScheduleBgLogWriterClose(JobContext* job_context) { + if (!job_context->logs_to_free.empty()) { + for (auto l : job_context->logs_to_free) { + AddToLogsToFreeQueue(l); } + job_context->logs_to_free.clear(); + SchedulePurge(); } +} - // get obsolete files - versions_->GetObsoleteFiles(&deletion_state.sst_delete_files); +Directory* DBImpl::Directories::GetDataDir(size_t path_id) { + assert(path_id < data_dirs_.size()); + Directory* ret_dir = data_dirs_[path_id].get(); + if (ret_dir == nullptr) { + // Should use db_dir_ + return db_dir_.get(); + } + return ret_dir; +} - // store the current filenum, lognum, etc - deletion_state.manifest_file_number = versions_->ManifestFileNumber(); - deletion_state.pending_manifest_file_number = - versions_->PendingManifestFileNumber(); - deletion_state.log_number = versions_->MinLogNumber(); - deletion_state.prev_log_number = versions_->PrevLogNumber(); +Status DBImpl::SetOptions(ColumnFamilyHandle* column_family, + const std::unordered_map& options_map) { +#ifdef ROCKSDB_LITE + return Status::NotSupported("Not supported in ROCKSDB LITE"); +#else + auto* cfd = reinterpret_cast(column_family)->cfd(); + if (options_map.empty()) { + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "SetOptions() on column family [%s], empty input", + cfd->GetName().c_str()); + return Status::InvalidArgument("empty input"); + } - if (!doing_the_full_scan && !deletion_state.HaveSomethingToDelete()) { - // avoid filling up sst_live if we're sure that we - // are not going to do the full scan and that we don't have - // anything to delete at the moment - return; + MutableCFOptions new_options; + Status s; + Status persist_options_status; + WriteThread::Writer w; + { + InstrumentedMutexLock l(&mutex_); + s = cfd->SetOptions(options_map); + if (s.ok()) { + new_options = *cfd->GetLatestMutableCFOptions(); + // Append new version to recompute compaction score. + VersionEdit dummy_edit; + versions_->LogAndApply(cfd, new_options, &dummy_edit, &mutex_, + directories_.GetDbDir()); + // Trigger possible flush/compactions. This has to be before we persist + // options to file, otherwise there will be a deadlock with writer + // thread. + auto* old_sv = + InstallSuperVersionAndScheduleWork(cfd, nullptr, new_options); + delete old_sv; + + persist_options_status = WriteOptionsFile( + false /*need_mutex_lock*/, true /*need_enter_write_thread*/); + } + } + + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "SetOptions() on column family [%s], inputs:", + cfd->GetName().c_str()); + for (const auto& o : options_map) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, "%s: %s\n", o.first.c_str(), + o.second.c_str()); + } + if (s.ok()) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "[%s] SetOptions() succeeded", cfd->GetName().c_str()); + new_options.Dump(immutable_db_options_.info_log.get()); + if (!persist_options_status.ok()) { + s = persist_options_status; + } + } else { + ROCKS_LOG_WARN(immutable_db_options_.info_log, "[%s] SetOptions() failed", + cfd->GetName().c_str()); } + LogFlush(immutable_db_options_.info_log); + return s; +#endif // ROCKSDB_LITE +} - // don't delete live files - for (auto pair : pending_outputs_) { - deletion_state.sst_live.emplace_back(pair.first, pair.second, 0); +Status DBImpl::SetDBOptions( + const std::unordered_map& options_map) { +#ifdef ROCKSDB_LITE + return Status::NotSupported("Not supported in ROCKSDB LITE"); +#else + if (options_map.empty()) { + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "SetDBOptions(), empty input."); + return Status::InvalidArgument("empty input"); } - /* deletion_state.sst_live.insert(pending_outputs_.begin(), - pending_outputs_.end());*/ - versions_->AddLiveFiles(&deletion_state.sst_live); - if (doing_the_full_scan) { - for (uint32_t path_id = 0; path_id < options_.db_paths.size(); path_id++) { - // set of all files in the directory. We'll exclude files that are still - // alive in the subsequent processings. - std::vector files; - env_->GetChildren(options_.db_paths[path_id].path, - &files); // Ignore errors - for (std::string file : files) { - deletion_state.candidate_files.emplace_back(file, path_id); + MutableDBOptions new_options; + Status s; + Status persist_options_status; + WriteThread::Writer w; + WriteContext write_context; + { + InstrumentedMutexLock l(&mutex_); + s = GetMutableDBOptionsFromStrings(mutable_db_options_, options_map, + &new_options); + if (s.ok()) { + if (new_options.max_background_compactions > + mutable_db_options_.max_background_compactions) { + env_->IncBackgroundThreadsIfNeeded( + new_options.max_background_compactions, Env::Priority::LOW); + MaybeScheduleFlushOrCompaction(); } - } - //Add log files in wal_dir - if (options_.wal_dir != dbname_) { - std::vector log_files; - env_->GetChildren(options_.wal_dir, &log_files); // Ignore errors - for (std::string log_file : log_files) { - deletion_state.candidate_files.emplace_back(log_file, 0); - } - } - // Add info log files in db_log_dir - if (!options_.db_log_dir.empty() && options_.db_log_dir != dbname_) { - std::vector info_log_files; - env_->GetChildren(options_.db_log_dir, &info_log_files); // Ignore errors - for (std::string log_file : info_log_files) { - deletion_state.candidate_files.emplace_back(log_file, 0); + write_controller_.set_max_delayed_write_rate(new_options.delayed_write_rate); + table_cache_.get()->SetCapacity(new_options.max_open_files == -1 + ? TableCache::kInfiniteCapacity + : new_options.max_open_files - 10); + + mutable_db_options_ = new_options; + + write_thread_.EnterUnbatched(&w, &mutex_); + if (total_log_size_ > GetMaxTotalWalSize()) { + Status purge_wal_status = HandleWALFull(&write_context); + if (!purge_wal_status.ok()) { + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "Unable to purge WAL files in SetDBOptions() -- %s", + purge_wal_status.ToString().c_str()); + } } + persist_options_status = WriteOptionsFile( + false /*need_mutex_lock*/, false /*need_enter_write_thread*/); + write_thread_.ExitUnbatched(&w); } } -} - -namespace { -bool CompareCandidateFile(const rocksdb::DBImpl::CandidateFileInfo& first, - const rocksdb::DBImpl::CandidateFileInfo& second) { - if (first.file_name > second.file_name) { - return true; - } else if (first.file_name < second.file_name) { - return false; + ROCKS_LOG_INFO(immutable_db_options_.info_log, "SetDBOptions(), inputs:"); + for (const auto& o : options_map) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, "%s: %s\n", o.first.c_str(), + o.second.c_str()); + } + if (s.ok()) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, "SetDBOptions() succeeded"); + new_options.Dump(immutable_db_options_.info_log.get()); + if (!persist_options_status.ok()) { + if (immutable_db_options_.fail_if_options_file_error) { + s = Status::IOError( + "SetDBOptions() succeeded, but unable to persist options", + persist_options_status.ToString()); + } + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "Unable to persist options in SetDBOptions() -- %s", + persist_options_status.ToString().c_str()); + } } else { - return (first.path_id > second.path_id); + ROCKS_LOG_WARN(immutable_db_options_.info_log, "SetDBOptions failed"); } + LogFlush(immutable_db_options_.info_log); + return s; +#endif // ROCKSDB_LITE } -}; // namespace - -// Diffs the files listed in filenames and those that do not -// belong to live files are posibly removed. Also, removes all the -// files in sst_delete_files and log_delete_files. -// It is not necessary to hold the mutex when invoking this method. -void DBImpl::PurgeObsoleteFiles(DeletionState& state) { - // we'd better have sth to delete - assert(state.HaveSomethingToDelete()); - - // this checks if FindObsoleteFiles() was run before. If not, don't do - // PurgeObsoleteFiles(). If FindObsoleteFiles() was run, we need to also - // run PurgeObsoleteFiles(), even if disable_delete_obsolete_files_ is true - if (state.manifest_file_number == 0) { - return; - } - // Now, convert live list to an unordered map, WITHOUT mutex held; - // set is slow. - std::unordered_map sst_live_map; - for (FileDescriptor& fd : state.sst_live) { - sst_live_map[fd.GetNumber()] = &fd; - } +// return the same level if it cannot be moved +int DBImpl::FindMinimumEmptyLevelFitting(ColumnFamilyData* cfd, + const MutableCFOptions& mutable_cf_options, int level) { + mutex_.AssertHeld(); + const auto* vstorage = cfd->current()->storage_info(); + int minimum_level = level; + for (int i = level - 1; i > 0; --i) { + // stop if level i is not empty + if (vstorage->NumLevelFiles(i) > 0) break; + // stop if level i is too small (cannot fit the level files) + if (vstorage->MaxBytesForLevel(i) < vstorage->NumLevelBytes(level)) { + break; + } - auto& candidate_files = state.candidate_files; - candidate_files.reserve( - candidate_files.size() + - state.sst_delete_files.size() + - state.log_delete_files.size()); - // We may ignore the dbname when generating the file names. - const char* kDumbDbName = ""; - for (auto file : state.sst_delete_files) { - candidate_files.emplace_back( - MakeTableFileName(kDumbDbName, file->fd.GetNumber()), - file->fd.GetPathId()); - delete file; + minimum_level = i; } + return minimum_level; +} - for (auto file_num : state.log_delete_files) { - if (file_num > 0) { - candidate_files.emplace_back(LogFileName(kDumbDbName, file_num).substr(1), - 0); +Status DBImpl::FlushWAL(bool sync) { + { + // We need to lock log_write_mutex_ since logs_ might change concurrently + InstrumentedMutexLock wl(&log_write_mutex_); + log::Writer* cur_log_writer = logs_.back().writer; + auto s = cur_log_writer->WriteBuffer(); + if (!s.ok()) { + ROCKS_LOG_ERROR(immutable_db_options_.info_log, "WAL flush error %s", + s.ToString().c_str()); + } + if (!sync) { + ROCKS_LOG_DEBUG(immutable_db_options_.info_log, "FlushWAL sync=false"); + return s; } } + // sync = true + ROCKS_LOG_DEBUG(immutable_db_options_.info_log, "FlushWAL sync=true"); + return SyncWAL(); +} - // dedup state.candidate_files so we don't try to delete the same - // file twice - sort(candidate_files.begin(), candidate_files.end(), CompareCandidateFile); - candidate_files.erase(unique(candidate_files.begin(), candidate_files.end()), - candidate_files.end()); +Status DBImpl::SyncWAL() { + autovector logs_to_sync; + bool need_log_dir_sync; + uint64_t current_log_number; - std::vector old_info_log_files; - InfoLogPrefix info_log_prefix(!options_.db_log_dir.empty(), dbname_); - for (const auto& candidate_file : candidate_files) { - std::string to_delete = candidate_file.file_name; - uint32_t path_id = candidate_file.path_id; - uint64_t number; - FileType type; - // Ignore file if we cannot recognize it. - if (!ParseFileName(to_delete, &number, info_log_prefix.prefix, &type)) { - continue; - } + { + InstrumentedMutexLock l(&mutex_); + assert(!logs_.empty()); - bool keep = true; - switch (type) { - case kLogFile: - keep = ((number >= state.log_number) || - (number == state.prev_log_number)); - break; - case kDescriptorFile: - // Keep my manifest file, and any newer incarnations' - // (can happen during manifest roll) - keep = (number >= state.manifest_file_number); - break; - case kTableFile: - keep = (sst_live_map.find(number) != sst_live_map.end()); - break; - case kTempFile: - // Any temp files that are currently being written to must - // be recorded in pending_outputs_, which is inserted into "live". - // Also, SetCurrentFile creates a temp file when writing out new - // manifest, which is equal to state.pending_manifest_file_number. We - // should not delete that file - keep = (sst_live_map.find(number) != sst_live_map.end()) || - (number == state.pending_manifest_file_number); - break; - case kInfoLogFile: - keep = true; - if (number != 0) { - old_info_log_files.push_back(to_delete); - } - break; - case kCurrentFile: - case kDBLockFile: - case kIdentityFile: - case kMetaDatabase: - keep = true; - break; - } + // This SyncWAL() call only cares about logs up to this number. + current_log_number = logfile_number_; - if (keep) { - continue; + while (logs_.front().number <= current_log_number && + logs_.front().getting_synced) { + log_sync_cv_.Wait(); } - - std::string fname; - if (type == kTableFile) { - // evict from cache - TableCache::Evict(table_cache_.get(), number); - fname = TableFileName(options_.db_paths, number, path_id); - } else { - fname = - ((type == kLogFile) ? options_.wal_dir : dbname_) + "/" + to_delete; + // First check that logs are safe to sync in background. + for (auto it = logs_.begin(); + it != logs_.end() && it->number <= current_log_number; ++it) { + if (!it->writer->file()->writable_file()->IsSyncThreadSafe()) { + return Status::NotSupported( + "SyncWAL() is not supported for this implementation of WAL file", + immutable_db_options_.allow_mmap_writes + ? "try setting Options::allow_mmap_writes to false" + : Slice()); + } } - - if (type == kLogFile && - (options_.WAL_ttl_seconds > 0 || options_.WAL_size_limit_MB > 0)) { - auto archived_log_name = ArchivedLogFileName(options_.wal_dir, number); - // The sync point below is used in (DBTest,TransactionLogIteratorRace) - TEST_SYNC_POINT("DBImpl::PurgeObsoleteFiles:1"); - Status s = env_->RenameFile(fname, archived_log_name); - // The sync point below is used in (DBTest,TransactionLogIteratorRace) - TEST_SYNC_POINT("DBImpl::PurgeObsoleteFiles:2"); - Log(options_.info_log, - "Move log file %s to %s -- %s\n", - fname.c_str(), archived_log_name.c_str(), s.ToString().c_str()); - } else { - Status s = env_->DeleteFile(fname); - Log(options_.info_log, "Delete %s type=%d #%" PRIu64 " -- %s\n", - fname.c_str(), type, number, s.ToString().c_str()); + for (auto it = logs_.begin(); + it != logs_.end() && it->number <= current_log_number; ++it) { + auto& log = *it; + assert(!log.getting_synced); + log.getting_synced = true; + logs_to_sync.push_back(log.writer); } + + need_log_dir_sync = !log_dir_synced_; } - // Delete old info log files. - size_t old_info_log_file_count = old_info_log_files.size(); - if (old_info_log_file_count >= options_.keep_log_file_num) { - std::sort(old_info_log_files.begin(), old_info_log_files.end()); - size_t end = old_info_log_file_count - options_.keep_log_file_num; - for (unsigned int i = 0; i <= end; i++) { - std::string& to_delete = old_info_log_files.at(i); - std::string full_path_to_delete = - (options_.db_log_dir.empty() ? dbname_ : options_.db_log_dir) + "/" + - to_delete; - Log(options_.info_log, "Delete info log file %s\n", - full_path_to_delete.c_str()); - Status s = env_->DeleteFile(full_path_to_delete); - if (!s.ok()) { - Log(options_.info_log, "Delete info log file %s FAILED -- %s\n", - to_delete.c_str(), s.ToString().c_str()); - } + TEST_SYNC_POINT("DBWALTest::SyncWALNotWaitWrite:1"); + RecordTick(stats_, WAL_FILE_SYNCED); + Status status; + for (log::Writer* log : logs_to_sync) { + status = log->file()->SyncWithoutFlush(immutable_db_options_.use_fsync); + if (!status.ok()) { + break; } } - PurgeObsoleteWALFiles(); - LogFlush(options_.info_log); + if (status.ok() && need_log_dir_sync) { + status = directories_.GetWalDir()->Fsync(); + } + TEST_SYNC_POINT("DBWALTest::SyncWALNotWaitWrite:2"); + + TEST_SYNC_POINT("DBImpl::SyncWAL:BeforeMarkLogsSynced:1"); + { + InstrumentedMutexLock l(&mutex_); + MarkLogsSynced(current_log_number, need_log_dir_sync, status); + } + TEST_SYNC_POINT("DBImpl::SyncWAL:BeforeMarkLogsSynced:2"); + + return status; } -void DBImpl::DeleteObsoleteFiles() { +void DBImpl::MarkLogsSynced( + uint64_t up_to, bool synced_dir, const Status& status) { mutex_.AssertHeld(); - DeletionState deletion_state; - FindObsoleteFiles(deletion_state, true); - if (deletion_state.HaveSomethingToDelete()) { - PurgeObsoleteFiles(deletion_state); + if (synced_dir && + logfile_number_ == up_to && + status.ok()) { + log_dir_synced_ = true; + } + for (auto it = logs_.begin(); it != logs_.end() && it->number <= up_to;) { + auto& log = *it; + assert(log.getting_synced); + if (status.ok() && logs_.size() > 1) { + logs_to_free_.push_back(log.ReleaseWriter()); + it = logs_.erase(it); + } else { + log.getting_synced = false; + ++it; + } } + assert(!status.ok() || logs_.empty() || logs_[0].number > up_to || + (logs_.size() == 1 && !logs_[0].getting_synced)); + log_sync_cv_.SignalAll(); } -#ifndef ROCKSDB_LITE -// 1. Go through all archived files and -// a. if ttl is enabled, delete outdated files -// b. if archive size limit is enabled, delete empty files, -// compute file number and size. -// 2. If size limit is enabled: -// a. compute how many files should be deleted -// b. get sorted non-empty archived logs -// c. delete what should be deleted -void DBImpl::PurgeObsoleteWALFiles() { - bool const ttl_enabled = options_.WAL_ttl_seconds > 0; - bool const size_limit_enabled = options_.WAL_size_limit_MB > 0; - if (!ttl_enabled && !size_limit_enabled) { - return; - } +SequenceNumber DBImpl::GetLatestSequenceNumber() const { + return versions_->LastSequence(); +} - int64_t current_time; - Status s = env_->GetCurrentTime(¤t_time); - if (!s.ok()) { - Log(options_.info_log, "Can't get current time: %s", s.ToString().c_str()); - assert(false); - return; - } - uint64_t const now_seconds = static_cast(current_time); - uint64_t const time_to_check = (ttl_enabled && !size_limit_enabled) ? - options_.WAL_ttl_seconds / 2 : default_interval_to_delete_obsolete_WAL_; - - if (purge_wal_files_last_run_ + time_to_check > now_seconds) { - return; - } - - purge_wal_files_last_run_ = now_seconds; - - std::string archival_dir = ArchivalDirectory(options_.wal_dir); - std::vector files; - s = env_->GetChildren(archival_dir, &files); - if (!s.ok()) { - Log(options_.info_log, "Can't get archive files: %s", s.ToString().c_str()); - assert(false); - return; - } - - size_t log_files_num = 0; - uint64_t log_file_size = 0; - - for (auto& f : files) { - uint64_t number; - FileType type; - if (ParseFileName(f, &number, &type) && type == kLogFile) { - std::string const file_path = archival_dir + "/" + f; - if (ttl_enabled) { - uint64_t file_m_time; - Status const s = env_->GetFileModificationTime(file_path, - &file_m_time); - if (!s.ok()) { - Log(options_.info_log, "Can't get file mod time: %s: %s", - file_path.c_str(), s.ToString().c_str()); - continue; - } - if (now_seconds - file_m_time > options_.WAL_ttl_seconds) { - Status const s = env_->DeleteFile(file_path); - if (!s.ok()) { - Log(options_.info_log, "Can't delete file: %s: %s", - file_path.c_str(), s.ToString().c_str()); - continue; - } else { - MutexLock l(&read_first_record_cache_mutex_); - read_first_record_cache_.erase(number); - } - continue; - } - } - - if (size_limit_enabled) { - uint64_t file_size; - Status const s = env_->GetFileSize(file_path, &file_size); - if (!s.ok()) { - Log(options_.info_log, "Can't get file size: %s: %s", - file_path.c_str(), s.ToString().c_str()); - return; - } else { - if (file_size > 0) { - log_file_size = std::max(log_file_size, file_size); - ++log_files_num; - } else { - Status s = env_->DeleteFile(file_path); - if (!s.ok()) { - Log(options_.info_log, "Can't delete file: %s: %s", - file_path.c_str(), s.ToString().c_str()); - continue; - } else { - MutexLock l(&read_first_record_cache_mutex_); - read_first_record_cache_.erase(number); - } - } - } - } - } - } - - if (0 == log_files_num || !size_limit_enabled) { - return; - } - - size_t const files_keep_num = options_.WAL_size_limit_MB * - 1024 * 1024 / log_file_size; - if (log_files_num <= files_keep_num) { - return; - } - - size_t files_del_num = log_files_num - files_keep_num; - VectorLogPtr archived_logs; - GetSortedWalsOfType(archival_dir, archived_logs, kArchivedLogFile); - - if (files_del_num > archived_logs.size()) { - Log(options_.info_log, "Trying to delete more archived log files than " - "exist. Deleting all"); - files_del_num = archived_logs.size(); - } - - for (size_t i = 0; i < files_del_num; ++i) { - std::string const file_path = archived_logs[i]->PathName(); - Status const s = DeleteFile(file_path); - if (!s.ok()) { - Log(options_.info_log, "Can't delete file: %s: %s", - file_path.c_str(), s.ToString().c_str()); - continue; - } else { - MutexLock l(&read_first_record_cache_mutex_); - read_first_record_cache_.erase(archived_logs[i]->LogNumber()); - } - } -} - -namespace { -struct CompareLogByPointer { - bool operator()(const unique_ptr& a, const unique_ptr& b) { - LogFileImpl* a_impl = dynamic_cast(a.get()); - LogFileImpl* b_impl = dynamic_cast(b.get()); - return *a_impl < *b_impl; - } -}; -} - -Status DBImpl::GetSortedWalsOfType(const std::string& path, - VectorLogPtr& log_files, - WalFileType log_type) { - std::vector all_files; - const Status status = env_->GetChildren(path, &all_files); - if (!status.ok()) { - return status; - } - log_files.reserve(all_files.size()); - for (const auto& f : all_files) { - uint64_t number; - FileType type; - if (ParseFileName(f, &number, &type) && type == kLogFile) { - SequenceNumber sequence; - Status s = ReadFirstRecord(log_type, number, &sequence); - if (!s.ok()) { - return s; - } - if (sequence == 0) { - // empty file - continue; - } - - // Reproduce the race condition where a log file is moved - // to archived dir, between these two sync points, used in - // (DBTest,TransactionLogIteratorRace) - TEST_SYNC_POINT("DBImpl::GetSortedWalsOfType:1"); - TEST_SYNC_POINT("DBImpl::GetSortedWalsOfType:2"); - - uint64_t size_bytes; - s = env_->GetFileSize(LogFileName(path, number), &size_bytes); - // re-try in case the alive log file has been moved to archive. - if (!s.ok() && log_type == kAliveLogFile && - env_->FileExists(ArchivedLogFileName(path, number))) { - s = env_->GetFileSize(ArchivedLogFileName(path, number), &size_bytes); - } - if (!s.ok()) { - return s; - } - - log_files.push_back(std::move(unique_ptr( - new LogFileImpl(number, log_type, sequence, size_bytes)))); - } - } - CompareLogByPointer compare_log_files; - std::sort(log_files.begin(), log_files.end(), compare_log_files); - return status; -} - -Status DBImpl::RetainProbableWalFiles(VectorLogPtr& all_logs, - const SequenceNumber target) { - int64_t start = 0; // signed to avoid overflow when target is < first file. - int64_t end = static_cast(all_logs.size()) - 1; - // Binary Search. avoid opening all files. - while (end >= start) { - int64_t mid = start + (end - start) / 2; // Avoid overflow. - SequenceNumber current_seq_num = all_logs.at(mid)->StartSequence(); - if (current_seq_num == target) { - end = mid; - break; - } else if (current_seq_num < target) { - start = mid + 1; - } else { - end = mid - 1; - } - } - // end could be -ve. - size_t start_index = std::max(static_cast(0), end); - // The last wal file is always included - all_logs.erase(all_logs.begin(), all_logs.begin() + start_index); - return Status::OK(); -} - -Status DBImpl::ReadFirstRecord(const WalFileType type, const uint64_t number, - SequenceNumber* sequence) { - if (type != kAliveLogFile && type != kArchivedLogFile) { - return Status::NotSupported("File Type Not Known " + std::to_string(type)); - } - { - MutexLock l(&read_first_record_cache_mutex_); - auto itr = read_first_record_cache_.find(number); - if (itr != read_first_record_cache_.end()) { - *sequence = itr->second; - return Status::OK(); - } - } - Status s; - if (type == kAliveLogFile) { - std::string fname = LogFileName(options_.wal_dir, number); - s = ReadFirstLine(fname, sequence); - if (env_->FileExists(fname) && !s.ok()) { - // return any error that is not caused by non-existing file - return s; - } - } - - if (type == kArchivedLogFile || !s.ok()) { - // check if the file got moved to archive. - std::string archived_file = ArchivedLogFileName(options_.wal_dir, number); - s = ReadFirstLine(archived_file, sequence); - } - - if (s.ok() && *sequence != 0) { - MutexLock l(&read_first_record_cache_mutex_); - read_first_record_cache_.insert({number, *sequence}); - } - return s; -} - -// the function returns status.ok() and sequence == 0 if the file exists, but is -// empty -Status DBImpl::ReadFirstLine(const std::string& fname, - SequenceNumber* sequence) { - struct LogReporter : public log::Reader::Reporter { - Env* env; - Logger* info_log; - const char* fname; - - Status* status; - bool ignore_error; // true if options_.paranoid_checks==false - virtual void Corruption(size_t bytes, const Status& s) { - Log(info_log, "%s%s: dropping %d bytes; %s", - (this->ignore_error ? "(ignoring error) " : ""), fname, - static_cast(bytes), s.ToString().c_str()); - if (this->status->ok()) { - // only keep the first error - *this->status = s; - } - } - }; - - unique_ptr file; - Status status = env_->NewSequentialFile(fname, &file, storage_options_); - - if (!status.ok()) { - return status; - } - - LogReporter reporter; - reporter.env = env_; - reporter.info_log = options_.info_log.get(); - reporter.fname = fname.c_str(); - reporter.status = &status; - reporter.ignore_error = !options_.paranoid_checks; - log::Reader reader(std::move(file), &reporter, true /*checksum*/, - 0 /*initial_offset*/); - std::string scratch; - Slice record; - - if (reader.ReadRecord(&record, &scratch) && - (status.ok() || !options_.paranoid_checks)) { - if (record.size() < 12) { - reporter.Corruption(record.size(), - Status::Corruption("log record too small")); - // TODO read record's till the first no corrupt entry? - } else { - WriteBatch batch; - WriteBatchInternal::SetContents(&batch, record); - *sequence = WriteBatchInternal::Sequence(&batch); - return Status::OK(); - } - } - - // ReadRecord returns false on EOF, which means that the log file is empty. we - // return status.ok() in that case and set sequence number to 0 - *sequence = 0; - return status; -} - -#endif // ROCKSDB_LITE - -Status DBImpl::Recover( - const std::vector& column_families, bool read_only, - bool error_if_log_file_exist) { - mutex_.AssertHeld(); - - bool is_new_db = false; - assert(db_lock_ == nullptr); - if (!read_only) { - // We call CreateDirIfMissing() as the directory may already exist (if we - // are reopening a DB), when this happens we don't want creating the - // directory to cause an error. However, we need to check if creating the - // directory fails or else we may get an obscure message about the lock - // file not existing. One real-world example of this occurring is if - // env->CreateDirIfMissing() doesn't create intermediate directories, e.g. - // when dbname_ is "dir/db" but when "dir" doesn't exist. - Status s = env_->CreateDirIfMissing(dbname_); - if (!s.ok()) { - return s; - } - - for (auto& db_path : options_.db_paths) { - s = env_->CreateDirIfMissing(db_path.path); - if (!s.ok()) { - return s; - } - } - - s = env_->NewDirectory(dbname_, &db_directory_); - if (!s.ok()) { - return s; - } - - s = env_->LockFile(LockFileName(dbname_), &db_lock_); - if (!s.ok()) { - return s; - } - - if (!env_->FileExists(CurrentFileName(dbname_))) { - if (options_.create_if_missing) { - s = NewDB(); - is_new_db = true; - if (!s.ok()) { - return s; - } - } else { - return Status::InvalidArgument( - dbname_, "does not exist (create_if_missing is false)"); - } - } else { - if (options_.error_if_exists) { - return Status::InvalidArgument( - dbname_, "exists (error_if_exists is true)"); - } - } - // Check for the IDENTITY file and create it if not there - if (!env_->FileExists(IdentityFileName(dbname_))) { - s = SetIdentityFile(env_, dbname_); - if (!s.ok()) { - return s; - } - } - } - - Status s = versions_->Recover(column_families, read_only); - if (options_.paranoid_checks && s.ok()) { - s = CheckConsistency(); - } - if (s.ok()) { - SequenceNumber max_sequence(0); - default_cf_handle_ = new ColumnFamilyHandleImpl( - versions_->GetColumnFamilySet()->GetDefault(), this, &mutex_); - default_cf_internal_stats_ = default_cf_handle_->cfd()->internal_stats(); - single_column_family_mode_ = - versions_->GetColumnFamilySet()->NumberOfColumnFamilies() == 1; - - // Recover from all newer log files than the ones named in the - // descriptor (new log files may have been added by the previous - // incarnation without registering them in the descriptor). - // - // Note that PrevLogNumber() is no longer used, but we pay - // attention to it in case we are recovering a database - // produced by an older version of rocksdb. - const uint64_t min_log = versions_->MinLogNumber(); - const uint64_t prev_log = versions_->PrevLogNumber(); - std::vector filenames; - s = env_->GetChildren(options_.wal_dir, &filenames); - if (!s.ok()) { - return s; - } - - std::vector logs; - for (size_t i = 0; i < filenames.size(); i++) { - uint64_t number; - FileType type; - if (ParseFileName(filenames[i], &number, &type) && type == kLogFile) { - if (is_new_db) { - return Status::Corruption( - "While creating a new Db, wal_dir contains " - "existing log file: ", - filenames[i]); - } else if ((number >= min_log) || (number == prev_log)) { - logs.push_back(number); - } - } - } - - if (logs.size() > 0 && error_if_log_file_exist) { - return Status::Corruption("" - "The db was opened in readonly mode with error_if_log_file_exist" - "flag but a log file already exists"); - } - - // Recover in the order in which the logs were generated - std::sort(logs.begin(), logs.end()); - for (const auto& log : logs) { - // The previous incarnation may not have written any MANIFEST - // records after allocating this log number. So we manually - // update the file number allocation counter in VersionSet. - versions_->MarkFileNumberUsed(log); - s = RecoverLogFile(log, &max_sequence, read_only); - } - SetTickerCount(stats_, SEQUENCE_NUMBER, versions_->LastSequence()); - } - - for (auto cfd : *versions_->GetColumnFamilySet()) { - max_total_in_memory_state_ += cfd->options()->write_buffer_size * - cfd->options()->max_write_buffer_number; - } - - return s; -} - -Status DBImpl::RecoverLogFile(uint64_t log_number, SequenceNumber* max_sequence, - bool read_only) { - struct LogReporter : public log::Reader::Reporter { - Env* env; - Logger* info_log; - const char* fname; - Status* status; // nullptr if options_.paranoid_checks==false or - // options_.skip_log_error_on_recovery==true - virtual void Corruption(size_t bytes, const Status& s) { - Log(info_log, "%s%s: dropping %d bytes; %s", - (this->status == nullptr ? "(ignoring error) " : ""), - fname, static_cast(bytes), s.ToString().c_str()); - if (this->status != nullptr && this->status->ok()) *this->status = s; - } - }; - - mutex_.AssertHeld(); - - std::unordered_map version_edits; - // no need to refcount because iteration is under mutex - for (auto cfd : *versions_->GetColumnFamilySet()) { - VersionEdit edit; - edit.SetColumnFamily(cfd->GetID()); - version_edits.insert({cfd->GetID(), edit}); - } - - // Open the log file - std::string fname = LogFileName(options_.wal_dir, log_number); - unique_ptr file; - Status status = env_->NewSequentialFile(fname, &file, storage_options_); - if (!status.ok()) { - MaybeIgnoreError(&status); - return status; - } - - // Create the log reader. - LogReporter reporter; - reporter.env = env_; - reporter.info_log = options_.info_log.get(); - reporter.fname = fname.c_str(); - reporter.status = (options_.paranoid_checks && - !options_.skip_log_error_on_recovery ? &status : nullptr); - // We intentially make log::Reader do checksumming even if - // paranoid_checks==false so that corruptions cause entire commits - // to be skipped instead of propagating bad information (like overly - // large sequence numbers). - log::Reader reader(std::move(file), &reporter, true/*checksum*/, - 0/*initial_offset*/); - Log(options_.info_log, "Recovering log #%" PRIu64 "", log_number); - - // Read all the records and add to a memtable - std::string scratch; - Slice record; - WriteBatch batch; - while (reader.ReadRecord(&record, &scratch)) { - if (record.size() < 12) { - reporter.Corruption(record.size(), - Status::Corruption("log record too small")); - continue; - } - WriteBatchInternal::SetContents(&batch, record); - - // If column family was not found, it might mean that the WAL write - // batch references to the column family that was dropped after the - // insert. We don't want to fail the whole write batch in that case -- we - // just ignore the update. That's why we set ignore missing column families - // to true - status = WriteBatchInternal::InsertInto( - &batch, column_family_memtables_.get(), - true /* ignore missing column families */, log_number); - - MaybeIgnoreError(&status); - if (!status.ok()) { - return status; - } - const SequenceNumber last_seq = - WriteBatchInternal::Sequence(&batch) + - WriteBatchInternal::Count(&batch) - 1; - if (last_seq > *max_sequence) { - *max_sequence = last_seq; - } - - if (!read_only) { - // no need to refcount since client still doesn't have access - // to the DB and can not drop column families while we iterate - for (auto cfd : *versions_->GetColumnFamilySet()) { - if (cfd->mem()->ShouldFlush()) { - // If this asserts, it means that InsertInto failed in - // filtering updates to already-flushed column families - assert(cfd->GetLogNumber() <= log_number); - auto iter = version_edits.find(cfd->GetID()); - assert(iter != version_edits.end()); - VersionEdit* edit = &iter->second; - status = WriteLevel0TableForRecovery(cfd, cfd->mem(), edit); - // we still want to clear the memtable, even if the recovery failed - cfd->CreateNewMemtable(); - if (!status.ok()) { - // Reflect errors immediately so that conditions like full - // file-systems cause the DB::Open() to fail. - return status; - } - } - } - } - } - - if (versions_->LastSequence() < *max_sequence) { - versions_->SetLastSequence(*max_sequence); - } - - if (!read_only) { - // no need to refcount since client still doesn't have access - // to the DB and can not drop column families while we iterate - for (auto cfd : *versions_->GetColumnFamilySet()) { - auto iter = version_edits.find(cfd->GetID()); - assert(iter != version_edits.end()); - VersionEdit* edit = &iter->second; - - if (cfd->GetLogNumber() > log_number) { - // Column family cfd has already flushed the data - // from log_number. Memtable has to be empty because - // we filter the updates based on log_number - // (in WriteBatch::InsertInto) - assert(cfd->mem()->GetFirstSequenceNumber() == 0); - assert(edit->NumEntries() == 0); - continue; - } - - // flush the final memtable (if non-empty) - if (cfd->mem()->GetFirstSequenceNumber() != 0) { - status = WriteLevel0TableForRecovery(cfd, cfd->mem(), edit); - } - // we still want to clear the memtable, even if the recovery failed - cfd->CreateNewMemtable(); - if (!status.ok()) { - return status; - } - - // write MANIFEST with update - // writing log number in the manifest means that any log file - // with number strongly less than (log_number + 1) is already - // recovered and should be ignored on next reincarnation. - // Since we already recovered log_number, we want all logs - // with numbers `<= log_number` (includes this one) to be ignored - edit->SetLogNumber(log_number + 1); - // we must mark the next log number as used, even though it's - // not actually used. that is because VersionSet assumes - // VersionSet::next_file_number_ always to be strictly greater than any - // log number - versions_->MarkFileNumberUsed(log_number + 1); - status = versions_->LogAndApply(cfd, edit, &mutex_); - if (!status.ok()) { - return status; - } - } - } - - return status; -} - -Status DBImpl::WriteLevel0TableForRecovery(ColumnFamilyData* cfd, MemTable* mem, - VersionEdit* edit) { - mutex_.AssertHeld(); - const uint64_t start_micros = env_->NowMicros(); - FileMetaData meta; - meta.fd = FileDescriptor(versions_->NewFileNumber(), 0, 0); - pending_outputs_[meta.fd.GetNumber()] = 0; // path 0 for level 0 file. - ReadOptions ro; - ro.total_order_seek = true; - Iterator* iter = mem->NewIterator(ro); - const SequenceNumber newest_snapshot = snapshots_.GetNewest(); - const SequenceNumber earliest_seqno_in_memtable = - mem->GetFirstSequenceNumber(); - Log(options_.info_log, "[%s] Level-0 table #%" PRIu64 ": started", - cfd->GetName().c_str(), meta.fd.GetNumber()); - - Status s; - { - mutex_.Unlock(); - s = BuildTable(dbname_, env_, *cfd->options(), storage_options_, - cfd->table_cache(), iter, &meta, cfd->internal_comparator(), - newest_snapshot, earliest_seqno_in_memtable, - GetCompressionFlush(*cfd->options()), Env::IO_HIGH); - LogFlush(options_.info_log); - mutex_.Lock(); - } - - Log(options_.info_log, - "[%s] Level-0 table #%" PRIu64 ": %" PRIu64 " bytes %s", - cfd->GetName().c_str(), meta.fd.GetNumber(), meta.fd.GetFileSize(), - s.ToString().c_str()); - delete iter; - - pending_outputs_.erase(meta.fd.GetNumber()); - - // Note that if file_size is zero, the file has been deleted and - // should not be added to the manifest. - int level = 0; - if (s.ok() && meta.fd.GetFileSize() > 0) { - edit->AddFile(level, meta.fd.GetNumber(), meta.fd.GetPathId(), - meta.fd.GetFileSize(), meta.smallest, meta.largest, - meta.smallest_seqno, meta.largest_seqno); - } - - InternalStats::CompactionStats stats(1); - stats.micros = env_->NowMicros() - start_micros; - stats.bytes_written = meta.fd.GetFileSize(); - stats.files_out_levelnp1 = 1; - cfd->internal_stats()->AddCompactionStats(level, stats); - cfd->internal_stats()->AddCFStats( - InternalStats::BYTES_FLUSHED, meta.fd.GetFileSize()); - RecordTick(stats_, COMPACT_WRITE_BYTES, meta.fd.GetFileSize()); - return s; -} - -Status DBImpl::WriteLevel0Table(ColumnFamilyData* cfd, - autovector& mems, VersionEdit* edit, - uint64_t* filenumber, LogBuffer* log_buffer) { - mutex_.AssertHeld(); - const uint64_t start_micros = env_->NowMicros(); - FileMetaData meta; - - meta.fd = FileDescriptor(versions_->NewFileNumber(), 0, 0); - *filenumber = meta.fd.GetNumber(); - pending_outputs_[meta.fd.GetNumber()] = 0; // path 0 for level 0 file. - - const SequenceNumber newest_snapshot = snapshots_.GetNewest(); - const SequenceNumber earliest_seqno_in_memtable = - mems[0]->GetFirstSequenceNumber(); - Version* base = cfd->current(); - base->Ref(); // it is likely that we do not need this reference - Status s; - { - mutex_.Unlock(); - log_buffer->FlushBufferToLog(); - std::vector memtables; - ReadOptions ro; - ro.total_order_seek = true; - for (MemTable* m : mems) { - Log(options_.info_log, - "[%s] Flushing memtable with next log file: %" PRIu64 "\n", - cfd->GetName().c_str(), m->GetNextLogNumber()); - memtables.push_back(m->NewIterator(ro)); - } - Iterator* iter = NewMergingIterator(&cfd->internal_comparator(), - &memtables[0], memtables.size()); - Log(options_.info_log, "[%s] Level-0 flush table #%" PRIu64 ": started", - cfd->GetName().c_str(), meta.fd.GetNumber()); - - s = BuildTable(dbname_, env_, *cfd->options(), storage_options_, - cfd->table_cache(), iter, &meta, cfd->internal_comparator(), - newest_snapshot, earliest_seqno_in_memtable, - GetCompressionFlush(*cfd->options()), Env::IO_HIGH); - LogFlush(options_.info_log); - delete iter; - Log(options_.info_log, - "[%s] Level-0 flush table #%" PRIu64 ": %" PRIu64 " bytes %s", - cfd->GetName().c_str(), meta.fd.GetNumber(), meta.fd.GetFileSize(), - s.ToString().c_str()); - - if (!options_.disableDataSync) { - db_directory_->Fsync(); - } - mutex_.Lock(); - } - base->Unref(); - - // re-acquire the most current version - base = cfd->current(); - - // There could be multiple threads writing to its own level-0 file. - // The pending_outputs cannot be cleared here, otherwise this newly - // created file might not be considered as a live-file by another - // compaction thread that is concurrently deleting obselete files. - // The pending_outputs can be cleared only after the new version is - // committed so that other threads can recognize this file as a - // valid one. - // pending_outputs_.erase(meta.number); - - // Note that if file_size is zero, the file has been deleted and - // should not be added to the manifest. - int level = 0; - if (s.ok() && meta.fd.GetFileSize() > 0) { - const Slice min_user_key = meta.smallest.user_key(); - const Slice max_user_key = meta.largest.user_key(); - // if we have more than 1 background thread, then we cannot - // insert files directly into higher levels because some other - // threads could be concurrently producing compacted files for - // that key range. - if (base != nullptr && options_.max_background_compactions <= 1 && - cfd->options()->compaction_style == kCompactionStyleLevel) { - level = base->PickLevelForMemTableOutput(min_user_key, max_user_key); - } - edit->AddFile(level, meta.fd.GetNumber(), meta.fd.GetPathId(), - meta.fd.GetFileSize(), meta.smallest, meta.largest, - meta.smallest_seqno, meta.largest_seqno); - } - - InternalStats::CompactionStats stats(1); - stats.micros = env_->NowMicros() - start_micros; - stats.bytes_written = meta.fd.GetFileSize(); - cfd->internal_stats()->AddCompactionStats(level, stats); - cfd->internal_stats()->AddCFStats( - InternalStats::BYTES_FLUSHED, meta.fd.GetFileSize()); - RecordTick(stats_, COMPACT_WRITE_BYTES, meta.fd.GetFileSize()); - return s; -} - -Status DBImpl::FlushMemTableToOutputFile(ColumnFamilyData* cfd, - bool* madeProgress, - DeletionState& deletion_state, - LogBuffer* log_buffer) { - mutex_.AssertHeld(); - assert(cfd->imm()->size() != 0); - assert(cfd->imm()->IsFlushPending()); - - // Save the contents of the earliest memtable as a new Table - uint64_t file_number; - autovector mems; - cfd->imm()->PickMemtablesToFlush(&mems); - if (mems.empty()) { - LogToBuffer(log_buffer, "[%s] Nothing in memtable to flush", - cfd->GetName().c_str()); - return Status::OK(); - } - - // record the logfile_number_ before we release the mutex - // entries mems are (implicitly) sorted in ascending order by their created - // time. We will use the first memtable's `edit` to keep the meta info for - // this flush. - MemTable* m = mems[0]; - VersionEdit* edit = m->GetEdits(); - edit->SetPrevLogNumber(0); - // SetLogNumber(log_num) indicates logs with number smaller than log_num - // will no longer be picked up for recovery. - edit->SetLogNumber(mems.back()->GetNextLogNumber()); - edit->SetColumnFamily(cfd->GetID()); - - // This will release and re-acquire the mutex. - Status s = WriteLevel0Table(cfd, mems, edit, &file_number, log_buffer); - - if (s.ok() && shutting_down_.Acquire_Load() && cfd->IsDropped()) { - s = Status::ShutdownInProgress( - "Database shutdown or Column family drop during flush"); - } - - if (!s.ok()) { - cfd->imm()->RollbackMemtableFlush(mems, file_number, &pending_outputs_); - } else { - // Replace immutable memtable with the generated Table - s = cfd->imm()->InstallMemtableFlushResults( - cfd, mems, versions_.get(), &mutex_, options_.info_log.get(), - file_number, &pending_outputs_, &deletion_state.memtables_to_free, - db_directory_.get(), log_buffer); - } - - if (s.ok()) { - InstallSuperVersion(cfd, deletion_state); - if (madeProgress) { - *madeProgress = 1; - } - Version::LevelSummaryStorage tmp; - LogToBuffer(log_buffer, "[%s] Level summary: %s\n", cfd->GetName().c_str(), - cfd->current()->LevelSummary(&tmp)); - - if (disable_delete_obsolete_files_ == 0) { - // add to deletion state - while (alive_log_files_.size() && - alive_log_files_.begin()->number < versions_->MinLogNumber()) { - const auto& earliest = *alive_log_files_.begin(); - deletion_state.log_delete_files.push_back(earliest.number); - total_log_size_ -= earliest.size; - alive_log_files_.pop_front(); - } - } - } - - if (!s.ok() && !s.IsShutdownInProgress() && options_.paranoid_checks && - bg_error_.ok()) { - // if a bad error happened (not ShutdownInProgress) and paranoid_checks is - // true, mark DB read-only - bg_error_ = s; - } - RecordFlushIOStats(); - return s; -} - -Status DBImpl::CompactRange(ColumnFamilyHandle* column_family, - const Slice* begin, const Slice* end, - bool reduce_level, int target_level, - uint32_t target_path_id) { - if (target_path_id >= options_.db_paths.size()) { - return Status::InvalidArgument("Invalid target path ID"); - } - - auto cfh = reinterpret_cast(column_family); - auto cfd = cfh->cfd(); - - Status s = FlushMemTable(cfd, FlushOptions()); - if (!s.ok()) { - LogFlush(options_.info_log); - return s; - } - - int max_level_with_files = 0; - { - MutexLock l(&mutex_); - Version* base = cfd->current(); - for (int level = 1; level < cfd->NumberLevels(); level++) { - if (base->OverlapInLevel(level, begin, end)) { - max_level_with_files = level; - } - } - } - for (int level = 0; level <= max_level_with_files; level++) { - // in case the compaction is unversal or if we're compacting the - // bottom-most level, the output level will be the same as input one. - // level 0 can never be the bottommost level (i.e. if all files are in level - // 0, we will compact to level 1) - if (cfd->options()->compaction_style == kCompactionStyleUniversal || - cfd->options()->compaction_style == kCompactionStyleFIFO || - (level == max_level_with_files && level > 0)) { - s = RunManualCompaction(cfd, level, level, target_path_id, begin, end); - } else { - s = RunManualCompaction(cfd, level, level + 1, target_path_id, begin, - end); - } - if (!s.ok()) { - LogFlush(options_.info_log); - return s; - } - } - - if (reduce_level) { - s = ReFitLevel(cfd, max_level_with_files, target_level); - } - LogFlush(options_.info_log); - - { - MutexLock l(&mutex_); - // an automatic compaction that has been scheduled might have been - // preempted by the manual compactions. Need to schedule it back. - MaybeScheduleFlushOrCompaction(); - } - - return s; -} - -// return the same level if it cannot be moved -int DBImpl::FindMinimumEmptyLevelFitting(ColumnFamilyData* cfd, int level) { - mutex_.AssertHeld(); - Version* current = cfd->current(); - int minimum_level = level; - for (int i = level - 1; i > 0; --i) { - // stop if level i is not empty - if (current->NumLevelFiles(i) > 0) break; - // stop if level i is too small (cannot fit the level files) - if (cfd->compaction_picker()->MaxBytesForLevel(i) < - current->NumLevelBytes(level)) { - break; - } - - minimum_level = i; - } - return minimum_level; -} - -Status DBImpl::ReFitLevel(ColumnFamilyData* cfd, int level, int target_level) { - assert(level < cfd->NumberLevels()); - - SuperVersion* superversion_to_free = nullptr; - SuperVersion* new_superversion = new SuperVersion(); - - mutex_.Lock(); - - // only allow one thread refitting - if (refitting_level_) { - mutex_.Unlock(); - Log(options_.info_log, "ReFitLevel: another thread is refitting"); - delete new_superversion; - return Status::NotSupported("another thread is refitting"); - } - refitting_level_ = true; - - // wait for all background threads to stop - bg_work_gate_closed_ = true; - while (bg_compaction_scheduled_ > 0 || bg_flush_scheduled_) { - Log(options_.info_log, - "RefitLevel: waiting for background threads to stop: %d %d", - bg_compaction_scheduled_, bg_flush_scheduled_); - bg_cv_.Wait(); - } - - // move to a smaller level - int to_level = target_level; - if (target_level < 0) { - to_level = FindMinimumEmptyLevelFitting(cfd, level); - } - - assert(to_level <= level); - - Status status; - if (to_level < level) { - Log(options_.info_log, "[%s] Before refitting:\n%s", cfd->GetName().c_str(), - cfd->current()->DebugString().data()); - - VersionEdit edit; - edit.SetColumnFamily(cfd->GetID()); - for (const auto& f : cfd->current()->files_[level]) { - edit.DeleteFile(level, f->fd.GetNumber()); - edit.AddFile(to_level, f->fd.GetNumber(), f->fd.GetPathId(), - f->fd.GetFileSize(), f->smallest, f->largest, - f->smallest_seqno, f->largest_seqno); - } - Log(options_.info_log, "[%s] Apply version edit:\n%s", - cfd->GetName().c_str(), edit.DebugString().data()); - - status = versions_->LogAndApply(cfd, &edit, &mutex_, db_directory_.get()); - superversion_to_free = cfd->InstallSuperVersion(new_superversion, &mutex_); - new_superversion = nullptr; - - Log(options_.info_log, "[%s] LogAndApply: %s\n", cfd->GetName().c_str(), - status.ToString().data()); - - if (status.ok()) { - Log(options_.info_log, "[%s] After refitting:\n%s", - cfd->GetName().c_str(), cfd->current()->DebugString().data()); - } - } - - refitting_level_ = false; - bg_work_gate_closed_ = false; - - mutex_.Unlock(); - delete superversion_to_free; - delete new_superversion; - return status; -} - -int DBImpl::NumberLevels(ColumnFamilyHandle* column_family) { - auto cfh = reinterpret_cast(column_family); - return cfh->cfd()->NumberLevels(); -} - -int DBImpl::MaxMemCompactionLevel(ColumnFamilyHandle* column_family) { - auto cfh = reinterpret_cast(column_family); - return cfh->cfd()->options()->max_mem_compaction_level; -} - -int DBImpl::Level0StopWriteTrigger(ColumnFamilyHandle* column_family) { - auto cfh = reinterpret_cast(column_family); - return cfh->cfd()->options()->level0_stop_writes_trigger; -} - -Status DBImpl::Flush(const FlushOptions& options, - ColumnFamilyHandle* column_family) { - auto cfh = reinterpret_cast(column_family); - return FlushMemTable(cfh->cfd(), options); -} - -SequenceNumber DBImpl::GetLatestSequenceNumber() const { - return versions_->LastSequence(); -} - -Status DBImpl::RunManualCompaction(ColumnFamilyData* cfd, int input_level, - int output_level, uint32_t output_path_id, - const Slice* begin, const Slice* end) { - assert(input_level >= 0); - - InternalKey begin_storage, end_storage; - - ManualCompaction manual; - manual.cfd = cfd; - manual.input_level = input_level; - manual.output_level = output_level; - manual.output_path_id = output_path_id; - manual.done = false; - manual.in_progress = false; - // For universal compaction, we enforce every manual compaction to compact - // all files. - if (begin == nullptr || - cfd->options()->compaction_style == kCompactionStyleUniversal || - cfd->options()->compaction_style == kCompactionStyleFIFO) { - manual.begin = nullptr; - } else { - begin_storage = InternalKey(*begin, kMaxSequenceNumber, kValueTypeForSeek); - manual.begin = &begin_storage; - } - if (end == nullptr || - cfd->options()->compaction_style == kCompactionStyleUniversal || - cfd->options()->compaction_style == kCompactionStyleFIFO) { - manual.end = nullptr; - } else { - end_storage = InternalKey(*end, 0, static_cast(0)); - manual.end = &end_storage; - } - - MutexLock l(&mutex_); - - // When a manual compaction arrives, temporarily disable scheduling of - // non-manual compactions and wait until the number of scheduled compaction - // jobs drops to zero. This is needed to ensure that this manual compaction - // can compact any range of keys/files. - // - // bg_manual_only_ is non-zero when at least one thread is inside - // RunManualCompaction(), i.e. during that time no other compaction will - // get scheduled (see MaybeScheduleFlushOrCompaction). - // - // Note that the following loop doesn't stop more that one thread calling - // RunManualCompaction() from getting to the second while loop below. - // However, only one of them will actually schedule compaction, while - // others will wait on a condition variable until it completes. - - ++bg_manual_only_; - while (bg_compaction_scheduled_ > 0) { - Log(options_.info_log, - "[%s] Manual compaction waiting for all other scheduled background " - "compactions to finish", - cfd->GetName().c_str()); - bg_cv_.Wait(); - } - - Log(options_.info_log, "[%s] Manual compaction starting", - cfd->GetName().c_str()); - - while (!manual.done && !shutting_down_.Acquire_Load() && bg_error_.ok()) { - assert(bg_manual_only_ > 0); - if (manual_compaction_ != nullptr) { - // Running either this or some other manual compaction - bg_cv_.Wait(); - } else { - manual_compaction_ = &manual; - assert(bg_compaction_scheduled_ == 0); - bg_compaction_scheduled_++; - env_->Schedule(&DBImpl::BGWorkCompaction, this, Env::Priority::LOW); - } - } - - assert(!manual.in_progress); - assert(bg_manual_only_ > 0); - --bg_manual_only_; - return manual.status; -} - -Status DBImpl::FlushMemTable(ColumnFamilyData* cfd, - const FlushOptions& options) { - Writer w(&mutex_); - w.batch = nullptr; - w.sync = false; - w.disableWAL = false; - w.in_batch_group = false; - w.done = false; - w.timeout_hint_us = kNoTimeOut; - - Status s; - { - WriteContext context; - MutexLock guard_lock(&mutex_); - s = BeginWrite(&w, 0); - assert(s.ok() && !w.done); // No timeout and nobody should do our job - - // SetNewMemtableAndNewLogFile() will release and reacquire mutex - // during execution - s = SetNewMemtableAndNewLogFile(cfd, &context); - cfd->imm()->FlushRequested(); - MaybeScheduleFlushOrCompaction(); - - assert(!writers_.empty()); - assert(writers_.front() == &w); - EndWrite(&w, &w, s); - } - - - if (s.ok() && options.wait) { - // Wait until the compaction completes - s = WaitForFlushMemTable(cfd); - } - return s; -} - -Status DBImpl::WaitForFlushMemTable(ColumnFamilyData* cfd) { - Status s; - // Wait until the compaction completes - MutexLock l(&mutex_); - while (cfd->imm()->size() > 0 && bg_error_.ok()) { - bg_cv_.Wait(); - } - if (!bg_error_.ok()) { - s = bg_error_; - } - return s; -} - -void DBImpl::MaybeScheduleFlushOrCompaction() { - mutex_.AssertHeld(); - bg_schedule_needed_ = false; - if (bg_work_gate_closed_) { - // gate closed for backgrond work - } else if (shutting_down_.Acquire_Load()) { - // DB is being deleted; no more background compactions - } else { - bool is_flush_pending = false; - // no need to refcount since we're under a mutex - for (auto cfd : *versions_->GetColumnFamilySet()) { - if (cfd->imm()->IsFlushPending()) { - is_flush_pending = true; - } - } - if (is_flush_pending) { - // memtable flush needed - if (bg_flush_scheduled_ < options_.max_background_flushes) { - bg_flush_scheduled_++; - env_->Schedule(&DBImpl::BGWorkFlush, this, Env::Priority::HIGH); - } else if (options_.max_background_flushes > 0) { - bg_schedule_needed_ = true; - } - } - bool is_compaction_needed = false; - // no need to refcount since we're under a mutex - for (auto cfd : *versions_->GetColumnFamilySet()) { - if (cfd->current()->NeedsCompaction()) { - is_compaction_needed = true; - break; - } - } - - // Schedule BGWorkCompaction if there's a compaction pending (or a memtable - // flush, but the HIGH pool is not enabled) - // Do it only if max_background_compactions hasn't been reached and - // bg_manual_only_ == 0 - if (!bg_manual_only_ && - (is_compaction_needed || - (is_flush_pending && options_.max_background_flushes == 0))) { - if (bg_compaction_scheduled_ < options_.max_background_compactions) { - bg_compaction_scheduled_++; - env_->Schedule(&DBImpl::BGWorkCompaction, this, Env::Priority::LOW); - } else { - bg_schedule_needed_ = true; - } - } - } -} - -void DBImpl::RecordFlushIOStats() { - RecordTick(stats_, FLUSH_WRITE_BYTES, IOSTATS(bytes_written)); - IOSTATS_RESET(bytes_written); -} - -void DBImpl::RecordCompactionIOStats() { - RecordTick(stats_, COMPACT_READ_BYTES, IOSTATS(bytes_read)); - IOSTATS_RESET(bytes_read); - RecordTick(stats_, COMPACT_WRITE_BYTES, IOSTATS(bytes_written)); - IOSTATS_RESET(bytes_written); -} - -void DBImpl::BGWorkFlush(void* db) { - IOSTATS_SET_THREAD_POOL_ID(Env::Priority::HIGH); - reinterpret_cast(db)->BackgroundCallFlush(); -} - -void DBImpl::BGWorkCompaction(void* db) { - IOSTATS_SET_THREAD_POOL_ID(Env::Priority::LOW); - reinterpret_cast(db)->BackgroundCallCompaction(); -} - -Status DBImpl::BackgroundFlush(bool* madeProgress, - DeletionState& deletion_state, - LogBuffer* log_buffer) { - mutex_.AssertHeld(); - // call_status is failure if at least one flush was a failure. even if - // flushing one column family reports a failure, we will continue flushing - // other column families. however, call_status will be a failure in that case. - Status call_status; - // refcounting in iteration - for (auto cfd : *versions_->GetColumnFamilySet()) { - cfd->Ref(); - Status flush_status; - while (flush_status.ok() && cfd->imm()->IsFlushPending()) { - LogToBuffer( - log_buffer, - "BackgroundCallFlush doing FlushMemTableToOutputFile with column " - "family [%s], flush slots available %d", - cfd->GetName().c_str(), - options_.max_background_flushes - bg_flush_scheduled_); - flush_status = FlushMemTableToOutputFile(cfd, madeProgress, - deletion_state, log_buffer); - } - if (call_status.ok() && !flush_status.ok()) { - call_status = flush_status; - } - cfd->Unref(); - } - versions_->GetColumnFamilySet()->FreeDeadColumnFamilies(); - return call_status; -} - -void DBImpl::BackgroundCallFlush() { - bool madeProgress = false; - DeletionState deletion_state(true); - assert(bg_flush_scheduled_); - - LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, options_.info_log.get()); - { - MutexLock l(&mutex_); - - Status s; - if (!shutting_down_.Acquire_Load()) { - s = BackgroundFlush(&madeProgress, deletion_state, &log_buffer); - if (!s.ok()) { - // Wait a little bit before retrying background compaction in - // case this is an environmental problem and we do not want to - // chew up resources for failed compactions for the duration of - // the problem. - uint64_t error_cnt = - default_cf_internal_stats_->BumpAndGetBackgroundErrorCount(); - bg_cv_.SignalAll(); // In case a waiter can proceed despite the error - mutex_.Unlock(); - Log(options_.info_log, - "Waiting after background flush error: %s" - "Accumulated background error counts: %" PRIu64, - s.ToString().c_str(), error_cnt); - log_buffer.FlushBufferToLog(); - LogFlush(options_.info_log); - env_->SleepForMicroseconds(1000000); - mutex_.Lock(); - } - } - - // If !s.ok(), this means that Flush failed. In that case, we want - // to delete all obsolete files and we force FindObsoleteFiles() - FindObsoleteFiles(deletion_state, !s.ok()); - // delete unnecessary files if any, this is done outside the mutex - if (deletion_state.HaveSomethingToDelete() || !log_buffer.IsEmpty()) { - mutex_.Unlock(); - // Have to flush the info logs before bg_flush_scheduled_-- - // because if bg_flush_scheduled_ becomes 0 and the lock is - // released, the deconstructor of DB can kick in and destroy all the - // states of DB so info_log might not be available after that point. - // It also applies to access other states that DB owns. - log_buffer.FlushBufferToLog(); - if (deletion_state.HaveSomethingToDelete()) { - PurgeObsoleteFiles(deletion_state); - } - mutex_.Lock(); - } - - bg_flush_scheduled_--; - // Any time the mutex is released After finding the work to do, another - // thread might execute MaybeScheduleFlushOrCompaction(). It is possible - // that there is a pending job but it is not scheduled because of the - // max thread limit. - if (madeProgress || bg_schedule_needed_) { - MaybeScheduleFlushOrCompaction(); - } - RecordFlushIOStats(); - bg_cv_.SignalAll(); - // IMPORTANT: there should be no code after calling SignalAll. This call may - // signal the DB destructor that it's OK to proceed with destruction. In - // that case, all DB variables will be dealloacated and referencing them - // will cause trouble. - } -} - -void DBImpl::BackgroundCallCompaction() { - bool madeProgress = false; - DeletionState deletion_state(true); - - MaybeDumpStats(); - LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, options_.info_log.get()); - { - MutexLock l(&mutex_); - assert(bg_compaction_scheduled_); - Status s; - if (!shutting_down_.Acquire_Load()) { - s = BackgroundCompaction(&madeProgress, deletion_state, &log_buffer); - if (!s.ok()) { - // Wait a little bit before retrying background compaction in - // case this is an environmental problem and we do not want to - // chew up resources for failed compactions for the duration of - // the problem. - uint64_t error_cnt = - default_cf_internal_stats_->BumpAndGetBackgroundErrorCount(); - bg_cv_.SignalAll(); // In case a waiter can proceed despite the error - mutex_.Unlock(); - log_buffer.FlushBufferToLog(); - Log(options_.info_log, - "Waiting after background compaction error: %s, " - "Accumulated background error counts: %" PRIu64, - s.ToString().c_str(), error_cnt); - LogFlush(options_.info_log); - env_->SleepForMicroseconds(1000000); - mutex_.Lock(); - } - } - - // If !s.ok(), this means that Compaction failed. In that case, we want - // to delete all obsolete files we might have created and we force - // FindObsoleteFiles(). This is because deletion_state does not catch - // all created files if compaction failed. - FindObsoleteFiles(deletion_state, !s.ok()); - - // delete unnecessary files if any, this is done outside the mutex - if (deletion_state.HaveSomethingToDelete() || !log_buffer.IsEmpty()) { - mutex_.Unlock(); - // Have to flush the info logs before bg_compaction_scheduled_-- - // because if bg_flush_scheduled_ becomes 0 and the lock is - // released, the deconstructor of DB can kick in and destroy all the - // states of DB so info_log might not be available after that point. - // It also applies to access other states that DB owns. - log_buffer.FlushBufferToLog(); - if (deletion_state.HaveSomethingToDelete()) { - PurgeObsoleteFiles(deletion_state); - } - mutex_.Lock(); - } - - bg_compaction_scheduled_--; - - versions_->GetColumnFamilySet()->FreeDeadColumnFamilies(); - - // Previous compaction may have produced too many files in a level, - // So reschedule another compaction if we made progress in the - // last compaction. - // - // Also, any time the mutex is released After finding the work to do, - // another thread might execute MaybeScheduleFlushOrCompaction(). It is - // possible that there is a pending job but it is not scheduled because of - // the max thread limit. - if (madeProgress || bg_schedule_needed_) { - MaybeScheduleFlushOrCompaction(); - } - if (madeProgress || bg_compaction_scheduled_ == 0 || bg_manual_only_ > 0) { - // signal if - // * madeProgress -- need to wakeup MakeRoomForWrite - // * bg_compaction_scheduled_ == 0 -- need to wakeup ~DBImpl - // * bg_manual_only_ > 0 -- need to wakeup RunManualCompaction - // If none of this is true, there is no need to signal since nobody is - // waiting for it - bg_cv_.SignalAll(); - } - // IMPORTANT: there should be no code after calling SignalAll. This call may - // signal the DB destructor that it's OK to proceed with destruction. In - // that case, all DB variables will be dealloacated and referencing them - // will cause trouble. - } -} - -Status DBImpl::BackgroundCompaction(bool* madeProgress, - DeletionState& deletion_state, - LogBuffer* log_buffer) { - *madeProgress = false; - mutex_.AssertHeld(); - - bool is_manual = (manual_compaction_ != nullptr) && - (manual_compaction_->in_progress == false); - - if (is_manual) { - // another thread cannot pick up the same work - manual_compaction_->in_progress = true; - } else if (manual_compaction_ != nullptr) { - // there should be no automatic compactions running when manual compaction - // is running - return Status::OK(); - } - - // FLUSH preempts compaction - Status flush_stat; - for (auto cfd : *versions_->GetColumnFamilySet()) { - while (cfd->imm()->IsFlushPending()) { - LogToBuffer( - log_buffer, - "BackgroundCompaction doing FlushMemTableToOutputFile, " - "compaction slots available %d", - options_.max_background_compactions - bg_compaction_scheduled_); - cfd->Ref(); - flush_stat = FlushMemTableToOutputFile(cfd, madeProgress, deletion_state, - log_buffer); - cfd->Unref(); - if (!flush_stat.ok()) { - if (is_manual) { - manual_compaction_->status = flush_stat; - manual_compaction_->done = true; - manual_compaction_->in_progress = false; - manual_compaction_ = nullptr; - } - return flush_stat; - } - } - } - - unique_ptr c; - InternalKey manual_end_storage; - InternalKey* manual_end = &manual_end_storage; - if (is_manual) { - ManualCompaction* m = manual_compaction_; - assert(m->in_progress); - c.reset(m->cfd->CompactRange(m->input_level, m->output_level, - m->output_path_id, m->begin, m->end, - &manual_end)); - if (!c) { - m->done = true; - } - LogToBuffer(log_buffer, - "[%s] Manual compaction from level-%d to level-%d from %s .. " - "%s; will stop at %s\n", - m->cfd->GetName().c_str(), m->input_level, m->output_level, - (m->begin ? m->begin->DebugString().c_str() : "(begin)"), - (m->end ? m->end->DebugString().c_str() : "(end)"), - ((m->done || manual_end == nullptr) - ? "(end)" - : manual_end->DebugString().c_str())); - } else { - // no need to refcount in iteration since it's always under a mutex - for (auto cfd : *versions_->GetColumnFamilySet()) { - if (!cfd->options()->disable_auto_compactions) { - c.reset(cfd->PickCompaction(log_buffer)); - if (c != nullptr) { - // update statistics - MeasureTime(stats_, NUM_FILES_IN_SINGLE_COMPACTION, - c->inputs(0)->size()); - break; - } - } - } - } - - Status status; - if (!c) { - // Nothing to do - LogToBuffer(log_buffer, "Compaction nothing to do"); - } else if (c->IsDeletionCompaction()) { - // TODO(icanadi) Do we want to honor snapshots here? i.e. not delete old - // file if there is alive snapshot pointing to it - assert(c->num_input_files(1) == 0); - assert(c->level() == 0); - assert(c->column_family_data()->options()->compaction_style == - kCompactionStyleFIFO); - for (const auto& f : *c->inputs(0)) { - c->edit()->DeleteFile(c->level(), f->fd.GetNumber()); - } - status = versions_->LogAndApply(c->column_family_data(), c->edit(), &mutex_, - db_directory_.get()); - InstallSuperVersion(c->column_family_data(), deletion_state); - LogToBuffer(log_buffer, "[%s] Deleted %d files\n", - c->column_family_data()->GetName().c_str(), - c->num_input_files(0)); - c->ReleaseCompactionFiles(status); - *madeProgress = true; - } else if (!is_manual && c->IsTrivialMove()) { - // Move file to next level - assert(c->num_input_files(0) == 1); - FileMetaData* f = c->input(0, 0); - c->edit()->DeleteFile(c->level(), f->fd.GetNumber()); - c->edit()->AddFile(c->level() + 1, f->fd.GetNumber(), f->fd.GetPathId(), - f->fd.GetFileSize(), f->smallest, f->largest, - f->smallest_seqno, f->largest_seqno); - status = versions_->LogAndApply(c->column_family_data(), c->edit(), &mutex_, - db_directory_.get()); - InstallSuperVersion(c->column_family_data(), deletion_state); - - Version::LevelSummaryStorage tmp; - LogToBuffer( - log_buffer, "[%s] Moved #%lld to level-%d %lld bytes %s: %s\n", - c->column_family_data()->GetName().c_str(), - static_cast(f->fd.GetNumber()), c->level() + 1, - static_cast(f->fd.GetFileSize()), - status.ToString().c_str(), c->input_version()->LevelSummary(&tmp)); - c->ReleaseCompactionFiles(status); - *madeProgress = true; - } else { - MaybeScheduleFlushOrCompaction(); // do more compaction work in parallel. - CompactionState* compact = new CompactionState(c.get()); - status = DoCompactionWork(compact, deletion_state, log_buffer); - CleanupCompaction(compact, status); - c->ReleaseCompactionFiles(status); - c->ReleaseInputs(); - *madeProgress = true; - } - c.reset(); - - if (status.ok()) { - // Done - } else if (status.IsShutdownInProgress()) { - // Ignore compaction errors found during shutting down - } else { - Log(InfoLogLevel::WARN_LEVEL, options_.info_log, "Compaction error: %s", - status.ToString().c_str()); - if (options_.paranoid_checks && bg_error_.ok()) { - bg_error_ = status; - } - } - - if (is_manual) { - ManualCompaction* m = manual_compaction_; - if (!status.ok()) { - m->status = status; - m->done = true; - } - // For universal compaction: - // Because universal compaction always happens at level 0, so one - // compaction will pick up all overlapped files. No files will be - // filtered out due to size limit and left for a successive compaction. - // So we can safely conclude the current compaction. - // - // Also note that, if we don't stop here, then the current compaction - // writes a new file back to level 0, which will be used in successive - // compaction. Hence the manual compaction will never finish. - // - // Stop the compaction if manual_end points to nullptr -- this means - // that we compacted the whole range. manual_end should always point - // to nullptr in case of universal compaction - if (manual_end == nullptr) { - m->done = true; - } - if (!m->done) { - // We only compacted part of the requested range. Update *m - // to the range that is left to be compacted. - // Universal and FIFO compactions should always compact the whole range - assert(m->cfd->options()->compaction_style != kCompactionStyleUniversal); - assert(m->cfd->options()->compaction_style != kCompactionStyleFIFO); - m->tmp_storage = *manual_end; - m->begin = &m->tmp_storage; - } - m->in_progress = false; // not being processed anymore - manual_compaction_ = nullptr; - } - return status; -} - -void DBImpl::CleanupCompaction(CompactionState* compact, Status status) { - mutex_.AssertHeld(); - if (compact->builder != nullptr) { - // May happen if we get a shutdown call in the middle of compaction - compact->builder->Abandon(); - compact->builder.reset(); - } else { - assert(compact->outfile == nullptr); - } - for (size_t i = 0; i < compact->outputs.size(); i++) { - const CompactionState::Output& out = compact->outputs[i]; - pending_outputs_.erase(out.number); - - // If this file was inserted into the table cache then remove - // them here because this compaction was not committed. - if (!status.ok()) { - TableCache::Evict(table_cache_.get(), out.number); - } - } - delete compact; -} - -// Allocate the file numbers for the output file. We allocate as -// many output file numbers as there are files in level+1 (at least one) -// Insert them into pending_outputs so that they do not get deleted. -void DBImpl::AllocateCompactionOutputFileNumbers(CompactionState* compact) { - mutex_.AssertHeld(); - assert(compact != nullptr); - assert(compact->builder == nullptr); - int filesNeeded = compact->compaction->num_input_files(1); - for (int i = 0; i < std::max(filesNeeded, 1); i++) { - uint64_t file_number = versions_->NewFileNumber(); - pending_outputs_[file_number] = compact->compaction->GetOutputPathId(); - compact->allocated_file_numbers.push_back(file_number); - } -} - -// Frees up unused file number. -void DBImpl::ReleaseCompactionUnusedFileNumbers(CompactionState* compact) { - mutex_.AssertHeld(); - for (const auto file_number : compact->allocated_file_numbers) { - pending_outputs_.erase(file_number); - } -} - -Status DBImpl::OpenCompactionOutputFile(CompactionState* compact) { - assert(compact != nullptr); - assert(compact->builder == nullptr); - uint64_t file_number; - // If we have not yet exhausted the pre-allocated file numbers, - // then use the one from the front. Otherwise, we have to acquire - // the heavyweight lock and allocate a new file number. - if (!compact->allocated_file_numbers.empty()) { - file_number = compact->allocated_file_numbers.front(); - compact->allocated_file_numbers.pop_front(); +InternalIterator* DBImpl::NewInternalIterator( + Arena* arena, RangeDelAggregator* range_del_agg, + ColumnFamilyHandle* column_family) { + ColumnFamilyData* cfd; + if (column_family == nullptr) { + cfd = default_cf_handle_->cfd(); } else { - mutex_.Lock(); - file_number = versions_->NewFileNumber(); - pending_outputs_[file_number] = compact->compaction->GetOutputPathId(); - mutex_.Unlock(); - } - CompactionState::Output out; - out.number = file_number; - out.path_id = compact->compaction->GetOutputPathId(); - out.smallest.Clear(); - out.largest.Clear(); - out.smallest_seqno = out.largest_seqno = 0; - compact->outputs.push_back(out); - - // Make the output file - std::string fname = TableFileName(options_.db_paths, file_number, - compact->compaction->GetOutputPathId()); - Status s = env_->NewWritableFile(fname, &compact->outfile, storage_options_); - - if (s.ok()) { - compact->outfile->SetIOPriority(Env::IO_LOW); - compact->outfile->SetPreallocationBlockSize( - compact->compaction->OutputFilePreallocationSize()); - - ColumnFamilyData* cfd = compact->compaction->column_family_data(); - compact->builder.reset(NewTableBuilder( - *cfd->options(), cfd->internal_comparator(), compact->outfile.get(), - compact->compaction->OutputCompressionType())); + auto cfh = reinterpret_cast(column_family); + cfd = cfh->cfd(); } - LogFlush(options_.info_log); - return s; -} - -Status DBImpl::FinishCompactionOutputFile(CompactionState* compact, - Iterator* input) { - assert(compact != nullptr); - assert(compact->outfile); - assert(compact->builder != nullptr); - const uint64_t output_number = compact->current_output()->number; - const uint32_t output_path_id = compact->current_output()->path_id; - assert(output_number != 0); - - // Check for iterator errors - Status s = input->status(); - const uint64_t current_entries = compact->builder->NumEntries(); - if (s.ok()) { - s = compact->builder->Finish(); - } else { - compact->builder->Abandon(); - } - const uint64_t current_bytes = compact->builder->FileSize(); - compact->current_output()->file_size = current_bytes; - compact->total_bytes += current_bytes; - compact->builder.reset(); - - // Finish and check for file errors - if (s.ok() && !options_.disableDataSync) { - if (options_.use_fsync) { - StopWatch sw(env_, stats_, COMPACTION_OUTFILE_SYNC_MICROS); - s = compact->outfile->Fsync(); - } else { - StopWatch sw(env_, stats_, COMPACTION_OUTFILE_SYNC_MICROS); - s = compact->outfile->Sync(); - } - } - if (s.ok()) { - s = compact->outfile->Close(); - } - compact->outfile.reset(); - - if (s.ok() && current_entries > 0) { - // Verify that the table is usable - ColumnFamilyData* cfd = compact->compaction->column_family_data(); - FileDescriptor fd(output_number, output_path_id, current_bytes); - Iterator* iter = cfd->table_cache()->NewIterator( - ReadOptions(), storage_options_, cfd->internal_comparator(), fd); - s = iter->status(); - delete iter; - if (s.ok()) { - Log(options_.info_log, "[%s] Generated table #%" PRIu64 ": %" PRIu64 - " keys, %" PRIu64 " bytes", - cfd->GetName().c_str(), output_number, current_entries, - current_bytes); - } - } - return s; + mutex_.Lock(); + SuperVersion* super_version = cfd->GetSuperVersion()->Ref(); + mutex_.Unlock(); + ReadOptions roptions; + return NewInternalIterator(roptions, cfd, super_version, arena, + range_del_agg); } - -Status DBImpl::InstallCompactionResults(CompactionState* compact, - LogBuffer* log_buffer) { +void DBImpl::SchedulePurge() { mutex_.AssertHeld(); + assert(opened_successfully_); - // paranoia: verify that the files that we started with - // still exist in the current version and in the same original level. - // This ensures that a concurrent compaction did not erroneously - // pick the same files to compact. - if (!versions_->VerifyCompactionFileConsistency(compact->compaction)) { - Log(options_.info_log, "[%s] Compaction %d@%d + %d@%d files aborted", - compact->compaction->column_family_data()->GetName().c_str(), - compact->compaction->num_input_files(0), compact->compaction->level(), - compact->compaction->num_input_files(1), - compact->compaction->output_level()); - return Status::Corruption("Compaction input files inconsistent"); - } - - LogToBuffer(log_buffer, "[%s] Compacted %d@%d + %d@%d files => %lld bytes", - compact->compaction->column_family_data()->GetName().c_str(), - compact->compaction->num_input_files(0), - compact->compaction->level(), - compact->compaction->num_input_files(1), - compact->compaction->output_level(), - static_cast(compact->total_bytes)); - - // Add compaction outputs - compact->compaction->AddInputDeletions(compact->compaction->edit()); - for (size_t i = 0; i < compact->outputs.size(); i++) { - const CompactionState::Output& out = compact->outputs[i]; - compact->compaction->edit()->AddFile(compact->compaction->output_level(), - out.number, out.path_id, out.file_size, - out.smallest, out.largest, - out.smallest_seqno, out.largest_seqno); - } - return versions_->LogAndApply(compact->compaction->column_family_data(), - compact->compaction->edit(), &mutex_, - db_directory_.get()); + // Purge operations are put into High priority queue + bg_purge_scheduled_++; + env_->Schedule(&DBImpl::BGWorkPurge, this, Env::Priority::HIGH, nullptr); } -// Given a sequence number, return the sequence number of the -// earliest snapshot that this sequence number is visible in. -// The snapshots themselves are arranged in ascending order of -// sequence numbers. -// Employ a sequential search because the total number of -// snapshots are typically small. -inline SequenceNumber DBImpl::findEarliestVisibleSnapshot( - SequenceNumber in, std::vector& snapshots, - SequenceNumber* prev_snapshot) { - SequenceNumber prev __attribute__((unused)) = 0; - for (const auto cur : snapshots) { - assert(prev <= cur); - if (cur >= in) { - *prev_snapshot = prev; - return cur; - } - prev = cur; // assignment - assert(prev); - } - Log(options_.info_log, - "Looking for seqid %" PRIu64 " but maxseqid is %" PRIu64 "", in, - snapshots[snapshots.size() - 1]); - assert(0); - return 0; -} - -uint64_t DBImpl::CallFlushDuringCompaction(ColumnFamilyData* cfd, - DeletionState& deletion_state, - LogBuffer* log_buffer) { - if (options_.max_background_flushes > 0) { - // flush thread will take care of this - return 0; - } - if (cfd->imm()->imm_flush_needed.NoBarrier_Load() != nullptr) { - const uint64_t imm_start = env_->NowMicros(); - mutex_.Lock(); - if (cfd->imm()->IsFlushPending()) { - cfd->Ref(); - FlushMemTableToOutputFile(cfd, nullptr, deletion_state, log_buffer); - cfd->Unref(); - bg_cv_.SignalAll(); // Wakeup MakeRoomForWrite() if necessary - } - mutex_.Unlock(); - log_buffer->FlushBufferToLog(); - return env_->NowMicros() - imm_start; - } - return 0; -} - -Status DBImpl::ProcessKeyValueCompaction( - bool is_snapshot_supported, - SequenceNumber visible_at_tip, - SequenceNumber earliest_snapshot, - SequenceNumber latest_snapshot, - DeletionState& deletion_state, - bool bottommost_level, - int64_t& imm_micros, - Iterator* input, - CompactionState* compact, - bool is_compaction_v2, - LogBuffer* log_buffer) { - size_t combined_idx = 0; - Status status; - std::string compaction_filter_value; - ParsedInternalKey ikey; - IterKey current_user_key; - bool has_current_user_key = false; - IterKey delete_key; - SequenceNumber last_sequence_for_key __attribute__((unused)) = - kMaxSequenceNumber; - SequenceNumber visible_in_snapshot = kMaxSequenceNumber; - ColumnFamilyData* cfd = compact->compaction->column_family_data(); - MergeHelper merge( - cfd->user_comparator(), cfd->options()->merge_operator.get(), - options_.info_log.get(), cfd->options()->min_partial_merge_operands, - false /* internal key corruption is expected */); - auto compaction_filter = cfd->options()->compaction_filter; - std::unique_ptr compaction_filter_from_factory = nullptr; - if (!compaction_filter) { - auto context = compact->GetFilterContextV1(); - compaction_filter_from_factory = - cfd->options()->compaction_filter_factory->CreateCompactionFilter( - context); - compaction_filter = compaction_filter_from_factory.get(); - } - - int64_t key_drop_user = 0; - int64_t key_drop_newer_entry = 0; - int64_t key_drop_obsolete = 0; - int64_t loop_cnt = 0; - while (input->Valid() && !shutting_down_.Acquire_Load() && - !cfd->IsDropped()) { - if (++loop_cnt > 1000) { - if (key_drop_user > 0) { - RecordTick(stats_, COMPACTION_KEY_DROP_USER, key_drop_user); - key_drop_user = 0; - } - if (key_drop_newer_entry > 0) { - RecordTick(stats_, COMPACTION_KEY_DROP_NEWER_ENTRY, - key_drop_newer_entry); - key_drop_newer_entry = 0; - } - if (key_drop_obsolete > 0) { - RecordTick(stats_, COMPACTION_KEY_DROP_OBSOLETE, key_drop_obsolete); - key_drop_obsolete = 0; - } - RecordCompactionIOStats(); - loop_cnt = 0; - } - // FLUSH preempts compaction - // TODO(icanadi) this currently only checks if flush is necessary on - // compacting column family. we should also check if flush is necessary on - // other column families, too - imm_micros += CallFlushDuringCompaction(cfd, deletion_state, log_buffer); - - Slice key; - Slice value; - // If is_compaction_v2 is on, kv-pairs are reset to the prefix batch. - // This prefix batch should contain results after calling - // compaction_filter_v2. - // - // If is_compaction_v2 is off, this function will go through all the - // kv-pairs in input. - if (!is_compaction_v2) { - key = input->key(); - value = input->value(); - } else { - if (combined_idx >= compact->combined_key_buf_.size()) { - break; - } - assert(combined_idx < compact->combined_key_buf_.size()); - key = compact->combined_key_buf_[combined_idx]; - value = compact->combined_value_buf_[combined_idx]; - - ++combined_idx; - } +void DBImpl::BackgroundCallPurge() { + mutex_.Lock(); - if (compact->compaction->ShouldStopBefore(key) && - compact->builder != nullptr) { - status = FinishCompactionOutputFile(compact, input); - if (!status.ok()) { - break; - } - } + // We use one single loop to clear both queues so that after existing the loop + // both queues are empty. This is stricter than what is needed, but can make + // it easier for us to reason the correctness. + while (!purge_queue_.empty() || !logs_to_free_queue_.empty()) { + if (!purge_queue_.empty()) { + auto purge_file = purge_queue_.begin(); + auto fname = purge_file->fname; + auto type = purge_file->type; + auto number = purge_file->number; + auto path_id = purge_file->path_id; + auto job_id = purge_file->job_id; + purge_queue_.pop_front(); - // Handle key/value, add to state, etc. - bool drop = false; - bool current_entry_is_merging = false; - if (!ParseInternalKey(key, &ikey)) { - // Do not hide error keys - // TODO: error key stays in db forever? Figure out the intention/rationale - // v10 error v8 : we cannot hide v8 even though it's pretty obvious. - current_user_key.Clear(); - has_current_user_key = false; - last_sequence_for_key = kMaxSequenceNumber; - visible_in_snapshot = kMaxSequenceNumber; + mutex_.Unlock(); + Status file_deletion_status; + DeleteObsoleteFileImpl(file_deletion_status, job_id, fname, type, number, + path_id); + mutex_.Lock(); } else { - if (!has_current_user_key || - cfd->user_comparator()->Compare(ikey.user_key, - current_user_key.GetKey()) != 0) { - // First occurrence of this user key - current_user_key.SetKey(ikey.user_key); - has_current_user_key = true; - last_sequence_for_key = kMaxSequenceNumber; - visible_in_snapshot = kMaxSequenceNumber; - // apply the compaction filter to the first occurrence of the user key - if (compaction_filter && !is_compaction_v2 && - ikey.type == kTypeValue && - (visible_at_tip || ikey.sequence > latest_snapshot)) { - // If the user has specified a compaction filter and the sequence - // number is greater than any external snapshot, then invoke the - // filter. - // If the return value of the compaction filter is true, replace - // the entry with a delete marker. - bool value_changed = false; - compaction_filter_value.clear(); - bool to_delete = compaction_filter->Filter( - compact->compaction->level(), ikey.user_key, value, - &compaction_filter_value, &value_changed); - if (to_delete) { - // make a copy of the original key and convert it to a delete - delete_key.SetInternalKey(ExtractUserKey(key), ikey.sequence, - kTypeDeletion); - // anchor the key again - key = delete_key.GetKey(); - // needed because ikey is backed by key - ParseInternalKey(key, &ikey); - // no value associated with delete - value.clear(); - ++key_drop_user; - } else if (value_changed) { - value = compaction_filter_value; - } - } - } - - // If there are no snapshots, then this kv affect visibility at tip. - // Otherwise, search though all existing snapshots to find - // the earlist snapshot that is affected by this kv. - SequenceNumber prev_snapshot = 0; // 0 means no previous snapshot - SequenceNumber visible = visible_at_tip ? visible_at_tip : - is_snapshot_supported ? findEarliestVisibleSnapshot(ikey.sequence, - compact->existing_snapshots, &prev_snapshot) - : 0; - - if (visible_in_snapshot == visible) { - // If the earliest snapshot is which this key is visible in - // is the same as the visibily of a previous instance of the - // same key, then this kv is not visible in any snapshot. - // Hidden by an newer entry for same user key - // TODO: why not > ? - assert(last_sequence_for_key >= ikey.sequence); - drop = true; // (A) - ++key_drop_newer_entry; - } else if (ikey.type == kTypeDeletion && - ikey.sequence <= earliest_snapshot && - compact->compaction->KeyNotExistsBeyondOutputLevel(ikey.user_key)) { - // For this user key: - // (1) there is no data in higher levels - // (2) data in lower levels will have larger sequence numbers - // (3) data in layers that are being compacted here and have - // smaller sequence numbers will be dropped in the next - // few iterations of this loop (by rule (A) above). - // Therefore this deletion marker is obsolete and can be dropped. - drop = true; - ++key_drop_obsolete; - } else if (ikey.type == kTypeMerge) { - if (!merge.HasOperator()) { - LogToBuffer(log_buffer, "Options::merge_operator is null."); - status = Status::InvalidArgument( - "merge_operator is not properly initialized."); - break; - } - // We know the merge type entry is not hidden, otherwise we would - // have hit (A) - // We encapsulate the merge related state machine in a different - // object to minimize change to the existing flow. Turn out this - // logic could also be nicely re-used for memtable flush purge - // optimization in BuildTable. - int steps = 0; - merge.MergeUntil(input, prev_snapshot, bottommost_level, - options_.statistics.get(), &steps); - // Skip the Merge ops - combined_idx = combined_idx - 1 + steps; - - current_entry_is_merging = true; - if (merge.IsSuccess()) { - // Successfully found Put/Delete/(end-of-key-range) while merging - // Get the merge result - key = merge.key(); - ParseInternalKey(key, &ikey); - value = merge.value(); - } else { - // Did not find a Put/Delete/(end-of-key-range) while merging - // We now have some stack of merge operands to write out. - // NOTE: key,value, and ikey are now referring to old entries. - // These will be correctly set below. - assert(!merge.keys().empty()); - assert(merge.keys().size() == merge.values().size()); - - // Hack to make sure last_sequence_for_key is correct - ParseInternalKey(merge.keys().front(), &ikey); - } - } - - last_sequence_for_key = ikey.sequence; - visible_in_snapshot = visible; - } - - if (!drop) { - // We may write a single key (e.g.: for Put/Delete or successful merge). - // Or we may instead have to write a sequence/list of keys. - // We have to write a sequence iff we have an unsuccessful merge - bool has_merge_list = current_entry_is_merging && !merge.IsSuccess(); - const std::deque* keys = nullptr; - const std::deque* values = nullptr; - std::deque::const_reverse_iterator key_iter; - std::deque::const_reverse_iterator value_iter; - if (has_merge_list) { - keys = &merge.keys(); - values = &merge.values(); - key_iter = keys->rbegin(); // The back (*rbegin()) is the first key - value_iter = values->rbegin(); - - key = Slice(*key_iter); - value = Slice(*value_iter); - } - - // If we have a list of keys to write, traverse the list. - // If we have a single key to write, simply write that key. - while (true) { - // Invariant: key,value,ikey will always be the next entry to write - char* kptr = (char*)key.data(); - std::string kstr; - - // Zeroing out the sequence number leads to better compression. - // If this is the bottommost level (no files in lower levels) - // and the earliest snapshot is larger than this seqno - // then we can squash the seqno to zero. - if (bottommost_level && ikey.sequence < earliest_snapshot && - ikey.type != kTypeMerge) { - assert(ikey.type != kTypeDeletion); - // make a copy because updating in place would cause problems - // with the priority queue that is managing the input key iterator - kstr.assign(key.data(), key.size()); - kptr = (char *)kstr.c_str(); - UpdateInternalKey(kptr, key.size(), (uint64_t)0, ikey.type); - } - - Slice newkey(kptr, key.size()); - assert((key.clear(), 1)); // we do not need 'key' anymore - - // Open output file if necessary - if (compact->builder == nullptr) { - status = OpenCompactionOutputFile(compact); - if (!status.ok()) { - break; - } - } - - SequenceNumber seqno = GetInternalKeySeqno(newkey); - if (compact->builder->NumEntries() == 0) { - compact->current_output()->smallest.DecodeFrom(newkey); - compact->current_output()->smallest_seqno = seqno; - } else { - compact->current_output()->smallest_seqno = - std::min(compact->current_output()->smallest_seqno, seqno); - } - compact->current_output()->largest.DecodeFrom(newkey); - compact->builder->Add(newkey, value); - compact->current_output()->largest_seqno = - std::max(compact->current_output()->largest_seqno, seqno); - - // Close output file if it is big enough - if (compact->builder->FileSize() >= - compact->compaction->MaxOutputFileSize()) { - status = FinishCompactionOutputFile(compact, input); - if (!status.ok()) { - break; - } - } - - // If we have a list of entries, move to next element - // If we only had one entry, then break the loop. - if (has_merge_list) { - ++key_iter; - ++value_iter; - - // If at end of list - if (key_iter == keys->rend() || value_iter == values->rend()) { - // Sanity Check: if one ends, then both end - assert(key_iter == keys->rend() && value_iter == values->rend()); - break; - } - - // Otherwise not at end of list. Update key, value, and ikey. - key = Slice(*key_iter); - value = Slice(*value_iter); - ParseInternalKey(key, &ikey); - - } else{ - // Only had one item to begin with (Put/Delete) - break; - } - } - } - - // MergeUntil has moved input to the next entry - if (!current_entry_is_merging) { - input->Next(); - } - } - if (key_drop_user > 0) { - RecordTick(stats_, COMPACTION_KEY_DROP_USER, key_drop_user); - } - if (key_drop_newer_entry > 0) { - RecordTick(stats_, COMPACTION_KEY_DROP_NEWER_ENTRY, key_drop_newer_entry); - } - if (key_drop_obsolete > 0) { - RecordTick(stats_, COMPACTION_KEY_DROP_OBSOLETE, key_drop_obsolete); - } - RecordCompactionIOStats(); - - return status; -} - -void DBImpl::CallCompactionFilterV2(CompactionState* compact, - CompactionFilterV2* compaction_filter_v2) { - if (compact == nullptr || compaction_filter_v2 == nullptr) { - return; - } - - // Assemble slice vectors for user keys and existing values. - // We also keep track of our parsed internal key structs because - // we may need to access the sequence number in the event that - // keys are garbage collected during the filter process. - std::vector ikey_buf; - std::vector user_key_buf; - std::vector existing_value_buf; - - for (const auto& key : compact->key_str_buf_) { - ParsedInternalKey ikey; - ParseInternalKey(Slice(key), &ikey); - ikey_buf.emplace_back(ikey); - user_key_buf.emplace_back(ikey.user_key); - } - for (const auto& value : compact->existing_value_str_buf_) { - existing_value_buf.emplace_back(Slice(value)); - } - - // If the user has specified a compaction filter and the sequence - // number is greater than any external snapshot, then invoke the - // filter. - // If the return value of the compaction filter is true, replace - // the entry with a delete marker. - compact->to_delete_buf_ = compaction_filter_v2->Filter( - compact->compaction->level(), - user_key_buf, existing_value_buf, - &compact->new_value_buf_, - &compact->value_changed_buf_); - - // new_value_buf_.size() <= to_delete__buf_.size(). "=" iff all - // kv-pairs in this compaction run needs to be deleted. - assert(compact->to_delete_buf_.size() == - compact->key_str_buf_.size()); - assert(compact->to_delete_buf_.size() == - compact->existing_value_str_buf_.size()); - assert(compact->to_delete_buf_.size() == - compact->value_changed_buf_.size()); - - int new_value_idx = 0; - for (unsigned int i = 0; i < compact->to_delete_buf_.size(); ++i) { - if (compact->to_delete_buf_[i]) { - // update the string buffer directly - // the Slice buffer points to the updated buffer - UpdateInternalKey(&compact->key_str_buf_[i][0], - compact->key_str_buf_[i].size(), - ikey_buf[i].sequence, - kTypeDeletion); - - // no value associated with delete - compact->existing_value_str_buf_[i].clear(); - RecordTick(stats_, COMPACTION_KEY_DROP_USER); - } else if (compact->value_changed_buf_[i]) { - compact->existing_value_str_buf_[i] = - compact->new_value_buf_[new_value_idx++]; + assert(!logs_to_free_queue_.empty()); + log::Writer* log_writer = *(logs_to_free_queue_.begin()); + logs_to_free_queue_.pop_front(); + mutex_.Unlock(); + delete log_writer; + mutex_.Lock(); } - } // for -} - -Status DBImpl::DoCompactionWork(CompactionState* compact, - DeletionState& deletion_state, - LogBuffer* log_buffer) { - assert(compact); - compact->CleanupBatchBuffer(); - compact->CleanupMergedBuffer(); - bool prefix_initialized = false; - - // Generate file_levels_ for compaction berfore making Iterator - compact->compaction->GenerateFileLevels(); - int64_t imm_micros = 0; // Micros spent doing imm_ compactions - ColumnFamilyData* cfd = compact->compaction->column_family_data(); - LogToBuffer( - log_buffer, - "[%s] Compacting %d@%d + %d@%d files, score %.2f slots available %d", - cfd->GetName().c_str(), compact->compaction->num_input_files(0), - compact->compaction->level(), compact->compaction->num_input_files(1), - compact->compaction->output_level(), compact->compaction->score(), - options_.max_background_compactions - bg_compaction_scheduled_); - char scratch[2345]; - compact->compaction->Summary(scratch, sizeof(scratch)); - LogToBuffer(log_buffer, "[%s] Compaction start summary: %s\n", - cfd->GetName().c_str(), scratch); - - assert(cfd->current()->NumLevelFiles(compact->compaction->level()) > 0); - assert(compact->builder == nullptr); - assert(!compact->outfile); - - SequenceNumber visible_at_tip = 0; - SequenceNumber earliest_snapshot; - SequenceNumber latest_snapshot = 0; - snapshots_.getAll(compact->existing_snapshots); - if (compact->existing_snapshots.size() == 0) { - // optimize for fast path if there are no snapshots - visible_at_tip = versions_->LastSequence(); - earliest_snapshot = visible_at_tip; - } else { - latest_snapshot = compact->existing_snapshots.back(); - // Add the current seqno as the 'latest' virtual - // snapshot to the end of this list. - compact->existing_snapshots.push_back(versions_->LastSequence()); - earliest_snapshot = compact->existing_snapshots[0]; } + bg_purge_scheduled_--; - // Is this compaction producing files at the bottommost level? - bool bottommost_level = compact->compaction->BottomMostLevel(); - - // Allocate the output file numbers before we release the lock - AllocateCompactionOutputFileNumbers(compact); - - bool is_snapshot_supported = IsSnapshotSupported(); - // Release mutex while we're actually doing the compaction work + bg_cv_.SignalAll(); + // IMPORTANT:there should be no code after calling SignalAll. This call may + // signal the DB destructor that it's OK to proceed with destruction. In + // that case, all DB variables will be dealloacated and referencing them + // will cause trouble. mutex_.Unlock(); - log_buffer->FlushBufferToLog(); - - const uint64_t start_micros = env_->NowMicros(); - unique_ptr input(versions_->MakeInputIterator(compact->compaction)); - input->SeekToFirst(); - shared_ptr backup_input( - versions_->MakeInputIterator(compact->compaction)); - backup_input->SeekToFirst(); - - Status status; - ParsedInternalKey ikey; - std::unique_ptr compaction_filter_from_factory_v2 - = nullptr; - auto context = compact->GetFilterContext(); - compaction_filter_from_factory_v2 = - cfd->options()->compaction_filter_factory_v2->CreateCompactionFilterV2( - context); - auto compaction_filter_v2 = - compaction_filter_from_factory_v2.get(); - - // temp_backup_input always point to the start of the current buffer - // temp_backup_input = backup_input; - // iterate through input, - // 1) buffer ineligible keys and value keys into 2 separate buffers; - // 2) send value_buffer to compaction filter and alternate the values; - // 3) merge value_buffer with ineligible_value_buffer; - // 4) run the modified "compaction" using the old for loop. - if (compaction_filter_v2) { - while (backup_input->Valid() && !shutting_down_.Acquire_Load() && - !cfd->IsDropped()) { - // FLUSH preempts compaction - // TODO(icanadi) this currently only checks if flush is necessary on - // compacting column family. we should also check if flush is necessary on - // other column families, too - imm_micros += CallFlushDuringCompaction(cfd, deletion_state, log_buffer); - - Slice key = backup_input->key(); - Slice value = backup_input->value(); - - if (!ParseInternalKey(key, &ikey)) { - // log error - Log(options_.info_log, "[%s] Failed to parse key: %s", - cfd->GetName().c_str(), key.ToString().c_str()); - continue; - } else { - const SliceTransform* transformer = - cfd->options()->compaction_filter_factory_v2->GetPrefixExtractor(); - const auto key_prefix = transformer->Transform(ikey.user_key); - if (!prefix_initialized) { - compact->cur_prefix_ = key_prefix.ToString(); - prefix_initialized = true; - } - // If the prefix remains the same, keep buffering - if (key_prefix.compare(Slice(compact->cur_prefix_)) == 0) { - // Apply the compaction filter V2 to all the kv pairs sharing - // the same prefix - if (ikey.type == kTypeValue && - (visible_at_tip || ikey.sequence > latest_snapshot)) { - // Buffer all keys sharing the same prefix for CompactionFilterV2 - // Iterate through keys to check prefix - compact->BufferKeyValueSlices(key, value); - } else { - // buffer ineligible keys - compact->BufferOtherKeyValueSlices(key, value); - } - backup_input->Next(); - continue; - // finish changing values for eligible keys - } else { - // Now prefix changes, this batch is done. - // Call compaction filter on the buffered values to change the value - if (compact->key_str_buf_.size() > 0) { - CallCompactionFilterV2(compact, compaction_filter_v2); - } - compact->cur_prefix_ = key_prefix.ToString(); - } - } - - // Merge this batch of data (values + ineligible keys) - compact->MergeKeyValueSliceBuffer(&cfd->internal_comparator()); - - // Done buffering for the current prefix. Spit it out to disk - // Now just iterate through all the kv-pairs - status = ProcessKeyValueCompaction( - is_snapshot_supported, - visible_at_tip, - earliest_snapshot, - latest_snapshot, - deletion_state, - bottommost_level, - imm_micros, - input.get(), - compact, - true, - log_buffer); - - if (!status.ok()) { - break; - } - - // After writing the kv-pairs, we can safely remove the reference - // to the string buffer and clean them up - compact->CleanupBatchBuffer(); - compact->CleanupMergedBuffer(); - // Buffer the key that triggers the mismatch in prefix - if (ikey.type == kTypeValue && - (visible_at_tip || ikey.sequence > latest_snapshot)) { - compact->BufferKeyValueSlices(key, value); - } else { - compact->BufferOtherKeyValueSlices(key, value); - } - backup_input->Next(); - if (!backup_input->Valid()) { - // If this is the single last value, we need to merge it. - if (compact->key_str_buf_.size() > 0) { - CallCompactionFilterV2(compact, compaction_filter_v2); - } - compact->MergeKeyValueSliceBuffer(&cfd->internal_comparator()); - - status = ProcessKeyValueCompaction( - is_snapshot_supported, - visible_at_tip, - earliest_snapshot, - latest_snapshot, - deletion_state, - bottommost_level, - imm_micros, - input.get(), - compact, - true, - log_buffer); - - compact->CleanupBatchBuffer(); - compact->CleanupMergedBuffer(); - } - } // done processing all prefix batches - // finish the last batch - if (compact->key_str_buf_.size() > 0) { - CallCompactionFilterV2(compact, compaction_filter_v2); - } - compact->MergeKeyValueSliceBuffer(&cfd->internal_comparator()); - status = ProcessKeyValueCompaction( - is_snapshot_supported, - visible_at_tip, - earliest_snapshot, - latest_snapshot, - deletion_state, - bottommost_level, - imm_micros, - input.get(), - compact, - true, - log_buffer); - } // checking for compaction filter v2 - - if (!compaction_filter_v2) { - status = ProcessKeyValueCompaction( - is_snapshot_supported, - visible_at_tip, - earliest_snapshot, - latest_snapshot, - deletion_state, - bottommost_level, - imm_micros, - input.get(), - compact, - false, - log_buffer); - } - - if (status.ok() && (shutting_down_.Acquire_Load() || cfd->IsDropped())) { - status = Status::ShutdownInProgress( - "Database shutdown or Column family drop during compaction"); - } - if (status.ok() && compact->builder != nullptr) { - status = FinishCompactionOutputFile(compact, input.get()); - } - if (status.ok()) { - status = input->status(); - } - input.reset(); - - if (!options_.disableDataSync) { - db_directory_->Fsync(); - } - - InternalStats::CompactionStats stats(1); - stats.micros = env_->NowMicros() - start_micros - imm_micros; - stats.files_in_leveln = compact->compaction->num_input_files(0); - stats.files_in_levelnp1 = compact->compaction->num_input_files(1); - MeasureTime(stats_, COMPACTION_TIME, stats.micros); - - int num_output_files = compact->outputs.size(); - if (compact->builder != nullptr) { - // An error occurred so ignore the last output. - assert(num_output_files > 0); - --num_output_files; - } - stats.files_out_levelnp1 = num_output_files; - - for (int i = 0; i < compact->compaction->num_input_files(0); i++) { - stats.bytes_readn += compact->compaction->input(0, i)->fd.GetFileSize(); - } - - for (int i = 0; i < compact->compaction->num_input_files(1); i++) { - stats.bytes_readnp1 += compact->compaction->input(1, i)->fd.GetFileSize(); - } - - for (int i = 0; i < num_output_files; i++) { - stats.bytes_written += compact->outputs[i].file_size; - } - - RecordCompactionIOStats(); - - LogFlush(options_.info_log); - mutex_.Lock(); - cfd->internal_stats()->AddCompactionStats( - compact->compaction->output_level(), stats); - - // if there were any unused file number (mostly in case of - // compaction error), free up the entry from pending_putputs - ReleaseCompactionUnusedFileNumbers(compact); - - if (status.ok()) { - status = InstallCompactionResults(compact, log_buffer); - InstallSuperVersion(cfd, deletion_state); - } - Version::LevelSummaryStorage tmp; - LogToBuffer( - log_buffer, - "[%s] compacted to: %s, %.1f MB/sec, level %d, files in(%d, %d) out(%d) " - "MB in(%.1f, %.1f) out(%.1f), read-write-amplify(%.1f) " - "write-amplify(%.1f) %s\n", - cfd->GetName().c_str(), cfd->current()->LevelSummary(&tmp), - (stats.bytes_readn + stats.bytes_readnp1 + stats.bytes_written) / - (double)stats.micros, - compact->compaction->output_level(), stats.files_in_leveln, - stats.files_in_levelnp1, stats.files_out_levelnp1, - stats.bytes_readn / 1048576.0, stats.bytes_readnp1 / 1048576.0, - stats.bytes_written / 1048576.0, - (stats.bytes_written + stats.bytes_readnp1 + stats.bytes_readn) / - (double)stats.bytes_readn, - stats.bytes_written / (double)stats.bytes_readn, - status.ToString().c_str()); - - return status; } namespace { struct IterState { - IterState(DBImpl* db, port::Mutex* mu, SuperVersion* super_version) - : db(db), mu(mu), super_version(super_version) {} + IterState(DBImpl* _db, InstrumentedMutex* _mu, SuperVersion* _super_version, + bool _background_purge) + : db(_db), + mu(_mu), + super_version(_super_version), + background_purge(_background_purge) {} DBImpl* db; - port::Mutex* mu; + InstrumentedMutex* mu; SuperVersion* super_version; + bool background_purge; }; static void CleanupIteratorState(void* arg1, void* arg2) { IterState* state = reinterpret_cast(arg1); if (state->super_version->Unref()) { - DBImpl::DeletionState deletion_state; + // Job id == 0 means that this is not our background process, but rather + // user thread + JobContext job_context(0); state->mu->Lock(); state->super_version->Cleanup(); - state->db->FindObsoleteFiles(deletion_state, false, true); + state->db->FindObsoleteFiles(&job_context, false, true); + if (state->background_purge) { + state->db->ScheduleBgLogWriterClose(&job_context); + } state->mu->Unlock(); delete state->super_version; - if (deletion_state.HaveSomethingToDelete()) { - state->db->PurgeObsoleteFiles(deletion_state); + if (job_context.HaveSomethingToDelete()) { + if (state->background_purge) { + // PurgeObsoleteFiles here does not delete files. Instead, it adds the + // files to be deleted to a job queue, and deletes it in a separate + // background thread. + state->db->PurgeObsoleteFiles(job_context, true /* schedule only */); + state->mu->Lock(); + state->db->SchedulePurge(); + state->mu->Unlock(); + } else { + state->db->PurgeObsoleteFiles(job_context); + } } + job_context.Clean(); } delete state; } } // namespace -Iterator* DBImpl::NewInternalIterator(const ReadOptions& options, - ColumnFamilyData* cfd, - SuperVersion* super_version, - Arena* arena) { - Iterator* internal_iter; - if (arena != nullptr) { - // Need to create internal iterator from the arena. - MergeIteratorBuilder merge_iter_builder(&cfd->internal_comparator(), arena); - // Collect iterator for mutable mem - merge_iter_builder.AddIterator( - super_version->mem->NewIterator(options, arena)); - // Collect all needed child iterators for immutable memtables - super_version->imm->AddIterators(options, &merge_iter_builder); +InternalIterator* DBImpl::NewInternalIterator( + const ReadOptions& read_options, ColumnFamilyData* cfd, + SuperVersion* super_version, Arena* arena, + RangeDelAggregator* range_del_agg) { + InternalIterator* internal_iter; + assert(arena != nullptr); + assert(range_del_agg != nullptr); + // Need to create internal iterator from the arena. + MergeIteratorBuilder merge_iter_builder( + &cfd->internal_comparator(), arena, + !read_options.total_order_seek && + cfd->ioptions()->prefix_extractor != nullptr); + // Collect iterator for mutable mem + merge_iter_builder.AddIterator( + super_version->mem->NewIterator(read_options, arena)); + std::unique_ptr range_del_iter; + Status s; + if (!read_options.ignore_range_deletions) { + range_del_iter.reset( + super_version->mem->NewRangeTombstoneIterator(read_options)); + s = range_del_agg->AddTombstones(std::move(range_del_iter)); + } + // Collect all needed child iterators for immutable memtables + if (s.ok()) { + super_version->imm->AddIterators(read_options, &merge_iter_builder); + if (!read_options.ignore_range_deletions) { + s = super_version->imm->AddRangeTombstoneIterators(read_options, arena, + range_del_agg); + } + } + if (s.ok()) { // Collect iterators for files in L0 - Ln - super_version->current->AddIterators(options, storage_options_, - &merge_iter_builder); + if (read_options.read_tier != kMemtableTier) { + super_version->current->AddIterators(read_options, env_options_, + &merge_iter_builder, range_del_agg); + } internal_iter = merge_iter_builder.Finish(); - } else { - // Need to create internal iterator using malloc. - std::vector iterator_list; - // Collect iterator for mutable mem - iterator_list.push_back(super_version->mem->NewIterator(options)); - // Collect all needed child iterators for immutable memtables - super_version->imm->AddIterators(options, &iterator_list); - // Collect iterators for files in L0 - Ln - super_version->current->AddIterators(options, storage_options_, - &iterator_list); - internal_iter = NewMergingIterator(&cfd->internal_comparator(), - &iterator_list[0], iterator_list.size()); - } - IterState* cleanup = new IterState(this, &mutex_, super_version); - internal_iter->RegisterCleanup(CleanupIteratorState, cleanup, nullptr); + IterState* cleanup = + new IterState(this, &mutex_, super_version, + read_options.background_purge_on_iterator_cleanup); + internal_iter->RegisterCleanup(CleanupIteratorState, cleanup, nullptr); - return internal_iter; + return internal_iter; + } + return NewErrorInternalIterator(s); } ColumnFamilyHandle* DBImpl::DefaultColumnFamily() const { return default_cf_handle_; } -Status DBImpl::Get(const ReadOptions& options, +Status DBImpl::Get(const ReadOptions& read_options, ColumnFamilyHandle* column_family, const Slice& key, - std::string* value) { - return GetImpl(options, column_family, key, value); -} - -// DeletionState gets created and destructed outside of the lock -- we -// use this convinently to: -// * malloc one SuperVersion() outside of the lock -- new_superversion -// * delete SuperVersion()s outside of the lock -- superversions_to_free -// -// However, if InstallSuperVersion() gets called twice with the same, -// deletion_state, we can't reuse the SuperVersion() that got malloced because -// first call already used it. In that rare case, we take a hit and create a -// new SuperVersion() inside of the mutex. We do similar thing -// for superversion_to_free -void DBImpl::InstallSuperVersion(ColumnFamilyData* cfd, - DeletionState& deletion_state) { - mutex_.AssertHeld(); - // if new_superversion == nullptr, it means somebody already used it - SuperVersion* new_superversion = - (deletion_state.new_superversion != nullptr) ? - deletion_state.new_superversion : new SuperVersion(); - SuperVersion* old_superversion = - cfd->InstallSuperVersion(new_superversion, &mutex_); - deletion_state.new_superversion = nullptr; - deletion_state.superversions_to_free.push_back(old_superversion); + PinnableSlice* value) { + return GetImpl(read_options, column_family, key, value); } -Status DBImpl::GetImpl(const ReadOptions& options, +Status DBImpl::GetImpl(const ReadOptions& read_options, ColumnFamilyHandle* column_family, const Slice& key, - std::string* value, bool* value_found) { + PinnableSlice* pinnable_val, bool* value_found, + bool* is_blob_index) { + assert(pinnable_val != nullptr); StopWatch sw(env_, stats_, DB_GET); PERF_TIMER_GUARD(get_snapshot_time); auto cfh = reinterpret_cast(column_family); auto cfd = cfh->cfd(); - SequenceNumber snapshot; - if (options.snapshot != nullptr) { - snapshot = reinterpret_cast(options.snapshot)->number_; - } else { - snapshot = versions_->LastSequence(); - } - // Acquire SuperVersion SuperVersion* sv = GetAndRefSuperVersion(cfd); + TEST_SYNC_POINT("DBImpl::GetImpl:1"); + TEST_SYNC_POINT("DBImpl::GetImpl:2"); + + SequenceNumber snapshot; + if (read_options.snapshot != nullptr) { + snapshot = reinterpret_cast( + read_options.snapshot)->number_; + } else { + // Since we get and reference the super version before getting + // the snapshot number, without a mutex protection, it is possible + // that a memtable switch happened in the middle and not all the + // data for this snapshot is available. But it will contain all + // the data available in the super version we have, which is also + // a valid snapshot to read from. + // We shouldn't get snapshot before finding and referencing the + // super versipon because a flush happening in between may compact + // away data for the snapshot, but the snapshot is earlier than the + // data overwriting it, so users may see wrong results. + snapshot = versions_->LastSequence(); + } + TEST_SYNC_POINT("DBImpl::GetImpl:3"); + TEST_SYNC_POINT("DBImpl::GetImpl:4"); + // Prepare to store a list of merge operations if merge occurs. MergeContext merge_context; + RangeDelAggregator range_del_agg(cfd->internal_comparator(), snapshot); Status s; // First look in the memtable, then in the immutable memtable (if any). @@ -3439,15 +955,31 @@ Status DBImpl::GetImpl(const ReadOptions& options, LookupKey lkey(key, snapshot); PERF_TIMER_STOP(get_snapshot_time); - if (sv->mem->Get(lkey, value, &s, merge_context, *cfd->options())) { - // Done - RecordTick(stats_, MEMTABLE_HIT); - } else if (sv->imm->Get(lkey, value, &s, merge_context, *cfd->options())) { - // Done - RecordTick(stats_, MEMTABLE_HIT); - } else { + bool skip_memtable = (read_options.read_tier == kPersistedTier && + has_unpersisted_data_.load(std::memory_order_relaxed)); + bool done = false; + if (!skip_memtable) { + if (sv->mem->Get(lkey, pinnable_val->GetSelf(), &s, &merge_context, + &range_del_agg, read_options, is_blob_index)) { + done = true; + pinnable_val->PinSelf(); + RecordTick(stats_, MEMTABLE_HIT); + } else if ((s.ok() || s.IsMergeInProgress()) && + sv->imm->Get(lkey, pinnable_val->GetSelf(), &s, &merge_context, + &range_del_agg, read_options, is_blob_index)) { + done = true; + pinnable_val->PinSelf(); + RecordTick(stats_, MEMTABLE_HIT); + } + if (!done && !s.ok() && !s.IsMergeInProgress()) { + return s; + } + } + if (!done) { PERF_TIMER_GUARD(get_from_output_files_time); - sv->current->Get(options, lkey, value, &s, &merge_context, value_found); + sv->current->Get(read_options, lkey, pinnable_val, &s, &merge_context, + &range_del_agg, value_found, nullptr, nullptr, + is_blob_index); RecordTick(stats_, MEMTABLE_MISS); } @@ -3457,13 +989,16 @@ Status DBImpl::GetImpl(const ReadOptions& options, ReturnAndCleanupSuperVersion(cfd, sv); RecordTick(stats_, NUMBER_KEYS_READ); - RecordTick(stats_, BYTES_READ, value->size()); + size_t size = pinnable_val->size(); + RecordTick(stats_, BYTES_READ, size); + MeasureTime(stats_, BYTES_PER_READ, size); + PERF_COUNTER_ADD(get_read_bytes, size); } return s; } std::vector DBImpl::MultiGet( - const ReadOptions& options, + const ReadOptions& read_options, const std::vector& column_family, const std::vector& keys, std::vector* values) { @@ -3489,8 +1024,9 @@ std::vector DBImpl::MultiGet( } mutex_.Lock(); - if (options.snapshot != nullptr) { - snapshot = reinterpret_cast(options.snapshot)->number_; + if (read_options.snapshot != nullptr) { + snapshot = reinterpret_cast( + read_options.snapshot)->number_; } else { snapshot = versions_->LastSequence(); } @@ -3523,19 +1059,34 @@ std::vector DBImpl::MultiGet( LookupKey lkey(keys[i], snapshot); auto cfh = reinterpret_cast(column_family[i]); + RangeDelAggregator range_del_agg(cfh->cfd()->internal_comparator(), + snapshot); auto mgd_iter = multiget_cf_data.find(cfh->cfd()->GetID()); assert(mgd_iter != multiget_cf_data.end()); auto mgd = mgd_iter->second; auto super_version = mgd->super_version; - auto cfd = mgd->cfd; - if (super_version->mem->Get(lkey, value, &s, merge_context, - *cfd->options())) { - // Done - } else if (super_version->imm->Get(lkey, value, &s, merge_context, - *cfd->options())) { - // Done - } else { - super_version->current->Get(options, lkey, value, &s, &merge_context); + bool skip_memtable = + (read_options.read_tier == kPersistedTier && + has_unpersisted_data_.load(std::memory_order_relaxed)); + bool done = false; + if (!skip_memtable) { + if (super_version->mem->Get(lkey, value, &s, &merge_context, + &range_del_agg, read_options)) { + done = true; + // TODO(?): RecordTick(stats_, MEMTABLE_HIT)? + } else if (super_version->imm->Get(lkey, value, &s, &merge_context, + &range_del_agg, read_options)) { + done = true; + // TODO(?): RecordTick(stats_, MEMTABLE_HIT)? + } + } + if (!done) { + PinnableSlice pinnable_val; + PERF_TIMER_GUARD(get_from_output_files_time); + super_version->current->Get(read_options, lkey, &pinnable_val, &s, + &merge_context, &range_del_agg); + value->assign(pinnable_val.data(), pinnable_val.size()); + // TODO(?): RecordTick(stats_, MEMTABLE_MISS)? } if (s.ok()) { @@ -3568,95 +1119,263 @@ std::vector DBImpl::MultiGet( RecordTick(stats_, NUMBER_MULTIGET_CALLS); RecordTick(stats_, NUMBER_MULTIGET_KEYS_READ, num_keys); RecordTick(stats_, NUMBER_MULTIGET_BYTES_READ, bytes_read); + MeasureTime(stats_, BYTES_PER_MULTIGET, bytes_read); + PERF_COUNTER_ADD(multiget_read_bytes, bytes_read); PERF_TIMER_STOP(get_post_process_time); return stat_list; } -Status DBImpl::CreateColumnFamily(const ColumnFamilyOptions& options, - const std::string& column_family_name, +Status DBImpl::CreateColumnFamily(const ColumnFamilyOptions& cf_options, + const std::string& column_family, ColumnFamilyHandle** handle) { + assert(handle != nullptr); + Status s = CreateColumnFamilyImpl(cf_options, column_family, handle); + if (s.ok()) { + s = WriteOptionsFile(true /*need_mutex_lock*/, + true /*need_enter_write_thread*/); + } + return s; +} + +Status DBImpl::CreateColumnFamilies( + const ColumnFamilyOptions& cf_options, + const std::vector& column_family_names, + std::vector* handles) { + assert(handles != nullptr); + handles->clear(); + size_t num_cf = column_family_names.size(); + Status s; + bool success_once = false; + for (size_t i = 0; i < num_cf; i++) { + ColumnFamilyHandle* handle; + s = CreateColumnFamilyImpl(cf_options, column_family_names[i], &handle); + if (!s.ok()) { + break; + } + handles->push_back(handle); + success_once = true; + } + if (success_once) { + Status persist_options_status = WriteOptionsFile( + true /*need_mutex_lock*/, true /*need_enter_write_thread*/); + if (s.ok() && !persist_options_status.ok()) { + s = persist_options_status; + } + } + return s; +} + +Status DBImpl::CreateColumnFamilies( + const std::vector& column_families, + std::vector* handles) { + assert(handles != nullptr); + handles->clear(); + size_t num_cf = column_families.size(); + Status s; + bool success_once = false; + for (size_t i = 0; i < num_cf; i++) { + ColumnFamilyHandle* handle; + s = CreateColumnFamilyImpl(column_families[i].options, + column_families[i].name, &handle); + if (!s.ok()) { + break; + } + handles->push_back(handle); + success_once = true; + } + if (success_once) { + Status persist_options_status = WriteOptionsFile( + true /*need_mutex_lock*/, true /*need_enter_write_thread*/); + if (s.ok() && !persist_options_status.ok()) { + s = persist_options_status; + } + } + return s; +} + +Status DBImpl::CreateColumnFamilyImpl(const ColumnFamilyOptions& cf_options, + const std::string& column_family_name, + ColumnFamilyHandle** handle) { + Status s; + Status persist_options_status; *handle = nullptr; - MutexLock l(&mutex_); - if (versions_->GetColumnFamilySet()->GetColumnFamily(column_family_name) != - nullptr) { - return Status::InvalidArgument("Column family already exists"); + s = CheckCompressionSupported(cf_options); + if (s.ok() && immutable_db_options_.allow_concurrent_memtable_write) { + s = CheckConcurrentWritesSupported(cf_options); } - VersionEdit edit; - edit.AddColumnFamily(column_family_name); - uint32_t new_id = versions_->GetColumnFamilySet()->GetNextColumnFamilyID(); - edit.SetColumnFamily(new_id); - edit.SetLogNumber(logfile_number_); - edit.SetComparatorName(options.comparator->Name()); - - // LogAndApply will both write the creation in MANIFEST and create - // ColumnFamilyData object - Status s = versions_->LogAndApply(nullptr, &edit, &mutex_, - db_directory_.get(), false, &options); + if (!s.ok()) { + return s; + } + + { + InstrumentedMutexLock l(&mutex_); + + if (versions_->GetColumnFamilySet()->GetColumnFamily(column_family_name) != + nullptr) { + return Status::InvalidArgument("Column family already exists"); + } + VersionEdit edit; + edit.AddColumnFamily(column_family_name); + uint32_t new_id = versions_->GetColumnFamilySet()->GetNextColumnFamilyID(); + edit.SetColumnFamily(new_id); + edit.SetLogNumber(logfile_number_); + edit.SetComparatorName(cf_options.comparator->Name()); + + // LogAndApply will both write the creation in MANIFEST and create + // ColumnFamilyData object + { // write thread + WriteThread::Writer w; + write_thread_.EnterUnbatched(&w, &mutex_); + // LogAndApply will both write the creation in MANIFEST and create + // ColumnFamilyData object + s = versions_->LogAndApply(nullptr, MutableCFOptions(cf_options), &edit, + &mutex_, directories_.GetDbDir(), false, + &cf_options); + write_thread_.ExitUnbatched(&w); + } + if (s.ok()) { + single_column_family_mode_ = false; + auto* cfd = + versions_->GetColumnFamilySet()->GetColumnFamily(column_family_name); + assert(cfd != nullptr); + delete InstallSuperVersionAndScheduleWork( + cfd, nullptr, *cfd->GetLatestMutableCFOptions()); + + if (!cfd->mem()->IsSnapshotSupported()) { + is_snapshot_supported_ = false; + } + + cfd->set_initialized(); + + *handle = new ColumnFamilyHandleImpl(cfd, this, &mutex_); + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "Created column family [%s] (ID %u)", + column_family_name.c_str(), (unsigned)cfd->GetID()); + } else { + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "Creating column family [%s] FAILED -- %s", + column_family_name.c_str(), s.ToString().c_str()); + } + } // InstrumentedMutexLock l(&mutex_) + + // this is outside the mutex if (s.ok()) { - single_column_family_mode_ = false; - auto cfd = - versions_->GetColumnFamilySet()->GetColumnFamily(column_family_name); - assert(cfd != nullptr); - delete cfd->InstallSuperVersion(new SuperVersion(), &mutex_); - *handle = new ColumnFamilyHandleImpl(cfd, this, &mutex_); - Log(options_.info_log, "Created column family [%s] (ID %u)", - column_family_name.c_str(), (unsigned)cfd->GetID()); - max_total_in_memory_state_ += cfd->options()->write_buffer_size * - cfd->options()->max_write_buffer_number; - } else { - Log(options_.info_log, "Creating column family [%s] FAILED -- %s", - column_family_name.c_str(), s.ToString().c_str()); + NewThreadStatusCfInfo( + reinterpret_cast(*handle)->cfd()); } return s; } Status DBImpl::DropColumnFamily(ColumnFamilyHandle* column_family) { + assert(column_family != nullptr); + Status s = DropColumnFamilyImpl(column_family); + if (s.ok()) { + s = WriteOptionsFile(true /*need_mutex_lock*/, + true /*need_enter_write_thread*/); + } + return s; +} + +Status DBImpl::DropColumnFamilies( + const std::vector& column_families) { + Status s; + bool success_once = false; + for (auto* handle : column_families) { + s = DropColumnFamilyImpl(handle); + if (!s.ok()) { + break; + } + success_once = true; + } + if (success_once) { + Status persist_options_status = WriteOptionsFile( + true /*need_mutex_lock*/, true /*need_enter_write_thread*/); + if (s.ok() && !persist_options_status.ok()) { + s = persist_options_status; + } + } + return s; +} + +Status DBImpl::DropColumnFamilyImpl(ColumnFamilyHandle* column_family) { auto cfh = reinterpret_cast(column_family); auto cfd = cfh->cfd(); if (cfd->GetID() == 0) { return Status::InvalidArgument("Can't drop default column family"); } + bool cf_support_snapshot = cfd->mem()->IsSnapshotSupported(); + VersionEdit edit; edit.DropColumnFamily(); edit.SetColumnFamily(cfd->GetID()); Status s; { - MutexLock l(&mutex_); + InstrumentedMutexLock l(&mutex_); if (cfd->IsDropped()) { s = Status::InvalidArgument("Column family already dropped!\n"); } if (s.ok()) { - s = versions_->LogAndApply(cfd, &edit, &mutex_); + // we drop column family from a single write thread + WriteThread::Writer w; + write_thread_.EnterUnbatched(&w, &mutex_); + s = versions_->LogAndApply(cfd, *cfd->GetLatestMutableCFOptions(), + &edit, &mutex_); + write_thread_.ExitUnbatched(&w); + } + if (s.ok()) { + auto* mutable_cf_options = cfd->GetLatestMutableCFOptions(); + max_total_in_memory_state_ -= mutable_cf_options->write_buffer_size * + mutable_cf_options->max_write_buffer_number; + } + + if (!cf_support_snapshot) { + // Dropped Column Family doesn't support snapshot. Need to recalculate + // is_snapshot_supported_. + bool new_is_snapshot_supported = true; + for (auto c : *versions_->GetColumnFamilySet()) { + if (!c->IsDropped() && !c->mem()->IsSnapshotSupported()) { + new_is_snapshot_supported = false; + break; + } + } + is_snapshot_supported_ = new_is_snapshot_supported; } } if (s.ok()) { + // Note that here we erase the associated cf_info of the to-be-dropped + // cfd before its ref-count goes to zero to avoid having to erase cf_info + // later inside db_mutex. + EraseThreadStatusCfInfo(cfd); assert(cfd->IsDropped()); - max_total_in_memory_state_ -= cfd->options()->write_buffer_size * - cfd->options()->max_write_buffer_number; - Log(options_.info_log, "Dropped column family with id %u\n", cfd->GetID()); + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "Dropped column family with id %u\n", cfd->GetID()); } else { - Log(options_.info_log, "Dropping column family with id %u FAILED -- %s\n", - cfd->GetID(), s.ToString().c_str()); + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "Dropping column family with id %u FAILED -- %s\n", + cfd->GetID(), s.ToString().c_str()); } return s; } -bool DBImpl::KeyMayExist(const ReadOptions& options, +bool DBImpl::KeyMayExist(const ReadOptions& read_options, ColumnFamilyHandle* column_family, const Slice& key, std::string* value, bool* value_found) { + assert(value != nullptr); if (value_found != nullptr) { // falsify later if key-may-exist but can't fetch value *value_found = true; } - ReadOptions roptions = options; + ReadOptions roptions = read_options; roptions.read_tier = kBlockCacheTier; // read from block cache only - auto s = GetImpl(roptions, column_family, key, value, value_found); + PinnableSlice pinnable_val; + auto s = GetImpl(roptions, column_family, key, &pinnable_val, value_found); + value->assign(pinnable_val.data(), pinnable_val.size()); // If block_cache is enabled and the index block of the table didn't // not present in block_cache, the return value will be Status::Incomplete. @@ -3664,774 +1383,210 @@ bool DBImpl::KeyMayExist(const ReadOptions& options, return s.ok() || s.IsIncomplete(); } -Iterator* DBImpl::NewIterator(const ReadOptions& options, +Iterator* DBImpl::NewIterator(const ReadOptions& read_options, ColumnFamilyHandle* column_family) { + if (read_options.read_tier == kPersistedTier) { + return NewErrorIterator(Status::NotSupported( + "ReadTier::kPersistedData is not yet supported in iterators.")); + } auto cfh = reinterpret_cast(column_family); auto cfd = cfh->cfd(); - - if (options.tailing) { + if (read_options.managed) { +#ifdef ROCKSDB_LITE + // not supported in lite version + return NewErrorIterator(Status::InvalidArgument( + "Managed Iterators not supported in RocksDBLite.")); +#else + if ((read_options.tailing) || (read_options.snapshot != nullptr) || + (is_snapshot_supported_)) { + return new ManagedIterator(this, read_options, cfd); + } + // Managed iter not supported + return NewErrorIterator(Status::InvalidArgument( + "Managed Iterators not supported without snapshots.")); +#endif + } else if (read_options.tailing) { #ifdef ROCKSDB_LITE // not supported in lite version return nullptr; #else - // TODO(ljin): remove tailing iterator - auto iter = new ForwardIterator(this, options, cfd); - return NewDBIterator(env_, *cfd->options(), cfd->user_comparator(), iter, - kMaxSequenceNumber); -// return new TailingIterator(env_, this, options, cfd); + SuperVersion* sv = cfd->GetReferencedSuperVersion(&mutex_); + auto iter = new ForwardIterator(this, read_options, cfd, sv); + return NewDBIterator( + env_, read_options, *cfd->ioptions(), cfd->user_comparator(), iter, + kMaxSequenceNumber, + sv->mutable_cf_options.max_sequential_skip_in_iterations); #endif } else { SequenceNumber latest_snapshot = versions_->LastSequence(); - SuperVersion* sv = nullptr; - sv = cfd->GetReferencedSuperVersion(&mutex_); - auto snapshot = - options.snapshot != nullptr - ? reinterpret_cast(options.snapshot)->number_ + read_options.snapshot != nullptr + ? reinterpret_cast(read_options.snapshot) + ->number_ : latest_snapshot; - - // Try to generate a DB iterator tree in continuous memory area to be - // cache friendly. Here is an example of result: - // +-------------------------------+ - // | | - // | ArenaWrappedDBIter | - // | + | - // | +---> Inner Iterator ------------+ - // | | | | - // | | +-- -- -- -- -- -- -- --+ | - // | +--- | Arena | | - // | | | | - // | Allocated Memory: | | - // | | +-------------------+ | - // | | | DBIter | <---+ - // | | + | - // | | | +-> iter_ ------------+ - // | | | | | - // | | +-------------------+ | - // | | | MergingIterator | <---+ - // | | + | - // | | | +->child iter1 ------------+ - // | | | | | | - // | | +->child iter2 ----------+ | - // | | | | | | | - // | | | +->child iter3 --------+ | | - // | | | | | | - // | | +-------------------+ | | | - // | | | Iterator1 | <--------+ - // | | +-------------------+ | | - // | | | Iterator2 | <------+ - // | | +-------------------+ | - // | | | Iterator3 | <----+ - // | | +-------------------+ - // | | | - // +-------+-----------------------+ - // - // ArenaWrappedDBIter inlines an arena area where all the iterartor in the - // the iterator tree is allocated in the order of being accessed when - // querying. - // Laying out the iterators in the order of being accessed makes it more - // likely that any iterator pointer is close to the iterator it points to so - // that they are likely to be in the same cache line and/or page. - ArenaWrappedDBIter* db_iter = NewArenaWrappedDbIterator( - env_, *cfd->options(), cfd->user_comparator(), snapshot); - Iterator* internal_iter = - NewInternalIterator(options, cfd, sv, db_iter->GetArena()); - db_iter->SetIterUnderDBIter(internal_iter); - - return db_iter; - } + return NewIteratorImpl(read_options, cfd, snapshot); + } + // To stop compiler from complaining + return nullptr; +} + +ArenaWrappedDBIter* DBImpl::NewIteratorImpl(const ReadOptions& read_options, + ColumnFamilyData* cfd, + SequenceNumber snapshot, + bool allow_blob) { + SuperVersion* sv = cfd->GetReferencedSuperVersion(&mutex_); + + // Try to generate a DB iterator tree in continuous memory area to be + // cache friendly. Here is an example of result: + // +-------------------------------+ + // | | + // | ArenaWrappedDBIter | + // | + | + // | +---> Inner Iterator ------------+ + // | | | | + // | | +-- -- -- -- -- -- -- --+ | + // | +--- | Arena | | + // | | | | + // | Allocated Memory: | | + // | | +-------------------+ | + // | | | DBIter | <---+ + // | | + | + // | | | +-> iter_ ------------+ + // | | | | | + // | | +-------------------+ | + // | | | MergingIterator | <---+ + // | | + | + // | | | +->child iter1 ------------+ + // | | | | | | + // | | +->child iter2 ----------+ | + // | | | | | | | + // | | | +->child iter3 --------+ | | + // | | | | | | + // | | +-------------------+ | | | + // | | | Iterator1 | <--------+ + // | | +-------------------+ | | + // | | | Iterator2 | <------+ + // | | +-------------------+ | + // | | | Iterator3 | <----+ + // | | +-------------------+ + // | | | + // +-------+-----------------------+ + // + // ArenaWrappedDBIter inlines an arena area where all the iterators in + // the iterator tree are allocated in the order of being accessed when + // querying. + // Laying out the iterators in the order of being accessed makes it more + // likely that any iterator pointer is close to the iterator it points to so + // that they are likely to be in the same cache line and/or page. + ArenaWrappedDBIter* db_iter = NewArenaWrappedDbIterator( + env_, read_options, *cfd->ioptions(), snapshot, + sv->mutable_cf_options.max_sequential_skip_in_iterations, + sv->version_number, ((read_options.snapshot != nullptr) ? nullptr : this), + cfd, allow_blob); + + InternalIterator* internal_iter = + NewInternalIterator(read_options, cfd, sv, db_iter->GetArena(), + db_iter->GetRangeDelAggregator()); + db_iter->SetIterUnderDBIter(internal_iter); + + return db_iter; } Status DBImpl::NewIterators( - const ReadOptions& options, + const ReadOptions& read_options, const std::vector& column_families, std::vector* iterators) { + if (read_options.read_tier == kPersistedTier) { + return Status::NotSupported( + "ReadTier::kPersistedData is not yet supported in iterators."); + } iterators->clear(); iterators->reserve(column_families.size()); - SequenceNumber latest_snapshot = 0; - std::vector super_versions; - super_versions.reserve(column_families.size()); - - if (!options.tailing) { - mutex_.Lock(); - latest_snapshot = versions_->LastSequence(); - for (auto cfh : column_families) { - auto cfd = reinterpret_cast(cfh)->cfd(); - super_versions.push_back(cfd->GetSuperVersion()->Ref()); - } - mutex_.Unlock(); - } - - if (options.tailing) { + if (read_options.managed) { #ifdef ROCKSDB_LITE return Status::InvalidArgument( - "Tailing interator not supported in RocksDB lite"); + "Managed interator not supported in RocksDB lite"); #else + if ((!read_options.tailing) && (read_options.snapshot == nullptr) && + (!is_snapshot_supported_)) { + return Status::InvalidArgument( + "Managed interator not supported without snapshots"); + } for (auto cfh : column_families) { auto cfd = reinterpret_cast(cfh)->cfd(); - auto iter = new ForwardIterator(this, options, cfd); - iterators->push_back( - NewDBIterator(env_, *cfd->options(), cfd->user_comparator(), iter, - kMaxSequenceNumber)); - } -#endif - } else { - for (size_t i = 0; i < column_families.size(); ++i) { - auto cfh = reinterpret_cast(column_families[i]); - auto cfd = cfh->cfd(); - - auto snapshot = - options.snapshot != nullptr - ? reinterpret_cast(options.snapshot)->number_ - : latest_snapshot; - - auto iter = NewInternalIterator(options, cfd, super_versions[i]); - iter = NewDBIterator(env_, *cfd->options(), - cfd->user_comparator(), iter, snapshot); + auto iter = new ManagedIterator(this, read_options, cfd); iterators->push_back(iter); } - } - - return Status::OK(); -} - -bool DBImpl::IsSnapshotSupported() const { - for (auto cfd : *versions_->GetColumnFamilySet()) { - if (!cfd->mem()->IsSnapshotSupported()) { - return false; - } - } - return true; -} - -const Snapshot* DBImpl::GetSnapshot() { - MutexLock l(&mutex_); - // returns null if the underlying memtable does not support snapshot. - if (!IsSnapshotSupported()) return nullptr; - return snapshots_.New(versions_->LastSequence()); -} - -void DBImpl::ReleaseSnapshot(const Snapshot* s) { - MutexLock l(&mutex_); - snapshots_.Delete(reinterpret_cast(s)); -} - -// Convenience methods -Status DBImpl::Put(const WriteOptions& o, ColumnFamilyHandle* column_family, - const Slice& key, const Slice& val) { - return DB::Put(o, column_family, key, val); -} - -Status DBImpl::Merge(const WriteOptions& o, ColumnFamilyHandle* column_family, - const Slice& key, const Slice& val) { - auto cfh = reinterpret_cast(column_family); - if (!cfh->cfd()->options()->merge_operator) { - return Status::NotSupported("Provide a merge_operator when opening DB"); - } else { - return DB::Merge(o, column_family, key, val); - } -} - -Status DBImpl::Delete(const WriteOptions& options, - ColumnFamilyHandle* column_family, const Slice& key) { - return DB::Delete(options, column_family, key); -} - -// REQUIRES: mutex_ is held -Status DBImpl::BeginWrite(Writer* w, uint64_t expiration_time) { - // the following code block pushes the current writer "w" into the writer - // queue "writers_" and wait until one of the following conditions met: - // 1. the job of "w" has been done by some other writers. - // 2. "w" becomes the first writer in "writers_" - // 3. "w" timed-out. - mutex_.AssertHeld(); - writers_.push_back(w); - - bool timed_out = false; - while (!w->done && w != writers_.front()) { - if (expiration_time == 0) { - w->cv.Wait(); - } else if (w->cv.TimedWait(expiration_time)) { - if (w->in_batch_group) { - // then it means the front writer is currently doing the - // write on behalf of this "timed-out" writer. Then it - // should wait until the write completes. - expiration_time = 0; - } else { - timed_out = true; - break; - } - } - } - - if (timed_out) { -#ifndef NDEBUG - bool found = false; -#endif - for (auto iter = writers_.begin(); iter != writers_.end(); iter++) { - if (*iter == w) { - writers_.erase(iter); -#ifndef NDEBUG - found = true; -#endif - break; - } - } -#ifndef NDEBUG - assert(found); -#endif - // writers_.front() might still be in cond_wait without a time-out. - // As a result, we need to signal it to wake it up. Otherwise no - // one else will wake him up, and RocksDB will hang. - if (!writers_.empty()) { - writers_.front()->cv.Signal(); - } - return Status::TimedOut(); - } - return Status::OK(); -} - -// REQUIRES: mutex_ is held -void DBImpl::EndWrite(Writer* w, Writer* last_writer, Status status) { - // Pop out the current writer and all writers being pushed before the - // current writer from the writer queue. - mutex_.AssertHeld(); - while (!writers_.empty()) { - Writer* ready = writers_.front(); - writers_.pop_front(); - if (ready != w) { - ready->status = status; - ready->done = true; - ready->cv.Signal(); - } - if (ready == last_writer) break; - } - - // Notify new head of write queue - if (!writers_.empty()) { - writers_.front()->cv.Signal(); - } -} - -Status DBImpl::Write(const WriteOptions& options, WriteBatch* my_batch) { - if (my_batch == nullptr) { - return Status::Corruption("Batch is nullptr!"); - } - PERF_TIMER_GUARD(write_pre_and_post_process_time); - Writer w(&mutex_); - w.batch = my_batch; - w.sync = options.sync; - w.disableWAL = options.disableWAL; - w.in_batch_group = false; - w.done = false; - w.timeout_hint_us = options.timeout_hint_us; - - uint64_t expiration_time = 0; - if (w.timeout_hint_us == 0) { - w.timeout_hint_us = kNoTimeOut; - } else { - expiration_time = env_->NowMicros() + w.timeout_hint_us; - } - - if (!options.disableWAL) { - RecordTick(stats_, WRITE_WITH_WAL); - default_cf_internal_stats_->AddDBStats(InternalStats::WRITE_WITH_WAL, 1); - } - - WriteContext context; - mutex_.Lock(); - Status status = BeginWrite(&w, expiration_time); - assert(status.ok() || status.IsTimedOut()); - if (status.IsTimedOut()) { - mutex_.Unlock(); - RecordTick(stats_, WRITE_TIMEDOUT); - return Status::TimedOut(); - } - if (w.done) { // write was done by someone else - default_cf_internal_stats_->AddDBStats(InternalStats::WRITE_DONE_BY_OTHER, - 1); - mutex_.Unlock(); - RecordTick(stats_, WRITE_DONE_BY_OTHER); - return w.status; - } - - RecordTick(stats_, WRITE_DONE_BY_SELF); - default_cf_internal_stats_->AddDBStats(InternalStats::WRITE_DONE_BY_SELF, 1); - - // Once reaches this point, the current writer "w" will try to do its write - // job. It may also pick up some of the remaining writers in the "writers_" - // when it finds suitable, and finish them in the same write batch. - // This is how a write job could be done by the other writer. - assert(!single_column_family_mode_ || - versions_->GetColumnFamilySet()->NumberOfColumnFamilies() == 1); - - uint64_t flush_column_family_if_log_file = 0; - uint64_t max_total_wal_size = (options_.max_total_wal_size == 0) - ? 4 * max_total_in_memory_state_ - : options_.max_total_wal_size; - if (UNLIKELY(!single_column_family_mode_) && - alive_log_files_.begin()->getting_flushed == false && - total_log_size_ > max_total_wal_size) { - flush_column_family_if_log_file = alive_log_files_.begin()->number; - alive_log_files_.begin()->getting_flushed = true; - Log(options_.info_log, - "Flushing all column families with data in WAL number %" PRIu64 - ". Total log size is %" PRIu64 " while max_total_wal_size is %" PRIu64, - flush_column_family_if_log_file, total_log_size_, max_total_wal_size); - } - - if (LIKELY(single_column_family_mode_)) { - // fast path - status = MakeRoomForWrite(default_cf_handle_->cfd(), - &context, expiration_time); - } else { - // refcounting cfd in iteration - bool dead_cfd = false; - for (auto cfd : *versions_->GetColumnFamilySet()) { - cfd->Ref(); - if (flush_column_family_if_log_file != 0 && - cfd->GetLogNumber() <= flush_column_family_if_log_file) { - // log size excedded limit and we need to do flush - // SetNewMemtableAndNewLogFie may temporarily unlock and wait - status = SetNewMemtableAndNewLogFile(cfd, &context); - cfd->imm()->FlushRequested(); - MaybeScheduleFlushOrCompaction(); - } else { - // May temporarily unlock and wait. - status = MakeRoomForWrite(cfd, &context, expiration_time); - } - - if (cfd->Unref()) { - dead_cfd = true; - } - if (!status.ok()) { - break; - } - } - if (dead_cfd) { - versions_->GetColumnFamilySet()->FreeDeadColumnFamilies(); - } - } - - uint64_t last_sequence = versions_->LastSequence(); - Writer* last_writer = &w; - if (status.ok()) { - autovector write_batch_group; - BuildBatchGroup(&last_writer, &write_batch_group); - - // Add to log and apply to memtable. We can release the lock - // during this phase since &w is currently responsible for logging - // and protects against concurrent loggers and concurrent writes - // into memtables - { - mutex_.Unlock(); - WriteBatch* updates = nullptr; - if (write_batch_group.size() == 1) { - updates = write_batch_group[0]; - } else { - updates = &tmp_batch_; - for (size_t i = 0; i < write_batch_group.size(); ++i) { - WriteBatchInternal::Append(updates, write_batch_group[i]); - } - } - - const SequenceNumber current_sequence = last_sequence + 1; - WriteBatchInternal::SetSequence(updates, current_sequence); - int my_batch_count = WriteBatchInternal::Count(updates); - last_sequence += my_batch_count; - const uint64_t batch_size = WriteBatchInternal::ByteSize(updates); - // Record statistics - RecordTick(stats_, NUMBER_KEYS_WRITTEN, my_batch_count); - RecordTick(stats_, BYTES_WRITTEN, WriteBatchInternal::ByteSize(updates)); - if (options.disableWAL) { - flush_on_destroy_ = true; - } - PERF_TIMER_STOP(write_pre_and_post_process_time); - - uint64_t log_size = 0; - if (!options.disableWAL) { - PERF_TIMER_GUARD(write_wal_time); - Slice log_entry = WriteBatchInternal::Contents(updates); - status = log_->AddRecord(log_entry); - total_log_size_ += log_entry.size(); - alive_log_files_.back().AddSize(log_entry.size()); - log_empty_ = false; - log_size = log_entry.size(); - RecordTick(stats_, WAL_FILE_SYNCED); - RecordTick(stats_, WAL_FILE_BYTES, log_size); - if (status.ok() && options.sync) { - if (options_.use_fsync) { - StopWatch(env_, stats_, WAL_FILE_SYNC_MICROS); - status = log_->file()->Fsync(); - } else { - StopWatch(env_, stats_, WAL_FILE_SYNC_MICROS); - status = log_->file()->Sync(); - } - } - } - if (status.ok()) { - PERF_TIMER_GUARD(write_memtable_time); - - status = WriteBatchInternal::InsertInto( - updates, column_family_memtables_.get(), - options.ignore_missing_column_families, 0, this, false); - // A non-OK status here indicates iteration failure (either in-memory - // writebatch corruption (very bad), or the client specified invalid - // column family). This will later on trigger bg_error_. - // - // Note that existing logic was not sound. Any partial failure writing - // into the memtable would result in a state that some write ops might - // have succeeded in memtable but Status reports error for all writes. - - SetTickerCount(stats_, SEQUENCE_NUMBER, last_sequence); - } - PERF_TIMER_START(write_pre_and_post_process_time); - if (updates == &tmp_batch_) { - tmp_batch_.Clear(); - } - mutex_.Lock(); - // internal stats - default_cf_internal_stats_->AddDBStats( - InternalStats::BYTES_WRITTEN, batch_size); - if (!options.disableWAL) { - default_cf_internal_stats_->AddDBStats( - InternalStats::WAL_FILE_SYNCED, 1); - default_cf_internal_stats_->AddDBStats( - InternalStats::WAL_FILE_BYTES, log_size); - } - if (status.ok()) { - versions_->SetLastSequence(last_sequence); - } - } - } - if (options_.paranoid_checks && !status.ok() && - !status.IsTimedOut() && bg_error_.ok()) { - bg_error_ = status; // stop compaction & fail any further writes - } - - EndWrite(&w, last_writer, status); - mutex_.Unlock(); - - if (status.IsTimedOut()) { - RecordTick(stats_, WRITE_TIMEDOUT); - } - - return status; -} - -// This function will be called only when the first writer succeeds. -// All writers in the to-be-built batch group will be processed. -// -// REQUIRES: Writer list must be non-empty -// REQUIRES: First writer must have a non-nullptr batch -void DBImpl::BuildBatchGroup(Writer** last_writer, - autovector* write_batch_group) { - assert(!writers_.empty()); - Writer* first = writers_.front(); - assert(first->batch != nullptr); - - size_t size = WriteBatchInternal::ByteSize(first->batch); - write_batch_group->push_back(first->batch); - - // Allow the group to grow up to a maximum size, but if the - // original write is small, limit the growth so we do not slow - // down the small write too much. - size_t max_size = 1 << 20; - if (size <= (128<<10)) { - max_size = size + (128<<10); - } - - *last_writer = first; - std::deque::iterator iter = writers_.begin(); - ++iter; // Advance past "first" - for (; iter != writers_.end(); ++iter) { - Writer* w = *iter; - if (w->sync && !first->sync) { - // Do not include a sync write into a batch handled by a non-sync write. - break; - } - - if (!w->disableWAL && first->disableWAL) { - // Do not include a write that needs WAL into a batch that has - // WAL disabled. - break; - } - - if (w->timeout_hint_us < first->timeout_hint_us) { - // Do not include those writes with shorter timeout. Otherwise, we might - // execute a write that should instead be aborted because of timeout. - break; - } - - if (w->batch == nullptr) { - // Do not include those writes with nullptr batch. Those are not writes, - // those are something else. They want to be alone - break; - } - - size += WriteBatchInternal::ByteSize(w->batch); - if (size > max_size) { - // Do not make batch too big - break; - } - - write_batch_group->push_back(w->batch); - w->in_batch_group = true; - *last_writer = w; - } -} - -// This function computes the amount of time in microseconds by which a write -// should be delayed based on the number of level-0 files according to the -// following formula: -// if n < bottom, return 0; -// if n >= top, return 1000; -// otherwise, let r = (n - bottom) / -// (top - bottom) -// and return r^2 * 1000. -// The goal of this formula is to gradually increase the rate at which writes -// are slowed. We also tried linear delay (r * 1000), but it seemed to do -// slightly worse. There is no other particular reason for choosing quadratic. -uint64_t DBImpl::SlowdownAmount(int n, double bottom, double top) { - uint64_t delay; - if (n >= top) { - delay = 1000; - } - else if (n < bottom) { - delay = 0; - } - else { - // If we are here, we know that: - // level0_start_slowdown <= n < level0_slowdown - // since the previous two conditions are false. - double how_much = - (double) (n - bottom) / - (top - bottom); - delay = std::max(how_much * how_much * 1000, 100.0); - } - assert(delay <= 1000); - return delay; -} - -// REQUIRES: mutex_ is held -// REQUIRES: this thread is currently at the front of the writer queue -Status DBImpl::MakeRoomForWrite(ColumnFamilyData* cfd, - WriteContext* context, - uint64_t expiration_time) { - mutex_.AssertHeld(); - assert(!writers_.empty()); - bool allow_delay = true; - bool allow_hard_rate_limit_delay = true; - bool allow_soft_rate_limit_delay = true; - uint64_t rate_limit_delay_millis = 0; - Status s; - double score; - // Once we schedule background work, we shouldn't schedule it again, since it - // might generate a tight feedback loop, constantly scheduling more background - // work, even if additional background work is not needed - bool schedule_background_work = true; - bool has_timeout = (expiration_time > 0); - - while (true) { - if (!bg_error_.ok()) { - // Yield previous error - s = bg_error_; - break; - } else if (has_timeout && env_->NowMicros() > expiration_time) { - s = Status::TimedOut(); - break; - } else if (allow_delay && cfd->NeedSlowdownForNumLevel0Files()) { - // We are getting close to hitting a hard limit on the number of - // L0 files. Rather than delaying a single write by several - // seconds when we hit the hard limit, start delaying each - // individual write by 0-1ms to reduce latency variance. Also, - // this delay hands over some CPU to the compaction thread in - // case it is sharing the same core as the writer. - uint64_t slowdown = - SlowdownAmount(cfd->current()->NumLevelFiles(0), - cfd->options()->level0_slowdown_writes_trigger, - cfd->options()->level0_stop_writes_trigger); - mutex_.Unlock(); - uint64_t delayed; - { - StopWatch sw(env_, stats_, STALL_L0_SLOWDOWN_COUNT, &delayed); - env_->SleepForMicroseconds(slowdown); - } - RecordTick(stats_, STALL_L0_SLOWDOWN_MICROS, delayed); - allow_delay = false; // Do not delay a single write more than once - mutex_.Lock(); - cfd->internal_stats()->AddCFStats( - InternalStats::LEVEL0_SLOWDOWN, delayed); - delayed_writes_++; - } else if (!cfd->mem()->ShouldFlush()) { - // There is room in current memtable - if (allow_delay) { - DelayLoggingAndReset(); - } - break; - } else if (cfd->NeedWaitForNumMemtables()) { - // We have filled up the current memtable, but the previous - // ones are still being flushed, so we wait. - DelayLoggingAndReset(); - Log(options_.info_log, "[%s] wait for memtable flush...\n", - cfd->GetName().c_str()); - if (schedule_background_work) { - MaybeScheduleFlushOrCompaction(); - schedule_background_work = false; - } - uint64_t stall; - { - StopWatch sw(env_, stats_, STALL_MEMTABLE_COMPACTION_COUNT, &stall); - if (!has_timeout) { - bg_cv_.Wait(); - } else { - bg_cv_.TimedWait(expiration_time); - } - } - RecordTick(stats_, STALL_MEMTABLE_COMPACTION_MICROS, stall); - cfd->internal_stats()->AddCFStats( - InternalStats::MEMTABLE_COMPACTION, stall); - } else if (cfd->NeedWaitForNumLevel0Files()) { - DelayLoggingAndReset(); - Log(options_.info_log, "[%s] wait for fewer level0 files...\n", - cfd->GetName().c_str()); - uint64_t stall; - { - StopWatch sw(env_, stats_, STALL_L0_NUM_FILES_COUNT, &stall); - if (!has_timeout) { - bg_cv_.Wait(); - } else { - bg_cv_.TimedWait(expiration_time); - } - } - RecordTick(stats_, STALL_L0_NUM_FILES_MICROS, stall); - cfd->internal_stats()->AddCFStats( - InternalStats::LEVEL0_NUM_FILES, stall); - } else if (allow_hard_rate_limit_delay && cfd->ExceedsHardRateLimit()) { - // Delay a write when the compaction score for any level is too large. - const int max_level = cfd->current()->MaxCompactionScoreLevel(); - score = cfd->current()->MaxCompactionScore(); - mutex_.Unlock(); - uint64_t delayed; - { - StopWatch sw(env_, stats_, HARD_RATE_LIMIT_DELAY_COUNT, &delayed); - env_->SleepForMicroseconds(1000); - } - // Make sure the following value doesn't round to zero. - uint64_t rate_limit = std::max((delayed / 1000), (uint64_t) 1); - rate_limit_delay_millis += rate_limit; - RecordTick(stats_, RATE_LIMIT_DELAY_MILLIS, rate_limit); - if (cfd->options()->rate_limit_delay_max_milliseconds > 0 && - rate_limit_delay_millis >= - (unsigned)cfd->options()->rate_limit_delay_max_milliseconds) { - allow_hard_rate_limit_delay = false; - } - mutex_.Lock(); - cfd->internal_stats()->RecordLevelNSlowdown(max_level, delayed, false); - } else if (allow_soft_rate_limit_delay && cfd->ExceedsSoftRateLimit()) { - const int max_level = cfd->current()->MaxCompactionScoreLevel(); - score = cfd->current()->MaxCompactionScore(); - // Delay a write when the compaction score for any level is too large. - // TODO: add statistics - uint64_t slowdown = SlowdownAmount(score, cfd->options()->soft_rate_limit, - cfd->options()->hard_rate_limit); - uint64_t elapsed = 0; - mutex_.Unlock(); - { - StopWatch sw(env_, stats_, SOFT_RATE_LIMIT_DELAY_COUNT, &elapsed); - env_->SleepForMicroseconds(slowdown); - rate_limit_delay_millis += slowdown; - } - allow_soft_rate_limit_delay = false; - mutex_.Lock(); - cfd->internal_stats()->RecordLevelNSlowdown(max_level, elapsed, true); - } else { - s = SetNewMemtableAndNewLogFile(cfd, context); - if (!s.ok()) { - break; - } - MaybeScheduleFlushOrCompaction(); - } - } - return s; -} - -// REQUIRES: mutex_ is held -// REQUIRES: this thread is currently at the front of the writer queue -Status DBImpl::SetNewMemtableAndNewLogFile(ColumnFamilyData* cfd, - WriteContext* context) { - mutex_.AssertHeld(); - unique_ptr lfile; - log::Writer* new_log = nullptr; - MemTable* new_mem = nullptr; - - // Attempt to switch to a new memtable and trigger flush of old. - // Do this without holding the dbmutex lock. - assert(versions_->PrevLogNumber() == 0); - bool creating_new_log = !log_empty_; - uint64_t new_log_number = - creating_new_log ? versions_->NewFileNumber() : logfile_number_; - SuperVersion* new_superversion = nullptr; - mutex_.Unlock(); - Status s; - { - DelayLoggingAndReset(); - if (creating_new_log) { - s = env_->NewWritableFile(LogFileName(options_.wal_dir, new_log_number), - &lfile, - env_->OptimizeForLogWrite(storage_options_)); - if (s.ok()) { - // Our final size should be less than write_buffer_size - // (compression, etc) but err on the side of caution. - lfile->SetPreallocationBlockSize(1.1 * - cfd->options()->write_buffer_size); - new_log = new log::Writer(std::move(lfile)); - } +#endif + } else if (read_options.tailing) { +#ifdef ROCKSDB_LITE + return Status::InvalidArgument( + "Tailing interator not supported in RocksDB lite"); +#else + for (auto cfh : column_families) { + auto cfd = reinterpret_cast(cfh)->cfd(); + SuperVersion* sv = cfd->GetReferencedSuperVersion(&mutex_); + auto iter = new ForwardIterator(this, read_options, cfd, sv); + iterators->push_back(NewDBIterator( + env_, read_options, *cfd->ioptions(), cfd->user_comparator(), iter, + kMaxSequenceNumber, + sv->mutable_cf_options.max_sequential_skip_in_iterations)); } +#endif + } else { + SequenceNumber latest_snapshot = versions_->LastSequence(); + auto snapshot = + read_options.snapshot != nullptr + ? reinterpret_cast(read_options.snapshot) + ->number_ + : latest_snapshot; - if (s.ok()) { - new_mem = new MemTable(cfd->internal_comparator(), *cfd->options()); - new_superversion = new SuperVersion(); + for (size_t i = 0; i < column_families.size(); ++i) { + auto* cfd = reinterpret_cast( + column_families[i])->cfd(); + iterators->push_back(NewIteratorImpl(read_options, cfd, snapshot)); } } - mutex_.Lock(); - if (!s.ok()) { - // how do we fail if we're not creating new log? - assert(creating_new_log); - // Avoid chewing through file number space in a tight loop. - versions_->ReuseLogFileNumber(new_log_number); - assert(!new_mem); - assert(!new_log); - return s; + + return Status::OK(); +} + +const Snapshot* DBImpl::GetSnapshot() { return GetSnapshotImpl(false); } + +#ifndef ROCKSDB_LITE +const Snapshot* DBImpl::GetSnapshotForWriteConflictBoundary() { + return GetSnapshotImpl(true); +} +#endif // ROCKSDB_LITE + +const Snapshot* DBImpl::GetSnapshotImpl(bool is_write_conflict_boundary) { + int64_t unix_time = 0; + env_->GetCurrentTime(&unix_time); // Ignore error + SnapshotImpl* s = new SnapshotImpl; + + InstrumentedMutexLock l(&mutex_); + // returns null if the underlying memtable does not support snapshot. + if (!is_snapshot_supported_) { + delete s; + return nullptr; } - if (creating_new_log) { - logfile_number_ = new_log_number; - assert(new_log != nullptr); - context->logs_to_free_.push_back(log_.release()); - log_.reset(new_log); - log_empty_ = true; - alive_log_files_.push_back(LogFileNumberSize(logfile_number_)); - for (auto cfd : *versions_->GetColumnFamilySet()) { - // all this is just optimization to delete logs that - // are no longer needed -- if CF is empty, that means it - // doesn't need that particular log to stay alive, so we just - // advance the log number. no need to persist this in the manifest - if (cfd->mem()->GetFirstSequenceNumber() == 0 && - cfd->imm()->size() == 0) { - cfd->SetLogNumber(logfile_number_); - } - } + return snapshots_.New(s, versions_->LastSequence(), unix_time, + is_write_conflict_boundary); +} + +void DBImpl::ReleaseSnapshot(const Snapshot* s) { + const SnapshotImpl* casted_s = reinterpret_cast(s); + { + InstrumentedMutexLock l(&mutex_); + snapshots_.Delete(casted_s); } - cfd->mem()->SetNextLogNumber(logfile_number_); - cfd->imm()->Add(cfd->mem()); - new_mem->Ref(); - cfd->SetMemtable(new_mem); - Log(options_.info_log, - "[%s] New memtable created with log file: #%" PRIu64 "\n", - cfd->GetName().c_str(), logfile_number_); - context->superversions_to_free_.push_back( - cfd->InstallSuperVersion(new_superversion, &mutex_)); - return s; + delete casted_s; +} + +bool DBImpl::HasActiveSnapshotInRange(SequenceNumber lower_bound, + SequenceNumber upper_bound) { + InstrumentedMutexLock l(&mutex_); + return snapshots_.HasSnapshotInRange(lower_bound, upper_bound); } #ifndef ROCKSDB_LITE @@ -4455,6 +1610,29 @@ Status DBImpl::GetPropertiesOfAllTables(ColumnFamilyHandle* column_family, return s; } + +Status DBImpl::GetPropertiesOfTablesInRange(ColumnFamilyHandle* column_family, + const Range* range, std::size_t n, + TablePropertiesCollection* props) { + auto cfh = reinterpret_cast(column_family); + auto cfd = cfh->cfd(); + + // Increment the ref count + mutex_.Lock(); + auto version = cfd->current(); + version->Ref(); + mutex_.Unlock(); + + auto s = version->GetPropertiesOfTablesInRange(range, n, props); + + // Decrement the ref count + mutex_.Lock(); + version->Unref(); + mutex_.Unlock(); + + return s; +} + #endif // ROCKSDB_LITE const std::string& DBImpl::GetName() const { @@ -4465,92 +1643,168 @@ Env* DBImpl::GetEnv() const { return env_; } -const Options& DBImpl::GetOptions(ColumnFamilyHandle* column_family) const { +Options DBImpl::GetOptions(ColumnFamilyHandle* column_family) const { + InstrumentedMutexLock l(&mutex_); auto cfh = reinterpret_cast(column_family); - return *cfh->cfd()->options(); + return Options(BuildDBOptions(immutable_db_options_, mutable_db_options_), + cfh->cfd()->GetLatestCFOptions()); +} + +DBOptions DBImpl::GetDBOptions() const { + InstrumentedMutexLock l(&mutex_); + return BuildDBOptions(immutable_db_options_, mutable_db_options_); } bool DBImpl::GetProperty(ColumnFamilyHandle* column_family, const Slice& property, std::string* value) { - bool is_int_property = false; - bool need_out_of_mutex = false; - DBPropertyType property_type = - GetPropertyType(property, &is_int_property, &need_out_of_mutex); - + const DBPropertyInfo* property_info = GetPropertyInfo(property); value->clear(); - if (is_int_property) { + auto cfd = reinterpret_cast(column_family)->cfd(); + if (property_info == nullptr) { + return false; + } else if (property_info->handle_int) { uint64_t int_value; - bool ret_value = GetIntPropertyInternal(column_family, property_type, - need_out_of_mutex, &int_value); + bool ret_value = + GetIntPropertyInternal(cfd, *property_info, false, &int_value); if (ret_value) { - *value = std::to_string(int_value); + *value = ToString(int_value); } return ret_value; - } else { - auto cfh = reinterpret_cast(column_family); - auto cfd = cfh->cfd(); - MutexLock l(&mutex_); - return cfd->internal_stats()->GetStringProperty(property_type, property, + } else if (property_info->handle_string) { + InstrumentedMutexLock l(&mutex_); + return cfd->internal_stats()->GetStringProperty(*property_info, property, value); } + // Shouldn't reach here since exactly one of handle_string and handle_int + // should be non-nullptr. + assert(false); + return false; +} + +bool DBImpl::GetMapProperty(ColumnFamilyHandle* column_family, + const Slice& property, + std::map* value) { + const DBPropertyInfo* property_info = GetPropertyInfo(property); + value->clear(); + auto cfd = reinterpret_cast(column_family)->cfd(); + if (property_info == nullptr) { + return false; + } else if (property_info->handle_map) { + InstrumentedMutexLock l(&mutex_); + return cfd->internal_stats()->GetMapProperty(*property_info, property, + value); + } + // If we reach this point it means that handle_map is not provided for the + // requested property + return false; } bool DBImpl::GetIntProperty(ColumnFamilyHandle* column_family, const Slice& property, uint64_t* value) { - bool is_int_property = false; - bool need_out_of_mutex = false; - DBPropertyType property_type = - GetPropertyType(property, &is_int_property, &need_out_of_mutex); - if (!is_int_property) { + const DBPropertyInfo* property_info = GetPropertyInfo(property); + if (property_info == nullptr || property_info->handle_int == nullptr) { return false; } - return GetIntPropertyInternal(column_family, property_type, need_out_of_mutex, - value); + auto cfd = reinterpret_cast(column_family)->cfd(); + return GetIntPropertyInternal(cfd, *property_info, false, value); } -bool DBImpl::GetIntPropertyInternal(ColumnFamilyHandle* column_family, - DBPropertyType property_type, - bool need_out_of_mutex, uint64_t* value) { - auto cfh = reinterpret_cast(column_family); - auto cfd = cfh->cfd(); - - if (!need_out_of_mutex) { - MutexLock l(&mutex_); - return cfd->internal_stats()->GetIntProperty(property_type, value, this); +bool DBImpl::GetIntPropertyInternal(ColumnFamilyData* cfd, + const DBPropertyInfo& property_info, + bool is_locked, uint64_t* value) { + assert(property_info.handle_int != nullptr); + if (!property_info.need_out_of_mutex) { + if (is_locked) { + mutex_.AssertHeld(); + return cfd->internal_stats()->GetIntProperty(property_info, value, this); + } else { + InstrumentedMutexLock l(&mutex_); + return cfd->internal_stats()->GetIntProperty(property_info, value, this); + } } else { - SuperVersion* sv = GetAndRefSuperVersion(cfd); + SuperVersion* sv = nullptr; + if (!is_locked) { + sv = GetAndRefSuperVersion(cfd); + } else { + sv = cfd->GetSuperVersion(); + } bool ret = cfd->internal_stats()->GetIntPropertyOutOfMutex( - property_type, sv->current, value); + property_info, sv->current, value); - ReturnAndCleanupSuperVersion(cfd, sv); + if (!is_locked) { + ReturnAndCleanupSuperVersion(cfd, sv); + } return ret; } } +#ifndef ROCKSDB_LITE +Status DBImpl::ResetStats() { + InstrumentedMutexLock l(&mutex_); + for (auto* cfd : *versions_->GetColumnFamilySet()) { + if (cfd->initialized()) { + cfd->internal_stats()->Clear(); + } + } + return Status::OK(); +} +#endif // ROCKSDB_LITE + +bool DBImpl::GetAggregatedIntProperty(const Slice& property, + uint64_t* aggregated_value) { + const DBPropertyInfo* property_info = GetPropertyInfo(property); + if (property_info == nullptr || property_info->handle_int == nullptr) { + return false; + } + + uint64_t sum = 0; + { + // Needs mutex to protect the list of column families. + InstrumentedMutexLock l(&mutex_); + uint64_t value; + for (auto* cfd : *versions_->GetColumnFamilySet()) { + if (!cfd->initialized()) { + continue; + } + if (GetIntPropertyInternal(cfd, *property_info, true, &value)) { + sum += value; + } else { + return false; + } + } + } + *aggregated_value = sum; + return true; +} + SuperVersion* DBImpl::GetAndRefSuperVersion(ColumnFamilyData* cfd) { // TODO(ljin): consider using GetReferencedSuperVersion() directly - if (LIKELY(options_.allow_thread_local)) { - return cfd->GetThreadLocalSuperVersion(&mutex_); - } else { - MutexLock l(&mutex_); - return cfd->GetSuperVersion()->Ref(); + return cfd->GetThreadLocalSuperVersion(&mutex_); +} + +// REQUIRED: this function should only be called on the write thread or if the +// mutex is held. +SuperVersion* DBImpl::GetAndRefSuperVersion(uint32_t column_family_id) { + auto column_family_set = versions_->GetColumnFamilySet(); + auto cfd = column_family_set->GetColumnFamily(column_family_id); + if (!cfd) { + return nullptr; } + + return GetAndRefSuperVersion(cfd); } void DBImpl::ReturnAndCleanupSuperVersion(ColumnFamilyData* cfd, SuperVersion* sv) { - bool unref_sv = true; - if (LIKELY(options_.allow_thread_local)) { - unref_sv = !cfd->ReturnThreadLocalSuperVersion(sv); - } + bool unref_sv = !cfd->ReturnThreadLocalSuperVersion(sv); if (unref_sv) { // Release SuperVersion if (sv->Unref()) { { - MutexLock l(&mutex_); + InstrumentedMutexLock l(&mutex_); sv->Cleanup(); } delete sv; @@ -4560,38 +1814,108 @@ void DBImpl::ReturnAndCleanupSuperVersion(ColumnFamilyData* cfd, } } +// REQUIRED: this function should only be called on the write thread. +void DBImpl::ReturnAndCleanupSuperVersion(uint32_t column_family_id, + SuperVersion* sv) { + auto column_family_set = versions_->GetColumnFamilySet(); + auto cfd = column_family_set->GetColumnFamily(column_family_id); + + // If SuperVersion is held, and we successfully fetched a cfd using + // GetAndRefSuperVersion(), it must still exist. + assert(cfd != nullptr); + ReturnAndCleanupSuperVersion(cfd, sv); +} + +// REQUIRED: this function should only be called on the write thread or if the +// mutex is held. +ColumnFamilyHandle* DBImpl::GetColumnFamilyHandle(uint32_t column_family_id) { + ColumnFamilyMemTables* cf_memtables = column_family_memtables_.get(); + + if (!cf_memtables->Seek(column_family_id)) { + return nullptr; + } + + return cf_memtables->GetColumnFamilyHandle(); +} + +// REQUIRED: mutex is NOT held. +ColumnFamilyHandle* DBImpl::GetColumnFamilyHandleUnlocked( + uint32_t column_family_id) { + ColumnFamilyMemTables* cf_memtables = column_family_memtables_.get(); + + InstrumentedMutexLock l(&mutex_); + + if (!cf_memtables->Seek(column_family_id)) { + return nullptr; + } + + return cf_memtables->GetColumnFamilyHandle(); +} + +void DBImpl::GetApproximateMemTableStats(ColumnFamilyHandle* column_family, + const Range& range, + uint64_t* const count, + uint64_t* const size) { + ColumnFamilyHandleImpl* cfh = + reinterpret_cast(column_family); + ColumnFamilyData* cfd = cfh->cfd(); + SuperVersion* sv = GetAndRefSuperVersion(cfd); + + // Convert user_key into a corresponding internal key. + InternalKey k1(range.start, kMaxSequenceNumber, kValueTypeForSeek); + InternalKey k2(range.limit, kMaxSequenceNumber, kValueTypeForSeek); + MemTable::MemTableStats memStats = + sv->mem->ApproximateStats(k1.Encode(), k2.Encode()); + MemTable::MemTableStats immStats = + sv->imm->ApproximateStats(k1.Encode(), k2.Encode()); + *count = memStats.count + immStats.count; + *size = memStats.size + immStats.size; + + ReturnAndCleanupSuperVersion(cfd, sv); +} + void DBImpl::GetApproximateSizes(ColumnFamilyHandle* column_family, - const Range* range, int n, uint64_t* sizes) { - // TODO(opt): better implementation + const Range* range, int n, uint64_t* sizes, + uint8_t include_flags) { + assert(include_flags & DB::SizeApproximationFlags::INCLUDE_FILES || + include_flags & DB::SizeApproximationFlags::INCLUDE_MEMTABLES); Version* v; auto cfh = reinterpret_cast(column_family); auto cfd = cfh->cfd(); - { - MutexLock l(&mutex_); - v = cfd->current(); - v->Ref(); - } + SuperVersion* sv = GetAndRefSuperVersion(cfd); + v = sv->current; for (int i = 0; i < n; i++) { // Convert user_key into a corresponding internal key. InternalKey k1(range[i].start, kMaxSequenceNumber, kValueTypeForSeek); InternalKey k2(range[i].limit, kMaxSequenceNumber, kValueTypeForSeek); - uint64_t start = versions_->ApproximateOffsetOf(v, k1); - uint64_t limit = versions_->ApproximateOffsetOf(v, k2); - sizes[i] = (limit >= start ? limit - start : 0); + sizes[i] = 0; + if (include_flags & DB::SizeApproximationFlags::INCLUDE_FILES) { + sizes[i] += versions_->ApproximateSize(v, k1.Encode(), k2.Encode()); + } + if (include_flags & DB::SizeApproximationFlags::INCLUDE_MEMTABLES) { + sizes[i] += sv->mem->ApproximateStats(k1.Encode(), k2.Encode()).size; + sizes[i] += sv->imm->ApproximateStats(k1.Encode(), k2.Encode()).size; + } } - { - MutexLock l(&mutex_); - v->Unref(); - } + ReturnAndCleanupSuperVersion(cfd, sv); } -inline void DBImpl::DelayLoggingAndReset() { - if (delayed_writes_ > 0) { - Log(options_.info_log, "delayed %d write...\n", delayed_writes_ ); - delayed_writes_ = 0; - } +std::list::iterator +DBImpl::CaptureCurrentFileNumberInPendingOutputs() { + // We need to remember the iterator of our insert, because after the + // background job is done, we need to remove that element from + // pending_outputs_. + pending_outputs_.push_back(versions_->current_next_file_number()); + auto pending_outputs_inserted_elem = pending_outputs_.end(); + --pending_outputs_inserted_elem; + return pending_outputs_inserted_elem; +} + +void DBImpl::ReleaseFileNumberFromPendingOutputs( + std::list::iterator v) { + pending_outputs_.erase(v); } #ifndef ROCKSDB_LITE @@ -4603,23 +1927,7 @@ Status DBImpl::GetUpdatesSince( if (seq > versions_->LastSequence()) { return Status::NotFound("Requested sequence not yet written in the db"); } - // Get all sorted Wal Files. - // Do binary search and open files and find the seq number. - - std::unique_ptr wal_files(new VectorLogPtr); - Status s = GetSortedWalFiles(*wal_files); - if (!s.ok()) { - return s; - } - - s = RetainProbableWalFiles(*wal_files, seq); - if (!s.ok()) { - return s; - } - iter->reset(new TransactionLogIteratorImpl(options_.wal_dir, &options_, - read_options, storage_options_, - seq, std::move(wal_files), this)); - return (*iter)->status(); + return wal_manager_.GetUpdatesSince(seq, iter, read_options, versions_.get()); } Status DBImpl::DeleteFile(std::string name) { @@ -4628,7 +1936,8 @@ Status DBImpl::DeleteFile(std::string name) { WalFileType log_type; if (!ParseFileName(name, &number, &type, &log_type) || (type != kTableFile && type != kLogFile)) { - Log(options_.info_log, "DeleteFile %s failed.\n", name.c_str()); + ROCKS_LOG_ERROR(immutable_db_options_.info_log, "DeleteFile %s failed.\n", + name.c_str()); return Status::InvalidArgument("Invalid file name"); } @@ -4636,14 +1945,17 @@ Status DBImpl::DeleteFile(std::string name) { if (type == kLogFile) { // Only allow deleting archived log files if (log_type != kArchivedLogFile) { - Log(options_.info_log, "DeleteFile %s failed - not archived log.\n", - name.c_str()); + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "DeleteFile %s failed - not archived log.\n", + name.c_str()); return Status::NotSupported("Delete only supported for archived logs"); } - status = env_->DeleteFile(options_.wal_dir + "/" + name.c_str()); + status = + env_->DeleteFile(immutable_db_options_.wal_dir + "/" + name.c_str()); if (!status.ok()) { - Log(options_.info_log, "DeleteFile %s failed -- %s.\n", - name.c_str(), status.ToString().c_str()); + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "DeleteFile %s failed -- %s.\n", name.c_str(), + status.ToString().c_str()); } return status; } @@ -4652,59 +1964,168 @@ Status DBImpl::DeleteFile(std::string name) { FileMetaData* metadata; ColumnFamilyData* cfd; VersionEdit edit; - DeletionState deletion_state(true); + JobContext job_context(next_job_id_.fetch_add(1), true); { - MutexLock l(&mutex_); + InstrumentedMutexLock l(&mutex_); status = versions_->GetMetadataForFile(number, &level, &metadata, &cfd); if (!status.ok()) { - Log(options_.info_log, "DeleteFile %s failed. File not found\n", - name.c_str()); + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "DeleteFile %s failed. File not found\n", name.c_str()); + job_context.Clean(); return Status::InvalidArgument("File not found"); } - assert((level > 0) && (level < cfd->NumberLevels())); + assert(level < cfd->NumberLevels()); // If the file is being compacted no need to delete. if (metadata->being_compacted) { - Log(options_.info_log, - "DeleteFile %s Skipped. File about to be compacted\n", name.c_str()); + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "DeleteFile %s Skipped. File about to be compacted\n", + name.c_str()); + job_context.Clean(); return Status::OK(); } // Only the files in the last level can be deleted externally. // This is to make sure that any deletion tombstones are not // lost. Check that the level passed is the last level. + auto* vstoreage = cfd->current()->storage_info(); for (int i = level + 1; i < cfd->NumberLevels(); i++) { - if (cfd->current()->NumLevelFiles(i) != 0) { - Log(options_.info_log, - "DeleteFile %s FAILED. File not in last level\n", name.c_str()); + if (vstoreage->NumLevelFiles(i) != 0) { + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "DeleteFile %s FAILED. File not in last level\n", + name.c_str()); + job_context.Clean(); return Status::InvalidArgument("File not in last level"); } } + // if level == 0, it has to be the oldest file + if (level == 0 && + vstoreage->LevelFiles(0).back()->fd.GetNumber() != number) { + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "DeleteFile %s failed ---" + " target file in level 0 must be the oldest.", + name.c_str()); + job_context.Clean(); + return Status::InvalidArgument("File in level 0, but not oldest"); + } + edit.SetColumnFamily(cfd->GetID()); edit.DeleteFile(level, number); - status = versions_->LogAndApply(cfd, &edit, &mutex_, db_directory_.get()); + status = versions_->LogAndApply(cfd, *cfd->GetLatestMutableCFOptions(), + &edit, &mutex_, directories_.GetDbDir()); if (status.ok()) { - InstallSuperVersion(cfd, deletion_state); + InstallSuperVersionAndScheduleWorkWrapper( + cfd, &job_context, *cfd->GetLatestMutableCFOptions()); } - FindObsoleteFiles(deletion_state, false); - } // lock released here - LogFlush(options_.info_log); + FindObsoleteFiles(&job_context, false); + } // lock released here + + LogFlush(immutable_db_options_.info_log); // remove files outside the db-lock - if (deletion_state.HaveSomethingToDelete()) { - PurgeObsoleteFiles(deletion_state); + if (job_context.HaveSomethingToDelete()) { + // Call PurgeObsoleteFiles() without holding mutex. + PurgeObsoleteFiles(job_context); } + job_context.Clean(); + return status; +} + +Status DBImpl::DeleteFilesInRange(ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) { + Status status; + auto cfh = reinterpret_cast(column_family); + ColumnFamilyData* cfd = cfh->cfd(); + VersionEdit edit; + std::vector deleted_files; + JobContext job_context(next_job_id_.fetch_add(1), true); { - MutexLock l(&mutex_); - // schedule flush if file deletion means we freed the space for flushes to - // continue - MaybeScheduleFlushOrCompaction(); + InstrumentedMutexLock l(&mutex_); + Version* input_version = cfd->current(); + + auto* vstorage = input_version->storage_info(); + for (int i = 1; i < cfd->NumberLevels(); i++) { + if (vstorage->LevelFiles(i).empty() || + !vstorage->OverlapInLevel(i, begin, end)) { + continue; + } + std::vector level_files; + InternalKey begin_storage, end_storage, *begin_key, *end_key; + if (begin == nullptr) { + begin_key = nullptr; + } else { + begin_storage.SetMaxPossibleForUserKey(*begin); + begin_key = &begin_storage; + } + if (end == nullptr) { + end_key = nullptr; + } else { + end_storage.SetMinPossibleForUserKey(*end); + end_key = &end_storage; + } + + vstorage->GetOverlappingInputs(i, begin_key, end_key, &level_files, -1, + nullptr, false); + FileMetaData* level_file; + for (uint32_t j = 0; j < level_files.size(); j++) { + level_file = level_files[j]; + if (((begin == nullptr) || + (cfd->internal_comparator().user_comparator()->Compare( + level_file->smallest.user_key(), *begin) >= 0)) && + ((end == nullptr) || + (cfd->internal_comparator().user_comparator()->Compare( + level_file->largest.user_key(), *end) <= 0))) { + if (level_file->being_compacted) { + continue; + } + edit.SetColumnFamily(cfd->GetID()); + edit.DeleteFile(i, level_file->fd.GetNumber()); + deleted_files.push_back(level_file); + level_file->being_compacted = true; + } + } + } + if (edit.GetDeletedFiles().empty()) { + job_context.Clean(); + return Status::OK(); + } + input_version->Ref(); + status = versions_->LogAndApply(cfd, *cfd->GetLatestMutableCFOptions(), + &edit, &mutex_, directories_.GetDbDir()); + if (status.ok()) { + InstallSuperVersionAndScheduleWorkWrapper( + cfd, &job_context, *cfd->GetLatestMutableCFOptions()); + } + for (auto* deleted_file : deleted_files) { + deleted_file->being_compacted = false; + } + input_version->Unref(); + FindObsoleteFiles(&job_context, false); + } // lock released here + + LogFlush(immutable_db_options_.info_log); + // remove files outside the db-lock + if (job_context.HaveSomethingToDelete()) { + // Call PurgeObsoleteFiles() without holding mutex. + PurgeObsoleteFiles(job_context); } + job_context.Clean(); return status; } void DBImpl::GetLiveFilesMetaData(std::vector* metadata) { - MutexLock l(&mutex_); + InstrumentedMutexLock l(&mutex_); versions_->GetLiveFilesMetaData(metadata); } + +void DBImpl::GetColumnFamilyMetaData( + ColumnFamilyHandle* column_family, + ColumnFamilyMetaData* cf_meta) { + assert(column_family); + auto* cfd = reinterpret_cast(column_family)->cfd(); + auto* sv = GetAndRefSuperVersion(cfd); + sv->current->GetColumnFamilyMetaData(cf_meta); + ReturnAndCleanupSuperVersion(cfd, sv); +} + #endif // ROCKSDB_LITE Status DBImpl::CheckConsistency() { @@ -4714,18 +2135,23 @@ Status DBImpl::CheckConsistency() { std::string corruption_messages; for (const auto& md : metadata) { - std::string file_path = md.db_path + "/" + md.name; + // md.name has a leading "/". + std::string file_path = md.db_path + md.name; uint64_t fsize = 0; Status s = env_->GetFileSize(file_path, &fsize); + if (!s.ok() && + env_->GetFileSize(Rocks2LevelTableFileName(file_path), &fsize).ok()) { + s = Status::OK(); + } if (!s.ok()) { corruption_messages += "Can't access " + md.name + ": " + s.ToString() + "\n"; } else if (fsize != md.size) { corruption_messages += "Sst file size mismatch: " + file_path + ". Size recorded in manifest " + - std::to_string(md.size) + ", actual size " + - std::to_string(fsize) + "\n"; + ToString(md.size) + ", actual size " + + ToString(fsize) + "\n"; } } if (corruption_messages.size() == 0) { @@ -4735,22 +2161,28 @@ Status DBImpl::CheckConsistency() { } } -Status DBImpl::GetDbIdentity(std::string& identity) { +Status DBImpl::GetDbIdentity(std::string& identity) const { std::string idfilename = IdentityFileName(dbname_); - unique_ptr idfile; const EnvOptions soptions; - Status s = env_->NewSequentialFile(idfilename, &idfile, soptions); - if (!s.ok()) { - return s; + unique_ptr id_file_reader; + Status s; + { + unique_ptr idfile; + s = env_->NewSequentialFile(idfilename, &idfile, soptions); + if (!s.ok()) { + return s; + } + id_file_reader.reset(new SequentialFileReader(std::move(idfile))); } + uint64_t file_size; s = env_->GetFileSize(idfilename, &file_size); if (!s.ok()) { return s; } - char buffer[file_size]; + char* buffer = reinterpret_cast(alloca(file_size)); Slice id; - s = idfile->Read(file_size, &id, buffer); + s = id_file_reader->Read(static_cast(file_size), &id, buffer); if (!s.ok()) { return s; } @@ -4762,208 +2194,42 @@ Status DBImpl::GetDbIdentity(std::string& identity) { return s; } -// Default implementations of convenience methods that subclasses of DB -// can call if they wish -Status DB::Put(const WriteOptions& opt, ColumnFamilyHandle* column_family, - const Slice& key, const Slice& value) { - // Pre-allocate size of write batch conservatively. - // 8 bytes are taken by header, 4 bytes for count, 1 byte for type, - // and we allocate 11 extra bytes for key length, as well as value length. - WriteBatch batch(key.size() + value.size() + 24); - batch.Put(column_family, key, value); - return Write(opt, &batch); -} - -Status DB::Delete(const WriteOptions& opt, ColumnFamilyHandle* column_family, - const Slice& key) { - WriteBatch batch; - batch.Delete(column_family, key); - return Write(opt, &batch); -} - -Status DB::Merge(const WriteOptions& opt, ColumnFamilyHandle* column_family, - const Slice& key, const Slice& value) { - WriteBatch batch; - batch.Merge(column_family, key, value); - return Write(opt, &batch); -} - // Default implementation -- returns not supported status -Status DB::CreateColumnFamily(const ColumnFamilyOptions& options, +Status DB::CreateColumnFamily(const ColumnFamilyOptions& cf_options, const std::string& column_family_name, ColumnFamilyHandle** handle) { return Status::NotSupported(""); } -Status DB::DropColumnFamily(ColumnFamilyHandle* column_family) { - return Status::NotSupported(""); -} - -DB::~DB() { } - -Status DB::Open(const Options& options, const std::string& dbname, DB** dbptr) { - DBOptions db_options(options); - ColumnFamilyOptions cf_options(options); - std::vector column_families; - column_families.push_back( - ColumnFamilyDescriptor(kDefaultColumnFamilyName, cf_options)); - std::vector handles; - Status s = DB::Open(db_options, dbname, column_families, &handles, dbptr); - if (s.ok()) { - assert(handles.size() == 1); - // i can delete the handle since DBImpl is always holding a reference to - // default column family - delete handles[0]; - } - return s; -} - -Status DB::Open(const DBOptions& db_options, const std::string& dbname, - const std::vector& column_families, - std::vector* handles, DB** dbptr) { - Status s = SanitizeDBOptionsByCFOptions(&db_options, column_families); - if (!s.ok()) { - return s; - } - if (db_options.db_paths.size() > 1) { - for (auto& cfd : column_families) { - if (cfd.options.compaction_style != kCompactionStyleUniversal) { - return Status::NotSupported( - "More than one DB paths are only supported in " - "universal compaction style. "); - } - } - - if (db_options.db_paths.size() > 4) { - return Status::NotSupported( - "More than four DB paths are not supported yet. "); - } - } - - *dbptr = nullptr; - handles->clear(); - - size_t max_write_buffer_size = 0; - for (auto cf : column_families) { - max_write_buffer_size = - std::max(max_write_buffer_size, cf.options.write_buffer_size); - } - DBImpl* impl = new DBImpl(db_options, dbname); - s = impl->env_->CreateDirIfMissing(impl->options_.wal_dir); - if (s.ok()) { - for (auto db_path : impl->options_.db_paths) { - s = impl->env_->CreateDirIfMissing(db_path.path); - if (!s.ok()) { - break; - } - } - } - - if (!s.ok()) { - delete impl; - return s; - } - - s = impl->CreateArchivalDirectory(); - if (!s.ok()) { - delete impl; - return s; - } - impl->mutex_.Lock(); - // Handles create_if_missing, error_if_exists - s = impl->Recover(column_families); - if (s.ok()) { - uint64_t new_log_number = impl->versions_->NewFileNumber(); - unique_ptr lfile; - EnvOptions soptions(db_options); - s = impl->options_.env->NewWritableFile( - LogFileName(impl->options_.wal_dir, new_log_number), &lfile, - impl->options_.env->OptimizeForLogWrite(soptions)); - if (s.ok()) { - lfile->SetPreallocationBlockSize(1.1 * max_write_buffer_size); - impl->logfile_number_ = new_log_number; - impl->log_.reset(new log::Writer(std::move(lfile))); - - // set column family handles - for (auto cf : column_families) { - auto cfd = - impl->versions_->GetColumnFamilySet()->GetColumnFamily(cf.name); - if (cfd != nullptr) { - handles->push_back( - new ColumnFamilyHandleImpl(cfd, impl, &impl->mutex_)); - } else { - if (db_options.create_missing_column_families) { - // missing column family, create it - ColumnFamilyHandle* handle; - impl->mutex_.Unlock(); - s = impl->CreateColumnFamily(cf.options, cf.name, &handle); - impl->mutex_.Lock(); - if (s.ok()) { - handles->push_back(handle); - } else { - break; - } - } else { - s = Status::InvalidArgument("Column family not found: ", cf.name); - break; - } - } - } - } - if (s.ok()) { - for (auto cfd : *impl->versions_->GetColumnFamilySet()) { - delete cfd->InstallSuperVersion(new SuperVersion(), &impl->mutex_); - } - impl->alive_log_files_.push_back( - DBImpl::LogFileNumberSize(impl->logfile_number_)); - impl->DeleteObsoleteFiles(); - impl->MaybeScheduleFlushOrCompaction(); - s = impl->db_directory_->Fsync(); - } - } - - if (s.ok()) { - for (auto cfd : *impl->versions_->GetColumnFamilySet()) { - if (cfd->options()->compaction_style == kCompactionStyleUniversal || - cfd->options()->compaction_style == kCompactionStyleFIFO) { - Version* current = cfd->current(); - for (int i = 1; i < current->NumberLevels(); ++i) { - int num_files = current->NumLevelFiles(i); - if (num_files > 0) { - s = Status::InvalidArgument( - "Not all files are at level 0. Cannot " - "open with universal or FIFO compaction style."); - break; - } - } - } - if (cfd->options()->merge_operator != nullptr && - !cfd->mem()->IsMergeOperatorSupported()) { - s = Status::InvalidArgument( - "The memtable of column family %s does not support merge operator " - "its options.merge_operator is non-null", cfd->GetName().c_str()); - } - if (!s.ok()) { - break; - } - } - } +Status DB::CreateColumnFamilies( + const ColumnFamilyOptions& cf_options, + const std::vector& column_family_names, + std::vector* handles) { + return Status::NotSupported(""); +} - impl->mutex_.Unlock(); +Status DB::CreateColumnFamilies( + const std::vector& column_families, + std::vector* handles) { + return Status::NotSupported(""); +} - if (s.ok()) { - impl->opened_successfully_ = true; - *dbptr = impl; - } else { - for (auto h : *handles) { - delete h; - } - handles->clear(); - delete impl; - } - return s; +Status DB::DropColumnFamily(ColumnFamilyHandle* column_family) { + return Status::NotSupported(""); +} + +Status DB::DropColumnFamilies( + const std::vector& column_families) { + return Status::NotSupported(""); +} + +Status DB::DestroyColumnFamilyHandle(ColumnFamilyHandle* column_family) { + delete column_family; + return Status::OK(); } +DB::~DB() { } + Status DB::ListColumnFamilies(const DBOptions& db_options, const std::string& name, std::vector* column_families) { @@ -4974,44 +2240,31 @@ Snapshot::~Snapshot() { } Status DestroyDB(const std::string& dbname, const Options& options) { - const InternalKeyComparator comparator(options.comparator); - const Options& soptions(SanitizeOptions(dbname, &comparator, options)); + const ImmutableDBOptions soptions(SanitizeOptions(dbname, options)); Env* env = soptions.env; std::vector filenames; - std::vector archiveFiles; - std::string archivedir = ArchivalDirectory(dbname); // Ignore error in case directory does not exist env->GetChildren(dbname, &filenames); - if (dbname != soptions.wal_dir) { - std::vector logfilenames; - env->GetChildren(soptions.wal_dir, &logfilenames); - filenames.insert(filenames.end(), logfilenames.begin(), logfilenames.end()); - archivedir = ArchivalDirectory(soptions.wal_dir); - } - - if (filenames.empty()) { - return Status::OK(); - } - FileLock* lock; const std::string lockname = LockFileName(dbname); Status result = env->LockFile(lockname, &lock); if (result.ok()) { uint64_t number; FileType type; - InfoLogPrefix info_log_prefix(!options.db_log_dir.empty(), dbname); + InfoLogPrefix info_log_prefix(!soptions.db_log_dir.empty(), dbname); for (size_t i = 0; i < filenames.size(); i++) { if (ParseFileName(filenames[i], &number, info_log_prefix.prefix, &type) && type != kDBLockFile) { // Lock file will be deleted at end Status del; + std::string path_to_delete = dbname + "/" + filenames[i]; if (type == kMetaDatabase) { - del = DestroyDB(dbname + "/" + filenames[i], options); - } else if (type == kLogFile) { - del = env->DeleteFile(soptions.wal_dir + "/" + filenames[i]); + del = DestroyDB(path_to_delete, options); + } else if (type == kTableFile) { + del = DeleteSSTFile(&soptions, path_to_delete, 0); } else { - del = env->DeleteFile(dbname + "/" + filenames[i]); + del = env->DeleteFile(path_to_delete); } if (result.ok() && !del.ok()) { result = del; @@ -5019,14 +2272,15 @@ Status DestroyDB(const std::string& dbname, const Options& options) { } } - for (auto& db_path : options.db_paths) { + for (size_t path_id = 0; path_id < options.db_paths.size(); path_id++) { + const auto& db_path = options.db_paths[path_id]; env->GetChildren(db_path.path, &filenames); - uint64_t number; - FileType type; for (size_t i = 0; i < filenames.size(); i++) { if (ParseFileName(filenames[i], &number, &type) && type == kTableFile) { // Lock file will be deleted at end - Status del = env->DeleteFile(db_path.path + "/" + filenames[i]); + std::string table_path = db_path.path + "/" + filenames[i]; + Status del = DeleteSSTFile(&soptions, table_path, + static_cast(path_id)); if (result.ok() && !del.ok()) { result = del; } @@ -5034,6 +2288,24 @@ Status DestroyDB(const std::string& dbname, const Options& options) { } } + std::vector walDirFiles; + std::string archivedir = ArchivalDirectory(dbname); + if (dbname != soptions.wal_dir) { + env->GetChildren(soptions.wal_dir, &walDirFiles); + archivedir = ArchivalDirectory(soptions.wal_dir); + } + + // Delete log files in the WAL dir + for (const auto& file : walDirFiles) { + if (ParseFileName(file, &number, &type) && type == kLogFile) { + Status del = env->DeleteFile(LogFileName(soptions.wal_dir, number)); + if (result.ok() && !del.ok()) { + result = del; + } + } + } + + std::vector archiveFiles; env->GetChildren(archivedir, &archiveFiles); // Delete archival files. for (size_t i = 0; i < archiveFiles.size(); ++i) { @@ -5045,7 +2317,8 @@ Status DestroyDB(const std::string& dbname, const Options& options) { } } } - // ignore case where no archival directory is present. + + // ignore case where no archival directory is present env->DeleteDir(archivedir); env->UnlockFile(lock); // Ignore error since state is already gone @@ -5056,15 +2329,495 @@ Status DestroyDB(const std::string& dbname, const Options& options) { return result; } +Status DBImpl::WriteOptionsFile(bool need_mutex_lock, + bool need_enter_write_thread) { +#ifndef ROCKSDB_LITE + WriteThread::Writer w; + if (need_mutex_lock) { + mutex_.Lock(); + } else { + mutex_.AssertHeld(); + } + if (need_enter_write_thread) { + write_thread_.EnterUnbatched(&w, &mutex_); + } + + std::vector cf_names; + std::vector cf_opts; + + // This part requires mutex to protect the column family options + for (auto cfd : *versions_->GetColumnFamilySet()) { + if (cfd->IsDropped()) { + continue; + } + cf_names.push_back(cfd->GetName()); + cf_opts.push_back(cfd->GetLatestCFOptions()); + } + + // Unlock during expensive operations. New writes cannot get here + // because the single write thread ensures all new writes get queued. + DBOptions db_options = + BuildDBOptions(immutable_db_options_, mutable_db_options_); + mutex_.Unlock(); + + TEST_SYNC_POINT("DBImpl::WriteOptionsFile:1"); + TEST_SYNC_POINT("DBImpl::WriteOptionsFile:2"); + + std::string file_name = + TempOptionsFileName(GetName(), versions_->NewFileNumber()); + Status s = + PersistRocksDBOptions(db_options, cf_names, cf_opts, file_name, GetEnv()); + + if (s.ok()) { + s = RenameTempFileToOptionsFile(file_name); + } + // restore lock + if (!need_mutex_lock) { + mutex_.Lock(); + } + if (need_enter_write_thread) { + write_thread_.ExitUnbatched(&w); + } + if (!s.ok()) { + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "Unnable to persist options -- %s", s.ToString().c_str()); + if (immutable_db_options_.fail_if_options_file_error) { + return Status::IOError("Unable to persist options.", + s.ToString().c_str()); + } + } +#endif // !ROCKSDB_LITE + return Status::OK(); +} + +#ifndef ROCKSDB_LITE +namespace { +void DeleteOptionsFilesHelper(const std::map& filenames, + const size_t num_files_to_keep, + const std::shared_ptr& info_log, + Env* env) { + if (filenames.size() <= num_files_to_keep) { + return; + } + for (auto iter = std::next(filenames.begin(), num_files_to_keep); + iter != filenames.end(); ++iter) { + if (!env->DeleteFile(iter->second).ok()) { + ROCKS_LOG_WARN(info_log, "Unable to delete options file %s", + iter->second.c_str()); + } + } +} +} // namespace +#endif // !ROCKSDB_LITE + +Status DBImpl::DeleteObsoleteOptionsFiles() { +#ifndef ROCKSDB_LITE + std::vector filenames; + // use ordered map to store keep the filenames sorted from the newest + // to the oldest. + std::map options_filenames; + Status s; + s = GetEnv()->GetChildren(GetName(), &filenames); + if (!s.ok()) { + return s; + } + for (auto& filename : filenames) { + uint64_t file_number; + FileType type; + if (ParseFileName(filename, &file_number, &type) && type == kOptionsFile) { + options_filenames.insert( + {std::numeric_limits::max() - file_number, + GetName() + "/" + filename}); + } + } + + // Keeps the latest 2 Options file + const size_t kNumOptionsFilesKept = 2; + DeleteOptionsFilesHelper(options_filenames, kNumOptionsFilesKept, + immutable_db_options_.info_log, GetEnv()); + return Status::OK(); +#else + return Status::OK(); +#endif // !ROCKSDB_LITE +} + +Status DBImpl::RenameTempFileToOptionsFile(const std::string& file_name) { +#ifndef ROCKSDB_LITE + Status s; + + versions_->options_file_number_ = versions_->NewFileNumber(); + std::string options_file_name = + OptionsFileName(GetName(), versions_->options_file_number_); + // Retry if the file name happen to conflict with an existing one. + s = GetEnv()->RenameFile(file_name, options_file_name); + + DeleteObsoleteOptionsFiles(); + return s; +#else + return Status::OK(); +#endif // !ROCKSDB_LITE +} + +#ifdef ROCKSDB_USING_THREAD_STATUS + +void DBImpl::NewThreadStatusCfInfo( + ColumnFamilyData* cfd) const { + if (immutable_db_options_.enable_thread_tracking) { + ThreadStatusUtil::NewColumnFamilyInfo(this, cfd, cfd->GetName(), + cfd->ioptions()->env); + } +} + +void DBImpl::EraseThreadStatusCfInfo( + ColumnFamilyData* cfd) const { + if (immutable_db_options_.enable_thread_tracking) { + ThreadStatusUtil::EraseColumnFamilyInfo(cfd); + } +} + +void DBImpl::EraseThreadStatusDbInfo() const { + if (immutable_db_options_.enable_thread_tracking) { + ThreadStatusUtil::EraseDatabaseInfo(this); + } +} + +#else +void DBImpl::NewThreadStatusCfInfo( + ColumnFamilyData* cfd) const { +} + +void DBImpl::EraseThreadStatusCfInfo( + ColumnFamilyData* cfd) const { +} + +void DBImpl::EraseThreadStatusDbInfo() const { +} +#endif // ROCKSDB_USING_THREAD_STATUS + // // A global method that can dump out the build version -void DumpLeveldbBuildVersion(Logger * log) { +void DumpRocksDBBuildVersion(Logger * log) { #if !defined(IOS_CROSS_COMPILE) - // if we compile with Xcode, we don't run build_detect_vesion, so we don't generate util/build_version.cc - Log(log, "Git sha %s", rocksdb_build_git_sha); - Log(log, "Compile time %s %s", - rocksdb_build_compile_time, rocksdb_build_compile_date); + // if we compile with Xcode, we don't run build_detect_version, so we don't + // generate util/build_version.cc + ROCKS_LOG_HEADER(log, "RocksDB version: %d.%d.%d\n", ROCKSDB_MAJOR, + ROCKSDB_MINOR, ROCKSDB_PATCH); + ROCKS_LOG_HEADER(log, "Git sha %s", rocksdb_build_git_sha); + ROCKS_LOG_HEADER(log, "Compile date %s", rocksdb_build_compile_date); +#endif +} + +#ifndef ROCKSDB_LITE +SequenceNumber DBImpl::GetEarliestMemTableSequenceNumber(SuperVersion* sv, + bool include_history) { + // Find the earliest sequence number that we know we can rely on reading + // from the memtable without needing to check sst files. + SequenceNumber earliest_seq = + sv->imm->GetEarliestSequenceNumber(include_history); + if (earliest_seq == kMaxSequenceNumber) { + earliest_seq = sv->mem->GetEarliestSequenceNumber(); + } + assert(sv->mem->GetEarliestSequenceNumber() >= earliest_seq); + + return earliest_seq; +} +#endif // ROCKSDB_LITE + +#ifndef ROCKSDB_LITE +Status DBImpl::GetLatestSequenceForKey(SuperVersion* sv, const Slice& key, + bool cache_only, SequenceNumber* seq, + bool* found_record_for_key, + bool* is_blob_index) { + Status s; + MergeContext merge_context; + RangeDelAggregator range_del_agg(sv->mem->GetInternalKeyComparator(), + kMaxSequenceNumber); + + ReadOptions read_options; + SequenceNumber current_seq = versions_->LastSequence(); + LookupKey lkey(key, current_seq); + + *seq = kMaxSequenceNumber; + *found_record_for_key = false; + + // Check if there is a record for this key in the latest memtable + sv->mem->Get(lkey, nullptr, &s, &merge_context, &range_del_agg, seq, + read_options, is_blob_index); + + if (!(s.ok() || s.IsNotFound() || s.IsMergeInProgress())) { + // unexpected error reading memtable. + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "Unexpected status returned from MemTable::Get: %s\n", + s.ToString().c_str()); + + return s; + } + + if (*seq != kMaxSequenceNumber) { + // Found a sequence number, no need to check immutable memtables + *found_record_for_key = true; + return Status::OK(); + } + + // Check if there is a record for this key in the immutable memtables + sv->imm->Get(lkey, nullptr, &s, &merge_context, &range_del_agg, seq, + read_options, is_blob_index); + + if (!(s.ok() || s.IsNotFound() || s.IsMergeInProgress())) { + // unexpected error reading memtable. + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "Unexpected status returned from MemTableList::Get: %s\n", + s.ToString().c_str()); + + return s; + } + + if (*seq != kMaxSequenceNumber) { + // Found a sequence number, no need to check memtable history + *found_record_for_key = true; + return Status::OK(); + } + + // Check if there is a record for this key in the immutable memtables + sv->imm->GetFromHistory(lkey, nullptr, &s, &merge_context, &range_del_agg, + seq, read_options, is_blob_index); + + if (!(s.ok() || s.IsNotFound() || s.IsMergeInProgress())) { + // unexpected error reading memtable. + ROCKS_LOG_ERROR( + immutable_db_options_.info_log, + "Unexpected status returned from MemTableList::GetFromHistory: %s\n", + s.ToString().c_str()); + + return s; + } + + if (*seq != kMaxSequenceNumber) { + // Found a sequence number, no need to check SST files + *found_record_for_key = true; + return Status::OK(); + } + + // TODO(agiardullo): possible optimization: consider checking cached + // SST files if cache_only=true? + if (!cache_only) { + // Check tables + sv->current->Get(read_options, lkey, nullptr, &s, &merge_context, + &range_del_agg, nullptr /* value_found */, + found_record_for_key, seq, is_blob_index); + + if (!(s.ok() || s.IsNotFound() || s.IsMergeInProgress())) { + // unexpected error reading SST files + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "Unexpected status returned from Version::Get: %s\n", + s.ToString().c_str()); + + return s; + } + } + + return Status::OK(); +} + +Status DBImpl::IngestExternalFile( + ColumnFamilyHandle* column_family, + const std::vector& external_files, + const IngestExternalFileOptions& ingestion_options) { + Status status; + auto cfh = reinterpret_cast(column_family); + auto cfd = cfh->cfd(); + + // Ingest should immediately fail if ingest_behind is requested, + // but the DB doesn't support it. + if (ingestion_options.ingest_behind) { + if (!immutable_db_options_.allow_ingest_behind) { + return Status::InvalidArgument( + "Can't ingest_behind file in DB with allow_ingest_behind=false"); + } + } + + ExternalSstFileIngestionJob ingestion_job(env_, versions_.get(), cfd, + immutable_db_options_, env_options_, + &snapshots_, ingestion_options); + + std::list::iterator pending_output_elem; + { + InstrumentedMutexLock l(&mutex_); + if (!bg_error_.ok()) { + // Don't ingest files when there is a bg_error + return bg_error_; + } + + // Make sure that bg cleanup wont delete the files that we are ingesting + pending_output_elem = CaptureCurrentFileNumberInPendingOutputs(); + } + + status = ingestion_job.Prepare(external_files); + if (!status.ok()) { + return status; + } + + TEST_SYNC_POINT("DBImpl::AddFile:Start"); + { + // Lock db mutex + InstrumentedMutexLock l(&mutex_); + TEST_SYNC_POINT("DBImpl::AddFile:MutexLock"); + + // Stop writes to the DB by entering both write threads + WriteThread::Writer w; + write_thread_.EnterUnbatched(&w, &mutex_); + WriteThread::Writer nonmem_w; + if (concurrent_prepare_) { + nonmem_write_thread_.EnterUnbatched(&nonmem_w, &mutex_); + } + + num_running_ingest_file_++; + + // We cannot ingest a file into a dropped CF + if (cfd->IsDropped()) { + status = Status::InvalidArgument( + "Cannot ingest an external file into a dropped CF"); + } + + // Figure out if we need to flush the memtable first + if (status.ok()) { + bool need_flush = false; + status = ingestion_job.NeedsFlush(&need_flush); + TEST_SYNC_POINT_CALLBACK("DBImpl::IngestExternalFile:NeedFlush", + &need_flush); + if (status.ok() && need_flush) { + mutex_.Unlock(); + status = FlushMemTable(cfd, FlushOptions(), true /* writes_stopped */); + mutex_.Lock(); + } + } + + // Run the ingestion job + if (status.ok()) { + status = ingestion_job.Run(); + } + + // Install job edit [Mutex will be unlocked here] + auto mutable_cf_options = cfd->GetLatestMutableCFOptions(); + if (status.ok()) { + status = + versions_->LogAndApply(cfd, *mutable_cf_options, ingestion_job.edit(), + &mutex_, directories_.GetDbDir()); + } + if (status.ok()) { + delete InstallSuperVersionAndScheduleWork(cfd, nullptr, + *mutable_cf_options); + } + + // Resume writes to the DB + if (concurrent_prepare_) { + nonmem_write_thread_.ExitUnbatched(&nonmem_w); + } + write_thread_.ExitUnbatched(&w); + + // Update stats + if (status.ok()) { + ingestion_job.UpdateStats(); + } + + ReleaseFileNumberFromPendingOutputs(pending_output_elem); + + num_running_ingest_file_--; + if (num_running_ingest_file_ == 0) { + bg_cv_.SignalAll(); + } + + TEST_SYNC_POINT("DBImpl::AddFile:MutexUnlock"); + } + // mutex_ is unlocked here + + // Cleanup + ingestion_job.Cleanup(status); + + if (status.ok()) { + NotifyOnExternalFileIngested(cfd, ingestion_job); + } + + return status; +} + +Status DBImpl::VerifyChecksum() { + Status s; + Options options; + EnvOptions env_options; + std::vector cfd_list; + { + InstrumentedMutexLock l(&mutex_); + for (auto cfd : *versions_->GetColumnFamilySet()) { + if (!cfd->IsDropped() && cfd->initialized()) { + cfd->Ref(); + cfd_list.push_back(cfd); + } + } + } + std::vector sv_list; + for (auto cfd : cfd_list) { + sv_list.push_back(cfd->GetReferencedSuperVersion(&mutex_)); + } + for (auto& sv : sv_list) { + VersionStorageInfo* vstorage = sv->current->storage_info(); + for (int i = 0; i < vstorage->num_non_empty_levels() && s.ok(); i++) { + for (size_t j = 0; j < vstorage->LevelFilesBrief(i).num_files && s.ok(); + j++) { + const auto& fd = vstorage->LevelFilesBrief(i).files[j].fd; + std::string fname = TableFileName(immutable_db_options_.db_paths, + fd.GetNumber(), fd.GetPathId()); + s = rocksdb::VerifySstFileChecksum(options, env_options, fname); + } + } + if (!s.ok()) { + break; + } + } + { + InstrumentedMutexLock l(&mutex_); + for (auto sv : sv_list) { + if (sv && sv->Unref()) { + sv->Cleanup(); + delete sv; + } + } + for (auto cfd : cfd_list) { + cfd->Unref(); + } + } + return s; +} + +void DBImpl::NotifyOnExternalFileIngested( + ColumnFamilyData* cfd, const ExternalSstFileIngestionJob& ingestion_job) { +#ifndef ROCKSDB_LITE + if (immutable_db_options_.listeners.empty()) { + return; + } + + for (const IngestedFileInfo& f : ingestion_job.files_to_ingest()) { + ExternalFileIngestionInfo info; + info.cf_name = cfd->GetName(); + info.external_file_path = f.external_file_path; + info.internal_file_path = f.internal_file_path; + info.global_seqno = f.assigned_seqno; + info.table_properties = f.table_properties; + for (auto listener : immutable_db_options_.listeners) { + listener->OnExternalFileIngested(this, info); + } + } + #endif } +void DBImpl::WaitForIngestFile() { + mutex_.AssertHeld(); + while (num_running_ingest_file_ > 0) { + bg_cv_.Wait(); + } +} + +#endif // ROCKSDB_LITE + } // namespace rocksdb diff --git a/db/db_impl.h b/db/db_impl.h index 086ac9fd4e4..f1730f9adbd 100644 --- a/db/db_impl.h +++ b/db/db_impl.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,37 +10,59 @@ #include #include +#include #include +#include +#include +#include #include +#include #include #include -#include +#include "db/column_family.h" +#include "db/compaction_job.h" #include "db/dbformat.h" +#include "db/external_sst_file_ingestion_job.h" +#include "db/flush_job.h" +#include "db/flush_scheduler.h" +#include "db/internal_stats.h" #include "db/log_writer.h" -#include "db/snapshot.h" -#include "db/column_family.h" +#include "db/snapshot_impl.h" #include "db/version_edit.h" +#include "db/wal_manager.h" +#include "db/write_controller.h" +#include "db/write_thread.h" #include "memtable_list.h" +#include "monitoring/instrumented_mutex.h" +#include "options/db_options.h" #include "port/port.h" #include "rocksdb/db.h" #include "rocksdb/env.h" #include "rocksdb/memtablerep.h" +#include "rocksdb/status.h" #include "rocksdb/transaction_log.h" +#include "rocksdb/write_buffer_manager.h" +#include "table/scoped_arena_iterator.h" #include "util/autovector.h" +#include "util/event_logger.h" +#include "util/hash.h" #include "util/stop_watch.h" #include "util/thread_local.h" -#include "db/internal_stats.h" namespace rocksdb { +class ArenaWrappedDBIter; class MemTable; class TableCache; class Version; class VersionEdit; class VersionSet; -class CompactionFilterV2; class Arena; +class WriteCallback; +struct JobContext; +struct ExternalSstFileInfo; +struct MemTableInfo; class DBImpl : public DB { public: @@ -51,30 +73,54 @@ class DBImpl : public DB { using DB::Put; virtual Status Put(const WriteOptions& options, ColumnFamilyHandle* column_family, const Slice& key, - const Slice& value); + const Slice& value) override; using DB::Merge; virtual Status Merge(const WriteOptions& options, ColumnFamilyHandle* column_family, const Slice& key, - const Slice& value); + const Slice& value) override; using DB::Delete; virtual Status Delete(const WriteOptions& options, - ColumnFamilyHandle* column_family, const Slice& key); + ColumnFamilyHandle* column_family, + const Slice& key) override; + using DB::SingleDelete; + virtual Status SingleDelete(const WriteOptions& options, + ColumnFamilyHandle* column_family, + const Slice& key) override; using DB::Write; - virtual Status Write(const WriteOptions& options, WriteBatch* updates); + virtual Status Write(const WriteOptions& options, + WriteBatch* updates) override; + using DB::Get; virtual Status Get(const ReadOptions& options, ColumnFamilyHandle* column_family, const Slice& key, - std::string* value); + PinnableSlice* value) override; + + // Function that Get and KeyMayExist call with no_io true or false + // Note: 'value_found' from KeyMayExist propagates here + Status GetImpl(const ReadOptions& options, ColumnFamilyHandle* column_family, + const Slice& key, PinnableSlice* value, + bool* value_found = nullptr, bool* is_blob_index = nullptr); + using DB::MultiGet; virtual std::vector MultiGet( const ReadOptions& options, const std::vector& column_family, - const std::vector& keys, std::vector* values); + const std::vector& keys, + std::vector* values) override; - virtual Status CreateColumnFamily(const ColumnFamilyOptions& options, + virtual Status CreateColumnFamily(const ColumnFamilyOptions& cf_options, const std::string& column_family, - ColumnFamilyHandle** handle); - virtual Status DropColumnFamily(ColumnFamilyHandle* column_family); + ColumnFamilyHandle** handle) override; + virtual Status CreateColumnFamilies( + const ColumnFamilyOptions& cf_options, + const std::vector& column_family_names, + std::vector* handles) override; + virtual Status CreateColumnFamilies( + const std::vector& column_families, + std::vector* handles) override; + virtual Status DropColumnFamily(ColumnFamilyHandle* column_family) override; + virtual Status DropColumnFamilies( + const std::vector& column_families) override; // Returns false if key doesn't exist in the database and true if it may. // If value_found is not passed in as null, then return the value if found in @@ -83,86 +129,240 @@ class DBImpl : public DB { using DB::KeyMayExist; virtual bool KeyMayExist(const ReadOptions& options, ColumnFamilyHandle* column_family, const Slice& key, - std::string* value, bool* value_found = nullptr); + std::string* value, + bool* value_found = nullptr) override; + using DB::NewIterator; virtual Iterator* NewIterator(const ReadOptions& options, - ColumnFamilyHandle* column_family); + ColumnFamilyHandle* column_family) override; virtual Status NewIterators( const ReadOptions& options, const std::vector& column_families, - std::vector* iterators); - virtual const Snapshot* GetSnapshot(); - virtual void ReleaseSnapshot(const Snapshot* snapshot); + std::vector* iterators) override; + ArenaWrappedDBIter* NewIteratorImpl(const ReadOptions& options, + ColumnFamilyData* cfd, + SequenceNumber snapshot, + bool allow_blob = false); + + virtual const Snapshot* GetSnapshot() override; + virtual void ReleaseSnapshot(const Snapshot* snapshot) override; using DB::GetProperty; virtual bool GetProperty(ColumnFamilyHandle* column_family, - const Slice& property, std::string* value); + const Slice& property, std::string* value) override; + using DB::GetMapProperty; + virtual bool GetMapProperty(ColumnFamilyHandle* column_family, + const Slice& property, + std::map* value) override; using DB::GetIntProperty; virtual bool GetIntProperty(ColumnFamilyHandle* column_family, const Slice& property, uint64_t* value) override; + using DB::GetAggregatedIntProperty; + virtual bool GetAggregatedIntProperty(const Slice& property, + uint64_t* aggregated_value) override; using DB::GetApproximateSizes; virtual void GetApproximateSizes(ColumnFamilyHandle* column_family, - const Range* range, int n, uint64_t* sizes); + const Range* range, int n, uint64_t* sizes, + uint8_t include_flags + = INCLUDE_FILES) override; + using DB::GetApproximateMemTableStats; + virtual void GetApproximateMemTableStats(ColumnFamilyHandle* column_family, + const Range& range, + uint64_t* const count, + uint64_t* const size) override; using DB::CompactRange; - virtual Status CompactRange(ColumnFamilyHandle* column_family, - const Slice* begin, const Slice* end, - bool reduce_level = false, int target_level = -1, - uint32_t target_path_id = 0); + virtual Status CompactRange(const CompactRangeOptions& options, + ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) override; + + using DB::CompactFiles; + virtual Status CompactFiles(const CompactionOptions& compact_options, + ColumnFamilyHandle* column_family, + const std::vector& input_file_names, + const int output_level, + const int output_path_id = -1) override; + + virtual Status PauseBackgroundWork() override; + virtual Status ContinueBackgroundWork() override; + + virtual Status EnableAutoCompaction( + const std::vector& column_family_handles) override; + + using DB::SetOptions; + Status SetOptions( + ColumnFamilyHandle* column_family, + const std::unordered_map& options_map) override; + + virtual Status SetDBOptions( + const std::unordered_map& options_map) override; using DB::NumberLevels; - virtual int NumberLevels(ColumnFamilyHandle* column_family); + virtual int NumberLevels(ColumnFamilyHandle* column_family) override; using DB::MaxMemCompactionLevel; - virtual int MaxMemCompactionLevel(ColumnFamilyHandle* column_family); + virtual int MaxMemCompactionLevel(ColumnFamilyHandle* column_family) override; using DB::Level0StopWriteTrigger; - virtual int Level0StopWriteTrigger(ColumnFamilyHandle* column_family); - virtual const std::string& GetName() const; - virtual Env* GetEnv() const; + virtual int Level0StopWriteTrigger( + ColumnFamilyHandle* column_family) override; + virtual const std::string& GetName() const override; + virtual Env* GetEnv() const override; using DB::GetOptions; - virtual const Options& GetOptions(ColumnFamilyHandle* column_family) const; + virtual Options GetOptions(ColumnFamilyHandle* column_family) const override; + using DB::GetDBOptions; + virtual DBOptions GetDBOptions() const override; using DB::Flush; virtual Status Flush(const FlushOptions& options, - ColumnFamilyHandle* column_family); + ColumnFamilyHandle* column_family) override; + virtual Status FlushWAL(bool sync) override; + virtual Status SyncWAL() override; - virtual SequenceNumber GetLatestSequenceNumber() const; + virtual SequenceNumber GetLatestSequenceNumber() const override; + + // Whether there is an active snapshot in range [lower_bound, upper_bound). + bool HasActiveSnapshotInRange(SequenceNumber lower_bound, + SequenceNumber upper_bound); #ifndef ROCKSDB_LITE - virtual Status DisableFileDeletions(); - virtual Status EnableFileDeletions(bool force); + using DB::ResetStats; + virtual Status ResetStats() override; + virtual Status DisableFileDeletions() override; + virtual Status EnableFileDeletions(bool force) override; virtual int IsFileDeletionsEnabled() const; // All the returned filenames start with "/" virtual Status GetLiveFiles(std::vector&, uint64_t* manifest_file_size, - bool flush_memtable = true); - virtual Status GetSortedWalFiles(VectorLogPtr& files); + bool flush_memtable = true) override; + virtual Status GetSortedWalFiles(VectorLogPtr& files) override; virtual Status GetUpdatesSince( SequenceNumber seq_number, unique_ptr* iter, const TransactionLogIterator::ReadOptions& - read_options = TransactionLogIterator::ReadOptions()); - virtual Status DeleteFile(std::string name); + read_options = TransactionLogIterator::ReadOptions()) override; + virtual Status DeleteFile(std::string name) override; + Status DeleteFilesInRange(ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end); + + virtual void GetLiveFilesMetaData( + std::vector* metadata) override; + + // Obtains the meta data of the specified column family of the DB. + // Status::NotFound() will be returned if the current DB does not have + // any column family match the specified name. + // TODO(yhchiang): output parameter is placed in the end in this codebase. + virtual void GetColumnFamilyMetaData( + ColumnFamilyHandle* column_family, + ColumnFamilyMetaData* metadata) override; + + Status SuggestCompactRange(ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) override; + + Status PromoteL0(ColumnFamilyHandle* column_family, + int target_level) override; + + // Similar to Write() but will call the callback once on the single write + // thread to determine whether it is safe to perform the write. + virtual Status WriteWithCallback(const WriteOptions& write_options, + WriteBatch* my_batch, + WriteCallback* callback); + + // Returns the sequence number that is guaranteed to be smaller than or equal + // to the sequence number of any key that could be inserted into the current + // memtables. It can then be assumed that any write with a larger(or equal) + // sequence number will be present in this memtable or a later memtable. + // + // If the earliest sequence number could not be determined, + // kMaxSequenceNumber will be returned. + // + // If include_history=true, will also search Memtables in MemTableList + // History. + SequenceNumber GetEarliestMemTableSequenceNumber(SuperVersion* sv, + bool include_history); + + // For a given key, check to see if there are any records for this key + // in the memtables, including memtable history. If cache_only is false, + // SST files will also be checked. + // + // If a key is found, *found_record_for_key will be set to true and + // *seq will be set to the stored sequence number for the latest + // operation on this key or kMaxSequenceNumber if unknown. + // If no key is found, *found_record_for_key will be set to false. + // + // Note: If cache_only=false, it is possible for *seq to be set to 0 if + // the sequence number has been cleared from the record. If the caller is + // holding an active db snapshot, we know the missing sequence must be less + // than the snapshot's sequence number (sequence numbers are only cleared + // when there are no earlier active snapshots). + // + // If NotFound is returned and found_record_for_key is set to false, then no + // record for this key was found. If the caller is holding an active db + // snapshot, we know that no key could have existing after this snapshot + // (since we do not compact keys that have an earlier snapshot). + // + // Returns OK or NotFound on success, + // other status on unexpected error. + // TODO(andrewkr): this API need to be aware of range deletion operations + Status GetLatestSequenceForKey(SuperVersion* sv, const Slice& key, + bool cache_only, SequenceNumber* seq, + bool* found_record_for_key, + bool* is_blob_index = nullptr); + + using DB::IngestExternalFile; + virtual Status IngestExternalFile( + ColumnFamilyHandle* column_family, + const std::vector& external_files, + const IngestExternalFileOptions& ingestion_options) override; + + virtual Status VerifyChecksum() override; - virtual void GetLiveFilesMetaData(std::vector* metadata); #endif // ROCKSDB_LITE + // Similar to GetSnapshot(), but also lets the db know that this snapshot + // will be used for transaction write-conflict checking. The DB can then + // make sure not to compact any keys that would prevent a write-conflict from + // being detected. + const Snapshot* GetSnapshotForWriteConflictBoundary(); + // checks if all live files exist on file system and that their file sizes // match to our in-memory records virtual Status CheckConsistency(); - virtual Status GetDbIdentity(std::string& identity); + virtual Status GetDbIdentity(std::string& identity) const override; Status RunManualCompaction(ColumnFamilyData* cfd, int input_level, int output_level, uint32_t output_path_id, - const Slice* begin, const Slice* end); + const Slice* begin, const Slice* end, + bool exclusive, + bool disallow_trivial_move = false); -#ifndef ROCKSDB_LITE + // Return an internal iterator over the current state of the database. + // The keys of this iterator are internal keys (see format.h). + // The returned iterator should be deleted when no longer needed. + InternalIterator* NewInternalIterator( + Arena* arena, RangeDelAggregator* range_del_agg, + ColumnFamilyHandle* column_family = nullptr); + +#ifndef NDEBUG // Extra methods (for testing) that are not in the public DB interface // Implemented in db_impl_debug.cc // Compact any files in the named level that overlap [*begin, *end] Status TEST_CompactRange(int level, const Slice* begin, const Slice* end, - ColumnFamilyHandle* column_family = nullptr); + ColumnFamilyHandle* column_family = nullptr, + bool disallow_trivial_move = false); + + void TEST_HandleWALFull(); + + bool TEST_UnableToFlushOldestLog() { + return unable_to_flush_oldest_log_; + } + + bool TEST_IsLogGettingFlushed() { + return alive_log_files_.begin()->getting_flushed; + } + + Status TEST_SwitchMemtable(ColumnFamilyData* cfd = nullptr); // Force current memtable contents to be flushed. - Status TEST_FlushMemTable(bool wait = true); + Status TEST_FlushMemTable(bool wait = true, + ColumnFamilyHandle* cfh = nullptr); // Wait for memtable compaction Status TEST_WaitForFlushMemTable(ColumnFamilyHandle* column_family = nullptr); @@ -170,12 +370,6 @@ class DBImpl : public DB { // Wait for any compaction Status TEST_WaitForCompact(); - // Return an internal iterator over the current state of the database. - // The keys of this iterator are internal keys (see format.h). - // The returned iterator should be deleted when no longer needed. - Iterator* TEST_NewInternalIterator(ColumnFamilyHandle* column_family = - nullptr); - // Return the maximum overlapping data (in bytes) at next level for any // file at a level >= 1. int64_t TEST_MaxNextLevelOverlappingBytes(ColumnFamilyHandle* column_family = @@ -184,394 +378,770 @@ class DBImpl : public DB { // Return the current manifest file no. uint64_t TEST_Current_Manifest_FileNo(); - // Trigger's a background call for testing. - void TEST_PurgeObsoleteteWAL(); - // get total level0 file size. Only for testing. uint64_t TEST_GetLevel0TotalSize(); - void TEST_SetDefaultTimeToCheck(uint64_t default_interval_to_delete_obsolete_WAL) - { - default_interval_to_delete_obsolete_WAL_ = default_interval_to_delete_obsolete_WAL; - } - void TEST_GetFilesMetaData(ColumnFamilyHandle* column_family, std::vector>* metadata); - Status TEST_ReadFirstRecord(const WalFileType type, const uint64_t number, - SequenceNumber* sequence); + void TEST_LockMutex(); - Status TEST_ReadFirstLine(const std::string& fname, SequenceNumber* sequence); -#endif // NDEBUG + void TEST_UnlockMutex(); - // Structure to store information for candidate files to delete. - struct CandidateFileInfo { - std::string file_name; - uint32_t path_id; - CandidateFileInfo(std::string name, uint32_t path) - : file_name(name), path_id(path) {} - bool operator==(const CandidateFileInfo& other) const { - return file_name == other.file_name && path_id == other.path_id; - } - }; + // REQUIRES: mutex locked + void* TEST_BeginWrite(); - // needed for CleanupIteratorState - struct DeletionState { - inline bool HaveSomethingToDelete() const { - return candidate_files.size() || - sst_delete_files.size() || - log_delete_files.size(); - } + // REQUIRES: mutex locked + // pass the pointer that you got from TEST_BeginWrite() + void TEST_EndWrite(void* w); - // a list of all files that we'll consider deleting - // (every once in a while this is filled up with all files - // in the DB directory) - std::vector candidate_files; + uint64_t TEST_MaxTotalInMemoryState() const { + return max_total_in_memory_state_; + } - // the list of all live sst files that cannot be deleted - std::vector sst_live; + size_t TEST_LogsToFreeSize(); - // a list of sst files that we need to delete - std::vector sst_delete_files; + uint64_t TEST_LogfileNumber(); - // a list of log files that we need to delete - std::vector log_delete_files; + uint64_t TEST_total_log_size() const { return total_log_size_; } - // a list of memtables to be free - autovector memtables_to_free; + // Returns column family name to ImmutableCFOptions map. + Status TEST_GetAllImmutableCFOptions( + std::unordered_map* iopts_map); - autovector superversions_to_free; + // Return the lastest MutableCFOptions of a column family + Status TEST_GetLatestMutableCFOptions(ColumnFamilyHandle* column_family, + MutableCFOptions* mutable_cf_options); - SuperVersion* new_superversion; // if nullptr no new superversion + Cache* TEST_table_cache() { return table_cache_.get(); } - // the current manifest_file_number, log_number and prev_log_number - // that corresponds to the set of files in 'live'. - uint64_t manifest_file_number, pending_manifest_file_number, log_number, - prev_log_number; + WriteController& TEST_write_controler() { return write_controller_; } - explicit DeletionState(bool create_superversion = false) { - manifest_file_number = 0; - pending_manifest_file_number = 0; - log_number = 0; - prev_log_number = 0; - new_superversion = create_superversion ? new SuperVersion() : nullptr; - } + uint64_t TEST_FindMinLogContainingOutstandingPrep(); + uint64_t TEST_FindMinPrepLogReferencedByMemTable(); - ~DeletionState() { - // free pending memtables - for (auto m : memtables_to_free) { - delete m; - } - // free superversions - for (auto s : superversions_to_free) { - delete s; - } - // if new_superversion was not used, it will be non-nullptr and needs - // to be freed here - delete new_superversion; - } + int TEST_BGCompactionsAllowed() const; + int TEST_BGFlushesAllowed() const; + +#endif // NDEBUG + + struct BGJobLimits { + int max_flushes; + int max_compactions; }; + // Returns maximum background flushes and compactions allowed to be scheduled + BGJobLimits GetBGJobLimits() const; + // Need a static version that can be called during SanitizeOptions(). + static BGJobLimits GetBGJobLimits(int max_background_flushes, + int max_background_compactions, + int max_background_jobs, + bool parallelize_compactions); + + // move logs pending closing from job_context to the DB queue and + // schedule a purge + void ScheduleBgLogWriterClose(JobContext* job_context); + + uint64_t MinLogNumberToKeep(); // Returns the list of live files in 'live' and the list // of all files in the filesystem in 'candidate_files'. // If force == false and the last call was less than - // options_.delete_obsolete_files_period_micros microseconds ago, - // it will not fill up the deletion_state - void FindObsoleteFiles(DeletionState& deletion_state, - bool force, + // db_options_.delete_obsolete_files_period_micros microseconds ago, + // it will not fill up the job_context + void FindObsoleteFiles(JobContext* job_context, bool force, bool no_full_scan = false); // Diffs the files listed in filenames and those that do not // belong to live files are posibly removed. Also, removes all the // files in sst_delete_files and log_delete_files. // It is not necessary to hold the mutex when invoking this method. - void PurgeObsoleteFiles(DeletionState& deletion_state); + void PurgeObsoleteFiles(const JobContext& background_contet, + bool schedule_only = false); + + void SchedulePurge(); + + ColumnFamilyHandle* DefaultColumnFamily() const override; + + const SnapshotList& snapshots() const { return snapshots_; } + + const ImmutableDBOptions& immutable_db_options() const { + return immutable_db_options_; + } + + void CancelAllBackgroundWork(bool wait); + + // Find Super version and reference it. Based on options, it might return + // the thread local cached one. + // Call ReturnAndCleanupSuperVersion() when it is no longer needed. + SuperVersion* GetAndRefSuperVersion(ColumnFamilyData* cfd); + + // Similar to the previous function but looks up based on a column family id. + // nullptr will be returned if this column family no longer exists. + // REQUIRED: this function should only be called on the write thread or if the + // mutex is held. + SuperVersion* GetAndRefSuperVersion(uint32_t column_family_id); + + // Un-reference the super version and return it to thread local cache if + // needed. If it is the last reference of the super version. Clean it up + // after un-referencing it. + void ReturnAndCleanupSuperVersion(ColumnFamilyData* cfd, SuperVersion* sv); + + // Similar to the previous function but looks up based on a column family id. + // nullptr will be returned if this column family no longer exists. + // REQUIRED: this function should only be called on the write thread. + void ReturnAndCleanupSuperVersion(uint32_t colun_family_id, SuperVersion* sv); + + // REQUIRED: this function should only be called on the write thread or if the + // mutex is held. Return value only valid until next call to this function or + // mutex is released. + ColumnFamilyHandle* GetColumnFamilyHandle(uint32_t column_family_id); + + // Same as above, should called without mutex held and not on write thread. + ColumnFamilyHandle* GetColumnFamilyHandleUnlocked(uint32_t column_family_id); + + // Returns the number of currently running flushes. + // REQUIREMENT: mutex_ must be held when calling this function. + int num_running_flushes() { + mutex_.AssertHeld(); + return num_running_flushes_; + } + + // Returns the number of currently running compactions. + // REQUIREMENT: mutex_ must be held when calling this function. + int num_running_compactions() { + mutex_.AssertHeld(); + return num_running_compactions_; + } + + const WriteController& write_controller() { return write_controller_; } + + InternalIterator* NewInternalIterator(const ReadOptions&, + ColumnFamilyData* cfd, + SuperVersion* super_version, + Arena* arena, + RangeDelAggregator* range_del_agg); + + // hollow transactions shell used for recovery. + // these will then be passed to TransactionDB so that + // locks can be reacquired before writing can resume. + struct RecoveredTransaction { + uint64_t log_number_; + std::string name_; + WriteBatch* batch_; + explicit RecoveredTransaction(const uint64_t log, const std::string& name, + WriteBatch* batch) + : log_number_(log), name_(name), batch_(batch) {} + + ~RecoveredTransaction() { delete batch_; } + }; - ColumnFamilyHandle* DefaultColumnFamily() const; + bool allow_2pc() const { return immutable_db_options_.allow_2pc; } + + std::unordered_map + recovered_transactions() { + return recovered_transactions_; + } + + RecoveredTransaction* GetRecoveredTransaction(const std::string& name) { + auto it = recovered_transactions_.find(name); + if (it == recovered_transactions_.end()) { + return nullptr; + } else { + return it->second; + } + } + + void InsertRecoveredTransaction(const uint64_t log, const std::string& name, + WriteBatch* batch) { + recovered_transactions_[name] = new RecoveredTransaction(log, name, batch); + MarkLogAsContainingPrepSection(log); + } + + void DeleteRecoveredTransaction(const std::string& name) { + auto it = recovered_transactions_.find(name); + assert(it != recovered_transactions_.end()); + auto* trx = it->second; + recovered_transactions_.erase(it); + MarkLogAsHavingPrepSectionFlushed(trx->log_number_); + delete trx; + } + + void DeleteAllRecoveredTransactions() { + for (auto it = recovered_transactions_.begin(); + it != recovered_transactions_.end(); it++) { + delete it->second; + } + recovered_transactions_.clear(); + } + + void MarkLogAsHavingPrepSectionFlushed(uint64_t log); + void MarkLogAsContainingPrepSection(uint64_t log); + void AddToLogsToFreeQueue(log::Writer* log_writer) { + logs_to_free_queue_.push_back(log_writer); + } + InstrumentedMutex* mutex() { return &mutex_; } + + Status NewDB(); protected: Env* const env_; const std::string dbname_; unique_ptr versions_; - const DBOptions options_; + const DBOptions initial_db_options_; + const ImmutableDBOptions immutable_db_options_; + MutableDBOptions mutable_db_options_; Statistics* stats_; + std::unordered_map + recovered_transactions_; + + // Except in DB::Open(), WriteOptionsFile can only be called when: + // Persist options to options file. + // If need_mutex_lock = false, the method will lock DB mutex. + // If need_enter_write_thread = false, the method will enter write thread. + Status WriteOptionsFile(bool need_mutex_lock, bool need_enter_write_thread); + + // The following two functions can only be called when: + // 1. WriteThread::Writer::EnterUnbatched() is used. + // 2. db_mutex is NOT held + Status RenameTempFileToOptionsFile(const std::string& file_name); + Status DeleteObsoleteOptionsFiles(); + + void NotifyOnFlushBegin(ColumnFamilyData* cfd, FileMetaData* file_meta, + const MutableCFOptions& mutable_cf_options, + int job_id, TableProperties prop); + + void NotifyOnFlushCompleted(ColumnFamilyData* cfd, FileMetaData* file_meta, + const MutableCFOptions& mutable_cf_options, + int job_id, TableProperties prop); + + void NotifyOnCompactionCompleted(ColumnFamilyData* cfd, + Compaction *c, const Status &st, + const CompactionJobStats& job_stats, + int job_id); + void NotifyOnMemTableSealed(ColumnFamilyData* cfd, + const MemTableInfo& mem_table_info); + +#ifndef ROCKSDB_LITE + void NotifyOnExternalFileIngested( + ColumnFamilyData* cfd, const ExternalSstFileIngestionJob& ingestion_job); +#endif // !ROCKSDB_LITE + + void NewThreadStatusCfInfo(ColumnFamilyData* cfd) const; + + void EraseThreadStatusCfInfo(ColumnFamilyData* cfd) const; - Iterator* NewInternalIterator(const ReadOptions&, ColumnFamilyData* cfd, - SuperVersion* super_version, - Arena* arena = nullptr); + void EraseThreadStatusDbInfo() const; + + Status WriteImpl(const WriteOptions& options, WriteBatch* updates, + WriteCallback* callback = nullptr, + uint64_t* log_used = nullptr, uint64_t log_ref = 0, + bool disable_memtable = false, uint64_t* seq_used = nullptr); + + Status PipelinedWriteImpl(const WriteOptions& options, WriteBatch* updates, + WriteCallback* callback = nullptr, + uint64_t* log_used = nullptr, uint64_t log_ref = 0, + bool disable_memtable = false, + uint64_t* seq_used = nullptr); + + Status WriteImplWALOnly(const WriteOptions& options, WriteBatch* updates, + WriteCallback* callback = nullptr, + uint64_t* log_used = nullptr, uint64_t log_ref = 0, + uint64_t* seq_used = nullptr); + + uint64_t FindMinLogContainingOutstandingPrep(); + uint64_t FindMinPrepLogReferencedByMemTable(); private: friend class DB; friend class InternalStats; + friend class PessimisticTransaction; + friend class WriteCommittedTxn; + friend class WritePreparedTxn; #ifndef ROCKSDB_LITE - friend class TailingIterator; friend class ForwardIterator; #endif friend struct SuperVersion; + friend class CompactedDBImpl; +#ifndef NDEBUG + friend class DBTest2_ReadCallbackTest_Test; + friend class XFTransactionWriteHandler; + friend class DBBlobIndexTest; +#endif struct CompactionState; - struct Writer; - struct WriteContext; - Status NewDB(); + struct WriteContext { + autovector superversions_to_free_; + autovector memtables_to_free_; + + ~WriteContext() { + for (auto& sv : superversions_to_free_) { + delete sv; + } + for (auto& m : memtables_to_free_) { + delete m; + } + } + }; + + struct PrepickedCompaction; + struct PurgeFileInfo; // Recover the descriptor from persistent storage. May do a significant // amount of work to recover recently logged updates. Any changes to // be made to the descriptor are added to *edit. Status Recover(const std::vector& column_families, - bool read_only = false, bool error_if_log_file_exist = false); + bool read_only = false, bool error_if_log_file_exist = false, + bool error_if_data_exists_in_logs = false); void MaybeIgnoreError(Status* s) const; const Status CreateArchivalDirectory(); + Status CreateColumnFamilyImpl(const ColumnFamilyOptions& cf_options, + const std::string& cf_name, + ColumnFamilyHandle** handle); + + Status DropColumnFamilyImpl(ColumnFamilyHandle* column_family); + // Delete any unneeded files and stale in-memory entries. void DeleteObsoleteFiles(); + // Delete obsolete files and log status and information of file deletion + void DeleteObsoleteFileImpl(Status file_deletion_status, int job_id, + const std::string& fname, FileType type, + uint64_t number, uint32_t path_id); + + // Background process needs to call + // auto x = CaptureCurrentFileNumberInPendingOutputs() + // auto file_num = versions_->NewFileNumber(); + // + // ReleaseFileNumberFromPendingOutputs(x) + // This will protect any file with number `file_num` or greater from being + // deleted while is running. + // ----------- + // This function will capture current file number and append it to + // pending_outputs_. This will prevent any background process to delete any + // file created after this point. + std::list::iterator CaptureCurrentFileNumberInPendingOutputs(); + // This function should be called with the result of + // CaptureCurrentFileNumberInPendingOutputs(). It then marks that any file + // created between the calls CaptureCurrentFileNumberInPendingOutputs() and + // ReleaseFileNumberFromPendingOutputs() can now be deleted (if it's not live + // and blocked by any other pending_outputs_ calls) + void ReleaseFileNumberFromPendingOutputs(std::list::iterator v); + + Status SyncClosedLogs(JobContext* job_context); // Flush the in-memory write buffer to storage. Switches to a new // log-file/memtable and writes a new descriptor iff successful. - Status FlushMemTableToOutputFile(ColumnFamilyData* cfd, bool* madeProgress, - DeletionState& deletion_state, + Status FlushMemTableToOutputFile(ColumnFamilyData* cfd, + const MutableCFOptions& mutable_cf_options, + bool* madeProgress, JobContext* job_context, LogBuffer* log_buffer); - Status RecoverLogFile(uint64_t log_number, SequenceNumber* max_sequence, - bool read_only); + // REQUIRES: log_numbers are sorted in ascending order + Status RecoverLogFiles(const std::vector& log_numbers, + SequenceNumber* next_sequence, bool read_only); // The following two methods are used to flush a memtable to - // storage. The first one is used atdatabase RecoveryTime (when the + // storage. The first one is used at database RecoveryTime (when the // database is opened) and is heavyweight because it holds the mutex // for the entire period. The second method WriteLevel0Table supports // concurrent flush memtables to storage. - Status WriteLevel0TableForRecovery(ColumnFamilyData* cfd, MemTable* mem, - VersionEdit* edit); - Status WriteLevel0Table(ColumnFamilyData* cfd, autovector& mems, - VersionEdit* edit, uint64_t* filenumber, - LogBuffer* log_buffer); - - uint64_t SlowdownAmount(int n, double bottom, double top); - - // Before applying write operation (such as DBImpl::Write, DBImpl::Flush) - // thread should grab the mutex_ and be the first on writers queue. - // BeginWrite is used for it. - // Be aware! Writer's job can be done by other thread (see DBImpl::Write - // for examples), so check it via w.done before applying changes. - // - // Writer* w: writer to be placed in the queue - // uint64_t expiration_time: maximum time to be in the queue - // See also: EndWrite - Status BeginWrite(Writer* w, uint64_t expiration_time); - - // After doing write job, we need to remove already used writers from - // writers_ queue and notify head of the queue about it. - // EndWrite is used for this. - // - // Writer* w: Writer, that was added by BeginWrite function - // Writer* last_writer: Since we can join a few Writers (as DBImpl::Write - // does) - // we should pass last_writer as a parameter to - // EndWrite - // (if you don't touch other writers, just pass w) - // Status status: Status of write operation - // See also: BeginWrite - void EndWrite(Writer* w, Writer* last_writer, Status status); - - Status MakeRoomForWrite(ColumnFamilyData* cfd, - WriteContext* context, - uint64_t expiration_time); - - Status SetNewMemtableAndNewLogFile(ColumnFamilyData* cfd, - WriteContext* context); - - void BuildBatchGroup(Writer** last_writer, - autovector* write_batch_group); + Status WriteLevel0TableForRecovery(int job_id, ColumnFamilyData* cfd, + MemTable* mem, VersionEdit* edit); + + // num_bytes: for slowdown case, delay time is calculated based on + // `num_bytes` going through. + Status DelayWrite(uint64_t num_bytes, const WriteOptions& write_options); + + Status ThrottleLowPriWritesIfNeeded(const WriteOptions& write_options, + WriteBatch* my_batch); + + Status ScheduleFlushes(WriteContext* context); + + Status SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context); // Force current memtable contents to be flushed. - Status FlushMemTable(ColumnFamilyData* cfd, const FlushOptions& options); + Status FlushMemTable(ColumnFamilyData* cfd, const FlushOptions& options, + bool writes_stopped = false); // Wait for memtable flushed Status WaitForFlushMemTable(ColumnFamilyData* cfd); - void RecordFlushIOStats(); - void RecordCompactionIOStats(); + // REQUIRES: mutex locked + Status HandleWALFull(WriteContext* write_context); - void MaybeScheduleFlushOrCompaction(); - static void BGWorkCompaction(void* db); - static void BGWorkFlush(void* db); - void BackgroundCallCompaction(); - void BackgroundCallFlush(); - Status BackgroundCompaction(bool* madeProgress, DeletionState& deletion_state, - LogBuffer* log_buffer); - Status BackgroundFlush(bool* madeProgress, DeletionState& deletion_state, - LogBuffer* log_buffer); - void CleanupCompaction(CompactionState* compact, Status status); - Status DoCompactionWork(CompactionState* compact, - DeletionState& deletion_state, - LogBuffer* log_buffer); - - // This function is called as part of compaction. It enables Flush process to - // preempt compaction, since it's higher prioirty - // Returns: micros spent executing - uint64_t CallFlushDuringCompaction(ColumnFamilyData* cfd, - DeletionState& deletion_state, - LogBuffer* log_buffer); - - // Call compaction filter if is_compaction_v2 is not true. Then iterate - // through input and compact the kv-pairs - Status ProcessKeyValueCompaction( - bool is_snapshot_supported, - SequenceNumber visible_at_tip, - SequenceNumber earliest_snapshot, - SequenceNumber latest_snapshot, - DeletionState& deletion_state, - bool bottommost_level, - int64_t& imm_micros, - Iterator* input, - CompactionState* compact, - bool is_compaction_v2, - LogBuffer* log_buffer); - - // Call compaction_filter_v2->Filter() on kv-pairs in compact - void CallCompactionFilterV2(CompactionState* compact, - CompactionFilterV2* compaction_filter_v2); - - Status OpenCompactionOutputFile(CompactionState* compact); - Status FinishCompactionOutputFile(CompactionState* compact, Iterator* input); - Status InstallCompactionResults(CompactionState* compact, - LogBuffer* log_buffer); - void AllocateCompactionOutputFileNumbers(CompactionState* compact); - void ReleaseCompactionUnusedFileNumbers(CompactionState* compact); - -#ifdef ROCKSDB_LITE - void PurgeObsoleteWALFiles() { - // this function is used for archiving WAL files. we don't need this in - // ROCKSDB_LITE - } -#else - void PurgeObsoleteWALFiles(); + // REQUIRES: mutex locked + Status HandleWriteBufferFull(WriteContext* write_context); + + // REQUIRES: mutex locked + Status PreprocessWrite(const WriteOptions& write_options, bool* need_log_sync, + WriteContext* write_context); + + WriteBatch* MergeBatch(const WriteThread::WriteGroup& write_group, + WriteBatch* tmp_batch, size_t* write_with_wal); + + Status WriteToWAL(const WriteBatch& merged_batch, log::Writer* log_writer, + uint64_t* log_used, uint64_t* log_size); + + Status WriteToWAL(const WriteThread::WriteGroup& write_group, + log::Writer* log_writer, uint64_t* log_used, + bool need_log_sync, bool need_log_dir_sync, + SequenceNumber sequence); + + Status ConcurrentWriteToWAL(const WriteThread::WriteGroup& write_group, + uint64_t* log_used, SequenceNumber* last_sequence, + int total_count); - Status GetSortedWalsOfType(const std::string& path, - VectorLogPtr& log_files, - WalFileType type); + // Used by WriteImpl to update bg_error_ if paranoid check is enabled. + void WriteCallbackStatusCheck(const Status& status); - // Requires: all_logs should be sorted with earliest log file first - // Retains all log files in all_logs which contain updates with seq no. - // Greater Than or Equal to the requested SequenceNumber. - Status RetainProbableWalFiles(VectorLogPtr& all_logs, - const SequenceNumber target); + // Used by WriteImpl to update bg_error_ in case of memtable insert error. + void MemTableInsertStatusCheck(const Status& memtable_insert_status); - Status ReadFirstRecord(const WalFileType type, const uint64_t number, - SequenceNumber* sequence); +#ifndef ROCKSDB_LITE + + Status CompactFilesImpl(const CompactionOptions& compact_options, + ColumnFamilyData* cfd, Version* version, + const std::vector& input_file_names, + const int output_level, int output_path_id, + JobContext* job_context, LogBuffer* log_buffer); - Status ReadFirstLine(const std::string& fname, SequenceNumber* sequence); + // Wait for current IngestExternalFile() calls to finish. + // REQUIRES: mutex_ held + void WaitForIngestFile(); + +#else + // IngestExternalFile is not supported in ROCKSDB_LITE so this function + // will be no-op + void WaitForIngestFile() {} #endif // ROCKSDB_LITE + ColumnFamilyData* GetColumnFamilyDataByName(const std::string& cf_name); + + void MaybeScheduleFlushOrCompaction(); + void SchedulePendingFlush(ColumnFamilyData* cfd); + void SchedulePendingCompaction(ColumnFamilyData* cfd); + void SchedulePendingPurge(std::string fname, FileType type, uint64_t number, + uint32_t path_id, int job_id); + static void BGWorkCompaction(void* arg); + // Runs a pre-chosen universal compaction involving bottom level in a + // separate, bottom-pri thread pool. + static void BGWorkBottomCompaction(void* arg); + static void BGWorkFlush(void* db); + static void BGWorkPurge(void* arg); + static void UnscheduleCallback(void* arg); + void BackgroundCallCompaction(PrepickedCompaction* prepicked_compaction, + Env::Priority bg_thread_pri); + void BackgroundCallFlush(); + void BackgroundCallPurge(); + Status BackgroundCompaction(bool* madeProgress, JobContext* job_context, + LogBuffer* log_buffer, + PrepickedCompaction* prepicked_compaction); + Status BackgroundFlush(bool* madeProgress, JobContext* job_context, + LogBuffer* log_buffer); + void PrintStatistics(); // dump rocksdb.stats to LOG void MaybeDumpStats(); - // Return true if the current db supports snapshot. If the current - // DB does not support snapshot, then calling GetSnapshot() will always - // return nullptr. - // - // @see GetSnapshot() - virtual bool IsSnapshotSupported() const; - // Return the minimum empty level that could hold the total data in the // input level. Return the input level, if such level could not be found. - int FindMinimumEmptyLevelFitting(ColumnFamilyData* cfd, int level); + int FindMinimumEmptyLevelFitting(ColumnFamilyData* cfd, + const MutableCFOptions& mutable_cf_options, int level); // Move the files in the input level to the target level. // If target_level < 0, automatically calculate the minimum level that could // hold the data set. Status ReFitLevel(ColumnFamilyData* cfd, int level, int target_level = -1); + // helper functions for adding and removing from flush & compaction queues + void AddToCompactionQueue(ColumnFamilyData* cfd); + ColumnFamilyData* PopFirstFromCompactionQueue(); + void AddToFlushQueue(ColumnFamilyData* cfd); + ColumnFamilyData* PopFirstFromFlushQueue(); + + // helper function to call after some of the logs_ were synced + void MarkLogsSynced(uint64_t up_to, bool synced_dir, const Status& status); + + const Snapshot* GetSnapshotImpl(bool is_write_conflict_boundary); + + uint64_t GetMaxTotalWalSize() const; + // table_cache_ provides its own synchronization std::shared_ptr table_cache_; // Lock over the persistent DB state. Non-nullptr iff successfully acquired. FileLock* db_lock_; + // In addition to mutex_, log_write_mutex_ protected writes to logs_ and + // logfile_number_. With concurrent_prepare it also protects alive_log_files_, + // and log_empty_. Refer to the definition of each variable below for more + // details. + InstrumentedMutex log_write_mutex_; // State below is protected by mutex_ - port::Mutex mutex_; - port::AtomicPointer shutting_down_; + // With concurrent_prepare enabled, some of the variables that accessed during + // WriteToWAL need different synchronization: log_empty_, alive_log_files_, + // logs_, logfile_number_. Refer to the definition of each variable below for + // more description. + mutable InstrumentedMutex mutex_; + + std::atomic shutting_down_; // This condition variable is signaled on these conditions: // * whenever bg_compaction_scheduled_ goes down to 0 - // * if bg_manual_only_ > 0, whenever a compaction finishes, even if it hasn't + // * if AnyManualCompaction, whenever a compaction finishes, even if it hasn't // made any progress // * whenever a compaction made any progress - // * whenever bg_flush_scheduled_ value decreases (i.e. whenever a flush is - // done, even if it didn't make any progress) - // * whenever there is an error in background flush or compaction - port::CondVar bg_cv_; + // * whenever bg_flush_scheduled_ or bg_purge_scheduled_ value decreases + // (i.e. whenever a flush is done, even if it didn't make any progress) + // * whenever there is an error in background purge, flush or compaction + // * whenever num_running_ingest_file_ goes to 0. + InstrumentedCondVar bg_cv_; + // Writes are protected by locking both mutex_ and log_write_mutex_, and reads + // must be under either mutex_ or log_write_mutex_. Since after ::Open, + // logfile_number_ is currently updated only in write_thread_, it can be read + // from the same write_thread_ without any locks. uint64_t logfile_number_; - unique_ptr log_; + std::deque + log_recycle_files; // a list of log files that we can recycle + bool log_dir_synced_; + // Without concurrent_prepare, read and writes to log_empty_ are protected by + // mutex_. Since it is currently updated/read only in write_thread_, it can be + // accessed from the same write_thread_ without any locks. With + // concurrent_prepare writes, where it can be updated in different threads, + // read and writes are protected by log_write_mutex_ instead. This is to avoid + // expesnive mutex_ lock during WAL write, which update log_empty_. bool log_empty_; ColumnFamilyHandleImpl* default_cf_handle_; InternalStats* default_cf_internal_stats_; unique_ptr column_family_memtables_; struct LogFileNumberSize { explicit LogFileNumberSize(uint64_t _number) - : number(_number), size(0), getting_flushed(false) {} + : number(_number) {} void AddSize(uint64_t new_size) { size += new_size; } uint64_t number; - uint64_t size; - bool getting_flushed; + uint64_t size = 0; + bool getting_flushed = false; }; + struct LogWriterNumber { + // pass ownership of _writer + LogWriterNumber(uint64_t _number, log::Writer* _writer) + : number(_number), writer(_writer) {} + + log::Writer* ReleaseWriter() { + auto* w = writer; + writer = nullptr; + return w; + } + void ClearWriter() { + delete writer; + writer = nullptr; + } + + uint64_t number; + // Visual Studio doesn't support deque's member to be noncopyable because + // of a unique_ptr as a member. + log::Writer* writer; // own + // true for some prefix of logs_ + bool getting_synced = false; + }; + // Without concurrent_prepare, read and writes to alive_log_files_ are + // protected by mutex_. However since back() is never popped, and push_back() + // is done only from write_thread_, the same thread can access the item + // reffered by back() without mutex_. With concurrent_prepare_, writes + // are protected by locking both mutex_ and log_write_mutex_, and reads must + // be under either mutex_ or log_write_mutex_. std::deque alive_log_files_; - uint64_t total_log_size_; + // Log files that aren't fully synced, and the current log file. + // Synchronization: + // - push_back() is done from write_thread_ with locked mutex_ and + // log_write_mutex_ + // - pop_front() is done from any thread with locked mutex_ and + // log_write_mutex_ + // - reads are done with either locked mutex_ or log_write_mutex_ + // - back() and items with getting_synced=true are not popped, + // - The same thread that sets getting_synced=true will reset it. + // - it follows that the object referred by back() can be safely read from + // the write_thread_ without using mutex + // - it follows that the items with getting_synced=true can be safely read + // from the same thread that has set getting_synced=true + std::deque logs_; + // Signaled when getting_synced becomes false for some of the logs_. + InstrumentedCondVar log_sync_cv_; + std::atomic total_log_size_; // only used for dynamically adjusting max_total_wal_size. it is a sum of // [write_buffer_size * max_write_buffer_number] over all column families uint64_t max_total_in_memory_state_; // If true, we have only one (default) column family. We use this to optimize // some code-paths bool single_column_family_mode_; + // If this is non-empty, we need to delete these log files in background + // threads. Protected by db mutex. + autovector logs_to_free_; + + bool is_snapshot_supported_; + + // Class to maintain directories for all database paths other than main one. + class Directories { + public: + Status SetDirectories(Env* env, const std::string& dbname, + const std::string& wal_dir, + const std::vector& data_paths); + + Directory* GetDataDir(size_t path_id); + + Directory* GetWalDir() { + if (wal_dir_) { + return wal_dir_.get(); + } + return db_dir_.get(); + } + + Directory* GetDbDir() { return db_dir_.get(); } + + private: + std::unique_ptr db_dir_; + std::vector> data_dirs_; + std::unique_ptr wal_dir_; + + Status CreateAndNewDirectory(Env* env, const std::string& dirname, + std::unique_ptr* directory) const; + }; + + Directories directories_; - std::unique_ptr db_directory_; + WriteBufferManager* write_buffer_manager_; - // Queue of writers. - std::deque writers_; + WriteThread write_thread_; WriteBatch tmp_batch_; + // The write thread when the writers have no memtable write. This will be used + // in 2PC to batch the prepares separately from the serial commit. + WriteThread nonmem_write_thread_; - SnapshotList snapshots_; + WriteController write_controller_; + + unique_ptr low_pri_write_rate_limiter_; + + // Size of the last batch group. In slowdown mode, next write needs to + // sleep if it uses up the quota. + // Note: This is to protect memtable and compaction. If the batch only writes + // to the WAL its size need not to be included in this. + uint64_t last_batch_group_size_; - // cache for ReadFirstRecord() calls - std::unordered_map read_first_record_cache_; - port::Mutex read_first_record_cache_mutex_; + FlushScheduler flush_scheduler_; - // Set of table files to protect from deletion because they are - // part of ongoing compactions. - // map from pending file number ID to their path IDs. - FileNumToPathIdMap pending_outputs_; + SnapshotList snapshots_; + + // For each background job, pending_outputs_ keeps the current file number at + // the time that background job started. + // FindObsoleteFiles()/PurgeObsoleteFiles() never deletes any file that has + // number bigger than any of the file number in pending_outputs_. Since file + // numbers grow monotonically, this also means that pending_outputs_ is always + // sorted. After a background job is done executing, its file number is + // deleted from pending_outputs_, which allows PurgeObsoleteFiles() to clean + // it up. + // State is protected with db mutex. + std::list pending_outputs_; + + // PurgeFileInfo is a structure to hold information of files to be deleted in + // purge_queue_ + struct PurgeFileInfo { + std::string fname; + FileType type; + uint64_t number; + uint32_t path_id; + int job_id; + PurgeFileInfo(std::string fn, FileType t, uint64_t num, uint32_t pid, + int jid) + : fname(fn), type(t), number(num), path_id(pid), job_id(jid) {} + }; - // At least one compaction or flush job is pending but not yet scheduled - // because of the max background thread limit. - bool bg_schedule_needed_; + // flush_queue_ and compaction_queue_ hold column families that we need to + // flush and compact, respectively. + // A column family is inserted into flush_queue_ when it satisfies condition + // cfd->imm()->IsFlushPending() + // A column family is inserted into compaction_queue_ when it satisfied + // condition cfd->NeedsCompaction() + // Column families in this list are all Ref()-erenced + // TODO(icanadi) Provide some kind of ReferencedColumnFamily class that will + // do RAII on ColumnFamilyData + // Column families are in this queue when they need to be flushed or + // compacted. Consumers of these queues are flush and compaction threads. When + // column family is put on this queue, we increase unscheduled_flushes_ and + // unscheduled_compactions_. When these variables are bigger than zero, that + // means we need to schedule background threads for compaction and thread. + // Once the background threads are scheduled, we decrease unscheduled_flushes_ + // and unscheduled_compactions_. That way we keep track of number of + // compaction and flush threads we need to schedule. This scheduling is done + // in MaybeScheduleFlushOrCompaction() + // invariant(column family present in flush_queue_ <==> + // ColumnFamilyData::pending_flush_ == true) + std::deque flush_queue_; + // invariant(column family present in compaction_queue_ <==> + // ColumnFamilyData::pending_compaction_ == true) + std::deque compaction_queue_; + + // A queue to store filenames of the files to be purged + std::deque purge_queue_; + + // A queue to store log writers to close + std::deque logs_to_free_queue_; + int unscheduled_flushes_; + int unscheduled_compactions_; + + // count how many background compactions are running or have been scheduled in + // the BOTTOM pool + int bg_bottom_compaction_scheduled_; // count how many background compactions are running or have been scheduled int bg_compaction_scheduled_; - // If non-zero, MaybeScheduleFlushOrCompaction() will only schedule manual - // compactions (if manual_compaction_ is not null). This mechanism enables - // manual compactions to wait until all other compactions are finished. - int bg_manual_only_; + // stores the number of compactions are currently running + int num_running_compactions_; // number of background memtable flush jobs, submitted to the HIGH pool int bg_flush_scheduled_; + // stores the number of flushes are currently running + int num_running_flushes_; + + // number of background obsolete file purge jobs, submitted to the HIGH pool + int bg_purge_scheduled_; + // Information for a manual compaction - struct ManualCompaction { + struct ManualCompactionState { ColumnFamilyData* cfd; int input_level; int output_level; uint32_t output_path_id; - bool done; Status status; - bool in_progress; // compaction request being processed? - const InternalKey* begin; // nullptr means beginning of key range - const InternalKey* end; // nullptr means end of key range - InternalKey tmp_storage; // Used to keep track of compaction progress + bool done; + bool in_progress; // compaction request being processed? + bool incomplete; // only part of requested range compacted + bool exclusive; // current behavior of only one manual + bool disallow_trivial_move; // Force actual compaction to run + const InternalKey* begin; // nullptr means beginning of key range + const InternalKey* end; // nullptr means end of key range + InternalKey* manual_end; // how far we are compacting + InternalKey tmp_storage; // Used to keep track of compaction progress + InternalKey tmp_storage1; // Used to keep track of compaction progress + }; + struct PrepickedCompaction { + // background compaction takes ownership of `compaction`. + Compaction* compaction; + // caller retains ownership of `manual_compaction_state` as it is reused + // across background compactions. + ManualCompactionState* manual_compaction_state; // nullptr if non-manual + }; + std::deque manual_compaction_dequeue_; + + struct CompactionArg { + // caller retains ownership of `db`. + DBImpl* db; + // background compaction takes ownership of `prepicked_compaction`. + PrepickedCompaction* prepicked_compaction; }; - ManualCompaction* manual_compaction_; // Have we encountered a background error in paranoid mode? Status bg_error_; @@ -584,33 +1154,55 @@ class DBImpl : public DB { // without any synchronization int disable_delete_obsolete_files_; - // last time when DeleteObsoleteFiles was invoked + // last time when DeleteObsoleteFiles with full scan was executed. Originaly + // initialized with startup time. uint64_t delete_obsolete_files_last_run_; - // last time when PurgeObsoleteWALFiles ran. - uint64_t purge_wal_files_last_run_; - // last time stats were dumped to LOG std::atomic last_stats_dump_time_microsec_; - // obsolete files will be deleted every this seconds if ttl deletion is - // enabled and archive size_limit is disabled. - uint64_t default_interval_to_delete_obsolete_WAL_; + // Each flush or compaction gets its own job id. this counter makes sure + // they're unique + std::atomic next_job_id_; - bool flush_on_destroy_; // Used when disableWAL is true. + // A flag indicating whether the current rocksdb database has any + // data that is not yet persisted into either WAL or SST file. + // Used when disableWAL is true. + std::atomic has_unpersisted_data_; + + // if an attempt was made to flush all column families that + // the oldest log depends on but uncommited data in the oldest + // log prevents the log from being released. + // We must attempt to free the dependent memtables again + // at a later time after the transaction in the oldest + // log is fully commited. + bool unable_to_flush_oldest_log_; static const int KEEP_LOG_FILE_NUM = 1000; - static const uint64_t kNoTimeOut = std::numeric_limits::max(); - std::string db_absolute_path_; + // MSVC version 1800 still does not have constexpr for ::max() + static const uint64_t kNoTimeOut = port::kMaxUint64; - // count of the number of contiguous delaying writes - int delayed_writes_; + std::string db_absolute_path_; // The options to access storage files - const EnvOptions storage_options_; + const EnvOptions env_options_; - // A value of true temporarily disables scheduling of background work - bool bg_work_gate_closed_; + // Number of running IngestExternalFile() calls. + // REQUIRES: mutex held + int num_running_ingest_file_; + +#ifndef ROCKSDB_LITE + WalManager wal_manager_; +#endif // ROCKSDB_LITE + + // Unified interface for logging events + EventLogger event_logger_; + + // A value of > 0 temporarily disables scheduling of background work + int bg_work_paused_; + + // A value of > 0 temporarily disables scheduling of background compaction + int bg_compaction_paused_; // Guard against multiple concurrent refitting bool refitting_level_; @@ -618,61 +1210,87 @@ class DBImpl : public DB { // Indicate DB was opened successfully bool opened_successfully_; + // minimum log number still containing prepared data. + // this is used by FindObsoleteFiles to determine which + // flushed logs we must keep around because they still + // contain prepared data which has not been flushed or rolled back + std::priority_queue, std::greater> + min_log_with_prep_; + + // to be used in conjunction with min_log_with_prep_. + // once a transaction with data in log L is committed or rolled back + // rather than removing the value from the heap we add that value + // to prepared_section_completed_ which maps LOG -> instance_count + // since a log could contain multiple prepared sections + // + // when trying to determine the minimum log still active we first + // consult min_log_with_prep_. while that root value maps to + // a value > 0 in prepared_section_completed_ we decrement the + // instance_count for that log and pop the root value in + // min_log_with_prep_. This will work the same as a min_heap + // where we are deleteing arbitrary elements and the up heaping. + std::unordered_map prepared_section_completed_; + std::mutex prep_heap_mutex_; + // No copying allowed DBImpl(const DBImpl&); void operator=(const DBImpl&); - // dump the delayed_writes_ to the log file and reset counter. - void DelayLoggingAndReset(); - - // Return the earliest snapshot where seqno is visible. - // Store the snapshot right before that, if any, in prev_snapshot - inline SequenceNumber findEarliestVisibleSnapshot( - SequenceNumber in, - std::vector& snapshots, - SequenceNumber* prev_snapshot); - // Background threads call this function, which is just a wrapper around - // the cfd->InstallSuperVersion() function. Background threads carry - // deletion_state which can have new_superversion already allocated. - void InstallSuperVersion(ColumnFamilyData* cfd, - DeletionState& deletion_state); - - // Find Super version and reference it. Based on options, it might return - // the thread local cached one. - inline SuperVersion* GetAndRefSuperVersion(ColumnFamilyData* cfd); - - // Un-reference the super version and return it to thread local cache if - // needed. If it is the last reference of the super version. Clean it up - // after un-referencing it. - inline void ReturnAndCleanupSuperVersion(ColumnFamilyData* cfd, - SuperVersion* sv); + // the InstallSuperVersion() function. Background threads carry + // job_context which can have new_superversion already + // allocated. + void InstallSuperVersionAndScheduleWorkWrapper( + ColumnFamilyData* cfd, JobContext* job_context, + const MutableCFOptions& mutable_cf_options); + + // All ColumnFamily state changes go through this function. Here we analyze + // the new state and we schedule background work if we detect that the new + // state needs flush or compaction. + SuperVersion* InstallSuperVersionAndScheduleWork( + ColumnFamilyData* cfd, SuperVersion* new_sv, + const MutableCFOptions& mutable_cf_options); #ifndef ROCKSDB_LITE using DB::GetPropertiesOfAllTables; virtual Status GetPropertiesOfAllTables(ColumnFamilyHandle* column_family, TablePropertiesCollection* props) override; + virtual Status GetPropertiesOfTablesInRange( + ColumnFamilyHandle* column_family, const Range* range, std::size_t n, + TablePropertiesCollection* props) override; + #endif // ROCKSDB_LITE - // Function that Get and KeyMayExist call with no_io true or false - // Note: 'value_found' from KeyMayExist propagates here - Status GetImpl(const ReadOptions& options, ColumnFamilyHandle* column_family, - const Slice& key, std::string* value, - bool* value_found = nullptr); + bool GetIntPropertyInternal(ColumnFamilyData* cfd, + const DBPropertyInfo& property_info, + bool is_locked, uint64_t* value); + + bool HasPendingManualCompaction(); + bool HasExclusiveManualCompaction(); + void AddManualCompaction(ManualCompactionState* m); + void RemoveManualCompaction(ManualCompactionState* m); + bool ShouldntRunManualCompaction(ManualCompactionState* m); + bool HaveManualCompaction(ColumnFamilyData* cfd); + bool MCOverlap(ManualCompactionState* m, ManualCompactionState* m1); + + size_t GetWalPreallocateBlockSize(uint64_t write_buffer_size) const; - bool GetIntPropertyInternal(ColumnFamilyHandle* column_family, - DBPropertyType property_type, - bool need_out_of_mutex, uint64_t* value); + // When set, we use a seprate queue for writes that dont write to memtable. In + // 2PC these are the writes at Prepare phase. + const bool concurrent_prepare_; + const bool manual_wal_flush_; }; -// Sanitize db options. The caller should delete result.info_log if -// it is not equal to src.info_log. extern Options SanitizeOptions(const std::string& db, - const InternalKeyComparator* icmp, const Options& src); + extern DBOptions SanitizeOptions(const std::string& db, const DBOptions& src); +extern CompressionType GetCompressionFlush( + const ImmutableCFOptions& ioptions, + const MutableCFOptions& mutable_cf_options); + // Fix user-supplied options to be reasonable template static void ClipToRange(T* ptr, V minvalue, V maxvalue) { @@ -680,8 +1298,4 @@ static void ClipToRange(T* ptr, V minvalue, V maxvalue) { if (static_cast(*ptr) < minvalue) *ptr = minvalue; } -// Dump db file summary, implemented in util/ -extern void DumpDBFileSummary(const DBOptions& options, - const std::string& dbname); - } // namespace rocksdb diff --git a/db/db_impl_compaction_flush.cc b/db/db_impl_compaction_flush.cc new file mode 100644 index 00000000000..3e686fe7039 --- /dev/null +++ b/db/db_impl_compaction_flush.cc @@ -0,0 +1,1910 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#include "db/db_impl.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif +#include + +#include "db/builder.h" +#include "db/event_helpers.h" +#include "monitoring/iostats_context_imp.h" +#include "monitoring/perf_context_imp.h" +#include "monitoring/thread_status_updater.h" +#include "monitoring/thread_status_util.h" +#include "util/sst_file_manager_impl.h" +#include "util/sync_point.h" + +namespace rocksdb { +Status DBImpl::SyncClosedLogs(JobContext* job_context) { + TEST_SYNC_POINT("DBImpl::SyncClosedLogs:Start"); + mutex_.AssertHeld(); + autovector logs_to_sync; + uint64_t current_log_number = logfile_number_; + while (logs_.front().number < current_log_number && + logs_.front().getting_synced) { + log_sync_cv_.Wait(); + } + for (auto it = logs_.begin(); + it != logs_.end() && it->number < current_log_number; ++it) { + auto& log = *it; + assert(!log.getting_synced); + log.getting_synced = true; + logs_to_sync.push_back(log.writer); + } + + Status s; + if (!logs_to_sync.empty()) { + mutex_.Unlock(); + + for (log::Writer* log : logs_to_sync) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "[JOB %d] Syncing log #%" PRIu64, job_context->job_id, + log->get_log_number()); + s = log->file()->Sync(immutable_db_options_.use_fsync); + } + if (s.ok()) { + s = directories_.GetWalDir()->Fsync(); + } + + mutex_.Lock(); + + // "number <= current_log_number - 1" is equivalent to + // "number < current_log_number". + MarkLogsSynced(current_log_number - 1, true, s); + if (!s.ok()) { + Status new_bg_error = s; + // may temporarily unlock and lock the mutex. + EventHelpers::NotifyOnBackgroundError(immutable_db_options_.listeners, + BackgroundErrorReason::kFlush, + &new_bg_error, &mutex_); + if (!new_bg_error.ok()) { + bg_error_ = new_bg_error; + } + TEST_SYNC_POINT("DBImpl::SyncClosedLogs:Failed"); + return s; + } + } + return s; +} + +Status DBImpl::FlushMemTableToOutputFile( + ColumnFamilyData* cfd, const MutableCFOptions& mutable_cf_options, + bool* made_progress, JobContext* job_context, LogBuffer* log_buffer) { + mutex_.AssertHeld(); + assert(cfd->imm()->NumNotFlushed() != 0); + assert(cfd->imm()->IsFlushPending()); + + SequenceNumber earliest_write_conflict_snapshot; + std::vector snapshot_seqs = + snapshots_.GetAll(&earliest_write_conflict_snapshot); + + FlushJob flush_job( + dbname_, cfd, immutable_db_options_, mutable_cf_options, env_options_, + versions_.get(), &mutex_, &shutting_down_, snapshot_seqs, + earliest_write_conflict_snapshot, job_context, log_buffer, + directories_.GetDbDir(), directories_.GetDataDir(0U), + GetCompressionFlush(*cfd->ioptions(), mutable_cf_options), stats_, + &event_logger_, mutable_cf_options.report_bg_io_stats); + + FileMetaData file_meta; + + flush_job.PickMemTable(); + +#ifndef ROCKSDB_LITE + // may temporarily unlock and lock the mutex. + NotifyOnFlushBegin(cfd, &file_meta, mutable_cf_options, job_context->job_id, + flush_job.GetTableProperties()); +#endif // ROCKSDB_LITE + + Status s; + if (logfile_number_ > 0 && + versions_->GetColumnFamilySet()->NumberOfColumnFamilies() > 0) { + // If there are more than one column families, we need to make sure that + // all the log files except the most recent one are synced. Otherwise if + // the host crashes after flushing and before WAL is persistent, the + // flushed SST may contain data from write batches whose updates to + // other column families are missing. + // SyncClosedLogs() may unlock and re-lock the db_mutex. + s = SyncClosedLogs(job_context); + } + + // Within flush_job.Run, rocksdb may call event listener to notify + // file creation and deletion. + // + // Note that flush_job.Run will unlock and lock the db_mutex, + // and EventListener callback will be called when the db_mutex + // is unlocked by the current thread. + if (s.ok()) { + s = flush_job.Run(&file_meta); + } else { + flush_job.Cancel(); + } + + if (s.ok()) { + InstallSuperVersionAndScheduleWorkWrapper(cfd, job_context, + mutable_cf_options); + if (made_progress) { + *made_progress = 1; + } + VersionStorageInfo::LevelSummaryStorage tmp; + ROCKS_LOG_BUFFER(log_buffer, "[%s] Level summary: %s\n", + cfd->GetName().c_str(), + cfd->current()->storage_info()->LevelSummary(&tmp)); + } + + if (!s.ok() && !s.IsShutdownInProgress() && + immutable_db_options_.paranoid_checks && bg_error_.ok()) { + Status new_bg_error = s; + // may temporarily unlock and lock the mutex. + EventHelpers::NotifyOnBackgroundError(immutable_db_options_.listeners, + BackgroundErrorReason::kFlush, + &new_bg_error, &mutex_); + if (!new_bg_error.ok()) { + // if a bad error happened (not ShutdownInProgress), paranoid_checks is + // true, and the error isn't handled by callback, mark DB read-only + bg_error_ = new_bg_error; + } + } + if (s.ok()) { +#ifndef ROCKSDB_LITE + // may temporarily unlock and lock the mutex. + NotifyOnFlushCompleted(cfd, &file_meta, mutable_cf_options, + job_context->job_id, flush_job.GetTableProperties()); + auto sfm = static_cast( + immutable_db_options_.sst_file_manager.get()); + if (sfm) { + // Notify sst_file_manager that a new file was added + std::string file_path = MakeTableFileName( + immutable_db_options_.db_paths[0].path, file_meta.fd.GetNumber()); + sfm->OnAddFile(file_path); + if (sfm->IsMaxAllowedSpaceReached() && bg_error_.ok()) { + Status new_bg_error = Status::IOError("Max allowed space was reached"); + TEST_SYNC_POINT_CALLBACK( + "DBImpl::FlushMemTableToOutputFile:MaxAllowedSpaceReached", + &new_bg_error); + // may temporarily unlock and lock the mutex. + EventHelpers::NotifyOnBackgroundError(immutable_db_options_.listeners, + BackgroundErrorReason::kFlush, + &new_bg_error, &mutex_); + if (!new_bg_error.ok()) { + bg_error_ = new_bg_error; + } + } + } +#endif // ROCKSDB_LITE + } + return s; +} + +void DBImpl::NotifyOnFlushBegin(ColumnFamilyData* cfd, FileMetaData* file_meta, + const MutableCFOptions& mutable_cf_options, + int job_id, TableProperties prop) { +#ifndef ROCKSDB_LITE + if (immutable_db_options_.listeners.size() == 0U) { + return; + } + mutex_.AssertHeld(); + if (shutting_down_.load(std::memory_order_acquire)) { + return; + } + bool triggered_writes_slowdown = + (cfd->current()->storage_info()->NumLevelFiles(0) >= + mutable_cf_options.level0_slowdown_writes_trigger); + bool triggered_writes_stop = + (cfd->current()->storage_info()->NumLevelFiles(0) >= + mutable_cf_options.level0_stop_writes_trigger); + // release lock while notifying events + mutex_.Unlock(); + { + FlushJobInfo info; + info.cf_name = cfd->GetName(); + // TODO(yhchiang): make db_paths dynamic in case flush does not + // go to L0 in the future. + info.file_path = MakeTableFileName(immutable_db_options_.db_paths[0].path, + file_meta->fd.GetNumber()); + info.thread_id = env_->GetThreadID(); + info.job_id = job_id; + info.triggered_writes_slowdown = triggered_writes_slowdown; + info.triggered_writes_stop = triggered_writes_stop; + info.smallest_seqno = file_meta->smallest_seqno; + info.largest_seqno = file_meta->largest_seqno; + info.table_properties = prop; + for (auto listener : immutable_db_options_.listeners) { + listener->OnFlushBegin(this, info); + } + } + mutex_.Lock(); +// no need to signal bg_cv_ as it will be signaled at the end of the +// flush process. +#endif // ROCKSDB_LITE +} + +void DBImpl::NotifyOnFlushCompleted(ColumnFamilyData* cfd, + FileMetaData* file_meta, + const MutableCFOptions& mutable_cf_options, + int job_id, TableProperties prop) { +#ifndef ROCKSDB_LITE + if (immutable_db_options_.listeners.size() == 0U) { + return; + } + mutex_.AssertHeld(); + if (shutting_down_.load(std::memory_order_acquire)) { + return; + } + bool triggered_writes_slowdown = + (cfd->current()->storage_info()->NumLevelFiles(0) >= + mutable_cf_options.level0_slowdown_writes_trigger); + bool triggered_writes_stop = + (cfd->current()->storage_info()->NumLevelFiles(0) >= + mutable_cf_options.level0_stop_writes_trigger); + // release lock while notifying events + mutex_.Unlock(); + { + FlushJobInfo info; + info.cf_name = cfd->GetName(); + // TODO(yhchiang): make db_paths dynamic in case flush does not + // go to L0 in the future. + info.file_path = MakeTableFileName(immutable_db_options_.db_paths[0].path, + file_meta->fd.GetNumber()); + info.thread_id = env_->GetThreadID(); + info.job_id = job_id; + info.triggered_writes_slowdown = triggered_writes_slowdown; + info.triggered_writes_stop = triggered_writes_stop; + info.smallest_seqno = file_meta->smallest_seqno; + info.largest_seqno = file_meta->largest_seqno; + info.table_properties = prop; + for (auto listener : immutable_db_options_.listeners) { + listener->OnFlushCompleted(this, info); + } + } + mutex_.Lock(); + // no need to signal bg_cv_ as it will be signaled at the end of the + // flush process. +#endif // ROCKSDB_LITE +} + +Status DBImpl::CompactRange(const CompactRangeOptions& options, + ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) { + if (options.target_path_id >= immutable_db_options_.db_paths.size()) { + return Status::InvalidArgument("Invalid target path ID"); + } + + auto cfh = reinterpret_cast(column_family); + auto cfd = cfh->cfd(); + bool exclusive = options.exclusive_manual_compaction; + + Status s = FlushMemTable(cfd, FlushOptions()); + if (!s.ok()) { + LogFlush(immutable_db_options_.info_log); + return s; + } + + int max_level_with_files = 0; + { + InstrumentedMutexLock l(&mutex_); + Version* base = cfd->current(); + for (int level = 1; level < base->storage_info()->num_non_empty_levels(); + level++) { + if (base->storage_info()->OverlapInLevel(level, begin, end)) { + max_level_with_files = level; + } + } + } + + int final_output_level = 0; + if (cfd->ioptions()->compaction_style == kCompactionStyleUniversal && + cfd->NumberLevels() > 1) { + // Always compact all files together. + final_output_level = cfd->NumberLevels() - 1; + // if bottom most level is reserved + if (immutable_db_options_.allow_ingest_behind) { + final_output_level--; + } + s = RunManualCompaction(cfd, ColumnFamilyData::kCompactAllLevels, + final_output_level, options.target_path_id, + begin, end, exclusive); + } else { + for (int level = 0; level <= max_level_with_files; level++) { + int output_level; + // in case the compaction is universal or if we're compacting the + // bottom-most level, the output level will be the same as input one. + // level 0 can never be the bottommost level (i.e. if all files are in + // level 0, we will compact to level 1) + if (cfd->ioptions()->compaction_style == kCompactionStyleUniversal || + cfd->ioptions()->compaction_style == kCompactionStyleFIFO) { + output_level = level; + } else if (level == max_level_with_files && level > 0) { + if (options.bottommost_level_compaction == + BottommostLevelCompaction::kSkip) { + // Skip bottommost level compaction + continue; + } else if (options.bottommost_level_compaction == + BottommostLevelCompaction::kIfHaveCompactionFilter && + cfd->ioptions()->compaction_filter == nullptr && + cfd->ioptions()->compaction_filter_factory == nullptr) { + // Skip bottommost level compaction since we don't have a compaction + // filter + continue; + } + output_level = level; + } else { + output_level = level + 1; + if (cfd->ioptions()->compaction_style == kCompactionStyleLevel && + cfd->ioptions()->level_compaction_dynamic_level_bytes && + level == 0) { + output_level = ColumnFamilyData::kCompactToBaseLevel; + } + } + s = RunManualCompaction(cfd, level, output_level, options.target_path_id, + begin, end, exclusive); + if (!s.ok()) { + break; + } + if (output_level == ColumnFamilyData::kCompactToBaseLevel) { + final_output_level = cfd->NumberLevels() - 1; + } else if (output_level > final_output_level) { + final_output_level = output_level; + } + TEST_SYNC_POINT("DBImpl::RunManualCompaction()::1"); + TEST_SYNC_POINT("DBImpl::RunManualCompaction()::2"); + } + } + if (!s.ok()) { + LogFlush(immutable_db_options_.info_log); + return s; + } + + if (options.change_level) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "[RefitLevel] waiting for background threads to stop"); + s = PauseBackgroundWork(); + if (s.ok()) { + s = ReFitLevel(cfd, final_output_level, options.target_level); + } + ContinueBackgroundWork(); + } + LogFlush(immutable_db_options_.info_log); + + { + InstrumentedMutexLock l(&mutex_); + // an automatic compaction that has been scheduled might have been + // preempted by the manual compactions. Need to schedule it back. + MaybeScheduleFlushOrCompaction(); + } + + return s; +} + +Status DBImpl::CompactFiles( + const CompactionOptions& compact_options, + ColumnFamilyHandle* column_family, + const std::vector& input_file_names, + const int output_level, const int output_path_id) { +#ifdef ROCKSDB_LITE + // not supported in lite version + return Status::NotSupported("Not supported in ROCKSDB LITE"); +#else + if (column_family == nullptr) { + return Status::InvalidArgument("ColumnFamilyHandle must be non-null."); + } + + auto cfd = reinterpret_cast(column_family)->cfd(); + assert(cfd); + + Status s; + JobContext job_context(0, true); + LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, + immutable_db_options_.info_log.get()); + + // Perform CompactFiles + SuperVersion* sv = cfd->GetReferencedSuperVersion(&mutex_); + { + InstrumentedMutexLock l(&mutex_); + + // This call will unlock/lock the mutex to wait for current running + // IngestExternalFile() calls to finish. + WaitForIngestFile(); + + s = CompactFilesImpl(compact_options, cfd, sv->current, + input_file_names, output_level, + output_path_id, &job_context, &log_buffer); + } + if (sv->Unref()) { + mutex_.Lock(); + sv->Cleanup(); + mutex_.Unlock(); + delete sv; + } + + // Find and delete obsolete files + { + InstrumentedMutexLock l(&mutex_); + // If !s.ok(), this means that Compaction failed. In that case, we want + // to delete all obsolete files we might have created and we force + // FindObsoleteFiles(). This is because job_context does not + // catch all created files if compaction failed. + FindObsoleteFiles(&job_context, !s.ok()); + } // release the mutex + + // delete unnecessary files if any, this is done outside the mutex + if (job_context.HaveSomethingToDelete() || !log_buffer.IsEmpty()) { + // Have to flush the info logs before bg_compaction_scheduled_-- + // because if bg_flush_scheduled_ becomes 0 and the lock is + // released, the deconstructor of DB can kick in and destroy all the + // states of DB so info_log might not be available after that point. + // It also applies to access other states that DB owns. + log_buffer.FlushBufferToLog(); + if (job_context.HaveSomethingToDelete()) { + // no mutex is locked here. No need to Unlock() and Lock() here. + PurgeObsoleteFiles(job_context); + } + job_context.Clean(); + } + + return s; +#endif // ROCKSDB_LITE +} + +#ifndef ROCKSDB_LITE +Status DBImpl::CompactFilesImpl( + const CompactionOptions& compact_options, ColumnFamilyData* cfd, + Version* version, const std::vector& input_file_names, + const int output_level, int output_path_id, JobContext* job_context, + LogBuffer* log_buffer) { + mutex_.AssertHeld(); + + if (shutting_down_.load(std::memory_order_acquire)) { + return Status::ShutdownInProgress(); + } + + std::unordered_set input_set; + for (auto file_name : input_file_names) { + input_set.insert(TableFileNameToNumber(file_name)); + } + + ColumnFamilyMetaData cf_meta; + // TODO(yhchiang): can directly use version here if none of the + // following functions call is pluggable to external developers. + version->GetColumnFamilyMetaData(&cf_meta); + + if (output_path_id < 0) { + if (immutable_db_options_.db_paths.size() == 1U) { + output_path_id = 0; + } else { + return Status::NotSupported( + "Automatic output path selection is not " + "yet supported in CompactFiles()"); + } + } + + Status s = cfd->compaction_picker()->SanitizeCompactionInputFiles( + &input_set, cf_meta, output_level); + if (!s.ok()) { + return s; + } + + std::vector input_files; + s = cfd->compaction_picker()->GetCompactionInputsFromFileNumbers( + &input_files, &input_set, version->storage_info(), compact_options); + if (!s.ok()) { + return s; + } + + for (auto inputs : input_files) { + if (cfd->compaction_picker()->AreFilesInCompaction(inputs.files)) { + return Status::Aborted( + "Some of the necessary compaction input " + "files are already being compacted"); + } + } + + // At this point, CompactFiles will be run. + bg_compaction_scheduled_++; + + unique_ptr c; + assert(cfd->compaction_picker()); + c.reset(cfd->compaction_picker()->CompactFiles( + compact_options, input_files, output_level, version->storage_info(), + *cfd->GetLatestMutableCFOptions(), output_path_id)); + if (!c) { + return Status::Aborted("Another Level 0 compaction is running"); + } + c->SetInputVersion(version); + // deletion compaction currently not allowed in CompactFiles. + assert(!c->deletion_compaction()); + + SequenceNumber earliest_write_conflict_snapshot; + std::vector snapshot_seqs = + snapshots_.GetAll(&earliest_write_conflict_snapshot); + + auto pending_outputs_inserted_elem = + CaptureCurrentFileNumberInPendingOutputs(); + + assert(is_snapshot_supported_ || snapshots_.empty()); + CompactionJob compaction_job( + job_context->job_id, c.get(), immutable_db_options_, env_options_, + versions_.get(), &shutting_down_, log_buffer, directories_.GetDbDir(), + directories_.GetDataDir(c->output_path_id()), stats_, &mutex_, &bg_error_, + snapshot_seqs, earliest_write_conflict_snapshot, table_cache_, + &event_logger_, c->mutable_cf_options()->paranoid_file_checks, + c->mutable_cf_options()->report_bg_io_stats, dbname_, + nullptr); // Here we pass a nullptr for CompactionJobStats because + // CompactFiles does not trigger OnCompactionCompleted(), + // which is the only place where CompactionJobStats is + // returned. The idea of not triggering OnCompationCompleted() + // is that CompactFiles runs in the caller thread, so the user + // should always know when it completes. As a result, it makes + // less sense to notify the users something they should already + // know. + // + // In the future, if we would like to add CompactionJobStats + // support for CompactFiles, we should have CompactFiles API + // pass a pointer of CompactionJobStats as the out-value + // instead of using EventListener. + + // Creating a compaction influences the compaction score because the score + // takes running compactions into account (by skipping files that are already + // being compacted). Since we just changed compaction score, we recalculate it + // here. + version->storage_info()->ComputeCompactionScore(*cfd->ioptions(), + *c->mutable_cf_options()); + + compaction_job.Prepare(); + + mutex_.Unlock(); + TEST_SYNC_POINT("CompactFilesImpl:0"); + TEST_SYNC_POINT("CompactFilesImpl:1"); + compaction_job.Run(); + TEST_SYNC_POINT("CompactFilesImpl:2"); + TEST_SYNC_POINT("CompactFilesImpl:3"); + mutex_.Lock(); + + Status status = compaction_job.Install(*c->mutable_cf_options()); + if (status.ok()) { + InstallSuperVersionAndScheduleWorkWrapper( + c->column_family_data(), job_context, *c->mutable_cf_options()); + } + c->ReleaseCompactionFiles(s); + + ReleaseFileNumberFromPendingOutputs(pending_outputs_inserted_elem); + + if (status.ok()) { + // Done + } else if (status.IsShutdownInProgress()) { + // Ignore compaction errors found during shutting down + } else { + ROCKS_LOG_WARN(immutable_db_options_.info_log, + "[%s] [JOB %d] Compaction error: %s", + c->column_family_data()->GetName().c_str(), + job_context->job_id, status.ToString().c_str()); + if (immutable_db_options_.paranoid_checks && bg_error_.ok()) { + Status new_bg_error = status; + // may temporarily unlock and lock the mutex. + EventHelpers::NotifyOnBackgroundError(immutable_db_options_.listeners, + BackgroundErrorReason::kCompaction, + &new_bg_error, &mutex_); + if (!new_bg_error.ok()) { + bg_error_ = new_bg_error; + } + } + } + + c.reset(); + + bg_compaction_scheduled_--; + if (bg_compaction_scheduled_ == 0) { + bg_cv_.SignalAll(); + } + + return status; +} +#endif // ROCKSDB_LITE + +Status DBImpl::PauseBackgroundWork() { + InstrumentedMutexLock guard_lock(&mutex_); + bg_compaction_paused_++; + while (bg_bottom_compaction_scheduled_ > 0 || bg_compaction_scheduled_ > 0 || + bg_flush_scheduled_ > 0) { + bg_cv_.Wait(); + } + bg_work_paused_++; + return Status::OK(); +} + +Status DBImpl::ContinueBackgroundWork() { + InstrumentedMutexLock guard_lock(&mutex_); + if (bg_work_paused_ == 0) { + return Status::InvalidArgument(); + } + assert(bg_work_paused_ > 0); + assert(bg_compaction_paused_ > 0); + bg_compaction_paused_--; + bg_work_paused_--; + // It's sufficient to check just bg_work_paused_ here since + // bg_work_paused_ is always no greater than bg_compaction_paused_ + if (bg_work_paused_ == 0) { + MaybeScheduleFlushOrCompaction(); + } + return Status::OK(); +} + +void DBImpl::NotifyOnCompactionCompleted( + ColumnFamilyData* cfd, Compaction *c, const Status &st, + const CompactionJobStats& compaction_job_stats, + const int job_id) { +#ifndef ROCKSDB_LITE + if (immutable_db_options_.listeners.size() == 0U) { + return; + } + mutex_.AssertHeld(); + if (shutting_down_.load(std::memory_order_acquire)) { + return; + } + // release lock while notifying events + mutex_.Unlock(); + TEST_SYNC_POINT("DBImpl::NotifyOnCompactionCompleted::UnlockMutex"); + { + CompactionJobInfo info; + info.cf_name = cfd->GetName(); + info.status = st; + info.thread_id = env_->GetThreadID(); + info.job_id = job_id; + info.base_input_level = c->start_level(); + info.output_level = c->output_level(); + info.stats = compaction_job_stats; + info.table_properties = c->GetOutputTableProperties(); + info.compaction_reason = c->compaction_reason(); + info.compression = c->output_compression(); + for (size_t i = 0; i < c->num_input_levels(); ++i) { + for (const auto fmd : *c->inputs(i)) { + auto fn = TableFileName(immutable_db_options_.db_paths, + fmd->fd.GetNumber(), fmd->fd.GetPathId()); + info.input_files.push_back(fn); + if (info.table_properties.count(fn) == 0) { + std::shared_ptr tp; + auto s = cfd->current()->GetTableProperties(&tp, fmd, &fn); + if (s.ok()) { + info.table_properties[fn] = tp; + } + } + } + } + for (const auto newf : c->edit()->GetNewFiles()) { + info.output_files.push_back(TableFileName(immutable_db_options_.db_paths, + newf.second.fd.GetNumber(), + newf.second.fd.GetPathId())); + } + for (auto listener : immutable_db_options_.listeners) { + listener->OnCompactionCompleted(this, info); + } + } + mutex_.Lock(); + // no need to signal bg_cv_ as it will be signaled at the end of the + // flush process. +#endif // ROCKSDB_LITE +} + +// REQUIREMENT: block all background work by calling PauseBackgroundWork() +// before calling this function +Status DBImpl::ReFitLevel(ColumnFamilyData* cfd, int level, int target_level) { + assert(level < cfd->NumberLevels()); + if (target_level >= cfd->NumberLevels()) { + return Status::InvalidArgument("Target level exceeds number of levels"); + } + + std::unique_ptr superversion_to_free; + std::unique_ptr new_superversion(new SuperVersion()); + + Status status; + + InstrumentedMutexLock guard_lock(&mutex_); + + // only allow one thread refitting + if (refitting_level_) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "[ReFitLevel] another thread is refitting"); + return Status::NotSupported("another thread is refitting"); + } + refitting_level_ = true; + + const MutableCFOptions mutable_cf_options = *cfd->GetLatestMutableCFOptions(); + // move to a smaller level + int to_level = target_level; + if (target_level < 0) { + to_level = FindMinimumEmptyLevelFitting(cfd, mutable_cf_options, level); + } + + auto* vstorage = cfd->current()->storage_info(); + if (to_level > level) { + if (level == 0) { + return Status::NotSupported( + "Cannot change from level 0 to other levels."); + } + // Check levels are empty for a trivial move + for (int l = level + 1; l <= to_level; l++) { + if (vstorage->NumLevelFiles(l) > 0) { + return Status::NotSupported( + "Levels between source and target are not empty for a move."); + } + } + } + if (to_level != level) { + ROCKS_LOG_DEBUG(immutable_db_options_.info_log, + "[%s] Before refitting:\n%s", cfd->GetName().c_str(), + cfd->current()->DebugString().data()); + + VersionEdit edit; + edit.SetColumnFamily(cfd->GetID()); + for (const auto& f : vstorage->LevelFiles(level)) { + edit.DeleteFile(level, f->fd.GetNumber()); + edit.AddFile(to_level, f->fd.GetNumber(), f->fd.GetPathId(), + f->fd.GetFileSize(), f->smallest, f->largest, + f->smallest_seqno, f->largest_seqno, + f->marked_for_compaction); + } + ROCKS_LOG_DEBUG(immutable_db_options_.info_log, + "[%s] Apply version edit:\n%s", cfd->GetName().c_str(), + edit.DebugString().data()); + + status = versions_->LogAndApply(cfd, mutable_cf_options, &edit, &mutex_, + directories_.GetDbDir()); + superversion_to_free.reset(InstallSuperVersionAndScheduleWork( + cfd, new_superversion.release(), mutable_cf_options)); + + ROCKS_LOG_DEBUG(immutable_db_options_.info_log, "[%s] LogAndApply: %s\n", + cfd->GetName().c_str(), status.ToString().data()); + + if (status.ok()) { + ROCKS_LOG_DEBUG(immutable_db_options_.info_log, + "[%s] After refitting:\n%s", cfd->GetName().c_str(), + cfd->current()->DebugString().data()); + } + } + + refitting_level_ = false; + + return status; +} + +int DBImpl::NumberLevels(ColumnFamilyHandle* column_family) { + auto cfh = reinterpret_cast(column_family); + return cfh->cfd()->NumberLevels(); +} + +int DBImpl::MaxMemCompactionLevel(ColumnFamilyHandle* column_family) { + return 0; +} + +int DBImpl::Level0StopWriteTrigger(ColumnFamilyHandle* column_family) { + auto cfh = reinterpret_cast(column_family); + InstrumentedMutexLock l(&mutex_); + return cfh->cfd()->GetSuperVersion()-> + mutable_cf_options.level0_stop_writes_trigger; +} + +Status DBImpl::Flush(const FlushOptions& flush_options, + ColumnFamilyHandle* column_family) { + auto cfh = reinterpret_cast(column_family); + return FlushMemTable(cfh->cfd(), flush_options); +} + +Status DBImpl::RunManualCompaction(ColumnFamilyData* cfd, int input_level, + int output_level, uint32_t output_path_id, + const Slice* begin, const Slice* end, + bool exclusive, bool disallow_trivial_move) { + assert(input_level == ColumnFamilyData::kCompactAllLevels || + input_level >= 0); + + InternalKey begin_storage, end_storage; + CompactionArg* ca; + + bool scheduled = false; + bool manual_conflict = false; + ManualCompactionState manual; + manual.cfd = cfd; + manual.input_level = input_level; + manual.output_level = output_level; + manual.output_path_id = output_path_id; + manual.done = false; + manual.in_progress = false; + manual.incomplete = false; + manual.exclusive = exclusive; + manual.disallow_trivial_move = disallow_trivial_move; + // For universal compaction, we enforce every manual compaction to compact + // all files. + if (begin == nullptr || + cfd->ioptions()->compaction_style == kCompactionStyleUniversal || + cfd->ioptions()->compaction_style == kCompactionStyleFIFO) { + manual.begin = nullptr; + } else { + begin_storage.SetMaxPossibleForUserKey(*begin); + manual.begin = &begin_storage; + } + if (end == nullptr || + cfd->ioptions()->compaction_style == kCompactionStyleUniversal || + cfd->ioptions()->compaction_style == kCompactionStyleFIFO) { + manual.end = nullptr; + } else { + end_storage.SetMinPossibleForUserKey(*end); + manual.end = &end_storage; + } + + TEST_SYNC_POINT("DBImpl::RunManualCompaction:0"); + TEST_SYNC_POINT("DBImpl::RunManualCompaction:1"); + InstrumentedMutexLock l(&mutex_); + + // When a manual compaction arrives, temporarily disable scheduling of + // non-manual compactions and wait until the number of scheduled compaction + // jobs drops to zero. This is needed to ensure that this manual compaction + // can compact any range of keys/files. + // + // HasPendingManualCompaction() is true when at least one thread is inside + // RunManualCompaction(), i.e. during that time no other compaction will + // get scheduled (see MaybeScheduleFlushOrCompaction). + // + // Note that the following loop doesn't stop more that one thread calling + // RunManualCompaction() from getting to the second while loop below. + // However, only one of them will actually schedule compaction, while + // others will wait on a condition variable until it completes. + + AddManualCompaction(&manual); + TEST_SYNC_POINT_CALLBACK("DBImpl::RunManualCompaction:NotScheduled", &mutex_); + if (exclusive) { + while (bg_bottom_compaction_scheduled_ > 0 || + bg_compaction_scheduled_ > 0) { + TEST_SYNC_POINT("DBImpl::RunManualCompaction:WaitScheduled"); + ROCKS_LOG_INFO( + immutable_db_options_.info_log, + "[%s] Manual compaction waiting for all other scheduled background " + "compactions to finish", + cfd->GetName().c_str()); + bg_cv_.Wait(); + } + } + + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "[%s] Manual compaction starting", cfd->GetName().c_str()); + + // We don't check bg_error_ here, because if we get the error in compaction, + // the compaction will set manual.status to bg_error_ and set manual.done to + // true. + while (!manual.done) { + assert(HasPendingManualCompaction()); + manual_conflict = false; + Compaction* compaction; + if (ShouldntRunManualCompaction(&manual) || (manual.in_progress == true) || + scheduled || + ((manual.manual_end = &manual.tmp_storage1) && + ((compaction = manual.cfd->CompactRange( + *manual.cfd->GetLatestMutableCFOptions(), manual.input_level, + manual.output_level, manual.output_path_id, manual.begin, + manual.end, &manual.manual_end, &manual_conflict)) == nullptr) && + manual_conflict)) { + // exclusive manual compactions should not see a conflict during + // CompactRange + assert(!exclusive || !manual_conflict); + // Running either this or some other manual compaction + bg_cv_.Wait(); + if (scheduled && manual.incomplete == true) { + assert(!manual.in_progress); + scheduled = false; + manual.incomplete = false; + } + } else if (!scheduled) { + if (compaction == nullptr) { + manual.done = true; + bg_cv_.SignalAll(); + continue; + } + ca = new CompactionArg; + ca->db = this; + ca->prepicked_compaction = new PrepickedCompaction; + ca->prepicked_compaction->manual_compaction_state = &manual; + ca->prepicked_compaction->compaction = compaction; + manual.incomplete = false; + bg_compaction_scheduled_++; + env_->Schedule(&DBImpl::BGWorkCompaction, ca, Env::Priority::LOW, this, + &DBImpl::UnscheduleCallback); + scheduled = true; + } + } + + assert(!manual.in_progress); + assert(HasPendingManualCompaction()); + RemoveManualCompaction(&manual); + bg_cv_.SignalAll(); + return manual.status; +} + +Status DBImpl::FlushMemTable(ColumnFamilyData* cfd, + const FlushOptions& flush_options, + bool writes_stopped) { + Status s; + { + WriteContext context; + InstrumentedMutexLock guard_lock(&mutex_); + + if (cfd->imm()->NumNotFlushed() == 0 && cfd->mem()->IsEmpty()) { + // Nothing to flush + return Status::OK(); + } + + WriteThread::Writer w; + if (!writes_stopped) { + write_thread_.EnterUnbatched(&w, &mutex_); + } + + // SwitchMemtable() will release and reacquire mutex + // during execution + s = SwitchMemtable(cfd, &context); + + if (!writes_stopped) { + write_thread_.ExitUnbatched(&w); + } + + cfd->imm()->FlushRequested(); + + // schedule flush + SchedulePendingFlush(cfd); + MaybeScheduleFlushOrCompaction(); + } + + if (s.ok() && flush_options.wait) { + // Wait until the compaction completes + s = WaitForFlushMemTable(cfd); + } + return s; +} + +Status DBImpl::WaitForFlushMemTable(ColumnFamilyData* cfd) { + Status s; + // Wait until the compaction completes + InstrumentedMutexLock l(&mutex_); + while (cfd->imm()->NumNotFlushed() > 0 && bg_error_.ok()) { + if (shutting_down_.load(std::memory_order_acquire)) { + return Status::ShutdownInProgress(); + } + if (cfd->IsDropped()) { + // FlushJob cannot flush a dropped CF, if we did not break here + // we will loop forever since cfd->imm()->NumNotFlushed() will never + // drop to zero + return Status::InvalidArgument("Cannot flush a dropped CF"); + } + bg_cv_.Wait(); + } + if (!bg_error_.ok()) { + s = bg_error_; + } + return s; +} + +Status DBImpl::EnableAutoCompaction( + const std::vector& column_family_handles) { + Status s; + for (auto cf_ptr : column_family_handles) { + Status status = + this->SetOptions(cf_ptr, {{"disable_auto_compactions", "false"}}); + if (!status.ok()) { + s = status; + } + } + + return s; +} + +void DBImpl::MaybeScheduleFlushOrCompaction() { + mutex_.AssertHeld(); + if (!opened_successfully_) { + // Compaction may introduce data race to DB open + return; + } + if (bg_work_paused_ > 0) { + // we paused the background work + return; + } else if (shutting_down_.load(std::memory_order_acquire)) { + // DB is being deleted; no more background compactions + return; + } + auto bg_job_limits = GetBGJobLimits(); + bool is_flush_pool_empty = + env_->GetBackgroundThreads(Env::Priority::HIGH) == 0; + while (!is_flush_pool_empty && unscheduled_flushes_ > 0 && + bg_flush_scheduled_ < bg_job_limits.max_flushes) { + unscheduled_flushes_--; + bg_flush_scheduled_++; + env_->Schedule(&DBImpl::BGWorkFlush, this, Env::Priority::HIGH, this); + } + + // special case -- if high-pri (flush) thread pool is empty, then schedule + // flushes in low-pri (compaction) thread pool. + if (is_flush_pool_empty) { + while (unscheduled_flushes_ > 0 && + bg_flush_scheduled_ + bg_compaction_scheduled_ < + bg_job_limits.max_flushes) { + unscheduled_flushes_--; + bg_flush_scheduled_++; + env_->Schedule(&DBImpl::BGWorkFlush, this, Env::Priority::LOW, this); + } + } + + if (bg_compaction_paused_ > 0) { + // we paused the background compaction + return; + } + + if (HasExclusiveManualCompaction()) { + // only manual compactions are allowed to run. don't schedule automatic + // compactions + return; + } + + while (bg_compaction_scheduled_ < bg_job_limits.max_compactions && + unscheduled_compactions_ > 0) { + CompactionArg* ca = new CompactionArg; + ca->db = this; + ca->prepicked_compaction = nullptr; + bg_compaction_scheduled_++; + unscheduled_compactions_--; + env_->Schedule(&DBImpl::BGWorkCompaction, ca, Env::Priority::LOW, this, + &DBImpl::UnscheduleCallback); + } +} + +DBImpl::BGJobLimits DBImpl::GetBGJobLimits() const { + mutex_.AssertHeld(); + return GetBGJobLimits(immutable_db_options_.max_background_flushes, + mutable_db_options_.max_background_compactions, + mutable_db_options_.max_background_jobs, + write_controller_.NeedSpeedupCompaction()); +} + +DBImpl::BGJobLimits DBImpl::GetBGJobLimits(int max_background_flushes, + int max_background_compactions, + int max_background_jobs, + bool parallelize_compactions) { + BGJobLimits res; + if (max_background_flushes == -1 && max_background_compactions == -1) { + // for our first stab implementing max_background_jobs, simply allocate a + // quarter of the threads to flushes. + res.max_flushes = std::max(1, max_background_jobs / 4); + res.max_compactions = std::max(1, max_background_jobs - res.max_flushes); + } else { + // compatibility code in case users haven't migrated to max_background_jobs, + // which automatically computes flush/compaction limits + res.max_flushes = std::max(1, max_background_flushes); + res.max_compactions = std::max(1, max_background_compactions); + } + if (!parallelize_compactions) { + // throttle background compactions until we deem necessary + res.max_compactions = 1; + } + return res; +} + +void DBImpl::AddToCompactionQueue(ColumnFamilyData* cfd) { + assert(!cfd->pending_compaction()); + cfd->Ref(); + compaction_queue_.push_back(cfd); + cfd->set_pending_compaction(true); +} + +ColumnFamilyData* DBImpl::PopFirstFromCompactionQueue() { + assert(!compaction_queue_.empty()); + auto cfd = *compaction_queue_.begin(); + compaction_queue_.pop_front(); + assert(cfd->pending_compaction()); + cfd->set_pending_compaction(false); + return cfd; +} + +void DBImpl::AddToFlushQueue(ColumnFamilyData* cfd) { + assert(!cfd->pending_flush()); + cfd->Ref(); + flush_queue_.push_back(cfd); + cfd->set_pending_flush(true); +} + +ColumnFamilyData* DBImpl::PopFirstFromFlushQueue() { + assert(!flush_queue_.empty()); + auto cfd = *flush_queue_.begin(); + flush_queue_.pop_front(); + assert(cfd->pending_flush()); + cfd->set_pending_flush(false); + return cfd; +} + +void DBImpl::SchedulePendingFlush(ColumnFamilyData* cfd) { + if (!cfd->pending_flush() && cfd->imm()->IsFlushPending()) { + AddToFlushQueue(cfd); + ++unscheduled_flushes_; + } +} + +void DBImpl::SchedulePendingCompaction(ColumnFamilyData* cfd) { + if (!cfd->pending_compaction() && cfd->NeedsCompaction()) { + AddToCompactionQueue(cfd); + ++unscheduled_compactions_; + } +} + +void DBImpl::SchedulePendingPurge(std::string fname, FileType type, + uint64_t number, uint32_t path_id, + int job_id) { + mutex_.AssertHeld(); + PurgeFileInfo file_info(fname, type, number, path_id, job_id); + purge_queue_.push_back(std::move(file_info)); +} + +void DBImpl::BGWorkFlush(void* db) { + IOSTATS_SET_THREAD_POOL_ID(Env::Priority::HIGH); + TEST_SYNC_POINT("DBImpl::BGWorkFlush"); + reinterpret_cast(db)->BackgroundCallFlush(); + TEST_SYNC_POINT("DBImpl::BGWorkFlush:done"); +} + +void DBImpl::BGWorkCompaction(void* arg) { + CompactionArg ca = *(reinterpret_cast(arg)); + delete reinterpret_cast(arg); + IOSTATS_SET_THREAD_POOL_ID(Env::Priority::LOW); + TEST_SYNC_POINT("DBImpl::BGWorkCompaction"); + auto prepicked_compaction = + static_cast(ca.prepicked_compaction); + reinterpret_cast(ca.db)->BackgroundCallCompaction( + prepicked_compaction, Env::Priority::LOW); + delete prepicked_compaction; +} + +void DBImpl::BGWorkBottomCompaction(void* arg) { + CompactionArg ca = *(static_cast(arg)); + delete static_cast(arg); + IOSTATS_SET_THREAD_POOL_ID(Env::Priority::BOTTOM); + TEST_SYNC_POINT("DBImpl::BGWorkBottomCompaction"); + auto* prepicked_compaction = ca.prepicked_compaction; + assert(prepicked_compaction && prepicked_compaction->compaction && + !prepicked_compaction->manual_compaction_state); + ca.db->BackgroundCallCompaction(prepicked_compaction, Env::Priority::BOTTOM); + delete prepicked_compaction; +} + +void DBImpl::BGWorkPurge(void* db) { + IOSTATS_SET_THREAD_POOL_ID(Env::Priority::HIGH); + TEST_SYNC_POINT("DBImpl::BGWorkPurge:start"); + reinterpret_cast(db)->BackgroundCallPurge(); + TEST_SYNC_POINT("DBImpl::BGWorkPurge:end"); +} + +void DBImpl::UnscheduleCallback(void* arg) { + CompactionArg ca = *(reinterpret_cast(arg)); + delete reinterpret_cast(arg); + if (ca.prepicked_compaction != nullptr) { + if (ca.prepicked_compaction->compaction != nullptr) { + delete ca.prepicked_compaction->compaction; + } + delete ca.prepicked_compaction; + } + TEST_SYNC_POINT("DBImpl::UnscheduleCallback"); +} + +Status DBImpl::BackgroundFlush(bool* made_progress, JobContext* job_context, + LogBuffer* log_buffer) { + mutex_.AssertHeld(); + + Status status = bg_error_; + if (status.ok() && shutting_down_.load(std::memory_order_acquire)) { + status = Status::ShutdownInProgress(); + } + + if (!status.ok()) { + return status; + } + + ColumnFamilyData* cfd = nullptr; + while (!flush_queue_.empty()) { + // This cfd is already referenced + auto first_cfd = PopFirstFromFlushQueue(); + + if (first_cfd->IsDropped() || !first_cfd->imm()->IsFlushPending()) { + // can't flush this CF, try next one + if (first_cfd->Unref()) { + delete first_cfd; + } + continue; + } + + // found a flush! + cfd = first_cfd; + break; + } + + if (cfd != nullptr) { + const MutableCFOptions mutable_cf_options = + *cfd->GetLatestMutableCFOptions(); + auto bg_job_limits = GetBGJobLimits(); + ROCKS_LOG_BUFFER( + log_buffer, + "Calling FlushMemTableToOutputFile with column " + "family [%s], flush slots available %d, compaction slots available %d, " + "flush slots scheduled %d, compaction slots scheduled %d", + cfd->GetName().c_str(), bg_job_limits.max_flushes, + bg_job_limits.max_compactions, bg_flush_scheduled_, + bg_compaction_scheduled_); + status = FlushMemTableToOutputFile(cfd, mutable_cf_options, made_progress, + job_context, log_buffer); + if (cfd->Unref()) { + delete cfd; + } + } + return status; +} + +void DBImpl::BackgroundCallFlush() { + bool made_progress = false; + JobContext job_context(next_job_id_.fetch_add(1), true); + + TEST_SYNC_POINT("DBImpl::BackgroundCallFlush:start"); + + LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, + immutable_db_options_.info_log.get()); + { + InstrumentedMutexLock l(&mutex_); + assert(bg_flush_scheduled_); + num_running_flushes_++; + + auto pending_outputs_inserted_elem = + CaptureCurrentFileNumberInPendingOutputs(); + + Status s = BackgroundFlush(&made_progress, &job_context, &log_buffer); + if (!s.ok() && !s.IsShutdownInProgress()) { + // Wait a little bit before retrying background flush in + // case this is an environmental problem and we do not want to + // chew up resources for failed flushes for the duration of + // the problem. + uint64_t error_cnt = + default_cf_internal_stats_->BumpAndGetBackgroundErrorCount(); + bg_cv_.SignalAll(); // In case a waiter can proceed despite the error + mutex_.Unlock(); + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "Waiting after background flush error: %s" + "Accumulated background error counts: %" PRIu64, + s.ToString().c_str(), error_cnt); + log_buffer.FlushBufferToLog(); + LogFlush(immutable_db_options_.info_log); + env_->SleepForMicroseconds(1000000); + mutex_.Lock(); + } + + ReleaseFileNumberFromPendingOutputs(pending_outputs_inserted_elem); + + // If flush failed, we want to delete all temporary files that we might have + // created. Thus, we force full scan in FindObsoleteFiles() + FindObsoleteFiles(&job_context, !s.ok() && !s.IsShutdownInProgress()); + // delete unnecessary files if any, this is done outside the mutex + if (job_context.HaveSomethingToDelete() || !log_buffer.IsEmpty()) { + mutex_.Unlock(); + // Have to flush the info logs before bg_flush_scheduled_-- + // because if bg_flush_scheduled_ becomes 0 and the lock is + // released, the deconstructor of DB can kick in and destroy all the + // states of DB so info_log might not be available after that point. + // It also applies to access other states that DB owns. + log_buffer.FlushBufferToLog(); + if (job_context.HaveSomethingToDelete()) { + PurgeObsoleteFiles(job_context); + } + job_context.Clean(); + mutex_.Lock(); + } + + assert(num_running_flushes_ > 0); + num_running_flushes_--; + bg_flush_scheduled_--; + // See if there's more work to be done + MaybeScheduleFlushOrCompaction(); + bg_cv_.SignalAll(); + // IMPORTANT: there should be no code after calling SignalAll. This call may + // signal the DB destructor that it's OK to proceed with destruction. In + // that case, all DB variables will be dealloacated and referencing them + // will cause trouble. + } +} + +void DBImpl::BackgroundCallCompaction(PrepickedCompaction* prepicked_compaction, + Env::Priority bg_thread_pri) { + bool made_progress = false; + JobContext job_context(next_job_id_.fetch_add(1), true); + TEST_SYNC_POINT("BackgroundCallCompaction:0"); + MaybeDumpStats(); + LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, + immutable_db_options_.info_log.get()); + { + InstrumentedMutexLock l(&mutex_); + + // This call will unlock/lock the mutex to wait for current running + // IngestExternalFile() calls to finish. + WaitForIngestFile(); + + num_running_compactions_++; + + auto pending_outputs_inserted_elem = + CaptureCurrentFileNumberInPendingOutputs(); + + assert((bg_thread_pri == Env::Priority::BOTTOM && + bg_bottom_compaction_scheduled_) || + (bg_thread_pri == Env::Priority::LOW && bg_compaction_scheduled_)); + Status s = BackgroundCompaction(&made_progress, &job_context, &log_buffer, + prepicked_compaction); + TEST_SYNC_POINT("BackgroundCallCompaction:1"); + if (!s.ok() && !s.IsShutdownInProgress()) { + // Wait a little bit before retrying background compaction in + // case this is an environmental problem and we do not want to + // chew up resources for failed compactions for the duration of + // the problem. + uint64_t error_cnt = + default_cf_internal_stats_->BumpAndGetBackgroundErrorCount(); + bg_cv_.SignalAll(); // In case a waiter can proceed despite the error + mutex_.Unlock(); + log_buffer.FlushBufferToLog(); + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "Waiting after background compaction error: %s, " + "Accumulated background error counts: %" PRIu64, + s.ToString().c_str(), error_cnt); + LogFlush(immutable_db_options_.info_log); + env_->SleepForMicroseconds(1000000); + mutex_.Lock(); + } + + ReleaseFileNumberFromPendingOutputs(pending_outputs_inserted_elem); + + // If compaction failed, we want to delete all temporary files that we might + // have created (they might not be all recorded in job_context in case of a + // failure). Thus, we force full scan in FindObsoleteFiles() + FindObsoleteFiles(&job_context, !s.ok() && !s.IsShutdownInProgress()); + + // delete unnecessary files if any, this is done outside the mutex + if (job_context.HaveSomethingToDelete() || !log_buffer.IsEmpty()) { + mutex_.Unlock(); + // Have to flush the info logs before bg_compaction_scheduled_-- + // because if bg_flush_scheduled_ becomes 0 and the lock is + // released, the deconstructor of DB can kick in and destroy all the + // states of DB so info_log might not be available after that point. + // It also applies to access other states that DB owns. + log_buffer.FlushBufferToLog(); + if (job_context.HaveSomethingToDelete()) { + PurgeObsoleteFiles(job_context); + } + job_context.Clean(); + mutex_.Lock(); + } + + assert(num_running_compactions_ > 0); + num_running_compactions_--; + if (bg_thread_pri == Env::Priority::LOW) { + bg_compaction_scheduled_--; + } else { + assert(bg_thread_pri == Env::Priority::BOTTOM); + bg_bottom_compaction_scheduled_--; + } + + versions_->GetColumnFamilySet()->FreeDeadColumnFamilies(); + + // See if there's more work to be done + MaybeScheduleFlushOrCompaction(); + if (made_progress || + (bg_compaction_scheduled_ == 0 && + bg_bottom_compaction_scheduled_ == 0) || + HasPendingManualCompaction()) { + // signal if + // * made_progress -- need to wakeup DelayWrite + // * bg_{bottom,}_compaction_scheduled_ == 0 -- need to wakeup ~DBImpl + // * HasPendingManualCompaction -- need to wakeup RunManualCompaction + // If none of this is true, there is no need to signal since nobody is + // waiting for it + bg_cv_.SignalAll(); + } + // IMPORTANT: there should be no code after calling SignalAll. This call may + // signal the DB destructor that it's OK to proceed with destruction. In + // that case, all DB variables will be dealloacated and referencing them + // will cause trouble. + } +} + +Status DBImpl::BackgroundCompaction(bool* made_progress, + JobContext* job_context, + LogBuffer* log_buffer, + PrepickedCompaction* prepicked_compaction) { + ManualCompactionState* manual_compaction = + prepicked_compaction == nullptr + ? nullptr + : prepicked_compaction->manual_compaction_state; + *made_progress = false; + mutex_.AssertHeld(); + TEST_SYNC_POINT("DBImpl::BackgroundCompaction:Start"); + + bool is_manual = (manual_compaction != nullptr); + unique_ptr c; + if (prepicked_compaction != nullptr && + prepicked_compaction->compaction != nullptr) { + c.reset(prepicked_compaction->compaction); + } + bool is_prepicked = is_manual || c; + + // (manual_compaction->in_progress == false); + bool trivial_move_disallowed = + is_manual && manual_compaction->disallow_trivial_move; + + CompactionJobStats compaction_job_stats; + Status status = bg_error_; + if (status.ok() && shutting_down_.load(std::memory_order_acquire)) { + status = Status::ShutdownInProgress(); + } + + if (!status.ok()) { + if (is_manual) { + manual_compaction->status = status; + manual_compaction->done = true; + manual_compaction->in_progress = false; + manual_compaction = nullptr; + } + return status; + } + + if (is_manual) { + // another thread cannot pick up the same work + manual_compaction->in_progress = true; + } + + // InternalKey manual_end_storage; + // InternalKey* manual_end = &manual_end_storage; + if (is_manual) { + ManualCompactionState* m = manual_compaction; + assert(m->in_progress); + if (!c) { + m->done = true; + m->manual_end = nullptr; + ROCKS_LOG_BUFFER(log_buffer, + "[%s] Manual compaction from level-%d from %s .. " + "%s; nothing to do\n", + m->cfd->GetName().c_str(), m->input_level, + (m->begin ? m->begin->DebugString().c_str() : "(begin)"), + (m->end ? m->end->DebugString().c_str() : "(end)")); + } else { + ROCKS_LOG_BUFFER( + log_buffer, + "[%s] Manual compaction from level-%d to level-%d from %s .. " + "%s; will stop at %s\n", + m->cfd->GetName().c_str(), m->input_level, c->output_level(), + (m->begin ? m->begin->DebugString().c_str() : "(begin)"), + (m->end ? m->end->DebugString().c_str() : "(end)"), + ((m->done || m->manual_end == nullptr) + ? "(end)" + : m->manual_end->DebugString().c_str())); + } + } else if (!is_prepicked && !compaction_queue_.empty()) { + if (HaveManualCompaction(compaction_queue_.front())) { + // Can't compact right now, but try again later + TEST_SYNC_POINT("DBImpl::BackgroundCompaction()::Conflict"); + + // Stay in the compaction queue. + unscheduled_compactions_++; + + return Status::OK(); + } + + // cfd is referenced here + auto cfd = PopFirstFromCompactionQueue(); + // We unreference here because the following code will take a Ref() on + // this cfd if it is going to use it (Compaction class holds a + // reference). + // This will all happen under a mutex so we don't have to be afraid of + // somebody else deleting it. + if (cfd->Unref()) { + delete cfd; + // This was the last reference of the column family, so no need to + // compact. + return Status::OK(); + } + + // Pick up latest mutable CF Options and use it throughout the + // compaction job + // Compaction makes a copy of the latest MutableCFOptions. It should be used + // throughout the compaction procedure to make sure consistency. It will + // eventually be installed into SuperVersion + auto* mutable_cf_options = cfd->GetLatestMutableCFOptions(); + if (!mutable_cf_options->disable_auto_compactions && !cfd->IsDropped()) { + // NOTE: try to avoid unnecessary copy of MutableCFOptions if + // compaction is not necessary. Need to make sure mutex is held + // until we make a copy in the following code + TEST_SYNC_POINT("DBImpl::BackgroundCompaction():BeforePickCompaction"); + c.reset(cfd->PickCompaction(*mutable_cf_options, log_buffer)); + TEST_SYNC_POINT("DBImpl::BackgroundCompaction():AfterPickCompaction"); + if (c != nullptr) { + // update statistics + MeasureTime(stats_, NUM_FILES_IN_SINGLE_COMPACTION, + c->inputs(0)->size()); + // There are three things that can change compaction score: + // 1) When flush or compaction finish. This case is covered by + // InstallSuperVersionAndScheduleWork + // 2) When MutableCFOptions changes. This case is also covered by + // InstallSuperVersionAndScheduleWork, because this is when the new + // options take effect. + // 3) When we Pick a new compaction, we "remove" those files being + // compacted from the calculation, which then influences compaction + // score. Here we check if we need the new compaction even without the + // files that are currently being compacted. If we need another + // compaction, we might be able to execute it in parallel, so we add it + // to the queue and schedule a new thread. + if (cfd->NeedsCompaction()) { + // Yes, we need more compactions! + AddToCompactionQueue(cfd); + ++unscheduled_compactions_; + MaybeScheduleFlushOrCompaction(); + } + } + } + } + + if (!c) { + // Nothing to do + ROCKS_LOG_BUFFER(log_buffer, "Compaction nothing to do"); + } else if (c->deletion_compaction()) { + // TODO(icanadi) Do we want to honor snapshots here? i.e. not delete old + // file if there is alive snapshot pointing to it + assert(c->num_input_files(1) == 0); + assert(c->level() == 0); + assert(c->column_family_data()->ioptions()->compaction_style == + kCompactionStyleFIFO); + + compaction_job_stats.num_input_files = c->num_input_files(0); + + for (const auto& f : *c->inputs(0)) { + c->edit()->DeleteFile(c->level(), f->fd.GetNumber()); + } + status = versions_->LogAndApply(c->column_family_data(), + *c->mutable_cf_options(), c->edit(), + &mutex_, directories_.GetDbDir()); + InstallSuperVersionAndScheduleWorkWrapper( + c->column_family_data(), job_context, *c->mutable_cf_options()); + ROCKS_LOG_BUFFER(log_buffer, "[%s] Deleted %d files\n", + c->column_family_data()->GetName().c_str(), + c->num_input_files(0)); + *made_progress = true; + } else if (!trivial_move_disallowed && c->IsTrivialMove()) { + TEST_SYNC_POINT("DBImpl::BackgroundCompaction:TrivialMove"); + // Instrument for event update + // TODO(yhchiang): add op details for showing trivial-move. + ThreadStatusUtil::SetColumnFamily( + c->column_family_data(), c->column_family_data()->ioptions()->env, + immutable_db_options_.enable_thread_tracking); + ThreadStatusUtil::SetThreadOperation(ThreadStatus::OP_COMPACTION); + + compaction_job_stats.num_input_files = c->num_input_files(0); + + // Move files to next level + int32_t moved_files = 0; + int64_t moved_bytes = 0; + for (unsigned int l = 0; l < c->num_input_levels(); l++) { + if (c->level(l) == c->output_level()) { + continue; + } + for (size_t i = 0; i < c->num_input_files(l); i++) { + FileMetaData* f = c->input(l, i); + c->edit()->DeleteFile(c->level(l), f->fd.GetNumber()); + c->edit()->AddFile(c->output_level(), f->fd.GetNumber(), + f->fd.GetPathId(), f->fd.GetFileSize(), f->smallest, + f->largest, f->smallest_seqno, f->largest_seqno, + f->marked_for_compaction); + + ROCKS_LOG_BUFFER(log_buffer, "[%s] Moving #%" PRIu64 + " to level-%d %" PRIu64 " bytes\n", + c->column_family_data()->GetName().c_str(), + f->fd.GetNumber(), c->output_level(), + f->fd.GetFileSize()); + ++moved_files; + moved_bytes += f->fd.GetFileSize(); + } + } + + status = versions_->LogAndApply(c->column_family_data(), + *c->mutable_cf_options(), c->edit(), + &mutex_, directories_.GetDbDir()); + // Use latest MutableCFOptions + InstallSuperVersionAndScheduleWorkWrapper( + c->column_family_data(), job_context, *c->mutable_cf_options()); + + VersionStorageInfo::LevelSummaryStorage tmp; + c->column_family_data()->internal_stats()->IncBytesMoved(c->output_level(), + moved_bytes); + { + event_logger_.LogToBuffer(log_buffer) + << "job" << job_context->job_id << "event" + << "trivial_move" + << "destination_level" << c->output_level() << "files" << moved_files + << "total_files_size" << moved_bytes; + } + ROCKS_LOG_BUFFER( + log_buffer, + "[%s] Moved #%d files to level-%d %" PRIu64 " bytes %s: %s\n", + c->column_family_data()->GetName().c_str(), moved_files, + c->output_level(), moved_bytes, status.ToString().c_str(), + c->column_family_data()->current()->storage_info()->LevelSummary(&tmp)); + *made_progress = true; + + // Clear Instrument + ThreadStatusUtil::ResetThreadStatus(); + } else if (c->column_family_data()->ioptions()->compaction_style == + kCompactionStyleUniversal && + !is_prepicked && c->output_level() > 0 && + c->output_level() == + c->column_family_data() + ->current() + ->storage_info() + ->MaxOutputLevel( + immutable_db_options_.allow_ingest_behind) && + env_->GetBackgroundThreads(Env::Priority::BOTTOM) > 0) { + // Forward universal compactions involving last level to the bottom pool + // if it exists, such that long-running compactions can't block short- + // lived ones, like L0->L0s. + TEST_SYNC_POINT("DBImpl::BackgroundCompaction:ForwardToBottomPriPool"); + CompactionArg* ca = new CompactionArg; + ca->db = this; + ca->prepicked_compaction = new PrepickedCompaction; + ca->prepicked_compaction->compaction = c.release(); + ca->prepicked_compaction->manual_compaction_state = nullptr; + ++bg_bottom_compaction_scheduled_; + env_->Schedule(&DBImpl::BGWorkBottomCompaction, ca, Env::Priority::BOTTOM, + this, &DBImpl::UnscheduleCallback); + } else { + int output_level __attribute__((unused)) = c->output_level(); + TEST_SYNC_POINT_CALLBACK("DBImpl::BackgroundCompaction:NonTrivial", + &output_level); + + SequenceNumber earliest_write_conflict_snapshot; + std::vector snapshot_seqs = + snapshots_.GetAll(&earliest_write_conflict_snapshot); + + assert(is_snapshot_supported_ || snapshots_.empty()); + CompactionJob compaction_job( + job_context->job_id, c.get(), immutable_db_options_, env_options_, + versions_.get(), &shutting_down_, log_buffer, directories_.GetDbDir(), + directories_.GetDataDir(c->output_path_id()), stats_, &mutex_, + &bg_error_, snapshot_seqs, earliest_write_conflict_snapshot, + table_cache_, &event_logger_, + c->mutable_cf_options()->paranoid_file_checks, + c->mutable_cf_options()->report_bg_io_stats, dbname_, + &compaction_job_stats); + compaction_job.Prepare(); + + mutex_.Unlock(); + compaction_job.Run(); + TEST_SYNC_POINT("DBImpl::BackgroundCompaction:NonTrivial:AfterRun"); + mutex_.Lock(); + + status = compaction_job.Install(*c->mutable_cf_options()); + if (status.ok()) { + InstallSuperVersionAndScheduleWorkWrapper( + c->column_family_data(), job_context, *c->mutable_cf_options()); + } + *made_progress = true; + } + if (c != nullptr) { + c->ReleaseCompactionFiles(status); + *made_progress = true; + NotifyOnCompactionCompleted( + c->column_family_data(), c.get(), status, + compaction_job_stats, job_context->job_id); + } + // this will unref its input_version and column_family_data + c.reset(); + + if (status.ok()) { + // Done + } else if (status.IsShutdownInProgress()) { + // Ignore compaction errors found during shutting down + } else { + ROCKS_LOG_WARN(immutable_db_options_.info_log, "Compaction error: %s", + status.ToString().c_str()); + if (immutable_db_options_.paranoid_checks && bg_error_.ok()) { + Status new_bg_error = status; + // may temporarily unlock and lock the mutex. + EventHelpers::NotifyOnBackgroundError(immutable_db_options_.listeners, + BackgroundErrorReason::kCompaction, + &new_bg_error, &mutex_); + if (!new_bg_error.ok()) { + bg_error_ = new_bg_error; + } + } + } + + if (is_manual) { + ManualCompactionState* m = manual_compaction; + if (!status.ok()) { + m->status = status; + m->done = true; + } + // For universal compaction: + // Because universal compaction always happens at level 0, so one + // compaction will pick up all overlapped files. No files will be + // filtered out due to size limit and left for a successive compaction. + // So we can safely conclude the current compaction. + // + // Also note that, if we don't stop here, then the current compaction + // writes a new file back to level 0, which will be used in successive + // compaction. Hence the manual compaction will never finish. + // + // Stop the compaction if manual_end points to nullptr -- this means + // that we compacted the whole range. manual_end should always point + // to nullptr in case of universal compaction + if (m->manual_end == nullptr) { + m->done = true; + } + if (!m->done) { + // We only compacted part of the requested range. Update *m + // to the range that is left to be compacted. + // Universal and FIFO compactions should always compact the whole range + assert(m->cfd->ioptions()->compaction_style != + kCompactionStyleUniversal || + m->cfd->ioptions()->num_levels > 1); + assert(m->cfd->ioptions()->compaction_style != kCompactionStyleFIFO); + m->tmp_storage = *m->manual_end; + m->begin = &m->tmp_storage; + m->incomplete = true; + } + m->in_progress = false; // not being processed anymore + } + TEST_SYNC_POINT("DBImpl::BackgroundCompaction:Finish"); + return status; +} + +bool DBImpl::HasPendingManualCompaction() { + return (!manual_compaction_dequeue_.empty()); +} + +void DBImpl::AddManualCompaction(DBImpl::ManualCompactionState* m) { + manual_compaction_dequeue_.push_back(m); +} + +void DBImpl::RemoveManualCompaction(DBImpl::ManualCompactionState* m) { + // Remove from queue + std::deque::iterator it = + manual_compaction_dequeue_.begin(); + while (it != manual_compaction_dequeue_.end()) { + if (m == (*it)) { + it = manual_compaction_dequeue_.erase(it); + return; + } + it++; + } + assert(false); + return; +} + +bool DBImpl::ShouldntRunManualCompaction(ManualCompactionState* m) { + if (num_running_ingest_file_ > 0) { + // We need to wait for other IngestExternalFile() calls to finish + // before running a manual compaction. + return true; + } + if (m->exclusive) { + return (bg_bottom_compaction_scheduled_ > 0 || + bg_compaction_scheduled_ > 0); + } + std::deque::iterator it = + manual_compaction_dequeue_.begin(); + bool seen = false; + while (it != manual_compaction_dequeue_.end()) { + if (m == (*it)) { + it++; + seen = true; + continue; + } else if (MCOverlap(m, (*it)) && (!seen && !(*it)->in_progress)) { + // Consider the other manual compaction *it, conflicts if: + // overlaps with m + // and (*it) is ahead in the queue and is not yet in progress + return true; + } + it++; + } + return false; +} + +bool DBImpl::HaveManualCompaction(ColumnFamilyData* cfd) { + // Remove from priority queue + std::deque::iterator it = + manual_compaction_dequeue_.begin(); + while (it != manual_compaction_dequeue_.end()) { + if ((*it)->exclusive) { + return true; + } + if ((cfd == (*it)->cfd) && (!((*it)->in_progress || (*it)->done))) { + // Allow automatic compaction if manual compaction is + // in progress + return true; + } + it++; + } + return false; +} + +bool DBImpl::HasExclusiveManualCompaction() { + // Remove from priority queue + std::deque::iterator it = + manual_compaction_dequeue_.begin(); + while (it != manual_compaction_dequeue_.end()) { + if ((*it)->exclusive) { + return true; + } + it++; + } + return false; +} + +bool DBImpl::MCOverlap(ManualCompactionState* m, ManualCompactionState* m1) { + if ((m->exclusive) || (m1->exclusive)) { + return true; + } + if (m->cfd != m1->cfd) { + return false; + } + return true; +} + +// JobContext gets created and destructed outside of the lock -- +// we +// use this convinently to: +// * malloc one SuperVersion() outside of the lock -- new_superversion +// * delete SuperVersion()s outside of the lock -- superversions_to_free +// +// However, if InstallSuperVersionAndScheduleWork() gets called twice with the +// same job_context, we can't reuse the SuperVersion() that got +// malloced because +// first call already used it. In that rare case, we take a hit and create a +// new SuperVersion() inside of the mutex. We do similar thing +// for superversion_to_free +void DBImpl::InstallSuperVersionAndScheduleWorkWrapper( + ColumnFamilyData* cfd, JobContext* job_context, + const MutableCFOptions& mutable_cf_options) { + mutex_.AssertHeld(); + SuperVersion* old_superversion = InstallSuperVersionAndScheduleWork( + cfd, job_context->new_superversion, mutable_cf_options); + job_context->new_superversion = nullptr; + job_context->superversions_to_free.push_back(old_superversion); +} + +SuperVersion* DBImpl::InstallSuperVersionAndScheduleWork( + ColumnFamilyData* cfd, SuperVersion* new_sv, + const MutableCFOptions& mutable_cf_options) { + mutex_.AssertHeld(); + + // Update max_total_in_memory_state_ + size_t old_memtable_size = 0; + auto* old_sv = cfd->GetSuperVersion(); + if (old_sv) { + old_memtable_size = old_sv->mutable_cf_options.write_buffer_size * + old_sv->mutable_cf_options.max_write_buffer_number; + } + + auto* old = cfd->InstallSuperVersion( + new_sv ? new_sv : new SuperVersion(), &mutex_, mutable_cf_options); + + // Whenever we install new SuperVersion, we might need to issue new flushes or + // compactions. + SchedulePendingFlush(cfd); + SchedulePendingCompaction(cfd); + MaybeScheduleFlushOrCompaction(); + + // Update max_total_in_memory_state_ + max_total_in_memory_state_ = + max_total_in_memory_state_ - old_memtable_size + + mutable_cf_options.write_buffer_size * + mutable_cf_options.max_write_buffer_number; + return old; +} +} // namespace rocksdb diff --git a/db/db_impl_debug.cc b/db/db_impl_debug.cc index 8df66f6c6d9..a4b378020aa 100644 --- a/db/db_impl_debug.cc +++ b/db/db_impl_debug.cc @@ -1,39 +1,28 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. -#ifndef ROCKSDB_LITE +#ifndef NDEBUG #include "db/db_impl.h" +#include "monitoring/thread_status_updater.h" namespace rocksdb { -void DBImpl::TEST_PurgeObsoleteteWAL() { PurgeObsoleteWALFiles(); } - uint64_t DBImpl::TEST_GetLevel0TotalSize() { - MutexLock l(&mutex_); - return default_cf_handle_->cfd()->current()->NumLevelBytes(0); + InstrumentedMutexLock l(&mutex_); + return default_cf_handle_->cfd()->current()->storage_info()->NumLevelBytes(0); } -Iterator* DBImpl::TEST_NewInternalIterator(ColumnFamilyHandle* column_family) { - ColumnFamilyData* cfd; - if (column_family == nullptr) { - cfd = default_cf_handle_->cfd(); - } else { - auto cfh = reinterpret_cast(column_family); - cfd = cfh->cfd(); - } - - mutex_.Lock(); - SuperVersion* super_version = cfd->GetSuperVersion()->Ref(); - mutex_.Unlock(); - ReadOptions roptions; - return NewInternalIterator(roptions, cfd, super_version); +void DBImpl::TEST_HandleWALFull() { + WriteContext write_context; + InstrumentedMutexLock l(&mutex_); + HandleWALFull(&write_context); } int64_t DBImpl::TEST_MaxNextLevelOverlappingBytes( @@ -45,8 +34,8 @@ int64_t DBImpl::TEST_MaxNextLevelOverlappingBytes( auto cfh = reinterpret_cast(column_family); cfd = cfh->cfd(); } - MutexLock l(&mutex_); - return cfd->current()->MaxNextLevelOverlappingBytes(); + InstrumentedMutexLock l(&mutex_); + return cfd->current()->storage_info()->MaxNextLevelOverlappingBytes(); } void DBImpl::TEST_GetFilesMetaData( @@ -54,10 +43,11 @@ void DBImpl::TEST_GetFilesMetaData( std::vector>* metadata) { auto cfh = reinterpret_cast(column_family); auto cfd = cfh->cfd(); - MutexLock l(&mutex_); + InstrumentedMutexLock l(&mutex_); metadata->resize(NumberLevels()); for (int level = 0; level < NumberLevels(); level++) { - const std::vector& files = cfd->current()->files_[level]; + const std::vector& files = + cfd->current()->storage_info()->LevelFiles(level); (*metadata)[level].clear(); for (const auto& f : files) { @@ -67,12 +57,13 @@ void DBImpl::TEST_GetFilesMetaData( } uint64_t DBImpl::TEST_Current_Manifest_FileNo() { - return versions_->ManifestFileNumber(); + return versions_->manifest_file_number(); } Status DBImpl::TEST_CompactRange(int level, const Slice* begin, const Slice* end, - ColumnFamilyHandle* column_family) { + ColumnFamilyHandle* column_family, + bool disallow_trivial_move) { ColumnFamilyData* cfd; if (column_family == nullptr) { cfd = default_cf_handle_->cfd(); @@ -81,17 +72,34 @@ Status DBImpl::TEST_CompactRange(int level, const Slice* begin, cfd = cfh->cfd(); } int output_level = - (cfd->options()->compaction_style == kCompactionStyleUniversal || - cfd->options()->compaction_style == kCompactionStyleFIFO) + (cfd->ioptions()->compaction_style == kCompactionStyleUniversal || + cfd->ioptions()->compaction_style == kCompactionStyleFIFO) ? level : level + 1; - return RunManualCompaction(cfd, level, output_level, 0, begin, end); + return RunManualCompaction(cfd, level, output_level, 0, begin, end, true, + disallow_trivial_move); +} + +Status DBImpl::TEST_SwitchMemtable(ColumnFamilyData* cfd) { + WriteContext write_context; + InstrumentedMutexLock l(&mutex_); + if (cfd == nullptr) { + cfd = default_cf_handle_->cfd(); + } + return SwitchMemtable(cfd, &write_context); } -Status DBImpl::TEST_FlushMemTable(bool wait) { +Status DBImpl::TEST_FlushMemTable(bool wait, ColumnFamilyHandle* cfh) { FlushOptions fo; fo.wait = wait; - return FlushMemTable(default_cf_handle_->cfd(), fo); + ColumnFamilyData* cfd; + if (cfh == nullptr) { + cfd = default_cf_handle_->cfd(); + } else { + auto cfhi = reinterpret_cast(cfh); + cfd = cfhi->cfd(); + } + return FlushMemTable(cfd, fo); } Status DBImpl::TEST_WaitForFlushMemTable(ColumnFamilyHandle* column_family) { @@ -112,22 +120,90 @@ Status DBImpl::TEST_WaitForCompact() { // wait for compact. It actually waits for scheduled compaction // OR flush to finish. - MutexLock l(&mutex_); - while ((bg_compaction_scheduled_ || bg_flush_scheduled_) && bg_error_.ok()) { + InstrumentedMutexLock l(&mutex_); + while ((bg_bottom_compaction_scheduled_ || bg_compaction_scheduled_ || + bg_flush_scheduled_) && + bg_error_.ok()) { bg_cv_.Wait(); } return bg_error_; } -Status DBImpl::TEST_ReadFirstRecord(const WalFileType type, - const uint64_t number, - SequenceNumber* sequence) { - return ReadFirstRecord(type, number, sequence); +void DBImpl::TEST_LockMutex() { + mutex_.Lock(); +} + +void DBImpl::TEST_UnlockMutex() { + mutex_.Unlock(); +} + +void* DBImpl::TEST_BeginWrite() { + auto w = new WriteThread::Writer(); + write_thread_.EnterUnbatched(w, &mutex_); + return reinterpret_cast(w); +} + +void DBImpl::TEST_EndWrite(void* w) { + auto writer = reinterpret_cast(w); + write_thread_.ExitUnbatched(writer); + delete writer; +} + +size_t DBImpl::TEST_LogsToFreeSize() { + InstrumentedMutexLock l(&mutex_); + return logs_to_free_.size(); +} + +uint64_t DBImpl::TEST_LogfileNumber() { + InstrumentedMutexLock l(&mutex_); + return logfile_number_; +} + +Status DBImpl::TEST_GetAllImmutableCFOptions( + std::unordered_map* iopts_map) { + std::vector cf_names; + std::vector iopts; + { + InstrumentedMutexLock l(&mutex_); + for (auto cfd : *versions_->GetColumnFamilySet()) { + cf_names.push_back(cfd->GetName()); + iopts.push_back(cfd->ioptions()); + } + } + iopts_map->clear(); + for (size_t i = 0; i < cf_names.size(); ++i) { + iopts_map->insert({cf_names[i], iopts[i]}); + } + + return Status::OK(); +} + +uint64_t DBImpl::TEST_FindMinLogContainingOutstandingPrep() { + return FindMinLogContainingOutstandingPrep(); +} + +uint64_t DBImpl::TEST_FindMinPrepLogReferencedByMemTable() { + return FindMinPrepLogReferencedByMemTable(); +} + +Status DBImpl::TEST_GetLatestMutableCFOptions( + ColumnFamilyHandle* column_family, MutableCFOptions* mutable_cf_options) { + InstrumentedMutexLock l(&mutex_); + + auto cfh = reinterpret_cast(column_family); + *mutable_cf_options = *cfh->cfd()->GetLatestMutableCFOptions(); + return Status::OK(); +} + +int DBImpl::TEST_BGCompactionsAllowed() const { + InstrumentedMutexLock l(&mutex_); + return GetBGJobLimits().max_compactions; } -Status DBImpl::TEST_ReadFirstLine(const std::string& fname, - SequenceNumber* sequence) { - return ReadFirstLine(fname, sequence); +int DBImpl::TEST_BGFlushesAllowed() const { + InstrumentedMutexLock l(&mutex_); + return GetBGJobLimits().max_flushes; } + } // namespace rocksdb -#endif // ROCKSDB_LITE +#endif // NDEBUG diff --git a/db/db_impl_experimental.cc b/db/db_impl_experimental.cc new file mode 100644 index 00000000000..0d010758e6e --- /dev/null +++ b/db/db_impl_experimental.cc @@ -0,0 +1,152 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_impl.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include + +#include "db/column_family.h" +#include "db/job_context.h" +#include "db/version_set.h" +#include "rocksdb/status.h" + +namespace rocksdb { + +#ifndef ROCKSDB_LITE +Status DBImpl::SuggestCompactRange(ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) { + auto cfh = reinterpret_cast(column_family); + auto cfd = cfh->cfd(); + InternalKey start_key, end_key; + if (begin != nullptr) { + start_key.SetMaxPossibleForUserKey(*begin); + } + if (end != nullptr) { + end_key.SetMinPossibleForUserKey(*end); + } + { + InstrumentedMutexLock l(&mutex_); + auto vstorage = cfd->current()->storage_info(); + for (int level = 0; level < vstorage->num_non_empty_levels() - 1; ++level) { + std::vector inputs; + vstorage->GetOverlappingInputs( + level, begin == nullptr ? nullptr : &start_key, + end == nullptr ? nullptr : &end_key, &inputs); + for (auto f : inputs) { + f->marked_for_compaction = true; + } + } + // Since we have some more files to compact, we should also recompute + // compaction score + vstorage->ComputeCompactionScore(*cfd->ioptions(), + *cfd->GetLatestMutableCFOptions()); + SchedulePendingCompaction(cfd); + MaybeScheduleFlushOrCompaction(); + } + return Status::OK(); +} + +Status DBImpl::PromoteL0(ColumnFamilyHandle* column_family, int target_level) { + assert(column_family); + + if (target_level < 1) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "PromoteL0 FAILED. Invalid target level %d\n", target_level); + return Status::InvalidArgument("Invalid target level"); + } + + Status status; + VersionEdit edit; + JobContext job_context(next_job_id_.fetch_add(1), true); + { + InstrumentedMutexLock l(&mutex_); + auto* cfd = static_cast(column_family)->cfd(); + const auto* vstorage = cfd->current()->storage_info(); + + if (target_level >= vstorage->num_levels()) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "PromoteL0 FAILED. Target level %d does not exist\n", + target_level); + job_context.Clean(); + return Status::InvalidArgument("Target level does not exist"); + } + + // Sort L0 files by range. + const InternalKeyComparator* icmp = &cfd->internal_comparator(); + auto l0_files = vstorage->LevelFiles(0); + std::sort(l0_files.begin(), l0_files.end(), + [icmp](FileMetaData* f1, FileMetaData* f2) { + return icmp->Compare(f1->largest, f2->largest) < 0; + }); + + // Check that no L0 file is being compacted and that they have + // non-overlapping ranges. + for (size_t i = 0; i < l0_files.size(); ++i) { + auto f = l0_files[i]; + if (f->being_compacted) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "PromoteL0 FAILED. File %" PRIu64 " being compacted\n", + f->fd.GetNumber()); + job_context.Clean(); + return Status::InvalidArgument("PromoteL0 called during L0 compaction"); + } + + if (i == 0) continue; + auto prev_f = l0_files[i - 1]; + if (icmp->Compare(prev_f->largest, f->smallest) >= 0) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "PromoteL0 FAILED. Files %" PRIu64 " and %" PRIu64 + " have overlapping ranges\n", + prev_f->fd.GetNumber(), f->fd.GetNumber()); + job_context.Clean(); + return Status::InvalidArgument("L0 has overlapping files"); + } + } + + // Check that all levels up to target_level are empty. + for (int level = 1; level <= target_level; ++level) { + if (vstorage->NumLevelFiles(level) > 0) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "PromoteL0 FAILED. Level %d not empty\n", level); + job_context.Clean(); + return Status::InvalidArgument( + "All levels up to target_level " + "must be empty"); + } + } + + edit.SetColumnFamily(cfd->GetID()); + for (const auto& f : l0_files) { + edit.DeleteFile(0, f->fd.GetNumber()); + edit.AddFile(target_level, f->fd.GetNumber(), f->fd.GetPathId(), + f->fd.GetFileSize(), f->smallest, f->largest, + f->smallest_seqno, f->largest_seqno, + f->marked_for_compaction); + } + + status = versions_->LogAndApply(cfd, *cfd->GetLatestMutableCFOptions(), + &edit, &mutex_, directories_.GetDbDir()); + if (status.ok()) { + InstallSuperVersionAndScheduleWorkWrapper( + cfd, &job_context, *cfd->GetLatestMutableCFOptions()); + } + } // lock released here + LogFlush(immutable_db_options_.info_log); + job_context.Clean(); + + return status; +} +#endif // ROCKSDB_LITE + +} // namespace rocksdb diff --git a/db/db_impl_files.cc b/db/db_impl_files.cc new file mode 100644 index 00000000000..e44e4231895 --- /dev/null +++ b/db/db_impl_files.cc @@ -0,0 +1,548 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#include "db/db_impl.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif +#include +#include "db/event_helpers.h" +#include "util/file_util.h" +#include "util/sst_file_manager_impl.h" + + +namespace rocksdb { +uint64_t DBImpl::FindMinPrepLogReferencedByMemTable() { + if (!allow_2pc()) { + return 0; + } + + uint64_t min_log = 0; + + // we must look through the memtables for two phase transactions + // that have been committed but not yet flushed + for (auto loop_cfd : *versions_->GetColumnFamilySet()) { + if (loop_cfd->IsDropped()) { + continue; + } + + auto log = loop_cfd->imm()->GetMinLogContainingPrepSection(); + + if (log > 0 && (min_log == 0 || log < min_log)) { + min_log = log; + } + + log = loop_cfd->mem()->GetMinLogContainingPrepSection(); + + if (log > 0 && (min_log == 0 || log < min_log)) { + min_log = log; + } + } + + return min_log; +} + +void DBImpl::MarkLogAsHavingPrepSectionFlushed(uint64_t log) { + assert(log != 0); + std::lock_guard lock(prep_heap_mutex_); + auto it = prepared_section_completed_.find(log); + assert(it != prepared_section_completed_.end()); + it->second += 1; +} + +void DBImpl::MarkLogAsContainingPrepSection(uint64_t log) { + assert(log != 0); + std::lock_guard lock(prep_heap_mutex_); + min_log_with_prep_.push(log); + auto it = prepared_section_completed_.find(log); + if (it == prepared_section_completed_.end()) { + prepared_section_completed_[log] = 0; + } +} + +uint64_t DBImpl::FindMinLogContainingOutstandingPrep() { + + if (!allow_2pc()) { + return 0; + } + + std::lock_guard lock(prep_heap_mutex_); + uint64_t min_log = 0; + + // first we look in the prepared heap where we keep + // track of transactions that have been prepared (written to WAL) + // but not yet committed. + while (!min_log_with_prep_.empty()) { + min_log = min_log_with_prep_.top(); + + auto it = prepared_section_completed_.find(min_log); + + // value was marked as 'deleted' from heap + if (it != prepared_section_completed_.end() && it->second > 0) { + it->second -= 1; + min_log_with_prep_.pop(); + + // back to squere one... + min_log = 0; + continue; + } else { + // found a valid value + break; + } + } + + return min_log; +} + +uint64_t DBImpl::MinLogNumberToKeep() { + uint64_t log_number = versions_->MinLogNumber(); + + if (allow_2pc()) { + // if are 2pc we must consider logs containing prepared + // sections of outstanding transactions. + // + // We must check min logs with outstanding prep before we check + // logs referneces by memtables because a log referenced by the + // first data structure could transition to the second under us. + // + // TODO(horuff): iterating over all column families under db mutex. + // should find more optimial solution + auto min_log_in_prep_heap = FindMinLogContainingOutstandingPrep(); + + if (min_log_in_prep_heap != 0 && min_log_in_prep_heap < log_number) { + log_number = min_log_in_prep_heap; + } + + auto min_log_refed_by_mem = FindMinPrepLogReferencedByMemTable(); + + if (min_log_refed_by_mem != 0 && min_log_refed_by_mem < log_number) { + log_number = min_log_refed_by_mem; + } + } + return log_number; +} + +// * Returns the list of live files in 'sst_live' +// If it's doing full scan: +// * Returns the list of all files in the filesystem in +// 'full_scan_candidate_files'. +// Otherwise, gets obsolete files from VersionSet. +// no_full_scan = true -- never do the full scan using GetChildren() +// force = false -- don't force the full scan, except every +// mutable_db_options_.delete_obsolete_files_period_micros +// force = true -- force the full scan +void DBImpl::FindObsoleteFiles(JobContext* job_context, bool force, + bool no_full_scan) { + mutex_.AssertHeld(); + + // if deletion is disabled, do nothing + if (disable_delete_obsolete_files_ > 0) { + return; + } + + bool doing_the_full_scan = false; + + // logic for figurint out if we're doing the full scan + if (no_full_scan) { + doing_the_full_scan = false; + } else if (force || + mutable_db_options_.delete_obsolete_files_period_micros == 0) { + doing_the_full_scan = true; + } else { + const uint64_t now_micros = env_->NowMicros(); + if ((delete_obsolete_files_last_run_ + + mutable_db_options_.delete_obsolete_files_period_micros) < + now_micros) { + doing_the_full_scan = true; + delete_obsolete_files_last_run_ = now_micros; + } + } + + // don't delete files that might be currently written to from compaction + // threads + // Since job_context->min_pending_output is set, until file scan finishes, + // mutex_ cannot be released. Otherwise, we might see no min_pending_output + // here but later find newer generated unfinalized files while scannint. + if (!pending_outputs_.empty()) { + job_context->min_pending_output = *pending_outputs_.begin(); + } else { + // delete all of them + job_context->min_pending_output = std::numeric_limits::max(); + } + + // Get obsolete files. This function will also update the list of + // pending files in VersionSet(). + versions_->GetObsoleteFiles(&job_context->sst_delete_files, + &job_context->manifest_delete_files, + job_context->min_pending_output); + + // store the current filenum, lognum, etc + job_context->manifest_file_number = versions_->manifest_file_number(); + job_context->pending_manifest_file_number = + versions_->pending_manifest_file_number(); + job_context->log_number = MinLogNumberToKeep(); + + job_context->prev_log_number = versions_->prev_log_number(); + + versions_->AddLiveFiles(&job_context->sst_live); + if (doing_the_full_scan) { + for (size_t path_id = 0; path_id < immutable_db_options_.db_paths.size(); + path_id++) { + // set of all files in the directory. We'll exclude files that are still + // alive in the subsequent processings. + std::vector files; + env_->GetChildren(immutable_db_options_.db_paths[path_id].path, + &files); // Ignore errors + for (std::string file : files) { + // TODO(icanadi) clean up this mess to avoid having one-off "/" prefixes + job_context->full_scan_candidate_files.emplace_back( + "/" + file, static_cast(path_id)); + } + } + + // Add log files in wal_dir + if (immutable_db_options_.wal_dir != dbname_) { + std::vector log_files; + env_->GetChildren(immutable_db_options_.wal_dir, + &log_files); // Ignore errors + for (std::string log_file : log_files) { + job_context->full_scan_candidate_files.emplace_back(log_file, 0); + } + } + // Add info log files in db_log_dir + if (!immutable_db_options_.db_log_dir.empty() && + immutable_db_options_.db_log_dir != dbname_) { + std::vector info_log_files; + // Ignore errors + env_->GetChildren(immutable_db_options_.db_log_dir, &info_log_files); + for (std::string log_file : info_log_files) { + job_context->full_scan_candidate_files.emplace_back(log_file, 0); + } + } + } + + // logs_ is empty when called during recovery, in which case there can't yet + // be any tracked obsolete logs + if (!alive_log_files_.empty() && !logs_.empty()) { + uint64_t min_log_number = job_context->log_number; + size_t num_alive_log_files = alive_log_files_.size(); + // find newly obsoleted log files + while (alive_log_files_.begin()->number < min_log_number) { + auto& earliest = *alive_log_files_.begin(); + if (immutable_db_options_.recycle_log_file_num > + log_recycle_files.size()) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "adding log %" PRIu64 " to recycle list\n", + earliest.number); + log_recycle_files.push_back(earliest.number); + } else { + job_context->log_delete_files.push_back(earliest.number); + } + if (job_context->size_log_to_delete == 0) { + job_context->prev_total_log_size = total_log_size_; + job_context->num_alive_log_files = num_alive_log_files; + } + job_context->size_log_to_delete += earliest.size; + total_log_size_ -= earliest.size; + if (concurrent_prepare_) { + log_write_mutex_.Lock(); + } + alive_log_files_.pop_front(); + if (concurrent_prepare_) { + log_write_mutex_.Unlock(); + } + // Current log should always stay alive since it can't have + // number < MinLogNumber(). + assert(alive_log_files_.size()); + } + while (!logs_.empty() && logs_.front().number < min_log_number) { + auto& log = logs_.front(); + if (log.getting_synced) { + log_sync_cv_.Wait(); + // logs_ could have changed while we were waiting. + continue; + } + logs_to_free_.push_back(log.ReleaseWriter()); + { + InstrumentedMutexLock wl(&log_write_mutex_); + logs_.pop_front(); + } + } + // Current log cannot be obsolete. + assert(!logs_.empty()); + } + + // We're just cleaning up for DB::Write(). + assert(job_context->logs_to_free.empty()); + job_context->logs_to_free = logs_to_free_; + job_context->log_recycle_files.assign(log_recycle_files.begin(), + log_recycle_files.end()); + logs_to_free_.clear(); +} + +namespace { +bool CompareCandidateFile(const JobContext::CandidateFileInfo& first, + const JobContext::CandidateFileInfo& second) { + if (first.file_name > second.file_name) { + return true; + } else if (first.file_name < second.file_name) { + return false; + } else { + return (first.path_id > second.path_id); + } +} +}; // namespace + +// Delete obsolete files and log status and information of file deletion +void DBImpl::DeleteObsoleteFileImpl(Status file_deletion_status, int job_id, + const std::string& fname, FileType type, + uint64_t number, uint32_t path_id) { + if (type == kTableFile) { + file_deletion_status = + DeleteSSTFile(&immutable_db_options_, fname, path_id); + } else { + file_deletion_status = env_->DeleteFile(fname); + } + if (file_deletion_status.ok()) { + ROCKS_LOG_DEBUG(immutable_db_options_.info_log, + "[JOB %d] Delete %s type=%d #%" PRIu64 " -- %s\n", job_id, + fname.c_str(), type, number, + file_deletion_status.ToString().c_str()); + } else if (env_->FileExists(fname).IsNotFound()) { + ROCKS_LOG_INFO( + immutable_db_options_.info_log, + "[JOB %d] Tried to delete a non-existing file %s type=%d #%" PRIu64 + " -- %s\n", + job_id, fname.c_str(), type, number, + file_deletion_status.ToString().c_str()); + } else { + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "[JOB %d] Failed to delete %s type=%d #%" PRIu64 " -- %s\n", + job_id, fname.c_str(), type, number, + file_deletion_status.ToString().c_str()); + } + if (type == kTableFile) { + EventHelpers::LogAndNotifyTableFileDeletion( + &event_logger_, job_id, number, fname, file_deletion_status, GetName(), + immutable_db_options_.listeners); + } +} + +// Diffs the files listed in filenames and those that do not +// belong to live files are posibly removed. Also, removes all the +// files in sst_delete_files and log_delete_files. +// It is not necessary to hold the mutex when invoking this method. +void DBImpl::PurgeObsoleteFiles(const JobContext& state, bool schedule_only) { + // we'd better have sth to delete + assert(state.HaveSomethingToDelete()); + + // this checks if FindObsoleteFiles() was run before. If not, don't do + // PurgeObsoleteFiles(). If FindObsoleteFiles() was run, we need to also + // run PurgeObsoleteFiles(), even if disable_delete_obsolete_files_ is true + if (state.manifest_file_number == 0) { + return; + } + + // Now, convert live list to an unordered map, WITHOUT mutex held; + // set is slow. + std::unordered_map sst_live_map; + for (const FileDescriptor& fd : state.sst_live) { + sst_live_map[fd.GetNumber()] = &fd; + } + std::unordered_set log_recycle_files_set( + state.log_recycle_files.begin(), state.log_recycle_files.end()); + + auto candidate_files = state.full_scan_candidate_files; + candidate_files.reserve( + candidate_files.size() + state.sst_delete_files.size() + + state.log_delete_files.size() + state.manifest_delete_files.size()); + // We may ignore the dbname when generating the file names. + const char* kDumbDbName = ""; + for (auto file : state.sst_delete_files) { + candidate_files.emplace_back( + MakeTableFileName(kDumbDbName, file->fd.GetNumber()), + file->fd.GetPathId()); + if (file->table_reader_handle) { + table_cache_->Release(file->table_reader_handle); + } + delete file; + } + + for (auto file_num : state.log_delete_files) { + if (file_num > 0) { + candidate_files.emplace_back(LogFileName(kDumbDbName, file_num), 0); + } + } + for (const auto& filename : state.manifest_delete_files) { + candidate_files.emplace_back(filename, 0); + } + + // dedup state.candidate_files so we don't try to delete the same + // file twice + std::sort(candidate_files.begin(), candidate_files.end(), + CompareCandidateFile); + candidate_files.erase( + std::unique(candidate_files.begin(), candidate_files.end()), + candidate_files.end()); + + if (state.prev_total_log_size > 0) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "[JOB %d] Try to delete WAL files size %" PRIu64 + ", prev total WAL file size %" PRIu64 + ", number of live WAL files %" ROCKSDB_PRIszt ".\n", + state.job_id, state.size_log_to_delete, + state.prev_total_log_size, state.num_alive_log_files); + } + + std::vector old_info_log_files; + InfoLogPrefix info_log_prefix(!immutable_db_options_.db_log_dir.empty(), + dbname_); + for (const auto& candidate_file : candidate_files) { + std::string to_delete = candidate_file.file_name; + uint32_t path_id = candidate_file.path_id; + uint64_t number; + FileType type; + // Ignore file if we cannot recognize it. + if (!ParseFileName(to_delete, &number, info_log_prefix.prefix, &type)) { + continue; + } + + bool keep = true; + switch (type) { + case kLogFile: + keep = ((number >= state.log_number) || + (number == state.prev_log_number) || + (log_recycle_files_set.find(number) != + log_recycle_files_set.end())); + break; + case kDescriptorFile: + // Keep my manifest file, and any newer incarnations' + // (can happen during manifest roll) + keep = (number >= state.manifest_file_number); + break; + case kTableFile: + // If the second condition is not there, this makes + // DontDeletePendingOutputs fail + keep = (sst_live_map.find(number) != sst_live_map.end()) || + number >= state.min_pending_output; + break; + case kTempFile: + // Any temp files that are currently being written to must + // be recorded in pending_outputs_, which is inserted into "live". + // Also, SetCurrentFile creates a temp file when writing out new + // manifest, which is equal to state.pending_manifest_file_number. We + // should not delete that file + // + // TODO(yhchiang): carefully modify the third condition to safely + // remove the temp options files. + keep = (sst_live_map.find(number) != sst_live_map.end()) || + (number == state.pending_manifest_file_number) || + (to_delete.find(kOptionsFileNamePrefix) != std::string::npos); + break; + case kInfoLogFile: + keep = true; + if (number != 0) { + old_info_log_files.push_back(to_delete); + } + break; + case kCurrentFile: + case kDBLockFile: + case kIdentityFile: + case kMetaDatabase: + case kOptionsFile: + case kBlobFile: + keep = true; + break; + } + + if (keep) { + continue; + } + + std::string fname; + if (type == kTableFile) { + // evict from cache + TableCache::Evict(table_cache_.get(), number); + fname = TableFileName(immutable_db_options_.db_paths, number, path_id); + } else { + fname = ((type == kLogFile) ? immutable_db_options_.wal_dir : dbname_) + + "/" + to_delete; + } + +#ifndef ROCKSDB_LITE + if (type == kLogFile && (immutable_db_options_.wal_ttl_seconds > 0 || + immutable_db_options_.wal_size_limit_mb > 0)) { + wal_manager_.ArchiveWALFile(fname, number); + continue; + } +#endif // !ROCKSDB_LITE + + Status file_deletion_status; + if (schedule_only) { + InstrumentedMutexLock guard_lock(&mutex_); + SchedulePendingPurge(fname, type, number, path_id, state.job_id); + } else { + DeleteObsoleteFileImpl(file_deletion_status, state.job_id, fname, type, + number, path_id); + } + } + + // Delete old info log files. + size_t old_info_log_file_count = old_info_log_files.size(); + if (old_info_log_file_count != 0 && + old_info_log_file_count >= immutable_db_options_.keep_log_file_num) { + std::sort(old_info_log_files.begin(), old_info_log_files.end()); + size_t end = + old_info_log_file_count - immutable_db_options_.keep_log_file_num; + for (unsigned int i = 0; i <= end; i++) { + std::string& to_delete = old_info_log_files.at(i); + std::string full_path_to_delete = + (immutable_db_options_.db_log_dir.empty() + ? dbname_ + : immutable_db_options_.db_log_dir) + + "/" + to_delete; + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "[JOB %d] Delete info log file %s\n", state.job_id, + full_path_to_delete.c_str()); + Status s = env_->DeleteFile(full_path_to_delete); + if (!s.ok()) { + if (env_->FileExists(full_path_to_delete).IsNotFound()) { + ROCKS_LOG_INFO( + immutable_db_options_.info_log, + "[JOB %d] Tried to delete non-existing info log file %s FAILED " + "-- %s\n", + state.job_id, to_delete.c_str(), s.ToString().c_str()); + } else { + ROCKS_LOG_ERROR(immutable_db_options_.info_log, + "[JOB %d] Delete info log file %s FAILED -- %s\n", + state.job_id, to_delete.c_str(), + s.ToString().c_str()); + } + } + } + } +#ifndef ROCKSDB_LITE + wal_manager_.PurgeObsoleteWALFiles(); +#endif // ROCKSDB_LITE + LogFlush(immutable_db_options_.info_log); +} + +void DBImpl::DeleteObsoleteFiles() { + mutex_.AssertHeld(); + JobContext job_context(next_job_id_.fetch_add(1)); + FindObsoleteFiles(&job_context, true); + + mutex_.Unlock(); + if (job_context.HaveSomethingToDelete()) { + PurgeObsoleteFiles(job_context); + } + job_context.Clean(); + mutex_.Lock(); +} +} // namespace rocksdb diff --git a/db/db_impl_open.cc b/db/db_impl_open.cc new file mode 100644 index 00000000000..bc94b6095f1 --- /dev/null +++ b/db/db_impl_open.cc @@ -0,0 +1,1129 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#include "db/db_impl.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif +#include + +#include "db/builder.h" +#include "options/options_helper.h" +#include "rocksdb/wal_filter.h" +#include "table/block_based_table_factory.h" +#include "util/rate_limiter.h" +#include "util/sst_file_manager_impl.h" +#include "util/sync_point.h" + +namespace rocksdb { +Options SanitizeOptions(const std::string& dbname, + const Options& src) { + auto db_options = SanitizeOptions(dbname, DBOptions(src)); + ImmutableDBOptions immutable_db_options(db_options); + auto cf_options = + SanitizeOptions(immutable_db_options, ColumnFamilyOptions(src)); + return Options(db_options, cf_options); +} + +DBOptions SanitizeOptions(const std::string& dbname, const DBOptions& src) { + DBOptions result(src); + + // result.max_open_files means an "infinite" open files. + if (result.max_open_files != -1) { + int max_max_open_files = port::GetMaxOpenFiles(); + if (max_max_open_files == -1) { + max_max_open_files = 0x400000; + } + ClipToRange(&result.max_open_files, 20, max_max_open_files); + } + + if (result.info_log == nullptr) { + Status s = CreateLoggerFromOptions(dbname, result, &result.info_log); + if (!s.ok()) { + // No place suitable for logging + result.info_log = nullptr; + } + } + + if (!result.write_buffer_manager) { + result.write_buffer_manager.reset( + new WriteBufferManager(result.db_write_buffer_size)); + } + auto bg_job_limits = DBImpl::GetBGJobLimits(result.max_background_flushes, + result.max_background_compactions, + result.max_background_jobs, + true /* parallelize_compactions */); + result.env->IncBackgroundThreadsIfNeeded(bg_job_limits.max_compactions, + Env::Priority::LOW); + result.env->IncBackgroundThreadsIfNeeded(bg_job_limits.max_flushes, + Env::Priority::HIGH); + + if (result.rate_limiter.get() != nullptr) { + if (result.bytes_per_sync == 0) { + result.bytes_per_sync = 1024 * 1024; + } + } + + if (result.delayed_write_rate == 0) { + if (result.rate_limiter.get() != nullptr) { + result.delayed_write_rate = result.rate_limiter->GetBytesPerSecond(); + } + if (result.delayed_write_rate == 0) { + result.delayed_write_rate = 16 * 1024 * 1024; + } + } + + if (result.WAL_ttl_seconds > 0 || result.WAL_size_limit_MB > 0) { + result.recycle_log_file_num = false; + } + + if (result.recycle_log_file_num && + (result.wal_recovery_mode == WALRecoveryMode::kPointInTimeRecovery || + result.wal_recovery_mode == WALRecoveryMode::kAbsoluteConsistency)) { + // kPointInTimeRecovery is indistinguishable from + // kTolerateCorruptedTailRecords in recycle mode since we define + // the "end" of the log as the first corrupt record we encounter. + // kAbsoluteConsistency doesn't make sense because even a clean + // shutdown leaves old junk at the end of the log file. + result.wal_recovery_mode = WALRecoveryMode::kTolerateCorruptedTailRecords; + } + + if (result.wal_dir.empty()) { + // Use dbname as default + result.wal_dir = dbname; + } + if (result.wal_dir.back() == '/') { + result.wal_dir = result.wal_dir.substr(0, result.wal_dir.size() - 1); + } + + if (result.db_paths.size() == 0) { + result.db_paths.emplace_back(dbname, std::numeric_limits::max()); + } + + if (result.use_direct_io_for_flush_and_compaction && + result.compaction_readahead_size == 0) { + TEST_SYNC_POINT_CALLBACK("SanitizeOptions:direct_io", nullptr); + result.compaction_readahead_size = 1024 * 1024 * 2; + } + + if (result.compaction_readahead_size > 0 || + result.use_direct_io_for_flush_and_compaction) { + result.new_table_reader_for_compaction_inputs = true; + } + + // Force flush on DB open if 2PC is enabled, since with 2PC we have no + // guarantee that consecutive log files have consecutive sequence id, which + // make recovery complicated. + if (result.allow_2pc) { + result.avoid_flush_during_recovery = false; + } + + return result; +} + +namespace { + +Status SanitizeOptionsByTable( + const DBOptions& db_opts, + const std::vector& column_families) { + Status s; + for (auto cf : column_families) { + s = cf.options.table_factory->SanitizeOptions(db_opts, cf.options); + if (!s.ok()) { + return s; + } + } + return Status::OK(); +} + +static Status ValidateOptions( + const DBOptions& db_options, + const std::vector& column_families) { + Status s; + + for (auto& cfd : column_families) { + s = CheckCompressionSupported(cfd.options); + if (s.ok() && db_options.allow_concurrent_memtable_write) { + s = CheckConcurrentWritesSupported(cfd.options); + } + if (!s.ok()) { + return s; + } + if (db_options.db_paths.size() > 1) { + if ((cfd.options.compaction_style != kCompactionStyleUniversal) && + (cfd.options.compaction_style != kCompactionStyleLevel)) { + return Status::NotSupported( + "More than one DB paths are only supported in " + "universal and level compaction styles. "); + } + } + if (cfd.options.compaction_options_fifo.ttl > 0) { + if (db_options.max_open_files != -1) { + return Status::NotSupported( + "FIFO Compaction with TTL is only supported when files are always " + "kept open (set max_open_files = -1). "); + } + if (cfd.options.table_factory->Name() != + BlockBasedTableFactory().Name()) { + return Status::NotSupported( + "FIFO Compaction with TTL is only supported in " + "Block-Based Table format. "); + } + } + } + + if (db_options.db_paths.size() > 4) { + return Status::NotSupported( + "More than four DB paths are not supported yet. "); + } + + if (db_options.allow_mmap_reads && db_options.use_direct_reads) { + // Protect against assert in PosixMMapReadableFile constructor + return Status::NotSupported( + "If memory mapped reads (allow_mmap_reads) are enabled " + "then direct I/O reads (use_direct_reads) must be disabled. "); + } + + if (db_options.allow_mmap_writes && + db_options.use_direct_io_for_flush_and_compaction) { + return Status::NotSupported( + "If memory mapped writes (allow_mmap_writes) are enabled " + "then direct I/O writes (use_direct_io_for_flush_and_compaction) must " + "be disabled. "); + } + + if (db_options.keep_log_file_num == 0) { + return Status::InvalidArgument("keep_log_file_num must be greater than 0"); + } + + return Status::OK(); +} +} // namespace +Status DBImpl::NewDB() { + VersionEdit new_db; + new_db.SetLogNumber(0); + new_db.SetNextFile(2); + new_db.SetLastSequence(0); + + Status s; + + ROCKS_LOG_INFO(immutable_db_options_.info_log, "Creating manifest 1 \n"); + const std::string manifest = DescriptorFileName(dbname_, 1); + { + unique_ptr file; + EnvOptions env_options = env_->OptimizeForManifestWrite(env_options_); + s = NewWritableFile(env_, manifest, &file, env_options); + if (!s.ok()) { + return s; + } + file->SetPreallocationBlockSize( + immutable_db_options_.manifest_preallocation_size); + unique_ptr file_writer( + new WritableFileWriter(std::move(file), env_options)); + log::Writer log(std::move(file_writer), 0, false); + std::string record; + new_db.EncodeTo(&record); + s = log.AddRecord(record); + if (s.ok()) { + s = SyncManifest(env_, &immutable_db_options_, log.file()); + } + } + if (s.ok()) { + // Make "CURRENT" file that points to the new manifest file. + s = SetCurrentFile(env_, dbname_, 1, directories_.GetDbDir()); + } else { + env_->DeleteFile(manifest); + } + return s; +} + +Status DBImpl::Directories::CreateAndNewDirectory( + Env* env, const std::string& dirname, + std::unique_ptr* directory) const { + // We call CreateDirIfMissing() as the directory may already exist (if we + // are reopening a DB), when this happens we don't want creating the + // directory to cause an error. However, we need to check if creating the + // directory fails or else we may get an obscure message about the lock + // file not existing. One real-world example of this occurring is if + // env->CreateDirIfMissing() doesn't create intermediate directories, e.g. + // when dbname_ is "dir/db" but when "dir" doesn't exist. + Status s = env->CreateDirIfMissing(dirname); + if (!s.ok()) { + return s; + } + return env->NewDirectory(dirname, directory); +} + +Status DBImpl::Directories::SetDirectories( + Env* env, const std::string& dbname, const std::string& wal_dir, + const std::vector& data_paths) { + Status s = CreateAndNewDirectory(env, dbname, &db_dir_); + if (!s.ok()) { + return s; + } + if (!wal_dir.empty() && dbname != wal_dir) { + s = CreateAndNewDirectory(env, wal_dir, &wal_dir_); + if (!s.ok()) { + return s; + } + } + + data_dirs_.clear(); + for (auto& p : data_paths) { + const std::string db_path = p.path; + if (db_path == dbname) { + data_dirs_.emplace_back(nullptr); + } else { + std::unique_ptr path_directory; + s = CreateAndNewDirectory(env, db_path, &path_directory); + if (!s.ok()) { + return s; + } + data_dirs_.emplace_back(path_directory.release()); + } + } + assert(data_dirs_.size() == data_paths.size()); + return Status::OK(); +} + +Status DBImpl::Recover( + const std::vector& column_families, bool read_only, + bool error_if_log_file_exist, bool error_if_data_exists_in_logs) { + mutex_.AssertHeld(); + + bool is_new_db = false; + assert(db_lock_ == nullptr); + if (!read_only) { + Status s = directories_.SetDirectories(env_, dbname_, + immutable_db_options_.wal_dir, + immutable_db_options_.db_paths); + if (!s.ok()) { + return s; + } + + s = env_->LockFile(LockFileName(dbname_), &db_lock_); + if (!s.ok()) { + return s; + } + + s = env_->FileExists(CurrentFileName(dbname_)); + if (s.IsNotFound()) { + if (immutable_db_options_.create_if_missing) { + s = NewDB(); + is_new_db = true; + if (!s.ok()) { + return s; + } + } else { + return Status::InvalidArgument( + dbname_, "does not exist (create_if_missing is false)"); + } + } else if (s.ok()) { + if (immutable_db_options_.error_if_exists) { + return Status::InvalidArgument( + dbname_, "exists (error_if_exists is true)"); + } + } else { + // Unexpected error reading file + assert(s.IsIOError()); + return s; + } + // Check for the IDENTITY file and create it if not there + s = env_->FileExists(IdentityFileName(dbname_)); + if (s.IsNotFound()) { + s = SetIdentityFile(env_, dbname_); + if (!s.ok()) { + return s; + } + } else if (!s.ok()) { + assert(s.IsIOError()); + return s; + } + } + + Status s = versions_->Recover(column_families, read_only); + if (immutable_db_options_.paranoid_checks && s.ok()) { + s = CheckConsistency(); + } + if (s.ok()) { + SequenceNumber next_sequence(kMaxSequenceNumber); + default_cf_handle_ = new ColumnFamilyHandleImpl( + versions_->GetColumnFamilySet()->GetDefault(), this, &mutex_); + default_cf_internal_stats_ = default_cf_handle_->cfd()->internal_stats(); + single_column_family_mode_ = + versions_->GetColumnFamilySet()->NumberOfColumnFamilies() == 1; + + // Recover from all newer log files than the ones named in the + // descriptor (new log files may have been added by the previous + // incarnation without registering them in the descriptor). + // + // Note that prev_log_number() is no longer used, but we pay + // attention to it in case we are recovering a database + // produced by an older version of rocksdb. + std::vector filenames; + s = env_->GetChildren(immutable_db_options_.wal_dir, &filenames); + if (!s.ok()) { + return s; + } + + std::vector logs; + for (size_t i = 0; i < filenames.size(); i++) { + uint64_t number; + FileType type; + if (ParseFileName(filenames[i], &number, &type) && type == kLogFile) { + if (is_new_db) { + return Status::Corruption( + "While creating a new Db, wal_dir contains " + "existing log file: ", + filenames[i]); + } else { + logs.push_back(number); + } + } + } + + if (logs.size() > 0) { + if (error_if_log_file_exist) { + return Status::Corruption( + "The db was opened in readonly mode with error_if_log_file_exist" + "flag but a log file already exists"); + } else if (error_if_data_exists_in_logs) { + for (auto& log : logs) { + std::string fname = LogFileName(immutable_db_options_.wal_dir, log); + uint64_t bytes; + s = env_->GetFileSize(fname, &bytes); + if (s.ok()) { + if (bytes > 0) { + return Status::Corruption( + "error_if_data_exists_in_logs is set but there are data " + " in log files."); + } + } + } + } + } + + if (!logs.empty()) { + // Recover in the order in which the logs were generated + std::sort(logs.begin(), logs.end()); + s = RecoverLogFiles(logs, &next_sequence, read_only); + if (!s.ok()) { + // Clear memtables if recovery failed + for (auto cfd : *versions_->GetColumnFamilySet()) { + cfd->CreateNewMemtable(*cfd->GetLatestMutableCFOptions(), + kMaxSequenceNumber); + } + } + } + } + + // Initial value + max_total_in_memory_state_ = 0; + for (auto cfd : *versions_->GetColumnFamilySet()) { + auto* mutable_cf_options = cfd->GetLatestMutableCFOptions(); + max_total_in_memory_state_ += mutable_cf_options->write_buffer_size * + mutable_cf_options->max_write_buffer_number; + } + + return s; +} + +// REQUIRES: log_numbers are sorted in ascending order +Status DBImpl::RecoverLogFiles(const std::vector& log_numbers, + SequenceNumber* next_sequence, bool read_only) { + struct LogReporter : public log::Reader::Reporter { + Env* env; + Logger* info_log; + const char* fname; + Status* status; // nullptr if immutable_db_options_.paranoid_checks==false + virtual void Corruption(size_t bytes, const Status& s) override { + ROCKS_LOG_WARN(info_log, "%s%s: dropping %d bytes; %s", + (this->status == nullptr ? "(ignoring error) " : ""), + fname, static_cast(bytes), s.ToString().c_str()); + if (this->status != nullptr && this->status->ok()) { + *this->status = s; + } + } + }; + + mutex_.AssertHeld(); + Status status; + std::unordered_map version_edits; + // no need to refcount because iteration is under mutex + for (auto cfd : *versions_->GetColumnFamilySet()) { + VersionEdit edit; + edit.SetColumnFamily(cfd->GetID()); + version_edits.insert({cfd->GetID(), edit}); + } + int job_id = next_job_id_.fetch_add(1); + { + auto stream = event_logger_.Log(); + stream << "job" << job_id << "event" + << "recovery_started"; + stream << "log_files"; + stream.StartArray(); + for (auto log_number : log_numbers) { + stream << log_number; + } + stream.EndArray(); + } + +#ifndef ROCKSDB_LITE + if (immutable_db_options_.wal_filter != nullptr) { + std::map cf_name_id_map; + std::map cf_lognumber_map; + for (auto cfd : *versions_->GetColumnFamilySet()) { + cf_name_id_map.insert( + std::make_pair(cfd->GetName(), cfd->GetID())); + cf_lognumber_map.insert( + std::make_pair(cfd->GetID(), cfd->GetLogNumber())); + } + + immutable_db_options_.wal_filter->ColumnFamilyLogNumberMap(cf_lognumber_map, + cf_name_id_map); + } +#endif + + bool stop_replay_by_wal_filter = false; + bool stop_replay_for_corruption = false; + bool flushed = false; + for (auto log_number : log_numbers) { + // The previous incarnation may not have written any MANIFEST + // records after allocating this log number. So we manually + // update the file number allocation counter in VersionSet. + versions_->MarkFileNumberUsedDuringRecovery(log_number); + // Open the log file + std::string fname = LogFileName(immutable_db_options_.wal_dir, log_number); + + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "Recovering log #%" PRIu64 " mode %d", log_number, + immutable_db_options_.wal_recovery_mode); + auto logFileDropped = [this, &fname]() { + uint64_t bytes; + if (env_->GetFileSize(fname, &bytes).ok()) { + auto info_log = immutable_db_options_.info_log.get(); + ROCKS_LOG_WARN(info_log, "%s: dropping %d bytes", fname.c_str(), + static_cast(bytes)); + } + }; + if (stop_replay_by_wal_filter) { + logFileDropped(); + continue; + } + + unique_ptr file_reader; + { + unique_ptr file; + status = env_->NewSequentialFile(fname, &file, + env_->OptimizeForLogRead(env_options_)); + if (!status.ok()) { + MaybeIgnoreError(&status); + if (!status.ok()) { + return status; + } else { + // Fail with one log file, but that's ok. + // Try next one. + continue; + } + } + file_reader.reset(new SequentialFileReader(std::move(file))); + } + + // Create the log reader. + LogReporter reporter; + reporter.env = env_; + reporter.info_log = immutable_db_options_.info_log.get(); + reporter.fname = fname.c_str(); + if (!immutable_db_options_.paranoid_checks || + immutable_db_options_.wal_recovery_mode == + WALRecoveryMode::kSkipAnyCorruptedRecords) { + reporter.status = nullptr; + } else { + reporter.status = &status; + } + // We intentially make log::Reader do checksumming even if + // paranoid_checks==false so that corruptions cause entire commits + // to be skipped instead of propagating bad information (like overly + // large sequence numbers). + log::Reader reader(immutable_db_options_.info_log, std::move(file_reader), + &reporter, true /*checksum*/, 0 /*initial_offset*/, + log_number); + + // Determine if we should tolerate incomplete records at the tail end of the + // Read all the records and add to a memtable + std::string scratch; + Slice record; + WriteBatch batch; + + while (!stop_replay_by_wal_filter && + reader.ReadRecord(&record, &scratch, + immutable_db_options_.wal_recovery_mode) && + status.ok()) { + if (record.size() < WriteBatchInternal::kHeader) { + reporter.Corruption(record.size(), + Status::Corruption("log record too small")); + continue; + } + WriteBatchInternal::SetContents(&batch, record); + SequenceNumber sequence = WriteBatchInternal::Sequence(&batch); + + if (immutable_db_options_.wal_recovery_mode == + WALRecoveryMode::kPointInTimeRecovery) { + // In point-in-time recovery mode, if sequence id of log files are + // consecutive, we continue recovery despite corruption. This could + // happen when we open and write to a corrupted DB, where sequence id + // will start from the last sequence id we recovered. + if (sequence == *next_sequence) { + stop_replay_for_corruption = false; + } + if (stop_replay_for_corruption) { + logFileDropped(); + break; + } + } + +#ifndef ROCKSDB_LITE + if (immutable_db_options_.wal_filter != nullptr) { + WriteBatch new_batch; + bool batch_changed = false; + + WalFilter::WalProcessingOption wal_processing_option = + immutable_db_options_.wal_filter->LogRecordFound( + log_number, fname, batch, &new_batch, &batch_changed); + + switch (wal_processing_option) { + case WalFilter::WalProcessingOption::kContinueProcessing: + // do nothing, proceeed normally + break; + case WalFilter::WalProcessingOption::kIgnoreCurrentRecord: + // skip current record + continue; + case WalFilter::WalProcessingOption::kStopReplay: + // skip current record and stop replay + stop_replay_by_wal_filter = true; + continue; + case WalFilter::WalProcessingOption::kCorruptedRecord: { + status = + Status::Corruption("Corruption reported by Wal Filter ", + immutable_db_options_.wal_filter->Name()); + MaybeIgnoreError(&status); + if (!status.ok()) { + reporter.Corruption(record.size(), status); + continue; + } + break; + } + default: { + assert(false); // unhandled case + status = Status::NotSupported( + "Unknown WalProcessingOption returned" + " by Wal Filter ", + immutable_db_options_.wal_filter->Name()); + MaybeIgnoreError(&status); + if (!status.ok()) { + return status; + } else { + // Ignore the error with current record processing. + continue; + } + } + } + + if (batch_changed) { + // Make sure that the count in the new batch is + // within the orignal count. + int new_count = WriteBatchInternal::Count(&new_batch); + int original_count = WriteBatchInternal::Count(&batch); + if (new_count > original_count) { + ROCKS_LOG_FATAL( + immutable_db_options_.info_log, + "Recovering log #%" PRIu64 + " mode %d log filter %s returned " + "more records (%d) than original (%d) which is not allowed. " + "Aborting recovery.", + log_number, immutable_db_options_.wal_recovery_mode, + immutable_db_options_.wal_filter->Name(), new_count, + original_count); + status = Status::NotSupported( + "More than original # of records " + "returned by Wal Filter ", + immutable_db_options_.wal_filter->Name()); + return status; + } + // Set the same sequence number in the new_batch + // as the original batch. + WriteBatchInternal::SetSequence(&new_batch, + WriteBatchInternal::Sequence(&batch)); + batch = new_batch; + } + } +#endif // ROCKSDB_LITE + + // If column family was not found, it might mean that the WAL write + // batch references to the column family that was dropped after the + // insert. We don't want to fail the whole write batch in that case -- + // we just ignore the update. + // That's why we set ignore missing column families to true + bool has_valid_writes = false; + status = WriteBatchInternal::InsertInto( + &batch, column_family_memtables_.get(), &flush_scheduler_, true, + log_number, this, false /* concurrent_memtable_writes */, + next_sequence, &has_valid_writes); + MaybeIgnoreError(&status); + if (!status.ok()) { + // We are treating this as a failure while reading since we read valid + // blocks that do not form coherent data + reporter.Corruption(record.size(), status); + continue; + } + + if (has_valid_writes && !read_only) { + // we can do this because this is called before client has access to the + // DB and there is only a single thread operating on DB + ColumnFamilyData* cfd; + + while ((cfd = flush_scheduler_.TakeNextColumnFamily()) != nullptr) { + cfd->Unref(); + // If this asserts, it means that InsertInto failed in + // filtering updates to already-flushed column families + assert(cfd->GetLogNumber() <= log_number); + auto iter = version_edits.find(cfd->GetID()); + assert(iter != version_edits.end()); + VersionEdit* edit = &iter->second; + status = WriteLevel0TableForRecovery(job_id, cfd, cfd->mem(), edit); + if (!status.ok()) { + // Reflect errors immediately so that conditions like full + // file-systems cause the DB::Open() to fail. + return status; + } + flushed = true; + + cfd->CreateNewMemtable(*cfd->GetLatestMutableCFOptions(), + *next_sequence); + } + } + } + + if (!status.ok()) { + if (immutable_db_options_.wal_recovery_mode == + WALRecoveryMode::kSkipAnyCorruptedRecords) { + // We should ignore all errors unconditionally + status = Status::OK(); + } else if (immutable_db_options_.wal_recovery_mode == + WALRecoveryMode::kPointInTimeRecovery) { + // We should ignore the error but not continue replaying + status = Status::OK(); + stop_replay_for_corruption = true; + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "Point in time recovered to log #%" PRIu64 + " seq #%" PRIu64, + log_number, *next_sequence); + } else { + assert(immutable_db_options_.wal_recovery_mode == + WALRecoveryMode::kTolerateCorruptedTailRecords || + immutable_db_options_.wal_recovery_mode == + WALRecoveryMode::kAbsoluteConsistency); + return status; + } + } + + flush_scheduler_.Clear(); + auto last_sequence = *next_sequence - 1; + if ((*next_sequence != kMaxSequenceNumber) && + (versions_->LastSequence() <= last_sequence)) { + versions_->SetLastToBeWrittenSequence(last_sequence); + versions_->SetLastSequence(last_sequence); + } + } + + // True if there's any data in the WALs; if not, we can skip re-processing + // them later + bool data_seen = false; + if (!read_only) { + // no need to refcount since client still doesn't have access + // to the DB and can not drop column families while we iterate + auto max_log_number = log_numbers.back(); + for (auto cfd : *versions_->GetColumnFamilySet()) { + auto iter = version_edits.find(cfd->GetID()); + assert(iter != version_edits.end()); + VersionEdit* edit = &iter->second; + + if (cfd->GetLogNumber() > max_log_number) { + // Column family cfd has already flushed the data + // from all logs. Memtable has to be empty because + // we filter the updates based on log_number + // (in WriteBatch::InsertInto) + assert(cfd->mem()->GetFirstSequenceNumber() == 0); + assert(edit->NumEntries() == 0); + continue; + } + + // flush the final memtable (if non-empty) + if (cfd->mem()->GetFirstSequenceNumber() != 0) { + // If flush happened in the middle of recovery (e.g. due to memtable + // being full), we flush at the end. Otherwise we'll need to record + // where we were on last flush, which make the logic complicated. + if (flushed || !immutable_db_options_.avoid_flush_during_recovery) { + status = WriteLevel0TableForRecovery(job_id, cfd, cfd->mem(), edit); + if (!status.ok()) { + // Recovery failed + break; + } + flushed = true; + + cfd->CreateNewMemtable(*cfd->GetLatestMutableCFOptions(), + versions_->LastSequence()); + } + data_seen = true; + } + + // write MANIFEST with update + // writing log_number in the manifest means that any log file + // with number strongly less than (log_number + 1) is already + // recovered and should be ignored on next reincarnation. + // Since we already recovered max_log_number, we want all logs + // with numbers `<= max_log_number` (includes this one) to be ignored + if (flushed || cfd->mem()->GetFirstSequenceNumber() == 0) { + edit->SetLogNumber(max_log_number + 1); + } + // we must mark the next log number as used, even though it's + // not actually used. that is because VersionSet assumes + // VersionSet::next_file_number_ always to be strictly greater than any + // log number + versions_->MarkFileNumberUsedDuringRecovery(max_log_number + 1); + status = versions_->LogAndApply( + cfd, *cfd->GetLatestMutableCFOptions(), edit, &mutex_); + if (!status.ok()) { + // Recovery failed + break; + } + } + } + + if (data_seen && !flushed) { + // Mark these as alive so they'll be considered for deletion later by + // FindObsoleteFiles() + if (concurrent_prepare_) { + log_write_mutex_.Lock(); + } + for (auto log_number : log_numbers) { + alive_log_files_.push_back(LogFileNumberSize(log_number)); + } + if (concurrent_prepare_) { + log_write_mutex_.Unlock(); + } + } + + event_logger_.Log() << "job" << job_id << "event" + << "recovery_finished"; + + return status; +} + +Status DBImpl::WriteLevel0TableForRecovery(int job_id, ColumnFamilyData* cfd, + MemTable* mem, VersionEdit* edit) { + mutex_.AssertHeld(); + const uint64_t start_micros = env_->NowMicros(); + FileMetaData meta; + auto pending_outputs_inserted_elem = + CaptureCurrentFileNumberInPendingOutputs(); + meta.fd = FileDescriptor(versions_->NewFileNumber(), 0, 0); + ReadOptions ro; + ro.total_order_seek = true; + Arena arena; + Status s; + TableProperties table_properties; + { + ScopedArenaIterator iter(mem->NewIterator(ro, &arena)); + ROCKS_LOG_DEBUG(immutable_db_options_.info_log, + "[%s] [WriteLevel0TableForRecovery]" + " Level-0 table #%" PRIu64 ": started", + cfd->GetName().c_str(), meta.fd.GetNumber()); + + // Get the latest mutable cf options while the mutex is still locked + const MutableCFOptions mutable_cf_options = + *cfd->GetLatestMutableCFOptions(); + bool paranoid_file_checks = + cfd->GetLatestMutableCFOptions()->paranoid_file_checks; + + int64_t _current_time = 0; + env_->GetCurrentTime(&_current_time); // ignore error + const uint64_t current_time = static_cast(_current_time); + + { + mutex_.Unlock(); + + SequenceNumber earliest_write_conflict_snapshot; + std::vector snapshot_seqs = + snapshots_.GetAll(&earliest_write_conflict_snapshot); + + EnvOptions optimized_env_options = + env_->OptimizeForCompactionTableWrite(env_options_, immutable_db_options_); + s = BuildTable( + dbname_, env_, *cfd->ioptions(), mutable_cf_options, + optimized_env_options, cfd->table_cache(), iter.get(), + std::unique_ptr(mem->NewRangeTombstoneIterator(ro)), + &meta, cfd->internal_comparator(), + cfd->int_tbl_prop_collector_factories(), cfd->GetID(), cfd->GetName(), + snapshot_seqs, earliest_write_conflict_snapshot, + GetCompressionFlush(*cfd->ioptions(), mutable_cf_options), + cfd->ioptions()->compression_opts, paranoid_file_checks, + cfd->internal_stats(), TableFileCreationReason::kRecovery, + &event_logger_, job_id, Env::IO_HIGH, nullptr /* table_properties */, + -1 /* level */, current_time); + LogFlush(immutable_db_options_.info_log); + ROCKS_LOG_DEBUG(immutable_db_options_.info_log, + "[%s] [WriteLevel0TableForRecovery]" + " Level-0 table #%" PRIu64 ": %" PRIu64 " bytes %s", + cfd->GetName().c_str(), meta.fd.GetNumber(), + meta.fd.GetFileSize(), s.ToString().c_str()); + mutex_.Lock(); + } + } + ReleaseFileNumberFromPendingOutputs(pending_outputs_inserted_elem); + + // Note that if file_size is zero, the file has been deleted and + // should not be added to the manifest. + int level = 0; + if (s.ok() && meta.fd.GetFileSize() > 0) { + edit->AddFile(level, meta.fd.GetNumber(), meta.fd.GetPathId(), + meta.fd.GetFileSize(), meta.smallest, meta.largest, + meta.smallest_seqno, meta.largest_seqno, + meta.marked_for_compaction); + } + + InternalStats::CompactionStats stats(1); + stats.micros = env_->NowMicros() - start_micros; + stats.bytes_written = meta.fd.GetFileSize(); + stats.num_output_files = 1; + cfd->internal_stats()->AddCompactionStats(level, stats); + cfd->internal_stats()->AddCFStats( + InternalStats::BYTES_FLUSHED, meta.fd.GetFileSize()); + RecordTick(stats_, COMPACT_WRITE_BYTES, meta.fd.GetFileSize()); + return s; +} + +Status DB::Open(const Options& options, const std::string& dbname, DB** dbptr) { + DBOptions db_options(options); + ColumnFamilyOptions cf_options(options); + std::vector column_families; + column_families.push_back( + ColumnFamilyDescriptor(kDefaultColumnFamilyName, cf_options)); + std::vector handles; + Status s = DB::Open(db_options, dbname, column_families, &handles, dbptr); + if (s.ok()) { + assert(handles.size() == 1); + // i can delete the handle since DBImpl is always holding a reference to + // default column family + delete handles[0]; + } + return s; +} + +Status DB::Open(const DBOptions& db_options, const std::string& dbname, + const std::vector& column_families, + std::vector* handles, DB** dbptr) { + Status s = SanitizeOptionsByTable(db_options, column_families); + if (!s.ok()) { + return s; + } + + s = ValidateOptions(db_options, column_families); + if (!s.ok()) { + return s; + } + + *dbptr = nullptr; + handles->clear(); + + size_t max_write_buffer_size = 0; + for (auto cf : column_families) { + max_write_buffer_size = + std::max(max_write_buffer_size, cf.options.write_buffer_size); + } + + DBImpl* impl = new DBImpl(db_options, dbname); + s = impl->env_->CreateDirIfMissing(impl->immutable_db_options_.wal_dir); + if (s.ok()) { + for (auto db_path : impl->immutable_db_options_.db_paths) { + s = impl->env_->CreateDirIfMissing(db_path.path); + if (!s.ok()) { + break; + } + } + } + + if (!s.ok()) { + delete impl; + return s; + } + + s = impl->CreateArchivalDirectory(); + if (!s.ok()) { + delete impl; + return s; + } + impl->mutex_.Lock(); + // Handles create_if_missing, error_if_exists + s = impl->Recover(column_families); + if (s.ok()) { + uint64_t new_log_number = impl->versions_->NewFileNumber(); + unique_ptr lfile; + EnvOptions soptions(db_options); + EnvOptions opt_env_options = + impl->immutable_db_options_.env->OptimizeForLogWrite( + soptions, BuildDBOptions(impl->immutable_db_options_, + impl->mutable_db_options_)); + s = NewWritableFile( + impl->immutable_db_options_.env, + LogFileName(impl->immutable_db_options_.wal_dir, new_log_number), + &lfile, opt_env_options); + if (s.ok()) { + lfile->SetPreallocationBlockSize( + impl->GetWalPreallocateBlockSize(max_write_buffer_size)); + { + InstrumentedMutexLock wl(&impl->log_write_mutex_); + impl->logfile_number_ = new_log_number; + unique_ptr file_writer( + new WritableFileWriter(std::move(lfile), opt_env_options)); + impl->logs_.emplace_back( + new_log_number, + new log::Writer( + std::move(file_writer), new_log_number, + impl->immutable_db_options_.recycle_log_file_num > 0)); + } + + // set column family handles + for (auto cf : column_families) { + auto cfd = + impl->versions_->GetColumnFamilySet()->GetColumnFamily(cf.name); + if (cfd != nullptr) { + handles->push_back( + new ColumnFamilyHandleImpl(cfd, impl, &impl->mutex_)); + impl->NewThreadStatusCfInfo(cfd); + } else { + if (db_options.create_missing_column_families) { + // missing column family, create it + ColumnFamilyHandle* handle; + impl->mutex_.Unlock(); + s = impl->CreateColumnFamily(cf.options, cf.name, &handle); + impl->mutex_.Lock(); + if (s.ok()) { + handles->push_back(handle); + } else { + break; + } + } else { + s = Status::InvalidArgument("Column family not found: ", cf.name); + break; + } + } + } + } + if (s.ok()) { + for (auto cfd : *impl->versions_->GetColumnFamilySet()) { + delete impl->InstallSuperVersionAndScheduleWork( + cfd, nullptr, *cfd->GetLatestMutableCFOptions()); + } + if (impl->concurrent_prepare_) { + impl->log_write_mutex_.Lock(); + } + impl->alive_log_files_.push_back( + DBImpl::LogFileNumberSize(impl->logfile_number_)); + if (impl->concurrent_prepare_) { + impl->log_write_mutex_.Unlock(); + } + impl->DeleteObsoleteFiles(); + s = impl->directories_.GetDbDir()->Fsync(); + } + } + + if (s.ok()) { + for (auto cfd : *impl->versions_->GetColumnFamilySet()) { + if (cfd->ioptions()->compaction_style == kCompactionStyleFIFO) { + auto* vstorage = cfd->current()->storage_info(); + for (int i = 1; i < vstorage->num_levels(); ++i) { + int num_files = vstorage->NumLevelFiles(i); + if (num_files > 0) { + s = Status::InvalidArgument( + "Not all files are at level 0. Cannot " + "open with FIFO compaction style."); + break; + } + } + } + if (!cfd->mem()->IsSnapshotSupported()) { + impl->is_snapshot_supported_ = false; + } + if (cfd->ioptions()->merge_operator != nullptr && + !cfd->mem()->IsMergeOperatorSupported()) { + s = Status::InvalidArgument( + "The memtable of column family %s does not support merge operator " + "its options.merge_operator is non-null", cfd->GetName().c_str()); + } + if (!s.ok()) { + break; + } + } + } + TEST_SYNC_POINT("DBImpl::Open:Opened"); + Status persist_options_status; + if (s.ok()) { + // Persist RocksDB Options before scheduling the compaction. + // The WriteOptionsFile() will release and lock the mutex internally. + persist_options_status = impl->WriteOptionsFile( + false /*need_mutex_lock*/, false /*need_enter_write_thread*/); + + *dbptr = impl; + impl->opened_successfully_ = true; + impl->MaybeScheduleFlushOrCompaction(); + } + impl->mutex_.Unlock(); + +#ifndef ROCKSDB_LITE + auto sfm = static_cast( + impl->immutable_db_options_.sst_file_manager.get()); + if (s.ok() && sfm) { + // Notify SstFileManager about all sst files that already exist in + // db_paths[0] when the DB is opened. + auto& db_path = impl->immutable_db_options_.db_paths[0]; + std::vector existing_files; + impl->immutable_db_options_.env->GetChildren(db_path.path, &existing_files); + for (auto& file_name : existing_files) { + uint64_t file_number; + FileType file_type; + std::string file_path = db_path.path + "/" + file_name; + if (ParseFileName(file_name, &file_number, &file_type) && + file_type == kTableFile) { + sfm->OnAddFile(file_path); + } + } + } +#endif // !ROCKSDB_LITE + + if (s.ok()) { + ROCKS_LOG_INFO(impl->immutable_db_options_.info_log, "DB pointer %p", impl); + LogFlush(impl->immutable_db_options_.info_log); + if (!persist_options_status.ok()) { + s = Status::IOError( + "DB::Open() failed --- Unable to persist Options file", + persist_options_status.ToString()); + } + } + if (!s.ok()) { + for (auto* h : *handles) { + delete h; + } + handles->clear(); + delete impl; + *dbptr = nullptr; + } + return s; +} +} // namespace rocksdb diff --git a/db/db_impl_readonly.cc b/db/db_impl_readonly.cc index 6c864aefd80..d69eecb988f 100644 --- a/db/db_impl_readonly.cc +++ b/db/db_impl_readonly.cc @@ -1,93 +1,79 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Copyright (c) 2012 Facebook. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #include "db/db_impl_readonly.h" -#include "db/db_impl.h" -#include -#include -#include -#include -#include -#include +#include "db/compacted_db_impl.h" +#include "db/db_impl.h" #include "db/db_iter.h" -#include "db/dbformat.h" -#include "db/filename.h" -#include "db/log_reader.h" -#include "db/log_writer.h" -#include "db/memtable.h" #include "db/merge_context.h" -#include "db/table_cache.h" -#include "db/version_set.h" -#include "db/write_batch_internal.h" -#include "rocksdb/db.h" -#include "rocksdb/env.h" -#include "rocksdb/status.h" -#include "rocksdb/table.h" -#include "rocksdb/merge_operator.h" -#include "port/port.h" -#include "table/block.h" -#include "table/merger.h" -#include "table/two_level_iterator.h" -#include "util/coding.h" -#include "util/logging.h" -#include "util/build_version.h" +#include "db/range_del_aggregator.h" +#include "monitoring/perf_context_imp.h" namespace rocksdb { -DBImplReadOnly::DBImplReadOnly(const DBOptions& options, +#ifndef ROCKSDB_LITE + +DBImplReadOnly::DBImplReadOnly(const DBOptions& db_options, const std::string& dbname) - : DBImpl(options, dbname) { - Log(options_.info_log, "Opening the db in read only mode"); + : DBImpl(db_options, dbname) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "Opening the db in read only mode"); + LogFlush(immutable_db_options_.info_log); } DBImplReadOnly::~DBImplReadOnly() { } // Implementations of the DB interface -Status DBImplReadOnly::Get(const ReadOptions& options, +Status DBImplReadOnly::Get(const ReadOptions& read_options, ColumnFamilyHandle* column_family, const Slice& key, - std::string* value) { + PinnableSlice* pinnable_val) { + assert(pinnable_val != nullptr); Status s; SequenceNumber snapshot = versions_->LastSequence(); auto cfh = reinterpret_cast(column_family); auto cfd = cfh->cfd(); SuperVersion* super_version = cfd->GetSuperVersion(); MergeContext merge_context; + RangeDelAggregator range_del_agg(cfd->internal_comparator(), snapshot); LookupKey lkey(key, snapshot); - if (super_version->mem->Get(lkey, value, &s, merge_context, - *cfd->options())) { + if (super_version->mem->Get(lkey, pinnable_val->GetSelf(), &s, &merge_context, + &range_del_agg, read_options)) { + pinnable_val->PinSelf(); } else { - super_version->current->Get(options, lkey, value, &s, &merge_context); + PERF_TIMER_GUARD(get_from_output_files_time); + super_version->current->Get(read_options, lkey, pinnable_val, &s, + &merge_context, &range_del_agg); } return s; } -Iterator* DBImplReadOnly::NewIterator(const ReadOptions& options, +Iterator* DBImplReadOnly::NewIterator(const ReadOptions& read_options, ColumnFamilyHandle* column_family) { auto cfh = reinterpret_cast(column_family); auto cfd = cfh->cfd(); SuperVersion* super_version = cfd->GetSuperVersion()->Ref(); SequenceNumber latest_snapshot = versions_->LastSequence(); auto db_iter = NewArenaWrappedDbIterator( - env_, *cfd->options(), cfd->user_comparator(), - (options.snapshot != nullptr - ? reinterpret_cast(options.snapshot)->number_ - : latest_snapshot)); + env_, read_options, *cfd->ioptions(), + (read_options.snapshot != nullptr + ? reinterpret_cast(read_options.snapshot) + ->number_ + : latest_snapshot), + super_version->mutable_cf_options.max_sequential_skip_in_iterations, + super_version->version_number); auto internal_iter = - NewInternalIterator(options, cfd, super_version, db_iter->GetArena()); + NewInternalIterator(read_options, cfd, super_version, db_iter->GetArena(), + db_iter->GetRangeDelAggregator()); db_iter->SetIterUnderDBIter(internal_iter); return db_iter; } Status DBImplReadOnly::NewIterators( - const ReadOptions& options, + const ReadOptions& read_options, const std::vector& column_families, std::vector* iterators) { if (iterators == nullptr) { @@ -98,14 +84,19 @@ Status DBImplReadOnly::NewIterators( SequenceNumber latest_snapshot = versions_->LastSequence(); for (auto cfh : column_families) { - auto cfd = reinterpret_cast(cfh)->cfd(); - auto db_iter = NewArenaWrappedDbIterator( - env_, *cfd->options(), cfd->user_comparator(), - options.snapshot != nullptr - ? reinterpret_cast(options.snapshot)->number_ - : latest_snapshot); - auto internal_iter = NewInternalIterator( - options, cfd, cfd->GetSuperVersion()->Ref(), db_iter->GetArena()); + auto* cfd = reinterpret_cast(cfh)->cfd(); + auto* sv = cfd->GetSuperVersion()->Ref(); + auto* db_iter = NewArenaWrappedDbIterator( + env_, read_options, *cfd->ioptions(), + (read_options.snapshot != nullptr + ? reinterpret_cast(read_options.snapshot) + ->number_ + : latest_snapshot), + sv->mutable_cf_options.max_sequential_skip_in_iterations, + sv->version_number); + auto* internal_iter = + NewInternalIterator(read_options, cfd, sv, db_iter->GetArena(), + db_iter->GetRangeDelAggregator()); db_iter->SetIterUnderDBIter(internal_iter); iterators->push_back(db_iter); } @@ -117,6 +108,13 @@ Status DB::OpenForReadOnly(const Options& options, const std::string& dbname, DB** dbptr, bool error_if_log_file_exist) { *dbptr = nullptr; + // Try to first open DB as fully compacted DB + Status s; + s = CompactedDBImpl::Open(options, dbname, dbptr); + if (s.ok()) { + return s; + } + DBOptions db_options(options); ColumnFamilyOptions cf_options(options); std::vector column_families; @@ -124,8 +122,7 @@ Status DB::OpenForReadOnly(const Options& options, const std::string& dbname, ColumnFamilyDescriptor(kDefaultColumnFamilyName, cf_options)); std::vector handles; - Status s = - DB::OpenForReadOnly(db_options, dbname, column_families, &handles, dbptr); + s = DB::OpenForReadOnly(db_options, dbname, column_families, &handles, dbptr); if (s.ok()) { assert(handles.size() == 1); // i can delete the handle since DBImpl is always holding a @@ -167,6 +164,10 @@ Status DB::OpenForReadOnly( impl->mutex_.Unlock(); if (s.ok()) { *dbptr = impl; + for (auto* h : *handles) { + impl->NewThreadStatusCfInfo( + reinterpret_cast(h)->cfd()); + } } else { for (auto h : *handles) { delete h; @@ -177,5 +178,20 @@ Status DB::OpenForReadOnly( return s; } +#else // !ROCKSDB_LITE + +Status DB::OpenForReadOnly(const Options& options, const std::string& dbname, + DB** dbptr, bool error_if_log_file_exist) { + return Status::NotSupported("Not supported in ROCKSDB_LITE."); +} + +Status DB::OpenForReadOnly( + const DBOptions& db_options, const std::string& dbname, + const std::vector& column_families, + std::vector* handles, DB** dbptr, + bool error_if_log_file_exist) { + return Status::NotSupported("Not supported in ROCKSDB_LITE."); +} +#endif // !ROCKSDB_LITE } // namespace rocksdb diff --git a/db/db_impl_readonly.h b/db/db_impl_readonly.h index 1dfdf422ef7..9bdc95cc874 100644 --- a/db/db_impl_readonly.h +++ b/db/db_impl_readonly.h @@ -1,25 +1,15 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Copyright (c) 2012 Facebook. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once -#include "db/db_impl.h" -#include -#include +#ifndef ROCKSDB_LITE + +#include "db/db_impl.h" #include #include -#include "db/dbformat.h" -#include "db/log_writer.h" -#include "db/snapshot.h" -#include "rocksdb/db.h" -#include "rocksdb/env.h" -#include "port/port.h" namespace rocksdb { @@ -32,7 +22,7 @@ class DBImplReadOnly : public DBImpl { using DB::Get; virtual Status Get(const ReadOptions& options, ColumnFamilyHandle* column_family, const Slice& key, - std::string* value) override; + PinnableSlice* value) override; // TODO: Implement ReadOnly MultiGet? @@ -63,22 +53,36 @@ class DBImplReadOnly : public DBImpl { const Slice& key) override { return Status::NotSupported("Not supported operation in read only mode."); } + using DBImpl::SingleDelete; + virtual Status SingleDelete(const WriteOptions& options, + ColumnFamilyHandle* column_family, + const Slice& key) override { + return Status::NotSupported("Not supported operation in read only mode."); + } virtual Status Write(const WriteOptions& options, WriteBatch* updates) override { return Status::NotSupported("Not supported operation in read only mode."); } using DBImpl::CompactRange; - virtual Status CompactRange(ColumnFamilyHandle* column_family, - const Slice* begin, const Slice* end, - bool reduce_level = false, int target_level = -1, - uint32_t target_path_id = 0) override { + virtual Status CompactRange(const CompactRangeOptions& options, + ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) override { + return Status::NotSupported("Not supported operation in read only mode."); + } + + using DBImpl::CompactFiles; + virtual Status CompactFiles( + const CompactionOptions& compact_options, + ColumnFamilyHandle* column_family, + const std::vector& input_file_names, + const int output_level, const int output_path_id = -1) override { return Status::NotSupported("Not supported operation in read only mode."); } -#ifndef ROCKSDB_LITE virtual Status DisableFileDeletions() override { return Status::NotSupported("Not supported operation in read only mode."); } + virtual Status EnableFileDeletions(bool force) override { return Status::NotSupported("Not supported operation in read only mode."); } @@ -87,7 +91,6 @@ class DBImplReadOnly : public DBImpl { bool flush_memtable = true) override { return Status::NotSupported("Not supported operation in read only mode."); } -#endif // ROCKSDB_LITE using DBImpl::Flush; virtual Status Flush(const FlushOptions& options, @@ -95,6 +98,19 @@ class DBImplReadOnly : public DBImpl { return Status::NotSupported("Not supported operation in read only mode."); } + using DBImpl::SyncWAL; + virtual Status SyncWAL() override { + return Status::NotSupported("Not supported operation in read only mode."); + } + + using DB::IngestExternalFile; + virtual Status IngestExternalFile( + ColumnFamilyHandle* column_family, + const std::vector& external_files, + const IngestExternalFileOptions& ingestion_options) override { + return Status::NotSupported("Not supported operation in read only mode."); + } + private: friend class DB; @@ -103,3 +119,5 @@ class DBImplReadOnly : public DBImpl { void operator=(const DBImplReadOnly&); }; } + +#endif // !ROCKSDB_LITE diff --git a/db/db_impl_write.cc b/db/db_impl_write.cc new file mode 100644 index 00000000000..2b06c7d710c --- /dev/null +++ b/db/db_impl_write.cc @@ -0,0 +1,1263 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#include "db/db_impl.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif +#include +#include "db/event_helpers.h" +#include "monitoring/perf_context_imp.h" +#include "options/options_helper.h" +#include "util/sync_point.h" + +namespace rocksdb { +// Convenience methods +Status DBImpl::Put(const WriteOptions& o, ColumnFamilyHandle* column_family, + const Slice& key, const Slice& val) { + return DB::Put(o, column_family, key, val); +} + +Status DBImpl::Merge(const WriteOptions& o, ColumnFamilyHandle* column_family, + const Slice& key, const Slice& val) { + auto cfh = reinterpret_cast(column_family); + if (!cfh->cfd()->ioptions()->merge_operator) { + return Status::NotSupported("Provide a merge_operator when opening DB"); + } else { + return DB::Merge(o, column_family, key, val); + } +} + +Status DBImpl::Delete(const WriteOptions& write_options, + ColumnFamilyHandle* column_family, const Slice& key) { + return DB::Delete(write_options, column_family, key); +} + +Status DBImpl::SingleDelete(const WriteOptions& write_options, + ColumnFamilyHandle* column_family, + const Slice& key) { + return DB::SingleDelete(write_options, column_family, key); +} + +Status DBImpl::Write(const WriteOptions& write_options, WriteBatch* my_batch) { + return WriteImpl(write_options, my_batch, nullptr, nullptr); +} + +#ifndef ROCKSDB_LITE +Status DBImpl::WriteWithCallback(const WriteOptions& write_options, + WriteBatch* my_batch, + WriteCallback* callback) { + return WriteImpl(write_options, my_batch, callback, nullptr); +} +#endif // ROCKSDB_LITE + +Status DBImpl::WriteImpl(const WriteOptions& write_options, + WriteBatch* my_batch, WriteCallback* callback, + uint64_t* log_used, uint64_t log_ref, + bool disable_memtable, uint64_t* seq_used) { + if (my_batch == nullptr) { + return Status::Corruption("Batch is nullptr!"); + } + if (concurrent_prepare_ && immutable_db_options_.enable_pipelined_write) { + return Status::NotSupported( + "pipelined_writes is not compatible with concurrent prepares"); + } + + Status status; + if (write_options.low_pri) { + status = ThrottleLowPriWritesIfNeeded(write_options, my_batch); + if (!status.ok()) { + return status; + } + } + + if (concurrent_prepare_ && disable_memtable) { + return WriteImplWALOnly(write_options, my_batch, callback, log_used, + log_ref, seq_used); + } + + if (immutable_db_options_.enable_pipelined_write) { + return PipelinedWriteImpl(write_options, my_batch, callback, log_used, + log_ref, disable_memtable, seq_used); + } + + PERF_TIMER_GUARD(write_pre_and_post_process_time); + WriteThread::Writer w(write_options, my_batch, callback, log_ref, + disable_memtable); + + if (!write_options.disableWAL) { + RecordTick(stats_, WRITE_WITH_WAL); + } + + StopWatch write_sw(env_, immutable_db_options_.statistics.get(), DB_WRITE); + + write_thread_.JoinBatchGroup(&w); + if (w.state == WriteThread::STATE_PARALLEL_MEMTABLE_WRITER) { + // we are a non-leader in a parallel group + PERF_TIMER_GUARD(write_memtable_time); + + if (w.ShouldWriteToMemtable()) { + ColumnFamilyMemTablesImpl column_family_memtables( + versions_->GetColumnFamilySet()); + w.status = WriteBatchInternal::InsertInto( + &w, w.sequence, &column_family_memtables, &flush_scheduler_, + write_options.ignore_missing_column_families, 0 /*log_number*/, this, + true /*concurrent_memtable_writes*/); + } + + if (write_thread_.CompleteParallelMemTableWriter(&w)) { + // we're responsible for exit batch group + auto last_sequence = w.write_group->last_sequence; + versions_->SetLastSequence(last_sequence); + MemTableInsertStatusCheck(w.status); + write_thread_.ExitAsBatchGroupFollower(&w); + } + assert(w.state == WriteThread::STATE_COMPLETED); + // STATE_COMPLETED conditional below handles exit + + status = w.FinalStatus(); + } + if (w.state == WriteThread::STATE_COMPLETED) { + if (log_used != nullptr) { + *log_used = w.log_used; + } + if (seq_used != nullptr) { + *seq_used = w.sequence; + } + // write is complete and leader has updated sequence + return w.FinalStatus(); + } + // else we are the leader of the write batch group + assert(w.state == WriteThread::STATE_GROUP_LEADER); + + // Once reaches this point, the current writer "w" will try to do its write + // job. It may also pick up some of the remaining writers in the "writers_" + // when it finds suitable, and finish them in the same write batch. + // This is how a write job could be done by the other writer. + WriteContext write_context; + WriteThread::WriteGroup write_group; + bool in_parallel_group = false; + uint64_t last_sequence = kMaxSequenceNumber; + if (!concurrent_prepare_) { + last_sequence = versions_->LastSequence(); + } + + mutex_.Lock(); + + bool need_log_sync = !write_options.disableWAL && write_options.sync; + bool need_log_dir_sync = need_log_sync && !log_dir_synced_; + if (!concurrent_prepare_ || !disable_memtable) { + // With concurrent writes we do preprocess only in the write thread that + // also does write to memtable to avoid sync issue on shared data structure + // with the other thread + status = PreprocessWrite(write_options, &need_log_sync, &write_context); + } + log::Writer* log_writer = logs_.back().writer; + + mutex_.Unlock(); + + // Add to log and apply to memtable. We can release the lock + // during this phase since &w is currently responsible for logging + // and protects against concurrent loggers and concurrent writes + // into memtables + + last_batch_group_size_ = + write_thread_.EnterAsBatchGroupLeader(&w, &write_group); + + if (status.ok()) { + // Rules for when we can update the memtable concurrently + // 1. supported by memtable + // 2. Puts are not okay if inplace_update_support + // 3. Merges are not okay + // + // Rules 1..2 are enforced by checking the options + // during startup (CheckConcurrentWritesSupported), so if + // options.allow_concurrent_memtable_write is true then they can be + // assumed to be true. Rule 3 is checked for each batch. We could + // relax rules 2 if we could prevent write batches from referring + // more than once to a particular key. + bool parallel = immutable_db_options_.allow_concurrent_memtable_write && + write_group.size > 1; + int total_count = 0; + uint64_t total_byte_size = 0; + for (auto* writer : write_group) { + if (writer->CheckCallback(this)) { + if (writer->ShouldWriteToMemtable()) { + total_count += WriteBatchInternal::Count(writer->batch); + parallel = parallel && !writer->batch->HasMerge(); + } + + total_byte_size = WriteBatchInternal::AppendedByteSize( + total_byte_size, WriteBatchInternal::ByteSize(writer->batch)); + } + } + + const bool concurrent_update = concurrent_prepare_; + // Update stats while we are an exclusive group leader, so we know + // that nobody else can be writing to these particular stats. + // We're optimistic, updating the stats before we successfully + // commit. That lets us release our leader status early. + auto stats = default_cf_internal_stats_; + stats->AddDBStats(InternalStats::NUMBER_KEYS_WRITTEN, total_count, + concurrent_update); + RecordTick(stats_, NUMBER_KEYS_WRITTEN, total_count); + stats->AddDBStats(InternalStats::BYTES_WRITTEN, total_byte_size, + concurrent_update); + RecordTick(stats_, BYTES_WRITTEN, total_byte_size); + stats->AddDBStats(InternalStats::WRITE_DONE_BY_SELF, 1, concurrent_update); + RecordTick(stats_, WRITE_DONE_BY_SELF); + auto write_done_by_other = write_group.size - 1; + if (write_done_by_other > 0) { + stats->AddDBStats(InternalStats::WRITE_DONE_BY_OTHER, write_done_by_other, + concurrent_update); + RecordTick(stats_, WRITE_DONE_BY_OTHER, write_done_by_other); + } + MeasureTime(stats_, BYTES_PER_WRITE, total_byte_size); + + if (write_options.disableWAL) { + has_unpersisted_data_.store(true, std::memory_order_relaxed); + } + + PERF_TIMER_STOP(write_pre_and_post_process_time); + + if (!concurrent_prepare_) { + if (status.ok() && !write_options.disableWAL) { + PERF_TIMER_GUARD(write_wal_time); + status = WriteToWAL(write_group, log_writer, log_used, need_log_sync, + need_log_dir_sync, last_sequence + 1); + } + } else { + if (status.ok() && !write_options.disableWAL) { + PERF_TIMER_GUARD(write_wal_time); + // LastToBeWrittenSequence is increased inside WriteToWAL under + // wal_write_mutex_ to ensure ordered events in WAL + status = ConcurrentWriteToWAL(write_group, log_used, &last_sequence, + total_count); + } else { + // Otherwise we inc seq number for memtable writes + last_sequence = versions_->FetchAddLastToBeWrittenSequence(total_count); + } + } + assert(last_sequence != kMaxSequenceNumber); + const SequenceNumber current_sequence = last_sequence + 1; + last_sequence += total_count; + + if (status.ok()) { + PERF_TIMER_GUARD(write_memtable_time); + + if (!parallel) { + w.status = WriteBatchInternal::InsertInto( + write_group, current_sequence, column_family_memtables_.get(), + &flush_scheduler_, write_options.ignore_missing_column_families, + 0 /*recovery_log_number*/, this); + } else { + SequenceNumber next_sequence = current_sequence; + for (auto* writer : write_group) { + if (writer->ShouldWriteToMemtable()) { + writer->sequence = next_sequence; + next_sequence += WriteBatchInternal::Count(writer->batch); + } + } + write_group.last_sequence = last_sequence; + write_group.running.store(static_cast(write_group.size), + std::memory_order_relaxed); + write_thread_.LaunchParallelMemTableWriters(&write_group); + in_parallel_group = true; + + // Each parallel follower is doing each own writes. The leader should + // also do its own. + if (w.ShouldWriteToMemtable()) { + ColumnFamilyMemTablesImpl column_family_memtables( + versions_->GetColumnFamilySet()); + assert(w.sequence == current_sequence); + w.status = WriteBatchInternal::InsertInto( + &w, w.sequence, &column_family_memtables, &flush_scheduler_, + write_options.ignore_missing_column_families, 0 /*log_number*/, + this, true /*concurrent_memtable_writes*/); + } + if (seq_used != nullptr) { + *seq_used = w.sequence; + } + } + } + } + PERF_TIMER_START(write_pre_and_post_process_time); + + if (!w.CallbackFailed()) { + WriteCallbackStatusCheck(status); + } + + if (need_log_sync) { + mutex_.Lock(); + MarkLogsSynced(logfile_number_, need_log_dir_sync, status); + mutex_.Unlock(); + // Requesting sync with concurrent_prepare_ is expected to be very rare. We + // hance provide a simple implementation that is not necessarily efficient. + if (concurrent_prepare_) { + if (manual_wal_flush_) { + status = FlushWAL(true); + } else { + status = SyncWAL(); + } + } + } + + bool should_exit_batch_group = true; + if (in_parallel_group) { + // CompleteParallelWorker returns true if this thread should + // handle exit, false means somebody else did + should_exit_batch_group = write_thread_.CompleteParallelMemTableWriter(&w); + } + if (should_exit_batch_group) { + if (status.ok()) { + versions_->SetLastSequence(last_sequence); + } + MemTableInsertStatusCheck(w.status); + write_thread_.ExitAsBatchGroupLeader(write_group, status); + } + + if (status.ok()) { + status = w.FinalStatus(); + } + return status; +} + +Status DBImpl::PipelinedWriteImpl(const WriteOptions& write_options, + WriteBatch* my_batch, WriteCallback* callback, + uint64_t* log_used, uint64_t log_ref, + bool disable_memtable, uint64_t* seq_used) { + PERF_TIMER_GUARD(write_pre_and_post_process_time); + StopWatch write_sw(env_, immutable_db_options_.statistics.get(), DB_WRITE); + + WriteContext write_context; + + WriteThread::Writer w(write_options, my_batch, callback, log_ref, + disable_memtable); + write_thread_.JoinBatchGroup(&w); + if (w.state == WriteThread::STATE_GROUP_LEADER) { + WriteThread::WriteGroup wal_write_group; + if (w.callback && !w.callback->AllowWriteBatching()) { + write_thread_.WaitForMemTableWriters(); + } + mutex_.Lock(); + bool need_log_sync = !write_options.disableWAL && write_options.sync; + bool need_log_dir_sync = need_log_sync && !log_dir_synced_; + w.status = PreprocessWrite(write_options, &need_log_sync, &write_context); + log::Writer* log_writer = logs_.back().writer; + mutex_.Unlock(); + + // This can set non-OK status if callback fail. + last_batch_group_size_ = + write_thread_.EnterAsBatchGroupLeader(&w, &wal_write_group); + const SequenceNumber current_sequence = + write_thread_.UpdateLastSequence(versions_->LastSequence()) + 1; + size_t total_count = 0; + size_t total_byte_size = 0; + + if (w.status.ok()) { + SequenceNumber next_sequence = current_sequence; + for (auto writer : wal_write_group) { + if (writer->CheckCallback(this)) { + if (writer->ShouldWriteToMemtable()) { + writer->sequence = next_sequence; + size_t count = WriteBatchInternal::Count(writer->batch); + next_sequence += count; + total_count += count; + } + total_byte_size = WriteBatchInternal::AppendedByteSize( + total_byte_size, WriteBatchInternal::ByteSize(writer->batch)); + } + } + if (w.disable_wal) { + has_unpersisted_data_.store(true, std::memory_order_relaxed); + } + write_thread_.UpdateLastSequence(current_sequence + total_count - 1); + } + + auto stats = default_cf_internal_stats_; + stats->AddDBStats(InternalStats::NUMBER_KEYS_WRITTEN, total_count); + RecordTick(stats_, NUMBER_KEYS_WRITTEN, total_count); + stats->AddDBStats(InternalStats::BYTES_WRITTEN, total_byte_size); + RecordTick(stats_, BYTES_WRITTEN, total_byte_size); + + PERF_TIMER_STOP(write_pre_and_post_process_time); + + if (w.ShouldWriteToWAL()) { + PERF_TIMER_GUARD(write_wal_time); + stats->AddDBStats(InternalStats::WRITE_DONE_BY_SELF, 1); + RecordTick(stats_, WRITE_DONE_BY_SELF, 1); + if (wal_write_group.size > 1) { + stats->AddDBStats(InternalStats::WRITE_DONE_BY_OTHER, + wal_write_group.size - 1); + RecordTick(stats_, WRITE_DONE_BY_OTHER, wal_write_group.size - 1); + } + w.status = WriteToWAL(wal_write_group, log_writer, log_used, + need_log_sync, need_log_dir_sync, current_sequence); + } + + if (!w.CallbackFailed()) { + WriteCallbackStatusCheck(w.status); + } + + if (need_log_sync) { + mutex_.Lock(); + MarkLogsSynced(logfile_number_, need_log_dir_sync, w.status); + mutex_.Unlock(); + } + + write_thread_.ExitAsBatchGroupLeader(wal_write_group, w.status); + } + + WriteThread::WriteGroup memtable_write_group; + if (w.state == WriteThread::STATE_MEMTABLE_WRITER_LEADER) { + PERF_TIMER_GUARD(write_memtable_time); + assert(w.status.ok()); + write_thread_.EnterAsMemTableWriter(&w, &memtable_write_group); + if (memtable_write_group.size > 1 && + immutable_db_options_.allow_concurrent_memtable_write) { + write_thread_.LaunchParallelMemTableWriters(&memtable_write_group); + } else { + memtable_write_group.status = WriteBatchInternal::InsertInto( + memtable_write_group, w.sequence, column_family_memtables_.get(), + &flush_scheduler_, write_options.ignore_missing_column_families, + 0 /*log_number*/, this); + versions_->SetLastSequence(memtable_write_group.last_sequence); + write_thread_.ExitAsMemTableWriter(&w, memtable_write_group); + } + } + + if (w.state == WriteThread::STATE_PARALLEL_MEMTABLE_WRITER) { + assert(w.ShouldWriteToMemtable()); + ColumnFamilyMemTablesImpl column_family_memtables( + versions_->GetColumnFamilySet()); + w.status = WriteBatchInternal::InsertInto( + &w, w.sequence, &column_family_memtables, &flush_scheduler_, + write_options.ignore_missing_column_families, 0 /*log_number*/, this, + true /*concurrent_memtable_writes*/); + if (write_thread_.CompleteParallelMemTableWriter(&w)) { + MemTableInsertStatusCheck(w.status); + versions_->SetLastSequence(w.write_group->last_sequence); + write_thread_.ExitAsMemTableWriter(&w, *w.write_group); + } + } + if (seq_used != nullptr) { + *seq_used = w.sequence; + } + + assert(w.state == WriteThread::STATE_COMPLETED); + return w.FinalStatus(); +} + +Status DBImpl::WriteImplWALOnly(const WriteOptions& write_options, + WriteBatch* my_batch, WriteCallback* callback, + uint64_t* log_used, uint64_t log_ref, + uint64_t* seq_used) { + Status status; + PERF_TIMER_GUARD(write_pre_and_post_process_time); + WriteThread::Writer w(write_options, my_batch, callback, log_ref, + true /* disable_memtable */); + if (write_options.disableWAL) { + return status; + } + RecordTick(stats_, WRITE_WITH_WAL); + StopWatch write_sw(env_, immutable_db_options_.statistics.get(), DB_WRITE); + + nonmem_write_thread_.JoinBatchGroup(&w); + assert(w.state != WriteThread::STATE_PARALLEL_MEMTABLE_WRITER); + if (w.state == WriteThread::STATE_COMPLETED) { + if (log_used != nullptr) { + *log_used = w.log_used; + } + if (seq_used != nullptr) { + *seq_used = w.sequence; + } + return w.FinalStatus(); + } + // else we are the leader of the write batch group + assert(w.state == WriteThread::STATE_GROUP_LEADER); + WriteContext write_context; + WriteThread::WriteGroup write_group; + uint64_t last_sequence; + nonmem_write_thread_.EnterAsBatchGroupLeader(&w, &write_group); + // Note: no need to update last_batch_group_size_ here since the batch writes + // to WAL only + + uint64_t total_byte_size = 0; + for (auto* writer : write_group) { + if (writer->CheckCallback(this)) { + total_byte_size = WriteBatchInternal::AppendedByteSize( + total_byte_size, WriteBatchInternal::ByteSize(writer->batch)); + } + } + + const bool concurrent_update = true; + // Update stats while we are an exclusive group leader, so we know + // that nobody else can be writing to these particular stats. + // We're optimistic, updating the stats before we successfully + // commit. That lets us release our leader status early. + auto stats = default_cf_internal_stats_; + stats->AddDBStats(InternalStats::BYTES_WRITTEN, total_byte_size, + concurrent_update); + RecordTick(stats_, BYTES_WRITTEN, total_byte_size); + stats->AddDBStats(InternalStats::WRITE_DONE_BY_SELF, 1, concurrent_update); + RecordTick(stats_, WRITE_DONE_BY_SELF); + auto write_done_by_other = write_group.size - 1; + if (write_done_by_other > 0) { + stats->AddDBStats(InternalStats::WRITE_DONE_BY_OTHER, write_done_by_other, + concurrent_update); + RecordTick(stats_, WRITE_DONE_BY_OTHER, write_done_by_other); + } + MeasureTime(stats_, BYTES_PER_WRITE, total_byte_size); + + PERF_TIMER_STOP(write_pre_and_post_process_time); + + PERF_TIMER_GUARD(write_wal_time); + // LastToBeWrittenSequence is increased inside WriteToWAL under + // wal_write_mutex_ to ensure ordered events in WAL + status = ConcurrentWriteToWAL(write_group, log_used, &last_sequence, + 0 /*total_count*/); + auto curr_seq = last_sequence + 1; + for (auto* writer : write_group) { + if (writer->CheckCallback(this)) { + writer->sequence = curr_seq; + curr_seq += WriteBatchInternal::Count(writer->batch); + } + } + if (status.ok() && write_options.sync) { + // Requesting sync with concurrent_prepare_ is expected to be very rare. We + // hance provide a simple implementation that is not necessarily efficient. + if (manual_wal_flush_) { + status = FlushWAL(true); + } else { + status = SyncWAL(); + } + } + PERF_TIMER_START(write_pre_and_post_process_time); + + if (!w.CallbackFailed()) { + WriteCallbackStatusCheck(status); + } + nonmem_write_thread_.ExitAsBatchGroupLeader(write_group, status); + if (status.ok()) { + status = w.FinalStatus(); + } + if (seq_used != nullptr) { + *seq_used = w.sequence; + } + return status; +} + +void DBImpl::WriteCallbackStatusCheck(const Status& status) { + // Is setting bg_error_ enough here? This will at least stop + // compaction and fail any further writes. + if (immutable_db_options_.paranoid_checks && !status.ok() && + !status.IsBusy() && !status.IsIncomplete()) { + mutex_.Lock(); + if (bg_error_.ok()) { + Status new_bg_error = status; + // may temporarily unlock and lock the mutex. + EventHelpers::NotifyOnBackgroundError(immutable_db_options_.listeners, + BackgroundErrorReason::kWriteCallback, + &new_bg_error, &mutex_); + if (!new_bg_error.ok()) { + bg_error_ = new_bg_error; // stop compaction & fail any further writes + } + } + mutex_.Unlock(); + } +} + +void DBImpl::MemTableInsertStatusCheck(const Status& status) { + // A non-OK status here indicates that the state implied by the + // WAL has diverged from the in-memory state. This could be + // because of a corrupt write_batch (very bad), or because the + // client specified an invalid column family and didn't specify + // ignore_missing_column_families. + if (!status.ok()) { + mutex_.Lock(); + assert(bg_error_.ok()); + Status new_bg_error = status; + // may temporarily unlock and lock the mutex. + EventHelpers::NotifyOnBackgroundError(immutable_db_options_.listeners, + BackgroundErrorReason::kMemTable, + &new_bg_error, &mutex_); + if (!new_bg_error.ok()) { + bg_error_ = new_bg_error; // stop compaction & fail any further writes + } + mutex_.Unlock(); + } +} + +Status DBImpl::PreprocessWrite(const WriteOptions& write_options, + bool* need_log_sync, + WriteContext* write_context) { + mutex_.AssertHeld(); + assert(write_context != nullptr && need_log_sync != nullptr); + Status status; + + assert(!single_column_family_mode_ || + versions_->GetColumnFamilySet()->NumberOfColumnFamilies() == 1); + if (UNLIKELY(status.ok() && !single_column_family_mode_ && + total_log_size_ > GetMaxTotalWalSize())) { + status = HandleWALFull(write_context); + } + + if (UNLIKELY(status.ok() && write_buffer_manager_->ShouldFlush())) { + // Before a new memtable is added in SwitchMemtable(), + // write_buffer_manager_->ShouldFlush() will keep returning true. If another + // thread is writing to another DB with the same write buffer, they may also + // be flushed. We may end up with flushing much more DBs than needed. It's + // suboptimal but still correct. + status = HandleWriteBufferFull(write_context); + } + + if (UNLIKELY(status.ok() && !bg_error_.ok())) { + return bg_error_; + } + + if (UNLIKELY(status.ok() && !flush_scheduler_.Empty())) { + status = ScheduleFlushes(write_context); + } + + if (UNLIKELY(status.ok() && (write_controller_.IsStopped() || + write_controller_.NeedsDelay()))) { + PERF_TIMER_GUARD(write_delay_time); + // We don't know size of curent batch so that we always use the size + // for previous one. It might create a fairness issue that expiration + // might happen for smaller writes but larger writes can go through. + // Can optimize it if it is an issue. + status = DelayWrite(last_batch_group_size_, write_options); + } + + if (status.ok() && *need_log_sync) { + // Wait until the parallel syncs are finished. Any sync process has to sync + // the front log too so it is enough to check the status of front() + // We do a while loop since log_sync_cv_ is signalled when any sync is + // finished + // Note: there does not seem to be a reason to wait for parallel sync at + // this early step but it is not important since parallel sync (SyncWAL) and + // need_log_sync are usually not used together. + while (logs_.front().getting_synced) { + log_sync_cv_.Wait(); + } + for (auto& log : logs_) { + assert(!log.getting_synced); + // This is just to prevent the logs to be synced by a parallel SyncWAL + // call. We will do the actual syncing later after we will write to the + // WAL. + // Note: there does not seem to be a reason to set this early before we + // actually write to the WAL + log.getting_synced = true; + } + } else { + *need_log_sync = false; + } + + return status; +} + +WriteBatch* DBImpl::MergeBatch(const WriteThread::WriteGroup& write_group, + WriteBatch* tmp_batch, size_t* write_with_wal) { + assert(write_with_wal != nullptr); + assert(tmp_batch != nullptr); + WriteBatch* merged_batch = nullptr; + *write_with_wal = 0; + auto* leader = write_group.leader; + if (write_group.size == 1 && leader->ShouldWriteToWAL() && + leader->batch->GetWalTerminationPoint().is_cleared()) { + // we simply write the first WriteBatch to WAL if the group only + // contains one batch, that batch should be written to the WAL, + // and the batch is not wanting to be truncated + merged_batch = leader->batch; + *write_with_wal = 1; + } else { + // WAL needs all of the batches flattened into a single batch. + // We could avoid copying here with an iov-like AddRecord + // interface + merged_batch = tmp_batch; + for (auto writer : write_group) { + if (writer->ShouldWriteToWAL()) { + WriteBatchInternal::Append(merged_batch, writer->batch, + /*WAL_only*/ true); + (*write_with_wal)++; + } + } + } + return merged_batch; +} + +// When concurrent_prepare_ is disabled, this function is called from the only +// write thread. Otherwise this must be called holding log_write_mutex_. +Status DBImpl::WriteToWAL(const WriteBatch& merged_batch, + log::Writer* log_writer, uint64_t* log_used, + uint64_t* log_size) { + assert(log_size != nullptr); + Slice log_entry = WriteBatchInternal::Contents(&merged_batch); + *log_size = log_entry.size(); + Status status = log_writer->AddRecord(log_entry); + if (log_used != nullptr) { + *log_used = logfile_number_; + } + total_log_size_ += log_entry.size(); + // TODO(myabandeh): it might be unsafe to access alive_log_files_.back() here + // since alive_log_files_ might be modified concurrently + alive_log_files_.back().AddSize(log_entry.size()); + log_empty_ = false; + return status; +} + +Status DBImpl::WriteToWAL(const WriteThread::WriteGroup& write_group, + log::Writer* log_writer, uint64_t* log_used, + bool need_log_sync, bool need_log_dir_sync, + SequenceNumber sequence) { + Status status; + + size_t write_with_wal = 0; + WriteBatch* merged_batch = + MergeBatch(write_group, &tmp_batch_, &write_with_wal); + if (merged_batch == write_group.leader->batch) { + write_group.leader->log_used = logfile_number_; + } else if (write_with_wal > 1) { + for (auto writer : write_group) { + writer->log_used = logfile_number_; + } + } + + WriteBatchInternal::SetSequence(merged_batch, sequence); + + uint64_t log_size; + status = WriteToWAL(*merged_batch, log_writer, log_used, &log_size); + + if (status.ok() && need_log_sync) { + StopWatch sw(env_, stats_, WAL_FILE_SYNC_MICROS); + // It's safe to access logs_ with unlocked mutex_ here because: + // - we've set getting_synced=true for all logs, + // so other threads won't pop from logs_ while we're here, + // - only writer thread can push to logs_, and we're in + // writer thread, so no one will push to logs_, + // - as long as other threads don't modify it, it's safe to read + // from std::deque from multiple threads concurrently. + for (auto& log : logs_) { + status = log.writer->file()->Sync(immutable_db_options_.use_fsync); + if (!status.ok()) { + break; + } + } + if (status.ok() && need_log_dir_sync) { + // We only sync WAL directory the first time WAL syncing is + // requested, so that in case users never turn on WAL sync, + // we can avoid the disk I/O in the write code path. + status = directories_.GetWalDir()->Fsync(); + } + } + + if (merged_batch == &tmp_batch_) { + tmp_batch_.Clear(); + } + if (status.ok()) { + auto stats = default_cf_internal_stats_; + if (need_log_sync) { + stats->AddDBStats(InternalStats::WAL_FILE_SYNCED, 1); + RecordTick(stats_, WAL_FILE_SYNCED); + } + stats->AddDBStats(InternalStats::WAL_FILE_BYTES, log_size); + RecordTick(stats_, WAL_FILE_BYTES, log_size); + stats->AddDBStats(InternalStats::WRITE_WITH_WAL, write_with_wal); + RecordTick(stats_, WRITE_WITH_WAL, write_with_wal); + } + return status; +} + +Status DBImpl::ConcurrentWriteToWAL(const WriteThread::WriteGroup& write_group, + uint64_t* log_used, + SequenceNumber* last_sequence, + int total_count) { + Status status; + + WriteBatch tmp_batch; + size_t write_with_wal = 0; + WriteBatch* merged_batch = + MergeBatch(write_group, &tmp_batch, &write_with_wal); + + // We need to lock log_write_mutex_ since logs_ and alive_log_files might be + // pushed back concurrently + log_write_mutex_.Lock(); + if (merged_batch == write_group.leader->batch) { + write_group.leader->log_used = logfile_number_; + } else if (write_with_wal > 1) { + for (auto writer : write_group) { + writer->log_used = logfile_number_; + } + } + *last_sequence = versions_->FetchAddLastToBeWrittenSequence(total_count); + auto sequence = *last_sequence + 1; + WriteBatchInternal::SetSequence(merged_batch, sequence); + + log::Writer* log_writer = logs_.back().writer; + uint64_t log_size; + status = WriteToWAL(*merged_batch, log_writer, log_used, &log_size); + log_write_mutex_.Unlock(); + + if (status.ok()) { + const bool concurrent = true; + auto stats = default_cf_internal_stats_; + stats->AddDBStats(InternalStats::WAL_FILE_BYTES, log_size, concurrent); + RecordTick(stats_, WAL_FILE_BYTES, log_size); + stats->AddDBStats(InternalStats::WRITE_WITH_WAL, write_with_wal, + concurrent); + RecordTick(stats_, WRITE_WITH_WAL, write_with_wal); + } + return status; +} + +Status DBImpl::HandleWALFull(WriteContext* write_context) { + mutex_.AssertHeld(); + assert(write_context != nullptr); + Status status; + + if (alive_log_files_.begin()->getting_flushed) { + return status; + } + + auto oldest_alive_log = alive_log_files_.begin()->number; + auto oldest_log_with_uncommited_prep = FindMinLogContainingOutstandingPrep(); + + if (allow_2pc() && + oldest_log_with_uncommited_prep > 0 && + oldest_log_with_uncommited_prep <= oldest_alive_log) { + if (unable_to_flush_oldest_log_) { + // we already attempted to flush all column families dependent on + // the oldest alive log but the log still contained uncommited transactions. + // the oldest alive log STILL contains uncommited transaction so there + // is still nothing that we can do. + return status; + } else { + ROCKS_LOG_WARN( + immutable_db_options_.info_log, + "Unable to release oldest log due to uncommited transaction"); + unable_to_flush_oldest_log_ = true; + } + } else { + // we only mark this log as getting flushed if we have successfully + // flushed all data in this log. If this log contains outstanding prepared + // transactions then we cannot flush this log until those transactions are commited. + unable_to_flush_oldest_log_ = false; + alive_log_files_.begin()->getting_flushed = true; + } + + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "Flushing all column families with data in WAL number %" PRIu64 + ". Total log size is %" PRIu64 + " while max_total_wal_size is %" PRIu64, + oldest_alive_log, total_log_size_.load(), GetMaxTotalWalSize()); + // no need to refcount because drop is happening in write thread, so can't + // happen while we're in the write thread + for (auto cfd : *versions_->GetColumnFamilySet()) { + if (cfd->IsDropped()) { + continue; + } + if (cfd->OldestLogToKeep() <= oldest_alive_log) { + status = SwitchMemtable(cfd, write_context); + if (!status.ok()) { + break; + } + cfd->imm()->FlushRequested(); + SchedulePendingFlush(cfd); + } + } + MaybeScheduleFlushOrCompaction(); + return status; +} + +Status DBImpl::HandleWriteBufferFull(WriteContext* write_context) { + mutex_.AssertHeld(); + assert(write_context != nullptr); + Status status; + + // Before a new memtable is added in SwitchMemtable(), + // write_buffer_manager_->ShouldFlush() will keep returning true. If another + // thread is writing to another DB with the same write buffer, they may also + // be flushed. We may end up with flushing much more DBs than needed. It's + // suboptimal but still correct. + ROCKS_LOG_INFO( + immutable_db_options_.info_log, + "Flushing column family with largest mem table size. Write buffer is " + "using %" PRIu64 " bytes out of a total of %" PRIu64 ".", + write_buffer_manager_->memory_usage(), + write_buffer_manager_->buffer_size()); + // no need to refcount because drop is happening in write thread, so can't + // happen while we're in the write thread + ColumnFamilyData* cfd_picked = nullptr; + SequenceNumber seq_num_for_cf_picked = kMaxSequenceNumber; + + for (auto cfd : *versions_->GetColumnFamilySet()) { + if (cfd->IsDropped()) { + continue; + } + if (!cfd->mem()->IsEmpty()) { + // We only consider active mem table, hoping immutable memtable is + // already in the process of flushing. + uint64_t seq = cfd->mem()->GetCreationSeq(); + if (cfd_picked == nullptr || seq < seq_num_for_cf_picked) { + cfd_picked = cfd; + seq_num_for_cf_picked = seq; + } + } + } + if (cfd_picked != nullptr) { + status = SwitchMemtable(cfd_picked, write_context); + if (status.ok()) { + cfd_picked->imm()->FlushRequested(); + SchedulePendingFlush(cfd_picked); + MaybeScheduleFlushOrCompaction(); + } + } + return status; +} + +uint64_t DBImpl::GetMaxTotalWalSize() const { + mutex_.AssertHeld(); + return mutable_db_options_.max_total_wal_size == 0 + ? 4 * max_total_in_memory_state_ + : mutable_db_options_.max_total_wal_size; +} + +// REQUIRES: mutex_ is held +// REQUIRES: this thread is currently at the front of the writer queue +Status DBImpl::DelayWrite(uint64_t num_bytes, + const WriteOptions& write_options) { + uint64_t time_delayed = 0; + bool delayed = false; + { + StopWatch sw(env_, stats_, WRITE_STALL, &time_delayed); + uint64_t delay = write_controller_.GetDelay(env_, num_bytes); + if (delay > 0) { + if (write_options.no_slowdown) { + return Status::Incomplete(); + } + TEST_SYNC_POINT("DBImpl::DelayWrite:Sleep"); + + mutex_.Unlock(); + // We will delay the write until we have slept for delay ms or + // we don't need a delay anymore + const uint64_t kDelayInterval = 1000; + uint64_t stall_end = sw.start_time() + delay; + while (write_controller_.NeedsDelay()) { + if (env_->NowMicros() >= stall_end) { + // We already delayed this write `delay` microseconds + break; + } + + delayed = true; + // Sleep for 0.001 seconds + env_->SleepForMicroseconds(kDelayInterval); + } + mutex_.Lock(); + } + + while (bg_error_.ok() && write_controller_.IsStopped()) { + if (write_options.no_slowdown) { + return Status::Incomplete(); + } + delayed = true; + TEST_SYNC_POINT("DBImpl::DelayWrite:Wait"); + bg_cv_.Wait(); + } + } + assert(!delayed || !write_options.no_slowdown); + if (delayed) { + default_cf_internal_stats_->AddDBStats(InternalStats::WRITE_STALL_MICROS, + time_delayed); + RecordTick(stats_, STALL_MICROS, time_delayed); + } + + return bg_error_; +} + +Status DBImpl::ThrottleLowPriWritesIfNeeded(const WriteOptions& write_options, + WriteBatch* my_batch) { + assert(write_options.low_pri); + // This is called outside the DB mutex. Although it is safe to make the call, + // the consistency condition is not guaranteed to hold. It's OK to live with + // it in this case. + // If we need to speed compaction, it means the compaction is left behind + // and we start to limit low pri writes to a limit. + if (write_controller_.NeedSpeedupCompaction()) { + if (allow_2pc() && (my_batch->HasCommit() || my_batch->HasRollback())) { + // For 2PC, we only rate limit prepare, not commit. + return Status::OK(); + } + if (write_options.no_slowdown) { + return Status::Incomplete(); + } else { + assert(my_batch != nullptr); + // Rate limit those writes. The reason that we don't completely wait + // is that in case the write is heavy, low pri writes may never have + // a chance to run. Now we guarantee we are still slowly making + // progress. + write_controller_.low_pri_rate_limiter()->Request( + my_batch->GetDataSize(), Env::IO_HIGH, nullptr /* stats */, + RateLimiter::OpType::kWrite); + } + } + return Status::OK(); +} + +Status DBImpl::ScheduleFlushes(WriteContext* context) { + ColumnFamilyData* cfd; + while ((cfd = flush_scheduler_.TakeNextColumnFamily()) != nullptr) { + auto status = SwitchMemtable(cfd, context); + if (cfd->Unref()) { + delete cfd; + } + if (!status.ok()) { + return status; + } + } + return Status::OK(); +} + +#ifndef ROCKSDB_LITE +void DBImpl::NotifyOnMemTableSealed(ColumnFamilyData* cfd, + const MemTableInfo& mem_table_info) { + if (immutable_db_options_.listeners.size() == 0U) { + return; + } + if (shutting_down_.load(std::memory_order_acquire)) { + return; + } + + for (auto listener : immutable_db_options_.listeners) { + listener->OnMemTableSealed(mem_table_info); + } +} +#endif // ROCKSDB_LITE + +// REQUIRES: mutex_ is held +// REQUIRES: this thread is currently at the front of the writer queue +Status DBImpl::SwitchMemtable(ColumnFamilyData* cfd, WriteContext* context) { + mutex_.AssertHeld(); + WriteThread::Writer nonmem_w; + if (concurrent_prepare_) { + // SwitchMemtable is a rare event. To simply the reasoning, we make sure + // that there is no concurrent thread writing to WAL. + nonmem_write_thread_.EnterUnbatched(&nonmem_w, &mutex_); + } + + unique_ptr lfile; + log::Writer* new_log = nullptr; + MemTable* new_mem = nullptr; + + // In case of pipelined write is enabled, wait for all pending memtable + // writers. + if (immutable_db_options_.enable_pipelined_write) { + write_thread_.WaitForMemTableWriters(); + } + + // Attempt to switch to a new memtable and trigger flush of old. + // Do this without holding the dbmutex lock. + assert(versions_->prev_log_number() == 0); + if (concurrent_prepare_) { + log_write_mutex_.Lock(); + } + bool creating_new_log = !log_empty_; + if (concurrent_prepare_) { + log_write_mutex_.Unlock(); + } + uint64_t recycle_log_number = 0; + if (creating_new_log && immutable_db_options_.recycle_log_file_num && + !log_recycle_files.empty()) { + recycle_log_number = log_recycle_files.front(); + log_recycle_files.pop_front(); + } + uint64_t new_log_number = + creating_new_log ? versions_->NewFileNumber() : logfile_number_; + SuperVersion* new_superversion = nullptr; + const MutableCFOptions mutable_cf_options = *cfd->GetLatestMutableCFOptions(); + + // Set memtable_info for memtable sealed callback +#ifndef ROCKSDB_LITE + MemTableInfo memtable_info; + memtable_info.cf_name = cfd->GetName(); + memtable_info.first_seqno = cfd->mem()->GetFirstSequenceNumber(); + memtable_info.earliest_seqno = cfd->mem()->GetEarliestSequenceNumber(); + memtable_info.num_entries = cfd->mem()->num_entries(); + memtable_info.num_deletes = cfd->mem()->num_deletes(); +#endif // ROCKSDB_LITE + // Log this later after lock release. It may be outdated, e.g., if background + // flush happens before logging, but that should be ok. + int num_imm_unflushed = cfd->imm()->NumNotFlushed(); + DBOptions db_options = + BuildDBOptions(immutable_db_options_, mutable_db_options_); + const auto preallocate_block_size = + GetWalPreallocateBlockSize(mutable_cf_options.write_buffer_size); + mutex_.Unlock(); + Status s; + { + if (creating_new_log) { + EnvOptions opt_env_opt = + env_->OptimizeForLogWrite(env_options_, db_options); + if (recycle_log_number) { + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "reusing log %" PRIu64 " from recycle list\n", + recycle_log_number); + s = env_->ReuseWritableFile( + LogFileName(immutable_db_options_.wal_dir, new_log_number), + LogFileName(immutable_db_options_.wal_dir, recycle_log_number), + &lfile, opt_env_opt); + } else { + s = NewWritableFile( + env_, LogFileName(immutable_db_options_.wal_dir, new_log_number), + &lfile, opt_env_opt); + } + if (s.ok()) { + // Our final size should be less than write_buffer_size + // (compression, etc) but err on the side of caution. + + // use preallocate_block_size instead + // of calling GetWalPreallocateBlockSize() + lfile->SetPreallocationBlockSize(preallocate_block_size); + unique_ptr file_writer( + new WritableFileWriter(std::move(lfile), opt_env_opt)); + new_log = new log::Writer( + std::move(file_writer), new_log_number, + immutable_db_options_.recycle_log_file_num > 0, manual_wal_flush_); + } + } + + if (s.ok()) { + SequenceNumber seq = versions_->LastSequence(); + new_mem = cfd->ConstructNewMemtable(mutable_cf_options, seq); + new_superversion = new SuperVersion(); + } + +#ifndef ROCKSDB_LITE + // PLEASE NOTE: We assume that there are no failable operations + // after lock is acquired below since we are already notifying + // client about mem table becoming immutable. + NotifyOnMemTableSealed(cfd, memtable_info); +#endif //ROCKSDB_LITE + } + ROCKS_LOG_INFO(immutable_db_options_.info_log, + "[%s] New memtable created with log file: #%" PRIu64 + ". Immutable memtables: %d.\n", + cfd->GetName().c_str(), new_log_number, num_imm_unflushed); + mutex_.Lock(); + if (!s.ok()) { + // how do we fail if we're not creating new log? + assert(creating_new_log); + assert(!new_mem); + assert(!new_log); + if (concurrent_prepare_) { + nonmem_write_thread_.ExitUnbatched(&nonmem_w); + } + return s; + } + if (creating_new_log) { + log_write_mutex_.Lock(); + logfile_number_ = new_log_number; + assert(new_log != nullptr); + log_empty_ = true; + log_dir_synced_ = false; + if (!logs_.empty()) { + // Alway flush the buffer of the last log before switching to a new one + log::Writer* cur_log_writer = logs_.back().writer; + cur_log_writer->WriteBuffer(); + } + logs_.emplace_back(logfile_number_, new_log); + alive_log_files_.push_back(LogFileNumberSize(logfile_number_)); + log_write_mutex_.Unlock(); + } + for (auto loop_cfd : *versions_->GetColumnFamilySet()) { + // all this is just optimization to delete logs that + // are no longer needed -- if CF is empty, that means it + // doesn't need that particular log to stay alive, so we just + // advance the log number. no need to persist this in the manifest + if (loop_cfd->mem()->GetFirstSequenceNumber() == 0 && + loop_cfd->imm()->NumNotFlushed() == 0) { + if (creating_new_log) { + loop_cfd->SetLogNumber(logfile_number_); + } + loop_cfd->mem()->SetCreationSeq(versions_->LastSequence()); + } + } + + cfd->mem()->SetNextLogNumber(logfile_number_); + cfd->imm()->Add(cfd->mem(), &context->memtables_to_free_); + new_mem->Ref(); + cfd->SetMemtable(new_mem); + context->superversions_to_free_.push_back(InstallSuperVersionAndScheduleWork( + cfd, new_superversion, mutable_cf_options)); + if (concurrent_prepare_) { + nonmem_write_thread_.ExitUnbatched(&nonmem_w); + } + return s; +} + +size_t DBImpl::GetWalPreallocateBlockSize(uint64_t write_buffer_size) const { + mutex_.AssertHeld(); + size_t bsize = write_buffer_size / 10 + write_buffer_size; + // Some users might set very high write_buffer_size and rely on + // max_total_wal_size or other parameters to control the WAL size. + if (mutable_db_options_.max_total_wal_size > 0) { + bsize = std::min(bsize, mutable_db_options_.max_total_wal_size); + } + if (immutable_db_options_.db_write_buffer_size > 0) { + bsize = std::min(bsize, immutable_db_options_.db_write_buffer_size); + } + if (immutable_db_options_.write_buffer_manager && + immutable_db_options_.write_buffer_manager->enabled()) { + bsize = std::min( + bsize, immutable_db_options_.write_buffer_manager->buffer_size()); + } + + return bsize; +} + +// Default implementations of convenience methods that subclasses of DB +// can call if they wish +Status DB::Put(const WriteOptions& opt, ColumnFamilyHandle* column_family, + const Slice& key, const Slice& value) { + // Pre-allocate size of write batch conservatively. + // 8 bytes are taken by header, 4 bytes for count, 1 byte for type, + // and we allocate 11 extra bytes for key length, as well as value length. + WriteBatch batch(key.size() + value.size() + 24); + batch.Put(column_family, key, value); + return Write(opt, &batch); +} + +Status DB::Delete(const WriteOptions& opt, ColumnFamilyHandle* column_family, + const Slice& key) { + WriteBatch batch; + batch.Delete(column_family, key); + return Write(opt, &batch); +} + +Status DB::SingleDelete(const WriteOptions& opt, + ColumnFamilyHandle* column_family, const Slice& key) { + WriteBatch batch; + batch.SingleDelete(column_family, key); + return Write(opt, &batch); +} + +Status DB::DeleteRange(const WriteOptions& opt, + ColumnFamilyHandle* column_family, + const Slice& begin_key, const Slice& end_key) { + WriteBatch batch; + batch.DeleteRange(column_family, begin_key, end_key); + return Write(opt, &batch); +} + +Status DB::Merge(const WriteOptions& opt, ColumnFamilyHandle* column_family, + const Slice& key, const Slice& value) { + WriteBatch batch; + batch.Merge(column_family, key, value); + return Write(opt, &batch); +} +} // namespace rocksdb diff --git a/util/db_info_dummper.cc b/db/db_info_dumper.cc similarity index 62% rename from util/db_info_dummper.cc rename to db/db_info_dumper.cc index d5dd97ad2c1..1668a1638ff 100644 --- a/util/db_info_dummper.cc +++ b/db/db_info_dumper.cc @@ -1,25 +1,27 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Must not be included from any .h files to avoid polluting the namespace -// with macros. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#ifndef __STDC_FORMAT_MACROS #define __STDC_FORMAT_MACROS +#endif + +#include "db/db_info_dumper.h" + #include #include #include #include #include -#include "rocksdb/options.h" #include "rocksdb/env.h" -#include "db/filename.h" +#include "util/filename.h" namespace rocksdb { -void DumpDBFileSummary(const DBOptions& options, const std::string& dbname) { +void DumpDBFileSummary(const ImmutableDBOptions& options, + const std::string& dbname) { if (options.info_log == nullptr) { return; } @@ -33,10 +35,11 @@ void DumpDBFileSummary(const DBOptions& options, const std::string& dbname) { uint64_t file_size; std::string file_info, wal_info; - Log(options.info_log, "DB SUMMARY\n"); + Header(options.info_log, "DB SUMMARY\n"); // Get files in dbname dir if (!env->GetChildren(dbname, &files).ok()) { - Log(options.info_log, "Error when reading %s dir\n", dbname.c_str()); + Error(options.info_log, + "Error when reading %s dir\n", dbname.c_str()); } std::sort(files.begin(), files.end()); for (std::string file : files) { @@ -45,22 +48,22 @@ void DumpDBFileSummary(const DBOptions& options, const std::string& dbname) { } switch (type) { case kCurrentFile: - Log(options.info_log, "CURRENT file: %s\n", file.c_str()); + Header(options.info_log, "CURRENT file: %s\n", file.c_str()); break; case kIdentityFile: - Log(options.info_log, "IDENTITY file: %s\n", file.c_str()); + Header(options.info_log, "IDENTITY file: %s\n", file.c_str()); break; case kDescriptorFile: env->GetFileSize(dbname + "/" + file, &file_size); - Log(options.info_log, "MANIFEST file: %s size: %" PRIu64 " Bytes\n", - file.c_str(), file_size); + Header(options.info_log, "MANIFEST file: %s size: %" PRIu64 " Bytes\n", + file.c_str(), file_size); break; case kLogFile: env->GetFileSize(dbname + "/" + file, &file_size); - char str[8]; + char str[16]; snprintf(str, sizeof(str), "%" PRIu64, file_size); wal_info.append(file).append(" size: "). - append(str, sizeof(str)).append(" ;"); + append(str).append(" ; "); break; case kTableFile: if (++file_num < 10) { @@ -76,7 +79,8 @@ void DumpDBFileSummary(const DBOptions& options, const std::string& dbname) { for (auto& db_path : options.db_paths) { if (dbname.compare(db_path.path) != 0) { if (!env->GetChildren(db_path.path, &files).ok()) { - Log(options.info_log, "Error when reading %s dir\n", + Error(options.info_log, + "Error when reading %s dir\n", db_path.path.c_str()); continue; } @@ -89,8 +93,9 @@ void DumpDBFileSummary(const DBOptions& options, const std::string& dbname) { } } } - Log(options.info_log, "SST files in %s dir, Total Num: %" PRIu64 ", files: %s\n", - db_path.path.c_str(), file_num, file_info.c_str()); + Header(options.info_log, + "SST files in %s dir, Total Num: %" PRIu64 ", files: %s\n", + db_path.path.c_str(), file_num, file_info.c_str()); file_num = 0; file_info.clear(); } @@ -98,7 +103,8 @@ void DumpDBFileSummary(const DBOptions& options, const std::string& dbname) { // Get wal file in wal_dir if (dbname.compare(options.wal_dir) != 0) { if (!env->GetChildren(options.wal_dir, &files).ok()) { - Log(options.info_log, "Error when reading %s dir\n", + Error(options.info_log, + "Error when reading %s dir\n", options.wal_dir.c_str()); return; } @@ -107,15 +113,15 @@ void DumpDBFileSummary(const DBOptions& options, const std::string& dbname) { if (ParseFileName(file, &number, &type)) { if (type == kLogFile) { env->GetFileSize(options.wal_dir + "/" + file, &file_size); - char str[8]; + char str[16]; snprintf(str, sizeof(str), "%" PRIu64, file_size); wal_info.append(file).append(" size: "). - append(str, sizeof(str)).append(" ;"); + append(str).append(" ; "); } } } } - Log(options.info_log, "Write Ahead Log file in %s: %s\n", - options.wal_dir.c_str(), wal_info.c_str()); + Header(options.info_log, "Write Ahead Log file in %s: %s\n", + options.wal_dir.c_str(), wal_info.c_str()); } } // namespace rocksdb diff --git a/db/db_info_dumper.h b/db/db_info_dumper.h new file mode 100644 index 00000000000..acff8f1b8f6 --- /dev/null +++ b/db/db_info_dumper.h @@ -0,0 +1,14 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once + +#include + +#include "options/db_options.h" + +namespace rocksdb { +void DumpDBFileSummary(const ImmutableDBOptions& options, + const std::string& dbname); +} // namespace rocksdb diff --git a/db/db_inplace_update_test.cc b/db/db_inplace_update_test.cc new file mode 100644 index 00000000000..c1f1b51e301 --- /dev/null +++ b/db/db_inplace_update_test.cc @@ -0,0 +1,177 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#include "db/db_test_util.h" +#include "port/stack_trace.h" + +namespace rocksdb { + +class DBTestInPlaceUpdate : public DBTestBase { + public: + DBTestInPlaceUpdate() : DBTestBase("/db_inplace_update_test") {} +}; + +TEST_F(DBTestInPlaceUpdate, InPlaceUpdate) { + do { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.inplace_update_support = true; + options.env = env_; + options.write_buffer_size = 100000; + options.allow_concurrent_memtable_write = false; + Reopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Update key with values of smaller size + int numValues = 10; + for (int i = numValues; i > 0; i--) { + std::string value = DummyString(i, 'a'); + ASSERT_OK(Put(1, "key", value)); + ASSERT_EQ(value, Get(1, "key")); + } + + // Only 1 instance for that key. + validateNumberOfEntries(1, 1); + } while (ChangeCompactOptions()); +} + +TEST_F(DBTestInPlaceUpdate, InPlaceUpdateLargeNewValue) { + do { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.inplace_update_support = true; + options.env = env_; + options.write_buffer_size = 100000; + options.allow_concurrent_memtable_write = false; + Reopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Update key with values of larger size + int numValues = 10; + for (int i = 0; i < numValues; i++) { + std::string value = DummyString(i, 'a'); + ASSERT_OK(Put(1, "key", value)); + ASSERT_EQ(value, Get(1, "key")); + } + + // All 10 updates exist in the internal iterator + validateNumberOfEntries(numValues, 1); + } while (ChangeCompactOptions()); +} + +TEST_F(DBTestInPlaceUpdate, InPlaceUpdateCallbackSmallerSize) { + do { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.inplace_update_support = true; + + options.env = env_; + options.write_buffer_size = 100000; + options.inplace_callback = + rocksdb::DBTestInPlaceUpdate::updateInPlaceSmallerSize; + options.allow_concurrent_memtable_write = false; + Reopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Update key with values of smaller size + int numValues = 10; + ASSERT_OK(Put(1, "key", DummyString(numValues, 'a'))); + ASSERT_EQ(DummyString(numValues, 'c'), Get(1, "key")); + + for (int i = numValues; i > 0; i--) { + ASSERT_OK(Put(1, "key", DummyString(i, 'a'))); + ASSERT_EQ(DummyString(i - 1, 'b'), Get(1, "key")); + } + + // Only 1 instance for that key. + validateNumberOfEntries(1, 1); + } while (ChangeCompactOptions()); +} + +TEST_F(DBTestInPlaceUpdate, InPlaceUpdateCallbackSmallerVarintSize) { + do { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.inplace_update_support = true; + + options.env = env_; + options.write_buffer_size = 100000; + options.inplace_callback = + rocksdb::DBTestInPlaceUpdate::updateInPlaceSmallerVarintSize; + options.allow_concurrent_memtable_write = false; + Reopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Update key with values of smaller varint size + int numValues = 265; + ASSERT_OK(Put(1, "key", DummyString(numValues, 'a'))); + ASSERT_EQ(DummyString(numValues, 'c'), Get(1, "key")); + + for (int i = numValues; i > 0; i--) { + ASSERT_OK(Put(1, "key", DummyString(i, 'a'))); + ASSERT_EQ(DummyString(1, 'b'), Get(1, "key")); + } + + // Only 1 instance for that key. + validateNumberOfEntries(1, 1); + } while (ChangeCompactOptions()); +} + +TEST_F(DBTestInPlaceUpdate, InPlaceUpdateCallbackLargeNewValue) { + do { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.inplace_update_support = true; + + options.env = env_; + options.write_buffer_size = 100000; + options.inplace_callback = + rocksdb::DBTestInPlaceUpdate::updateInPlaceLargerSize; + options.allow_concurrent_memtable_write = false; + Reopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Update key with values of larger size + int numValues = 10; + for (int i = 0; i < numValues; i++) { + ASSERT_OK(Put(1, "key", DummyString(i, 'a'))); + ASSERT_EQ(DummyString(i, 'c'), Get(1, "key")); + } + + // No inplace updates. All updates are puts with new seq number + // All 10 updates exist in the internal iterator + validateNumberOfEntries(numValues, 1); + } while (ChangeCompactOptions()); +} + +TEST_F(DBTestInPlaceUpdate, InPlaceUpdateCallbackNoAction) { + do { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.inplace_update_support = true; + + options.env = env_; + options.write_buffer_size = 100000; + options.inplace_callback = + rocksdb::DBTestInPlaceUpdate::updateInPlaceNoAction; + options.allow_concurrent_memtable_write = false; + Reopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Callback function requests no actions from db + ASSERT_OK(Put(1, "key", DummyString(1, 'a'))); + ASSERT_EQ(Get(1, "key"), "NOT_FOUND"); + } while (ChangeCompactOptions()); +} +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_io_failure_test.cc b/db/db_io_failure_test.cc new file mode 100644 index 00000000000..9f4dcc5d056 --- /dev/null +++ b/db/db_io_failure_test.cc @@ -0,0 +1,568 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_test_util.h" +#include "port/stack_trace.h" + +namespace rocksdb { + +class DBIOFailureTest : public DBTestBase { + public: + DBIOFailureTest() : DBTestBase("/db_io_failure_test") {} +}; + +#ifndef ROCKSDB_LITE +// Check that number of files does not grow when writes are dropped +TEST_F(DBIOFailureTest, DropWrites) { + do { + Options options = CurrentOptions(); + options.env = env_; + options.paranoid_checks = false; + Reopen(options); + + ASSERT_OK(Put("foo", "v1")); + ASSERT_EQ("v1", Get("foo")); + Compact("a", "z"); + const size_t num_files = CountFiles(); + // Force out-of-space errors + env_->drop_writes_.store(true, std::memory_order_release); + env_->sleep_counter_.Reset(); + env_->no_slowdown_ = true; + for (int i = 0; i < 5; i++) { + if (option_config_ != kUniversalCompactionMultiLevel && + option_config_ != kUniversalSubcompactions) { + for (int level = 0; level < dbfull()->NumberLevels(); level++) { + if (level > 0 && level == dbfull()->NumberLevels() - 1) { + break; + } + dbfull()->TEST_CompactRange(level, nullptr, nullptr, nullptr, + true /* disallow trivial move */); + } + } else { + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + } + } + + std::string property_value; + ASSERT_TRUE(db_->GetProperty("rocksdb.background-errors", &property_value)); + ASSERT_EQ("5", property_value); + + env_->drop_writes_.store(false, std::memory_order_release); + ASSERT_LT(CountFiles(), num_files + 3); + + // Check that compaction attempts slept after errors + // TODO @krad: Figure out why ASSERT_EQ 5 keeps failing in certain compiler + // versions + ASSERT_GE(env_->sleep_counter_.Read(), 4); + } while (ChangeCompactOptions()); +} + +// Check background error counter bumped on flush failures. +TEST_F(DBIOFailureTest, DropWritesFlush) { + do { + Options options = CurrentOptions(); + options.env = env_; + options.max_background_flushes = 1; + Reopen(options); + + ASSERT_OK(Put("foo", "v1")); + // Force out-of-space errors + env_->drop_writes_.store(true, std::memory_order_release); + + std::string property_value; + // Background error count is 0 now. + ASSERT_TRUE(db_->GetProperty("rocksdb.background-errors", &property_value)); + ASSERT_EQ("0", property_value); + + dbfull()->TEST_FlushMemTable(true); + + ASSERT_TRUE(db_->GetProperty("rocksdb.background-errors", &property_value)); + ASSERT_EQ("1", property_value); + + env_->drop_writes_.store(false, std::memory_order_release); + } while (ChangeCompactOptions()); +} +#endif // ROCKSDB_LITE + +// Check that CompactRange() returns failure if there is not enough space left +// on device +TEST_F(DBIOFailureTest, NoSpaceCompactRange) { + do { + Options options = CurrentOptions(); + options.env = env_; + options.disable_auto_compactions = true; + Reopen(options); + + // generate 5 tables + for (int i = 0; i < 5; ++i) { + ASSERT_OK(Put(Key(i), Key(i) + "v")); + ASSERT_OK(Flush()); + } + + // Force out-of-space errors + env_->no_space_.store(true, std::memory_order_release); + + Status s = dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, + true /* disallow trivial move */); + ASSERT_TRUE(s.IsIOError()); + ASSERT_TRUE(s.IsNoSpace()); + + env_->no_space_.store(false, std::memory_order_release); + } while (ChangeCompactOptions()); +} + +TEST_F(DBIOFailureTest, NonWritableFileSystem) { + do { + Options options = CurrentOptions(); + options.write_buffer_size = 4096; + options.arena_block_size = 4096; + options.env = env_; + Reopen(options); + ASSERT_OK(Put("foo", "v1")); + env_->non_writeable_rate_.store(100); + std::string big(100000, 'x'); + int errors = 0; + for (int i = 0; i < 20; i++) { + if (!Put("foo", big).ok()) { + errors++; + env_->SleepForMicroseconds(100000); + } + } + ASSERT_GT(errors, 0); + env_->non_writeable_rate_.store(0); + } while (ChangeCompactOptions()); +} + +#ifndef ROCKSDB_LITE +TEST_F(DBIOFailureTest, ManifestWriteError) { + // Test for the following problem: + // (a) Compaction produces file F + // (b) Log record containing F is written to MANIFEST file, but Sync() fails + // (c) GC deletes F + // (d) After reopening DB, reads fail since deleted F is named in log record + + // We iterate twice. In the second iteration, everything is the + // same except the log record never makes it to the MANIFEST file. + for (int iter = 0; iter < 2; iter++) { + std::atomic* error_type = (iter == 0) ? &env_->manifest_sync_error_ + : &env_->manifest_write_error_; + + // Insert foo=>bar mapping + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.error_if_exists = false; + options.paranoid_checks = true; + DestroyAndReopen(options); + ASSERT_OK(Put("foo", "bar")); + ASSERT_EQ("bar", Get("foo")); + + // Memtable compaction (will succeed) + Flush(); + ASSERT_EQ("bar", Get("foo")); + const int last = 2; + MoveFilesToLevel(2); + ASSERT_EQ(NumTableFilesAtLevel(last), 1); // foo=>bar is now in last level + + // Merging compaction (will fail) + error_type->store(true, std::memory_order_release); + dbfull()->TEST_CompactRange(last, nullptr, nullptr); // Should fail + ASSERT_EQ("bar", Get("foo")); + + error_type->store(false, std::memory_order_release); + + // Since paranoid_checks=true, writes should fail + ASSERT_NOK(Put("foo2", "bar2")); + + // Recovery: should not lose data + ASSERT_EQ("bar", Get("foo")); + + // Try again with paranoid_checks=false + Close(); + options.paranoid_checks = false; + Reopen(options); + + // Merging compaction (will fail) + error_type->store(true, std::memory_order_release); + dbfull()->TEST_CompactRange(last, nullptr, nullptr); // Should fail + ASSERT_EQ("bar", Get("foo")); + + // Recovery: should not lose data + error_type->store(false, std::memory_order_release); + Reopen(options); + ASSERT_EQ("bar", Get("foo")); + + // Since paranoid_checks=false, writes should succeed + ASSERT_OK(Put("foo2", "bar2")); + ASSERT_EQ("bar", Get("foo")); + ASSERT_EQ("bar2", Get("foo2")); + } +} + +TEST_F(DBIOFailureTest, PutFailsParanoid) { + // Test the following: + // (a) A random put fails in paranoid mode (simulate by sync fail) + // (b) All other puts have to fail, even if writes would succeed + // (c) All of that should happen ONLY if paranoid_checks = true + + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.error_if_exists = false; + options.paranoid_checks = true; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + Status s; + + ASSERT_OK(Put(1, "foo", "bar")); + ASSERT_OK(Put(1, "foo1", "bar1")); + // simulate error + env_->log_write_error_.store(true, std::memory_order_release); + s = Put(1, "foo2", "bar2"); + ASSERT_TRUE(!s.ok()); + env_->log_write_error_.store(false, std::memory_order_release); + s = Put(1, "foo3", "bar3"); + // the next put should fail, too + ASSERT_TRUE(!s.ok()); + // but we're still able to read + ASSERT_EQ("bar", Get(1, "foo")); + + // do the same thing with paranoid checks off + options.paranoid_checks = false; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + ASSERT_OK(Put(1, "foo", "bar")); + ASSERT_OK(Put(1, "foo1", "bar1")); + // simulate error + env_->log_write_error_.store(true, std::memory_order_release); + s = Put(1, "foo2", "bar2"); + ASSERT_TRUE(!s.ok()); + env_->log_write_error_.store(false, std::memory_order_release); + s = Put(1, "foo3", "bar3"); + // the next put should NOT fail + ASSERT_TRUE(s.ok()); +} +#if !(defined NDEBUG) || !defined(OS_WIN) +TEST_F(DBIOFailureTest, FlushSstRangeSyncError) { + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.error_if_exists = false; + options.paranoid_checks = true; + options.write_buffer_size = 256 * 1024 * 1024; + options.writable_file_max_buffer_size = 128 * 1024; + options.bytes_per_sync = 128 * 1024; + options.level0_file_num_compaction_trigger = 4; + options.memtable_factory.reset(new SpecialSkipListFactory(10)); + BlockBasedTableOptions table_options; + table_options.filter_policy.reset(NewBloomFilterPolicy(10)); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + Status s; + + std::atomic range_sync_called(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "SpecialEnv::SStableFile::RangeSync", [&](void* arg) { + if (range_sync_called.fetch_add(1) == 0) { + Status* st = static_cast(arg); + *st = Status::IOError("range sync dummy error"); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Random rnd(301); + std::string rnd_str = + RandomString(&rnd, static_cast(options.bytes_per_sync / 2)); + std::string rnd_str_512kb = RandomString(&rnd, 512 * 1024); + + ASSERT_OK(Put(1, "foo", "bar")); + // First 1MB doesn't get range synced + ASSERT_OK(Put(1, "foo0_0", rnd_str_512kb)); + ASSERT_OK(Put(1, "foo0_1", rnd_str_512kb)); + ASSERT_OK(Put(1, "foo1_1", rnd_str)); + ASSERT_OK(Put(1, "foo1_2", rnd_str)); + ASSERT_OK(Put(1, "foo1_3", rnd_str)); + ASSERT_OK(Put(1, "foo2", "bar")); + ASSERT_OK(Put(1, "foo3_1", rnd_str)); + ASSERT_OK(Put(1, "foo3_2", rnd_str)); + ASSERT_OK(Put(1, "foo3_3", rnd_str)); + ASSERT_OK(Put(1, "foo4", "bar")); + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + + // Following writes should fail as flush failed. + ASSERT_NOK(Put(1, "foo2", "bar3")); + ASSERT_EQ("bar", Get(1, "foo")); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + ASSERT_GE(1, range_sync_called.load()); + + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_EQ("bar", Get(1, "foo")); +} + +TEST_F(DBIOFailureTest, CompactSstRangeSyncError) { + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.error_if_exists = false; + options.paranoid_checks = true; + options.write_buffer_size = 256 * 1024 * 1024; + options.writable_file_max_buffer_size = 128 * 1024; + options.bytes_per_sync = 128 * 1024; + options.level0_file_num_compaction_trigger = 2; + options.target_file_size_base = 256 * 1024 * 1024; + options.disable_auto_compactions = true; + BlockBasedTableOptions table_options; + table_options.filter_policy.reset(NewBloomFilterPolicy(10)); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + Status s; + + Random rnd(301); + std::string rnd_str = + RandomString(&rnd, static_cast(options.bytes_per_sync / 2)); + std::string rnd_str_512kb = RandomString(&rnd, 512 * 1024); + + ASSERT_OK(Put(1, "foo", "bar")); + // First 1MB doesn't get range synced + ASSERT_OK(Put(1, "foo0_0", rnd_str_512kb)); + ASSERT_OK(Put(1, "foo0_1", rnd_str_512kb)); + ASSERT_OK(Put(1, "foo1_1", rnd_str)); + ASSERT_OK(Put(1, "foo1_2", rnd_str)); + ASSERT_OK(Put(1, "foo1_3", rnd_str)); + Flush(1); + ASSERT_OK(Put(1, "foo", "bar")); + ASSERT_OK(Put(1, "foo3_1", rnd_str)); + ASSERT_OK(Put(1, "foo3_2", rnd_str)); + ASSERT_OK(Put(1, "foo3_3", rnd_str)); + ASSERT_OK(Put(1, "foo4", "bar")); + Flush(1); + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + + std::atomic range_sync_called(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "SpecialEnv::SStableFile::RangeSync", [&](void* arg) { + if (range_sync_called.fetch_add(1) == 0) { + Status* st = static_cast(arg); + *st = Status::IOError("range sync dummy error"); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + ASSERT_OK(dbfull()->SetOptions(handles_[1], + { + {"disable_auto_compactions", "false"}, + })); + dbfull()->TEST_WaitForCompact(); + + // Following writes should fail as flush failed. + ASSERT_NOK(Put(1, "foo2", "bar3")); + ASSERT_EQ("bar", Get(1, "foo")); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + ASSERT_GE(1, range_sync_called.load()); + + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_EQ("bar", Get(1, "foo")); +} + +TEST_F(DBIOFailureTest, FlushSstCloseError) { + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.error_if_exists = false; + options.paranoid_checks = true; + options.level0_file_num_compaction_trigger = 4; + options.memtable_factory.reset(new SpecialSkipListFactory(2)); + + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + Status s; + std::atomic close_called(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "SpecialEnv::SStableFile::Close", [&](void* arg) { + if (close_called.fetch_add(1) == 0) { + Status* st = static_cast(arg); + *st = Status::IOError("close dummy error"); + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + ASSERT_OK(Put(1, "foo", "bar")); + ASSERT_OK(Put(1, "foo1", "bar1")); + ASSERT_OK(Put(1, "foo", "bar2")); + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + + // Following writes should fail as flush failed. + ASSERT_NOK(Put(1, "foo2", "bar3")); + ASSERT_EQ("bar2", Get(1, "foo")); + ASSERT_EQ("bar1", Get(1, "foo1")); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_EQ("bar2", Get(1, "foo")); + ASSERT_EQ("bar1", Get(1, "foo1")); +} + +TEST_F(DBIOFailureTest, CompactionSstCloseError) { + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.error_if_exists = false; + options.paranoid_checks = true; + options.level0_file_num_compaction_trigger = 2; + options.disable_auto_compactions = true; + + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + Status s; + + ASSERT_OK(Put(1, "foo", "bar")); + ASSERT_OK(Put(1, "foo2", "bar")); + Flush(1); + ASSERT_OK(Put(1, "foo", "bar2")); + ASSERT_OK(Put(1, "foo2", "bar")); + Flush(1); + ASSERT_OK(Put(1, "foo", "bar3")); + ASSERT_OK(Put(1, "foo2", "bar")); + Flush(1); + dbfull()->TEST_WaitForCompact(); + + std::atomic close_called(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "SpecialEnv::SStableFile::Close", [&](void* arg) { + if (close_called.fetch_add(1) == 0) { + Status* st = static_cast(arg); + *st = Status::IOError("close dummy error"); + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + ASSERT_OK(dbfull()->SetOptions(handles_[1], + { + {"disable_auto_compactions", "false"}, + })); + dbfull()->TEST_WaitForCompact(); + + // Following writes should fail as compaction failed. + ASSERT_NOK(Put(1, "foo2", "bar3")); + ASSERT_EQ("bar3", Get(1, "foo")); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_EQ("bar3", Get(1, "foo")); +} + +TEST_F(DBIOFailureTest, FlushSstSyncError) { + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.error_if_exists = false; + options.paranoid_checks = true; + options.use_fsync = false; + options.level0_file_num_compaction_trigger = 4; + options.memtable_factory.reset(new SpecialSkipListFactory(2)); + + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + Status s; + std::atomic sync_called(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "SpecialEnv::SStableFile::Sync", [&](void* arg) { + if (sync_called.fetch_add(1) == 0) { + Status* st = static_cast(arg); + *st = Status::IOError("sync dummy error"); + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + ASSERT_OK(Put(1, "foo", "bar")); + ASSERT_OK(Put(1, "foo1", "bar1")); + ASSERT_OK(Put(1, "foo", "bar2")); + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + + // Following writes should fail as flush failed. + ASSERT_NOK(Put(1, "foo2", "bar3")); + ASSERT_EQ("bar2", Get(1, "foo")); + ASSERT_EQ("bar1", Get(1, "foo1")); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_EQ("bar2", Get(1, "foo")); + ASSERT_EQ("bar1", Get(1, "foo1")); +} + +TEST_F(DBIOFailureTest, CompactionSstSyncError) { + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.error_if_exists = false; + options.paranoid_checks = true; + options.level0_file_num_compaction_trigger = 2; + options.disable_auto_compactions = true; + options.use_fsync = false; + + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + Status s; + + ASSERT_OK(Put(1, "foo", "bar")); + ASSERT_OK(Put(1, "foo2", "bar")); + Flush(1); + ASSERT_OK(Put(1, "foo", "bar2")); + ASSERT_OK(Put(1, "foo2", "bar")); + Flush(1); + ASSERT_OK(Put(1, "foo", "bar3")); + ASSERT_OK(Put(1, "foo2", "bar")); + Flush(1); + dbfull()->TEST_WaitForCompact(); + + std::atomic sync_called(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "SpecialEnv::SStableFile::Sync", [&](void* arg) { + if (sync_called.fetch_add(1) == 0) { + Status* st = static_cast(arg); + *st = Status::IOError("close dummy error"); + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + ASSERT_OK(dbfull()->SetOptions(handles_[1], + { + {"disable_auto_compactions", "false"}, + })); + dbfull()->TEST_WaitForCompact(); + + // Following writes should fail as compaction failed. + ASSERT_NOK(Put(1, "foo2", "bar3")); + ASSERT_EQ("bar3", Get(1, "foo")); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_EQ("bar3", Get(1, "foo")); +} +#endif // !(defined NDEBUG) || !defined(OS_WIN) +#endif // ROCKSDB_LITE +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_iter.cc b/db/db_iter.cc index 599a56a99c0..e4a6c92a7dd 100644 --- a/db/db_iter.cc +++ b/db/db_iter.cc @@ -1,29 +1,31 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #include "db/db_iter.h" -#include -#include #include #include -#include "db/filename.h" #include "db/dbformat.h" +#include "db/merge_context.h" +#include "db/merge_helper.h" +#include "db/pinned_iterators_manager.h" +#include "monitoring/perf_context_imp.h" #include "rocksdb/env.h" -#include "rocksdb/options.h" #include "rocksdb/iterator.h" #include "rocksdb/merge_operator.h" -#include "port/port.h" +#include "rocksdb/options.h" +#include "table/internal_iterator.h" #include "util/arena.h" +#include "util/filename.h" #include "util/logging.h" #include "util/mutexlock.h" -#include "util/perf_context_imp.h" +#include "util/string_util.h" namespace rocksdb { @@ -58,70 +60,187 @@ class DBIter: public Iterator { kReverse }; - DBIter(Env* env, const Options& options, const Comparator* cmp, - Iterator* iter, SequenceNumber s, bool arena_mode) + // LocalStatistics contain Statistics counters that will be aggregated per + // each iterator instance and then will be sent to the global statistics when + // the iterator is destroyed. + // + // The purpose of this approach is to avoid perf regression happening + // when multiple threads bump the atomic counters from a DBIter::Next(). + struct LocalStatistics { + explicit LocalStatistics() { ResetCounters(); } + + void ResetCounters() { + next_count_ = 0; + next_found_count_ = 0; + prev_count_ = 0; + prev_found_count_ = 0; + bytes_read_ = 0; + } + + void BumpGlobalStatistics(Statistics* global_statistics) { + RecordTick(global_statistics, NUMBER_DB_NEXT, next_count_); + RecordTick(global_statistics, NUMBER_DB_NEXT_FOUND, next_found_count_); + RecordTick(global_statistics, NUMBER_DB_PREV, prev_count_); + RecordTick(global_statistics, NUMBER_DB_PREV_FOUND, prev_found_count_); + RecordTick(global_statistics, ITER_BYTES_READ, bytes_read_); + PERF_COUNTER_ADD(iter_read_bytes, bytes_read_); + ResetCounters(); + } + + // Map to Tickers::NUMBER_DB_NEXT + uint64_t next_count_; + // Map to Tickers::NUMBER_DB_NEXT_FOUND + uint64_t next_found_count_; + // Map to Tickers::NUMBER_DB_PREV + uint64_t prev_count_; + // Map to Tickers::NUMBER_DB_PREV_FOUND + uint64_t prev_found_count_; + // Map to Tickers::ITER_BYTES_READ + uint64_t bytes_read_; + }; + + DBIter(Env* _env, const ReadOptions& read_options, + const ImmutableCFOptions& cf_options, const Comparator* cmp, + InternalIterator* iter, SequenceNumber s, bool arena_mode, + uint64_t max_sequential_skip_in_iterations, bool allow_blob) : arena_mode_(arena_mode), - env_(env), - logger_(options.info_log.get()), + env_(_env), + logger_(cf_options.info_log), user_comparator_(cmp), - user_merge_operator_(options.merge_operator.get()), + merge_operator_(cf_options.merge_operator), iter_(iter), sequence_(s), direction_(kForward), valid_(false), current_entry_is_merged_(false), - statistics_(options.statistics.get()) { + statistics_(cf_options.statistics), + iterate_upper_bound_(read_options.iterate_upper_bound), + prefix_same_as_start_(read_options.prefix_same_as_start), + pin_thru_lifetime_(read_options.pin_data), + total_order_seek_(read_options.total_order_seek), + range_del_agg_(cf_options.internal_comparator, s, + true /* collapse_deletions */), + allow_blob_(allow_blob) { RecordTick(statistics_, NO_ITERATORS); - has_prefix_extractor_ = (options.prefix_extractor.get() != nullptr); - max_skip_ = options.max_sequential_skip_in_iterations; + prefix_extractor_ = cf_options.prefix_extractor; + max_skip_ = max_sequential_skip_in_iterations; + max_skippable_internal_keys_ = read_options.max_skippable_internal_keys; + if (pin_thru_lifetime_) { + pinned_iters_mgr_.StartPinning(); + } + if (iter_) { + iter_->SetPinnedItersMgr(&pinned_iters_mgr_); + } } virtual ~DBIter() { + // Release pinned data if any + if (pinned_iters_mgr_.PinningEnabled()) { + pinned_iters_mgr_.ReleasePinnedData(); + } RecordTick(statistics_, NO_ITERATORS, -1); + local_stats_.BumpGlobalStatistics(statistics_); if (!arena_mode_) { delete iter_; } else { - iter_->~Iterator(); + iter_->~InternalIterator(); } } - virtual void SetIter(Iterator* iter) { + virtual void SetIter(InternalIterator* iter) { assert(iter_ == nullptr); iter_ = iter; + iter_->SetPinnedItersMgr(&pinned_iters_mgr_); + } + virtual RangeDelAggregator* GetRangeDelAggregator() { + return &range_del_agg_; } - virtual bool Valid() const { return valid_; } - virtual Slice key() const { + + virtual bool Valid() const override { return valid_; } + virtual Slice key() const override { assert(valid_); - return saved_key_.GetKey(); + return saved_key_.GetUserKey(); } - virtual Slice value() const { + virtual Slice value() const override { assert(valid_); - return (direction_ == kForward && !current_entry_is_merged_) ? - iter_->value() : saved_value_; + if (current_entry_is_merged_) { + // If pinned_value_ is set then the result of merge operator is one of + // the merge operands and we should return it. + return pinned_value_.data() ? pinned_value_ : saved_value_; + } else if (direction_ == kReverse) { + return pinned_value_; + } else { + return iter_->value(); + } } - virtual Status status() const { + virtual Status status() const override { if (status_.ok()) { return iter_->status(); } else { return status_; } } + bool IsBlob() const { + assert(valid_ && (allow_blob_ || !is_blob_)); + return is_blob_; + } - virtual void Next(); - virtual void Prev(); - virtual void Seek(const Slice& target); - virtual void SeekToFirst(); - virtual void SeekToLast(); + virtual Status GetProperty(std::string prop_name, + std::string* prop) override { + if (prop == nullptr) { + return Status::InvalidArgument("prop is nullptr"); + } + if (prop_name == "rocksdb.iterator.super-version-number") { + // First try to pass the value returned from inner iterator. + return iter_->GetProperty(prop_name, prop); + } else if (prop_name == "rocksdb.iterator.is-key-pinned") { + if (valid_) { + *prop = (pin_thru_lifetime_ && saved_key_.IsKeyPinned()) ? "1" : "0"; + } else { + *prop = "Iterator is not valid."; + } + return Status::OK(); + } + return Status::InvalidArgument("Undentified property."); + } + + virtual void Next() override; + virtual void Prev() override; + virtual void Seek(const Slice& target) override; + virtual void SeekForPrev(const Slice& target) override; + virtual void SeekToFirst() override; + virtual void SeekToLast() override; + Env* env() { return env_; } + void set_sequence(uint64_t s) { sequence_ = s; } + void set_valid(bool v) { valid_ = v; } private: + void ReverseToForward(); + void ReverseToBackward(); void PrevInternal(); void FindParseableKey(ParsedInternalKey* ikey, Direction direction); bool FindValueForCurrentKey(); bool FindValueForCurrentKeyUsingSeek(); void FindPrevUserKey(); void FindNextUserKey(); - inline void FindNextUserEntry(bool skipping); - void FindNextUserEntryInternal(bool skipping); + inline void FindNextUserEntry(bool skipping, bool prefix_check); + void FindNextUserEntryInternal(bool skipping, bool prefix_check); bool ParseKey(ParsedInternalKey* key); void MergeValuesNewToOld(); + bool TooManyInternalKeysSkipped(bool increment = true); + + // Temporarily pin the blocks that we encounter until ReleaseTempPinnedData() + // is called + void TempPinData() { + if (!pin_thru_lifetime_) { + pinned_iters_mgr_.StartPinning(); + } + } + + // Release blocks pinned by TempPinData() + void ReleaseTempPinnedData() { + if (!pin_thru_lifetime_ && pinned_iters_mgr_.PinningEnabled()) { + pinned_iters_mgr_.ReleasePinnedData(); + } + } inline void ClearSavedValue() { if (saved_value_.capacity() > 1048576) { @@ -132,23 +251,46 @@ class DBIter: public Iterator { } } - bool has_prefix_extractor_; + inline void ResetInternalKeysSkippedCounter() { + num_internal_keys_skipped_ = 0; + } + + const SliceTransform* prefix_extractor_; bool arena_mode_; Env* const env_; Logger* logger_; const Comparator* const user_comparator_; - const MergeOperator* const user_merge_operator_; - Iterator* iter_; - SequenceNumber const sequence_; + const MergeOperator* const merge_operator_; + InternalIterator* iter_; + SequenceNumber sequence_; Status status_; IterKey saved_key_; std::string saved_value_; + Slice pinned_value_; Direction direction_; bool valid_; bool current_entry_is_merged_; + // for prefix seek mode to support prev() Statistics* statistics_; uint64_t max_skip_; + uint64_t max_skippable_internal_keys_; + uint64_t num_internal_keys_skipped_; + const Slice* iterate_upper_bound_; + IterKey prefix_start_buf_; + Slice prefix_start_key_; + const bool prefix_same_as_start_; + // Means that we will pin all data blocks we read as long the Iterator + // is not deleted, will be true if ReadOptions::pin_data is true + const bool pin_thru_lifetime_; + const bool total_order_seek_; + // List of operands for merge operator. + MergeContext merge_context_; + RangeDelAggregator range_del_agg_; + LocalStatistics local_stats_; + PinnedIteratorsManager pinned_iters_mgr_; + bool allow_blob_; + bool is_blob_; // No copying allowed DBIter(const DBIter&); @@ -158,8 +300,8 @@ class DBIter: public Iterator { inline bool DBIter::ParseKey(ParsedInternalKey* ikey) { if (!ParseInternalKey(iter_->key(), ikey)) { status_ = Status::Corruption("corrupted internal key in DBIter"); - Log(logger_, "corrupted internal key in DBIter: %s", - iter_->key().ToString(true).c_str()); + ROCKS_LOG_ERROR(logger_, "corrupted internal key in DBIter: %s", + iter_->key().ToString(true).c_str()); return false; } else { return true; @@ -169,20 +311,35 @@ inline bool DBIter::ParseKey(ParsedInternalKey* ikey) { void DBIter::Next() { assert(valid_); + // Release temporarily pinned blocks from last operation + ReleaseTempPinnedData(); + ResetInternalKeysSkippedCounter(); if (direction_ == kReverse) { - FindNextUserKey(); - direction_ = kForward; - if (!iter_->Valid()) { - iter_->SeekToFirst(); - } + ReverseToForward(); + } else if (iter_->Valid() && !current_entry_is_merged_) { + // If the current value is not a merge, the iter position is the + // current key, which is already returned. We can safely issue a + // Next() without checking the current key. + // If the current key is a merge, very likely iter already points + // to the next internal position. + iter_->Next(); + PERF_COUNTER_ADD(internal_key_skipped_count, 1); } - // If the current value is merged, we might already hit end of iter_ + if (statistics_ != nullptr) { + local_stats_.next_count_++; + } + // Now we point to the next internal position, for both of merge and + // not merge cases. if (!iter_->Valid()) { valid_ = false; return; } - FindNextUserEntry(true /* skipping the current user key */); + FindNextUserEntry(true /* skipping the current user key */, prefix_same_as_start_); + if (statistics_ != nullptr && valid_) { + local_stats_.next_found_count_++; + local_stats_.bytes_read_ += (key().size() + value().size()); + } } // PRE: saved_key_ has the current user key if skipping @@ -192,63 +349,179 @@ void DBIter::Next() { // saved_value_ => the merged value // // NOTE: In between, saved_key_ can point to a user key that has -// a delete marker -inline void DBIter::FindNextUserEntry(bool skipping) { +// a delete marker or a sequence number higher than sequence_ +// saved_key_ MUST have a proper user_key before calling this function +// +// The prefix_check parameter controls whether we check the iterated +// keys against the prefix of the seeked key. Set to false when +// performing a seek without a key (e.g. SeekToFirst). Set to +// prefix_same_as_start_ for other iterations. +inline void DBIter::FindNextUserEntry(bool skipping, bool prefix_check) { PERF_TIMER_GUARD(find_next_user_entry_time); - FindNextUserEntryInternal(skipping); + FindNextUserEntryInternal(skipping, prefix_check); } // Actual implementation of DBIter::FindNextUserEntry() -void DBIter::FindNextUserEntryInternal(bool skipping) { +void DBIter::FindNextUserEntryInternal(bool skipping, bool prefix_check) { // Loop until we hit an acceptable entry to yield assert(iter_->Valid()); assert(direction_ == kForward); current_entry_is_merged_ = false; + + // How many times in a row we have skipped an entry with user key less than + // or equal to saved_key_. We could skip these entries either because + // sequence numbers were too high or because skipping = true. + // What saved_key_ contains throughout this method: + // - if skipping : saved_key_ contains the key that we need to skip, + // and we haven't seen any keys greater than that, + // - if num_skipped > 0 : saved_key_ contains the key that we have skipped + // num_skipped times, and we haven't seen any keys + // greater than that, + // - none of the above : saved_key_ can contain anything, it doesn't matter. uint64_t num_skipped = 0; + + is_blob_ = false; + do { ParsedInternalKey ikey; - if (ParseKey(&ikey) && ikey.sequence <= sequence_) { + + if (!ParseKey(&ikey)) { + // Skip corrupted keys. + iter_->Next(); + continue; + } + + if (iterate_upper_bound_ != nullptr && + user_comparator_->Compare(ikey.user_key, *iterate_upper_bound_) >= 0) { + break; + } + + if (prefix_extractor_ && prefix_check && + prefix_extractor_->Transform(ikey.user_key) + .compare(prefix_start_key_) != 0) { + break; + } + + if (TooManyInternalKeysSkipped()) { + return; + } + + if (ikey.sequence <= sequence_) { if (skipping && - user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) <= 0) { - num_skipped++; // skip this entry + user_comparator_->Compare(ikey.user_key, saved_key_.GetUserKey()) <= + 0) { + num_skipped++; // skip this entry PERF_COUNTER_ADD(internal_key_skipped_count, 1); } else { - skipping = false; + num_skipped = 0; switch (ikey.type) { case kTypeDeletion: + case kTypeSingleDeletion: // Arrange to skip all upcoming entries for this key since // they are hidden by this deletion. - saved_key_.SetKey(ikey.user_key); + saved_key_.SetUserKey( + ikey.user_key, + !iter_->IsKeyPinned() || !pin_thru_lifetime_ /* copy */); skipping = true; - num_skipped = 0; PERF_COUNTER_ADD(internal_delete_skipped_count, 1); break; case kTypeValue: - valid_ = true; - saved_key_.SetKey(ikey.user_key); - return; + case kTypeBlobIndex: + saved_key_.SetUserKey( + ikey.user_key, + !iter_->IsKeyPinned() || !pin_thru_lifetime_ /* copy */); + if (range_del_agg_.ShouldDelete( + ikey, RangeDelAggregator::RangePositioningMode:: + kForwardTraversal)) { + // Arrange to skip all upcoming entries for this key since + // they are hidden by this deletion. + skipping = true; + num_skipped = 0; + PERF_COUNTER_ADD(internal_delete_skipped_count, 1); + } else if (ikey.type == kTypeBlobIndex) { + if (!allow_blob_) { + ROCKS_LOG_ERROR(logger_, "Encounter unexpected blob index."); + status_ = Status::NotSupported( + "Encounter unexpected blob index. Please open DB with " + "rocksdb::blob_db::BlobDB instead."); + valid_ = false; + } else { + is_blob_ = true; + valid_ = true; + } + return; + } else { + valid_ = true; + return; + } + break; case kTypeMerge: - // By now, we are sure the current ikey is going to yield a value - saved_key_.SetKey(ikey.user_key); - current_entry_is_merged_ = true; - valid_ = true; - MergeValuesNewToOld(); // Go to a different state machine - return; + saved_key_.SetUserKey( + ikey.user_key, + !iter_->IsKeyPinned() || !pin_thru_lifetime_ /* copy */); + if (range_del_agg_.ShouldDelete( + ikey, RangeDelAggregator::RangePositioningMode:: + kForwardTraversal)) { + // Arrange to skip all upcoming entries for this key since + // they are hidden by this deletion. + skipping = true; + num_skipped = 0; + PERF_COUNTER_ADD(internal_delete_skipped_count, 1); + } else { + // By now, we are sure the current ikey is going to yield a + // value + current_entry_is_merged_ = true; + valid_ = true; + MergeValuesNewToOld(); // Go to a different state machine + return; + } + break; default: assert(false); break; } } + } else { + // This key was inserted after our snapshot was taken. + PERF_COUNTER_ADD(internal_recent_skipped_count, 1); + + // Here saved_key_ may contain some old key, or the default empty key, or + // key assigned by some random other method. We don't care. + if (user_comparator_->Compare(ikey.user_key, saved_key_.GetUserKey()) <= + 0) { + num_skipped++; + } else { + saved_key_.SetUserKey( + ikey.user_key, + !iter_->IsKeyPinned() || !pin_thru_lifetime_ /* copy */); + skipping = false; + num_skipped = 0; + } } - // If we have sequentially iterated via numerous keys and still not - // found the next user-key, then it is better to seek so that we can - // avoid too many key comparisons. We seek to the last occurence of - // our current key by looking for sequence number 0. - if (skipping && num_skipped > max_skip_) { + + // If we have sequentially iterated via numerous equal keys, then it's + // better to seek so that we can avoid too many key comparisons. + if (num_skipped > max_skip_) { num_skipped = 0; std::string last_key; - AppendInternalKey(&last_key, ParsedInternalKey(saved_key_.GetKey(), 0, - kValueTypeForSeek)); + if (skipping) { + // We're looking for the next user-key but all we see are the same + // user-key with decreasing sequence numbers. Fast forward to + // sequence number 0 and type deletion (the smallest type). + AppendInternalKey(&last_key, ParsedInternalKey(saved_key_.GetUserKey(), + 0, kTypeDeletion)); + // Don't set skipping = false because we may still see more user-keys + // equal to saved_key_. + } else { + // We saw multiple entries with this user key and sequence numbers + // higher than sequence_. Fast forward to sequence_. + // Note that this only covers a case when a higher key was overwritten + // many times since our snapshot was taken, not the case when a lot of + // different keys were inserted after our snapshot was taken. + AppendInternalKey(&last_key, + ParsedInternalKey(saved_key_.GetUserKey(), sequence_, + kValueTypeForSeek)); + } iter_->Seek(last_key); RecordTick(statistics_, NUMBER_OF_RESEEKS_IN_ITERATION); } else { @@ -265,53 +538,73 @@ void DBIter::FindNextUserEntryInternal(bool skipping) { // POST: saved_value_ has the merged value for the user key // iter_ points to the next entry (or invalid) void DBIter::MergeValuesNewToOld() { - if (!user_merge_operator_) { - Log(logger_, "Options::merge_operator is null."); - throw std::logic_error("DBIter::MergeValuesNewToOld() with" - " Options::merge_operator null"); + if (!merge_operator_) { + ROCKS_LOG_ERROR(logger_, "Options::merge_operator is null."); + status_ = Status::InvalidArgument("merge_operator_ must be set."); + valid_ = false; + return; } + // Temporarily pin the blocks that hold merge operands + TempPinData(); + merge_context_.Clear(); // Start the merge process by pushing the first operand - std::deque operands; - operands.push_front(iter_->value().ToString()); + merge_context_.PushOperand(iter_->value(), + iter_->IsValuePinned() /* operand_pinned */); - std::string merge_result; // Temporary string to hold merge result later ParsedInternalKey ikey; + Status s; for (iter_->Next(); iter_->Valid(); iter_->Next()) { if (!ParseKey(&ikey)) { // skip corrupted key continue; } - if (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) != 0) { + if (!user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey())) { // hit the next user key, stop right here break; - } - - if (kTypeDeletion == ikey.type) { + } else if (kTypeDeletion == ikey.type || kTypeSingleDeletion == ikey.type || + range_del_agg_.ShouldDelete( + ikey, RangeDelAggregator::RangePositioningMode:: + kForwardTraversal)) { // hit a delete with the same user key, stop right here // iter_ is positioned after delete iter_->Next(); break; - } - - if (kTypeValue == ikey.type) { + } else if (kTypeValue == ikey.type) { // hit a put, merge the put value with operands and store the // final result in saved_value_. We are done! // ignore corruption if there is any. - const Slice value = iter_->value(); - user_merge_operator_->FullMerge(ikey.user_key, &value, operands, - &saved_value_, logger_); + const Slice val = iter_->value(); + s = MergeHelper::TimedFullMerge( + merge_operator_, ikey.user_key, &val, merge_context_.GetOperands(), + &saved_value_, logger_, statistics_, env_, &pinned_value_, true); + if (!s.ok()) { + status_ = s; + } // iter_ is positioned after put iter_->Next(); return; - } - - if (kTypeMerge == ikey.type) { + } else if (kTypeMerge == ikey.type) { // hit a merge, add the value as an operand and run associative merge. // when complete, add result to operands and continue. - const Slice& value = iter_->value(); - operands.push_front(value.ToString()); + merge_context_.PushOperand(iter_->value(), + iter_->IsValuePinned() /* operand_pinned */); + PERF_COUNTER_ADD(internal_merge_count, 1); + } else if (kTypeBlobIndex == ikey.type) { + if (!allow_blob_) { + ROCKS_LOG_ERROR(logger_, "Encounter unexpected blob index."); + status_ = Status::NotSupported( + "Encounter unexpected blob index. Please open DB with " + "rocksdb::blob_db::BlobDB instead."); + } else { + status_ = + Status::NotSupported("Blob DB does not support merge operator."); + } + valid_ = false; + return; + } else { + assert(false); } } @@ -319,17 +612,87 @@ void DBIter::MergeValuesNewToOld() { // a deletion marker. // feed null as the existing value to the merge operator, such that // client can differentiate this scenario and do things accordingly. - user_merge_operator_->FullMerge(saved_key_.GetKey(), nullptr, operands, - &saved_value_, logger_); + s = MergeHelper::TimedFullMerge(merge_operator_, saved_key_.GetUserKey(), + nullptr, merge_context_.GetOperands(), + &saved_value_, logger_, statistics_, env_, + &pinned_value_, true); + if (!s.ok()) { + status_ = s; + } } void DBIter::Prev() { assert(valid_); + ReleaseTempPinnedData(); + ResetInternalKeysSkippedCounter(); if (direction_ == kForward) { - FindPrevUserKey(); - direction_ = kReverse; + ReverseToBackward(); } PrevInternal(); + if (statistics_ != nullptr) { + local_stats_.prev_count_++; + if (valid_) { + local_stats_.prev_found_count_++; + local_stats_.bytes_read_ += (key().size() + value().size()); + } + } +} + +void DBIter::ReverseToForward() { + if (prefix_extractor_ != nullptr && !total_order_seek_) { + IterKey last_key; + last_key.SetInternalKey(ParsedInternalKey( + saved_key_.GetUserKey(), kMaxSequenceNumber, kValueTypeForSeek)); + iter_->Seek(last_key.GetInternalKey()); + } + FindNextUserKey(); + direction_ = kForward; + if (!iter_->Valid()) { + iter_->SeekToFirst(); + range_del_agg_.InvalidateTombstoneMapPositions(); + } +} + +void DBIter::ReverseToBackward() { + if (prefix_extractor_ != nullptr && !total_order_seek_) { + IterKey last_key; + last_key.SetInternalKey(ParsedInternalKey(saved_key_.GetUserKey(), 0, + kValueTypeForSeekForPrev)); + iter_->SeekForPrev(last_key.GetInternalKey()); + } + if (current_entry_is_merged_) { + // Not placed in the same key. Need to call Prev() until finding the + // previous key. + if (!iter_->Valid()) { + iter_->SeekToLast(); + range_del_agg_.InvalidateTombstoneMapPositions(); + } + ParsedInternalKey ikey; + FindParseableKey(&ikey, kReverse); + while (iter_->Valid() && + user_comparator_->Compare(ikey.user_key, saved_key_.GetUserKey()) > + 0) { + assert(ikey.sequence != kMaxSequenceNumber); + if (ikey.sequence > sequence_) { + PERF_COUNTER_ADD(internal_recent_skipped_count, 1); + } else { + PERF_COUNTER_ADD(internal_key_skipped_count, 1); + } + iter_->Prev(); + FindParseableKey(&ikey, kReverse); + } + } +#ifndef NDEBUG + if (iter_->Valid()) { + ParsedInternalKey ikey; + assert(ParseKey(&ikey)); + assert(user_comparator_->Compare(ikey.user_key, saved_key_.GetUserKey()) <= + 0); + } +#endif + + FindPrevUserKey(); + direction_ = kReverse; } void DBIter::PrevInternal() { @@ -341,48 +704,69 @@ void DBIter::PrevInternal() { ParsedInternalKey ikey; while (iter_->Valid()) { - saved_key_.SetKey(ExtractUserKey(iter_->key())); + saved_key_.SetUserKey( + ExtractUserKey(iter_->key()), + !iter_->IsKeyPinned() || !pin_thru_lifetime_ /* copy */); + if (FindValueForCurrentKey()) { - valid_ = true; if (!iter_->Valid()) { return; } FindParseableKey(&ikey, kReverse); - if (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0) { + if (user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey())) { FindPrevUserKey(); } + if (valid_ && prefix_extractor_ && prefix_same_as_start_ && + prefix_extractor_->Transform(saved_key_.GetUserKey()) + .compare(prefix_start_key_) != 0) { + valid_ = false; + } + return; + } + + if (TooManyInternalKeysSkipped(false)) { return; } + if (!iter_->Valid()) { break; } FindParseableKey(&ikey, kReverse); - if (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0) { - + if (user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey())) { FindPrevUserKey(); } } // We haven't found any key - iterator is not valid + // Or the prefix is different than start prefix assert(!iter_->Valid()); valid_ = false; } // This function checks, if the entry with biggest sequence_number <= sequence_ -// is non kTypeDeletion. If it's not, we save value in saved_value_ +// is non kTypeDeletion or kTypeSingleDeletion. If it's not, we save value in +// saved_value_ bool DBIter::FindValueForCurrentKey() { assert(iter_->Valid()); - // Contains operands for merge operator. - std::deque operands; - // last entry before merge (could be kTypeDeletion or kTypeValue) + merge_context_.Clear(); + current_entry_is_merged_ = false; + // last entry before merge (could be kTypeDeletion, kTypeSingleDeletion or + // kTypeValue) ValueType last_not_merge_type = kTypeDeletion; ValueType last_key_entry_type = kTypeDeletion; ParsedInternalKey ikey; FindParseableKey(&ikey, kReverse); + // Temporarily pin blocks that hold (merge operands / the value) + ReleaseTempPinnedData(); + TempPinData(); size_t num_skipped = 0; while (iter_->Valid() && ikey.sequence <= sequence_ && - (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0)) { + user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey())) { + if (TooManyInternalKeysSkipped()) { + return false; + } + // We iterate too much: let's use Seek() to avoid too much key comparisons if (num_skipped >= max_skip_) { return FindValueForCurrentKeyUsingSeek(); @@ -391,61 +775,122 @@ bool DBIter::FindValueForCurrentKey() { last_key_entry_type = ikey.type; switch (last_key_entry_type) { case kTypeValue: - operands.clear(); - saved_value_ = iter_->value().ToString(); - last_not_merge_type = kTypeValue; + case kTypeBlobIndex: + if (range_del_agg_.ShouldDelete( + ikey, + RangeDelAggregator::RangePositioningMode::kBackwardTraversal)) { + last_key_entry_type = kTypeRangeDeletion; + PERF_COUNTER_ADD(internal_delete_skipped_count, 1); + } else { + assert(iter_->IsValuePinned()); + pinned_value_ = iter_->value(); + } + merge_context_.Clear(); + last_not_merge_type = last_key_entry_type; break; case kTypeDeletion: - operands.clear(); - last_not_merge_type = kTypeDeletion; + case kTypeSingleDeletion: + merge_context_.Clear(); + last_not_merge_type = last_key_entry_type; + PERF_COUNTER_ADD(internal_delete_skipped_count, 1); break; case kTypeMerge: - assert(user_merge_operator_ != nullptr); - operands.push_back(iter_->value().ToString()); + if (range_del_agg_.ShouldDelete( + ikey, + RangeDelAggregator::RangePositioningMode::kBackwardTraversal)) { + merge_context_.Clear(); + last_key_entry_type = kTypeRangeDeletion; + last_not_merge_type = last_key_entry_type; + PERF_COUNTER_ADD(internal_delete_skipped_count, 1); + } else { + assert(merge_operator_ != nullptr); + merge_context_.PushOperandBack( + iter_->value(), iter_->IsValuePinned() /* operand_pinned */); + PERF_COUNTER_ADD(internal_merge_count, 1); + } break; default: assert(false); } - assert(user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0); + PERF_COUNTER_ADD(internal_key_skipped_count, 1); + assert(user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey())); iter_->Prev(); ++num_skipped; FindParseableKey(&ikey, kReverse); } + Status s; + is_blob_ = false; switch (last_key_entry_type) { case kTypeDeletion: + case kTypeSingleDeletion: + case kTypeRangeDeletion: valid_ = false; return false; case kTypeMerge: - if (last_not_merge_type == kTypeDeletion) { - user_merge_operator_->FullMerge(saved_key_.GetKey(), nullptr, operands, - &saved_value_, logger_); + current_entry_is_merged_ = true; + if (last_not_merge_type == kTypeDeletion || + last_not_merge_type == kTypeSingleDeletion || + last_not_merge_type == kTypeRangeDeletion) { + s = MergeHelper::TimedFullMerge( + merge_operator_, saved_key_.GetUserKey(), nullptr, + merge_context_.GetOperands(), &saved_value_, logger_, statistics_, + env_, &pinned_value_, true); + } else if (last_not_merge_type == kTypeBlobIndex) { + if (!allow_blob_) { + ROCKS_LOG_ERROR(logger_, "Encounter unexpected blob index."); + status_ = Status::NotSupported( + "Encounter unexpected blob index. Please open DB with " + "rocksdb::blob_db::BlobDB instead."); + } else { + status_ = + Status::NotSupported("Blob DB does not support merge operator."); + } + valid_ = false; + return true; } else { assert(last_not_merge_type == kTypeValue); - std::string last_put_value = saved_value_; - Slice temp_slice(last_put_value); - user_merge_operator_->FullMerge(saved_key_.GetKey(), &temp_slice, - operands, &saved_value_, logger_); + s = MergeHelper::TimedFullMerge( + merge_operator_, saved_key_.GetUserKey(), &pinned_value_, + merge_context_.GetOperands(), &saved_value_, logger_, statistics_, + env_, &pinned_value_, true); } break; case kTypeValue: // do nothing - we've already has value in saved_value_ break; + case kTypeBlobIndex: + if (!allow_blob_) { + ROCKS_LOG_ERROR(logger_, "Encounter unexpected blob index."); + status_ = Status::NotSupported( + "Encounter unexpected blob index. Please open DB with " + "rocksdb::blob_db::BlobDB instead."); + valid_ = false; + return true; + } + is_blob_ = true; + break; default: assert(false); break; } valid_ = true; + if (!s.ok()) { + status_ = s; + } return true; } // This function is used in FindValueForCurrentKey. // We use Seek() function instead of Prev() to find necessary value bool DBIter::FindValueForCurrentKeyUsingSeek() { + // FindValueForCurrentKey will enable pinning before calling + // FindValueForCurrentKeyUsingSeek() + assert(pinned_iters_mgr_.PinningEnabled()); std::string last_key; - AppendInternalKey(&last_key, ParsedInternalKey(saved_key_.GetKey(), sequence_, - kValueTypeForSeek)); + AppendInternalKey(&last_key, ParsedInternalKey(saved_key_.GetUserKey(), + sequence_, kValueTypeForSeek)); iter_->Seek(last_key); RecordTick(statistics_, NUMBER_OF_RESEEKS_IN_ITERATION); @@ -453,47 +898,76 @@ bool DBIter::FindValueForCurrentKeyUsingSeek() { ParsedInternalKey ikey; FindParseableKey(&ikey, kForward); - if (ikey.type == kTypeValue || ikey.type == kTypeDeletion) { - if (ikey.type == kTypeValue) { - saved_value_ = iter_->value().ToString(); - valid_ = true; - return true; - } + if (ikey.type == kTypeDeletion || ikey.type == kTypeSingleDeletion || + range_del_agg_.ShouldDelete( + ikey, RangeDelAggregator::RangePositioningMode::kBackwardTraversal)) { valid_ = false; return false; } + if (ikey.type == kTypeBlobIndex && !allow_blob_) { + ROCKS_LOG_ERROR(logger_, "Encounter unexpected blob index."); + status_ = Status::NotSupported( + "Encounter unexpected blob index. Please open DB with " + "rocksdb::blob_db::BlobDB instead."); + valid_ = false; + return true; + } + if (ikey.type == kTypeValue || ikey.type == kTypeBlobIndex) { + assert(iter_->IsValuePinned()); + pinned_value_ = iter_->value(); + valid_ = true; + return true; + } // kTypeMerge. We need to collect all kTypeMerge values and save them // in operands - std::deque operands; - while (iter_->Valid() && - (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0) && - ikey.type == kTypeMerge) { - operands.push_front(iter_->value().ToString()); + current_entry_is_merged_ = true; + merge_context_.Clear(); + while ( + iter_->Valid() && + user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey()) && + ikey.type == kTypeMerge && + !range_del_agg_.ShouldDelete( + ikey, RangeDelAggregator::RangePositioningMode::kBackwardTraversal)) { + merge_context_.PushOperand(iter_->value(), + iter_->IsValuePinned() /* operand_pinned */); + PERF_COUNTER_ADD(internal_merge_count, 1); iter_->Next(); FindParseableKey(&ikey, kForward); } + Status s; if (!iter_->Valid() || - (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) != 0) || - ikey.type == kTypeDeletion) { - user_merge_operator_->FullMerge(saved_key_.GetKey(), nullptr, operands, - &saved_value_, logger_); - + !user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey()) || + ikey.type == kTypeDeletion || ikey.type == kTypeSingleDeletion || + range_del_agg_.ShouldDelete( + ikey, RangeDelAggregator::RangePositioningMode::kBackwardTraversal)) { + s = MergeHelper::TimedFullMerge(merge_operator_, saved_key_.GetUserKey(), + nullptr, merge_context_.GetOperands(), + &saved_value_, logger_, statistics_, env_, + &pinned_value_, true); // Make iter_ valid and point to saved_key_ if (!iter_->Valid() || - (user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) != 0)) { + !user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey())) { iter_->Seek(last_key); RecordTick(statistics_, NUMBER_OF_RESEEKS_IN_ITERATION); } valid_ = true; + if (!s.ok()) { + status_ = s; + } return true; } - const Slice& value = iter_->value(); - user_merge_operator_->FullMerge(saved_key_.GetKey(), &value, operands, - &saved_value_, logger_); + const Slice& val = iter_->value(); + s = MergeHelper::TimedFullMerge(merge_operator_, saved_key_.GetUserKey(), + &val, merge_context_.GetOperands(), + &saved_value_, logger_, statistics_, env_, + &pinned_value_, true); valid_ = true; + if (!s.ok()) { + status_ = s; + } return true; } @@ -508,7 +982,7 @@ void DBIter::FindNextUserKey() { ParsedInternalKey ikey; FindParseableKey(&ikey, kForward); while (iter_->Valid() && - user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) != 0) { + !user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey())) { iter_->Next(); FindParseableKey(&ikey, kForward); } @@ -522,23 +996,50 @@ void DBIter::FindPrevUserKey() { size_t num_skipped = 0; ParsedInternalKey ikey; FindParseableKey(&ikey, kReverse); + int cmp; while (iter_->Valid() && - user_comparator_->Compare(ikey.user_key, saved_key_.GetKey()) == 0) { - if (num_skipped >= max_skip_) { - num_skipped = 0; - IterKey last_key; - last_key.SetInternalKey(ParsedInternalKey( - saved_key_.GetKey(), kMaxSequenceNumber, kValueTypeForSeek)); - iter_->Seek(last_key.GetKey()); - RecordTick(statistics_, NUMBER_OF_RESEEKS_IN_ITERATION); + ((cmp = user_comparator_->Compare(ikey.user_key, + saved_key_.GetUserKey())) == 0 || + (cmp > 0 && ikey.sequence > sequence_))) { + if (TooManyInternalKeysSkipped()) { + return; } + if (cmp == 0) { + if (num_skipped >= max_skip_) { + num_skipped = 0; + IterKey last_key; + last_key.SetInternalKey(ParsedInternalKey( + saved_key_.GetUserKey(), kMaxSequenceNumber, kValueTypeForSeek)); + iter_->Seek(last_key.GetInternalKey()); + RecordTick(statistics_, NUMBER_OF_RESEEKS_IN_ITERATION); + } else { + ++num_skipped; + } + } + assert(ikey.sequence != kMaxSequenceNumber); + if (ikey.sequence > sequence_) { + PERF_COUNTER_ADD(internal_recent_skipped_count, 1); + } else { + PERF_COUNTER_ADD(internal_key_skipped_count, 1); + } iter_->Prev(); - ++num_skipped; FindParseableKey(&ikey, kReverse); } } +bool DBIter::TooManyInternalKeysSkipped(bool increment) { + if ((max_skippable_internal_keys_ > 0) && + (num_internal_keys_skipped_ > max_skippable_internal_keys_)) { + valid_ = false; + status_ = Status::Incomplete("Too many internal keys skipped."); + return true; + } else if (increment) { + num_internal_keys_skipped_++; + } + return false; +} + // Skip all unparseable keys void DBIter::FindParseableKey(ParsedInternalKey* ikey, Direction direction) { while (iter_->Valid() && !ParseKey(ikey)) { @@ -552,76 +1053,191 @@ void DBIter::FindParseableKey(ParsedInternalKey* ikey, Direction direction) { void DBIter::Seek(const Slice& target) { StopWatch sw(env_, statistics_, DB_SEEK); - + ReleaseTempPinnedData(); + ResetInternalKeysSkippedCounter(); saved_key_.Clear(); - // now savved_key is used to store internal key. saved_key_.SetInternalKey(target, sequence_); { PERF_TIMER_GUARD(seek_internal_seek_time); - iter_->Seek(saved_key_.GetKey()); + iter_->Seek(saved_key_.GetInternalKey()); + range_del_agg_.InvalidateTombstoneMapPositions(); } - + RecordTick(statistics_, NUMBER_DB_SEEK); if (iter_->Valid()) { + if (prefix_extractor_ && prefix_same_as_start_) { + prefix_start_key_ = prefix_extractor_->Transform(target); + } direction_ = kForward; ClearSavedValue(); - FindNextUserEntry(false /*not skipping */); + FindNextUserEntry(false /* not skipping */, prefix_same_as_start_); + if (!valid_) { + prefix_start_key_.clear(); + } + if (statistics_ != nullptr) { + if (valid_) { + RecordTick(statistics_, NUMBER_DB_SEEK_FOUND); + RecordTick(statistics_, ITER_BYTES_READ, key().size() + value().size()); + PERF_COUNTER_ADD(iter_read_bytes, key().size() + value().size()); + } + } + } else { + valid_ = false; + } + + if (valid_ && prefix_extractor_ && prefix_same_as_start_) { + prefix_start_buf_.SetUserKey(prefix_start_key_); + prefix_start_key_ = prefix_start_buf_.GetUserKey(); + } +} + +void DBIter::SeekForPrev(const Slice& target) { + StopWatch sw(env_, statistics_, DB_SEEK); + ReleaseTempPinnedData(); + ResetInternalKeysSkippedCounter(); + saved_key_.Clear(); + // now saved_key is used to store internal key. + saved_key_.SetInternalKey(target, 0 /* sequence_number */, + kValueTypeForSeekForPrev); + + { + PERF_TIMER_GUARD(seek_internal_seek_time); + iter_->SeekForPrev(saved_key_.GetInternalKey()); + range_del_agg_.InvalidateTombstoneMapPositions(); + } + + RecordTick(statistics_, NUMBER_DB_SEEK); + if (iter_->Valid()) { + if (prefix_extractor_ && prefix_same_as_start_) { + prefix_start_key_ = prefix_extractor_->Transform(target); + } + direction_ = kReverse; + ClearSavedValue(); + PrevInternal(); + if (!valid_) { + prefix_start_key_.clear(); + } + if (statistics_ != nullptr) { + if (valid_) { + RecordTick(statistics_, NUMBER_DB_SEEK_FOUND); + RecordTick(statistics_, ITER_BYTES_READ, key().size() + value().size()); + PERF_COUNTER_ADD(iter_read_bytes, key().size() + value().size()); + } + } } else { valid_ = false; } + if (valid_ && prefix_extractor_ && prefix_same_as_start_) { + prefix_start_buf_.SetUserKey(prefix_start_key_); + prefix_start_key_ = prefix_start_buf_.GetUserKey(); + } } void DBIter::SeekToFirst() { // Don't use iter_::Seek() if we set a prefix extractor - // because prefix seek wiil be used. - if (has_prefix_extractor_) { + // because prefix seek will be used. + if (prefix_extractor_ != nullptr) { max_skip_ = std::numeric_limits::max(); } direction_ = kForward; + ReleaseTempPinnedData(); + ResetInternalKeysSkippedCounter(); ClearSavedValue(); { PERF_TIMER_GUARD(seek_internal_seek_time); iter_->SeekToFirst(); + range_del_agg_.InvalidateTombstoneMapPositions(); } + RecordTick(statistics_, NUMBER_DB_SEEK); if (iter_->Valid()) { - FindNextUserEntry(false /* not skipping */); + saved_key_.SetUserKey( + ExtractUserKey(iter_->key()), + !iter_->IsKeyPinned() || !pin_thru_lifetime_ /* copy */); + FindNextUserEntry(false /* not skipping */, false /* no prefix check */); + if (statistics_ != nullptr) { + if (valid_) { + RecordTick(statistics_, NUMBER_DB_SEEK_FOUND); + RecordTick(statistics_, ITER_BYTES_READ, key().size() + value().size()); + PERF_COUNTER_ADD(iter_read_bytes, key().size() + value().size()); + } + } } else { valid_ = false; } + if (valid_ && prefix_extractor_ && prefix_same_as_start_) { + prefix_start_buf_.SetUserKey( + prefix_extractor_->Transform(saved_key_.GetUserKey())); + prefix_start_key_ = prefix_start_buf_.GetUserKey(); + } } void DBIter::SeekToLast() { // Don't use iter_::Seek() if we set a prefix extractor - // because prefix seek wiil be used. - if (has_prefix_extractor_) { + // because prefix seek will be used. + if (prefix_extractor_ != nullptr) { max_skip_ = std::numeric_limits::max(); } direction_ = kReverse; + ReleaseTempPinnedData(); + ResetInternalKeysSkippedCounter(); ClearSavedValue(); { PERF_TIMER_GUARD(seek_internal_seek_time); iter_->SeekToLast(); + range_del_agg_.InvalidateTombstoneMapPositions(); + } + // When the iterate_upper_bound is set to a value, + // it will seek to the last key before the + // ReadOptions.iterate_upper_bound + if (iter_->Valid() && iterate_upper_bound_ != nullptr) { + SeekForPrev(*iterate_upper_bound_); + range_del_agg_.InvalidateTombstoneMapPositions(); + if (!Valid()) { + return; + } else if (user_comparator_->Equal(*iterate_upper_bound_, key())) { + Prev(); + } + } else { + PrevInternal(); + } + if (statistics_ != nullptr) { + RecordTick(statistics_, NUMBER_DB_SEEK); + if (valid_) { + RecordTick(statistics_, NUMBER_DB_SEEK_FOUND); + RecordTick(statistics_, ITER_BYTES_READ, key().size() + value().size()); + PERF_COUNTER_ADD(iter_read_bytes, key().size() + value().size()); + } + } + if (valid_ && prefix_extractor_ && prefix_same_as_start_) { + prefix_start_buf_.SetUserKey( + prefix_extractor_->Transform(saved_key_.GetUserKey())); + prefix_start_key_ = prefix_start_buf_.GetUserKey(); } - - PrevInternal(); } -Iterator* NewDBIterator(Env* env, const Options& options, +Iterator* NewDBIterator(Env* env, const ReadOptions& read_options, + const ImmutableCFOptions& cf_options, const Comparator* user_key_comparator, - Iterator* internal_iter, - const SequenceNumber& sequence) { - return new DBIter(env, options, user_key_comparator, internal_iter, sequence, - false); + InternalIterator* internal_iter, + const SequenceNumber& sequence, + uint64_t max_sequential_skip_in_iterations, + bool allow_blob) { + DBIter* db_iter = new DBIter( + env, read_options, cf_options, user_key_comparator, internal_iter, + sequence, false, max_sequential_skip_in_iterations, allow_blob); + return db_iter; } ArenaWrappedDBIter::~ArenaWrappedDBIter() { db_iter_->~DBIter(); } -void ArenaWrappedDBIter::SetDBIter(DBIter* iter) { db_iter_ = iter; } +RangeDelAggregator* ArenaWrappedDBIter::GetRangeDelAggregator() { + return db_iter_->GetRangeDelAggregator(); +} -void ArenaWrappedDBIter::SetIterUnderDBIter(Iterator* iter) { +void ArenaWrappedDBIter::SetIterUnderDBIter(InternalIterator* iter) { static_cast(db_iter_)->SetIter(iter); } @@ -631,25 +1247,79 @@ inline void ArenaWrappedDBIter::SeekToLast() { db_iter_->SeekToLast(); } inline void ArenaWrappedDBIter::Seek(const Slice& target) { db_iter_->Seek(target); } +inline void ArenaWrappedDBIter::SeekForPrev(const Slice& target) { + db_iter_->SeekForPrev(target); +} inline void ArenaWrappedDBIter::Next() { db_iter_->Next(); } inline void ArenaWrappedDBIter::Prev() { db_iter_->Prev(); } inline Slice ArenaWrappedDBIter::key() const { return db_iter_->key(); } inline Slice ArenaWrappedDBIter::value() const { return db_iter_->value(); } inline Status ArenaWrappedDBIter::status() const { return db_iter_->status(); } -void ArenaWrappedDBIter::RegisterCleanup(CleanupFunction function, void* arg1, - void* arg2) { - db_iter_->RegisterCleanup(function, arg1, arg2); +bool ArenaWrappedDBIter::IsBlob() const { return db_iter_->IsBlob(); } +inline Status ArenaWrappedDBIter::GetProperty(std::string prop_name, + std::string* prop) { + if (prop_name == "rocksdb.iterator.super-version-number") { + // First try to pass the value returned from inner iterator. + if (!db_iter_->GetProperty(prop_name, prop).ok()) { + *prop = ToString(sv_number_); + } + return Status::OK(); + } + return db_iter_->GetProperty(prop_name, prop); +} + +void ArenaWrappedDBIter::Init(Env* env, const ReadOptions& read_options, + const ImmutableCFOptions& cf_options, + const SequenceNumber& sequence, + uint64_t max_sequential_skip_in_iteration, + uint64_t version_number, bool allow_blob) { + auto mem = arena_.AllocateAligned(sizeof(DBIter)); + db_iter_ = new (mem) + DBIter(env, read_options, cf_options, cf_options.user_comparator, nullptr, + sequence, true, max_sequential_skip_in_iteration, allow_blob); + sv_number_ = version_number; +} + +Status ArenaWrappedDBIter::Refresh() { + if (cfd_ == nullptr || db_impl_ == nullptr) { + return Status::NotSupported("Creating renew iterator is not allowed."); + } + assert(db_iter_ != nullptr); + SequenceNumber latest_seq = db_impl_->GetLatestSequenceNumber(); + uint64_t cur_sv_number = cfd_->GetSuperVersionNumber(); + if (sv_number_ != cur_sv_number) { + Env* env = db_iter_->env(); + db_iter_->~DBIter(); + arena_.~Arena(); + new (&arena_) Arena(); + + SuperVersion* sv = cfd_->GetReferencedSuperVersion(db_impl_->mutex()); + Init(env, read_options_, *(cfd_->ioptions()), latest_seq, + sv->mutable_cf_options.max_sequential_skip_in_iterations, + cur_sv_number, allow_blob_); + + InternalIterator* internal_iter = db_impl_->NewInternalIterator( + read_options_, cfd_, sv, &arena_, db_iter_->GetRangeDelAggregator()); + SetIterUnderDBIter(internal_iter); + } else { + db_iter_->set_sequence(latest_seq); + db_iter_->set_valid(false); + } + return Status::OK(); } ArenaWrappedDBIter* NewArenaWrappedDbIterator( - Env* env, const Options& options, const Comparator* user_key_comparator, - const SequenceNumber& sequence) { + Env* env, const ReadOptions& read_options, + const ImmutableCFOptions& cf_options, const SequenceNumber& sequence, + uint64_t max_sequential_skip_in_iterations, uint64_t version_number, + DBImpl* db_impl, ColumnFamilyData* cfd, bool allow_blob) { ArenaWrappedDBIter* iter = new ArenaWrappedDBIter(); - Arena* arena = iter->GetArena(); - auto mem = arena->AllocateAligned(sizeof(DBIter)); - DBIter* db_iter = new (mem) - DBIter(env, options, user_key_comparator, nullptr, sequence, true); - iter->SetDBIter(db_iter); + iter->Init(env, read_options, cf_options, sequence, + max_sequential_skip_in_iterations, version_number, allow_blob); + if (db_impl != nullptr && cfd != nullptr) { + iter->StoreRefreshInfo(read_options, db_impl, cfd, allow_blob); + } + return iter; } diff --git a/db/db_iter.h b/db/db_iter.h index cb9840324ff..26fcd44cbd2 100644 --- a/db/db_iter.h +++ b/db/db_iter.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,8 +9,13 @@ #pragma once #include -#include "rocksdb/db.h" +#include +#include "db/db_impl.h" #include "db/dbformat.h" +#include "db/range_del_aggregator.h" +#include "options/cf_options.h" +#include "rocksdb/db.h" +#include "rocksdb/iterator.h" #include "util/arena.h" #include "util/autovector.h" @@ -18,16 +23,18 @@ namespace rocksdb { class Arena; class DBIter; +class InternalIterator; // Return a new iterator that converts internal keys (yielded by // "*internal_iter") that were live at the specified "sequence" number // into appropriate user keys. -extern Iterator* NewDBIterator( - Env* env, - const Options& options, - const Comparator *user_key_comparator, - Iterator* internal_iter, - const SequenceNumber& sequence); +extern Iterator* NewDBIterator(Env* env, const ReadOptions& read_options, + const ImmutableCFOptions& cf_options, + const Comparator* user_key_comparator, + InternalIterator* internal_iter, + const SequenceNumber& sequence, + uint64_t max_sequential_skip_in_iterations, + bool allow_blob = false); // A wrapper iterator which wraps DB Iterator and the arena, with which the DB // iterator is supposed be allocated. This class is used as an entry point of @@ -41,33 +48,58 @@ class ArenaWrappedDBIter : public Iterator { // Get the arena to be used to allocate memory for DBIter to be wrapped, // as well as child iterators in it. virtual Arena* GetArena() { return &arena_; } - - // Set the DB Iterator to be wrapped - - virtual void SetDBIter(DBIter* iter); + virtual RangeDelAggregator* GetRangeDelAggregator(); // Set the internal iterator wrapped inside the DB Iterator. Usually it is // a merging iterator. - virtual void SetIterUnderDBIter(Iterator* iter); + virtual void SetIterUnderDBIter(InternalIterator* iter); virtual bool Valid() const override; virtual void SeekToFirst() override; virtual void SeekToLast() override; virtual void Seek(const Slice& target) override; + virtual void SeekForPrev(const Slice& target) override; virtual void Next() override; virtual void Prev() override; virtual Slice key() const override; virtual Slice value() const override; virtual Status status() const override; - void RegisterCleanup(CleanupFunction function, void* arg1, void* arg2); + virtual Status Refresh() override; + bool IsBlob() const; + + virtual Status GetProperty(std::string prop_name, std::string* prop) override; + + void Init(Env* env, const ReadOptions& read_options, + const ImmutableCFOptions& cf_options, + const SequenceNumber& sequence, + uint64_t max_sequential_skip_in_iterations, uint64_t version_number, + bool allow_blob); + + void StoreRefreshInfo(const ReadOptions& read_options, DBImpl* db_impl, + ColumnFamilyData* cfd, bool allow_blob) { + read_options_ = read_options; + db_impl_ = db_impl; + cfd_ = cfd; + allow_blob_ = allow_blob; + } private: DBIter* db_iter_; Arena arena_; + uint64_t sv_number_; + ColumnFamilyData* cfd_ = nullptr; + DBImpl* db_impl_ = nullptr; + ReadOptions read_options_; + bool allow_blob_ = false; }; // Generate the arena wrapped iterator class. +// `db_impl` and `cfd` are used for reneweal. If left null, renewal will not +// be supported. extern ArenaWrappedDBIter* NewArenaWrappedDbIterator( - Env* env, const Options& options, const Comparator* user_key_comparator, - const SequenceNumber& sequence); + Env* env, const ReadOptions& read_options, + const ImmutableCFOptions& cf_options, const SequenceNumber& sequence, + uint64_t max_sequential_skip_in_iterations, uint64_t version_number, + DBImpl* db_impl = nullptr, ColumnFamilyData* cfd = nullptr, + bool allow_blob = false); } // namespace rocksdb diff --git a/db/db_iter_test.cc b/db/db_iter_test.cc index 4ce79da1bac..6db3b4a9bbb 100644 --- a/db/db_iter_test.cc +++ b/db/db_iter_test.cc @@ -1,30 +1,35 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #include #include #include #include +#include "db/db_iter.h" #include "db/dbformat.h" #include "rocksdb/comparator.h" #include "rocksdb/options.h" +#include "rocksdb/perf_context.h" #include "rocksdb/slice.h" #include "rocksdb/statistics.h" -#include "db/db_iter.h" +#include "table/iterator_wrapper.h" +#include "table/merging_iterator.h" +#include "util/string_util.h" +#include "util/sync_point.h" #include "util/testharness.h" #include "utilities/merge_operators.h" namespace rocksdb { -static uint32_t TestGetTickerCount(const Options& options, +static uint64_t TestGetTickerCount(const Options& options, Tickers ticker_type) { return options.statistics->getTickerCount(ticker_type); } -class TestIterator : public Iterator { +class TestIterator : public InternalIterator { public: explicit TestIterator(const Comparator* comparator) : initialized_(false), @@ -33,21 +38,40 @@ class TestIterator : public Iterator { iter_(0), cmp(comparator) {} - void AddMerge(std::string key, std::string value) { - Add(key, kTypeMerge, value); + void AddPut(std::string argkey, std::string argvalue) { + Add(argkey, kTypeValue, argvalue); + } + + void AddDeletion(std::string argkey) { + Add(argkey, kTypeDeletion, std::string()); } - void AddDeletion(std::string key) { Add(key, kTypeDeletion, std::string()); } + void AddSingleDeletion(std::string argkey) { + Add(argkey, kTypeSingleDeletion, std::string()); + } + + void AddMerge(std::string argkey, std::string argvalue) { + Add(argkey, kTypeMerge, argvalue); + } - void AddPut(std::string key, std::string value) { - Add(key, kTypeValue, value); + void Add(std::string argkey, ValueType type, std::string argvalue) { + Add(argkey, type, argvalue, sequence_number_++); } - void Add(std::string key, ValueType type, std::string value) { + void Add(std::string argkey, ValueType type, std::string argvalue, + size_t seq_num, bool update_iter = false) { valid_ = true; - ParsedInternalKey internal_key(key, sequence_number_++, type); - data_.push_back(std::pair(std::string(), value)); + ParsedInternalKey internal_key(argkey, seq_num, type); + data_.push_back( + std::pair(std::string(), argvalue)); AppendInternalKey(&data_.back().first, internal_key); + if (update_iter && valid_ && cmp.Compare(data_.back().first, key()) < 0) { + // insert a key smaller than current key + Finish(); + // data_[iter_] is not anymore the current element of the iterator. + // Increment it to reposition it to the right position. + iter_++; + } } // should be called before operations with iterator @@ -93,6 +117,11 @@ class TestIterator : public Iterator { } } + virtual void SeekForPrev(const Slice& target) override { + assert(initialized_); + SeekForPrevImpl(target, &cmp); + } + virtual void Next() override { assert(initialized_); if (data_.empty() || (iter_ == data_.size() - 1)) { @@ -126,6 +155,9 @@ class TestIterator : public Iterator { return Status::OK(); } + virtual bool IsKeyPinned() const override { return true; } + virtual bool IsValuePinned() const override { return true; } + private: bool initialized_; bool valid_; @@ -136,15 +168,16 @@ class TestIterator : public Iterator { std::vector> data_; }; -class DBIteratorTest { +class DBIteratorTest : public testing::Test { public: Env* env_; DBIteratorTest() : env_(Env::Default()) {} }; -TEST(DBIteratorTest, DBIteratorPrevNext) { +TEST_F(DBIteratorTest, DBIteratorPrevNext) { Options options; + ImmutableCFOptions cf_options = ImmutableCFOptions(options); { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); @@ -157,8 +190,10 @@ TEST(DBIteratorTest, DBIteratorPrevNext) { internal_iter->AddPut("b", "val_b"); internal_iter->Finish(); + ReadOptions ro; std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 10)); + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); @@ -178,6 +213,273 @@ TEST(DBIteratorTest, DBIteratorPrevNext) { db_iter->Next(); ASSERT_TRUE(!db_iter->Valid()); } + // Test to check the SeekToLast() with iterate_upper_bound not set + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("b", "val_b"); + internal_iter->AddPut("b", "val_b"); + internal_iter->AddPut("c", "val_c"); + internal_iter->Finish(); + + ReadOptions ro; + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); + + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + } + + // Test to check the SeekToLast() with iterate_upper_bound set + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("b", "val_b"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddPut("d", "val_d"); + internal_iter->AddPut("e", "val_e"); + internal_iter->AddPut("f", "val_f"); + internal_iter->Finish(); + + Slice prefix("d"); + + ReadOptions ro; + ro.iterate_upper_bound = &prefix; + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); + + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + + db_iter->Next(); + ASSERT_TRUE(!db_iter->Valid()); + + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + } + // Test to check the SeekToLast() iterate_upper_bound set to a key that + // is not Put yet + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("b", "val_b"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddPut("d", "val_d"); + internal_iter->Finish(); + + Slice prefix("z"); + + ReadOptions ro; + ro.iterate_upper_bound = &prefix; + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); + + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "d"); + + db_iter->Next(); + ASSERT_TRUE(!db_iter->Valid()); + + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "d"); + + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + } + // Test to check the SeekToLast() with iterate_upper_bound set to the + // first key + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("b", "val_b"); + internal_iter->AddPut("b", "val_b"); + internal_iter->Finish(); + + Slice prefix("a"); + + ReadOptions ro; + ro.iterate_upper_bound = &prefix; + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); + + db_iter->SeekToLast(); + ASSERT_TRUE(!db_iter->Valid()); + } + // Test case to check SeekToLast with iterate_upper_bound set + // (same key put may times - SeekToLast should start with the + // maximum sequence id of the upper bound) + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("b", "val_b"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddPut("c", "val_c"); + internal_iter->Finish(); + + Slice prefix("c"); + + ReadOptions ro; + ro.iterate_upper_bound = &prefix; + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 7, options.max_sequential_skip_in_iterations)); + + SetPerfLevel(kEnableCount); + ASSERT_TRUE(GetPerfLevel() == kEnableCount); + + get_perf_context()->Reset(); + db_iter->SeekToLast(); + + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(static_cast(get_perf_context()->internal_key_skipped_count), 7); + ASSERT_EQ(db_iter->key().ToString(), "b"); + + SetPerfLevel(kDisable); + } + // Test to check the SeekToLast() with the iterate_upper_bound set + // (Checking the value of the key which has sequence ids greater than + // and less that the iterator's sequence id) + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + + internal_iter->AddPut("a", "val_a1"); + internal_iter->AddPut("a", "val_a2"); + internal_iter->AddPut("b", "val_b1"); + internal_iter->AddPut("c", "val_c1"); + internal_iter->AddPut("c", "val_c2"); + internal_iter->AddPut("c", "val_c3"); + internal_iter->AddPut("b", "val_b2"); + internal_iter->AddPut("d", "val_d1"); + internal_iter->Finish(); + + Slice prefix("c"); + + ReadOptions ro; + ro.iterate_upper_bound = &prefix; + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 4, options.max_sequential_skip_in_iterations)); + + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "val_b1"); + } + + // Test to check the SeekToLast() with the iterate_upper_bound set to the + // key that is deleted + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddDeletion("a"); + internal_iter->AddPut("b", "val_b"); + internal_iter->AddPut("c", "val_c"); + internal_iter->Finish(); + + Slice prefix("a"); + + ReadOptions ro; + ro.iterate_upper_bound = &prefix; + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); + + db_iter->SeekToLast(); + ASSERT_TRUE(!db_iter->Valid()); + } + // Test to check the SeekToLast() with the iterate_upper_bound set + // (Deletion cases) + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("b", "val_b"); + internal_iter->AddDeletion("b"); + internal_iter->AddPut("c", "val_c"); + internal_iter->Finish(); + + Slice prefix("c"); + + ReadOptions ro; + ro.iterate_upper_bound = &prefix; + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); + + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + + db_iter->Next(); + ASSERT_TRUE(!db_iter->Valid()); + + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + } + // Test to check the SeekToLast() with iterate_upper_bound set + // (Deletion cases - Lot of internal keys after the upper_bound + // is deleted) + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("b", "val_b"); + internal_iter->AddDeletion("c"); + internal_iter->AddDeletion("d"); + internal_iter->AddDeletion("e"); + internal_iter->AddDeletion("f"); + internal_iter->AddDeletion("g"); + internal_iter->AddDeletion("h"); + internal_iter->Finish(); + + Slice prefix("c"); + + ReadOptions ro; + ro.iterate_upper_bound = &prefix; + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 7, options.max_sequential_skip_in_iterations)); + + SetPerfLevel(kEnableCount); + ASSERT_TRUE(GetPerfLevel() == kEnableCount); + + get_perf_context()->Reset(); + db_iter->SeekToLast(); + + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(static_cast(get_perf_context()->internal_delete_skipped_count), 1); + ASSERT_EQ(db_iter->key().ToString(), "b"); + + SetPerfLevel(kDisable); + } { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); @@ -190,8 +492,10 @@ TEST(DBIteratorTest, DBIteratorPrevNext) { internal_iter->AddPut("b", "val_b"); internal_iter->Finish(); + ReadOptions ro; std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 10)); + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); db_iter->SeekToFirst(); ASSERT_TRUE(db_iter->Valid()); @@ -213,7 +517,6 @@ TEST(DBIteratorTest, DBIteratorPrevNext) { } { - Options options; TestIterator* internal_iter = new TestIterator(BytewiseComparator()); internal_iter->AddPut("a", "val_a"); internal_iter->AddPut("b", "val_b"); @@ -231,8 +534,10 @@ TEST(DBIteratorTest, DBIteratorPrevNext) { internal_iter->AddPut("b", "val_b"); internal_iter->Finish(); + ReadOptions ro; std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 2)); + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 2, options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "b"); @@ -248,7 +553,6 @@ TEST(DBIteratorTest, DBIteratorPrevNext) { } { - Options options; TestIterator* internal_iter = new TestIterator(BytewiseComparator()); internal_iter->AddPut("a", "val_a"); internal_iter->AddPut("a", "val_a"); @@ -261,8 +565,10 @@ TEST(DBIteratorTest, DBIteratorPrevNext) { internal_iter->AddPut("c", "val_c"); internal_iter->Finish(); + ReadOptions ro; std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 10)); + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "c"); @@ -280,15 +586,18 @@ TEST(DBIteratorTest, DBIteratorPrevNext) { } } -TEST(DBIteratorTest, DBIteratorEmpty) { +TEST_F(DBIteratorTest, DBIteratorEmpty) { Options options; + ImmutableCFOptions cf_options = ImmutableCFOptions(options); + ReadOptions ro; { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); internal_iter->Finish(); std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 0)); + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 0, options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(!db_iter->Valid()); } @@ -298,13 +607,15 @@ TEST(DBIteratorTest, DBIteratorEmpty) { internal_iter->Finish(); std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 0)); + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 0, options.max_sequential_skip_in_iterations)); db_iter->SeekToFirst(); ASSERT_TRUE(!db_iter->Valid()); } } -TEST(DBIteratorTest, DBIteratorUseSkipCountSkips) { +TEST_F(DBIteratorTest, DBIteratorUseSkipCountSkips) { + ReadOptions ro; Options options; options.statistics = rocksdb::CreateDBStatistics(); options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); @@ -317,8 +628,9 @@ TEST(DBIteratorTest, DBIteratorUseSkipCountSkips) { } internal_iter->Finish(); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 2)); + std::unique_ptr db_iter(NewDBIterator( + env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 2, options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "c"); @@ -342,27 +654,31 @@ TEST(DBIteratorTest, DBIteratorUseSkipCountSkips) { ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 3u); } -TEST(DBIteratorTest, DBIteratorUseSkip) { +TEST_F(DBIteratorTest, DBIteratorUseSkip) { + ReadOptions ro; Options options; options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); + ImmutableCFOptions cf_options = ImmutableCFOptions(options); + { for (size_t i = 0; i < 200; ++i) { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); internal_iter->AddMerge("b", "merge_1"); internal_iter->AddMerge("a", "merge_2"); - for (size_t i = 0; i < 200; ++i) { - internal_iter->AddPut("c", std::to_string(i)); + for (size_t k = 0; k < 200; ++k) { + internal_iter->AddPut("c", ToString(k)); } internal_iter->Finish(); options.statistics = rocksdb::CreateDBStatistics(); std::unique_ptr db_iter(NewDBIterator( - env_, options, BytewiseComparator(), internal_iter, i + 2)); + env_, ro, cf_options, BytewiseComparator(), internal_iter, i + 2, + options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "c"); - ASSERT_EQ(db_iter->value().ToString(), std::to_string(i)); + ASSERT_EQ(db_iter->value().ToString(), ToString(i)); db_iter->Prev(); ASSERT_TRUE(db_iter->Valid()); @@ -384,14 +700,15 @@ TEST(DBIteratorTest, DBIteratorUseSkip) { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); internal_iter->AddMerge("b", "merge_1"); internal_iter->AddMerge("a", "merge_2"); - for (size_t i = 0; i < 200; ++i) { + for (size_t k = 0; k < 200; ++k) { internal_iter->AddDeletion("c"); } internal_iter->AddPut("c", "200"); internal_iter->Finish(); std::unique_ptr db_iter(NewDBIterator( - env_, options, BytewiseComparator(), internal_iter, i + 2)); + env_, ro, cf_options, BytewiseComparator(), internal_iter, i + 2, + options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); @@ -418,7 +735,8 @@ TEST(DBIteratorTest, DBIteratorUseSkip) { internal_iter->Finish(); std::unique_ptr db_iter(NewDBIterator( - env_, options, BytewiseComparator(), internal_iter, 202)); + env_, ro, cf_options, BytewiseComparator(), internal_iter, 202, + options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); @@ -443,13 +761,14 @@ TEST(DBIteratorTest, DBIteratorUseSkip) { { for (size_t i = 0; i < 200; ++i) { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - for (size_t i = 0; i < 200; ++i) { + for (size_t k = 0; k < 200; ++k) { internal_iter->AddDeletion("c"); } internal_iter->AddPut("c", "200"); internal_iter->Finish(); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, i)); + std::unique_ptr db_iter(NewDBIterator( + env_, ro, cf_options, BytewiseComparator(), internal_iter, i, + options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(!db_iter->Valid()); @@ -464,7 +783,8 @@ TEST(DBIteratorTest, DBIteratorUseSkip) { internal_iter->AddPut("c", "200"); internal_iter->Finish(); std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 200)); + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 200, options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "c"); @@ -487,22 +807,23 @@ TEST(DBIteratorTest, DBIteratorUseSkip) { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); internal_iter->AddMerge("b", "merge_1"); internal_iter->AddMerge("a", "merge_2"); - for (size_t i = 0; i < 200; ++i) { - internal_iter->AddPut("d", std::to_string(i)); + for (size_t k = 0; k < 200; ++k) { + internal_iter->AddPut("d", ToString(k)); } - for (size_t i = 0; i < 200; ++i) { - internal_iter->AddPut("c", std::to_string(i)); + for (size_t k = 0; k < 200; ++k) { + internal_iter->AddPut("c", ToString(k)); } internal_iter->Finish(); std::unique_ptr db_iter(NewDBIterator( - env_, options, BytewiseComparator(), internal_iter, i + 2)); + env_, ro, cf_options, BytewiseComparator(), internal_iter, i + 2, + options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "d"); - ASSERT_EQ(db_iter->value().ToString(), std::to_string(i)); + ASSERT_EQ(db_iter->value().ToString(), ToString(i)); db_iter->Prev(); ASSERT_TRUE(db_iter->Valid()); @@ -524,20 +845,21 @@ TEST(DBIteratorTest, DBIteratorUseSkip) { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); internal_iter->AddMerge("b", "b"); internal_iter->AddMerge("a", "a"); - for (size_t i = 0; i < 200; ++i) { - internal_iter->AddMerge("c", std::to_string(i)); + for (size_t k = 0; k < 200; ++k) { + internal_iter->AddMerge("c", ToString(k)); } internal_iter->Finish(); std::unique_ptr db_iter(NewDBIterator( - env_, options, BytewiseComparator(), internal_iter, i + 2)); + env_, ro, cf_options, BytewiseComparator(), internal_iter, i + 2, + options.max_sequential_skip_in_iterations)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "c"); std::string merge_result = "0"; for (size_t j = 1; j <= i; ++j) { - merge_result += "," + std::to_string(j); + merge_result += "," + ToString(j); } ASSERT_EQ(db_iter->value().ToString(), merge_result); @@ -557,778 +879,1926 @@ TEST(DBIteratorTest, DBIteratorUseSkip) { } } -TEST(DBIteratorTest, DBIterator) { +TEST_F(DBIteratorTest, DBIteratorSkipInternalKeys) { Options options; - options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); + ImmutableCFOptions cf_options = ImmutableCFOptions(options); + ReadOptions ro; + + // Basic test case ... Make sure explicityly passing the default value works. + // Skipping internal keys is disabled by default, when the value is 0. { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddPut("a", "0"); - internal_iter->AddPut("b", "0"); + internal_iter->AddPut("a", "val_a"); internal_iter->AddDeletion("b"); - internal_iter->AddMerge("a", "1"); - internal_iter->AddMerge("b", "2"); + internal_iter->AddDeletion("b"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddDeletion("c"); + internal_iter->AddPut("d", "val_d"); internal_iter->Finish(); + ro.max_skippable_internal_keys = 0; std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 1)); + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); + db_iter->SeekToFirst(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "0"); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); + db_iter->Next(); ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "b"); - } + ASSERT_EQ(db_iter->key().ToString(), "d"); + ASSERT_EQ(db_iter->value().ToString(), "val_d"); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddPut("a", "0"); - internal_iter->AddPut("b", "0"); - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("a", "1"); - internal_iter->AddMerge("b", "2"); - internal_iter->Finish(); + db_iter->Next(); + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().ok()); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 0)); - db_iter->SeekToFirst(); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "d"); + ASSERT_EQ(db_iter->value().ToString(), "val_d"); + + db_iter->Prev(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "0"); - db_iter->Next(); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); + + db_iter->Prev(); ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().ok()); } + // Test to make sure that the request will *not* fail as incomplete if + // num_internal_keys_skipped is *equal* to max_skippable_internal_keys + // threshold. (It will fail as incomplete only when the threshold is + // exceeded.) { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddPut("a", "0"); - internal_iter->AddPut("b", "0"); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddDeletion("b"); internal_iter->AddDeletion("b"); - internal_iter->AddMerge("a", "1"); - internal_iter->AddMerge("b", "2"); + internal_iter->AddPut("c", "val_c"); internal_iter->Finish(); + ro.max_skippable_internal_keys = 2; std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 2)); + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); + db_iter->SeekToFirst(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "0"); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); + + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "val_c"); + db_iter->Next(); ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().ok()); + + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "val_c"); + + db_iter->Prev(); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); + + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().ok()); } + // Fail the request as incomplete when num_internal_keys_skipped > + // max_skippable_internal_keys { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddPut("a", "0"); - internal_iter->AddPut("b", "0"); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddDeletion("b"); + internal_iter->AddDeletion("b"); internal_iter->AddDeletion("b"); - internal_iter->AddMerge("a", "1"); - internal_iter->AddMerge("b", "2"); + internal_iter->AddPut("c", "val_c"); internal_iter->Finish(); + ro.max_skippable_internal_keys = 2; std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 4)); + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); + db_iter->SeekToFirst(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "0,1"); - db_iter->Next(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "b"); - ASSERT_EQ(db_iter->value().ToString(), "2"); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); + db_iter->Next(); ASSERT_TRUE(!db_iter->Valid()); - } + ASSERT_TRUE(db_iter->status().IsIncomplete()); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "val_c"); + + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); + } + + // Test that the num_internal_keys_skipped counter resets after a successful + // read. { - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddPut("a", "put_1"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddDeletion("b"); + internal_iter->AddDeletion("b"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddDeletion("d"); + internal_iter->AddDeletion("d"); + internal_iter->AddDeletion("d"); + internal_iter->AddPut("e", "val_e"); + internal_iter->Finish(); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 0)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + ro.max_skippable_internal_keys = 2; + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddPut("a", "put_1"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + db_iter->SeekToFirst(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 1)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1,merge_2"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "val_c"); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddPut("a", "put_1"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + db_iter->Next(); // num_internal_keys_skipped counter resets here. + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); + } - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 2)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1,merge_2,merge_3"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + // Test that the num_internal_keys_skipped counter resets after a successful + // read. + // Reverse direction + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddDeletion("b"); + internal_iter->AddDeletion("b"); + internal_iter->AddDeletion("b"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddDeletion("d"); + internal_iter->AddDeletion("d"); + internal_iter->AddPut("e", "val_e"); + internal_iter->Finish(); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddPut("a", "put_1"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + ro.max_skippable_internal_keys = 2; + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 3)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "put_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "e"); + ASSERT_EQ(db_iter->value().ToString(), "val_e"); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddPut("a", "put_1"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "val_c"); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 4)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "put_1,merge_4"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + db_iter->Prev(); // num_internal_keys_skipped counter resets here. + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); + } - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddPut("a", "put_1"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + // Test that skipping separate keys is handled + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddDeletion("b"); + internal_iter->AddDeletion("c"); + internal_iter->AddDeletion("d"); + internal_iter->AddPut("e", "val_e"); + internal_iter->Finish(); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 5)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "put_1,merge_4,merge_5"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + ro.max_skippable_internal_keys = 2; + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddPut("a", "put_1"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + db_iter->SeekToFirst(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 6)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "put_1,merge_4,merge_5,merge_6"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + db_iter->Next(); + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); + + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "e"); + ASSERT_EQ(db_iter->value().ToString(), "val_e"); + + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); } + // Test if alternating puts and deletes of the same key are handled correctly. { - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddDeletion("a"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddPut("b", "val_b"); + internal_iter->AddDeletion("b"); + internal_iter->AddPut("c", "val_c"); + internal_iter->AddDeletion("c"); + internal_iter->AddPut("d", "val_d"); + internal_iter->AddDeletion("d"); + internal_iter->AddPut("e", "val_e"); + internal_iter->Finish(); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 0)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + ro.max_skippable_internal_keys = 2; + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddDeletion("a"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + db_iter->SeekToFirst(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 1)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1,merge_2"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + db_iter->Next(); + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddDeletion("a"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "e"); + ASSERT_EQ(db_iter->value().ToString(), "val_e"); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 2)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1,merge_2,merge_3"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); + } - { + // Test for large number of skippable internal keys with *default* + // max_sequential_skip_in_iterations. + { + for (size_t i = 1; i <= 200; ++i) { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddDeletion("a"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); + internal_iter->AddPut("a", "val_a"); + for (size_t j = 1; j <= i; ++j) { + internal_iter->AddPut("b", "val_b"); + internal_iter->AddDeletion("b"); + } + internal_iter->AddPut("c", "val_c"); internal_iter->Finish(); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 3)); - db_iter->SeekToLast(); - ASSERT_TRUE(!db_iter->Valid()); - } - - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddDeletion("a"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + ro.max_skippable_internal_keys = i; + std::unique_ptr db_iter(NewDBIterator( + env_, ro, cf_options, BytewiseComparator(), internal_iter, 2 * i + 1, + options.max_sequential_skip_in_iterations)); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 4)); - db_iter->SeekToLast(); + db_iter->SeekToFirst(); ASSERT_TRUE(db_iter->Valid()); ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_4"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } - - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddDeletion("a"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); + + db_iter->Next(); + if ((options.max_sequential_skip_in_iterations + 1) >= + ro.max_skippable_internal_keys) { + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); + } else { + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "val_c"); + } - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 5)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } - - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddMerge("a", "merge_2"); - internal_iter->AddMerge("a", "merge_3"); - internal_iter->AddDeletion("a"); - internal_iter->AddMerge("a", "merge_4"); - internal_iter->AddMerge("a", "merge_5"); - internal_iter->AddMerge("a", "merge_6"); - internal_iter->Finish(); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "val_c"); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 6)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5,merge_6"); db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); + if ((options.max_sequential_skip_in_iterations + 1) >= + ro.max_skippable_internal_keys) { + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); + } else { + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); + } } } + // Test for large number of skippable internal keys with a *non-default* + // max_sequential_skip_in_iterations. { - { + for (size_t i = 1; i <= 200; ++i) { TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddPut("b", "val"); - internal_iter->AddMerge("b", "merge_2"); - - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_3"); + internal_iter->AddPut("a", "val_a"); + for (size_t j = 1; j <= i; ++j) { + internal_iter->AddPut("b", "val_b"); + internal_iter->AddDeletion("b"); + } + internal_iter->AddPut("c", "val_c"); + internal_iter->Finish(); - internal_iter->AddMerge("c", "merge_4"); - internal_iter->AddMerge("c", "merge_5"); + options.max_sequential_skip_in_iterations = 1000; + ro.max_skippable_internal_keys = i; + std::unique_ptr db_iter(NewDBIterator( + env_, ro, cf_options, BytewiseComparator(), internal_iter, 2 * i + 1, + options.max_sequential_skip_in_iterations)); - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_6"); - internal_iter->AddMerge("b", "merge_7"); - internal_iter->AddMerge("b", "merge_8"); - internal_iter->AddMerge("b", "merge_9"); - internal_iter->AddMerge("b", "merge_10"); - internal_iter->AddMerge("b", "merge_11"); + db_iter->SeekToFirst(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "val_a"); - internal_iter->AddDeletion("c"); - internal_iter->Finish(); + db_iter->Next(); + ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 0)); db_iter->SeekToLast(); ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "val_c"); + db_iter->Prev(); ASSERT_TRUE(!db_iter->Valid()); + ASSERT_TRUE(db_iter->status().IsIncomplete()); } + } +} - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddPut("b", "val"); - internal_iter->AddMerge("b", "merge_2"); +TEST_F(DBIteratorTest, DBIterator1) { + ReadOptions ro; + Options options; + options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_3"); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "0"); + internal_iter->AddPut("b", "0"); + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("a", "1"); + internal_iter->AddMerge("b", "2"); + internal_iter->Finish(); - internal_iter->AddMerge("c", "merge_4"); - internal_iter->AddMerge("c", "merge_5"); + std::unique_ptr db_iter(NewDBIterator( + env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 1, options.max_sequential_skip_in_iterations)); + db_iter->SeekToFirst(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "0"); + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + db_iter->Next(); + ASSERT_FALSE(db_iter->Valid()); +} - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_6"); - internal_iter->AddMerge("b", "merge_7"); - internal_iter->AddMerge("b", "merge_8"); - internal_iter->AddMerge("b", "merge_9"); - internal_iter->AddMerge("b", "merge_10"); - internal_iter->AddMerge("b", "merge_11"); +TEST_F(DBIteratorTest, DBIterator2) { + ReadOptions ro; + Options options; + options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); - internal_iter->AddDeletion("c"); - internal_iter->Finish(); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "0"); + internal_iter->AddPut("b", "0"); + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("a", "1"); + internal_iter->AddMerge("b", "2"); + internal_iter->Finish(); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 2)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); + std::unique_ptr db_iter(NewDBIterator( + env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 0, options.max_sequential_skip_in_iterations)); + db_iter->SeekToFirst(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "0"); + db_iter->Next(); + ASSERT_TRUE(!db_iter->Valid()); +} - ASSERT_EQ(db_iter->key().ToString(), "b"); - ASSERT_EQ(db_iter->value().ToString(), "val,merge_2"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); +TEST_F(DBIteratorTest, DBIterator3) { + ReadOptions ro; + Options options; + options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "0"); + internal_iter->AddPut("b", "0"); + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("a", "1"); + internal_iter->AddMerge("b", "2"); + internal_iter->Finish(); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddPut("b", "val"); - internal_iter->AddMerge("b", "merge_2"); + std::unique_ptr db_iter(NewDBIterator( + env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 2, options.max_sequential_skip_in_iterations)); + db_iter->SeekToFirst(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "0"); + db_iter->Next(); + ASSERT_TRUE(!db_iter->Valid()); +} - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_3"); +TEST_F(DBIteratorTest, DBIterator4) { + ReadOptions ro; + Options options; + options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); - internal_iter->AddMerge("c", "merge_4"); - internal_iter->AddMerge("c", "merge_5"); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "0"); + internal_iter->AddPut("b", "0"); + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("a", "1"); + internal_iter->AddMerge("b", "2"); + internal_iter->Finish(); - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_6"); - internal_iter->AddMerge("b", "merge_7"); - internal_iter->AddMerge("b", "merge_8"); - internal_iter->AddMerge("b", "merge_9"); - internal_iter->AddMerge("b", "merge_10"); - internal_iter->AddMerge("b", "merge_11"); + std::unique_ptr db_iter(NewDBIterator( + env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 4, options.max_sequential_skip_in_iterations)); + db_iter->SeekToFirst(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "0,1"); + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "2"); + db_iter->Next(); + ASSERT_TRUE(!db_iter->Valid()); +} - internal_iter->AddDeletion("c"); - internal_iter->Finish(); +TEST_F(DBIteratorTest, DBIterator5) { + ReadOptions ro; + Options options; + options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); + ImmutableCFOptions cf_options = ImmutableCFOptions(options); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 4)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddPut("a", "put_1"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); - ASSERT_EQ(db_iter->key().ToString(), "b"); - ASSERT_EQ(db_iter->value().ToString(), "merge_3"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 0, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddPut("a", "put_1"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 1, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1,merge_2"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddPut("a", "put_1"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 2, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1,merge_2,merge_3"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddPut("a", "put_1"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 3, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "put_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddPut("a", "put_1"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 4, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "put_1,merge_4"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddPut("a", "put_1"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 5, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "put_1,merge_4,merge_5"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddPut("a", "put_1"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 6, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "put_1,merge_4,merge_5,merge_6"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + // put, singledelete, merge + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "val_a"); + internal_iter->AddSingleDeletion("a"); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddPut("b", "val_b"); + internal_iter->Finish(); + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 10, options.max_sequential_skip_in_iterations)); + db_iter->Seek("b"); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + } +} + +TEST_F(DBIteratorTest, DBIterator6) { + ReadOptions ro; + Options options; + options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); + ImmutableCFOptions cf_options = ImmutableCFOptions(options); + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddDeletion("a"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 0, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddDeletion("a"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 1, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1,merge_2"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddDeletion("a"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 2, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1,merge_2,merge_3"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddDeletion("a"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 3, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddDeletion("a"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 4, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_4"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddDeletion("a"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 5, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("a", "merge_3"); + internal_iter->AddDeletion("a"); + internal_iter->AddMerge("a", "merge_4"); + internal_iter->AddMerge("a", "merge_5"); + internal_iter->AddMerge("a", "merge_6"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 6, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5,merge_6"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } +} + +TEST_F(DBIteratorTest, DBIterator7) { + ReadOptions ro; + Options options; + options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); + ImmutableCFOptions cf_options = ImmutableCFOptions(options); + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddPut("b", "val"); + internal_iter->AddMerge("b", "merge_2"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_3"); + + internal_iter->AddMerge("c", "merge_4"); + internal_iter->AddMerge("c", "merge_5"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_6"); + internal_iter->AddMerge("b", "merge_7"); + internal_iter->AddMerge("b", "merge_8"); + internal_iter->AddMerge("b", "merge_9"); + internal_iter->AddMerge("b", "merge_10"); + internal_iter->AddMerge("b", "merge_11"); + + internal_iter->AddDeletion("c"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 0, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddPut("b", "val"); + internal_iter->AddMerge("b", "merge_2"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_3"); + + internal_iter->AddMerge("c", "merge_4"); + internal_iter->AddMerge("c", "merge_5"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_6"); + internal_iter->AddMerge("b", "merge_7"); + internal_iter->AddMerge("b", "merge_8"); + internal_iter->AddMerge("b", "merge_9"); + internal_iter->AddMerge("b", "merge_10"); + internal_iter->AddMerge("b", "merge_11"); + + internal_iter->AddDeletion("c"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 2, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "val,merge_2"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddPut("b", "val"); + internal_iter->AddMerge("b", "merge_2"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_3"); + + internal_iter->AddMerge("c", "merge_4"); + internal_iter->AddMerge("c", "merge_5"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_6"); + internal_iter->AddMerge("b", "merge_7"); + internal_iter->AddMerge("b", "merge_8"); + internal_iter->AddMerge("b", "merge_9"); + internal_iter->AddMerge("b", "merge_10"); + internal_iter->AddMerge("b", "merge_11"); + + internal_iter->AddDeletion("c"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 4, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "merge_3"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddPut("b", "val"); + internal_iter->AddMerge("b", "merge_2"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_3"); + + internal_iter->AddMerge("c", "merge_4"); + internal_iter->AddMerge("c", "merge_5"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_6"); + internal_iter->AddMerge("b", "merge_7"); + internal_iter->AddMerge("b", "merge_8"); + internal_iter->AddMerge("b", "merge_9"); + internal_iter->AddMerge("b", "merge_10"); + internal_iter->AddMerge("b", "merge_11"); + + internal_iter->AddDeletion("c"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 5, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "merge_4"); + db_iter->Prev(); + + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "merge_3"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddPut("b", "val"); + internal_iter->AddMerge("b", "merge_2"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_3"); + + internal_iter->AddMerge("c", "merge_4"); + internal_iter->AddMerge("c", "merge_5"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_6"); + internal_iter->AddMerge("b", "merge_7"); + internal_iter->AddMerge("b", "merge_8"); + internal_iter->AddMerge("b", "merge_9"); + internal_iter->AddMerge("b", "merge_10"); + internal_iter->AddMerge("b", "merge_11"); + + internal_iter->AddDeletion("c"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 6, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "merge_3"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddPut("b", "val"); + internal_iter->AddMerge("b", "merge_2"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_3"); + + internal_iter->AddMerge("c", "merge_4"); + internal_iter->AddMerge("c", "merge_5"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_6"); + internal_iter->AddMerge("b", "merge_7"); + internal_iter->AddMerge("b", "merge_8"); + internal_iter->AddMerge("b", "merge_9"); + internal_iter->AddMerge("b", "merge_10"); + internal_iter->AddMerge("b", "merge_11"); + + internal_iter->AddDeletion("c"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 7, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddPut("b", "val"); + internal_iter->AddMerge("b", "merge_2"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_3"); + + internal_iter->AddMerge("c", "merge_4"); + internal_iter->AddMerge("c", "merge_5"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_6"); + internal_iter->AddMerge("b", "merge_7"); + internal_iter->AddMerge("b", "merge_8"); + internal_iter->AddMerge("b", "merge_9"); + internal_iter->AddMerge("b", "merge_10"); + internal_iter->AddMerge("b", "merge_11"); + + internal_iter->AddDeletion("c"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 9, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "merge_6,merge_7"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } + + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddPut("b", "val"); + internal_iter->AddMerge("b", "merge_2"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_3"); + + internal_iter->AddMerge("c", "merge_4"); + internal_iter->AddMerge("c", "merge_5"); + + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_6"); + internal_iter->AddMerge("b", "merge_7"); + internal_iter->AddMerge("b", "merge_8"); + internal_iter->AddMerge("b", "merge_9"); + internal_iter->AddMerge("b", "merge_10"); + internal_iter->AddMerge("b", "merge_11"); + + internal_iter->AddDeletion("c"); + internal_iter->Finish(); + + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 13, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddPut("b", "val"); - internal_iter->AddMerge("b", "merge_2"); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_3"); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), + "merge_6,merge_7,merge_8,merge_9,merge_10,merge_11"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); - internal_iter->AddMerge("c", "merge_4"); - internal_iter->AddMerge("c", "merge_5"); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_6"); - internal_iter->AddMerge("b", "merge_7"); - internal_iter->AddMerge("b", "merge_8"); - internal_iter->AddMerge("b", "merge_9"); - internal_iter->AddMerge("b", "merge_10"); - internal_iter->AddMerge("b", "merge_11"); + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddPut("b", "val"); + internal_iter->AddMerge("b", "merge_2"); - internal_iter->AddDeletion("c"); - internal_iter->Finish(); + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_3"); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 5)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); + internal_iter->AddMerge("c", "merge_4"); + internal_iter->AddMerge("c", "merge_5"); - ASSERT_EQ(db_iter->key().ToString(), "c"); - ASSERT_EQ(db_iter->value().ToString(), "merge_4"); - db_iter->Prev(); + internal_iter->AddDeletion("b"); + internal_iter->AddMerge("b", "merge_6"); + internal_iter->AddMerge("b", "merge_7"); + internal_iter->AddMerge("b", "merge_8"); + internal_iter->AddMerge("b", "merge_9"); + internal_iter->AddMerge("b", "merge_10"); + internal_iter->AddMerge("b", "merge_11"); + + internal_iter->AddDeletion("c"); + internal_iter->Finish(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "b"); - ASSERT_EQ(db_iter->value().ToString(), "merge_3"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); + std::unique_ptr db_iter( + NewDBIterator(env_, ro, cf_options, BytewiseComparator(), internal_iter, + 14, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), + "merge_6,merge_7,merge_8,merge_9,merge_10,merge_11"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddPut("b", "val"); - internal_iter->AddMerge("b", "merge_2"); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1"); + db_iter->Prev(); + ASSERT_TRUE(!db_iter->Valid()); + } +} - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_3"); +TEST_F(DBIteratorTest, DBIterator8) { + ReadOptions ro; + Options options; + options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); - internal_iter->AddMerge("c", "merge_4"); - internal_iter->AddMerge("c", "merge_5"); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddDeletion("a"); + internal_iter->AddPut("a", "0"); + internal_iter->AddPut("b", "0"); + internal_iter->Finish(); - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_6"); - internal_iter->AddMerge("b", "merge_7"); - internal_iter->AddMerge("b", "merge_8"); - internal_iter->AddMerge("b", "merge_9"); - internal_iter->AddMerge("b", "merge_10"); - internal_iter->AddMerge("b", "merge_11"); + std::unique_ptr db_iter(NewDBIterator( + env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 10, options.max_sequential_skip_in_iterations)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "0"); - internal_iter->AddDeletion("c"); - internal_iter->Finish(); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "0"); +} - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 6)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); +// TODO(3.13): fix the issue of Seek() then Prev() which might not necessary +// return the biggest element smaller than the seek key. +TEST_F(DBIteratorTest, DBIterator9) { + ReadOptions ro; + Options options; + options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); + { + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddMerge("a", "merge_1"); + internal_iter->AddMerge("a", "merge_2"); + internal_iter->AddMerge("b", "merge_3"); + internal_iter->AddMerge("b", "merge_4"); + internal_iter->AddMerge("d", "merge_5"); + internal_iter->AddMerge("d", "merge_6"); + internal_iter->Finish(); - ASSERT_EQ(db_iter->key().ToString(), "c"); - ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); + std::unique_ptr db_iter(NewDBIterator( + env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 10, options.max_sequential_skip_in_iterations)); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "b"); - ASSERT_EQ(db_iter->value().ToString(), "merge_3"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "merge_3,merge_4"); + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "d"); + ASSERT_EQ(db_iter->value().ToString(), "merge_5,merge_6"); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + db_iter->Seek("b"); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "merge_3,merge_4"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "merge_1,merge_2"); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddPut("b", "val"); - internal_iter->AddMerge("b", "merge_2"); + db_iter->SeekForPrev("b"); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "merge_3,merge_4"); + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "d"); + ASSERT_EQ(db_iter->value().ToString(), "merge_5,merge_6"); - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_3"); + db_iter->Seek("c"); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "d"); + ASSERT_EQ(db_iter->value().ToString(), "merge_5,merge_6"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "merge_3,merge_4"); - internal_iter->AddMerge("c", "merge_4"); - internal_iter->AddMerge("c", "merge_5"); + db_iter->SeekForPrev("c"); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "merge_3,merge_4"); + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "d"); + ASSERT_EQ(db_iter->value().ToString(), "merge_5,merge_6"); + } +} - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_6"); - internal_iter->AddMerge("b", "merge_7"); - internal_iter->AddMerge("b", "merge_8"); - internal_iter->AddMerge("b", "merge_9"); - internal_iter->AddMerge("b", "merge_10"); - internal_iter->AddMerge("b", "merge_11"); +// TODO(3.13): fix the issue of Seek() then Prev() which might not necessary +// return the biggest element smaller than the seek key. +TEST_F(DBIteratorTest, DBIterator10) { + ReadOptions ro; + Options options; - internal_iter->AddDeletion("c"); - internal_iter->Finish(); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "1"); + internal_iter->AddPut("b", "2"); + internal_iter->AddPut("c", "3"); + internal_iter->AddPut("d", "4"); + internal_iter->Finish(); - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 7)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); + std::unique_ptr db_iter(NewDBIterator( + env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 10, options.max_sequential_skip_in_iterations)); - ASSERT_EQ(db_iter->key().ToString(), "c"); - ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); + db_iter->Seek("c"); + ASSERT_TRUE(db_iter->Valid()); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "2"); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "3"); - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddPut("b", "val"); - internal_iter->AddMerge("b", "merge_2"); + db_iter->SeekForPrev("c"); + ASSERT_TRUE(db_iter->Valid()); + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "d"); + ASSERT_EQ(db_iter->value().ToString(), "4"); - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_3"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "3"); +} - internal_iter->AddMerge("c", "merge_4"); - internal_iter->AddMerge("c", "merge_5"); +TEST_F(DBIteratorTest, SeekToLastOccurrenceSeq0) { + ReadOptions ro; + Options options; + options.merge_operator = nullptr; - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_6"); - internal_iter->AddMerge("b", "merge_7"); - internal_iter->AddMerge("b", "merge_8"); - internal_iter->AddMerge("b", "merge_9"); - internal_iter->AddMerge("b", "merge_10"); - internal_iter->AddMerge("b", "merge_11"); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "1"); + internal_iter->AddPut("b", "2"); + internal_iter->Finish(); - internal_iter->AddDeletion("c"); - internal_iter->Finish(); + std::unique_ptr db_iter( + NewDBIterator(env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 10, 0 /* force seek */)); + db_iter->SeekToFirst(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "1"); + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "2"); + db_iter->Next(); + ASSERT_FALSE(db_iter->Valid()); +} - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 9)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); +TEST_F(DBIteratorTest, DBIterator11) { + ReadOptions ro; + Options options; + options.merge_operator = MergeOperators::CreateFromStringId("stringappend"); - ASSERT_EQ(db_iter->key().ToString(), "c"); - ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "0"); + internal_iter->AddPut("b", "0"); + internal_iter->AddSingleDeletion("b"); + internal_iter->AddMerge("a", "1"); + internal_iter->AddMerge("b", "2"); + internal_iter->Finish(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "b"); - ASSERT_EQ(db_iter->value().ToString(), "merge_6,merge_7"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); + std::unique_ptr db_iter(NewDBIterator( + env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 1, options.max_sequential_skip_in_iterations)); + db_iter->SeekToFirst(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "0"); + db_iter->Next(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + db_iter->Next(); + ASSERT_FALSE(db_iter->Valid()); +} - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } +TEST_F(DBIteratorTest, DBIterator12) { + ReadOptions ro; + Options options; + options.merge_operator = nullptr; - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddPut("b", "val"); - internal_iter->AddMerge("b", "merge_2"); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("a", "1"); + internal_iter->AddPut("b", "2"); + internal_iter->AddPut("c", "3"); + internal_iter->AddSingleDeletion("b"); + internal_iter->Finish(); - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_3"); + std::unique_ptr db_iter( + NewDBIterator(env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 10, 0)); + db_iter->SeekToLast(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "c"); + ASSERT_EQ(db_iter->value().ToString(), "3"); + db_iter->Prev(); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "1"); + db_iter->Prev(); + ASSERT_FALSE(db_iter->Valid()); +} - internal_iter->AddMerge("c", "merge_4"); - internal_iter->AddMerge("c", "merge_5"); +TEST_F(DBIteratorTest, DBIterator13) { + ReadOptions ro; + Options options; + options.merge_operator = nullptr; - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_6"); - internal_iter->AddMerge("b", "merge_7"); - internal_iter->AddMerge("b", "merge_8"); - internal_iter->AddMerge("b", "merge_9"); - internal_iter->AddMerge("b", "merge_10"); - internal_iter->AddMerge("b", "merge_11"); + std::string key; + key.resize(9); + key.assign(9, static_cast(0)); + key[0] = 'b'; - internal_iter->AddDeletion("c"); - internal_iter->Finish(); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut(key, "0"); + internal_iter->AddPut(key, "1"); + internal_iter->AddPut(key, "2"); + internal_iter->AddPut(key, "3"); + internal_iter->AddPut(key, "4"); + internal_iter->AddPut(key, "5"); + internal_iter->AddPut(key, "6"); + internal_iter->AddPut(key, "7"); + internal_iter->AddPut(key, "8"); + internal_iter->Finish(); - std::unique_ptr db_iter(NewDBIterator( - env_, options, BytewiseComparator(), internal_iter, 13)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); + std::unique_ptr db_iter( + NewDBIterator(env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 2, 3)); + db_iter->Seek("b"); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), key); + ASSERT_EQ(db_iter->value().ToString(), "2"); +} - ASSERT_EQ(db_iter->key().ToString(), "c"); - ASSERT_EQ(db_iter->value().ToString(), "merge_4,merge_5"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); +TEST_F(DBIteratorTest, DBIterator14) { + ReadOptions ro; + Options options; + options.merge_operator = nullptr; - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "b"); - ASSERT_EQ(db_iter->value().ToString(), - "merge_6,merge_7,merge_8,merge_9,merge_10,merge_11"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); + std::string key("b"); + TestIterator* internal_iter = new TestIterator(BytewiseComparator()); + internal_iter->AddPut("b", "0"); + internal_iter->AddPut("b", "1"); + internal_iter->AddPut("b", "2"); + internal_iter->AddPut("b", "3"); + internal_iter->AddPut("a", "4"); + internal_iter->AddPut("a", "5"); + internal_iter->AddPut("a", "6"); + internal_iter->AddPut("c", "7"); + internal_iter->AddPut("c", "8"); + internal_iter->AddPut("c", "9"); + internal_iter->Finish(); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } + std::unique_ptr db_iter( + NewDBIterator(env_, ro, ImmutableCFOptions(options), BytewiseComparator(), + internal_iter, 4, 1)); + db_iter->Seek("b"); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_EQ(db_iter->key().ToString(), "b"); + ASSERT_EQ(db_iter->value().ToString(), "3"); + db_iter->SeekToFirst(); + ASSERT_EQ(db_iter->key().ToString(), "a"); + ASSERT_EQ(db_iter->value().ToString(), "4"); +} - { - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddMerge("a", "merge_1"); - internal_iter->AddPut("b", "val"); - internal_iter->AddMerge("b", "merge_2"); +class DBIterWithMergeIterTest : public testing::Test { + public: + DBIterWithMergeIterTest() + : env_(Env::Default()), icomp_(BytewiseComparator()) { + options_.merge_operator = nullptr; + + internal_iter1_ = new TestIterator(BytewiseComparator()); + internal_iter1_->Add("a", kTypeValue, "1", 3u); + internal_iter1_->Add("f", kTypeValue, "2", 5u); + internal_iter1_->Add("g", kTypeValue, "3", 7u); + internal_iter1_->Finish(); + + internal_iter2_ = new TestIterator(BytewiseComparator()); + internal_iter2_->Add("a", kTypeValue, "4", 6u); + internal_iter2_->Add("b", kTypeValue, "5", 1u); + internal_iter2_->Add("c", kTypeValue, "6", 2u); + internal_iter2_->Add("d", kTypeValue, "7", 3u); + internal_iter2_->Finish(); + + std::vector child_iters; + child_iters.push_back(internal_iter1_); + child_iters.push_back(internal_iter2_); + InternalKeyComparator icomp(BytewiseComparator()); + InternalIterator* merge_iter = + NewMergingIterator(&icomp_, &child_iters[0], 2u); + + db_iter_.reset(NewDBIterator(env_, ro_, ImmutableCFOptions(options_), + BytewiseComparator(), merge_iter, + 8 /* read data earlier than seqId 8 */, + 3 /* max iterators before reseek */)); + } - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_3"); + Env* env_; + ReadOptions ro_; + Options options_; + TestIterator* internal_iter1_; + TestIterator* internal_iter2_; + InternalKeyComparator icomp_; + Iterator* merge_iter_; + std::unique_ptr db_iter_; +}; - internal_iter->AddMerge("c", "merge_4"); - internal_iter->AddMerge("c", "merge_5"); +TEST_F(DBIterWithMergeIterTest, InnerMergeIterator1) { + db_iter_->SeekToFirst(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "a"); + ASSERT_EQ(db_iter_->value().ToString(), "4"); + db_iter_->Next(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "b"); + ASSERT_EQ(db_iter_->value().ToString(), "5"); + db_iter_->Next(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "c"); + ASSERT_EQ(db_iter_->value().ToString(), "6"); + db_iter_->Next(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "d"); + ASSERT_EQ(db_iter_->value().ToString(), "7"); + db_iter_->Next(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "f"); + ASSERT_EQ(db_iter_->value().ToString(), "2"); + db_iter_->Next(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "g"); + ASSERT_EQ(db_iter_->value().ToString(), "3"); + db_iter_->Next(); + ASSERT_FALSE(db_iter_->Valid()); +} - internal_iter->AddDeletion("b"); - internal_iter->AddMerge("b", "merge_6"); - internal_iter->AddMerge("b", "merge_7"); - internal_iter->AddMerge("b", "merge_8"); - internal_iter->AddMerge("b", "merge_9"); - internal_iter->AddMerge("b", "merge_10"); - internal_iter->AddMerge("b", "merge_11"); +TEST_F(DBIterWithMergeIterTest, InnerMergeIterator2) { + // Test Prev() when one child iterator is at its end. + db_iter_->SeekForPrev("g"); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "g"); + ASSERT_EQ(db_iter_->value().ToString(), "3"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "f"); + ASSERT_EQ(db_iter_->value().ToString(), "2"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "d"); + ASSERT_EQ(db_iter_->value().ToString(), "7"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "c"); + ASSERT_EQ(db_iter_->value().ToString(), "6"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "b"); + ASSERT_EQ(db_iter_->value().ToString(), "5"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "a"); + ASSERT_EQ(db_iter_->value().ToString(), "4"); +} - internal_iter->AddDeletion("c"); - internal_iter->Finish(); +TEST_F(DBIterWithMergeIterTest, InnerMergeIteratorDataRace1) { + // Test Prev() when one child iterator is at its end but more rows + // are added. + db_iter_->Seek("f"); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "f"); + ASSERT_EQ(db_iter_->value().ToString(), "2"); + + // Test call back inserts a key in the end of the mem table after + // MergeIterator::Prev() realized the mem table iterator is at its end + // and before an SeekToLast() is called. + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "MergeIterator::Prev:BeforeSeekToLast", + [&](void* arg) { internal_iter2_->Add("z", kTypeValue, "7", 12u); }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "d"); + ASSERT_EQ(db_iter_->value().ToString(), "7"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "c"); + ASSERT_EQ(db_iter_->value().ToString(), "6"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "b"); + ASSERT_EQ(db_iter_->value().ToString(), "5"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "a"); + ASSERT_EQ(db_iter_->value().ToString(), "4"); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} - std::unique_ptr db_iter(NewDBIterator( - env_, options, BytewiseComparator(), internal_iter, 14)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); +TEST_F(DBIterWithMergeIterTest, InnerMergeIteratorDataRace2) { + // Test Prev() when one child iterator is at its end but more rows + // are added. + db_iter_->Seek("f"); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "f"); + ASSERT_EQ(db_iter_->value().ToString(), "2"); + + // Test call back inserts entries for update a key in the end of the + // mem table after MergeIterator::Prev() realized the mem tableiterator is at + // its end and before an SeekToLast() is called. + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "MergeIterator::Prev:BeforeSeekToLast", [&](void* arg) { + internal_iter2_->Add("z", kTypeValue, "7", 12u); + internal_iter2_->Add("z", kTypeValue, "7", 11u); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "d"); + ASSERT_EQ(db_iter_->value().ToString(), "7"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "c"); + ASSERT_EQ(db_iter_->value().ToString(), "6"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "b"); + ASSERT_EQ(db_iter_->value().ToString(), "5"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "a"); + ASSERT_EQ(db_iter_->value().ToString(), "4"); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} - ASSERT_EQ(db_iter->key().ToString(), "b"); - ASSERT_EQ(db_iter->value().ToString(), - "merge_6,merge_7,merge_8,merge_9,merge_10,merge_11"); - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); +TEST_F(DBIterWithMergeIterTest, InnerMergeIteratorDataRace3) { + // Test Prev() when one child iterator is at its end but more rows + // are added and max_skipped is triggered. + db_iter_->Seek("f"); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "f"); + ASSERT_EQ(db_iter_->value().ToString(), "2"); + + // Test call back inserts entries for update a key in the end of the + // mem table after MergeIterator::Prev() realized the mem table iterator is at + // its end and before an SeekToLast() is called. + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "MergeIterator::Prev:BeforeSeekToLast", [&](void* arg) { + internal_iter2_->Add("z", kTypeValue, "7", 16u, true); + internal_iter2_->Add("z", kTypeValue, "7", 15u, true); + internal_iter2_->Add("z", kTypeValue, "7", 14u, true); + internal_iter2_->Add("z", kTypeValue, "7", 13u, true); + internal_iter2_->Add("z", kTypeValue, "7", 12u, true); + internal_iter2_->Add("z", kTypeValue, "7", 11u, true); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "d"); + ASSERT_EQ(db_iter_->value().ToString(), "7"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "c"); + ASSERT_EQ(db_iter_->value().ToString(), "6"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "b"); + ASSERT_EQ(db_iter_->value().ToString(), "5"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "a"); + ASSERT_EQ(db_iter_->value().ToString(), "4"); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "merge_1"); - db_iter->Prev(); - ASSERT_TRUE(!db_iter->Valid()); - } - } +TEST_F(DBIterWithMergeIterTest, InnerMergeIteratorDataRace4) { + // Test Prev() when one child iterator has more rows inserted + // between Seek() and Prev() when changing directions. + internal_iter2_->Add("z", kTypeValue, "9", 4u); + + db_iter_->Seek("g"); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "g"); + ASSERT_EQ(db_iter_->value().ToString(), "3"); + + // Test call back inserts entries for update a key before "z" in + // mem table after MergeIterator::Prev() calls mem table iterator's + // Seek() and before calling Prev() + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "MergeIterator::Prev:BeforePrev", [&](void* arg) { + IteratorWrapper* it = reinterpret_cast(arg); + if (it->key().starts_with("z")) { + internal_iter2_->Add("x", kTypeValue, "7", 16u, true); + internal_iter2_->Add("x", kTypeValue, "7", 15u, true); + internal_iter2_->Add("x", kTypeValue, "7", 14u, true); + internal_iter2_->Add("x", kTypeValue, "7", 13u, true); + internal_iter2_->Add("x", kTypeValue, "7", 12u, true); + internal_iter2_->Add("x", kTypeValue, "7", 11u, true); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "f"); + ASSERT_EQ(db_iter_->value().ToString(), "2"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "d"); + ASSERT_EQ(db_iter_->value().ToString(), "7"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "c"); + ASSERT_EQ(db_iter_->value().ToString(), "6"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "b"); + ASSERT_EQ(db_iter_->value().ToString(), "5"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "a"); + ASSERT_EQ(db_iter_->value().ToString(), "4"); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} - { - Options options; - TestIterator* internal_iter = new TestIterator(BytewiseComparator()); - internal_iter->AddDeletion("a"); - internal_iter->AddPut("a", "0"); - internal_iter->AddPut("b", "0"); - internal_iter->Finish(); +TEST_F(DBIterWithMergeIterTest, InnerMergeIteratorDataRace5) { + internal_iter2_->Add("z", kTypeValue, "9", 4u); + + // Test Prev() when one child iterator has more rows inserted + // between Seek() and Prev() when changing directions. + db_iter_->Seek("g"); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "g"); + ASSERT_EQ(db_iter_->value().ToString(), "3"); + + // Test call back inserts entries for update a key before "z" in + // mem table after MergeIterator::Prev() calls mem table iterator's + // Seek() and before calling Prev() + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "MergeIterator::Prev:BeforePrev", [&](void* arg) { + IteratorWrapper* it = reinterpret_cast(arg); + if (it->key().starts_with("z")) { + internal_iter2_->Add("x", kTypeValue, "7", 16u, true); + internal_iter2_->Add("x", kTypeValue, "7", 15u, true); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "f"); + ASSERT_EQ(db_iter_->value().ToString(), "2"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "d"); + ASSERT_EQ(db_iter_->value().ToString(), "7"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "c"); + ASSERT_EQ(db_iter_->value().ToString(), "6"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "b"); + ASSERT_EQ(db_iter_->value().ToString(), "5"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "a"); + ASSERT_EQ(db_iter_->value().ToString(), "4"); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} - std::unique_ptr db_iter( - NewDBIterator(env_, options, BytewiseComparator(), internal_iter, 10)); - db_iter->SeekToLast(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "b"); - ASSERT_EQ(db_iter->value().ToString(), "0"); +TEST_F(DBIterWithMergeIterTest, InnerMergeIteratorDataRace6) { + internal_iter2_->Add("z", kTypeValue, "9", 4u); + + // Test Prev() when one child iterator has more rows inserted + // between Seek() and Prev() when changing directions. + db_iter_->Seek("g"); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "g"); + ASSERT_EQ(db_iter_->value().ToString(), "3"); + + // Test call back inserts an entry for update a key before "z" in + // mem table after MergeIterator::Prev() calls mem table iterator's + // Seek() and before calling Prev() + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "MergeIterator::Prev:BeforePrev", [&](void* arg) { + IteratorWrapper* it = reinterpret_cast(arg); + if (it->key().starts_with("z")) { + internal_iter2_->Add("x", kTypeValue, "7", 16u, true); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "f"); + ASSERT_EQ(db_iter_->value().ToString(), "2"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "d"); + ASSERT_EQ(db_iter_->value().ToString(), "7"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "c"); + ASSERT_EQ(db_iter_->value().ToString(), "6"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "b"); + ASSERT_EQ(db_iter_->value().ToString(), "5"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "a"); + ASSERT_EQ(db_iter_->value().ToString(), "4"); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} - db_iter->Prev(); - ASSERT_TRUE(db_iter->Valid()); - ASSERT_EQ(db_iter->key().ToString(), "a"); - ASSERT_EQ(db_iter->value().ToString(), "0"); - } +TEST_F(DBIterWithMergeIterTest, InnerMergeIteratorDataRace7) { + internal_iter1_->Add("u", kTypeValue, "10", 4u); + internal_iter1_->Add("v", kTypeValue, "11", 4u); + internal_iter1_->Add("w", kTypeValue, "12", 4u); + internal_iter2_->Add("z", kTypeValue, "9", 4u); + + // Test Prev() when one child iterator has more rows inserted + // between Seek() and Prev() when changing directions. + db_iter_->Seek("g"); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "g"); + ASSERT_EQ(db_iter_->value().ToString(), "3"); + + // Test call back inserts entries for update a key before "z" in + // mem table after MergeIterator::Prev() calls mem table iterator's + // Seek() and before calling Prev() + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "MergeIterator::Prev:BeforePrev", [&](void* arg) { + IteratorWrapper* it = reinterpret_cast(arg); + if (it->key().starts_with("z")) { + internal_iter2_->Add("x", kTypeValue, "7", 16u, true); + internal_iter2_->Add("x", kTypeValue, "7", 15u, true); + internal_iter2_->Add("x", kTypeValue, "7", 14u, true); + internal_iter2_->Add("x", kTypeValue, "7", 13u, true); + internal_iter2_->Add("x", kTypeValue, "7", 12u, true); + internal_iter2_->Add("x", kTypeValue, "7", 11u, true); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "f"); + ASSERT_EQ(db_iter_->value().ToString(), "2"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "d"); + ASSERT_EQ(db_iter_->value().ToString(), "7"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "c"); + ASSERT_EQ(db_iter_->value().ToString(), "6"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "b"); + ASSERT_EQ(db_iter_->value().ToString(), "5"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "a"); + ASSERT_EQ(db_iter_->value().ToString(), "4"); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); } +TEST_F(DBIterWithMergeIterTest, InnerMergeIteratorDataRace8) { + // internal_iter1_: a, f, g + // internal_iter2_: a, b, c, d, adding (z) + internal_iter2_->Add("z", kTypeValue, "9", 4u); + + // Test Prev() when one child iterator has more rows inserted + // between Seek() and Prev() when changing directions. + db_iter_->Seek("g"); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "g"); + ASSERT_EQ(db_iter_->value().ToString(), "3"); + + // Test call back inserts two keys before "z" in mem table after + // MergeIterator::Prev() calls mem table iterator's Seek() and + // before calling Prev() + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "MergeIterator::Prev:BeforePrev", [&](void* arg) { + IteratorWrapper* it = reinterpret_cast(arg); + if (it->key().starts_with("z")) { + internal_iter2_->Add("x", kTypeValue, "7", 16u, true); + internal_iter2_->Add("y", kTypeValue, "7", 17u, true); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "f"); + ASSERT_EQ(db_iter_->value().ToString(), "2"); + db_iter_->Prev(); + ASSERT_TRUE(db_iter_->Valid()); + ASSERT_EQ(db_iter_->key().ToString(), "d"); + ASSERT_EQ(db_iter_->value().ToString(), "7"); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} } // namespace rocksdb -int main(int argc, char** argv) { return rocksdb::test::RunAllTests(); } +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_iterator_test.cc b/db/db_iterator_test.cc new file mode 100644 index 00000000000..d3bd164a2c0 --- /dev/null +++ b/db/db_iterator_test.cc @@ -0,0 +1,1988 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include + +#include "db/db_test_util.h" +#include "port/port.h" +#include "port/stack_trace.h" +#include "rocksdb/iostats_context.h" +#include "rocksdb/perf_context.h" + +namespace rocksdb { + +class DBIteratorTest : public DBTestBase { + public: + DBIteratorTest() : DBTestBase("/db_iterator_test") {} +}; + +class FlushBlockEveryKeyPolicy : public FlushBlockPolicy { + public: + virtual bool Update(const Slice& key, const Slice& value) override { + if (!start_) { + start_ = true; + return false; + } + return true; + } + private: + bool start_ = false; +}; + +class FlushBlockEveryKeyPolicyFactory : public FlushBlockPolicyFactory { + public: + explicit FlushBlockEveryKeyPolicyFactory() {} + + const char* Name() const override { + return "FlushBlockEveryKeyPolicyFactory"; + } + + FlushBlockPolicy* NewFlushBlockPolicy( + const BlockBasedTableOptions& table_options, + const BlockBuilder& data_block_builder) const override { + return new FlushBlockEveryKeyPolicy; + } +}; + +TEST_F(DBIteratorTest, IteratorProperty) { + // The test needs to be changed if kPersistedTier is supported in iterator. + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); + Put(1, "1", "2"); + ReadOptions ropt; + ropt.pin_data = false; + { + unique_ptr iter(db_->NewIterator(ropt, handles_[1])); + iter->SeekToFirst(); + std::string prop_value; + ASSERT_NOK(iter->GetProperty("non_existing.value", &prop_value)); + ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value)); + ASSERT_EQ("0", prop_value); + iter->Next(); + ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value)); + ASSERT_EQ("Iterator is not valid.", prop_value); + } + Close(); +} + +TEST_F(DBIteratorTest, PersistedTierOnIterator) { + // The test needs to be changed if kPersistedTier is supported in iterator. + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); + ReadOptions ropt; + ropt.read_tier = kPersistedTier; + + auto* iter = db_->NewIterator(ropt, handles_[1]); + ASSERT_TRUE(iter->status().IsNotSupported()); + delete iter; + + std::vector iters; + ASSERT_TRUE(db_->NewIterators(ropt, {handles_[1]}, &iters).IsNotSupported()); + Close(); +} + +TEST_F(DBIteratorTest, NonBlockingIteration) { + do { + ReadOptions non_blocking_opts, regular_opts; + Options options = CurrentOptions(); + options.statistics = rocksdb::CreateDBStatistics(); + non_blocking_opts.read_tier = kBlockCacheTier; + CreateAndReopenWithCF({"pikachu"}, options); + // write one kv to the database. + ASSERT_OK(Put(1, "a", "b")); + + // scan using non-blocking iterator. We should find it because + // it is in memtable. + Iterator* iter = db_->NewIterator(non_blocking_opts, handles_[1]); + int count = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_OK(iter->status()); + count++; + } + ASSERT_EQ(count, 1); + delete iter; + + // flush memtable to storage. Now, the key should not be in the + // memtable neither in the block cache. + ASSERT_OK(Flush(1)); + + // verify that a non-blocking iterator does not find any + // kvs. Neither does it do any IOs to storage. + uint64_t numopen = TestGetTickerCount(options, NO_FILE_OPENS); + uint64_t cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); + iter = db_->NewIterator(non_blocking_opts, handles_[1]); + count = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + count++; + } + ASSERT_EQ(count, 0); + ASSERT_TRUE(iter->status().IsIncomplete()); + ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); + ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); + delete iter; + + // read in the specified block via a regular get + ASSERT_EQ(Get(1, "a"), "b"); + + // verify that we can find it via a non-blocking scan + numopen = TestGetTickerCount(options, NO_FILE_OPENS); + cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); + iter = db_->NewIterator(non_blocking_opts, handles_[1]); + count = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_OK(iter->status()); + count++; + } + ASSERT_EQ(count, 1); + ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); + ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); + delete iter; + + // This test verifies block cache behaviors, which is not used by plain + // table format. + // Exclude kHashCuckoo as it does not support iteration currently + } while (ChangeOptions(kSkipPlainTable | kSkipNoSeekToLast | kSkipHashCuckoo | + kSkipMmapReads)); +} + +#ifndef ROCKSDB_LITE +TEST_F(DBIteratorTest, ManagedNonBlockingIteration) { + do { + ReadOptions non_blocking_opts, regular_opts; + Options options = CurrentOptions(); + options.statistics = rocksdb::CreateDBStatistics(); + non_blocking_opts.read_tier = kBlockCacheTier; + non_blocking_opts.managed = true; + CreateAndReopenWithCF({"pikachu"}, options); + // write one kv to the database. + ASSERT_OK(Put(1, "a", "b")); + + // scan using non-blocking iterator. We should find it because + // it is in memtable. + Iterator* iter = db_->NewIterator(non_blocking_opts, handles_[1]); + int count = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_OK(iter->status()); + count++; + } + ASSERT_EQ(count, 1); + delete iter; + + // flush memtable to storage. Now, the key should not be in the + // memtable neither in the block cache. + ASSERT_OK(Flush(1)); + + // verify that a non-blocking iterator does not find any + // kvs. Neither does it do any IOs to storage. + int64_t numopen = TestGetTickerCount(options, NO_FILE_OPENS); + int64_t cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); + iter = db_->NewIterator(non_blocking_opts, handles_[1]); + count = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + count++; + } + ASSERT_EQ(count, 0); + ASSERT_TRUE(iter->status().IsIncomplete()); + ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); + ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); + delete iter; + + // read in the specified block via a regular get + ASSERT_EQ(Get(1, "a"), "b"); + + // verify that we can find it via a non-blocking scan + numopen = TestGetTickerCount(options, NO_FILE_OPENS); + cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); + iter = db_->NewIterator(non_blocking_opts, handles_[1]); + count = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_OK(iter->status()); + count++; + } + ASSERT_EQ(count, 1); + ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); + ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); + delete iter; + + // This test verifies block cache behaviors, which is not used by plain + // table format. + // Exclude kHashCuckoo as it does not support iteration currently + } while (ChangeOptions(kSkipPlainTable | kSkipNoSeekToLast | kSkipHashCuckoo | + kSkipMmapReads)); +} +#endif // ROCKSDB_LITE + +TEST_F(DBIteratorTest, IterSeekBeforePrev) { + ASSERT_OK(Put("a", "b")); + ASSERT_OK(Put("c", "d")); + dbfull()->Flush(FlushOptions()); + ASSERT_OK(Put("0", "f")); + ASSERT_OK(Put("1", "h")); + dbfull()->Flush(FlushOptions()); + ASSERT_OK(Put("2", "j")); + auto iter = db_->NewIterator(ReadOptions()); + iter->Seek(Slice("c")); + iter->Prev(); + iter->Seek(Slice("a")); + iter->Prev(); + delete iter; +} + +TEST_F(DBIteratorTest, IterSeekForPrevBeforeNext) { + ASSERT_OK(Put("a", "b")); + ASSERT_OK(Put("c", "d")); + dbfull()->Flush(FlushOptions()); + ASSERT_OK(Put("0", "f")); + ASSERT_OK(Put("1", "h")); + dbfull()->Flush(FlushOptions()); + ASSERT_OK(Put("2", "j")); + auto iter = db_->NewIterator(ReadOptions()); + iter->SeekForPrev(Slice("0")); + iter->Next(); + iter->SeekForPrev(Slice("1")); + iter->Next(); + delete iter; +} + +namespace { +std::string MakeLongKey(size_t length, char c) { + return std::string(length, c); +} +} // namespace + +TEST_F(DBIteratorTest, IterLongKeys) { + ASSERT_OK(Put(MakeLongKey(20, 0), "0")); + ASSERT_OK(Put(MakeLongKey(32, 2), "2")); + ASSERT_OK(Put("a", "b")); + dbfull()->Flush(FlushOptions()); + ASSERT_OK(Put(MakeLongKey(50, 1), "1")); + ASSERT_OK(Put(MakeLongKey(127, 3), "3")); + ASSERT_OK(Put(MakeLongKey(64, 4), "4")); + auto iter = db_->NewIterator(ReadOptions()); + + // Create a key that needs to be skipped for Seq too new + iter->Seek(MakeLongKey(20, 0)); + ASSERT_EQ(IterStatus(iter), MakeLongKey(20, 0) + "->0"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), MakeLongKey(50, 1) + "->1"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), MakeLongKey(32, 2) + "->2"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), MakeLongKey(127, 3) + "->3"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), MakeLongKey(64, 4) + "->4"); + + iter->SeekForPrev(MakeLongKey(127, 3)); + ASSERT_EQ(IterStatus(iter), MakeLongKey(127, 3) + "->3"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), MakeLongKey(32, 2) + "->2"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), MakeLongKey(50, 1) + "->1"); + delete iter; + + iter = db_->NewIterator(ReadOptions()); + iter->Seek(MakeLongKey(50, 1)); + ASSERT_EQ(IterStatus(iter), MakeLongKey(50, 1) + "->1"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), MakeLongKey(32, 2) + "->2"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), MakeLongKey(127, 3) + "->3"); + delete iter; +} + +TEST_F(DBIteratorTest, IterNextWithNewerSeq) { + ASSERT_OK(Put("0", "0")); + dbfull()->Flush(FlushOptions()); + ASSERT_OK(Put("a", "b")); + ASSERT_OK(Put("c", "d")); + ASSERT_OK(Put("d", "e")); + auto iter = db_->NewIterator(ReadOptions()); + + // Create a key that needs to be skipped for Seq too new + for (uint64_t i = 0; i < last_options_.max_sequential_skip_in_iterations + 1; + i++) { + ASSERT_OK(Put("b", "f")); + } + + iter->Seek(Slice("a")); + ASSERT_EQ(IterStatus(iter), "a->b"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "c->d"); + iter->SeekForPrev(Slice("b")); + ASSERT_EQ(IterStatus(iter), "a->b"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "c->d"); + + delete iter; +} + +TEST_F(DBIteratorTest, IterPrevWithNewerSeq) { + ASSERT_OK(Put("0", "0")); + dbfull()->Flush(FlushOptions()); + ASSERT_OK(Put("a", "b")); + ASSERT_OK(Put("c", "d")); + ASSERT_OK(Put("d", "e")); + auto iter = db_->NewIterator(ReadOptions()); + + // Create a key that needs to be skipped for Seq too new + for (uint64_t i = 0; i < last_options_.max_sequential_skip_in_iterations + 1; + i++) { + ASSERT_OK(Put("b", "f")); + } + + iter->Seek(Slice("d")); + ASSERT_EQ(IterStatus(iter), "d->e"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "c->d"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "a->b"); + iter->Prev(); + iter->SeekForPrev(Slice("d")); + ASSERT_EQ(IterStatus(iter), "d->e"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "c->d"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "a->b"); + iter->Prev(); + delete iter; +} + +TEST_F(DBIteratorTest, IterPrevWithNewerSeq2) { + ASSERT_OK(Put("0", "0")); + dbfull()->Flush(FlushOptions()); + ASSERT_OK(Put("a", "b")); + ASSERT_OK(Put("c", "d")); + ASSERT_OK(Put("e", "f")); + auto iter = db_->NewIterator(ReadOptions()); + auto iter2 = db_->NewIterator(ReadOptions()); + iter->Seek(Slice("c")); + iter2->SeekForPrev(Slice("d")); + ASSERT_EQ(IterStatus(iter), "c->d"); + ASSERT_EQ(IterStatus(iter2), "c->d"); + + // Create a key that needs to be skipped for Seq too new + for (uint64_t i = 0; i < last_options_.max_sequential_skip_in_iterations + 1; + i++) { + ASSERT_OK(Put("b", "f")); + } + + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "a->b"); + iter->Prev(); + iter2->Prev(); + ASSERT_EQ(IterStatus(iter2), "a->b"); + iter2->Prev(); + delete iter; + delete iter2; +} + +TEST_F(DBIteratorTest, IterEmpty) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); + + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->Seek("foo"); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->SeekForPrev("foo"); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + delete iter; + } while (ChangeCompactOptions()); +} + +TEST_F(DBIteratorTest, IterSingle) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "a", "va")); + Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); + + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->Seek(""); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekForPrev(""); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->Seek("a"); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekForPrev("a"); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->Seek("b"); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekForPrev("b"); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + delete iter; + } while (ChangeCompactOptions()); +} + +TEST_F(DBIteratorTest, IterMulti) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "a", "va")); + ASSERT_OK(Put(1, "b", "vb")); + ASSERT_OK(Put(1, "c", "vc")); + Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); + + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->Seek(""); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Seek("a"); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Seek("ax"); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->SeekForPrev("d"); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->SeekForPrev("c"); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->SeekForPrev("bx"); + ASSERT_EQ(IterStatus(iter), "b->vb"); + + iter->Seek("b"); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Seek("z"); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekForPrev("b"); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->SeekForPrev(""); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + // Switch from reverse to forward + iter->SeekToLast(); + iter->Prev(); + iter->Prev(); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + + // Switch from forward to reverse + iter->SeekToFirst(); + iter->Next(); + iter->Next(); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + + // Make sure iter stays at snapshot + ASSERT_OK(Put(1, "a", "va2")); + ASSERT_OK(Put(1, "a2", "va3")); + ASSERT_OK(Put(1, "b", "vb2")); + ASSERT_OK(Put(1, "c", "vc2")); + ASSERT_OK(Delete(1, "b")); + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "b->vb"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + delete iter; + } while (ChangeCompactOptions()); +} + +// Check that we can skip over a run of user keys +// by using reseek rather than sequential scan +TEST_F(DBIteratorTest, IterReseek) { + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; + Options options = CurrentOptions(options_override); + options.max_sequential_skip_in_iterations = 3; + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // insert three keys with same userkey and verify that + // reseek is not invoked. For each of these test cases, + // verify that we can find the next key "b". + ASSERT_OK(Put(1, "a", "zero")); + ASSERT_OK(Put(1, "a", "one")); + ASSERT_OK(Put(1, "a", "two")); + ASSERT_OK(Put(1, "b", "bone")); + Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); + iter->SeekToFirst(); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 0); + ASSERT_EQ(IterStatus(iter), "a->two"); + iter->Next(); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 0); + ASSERT_EQ(IterStatus(iter), "b->bone"); + delete iter; + + // insert a total of three keys with same userkey and verify + // that reseek is still not invoked. + ASSERT_OK(Put(1, "a", "three")); + iter = db_->NewIterator(ReadOptions(), handles_[1]); + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->three"); + iter->Next(); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 0); + ASSERT_EQ(IterStatus(iter), "b->bone"); + delete iter; + + // insert a total of four keys with same userkey and verify + // that reseek is invoked. + ASSERT_OK(Put(1, "a", "four")); + iter = db_->NewIterator(ReadOptions(), handles_[1]); + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->four"); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 0); + iter->Next(); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 1); + ASSERT_EQ(IterStatus(iter), "b->bone"); + delete iter; + + // Testing reverse iterator + // At this point, we have three versions of "a" and one version of "b". + // The reseek statistics is already at 1. + int num_reseeks = static_cast( + TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION)); + + // Insert another version of b and assert that reseek is not invoked + ASSERT_OK(Put(1, "b", "btwo")); + iter = db_->NewIterator(ReadOptions(), handles_[1]); + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "b->btwo"); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), + num_reseeks); + iter->Prev(); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), + num_reseeks + 1); + ASSERT_EQ(IterStatus(iter), "a->four"); + delete iter; + + // insert two more versions of b. This makes a total of 4 versions + // of b and 4 versions of a. + ASSERT_OK(Put(1, "b", "bthree")); + ASSERT_OK(Put(1, "b", "bfour")); + iter = db_->NewIterator(ReadOptions(), handles_[1]); + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "b->bfour"); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), + num_reseeks + 2); + iter->Prev(); + + // the previous Prev call should have invoked reseek + ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), + num_reseeks + 3); + ASSERT_EQ(IterStatus(iter), "a->four"); + delete iter; +} + +TEST_F(DBIteratorTest, IterSmallAndLargeMix) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "a", "va")); + ASSERT_OK(Put(1, "b", std::string(100000, 'b'))); + ASSERT_OK(Put(1, "c", "vc")); + ASSERT_OK(Put(1, "d", std::string(100000, 'd'))); + ASSERT_OK(Put(1, "e", std::string(100000, 'e'))); + + Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); + + iter->SeekToFirst(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "b->" + std::string(100000, 'b')); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "d->" + std::string(100000, 'd')); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "e->" + std::string(100000, 'e')); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), "e->" + std::string(100000, 'e')); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "d->" + std::string(100000, 'd')); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "c->vc"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "b->" + std::string(100000, 'b')); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "a->va"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "(invalid)"); + + delete iter; + } while (ChangeCompactOptions()); +} + +TEST_F(DBIteratorTest, IterMultiWithDelete) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "ka", "va")); + ASSERT_OK(Put(1, "kb", "vb")); + ASSERT_OK(Put(1, "kc", "vc")); + ASSERT_OK(Delete(1, "kb")); + ASSERT_EQ("NOT_FOUND", Get(1, "kb")); + + Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); + iter->Seek("kc"); + ASSERT_EQ(IterStatus(iter), "kc->vc"); + if (!CurrentOptions().merge_operator) { + // TODO: merge operator does not support backward iteration yet + if (kPlainTableAllBytesPrefix != option_config_ && + kBlockBasedTableWithWholeKeyHashIndex != option_config_ && + kHashLinkList != option_config_ && + kHashSkipList != option_config_) { // doesn't support SeekToLast + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "ka->va"); + } + } + delete iter; + } while (ChangeOptions()); +} + +TEST_F(DBIteratorTest, IterPrevMaxSkip) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + for (int i = 0; i < 2; i++) { + ASSERT_OK(Put(1, "key1", "v1")); + ASSERT_OK(Put(1, "key2", "v2")); + ASSERT_OK(Put(1, "key3", "v3")); + ASSERT_OK(Put(1, "key4", "v4")); + ASSERT_OK(Put(1, "key5", "v5")); + } + + VerifyIterLast("key5->v5", 1); + + ASSERT_OK(Delete(1, "key5")); + VerifyIterLast("key4->v4", 1); + + ASSERT_OK(Delete(1, "key4")); + VerifyIterLast("key3->v3", 1); + + ASSERT_OK(Delete(1, "key3")); + VerifyIterLast("key2->v2", 1); + + ASSERT_OK(Delete(1, "key2")); + VerifyIterLast("key1->v1", 1); + + ASSERT_OK(Delete(1, "key1")); + VerifyIterLast("(invalid)", 1); + } while (ChangeOptions(kSkipMergePut | kSkipNoSeekToLast)); +} + +TEST_F(DBIteratorTest, IterWithSnapshot) { + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions(options_override)); + ASSERT_OK(Put(1, "key1", "val1")); + ASSERT_OK(Put(1, "key2", "val2")); + ASSERT_OK(Put(1, "key3", "val3")); + ASSERT_OK(Put(1, "key4", "val4")); + ASSERT_OK(Put(1, "key5", "val5")); + + const Snapshot* snapshot = db_->GetSnapshot(); + ReadOptions options; + options.snapshot = snapshot; + Iterator* iter = db_->NewIterator(options, handles_[1]); + + ASSERT_OK(Put(1, "key0", "val0")); + // Put more values after the snapshot + ASSERT_OK(Put(1, "key100", "val100")); + ASSERT_OK(Put(1, "key101", "val101")); + + iter->Seek("key5"); + ASSERT_EQ(IterStatus(iter), "key5->val5"); + if (!CurrentOptions().merge_operator) { + // TODO: merge operator does not support backward iteration yet + if (kPlainTableAllBytesPrefix != option_config_ && + kBlockBasedTableWithWholeKeyHashIndex != option_config_ && + kHashLinkList != option_config_ && kHashSkipList != option_config_) { + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "key4->val4"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "key3->val3"); + + iter->Next(); + ASSERT_EQ(IterStatus(iter), "key4->val4"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "key5->val5"); + } + iter->Next(); + ASSERT_TRUE(!iter->Valid()); + } + + if (!CurrentOptions().merge_operator) { + // TODO(gzh): merge operator does not support backward iteration yet + if (kPlainTableAllBytesPrefix != option_config_ && + kBlockBasedTableWithWholeKeyHashIndex != option_config_ && + kHashLinkList != option_config_ && kHashSkipList != option_config_) { + iter->SeekForPrev("key1"); + ASSERT_EQ(IterStatus(iter), "key1->val1"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "key2->val2"); + iter->Next(); + ASSERT_EQ(IterStatus(iter), "key3->val3"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "key2->val2"); + iter->Prev(); + ASSERT_EQ(IterStatus(iter), "key1->val1"); + iter->Prev(); + ASSERT_TRUE(!iter->Valid()); + } + } + db_->ReleaseSnapshot(snapshot); + delete iter; + // skip as HashCuckooRep does not support snapshot + } while (ChangeOptions(kSkipHashCuckoo)); +} + +TEST_F(DBIteratorTest, IteratorPinsRef) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + Put(1, "foo", "hello"); + + // Get iterator that will yield the current contents of the DB. + Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); + + // Write to force compactions + Put(1, "foo", "newvalue1"); + for (int i = 0; i < 100; i++) { + // 100K values + ASSERT_OK(Put(1, Key(i), Key(i) + std::string(100000, 'v'))); + } + Put(1, "foo", "newvalue2"); + + iter->SeekToFirst(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ("foo", iter->key().ToString()); + ASSERT_EQ("hello", iter->value().ToString()); + iter->Next(); + ASSERT_TRUE(!iter->Valid()); + delete iter; + } while (ChangeCompactOptions()); +} + +TEST_F(DBIteratorTest, DBIteratorBoundTest) { + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + + options.prefix_extractor = nullptr; + DestroyAndReopen(options); + ASSERT_OK(Put("a", "0")); + ASSERT_OK(Put("foo", "bar")); + ASSERT_OK(Put("foo1", "bar1")); + ASSERT_OK(Put("g1", "0")); + + // testing basic case with no iterate_upper_bound and no prefix_extractor + { + ReadOptions ro; + ro.iterate_upper_bound = nullptr; + + std::unique_ptr iter(db_->NewIterator(ro)); + + iter->Seek("foo"); + + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("foo")), 0); + + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("foo1")), 0); + + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("g1")), 0); + + iter->SeekForPrev("g1"); + + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("g1")), 0); + + iter->Prev(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("foo1")), 0); + + iter->Prev(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("foo")), 0); + } + + // testing iterate_upper_bound and forward iterator + // to make sure it stops at bound + { + ReadOptions ro; + // iterate_upper_bound points beyond the last expected entry + Slice prefix("foo2"); + ro.iterate_upper_bound = &prefix; + + std::unique_ptr iter(db_->NewIterator(ro)); + + iter->Seek("foo"); + + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("foo")), 0); + + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(("foo1")), 0); + + iter->Next(); + // should stop here... + ASSERT_TRUE(!iter->Valid()); + } + // Testing SeekToLast with iterate_upper_bound set + { + ReadOptions ro; + + Slice prefix("foo"); + ro.iterate_upper_bound = &prefix; + + std::unique_ptr iter(db_->NewIterator(ro)); + + iter->SeekToLast(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("a")), 0); + } + + // prefix is the first letter of the key + options.prefix_extractor.reset(NewFixedPrefixTransform(1)); + + DestroyAndReopen(options); + ASSERT_OK(Put("a", "0")); + ASSERT_OK(Put("foo", "bar")); + ASSERT_OK(Put("foo1", "bar1")); + ASSERT_OK(Put("g1", "0")); + + // testing with iterate_upper_bound and prefix_extractor + // Seek target and iterate_upper_bound are not is same prefix + // This should be an error + { + ReadOptions ro; + Slice upper_bound("g"); + ro.iterate_upper_bound = &upper_bound; + + std::unique_ptr iter(db_->NewIterator(ro)); + + iter->Seek("foo"); + + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ("foo", iter->key().ToString()); + + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ("foo1", iter->key().ToString()); + + iter->Next(); + ASSERT_TRUE(!iter->Valid()); + } + + // testing that iterate_upper_bound prevents iterating over deleted items + // if the bound has already reached + { + options.prefix_extractor = nullptr; + DestroyAndReopen(options); + ASSERT_OK(Put("a", "0")); + ASSERT_OK(Put("b", "0")); + ASSERT_OK(Put("b1", "0")); + ASSERT_OK(Put("c", "0")); + ASSERT_OK(Put("d", "0")); + ASSERT_OK(Put("e", "0")); + ASSERT_OK(Delete("c")); + ASSERT_OK(Delete("d")); + + // base case with no bound + ReadOptions ro; + ro.iterate_upper_bound = nullptr; + + std::unique_ptr iter(db_->NewIterator(ro)); + + iter->Seek("b"); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("b")), 0); + + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(("b1")), 0); + + get_perf_context()->Reset(); + iter->Next(); + + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(static_cast(get_perf_context()->internal_delete_skipped_count), 2); + + // now testing with iterate_bound + Slice prefix("c"); + ro.iterate_upper_bound = &prefix; + + iter.reset(db_->NewIterator(ro)); + + get_perf_context()->Reset(); + + iter->Seek("b"); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("b")), 0); + + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(("b1")), 0); + + iter->Next(); + // the iteration should stop as soon as the bound key is reached + // even though the key is deleted + // hence internal_delete_skipped_count should be 0 + ASSERT_TRUE(!iter->Valid()); + ASSERT_EQ(static_cast(get_perf_context()->internal_delete_skipped_count), 0); + } +} + +TEST_F(DBIteratorTest, DBIteratorBoundOptimizationTest) { + int upper_bound_hits = 0; + Options options = CurrentOptions(); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "BlockBasedTable::BlockEntryIteratorState::KeyReachedUpperBound", + [&upper_bound_hits](void* arg) { + assert(arg != nullptr); + upper_bound_hits += (*static_cast(arg) ? 1 : 0); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + options.env = env_; + options.create_if_missing = true; + options.prefix_extractor = nullptr; + BlockBasedTableOptions table_options; + table_options.flush_block_policy_factory = + std::make_shared(); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + DestroyAndReopen(options); + ASSERT_OK(Put("foo1", "bar1")); + ASSERT_OK(Put("foo2", "bar2")); + ASSERT_OK(Put("foo4", "bar4")); + ASSERT_OK(Flush()); + + Slice ub("foo3"); + ReadOptions ro; + ro.iterate_upper_bound = &ub; + + std::unique_ptr iter(db_->NewIterator(ro)); + + iter->Seek("foo"); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("foo1")), 0); + ASSERT_EQ(upper_bound_hits, 0); + + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("foo2")), 0); + ASSERT_EQ(upper_bound_hits, 0); + + iter->Next(); + ASSERT_FALSE(iter->Valid()); + ASSERT_EQ(upper_bound_hits, 1); +} +// TODO(3.13): fix the issue of Seek() + Prev() which might not necessary +// return the biggest key which is smaller than the seek key. +TEST_F(DBIteratorTest, PrevAfterAndNextAfterMerge) { + Options options; + options.create_if_missing = true; + options.merge_operator = MergeOperators::CreatePutOperator(); + options.env = env_; + DestroyAndReopen(options); + + // write three entries with different keys using Merge() + WriteOptions wopts; + db_->Merge(wopts, "1", "data1"); + db_->Merge(wopts, "2", "data2"); + db_->Merge(wopts, "3", "data3"); + + std::unique_ptr it(db_->NewIterator(ReadOptions())); + + it->Seek("2"); + ASSERT_TRUE(it->Valid()); + ASSERT_EQ("2", it->key().ToString()); + + it->Prev(); + ASSERT_TRUE(it->Valid()); + ASSERT_EQ("1", it->key().ToString()); + + it->SeekForPrev("1"); + ASSERT_TRUE(it->Valid()); + ASSERT_EQ("1", it->key().ToString()); + + it->Next(); + ASSERT_TRUE(it->Valid()); + ASSERT_EQ("2", it->key().ToString()); +} + +TEST_F(DBIteratorTest, PinnedDataIteratorRandomized) { + enum TestConfig { + NORMAL, + CLOSE_AND_OPEN, + COMPACT_BEFORE_READ, + FLUSH_EVERY_1000, + MAX + }; + + // Generate Random data + Random rnd(301); + + int puts = 100000; + int key_pool = static_cast(puts * 0.7); + int key_size = 100; + int val_size = 1000; + int seeks_percentage = 20; // 20% of keys will be used to test seek() + int delete_percentage = 20; // 20% of keys will be deleted + int merge_percentage = 20; // 20% of keys will be added using Merge() + + for (int run_config = 0; run_config < TestConfig::MAX; run_config++) { + Options options = CurrentOptions(); + BlockBasedTableOptions table_options; + table_options.use_delta_encoding = false; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.merge_operator = MergeOperators::CreatePutOperator(); + DestroyAndReopen(options); + + std::vector generated_keys(key_pool); + for (int i = 0; i < key_pool; i++) { + generated_keys[i] = RandomString(&rnd, key_size); + } + + std::map true_data; + std::vector random_keys; + std::vector deleted_keys; + for (int i = 0; i < puts; i++) { + auto& k = generated_keys[rnd.Next() % key_pool]; + auto v = RandomString(&rnd, val_size); + + // Insert data to true_data map and to DB + true_data[k] = v; + if (rnd.OneIn(static_cast(100.0 / merge_percentage))) { + ASSERT_OK(db_->Merge(WriteOptions(), k, v)); + } else { + ASSERT_OK(Put(k, v)); + } + + // Pick random keys to be used to test Seek() + if (rnd.OneIn(static_cast(100.0 / seeks_percentage))) { + random_keys.push_back(k); + } + + // Delete some random keys + if (rnd.OneIn(static_cast(100.0 / delete_percentage))) { + deleted_keys.push_back(k); + true_data.erase(k); + ASSERT_OK(Delete(k)); + } + + if (run_config == TestConfig::FLUSH_EVERY_1000) { + if (i && i % 1000 == 0) { + Flush(); + } + } + } + + if (run_config == TestConfig::CLOSE_AND_OPEN) { + Close(); + Reopen(options); + } else if (run_config == TestConfig::COMPACT_BEFORE_READ) { + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + } + + ReadOptions ro; + ro.pin_data = true; + auto iter = db_->NewIterator(ro); + + { + // Test Seek to random keys + std::vector keys_slices; + std::vector true_keys; + for (auto& k : random_keys) { + iter->Seek(k); + if (!iter->Valid()) { + ASSERT_EQ(true_data.lower_bound(k), true_data.end()); + continue; + } + std::string prop_value; + ASSERT_OK( + iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value)); + ASSERT_EQ("1", prop_value); + keys_slices.push_back(iter->key()); + true_keys.push_back(true_data.lower_bound(k)->first); + } + + for (size_t i = 0; i < keys_slices.size(); i++) { + ASSERT_EQ(keys_slices[i].ToString(), true_keys[i]); + } + } + + { + // Test SeekForPrev to random keys + std::vector keys_slices; + std::vector true_keys; + for (auto& k : random_keys) { + iter->SeekForPrev(k); + if (!iter->Valid()) { + ASSERT_EQ(true_data.upper_bound(k), true_data.begin()); + continue; + } + std::string prop_value; + ASSERT_OK( + iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value)); + ASSERT_EQ("1", prop_value); + keys_slices.push_back(iter->key()); + true_keys.push_back((--true_data.upper_bound(k))->first); + } + + for (size_t i = 0; i < keys_slices.size(); i++) { + ASSERT_EQ(keys_slices[i].ToString(), true_keys[i]); + } + } + + { + // Test iterating all data forward + std::vector all_keys; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + std::string prop_value; + ASSERT_OK( + iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value)); + ASSERT_EQ("1", prop_value); + all_keys.push_back(iter->key()); + } + ASSERT_EQ(all_keys.size(), true_data.size()); + + // Verify that all keys slices are valid + auto data_iter = true_data.begin(); + for (size_t i = 0; i < all_keys.size(); i++) { + ASSERT_EQ(all_keys[i].ToString(), data_iter->first); + data_iter++; + } + } + + { + // Test iterating all data backward + std::vector all_keys; + for (iter->SeekToLast(); iter->Valid(); iter->Prev()) { + std::string prop_value; + ASSERT_OK( + iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value)); + ASSERT_EQ("1", prop_value); + all_keys.push_back(iter->key()); + } + ASSERT_EQ(all_keys.size(), true_data.size()); + + // Verify that all keys slices are valid (backward) + auto data_iter = true_data.rbegin(); + for (size_t i = 0; i < all_keys.size(); i++) { + ASSERT_EQ(all_keys[i].ToString(), data_iter->first); + data_iter++; + } + } + + delete iter; + } +} + +#ifndef ROCKSDB_LITE +TEST_F(DBIteratorTest, PinnedDataIteratorMultipleFiles) { + Options options = CurrentOptions(); + BlockBasedTableOptions table_options; + table_options.use_delta_encoding = false; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.disable_auto_compactions = true; + options.write_buffer_size = 1024 * 1024 * 10; // 10 Mb + DestroyAndReopen(options); + + std::map true_data; + + // Generate 4 sst files in L2 + Random rnd(301); + for (int i = 1; i <= 1000; i++) { + std::string k = Key(i * 3); + std::string v = RandomString(&rnd, 100); + ASSERT_OK(Put(k, v)); + true_data[k] = v; + if (i % 250 == 0) { + ASSERT_OK(Flush()); + } + } + ASSERT_EQ(FilesPerLevel(0), "4"); + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ(FilesPerLevel(0), "0,4"); + + // Generate 4 sst files in L0 + for (int i = 1; i <= 1000; i++) { + std::string k = Key(i * 2); + std::string v = RandomString(&rnd, 100); + ASSERT_OK(Put(k, v)); + true_data[k] = v; + if (i % 250 == 0) { + ASSERT_OK(Flush()); + } + } + ASSERT_EQ(FilesPerLevel(0), "4,4"); + + // Add some keys/values in memtables + for (int i = 1; i <= 1000; i++) { + std::string k = Key(i); + std::string v = RandomString(&rnd, 100); + ASSERT_OK(Put(k, v)); + true_data[k] = v; + } + ASSERT_EQ(FilesPerLevel(0), "4,4"); + + ReadOptions ro; + ro.pin_data = true; + auto iter = db_->NewIterator(ro); + + std::vector> results; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + std::string prop_value; + ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value)); + ASSERT_EQ("1", prop_value); + results.emplace_back(iter->key(), iter->value().ToString()); + } + + ASSERT_EQ(results.size(), true_data.size()); + auto data_iter = true_data.begin(); + for (size_t i = 0; i < results.size(); i++, data_iter++) { + auto& kv = results[i]; + ASSERT_EQ(kv.first, data_iter->first); + ASSERT_EQ(kv.second, data_iter->second); + } + + delete iter; +} +#endif + +TEST_F(DBIteratorTest, PinnedDataIteratorMergeOperator) { + Options options = CurrentOptions(); + BlockBasedTableOptions table_options; + table_options.use_delta_encoding = false; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.merge_operator = MergeOperators::CreateUInt64AddOperator(); + DestroyAndReopen(options); + + std::string numbers[7]; + for (int val = 0; val <= 6; val++) { + PutFixed64(numbers + val, val); + } + + // +1 all keys in range [ 0 => 999] + for (int i = 0; i < 1000; i++) { + WriteOptions wo; + ASSERT_OK(db_->Merge(wo, Key(i), numbers[1])); + } + + // +2 all keys divisible by 2 in range [ 0 => 999] + for (int i = 0; i < 1000; i += 2) { + WriteOptions wo; + ASSERT_OK(db_->Merge(wo, Key(i), numbers[2])); + } + + // +3 all keys divisible by 5 in range [ 0 => 999] + for (int i = 0; i < 1000; i += 5) { + WriteOptions wo; + ASSERT_OK(db_->Merge(wo, Key(i), numbers[3])); + } + + ReadOptions ro; + ro.pin_data = true; + auto iter = db_->NewIterator(ro); + + std::vector> results; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + std::string prop_value; + ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value)); + ASSERT_EQ("1", prop_value); + results.emplace_back(iter->key(), iter->value().ToString()); + } + + ASSERT_EQ(results.size(), 1000); + for (size_t i = 0; i < results.size(); i++) { + auto& kv = results[i]; + ASSERT_EQ(kv.first, Key(static_cast(i))); + int expected_val = 1; + if (i % 2 == 0) { + expected_val += 2; + } + if (i % 5 == 0) { + expected_val += 3; + } + ASSERT_EQ(kv.second, numbers[expected_val]); + } + + delete iter; +} + +TEST_F(DBIteratorTest, PinnedDataIteratorReadAfterUpdate) { + Options options = CurrentOptions(); + BlockBasedTableOptions table_options; + table_options.use_delta_encoding = false; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.write_buffer_size = 100000; + DestroyAndReopen(options); + + Random rnd(301); + + std::map true_data; + for (int i = 0; i < 1000; i++) { + std::string k = RandomString(&rnd, 10); + std::string v = RandomString(&rnd, 1000); + ASSERT_OK(Put(k, v)); + true_data[k] = v; + } + + ReadOptions ro; + ro.pin_data = true; + auto iter = db_->NewIterator(ro); + + // Delete 50% of the keys and update the other 50% + for (auto& kv : true_data) { + if (rnd.OneIn(2)) { + ASSERT_OK(Delete(kv.first)); + } else { + std::string new_val = RandomString(&rnd, 1000); + ASSERT_OK(Put(kv.first, new_val)); + } + } + + std::vector> results; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + std::string prop_value; + ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value)); + ASSERT_EQ("1", prop_value); + results.emplace_back(iter->key(), iter->value().ToString()); + } + + auto data_iter = true_data.begin(); + for (size_t i = 0; i < results.size(); i++, data_iter++) { + auto& kv = results[i]; + ASSERT_EQ(kv.first, data_iter->first); + ASSERT_EQ(kv.second, data_iter->second); + } + + delete iter; +} + +TEST_F(DBIteratorTest, IterSeekForPrevCrossingFiles) { + Options options = CurrentOptions(); + options.prefix_extractor.reset(NewFixedPrefixTransform(1)); + options.disable_auto_compactions = true; + // Enable prefix bloom for SST files + BlockBasedTableOptions table_options; + table_options.filter_policy.reset(NewBloomFilterPolicy(10, true)); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + DestroyAndReopen(options); + + ASSERT_OK(Put("a1", "va1")); + ASSERT_OK(Put("a2", "va2")); + ASSERT_OK(Put("a3", "va3")); + ASSERT_OK(Flush()); + + ASSERT_OK(Put("b1", "vb1")); + ASSERT_OK(Put("b2", "vb2")); + ASSERT_OK(Put("b3", "vb3")); + ASSERT_OK(Flush()); + + ASSERT_OK(Put("b4", "vb4")); + ASSERT_OK(Put("d1", "vd1")); + ASSERT_OK(Put("d2", "vd2")); + ASSERT_OK(Put("d4", "vd4")); + ASSERT_OK(Flush()); + + MoveFilesToLevel(1); + { + ReadOptions ro; + Iterator* iter = db_->NewIterator(ro); + + iter->SeekForPrev("a4"); + ASSERT_EQ(iter->key().ToString(), "a3"); + ASSERT_EQ(iter->value().ToString(), "va3"); + + iter->SeekForPrev("c2"); + ASSERT_EQ(iter->key().ToString(), "b3"); + iter->SeekForPrev("d3"); + ASSERT_EQ(iter->key().ToString(), "d2"); + iter->SeekForPrev("b5"); + ASSERT_EQ(iter->key().ToString(), "b4"); + delete iter; + } + + { + ReadOptions ro; + ro.prefix_same_as_start = true; + Iterator* iter = db_->NewIterator(ro); + iter->SeekForPrev("c2"); + ASSERT_TRUE(!iter->Valid()); + delete iter; + } +} + +TEST_F(DBIteratorTest, IterPrevKeyCrossingBlocks) { + Options options = CurrentOptions(); + BlockBasedTableOptions table_options; + table_options.block_size = 1; // every block will contain one entry + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.merge_operator = MergeOperators::CreateStringAppendTESTOperator(); + options.disable_auto_compactions = true; + options.max_sequential_skip_in_iterations = 8; + + DestroyAndReopen(options); + + // Putting such deletes will force DBIter::Prev() to fallback to a Seek + for (int file_num = 0; file_num < 10; file_num++) { + ASSERT_OK(Delete("key4")); + ASSERT_OK(Flush()); + } + + // First File containing 5 blocks of puts + ASSERT_OK(Put("key1", "val1.0")); + ASSERT_OK(Put("key2", "val2.0")); + ASSERT_OK(Put("key3", "val3.0")); + ASSERT_OK(Put("key4", "val4.0")); + ASSERT_OK(Put("key5", "val5.0")); + ASSERT_OK(Flush()); + + // Second file containing 9 blocks of merge operands + ASSERT_OK(db_->Merge(WriteOptions(), "key1", "val1.1")); + ASSERT_OK(db_->Merge(WriteOptions(), "key1", "val1.2")); + + ASSERT_OK(db_->Merge(WriteOptions(), "key2", "val2.1")); + ASSERT_OK(db_->Merge(WriteOptions(), "key2", "val2.2")); + ASSERT_OK(db_->Merge(WriteOptions(), "key2", "val2.3")); + + ASSERT_OK(db_->Merge(WriteOptions(), "key3", "val3.1")); + ASSERT_OK(db_->Merge(WriteOptions(), "key3", "val3.2")); + ASSERT_OK(db_->Merge(WriteOptions(), "key3", "val3.3")); + ASSERT_OK(db_->Merge(WriteOptions(), "key3", "val3.4")); + ASSERT_OK(Flush()); + + { + ReadOptions ro; + ro.fill_cache = false; + Iterator* iter = db_->NewIterator(ro); + + iter->SeekToLast(); + ASSERT_EQ(iter->key().ToString(), "key5"); + ASSERT_EQ(iter->value().ToString(), "val5.0"); + + iter->Prev(); + ASSERT_EQ(iter->key().ToString(), "key4"); + ASSERT_EQ(iter->value().ToString(), "val4.0"); + + iter->Prev(); + ASSERT_EQ(iter->key().ToString(), "key3"); + ASSERT_EQ(iter->value().ToString(), "val3.0,val3.1,val3.2,val3.3,val3.4"); + + iter->Prev(); + ASSERT_EQ(iter->key().ToString(), "key2"); + ASSERT_EQ(iter->value().ToString(), "val2.0,val2.1,val2.2,val2.3"); + + iter->Prev(); + ASSERT_EQ(iter->key().ToString(), "key1"); + ASSERT_EQ(iter->value().ToString(), "val1.0,val1.1,val1.2"); + + delete iter; + } +} + +TEST_F(DBIteratorTest, IterPrevKeyCrossingBlocksRandomized) { + Options options = CurrentOptions(); + options.merge_operator = MergeOperators::CreateStringAppendTESTOperator(); + options.disable_auto_compactions = true; + options.level0_slowdown_writes_trigger = (1 << 30); + options.level0_stop_writes_trigger = (1 << 30); + options.max_sequential_skip_in_iterations = 8; + DestroyAndReopen(options); + + const int kNumKeys = 500; + // Small number of merge operands to make sure that DBIter::Prev() dont + // fall back to Seek() + const int kNumMergeOperands = 3; + // Use value size that will make sure that every block contain 1 key + const int kValSize = + static_cast(BlockBasedTableOptions().block_size) * 4; + // Percentage of keys that wont get merge operations + const int kNoMergeOpPercentage = 20; + // Percentage of keys that will be deleted + const int kDeletePercentage = 10; + + // For half of the key range we will write multiple deletes first to + // force DBIter::Prev() to fall back to Seek() + for (int file_num = 0; file_num < 10; file_num++) { + for (int i = 0; i < kNumKeys; i += 2) { + ASSERT_OK(Delete(Key(i))); + } + ASSERT_OK(Flush()); + } + + Random rnd(301); + std::map true_data; + std::string gen_key; + std::string gen_val; + + for (int i = 0; i < kNumKeys; i++) { + gen_key = Key(i); + gen_val = RandomString(&rnd, kValSize); + + ASSERT_OK(Put(gen_key, gen_val)); + true_data[gen_key] = gen_val; + } + ASSERT_OK(Flush()); + + // Separate values and merge operands in different file so that we + // make sure that we dont merge them while flushing but actually + // merge them in the read path + for (int i = 0; i < kNumKeys; i++) { + if (rnd.OneIn(static_cast(100.0 / kNoMergeOpPercentage))) { + // Dont give merge operations for some keys + continue; + } + + for (int j = 0; j < kNumMergeOperands; j++) { + gen_key = Key(i); + gen_val = RandomString(&rnd, kValSize); + + ASSERT_OK(db_->Merge(WriteOptions(), gen_key, gen_val)); + true_data[gen_key] += "," + gen_val; + } + } + ASSERT_OK(Flush()); + + for (int i = 0; i < kNumKeys; i++) { + if (rnd.OneIn(static_cast(100.0 / kDeletePercentage))) { + gen_key = Key(i); + + ASSERT_OK(Delete(gen_key)); + true_data.erase(gen_key); + } + } + ASSERT_OK(Flush()); + + { + ReadOptions ro; + ro.fill_cache = false; + Iterator* iter = db_->NewIterator(ro); + auto data_iter = true_data.rbegin(); + + for (iter->SeekToLast(); iter->Valid(); iter->Prev()) { + ASSERT_EQ(iter->key().ToString(), data_iter->first); + ASSERT_EQ(iter->value().ToString(), data_iter->second); + data_iter++; + } + ASSERT_EQ(data_iter, true_data.rend()); + + delete iter; + } + + { + ReadOptions ro; + ro.fill_cache = false; + Iterator* iter = db_->NewIterator(ro); + auto data_iter = true_data.rbegin(); + + int entries_right = 0; + std::string seek_key; + for (iter->SeekToLast(); iter->Valid(); iter->Prev()) { + // Verify key/value of current position + ASSERT_EQ(iter->key().ToString(), data_iter->first); + ASSERT_EQ(iter->value().ToString(), data_iter->second); + + bool restore_position_with_seek = rnd.Uniform(2); + if (restore_position_with_seek) { + seek_key = iter->key().ToString(); + } + + // Do some Next() operations the restore the iterator to orignal position + int next_count = + entries_right > 0 ? rnd.Uniform(std::min(entries_right, 10)) : 0; + for (int i = 0; i < next_count; i++) { + iter->Next(); + data_iter--; + + ASSERT_EQ(iter->key().ToString(), data_iter->first); + ASSERT_EQ(iter->value().ToString(), data_iter->second); + } + + if (restore_position_with_seek) { + // Restore orignal position using Seek() + iter->Seek(seek_key); + for (int i = 0; i < next_count; i++) { + data_iter++; + } + + ASSERT_EQ(iter->key().ToString(), data_iter->first); + ASSERT_EQ(iter->value().ToString(), data_iter->second); + } else { + // Restore original position using Prev() + for (int i = 0; i < next_count; i++) { + iter->Prev(); + data_iter++; + + ASSERT_EQ(iter->key().ToString(), data_iter->first); + ASSERT_EQ(iter->value().ToString(), data_iter->second); + } + } + + entries_right++; + data_iter++; + } + ASSERT_EQ(data_iter, true_data.rend()); + + delete iter; + } +} + +TEST_F(DBIteratorTest, IteratorWithLocalStatistics) { + Options options = CurrentOptions(); + options.statistics = rocksdb::CreateDBStatistics(); + DestroyAndReopen(options); + + Random rnd(301); + for (int i = 0; i < 1000; i++) { + // Key 10 bytes / Value 10 bytes + ASSERT_OK(Put(RandomString(&rnd, 10), RandomString(&rnd, 10))); + } + + std::atomic total_next(0); + std::atomic total_next_found(0); + std::atomic total_prev(0); + std::atomic total_prev_found(0); + std::atomic total_bytes(0); + + std::vector threads; + std::function reader_func_next = [&]() { + SetPerfLevel(kEnableCount); + get_perf_context()->Reset(); + Iterator* iter = db_->NewIterator(ReadOptions()); + + iter->SeekToFirst(); + // Seek will bump ITER_BYTES_READ + uint64_t bytes = 0; + bytes += iter->key().size(); + bytes += iter->value().size(); + while (true) { + iter->Next(); + total_next++; + + if (!iter->Valid()) { + break; + } + total_next_found++; + bytes += iter->key().size(); + bytes += iter->value().size(); + } + + delete iter; + ASSERT_EQ(bytes, get_perf_context()->iter_read_bytes); + SetPerfLevel(kDisable); + total_bytes += bytes; + }; + + std::function reader_func_prev = [&]() { + SetPerfLevel(kEnableCount); + Iterator* iter = db_->NewIterator(ReadOptions()); + + iter->SeekToLast(); + // Seek will bump ITER_BYTES_READ + uint64_t bytes = 0; + bytes += iter->key().size(); + bytes += iter->value().size(); + while (true) { + iter->Prev(); + total_prev++; + + if (!iter->Valid()) { + break; + } + total_prev_found++; + bytes += iter->key().size(); + bytes += iter->value().size(); + } + + delete iter; + ASSERT_EQ(bytes, get_perf_context()->iter_read_bytes); + SetPerfLevel(kDisable); + total_bytes += bytes; + }; + + for (int i = 0; i < 10; i++) { + threads.emplace_back(reader_func_next); + } + for (int i = 0; i < 15; i++) { + threads.emplace_back(reader_func_prev); + } + + for (auto& t : threads) { + t.join(); + } + + ASSERT_EQ(TestGetTickerCount(options, NUMBER_DB_NEXT), (uint64_t)total_next); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_DB_NEXT_FOUND), + (uint64_t)total_next_found); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_DB_PREV), (uint64_t)total_prev); + ASSERT_EQ(TestGetTickerCount(options, NUMBER_DB_PREV_FOUND), + (uint64_t)total_prev_found); + ASSERT_EQ(TestGetTickerCount(options, ITER_BYTES_READ), (uint64_t)total_bytes); + +} + +TEST_F(DBIteratorTest, ReadAhead) { + Options options; + env_->count_random_reads_ = true; + options.env = env_; + options.disable_auto_compactions = true; + options.write_buffer_size = 4 << 20; + options.statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions table_options; + table_options.block_size = 1024; + table_options.no_block_cache = true; + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + Reopen(options); + + std::string value(1024, 'a'); + for (int i = 0; i < 100; i++) { + Put(Key(i), value); + } + ASSERT_OK(Flush()); + MoveFilesToLevel(2); + + for (int i = 0; i < 100; i++) { + Put(Key(i), value); + } + ASSERT_OK(Flush()); + MoveFilesToLevel(1); + + for (int i = 0; i < 100; i++) { + Put(Key(i), value); + } + ASSERT_OK(Flush()); +#ifndef ROCKSDB_LITE + ASSERT_EQ("1,1,1", FilesPerLevel()); +#endif // !ROCKSDB_LITE + + env_->random_read_bytes_counter_ = 0; + options.statistics->setTickerCount(NO_FILE_OPENS, 0); + ReadOptions read_options; + auto* iter = db_->NewIterator(read_options); + iter->SeekToFirst(); + int64_t num_file_opens = TestGetTickerCount(options, NO_FILE_OPENS); + size_t bytes_read = env_->random_read_bytes_counter_; + delete iter; + + env_->random_read_bytes_counter_ = 0; + options.statistics->setTickerCount(NO_FILE_OPENS, 0); + read_options.readahead_size = 1024 * 10; + iter = db_->NewIterator(read_options); + iter->SeekToFirst(); + int64_t num_file_opens_readahead = TestGetTickerCount(options, NO_FILE_OPENS); + size_t bytes_read_readahead = env_->random_read_bytes_counter_; + delete iter; + ASSERT_EQ(num_file_opens + 3, num_file_opens_readahead); + ASSERT_GT(bytes_read_readahead, bytes_read); + ASSERT_GT(bytes_read_readahead, read_options.readahead_size * 3); + + // Verify correctness. + iter = db_->NewIterator(read_options); + int count = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_EQ(value, iter->value()); + count++; + } + ASSERT_EQ(100, count); + for (int i = 0; i < 100; i++) { + iter->Seek(Key(i)); + ASSERT_EQ(value, iter->value()); + } + delete iter; +} + +// Insert a key, create a snapshot iterator, overwrite key lots of times, +// seek to a smaller key. Expect DBIter to fall back to a seek instead of +// going through all the overwrites linearly. +TEST_F(DBIteratorTest, DBIteratorSkipRecentDuplicatesTest) { + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.max_sequential_skip_in_iterations = 3; + options.prefix_extractor = nullptr; + options.write_buffer_size = 1 << 27; // big enough to avoid flush + options.statistics = rocksdb::CreateDBStatistics(); + DestroyAndReopen(options); + + // Insert. + ASSERT_OK(Put("b", "0")); + + // Create iterator. + ReadOptions ro; + std::unique_ptr iter(db_->NewIterator(ro)); + + // Insert a lot. + for (int i = 0; i < 100; ++i) { + ASSERT_OK(Put("b", std::to_string(i + 1).c_str())); + } + +#ifndef ROCKSDB_LITE + // Check that memtable wasn't flushed. + std::string val; + ASSERT_TRUE(db_->GetProperty("rocksdb.num-files-at-level0", &val)); + EXPECT_EQ("0", val); +#endif + + // Seek iterator to a smaller key. + get_perf_context()->Reset(); + iter->Seek("a"); + ASSERT_TRUE(iter->Valid()); + EXPECT_EQ("b", iter->key().ToString()); + EXPECT_EQ("0", iter->value().ToString()); + + // Check that the seek didn't do too much work. + // Checks are not tight, just make sure that everything is well below 100. + EXPECT_LT(get_perf_context()->internal_key_skipped_count, 4); + EXPECT_LT(get_perf_context()->internal_recent_skipped_count, 8); + EXPECT_LT(get_perf_context()->seek_on_memtable_count, 10); + EXPECT_LT(get_perf_context()->next_on_memtable_count, 10); + EXPECT_LT(get_perf_context()->prev_on_memtable_count, 10); + + // Check that iterator did something like what we expect. + EXPECT_EQ(get_perf_context()->internal_delete_skipped_count, 0); + EXPECT_EQ(get_perf_context()->internal_merge_count, 0); + EXPECT_GE(get_perf_context()->internal_recent_skipped_count, 2); + EXPECT_GE(get_perf_context()->seek_on_memtable_count, 2); + EXPECT_EQ(1, options.statistics->getTickerCount( + NUMBER_OF_RESEEKS_IN_ITERATION)); +} + +TEST_F(DBIteratorTest, Refresh) { + ASSERT_OK(Put("x", "y")); + + std::unique_ptr iter(db_->NewIterator(ReadOptions())); + iter->Seek(Slice("a")); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("x")), 0); + iter->Next(); + ASSERT_FALSE(iter->Valid()); + + ASSERT_OK(Put("c", "d")); + + iter->Seek(Slice("a")); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("x")), 0); + iter->Next(); + ASSERT_FALSE(iter->Valid()); + + iter->Refresh(); + + iter->Seek(Slice("a")); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("c")), 0); + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("x")), 0); + iter->Next(); + ASSERT_FALSE(iter->Valid()); + + dbfull()->Flush(FlushOptions()); + + ASSERT_OK(Put("m", "n")); + + iter->Seek(Slice("a")); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("c")), 0); + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("x")), 0); + iter->Next(); + ASSERT_FALSE(iter->Valid()); + + iter->Refresh(); + + iter->Seek(Slice("a")); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("c")), 0); + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("m")), 0); + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Slice("x")), 0); + iter->Next(); + ASSERT_FALSE(iter->Valid()); + + iter.reset(); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_log_iter_test.cc b/db/db_log_iter_test.cc new file mode 100644 index 00000000000..e7f94c4c423 --- /dev/null +++ b/db/db_log_iter_test.cc @@ -0,0 +1,294 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +// Introduction of SyncPoint effectively disabled building and running this test +// in Release build. +// which is a pity, it is a good test +#if !defined(ROCKSDB_LITE) + +#include "db/db_test_util.h" +#include "port/stack_trace.h" + +namespace rocksdb { + +class DBTestXactLogIterator : public DBTestBase { + public: + DBTestXactLogIterator() : DBTestBase("/db_log_iter_test") {} + + std::unique_ptr OpenTransactionLogIter( + const SequenceNumber seq) { + unique_ptr iter; + Status status = dbfull()->GetUpdatesSince(seq, &iter); + EXPECT_OK(status); + EXPECT_TRUE(iter->Valid()); + return iter; + } +}; + +namespace { +SequenceNumber ReadRecords( + std::unique_ptr& iter, + int& count) { + count = 0; + SequenceNumber lastSequence = 0; + BatchResult res; + while (iter->Valid()) { + res = iter->GetBatch(); + EXPECT_TRUE(res.sequence > lastSequence); + ++count; + lastSequence = res.sequence; + EXPECT_OK(iter->status()); + iter->Next(); + } + return res.sequence; +} + +void ExpectRecords( + const int expected_no_records, + std::unique_ptr& iter) { + int num_records; + ReadRecords(iter, num_records); + ASSERT_EQ(num_records, expected_no_records); +} +} // namespace + +TEST_F(DBTestXactLogIterator, TransactionLogIterator) { + do { + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + Put(0, "key1", DummyString(1024)); + Put(1, "key2", DummyString(1024)); + Put(1, "key2", DummyString(1024)); + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 3U); + { + auto iter = OpenTransactionLogIter(0); + ExpectRecords(3, iter); + } + ReopenWithColumnFamilies({"default", "pikachu"}, options); + env_->SleepForMicroseconds(2 * 1000 * 1000); + { + Put(0, "key4", DummyString(1024)); + Put(1, "key5", DummyString(1024)); + Put(0, "key6", DummyString(1024)); + } + { + auto iter = OpenTransactionLogIter(0); + ExpectRecords(6, iter); + } + } while (ChangeCompactOptions()); +} + +#ifndef NDEBUG // sync point is not included with DNDEBUG build +TEST_F(DBTestXactLogIterator, TransactionLogIteratorRace) { + static const int LOG_ITERATOR_RACE_TEST_COUNT = 2; + static const char* sync_points[LOG_ITERATOR_RACE_TEST_COUNT][4] = { + {"WalManager::GetSortedWalFiles:1", "WalManager::PurgeObsoleteFiles:1", + "WalManager::PurgeObsoleteFiles:2", "WalManager::GetSortedWalFiles:2"}, + {"WalManager::GetSortedWalsOfType:1", + "WalManager::PurgeObsoleteFiles:1", + "WalManager::PurgeObsoleteFiles:2", + "WalManager::GetSortedWalsOfType:2"}}; + for (int test = 0; test < LOG_ITERATOR_RACE_TEST_COUNT; ++test) { + // Setup sync point dependency to reproduce the race condition of + // a log file moved to archived dir, in the middle of GetSortedWalFiles + rocksdb::SyncPoint::GetInstance()->LoadDependency( + { { sync_points[test][0], sync_points[test][1] }, + { sync_points[test][2], sync_points[test][3] }, + }); + + do { + rocksdb::SyncPoint::GetInstance()->ClearTrace(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + Put("key1", DummyString(1024)); + dbfull()->Flush(FlushOptions()); + Put("key2", DummyString(1024)); + dbfull()->Flush(FlushOptions()); + Put("key3", DummyString(1024)); + dbfull()->Flush(FlushOptions()); + Put("key4", DummyString(1024)); + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 4U); + dbfull()->FlushWAL(false); + + { + auto iter = OpenTransactionLogIter(0); + ExpectRecords(4, iter); + } + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + // trigger async flush, and log move. Well, log move will + // wait until the GetSortedWalFiles:1 to reproduce the race + // condition + FlushOptions flush_options; + flush_options.wait = false; + dbfull()->Flush(flush_options); + + // "key5" would be written in a new memtable and log + Put("key5", DummyString(1024)); + dbfull()->FlushWAL(false); + { + // this iter would miss "key4" if not fixed + auto iter = OpenTransactionLogIter(0); + ExpectRecords(5, iter); + } + } while (ChangeCompactOptions()); + } +} +#endif + +TEST_F(DBTestXactLogIterator, TransactionLogIteratorStallAtLastRecord) { + do { + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + Put("key1", DummyString(1024)); + auto iter = OpenTransactionLogIter(0); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + iter->Next(); + ASSERT_TRUE(!iter->Valid()); + ASSERT_OK(iter->status()); + Put("key2", DummyString(1024)); + iter->Next(); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + } while (ChangeCompactOptions()); +} + +TEST_F(DBTestXactLogIterator, TransactionLogIteratorCheckAfterRestart) { + do { + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + Put("key1", DummyString(1024)); + Put("key2", DummyString(1023)); + dbfull()->Flush(FlushOptions()); + Reopen(options); + auto iter = OpenTransactionLogIter(0); + ExpectRecords(2, iter); + } while (ChangeCompactOptions()); +} + +TEST_F(DBTestXactLogIterator, TransactionLogIteratorCorruptedLog) { + do { + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + for (int i = 0; i < 1024; i++) { + Put("key"+ToString(i), DummyString(10)); + } + dbfull()->Flush(FlushOptions()); + dbfull()->FlushWAL(false); + // Corrupt this log to create a gap + rocksdb::VectorLogPtr wal_files; + ASSERT_OK(dbfull()->GetSortedWalFiles(wal_files)); + const auto logfile_path = dbname_ + "/" + wal_files.front()->PathName(); + if (mem_env_) { + mem_env_->Truncate(logfile_path, wal_files.front()->SizeFileBytes() / 2); + } else { + ASSERT_EQ(0, truncate(logfile_path.c_str(), + wal_files.front()->SizeFileBytes() / 2)); + } + + // Insert a new entry to a new log file + Put("key1025", DummyString(10)); + dbfull()->FlushWAL(false); + // Try to read from the beginning. Should stop before the gap and read less + // than 1025 entries + auto iter = OpenTransactionLogIter(0); + int count; + SequenceNumber last_sequence_read = ReadRecords(iter, count); + ASSERT_LT(last_sequence_read, 1025U); + // Try to read past the gap, should be able to seek to key1025 + auto iter2 = OpenTransactionLogIter(last_sequence_read + 1); + ExpectRecords(1, iter2); + } while (ChangeCompactOptions()); +} + +TEST_F(DBTestXactLogIterator, TransactionLogIteratorBatchOperations) { + do { + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + WriteBatch batch; + batch.Put(handles_[1], "key1", DummyString(1024)); + batch.Put(handles_[0], "key2", DummyString(1024)); + batch.Put(handles_[1], "key3", DummyString(1024)); + batch.Delete(handles_[0], "key2"); + dbfull()->Write(WriteOptions(), &batch); + Flush(1); + Flush(0); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + Put(1, "key4", DummyString(1024)); + auto iter = OpenTransactionLogIter(3); + ExpectRecords(2, iter); + } while (ChangeCompactOptions()); +} + +TEST_F(DBTestXactLogIterator, TransactionLogIteratorBlobs) { + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + { + WriteBatch batch; + batch.Put(handles_[1], "key1", DummyString(1024)); + batch.Put(handles_[0], "key2", DummyString(1024)); + batch.PutLogData(Slice("blob1")); + batch.Put(handles_[1], "key3", DummyString(1024)); + batch.PutLogData(Slice("blob2")); + batch.Delete(handles_[0], "key2"); + dbfull()->Write(WriteOptions(), &batch); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + } + + auto res = OpenTransactionLogIter(0)->GetBatch(); + struct Handler : public WriteBatch::Handler { + std::string seen; + virtual Status PutCF(uint32_t cf, const Slice& key, + const Slice& value) override { + seen += "Put(" + ToString(cf) + ", " + key.ToString() + ", " + + ToString(value.size()) + ")"; + return Status::OK(); + } + virtual Status MergeCF(uint32_t cf, const Slice& key, + const Slice& value) override { + seen += "Merge(" + ToString(cf) + ", " + key.ToString() + ", " + + ToString(value.size()) + ")"; + return Status::OK(); + } + virtual void LogData(const Slice& blob) override { + seen += "LogData(" + blob.ToString() + ")"; + } + virtual Status DeleteCF(uint32_t cf, const Slice& key) override { + seen += "Delete(" + ToString(cf) + ", " + key.ToString() + ")"; + return Status::OK(); + } + } handler; + res.writeBatchPtr->Iterate(&handler); + ASSERT_EQ( + "Put(1, key1, 1024)" + "Put(0, key2, 1024)" + "LogData(blob1)" + "Put(1, key3, 1024)" + "LogData(blob2)" + "Delete(0, key2)", + handler.seen); +} +} // namespace rocksdb + +#endif // !defined(ROCKSDB_LITE) + +int main(int argc, char** argv) { +#if !defined(ROCKSDB_LITE) + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +#else + return 0; +#endif +} diff --git a/db/db_memtable_test.cc b/db/db_memtable_test.cc new file mode 100644 index 00000000000..63d274f6ab5 --- /dev/null +++ b/db/db_memtable_test.cc @@ -0,0 +1,195 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include +#include + +#include "db/db_test_util.h" +#include "db/memtable.h" +#include "port/stack_trace.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/slice_transform.h" + +namespace rocksdb { + +class DBMemTableTest : public DBTestBase { + public: + DBMemTableTest() : DBTestBase("/db_memtable_test") {} +}; + +class MockMemTableRep : public MemTableRep { + public: + explicit MockMemTableRep(Allocator* allocator, MemTableRep* rep) + : MemTableRep(allocator), rep_(rep), num_insert_with_hint_(0) {} + + virtual KeyHandle Allocate(const size_t len, char** buf) override { + return rep_->Allocate(len, buf); + } + + virtual void Insert(KeyHandle handle) override { + return rep_->Insert(handle); + } + + virtual void InsertWithHint(KeyHandle handle, void** hint) override { + num_insert_with_hint_++; + ASSERT_NE(nullptr, hint); + last_hint_in_ = *hint; + rep_->InsertWithHint(handle, hint); + last_hint_out_ = *hint; + } + + virtual bool Contains(const char* key) const override { + return rep_->Contains(key); + } + + virtual void Get(const LookupKey& k, void* callback_args, + bool (*callback_func)(void* arg, + const char* entry)) override { + rep_->Get(k, callback_args, callback_func); + } + + virtual size_t ApproximateMemoryUsage() override { + return rep_->ApproximateMemoryUsage(); + } + + virtual Iterator* GetIterator(Arena* arena) override { + return rep_->GetIterator(arena); + } + + void* last_hint_in() { return last_hint_in_; } + void* last_hint_out() { return last_hint_out_; } + int num_insert_with_hint() { return num_insert_with_hint_; } + + private: + std::unique_ptr rep_; + void* last_hint_in_; + void* last_hint_out_; + int num_insert_with_hint_; +}; + +class MockMemTableRepFactory : public MemTableRepFactory { + public: + virtual MemTableRep* CreateMemTableRep(const MemTableRep::KeyComparator& cmp, + Allocator* allocator, + const SliceTransform* transform, + Logger* logger) override { + SkipListFactory factory; + MemTableRep* skiplist_rep = + factory.CreateMemTableRep(cmp, allocator, transform, logger); + mock_rep_ = new MockMemTableRep(allocator, skiplist_rep); + return mock_rep_; + } + + virtual MemTableRep* CreateMemTableRep(const MemTableRep::KeyComparator& cmp, + Allocator* allocator, + const SliceTransform* transform, + Logger* logger, + uint32_t column_family_id) override { + last_column_family_id_ = column_family_id; + return CreateMemTableRep(cmp, allocator, transform, logger); + } + + virtual const char* Name() const override { return "MockMemTableRepFactory"; } + + MockMemTableRep* rep() { return mock_rep_; } + + bool IsInsertConcurrentlySupported() const override { return false; } + + uint32_t GetLastColumnFamilyId() { return last_column_family_id_; } + + private: + MockMemTableRep* mock_rep_; + // workaround since there's no port::kMaxUint32 yet. + uint32_t last_column_family_id_ = static_cast(-1); +}; + +class TestPrefixExtractor : public SliceTransform { + public: + virtual const char* Name() const override { return "TestPrefixExtractor"; } + + virtual Slice Transform(const Slice& key) const override { + const char* p = separator(key); + if (p == nullptr) { + return Slice(); + } + return Slice(key.data(), p - key.data() + 1); + } + + virtual bool InDomain(const Slice& key) const override { + return separator(key) != nullptr; + } + + virtual bool InRange(const Slice& key) const override { return false; } + + private: + const char* separator(const Slice& key) const { + return reinterpret_cast(memchr(key.data(), '_', key.size())); + } +}; + +TEST_F(DBMemTableTest, InsertWithHint) { + Options options; + options.allow_concurrent_memtable_write = false; + options.create_if_missing = true; + options.memtable_factory.reset(new MockMemTableRepFactory()); + options.memtable_insert_with_hint_prefix_extractor.reset( + new TestPrefixExtractor()); + options.env = env_; + Reopen(options); + MockMemTableRep* rep = + reinterpret_cast(options.memtable_factory.get()) + ->rep(); + ASSERT_OK(Put("foo_k1", "foo_v1")); + ASSERT_EQ(nullptr, rep->last_hint_in()); + void* hint_foo = rep->last_hint_out(); + ASSERT_OK(Put("foo_k2", "foo_v2")); + ASSERT_EQ(hint_foo, rep->last_hint_in()); + ASSERT_EQ(hint_foo, rep->last_hint_out()); + ASSERT_OK(Put("foo_k3", "foo_v3")); + ASSERT_EQ(hint_foo, rep->last_hint_in()); + ASSERT_EQ(hint_foo, rep->last_hint_out()); + ASSERT_OK(Put("bar_k1", "bar_v1")); + ASSERT_EQ(nullptr, rep->last_hint_in()); + void* hint_bar = rep->last_hint_out(); + ASSERT_NE(hint_foo, hint_bar); + ASSERT_OK(Put("bar_k2", "bar_v2")); + ASSERT_EQ(hint_bar, rep->last_hint_in()); + ASSERT_EQ(hint_bar, rep->last_hint_out()); + ASSERT_EQ(5, rep->num_insert_with_hint()); + ASSERT_OK(Put("whitelisted", "vvv")); + ASSERT_EQ(5, rep->num_insert_with_hint()); + ASSERT_EQ("foo_v1", Get("foo_k1")); + ASSERT_EQ("foo_v2", Get("foo_k2")); + ASSERT_EQ("foo_v3", Get("foo_k3")); + ASSERT_EQ("bar_v1", Get("bar_k1")); + ASSERT_EQ("bar_v2", Get("bar_k2")); + ASSERT_EQ("vvv", Get("whitelisted")); +} + +TEST_F(DBMemTableTest, ColumnFamilyId) { + // Verifies MemTableRepFactory is told the right column family id. + Options options; + options.allow_concurrent_memtable_write = false; + options.create_if_missing = true; + options.memtable_factory.reset(new MockMemTableRepFactory()); + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + for (int cf = 0; cf < 2; ++cf) { + ASSERT_OK(Put(cf, "key", "val")); + ASSERT_OK(Flush(cf)); + ASSERT_EQ( + cf, static_cast(options.memtable_factory.get()) + ->GetLastColumnFamilyId()); + } +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_merge_operator_test.cc b/db/db_merge_operator_test.cc new file mode 100644 index 00000000000..de286191064 --- /dev/null +++ b/db/db_merge_operator_test.cc @@ -0,0 +1,367 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#include +#include + +#include "db/db_test_util.h" +#include "db/forward_iterator.h" +#include "port/stack_trace.h" +#include "utilities/merge_operators.h" + +namespace rocksdb { + +// Test merge operator functionality. +class DBMergeOperatorTest : public DBTestBase { + public: + DBMergeOperatorTest() : DBTestBase("/db_merge_operator_test") {} +}; + +TEST_F(DBMergeOperatorTest, MergeErrorOnRead) { + Options options; + options.create_if_missing = true; + options.merge_operator.reset(new TestPutOperator()); + options.env = env_; + Reopen(options); + ASSERT_OK(Merge("k1", "v1")); + ASSERT_OK(Merge("k1", "corrupted")); + std::string value; + ASSERT_TRUE(db_->Get(ReadOptions(), "k1", &value).IsCorruption()); + VerifyDBInternal({{"k1", "corrupted"}, {"k1", "v1"}}); +} + +TEST_F(DBMergeOperatorTest, MergeErrorOnWrite) { + Options options; + options.create_if_missing = true; + options.merge_operator.reset(new TestPutOperator()); + options.max_successive_merges = 3; + options.env = env_; + Reopen(options); + ASSERT_OK(Merge("k1", "v1")); + ASSERT_OK(Merge("k1", "v2")); + // Will trigger a merge when hitting max_successive_merges and the merge + // will fail. The delta will be inserted nevertheless. + ASSERT_OK(Merge("k1", "corrupted")); + // Data should stay unmerged after the error. + VerifyDBInternal({{"k1", "corrupted"}, {"k1", "v2"}, {"k1", "v1"}}); +} + +TEST_F(DBMergeOperatorTest, MergeErrorOnIteration) { + Options options; + options.create_if_missing = true; + options.merge_operator.reset(new TestPutOperator()); + options.env = env_; + + DestroyAndReopen(options); + ASSERT_OK(Merge("k1", "v1")); + ASSERT_OK(Merge("k1", "corrupted")); + ASSERT_OK(Put("k2", "v2")); + VerifyDBFromMap({{"k1", ""}, {"k2", "v2"}}, nullptr, false, + {{"k1", Status::Corruption()}}); + VerifyDBInternal({{"k1", "corrupted"}, {"k1", "v1"}, {"k2", "v2"}}); + + DestroyAndReopen(options); + ASSERT_OK(Merge("k1", "v1")); + ASSERT_OK(Put("k2", "v2")); + ASSERT_OK(Merge("k2", "corrupted")); + VerifyDBFromMap({{"k1", "v1"}, {"k2", ""}}, nullptr, false, + {{"k2", Status::Corruption()}}); + VerifyDBInternal({{"k1", "v1"}, {"k2", "corrupted"}, {"k2", "v2"}}); +} + + +class MergeOperatorPinningTest : public DBMergeOperatorTest, + public testing::WithParamInterface { + public: + MergeOperatorPinningTest() { disable_block_cache_ = GetParam(); } + + bool disable_block_cache_; +}; + +INSTANTIATE_TEST_CASE_P(MergeOperatorPinningTest, MergeOperatorPinningTest, + ::testing::Bool()); + +#ifndef ROCKSDB_LITE +TEST_P(MergeOperatorPinningTest, OperandsMultiBlocks) { + Options options = CurrentOptions(); + BlockBasedTableOptions table_options; + table_options.block_size = 1; // every block will contain one entry + table_options.no_block_cache = disable_block_cache_; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.merge_operator = MergeOperators::CreateStringAppendTESTOperator(); + options.level0_slowdown_writes_trigger = (1 << 30); + options.level0_stop_writes_trigger = (1 << 30); + options.disable_auto_compactions = true; + DestroyAndReopen(options); + + const int kKeysPerFile = 10; + const int kOperandsPerKeyPerFile = 7; + const int kOperandSize = 100; + // Filse to write in L0 before compacting to lower level + const int kFilesPerLevel = 3; + + Random rnd(301); + std::map true_data; + int batch_num = 1; + int lvl_to_fill = 4; + int key_id = 0; + while (true) { + for (int j = 0; j < kKeysPerFile; j++) { + std::string key = Key(key_id % 35); + key_id++; + for (int k = 0; k < kOperandsPerKeyPerFile; k++) { + std::string val = RandomString(&rnd, kOperandSize); + ASSERT_OK(db_->Merge(WriteOptions(), key, val)); + if (true_data[key].size() == 0) { + true_data[key] = val; + } else { + true_data[key] += "," + val; + } + } + } + + if (lvl_to_fill == -1) { + // Keep last batch in memtable and stop + break; + } + + ASSERT_OK(Flush()); + if (batch_num % kFilesPerLevel == 0) { + if (lvl_to_fill != 0) { + MoveFilesToLevel(lvl_to_fill); + } + lvl_to_fill--; + } + batch_num++; + } + + // 3 L0 files + // 1 L1 file + // 3 L2 files + // 1 L3 file + // 3 L4 Files + ASSERT_EQ(FilesPerLevel(), "3,1,3,1,3"); + + VerifyDBFromMap(true_data); +} + +TEST_P(MergeOperatorPinningTest, Randomized) { + do { + Options options = CurrentOptions(); + options.merge_operator = MergeOperators::CreateMaxOperator(); + BlockBasedTableOptions table_options; + table_options.no_block_cache = disable_block_cache_; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + DestroyAndReopen(options); + + Random rnd(301); + std::map true_data; + + const int kTotalMerges = 10000; + // Every key gets ~10 operands + const int kKeyRange = kTotalMerges / 10; + const int kOperandSize = 20; + const int kNumPutBefore = kKeyRange / 10; // 10% value + const int kNumPutAfter = kKeyRange / 10; // 10% overwrite + const int kNumDelete = kKeyRange / 10; // 10% delete + + // kNumPutBefore keys will have base values + for (int i = 0; i < kNumPutBefore; i++) { + std::string key = Key(rnd.Next() % kKeyRange); + std::string value = RandomString(&rnd, kOperandSize); + ASSERT_OK(db_->Put(WriteOptions(), key, value)); + + true_data[key] = value; + } + + // Do kTotalMerges merges + for (int i = 0; i < kTotalMerges; i++) { + std::string key = Key(rnd.Next() % kKeyRange); + std::string value = RandomString(&rnd, kOperandSize); + ASSERT_OK(db_->Merge(WriteOptions(), key, value)); + + if (true_data[key] < value) { + true_data[key] = value; + } + } + + // Overwrite random kNumPutAfter keys + for (int i = 0; i < kNumPutAfter; i++) { + std::string key = Key(rnd.Next() % kKeyRange); + std::string value = RandomString(&rnd, kOperandSize); + ASSERT_OK(db_->Put(WriteOptions(), key, value)); + + true_data[key] = value; + } + + // Delete random kNumDelete keys + for (int i = 0; i < kNumDelete; i++) { + std::string key = Key(rnd.Next() % kKeyRange); + ASSERT_OK(db_->Delete(WriteOptions(), key)); + + true_data.erase(key); + } + + VerifyDBFromMap(true_data); + + // Skip HashCuckoo since it does not support merge operators + } while (ChangeOptions(kSkipMergePut | kSkipHashCuckoo)); +} + +class MergeOperatorHook : public MergeOperator { + public: + explicit MergeOperatorHook(std::shared_ptr _merge_op) + : merge_op_(_merge_op) {} + + virtual bool FullMergeV2(const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const override { + before_merge_(); + bool res = merge_op_->FullMergeV2(merge_in, merge_out); + after_merge_(); + return res; + } + + virtual const char* Name() const override { return merge_op_->Name(); } + + std::shared_ptr merge_op_; + std::function before_merge_ = []() {}; + std::function after_merge_ = []() {}; +}; + +TEST_P(MergeOperatorPinningTest, EvictCacheBeforeMerge) { + Options options = CurrentOptions(); + + auto merge_hook = + std::make_shared(MergeOperators::CreateMaxOperator()); + options.merge_operator = merge_hook; + options.disable_auto_compactions = true; + options.level0_slowdown_writes_trigger = (1 << 30); + options.level0_stop_writes_trigger = (1 << 30); + options.max_open_files = 20; + BlockBasedTableOptions bbto; + bbto.no_block_cache = disable_block_cache_; + if (bbto.no_block_cache == false) { + bbto.block_cache = NewLRUCache(64 * 1024 * 1024); + } else { + bbto.block_cache = nullptr; + } + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + DestroyAndReopen(options); + + const int kNumOperands = 30; + const int kNumKeys = 1000; + const int kOperandSize = 100; + Random rnd(301); + + // 1000 keys every key have 30 operands, every operand is in a different file + std::map true_data; + for (int i = 0; i < kNumOperands; i++) { + for (int j = 0; j < kNumKeys; j++) { + std::string k = Key(j); + std::string v = RandomString(&rnd, kOperandSize); + ASSERT_OK(db_->Merge(WriteOptions(), k, v)); + + true_data[k] = std::max(true_data[k], v); + } + ASSERT_OK(Flush()); + } + + std::vector file_numbers = ListTableFiles(env_, dbname_); + ASSERT_EQ(file_numbers.size(), kNumOperands); + int merge_cnt = 0; + + // Code executed before merge operation + merge_hook->before_merge_ = [&]() { + // Evict all tables from cache before every merge operation + for (uint64_t num : file_numbers) { + TableCache::Evict(dbfull()->TEST_table_cache(), num); + } + // Decrease cache capacity to force all unrefed blocks to be evicted + if (bbto.block_cache) { + bbto.block_cache->SetCapacity(1); + } + merge_cnt++; + }; + + // Code executed after merge operation + merge_hook->after_merge_ = [&]() { + // Increase capacity again after doing the merge + if (bbto.block_cache) { + bbto.block_cache->SetCapacity(64 * 1024 * 1024); + } + }; + + size_t total_reads; + VerifyDBFromMap(true_data, &total_reads); + ASSERT_EQ(merge_cnt, total_reads); + + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + + VerifyDBFromMap(true_data, &total_reads); +} + +TEST_P(MergeOperatorPinningTest, TailingIterator) { + Options options = CurrentOptions(); + options.merge_operator = MergeOperators::CreateMaxOperator(); + BlockBasedTableOptions bbto; + bbto.no_block_cache = disable_block_cache_; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + DestroyAndReopen(options); + + const int kNumOperands = 100; + const int kNumWrites = 100000; + + std::function writer_func = [&]() { + int k = 0; + for (int i = 0; i < kNumWrites; i++) { + db_->Merge(WriteOptions(), Key(k), Key(k)); + + if (i && i % kNumOperands == 0) { + k++; + } + if (i && i % 127 == 0) { + ASSERT_OK(Flush()); + } + if (i && i % 317 == 0) { + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + } + } + }; + + std::function reader_func = [&]() { + ReadOptions ro; + ro.tailing = true; + Iterator* iter = db_->NewIterator(ro); + + iter->SeekToFirst(); + for (int i = 0; i < (kNumWrites / kNumOperands); i++) { + while (!iter->Valid()) { + // wait for the key to be written + env_->SleepForMicroseconds(100); + iter->Seek(Key(i)); + } + ASSERT_EQ(iter->key(), Key(i)); + ASSERT_EQ(iter->value(), Key(i)); + + iter->Next(); + } + + delete iter; + }; + + rocksdb::port::Thread writer_thread(writer_func); + rocksdb::port::Thread reader_thread(reader_func); + + writer_thread.join(); + reader_thread.join(); +} +#endif // ROCKSDB_LITE + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_options_test.cc b/db/db_options_test.cc new file mode 100644 index 00000000000..243748f9fa4 --- /dev/null +++ b/db/db_options_test.cc @@ -0,0 +1,452 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#include +#include +#include + +#include "db/column_family.h" +#include "db/db_impl.h" +#include "db/db_test_util.h" +#include "options/options_helper.h" +#include "port/stack_trace.h" +#include "rocksdb/cache.h" +#include "rocksdb/convenience.h" +#include "rocksdb/rate_limiter.h" +#include "util/random.h" +#include "util/sync_point.h" +#include "util/testutil.h" + +namespace rocksdb { + +class DBOptionsTest : public DBTestBase { + public: + DBOptionsTest() : DBTestBase("/db_options_test") {} + +#ifndef ROCKSDB_LITE + std::unordered_map GetMutableDBOptionsMap( + const DBOptions& options) { + std::string options_str; + GetStringFromDBOptions(&options_str, options); + std::unordered_map options_map; + StringToMap(options_str, &options_map); + std::unordered_map mutable_map; + for (const auto opt : db_options_type_info) { + if (opt.second.is_mutable && + opt.second.verification != OptionVerificationType::kDeprecated) { + mutable_map[opt.first] = options_map[opt.first]; + } + } + return mutable_map; + } + + std::unordered_map GetMutableCFOptionsMap( + const ColumnFamilyOptions& options) { + std::string options_str; + GetStringFromColumnFamilyOptions(&options_str, options); + std::unordered_map options_map; + StringToMap(options_str, &options_map); + std::unordered_map mutable_map; + for (const auto opt : cf_options_type_info) { + if (opt.second.is_mutable && + opt.second.verification != OptionVerificationType::kDeprecated) { + mutable_map[opt.first] = options_map[opt.first]; + } + } + return mutable_map; + } + + std::unordered_map GetRandomizedMutableCFOptionsMap( + Random* rnd) { + Options options; + options.env = env_; + ImmutableDBOptions db_options(options); + test::RandomInitCFOptions(&options, rnd); + auto sanitized_options = SanitizeOptions(db_options, options); + auto opt_map = GetMutableCFOptionsMap(sanitized_options); + delete options.compaction_filter; + return opt_map; + } + + std::unordered_map GetRandomizedMutableDBOptionsMap( + Random* rnd) { + DBOptions db_options; + test::RandomInitDBOptions(&db_options, rnd); + auto sanitized_options = SanitizeOptions(dbname_, db_options); + return GetMutableDBOptionsMap(sanitized_options); + } +#endif // ROCKSDB_LITE +}; + +// RocksDB lite don't support dynamic options. +#ifndef ROCKSDB_LITE + +TEST_F(DBOptionsTest, GetLatestDBOptions) { + // GetOptions should be able to get latest option changed by SetOptions. + Options options; + options.create_if_missing = true; + options.env = env_; + Random rnd(228); + Reopen(options); + auto new_options = GetRandomizedMutableDBOptionsMap(&rnd); + ASSERT_OK(dbfull()->SetDBOptions(new_options)); + ASSERT_EQ(new_options, GetMutableDBOptionsMap(dbfull()->GetDBOptions())); +} + +TEST_F(DBOptionsTest, GetLatestCFOptions) { + // GetOptions should be able to get latest option changed by SetOptions. + Options options; + options.create_if_missing = true; + options.env = env_; + Random rnd(228); + Reopen(options); + CreateColumnFamilies({"foo"}, options); + ReopenWithColumnFamilies({"default", "foo"}, options); + auto options_default = GetRandomizedMutableCFOptionsMap(&rnd); + auto options_foo = GetRandomizedMutableCFOptionsMap(&rnd); + ASSERT_OK(dbfull()->SetOptions(handles_[0], options_default)); + ASSERT_OK(dbfull()->SetOptions(handles_[1], options_foo)); + ASSERT_EQ(options_default, + GetMutableCFOptionsMap(dbfull()->GetOptions(handles_[0]))); + ASSERT_EQ(options_foo, + GetMutableCFOptionsMap(dbfull()->GetOptions(handles_[1]))); +} + +TEST_F(DBOptionsTest, SetOptionsAndReopen) { + Random rnd(1044); + auto rand_opts = GetRandomizedMutableCFOptionsMap(&rnd); + ASSERT_OK(dbfull()->SetOptions(rand_opts)); + // Verify if DB can be reopen after setting options. + Options options; + options.env = env_; + ASSERT_OK(TryReopen(options)); +} + +TEST_F(DBOptionsTest, EnableAutoCompactionAndTriggerStall) { + const std::string kValue(1024, 'v'); + for (int method_type = 0; method_type < 2; method_type++) { + for (int option_type = 0; option_type < 4; option_type++) { + Options options; + options.create_if_missing = true; + options.disable_auto_compactions = true; + options.write_buffer_size = 1024 * 1024 * 10; + options.compression = CompressionType::kNoCompression; + options.level0_file_num_compaction_trigger = 1; + options.level0_stop_writes_trigger = std::numeric_limits::max(); + options.level0_slowdown_writes_trigger = std::numeric_limits::max(); + options.hard_pending_compaction_bytes_limit = + std::numeric_limits::max(); + options.soft_pending_compaction_bytes_limit = + std::numeric_limits::max(); + options.env = env_; + + DestroyAndReopen(options); + int i = 0; + for (; i < 1024; i++) { + Put(Key(i), kValue); + } + Flush(); + for (; i < 1024 * 2; i++) { + Put(Key(i), kValue); + } + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(2, NumTableFilesAtLevel(0)); + uint64_t l0_size = SizeAtLevel(0); + + switch (option_type) { + case 0: + // test with level0_stop_writes_trigger + options.level0_stop_writes_trigger = 2; + options.level0_slowdown_writes_trigger = 2; + break; + case 1: + options.level0_slowdown_writes_trigger = 2; + break; + case 2: + options.hard_pending_compaction_bytes_limit = l0_size; + options.soft_pending_compaction_bytes_limit = l0_size; + break; + case 3: + options.soft_pending_compaction_bytes_limit = l0_size; + break; + } + Reopen(options); + dbfull()->TEST_WaitForCompact(); + ASSERT_FALSE(dbfull()->TEST_write_controler().IsStopped()); + ASSERT_FALSE(dbfull()->TEST_write_controler().NeedsDelay()); + + SyncPoint::GetInstance()->LoadDependency( + {{"DBOptionsTest::EnableAutoCompactionAndTriggerStall:1", + "BackgroundCallCompaction:0"}, + {"DBImpl::BackgroundCompaction():BeforePickCompaction", + "DBOptionsTest::EnableAutoCompactionAndTriggerStall:2"}, + {"DBOptionsTest::EnableAutoCompactionAndTriggerStall:3", + "DBImpl::BackgroundCompaction():AfterPickCompaction"}}); + // Block background compaction. + SyncPoint::GetInstance()->EnableProcessing(); + + switch (method_type) { + case 0: + ASSERT_OK( + dbfull()->SetOptions({{"disable_auto_compactions", "false"}})); + break; + case 1: + ASSERT_OK(dbfull()->EnableAutoCompaction( + {dbfull()->DefaultColumnFamily()})); + break; + } + TEST_SYNC_POINT("DBOptionsTest::EnableAutoCompactionAndTriggerStall:1"); + // Wait for stall condition recalculate. + TEST_SYNC_POINT("DBOptionsTest::EnableAutoCompactionAndTriggerStall:2"); + + switch (option_type) { + case 0: + ASSERT_TRUE(dbfull()->TEST_write_controler().IsStopped()); + break; + case 1: + ASSERT_FALSE(dbfull()->TEST_write_controler().IsStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + break; + case 2: + ASSERT_TRUE(dbfull()->TEST_write_controler().IsStopped()); + break; + case 3: + ASSERT_FALSE(dbfull()->TEST_write_controler().IsStopped()); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); + break; + } + TEST_SYNC_POINT("DBOptionsTest::EnableAutoCompactionAndTriggerStall:3"); + + // Background compaction executed. + dbfull()->TEST_WaitForCompact(); + ASSERT_FALSE(dbfull()->TEST_write_controler().IsStopped()); + ASSERT_FALSE(dbfull()->TEST_write_controler().NeedsDelay()); + } + } +} + +TEST_F(DBOptionsTest, SetOptionsMayTriggerCompaction) { + Options options; + options.create_if_missing = true; + options.level0_file_num_compaction_trigger = 1000; + options.env = env_; + Reopen(options); + for (int i = 0; i < 3; i++) { + // Need to insert two keys to avoid trivial move. + ASSERT_OK(Put("foo", ToString(i))); + ASSERT_OK(Put("bar", ToString(i))); + Flush(); + } + ASSERT_EQ("3", FilesPerLevel()); + ASSERT_OK( + dbfull()->SetOptions({{"level0_file_num_compaction_trigger", "3"}})); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ("0,1", FilesPerLevel()); +} + +TEST_F(DBOptionsTest, SetBackgroundCompactionThreads) { + Options options; + options.create_if_missing = true; + options.max_background_compactions = 1; // default value + options.env = env_; + Reopen(options); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + ASSERT_OK(dbfull()->SetDBOptions({{"max_background_compactions", "3"}})); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + auto stop_token = dbfull()->TEST_write_controler().GetStopToken(); + ASSERT_EQ(3, dbfull()->TEST_BGCompactionsAllowed()); +} + +TEST_F(DBOptionsTest, SetBackgroundJobs) { + Options options; + options.create_if_missing = true; + options.max_background_jobs = 8; + options.env = env_; + Reopen(options); + + for (int i = 0; i < 2; ++i) { + if (i > 0) { + options.max_background_jobs = 12; + ASSERT_OK(dbfull()->SetDBOptions( + {{"max_background_jobs", + std::to_string(options.max_background_jobs)}})); + } + + ASSERT_EQ(options.max_background_jobs / 4, + dbfull()->TEST_BGFlushesAllowed()); + ASSERT_EQ(1, dbfull()->TEST_BGCompactionsAllowed()); + + auto stop_token = dbfull()->TEST_write_controler().GetStopToken(); + + ASSERT_EQ(options.max_background_jobs / 4, + dbfull()->TEST_BGFlushesAllowed()); + ASSERT_EQ(3 * options.max_background_jobs / 4, + dbfull()->TEST_BGCompactionsAllowed()); + } +} + +TEST_F(DBOptionsTest, AvoidFlushDuringShutdown) { + Options options; + options.create_if_missing = true; + options.disable_auto_compactions = true; + options.env = env_; + WriteOptions write_without_wal; + write_without_wal.disableWAL = true; + + ASSERT_FALSE(options.avoid_flush_during_shutdown); + DestroyAndReopen(options); + ASSERT_OK(Put("foo", "v1", write_without_wal)); + Reopen(options); + ASSERT_EQ("v1", Get("foo")); + ASSERT_EQ("1", FilesPerLevel()); + + DestroyAndReopen(options); + ASSERT_OK(Put("foo", "v2", write_without_wal)); + ASSERT_OK(dbfull()->SetDBOptions({{"avoid_flush_during_shutdown", "true"}})); + Reopen(options); + ASSERT_EQ("NOT_FOUND", Get("foo")); + ASSERT_EQ("", FilesPerLevel()); +} + +TEST_F(DBOptionsTest, SetDelayedWriteRateOption) { + Options options; + options.create_if_missing = true; + options.delayed_write_rate = 2 * 1024U * 1024U; + options.env = env_; + Reopen(options); + ASSERT_EQ(2 * 1024U * 1024U, dbfull()->TEST_write_controler().max_delayed_write_rate()); + + ASSERT_OK(dbfull()->SetDBOptions({{"delayed_write_rate", "20000"}})); + ASSERT_EQ(20000, dbfull()->TEST_write_controler().max_delayed_write_rate()); +} + +TEST_F(DBOptionsTest, MaxTotalWalSizeChange) { + Random rnd(1044); + const auto value_size = size_t(1024); + std::string value; + test::RandomString(&rnd, value_size, &value); + + Options options; + options.create_if_missing = true; + options.env = env_; + CreateColumnFamilies({"1", "2", "3"}, options); + ReopenWithColumnFamilies({"default", "1", "2", "3"}, options); + + WriteOptions write_options; + + const int key_count = 100; + for (int i = 0; i < key_count; ++i) { + for (size_t cf = 0; cf < handles_.size(); ++cf) { + ASSERT_OK(Put(static_cast(cf), Key(i), value)); + } + } + ASSERT_OK(dbfull()->SetDBOptions({{"max_total_wal_size", "10"}})); + + for (size_t cf = 0; cf < handles_.size(); ++cf) { + dbfull()->TEST_WaitForFlushMemTable(handles_[cf]); + ASSERT_EQ("1", FilesPerLevel(static_cast(cf))); + } +} + +TEST_F(DBOptionsTest, SetStatsDumpPeriodSec) { + Options options; + options.create_if_missing = true; + options.stats_dump_period_sec = 5; + options.env = env_; + Reopen(options); + ASSERT_EQ(5, dbfull()->GetDBOptions().stats_dump_period_sec); + + for (int i = 0; i < 20; i++) { + int num = rand() % 5000 + 1; + ASSERT_OK(dbfull()->SetDBOptions( + {{"stats_dump_period_sec", std::to_string(num)}})); + ASSERT_EQ(num, dbfull()->GetDBOptions().stats_dump_period_sec); + } +} + +static void assert_candidate_files_empty(DBImpl* dbfull, const bool empty) { + dbfull->TEST_LockMutex(); + JobContext job_context(0); + dbfull->FindObsoleteFiles(&job_context, false); + ASSERT_EQ(empty, job_context.full_scan_candidate_files.empty()); + job_context.Clean(); + dbfull->TEST_UnlockMutex(); +} + +TEST_F(DBOptionsTest, DeleteObsoleteFilesPeriodChange) { + SpecialEnv env(env_); + env.time_elapse_only_sleep_ = true; + Options options; + options.env = &env; + options.create_if_missing = true; + ASSERT_OK(TryReopen(options)); + + // Verify that candidate files set is empty when no full scan requested. + assert_candidate_files_empty(dbfull(), true); + + ASSERT_OK( + dbfull()->SetDBOptions({{"delete_obsolete_files_period_micros", "0"}})); + + // After delete_obsolete_files_period_micros updated to 0, the next call + // to FindObsoleteFiles should make a full scan + assert_candidate_files_empty(dbfull(), false); + + ASSERT_OK( + dbfull()->SetDBOptions({{"delete_obsolete_files_period_micros", "20"}})); + + assert_candidate_files_empty(dbfull(), true); + + env.addon_time_.store(20); + assert_candidate_files_empty(dbfull(), true); + + env.addon_time_.store(21); + assert_candidate_files_empty(dbfull(), false); + + Close(); +} + +TEST_F(DBOptionsTest, MaxOpenFilesChange) { + SpecialEnv env(env_); + Options options; + options.env = CurrentOptions().env; + options.max_open_files = -1; + + Reopen(options); + + Cache* tc = dbfull()->TEST_table_cache(); + + ASSERT_EQ(-1, dbfull()->GetDBOptions().max_open_files); + ASSERT_LT(2000, tc->GetCapacity()); + ASSERT_OK(dbfull()->SetDBOptions({{"max_open_files", "1024"}})); + ASSERT_EQ(1024, dbfull()->GetDBOptions().max_open_files); + // examine the table cache (actual size should be 1014) + ASSERT_GT(1500, tc->GetCapacity()); + Close(); +} + +TEST_F(DBOptionsTest, SanitizeDelayedWriteRate) { + Options options; + options.delayed_write_rate = 0; + Reopen(options); + ASSERT_EQ(16 * 1024 * 1024, dbfull()->GetDBOptions().delayed_write_rate); + + options.rate_limiter.reset(NewGenericRateLimiter(31 * 1024 * 1024)); + Reopen(options); + ASSERT_EQ(31 * 1024 * 1024, dbfull()->GetDBOptions().delayed_write_rate); +} + +#endif // ROCKSDB_LITE + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_properties_test.cc b/db/db_properties_test.cc new file mode 100644 index 00000000000..0da64b13656 --- /dev/null +++ b/db/db_properties_test.cc @@ -0,0 +1,1393 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include + +#include +#include + +#include "db/db_test_util.h" +#include "port/stack_trace.h" +#include "rocksdb/options.h" +#include "rocksdb/perf_context.h" +#include "rocksdb/perf_level.h" +#include "rocksdb/table.h" +#include "util/random.h" +#include "util/string_util.h" + +namespace rocksdb { + +class DBPropertiesTest : public DBTestBase { + public: + DBPropertiesTest() : DBTestBase("/db_properties_test") {} +}; + +#ifndef ROCKSDB_LITE +TEST_F(DBPropertiesTest, Empty) { + do { + Options options; + options.env = env_; + options.write_buffer_size = 100000; // Small write buffer + options.allow_concurrent_memtable_write = false; + options = CurrentOptions(options); + CreateAndReopenWithCF({"pikachu"}, options); + + std::string num; + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], "rocksdb.num-entries-active-mem-table", &num)); + ASSERT_EQ("0", num); + + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], "rocksdb.num-entries-active-mem-table", &num)); + ASSERT_EQ("1", num); + + // Block sync calls + env_->delay_sstable_sync_.store(true, std::memory_order_release); + Put(1, "k1", std::string(100000, 'x')); // Fill memtable + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], "rocksdb.num-entries-active-mem-table", &num)); + ASSERT_EQ("2", num); + + Put(1, "k2", std::string(100000, 'y')); // Trigger compaction + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], "rocksdb.num-entries-active-mem-table", &num)); + ASSERT_EQ("1", num); + + ASSERT_EQ("v1", Get(1, "foo")); + // Release sync calls + env_->delay_sstable_sync_.store(false, std::memory_order_release); + + ASSERT_OK(db_->DisableFileDeletions()); + ASSERT_TRUE( + dbfull()->GetProperty("rocksdb.is-file-deletions-enabled", &num)); + ASSERT_EQ("1", num); + + ASSERT_OK(db_->DisableFileDeletions()); + ASSERT_TRUE( + dbfull()->GetProperty("rocksdb.is-file-deletions-enabled", &num)); + ASSERT_EQ("2", num); + + ASSERT_OK(db_->DisableFileDeletions()); + ASSERT_TRUE( + dbfull()->GetProperty("rocksdb.is-file-deletions-enabled", &num)); + ASSERT_EQ("3", num); + + ASSERT_OK(db_->EnableFileDeletions(false)); + ASSERT_TRUE( + dbfull()->GetProperty("rocksdb.is-file-deletions-enabled", &num)); + ASSERT_EQ("2", num); + + ASSERT_OK(db_->EnableFileDeletions()); + ASSERT_TRUE( + dbfull()->GetProperty("rocksdb.is-file-deletions-enabled", &num)); + ASSERT_EQ("0", num); + } while (ChangeOptions()); +} + +TEST_F(DBPropertiesTest, CurrentVersionNumber) { + uint64_t v1, v2, v3; + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.current-super-version-number", &v1)); + Put("12345678", ""); + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.current-super-version-number", &v2)); + Flush(); + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.current-super-version-number", &v3)); + + ASSERT_EQ(v1, v2); + ASSERT_GT(v3, v2); +} + +TEST_F(DBPropertiesTest, GetAggregatedIntPropertyTest) { + const int kKeySize = 100; + const int kValueSize = 500; + const int kKeyNum = 100; + + Options options; + options.env = env_; + options.create_if_missing = true; + options.write_buffer_size = (kKeySize + kValueSize) * kKeyNum / 10; + // Make them never flush + options.min_write_buffer_number_to_merge = 1000; + options.max_write_buffer_number = 1000; + options = CurrentOptions(options); + CreateAndReopenWithCF({"one", "two", "three", "four"}, options); + + Random rnd(301); + for (auto* handle : handles_) { + for (int i = 0; i < kKeyNum; ++i) { + db_->Put(WriteOptions(), handle, RandomString(&rnd, kKeySize), + RandomString(&rnd, kValueSize)); + } + } + + uint64_t manual_sum = 0; + uint64_t api_sum = 0; + uint64_t value = 0; + for (auto* handle : handles_) { + ASSERT_TRUE( + db_->GetIntProperty(handle, DB::Properties::kSizeAllMemTables, &value)); + manual_sum += value; + } + ASSERT_TRUE(db_->GetAggregatedIntProperty(DB::Properties::kSizeAllMemTables, + &api_sum)); + ASSERT_GT(manual_sum, 0); + ASSERT_EQ(manual_sum, api_sum); + + ASSERT_FALSE(db_->GetAggregatedIntProperty(DB::Properties::kDBStats, &value)); + + uint64_t before_flush_trm; + uint64_t after_flush_trm; + for (auto* handle : handles_) { + ASSERT_TRUE(db_->GetAggregatedIntProperty( + DB::Properties::kEstimateTableReadersMem, &before_flush_trm)); + + // Issue flush and expect larger memory usage of table readers. + db_->Flush(FlushOptions(), handle); + + ASSERT_TRUE(db_->GetAggregatedIntProperty( + DB::Properties::kEstimateTableReadersMem, &after_flush_trm)); + ASSERT_GT(after_flush_trm, before_flush_trm); + } +} + +namespace { +void ResetTableProperties(TableProperties* tp) { + tp->data_size = 0; + tp->index_size = 0; + tp->filter_size = 0; + tp->raw_key_size = 0; + tp->raw_value_size = 0; + tp->num_data_blocks = 0; + tp->num_entries = 0; +} + +void ParseTablePropertiesString(std::string tp_string, TableProperties* tp) { + double dummy_double; + std::replace(tp_string.begin(), tp_string.end(), ';', ' '); + std::replace(tp_string.begin(), tp_string.end(), '=', ' '); + ResetTableProperties(tp); + + sscanf(tp_string.c_str(), + "# data blocks %" SCNu64 " # entries %" SCNu64 " raw key size %" SCNu64 + " raw average key size %lf " + " raw value size %" SCNu64 + " raw average value size %lf " + " data block size %" SCNu64 " index block size %" SCNu64 + " filter block size %" SCNu64, + &tp->num_data_blocks, &tp->num_entries, &tp->raw_key_size, + &dummy_double, &tp->raw_value_size, &dummy_double, &tp->data_size, + &tp->index_size, &tp->filter_size); +} + +void VerifySimilar(uint64_t a, uint64_t b, double bias) { + ASSERT_EQ(a == 0U, b == 0U); + if (a == 0) { + return; + } + double dbl_a = static_cast(a); + double dbl_b = static_cast(b); + if (dbl_a > dbl_b) { + ASSERT_LT(static_cast(dbl_a - dbl_b) / (dbl_a + dbl_b), bias); + } else { + ASSERT_LT(static_cast(dbl_b - dbl_a) / (dbl_a + dbl_b), bias); + } +} + +void VerifyTableProperties(const TableProperties& base_tp, + const TableProperties& new_tp, + double filter_size_bias = 0.1, + double index_size_bias = 0.1, + double data_size_bias = 0.1, + double num_data_blocks_bias = 0.05) { + VerifySimilar(base_tp.data_size, new_tp.data_size, data_size_bias); + VerifySimilar(base_tp.index_size, new_tp.index_size, index_size_bias); + VerifySimilar(base_tp.filter_size, new_tp.filter_size, filter_size_bias); + VerifySimilar(base_tp.num_data_blocks, new_tp.num_data_blocks, + num_data_blocks_bias); + ASSERT_EQ(base_tp.raw_key_size, new_tp.raw_key_size); + ASSERT_EQ(base_tp.raw_value_size, new_tp.raw_value_size); + ASSERT_EQ(base_tp.num_entries, new_tp.num_entries); +} + +void GetExpectedTableProperties(TableProperties* expected_tp, + const int kKeySize, const int kValueSize, + const int kKeysPerTable, const int kTableCount, + const int kBloomBitsPerKey, + const size_t kBlockSize) { + const int kKeyCount = kTableCount * kKeysPerTable; + const int kAvgSuccessorSize = kKeySize / 5; + const int kEncodingSavePerKey = kKeySize / 4; + expected_tp->raw_key_size = kKeyCount * (kKeySize + 8); + expected_tp->raw_value_size = kKeyCount * kValueSize; + expected_tp->num_entries = kKeyCount; + expected_tp->num_data_blocks = + kTableCount * + (kKeysPerTable * (kKeySize - kEncodingSavePerKey + kValueSize)) / + kBlockSize; + expected_tp->data_size = + kTableCount * (kKeysPerTable * (kKeySize + 8 + kValueSize)); + expected_tp->index_size = + expected_tp->num_data_blocks * (kAvgSuccessorSize + 8); + expected_tp->filter_size = + kTableCount * (kKeysPerTable * kBloomBitsPerKey / 8); +} +} // anonymous namespace + +TEST_F(DBPropertiesTest, ValidatePropertyInfo) { + for (const auto& ppt_name_and_info : InternalStats::ppt_name_to_info) { + // If C++ gets a std::string_literal, this would be better to check at + // compile-time using static_assert. + ASSERT_TRUE(ppt_name_and_info.first.empty() || + !isdigit(ppt_name_and_info.first.back())); + + ASSERT_TRUE((ppt_name_and_info.second.handle_string == nullptr) != + (ppt_name_and_info.second.handle_int == nullptr)); + } +} + +TEST_F(DBPropertiesTest, ValidateSampleNumber) { + // When "max_open_files" is -1, we read all the files for + // "rocksdb.estimate-num-keys" computation, which is the ground truth. + // Otherwise, we sample 20 newest files to make an estimation. + // Formula: lastest_20_files_active_key_ratio * total_files + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.level0_stop_writes_trigger = 1000; + DestroyAndReopen(options); + int key = 0; + for (int files = 20; files >= 10; files -= 10) { + for (int i = 0; i < files; i++) { + int rows = files / 10; + for (int j = 0; j < rows; j++) { + db_->Put(WriteOptions(), std::to_string(++key), "foo"); + } + db_->Flush(FlushOptions()); + } + } + std::string num; + Reopen(options); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num)); + ASSERT_EQ("45", num); + options.max_open_files = -1; + Reopen(options); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num)); + ASSERT_EQ("50", num); +} + +TEST_F(DBPropertiesTest, AggregatedTableProperties) { + for (int kTableCount = 40; kTableCount <= 100; kTableCount += 30) { + const int kKeysPerTable = 100; + const int kKeySize = 80; + const int kValueSize = 200; + const int kBloomBitsPerKey = 20; + + Options options = CurrentOptions(); + options.level0_file_num_compaction_trigger = 8; + options.compression = kNoCompression; + options.create_if_missing = true; + + BlockBasedTableOptions table_options; + table_options.filter_policy.reset( + NewBloomFilterPolicy(kBloomBitsPerKey, false)); + table_options.block_size = 1024; + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + + DestroyAndReopen(options); + + Random rnd(5632); + for (int table = 1; table <= kTableCount; ++table) { + for (int i = 0; i < kKeysPerTable; ++i) { + db_->Put(WriteOptions(), RandomString(&rnd, kKeySize), + RandomString(&rnd, kValueSize)); + } + db_->Flush(FlushOptions()); + } + std::string property; + db_->GetProperty(DB::Properties::kAggregatedTableProperties, &property); + + TableProperties expected_tp; + GetExpectedTableProperties(&expected_tp, kKeySize, kValueSize, + kKeysPerTable, kTableCount, kBloomBitsPerKey, + table_options.block_size); + + TableProperties output_tp; + ParseTablePropertiesString(property, &output_tp); + + VerifyTableProperties(expected_tp, output_tp); + } +} + +TEST_F(DBPropertiesTest, ReadLatencyHistogramByLevel) { + Options options = CurrentOptions(); + options.write_buffer_size = 110 << 10; + options.level0_file_num_compaction_trigger = 6; + options.num_levels = 4; + options.compression = kNoCompression; + options.max_bytes_for_level_base = 4500 << 10; + options.target_file_size_base = 98 << 10; + options.max_write_buffer_number = 2; + options.statistics = rocksdb::CreateDBStatistics(); + options.max_open_files = 100; + + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + + CreateAndReopenWithCF({"pikachu"}, options); + int key_index = 0; + Random rnd(301); + for (int num = 0; num < 8; num++) { + Put("foo", "bar"); + GenerateNewFile(&rnd, &key_index); + dbfull()->TEST_WaitForCompact(); + } + dbfull()->TEST_WaitForCompact(); + + std::string prop; + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.dbstats", &prop)); + + // Get() after flushes, See latency histogram tracked. + for (int key = 0; key < key_index; key++) { + Get(Key(key)); + } + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.cfstats", &prop)); + ASSERT_NE(std::string::npos, prop.find("** Level 0 read latency histogram")); + ASSERT_NE(std::string::npos, prop.find("** Level 1 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 2 read latency histogram")); + + // Reopen and issue Get(). See thee latency tracked + ReopenWithColumnFamilies({"default", "pikachu"}, options); + dbfull()->TEST_WaitForCompact(); + for (int key = 0; key < key_index; key++) { + Get(Key(key)); + } + ASSERT_TRUE(dbfull()->GetProperty(dbfull()->DefaultColumnFamily(), + "rocksdb.cf-file-histogram", &prop)); + ASSERT_NE(std::string::npos, prop.find("** Level 0 read latency histogram")); + ASSERT_NE(std::string::npos, prop.find("** Level 1 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 2 read latency histogram")); + + // Reopen and issue iterating. See thee latency tracked + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.cf-file-histogram", &prop)); + ASSERT_EQ(std::string::npos, prop.find("** Level 0 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 1 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 2 read latency histogram")); + { + unique_ptr iter(db_->NewIterator(ReadOptions())); + for (iter->Seek(Key(0)); iter->Valid(); iter->Next()) { + } + } + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.cf-file-histogram", &prop)); + ASSERT_NE(std::string::npos, prop.find("** Level 0 read latency histogram")); + ASSERT_NE(std::string::npos, prop.find("** Level 1 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 2 read latency histogram")); + + // CF 1 should show no histogram. + ASSERT_TRUE( + dbfull()->GetProperty(handles_[1], "rocksdb.cf-file-histogram", &prop)); + ASSERT_EQ(std::string::npos, prop.find("** Level 0 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 1 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 2 read latency histogram")); + // put something and read it back , CF 1 should show histogram. + Put(1, "foo", "bar"); + Flush(1); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ("bar", Get(1, "foo")); + + ASSERT_TRUE( + dbfull()->GetProperty(handles_[1], "rocksdb.cf-file-histogram", &prop)); + ASSERT_NE(std::string::npos, prop.find("** Level 0 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 1 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 2 read latency histogram")); + + // options.max_open_files preloads table readers. + options.max_open_files = -1; + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_TRUE(dbfull()->GetProperty(dbfull()->DefaultColumnFamily(), + "rocksdb.cf-file-histogram", &prop)); + ASSERT_NE(std::string::npos, prop.find("** Level 0 read latency histogram")); + ASSERT_NE(std::string::npos, prop.find("** Level 1 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 2 read latency histogram")); + for (int key = 0; key < key_index; key++) { + Get(Key(key)); + } + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.cfstats", &prop)); + ASSERT_NE(std::string::npos, prop.find("** Level 0 read latency histogram")); + ASSERT_NE(std::string::npos, prop.find("** Level 1 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 2 read latency histogram")); + + // Clear internal stats + dbfull()->ResetStats(); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.cfstats", &prop)); + ASSERT_EQ(std::string::npos, prop.find("** Level 0 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 1 read latency histogram")); + ASSERT_EQ(std::string::npos, prop.find("** Level 2 read latency histogram")); +} + +TEST_F(DBPropertiesTest, AggregatedTablePropertiesAtLevel) { + const int kTableCount = 100; + const int kKeysPerTable = 10; + const int kKeySize = 50; + const int kValueSize = 400; + const int kMaxLevel = 7; + const int kBloomBitsPerKey = 20; + Random rnd(301); + Options options = CurrentOptions(); + options.level0_file_num_compaction_trigger = 8; + options.compression = kNoCompression; + options.create_if_missing = true; + options.level0_file_num_compaction_trigger = 2; + options.target_file_size_base = 8192; + options.max_bytes_for_level_base = 10000; + options.max_bytes_for_level_multiplier = 2; + // This ensures there no compaction happening when we call GetProperty(). + options.disable_auto_compactions = true; + + BlockBasedTableOptions table_options; + table_options.filter_policy.reset( + NewBloomFilterPolicy(kBloomBitsPerKey, false)); + table_options.block_size = 1024; + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + + DestroyAndReopen(options); + + std::string level_tp_strings[kMaxLevel]; + std::string tp_string; + TableProperties level_tps[kMaxLevel]; + TableProperties tp, sum_tp, expected_tp; + for (int table = 1; table <= kTableCount; ++table) { + for (int i = 0; i < kKeysPerTable; ++i) { + db_->Put(WriteOptions(), RandomString(&rnd, kKeySize), + RandomString(&rnd, kValueSize)); + } + db_->Flush(FlushOptions()); + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ResetTableProperties(&sum_tp); + for (int level = 0; level < kMaxLevel; ++level) { + db_->GetProperty( + DB::Properties::kAggregatedTablePropertiesAtLevel + ToString(level), + &level_tp_strings[level]); + ParseTablePropertiesString(level_tp_strings[level], &level_tps[level]); + sum_tp.data_size += level_tps[level].data_size; + sum_tp.index_size += level_tps[level].index_size; + sum_tp.filter_size += level_tps[level].filter_size; + sum_tp.raw_key_size += level_tps[level].raw_key_size; + sum_tp.raw_value_size += level_tps[level].raw_value_size; + sum_tp.num_data_blocks += level_tps[level].num_data_blocks; + sum_tp.num_entries += level_tps[level].num_entries; + } + db_->GetProperty(DB::Properties::kAggregatedTableProperties, &tp_string); + ParseTablePropertiesString(tp_string, &tp); + ASSERT_EQ(sum_tp.data_size, tp.data_size); + ASSERT_EQ(sum_tp.index_size, tp.index_size); + ASSERT_EQ(sum_tp.filter_size, tp.filter_size); + ASSERT_EQ(sum_tp.raw_key_size, tp.raw_key_size); + ASSERT_EQ(sum_tp.raw_value_size, tp.raw_value_size); + ASSERT_EQ(sum_tp.num_data_blocks, tp.num_data_blocks); + ASSERT_EQ(sum_tp.num_entries, tp.num_entries); + if (table > 3) { + GetExpectedTableProperties(&expected_tp, kKeySize, kValueSize, + kKeysPerTable, table, kBloomBitsPerKey, + table_options.block_size); + // Gives larger bias here as index block size, filter block size, + // and data block size become much harder to estimate in this test. + VerifyTableProperties(tp, expected_tp, 0.5, 0.4, 0.4, 0.25); + } + } +} + +TEST_F(DBPropertiesTest, NumImmutableMemTable) { + do { + Options options = CurrentOptions(); + WriteOptions writeOpt = WriteOptions(); + writeOpt.disableWAL = true; + options.max_write_buffer_number = 4; + options.min_write_buffer_number_to_merge = 3; + options.max_write_buffer_number_to_maintain = 4; + options.write_buffer_size = 1000000; + CreateAndReopenWithCF({"pikachu"}, options); + + std::string big_value(1000000 * 2, 'x'); + std::string num; + uint64_t value; + SetPerfLevel(kEnableTime); + ASSERT_TRUE(GetPerfLevel() == kEnableTime); + + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "k1", big_value)); + ASSERT_TRUE(dbfull()->GetProperty(handles_[1], + "rocksdb.num-immutable-mem-table", &num)); + ASSERT_EQ(num, "0"); + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], DB::Properties::kNumImmutableMemTableFlushed, &num)); + ASSERT_EQ(num, "0"); + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], "rocksdb.num-entries-active-mem-table", &num)); + ASSERT_EQ(num, "1"); + get_perf_context()->Reset(); + Get(1, "k1"); + ASSERT_EQ(1, static_cast(get_perf_context()->get_from_memtable_count)); + + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "k2", big_value)); + ASSERT_TRUE(dbfull()->GetProperty(handles_[1], + "rocksdb.num-immutable-mem-table", &num)); + ASSERT_EQ(num, "1"); + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], "rocksdb.num-entries-active-mem-table", &num)); + ASSERT_EQ(num, "1"); + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], "rocksdb.num-entries-imm-mem-tables", &num)); + ASSERT_EQ(num, "1"); + + get_perf_context()->Reset(); + Get(1, "k1"); + ASSERT_EQ(2, static_cast(get_perf_context()->get_from_memtable_count)); + get_perf_context()->Reset(); + Get(1, "k2"); + ASSERT_EQ(1, static_cast(get_perf_context()->get_from_memtable_count)); + + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "k3", big_value)); + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], "rocksdb.cur-size-active-mem-table", &num)); + ASSERT_TRUE(dbfull()->GetProperty(handles_[1], + "rocksdb.num-immutable-mem-table", &num)); + ASSERT_EQ(num, "2"); + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], "rocksdb.num-entries-active-mem-table", &num)); + ASSERT_EQ(num, "1"); + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], "rocksdb.num-entries-imm-mem-tables", &num)); + ASSERT_EQ(num, "2"); + get_perf_context()->Reset(); + Get(1, "k2"); + ASSERT_EQ(2, static_cast(get_perf_context()->get_from_memtable_count)); + get_perf_context()->Reset(); + Get(1, "k3"); + ASSERT_EQ(1, static_cast(get_perf_context()->get_from_memtable_count)); + get_perf_context()->Reset(); + Get(1, "k1"); + ASSERT_EQ(3, static_cast(get_perf_context()->get_from_memtable_count)); + + ASSERT_OK(Flush(1)); + ASSERT_TRUE(dbfull()->GetProperty(handles_[1], + "rocksdb.num-immutable-mem-table", &num)); + ASSERT_EQ(num, "0"); + ASSERT_TRUE(dbfull()->GetProperty( + handles_[1], DB::Properties::kNumImmutableMemTableFlushed, &num)); + ASSERT_EQ(num, "3"); + ASSERT_TRUE(dbfull()->GetIntProperty( + handles_[1], "rocksdb.cur-size-active-mem-table", &value)); + // "192" is the size of the metadata of two empty skiplists, this would + // break if we change the default skiplist implementation + ASSERT_GE(value, 192); + + uint64_t int_num; + uint64_t base_total_size; + ASSERT_TRUE(dbfull()->GetIntProperty( + handles_[1], "rocksdb.estimate-num-keys", &base_total_size)); + + ASSERT_OK(dbfull()->Delete(writeOpt, handles_[1], "k2")); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "k3", "")); + ASSERT_OK(dbfull()->Delete(writeOpt, handles_[1], "k3")); + ASSERT_TRUE(dbfull()->GetIntProperty( + handles_[1], "rocksdb.num-deletes-active-mem-table", &int_num)); + ASSERT_EQ(int_num, 2U); + ASSERT_TRUE(dbfull()->GetIntProperty( + handles_[1], "rocksdb.num-entries-active-mem-table", &int_num)); + ASSERT_EQ(int_num, 3U); + + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "k2", big_value)); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "k2", big_value)); + ASSERT_TRUE(dbfull()->GetIntProperty( + handles_[1], "rocksdb.num-entries-imm-mem-tables", &int_num)); + ASSERT_EQ(int_num, 4U); + ASSERT_TRUE(dbfull()->GetIntProperty( + handles_[1], "rocksdb.num-deletes-imm-mem-tables", &int_num)); + ASSERT_EQ(int_num, 2U); + + ASSERT_TRUE(dbfull()->GetIntProperty( + handles_[1], "rocksdb.estimate-num-keys", &int_num)); + ASSERT_EQ(int_num, base_total_size + 1); + + SetPerfLevel(kDisable); + ASSERT_TRUE(GetPerfLevel() == kDisable); + } while (ChangeCompactOptions()); +} + +// TODO(techdept) : Disabled flaky test #12863555 +TEST_F(DBPropertiesTest, DISABLED_GetProperty) { + // Set sizes to both background thread pool to be 1 and block them. + env_->SetBackgroundThreads(1, Env::HIGH); + env_->SetBackgroundThreads(1, Env::LOW); + test::SleepingBackgroundTask sleeping_task_low; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + test::SleepingBackgroundTask sleeping_task_high; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, + &sleeping_task_high, Env::Priority::HIGH); + + Options options = CurrentOptions(); + WriteOptions writeOpt = WriteOptions(); + writeOpt.disableWAL = true; + options.compaction_style = kCompactionStyleUniversal; + options.level0_file_num_compaction_trigger = 1; + options.compaction_options_universal.size_ratio = 50; + options.max_background_compactions = 1; + options.max_background_flushes = 1; + options.max_write_buffer_number = 10; + options.min_write_buffer_number_to_merge = 1; + options.max_write_buffer_number_to_maintain = 0; + options.write_buffer_size = 1000000; + Reopen(options); + + std::string big_value(1000000 * 2, 'x'); + std::string num; + uint64_t int_num; + SetPerfLevel(kEnableTime); + + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); + ASSERT_EQ(int_num, 0U); + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.estimate-live-data-size", &int_num)); + ASSERT_EQ(int_num, 0U); + + ASSERT_OK(dbfull()->Put(writeOpt, "k1", big_value)); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.num-immutable-mem-table", &num)); + ASSERT_EQ(num, "0"); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.mem-table-flush-pending", &num)); + ASSERT_EQ(num, "0"); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.compaction-pending", &num)); + ASSERT_EQ(num, "0"); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num)); + ASSERT_EQ(num, "1"); + get_perf_context()->Reset(); + + ASSERT_OK(dbfull()->Put(writeOpt, "k2", big_value)); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.num-immutable-mem-table", &num)); + ASSERT_EQ(num, "1"); + ASSERT_OK(dbfull()->Delete(writeOpt, "k-non-existing")); + ASSERT_OK(dbfull()->Put(writeOpt, "k3", big_value)); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.num-immutable-mem-table", &num)); + ASSERT_EQ(num, "2"); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.mem-table-flush-pending", &num)); + ASSERT_EQ(num, "1"); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.compaction-pending", &num)); + ASSERT_EQ(num, "0"); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num)); + ASSERT_EQ(num, "2"); + // Verify the same set of properties through GetIntProperty + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.num-immutable-mem-table", &int_num)); + ASSERT_EQ(int_num, 2U); + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.mem-table-flush-pending", &int_num)); + ASSERT_EQ(int_num, 1U); + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.compaction-pending", &int_num)); + ASSERT_EQ(int_num, 0U); + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.estimate-num-keys", &int_num)); + ASSERT_EQ(int_num, 2U); + + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); + ASSERT_EQ(int_num, 0U); + + sleeping_task_high.WakeUp(); + sleeping_task_high.WaitUntilDone(); + dbfull()->TEST_WaitForFlushMemTable(); + + ASSERT_OK(dbfull()->Put(writeOpt, "k4", big_value)); + ASSERT_OK(dbfull()->Put(writeOpt, "k5", big_value)); + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.mem-table-flush-pending", &num)); + ASSERT_EQ(num, "0"); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.compaction-pending", &num)); + ASSERT_EQ(num, "1"); + ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num)); + ASSERT_EQ(num, "4"); + + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); + ASSERT_GT(int_num, 0U); + + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilDone(); + + // Wait for compaction to be done. This is important because otherwise RocksDB + // might schedule a compaction when reopening the database, failing assertion + // (A) as a result. + dbfull()->TEST_WaitForCompact(); + options.max_open_files = 10; + Reopen(options); + // After reopening, no table reader is loaded, so no memory for table readers + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); + ASSERT_EQ(int_num, 0U); // (A) + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.estimate-num-keys", &int_num)); + ASSERT_GT(int_num, 0U); + + // After reading a key, at least one table reader is loaded. + Get("k5"); + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); + ASSERT_GT(int_num, 0U); + + // Test rocksdb.num-live-versions + { + options.level0_file_num_compaction_trigger = 20; + Reopen(options); + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.num-live-versions", &int_num)); + ASSERT_EQ(int_num, 1U); + + // Use an iterator to hold current version + std::unique_ptr iter1(dbfull()->NewIterator(ReadOptions())); + + ASSERT_OK(dbfull()->Put(writeOpt, "k6", big_value)); + Flush(); + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.num-live-versions", &int_num)); + ASSERT_EQ(int_num, 2U); + + // Use an iterator to hold current version + std::unique_ptr iter2(dbfull()->NewIterator(ReadOptions())); + + ASSERT_OK(dbfull()->Put(writeOpt, "k7", big_value)); + Flush(); + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.num-live-versions", &int_num)); + ASSERT_EQ(int_num, 3U); + + iter2.reset(); + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.num-live-versions", &int_num)); + ASSERT_EQ(int_num, 2U); + + iter1.reset(); + ASSERT_TRUE( + dbfull()->GetIntProperty("rocksdb.num-live-versions", &int_num)); + ASSERT_EQ(int_num, 1U); + } +} + +TEST_F(DBPropertiesTest, ApproximateMemoryUsage) { + const int kNumRounds = 10; + // TODO(noetzli) kFlushesPerRound does not really correlate with how many + // flushes happen. + const int kFlushesPerRound = 10; + const int kWritesPerFlush = 10; + const int kKeySize = 100; + const int kValueSize = 1000; + Options options; + options.write_buffer_size = 1000; // small write buffer + options.min_write_buffer_number_to_merge = 4; + options.compression = kNoCompression; + options.create_if_missing = true; + options = CurrentOptions(options); + DestroyAndReopen(options); + + Random rnd(301); + + std::vector iters; + + uint64_t active_mem; + uint64_t unflushed_mem; + uint64_t all_mem; + uint64_t prev_all_mem; + + // Phase 0. The verify the initial value of all these properties are the same + // as we have no mem-tables. + dbfull()->GetIntProperty("rocksdb.cur-size-active-mem-table", &active_mem); + dbfull()->GetIntProperty("rocksdb.cur-size-all-mem-tables", &unflushed_mem); + dbfull()->GetIntProperty("rocksdb.size-all-mem-tables", &all_mem); + ASSERT_EQ(all_mem, active_mem); + ASSERT_EQ(all_mem, unflushed_mem); + + // Phase 1. Simply issue Put() and expect "cur-size-all-mem-tables" equals to + // "size-all-mem-tables" + for (int r = 0; r < kNumRounds; ++r) { + for (int f = 0; f < kFlushesPerRound; ++f) { + for (int w = 0; w < kWritesPerFlush; ++w) { + Put(RandomString(&rnd, kKeySize), RandomString(&rnd, kValueSize)); + } + } + // Make sure that there is no flush between getting the two properties. + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->GetIntProperty("rocksdb.cur-size-all-mem-tables", &unflushed_mem); + dbfull()->GetIntProperty("rocksdb.size-all-mem-tables", &all_mem); + // in no iterator case, these two number should be the same. + ASSERT_EQ(unflushed_mem, all_mem); + } + prev_all_mem = all_mem; + + // Phase 2. Keep issuing Put() but also create new iterators. This time we + // expect "size-all-mem-tables" > "cur-size-all-mem-tables". + for (int r = 0; r < kNumRounds; ++r) { + iters.push_back(db_->NewIterator(ReadOptions())); + for (int f = 0; f < kFlushesPerRound; ++f) { + for (int w = 0; w < kWritesPerFlush; ++w) { + Put(RandomString(&rnd, kKeySize), RandomString(&rnd, kValueSize)); + } + } + // Force flush to prevent flush from happening between getting the + // properties or after getting the properties and before the new round. + Flush(); + + // In the second round, add iterators. + dbfull()->GetIntProperty("rocksdb.cur-size-active-mem-table", &active_mem); + dbfull()->GetIntProperty("rocksdb.cur-size-all-mem-tables", &unflushed_mem); + dbfull()->GetIntProperty("rocksdb.size-all-mem-tables", &all_mem); + ASSERT_GT(all_mem, active_mem); + ASSERT_GT(all_mem, unflushed_mem); + ASSERT_GT(all_mem, prev_all_mem); + prev_all_mem = all_mem; + } + + // Phase 3. Delete iterators and expect "size-all-mem-tables" shrinks + // whenever we release an iterator. + for (auto* iter : iters) { + delete iter; + dbfull()->GetIntProperty("rocksdb.size-all-mem-tables", &all_mem); + // Expect the size shrinking + ASSERT_LT(all_mem, prev_all_mem); + prev_all_mem = all_mem; + } + + // Expect all these three counters to be the same. + dbfull()->GetIntProperty("rocksdb.cur-size-active-mem-table", &active_mem); + dbfull()->GetIntProperty("rocksdb.cur-size-all-mem-tables", &unflushed_mem); + dbfull()->GetIntProperty("rocksdb.size-all-mem-tables", &all_mem); + ASSERT_EQ(active_mem, unflushed_mem); + ASSERT_EQ(unflushed_mem, all_mem); + + // Phase 5. Reopen, and expect all these three counters to be the same again. + Reopen(options); + dbfull()->GetIntProperty("rocksdb.cur-size-active-mem-table", &active_mem); + dbfull()->GetIntProperty("rocksdb.cur-size-all-mem-tables", &unflushed_mem); + dbfull()->GetIntProperty("rocksdb.size-all-mem-tables", &all_mem); + ASSERT_EQ(active_mem, unflushed_mem); + ASSERT_EQ(unflushed_mem, all_mem); +} + +TEST_F(DBPropertiesTest, EstimatePendingCompBytes) { + // Set sizes to both background thread pool to be 1 and block them. + env_->SetBackgroundThreads(1, Env::HIGH); + env_->SetBackgroundThreads(1, Env::LOW); + test::SleepingBackgroundTask sleeping_task_low; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + + Options options = CurrentOptions(); + WriteOptions writeOpt = WriteOptions(); + writeOpt.disableWAL = true; + options.compaction_style = kCompactionStyleLevel; + options.level0_file_num_compaction_trigger = 2; + options.max_background_compactions = 1; + options.max_background_flushes = 1; + options.max_write_buffer_number = 10; + options.min_write_buffer_number_to_merge = 1; + options.max_write_buffer_number_to_maintain = 0; + options.write_buffer_size = 1000000; + Reopen(options); + + std::string big_value(1000000 * 2, 'x'); + std::string num; + uint64_t int_num; + + ASSERT_OK(dbfull()->Put(writeOpt, "k1", big_value)); + Flush(); + ASSERT_TRUE(dbfull()->GetIntProperty( + "rocksdb.estimate-pending-compaction-bytes", &int_num)); + ASSERT_EQ(int_num, 0U); + + ASSERT_OK(dbfull()->Put(writeOpt, "k2", big_value)); + Flush(); + ASSERT_TRUE(dbfull()->GetIntProperty( + "rocksdb.estimate-pending-compaction-bytes", &int_num)); + ASSERT_GT(int_num, 0U); + + ASSERT_OK(dbfull()->Put(writeOpt, "k3", big_value)); + Flush(); + ASSERT_TRUE(dbfull()->GetIntProperty( + "rocksdb.estimate-pending-compaction-bytes", &int_num)); + ASSERT_GT(int_num, 0U); + + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilDone(); + + dbfull()->TEST_WaitForCompact(); + ASSERT_TRUE(dbfull()->GetIntProperty( + "rocksdb.estimate-pending-compaction-bytes", &int_num)); + ASSERT_EQ(int_num, 0U); +} + +TEST_F(DBPropertiesTest, EstimateCompressionRatio) { + if (!Snappy_Supported()) { + return; + } + const int kNumL0Files = 3; + const int kNumEntriesPerFile = 1000; + + Options options = CurrentOptions(); + options.compression_per_level = {kNoCompression, kSnappyCompression}; + options.disable_auto_compactions = true; + options.num_levels = 2; + Reopen(options); + + // compression ratio is -1.0 when no open files at level + ASSERT_EQ(CompressionRatioAtLevel(0), -1.0); + + const std::string kVal(100, 'a'); + for (int i = 0; i < kNumL0Files; ++i) { + for (int j = 0; j < kNumEntriesPerFile; ++j) { + // Put common data ("key") at end to prevent delta encoding from + // compressing the key effectively + std::string key = ToString(i) + ToString(j) + "key"; + ASSERT_OK(dbfull()->Put(WriteOptions(), key, kVal)); + } + Flush(); + } + + // no compression at L0, so ratio is less than one + ASSERT_LT(CompressionRatioAtLevel(0), 1.0); + ASSERT_GT(CompressionRatioAtLevel(0), 0.0); + ASSERT_EQ(CompressionRatioAtLevel(1), -1.0); + + dbfull()->TEST_CompactRange(0, nullptr, nullptr); + + ASSERT_EQ(CompressionRatioAtLevel(0), -1.0); + // Data at L1 should be highly compressed thanks to Snappy and redundant data + // in values (ratio is 12.846 as of 4/19/2016). + ASSERT_GT(CompressionRatioAtLevel(1), 10.0); +} + +#endif // ROCKSDB_LITE + +class CountingUserTblPropCollector : public TablePropertiesCollector { + public: + const char* Name() const override { return "CountingUserTblPropCollector"; } + + Status Finish(UserCollectedProperties* properties) override { + std::string encoded; + PutVarint32(&encoded, count_); + *properties = UserCollectedProperties{ + {"CountingUserTblPropCollector", message_}, {"Count", encoded}, + }; + return Status::OK(); + } + + Status AddUserKey(const Slice& user_key, const Slice& value, EntryType type, + SequenceNumber seq, uint64_t file_size) override { + ++count_; + return Status::OK(); + } + + virtual UserCollectedProperties GetReadableProperties() const override { + return UserCollectedProperties{}; + } + + private: + std::string message_ = "Rocksdb"; + uint32_t count_ = 0; +}; + +class CountingUserTblPropCollectorFactory + : public TablePropertiesCollectorFactory { + public: + explicit CountingUserTblPropCollectorFactory( + uint32_t expected_column_family_id) + : expected_column_family_id_(expected_column_family_id), + num_created_(0) {} + virtual TablePropertiesCollector* CreateTablePropertiesCollector( + TablePropertiesCollectorFactory::Context context) override { + EXPECT_EQ(expected_column_family_id_, context.column_family_id); + num_created_++; + return new CountingUserTblPropCollector(); + } + const char* Name() const override { + return "CountingUserTblPropCollectorFactory"; + } + void set_expected_column_family_id(uint32_t v) { + expected_column_family_id_ = v; + } + uint32_t expected_column_family_id_; + uint32_t num_created_; +}; + +class CountingDeleteTabPropCollector : public TablePropertiesCollector { + public: + const char* Name() const override { return "CountingDeleteTabPropCollector"; } + + Status AddUserKey(const Slice& user_key, const Slice& value, EntryType type, + SequenceNumber seq, uint64_t file_size) override { + if (type == kEntryDelete) { + num_deletes_++; + } + return Status::OK(); + } + + bool NeedCompact() const override { return num_deletes_ > 10; } + + UserCollectedProperties GetReadableProperties() const override { + return UserCollectedProperties{}; + } + + Status Finish(UserCollectedProperties* properties) override { + *properties = + UserCollectedProperties{{"num_delete", ToString(num_deletes_)}}; + return Status::OK(); + } + + private: + uint32_t num_deletes_ = 0; +}; + +class CountingDeleteTabPropCollectorFactory + : public TablePropertiesCollectorFactory { + public: + virtual TablePropertiesCollector* CreateTablePropertiesCollector( + TablePropertiesCollectorFactory::Context context) override { + return new CountingDeleteTabPropCollector(); + } + const char* Name() const override { + return "CountingDeleteTabPropCollectorFactory"; + } +}; + +#ifndef ROCKSDB_LITE +TEST_F(DBPropertiesTest, GetUserDefinedTableProperties) { + Options options = CurrentOptions(); + options.level0_file_num_compaction_trigger = (1 << 30); + options.table_properties_collector_factories.resize(1); + std::shared_ptr collector_factory = + std::make_shared(0); + options.table_properties_collector_factories[0] = collector_factory; + Reopen(options); + // Create 4 tables + for (int table = 0; table < 4; ++table) { + for (int i = 0; i < 10 + table; ++i) { + db_->Put(WriteOptions(), ToString(table * 100 + i), "val"); + } + db_->Flush(FlushOptions()); + } + + TablePropertiesCollection props; + ASSERT_OK(db_->GetPropertiesOfAllTables(&props)); + ASSERT_EQ(4U, props.size()); + uint32_t sum = 0; + for (const auto& item : props) { + auto& user_collected = item.second->user_collected_properties; + ASSERT_TRUE(user_collected.find("CountingUserTblPropCollector") != + user_collected.end()); + ASSERT_EQ(user_collected.at("CountingUserTblPropCollector"), "Rocksdb"); + ASSERT_TRUE(user_collected.find("Count") != user_collected.end()); + Slice key(user_collected.at("Count")); + uint32_t count; + ASSERT_TRUE(GetVarint32(&key, &count)); + sum += count; + } + ASSERT_EQ(10u + 11u + 12u + 13u, sum); + + ASSERT_GT(collector_factory->num_created_, 0U); + collector_factory->num_created_ = 0; + dbfull()->TEST_CompactRange(0, nullptr, nullptr); + ASSERT_GT(collector_factory->num_created_, 0U); +} +#endif // ROCKSDB_LITE + +TEST_F(DBPropertiesTest, UserDefinedTablePropertiesContext) { + Options options = CurrentOptions(); + options.level0_file_num_compaction_trigger = 3; + options.table_properties_collector_factories.resize(1); + std::shared_ptr collector_factory = + std::make_shared(1); + options.table_properties_collector_factories[0] = collector_factory, + CreateAndReopenWithCF({"pikachu"}, options); + // Create 2 files + for (int table = 0; table < 2; ++table) { + for (int i = 0; i < 10 + table; ++i) { + Put(1, ToString(table * 100 + i), "val"); + } + Flush(1); + } + ASSERT_GT(collector_factory->num_created_, 0U); + + collector_factory->num_created_ = 0; + // Trigger automatic compactions. + for (int table = 0; table < 3; ++table) { + for (int i = 0; i < 10 + table; ++i) { + Put(1, ToString(table * 100 + i), "val"); + } + Flush(1); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_GT(collector_factory->num_created_, 0U); + + collector_factory->num_created_ = 0; + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); + ASSERT_GT(collector_factory->num_created_, 0U); + + // Come back to write to default column family + collector_factory->num_created_ = 0; + collector_factory->set_expected_column_family_id(0); // default CF + // Create 4 tables in default column family + for (int table = 0; table < 2; ++table) { + for (int i = 0; i < 10 + table; ++i) { + Put(ToString(table * 100 + i), "val"); + } + Flush(); + } + ASSERT_GT(collector_factory->num_created_, 0U); + + collector_factory->num_created_ = 0; + // Trigger automatic compactions. + for (int table = 0; table < 3; ++table) { + for (int i = 0; i < 10 + table; ++i) { + Put(ToString(table * 100 + i), "val"); + } + Flush(); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_GT(collector_factory->num_created_, 0U); + + collector_factory->num_created_ = 0; + dbfull()->TEST_CompactRange(0, nullptr, nullptr); + ASSERT_GT(collector_factory->num_created_, 0U); +} + +#ifndef ROCKSDB_LITE +TEST_F(DBPropertiesTest, TablePropertiesNeedCompactTest) { + Random rnd(301); + + Options options; + options.create_if_missing = true; + options.write_buffer_size = 4096; + options.max_write_buffer_number = 8; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 2; + options.level0_stop_writes_trigger = 4; + options.target_file_size_base = 2048; + options.max_bytes_for_level_base = 10240; + options.max_bytes_for_level_multiplier = 4; + options.soft_pending_compaction_bytes_limit = 1024 * 1024; + options.num_levels = 8; + options.env = env_; + + std::shared_ptr collector_factory = + std::make_shared(); + options.table_properties_collector_factories.resize(1); + options.table_properties_collector_factories[0] = collector_factory; + + DestroyAndReopen(options); + + const int kMaxKey = 1000; + for (int i = 0; i < kMaxKey; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 102))); + ASSERT_OK(Put(Key(kMaxKey + i), RandomString(&rnd, 102))); + } + Flush(); + dbfull()->TEST_WaitForCompact(); + if (NumTableFilesAtLevel(0) == 1) { + // Clear Level 0 so that when later flush a file with deletions, + // we don't trigger an organic compaction. + ASSERT_OK(Put(Key(0), "")); + ASSERT_OK(Put(Key(kMaxKey * 2), "")); + Flush(); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + + { + int c = 0; + std::unique_ptr iter(db_->NewIterator(ReadOptions())); + iter->Seek(Key(kMaxKey - 100)); + while (iter->Valid() && iter->key().compare(Key(kMaxKey + 100)) < 0) { + iter->Next(); + ++c; + } + ASSERT_EQ(c, 200); + } + + Delete(Key(0)); + for (int i = kMaxKey - 100; i < kMaxKey + 100; i++) { + Delete(Key(i)); + } + Delete(Key(kMaxKey * 2)); + + Flush(); + dbfull()->TEST_WaitForCompact(); + + { + SetPerfLevel(kEnableCount); + get_perf_context()->Reset(); + int c = 0; + std::unique_ptr iter(db_->NewIterator(ReadOptions())); + iter->Seek(Key(kMaxKey - 100)); + while (iter->Valid() && iter->key().compare(Key(kMaxKey + 100)) < 0) { + iter->Next(); + } + ASSERT_EQ(c, 0); + ASSERT_LT(get_perf_context()->internal_delete_skipped_count, 30u); + ASSERT_LT(get_perf_context()->internal_key_skipped_count, 30u); + SetPerfLevel(kDisable); + } +} + +TEST_F(DBPropertiesTest, NeedCompactHintPersistentTest) { + Random rnd(301); + + Options options; + options.create_if_missing = true; + options.max_write_buffer_number = 8; + options.level0_file_num_compaction_trigger = 10; + options.level0_slowdown_writes_trigger = 10; + options.level0_stop_writes_trigger = 10; + options.disable_auto_compactions = true; + options.env = env_; + + std::shared_ptr collector_factory = + std::make_shared(); + options.table_properties_collector_factories.resize(1); + options.table_properties_collector_factories[0] = collector_factory; + + DestroyAndReopen(options); + + const int kMaxKey = 100; + for (int i = 0; i < kMaxKey; i++) { + ASSERT_OK(Put(Key(i), "")); + } + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + + for (int i = 1; i < kMaxKey - 1; i++) { + Delete(Key(i)); + } + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(NumTableFilesAtLevel(0), 2); + + // Restart the DB. Although number of files didn't reach + // options.level0_file_num_compaction_trigger, compaction should + // still be triggered because of the need-compaction hint. + options.disable_auto_compactions = false; + Reopen(options); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + { + SetPerfLevel(kEnableCount); + get_perf_context()->Reset(); + int c = 0; + std::unique_ptr iter(db_->NewIterator(ReadOptions())); + for (iter->Seek(Key(0)); iter->Valid(); iter->Next()) { + c++; + } + ASSERT_EQ(c, 2); + ASSERT_EQ(get_perf_context()->internal_delete_skipped_count, 0); + // We iterate every key twice. Is it a bug? + ASSERT_LE(get_perf_context()->internal_key_skipped_count, 2); + SetPerfLevel(kDisable); + } +} + +TEST_F(DBPropertiesTest, EstimateNumKeysUnderflow) { + Options options; + Reopen(options); + Put("foo", "bar"); + Delete("foo"); + Delete("foo"); + uint64_t num_keys = 0; + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.estimate-num-keys", &num_keys)); + ASSERT_EQ(0, num_keys); +} + +TEST_F(DBPropertiesTest, EstimateOldestKeyTime) { + std::unique_ptr mock_env(new MockTimeEnv(Env::Default())); + uint64_t oldest_key_time = 0; + Options options; + options.env = mock_env.get(); + + // "rocksdb.estimate-oldest-key-time" only available to fifo compaction. + mock_env->set_current_time(100); + for (auto compaction : {kCompactionStyleLevel, kCompactionStyleUniversal, + kCompactionStyleNone}) { + options.compaction_style = compaction; + options.create_if_missing = true; + DestroyAndReopen(options); + ASSERT_OK(Put("foo", "bar")); + ASSERT_FALSE(dbfull()->GetIntProperty( + DB::Properties::kEstimateOldestKeyTime, &oldest_key_time)); + } + + options.compaction_style = kCompactionStyleFIFO; + options.compaction_options_fifo.ttl = 300; + options.compaction_options_fifo.allow_compaction = false; + DestroyAndReopen(options); + + mock_env->set_current_time(100); + ASSERT_OK(Put("k1", "v1")); + ASSERT_TRUE(dbfull()->GetIntProperty(DB::Properties::kEstimateOldestKeyTime, + &oldest_key_time)); + ASSERT_EQ(100, oldest_key_time); + ASSERT_OK(Flush()); + ASSERT_EQ("1", FilesPerLevel()); + ASSERT_TRUE(dbfull()->GetIntProperty(DB::Properties::kEstimateOldestKeyTime, + &oldest_key_time)); + ASSERT_EQ(100, oldest_key_time); + + mock_env->set_current_time(200); + ASSERT_OK(Put("k2", "v2")); + ASSERT_OK(Flush()); + ASSERT_EQ("2", FilesPerLevel()); + ASSERT_TRUE(dbfull()->GetIntProperty(DB::Properties::kEstimateOldestKeyTime, + &oldest_key_time)); + ASSERT_EQ(100, oldest_key_time); + + mock_env->set_current_time(300); + ASSERT_OK(Put("k3", "v3")); + ASSERT_OK(Flush()); + ASSERT_EQ("3", FilesPerLevel()); + ASSERT_TRUE(dbfull()->GetIntProperty(DB::Properties::kEstimateOldestKeyTime, + &oldest_key_time)); + ASSERT_EQ(100, oldest_key_time); + + mock_env->set_current_time(450); + ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ("2", FilesPerLevel()); + ASSERT_TRUE(dbfull()->GetIntProperty(DB::Properties::kEstimateOldestKeyTime, + &oldest_key_time)); + ASSERT_EQ(200, oldest_key_time); + + mock_env->set_current_time(550); + ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ("1", FilesPerLevel()); + ASSERT_TRUE(dbfull()->GetIntProperty(DB::Properties::kEstimateOldestKeyTime, + &oldest_key_time)); + ASSERT_EQ(300, oldest_key_time); + + mock_env->set_current_time(650); + ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ("", FilesPerLevel()); + ASSERT_FALSE(dbfull()->GetIntProperty(DB::Properties::kEstimateOldestKeyTime, + &oldest_key_time)); + + // Close before mock_env destructs. + Close(); +} + +#endif // ROCKSDB_LITE +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_range_del_test.cc b/db/db_range_del_test.cc new file mode 100644 index 00000000000..982cbb85ab2 --- /dev/null +++ b/db/db_range_del_test.cc @@ -0,0 +1,1007 @@ +// Copyright (c) 2016-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/db_test_util.h" +#include "port/stack_trace.h" +#include "util/testutil.h" +#include "utilities/merge_operators.h" + +namespace rocksdb { + +class DBRangeDelTest : public DBTestBase { + public: + DBRangeDelTest() : DBTestBase("/db_range_del_test") {} + + std::string GetNumericStr(int key) { + uint64_t uint64_key = static_cast(key); + std::string str; + str.resize(8); + memcpy(&str[0], static_cast(&uint64_key), 8); + return str; + } +}; + +// PlainTableFactory and NumTableFilesAtLevel() are not supported in +// ROCKSDB_LITE +#ifndef ROCKSDB_LITE +TEST_F(DBRangeDelTest, NonBlockBasedTableNotSupported) { + if (!IsMemoryMappedAccessSupported()) { + return; + } + Options opts = CurrentOptions(); + opts.table_factory.reset(new PlainTableFactory()); + opts.prefix_extractor.reset(NewNoopTransform()); + opts.allow_mmap_reads = true; + opts.max_sequential_skip_in_iterations = 999999; + Reopen(opts); + + ASSERT_TRUE( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "dr1", "dr1") + .IsNotSupported()); +} + +TEST_F(DBRangeDelTest, FlushOutputHasOnlyRangeTombstones) { + ASSERT_OK(db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "dr1", + "dr2")); + ASSERT_OK(db_->Flush(FlushOptions())); + ASSERT_EQ(1, NumTableFilesAtLevel(0)); +} + +TEST_F(DBRangeDelTest, CompactionOutputHasOnlyRangeTombstone) { + Options opts = CurrentOptions(); + opts.disable_auto_compactions = true; + opts.statistics = CreateDBStatistics(); + Reopen(opts); + + // snapshot protects range tombstone from dropping due to becoming obsolete. + const Snapshot* snapshot = db_->GetSnapshot(); + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z"); + db_->Flush(FlushOptions()); + + ASSERT_EQ(1, NumTableFilesAtLevel(0)); + ASSERT_EQ(0, NumTableFilesAtLevel(1)); + dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, + true /* disallow_trivial_move */); + ASSERT_EQ(0, NumTableFilesAtLevel(0)); + ASSERT_EQ(1, NumTableFilesAtLevel(1)); + ASSERT_EQ(0, TestGetTickerCount(opts, COMPACTION_RANGE_DEL_DROP_OBSOLETE)); + db_->ReleaseSnapshot(snapshot); +} + +TEST_F(DBRangeDelTest, CompactionOutputFilesExactlyFilled) { + // regression test for exactly filled compaction output files. Previously + // another file would be generated containing all range deletions, which + // could invalidate the non-overlapping file boundary invariant. + const int kNumPerFile = 4, kNumFiles = 2, kFileBytes = 9 << 10; + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.level0_file_num_compaction_trigger = kNumFiles; + options.memtable_factory.reset(new SpecialSkipListFactory(kNumPerFile)); + options.num_levels = 2; + options.target_file_size_base = kFileBytes; + BlockBasedTableOptions table_options; + table_options.block_size_deviation = 50; // each block holds two keys + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + Reopen(options); + + // snapshot protects range tombstone from dropping due to becoming obsolete. + const Snapshot* snapshot = db_->GetSnapshot(); + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), Key(0), Key(1)); + + Random rnd(301); + for (int i = 0; i < kNumFiles; ++i) { + std::vector values; + // Write 12K (4 values, each 3K) + for (int j = 0; j < kNumPerFile; j++) { + values.push_back(RandomString(&rnd, 3 << 10)); + ASSERT_OK(Put(Key(i * kNumPerFile + j), values[j])); + if (j == 0 && i > 0) { + dbfull()->TEST_WaitForFlushMemTable(); + } + } + } + // put extra key to trigger final flush + ASSERT_OK(Put("", "")); + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(kNumFiles, NumTableFilesAtLevel(0)); + ASSERT_EQ(0, NumTableFilesAtLevel(1)); + + dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, + true /* disallow_trivial_move */); + ASSERT_EQ(0, NumTableFilesAtLevel(0)); + ASSERT_EQ(2, NumTableFilesAtLevel(1)); + db_->ReleaseSnapshot(snapshot); +} + +TEST_F(DBRangeDelTest, MaxCompactionBytesCutsOutputFiles) { + // Ensures range deletion spanning multiple compaction output files that are + // cut by max_compaction_bytes will have non-overlapping key-ranges. + // https://github.com/facebook/rocksdb/issues/1778 + const int kNumFiles = 2, kNumPerFile = 1 << 8, kBytesPerVal = 1 << 12; + Options opts = CurrentOptions(); + opts.comparator = test::Uint64Comparator(); + opts.disable_auto_compactions = true; + opts.level0_file_num_compaction_trigger = kNumFiles; + opts.max_compaction_bytes = kNumPerFile * kBytesPerVal; + opts.memtable_factory.reset(new SpecialSkipListFactory(kNumPerFile)); + // Want max_compaction_bytes to trigger the end of compaction output file, not + // target_file_size_base, so make the latter much bigger + opts.target_file_size_base = 100 * opts.max_compaction_bytes; + Reopen(opts); + + // snapshot protects range tombstone from dropping due to becoming obsolete. + const Snapshot* snapshot = db_->GetSnapshot(); + + // It spans the whole key-range, thus will be included in all output files + ASSERT_OK(db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + GetNumericStr(0), + GetNumericStr(kNumFiles * kNumPerFile - 1))); + Random rnd(301); + for (int i = 0; i < kNumFiles; ++i) { + std::vector values; + // Write 1MB (256 values, each 4K) + for (int j = 0; j < kNumPerFile; j++) { + values.push_back(RandomString(&rnd, kBytesPerVal)); + ASSERT_OK(Put(GetNumericStr(kNumPerFile * i + j), values[j])); + } + // extra entry to trigger SpecialSkipListFactory's flush + ASSERT_OK(Put(GetNumericStr(kNumPerFile), "")); + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(i + 1, NumTableFilesAtLevel(0)); + } + + dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, + true /* disallow_trivial_move */); + ASSERT_EQ(0, NumTableFilesAtLevel(0)); + ASSERT_GE(NumTableFilesAtLevel(1), 2); + + std::vector> files; + dbfull()->TEST_GetFilesMetaData(db_->DefaultColumnFamily(), &files); + + for (size_t i = 0; i < files[1].size() - 1; ++i) { + ASSERT_TRUE(InternalKeyComparator(opts.comparator) + .Compare(files[1][i].largest, files[1][i + 1].smallest) < + 0); + } + db_->ReleaseSnapshot(snapshot); +} + +TEST_F(DBRangeDelTest, SentinelsOmittedFromOutputFile) { + // Regression test for bug where sentinel range deletions (i.e., ones with + // sequence number of zero) were included in output files. + // snapshot protects range tombstone from dropping due to becoming obsolete. + const Snapshot* snapshot = db_->GetSnapshot(); + + // gaps between ranges creates sentinels in our internal representation + std::vector> range_dels = {{"a", "b"}, {"c", "d"}, {"e", "f"}}; + for (const auto& range_del : range_dels) { + ASSERT_OK(db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + range_del.first, range_del.second)); + } + ASSERT_OK(db_->Flush(FlushOptions())); + ASSERT_EQ(1, NumTableFilesAtLevel(0)); + + std::vector> files; + dbfull()->TEST_GetFilesMetaData(db_->DefaultColumnFamily(), &files); + ASSERT_GT(files[0][0].smallest_seqno, 0); + + db_->ReleaseSnapshot(snapshot); +} + +TEST_F(DBRangeDelTest, FlushRangeDelsSameStartKey) { + db_->Put(WriteOptions(), "b1", "val"); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "c")); + db_->Put(WriteOptions(), "b2", "val"); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "b")); + // first iteration verifies query correctness in memtable, second verifies + // query correctness for a single SST file + for (int i = 0; i < 2; ++i) { + if (i > 0) { + ASSERT_OK(db_->Flush(FlushOptions())); + ASSERT_EQ(1, NumTableFilesAtLevel(0)); + } + std::string value; + ASSERT_TRUE(db_->Get(ReadOptions(), "b1", &value).IsNotFound()); + ASSERT_OK(db_->Get(ReadOptions(), "b2", &value)); + } +} + +TEST_F(DBRangeDelTest, CompactRangeDelsSameStartKey) { + db_->Put(WriteOptions(), "unused", "val"); // prevents empty after compaction + db_->Put(WriteOptions(), "b1", "val"); + ASSERT_OK(db_->Flush(FlushOptions())); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "c")); + ASSERT_OK(db_->Flush(FlushOptions())); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "b")); + ASSERT_OK(db_->Flush(FlushOptions())); + ASSERT_EQ(3, NumTableFilesAtLevel(0)); + + for (int i = 0; i < 2; ++i) { + if (i > 0) { + dbfull()->TEST_CompactRange(0, nullptr, nullptr, nullptr, + true /* disallow_trivial_move */); + ASSERT_EQ(0, NumTableFilesAtLevel(0)); + ASSERT_EQ(1, NumTableFilesAtLevel(1)); + } + std::string value; + ASSERT_TRUE(db_->Get(ReadOptions(), "b1", &value).IsNotFound()); + } +} +#endif // ROCKSDB_LITE + +TEST_F(DBRangeDelTest, FlushRemovesCoveredKeys) { + const int kNum = 300, kRangeBegin = 50, kRangeEnd = 250; + Options opts = CurrentOptions(); + opts.comparator = test::Uint64Comparator(); + Reopen(opts); + + // Write a third before snapshot, a third between snapshot and tombstone, and + // a third after the tombstone. Keys older than snapshot or newer than the + // tombstone should be preserved. + const Snapshot* snapshot = nullptr; + for (int i = 0; i < kNum; ++i) { + if (i == kNum / 3) { + snapshot = db_->GetSnapshot(); + } else if (i == 2 * kNum / 3) { + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + GetNumericStr(kRangeBegin), GetNumericStr(kRangeEnd)); + } + db_->Put(WriteOptions(), GetNumericStr(i), "val"); + } + db_->Flush(FlushOptions()); + + for (int i = 0; i < kNum; ++i) { + ReadOptions read_opts; + read_opts.ignore_range_deletions = true; + std::string value; + if (i < kRangeBegin || i > kRangeEnd || i < kNum / 3 || i >= 2 * kNum / 3) { + ASSERT_OK(db_->Get(read_opts, GetNumericStr(i), &value)); + } else { + ASSERT_TRUE(db_->Get(read_opts, GetNumericStr(i), &value).IsNotFound()); + } + } + db_->ReleaseSnapshot(snapshot); +} + +// NumTableFilesAtLevel() is not supported in ROCKSDB_LITE +#ifndef ROCKSDB_LITE +TEST_F(DBRangeDelTest, CompactionRemovesCoveredKeys) { + const int kNumPerFile = 100, kNumFiles = 4; + Options opts = CurrentOptions(); + opts.comparator = test::Uint64Comparator(); + opts.disable_auto_compactions = true; + opts.memtable_factory.reset(new SpecialSkipListFactory(kNumPerFile)); + opts.num_levels = 2; + opts.statistics = CreateDBStatistics(); + Reopen(opts); + + for (int i = 0; i < kNumFiles; ++i) { + if (i > 0) { + // range tombstone covers first half of the previous file + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + GetNumericStr((i - 1) * kNumPerFile), + GetNumericStr((i - 1) * kNumPerFile + kNumPerFile / 2)); + } + // Make sure a given key appears in each file so compaction won't be able to + // use trivial move, which would happen if the ranges were non-overlapping. + // Also, we need an extra element since flush is only triggered when the + // number of keys is one greater than SpecialSkipListFactory's limit. + // We choose a key outside the key-range used by the test to avoid conflict. + db_->Put(WriteOptions(), GetNumericStr(kNumPerFile * kNumFiles), "val"); + + for (int j = 0; j < kNumPerFile; ++j) { + db_->Put(WriteOptions(), GetNumericStr(i * kNumPerFile + j), "val"); + } + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(i + 1, NumTableFilesAtLevel(0)); + } + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ(0, NumTableFilesAtLevel(0)); + ASSERT_GT(NumTableFilesAtLevel(1), 0); + ASSERT_EQ((kNumFiles - 1) * kNumPerFile / 2, + TestGetTickerCount(opts, COMPACTION_KEY_DROP_RANGE_DEL)); + + for (int i = 0; i < kNumFiles; ++i) { + for (int j = 0; j < kNumPerFile; ++j) { + ReadOptions read_opts; + read_opts.ignore_range_deletions = true; + std::string value; + if (i == kNumFiles - 1 || j >= kNumPerFile / 2) { + ASSERT_OK( + db_->Get(read_opts, GetNumericStr(i * kNumPerFile + j), &value)); + } else { + ASSERT_TRUE( + db_->Get(read_opts, GetNumericStr(i * kNumPerFile + j), &value) + .IsNotFound()); + } + } + } +} + +TEST_F(DBRangeDelTest, ValidLevelSubcompactionBoundaries) { + const int kNumPerFile = 100, kNumFiles = 4, kFileBytes = 100 << 10; + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.level0_file_num_compaction_trigger = kNumFiles; + options.max_bytes_for_level_base = 2 * kFileBytes; + options.max_subcompactions = 4; + options.memtable_factory.reset(new SpecialSkipListFactory(kNumPerFile)); + options.num_levels = 3; + options.target_file_size_base = kFileBytes; + options.target_file_size_multiplier = 1; + Reopen(options); + + Random rnd(301); + for (int i = 0; i < 2; ++i) { + for (int j = 0; j < kNumFiles; ++j) { + if (i > 0) { + // delete [95,105) in two files, [295,305) in next two + int mid = (j + (1 - j % 2)) * kNumPerFile; + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + Key(mid - 5), Key(mid + 5)); + } + std::vector values; + // Write 100KB (100 values, each 1K) + for (int k = 0; k < kNumPerFile; k++) { + values.push_back(RandomString(&rnd, 990)); + ASSERT_OK(Put(Key(j * kNumPerFile + k), values[k])); + } + // put extra key to trigger flush + ASSERT_OK(Put("", "")); + dbfull()->TEST_WaitForFlushMemTable(); + if (j < kNumFiles - 1) { + // background compaction may happen early for kNumFiles'th file + ASSERT_EQ(NumTableFilesAtLevel(0), j + 1); + } + if (j == options.level0_file_num_compaction_trigger - 1) { + // When i == 1, compaction will output some files to L1, at which point + // L1 is not bottommost so range deletions cannot be compacted away. The + // new L1 files must be generated with non-overlapping key ranges even + // though multiple subcompactions see the same ranges deleted, else an + // assertion will fail. + // + // Only enable auto-compactions when we're ready; otherwise, the + // oversized L0 (relative to base_level) causes the compaction to run + // earlier. + ASSERT_OK(db_->EnableAutoCompaction({db_->DefaultColumnFamily()})); + dbfull()->TEST_WaitForCompact(); + ASSERT_OK(db_->SetOptions(db_->DefaultColumnFamily(), + {{"disable_auto_compactions", "true"}})); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + ASSERT_GT(NumTableFilesAtLevel(1), 0); + ASSERT_GT(NumTableFilesAtLevel(2), 0); + } + } + } +} + +TEST_F(DBRangeDelTest, ValidUniversalSubcompactionBoundaries) { + const int kNumPerFile = 100, kFilesPerLevel = 4, kNumLevels = 4; + Options options = CurrentOptions(); + options.compaction_options_universal.min_merge_width = kFilesPerLevel; + options.compaction_options_universal.max_merge_width = kFilesPerLevel; + options.compaction_options_universal.size_ratio = 10; + options.compaction_style = kCompactionStyleUniversal; + options.level0_file_num_compaction_trigger = kFilesPerLevel; + options.max_subcompactions = 4; + options.memtable_factory.reset(new SpecialSkipListFactory(kNumPerFile)); + options.num_levels = kNumLevels; + options.target_file_size_base = kNumPerFile << 10; + options.target_file_size_multiplier = 1; + Reopen(options); + + Random rnd(301); + for (int i = 0; i < kNumLevels - 1; ++i) { + for (int j = 0; j < kFilesPerLevel; ++j) { + if (i == kNumLevels - 2) { + // insert range deletions [95,105) in two files, [295,305) in next two + // to prepare L1 for later manual compaction. + int mid = (j + (1 - j % 2)) * kNumPerFile; + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + Key(mid - 5), Key(mid + 5)); + } + std::vector values; + // Write 100KB (100 values, each 1K) + for (int k = 0; k < kNumPerFile; k++) { + values.push_back(RandomString(&rnd, 990)); + ASSERT_OK(Put(Key(j * kNumPerFile + k), values[k])); + } + // put extra key to trigger flush + ASSERT_OK(Put("", "")); + dbfull()->TEST_WaitForFlushMemTable(); + if (j < kFilesPerLevel - 1) { + // background compaction may happen early for kFilesPerLevel'th file + ASSERT_EQ(NumTableFilesAtLevel(0), j + 1); + } + } + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + ASSERT_GT(NumTableFilesAtLevel(kNumLevels - 1 - i), kFilesPerLevel - 1); + } + // Now L1-L3 are full, when we compact L1->L2 we should see (1) subcompactions + // happen since input level > 0; (2) range deletions are not dropped since + // output level is not bottommost. If no file boundary assertion fails, that + // probably means universal compaction + subcompaction + range deletion are + // compatible. + ASSERT_OK(dbfull()->RunManualCompaction( + reinterpret_cast(db_->DefaultColumnFamily()) + ->cfd(), + 1 /* input_level */, 2 /* output_level */, 0 /* output_path_id */, + nullptr /* begin */, nullptr /* end */, true /* exclusive */, + true /* disallow_trivial_move */)); +} +#endif // ROCKSDB_LITE + +TEST_F(DBRangeDelTest, CompactionRemovesCoveredMergeOperands) { + const int kNumPerFile = 3, kNumFiles = 3; + Options opts = CurrentOptions(); + opts.disable_auto_compactions = true; + opts.memtable_factory.reset(new SpecialSkipListFactory(2 * kNumPerFile)); + opts.merge_operator = MergeOperators::CreateUInt64AddOperator(); + opts.num_levels = 2; + Reopen(opts); + + // Iterates kNumFiles * kNumPerFile + 1 times since flushing the last file + // requires an extra entry. + for (int i = 0; i <= kNumFiles * kNumPerFile; ++i) { + if (i % kNumPerFile == 0 && i / kNumPerFile == kNumFiles - 1) { + // Delete merge operands from all but the last file + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "key", + "key_"); + } + std::string val; + PutFixed64(&val, i); + db_->Merge(WriteOptions(), "key", val); + // we need to prevent trivial move using Puts so compaction will actually + // process the merge operands. + db_->Put(WriteOptions(), "prevent_trivial_move", ""); + if (i > 0 && i % kNumPerFile == 0) { + dbfull()->TEST_WaitForFlushMemTable(); + } + } + + ReadOptions read_opts; + read_opts.ignore_range_deletions = true; + std::string expected, actual; + ASSERT_OK(db_->Get(read_opts, "key", &actual)); + PutFixed64(&expected, 45); // 1+2+...+9 + ASSERT_EQ(expected, actual); + + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + + expected.clear(); + ASSERT_OK(db_->Get(read_opts, "key", &actual)); + uint64_t tmp; + Slice tmp2(actual); + GetFixed64(&tmp2, &tmp); + PutFixed64(&expected, 30); // 6+7+8+9 (earlier operands covered by tombstone) + ASSERT_EQ(expected, actual); +} + +// NumTableFilesAtLevel() is not supported in ROCKSDB_LITE +#ifndef ROCKSDB_LITE +TEST_F(DBRangeDelTest, ObsoleteTombstoneCleanup) { + // During compaction to bottommost level, verify range tombstones older than + // the oldest snapshot are removed, while others are preserved. + Options opts = CurrentOptions(); + opts.disable_auto_compactions = true; + opts.num_levels = 2; + opts.statistics = CreateDBStatistics(); + Reopen(opts); + + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "dr1", + "dr1"); // obsolete after compaction + db_->Put(WriteOptions(), "key", "val"); + db_->Flush(FlushOptions()); + const Snapshot* snapshot = db_->GetSnapshot(); + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "dr2", + "dr2"); // protected by snapshot + db_->Put(WriteOptions(), "key", "val"); + db_->Flush(FlushOptions()); + + ASSERT_EQ(2, NumTableFilesAtLevel(0)); + ASSERT_EQ(0, NumTableFilesAtLevel(1)); + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ(0, NumTableFilesAtLevel(0)); + ASSERT_EQ(1, NumTableFilesAtLevel(1)); + ASSERT_EQ(1, TestGetTickerCount(opts, COMPACTION_RANGE_DEL_DROP_OBSOLETE)); + + db_->ReleaseSnapshot(snapshot); +} + +TEST_F(DBRangeDelTest, TableEvictedDuringScan) { + // The RangeDelAggregator holds pointers into range deletion blocks created by + // table readers. This test ensures the aggregator can still access those + // blocks even if it outlives the table readers that created them. + // + // DBIter always keeps readers open for L0 files. So, in order to test + // aggregator outliving reader, we need to have deletions in L1 files, which + // are opened/closed on-demand during the scan. This is accomplished by + // setting kNumRanges > level0_stop_writes_trigger, which prevents deletions + // from all lingering in L0 (there is at most one range deletion per L0 file). + // + // The first L1 file will contain a range deletion since its begin key is 0. + // SeekToFirst() references that table's reader and adds its range tombstone + // to the aggregator. Upon advancing beyond that table's key-range via Next(), + // the table reader will be unreferenced by the iterator. Since we manually + // call Evict() on all readers before the full scan, this unreference causes + // the reader's refcount to drop to zero and thus be destroyed. + // + // When it is destroyed, we do not remove its range deletions from the + // aggregator. So, subsequent calls to Next() must be able to use these + // deletions to decide whether a key is covered. This will work as long as + // the aggregator properly references the range deletion block. + const int kNum = 25, kRangeBegin = 0, kRangeEnd = 7, kNumRanges = 5; + Options opts = CurrentOptions(); + opts.comparator = test::Uint64Comparator(); + opts.level0_file_num_compaction_trigger = 4; + opts.level0_stop_writes_trigger = 4; + opts.memtable_factory.reset(new SpecialSkipListFactory(1)); + opts.num_levels = 2; + BlockBasedTableOptions bbto; + bbto.cache_index_and_filter_blocks = true; + bbto.block_cache = NewLRUCache(8 << 20); + opts.table_factory.reset(NewBlockBasedTableFactory(bbto)); + Reopen(opts); + + // Hold a snapshot so range deletions can't become obsolete during compaction + // to bottommost level (i.e., L1). + const Snapshot* snapshot = db_->GetSnapshot(); + for (int i = 0; i < kNum; ++i) { + db_->Put(WriteOptions(), GetNumericStr(i), "val"); + if (i > 0) { + dbfull()->TEST_WaitForFlushMemTable(); + } + if (i >= kNum / 2 && i < kNum / 2 + kNumRanges) { + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + GetNumericStr(kRangeBegin), GetNumericStr(kRangeEnd)); + } + } + // Must be > 1 so the first L1 file can be closed before scan finishes + dbfull()->TEST_WaitForCompact(); + ASSERT_GT(NumTableFilesAtLevel(1), 1); + std::vector file_numbers = ListTableFiles(env_, dbname_); + + ReadOptions read_opts; + auto* iter = db_->NewIterator(read_opts); + int expected = kRangeEnd; + iter->SeekToFirst(); + for (auto file_number : file_numbers) { + // This puts table caches in the state of being externally referenced only + // so they are destroyed immediately upon iterator unreferencing. + TableCache::Evict(dbfull()->TEST_table_cache(), file_number); + } + for (; iter->Valid(); iter->Next()) { + ASSERT_EQ(GetNumericStr(expected), iter->key()); + ++expected; + // Keep clearing block cache's LRU so range deletion block can be freed as + // soon as its refcount drops to zero. + bbto.block_cache->EraseUnRefEntries(); + } + ASSERT_EQ(kNum, expected); + delete iter; + db_->ReleaseSnapshot(snapshot); +} + +TEST_F(DBRangeDelTest, GetCoveredKeyFromMutableMemtable) { + db_->Put(WriteOptions(), "key", "val"); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z")); + + ReadOptions read_opts; + std::string value; + ASSERT_TRUE(db_->Get(read_opts, "key", &value).IsNotFound()); +} + +TEST_F(DBRangeDelTest, GetCoveredKeyFromImmutableMemtable) { + Options opts = CurrentOptions(); + opts.max_write_buffer_number = 3; + opts.min_write_buffer_number_to_merge = 2; + // SpecialSkipListFactory lets us specify maximum number of elements the + // memtable can hold. It switches the active memtable to immutable (flush is + // prevented by the above options) upon inserting an element that would + // overflow the memtable. + opts.memtable_factory.reset(new SpecialSkipListFactory(1)); + Reopen(opts); + + db_->Put(WriteOptions(), "key", "val"); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z")); + db_->Put(WriteOptions(), "blah", "val"); + + ReadOptions read_opts; + std::string value; + ASSERT_TRUE(db_->Get(read_opts, "key", &value).IsNotFound()); +} + +TEST_F(DBRangeDelTest, GetCoveredKeyFromSst) { + db_->Put(WriteOptions(), "key", "val"); + // snapshot prevents key from being deleted during flush + const Snapshot* snapshot = db_->GetSnapshot(); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z")); + ASSERT_OK(db_->Flush(FlushOptions())); + + ReadOptions read_opts; + std::string value; + ASSERT_TRUE(db_->Get(read_opts, "key", &value).IsNotFound()); + db_->ReleaseSnapshot(snapshot); +} + +TEST_F(DBRangeDelTest, GetCoveredMergeOperandFromMemtable) { + const int kNumMergeOps = 10; + Options opts = CurrentOptions(); + opts.merge_operator = MergeOperators::CreateUInt64AddOperator(); + Reopen(opts); + + for (int i = 0; i < kNumMergeOps; ++i) { + std::string val; + PutFixed64(&val, i); + db_->Merge(WriteOptions(), "key", val); + if (i == kNumMergeOps / 2) { + // deletes [0, 5] + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "key", + "key_"); + } + } + + ReadOptions read_opts; + std::string expected, actual; + ASSERT_OK(db_->Get(read_opts, "key", &actual)); + PutFixed64(&expected, 30); // 6+7+8+9 + ASSERT_EQ(expected, actual); + + expected.clear(); + read_opts.ignore_range_deletions = true; + ASSERT_OK(db_->Get(read_opts, "key", &actual)); + PutFixed64(&expected, 45); // 0+1+2+...+9 + ASSERT_EQ(expected, actual); +} + +TEST_F(DBRangeDelTest, GetIgnoresRangeDeletions) { + Options opts = CurrentOptions(); + opts.max_write_buffer_number = 4; + opts.min_write_buffer_number_to_merge = 3; + opts.memtable_factory.reset(new SpecialSkipListFactory(1)); + Reopen(opts); + + db_->Put(WriteOptions(), "sst_key", "val"); + // snapshot prevents key from being deleted during flush + const Snapshot* snapshot = db_->GetSnapshot(); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z")); + ASSERT_OK(db_->Flush(FlushOptions())); + db_->Put(WriteOptions(), "imm_key", "val"); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z")); + db_->Put(WriteOptions(), "mem_key", "val"); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z")); + + ReadOptions read_opts; + read_opts.ignore_range_deletions = true; + for (std::string key : {"sst_key", "imm_key", "mem_key"}) { + std::string value; + ASSERT_OK(db_->Get(read_opts, key, &value)); + } + db_->ReleaseSnapshot(snapshot); +} + +TEST_F(DBRangeDelTest, IteratorRemovesCoveredKeys) { + const int kNum = 200, kRangeBegin = 50, kRangeEnd = 150, kNumPerFile = 25; + Options opts = CurrentOptions(); + opts.comparator = test::Uint64Comparator(); + opts.memtable_factory.reset(new SpecialSkipListFactory(kNumPerFile)); + Reopen(opts); + + // Write half of the keys before the tombstone and half after the tombstone. + // Only covered keys (i.e., within the range and older than the tombstone) + // should be deleted. + for (int i = 0; i < kNum; ++i) { + if (i == kNum / 2) { + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + GetNumericStr(kRangeBegin), GetNumericStr(kRangeEnd)); + } + db_->Put(WriteOptions(), GetNumericStr(i), "val"); + } + ReadOptions read_opts; + auto* iter = db_->NewIterator(read_opts); + + int expected = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_EQ(GetNumericStr(expected), iter->key()); + if (expected == kRangeBegin - 1) { + expected = kNum / 2; + } else { + ++expected; + } + } + ASSERT_EQ(kNum, expected); + delete iter; +} + +TEST_F(DBRangeDelTest, IteratorOverUserSnapshot) { + const int kNum = 200, kRangeBegin = 50, kRangeEnd = 150, kNumPerFile = 25; + Options opts = CurrentOptions(); + opts.comparator = test::Uint64Comparator(); + opts.memtable_factory.reset(new SpecialSkipListFactory(kNumPerFile)); + Reopen(opts); + + const Snapshot* snapshot = nullptr; + // Put a snapshot before the range tombstone, verify an iterator using that + // snapshot sees all inserted keys. + for (int i = 0; i < kNum; ++i) { + if (i == kNum / 2) { + snapshot = db_->GetSnapshot(); + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + GetNumericStr(kRangeBegin), GetNumericStr(kRangeEnd)); + } + db_->Put(WriteOptions(), GetNumericStr(i), "val"); + } + ReadOptions read_opts; + read_opts.snapshot = snapshot; + auto* iter = db_->NewIterator(read_opts); + + int expected = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_EQ(GetNumericStr(expected), iter->key()); + ++expected; + } + ASSERT_EQ(kNum / 2, expected); + delete iter; + db_->ReleaseSnapshot(snapshot); +} + +TEST_F(DBRangeDelTest, IteratorIgnoresRangeDeletions) { + Options opts = CurrentOptions(); + opts.max_write_buffer_number = 4; + opts.min_write_buffer_number_to_merge = 3; + opts.memtable_factory.reset(new SpecialSkipListFactory(1)); + Reopen(opts); + + db_->Put(WriteOptions(), "sst_key", "val"); + // snapshot prevents key from being deleted during flush + const Snapshot* snapshot = db_->GetSnapshot(); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z")); + ASSERT_OK(db_->Flush(FlushOptions())); + db_->Put(WriteOptions(), "imm_key", "val"); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z")); + db_->Put(WriteOptions(), "mem_key", "val"); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z")); + + ReadOptions read_opts; + read_opts.ignore_range_deletions = true; + auto* iter = db_->NewIterator(read_opts); + int i = 0; + std::string expected[] = {"imm_key", "mem_key", "sst_key"}; + for (iter->SeekToFirst(); iter->Valid(); iter->Next(), ++i) { + std::string key; + ASSERT_EQ(expected[i], iter->key()); + } + ASSERT_EQ(3, i); + delete iter; + db_->ReleaseSnapshot(snapshot); +} + +#ifndef ROCKSDB_UBSAN_RUN +TEST_F(DBRangeDelTest, TailingIteratorRangeTombstoneUnsupported) { + db_->Put(WriteOptions(), "key", "val"); + // snapshot prevents key from being deleted during flush + const Snapshot* snapshot = db_->GetSnapshot(); + ASSERT_OK( + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), "a", "z")); + + // iterations check unsupported in memtable, l0, and then l1 + for (int i = 0; i < 3; ++i) { + ReadOptions read_opts; + read_opts.tailing = true; + auto* iter = db_->NewIterator(read_opts); + if (i == 2) { + // For L1+, iterators over files are created on-demand, so need seek + iter->SeekToFirst(); + } + ASSERT_TRUE(iter->status().IsNotSupported()); + delete iter; + if (i == 0) { + ASSERT_OK(db_->Flush(FlushOptions())); + } else if (i == 1) { + MoveFilesToLevel(1); + } + } + db_->ReleaseSnapshot(snapshot); +} + +#endif // !ROCKSDB_UBSAN_RUN + +TEST_F(DBRangeDelTest, SubcompactionHasEmptyDedicatedRangeDelFile) { + const int kNumFiles = 2, kNumKeysPerFile = 4; + Options options = CurrentOptions(); + options.compression = kNoCompression; + options.disable_auto_compactions = true; + options.level0_file_num_compaction_trigger = kNumFiles; + options.max_subcompactions = 2; + options.num_levels = 2; + options.target_file_size_base = 4096; + Reopen(options); + + // need a L1 file for subcompaction to be triggered + ASSERT_OK( + db_->Put(WriteOptions(), db_->DefaultColumnFamily(), Key(0), "val")); + ASSERT_OK(db_->Flush(FlushOptions())); + MoveFilesToLevel(1); + + // put enough keys to fill up the first subcompaction, and later range-delete + // them so that the first subcompaction outputs no key-values. In that case + // it'll consider making an SST file dedicated to range deletions. + for (int i = 0; i < kNumKeysPerFile; ++i) { + ASSERT_OK(db_->Put(WriteOptions(), db_->DefaultColumnFamily(), Key(i), + std::string(1024, 'a'))); + } + ASSERT_OK(db_->Flush(FlushOptions())); + ASSERT_OK(db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), Key(0), + Key(kNumKeysPerFile))); + + // the above range tombstone can be dropped, so that one alone won't cause a + // dedicated file to be opened. We can make one protected by snapshot that + // must be considered. Make its range outside the first subcompaction's range + // to exercise the tricky part of the code. + const Snapshot* snapshot = db_->GetSnapshot(); + ASSERT_OK(db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + Key(kNumKeysPerFile + 1), + Key(kNumKeysPerFile + 2))); + ASSERT_OK(db_->Flush(FlushOptions())); + + ASSERT_EQ(kNumFiles, NumTableFilesAtLevel(0)); + ASSERT_EQ(1, NumTableFilesAtLevel(1)); + + db_->EnableAutoCompaction({db_->DefaultColumnFamily()}); + dbfull()->TEST_WaitForCompact(); + db_->ReleaseSnapshot(snapshot); +} + +TEST_F(DBRangeDelTest, MemtableBloomFilter) { + // regression test for #2743. the range delete tombstones in memtable should + // be added even when Get() skips searching due to its prefix bloom filter + const int kMemtableSize = 1 << 20; // 1MB + const int kMemtablePrefixFilterSize = 1 << 13; // 8KB + const int kNumKeys = 1000; + const int kPrefixLen = 8; + Options options = CurrentOptions(); + options.memtable_prefix_bloom_size_ratio = + static_cast(kMemtablePrefixFilterSize) / kMemtableSize; + options.prefix_extractor.reset(rocksdb::NewFixedPrefixTransform(kPrefixLen)); + options.write_buffer_size = kMemtableSize; + Reopen(options); + + for (int i = 0; i < kNumKeys; ++i) { + ASSERT_OK(Put(Key(i), "val")); + } + Flush(); + ASSERT_OK(db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), Key(0), + Key(kNumKeys))); + for (int i = 0; i < kNumKeys; ++i) { + std::string value; + ASSERT_TRUE(db_->Get(ReadOptions(), Key(i), &value).IsNotFound()); + } +} + +TEST_F(DBRangeDelTest, CompactionTreatsSplitInputLevelDeletionAtomically) { + // make sure compaction treats files containing a split range deletion in the + // input level as an atomic unit. I.e., compacting any input-level file(s) + // containing a portion of the range deletion causes all other input-level + // files containing portions of that same range deletion to be included in the + // compaction. + const int kNumFilesPerLevel = 4, kValueBytes = 4 << 10; + Options options = CurrentOptions(); + options.compression = kNoCompression; + options.level0_file_num_compaction_trigger = kNumFilesPerLevel; + options.memtable_factory.reset( + new SpecialSkipListFactory(2 /* num_entries_flush */)); + options.target_file_size_base = kValueBytes; + // i == 0: CompactFiles + // i == 1: CompactRange + // i == 2: automatic compaction + for (int i = 0; i < 3; ++i) { + DestroyAndReopen(options); + + ASSERT_OK(Put(Key(0), "")); + ASSERT_OK(db_->Flush(FlushOptions())); + MoveFilesToLevel(2); + ASSERT_EQ(1, NumTableFilesAtLevel(2)); + + // snapshot protects range tombstone from dropping due to becoming obsolete. + const Snapshot* snapshot = db_->GetSnapshot(); + db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), Key(0), + Key(2 * kNumFilesPerLevel)); + + Random rnd(301); + std::string value = RandomString(&rnd, kValueBytes); + for (int j = 0; j < kNumFilesPerLevel; ++j) { + // give files overlapping key-ranges to prevent trivial move + ASSERT_OK(Put(Key(j), value)); + ASSERT_OK(Put(Key(2 * kNumFilesPerLevel - 1 - j), value)); + if (j > 0) { + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(j, NumTableFilesAtLevel(0)); + } + } + // put extra key to trigger final flush + ASSERT_OK(Put("", "")); + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(0, NumTableFilesAtLevel(0)); + ASSERT_EQ(kNumFilesPerLevel, NumTableFilesAtLevel(1)); + + ColumnFamilyMetaData meta; + db_->GetColumnFamilyMetaData(&meta); + if (i == 0) { + ASSERT_OK(db_->CompactFiles( + CompactionOptions(), {meta.levels[1].files[0].name}, 2 /* level */)); + } else if (i == 1) { + auto begin_str = Key(0), end_str = Key(1); + Slice begin = begin_str, end = end_str; + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), &begin, &end)); + } else if (i == 2) { + ASSERT_OK(db_->SetOptions(db_->DefaultColumnFamily(), + {{"max_bytes_for_level_base", "10000"}})); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_EQ(0, NumTableFilesAtLevel(1)); + ASSERT_GT(NumTableFilesAtLevel(2), 0); + + db_->ReleaseSnapshot(snapshot); + } +} + +TEST_F(DBRangeDelTest, UnorderedTombstones) { + // Regression test for #2752. Range delete tombstones between + // different snapshot stripes are not stored in order, so the first + // tombstone of each snapshot stripe should be checked as a smallest + // candidate. + Options options = CurrentOptions(); + DestroyAndReopen(options); + + auto cf = db_->DefaultColumnFamily(); + + ASSERT_OK(db_->Put(WriteOptions(), cf, "a", "a")); + ASSERT_OK(db_->Flush(FlushOptions(), cf)); + ASSERT_EQ(1, NumTableFilesAtLevel(0)); + ASSERT_OK(dbfull()->TEST_CompactRange(0, nullptr, nullptr)); + ASSERT_EQ(1, NumTableFilesAtLevel(1)); + + ASSERT_OK(db_->DeleteRange(WriteOptions(), cf, "b", "c")); + // Hold a snapshot to separate these two delete ranges. + auto snapshot = db_->GetSnapshot(); + ASSERT_OK(db_->DeleteRange(WriteOptions(), cf, "a", "b")); + ASSERT_OK(db_->Flush(FlushOptions(), cf)); + db_->ReleaseSnapshot(snapshot); + + std::vector> files; + dbfull()->TEST_GetFilesMetaData(cf, &files); + ASSERT_EQ(1, files[0].size()); + ASSERT_EQ("a", files[0][0].smallest.user_key()); + ASSERT_EQ("c", files[0][0].largest.user_key()); + + std::string v; + auto s = db_->Get(ReadOptions(), "a", &v); + ASSERT_TRUE(s.IsNotFound()); +} + +#endif // ROCKSDB_LITE + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_sst_test.cc b/db/db_sst_test.cc new file mode 100644 index 00000000000..e01754c44e4 --- /dev/null +++ b/db/db_sst_test.cc @@ -0,0 +1,849 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_test_util.h" +#include "port/port.h" +#include "port/stack_trace.h" +#include "rocksdb/sst_file_manager.h" +#include "util/sst_file_manager_impl.h" + +namespace rocksdb { + +class DBSSTTest : public DBTestBase { + public: + DBSSTTest() : DBTestBase("/db_sst_test") {} +}; + +TEST_F(DBSSTTest, DontDeletePendingOutputs) { + Options options; + options.env = env_; + options.create_if_missing = true; + DestroyAndReopen(options); + + // Every time we write to a table file, call FOF/POF with full DB scan. This + // will make sure our pending_outputs_ protection work correctly + std::function purge_obsolete_files_function = [&]() { + JobContext job_context(0); + dbfull()->TEST_LockMutex(); + dbfull()->FindObsoleteFiles(&job_context, true /*force*/); + dbfull()->TEST_UnlockMutex(); + dbfull()->PurgeObsoleteFiles(job_context); + job_context.Clean(); + }; + + env_->table_write_callback_ = &purge_obsolete_files_function; + + for (int i = 0; i < 2; ++i) { + ASSERT_OK(Put("a", "begin")); + ASSERT_OK(Put("z", "end")); + ASSERT_OK(Flush()); + } + + // If pending output guard does not work correctly, PurgeObsoleteFiles() will + // delete the file that Compaction is trying to create, causing this: error + // db/db_test.cc:975: IO error: + // /tmp/rocksdbtest-1552237650/db_test/000009.sst: No such file or directory + Compact("a", "b"); +} + +// 1 Create some SST files by inserting K-V pairs into DB +// 2 Close DB and change suffix from ".sst" to ".ldb" for every other SST file +// 3 Open DB and check if all key can be read +TEST_F(DBSSTTest, SSTsWithLdbSuffixHandling) { + Options options = CurrentOptions(); + options.write_buffer_size = 110 << 10; // 110KB + options.num_levels = 4; + DestroyAndReopen(options); + + Random rnd(301); + int key_id = 0; + for (int i = 0; i < 10; ++i) { + GenerateNewFile(&rnd, &key_id, false); + } + Flush(); + Close(); + int const num_files = GetSstFileCount(dbname_); + ASSERT_GT(num_files, 0); + + std::vector filenames; + GetSstFiles(dbname_, &filenames); + int num_ldb_files = 0; + for (size_t i = 0; i < filenames.size(); ++i) { + if (i & 1) { + continue; + } + std::string const rdb_name = dbname_ + "/" + filenames[i]; + std::string const ldb_name = Rocks2LevelTableFileName(rdb_name); + ASSERT_TRUE(env_->RenameFile(rdb_name, ldb_name).ok()); + ++num_ldb_files; + } + ASSERT_GT(num_ldb_files, 0); + ASSERT_EQ(num_files, GetSstFileCount(dbname_)); + + Reopen(options); + for (int k = 0; k < key_id; ++k) { + ASSERT_NE("NOT_FOUND", Get(Key(k))); + } + Destroy(options); +} + +#ifndef ROCKSDB_LITE +TEST_F(DBSSTTest, DontDeleteMovedFile) { + // This test triggers move compaction and verifies that the file is not + // deleted when it's part of move compaction + Options options = CurrentOptions(); + options.env = env_; + options.create_if_missing = true; + options.max_bytes_for_level_base = 1024 * 1024; // 1 MB + options.level0_file_num_compaction_trigger = + 2; // trigger compaction when we have 2 files + DestroyAndReopen(options); + + Random rnd(301); + // Create two 1MB sst files + for (int i = 0; i < 2; ++i) { + // Create 1MB sst file + for (int j = 0; j < 100; ++j) { + ASSERT_OK(Put(Key(i * 50 + j), RandomString(&rnd, 10 * 1024))); + } + ASSERT_OK(Flush()); + } + // this should execute both L0->L1 and L1->(move)->L2 compactions + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ("0,0,1", FilesPerLevel(0)); + + // If the moved file is actually deleted (the move-safeguard in + // ~Version::Version() is not there), we get this failure: + // Corruption: Can't access /000009.sst + Reopen(options); +} + +// This reproduces a bug where we don't delete a file because when it was +// supposed to be deleted, it was blocked by pending_outputs +// Consider: +// 1. current file_number is 13 +// 2. compaction (1) starts, blocks deletion of all files starting with 13 +// (pending outputs) +// 3. file 13 is created by compaction (2) +// 4. file 13 is consumed by compaction (3) and file 15 was created. Since file +// 13 has no references, it is put into VersionSet::obsolete_files_ +// 5. FindObsoleteFiles() gets file 13 from VersionSet::obsolete_files_. File 13 +// is deleted from obsolete_files_ set. +// 6. PurgeObsoleteFiles() tries to delete file 13, but this file is blocked by +// pending outputs since compaction (1) is still running. It is not deleted and +// it is not present in obsolete_files_ anymore. Therefore, we never delete it. +TEST_F(DBSSTTest, DeleteObsoleteFilesPendingOutputs) { + Options options = CurrentOptions(); + options.env = env_; + options.write_buffer_size = 2 * 1024 * 1024; // 2 MB + options.max_bytes_for_level_base = 1024 * 1024; // 1 MB + options.level0_file_num_compaction_trigger = + 2; // trigger compaction when we have 2 files + options.max_background_flushes = 2; + options.max_background_compactions = 2; + + OnFileDeletionListener* listener = new OnFileDeletionListener(); + options.listeners.emplace_back(listener); + + Reopen(options); + + Random rnd(301); + // Create two 1MB sst files + for (int i = 0; i < 2; ++i) { + // Create 1MB sst file + for (int j = 0; j < 100; ++j) { + ASSERT_OK(Put(Key(i * 50 + j), RandomString(&rnd, 10 * 1024))); + } + ASSERT_OK(Flush()); + } + // this should execute both L0->L1 and L1->(move)->L2 compactions + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ("0,0,1", FilesPerLevel(0)); + + test::SleepingBackgroundTask blocking_thread; + port::Mutex mutex_; + bool already_blocked(false); + + // block the flush + std::function block_first_time = [&]() { + bool blocking = false; + { + MutexLock l(&mutex_); + if (!already_blocked) { + blocking = true; + already_blocked = true; + } + } + if (blocking) { + blocking_thread.DoSleep(); + } + }; + env_->table_write_callback_ = &block_first_time; + // Insert 2.5MB data, which should trigger a flush because we exceed + // write_buffer_size. The flush will be blocked with block_first_time + // pending_file is protecting all the files created after + for (int j = 0; j < 256; ++j) { + ASSERT_OK(Put(Key(j), RandomString(&rnd, 10 * 1024))); + } + blocking_thread.WaitUntilSleeping(); + + ASSERT_OK(dbfull()->TEST_CompactRange(2, nullptr, nullptr)); + + ASSERT_EQ("0,0,0,1", FilesPerLevel(0)); + std::vector metadata; + db_->GetLiveFilesMetaData(&metadata); + ASSERT_EQ(metadata.size(), 1U); + auto file_on_L2 = metadata[0].name; + listener->SetExpectedFileName(dbname_ + file_on_L2); + + ASSERT_OK(dbfull()->TEST_CompactRange(3, nullptr, nullptr, nullptr, + true /* disallow trivial move */)); + ASSERT_EQ("0,0,0,0,1", FilesPerLevel(0)); + + // finish the flush! + blocking_thread.WakeUp(); + blocking_thread.WaitUntilDone(); + dbfull()->TEST_WaitForFlushMemTable(); + // File just flushed is too big for L0 and L1 so gets moved to L2. + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ("0,0,1,0,1", FilesPerLevel(0)); + + metadata.clear(); + db_->GetLiveFilesMetaData(&metadata); + ASSERT_EQ(metadata.size(), 2U); + + // This file should have been deleted during last compaction + ASSERT_EQ(Status::NotFound(), env_->FileExists(dbname_ + file_on_L2)); + listener->VerifyMatchedCount(1); +} + +TEST_F(DBSSTTest, DBWithSstFileManager) { + std::shared_ptr sst_file_manager(NewSstFileManager(env_)); + auto sfm = static_cast(sst_file_manager.get()); + + int files_added = 0; + int files_deleted = 0; + int files_moved = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "SstFileManagerImpl::OnAddFile", [&](void* arg) { files_added++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "SstFileManagerImpl::OnDeleteFile", [&](void* arg) { files_deleted++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "SstFileManagerImpl::OnMoveFile", [&](void* arg) { files_moved++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.sst_file_manager = sst_file_manager; + DestroyAndReopen(options); + + Random rnd(301); + for (int i = 0; i < 25; i++) { + GenerateNewRandomFile(&rnd); + ASSERT_OK(Flush()); + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + // Verify that we are tracking all sst files in dbname_ + ASSERT_EQ(sfm->GetTrackedFiles(), GetAllSSTFiles()); + } + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + + auto files_in_db = GetAllSSTFiles(); + // Verify that we are tracking all sst files in dbname_ + ASSERT_EQ(sfm->GetTrackedFiles(), files_in_db); + // Verify the total files size + uint64_t total_files_size = 0; + for (auto& file_to_size : files_in_db) { + total_files_size += file_to_size.second; + } + ASSERT_EQ(sfm->GetTotalSize(), total_files_size); + // We flushed at least 25 files + ASSERT_GE(files_added, 25); + // Compaction must have deleted some files + ASSERT_GT(files_deleted, 0); + // No files were moved + ASSERT_EQ(files_moved, 0); + + Close(); + Reopen(options); + ASSERT_EQ(sfm->GetTrackedFiles(), files_in_db); + ASSERT_EQ(sfm->GetTotalSize(), total_files_size); + + // Verify that we track all the files again after the DB is closed and opened + Close(); + sst_file_manager.reset(NewSstFileManager(env_)); + options.sst_file_manager = sst_file_manager; + sfm = static_cast(sst_file_manager.get()); + + Reopen(options); + ASSERT_EQ(sfm->GetTrackedFiles(), files_in_db); + ASSERT_EQ(sfm->GetTotalSize(), total_files_size); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(DBSSTTest, RateLimitedDelete) { + Destroy(last_options_); + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"DBSSTTest::RateLimitedDelete:1", + "DeleteScheduler::BackgroundEmptyTrash"}, + }); + + std::vector penalties; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DeleteScheduler::BackgroundEmptyTrash:Wait", + [&](void* arg) { penalties.push_back(*(static_cast(arg))); }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "InstrumentedCondVar::TimedWaitInternal", [&](void* arg) { + // Turn timed wait into a simulated sleep + uint64_t* abs_time_us = static_cast(arg); + int64_t cur_time = 0; + env_->GetCurrentTime(&cur_time); + if (*abs_time_us > static_cast(cur_time)) { + env_->addon_time_.fetch_add(*abs_time_us - + static_cast(cur_time)); + } + + // Randomly sleep shortly + env_->addon_time_.fetch_add( + static_cast(Random::GetTLSInstance()->Uniform(10))); + + // Set wait until time to before current to force not to sleep. + int64_t real_cur_time = 0; + Env::Default()->GetCurrentTime(&real_cur_time); + *abs_time_us = static_cast(real_cur_time); + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + env_->no_slowdown_ = true; + env_->time_elapse_only_sleep_ = true; + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.env = env_; + + std::string trash_dir = test::TmpDir(env_) + "/trash"; + int64_t rate_bytes_per_sec = 1024 * 10; // 10 Kbs / Sec + Status s; + options.sst_file_manager.reset( + NewSstFileManager(env_, nullptr, trash_dir, 0, false, &s)); + ASSERT_OK(s); + options.sst_file_manager->SetDeleteRateBytesPerSecond(rate_bytes_per_sec); + auto sfm = static_cast(options.sst_file_manager.get()); + sfm->delete_scheduler()->TEST_SetMaxTrashDBRatio(1.1); + + ASSERT_OK(TryReopen(options)); + // Create 4 files in L0 + for (char v = 'a'; v <= 'd'; v++) { + ASSERT_OK(Put("Key2", DummyString(1024, v))); + ASSERT_OK(Put("Key3", DummyString(1024, v))); + ASSERT_OK(Put("Key4", DummyString(1024, v))); + ASSERT_OK(Put("Key1", DummyString(1024, v))); + ASSERT_OK(Put("Key4", DummyString(1024, v))); + ASSERT_OK(Flush()); + } + // We created 4 sst files in L0 + ASSERT_EQ("4", FilesPerLevel(0)); + + std::vector metadata; + db_->GetLiveFilesMetaData(&metadata); + + // Compaction will move the 4 files in L0 to trash and create 1 L1 file + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ("0,1", FilesPerLevel(0)); + + uint64_t delete_start_time = env_->NowMicros(); + // Hold BackgroundEmptyTrash + TEST_SYNC_POINT("DBSSTTest::RateLimitedDelete:1"); + sfm->WaitForEmptyTrash(); + uint64_t time_spent_deleting = env_->NowMicros() - delete_start_time; + + uint64_t total_files_size = 0; + uint64_t expected_penlty = 0; + ASSERT_EQ(penalties.size(), metadata.size()); + for (size_t i = 0; i < metadata.size(); i++) { + total_files_size += metadata[i].size; + expected_penlty = ((total_files_size * 1000000) / rate_bytes_per_sec); + ASSERT_EQ(expected_penlty, penalties[i]); + } + ASSERT_GT(time_spent_deleting, expected_penlty * 0.9); + ASSERT_LT(time_spent_deleting, expected_penlty * 1.1); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +// Create a DB with 2 db_paths, and generate multiple files in the 2 +// db_paths using CompactRangeOptions, make sure that files that were +// deleted from first db_path were deleted using DeleteScheduler and +// files in the second path were not. +TEST_F(DBSSTTest, DeleteSchedulerMultipleDBPaths) { + int bg_delete_file = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DeleteScheduler::DeleteTrashFile:DeleteFile", + [&](void* arg) { bg_delete_file++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.db_paths.emplace_back(dbname_, 1024 * 100); + options.db_paths.emplace_back(dbname_ + "_2", 1024 * 100); + options.env = env_; + + std::string trash_dir = test::TmpDir(env_) + "/trash"; + int64_t rate_bytes_per_sec = 1024 * 1024; // 1 Mb / Sec + Status s; + options.sst_file_manager.reset(NewSstFileManager( + env_, nullptr, trash_dir, rate_bytes_per_sec, false, &s)); + ASSERT_OK(s); + auto sfm = static_cast(options.sst_file_manager.get()); + sfm->delete_scheduler()->TEST_SetMaxTrashDBRatio(1.1); + + DestroyAndReopen(options); + + // Create 4 files in L0 + for (int i = 0; i < 4; i++) { + ASSERT_OK(Put("Key" + ToString(i), DummyString(1024, 'A'))); + ASSERT_OK(Flush()); + } + // We created 4 sst files in L0 + ASSERT_EQ("4", FilesPerLevel(0)); + // Compaction will delete files from L0 in first db path and generate a new + // file in L1 in second db path + CompactRangeOptions compact_options; + compact_options.target_path_id = 1; + Slice begin("Key0"); + Slice end("Key3"); + ASSERT_OK(db_->CompactRange(compact_options, &begin, &end)); + ASSERT_EQ("0,1", FilesPerLevel(0)); + + // Create 4 files in L0 + for (int i = 4; i < 8; i++) { + ASSERT_OK(Put("Key" + ToString(i), DummyString(1024, 'B'))); + ASSERT_OK(Flush()); + } + ASSERT_EQ("4,1", FilesPerLevel(0)); + + // Compaction will delete files from L0 in first db path and generate a new + // file in L1 in second db path + begin = "Key4"; + end = "Key7"; + ASSERT_OK(db_->CompactRange(compact_options, &begin, &end)); + ASSERT_EQ("0,2", FilesPerLevel(0)); + + sfm->WaitForEmptyTrash(); + ASSERT_EQ(bg_delete_file, 8); + + compact_options.bottommost_level_compaction = + BottommostLevelCompaction::kForce; + ASSERT_OK(db_->CompactRange(compact_options, nullptr, nullptr)); + ASSERT_EQ("0,1", FilesPerLevel(0)); + + sfm->WaitForEmptyTrash(); + ASSERT_EQ(bg_delete_file, 8); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(DBSSTTest, DestroyDBWithRateLimitedDelete) { + int bg_delete_file = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DeleteScheduler::DeleteTrashFile:DeleteFile", + [&](void* arg) { bg_delete_file++; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Status s; + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.env = env_; + std::string trash_dir = test::TmpDir(env_) + "/trash"; + options.sst_file_manager.reset( + NewSstFileManager(env_, nullptr, trash_dir, 0, false, &s)); + ASSERT_OK(s); + DestroyAndReopen(options); + + // Create 4 files in L0 + for (int i = 0; i < 4; i++) { + ASSERT_OK(Put("Key" + ToString(i), DummyString(1024, 'A'))); + ASSERT_OK(Flush()); + } + // We created 4 sst files in L0 + ASSERT_EQ("4", FilesPerLevel(0)); + + // Close DB and destroy it using DeleteScheduler + Close(); + + auto sfm = static_cast(options.sst_file_manager.get()); + + sfm->SetDeleteRateBytesPerSecond(1024 * 1024); + sfm->delete_scheduler()->TEST_SetMaxTrashDBRatio(1.1); + ASSERT_OK(DestroyDB(dbname_, options)); + sfm->WaitForEmptyTrash(); + // We have deleted the 4 sst files in the delete_scheduler + ASSERT_EQ(bg_delete_file, 4); +} + +TEST_F(DBSSTTest, DBWithMaxSpaceAllowed) { + std::shared_ptr sst_file_manager(NewSstFileManager(env_)); + auto sfm = static_cast(sst_file_manager.get()); + + Options options = CurrentOptions(); + options.sst_file_manager = sst_file_manager; + options.disable_auto_compactions = true; + DestroyAndReopen(options); + + Random rnd(301); + + // Generate a file containing 100 keys. + for (int i = 0; i < 100; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 50))); + } + ASSERT_OK(Flush()); + + uint64_t first_file_size = 0; + auto files_in_db = GetAllSSTFiles(&first_file_size); + ASSERT_EQ(sfm->GetTotalSize(), first_file_size); + + // Set the maximum allowed space usage to the current total size + sfm->SetMaxAllowedSpaceUsage(first_file_size + 1); + + ASSERT_OK(Put("key1", "val1")); + // This flush will cause bg_error_ and will fail + ASSERT_NOK(Flush()); +} + +TEST_F(DBSSTTest, DBWithMaxSpaceAllowedRandomized) { + // This test will set a maximum allowed space for the DB, then it will + // keep filling the DB until the limit is reached and bg_error_ is set. + // When bg_error_ is set we will verify that the DB size is greater + // than the limit. + + std::vector max_space_limits_mbs = {1, 2, 4, 8, 10}; + decltype(max_space_limits_mbs)::value_type limit_mb_cb; + bool bg_error_set = false; + uint64_t total_sst_files_size = 0; + + std::atomic estimate_multiplier(1); + int reached_max_space_on_flush = 0; + int reached_max_space_on_compaction = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::FlushMemTableToOutputFile:MaxAllowedSpaceReached", + [&](void* arg) { + Status* bg_error = static_cast(arg); + bg_error_set = true; + GetAllSSTFiles(&total_sst_files_size); + reached_max_space_on_flush++; + // low limit for size calculated using sst files + ASSERT_GE(total_sst_files_size, limit_mb_cb * 1024 * 1024); + // clear error to ensure compaction callback is called + *bg_error = Status::OK(); + estimate_multiplier++; // used in the main loop assert + }); + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "CompactionJob::FinishCompactionOutputFile:MaxAllowedSpaceReached", + [&](void* arg) { + bg_error_set = true; + GetAllSSTFiles(&total_sst_files_size); + reached_max_space_on_compaction++; + }); + + for (auto limit_mb : max_space_limits_mbs) { + bg_error_set = false; + total_sst_files_size = 0; + estimate_multiplier = 1; + limit_mb_cb = limit_mb; + rocksdb::SyncPoint::GetInstance()->ClearTrace(); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + std::shared_ptr sst_file_manager(NewSstFileManager(env_)); + auto sfm = static_cast(sst_file_manager.get()); + + Options options = CurrentOptions(); + options.sst_file_manager = sst_file_manager; + options.write_buffer_size = 1024 * 512; // 512 Kb + DestroyAndReopen(options); + Random rnd(301); + + sfm->SetMaxAllowedSpaceUsage(limit_mb * 1024 * 1024); + + int keys_written = 0; + uint64_t estimated_db_size = 0; + while (true) { + auto s = Put(RandomString(&rnd, 10), RandomString(&rnd, 50)); + if (!s.ok()) { + break; + } + keys_written++; + // Check the estimated db size vs the db limit just to make sure we + // dont run into an infinite loop + estimated_db_size = keys_written * 60; // ~60 bytes per key + ASSERT_LT(estimated_db_size, + estimate_multiplier * limit_mb * 1024 * 1024 * 2); + } + ASSERT_TRUE(bg_error_set); + ASSERT_GE(total_sst_files_size, limit_mb * 1024 * 1024); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + } + + ASSERT_GT(reached_max_space_on_flush, 0); + ASSERT_GT(reached_max_space_on_compaction, 0); +} + +TEST_F(DBSSTTest, OpenDBWithInfiniteMaxOpenFiles) { + // Open DB with infinite max open files + // - First iteration use 1 thread to open files + // - Second iteration use 5 threads to open files + for (int iter = 0; iter < 2; iter++) { + Options options; + options.create_if_missing = true; + options.write_buffer_size = 100000; + options.disable_auto_compactions = true; + options.max_open_files = -1; + if (iter == 0) { + options.max_file_opening_threads = 1; + } else { + options.max_file_opening_threads = 5; + } + options = CurrentOptions(options); + DestroyAndReopen(options); + + // Create 12 Files in L0 (then move then to L2) + for (int i = 0; i < 12; i++) { + std::string k = "L2_" + Key(i); + ASSERT_OK(Put(k, k + std::string(1000, 'a'))); + ASSERT_OK(Flush()); + } + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 2; + db_->CompactRange(compact_options, nullptr, nullptr); + + // Create 12 Files in L0 + for (int i = 0; i < 12; i++) { + std::string k = "L0_" + Key(i); + ASSERT_OK(Put(k, k + std::string(1000, 'a'))); + ASSERT_OK(Flush()); + } + Close(); + + // Reopening the DB will load all existing files + Reopen(options); + ASSERT_EQ("12,0,12", FilesPerLevel(0)); + std::vector> files; + dbfull()->TEST_GetFilesMetaData(db_->DefaultColumnFamily(), &files); + + for (const auto& level : files) { + for (const auto& file : level) { + ASSERT_TRUE(file.table_reader_handle != nullptr); + } + } + + for (int i = 0; i < 12; i++) { + ASSERT_EQ(Get("L0_" + Key(i)), "L0_" + Key(i) + std::string(1000, 'a')); + ASSERT_EQ(Get("L2_" + Key(i)), "L2_" + Key(i) + std::string(1000, 'a')); + } + } +} + +TEST_F(DBSSTTest, GetTotalSstFilesSize) { + // We don't propagate oldest-key-time table property on compaction and + // just write 0 as default value. This affect the exact table size, since + // we encode table properties as varint64. Force time to be 0 to work around + // it. Should remove the workaround after we propagate the property on + // compaction. + std::unique_ptr mock_env(new MockTimeEnv(Env::Default())); + mock_env->set_current_time(0); + + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.compression = kNoCompression; + options.env = mock_env.get(); + DestroyAndReopen(options); + // Generate 5 files in L0 + for (int i = 0; i < 5; i++) { + for (int j = 0; j < 10; j++) { + std::string val = "val_file_" + ToString(i); + ASSERT_OK(Put(Key(j), val)); + } + Flush(); + } + ASSERT_EQ("5", FilesPerLevel(0)); + + std::vector live_files_meta; + dbfull()->GetLiveFilesMetaData(&live_files_meta); + ASSERT_EQ(live_files_meta.size(), 5); + uint64_t single_file_size = live_files_meta[0].size; + + uint64_t live_sst_files_size = 0; + uint64_t total_sst_files_size = 0; + for (const auto& file_meta : live_files_meta) { + live_sst_files_size += file_meta.size; + } + + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.total-sst-files-size", + &total_sst_files_size)); + // Live SST files = 5 + // Total SST files = 5 + ASSERT_EQ(live_sst_files_size, 5 * single_file_size); + ASSERT_EQ(total_sst_files_size, 5 * single_file_size); + + // hold current version + std::unique_ptr iter1(dbfull()->NewIterator(ReadOptions())); + + // Compact 5 files into 1 file in L0 + ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ("0,1", FilesPerLevel(0)); + + live_files_meta.clear(); + dbfull()->GetLiveFilesMetaData(&live_files_meta); + ASSERT_EQ(live_files_meta.size(), 1); + + live_sst_files_size = 0; + total_sst_files_size = 0; + for (const auto& file_meta : live_files_meta) { + live_sst_files_size += file_meta.size; + } + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.total-sst-files-size", + &total_sst_files_size)); + // Live SST files = 1 (compacted file) + // Total SST files = 6 (5 original files + compacted file) + ASSERT_EQ(live_sst_files_size, 1 * single_file_size); + ASSERT_EQ(total_sst_files_size, 6 * single_file_size); + + // hold current version + std::unique_ptr iter2(dbfull()->NewIterator(ReadOptions())); + + // Delete all keys and compact, this will delete all live files + for (int i = 0; i < 10; i++) { + ASSERT_OK(Delete(Key(i))); + } + Flush(); + ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ("", FilesPerLevel(0)); + + live_files_meta.clear(); + dbfull()->GetLiveFilesMetaData(&live_files_meta); + ASSERT_EQ(live_files_meta.size(), 0); + + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.total-sst-files-size", + &total_sst_files_size)); + // Live SST files = 0 + // Total SST files = 6 (5 original files + compacted file) + ASSERT_EQ(total_sst_files_size, 6 * single_file_size); + + iter1.reset(); + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.total-sst-files-size", + &total_sst_files_size)); + // Live SST files = 0 + // Total SST files = 1 (compacted file) + ASSERT_EQ(total_sst_files_size, 1 * single_file_size); + + iter2.reset(); + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.total-sst-files-size", + &total_sst_files_size)); + // Live SST files = 0 + // Total SST files = 0 + ASSERT_EQ(total_sst_files_size, 0); + + // Close db before mock_env destruct. + Close(); +} + +TEST_F(DBSSTTest, GetTotalSstFilesSizeVersionsFilesShared) { + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.compression = kNoCompression; + DestroyAndReopen(options); + // Generate 5 files in L0 + for (int i = 0; i < 5; i++) { + ASSERT_OK(Put(Key(i), "val")); + Flush(); + } + ASSERT_EQ("5", FilesPerLevel(0)); + + std::vector live_files_meta; + dbfull()->GetLiveFilesMetaData(&live_files_meta); + ASSERT_EQ(live_files_meta.size(), 5); + uint64_t single_file_size = live_files_meta[0].size; + + uint64_t live_sst_files_size = 0; + uint64_t total_sst_files_size = 0; + for (const auto& file_meta : live_files_meta) { + live_sst_files_size += file_meta.size; + } + + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.total-sst-files-size", + &total_sst_files_size)); + + // Live SST files = 5 + // Total SST files = 5 + ASSERT_EQ(live_sst_files_size, 5 * single_file_size); + ASSERT_EQ(total_sst_files_size, 5 * single_file_size); + + // hold current version + std::unique_ptr iter1(dbfull()->NewIterator(ReadOptions())); + + // Compaction will do trivial move from L0 to L1 + ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ("0,5", FilesPerLevel(0)); + + live_files_meta.clear(); + dbfull()->GetLiveFilesMetaData(&live_files_meta); + ASSERT_EQ(live_files_meta.size(), 5); + + live_sst_files_size = 0; + total_sst_files_size = 0; + for (const auto& file_meta : live_files_meta) { + live_sst_files_size += file_meta.size; + } + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.total-sst-files-size", + &total_sst_files_size)); + // Live SST files = 5 + // Total SST files = 5 (used in 2 version) + ASSERT_EQ(live_sst_files_size, 5 * single_file_size); + ASSERT_EQ(total_sst_files_size, 5 * single_file_size); + + // hold current version + std::unique_ptr iter2(dbfull()->NewIterator(ReadOptions())); + + // Delete all keys and compact, this will delete all live files + for (int i = 0; i < 5; i++) { + ASSERT_OK(Delete(Key(i))); + } + Flush(); + ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + ASSERT_EQ("", FilesPerLevel(0)); + + live_files_meta.clear(); + dbfull()->GetLiveFilesMetaData(&live_files_meta); + ASSERT_EQ(live_files_meta.size(), 0); + + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.total-sst-files-size", + &total_sst_files_size)); + // Live SST files = 0 + // Total SST files = 5 (used in 2 version) + ASSERT_EQ(total_sst_files_size, 5 * single_file_size); + + iter1.reset(); + iter2.reset(); + + ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.total-sst-files-size", + &total_sst_files_size)); + // Live SST files = 0 + // Total SST files = 0 + ASSERT_EQ(total_sst_files_size, 0); +} + +#endif // ROCKSDB_LITE + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_statistics_test.cc b/db/db_statistics_test.cc new file mode 100644 index 00000000000..237a2c68141 --- /dev/null +++ b/db/db_statistics_test.cc @@ -0,0 +1,149 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include + +#include "db/db_test_util.h" +#include "monitoring/thread_status_util.h" +#include "port/stack_trace.h" +#include "rocksdb/statistics.h" + +namespace rocksdb { + +class DBStatisticsTest : public DBTestBase { + public: + DBStatisticsTest() : DBTestBase("/db_statistics_test") {} +}; + +TEST_F(DBStatisticsTest, CompressionStatsTest) { + CompressionType type; + + if (Snappy_Supported()) { + type = kSnappyCompression; + fprintf(stderr, "using snappy\n"); + } else if (Zlib_Supported()) { + type = kZlibCompression; + fprintf(stderr, "using zlib\n"); + } else if (BZip2_Supported()) { + type = kBZip2Compression; + fprintf(stderr, "using bzip2\n"); + } else if (LZ4_Supported()) { + type = kLZ4Compression; + fprintf(stderr, "using lz4\n"); + } else if (XPRESS_Supported()) { + type = kXpressCompression; + fprintf(stderr, "using xpress\n"); + } else if (ZSTD_Supported()) { + type = kZSTD; + fprintf(stderr, "using ZSTD\n"); + } else { + fprintf(stderr, "skipping test, compression disabled\n"); + return; + } + + Options options = CurrentOptions(); + options.compression = type; + options.statistics = rocksdb::CreateDBStatistics(); + options.statistics->stats_level_ = StatsLevel::kExceptTimeForMutex; + DestroyAndReopen(options); + + int kNumKeysWritten = 100000; + + // Check that compressions occur and are counted when compression is turned on + Random rnd(301); + for (int i = 0; i < kNumKeysWritten; ++i) { + // compressible string + ASSERT_OK(Put(Key(i), RandomString(&rnd, 128) + std::string(128, 'a'))); + } + ASSERT_OK(Flush()); + ASSERT_GT(options.statistics->getTickerCount(NUMBER_BLOCK_COMPRESSED), 0); + + for (int i = 0; i < kNumKeysWritten; ++i) { + auto r = Get(Key(i)); + } + ASSERT_GT(options.statistics->getTickerCount(NUMBER_BLOCK_DECOMPRESSED), 0); + + options.compression = kNoCompression; + DestroyAndReopen(options); + uint64_t currentCompressions = + options.statistics->getTickerCount(NUMBER_BLOCK_COMPRESSED); + uint64_t currentDecompressions = + options.statistics->getTickerCount(NUMBER_BLOCK_DECOMPRESSED); + + // Check that compressions do not occur when turned off + for (int i = 0; i < kNumKeysWritten; ++i) { + // compressible string + ASSERT_OK(Put(Key(i), RandomString(&rnd, 128) + std::string(128, 'a'))); + } + ASSERT_OK(Flush()); + ASSERT_EQ(options.statistics->getTickerCount(NUMBER_BLOCK_COMPRESSED) + - currentCompressions, 0); + + for (int i = 0; i < kNumKeysWritten; ++i) { + auto r = Get(Key(i)); + } + ASSERT_EQ(options.statistics->getTickerCount(NUMBER_BLOCK_DECOMPRESSED) + - currentDecompressions, 0); +} + +TEST_F(DBStatisticsTest, MutexWaitStatsDisabledByDefault) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + CreateAndReopenWithCF({"pikachu"}, options); + const uint64_t kMutexWaitDelay = 100; + ThreadStatusUtil::TEST_SetStateDelay(ThreadStatus::STATE_MUTEX_WAIT, + kMutexWaitDelay); + ASSERT_OK(Put("hello", "rocksdb")); + ASSERT_EQ(TestGetTickerCount(options, DB_MUTEX_WAIT_MICROS), 0); + ThreadStatusUtil::TEST_SetStateDelay(ThreadStatus::STATE_MUTEX_WAIT, 0); +} + +TEST_F(DBStatisticsTest, MutexWaitStats) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + options.statistics->stats_level_ = StatsLevel::kAll; + CreateAndReopenWithCF({"pikachu"}, options); + const uint64_t kMutexWaitDelay = 100; + ThreadStatusUtil::TEST_SetStateDelay(ThreadStatus::STATE_MUTEX_WAIT, + kMutexWaitDelay); + ASSERT_OK(Put("hello", "rocksdb")); + ASSERT_GE(TestGetTickerCount(options, DB_MUTEX_WAIT_MICROS), kMutexWaitDelay); + ThreadStatusUtil::TEST_SetStateDelay(ThreadStatus::STATE_MUTEX_WAIT, 0); +} + +TEST_F(DBStatisticsTest, ResetStats) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + DestroyAndReopen(options); + for (int i = 0; i < 2; ++i) { + // pick arbitrary ticker and histogram. On first iteration they're zero + // because db is unused. On second iteration they're zero due to Reset(). + ASSERT_EQ(0, TestGetTickerCount(options, NUMBER_KEYS_WRITTEN)); + HistogramData histogram_data; + options.statistics->histogramData(DB_WRITE, &histogram_data); + ASSERT_EQ(0.0, histogram_data.max); + + if (i == 0) { + // The Put() makes some of the ticker/histogram stats nonzero until we + // Reset(). + ASSERT_OK(Put("hello", "rocksdb")); + ASSERT_EQ(1, TestGetTickerCount(options, NUMBER_KEYS_WRITTEN)); + options.statistics->histogramData(DB_WRITE, &histogram_data); + ASSERT_GT(histogram_data.max, 0.0); + options.statistics->Reset(); + } + } +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_table_properties_test.cc b/db/db_table_properties_test.cc new file mode 100644 index 00000000000..265e9cb2e1a --- /dev/null +++ b/db/db_table_properties_test.cc @@ -0,0 +1,261 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include + +#include "db/db_test_util.h" +#include "port/stack_trace.h" +#include "rocksdb/db.h" +#include "util/testharness.h" +#include "util/testutil.h" + +#ifndef ROCKSDB_LITE + +namespace rocksdb { + +// A helper function that ensures the table properties returned in +// `GetPropertiesOfAllTablesTest` is correct. +// This test assumes entries size is different for each of the tables. +namespace { + +void VerifyTableProperties(DB* db, uint64_t expected_entries_size) { + TablePropertiesCollection props; + ASSERT_OK(db->GetPropertiesOfAllTables(&props)); + + ASSERT_EQ(4U, props.size()); + std::unordered_set unique_entries; + + // Indirect test + uint64_t sum = 0; + for (const auto& item : props) { + unique_entries.insert(item.second->num_entries); + sum += item.second->num_entries; + } + + ASSERT_EQ(props.size(), unique_entries.size()); + ASSERT_EQ(expected_entries_size, sum); +} +} // namespace + +class DBTablePropertiesTest : public DBTestBase { + public: + DBTablePropertiesTest() : DBTestBase("/db_table_properties_test") {} + TablePropertiesCollection TestGetPropertiesOfTablesInRange( + std::vector ranges, std::size_t* num_properties = nullptr, + std::size_t* num_files = nullptr); +}; + +TEST_F(DBTablePropertiesTest, GetPropertiesOfAllTablesTest) { + Options options = CurrentOptions(); + options.level0_file_num_compaction_trigger = 8; + Reopen(options); + // Create 4 tables + for (int table = 0; table < 4; ++table) { + for (int i = 0; i < 10 + table; ++i) { + db_->Put(WriteOptions(), ToString(table * 100 + i), "val"); + } + db_->Flush(FlushOptions()); + } + + // 1. Read table properties directly from file + Reopen(options); + VerifyTableProperties(db_, 10 + 11 + 12 + 13); + + // 2. Put two tables to table cache and + Reopen(options); + // fetch key from 1st and 2nd table, which will internally place that table to + // the table cache. + for (int i = 0; i < 2; ++i) { + Get(ToString(i * 100 + 0)); + } + + VerifyTableProperties(db_, 10 + 11 + 12 + 13); + + // 3. Put all tables to table cache + Reopen(options); + // fetch key from 1st and 2nd table, which will internally place that table to + // the table cache. + for (int i = 0; i < 4; ++i) { + Get(ToString(i * 100 + 0)); + } + VerifyTableProperties(db_, 10 + 11 + 12 + 13); +} + +TablePropertiesCollection +DBTablePropertiesTest::TestGetPropertiesOfTablesInRange( + std::vector ranges, std::size_t* num_properties, + std::size_t* num_files) { + + // Since we deref zero element in the vector it can not be empty + // otherwise we pass an address to some random memory + EXPECT_GT(ranges.size(), 0U); + // run the query + TablePropertiesCollection props; + EXPECT_OK(db_->GetPropertiesOfTablesInRange( + db_->DefaultColumnFamily(), &ranges[0], ranges.size(), &props)); + + // Make sure that we've received properties for those and for those files + // only which fall within requested ranges + std::vector vmd; + db_->GetLiveFilesMetaData(&vmd); + for (auto& md : vmd) { + std::string fn = md.db_path + md.name; + bool in_range = false; + for (auto& r : ranges) { + // smallestkey < limit && largestkey >= start + if (r.limit.compare(md.smallestkey) >= 0 && + r.start.compare(md.largestkey) <= 0) { + in_range = true; + EXPECT_GT(props.count(fn), 0); + } + } + if (!in_range) { + EXPECT_EQ(props.count(fn), 0); + } + } + + if (num_properties) { + *num_properties = props.size(); + } + + if (num_files) { + *num_files = vmd.size(); + } + return props; +} + +TEST_F(DBTablePropertiesTest, GetPropertiesOfTablesInRange) { + // Fixed random sead + Random rnd(301); + + Options options; + options.create_if_missing = true; + options.write_buffer_size = 4096; + options.max_write_buffer_number = 3; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 2; + options.level0_stop_writes_trigger = 4; + options.target_file_size_base = 2048; + options.max_bytes_for_level_base = 10240; + options.max_bytes_for_level_multiplier = 4; + options.hard_pending_compaction_bytes_limit = 16 * 1024; + options.num_levels = 8; + options.env = env_; + + DestroyAndReopen(options); + + // build a decent LSM + for (int i = 0; i < 10000; i++) { + ASSERT_OK(Put(test::RandomKey(&rnd, 5), RandomString(&rnd, 102))); + } + Flush(); + dbfull()->TEST_WaitForCompact(); + if (NumTableFilesAtLevel(0) == 0) { + ASSERT_OK(Put(test::RandomKey(&rnd, 5), RandomString(&rnd, 102))); + Flush(); + } + + db_->PauseBackgroundWork(); + + // Ensure that we have at least L0, L1 and L2 + ASSERT_GT(NumTableFilesAtLevel(0), 0); + ASSERT_GT(NumTableFilesAtLevel(1), 0); + ASSERT_GT(NumTableFilesAtLevel(2), 0); + + // Query the largest range + std::size_t num_properties, num_files; + TestGetPropertiesOfTablesInRange( + {Range(test::RandomKey(&rnd, 5, test::RandomKeyType::SMALLEST), + test::RandomKey(&rnd, 5, test::RandomKeyType::LARGEST))}, + &num_properties, &num_files); + ASSERT_EQ(num_properties, num_files); + + // Query the empty range + TestGetPropertiesOfTablesInRange( + {Range(test::RandomKey(&rnd, 5, test::RandomKeyType::LARGEST), + test::RandomKey(&rnd, 5, test::RandomKeyType::SMALLEST))}, + &num_properties, &num_files); + ASSERT_GT(num_files, 0); + ASSERT_EQ(num_properties, 0); + + // Query the middle rangee + TestGetPropertiesOfTablesInRange( + {Range(test::RandomKey(&rnd, 5, test::RandomKeyType::MIDDLE), + test::RandomKey(&rnd, 5, test::RandomKeyType::LARGEST))}, + &num_properties, &num_files); + ASSERT_GT(num_files, 0); + ASSERT_GT(num_files, num_properties); + ASSERT_GT(num_properties, 0); + + // Query a bunch of random ranges + for (int j = 0; j < 100; j++) { + // create a bunch of ranges + std::vector random_keys; + // Random returns numbers with zero included + // when we pass empty ranges TestGetPropertiesOfTablesInRange() + // derefs random memory in the empty ranges[0] + // so want to be greater than zero and even since + // the below loop requires that random_keys.size() to be even. + auto n = 2 * (rnd.Uniform(50) + 1); + + for (uint32_t i = 0; i < n; ++i) { + random_keys.push_back(test::RandomKey(&rnd, 5)); + } + + ASSERT_GT(random_keys.size(), 0U); + ASSERT_EQ((random_keys.size() % 2), 0U); + + std::vector ranges; + auto it = random_keys.begin(); + while (it != random_keys.end()) { + ranges.push_back(Range(*it, *(it + 1))); + it += 2; + } + + TestGetPropertiesOfTablesInRange(std::move(ranges)); + } +} + +TEST_F(DBTablePropertiesTest, GetColumnFamilyNameProperty) { + std::string kExtraCfName = "pikachu"; + CreateAndReopenWithCF({kExtraCfName}, CurrentOptions()); + + // Create one table per CF, then verify it was created with the column family + // name property. + for (int cf = 0; cf < 2; ++cf) { + Put(cf, "key", "val"); + Flush(cf); + + TablePropertiesCollection fname_to_props; + ASSERT_OK(db_->GetPropertiesOfAllTables(handles_[cf], &fname_to_props)); + ASSERT_EQ(1U, fname_to_props.size()); + + std::string expected_cf_name; + if (cf > 0) { + expected_cf_name = kExtraCfName; + } else { + expected_cf_name = kDefaultColumnFamilyName; + } + ASSERT_EQ(expected_cf_name, + fname_to_props.begin()->second->column_family_name); + ASSERT_EQ(cf, static_cast( + fname_to_props.begin()->second->column_family_id)); + } +} + +} // namespace rocksdb + +#endif // ROCKSDB_LITE + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_tailing_iter_test.cc b/db/db_tailing_iter_test.cc new file mode 100644 index 00000000000..d217828db9d --- /dev/null +++ b/db/db_tailing_iter_test.cc @@ -0,0 +1,814 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +// Introduction of SyncPoint effectively disabled building and running this test +// in Release build. +// which is a pity, it is a good test +#if !defined(ROCKSDB_LITE) + +#include "db/db_test_util.h" +#include "db/forward_iterator.h" +#include "port/stack_trace.h" + +namespace rocksdb { + +class DBTestTailingIterator : public DBTestBase { + public: + DBTestTailingIterator() : DBTestBase("/db_tailing_iterator_test") {} +}; + +TEST_F(DBTestTailingIterator, TailingIteratorSingle) { + ReadOptions read_options; + read_options.tailing = true; + + std::unique_ptr iter(db_->NewIterator(read_options)); + iter->SeekToFirst(); + ASSERT_TRUE(!iter->Valid()); + + // add a record and check that iter can see it + ASSERT_OK(db_->Put(WriteOptions(), "mirko", "fodor")); + iter->SeekToFirst(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().ToString(), "mirko"); + + iter->Next(); + ASSERT_TRUE(!iter->Valid()); +} + +TEST_F(DBTestTailingIterator, TailingIteratorKeepAdding) { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ReadOptions read_options; + read_options.tailing = true; + + std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); + std::string value(1024, 'a'); + + const int num_records = 10000; + for (int i = 0; i < num_records; ++i) { + char buf[32]; + snprintf(buf, sizeof(buf), "%016d", i); + + Slice key(buf, 16); + ASSERT_OK(Put(1, key, value)); + + iter->Seek(key); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(key), 0); + } +} + +TEST_F(DBTestTailingIterator, TailingIteratorSeekToNext) { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ReadOptions read_options; + read_options.tailing = true; + + std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); + std::unique_ptr itern(db_->NewIterator(read_options, handles_[1])); + std::string value(1024, 'a'); + + const int num_records = 1000; + for (int i = 1; i < num_records; ++i) { + char buf1[32]; + char buf2[32]; + snprintf(buf1, sizeof(buf1), "00a0%016d", i * 5); + + Slice key(buf1, 20); + ASSERT_OK(Put(1, key, value)); + + if (i % 100 == 99) { + ASSERT_OK(Flush(1)); + } + + snprintf(buf2, sizeof(buf2), "00a0%016d", i * 5 - 2); + Slice target(buf2, 20); + iter->Seek(target); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(key), 0); + if (i == 1) { + itern->SeekToFirst(); + } else { + itern->Next(); + } + ASSERT_TRUE(itern->Valid()); + ASSERT_EQ(itern->key().compare(key), 0); + } + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + for (int i = 2 * num_records; i > 0; --i) { + char buf1[32]; + char buf2[32]; + snprintf(buf1, sizeof(buf1), "00a0%016d", i * 5); + + Slice key(buf1, 20); + ASSERT_OK(Put(1, key, value)); + + if (i % 100 == 99) { + ASSERT_OK(Flush(1)); + } + + snprintf(buf2, sizeof(buf2), "00a0%016d", i * 5 - 2); + Slice target(buf2, 20); + iter->Seek(target); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(key), 0); + } +} + +TEST_F(DBTestTailingIterator, TailingIteratorTrimSeekToNext) { + const uint64_t k150KB = 150 * 1024; + Options options; + options.write_buffer_size = k150KB; + options.max_write_buffer_number = 3; + options.min_write_buffer_number_to_merge = 2; + options.env = env_; + CreateAndReopenWithCF({"pikachu"}, options); + ReadOptions read_options; + read_options.tailing = true; + int num_iters, deleted_iters; + + char bufe[32]; + snprintf(bufe, sizeof(bufe), "00b0%016d", 0); + Slice keyu(bufe, 20); + read_options.iterate_upper_bound = &keyu; + std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); + std::unique_ptr itern(db_->NewIterator(read_options, handles_[1])); + std::unique_ptr iterh(db_->NewIterator(read_options, handles_[1])); + std::string value(1024, 'a'); + bool file_iters_deleted = false; + bool file_iters_renewed_null = false; + bool file_iters_renewed_copy = false; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "ForwardIterator::SeekInternal:Return", [&](void* arg) { + ForwardIterator* fiter = reinterpret_cast(arg); + ASSERT_TRUE(!file_iters_deleted || + fiter->TEST_CheckDeletedIters(&deleted_iters, &num_iters)); + }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "ForwardIterator::Next:Return", [&](void* arg) { + ForwardIterator* fiter = reinterpret_cast(arg); + ASSERT_TRUE(!file_iters_deleted || + fiter->TEST_CheckDeletedIters(&deleted_iters, &num_iters)); + }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "ForwardIterator::RenewIterators:Null", + [&](void* arg) { file_iters_renewed_null = true; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "ForwardIterator::RenewIterators:Copy", + [&](void* arg) { file_iters_renewed_copy = true; }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + const int num_records = 1000; + for (int i = 1; i < num_records; ++i) { + char buf1[32]; + char buf2[32]; + char buf3[32]; + char buf4[32]; + snprintf(buf1, sizeof(buf1), "00a0%016d", i * 5); + snprintf(buf3, sizeof(buf3), "00b0%016d", i * 5); + + Slice key(buf1, 20); + ASSERT_OK(Put(1, key, value)); + Slice keyn(buf3, 20); + ASSERT_OK(Put(1, keyn, value)); + + if (i % 100 == 99) { + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + if (i == 299) { + file_iters_deleted = true; + } + snprintf(buf4, sizeof(buf4), "00a0%016d", i * 5 / 2); + Slice target(buf4, 20); + iterh->Seek(target); + ASSERT_TRUE(iter->Valid()); + for (int j = (i + 1) * 5 / 2; j < i * 5; j += 5) { + iterh->Next(); + ASSERT_TRUE(iterh->Valid()); + } + if (i == 299) { + file_iters_deleted = false; + } + } + + file_iters_deleted = true; + snprintf(buf2, sizeof(buf2), "00a0%016d", i * 5 - 2); + Slice target(buf2, 20); + iter->Seek(target); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(key), 0); + ASSERT_LE(num_iters, 1); + if (i == 1) { + itern->SeekToFirst(); + } else { + itern->Next(); + } + ASSERT_TRUE(itern->Valid()); + ASSERT_EQ(itern->key().compare(key), 0); + ASSERT_LE(num_iters, 1); + file_iters_deleted = false; + } + ASSERT_TRUE(file_iters_renewed_null); + ASSERT_TRUE(file_iters_renewed_copy); + iter = 0; + itern = 0; + iterh = 0; + BlockBasedTableOptions table_options; + table_options.no_block_cache = true; + table_options.block_cache_compressed = nullptr; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + read_options.read_tier = kBlockCacheTier; + std::unique_ptr iteri(db_->NewIterator(read_options, handles_[1])); + char buf5[32]; + snprintf(buf5, sizeof(buf5), "00a0%016d", (num_records / 2) * 5 - 2); + Slice target1(buf5, 20); + iteri->Seek(target1); + ASSERT_TRUE(iteri->status().IsIncomplete()); + iteri = 0; + + read_options.read_tier = kReadAllTier; + options.table_factory.reset(NewBlockBasedTableFactory()); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + iter.reset(db_->NewIterator(read_options, handles_[1])); + for (int i = 2 * num_records; i > 0; --i) { + char buf1[32]; + char buf2[32]; + snprintf(buf1, sizeof(buf1), "00a0%016d", i * 5); + + Slice key(buf1, 20); + ASSERT_OK(Put(1, key, value)); + + if (i % 100 == 99) { + ASSERT_OK(Flush(1)); + } + + snprintf(buf2, sizeof(buf2), "00a0%016d", i * 5 - 2); + Slice target(buf2, 20); + iter->Seek(target); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(key), 0); + } +} + +TEST_F(DBTestTailingIterator, TailingIteratorDeletes) { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ReadOptions read_options; + read_options.tailing = true; + + std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); + + // write a single record, read it using the iterator, then delete it + ASSERT_OK(Put(1, "0test", "test")); + iter->SeekToFirst(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().ToString(), "0test"); + ASSERT_OK(Delete(1, "0test")); + + // write many more records + const int num_records = 10000; + std::string value(1024, 'A'); + + for (int i = 0; i < num_records; ++i) { + char buf[32]; + snprintf(buf, sizeof(buf), "1%015d", i); + + Slice key(buf, 16); + ASSERT_OK(Put(1, key, value)); + } + + // force a flush to make sure that no records are read from memtable + ASSERT_OK(Flush(1)); + + // skip "0test" + iter->Next(); + + // make sure we can read all new records using the existing iterator + int count = 0; + for (; iter->Valid(); iter->Next(), ++count) ; + + ASSERT_EQ(count, num_records); +} + +TEST_F(DBTestTailingIterator, TailingIteratorPrefixSeek) { + ReadOptions read_options; + read_options.tailing = true; + + Options options = CurrentOptions(); + options.create_if_missing = true; + options.disable_auto_compactions = true; + options.prefix_extractor.reset(NewFixedPrefixTransform(2)); + options.memtable_factory.reset(NewHashSkipListRepFactory(16)); + options.allow_concurrent_memtable_write = false; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); + ASSERT_OK(Put(1, "0101", "test")); + + ASSERT_OK(Flush(1)); + + ASSERT_OK(Put(1, "0202", "test")); + + // Seek(0102) shouldn't find any records since 0202 has a different prefix + iter->Seek("0102"); + ASSERT_TRUE(!iter->Valid()); + + iter->Seek("0202"); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().ToString(), "0202"); + + iter->Next(); + ASSERT_TRUE(!iter->Valid()); +} + +TEST_F(DBTestTailingIterator, TailingIteratorIncomplete) { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ReadOptions read_options; + read_options.tailing = true; + read_options.read_tier = kBlockCacheTier; + + std::string key("key"); + std::string value("value"); + + ASSERT_OK(db_->Put(WriteOptions(), key, value)); + + std::unique_ptr iter(db_->NewIterator(read_options)); + iter->SeekToFirst(); + // we either see the entry or it's not in cache + ASSERT_TRUE(iter->Valid() || iter->status().IsIncomplete()); + + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + iter->SeekToFirst(); + // should still be true after compaction + ASSERT_TRUE(iter->Valid() || iter->status().IsIncomplete()); +} + +TEST_F(DBTestTailingIterator, TailingIteratorSeekToSame) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.write_buffer_size = 1000; + CreateAndReopenWithCF({"pikachu"}, options); + + ReadOptions read_options; + read_options.tailing = true; + + const int NROWS = 10000; + // Write rows with keys 00000, 00002, 00004 etc. + for (int i = 0; i < NROWS; ++i) { + char buf[100]; + snprintf(buf, sizeof(buf), "%05d", 2*i); + std::string key(buf); + std::string value("value"); + ASSERT_OK(db_->Put(WriteOptions(), key, value)); + } + + std::unique_ptr iter(db_->NewIterator(read_options)); + // Seek to 00001. We expect to find 00002. + std::string start_key = "00001"; + iter->Seek(start_key); + ASSERT_TRUE(iter->Valid()); + + std::string found = iter->key().ToString(); + ASSERT_EQ("00002", found); + + // Now seek to the same key. The iterator should remain in the same + // position. + iter->Seek(found); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(found, iter->key().ToString()); +} + +// Sets iterate_upper_bound and verifies that ForwardIterator doesn't call +// Seek() on immutable iterators when target key is >= prev_key and all +// iterators, including the memtable iterator, are over the upper bound. +TEST_F(DBTestTailingIterator, TailingIteratorUpperBound) { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + + const Slice upper_bound("20", 3); + ReadOptions read_options; + read_options.tailing = true; + read_options.iterate_upper_bound = &upper_bound; + + ASSERT_OK(Put(1, "11", "11")); + ASSERT_OK(Put(1, "12", "12")); + ASSERT_OK(Put(1, "22", "22")); + ASSERT_OK(Flush(1)); // flush all those keys to an immutable SST file + + // Add another key to the memtable. + ASSERT_OK(Put(1, "21", "21")); + + std::unique_ptr it(db_->NewIterator(read_options, handles_[1])); + it->Seek("12"); + ASSERT_TRUE(it->Valid()); + ASSERT_EQ("12", it->key().ToString()); + + it->Next(); + // Not valid since "21" is over the upper bound. + ASSERT_FALSE(it->Valid()); + + // This keeps track of the number of times NeedToSeekImmutable() was true. + int immutable_seeks = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "ForwardIterator::SeekInternal:Immutable", + [&](void* arg) { ++immutable_seeks; }); + + // Seek to 13. This should not require any immutable seeks. + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + it->Seek("13"); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + ASSERT_FALSE(it->Valid()); + ASSERT_EQ(0, immutable_seeks); +} + +TEST_F(DBTestTailingIterator, TailingIteratorGap) { + // level 1: [20, 25] [35, 40] + // level 2: [10 - 15] [45 - 50] + // level 3: [20, 30, 40] + // Previously there is a bug in tailing_iterator that if there is a gap in + // lower level, the key will be skipped if it is within the range between + // the largest key of index n file and the smallest key of index n+1 file + // if both file fit in that gap. In this example, 25 < key < 35 + // https://github.com/facebook/rocksdb/issues/1372 + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + + ReadOptions read_options; + read_options.tailing = true; + + ASSERT_OK(Put(1, "20", "20")); + ASSERT_OK(Put(1, "30", "30")); + ASSERT_OK(Put(1, "40", "40")); + ASSERT_OK(Flush(1)); + MoveFilesToLevel(3, 1); + + ASSERT_OK(Put(1, "10", "10")); + ASSERT_OK(Put(1, "15", "15")); + ASSERT_OK(Flush(1)); + ASSERT_OK(Put(1, "45", "45")); + ASSERT_OK(Put(1, "50", "50")); + ASSERT_OK(Flush(1)); + MoveFilesToLevel(2, 1); + + ASSERT_OK(Put(1, "20", "20")); + ASSERT_OK(Put(1, "25", "25")); + ASSERT_OK(Flush(1)); + ASSERT_OK(Put(1, "35", "35")); + ASSERT_OK(Put(1, "40", "40")); + ASSERT_OK(Flush(1)); + MoveFilesToLevel(1, 1); + + ColumnFamilyMetaData meta; + db_->GetColumnFamilyMetaData(handles_[1], &meta); + + std::unique_ptr it(db_->NewIterator(read_options, handles_[1])); + it->Seek("30"); + ASSERT_TRUE(it->Valid()); + ASSERT_EQ("30", it->key().ToString()); + + it->Next(); + ASSERT_TRUE(it->Valid()); + ASSERT_EQ("35", it->key().ToString()); + + it->Next(); + ASSERT_TRUE(it->Valid()); + ASSERT_EQ("40", it->key().ToString()); +} + +TEST_F(DBTestTailingIterator, ManagedTailingIteratorSingle) { + ReadOptions read_options; + read_options.tailing = true; + read_options.managed = true; + + std::unique_ptr iter(db_->NewIterator(read_options)); + iter->SeekToFirst(); + ASSERT_TRUE(!iter->Valid()); + + // add a record and check that iter can see it + ASSERT_OK(db_->Put(WriteOptions(), "mirko", "fodor")); + iter->SeekToFirst(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().ToString(), "mirko"); + + iter->Next(); + ASSERT_TRUE(!iter->Valid()); +} + +TEST_F(DBTestTailingIterator, ManagedTailingIteratorKeepAdding) { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ReadOptions read_options; + read_options.tailing = true; + read_options.managed = true; + + std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); + std::string value(1024, 'a'); + + const int num_records = 10000; + for (int i = 0; i < num_records; ++i) { + char buf[32]; + snprintf(buf, sizeof(buf), "%016d", i); + + Slice key(buf, 16); + ASSERT_OK(Put(1, key, value)); + + iter->Seek(key); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(key), 0); + } +} + +TEST_F(DBTestTailingIterator, ManagedTailingIteratorSeekToNext) { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ReadOptions read_options; + read_options.tailing = true; + read_options.managed = true; + + std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); + std::string value(1024, 'a'); + + const int num_records = 1000; + for (int i = 1; i < num_records; ++i) { + char buf1[32]; + char buf2[32]; + snprintf(buf1, sizeof(buf1), "00a0%016d", i * 5); + + Slice key(buf1, 20); + ASSERT_OK(Put(1, key, value)); + + if (i % 100 == 99) { + ASSERT_OK(Flush(1)); + } + + snprintf(buf2, sizeof(buf2), "00a0%016d", i * 5 - 2); + Slice target(buf2, 20); + iter->Seek(target); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(key), 0); + } + for (int i = 2 * num_records; i > 0; --i) { + char buf1[32]; + char buf2[32]; + snprintf(buf1, sizeof(buf1), "00a0%016d", i * 5); + + Slice key(buf1, 20); + ASSERT_OK(Put(1, key, value)); + + if (i % 100 == 99) { + ASSERT_OK(Flush(1)); + } + + snprintf(buf2, sizeof(buf2), "00a0%016d", i * 5 - 2); + Slice target(buf2, 20); + iter->Seek(target); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(key), 0); + } +} + +TEST_F(DBTestTailingIterator, ManagedTailingIteratorDeletes) { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ReadOptions read_options; + read_options.tailing = true; + read_options.managed = true; + + std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); + + // write a single record, read it using the iterator, then delete it + ASSERT_OK(Put(1, "0test", "test")); + iter->SeekToFirst(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().ToString(), "0test"); + ASSERT_OK(Delete(1, "0test")); + + // write many more records + const int num_records = 10000; + std::string value(1024, 'A'); + + for (int i = 0; i < num_records; ++i) { + char buf[32]; + snprintf(buf, sizeof(buf), "1%015d", i); + + Slice key(buf, 16); + ASSERT_OK(Put(1, key, value)); + } + + // force a flush to make sure that no records are read from memtable + ASSERT_OK(Flush(1)); + + // skip "0test" + iter->Next(); + + // make sure we can read all new records using the existing iterator + int count = 0; + for (; iter->Valid(); iter->Next(), ++count) { + } + + ASSERT_EQ(count, num_records); +} + +TEST_F(DBTestTailingIterator, ManagedTailingIteratorPrefixSeek) { + ReadOptions read_options; + read_options.tailing = true; + read_options.managed = true; + + Options options = CurrentOptions(); + options.create_if_missing = true; + options.disable_auto_compactions = true; + options.prefix_extractor.reset(NewFixedPrefixTransform(2)); + options.memtable_factory.reset(NewHashSkipListRepFactory(16)); + options.allow_concurrent_memtable_write = false; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); + ASSERT_OK(Put(1, "0101", "test")); + + ASSERT_OK(Flush(1)); + + ASSERT_OK(Put(1, "0202", "test")); + + // Seek(0102) shouldn't find any records since 0202 has a different prefix + iter->Seek("0102"); + ASSERT_TRUE(!iter->Valid()); + + iter->Seek("0202"); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().ToString(), "0202"); + + iter->Next(); + ASSERT_TRUE(!iter->Valid()); +} + +TEST_F(DBTestTailingIterator, ManagedTailingIteratorIncomplete) { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ReadOptions read_options; + read_options.tailing = true; + read_options.managed = true; + read_options.read_tier = kBlockCacheTier; + + std::string key = "key"; + std::string value = "value"; + + ASSERT_OK(db_->Put(WriteOptions(), key, value)); + + std::unique_ptr iter(db_->NewIterator(read_options)); + iter->SeekToFirst(); + // we either see the entry or it's not in cache + ASSERT_TRUE(iter->Valid() || iter->status().IsIncomplete()); + + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + iter->SeekToFirst(); + // should still be true after compaction + ASSERT_TRUE(iter->Valid() || iter->status().IsIncomplete()); +} + +TEST_F(DBTestTailingIterator, ManagedTailingIteratorSeekToSame) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.write_buffer_size = 1000; + CreateAndReopenWithCF({"pikachu"}, options); + + ReadOptions read_options; + read_options.tailing = true; + read_options.managed = true; + + const int NROWS = 10000; + // Write rows with keys 00000, 00002, 00004 etc. + for (int i = 0; i < NROWS; ++i) { + char buf[100]; + snprintf(buf, sizeof(buf), "%05d", 2 * i); + std::string key(buf); + std::string value("value"); + ASSERT_OK(db_->Put(WriteOptions(), key, value)); + } + + std::unique_ptr iter(db_->NewIterator(read_options)); + // Seek to 00001. We expect to find 00002. + std::string start_key = "00001"; + iter->Seek(start_key); + ASSERT_TRUE(iter->Valid()); + + std::string found = iter->key().ToString(); + ASSERT_EQ("00002", found); + + // Now seek to the same key. The iterator should remain in the same + // position. + iter->Seek(found); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(found, iter->key().ToString()); +} + +TEST_F(DBTestTailingIterator, ForwardIteratorVersionProperty) { + Options options = CurrentOptions(); + options.write_buffer_size = 1000; + + ReadOptions read_options; + read_options.tailing = true; + + Put("foo", "bar"); + + uint64_t v1, v2, v3, v4; + { + std::unique_ptr iter(db_->NewIterator(read_options)); + iter->Seek("foo"); + std::string prop_value; + ASSERT_OK(iter->GetProperty("rocksdb.iterator.super-version-number", + &prop_value)); + v1 = static_cast(std::atoi(prop_value.c_str())); + + Put("foo1", "bar1"); + Flush(); + + ASSERT_OK(iter->GetProperty("rocksdb.iterator.super-version-number", + &prop_value)); + v2 = static_cast(std::atoi(prop_value.c_str())); + + iter->Seek("f"); + + ASSERT_OK(iter->GetProperty("rocksdb.iterator.super-version-number", + &prop_value)); + v3 = static_cast(std::atoi(prop_value.c_str())); + + ASSERT_EQ(v1, v2); + ASSERT_GT(v3, v2); + } + + { + std::unique_ptr iter(db_->NewIterator(read_options)); + iter->Seek("foo"); + std::string prop_value; + ASSERT_OK(iter->GetProperty("rocksdb.iterator.super-version-number", + &prop_value)); + v4 = static_cast(std::atoi(prop_value.c_str())); + } + ASSERT_EQ(v3, v4); +} + +TEST_F(DBTestTailingIterator, SeekWithUpperBoundBug) { + ReadOptions read_options; + read_options.tailing = true; + const Slice upper_bound("cc", 3); + read_options.iterate_upper_bound = &upper_bound; + + + // 1st L0 file + ASSERT_OK(db_->Put(WriteOptions(), "aa", "SEEN")); + ASSERT_OK(Flush()); + + // 2nd L0 file + ASSERT_OK(db_->Put(WriteOptions(), "zz", "NOT-SEEN")); + ASSERT_OK(Flush()); + + std::unique_ptr iter(db_->NewIterator(read_options)); + + iter->Seek("aa"); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().ToString(), "aa"); +} + +TEST_F(DBTestTailingIterator, SeekToFirstWithUpperBoundBug) { + ReadOptions read_options; + read_options.tailing = true; + const Slice upper_bound("cc", 3); + read_options.iterate_upper_bound = &upper_bound; + + + // 1st L0 file + ASSERT_OK(db_->Put(WriteOptions(), "aa", "SEEN")); + ASSERT_OK(Flush()); + + // 2nd L0 file + ASSERT_OK(db_->Put(WriteOptions(), "zz", "NOT-SEEN")); + ASSERT_OK(Flush()); + + std::unique_ptr iter(db_->NewIterator(read_options)); + + iter->SeekToFirst(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().ToString(), "aa"); + + iter->Next(); + ASSERT_FALSE(iter->Valid()); + + iter->SeekToFirst(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().ToString(), "aa"); +} + +} // namespace rocksdb + +#endif // !defined(ROCKSDB_LITE) + +int main(int argc, char** argv) { +#if !defined(ROCKSDB_LITE) + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +#else + return 0; +#endif +} diff --git a/db/db_test.cc b/db/db_test.cc index 6295f5921ee..193101d460d 100644 --- a/db/db_test.cc +++ b/db/db_test.cc @@ -1,1418 +1,560 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. +// Introduction of SyncPoint effectively disabled building and running this test +// in Release build. +// which is a pity, it is a good test +#include #include -#include #include -#include +#include #include #include +#ifndef OS_WIN +#include +#endif +#ifdef OS_SOLARIS +#include +#endif -#include "db/dbformat.h" +#include "cache/lru_cache.h" #include "db/db_impl.h" -#include "db/filename.h" +#include "db/db_test_util.h" +#include "db/dbformat.h" +#include "db/job_context.h" #include "db/version_set.h" #include "db/write_batch_internal.h" +#include "env/mock_env.h" +#include "memtable/hash_linklist_rep.h" +#include "monitoring/thread_status_util.h" +#include "port/port.h" +#include "port/stack_trace.h" #include "rocksdb/cache.h" #include "rocksdb/compaction_filter.h" +#include "rocksdb/convenience.h" #include "rocksdb/db.h" #include "rocksdb/env.h" +#include "rocksdb/experimental.h" #include "rocksdb/filter_policy.h" +#include "rocksdb/options.h" #include "rocksdb/perf_context.h" #include "rocksdb/slice.h" #include "rocksdb/slice_transform.h" +#include "rocksdb/snapshot.h" #include "rocksdb/table.h" -#include "rocksdb/options.h" #include "rocksdb/table_properties.h" +#include "rocksdb/thread_status.h" +#include "rocksdb/utilities/checkpoint.h" +#include "rocksdb/utilities/optimistic_transaction_db.h" #include "rocksdb/utilities/write_batch_with_index.h" #include "table/block_based_table_factory.h" +#include "table/mock_table.h" #include "table/plain_table_factory.h" +#include "table/scoped_arena_iterator.h" +#include "util/compression.h" +#include "util/file_reader_writer.h" +#include "util/filename.h" #include "util/hash.h" -#include "util/hash_linklist_rep.h" -#include "utilities/merge_operators.h" -#include "util/logging.h" #include "util/mutexlock.h" #include "util/rate_limiter.h" -#include "util/statistics.h" -#include "util/testharness.h" +#include "util/string_util.h" #include "util/sync_point.h" +#include "util/testharness.h" #include "util/testutil.h" +#include "utilities/merge_operators.h" namespace rocksdb { -static bool SnappyCompressionSupported(const CompressionOptions& options) { - std::string out; - Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; - return port::Snappy_Compress(options, in.data(), in.size(), &out); -} +class DBTest : public DBTestBase { + public: + DBTest() : DBTestBase("/db_test") {} +}; -static bool ZlibCompressionSupported(const CompressionOptions& options) { - std::string out; - Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; - return port::Zlib_Compress(options, in.data(), in.size(), &out); -} +class DBTestWithParam + : public DBTest, + public testing::WithParamInterface> { + public: + DBTestWithParam() { + max_subcompactions_ = std::get<0>(GetParam()); + exclusive_manual_compaction_ = std::get<1>(GetParam()); + } -static bool BZip2CompressionSupported(const CompressionOptions& options) { - std::string out; - Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; - return port::BZip2_Compress(options, in.data(), in.size(), &out); -} + // Required if inheriting from testing::WithParamInterface<> + static void SetUpTestCase() {} + static void TearDownTestCase() {} -static bool LZ4CompressionSupported(const CompressionOptions &options) { - std::string out; - Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; - return port::LZ4_Compress(options, in.data(), in.size(), &out); -} + uint32_t max_subcompactions_; + bool exclusive_manual_compaction_; +}; -static bool LZ4HCCompressionSupported(const CompressionOptions &options) { - std::string out; - Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; - return port::LZ4HC_Compress(options, in.data(), in.size(), &out); -} +TEST_F(DBTest, MockEnvTest) { + unique_ptr env{new MockEnv(Env::Default())}; + Options options; + options.create_if_missing = true; + options.env = env.get(); + DB* db; -static std::string RandomString(Random *rnd, int len) { - std::string r; - test::RandomString(rnd, len, &r); - return r; -} + const Slice keys[] = {Slice("aaa"), Slice("bbb"), Slice("ccc")}; + const Slice vals[] = {Slice("foo"), Slice("bar"), Slice("baz")}; -namespace anon { -class AtomicCounter { - private: - port::Mutex mu_; - int count_; - public: - AtomicCounter() : count_(0) { } - void Increment() { - MutexLock l(&mu_); - count_++; + ASSERT_OK(DB::Open(options, "/dir/db", &db)); + for (size_t i = 0; i < 3; ++i) { + ASSERT_OK(db->Put(WriteOptions(), keys[i], vals[i])); } - int Read() { - MutexLock l(&mu_); - return count_; + + for (size_t i = 0; i < 3; ++i) { + std::string res; + ASSERT_OK(db->Get(ReadOptions(), keys[i], &res)); + ASSERT_TRUE(res == vals[i]); } - void Reset() { - MutexLock l(&mu_); - count_ = 0; + + Iterator* iterator = db->NewIterator(ReadOptions()); + iterator->SeekToFirst(); + for (size_t i = 0; i < 3; ++i) { + ASSERT_TRUE(iterator->Valid()); + ASSERT_TRUE(keys[i] == iterator->key()); + ASSERT_TRUE(vals[i] == iterator->value()); + iterator->Next(); } -}; + ASSERT_TRUE(!iterator->Valid()); + delete iterator; -struct OptionsOverride { - std::shared_ptr filter_policy = nullptr; -}; +// TEST_FlushMemTable() is not supported in ROCKSDB_LITE +#ifndef ROCKSDB_LITE + DBImpl* dbi = reinterpret_cast(db); + ASSERT_OK(dbi->TEST_FlushMemTable()); -} // namespace anon + for (size_t i = 0; i < 3; ++i) { + std::string res; + ASSERT_OK(db->Get(ReadOptions(), keys[i], &res)); + ASSERT_TRUE(res == vals[i]); + } +#endif // ROCKSDB_LITE -static std::string Key(int i) { - char buf[100]; - snprintf(buf, sizeof(buf), "key%06d", i); - return std::string(buf); + delete db; } -// Special Env used to delay background operations -class SpecialEnv : public EnvWrapper { - public: - // sstable Sync() calls are blocked while this pointer is non-nullptr. - port::AtomicPointer delay_sstable_sync_; - - // Simulate no-space errors while this pointer is non-nullptr. - port::AtomicPointer no_space_; - - // Simulate non-writable file system while this pointer is non-nullptr - port::AtomicPointer non_writable_; +// NewMemEnv returns nullptr in ROCKSDB_LITE since class InMemoryEnv isn't +// defined. +#ifndef ROCKSDB_LITE +TEST_F(DBTest, MemEnvTest) { + unique_ptr env{NewMemEnv(Env::Default())}; + Options options; + options.create_if_missing = true; + options.env = env.get(); + DB* db; - // Force sync of manifest files to fail while this pointer is non-nullptr - port::AtomicPointer manifest_sync_error_; + const Slice keys[] = {Slice("aaa"), Slice("bbb"), Slice("ccc")}; + const Slice vals[] = {Slice("foo"), Slice("bar"), Slice("baz")}; - // Force write to manifest files to fail while this pointer is non-nullptr - port::AtomicPointer manifest_write_error_; + ASSERT_OK(DB::Open(options, "/dir/db", &db)); + for (size_t i = 0; i < 3; ++i) { + ASSERT_OK(db->Put(WriteOptions(), keys[i], vals[i])); + } - // Force write to log files to fail while this pointer is non-nullptr - port::AtomicPointer log_write_error_; + for (size_t i = 0; i < 3; ++i) { + std::string res; + ASSERT_OK(db->Get(ReadOptions(), keys[i], &res)); + ASSERT_TRUE(res == vals[i]); + } - bool count_random_reads_; - anon::AtomicCounter random_read_counter_; + Iterator* iterator = db->NewIterator(ReadOptions()); + iterator->SeekToFirst(); + for (size_t i = 0; i < 3; ++i) { + ASSERT_TRUE(iterator->Valid()); + ASSERT_TRUE(keys[i] == iterator->key()); + ASSERT_TRUE(vals[i] == iterator->value()); + iterator->Next(); + } + ASSERT_TRUE(!iterator->Valid()); + delete iterator; - bool count_sequential_reads_; - anon::AtomicCounter sequential_read_counter_; + DBImpl* dbi = reinterpret_cast(db); + ASSERT_OK(dbi->TEST_FlushMemTable()); - anon::AtomicCounter sleep_counter_; + for (size_t i = 0; i < 3; ++i) { + std::string res; + ASSERT_OK(db->Get(ReadOptions(), keys[i], &res)); + ASSERT_TRUE(res == vals[i]); + } - std::atomic bytes_written_; + delete db; - explicit SpecialEnv(Env* base) : EnvWrapper(base) { - delay_sstable_sync_.Release_Store(nullptr); - no_space_.Release_Store(nullptr); - non_writable_.Release_Store(nullptr); - count_random_reads_ = false; - count_sequential_reads_ = false; - manifest_sync_error_.Release_Store(nullptr); - manifest_write_error_.Release_Store(nullptr); - log_write_error_.Release_Store(nullptr); - bytes_written_ = 0; + options.create_if_missing = false; + ASSERT_OK(DB::Open(options, "/dir/db", &db)); + for (size_t i = 0; i < 3; ++i) { + std::string res; + ASSERT_OK(db->Get(ReadOptions(), keys[i], &res)); + ASSERT_TRUE(res == vals[i]); } + delete db; +} +#endif // ROCKSDB_LITE - Status NewWritableFile(const std::string& f, unique_ptr* r, - const EnvOptions& soptions) { - class SSTableFile : public WritableFile { - private: - SpecialEnv* env_; - unique_ptr base_; +TEST_F(DBTest, WriteEmptyBatch) { + Options options = CurrentOptions(); + options.env = env_; + options.write_buffer_size = 100000; + CreateAndReopenWithCF({"pikachu"}, options); - public: - SSTableFile(SpecialEnv* env, unique_ptr&& base) - : env_(env), - base_(std::move(base)) { - } - Status Append(const Slice& data) { - if (env_->no_space_.Acquire_Load() != nullptr) { - // Drop writes on the floor - return Status::OK(); - } else { - env_->bytes_written_ += data.size(); - return base_->Append(data); - } - } - Status Close() { return base_->Close(); } - Status Flush() { return base_->Flush(); } - Status Sync() { - while (env_->delay_sstable_sync_.Acquire_Load() != nullptr) { - env_->SleepForMicroseconds(100000); - } - return base_->Sync(); - } - void SetIOPriority(Env::IOPriority pri) { - base_->SetIOPriority(pri); - } - }; - class ManifestFile : public WritableFile { - private: - SpecialEnv* env_; - unique_ptr base_; - public: - ManifestFile(SpecialEnv* env, unique_ptr&& b) - : env_(env), base_(std::move(b)) { } - Status Append(const Slice& data) { - if (env_->manifest_write_error_.Acquire_Load() != nullptr) { - return Status::IOError("simulated writer error"); - } else { - return base_->Append(data); - } - } - Status Close() { return base_->Close(); } - Status Flush() { return base_->Flush(); } - Status Sync() { - if (env_->manifest_sync_error_.Acquire_Load() != nullptr) { - return Status::IOError("simulated sync error"); - } else { - return base_->Sync(); - } - } - }; - class LogFile : public WritableFile { - private: - SpecialEnv* env_; - unique_ptr base_; - public: - LogFile(SpecialEnv* env, unique_ptr&& b) - : env_(env), base_(std::move(b)) { } - Status Append(const Slice& data) { - if (env_->log_write_error_.Acquire_Load() != nullptr) { - return Status::IOError("simulated writer error"); - } else { - return base_->Append(data); - } - } - Status Close() { return base_->Close(); } - Status Flush() { return base_->Flush(); } - Status Sync() { return base_->Sync(); } - }; + ASSERT_OK(Put(1, "foo", "bar")); + WriteOptions wo; + wo.sync = true; + wo.disableWAL = false; + WriteBatch empty_batch; + ASSERT_OK(dbfull()->Write(wo, &empty_batch)); - if (non_writable_.Acquire_Load() != nullptr) { - return Status::IOError("simulated write error"); - } + // make sure we can re-open it. + ASSERT_OK(TryReopenWithColumnFamilies({"default", "pikachu"}, options)); + ASSERT_EQ("bar", Get(1, "foo")); +} - Status s = target()->NewWritableFile(f, r, soptions); - if (s.ok()) { - if (strstr(f.c_str(), ".sst") != nullptr) { - r->reset(new SSTableFile(this, std::move(*r))); - } else if (strstr(f.c_str(), "MANIFEST") != nullptr) { - r->reset(new ManifestFile(this, std::move(*r))); - } else if (strstr(f.c_str(), "log") != nullptr) { - r->reset(new LogFile(this, std::move(*r))); - } - } - return s; - } +TEST_F(DBTest, SkipDelay) { + Options options = CurrentOptions(); + options.env = env_; + options.write_buffer_size = 100000; + CreateAndReopenWithCF({"pikachu"}, options); + + for (bool sync : {true, false}) { + for (bool disableWAL : {true, false}) { + // Use a small number to ensure a large delay that is still effective + // when we do Put + // TODO(myabandeh): this is time dependent and could potentially make + // the test flaky + auto token = dbfull()->TEST_write_controler().GetDelayToken(1); + std::atomic sleep_count(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::DelayWrite:Sleep", + [&](void* arg) { sleep_count.fetch_add(1); }); + std::atomic wait_count(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::DelayWrite:Wait", + [&](void* arg) { wait_count.fetch_add(1); }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); - Status NewRandomAccessFile(const std::string& f, - unique_ptr* r, - const EnvOptions& soptions) { - class CountingFile : public RandomAccessFile { - private: - unique_ptr target_; - anon::AtomicCounter* counter_; - public: - CountingFile(unique_ptr&& target, - anon::AtomicCounter* counter) - : target_(std::move(target)), counter_(counter) { - } - virtual Status Read(uint64_t offset, size_t n, Slice* result, - char* scratch) const { - counter_->Increment(); - return target_->Read(offset, n, result, scratch); - } - }; + WriteOptions wo; + wo.sync = sync; + wo.disableWAL = disableWAL; + wo.no_slowdown = true; + dbfull()->Put(wo, "foo", "bar"); + // We need the 2nd write to trigger delay. This is because delay is + // estimated based on the last write size which is 0 for the first write. + ASSERT_NOK(dbfull()->Put(wo, "foo2", "bar2")); + ASSERT_GE(sleep_count.load(), 0); + ASSERT_GE(wait_count.load(), 0); + token.reset(); - Status s = target()->NewRandomAccessFile(f, r, soptions); - if (s.ok() && count_random_reads_) { - r->reset(new CountingFile(std::move(*r), &random_read_counter_)); + token = dbfull()->TEST_write_controler().GetDelayToken(1000000000); + wo.no_slowdown = false; + ASSERT_OK(dbfull()->Put(wo, "foo3", "bar3")); + ASSERT_GE(sleep_count.load(), 1); + token.reset(); } - return s; } +} - Status NewSequentialFile(const std::string& f, unique_ptr* r, - const EnvOptions& soptions) { - class CountingFile : public SequentialFile { - private: - unique_ptr target_; - anon::AtomicCounter* counter_; - - public: - CountingFile(unique_ptr&& target, - anon::AtomicCounter* counter) - : target_(std::move(target)), counter_(counter) {} - virtual Status Read(size_t n, Slice* result, char* scratch) { - counter_->Increment(); - return target_->Read(n, result, scratch); - } - virtual Status Skip(uint64_t n) { return target_->Skip(n); } - }; +#ifndef ROCKSDB_LITE - Status s = target()->NewSequentialFile(f, r, soptions); - if (s.ok() && count_sequential_reads_) { - r->reset(new CountingFile(std::move(*r), &sequential_read_counter_)); - } - return s; - } +TEST_F(DBTest, LevelLimitReopen) { + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); - virtual void SleepForMicroseconds(int micros) { - sleep_counter_.Increment(); - target()->SleepForMicroseconds(micros); + const std::string value(1024 * 1024, ' '); + int i = 0; + while (NumTableFilesAtLevel(2, 1) == 0) { + ASSERT_OK(Put(1, Key(i++), value)); } -}; -class DBTest { - protected: - // Sequence of option configurations to try - enum OptionConfig { - kDefault = 0, - kBlockBasedTableWithPrefixHashIndex = 1, - kBlockBasedTableWithWholeKeyHashIndex = 2, - kPlainTableFirstBytePrefix = 3, - kPlainTableAllBytesPrefix = 4, - kVectorRep = 5, - kHashLinkList = 6, - kHashCuckoo = 7, - kMergePut = 8, - kFilter = 9, - kUncompressed = 10, - kNumLevel_3 = 11, - kDBLogDir = 12, - kWalDir = 13, - kManifestFileSize = 14, - kCompactOnFlush = 15, - kPerfOptions = 16, - kDeletesFilterFirst = 17, - kHashSkipList = 18, - kUniversalCompaction = 19, - kCompressedBlockCache = 20, - kInfiniteMaxOpenFiles = 21, - kxxHashChecksum = 22, - kFIFOCompaction = 23, - kEnd = 24 - }; - int option_config_; + options.num_levels = 1; + options.max_bytes_for_level_multiplier_additional.resize(1, 1); + Status s = TryReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_EQ(s.IsInvalidArgument(), true); + ASSERT_EQ(s.ToString(), + "Invalid argument: db has more levels than options.num_levels"); - public: - std::string dbname_; - SpecialEnv* env_; - DB* db_; - std::vector handles_; - - Options last_options_; - - // Skip some options, as they may not be applicable to a specific test. - // To add more skip constants, use values 4, 8, 16, etc. - enum OptionSkip { - kNoSkip = 0, - kSkipDeletesFilterFirst = 1, - kSkipUniversalCompaction = 2, - kSkipMergePut = 4, - kSkipPlainTable = 8, - kSkipHashIndex = 16, - kSkipNoSeekToLast = 32, - kSkipHashCuckoo = 64, - kSkipFIFOCompaction = 128, - }; + options.num_levels = 10; + options.max_bytes_for_level_multiplier_additional.resize(10, 1); + ASSERT_OK(TryReopenWithColumnFamilies({"default", "pikachu"}, options)); +} +#endif // ROCKSDB_LITE - DBTest() : option_config_(kDefault), - env_(new SpecialEnv(Env::Default())) { - dbname_ = test::TmpDir() + "/db_test"; - ASSERT_OK(DestroyDB(dbname_, Options())); - db_ = nullptr; - Reopen(); - } +TEST_F(DBTest, PutSingleDeleteGet) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_OK(Put(1, "foo2", "v2")); + ASSERT_EQ("v2", Get(1, "foo2")); + ASSERT_OK(SingleDelete(1, "foo")); + ASSERT_EQ("NOT_FOUND", Get(1, "foo")); + // Skip HashCuckooRep as it does not support single delete. FIFO and + // universal compaction do not apply to the test case. Skip MergePut + // because single delete does not get removed when it encounters a merge. + } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction | + kSkipUniversalCompaction | kSkipMergePut)); +} - ~DBTest() { - Close(); - Options options; - options.db_paths.emplace_back(dbname_, 0); - options.db_paths.emplace_back(dbname_ + "_2", 0); - options.db_paths.emplace_back(dbname_ + "_3", 0); - options.db_paths.emplace_back(dbname_ + "_4", 0); - ASSERT_OK(DestroyDB(dbname_, options)); - delete env_; - } - - // Switch to a fresh database with the next option configuration to - // test. Return false if there are no more configurations to test. - bool ChangeOptions(int skip_mask = kNoSkip) { - for(option_config_++; option_config_ < kEnd; option_config_++) { - if ((skip_mask & kSkipDeletesFilterFirst) && - option_config_ == kDeletesFilterFirst) { - continue; - } - if ((skip_mask & kSkipUniversalCompaction) && - option_config_ == kUniversalCompaction) { - continue; - } - if ((skip_mask & kSkipMergePut) && option_config_ == kMergePut) { - continue; - } - if ((skip_mask & kSkipNoSeekToLast) && - (option_config_ == kHashLinkList || - option_config_ == kHashSkipList)) {; - continue; - } - if ((skip_mask & kSkipPlainTable) - && (option_config_ == kPlainTableAllBytesPrefix - || option_config_ == kPlainTableFirstBytePrefix)) { - continue; - } - if ((skip_mask & kSkipHashIndex) && - (option_config_ == kBlockBasedTableWithPrefixHashIndex || - option_config_ == kBlockBasedTableWithWholeKeyHashIndex)) { - continue; - } - if ((skip_mask & kSkipHashCuckoo) && (option_config_ == kHashCuckoo)) { - continue; - } - if ((skip_mask & kSkipFIFOCompaction) && - option_config_ == kFIFOCompaction) { - continue; +TEST_F(DBTest, ReadFromPersistedTier) { + do { + Random rnd(301); + Options options = CurrentOptions(); + for (int disableWAL = 0; disableWAL <= 1; ++disableWAL) { + CreateAndReopenWithCF({"pikachu"}, options); + WriteOptions wopt; + wopt.disableWAL = (disableWAL == 1); + // 1st round: put but not flush + ASSERT_OK(db_->Put(wopt, handles_[1], "foo", "first")); + ASSERT_OK(db_->Put(wopt, handles_[1], "bar", "one")); + ASSERT_EQ("first", Get(1, "foo")); + ASSERT_EQ("one", Get(1, "bar")); + + // Read directly from persited data. + ReadOptions ropt; + ropt.read_tier = kPersistedTier; + std::string value; + if (wopt.disableWAL) { + // as data has not yet being flushed, we expect not found. + ASSERT_TRUE(db_->Get(ropt, handles_[1], "foo", &value).IsNotFound()); + ASSERT_TRUE(db_->Get(ropt, handles_[1], "bar", &value).IsNotFound()); + } else { + ASSERT_OK(db_->Get(ropt, handles_[1], "foo", &value)); + ASSERT_OK(db_->Get(ropt, handles_[1], "bar", &value)); } - break; - } - if (option_config_ >= kEnd) { - Destroy(&last_options_); - return false; - } else { - DestroyAndReopen(); - return true; - } - } + // Multiget + std::vector multiget_cfs; + multiget_cfs.push_back(handles_[1]); + multiget_cfs.push_back(handles_[1]); + std::vector multiget_keys; + multiget_keys.push_back("foo"); + multiget_keys.push_back("bar"); + std::vector multiget_values; + auto statuses = + db_->MultiGet(ropt, multiget_cfs, multiget_keys, &multiget_values); + if (wopt.disableWAL) { + ASSERT_TRUE(statuses[0].IsNotFound()); + ASSERT_TRUE(statuses[1].IsNotFound()); + } else { + ASSERT_OK(statuses[0]); + ASSERT_OK(statuses[1]); + } - // Switch between different compaction styles (we have only 2 now). - bool ChangeCompactOptions(Options* prev_options = nullptr) { - if (option_config_ == kDefault) { - option_config_ = kUniversalCompaction; - if (prev_options == nullptr) { - prev_options = &last_options_; + // 2nd round: flush and put a new value in memtable. + ASSERT_OK(Flush(1)); + ASSERT_OK(db_->Put(wopt, handles_[1], "rocksdb", "hello")); + + // once the data has been flushed, we are able to get the + // data when kPersistedTier is used. + ASSERT_TRUE(db_->Get(ropt, handles_[1], "foo", &value).ok()); + ASSERT_EQ(value, "first"); + ASSERT_TRUE(db_->Get(ropt, handles_[1], "bar", &value).ok()); + ASSERT_EQ(value, "one"); + if (wopt.disableWAL) { + ASSERT_TRUE( + db_->Get(ropt, handles_[1], "rocksdb", &value).IsNotFound()); + } else { + ASSERT_OK(db_->Get(ropt, handles_[1], "rocksdb", &value)); + ASSERT_EQ(value, "hello"); } - Destroy(prev_options); - TryReopen(); - return true; - } else { - return false; - } - } - // Return the current option configuration. - Options CurrentOptions( - const anon::OptionsOverride& options_override = anon::OptionsOverride()) { - Options options; - return CurrentOptions(options, options_override); - } - - Options CurrentOptions( - const Options& defaultOptions, - const anon::OptionsOverride& options_override = anon::OptionsOverride()) { - // this redudant copy is to minimize code change w/o having lint error. - Options options = defaultOptions; - BlockBasedTableOptions table_options; - bool set_block_based_table_factory = true; - switch (option_config_) { - case kHashSkipList: - options.prefix_extractor.reset(NewFixedPrefixTransform(1)); - options.memtable_factory.reset( - NewHashSkipListRepFactory(16)); - break; - case kPlainTableFirstBytePrefix: - options.table_factory.reset(new PlainTableFactory()); - options.prefix_extractor.reset(NewFixedPrefixTransform(1)); - options.allow_mmap_reads = true; - options.max_sequential_skip_in_iterations = 999999; - set_block_based_table_factory = false; - break; - case kPlainTableAllBytesPrefix: - options.table_factory.reset(new PlainTableFactory()); - options.prefix_extractor.reset(NewNoopTransform()); - options.allow_mmap_reads = true; - options.max_sequential_skip_in_iterations = 999999; - set_block_based_table_factory = false; - break; - case kMergePut: - options.merge_operator = MergeOperators::CreatePutOperator(); - break; - case kFilter: - table_options.filter_policy.reset(NewBloomFilterPolicy(10)); - break; - case kUncompressed: - options.compression = kNoCompression; - break; - case kNumLevel_3: - options.num_levels = 3; - break; - case kDBLogDir: - options.db_log_dir = test::TmpDir(); - break; - case kWalDir: - options.wal_dir = test::TmpDir() + "/wal"; - break; - case kManifestFileSize: - options.max_manifest_file_size = 50; // 50 bytes - case kCompactOnFlush: - options.purge_redundant_kvs_while_flush = - !options.purge_redundant_kvs_while_flush; - break; - case kPerfOptions: - options.hard_rate_limit = 2.0; - options.rate_limit_delay_max_milliseconds = 2; - // TODO -- test more options - break; - case kDeletesFilterFirst: - options.filter_deletes = true; - break; - case kVectorRep: - options.memtable_factory.reset(new VectorRepFactory(100)); - break; - case kHashLinkList: - options.prefix_extractor.reset(NewFixedPrefixTransform(1)); - options.memtable_factory.reset( - NewHashLinkListRepFactory(4, 0, 3, true, 4)); - break; - case kHashCuckoo: - options.memtable_factory.reset( - NewHashCuckooRepFactory(options.write_buffer_size)); - break; - case kUniversalCompaction: - options.compaction_style = kCompactionStyleUniversal; - break; - case kCompressedBlockCache: - options.allow_mmap_writes = true; - table_options.block_cache_compressed = NewLRUCache(8*1024*1024); - break; - case kInfiniteMaxOpenFiles: - options.max_open_files = -1; - break; - case kxxHashChecksum: { - table_options.checksum = kxxHash; - break; + // Expect same result in multiget + multiget_cfs.push_back(handles_[1]); + multiget_keys.push_back("rocksdb"); + statuses = + db_->MultiGet(ropt, multiget_cfs, multiget_keys, &multiget_values); + ASSERT_TRUE(statuses[0].ok()); + ASSERT_EQ("first", multiget_values[0]); + ASSERT_TRUE(statuses[1].ok()); + ASSERT_EQ("one", multiget_values[1]); + if (wopt.disableWAL) { + ASSERT_TRUE(statuses[2].IsNotFound()); + } else { + ASSERT_OK(statuses[2]); } - case kFIFOCompaction: { - options.compaction_style = kCompactionStyleFIFO; - break; + + // 3rd round: delete and flush + ASSERT_OK(db_->Delete(wopt, handles_[1], "foo")); + Flush(1); + ASSERT_OK(db_->Delete(wopt, handles_[1], "bar")); + + ASSERT_TRUE(db_->Get(ropt, handles_[1], "foo", &value).IsNotFound()); + if (wopt.disableWAL) { + // Still expect finding the value as its delete has not yet being + // flushed. + ASSERT_TRUE(db_->Get(ropt, handles_[1], "bar", &value).ok()); + ASSERT_EQ(value, "one"); + } else { + ASSERT_TRUE(db_->Get(ropt, handles_[1], "bar", &value).IsNotFound()); } - case kBlockBasedTableWithPrefixHashIndex: { - table_options.index_type = BlockBasedTableOptions::kHashSearch; - options.prefix_extractor.reset(NewFixedPrefixTransform(1)); - break; + ASSERT_TRUE(db_->Get(ropt, handles_[1], "rocksdb", &value).ok()); + ASSERT_EQ(value, "hello"); + + statuses = + db_->MultiGet(ropt, multiget_cfs, multiget_keys, &multiget_values); + ASSERT_TRUE(statuses[0].IsNotFound()); + if (wopt.disableWAL) { + ASSERT_TRUE(statuses[1].ok()); + ASSERT_EQ("one", multiget_values[1]); + } else { + ASSERT_TRUE(statuses[1].IsNotFound()); } - case kBlockBasedTableWithWholeKeyHashIndex: { - table_options.index_type = BlockBasedTableOptions::kHashSearch; - options.prefix_extractor.reset(NewNoopTransform()); - break; + ASSERT_TRUE(statuses[2].ok()); + ASSERT_EQ("hello", multiget_values[2]); + if (wopt.disableWAL == 0) { + DestroyAndReopen(options); } - default: - break; - } - - if (options_override.filter_policy) { - table_options.filter_policy = options_override.filter_policy; - } - if (set_block_based_table_factory) { - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); } - return options; - } + } while (ChangeOptions(kSkipHashCuckoo)); +} - DBImpl* dbfull() { - return reinterpret_cast(db_); - } +TEST_F(DBTest, SingleDeleteFlush) { + // Test to check whether flushing preserves a single delete hidden + // behind a put. + do { + Random rnd(301); - void CreateColumnFamilies(const std::vector& cfs, - const ColumnFamilyOptions* options = nullptr) { - ColumnFamilyOptions cf_opts; - if (options != nullptr) { - cf_opts = ColumnFamilyOptions(*options); - } else { - cf_opts = ColumnFamilyOptions(CurrentOptions()); - } - int cfi = handles_.size(); - handles_.resize(cfi + cfs.size()); - for (auto cf : cfs) { - ASSERT_OK(db_->CreateColumnFamily(cf_opts, cf, &handles_[cfi++])); - } - } + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + CreateAndReopenWithCF({"pikachu"}, options); - void CreateAndReopenWithCF(const std::vector& cfs, - const Options* options = nullptr) { - CreateColumnFamilies(cfs, options); - std::vector cfs_plus_default = cfs; - cfs_plus_default.insert(cfs_plus_default.begin(), kDefaultColumnFamilyName); - ReopenWithColumnFamilies(cfs_plus_default, options); - } + // Put values on second level (so that they will not be in the same + // compaction as the other operations. + Put(1, "foo", "first"); + Put(1, "bar", "one"); + ASSERT_OK(Flush(1)); + MoveFilesToLevel(2, 1); - void ReopenWithColumnFamilies(const std::vector& cfs, - const std::vector& options) { - ASSERT_OK(TryReopenWithColumnFamilies(cfs, options)); - } + // (Single) delete hidden by a put + SingleDelete(1, "foo"); + Put(1, "foo", "second"); + Delete(1, "bar"); + Put(1, "bar", "two"); + ASSERT_OK(Flush(1)); - void ReopenWithColumnFamilies(const std::vector& cfs, - const Options* options = nullptr) { - ASSERT_OK(TryReopenWithColumnFamilies(cfs, options)); - } + SingleDelete(1, "foo"); + Delete(1, "bar"); + ASSERT_OK(Flush(1)); - Status TryReopenWithColumnFamilies( - const std::vector& cfs, - const std::vector& options) { - Close(); - ASSERT_EQ(cfs.size(), options.size()); - std::vector column_families; - for (size_t i = 0; i < cfs.size(); ++i) { - column_families.push_back(ColumnFamilyDescriptor(cfs[i], *options[i])); - } - DBOptions db_opts = DBOptions(*options[0]); - return DB::Open(db_opts, dbname_, column_families, &handles_, &db_); - } + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); - Status TryReopenWithColumnFamilies(const std::vector& cfs, - const Options* options = nullptr) { - Close(); - Options opts = (options == nullptr) ? CurrentOptions() : *options; - std::vector v_opts(cfs.size(), &opts); - return TryReopenWithColumnFamilies(cfs, v_opts); - } + ASSERT_EQ("NOT_FOUND", Get(1, "bar")); + ASSERT_EQ("NOT_FOUND", Get(1, "foo")); + // Skip HashCuckooRep as it does not support single delete. FIFO and + // universal compaction do not apply to the test case. Skip MergePut + // because merges cannot be combined with single deletions. + } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction | + kSkipUniversalCompaction | kSkipMergePut)); +} - void Reopen(Options* options = nullptr) { - ASSERT_OK(TryReopen(options)); - } +TEST_F(DBTest, SingleDeletePutFlush) { + // Single deletes that encounter the matching put in a flush should get + // removed. + do { + Random rnd(301); - void Close() { - for (auto h : handles_) { - delete h; - } - handles_.clear(); - delete db_; - db_ = nullptr; - } + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + CreateAndReopenWithCF({"pikachu"}, options); - void DestroyAndReopen(Options* options = nullptr) { - //Destroy using last options - Destroy(&last_options_); - ASSERT_OK(TryReopen(options)); - } + Put(1, "foo", Slice()); + Put(1, "a", Slice()); + SingleDelete(1, "a"); + ASSERT_OK(Flush(1)); - void Destroy(Options* options) { - Close(); - ASSERT_OK(DestroyDB(dbname_, *options)); - } + ASSERT_EQ("[ ]", AllEntriesFor("a", 1)); + // Skip HashCuckooRep as it does not support single delete. FIFO and + // universal compaction do not apply to the test case. Skip MergePut + // because merges cannot be combined with single deletions. + } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction | + kSkipUniversalCompaction | kSkipMergePut)); +} - Status ReadOnlyReopen(Options* options) { - return DB::OpenForReadOnly(*options, dbname_, &db_); - } +// Disable because not all platform can run it. +// It requires more than 9GB memory to run it, With single allocation +// of more than 3GB. +TEST_F(DBTest, DISABLED_VeryLargeValue) { + const size_t kValueSize = 3221225472u; // 3GB value + const size_t kKeySize = 8388608u; // 8MB key + std::string raw(kValueSize, 'v'); + std::string key1(kKeySize, 'c'); + std::string key2(kKeySize, 'd'); - Status TryReopen(Options* options = nullptr) { - Close(); - Options opts; - if (options != nullptr) { - opts = *options; - } else { - opts = CurrentOptions(); - opts.create_if_missing = true; - } - last_options_ = opts; - return DB::Open(opts, dbname_, &db_); - } + Options options = CurrentOptions(); + options.env = env_; + options.write_buffer_size = 100000; // Small write buffer + options.paranoid_checks = true; + DestroyAndReopen(options); - Status Flush(int cf = 0) { - if (cf == 0) { - return db_->Flush(FlushOptions()); - } else { - return db_->Flush(FlushOptions(), handles_[cf]); - } - } + ASSERT_OK(Put("boo", "v1")); + ASSERT_OK(Put("foo", "v1")); + ASSERT_OK(Put(key1, raw)); + raw[0] = 'w'; + ASSERT_OK(Put(key2, raw)); + dbfull()->TEST_WaitForFlushMemTable(); - Status Put(const Slice& k, const Slice& v, WriteOptions wo = WriteOptions()) { - if (kMergePut == option_config_ ) { - return db_->Merge(wo, k, v); - } else { - return db_->Put(wo, k, v); - } - } + ASSERT_EQ(1, NumTableFilesAtLevel(0)); - Status Put(int cf, const Slice& k, const Slice& v, - WriteOptions wo = WriteOptions()) { - if (kMergePut == option_config_) { - return db_->Merge(wo, handles_[cf], k, v); - } else { - return db_->Put(wo, handles_[cf], k, v); - } - } + std::string value; + Status s = db_->Get(ReadOptions(), key1, &value); + ASSERT_OK(s); + ASSERT_EQ(kValueSize, value.size()); + ASSERT_EQ('v', value[0]); - Status Delete(const std::string& k) { - return db_->Delete(WriteOptions(), k); - } + s = db_->Get(ReadOptions(), key2, &value); + ASSERT_OK(s); + ASSERT_EQ(kValueSize, value.size()); + ASSERT_EQ('w', value[0]); - Status Delete(int cf, const std::string& k) { - return db_->Delete(WriteOptions(), handles_[cf], k); - } + // Compact all files. + Flush(); + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); - std::string Get(const std::string& k, const Snapshot* snapshot = nullptr) { - ReadOptions options; - options.verify_checksums = true; - options.snapshot = snapshot; - std::string result; - Status s = db_->Get(options, k, &result); - if (s.IsNotFound()) { - result = "NOT_FOUND"; - } else if (!s.ok()) { - result = s.ToString(); - } - return result; - } - - std::string Get(int cf, const std::string& k, - const Snapshot* snapshot = nullptr) { - ReadOptions options; - options.verify_checksums = true; - options.snapshot = snapshot; - std::string result; - Status s = db_->Get(options, handles_[cf], k, &result); - if (s.IsNotFound()) { - result = "NOT_FOUND"; - } else if (!s.ok()) { - result = s.ToString(); - } - return result; - } - - // Return a string that contains all key,value pairs in order, - // formatted like "(k1->v1)(k2->v2)". - std::string Contents(int cf = 0) { - std::vector forward; - std::string result; - Iterator* iter = (cf == 0) ? db_->NewIterator(ReadOptions()) - : db_->NewIterator(ReadOptions(), handles_[cf]); - for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { - std::string s = IterStatus(iter); - result.push_back('('); - result.append(s); - result.push_back(')'); - forward.push_back(s); - } + // Check DB is not in read-only state. + ASSERT_OK(Put("boo", "v1")); - // Check reverse iteration results are the reverse of forward results - unsigned int matched = 0; - for (iter->SeekToLast(); iter->Valid(); iter->Prev()) { - ASSERT_LT(matched, forward.size()); - ASSERT_EQ(IterStatus(iter), forward[forward.size() - matched - 1]); - matched++; - } - ASSERT_EQ(matched, forward.size()); + s = db_->Get(ReadOptions(), key1, &value); + ASSERT_OK(s); + ASSERT_EQ(kValueSize, value.size()); + ASSERT_EQ('v', value[0]); - delete iter; - return result; - } + s = db_->Get(ReadOptions(), key2, &value); + ASSERT_OK(s); + ASSERT_EQ(kValueSize, value.size()); + ASSERT_EQ('w', value[0]); +} - std::string AllEntriesFor(const Slice& user_key, int cf = 0) { - Iterator* iter; - if (cf == 0) { - iter = dbfull()->TEST_NewInternalIterator(); - } else { - iter = dbfull()->TEST_NewInternalIterator(handles_[cf]); - } - InternalKey target(user_key, kMaxSequenceNumber, kTypeValue); - iter->Seek(target.Encode()); - std::string result; - if (!iter->status().ok()) { - result = iter->status().ToString(); - } else { - result = "[ "; - bool first = true; - while (iter->Valid()) { - ParsedInternalKey ikey(Slice(), 0, kTypeValue); - if (!ParseInternalKey(iter->key(), &ikey)) { - result += "CORRUPTED"; - } else { - if (last_options_.comparator->Compare(ikey.user_key, user_key) != 0) { - break; - } - if (!first) { - result += ", "; - } - first = false; - switch (ikey.type) { - case kTypeValue: - result += iter->value().ToString(); - break; - case kTypeMerge: - // keep it the same as kTypeValue for testing kMergePut - result += iter->value().ToString(); - break; - case kTypeDeletion: - result += "DEL"; - break; - default: - assert(false); - break; - } - } - iter->Next(); - } - if (!first) { - result += " "; - } - result += "]"; - } - delete iter; - return result; - } +TEST_F(DBTest, GetFromImmutableLayer) { + do { + Options options = CurrentOptions(); + options.env = env_; + CreateAndReopenWithCF({"pikachu"}, options); - int NumTableFilesAtLevel(int level, int cf = 0) { - std::string property; - if (cf == 0) { - // default cfd - ASSERT_TRUE(db_->GetProperty( - "rocksdb.num-files-at-level" + NumberToString(level), &property)); - } else { - ASSERT_TRUE(db_->GetProperty( - handles_[cf], "rocksdb.num-files-at-level" + NumberToString(level), - &property)); - } - return atoi(property.c_str()); - } + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_EQ("v1", Get(1, "foo")); - int TotalTableFiles(int cf = 0, int levels = -1) { - if (levels == -1) { - levels = CurrentOptions().num_levels; - } - int result = 0; - for (int level = 0; level < levels; level++) { - result += NumTableFilesAtLevel(level, cf); - } - return result; - } - - // Return spread of files per level - std::string FilesPerLevel(int cf = 0) { - int num_levels = - (cf == 0) ? db_->NumberLevels() : db_->NumberLevels(handles_[1]); - std::string result; - int last_non_zero_offset = 0; - for (int level = 0; level < num_levels; level++) { - int f = NumTableFilesAtLevel(level, cf); - char buf[100]; - snprintf(buf, sizeof(buf), "%s%d", (level ? "," : ""), f); - result += buf; - if (f > 0) { - last_non_zero_offset = result.size(); - } - } - result.resize(last_non_zero_offset); - return result; - } + // Block sync calls + env_->delay_sstable_sync_.store(true, std::memory_order_release); + Put(1, "k1", std::string(100000, 'x')); // Fill memtable + Put(1, "k2", std::string(100000, 'y')); // Trigger flush + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_EQ("NOT_FOUND", Get(0, "foo")); + // Release sync calls + env_->delay_sstable_sync_.store(false, std::memory_order_release); + } while (ChangeOptions()); +} - int CountFiles() { - std::vector files; - env_->GetChildren(dbname_, &files); - std::vector logfiles; - if (dbname_ != last_options_.wal_dir) { - env_->GetChildren(last_options_.wal_dir, &logfiles); - } - - return static_cast(files.size() + logfiles.size()); - } - - int CountLiveFiles() { - std::vector metadata; - db_->GetLiveFilesMetaData(&metadata); - return metadata.size(); - } - - uint64_t Size(const Slice& start, const Slice& limit, int cf = 0) { - Range r(start, limit); - uint64_t size; - if (cf == 0) { - db_->GetApproximateSizes(&r, 1, &size); - } else { - db_->GetApproximateSizes(handles_[1], &r, 1, &size); - } - return size; - } - - void Compact(int cf, const Slice& start, const Slice& limit) { - ASSERT_OK(db_->CompactRange(handles_[cf], &start, &limit)); - } - - void Compact(const Slice& start, const Slice& limit) { - ASSERT_OK(db_->CompactRange(&start, &limit)); - } - - // Do n memtable compactions, each of which produces an sstable - // covering the range [small,large]. - void MakeTables(int n, const std::string& small, const std::string& large, - int cf = 0) { - for (int i = 0; i < n; i++) { - ASSERT_OK(Put(cf, small, "begin")); - ASSERT_OK(Put(cf, large, "end")); - ASSERT_OK(Flush(cf)); - } - } - - // Prevent pushing of new sstables into deeper levels by adding - // tables that cover a specified range to all levels. - void FillLevels(const std::string& smallest, const std::string& largest, - int cf) { - MakeTables(db_->NumberLevels(handles_[cf]), smallest, largest, cf); - } - - void DumpFileCounts(const char* label) { - fprintf(stderr, "---\n%s:\n", label); - fprintf(stderr, "maxoverlap: %lld\n", - static_cast( - dbfull()->TEST_MaxNextLevelOverlappingBytes())); - for (int level = 0; level < db_->NumberLevels(); level++) { - int num = NumTableFilesAtLevel(level); - if (num > 0) { - fprintf(stderr, " level %3d : %d files\n", level, num); - } - } - } - - std::string DumpSSTableList() { - std::string property; - db_->GetProperty("rocksdb.sstables", &property); - return property; - } - - int GetSstFileCount(std::string path) { - std::vector files; - env_->GetChildren(path, &files); - - int sst_count = 0; - uint64_t number; - FileType type; - for (size_t i = 0; i < files.size(); i++) { - if (ParseFileName(files[i], &number, &type) && type == kTableFile) { - sst_count++; - } - } - return sst_count; - } - - void GenerateNewFile(Random* rnd, int* key_idx) { - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(Key(*key_idx), RandomString(rnd, (i == 10) ? 1 : 10000))); - (*key_idx)++; - } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_WaitForCompact(); - } - - std::string IterStatus(Iterator* iter) { - std::string result; - if (iter->Valid()) { - result = iter->key().ToString() + "->" + iter->value().ToString(); - } else { - result = "(invalid)"; - } - return result; - } - - Options OptionsForLogIterTest() { - Options options = CurrentOptions(); - options.create_if_missing = true; - options.WAL_ttl_seconds = 1000; - return options; - } - - std::unique_ptr OpenTransactionLogIter( - const SequenceNumber seq) { - unique_ptr iter; - Status status = dbfull()->GetUpdatesSince(seq, &iter); - ASSERT_OK(status); - ASSERT_TRUE(iter->Valid()); - return std::move(iter); - } - - std::string DummyString(size_t len, char c = 'a') { - return std::string(len, c); - } - - void VerifyIterLast(std::string expected_key, int cf = 0) { - Iterator* iter; - ReadOptions ro; - if (cf == 0) { - iter = db_->NewIterator(ro); - } else { - iter = db_->NewIterator(ro, handles_[cf]); - } - iter->SeekToLast(); - ASSERT_EQ(IterStatus(iter), expected_key); - delete iter; - } - - // Used to test InplaceUpdate - - // If previous value is nullptr or delta is > than previous value, - // sets newValue with delta - // If previous value is not empty, - // updates previous value with 'b' string of previous value size - 1. - static UpdateStatus - updateInPlaceSmallerSize(char* prevValue, uint32_t* prevSize, - Slice delta, std::string* newValue) { - if (prevValue == nullptr) { - *newValue = std::string(delta.size(), 'c'); - return UpdateStatus::UPDATED; - } else { - *prevSize = *prevSize - 1; - std::string str_b = std::string(*prevSize, 'b'); - memcpy(prevValue, str_b.c_str(), str_b.size()); - return UpdateStatus::UPDATED_INPLACE; - } - } - - static UpdateStatus - updateInPlaceSmallerVarintSize(char* prevValue, uint32_t* prevSize, - Slice delta, std::string* newValue) { - if (prevValue == nullptr) { - *newValue = std::string(delta.size(), 'c'); - return UpdateStatus::UPDATED; - } else { - *prevSize = 1; - std::string str_b = std::string(*prevSize, 'b'); - memcpy(prevValue, str_b.c_str(), str_b.size()); - return UpdateStatus::UPDATED_INPLACE; - } - } - - static UpdateStatus - updateInPlaceLargerSize(char* prevValue, uint32_t* prevSize, - Slice delta, std::string* newValue) { - *newValue = std::string(delta.size(), 'c'); - return UpdateStatus::UPDATED; - } - - static UpdateStatus - updateInPlaceNoAction(char* prevValue, uint32_t* prevSize, - Slice delta, std::string* newValue) { - return UpdateStatus::UPDATE_FAILED; - } - - // Utility method to test InplaceUpdate - void validateNumberOfEntries(int numValues, int cf = 0) { - Iterator* iter; - if (cf != 0) { - iter = dbfull()->TEST_NewInternalIterator(handles_[cf]); - } else { - iter = dbfull()->TEST_NewInternalIterator(); - } - iter->SeekToFirst(); - ASSERT_EQ(iter->status().ok(), true); - int seq = numValues; - while (iter->Valid()) { - ParsedInternalKey ikey; - ikey.sequence = -1; - ASSERT_EQ(ParseInternalKey(iter->key(), &ikey), true); - - // checks sequence number for updates - ASSERT_EQ(ikey.sequence, (unsigned)seq--); - iter->Next(); - } - delete iter; - ASSERT_EQ(0, seq); - } - - void CopyFile(const std::string& source, const std::string& destination, - uint64_t size = 0) { - const EnvOptions soptions; - unique_ptr srcfile; - ASSERT_OK(env_->NewSequentialFile(source, &srcfile, soptions)); - unique_ptr destfile; - ASSERT_OK(env_->NewWritableFile(destination, &destfile, soptions)); - - if (size == 0) { - // default argument means copy everything - ASSERT_OK(env_->GetFileSize(source, &size)); - } - - char buffer[4096]; - Slice slice; - while (size > 0) { - uint64_t one = std::min(uint64_t(sizeof(buffer)), size); - ASSERT_OK(srcfile->Read(one, &slice, buffer)); - ASSERT_OK(destfile->Append(slice)); - size -= slice.size(); - } - ASSERT_OK(destfile->Close()); - } - -}; - -static long TestGetTickerCount(const Options& options, Tickers ticker_type) { - return options.statistics->getTickerCount(ticker_type); -} - -// A helper function that ensures the table properties returned in -// `GetPropertiesOfAllTablesTest` is correct. -// This test assumes entries size is differnt for each of the tables. -namespace { -void VerifyTableProperties(DB* db, uint64_t expected_entries_size) { - TablePropertiesCollection props; - ASSERT_OK(db->GetPropertiesOfAllTables(&props)); - - ASSERT_EQ(4U, props.size()); - std::unordered_set unique_entries; - - // Indirect test - uint64_t sum = 0; - for (const auto& item : props) { - unique_entries.insert(item.second->num_entries); - sum += item.second->num_entries; - } - - ASSERT_EQ(props.size(), unique_entries.size()); - ASSERT_EQ(expected_entries_size, sum); -} -} // namespace - -TEST(DBTest, Empty) { - do { - Options options; - options.env = env_; - options.write_buffer_size = 100000; // Small write buffer - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - std::string num; - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.num-entries-active-mem-table", &num)); - ASSERT_EQ("0", num); - - ASSERT_OK(Put(1, "foo", "v1")); - ASSERT_EQ("v1", Get(1, "foo")); - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.num-entries-active-mem-table", &num)); - ASSERT_EQ("1", num); - - env_->delay_sstable_sync_.Release_Store(env_); // Block sync calls - Put(1, "k1", std::string(100000, 'x')); // Fill memtable - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.num-entries-active-mem-table", &num)); - ASSERT_EQ("2", num); - - Put(1, "k2", std::string(100000, 'y')); // Trigger compaction - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.num-entries-active-mem-table", &num)); - ASSERT_EQ("1", num); - - ASSERT_EQ("v1", Get(1, "foo")); - env_->delay_sstable_sync_.Release_Store(nullptr); // Release sync calls - - ASSERT_OK(db_->DisableFileDeletions()); - ASSERT_TRUE( - dbfull()->GetProperty("rocksdb.is-file-deletions-enabled", &num)); - ASSERT_EQ("1", num); - - ASSERT_OK(db_->DisableFileDeletions()); - ASSERT_TRUE( - dbfull()->GetProperty("rocksdb.is-file-deletions-enabled", &num)); - ASSERT_EQ("2", num); - - ASSERT_OK(db_->DisableFileDeletions()); - ASSERT_TRUE( - dbfull()->GetProperty("rocksdb.is-file-deletions-enabled", &num)); - ASSERT_EQ("3", num); - - ASSERT_OK(db_->EnableFileDeletions(false)); - ASSERT_TRUE( - dbfull()->GetProperty("rocksdb.is-file-deletions-enabled", &num)); - ASSERT_EQ("2", num); - - ASSERT_OK(db_->EnableFileDeletions()); - ASSERT_TRUE( - dbfull()->GetProperty("rocksdb.is-file-deletions-enabled", &num)); - ASSERT_EQ("0", num); - } while (ChangeOptions()); -} - -TEST(DBTest, ReadOnlyDB) { - ASSERT_OK(Put("foo", "v1")); - ASSERT_OK(Put("bar", "v2")); - ASSERT_OK(Put("foo", "v3")); - Close(); - - Options options; - ASSERT_OK(ReadOnlyReopen(&options)); - ASSERT_EQ("v3", Get("foo")); - ASSERT_EQ("v2", Get("bar")); - Iterator* iter = db_->NewIterator(ReadOptions()); - int count = 0; - for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { - ASSERT_OK(iter->status()); - ++count; - } - ASSERT_EQ(count, 2); - delete iter; - Close(); - - // Reopen and flush memtable. - Reopen(); - Flush(); - Close(); - // Now check keys in read only mode. - ASSERT_OK(ReadOnlyReopen(&options)); - ASSERT_EQ("v3", Get("foo")); - ASSERT_EQ("v2", Get("bar")); -} - -// Make sure that when options.block_cache is set, after a new table is -// created its index/filter blocks are added to block cache. -TEST(DBTest, IndexAndFilterBlocksOfNewTableAddedToCache) { - Options options = CurrentOptions(); - options.create_if_missing = true; - options.statistics = rocksdb::CreateDBStatistics(); - BlockBasedTableOptions table_options; - table_options.cache_index_and_filter_blocks = true; - table_options.filter_policy.reset(NewBloomFilterPolicy(20)); - options.table_factory.reset(new BlockBasedTableFactory(table_options)); - CreateAndReopenWithCF({"pikachu"}, &options); - - ASSERT_OK(Put(1, "key", "val")); - // Create a new table. - ASSERT_OK(Flush(1)); - - // index/filter blocks added to block cache right after table creation. - ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); - ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); - ASSERT_EQ(2, /* only index/filter were added */ - TestGetTickerCount(options, BLOCK_CACHE_ADD)); - ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_DATA_MISS)); - uint64_t int_num; - ASSERT_TRUE( - dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); - ASSERT_EQ(int_num, 0U); - - // Make sure filter block is in cache. - std::string value; - ReadOptions ropt; - db_->KeyMayExist(ReadOptions(), handles_[1], "key", &value); - - // Miss count should remain the same. - ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); - ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); - - db_->KeyMayExist(ReadOptions(), handles_[1], "key", &value); - ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); - ASSERT_EQ(2, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); - - // Make sure index block is in cache. - auto index_block_hit = TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT); - value = Get(1, "key"); - ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); - ASSERT_EQ(index_block_hit + 1, - TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); - - value = Get(1, "key"); - ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); - ASSERT_EQ(index_block_hit + 2, - TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); -} - -TEST(DBTest, GetPropertiesOfAllTablesTest) { - Options options = CurrentOptions(); - Reopen(&options); - // Create 4 tables - for (int table = 0; table < 4; ++table) { - for (int i = 0; i < 10 + table; ++i) { - db_->Put(WriteOptions(), std::to_string(table * 100 + i), "val"); - } - db_->Flush(FlushOptions()); - } - - // 1. Read table properties directly from file - Reopen(&options); - VerifyTableProperties(db_, 10 + 11 + 12 + 13); - - // 2. Put two tables to table cache and - Reopen(&options); - // fetch key from 1st and 2nd table, which will internally place that table to - // the table cache. - for (int i = 0; i < 2; ++i) { - Get(std::to_string(i * 100 + 0)); - } - - VerifyTableProperties(db_, 10 + 11 + 12 + 13); - - // 3. Put all tables to table cache - Reopen(&options); - // fetch key from 1st and 2nd table, which will internally place that table to - // the table cache. - for (int i = 0; i < 4; ++i) { - Get(std::to_string(i * 100 + 0)); - } - VerifyTableProperties(db_, 10 + 11 + 12 + 13); -} - -TEST(DBTest, LevelLimitReopen) { - Options options = CurrentOptions(); - CreateAndReopenWithCF({"pikachu"}, &options); - - const std::string value(1024 * 1024, ' '); - int i = 0; - while (NumTableFilesAtLevel(2, 1) == 0) { - ASSERT_OK(Put(1, Key(i++), value)); - } - - options.num_levels = 1; - options.max_bytes_for_level_multiplier_additional.resize(1, 1); - Status s = TryReopenWithColumnFamilies({"default", "pikachu"}, &options); - ASSERT_EQ(s.IsInvalidArgument(), true); - ASSERT_EQ(s.ToString(), - "Invalid argument: db has more levels than options.num_levels"); - - options.num_levels = 10; - options.max_bytes_for_level_multiplier_additional.resize(10, 1); - ASSERT_OK(TryReopenWithColumnFamilies({"default", "pikachu"}, &options)); -} - -TEST(DBTest, Preallocation) { - const std::string src = dbname_ + "/alloc_test"; - unique_ptr srcfile; - const EnvOptions soptions; - ASSERT_OK(env_->NewWritableFile(src, &srcfile, soptions)); - srcfile->SetPreallocationBlockSize(1024 * 1024); - - // No writes should mean no preallocation - size_t block_size, last_allocated_block; - srcfile->GetPreallocationStatus(&block_size, &last_allocated_block); - ASSERT_EQ(last_allocated_block, 0UL); - - // Small write should preallocate one block - srcfile->Append("test"); - srcfile->GetPreallocationStatus(&block_size, &last_allocated_block); - ASSERT_EQ(last_allocated_block, 1UL); - - // Write an entire preallocation block, make sure we increased by two. - std::string buf(block_size, ' '); - srcfile->Append(buf); - srcfile->GetPreallocationStatus(&block_size, &last_allocated_block); - ASSERT_EQ(last_allocated_block, 2UL); - - // Write five more blocks at once, ensure we're where we need to be. - buf = std::string(block_size * 5, ' '); - srcfile->Append(buf); - srcfile->GetPreallocationStatus(&block_size, &last_allocated_block); - ASSERT_EQ(last_allocated_block, 7UL); -} - -TEST(DBTest, PutDeleteGet) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "foo", "v1")); - ASSERT_EQ("v1", Get(1, "foo")); - ASSERT_OK(Put(1, "foo", "v2")); - ASSERT_EQ("v2", Get(1, "foo")); - ASSERT_OK(Delete(1, "foo")); - ASSERT_EQ("NOT_FOUND", Get(1, "foo")); - } while (ChangeOptions()); -} - - -TEST(DBTest, GetFromImmutableLayer) { - do { - Options options; - options.env = env_; - options.write_buffer_size = 100000; // Small write buffer - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - ASSERT_OK(Put(1, "foo", "v1")); - ASSERT_EQ("v1", Get(1, "foo")); - - env_->delay_sstable_sync_.Release_Store(env_); // Block sync calls - Put(1, "k1", std::string(100000, 'x')); // Fill memtable - Put(1, "k2", std::string(100000, 'y')); // Trigger flush - ASSERT_EQ("v1", Get(1, "foo")); - ASSERT_EQ("NOT_FOUND", Get(0, "foo")); - env_->delay_sstable_sync_.Release_Store(nullptr); // Release sync calls - } while (ChangeOptions()); -} - -TEST(DBTest, GetFromVersions) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "foo", "v1")); - ASSERT_OK(Flush(1)); - ASSERT_EQ("v1", Get(1, "foo")); - ASSERT_EQ("NOT_FOUND", Get(0, "foo")); - } while (ChangeOptions()); -} - -TEST(DBTest, GetSnapshot) { - do { - CreateAndReopenWithCF({"pikachu"}); - // Try with both a short key and a long key - for (int i = 0; i < 2; i++) { - std::string key = (i == 0) ? std::string("foo") : std::string(200, 'x'); - ASSERT_OK(Put(1, key, "v1")); - const Snapshot* s1 = db_->GetSnapshot(); - ASSERT_OK(Put(1, key, "v2")); - ASSERT_EQ("v2", Get(1, key)); - ASSERT_EQ("v1", Get(1, key, s1)); - ASSERT_OK(Flush(1)); - ASSERT_EQ("v2", Get(1, key)); - ASSERT_EQ("v1", Get(1, key, s1)); - db_->ReleaseSnapshot(s1); - } - // skip as HashCuckooRep does not support snapshot - } while (ChangeOptions(kSkipHashCuckoo)); -} - -TEST(DBTest, GetLevel0Ordering) { +TEST_F(DBTest, GetLevel0Ordering) { do { - CreateAndReopenWithCF({"pikachu"}); + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); // Check that we process level-0 files in correct order. The code // below generates two level-0 files where the earlier one comes // before the later one in the level-0 file list since the earlier @@ -1426,9 +568,20 @@ TEST(DBTest, GetLevel0Ordering) { } while (ChangeOptions()); } -TEST(DBTest, GetOrderedByLevels) { +TEST_F(DBTest, WrongLevel0Config) { + Options options = CurrentOptions(); + Close(); + ASSERT_OK(DestroyDB(dbname_, options)); + options.level0_stop_writes_trigger = 1; + options.level0_slowdown_writes_trigger = 2; + options.level0_file_num_compaction_trigger = 3; + ASSERT_OK(DB::Open(options, dbname_, &db_)); +} + +#ifndef ROCKSDB_LITE +TEST_F(DBTest, GetOrderedByLevels) { do { - CreateAndReopenWithCF({"pikachu"}); + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); ASSERT_OK(Put(1, "foo", "v1")); Compact(1, "a", "z"); ASSERT_EQ("v1", Get(1, "foo")); @@ -1439,9 +592,9 @@ TEST(DBTest, GetOrderedByLevels) { } while (ChangeOptions()); } -TEST(DBTest, GetPicksCorrectFile) { +TEST_F(DBTest, GetPicksCorrectFile) { do { - CreateAndReopenWithCF({"pikachu"}); + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); // Arrange to have multiple files in a non-level-0 level. ASSERT_OK(Put(1, "a", "va")); Compact(1, "a", "b"); @@ -1455,26 +608,29 @@ TEST(DBTest, GetPicksCorrectFile) { } while (ChangeOptions()); } -TEST(DBTest, GetEncountersEmptyLevel) { +TEST_F(DBTest, GetEncountersEmptyLevel) { do { - CreateAndReopenWithCF({"pikachu"}); + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); // Arrange for the following to happen: // * sstable A in level 0 // * nothing in level 1 // * sstable B in level 2 // Then do enough Get() calls to arrange for an automatic compaction // of sstable A. A bug would cause the compaction to be marked as - // occuring at level 1 (instead of the correct level 0). + // occurring at level 1 (instead of the correct level 0). // Step 1: First place sstables in levels 0 and 2 - int compaction_count = 0; - while (NumTableFilesAtLevel(0, 1) == 0 || NumTableFilesAtLevel(2, 1) == 0) { - ASSERT_LE(compaction_count, 100) << "could not fill levels 0 and 2"; - compaction_count++; - Put(1, "a", "begin"); - Put(1, "z", "end"); - ASSERT_OK(Flush(1)); - } + Put(1, "a", "begin"); + Put(1, "z", "end"); + ASSERT_OK(Flush(1)); + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); + dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); + Put(1, "a", "begin"); + Put(1, "z", "end"); + ASSERT_OK(Flush(1)); + ASSERT_GT(NumTableFilesAtLevel(0, 1), 0); + ASSERT_GT(NumTableFilesAtLevel(2, 1), 0); // Step 2: clear level 1 if necessary. dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); @@ -1488,6263 +644,4855 @@ TEST(DBTest, GetEncountersEmptyLevel) { } // Step 4: Wait for compaction to finish - env_->SleepForMicroseconds(1000000); + dbfull()->TEST_WaitForCompact(); ASSERT_EQ(NumTableFilesAtLevel(0, 1), 1); // XXX } while (ChangeOptions(kSkipUniversalCompaction | kSkipFIFOCompaction)); } +#endif // ROCKSDB_LITE -// KeyMayExist can lead to a few false positives, but not false negatives. -// To make test deterministic, use a much larger number of bits per key-20 than -// bits in the key, so that false positives are eliminated -TEST(DBTest, KeyMayExist) { +TEST_F(DBTest, FlushMultipleMemtable) { do { - ReadOptions ropts; - std::string value; - anon::OptionsOverride options_override; - options_override.filter_policy.reset(NewBloomFilterPolicy(20)); - Options options = CurrentOptions(options_override); - options.statistics = rocksdb::CreateDBStatistics(); - CreateAndReopenWithCF({"pikachu"}, &options); - - ASSERT_TRUE(!db_->KeyMayExist(ropts, handles_[1], "a", &value)); - - ASSERT_OK(Put(1, "a", "b")); - bool value_found = false; - ASSERT_TRUE( - db_->KeyMayExist(ropts, handles_[1], "a", &value, &value_found)); - ASSERT_TRUE(value_found); - ASSERT_EQ("b", value); - + Options options = CurrentOptions(); + WriteOptions writeOpt = WriteOptions(); + writeOpt.disableWAL = true; + options.max_write_buffer_number = 4; + options.min_write_buffer_number_to_merge = 3; + options.max_write_buffer_number_to_maintain = -1; + CreateAndReopenWithCF({"pikachu"}, options); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v1")); ASSERT_OK(Flush(1)); - value.clear(); - - long numopen = TestGetTickerCount(options, NO_FILE_OPENS); - long cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); - ASSERT_TRUE( - db_->KeyMayExist(ropts, handles_[1], "a", &value, &value_found)); - ASSERT_TRUE(!value_found); - // assert that no new files were opened and no new blocks were - // read into block cache. - ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); - ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); - - ASSERT_OK(Delete(1, "a")); - - numopen = TestGetTickerCount(options, NO_FILE_OPENS); - cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); - ASSERT_TRUE(!db_->KeyMayExist(ropts, handles_[1], "a", &value)); - ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); - ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v1")); + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_EQ("v1", Get(1, "bar")); ASSERT_OK(Flush(1)); - db_->CompactRange(handles_[1], nullptr, nullptr); - - numopen = TestGetTickerCount(options, NO_FILE_OPENS); - cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); - ASSERT_TRUE(!db_->KeyMayExist(ropts, handles_[1], "a", &value)); - ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); - ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); - - ASSERT_OK(Delete(1, "c")); - - numopen = TestGetTickerCount(options, NO_FILE_OPENS); - cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); - ASSERT_TRUE(!db_->KeyMayExist(ropts, handles_[1], "c", &value)); - ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); - ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); - - // KeyMayExist function only checks data in block caches, which is not used - // by plain table format. - } while ( - ChangeOptions(kSkipPlainTable | kSkipHashIndex | kSkipFIFOCompaction)); + } while (ChangeCompactOptions()); } - -TEST(DBTest, NonBlockingIteration) { - do { - ReadOptions non_blocking_opts, regular_opts; - Options options = CurrentOptions(); - options.statistics = rocksdb::CreateDBStatistics(); - non_blocking_opts.read_tier = kBlockCacheTier; - CreateAndReopenWithCF({"pikachu"}, &options); - // write one kv to the database. - ASSERT_OK(Put(1, "a", "b")); - - // scan using non-blocking iterator. We should find it because - // it is in memtable. - Iterator* iter = db_->NewIterator(non_blocking_opts, handles_[1]); - int count = 0; - for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { - ASSERT_OK(iter->status()); - count++; +#ifndef ROCKSDB_LITE +TEST_F(DBTest, FlushSchedule) { + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.level0_stop_writes_trigger = 1 << 10; + options.level0_slowdown_writes_trigger = 1 << 10; + options.min_write_buffer_number_to_merge = 1; + options.max_write_buffer_number_to_maintain = 1; + options.max_write_buffer_number = 2; + options.write_buffer_size = 120 * 1024; + CreateAndReopenWithCF({"pikachu"}, options); + std::vector threads; + + std::atomic thread_num(0); + // each column family will have 5 thread, each thread generating 2 memtables. + // each column family should end up with 10 table files + std::function fill_memtable_func = [&]() { + int a = thread_num.fetch_add(1); + Random rnd(a); + WriteOptions wo; + // this should fill up 2 memtables + for (int k = 0; k < 5000; ++k) { + ASSERT_OK(db_->Put(wo, handles_[a & 1], RandomString(&rnd, 13), "")); } - ASSERT_EQ(count, 1); - delete iter; + }; - // flush memtable to storage. Now, the key should not be in the - // memtable neither in the block cache. - ASSERT_OK(Flush(1)); + for (int i = 0; i < 10; ++i) { + threads.emplace_back(fill_memtable_func); + } - // verify that a non-blocking iterator does not find any - // kvs. Neither does it do any IOs to storage. - long numopen = TestGetTickerCount(options, NO_FILE_OPENS); - long cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); - iter = db_->NewIterator(non_blocking_opts, handles_[1]); - count = 0; - for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { - count++; - } - ASSERT_EQ(count, 0); - ASSERT_TRUE(iter->status().IsIncomplete()); - ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); - ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); - delete iter; - - // read in the specified block via a regular get - ASSERT_EQ(Get(1, "a"), "b"); - - // verify that we can find it via a non-blocking scan - numopen = TestGetTickerCount(options, NO_FILE_OPENS); - cache_added = TestGetTickerCount(options, BLOCK_CACHE_ADD); - iter = db_->NewIterator(non_blocking_opts, handles_[1]); - count = 0; - for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { - ASSERT_OK(iter->status()); - count++; - } - ASSERT_EQ(count, 1); - ASSERT_EQ(numopen, TestGetTickerCount(options, NO_FILE_OPENS)); - ASSERT_EQ(cache_added, TestGetTickerCount(options, BLOCK_CACHE_ADD)); - delete iter; - - // This test verifies block cache behaviors, which is not used by plain - // table format. - // Exclude kHashCuckoo as it does not support iteration currently - } while (ChangeOptions(kSkipPlainTable | kSkipNoSeekToLast | - kSkipHashCuckoo)); -} + for (auto& t : threads) { + t.join(); + } -// A delete is skipped for key if KeyMayExist(key) returns False -// Tests Writebatch consistency and proper delete behaviour -TEST(DBTest, FilterDeletes) { - do { - anon::OptionsOverride options_override; - options_override.filter_policy.reset(NewBloomFilterPolicy(20)); - Options options = CurrentOptions(options_override); - options.filter_deletes = true; - CreateAndReopenWithCF({"pikachu"}, &options); - WriteBatch batch; + auto default_tables = GetNumberOfSstFilesForColumnFamily(db_, "default"); + auto pikachu_tables = GetNumberOfSstFilesForColumnFamily(db_, "pikachu"); + ASSERT_LE(default_tables, static_cast(10)); + ASSERT_GT(default_tables, static_cast(0)); + ASSERT_LE(pikachu_tables, static_cast(10)); + ASSERT_GT(pikachu_tables, static_cast(0)); +} +#endif // ROCKSDB_LITE - batch.Delete(handles_[1], "a"); - dbfull()->Write(WriteOptions(), &batch); - ASSERT_EQ(AllEntriesFor("a", 1), "[ ]"); // Delete skipped - batch.Clear(); - - batch.Put(handles_[1], "a", "b"); - batch.Delete(handles_[1], "a"); - dbfull()->Write(WriteOptions(), &batch); - ASSERT_EQ(Get(1, "a"), "NOT_FOUND"); - ASSERT_EQ(AllEntriesFor("a", 1), "[ DEL, b ]"); // Delete issued - batch.Clear(); - - batch.Delete(handles_[1], "c"); - batch.Put(handles_[1], "c", "d"); - dbfull()->Write(WriteOptions(), &batch); - ASSERT_EQ(Get(1, "c"), "d"); - ASSERT_EQ(AllEntriesFor("c", 1), "[ d ]"); // Delete skipped - batch.Clear(); - - ASSERT_OK(Flush(1)); // A stray Flush - - batch.Delete(handles_[1], "c"); - dbfull()->Write(WriteOptions(), &batch); - ASSERT_EQ(AllEntriesFor("c", 1), "[ DEL, d ]"); // Delete issued - batch.Clear(); - } while (ChangeCompactOptions()); -} +namespace { +class KeepFilter : public CompactionFilter { + public: + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, + bool* value_changed) const override { + return false; + } + virtual const char* Name() const override { return "KeepFilter"; } +}; -TEST(DBTest, IterSeekBeforePrev) { - ASSERT_OK(Put("a", "b")); - ASSERT_OK(Put("c", "d")); - dbfull()->Flush(FlushOptions()); - ASSERT_OK(Put("0", "f")); - ASSERT_OK(Put("1", "h")); - dbfull()->Flush(FlushOptions()); - ASSERT_OK(Put("2", "j")); - auto iter = db_->NewIterator(ReadOptions()); - iter->Seek(Slice("c")); - iter->Prev(); - iter->Seek(Slice("a")); - iter->Prev(); - delete iter; -} +class KeepFilterFactory : public CompactionFilterFactory { + public: + explicit KeepFilterFactory(bool check_context = false) + : check_context_(check_context) {} -namespace { -std::string MakeLongKey(size_t length, char c) { - return std::string(length, c); -} -} // namespace + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + if (check_context_) { + EXPECT_EQ(expect_full_compaction_.load(), context.is_full_compaction); + EXPECT_EQ(expect_manual_compaction_.load(), context.is_manual_compaction); + } + return std::unique_ptr(new KeepFilter()); + } -TEST(DBTest, IterLongKeys) { - ASSERT_OK(Put(MakeLongKey(20, 0), "0")); - ASSERT_OK(Put(MakeLongKey(32, 2), "2")); - ASSERT_OK(Put("a", "b")); - dbfull()->Flush(FlushOptions()); - ASSERT_OK(Put(MakeLongKey(50, 1), "1")); - ASSERT_OK(Put(MakeLongKey(127, 3), "3")); - ASSERT_OK(Put(MakeLongKey(64, 4), "4")); - auto iter = db_->NewIterator(ReadOptions()); - - // Create a key that needs to be skipped for Seq too new - iter->Seek(MakeLongKey(20, 0)); - ASSERT_EQ(IterStatus(iter), MakeLongKey(20, 0) + "->0"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), MakeLongKey(50, 1) + "->1"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), MakeLongKey(32, 2) + "->2"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), MakeLongKey(127, 3) + "->3"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), MakeLongKey(64, 4) + "->4"); - delete iter; - - iter = db_->NewIterator(ReadOptions()); - iter->Seek(MakeLongKey(50, 1)); - ASSERT_EQ(IterStatus(iter), MakeLongKey(50, 1) + "->1"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), MakeLongKey(32, 2) + "->2"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), MakeLongKey(127, 3) + "->3"); - delete iter; -} + virtual const char* Name() const override { return "KeepFilterFactory"; } + bool check_context_; + std::atomic_bool expect_full_compaction_; + std::atomic_bool expect_manual_compaction_; +}; + +class DelayFilter : public CompactionFilter { + public: + explicit DelayFilter(DBTestBase* d) : db_test(d) {} + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, + bool* value_changed) const override { + db_test->env_->addon_time_.fetch_add(1000); + return true; + } + virtual const char* Name() const override { return "DelayFilter"; } -TEST(DBTest, IterNextWithNewerSeq) { - ASSERT_OK(Put("0", "0")); - dbfull()->Flush(FlushOptions()); - ASSERT_OK(Put("a", "b")); - ASSERT_OK(Put("c", "d")); - ASSERT_OK(Put("d", "e")); - auto iter = db_->NewIterator(ReadOptions()); + private: + DBTestBase* db_test; +}; - // Create a key that needs to be skipped for Seq too new - for (uint64_t i = 0; i < last_options_.max_sequential_skip_in_iterations + 1; - i++) { - ASSERT_OK(Put("b", "f")); +class DelayFilterFactory : public CompactionFilterFactory { + public: + explicit DelayFilterFactory(DBTestBase* d) : db_test(d) {} + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + return std::unique_ptr(new DelayFilter(db_test)); } - iter->Seek(Slice("a")); - ASSERT_EQ(IterStatus(iter), "a->b"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "c->d"); - delete iter; + virtual const char* Name() const override { return "DelayFilterFactory"; } + + private: + DBTestBase* db_test; +}; +} // namespace + +#ifndef ROCKSDB_LITE + +static std::string CompressibleString(Random* rnd, int len) { + std::string r; + test::CompressibleString(rnd, 0.8, len, &r); + return r; } +#endif // ROCKSDB_LITE -TEST(DBTest, IterPrevWithNewerSeq) { - ASSERT_OK(Put("0", "0")); - dbfull()->Flush(FlushOptions()); - ASSERT_OK(Put("a", "b")); - ASSERT_OK(Put("c", "d")); - ASSERT_OK(Put("d", "e")); - auto iter = db_->NewIterator(ReadOptions()); +TEST_F(DBTest, FailMoreDbPaths) { + Options options = CurrentOptions(); + options.db_paths.emplace_back(dbname_, 10000000); + options.db_paths.emplace_back(dbname_ + "_2", 1000000); + options.db_paths.emplace_back(dbname_ + "_3", 1000000); + options.db_paths.emplace_back(dbname_ + "_4", 1000000); + options.db_paths.emplace_back(dbname_ + "_5", 1000000); + ASSERT_TRUE(TryReopen(options).IsNotSupported()); +} - // Create a key that needs to be skipped for Seq too new - for (uint64_t i = 0; i < last_options_.max_sequential_skip_in_iterations + 1; - i++) { - ASSERT_OK(Put("b", "f")); +void CheckColumnFamilyMeta(const ColumnFamilyMetaData& cf_meta) { + uint64_t cf_size = 0; + uint64_t cf_csize = 0; + size_t file_count = 0; + for (auto level_meta : cf_meta.levels) { + uint64_t level_size = 0; + uint64_t level_csize = 0; + file_count += level_meta.files.size(); + for (auto file_meta : level_meta.files) { + level_size += file_meta.size; + } + ASSERT_EQ(level_meta.size, level_size); + cf_size += level_size; + cf_csize += level_csize; } + ASSERT_EQ(cf_meta.file_count, file_count); + ASSERT_EQ(cf_meta.size, cf_size); +} - iter->Seek(Slice("d")); - ASSERT_EQ(IterStatus(iter), "d->e"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "c->d"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "a->b"); +#ifndef ROCKSDB_LITE +TEST_F(DBTest, ColumnFamilyMetaDataTest) { + Options options = CurrentOptions(); + options.create_if_missing = true; + DestroyAndReopen(options); - iter->Prev(); - delete iter; + Random rnd(301); + int key_index = 0; + ColumnFamilyMetaData cf_meta; + for (int i = 0; i < 100; ++i) { + GenerateNewFile(&rnd, &key_index); + db_->GetColumnFamilyMetaData(&cf_meta); + CheckColumnFamilyMeta(cf_meta); + } } -TEST(DBTest, IterPrevWithNewerSeq2) { - ASSERT_OK(Put("0", "0")); - dbfull()->Flush(FlushOptions()); - ASSERT_OK(Put("a", "b")); - ASSERT_OK(Put("c", "d")); - ASSERT_OK(Put("d", "e")); - auto iter = db_->NewIterator(ReadOptions()); - iter->Seek(Slice("c")); - ASSERT_EQ(IterStatus(iter), "c->d"); +namespace { +void MinLevelHelper(DBTest* self, Options& options) { + Random rnd(301); - // Create a key that needs to be skipped for Seq too new - for (uint64_t i = 0; i < last_options_.max_sequential_skip_in_iterations + 1; - i++) { - ASSERT_OK(Put("b", "f")); + for (int num = 0; num < options.level0_file_num_compaction_trigger - 1; + num++) { + std::vector values; + // Write 120KB (12 values, each 10K) + for (int i = 0; i < 12; i++) { + values.push_back(DBTestBase::RandomString(&rnd, 10000)); + ASSERT_OK(self->Put(DBTestBase::Key(i), values[i])); + } + self->dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(self->NumTableFilesAtLevel(0), num + 1); } - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "a->b"); + // generate one more file in level-0, and should trigger level-0 compaction + std::vector values; + for (int i = 0; i < 12; i++) { + values.push_back(DBTestBase::RandomString(&rnd, 10000)); + ASSERT_OK(self->Put(DBTestBase::Key(i), values[i])); + } + self->dbfull()->TEST_WaitForCompact(); - iter->Prev(); - delete iter; + ASSERT_EQ(self->NumTableFilesAtLevel(0), 0); + ASSERT_EQ(self->NumTableFilesAtLevel(1), 1); } -TEST(DBTest, IterEmpty) { - do { - CreateAndReopenWithCF({"pikachu"}); - Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); +// returns false if the calling-Test should be skipped +bool MinLevelToCompress(CompressionType& type, Options& options, int wbits, + int lev, int strategy) { + fprintf(stderr, + "Test with compression options : window_bits = %d, level = %d, " + "strategy = %d}\n", + wbits, lev, strategy); + options.write_buffer_size = 100 << 10; // 100KB + options.arena_block_size = 4096; + options.num_levels = 3; + options.level0_file_num_compaction_trigger = 3; + options.create_if_missing = true; - iter->SeekToFirst(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); + if (Snappy_Supported()) { + type = kSnappyCompression; + fprintf(stderr, "using snappy\n"); + } else if (Zlib_Supported()) { + type = kZlibCompression; + fprintf(stderr, "using zlib\n"); + } else if (BZip2_Supported()) { + type = kBZip2Compression; + fprintf(stderr, "using bzip2\n"); + } else if (LZ4_Supported()) { + type = kLZ4Compression; + fprintf(stderr, "using lz4\n"); + } else if (XPRESS_Supported()) { + type = kXpressCompression; + fprintf(stderr, "using xpress\n"); + } else if (ZSTD_Supported()) { + type = kZSTD; + fprintf(stderr, "using ZSTD\n"); + } else { + fprintf(stderr, "skipping test, compression disabled\n"); + return false; + } + options.compression_per_level.resize(options.num_levels); - iter->SeekToLast(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); + // do not compress L0 + for (int i = 0; i < 1; i++) { + options.compression_per_level[i] = kNoCompression; + } + for (int i = 1; i < options.num_levels; i++) { + options.compression_per_level[i] = type; + } + return true; +} +} // namespace - iter->Seek("foo"); - ASSERT_EQ(IterStatus(iter), "(invalid)"); +TEST_F(DBTest, MinLevelToCompress1) { + Options options = CurrentOptions(); + CompressionType type = kSnappyCompression; + if (!MinLevelToCompress(type, options, -14, -1, 0)) { + return; + } + Reopen(options); + MinLevelHelper(this, options); - delete iter; - } while (ChangeCompactOptions()); + // do not compress L0 and L1 + for (int i = 0; i < 2; i++) { + options.compression_per_level[i] = kNoCompression; + } + for (int i = 2; i < options.num_levels; i++) { + options.compression_per_level[i] = type; + } + DestroyAndReopen(options); + MinLevelHelper(this, options); } -TEST(DBTest, IterSingle) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "a", "va")); - Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); +TEST_F(DBTest, MinLevelToCompress2) { + Options options = CurrentOptions(); + CompressionType type = kSnappyCompression; + if (!MinLevelToCompress(type, options, 15, -1, 0)) { + return; + } + Reopen(options); + MinLevelHelper(this, options); - iter->SeekToFirst(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - iter->SeekToFirst(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - - iter->SeekToLast(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - iter->SeekToLast(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - - iter->Seek(""); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); + // do not compress L0 and L1 + for (int i = 0; i < 2; i++) { + options.compression_per_level[i] = kNoCompression; + } + for (int i = 2; i < options.num_levels; i++) { + options.compression_per_level[i] = type; + } + DestroyAndReopen(options); + MinLevelHelper(this, options); +} - iter->Seek("a"); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); +// This test may fail because of a legit case that multiple L0 files +// are trivial moved to L1. +TEST_F(DBTest, DISABLED_RepeatedWritesToSameKey) { + do { + Options options = CurrentOptions(); + options.env = env_; + options.write_buffer_size = 100000; // Small write buffer + CreateAndReopenWithCF({"pikachu"}, options); - iter->Seek("b"); - ASSERT_EQ(IterStatus(iter), "(invalid)"); + // We must have at most one file per level except for level-0, + // which may have up to kL0_StopWritesTrigger files. + const int kMaxFiles = + options.num_levels + options.level0_stop_writes_trigger; - delete iter; + Random rnd(301); + std::string value = + RandomString(&rnd, static_cast(2 * options.write_buffer_size)); + for (int i = 0; i < 5 * kMaxFiles; i++) { + ASSERT_OK(Put(1, "key", value)); + ASSERT_LE(TotalTableFiles(1), kMaxFiles); + } } while (ChangeCompactOptions()); } +#endif // ROCKSDB_LITE -TEST(DBTest, IterMulti) { +TEST_F(DBTest, SparseMerge) { do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "a", "va")); - ASSERT_OK(Put(1, "b", "vb")); - ASSERT_OK(Put(1, "c", "vc")); - Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); + Options options = CurrentOptions(); + options.compression = kNoCompression; + CreateAndReopenWithCF({"pikachu"}, options); - iter->SeekToFirst(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "b->vb"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "c->vc"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - iter->SeekToFirst(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - - iter->SeekToLast(); - ASSERT_EQ(IterStatus(iter), "c->vc"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "b->vb"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - iter->SeekToLast(); - ASSERT_EQ(IterStatus(iter), "c->vc"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - - iter->Seek(""); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Seek("a"); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Seek("ax"); - ASSERT_EQ(IterStatus(iter), "b->vb"); - - iter->Seek("b"); - ASSERT_EQ(IterStatus(iter), "b->vb"); - iter->Seek("z"); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - - // Switch from reverse to forward - iter->SeekToLast(); - iter->Prev(); - iter->Prev(); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "b->vb"); + FillLevels("A", "Z", 1); - // Switch from forward to reverse - iter->SeekToFirst(); - iter->Next(); - iter->Next(); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "b->vb"); - - // Make sure iter stays at snapshot - ASSERT_OK(Put(1, "a", "va2")); - ASSERT_OK(Put(1, "a2", "va3")); - ASSERT_OK(Put(1, "b", "vb2")); - ASSERT_OK(Put(1, "c", "vc2")); - ASSERT_OK(Delete(1, "b")); - iter->SeekToFirst(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "b->vb"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "c->vc"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - iter->SeekToLast(); - ASSERT_EQ(IterStatus(iter), "c->vc"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "b->vb"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - - delete iter; + // Suppose there is: + // small amount of data with prefix A + // large amount of data with prefix B + // small amount of data with prefix C + // and that recent updates have made small changes to all three prefixes. + // Check that we do not do a compaction that merges all of B in one shot. + const std::string value(1000, 'x'); + Put(1, "A", "va"); + // Write approximately 100MB of "B" values + for (int i = 0; i < 100000; i++) { + char key[100]; + snprintf(key, sizeof(key), "B%010d", i); + Put(1, key, value); + } + Put(1, "C", "vc"); + ASSERT_OK(Flush(1)); + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); + + // Make sparse update + Put(1, "A", "va2"); + Put(1, "B100", "bvalue2"); + Put(1, "C", "vc2"); + ASSERT_OK(Flush(1)); + + // Compactions should not cause us to create a situation where + // a file overlaps too much data at the next level. + ASSERT_LE(dbfull()->TEST_MaxNextLevelOverlappingBytes(handles_[1]), + 20 * 1048576); + dbfull()->TEST_CompactRange(0, nullptr, nullptr); + ASSERT_LE(dbfull()->TEST_MaxNextLevelOverlappingBytes(handles_[1]), + 20 * 1048576); + dbfull()->TEST_CompactRange(1, nullptr, nullptr); + ASSERT_LE(dbfull()->TEST_MaxNextLevelOverlappingBytes(handles_[1]), + 20 * 1048576); } while (ChangeCompactOptions()); } -// Check that we can skip over a run of user keys -// by using reseek rather than sequential scan -TEST(DBTest, IterReseek) { +#ifndef ROCKSDB_LITE +static bool Between(uint64_t val, uint64_t low, uint64_t high) { + bool result = (val >= low) && (val <= high); + if (!result) { + fprintf(stderr, "Value %llu is not in range [%llu, %llu]\n", + (unsigned long long)(val), (unsigned long long)(low), + (unsigned long long)(high)); + } + return result; +} + +TEST_F(DBTest, ApproximateSizesMemTable) { Options options = CurrentOptions(); - options.max_sequential_skip_in_iterations = 3; + options.write_buffer_size = 100000000; // Large write buffer + options.compression = kNoCompression; options.create_if_missing = true; - options.statistics = rocksdb::CreateDBStatistics(); - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); - - // insert two keys with same userkey and verify that - // reseek is not invoked. For each of these test cases, - // verify that we can find the next key "b". - ASSERT_OK(Put(1, "a", "one")); - ASSERT_OK(Put(1, "a", "two")); - ASSERT_OK(Put(1, "b", "bone")); - Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); - iter->SeekToFirst(); - ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 0); - ASSERT_EQ(IterStatus(iter), "a->two"); - iter->Next(); - ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 0); - ASSERT_EQ(IterStatus(iter), "b->bone"); - delete iter; - - // insert a total of three keys with same userkey and verify - // that reseek is still not invoked. - ASSERT_OK(Put(1, "a", "three")); - iter = db_->NewIterator(ReadOptions(), handles_[1]); - iter->SeekToFirst(); - ASSERT_EQ(IterStatus(iter), "a->three"); - iter->Next(); - ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 0); - ASSERT_EQ(IterStatus(iter), "b->bone"); - delete iter; - - // insert a total of four keys with same userkey and verify - // that reseek is invoked. - ASSERT_OK(Put(1, "a", "four")); - iter = db_->NewIterator(ReadOptions(), handles_[1]); - iter->SeekToFirst(); - ASSERT_EQ(IterStatus(iter), "a->four"); - ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 0); - iter->Next(); - ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), 1); - ASSERT_EQ(IterStatus(iter), "b->bone"); - delete iter; - - // Testing reverse iterator - // At this point, we have three versions of "a" and one version of "b". - // The reseek statistics is already at 1. - int num_reseeks = - (int)TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION); - - // Insert another version of b and assert that reseek is not invoked - ASSERT_OK(Put(1, "b", "btwo")); - iter = db_->NewIterator(ReadOptions(), handles_[1]); - iter->SeekToLast(); - ASSERT_EQ(IterStatus(iter), "b->btwo"); - ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), - num_reseeks); - iter->Prev(); - ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), - num_reseeks + 1); - ASSERT_EQ(IterStatus(iter), "a->four"); - delete iter; - - // insert two more versions of b. This makes a total of 4 versions - // of b and 4 versions of a. - ASSERT_OK(Put(1, "b", "bthree")); - ASSERT_OK(Put(1, "b", "bfour")); - iter = db_->NewIterator(ReadOptions(), handles_[1]); - iter->SeekToLast(); - ASSERT_EQ(IterStatus(iter), "b->bfour"); - ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), - num_reseeks + 2); - iter->Prev(); - - // the previous Prev call should have invoked reseek - ASSERT_EQ(TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION), - num_reseeks + 3); - ASSERT_EQ(IterStatus(iter), "a->four"); - delete iter; -} + DestroyAndReopen(options); -TEST(DBTest, IterSmallAndLargeMix) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "a", "va")); - ASSERT_OK(Put(1, "b", std::string(100000, 'b'))); - ASSERT_OK(Put(1, "c", "vc")); - ASSERT_OK(Put(1, "d", std::string(100000, 'd'))); - ASSERT_OK(Put(1, "e", std::string(100000, 'e'))); + const int N = 128; + Random rnd(301); + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 1024))); + } + + uint64_t size; + std::string start = Key(50); + std::string end = Key(60); + Range r(start, end); + uint8_t include_both = DB::SizeApproximationFlags::INCLUDE_FILES | + DB::SizeApproximationFlags::INCLUDE_MEMTABLES; + db_->GetApproximateSizes(&r, 1, &size, include_both); + ASSERT_GT(size, 6000); + ASSERT_LT(size, 204800); + // Zero if not including mem table + db_->GetApproximateSizes(&r, 1, &size); + ASSERT_EQ(size, 0); + + start = Key(500); + end = Key(600); + r = Range(start, end); + db_->GetApproximateSizes(&r, 1, &size, include_both); + ASSERT_EQ(size, 0); + + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(Key(1000 + i), RandomString(&rnd, 1024))); + } + + start = Key(500); + end = Key(600); + r = Range(start, end); + db_->GetApproximateSizes(&r, 1, &size, include_both); + ASSERT_EQ(size, 0); + + start = Key(100); + end = Key(1020); + r = Range(start, end); + db_->GetApproximateSizes(&r, 1, &size, include_both); + ASSERT_GT(size, 6000); + + options.max_write_buffer_number = 8; + options.min_write_buffer_number_to_merge = 5; + options.write_buffer_size = 1024 * N; // Not very large + DestroyAndReopen(options); + + int keys[N * 3]; + for (int i = 0; i < N; i++) { + keys[i * 3] = i * 5; + keys[i * 3 + 1] = i * 5 + 1; + keys[i * 3 + 2] = i * 5 + 2; + } + std::random_shuffle(std::begin(keys), std::end(keys)); + + for (int i = 0; i < N * 3; i++) { + ASSERT_OK(Put(Key(keys[i] + 1000), RandomString(&rnd, 1024))); + } + + start = Key(100); + end = Key(300); + r = Range(start, end); + db_->GetApproximateSizes(&r, 1, &size, include_both); + ASSERT_EQ(size, 0); + + start = Key(1050); + end = Key(1080); + r = Range(start, end); + db_->GetApproximateSizes(&r, 1, &size, include_both); + ASSERT_GT(size, 6000); + + start = Key(2100); + end = Key(2300); + r = Range(start, end); + db_->GetApproximateSizes(&r, 1, &size, include_both); + ASSERT_EQ(size, 0); + + start = Key(1050); + end = Key(1080); + r = Range(start, end); + uint64_t size_with_mt, size_without_mt; + db_->GetApproximateSizes(&r, 1, &size_with_mt, include_both); + ASSERT_GT(size_with_mt, 6000); + db_->GetApproximateSizes(&r, 1, &size_without_mt); + ASSERT_EQ(size_without_mt, 0); - Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); + Flush(); - iter->SeekToFirst(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "b->" + std::string(100000, 'b')); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "c->vc"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "d->" + std::string(100000, 'd')); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "e->" + std::string(100000, 'e')); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - - iter->SeekToLast(); - ASSERT_EQ(IterStatus(iter), "e->" + std::string(100000, 'e')); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "d->" + std::string(100000, 'd')); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "c->vc"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "b->" + std::string(100000, 'b')); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "a->va"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "(invalid)"); - - delete iter; - } while (ChangeCompactOptions()); -} + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(Key(i + 1000), RandomString(&rnd, 1024))); + } -TEST(DBTest, IterMultiWithDelete) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "ka", "va")); - ASSERT_OK(Put(1, "kb", "vb")); - ASSERT_OK(Put(1, "kc", "vc")); - ASSERT_OK(Delete(1, "kb")); - ASSERT_EQ("NOT_FOUND", Get(1, "kb")); - - Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); - iter->Seek("kc"); - ASSERT_EQ(IterStatus(iter), "kc->vc"); - if (!CurrentOptions().merge_operator) { - // TODO: merge operator does not support backward iteration yet - if (kPlainTableAllBytesPrefix != option_config_&& - kBlockBasedTableWithWholeKeyHashIndex != option_config_ && - kHashLinkList != option_config_) { - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "ka->va"); - } - } - delete iter; - } while (ChangeOptions()); + start = Key(1050); + end = Key(1080); + r = Range(start, end); + db_->GetApproximateSizes(&r, 1, &size_with_mt, include_both); + db_->GetApproximateSizes(&r, 1, &size_without_mt); + ASSERT_GT(size_with_mt, size_without_mt); + ASSERT_GT(size_without_mt, 6000); } -TEST(DBTest, IterPrevMaxSkip) { - do { - CreateAndReopenWithCF({"pikachu"}); - for (int i = 0; i < 2; i++) { - ASSERT_OK(Put(1, "key1", "v1")); - ASSERT_OK(Put(1, "key2", "v2")); - ASSERT_OK(Put(1, "key3", "v3")); - ASSERT_OK(Put(1, "key4", "v4")); - ASSERT_OK(Put(1, "key5", "v5")); - } - - VerifyIterLast("key5->v5", 1); +TEST_F(DBTest, GetApproximateMemTableStats) { + Options options = CurrentOptions(); + options.write_buffer_size = 100000000; + options.compression = kNoCompression; + options.create_if_missing = true; + DestroyAndReopen(options); - ASSERT_OK(Delete(1, "key5")); - VerifyIterLast("key4->v4", 1); + const int N = 128; + Random rnd(301); + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 1024))); + } + + uint64_t count; + uint64_t size; + + std::string start = Key(50); + std::string end = Key(60); + Range r(start, end); + db_->GetApproximateMemTableStats(r, &count, &size); + ASSERT_GT(count, 0); + ASSERT_LE(count, N); + ASSERT_GT(size, 6000); + ASSERT_LT(size, 204800); + + start = Key(500); + end = Key(600); + r = Range(start, end); + db_->GetApproximateMemTableStats(r, &count, &size); + ASSERT_EQ(count, 0); + ASSERT_EQ(size, 0); - ASSERT_OK(Delete(1, "key4")); - VerifyIterLast("key3->v3", 1); + Flush(); - ASSERT_OK(Delete(1, "key3")); - VerifyIterLast("key2->v2", 1); + start = Key(50); + end = Key(60); + r = Range(start, end); + db_->GetApproximateMemTableStats(r, &count, &size); + ASSERT_EQ(count, 0); + ASSERT_EQ(size, 0); - ASSERT_OK(Delete(1, "key2")); - VerifyIterLast("key1->v1", 1); + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(Key(1000 + i), RandomString(&rnd, 1024))); + } - ASSERT_OK(Delete(1, "key1")); - VerifyIterLast("(invalid)", 1); - } while (ChangeOptions(kSkipMergePut | kSkipNoSeekToLast)); + start = Key(100); + end = Key(1020); + r = Range(start, end); + db_->GetApproximateMemTableStats(r, &count, &size); + ASSERT_GT(count, 20); + ASSERT_GT(size, 6000); } -TEST(DBTest, IterWithSnapshot) { +TEST_F(DBTest, ApproximateSizes) { do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "key1", "val1")); - ASSERT_OK(Put(1, "key2", "val2")); - ASSERT_OK(Put(1, "key3", "val3")); - ASSERT_OK(Put(1, "key4", "val4")); - ASSERT_OK(Put(1, "key5", "val5")); - - const Snapshot *snapshot = db_->GetSnapshot(); - ReadOptions options; - options.snapshot = snapshot; - Iterator* iter = db_->NewIterator(options, handles_[1]); - - // Put more values after the snapshot - ASSERT_OK(Put(1, "key100", "val100")); - ASSERT_OK(Put(1, "key101", "val101")); - - iter->Seek("key5"); - ASSERT_EQ(IterStatus(iter), "key5->val5"); - if (!CurrentOptions().merge_operator) { - // TODO: merge operator does not support backward iteration yet - if (kPlainTableAllBytesPrefix != option_config_&& - kBlockBasedTableWithWholeKeyHashIndex != option_config_ && - kHashLinkList != option_config_) { - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "key4->val4"); - iter->Prev(); - ASSERT_EQ(IterStatus(iter), "key3->val3"); - - iter->Next(); - ASSERT_EQ(IterStatus(iter), "key4->val4"); - iter->Next(); - ASSERT_EQ(IterStatus(iter), "key5->val5"); - } - iter->Next(); - ASSERT_TRUE(!iter->Valid()); - } - db_->ReleaseSnapshot(snapshot); - delete iter; - // skip as HashCuckooRep does not support snapshot - } while (ChangeOptions(kSkipHashCuckoo)); -} + Options options = CurrentOptions(); + options.write_buffer_size = 100000000; // Large write buffer + options.compression = kNoCompression; + options.create_if_missing = true; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); -TEST(DBTest, Recover) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "foo", "v1")); - ASSERT_OK(Put(1, "baz", "v5")); + ASSERT_TRUE(Between(Size("", "xyz", 1), 0, 0)); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_TRUE(Between(Size("", "xyz", 1), 0, 0)); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_EQ("v1", Get(1, "foo")); + // Write 8MB (80 values, each 100K) + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + const int N = 80; + static const int S1 = 100000; + static const int S2 = 105000; // Allow some expansion from metadata + Random rnd(301); + for (int i = 0; i < N; i++) { + ASSERT_OK(Put(1, Key(i), RandomString(&rnd, S1))); + } - ASSERT_EQ("v1", Get(1, "foo")); - ASSERT_EQ("v5", Get(1, "baz")); - ASSERT_OK(Put(1, "bar", "v2")); - ASSERT_OK(Put(1, "foo", "v3")); - - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_EQ("v3", Get(1, "foo")); - ASSERT_OK(Put(1, "foo", "v4")); - ASSERT_EQ("v4", Get(1, "foo")); - ASSERT_EQ("v2", Get(1, "bar")); - ASSERT_EQ("v5", Get(1, "baz")); - } while (ChangeOptions()); -} + // 0 because GetApproximateSizes() does not account for memtable space + ASSERT_TRUE(Between(Size("", Key(50), 1), 0, 0)); -TEST(DBTest, RecoverWithTableHandle) { - do { - Options options; - options.create_if_missing = true; - options.write_buffer_size = 100; - options.disable_auto_compactions = true; - options = CurrentOptions(options); - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); + // Check sizes across recovery by reopening a few times + for (int run = 0; run < 3; run++) { + ReopenWithColumnFamilies({"default", "pikachu"}, options); - ASSERT_OK(Put(1, "foo", "v1")); - ASSERT_OK(Put(1, "bar", "v2")); - ASSERT_OK(Flush(1)); - ASSERT_OK(Put(1, "foo", "v3")); - ASSERT_OK(Put(1, "bar", "v4")); - ASSERT_OK(Flush(1)); - ASSERT_OK(Put(1, "big", std::string(100, 'a'))); - ReopenWithColumnFamilies({"default", "pikachu"}); - - std::vector> files; - dbfull()->TEST_GetFilesMetaData(handles_[1], &files); - int total_files = 0; - for (const auto& level : files) { - total_files += level.size(); - } - ASSERT_EQ(total_files, 3); - for (const auto& level : files) { - for (const auto& file : level) { - if (kInfiniteMaxOpenFiles == option_config_) { - ASSERT_TRUE(file.table_reader_handle != nullptr); - } else { - ASSERT_TRUE(file.table_reader_handle == nullptr); + for (int compact_start = 0; compact_start < N; compact_start += 10) { + for (int i = 0; i < N; i += 10) { + ASSERT_TRUE(Between(Size("", Key(i), 1), S1 * i, S2 * i)); + ASSERT_TRUE(Between(Size("", Key(i) + ".suffix", 1), S1 * (i + 1), + S2 * (i + 1))); + ASSERT_TRUE(Between(Size(Key(i), Key(i + 10), 1), S1 * 10, S2 * 10)); } - } - } - } while (ChangeOptions()); -} + ASSERT_TRUE(Between(Size("", Key(50), 1), S1 * 50, S2 * 50)); + ASSERT_TRUE( + Between(Size("", Key(50) + ".suffix", 1), S1 * 50, S2 * 50)); -TEST(DBTest, IgnoreRecoveredLog) { - std::string backup_logs = dbname_ + "/backup_logs"; + std::string cstart_str = Key(compact_start); + std::string cend_str = Key(compact_start + 9); + Slice cstart = cstart_str; + Slice cend = cend_str; + dbfull()->TEST_CompactRange(0, &cstart, &cend, handles_[1]); + } - // delete old files in backup_logs directory - env_->CreateDirIfMissing(backup_logs); - std::vector old_files; - env_->GetChildren(backup_logs, &old_files); - for (auto& file : old_files) { - if (file != "." && file != "..") { - env_->DeleteFile(backup_logs + "/" + file); + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + ASSERT_GT(NumTableFilesAtLevel(1, 1), 0); } - } + // ApproximateOffsetOf() is not yet implemented in plain table format. + } while (ChangeOptions(kSkipUniversalCompaction | kSkipFIFOCompaction | + kSkipPlainTable | kSkipHashIndex)); +} +TEST_F(DBTest, ApproximateSizes_MixOfSmallAndLarge) { do { Options options = CurrentOptions(); - options.create_if_missing = true; - options.merge_operator = MergeOperators::CreateUInt64AddOperator(); - options.wal_dir = dbname_ + "/logs"; - DestroyAndReopen(&options); - - // fill up the DB - std::string one, two; - PutFixed64(&one, 1); - PutFixed64(&two, 2); - ASSERT_OK(db_->Merge(WriteOptions(), Slice("foo"), Slice(one))); - ASSERT_OK(db_->Merge(WriteOptions(), Slice("foo"), Slice(one))); - ASSERT_OK(db_->Merge(WriteOptions(), Slice("bar"), Slice(one))); - - // copy the logs to backup - std::vector logs; - env_->GetChildren(options.wal_dir, &logs); - for (auto& log : logs) { - if (log != ".." && log != ".") { - CopyFile(options.wal_dir + "/" + log, backup_logs + "/" + log); - } - } + options.compression = kNoCompression; + CreateAndReopenWithCF({"pikachu"}, options); - // recover the DB - Reopen(&options); - ASSERT_EQ(two, Get("foo")); - ASSERT_EQ(one, Get("bar")); - Close(); + Random rnd(301); + std::string big1 = RandomString(&rnd, 100000); + ASSERT_OK(Put(1, Key(0), RandomString(&rnd, 10000))); + ASSERT_OK(Put(1, Key(1), RandomString(&rnd, 10000))); + ASSERT_OK(Put(1, Key(2), big1)); + ASSERT_OK(Put(1, Key(3), RandomString(&rnd, 10000))); + ASSERT_OK(Put(1, Key(4), big1)); + ASSERT_OK(Put(1, Key(5), RandomString(&rnd, 10000))); + ASSERT_OK(Put(1, Key(6), RandomString(&rnd, 300000))); + ASSERT_OK(Put(1, Key(7), RandomString(&rnd, 10000))); - // copy the logs from backup back to wal dir - for (auto& log : logs) { - if (log != ".." && log != ".") { - CopyFile(backup_logs + "/" + log, options.wal_dir + "/" + log); - } - } - // this should ignore the log files, recovery should not happen again - // if the recovery happens, the same merge operator would be called twice, - // leading to incorrect results - Reopen(&options); - ASSERT_EQ(two, Get("foo")); - ASSERT_EQ(one, Get("bar")); - Close(); - Destroy(&options); - Reopen(&options); - Close(); - - // copy the logs from backup back to wal dir - env_->CreateDirIfMissing(options.wal_dir); - for (auto& log : logs) { - if (log != ".." && log != ".") { - CopyFile(backup_logs + "/" + log, options.wal_dir + "/" + log); - } - } - // assert that we successfully recovered only from logs, even though we - // destroyed the DB - Reopen(&options); - ASSERT_EQ(two, Get("foo")); - ASSERT_EQ(one, Get("bar")); - - // Recovery will fail if DB directory doesn't exist. - Destroy(&options); - // copy the logs from backup back to wal dir - env_->CreateDirIfMissing(options.wal_dir); - for (auto& log : logs) { - if (log != ".." && log != ".") { - CopyFile(backup_logs + "/" + log, options.wal_dir + "/" + log); - // we won't be needing this file no more - env_->DeleteFile(backup_logs + "/" + log); - } - } - Status s = TryReopen(&options); - ASSERT_TRUE(!s.ok()); - } while (ChangeOptions(kSkipHashCuckoo)); -} + // Check sizes across recovery by reopening a few times + for (int run = 0; run < 3; run++) { + ReopenWithColumnFamilies({"default", "pikachu"}, options); -TEST(DBTest, RollLog) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "foo", "v1")); - ASSERT_OK(Put(1, "baz", "v5")); + ASSERT_TRUE(Between(Size("", Key(0), 1), 0, 0)); + ASSERT_TRUE(Between(Size("", Key(1), 1), 10000, 11000)); + ASSERT_TRUE(Between(Size("", Key(2), 1), 20000, 21000)); + ASSERT_TRUE(Between(Size("", Key(3), 1), 120000, 121000)); + ASSERT_TRUE(Between(Size("", Key(4), 1), 130000, 131000)); + ASSERT_TRUE(Between(Size("", Key(5), 1), 230000, 231000)); + ASSERT_TRUE(Between(Size("", Key(6), 1), 240000, 241000)); + ASSERT_TRUE(Between(Size("", Key(7), 1), 540000, 541000)); + ASSERT_TRUE(Between(Size("", Key(8), 1), 550000, 560000)); - ReopenWithColumnFamilies({"default", "pikachu"}); - for (int i = 0; i < 10; i++) { - ReopenWithColumnFamilies({"default", "pikachu"}); - } - ASSERT_OK(Put(1, "foo", "v4")); - for (int i = 0; i < 10; i++) { - ReopenWithColumnFamilies({"default", "pikachu"}); + ASSERT_TRUE(Between(Size(Key(3), Key(5), 1), 110000, 111000)); + + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); } - } while (ChangeOptions()); + // ApproximateOffsetOf() is not yet implemented in plain table format. + } while (ChangeOptions(kSkipPlainTable)); } +#endif // ROCKSDB_LITE -TEST(DBTest, WAL) { +#ifndef ROCKSDB_LITE +TEST_F(DBTest, Snapshot) { + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; do { - CreateAndReopenWithCF({"pikachu"}); - WriteOptions writeOpt = WriteOptions(); - writeOpt.disableWAL = true; - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v1")); - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v1")); + CreateAndReopenWithCF({"pikachu"}, CurrentOptions(options_override)); + Put(0, "foo", "0v1"); + Put(1, "foo", "1v1"); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_EQ("v1", Get(1, "foo")); - ASSERT_EQ("v1", Get(1, "bar")); + const Snapshot* s1 = db_->GetSnapshot(); + ASSERT_EQ(1U, GetNumSnapshots()); + uint64_t time_snap1 = GetTimeOldestSnapshots(); + ASSERT_GT(time_snap1, 0U); + Put(0, "foo", "0v2"); + Put(1, "foo", "1v2"); - writeOpt.disableWAL = false; - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v2")); - writeOpt.disableWAL = true; - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v2")); + env_->addon_time_.fetch_add(1); - ReopenWithColumnFamilies({"default", "pikachu"}); - // Both value's should be present. - ASSERT_EQ("v2", Get(1, "bar")); - ASSERT_EQ("v2", Get(1, "foo")); + const Snapshot* s2 = db_->GetSnapshot(); + ASSERT_EQ(2U, GetNumSnapshots()); + ASSERT_EQ(time_snap1, GetTimeOldestSnapshots()); + Put(0, "foo", "0v3"); + Put(1, "foo", "1v3"); - writeOpt.disableWAL = true; - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v3")); - writeOpt.disableWAL = false; - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v3")); - - ReopenWithColumnFamilies({"default", "pikachu"}); - // again both values should be present. - ASSERT_EQ("v3", Get(1, "foo")); - ASSERT_EQ("v3", Get(1, "bar")); - } while (ChangeCompactOptions()); -} + { + ManagedSnapshot s3(db_); + ASSERT_EQ(3U, GetNumSnapshots()); + ASSERT_EQ(time_snap1, GetTimeOldestSnapshots()); + + Put(0, "foo", "0v4"); + Put(1, "foo", "1v4"); + ASSERT_EQ("0v1", Get(0, "foo", s1)); + ASSERT_EQ("1v1", Get(1, "foo", s1)); + ASSERT_EQ("0v2", Get(0, "foo", s2)); + ASSERT_EQ("1v2", Get(1, "foo", s2)); + ASSERT_EQ("0v3", Get(0, "foo", s3.snapshot())); + ASSERT_EQ("1v3", Get(1, "foo", s3.snapshot())); + ASSERT_EQ("0v4", Get(0, "foo")); + ASSERT_EQ("1v4", Get(1, "foo")); + } + + ASSERT_EQ(2U, GetNumSnapshots()); + ASSERT_EQ(time_snap1, GetTimeOldestSnapshots()); + ASSERT_EQ("0v1", Get(0, "foo", s1)); + ASSERT_EQ("1v1", Get(1, "foo", s1)); + ASSERT_EQ("0v2", Get(0, "foo", s2)); + ASSERT_EQ("1v2", Get(1, "foo", s2)); + ASSERT_EQ("0v4", Get(0, "foo")); + ASSERT_EQ("1v4", Get(1, "foo")); -TEST(DBTest, CheckLock) { - do { - DB* localdb; - Options options = CurrentOptions(); - ASSERT_OK(TryReopen(&options)); + db_->ReleaseSnapshot(s1); + ASSERT_EQ("0v2", Get(0, "foo", s2)); + ASSERT_EQ("1v2", Get(1, "foo", s2)); + ASSERT_EQ("0v4", Get(0, "foo")); + ASSERT_EQ("1v4", Get(1, "foo")); + ASSERT_EQ(1U, GetNumSnapshots()); + ASSERT_LT(time_snap1, GetTimeOldestSnapshots()); - // second open should fail - ASSERT_TRUE(!(DB::Open(options, dbname_, &localdb)).ok()); - } while (ChangeCompactOptions()); + db_->ReleaseSnapshot(s2); + ASSERT_EQ(0U, GetNumSnapshots()); + ASSERT_EQ("0v4", Get(0, "foo")); + ASSERT_EQ("1v4", Get(1, "foo")); + } while (ChangeOptions(kSkipHashCuckoo)); } -TEST(DBTest, FlushMultipleMemtable) { +TEST_F(DBTest, HiddenValuesAreRemoved) { + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; do { - Options options = CurrentOptions(); - WriteOptions writeOpt = WriteOptions(); - writeOpt.disableWAL = true; - options.max_write_buffer_number = 4; - options.min_write_buffer_number_to_merge = 3; - CreateAndReopenWithCF({"pikachu"}, &options); - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v1")); - ASSERT_OK(Flush(1)); - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v1")); - - ASSERT_EQ("v1", Get(1, "foo")); - ASSERT_EQ("v1", Get(1, "bar")); - ASSERT_OK(Flush(1)); - } while (ChangeCompactOptions()); -} + Options options = CurrentOptions(options_override); + CreateAndReopenWithCF({"pikachu"}, options); + Random rnd(301); + FillLevels("a", "z", 1); -TEST(DBTest, NumImmutableMemTable) { - do { - Options options = CurrentOptions(); - WriteOptions writeOpt = WriteOptions(); - writeOpt.disableWAL = true; - options.max_write_buffer_number = 4; - options.min_write_buffer_number_to_merge = 3; - options.write_buffer_size = 1000000; - CreateAndReopenWithCF({"pikachu"}, &options); - - std::string big_value(1000000 * 2, 'x'); - std::string num; - SetPerfLevel(kEnableTime);; - ASSERT_TRUE(GetPerfLevel() == kEnableTime); - - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "k1", big_value)); - ASSERT_TRUE(dbfull()->GetProperty(handles_[1], - "rocksdb.num-immutable-mem-table", &num)); - ASSERT_EQ(num, "0"); - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.num-entries-active-mem-table", &num)); - ASSERT_EQ(num, "1"); - perf_context.Reset(); - Get(1, "k1"); - ASSERT_EQ(1, (int) perf_context.get_from_memtable_count); - - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "k2", big_value)); - ASSERT_TRUE(dbfull()->GetProperty(handles_[1], - "rocksdb.num-immutable-mem-table", &num)); - ASSERT_EQ(num, "1"); - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.num-entries-active-mem-table", &num)); - ASSERT_EQ(num, "1"); - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.num-entries-imm-mem-tables", &num)); - ASSERT_EQ(num, "1"); - - perf_context.Reset(); - Get(1, "k1"); - ASSERT_EQ(2, (int) perf_context.get_from_memtable_count); - perf_context.Reset(); - Get(1, "k2"); - ASSERT_EQ(1, (int) perf_context.get_from_memtable_count); - - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "k3", big_value)); - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.cur-size-active-mem-table", &num)); - ASSERT_TRUE(dbfull()->GetProperty(handles_[1], - "rocksdb.num-immutable-mem-table", &num)); - ASSERT_EQ(num, "2"); - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.num-entries-active-mem-table", &num)); - ASSERT_EQ(num, "1"); - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.num-entries-imm-mem-tables", &num)); - ASSERT_EQ(num, "2"); - perf_context.Reset(); - Get(1, "k2"); - ASSERT_EQ(2, (int) perf_context.get_from_memtable_count); - perf_context.Reset(); - Get(1, "k3"); - ASSERT_EQ(1, (int) perf_context.get_from_memtable_count); - perf_context.Reset(); - Get(1, "k1"); - ASSERT_EQ(3, (int) perf_context.get_from_memtable_count); + std::string big = RandomString(&rnd, 50000); + Put(1, "foo", big); + Put(1, "pastfoo", "v"); + const Snapshot* snapshot = db_->GetSnapshot(); + Put(1, "foo", "tiny"); + Put(1, "pastfoo2", "v2"); // Advance sequence number one more ASSERT_OK(Flush(1)); - ASSERT_TRUE(dbfull()->GetProperty(handles_[1], - "rocksdb.num-immutable-mem-table", &num)); - ASSERT_EQ(num, "0"); - ASSERT_TRUE(dbfull()->GetProperty( - handles_[1], "rocksdb.cur-size-active-mem-table", &num)); - // "200" is the size of the metadata of an empty skiplist, this would - // break if we change the default skiplist implementation - ASSERT_EQ(num, "200"); - SetPerfLevel(kDisable); - ASSERT_TRUE(GetPerfLevel() == kDisable); - } while (ChangeCompactOptions()); -} - -class SleepingBackgroundTask { - public: - SleepingBackgroundTask() - : bg_cv_(&mutex_), should_sleep_(true), done_with_sleep_(false) {} - void DoSleep() { - MutexLock l(&mutex_); - while (should_sleep_) { - bg_cv_.Wait(); - } - done_with_sleep_ = true; - bg_cv_.SignalAll(); - } - void WakeUp() { - MutexLock l(&mutex_); - should_sleep_ = false; - bg_cv_.SignalAll(); - } - void WaitUntilDone() { - MutexLock l(&mutex_); - while (!done_with_sleep_) { - bg_cv_.Wait(); - } - } - - static void DoSleepTask(void* arg) { - reinterpret_cast(arg)->DoSleep(); - } - - private: - port::Mutex mutex_; - port::CondVar bg_cv_; // Signalled when background work finishes - bool should_sleep_; - bool done_with_sleep_; -}; - -TEST(DBTest, GetProperty) { - // Set sizes to both background thread pool to be 1 and block them. - env_->SetBackgroundThreads(1, Env::HIGH); - env_->SetBackgroundThreads(1, Env::LOW); - SleepingBackgroundTask sleeping_task_low; - env_->Schedule(&SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, - Env::Priority::LOW); - SleepingBackgroundTask sleeping_task_high; - env_->Schedule(&SleepingBackgroundTask::DoSleepTask, &sleeping_task_high, - Env::Priority::HIGH); - - Options options = CurrentOptions(); - WriteOptions writeOpt = WriteOptions(); - writeOpt.disableWAL = true; - options.compaction_style = kCompactionStyleUniversal; - options.level0_file_num_compaction_trigger = 1; - options.compaction_options_universal.size_ratio = 50; - options.max_background_compactions = 1; - options.max_background_flushes = 1; - options.max_write_buffer_number = 10; - options.min_write_buffer_number_to_merge = 1; - options.write_buffer_size = 1000000; - Reopen(&options); - - std::string big_value(1000000 * 2, 'x'); - std::string num; - uint64_t int_num; - SetPerfLevel(kEnableTime); - - ASSERT_TRUE( - dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); - ASSERT_EQ(int_num, 0U); - - ASSERT_OK(dbfull()->Put(writeOpt, "k1", big_value)); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.num-immutable-mem-table", &num)); - ASSERT_EQ(num, "0"); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.mem-table-flush-pending", &num)); - ASSERT_EQ(num, "0"); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.compaction-pending", &num)); - ASSERT_EQ(num, "0"); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num)); - ASSERT_EQ(num, "1"); - perf_context.Reset(); - - ASSERT_OK(dbfull()->Put(writeOpt, "k2", big_value)); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.num-immutable-mem-table", &num)); - ASSERT_EQ(num, "1"); - ASSERT_OK(dbfull()->Delete(writeOpt, "k-non-existing")); - ASSERT_OK(dbfull()->Put(writeOpt, "k3", big_value)); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.num-immutable-mem-table", &num)); - ASSERT_EQ(num, "2"); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.mem-table-flush-pending", &num)); - ASSERT_EQ(num, "1"); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.compaction-pending", &num)); - ASSERT_EQ(num, "0"); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num)); - ASSERT_EQ(num, "4"); - // Verify the same set of properties through GetIntProperty - ASSERT_TRUE( - dbfull()->GetIntProperty("rocksdb.num-immutable-mem-table", &int_num)); - ASSERT_EQ(int_num, 2U); - ASSERT_TRUE( - dbfull()->GetIntProperty("rocksdb.mem-table-flush-pending", &int_num)); - ASSERT_EQ(int_num, 1U); - ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.compaction-pending", &int_num)); - ASSERT_EQ(int_num, 0U); - ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.estimate-num-keys", &int_num)); - ASSERT_EQ(int_num, 4U); - - ASSERT_TRUE( - dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); - ASSERT_EQ(int_num, 0U); - - sleeping_task_high.WakeUp(); - sleeping_task_high.WaitUntilDone(); - dbfull()->TEST_WaitForFlushMemTable(); - - ASSERT_OK(dbfull()->Put(writeOpt, "k4", big_value)); - ASSERT_OK(dbfull()->Put(writeOpt, "k5", big_value)); - dbfull()->TEST_WaitForFlushMemTable(); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.mem-table-flush-pending", &num)); - ASSERT_EQ(num, "0"); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.compaction-pending", &num)); - ASSERT_EQ(num, "1"); - ASSERT_TRUE(dbfull()->GetProperty("rocksdb.estimate-num-keys", &num)); - ASSERT_EQ(num, "4"); - - ASSERT_TRUE( - dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); - ASSERT_GT(int_num, 0U); + ASSERT_GT(NumTableFilesAtLevel(0, 1), 0); - sleeping_task_low.WakeUp(); - sleeping_task_low.WaitUntilDone(); + ASSERT_EQ(big, Get(1, "foo", snapshot)); + ASSERT_TRUE(Between(Size("", "pastfoo", 1), 50000, 60000)); + db_->ReleaseSnapshot(snapshot); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ tiny, " + big + " ]"); + Slice x("x"); + dbfull()->TEST_CompactRange(0, nullptr, &x, handles_[1]); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ tiny ]"); + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + ASSERT_GE(NumTableFilesAtLevel(1, 1), 1); + dbfull()->TEST_CompactRange(1, nullptr, &x, handles_[1]); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ tiny ]"); - dbfull()->TEST_WaitForFlushMemTable(); - options.max_open_files = 10; - Reopen(&options); - // After reopening, no table reader is loaded, so no memory for table readers - ASSERT_TRUE( - dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); - ASSERT_EQ(int_num, 0U); - ASSERT_TRUE(dbfull()->GetIntProperty("rocksdb.estimate-num-keys", &int_num)); - ASSERT_GT(int_num, 0U); - - // After reading a key, at least one table reader is loaded. - Get("k5"); - ASSERT_TRUE( - dbfull()->GetIntProperty("rocksdb.estimate-table-readers-mem", &int_num)); - ASSERT_GT(int_num, 0U); + ASSERT_TRUE(Between(Size("", "pastfoo", 1), 0, 1000)); + // ApproximateOffsetOf() is not yet implemented in plain table format, + // which is used by Size(). + // skip HashCuckooRep as it does not support snapshot + } while (ChangeOptions(kSkipUniversalCompaction | kSkipFIFOCompaction | + kSkipPlainTable | kSkipHashCuckoo)); } +#endif // ROCKSDB_LITE -TEST(DBTest, FLUSH) { +TEST_F(DBTest, UnremovableSingleDelete) { + // If we compact: + // + // Put(A, v1) Snapshot SingleDelete(A) Put(A, v2) + // + // We do not want to end up with: + // + // Put(A, v1) Snapshot Put(A, v2) + // + // Because a subsequent SingleDelete(A) would delete the Put(A, v2) + // but not Put(A, v1), so Get(A) would return v1. + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; do { - CreateAndReopenWithCF({"pikachu"}); - WriteOptions writeOpt = WriteOptions(); - writeOpt.disableWAL = true; - SetPerfLevel(kEnableTime);; - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v1")); - // this will now also flush the last 2 writes - ASSERT_OK(Flush(1)); - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v1")); + Options options = CurrentOptions(options_override); + options.disable_auto_compactions = true; + CreateAndReopenWithCF({"pikachu"}, options); - perf_context.Reset(); - Get(1, "foo"); - ASSERT_TRUE((int) perf_context.get_from_output_files_time > 0); + Put(1, "foo", "first"); + const Snapshot* snapshot = db_->GetSnapshot(); + SingleDelete(1, "foo"); + Put(1, "foo", "second"); + ASSERT_OK(Flush(1)); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_EQ("v1", Get(1, "foo")); - ASSERT_EQ("v1", Get(1, "bar")); + ASSERT_EQ("first", Get(1, "foo", snapshot)); + ASSERT_EQ("second", Get(1, "foo")); - writeOpt.disableWAL = true; - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v2")); - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v2")); - ASSERT_OK(Flush(1)); + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); + ASSERT_EQ("[ second, SDEL, first ]", AllEntriesFor("foo", 1)); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_EQ("v2", Get(1, "bar")); - perf_context.Reset(); - ASSERT_EQ("v2", Get(1, "foo")); - ASSERT_TRUE((int) perf_context.get_from_output_files_time > 0); + SingleDelete(1, "foo"); - writeOpt.disableWAL = false; - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v3")); - ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v3")); - ASSERT_OK(Flush(1)); + ASSERT_EQ("first", Get(1, "foo", snapshot)); + ASSERT_EQ("NOT_FOUND", Get(1, "foo")); - ReopenWithColumnFamilies({"default", "pikachu"}); - // 'foo' should be there because its put - // has WAL enabled. - ASSERT_EQ("v3", Get(1, "foo")); - ASSERT_EQ("v3", Get(1, "bar")); + dbfull()->CompactRange(CompactRangeOptions(), handles_[1], nullptr, + nullptr); - SetPerfLevel(kDisable); - } while (ChangeCompactOptions()); + ASSERT_EQ("first", Get(1, "foo", snapshot)); + ASSERT_EQ("NOT_FOUND", Get(1, "foo")); + db_->ReleaseSnapshot(snapshot); + // Skip HashCuckooRep as it does not support single delete. FIFO and + // universal compaction do not apply to the test case. Skip MergePut + // because single delete does not get removed when it encounters a merge. + } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction | + kSkipUniversalCompaction | kSkipMergePut)); } -TEST(DBTest, RecoveryWithEmptyLog) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "foo", "v1")); - ASSERT_OK(Put(1, "foo", "v2")); - ReopenWithColumnFamilies({"default", "pikachu"}); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_OK(Put(1, "foo", "v3")); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_EQ("v3", Get(1, "foo")); - } while (ChangeOptions()); +#ifndef ROCKSDB_LITE +TEST_F(DBTest, DeletionMarkers1) { + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); + Put(1, "foo", "v1"); + ASSERT_OK(Flush(1)); + const int last = 2; + MoveFilesToLevel(last, 1); + // foo => v1 is now in last level + ASSERT_EQ(NumTableFilesAtLevel(last, 1), 1); + + // Place a table at level last-1 to prevent merging with preceding mutation + Put(1, "a", "begin"); + Put(1, "z", "end"); + Flush(1); + MoveFilesToLevel(last - 1, 1); + ASSERT_EQ(NumTableFilesAtLevel(last, 1), 1); + ASSERT_EQ(NumTableFilesAtLevel(last - 1, 1), 1); + + Delete(1, "foo"); + Put(1, "foo", "v2"); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, DEL, v1 ]"); + ASSERT_OK(Flush(1)); // Moves to level last-2 + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, v1 ]"); + Slice z("z"); + dbfull()->TEST_CompactRange(last - 2, nullptr, &z, handles_[1]); + // DEL eliminated, but v1 remains because we aren't compacting that level + // (DEL can be eliminated because v2 hides v1). + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, v1 ]"); + dbfull()->TEST_CompactRange(last - 1, nullptr, nullptr, handles_[1]); + // Merging last-1 w/ last, so we are the base level for "foo", so + // DEL is removed. (as is v1). + ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2 ]"); } -// Check that writes done during a memtable compaction are recovered -// if the database is shutdown during the memtable compaction. -TEST(DBTest, RecoverDuringMemtableCompaction) { - do { - Options options; - options.env = env_; - options.write_buffer_size = 1000000; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); +TEST_F(DBTest, DeletionMarkers2) { + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); + Put(1, "foo", "v1"); + ASSERT_OK(Flush(1)); + const int last = 2; + MoveFilesToLevel(last, 1); + // foo => v1 is now in last level + ASSERT_EQ(NumTableFilesAtLevel(last, 1), 1); - // Trigger a long memtable compaction and reopen the database during it - ASSERT_OK(Put(1, "foo", "v1")); // Goes to 1st log file - ASSERT_OK(Put(1, "big1", std::string(10000000, 'x'))); // Fills memtable - ASSERT_OK(Put(1, "big2", std::string(1000, 'y'))); // Triggers compaction - ASSERT_OK(Put(1, "bar", "v2")); // Goes to new log file + // Place a table at level last-1 to prevent merging with preceding mutation + Put(1, "a", "begin"); + Put(1, "z", "end"); + Flush(1); + MoveFilesToLevel(last - 1, 1); + ASSERT_EQ(NumTableFilesAtLevel(last, 1), 1); + ASSERT_EQ(NumTableFilesAtLevel(last - 1, 1), 1); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - ASSERT_EQ("v1", Get(1, "foo")); - ASSERT_EQ("v2", Get(1, "bar")); - ASSERT_EQ(std::string(10000000, 'x'), Get(1, "big1")); - ASSERT_EQ(std::string(1000, 'y'), Get(1, "big2")); - } while (ChangeOptions()); + Delete(1, "foo"); + ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, v1 ]"); + ASSERT_OK(Flush(1)); // Moves to level last-2 + ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, v1 ]"); + dbfull()->TEST_CompactRange(last - 2, nullptr, nullptr, handles_[1]); + // DEL kept: "last" file overlaps + ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, v1 ]"); + dbfull()->TEST_CompactRange(last - 1, nullptr, nullptr, handles_[1]); + // Merging last-1 w/ last, so we are the base level for "foo", so + // DEL is removed. (as is v1). + ASSERT_EQ(AllEntriesFor("foo", 1), "[ ]"); } -TEST(DBTest, MinorCompactionsHappen) { +TEST_F(DBTest, OverlapInLevel0) { do { - Options options; - options.write_buffer_size = 10000; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); - const int N = 500; + // Fill levels 1 and 2 to disable the pushing of new memtables to levels > + // 0. + ASSERT_OK(Put(1, "100", "v100")); + ASSERT_OK(Put(1, "999", "v999")); + Flush(1); + MoveFilesToLevel(2, 1); + ASSERT_OK(Delete(1, "100")); + ASSERT_OK(Delete(1, "999")); + Flush(1); + MoveFilesToLevel(1, 1); + ASSERT_EQ("0,1,1", FilesPerLevel(1)); - int starting_num_tables = TotalTableFiles(1); - for (int i = 0; i < N; i++) { - ASSERT_OK(Put(1, Key(i), Key(i) + std::string(1000, 'v'))); - } - int ending_num_tables = TotalTableFiles(1); - ASSERT_GT(ending_num_tables, starting_num_tables); + // Make files spanning the following ranges in level-0: + // files[0] 200 .. 900 + // files[1] 300 .. 500 + // Note that files are sorted by smallest key. + ASSERT_OK(Put(1, "300", "v300")); + ASSERT_OK(Put(1, "500", "v500")); + Flush(1); + ASSERT_OK(Put(1, "200", "v200")); + ASSERT_OK(Put(1, "600", "v600")); + ASSERT_OK(Put(1, "900", "v900")); + Flush(1); + ASSERT_EQ("2,1,1", FilesPerLevel(1)); - for (int i = 0; i < N; i++) { - ASSERT_EQ(Key(i) + std::string(1000, 'v'), Get(1, Key(i))); - } + // Compact away the placeholder files we created initially + dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); + dbfull()->TEST_CompactRange(2, nullptr, nullptr, handles_[1]); + ASSERT_EQ("2", FilesPerLevel(1)); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); + // Do a memtable compaction. Before bug-fix, the compaction would + // not detect the overlap with level-0 files and would incorrectly place + // the deletion in a deeper level. + ASSERT_OK(Delete(1, "600")); + Flush(1); + ASSERT_EQ("3", FilesPerLevel(1)); + ASSERT_EQ("NOT_FOUND", Get(1, "600")); + } while (ChangeOptions(kSkipUniversalCompaction | kSkipFIFOCompaction)); +} +#endif // ROCKSDB_LITE - for (int i = 0; i < N; i++) { - ASSERT_EQ(Key(i) + std::string(1000, 'v'), Get(1, Key(i))); +TEST_F(DBTest, ComparatorCheck) { + class NewComparator : public Comparator { + public: + virtual const char* Name() const override { + return "rocksdb.NewComparator"; + } + virtual int Compare(const Slice& a, const Slice& b) const override { + return BytewiseComparator()->Compare(a, b); + } + virtual void FindShortestSeparator(std::string* s, + const Slice& l) const override { + BytewiseComparator()->FindShortestSeparator(s, l); } + virtual void FindShortSuccessor(std::string* key) const override { + BytewiseComparator()->FindShortSuccessor(key); + } + }; + Options new_options, options; + NewComparator cmp; + do { + options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); + new_options = CurrentOptions(); + new_options.comparator = &cmp; + // only the non-default column family has non-matching comparator + Status s = TryReopenWithColumnFamilies( + {"default", "pikachu"}, std::vector({options, new_options})); + ASSERT_TRUE(!s.ok()); + ASSERT_TRUE(s.ToString().find("comparator") != std::string::npos) + << s.ToString(); } while (ChangeCompactOptions()); } -TEST(DBTest, ManifestRollOver) { - do { - Options options; - options.max_manifest_file_size = 10 ; // 10 bytes - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - { - ASSERT_OK(Put(1, "manifest_key1", std::string(1000, '1'))); - ASSERT_OK(Put(1, "manifest_key2", std::string(1000, '2'))); - ASSERT_OK(Put(1, "manifest_key3", std::string(1000, '3'))); - uint64_t manifest_before_flush = dbfull()->TEST_Current_Manifest_FileNo(); - ASSERT_OK(Flush(1)); // This should trigger LogAndApply. - uint64_t manifest_after_flush = dbfull()->TEST_Current_Manifest_FileNo(); - ASSERT_GT(manifest_after_flush, manifest_before_flush); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - ASSERT_GT(dbfull()->TEST_Current_Manifest_FileNo(), manifest_after_flush); - // check if a new manifest file got inserted or not. - ASSERT_EQ(std::string(1000, '1'), Get(1, "manifest_key1")); - ASSERT_EQ(std::string(1000, '2'), Get(1, "manifest_key2")); - ASSERT_EQ(std::string(1000, '3'), Get(1, "manifest_key3")); +TEST_F(DBTest, CustomComparator) { + class NumberComparator : public Comparator { + public: + virtual const char* Name() const override { + return "test.NumberComparator"; + } + virtual int Compare(const Slice& a, const Slice& b) const override { + return ToNumber(a) - ToNumber(b); + } + virtual void FindShortestSeparator(std::string* s, + const Slice& l) const override { + ToNumber(*s); // Check format + ToNumber(l); // Check format + } + virtual void FindShortSuccessor(std::string* key) const override { + ToNumber(*key); // Check format } - } while (ChangeCompactOptions()); -} -TEST(DBTest, IdentityAcrossRestarts) { + private: + static int ToNumber(const Slice& x) { + // Check that there are no extra characters. + EXPECT_TRUE(x.size() >= 2 && x[0] == '[' && x[x.size() - 1] == ']') + << EscapeString(x); + int val; + char ignored; + EXPECT_TRUE(sscanf(x.ToString().c_str(), "[%i]%c", &val, &ignored) == 1) + << EscapeString(x); + return val; + } + }; + Options new_options; + NumberComparator cmp; do { - std::string id1; - ASSERT_OK(db_->GetDbIdentity(id1)); + new_options = CurrentOptions(); + new_options.create_if_missing = true; + new_options.comparator = &cmp; + new_options.write_buffer_size = 4096; // Compact more often + new_options.arena_block_size = 4096; + new_options = CurrentOptions(new_options); + DestroyAndReopen(new_options); + CreateAndReopenWithCF({"pikachu"}, new_options); + ASSERT_OK(Put(1, "[10]", "ten")); + ASSERT_OK(Put(1, "[0x14]", "twenty")); + for (int i = 0; i < 2; i++) { + ASSERT_EQ("ten", Get(1, "[10]")); + ASSERT_EQ("ten", Get(1, "[0xa]")); + ASSERT_EQ("twenty", Get(1, "[20]")); + ASSERT_EQ("twenty", Get(1, "[0x14]")); + ASSERT_EQ("NOT_FOUND", Get(1, "[15]")); + ASSERT_EQ("NOT_FOUND", Get(1, "[0xf]")); + Compact(1, "[0]", "[9999]"); + } - Options options = CurrentOptions(); - Reopen(&options); - std::string id2; - ASSERT_OK(db_->GetDbIdentity(id2)); - // id1 should match id2 because identity was not regenerated - ASSERT_EQ(id1.compare(id2), 0); - - std::string idfilename = IdentityFileName(dbname_); - ASSERT_OK(env_->DeleteFile(idfilename)); - Reopen(&options); - std::string id3; - ASSERT_OK(db_->GetDbIdentity(id3)); - // id1 should NOT match id3 because identity was regenerated - ASSERT_NE(id1.compare(id3), 0); + for (int run = 0; run < 2; run++) { + for (int i = 0; i < 1000; i++) { + char buf[100]; + snprintf(buf, sizeof(buf), "[%d]", i * 10); + ASSERT_OK(Put(1, buf, buf)); + } + Compact(1, "[0]", "[1000000]"); + } } while (ChangeCompactOptions()); } -TEST(DBTest, RecoverWithLargeLog) { - do { - { - Options options = CurrentOptions(); - CreateAndReopenWithCF({"pikachu"}, &options); - ASSERT_OK(Put(1, "big1", std::string(200000, '1'))); - ASSERT_OK(Put(1, "big2", std::string(200000, '2'))); - ASSERT_OK(Put(1, "small3", std::string(10, '3'))); - ASSERT_OK(Put(1, "small4", std::string(10, '4'))); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - } +TEST_F(DBTest, DBOpen_Options) { + Options options = CurrentOptions(); + std::string dbname = test::TmpDir(env_) + "/db_options_test"; + ASSERT_OK(DestroyDB(dbname, options)); - // Make sure that if we re-open with a small write buffer size that - // we flush table files in the middle of a large log file. - Options options; - options.write_buffer_size = 100000; - options = CurrentOptions(options); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 3); - ASSERT_EQ(std::string(200000, '1'), Get(1, "big1")); - ASSERT_EQ(std::string(200000, '2'), Get(1, "big2")); - ASSERT_EQ(std::string(10, '3'), Get(1, "small3")); - ASSERT_EQ(std::string(10, '4'), Get(1, "small4")); - ASSERT_GT(NumTableFilesAtLevel(0, 1), 1); - } while (ChangeCompactOptions()); -} + // Does not exist, and create_if_missing == false: error + DB* db = nullptr; + options.create_if_missing = false; + Status s = DB::Open(options, dbname, &db); + ASSERT_TRUE(strstr(s.ToString().c_str(), "does not exist") != nullptr); + ASSERT_TRUE(db == nullptr); -TEST(DBTest, CompactionsGenerateMultipleFiles) { - Options options; - options.write_buffer_size = 100000000; // Large write buffer - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); + // Does not exist, and create_if_missing == true: OK + options.create_if_missing = true; + s = DB::Open(options, dbname, &db); + ASSERT_OK(s); + ASSERT_TRUE(db != nullptr); - Random rnd(301); + delete db; + db = nullptr; - // Write 8MB (80 values, each 100K) - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - std::vector values; - for (int i = 0; i < 80; i++) { - values.push_back(RandomString(&rnd, 100000)); - ASSERT_OK(Put(1, Key(i), values[i])); - } + // Does exist, and error_if_exists == true: error + options.create_if_missing = false; + options.error_if_exists = true; + s = DB::Open(options, dbname, &db); + ASSERT_TRUE(strstr(s.ToString().c_str(), "exists") != nullptr); + ASSERT_TRUE(db == nullptr); - // Reopening moves updates to level-0 - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); + // Does exist, and error_if_exists == false: OK + options.create_if_missing = true; + options.error_if_exists = false; + s = DB::Open(options, dbname, &db); + ASSERT_OK(s); + ASSERT_TRUE(db != nullptr); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - ASSERT_GT(NumTableFilesAtLevel(1, 1), 1); - for (int i = 0; i < 80; i++) { - ASSERT_EQ(Get(1, Key(i)), values[i]); - } + delete db; + db = nullptr; } -TEST(DBTest, CompactionTrigger) { - Options options; - options.write_buffer_size = 100<<10; //100KB - options.num_levels = 3; - options.max_mem_compaction_level = 0; - options.level0_file_num_compaction_trigger = 3; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); +TEST_F(DBTest, DBOpen_Change_NumLevels) { + Options options = CurrentOptions(); + options.create_if_missing = true; + DestroyAndReopen(options); + ASSERT_TRUE(db_ != nullptr); + CreateAndReopenWithCF({"pikachu"}, options); - Random rnd(301); + ASSERT_OK(Put(1, "a", "123")); + ASSERT_OK(Put(1, "b", "234")); + Flush(1); + MoveFilesToLevel(3, 1); + Close(); - for (int num = 0; num < options.level0_file_num_compaction_trigger - 1; - num++) { - std::vector values; - // Write 120KB (12 values, each 10K) - for (int i = 0; i < 12; i++) { - values.push_back(RandomString(&rnd, 10000)); - ASSERT_OK(Put(1, Key(i), values[i])); - } - dbfull()->TEST_WaitForFlushMemTable(handles_[1]); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), num + 1); - } + options.create_if_missing = false; + options.num_levels = 2; + Status s = TryReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_TRUE(strstr(s.ToString().c_str(), "Invalid argument") != nullptr); + ASSERT_TRUE(db_ == nullptr); +} - //generate one more file in level-0, and should trigger level-0 compaction - std::vector values; - for (int i = 0; i < 12; i++) { - values.push_back(RandomString(&rnd, 10000)); - ASSERT_OK(Put(1, Key(i), values[i])); - } - dbfull()->TEST_WaitForCompact(); +TEST_F(DBTest, DestroyDBMetaDatabase) { + std::string dbname = test::TmpDir(env_) + "/db_meta"; + ASSERT_OK(env_->CreateDirIfMissing(dbname)); + std::string metadbname = MetaDatabaseName(dbname, 0); + ASSERT_OK(env_->CreateDirIfMissing(metadbname)); + std::string metametadbname = MetaDatabaseName(metadbname, 0); + ASSERT_OK(env_->CreateDirIfMissing(metametadbname)); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - ASSERT_EQ(NumTableFilesAtLevel(1, 1), 1); -} + // Destroy previous versions if they exist. Using the long way. + Options options = CurrentOptions(); + ASSERT_OK(DestroyDB(metametadbname, options)); + ASSERT_OK(DestroyDB(metadbname, options)); + ASSERT_OK(DestroyDB(dbname, options)); -namespace { -static const int kCDTValueSize = 1000; -static const int kCDTKeysPerBuffer = 4; -static const int kCDTNumLevels = 8; -Options DeletionTriggerOptions() { - Options options; - options.compression = kNoCompression; - options.write_buffer_size = kCDTKeysPerBuffer * (kCDTValueSize + 24); - options.min_write_buffer_number_to_merge = 1; - options.num_levels = kCDTNumLevels; - options.max_mem_compaction_level = 0; - options.level0_file_num_compaction_trigger = 1; - options.target_file_size_base = options.write_buffer_size * 2; - options.target_file_size_multiplier = 2; - options.max_bytes_for_level_base = - options.target_file_size_base * options.target_file_size_multiplier; - options.max_bytes_for_level_multiplier = 2; - options.disable_auto_compactions = false; - return options; -} -} // anonymous namespace + // Setup databases + DB* db = nullptr; + ASSERT_OK(DB::Open(options, dbname, &db)); + delete db; + db = nullptr; + ASSERT_OK(DB::Open(options, metadbname, &db)); + delete db; + db = nullptr; + ASSERT_OK(DB::Open(options, metametadbname, &db)); + delete db; + db = nullptr; -TEST(DBTest, CompactionDeletionTrigger) { - Options options = DeletionTriggerOptions(); - options.create_if_missing = true; + // Delete databases + ASSERT_OK(DestroyDB(dbname, options)); + + // Check if deletion worked. + options.create_if_missing = false; + ASSERT_TRUE(!(DB::Open(options, dbname, &db)).ok()); + ASSERT_TRUE(!(DB::Open(options, metadbname, &db)).ok()); + ASSERT_TRUE(!(DB::Open(options, metametadbname, &db)).ok()); +} - for (int tid = 0; tid < 2; ++tid) { - uint64_t db_size[2]; +#ifndef ROCKSDB_LITE +TEST_F(DBTest, SnapshotFiles) { + do { + Options options = CurrentOptions(); + options.write_buffer_size = 100000000; // Large write buffer + CreateAndReopenWithCF({"pikachu"}, options); - DestroyAndReopen(&options); Random rnd(301); - const int kTestSize = kCDTKeysPerBuffer * 512; + // Write 8MB (80 values, each 100K) + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); std::vector values; - for (int k = 0; k < kTestSize; ++k) { - values.push_back(RandomString(&rnd, kCDTValueSize)); - ASSERT_OK(Put(Key(k), values[k])); + for (int i = 0; i < 80; i++) { + values.push_back(RandomString(&rnd, 100000)); + ASSERT_OK(Put((i < 40), Key(i), values[i])); } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_WaitForCompact(); - db_size[0] = Size(Key(0), Key(kTestSize - 1)); - for (int k = 0; k < kTestSize; ++k) { - ASSERT_OK(Delete(Key(k))); - } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_WaitForCompact(); - db_size[1] = Size(Key(0), Key(kTestSize - 1)); + // assert that nothing makes it to disk yet. + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - // must have much smaller db size. - ASSERT_GT(db_size[0] / 3, db_size[1]); + // get a file snapshot + uint64_t manifest_number = 0; + uint64_t manifest_size = 0; + std::vector files; + dbfull()->DisableFileDeletions(); + dbfull()->GetLiveFiles(files, &manifest_size); - // repeat the test with universal compaction - options.compaction_style = kCompactionStyleUniversal; - options.num_levels = 1; - } -} + // CURRENT, MANIFEST, OPTIONS, *.sst files (one for each CF) + ASSERT_EQ(files.size(), 5U); -TEST(DBTest, CompactionDeletionTriggerReopen) { - for (int tid = 0; tid < 2; ++tid) { - uint64_t db_size[3]; - Options options = DeletionTriggerOptions(); - options.create_if_missing = true; + uint64_t number = 0; + FileType type; - DestroyAndReopen(&options); - Random rnd(301); + // copy these files to a new snapshot directory + std::string snapdir = dbname_ + ".snapdir/"; + ASSERT_OK(env_->CreateDirIfMissing(snapdir)); - // round 1 --- insert key/value pairs. - const int kTestSize = kCDTKeysPerBuffer * 512; - std::vector values; - for (int k = 0; k < kTestSize; ++k) { - values.push_back(RandomString(&rnd, kCDTValueSize)); - ASSERT_OK(Put(Key(k), values[k])); - } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_WaitForCompact(); - db_size[0] = Size(Key(0), Key(kTestSize - 1)); - Close(); + for (size_t i = 0; i < files.size(); i++) { + // our clients require that GetLiveFiles returns + // files with "/" as first character! + ASSERT_EQ(files[i][0], '/'); + std::string src = dbname_ + files[i]; + std::string dest = snapdir + files[i]; - // round 2 --- disable auto-compactions and issue deletions. - options.create_if_missing = false; - options.disable_auto_compactions = true; - Reopen(&options); + uint64_t size; + ASSERT_OK(env_->GetFileSize(src, &size)); - for (int k = 0; k < kTestSize; ++k) { - ASSERT_OK(Delete(Key(k))); - } - db_size[1] = Size(Key(0), Key(kTestSize - 1)); - Close(); - // as auto_compaction is off, we shouldn't see too much reduce - // in db size. - ASSERT_LT(db_size[0] / 3, db_size[1]); - - // round 3 --- reopen db with auto_compaction on and see if - // deletion compensation still work. - options.disable_auto_compactions = false; - Reopen(&options); - // insert relatively small amount of data to trigger auto compaction. - for (int k = 0; k < kTestSize / 10; ++k) { - ASSERT_OK(Put(Key(k), values[k])); + // record the number and the size of the + // latest manifest file + if (ParseFileName(files[i].substr(1), &number, &type)) { + if (type == kDescriptorFile) { + if (number > manifest_number) { + manifest_number = number; + ASSERT_GE(size, manifest_size); + size = manifest_size; // copy only valid MANIFEST data + } + } + } + CopyFile(src, dest, size); } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_WaitForCompact(); - db_size[2] = Size(Key(0), Key(kTestSize - 1)); - // this time we're expecting significant drop in size. - ASSERT_GT(db_size[0] / 3, db_size[2]); - - // repeat the test with universal compaction - options.compaction_style = kCompactionStyleUniversal; - options.num_levels = 1; - } -} - -// This is a static filter used for filtering -// kvs during the compaction process. -static int cfilter_count; -static std::string NEW_VALUE = "NewValue"; - -class KeepFilter : public CompactionFilter { - public: - virtual bool Filter(int level, const Slice& key, const Slice& value, - std::string* new_value, bool* value_changed) const - override { - cfilter_count++; - return false; - } - - virtual const char* Name() const override { return "KeepFilter"; } -}; - -class DeleteFilter : public CompactionFilter { - public: - virtual bool Filter(int level, const Slice& key, const Slice& value, - std::string* new_value, bool* value_changed) const - override { - cfilter_count++; - return true; - } - - virtual const char* Name() const override { return "DeleteFilter"; } -}; -class ChangeFilter : public CompactionFilter { - public: - explicit ChangeFilter() {} + // release file snapshot + dbfull()->DisableFileDeletions(); + // overwrite one key, this key should not appear in the snapshot + std::vector extras; + for (unsigned int i = 0; i < 1; i++) { + extras.push_back(RandomString(&rnd, 100000)); + ASSERT_OK(Put(0, Key(i), extras[i])); + } - virtual bool Filter(int level, const Slice& key, const Slice& value, - std::string* new_value, bool* value_changed) const - override { - assert(new_value != nullptr); - *new_value = NEW_VALUE; - *value_changed = true; - return false; - } + // verify that data in the snapshot are correct + std::vector column_families; + column_families.emplace_back("default", ColumnFamilyOptions()); + column_families.emplace_back("pikachu", ColumnFamilyOptions()); + std::vector cf_handles; + DB* snapdb; + DBOptions opts; + opts.env = env_; + opts.create_if_missing = false; + Status stat = + DB::Open(opts, snapdir, column_families, &cf_handles, &snapdb); + ASSERT_OK(stat); - virtual const char* Name() const override { return "ChangeFilter"; } -}; + ReadOptions roptions; + std::string val; + for (unsigned int i = 0; i < 80; i++) { + stat = snapdb->Get(roptions, cf_handles[i < 40], Key(i), &val); + ASSERT_EQ(values[i].compare(val), 0); + } + for (auto cfh : cf_handles) { + delete cfh; + } + delete snapdb; -class KeepFilterFactory : public CompactionFilterFactory { - public: - explicit KeepFilterFactory(bool check_context = false) - : check_context_(check_context) {} + // look at the new live files after we added an 'extra' key + // and after we took the first snapshot. + uint64_t new_manifest_number = 0; + uint64_t new_manifest_size = 0; + std::vector newfiles; + dbfull()->DisableFileDeletions(); + dbfull()->GetLiveFiles(newfiles, &new_manifest_size); - virtual std::unique_ptr CreateCompactionFilter( - const CompactionFilter::Context& context) override { - if (check_context_) { - ASSERT_EQ(expect_full_compaction_.load(), context.is_full_compaction); - ASSERT_EQ(expect_manual_compaction_.load(), context.is_manual_compaction); + // find the new manifest file. assert that this manifest file is + // the same one as in the previous snapshot. But its size should be + // larger because we added an extra key after taking the + // previous shapshot. + for (size_t i = 0; i < newfiles.size(); i++) { + std::string src = dbname_ + "/" + newfiles[i]; + // record the lognumber and the size of the + // latest manifest file + if (ParseFileName(newfiles[i].substr(1), &number, &type)) { + if (type == kDescriptorFile) { + if (number > new_manifest_number) { + uint64_t size; + new_manifest_number = number; + ASSERT_OK(env_->GetFileSize(src, &size)); + ASSERT_GE(size, new_manifest_size); + } + } + } } - return std::unique_ptr(new KeepFilter()); - } + ASSERT_EQ(manifest_number, new_manifest_number); + ASSERT_GT(new_manifest_size, manifest_size); - virtual const char* Name() const override { return "KeepFilterFactory"; } - bool check_context_; - std::atomic_bool expect_full_compaction_; - std::atomic_bool expect_manual_compaction_; -}; + // release file snapshot + dbfull()->DisableFileDeletions(); + } while (ChangeCompactOptions()); +} +#endif -class DeleteFilterFactory : public CompactionFilterFactory { - public: - virtual std::unique_ptr CreateCompactionFilter( - const CompactionFilter::Context& context) override { - if (context.is_manual_compaction) { - return std::unique_ptr(new DeleteFilter()); +TEST_F(DBTest, PurgeInfoLogs) { + Options options = CurrentOptions(); + options.keep_log_file_num = 5; + options.create_if_missing = true; + for (int mode = 0; mode <= 1; mode++) { + if (mode == 1) { + options.db_log_dir = dbname_ + "_logs"; + env_->CreateDirIfMissing(options.db_log_dir); } else { - return std::unique_ptr(nullptr); + options.db_log_dir = ""; } - } - - virtual const char* Name() const override { return "DeleteFilterFactory"; } -}; - -class ChangeFilterFactory : public CompactionFilterFactory { - public: - explicit ChangeFilterFactory() {} - - virtual std::unique_ptr CreateCompactionFilter( - const CompactionFilter::Context& context) override { - return std::unique_ptr(new ChangeFilter()); - } - - virtual const char* Name() const override { return "ChangeFilterFactory"; } -}; - -// TODO(kailiu) The tests on UniversalCompaction has some issues: -// 1. A lot of magic numbers ("11" or "12"). -// 2. Made assumption on the memtable flush conidtions, which may change from -// time to time. -TEST(DBTest, UniversalCompactionTrigger) { - Options options; - options.compaction_style = kCompactionStyleUniversal; - options.write_buffer_size = 100<<10; //100KB - // trigger compaction if there are >= 4 files - options.level0_file_num_compaction_trigger = 4; - KeepFilterFactory* filter = new KeepFilterFactory(true); - filter->expect_manual_compaction_.store(false); - options.compaction_filter_factory.reset(filter); - - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - Random rnd(301); - int key_idx = 0; - - filter->expect_full_compaction_.store(true); - // Stage 1: - // Generate a set of files at level 0, but don't trigger level-0 - // compaction. - for (int num = 0; num < options.level0_file_num_compaction_trigger - 1; - num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 12; i++) { - ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; + for (int i = 0; i < 8; i++) { + Reopen(options); } - dbfull()->TEST_WaitForFlushMemTable(handles_[1]); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), num + 1); - } - // Generate one more file at level-0, which should trigger level-0 - // compaction. - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForCompact(); - // Suppose each file flushed from mem table has size 1. Now we compact - // (level0_file_num_compaction_trigger+1)=4 files and should have a big - // file of size 4. - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 1); - for (int i = 1; i < options.num_levels ; i++) { - ASSERT_EQ(NumTableFilesAtLevel(i, 1), 0); - } - - // Stage 2: - // Now we have one file at level 0, with size 4. We also have some data in - // mem table. Let's continue generating new files at level 0, but don't - // trigger level-0 compaction. - // First, clean up memtable before inserting new data. This will generate - // a level-0 file, with size around 0.4 (according to previously written - // data amount). - filter->expect_full_compaction_.store(false); - ASSERT_OK(Flush(1)); - for (int num = 0; num < options.level0_file_num_compaction_trigger - 3; - num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; + std::vector files; + env_->GetChildren(options.db_log_dir.empty() ? dbname_ : options.db_log_dir, + &files); + int info_log_count = 0; + for (std::string file : files) { + if (file.find("LOG") != std::string::npos) { + info_log_count++; + } } - dbfull()->TEST_WaitForFlushMemTable(handles_[1]); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), num + 3); - } - - // Generate one more file at level-0, which should trigger level-0 - // compaction. - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForCompact(); - // Before compaction, we have 4 files at level 0, with size 4, 0.4, 1, 1. - // After comapction, we should have 2 files, with size 4, 2.4. - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 2); - for (int i = 1; i < options.num_levels ; i++) { - ASSERT_EQ(NumTableFilesAtLevel(i, 1), 0); - } + ASSERT_EQ(5, info_log_count); - // Stage 3: - // Now we have 2 files at level 0, with size 4 and 2.4. Continue - // generating new files at level 0. - for (int num = 0; num < options.level0_file_num_compaction_trigger - 3; - num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; + Destroy(options); + // For mode (1), test DestroyDB() to delete all the logs under DB dir. + // For mode (2), no info log file should have been put under DB dir. + std::vector db_files; + env_->GetChildren(dbname_, &db_files); + for (std::string file : db_files) { + ASSERT_TRUE(file.find("LOG") == std::string::npos); } - dbfull()->TEST_WaitForFlushMemTable(handles_[1]); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), num + 3); - } - - // Generate one more file at level-0, which should trigger level-0 - // compaction. - for (int i = 0; i < 12; i++) { - ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForCompact(); - // Before compaction, we have 4 files at level 0, with size 4, 2.4, 1, 1. - // After comapction, we should have 3 files, with size 4, 2.4, 2. - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 3); - for (int i = 1; i < options.num_levels ; i++) { - ASSERT_EQ(NumTableFilesAtLevel(i, 1), 0); - } - // Stage 4: - // Now we have 3 files at level 0, with size 4, 2.4, 2. Let's generate a - // new file of size 1. - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForCompact(); - // Level-0 compaction is triggered, but no file will be picked up. - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 4); - for (int i = 1; i < options.num_levels ; i++) { - ASSERT_EQ(NumTableFilesAtLevel(i, 1), 0); - } - - // Stage 5: - // Now we have 4 files at level 0, with size 4, 2.4, 2, 1. Let's generate - // a new file of size 1. - filter->expect_full_compaction_.store(true); - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForCompact(); - // All files at level 0 will be compacted into a single one. - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 1); - for (int i = 1; i < options.num_levels ; i++) { - ASSERT_EQ(NumTableFilesAtLevel(i, 1), 0); + if (mode == 1) { + // Cleaning up + env_->GetChildren(options.db_log_dir, &files); + for (std::string file : files) { + env_->DeleteFile(options.db_log_dir + "/" + file); + } + env_->DeleteDir(options.db_log_dir); + } } } -TEST(DBTest, UniversalCompactionSizeAmplification) { - Options options; - options.compaction_style = kCompactionStyleUniversal; - options.write_buffer_size = 100<<10; //100KB - options.level0_file_num_compaction_trigger = 3; - CreateAndReopenWithCF({"pikachu"}, &options); +#ifndef ROCKSDB_LITE +// Multi-threaded test: +namespace { - // Trigger compaction if size amplification exceeds 110% - options.compaction_options_universal.max_size_amplification_percent = 110; - options = CurrentOptions(options); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); +static const int kColumnFamilies = 10; +static const int kNumThreads = 10; +static const int kTestSeconds = 10; +static const int kNumKeys = 1000; - Random rnd(301); - int key_idx = 0; +struct MTState { + DBTest* test; + std::atomic stop; + std::atomic counter[kNumThreads]; + std::atomic thread_done[kNumThreads]; +}; - // Generate two files in Level 0. Both files are approx the same size. - for (int num = 0; num < options.level0_file_num_compaction_trigger - 1; - num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForFlushMemTable(handles_[1]); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), num + 1); - } - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 2); - - // Flush whatever is remaining in memtable. This is typically - // small, which should not trigger size ratio based compaction - // but will instead trigger size amplification. - ASSERT_OK(Flush(1)); - - dbfull()->TEST_WaitForCompact(); +struct MTThread { + MTState* state; + int id; +}; - // Verify that size amplification did occur - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 1); -} +static void MTThreadBody(void* arg) { + MTThread* t = reinterpret_cast(arg); + int id = t->id; + DB* db = t->state->test->db_; + int counter = 0; + fprintf(stderr, "... starting thread %d\n", id); + Random rnd(1000 + id); + char valbuf[1500]; + while (t->state->stop.load(std::memory_order_acquire) == false) { + t->state->counter[id].store(counter, std::memory_order_release); -TEST(DBTest, UniversalCompactionOptions) { - Options options; - options.compaction_style = kCompactionStyleUniversal; - options.write_buffer_size = 100<<10; //100KB - options.level0_file_num_compaction_trigger = 4; - options.num_levels = 1; - options.compaction_options_universal.compression_size_percent = -1; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); + int key = rnd.Uniform(kNumKeys); + char keybuf[20]; + snprintf(keybuf, sizeof(keybuf), "%016d", key); - Random rnd(301); - int key_idx = 0; + if (rnd.OneIn(2)) { + // Write values of the form . + // into each of the CFs + // We add some padding for force compactions. + int unique_id = rnd.Uniform(1000000); - for (int num = 0; num < options.level0_file_num_compaction_trigger; num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; + // Half of the time directly use WriteBatch. Half of the time use + // WriteBatchWithIndex. + if (rnd.OneIn(2)) { + WriteBatch batch; + for (int cf = 0; cf < kColumnFamilies; ++cf) { + snprintf(valbuf, sizeof(valbuf), "%d.%d.%d.%d.%-1000d", key, id, + static_cast(counter), cf, unique_id); + batch.Put(t->state->test->handles_[cf], Slice(keybuf), Slice(valbuf)); + } + ASSERT_OK(db->Write(WriteOptions(), &batch)); + } else { + WriteBatchWithIndex batch(db->GetOptions().comparator); + for (int cf = 0; cf < kColumnFamilies; ++cf) { + snprintf(valbuf, sizeof(valbuf), "%d.%d.%d.%d.%-1000d", key, id, + static_cast(counter), cf, unique_id); + batch.Put(t->state->test->handles_[cf], Slice(keybuf), Slice(valbuf)); + } + ASSERT_OK(db->Write(WriteOptions(), batch.GetWriteBatch())); + } + } else { + // Read a value and verify that it matches the pattern written above + // and that writes to all column families were atomic (unique_id is the + // same) + std::vector keys(kColumnFamilies, Slice(keybuf)); + std::vector values; + std::vector statuses = + db->MultiGet(ReadOptions(), t->state->test->handles_, keys, &values); + Status s = statuses[0]; + // all statuses have to be the same + for (size_t i = 1; i < statuses.size(); ++i) { + // they are either both ok or both not-found + ASSERT_TRUE((s.ok() && statuses[i].ok()) || + (s.IsNotFound() && statuses[i].IsNotFound())); + } + if (s.IsNotFound()) { + // Key has not yet been written + } else { + // Check that the writer thread counter is >= the counter in the value + ASSERT_OK(s); + int unique_id = -1; + for (int i = 0; i < kColumnFamilies; ++i) { + int k, w, c, cf, u; + ASSERT_EQ(5, sscanf(values[i].c_str(), "%d.%d.%d.%d.%d", &k, &w, &c, + &cf, &u)) + << values[i]; + ASSERT_EQ(k, key); + ASSERT_GE(w, 0); + ASSERT_LT(w, kNumThreads); + ASSERT_LE(c, t->state->counter[w].load(std::memory_order_acquire)); + ASSERT_EQ(cf, i); + if (i == 0) { + unique_id = u; + } else { + // this checks that updates across column families happened + // atomically -- all unique ids are the same + ASSERT_EQ(u, unique_id); + } + } + } } - dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + counter++; + } + t->state->thread_done[id].store(true, std::memory_order_release); + fprintf(stderr, "... stopping thread %d after %d ops\n", id, int(counter)); +} - if (num < options.level0_file_num_compaction_trigger - 1) { - ASSERT_EQ(NumTableFilesAtLevel(0, 1), num + 1); +} // namespace + +class MultiThreadedDBTest : public DBTest, + public ::testing::WithParamInterface { + public: + virtual void SetUp() override { option_config_ = GetParam(); } + + static std::vector GenerateOptionConfigs() { + std::vector optionConfigs; + for (int optionConfig = kDefault; optionConfig < kEnd; ++optionConfig) { + // skip as HashCuckooRep does not support snapshot + if (optionConfig != kHashCuckoo) { + optionConfigs.push_back(optionConfig); + } } + return optionConfigs; } +}; - dbfull()->TEST_WaitForCompact(); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 1); - for (int i = 1; i < options.num_levels ; i++) { - ASSERT_EQ(NumTableFilesAtLevel(i, 1), 0); +TEST_P(MultiThreadedDBTest, MultiThreaded) { + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; + Options options = CurrentOptions(options_override); + std::vector cfs; + for (int i = 1; i < kColumnFamilies; ++i) { + cfs.push_back(ToString(i)); + } + Reopen(options); + CreateAndReopenWithCF(cfs, options); + // Initialize state + MTState mt; + mt.test = this; + mt.stop.store(false, std::memory_order_release); + for (int id = 0; id < kNumThreads; id++) { + mt.counter[id].store(0, std::memory_order_release); + mt.thread_done[id].store(false, std::memory_order_release); + } + + // Start threads + MTThread thread[kNumThreads]; + for (int id = 0; id < kNumThreads; id++) { + thread[id].state = &mt; + thread[id].id = id; + env_->StartThread(MTThreadBody, &thread[id]); + } + + // Let them run for a while + env_->SleepForMicroseconds(kTestSeconds * 1000000); + + // Stop the threads and wait for them to finish + mt.stop.store(true, std::memory_order_release); + for (int id = 0; id < kNumThreads; id++) { + while (mt.thread_done[id].load(std::memory_order_acquire) == false) { + env_->SleepForMicroseconds(100000); + } } } -TEST(DBTest, UniversalCompactionStopStyleSimilarSize) { - Options options = CurrentOptions(); - options.compaction_style = kCompactionStyleUniversal; - options.write_buffer_size = 100<<10; //100KB - // trigger compaction if there are >= 4 files - options.level0_file_num_compaction_trigger = 4; - options.compaction_options_universal.size_ratio = 10; - options.compaction_options_universal.stop_style = kCompactionStopStyleSimilarSize; - options.num_levels=1; - Reopen(&options); +INSTANTIATE_TEST_CASE_P( + MultiThreaded, MultiThreadedDBTest, + ::testing::ValuesIn(MultiThreadedDBTest::GenerateOptionConfigs())); +#endif // ROCKSDB_LITE - Random rnd(301); - int key_idx = 0; +// Group commit test: +namespace { - // Stage 1: - // Generate a set of files at level 0, but don't trigger level-0 - // compaction. - for (int num = 0; - num < options.level0_file_num_compaction_trigger-1; - num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForFlushMemTable(); - ASSERT_EQ(NumTableFilesAtLevel(0), num + 1); - } +static const int kGCNumThreads = 4; +static const int kGCNumKeys = 1000; - // Generate one more file at level-0, which should trigger level-0 - // compaction. - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForCompact(); - // Suppose each file flushed from mem table has size 1. Now we compact - // (level0_file_num_compaction_trigger+1)=4 files and should have a big - // file of size 4. - ASSERT_EQ(NumTableFilesAtLevel(0), 1); +struct GCThread { + DB* db; + int id; + std::atomic done; +}; - // Stage 2: - // Now we have one file at level 0, with size 4. We also have some data in - // mem table. Let's continue generating new files at level 0, but don't - // trigger level-0 compaction. - // First, clean up memtable before inserting new data. This will generate - // a level-0 file, with size around 0.4 (according to previously written - // data amount). - dbfull()->Flush(FlushOptions()); - for (int num = 0; - num < options.level0_file_num_compaction_trigger-3; - num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForFlushMemTable(); - ASSERT_EQ(NumTableFilesAtLevel(0), num + 3); - } +static void GCThreadBody(void* arg) { + GCThread* t = reinterpret_cast(arg); + int id = t->id; + DB* db = t->db; + WriteOptions wo; - // Generate one more file at level-0, which should trigger level-0 - // compaction. - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForCompact(); - // Before compaction, we have 4 files at level 0, with size 4, 0.4, 1, 1. - // After compaction, we should have 3 files, with size 4, 0.4, 2. - ASSERT_EQ(NumTableFilesAtLevel(0), 3); - // Stage 3: - // Now we have 3 files at level 0, with size 4, 0.4, 2. Generate one - // more file at level-0, which should trigger level-0 compaction. - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(Key(key_idx), RandomString(&rnd, 10000))); - key_idx++; + for (int i = 0; i < kGCNumKeys; ++i) { + std::string kv(ToString(i + id * kGCNumKeys)); + ASSERT_OK(db->Put(wo, kv, kv)); } - dbfull()->TEST_WaitForCompact(); - // Level-0 compaction is triggered, but no file will be picked up. - ASSERT_EQ(NumTableFilesAtLevel(0), 4); + t->done = true; } -#if defined(SNAPPY) -TEST(DBTest, CompressedCache) { - int num_iter = 80; - - // Run this test three iterations. - // Iteration 1: only a uncompressed block cache - // Iteration 2: only a compressed block cache - // Iteration 3: both block cache and compressed cache - // Iteration 4: both block cache and compressed cache, but DB is not - // compressed - for (int iter = 0; iter < 4; iter++) { - Options options; - options.write_buffer_size = 64*1024; // small write buffer +} // namespace + +TEST_F(DBTest, GroupCommitTest) { + do { + Options options = CurrentOptions(); + options.env = env_; + env_->log_write_slowdown_.store(100); options.statistics = rocksdb::CreateDBStatistics(); + Reopen(options); - BlockBasedTableOptions table_options; - switch (iter) { - case 0: - // only uncompressed block cache - table_options.block_cache = NewLRUCache(8*1024); - table_options.block_cache_compressed = nullptr; - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - break; - case 1: - // no block cache, only compressed cache - table_options.no_block_cache = true; - table_options.block_cache = nullptr; - table_options.block_cache_compressed = NewLRUCache(8*1024); - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - break; - case 2: - // both compressed and uncompressed block cache - table_options.block_cache = NewLRUCache(1024); - table_options.block_cache_compressed = NewLRUCache(8*1024); - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - break; - case 3: - // both block cache and compressed cache, but DB is not compressed - // also, make block cache sizes bigger, to trigger block cache hits - table_options.block_cache = NewLRUCache(1024 * 1024); - table_options.block_cache_compressed = NewLRUCache(8 * 1024 * 1024); - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - options.compression = kNoCompression; - break; - default: - ASSERT_TRUE(false); + // Start threads + GCThread thread[kGCNumThreads]; + for (int id = 0; id < kGCNumThreads; id++) { + thread[id].id = id; + thread[id].db = db_; + thread[id].done = false; + env_->StartThread(GCThreadBody, &thread[id]); } - CreateAndReopenWithCF({"pikachu"}, &options); - // default column family doesn't have block cache - Options no_block_cache_opts; - no_block_cache_opts.statistics = options.statistics; - BlockBasedTableOptions table_options_no_bc; - table_options_no_bc.no_block_cache = true; - no_block_cache_opts.table_factory.reset( - NewBlockBasedTableFactory(table_options_no_bc)); - ReopenWithColumnFamilies({"default", "pikachu"}, - {&no_block_cache_opts, &options}); - - Random rnd(301); - // Write 8MB (80 values, each 100K) - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - std::vector values; - std::string str; - for (int i = 0; i < num_iter; i++) { - if (i % 4 == 0) { // high compression ratio - str = RandomString(&rnd, 1000); + for (int id = 0; id < kGCNumThreads; id++) { + while (thread[id].done == false) { + env_->SleepForMicroseconds(100000); } - values.push_back(str); - ASSERT_OK(Put(1, Key(i), values[i])); } + env_->log_write_slowdown_.store(0); - // flush all data from memtable so that reads are from block cache - ASSERT_OK(Flush(1)); + ASSERT_GT(TestGetTickerCount(options, WRITE_DONE_BY_OTHER), 0); - for (int i = 0; i < num_iter; i++) { - ASSERT_EQ(Get(1, Key(i)), values[i]); + std::vector expected_db; + for (int i = 0; i < kGCNumThreads * kGCNumKeys; ++i) { + expected_db.push_back(ToString(i)); } + std::sort(expected_db.begin(), expected_db.end()); - // check that we triggered the appropriate code paths in the cache - switch (iter) { - case 0: - // only uncompressed block cache - ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_MISS), 0); - ASSERT_EQ(TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_MISS), 0); - break; - case 1: - // no block cache, only compressed cache - ASSERT_EQ(TestGetTickerCount(options, BLOCK_CACHE_MISS), 0); - ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_MISS), 0); - break; - case 2: - // both compressed and uncompressed block cache - ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_MISS), 0); - ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_MISS), 0); - break; - case 3: - // both compressed and uncompressed block cache - ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_MISS), 0); - ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_HIT), 0); - ASSERT_GT(TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_MISS), 0); - // compressed doesn't have any hits since blocks are not compressed on - // storage - ASSERT_EQ(TestGetTickerCount(options, BLOCK_CACHE_COMPRESSED_HIT), 0); - break; - default: - ASSERT_TRUE(false); + Iterator* itr = db_->NewIterator(ReadOptions()); + itr->SeekToFirst(); + for (auto x : expected_db) { + ASSERT_TRUE(itr->Valid()); + ASSERT_EQ(itr->key().ToString(), x); + ASSERT_EQ(itr->value().ToString(), x); + itr->Next(); } + ASSERT_TRUE(!itr->Valid()); + delete itr; - options.create_if_missing = true; - DestroyAndReopen(&options); - } + HistogramData hist_data; + options.statistics->histogramData(DB_WRITE, &hist_data); + ASSERT_GT(hist_data.average, 0.0); + } while (ChangeOptions(kSkipNoSeekToLast)); } -static std::string CompressibleString(Random* rnd, int len) { - std::string r; - test::CompressibleString(rnd, 0.8, len, &r); - return r; +namespace { +typedef std::map KVMap; } -TEST(DBTest, UniversalCompactionCompressRatio1) { - Options options; - options.compaction_style = kCompactionStyleUniversal; - options.write_buffer_size = 100<<10; //100KB - options.level0_file_num_compaction_trigger = 2; - options.num_levels = 1; - options.compaction_options_universal.compression_size_percent = 70; - options = CurrentOptions(options); - Reopen(&options); - - Random rnd(301); - int key_idx = 0; +class ModelDB : public DB { + public: + class ModelSnapshot : public Snapshot { + public: + KVMap map_; - // The first compaction (2) is compressed. - for (int num = 0; num < 2; num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(Key(key_idx), CompressibleString(&rnd, 10000))); - key_idx++; + virtual SequenceNumber GetSequenceNumber() const override { + // no need to call this + assert(false); + return 0; } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_WaitForCompact(); - } - ASSERT_LT((int)dbfull()->TEST_GetLevel0TotalSize(), 110000 * 2 * 0.9); + }; - // The second compaction (4) is compressed - for (int num = 0; num < 2; num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(Key(key_idx), CompressibleString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_WaitForCompact(); + explicit ModelDB(const Options& options) : options_(options) {} + using DB::Put; + virtual Status Put(const WriteOptions& o, ColumnFamilyHandle* cf, + const Slice& k, const Slice& v) override { + WriteBatch batch; + batch.Put(cf, k, v); + return Write(o, &batch); + } + using DB::Delete; + virtual Status Delete(const WriteOptions& o, ColumnFamilyHandle* cf, + const Slice& key) override { + WriteBatch batch; + batch.Delete(cf, key); + return Write(o, &batch); + } + using DB::SingleDelete; + virtual Status SingleDelete(const WriteOptions& o, ColumnFamilyHandle* cf, + const Slice& key) override { + WriteBatch batch; + batch.SingleDelete(cf, key); + return Write(o, &batch); + } + using DB::Merge; + virtual Status Merge(const WriteOptions& o, ColumnFamilyHandle* cf, + const Slice& k, const Slice& v) override { + WriteBatch batch; + batch.Merge(cf, k, v); + return Write(o, &batch); + } + using DB::Get; + virtual Status Get(const ReadOptions& options, ColumnFamilyHandle* cf, + const Slice& key, PinnableSlice* value) override { + return Status::NotSupported(key); } - ASSERT_LT((int)dbfull()->TEST_GetLevel0TotalSize(), 110000 * 4 * 0.9); - // The third compaction (2 4) is compressed since this time it is - // (1 1 3.2) and 3.2/5.2 doesn't reach ratio. - for (int num = 0; num < 2; num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(Key(key_idx), CompressibleString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_WaitForCompact(); + using DB::MultiGet; + virtual std::vector MultiGet( + const ReadOptions& options, + const std::vector& column_family, + const std::vector& keys, + std::vector* values) override { + std::vector s(keys.size(), + Status::NotSupported("Not implemented.")); + return s; } - ASSERT_LT((int)dbfull()->TEST_GetLevel0TotalSize(), 110000 * 6 * 0.9); - // When we start for the compaction up to (2 4 8), the latest - // compressed is not compressed. - for (int num = 0; num < 8; num++) { - // Write 110KB (11 values, each 10K) - for (int i = 0; i < 11; i++) { - ASSERT_OK(Put(Key(key_idx), CompressibleString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_WaitForCompact(); +#ifndef ROCKSDB_LITE + using DB::IngestExternalFile; + virtual Status IngestExternalFile( + ColumnFamilyHandle* column_family, + const std::vector& external_files, + const IngestExternalFileOptions& options) override { + return Status::NotSupported("Not implemented."); } - ASSERT_GT((int)dbfull()->TEST_GetLevel0TotalSize(), - 110000 * 11 * 0.8 + 110000 * 2); -} -TEST(DBTest, UniversalCompactionCompressRatio2) { - Options options; - options.compaction_style = kCompactionStyleUniversal; - options.write_buffer_size = 100<<10; //100KB - options.level0_file_num_compaction_trigger = 2; - options.num_levels = 1; - options.compaction_options_universal.compression_size_percent = 95; - options = CurrentOptions(options); - Reopen(&options); + virtual Status VerifyChecksum() override { + return Status::NotSupported("Not implemented."); + } - Random rnd(301); - int key_idx = 0; + using DB::GetPropertiesOfAllTables; + virtual Status GetPropertiesOfAllTables( + ColumnFamilyHandle* column_family, + TablePropertiesCollection* props) override { + return Status(); + } - // When we start for the compaction up to (2 4 8), the latest - // compressed is compressed given the size ratio to compress. - for (int num = 0; num < 14; num++) { - // Write 120KB (12 values, each 10K) - for (int i = 0; i < 12; i++) { - ASSERT_OK(Put(Key(key_idx), CompressibleString(&rnd, 10000))); - key_idx++; - } - dbfull()->TEST_WaitForFlushMemTable(); - dbfull()->TEST_WaitForCompact(); + virtual Status GetPropertiesOfTablesInRange( + ColumnFamilyHandle* column_family, const Range* range, std::size_t n, + TablePropertiesCollection* props) override { + return Status(); } - ASSERT_LT((int)dbfull()->TEST_GetLevel0TotalSize(), - 120000 * 12 * 0.8 + 120000 * 2); -} +#endif // ROCKSDB_LITE -TEST(DBTest, FailMoreDbPaths) { - Options options; - options.db_paths.emplace_back(dbname_, 10000000); - options.db_paths.emplace_back(dbname_ + "_2", 1000000); - options.db_paths.emplace_back(dbname_ + "_3", 1000000); - options.db_paths.emplace_back(dbname_ + "_4", 1000000); - options.db_paths.emplace_back(dbname_ + "_5", 1000000); - ASSERT_TRUE(TryReopen(&options).IsNotSupported()); -} - -TEST(DBTest, UniversalCompactionSecondPathRatio) { - Options options; - options.db_paths.emplace_back(dbname_, 500 * 1024); - options.db_paths.emplace_back(dbname_ + "_2", 1024 * 1024 * 1024); - options.compaction_style = kCompactionStyleUniversal; - options.write_buffer_size = 100 << 10; // 100KB - options.level0_file_num_compaction_trigger = 2; - options.num_levels = 1; - options = CurrentOptions(options); + using DB::KeyMayExist; + virtual bool KeyMayExist(const ReadOptions& options, + ColumnFamilyHandle* column_family, const Slice& key, + std::string* value, + bool* value_found = nullptr) override { + if (value_found != nullptr) { + *value_found = false; + } + return true; // Not Supported directly + } + using DB::NewIterator; + virtual Iterator* NewIterator(const ReadOptions& options, + ColumnFamilyHandle* column_family) override { + if (options.snapshot == nullptr) { + KVMap* saved = new KVMap; + *saved = map_; + return new ModelIter(saved, true); + } else { + const KVMap* snapshot_state = + &(reinterpret_cast(options.snapshot)->map_); + return new ModelIter(snapshot_state, false); + } + } + virtual Status NewIterators( + const ReadOptions& options, + const std::vector& column_family, + std::vector* iterators) override { + return Status::NotSupported("Not supported yet"); + } + virtual const Snapshot* GetSnapshot() override { + ModelSnapshot* snapshot = new ModelSnapshot; + snapshot->map_ = map_; + return snapshot; + } - std::vector filenames; - env_->GetChildren(options.db_paths[1].path, &filenames); - // Delete archival files. - for (size_t i = 0; i < filenames.size(); ++i) { - env_->DeleteFile(options.db_paths[1].path + "/" + filenames[i]); + virtual void ReleaseSnapshot(const Snapshot* snapshot) override { + delete reinterpret_cast(snapshot); } - env_->DeleteDir(options.db_paths[1].path); - Reopen(&options); - Random rnd(301); - int key_idx = 0; + virtual Status Write(const WriteOptions& options, + WriteBatch* batch) override { + class Handler : public WriteBatch::Handler { + public: + KVMap* map_; + virtual void Put(const Slice& key, const Slice& value) override { + (*map_)[key.ToString()] = value.ToString(); + } + virtual void Merge(const Slice& key, const Slice& value) override { + // ignore merge for now + // (*map_)[key.ToString()] = value.ToString(); + } + virtual void Delete(const Slice& key) override { + map_->erase(key.ToString()); + } + }; + Handler handler; + handler.map_ = &map_; + return batch->Iterate(&handler); + } - // First three 110KB files are not going to second path. - // After that, (100K, 200K) - for (int num = 0; num < 3; num++) { - GenerateNewFile(&rnd, &key_idx); + using DB::GetProperty; + virtual bool GetProperty(ColumnFamilyHandle* column_family, + const Slice& property, std::string* value) override { + return false; + } + using DB::GetIntProperty; + virtual bool GetIntProperty(ColumnFamilyHandle* column_family, + const Slice& property, uint64_t* value) override { + return false; + } + using DB::GetMapProperty; + virtual bool GetMapProperty(ColumnFamilyHandle* column_family, + const Slice& property, + std::map* value) override { + return false; + } + using DB::GetAggregatedIntProperty; + virtual bool GetAggregatedIntProperty(const Slice& property, + uint64_t* value) override { + return false; + } + using DB::GetApproximateSizes; + virtual void GetApproximateSizes(ColumnFamilyHandle* column_family, + const Range* range, int n, uint64_t* sizes, + uint8_t include_flags + = INCLUDE_FILES) override { + for (int i = 0; i < n; i++) { + sizes[i] = 0; + } + } + using DB::GetApproximateMemTableStats; + virtual void GetApproximateMemTableStats(ColumnFamilyHandle* column_family, + const Range& range, + uint64_t* const count, + uint64_t* const size) override { + *count = 0; + *size = 0; + } + using DB::CompactRange; + virtual Status CompactRange(const CompactRangeOptions& options, + ColumnFamilyHandle* column_family, + const Slice* start, const Slice* end) override { + return Status::NotSupported("Not supported operation."); } - // Another 110KB triggers a compaction to 400K file to second path - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + virtual Status SetDBOptions( + const std::unordered_map& new_options) + override { + return Status::NotSupported("Not supported operation."); + } - // (1, 4) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(1, GetSstFileCount(dbname_)); + using DB::CompactFiles; + virtual Status CompactFiles(const CompactionOptions& compact_options, + ColumnFamilyHandle* column_family, + const std::vector& input_file_names, + const int output_level, + const int output_path_id = -1) override { + return Status::NotSupported("Not supported operation."); + } - // (1,1,4) -> (2, 4) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(1, GetSstFileCount(dbname_)); + Status PauseBackgroundWork() override { + return Status::NotSupported("Not supported operation."); + } - // (1, 2, 4) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(2, GetSstFileCount(dbname_)); + Status ContinueBackgroundWork() override { + return Status::NotSupported("Not supported operation."); + } - // (1, 1, 2, 4) -> (8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(0, GetSstFileCount(dbname_)); + Status EnableAutoCompaction( + const std::vector& column_family_handles) override { + return Status::NotSupported("Not supported operation."); + } - // (1, 8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(1, GetSstFileCount(dbname_)); + using DB::NumberLevels; + virtual int NumberLevels(ColumnFamilyHandle* column_family) override { + return 1; + } - // (1, 1, 8) -> (2, 8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(1, GetSstFileCount(dbname_)); + using DB::MaxMemCompactionLevel; + virtual int MaxMemCompactionLevel( + ColumnFamilyHandle* column_family) override { + return 1; + } - // (1, 2, 8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(2, GetSstFileCount(dbname_)); + using DB::Level0StopWriteTrigger; + virtual int Level0StopWriteTrigger( + ColumnFamilyHandle* column_family) override { + return -1; + } - // (1, 1, 2, 8) -> (4, 8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(2, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(0, GetSstFileCount(dbname_)); + virtual const std::string& GetName() const override { return name_; } - // (1, 4, 8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(2, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(1, GetSstFileCount(dbname_)); + virtual Env* GetEnv() const override { return nullptr; } - for (int i = 0; i < key_idx; i++) { - auto v = Get(Key(i)); - ASSERT_NE(v, "NOT_FOUND"); - ASSERT_TRUE(v.size() == 1 || v.size() == 10000); + using DB::GetOptions; + virtual Options GetOptions(ColumnFamilyHandle* column_family) const override { + return options_; } - Reopen(&options); + using DB::GetDBOptions; + virtual DBOptions GetDBOptions() const override { return options_; } - for (int i = 0; i < key_idx; i++) { - auto v = Get(Key(i)); - ASSERT_NE(v, "NOT_FOUND"); - ASSERT_TRUE(v.size() == 1 || v.size() == 10000); + using DB::Flush; + virtual Status Flush(const rocksdb::FlushOptions& options, + ColumnFamilyHandle* column_family) override { + Status ret; + return ret; } - Destroy(&options); -} + virtual Status SyncWAL() override { return Status::OK(); } -TEST(DBTest, UniversalCompactionFourPaths) { - Options options; - options.db_paths.emplace_back(dbname_, 300 * 1024); - options.db_paths.emplace_back(dbname_ + "_2", 300 * 1024); - options.db_paths.emplace_back(dbname_ + "_3", 500 * 1024); - options.db_paths.emplace_back(dbname_ + "_4", 1024 * 1024 * 1024); - options.compaction_style = kCompactionStyleUniversal; - options.write_buffer_size = 100 << 10; // 100KB - options.level0_file_num_compaction_trigger = 2; - options.num_levels = 1; - options = CurrentOptions(options); +#ifndef ROCKSDB_LITE + virtual Status DisableFileDeletions() override { return Status::OK(); } - std::vector filenames; - env_->GetChildren(options.db_paths[1].path, &filenames); - // Delete archival files. - for (size_t i = 0; i < filenames.size(); ++i) { - env_->DeleteFile(options.db_paths[1].path + "/" + filenames[i]); + virtual Status EnableFileDeletions(bool force) override { + return Status::OK(); } - env_->DeleteDir(options.db_paths[1].path); - Reopen(&options); - - Random rnd(301); - int key_idx = 0; - - // First three 110KB files are not going to second path. - // After that, (100K, 200K) - for (int num = 0; num < 3; num++) { - GenerateNewFile(&rnd, &key_idx); + virtual Status GetLiveFiles(std::vector&, uint64_t* size, + bool flush_memtable = true) override { + return Status::OK(); } - // Another 110KB triggers a compaction to 400K file to second path - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); + virtual Status GetSortedWalFiles(VectorLogPtr& files) override { + return Status::OK(); + } - // (1, 4) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); - ASSERT_EQ(1, GetSstFileCount(dbname_)); + virtual Status DeleteFile(std::string name) override { return Status::OK(); } - // (1,1,4) -> (2, 4) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(0, GetSstFileCount(dbname_)); + virtual Status GetUpdatesSince( + rocksdb::SequenceNumber, unique_ptr*, + const TransactionLogIterator::ReadOptions& read_options = + TransactionLogIterator::ReadOptions()) override { + return Status::NotSupported("Not supported in Model DB"); + } - // (1, 2, 4) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(1, GetSstFileCount(dbname_)); + virtual void GetColumnFamilyMetaData( + ColumnFamilyHandle* column_family, + ColumnFamilyMetaData* metadata) override {} +#endif // ROCKSDB_LITE - // (1, 1, 2, 4) -> (8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); + virtual Status GetDbIdentity(std::string& identity) const override { + return Status::OK(); + } - // (1, 8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); - ASSERT_EQ(1, GetSstFileCount(dbname_)); + virtual SequenceNumber GetLatestSequenceNumber() const override { return 0; } - // (1, 1, 8) -> (2, 8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + virtual ColumnFamilyHandle* DefaultColumnFamily() const override { + return nullptr; + } - // (1, 2, 8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - ASSERT_EQ(1, GetSstFileCount(dbname_)); + private: + class ModelIter : public Iterator { + public: + ModelIter(const KVMap* map, bool owned) + : map_(map), owned_(owned), iter_(map_->end()) {} + ~ModelIter() { + if (owned_) delete map_; + } + virtual bool Valid() const override { return iter_ != map_->end(); } + virtual void SeekToFirst() override { iter_ = map_->begin(); } + virtual void SeekToLast() override { + if (map_->empty()) { + iter_ = map_->end(); + } else { + iter_ = map_->find(map_->rbegin()->first); + } + } + virtual void Seek(const Slice& k) override { + iter_ = map_->lower_bound(k.ToString()); + } + virtual void SeekForPrev(const Slice& k) override { + iter_ = map_->upper_bound(k.ToString()); + Prev(); + } + virtual void Next() override { ++iter_; } + virtual void Prev() override { + if (iter_ == map_->begin()) { + iter_ = map_->end(); + return; + } + --iter_; + } - // (1, 1, 2, 8) -> (4, 8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); + virtual Slice key() const override { return iter_->first; } + virtual Slice value() const override { return iter_->second; } + virtual Status status() const override { return Status::OK(); } - // (1, 4, 8) - GenerateNewFile(&rnd, &key_idx); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); - ASSERT_EQ(1, GetSstFileCount(dbname_)); + private: + const KVMap* const map_; + const bool owned_; // Do we own map_ + KVMap::const_iterator iter_; + }; + const Options options_; + KVMap map_; + std::string name_ = ""; +}; - for (int i = 0; i < key_idx; i++) { - auto v = Get(Key(i)); - ASSERT_NE(v, "NOT_FOUND"); - ASSERT_TRUE(v.size() == 1 || v.size() == 10000); - } +static std::string RandomKey(Random* rnd, int minimum = 0) { + int len; + do { + len = (rnd->OneIn(3) + ? 1 // Short sometimes to encourage collisions + : (rnd->OneIn(100) ? rnd->Skewed(10) : rnd->Uniform(10))); + } while (len < minimum); + return test::RandomKey(rnd, len); +} - Reopen(&options); +static bool CompareIterators(int step, DB* model, DB* db, + const Snapshot* model_snap, + const Snapshot* db_snap) { + ReadOptions options; + options.snapshot = model_snap; + Iterator* miter = model->NewIterator(options); + options.snapshot = db_snap; + Iterator* dbiter = db->NewIterator(options); + bool ok = true; + int count = 0; + for (miter->SeekToFirst(), dbiter->SeekToFirst(); + ok && miter->Valid() && dbiter->Valid(); miter->Next(), dbiter->Next()) { + count++; + if (miter->key().compare(dbiter->key()) != 0) { + fprintf(stderr, "step %d: Key mismatch: '%s' vs. '%s'\n", step, + EscapeString(miter->key()).c_str(), + EscapeString(dbiter->key()).c_str()); + ok = false; + break; + } - for (int i = 0; i < key_idx; i++) { - auto v = Get(Key(i)); - ASSERT_NE(v, "NOT_FOUND"); - ASSERT_TRUE(v.size() == 1 || v.size() == 10000); + if (miter->value().compare(dbiter->value()) != 0) { + fprintf(stderr, "step %d: Value mismatch for key '%s': '%s' vs. '%s'\n", + step, EscapeString(miter->key()).c_str(), + EscapeString(miter->value()).c_str(), + EscapeString(miter->value()).c_str()); + ok = false; + } } - Destroy(&options); + if (ok) { + if (miter->Valid() != dbiter->Valid()) { + fprintf(stderr, "step %d: Mismatch at end of iterators: %d vs. %d\n", + step, miter->Valid(), dbiter->Valid()); + ok = false; + } + } + delete miter; + delete dbiter; + return ok; } -#endif -TEST(DBTest, ConvertCompactionStyle) { - Random rnd(301); - int max_key_level_insert = 200; - int max_key_universal_insert = 600; - - // Stage 1: generate a db with level compaction - Options options; - options.write_buffer_size = 100<<10; //100KB - options.num_levels = 4; - options.level0_file_num_compaction_trigger = 3; - options.max_bytes_for_level_base = 500<<10; // 500KB - options.max_bytes_for_level_multiplier = 1; - options.target_file_size_base = 200<<10; // 200KB - options.target_file_size_multiplier = 1; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); +class DBTestRandomized : public DBTest, + public ::testing::WithParamInterface { + public: + virtual void SetUp() override { option_config_ = GetParam(); } - for (int i = 0; i <= max_key_level_insert; i++) { - // each value is 10K - ASSERT_OK(Put(1, Key(i), RandomString(&rnd, 10000))); + static std::vector GenerateOptionConfigs() { + std::vector option_configs; + // skip cuckoo hash as it does not support snapshot. + for (int option_config = kDefault; option_config < kEnd; ++option_config) { + if (!ShouldSkipOptions(option_config, kSkipDeletesFilterFirst | + kSkipNoSeekToLast | + kSkipHashCuckoo)) { + option_configs.push_back(option_config); + } + } + option_configs.push_back(kBlockBasedTableWithIndexRestartInterval); + return option_configs; } - ASSERT_OK(Flush(1)); - dbfull()->TEST_WaitForCompact(); +}; - ASSERT_GT(TotalTableFiles(1, 4), 1); - int non_level0_num_files = 0; - for (int i = 1; i < options.num_levels; i++) { - non_level0_num_files += NumTableFilesAtLevel(i, 1); - } - ASSERT_GT(non_level0_num_files, 0); +INSTANTIATE_TEST_CASE_P( + DBTestRandomized, DBTestRandomized, + ::testing::ValuesIn(DBTestRandomized::GenerateOptionConfigs())); + +TEST_P(DBTestRandomized, Randomized) { + anon::OptionsOverride options_override; + options_override.skip_policy = kSkipNoSnapshot; + Options options = CurrentOptions(options_override); + DestroyAndReopen(options); + + Random rnd(test::RandomSeed() + GetParam()); + ModelDB model(options); + const int N = 10000; + const Snapshot* model_snap = nullptr; + const Snapshot* db_snap = nullptr; + std::string k, v; + for (int step = 0; step < N; step++) { + // TODO(sanjay): Test Get() works + int p = rnd.Uniform(100); + int minimum = 0; + if (option_config_ == kHashSkipList || option_config_ == kHashLinkList || + option_config_ == kHashCuckoo || + option_config_ == kPlainTableFirstBytePrefix || + option_config_ == kBlockBasedTableWithWholeKeyHashIndex || + option_config_ == kBlockBasedTableWithPrefixHashIndex) { + minimum = 1; + } + if (p < 45) { // Put + k = RandomKey(&rnd, minimum); + v = RandomString(&rnd, + rnd.OneIn(20) ? 100 + rnd.Uniform(100) : rnd.Uniform(8)); + ASSERT_OK(model.Put(WriteOptions(), k, v)); + ASSERT_OK(db_->Put(WriteOptions(), k, v)); + } else if (p < 90) { // Delete + k = RandomKey(&rnd, minimum); + ASSERT_OK(model.Delete(WriteOptions(), k)); + ASSERT_OK(db_->Delete(WriteOptions(), k)); + } else { // Multi-element batch + WriteBatch b; + const int num = rnd.Uniform(8); + for (int i = 0; i < num; i++) { + if (i == 0 || !rnd.OneIn(10)) { + k = RandomKey(&rnd, minimum); + } else { + // Periodically re-use the same key from the previous iter, so + // we have multiple entries in the write batch for the same key + } + if (rnd.OneIn(2)) { + v = RandomString(&rnd, rnd.Uniform(10)); + b.Put(k, v); + } else { + b.Delete(k); + } + } + ASSERT_OK(model.Write(WriteOptions(), &b)); + ASSERT_OK(db_->Write(WriteOptions(), &b)); + } - // Stage 2: reopen with universal compaction - should fail - options = CurrentOptions(); - options.compaction_style = kCompactionStyleUniversal; - options = CurrentOptions(options); - Status s = TryReopenWithColumnFamilies({"default", "pikachu"}, &options); - ASSERT_TRUE(s.IsInvalidArgument()); + if ((step % 100) == 0) { + // For DB instances that use the hash index + block-based table, the + // iterator will be invalid right when seeking a non-existent key, right + // than return a key that is close to it. + if (option_config_ != kBlockBasedTableWithWholeKeyHashIndex && + option_config_ != kBlockBasedTableWithPrefixHashIndex) { + ASSERT_TRUE(CompareIterators(step, &model, db_, nullptr, nullptr)); + ASSERT_TRUE(CompareIterators(step, &model, db_, model_snap, db_snap)); + } - // Stage 3: compact into a single file and move the file to level 0 - options = CurrentOptions(); - options.disable_auto_compactions = true; - options.target_file_size_base = INT_MAX; - options.target_file_size_multiplier = 1; - options.max_bytes_for_level_base = INT_MAX; - options.max_bytes_for_level_multiplier = 1; - options = CurrentOptions(options); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); + // Save a snapshot from each DB this time that we'll use next + // time we compare things, to make sure the current state is + // preserved with the snapshot + if (model_snap != nullptr) model.ReleaseSnapshot(model_snap); + if (db_snap != nullptr) db_->ReleaseSnapshot(db_snap); - dbfull()->CompactRange(handles_[1], nullptr, nullptr, true /* reduce level */, - 0 /* reduce to level 0 */); + Reopen(options); + ASSERT_TRUE(CompareIterators(step, &model, db_, nullptr, nullptr)); - for (int i = 0; i < options.num_levels; i++) { - int num = NumTableFilesAtLevel(i, 1); - if (i == 0) { - ASSERT_EQ(num, 1); - } else { - ASSERT_EQ(num, 0); + model_snap = model.GetSnapshot(); + db_snap = db_->GetSnapshot(); } } + if (model_snap != nullptr) model.ReleaseSnapshot(model_snap); + if (db_snap != nullptr) db_->ReleaseSnapshot(db_snap); +} - // Stage 4: re-open in universal compaction style and do some db operations - options = CurrentOptions(); - options.compaction_style = kCompactionStyleUniversal; - options.write_buffer_size = 100<<10; //100KB - options.level0_file_num_compaction_trigger = 3; - options = CurrentOptions(options); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - - for (int i = max_key_level_insert / 2; i <= max_key_universal_insert; i++) { - ASSERT_OK(Put(1, Key(i), RandomString(&rnd, 10000))); - } - dbfull()->Flush(FlushOptions()); - ASSERT_OK(Flush(1)); - dbfull()->TEST_WaitForCompact(); - - for (int i = 1; i < options.num_levels; i++) { - ASSERT_EQ(NumTableFilesAtLevel(i, 1), 0); - } - - // verify keys inserted in both level compaction style and universal - // compaction style - std::string keys_in_db; - Iterator* iter = dbfull()->NewIterator(ReadOptions(), handles_[1]); - for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { - keys_in_db.append(iter->key().ToString()); - keys_in_db.push_back(','); - } - delete iter; - - std::string expected_keys; - for (int i = 0; i <= max_key_universal_insert; i++) { - expected_keys.append(Key(i)); - expected_keys.push_back(','); - } - - ASSERT_EQ(keys_in_db, expected_keys); -} - -namespace { -void MinLevelHelper(DBTest* self, Options& options) { - Random rnd(301); - - for (int num = 0; - num < options.level0_file_num_compaction_trigger - 1; - num++) - { - std::vector values; - // Write 120KB (12 values, each 10K) - for (int i = 0; i < 12; i++) { - values.push_back(RandomString(&rnd, 10000)); - ASSERT_OK(self->Put(Key(i), values[i])); - } - self->dbfull()->TEST_WaitForFlushMemTable(); - ASSERT_EQ(self->NumTableFilesAtLevel(0), num + 1); - } - - //generate one more file in level-0, and should trigger level-0 compaction - std::vector values; - for (int i = 0; i < 12; i++) { - values.push_back(RandomString(&rnd, 10000)); - ASSERT_OK(self->Put(Key(i), values[i])); - } - self->dbfull()->TEST_WaitForCompact(); - - ASSERT_EQ(self->NumTableFilesAtLevel(0), 0); - ASSERT_EQ(self->NumTableFilesAtLevel(1), 1); -} - -// returns false if the calling-Test should be skipped -bool MinLevelToCompress(CompressionType& type, Options& options, int wbits, - int lev, int strategy) { - fprintf(stderr, "Test with compression options : window_bits = %d, level = %d, strategy = %d}\n", wbits, lev, strategy); - options.write_buffer_size = 100<<10; //100KB - options.num_levels = 3; - options.max_mem_compaction_level = 0; - options.level0_file_num_compaction_trigger = 3; - options.create_if_missing = true; - - if (SnappyCompressionSupported(CompressionOptions(wbits, lev, strategy))) { - type = kSnappyCompression; - fprintf(stderr, "using snappy\n"); - } else if (ZlibCompressionSupported( - CompressionOptions(wbits, lev, strategy))) { - type = kZlibCompression; - fprintf(stderr, "using zlib\n"); - } else if (BZip2CompressionSupported( - CompressionOptions(wbits, lev, strategy))) { - type = kBZip2Compression; - fprintf(stderr, "using bzip2\n"); - } else if (LZ4CompressionSupported( - CompressionOptions(wbits, lev, strategy))) { - type = kLZ4Compression; - fprintf(stderr, "using lz4\n"); - } else if (LZ4HCCompressionSupported( - CompressionOptions(wbits, lev, strategy))) { - type = kLZ4HCCompression; - fprintf(stderr, "using lz4hc\n"); - } else { - fprintf(stderr, "skipping test, compression disabled\n"); - return false; - } - options.compression_per_level.resize(options.num_levels); - - // do not compress L0 - for (int i = 0; i < 1; i++) { - options.compression_per_level[i] = kNoCompression; - } - for (int i = 1; i < options.num_levels; i++) { - options.compression_per_level[i] = type; - } - return true; -} -} // namespace - -TEST(DBTest, MinLevelToCompress1) { - Options options = CurrentOptions(); - CompressionType type; - if (!MinLevelToCompress(type, options, -14, -1, 0)) { - return; - } - Reopen(&options); - MinLevelHelper(this, options); - - // do not compress L0 and L1 - for (int i = 0; i < 2; i++) { - options.compression_per_level[i] = kNoCompression; - } - for (int i = 2; i < options.num_levels; i++) { - options.compression_per_level[i] = type; - } - DestroyAndReopen(&options); - MinLevelHelper(this, options); -} - -TEST(DBTest, MinLevelToCompress2) { +TEST_F(DBTest, BlockBasedTablePrefixIndexTest) { + // create a DB with block prefix index + BlockBasedTableOptions table_options; Options options = CurrentOptions(); - CompressionType type; - if (!MinLevelToCompress(type, options, 15, -1, 0)) { - return; - } - Reopen(&options); - MinLevelHelper(this, options); - - // do not compress L0 and L1 - for (int i = 0; i < 2; i++) { - options.compression_per_level[i] = kNoCompression; - } - for (int i = 2; i < options.num_levels; i++) { - options.compression_per_level[i] = type; - } - DestroyAndReopen(&options); - MinLevelHelper(this, options); -} - -TEST(DBTest, RepeatedWritesToSameKey) { - do { - Options options; - options.env = env_; - options.write_buffer_size = 100000; // Small write buffer - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - // We must have at most one file per level except for level-0, - // which may have up to kL0_StopWritesTrigger files. - const int kMaxFiles = - options.num_levels + options.level0_stop_writes_trigger; - - Random rnd(301); - std::string value = RandomString(&rnd, 2 * options.write_buffer_size); - for (int i = 0; i < 5 * kMaxFiles; i++) { - ASSERT_OK(Put(1, "key", value)); - ASSERT_LE(TotalTableFiles(1), kMaxFiles); - } - } while (ChangeCompactOptions()); -} - -TEST(DBTest, InPlaceUpdate) { - do { - Options options; - options.create_if_missing = true; - options.inplace_update_support = true; - options.env = env_; - options.write_buffer_size = 100000; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - // Update key with values of smaller size - int numValues = 10; - for (int i = numValues; i > 0; i--) { - std::string value = DummyString(i, 'a'); - ASSERT_OK(Put(1, "key", value)); - ASSERT_EQ(value, Get(1, "key")); - } - - // Only 1 instance for that key. - validateNumberOfEntries(1, 1); - - } while (ChangeCompactOptions()); -} - -TEST(DBTest, InPlaceUpdateLargeNewValue) { - do { - Options options; - options.create_if_missing = true; - options.inplace_update_support = true; - options.env = env_; - options.write_buffer_size = 100000; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - // Update key with values of larger size - int numValues = 10; - for (int i = 0; i < numValues; i++) { - std::string value = DummyString(i, 'a'); - ASSERT_OK(Put(1, "key", value)); - ASSERT_EQ(value, Get(1, "key")); - } - - // All 10 updates exist in the internal iterator - validateNumberOfEntries(numValues, 1); - - } while (ChangeCompactOptions()); -} - - -TEST(DBTest, InPlaceUpdateCallbackSmallerSize) { - do { - Options options; - options.create_if_missing = true; - options.inplace_update_support = true; - - options.env = env_; - options.write_buffer_size = 100000; - options.inplace_callback = - rocksdb::DBTest::updateInPlaceSmallerSize; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - // Update key with values of smaller size - int numValues = 10; - ASSERT_OK(Put(1, "key", DummyString(numValues, 'a'))); - ASSERT_EQ(DummyString(numValues, 'c'), Get(1, "key")); - - for (int i = numValues; i > 0; i--) { - ASSERT_OK(Put(1, "key", DummyString(i, 'a'))); - ASSERT_EQ(DummyString(i - 1, 'b'), Get(1, "key")); - } - - // Only 1 instance for that key. - validateNumberOfEntries(1, 1); - - } while (ChangeCompactOptions()); -} - -TEST(DBTest, InPlaceUpdateCallbackSmallerVarintSize) { - do { - Options options; - options.create_if_missing = true; - options.inplace_update_support = true; - - options.env = env_; - options.write_buffer_size = 100000; - options.inplace_callback = - rocksdb::DBTest::updateInPlaceSmallerVarintSize; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - // Update key with values of smaller varint size - int numValues = 265; - ASSERT_OK(Put(1, "key", DummyString(numValues, 'a'))); - ASSERT_EQ(DummyString(numValues, 'c'), Get(1, "key")); + table_options.index_type = BlockBasedTableOptions::kHashSearch; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.prefix_extractor.reset(NewFixedPrefixTransform(1)); - for (int i = numValues; i > 0; i--) { - ASSERT_OK(Put(1, "key", DummyString(i, 'a'))); - ASSERT_EQ(DummyString(1, 'b'), Get(1, "key")); - } + Reopen(options); + ASSERT_OK(Put("k1", "v1")); + Flush(); + ASSERT_OK(Put("k2", "v2")); - // Only 1 instance for that key. - validateNumberOfEntries(1, 1); + // Reopen it without prefix extractor, make sure everything still works. + // RocksDB should just fall back to the binary index. + table_options.index_type = BlockBasedTableOptions::kBinarySearch; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.prefix_extractor.reset(); - } while (ChangeCompactOptions()); + Reopen(options); + ASSERT_EQ("v1", Get("k1")); + ASSERT_EQ("v2", Get("k2")); } -TEST(DBTest, InPlaceUpdateCallbackLargeNewValue) { - do { - Options options; - options.create_if_missing = true; - options.inplace_update_support = true; +TEST_F(DBTest, ChecksumTest) { + BlockBasedTableOptions table_options; + Options options = CurrentOptions(); - options.env = env_; - options.write_buffer_size = 100000; - options.inplace_callback = - rocksdb::DBTest::updateInPlaceLargerSize; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); + table_options.checksum = kCRC32c; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + Reopen(options); + ASSERT_OK(Put("a", "b")); + ASSERT_OK(Put("c", "d")); + ASSERT_OK(Flush()); // table with crc checksum - // Update key with values of larger size - int numValues = 10; - for (int i = 0; i < numValues; i++) { - ASSERT_OK(Put(1, "key", DummyString(i, 'a'))); - ASSERT_EQ(DummyString(i, 'c'), Get(1, "key")); - } + table_options.checksum = kxxHash; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + Reopen(options); + ASSERT_OK(Put("e", "f")); + ASSERT_OK(Put("g", "h")); + ASSERT_OK(Flush()); // table with xxhash checksum - // No inplace updates. All updates are puts with new seq number - // All 10 updates exist in the internal iterator - validateNumberOfEntries(numValues, 1); + table_options.checksum = kCRC32c; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + Reopen(options); + ASSERT_EQ("b", Get("a")); + ASSERT_EQ("d", Get("c")); + ASSERT_EQ("f", Get("e")); + ASSERT_EQ("h", Get("g")); - } while (ChangeCompactOptions()); + table_options.checksum = kCRC32c; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + Reopen(options); + ASSERT_EQ("b", Get("a")); + ASSERT_EQ("d", Get("c")); + ASSERT_EQ("f", Get("e")); + ASSERT_EQ("h", Get("g")); } -TEST(DBTest, InPlaceUpdateCallbackNoAction) { - do { +#ifndef ROCKSDB_LITE +TEST_P(DBTestWithParam, FIFOCompactionTest) { + for (int iter = 0; iter < 2; ++iter) { + // first iteration -- auto compaction + // second iteration -- manual compaction Options options; - options.create_if_missing = true; - options.inplace_update_support = true; - - options.env = env_; - options.write_buffer_size = 100000; - options.inplace_callback = - rocksdb::DBTest::updateInPlaceNoAction; - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - // Callback function requests no actions from db - ASSERT_OK(Put(1, "key", DummyString(1, 'a'))); - ASSERT_EQ(Get(1, "key"), "NOT_FOUND"); - - } while (ChangeCompactOptions()); -} - -TEST(DBTest, CompactionFilter) { - Options options = CurrentOptions(); - options.max_open_files = -1; - options.num_levels = 3; - options.max_mem_compaction_level = 0; - options.compaction_filter_factory = std::make_shared(); - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - // Write 100K keys, these are written to a few files in L0. - const std::string value(10, 'x'); - for (int i = 0; i < 100000; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%010d", i); - Put(1, key, value); - } - ASSERT_OK(Flush(1)); - - // Push all files to the highest level L2. Verify that - // the compaction is each level invokes the filter for - // all the keys in that level. - cfilter_count = 0; - dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); - ASSERT_EQ(cfilter_count, 100000); - cfilter_count = 0; - dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); - ASSERT_EQ(cfilter_count, 100000); - - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - ASSERT_EQ(NumTableFilesAtLevel(1, 1), 0); - ASSERT_NE(NumTableFilesAtLevel(2, 1), 0); - cfilter_count = 0; - - // All the files are in the lowest level. - // Verify that all but the 100001st record - // has sequence number zero. The 100001st record - // is at the tip of this snapshot and cannot - // be zeroed out. - // TODO: figure out sequence number squashtoo - int count = 0; - int total = 0; - Iterator* iter = dbfull()->TEST_NewInternalIterator(handles_[1]); - iter->SeekToFirst(); - ASSERT_OK(iter->status()); - while (iter->Valid()) { - ParsedInternalKey ikey(Slice(), 0, kTypeValue); - ikey.sequence = -1; - ASSERT_EQ(ParseInternalKey(iter->key(), &ikey), true); - total++; - if (ikey.sequence != 0) { - count++; - } - iter->Next(); - } - ASSERT_EQ(total, 100000); - ASSERT_EQ(count, 1); - delete iter; - - // overwrite all the 100K keys once again. - for (int i = 0; i < 100000; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%010d", i); - ASSERT_OK(Put(1, key, value)); - } - ASSERT_OK(Flush(1)); - - // push all files to the highest level L2. This - // means that all keys should pass at least once - // via the compaction filter - cfilter_count = 0; - dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); - ASSERT_EQ(cfilter_count, 100000); - cfilter_count = 0; - dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); - ASSERT_EQ(cfilter_count, 100000); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - ASSERT_EQ(NumTableFilesAtLevel(1, 1), 0); - ASSERT_NE(NumTableFilesAtLevel(2, 1), 0); - - // create a new database with the compaction - // filter in such a way that it deletes all keys - options.compaction_filter_factory = std::make_shared(); - options.create_if_missing = true; - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); - - // write all the keys once again. - for (int i = 0; i < 100000; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%010d", i); - ASSERT_OK(Put(1, key, value)); - } - ASSERT_OK(Flush(1)); - ASSERT_NE(NumTableFilesAtLevel(0, 1), 0); - ASSERT_EQ(NumTableFilesAtLevel(1, 1), 0); - ASSERT_EQ(NumTableFilesAtLevel(2, 1), 0); - - // Push all files to the highest level L2. This - // triggers the compaction filter to delete all keys, - // verify that at the end of the compaction process, - // nothing is left. - cfilter_count = 0; - dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); - ASSERT_EQ(cfilter_count, 100000); - cfilter_count = 0; - dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); - ASSERT_EQ(cfilter_count, 0); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - ASSERT_EQ(NumTableFilesAtLevel(1, 1), 0); - - // Scan the entire database to ensure that nothing is left - iter = db_->NewIterator(ReadOptions(), handles_[1]); - iter->SeekToFirst(); - count = 0; - while (iter->Valid()) { - count++; - iter->Next(); - } - ASSERT_EQ(count, 0); - delete iter; - - // The sequence number of the remaining record - // is not zeroed out even though it is at the - // level Lmax because this record is at the tip - // TODO: remove the following or design a different - // test - count = 0; - iter = dbfull()->TEST_NewInternalIterator(handles_[1]); - iter->SeekToFirst(); - ASSERT_OK(iter->status()); - while (iter->Valid()) { - ParsedInternalKey ikey(Slice(), 0, kTypeValue); - ASSERT_EQ(ParseInternalKey(iter->key(), &ikey), true); - ASSERT_NE(ikey.sequence, (unsigned)0); - count++; - iter->Next(); - } - ASSERT_EQ(count, 0); - delete iter; -} - -// Tests the edge case where compaction does not produce any output -- all -// entries are deleted. The compaction should create bunch of 'DeleteFile' -// entries in VersionEdit, but none of the 'AddFile's. -TEST(DBTest, CompactionFilterDeletesAll) { - Options options; - options.compaction_filter_factory = std::make_shared(); - options.disable_auto_compactions = true; - options.create_if_missing = true; - DestroyAndReopen(&options); - - // put some data - for (int table = 0; table < 4; ++table) { - for (int i = 0; i < 10 + table; ++i) { - Put(std::to_string(table * 100 + i), "val"); - } - Flush(); - } - - // this will produce empty file (delete compaction filter) - ASSERT_OK(db_->CompactRange(nullptr, nullptr)); - ASSERT_EQ(0, CountLiveFiles()); - - Reopen(&options); - - Iterator* itr = db_->NewIterator(ReadOptions()); - itr->SeekToFirst(); - // empty db - ASSERT_TRUE(!itr->Valid()); - - delete itr; -} - -TEST(DBTest, CompactionFilterWithValueChange) { - do { - Options options; - options.num_levels = 3; - options.max_mem_compaction_level = 0; - options.compaction_filter_factory = - std::make_shared(); - options = CurrentOptions(options); - CreateAndReopenWithCF({"pikachu"}, &options); - - // Write 100K+1 keys, these are written to a few files - // in L0. We do this so that the current snapshot points - // to the 100001 key.The compaction filter is not invoked - // on keys that are visible via a snapshot because we - // anyways cannot delete it. - const std::string value(10, 'x'); - for (int i = 0; i < 100001; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%010d", i); - Put(1, key, value); - } - - // push all files to lower levels - ASSERT_OK(Flush(1)); - dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); - dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); - - // re-write all data again - for (int i = 0; i < 100001; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%010d", i); - Put(1, key, value); - } - - // push all files to lower levels. This should - // invoke the compaction filter for all 100000 keys. - ASSERT_OK(Flush(1)); - dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); - dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); - - // verify that all keys now have the new value that - // was set by the compaction process. - for (int i = 0; i < 100001; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%010d", i); - std::string newvalue = Get(1, key); - ASSERT_EQ(newvalue.compare(NEW_VALUE), 0); - } - } while (ChangeCompactOptions()); -} - -TEST(DBTest, CompactionFilterContextManual) { - KeepFilterFactory* filter = new KeepFilterFactory(); - - Options options = CurrentOptions(); - options.compaction_style = kCompactionStyleUniversal; - options.compaction_filter_factory.reset(filter); - options.compression = kNoCompression; - options.level0_file_num_compaction_trigger = 8; - Reopen(&options); - int num_keys_per_file = 400; - for (int j = 0; j < 3; j++) { - // Write several keys. - const std::string value(10, 'x'); - for (int i = 0; i < num_keys_per_file; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%08d%02d", i, j); - Put(key, value); - } - dbfull()->TEST_FlushMemTable(); - // Make sure next file is much smaller so automatic compaction will not - // be triggered. - num_keys_per_file /= 2; - } - - // Force a manual compaction - cfilter_count = 0; - filter->expect_manual_compaction_.store(true); - filter->expect_full_compaction_.store(false); // Manual compaction always - // set this flag. - dbfull()->CompactRange(nullptr, nullptr); - ASSERT_EQ(cfilter_count, 700); - ASSERT_EQ(NumTableFilesAtLevel(0), 1); - - // Verify total number of keys is correct after manual compaction. - int count = 0; - int total = 0; - Iterator* iter = dbfull()->TEST_NewInternalIterator(); - iter->SeekToFirst(); - ASSERT_OK(iter->status()); - while (iter->Valid()) { - ParsedInternalKey ikey(Slice(), 0, kTypeValue); - ikey.sequence = -1; - ASSERT_EQ(ParseInternalKey(iter->key(), &ikey), true); - total++; - if (ikey.sequence != 0) { - count++; - } - iter->Next(); - } - ASSERT_EQ(total, 700); - ASSERT_EQ(count, 1); - delete iter; -} - -class KeepFilterV2 : public CompactionFilterV2 { - public: - virtual std::vector Filter(int level, - const SliceVector& keys, - const SliceVector& existing_values, - std::vector* new_values, - std::vector* values_changed) - const override { - cfilter_count++; - std::vector ret; - new_values->clear(); - values_changed->clear(); - for (unsigned int i = 0; i < keys.size(); ++i) { - values_changed->push_back(false); - ret.push_back(false); - } - return ret; - } - - virtual const char* Name() const override { - return "KeepFilterV2"; - } -}; - -class DeleteFilterV2 : public CompactionFilterV2 { - public: - virtual std::vector Filter(int level, - const SliceVector& keys, - const SliceVector& existing_values, - std::vector* new_values, - std::vector* values_changed) - const override { - cfilter_count++; - new_values->clear(); - values_changed->clear(); - std::vector ret; - for (unsigned int i = 0; i < keys.size(); ++i) { - values_changed->push_back(false); - ret.push_back(true); - } - return ret; - } - - virtual const char* Name() const override { - return "DeleteFilterV2"; - } -}; - -class ChangeFilterV2 : public CompactionFilterV2 { - public: - virtual std::vector Filter(int level, - const SliceVector& keys, - const SliceVector& existing_values, - std::vector* new_values, - std::vector* values_changed) - const override { - std::vector ret; - new_values->clear(); - values_changed->clear(); - for (unsigned int i = 0; i < keys.size(); ++i) { - values_changed->push_back(true); - new_values->push_back(NEW_VALUE); - ret.push_back(false); - } - return ret; - } - - virtual const char* Name() const override { - return "ChangeFilterV2"; - } -}; - -class KeepFilterFactoryV2 : public CompactionFilterFactoryV2 { - public: - explicit KeepFilterFactoryV2(const SliceTransform* prefix_extractor) - : CompactionFilterFactoryV2(prefix_extractor) { } - - virtual std::unique_ptr - CreateCompactionFilterV2( - const CompactionFilterContext& context) override { - return std::unique_ptr(new KeepFilterV2()); - } - - virtual const char* Name() const override { - return "KeepFilterFactoryV2"; - } -}; - -class DeleteFilterFactoryV2 : public CompactionFilterFactoryV2 { - public: - explicit DeleteFilterFactoryV2(const SliceTransform* prefix_extractor) - : CompactionFilterFactoryV2(prefix_extractor) { } - - virtual std::unique_ptr - CreateCompactionFilterV2( - const CompactionFilterContext& context) override { - return std::unique_ptr(new DeleteFilterV2()); - } - - virtual const char* Name() const override { - return "DeleteFilterFactoryV2"; - } -}; - -class ChangeFilterFactoryV2 : public CompactionFilterFactoryV2 { - public: - explicit ChangeFilterFactoryV2(const SliceTransform* prefix_extractor) - : CompactionFilterFactoryV2(prefix_extractor) { } - - virtual std::unique_ptr - CreateCompactionFilterV2( - const CompactionFilterContext& context) override { - return std::unique_ptr(new ChangeFilterV2()); - } - - virtual const char* Name() const override { - return "ChangeFilterFactoryV2"; - } -}; - -TEST(DBTest, CompactionFilterV2) { - Options options = CurrentOptions(); - options.num_levels = 3; - options.max_mem_compaction_level = 0; - // extract prefix - std::unique_ptr prefix_extractor; - prefix_extractor.reset(NewFixedPrefixTransform(8)); - - options.compaction_filter_factory_v2 - = std::make_shared(prefix_extractor.get()); - // In a testing environment, we can only flush the application - // compaction filter buffer using universal compaction - option_config_ = kUniversalCompaction; - options.compaction_style = (rocksdb::CompactionStyle)1; - Reopen(&options); - - // Write 100K keys, these are written to a few files in L0. - const std::string value(10, 'x'); - for (int i = 0; i < 100000; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%08d%010d", i , i); - Put(key, value); - } - - dbfull()->TEST_FlushMemTable(); - - dbfull()->TEST_CompactRange(0, nullptr, nullptr); - dbfull()->TEST_CompactRange(1, nullptr, nullptr); - - ASSERT_EQ(NumTableFilesAtLevel(0), 1); - - // All the files are in the lowest level. - int count = 0; - int total = 0; - Iterator* iter = dbfull()->TEST_NewInternalIterator(); - iter->SeekToFirst(); - ASSERT_OK(iter->status()); - while (iter->Valid()) { - ParsedInternalKey ikey(Slice(), 0, kTypeValue); - ikey.sequence = -1; - ASSERT_EQ(ParseInternalKey(iter->key(), &ikey), true); - total++; - if (ikey.sequence != 0) { - count++; - } - iter->Next(); - } - - ASSERT_EQ(total, 100000); - // 1 snapshot only. Since we are using universal compacton, - // the sequence no is cleared for better compression - ASSERT_EQ(count, 1); - delete iter; - - // create a new database with the compaction - // filter in such a way that it deletes all keys - options.compaction_filter_factory_v2 = - std::make_shared(prefix_extractor.get()); - options.create_if_missing = true; - DestroyAndReopen(&options); - - // write all the keys once again. - for (int i = 0; i < 100000; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%08d%010d", i, i); - Put(key, value); - } - - dbfull()->TEST_FlushMemTable(); - ASSERT_NE(NumTableFilesAtLevel(0), 0); - - dbfull()->TEST_CompactRange(0, nullptr, nullptr); - dbfull()->TEST_CompactRange(1, nullptr, nullptr); - ASSERT_EQ(NumTableFilesAtLevel(1), 0); - - // Scan the entire database to ensure that nothing is left - iter = db_->NewIterator(ReadOptions()); - iter->SeekToFirst(); - count = 0; - while (iter->Valid()) { - count++; - iter->Next(); - } - - ASSERT_EQ(count, 0); - delete iter; -} - -TEST(DBTest, CompactionFilterV2WithValueChange) { - Options options = CurrentOptions(); - options.num_levels = 3; - options.max_mem_compaction_level = 0; - std::unique_ptr prefix_extractor; - prefix_extractor.reset(NewFixedPrefixTransform(8)); - options.compaction_filter_factory_v2 = - std::make_shared(prefix_extractor.get()); - // In a testing environment, we can only flush the application - // compaction filter buffer using universal compaction - option_config_ = kUniversalCompaction; - options.compaction_style = (rocksdb::CompactionStyle)1; - options = CurrentOptions(options); - Reopen(&options); - - // Write 100K+1 keys, these are written to a few files - // in L0. We do this so that the current snapshot points - // to the 100001 key.The compaction filter is not invoked - // on keys that are visible via a snapshot because we - // anyways cannot delete it. - const std::string value(10, 'x'); - for (int i = 0; i < 100001; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%08d%010d", i, i); - Put(key, value); - } - - // push all files to lower levels - dbfull()->TEST_FlushMemTable(); - dbfull()->TEST_CompactRange(0, nullptr, nullptr); - dbfull()->TEST_CompactRange(1, nullptr, nullptr); - - // verify that all keys now have the new value that - // was set by the compaction process. - for (int i = 0; i < 100001; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%08d%010d", i, i); - std::string newvalue = Get(key); - ASSERT_EQ(newvalue.compare(NEW_VALUE), 0); - } -} - -TEST(DBTest, CompactionFilterV2NULLPrefix) { - Options options = CurrentOptions(); - options.num_levels = 3; - options.max_mem_compaction_level = 0; - std::unique_ptr prefix_extractor; - prefix_extractor.reset(NewFixedPrefixTransform(8)); - options.compaction_filter_factory_v2 = - std::make_shared(prefix_extractor.get()); - // In a testing environment, we can only flush the application - // compaction filter buffer using universal compaction - option_config_ = kUniversalCompaction; - options.compaction_style = (rocksdb::CompactionStyle)1; - Reopen(&options); - - // Write 100K+1 keys, these are written to a few files - // in L0. We do this so that the current snapshot points - // to the 100001 key.The compaction filter is not invoked - // on keys that are visible via a snapshot because we - // anyways cannot delete it. - const std::string value(10, 'x'); - char first_key[100]; - snprintf(first_key, sizeof(first_key), "%s0000%010d", "NULL", 1); - Put(first_key, value); - for (int i = 1; i < 100000; i++) { - char key[100]; - snprintf(key, sizeof(key), "%08d%010d", i, i); - Put(key, value); - } - - char last_key[100]; - snprintf(last_key, sizeof(last_key), "%s0000%010d", "NULL", 2); - Put(last_key, value); - - // push all files to lower levels - dbfull()->TEST_FlushMemTable(); - dbfull()->TEST_CompactRange(0, nullptr, nullptr); - - // verify that all keys now have the new value that - // was set by the compaction process. - std::string newvalue = Get(first_key); - ASSERT_EQ(newvalue.compare(NEW_VALUE), 0); - newvalue = Get(last_key); - ASSERT_EQ(newvalue.compare(NEW_VALUE), 0); - for (int i = 1; i < 100000; i++) { - char key[100]; - snprintf(key, sizeof(key), "%08d%010d", i, i); - std::string newvalue = Get(key); - ASSERT_EQ(newvalue.compare(NEW_VALUE), 0); - } -} - -TEST(DBTest, SparseMerge) { - do { - Options options = CurrentOptions(); - options.compression = kNoCompression; - CreateAndReopenWithCF({"pikachu"}, &options); - - FillLevels("A", "Z", 1); - - // Suppose there is: - // small amount of data with prefix A - // large amount of data with prefix B - // small amount of data with prefix C - // and that recent updates have made small changes to all three prefixes. - // Check that we do not do a compaction that merges all of B in one shot. - const std::string value(1000, 'x'); - Put(1, "A", "va"); - // Write approximately 100MB of "B" values - for (int i = 0; i < 100000; i++) { - char key[100]; - snprintf(key, sizeof(key), "B%010d", i); - Put(1, key, value); - } - Put(1, "C", "vc"); - ASSERT_OK(Flush(1)); - dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); - - // Make sparse update - Put(1, "A", "va2"); - Put(1, "B100", "bvalue2"); - Put(1, "C", "vc2"); - ASSERT_OK(Flush(1)); - - // Compactions should not cause us to create a situation where - // a file overlaps too much data at the next level. - ASSERT_LE(dbfull()->TEST_MaxNextLevelOverlappingBytes(handles_[1]), - 20 * 1048576); - dbfull()->TEST_CompactRange(0, nullptr, nullptr); - ASSERT_LE(dbfull()->TEST_MaxNextLevelOverlappingBytes(handles_[1]), - 20 * 1048576); - dbfull()->TEST_CompactRange(1, nullptr, nullptr); - ASSERT_LE(dbfull()->TEST_MaxNextLevelOverlappingBytes(handles_[1]), - 20 * 1048576); - } while (ChangeCompactOptions()); -} - -static bool Between(uint64_t val, uint64_t low, uint64_t high) { - bool result = (val >= low) && (val <= high); - if (!result) { - fprintf(stderr, "Value %llu is not in range [%llu, %llu]\n", - (unsigned long long)(val), - (unsigned long long)(low), - (unsigned long long)(high)); - } - return result; -} - -TEST(DBTest, ApproximateSizes) { - do { - Options options; - options.write_buffer_size = 100000000; // Large write buffer + options.compaction_style = kCompactionStyleFIFO; + options.write_buffer_size = 100 << 10; // 100KB + options.arena_block_size = 4096; + options.compaction_options_fifo.max_table_files_size = 500 << 10; // 500KB options.compression = kNoCompression; - options = CurrentOptions(options); - DestroyAndReopen(); - CreateAndReopenWithCF({"pikachu"}, &options); - - ASSERT_TRUE(Between(Size("", "xyz", 1), 0, 0)); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - ASSERT_TRUE(Between(Size("", "xyz", 1), 0, 0)); - - // Write 8MB (80 values, each 100K) - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - const int N = 80; - static const int S1 = 100000; - static const int S2 = 105000; // Allow some expansion from metadata - Random rnd(301); - for (int i = 0; i < N; i++) { - ASSERT_OK(Put(1, Key(i), RandomString(&rnd, S1))); - } - - // 0 because GetApproximateSizes() does not account for memtable space - ASSERT_TRUE(Between(Size("", Key(50), 1), 0, 0)); - - // Check sizes across recovery by reopening a few times - for (int run = 0; run < 3; run++) { - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - - for (int compact_start = 0; compact_start < N; compact_start += 10) { - for (int i = 0; i < N; i += 10) { - ASSERT_TRUE(Between(Size("", Key(i), 1), S1 * i, S2 * i)); - ASSERT_TRUE(Between(Size("", Key(i) + ".suffix", 1), S1 * (i + 1), - S2 * (i + 1))); - ASSERT_TRUE(Between(Size(Key(i), Key(i + 10), 1), S1 * 10, S2 * 10)); - } - ASSERT_TRUE(Between(Size("", Key(50), 1), S1 * 50, S2 * 50)); - ASSERT_TRUE( - Between(Size("", Key(50) + ".suffix", 1), S1 * 50, S2 * 50)); - - std::string cstart_str = Key(compact_start); - std::string cend_str = Key(compact_start + 9); - Slice cstart = cstart_str; - Slice cend = cend_str; - dbfull()->TEST_CompactRange(0, &cstart, &cend, handles_[1]); - } - - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - ASSERT_GT(NumTableFilesAtLevel(1, 1), 0); - } - // ApproximateOffsetOf() is not yet implemented in plain table format. - } while (ChangeOptions(kSkipUniversalCompaction | kSkipFIFOCompaction | - kSkipPlainTable | kSkipHashIndex)); -} - -TEST(DBTest, ApproximateSizes_MixOfSmallAndLarge) { - do { - Options options = CurrentOptions(); - options.compression = kNoCompression; - CreateAndReopenWithCF({"pikachu"}, &options); - - Random rnd(301); - std::string big1 = RandomString(&rnd, 100000); - ASSERT_OK(Put(1, Key(0), RandomString(&rnd, 10000))); - ASSERT_OK(Put(1, Key(1), RandomString(&rnd, 10000))); - ASSERT_OK(Put(1, Key(2), big1)); - ASSERT_OK(Put(1, Key(3), RandomString(&rnd, 10000))); - ASSERT_OK(Put(1, Key(4), big1)); - ASSERT_OK(Put(1, Key(5), RandomString(&rnd, 10000))); - ASSERT_OK(Put(1, Key(6), RandomString(&rnd, 300000))); - ASSERT_OK(Put(1, Key(7), RandomString(&rnd, 10000))); - - // Check sizes across recovery by reopening a few times - for (int run = 0; run < 3; run++) { - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - - ASSERT_TRUE(Between(Size("", Key(0), 1), 0, 0)); - ASSERT_TRUE(Between(Size("", Key(1), 1), 10000, 11000)); - ASSERT_TRUE(Between(Size("", Key(2), 1), 20000, 21000)); - ASSERT_TRUE(Between(Size("", Key(3), 1), 120000, 121000)); - ASSERT_TRUE(Between(Size("", Key(4), 1), 130000, 131000)); - ASSERT_TRUE(Between(Size("", Key(5), 1), 230000, 231000)); - ASSERT_TRUE(Between(Size("", Key(6), 1), 240000, 241000)); - ASSERT_TRUE(Between(Size("", Key(7), 1), 540000, 541000)); - ASSERT_TRUE(Between(Size("", Key(8), 1), 550000, 560000)); - - ASSERT_TRUE(Between(Size(Key(3), Key(5), 1), 110000, 111000)); - - dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); - } - // ApproximateOffsetOf() is not yet implemented in plain table format. - } while (ChangeOptions(kSkipPlainTable)); -} - -TEST(DBTest, IteratorPinsRef) { - do { - CreateAndReopenWithCF({"pikachu"}); - Put(1, "foo", "hello"); - - // Get iterator that will yield the current contents of the DB. - Iterator* iter = db_->NewIterator(ReadOptions(), handles_[1]); - - // Write to force compactions - Put(1, "foo", "newvalue1"); - for (int i = 0; i < 100; i++) { - // 100K values - ASSERT_OK(Put(1, Key(i), Key(i) + std::string(100000, 'v'))); - } - Put(1, "foo", "newvalue2"); - - iter->SeekToFirst(); - ASSERT_TRUE(iter->Valid()); - ASSERT_EQ("foo", iter->key().ToString()); - ASSERT_EQ("hello", iter->value().ToString()); - iter->Next(); - ASSERT_TRUE(!iter->Valid()); - delete iter; - } while (ChangeCompactOptions()); -} - -TEST(DBTest, Snapshot) { - do { - CreateAndReopenWithCF({"pikachu"}); - Put(0, "foo", "0v1"); - Put(1, "foo", "1v1"); - const Snapshot* s1 = db_->GetSnapshot(); - Put(0, "foo", "0v2"); - Put(1, "foo", "1v2"); - const Snapshot* s2 = db_->GetSnapshot(); - Put(0, "foo", "0v3"); - Put(1, "foo", "1v3"); - const Snapshot* s3 = db_->GetSnapshot(); - - Put(0, "foo", "0v4"); - Put(1, "foo", "1v4"); - ASSERT_EQ("0v1", Get(0, "foo", s1)); - ASSERT_EQ("1v1", Get(1, "foo", s1)); - ASSERT_EQ("0v2", Get(0, "foo", s2)); - ASSERT_EQ("1v2", Get(1, "foo", s2)); - ASSERT_EQ("0v3", Get(0, "foo", s3)); - ASSERT_EQ("1v3", Get(1, "foo", s3)); - ASSERT_EQ("0v4", Get(0, "foo")); - ASSERT_EQ("1v4", Get(1, "foo")); - - db_->ReleaseSnapshot(s3); - ASSERT_EQ("0v1", Get(0, "foo", s1)); - ASSERT_EQ("1v1", Get(1, "foo", s1)); - ASSERT_EQ("0v2", Get(0, "foo", s2)); - ASSERT_EQ("1v2", Get(1, "foo", s2)); - ASSERT_EQ("0v4", Get(0, "foo")); - ASSERT_EQ("1v4", Get(1, "foo")); - - db_->ReleaseSnapshot(s1); - ASSERT_EQ("0v2", Get(0, "foo", s2)); - ASSERT_EQ("1v2", Get(1, "foo", s2)); - ASSERT_EQ("0v4", Get(0, "foo")); - ASSERT_EQ("1v4", Get(1, "foo")); - - db_->ReleaseSnapshot(s2); - ASSERT_EQ("0v4", Get(0, "foo")); - ASSERT_EQ("1v4", Get(1, "foo")); - } while (ChangeOptions(kSkipHashCuckoo)); -} - -TEST(DBTest, HiddenValuesAreRemoved) { - do { - CreateAndReopenWithCF({"pikachu"}); - Random rnd(301); - FillLevels("a", "z", 1); - - std::string big = RandomString(&rnd, 50000); - Put(1, "foo", big); - Put(1, "pastfoo", "v"); - const Snapshot* snapshot = db_->GetSnapshot(); - Put(1, "foo", "tiny"); - Put(1, "pastfoo2", "v2"); // Advance sequence number one more - - ASSERT_OK(Flush(1)); - ASSERT_GT(NumTableFilesAtLevel(0, 1), 0); - - ASSERT_EQ(big, Get(1, "foo", snapshot)); - ASSERT_TRUE(Between(Size("", "pastfoo", 1), 50000, 60000)); - db_->ReleaseSnapshot(snapshot); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ tiny, " + big + " ]"); - Slice x("x"); - dbfull()->TEST_CompactRange(0, nullptr, &x, handles_[1]); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ tiny ]"); - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - ASSERT_GE(NumTableFilesAtLevel(1, 1), 1); - dbfull()->TEST_CompactRange(1, nullptr, &x, handles_[1]); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ tiny ]"); - - ASSERT_TRUE(Between(Size("", "pastfoo", 1), 0, 1000)); - // ApproximateOffsetOf() is not yet implemented in plain table format, - // which is used by Size(). - // skip HashCuckooRep as it does not support snapshot - } while (ChangeOptions(kSkipUniversalCompaction | kSkipFIFOCompaction | - kSkipPlainTable | kSkipHashCuckoo)); -} - -TEST(DBTest, CompactBetweenSnapshots) { - do { - Options options = CurrentOptions(); - options.disable_auto_compactions = true; - CreateAndReopenWithCF({"pikachu"}); - Random rnd(301); - FillLevels("a", "z", 1); - - Put(1, "foo", "first"); - const Snapshot* snapshot1 = db_->GetSnapshot(); - Put(1, "foo", "second"); - Put(1, "foo", "third"); - Put(1, "foo", "fourth"); - const Snapshot* snapshot2 = db_->GetSnapshot(); - Put(1, "foo", "fifth"); - Put(1, "foo", "sixth"); - - // All entries (including duplicates) exist - // before any compaction is triggered. - ASSERT_OK(Flush(1)); - ASSERT_EQ("sixth", Get(1, "foo")); - ASSERT_EQ("fourth", Get(1, "foo", snapshot2)); - ASSERT_EQ("first", Get(1, "foo", snapshot1)); - ASSERT_EQ(AllEntriesFor("foo", 1), - "[ sixth, fifth, fourth, third, second, first ]"); - - // After a compaction, "second", "third" and "fifth" should - // be removed - FillLevels("a", "z", 1); - dbfull()->CompactRange(handles_[1], nullptr, nullptr); - ASSERT_EQ("sixth", Get(1, "foo")); - ASSERT_EQ("fourth", Get(1, "foo", snapshot2)); - ASSERT_EQ("first", Get(1, "foo", snapshot1)); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ sixth, fourth, first ]"); - - // after we release the snapshot1, only two values left - db_->ReleaseSnapshot(snapshot1); - FillLevels("a", "z", 1); - dbfull()->CompactRange(handles_[1], nullptr, nullptr); - - // We have only one valid snapshot snapshot2. Since snapshot1 is - // not valid anymore, "first" should be removed by a compaction. - ASSERT_EQ("sixth", Get(1, "foo")); - ASSERT_EQ("fourth", Get(1, "foo", snapshot2)); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ sixth, fourth ]"); - - // after we release the snapshot2, only one value should be left - db_->ReleaseSnapshot(snapshot2); - FillLevels("a", "z", 1); - dbfull()->CompactRange(handles_[1], nullptr, nullptr); - ASSERT_EQ("sixth", Get(1, "foo")); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ sixth ]"); - // skip HashCuckooRep as it does not support snapshot - } while (ChangeOptions(kSkipHashCuckoo | kSkipFIFOCompaction)); -} - -TEST(DBTest, DeletionMarkers1) { - CreateAndReopenWithCF({"pikachu"}); - Put(1, "foo", "v1"); - ASSERT_OK(Flush(1)); - const int last = CurrentOptions().max_mem_compaction_level; - // foo => v1 is now in last level - ASSERT_EQ(NumTableFilesAtLevel(last, 1), 1); - - // Place a table at level last-1 to prevent merging with preceding mutation - Put(1, "a", "begin"); - Put(1, "z", "end"); - Flush(1); - ASSERT_EQ(NumTableFilesAtLevel(last, 1), 1); - ASSERT_EQ(NumTableFilesAtLevel(last - 1, 1), 1); - - Delete(1, "foo"); - Put(1, "foo", "v2"); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, DEL, v1 ]"); - ASSERT_OK(Flush(1)); // Moves to level last-2 - if (CurrentOptions().purge_redundant_kvs_while_flush) { - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, v1 ]"); - } else { - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, DEL, v1 ]"); - } - Slice z("z"); - dbfull()->TEST_CompactRange(last - 2, nullptr, &z, handles_[1]); - // DEL eliminated, but v1 remains because we aren't compacting that level - // (DEL can be eliminated because v2 hides v1). - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, v1 ]"); - dbfull()->TEST_CompactRange(last - 1, nullptr, nullptr, handles_[1]); - // Merging last-1 w/ last, so we are the base level for "foo", so - // DEL is removed. (as is v1). - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2 ]"); -} - -TEST(DBTest, DeletionMarkers2) { - CreateAndReopenWithCF({"pikachu"}); - Put(1, "foo", "v1"); - ASSERT_OK(Flush(1)); - const int last = CurrentOptions().max_mem_compaction_level; - // foo => v1 is now in last level - ASSERT_EQ(NumTableFilesAtLevel(last, 1), 1); - - // Place a table at level last-1 to prevent merging with preceding mutation - Put(1, "a", "begin"); - Put(1, "z", "end"); - Flush(1); - ASSERT_EQ(NumTableFilesAtLevel(last, 1), 1); - ASSERT_EQ(NumTableFilesAtLevel(last - 1, 1), 1); - - Delete(1, "foo"); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, v1 ]"); - ASSERT_OK(Flush(1)); // Moves to level last-2 - ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, v1 ]"); - dbfull()->TEST_CompactRange(last - 2, nullptr, nullptr, handles_[1]); - // DEL kept: "last" file overlaps - ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, v1 ]"); - dbfull()->TEST_CompactRange(last - 1, nullptr, nullptr, handles_[1]); - // Merging last-1 w/ last, so we are the base level for "foo", so - // DEL is removed. (as is v1). - ASSERT_EQ(AllEntriesFor("foo", 1), "[ ]"); -} - -TEST(DBTest, OverlapInLevel0) { - do { - CreateAndReopenWithCF({"pikachu"}); - int tmp = CurrentOptions().max_mem_compaction_level; - ASSERT_EQ(tmp, 2) << "Fix test to match config"; - - //Fill levels 1 and 2 to disable the pushing of new memtables to levels > 0. - ASSERT_OK(Put(1, "100", "v100")); - ASSERT_OK(Put(1, "999", "v999")); - Flush(1); - ASSERT_OK(Delete(1, "100")); - ASSERT_OK(Delete(1, "999")); - Flush(1); - ASSERT_EQ("0,1,1", FilesPerLevel(1)); - - // Make files spanning the following ranges in level-0: - // files[0] 200 .. 900 - // files[1] 300 .. 500 - // Note that files are sorted by smallest key. - ASSERT_OK(Put(1, "300", "v300")); - ASSERT_OK(Put(1, "500", "v500")); - Flush(1); - ASSERT_OK(Put(1, "200", "v200")); - ASSERT_OK(Put(1, "600", "v600")); - ASSERT_OK(Put(1, "900", "v900")); - Flush(1); - ASSERT_EQ("2,1,1", FilesPerLevel(1)); - - // Compact away the placeholder files we created initially - dbfull()->TEST_CompactRange(1, nullptr, nullptr, handles_[1]); - dbfull()->TEST_CompactRange(2, nullptr, nullptr, handles_[1]); - ASSERT_EQ("2", FilesPerLevel(1)); - - // Do a memtable compaction. Before bug-fix, the compaction would - // not detect the overlap with level-0 files and would incorrectly place - // the deletion in a deeper level. - ASSERT_OK(Delete(1, "600")); - Flush(1); - ASSERT_EQ("3", FilesPerLevel(1)); - ASSERT_EQ("NOT_FOUND", Get(1, "600")); - } while (ChangeOptions(kSkipUniversalCompaction | kSkipFIFOCompaction)); -} - -TEST(DBTest, L0_CompactionBug_Issue44_a) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "b", "v")); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_OK(Delete(1, "b")); - ASSERT_OK(Delete(1, "a")); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_OK(Delete(1, "a")); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_OK(Put(1, "a", "v")); - ReopenWithColumnFamilies({"default", "pikachu"}); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_EQ("(a->v)", Contents(1)); - env_->SleepForMicroseconds(1000000); // Wait for compaction to finish - ASSERT_EQ("(a->v)", Contents(1)); - } while (ChangeCompactOptions()); -} - -TEST(DBTest, L0_CompactionBug_Issue44_b) { - do { - CreateAndReopenWithCF({"pikachu"}); - Put(1, "", ""); - ReopenWithColumnFamilies({"default", "pikachu"}); - Delete(1, "e"); - Put(1, "", ""); - ReopenWithColumnFamilies({"default", "pikachu"}); - Put(1, "c", "cv"); - ReopenWithColumnFamilies({"default", "pikachu"}); - Put(1, "", ""); - ReopenWithColumnFamilies({"default", "pikachu"}); - Put(1, "", ""); - env_->SleepForMicroseconds(1000000); // Wait for compaction to finish - ReopenWithColumnFamilies({"default", "pikachu"}); - Put(1, "d", "dv"); - ReopenWithColumnFamilies({"default", "pikachu"}); - Put(1, "", ""); - ReopenWithColumnFamilies({"default", "pikachu"}); - Delete(1, "d"); - Delete(1, "b"); - ReopenWithColumnFamilies({"default", "pikachu"}); - ASSERT_EQ("(->)(c->cv)", Contents(1)); - env_->SleepForMicroseconds(1000000); // Wait for compaction to finish - ASSERT_EQ("(->)(c->cv)", Contents(1)); - } while (ChangeCompactOptions()); -} - -TEST(DBTest, ComparatorCheck) { - class NewComparator : public Comparator { - public: - virtual const char* Name() const { return "rocksdb.NewComparator"; } - virtual int Compare(const Slice& a, const Slice& b) const { - return BytewiseComparator()->Compare(a, b); - } - virtual void FindShortestSeparator(std::string* s, const Slice& l) const { - BytewiseComparator()->FindShortestSeparator(s, l); - } - virtual void FindShortSuccessor(std::string* key) const { - BytewiseComparator()->FindShortSuccessor(key); - } - }; - Options new_options, options; - NewComparator cmp; - do { - CreateAndReopenWithCF({"pikachu"}); - options = CurrentOptions(); - new_options = CurrentOptions(); - new_options.comparator = &cmp; - // only the non-default column family has non-matching comparator - Status s = TryReopenWithColumnFamilies({"default", "pikachu"}, - {&options, &new_options}); - ASSERT_TRUE(!s.ok()); - ASSERT_TRUE(s.ToString().find("comparator") != std::string::npos) - << s.ToString(); - } while (ChangeCompactOptions(&new_options)); -} - -TEST(DBTest, CustomComparator) { - class NumberComparator : public Comparator { - public: - virtual const char* Name() const { return "test.NumberComparator"; } - virtual int Compare(const Slice& a, const Slice& b) const { - return ToNumber(a) - ToNumber(b); - } - virtual void FindShortestSeparator(std::string* s, const Slice& l) const { - ToNumber(*s); // Check format - ToNumber(l); // Check format - } - virtual void FindShortSuccessor(std::string* key) const { - ToNumber(*key); // Check format - } - private: - static int ToNumber(const Slice& x) { - // Check that there are no extra characters. - ASSERT_TRUE(x.size() >= 2 && x[0] == '[' && x[x.size()-1] == ']') - << EscapeString(x); - int val; - char ignored; - ASSERT_TRUE(sscanf(x.ToString().c_str(), "[%i]%c", &val, &ignored) == 1) - << EscapeString(x); - return val; - } - }; - Options new_options; - NumberComparator cmp; - do { - new_options = CurrentOptions(); - new_options.create_if_missing = true; - new_options.comparator = &cmp; - new_options.write_buffer_size = 1000; // Compact more often - new_options = CurrentOptions(new_options); - DestroyAndReopen(&new_options); - CreateAndReopenWithCF({"pikachu"}, &new_options); - ASSERT_OK(Put(1, "[10]", "ten")); - ASSERT_OK(Put(1, "[0x14]", "twenty")); - for (int i = 0; i < 2; i++) { - ASSERT_EQ("ten", Get(1, "[10]")); - ASSERT_EQ("ten", Get(1, "[0xa]")); - ASSERT_EQ("twenty", Get(1, "[20]")); - ASSERT_EQ("twenty", Get(1, "[0x14]")); - ASSERT_EQ("NOT_FOUND", Get(1, "[15]")); - ASSERT_EQ("NOT_FOUND", Get(1, "[0xf]")); - Compact(1, "[0]", "[9999]"); + options.create_if_missing = true; + options.max_subcompactions = max_subcompactions_; + if (iter == 1) { + options.disable_auto_compactions = true; } + options = CurrentOptions(options); + DestroyAndReopen(options); - for (int run = 0; run < 2; run++) { - for (int i = 0; i < 1000; i++) { - char buf[100]; - snprintf(buf, sizeof(buf), "[%d]", i*10); - ASSERT_OK(Put(1, buf, buf)); + Random rnd(301); + for (int i = 0; i < 6; ++i) { + for (int j = 0; j < 110; ++j) { + ASSERT_OK(Put(ToString(i * 100 + j), RandomString(&rnd, 980))); } - Compact(1, "[0]", "[1000000]"); + // flush should happen here + ASSERT_OK(dbfull()->TEST_WaitForFlushMemTable()); + } + if (iter == 0) { + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + } else { + CompactRangeOptions cro; + cro.exclusive_manual_compaction = exclusive_manual_compaction_; + ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr)); } - } while (ChangeCompactOptions(&new_options)); + // only 5 files should survive + ASSERT_EQ(NumTableFilesAtLevel(0), 5); + for (int i = 0; i < 50; ++i) { + // these keys should be deleted in previous compaction + ASSERT_EQ("NOT_FOUND", Get(ToString(i))); + } + } } -TEST(DBTest, ManualCompaction) { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_EQ(dbfull()->MaxMemCompactionLevel(), 2) - << "Need to update this test to match kMaxMemCompactLevel"; - - // iter - 0 with 7 levels - // iter - 1 with 3 levels - for (int iter = 0; iter < 2; ++iter) { - MakeTables(3, "p", "q", 1); - ASSERT_EQ("1,1,1", FilesPerLevel(1)); - - // Compaction range falls before files - Compact(1, "", "c"); - ASSERT_EQ("1,1,1", FilesPerLevel(1)); - - // Compaction range falls after files - Compact(1, "r", "z"); - ASSERT_EQ("1,1,1", FilesPerLevel(1)); - - // Compaction range overlaps files - Compact(1, "p1", "p9"); - ASSERT_EQ("0,0,1", FilesPerLevel(1)); - - // Populate a different range - MakeTables(3, "c", "e", 1); - ASSERT_EQ("1,1,2", FilesPerLevel(1)); - - // Compact just the new range - Compact(1, "b", "f"); - ASSERT_EQ("0,0,2", FilesPerLevel(1)); +TEST_F(DBTest, FIFOCompactionTestWithCompaction) { + Options options; + options.compaction_style = kCompactionStyleFIFO; + options.write_buffer_size = 20 << 10; // 20K + options.arena_block_size = 4096; + options.compaction_options_fifo.max_table_files_size = 1500 << 10; // 1MB + options.compaction_options_fifo.allow_compaction = true; + options.level0_file_num_compaction_trigger = 6; + options.compression = kNoCompression; + options.create_if_missing = true; + options = CurrentOptions(options); + DestroyAndReopen(options); - // Compact all - MakeTables(1, "a", "z", 1); - ASSERT_EQ("0,1,2", FilesPerLevel(1)); - db_->CompactRange(handles_[1], nullptr, nullptr); - ASSERT_EQ("0,0,1", FilesPerLevel(1)); + Random rnd(301); + for (int i = 0; i < 60; i++) { + // Generate and flush a file about 20KB. + for (int j = 0; j < 20; j++) { + ASSERT_OK(Put(ToString(i * 20 + j), RandomString(&rnd, 980))); + } + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + } + // It should be compacted to 10 files. + ASSERT_EQ(NumTableFilesAtLevel(0), 10); - if (iter == 0) { - Options options = CurrentOptions(); - options.num_levels = 3; - options.create_if_missing = true; - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); + for (int i = 0; i < 60; i++) { + // Generate and flush a file about 20KB. + for (int j = 0; j < 20; j++) { + ASSERT_OK(Put(ToString(i * 20 + j + 2000), RandomString(&rnd, 980))); } + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); } + // It should be compacted to no more than 20 files. + ASSERT_GT(NumTableFilesAtLevel(0), 10); + ASSERT_LT(NumTableFilesAtLevel(0), 18); + // Size limit is still guaranteed. + ASSERT_LE(SizeAtLevel(0), + options.compaction_options_fifo.max_table_files_size); } -TEST(DBTest, ManualCompactionOutputPathId) { - Options options = CurrentOptions(); +TEST_F(DBTest, FIFOCompactionStyleWithCompactionAndDelete) { + Options options; + options.compaction_style = kCompactionStyleFIFO; + options.write_buffer_size = 20 << 10; // 20K + options.arena_block_size = 4096; + options.compaction_options_fifo.max_table_files_size = 1500 << 10; // 1MB + options.compaction_options_fifo.allow_compaction = true; + options.level0_file_num_compaction_trigger = 3; + options.compression = kNoCompression; options.create_if_missing = true; - options.db_paths.emplace_back(dbname_, 1000000000); - options.db_paths.emplace_back(dbname_ + "_2", 1000000000); - options.compaction_style = kCompactionStyleUniversal; - options.level0_file_num_compaction_trigger = 10; - Destroy(&options); - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); - MakeTables(3, "p", "q", 1); - dbfull()->TEST_WaitForCompact(); - ASSERT_EQ("3", FilesPerLevel(1)); - ASSERT_EQ(3, GetSstFileCount(options.db_paths[0].path)); - ASSERT_EQ(0, GetSstFileCount(options.db_paths[1].path)); - - // Full compaction to DB path 0 - db_->CompactRange(handles_[1], nullptr, nullptr, false, -1, 1); - ASSERT_EQ("1", FilesPerLevel(1)); - ASSERT_EQ(0, GetSstFileCount(options.db_paths[0].path)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - - ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, &options); - ASSERT_EQ("1", FilesPerLevel(1)); - ASSERT_EQ(0, GetSstFileCount(options.db_paths[0].path)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - - MakeTables(1, "p", "q", 1); - ASSERT_EQ("2", FilesPerLevel(1)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[0].path)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - - ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, &options); - ASSERT_EQ("2", FilesPerLevel(1)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[0].path)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); - - // Full compaction to DB path 0 - db_->CompactRange(handles_[1], nullptr, nullptr, false, -1, 0); - ASSERT_EQ("1", FilesPerLevel(1)); - ASSERT_EQ(1, GetSstFileCount(options.db_paths[0].path)); - ASSERT_EQ(0, GetSstFileCount(options.db_paths[1].path)); - - // Fail when compacting to an invalid path ID - ASSERT_TRUE(db_->CompactRange(handles_[1], nullptr, nullptr, false, -1, 2) - .IsInvalidArgument()); + options = CurrentOptions(options); + DestroyAndReopen(options); + + Random rnd(301); + for (int i = 0; i < 3; i++) { + // Each file contains a different key which will be dropped later. + ASSERT_OK(Put("a" + ToString(i), RandomString(&rnd, 500))); + ASSERT_OK(Put("key" + ToString(i), "")); + ASSERT_OK(Put("z" + ToString(i), RandomString(&rnd, 500))); + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + } + ASSERT_EQ(NumTableFilesAtLevel(0), 1); + for (int i = 0; i < 3; i++) { + ASSERT_EQ("", Get("key" + ToString(i))); + } + for (int i = 0; i < 3; i++) { + // Each file contains a different key which will be dropped later. + ASSERT_OK(Put("a" + ToString(i), RandomString(&rnd, 500))); + ASSERT_OK(Delete("key" + ToString(i))); + ASSERT_OK(Put("z" + ToString(i), RandomString(&rnd, 500))); + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + } + ASSERT_EQ(NumTableFilesAtLevel(0), 2); + for (int i = 0; i < 3; i++) { + ASSERT_EQ("NOT_FOUND", Get("key" + ToString(i))); + } } -TEST(DBTest, DBOpen_Options) { - std::string dbname = test::TmpDir() + "/db_options_test"; - ASSERT_OK(DestroyDB(dbname, Options())); +// Check that FIFO-with-TTL is not supported with max_open_files != -1. +TEST_F(DBTest, FIFOCompactionWithTTLAndMaxOpenFilesTest) { + Options options; + options.compaction_style = kCompactionStyleFIFO; + options.create_if_missing = true; + options.compaction_options_fifo.ttl = 600; // seconds - // Does not exist, and create_if_missing == false: error - DB* db = nullptr; - Options opts; - opts.create_if_missing = false; - Status s = DB::Open(opts, dbname, &db); - ASSERT_TRUE(strstr(s.ToString().c_str(), "does not exist") != nullptr); - ASSERT_TRUE(db == nullptr); + // Check that it is not supported with max_open_files != -1. + options.max_open_files = 100; + options = CurrentOptions(options); + ASSERT_TRUE(TryReopen(options).IsNotSupported()); - // Does not exist, and create_if_missing == true: OK - opts.create_if_missing = true; - s = DB::Open(opts, dbname, &db); - ASSERT_OK(s); - ASSERT_TRUE(db != nullptr); + options.max_open_files = -1; + ASSERT_OK(TryReopen(options)); +} - delete db; - db = nullptr; +// Check that FIFO-with-TTL is supported only with BlockBasedTableFactory. +TEST_F(DBTest, FIFOCompactionWithTTLAndVariousTableFormatsTest) { + Options options; + options.compaction_style = kCompactionStyleFIFO; + options.create_if_missing = true; + options.compaction_options_fifo.ttl = 600; // seconds - // Does exist, and error_if_exists == true: error - opts.create_if_missing = false; - opts.error_if_exists = true; - s = DB::Open(opts, dbname, &db); - ASSERT_TRUE(strstr(s.ToString().c_str(), "exists") != nullptr); - ASSERT_TRUE(db == nullptr); + options = CurrentOptions(options); + options.table_factory.reset(NewBlockBasedTableFactory()); + ASSERT_OK(TryReopen(options)); - // Does exist, and error_if_exists == false: OK - opts.create_if_missing = true; - opts.error_if_exists = false; - s = DB::Open(opts, dbname, &db); - ASSERT_OK(s); - ASSERT_TRUE(db != nullptr); + Destroy(options); + options.table_factory.reset(NewPlainTableFactory()); + ASSERT_TRUE(TryReopen(options).IsNotSupported()); - delete db; - db = nullptr; -} + Destroy(options); + options.table_factory.reset(NewCuckooTableFactory()); + ASSERT_TRUE(TryReopen(options).IsNotSupported()); -TEST(DBTest, DBOpen_Change_NumLevels) { - Options opts; - opts.create_if_missing = true; - DestroyAndReopen(&opts); - ASSERT_TRUE(db_ != nullptr); - CreateAndReopenWithCF({"pikachu"}, &opts); + Destroy(options); + options.table_factory.reset(NewAdaptiveTableFactory()); + ASSERT_TRUE(TryReopen(options).IsNotSupported()); +} - ASSERT_OK(Put(1, "a", "123")); - ASSERT_OK(Put(1, "b", "234")); - db_->CompactRange(handles_[1], nullptr, nullptr); - Close(); +TEST_F(DBTest, FIFOCompactionWithTTLTest) { + Options options; + options.compaction_style = kCompactionStyleFIFO; + options.write_buffer_size = 10 << 10; // 10KB + options.arena_block_size = 4096; + options.compression = kNoCompression; + options.create_if_missing = true; + env_->time_elapse_only_sleep_ = false; + options.env = env_; - opts.create_if_missing = false; - opts.num_levels = 2; - Status s = TryReopenWithColumnFamilies({"default", "pikachu"}, &opts); - ASSERT_TRUE(strstr(s.ToString().c_str(), "Invalid argument") != nullptr); - ASSERT_TRUE(db_ == nullptr); -} + // Test to make sure that all files with expired ttl are deleted on next + // manual compaction. + { + env_->addon_time_.store(0); + options.compaction_options_fifo.max_table_files_size = 150 << 10; // 150KB + options.compaction_options_fifo.allow_compaction = false; + options.compaction_options_fifo.ttl = 1 * 60 * 60 ; // 1 hour + options = CurrentOptions(options); + DestroyAndReopen(options); -TEST(DBTest, DestroyDBMetaDatabase) { - std::string dbname = test::TmpDir() + "/db_meta"; - std::string metadbname = MetaDatabaseName(dbname, 0); - std::string metametadbname = MetaDatabaseName(metadbname, 0); + Random rnd(301); + for (int i = 0; i < 10; i++) { + // Generate and flush a file about 10KB. + for (int j = 0; j < 10; j++) { + ASSERT_OK(Put(ToString(i * 20 + j), RandomString(&rnd, 980))); + } + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + } + ASSERT_EQ(NumTableFilesAtLevel(0), 10); - // Destroy previous versions if they exist. Using the long way. - ASSERT_OK(DestroyDB(metametadbname, Options())); - ASSERT_OK(DestroyDB(metadbname, Options())); - ASSERT_OK(DestroyDB(dbname, Options())); + // Sleep for 2 hours -- which is much greater than TTL. + // Note: Couldn't use SleepForMicroseconds because it takes an int instead + // of uint64_t. Hence used addon_time_ directly. + // env_->SleepForMicroseconds(2 * 60 * 60 * 1000 * 1000); + env_->addon_time_.fetch_add(2 * 60 * 60); - // Setup databases - Options opts; - opts.create_if_missing = true; - DB* db = nullptr; - ASSERT_OK(DB::Open(opts, dbname, &db)); - delete db; - db = nullptr; - ASSERT_OK(DB::Open(opts, metadbname, &db)); - delete db; - db = nullptr; - ASSERT_OK(DB::Open(opts, metametadbname, &db)); - delete db; - db = nullptr; + // Since no flushes and compactions have run, the db should still be in + // the same state even after considerable time has passed. + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + ASSERT_EQ(NumTableFilesAtLevel(0), 10); - // Delete databases - ASSERT_OK(DestroyDB(dbname, Options())); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + } - // Check if deletion worked. - opts.create_if_missing = false; - ASSERT_TRUE(!(DB::Open(opts, dbname, &db)).ok()); - ASSERT_TRUE(!(DB::Open(opts, metadbname, &db)).ok()); - ASSERT_TRUE(!(DB::Open(opts, metametadbname, &db)).ok()); -} + // Test to make sure that all files with expired ttl are deleted on next + // automatic compaction. + { + options.compaction_options_fifo.max_table_files_size = 150 << 10; // 150KB + options.compaction_options_fifo.allow_compaction = false; + options.compaction_options_fifo.ttl = 1 * 60 * 60; // 1 hour + options = CurrentOptions(options); + DestroyAndReopen(options); -// Check that number of files does not grow when we are out of space -TEST(DBTest, NoSpace) { - do { - Options options = CurrentOptions(); - options.env = env_; - options.paranoid_checks = false; - Reopen(&options); - - ASSERT_OK(Put("foo", "v1")); - ASSERT_EQ("v1", Get("foo")); - Compact("a", "z"); - const int num_files = CountFiles(); - env_->no_space_.Release_Store(env_); // Force out-of-space errors - env_->sleep_counter_.Reset(); - for (int i = 0; i < 5; i++) { - for (int level = 0; level < dbfull()->NumberLevels()-1; level++) { - dbfull()->TEST_CompactRange(level, nullptr, nullptr); + Random rnd(301); + for (int i = 0; i < 10; i++) { + // Generate and flush a file about 10KB. + for (int j = 0; j < 10; j++) { + ASSERT_OK(Put(ToString(i * 20 + j), RandomString(&rnd, 980))); } + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); } + ASSERT_EQ(NumTableFilesAtLevel(0), 10); - std::string property_value; - ASSERT_TRUE(db_->GetProperty("rocksdb.background-errors", &property_value)); - ASSERT_EQ("5", property_value); + // Sleep for 2 hours -- which is much greater than TTL. + env_->addon_time_.fetch_add(2 * 60 * 60); + // Just to make sure that we are in the same state even after sleeping. + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + ASSERT_EQ(NumTableFilesAtLevel(0), 10); - env_->no_space_.Release_Store(nullptr); - ASSERT_LT(CountFiles(), num_files + 3); + // Create 1 more file to trigger TTL compaction. The old files are dropped. + for (int i = 0; i < 1; i++) { + for (int j = 0; j < 10; j++) { + ASSERT_OK(Put(ToString(i * 20 + j), RandomString(&rnd, 980))); + } + Flush(); + } - // Check that compaction attempts slept after errors - ASSERT_GE(env_->sleep_counter_.Read(), 5); - } while (ChangeCompactOptions()); -} + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + // Only the new 10 files remain. + ASSERT_EQ(NumTableFilesAtLevel(0), 1); + ASSERT_LE(SizeAtLevel(0), + options.compaction_options_fifo.max_table_files_size); + } -// Check background error counter bumped on flush failures. -TEST(DBTest, NoSpaceFlush) { - do { - Options options = CurrentOptions(); - options.env = env_; - options.max_background_flushes = 1; - Reopen(&options); - - ASSERT_OK(Put("foo", "v1")); - env_->no_space_.Release_Store(env_); // Force out-of-space errors - - std::string property_value; - // Background error count is 0 now. - ASSERT_TRUE(db_->GetProperty("rocksdb.background-errors", &property_value)); - ASSERT_EQ("0", property_value); - - dbfull()->TEST_FlushMemTable(false); - - // Wait 300 milliseconds or background-errors turned 1 from 0. - int time_to_sleep_limit = 300000; - while (time_to_sleep_limit > 0) { - int to_sleep = (time_to_sleep_limit > 1000) ? 1000 : time_to_sleep_limit; - time_to_sleep_limit -= to_sleep; - env_->SleepForMicroseconds(to_sleep); - - ASSERT_TRUE( - db_->GetProperty("rocksdb.background-errors", &property_value)); - if (property_value == "1") { - break; + // Test that shows the fall back to size-based FIFO compaction if TTL-based + // deletion doesn't move the total size to be less than max_table_files_size. + { + options.write_buffer_size = 10 << 10; // 10KB + options.compaction_options_fifo.max_table_files_size = 150 << 10; // 150KB + options.compaction_options_fifo.allow_compaction = false; + options.compaction_options_fifo.ttl = 1 * 60 * 60; // 1 hour + options = CurrentOptions(options); + DestroyAndReopen(options); + + Random rnd(301); + for (int i = 0; i < 3; i++) { + // Generate and flush a file about 10KB. + for (int j = 0; j < 10; j++) { + ASSERT_OK(Put(ToString(i * 20 + j), RandomString(&rnd, 980))); } + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); } - ASSERT_EQ("1", property_value); + ASSERT_EQ(NumTableFilesAtLevel(0), 3); - env_->no_space_.Release_Store(nullptr); - } while (ChangeCompactOptions()); -} + // Sleep for 2 hours -- which is much greater than TTL. + env_->addon_time_.fetch_add(2 * 60 * 60); + // Just to make sure that we are in the same state even after sleeping. + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + ASSERT_EQ(NumTableFilesAtLevel(0), 3); -TEST(DBTest, NonWritableFileSystem) { - do { - Options options = CurrentOptions(); - options.write_buffer_size = 1000; - options.env = env_; - Reopen(&options); - ASSERT_OK(Put("foo", "v1")); - env_->non_writable_.Release_Store(env_); // Force errors for new files - std::string big(100000, 'x'); - int errors = 0; - for (int i = 0; i < 20; i++) { - if (!Put("foo", big).ok()) { - errors++; - env_->SleepForMicroseconds(100000); + for (int i = 0; i < 5; i++) { + for (int j = 0; j < 140; j++) { + ASSERT_OK(Put(ToString(i * 20 + j), RandomString(&rnd, 980))); } + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); } - ASSERT_GT(errors, 0); - env_->non_writable_.Release_Store(nullptr); - } while (ChangeCompactOptions()); -} + // Size limit is still guaranteed. + ASSERT_LE(SizeAtLevel(0), + options.compaction_options_fifo.max_table_files_size); + } -TEST(DBTest, ManifestWriteError) { - // Test for the following problem: - // (a) Compaction produces file F - // (b) Log record containing F is written to MANIFEST file, but Sync() fails - // (c) GC deletes F - // (d) After reopening DB, reads fail since deleted F is named in log record - - // We iterate twice. In the second iteration, everything is the - // same except the log record never makes it to the MANIFEST file. - for (int iter = 0; iter < 2; iter++) { - port::AtomicPointer* error_type = (iter == 0) - ? &env_->manifest_sync_error_ - : &env_->manifest_write_error_; - - // Insert foo=>bar mapping - Options options = CurrentOptions(); - options.env = env_; - options.create_if_missing = true; - options.error_if_exists = false; - DestroyAndReopen(&options); - ASSERT_OK(Put("foo", "bar")); - ASSERT_EQ("bar", Get("foo")); + // Test with TTL + Intra-L0 compactions. + { + options.compaction_options_fifo.max_table_files_size = 150 << 10; // 150KB + options.compaction_options_fifo.allow_compaction = true; + options.compaction_options_fifo.ttl = 1 * 60 * 60; // 1 hour + options.level0_file_num_compaction_trigger = 6; + options = CurrentOptions(options); + DestroyAndReopen(options); - // Memtable compaction (will succeed) - Flush(); - ASSERT_EQ("bar", Get("foo")); - const int last = dbfull()->MaxMemCompactionLevel(); - ASSERT_EQ(NumTableFilesAtLevel(last), 1); // foo=>bar is now in last level + Random rnd(301); + for (int i = 0; i < 10; i++) { + // Generate and flush a file about 10KB. + for (int j = 0; j < 10; j++) { + ASSERT_OK(Put(ToString(i * 20 + j), RandomString(&rnd, 980))); + } + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + } + // With Intra-L0 compaction, out of 10 files, 6 files will be compacted to 1 + // (due to level0_file_num_compaction_trigger = 6). + // So total files = 1 + remaining 4 = 5. + ASSERT_EQ(NumTableFilesAtLevel(0), 5); - // Merging compaction (will fail) - error_type->Release_Store(env_); - dbfull()->TEST_CompactRange(last, nullptr, nullptr); // Should fail - ASSERT_EQ("bar", Get("foo")); + // Sleep for 2 hours -- which is much greater than TTL. + env_->addon_time_.fetch_add(2 * 60 * 60); + // Just to make sure that we are in the same state even after sleeping. + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + ASSERT_EQ(NumTableFilesAtLevel(0), 5); - // Recovery: should not lose data - error_type->Release_Store(nullptr); - Reopen(&options); - ASSERT_EQ("bar", Get("foo")); + // Create 10 more files. The old 5 files are dropped as their ttl expired. + for (int i = 0; i < 10; i++) { + for (int j = 0; j < 10; j++) { + ASSERT_OK(Put(ToString(i * 20 + j), RandomString(&rnd, 980))); + } + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + } + ASSERT_EQ(NumTableFilesAtLevel(0), 5); + ASSERT_LE(SizeAtLevel(0), + options.compaction_options_fifo.max_table_files_size); } -} - -TEST(DBTest, PutFailsParanoid) { - // Test the following: - // (a) A random put fails in paranoid mode (simulate by sync fail) - // (b) All other puts have to fail, even if writes would succeed - // (c) All of that should happen ONLY if paranoid_checks = true - Options options = CurrentOptions(); - options.env = env_; - options.create_if_missing = true; - options.error_if_exists = false; - options.paranoid_checks = true; - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); - Status s; - - ASSERT_OK(Put(1, "foo", "bar")); - ASSERT_OK(Put(1, "foo1", "bar1")); - // simulate error - env_->log_write_error_.Release_Store(env_); - s = Put(1, "foo2", "bar2"); - ASSERT_TRUE(!s.ok()); - env_->log_write_error_.Release_Store(nullptr); - s = Put(1, "foo3", "bar3"); - // the next put should fail, too - ASSERT_TRUE(!s.ok()); - // but we're still able to read - ASSERT_EQ("bar", Get(1, "foo")); - - // do the same thing with paranoid checks off - options.paranoid_checks = false; - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); + // Test with large TTL + Intra-L0 compactions. + // Files dropped based on size, as ttl doesn't kick in. + { + options.write_buffer_size = 20 << 10; // 20K + options.compaction_options_fifo.max_table_files_size = 1500 << 10; // 1.5MB + options.compaction_options_fifo.allow_compaction = true; + options.compaction_options_fifo.ttl = 1 * 60 * 60; // 1 hour + options.level0_file_num_compaction_trigger = 6; + options = CurrentOptions(options); + DestroyAndReopen(options); - ASSERT_OK(Put(1, "foo", "bar")); - ASSERT_OK(Put(1, "foo1", "bar1")); - // simulate error - env_->log_write_error_.Release_Store(env_); - s = Put(1, "foo2", "bar2"); - ASSERT_TRUE(!s.ok()); - env_->log_write_error_.Release_Store(nullptr); - s = Put(1, "foo3", "bar3"); - // the next put should NOT fail - ASSERT_TRUE(s.ok()); -} + Random rnd(301); + for (int i = 0; i < 60; i++) { + // Generate and flush a file about 20KB. + for (int j = 0; j < 20; j++) { + ASSERT_OK(Put(ToString(i * 20 + j), RandomString(&rnd, 980))); + } + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + } + // It should be compacted to 10 files. + ASSERT_EQ(NumTableFilesAtLevel(0), 10); -TEST(DBTest, FilesDeletedAfterCompaction) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "foo", "v2")); - Compact(1, "a", "z"); - const int num_files = CountLiveFiles(); - for (int i = 0; i < 10; i++) { - ASSERT_OK(Put(1, "foo", "v2")); - Compact(1, "a", "z"); + for (int i = 0; i < 60; i++) { + // Generate and flush a file about 20KB. + for (int j = 0; j < 20; j++) { + ASSERT_OK(Put(ToString(i * 20 + j + 2000), RandomString(&rnd, 980))); + } + Flush(); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); } - ASSERT_EQ(CountLiveFiles(), num_files); - } while (ChangeCompactOptions()); + + // It should be compacted to no more than 20 files. + ASSERT_GT(NumTableFilesAtLevel(0), 10); + ASSERT_LT(NumTableFilesAtLevel(0), 18); + // Size limit is still guaranteed. + ASSERT_LE(SizeAtLevel(0), + options.compaction_options_fifo.max_table_files_size); + } } +#endif // ROCKSDB_LITE -TEST(DBTest, BloomFilter) { - do { - Options options = CurrentOptions(); - env_->count_random_reads_ = true; - options.env = env_; - // ChangeCompactOptions() only changes compaction style, which does not - // trigger reset of table_factory - BlockBasedTableOptions table_options; - table_options.no_block_cache = true; - table_options.filter_policy.reset(NewBloomFilterPolicy(10)); - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); +#ifndef ROCKSDB_LITE +/* + * This test is not reliable enough as it heavily depends on disk behavior. + * Disable as it is flaky. + */ +TEST_F(DBTest, DISABLED_RateLimitingTest) { + Options options = CurrentOptions(); + options.write_buffer_size = 1 << 20; // 1MB + options.level0_file_num_compaction_trigger = 2; + options.target_file_size_base = 1 << 20; // 1MB + options.max_bytes_for_level_base = 4 << 20; // 4MB + options.max_bytes_for_level_multiplier = 4; + options.compression = kNoCompression; + options.create_if_missing = true; + options.env = env_; + options.statistics = rocksdb::CreateDBStatistics(); + options.IncreaseParallelism(4); + DestroyAndReopen(options); - CreateAndReopenWithCF({"pikachu"}, &options); + WriteOptions wo; + wo.disableWAL = true; - // Populate multiple layers - const int N = 10000; - for (int i = 0; i < N; i++) { - ASSERT_OK(Put(1, Key(i), Key(i))); - } - Compact(1, "a", "z"); - for (int i = 0; i < N; i += 100) { - ASSERT_OK(Put(1, Key(i), Key(i))); - } - Flush(1); + // # no rate limiting + Random rnd(301); + uint64_t start = env_->NowMicros(); + // Write ~96M data + for (int64_t i = 0; i < (96 << 10); ++i) { + ASSERT_OK( + Put(RandomString(&rnd, 32), RandomString(&rnd, (1 << 10) + 1), wo)); + } + uint64_t elapsed = env_->NowMicros() - start; + double raw_rate = env_->bytes_written_ * 1000000.0 / elapsed; + uint64_t rate_limiter_drains = + TestGetTickerCount(options, NUMBER_RATE_LIMITER_DRAINS); + ASSERT_EQ(0, rate_limiter_drains); + Close(); - // Prevent auto compactions triggered by seeks - env_->delay_sstable_sync_.Release_Store(env_); + // # rate limiting with 0.7 x threshold + options.rate_limiter.reset( + NewGenericRateLimiter(static_cast(0.7 * raw_rate))); + env_->bytes_written_ = 0; + DestroyAndReopen(options); - // Lookup present keys. Should rarely read from small sstable. - env_->random_read_counter_.Reset(); - for (int i = 0; i < N; i++) { - ASSERT_EQ(Key(i), Get(1, Key(i))); - } - int reads = env_->random_read_counter_.Read(); - fprintf(stderr, "%d present => %d reads\n", N, reads); - ASSERT_GE(reads, N); - ASSERT_LE(reads, N + 2*N/100); + start = env_->NowMicros(); + // Write ~96M data + for (int64_t i = 0; i < (96 << 10); ++i) { + ASSERT_OK( + Put(RandomString(&rnd, 32), RandomString(&rnd, (1 << 10) + 1), wo)); + } + rate_limiter_drains = + TestGetTickerCount(options, NUMBER_RATE_LIMITER_DRAINS) - + rate_limiter_drains; + elapsed = env_->NowMicros() - start; + Close(); + ASSERT_EQ(options.rate_limiter->GetTotalBytesThrough(), env_->bytes_written_); + // Most intervals should've been drained (interval time is 100ms, elapsed is + // micros) + ASSERT_GT(rate_limiter_drains, 0); + ASSERT_LE(rate_limiter_drains, elapsed / 100000 + 1); + double ratio = env_->bytes_written_ * 1000000 / elapsed / raw_rate; + fprintf(stderr, "write rate ratio = %.2lf, expected 0.7\n", ratio); + ASSERT_TRUE(ratio < 0.8); - // Lookup present keys. Should rarely read from either sstable. - env_->random_read_counter_.Reset(); - for (int i = 0; i < N; i++) { - ASSERT_EQ("NOT_FOUND", Get(1, Key(i) + ".missing")); - } - reads = env_->random_read_counter_.Read(); - fprintf(stderr, "%d missing => %d reads\n", N, reads); - ASSERT_LE(reads, 3*N/100); + // # rate limiting with half of the raw_rate + options.rate_limiter.reset( + NewGenericRateLimiter(static_cast(raw_rate / 2))); + env_->bytes_written_ = 0; + DestroyAndReopen(options); - env_->delay_sstable_sync_.Release_Store(nullptr); - Close(); - } while (ChangeCompactOptions()); + start = env_->NowMicros(); + // Write ~96M data + for (int64_t i = 0; i < (96 << 10); ++i) { + ASSERT_OK( + Put(RandomString(&rnd, 32), RandomString(&rnd, (1 << 10) + 1), wo)); + } + elapsed = env_->NowMicros() - start; + rate_limiter_drains = + TestGetTickerCount(options, NUMBER_RATE_LIMITER_DRAINS) - + rate_limiter_drains; + Close(); + ASSERT_EQ(options.rate_limiter->GetTotalBytesThrough(), env_->bytes_written_); + // Most intervals should've been drained (interval time is 100ms, elapsed is + // micros) + ASSERT_GT(rate_limiter_drains, elapsed / 100000 / 2); + ASSERT_LE(rate_limiter_drains, elapsed / 100000 + 1); + ratio = env_->bytes_written_ * 1000000 / elapsed / raw_rate; + fprintf(stderr, "write rate ratio = %.2lf, expected 0.5\n", ratio); + ASSERT_LT(ratio, 0.6); } -TEST(DBTest, SnapshotFiles) { - do { - Options options = CurrentOptions(); - options.write_buffer_size = 100000000; // Large write buffer - CreateAndReopenWithCF({"pikachu"}, &options); +TEST_F(DBTest, TableOptionsSanitizeTest) { + Options options = CurrentOptions(); + options.create_if_missing = true; + DestroyAndReopen(options); + ASSERT_EQ(db_->GetOptions().allow_mmap_reads, false); - Random rnd(301); + options.table_factory.reset(new PlainTableFactory()); + options.prefix_extractor.reset(NewNoopTransform()); + Destroy(options); + ASSERT_TRUE(!TryReopen(options).IsNotSupported()); - // Write 8MB (80 values, each 100K) - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); - std::vector values; - for (int i = 0; i < 80; i++) { - values.push_back(RandomString(&rnd, 100000)); - ASSERT_OK(Put((i < 40), Key(i), values[i])); - } + // Test for check of prefix_extractor when hash index is used for + // block-based table + BlockBasedTableOptions to; + to.index_type = BlockBasedTableOptions::kHashSearch; + options = CurrentOptions(); + options.create_if_missing = true; + options.table_factory.reset(NewBlockBasedTableFactory(to)); + ASSERT_TRUE(TryReopen(options).IsInvalidArgument()); + options.prefix_extractor.reset(NewFixedPrefixTransform(1)); + ASSERT_OK(TryReopen(options)); +} - // assert that nothing makes it to disk yet. - ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); +TEST_F(DBTest, ConcurrentMemtableNotSupported) { + Options options = CurrentOptions(); + options.allow_concurrent_memtable_write = true; + options.soft_pending_compaction_bytes_limit = 0; + options.hard_pending_compaction_bytes_limit = 100; + options.create_if_missing = true; - // get a file snapshot - uint64_t manifest_number = 0; - uint64_t manifest_size = 0; - std::vector files; - dbfull()->DisableFileDeletions(); - dbfull()->GetLiveFiles(files, &manifest_size); + DestroyDB(dbname_, options); + options.memtable_factory.reset(NewHashLinkListRepFactory(4, 0, 3, true, 4)); + ASSERT_NOK(TryReopen(options)); - // CURRENT, MANIFEST, *.sst files (one for each CF) - ASSERT_EQ(files.size(), 4U); + options.memtable_factory.reset(new SkipListFactory); + ASSERT_OK(TryReopen(options)); - uint64_t number = 0; - FileType type; + ColumnFamilyOptions cf_options(options); + cf_options.memtable_factory.reset( + NewHashLinkListRepFactory(4, 0, 3, true, 4)); + ColumnFamilyHandle* handle; + ASSERT_NOK(db_->CreateColumnFamily(cf_options, "name", &handle)); +} - // copy these files to a new snapshot directory - std::string snapdir = dbname_ + ".snapdir/"; - std::string mkdir = "mkdir -p " + snapdir; - ASSERT_EQ(system(mkdir.c_str()), 0); +#endif // ROCKSDB_LITE - for (unsigned int i = 0; i < files.size(); i++) { - // our clients require that GetLiveFiles returns - // files with "/" as first character! - ASSERT_EQ(files[i][0], '/'); - std::string src = dbname_ + files[i]; - std::string dest = snapdir + files[i]; +TEST_F(DBTest, SanitizeNumThreads) { + for (int attempt = 0; attempt < 2; attempt++) { + const size_t kTotalTasks = 8; + test::SleepingBackgroundTask sleeping_tasks[kTotalTasks]; - uint64_t size; - ASSERT_OK(env_->GetFileSize(src, &size)); + Options options = CurrentOptions(); + if (attempt == 0) { + options.max_background_compactions = 3; + options.max_background_flushes = 2; + } + options.create_if_missing = true; + DestroyAndReopen(options); - // record the number and the size of the - // latest manifest file - if (ParseFileName(files[i].substr(1), &number, &type)) { - if (type == kDescriptorFile) { - if (number > manifest_number) { - manifest_number = number; - ASSERT_GE(size, manifest_size); - size = manifest_size; // copy only valid MANIFEST data - } - } - } - CopyFile(src, dest, size); + for (size_t i = 0; i < kTotalTasks; i++) { + // Insert 5 tasks to low priority queue and 5 tasks to high priority queue + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, + &sleeping_tasks[i], + (i < 4) ? Env::Priority::LOW : Env::Priority::HIGH); } - // release file snapshot - dbfull()->DisableFileDeletions(); + // Wait 100 milliseconds for they are scheduled. + env_->SleepForMicroseconds(100000); - // overwrite one key, this key should not appear in the snapshot - std::vector extras; - for (unsigned int i = 0; i < 1; i++) { - extras.push_back(RandomString(&rnd, 100000)); - ASSERT_OK(Put(0, Key(i), extras[i])); + // pool size 3, total task 4. Queue size should be 1. + ASSERT_EQ(1U, options.env->GetThreadPoolQueueLen(Env::Priority::LOW)); + // pool size 2, total task 4. Queue size should be 2. + ASSERT_EQ(2U, options.env->GetThreadPoolQueueLen(Env::Priority::HIGH)); + + for (size_t i = 0; i < kTotalTasks; i++) { + sleeping_tasks[i].WakeUp(); + sleeping_tasks[i].WaitUntilDone(); } - // verify that data in the snapshot are correct - std::vector column_families; - column_families.emplace_back("default", ColumnFamilyOptions()); - column_families.emplace_back("pikachu", ColumnFamilyOptions()); - std::vector cf_handles; - DB* snapdb; - DBOptions opts; - opts.create_if_missing = false; - Status stat = - DB::Open(opts, snapdir, column_families, &cf_handles, &snapdb); - ASSERT_OK(stat); + ASSERT_OK(Put("abc", "def")); + ASSERT_EQ("def", Get("abc")); + Flush(); + ASSERT_EQ("def", Get("abc")); + } +} - ReadOptions roptions; - std::string val; - for (unsigned int i = 0; i < 80; i++) { - stat = snapdb->Get(roptions, cf_handles[i < 40], Key(i), &val); - ASSERT_EQ(values[i].compare(val), 0); - } - for (auto cfh : cf_handles) { - delete cfh; - } - delete snapdb; +TEST_F(DBTest, WriteSingleThreadEntry) { + std::vector threads; + dbfull()->TEST_LockMutex(); + auto w = dbfull()->TEST_BeginWrite(); + threads.emplace_back([&] { Put("a", "b"); }); + env_->SleepForMicroseconds(10000); + threads.emplace_back([&] { Flush(); }); + env_->SleepForMicroseconds(10000); + dbfull()->TEST_UnlockMutex(); + dbfull()->TEST_LockMutex(); + dbfull()->TEST_EndWrite(w); + dbfull()->TEST_UnlockMutex(); - // look at the new live files after we added an 'extra' key - // and after we took the first snapshot. - uint64_t new_manifest_number = 0; - uint64_t new_manifest_size = 0; - std::vector newfiles; - dbfull()->DisableFileDeletions(); - dbfull()->GetLiveFiles(newfiles, &new_manifest_size); + for (auto& t : threads) { + t.join(); + } +} - // find the new manifest file. assert that this manifest file is - // the same one as in the previous snapshot. But its size should be - // larger because we added an extra key after taking the - // previous shapshot. - for (unsigned int i = 0; i < newfiles.size(); i++) { - std::string src = dbname_ + "/" + newfiles[i]; - // record the lognumber and the size of the - // latest manifest file - if (ParseFileName(newfiles[i].substr(1), &number, &type)) { - if (type == kDescriptorFile) { - if (number > new_manifest_number) { - uint64_t size; - new_manifest_number = number; - ASSERT_OK(env_->GetFileSize(src, &size)); - ASSERT_GE(size, new_manifest_size); - } - } +#ifndef ROCKSDB_LITE +TEST_F(DBTest, DynamicMemtableOptions) { + const uint64_t k64KB = 1 << 16; + const uint64_t k128KB = 1 << 17; + const uint64_t k5KB = 5 * 1024; + Options options; + options.env = env_; + options.create_if_missing = true; + options.compression = kNoCompression; + options.max_background_compactions = 1; + options.write_buffer_size = k64KB; + options.arena_block_size = 16 * 1024; + options.max_write_buffer_number = 2; + // Don't trigger compact/slowdown/stop + options.level0_file_num_compaction_trigger = 1024; + options.level0_slowdown_writes_trigger = 1024; + options.level0_stop_writes_trigger = 1024; + DestroyAndReopen(options); + + auto gen_l0_kb = [this](int size) { + const int kNumPutsBeforeWaitForFlush = 64; + Random rnd(301); + for (int i = 0; i < size; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 1024))); + + // The following condition prevents a race condition between flush jobs + // acquiring work and this thread filling up multiple memtables. Without + // this, the flush might produce less files than expected because + // multiple memtables are flushed into a single L0 file. This race + // condition affects assertion (A). + if (i % kNumPutsBeforeWaitForFlush == kNumPutsBeforeWaitForFlush - 1) { + dbfull()->TEST_WaitForFlushMemTable(); } } - ASSERT_EQ(manifest_number, new_manifest_number); - ASSERT_GT(new_manifest_size, manifest_size); + dbfull()->TEST_WaitForFlushMemTable(); + }; - // release file snapshot - dbfull()->DisableFileDeletions(); - } while (ChangeCompactOptions()); -} + // Test write_buffer_size + gen_l0_kb(64); + ASSERT_EQ(NumTableFilesAtLevel(0), 1); + ASSERT_LT(SizeAtLevel(0), k64KB + k5KB); + ASSERT_GT(SizeAtLevel(0), k64KB - k5KB * 2); + + // Clean up L0 + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + + // Increase buffer size + ASSERT_OK(dbfull()->SetOptions({ + {"write_buffer_size", "131072"}, + })); + + // The existing memtable inflated 64KB->128KB when we invoked SetOptions(). + // Write 192KB, we should have a 128KB L0 file and a memtable with 64KB data. + gen_l0_kb(192); + ASSERT_EQ(NumTableFilesAtLevel(0), 1); // (A) + ASSERT_LT(SizeAtLevel(0), k128KB + 2 * k5KB); + ASSERT_GT(SizeAtLevel(0), k128KB - 4 * k5KB); + + // Decrease buffer size below current usage + ASSERT_OK(dbfull()->SetOptions({ + {"write_buffer_size", "65536"}, + })); + // The existing memtable became eligible for flush when we reduced its + // capacity to 64KB. Two keys need to be added to trigger flush: first causes + // memtable to be marked full, second schedules the flush. Then we should have + // a 128KB L0 file, a 64KB L0 file, and a memtable with just one key. + gen_l0_kb(2); + ASSERT_EQ(NumTableFilesAtLevel(0), 2); + ASSERT_LT(SizeAtLevel(0), k128KB + k64KB + 2 * k5KB); + ASSERT_GT(SizeAtLevel(0), k128KB + k64KB - 4 * k5KB); + + // Test max_write_buffer_number + // Block compaction thread, which will also block the flushes because + // max_background_flushes == 0, so flushes are getting executed by the + // compaction thread + env_->SetBackgroundThreads(1, Env::LOW); + test::SleepingBackgroundTask sleeping_task_low; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + // Start from scratch and disable compaction/flush. Flush can only happen + // during compaction but trigger is pretty high + options.disable_auto_compactions = true; + DestroyAndReopen(options); + env_->SetBackgroundThreads(0, Env::HIGH); -TEST(DBTest, CompactOnFlush) { - do { - Options options = CurrentOptions(); - options.purge_redundant_kvs_while_flush = true; - options.disable_auto_compactions = true; - CreateAndReopenWithCF({"pikachu"}, &options); + // Put until writes are stopped, bounded by 256 puts. We should see stop at + // ~128KB + int count = 0; + Random rnd(301); - Put(1, "foo", "v1"); - ASSERT_OK(Flush(1)); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v1 ]"); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::DelayWrite:Wait", + [&](void* arg) { sleeping_task_low.WakeUp(); }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); - // Write two new keys - Put(1, "a", "begin"); - Put(1, "z", "end"); - Flush(1); + while (!sleeping_task_low.WokenUp() && count < 256) { + ASSERT_OK(Put(Key(count), RandomString(&rnd, 1024), WriteOptions())); + count++; + } + ASSERT_GT(static_cast(count), 128 * 0.8); + ASSERT_LT(static_cast(count), 128 * 1.2); - // Case1: Delete followed by a put - Delete(1, "foo"); - Put(1, "foo", "v2"); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, DEL, v1 ]"); + sleeping_task_low.WaitUntilDone(); - // After the current memtable is flushed, the DEL should - // have been removed - ASSERT_OK(Flush(1)); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2, v1 ]"); + // Increase + ASSERT_OK(dbfull()->SetOptions({ + {"max_write_buffer_number", "8"}, + })); + // Clean up memtable and L0 + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); - dbfull()->CompactRange(handles_[1], nullptr, nullptr); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v2 ]"); + sleeping_task_low.Reset(); + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + count = 0; + while (!sleeping_task_low.WokenUp() && count < 1024) { + ASSERT_OK(Put(Key(count), RandomString(&rnd, 1024), WriteOptions())); + count++; + } +// Windows fails this test. Will tune in the future and figure out +// approp number +#ifndef OS_WIN + ASSERT_GT(static_cast(count), 512 * 0.8); + ASSERT_LT(static_cast(count), 512 * 1.2); +#endif + sleeping_task_low.WaitUntilDone(); - // Case 2: Delete followed by another delete - Delete(1, "foo"); - Delete(1, "foo"); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, DEL, v2 ]"); - ASSERT_OK(Flush(1)); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, v2 ]"); - dbfull()->CompactRange(handles_[1], nullptr, nullptr); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ ]"); - - // Case 3: Put followed by a delete - Put(1, "foo", "v3"); - Delete(1, "foo"); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL, v3 ]"); - ASSERT_OK(Flush(1)); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ DEL ]"); - dbfull()->CompactRange(handles_[1], nullptr, nullptr); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ ]"); - - // Case 4: Put followed by another Put - Put(1, "foo", "v4"); - Put(1, "foo", "v5"); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v5, v4 ]"); - ASSERT_OK(Flush(1)); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v5 ]"); - dbfull()->CompactRange(handles_[1], nullptr, nullptr); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v5 ]"); - - // clear database - Delete(1, "foo"); - dbfull()->CompactRange(handles_[1], nullptr, nullptr); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ ]"); - - // Case 5: Put followed by snapshot followed by another Put - // Both puts should remain. - Put(1, "foo", "v6"); - const Snapshot* snapshot = db_->GetSnapshot(); - Put(1, "foo", "v7"); - ASSERT_OK(Flush(1)); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v7, v6 ]"); - db_->ReleaseSnapshot(snapshot); + // Decrease + ASSERT_OK(dbfull()->SetOptions({ + {"max_write_buffer_number", "4"}, + })); + // Clean up memtable and L0 + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); - // clear database - Delete(1, "foo"); - dbfull()->CompactRange(handles_[1], nullptr, nullptr); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ ]"); + sleeping_task_low.Reset(); + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); - // Case 5: snapshot followed by a put followed by another Put - // Only the last put should remain. - const Snapshot* snapshot1 = db_->GetSnapshot(); - Put(1, "foo", "v8"); - Put(1, "foo", "v9"); - ASSERT_OK(Flush(1)); - ASSERT_EQ(AllEntriesFor("foo", 1), "[ v9 ]"); - db_->ReleaseSnapshot(snapshot1); - } while (ChangeCompactOptions()); + count = 0; + while (!sleeping_task_low.WokenUp() && count < 1024) { + ASSERT_OK(Put(Key(count), RandomString(&rnd, 1024), WriteOptions())); + count++; + } +// Windows fails this test. Will tune in the future and figure out +// approp number +#ifndef OS_WIN + ASSERT_GT(static_cast(count), 256 * 0.8); + ASSERT_LT(static_cast(count), 266 * 1.2); +#endif + sleeping_task_low.WaitUntilDone(); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); } +#endif // ROCKSDB_LITE +#ifdef ROCKSDB_USING_THREAD_STATUS namespace { -std::vector ListSpecificFiles( - Env* env, const std::string& path, const FileType expected_file_type) { - std::vector files; - std::vector log_files; - env->GetChildren(path, &files); - uint64_t number; - FileType type; - for (size_t i = 0; i < files.size(); ++i) { - if (ParseFileName(files[i], &number, &type)) { - if (type == expected_file_type) { - log_files.push_back(number); - } +void VerifyOperationCount(Env* env, ThreadStatus::OperationType op_type, + int expected_count) { + int op_count = 0; + std::vector thread_list; + ASSERT_OK(env->GetThreadList(&thread_list)); + for (auto thread : thread_list) { + if (thread.operation_type == op_type) { + op_count++; } } - return std::move(log_files); -} - -std::vector ListLogFiles(Env* env, const std::string& path) { - return ListSpecificFiles(env, path, kLogFile); -} - -std::vector ListTableFiles(Env* env, const std::string& path) { - return ListSpecificFiles(env, path, kTableFile); + ASSERT_EQ(op_count, expected_count); } } // namespace -TEST(DBTest, FlushOneColumnFamily) { +TEST_F(DBTest, GetThreadStatus) { Options options; - CreateAndReopenWithCF({"pikachu", "ilya", "muromec", "dobrynia", "nikitich", - "alyosha", "popovich"}, - &options); - - ASSERT_OK(Put(0, "Default", "Default")); - ASSERT_OK(Put(1, "pikachu", "pikachu")); - ASSERT_OK(Put(2, "ilya", "ilya")); - ASSERT_OK(Put(3, "muromec", "muromec")); - ASSERT_OK(Put(4, "dobrynia", "dobrynia")); - ASSERT_OK(Put(5, "nikitich", "nikitich")); - ASSERT_OK(Put(6, "alyosha", "alyosha")); - ASSERT_OK(Put(7, "popovich", "popovich")); - - for (size_t i = 0; i < 8; ++i) { - Flush(i); - auto tables = ListTableFiles(env_, dbname_); - ASSERT_EQ(tables.size(), i + 1U); - } -} - -TEST(DBTest, WALArchivalTtl) { - do { - Options options = CurrentOptions(); - options.create_if_missing = true; - options.WAL_ttl_seconds = 1000; - DestroyAndReopen(&options); - - // TEST : Create DB with a ttl and no size limit. - // Put some keys. Count the log files present in the DB just after insert. - // Re-open db. Causes deletion/archival to take place. - // Assert that the files moved under "/archive". - // Reopen db with small ttl. - // Assert that archive was removed. + options.env = env_; + options.enable_thread_tracking = true; + TryReopen(options); - std::string archiveDir = ArchivalDirectory(dbname_); + std::vector thread_list; + Status s = env_->GetThreadList(&thread_list); - for (int i = 0; i < 10; ++i) { - for (int j = 0; j < 10; ++j) { - ASSERT_OK(Put(Key(10 * i + j), DummyString(1024))); + for (int i = 0; i < 2; ++i) { + // repeat the test with differet number of high / low priority threads + const int kTestCount = 3; + const unsigned int kHighPriCounts[kTestCount] = {3, 2, 5}; + const unsigned int kLowPriCounts[kTestCount] = {10, 15, 3}; + for (int test = 0; test < kTestCount; ++test) { + // Change the number of threads in high / low priority pool. + env_->SetBackgroundThreads(kHighPriCounts[test], Env::HIGH); + env_->SetBackgroundThreads(kLowPriCounts[test], Env::LOW); + // Wait to ensure the all threads has been registered + unsigned int thread_type_counts[ThreadStatus::NUM_THREAD_TYPES]; + // Try up to 60 seconds. + for (int num_try = 0; num_try < 60000; num_try++) { + env_->SleepForMicroseconds(1000); + thread_list.clear(); + s = env_->GetThreadList(&thread_list); + ASSERT_OK(s); + memset(thread_type_counts, 0, sizeof(thread_type_counts)); + for (auto thread : thread_list) { + ASSERT_LT(thread.thread_type, ThreadStatus::NUM_THREAD_TYPES); + thread_type_counts[thread.thread_type]++; + } + if (thread_type_counts[ThreadStatus::HIGH_PRIORITY] == + kHighPriCounts[test] && + thread_type_counts[ThreadStatus::LOW_PRIORITY] == + kLowPriCounts[test]) { + break; + } } + // Verify the total number of threades + ASSERT_EQ(thread_type_counts[ThreadStatus::HIGH_PRIORITY] + + thread_type_counts[ThreadStatus::LOW_PRIORITY], + kHighPriCounts[test] + kLowPriCounts[test]); + // Verify the number of high-priority threads + ASSERT_EQ(thread_type_counts[ThreadStatus::HIGH_PRIORITY], + kHighPriCounts[test]); + // Verify the number of low-priority threads + ASSERT_EQ(thread_type_counts[ThreadStatus::LOW_PRIORITY], + kLowPriCounts[test]); + } + if (i == 0) { + // repeat the test with multiple column families + CreateAndReopenWithCF({"pikachu", "about-to-remove"}, options); + env_->GetThreadStatusUpdater()->TEST_VerifyColumnFamilyInfoMap(handles_, + true); + } + } + db_->DropColumnFamily(handles_[2]); + delete handles_[2]; + handles_.erase(handles_.begin() + 2); + env_->GetThreadStatusUpdater()->TEST_VerifyColumnFamilyInfoMap(handles_, + true); + Close(); + env_->GetThreadStatusUpdater()->TEST_VerifyColumnFamilyInfoMap(handles_, + true); +} - std::vector log_files = ListLogFiles(env_, dbname_); +TEST_F(DBTest, DisableThreadStatus) { + Options options; + options.env = env_; + options.enable_thread_tracking = false; + TryReopen(options); + CreateAndReopenWithCF({"pikachu", "about-to-remove"}, options); + // Verify non of the column family info exists + env_->GetThreadStatusUpdater()->TEST_VerifyColumnFamilyInfoMap(handles_, + false); +} - options.create_if_missing = false; - Reopen(&options); +TEST_F(DBTest, ThreadStatusFlush) { + Options options; + options.env = env_; + options.write_buffer_size = 100000; // Small write buffer + options.enable_thread_tracking = true; + options = CurrentOptions(options); - std::vector logs = ListLogFiles(env_, archiveDir); - std::set archivedFiles(logs.begin(), logs.end()); + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"FlushJob::FlushJob()", "DBTest::ThreadStatusFlush:1"}, + {"DBTest::ThreadStatusFlush:2", "FlushJob::WriteLevel0Table"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + CreateAndReopenWithCF({"pikachu"}, options); + VerifyOperationCount(env_, ThreadStatus::OP_FLUSH, 0); + + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_EQ("v1", Get(1, "foo")); + VerifyOperationCount(env_, ThreadStatus::OP_FLUSH, 0); + + uint64_t num_running_flushes = 0; + db_->GetIntProperty(DB::Properties::kNumRunningFlushes, &num_running_flushes); + ASSERT_EQ(num_running_flushes, 0); + + Put(1, "k1", std::string(100000, 'x')); // Fill memtable + Put(1, "k2", std::string(100000, 'y')); // Trigger flush + + // The first sync point is to make sure there's one flush job + // running when we perform VerifyOperationCount(). + TEST_SYNC_POINT("DBTest::ThreadStatusFlush:1"); + VerifyOperationCount(env_, ThreadStatus::OP_FLUSH, 1); + db_->GetIntProperty(DB::Properties::kNumRunningFlushes, &num_running_flushes); + ASSERT_EQ(num_running_flushes, 1); + // This second sync point is to ensure the flush job will not + // be completed until we already perform VerifyOperationCount(). + TEST_SYNC_POINT("DBTest::ThreadStatusFlush:2"); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_P(DBTestWithParam, ThreadStatusSingleCompaction) { + const int kTestKeySize = 16; + const int kTestValueSize = 984; + const int kEntrySize = kTestKeySize + kTestValueSize; + const int kEntriesPerBuffer = 100; + Options options; + options.create_if_missing = true; + options.write_buffer_size = kEntrySize * kEntriesPerBuffer; + options.compaction_style = kCompactionStyleLevel; + options.target_file_size_base = options.write_buffer_size; + options.max_bytes_for_level_base = options.target_file_size_base * 2; + options.max_bytes_for_level_multiplier = 2; + options.compression = kNoCompression; + options = CurrentOptions(options); + options.env = env_; + options.enable_thread_tracking = true; + const int kNumL0Files = 4; + options.level0_file_num_compaction_trigger = kNumL0Files; + options.max_subcompactions = max_subcompactions_; + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"DBTest::ThreadStatusSingleCompaction:0", "DBImpl::BGWorkCompaction"}, + {"CompactionJob::Run():Start", "DBTest::ThreadStatusSingleCompaction:1"}, + {"DBTest::ThreadStatusSingleCompaction:2", "CompactionJob::Run():End"}, + }); + for (int tests = 0; tests < 2; ++tests) { + DestroyAndReopen(options); + rocksdb::SyncPoint::GetInstance()->ClearTrace(); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); - for (auto& log : log_files) { - ASSERT_TRUE(archivedFiles.find(log) != archivedFiles.end()); + Random rnd(301); + // The Put Phase. + for (int file = 0; file < kNumL0Files; ++file) { + for (int key = 0; key < kEntriesPerBuffer; ++key) { + ASSERT_OK(Put(ToString(key + file * kEntriesPerBuffer), + RandomString(&rnd, kTestValueSize))); } + Flush(); + } + // This makes sure a compaction won't be scheduled until + // we have done with the above Put Phase. + uint64_t num_running_compactions = 0; + db_->GetIntProperty(DB::Properties::kNumRunningCompactions, + &num_running_compactions); + ASSERT_EQ(num_running_compactions, 0); + TEST_SYNC_POINT("DBTest::ThreadStatusSingleCompaction:0"); + ASSERT_GE(NumTableFilesAtLevel(0), + options.level0_file_num_compaction_trigger); + + // This makes sure at least one compaction is running. + TEST_SYNC_POINT("DBTest::ThreadStatusSingleCompaction:1"); + + if (options.enable_thread_tracking) { + // expecting one single L0 to L1 compaction + VerifyOperationCount(env_, ThreadStatus::OP_COMPACTION, 1); + } else { + // If thread tracking is not enabled, compaction count should be 0. + VerifyOperationCount(env_, ThreadStatus::OP_COMPACTION, 0); } + db_->GetIntProperty(DB::Properties::kNumRunningCompactions, + &num_running_compactions); + ASSERT_EQ(num_running_compactions, 1); + // TODO(yhchiang): adding assert to verify each compaction stage. + TEST_SYNC_POINT("DBTest::ThreadStatusSingleCompaction:2"); - std::vector log_files = ListLogFiles(env_, archiveDir); - ASSERT_TRUE(log_files.size() > 0); + // repeat the test with disabling thread tracking. + options.enable_thread_tracking = false; + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + } +} - options.WAL_ttl_seconds = 1; - env_->SleepForMicroseconds(2 * 1000 * 1000); - Reopen(&options); +TEST_P(DBTestWithParam, PreShutdownManualCompaction) { + Options options = CurrentOptions(); + options.max_subcompactions = max_subcompactions_; + CreateAndReopenWithCF({"pikachu"}, options); - log_files = ListLogFiles(env_, archiveDir); - ASSERT_TRUE(log_files.empty()); - } while (ChangeCompactOptions()); -} + // iter - 0 with 7 levels + // iter - 1 with 3 levels + for (int iter = 0; iter < 2; ++iter) { + MakeTables(3, "p", "q", 1); + ASSERT_EQ("1,1,1", FilesPerLevel(1)); -namespace { -uint64_t GetLogDirSize(std::string dir_path, SpecialEnv* env) { - uint64_t dir_size = 0; - std::vector files; - env->GetChildren(dir_path, &files); - for (auto& f : files) { - uint64_t number; - FileType type; - if (ParseFileName(f, &number, &type) && type == kLogFile) { - std::string const file_path = dir_path + "/" + f; - uint64_t file_size; - env->GetFileSize(file_path, &file_size); - dir_size += file_size; - } - } - return dir_size; -} -} // namespace + // Compaction range falls before files + Compact(1, "", "c"); + ASSERT_EQ("1,1,1", FilesPerLevel(1)); -TEST(DBTest, WALArchivalSizeLimit) { - do { - Options options = CurrentOptions(); - options.create_if_missing = true; - options.WAL_ttl_seconds = 0; - options.WAL_size_limit_MB = 1000; - - // TEST : Create DB with huge size limit and no ttl. - // Put some keys. Count the archived log files present in the DB - // just after insert. Assert that there are many enough. - // Change size limit. Re-open db. - // Assert that archive is not greater than WAL_size_limit_MB. - // Set ttl and time_to_check_ to small values. Re-open db. - // Assert that there are no archived logs left. - - DestroyAndReopen(&options); - for (int i = 0; i < 128 * 128; ++i) { - ASSERT_OK(Put(Key(i), DummyString(1024))); - } - Reopen(&options); + // Compaction range falls after files + Compact(1, "r", "z"); + ASSERT_EQ("1,1,1", FilesPerLevel(1)); - std::string archive_dir = ArchivalDirectory(dbname_); - std::vector log_files = ListLogFiles(env_, archive_dir); - ASSERT_TRUE(log_files.size() > 2); + // Compaction range overlaps files + Compact(1, "p1", "p9"); + ASSERT_EQ("0,0,1", FilesPerLevel(1)); - options.WAL_size_limit_MB = 8; - Reopen(&options); - dbfull()->TEST_PurgeObsoleteteWAL(); + // Populate a different range + MakeTables(3, "c", "e", 1); + ASSERT_EQ("1,1,2", FilesPerLevel(1)); - uint64_t archive_size = GetLogDirSize(archive_dir, env_); - ASSERT_TRUE(archive_size <= options.WAL_size_limit_MB * 1024 * 1024); + // Compact just the new range + Compact(1, "b", "f"); + ASSERT_EQ("0,0,2", FilesPerLevel(1)); - options.WAL_ttl_seconds = 1; - dbfull()->TEST_SetDefaultTimeToCheck(1); - env_->SleepForMicroseconds(2 * 1000 * 1000); - Reopen(&options); - dbfull()->TEST_PurgeObsoleteteWAL(); + // Compact all + MakeTables(1, "a", "z", 1); + ASSERT_EQ("1,0,2", FilesPerLevel(1)); + CancelAllBackgroundWork(db_); + db_->CompactRange(CompactRangeOptions(), handles_[1], nullptr, nullptr); + ASSERT_EQ("1,0,2", FilesPerLevel(1)); - log_files = ListLogFiles(env_, archive_dir); - ASSERT_TRUE(log_files.empty()); - } while (ChangeCompactOptions()); + if (iter == 0) { + options = CurrentOptions(); + options.num_levels = 3; + options.create_if_missing = true; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + } + } } -TEST(DBTest, PurgeInfoLogs) { +TEST_F(DBTest, PreShutdownFlush) { Options options = CurrentOptions(); - options.keep_log_file_num = 5; + CreateAndReopenWithCF({"pikachu"}, options); + ASSERT_OK(Put(1, "key", "value")); + CancelAllBackgroundWork(db_); + Status s = + db_->CompactRange(CompactRangeOptions(), handles_[1], nullptr, nullptr); + ASSERT_TRUE(s.IsShutdownInProgress()); +} + +TEST_P(DBTestWithParam, PreShutdownMultipleCompaction) { + const int kTestKeySize = 16; + const int kTestValueSize = 984; + const int kEntrySize = kTestKeySize + kTestValueSize; + const int kEntriesPerBuffer = 40; + const int kNumL0Files = 4; + + const int kHighPriCount = 3; + const int kLowPriCount = 5; + env_->SetBackgroundThreads(kHighPriCount, Env::HIGH); + env_->SetBackgroundThreads(kLowPriCount, Env::LOW); + + Options options; options.create_if_missing = true; - for (int mode = 0; mode <= 1; mode++) { - if (mode == 1) { - options.db_log_dir = dbname_ + "_logs"; - env_->CreateDirIfMissing(options.db_log_dir); - } else { - options.db_log_dir = ""; - } - for (int i = 0; i < 8; i++) { - Reopen(&options); - } + options.write_buffer_size = kEntrySize * kEntriesPerBuffer; + options.compaction_style = kCompactionStyleLevel; + options.target_file_size_base = options.write_buffer_size; + options.max_bytes_for_level_base = + options.target_file_size_base * kNumL0Files; + options.compression = kNoCompression; + options = CurrentOptions(options); + options.env = env_; + options.enable_thread_tracking = true; + options.level0_file_num_compaction_trigger = kNumL0Files; + options.max_bytes_for_level_multiplier = 2; + options.max_background_compactions = kLowPriCount; + options.level0_stop_writes_trigger = 1 << 10; + options.level0_slowdown_writes_trigger = 1 << 10; + options.max_subcompactions = max_subcompactions_; - std::vector files; - env_->GetChildren(options.db_log_dir.empty() ? dbname_ : options.db_log_dir, - &files); - int info_log_count = 0; - for (std::string file : files) { - if (file.find("LOG") != std::string::npos) { - info_log_count++; - } - } - ASSERT_EQ(5, info_log_count); + TryReopen(options); + Random rnd(301); - Destroy(&options); - // For mode (1), test DestoryDB() to delete all the logs under DB dir. - // For mode (2), no info log file should have been put under DB dir. - std::vector db_files; - env_->GetChildren(dbname_, &db_files); - for (std::string file : db_files) { - ASSERT_TRUE(file.find("LOG") == std::string::npos); + std::vector thread_list; + // Delay both flush and compaction + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"FlushJob::FlushJob()", "CompactionJob::Run():Start"}, + {"CompactionJob::Run():Start", + "DBTest::PreShutdownMultipleCompaction:Preshutdown"}, + {"CompactionJob::Run():Start", + "DBTest::PreShutdownMultipleCompaction:VerifyCompaction"}, + {"DBTest::PreShutdownMultipleCompaction:Preshutdown", + "CompactionJob::Run():End"}, + {"CompactionJob::Run():End", + "DBTest::PreShutdownMultipleCompaction:VerifyPreshutdown"}}); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Make rocksdb busy + int key = 0; + // check how many threads are doing compaction using GetThreadList + int operation_count[ThreadStatus::NUM_OP_TYPES] = {0}; + for (int file = 0; file < 16 * kNumL0Files; ++file) { + for (int k = 0; k < kEntriesPerBuffer; ++k) { + ASSERT_OK(Put(ToString(key++), RandomString(&rnd, kTestValueSize))); + } + + Status s = env_->GetThreadList(&thread_list); + for (auto thread : thread_list) { + operation_count[thread.operation_type]++; + } + + // Speed up the test + if (operation_count[ThreadStatus::OP_FLUSH] > 1 && + operation_count[ThreadStatus::OP_COMPACTION] > + 0.6 * options.max_background_compactions) { + break; } - - if (mode == 1) { - // Cleaning up - env_->GetChildren(options.db_log_dir, &files); - for (std::string file : files) { - env_->DeleteFile(options.db_log_dir + "/" + file); - } - env_->DeleteDir(options.db_log_dir); + if (file == 15 * kNumL0Files) { + TEST_SYNC_POINT("DBTest::PreShutdownMultipleCompaction:Preshutdown"); } } -} -namespace { -SequenceNumber ReadRecords( - std::unique_ptr& iter, - int& count) { - count = 0; - SequenceNumber lastSequence = 0; - BatchResult res; - while (iter->Valid()) { - res = iter->GetBatch(); - ASSERT_TRUE(res.sequence > lastSequence); - ++count; - lastSequence = res.sequence; - ASSERT_OK(iter->status()); - iter->Next(); + TEST_SYNC_POINT("DBTest::PreShutdownMultipleCompaction:Preshutdown"); + ASSERT_GE(operation_count[ThreadStatus::OP_COMPACTION], 1); + CancelAllBackgroundWork(db_); + TEST_SYNC_POINT("DBTest::PreShutdownMultipleCompaction:VerifyPreshutdown"); + dbfull()->TEST_WaitForCompact(); + // Record the number of compactions at a time. + for (int i = 0; i < ThreadStatus::NUM_OP_TYPES; ++i) { + operation_count[i] = 0; + } + Status s = env_->GetThreadList(&thread_list); + for (auto thread : thread_list) { + operation_count[thread.operation_type]++; } - return res.sequence; + ASSERT_EQ(operation_count[ThreadStatus::OP_COMPACTION], 0); } -void ExpectRecords( - const int expected_no_records, - std::unique_ptr& iter) { - int num_records; - ReadRecords(iter, num_records); - ASSERT_EQ(num_records, expected_no_records); -} -} // namespace +TEST_P(DBTestWithParam, PreShutdownCompactionMiddle) { + const int kTestKeySize = 16; + const int kTestValueSize = 984; + const int kEntrySize = kTestKeySize + kTestValueSize; + const int kEntriesPerBuffer = 40; + const int kNumL0Files = 4; -TEST(DBTest, TransactionLogIterator) { - do { - Options options = OptionsForLogIterTest(); - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); - Put(0, "key1", DummyString(1024)); - Put(1, "key2", DummyString(1024)); - Put(1, "key2", DummyString(1024)); - ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 3U); - { - auto iter = OpenTransactionLogIter(0); - ExpectRecords(3, iter); - } - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - env_->SleepForMicroseconds(2 * 1000 * 1000); - { - Put(0, "key4", DummyString(1024)); - Put(1, "key5", DummyString(1024)); - Put(0, "key6", DummyString(1024)); - } - { - auto iter = OpenTransactionLogIter(0); - ExpectRecords(6, iter); - } - } while (ChangeCompactOptions()); -} + const int kHighPriCount = 3; + const int kLowPriCount = 5; + env_->SetBackgroundThreads(kHighPriCount, Env::HIGH); + env_->SetBackgroundThreads(kLowPriCount, Env::LOW); -#ifndef NDEBUG // sync point is not included with DNDEBUG build -TEST(DBTest, TransactionLogIteratorRace) { - static const int LOG_ITERATOR_RACE_TEST_COUNT = 2; - static const char* sync_points[LOG_ITERATOR_RACE_TEST_COUNT][4] = - { { "DBImpl::GetSortedWalFiles:1", "DBImpl::PurgeObsoleteFiles:1", - "DBImpl::PurgeObsoleteFiles:2", "DBImpl::GetSortedWalFiles:2" }, - { "DBImpl::GetSortedWalsOfType:1", "DBImpl::PurgeObsoleteFiles:1", - "DBImpl::PurgeObsoleteFiles:2", "DBImpl::GetSortedWalsOfType:2" }}; - for (int test = 0; test < LOG_ITERATOR_RACE_TEST_COUNT; ++test) { - // Setup sync point dependency to reproduce the race condition of - // a log file moved to archived dir, in the middle of GetSortedWalFiles - rocksdb::SyncPoint::GetInstance()->LoadDependency( - { { sync_points[test][0], sync_points[test][1] }, - { sync_points[test][2], sync_points[test][3] }, - }); + Options options; + options.create_if_missing = true; + options.write_buffer_size = kEntrySize * kEntriesPerBuffer; + options.compaction_style = kCompactionStyleLevel; + options.target_file_size_base = options.write_buffer_size; + options.max_bytes_for_level_base = + options.target_file_size_base * kNumL0Files; + options.compression = kNoCompression; + options = CurrentOptions(options); + options.env = env_; + options.enable_thread_tracking = true; + options.level0_file_num_compaction_trigger = kNumL0Files; + options.max_bytes_for_level_multiplier = 2; + options.max_background_compactions = kLowPriCount; + options.level0_stop_writes_trigger = 1 << 10; + options.level0_slowdown_writes_trigger = 1 << 10; + options.max_subcompactions = max_subcompactions_; - do { - rocksdb::SyncPoint::GetInstance()->ClearTrace(); - rocksdb::SyncPoint::GetInstance()->DisableProcessing(); - Options options = OptionsForLogIterTest(); - DestroyAndReopen(&options); - Put("key1", DummyString(1024)); - dbfull()->Flush(FlushOptions()); - Put("key2", DummyString(1024)); - dbfull()->Flush(FlushOptions()); - Put("key3", DummyString(1024)); - dbfull()->Flush(FlushOptions()); - Put("key4", DummyString(1024)); - ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 4U); - - { - auto iter = OpenTransactionLogIter(0); - ExpectRecords(4, iter); - } + TryReopen(options); + Random rnd(301); - rocksdb::SyncPoint::GetInstance()->EnableProcessing(); - // trigger async flush, and log move. Well, log move will - // wait until the GetSortedWalFiles:1 to reproduce the race - // condition - FlushOptions flush_options; - flush_options.wait = false; - dbfull()->Flush(flush_options); - - // "key5" would be written in a new memtable and log - Put("key5", DummyString(1024)); - { - // this iter would miss "key4" if not fixed - auto iter = OpenTransactionLogIter(0); - ExpectRecords(5, iter); - } - } while (ChangeCompactOptions()); + std::vector thread_list; + // Delay both flush and compaction + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBTest::PreShutdownCompactionMiddle:Preshutdown", + "CompactionJob::Run():Inprogress"}, + {"CompactionJob::Run():Start", + "DBTest::PreShutdownCompactionMiddle:VerifyCompaction"}, + {"CompactionJob::Run():Inprogress", "CompactionJob::Run():End"}, + {"CompactionJob::Run():End", + "DBTest::PreShutdownCompactionMiddle:VerifyPreshutdown"}}); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Make rocksdb busy + int key = 0; + // check how many threads are doing compaction using GetThreadList + int operation_count[ThreadStatus::NUM_OP_TYPES] = {0}; + for (int file = 0; file < 16 * kNumL0Files; ++file) { + for (int k = 0; k < kEntriesPerBuffer; ++k) { + ASSERT_OK(Put(ToString(key++), RandomString(&rnd, kTestValueSize))); + } + + Status s = env_->GetThreadList(&thread_list); + for (auto thread : thread_list) { + operation_count[thread.operation_type]++; + } + + // Speed up the test + if (operation_count[ThreadStatus::OP_FLUSH] > 1 && + operation_count[ThreadStatus::OP_COMPACTION] > + 0.6 * options.max_background_compactions) { + break; + } + if (file == 15 * kNumL0Files) { + TEST_SYNC_POINT("DBTest::PreShutdownCompactionMiddle:VerifyCompaction"); + } } -} -#endif - -TEST(DBTest, TransactionLogIteratorMoveOverZeroFiles) { - do { - Options options = OptionsForLogIterTest(); - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); - // Do a plain Reopen. - Put(1, "key1", DummyString(1024)); - // Two reopens should create a zero record WAL file. - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - - Put(1, "key2", DummyString(1024)); - - auto iter = OpenTransactionLogIter(0); - ExpectRecords(2, iter); - } while (ChangeCompactOptions()); -} - -TEST(DBTest, TransactionLogIteratorStallAtLastRecord) { - do { - Options options = OptionsForLogIterTest(); - DestroyAndReopen(&options); - Put("key1", DummyString(1024)); - auto iter = OpenTransactionLogIter(0); - ASSERT_OK(iter->status()); - ASSERT_TRUE(iter->Valid()); - iter->Next(); - ASSERT_TRUE(!iter->Valid()); - ASSERT_OK(iter->status()); - Put("key2", DummyString(1024)); - iter->Next(); - ASSERT_OK(iter->status()); - ASSERT_TRUE(iter->Valid()); - } while (ChangeCompactOptions()); -} - -TEST(DBTest, TransactionLogIteratorJustEmptyFile) { - do { - Options options = OptionsForLogIterTest(); - DestroyAndReopen(&options); - unique_ptr iter; - Status status = dbfull()->GetUpdatesSince(0, &iter); - // Check that an empty iterator is returned - ASSERT_TRUE(!iter->Valid()); - } while (ChangeCompactOptions()); -} -TEST(DBTest, TransactionLogIteratorCheckAfterRestart) { - do { - Options options = OptionsForLogIterTest(); - DestroyAndReopen(&options); - Put("key1", DummyString(1024)); - Put("key2", DummyString(1023)); - dbfull()->Flush(FlushOptions()); - Reopen(&options); - auto iter = OpenTransactionLogIter(0); - ExpectRecords(2, iter); - } while (ChangeCompactOptions()); + ASSERT_GE(operation_count[ThreadStatus::OP_COMPACTION], 1); + CancelAllBackgroundWork(db_); + TEST_SYNC_POINT("DBTest::PreShutdownCompactionMiddle:Preshutdown"); + TEST_SYNC_POINT("DBTest::PreShutdownCompactionMiddle:VerifyPreshutdown"); + dbfull()->TEST_WaitForCompact(); + // Record the number of compactions at a time. + for (int i = 0; i < ThreadStatus::NUM_OP_TYPES; ++i) { + operation_count[i] = 0; + } + Status s = env_->GetThreadList(&thread_list); + for (auto thread : thread_list) { + operation_count[thread.operation_type]++; + } + ASSERT_EQ(operation_count[ThreadStatus::OP_COMPACTION], 0); } -TEST(DBTest, TransactionLogIteratorCorruptedLog) { - do { - Options options = OptionsForLogIterTest(); - DestroyAndReopen(&options); - for (int i = 0; i < 1024; i++) { - Put("key"+std::to_string(i), DummyString(10)); - } - dbfull()->Flush(FlushOptions()); - // Corrupt this log to create a gap - rocksdb::VectorLogPtr wal_files; - ASSERT_OK(dbfull()->GetSortedWalFiles(wal_files)); - const auto logfilePath = dbname_ + "/" + wal_files.front()->PathName(); - ASSERT_EQ( - 0, - truncate(logfilePath.c_str(), wal_files.front()->SizeFileBytes() / 2)); - // Insert a new entry to a new log file - Put("key1025", DummyString(10)); - // Try to read from the beginning. Should stop before the gap and read less - // than 1025 entries - auto iter = OpenTransactionLogIter(0); - int count; - int last_sequence_read = ReadRecords(iter, count); - ASSERT_LT(last_sequence_read, 1025); - // Try to read past the gap, should be able to seek to key1025 - auto iter2 = OpenTransactionLogIter(last_sequence_read + 1); - ExpectRecords(1, iter2); - } while (ChangeCompactOptions()); -} +#endif // ROCKSDB_USING_THREAD_STATUS -TEST(DBTest, TransactionLogIteratorBatchOperations) { - do { - Options options = OptionsForLogIterTest(); - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); - WriteBatch batch; - batch.Put(handles_[1], "key1", DummyString(1024)); - batch.Put(handles_[0], "key2", DummyString(1024)); - batch.Put(handles_[1], "key3", DummyString(1024)); - batch.Delete(handles_[0], "key2"); - dbfull()->Write(WriteOptions(), &batch); - Flush(1); - Flush(0); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - Put(1, "key4", DummyString(1024)); - auto iter = OpenTransactionLogIter(3); - ExpectRecords(2, iter); - } while (ChangeCompactOptions()); +#ifndef ROCKSDB_LITE +TEST_F(DBTest, FlushOnDestroy) { + WriteOptions wo; + wo.disableWAL = true; + ASSERT_OK(Put("foo", "v1", wo)); + CancelAllBackgroundWork(db_); } -TEST(DBTest, TransactionLogIteratorBlobs) { - Options options = OptionsForLogIterTest(); - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); - { - WriteBatch batch; - batch.Put(handles_[1], "key1", DummyString(1024)); - batch.Put(handles_[0], "key2", DummyString(1024)); - batch.PutLogData(Slice("blob1")); - batch.Put(handles_[1], "key3", DummyString(1024)); - batch.PutLogData(Slice("blob2")); - batch.Delete(handles_[0], "key2"); - dbfull()->Write(WriteOptions(), &batch); - ReopenWithColumnFamilies({"default", "pikachu"}, &options); - } - - auto res = OpenTransactionLogIter(0)->GetBatch(); - struct Handler : public WriteBatch::Handler { - std::string seen; - virtual Status PutCF(uint32_t cf, const Slice& key, const Slice& value) { - seen += "Put(" + std::to_string(cf) + ", " + key.ToString() + ", " + - std::to_string(value.size()) + ")"; - return Status::OK(); - } - virtual Status MergeCF(uint32_t cf, const Slice& key, const Slice& value) { - seen += "Merge(" + std::to_string(cf) + ", " + key.ToString() + ", " + - std::to_string(value.size()) + ")"; - return Status::OK(); - } - virtual void LogData(const Slice& blob) { - seen += "LogData(" + blob.ToString() + ")"; - } - virtual Status DeleteCF(uint32_t cf, const Slice& key) { - seen += "Delete(" + std::to_string(cf) + ", " + key.ToString() + ")"; - return Status::OK(); - } - } handler; - res.writeBatchPtr->Iterate(&handler); - ASSERT_EQ( - "Put(1, key1, 1024)" - "Put(0, key2, 1024)" - "LogData(blob1)" - "Put(1, key3, 1024)" - "LogData(blob2)" - "Delete(0, key2)", - handler.seen); -} +TEST_F(DBTest, DynamicLevelCompressionPerLevel) { + if (!Snappy_Supported()) { + return; + } + const int kNKeys = 120; + int keys[kNKeys]; + for (int i = 0; i < kNKeys; i++) { + keys[i] = i; + } + std::random_shuffle(std::begin(keys), std::end(keys)); -TEST(DBTest, ReadFirstRecordCache) { - Options options = CurrentOptions(); - options.env = env_; + Random rnd(301); + Options options; options.create_if_missing = true; - DestroyAndReopen(&options); - - std::string path = dbname_ + "/000001.log"; - unique_ptr file; - ASSERT_OK(env_->NewWritableFile(path, &file, EnvOptions())); - - SequenceNumber s; - ASSERT_OK(dbfull()->TEST_ReadFirstLine(path, &s)); - ASSERT_EQ(s, 0U); - - ASSERT_OK(dbfull()->TEST_ReadFirstRecord(kAliveLogFile, 1, &s)); - ASSERT_EQ(s, 0U); - - log::Writer writer(std::move(file)); - WriteBatch batch; - batch.Put("foo", "bar"); - WriteBatchInternal::SetSequence(&batch, 10); - writer.AddRecord(WriteBatchInternal::Contents(&batch)); - - env_->count_sequential_reads_ = true; - // sequential_read_counter_ sanity test - ASSERT_EQ(env_->sequential_read_counter_.Read(), 0); - - ASSERT_OK(dbfull()->TEST_ReadFirstRecord(kAliveLogFile, 1, &s)); - ASSERT_EQ(s, 10U); - // did a read - ASSERT_EQ(env_->sequential_read_counter_.Read(), 1); - - ASSERT_OK(dbfull()->TEST_ReadFirstRecord(kAliveLogFile, 1, &s)); - ASSERT_EQ(s, 10U); - // no new reads since the value is cached - ASSERT_EQ(env_->sequential_read_counter_.Read(), 1); -} + options.db_write_buffer_size = 20480; + options.write_buffer_size = 20480; + options.max_write_buffer_number = 2; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 2; + options.level0_stop_writes_trigger = 2; + options.target_file_size_base = 20480; + options.level_compaction_dynamic_level_bytes = true; + options.max_bytes_for_level_base = 102400; + options.max_bytes_for_level_multiplier = 4; + options.max_background_compactions = 1; + options.num_levels = 5; -// Multi-threaded test: -namespace { + options.compression_per_level.resize(3); + options.compression_per_level[0] = kNoCompression; + options.compression_per_level[1] = kNoCompression; + options.compression_per_level[2] = kSnappyCompression; -static const int kColumnFamilies = 10; -static const int kNumThreads = 10; -static const int kTestSeconds = 10; -static const int kNumKeys = 1000; + OnFileDeletionListener* listener = new OnFileDeletionListener(); + options.listeners.emplace_back(listener); -struct MTState { - DBTest* test; - port::AtomicPointer stop; - port::AtomicPointer counter[kNumThreads]; - port::AtomicPointer thread_done[kNumThreads]; -}; + DestroyAndReopen(options); -struct MTThread { - MTState* state; - int id; -}; + // Insert more than 80K. L4 should be base level. Neither L0 nor L4 should + // be compressed, so total data size should be more than 80K. + for (int i = 0; i < 20; i++) { + ASSERT_OK(Put(Key(keys[i]), CompressibleString(&rnd, 4000))); + } + Flush(); + dbfull()->TEST_WaitForCompact(); -static void MTThreadBody(void* arg) { - MTThread* t = reinterpret_cast(arg); - int id = t->id; - DB* db = t->state->test->db_; - uintptr_t counter = 0; - fprintf(stderr, "... starting thread %d\n", id); - Random rnd(1000 + id); - char valbuf[1500]; - while (t->state->stop.Acquire_Load() == nullptr) { - t->state->counter[id].Release_Store(reinterpret_cast(counter)); + ASSERT_EQ(NumTableFilesAtLevel(1), 0); + ASSERT_EQ(NumTableFilesAtLevel(2), 0); + ASSERT_EQ(NumTableFilesAtLevel(3), 0); + // Assuming each files' metadata is at least 50 bytes/ + ASSERT_GT(SizeAtLevel(0) + SizeAtLevel(4), 20U * 4000U + 50U * 4); - int key = rnd.Uniform(kNumKeys); - char keybuf[20]; - snprintf(keybuf, sizeof(keybuf), "%016d", key); + // Insert 400KB. Some data will be compressed + for (int i = 21; i < 120; i++) { + ASSERT_OK(Put(Key(keys[i]), CompressibleString(&rnd, 4000))); + } + Flush(); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(NumTableFilesAtLevel(1), 0); + ASSERT_EQ(NumTableFilesAtLevel(2), 0); + ASSERT_LT(SizeAtLevel(0) + SizeAtLevel(3) + SizeAtLevel(4), + 120U * 4000U + 50U * 24); + // Make sure data in files in L3 is not compacted by removing all files + // in L4 and calculate number of rows + ASSERT_OK(dbfull()->SetOptions({ + {"disable_auto_compactions", "true"}, + })); + ColumnFamilyMetaData cf_meta; + db_->GetColumnFamilyMetaData(&cf_meta); + for (auto file : cf_meta.levels[4].files) { + listener->SetExpectedFileName(dbname_ + file.name); + ASSERT_OK(dbfull()->DeleteFile(file.name)); + } + listener->VerifyMatchedCount(cf_meta.levels[4].files.size()); + + int num_keys = 0; + std::unique_ptr iter(db_->NewIterator(ReadOptions())); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + num_keys++; + } + ASSERT_OK(iter->status()); + ASSERT_GT(SizeAtLevel(0) + SizeAtLevel(3), num_keys * 4000U + num_keys * 10U); +} - if (rnd.OneIn(2)) { - // Write values of the form . - // into each of the CFs - // We add some padding for force compactions. - int unique_id = rnd.Uniform(1000000); +TEST_F(DBTest, DynamicLevelCompressionPerLevel2) { + if (!Snappy_Supported() || !LZ4_Supported() || !Zlib_Supported()) { + return; + } + const int kNKeys = 500; + int keys[kNKeys]; + for (int i = 0; i < kNKeys; i++) { + keys[i] = i; + } + std::random_shuffle(std::begin(keys), std::end(keys)); - // Half of the time directly use WriteBatch. Half of the time use - // WriteBatchWithIndex. - if (rnd.OneIn(2)) { - WriteBatch batch; - for (int cf = 0; cf < kColumnFamilies; ++cf) { - snprintf(valbuf, sizeof(valbuf), "%d.%d.%d.%d.%-1000d", key, id, - static_cast(counter), cf, unique_id); - batch.Put(t->state->test->handles_[cf], Slice(keybuf), Slice(valbuf)); - } - ASSERT_OK(db->Write(WriteOptions(), &batch)); - } else { - WriteBatchWithIndex batch(db->GetOptions().comparator); - for (int cf = 0; cf < kColumnFamilies; ++cf) { - snprintf(valbuf, sizeof(valbuf), "%d.%d.%d.%d.%-1000d", key, id, - static_cast(counter), cf, unique_id); - batch.Put(t->state->test->handles_[cf], Slice(keybuf), Slice(valbuf)); + Random rnd(301); + Options options; + options.create_if_missing = true; + options.db_write_buffer_size = 6000000; + options.write_buffer_size = 600000; + options.max_write_buffer_number = 2; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 2; + options.level0_stop_writes_trigger = 2; + options.soft_pending_compaction_bytes_limit = 1024 * 1024; + options.target_file_size_base = 20; + + options.level_compaction_dynamic_level_bytes = true; + options.max_bytes_for_level_base = 200; + options.max_bytes_for_level_multiplier = 8; + options.max_background_compactions = 1; + options.num_levels = 5; + std::shared_ptr mtf(new mock::MockTableFactory); + options.table_factory = mtf; + + options.compression_per_level.resize(3); + options.compression_per_level[0] = kNoCompression; + options.compression_per_level[1] = kLZ4Compression; + options.compression_per_level[2] = kZlibCompression; + + DestroyAndReopen(options); + // When base level is L4, L4 is LZ4. + std::atomic num_zlib(0); + std::atomic num_lz4(0); + std::atomic num_no(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "LevelCompactionPicker::PickCompaction:Return", [&](void* arg) { + Compaction* compaction = reinterpret_cast(arg); + if (compaction->output_level() == 4) { + ASSERT_TRUE(compaction->output_compression() == kLZ4Compression); + num_lz4.fetch_add(1); } - ASSERT_OK(db->Write(WriteOptions(), batch.GetWriteBatch())); - } - } else { - // Read a value and verify that it matches the pattern written above - // and that writes to all column families were atomic (unique_id is the - // same) - std::vector keys(kColumnFamilies, Slice(keybuf)); - std::vector values; - std::vector statuses = - db->MultiGet(ReadOptions(), t->state->test->handles_, keys, &values); - Status s = statuses[0]; - // all statuses have to be the same - for (size_t i = 1; i < statuses.size(); ++i) { - // they are either both ok or both not-found - ASSERT_TRUE((s.ok() && statuses[i].ok()) || - (s.IsNotFound() && statuses[i].IsNotFound())); - } - if (s.IsNotFound()) { - // Key has not yet been written - } else { - // Check that the writer thread counter is >= the counter in the value - ASSERT_OK(s); - int unique_id = -1; - for (int i = 0; i < kColumnFamilies; ++i) { - int k, w, c, cf, u; - ASSERT_EQ(5, sscanf(values[i].c_str(), "%d.%d.%d.%d.%d", &k, &w, - &c, &cf, &u)) - << values[i]; - ASSERT_EQ(k, key); - ASSERT_GE(w, 0); - ASSERT_LT(w, kNumThreads); - ASSERT_LE((unsigned int)c, reinterpret_cast( - t->state->counter[w].Acquire_Load())); - ASSERT_EQ(cf, i); - if (i == 0) { - unique_id = u; - } else { - // this checks that updates across column families happened - // atomically -- all unique ids are the same - ASSERT_EQ(u, unique_id); - } + }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "FlushJob::WriteLevel0Table:output_compression", [&](void* arg) { + auto* compression = reinterpret_cast(arg); + ASSERT_TRUE(*compression == kNoCompression); + num_no.fetch_add(1); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + for (int i = 0; i < 100; i++) { + std::string value = RandomString(&rnd, 200); + ASSERT_OK(Put(Key(keys[i]), value)); + if (i % 25 == 24) { + Flush(); + dbfull()->TEST_WaitForCompact(); + } + } + + Flush(); + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); + + ASSERT_EQ(NumTableFilesAtLevel(1), 0); + ASSERT_EQ(NumTableFilesAtLevel(2), 0); + ASSERT_EQ(NumTableFilesAtLevel(3), 0); + ASSERT_GT(NumTableFilesAtLevel(4), 0); + ASSERT_GT(num_no.load(), 2); + ASSERT_GT(num_lz4.load(), 0); + int prev_num_files_l4 = NumTableFilesAtLevel(4); + + // After base level turn L4->L3, L3 becomes LZ4 and L4 becomes Zlib + num_lz4.store(0); + num_no.store(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "LevelCompactionPicker::PickCompaction:Return", [&](void* arg) { + Compaction* compaction = reinterpret_cast(arg); + if (compaction->output_level() == 4 && compaction->start_level() == 3) { + ASSERT_TRUE(compaction->output_compression() == kZlibCompression); + num_zlib.fetch_add(1); + } else { + ASSERT_TRUE(compaction->output_compression() == kLZ4Compression); + num_lz4.fetch_add(1); } - } + }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "FlushJob::WriteLevel0Table:output_compression", [&](void* arg) { + auto* compression = reinterpret_cast(arg); + ASSERT_TRUE(*compression == kNoCompression); + num_no.fetch_add(1); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + for (int i = 101; i < 500; i++) { + std::string value = RandomString(&rnd, 200); + ASSERT_OK(Put(Key(keys[i]), value)); + if (i % 100 == 99) { + Flush(); + dbfull()->TEST_WaitForCompact(); } - counter++; } - t->state->thread_done[id].Release_Store(t); - fprintf(stderr, "... stopping thread %d after %d ops\n", id, int(counter)); -} -} // namespace + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + ASSERT_EQ(NumTableFilesAtLevel(1), 0); + ASSERT_EQ(NumTableFilesAtLevel(2), 0); + ASSERT_GT(NumTableFilesAtLevel(3), 0); + ASSERT_GT(NumTableFilesAtLevel(4), prev_num_files_l4); + ASSERT_GT(num_no.load(), 2); + ASSERT_GT(num_lz4.load(), 0); + ASSERT_GT(num_zlib.load(), 0); +} + +TEST_F(DBTest, DynamicCompactionOptions) { + // minimum write buffer size is enforced at 64KB + const uint64_t k32KB = 1 << 15; + const uint64_t k64KB = 1 << 16; + const uint64_t k128KB = 1 << 17; + const uint64_t k1MB = 1 << 20; + const uint64_t k4KB = 1 << 12; + Options options; + options.env = env_; + options.create_if_missing = true; + options.compression = kNoCompression; + options.soft_pending_compaction_bytes_limit = 1024 * 1024; + options.write_buffer_size = k64KB; + options.arena_block_size = 4 * k4KB; + options.max_write_buffer_number = 2; + // Compaction related options + options.level0_file_num_compaction_trigger = 3; + options.level0_slowdown_writes_trigger = 4; + options.level0_stop_writes_trigger = 8; + options.target_file_size_base = k64KB; + options.max_compaction_bytes = options.target_file_size_base * 10; + options.target_file_size_multiplier = 1; + options.max_bytes_for_level_base = k128KB; + options.max_bytes_for_level_multiplier = 4; -TEST(DBTest, MultiThreaded) { - do { - std::vector cfs; - for (int i = 1; i < kColumnFamilies; ++i) { - cfs.push_back(std::to_string(i)); - } - CreateAndReopenWithCF(cfs); - // Initialize state - MTState mt; - mt.test = this; - mt.stop.Release_Store(0); - for (int id = 0; id < kNumThreads; id++) { - mt.counter[id].Release_Store(0); - mt.thread_done[id].Release_Store(0); - } + // Block flush thread and disable compaction thread + env_->SetBackgroundThreads(1, Env::LOW); + env_->SetBackgroundThreads(1, Env::HIGH); + DestroyAndReopen(options); - // Start threads - MTThread thread[kNumThreads]; - for (int id = 0; id < kNumThreads; id++) { - thread[id].state = &mt; - thread[id].id = id; - env_->StartThread(MTThreadBody, &thread[id]); + auto gen_l0_kb = [this](int start, int size, int stride) { + Random rnd(301); + for (int i = 0; i < size; i++) { + ASSERT_OK(Put(Key(start + stride * i), RandomString(&rnd, 1024))); } + dbfull()->TEST_WaitForFlushMemTable(); + }; - // Let them run for a while - env_->SleepForMicroseconds(kTestSeconds * 1000000); - - // Stop the threads and wait for them to finish - mt.stop.Release_Store(&mt); - for (int id = 0; id < kNumThreads; id++) { - while (mt.thread_done[id].Acquire_Load() == nullptr) { - env_->SleepForMicroseconds(100000); - } + // Write 3 files that have the same key range. + // Since level0_file_num_compaction_trigger is 3, compaction should be + // triggered. The compaction should result in one L1 file + gen_l0_kb(0, 64, 1); + ASSERT_EQ(NumTableFilesAtLevel(0), 1); + gen_l0_kb(0, 64, 1); + ASSERT_EQ(NumTableFilesAtLevel(0), 2); + gen_l0_kb(0, 64, 1); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ("0,1", FilesPerLevel()); + std::vector metadata; + db_->GetLiveFilesMetaData(&metadata); + ASSERT_EQ(1U, metadata.size()); + ASSERT_LE(metadata[0].size, k64KB + k4KB); + ASSERT_GE(metadata[0].size, k64KB - k4KB); + + // Test compaction trigger and target_file_size_base + // Reduce compaction trigger to 2, and reduce L1 file size to 32KB. + // Writing to 64KB L0 files should trigger a compaction. Since these + // 2 L0 files have the same key range, compaction merge them and should + // result in 2 32KB L1 files. + ASSERT_OK(dbfull()->SetOptions({{"level0_file_num_compaction_trigger", "2"}, + {"target_file_size_base", ToString(k32KB)}})); + + gen_l0_kb(0, 64, 1); + ASSERT_EQ("1,1", FilesPerLevel()); + gen_l0_kb(0, 64, 1); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ("0,2", FilesPerLevel()); + metadata.clear(); + db_->GetLiveFilesMetaData(&metadata); + ASSERT_EQ(2U, metadata.size()); + ASSERT_LE(metadata[0].size, k32KB + k4KB); + ASSERT_GE(metadata[0].size, k32KB - k4KB); + ASSERT_LE(metadata[1].size, k32KB + k4KB); + ASSERT_GE(metadata[1].size, k32KB - k4KB); + + // Test max_bytes_for_level_base + // Increase level base size to 256KB and write enough data that will + // fill L1 and L2. L1 size should be around 256KB while L2 size should be + // around 256KB x 4. + ASSERT_OK( + dbfull()->SetOptions({{"max_bytes_for_level_base", ToString(k1MB)}})); + + // writing 96 x 64KB => 6 * 1024KB + // (L1 + L2) = (1 + 4) * 1024KB + for (int i = 0; i < 96; ++i) { + gen_l0_kb(i, 64, 96); + } + dbfull()->TEST_WaitForCompact(); + ASSERT_GT(SizeAtLevel(1), k1MB / 2); + ASSERT_LT(SizeAtLevel(1), k1MB + k1MB / 2); + + // Within (0.5, 1.5) of 4MB. + ASSERT_GT(SizeAtLevel(2), 2 * k1MB); + ASSERT_LT(SizeAtLevel(2), 6 * k1MB); + + // Test max_bytes_for_level_multiplier and + // max_bytes_for_level_base. Now, reduce both mulitplier and level base, + // After filling enough data that can fit in L1 - L3, we should see L1 size + // reduces to 128KB from 256KB which was asserted previously. Same for L2. + ASSERT_OK( + dbfull()->SetOptions({{"max_bytes_for_level_multiplier", "2"}, + {"max_bytes_for_level_base", ToString(k128KB)}})); + + // writing 20 x 64KB = 10 x 128KB + // (L1 + L2 + L3) = (1 + 2 + 4) * 128KB + for (int i = 0; i < 20; ++i) { + gen_l0_kb(i, 64, 32); + } + dbfull()->TEST_WaitForCompact(); + uint64_t total_size = SizeAtLevel(1) + SizeAtLevel(2) + SizeAtLevel(3); + ASSERT_TRUE(total_size < k128KB * 7 * 1.5); + + // Test level0_stop_writes_trigger. + // Clean up memtable and L0. Block compaction threads. If continue to write + // and flush memtables. We should see put stop after 8 memtable flushes + // since level0_stop_writes_trigger = 8 + dbfull()->TEST_FlushMemTable(true); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + // Block compaction + test::SleepingBackgroundTask sleeping_task_low; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + sleeping_task_low.WaitUntilSleeping(); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + int count = 0; + Random rnd(301); + WriteOptions wo; + while (count < 64) { + ASSERT_OK(Put(Key(count), RandomString(&rnd, 1024), wo)); + dbfull()->TEST_FlushMemTable(true); + count++; + if (dbfull()->TEST_write_controler().IsStopped()) { + sleeping_task_low.WakeUp(); + break; } - // skip as HashCuckooRep does not support snapshot - } while (ChangeOptions(kSkipHashCuckoo)); -} + } + // Stop trigger = 8 + ASSERT_EQ(count, 8); + // Unblock + sleeping_task_low.WaitUntilDone(); -// Group commit test: -namespace { + // Now reduce level0_stop_writes_trigger to 6. Clear up memtables and L0. + // Block compaction thread again. Perform the put and memtable flushes + // until we see the stop after 6 memtable flushes. + ASSERT_OK(dbfull()->SetOptions({{"level0_stop_writes_trigger", "6"}})); + dbfull()->TEST_FlushMemTable(true); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); + + // Block compaction again + sleeping_task_low.Reset(); + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + sleeping_task_low.WaitUntilSleeping(); + count = 0; + while (count < 64) { + ASSERT_OK(Put(Key(count), RandomString(&rnd, 1024), wo)); + dbfull()->TEST_FlushMemTable(true); + count++; + if (dbfull()->TEST_write_controler().IsStopped()) { + sleeping_task_low.WakeUp(); + break; + } + } + ASSERT_EQ(count, 6); + // Unblock + sleeping_task_low.WaitUntilDone(); -static const int kGCNumThreads = 4; -static const int kGCNumKeys = 1000; + // Test disable_auto_compactions + // Compaction thread is unblocked but auto compaction is disabled. Write + // 4 L0 files and compaction should be triggered. If auto compaction is + // disabled, then TEST_WaitForCompact will be waiting for nothing. Number of + // L0 files do not change after the call. + ASSERT_OK(dbfull()->SetOptions({{"disable_auto_compactions", "true"}})); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); -struct GCThread { - DB* db; - int id; - std::atomic done; -}; + for (int i = 0; i < 4; ++i) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 1024))); + // Wait for compaction so that put won't stop + dbfull()->TEST_FlushMemTable(true); + } + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(NumTableFilesAtLevel(0), 4); -static void GCThreadBody(void* arg) { - GCThread* t = reinterpret_cast(arg); - int id = t->id; - DB* db = t->db; - WriteOptions wo; + // Enable auto compaction and perform the same test, # of L0 files should be + // reduced after compaction. + ASSERT_OK(dbfull()->SetOptions({{"disable_auto_compactions", "false"}})); + dbfull()->CompactRange(CompactRangeOptions(), nullptr, nullptr); + ASSERT_EQ(NumTableFilesAtLevel(0), 0); - for (int i = 0; i < kGCNumKeys; ++i) { - std::string kv(std::to_string(i + id * kGCNumKeys)); - ASSERT_OK(db->Put(wo, kv, kv)); + for (int i = 0; i < 4; ++i) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 1024))); + // Wait for compaction so that put won't stop + dbfull()->TEST_FlushMemTable(true); } - t->done = true; + dbfull()->TEST_WaitForCompact(); + ASSERT_LT(NumTableFilesAtLevel(0), 4); } +#endif // ROCKSDB_LITE -} // namespace +TEST_F(DBTest, FileCreationRandomFailure) { + Options options; + options.env = env_; + options.create_if_missing = true; + options.write_buffer_size = 100000; // Small write buffer + options.target_file_size_base = 200000; + options.max_bytes_for_level_base = 1000000; + options.max_bytes_for_level_multiplier = 2; -TEST(DBTest, GroupCommitTest) { - do { - Options options = CurrentOptions(); - options.statistics = rocksdb::CreateDBStatistics(); - Reopen(&options); + DestroyAndReopen(options); + Random rnd(301); - // Start threads - GCThread thread[kGCNumThreads]; - for (int id = 0; id < kGCNumThreads; id++) { - thread[id].id = id; - thread[id].db = db_; - thread[id].done = false; - env_->StartThread(GCThreadBody, &thread[id]); + const int kCDTKeysPerBuffer = 4; + const int kTestSize = kCDTKeysPerBuffer * 4096; + const int kTotalIteration = 100; + // the second half of the test involves in random failure + // of file creation. + const int kRandomFailureTest = kTotalIteration / 2; + std::vector values; + for (int i = 0; i < kTestSize; ++i) { + values.push_back("NOT_FOUND"); + } + for (int j = 0; j < kTotalIteration; ++j) { + if (j == kRandomFailureTest) { + env_->non_writeable_rate_.store(90); } - - for (int id = 0; id < kGCNumThreads; id++) { - while (thread[id].done == false) { - env_->SleepForMicroseconds(100000); + for (int k = 0; k < kTestSize; ++k) { + // here we expect some of the Put fails. + std::string value = RandomString(&rnd, 100); + Status s = Put(Key(k), Slice(value)); + if (s.ok()) { + // update the latest successful put + values[k] = value; + } + // But everything before we simulate the failure-test should succeed. + if (j < kRandomFailureTest) { + ASSERT_OK(s); } } - ASSERT_GT(TestGetTickerCount(options, WRITE_DONE_BY_OTHER), 0); + } - std::vector expected_db; - for (int i = 0; i < kGCNumThreads * kGCNumKeys; ++i) { - expected_db.push_back(std::to_string(i)); - } - sort(expected_db.begin(), expected_db.end()); + // If rocksdb does not do the correct job, internal assert will fail here. + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); - Iterator* itr = db_->NewIterator(ReadOptions()); - itr->SeekToFirst(); - for (auto x : expected_db) { - ASSERT_TRUE(itr->Valid()); - ASSERT_EQ(itr->key().ToString(), x); - ASSERT_EQ(itr->value().ToString(), x); - itr->Next(); - } - ASSERT_TRUE(!itr->Valid()); - delete itr; + // verify we have the latest successful update + for (int k = 0; k < kTestSize; ++k) { + auto v = Get(Key(k)); + ASSERT_EQ(v, values[k]); + } - } while (ChangeOptions(kSkipNoSeekToLast)); + // reopen and reverify we have the latest successful update + env_->non_writeable_rate_.store(0); + Reopen(options); + for (int k = 0; k < kTestSize; ++k) { + auto v = Get(Key(k)); + ASSERT_EQ(v, values[k]); + } } -namespace { -typedef std::map KVMap; -} +#ifndef ROCKSDB_LITE +TEST_F(DBTest, DynamicMiscOptions) { + // Test max_sequential_skip_in_iterations + Options options; + options.env = env_; + options.create_if_missing = true; + options.max_sequential_skip_in_iterations = 16; + options.compression = kNoCompression; + options.statistics = rocksdb::CreateDBStatistics(); + DestroyAndReopen(options); -class ModelDB: public DB { - public: - class ModelSnapshot : public Snapshot { - public: - KVMap map_; + auto assert_reseek_count = [this, &options](int key_start, int num_reseek) { + int key0 = key_start; + int key1 = key_start + 1; + int key2 = key_start + 2; + Random rnd(301); + ASSERT_OK(Put(Key(key0), RandomString(&rnd, 8))); + for (int i = 0; i < 10; ++i) { + ASSERT_OK(Put(Key(key1), RandomString(&rnd, 8))); + } + ASSERT_OK(Put(Key(key2), RandomString(&rnd, 8))); + std::unique_ptr iter(db_->NewIterator(ReadOptions())); + iter->Seek(Key(key1)); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Key(key1)), 0); + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(iter->key().compare(Key(key2)), 0); + ASSERT_EQ(num_reseek, + TestGetTickerCount(options, NUMBER_OF_RESEEKS_IN_ITERATION)); }; + // No reseek + assert_reseek_count(100, 0); + + ASSERT_OK(dbfull()->SetOptions({{"max_sequential_skip_in_iterations", "4"}})); + // Clear memtable and make new option effective + dbfull()->TEST_FlushMemTable(true); + // Trigger reseek + assert_reseek_count(200, 1); + + ASSERT_OK( + dbfull()->SetOptions({{"max_sequential_skip_in_iterations", "16"}})); + // Clear memtable and make new option effective + dbfull()->TEST_FlushMemTable(true); + // No reseek + assert_reseek_count(300, 1); + + MutableCFOptions mutable_cf_options; + CreateAndReopenWithCF({"pikachu"}, options); + // Test soft_pending_compaction_bytes_limit, + // hard_pending_compaction_bytes_limit + ASSERT_OK(dbfull()->SetOptions( + handles_[1], {{"soft_pending_compaction_bytes_limit", "200"}, + {"hard_pending_compaction_bytes_limit", "300"}})); + ASSERT_OK(dbfull()->TEST_GetLatestMutableCFOptions(handles_[1], + &mutable_cf_options)); + ASSERT_EQ(200, mutable_cf_options.soft_pending_compaction_bytes_limit); + ASSERT_EQ(300, mutable_cf_options.hard_pending_compaction_bytes_limit); + // Test report_bg_io_stats + ASSERT_OK( + dbfull()->SetOptions(handles_[1], {{"report_bg_io_stats", "true"}})); + // sanity check + ASSERT_OK(dbfull()->TEST_GetLatestMutableCFOptions(handles_[1], + &mutable_cf_options)); + ASSERT_TRUE(mutable_cf_options.report_bg_io_stats); + // Test compression + // sanity check + ASSERT_OK(dbfull()->SetOptions({{"compression", "kNoCompression"}})); + ASSERT_OK(dbfull()->TEST_GetLatestMutableCFOptions(handles_[0], + &mutable_cf_options)); + ASSERT_EQ(CompressionType::kNoCompression, mutable_cf_options.compression); + ASSERT_OK(dbfull()->SetOptions({{"compression", "kSnappyCompression"}})); + ASSERT_OK(dbfull()->TEST_GetLatestMutableCFOptions(handles_[0], + &mutable_cf_options)); + ASSERT_EQ(CompressionType::kSnappyCompression, + mutable_cf_options.compression); + // Test paranoid_file_checks already done in db_block_cache_test + ASSERT_OK( + dbfull()->SetOptions(handles_[1], {{"paranoid_file_checks", "true"}})); + ASSERT_OK(dbfull()->TEST_GetLatestMutableCFOptions(handles_[1], + &mutable_cf_options)); + ASSERT_TRUE(mutable_cf_options.report_bg_io_stats); +} +#endif // ROCKSDB_LITE + +TEST_F(DBTest, L0L1L2AndUpHitCounter) { + Options options = CurrentOptions(); + options.write_buffer_size = 32 * 1024; + options.target_file_size_base = 32 * 1024; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 2; + options.level0_stop_writes_trigger = 4; + options.max_bytes_for_level_base = 64 * 1024; + options.max_write_buffer_number = 2; + options.max_background_compactions = 8; + options.max_background_flushes = 8; + options.statistics = rocksdb::CreateDBStatistics(); + CreateAndReopenWithCF({"mypikachu"}, options); - explicit ModelDB(const Options& options) : options_(options) {} - using DB::Put; - virtual Status Put(const WriteOptions& o, ColumnFamilyHandle* cf, - const Slice& k, const Slice& v) { - WriteBatch batch; - batch.Put(cf, k, v); - return Write(o, &batch); - } - using DB::Merge; - virtual Status Merge(const WriteOptions& o, ColumnFamilyHandle* cf, - const Slice& k, const Slice& v) { - WriteBatch batch; - batch.Merge(cf, k, v); - return Write(o, &batch); - } - using DB::Delete; - virtual Status Delete(const WriteOptions& o, ColumnFamilyHandle* cf, - const Slice& key) { - WriteBatch batch; - batch.Delete(cf, key); - return Write(o, &batch); - } - using DB::Get; - virtual Status Get(const ReadOptions& options, ColumnFamilyHandle* cf, - const Slice& key, std::string* value) { - return Status::NotSupported(key); - } - - using DB::MultiGet; - virtual std::vector MultiGet( - const ReadOptions& options, - const std::vector& column_family, - const std::vector& keys, std::vector* values) { - std::vector s(keys.size(), - Status::NotSupported("Not implemented.")); - return s; + int numkeys = 20000; + for (int i = 0; i < numkeys; i++) { + ASSERT_OK(Put(1, Key(i), "val")); } + ASSERT_EQ(0, TestGetTickerCount(options, GET_HIT_L0)); + ASSERT_EQ(0, TestGetTickerCount(options, GET_HIT_L1)); + ASSERT_EQ(0, TestGetTickerCount(options, GET_HIT_L2_AND_UP)); - using DB::GetPropertiesOfAllTables; - virtual Status GetPropertiesOfAllTables(ColumnFamilyHandle* column_family, - TablePropertiesCollection* props) { - return Status(); - } + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); - using DB::KeyMayExist; - virtual bool KeyMayExist(const ReadOptions& options, - ColumnFamilyHandle* column_family, const Slice& key, - std::string* value, bool* value_found = nullptr) { - if (value_found != nullptr) { - *value_found = false; - } - return true; // Not Supported directly - } - using DB::NewIterator; - virtual Iterator* NewIterator(const ReadOptions& options, - ColumnFamilyHandle* column_family) { - if (options.snapshot == nullptr) { - KVMap* saved = new KVMap; - *saved = map_; - return new ModelIter(saved, true); - } else { - const KVMap* snapshot_state = - &(reinterpret_cast(options.snapshot)->map_); - return new ModelIter(snapshot_state, false); - } - } - virtual Status NewIterators( - const ReadOptions& options, - const std::vector& column_family, - std::vector* iterators) { - return Status::NotSupported("Not supported yet"); - } - virtual const Snapshot* GetSnapshot() { - ModelSnapshot* snapshot = new ModelSnapshot; - snapshot->map_ = map_; - return snapshot; - } + for (int i = 0; i < numkeys; i++) { + ASSERT_EQ(Get(1, Key(i)), "val"); + } + + ASSERT_GT(TestGetTickerCount(options, GET_HIT_L0), 100); + ASSERT_GT(TestGetTickerCount(options, GET_HIT_L1), 100); + ASSERT_GT(TestGetTickerCount(options, GET_HIT_L2_AND_UP), 100); + + ASSERT_EQ(numkeys, TestGetTickerCount(options, GET_HIT_L0) + + TestGetTickerCount(options, GET_HIT_L1) + + TestGetTickerCount(options, GET_HIT_L2_AND_UP)); +} + +TEST_F(DBTest, EncodeDecompressedBlockSizeTest) { + // iter 0 -- zlib + // iter 1 -- bzip2 + // iter 2 -- lz4 + // iter 3 -- lz4HC + // iter 4 -- xpress + CompressionType compressions[] = {kZlibCompression, kBZip2Compression, + kLZ4Compression, kLZ4HCCompression, + kXpressCompression}; + for (auto comp : compressions) { + if (!CompressionTypeSupported(comp)) { + continue; + } + // first_table_version 1 -- generate with table_version == 1, read with + // table_version == 2 + // first_table_version 2 -- generate with table_version == 2, read with + // table_version == 1 + for (int first_table_version = 1; first_table_version <= 2; + ++first_table_version) { + BlockBasedTableOptions table_options; + table_options.format_version = first_table_version; + table_options.filter_policy.reset(NewBloomFilterPolicy(10)); + Options options = CurrentOptions(); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.create_if_missing = true; + options.compression = comp; + DestroyAndReopen(options); - virtual void ReleaseSnapshot(const Snapshot* snapshot) { - delete reinterpret_cast(snapshot); - } + int kNumKeysWritten = 100000; - virtual Status Write(const WriteOptions& options, WriteBatch* batch) { - class Handler : public WriteBatch::Handler { - public: - KVMap* map_; - virtual void Put(const Slice& key, const Slice& value) { - (*map_)[key.ToString()] = value.ToString(); - } - virtual void Merge(const Slice& key, const Slice& value) { - // ignore merge for now - //(*map_)[key.ToString()] = value.ToString(); - } - virtual void Delete(const Slice& key) { - map_->erase(key.ToString()); + Random rnd(301); + for (int i = 0; i < kNumKeysWritten; ++i) { + // compressible string + ASSERT_OK(Put(Key(i), RandomString(&rnd, 128) + std::string(128, 'a'))); } - }; - Handler handler; - handler.map_ = &map_; - return batch->Iterate(&handler); - } - using DB::GetProperty; - virtual bool GetProperty(ColumnFamilyHandle* column_family, - const Slice& property, std::string* value) { - return false; - } - using DB::GetIntProperty; - virtual bool GetIntProperty(ColumnFamilyHandle* column_family, - const Slice& property, uint64_t* value) override { - return false; - } - using DB::GetApproximateSizes; - virtual void GetApproximateSizes(ColumnFamilyHandle* column_family, - const Range* range, int n, uint64_t* sizes) { - for (int i = 0; i < n; i++) { - sizes[i] = 0; + table_options.format_version = first_table_version == 1 ? 2 : 1; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + Reopen(options); + for (int i = 0; i < kNumKeysWritten; ++i) { + auto r = Get(Key(i)); + ASSERT_EQ(r.substr(128), std::string(128, 'a')); + } } } - using DB::CompactRange; - virtual Status CompactRange(ColumnFamilyHandle* column_family, - const Slice* start, const Slice* end, - bool reduce_level, int target_level, - uint32_t output_path_id) { - return Status::NotSupported("Not supported operation."); +} + +TEST_F(DBTest, CloseSpeedup) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleLevel; + options.write_buffer_size = 110 << 10; // 110KB + options.arena_block_size = 4 << 10; + options.level0_file_num_compaction_trigger = 2; + options.num_levels = 4; + options.max_bytes_for_level_base = 400 * 1024; + options.max_write_buffer_number = 16; + + // Block background threads + env_->SetBackgroundThreads(1, Env::LOW); + env_->SetBackgroundThreads(1, Env::HIGH); + test::SleepingBackgroundTask sleeping_task_low; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + test::SleepingBackgroundTask sleeping_task_high; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, + &sleeping_task_high, Env::Priority::HIGH); + + std::vector filenames; + env_->GetChildren(dbname_, &filenames); + // Delete archival files. + for (size_t i = 0; i < filenames.size(); ++i) { + env_->DeleteFile(dbname_ + "/" + filenames[i]); } + env_->DeleteDir(dbname_); + DestroyAndReopen(options); - using DB::NumberLevels; - virtual int NumberLevels(ColumnFamilyHandle* column_family) { return 1; } + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + env_->SetBackgroundThreads(1, Env::LOW); + env_->SetBackgroundThreads(1, Env::HIGH); + Random rnd(301); + int key_idx = 0; - using DB::MaxMemCompactionLevel; - virtual int MaxMemCompactionLevel(ColumnFamilyHandle* column_family) { - return 1; + // First three 110KB files are not going to level 2 + // After that, (100K, 200K) + for (int num = 0; num < 5; num++) { + GenerateNewFile(&rnd, &key_idx, true); } - using DB::Level0StopWriteTrigger; - virtual int Level0StopWriteTrigger(ColumnFamilyHandle* column_family) { - return -1; - } + ASSERT_EQ(0, GetSstFileCount(dbname_)); - virtual const std::string& GetName() const { - return name_; - } + Close(); + ASSERT_EQ(0, GetSstFileCount(dbname_)); - virtual Env* GetEnv() const { - return nullptr; - } + // Unblock background threads + sleeping_task_high.WakeUp(); + sleeping_task_high.WaitUntilDone(); + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilDone(); - using DB::GetOptions; - virtual const Options& GetOptions(ColumnFamilyHandle* column_family) const { - return options_; - } + Destroy(options); +} - using DB::Flush; - virtual Status Flush(const rocksdb::FlushOptions& options, - ColumnFamilyHandle* column_family) { - Status ret; - return ret; - } +class DelayedMergeOperator : public MergeOperator { + private: + DBTest* db_test_; - virtual Status DisableFileDeletions() { - return Status::OK(); - } - virtual Status EnableFileDeletions(bool force) { - return Status::OK(); - } - virtual Status GetLiveFiles(std::vector&, uint64_t* size, - bool flush_memtable = true) { - return Status::OK(); - } + public: + explicit DelayedMergeOperator(DBTest* d) : db_test_(d) {} - virtual Status GetSortedWalFiles(VectorLogPtr& files) { - return Status::OK(); + virtual bool FullMergeV2(const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const override { + db_test_->env_->addon_time_.fetch_add(1000); + merge_out->new_value = ""; + return true; } - virtual Status DeleteFile(std::string name) { - return Status::OK(); - } + virtual const char* Name() const override { return "DelayedMergeOperator"; } +}; - virtual Status GetDbIdentity(std::string& identity) { - return Status::OK(); - } +TEST_F(DBTest, MergeTestTime) { + std::string one, two, three; + PutFixed64(&one, 1); + PutFixed64(&two, 2); + PutFixed64(&three, 3); - virtual SequenceNumber GetLatestSequenceNumber() const { - return 0; - } - virtual Status GetUpdatesSince( - rocksdb::SequenceNumber, unique_ptr*, - const TransactionLogIterator::ReadOptions& - read_options = TransactionLogIterator::ReadOptions()) { - return Status::NotSupported("Not supported in Model DB"); - } + // Enable time profiling + SetPerfLevel(kEnableTime); + this->env_->addon_time_.store(0); + this->env_->time_elapse_only_sleep_ = true; + this->env_->no_slowdown_ = true; + Options options = CurrentOptions(); + options.statistics = rocksdb::CreateDBStatistics(); + options.merge_operator.reset(new DelayedMergeOperator(this)); + DestroyAndReopen(options); - virtual ColumnFamilyHandle* DefaultColumnFamily() const { return nullptr; } + ASSERT_EQ(TestGetTickerCount(options, MERGE_OPERATION_TOTAL_TIME), 0); + db_->Put(WriteOptions(), "foo", one); + ASSERT_OK(Flush()); + ASSERT_OK(db_->Merge(WriteOptions(), "foo", two)); + ASSERT_OK(Flush()); + ASSERT_OK(db_->Merge(WriteOptions(), "foo", three)); + ASSERT_OK(Flush()); - private: - class ModelIter: public Iterator { - public: - ModelIter(const KVMap* map, bool owned) - : map_(map), owned_(owned), iter_(map_->end()) { - } - ~ModelIter() { - if (owned_) delete map_; - } - virtual bool Valid() const { return iter_ != map_->end(); } - virtual void SeekToFirst() { iter_ = map_->begin(); } - virtual void SeekToLast() { - if (map_->empty()) { - iter_ = map_->end(); - } else { - iter_ = map_->find(map_->rbegin()->first); - } - } - virtual void Seek(const Slice& k) { - iter_ = map_->lower_bound(k.ToString()); - } - virtual void Next() { ++iter_; } - virtual void Prev() { - if (iter_ == map_->begin()) { - iter_ = map_->end(); - return; - } - --iter_; - } + ReadOptions opt; + opt.verify_checksums = true; + opt.snapshot = nullptr; + std::string result; + db_->Get(opt, "foo", &result); - virtual Slice key() const { return iter_->first; } - virtual Slice value() const { return iter_->second; } - virtual Status status() const { return Status::OK(); } - private: - const KVMap* const map_; - const bool owned_; // Do we own map_ - KVMap::const_iterator iter_; - }; - const Options options_; - KVMap map_; - std::string name_ = ""; -}; + ASSERT_EQ(1000000, TestGetTickerCount(options, MERGE_OPERATION_TOTAL_TIME)); -static std::string RandomKey(Random* rnd, int minimum = 0) { - int len; - do { - len = (rnd->OneIn(3) - ? 1 // Short sometimes to encourage collisions - : (rnd->OneIn(100) ? rnd->Skewed(10) : rnd->Uniform(10))); - } while (len < minimum); - return test::RandomKey(rnd, len); + ReadOptions read_options; + std::unique_ptr iter(db_->NewIterator(read_options)); + int count = 0; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_OK(iter->status()); + ++count; + } + + ASSERT_EQ(1, count); + ASSERT_EQ(2000000, TestGetTickerCount(options, MERGE_OPERATION_TOTAL_TIME)); +#ifdef ROCKSDB_USING_THREAD_STATUS + ASSERT_GT(TestGetTickerCount(options, FLUSH_WRITE_BYTES), 0); +#endif // ROCKSDB_USING_THREAD_STATUS + this->env_->time_elapse_only_sleep_ = false; } -static bool CompareIterators(int step, - DB* model, - DB* db, - const Snapshot* model_snap, - const Snapshot* db_snap) { - ReadOptions options; - options.snapshot = model_snap; - Iterator* miter = model->NewIterator(options); - options.snapshot = db_snap; - Iterator* dbiter = db->NewIterator(options); - bool ok = true; - int count = 0; - for (miter->SeekToFirst(), dbiter->SeekToFirst(); - ok && miter->Valid() && dbiter->Valid(); - miter->Next(), dbiter->Next()) { - count++; - if (miter->key().compare(dbiter->key()) != 0) { - fprintf(stderr, "step %d: Key mismatch: '%s' vs. '%s'\n", - step, - EscapeString(miter->key()).c_str(), - EscapeString(dbiter->key()).c_str()); - ok = false; - break; - } +#ifndef ROCKSDB_LITE +TEST_P(DBTestWithParam, MergeCompactionTimeTest) { + SetPerfLevel(kEnableTime); + Options options = CurrentOptions(); + options.compaction_filter_factory = std::make_shared(); + options.statistics = rocksdb::CreateDBStatistics(); + options.merge_operator.reset(new DelayedMergeOperator(this)); + options.compaction_style = kCompactionStyleUniversal; + options.max_subcompactions = max_subcompactions_; + DestroyAndReopen(options); - if (miter->value().compare(dbiter->value()) != 0) { - fprintf(stderr, "step %d: Value mismatch for key '%s': '%s' vs. '%s'\n", - step, - EscapeString(miter->key()).c_str(), - EscapeString(miter->value()).c_str(), - EscapeString(miter->value()).c_str()); - ok = false; - } + for (int i = 0; i < 1000; i++) { + ASSERT_OK(db_->Merge(WriteOptions(), "foo", "TEST")); + ASSERT_OK(Flush()); } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); - if (ok) { - if (miter->Valid() != dbiter->Valid()) { - fprintf(stderr, "step %d: Mismatch at end of iterators: %d vs. %d\n", - step, miter->Valid(), dbiter->Valid()); - ok = false; - } - } - delete miter; - delete dbiter; - return ok; + ASSERT_NE(TestGetTickerCount(options, MERGE_OPERATION_TOTAL_TIME), 0); } -TEST(DBTest, Randomized) { - Random rnd(test::RandomSeed()); - do { - ModelDB model(CurrentOptions()); - const int N = 10000; - const Snapshot* model_snap = nullptr; - const Snapshot* db_snap = nullptr; - std::string k, v; - for (int step = 0; step < N; step++) { - // TODO(sanjay): Test Get() works - int p = rnd.Uniform(100); - int minimum = 0; - if (option_config_ == kHashSkipList || - option_config_ == kHashLinkList || - option_config_ == kHashCuckoo || - option_config_ == kPlainTableFirstBytePrefix || - option_config_ == kBlockBasedTableWithWholeKeyHashIndex || - option_config_ == kBlockBasedTableWithPrefixHashIndex) { - minimum = 1; - } - if (p < 45) { // Put - k = RandomKey(&rnd, minimum); - v = RandomString(&rnd, - rnd.OneIn(20) - ? 100 + rnd.Uniform(100) - : rnd.Uniform(8)); - ASSERT_OK(model.Put(WriteOptions(), k, v)); - ASSERT_OK(db_->Put(WriteOptions(), k, v)); - - } else if (p < 90) { // Delete - k = RandomKey(&rnd, minimum); - ASSERT_OK(model.Delete(WriteOptions(), k)); - ASSERT_OK(db_->Delete(WriteOptions(), k)); - - - } else { // Multi-element batch - WriteBatch b; - const int num = rnd.Uniform(8); - for (int i = 0; i < num; i++) { - if (i == 0 || !rnd.OneIn(10)) { - k = RandomKey(&rnd, minimum); - } else { - // Periodically re-use the same key from the previous iter, so - // we have multiple entries in the write batch for the same key - } - if (rnd.OneIn(2)) { - v = RandomString(&rnd, rnd.Uniform(10)); - b.Put(k, v); - } else { - b.Delete(k); - } - } - ASSERT_OK(model.Write(WriteOptions(), &b)); - ASSERT_OK(db_->Write(WriteOptions(), &b)); - } - - if ((step % 100) == 0) { - // For DB instances that use the hash index + block-based table, the - // iterator will be invalid right when seeking a non-existent key, right - // than return a key that is close to it. - if (option_config_ != kBlockBasedTableWithWholeKeyHashIndex && - option_config_ != kBlockBasedTableWithPrefixHashIndex) { - ASSERT_TRUE(CompareIterators(step, &model, db_, nullptr, nullptr)); - ASSERT_TRUE(CompareIterators(step, &model, db_, model_snap, db_snap)); - } +TEST_P(DBTestWithParam, FilterCompactionTimeTest) { + Options options = CurrentOptions(); + options.compaction_filter_factory = + std::make_shared(this); + options.disable_auto_compactions = true; + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + options.max_subcompactions = max_subcompactions_; + DestroyAndReopen(options); - // Save a snapshot from each DB this time that we'll use next - // time we compare things, to make sure the current state is - // preserved with the snapshot - if (model_snap != nullptr) model.ReleaseSnapshot(model_snap); - if (db_snap != nullptr) db_->ReleaseSnapshot(db_snap); + // put some data + for (int table = 0; table < 4; ++table) { + for (int i = 0; i < 10 + table; ++i) { + Put(ToString(table * 100 + i), "val"); + } + Flush(); + } - Reopen(); - ASSERT_TRUE(CompareIterators(step, &model, db_, nullptr, nullptr)); + CompactRangeOptions cro; + cro.exclusive_manual_compaction = exclusive_manual_compaction_; + ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr)); + ASSERT_EQ(0U, CountLiveFiles()); - model_snap = model.GetSnapshot(); - db_snap = db_->GetSnapshot(); - } + Reopen(options); - if ((step % 2000) == 0) { - fprintf(stdout, - "DBTest.Randomized, option ID: %d, step: %d out of %d\n", - option_config_, step, N); - } - } - if (model_snap != nullptr) model.ReleaseSnapshot(model_snap); - if (db_snap != nullptr) db_->ReleaseSnapshot(db_snap); - // skip cuckoo hash as it does not support snapshot. - } while (ChangeOptions(kSkipDeletesFilterFirst | kSkipNoSeekToLast | - kSkipHashCuckoo)); + Iterator* itr = db_->NewIterator(ReadOptions()); + itr->SeekToFirst(); + ASSERT_NE(TestGetTickerCount(options, FILTER_OPERATION_TOTAL_TIME), 0); + delete itr; } +#endif // ROCKSDB_LITE -TEST(DBTest, MultiGetSimple) { - do { - CreateAndReopenWithCF({"pikachu"}); - ASSERT_OK(Put(1, "k1", "v1")); - ASSERT_OK(Put(1, "k2", "v2")); - ASSERT_OK(Put(1, "k3", "v3")); - ASSERT_OK(Put(1, "k4", "v4")); - ASSERT_OK(Delete(1, "k4")); - ASSERT_OK(Put(1, "k5", "v5")); - ASSERT_OK(Delete(1, "no_key")); - - std::vector keys({"k1", "k2", "k3", "k4", "k5", "no_key"}); - - std::vector values(20, "Temporary data to be overwritten"); - std::vector cfs(keys.size(), handles_[1]); - - std::vector s = db_->MultiGet(ReadOptions(), cfs, keys, &values); - ASSERT_EQ(values.size(), keys.size()); - ASSERT_EQ(values[0], "v1"); - ASSERT_EQ(values[1], "v2"); - ASSERT_EQ(values[2], "v3"); - ASSERT_EQ(values[4], "v5"); - - ASSERT_OK(s[0]); - ASSERT_OK(s[1]); - ASSERT_OK(s[2]); - ASSERT_TRUE(s[3].IsNotFound()); - ASSERT_OK(s[4]); - ASSERT_TRUE(s[5].IsNotFound()); - } while (ChangeCompactOptions()); -} +TEST_F(DBTest, TestLogCleanup) { + Options options = CurrentOptions(); + options.write_buffer_size = 64 * 1024; // very small + // only two memtables allowed ==> only two log files + options.max_write_buffer_number = 2; + Reopen(options); -TEST(DBTest, MultiGetEmpty) { - do { - CreateAndReopenWithCF({"pikachu"}); - // Empty Key Set - std::vector keys; - std::vector values; - std::vector cfs; - std::vector s = db_->MultiGet(ReadOptions(), cfs, keys, &values); - ASSERT_EQ(s.size(), 0U); - - // Empty Database, Empty Key Set - DestroyAndReopen(); - CreateAndReopenWithCF({"pikachu"}); - s = db_->MultiGet(ReadOptions(), cfs, keys, &values); - ASSERT_EQ(s.size(), 0U); - - // Empty Database, Search for Keys - keys.resize(2); - keys[0] = "a"; - keys[1] = "b"; - cfs.push_back(handles_[0]); - cfs.push_back(handles_[1]); - s = db_->MultiGet(ReadOptions(), cfs, keys, &values); - ASSERT_EQ((int)s.size(), 2); - ASSERT_TRUE(s[0].IsNotFound() && s[1].IsNotFound()); - } while (ChangeCompactOptions()); + for (int i = 0; i < 100000; ++i) { + Put(Key(i), "val"); + // only 2 memtables will be alive, so logs_to_free needs to always be below + // 2 + ASSERT_LT(dbfull()->TEST_LogsToFreeSize(), static_cast(3)); + } } -namespace { -void PrefixScanInit(DBTest *dbtest) { - char buf[100]; - std::string keystr; - const int small_range_sstfiles = 5; - const int big_range_sstfiles = 5; - - // Generate 11 sst files with the following prefix ranges. - // GROUP 0: [0,10] (level 1) - // GROUP 1: [1,2], [2,3], [3,4], [4,5], [5, 6] (level 0) - // GROUP 2: [0,6], [0,7], [0,8], [0,9], [0,10] (level 0) - // - // A seek with the previous API would do 11 random I/Os (to all the - // files). With the new API and a prefix filter enabled, we should - // only do 2 random I/O, to the 2 files containing the key. - - // GROUP 0 - snprintf(buf, sizeof(buf), "%02d______:start", 0); - keystr = std::string(buf); - ASSERT_OK(dbtest->Put(keystr, keystr)); - snprintf(buf, sizeof(buf), "%02d______:end", 10); - keystr = std::string(buf); - ASSERT_OK(dbtest->Put(keystr, keystr)); - dbtest->Flush(); - dbtest->dbfull()->CompactRange(nullptr, nullptr); // move to level 1 - - // GROUP 1 - for (int i = 1; i <= small_range_sstfiles; i++) { - snprintf(buf, sizeof(buf), "%02d______:start", i); - keystr = std::string(buf); - ASSERT_OK(dbtest->Put(keystr, keystr)); - snprintf(buf, sizeof(buf), "%02d______:end", i+1); - keystr = std::string(buf); - ASSERT_OK(dbtest->Put(keystr, keystr)); - dbtest->Flush(); - } - - // GROUP 2 - for (int i = 1; i <= big_range_sstfiles; i++) { - std::string keystr; - snprintf(buf, sizeof(buf), "%02d______:start", 0); - keystr = std::string(buf); - ASSERT_OK(dbtest->Put(keystr, keystr)); - snprintf(buf, sizeof(buf), "%02d______:end", - small_range_sstfiles+i+1); - keystr = std::string(buf); - ASSERT_OK(dbtest->Put(keystr, keystr)); - dbtest->Flush(); - } +#ifndef ROCKSDB_LITE +TEST_F(DBTest, EmptyCompactedDB) { + Options options = CurrentOptions(); + options.max_open_files = -1; + Close(); + ASSERT_OK(ReadOnlyReopen(options)); + Status s = Put("new", "value"); + ASSERT_TRUE(s.IsNotSupported()); + Close(); } -} // namespace +#endif // ROCKSDB_LITE + +#ifndef ROCKSDB_LITE +TEST_F(DBTest, SuggestCompactRangeTest) { + class CompactionFilterFactoryGetContext : public CompactionFilterFactory { + public: + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + saved_context = context; + std::unique_ptr empty_filter; + return empty_filter; + } + const char* Name() const override { + return "CompactionFilterFactoryGetContext"; + } + static bool IsManual(CompactionFilterFactory* compaction_filter_factory) { + return reinterpret_cast( + compaction_filter_factory) + ->saved_context.is_manual_compaction; + } + CompactionFilter::Context saved_context; + }; -TEST(DBTest, PrefixScan) { - int count; - Slice prefix; - Slice key; - char buf[100]; - Iterator* iter; - snprintf(buf, sizeof(buf), "03______:"); - prefix = Slice(buf, 8); - key = Slice(buf, 9); - // db configs - env_->count_random_reads_ = true; Options options = CurrentOptions(); - options.env = env_; - options.prefix_extractor.reset(NewFixedPrefixTransform(8)); - options.disable_auto_compactions = true; - options.max_background_compactions = 2; - options.create_if_missing = true; - options.memtable_factory.reset(NewHashSkipListRepFactory(16)); + options.memtable_factory.reset( + new SpecialSkipListFactory(DBTestBase::kNumKeysByGenerateNewRandomFile)); + options.compaction_style = kCompactionStyleLevel; + options.compaction_filter_factory.reset( + new CompactionFilterFactoryGetContext()); + options.write_buffer_size = 200 << 10; + options.arena_block_size = 4 << 10; + options.level0_file_num_compaction_trigger = 4; + options.num_levels = 4; + options.compression = kNoCompression; + options.max_bytes_for_level_base = 450 << 10; + options.target_file_size_base = 98 << 10; + options.max_compaction_bytes = static_cast(1) << 60; // inf - BlockBasedTableOptions table_options; - table_options.no_block_cache = true; - table_options.filter_policy.reset(NewBloomFilterPolicy(10)); - table_options.whole_key_filtering = false; - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + Reopen(options); - // 11 RAND I/Os - DestroyAndReopen(&options); - PrefixScanInit(this); - count = 0; - env_->random_read_counter_.Reset(); - iter = db_->NewIterator(ReadOptions()); - for (iter->Seek(prefix); iter->Valid(); iter->Next()) { - if (! iter->key().starts_with(prefix)) { - break; - } - count++; + Random rnd(301); + + for (int num = 0; num < 3; num++) { + GenerateNewRandomFile(&rnd); } - ASSERT_OK(iter->status()); - delete iter; - ASSERT_EQ(count, 2); - ASSERT_EQ(env_->random_read_counter_.Read(), 2); - Close(); -} -TEST(DBTest, TailingIteratorSingle) { - ReadOptions read_options; - read_options.tailing = true; + GenerateNewRandomFile(&rnd); + ASSERT_EQ("0,4", FilesPerLevel(0)); + ASSERT_TRUE(!CompactionFilterFactoryGetContext::IsManual( + options.compaction_filter_factory.get())); - std::unique_ptr iter(db_->NewIterator(read_options)); - iter->SeekToFirst(); - ASSERT_TRUE(!iter->Valid()); + GenerateNewRandomFile(&rnd); + ASSERT_EQ("1,4", FilesPerLevel(0)); - // add a record and check that iter can see it - ASSERT_OK(db_->Put(WriteOptions(), "mirko", "fodor")); - iter->SeekToFirst(); - ASSERT_TRUE(iter->Valid()); - ASSERT_EQ(iter->key().ToString(), "mirko"); + GenerateNewRandomFile(&rnd); + ASSERT_EQ("2,4", FilesPerLevel(0)); - iter->Next(); - ASSERT_TRUE(!iter->Valid()); -} + GenerateNewRandomFile(&rnd); + ASSERT_EQ("3,4", FilesPerLevel(0)); -TEST(DBTest, TailingIteratorKeepAdding) { - CreateAndReopenWithCF({"pikachu"}); - ReadOptions read_options; - read_options.tailing = true; + GenerateNewRandomFile(&rnd); + ASSERT_EQ("0,4,4", FilesPerLevel(0)); - std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); - std::string value(1024, 'a'); + GenerateNewRandomFile(&rnd); + ASSERT_EQ("1,4,4", FilesPerLevel(0)); - const int num_records = 10000; - for (int i = 0; i < num_records; ++i) { - char buf[32]; - snprintf(buf, sizeof(buf), "%016d", i); + GenerateNewRandomFile(&rnd); + ASSERT_EQ("2,4,4", FilesPerLevel(0)); - Slice key(buf, 16); - ASSERT_OK(Put(1, key, value)); + GenerateNewRandomFile(&rnd); + ASSERT_EQ("3,4,4", FilesPerLevel(0)); - iter->Seek(key); - ASSERT_TRUE(iter->Valid()); - ASSERT_EQ(iter->key().compare(key), 0); + GenerateNewRandomFile(&rnd); + ASSERT_EQ("0,4,8", FilesPerLevel(0)); + + GenerateNewRandomFile(&rnd); + ASSERT_EQ("1,4,8", FilesPerLevel(0)); + + // compact it three times + for (int i = 0; i < 3; ++i) { + ASSERT_OK(experimental::SuggestCompactRange(db_, nullptr, nullptr)); + dbfull()->TEST_WaitForCompact(); } -} -TEST(DBTest, TailingIteratorSeekToNext) { - CreateAndReopenWithCF({"pikachu"}); - ReadOptions read_options; - read_options.tailing = true; + // All files are compacted + ASSERT_EQ(0, NumTableFilesAtLevel(0)); + ASSERT_EQ(0, NumTableFilesAtLevel(1)); - std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); - std::string value(1024, 'a'); + GenerateNewRandomFile(&rnd); + ASSERT_EQ(1, NumTableFilesAtLevel(0)); - const int num_records = 1000; - for (int i = 1; i < num_records; ++i) { - char buf1[32]; - char buf2[32]; - snprintf(buf1, sizeof(buf1), "00a0%016d", i * 5); + // nonoverlapping with the file on level 0 + Slice start("a"), end("b"); + ASSERT_OK(experimental::SuggestCompactRange(db_, &start, &end)); + dbfull()->TEST_WaitForCompact(); - Slice key(buf1, 20); - ASSERT_OK(Put(1, key, value)); + // should not compact the level 0 file + ASSERT_EQ(1, NumTableFilesAtLevel(0)); - if (i % 100 == 99) { - ASSERT_OK(Flush(1)); + start = Slice("j"); + end = Slice("m"); + ASSERT_OK(experimental::SuggestCompactRange(db_, &start, &end)); + dbfull()->TEST_WaitForCompact(); + ASSERT_TRUE(CompactionFilterFactoryGetContext::IsManual( + options.compaction_filter_factory.get())); + + // now it should compact the level 0 file + ASSERT_EQ(0, NumTableFilesAtLevel(0)); + ASSERT_EQ(1, NumTableFilesAtLevel(1)); +} + +TEST_F(DBTest, PromoteL0) { + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.write_buffer_size = 10 * 1024 * 1024; + DestroyAndReopen(options); + + // non overlapping ranges + std::vector> ranges = { + {81, 160}, {0, 80}, {161, 240}, {241, 320}}; + + int32_t value_size = 10 * 1024; // 10 KB + + Random rnd(301); + std::map values; + for (const auto& range : ranges) { + for (int32_t j = range.first; j < range.second; j++) { + values[j] = RandomString(&rnd, value_size); + ASSERT_OK(Put(Key(j), values[j])); } + ASSERT_OK(Flush()); + } - snprintf(buf2, sizeof(buf2), "00a0%016d", i * 5 - 2); - Slice target(buf2, 20); - iter->Seek(target); - ASSERT_TRUE(iter->Valid()); - ASSERT_EQ(iter->key().compare(key), 0); + int32_t level0_files = NumTableFilesAtLevel(0, 0); + ASSERT_EQ(level0_files, ranges.size()); + ASSERT_EQ(NumTableFilesAtLevel(1, 0), 0); // No files in L1 + + // Promote L0 level to L2. + ASSERT_OK(experimental::PromoteL0(db_, db_->DefaultColumnFamily(), 2)); + // We expect that all the files were trivially moved from L0 to L2 + ASSERT_EQ(NumTableFilesAtLevel(0, 0), 0); + ASSERT_EQ(NumTableFilesAtLevel(2, 0), level0_files); + + for (const auto& kv : values) { + ASSERT_EQ(Get(Key(kv.first)), kv.second); + } +} + +TEST_F(DBTest, PromoteL0Failure) { + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.write_buffer_size = 10 * 1024 * 1024; + DestroyAndReopen(options); + + // Produce two L0 files with overlapping ranges. + ASSERT_OK(Put(Key(0), "")); + ASSERT_OK(Put(Key(3), "")); + ASSERT_OK(Flush()); + ASSERT_OK(Put(Key(1), "")); + ASSERT_OK(Flush()); + + Status status; + // Fails because L0 has overlapping files. + status = experimental::PromoteL0(db_, db_->DefaultColumnFamily()); + ASSERT_TRUE(status.IsInvalidArgument()); + + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + // Now there is a file in L1. + ASSERT_GE(NumTableFilesAtLevel(1, 0), 1); + + ASSERT_OK(Put(Key(5), "")); + ASSERT_OK(Flush()); + // Fails because L1 is non-empty. + status = experimental::PromoteL0(db_, db_->DefaultColumnFamily()); + ASSERT_TRUE(status.IsInvalidArgument()); +} +#endif // ROCKSDB_LITE + +// Github issue #596 +TEST_F(DBTest, HugeNumberOfLevels) { + Options options = CurrentOptions(); + options.write_buffer_size = 2 * 1024 * 1024; // 2MB + options.max_bytes_for_level_base = 2 * 1024 * 1024; // 2MB + options.num_levels = 12; + options.max_background_compactions = 10; + options.max_bytes_for_level_multiplier = 2; + options.level_compaction_dynamic_level_bytes = true; + DestroyAndReopen(options); + + Random rnd(301); + for (int i = 0; i < 300000; ++i) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 1024))); } - for (int i = 2 * num_records; i > 0; --i) { - char buf1[32]; - char buf2[32]; - snprintf(buf1, sizeof(buf1), "00a0%016d", i * 5); - Slice key(buf1, 20); - ASSERT_OK(Put(1, key, value)); + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); +} - if (i % 100 == 99) { - ASSERT_OK(Flush(1)); - } +TEST_F(DBTest, AutomaticConflictsWithManualCompaction) { + Options options = CurrentOptions(); + options.write_buffer_size = 2 * 1024 * 1024; // 2MB + options.max_bytes_for_level_base = 2 * 1024 * 1024; // 2MB + options.num_levels = 12; + options.max_background_compactions = 10; + options.max_bytes_for_level_multiplier = 2; + options.level_compaction_dynamic_level_bytes = true; + DestroyAndReopen(options); + + Random rnd(301); + for (int i = 0; i < 300000; ++i) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 1024))); + } - snprintf(buf2, sizeof(buf2), "00a0%016d", i * 5 - 2); - Slice target(buf2, 20); - iter->Seek(target); - ASSERT_TRUE(iter->Valid()); - ASSERT_EQ(iter->key().compare(key), 0); + std::atomic callback_count(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction()::Conflict", + [&](void* arg) { callback_count.fetch_add(1); }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + CompactRangeOptions croptions; + croptions.exclusive_manual_compaction = false; + ASSERT_OK(db_->CompactRange(croptions, nullptr, nullptr)); + ASSERT_GE(callback_count.load(), 1); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + for (int i = 0; i < 300000; ++i) { + ASSERT_NE("NOT_FOUND", Get(Key(i))); } } -TEST(DBTest, TailingIteratorDeletes) { - CreateAndReopenWithCF({"pikachu"}); - ReadOptions read_options; - read_options.tailing = true; +// Github issue #595 +// Large write batch with column families +TEST_F(DBTest, LargeBatchWithColumnFamilies) { + Options options = CurrentOptions(); + options.env = env_; + options.write_buffer_size = 100000; // Small write buffer + CreateAndReopenWithCF({"pikachu"}, options); + int64_t j = 0; + for (int i = 0; i < 5; i++) { + for (int pass = 1; pass <= 3; pass++) { + WriteBatch batch; + size_t write_size = 1024 * 1024 * (5 + i); + fprintf(stderr, "prepare: %" ROCKSDB_PRIszt " MB, pass:%d\n", + (write_size / 1024 / 1024), pass); + for (;;) { + std::string data(3000, j++ % 127 + 20); + data += ToString(j); + batch.Put(handles_[0], Slice(data), Slice(data)); + if (batch.GetDataSize() > write_size) { + break; + } + } + fprintf(stderr, "write: %" ROCKSDB_PRIszt " MB\n", + (batch.GetDataSize() / 1024 / 1024)); + ASSERT_OK(dbfull()->Write(WriteOptions(), &batch)); + fprintf(stderr, "done\n"); + } + } + // make sure we can re-open it. + ASSERT_OK(TryReopenWithColumnFamilies({"default", "pikachu"}, options)); +} - std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); +// Make sure that Flushes can proceed in parallel with CompactRange() +TEST_F(DBTest, FlushesInParallelWithCompactRange) { + // iter == 0 -- leveled + // iter == 1 -- leveled, but throw in a flush between two levels compacting + // iter == 2 -- universal + for (int iter = 0; iter < 3; ++iter) { + Options options = CurrentOptions(); + if (iter < 2) { + options.compaction_style = kCompactionStyleLevel; + } else { + options.compaction_style = kCompactionStyleUniversal; + } + options.write_buffer_size = 110 << 10; + options.level0_file_num_compaction_trigger = 4; + options.num_levels = 4; + options.compression = kNoCompression; + options.max_bytes_for_level_base = 450 << 10; + options.target_file_size_base = 98 << 10; + options.max_write_buffer_number = 2; - // write a single record, read it using the iterator, then delete it - ASSERT_OK(Put(1, "0test", "test")); - iter->SeekToFirst(); - ASSERT_TRUE(iter->Valid()); - ASSERT_EQ(iter->key().ToString(), "0test"); - ASSERT_OK(Delete(1, "0test")); + DestroyAndReopen(options); - // write many more records - const int num_records = 10000; - std::string value(1024, 'A'); + Random rnd(301); + for (int num = 0; num < 14; num++) { + GenerateNewRandomFile(&rnd); + } - for (int i = 0; i < num_records; ++i) { - char buf[32]; - snprintf(buf, sizeof(buf), "1%015d", i); + if (iter == 1) { + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::RunManualCompaction()::1", + "DBTest::FlushesInParallelWithCompactRange:1"}, + {"DBTest::FlushesInParallelWithCompactRange:2", + "DBImpl::RunManualCompaction()::2"}}); + } else { + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"CompactionJob::Run():Start", + "DBTest::FlushesInParallelWithCompactRange:1"}, + {"DBTest::FlushesInParallelWithCompactRange:2", + "CompactionJob::Run():End"}}); + } + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); - Slice key(buf, 16); - ASSERT_OK(Put(1, key, value)); - } + std::vector threads; + threads.emplace_back([&]() { Compact("a", "z"); }); - // force a flush to make sure that no records are read from memtable - ASSERT_OK(Flush(1)); + TEST_SYNC_POINT("DBTest::FlushesInParallelWithCompactRange:1"); - // skip "0test" - iter->Next(); + // this has to start a flush. if flushes are blocked, this will try to + // create + // 3 memtables, and that will fail because max_write_buffer_number is 2 + for (int num = 0; num < 3; num++) { + GenerateNewRandomFile(&rnd, /* nowait */ true); + } - // make sure we can read all new records using the existing iterator - int count = 0; - for (; iter->Valid(); iter->Next(), ++count) ; + TEST_SYNC_POINT("DBTest::FlushesInParallelWithCompactRange:2"); - ASSERT_EQ(count, num_records); + for (auto& t : threads) { + t.join(); + } + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + } } -TEST(DBTest, TailingIteratorPrefixSeek) { - ReadOptions read_options; - read_options.tailing = true; +TEST_F(DBTest, DelayedWriteRate) { + const int kEntriesPerMemTable = 100; + const int kTotalFlushes = 12; Options options = CurrentOptions(); + env_->SetBackgroundThreads(1, Env::LOW); options.env = env_; - options.create_if_missing = true; - options.disable_auto_compactions = true; - options.prefix_extractor.reset(NewFixedPrefixTransform(2)); - options.memtable_factory.reset(NewHashSkipListRepFactory(16)); - DestroyAndReopen(&options); - CreateAndReopenWithCF({"pikachu"}, &options); - - std::unique_ptr iter(db_->NewIterator(read_options, handles_[1])); - ASSERT_OK(Put(1, "0101", "test")); - - ASSERT_OK(Flush(1)); + env_->no_slowdown_ = true; + options.write_buffer_size = 100000000; + options.max_write_buffer_number = 256; + options.max_background_compactions = 1; + options.level0_file_num_compaction_trigger = 3; + options.level0_slowdown_writes_trigger = 3; + options.level0_stop_writes_trigger = 999999; + options.delayed_write_rate = 20000000; // Start with 200MB/s + options.memtable_factory.reset( + new SpecialSkipListFactory(kEntriesPerMemTable)); - ASSERT_OK(Put(1, "0202", "test")); + CreateAndReopenWithCF({"pikachu"}, options); - // Seek(0102) shouldn't find any records since 0202 has a different prefix - iter->Seek("0102"); - ASSERT_TRUE(!iter->Valid()); + // Block compactions + test::SleepingBackgroundTask sleeping_task_low; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); - iter->Seek("0202"); - ASSERT_TRUE(iter->Valid()); - ASSERT_EQ(iter->key().ToString(), "0202"); + for (int i = 0; i < 3; i++) { + Put(Key(i), std::string(10000, 'x')); + Flush(); + } - iter->Next(); - ASSERT_TRUE(!iter->Valid()); + // These writes will be slowed down to 1KB/s + uint64_t estimated_sleep_time = 0; + Random rnd(301); + Put("", ""); + uint64_t cur_rate = options.delayed_write_rate; + for (int i = 0; i < kTotalFlushes; i++) { + uint64_t size_memtable = 0; + for (int j = 0; j < kEntriesPerMemTable; j++) { + auto rand_num = rnd.Uniform(20); + // Spread the size range to more. + size_t entry_size = rand_num * rand_num * rand_num; + WriteOptions wo; + Put(Key(i), std::string(entry_size, 'x'), wo); + size_memtable += entry_size + 18; + // Occasionally sleep a while + if (rnd.Uniform(20) == 6) { + env_->SleepForMicroseconds(2666); + } + } + dbfull()->TEST_WaitForFlushMemTable(); + estimated_sleep_time += size_memtable * 1000000u / cur_rate; + // Slow down twice. One for memtable switch and one for flush finishes. + cur_rate = static_cast(static_cast(cur_rate) * + kIncSlowdownRatio * kIncSlowdownRatio); + } + // Estimate the total sleep time fall into the rough range. + ASSERT_GT(env_->addon_time_.load(), + static_cast(estimated_sleep_time / 2)); + ASSERT_LT(env_->addon_time_.load(), + static_cast(estimated_sleep_time * 2)); + + env_->no_slowdown_ = false; + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilDone(); } -TEST(DBTest, TailingIteratorIncomplete) { - CreateAndReopenWithCF({"pikachu"}); - ReadOptions read_options; - read_options.tailing = true; - read_options.read_tier = kBlockCacheTier; +TEST_F(DBTest, HardLimit) { + Options options = CurrentOptions(); + options.env = env_; + env_->SetBackgroundThreads(1, Env::LOW); + options.max_write_buffer_number = 256; + options.write_buffer_size = 110 << 10; // 110KB + options.arena_block_size = 4 * 1024; + options.level0_file_num_compaction_trigger = 4; + options.level0_slowdown_writes_trigger = 999999; + options.level0_stop_writes_trigger = 999999; + options.hard_pending_compaction_bytes_limit = 800 << 10; + options.max_bytes_for_level_base = 10000000000u; + options.max_background_compactions = 1; + options.memtable_factory.reset( + new SpecialSkipListFactory(KNumKeysByGenerateNewFile - 1)); - std::string key("key"); - std::string value("value"); + env_->SetBackgroundThreads(1, Env::LOW); + test::SleepingBackgroundTask sleeping_task_low; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); - ASSERT_OK(db_->Put(WriteOptions(), key, value)); + CreateAndReopenWithCF({"pikachu"}, options); - std::unique_ptr iter(db_->NewIterator(read_options)); - iter->SeekToFirst(); - // we either see the entry or it's not in cache - ASSERT_TRUE(iter->Valid() || iter->status().IsIncomplete()); - - ASSERT_OK(db_->CompactRange(nullptr, nullptr)); - iter->SeekToFirst(); - // should still be true after compaction - ASSERT_TRUE(iter->Valid() || iter->status().IsIncomplete()); -} + std::atomic callback_count(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack("DBImpl::DelayWrite:Wait", + [&](void* arg) { + callback_count.fetch_add(1); + sleeping_task_low.WakeUp(); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); -TEST(DBTest, TailingIteratorSeekToSame) { - Options options = CurrentOptions(); - options.compaction_style = kCompactionStyleUniversal; - options.write_buffer_size = 1000; - CreateAndReopenWithCF({"pikachu"}, &options); + Random rnd(301); + int key_idx = 0; + for (int num = 0; num < 5; num++) { + GenerateNewFile(&rnd, &key_idx, true); + dbfull()->TEST_WaitForFlushMemTable(); + } - ReadOptions read_options; - read_options.tailing = true; + ASSERT_EQ(0, callback_count.load()); - const int NROWS = 10000; - // Write rows with keys 00000, 00002, 00004 etc. - for (int i = 0; i < NROWS; ++i) { - char buf[100]; - snprintf(buf, sizeof(buf), "%05d", 2*i); - std::string key(buf); - std::string value("value"); - ASSERT_OK(db_->Put(WriteOptions(), key, value)); + for (int num = 0; num < 5; num++) { + GenerateNewFile(&rnd, &key_idx, true); + dbfull()->TEST_WaitForFlushMemTable(); } + ASSERT_GE(callback_count.load(), 1); - std::unique_ptr iter(db_->NewIterator(read_options)); - // Seek to 00001. We expect to find 00002. - std::string start_key = "00001"; - iter->Seek(start_key); - ASSERT_TRUE(iter->Valid()); - - std::string found = iter->key().ToString(); - ASSERT_EQ("00002", found); - - // Now seek to the same key. The iterator should remain in the same - // position. - iter->Seek(found); - ASSERT_TRUE(iter->Valid()); - ASSERT_EQ(found, iter->key().ToString()); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + sleeping_task_low.WaitUntilDone(); } -TEST(DBTest, BlockBasedTablePrefixIndexTest) { - // create a DB with block prefix index - BlockBasedTableOptions table_options; +#ifndef ROCKSDB_LITE +TEST_F(DBTest, SoftLimit) { Options options = CurrentOptions(); - table_options.index_type = BlockBasedTableOptions::kHashSearch; - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - options.prefix_extractor.reset(NewFixedPrefixTransform(1)); - + options.env = env_; + options.write_buffer_size = 100000; // Small write buffer + options.max_write_buffer_number = 256; + options.level0_file_num_compaction_trigger = 1; + options.level0_slowdown_writes_trigger = 3; + options.level0_stop_writes_trigger = 999999; + options.delayed_write_rate = 20000; // About 200KB/s limited rate + options.soft_pending_compaction_bytes_limit = 160000; + options.target_file_size_base = 99999999; // All into one file + options.max_bytes_for_level_base = 50000; + options.max_bytes_for_level_multiplier = 10; + options.max_background_compactions = 1; + options.compression = kNoCompression; - Reopen(&options); - ASSERT_OK(Put("k1", "v1")); - Flush(); - ASSERT_OK(Put("k2", "v2")); + Reopen(options); - // Reopen it without prefix extractor, make sure everything still works. - // RocksDB should just fall back to the binary index. - table_options.index_type = BlockBasedTableOptions::kBinarySearch; - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - options.prefix_extractor.reset(); + // Generating 360KB in Level 3 + for (int i = 0; i < 72; i++) { + Put(Key(i), std::string(5000, 'x')); + if (i % 10 == 0) { + Flush(); + } + } + dbfull()->TEST_WaitForCompact(); + MoveFilesToLevel(3); - Reopen(&options); - ASSERT_EQ("v1", Get("k1")); - ASSERT_EQ("v2", Get("k2")); -} + // Generating 360KB in Level 2 + for (int i = 0; i < 72; i++) { + Put(Key(i), std::string(5000, 'x')); + if (i % 10 == 0) { + Flush(); + } + } + dbfull()->TEST_WaitForCompact(); + MoveFilesToLevel(2); -TEST(DBTest, ChecksumTest) { - BlockBasedTableOptions table_options; - Options options = CurrentOptions(); + Put(Key(0), ""); - table_options.checksum = kCRC32c; - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - Reopen(&options); - ASSERT_OK(Put("a", "b")); - ASSERT_OK(Put("c", "d")); - ASSERT_OK(Flush()); // table with crc checksum + test::SleepingBackgroundTask sleeping_task_low; + // Block compactions + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + sleeping_task_low.WaitUntilSleeping(); - table_options.checksum = kxxHash; - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - Reopen(&options); - ASSERT_OK(Put("e", "f")); - ASSERT_OK(Put("g", "h")); - ASSERT_OK(Flush()); // table with xxhash checksum + // Create 3 L0 files, making score of L0 to be 3. + for (int i = 0; i < 3; i++) { + Put(Key(i), std::string(5000, 'x')); + Put(Key(100 - i), std::string(5000, 'x')); + // Flush the file. File size is around 30KB. + Flush(); + } + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); - table_options.checksum = kCRC32c; - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - Reopen(&options); - ASSERT_EQ("b", Get("a")); - ASSERT_EQ("d", Get("c")); - ASSERT_EQ("f", Get("e")); - ASSERT_EQ("h", Get("g")); + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilDone(); + sleeping_task_low.Reset(); + dbfull()->TEST_WaitForCompact(); - table_options.checksum = kCRC32c; - options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - Reopen(&options); - ASSERT_EQ("b", Get("a")); - ASSERT_EQ("d", Get("c")); - ASSERT_EQ("f", Get("e")); - ASSERT_EQ("h", Get("g")); -} + // Now there is one L1 file but doesn't trigger soft_rate_limit + // The L1 file size is around 30KB. + ASSERT_EQ(NumTableFilesAtLevel(1), 1); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + + // Only allow one compactin going through. + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "BackgroundCallCompaction:0", [&](void* arg) { + // Schedule a sleeping task. + sleeping_task_low.Reset(); + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, + &sleeping_task_low, Env::Priority::LOW); + }); -TEST(DBTest, FIFOCompactionTest) { - for (int iter = 0; iter < 2; ++iter) { - // first iteration -- auto compaction - // second iteration -- manual compaction - Options options; - options.compaction_style = kCompactionStyleFIFO; - options.write_buffer_size = 100 << 10; // 100KB - options.compaction_options_fifo.max_table_files_size = 500 << 10; // 500KB - options.compression = kNoCompression; - options.create_if_missing = true; - if (iter == 1) { - options.disable_auto_compactions = true; - } - DestroyAndReopen(&options); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); - Random rnd(301); - for (int i = 0; i < 6; ++i) { - for (int j = 0; j < 100; ++j) { - ASSERT_OK(Put(std::to_string(i * 100 + j), RandomString(&rnd, 1024))); - } - // flush should happen here - } - if (iter == 0) { - ASSERT_OK(dbfull()->TEST_WaitForCompact()); - } else { - ASSERT_OK(db_->CompactRange(nullptr, nullptr)); - } - // only 5 files should survive - ASSERT_EQ(NumTableFilesAtLevel(0), 5); - for (int i = 0; i < 50; ++i) { - // these keys should be deleted in previous compaction - ASSERT_EQ("NOT_FOUND", Get(std::to_string(i))); - } + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + sleeping_task_low.WaitUntilSleeping(); + // Create 3 L0 files, making score of L0 to be 3 + for (int i = 0; i < 3; i++) { + Put(Key(10 + i), std::string(5000, 'x')); + Put(Key(90 - i), std::string(5000, 'x')); + // Flush the file. File size is around 30KB. + Flush(); } -} -TEST(DBTest, SimpleWriteTimeoutTest) { - Options options; - options.env = env_; - options.create_if_missing = true; - options.write_buffer_size = 100000; - options.max_background_flushes = 0; - options.max_write_buffer_number = 2; - options.min_write_buffer_number_to_merge = 3; - options.max_total_wal_size = std::numeric_limits::max(); - WriteOptions write_opt = WriteOptions(); - write_opt.timeout_hint_us = 0; - DestroyAndReopen(&options); - // fill the two write buffer - ASSERT_OK(Put(Key(1), Key(1) + std::string(100000, 'v'), write_opt)); - ASSERT_OK(Put(Key(2), Key(2) + std::string(100000, 'v'), write_opt)); - // As the only two write buffers are full in this moment, the third - // Put is expected to be timed-out. - write_opt.timeout_hint_us = 50; - ASSERT_TRUE( - Put(Key(3), Key(3) + std::string(100000, 'v'), write_opt).IsTimedOut()); -} + // Wake up sleep task to enable compaction to run and waits + // for it to go to sleep state again to make sure one compaction + // goes through. + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilSleeping(); + + // Now there is one L1 file (around 60KB) which exceeds 50KB base by 10KB + // Given level multiplier 10, estimated pending compaction is around 100KB + // doesn't trigger soft_pending_compaction_bytes_limit + ASSERT_EQ(NumTableFilesAtLevel(1), 1); + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); + + // Create 3 L0 files, making score of L0 to be 3, higher than L0. + for (int i = 0; i < 3; i++) { + Put(Key(20 + i), std::string(5000, 'x')); + Put(Key(80 - i), std::string(5000, 'x')); + // Flush the file. File size is around 30KB. + Flush(); + } + // Wake up sleep task to enable compaction to run and waits + // for it to go to sleep state again to make sure one compaction + // goes through. + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilSleeping(); -// Multi-threaded Timeout Test -namespace { + // Now there is one L1 file (around 90KB) which exceeds 50KB base by 40KB + // L2 size is 360KB, so the estimated level fanout 4, estimated pending + // compaction is around 200KB + // triggerring soft_pending_compaction_bytes_limit + ASSERT_EQ(NumTableFilesAtLevel(1), 1); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); -static const int kValueSize = 1000; -static const int kWriteBufferSize = 100000; + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilSleeping(); -struct TimeoutWriterState { - int id; - DB* db; - std::atomic done; - std::map success_kvs; -}; + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); -static void RandomTimeoutWriter(void* arg) { - TimeoutWriterState* state = reinterpret_cast(arg); - static const uint64_t kTimerBias = 50; - int thread_id = state->id; - DB* db = state->db; - - Random rnd(1000 + thread_id); - WriteOptions write_opt = WriteOptions(); - write_opt.timeout_hint_us = 500; - int timeout_count = 0; - int num_keys = kNumKeys * 5; - - for (int k = 0; k < num_keys; ++k) { - int key = k + thread_id * num_keys; - std::string value = RandomString(&rnd, kValueSize); - // only the second-half is randomized - if (k > num_keys / 2) { - switch (rnd.Next() % 5) { - case 0: - write_opt.timeout_hint_us = 500 * thread_id; - break; - case 1: - write_opt.timeout_hint_us = num_keys - k; - break; - case 2: - write_opt.timeout_hint_us = 1; - break; - default: - write_opt.timeout_hint_us = 0; - state->success_kvs.insert({key, value}); - } - } + // shrink level base so L2 will hit soft limit easier. + ASSERT_OK(dbfull()->SetOptions({ + {"max_bytes_for_level_base", "5000"}, + })); - uint64_t time_before_put = db->GetEnv()->NowMicros(); - Status s = db->Put(write_opt, Key(key), value); - uint64_t put_duration = db->GetEnv()->NowMicros() - time_before_put; - if (write_opt.timeout_hint_us == 0 || - put_duration + kTimerBias < write_opt.timeout_hint_us) { - ASSERT_OK(s); - std::string result; - } - if (s.IsTimedOut()) { - timeout_count++; - ASSERT_GT(put_duration + kTimerBias, write_opt.timeout_hint_us); - } - } + Put("", ""); + Flush(); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); - state->done = true; + sleeping_task_low.WaitUntilSleeping(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilDone(); } -TEST(DBTest, MTRandomTimeoutTest) { - Options options; +TEST_F(DBTest, LastWriteBufferDelay) { + Options options = CurrentOptions(); options.env = env_; - options.create_if_missing = true; - options.max_write_buffer_number = 2; + options.write_buffer_size = 100000; + options.max_write_buffer_number = 4; + options.delayed_write_rate = 20000; options.compression = kNoCompression; - options.level0_slowdown_writes_trigger = 10; - options.level0_stop_writes_trigger = 20; - options.write_buffer_size = kWriteBufferSize; - DestroyAndReopen(&options); - - TimeoutWriterState thread_states[kNumThreads]; - for (int tid = 0; tid < kNumThreads; ++tid) { - thread_states[tid].id = tid; - thread_states[tid].db = db_; - thread_states[tid].done = false; - env_->StartThread(RandomTimeoutWriter, &thread_states[tid]); - } + options.disable_auto_compactions = true; + int kNumKeysPerMemtable = 3; + options.memtable_factory.reset( + new SpecialSkipListFactory(kNumKeysPerMemtable)); + + Reopen(options); + test::SleepingBackgroundTask sleeping_task; + // Block flushes + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task, + Env::Priority::HIGH); + sleeping_task.WaitUntilSleeping(); - for (int tid = 0; tid < kNumThreads; ++tid) { - while (thread_states[tid].done == false) { - env_->SleepForMicroseconds(100000); + // Create 3 L0 files, making score of L0 to be 3. + for (int i = 0; i < 3; i++) { + // Fill one mem table + for (int j = 0; j < kNumKeysPerMemtable; j++) { + Put(Key(j), ""); } + ASSERT_TRUE(!dbfull()->TEST_write_controler().NeedsDelay()); } + // Inserting a new entry would create a new mem table, triggering slow down. + Put(Key(0), ""); + ASSERT_TRUE(dbfull()->TEST_write_controler().NeedsDelay()); - Flush(); + sleeping_task.WakeUp(); + sleeping_task.WaitUntilDone(); +} +#endif // ROCKSDB_LITE - for (int tid = 0; tid < kNumThreads; ++tid) { - auto& success_kvs = thread_states[tid].success_kvs; - for (auto it = success_kvs.begin(); it != success_kvs.end(); ++it) { - ASSERT_EQ(Get(Key(it->first)), it->second); +TEST_F(DBTest, FailWhenCompressionNotSupportedTest) { + CompressionType compressions[] = {kZlibCompression, kBZip2Compression, + kLZ4Compression, kLZ4HCCompression, + kXpressCompression}; + for (auto comp : compressions) { + if (!CompressionTypeSupported(comp)) { + // not supported, we should fail the Open() + Options options = CurrentOptions(); + options.compression = comp; + ASSERT_TRUE(!TryReopen(options).ok()); + // Try if CreateColumnFamily also fails + options.compression = kNoCompression; + ASSERT_OK(TryReopen(options)); + ColumnFamilyOptions cf_options(options); + cf_options.compression = comp; + ColumnFamilyHandle* handle; + ASSERT_TRUE(!db_->CreateColumnFamily(cf_options, "name", &handle).ok()); } } } -} // anonymous namespace - -/* - * This test is not reliable enough as it heavily depends on disk behavior. - */ -TEST(DBTest, RateLimitingTest) { +#ifndef ROCKSDB_LITE +TEST_F(DBTest, RowCache) { Options options = CurrentOptions(); - options.write_buffer_size = 1 << 20; // 1MB - options.level0_file_num_compaction_trigger = 2; - options.target_file_size_base = 1 << 20; // 1MB - options.max_bytes_for_level_base = 4 << 20; // 4MB - options.max_bytes_for_level_multiplier = 4; - options.compression = kNoCompression; - options.create_if_missing = true; - options.env = env_; - options.IncreaseParallelism(4); - DestroyAndReopen(&options); + options.statistics = rocksdb::CreateDBStatistics(); + options.row_cache = NewLRUCache(8192); + DestroyAndReopen(options); - WriteOptions wo; - wo.disableWAL = true; + ASSERT_OK(Put("foo", "bar")); + ASSERT_OK(Flush()); - // # no rate limiting - Random rnd(301); - uint64_t start = env_->NowMicros(); - // Write ~96M data - for (int64_t i = 0; i < (96 << 10); ++i) { - ASSERT_OK(Put(RandomString(&rnd, 32), - RandomString(&rnd, (1 << 10) + 1), wo)); - } - uint64_t elapsed = env_->NowMicros() - start; - double raw_rate = env_->bytes_written_ * 1000000 / elapsed; - Close(); + ASSERT_EQ(TestGetTickerCount(options, ROW_CACHE_HIT), 0); + ASSERT_EQ(TestGetTickerCount(options, ROW_CACHE_MISS), 0); + ASSERT_EQ(Get("foo"), "bar"); + ASSERT_EQ(TestGetTickerCount(options, ROW_CACHE_HIT), 0); + ASSERT_EQ(TestGetTickerCount(options, ROW_CACHE_MISS), 1); + ASSERT_EQ(Get("foo"), "bar"); + ASSERT_EQ(TestGetTickerCount(options, ROW_CACHE_HIT), 1); + ASSERT_EQ(TestGetTickerCount(options, ROW_CACHE_MISS), 1); +} - // # rate limiting with 0.7 x threshold - options.rate_limiter.reset( - NewGenericRateLimiter(static_cast(0.7 * raw_rate))); - env_->bytes_written_ = 0; - DestroyAndReopen(&options); +TEST_F(DBTest, PinnableSliceAndRowCache) { + Options options = CurrentOptions(); + options.statistics = rocksdb::CreateDBStatistics(); + options.row_cache = NewLRUCache(8192); + DestroyAndReopen(options); - start = env_->NowMicros(); - // Write ~96M data - for (int64_t i = 0; i < (96 << 10); ++i) { - ASSERT_OK(Put(RandomString(&rnd, 32), - RandomString(&rnd, (1 << 10) + 1), wo)); - } - elapsed = env_->NowMicros() - start; - Close(); - ASSERT_TRUE(options.rate_limiter->GetTotalBytesThrough() == - env_->bytes_written_); - double ratio = env_->bytes_written_ * 1000000 / elapsed / raw_rate; - fprintf(stderr, "write rate ratio = %.2lf, expected 0.7\n", ratio); - ASSERT_TRUE(ratio < 0.8); + ASSERT_OK(Put("foo", "bar")); + ASSERT_OK(Flush()); - // # rate limiting with half of the raw_rate - options.rate_limiter.reset( - NewGenericRateLimiter(static_cast(raw_rate / 2))); - env_->bytes_written_ = 0; - DestroyAndReopen(&options); + ASSERT_EQ(Get("foo"), "bar"); + ASSERT_EQ( + reinterpret_cast(options.row_cache.get())->TEST_GetLRUSize(), + 1); - start = env_->NowMicros(); - // Write ~96M data - for (int64_t i = 0; i < (96 << 10); ++i) { - ASSERT_OK(Put(RandomString(&rnd, 32), - RandomString(&rnd, (1 << 10) + 1), wo)); + { + PinnableSlice pin_slice; + ASSERT_EQ(Get("foo", &pin_slice), Status::OK()); + ASSERT_EQ(pin_slice.ToString(), "bar"); + // Entry is already in cache, lookup will remove the element from lru + ASSERT_EQ( + reinterpret_cast(options.row_cache.get())->TEST_GetLRUSize(), + 0); } - elapsed = env_->NowMicros() - start; - Close(); - ASSERT_TRUE(options.rate_limiter->GetTotalBytesThrough() == - env_->bytes_written_); - ratio = env_->bytes_written_ * 1000000 / elapsed / raw_rate; - fprintf(stderr, "write rate ratio = %.2lf, expected 0.5\n", ratio); - ASSERT_TRUE(ratio < 0.6); + // After PinnableSlice destruction element is added back in LRU + ASSERT_EQ( + reinterpret_cast(options.row_cache.get())->TEST_GetLRUSize(), + 1); } -TEST(DBTest, TableOptionsSanitizeTest) { +#endif // ROCKSDB_LITE + +TEST_F(DBTest, DeletingOldWalAfterDrop) { + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"Test:AllowFlushes", "DBImpl::BGWorkFlush"}, + {"DBImpl::BGWorkFlush:done", "Test:WaitForFlush"}}); + rocksdb::SyncPoint::GetInstance()->ClearTrace(); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); Options options = CurrentOptions(); - options.create_if_missing = true; - DestroyAndReopen(&options); - ASSERT_EQ(db_->GetOptions().allow_mmap_reads, false); + options.max_total_wal_size = 8192; + options.compression = kNoCompression; + options.write_buffer_size = 1 << 20; + options.level0_file_num_compaction_trigger = (1 << 30); + options.level0_slowdown_writes_trigger = (1 << 30); + options.level0_stop_writes_trigger = (1 << 30); + options.disable_auto_compactions = true; + DestroyAndReopen(options); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + CreateColumnFamilies({"cf1", "cf2"}, options); + ASSERT_OK(Put(0, "key1", DummyString(8192))); + ASSERT_OK(Put(0, "key2", DummyString(8192))); + // the oldest wal should now be getting_flushed + ASSERT_OK(db_->DropColumnFamily(handles_[0])); + // all flushes should now do nothing because their CF is dropped + TEST_SYNC_POINT("Test:AllowFlushes"); + TEST_SYNC_POINT("Test:WaitForFlush"); + uint64_t lognum1 = dbfull()->TEST_LogfileNumber(); + ASSERT_OK(Put(1, "key3", DummyString(8192))); + ASSERT_OK(Put(1, "key4", DummyString(8192))); + // new wal should have been created + uint64_t lognum2 = dbfull()->TEST_LogfileNumber(); + EXPECT_GT(lognum2, lognum1); +} + +TEST_F(DBTest, UnsupportedManualSync) { + DestroyAndReopen(CurrentOptions()); + env_->is_wal_sync_thread_safe_.store(false); + Status s = db_->SyncWAL(); + ASSERT_TRUE(s.IsNotSupported()); +} + +INSTANTIATE_TEST_CASE_P(DBTestWithParam, DBTestWithParam, + ::testing::Combine(::testing::Values(1, 4), + ::testing::Bool())); + +TEST_F(DBTest, PauseBackgroundWorkTest) { + Options options = CurrentOptions(); + options.write_buffer_size = 100000; // Small write buffer + Reopen(options); - options.table_factory.reset(new PlainTableFactory()); - options.prefix_extractor.reset(NewNoopTransform()); - Destroy(&options); - ASSERT_TRUE(TryReopen(&options).IsNotSupported()); + std::vector threads; + std::atomic done(false); + db_->PauseBackgroundWork(); + threads.emplace_back([&]() { + Random rnd(301); + for (int i = 0; i < 10000; ++i) { + Put(RandomString(&rnd, 10), RandomString(&rnd, 10)); + } + done.store(true); + }); + env_->SleepForMicroseconds(200000); + // make sure the thread is not done + ASSERT_FALSE(done.load()); + db_->ContinueBackgroundWork(); + for (auto& t : threads) { + t.join(); + } + // now it's done + ASSERT_TRUE(done.load()); } - } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/db_test2.cc b/db/db_test2.cc new file mode 100644 index 00000000000..30afd5a690c --- /dev/null +++ b/db/db_test2.cc @@ -0,0 +1,2340 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#include +#include +#include + +#include "db/db_test_util.h" +#include "port/port.h" +#include "port/stack_trace.h" +#include "rocksdb/persistent_cache.h" +#include "rocksdb/wal_filter.h" + +namespace rocksdb { + +class DBTest2 : public DBTestBase { + public: + DBTest2() : DBTestBase("/db_test2") {} +}; + +class PrefixFullBloomWithReverseComparator + : public DBTestBase, + public ::testing::WithParamInterface { + public: + PrefixFullBloomWithReverseComparator() + : DBTestBase("/prefix_bloom_reverse") {} + virtual void SetUp() override { if_cache_filter_ = GetParam(); } + bool if_cache_filter_; +}; + +TEST_P(PrefixFullBloomWithReverseComparator, + PrefixFullBloomWithReverseComparator) { + Options options = last_options_; + options.comparator = ReverseBytewiseComparator(); + options.prefix_extractor.reset(NewCappedPrefixTransform(3)); + options.statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions bbto; + if (if_cache_filter_) { + bbto.no_block_cache = false; + bbto.cache_index_and_filter_blocks = true; + bbto.block_cache = NewLRUCache(1); + } + bbto.filter_policy.reset(NewBloomFilterPolicy(10, false)); + bbto.whole_key_filtering = false; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + DestroyAndReopen(options); + + ASSERT_OK(dbfull()->Put(WriteOptions(), "bar123", "foo")); + ASSERT_OK(dbfull()->Put(WriteOptions(), "bar234", "foo2")); + ASSERT_OK(dbfull()->Put(WriteOptions(), "foo123", "foo3")); + + dbfull()->Flush(FlushOptions()); + + if (bbto.block_cache) { + bbto.block_cache->EraseUnRefEntries(); + } + + unique_ptr iter(db_->NewIterator(ReadOptions())); + iter->Seek("bar345"); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ("bar234", iter->key().ToString()); + ASSERT_EQ("foo2", iter->value().ToString()); + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ("bar123", iter->key().ToString()); + ASSERT_EQ("foo", iter->value().ToString()); + + iter->Seek("foo234"); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ("foo123", iter->key().ToString()); + ASSERT_EQ("foo3", iter->value().ToString()); + + iter->Seek("bar"); + ASSERT_OK(iter->status()); + ASSERT_TRUE(!iter->Valid()); +} + +INSTANTIATE_TEST_CASE_P(PrefixFullBloomWithReverseComparator, + PrefixFullBloomWithReverseComparator, testing::Bool()); + +TEST_F(DBTest2, IteratorPropertyVersionNumber) { + Put("", ""); + Iterator* iter1 = db_->NewIterator(ReadOptions()); + std::string prop_value; + ASSERT_OK( + iter1->GetProperty("rocksdb.iterator.super-version-number", &prop_value)); + uint64_t version_number1 = + static_cast(std::atoi(prop_value.c_str())); + + Put("", ""); + Flush(); + + Iterator* iter2 = db_->NewIterator(ReadOptions()); + ASSERT_OK( + iter2->GetProperty("rocksdb.iterator.super-version-number", &prop_value)); + uint64_t version_number2 = + static_cast(std::atoi(prop_value.c_str())); + + ASSERT_GT(version_number2, version_number1); + + Put("", ""); + + Iterator* iter3 = db_->NewIterator(ReadOptions()); + ASSERT_OK( + iter3->GetProperty("rocksdb.iterator.super-version-number", &prop_value)); + uint64_t version_number3 = + static_cast(std::atoi(prop_value.c_str())); + + ASSERT_EQ(version_number2, version_number3); + + iter1->SeekToFirst(); + ASSERT_OK( + iter1->GetProperty("rocksdb.iterator.super-version-number", &prop_value)); + uint64_t version_number1_new = + static_cast(std::atoi(prop_value.c_str())); + ASSERT_EQ(version_number1, version_number1_new); + + delete iter1; + delete iter2; + delete iter3; +} + +TEST_F(DBTest2, CacheIndexAndFilterWithDBRestart) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions table_options; + table_options.cache_index_and_filter_blocks = true; + table_options.filter_policy.reset(NewBloomFilterPolicy(20)); + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + CreateAndReopenWithCF({"pikachu"}, options); + + Put(1, "a", "begin"); + Put(1, "z", "end"); + ASSERT_OK(Flush(1)); + TryReopenWithColumnFamilies({"default", "pikachu"}, options); + + std::string value; + value = Get(1, "a"); +} + +TEST_F(DBTest2, MaxSuccessiveMergesChangeWithDBRecovery) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + options.max_successive_merges = 3; + options.merge_operator = MergeOperators::CreatePutOperator(); + options.disable_auto_compactions = true; + DestroyAndReopen(options); + Put("poi", "Finch"); + db_->Merge(WriteOptions(), "poi", "Reese"); + db_->Merge(WriteOptions(), "poi", "Shaw"); + db_->Merge(WriteOptions(), "poi", "Root"); + options.max_successive_merges = 2; + Reopen(options); +} + +#ifndef ROCKSDB_LITE +class DBTestSharedWriteBufferAcrossCFs + : public DBTestBase, + public testing::WithParamInterface> { + public: + DBTestSharedWriteBufferAcrossCFs() + : DBTestBase("/db_test_shared_write_buffer") {} + void SetUp() override { + use_old_interface_ = std::get<0>(GetParam()); + cost_cache_ = std::get<1>(GetParam()); + } + bool use_old_interface_; + bool cost_cache_; +}; + +TEST_P(DBTestSharedWriteBufferAcrossCFs, SharedWriteBufferAcrossCFs) { + Options options = CurrentOptions(); + options.arena_block_size = 4096; + + // Avoid undeterministic value by malloc_usable_size(); + // Force arena block size to 1 + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "Arena::Arena:0", [&](void* arg) { + size_t* block_size = static_cast(arg); + *block_size = 1; + }); + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "Arena::AllocateNewBlock:0", [&](void* arg) { + std::pair* pair = + static_cast*>(arg); + *std::get<0>(*pair) = *std::get<1>(*pair); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // The total soft write buffer size is about 105000 + std::shared_ptr cache = NewLRUCache(4 * 1024 * 1024, 2); + ASSERT_LT(cache->GetUsage(), 1024 * 1024); + + if (use_old_interface_) { + options.db_write_buffer_size = 120000; // this is the real limit + } else if (!cost_cache_) { + options.write_buffer_manager.reset(new WriteBufferManager(114285)); + } else { + options.write_buffer_manager.reset(new WriteBufferManager(114285, cache)); + } + options.write_buffer_size = 500000; // this is never hit + CreateAndReopenWithCF({"pikachu", "dobrynia", "nikitich"}, options); + + WriteOptions wo; + wo.disableWAL = true; + + std::function wait_flush = [&]() { + dbfull()->TEST_WaitForFlushMemTable(handles_[0]); + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + dbfull()->TEST_WaitForFlushMemTable(handles_[2]); + dbfull()->TEST_WaitForFlushMemTable(handles_[3]); + }; + + // Create some data and flush "default" and "nikitich" so that they + // are newer CFs created. + ASSERT_OK(Put(3, Key(1), DummyString(1), wo)); + Flush(3); + ASSERT_OK(Put(3, Key(1), DummyString(1), wo)); + ASSERT_OK(Put(0, Key(1), DummyString(1), wo)); + Flush(0); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "nikitich"), + static_cast(1)); + + ASSERT_OK(Put(3, Key(1), DummyString(30000), wo)); + if (cost_cache_) { + ASSERT_GE(cache->GetUsage(), 1024 * 1024); + ASSERT_LE(cache->GetUsage(), 2 * 1024 * 1024); + } + wait_flush(); + ASSERT_OK(Put(0, Key(1), DummyString(60000), wo)); + if (cost_cache_) { + ASSERT_GE(cache->GetUsage(), 1024 * 1024); + ASSERT_LE(cache->GetUsage(), 2 * 1024 * 1024); + } + wait_flush(); + ASSERT_OK(Put(2, Key(1), DummyString(1), wo)); + // No flush should trigger + wait_flush(); + { + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "pikachu"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "dobrynia"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "nikitich"), + static_cast(1)); + } + + // Trigger a flush. Flushing "nikitich". + ASSERT_OK(Put(3, Key(2), DummyString(30000), wo)); + wait_flush(); + ASSERT_OK(Put(0, Key(1), DummyString(1), wo)); + wait_flush(); + { + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "pikachu"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "dobrynia"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "nikitich"), + static_cast(2)); + } + + // Without hitting the threshold, no flush should trigger. + ASSERT_OK(Put(2, Key(1), DummyString(30000), wo)); + wait_flush(); + ASSERT_OK(Put(2, Key(1), DummyString(1), wo)); + wait_flush(); + ASSERT_OK(Put(2, Key(1), DummyString(1), wo)); + wait_flush(); + { + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "pikachu"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "dobrynia"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "nikitich"), + static_cast(2)); + } + + // Hit the write buffer limit again. "default" + // will have been flushed. + ASSERT_OK(Put(2, Key(2), DummyString(10000), wo)); + wait_flush(); + ASSERT_OK(Put(3, Key(1), DummyString(1), wo)); + wait_flush(); + ASSERT_OK(Put(0, Key(1), DummyString(1), wo)); + wait_flush(); + ASSERT_OK(Put(0, Key(1), DummyString(1), wo)); + wait_flush(); + ASSERT_OK(Put(0, Key(1), DummyString(1), wo)); + wait_flush(); + { + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default"), + static_cast(2)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "pikachu"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "dobrynia"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "nikitich"), + static_cast(2)); + } + + // Trigger another flush. This time "dobrynia". "pikachu" should not + // be flushed, althrough it was never flushed. + ASSERT_OK(Put(1, Key(1), DummyString(1), wo)); + wait_flush(); + ASSERT_OK(Put(2, Key(1), DummyString(80000), wo)); + wait_flush(); + ASSERT_OK(Put(1, Key(1), DummyString(1), wo)); + wait_flush(); + ASSERT_OK(Put(2, Key(1), DummyString(1), wo)); + wait_flush(); + + { + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default"), + static_cast(2)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "pikachu"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "dobrynia"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "nikitich"), + static_cast(2)); + } + if (cost_cache_) { + ASSERT_GE(cache->GetUsage(), 1024 * 1024); + Close(); + options.write_buffer_manager.reset(); + ASSERT_LT(cache->GetUsage(), 1024 * 1024); + } + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +INSTANTIATE_TEST_CASE_P(DBTestSharedWriteBufferAcrossCFs, + DBTestSharedWriteBufferAcrossCFs, + ::testing::Values(std::make_tuple(true, false), + std::make_tuple(false, false), + std::make_tuple(false, true))); + +TEST_F(DBTest2, SharedWriteBufferLimitAcrossDB) { + std::string dbname2 = test::TmpDir(env_) + "/db_shared_wb_db2"; + Options options = CurrentOptions(); + options.arena_block_size = 4096; + // Avoid undeterministic value by malloc_usable_size(); + // Force arena block size to 1 + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "Arena::Arena:0", [&](void* arg) { + size_t* block_size = static_cast(arg); + *block_size = 1; + }); + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "Arena::AllocateNewBlock:0", [&](void* arg) { + std::pair* pair = + static_cast*>(arg); + *std::get<0>(*pair) = *std::get<1>(*pair); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + options.write_buffer_size = 500000; // this is never hit + // Use a write buffer total size so that the soft limit is about + // 105000. + options.write_buffer_manager.reset(new WriteBufferManager(120000)); + CreateAndReopenWithCF({"cf1", "cf2"}, options); + + ASSERT_OK(DestroyDB(dbname2, options)); + DB* db2 = nullptr; + ASSERT_OK(DB::Open(options, dbname2, &db2)); + + WriteOptions wo; + wo.disableWAL = true; + + std::function wait_flush = [&]() { + dbfull()->TEST_WaitForFlushMemTable(handles_[0]); + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + dbfull()->TEST_WaitForFlushMemTable(handles_[2]); + static_cast(db2)->TEST_WaitForFlushMemTable(); + }; + + // Trigger a flush on cf2 + ASSERT_OK(Put(2, Key(1), DummyString(70000), wo)); + wait_flush(); + ASSERT_OK(Put(0, Key(1), DummyString(20000), wo)); + wait_flush(); + + // Insert to DB2 + ASSERT_OK(db2->Put(wo, Key(2), DummyString(20000))); + wait_flush(); + + ASSERT_OK(Put(2, Key(1), DummyString(1), wo)); + wait_flush(); + static_cast(db2)->TEST_WaitForFlushMemTable(); + { + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default") + + GetNumberOfSstFilesForColumnFamily(db_, "cf1") + + GetNumberOfSstFilesForColumnFamily(db_, "cf2"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db2, "default"), + static_cast(0)); + } + + // Triggering to flush another CF in DB1 + ASSERT_OK(db2->Put(wo, Key(2), DummyString(70000))); + wait_flush(); + ASSERT_OK(Put(2, Key(1), DummyString(1), wo)); + wait_flush(); + { + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "cf1"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "cf2"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db2, "default"), + static_cast(0)); + } + + // Triggering flush in DB2. + ASSERT_OK(db2->Put(wo, Key(3), DummyString(40000))); + wait_flush(); + ASSERT_OK(db2->Put(wo, Key(1), DummyString(1))); + wait_flush(); + static_cast(db2)->TEST_WaitForFlushMemTable(); + { + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "cf1"), + static_cast(0)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "cf2"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db2, "default"), + static_cast(1)); + } + + delete db2; + ASSERT_OK(DestroyDB(dbname2, options)); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +namespace { + void ValidateKeyExistence(DB* db, const std::vector& keys_must_exist, + const std::vector& keys_must_not_exist) { + // Ensure that expected keys exist + std::vector values; + if (keys_must_exist.size() > 0) { + std::vector status_list = + db->MultiGet(ReadOptions(), keys_must_exist, &values); + for (size_t i = 0; i < keys_must_exist.size(); i++) { + ASSERT_OK(status_list[i]); + } + } + + // Ensure that given keys don't exist + if (keys_must_not_exist.size() > 0) { + std::vector status_list = + db->MultiGet(ReadOptions(), keys_must_not_exist, &values); + for (size_t i = 0; i < keys_must_not_exist.size(); i++) { + ASSERT_TRUE(status_list[i].IsNotFound()); + } + } + } + +} // namespace + +TEST_F(DBTest2, WalFilterTest) { + class TestWalFilter : public WalFilter { + private: + // Processing option that is requested to be applied at the given index + WalFilter::WalProcessingOption wal_processing_option_; + // Index at which to apply wal_processing_option_ + // At other indexes default wal_processing_option::kContinueProcessing is + // returned. + size_t apply_option_at_record_index_; + // Current record index, incremented with each record encountered. + size_t current_record_index_; + + public: + TestWalFilter(WalFilter::WalProcessingOption wal_processing_option, + size_t apply_option_for_record_index) + : wal_processing_option_(wal_processing_option), + apply_option_at_record_index_(apply_option_for_record_index), + current_record_index_(0) {} + + virtual WalProcessingOption LogRecord(const WriteBatch& batch, + WriteBatch* new_batch, + bool* batch_changed) const override { + WalFilter::WalProcessingOption option_to_return; + + if (current_record_index_ == apply_option_at_record_index_) { + option_to_return = wal_processing_option_; + } + else { + option_to_return = WalProcessingOption::kContinueProcessing; + } + + // Filter is passed as a const object for RocksDB to not modify the + // object, however we modify it for our own purpose here and hence + // cast the constness away. + (const_cast(this)->current_record_index_)++; + + return option_to_return; + } + + virtual const char* Name() const override { return "TestWalFilter"; } + }; + + // Create 3 batches with two keys each + std::vector> batch_keys(3); + + batch_keys[0].push_back("key1"); + batch_keys[0].push_back("key2"); + batch_keys[1].push_back("key3"); + batch_keys[1].push_back("key4"); + batch_keys[2].push_back("key5"); + batch_keys[2].push_back("key6"); + + // Test with all WAL processing options + for (int option = 0; + option < static_cast( + WalFilter::WalProcessingOption::kWalProcessingOptionMax); + option++) { + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + CreateAndReopenWithCF({ "pikachu" }, options); + + // Write given keys in given batches + for (size_t i = 0; i < batch_keys.size(); i++) { + WriteBatch batch; + for (size_t j = 0; j < batch_keys[i].size(); j++) { + batch.Put(handles_[0], batch_keys[i][j], DummyString(1024)); + } + dbfull()->Write(WriteOptions(), &batch); + } + + WalFilter::WalProcessingOption wal_processing_option = + static_cast(option); + + // Create a test filter that would apply wal_processing_option at the first + // record + size_t apply_option_for_record_index = 1; + TestWalFilter test_wal_filter(wal_processing_option, + apply_option_for_record_index); + + // Reopen database with option to use WAL filter + options = OptionsForLogIterTest(); + options.wal_filter = &test_wal_filter; + Status status = + TryReopenWithColumnFamilies({ "default", "pikachu" }, options); + if (wal_processing_option == + WalFilter::WalProcessingOption::kCorruptedRecord) { + assert(!status.ok()); + // In case of corruption we can turn off paranoid_checks to reopen + // databse + options.paranoid_checks = false; + ReopenWithColumnFamilies({ "default", "pikachu" }, options); + } + else { + assert(status.ok()); + } + + // Compute which keys we expect to be found + // and which we expect not to be found after recovery. + std::vector keys_must_exist; + std::vector keys_must_not_exist; + switch (wal_processing_option) { + case WalFilter::WalProcessingOption::kCorruptedRecord: + case WalFilter::WalProcessingOption::kContinueProcessing: { + fprintf(stderr, "Testing with complete WAL processing\n"); + // we expect all records to be processed + for (size_t i = 0; i < batch_keys.size(); i++) { + for (size_t j = 0; j < batch_keys[i].size(); j++) { + keys_must_exist.push_back(Slice(batch_keys[i][j])); + } + } + break; + } + case WalFilter::WalProcessingOption::kIgnoreCurrentRecord: { + fprintf(stderr, + "Testing with ignoring record %" ROCKSDB_PRIszt " only\n", + apply_option_for_record_index); + // We expect the record with apply_option_for_record_index to be not + // found. + for (size_t i = 0; i < batch_keys.size(); i++) { + for (size_t j = 0; j < batch_keys[i].size(); j++) { + if (i == apply_option_for_record_index) { + keys_must_not_exist.push_back(Slice(batch_keys[i][j])); + } + else { + keys_must_exist.push_back(Slice(batch_keys[i][j])); + } + } + } + break; + } + case WalFilter::WalProcessingOption::kStopReplay: { + fprintf(stderr, + "Testing with stopping replay from record %" ROCKSDB_PRIszt + "\n", + apply_option_for_record_index); + // We expect records beyond apply_option_for_record_index to be not + // found. + for (size_t i = 0; i < batch_keys.size(); i++) { + for (size_t j = 0; j < batch_keys[i].size(); j++) { + if (i >= apply_option_for_record_index) { + keys_must_not_exist.push_back(Slice(batch_keys[i][j])); + } + else { + keys_must_exist.push_back(Slice(batch_keys[i][j])); + } + } + } + break; + } + default: + assert(false); // unhandled case + } + + bool checked_after_reopen = false; + + while (true) { + // Ensure that expected keys exists + // and not expected keys don't exist after recovery + ValidateKeyExistence(db_, keys_must_exist, keys_must_not_exist); + + if (checked_after_reopen) { + break; + } + + // reopen database again to make sure previous log(s) are not used + //(even if they were skipped) + // reopn database with option to use WAL filter + options = OptionsForLogIterTest(); + ReopenWithColumnFamilies({ "default", "pikachu" }, options); + + checked_after_reopen = true; + } + } +} + +TEST_F(DBTest2, WalFilterTestWithChangeBatch) { + class ChangeBatchHandler : public WriteBatch::Handler { + private: + // Batch to insert keys in + WriteBatch* new_write_batch_; + // Number of keys to add in the new batch + size_t num_keys_to_add_in_new_batch_; + // Number of keys added to new batch + size_t num_keys_added_; + + public: + ChangeBatchHandler(WriteBatch* new_write_batch, + size_t num_keys_to_add_in_new_batch) + : new_write_batch_(new_write_batch), + num_keys_to_add_in_new_batch_(num_keys_to_add_in_new_batch), + num_keys_added_(0) {} + virtual void Put(const Slice& key, const Slice& value) override { + if (num_keys_added_ < num_keys_to_add_in_new_batch_) { + new_write_batch_->Put(key, value); + ++num_keys_added_; + } + } + }; + + class TestWalFilterWithChangeBatch : public WalFilter { + private: + // Index at which to start changing records + size_t change_records_from_index_; + // Number of keys to add in the new batch + size_t num_keys_to_add_in_new_batch_; + // Current record index, incremented with each record encountered. + size_t current_record_index_; + + public: + TestWalFilterWithChangeBatch(size_t change_records_from_index, + size_t num_keys_to_add_in_new_batch) + : change_records_from_index_(change_records_from_index), + num_keys_to_add_in_new_batch_(num_keys_to_add_in_new_batch), + current_record_index_(0) {} + + virtual WalProcessingOption LogRecord(const WriteBatch& batch, + WriteBatch* new_batch, + bool* batch_changed) const override { + if (current_record_index_ >= change_records_from_index_) { + ChangeBatchHandler handler(new_batch, num_keys_to_add_in_new_batch_); + batch.Iterate(&handler); + *batch_changed = true; + } + + // Filter is passed as a const object for RocksDB to not modify the + // object, however we modify it for our own purpose here and hence + // cast the constness away. + (const_cast(this) + ->current_record_index_)++; + + return WalProcessingOption::kContinueProcessing; + } + + virtual const char* Name() const override { + return "TestWalFilterWithChangeBatch"; + } + }; + + std::vector> batch_keys(3); + + batch_keys[0].push_back("key1"); + batch_keys[0].push_back("key2"); + batch_keys[1].push_back("key3"); + batch_keys[1].push_back("key4"); + batch_keys[2].push_back("key5"); + batch_keys[2].push_back("key6"); + + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + CreateAndReopenWithCF({ "pikachu" }, options); + + // Write given keys in given batches + for (size_t i = 0; i < batch_keys.size(); i++) { + WriteBatch batch; + for (size_t j = 0; j < batch_keys[i].size(); j++) { + batch.Put(handles_[0], batch_keys[i][j], DummyString(1024)); + } + dbfull()->Write(WriteOptions(), &batch); + } + + // Create a test filter that would apply wal_processing_option at the first + // record + size_t change_records_from_index = 1; + size_t num_keys_to_add_in_new_batch = 1; + TestWalFilterWithChangeBatch test_wal_filter_with_change_batch( + change_records_from_index, num_keys_to_add_in_new_batch); + + // Reopen database with option to use WAL filter + options = OptionsForLogIterTest(); + options.wal_filter = &test_wal_filter_with_change_batch; + ReopenWithColumnFamilies({ "default", "pikachu" }, options); + + // Ensure that all keys exist before change_records_from_index_ + // And after that index only single key exists + // as our filter adds only single key for each batch + std::vector keys_must_exist; + std::vector keys_must_not_exist; + + for (size_t i = 0; i < batch_keys.size(); i++) { + for (size_t j = 0; j < batch_keys[i].size(); j++) { + if (i >= change_records_from_index && j >= num_keys_to_add_in_new_batch) { + keys_must_not_exist.push_back(Slice(batch_keys[i][j])); + } + else { + keys_must_exist.push_back(Slice(batch_keys[i][j])); + } + } + } + + bool checked_after_reopen = false; + + while (true) { + // Ensure that expected keys exists + // and not expected keys don't exist after recovery + ValidateKeyExistence(db_, keys_must_exist, keys_must_not_exist); + + if (checked_after_reopen) { + break; + } + + // reopen database again to make sure previous log(s) are not used + //(even if they were skipped) + // reopn database with option to use WAL filter + options = OptionsForLogIterTest(); + ReopenWithColumnFamilies({ "default", "pikachu" }, options); + + checked_after_reopen = true; + } +} + +TEST_F(DBTest2, WalFilterTestWithChangeBatchExtraKeys) { + class TestWalFilterWithChangeBatchAddExtraKeys : public WalFilter { + public: + virtual WalProcessingOption LogRecord(const WriteBatch& batch, + WriteBatch* new_batch, + bool* batch_changed) const override { + *new_batch = batch; + new_batch->Put("key_extra", "value_extra"); + *batch_changed = true; + return WalProcessingOption::kContinueProcessing; + } + + virtual const char* Name() const override { + return "WalFilterTestWithChangeBatchExtraKeys"; + } + }; + + std::vector> batch_keys(3); + + batch_keys[0].push_back("key1"); + batch_keys[0].push_back("key2"); + batch_keys[1].push_back("key3"); + batch_keys[1].push_back("key4"); + batch_keys[2].push_back("key5"); + batch_keys[2].push_back("key6"); + + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + CreateAndReopenWithCF({ "pikachu" }, options); + + // Write given keys in given batches + for (size_t i = 0; i < batch_keys.size(); i++) { + WriteBatch batch; + for (size_t j = 0; j < batch_keys[i].size(); j++) { + batch.Put(handles_[0], batch_keys[i][j], DummyString(1024)); + } + dbfull()->Write(WriteOptions(), &batch); + } + + // Create a test filter that would add extra keys + TestWalFilterWithChangeBatchAddExtraKeys test_wal_filter_extra_keys; + + // Reopen database with option to use WAL filter + options = OptionsForLogIterTest(); + options.wal_filter = &test_wal_filter_extra_keys; + Status status = TryReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_TRUE(status.IsNotSupported()); + + // Reopen without filter, now reopen should succeed - previous + // attempt to open must not have altered the db. + options = OptionsForLogIterTest(); + ReopenWithColumnFamilies({ "default", "pikachu" }, options); + + std::vector keys_must_exist; + std::vector keys_must_not_exist; // empty vector + + for (size_t i = 0; i < batch_keys.size(); i++) { + for (size_t j = 0; j < batch_keys[i].size(); j++) { + keys_must_exist.push_back(Slice(batch_keys[i][j])); + } + } + + ValidateKeyExistence(db_, keys_must_exist, keys_must_not_exist); +} + +TEST_F(DBTest2, WalFilterTestWithColumnFamilies) { + class TestWalFilterWithColumnFamilies : public WalFilter { + private: + // column_family_id -> log_number map (provided to WALFilter) + std::map cf_log_number_map_; + // column_family_name -> column_family_id map (provided to WALFilter) + std::map cf_name_id_map_; + // column_family_name -> keys_found_in_wal map + // We store keys that are applicable to the column_family + // during recovery (i.e. aren't already flushed to SST file(s)) + // for verification against the keys we expect. + std::map> cf_wal_keys_; + public: + virtual void ColumnFamilyLogNumberMap( + const std::map& cf_lognumber_map, + const std::map& cf_name_id_map) override { + cf_log_number_map_ = cf_lognumber_map; + cf_name_id_map_ = cf_name_id_map; + } + + virtual WalProcessingOption LogRecordFound(unsigned long long log_number, + const std::string& log_file_name, + const WriteBatch& batch, + WriteBatch* new_batch, + bool* batch_changed) override { + class LogRecordBatchHandler : public WriteBatch::Handler { + private: + const std::map & cf_log_number_map_; + std::map> & cf_wal_keys_; + unsigned long long log_number_; + public: + LogRecordBatchHandler(unsigned long long current_log_number, + const std::map & cf_log_number_map, + std::map> & cf_wal_keys) : + cf_log_number_map_(cf_log_number_map), + cf_wal_keys_(cf_wal_keys), + log_number_(current_log_number){} + + virtual Status PutCF(uint32_t column_family_id, const Slice& key, + const Slice& /*value*/) override { + auto it = cf_log_number_map_.find(column_family_id); + assert(it != cf_log_number_map_.end()); + unsigned long long log_number_for_cf = it->second; + // If the current record is applicable for column_family_id + // (i.e. isn't flushed to SST file(s) for column_family_id) + // add it to the cf_wal_keys_ map for verification. + if (log_number_ >= log_number_for_cf) { + cf_wal_keys_[column_family_id].push_back(std::string(key.data(), + key.size())); + } + return Status::OK(); + } + } handler(log_number, cf_log_number_map_, cf_wal_keys_); + + batch.Iterate(&handler); + + return WalProcessingOption::kContinueProcessing; + } + + virtual const char* Name() const override { + return "WalFilterTestWithColumnFamilies"; + } + + const std::map>& GetColumnFamilyKeys() { + return cf_wal_keys_; + } + + const std::map & GetColumnFamilyNameIdMap() { + return cf_name_id_map_; + } + }; + + std::vector> batch_keys_pre_flush(3); + + batch_keys_pre_flush[0].push_back("key1"); + batch_keys_pre_flush[0].push_back("key2"); + batch_keys_pre_flush[1].push_back("key3"); + batch_keys_pre_flush[1].push_back("key4"); + batch_keys_pre_flush[2].push_back("key5"); + batch_keys_pre_flush[2].push_back("key6"); + + Options options = OptionsForLogIterTest(); + DestroyAndReopen(options); + CreateAndReopenWithCF({ "pikachu" }, options); + + // Write given keys in given batches + for (size_t i = 0; i < batch_keys_pre_flush.size(); i++) { + WriteBatch batch; + for (size_t j = 0; j < batch_keys_pre_flush[i].size(); j++) { + batch.Put(handles_[0], batch_keys_pre_flush[i][j], DummyString(1024)); + batch.Put(handles_[1], batch_keys_pre_flush[i][j], DummyString(1024)); + } + dbfull()->Write(WriteOptions(), &batch); + } + + //Flush default column-family + db_->Flush(FlushOptions(), handles_[0]); + + // Do some more writes + std::vector> batch_keys_post_flush(3); + + batch_keys_post_flush[0].push_back("key7"); + batch_keys_post_flush[0].push_back("key8"); + batch_keys_post_flush[1].push_back("key9"); + batch_keys_post_flush[1].push_back("key10"); + batch_keys_post_flush[2].push_back("key11"); + batch_keys_post_flush[2].push_back("key12"); + + // Write given keys in given batches + for (size_t i = 0; i < batch_keys_post_flush.size(); i++) { + WriteBatch batch; + for (size_t j = 0; j < batch_keys_post_flush[i].size(); j++) { + batch.Put(handles_[0], batch_keys_post_flush[i][j], DummyString(1024)); + batch.Put(handles_[1], batch_keys_post_flush[i][j], DummyString(1024)); + } + dbfull()->Write(WriteOptions(), &batch); + } + + // On Recovery we should only find the second batch applicable to default CF + // But both batches applicable to pikachu CF + + // Create a test filter that would add extra keys + TestWalFilterWithColumnFamilies test_wal_filter_column_families; + + // Reopen database with option to use WAL filter + options = OptionsForLogIterTest(); + options.wal_filter = &test_wal_filter_column_families; + Status status = + TryReopenWithColumnFamilies({ "default", "pikachu" }, options); + ASSERT_TRUE(status.ok()); + + // verify that handles_[0] only has post_flush keys + // while handles_[1] has pre and post flush keys + auto cf_wal_keys = test_wal_filter_column_families.GetColumnFamilyKeys(); + auto name_id_map = test_wal_filter_column_families.GetColumnFamilyNameIdMap(); + size_t index = 0; + auto keys_cf = cf_wal_keys[name_id_map[kDefaultColumnFamilyName]]; + //default column-family, only post_flush keys are expected + for (size_t i = 0; i < batch_keys_post_flush.size(); i++) { + for (size_t j = 0; j < batch_keys_post_flush[i].size(); j++) { + Slice key_from_the_log(keys_cf[index++]); + Slice batch_key(batch_keys_post_flush[i][j]); + ASSERT_TRUE(key_from_the_log.compare(batch_key) == 0); + } + } + ASSERT_TRUE(index == keys_cf.size()); + + index = 0; + keys_cf = cf_wal_keys[name_id_map["pikachu"]]; + //pikachu column-family, all keys are expected + for (size_t i = 0; i < batch_keys_pre_flush.size(); i++) { + for (size_t j = 0; j < batch_keys_pre_flush[i].size(); j++) { + Slice key_from_the_log(keys_cf[index++]); + Slice batch_key(batch_keys_pre_flush[i][j]); + ASSERT_TRUE(key_from_the_log.compare(batch_key) == 0); + } + } + + for (size_t i = 0; i < batch_keys_post_flush.size(); i++) { + for (size_t j = 0; j < batch_keys_post_flush[i].size(); j++) { + Slice key_from_the_log(keys_cf[index++]); + Slice batch_key(batch_keys_post_flush[i][j]); + ASSERT_TRUE(key_from_the_log.compare(batch_key) == 0); + } + } + ASSERT_TRUE(index == keys_cf.size()); +} + +TEST_F(DBTest2, PresetCompressionDict) { + const size_t kBlockSizeBytes = 4 << 10; + const size_t kL0FileBytes = 128 << 10; + const size_t kApproxPerBlockOverheadBytes = 50; + const int kNumL0Files = 5; + + Options options; + options.env = CurrentOptions().env; // Make sure to use any custom env that the test is configured with. + options.allow_concurrent_memtable_write = false; + options.arena_block_size = kBlockSizeBytes; + options.compaction_style = kCompactionStyleUniversal; + options.create_if_missing = true; + options.disable_auto_compactions = true; + options.level0_file_num_compaction_trigger = kNumL0Files; + options.memtable_factory.reset( + new SpecialSkipListFactory(kL0FileBytes / kBlockSizeBytes)); + options.num_levels = 2; + options.target_file_size_base = kL0FileBytes; + options.target_file_size_multiplier = 2; + options.write_buffer_size = kL0FileBytes; + BlockBasedTableOptions table_options; + table_options.block_size = kBlockSizeBytes; + std::vector compression_types; + if (Zlib_Supported()) { + compression_types.push_back(kZlibCompression); + } +#if LZ4_VERSION_NUMBER >= 10400 // r124+ + compression_types.push_back(kLZ4Compression); + compression_types.push_back(kLZ4HCCompression); +#endif // LZ4_VERSION_NUMBER >= 10400 + if (ZSTD_Supported()) { + compression_types.push_back(kZSTD); + } + + for (auto compression_type : compression_types) { + options.compression = compression_type; + size_t prev_out_bytes; + for (int i = 0; i < 2; ++i) { + // First iteration: compress without preset dictionary + // Second iteration: compress with preset dictionary + // To make sure the compression dictionary was actually used, we verify + // the compressed size is smaller in the second iteration. Also in the + // second iteration, verify the data we get out is the same data we put + // in. + if (i) { + options.compression_opts.max_dict_bytes = kBlockSizeBytes; + } else { + options.compression_opts.max_dict_bytes = 0; + } + + options.statistics = rocksdb::CreateDBStatistics(); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + CreateAndReopenWithCF({"pikachu"}, options); + Random rnd(301); + std::string seq_data = + RandomString(&rnd, kBlockSizeBytes - kApproxPerBlockOverheadBytes); + + ASSERT_EQ(0, NumTableFilesAtLevel(0, 1)); + for (int j = 0; j < kNumL0Files; ++j) { + for (size_t k = 0; k < kL0FileBytes / kBlockSizeBytes + 1; ++k) { + ASSERT_OK(Put(1, Key(static_cast( + j * (kL0FileBytes / kBlockSizeBytes) + k)), + seq_data)); + } + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + ASSERT_EQ(j + 1, NumTableFilesAtLevel(0, 1)); + } + db_->CompactRange(CompactRangeOptions(), handles_[1], nullptr, nullptr); + ASSERT_EQ(0, NumTableFilesAtLevel(0, 1)); + ASSERT_GT(NumTableFilesAtLevel(1, 1), 0); + + size_t out_bytes = 0; + std::vector files; + GetSstFiles(dbname_, &files); + for (const auto& file : files) { + uint64_t curr_bytes; + env_->GetFileSize(dbname_ + "/" + file, &curr_bytes); + out_bytes += static_cast(curr_bytes); + } + + for (size_t j = 0; j < kNumL0Files * (kL0FileBytes / kBlockSizeBytes); + j++) { + ASSERT_EQ(seq_data, Get(1, Key(static_cast(j)))); + } + if (i) { + ASSERT_GT(prev_out_bytes, out_bytes); + } + prev_out_bytes = out_bytes; + DestroyAndReopen(options); + } + } +} + +class CompactionCompressionListener : public EventListener { + public: + explicit CompactionCompressionListener(Options* db_options) + : db_options_(db_options) {} + + void OnCompactionCompleted(DB* db, const CompactionJobInfo& ci) override { + // Figure out last level with files + int bottommost_level = 0; + for (int level = 0; level < db->NumberLevels(); level++) { + std::string files_at_level; + ASSERT_TRUE( + db->GetProperty("rocksdb.num-files-at-level" + NumberToString(level), + &files_at_level)); + if (files_at_level != "0") { + bottommost_level = level; + } + } + + if (db_options_->bottommost_compression != kDisableCompressionOption && + ci.output_level == bottommost_level && ci.output_level >= 2) { + ASSERT_EQ(ci.compression, db_options_->bottommost_compression); + } else if (db_options_->compression_per_level.size() != 0) { + ASSERT_EQ(ci.compression, + db_options_->compression_per_level[ci.output_level]); + } else { + ASSERT_EQ(ci.compression, db_options_->compression); + } + max_level_checked = std::max(max_level_checked, ci.output_level); + } + + int max_level_checked = 0; + const Options* db_options_; +}; + +TEST_F(DBTest2, CompressionOptions) { + if (!Zlib_Supported() || !Snappy_Supported()) { + return; + } + + Options options = CurrentOptions(); + options.level0_file_num_compaction_trigger = 2; + options.max_bytes_for_level_base = 100; + options.max_bytes_for_level_multiplier = 2; + options.num_levels = 7; + options.max_background_compactions = 1; + + CompactionCompressionListener* listener = + new CompactionCompressionListener(&options); + options.listeners.emplace_back(listener); + + const int kKeySize = 5; + const int kValSize = 20; + Random rnd(301); + + for (int iter = 0; iter <= 2; iter++) { + listener->max_level_checked = 0; + + if (iter == 0) { + // Use different compression algorithms for different levels but + // always use Zlib for bottommost level + options.compression_per_level = {kNoCompression, kNoCompression, + kNoCompression, kSnappyCompression, + kSnappyCompression, kSnappyCompression, + kZlibCompression}; + options.compression = kNoCompression; + options.bottommost_compression = kZlibCompression; + } else if (iter == 1) { + // Use Snappy except for bottommost level use ZLib + options.compression_per_level = {}; + options.compression = kSnappyCompression; + options.bottommost_compression = kZlibCompression; + } else if (iter == 2) { + // Use Snappy everywhere + options.compression_per_level = {}; + options.compression = kSnappyCompression; + options.bottommost_compression = kDisableCompressionOption; + } + + DestroyAndReopen(options); + // Write 10 random files + for (int i = 0; i < 10; i++) { + for (int j = 0; j < 5; j++) { + ASSERT_OK( + Put(RandomString(&rnd, kKeySize), RandomString(&rnd, kValSize))); + } + ASSERT_OK(Flush()); + dbfull()->TEST_WaitForCompact(); + } + + // Make sure that we wrote enough to check all 7 levels + ASSERT_EQ(listener->max_level_checked, 6); + } +} + +class CompactionStallTestListener : public EventListener { + public: + CompactionStallTestListener() : compacted_files_cnt_(0) {} + + void OnCompactionCompleted(DB* db, const CompactionJobInfo& ci) override { + ASSERT_EQ(ci.cf_name, "default"); + ASSERT_EQ(ci.base_input_level, 0); + ASSERT_EQ(ci.compaction_reason, CompactionReason::kLevelL0FilesNum); + compacted_files_cnt_ += ci.input_files.size(); + } + std::atomic compacted_files_cnt_; +}; + +TEST_F(DBTest2, CompactionStall) { + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::BGWorkCompaction", "DBTest2::CompactionStall:0"}, + {"DBImpl::BGWorkCompaction", "DBTest2::CompactionStall:1"}, + {"DBTest2::CompactionStall:2", + "DBImpl::NotifyOnCompactionCompleted::UnlockMutex"}}); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.level0_file_num_compaction_trigger = 4; + options.max_background_compactions = 40; + CompactionStallTestListener* listener = new CompactionStallTestListener(); + options.listeners.emplace_back(listener); + DestroyAndReopen(options); + // make sure all background compaction jobs can be scheduled + auto stop_token = + dbfull()->TEST_write_controler().GetCompactionPressureToken(); + + Random rnd(301); + + // 4 Files in L0 + for (int i = 0; i < 4; i++) { + for (int j = 0; j < 10; j++) { + ASSERT_OK(Put(RandomString(&rnd, 10), RandomString(&rnd, 10))); + } + ASSERT_OK(Flush()); + } + + // Wait for compaction to be triggered + TEST_SYNC_POINT("DBTest2::CompactionStall:0"); + + // Clear "DBImpl::BGWorkCompaction" SYNC_POINT since we want to hold it again + // at DBTest2::CompactionStall::1 + rocksdb::SyncPoint::GetInstance()->ClearTrace(); + + // Another 6 L0 files to trigger compaction again + for (int i = 0; i < 6; i++) { + for (int j = 0; j < 10; j++) { + ASSERT_OK(Put(RandomString(&rnd, 10), RandomString(&rnd, 10))); + } + ASSERT_OK(Flush()); + } + + // Wait for another compaction to be triggered + TEST_SYNC_POINT("DBTest2::CompactionStall:1"); + + // Hold NotifyOnCompactionCompleted in the unlock mutex section + TEST_SYNC_POINT("DBTest2::CompactionStall:2"); + + dbfull()->TEST_WaitForCompact(); + ASSERT_LT(NumTableFilesAtLevel(0), + options.level0_file_num_compaction_trigger); + ASSERT_GT(listener->compacted_files_cnt_.load(), + 10 - options.level0_file_num_compaction_trigger); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +#endif // ROCKSDB_LITE + +TEST_F(DBTest2, FirstSnapshotTest) { + Options options; + options.write_buffer_size = 100000; // Small write buffer + options = CurrentOptions(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // This snapshot will have sequence number 0 what is expected behaviour. + const Snapshot* s1 = db_->GetSnapshot(); + + Put(1, "k1", std::string(100000, 'x')); // Fill memtable + Put(1, "k2", std::string(100000, 'y')); // Trigger flush + + db_->ReleaseSnapshot(s1); +} + +class PinL0IndexAndFilterBlocksTest : public DBTestBase, + public testing::WithParamInterface { + public: + PinL0IndexAndFilterBlocksTest() : DBTestBase("/db_pin_l0_index_bloom_test") {} + virtual void SetUp() override { infinite_max_files_ = GetParam(); } + + void CreateTwoLevels(Options* options, bool close_afterwards) { + if (infinite_max_files_) { + options->max_open_files = -1; + } + options->create_if_missing = true; + options->statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions table_options; + table_options.cache_index_and_filter_blocks = true; + table_options.pin_l0_filter_and_index_blocks_in_cache = true; + table_options.filter_policy.reset(NewBloomFilterPolicy(20)); + options->table_factory.reset(new BlockBasedTableFactory(table_options)); + CreateAndReopenWithCF({"pikachu"}, *options); + + Put(1, "a", "begin"); + Put(1, "z", "end"); + ASSERT_OK(Flush(1)); + // move this table to L1 + dbfull()->TEST_CompactRange(0, nullptr, nullptr, handles_[1]); + + // reset block cache + table_options.block_cache = NewLRUCache(64 * 1024); + options->table_factory.reset(NewBlockBasedTableFactory(table_options)); + TryReopenWithColumnFamilies({"default", "pikachu"}, *options); + // create new table at L0 + Put(1, "a2", "begin2"); + Put(1, "z2", "end2"); + ASSERT_OK(Flush(1)); + + if (close_afterwards) { + Close(); // This ensures that there is no ref to block cache entries + } + table_options.block_cache->EraseUnRefEntries(); + } + + bool infinite_max_files_; +}; + +TEST_P(PinL0IndexAndFilterBlocksTest, + IndexAndFilterBlocksOfNewTableAddedToCacheWithPinning) { + Options options = CurrentOptions(); + if (infinite_max_files_) { + options.max_open_files = -1; + } + options.create_if_missing = true; + options.statistics = rocksdb::CreateDBStatistics(); + BlockBasedTableOptions table_options; + table_options.cache_index_and_filter_blocks = true; + table_options.pin_l0_filter_and_index_blocks_in_cache = true; + table_options.filter_policy.reset(NewBloomFilterPolicy(20)); + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + CreateAndReopenWithCF({"pikachu"}, options); + + ASSERT_OK(Put(1, "key", "val")); + // Create a new table. + ASSERT_OK(Flush(1)); + + // index/filter blocks added to block cache right after table creation. + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); + + // only index/filter were added + ASSERT_EQ(2, TestGetTickerCount(options, BLOCK_CACHE_ADD)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_DATA_MISS)); + + std::string value; + // Miss and hit count should remain the same, they're all pinned. + db_->KeyMayExist(ReadOptions(), handles_[1], "key", &value); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); + + // Miss and hit count should remain the same, they're all pinned. + value = Get(1, "key"); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(0, TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); +} + +TEST_P(PinL0IndexAndFilterBlocksTest, + MultiLevelIndexAndFilterBlocksCachedWithPinning) { + Options options = CurrentOptions(); + PinL0IndexAndFilterBlocksTest::CreateTwoLevels(&options, false); + // get base cache values + uint64_t fm = TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS); + uint64_t fh = TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT); + uint64_t im = TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS); + uint64_t ih = TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT); + + std::string value; + // this should be read from L0 + // so cache values don't change + value = Get(1, "a2"); + ASSERT_EQ(fm, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(fh, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(im, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(ih, TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); + + // this should be read from L1 + // the file is opened, prefetching results in a cache filter miss + // the block is loaded and added to the cache, + // then the get results in a cache hit for L1 + // When we have inifinite max_files, there is still cache miss because we have + // reset the block cache + value = Get(1, "a"); + ASSERT_EQ(fm + 1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(im + 1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); +} + +TEST_P(PinL0IndexAndFilterBlocksTest, DisablePrefetchingNonL0IndexAndFilter) { + Options options = CurrentOptions(); + // This ensures that db does not ref anything in the block cache, so + // EraseUnRefEntries could clear them up. + bool close_afterwards = true; + PinL0IndexAndFilterBlocksTest::CreateTwoLevels(&options, close_afterwards); + + // Get base cache values + uint64_t fm = TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS); + uint64_t fh = TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT); + uint64_t im = TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS); + uint64_t ih = TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT); + + // Reopen database. If max_open_files is set as -1, table readers will be + // preloaded. This will trigger a BlockBasedTable::Open() and prefetch + // L0 index and filter. Level 1's prefetching is disabled in DB::Open() + TryReopenWithColumnFamilies({"default", "pikachu"}, options); + + if (infinite_max_files_) { + // After reopen, cache miss are increased by one because we read (and only + // read) filter and index on L0 + ASSERT_EQ(fm + 1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(fh, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(im + 1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(ih, TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); + } else { + // If max_open_files is not -1, we do not preload table readers, so there is + // no change. + ASSERT_EQ(fm, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(fh, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(im, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(ih, TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); + } + std::string value; + // this should be read from L0 + value = Get(1, "a2"); + // If max_open_files is -1, we have pinned index and filter in Rep, so there + // will not be changes in index and filter misses or hits. If max_open_files + // is not -1, Get() will open a TableReader and prefetch index and filter. + ASSERT_EQ(fm + 1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(fh, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(im + 1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(ih, TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); + + // this should be read from L1 + value = Get(1, "a"); + if (infinite_max_files_) { + // In inifinite max files case, there's a cache miss in executing Get() + // because index and filter are not prefetched before. + ASSERT_EQ(fm + 2, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(fh, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(im + 2, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(ih, TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); + } else { + // In this case, cache miss will be increased by one in + // BlockBasedTable::Open() because this is not in DB::Open() code path so we + // will prefetch L1's index and filter. Cache hit will also be increased by + // one because Get() will read index and filter from the block cache + // prefetched in previous Open() call. + ASSERT_EQ(fm + 2, TestGetTickerCount(options, BLOCK_CACHE_FILTER_MISS)); + ASSERT_EQ(fh + 1, TestGetTickerCount(options, BLOCK_CACHE_FILTER_HIT)); + ASSERT_EQ(im + 2, TestGetTickerCount(options, BLOCK_CACHE_INDEX_MISS)); + ASSERT_EQ(ih + 1, TestGetTickerCount(options, BLOCK_CACHE_INDEX_HIT)); + } +} + +INSTANTIATE_TEST_CASE_P(PinL0IndexAndFilterBlocksTest, + PinL0IndexAndFilterBlocksTest, ::testing::Bool()); + +#ifndef ROCKSDB_LITE +TEST_F(DBTest2, MaxCompactionBytesTest) { + Options options = CurrentOptions(); + options.memtable_factory.reset( + new SpecialSkipListFactory(DBTestBase::kNumKeysByGenerateNewRandomFile)); + options.compaction_style = kCompactionStyleLevel; + options.write_buffer_size = 200 << 10; + options.arena_block_size = 4 << 10; + options.level0_file_num_compaction_trigger = 4; + options.num_levels = 4; + options.compression = kNoCompression; + options.max_bytes_for_level_base = 450 << 10; + options.target_file_size_base = 100 << 10; + // Infinite for full compaction. + options.max_compaction_bytes = options.target_file_size_base * 100; + + Reopen(options); + + Random rnd(301); + + for (int num = 0; num < 8; num++) { + GenerateNewRandomFile(&rnd); + } + CompactRangeOptions cro; + cro.bottommost_level_compaction = BottommostLevelCompaction::kForce; + ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr)); + ASSERT_EQ("0,0,8", FilesPerLevel(0)); + + // When compact from Ln -> Ln+1, cut a file if the file overlaps with + // more than three files in Ln+1. + options.max_compaction_bytes = options.target_file_size_base * 3; + Reopen(options); + + GenerateNewRandomFile(&rnd); + // Add three more small files that overlap with the previous file + for (int i = 0; i < 3; i++) { + Put("a", "z"); + ASSERT_OK(Flush()); + } + dbfull()->TEST_WaitForCompact(); + + // Output files to L1 are cut to three pieces, according to + // options.max_compaction_bytes + ASSERT_EQ("0,3,8", FilesPerLevel(0)); +} + +static void UniqueIdCallback(void* arg) { + int* result = reinterpret_cast(arg); + if (*result == -1) { + *result = 0; + } + + rocksdb::SyncPoint::GetInstance()->ClearTrace(); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "GetUniqueIdFromFile:FS_IOC_GETVERSION", UniqueIdCallback); +} + +class MockPersistentCache : public PersistentCache { + public: + explicit MockPersistentCache(const bool is_compressed, const size_t max_size) + : is_compressed_(is_compressed), max_size_(max_size) { + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "GetUniqueIdFromFile:FS_IOC_GETVERSION", UniqueIdCallback); + } + + virtual ~MockPersistentCache() {} + + PersistentCache::StatsType Stats() override { + return PersistentCache::StatsType(); + } + + Status Insert(const Slice& page_key, const char* data, + const size_t size) override { + MutexLock _(&lock_); + + if (size_ > max_size_) { + size_ -= data_.begin()->second.size(); + data_.erase(data_.begin()); + } + + data_.insert(std::make_pair(page_key.ToString(), std::string(data, size))); + size_ += size; + return Status::OK(); + } + + Status Lookup(const Slice& page_key, std::unique_ptr* data, + size_t* size) override { + MutexLock _(&lock_); + auto it = data_.find(page_key.ToString()); + if (it == data_.end()) { + return Status::NotFound(); + } + + assert(page_key.ToString() == it->first); + data->reset(new char[it->second.size()]); + memcpy(data->get(), it->second.c_str(), it->second.size()); + *size = it->second.size(); + return Status::OK(); + } + + bool IsCompressed() override { return is_compressed_; } + + std::string GetPrintableOptions() const override { + return "MockPersistentCache"; + } + + port::Mutex lock_; + std::map data_; + const bool is_compressed_ = true; + size_t size_ = 0; + const size_t max_size_ = 10 * 1024; // 10KiB +}; + +#ifndef OS_SOLARIS // GetUniqueIdFromFile is not implemented +TEST_F(DBTest2, PersistentCache) { + int num_iter = 80; + + Options options; + options.write_buffer_size = 64 * 1024; // small write buffer + options.statistics = rocksdb::CreateDBStatistics(); + options = CurrentOptions(options); + + auto bsizes = {/*no block cache*/ 0, /*1M*/ 1 * 1024 * 1024}; + auto types = {/*compressed*/ 1, /*uncompressed*/ 0}; + for (auto bsize : bsizes) { + for (auto type : types) { + BlockBasedTableOptions table_options; + table_options.persistent_cache.reset( + new MockPersistentCache(type, 10 * 1024)); + table_options.no_block_cache = true; + table_options.block_cache = bsize ? NewLRUCache(bsize) : nullptr; + table_options.block_cache_compressed = nullptr; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + // default column family doesn't have block cache + Options no_block_cache_opts; + no_block_cache_opts.statistics = options.statistics; + no_block_cache_opts = CurrentOptions(no_block_cache_opts); + BlockBasedTableOptions table_options_no_bc; + table_options_no_bc.no_block_cache = true; + no_block_cache_opts.table_factory.reset( + NewBlockBasedTableFactory(table_options_no_bc)); + ReopenWithColumnFamilies( + {"default", "pikachu"}, + std::vector({no_block_cache_opts, options})); + + Random rnd(301); + + // Write 8MB (80 values, each 100K) + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + std::vector values; + std::string str; + for (int i = 0; i < num_iter; i++) { + if (i % 4 == 0) { // high compression ratio + str = RandomString(&rnd, 1000); + } + values.push_back(str); + ASSERT_OK(Put(1, Key(i), values[i])); + } + + // flush all data from memtable so that reads are from block cache + ASSERT_OK(Flush(1)); + + for (int i = 0; i < num_iter; i++) { + ASSERT_EQ(Get(1, Key(i)), values[i]); + } + + auto hit = options.statistics->getTickerCount(PERSISTENT_CACHE_HIT); + auto miss = options.statistics->getTickerCount(PERSISTENT_CACHE_MISS); + + ASSERT_GT(hit, 0); + ASSERT_GT(miss, 0); + } + } +} +#endif // !OS_SOLARIS + +namespace { +void CountSyncPoint() { + TEST_SYNC_POINT_CALLBACK("DBTest2::MarkedPoint", nullptr /* arg */); +} +} // namespace + +TEST_F(DBTest2, SyncPointMarker) { + std::atomic sync_point_called(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBTest2::MarkedPoint", + [&](void* arg) { sync_point_called.fetch_add(1); }); + + // The first dependency enforces Marker can be loaded before MarkedPoint. + // The second checks that thread 1's MarkedPoint should be disabled here. + // Execution order: + // | Thread 1 | Thread 2 | + // | | Marker | + // | MarkedPoint | | + // | Thread1First | | + // | | MarkedPoint | + rocksdb::SyncPoint::GetInstance()->LoadDependencyAndMarkers( + {{"DBTest2::SyncPointMarker:Thread1First", "DBTest2::MarkedPoint"}}, + {{"DBTest2::SyncPointMarker:Marker", "DBTest2::MarkedPoint"}}); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + std::function func1 = [&]() { + CountSyncPoint(); + TEST_SYNC_POINT("DBTest2::SyncPointMarker:Thread1First"); + }; + + std::function func2 = [&]() { + TEST_SYNC_POINT("DBTest2::SyncPointMarker:Marker"); + CountSyncPoint(); + }; + + auto thread1 = port::Thread(func1); + auto thread2 = port::Thread(func2); + thread1.join(); + thread2.join(); + + // Callback is only executed once + ASSERT_EQ(sync_point_called.load(), 1); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} +#endif + +size_t GetEncodedEntrySize(size_t key_size, size_t value_size) { + std::string buffer; + + PutVarint32(&buffer, static_cast(0)); + PutVarint32(&buffer, static_cast(key_size)); + PutVarint32(&buffer, static_cast(value_size)); + + return buffer.size() + key_size + value_size; +} + +TEST_F(DBTest2, ReadAmpBitmap) { + Options options = CurrentOptions(); + BlockBasedTableOptions bbto; + uint32_t bytes_per_bit[2] = {1, 16}; + for (size_t k = 0; k < 2; k++) { + // Disable delta encoding to make it easier to calculate read amplification + bbto.use_delta_encoding = false; + // Huge block cache to make it easier to calculate read amplification + bbto.block_cache = NewLRUCache(1024 * 1024 * 1024); + bbto.read_amp_bytes_per_bit = bytes_per_bit[k]; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + options.statistics = rocksdb::CreateDBStatistics(); + DestroyAndReopen(options); + + const size_t kNumEntries = 10000; + + Random rnd(301); + for (size_t i = 0; i < kNumEntries; i++) { + ASSERT_OK(Put(Key(static_cast(i)), RandomString(&rnd, 100))); + } + ASSERT_OK(Flush()); + + Close(); + Reopen(options); + + // Read keys/values randomly and verify that reported read amp error + // is less than 2% + uint64_t total_useful_bytes = 0; + std::set read_keys; + std::string value; + for (size_t i = 0; i < kNumEntries * 5; i++) { + int key_idx = rnd.Next() % kNumEntries; + std::string key = Key(key_idx); + ASSERT_OK(db_->Get(ReadOptions(), key, &value)); + + if (read_keys.find(key_idx) == read_keys.end()) { + auto internal_key = InternalKey(key, 0, ValueType::kTypeValue); + total_useful_bytes += + GetEncodedEntrySize(internal_key.size(), value.size()); + read_keys.insert(key_idx); + } + + double expected_read_amp = + static_cast(total_useful_bytes) / + options.statistics->getTickerCount(READ_AMP_TOTAL_READ_BYTES); + + double read_amp = + static_cast(options.statistics->getTickerCount( + READ_AMP_ESTIMATE_USEFUL_BYTES)) / + options.statistics->getTickerCount(READ_AMP_TOTAL_READ_BYTES); + + double error_pct = fabs(expected_read_amp - read_amp) * 100; + // Error between reported read amp and real read amp should be less than + // 2% + EXPECT_LE(error_pct, 2); + } + + // Make sure we read every thing in the DB (which is smaller than our cache) + Iterator* iter = db_->NewIterator(ReadOptions()); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_EQ(iter->value().ToString(), Get(iter->key().ToString())); + } + delete iter; + + // Read amp is on average 100% since we read all what we loaded in memory + if (k == 0) { + ASSERT_EQ( + options.statistics->getTickerCount(READ_AMP_ESTIMATE_USEFUL_BYTES), + options.statistics->getTickerCount(READ_AMP_TOTAL_READ_BYTES)); + } else { + ASSERT_NEAR( + options.statistics->getTickerCount(READ_AMP_ESTIMATE_USEFUL_BYTES) * + 1.0f / + options.statistics->getTickerCount(READ_AMP_TOTAL_READ_BYTES), + 1, .01); + } + } +} + +#ifndef OS_SOLARIS // GetUniqueIdFromFile is not implemented +TEST_F(DBTest2, ReadAmpBitmapLiveInCacheAfterDBClose) { + if (dbname_.find("dev/shm") != std::string::npos) { + // /dev/shm dont support getting a unique file id, this mean that + // running this test on /dev/shm will fail because lru_cache will load + // the blocks again regardless of them being already in the cache + return; + } + uint32_t bytes_per_bit[2] = {1, 16}; + for (size_t k = 0; k < 2; k++) { + std::shared_ptr lru_cache = NewLRUCache(1024 * 1024 * 1024); + std::shared_ptr stats = rocksdb::CreateDBStatistics(); + + Options options = CurrentOptions(); + BlockBasedTableOptions bbto; + // Disable delta encoding to make it easier to calculate read amplification + bbto.use_delta_encoding = false; + // Huge block cache to make it easier to calculate read amplification + bbto.block_cache = lru_cache; + bbto.read_amp_bytes_per_bit = bytes_per_bit[k]; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + options.statistics = stats; + DestroyAndReopen(options); + + const int kNumEntries = 10000; + + Random rnd(301); + for (int i = 0; i < kNumEntries; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 100))); + } + ASSERT_OK(Flush()); + + Close(); + Reopen(options); + + uint64_t total_useful_bytes = 0; + std::set read_keys; + std::string value; + // Iter1: Read half the DB, Read even keys + // Key(0), Key(2), Key(4), Key(6), Key(8), ... + for (int i = 0; i < kNumEntries; i += 2) { + std::string key = Key(i); + ASSERT_OK(db_->Get(ReadOptions(), key, &value)); + + if (read_keys.find(i) == read_keys.end()) { + auto internal_key = InternalKey(key, 0, ValueType::kTypeValue); + total_useful_bytes += + GetEncodedEntrySize(internal_key.size(), value.size()); + read_keys.insert(i); + } + } + + size_t total_useful_bytes_iter1 = + options.statistics->getTickerCount(READ_AMP_ESTIMATE_USEFUL_BYTES); + size_t total_loaded_bytes_iter1 = + options.statistics->getTickerCount(READ_AMP_TOTAL_READ_BYTES); + + Close(); + std::shared_ptr new_statistics = rocksdb::CreateDBStatistics(); + // Destroy old statistics obj that the blocks in lru_cache are pointing to + options.statistics.reset(); + // Use the statistics object that we just created + options.statistics = new_statistics; + Reopen(options); + + // Iter2: Read half the DB, Read odd keys + // Key(1), Key(3), Key(5), Key(7), Key(9), ... + for (int i = 1; i < kNumEntries; i += 2) { + std::string key = Key(i); + ASSERT_OK(db_->Get(ReadOptions(), key, &value)); + + if (read_keys.find(i) == read_keys.end()) { + auto internal_key = InternalKey(key, 0, ValueType::kTypeValue); + total_useful_bytes += + GetEncodedEntrySize(internal_key.size(), value.size()); + read_keys.insert(i); + } + } + + size_t total_useful_bytes_iter2 = + options.statistics->getTickerCount(READ_AMP_ESTIMATE_USEFUL_BYTES); + size_t total_loaded_bytes_iter2 = + options.statistics->getTickerCount(READ_AMP_TOTAL_READ_BYTES); + + + // Read amp is on average 100% since we read all what we loaded in memory + if (k == 0) { + ASSERT_EQ(total_useful_bytes_iter1 + total_useful_bytes_iter2, + total_loaded_bytes_iter1 + total_loaded_bytes_iter2); + } else { + ASSERT_NEAR((total_useful_bytes_iter1 + total_useful_bytes_iter2) * 1.0f / + (total_loaded_bytes_iter1 + total_loaded_bytes_iter2), + 1, .01); + } + } +} +#endif // !OS_SOLARIS + +#ifndef ROCKSDB_LITE +TEST_F(DBTest2, AutomaticCompactionOverlapManualCompaction) { + Options options = CurrentOptions(); + options.num_levels = 3; + options.IncreaseParallelism(20); + DestroyAndReopen(options); + + ASSERT_OK(Put(Key(0), "a")); + ASSERT_OK(Put(Key(5), "a")); + ASSERT_OK(Flush()); + + ASSERT_OK(Put(Key(10), "a")); + ASSERT_OK(Put(Key(15), "a")); + ASSERT_OK(Flush()); + + CompactRangeOptions cro; + cro.change_level = true; + cro.target_level = 2; + ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr)); + + auto get_stat = [](std::string level_str, LevelStatType type, + std::map props) { + auto prop_str = + level_str + "." + + InternalStats::compaction_level_stats.at(type).property_name.c_str(); + auto prop_item = props.find(prop_str); + return prop_item == props.end() ? 0 : prop_item->second; + }; + + // Trivial move 2 files to L2 + ASSERT_EQ("0,0,2", FilesPerLevel()); + // Also test that the stats GetMapProperty API reporting the same result + { + std::map prop; + ASSERT_TRUE(dbfull()->GetMapProperty("rocksdb.cfstats", &prop)); + ASSERT_EQ(0, get_stat("L0", LevelStatType::NUM_FILES, prop)); + ASSERT_EQ(0, get_stat("L1", LevelStatType::NUM_FILES, prop)); + ASSERT_EQ(2, get_stat("L2", LevelStatType::NUM_FILES, prop)); + ASSERT_EQ(2, get_stat("Sum", LevelStatType::NUM_FILES, prop)); + } + + // While the compaction is running, we will create 2 new files that + // can fit in L2, these 2 files will be moved to L2 and overlap with + // the running compaction and break the LSM consistency. + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "CompactionJob::Run():Start", [&](void* arg) { + ASSERT_OK( + dbfull()->SetOptions({{"level0_file_num_compaction_trigger", "2"}, + {"max_bytes_for_level_base", "1"}})); + ASSERT_OK(Put(Key(6), "a")); + ASSERT_OK(Put(Key(7), "a")); + ASSERT_OK(Flush()); + + ASSERT_OK(Put(Key(8), "a")); + ASSERT_OK(Put(Key(9), "a")); + ASSERT_OK(Flush()); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Run a manual compaction that will compact the 2 files in L2 + // into 1 file in L2 + cro.exclusive_manual_compaction = false; + cro.bottommost_level_compaction = BottommostLevelCompaction::kForce; + ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr)); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + // Test that the stats GetMapProperty API reporting 1 file in L2 + { + std::map prop; + ASSERT_TRUE(dbfull()->GetMapProperty("rocksdb.cfstats", &prop)); + ASSERT_EQ(1, get_stat("L2", LevelStatType::NUM_FILES, prop)); + } +} + +TEST_F(DBTest2, ManualCompactionOverlapManualCompaction) { + Options options = CurrentOptions(); + options.num_levels = 2; + options.IncreaseParallelism(20); + options.disable_auto_compactions = true; + DestroyAndReopen(options); + + ASSERT_OK(Put(Key(0), "a")); + ASSERT_OK(Put(Key(5), "a")); + ASSERT_OK(Flush()); + + ASSERT_OK(Put(Key(10), "a")); + ASSERT_OK(Put(Key(15), "a")); + ASSERT_OK(Flush()); + + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + + // Trivial move 2 files to L1 + ASSERT_EQ("0,2", FilesPerLevel()); + + std::function bg_manual_compact = [&]() { + std::string k1 = Key(6); + std::string k2 = Key(9); + Slice k1s(k1); + Slice k2s(k2); + CompactRangeOptions cro; + cro.exclusive_manual_compaction = false; + ASSERT_OK(db_->CompactRange(cro, &k1s, &k2s)); + }; + rocksdb::port::Thread bg_thread; + + // While the compaction is running, we will create 2 new files that + // can fit in L1, these 2 files will be moved to L1 and overlap with + // the running compaction and break the LSM consistency. + std::atomic flag(false); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "CompactionJob::Run():Start", [&](void* arg) { + if (flag.exchange(true)) { + // We want to make sure to call this callback only once + return; + } + ASSERT_OK(Put(Key(6), "a")); + ASSERT_OK(Put(Key(7), "a")); + ASSERT_OK(Flush()); + + ASSERT_OK(Put(Key(8), "a")); + ASSERT_OK(Put(Key(9), "a")); + ASSERT_OK(Flush()); + + // Start a non-exclusive manual compaction in a bg thread + bg_thread = port::Thread(bg_manual_compact); + // This manual compaction conflict with the other manual compaction + // so it should wait until the first compaction finish + env_->SleepForMicroseconds(1000000); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Run a manual compaction that will compact the 2 files in L1 + // into 1 file in L1 + CompactRangeOptions cro; + cro.exclusive_manual_compaction = false; + cro.bottommost_level_compaction = BottommostLevelCompaction::kForce; + ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr)); + bg_thread.join(); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(DBTest2, OptimizeForPointLookup) { + Options options = CurrentOptions(); + Close(); + options.OptimizeForPointLookup(2); + ASSERT_OK(DB::Open(options, dbname_, &db_)); + + ASSERT_OK(Put("foo", "v1")); + ASSERT_EQ("v1", Get("foo")); + Flush(); + ASSERT_EQ("v1", Get("foo")); +} + +#endif // ROCKSDB_LITE + +TEST_F(DBTest2, GetRaceFlush1) { + ASSERT_OK(Put("foo", "v1")); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::GetImpl:1", "DBTest2::GetRaceFlush:1"}, + {"DBTest2::GetRaceFlush:2", "DBImpl::GetImpl:2"}}); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + rocksdb::port::Thread t1([&] { + TEST_SYNC_POINT("DBTest2::GetRaceFlush:1"); + ASSERT_OK(Put("foo", "v2")); + Flush(); + TEST_SYNC_POINT("DBTest2::GetRaceFlush:2"); + }); + + // Get() is issued after the first Put(), so it should see either + // "v1" or "v2". + ASSERT_NE("NOT_FOUND", Get("foo")); + t1.join(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(DBTest2, GetRaceFlush2) { + ASSERT_OK(Put("foo", "v1")); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::GetImpl:3", "DBTest2::GetRaceFlush:1"}, + {"DBTest2::GetRaceFlush:2", "DBImpl::GetImpl:4"}}); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + port::Thread t1([&] { + TEST_SYNC_POINT("DBTest2::GetRaceFlush:1"); + ASSERT_OK(Put("foo", "v2")); + Flush(); + TEST_SYNC_POINT("DBTest2::GetRaceFlush:2"); + }); + + // Get() is issued after the first Put(), so it should see either + // "v1" or "v2". + ASSERT_NE("NOT_FOUND", Get("foo")); + t1.join(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(DBTest2, DirectIO) { + if (!IsDirectIOSupported()) { + return; + } + Options options = CurrentOptions(); + options.use_direct_reads = options.use_direct_io_for_flush_and_compaction = + true; + options.allow_mmap_reads = options.allow_mmap_writes = false; + DestroyAndReopen(options); + + ASSERT_OK(Put(Key(0), "a")); + ASSERT_OK(Put(Key(5), "a")); + ASSERT_OK(Flush()); + + ASSERT_OK(Put(Key(10), "a")); + ASSERT_OK(Put(Key(15), "a")); + ASSERT_OK(Flush()); + + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + Reopen(options); +} + +TEST_F(DBTest2, MemtableOnlyIterator) { + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); + + ASSERT_OK(Put(1, "foo", "first")); + ASSERT_OK(Put(1, "bar", "second")); + + ReadOptions ropt; + ropt.read_tier = kMemtableTier; + std::string value; + Iterator* it = nullptr; + + // Before flushing + // point lookups + ASSERT_OK(db_->Get(ropt, handles_[1], "foo", &value)); + ASSERT_EQ("first", value); + ASSERT_OK(db_->Get(ropt, handles_[1], "bar", &value)); + ASSERT_EQ("second", value); + + // Memtable-only iterator (read_tier=kMemtableTier); data not flushed yet. + it = db_->NewIterator(ropt, handles_[1]); + int count = 0; + for (it->SeekToFirst(); it->Valid(); it->Next()) { + ASSERT_TRUE(it->Valid()); + count++; + } + ASSERT_TRUE(!it->Valid()); + ASSERT_EQ(2, count); + delete it; + + Flush(1); + + // After flushing + // point lookups + ASSERT_OK(db_->Get(ropt, handles_[1], "foo", &value)); + ASSERT_EQ("first", value); + ASSERT_OK(db_->Get(ropt, handles_[1], "bar", &value)); + ASSERT_EQ("second", value); + // nothing should be returned using memtable-only iterator after flushing. + it = db_->NewIterator(ropt, handles_[1]); + count = 0; + for (it->SeekToFirst(); it->Valid(); it->Next()) { + ASSERT_TRUE(it->Valid()); + count++; + } + ASSERT_TRUE(!it->Valid()); + ASSERT_EQ(0, count); + delete it; + + // Add a key to memtable + ASSERT_OK(Put(1, "foobar", "third")); + it = db_->NewIterator(ropt, handles_[1]); + count = 0; + for (it->SeekToFirst(); it->Valid(); it->Next()) { + ASSERT_TRUE(it->Valid()); + ASSERT_EQ("foobar", it->key().ToString()); + ASSERT_EQ("third", it->value().ToString()); + count++; + } + ASSERT_TRUE(!it->Valid()); + ASSERT_EQ(1, count); + delete it; +} + +TEST_F(DBTest2, LowPriWrite) { + Options options = CurrentOptions(); + // Compaction pressure should trigger since 6 files + options.level0_file_num_compaction_trigger = 4; + options.level0_slowdown_writes_trigger = 12; + options.level0_stop_writes_trigger = 30; + options.delayed_write_rate = 8 * 1024 * 1024; + Reopen(options); + + std::atomic rate_limit_count(0); + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "GenericRateLimiter::Request:1", [&](void* arg) { + rate_limit_count.fetch_add(1); + int64_t* rate_bytes_per_sec = static_cast(arg); + ASSERT_EQ(1024 * 1024, *rate_bytes_per_sec); + }); + // Block compaction + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"DBTest.LowPriWrite:0", "DBImpl::BGWorkCompaction"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + WriteOptions wo; + for (int i = 0; i < 6; i++) { + wo.low_pri = false; + Put("", "", wo); + wo.low_pri = true; + Put("", "", wo); + Flush(); + } + ASSERT_EQ(0, rate_limit_count.load()); + wo.low_pri = true; + Put("", "", wo); + ASSERT_EQ(1, rate_limit_count.load()); + wo.low_pri = false; + Put("", "", wo); + ASSERT_EQ(1, rate_limit_count.load()); + + TEST_SYNC_POINT("DBTest.LowPriWrite:0"); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + dbfull()->TEST_WaitForCompact(); + wo.low_pri = true; + Put("", "", wo); + ASSERT_EQ(1, rate_limit_count.load()); + wo.low_pri = false; + Put("", "", wo); + ASSERT_EQ(1, rate_limit_count.load()); +} + +#ifndef ROCKSDB_LITE +TEST_F(DBTest2, RateLimitedCompactionReads) { + // compaction input has 512KB data + const int kNumKeysPerFile = 128; + const int kBytesPerKey = 1024; + const int kNumL0Files = 4; + + for (auto use_direct_io : {false, true}) { + if (use_direct_io && !IsDirectIOSupported()) { + continue; + } + Options options = CurrentOptions(); + options.compression = kNoCompression; + options.level0_file_num_compaction_trigger = kNumL0Files; + options.memtable_factory.reset(new SpecialSkipListFactory(kNumKeysPerFile)); + options.new_table_reader_for_compaction_inputs = true; + // takes roughly one second, split into 100 x 10ms intervals. Each interval + // permits 5.12KB, which is smaller than the block size, so this test + // exercises the code for chunking reads. + options.rate_limiter.reset(NewGenericRateLimiter( + static_cast(kNumL0Files * kNumKeysPerFile * + kBytesPerKey) /* rate_bytes_per_sec */, + 10 * 1000 /* refill_period_us */, 10 /* fairness */, + RateLimiter::Mode::kReadsOnly)); + options.use_direct_io_for_flush_and_compaction = use_direct_io; + BlockBasedTableOptions bbto; + bbto.block_size = 16384; + bbto.no_block_cache = true; + options.table_factory.reset(new BlockBasedTableFactory(bbto)); + DestroyAndReopen(options); + + for (int i = 0; i < kNumL0Files; ++i) { + for (int j = 0; j <= kNumKeysPerFile; ++j) { + ASSERT_OK(Put(Key(j), DummyString(kBytesPerKey))); + } + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(i + 1, NumTableFilesAtLevel(0)); + } + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(0, NumTableFilesAtLevel(0)); + + ASSERT_EQ(0, options.rate_limiter->GetTotalBytesThrough(Env::IO_HIGH)); + // should be slightly above 512KB due to non-data blocks read. Arbitrarily + // chose 1MB as the upper bound on the total bytes read. + size_t rate_limited_bytes = + options.rate_limiter->GetTotalBytesThrough(Env::IO_LOW); + // Include the explict prefetch of the footer in direct I/O case. + size_t direct_io_extra = use_direct_io ? 512 * 1024 : 0; + ASSERT_GE(rate_limited_bytes, + static_cast(kNumKeysPerFile * kBytesPerKey * kNumL0Files + + direct_io_extra)); + ASSERT_LT( + rate_limited_bytes, + static_cast(2 * kNumKeysPerFile * kBytesPerKey * kNumL0Files + + direct_io_extra)); + + Iterator* iter = db_->NewIterator(ReadOptions()); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_EQ(iter->value().ToString(), DummyString(kBytesPerKey)); + } + delete iter; + // bytes read for user iterator shouldn't count against the rate limit. + ASSERT_EQ(rate_limited_bytes, + static_cast( + options.rate_limiter->GetTotalBytesThrough(Env::IO_LOW))); + } +} +#endif // ROCKSDB_LITE + +// Make sure DB can be reopen with reduced number of levels, given no file +// is on levels higher than the new num_levels. +TEST_F(DBTest2, ReduceLevel) { + Options options; + options.disable_auto_compactions = true; + options.num_levels = 7; + Reopen(options); + Put("foo", "bar"); + Flush(); + MoveFilesToLevel(6); +#ifndef ROCKSDB_LITE + ASSERT_EQ("0,0,0,0,0,0,1", FilesPerLevel()); +#endif // !ROCKSDB_LITE + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 1; + dbfull()->CompactRange(compact_options, nullptr, nullptr); +#ifndef ROCKSDB_LITE + ASSERT_EQ("0,1", FilesPerLevel()); +#endif // !ROCKSDB_LITE + options.num_levels = 3; + Reopen(options); +#ifndef ROCKSDB_LITE + ASSERT_EQ("0,1", FilesPerLevel()); +#endif // !ROCKSDB_LITE +} +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_test_util.cc b/db/db_test_util.cc new file mode 100644 index 00000000000..c4d465ba117 --- /dev/null +++ b/db/db_test_util.cc @@ -0,0 +1,1395 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_test_util.h" +#include "db/forward_iterator.h" +#include "rocksdb/env_encryption.h" + +namespace rocksdb { + +// Special Env used to delay background operations + +SpecialEnv::SpecialEnv(Env* base) + : EnvWrapper(base), + rnd_(301), + sleep_counter_(this), + addon_time_(0), + time_elapse_only_sleep_(false), + no_slowdown_(false) { + delay_sstable_sync_.store(false, std::memory_order_release); + drop_writes_.store(false, std::memory_order_release); + no_space_.store(false, std::memory_order_release); + non_writable_.store(false, std::memory_order_release); + count_random_reads_ = false; + count_sequential_reads_ = false; + manifest_sync_error_.store(false, std::memory_order_release); + manifest_write_error_.store(false, std::memory_order_release); + log_write_error_.store(false, std::memory_order_release); + random_file_open_counter_.store(0, std::memory_order_relaxed); + delete_count_.store(0, std::memory_order_relaxed); + num_open_wal_file_.store(0); + log_write_slowdown_ = 0; + bytes_written_ = 0; + sync_counter_ = 0; + non_writeable_rate_ = 0; + new_writable_count_ = 0; + non_writable_count_ = 0; + table_write_callback_ = nullptr; +} +#ifndef ROCKSDB_LITE +ROT13BlockCipher rot13Cipher_(16); +#endif // ROCKSDB_LITE + +DBTestBase::DBTestBase(const std::string path) + : mem_env_(!getenv("MEM_ENV") ? nullptr : new MockEnv(Env::Default())), +#ifndef ROCKSDB_LITE + encrypted_env_( + !getenv("ENCRYPTED_ENV") + ? nullptr + : NewEncryptedEnv(mem_env_ ? mem_env_ : Env::Default(), + new CTREncryptionProvider(rot13Cipher_))), +#else + encrypted_env_(nullptr), +#endif // ROCKSDB_LITE + env_(new SpecialEnv(encrypted_env_ + ? encrypted_env_ + : (mem_env_ ? mem_env_ : Env::Default()))), + option_config_(kDefault) { + env_->SetBackgroundThreads(1, Env::LOW); + env_->SetBackgroundThreads(1, Env::HIGH); + dbname_ = test::TmpDir(env_) + path; + alternative_wal_dir_ = dbname_ + "/wal"; + alternative_db_log_dir_ = dbname_ + "/db_log_dir"; + auto options = CurrentOptions(); + options.env = env_; + auto delete_options = options; + delete_options.wal_dir = alternative_wal_dir_; + EXPECT_OK(DestroyDB(dbname_, delete_options)); + // Destroy it for not alternative WAL dir is used. + EXPECT_OK(DestroyDB(dbname_, options)); + db_ = nullptr; + Reopen(options); + Random::GetTLSInstance()->Reset(0xdeadbeef); +} + +DBTestBase::~DBTestBase() { + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + rocksdb::SyncPoint::GetInstance()->LoadDependency({}); + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); + Close(); + Options options; + options.db_paths.emplace_back(dbname_, 0); + options.db_paths.emplace_back(dbname_ + "_2", 0); + options.db_paths.emplace_back(dbname_ + "_3", 0); + options.db_paths.emplace_back(dbname_ + "_4", 0); + options.env = env_; + + if (getenv("KEEP_DB")) { + printf("DB is still at %s\n", dbname_.c_str()); + } else { + EXPECT_OK(DestroyDB(dbname_, options)); + } + delete env_; +} + +bool DBTestBase::ShouldSkipOptions(int option_config, int skip_mask) { +#ifdef ROCKSDB_LITE + // These options are not supported in ROCKSDB_LITE + if (option_config == kHashSkipList || + option_config == kPlainTableFirstBytePrefix || + option_config == kPlainTableCappedPrefix || + option_config == kPlainTableCappedPrefixNonMmap || + option_config == kPlainTableAllBytesPrefix || + option_config == kVectorRep || option_config == kHashLinkList || + option_config == kHashCuckoo || option_config == kUniversalCompaction || + option_config == kUniversalCompactionMultiLevel || + option_config == kUniversalSubcompactions || + option_config == kFIFOCompaction || + option_config == kConcurrentSkipList) { + return true; + } +#endif + + if ((skip_mask & kSkipUniversalCompaction) && + (option_config == kUniversalCompaction || + option_config == kUniversalCompactionMultiLevel)) { + return true; + } + if ((skip_mask & kSkipMergePut) && option_config == kMergePut) { + return true; + } + if ((skip_mask & kSkipNoSeekToLast) && + (option_config == kHashLinkList || option_config == kHashSkipList)) { + return true; + } + if ((skip_mask & kSkipPlainTable) && + (option_config == kPlainTableAllBytesPrefix || + option_config == kPlainTableFirstBytePrefix || + option_config == kPlainTableCappedPrefix || + option_config == kPlainTableCappedPrefixNonMmap)) { + return true; + } + if ((skip_mask & kSkipHashIndex) && + (option_config == kBlockBasedTableWithPrefixHashIndex || + option_config == kBlockBasedTableWithWholeKeyHashIndex)) { + return true; + } + if ((skip_mask & kSkipHashCuckoo) && (option_config == kHashCuckoo)) { + return true; + } + if ((skip_mask & kSkipFIFOCompaction) && option_config == kFIFOCompaction) { + return true; + } + if ((skip_mask & kSkipMmapReads) && option_config == kWalDirAndMmapReads) { + return true; + } + return false; +} + +// Switch to a fresh database with the next option configuration to +// test. Return false if there are no more configurations to test. +bool DBTestBase::ChangeOptions(int skip_mask) { + for (option_config_++; option_config_ < kEnd; option_config_++) { + if (ShouldSkipOptions(option_config_, skip_mask)) { + continue; + } + break; + } + + if (option_config_ >= kEnd) { + Destroy(last_options_); + return false; + } else { + auto options = CurrentOptions(); + options.create_if_missing = true; + DestroyAndReopen(options); + return true; + } +} + +// Switch between different compaction styles. +bool DBTestBase::ChangeCompactOptions() { + if (option_config_ == kDefault) { + option_config_ = kUniversalCompaction; + Destroy(last_options_); + auto options = CurrentOptions(); + options.create_if_missing = true; + TryReopen(options); + return true; + } else if (option_config_ == kUniversalCompaction) { + option_config_ = kUniversalCompactionMultiLevel; + Destroy(last_options_); + auto options = CurrentOptions(); + options.create_if_missing = true; + TryReopen(options); + return true; + } else if (option_config_ == kUniversalCompactionMultiLevel) { + option_config_ = kLevelSubcompactions; + Destroy(last_options_); + auto options = CurrentOptions(); + assert(options.max_subcompactions > 1); + TryReopen(options); + return true; + } else if (option_config_ == kLevelSubcompactions) { + option_config_ = kUniversalSubcompactions; + Destroy(last_options_); + auto options = CurrentOptions(); + assert(options.max_subcompactions > 1); + TryReopen(options); + return true; + } else { + return false; + } +} + +// Switch between different WAL settings +bool DBTestBase::ChangeWalOptions() { + if (option_config_ == kDefault) { + option_config_ = kDBLogDir; + Destroy(last_options_); + auto options = CurrentOptions(); + Destroy(options); + options.create_if_missing = true; + TryReopen(options); + return true; + } else if (option_config_ == kDBLogDir) { + option_config_ = kWalDirAndMmapReads; + Destroy(last_options_); + auto options = CurrentOptions(); + Destroy(options); + options.create_if_missing = true; + TryReopen(options); + return true; + } else if (option_config_ == kWalDirAndMmapReads) { + option_config_ = kRecycleLogFiles; + Destroy(last_options_); + auto options = CurrentOptions(); + Destroy(options); + TryReopen(options); + return true; + } else { + return false; + } +} + +// Switch between different filter policy +// Jump from kDefault to kFilter to kFullFilter +bool DBTestBase::ChangeFilterOptions() { + if (option_config_ == kDefault) { + option_config_ = kFilter; + } else if (option_config_ == kFilter) { + option_config_ = kFullFilterWithNewTableReaderForCompactions; + } else if (option_config_ == kFullFilterWithNewTableReaderForCompactions) { + option_config_ = kPartitionedFilterWithNewTableReaderForCompactions; + } else { + return false; + } + Destroy(last_options_); + + auto options = CurrentOptions(); + options.create_if_missing = true; + TryReopen(options); + return true; +} + +// Return the current option configuration. +Options DBTestBase::CurrentOptions( + const anon::OptionsOverride& options_override) const { + return GetOptions(option_config_, GetDefaultOptions(), options_override); +} + +Options DBTestBase::CurrentOptions( + const Options& default_options, + const anon::OptionsOverride& options_override) const { + return GetOptions(option_config_, default_options, options_override); +} + +Options DBTestBase::GetDefaultOptions() { + Options options; + options.write_buffer_size = 4090 * 4096; + options.target_file_size_base = 2 * 1024 * 1024; + options.max_bytes_for_level_base = 10 * 1024 * 1024; + options.max_open_files = 5000; + options.wal_recovery_mode = WALRecoveryMode::kTolerateCorruptedTailRecords; + options.compaction_pri = CompactionPri::kByCompensatedSize; + return options; +} + +Options DBTestBase::GetOptions( + int option_config, const Options& default_options, + const anon::OptionsOverride& options_override) const { + // this redundant copy is to minimize code change w/o having lint error. + Options options = default_options; + BlockBasedTableOptions table_options; + bool set_block_based_table_factory = true; +#if !defined(OS_MACOSX) && !defined(OS_WIN) && !defined(OS_SOLARIS) && \ + !defined(OS_AIX) + rocksdb::SyncPoint::GetInstance()->ClearCallBack( + "NewRandomAccessFile:O_DIRECT"); + rocksdb::SyncPoint::GetInstance()->ClearCallBack( + "NewWritableFile:O_DIRECT"); +#endif + + bool can_allow_mmap = IsMemoryMappedAccessSupported(); + switch (option_config) { +#ifndef ROCKSDB_LITE + case kHashSkipList: + options.prefix_extractor.reset(NewFixedPrefixTransform(1)); + options.memtable_factory.reset(NewHashSkipListRepFactory(16)); + options.allow_concurrent_memtable_write = false; + break; + case kPlainTableFirstBytePrefix: + options.table_factory.reset(new PlainTableFactory()); + options.prefix_extractor.reset(NewFixedPrefixTransform(1)); + options.allow_mmap_reads = can_allow_mmap; + options.max_sequential_skip_in_iterations = 999999; + set_block_based_table_factory = false; + break; + case kPlainTableCappedPrefix: + options.table_factory.reset(new PlainTableFactory()); + options.prefix_extractor.reset(NewCappedPrefixTransform(8)); + options.allow_mmap_reads = can_allow_mmap; + options.max_sequential_skip_in_iterations = 999999; + set_block_based_table_factory = false; + break; + case kPlainTableCappedPrefixNonMmap: + options.table_factory.reset(new PlainTableFactory()); + options.prefix_extractor.reset(NewCappedPrefixTransform(8)); + options.allow_mmap_reads = false; + options.max_sequential_skip_in_iterations = 999999; + set_block_based_table_factory = false; + break; + case kPlainTableAllBytesPrefix: + options.table_factory.reset(new PlainTableFactory()); + options.prefix_extractor.reset(NewNoopTransform()); + options.allow_mmap_reads = can_allow_mmap; + options.max_sequential_skip_in_iterations = 999999; + set_block_based_table_factory = false; + break; + case kVectorRep: + options.memtable_factory.reset(new VectorRepFactory(100)); + options.allow_concurrent_memtable_write = false; + break; + case kHashLinkList: + options.prefix_extractor.reset(NewFixedPrefixTransform(1)); + options.memtable_factory.reset( + NewHashLinkListRepFactory(4, 0, 3, true, 4)); + options.allow_concurrent_memtable_write = false; + break; + case kHashCuckoo: + options.memtable_factory.reset( + NewHashCuckooRepFactory(options.write_buffer_size)); + options.allow_concurrent_memtable_write = false; + break; +#endif // ROCKSDB_LITE + case kMergePut: + options.merge_operator = MergeOperators::CreatePutOperator(); + break; + case kFilter: + table_options.filter_policy.reset(NewBloomFilterPolicy(10, true)); + break; + case kFullFilterWithNewTableReaderForCompactions: + table_options.filter_policy.reset(NewBloomFilterPolicy(10, false)); + options.new_table_reader_for_compaction_inputs = true; + options.compaction_readahead_size = 10 * 1024 * 1024; + break; + case kPartitionedFilterWithNewTableReaderForCompactions: + table_options.filter_policy.reset(NewBloomFilterPolicy(10, false)); + table_options.partition_filters = true; + table_options.index_type = + BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch; + options.new_table_reader_for_compaction_inputs = true; + options.compaction_readahead_size = 10 * 1024 * 1024; + break; + case kUncompressed: + options.compression = kNoCompression; + break; + case kNumLevel_3: + options.num_levels = 3; + break; + case kDBLogDir: + options.db_log_dir = alternative_db_log_dir_; + break; + case kWalDirAndMmapReads: + options.wal_dir = alternative_wal_dir_; + // mmap reads should be orthogonal to WalDir setting, so we piggyback to + // this option config to test mmap reads as well + options.allow_mmap_reads = can_allow_mmap; + break; + case kManifestFileSize: + options.max_manifest_file_size = 50; // 50 bytes + break; + case kPerfOptions: + options.soft_rate_limit = 2.0; + options.delayed_write_rate = 8 * 1024 * 1024; + options.report_bg_io_stats = true; + // TODO(3.13) -- test more options + break; + case kUniversalCompaction: + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 1; + break; + case kUniversalCompactionMultiLevel: + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 8; + break; + case kCompressedBlockCache: + options.allow_mmap_writes = can_allow_mmap; + table_options.block_cache_compressed = NewLRUCache(8 * 1024 * 1024); + break; + case kInfiniteMaxOpenFiles: + options.max_open_files = -1; + break; + case kxxHashChecksum: { + table_options.checksum = kxxHash; + break; + } + case kFIFOCompaction: { + options.compaction_style = kCompactionStyleFIFO; + break; + } + case kBlockBasedTableWithPrefixHashIndex: { + table_options.index_type = BlockBasedTableOptions::kHashSearch; + options.prefix_extractor.reset(NewFixedPrefixTransform(1)); + break; + } + case kBlockBasedTableWithWholeKeyHashIndex: { + table_options.index_type = BlockBasedTableOptions::kHashSearch; + options.prefix_extractor.reset(NewNoopTransform()); + break; + } + case kBlockBasedTableWithPartitionedIndex: { + table_options.index_type = BlockBasedTableOptions::kTwoLevelIndexSearch; + options.prefix_extractor.reset(NewNoopTransform()); + break; + } + case kBlockBasedTableWithIndexRestartInterval: { + table_options.index_block_restart_interval = 8; + break; + } + case kOptimizeFiltersForHits: { + options.optimize_filters_for_hits = true; + set_block_based_table_factory = true; + break; + } + case kRowCache: { + options.row_cache = NewLRUCache(1024 * 1024); + break; + } + case kRecycleLogFiles: { + options.recycle_log_file_num = 2; + break; + } + case kLevelSubcompactions: { + options.max_subcompactions = 4; + break; + } + case kUniversalSubcompactions: { + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 8; + options.max_subcompactions = 4; + break; + } + case kConcurrentSkipList: { + options.allow_concurrent_memtable_write = true; + options.enable_write_thread_adaptive_yield = true; + break; + } + case kDirectIO: { + options.use_direct_reads = true; + options.use_direct_io_for_flush_and_compaction = true; + options.compaction_readahead_size = 2 * 1024 * 1024; +#if !defined(OS_MACOSX) && !defined(OS_WIN) && !defined(OS_SOLARIS) && \ + !defined(OS_AIX) + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "NewWritableFile:O_DIRECT", [&](void* arg) { + int* val = static_cast(arg); + *val &= ~O_DIRECT; + }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "NewRandomAccessFile:O_DIRECT", [&](void* arg) { + int* val = static_cast(arg); + *val &= ~O_DIRECT; + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); +#endif + break; + } + case kPipelinedWrite: { + options.enable_pipelined_write = true; + break; + } + case kConcurrentWALWrites: { + // This options optimize 2PC commit path + options.concurrent_prepare = true; + options.manual_wal_flush = true; + break; + } + + default: + break; + } + + if (options_override.filter_policy) { + table_options.filter_policy = options_override.filter_policy; + table_options.partition_filters = options_override.partition_filters; + table_options.metadata_block_size = options_override.metadata_block_size; + } + if (set_block_based_table_factory) { + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + } + options.env = env_; + options.create_if_missing = true; + options.fail_if_options_file_error = true; + return options; +} + +void DBTestBase::CreateColumnFamilies(const std::vector& cfs, + const Options& options) { + ColumnFamilyOptions cf_opts(options); + size_t cfi = handles_.size(); + handles_.resize(cfi + cfs.size()); + for (auto cf : cfs) { + ASSERT_OK(db_->CreateColumnFamily(cf_opts, cf, &handles_[cfi++])); + } +} + +void DBTestBase::CreateAndReopenWithCF(const std::vector& cfs, + const Options& options) { + CreateColumnFamilies(cfs, options); + std::vector cfs_plus_default = cfs; + cfs_plus_default.insert(cfs_plus_default.begin(), kDefaultColumnFamilyName); + ReopenWithColumnFamilies(cfs_plus_default, options); +} + +void DBTestBase::ReopenWithColumnFamilies(const std::vector& cfs, + const std::vector& options) { + ASSERT_OK(TryReopenWithColumnFamilies(cfs, options)); +} + +void DBTestBase::ReopenWithColumnFamilies(const std::vector& cfs, + const Options& options) { + ASSERT_OK(TryReopenWithColumnFamilies(cfs, options)); +} + +Status DBTestBase::TryReopenWithColumnFamilies( + const std::vector& cfs, const std::vector& options) { + Close(); + EXPECT_EQ(cfs.size(), options.size()); + std::vector column_families; + for (size_t i = 0; i < cfs.size(); ++i) { + column_families.push_back(ColumnFamilyDescriptor(cfs[i], options[i])); + } + DBOptions db_opts = DBOptions(options[0]); + return DB::Open(db_opts, dbname_, column_families, &handles_, &db_); +} + +Status DBTestBase::TryReopenWithColumnFamilies( + const std::vector& cfs, const Options& options) { + Close(); + std::vector v_opts(cfs.size(), options); + return TryReopenWithColumnFamilies(cfs, v_opts); +} + +void DBTestBase::Reopen(const Options& options) { + ASSERT_OK(TryReopen(options)); +} + +void DBTestBase::Close() { + for (auto h : handles_) { + db_->DestroyColumnFamilyHandle(h); + } + handles_.clear(); + delete db_; + db_ = nullptr; +} + +void DBTestBase::DestroyAndReopen(const Options& options) { + // Destroy using last options + Destroy(last_options_); + ASSERT_OK(TryReopen(options)); +} + +void DBTestBase::Destroy(const Options& options) { + Close(); + ASSERT_OK(DestroyDB(dbname_, options)); +} + +Status DBTestBase::ReadOnlyReopen(const Options& options) { + return DB::OpenForReadOnly(options, dbname_, &db_); +} + +Status DBTestBase::TryReopen(const Options& options) { + Close(); + last_options_.table_factory.reset(); + // Note: operator= is an unsafe approach here since it destructs shared_ptr in + // the same order of their creation, in contrast to destructors which + // destructs them in the opposite order of creation. One particular problme is + // that the cache destructor might invoke callback functions that use Option + // members such as statistics. To work around this problem, we manually call + // destructor of table_facotry which eventually clears the block cache. + last_options_ = options; + return DB::Open(options, dbname_, &db_); +} + +bool DBTestBase::IsDirectIOSupported() { + EnvOptions env_options; + env_options.use_mmap_writes = false; + env_options.use_direct_writes = true; + std::string tmp = TempFileName(dbname_, 999); + Status s; + { + unique_ptr file; + s = env_->NewWritableFile(tmp, &file, env_options); + } + if (s.ok()) { + s = env_->DeleteFile(tmp); + } + return s.ok(); +} + +bool DBTestBase::IsMemoryMappedAccessSupported() const { + return (!encrypted_env_); +} + +Status DBTestBase::Flush(int cf) { + if (cf == 0) { + return db_->Flush(FlushOptions()); + } else { + return db_->Flush(FlushOptions(), handles_[cf]); + } +} + +Status DBTestBase::Put(const Slice& k, const Slice& v, WriteOptions wo) { + if (kMergePut == option_config_) { + return db_->Merge(wo, k, v); + } else { + return db_->Put(wo, k, v); + } +} + +Status DBTestBase::Put(int cf, const Slice& k, const Slice& v, + WriteOptions wo) { + if (kMergePut == option_config_) { + return db_->Merge(wo, handles_[cf], k, v); + } else { + return db_->Put(wo, handles_[cf], k, v); + } +} + +Status DBTestBase::Merge(const Slice& k, const Slice& v, WriteOptions wo) { + return db_->Merge(wo, k, v); +} + +Status DBTestBase::Merge(int cf, const Slice& k, const Slice& v, + WriteOptions wo) { + return db_->Merge(wo, handles_[cf], k, v); +} + +Status DBTestBase::Delete(const std::string& k) { + return db_->Delete(WriteOptions(), k); +} + +Status DBTestBase::Delete(int cf, const std::string& k) { + return db_->Delete(WriteOptions(), handles_[cf], k); +} + +Status DBTestBase::SingleDelete(const std::string& k) { + return db_->SingleDelete(WriteOptions(), k); +} + +Status DBTestBase::SingleDelete(int cf, const std::string& k) { + return db_->SingleDelete(WriteOptions(), handles_[cf], k); +} + +std::string DBTestBase::Get(const std::string& k, const Snapshot* snapshot) { + ReadOptions options; + options.verify_checksums = true; + options.snapshot = snapshot; + std::string result; + Status s = db_->Get(options, k, &result); + if (s.IsNotFound()) { + result = "NOT_FOUND"; + } else if (!s.ok()) { + result = s.ToString(); + } + return result; +} + +std::string DBTestBase::Get(int cf, const std::string& k, + const Snapshot* snapshot) { + ReadOptions options; + options.verify_checksums = true; + options.snapshot = snapshot; + std::string result; + Status s = db_->Get(options, handles_[cf], k, &result); + if (s.IsNotFound()) { + result = "NOT_FOUND"; + } else if (!s.ok()) { + result = s.ToString(); + } + return result; +} + +Status DBTestBase::Get(const std::string& k, PinnableSlice* v) { + ReadOptions options; + options.verify_checksums = true; + Status s = dbfull()->Get(options, dbfull()->DefaultColumnFamily(), k, v); + return s; +} + +uint64_t DBTestBase::GetNumSnapshots() { + uint64_t int_num; + EXPECT_TRUE(dbfull()->GetIntProperty("rocksdb.num-snapshots", &int_num)); + return int_num; +} + +uint64_t DBTestBase::GetTimeOldestSnapshots() { + uint64_t int_num; + EXPECT_TRUE( + dbfull()->GetIntProperty("rocksdb.oldest-snapshot-time", &int_num)); + return int_num; +} + +// Return a string that contains all key,value pairs in order, +// formatted like "(k1->v1)(k2->v2)". +std::string DBTestBase::Contents(int cf) { + std::vector forward; + std::string result; + Iterator* iter = (cf == 0) ? db_->NewIterator(ReadOptions()) + : db_->NewIterator(ReadOptions(), handles_[cf]); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + std::string s = IterStatus(iter); + result.push_back('('); + result.append(s); + result.push_back(')'); + forward.push_back(s); + } + + // Check reverse iteration results are the reverse of forward results + unsigned int matched = 0; + for (iter->SeekToLast(); iter->Valid(); iter->Prev()) { + EXPECT_LT(matched, forward.size()); + EXPECT_EQ(IterStatus(iter), forward[forward.size() - matched - 1]); + matched++; + } + EXPECT_EQ(matched, forward.size()); + + delete iter; + return result; +} + +std::string DBTestBase::AllEntriesFor(const Slice& user_key, int cf) { + Arena arena; + auto options = CurrentOptions(); + InternalKeyComparator icmp(options.comparator); + RangeDelAggregator range_del_agg(icmp, {} /* snapshots */); + ScopedArenaIterator iter; + if (cf == 0) { + iter.set(dbfull()->NewInternalIterator(&arena, &range_del_agg)); + } else { + iter.set( + dbfull()->NewInternalIterator(&arena, &range_del_agg, handles_[cf])); + } + InternalKey target(user_key, kMaxSequenceNumber, kTypeValue); + iter->Seek(target.Encode()); + std::string result; + if (!iter->status().ok()) { + result = iter->status().ToString(); + } else { + result = "[ "; + bool first = true; + while (iter->Valid()) { + ParsedInternalKey ikey(Slice(), 0, kTypeValue); + if (!ParseInternalKey(iter->key(), &ikey)) { + result += "CORRUPTED"; + } else { + if (!last_options_.comparator->Equal(ikey.user_key, user_key)) { + break; + } + if (!first) { + result += ", "; + } + first = false; + switch (ikey.type) { + case kTypeValue: + result += iter->value().ToString(); + break; + case kTypeMerge: + // keep it the same as kTypeValue for testing kMergePut + result += iter->value().ToString(); + break; + case kTypeDeletion: + result += "DEL"; + break; + case kTypeSingleDeletion: + result += "SDEL"; + break; + default: + assert(false); + break; + } + } + iter->Next(); + } + if (!first) { + result += " "; + } + result += "]"; + } + return result; +} + +#ifndef ROCKSDB_LITE +int DBTestBase::NumSortedRuns(int cf) { + ColumnFamilyMetaData cf_meta; + if (cf == 0) { + db_->GetColumnFamilyMetaData(&cf_meta); + } else { + db_->GetColumnFamilyMetaData(handles_[cf], &cf_meta); + } + int num_sr = static_cast(cf_meta.levels[0].files.size()); + for (size_t i = 1U; i < cf_meta.levels.size(); i++) { + if (cf_meta.levels[i].files.size() > 0) { + num_sr++; + } + } + return num_sr; +} + +uint64_t DBTestBase::TotalSize(int cf) { + ColumnFamilyMetaData cf_meta; + if (cf == 0) { + db_->GetColumnFamilyMetaData(&cf_meta); + } else { + db_->GetColumnFamilyMetaData(handles_[cf], &cf_meta); + } + return cf_meta.size; +} + +uint64_t DBTestBase::SizeAtLevel(int level) { + std::vector metadata; + db_->GetLiveFilesMetaData(&metadata); + uint64_t sum = 0; + for (const auto& m : metadata) { + if (m.level == level) { + sum += m.size; + } + } + return sum; +} + +size_t DBTestBase::TotalLiveFiles(int cf) { + ColumnFamilyMetaData cf_meta; + if (cf == 0) { + db_->GetColumnFamilyMetaData(&cf_meta); + } else { + db_->GetColumnFamilyMetaData(handles_[cf], &cf_meta); + } + size_t num_files = 0; + for (auto& level : cf_meta.levels) { + num_files += level.files.size(); + } + return num_files; +} + +size_t DBTestBase::CountLiveFiles() { + std::vector metadata; + db_->GetLiveFilesMetaData(&metadata); + return metadata.size(); +} +#endif // ROCKSDB_LITE + +int DBTestBase::NumTableFilesAtLevel(int level, int cf) { + std::string property; + if (cf == 0) { + // default cfd + EXPECT_TRUE(db_->GetProperty( + "rocksdb.num-files-at-level" + NumberToString(level), &property)); + } else { + EXPECT_TRUE(db_->GetProperty( + handles_[cf], "rocksdb.num-files-at-level" + NumberToString(level), + &property)); + } + return atoi(property.c_str()); +} + +double DBTestBase::CompressionRatioAtLevel(int level, int cf) { + std::string property; + if (cf == 0) { + // default cfd + EXPECT_TRUE(db_->GetProperty( + "rocksdb.compression-ratio-at-level" + NumberToString(level), + &property)); + } else { + EXPECT_TRUE(db_->GetProperty( + handles_[cf], + "rocksdb.compression-ratio-at-level" + NumberToString(level), + &property)); + } + return std::stod(property); +} + +int DBTestBase::TotalTableFiles(int cf, int levels) { + if (levels == -1) { + levels = (cf == 0) ? db_->NumberLevels() : db_->NumberLevels(handles_[1]); + } + int result = 0; + for (int level = 0; level < levels; level++) { + result += NumTableFilesAtLevel(level, cf); + } + return result; +} + +// Return spread of files per level +std::string DBTestBase::FilesPerLevel(int cf) { + int num_levels = + (cf == 0) ? db_->NumberLevels() : db_->NumberLevels(handles_[1]); + std::string result; + size_t last_non_zero_offset = 0; + for (int level = 0; level < num_levels; level++) { + int f = NumTableFilesAtLevel(level, cf); + char buf[100]; + snprintf(buf, sizeof(buf), "%s%d", (level ? "," : ""), f); + result += buf; + if (f > 0) { + last_non_zero_offset = result.size(); + } + } + result.resize(last_non_zero_offset); + return result; +} + +size_t DBTestBase::CountFiles() { + std::vector files; + env_->GetChildren(dbname_, &files); + + std::vector logfiles; + if (dbname_ != last_options_.wal_dir) { + env_->GetChildren(last_options_.wal_dir, &logfiles); + } + + return files.size() + logfiles.size(); +} + +uint64_t DBTestBase::Size(const Slice& start, const Slice& limit, int cf) { + Range r(start, limit); + uint64_t size; + if (cf == 0) { + db_->GetApproximateSizes(&r, 1, &size); + } else { + db_->GetApproximateSizes(handles_[1], &r, 1, &size); + } + return size; +} + +void DBTestBase::Compact(int cf, const Slice& start, const Slice& limit, + uint32_t target_path_id) { + CompactRangeOptions compact_options; + compact_options.target_path_id = target_path_id; + ASSERT_OK(db_->CompactRange(compact_options, handles_[cf], &start, &limit)); +} + +void DBTestBase::Compact(int cf, const Slice& start, const Slice& limit) { + ASSERT_OK( + db_->CompactRange(CompactRangeOptions(), handles_[cf], &start, &limit)); +} + +void DBTestBase::Compact(const Slice& start, const Slice& limit) { + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), &start, &limit)); +} + +// Do n memtable compactions, each of which produces an sstable +// covering the range [small,large]. +void DBTestBase::MakeTables(int n, const std::string& small, + const std::string& large, int cf) { + for (int i = 0; i < n; i++) { + ASSERT_OK(Put(cf, small, "begin")); + ASSERT_OK(Put(cf, large, "end")); + ASSERT_OK(Flush(cf)); + MoveFilesToLevel(n - i - 1, cf); + } +} + +// Prevent pushing of new sstables into deeper levels by adding +// tables that cover a specified range to all levels. +void DBTestBase::FillLevels(const std::string& smallest, + const std::string& largest, int cf) { + MakeTables(db_->NumberLevels(handles_[cf]), smallest, largest, cf); +} + +void DBTestBase::MoveFilesToLevel(int level, int cf) { + for (int l = 0; l < level; ++l) { + if (cf > 0) { + dbfull()->TEST_CompactRange(l, nullptr, nullptr, handles_[cf]); + } else { + dbfull()->TEST_CompactRange(l, nullptr, nullptr); + } + } +} + +void DBTestBase::DumpFileCounts(const char* label) { + fprintf(stderr, "---\n%s:\n", label); + fprintf(stderr, "maxoverlap: %" PRIu64 "\n", + dbfull()->TEST_MaxNextLevelOverlappingBytes()); + for (int level = 0; level < db_->NumberLevels(); level++) { + int num = NumTableFilesAtLevel(level); + if (num > 0) { + fprintf(stderr, " level %3d : %d files\n", level, num); + } + } +} + +std::string DBTestBase::DumpSSTableList() { + std::string property; + db_->GetProperty("rocksdb.sstables", &property); + return property; +} + +void DBTestBase::GetSstFiles(std::string path, + std::vector* files) { + env_->GetChildren(path, files); + + files->erase( + std::remove_if(files->begin(), files->end(), [](std::string name) { + uint64_t number; + FileType type; + return !(ParseFileName(name, &number, &type) && type == kTableFile); + }), files->end()); +} + +int DBTestBase::GetSstFileCount(std::string path) { + std::vector files; + GetSstFiles(path, &files); + return static_cast(files.size()); +} + +// this will generate non-overlapping files since it keeps increasing key_idx +void DBTestBase::GenerateNewFile(int cf, Random* rnd, int* key_idx, + bool nowait) { + for (int i = 0; i < KNumKeysByGenerateNewFile; i++) { + ASSERT_OK(Put(cf, Key(*key_idx), RandomString(rnd, (i == 99) ? 1 : 990))); + (*key_idx)++; + } + if (!nowait) { + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } +} + +// this will generate non-overlapping files since it keeps increasing key_idx +void DBTestBase::GenerateNewFile(Random* rnd, int* key_idx, bool nowait) { + for (int i = 0; i < KNumKeysByGenerateNewFile; i++) { + ASSERT_OK(Put(Key(*key_idx), RandomString(rnd, (i == 99) ? 1 : 990))); + (*key_idx)++; + } + if (!nowait) { + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } +} + +const int DBTestBase::kNumKeysByGenerateNewRandomFile = 51; + +void DBTestBase::GenerateNewRandomFile(Random* rnd, bool nowait) { + for (int i = 0; i < kNumKeysByGenerateNewRandomFile; i++) { + ASSERT_OK(Put("key" + RandomString(rnd, 7), RandomString(rnd, 2000))); + } + ASSERT_OK(Put("key" + RandomString(rnd, 7), RandomString(rnd, 200))); + if (!nowait) { + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } +} + +std::string DBTestBase::IterStatus(Iterator* iter) { + std::string result; + if (iter->Valid()) { + result = iter->key().ToString() + "->" + iter->value().ToString(); + } else { + result = "(invalid)"; + } + return result; +} + +Options DBTestBase::OptionsForLogIterTest() { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.WAL_ttl_seconds = 1000; + return options; +} + +std::string DBTestBase::DummyString(size_t len, char c) { + return std::string(len, c); +} + +void DBTestBase::VerifyIterLast(std::string expected_key, int cf) { + Iterator* iter; + ReadOptions ro; + if (cf == 0) { + iter = db_->NewIterator(ro); + } else { + iter = db_->NewIterator(ro, handles_[cf]); + } + iter->SeekToLast(); + ASSERT_EQ(IterStatus(iter), expected_key); + delete iter; +} + +// Used to test InplaceUpdate + +// If previous value is nullptr or delta is > than previous value, +// sets newValue with delta +// If previous value is not empty, +// updates previous value with 'b' string of previous value size - 1. +UpdateStatus DBTestBase::updateInPlaceSmallerSize(char* prevValue, + uint32_t* prevSize, + Slice delta, + std::string* newValue) { + if (prevValue == nullptr) { + *newValue = std::string(delta.size(), 'c'); + return UpdateStatus::UPDATED; + } else { + *prevSize = *prevSize - 1; + std::string str_b = std::string(*prevSize, 'b'); + memcpy(prevValue, str_b.c_str(), str_b.size()); + return UpdateStatus::UPDATED_INPLACE; + } +} + +UpdateStatus DBTestBase::updateInPlaceSmallerVarintSize(char* prevValue, + uint32_t* prevSize, + Slice delta, + std::string* newValue) { + if (prevValue == nullptr) { + *newValue = std::string(delta.size(), 'c'); + return UpdateStatus::UPDATED; + } else { + *prevSize = 1; + std::string str_b = std::string(*prevSize, 'b'); + memcpy(prevValue, str_b.c_str(), str_b.size()); + return UpdateStatus::UPDATED_INPLACE; + } +} + +UpdateStatus DBTestBase::updateInPlaceLargerSize(char* prevValue, + uint32_t* prevSize, + Slice delta, + std::string* newValue) { + *newValue = std::string(delta.size(), 'c'); + return UpdateStatus::UPDATED; +} + +UpdateStatus DBTestBase::updateInPlaceNoAction(char* prevValue, + uint32_t* prevSize, Slice delta, + std::string* newValue) { + return UpdateStatus::UPDATE_FAILED; +} + +// Utility method to test InplaceUpdate +void DBTestBase::validateNumberOfEntries(int numValues, int cf) { + ScopedArenaIterator iter; + Arena arena; + auto options = CurrentOptions(); + InternalKeyComparator icmp(options.comparator); + RangeDelAggregator range_del_agg(icmp, {} /* snapshots */); + if (cf != 0) { + iter.set( + dbfull()->NewInternalIterator(&arena, &range_del_agg, handles_[cf])); + } else { + iter.set(dbfull()->NewInternalIterator(&arena, &range_del_agg)); + } + iter->SeekToFirst(); + ASSERT_EQ(iter->status().ok(), true); + int seq = numValues; + while (iter->Valid()) { + ParsedInternalKey ikey; + ikey.sequence = -1; + ASSERT_EQ(ParseInternalKey(iter->key(), &ikey), true); + + // checks sequence number for updates + ASSERT_EQ(ikey.sequence, (unsigned)seq--); + iter->Next(); + } + ASSERT_EQ(0, seq); +} + +void DBTestBase::CopyFile(const std::string& source, + const std::string& destination, uint64_t size) { + const EnvOptions soptions; + unique_ptr srcfile; + ASSERT_OK(env_->NewSequentialFile(source, &srcfile, soptions)); + unique_ptr destfile; + ASSERT_OK(env_->NewWritableFile(destination, &destfile, soptions)); + + if (size == 0) { + // default argument means copy everything + ASSERT_OK(env_->GetFileSize(source, &size)); + } + + char buffer[4096]; + Slice slice; + while (size > 0) { + uint64_t one = std::min(uint64_t(sizeof(buffer)), size); + ASSERT_OK(srcfile->Read(one, &slice, buffer)); + ASSERT_OK(destfile->Append(slice)); + size -= slice.size(); + } + ASSERT_OK(destfile->Close()); +} + +std::unordered_map DBTestBase::GetAllSSTFiles( + uint64_t* total_size) { + std::unordered_map res; + + if (total_size) { + *total_size = 0; + } + std::vector files; + env_->GetChildren(dbname_, &files); + for (auto& file_name : files) { + uint64_t number; + FileType type; + std::string file_path = dbname_ + "/" + file_name; + if (ParseFileName(file_name, &number, &type) && type == kTableFile) { + uint64_t file_size = 0; + env_->GetFileSize(file_path, &file_size); + res[file_path] = file_size; + if (total_size) { + *total_size += file_size; + } + } + } + return res; +} + +std::vector DBTestBase::ListTableFiles(Env* env, + const std::string& path) { + std::vector files; + std::vector file_numbers; + env->GetChildren(path, &files); + uint64_t number; + FileType type; + for (size_t i = 0; i < files.size(); ++i) { + if (ParseFileName(files[i], &number, &type)) { + if (type == kTableFile) { + file_numbers.push_back(number); + } + } + } + return file_numbers; +} + +void DBTestBase::VerifyDBFromMap(std::map true_data, + size_t* total_reads_res, bool tailing_iter, + std::map status) { + size_t total_reads = 0; + + for (auto& kv : true_data) { + Status s = status[kv.first]; + if (s.ok()) { + ASSERT_EQ(Get(kv.first), kv.second); + } else { + std::string value; + ASSERT_EQ(s, db_->Get(ReadOptions(), kv.first, &value)); + } + total_reads++; + } + + // Normal Iterator + { + int iter_cnt = 0; + ReadOptions ro; + ro.total_order_seek = true; + Iterator* iter = db_->NewIterator(ro); + // Verify Iterator::Next() + iter_cnt = 0; + auto data_iter = true_data.begin(); + Status s; + for (iter->SeekToFirst(); iter->Valid(); iter->Next(), data_iter++) { + ASSERT_EQ(iter->key().ToString(), data_iter->first); + Status current_status = status[data_iter->first]; + if (!current_status.ok()) { + s = current_status; + } + ASSERT_EQ(iter->status(), s); + if (current_status.ok()) { + ASSERT_EQ(iter->value().ToString(), data_iter->second); + } + iter_cnt++; + total_reads++; + } + ASSERT_EQ(data_iter, true_data.end()) << iter_cnt << " / " + << true_data.size(); + delete iter; + + // Verify Iterator::Prev() + // Use a new iterator to make sure its status is clean. + iter = db_->NewIterator(ro); + iter_cnt = 0; + s = Status::OK(); + auto data_rev = true_data.rbegin(); + for (iter->SeekToLast(); iter->Valid(); iter->Prev(), data_rev++) { + ASSERT_EQ(iter->key().ToString(), data_rev->first); + Status current_status = status[data_rev->first]; + if (!current_status.ok()) { + s = current_status; + } + ASSERT_EQ(iter->status(), s); + if (current_status.ok()) { + ASSERT_EQ(iter->value().ToString(), data_rev->second); + } + iter_cnt++; + total_reads++; + } + ASSERT_EQ(data_rev, true_data.rend()) << iter_cnt << " / " + << true_data.size(); + + // Verify Iterator::Seek() + for (auto kv : true_data) { + iter->Seek(kv.first); + ASSERT_EQ(kv.first, iter->key().ToString()); + ASSERT_EQ(kv.second, iter->value().ToString()); + total_reads++; + } + delete iter; + } + + if (tailing_iter) { +#ifndef ROCKSDB_LITE + // Tailing iterator + int iter_cnt = 0; + ReadOptions ro; + ro.tailing = true; + ro.total_order_seek = true; + Iterator* iter = db_->NewIterator(ro); + + // Verify ForwardIterator::Next() + iter_cnt = 0; + auto data_iter = true_data.begin(); + for (iter->SeekToFirst(); iter->Valid(); iter->Next(), data_iter++) { + ASSERT_EQ(iter->key().ToString(), data_iter->first); + ASSERT_EQ(iter->value().ToString(), data_iter->second); + iter_cnt++; + total_reads++; + } + ASSERT_EQ(data_iter, true_data.end()) << iter_cnt << " / " + << true_data.size(); + + // Verify ForwardIterator::Seek() + for (auto kv : true_data) { + iter->Seek(kv.first); + ASSERT_EQ(kv.first, iter->key().ToString()); + ASSERT_EQ(kv.second, iter->value().ToString()); + total_reads++; + } + + delete iter; +#endif // ROCKSDB_LITE + } + + if (total_reads_res) { + *total_reads_res = total_reads; + } +} + +void DBTestBase::VerifyDBInternal( + std::vector> true_data) { + Arena arena; + InternalKeyComparator icmp(last_options_.comparator); + RangeDelAggregator range_del_agg(icmp, {}); + auto iter = dbfull()->NewInternalIterator(&arena, &range_del_agg); + iter->SeekToFirst(); + for (auto p : true_data) { + ASSERT_TRUE(iter->Valid()); + ParsedInternalKey ikey; + ASSERT_TRUE(ParseInternalKey(iter->key(), &ikey)); + ASSERT_EQ(p.first, ikey.user_key); + ASSERT_EQ(p.second, iter->value()); + iter->Next(); + }; + ASSERT_FALSE(iter->Valid()); + iter->~InternalIterator(); +} + +#ifndef ROCKSDB_LITE + +uint64_t DBTestBase::GetNumberOfSstFilesForColumnFamily( + DB* db, std::string column_family_name) { + std::vector metadata; + db->GetLiveFilesMetaData(&metadata); + uint64_t result = 0; + for (auto& fileMetadata : metadata) { + result += (fileMetadata.column_family_name == column_family_name); + } + return result; +} +#endif // ROCKSDB_LITE + +} // namespace rocksdb diff --git a/db/db_test_util.h b/db/db_test_util.h new file mode 100644 index 00000000000..f2caa46ca29 --- /dev/null +++ b/db/db_test_util.h @@ -0,0 +1,970 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "db/db_impl.h" +#include "db/dbformat.h" +#include "env/mock_env.h" +#include "memtable/hash_linklist_rep.h" +#include "rocksdb/cache.h" +#include "rocksdb/compaction_filter.h" +#include "rocksdb/convenience.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/filter_policy.h" +#include "rocksdb/options.h" +#include "rocksdb/slice.h" +#include "rocksdb/sst_file_writer.h" +#include "rocksdb/statistics.h" +#include "rocksdb/table.h" +#include "rocksdb/utilities/checkpoint.h" +#include "table/block_based_table_factory.h" +#include "table/mock_table.h" +#include "table/plain_table_factory.h" +#include "table/scoped_arena_iterator.h" +#include "util/compression.h" +#include "util/filename.h" +#include "util/mutexlock.h" + +#include "util/string_util.h" +#include "util/sync_point.h" +#include "util/testharness.h" +#include "util/testutil.h" +#include "utilities/merge_operators.h" + +namespace rocksdb { + +namespace anon { +class AtomicCounter { + public: + explicit AtomicCounter(Env* env = NULL) + : env_(env), cond_count_(&mu_), count_(0) {} + + void Increment() { + MutexLock l(&mu_); + count_++; + cond_count_.SignalAll(); + } + + int Read() { + MutexLock l(&mu_); + return count_; + } + + bool WaitFor(int count) { + MutexLock l(&mu_); + + uint64_t start = env_->NowMicros(); + while (count_ < count) { + uint64_t now = env_->NowMicros(); + cond_count_.TimedWait(now + /*1s*/ 1 * 1000 * 1000); + if (env_->NowMicros() - start > /*10s*/ 10 * 1000 * 1000) { + return false; + } + if (count_ < count) { + GTEST_LOG_(WARNING) << "WaitFor is taking more time than usual"; + } + } + + return true; + } + + void Reset() { + MutexLock l(&mu_); + count_ = 0; + cond_count_.SignalAll(); + } + + private: + Env* env_; + port::Mutex mu_; + port::CondVar cond_count_; + int count_; +}; + +struct OptionsOverride { + std::shared_ptr filter_policy = nullptr; + // These will be used only if filter_policy is set + bool partition_filters = false; + uint64_t metadata_block_size = 1024; + BlockBasedTableOptions::IndexType index_type = + BlockBasedTableOptions::IndexType::kBinarySearch; + + // Used as a bit mask of individual enums in which to skip an XF test point + int skip_policy = 0; +}; + +} // namespace anon + +enum SkipPolicy { kSkipNone = 0, kSkipNoSnapshot = 1, kSkipNoPrefix = 2 }; + +// A hacky skip list mem table that triggers flush after number of entries. +class SpecialMemTableRep : public MemTableRep { + public: + explicit SpecialMemTableRep(Allocator* allocator, MemTableRep* memtable, + int num_entries_flush) + : MemTableRep(allocator), + memtable_(memtable), + num_entries_flush_(num_entries_flush), + num_entries_(0) {} + + virtual KeyHandle Allocate(const size_t len, char** buf) override { + return memtable_->Allocate(len, buf); + } + + // Insert key into the list. + // REQUIRES: nothing that compares equal to key is currently in the list. + virtual void Insert(KeyHandle handle) override { + memtable_->Insert(handle); + num_entries_++; + } + + // Returns true iff an entry that compares equal to key is in the list. + virtual bool Contains(const char* key) const override { + return memtable_->Contains(key); + } + + virtual size_t ApproximateMemoryUsage() override { + // Return a high memory usage when number of entries exceeds the threshold + // to trigger a flush. + return (num_entries_ < num_entries_flush_) ? 0 : 1024 * 1024 * 1024; + } + + virtual void Get(const LookupKey& k, void* callback_args, + bool (*callback_func)(void* arg, + const char* entry)) override { + memtable_->Get(k, callback_args, callback_func); + } + + uint64_t ApproximateNumEntries(const Slice& start_ikey, + const Slice& end_ikey) override { + return memtable_->ApproximateNumEntries(start_ikey, end_ikey); + } + + virtual MemTableRep::Iterator* GetIterator(Arena* arena = nullptr) override { + return memtable_->GetIterator(arena); + } + + virtual ~SpecialMemTableRep() override {} + + private: + unique_ptr memtable_; + int num_entries_flush_; + int num_entries_; +}; + +// The factory for the hacky skip list mem table that triggers flush after +// number of entries exceeds a threshold. +class SpecialSkipListFactory : public MemTableRepFactory { + public: + // After number of inserts exceeds `num_entries_flush` in a mem table, trigger + // flush. + explicit SpecialSkipListFactory(int num_entries_flush) + : num_entries_flush_(num_entries_flush) {} + + using MemTableRepFactory::CreateMemTableRep; + virtual MemTableRep* CreateMemTableRep( + const MemTableRep::KeyComparator& compare, Allocator* allocator, + const SliceTransform* transform, Logger* logger) override { + return new SpecialMemTableRep( + allocator, factory_.CreateMemTableRep(compare, allocator, transform, 0), + num_entries_flush_); + } + virtual const char* Name() const override { return "SkipListFactory"; } + + bool IsInsertConcurrentlySupported() const override { + return factory_.IsInsertConcurrentlySupported(); + } + + private: + SkipListFactory factory_; + int num_entries_flush_; +}; + +// Special Env used to delay background operations +class SpecialEnv : public EnvWrapper { + public: + explicit SpecialEnv(Env* base); + + Status NewWritableFile(const std::string& f, unique_ptr* r, + const EnvOptions& soptions) override { + class SSTableFile : public WritableFile { + private: + SpecialEnv* env_; + unique_ptr base_; + + public: + SSTableFile(SpecialEnv* env, unique_ptr&& base) + : env_(env), base_(std::move(base)) {} + Status Append(const Slice& data) override { + if (env_->table_write_callback_) { + (*env_->table_write_callback_)(); + } + if (env_->drop_writes_.load(std::memory_order_acquire)) { + // Drop writes on the floor + return Status::OK(); + } else if (env_->no_space_.load(std::memory_order_acquire)) { + return Status::NoSpace("No space left on device"); + } else { + env_->bytes_written_ += data.size(); + return base_->Append(data); + } + } + Status PositionedAppend(const Slice& data, uint64_t offset) override { + if (env_->table_write_callback_) { + (*env_->table_write_callback_)(); + } + if (env_->drop_writes_.load(std::memory_order_acquire)) { + // Drop writes on the floor + return Status::OK(); + } else if (env_->no_space_.load(std::memory_order_acquire)) { + return Status::NoSpace("No space left on device"); + } else { + env_->bytes_written_ += data.size(); + return base_->PositionedAppend(data, offset); + } + } + Status Truncate(uint64_t size) override { return base_->Truncate(size); } + Status RangeSync(uint64_t offset, uint64_t nbytes) override { + Status s = base_->RangeSync(offset, nbytes); +#if !(defined NDEBUG) || !defined(OS_WIN) + TEST_SYNC_POINT_CALLBACK("SpecialEnv::SStableFile::RangeSync", &s); +#endif // !(defined NDEBUG) || !defined(OS_WIN) + return s; + } + Status Close() override { +// SyncPoint is not supported in Released Windows Mode. +#if !(defined NDEBUG) || !defined(OS_WIN) + // Check preallocation size + // preallocation size is never passed to base file. + size_t preallocation_size = preallocation_block_size(); + TEST_SYNC_POINT_CALLBACK("DBTestWritableFile.GetPreallocationStatus", + &preallocation_size); +#endif // !(defined NDEBUG) || !defined(OS_WIN) + Status s = base_->Close(); +#if !(defined NDEBUG) || !defined(OS_WIN) + TEST_SYNC_POINT_CALLBACK("SpecialEnv::SStableFile::Close", &s); +#endif // !(defined NDEBUG) || !defined(OS_WIN) + return s; + } + Status Flush() override { return base_->Flush(); } + Status Sync() override { + ++env_->sync_counter_; + while (env_->delay_sstable_sync_.load(std::memory_order_acquire)) { + env_->SleepForMicroseconds(100000); + } + Status s = base_->Sync(); +#if !(defined NDEBUG) || !defined(OS_WIN) + TEST_SYNC_POINT_CALLBACK("SpecialEnv::SStableFile::Sync", &s); +#endif // !(defined NDEBUG) || !defined(OS_WIN) + return s; + } + void SetIOPriority(Env::IOPriority pri) override { + base_->SetIOPriority(pri); + } + Env::IOPriority GetIOPriority() override { + return base_->GetIOPriority(); + } + bool use_direct_io() const override { + return base_->use_direct_io(); + } + Status Allocate(uint64_t offset, uint64_t len) override { + return base_->Allocate(offset, len); + } + }; + class ManifestFile : public WritableFile { + public: + ManifestFile(SpecialEnv* env, unique_ptr&& b) + : env_(env), base_(std::move(b)) {} + Status Append(const Slice& data) override { + if (env_->manifest_write_error_.load(std::memory_order_acquire)) { + return Status::IOError("simulated writer error"); + } else { + return base_->Append(data); + } + } + Status Truncate(uint64_t size) override { return base_->Truncate(size); } + Status Close() override { return base_->Close(); } + Status Flush() override { return base_->Flush(); } + Status Sync() override { + ++env_->sync_counter_; + if (env_->manifest_sync_error_.load(std::memory_order_acquire)) { + return Status::IOError("simulated sync error"); + } else { + return base_->Sync(); + } + } + uint64_t GetFileSize() override { return base_->GetFileSize(); } + + private: + SpecialEnv* env_; + unique_ptr base_; + }; + class WalFile : public WritableFile { + public: + WalFile(SpecialEnv* env, unique_ptr&& b) + : env_(env), base_(std::move(b)) { + env_->num_open_wal_file_.fetch_add(1); + } + virtual ~WalFile() { env_->num_open_wal_file_.fetch_add(-1); } + Status Append(const Slice& data) override { +#if !(defined NDEBUG) || !defined(OS_WIN) + TEST_SYNC_POINT("SpecialEnv::WalFile::Append:1"); +#endif + Status s; + if (env_->log_write_error_.load(std::memory_order_acquire)) { + s = Status::IOError("simulated writer error"); + } else { + int slowdown = + env_->log_write_slowdown_.load(std::memory_order_acquire); + if (slowdown > 0) { + env_->SleepForMicroseconds(slowdown); + } + s = base_->Append(data); + } +#if !(defined NDEBUG) || !defined(OS_WIN) + TEST_SYNC_POINT("SpecialEnv::WalFile::Append:2"); +#endif + return s; + } + Status Truncate(uint64_t size) override { return base_->Truncate(size); } + Status Close() override { +// SyncPoint is not supported in Released Windows Mode. +#if !(defined NDEBUG) || !defined(OS_WIN) + // Check preallocation size + // preallocation size is never passed to base file. + size_t preallocation_size = preallocation_block_size(); + TEST_SYNC_POINT_CALLBACK("DBTestWalFile.GetPreallocationStatus", + &preallocation_size); +#endif // !(defined NDEBUG) || !defined(OS_WIN) + + return base_->Close(); + } + Status Flush() override { return base_->Flush(); } + Status Sync() override { + ++env_->sync_counter_; + return base_->Sync(); + } + bool IsSyncThreadSafe() const override { + return env_->is_wal_sync_thread_safe_.load(); + } + + private: + SpecialEnv* env_; + unique_ptr base_; + }; + + if (non_writeable_rate_.load(std::memory_order_acquire) > 0) { + uint32_t random_number; + { + MutexLock l(&rnd_mutex_); + random_number = rnd_.Uniform(100); + } + if (random_number < non_writeable_rate_.load()) { + return Status::IOError("simulated random write error"); + } + } + + new_writable_count_++; + + if (non_writable_count_.load() > 0) { + non_writable_count_--; + return Status::IOError("simulated write error"); + } + + EnvOptions optimized = soptions; + if (strstr(f.c_str(), "MANIFEST") != nullptr || + strstr(f.c_str(), "log") != nullptr) { + optimized.use_mmap_writes = false; + optimized.use_direct_writes = false; + } + + Status s = target()->NewWritableFile(f, r, optimized); + if (s.ok()) { + if (strstr(f.c_str(), ".sst") != nullptr) { + r->reset(new SSTableFile(this, std::move(*r))); + } else if (strstr(f.c_str(), "MANIFEST") != nullptr) { + r->reset(new ManifestFile(this, std::move(*r))); + } else if (strstr(f.c_str(), "log") != nullptr) { + r->reset(new WalFile(this, std::move(*r))); + } + } + return s; + } + + Status NewRandomAccessFile(const std::string& f, + unique_ptr* r, + const EnvOptions& soptions) override { + class CountingFile : public RandomAccessFile { + public: + CountingFile(unique_ptr&& target, + anon::AtomicCounter* counter, + std::atomic* bytes_read) + : target_(std::move(target)), + counter_(counter), + bytes_read_(bytes_read) {} + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const override { + counter_->Increment(); + Status s = target_->Read(offset, n, result, scratch); + *bytes_read_ += result->size(); + return s; + } + + private: + unique_ptr target_; + anon::AtomicCounter* counter_; + std::atomic* bytes_read_; + }; + + Status s = target()->NewRandomAccessFile(f, r, soptions); + random_file_open_counter_++; + if (s.ok() && count_random_reads_) { + r->reset(new CountingFile(std::move(*r), &random_read_counter_, + &random_read_bytes_counter_)); + } + return s; + } + + Status NewSequentialFile(const std::string& f, unique_ptr* r, + const EnvOptions& soptions) override { + class CountingFile : public SequentialFile { + public: + CountingFile(unique_ptr&& target, + anon::AtomicCounter* counter) + : target_(std::move(target)), counter_(counter) {} + virtual Status Read(size_t n, Slice* result, char* scratch) override { + counter_->Increment(); + return target_->Read(n, result, scratch); + } + virtual Status Skip(uint64_t n) override { return target_->Skip(n); } + + private: + unique_ptr target_; + anon::AtomicCounter* counter_; + }; + + Status s = target()->NewSequentialFile(f, r, soptions); + if (s.ok() && count_sequential_reads_) { + r->reset(new CountingFile(std::move(*r), &sequential_read_counter_)); + } + return s; + } + + virtual void SleepForMicroseconds(int micros) override { + sleep_counter_.Increment(); + if (no_slowdown_ || time_elapse_only_sleep_) { + addon_time_.fetch_add(micros); + } + if (!no_slowdown_) { + target()->SleepForMicroseconds(micros); + } + } + + virtual Status GetCurrentTime(int64_t* unix_time) override { + Status s; + if (!time_elapse_only_sleep_) { + s = target()->GetCurrentTime(unix_time); + } + if (s.ok()) { + *unix_time += addon_time_.load(); + } + return s; + } + + virtual uint64_t NowNanos() override { + return (time_elapse_only_sleep_ ? 0 : target()->NowNanos()) + + addon_time_.load() * 1000; + } + + virtual uint64_t NowMicros() override { + return (time_elapse_only_sleep_ ? 0 : target()->NowMicros()) + + addon_time_.load(); + } + + virtual Status DeleteFile(const std::string& fname) override { + delete_count_.fetch_add(1); + return target()->DeleteFile(fname); + } + + Random rnd_; + port::Mutex rnd_mutex_; // Lock to pretect rnd_ + + // sstable Sync() calls are blocked while this pointer is non-nullptr. + std::atomic delay_sstable_sync_; + + // Drop writes on the floor while this pointer is non-nullptr. + std::atomic drop_writes_; + + // Simulate no-space errors while this pointer is non-nullptr. + std::atomic no_space_; + + // Simulate non-writable file system while this pointer is non-nullptr + std::atomic non_writable_; + + // Force sync of manifest files to fail while this pointer is non-nullptr + std::atomic manifest_sync_error_; + + // Force write to manifest files to fail while this pointer is non-nullptr + std::atomic manifest_write_error_; + + // Force write to log files to fail while this pointer is non-nullptr + std::atomic log_write_error_; + + // Slow down every log write, in micro-seconds. + std::atomic log_write_slowdown_; + + // Number of WAL files that are still open for write. + std::atomic num_open_wal_file_; + + bool count_random_reads_; + anon::AtomicCounter random_read_counter_; + std::atomic random_read_bytes_counter_; + std::atomic random_file_open_counter_; + + bool count_sequential_reads_; + anon::AtomicCounter sequential_read_counter_; + + anon::AtomicCounter sleep_counter_; + + std::atomic bytes_written_; + + std::atomic sync_counter_; + + std::atomic non_writeable_rate_; + + std::atomic new_writable_count_; + + std::atomic non_writable_count_; + + std::function* table_write_callback_; + + std::atomic addon_time_; + + std::atomic delete_count_; + + bool time_elapse_only_sleep_; + + bool no_slowdown_; + + std::atomic is_wal_sync_thread_safe_{true}; +}; + +class MockTimeEnv : public EnvWrapper { + public: + explicit MockTimeEnv(Env* base) : EnvWrapper(base) {} + + virtual Status GetCurrentTime(int64_t* time) override { + assert(time != nullptr); + assert(current_time_ <= + static_cast(std::numeric_limits::max())); + *time = static_cast(current_time_); + return Status::OK(); + } + + virtual uint64_t NowMicros() override { + assert(current_time_ <= std::numeric_limits::max() / 1000000); + return current_time_ * 1000000; + } + + virtual uint64_t NowNanos() override { + assert(current_time_ <= std::numeric_limits::max() / 1000000000); + return current_time_ * 1000000000; + } + + void set_current_time(uint64_t time) { + assert(time >= current_time_); + current_time_ = time; + } + + private: + uint64_t current_time_ = 0; +}; + +#ifndef ROCKSDB_LITE +class OnFileDeletionListener : public EventListener { + public: + OnFileDeletionListener() : matched_count_(0), expected_file_name_("") {} + + void SetExpectedFileName(const std::string file_name) { + expected_file_name_ = file_name; + } + + void VerifyMatchedCount(size_t expected_value) { + ASSERT_EQ(matched_count_, expected_value); + } + + void OnTableFileDeleted(const TableFileDeletionInfo& info) override { + if (expected_file_name_ != "") { + ASSERT_EQ(expected_file_name_, info.file_path); + expected_file_name_ = ""; + matched_count_++; + } + } + + private: + size_t matched_count_; + std::string expected_file_name_; +}; +#endif + +// A test merge operator mimics put but also fails if one of merge operands is +// "corrupted". +class TestPutOperator : public MergeOperator { + public: + virtual bool FullMergeV2(const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const override { + if (merge_in.existing_value != nullptr && + *(merge_in.existing_value) == "corrupted") { + return false; + } + for (auto value : merge_in.operand_list) { + if (value == "corrupted") { + return false; + } + } + merge_out->existing_operand = merge_in.operand_list.back(); + return true; + } + + virtual const char* Name() const override { return "TestPutOperator"; } +}; + +class DBTestBase : public testing::Test { + public: + // Sequence of option configurations to try + enum OptionConfig : int { + kDefault = 0, + kBlockBasedTableWithPrefixHashIndex = 1, + kBlockBasedTableWithWholeKeyHashIndex = 2, + kPlainTableFirstBytePrefix = 3, + kPlainTableCappedPrefix = 4, + kPlainTableCappedPrefixNonMmap = 5, + kPlainTableAllBytesPrefix = 6, + kVectorRep = 7, + kHashLinkList = 8, + kHashCuckoo = 9, + kMergePut = 10, + kFilter = 11, + kFullFilterWithNewTableReaderForCompactions = 12, + kUncompressed = 13, + kNumLevel_3 = 14, + kDBLogDir = 15, + kWalDirAndMmapReads = 16, + kManifestFileSize = 17, + kPerfOptions = 18, + kHashSkipList = 19, + kUniversalCompaction = 20, + kUniversalCompactionMultiLevel = 21, + kCompressedBlockCache = 22, + kInfiniteMaxOpenFiles = 23, + kxxHashChecksum = 24, + kFIFOCompaction = 25, + kOptimizeFiltersForHits = 26, + kRowCache = 27, + kRecycleLogFiles = 28, + kConcurrentSkipList = 29, + kPipelinedWrite = 30, + kConcurrentWALWrites = 31, + kEnd = 32, + kDirectIO = 33, + kLevelSubcompactions = 34, + kUniversalSubcompactions = 35, + kBlockBasedTableWithIndexRestartInterval = 36, + kBlockBasedTableWithPartitionedIndex = 37, + kPartitionedFilterWithNewTableReaderForCompactions = 38, + }; + + public: + std::string dbname_; + std::string alternative_wal_dir_; + std::string alternative_db_log_dir_; + MockEnv* mem_env_; + Env* encrypted_env_; + SpecialEnv* env_; + DB* db_; + std::vector handles_; + + int option_config_; + Options last_options_; + + // Skip some options, as they may not be applicable to a specific test. + // To add more skip constants, use values 4, 8, 16, etc. + enum OptionSkip { + kNoSkip = 0, + kSkipDeletesFilterFirst = 1, + kSkipUniversalCompaction = 2, + kSkipMergePut = 4, + kSkipPlainTable = 8, + kSkipHashIndex = 16, + kSkipNoSeekToLast = 32, + kSkipHashCuckoo = 64, + kSkipFIFOCompaction = 128, + kSkipMmapReads = 256, + }; + + explicit DBTestBase(const std::string path); + + ~DBTestBase(); + + static std::string RandomString(Random* rnd, int len) { + std::string r; + test::RandomString(rnd, len, &r); + return r; + } + + static std::string Key(int i) { + char buf[100]; + snprintf(buf, sizeof(buf), "key%06d", i); + return std::string(buf); + } + + static bool ShouldSkipOptions(int option_config, int skip_mask = kNoSkip); + + // Switch to a fresh database with the next option configuration to + // test. Return false if there are no more configurations to test. + bool ChangeOptions(int skip_mask = kNoSkip); + + // Switch between different compaction styles. + bool ChangeCompactOptions(); + + // Switch between different WAL-realted options. + bool ChangeWalOptions(); + + // Switch between different filter policy + // Jump from kDefault to kFilter to kFullFilter + bool ChangeFilterOptions(); + + // Return the current option configuration. + Options CurrentOptions(const anon::OptionsOverride& options_override = + anon::OptionsOverride()) const; + + Options CurrentOptions(const Options& default_options, + const anon::OptionsOverride& options_override = + anon::OptionsOverride()) const; + + static Options GetDefaultOptions(); + + Options GetOptions(int option_config, + const Options& default_options = GetDefaultOptions(), + const anon::OptionsOverride& options_override = + anon::OptionsOverride()) const; + + DBImpl* dbfull() { return reinterpret_cast(db_); } + + void CreateColumnFamilies(const std::vector& cfs, + const Options& options); + + void CreateAndReopenWithCF(const std::vector& cfs, + const Options& options); + + void ReopenWithColumnFamilies(const std::vector& cfs, + const std::vector& options); + + void ReopenWithColumnFamilies(const std::vector& cfs, + const Options& options); + + Status TryReopenWithColumnFamilies(const std::vector& cfs, + const std::vector& options); + + Status TryReopenWithColumnFamilies(const std::vector& cfs, + const Options& options); + + void Reopen(const Options& options); + + void Close(); + + void DestroyAndReopen(const Options& options); + + void Destroy(const Options& options); + + Status ReadOnlyReopen(const Options& options); + + Status TryReopen(const Options& options); + + bool IsDirectIOSupported(); + + bool IsMemoryMappedAccessSupported() const; + + Status Flush(int cf = 0); + + Status Put(const Slice& k, const Slice& v, WriteOptions wo = WriteOptions()); + + Status Put(int cf, const Slice& k, const Slice& v, + WriteOptions wo = WriteOptions()); + + Status Merge(const Slice& k, const Slice& v, + WriteOptions wo = WriteOptions()); + + Status Merge(int cf, const Slice& k, const Slice& v, + WriteOptions wo = WriteOptions()); + + Status Delete(const std::string& k); + + Status Delete(int cf, const std::string& k); + + Status SingleDelete(const std::string& k); + + Status SingleDelete(int cf, const std::string& k); + + std::string Get(const std::string& k, const Snapshot* snapshot = nullptr); + + std::string Get(int cf, const std::string& k, + const Snapshot* snapshot = nullptr); + + Status Get(const std::string& k, PinnableSlice* v); + + uint64_t GetNumSnapshots(); + + uint64_t GetTimeOldestSnapshots(); + + // Return a string that contains all key,value pairs in order, + // formatted like "(k1->v1)(k2->v2)". + std::string Contents(int cf = 0); + + std::string AllEntriesFor(const Slice& user_key, int cf = 0); + +#ifndef ROCKSDB_LITE + int NumSortedRuns(int cf = 0); + + uint64_t TotalSize(int cf = 0); + + uint64_t SizeAtLevel(int level); + + size_t TotalLiveFiles(int cf = 0); + + size_t CountLiveFiles(); +#endif // ROCKSDB_LITE + + int NumTableFilesAtLevel(int level, int cf = 0); + + double CompressionRatioAtLevel(int level, int cf = 0); + + int TotalTableFiles(int cf = 0, int levels = -1); + + // Return spread of files per level + std::string FilesPerLevel(int cf = 0); + + size_t CountFiles(); + + uint64_t Size(const Slice& start, const Slice& limit, int cf = 0); + + void Compact(int cf, const Slice& start, const Slice& limit, + uint32_t target_path_id); + + void Compact(int cf, const Slice& start, const Slice& limit); + + void Compact(const Slice& start, const Slice& limit); + + // Do n memtable compactions, each of which produces an sstable + // covering the range [small,large]. + void MakeTables(int n, const std::string& small, const std::string& large, + int cf = 0); + + // Prevent pushing of new sstables into deeper levels by adding + // tables that cover a specified range to all levels. + void FillLevels(const std::string& smallest, const std::string& largest, + int cf); + + void MoveFilesToLevel(int level, int cf = 0); + + void DumpFileCounts(const char* label); + + std::string DumpSSTableList(); + + void GetSstFiles(std::string path, std::vector* files); + + int GetSstFileCount(std::string path); + + // this will generate non-overlapping files since it keeps increasing key_idx + void GenerateNewFile(Random* rnd, int* key_idx, bool nowait = false); + + void GenerateNewFile(int fd, Random* rnd, int* key_idx, bool nowait = false); + + static const int kNumKeysByGenerateNewRandomFile; + static const int KNumKeysByGenerateNewFile = 100; + + void GenerateNewRandomFile(Random* rnd, bool nowait = false); + + std::string IterStatus(Iterator* iter); + + Options OptionsForLogIterTest(); + + std::string DummyString(size_t len, char c = 'a'); + + void VerifyIterLast(std::string expected_key, int cf = 0); + + // Used to test InplaceUpdate + + // If previous value is nullptr or delta is > than previous value, + // sets newValue with delta + // If previous value is not empty, + // updates previous value with 'b' string of previous value size - 1. + static UpdateStatus updateInPlaceSmallerSize(char* prevValue, + uint32_t* prevSize, Slice delta, + std::string* newValue); + + static UpdateStatus updateInPlaceSmallerVarintSize(char* prevValue, + uint32_t* prevSize, + Slice delta, + std::string* newValue); + + static UpdateStatus updateInPlaceLargerSize(char* prevValue, + uint32_t* prevSize, Slice delta, + std::string* newValue); + + static UpdateStatus updateInPlaceNoAction(char* prevValue, uint32_t* prevSize, + Slice delta, std::string* newValue); + + // Utility method to test InplaceUpdate + void validateNumberOfEntries(int numValues, int cf = 0); + + void CopyFile(const std::string& source, const std::string& destination, + uint64_t size = 0); + + std::unordered_map GetAllSSTFiles( + uint64_t* total_size = nullptr); + + std::vector ListTableFiles(Env* env, const std::string& path); + + void VerifyDBFromMap( + std::map true_data, + size_t* total_reads_res = nullptr, bool tailing_iter = false, + std::map status = std::map()); + + void VerifyDBInternal( + std::vector> true_data); + +#ifndef ROCKSDB_LITE + uint64_t GetNumberOfSstFilesForColumnFamily(DB* db, + std::string column_family_name); +#endif // ROCKSDB_LITE + + uint64_t TestGetTickerCount(const Options& options, Tickers ticker_type) { + return options.statistics->getTickerCount(ticker_type); + } +}; + +} // namespace rocksdb diff --git a/db/db_universal_compaction_test.cc b/db/db_universal_compaction_test.cc new file mode 100644 index 00000000000..58fda80d549 --- /dev/null +++ b/db/db_universal_compaction_test.cc @@ -0,0 +1,1589 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_test_util.h" +#include "port/stack_trace.h" +#if !defined(ROCKSDB_LITE) +#include "util/sync_point.h" + +namespace rocksdb { + +static std::string CompressibleString(Random* rnd, int len) { + std::string r; + test::CompressibleString(rnd, 0.8, len, &r); + return r; +} + +class DBTestUniversalCompactionBase + : public DBTestBase, + public ::testing::WithParamInterface> { + public: + explicit DBTestUniversalCompactionBase( + const std::string& path) : DBTestBase(path) {} + virtual void SetUp() override { + num_levels_ = std::get<0>(GetParam()); + exclusive_manual_compaction_ = std::get<1>(GetParam()); + } + int num_levels_; + bool exclusive_manual_compaction_; +}; + +class DBTestUniversalCompaction : public DBTestUniversalCompactionBase { + public: + DBTestUniversalCompaction() : + DBTestUniversalCompactionBase("/db_universal_compaction_test") {} +}; + +namespace { +void VerifyCompactionResult( + const ColumnFamilyMetaData& cf_meta, + const std::set& overlapping_file_numbers) { +#ifndef NDEBUG + for (auto& level : cf_meta.levels) { + for (auto& file : level.files) { + assert(overlapping_file_numbers.find(file.name) == + overlapping_file_numbers.end()); + } + } +#endif +} + +class KeepFilter : public CompactionFilter { + public: + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, bool* value_changed) const + override { + return false; + } + + virtual const char* Name() const override { return "KeepFilter"; } +}; + +class KeepFilterFactory : public CompactionFilterFactory { + public: + explicit KeepFilterFactory(bool check_context = false) + : check_context_(check_context) {} + + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + if (check_context_) { + EXPECT_EQ(expect_full_compaction_.load(), context.is_full_compaction); + EXPECT_EQ(expect_manual_compaction_.load(), context.is_manual_compaction); + } + return std::unique_ptr(new KeepFilter()); + } + + virtual const char* Name() const override { return "KeepFilterFactory"; } + bool check_context_; + std::atomic_bool expect_full_compaction_; + std::atomic_bool expect_manual_compaction_; +}; + +class DelayFilter : public CompactionFilter { + public: + explicit DelayFilter(DBTestBase* d) : db_test(d) {} + virtual bool Filter(int level, const Slice& key, const Slice& value, + std::string* new_value, + bool* value_changed) const override { + db_test->env_->addon_time_.fetch_add(1000); + return true; + } + + virtual const char* Name() const override { return "DelayFilter"; } + + private: + DBTestBase* db_test; +}; + +class DelayFilterFactory : public CompactionFilterFactory { + public: + explicit DelayFilterFactory(DBTestBase* d) : db_test(d) {} + virtual std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override { + return std::unique_ptr(new DelayFilter(db_test)); + } + + virtual const char* Name() const override { return "DelayFilterFactory"; } + + private: + DBTestBase* db_test; +}; +} // namespace + +// Make sure we don't trigger a problem if the trigger condtion is given +// to be 0, which is invalid. +TEST_P(DBTestUniversalCompaction, UniversalCompactionSingleSortedRun) { + Options options = CurrentOptions(); + + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = num_levels_; + // Config universal compaction to always compact to one single sorted run. + options.level0_file_num_compaction_trigger = 0; + options.compaction_options_universal.size_ratio = 10; + options.compaction_options_universal.min_merge_width = 2; + options.compaction_options_universal.max_size_amplification_percent = 0; + + options.write_buffer_size = 105 << 10; // 105KB + options.arena_block_size = 4 << 10; + options.target_file_size_base = 32 << 10; // 32KB + // trigger compaction if there are >= 4 files + KeepFilterFactory* filter = new KeepFilterFactory(true); + filter->expect_manual_compaction_.store(false); + options.compaction_filter_factory.reset(filter); + + DestroyAndReopen(options); + ASSERT_EQ(1, db_->GetOptions().level0_file_num_compaction_trigger); + + Random rnd(301); + int key_idx = 0; + + filter->expect_full_compaction_.store(true); + + for (int num = 0; num < 16; num++) { + // Write 100KB file. And immediately it should be compacted to one file. + GenerateNewFile(&rnd, &key_idx); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(NumSortedRuns(0), 1); + } + ASSERT_OK(Put(Key(key_idx), "")); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(NumSortedRuns(0), 1); +} + +TEST_P(DBTestUniversalCompaction, OptimizeFiltersForHits) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.compaction_options_universal.size_ratio = 5; + options.num_levels = num_levels_; + options.write_buffer_size = 105 << 10; // 105KB + options.arena_block_size = 4 << 10; + options.target_file_size_base = 32 << 10; // 32KB + // trigger compaction if there are >= 4 files + options.level0_file_num_compaction_trigger = 4; + BlockBasedTableOptions bbto; + bbto.cache_index_and_filter_blocks = true; + bbto.filter_policy.reset(NewBloomFilterPolicy(10, false)); + bbto.whole_key_filtering = true; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + options.optimize_filters_for_hits = true; + options.statistics = rocksdb::CreateDBStatistics(); + options.memtable_factory.reset(new SpecialSkipListFactory(3)); + + DestroyAndReopen(options); + + // block compaction from happening + env_->SetBackgroundThreads(1, Env::LOW); + test::SleepingBackgroundTask sleeping_task_low; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::LOW); + + for (int num = 0; num < options.level0_file_num_compaction_trigger; num++) { + Put(Key(num * 10), "val"); + if (num) { + dbfull()->TEST_WaitForFlushMemTable(); + } + Put(Key(30 + num * 10), "val"); + Put(Key(60 + num * 10), "val"); + } + Put("", ""); + dbfull()->TEST_WaitForFlushMemTable(); + + // Query set of non existing keys + for (int i = 5; i < 90; i += 10) { + ASSERT_EQ(Get(Key(i)), "NOT_FOUND"); + } + + // Make sure bloom filter is used at least once. + ASSERT_GT(TestGetTickerCount(options, BLOOM_FILTER_USEFUL), 0); + auto prev_counter = TestGetTickerCount(options, BLOOM_FILTER_USEFUL); + + // Make sure bloom filter is used for all but the last L0 file when looking + // up a non-existent key that's in the range of all L0 files. + ASSERT_EQ(Get(Key(35)), "NOT_FOUND"); + ASSERT_EQ(prev_counter + NumTableFilesAtLevel(0) - 1, + TestGetTickerCount(options, BLOOM_FILTER_USEFUL)); + prev_counter = TestGetTickerCount(options, BLOOM_FILTER_USEFUL); + + // Unblock compaction and wait it for happening. + sleeping_task_low.WakeUp(); + dbfull()->TEST_WaitForCompact(); + + // The same queries will not trigger bloom filter + for (int i = 5; i < 90; i += 10) { + ASSERT_EQ(Get(Key(i)), "NOT_FOUND"); + } + ASSERT_EQ(prev_counter, TestGetTickerCount(options, BLOOM_FILTER_USEFUL)); +} + +// TODO(kailiu) The tests on UniversalCompaction has some issues: +// 1. A lot of magic numbers ("11" or "12"). +// 2. Made assumption on the memtable flush conditions, which may change from +// time to time. +TEST_P(DBTestUniversalCompaction, UniversalCompactionTrigger) { + Options options; + options.compaction_style = kCompactionStyleUniversal; + options.compaction_options_universal.size_ratio = 5; + options.num_levels = num_levels_; + options.write_buffer_size = 105 << 10; // 105KB + options.arena_block_size = 4 << 10; + options.target_file_size_base = 32 << 10; // 32KB + // trigger compaction if there are >= 4 files + options.level0_file_num_compaction_trigger = 4; + KeepFilterFactory* filter = new KeepFilterFactory(true); + filter->expect_manual_compaction_.store(false); + options.compaction_filter_factory.reset(filter); + + options = CurrentOptions(options); + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBTestWritableFile.GetPreallocationStatus", [&](void* arg) { + ASSERT_TRUE(arg != nullptr); + size_t preallocation_size = *(static_cast(arg)); + if (num_levels_ > 3) { + ASSERT_LE(preallocation_size, options.target_file_size_base * 1.1); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Random rnd(301); + int key_idx = 0; + + filter->expect_full_compaction_.store(true); + // Stage 1: + // Generate a set of files at level 0, but don't trigger level-0 + // compaction. + for (int num = 0; num < options.level0_file_num_compaction_trigger - 1; + num++) { + // Write 100KB + GenerateNewFile(1, &rnd, &key_idx); + } + + // Generate one more file at level-0, which should trigger level-0 + // compaction. + GenerateNewFile(1, &rnd, &key_idx); + // Suppose each file flushed from mem table has size 1. Now we compact + // (level0_file_num_compaction_trigger+1)=4 files and should have a big + // file of size 4. + ASSERT_EQ(NumSortedRuns(1), 1); + + // Stage 2: + // Now we have one file at level 0, with size 4. We also have some data in + // mem table. Let's continue generating new files at level 0, but don't + // trigger level-0 compaction. + // First, clean up memtable before inserting new data. This will generate + // a level-0 file, with size around 0.4 (according to previously written + // data amount). + filter->expect_full_compaction_.store(false); + ASSERT_OK(Flush(1)); + for (int num = 0; num < options.level0_file_num_compaction_trigger - 3; + num++) { + GenerateNewFile(1, &rnd, &key_idx); + ASSERT_EQ(NumSortedRuns(1), num + 3); + } + + // Generate one more file at level-0, which should trigger level-0 + // compaction. + GenerateNewFile(1, &rnd, &key_idx); + // Before compaction, we have 4 files at level 0, with size 4, 0.4, 1, 1. + // After compaction, we should have 2 files, with size 4, 2.4. + ASSERT_EQ(NumSortedRuns(1), 2); + + // Stage 3: + // Now we have 2 files at level 0, with size 4 and 2.4. Continue + // generating new files at level 0. + for (int num = 0; num < options.level0_file_num_compaction_trigger - 3; + num++) { + GenerateNewFile(1, &rnd, &key_idx); + ASSERT_EQ(NumSortedRuns(1), num + 3); + } + + // Generate one more file at level-0, which should trigger level-0 + // compaction. + GenerateNewFile(1, &rnd, &key_idx); + // Before compaction, we have 4 files at level 0, with size 4, 2.4, 1, 1. + // After compaction, we should have 3 files, with size 4, 2.4, 2. + ASSERT_EQ(NumSortedRuns(1), 3); + + // Stage 4: + // Now we have 3 files at level 0, with size 4, 2.4, 2. Let's generate a + // new file of size 1. + GenerateNewFile(1, &rnd, &key_idx); + dbfull()->TEST_WaitForCompact(); + // Level-0 compaction is triggered, but no file will be picked up. + ASSERT_EQ(NumSortedRuns(1), 4); + + // Stage 5: + // Now we have 4 files at level 0, with size 4, 2.4, 2, 1. Let's generate + // a new file of size 1. + filter->expect_full_compaction_.store(true); + GenerateNewFile(1, &rnd, &key_idx); + dbfull()->TEST_WaitForCompact(); + // All files at level 0 will be compacted into a single one. + ASSERT_EQ(NumSortedRuns(1), 1); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_P(DBTestUniversalCompaction, UniversalCompactionSizeAmplification) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = num_levels_; + options.write_buffer_size = 100 << 10; // 100KB + options.target_file_size_base = 32 << 10; // 32KB + options.level0_file_num_compaction_trigger = 3; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Trigger compaction if size amplification exceeds 110% + options.compaction_options_universal.max_size_amplification_percent = 110; + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + Random rnd(301); + int key_idx = 0; + + // Generate two files in Level 0. Both files are approx the same size. + for (int num = 0; num < options.level0_file_num_compaction_trigger - 1; + num++) { + // Write 110KB (11 values, each 10K) + for (int i = 0; i < 11; i++) { + ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 10000))); + key_idx++; + } + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + ASSERT_EQ(NumSortedRuns(1), num + 1); + } + ASSERT_EQ(NumSortedRuns(1), 2); + + // Flush whatever is remaining in memtable. This is typically + // small, which should not trigger size ratio based compaction + // but will instead trigger size amplification. + ASSERT_OK(Flush(1)); + + dbfull()->TEST_WaitForCompact(); + + // Verify that size amplification did occur + ASSERT_EQ(NumSortedRuns(1), 1); +} + +TEST_P(DBTestUniversalCompaction, CompactFilesOnUniversalCompaction) { + const int kTestKeySize = 16; + const int kTestValueSize = 984; + const int kEntrySize = kTestKeySize + kTestValueSize; + const int kEntriesPerBuffer = 10; + + ChangeCompactOptions(); + Options options; + options.create_if_missing = true; + options.compaction_style = kCompactionStyleLevel; + options.num_levels = 1; + options.target_file_size_base = options.write_buffer_size; + options.compression = kNoCompression; + options = CurrentOptions(options); + options.write_buffer_size = kEntrySize * kEntriesPerBuffer; + CreateAndReopenWithCF({"pikachu"}, options); + ASSERT_EQ(options.compaction_style, kCompactionStyleUniversal); + Random rnd(301); + for (int key = 1024 * kEntriesPerBuffer; key >= 0; --key) { + ASSERT_OK(Put(1, ToString(key), RandomString(&rnd, kTestValueSize))); + } + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + dbfull()->TEST_WaitForCompact(); + ColumnFamilyMetaData cf_meta; + dbfull()->GetColumnFamilyMetaData(handles_[1], &cf_meta); + std::vector compaction_input_file_names; + for (auto file : cf_meta.levels[0].files) { + if (rnd.OneIn(2)) { + compaction_input_file_names.push_back(file.name); + } + } + + if (compaction_input_file_names.size() == 0) { + compaction_input_file_names.push_back( + cf_meta.levels[0].files[0].name); + } + + // expect fail since universal compaction only allow L0 output + ASSERT_FALSE(dbfull() + ->CompactFiles(CompactionOptions(), handles_[1], + compaction_input_file_names, 1) + .ok()); + + // expect ok and verify the compacted files no longer exist. + ASSERT_OK(dbfull()->CompactFiles( + CompactionOptions(), handles_[1], + compaction_input_file_names, 0)); + + dbfull()->GetColumnFamilyMetaData(handles_[1], &cf_meta); + VerifyCompactionResult( + cf_meta, + std::set(compaction_input_file_names.begin(), + compaction_input_file_names.end())); + + compaction_input_file_names.clear(); + + // Pick the first and the last file, expect everything is + // compacted into one single file. + compaction_input_file_names.push_back( + cf_meta.levels[0].files[0].name); + compaction_input_file_names.push_back( + cf_meta.levels[0].files[ + cf_meta.levels[0].files.size() - 1].name); + ASSERT_OK(dbfull()->CompactFiles( + CompactionOptions(), handles_[1], + compaction_input_file_names, 0)); + + dbfull()->GetColumnFamilyMetaData(handles_[1], &cf_meta); + ASSERT_EQ(cf_meta.levels[0].files.size(), 1U); +} + +TEST_P(DBTestUniversalCompaction, UniversalCompactionTargetLevel) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.write_buffer_size = 100 << 10; // 100KB + options.num_levels = 7; + options.disable_auto_compactions = true; + DestroyAndReopen(options); + + // Generate 3 overlapping files + Random rnd(301); + for (int i = 0; i < 210; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 100))); + } + ASSERT_OK(Flush()); + + for (int i = 200; i < 300; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 100))); + } + ASSERT_OK(Flush()); + + for (int i = 250; i < 260; i++) { + ASSERT_OK(Put(Key(i), RandomString(&rnd, 100))); + } + ASSERT_OK(Flush()); + + ASSERT_EQ("3", FilesPerLevel(0)); + // Compact all files into 1 file and put it in L4 + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 4; + compact_options.exclusive_manual_compaction = exclusive_manual_compaction_; + db_->CompactRange(compact_options, nullptr, nullptr); + ASSERT_EQ("0,0,0,0,1", FilesPerLevel(0)); +} + + +class DBTestUniversalCompactionMultiLevels + : public DBTestUniversalCompactionBase { + public: + DBTestUniversalCompactionMultiLevels() : + DBTestUniversalCompactionBase( + "/db_universal_compaction_multi_levels_test") {} +}; + +TEST_P(DBTestUniversalCompactionMultiLevels, UniversalCompactionMultiLevels) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = num_levels_; + options.write_buffer_size = 100 << 10; // 100KB + options.level0_file_num_compaction_trigger = 8; + options.max_background_compactions = 3; + options.target_file_size_base = 32 * 1024; + CreateAndReopenWithCF({"pikachu"}, options); + + // Trigger compaction if size amplification exceeds 110% + options.compaction_options_universal.max_size_amplification_percent = 110; + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + Random rnd(301); + int num_keys = 100000; + for (int i = 0; i < num_keys * 2; i++) { + ASSERT_OK(Put(1, Key(i % num_keys), Key(i))); + } + + dbfull()->TEST_WaitForCompact(); + + for (int i = num_keys; i < num_keys * 2; i++) { + ASSERT_EQ(Get(1, Key(i % num_keys)), Key(i)); + } +} +// Tests universal compaction with trivial move enabled +TEST_P(DBTestUniversalCompactionMultiLevels, UniversalCompactionTrivialMove) { + int32_t trivial_move = 0; + int32_t non_trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", [&](void* arg) { + non_trivial_move++; + ASSERT_TRUE(arg != nullptr); + int output_level = *(static_cast(arg)); + ASSERT_EQ(output_level, 0); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.compaction_options_universal.allow_trivial_move = true; + options.num_levels = 3; + options.write_buffer_size = 100 << 10; // 100KB + options.level0_file_num_compaction_trigger = 3; + options.max_background_compactions = 2; + options.target_file_size_base = 32 * 1024; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Trigger compaction if size amplification exceeds 110% + options.compaction_options_universal.max_size_amplification_percent = 110; + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + Random rnd(301); + int num_keys = 150000; + for (int i = 0; i < num_keys; i++) { + ASSERT_OK(Put(1, Key(i), Key(i))); + } + std::vector values; + + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + + ASSERT_GT(trivial_move, 0); + ASSERT_GT(non_trivial_move, 0); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +INSTANTIATE_TEST_CASE_P(DBTestUniversalCompactionMultiLevels, + DBTestUniversalCompactionMultiLevels, + ::testing::Combine(::testing::Values(3, 20), + ::testing::Bool())); + +class DBTestUniversalCompactionParallel : + public DBTestUniversalCompactionBase { + public: + DBTestUniversalCompactionParallel() : + DBTestUniversalCompactionBase( + "/db_universal_compaction_prallel_test") {} +}; + +TEST_P(DBTestUniversalCompactionParallel, UniversalCompactionParallel) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = num_levels_; + options.write_buffer_size = 1 << 10; // 1KB + options.level0_file_num_compaction_trigger = 3; + options.max_background_compactions = 3; + options.max_background_flushes = 3; + options.target_file_size_base = 1 * 1024; + options.compaction_options_universal.max_size_amplification_percent = 110; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Delay every compaction so multiple compactions will happen. + std::atomic num_compactions_running(0); + std::atomic has_parallel(false); + rocksdb::SyncPoint::GetInstance()->SetCallBack("CompactionJob::Run():Start", + [&](void* arg) { + if (num_compactions_running.fetch_add(1) > 0) { + has_parallel.store(true); + return; + } + for (int nwait = 0; nwait < 20000; nwait++) { + if (has_parallel.load() || num_compactions_running.load() > 1) { + has_parallel.store(true); + break; + } + env_->SleepForMicroseconds(1000); + } + }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "CompactionJob::Run():End", + [&](void* arg) { num_compactions_running.fetch_add(-1); }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + Random rnd(301); + int num_keys = 30000; + for (int i = 0; i < num_keys * 2; i++) { + ASSERT_OK(Put(1, Key(i % num_keys), Key(i))); + } + dbfull()->TEST_WaitForCompact(); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + ASSERT_EQ(num_compactions_running.load(), 0); + ASSERT_TRUE(has_parallel.load()); + + for (int i = num_keys; i < num_keys * 2; i++) { + ASSERT_EQ(Get(1, Key(i % num_keys)), Key(i)); + } + + // Reopen and check. + ReopenWithColumnFamilies({"default", "pikachu"}, options); + for (int i = num_keys; i < num_keys * 2; i++) { + ASSERT_EQ(Get(1, Key(i % num_keys)), Key(i)); + } +} + +TEST_P(DBTestUniversalCompactionParallel, PickByFileNumberBug) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = num_levels_; + options.write_buffer_size = 1 * 1024; // 1KB + options.level0_file_num_compaction_trigger = 7; + options.max_background_compactions = 2; + options.target_file_size_base = 1024 * 1024; // 1MB + + // Disable size amplifiction compaction + options.compaction_options_universal.max_size_amplification_percent = + UINT_MAX; + DestroyAndReopen(options); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBTestUniversalCompactionParallel::PickByFileNumberBug:0", + "BackgroundCallCompaction:0"}, + {"UniversalCompactionPicker::PickCompaction:Return", + "DBTestUniversalCompactionParallel::PickByFileNumberBug:1"}, + {"DBTestUniversalCompactionParallel::PickByFileNumberBug:2", + "CompactionJob::Run():Start"}}); + + int total_picked_compactions = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "UniversalCompactionPicker::PickCompaction:Return", [&](void* arg) { + if (arg) { + total_picked_compactions++; + } + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Write 7 files to trigger compaction + int key_idx = 1; + for (int i = 1; i <= 70; i++) { + std::string k = Key(key_idx++); + ASSERT_OK(Put(k, k)); + if (i % 10 == 0) { + ASSERT_OK(Flush()); + } + } + + // Wait for the 1st background compaction process to start + TEST_SYNC_POINT("DBTestUniversalCompactionParallel::PickByFileNumberBug:0"); + TEST_SYNC_POINT("DBTestUniversalCompactionParallel::PickByFileNumberBug:1"); + rocksdb::SyncPoint::GetInstance()->ClearTrace(); + + // Write 3 files while 1st compaction is held + // These 3 files have different sizes to avoid compacting based on size_ratio + int num_keys = 1000; + for (int i = 0; i < 3; i++) { + for (int j = 1; j <= num_keys; j++) { + std::string k = Key(key_idx++); + ASSERT_OK(Put(k, k)); + } + ASSERT_OK(Flush()); + num_keys -= 100; + } + + // Hold the 1st compaction from finishing + TEST_SYNC_POINT("DBTestUniversalCompactionParallel::PickByFileNumberBug:2"); + dbfull()->TEST_WaitForCompact(); + + // There should only be one picked compaction as the score drops below one + // after the first one is picked. + EXPECT_EQ(total_picked_compactions, 1); + EXPECT_EQ(TotalTableFiles(), 4); + + // Stop SyncPoint and destroy the DB and reopen it again + rocksdb::SyncPoint::GetInstance()->ClearTrace(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + key_idx = 1; + total_picked_compactions = 0; + DestroyAndReopen(options); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Write 7 files to trigger compaction + for (int i = 1; i <= 70; i++) { + std::string k = Key(key_idx++); + ASSERT_OK(Put(k, k)); + if (i % 10 == 0) { + ASSERT_OK(Flush()); + } + } + + // Wait for the 1st background compaction process to start + TEST_SYNC_POINT("DBTestUniversalCompactionParallel::PickByFileNumberBug:0"); + TEST_SYNC_POINT("DBTestUniversalCompactionParallel::PickByFileNumberBug:1"); + rocksdb::SyncPoint::GetInstance()->ClearTrace(); + + // Write 8 files while 1st compaction is held + // These 8 files have different sizes to avoid compacting based on size_ratio + num_keys = 1000; + for (int i = 0; i < 8; i++) { + for (int j = 1; j <= num_keys; j++) { + std::string k = Key(key_idx++); + ASSERT_OK(Put(k, k)); + } + ASSERT_OK(Flush()); + num_keys -= 100; + } + + // Wait for the 2nd background compaction process to start + TEST_SYNC_POINT("DBTestUniversalCompactionParallel::PickByFileNumberBug:0"); + TEST_SYNC_POINT("DBTestUniversalCompactionParallel::PickByFileNumberBug:1"); + + // Hold the 1st and 2nd compaction from finishing + TEST_SYNC_POINT("DBTestUniversalCompactionParallel::PickByFileNumberBug:2"); + dbfull()->TEST_WaitForCompact(); + + // This time we will trigger a compaction because of size ratio and + // another compaction because of number of files that are not compacted + // greater than 7 + EXPECT_GE(total_picked_compactions, 2); +} + +INSTANTIATE_TEST_CASE_P(DBTestUniversalCompactionParallel, + DBTestUniversalCompactionParallel, + ::testing::Combine(::testing::Values(1, 10), + ::testing::Values(false))); + +TEST_P(DBTestUniversalCompaction, UniversalCompactionOptions) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.write_buffer_size = 105 << 10; // 105KB + options.arena_block_size = 4 << 10; // 4KB + options.target_file_size_base = 32 << 10; // 32KB + options.level0_file_num_compaction_trigger = 4; + options.num_levels = num_levels_; + options.compaction_options_universal.compression_size_percent = -1; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + Random rnd(301); + int key_idx = 0; + + for (int num = 0; num < options.level0_file_num_compaction_trigger; num++) { + // Write 100KB (100 values, each 1K) + for (int i = 0; i < 100; i++) { + ASSERT_OK(Put(1, Key(key_idx), RandomString(&rnd, 990))); + key_idx++; + } + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + + if (num < options.level0_file_num_compaction_trigger - 1) { + ASSERT_EQ(NumSortedRuns(1), num + 1); + } + } + + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(NumSortedRuns(1), 1); +} + +TEST_P(DBTestUniversalCompaction, UniversalCompactionStopStyleSimilarSize) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.write_buffer_size = 105 << 10; // 105KB + options.arena_block_size = 4 << 10; // 4KB + options.target_file_size_base = 32 << 10; // 32KB + // trigger compaction if there are >= 4 files + options.level0_file_num_compaction_trigger = 4; + options.compaction_options_universal.size_ratio = 10; + options.compaction_options_universal.stop_style = + kCompactionStopStyleSimilarSize; + options.num_levels = num_levels_; + DestroyAndReopen(options); + + Random rnd(301); + int key_idx = 0; + + // Stage 1: + // Generate a set of files at level 0, but don't trigger level-0 + // compaction. + for (int num = 0; num < options.level0_file_num_compaction_trigger - 1; + num++) { + // Write 100KB (100 values, each 1K) + for (int i = 0; i < 100; i++) { + ASSERT_OK(Put(Key(key_idx), RandomString(&rnd, 990))); + key_idx++; + } + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(NumSortedRuns(), num + 1); + } + + // Generate one more file at level-0, which should trigger level-0 + // compaction. + for (int i = 0; i < 100; i++) { + ASSERT_OK(Put(Key(key_idx), RandomString(&rnd, 990))); + key_idx++; + } + dbfull()->TEST_WaitForCompact(); + // Suppose each file flushed from mem table has size 1. Now we compact + // (level0_file_num_compaction_trigger+1)=4 files and should have a big + // file of size 4. + ASSERT_EQ(NumSortedRuns(), 1); + + // Stage 2: + // Now we have one file at level 0, with size 4. We also have some data in + // mem table. Let's continue generating new files at level 0, but don't + // trigger level-0 compaction. + // First, clean up memtable before inserting new data. This will generate + // a level-0 file, with size around 0.4 (according to previously written + // data amount). + dbfull()->Flush(FlushOptions()); + for (int num = 0; num < options.level0_file_num_compaction_trigger - 3; + num++) { + // Write 110KB (11 values, each 10K) + for (int i = 0; i < 100; i++) { + ASSERT_OK(Put(Key(key_idx), RandomString(&rnd, 990))); + key_idx++; + } + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(NumSortedRuns(), num + 3); + } + + // Generate one more file at level-0, which should trigger level-0 + // compaction. + for (int i = 0; i < 100; i++) { + ASSERT_OK(Put(Key(key_idx), RandomString(&rnd, 990))); + key_idx++; + } + dbfull()->TEST_WaitForCompact(); + // Before compaction, we have 4 files at level 0, with size 4, 0.4, 1, 1. + // After compaction, we should have 3 files, with size 4, 0.4, 2. + ASSERT_EQ(NumSortedRuns(), 3); + // Stage 3: + // Now we have 3 files at level 0, with size 4, 0.4, 2. Generate one + // more file at level-0, which should trigger level-0 compaction. + for (int i = 0; i < 100; i++) { + ASSERT_OK(Put(Key(key_idx), RandomString(&rnd, 990))); + key_idx++; + } + dbfull()->TEST_WaitForCompact(); + // Level-0 compaction is triggered, but no file will be picked up. + ASSERT_EQ(NumSortedRuns(), 4); +} + +TEST_P(DBTestUniversalCompaction, UniversalCompactionCompressRatio1) { + if (!Snappy_Supported()) { + return; + } + + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.write_buffer_size = 100 << 10; // 100KB + options.target_file_size_base = 32 << 10; // 32KB + options.level0_file_num_compaction_trigger = 2; + options.num_levels = num_levels_; + options.compaction_options_universal.compression_size_percent = 70; + DestroyAndReopen(options); + + Random rnd(301); + int key_idx = 0; + + // The first compaction (2) is compressed. + for (int num = 0; num < 2; num++) { + // Write 110KB (11 values, each 10K) + for (int i = 0; i < 11; i++) { + ASSERT_OK(Put(Key(key_idx), CompressibleString(&rnd, 10000))); + key_idx++; + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_LT(TotalSize(), 110000U * 2 * 0.9); + + // The second compaction (4) is compressed + for (int num = 0; num < 2; num++) { + // Write 110KB (11 values, each 10K) + for (int i = 0; i < 11; i++) { + ASSERT_OK(Put(Key(key_idx), CompressibleString(&rnd, 10000))); + key_idx++; + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_LT(TotalSize(), 110000 * 4 * 0.9); + + // The third compaction (2 4) is compressed since this time it is + // (1 1 3.2) and 3.2/5.2 doesn't reach ratio. + for (int num = 0; num < 2; num++) { + // Write 110KB (11 values, each 10K) + for (int i = 0; i < 11; i++) { + ASSERT_OK(Put(Key(key_idx), CompressibleString(&rnd, 10000))); + key_idx++; + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_LT(TotalSize(), 110000 * 6 * 0.9); + + // When we start for the compaction up to (2 4 8), the latest + // compressed is not compressed. + for (int num = 0; num < 8; num++) { + // Write 110KB (11 values, each 10K) + for (int i = 0; i < 11; i++) { + ASSERT_OK(Put(Key(key_idx), CompressibleString(&rnd, 10000))); + key_idx++; + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_GT(TotalSize(), 110000 * 11 * 0.8 + 110000 * 2); +} + +TEST_P(DBTestUniversalCompaction, UniversalCompactionCompressRatio2) { + if (!Snappy_Supported()) { + return; + } + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.write_buffer_size = 100 << 10; // 100KB + options.target_file_size_base = 32 << 10; // 32KB + options.level0_file_num_compaction_trigger = 2; + options.num_levels = num_levels_; + options.compaction_options_universal.compression_size_percent = 95; + DestroyAndReopen(options); + + Random rnd(301); + int key_idx = 0; + + // When we start for the compaction up to (2 4 8), the latest + // compressed is compressed given the size ratio to compress. + for (int num = 0; num < 14; num++) { + // Write 120KB (12 values, each 10K) + for (int i = 0; i < 12; i++) { + ASSERT_OK(Put(Key(key_idx), CompressibleString(&rnd, 10000))); + key_idx++; + } + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_LT(TotalSize(), 120000U * 12 * 0.8 + 120000 * 2); +} + +// Test that checks trivial move in universal compaction +TEST_P(DBTestUniversalCompaction, UniversalCompactionTrivialMoveTest1) { + int32_t trivial_move = 0; + int32_t non_trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", [&](void* arg) { + non_trivial_move++; + ASSERT_TRUE(arg != nullptr); + int output_level = *(static_cast(arg)); + ASSERT_EQ(output_level, 0); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.compaction_options_universal.allow_trivial_move = true; + options.num_levels = 2; + options.write_buffer_size = 100 << 10; // 100KB + options.level0_file_num_compaction_trigger = 3; + options.max_background_compactions = 1; + options.target_file_size_base = 32 * 1024; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Trigger compaction if size amplification exceeds 110% + options.compaction_options_universal.max_size_amplification_percent = 110; + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + Random rnd(301); + int num_keys = 250000; + for (int i = 0; i < num_keys; i++) { + ASSERT_OK(Put(1, Key(i), Key(i))); + } + std::vector values; + + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + + ASSERT_GT(trivial_move, 0); + ASSERT_GT(non_trivial_move, 0); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} +// Test that checks trivial move in universal compaction +TEST_P(DBTestUniversalCompaction, UniversalCompactionTrivialMoveTest2) { + int32_t trivial_move = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:TrivialMove", + [&](void* arg) { trivial_move++; }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:NonTrivial", [&](void* arg) { + ASSERT_TRUE(arg != nullptr); + int output_level = *(static_cast(arg)); + ASSERT_EQ(output_level, 0); + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.compaction_options_universal.allow_trivial_move = true; + options.num_levels = 15; + options.write_buffer_size = 100 << 10; // 100KB + options.level0_file_num_compaction_trigger = 8; + options.max_background_compactions = 2; + options.target_file_size_base = 64 * 1024; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + // Trigger compaction if size amplification exceeds 110% + options.compaction_options_universal.max_size_amplification_percent = 110; + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + Random rnd(301); + int num_keys = 500000; + for (int i = 0; i < num_keys; i++) { + ASSERT_OK(Put(1, Key(i), Key(i))); + } + std::vector values; + + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + + ASSERT_GT(trivial_move, 0); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_P(DBTestUniversalCompaction, UniversalCompactionFourPaths) { + Options options = CurrentOptions(); + options.db_paths.emplace_back(dbname_, 300 * 1024); + options.db_paths.emplace_back(dbname_ + "_2", 300 * 1024); + options.db_paths.emplace_back(dbname_ + "_3", 500 * 1024); + options.db_paths.emplace_back(dbname_ + "_4", 1024 * 1024 * 1024); + options.memtable_factory.reset( + new SpecialSkipListFactory(KNumKeysByGenerateNewFile - 1)); + options.compaction_style = kCompactionStyleUniversal; + options.compaction_options_universal.size_ratio = 5; + options.write_buffer_size = 111 << 10; // 114KB + options.arena_block_size = 4 << 10; + options.level0_file_num_compaction_trigger = 2; + options.num_levels = 1; + + std::vector filenames; + env_->GetChildren(options.db_paths[1].path, &filenames); + // Delete archival files. + for (size_t i = 0; i < filenames.size(); ++i) { + env_->DeleteFile(options.db_paths[1].path + "/" + filenames[i]); + } + env_->DeleteDir(options.db_paths[1].path); + Reopen(options); + + Random rnd(301); + int key_idx = 0; + + // First three 110KB files are not going to second path. + // After that, (100K, 200K) + for (int num = 0; num < 3; num++) { + GenerateNewFile(&rnd, &key_idx); + } + + // Another 110KB triggers a compaction to 400K file to second path + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); + + // (1, 4) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1,1,4) -> (2, 4) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + // (1, 2, 4) -> (3, 4) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + // (1, 3, 4) -> (8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); + + // (1, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 1, 8) -> (2, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + + // (1, 2, 8) -> (3, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + // (1, 3, 8) -> (4, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); + + // (1, 4, 8) -> (5, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[3].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[2].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + for (int i = 0; i < key_idx; i++) { + auto v = Get(Key(i)); + ASSERT_NE(v, "NOT_FOUND"); + ASSERT_TRUE(v.size() == 1 || v.size() == 990); + } + + Reopen(options); + + for (int i = 0; i < key_idx; i++) { + auto v = Get(Key(i)); + ASSERT_NE(v, "NOT_FOUND"); + ASSERT_TRUE(v.size() == 1 || v.size() == 990); + } + + Destroy(options); +} + +TEST_P(DBTestUniversalCompaction, IncreaseUniversalCompactionNumLevels) { + std::function verify_func = [&](int num_keys_in_db) { + std::string keys_in_db; + Iterator* iter = dbfull()->NewIterator(ReadOptions(), handles_[1]); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + keys_in_db.append(iter->key().ToString()); + keys_in_db.push_back(','); + } + delete iter; + + std::string expected_keys; + for (int i = 0; i <= num_keys_in_db; i++) { + expected_keys.append(Key(i)); + expected_keys.push_back(','); + } + + ASSERT_EQ(keys_in_db, expected_keys); + }; + + Random rnd(301); + int max_key1 = 200; + int max_key2 = 600; + int max_key3 = 800; + const int KNumKeysPerFile = 10; + + // Stage 1: open a DB with universal compaction, num_levels=1 + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 1; + options.write_buffer_size = 200 << 10; // 200KB + options.level0_file_num_compaction_trigger = 3; + options.memtable_factory.reset(new SpecialSkipListFactory(KNumKeysPerFile)); + options = CurrentOptions(options); + CreateAndReopenWithCF({"pikachu"}, options); + + for (int i = 0; i <= max_key1; i++) { + // each value is 10K + ASSERT_OK(Put(1, Key(i), RandomString(&rnd, 10000))); + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + + // Stage 2: reopen with universal compaction, num_levels=4 + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 4; + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + verify_func(max_key1); + + // Insert more keys + for (int i = max_key1 + 1; i <= max_key2; i++) { + // each value is 10K + ASSERT_OK(Put(1, Key(i), RandomString(&rnd, 10000))); + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + + verify_func(max_key2); + // Compaction to non-L0 has happened. + ASSERT_GT(NumTableFilesAtLevel(options.num_levels - 1, 1), 0); + + // Stage 3: Revert it back to one level and revert to num_levels=1. + options.num_levels = 4; + options.target_file_size_base = INT_MAX; + ReopenWithColumnFamilies({"default", "pikachu"}, options); + // Compact all to level 0 + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 0; + compact_options.exclusive_manual_compaction = exclusive_manual_compaction_; + dbfull()->CompactRange(compact_options, handles_[1], nullptr, nullptr); + // Need to restart it once to remove higher level records in manifest. + ReopenWithColumnFamilies({"default", "pikachu"}, options); + // Final reopen + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 1; + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + + // Insert more keys + for (int i = max_key2 + 1; i <= max_key3; i++) { + // each value is 10K + ASSERT_OK(Put(1, Key(i), RandomString(&rnd, 10000))); + dbfull()->TEST_WaitForFlushMemTable(handles_[1]); + dbfull()->TEST_WaitForCompact(); + } + ASSERT_OK(Flush(1)); + dbfull()->TEST_WaitForCompact(); + verify_func(max_key3); +} + + +TEST_P(DBTestUniversalCompaction, UniversalCompactionSecondPathRatio) { + if (!Snappy_Supported()) { + return; + } + Options options = CurrentOptions(); + options.db_paths.emplace_back(dbname_, 500 * 1024); + options.db_paths.emplace_back(dbname_ + "_2", 1024 * 1024 * 1024); + options.compaction_style = kCompactionStyleUniversal; + options.compaction_options_universal.size_ratio = 5; + options.write_buffer_size = 111 << 10; // 114KB + options.arena_block_size = 4 << 10; + options.level0_file_num_compaction_trigger = 2; + options.num_levels = 1; + options.memtable_factory.reset( + new SpecialSkipListFactory(KNumKeysByGenerateNewFile - 1)); + + std::vector filenames; + env_->GetChildren(options.db_paths[1].path, &filenames); + // Delete archival files. + for (size_t i = 0; i < filenames.size(); ++i) { + env_->DeleteFile(options.db_paths[1].path + "/" + filenames[i]); + } + env_->DeleteDir(options.db_paths[1].path); + Reopen(options); + + Random rnd(301); + int key_idx = 0; + + // First three 110KB files are not going to second path. + // After that, (100K, 200K) + for (int num = 0; num < 3; num++) { + GenerateNewFile(&rnd, &key_idx); + } + + // Another 110KB triggers a compaction to 400K file to second path + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + + // (1, 4) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1,1,4) -> (2, 4) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 2, 4) -> (3, 4) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(2, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + // (1, 3, 4) -> (8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + // (1, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 1, 8) -> (2, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(1, GetSstFileCount(dbname_)); + + // (1, 2, 8) -> (3, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(2, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + // (1, 3, 8) -> (4, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(2, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + // (1, 4, 8) -> (5, 8) + GenerateNewFile(&rnd, &key_idx); + ASSERT_EQ(2, GetSstFileCount(options.db_paths[1].path)); + ASSERT_EQ(0, GetSstFileCount(dbname_)); + + for (int i = 0; i < key_idx; i++) { + auto v = Get(Key(i)); + ASSERT_NE(v, "NOT_FOUND"); + ASSERT_TRUE(v.size() == 1 || v.size() == 990); + } + + Reopen(options); + + for (int i = 0; i < key_idx; i++) { + auto v = Get(Key(i)); + ASSERT_NE(v, "NOT_FOUND"); + ASSERT_TRUE(v.size() == 1 || v.size() == 990); + } + + Destroy(options); +} + +TEST_P(DBTestUniversalCompaction, FullCompactionInBottomPriThreadPool) { + const int kNumFilesTrigger = 3; + Env::Default()->SetBackgroundThreads(1, Env::Priority::BOTTOM); + for (bool allow_ingest_behind : {false, true}) { + Options options = CurrentOptions(); + options.allow_ingest_behind = allow_ingest_behind; + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = num_levels_; + options.write_buffer_size = 100 << 10; // 100KB + options.target_file_size_base = 32 << 10; // 32KB + options.level0_file_num_compaction_trigger = kNumFilesTrigger; + // Trigger compaction if size amplification exceeds 110% + options.compaction_options_universal.max_size_amplification_percent = 110; + DestroyAndReopen(options); + + int num_bottom_pri_compactions = 0; + SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BGWorkBottomCompaction", + [&](void* arg) { ++num_bottom_pri_compactions; }); + SyncPoint::GetInstance()->EnableProcessing(); + + Random rnd(301); + for (int num = 0; num < kNumFilesTrigger; num++) { + ASSERT_EQ(NumSortedRuns(), num); + int key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + } + dbfull()->TEST_WaitForCompact(); + + if (allow_ingest_behind || num_levels_ > 1) { + // allow_ingest_behind increases number of levels while sanitizing. + ASSERT_EQ(1, num_bottom_pri_compactions); + } else { + // for single-level universal, everything's bottom level so nothing should + // be executed in bottom-pri thread pool. + ASSERT_EQ(0, num_bottom_pri_compactions); + } + // Verify that size amplification did occur + ASSERT_EQ(NumSortedRuns(), 1); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + } + Env::Default()->SetBackgroundThreads(0, Env::Priority::BOTTOM); +} + +TEST_P(DBTestUniversalCompaction, ConcurrentBottomPriLowPriCompactions) { + if (num_levels_ == 1) { + // for single-level universal, everything's bottom level so nothing should + // be executed in bottom-pri thread pool. + return; + } + const int kNumFilesTrigger = 3; + Env::Default()->SetBackgroundThreads(1, Env::Priority::BOTTOM); + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = num_levels_; + options.write_buffer_size = 100 << 10; // 100KB + options.target_file_size_base = 32 << 10; // 32KB + options.level0_file_num_compaction_trigger = kNumFilesTrigger; + // Trigger compaction if size amplification exceeds 110% + options.compaction_options_universal.max_size_amplification_percent = 110; + DestroyAndReopen(options); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {// wait for the full compaction to be picked before adding files intended + // for the second one. + {"DBImpl::BackgroundCompaction:ForwardToBottomPriPool", + "DBTestUniversalCompaction:ConcurrentBottomPriLowPriCompactions:0"}, + // the full (bottom-pri) compaction waits until a partial (low-pri) + // compaction has started to verify they can run in parallel. + {"DBImpl::BackgroundCompaction:NonTrivial", + "DBImpl::BGWorkBottomCompaction"}}); + SyncPoint::GetInstance()->EnableProcessing(); + + Random rnd(301); + for (int i = 0; i < 2; ++i) { + for (int num = 0; num < kNumFilesTrigger; num++) { + int key_idx = 0; + GenerateNewFile(&rnd, &key_idx, true /* no_wait */); + // use no_wait above because that one waits for flush and compaction. We + // don't want to wait for compaction because the full compaction is + // intentionally blocked while more files are flushed. + dbfull()->TEST_WaitForFlushMemTable(); + } + if (i == 0) { + TEST_SYNC_POINT( + "DBTestUniversalCompaction:ConcurrentBottomPriLowPriCompactions:0"); + } + } + dbfull()->TEST_WaitForCompact(); + + // First compaction should output to bottom level. Second should output to L0 + // since older L0 files pending compaction prevent it from being placed lower. + ASSERT_EQ(NumSortedRuns(), 2); + ASSERT_GT(NumTableFilesAtLevel(0), 0); + ASSERT_GT(NumTableFilesAtLevel(num_levels_ - 1), 0); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + Env::Default()->SetBackgroundThreads(0, Env::Priority::BOTTOM); +} + +TEST_P(DBTestUniversalCompaction, RecalculateScoreAfterPicking) { + // Regression test for extra compactions scheduled. Once enough compactions + // have been scheduled to bring the score below one, we should stop + // scheduling more; otherwise, other CFs/DBs may be delayed unnecessarily. + const int kNumFilesTrigger = 8; + Options options = CurrentOptions(); + options.compaction_options_universal.max_merge_width = kNumFilesTrigger / 2; + options.compaction_options_universal.max_size_amplification_percent = + static_cast(-1); + options.compaction_style = kCompactionStyleUniversal; + options.level0_file_num_compaction_trigger = kNumFilesTrigger; + options.num_levels = num_levels_; + options.write_buffer_size = 100 << 10; // 100KB + Reopen(options); + + std::atomic num_compactions_attempted(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:Start", [&](void* arg) { + ++num_compactions_attempted; + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + Random rnd(301); + for (int num = 0; num < kNumFilesTrigger; num++) { + ASSERT_EQ(NumSortedRuns(), num); + int key_idx = 0; + GenerateNewFile(&rnd, &key_idx); + } + dbfull()->TEST_WaitForCompact(); + // Compacting the first four files was enough to bring the score below one so + // there's no need to schedule any more compactions. + ASSERT_EQ(1, num_compactions_attempted); + ASSERT_EQ(NumSortedRuns(), 5); +} + +INSTANTIATE_TEST_CASE_P(UniversalCompactionNumLevels, DBTestUniversalCompaction, + ::testing::Combine(::testing::Values(1, 3, 5), + ::testing::Bool())); + +class DBTestUniversalManualCompactionOutputPathId + : public DBTestUniversalCompactionBase { + public: + DBTestUniversalManualCompactionOutputPathId() : + DBTestUniversalCompactionBase( + "/db_universal_compaction_manual_pid_test") {} +}; + +TEST_P(DBTestUniversalManualCompactionOutputPathId, + ManualCompactionOutputPathId) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.db_paths.emplace_back(dbname_, 1000000000); + options.db_paths.emplace_back(dbname_ + "_2", 1000000000); + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = num_levels_; + options.target_file_size_base = 1 << 30; // Big size + options.level0_file_num_compaction_trigger = 10; + Destroy(options); + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + MakeTables(3, "p", "q", 1); + dbfull()->TEST_WaitForCompact(); + ASSERT_EQ(2, TotalLiveFiles(1)); + ASSERT_EQ(2, GetSstFileCount(options.db_paths[0].path)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[1].path)); + + // Full compaction to DB path 0 + CompactRangeOptions compact_options; + compact_options.target_path_id = 1; + compact_options.exclusive_manual_compaction = exclusive_manual_compaction_; + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr); + ASSERT_EQ(1, TotalLiveFiles(1)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[0].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + + ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, options); + ASSERT_EQ(1, TotalLiveFiles(1)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[0].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + + MakeTables(1, "p", "q", 1); + ASSERT_EQ(2, TotalLiveFiles(1)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[0].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + + ReopenWithColumnFamilies({kDefaultColumnFamilyName, "pikachu"}, options); + ASSERT_EQ(2, TotalLiveFiles(1)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[0].path)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[1].path)); + + // Full compaction to DB path 0 + compact_options.target_path_id = 0; + compact_options.exclusive_manual_compaction = exclusive_manual_compaction_; + db_->CompactRange(compact_options, handles_[1], nullptr, nullptr); + ASSERT_EQ(1, TotalLiveFiles(1)); + ASSERT_EQ(1, GetSstFileCount(options.db_paths[0].path)); + ASSERT_EQ(0, GetSstFileCount(options.db_paths[1].path)); + + // Fail when compacting to an invalid path ID + compact_options.target_path_id = 2; + compact_options.exclusive_manual_compaction = exclusive_manual_compaction_; + ASSERT_TRUE(db_->CompactRange(compact_options, handles_[1], nullptr, nullptr) + .IsInvalidArgument()); +} + +INSTANTIATE_TEST_CASE_P(DBTestUniversalManualCompactionOutputPathId, + DBTestUniversalManualCompactionOutputPathId, + ::testing::Combine(::testing::Values(1, 8), + ::testing::Bool())); + +} // namespace rocksdb + +#endif // !defined(ROCKSDB_LITE) + +int main(int argc, char** argv) { +#if !defined(ROCKSDB_LITE) + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +#else + return 0; +#endif +} diff --git a/db/db_wal_test.cc b/db/db_wal_test.cc new file mode 100644 index 00000000000..461fe467391 --- /dev/null +++ b/db/db_wal_test.cc @@ -0,0 +1,1229 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/db_test_util.h" +#include "options/options_helper.h" +#include "port/port.h" +#include "port/stack_trace.h" +#include "util/fault_injection_test_env.h" +#include "util/sync_point.h" + +namespace rocksdb { +class DBWALTest : public DBTestBase { + public: + DBWALTest() : DBTestBase("/db_wal_test") {} +}; + +TEST_F(DBWALTest, WAL) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + WriteOptions writeOpt = WriteOptions(); + writeOpt.disableWAL = true; + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v1")); + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v1")); + + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_EQ("v1", Get(1, "bar")); + + writeOpt.disableWAL = false; + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v2")); + writeOpt.disableWAL = true; + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v2")); + + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + // Both value's should be present. + ASSERT_EQ("v2", Get(1, "bar")); + ASSERT_EQ("v2", Get(1, "foo")); + + writeOpt.disableWAL = true; + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "bar", "v3")); + writeOpt.disableWAL = false; + ASSERT_OK(dbfull()->Put(writeOpt, handles_[1], "foo", "v3")); + + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + // again both values should be present. + ASSERT_EQ("v3", Get(1, "foo")); + ASSERT_EQ("v3", Get(1, "bar")); + } while (ChangeWalOptions()); +} + +TEST_F(DBWALTest, RollLog) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_OK(Put(1, "baz", "v5")); + + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + for (int i = 0; i < 10; i++) { + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + } + ASSERT_OK(Put(1, "foo", "v4")); + for (int i = 0; i < 10; i++) { + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + } + } while (ChangeWalOptions()); +} + +TEST_F(DBWALTest, SyncWALNotBlockWrite) { + Options options = CurrentOptions(); + options.max_write_buffer_number = 4; + DestroyAndReopen(options); + + ASSERT_OK(Put("foo1", "bar1")); + ASSERT_OK(Put("foo5", "bar5")); + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"WritableFileWriter::SyncWithoutFlush:1", + "DBWALTest::SyncWALNotBlockWrite:1"}, + {"DBWALTest::SyncWALNotBlockWrite:2", + "WritableFileWriter::SyncWithoutFlush:2"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + rocksdb::port::Thread thread([&]() { ASSERT_OK(db_->SyncWAL()); }); + + TEST_SYNC_POINT("DBWALTest::SyncWALNotBlockWrite:1"); + ASSERT_OK(Put("foo2", "bar2")); + ASSERT_OK(Put("foo3", "bar3")); + FlushOptions fo; + fo.wait = false; + ASSERT_OK(db_->Flush(fo)); + ASSERT_OK(Put("foo4", "bar4")); + + TEST_SYNC_POINT("DBWALTest::SyncWALNotBlockWrite:2"); + + thread.join(); + + ASSERT_EQ(Get("foo1"), "bar1"); + ASSERT_EQ(Get("foo2"), "bar2"); + ASSERT_EQ(Get("foo3"), "bar3"); + ASSERT_EQ(Get("foo4"), "bar4"); + ASSERT_EQ(Get("foo5"), "bar5"); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(DBWALTest, SyncWALNotWaitWrite) { + ASSERT_OK(Put("foo1", "bar1")); + ASSERT_OK(Put("foo3", "bar3")); + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"SpecialEnv::WalFile::Append:1", "DBWALTest::SyncWALNotWaitWrite:1"}, + {"DBWALTest::SyncWALNotWaitWrite:2", "SpecialEnv::WalFile::Append:2"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + rocksdb::port::Thread thread([&]() { ASSERT_OK(Put("foo2", "bar2")); }); + // Moving this to SyncWAL before the actual fsync + // TEST_SYNC_POINT("DBWALTest::SyncWALNotWaitWrite:1"); + ASSERT_OK(db_->SyncWAL()); + // Moving this to SyncWAL after actual fsync + // TEST_SYNC_POINT("DBWALTest::SyncWALNotWaitWrite:2"); + + thread.join(); + + ASSERT_EQ(Get("foo1"), "bar1"); + ASSERT_EQ(Get("foo2"), "bar2"); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(DBWALTest, Recover) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_OK(Put(1, "baz", "v5")); + + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_EQ("v1", Get(1, "foo")); + + ASSERT_EQ("v1", Get(1, "foo")); + ASSERT_EQ("v5", Get(1, "baz")); + ASSERT_OK(Put(1, "bar", "v2")); + ASSERT_OK(Put(1, "foo", "v3")); + + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_EQ("v3", Get(1, "foo")); + ASSERT_OK(Put(1, "foo", "v4")); + ASSERT_EQ("v4", Get(1, "foo")); + ASSERT_EQ("v2", Get(1, "bar")); + ASSERT_EQ("v5", Get(1, "baz")); + } while (ChangeWalOptions()); +} + +TEST_F(DBWALTest, RecoverWithTableHandle) { + do { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.disable_auto_compactions = true; + options.avoid_flush_during_recovery = false; + DestroyAndReopen(options); + CreateAndReopenWithCF({"pikachu"}, options); + + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_OK(Put(1, "bar", "v2")); + ASSERT_OK(Flush(1)); + ASSERT_OK(Put(1, "foo", "v3")); + ASSERT_OK(Put(1, "bar", "v4")); + ASSERT_OK(Flush(1)); + ASSERT_OK(Put(1, "big", std::string(100, 'a'))); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + + std::vector> files; + dbfull()->TEST_GetFilesMetaData(handles_[1], &files); + size_t total_files = 0; + for (const auto& level : files) { + total_files += level.size(); + } + ASSERT_EQ(total_files, 3); + for (const auto& level : files) { + for (const auto& file : level) { + if (kInfiniteMaxOpenFiles == option_config_) { + ASSERT_TRUE(file.table_reader_handle != nullptr); + } else { + ASSERT_TRUE(file.table_reader_handle == nullptr); + } + } + } + } while (ChangeWalOptions()); +} + +TEST_F(DBWALTest, IgnoreRecoveredLog) { + std::string backup_logs = dbname_ + "/backup_logs"; + + do { + // delete old files in backup_logs directory + env_->CreateDirIfMissing(backup_logs); + std::vector old_files; + env_->GetChildren(backup_logs, &old_files); + for (auto& file : old_files) { + if (file != "." && file != "..") { + env_->DeleteFile(backup_logs + "/" + file); + } + } + Options options = CurrentOptions(); + options.create_if_missing = true; + options.merge_operator = MergeOperators::CreateUInt64AddOperator(); + options.wal_dir = dbname_ + "/logs"; + DestroyAndReopen(options); + + // fill up the DB + std::string one, two; + PutFixed64(&one, 1); + PutFixed64(&two, 2); + ASSERT_OK(db_->Merge(WriteOptions(), Slice("foo"), Slice(one))); + ASSERT_OK(db_->Merge(WriteOptions(), Slice("foo"), Slice(one))); + ASSERT_OK(db_->Merge(WriteOptions(), Slice("bar"), Slice(one))); + + // copy the logs to backup + std::vector logs; + env_->GetChildren(options.wal_dir, &logs); + for (auto& log : logs) { + if (log != ".." && log != ".") { + CopyFile(options.wal_dir + "/" + log, backup_logs + "/" + log); + } + } + + // recover the DB + Reopen(options); + ASSERT_EQ(two, Get("foo")); + ASSERT_EQ(one, Get("bar")); + Close(); + + // copy the logs from backup back to wal dir + for (auto& log : logs) { + if (log != ".." && log != ".") { + CopyFile(backup_logs + "/" + log, options.wal_dir + "/" + log); + } + } + // this should ignore the log files, recovery should not happen again + // if the recovery happens, the same merge operator would be called twice, + // leading to incorrect results + Reopen(options); + ASSERT_EQ(two, Get("foo")); + ASSERT_EQ(one, Get("bar")); + Close(); + Destroy(options); + Reopen(options); + Close(); + + // copy the logs from backup back to wal dir + env_->CreateDirIfMissing(options.wal_dir); + for (auto& log : logs) { + if (log != ".." && log != ".") { + CopyFile(backup_logs + "/" + log, options.wal_dir + "/" + log); + } + } + // assert that we successfully recovered only from logs, even though we + // destroyed the DB + Reopen(options); + ASSERT_EQ(two, Get("foo")); + ASSERT_EQ(one, Get("bar")); + + // Recovery will fail if DB directory doesn't exist. + Destroy(options); + // copy the logs from backup back to wal dir + env_->CreateDirIfMissing(options.wal_dir); + for (auto& log : logs) { + if (log != ".." && log != ".") { + CopyFile(backup_logs + "/" + log, options.wal_dir + "/" + log); + // we won't be needing this file no more + env_->DeleteFile(backup_logs + "/" + log); + } + } + Status s = TryReopen(options); + ASSERT_TRUE(!s.ok()); + Destroy(options); + } while (ChangeWalOptions()); +} + +TEST_F(DBWALTest, RecoveryWithEmptyLog) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_OK(Put(1, "foo", "v2")); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "foo", "v3")); + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + ASSERT_EQ("v3", Get(1, "foo")); + } while (ChangeWalOptions()); +} + +#if !(defined NDEBUG) || !defined(OS_WIN) +TEST_F(DBWALTest, PreallocateBlock) { + Options options = CurrentOptions(); + options.write_buffer_size = 10 * 1000 * 1000; + options.max_total_wal_size = 0; + + size_t expected_preallocation_size = static_cast( + options.write_buffer_size + options.write_buffer_size / 10); + + DestroyAndReopen(options); + + std::atomic called(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBTestWalFile.GetPreallocationStatus", [&](void* arg) { + ASSERT_TRUE(arg != nullptr); + size_t preallocation_size = *(static_cast(arg)); + ASSERT_EQ(expected_preallocation_size, preallocation_size); + called.fetch_add(1); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + Put("", ""); + Flush(); + Put("", ""); + Close(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + ASSERT_EQ(2, called.load()); + + options.max_total_wal_size = 1000 * 1000; + expected_preallocation_size = static_cast(options.max_total_wal_size); + Reopen(options); + called.store(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBTestWalFile.GetPreallocationStatus", [&](void* arg) { + ASSERT_TRUE(arg != nullptr); + size_t preallocation_size = *(static_cast(arg)); + ASSERT_EQ(expected_preallocation_size, preallocation_size); + called.fetch_add(1); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + Put("", ""); + Flush(); + Put("", ""); + Close(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + ASSERT_EQ(2, called.load()); + + options.db_write_buffer_size = 800 * 1000; + expected_preallocation_size = + static_cast(options.db_write_buffer_size); + Reopen(options); + called.store(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBTestWalFile.GetPreallocationStatus", [&](void* arg) { + ASSERT_TRUE(arg != nullptr); + size_t preallocation_size = *(static_cast(arg)); + ASSERT_EQ(expected_preallocation_size, preallocation_size); + called.fetch_add(1); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + Put("", ""); + Flush(); + Put("", ""); + Close(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + ASSERT_EQ(2, called.load()); + + expected_preallocation_size = 700 * 1000; + std::shared_ptr write_buffer_manager = + std::make_shared(static_cast(700 * 1000)); + options.write_buffer_manager = write_buffer_manager; + Reopen(options); + called.store(0); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBTestWalFile.GetPreallocationStatus", [&](void* arg) { + ASSERT_TRUE(arg != nullptr); + size_t preallocation_size = *(static_cast(arg)); + ASSERT_EQ(expected_preallocation_size, preallocation_size); + called.fetch_add(1); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + Put("", ""); + Flush(); + Put("", ""); + Close(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + ASSERT_EQ(2, called.load()); +} +#endif // !(defined NDEBUG) || !defined(OS_WIN) + +#ifndef ROCKSDB_LITE +TEST_F(DBWALTest, FullPurgePreservesRecycledLog) { + // For github issue #1303 + for (int i = 0; i < 2; ++i) { + Options options = CurrentOptions(); + options.create_if_missing = true; + options.recycle_log_file_num = 2; + if (i != 0) { + options.wal_dir = alternative_wal_dir_; + } + + DestroyAndReopen(options); + ASSERT_OK(Put("foo", "v1")); + VectorLogPtr log_files; + ASSERT_OK(dbfull()->GetSortedWalFiles(log_files)); + ASSERT_GT(log_files.size(), 0); + ASSERT_OK(Flush()); + + // Now the original WAL is in log_files[0] and should be marked for + // recycling. + // Verify full purge cannot remove this file. + JobContext job_context(0); + dbfull()->TEST_LockMutex(); + dbfull()->FindObsoleteFiles(&job_context, true /* force */); + dbfull()->TEST_UnlockMutex(); + dbfull()->PurgeObsoleteFiles(job_context); + + if (i == 0) { + ASSERT_OK( + env_->FileExists(LogFileName(dbname_, log_files[0]->LogNumber()))); + } else { + ASSERT_OK(env_->FileExists( + LogFileName(alternative_wal_dir_, log_files[0]->LogNumber()))); + } + } +} + +TEST_F(DBWALTest, GetSortedWalFiles) { + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + VectorLogPtr log_files; + ASSERT_OK(dbfull()->GetSortedWalFiles(log_files)); + ASSERT_EQ(0, log_files.size()); + + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_OK(dbfull()->GetSortedWalFiles(log_files)); + ASSERT_EQ(1, log_files.size()); + } while (ChangeWalOptions()); +} + +TEST_F(DBWALTest, RecoveryWithLogDataForSomeCFs) { + // Test for regression of WAL cleanup missing files that don't contain data + // for every column family. + do { + CreateAndReopenWithCF({"pikachu"}, CurrentOptions()); + ASSERT_OK(Put(1, "foo", "v1")); + ASSERT_OK(Put(1, "foo", "v2")); + uint64_t earliest_log_nums[2]; + for (int i = 0; i < 2; ++i) { + if (i > 0) { + ReopenWithColumnFamilies({"default", "pikachu"}, CurrentOptions()); + } + VectorLogPtr log_files; + ASSERT_OK(dbfull()->GetSortedWalFiles(log_files)); + if (log_files.size() > 0) { + earliest_log_nums[i] = log_files[0]->LogNumber(); + } else { + earliest_log_nums[i] = port::kMaxUint64; + } + } + // Check at least the first WAL was cleaned up during the recovery. + ASSERT_LT(earliest_log_nums[0], earliest_log_nums[1]); + } while (ChangeWalOptions()); +} + +TEST_F(DBWALTest, RecoverWithLargeLog) { + do { + { + Options options = CurrentOptions(); + CreateAndReopenWithCF({"pikachu"}, options); + ASSERT_OK(Put(1, "big1", std::string(200000, '1'))); + ASSERT_OK(Put(1, "big2", std::string(200000, '2'))); + ASSERT_OK(Put(1, "small3", std::string(10, '3'))); + ASSERT_OK(Put(1, "small4", std::string(10, '4'))); + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 0); + } + + // Make sure that if we re-open with a small write buffer size that + // we flush table files in the middle of a large log file. + Options options; + options.write_buffer_size = 100000; + options = CurrentOptions(options); + ReopenWithColumnFamilies({"default", "pikachu"}, options); + ASSERT_EQ(NumTableFilesAtLevel(0, 1), 3); + ASSERT_EQ(std::string(200000, '1'), Get(1, "big1")); + ASSERT_EQ(std::string(200000, '2'), Get(1, "big2")); + ASSERT_EQ(std::string(10, '3'), Get(1, "small3")); + ASSERT_EQ(std::string(10, '4'), Get(1, "small4")); + ASSERT_GT(NumTableFilesAtLevel(0, 1), 1); + } while (ChangeWalOptions()); +} + +// In https://reviews.facebook.net/D20661 we change +// recovery behavior: previously for each log file each column family +// memtable was flushed, even it was empty. Now it's changed: +// we try to create the smallest number of table files by merging +// updates from multiple logs +TEST_F(DBWALTest, RecoverCheckFileAmountWithSmallWriteBuffer) { + Options options = CurrentOptions(); + options.write_buffer_size = 5000000; + CreateAndReopenWithCF({"pikachu", "dobrynia", "nikitich"}, options); + + // Since we will reopen DB with smaller write_buffer_size, + // each key will go to new SST file + ASSERT_OK(Put(1, Key(10), DummyString(1000000))); + ASSERT_OK(Put(1, Key(10), DummyString(1000000))); + ASSERT_OK(Put(1, Key(10), DummyString(1000000))); + ASSERT_OK(Put(1, Key(10), DummyString(1000000))); + + ASSERT_OK(Put(3, Key(10), DummyString(1))); + // Make 'dobrynia' to be flushed and new WAL file to be created + ASSERT_OK(Put(2, Key(10), DummyString(7500000))); + ASSERT_OK(Put(2, Key(1), DummyString(1))); + dbfull()->TEST_WaitForFlushMemTable(handles_[2]); + { + auto tables = ListTableFiles(env_, dbname_); + ASSERT_EQ(tables.size(), static_cast(1)); + // Make sure 'dobrynia' was flushed: check sst files amount + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "dobrynia"), + static_cast(1)); + } + // New WAL file + ASSERT_OK(Put(1, Key(1), DummyString(1))); + ASSERT_OK(Put(1, Key(1), DummyString(1))); + ASSERT_OK(Put(3, Key(10), DummyString(1))); + ASSERT_OK(Put(3, Key(10), DummyString(1))); + ASSERT_OK(Put(3, Key(10), DummyString(1))); + + options.write_buffer_size = 4096; + options.arena_block_size = 4096; + ReopenWithColumnFamilies({"default", "pikachu", "dobrynia", "nikitich"}, + options); + { + // No inserts => default is empty + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default"), + static_cast(0)); + // First 4 keys goes to separate SSTs + 1 more SST for 2 smaller keys + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "pikachu"), + static_cast(5)); + // 1 SST for big key + 1 SST for small one + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "dobrynia"), + static_cast(2)); + // 1 SST for all keys + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "nikitich"), + static_cast(1)); + } +} + +// In https://reviews.facebook.net/D20661 we change +// recovery behavior: previously for each log file each column family +// memtable was flushed, even it wasn't empty. Now it's changed: +// we try to create the smallest number of table files by merging +// updates from multiple logs +TEST_F(DBWALTest, RecoverCheckFileAmount) { + Options options = CurrentOptions(); + options.write_buffer_size = 100000; + options.arena_block_size = 4 * 1024; + options.avoid_flush_during_recovery = false; + CreateAndReopenWithCF({"pikachu", "dobrynia", "nikitich"}, options); + + ASSERT_OK(Put(0, Key(1), DummyString(1))); + ASSERT_OK(Put(1, Key(1), DummyString(1))); + ASSERT_OK(Put(2, Key(1), DummyString(1))); + + // Make 'nikitich' memtable to be flushed + ASSERT_OK(Put(3, Key(10), DummyString(1002400))); + ASSERT_OK(Put(3, Key(1), DummyString(1))); + dbfull()->TEST_WaitForFlushMemTable(handles_[3]); + // 4 memtable are not flushed, 1 sst file + { + auto tables = ListTableFiles(env_, dbname_); + ASSERT_EQ(tables.size(), static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "nikitich"), + static_cast(1)); + } + // Memtable for 'nikitich' has flushed, new WAL file has opened + // 4 memtable still not flushed + + // Write to new WAL file + ASSERT_OK(Put(0, Key(1), DummyString(1))); + ASSERT_OK(Put(1, Key(1), DummyString(1))); + ASSERT_OK(Put(2, Key(1), DummyString(1))); + + // Fill up 'nikitich' one more time + ASSERT_OK(Put(3, Key(10), DummyString(1002400))); + // make it flush + ASSERT_OK(Put(3, Key(1), DummyString(1))); + dbfull()->TEST_WaitForFlushMemTable(handles_[3]); + // There are still 4 memtable not flushed, and 2 sst tables + ASSERT_OK(Put(0, Key(1), DummyString(1))); + ASSERT_OK(Put(1, Key(1), DummyString(1))); + ASSERT_OK(Put(2, Key(1), DummyString(1))); + + { + auto tables = ListTableFiles(env_, dbname_); + ASSERT_EQ(tables.size(), static_cast(2)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "nikitich"), + static_cast(2)); + } + + ReopenWithColumnFamilies({"default", "pikachu", "dobrynia", "nikitich"}, + options); + { + std::vector table_files = ListTableFiles(env_, dbname_); + // Check, that records for 'default', 'dobrynia' and 'pikachu' from + // first, second and third WALs went to the same SST. + // So, there is 6 SSTs: three for 'nikitich', one for 'default', one for + // 'dobrynia', one for 'pikachu' + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "default"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "nikitich"), + static_cast(3)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "dobrynia"), + static_cast(1)); + ASSERT_EQ(GetNumberOfSstFilesForColumnFamily(db_, "pikachu"), + static_cast(1)); + } +} + +TEST_F(DBWALTest, SyncMultipleLogs) { + const uint64_t kNumBatches = 2; + const int kBatchSize = 1000; + + Options options = CurrentOptions(); + options.create_if_missing = true; + options.write_buffer_size = 4096; + Reopen(options); + + WriteBatch batch; + WriteOptions wo; + wo.sync = true; + + for (uint64_t b = 0; b < kNumBatches; b++) { + batch.Clear(); + for (int i = 0; i < kBatchSize; i++) { + batch.Put(Key(i), DummyString(128)); + } + + dbfull()->Write(wo, &batch); + } + + ASSERT_OK(dbfull()->SyncWAL()); +} + +// Github issue 1339. Prior the fix we read sequence id from the first log to +// a local variable, then keep increase the variable as we replay logs, +// ignoring actual sequence id of the records. This is incorrect if some writes +// come with WAL disabled. +TEST_F(DBWALTest, PartOfWritesWithWALDisabled) { + std::unique_ptr fault_env( + new FaultInjectionTestEnv(env_)); + Options options = CurrentOptions(); + options.env = fault_env.get(); + options.disable_auto_compactions = true; + WriteOptions wal_on, wal_off; + wal_on.sync = true; + wal_on.disableWAL = false; + wal_off.disableWAL = true; + CreateAndReopenWithCF({"dummy"}, options); + ASSERT_OK(Put(1, "dummy", "d1", wal_on)); // seq id 1 + ASSERT_OK(Put(1, "dummy", "d2", wal_off)); + ASSERT_OK(Put(1, "dummy", "d3", wal_off)); + ASSERT_OK(Put(0, "key", "v4", wal_on)); // seq id 4 + ASSERT_OK(Flush(0)); + ASSERT_OK(Put(0, "key", "v5", wal_on)); // seq id 5 + ASSERT_EQ("v5", Get(0, "key")); + dbfull()->FlushWAL(false); + // Simulate a crash. + fault_env->SetFilesystemActive(false); + Close(); + fault_env->ResetState(); + ReopenWithColumnFamilies({"default", "dummy"}, options); + // Prior to the fix, we may incorrectly recover "v5" with sequence id = 3. + ASSERT_EQ("v5", Get(0, "key")); + // Destroy DB before destruct fault_env. + Destroy(options); +} + +// +// Test WAL recovery for the various modes available +// +class RecoveryTestHelper { + public: + // Number of WAL files to generate + static const int kWALFilesCount = 10; + // Starting number for the WAL file name like 00010.log + static const int kWALFileOffset = 10; + // Keys to be written per WAL file + static const int kKeysPerWALFile = 133; + // Size of the value + static const int kValueSize = 96; + + // Create WAL files with values filled in + static void FillData(DBWALTest* test, const Options& options, + const size_t wal_count, size_t* count) { + const ImmutableDBOptions db_options(options); + + *count = 0; + + shared_ptr table_cache = NewLRUCache(50, 0); + EnvOptions env_options; + WriteBufferManager write_buffer_manager(db_options.db_write_buffer_size); + + unique_ptr versions; + unique_ptr wal_manager; + WriteController write_controller; + + versions.reset(new VersionSet(test->dbname_, &db_options, env_options, + table_cache.get(), &write_buffer_manager, + &write_controller)); + + wal_manager.reset(new WalManager(db_options, env_options)); + + std::unique_ptr current_log_writer; + + for (size_t j = kWALFileOffset; j < wal_count + kWALFileOffset; j++) { + uint64_t current_log_number = j; + std::string fname = LogFileName(test->dbname_, current_log_number); + unique_ptr file; + ASSERT_OK(db_options.env->NewWritableFile(fname, &file, env_options)); + unique_ptr file_writer( + new WritableFileWriter(std::move(file), env_options)); + current_log_writer.reset( + new log::Writer(std::move(file_writer), current_log_number, + db_options.recycle_log_file_num > 0)); + + WriteBatch batch; + for (int i = 0; i < kKeysPerWALFile; i++) { + std::string key = "key" + ToString((*count)++); + std::string value = test->DummyString(kValueSize); + assert(current_log_writer.get() != nullptr); + uint64_t seq = versions->LastSequence() + 1; + batch.Clear(); + batch.Put(key, value); + WriteBatchInternal::SetSequence(&batch, seq); + current_log_writer->AddRecord(WriteBatchInternal::Contents(&batch)); + versions->SetLastToBeWrittenSequence(seq); + versions->SetLastSequence(seq); + } + } + } + + // Recreate and fill the store with some data + static size_t FillData(DBWALTest* test, Options* options) { + options->create_if_missing = true; + test->DestroyAndReopen(*options); + test->Close(); + + size_t count = 0; + FillData(test, *options, kWALFilesCount, &count); + return count; + } + + // Read back all the keys we wrote and return the number of keys found + static size_t GetData(DBWALTest* test) { + size_t count = 0; + for (size_t i = 0; i < kWALFilesCount * kKeysPerWALFile; i++) { + if (test->Get("key" + ToString(i)) != "NOT_FOUND") { + ++count; + } + } + return count; + } + + // Manuall corrupt the specified WAL + static void CorruptWAL(DBWALTest* test, const Options& options, + const double off, const double len, + const int wal_file_id, const bool trunc = false) { + Env* env = options.env; + std::string fname = LogFileName(test->dbname_, wal_file_id); + uint64_t size; + ASSERT_OK(env->GetFileSize(fname, &size)); + ASSERT_GT(size, 0); +#ifdef OS_WIN + // Windows disk cache behaves differently. When we truncate + // the original content is still in the cache due to the original + // handle is still open. Generally, in Windows, one prohibits + // shared access to files and it is not needed for WAL but we allow + // it to induce corruption at various tests. + test->Close(); +#endif + if (trunc) { + ASSERT_EQ(0, truncate(fname.c_str(), static_cast(size * off))); + } else { + InduceCorruption(fname, static_cast(size * off + 8), + static_cast(size * len)); + } + } + + // Overwrite data with 'a' from offset for length len + static void InduceCorruption(const std::string& filename, size_t offset, + size_t len) { + ASSERT_GT(len, 0U); + + int fd = open(filename.c_str(), O_RDWR); + + // On windows long is 32-bit + ASSERT_LE(offset, std::numeric_limits::max()); + + ASSERT_GT(fd, 0); + ASSERT_EQ(offset, lseek(fd, static_cast(offset), SEEK_SET)); + + void* buf = alloca(len); + memset(buf, 'b', len); + ASSERT_EQ(len, write(fd, buf, static_cast(len))); + + close(fd); + } +}; + +// Test scope: +// - We expect to open the data store when there is incomplete trailing writes +// at the end of any of the logs +// - We do not expect to open the data store for corruption +TEST_F(DBWALTest, kTolerateCorruptedTailRecords) { + const int jstart = RecoveryTestHelper::kWALFileOffset; + const int jend = jstart + RecoveryTestHelper::kWALFilesCount; + + for (auto trunc : {true, false}) { /* Corruption style */ + for (int i = 0; i < 3; i++) { /* Corruption offset position */ + for (int j = jstart; j < jend; j++) { /* WAL file */ + // Fill data for testing + Options options = CurrentOptions(); + const size_t row_count = RecoveryTestHelper::FillData(this, &options); + // test checksum failure or parsing + RecoveryTestHelper::CorruptWAL(this, options, /*off=*/i * .3, + /*len%=*/.1, /*wal=*/j, trunc); + + if (trunc) { + options.wal_recovery_mode = + WALRecoveryMode::kTolerateCorruptedTailRecords; + options.create_if_missing = false; + ASSERT_OK(TryReopen(options)); + const size_t recovered_row_count = RecoveryTestHelper::GetData(this); + ASSERT_TRUE(i == 0 || recovered_row_count > 0); + ASSERT_LT(recovered_row_count, row_count); + } else { + options.wal_recovery_mode = + WALRecoveryMode::kTolerateCorruptedTailRecords; + ASSERT_NOK(TryReopen(options)); + } + } + } + } +} + +// Test scope: +// We don't expect the data store to be opened if there is any corruption +// (leading, middle or trailing -- incomplete writes or corruption) +TEST_F(DBWALTest, kAbsoluteConsistency) { + const int jstart = RecoveryTestHelper::kWALFileOffset; + const int jend = jstart + RecoveryTestHelper::kWALFilesCount; + + // Verify clean slate behavior + Options options = CurrentOptions(); + const size_t row_count = RecoveryTestHelper::FillData(this, &options); + options.wal_recovery_mode = WALRecoveryMode::kAbsoluteConsistency; + options.create_if_missing = false; + ASSERT_OK(TryReopen(options)); + ASSERT_EQ(RecoveryTestHelper::GetData(this), row_count); + + for (auto trunc : {true, false}) { /* Corruption style */ + for (int i = 0; i < 4; i++) { /* Corruption offset position */ + if (trunc && i == 0) { + continue; + } + + for (int j = jstart; j < jend; j++) { /* wal files */ + // fill with new date + RecoveryTestHelper::FillData(this, &options); + // corrupt the wal + RecoveryTestHelper::CorruptWAL(this, options, /*off=*/i * .3, + /*len%=*/.1, j, trunc); + // verify + options.wal_recovery_mode = WALRecoveryMode::kAbsoluteConsistency; + options.create_if_missing = false; + ASSERT_NOK(TryReopen(options)); + } + } + } +} + +// Test scope: +// - We expect to open data store under all circumstances +// - We expect only data upto the point where the first error was encountered +TEST_F(DBWALTest, kPointInTimeRecovery) { + const int jstart = RecoveryTestHelper::kWALFileOffset; + const int jend = jstart + RecoveryTestHelper::kWALFilesCount; + const int maxkeys = + RecoveryTestHelper::kWALFilesCount * RecoveryTestHelper::kKeysPerWALFile; + + for (auto trunc : {true, false}) { /* Corruption style */ + for (int i = 0; i < 4; i++) { /* Offset of corruption */ + for (int j = jstart; j < jend; j++) { /* WAL file */ + // Fill data for testing + Options options = CurrentOptions(); + const size_t row_count = RecoveryTestHelper::FillData(this, &options); + + // Corrupt the wal + RecoveryTestHelper::CorruptWAL(this, options, /*off=*/i * .3, + /*len%=*/.1, j, trunc); + + // Verify + options.wal_recovery_mode = WALRecoveryMode::kPointInTimeRecovery; + options.create_if_missing = false; + ASSERT_OK(TryReopen(options)); + + // Probe data for invariants + size_t recovered_row_count = RecoveryTestHelper::GetData(this); + ASSERT_LT(recovered_row_count, row_count); + + bool expect_data = true; + for (size_t k = 0; k < maxkeys; ++k) { + bool found = Get("key" + ToString(i)) != "NOT_FOUND"; + if (expect_data && !found) { + expect_data = false; + } + ASSERT_EQ(found, expect_data); + } + + const size_t min = RecoveryTestHelper::kKeysPerWALFile * + (j - RecoveryTestHelper::kWALFileOffset); + ASSERT_GE(recovered_row_count, min); + if (!trunc && i != 0) { + const size_t max = RecoveryTestHelper::kKeysPerWALFile * + (j - RecoveryTestHelper::kWALFileOffset + 1); + ASSERT_LE(recovered_row_count, max); + } + } + } + } +} + +// Test scope: +// - We expect to open the data store under all scenarios +// - We expect to have recovered records past the corruption zone +TEST_F(DBWALTest, kSkipAnyCorruptedRecords) { + const int jstart = RecoveryTestHelper::kWALFileOffset; + const int jend = jstart + RecoveryTestHelper::kWALFilesCount; + + for (auto trunc : {true, false}) { /* Corruption style */ + for (int i = 0; i < 4; i++) { /* Corruption offset */ + for (int j = jstart; j < jend; j++) { /* wal files */ + // Fill data for testing + Options options = CurrentOptions(); + const size_t row_count = RecoveryTestHelper::FillData(this, &options); + + // Corrupt the WAL + RecoveryTestHelper::CorruptWAL(this, options, /*off=*/i * .3, + /*len%=*/.1, j, trunc); + + // Verify behavior + options.wal_recovery_mode = WALRecoveryMode::kSkipAnyCorruptedRecords; + options.create_if_missing = false; + ASSERT_OK(TryReopen(options)); + + // Probe data for invariants + size_t recovered_row_count = RecoveryTestHelper::GetData(this); + ASSERT_LT(recovered_row_count, row_count); + + if (!trunc) { + ASSERT_TRUE(i != 0 || recovered_row_count > 0); + } + } + } + } +} + +TEST_F(DBWALTest, AvoidFlushDuringRecovery) { + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.avoid_flush_during_recovery = false; + + // Test with flush after recovery. + Reopen(options); + ASSERT_OK(Put("foo", "v1")); + ASSERT_OK(Put("bar", "v2")); + ASSERT_OK(Flush()); + ASSERT_OK(Put("foo", "v3")); + ASSERT_OK(Put("bar", "v4")); + ASSERT_EQ(1, TotalTableFiles()); + // Reopen DB. Check if WAL logs flushed. + Reopen(options); + ASSERT_EQ("v3", Get("foo")); + ASSERT_EQ("v4", Get("bar")); + ASSERT_EQ(2, TotalTableFiles()); + + // Test without flush after recovery. + options.avoid_flush_during_recovery = true; + DestroyAndReopen(options); + ASSERT_OK(Put("foo", "v5")); + ASSERT_OK(Put("bar", "v6")); + ASSERT_OK(Flush()); + ASSERT_OK(Put("foo", "v7")); + ASSERT_OK(Put("bar", "v8")); + ASSERT_EQ(1, TotalTableFiles()); + // Reopen DB. WAL logs should not be flushed this time. + Reopen(options); + ASSERT_EQ("v7", Get("foo")); + ASSERT_EQ("v8", Get("bar")); + ASSERT_EQ(1, TotalTableFiles()); + + // Force flush with allow_2pc. + options.avoid_flush_during_recovery = true; + options.allow_2pc = true; + ASSERT_OK(Put("foo", "v9")); + ASSERT_OK(Put("bar", "v10")); + ASSERT_OK(Flush()); + ASSERT_OK(Put("foo", "v11")); + ASSERT_OK(Put("bar", "v12")); + Reopen(options); + ASSERT_EQ("v11", Get("foo")); + ASSERT_EQ("v12", Get("bar")); + ASSERT_EQ(2, TotalTableFiles()); +} + +TEST_F(DBWALTest, WalCleanupAfterAvoidFlushDuringRecovery) { + // Verifies WAL files that were present during recovery, but not flushed due + // to avoid_flush_during_recovery, will be considered for deletion at a later + // stage. We check at least one such file is deleted during Flush(). + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + options.avoid_flush_during_recovery = true; + Reopen(options); + + ASSERT_OK(Put("foo", "v1")); + Reopen(options); + for (int i = 0; i < 2; ++i) { + if (i > 0) { + // Flush() triggers deletion of obsolete tracked files + Flush(); + } + VectorLogPtr log_files; + ASSERT_OK(dbfull()->GetSortedWalFiles(log_files)); + if (i == 0) { + ASSERT_GT(log_files.size(), 0); + } else { + ASSERT_EQ(0, log_files.size()); + } + } +} + +TEST_F(DBWALTest, RecoverWithoutFlush) { + Options options = CurrentOptions(); + options.avoid_flush_during_recovery = true; + options.create_if_missing = false; + options.disable_auto_compactions = true; + options.write_buffer_size = 64 * 1024 * 1024; + + size_t count = RecoveryTestHelper::FillData(this, &options); + auto validateData = [this, count]() { + for (size_t i = 0; i < count; i++) { + ASSERT_NE(Get("key" + ToString(i)), "NOT_FOUND"); + } + }; + Reopen(options); + validateData(); + // Insert some data without flush + ASSERT_OK(Put("foo", "foo_v1")); + ASSERT_OK(Put("bar", "bar_v1")); + Reopen(options); + validateData(); + ASSERT_EQ(Get("foo"), "foo_v1"); + ASSERT_EQ(Get("bar"), "bar_v1"); + // Insert again and reopen + ASSERT_OK(Put("foo", "foo_v2")); + ASSERT_OK(Put("bar", "bar_v2")); + Reopen(options); + validateData(); + ASSERT_EQ(Get("foo"), "foo_v2"); + ASSERT_EQ(Get("bar"), "bar_v2"); + // manual flush and insert again + Flush(); + ASSERT_EQ(Get("foo"), "foo_v2"); + ASSERT_EQ(Get("bar"), "bar_v2"); + ASSERT_OK(Put("foo", "foo_v3")); + ASSERT_OK(Put("bar", "bar_v3")); + Reopen(options); + validateData(); + ASSERT_EQ(Get("foo"), "foo_v3"); + ASSERT_EQ(Get("bar"), "bar_v3"); +} + +TEST_F(DBWALTest, RecoverWithoutFlushMultipleCF) { + const std::string kSmallValue = "v"; + const std::string kLargeValue = DummyString(1024); + Options options = CurrentOptions(); + options.avoid_flush_during_recovery = true; + options.create_if_missing = false; + options.disable_auto_compactions = true; + + auto countWalFiles = [this]() { + VectorLogPtr log_files; + dbfull()->GetSortedWalFiles(log_files); + return log_files.size(); + }; + + // Create DB with multiple column families and multiple log files. + CreateAndReopenWithCF({"one", "two"}, options); + ASSERT_OK(Put(0, "key1", kSmallValue)); + ASSERT_OK(Put(1, "key2", kLargeValue)); + Flush(1); + ASSERT_EQ(1, countWalFiles()); + ASSERT_OK(Put(0, "key3", kSmallValue)); + ASSERT_OK(Put(2, "key4", kLargeValue)); + Flush(2); + ASSERT_EQ(2, countWalFiles()); + + // Reopen, insert and flush. + options.db_write_buffer_size = 64 * 1024 * 1024; + ReopenWithColumnFamilies({"default", "one", "two"}, options); + ASSERT_EQ(Get(0, "key1"), kSmallValue); + ASSERT_EQ(Get(1, "key2"), kLargeValue); + ASSERT_EQ(Get(0, "key3"), kSmallValue); + ASSERT_EQ(Get(2, "key4"), kLargeValue); + // Insert more data. + ASSERT_OK(Put(0, "key5", kLargeValue)); + ASSERT_OK(Put(1, "key6", kLargeValue)); + ASSERT_EQ(3, countWalFiles()); + Flush(1); + ASSERT_OK(Put(2, "key7", kLargeValue)); + dbfull()->FlushWAL(false); + ASSERT_EQ(4, countWalFiles()); + + // Reopen twice and validate. + for (int i = 0; i < 2; i++) { + ReopenWithColumnFamilies({"default", "one", "two"}, options); + ASSERT_EQ(Get(0, "key1"), kSmallValue); + ASSERT_EQ(Get(1, "key2"), kLargeValue); + ASSERT_EQ(Get(0, "key3"), kSmallValue); + ASSERT_EQ(Get(2, "key4"), kLargeValue); + ASSERT_EQ(Get(0, "key5"), kLargeValue); + ASSERT_EQ(Get(1, "key6"), kLargeValue); + ASSERT_EQ(Get(2, "key7"), kLargeValue); + ASSERT_EQ(4, countWalFiles()); + } +} + +// In this test we are trying to do the following: +// 1. Create a DB with corrupted WAL log; +// 2. Open with avoid_flush_during_recovery = true; +// 3. Append more data without flushing, which creates new WAL log. +// 4. Open again. See if it can correctly handle previous corruption. +TEST_F(DBWALTest, RecoverFromCorruptedWALWithoutFlush) { + const int jstart = RecoveryTestHelper::kWALFileOffset; + const int jend = jstart + RecoveryTestHelper::kWALFilesCount; + const int kAppendKeys = 100; + Options options = CurrentOptions(); + options.avoid_flush_during_recovery = true; + options.create_if_missing = false; + options.disable_auto_compactions = true; + options.write_buffer_size = 64 * 1024 * 1024; + + auto getAll = [this]() { + std::vector> data; + ReadOptions ropt; + Iterator* iter = dbfull()->NewIterator(ropt); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + data.push_back( + std::make_pair(iter->key().ToString(), iter->value().ToString())); + } + delete iter; + return data; + }; + for (auto& mode : wal_recovery_mode_string_map) { + options.wal_recovery_mode = mode.second; + for (auto trunc : {true, false}) { + for (int i = 0; i < 4; i++) { + for (int j = jstart; j < jend; j++) { + // Create corrupted WAL + RecoveryTestHelper::FillData(this, &options); + RecoveryTestHelper::CorruptWAL(this, options, /*off=*/i * .3, + /*len%=*/.1, /*wal=*/j, trunc); + // Skip the test if DB won't open. + if (!TryReopen(options).ok()) { + ASSERT_TRUE(options.wal_recovery_mode == + WALRecoveryMode::kAbsoluteConsistency || + (!trunc && + options.wal_recovery_mode == + WALRecoveryMode::kTolerateCorruptedTailRecords)); + continue; + } + ASSERT_OK(TryReopen(options)); + // Append some more data. + for (int k = 0; k < kAppendKeys; k++) { + std::string key = "extra_key" + ToString(k); + std::string value = DummyString(RecoveryTestHelper::kValueSize); + ASSERT_OK(Put(key, value)); + } + // Save data for comparison. + auto data = getAll(); + // Reopen. Verify data. + ASSERT_OK(TryReopen(options)); + auto actual_data = getAll(); + ASSERT_EQ(data, actual_data); + } + } + } + } +} + +#endif // ROCKSDB_LITE + +TEST_F(DBWALTest, WalTermTest) { + Options options = CurrentOptions(); + options.env = env_; + CreateAndReopenWithCF({"pikachu"}, options); + + ASSERT_OK(Put(1, "foo", "bar")); + + WriteOptions wo; + wo.sync = true; + wo.disableWAL = false; + + WriteBatch batch; + batch.Put("foo", "bar"); + batch.MarkWalTerminationPoint(); + batch.Put("foo2", "bar2"); + + ASSERT_OK(dbfull()->Write(wo, &batch)); + + // make sure we can re-open it. + ASSERT_OK(TryReopenWithColumnFamilies({"default", "pikachu"}, options)); + ASSERT_EQ("bar", Get(1, "foo")); + ASSERT_EQ("NOT_FOUND", Get(1, "foo2")); +} +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/db_write_test.cc b/db/db_write_test.cc new file mode 100644 index 00000000000..e3e8ad829d7 --- /dev/null +++ b/db/db_write_test.cc @@ -0,0 +1,129 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include +#include +#include +#include +#include "db/db_test_util.h" +#include "db/write_batch_internal.h" +#include "db/write_thread.h" +#include "port/port.h" +#include "port/stack_trace.h" +#include "util/fault_injection_test_env.h" +#include "util/string_util.h" +#include "util/sync_point.h" + +namespace rocksdb { + +// Test variations of WriteImpl. +class DBWriteTest : public DBTestBase, public testing::WithParamInterface { + public: + DBWriteTest() : DBTestBase("/db_write_test") {} + + Options GetOptions() { return DBTestBase::GetOptions(GetParam()); } + + void Open() { DBTestBase::Reopen(GetOptions()); } +}; + +// Sequence number should be return through input write batch. +TEST_P(DBWriteTest, ReturnSeuqneceNumber) { + Random rnd(4422); + Open(); + for (int i = 0; i < 100; i++) { + WriteBatch batch; + batch.Put("key" + ToString(i), test::RandomHumanReadableString(&rnd, 10)); + ASSERT_OK(dbfull()->Write(WriteOptions(), &batch)); + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), + WriteBatchInternal::Sequence(&batch)); + } +} + +TEST_P(DBWriteTest, ReturnSeuqneceNumberMultiThreaded) { + constexpr size_t kThreads = 16; + constexpr size_t kNumKeys = 1000; + Open(); + ASSERT_EQ(0, dbfull()->GetLatestSequenceNumber()); + // Check each sequence is used once and only once. + std::vector flags(kNumKeys * kThreads + 1); + for (size_t i = 0; i < flags.size(); i++) { + flags[i].clear(); + } + auto writer = [&](size_t id) { + Random rnd(4422 + static_cast(id)); + for (size_t k = 0; k < kNumKeys; k++) { + WriteBatch batch; + batch.Put("key" + ToString(id) + "-" + ToString(k), + test::RandomHumanReadableString(&rnd, 10)); + ASSERT_OK(dbfull()->Write(WriteOptions(), &batch)); + SequenceNumber sequence = WriteBatchInternal::Sequence(&batch); + ASSERT_GT(sequence, 0); + ASSERT_LE(sequence, kNumKeys * kThreads); + // The sequence isn't consumed by someone else. + ASSERT_FALSE(flags[sequence].test_and_set()); + } + }; + std::vector threads; + for (size_t i = 0; i < kThreads; i++) { + threads.emplace_back(writer, i); + } + for (size_t i = 0; i < kThreads; i++) { + threads[i].join(); + } +} + +TEST_P(DBWriteTest, IOErrorOnWALWritePropagateToWriteThreadFollower) { + constexpr int kNumThreads = 5; + std::unique_ptr mock_env( + new FaultInjectionTestEnv(Env::Default())); + Options options = GetOptions(); + options.env = mock_env.get(); + Reopen(options); + std::atomic ready_count{0}; + std::atomic leader_count{0}; + std::vector threads; + mock_env->SetFilesystemActive(false); + // Wait until all threads linked to write threads, to make sure + // all threads join the same batch group. + SyncPoint::GetInstance()->SetCallBack( + "WriteThread::JoinBatchGroup:Wait", [&](void* arg) { + ready_count++; + auto* w = reinterpret_cast(arg); + if (w->state == WriteThread::STATE_GROUP_LEADER) { + leader_count++; + while (ready_count < kNumThreads) { + // busy waiting + } + } + }); + SyncPoint::GetInstance()->EnableProcessing(); + for (int i = 0; i < kNumThreads; i++) { + threads.push_back(port::Thread( + [&](int index) { + // All threads should fail. + ASSERT_FALSE(Put("key" + ToString(index), "value").ok()); + }, + i)); + } + for (int i = 0; i < kNumThreads; i++) { + threads[i].join(); + } + ASSERT_EQ(1, leader_count); + // Close before mock_env destruct. + Close(); +} + +INSTANTIATE_TEST_CASE_P(DBWriteTestInstance, DBWriteTest, + testing::Values(DBTestBase::kDefault, + DBTestBase::kConcurrentWALWrites, + DBTestBase::kPipelinedWrite)); + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/dbformat.cc b/db/dbformat.cc index baeb868027e..f287ae9f4e0 100644 --- a/db/dbformat.cc +++ b/db/dbformat.cc @@ -1,36 +1,63 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #include "db/dbformat.h" +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include #include +#include "monitoring/perf_context_imp.h" #include "port/port.h" #include "util/coding.h" -#include "util/perf_context_imp.h" +#include "util/string_util.h" namespace rocksdb { +// kValueTypeForSeek defines the ValueType that should be passed when +// constructing a ParsedInternalKey object for seeking to a particular +// sequence number (since we sort sequence numbers in decreasing order +// and the value type is embedded as the low 8 bits in the sequence +// number in internal keys, we need to use the highest-numbered +// ValueType, not the lowest). +const ValueType kValueTypeForSeek = kTypeBlobIndex; +const ValueType kValueTypeForSeekForPrev = kTypeDeletion; + uint64_t PackSequenceAndType(uint64_t seq, ValueType t) { assert(seq <= kMaxSequenceNumber); - assert(t <= kValueTypeForSeek); + assert(IsExtendedValueType(t)); return (seq << 8) | t; } +void UnPackSequenceAndType(uint64_t packed, uint64_t* seq, ValueType* t) { + *seq = packed >> 8; + *t = static_cast(packed & 0xff); + + assert(*seq <= kMaxSequenceNumber); + assert(IsExtendedValueType(*t)); +} + void AppendInternalKey(std::string* result, const ParsedInternalKey& key) { result->append(key.user_key.data(), key.user_key.size()); PutFixed64(result, PackSequenceAndType(key.sequence, key.type)); } +void AppendInternalKeyFooter(std::string* result, SequenceNumber s, + ValueType t) { + PutFixed64(result, PackSequenceAndType(s, t)); +} + std::string ParsedInternalKey::DebugString(bool hex) const { char buf[50]; - snprintf(buf, sizeof(buf), "' @ %llu : %d", - (unsigned long long) sequence, - int(type)); + snprintf(buf, sizeof(buf), "' seq:%" PRIu64 ", type:%d", sequence, + static_cast(type)); std::string result = "'"; result += user_key.ToString(hex); result += buf; @@ -102,7 +129,7 @@ void InternalKeyComparator::FindShortestSeparator( Slice user_limit = ExtractUserKey(limit); std::string tmp(user_start.data(), user_start.size()); user_comparator_->FindShortestSeparator(&tmp, user_limit); - if (tmp.size() < user_start.size() && + if (tmp.size() <= user_start.size() && user_comparator_->Compare(user_start, tmp) < 0) { // User key has become shorter physically, but larger logically. // Tack on the earliest possible number to the shortened user key. @@ -117,7 +144,7 @@ void InternalKeyComparator::FindShortSuccessor(std::string* key) const { Slice user_key = ExtractUserKey(*key); std::string tmp(user_key.data(), user_key.size()); user_comparator_->FindShortSuccessor(&tmp); - if (tmp.size() < user_key.size() && + if (tmp.size() <= user_key.size() && user_comparator_->Compare(user_key, tmp) < 0) { // User key has become shorter physically, but larger logically. // Tack on the earliest possible number to the shortened user key. @@ -127,8 +154,8 @@ void InternalKeyComparator::FindShortSuccessor(std::string* key) const { } } -LookupKey::LookupKey(const Slice& user_key, SequenceNumber s) { - size_t usize = user_key.size(); +LookupKey::LookupKey(const Slice& _user_key, SequenceNumber s) { + size_t usize = _user_key.size(); size_t needed = usize + 13; // A conservative estimate char* dst; if (needed <= sizeof(space_)) { @@ -137,9 +164,10 @@ LookupKey::LookupKey(const Slice& user_key, SequenceNumber s) { dst = new char[needed]; } start_ = dst; - dst = EncodeVarint32(dst, usize + 8); + // NOTE: We don't support users keys of more than 2GB :) + dst = EncodeVarint32(dst, static_cast(usize + 8)); kstart_ = dst; - memcpy(dst, user_key.data(), usize); + memcpy(dst, _user_key.data(), usize); dst += usize; EncodeFixed64(dst, PackSequenceAndType(s, kValueTypeForSeek)); dst += 8; diff --git a/db/dbformat.h b/db/dbformat.h index eb5d8ed534b..c58b8363ab5 100644 --- a/db/dbformat.h +++ b/db/dbformat.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,6 +10,7 @@ #pragma once #include #include +#include #include "rocksdb/comparator.h" #include "rocksdb/db.h" #include "rocksdb/filter_policy.h" @@ -33,37 +34,64 @@ enum ValueType : unsigned char { kTypeDeletion = 0x0, kTypeValue = 0x1, kTypeMerge = 0x2, - // Following types are used only in write ahead logs. They are not used in - // memtables or sst files: - kTypeLogData = 0x3, - kTypeColumnFamilyDeletion = 0x4, - kTypeColumnFamilyValue = 0x5, - kTypeColumnFamilyMerge = 0x6, - kMaxValue = 0x7F + kTypeLogData = 0x3, // WAL only. + kTypeColumnFamilyDeletion = 0x4, // WAL only. + kTypeColumnFamilyValue = 0x5, // WAL only. + kTypeColumnFamilyMerge = 0x6, // WAL only. + kTypeSingleDeletion = 0x7, + kTypeColumnFamilySingleDeletion = 0x8, // WAL only. + kTypeBeginPrepareXID = 0x9, // WAL only. + kTypeEndPrepareXID = 0xA, // WAL only. + kTypeCommitXID = 0xB, // WAL only. + kTypeRollbackXID = 0xC, // WAL only. + kTypeNoop = 0xD, // WAL only. + kTypeColumnFamilyRangeDeletion = 0xE, // WAL only. + kTypeRangeDeletion = 0xF, // meta block + kTypeColumnFamilyBlobIndex = 0x10, // Blob DB only + kTypeBlobIndex = 0x11, // Blob DB only + kMaxValue = 0x7F // Not used for storing records. }; -// kValueTypeForSeek defines the ValueType that should be passed when -// constructing a ParsedInternalKey object for seeking to a particular -// sequence number (since we sort sequence numbers in decreasing order -// and the value type is embedded as the low 8 bits in the sequence -// number in internal keys, we need to use the highest-numbered -// ValueType, not the lowest). -static const ValueType kValueTypeForSeek = kTypeMerge; +// Defined in dbformat.cc +extern const ValueType kValueTypeForSeek; +extern const ValueType kValueTypeForSeekForPrev; + +// Checks whether a type is an inline value type +// (i.e. a type used in memtable skiplist and sst file datablock). +inline bool IsValueType(ValueType t) { + return t <= kTypeMerge || t == kTypeSingleDeletion || t == kTypeBlobIndex; +} + +// Checks whether a type is from user operation +// kTypeRangeDeletion is in meta block so this API is separated from above +inline bool IsExtendedValueType(ValueType t) { + return IsValueType(t) || t == kTypeRangeDeletion; +} // We leave eight bits empty at the bottom so a type and sequence# // can be packed together into 64-bits. static const SequenceNumber kMaxSequenceNumber = ((0x1ull << 56) - 1); +static const SequenceNumber kDisableGlobalSequenceNumber = port::kMaxUint64; + struct ParsedInternalKey { Slice user_key; SequenceNumber sequence; ValueType type; - ParsedInternalKey() { } // Intentionally left uninitialized (for speed) + ParsedInternalKey() + : sequence(kMaxSequenceNumber) // Make code analyzer happy + {} // Intentionally left uninitialized (for speed) ParsedInternalKey(const Slice& u, const SequenceNumber& seq, ValueType t) : user_key(u), sequence(seq), type(t) { } std::string DebugString(bool hex = false) const; + + void clear() { + user_key.clear(); + sequence = 0; + type = kTypeDeletion; + } }; // Return the length of the encoding of "key". @@ -71,11 +99,21 @@ inline size_t InternalKeyEncodingLength(const ParsedInternalKey& key) { return key.user_key.size() + 8; } +// Pack a sequence number and a ValueType into a uint64_t extern uint64_t PackSequenceAndType(uint64_t seq, ValueType t); +// Given the result of PackSequenceAndType, store the sequence number in *seq +// and the ValueType in *t. +extern void UnPackSequenceAndType(uint64_t packed, uint64_t* seq, ValueType* t); + // Append the serialization of "key" to *result. extern void AppendInternalKey(std::string* result, const ParsedInternalKey& key); +// Serialized internal key consists of user key followed by footer. +// This function appends the footer to *result, assuming that *result already +// contains the user key at the end. +extern void AppendInternalKeyFooter(std::string* result, SequenceNumber s, + ValueType t); // Attempt to parse an internal key from "internal_key". On success, // stores the parsed data in "*result", and returns true. @@ -111,17 +149,19 @@ class InternalKeyComparator : public Comparator { } virtual ~InternalKeyComparator() {} - virtual const char* Name() const; - virtual int Compare(const Slice& a, const Slice& b) const; - virtual void FindShortestSeparator( - std::string* start, - const Slice& limit) const; - virtual void FindShortSuccessor(std::string* key) const; + virtual const char* Name() const override; + virtual int Compare(const Slice& a, const Slice& b) const override; + virtual void FindShortestSeparator(std::string* start, + const Slice& limit) const override; + virtual void FindShortSuccessor(std::string* key) const override; const Comparator* user_comparator() const { return user_comparator_; } int Compare(const InternalKey& a, const InternalKey& b) const; int Compare(const ParsedInternalKey& a, const ParsedInternalKey& b) const; + virtual const Comparator* GetRootComparator() const override { + return user_comparator_->GetRootComparator(); + } }; // Modules in this directory should keep internal keys wrapped inside @@ -132,8 +172,22 @@ class InternalKey { std::string rep_; public: InternalKey() { } // Leave rep_ as empty to indicate it is invalid - InternalKey(const Slice& user_key, SequenceNumber s, ValueType t) { - AppendInternalKey(&rep_, ParsedInternalKey(user_key, s, t)); + InternalKey(const Slice& _user_key, SequenceNumber s, ValueType t) { + AppendInternalKey(&rep_, ParsedInternalKey(_user_key, s, t)); + } + + // sets the internal key to be bigger or equal to all internal keys with this + // user key + void SetMaxPossibleForUserKey(const Slice& _user_key) { + AppendInternalKey(&rep_, ParsedInternalKey(_user_key, kMaxSequenceNumber, + kValueTypeForSeek)); + } + + // sets the internal key to be smaller or equal to all internal keys with this + // user key + void SetMinPossibleForUserKey(const Slice& _user_key) { + AppendInternalKey( + &rep_, ParsedInternalKey(_user_key, 0, static_cast(0))); } bool Valid() const { @@ -148,6 +202,11 @@ class InternalKey { } Slice user_key() const { return ExtractUserKey(rep_); } + size_t size() { return rep_.size(); } + + void Set(const Slice& _user_key, SequenceNumber s, ValueType t) { + SetFrom(ParsedInternalKey(_user_key, s, t)); + } void SetFrom(const ParsedInternalKey& p) { rep_.clear(); @@ -156,6 +215,16 @@ class InternalKey { void Clear() { rep_.clear(); } + // The underlying representation. + // Intended only to be used together with ConvertFromUserKey(). + std::string* rep() { return &rep_; } + + // Assuming that *rep() contains a user key, this method makes internal key + // out of it in-place. This saves a memcpy compared to Set()/SetFrom(). + void ConvertFromUserKey(SequenceNumber s, ValueType t) { + AppendInternalKeyFooter(&rep_, s, t); + } + std::string DebugString(bool hex = false) const; }; @@ -174,17 +243,19 @@ inline bool ParseInternalKey(const Slice& internal_key, result->type = static_cast(c); assert(result->type <= ValueType::kMaxValue); result->user_key = Slice(internal_key.data(), n - 8); - return (c <= static_cast(kValueTypeForSeek)); + return IsExtendedValueType(result->type); } -// Update the sequence number in the internal key -inline void UpdateInternalKey(char* internal_key, - const size_t internal_key_size, - uint64_t seq, ValueType t) { - assert(internal_key_size >= 8); - char* seqtype = internal_key + internal_key_size - 8; +// Update the sequence number in the internal key. +// Guarantees not to invalidate ikey.data(). +inline void UpdateInternalKey(std::string* ikey, uint64_t seq, ValueType t) { + size_t ikey_sz = ikey->size(); + assert(ikey_sz >= 8); uint64_t newval = (seq << 8) | t; - EncodeFixed64(seqtype, newval); + + // Note: Since C++11, strings are guaranteed to be stored contiguously and + // string::operator[]() is guaranteed not to change ikey.data(). + EncodeFixed64(&(*ikey)[ikey_sz - 8], newval); } // Get the sequence number from the internal key @@ -201,18 +272,24 @@ class LookupKey { public: // Initialize *this for looking up user_key at a snapshot with // the specified sequence number. - LookupKey(const Slice& user_key, SequenceNumber sequence); + LookupKey(const Slice& _user_key, SequenceNumber sequence); ~LookupKey(); // Return a key suitable for lookup in a MemTable. - Slice memtable_key() const { return Slice(start_, end_ - start_); } + Slice memtable_key() const { + return Slice(start_, static_cast(end_ - start_)); + } // Return an internal key (suitable for passing to an internal iterator) - Slice internal_key() const { return Slice(kstart_, end_ - kstart_); } + Slice internal_key() const { + return Slice(kstart_, static_cast(end_ - kstart_)); + } // Return the user key - Slice user_key() const { return Slice(kstart_, end_ - kstart_ - 8); } + Slice user_key() const { + return Slice(kstart_, static_cast(end_ - kstart_ - 8)); + } private: // We construct a char array of the form: @@ -238,13 +315,30 @@ inline LookupKey::~LookupKey() { class IterKey { public: - IterKey() : key_(space_), buf_size_(sizeof(space_)), key_size_(0) {} + IterKey() + : buf_(space_), + buf_size_(sizeof(space_)), + key_(buf_), + key_size_(0), + is_user_key_(true) {} ~IterKey() { ResetBuffer(); } - Slice GetKey() const { return Slice(key_, key_size_); } + Slice GetInternalKey() const { + assert(!IsUserKey()); + return Slice(key_, key_size_); + } - const size_t Size() { return key_size_; } + Slice GetUserKey() const { + if (IsUserKey()) { + return Slice(key_, key_size_); + } else { + assert(key_size_ >= 8); + return Slice(key_, key_size_ - 8); + } + } + + size_t Size() const { return key_size_; } void Clear() { key_size_ = 0; } @@ -255,34 +349,70 @@ class IterKey { void TrimAppend(const size_t shared_len, const char* non_shared_data, const size_t non_shared_len) { assert(shared_len <= key_size_); - size_t total_size = shared_len + non_shared_len; - if (total_size <= buf_size_) { - key_size_ = total_size; - } else { + + if (IsKeyPinned() /* key is not in buf_ */) { + // Copy the key from external memory to buf_ (copy shared_len bytes) + EnlargeBufferIfNeeded(total_size); + memcpy(buf_, key_, shared_len); + } else if (total_size > buf_size_) { // Need to allocate space, delete previous space char* p = new char[total_size]; memcpy(p, key_, shared_len); - if (key_ != nullptr && key_ != space_) { - delete[] key_; + if (buf_ != space_) { + delete[] buf_; } - key_ = p; - key_size_ = total_size; + buf_ = p; buf_size_ = total_size; } - memcpy(key_ + shared_len, non_shared_data, non_shared_len); + memcpy(buf_ + shared_len, non_shared_data, non_shared_len); + key_ = buf_; + key_size_ = total_size; } - void SetKey(const Slice& key) { - size_t size = key.size(); - EnlargeBufferIfNeeded(size); - memcpy(key_, key.data(), size); - key_size_ = size; + Slice SetUserKey(const Slice& key, bool copy = true) { + is_user_key_ = true; + return SetKeyImpl(key, copy); + } + + Slice SetInternalKey(const Slice& key, bool copy = true) { + is_user_key_ = false; + return SetKeyImpl(key, copy); } + // Copies the content of key, updates the reference to the user key in ikey + // and returns a Slice referencing the new copy. + Slice SetInternalKey(const Slice& key, ParsedInternalKey* ikey) { + size_t key_n = key.size(); + assert(key_n >= 8); + SetInternalKey(key); + ikey->user_key = Slice(key_, key_n - 8); + return Slice(key_, key_n); + } + + // Copy the key into IterKey own buf_ + void OwnKey() { + assert(IsKeyPinned() == true); + + Reserve(key_size_); + memcpy(buf_, key_, key_size_); + key_ = buf_; + } + + // Update the sequence number in the internal key. Guarantees not to + // invalidate slices to the key (and the user key). + void UpdateInternalKey(uint64_t seq, ValueType t) { + assert(!IsKeyPinned()); + assert(key_size_ >= 8); + uint64_t newval = (seq << 8) | t; + EncodeFixed64(&buf_[key_size_ - 8], newval); + } + + bool IsKeyPinned() const { return (key_ != buf_); } + void SetInternalKey(const Slice& key_prefix, const Slice& user_key, SequenceNumber s, ValueType value_type = kValueTypeForSeek) { @@ -290,11 +420,14 @@ class IterKey { size_t usize = user_key.size(); EnlargeBufferIfNeeded(psize + usize + sizeof(uint64_t)); if (psize > 0) { - memcpy(key_, key_prefix.data(), psize); + memcpy(buf_, key_prefix.data(), psize); } - memcpy(key_ + psize, user_key.data(), usize); - EncodeFixed64(key_ + usize + psize, PackSequenceAndType(s, value_type)); + memcpy(buf_ + psize, user_key.data(), usize); + EncodeFixed64(buf_ + usize + psize, PackSequenceAndType(s, value_type)); + + key_ = buf_; key_size_ = psize + usize + sizeof(uint64_t); + is_user_key_ = false; } void SetInternalKey(const Slice& user_key, SequenceNumber s, @@ -319,22 +452,43 @@ class IterKey { void EncodeLengthPrefixedKey(const Slice& key) { auto size = key.size(); - EnlargeBufferIfNeeded(size + VarintLength(size)); - char* ptr = EncodeVarint32(key_, size); + EnlargeBufferIfNeeded(size + static_cast(VarintLength(size))); + char* ptr = EncodeVarint32(buf_, static_cast(size)); memcpy(ptr, key.data(), size); + key_ = buf_; + is_user_key_ = true; } + bool IsUserKey() const { return is_user_key_; } + private: - char* key_; + char* buf_; size_t buf_size_; + const char* key_; size_t key_size_; char space_[32]; // Avoid allocation for short keys + bool is_user_key_; + + Slice SetKeyImpl(const Slice& key, bool copy) { + size_t size = key.size(); + if (copy) { + // Copy key to buf_ + EnlargeBufferIfNeeded(size); + memcpy(buf_, key.data(), size); + key_ = buf_; + } else { + // Update key_ to point to external memory + key_ = key.data(); + } + key_size_ = size; + return Slice(key_, key_size_); + } void ResetBuffer() { - if (key_ != nullptr && key_ != space_) { - delete[] key_; + if (buf_ != space_) { + delete[] buf_; + buf_ = space_; } - key_ = space_; buf_size_ = sizeof(space_); key_size_ = 0; } @@ -350,7 +504,7 @@ class IterKey { if (key_size > buf_size_) { // Need to enlarge the buffer. ResetBuffer(); - key_ = new char[key_size]; + buf_ = new char[key_size]; buf_size_ = key_size; } } @@ -365,19 +519,19 @@ class InternalKeySliceTransform : public SliceTransform { explicit InternalKeySliceTransform(const SliceTransform* transform) : transform_(transform) {} - virtual const char* Name() const { return transform_->Name(); } + virtual const char* Name() const override { return transform_->Name(); } - virtual Slice Transform(const Slice& src) const { + virtual Slice Transform(const Slice& src) const override { auto user_key = ExtractUserKey(src); return transform_->Transform(user_key); } - virtual bool InDomain(const Slice& src) const { + virtual bool InDomain(const Slice& src) const override { auto user_key = ExtractUserKey(src); return transform_->InDomain(user_key); } - virtual bool InRange(const Slice& dst) const { + virtual bool InRange(const Slice& dst) const override { auto user_key = ExtractUserKey(dst); return transform_->InRange(user_key); } @@ -390,6 +544,12 @@ class InternalKeySliceTransform : public SliceTransform { const SliceTransform* const transform_; }; +// Read the key of a record from a write batch. +// if this record represent the default column family then cf_record +// must be passed as false, otherwise it must be passed as true. +extern bool ReadKeyFromWriteBatchEntry(Slice* input, Slice* key, + bool cf_record); + // Read record from a write batch piece from input. // tag, column_family, key, value and blob are return values. Callers own the // Slice they point to. @@ -397,5 +557,42 @@ class InternalKeySliceTransform : public SliceTransform { // input will be advanced to after the record. extern Status ReadRecordFromWriteBatch(Slice* input, char* tag, uint32_t* column_family, Slice* key, - Slice* value, Slice* blob); + Slice* value, Slice* blob, Slice* xid); + +// When user call DeleteRange() to delete a range of keys, +// we will store a serialized RangeTombstone in MemTable and SST. +// the struct here is a easy-understood form +// start/end_key_ is the start/end user key of the range to be deleted +struct RangeTombstone { + Slice start_key_; + Slice end_key_; + SequenceNumber seq_; + RangeTombstone() = default; + RangeTombstone(Slice sk, Slice ek, SequenceNumber sn) + : start_key_(sk), end_key_(ek), seq_(sn) {} + + RangeTombstone(ParsedInternalKey parsed_key, Slice value) { + start_key_ = parsed_key.user_key; + seq_ = parsed_key.sequence; + end_key_ = value; + } + + // be careful to use Serialize(), allocates new memory + std::pair Serialize() const { + auto key = InternalKey(start_key_, seq_, kTypeRangeDeletion); + Slice value = end_key_; + return std::make_pair(std::move(key), std::move(value)); + } + + // be careful to use SerializeKey(), allocates new memory + InternalKey SerializeKey() const { + return InternalKey(start_key_, seq_, kTypeRangeDeletion); + } + + // be careful to use SerializeEndKey(), allocates new memory + InternalKey SerializeEndKey() const { + return InternalKey(end_key_, seq_, kTypeRangeDeletion); + } +}; + } // namespace rocksdb diff --git a/db/dbformat_test.cc b/db/dbformat_test.cc index 332d7723cfd..d96b5757afd 100644 --- a/db/dbformat_test.cc +++ b/db/dbformat_test.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -49,9 +49,9 @@ static void TestKey(const std::string& key, ASSERT_TRUE(!ParseInternalKey(Slice("bar"), &decoded)); } -class FormatTest { }; +class FormatTest : public testing::Test {}; -TEST(FormatTest, InternalKey_EncodeDecode) { +TEST_F(FormatTest, InternalKey_EncodeDecode) { const char* keys[] = { "", "k", "hello", "longggggggggggggggggggggg" }; const uint64_t seq[] = { 1, 2, 3, @@ -67,7 +67,7 @@ TEST(FormatTest, InternalKey_EncodeDecode) { } } -TEST(FormatTest, InternalKeyShortSeparator) { +TEST_F(FormatTest, InternalKeyShortSeparator) { // When user keys are same ASSERT_EQ(IKey("foo", 100, kTypeValue), Shorten(IKey("foo", 100, kTypeValue), @@ -92,6 +92,30 @@ TEST(FormatTest, InternalKeyShortSeparator) { Shorten(IKey("foo", 100, kTypeValue), IKey("hello", 200, kTypeValue))); + ASSERT_EQ(IKey("ABC2", kMaxSequenceNumber, kValueTypeForSeek), + Shorten(IKey("ABC1AAAAA", 100, kTypeValue), + IKey("ABC2ABB", 200, kTypeValue))); + + ASSERT_EQ(IKey("AAA2", kMaxSequenceNumber, kValueTypeForSeek), + Shorten(IKey("AAA1AAA", 100, kTypeValue), + IKey("AAA2AA", 200, kTypeValue))); + + ASSERT_EQ( + IKey("AAA2", kMaxSequenceNumber, kValueTypeForSeek), + Shorten(IKey("AAA1AAA", 100, kTypeValue), IKey("AAA4", 200, kTypeValue))); + + ASSERT_EQ( + IKey("AAA1B", kMaxSequenceNumber, kValueTypeForSeek), + Shorten(IKey("AAA1AAA", 100, kTypeValue), IKey("AAA2", 200, kTypeValue))); + + ASSERT_EQ(IKey("AAA2", kMaxSequenceNumber, kValueTypeForSeek), + Shorten(IKey("AAA1AAA", 100, kTypeValue), + IKey("AAA2A", 200, kTypeValue))); + + ASSERT_EQ( + IKey("AAA1", 100, kTypeValue), + Shorten(IKey("AAA1", 100, kTypeValue), IKey("AAA2", 200, kTypeValue))); + // When start user key is prefix of limit user key ASSERT_EQ(IKey("foo", 100, kTypeValue), Shorten(IKey("foo", 100, kTypeValue), @@ -103,54 +127,74 @@ TEST(FormatTest, InternalKeyShortSeparator) { IKey("foo", 200, kTypeValue))); } -TEST(FormatTest, InternalKeyShortestSuccessor) { +TEST_F(FormatTest, InternalKeyShortestSuccessor) { ASSERT_EQ(IKey("g", kMaxSequenceNumber, kValueTypeForSeek), ShortSuccessor(IKey("foo", 100, kTypeValue))); ASSERT_EQ(IKey("\xff\xff", 100, kTypeValue), ShortSuccessor(IKey("\xff\xff", 100, kTypeValue))); } -TEST(FormatTest, IterKeyOperation) { +TEST_F(FormatTest, IterKeyOperation) { IterKey k; const char p[] = "abcdefghijklmnopqrstuvwxyz"; const char q[] = "0123456789"; - ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()), + ASSERT_EQ(std::string(k.GetUserKey().data(), k.GetUserKey().size()), std::string("")); k.TrimAppend(0, p, 3); - ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()), + ASSERT_EQ(std::string(k.GetUserKey().data(), k.GetUserKey().size()), std::string("abc")); k.TrimAppend(1, p, 3); - ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()), + ASSERT_EQ(std::string(k.GetUserKey().data(), k.GetUserKey().size()), std::string("aabc")); k.TrimAppend(0, p, 26); - ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()), + ASSERT_EQ(std::string(k.GetUserKey().data(), k.GetUserKey().size()), std::string("abcdefghijklmnopqrstuvwxyz")); k.TrimAppend(26, q, 10); - ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()), + ASSERT_EQ(std::string(k.GetUserKey().data(), k.GetUserKey().size()), std::string("abcdefghijklmnopqrstuvwxyz0123456789")); k.TrimAppend(36, q, 1); - ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()), + ASSERT_EQ(std::string(k.GetUserKey().data(), k.GetUserKey().size()), std::string("abcdefghijklmnopqrstuvwxyz01234567890")); k.TrimAppend(26, q, 1); - ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()), + ASSERT_EQ(std::string(k.GetUserKey().data(), k.GetUserKey().size()), std::string("abcdefghijklmnopqrstuvwxyz0")); // Size going up, memory allocation is triggered k.TrimAppend(27, p, 26); - ASSERT_EQ(std::string(k.GetKey().data(), k.GetKey().size()), + ASSERT_EQ(std::string(k.GetUserKey().data(), k.GetUserKey().size()), std::string("abcdefghijklmnopqrstuvwxyz0" - "abcdefghijklmnopqrstuvwxyz")); + "abcdefghijklmnopqrstuvwxyz")); +} + +TEST_F(FormatTest, UpdateInternalKey) { + std::string user_key("abcdefghijklmnopqrstuvwxyz"); + uint64_t new_seq = 0x123456; + ValueType new_val_type = kTypeDeletion; + + std::string ikey; + AppendInternalKey(&ikey, ParsedInternalKey(user_key, 100U, kTypeValue)); + size_t ikey_size = ikey.size(); + UpdateInternalKey(&ikey, new_seq, new_val_type); + ASSERT_EQ(ikey_size, ikey.size()); + + Slice in(ikey); + ParsedInternalKey decoded; + ASSERT_TRUE(ParseInternalKey(in, &decoded)); + ASSERT_EQ(user_key, decoded.user_key.ToString()); + ASSERT_EQ(new_seq, decoded.sequence); + ASSERT_EQ(new_val_type, decoded.type); } } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/deletefile_test.cc b/db/deletefile_test.cc index 14f0324c172..989c0c4118b 100644 --- a/db/deletefile_test.cc +++ b/db/deletefile_test.cc @@ -1,29 +1,33 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. -#include "rocksdb/db.h" +#ifndef ROCKSDB_LITE + +#include +#include +#include +#include #include "db/db_impl.h" -#include "db/filename.h" #include "db/version_set.h" #include "db/write_batch_internal.h" -#include "util/testharness.h" -#include "util/testutil.h" +#include "rocksdb/db.h" #include "rocksdb/env.h" #include "rocksdb/transaction_log.h" -#include -#include -#include -#include +#include "util/filename.h" +#include "util/string_util.h" +#include "util/sync_point.h" +#include "util/testharness.h" +#include "util/testutil.h" namespace rocksdb { -class DeleteFileTest { +class DeleteFileTest : public testing::Test { public: std::string dbname_; Options options_; @@ -34,6 +38,8 @@ class DeleteFileTest { DeleteFileTest() { db_ = nullptr; env_ = Env::Default(); + options_.delete_obsolete_files_period_micros = 0; // always do full purge + options_.enable_thread_tracking = true; options_.write_buffer_size = 1024*1024*1000; options_.target_file_size_base = 1024*1024*1000; options_.max_bytes_for_level_base = 1024*1024*1000; @@ -55,7 +61,7 @@ class DeleteFileTest { DestroyDB(dbname_, options_); numlevels_ = 7; - ASSERT_OK(ReopenDB(true)); + EXPECT_OK(ReopenDB(true)); } Status ReopenDB(bool create) { @@ -70,6 +76,7 @@ class DeleteFileTest { void CloseDB() { delete db_; + db_ = nullptr; } void AddKeys(int numkeys, int startkey = 0) { @@ -77,7 +84,7 @@ class DeleteFileTest { options.sync = false; ReadOptions roptions; for (int i = startkey; i < (numkeys + startkey) ; i++) { - std::string temp = std::to_string(i); + std::string temp = ToString(i); Slice key(temp); Slice value(temp); ASSERT_OK(db_->Put(options, key, value)); @@ -114,10 +121,14 @@ class DeleteFileTest { DBImpl* dbi = reinterpret_cast(db_); ASSERT_OK(dbi->TEST_FlushMemTable()); ASSERT_OK(dbi->TEST_WaitForFlushMemTable()); + for (int i = 0; i < 2; ++i) { + ASSERT_OK(dbi->TEST_CompactRange(i, nullptr, nullptr)); + } AddKeys(50000, 10000); ASSERT_OK(dbi->TEST_FlushMemTable()); ASSERT_OK(dbi->TEST_WaitForFlushMemTable()); + ASSERT_OK(dbi->TEST_CompactRange(0, nullptr, nullptr)); } void CheckFileTypeCounts(std::string& dir, @@ -142,12 +153,20 @@ class DeleteFileTest { ASSERT_EQ(required_manifest, manifest_cnt); } + static void DoSleep(void* arg) { + auto test = reinterpret_cast(arg); + test->env_->SleepForMicroseconds(2 * 1000 * 1000); + } + + // An empty job to guard all jobs are processed + static void GuardFinish(void* arg) { + TEST_SYNC_POINT("DeleteFileTest::GuardFinish"); + } }; -TEST(DeleteFileTest, AddKeysAndQueryLevels) { +TEST_F(DeleteFileTest, AddKeysAndQueryLevels) { CreateTwoLevels(); std::vector metadata; - std::vector keysinlevel; db_->GetLiveFilesMetaData(&metadata); std::string level1file = ""; @@ -191,7 +210,7 @@ TEST(DeleteFileTest, AddKeysAndQueryLevels) { CloseDB(); } -TEST(DeleteFileTest, PurgeObsoleteFilesTest) { +TEST_F(DeleteFileTest, PurgeObsoleteFilesTest) { CreateTwoLevels(); // there should be only one (empty) log file because CreateTwoLevels() // flushes the memtables to disk @@ -199,8 +218,11 @@ TEST(DeleteFileTest, PurgeObsoleteFilesTest) { // 2 ssts, 1 manifest CheckFileTypeCounts(dbname_, 0, 2, 1); std::string first("0"), last("999999"); + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 2; Slice first_slice(first), last_slice(last); - db_->CompactRange(&first_slice, &last_slice, true, 2); + db_->CompactRange(compact_options, &first_slice, &last_slice); // 1 sst after compaction CheckFileTypeCounts(dbname_, 0, 1, 1); @@ -209,7 +231,7 @@ TEST(DeleteFileTest, PurgeObsoleteFilesTest) { Iterator *itr = 0; CreateTwoLevels(); itr = db_->NewIterator(ReadOptions()); - db_->CompactRange(&first_slice, &last_slice, true, 2); + db_->CompactRange(compact_options, &first_slice, &last_slice); // 3 sst after compaction with live iterator CheckFileTypeCounts(dbname_, 0, 3, 1); delete itr; @@ -219,7 +241,119 @@ TEST(DeleteFileTest, PurgeObsoleteFilesTest) { CloseDB(); } -TEST(DeleteFileTest, DeleteFileWithIterator) { +TEST_F(DeleteFileTest, BackgroundPurgeTest) { + std::string first("0"), last("999999"); + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 2; + Slice first_slice(first), last_slice(last); + + // We keep an iterator alive + Iterator* itr = 0; + CreateTwoLevels(); + ReadOptions options; + options.background_purge_on_iterator_cleanup = true; + itr = db_->NewIterator(options); + db_->CompactRange(compact_options, &first_slice, &last_slice); + // 3 sst after compaction with live iterator + CheckFileTypeCounts(dbname_, 0, 3, 1); + test::SleepingBackgroundTask sleeping_task_before; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, + &sleeping_task_before, Env::Priority::HIGH); + delete itr; + test::SleepingBackgroundTask sleeping_task_after; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, + &sleeping_task_after, Env::Priority::HIGH); + + // Make sure no purges are executed foreground + CheckFileTypeCounts(dbname_, 0, 3, 1); + sleeping_task_before.WakeUp(); + sleeping_task_before.WaitUntilDone(); + + // Make sure all background purges are executed + sleeping_task_after.WakeUp(); + sleeping_task_after.WaitUntilDone(); + // 1 sst after iterator deletion + CheckFileTypeCounts(dbname_, 0, 1, 1); + + CloseDB(); +} + +// This test is to reproduce a bug that read invalid ReadOption in iterator +// cleanup function +TEST_F(DeleteFileTest, BackgroundPurgeCopyOptions) { + std::string first("0"), last("999999"); + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 2; + Slice first_slice(first), last_slice(last); + + // We keep an iterator alive + Iterator* itr = 0; + CreateTwoLevels(); + ReadOptions* options = new ReadOptions(); + options->background_purge_on_iterator_cleanup = true; + itr = db_->NewIterator(*options); + // ReadOptions is deleted, but iterator cleanup function should not be + // affected + delete options; + + db_->CompactRange(compact_options, &first_slice, &last_slice); + // 3 sst after compaction with live iterator + CheckFileTypeCounts(dbname_, 0, 3, 1); + delete itr; + + test::SleepingBackgroundTask sleeping_task_after; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, + &sleeping_task_after, Env::Priority::HIGH); + + // Make sure all background purges are executed + sleeping_task_after.WakeUp(); + sleeping_task_after.WaitUntilDone(); + // 1 sst after iterator deletion + CheckFileTypeCounts(dbname_, 0, 1, 1); + + CloseDB(); +} + +TEST_F(DeleteFileTest, BackgroundPurgeTestMultipleJobs) { + std::string first("0"), last("999999"); + CompactRangeOptions compact_options; + compact_options.change_level = true; + compact_options.target_level = 2; + Slice first_slice(first), last_slice(last); + + // We keep an iterator alive + CreateTwoLevels(); + ReadOptions options; + options.background_purge_on_iterator_cleanup = true; + Iterator* itr1 = db_->NewIterator(options); + CreateTwoLevels(); + Iterator* itr2 = db_->NewIterator(options); + db_->CompactRange(compact_options, &first_slice, &last_slice); + // 5 sst files after 2 compactions with 2 live iterators + CheckFileTypeCounts(dbname_, 0, 5, 1); + + // ~DBImpl should wait until all BGWorkPurge are finished + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::~DBImpl:WaitJob", "DBImpl::BGWorkPurge"}, + {"DeleteFileTest::GuardFinish", + "DeleteFileTest::BackgroundPurgeTestMultipleJobs:DBClose"}}); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + delete itr1; + env_->Schedule(&DeleteFileTest::DoSleep, this, Env::Priority::HIGH); + delete itr2; + env_->Schedule(&DeleteFileTest::GuardFinish, nullptr, Env::Priority::HIGH); + CloseDB(); + + TEST_SYNC_POINT("DeleteFileTest::BackgroundPurgeTestMultipleJobs:DBClose"); + // 1 sst after iterator deletion + CheckFileTypeCounts(dbname_, 0, 1, 1); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(DeleteFileTest, DeleteFileWithIterator) { CreateTwoLevels(); ReadOptions options; Iterator* it = db_->NewIterator(options); @@ -250,7 +384,7 @@ TEST(DeleteFileTest, DeleteFileWithIterator) { CloseDB(); } -TEST(DeleteFileTest, DeleteLogFiles) { +TEST_F(DeleteFileTest, DeleteLogFiles) { AddKeys(10, 0); VectorLogPtr logfiles; db_->GetSortedWalFiles(logfiles); @@ -259,11 +393,11 @@ TEST(DeleteFileTest, DeleteLogFiles) { // Should not succeed because live logs are not allowed to be deleted std::unique_ptr alive_log = std::move(logfiles.back()); ASSERT_EQ(alive_log->Type(), kAliveLogFile); - ASSERT_TRUE(env_->FileExists(options_.wal_dir + "/" + alive_log->PathName())); + ASSERT_OK(env_->FileExists(options_.wal_dir + "/" + alive_log->PathName())); fprintf(stdout, "Deleting alive log file %s\n", alive_log->PathName().c_str()); ASSERT_TRUE(!db_->DeleteFile(alive_log->PathName()).ok()); - ASSERT_TRUE(env_->FileExists(options_.wal_dir + "/" + alive_log->PathName())); + ASSERT_OK(env_->FileExists(options_.wal_dir + "/" + alive_log->PathName())); logfiles.clear(); // Call Flush to bring about a new working log file and add more keys @@ -277,19 +411,99 @@ TEST(DeleteFileTest, DeleteLogFiles) { ASSERT_GT(logfiles.size(), 0UL); std::unique_ptr archived_log = std::move(logfiles.front()); ASSERT_EQ(archived_log->Type(), kArchivedLogFile); - ASSERT_TRUE(env_->FileExists(options_.wal_dir + "/" + - archived_log->PathName())); + ASSERT_OK( + env_->FileExists(options_.wal_dir + "/" + archived_log->PathName())); fprintf(stdout, "Deleting archived log file %s\n", archived_log->PathName().c_str()); ASSERT_OK(db_->DeleteFile(archived_log->PathName())); - ASSERT_TRUE(!env_->FileExists(options_.wal_dir + "/" + - archived_log->PathName())); + ASSERT_EQ(Status::NotFound(), env_->FileExists(options_.wal_dir + "/" + + archived_log->PathName())); + CloseDB(); +} + +TEST_F(DeleteFileTest, DeleteNonDefaultColumnFamily) { CloseDB(); + DBOptions db_options; + db_options.create_if_missing = true; + db_options.create_missing_column_families = true; + std::vector column_families; + column_families.emplace_back(); + column_families.emplace_back("new_cf", ColumnFamilyOptions()); + + std::vector handles; + rocksdb::DB* db; + ASSERT_OK(DB::Open(db_options, dbname_, column_families, &handles, &db)); + + Random rnd(5); + for (int i = 0; i < 1000; ++i) { + ASSERT_OK(db->Put(WriteOptions(), handles[1], test::RandomKey(&rnd, 10), + test::RandomKey(&rnd, 10))); + } + ASSERT_OK(db->Flush(FlushOptions(), handles[1])); + for (int i = 0; i < 1000; ++i) { + ASSERT_OK(db->Put(WriteOptions(), handles[1], test::RandomKey(&rnd, 10), + test::RandomKey(&rnd, 10))); + } + ASSERT_OK(db->Flush(FlushOptions(), handles[1])); + + std::vector metadata; + db->GetLiveFilesMetaData(&metadata); + ASSERT_EQ(2U, metadata.size()); + ASSERT_EQ("new_cf", metadata[0].column_family_name); + ASSERT_EQ("new_cf", metadata[1].column_family_name); + auto old_file = metadata[0].smallest_seqno < metadata[1].smallest_seqno + ? metadata[0].name + : metadata[1].name; + auto new_file = metadata[0].smallest_seqno > metadata[1].smallest_seqno + ? metadata[0].name + : metadata[1].name; + ASSERT_TRUE(db->DeleteFile(new_file).IsInvalidArgument()); + ASSERT_OK(db->DeleteFile(old_file)); + + { + std::unique_ptr itr(db->NewIterator(ReadOptions(), handles[1])); + int count = 0; + for (itr->SeekToFirst(); itr->Valid(); itr->Next()) { + ASSERT_OK(itr->status()); + ++count; + } + ASSERT_EQ(count, 1000); + } + + delete handles[0]; + delete handles[1]; + delete db; + + ASSERT_OK(DB::Open(db_options, dbname_, column_families, &handles, &db)); + { + std::unique_ptr itr(db->NewIterator(ReadOptions(), handles[1])); + int count = 0; + for (itr->SeekToFirst(); itr->Valid(); itr->Next()) { + ASSERT_OK(itr->status()); + ++count; + } + ASSERT_EQ(count, 1000); + } + + delete handles[0]; + delete handles[1]; + delete db; } } //namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, + "SKIPPED as DBImpl::DeleteFile is not supported in ROCKSDB_LITE\n"); + return 0; } +#endif // !ROCKSDB_LITE diff --git a/db/event_helpers.cc b/db/event_helpers.cc new file mode 100644 index 00000000000..1b79acb0f2c --- /dev/null +++ b/db/event_helpers.cc @@ -0,0 +1,155 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/event_helpers.h" + +namespace rocksdb { + +namespace { +template +inline T SafeDivide(T a, T b) { + return b == 0 ? 0 : a / b; +} +} // namespace + +void EventHelpers::AppendCurrentTime(JSONWriter* jwriter) { + *jwriter << "time_micros" + << std::chrono::duration_cast( + std::chrono::system_clock::now().time_since_epoch()).count(); +} + +#ifndef ROCKSDB_LITE +void EventHelpers::NotifyTableFileCreationStarted( + const std::vector>& listeners, + const std::string& db_name, const std::string& cf_name, + const std::string& file_path, int job_id, TableFileCreationReason reason) { + TableFileCreationBriefInfo info; + info.db_name = db_name; + info.cf_name = cf_name; + info.file_path = file_path; + info.job_id = job_id; + info.reason = reason; + for (auto& listener : listeners) { + listener->OnTableFileCreationStarted(info); + } +} +#endif // !ROCKSDB_LITE + +void EventHelpers::NotifyOnBackgroundError( + const std::vector>& listeners, + BackgroundErrorReason reason, Status* bg_error, + InstrumentedMutex* db_mutex) { +#ifndef ROCKSDB_LITE + if (listeners.size() == 0U) { + return; + } + db_mutex->AssertHeld(); + // release lock while notifying events + db_mutex->Unlock(); + for (auto& listener : listeners) { + listener->OnBackgroundError(reason, bg_error); + } + db_mutex->Lock(); +#endif // ROCKSDB_LITE +} + +void EventHelpers::LogAndNotifyTableFileCreationFinished( + EventLogger* event_logger, + const std::vector>& listeners, + const std::string& db_name, const std::string& cf_name, + const std::string& file_path, int job_id, const FileDescriptor& fd, + const TableProperties& table_properties, TableFileCreationReason reason, + const Status& s) { + if (s.ok() && event_logger) { + JSONWriter jwriter; + AppendCurrentTime(&jwriter); + jwriter << "cf_name" << cf_name << "job" << job_id << "event" + << "table_file_creation" + << "file_number" << fd.GetNumber() << "file_size" + << fd.GetFileSize(); + + // table_properties + { + jwriter << "table_properties"; + jwriter.StartObject(); + + // basic properties: + jwriter << "data_size" << table_properties.data_size << "index_size" + << table_properties.index_size << "filter_size" + << table_properties.filter_size << "raw_key_size" + << table_properties.raw_key_size << "raw_average_key_size" + << SafeDivide(table_properties.raw_key_size, + table_properties.num_entries) + << "raw_value_size" << table_properties.raw_value_size + << "raw_average_value_size" + << SafeDivide(table_properties.raw_value_size, + table_properties.num_entries) + << "num_data_blocks" << table_properties.num_data_blocks + << "num_entries" << table_properties.num_entries + << "filter_policy_name" << table_properties.filter_policy_name; + + // user collected properties + for (const auto& prop : table_properties.readable_properties) { + jwriter << prop.first << prop.second; + } + jwriter.EndObject(); + } + jwriter.EndObject(); + + event_logger->Log(jwriter); + } + +#ifndef ROCKSDB_LITE + if (listeners.size() == 0) { + return; + } + TableFileCreationInfo info; + info.db_name = db_name; + info.cf_name = cf_name; + info.file_path = file_path; + info.file_size = fd.file_size; + info.job_id = job_id; + info.table_properties = table_properties; + info.reason = reason; + info.status = s; + for (auto& listener : listeners) { + listener->OnTableFileCreated(info); + } +#endif // !ROCKSDB_LITE +} + +void EventHelpers::LogAndNotifyTableFileDeletion( + EventLogger* event_logger, int job_id, + uint64_t file_number, const std::string& file_path, + const Status& status, const std::string& dbname, + const std::vector>& listeners) { + + JSONWriter jwriter; + AppendCurrentTime(&jwriter); + + jwriter << "job" << job_id + << "event" << "table_file_deletion" + << "file_number" << file_number; + if (!status.ok()) { + jwriter << "status" << status.ToString(); + } + + jwriter.EndObject(); + + event_logger->Log(jwriter); + +#ifndef ROCKSDB_LITE + TableFileDeletionInfo info; + info.db_name = dbname; + info.job_id = job_id; + info.file_path = file_path; + info.status = status; + for (auto& listener : listeners) { + listener->OnTableFileDeleted(info); + } +#endif // !ROCKSDB_LITE +} + +} // namespace rocksdb diff --git a/db/event_helpers.h b/db/event_helpers.h new file mode 100644 index 00000000000..674e6c5f6fc --- /dev/null +++ b/db/event_helpers.h @@ -0,0 +1,52 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once + +#include +#include +#include + +#include "db/column_family.h" +#include "db/version_edit.h" +#include "rocksdb/listener.h" +#include "rocksdb/table_properties.h" +#include "util/event_logger.h" + +namespace rocksdb { + +class EventHelpers { + public: + static void AppendCurrentTime(JSONWriter* json_writer); +#ifndef ROCKSDB_LITE + static void NotifyTableFileCreationStarted( + const std::vector>& listeners, + const std::string& db_name, const std::string& cf_name, + const std::string& file_path, int job_id, TableFileCreationReason reason); +#endif // !ROCKSDB_LITE + static void NotifyOnBackgroundError( + const std::vector>& listeners, + BackgroundErrorReason reason, Status* bg_error, + InstrumentedMutex* db_mutex); + static void LogAndNotifyTableFileCreationFinished( + EventLogger* event_logger, + const std::vector>& listeners, + const std::string& db_name, const std::string& cf_name, + const std::string& file_path, int job_id, const FileDescriptor& fd, + const TableProperties& table_properties, TableFileCreationReason reason, + const Status& s); + static void LogAndNotifyTableFileDeletion( + EventLogger* event_logger, int job_id, + uint64_t file_number, const std::string& file_path, + const Status& status, const std::string& db_name, + const std::vector>& listeners); + + private: + static void LogAndNotifyTableFileCreation( + EventLogger* event_logger, + const std::vector>& listeners, + const FileDescriptor& fd, const TableFileCreationInfo& info); +}; + +} // namespace rocksdb diff --git a/db/experimental.cc b/db/experimental.cc new file mode 100644 index 00000000000..effe9d7c355 --- /dev/null +++ b/db/experimental.cc @@ -0,0 +1,49 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "rocksdb/experimental.h" + +#include "db/db_impl.h" + +namespace rocksdb { +namespace experimental { + +#ifndef ROCKSDB_LITE + +Status SuggestCompactRange(DB* db, ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) { + if (db == nullptr) { + return Status::InvalidArgument("DB is empty"); + } + + return db->SuggestCompactRange(column_family, begin, end); +} + +Status PromoteL0(DB* db, ColumnFamilyHandle* column_family, int target_level) { + if (db == nullptr) { + return Status::InvalidArgument("Didn't recognize DB object"); + } + return db->PromoteL0(column_family, target_level); +} + +#else // ROCKSDB_LITE + +Status SuggestCompactRange(DB* db, ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) { + return Status::NotSupported("Not supported in RocksDB LITE"); +} + +Status PromoteL0(DB* db, ColumnFamilyHandle* column_family, int target_level) { + return Status::NotSupported("Not supported in RocksDB LITE"); +} + +#endif // ROCKSDB_LITE + +Status SuggestCompactRange(DB* db, const Slice* begin, const Slice* end) { + return SuggestCompactRange(db, db->DefaultColumnFamily(), begin, end); +} + +} // namespace experimental +} // namespace rocksdb diff --git a/db/external_sst_file_basic_test.cc b/db/external_sst_file_basic_test.cc new file mode 100644 index 00000000000..534e8a0bf76 --- /dev/null +++ b/db/external_sst_file_basic_test.cc @@ -0,0 +1,617 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include + +#include "db/db_test_util.h" +#include "port/port.h" +#include "port/stack_trace.h" +#include "rocksdb/sst_file_writer.h" +#include "util/testutil.h" + +namespace rocksdb { + +#ifndef ROCKSDB_LITE +class ExternalSSTFileBasicTest : public DBTestBase { + public: + ExternalSSTFileBasicTest() : DBTestBase("/external_sst_file_test") { + sst_files_dir_ = dbname_ + "/sst_files/"; + DestroyAndRecreateExternalSSTFilesDir(); + } + + void DestroyAndRecreateExternalSSTFilesDir() { + test::DestroyDir(env_, sst_files_dir_); + env_->CreateDir(sst_files_dir_); + } + + Status DeprecatedAddFile(const std::vector& files, + bool move_files = false, + bool skip_snapshot_check = false) { + IngestExternalFileOptions opts; + opts.move_files = move_files; + opts.snapshot_consistency = !skip_snapshot_check; + opts.allow_global_seqno = false; + opts.allow_blocking_flush = false; + return db_->IngestExternalFile(files, opts); + } + + Status GenerateAndAddExternalFile( + const Options options, std::vector keys, + const std::vector& value_types, int file_id, + std::map* true_data) { + assert(value_types.size() == 1 || keys.size() == value_types.size()); + std::string file_path = sst_files_dir_ + ToString(file_id); + SstFileWriter sst_file_writer(EnvOptions(), options); + + Status s = sst_file_writer.Open(file_path); + if (!s.ok()) { + return s; + } + for (size_t i = 0; i < keys.size(); i++) { + std::string key = Key(keys[i]); + std::string value = Key(keys[i]) + ToString(file_id); + ValueType value_type = + (value_types.size() == 1 ? value_types[0] : value_types[i]); + switch (value_type) { + case ValueType::kTypeValue: + s = sst_file_writer.Put(key, value); + (*true_data)[key] = value; + break; + case ValueType::kTypeMerge: + s = sst_file_writer.Merge(key, value); + // we only use TestPutOperator in this test + (*true_data)[key] = value; + break; + case ValueType::kTypeDeletion: + s = sst_file_writer.Delete(key); + true_data->erase(key); + break; + default: + return Status::InvalidArgument("Value type is not supported"); + } + if (!s.ok()) { + sst_file_writer.Finish(); + return s; + } + } + s = sst_file_writer.Finish(); + + if (s.ok()) { + IngestExternalFileOptions ifo; + ifo.allow_global_seqno = true; + s = db_->IngestExternalFile({file_path}, ifo); + } + return s; + } + + Status GenerateAndAddExternalFile( + const Options options, std::vector keys, const ValueType value_type, + int file_id, std::map* true_data) { + return GenerateAndAddExternalFile(options, keys, + std::vector(1, value_type), + file_id, true_data); + } + + ~ExternalSSTFileBasicTest() { test::DestroyDir(env_, sst_files_dir_); } + + protected: + std::string sst_files_dir_; +}; + +TEST_F(ExternalSSTFileBasicTest, Basic) { + Options options = CurrentOptions(); + + SstFileWriter sst_file_writer(EnvOptions(), options); + + // Current file size should be 0 after sst_file_writer init and before open a + // file. + ASSERT_EQ(sst_file_writer.FileSize(), 0); + + // file1.sst (0 => 99) + std::string file1 = sst_files_dir_ + "file1.sst"; + ASSERT_OK(sst_file_writer.Open(file1)); + for (int k = 0; k < 100; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + ExternalSstFileInfo file1_info; + Status s = sst_file_writer.Finish(&file1_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + + // Current file size should be non-zero after success write. + ASSERT_GT(sst_file_writer.FileSize(), 0); + + ASSERT_EQ(file1_info.file_path, file1); + ASSERT_EQ(file1_info.num_entries, 100); + ASSERT_EQ(file1_info.smallest_key, Key(0)); + ASSERT_EQ(file1_info.largest_key, Key(99)); + // sst_file_writer already finished, cannot add this value + s = sst_file_writer.Put(Key(100), "bad_val"); + ASSERT_FALSE(s.ok()) << s.ToString(); + + DestroyAndReopen(options); + // Add file using file path + s = DeprecatedAddFile({file1}); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(db_->GetLatestSequenceNumber(), 0U); + for (int k = 0; k < 100; k++) { + ASSERT_EQ(Get(Key(k)), Key(k) + "_val"); + } + + DestroyAndRecreateExternalSSTFilesDir(); +} + +TEST_F(ExternalSSTFileBasicTest, NoCopy) { + Options options = CurrentOptions(); + const ImmutableCFOptions ioptions(options); + + SstFileWriter sst_file_writer(EnvOptions(), options); + + // file1.sst (0 => 99) + std::string file1 = sst_files_dir_ + "file1.sst"; + ASSERT_OK(sst_file_writer.Open(file1)); + for (int k = 0; k < 100; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + ExternalSstFileInfo file1_info; + Status s = sst_file_writer.Finish(&file1_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file1_info.file_path, file1); + ASSERT_EQ(file1_info.num_entries, 100); + ASSERT_EQ(file1_info.smallest_key, Key(0)); + ASSERT_EQ(file1_info.largest_key, Key(99)); + + // file2.sst (100 => 299) + std::string file2 = sst_files_dir_ + "file2.sst"; + ASSERT_OK(sst_file_writer.Open(file2)); + for (int k = 100; k < 300; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + ExternalSstFileInfo file2_info; + s = sst_file_writer.Finish(&file2_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file2_info.file_path, file2); + ASSERT_EQ(file2_info.num_entries, 200); + ASSERT_EQ(file2_info.smallest_key, Key(100)); + ASSERT_EQ(file2_info.largest_key, Key(299)); + + // file3.sst (110 => 124) .. overlap with file2.sst + std::string file3 = sst_files_dir_ + "file3.sst"; + ASSERT_OK(sst_file_writer.Open(file3)); + for (int k = 110; k < 125; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val_overlap")); + } + ExternalSstFileInfo file3_info; + s = sst_file_writer.Finish(&file3_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file3_info.file_path, file3); + ASSERT_EQ(file3_info.num_entries, 15); + ASSERT_EQ(file3_info.smallest_key, Key(110)); + ASSERT_EQ(file3_info.largest_key, Key(124)); + s = DeprecatedAddFile({file1}, true /* move file */); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(Status::NotFound(), env_->FileExists(file1)); + + s = DeprecatedAddFile({file2}, false /* copy file */); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_OK(env_->FileExists(file2)); + + // This file have overlapping values with the existing data + s = DeprecatedAddFile({file2}, true /* move file */); + ASSERT_FALSE(s.ok()) << s.ToString(); + ASSERT_OK(env_->FileExists(file3)); + + for (int k = 0; k < 300; k++) { + ASSERT_EQ(Get(Key(k)), Key(k) + "_val"); + } +} + +TEST_F(ExternalSSTFileBasicTest, IngestFileWithGlobalSeqnoPickedSeqno) { + do { + Options options = CurrentOptions(); + DestroyAndReopen(options); + std::map true_data; + + int file_id = 1; + + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 2, 3, 4, 5, 6}, + ValueType::kTypeValue, file_id++, + &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 0); + + ASSERT_OK(GenerateAndAddExternalFile(options, {10, 11, 12, 13}, + ValueType::kTypeValue, file_id++, + + &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 0); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {1, 4, 6}, ValueType::kTypeValue, file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 1); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {11, 15, 19}, ValueType::kTypeValue, file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 2); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {120, 130}, ValueType::kTypeValue, file_id++, &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 2); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {1, 130}, ValueType::kTypeValue, file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 3); + + // Write some keys through normal write path + for (int i = 0; i < 50; i++) { + ASSERT_OK(Put(Key(i), "memtable")); + true_data[Key(i)] = "memtable"; + } + SequenceNumber last_seqno = dbfull()->GetLatestSequenceNumber(); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {60, 61, 62}, ValueType::kTypeValue, file_id++, &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {40, 41, 42}, ValueType::kTypeValue, file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 1); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {20, 30, 40}, ValueType::kTypeValue, file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 2); + + const Snapshot* snapshot = db_->GetSnapshot(); + + // We will need a seqno for the file regardless if the file overwrite + // keys in the DB or not because we have a snapshot + ASSERT_OK(GenerateAndAddExternalFile( + options, {1000, 1002}, ValueType::kTypeValue, file_id++, &true_data)); + // A global seqno will be assigned anyway because of the snapshot + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 3); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {2000, 3002}, ValueType::kTypeValue, file_id++, &true_data)); + // A global seqno will be assigned anyway because of the snapshot + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 4); + + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 20, 40, 100, 150}, + ValueType::kTypeValue, file_id++, + &true_data)); + // A global seqno will be assigned anyway because of the snapshot + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 5); + + db_->ReleaseSnapshot(snapshot); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {5000, 5001}, ValueType::kTypeValue, file_id++, &true_data)); + // No snapshot anymore, no need to assign a seqno + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 5); + + size_t kcnt = 0; + VerifyDBFromMap(true_data, &kcnt, false); + } while (ChangeCompactOptions()); +} + +TEST_F(ExternalSSTFileBasicTest, IngestFileWithMultipleValueType) { + do { + Options options = CurrentOptions(); + options.merge_operator.reset(new TestPutOperator()); + DestroyAndReopen(options); + std::map true_data; + + int file_id = 1; + + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 2, 3, 4, 5, 6}, + ValueType::kTypeValue, file_id++, + &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 0); + + ASSERT_OK(GenerateAndAddExternalFile(options, {10, 11, 12, 13}, + ValueType::kTypeValue, file_id++, + + &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 0); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {1, 4, 6}, ValueType::kTypeMerge, file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 1); + + ASSERT_OK(GenerateAndAddExternalFile(options, {11, 15, 19}, + ValueType::kTypeDeletion, file_id++, + &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 2); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {120, 130}, ValueType::kTypeMerge, file_id++, &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 2); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {1, 130}, ValueType::kTypeDeletion, file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 3); + + // Write some keys through normal write path + for (int i = 0; i < 50; i++) { + ASSERT_OK(Put(Key(i), "memtable")); + true_data[Key(i)] = "memtable"; + } + SequenceNumber last_seqno = dbfull()->GetLatestSequenceNumber(); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {60, 61, 62}, ValueType::kTypeValue, file_id++, &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {40, 41, 42}, ValueType::kTypeMerge, file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 1); + + ASSERT_OK(GenerateAndAddExternalFile(options, {20, 30, 40}, + ValueType::kTypeDeletion, file_id++, + &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 2); + + const Snapshot* snapshot = db_->GetSnapshot(); + + // We will need a seqno for the file regardless if the file overwrite + // keys in the DB or not because we have a snapshot + ASSERT_OK(GenerateAndAddExternalFile( + options, {1000, 1002}, ValueType::kTypeMerge, file_id++, &true_data)); + // A global seqno will be assigned anyway because of the snapshot + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 3); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {2000, 3002}, ValueType::kTypeMerge, file_id++, &true_data)); + // A global seqno will be assigned anyway because of the snapshot + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 4); + + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 20, 40, 100, 150}, + ValueType::kTypeMerge, file_id++, + &true_data)); + // A global seqno will be assigned anyway because of the snapshot + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 5); + + db_->ReleaseSnapshot(snapshot); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {5000, 5001}, ValueType::kTypeValue, file_id++, &true_data)); + // No snapshot anymore, no need to assign a seqno + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 5); + + size_t kcnt = 0; + VerifyDBFromMap(true_data, &kcnt, false); + } while (ChangeCompactOptions()); +} + +TEST_F(ExternalSSTFileBasicTest, IngestFileWithMixedValueType) { + do { + Options options = CurrentOptions(); + options.merge_operator.reset(new TestPutOperator()); + DestroyAndReopen(options); + std::map true_data; + + int file_id = 1; + + ASSERT_OK(GenerateAndAddExternalFile( + options, {1, 2, 3, 4, 5, 6}, + {ValueType::kTypeValue, ValueType::kTypeMerge, ValueType::kTypeValue, + ValueType::kTypeMerge, ValueType::kTypeValue, ValueType::kTypeMerge}, + file_id++, &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 0); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {10, 11, 12, 13}, + {ValueType::kTypeValue, ValueType::kTypeMerge, ValueType::kTypeValue, + ValueType::kTypeMerge}, + file_id++, &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 0); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {1, 4, 6}, {ValueType::kTypeDeletion, ValueType::kTypeValue, + ValueType::kTypeMerge}, + file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 1); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {11, 15, 19}, {ValueType::kTypeDeletion, ValueType::kTypeMerge, + ValueType::kTypeValue}, + file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 2); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {120, 130}, {ValueType::kTypeValue, ValueType::kTypeMerge}, + file_id++, &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 2); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {1, 130}, {ValueType::kTypeMerge, ValueType::kTypeDeletion}, + file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), 3); + + // Write some keys through normal write path + for (int i = 0; i < 50; i++) { + ASSERT_OK(Put(Key(i), "memtable")); + true_data[Key(i)] = "memtable"; + } + SequenceNumber last_seqno = dbfull()->GetLatestSequenceNumber(); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {60, 61, 62}, + {ValueType::kTypeValue, ValueType::kTypeMerge, ValueType::kTypeValue}, + file_id++, &true_data)); + // File dont overwrite any keys, No seqno needed + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {40, 41, 42}, {ValueType::kTypeValue, ValueType::kTypeDeletion, + ValueType::kTypeDeletion}, + file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 1); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {20, 30, 40}, + {ValueType::kTypeDeletion, ValueType::kTypeDeletion, + ValueType::kTypeDeletion}, + file_id++, &true_data)); + // File overwrite some keys, a seqno will be assigned + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 2); + + const Snapshot* snapshot = db_->GetSnapshot(); + + // We will need a seqno for the file regardless if the file overwrite + // keys in the DB or not because we have a snapshot + ASSERT_OK(GenerateAndAddExternalFile( + options, {1000, 1002}, {ValueType::kTypeValue, ValueType::kTypeMerge}, + file_id++, &true_data)); + // A global seqno will be assigned anyway because of the snapshot + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 3); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {2000, 3002}, {ValueType::kTypeValue, ValueType::kTypeMerge}, + file_id++, &true_data)); + // A global seqno will be assigned anyway because of the snapshot + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 4); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {1, 20, 40, 100, 150}, + {ValueType::kTypeDeletion, ValueType::kTypeDeletion, + ValueType::kTypeValue, ValueType::kTypeMerge, ValueType::kTypeMerge}, + file_id++, &true_data)); + // A global seqno will be assigned anyway because of the snapshot + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 5); + + db_->ReleaseSnapshot(snapshot); + + ASSERT_OK(GenerateAndAddExternalFile( + options, {5000, 5001}, {ValueType::kTypeValue, ValueType::kTypeMerge}, + file_id++, &true_data)); + // No snapshot anymore, no need to assign a seqno + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno + 5); + + size_t kcnt = 0; + VerifyDBFromMap(true_data, &kcnt, false); + } while (ChangeCompactOptions()); +} + +TEST_F(ExternalSSTFileBasicTest, FadviseTrigger) { + Options options = CurrentOptions(); + const int kNumKeys = 10000; + + size_t total_fadvised_bytes = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "SstFileWriter::Rep::InvalidatePageCache", [&](void* arg) { + size_t fadvise_size = *(reinterpret_cast(arg)); + total_fadvised_bytes += fadvise_size; + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + std::unique_ptr sst_file_writer; + + std::string sst_file_path = sst_files_dir_ + "file_fadvise_disable.sst"; + sst_file_writer.reset( + new SstFileWriter(EnvOptions(), options, nullptr, false)); + ASSERT_OK(sst_file_writer->Open(sst_file_path)); + for (int i = 0; i < kNumKeys; i++) { + ASSERT_OK(sst_file_writer->Put(Key(i), Key(i))); + } + ASSERT_OK(sst_file_writer->Finish()); + // fadvise disabled + ASSERT_EQ(total_fadvised_bytes, 0); + + sst_file_path = sst_files_dir_ + "file_fadvise_enable.sst"; + sst_file_writer.reset( + new SstFileWriter(EnvOptions(), options, nullptr, true)); + ASSERT_OK(sst_file_writer->Open(sst_file_path)); + for (int i = 0; i < kNumKeys; i++) { + ASSERT_OK(sst_file_writer->Put(Key(i), Key(i))); + } + ASSERT_OK(sst_file_writer->Finish()); + // fadvise enabled + ASSERT_EQ(total_fadvised_bytes, sst_file_writer->FileSize()); + ASSERT_GT(total_fadvised_bytes, 0); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(ExternalSSTFileBasicTest, IngestionWithRangeDeletions) { + Options options = CurrentOptions(); + options.disable_auto_compactions = true; + Reopen(options); + + std::map true_data; + int file_id = 1; + // prevent range deletions from being dropped due to becoming obsolete. + const Snapshot* snapshot = db_->GetSnapshot(); + + // range del [0, 50) in L0 file, [50, 100) in memtable + for (int i = 0; i < 2; i++) { + if (i == 1) { + db_->Flush(FlushOptions()); + } + ASSERT_OK(db_->DeleteRange(WriteOptions(), db_->DefaultColumnFamily(), + Key(50 * i), Key(50 * (i + 1)))); + } + ASSERT_EQ(1, NumTableFilesAtLevel(0)); + + // overlaps with L0 file but not memtable, so flush is skipped + SequenceNumber last_seqno = dbfull()->GetLatestSequenceNumber(); + ASSERT_OK(GenerateAndAddExternalFile( + options, {10, 40}, {ValueType::kTypeValue, ValueType::kTypeValue}, + file_id++, &true_data)); + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), ++last_seqno); + ASSERT_EQ(2, NumTableFilesAtLevel(0)); + + // overlaps with memtable, so flush is triggered (thus file count increases by + // two at this step). + ASSERT_OK(GenerateAndAddExternalFile( + options, {50, 90}, {ValueType::kTypeValue, ValueType::kTypeValue}, + file_id++, &true_data)); + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), ++last_seqno); + ASSERT_EQ(4, NumTableFilesAtLevel(0)); + + // snapshot unneeded now that both range deletions are persisted + db_->ReleaseSnapshot(snapshot); + + // overlaps with nothing, so places at bottom level and skips incrementing + // seqnum. + ASSERT_OK(GenerateAndAddExternalFile( + options, {101, 125}, {ValueType::kTypeValue, ValueType::kTypeValue}, + file_id++, &true_data)); + ASSERT_EQ(dbfull()->GetLatestSequenceNumber(), last_seqno); + ASSERT_EQ(4, NumTableFilesAtLevel(0)); + ASSERT_EQ(1, NumTableFilesAtLevel(options.num_levels - 1)); +} + +#endif // ROCKSDB_LITE + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/external_sst_file_ingestion_job.cc b/db/external_sst_file_ingestion_job.cc new file mode 100644 index 00000000000..58fa354463a --- /dev/null +++ b/db/external_sst_file_ingestion_job.cc @@ -0,0 +1,665 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include "db/external_sst_file_ingestion_job.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include +#include +#include + +#include "db/version_edit.h" +#include "table/merging_iterator.h" +#include "table/scoped_arena_iterator.h" +#include "table/sst_file_writer_collectors.h" +#include "table/table_builder.h" +#include "util/file_reader_writer.h" +#include "util/file_util.h" +#include "util/stop_watch.h" +#include "util/sync_point.h" + +namespace rocksdb { + +Status ExternalSstFileIngestionJob::Prepare( + const std::vector& external_files_paths) { + Status status; + + // Read the information of files we are ingesting + for (const std::string& file_path : external_files_paths) { + IngestedFileInfo file_to_ingest; + status = GetIngestedFileInfo(file_path, &file_to_ingest); + if (!status.ok()) { + return status; + } + files_to_ingest_.push_back(file_to_ingest); + } + + for (const IngestedFileInfo& f : files_to_ingest_) { + if (f.cf_id != + TablePropertiesCollectorFactory::Context::kUnknownColumnFamily && + f.cf_id != cfd_->GetID()) { + return Status::InvalidArgument( + "External file column family id dont match"); + } + } + + const Comparator* ucmp = cfd_->internal_comparator().user_comparator(); + auto num_files = files_to_ingest_.size(); + if (num_files == 0) { + return Status::InvalidArgument("The list of files is empty"); + } else if (num_files > 1) { + // Verify that passed files dont have overlapping ranges + autovector sorted_files; + for (size_t i = 0; i < num_files; i++) { + sorted_files.push_back(&files_to_ingest_[i]); + } + + std::sort( + sorted_files.begin(), sorted_files.end(), + [&ucmp](const IngestedFileInfo* info1, const IngestedFileInfo* info2) { + return ucmp->Compare(info1->smallest_user_key, + info2->smallest_user_key) < 0; + }); + + for (size_t i = 0; i < num_files - 1; i++) { + if (ucmp->Compare(sorted_files[i]->largest_user_key, + sorted_files[i + 1]->smallest_user_key) >= 0) { + return Status::NotSupported("Files have overlapping ranges"); + } + } + } + + for (IngestedFileInfo& f : files_to_ingest_) { + if (f.num_entries == 0) { + return Status::InvalidArgument("File contain no entries"); + } + + if (!f.smallest_internal_key().Valid() || + !f.largest_internal_key().Valid()) { + return Status::Corruption("Generated table have corrupted keys"); + } + } + + // Copy/Move external files into DB + for (IngestedFileInfo& f : files_to_ingest_) { + f.fd = FileDescriptor(versions_->NewFileNumber(), 0, f.file_size); + + const std::string path_outside_db = f.external_file_path; + const std::string path_inside_db = + TableFileName(db_options_.db_paths, f.fd.GetNumber(), f.fd.GetPathId()); + + if (ingestion_options_.move_files) { + status = env_->LinkFile(path_outside_db, path_inside_db); + if (status.IsNotSupported()) { + // Original file is on a different FS, use copy instead of hard linking + status = CopyFile(env_, path_outside_db, path_inside_db, 0, + db_options_.use_fsync); + } + } else { + status = CopyFile(env_, path_outside_db, path_inside_db, 0, + db_options_.use_fsync); + } + TEST_SYNC_POINT("DBImpl::AddFile:FileCopied"); + if (!status.ok()) { + break; + } + f.internal_file_path = path_inside_db; + } + + if (!status.ok()) { + // We failed, remove all files that we copied into the db + for (IngestedFileInfo& f : files_to_ingest_) { + if (f.internal_file_path == "") { + break; + } + Status s = env_->DeleteFile(f.internal_file_path); + if (!s.ok()) { + ROCKS_LOG_WARN(db_options_.info_log, + "AddFile() clean up for file %s failed : %s", + f.internal_file_path.c_str(), s.ToString().c_str()); + } + } + } + + return status; +} + +Status ExternalSstFileIngestionJob::NeedsFlush(bool* flush_needed) { + SuperVersion* super_version = cfd_->GetSuperVersion(); + Status status = + IngestedFilesOverlapWithMemtables(super_version, flush_needed); + + if (status.ok() && *flush_needed && + !ingestion_options_.allow_blocking_flush) { + status = Status::InvalidArgument("External file requires flush"); + } + return status; +} + +// REQUIRES: we have become the only writer by entering both write_thread_ and +// nonmem_write_thread_ +Status ExternalSstFileIngestionJob::Run() { + Status status; +#ifndef NDEBUG + // We should never run the job with a memtable that is overlapping + // with the files we are ingesting + bool need_flush = false; + status = NeedsFlush(&need_flush); + assert(status.ok() && need_flush == false); +#endif + + bool consumed_seqno = false; + bool force_global_seqno = false; + + if (ingestion_options_.snapshot_consistency && !db_snapshots_->empty()) { + // We need to assign a global sequence number to all the files even + // if the dont overlap with any ranges since we have snapshots + force_global_seqno = true; + } + // It is safe to use this instead of LastToBeWrittenSequence since we are + // the only active writer, and hence they are equal + const SequenceNumber last_seqno = versions_->LastSequence(); + SuperVersion* super_version = cfd_->GetSuperVersion(); + edit_.SetColumnFamily(cfd_->GetID()); + // The levels that the files will be ingested into + + for (IngestedFileInfo& f : files_to_ingest_) { + SequenceNumber assigned_seqno = 0; + if (ingestion_options_.ingest_behind) { + status = CheckLevelForIngestedBehindFile(&f); + } else { + status = AssignLevelAndSeqnoForIngestedFile( + super_version, force_global_seqno, cfd_->ioptions()->compaction_style, + &f, &assigned_seqno); + } + if (!status.ok()) { + return status; + } + status = AssignGlobalSeqnoForIngestedFile(&f, assigned_seqno); + TEST_SYNC_POINT_CALLBACK("ExternalSstFileIngestionJob::Run", + &assigned_seqno); + if (assigned_seqno == last_seqno + 1) { + consumed_seqno = true; + } + if (!status.ok()) { + return status; + } + edit_.AddFile(f.picked_level, f.fd.GetNumber(), f.fd.GetPathId(), + f.fd.GetFileSize(), f.smallest_internal_key(), + f.largest_internal_key(), f.assigned_seqno, f.assigned_seqno, + false); + } + + if (consumed_seqno) { + versions_->SetLastToBeWrittenSequence(last_seqno + 1); + versions_->SetLastSequence(last_seqno + 1); + } + + return status; +} + +void ExternalSstFileIngestionJob::UpdateStats() { + // Update internal stats for new ingested files + uint64_t total_keys = 0; + uint64_t total_l0_files = 0; + uint64_t total_time = env_->NowMicros() - job_start_time_; + for (IngestedFileInfo& f : files_to_ingest_) { + InternalStats::CompactionStats stats(1); + stats.micros = total_time; + stats.bytes_written = f.fd.GetFileSize(); + stats.num_output_files = 1; + cfd_->internal_stats()->AddCompactionStats(f.picked_level, stats); + cfd_->internal_stats()->AddCFStats(InternalStats::BYTES_INGESTED_ADD_FILE, + f.fd.GetFileSize()); + total_keys += f.num_entries; + if (f.picked_level == 0) { + total_l0_files += 1; + } + ROCKS_LOG_INFO( + db_options_.info_log, + "[AddFile] External SST file %s was ingested in L%d with path %s " + "(global_seqno=%" PRIu64 ")\n", + f.external_file_path.c_str(), f.picked_level, + f.internal_file_path.c_str(), f.assigned_seqno); + } + cfd_->internal_stats()->AddCFStats(InternalStats::INGESTED_NUM_KEYS_TOTAL, + total_keys); + cfd_->internal_stats()->AddCFStats(InternalStats::INGESTED_NUM_FILES_TOTAL, + files_to_ingest_.size()); + cfd_->internal_stats()->AddCFStats( + InternalStats::INGESTED_LEVEL0_NUM_FILES_TOTAL, total_l0_files); +} + +void ExternalSstFileIngestionJob::Cleanup(const Status& status) { + if (!status.ok()) { + // We failed to add the files to the database + // remove all the files we copied + for (IngestedFileInfo& f : files_to_ingest_) { + Status s = env_->DeleteFile(f.internal_file_path); + if (!s.ok()) { + ROCKS_LOG_WARN(db_options_.info_log, + "AddFile() clean up for file %s failed : %s", + f.internal_file_path.c_str(), s.ToString().c_str()); + } + } + } else if (status.ok() && ingestion_options_.move_files) { + // The files were moved and added successfully, remove original file links + for (IngestedFileInfo& f : files_to_ingest_) { + Status s = env_->DeleteFile(f.external_file_path); + if (!s.ok()) { + ROCKS_LOG_WARN( + db_options_.info_log, + "%s was added to DB successfully but failed to remove original " + "file link : %s", + f.external_file_path.c_str(), s.ToString().c_str()); + } + } + } +} + +Status ExternalSstFileIngestionJob::GetIngestedFileInfo( + const std::string& external_file, IngestedFileInfo* file_to_ingest) { + file_to_ingest->external_file_path = external_file; + + // Get external file size + Status status = env_->GetFileSize(external_file, &file_to_ingest->file_size); + if (!status.ok()) { + return status; + } + + // Create TableReader for external file + std::unique_ptr table_reader; + std::unique_ptr sst_file; + std::unique_ptr sst_file_reader; + + status = env_->NewRandomAccessFile(external_file, &sst_file, env_options_); + if (!status.ok()) { + return status; + } + sst_file_reader.reset(new RandomAccessFileReader(std::move(sst_file), + external_file)); + + status = cfd_->ioptions()->table_factory->NewTableReader( + TableReaderOptions(*cfd_->ioptions(), env_options_, + cfd_->internal_comparator()), + std::move(sst_file_reader), file_to_ingest->file_size, &table_reader); + if (!status.ok()) { + return status; + } + + // Get the external file properties + auto props = table_reader->GetTableProperties(); + const auto& uprops = props->user_collected_properties; + + // Get table version + auto version_iter = uprops.find(ExternalSstFilePropertyNames::kVersion); + if (version_iter == uprops.end()) { + return Status::Corruption("External file version not found"); + } + file_to_ingest->version = DecodeFixed32(version_iter->second.c_str()); + + auto seqno_iter = uprops.find(ExternalSstFilePropertyNames::kGlobalSeqno); + if (file_to_ingest->version == 2) { + // version 2 imply that we have global sequence number + if (seqno_iter == uprops.end()) { + return Status::Corruption( + "External file global sequence number not found"); + } + + // Set the global sequence number + file_to_ingest->original_seqno = DecodeFixed64(seqno_iter->second.c_str()); + file_to_ingest->global_seqno_offset = props->properties_offsets.at( + ExternalSstFilePropertyNames::kGlobalSeqno); + + if (file_to_ingest->global_seqno_offset == 0) { + return Status::Corruption("Was not able to find file global seqno field"); + } + } else if (file_to_ingest->version == 1) { + // SST file V1 should not have global seqno field + assert(seqno_iter == uprops.end()); + file_to_ingest->original_seqno = 0; + if (ingestion_options_.allow_blocking_flush || + ingestion_options_.allow_global_seqno) { + return Status::InvalidArgument( + "External SST file V1 does not support global seqno"); + } + } else { + return Status::InvalidArgument("External file version is not supported"); + } + // Get number of entries in table + file_to_ingest->num_entries = props->num_entries; + + ParsedInternalKey key; + ReadOptions ro; + // During reading the external file we can cache blocks that we read into + // the block cache, if we later change the global seqno of this file, we will + // have block in cache that will include keys with wrong seqno. + // We need to disable fill_cache so that we read from the file without + // updating the block cache. + ro.fill_cache = false; + std::unique_ptr iter(table_reader->NewIterator(ro)); + + // Get first (smallest) key from file + iter->SeekToFirst(); + if (!ParseInternalKey(iter->key(), &key)) { + return Status::Corruption("external file have corrupted keys"); + } + if (key.sequence != 0) { + return Status::Corruption("external file have non zero sequence number"); + } + file_to_ingest->smallest_user_key = key.user_key.ToString(); + + // Get last (largest) key from file + iter->SeekToLast(); + if (!ParseInternalKey(iter->key(), &key)) { + return Status::Corruption("external file have corrupted keys"); + } + if (key.sequence != 0) { + return Status::Corruption("external file have non zero sequence number"); + } + file_to_ingest->largest_user_key = key.user_key.ToString(); + + file_to_ingest->cf_id = static_cast(props->column_family_id); + + file_to_ingest->table_properties = *props; + + return status; +} + +Status ExternalSstFileIngestionJob::IngestedFilesOverlapWithMemtables( + SuperVersion* sv, bool* overlap) { + // Create an InternalIterator over all memtables + Arena arena; + ReadOptions ro; + ro.total_order_seek = true; + MergeIteratorBuilder merge_iter_builder(&cfd_->internal_comparator(), &arena); + merge_iter_builder.AddIterator(sv->mem->NewIterator(ro, &arena)); + sv->imm->AddIterators(ro, &merge_iter_builder); + ScopedArenaIterator memtable_iter(merge_iter_builder.Finish()); + + std::vector memtable_range_del_iters; + auto* active_range_del_iter = sv->mem->NewRangeTombstoneIterator(ro); + if (active_range_del_iter != nullptr) { + memtable_range_del_iters.push_back(active_range_del_iter); + } + sv->imm->AddRangeTombstoneIterators(ro, &memtable_range_del_iters); + std::unique_ptr memtable_range_del_iter(NewMergingIterator( + &cfd_->internal_comparator(), + memtable_range_del_iters.empty() ? nullptr : &memtable_range_del_iters[0], + static_cast(memtable_range_del_iters.size()))); + + Status status; + *overlap = false; + for (IngestedFileInfo& f : files_to_ingest_) { + status = + IngestedFileOverlapWithIteratorRange(&f, memtable_iter.get(), overlap); + if (!status.ok() || *overlap == true) { + break; + } + status = IngestedFileOverlapWithRangeDeletions( + &f, memtable_range_del_iter.get(), overlap); + if (!status.ok() || *overlap == true) { + break; + } + } + + return status; +} + +Status ExternalSstFileIngestionJob::AssignLevelAndSeqnoForIngestedFile( + SuperVersion* sv, bool force_global_seqno, CompactionStyle compaction_style, + IngestedFileInfo* file_to_ingest, SequenceNumber* assigned_seqno) { + Status status; + *assigned_seqno = 0; + const SequenceNumber last_seqno = versions_->LastSequence(); + if (force_global_seqno) { + *assigned_seqno = last_seqno + 1; + if (compaction_style == kCompactionStyleUniversal) { + file_to_ingest->picked_level = 0; + return status; + } + } + + bool overlap_with_db = false; + Arena arena; + ReadOptions ro; + ro.total_order_seek = true; + int target_level = 0; + auto* vstorage = cfd_->current()->storage_info(); + + for (int lvl = 0; lvl < cfd_->NumberLevels(); lvl++) { + if (lvl > 0 && lvl < vstorage->base_level()) { + continue; + } + + if (vstorage->NumLevelFiles(lvl) > 0) { + bool overlap_with_level = false; + status = IngestedFileOverlapWithLevel(sv, file_to_ingest, lvl, + &overlap_with_level); + if (!status.ok()) { + return status; + } + if (overlap_with_level) { + // We must use L0 or any level higher than `lvl` to be able to overwrite + // the keys that we overlap with in this level, We also need to assign + // this file a seqno to overwrite the existing keys in level `lvl` + overlap_with_db = true; + break; + } + + if (compaction_style == kCompactionStyleUniversal && lvl != 0) { + const std::vector& level_files = + vstorage->LevelFiles(lvl); + const SequenceNumber level_largest_seqno = + (*max_element(level_files.begin(), level_files.end(), + [](FileMetaData* f1, FileMetaData* f2) { + return f1->largest_seqno < f2->largest_seqno; + })) + ->largest_seqno; + if (level_largest_seqno != 0) { + *assigned_seqno = level_largest_seqno; + } else { + continue; + } + } + } else if (compaction_style == kCompactionStyleUniversal) { + continue; + } + + // We dont overlap with any keys in this level, but we still need to check + // if our file can fit in it + if (IngestedFileFitInLevel(file_to_ingest, lvl)) { + target_level = lvl; + } + } + TEST_SYNC_POINT_CALLBACK( + "ExternalSstFileIngestionJob::AssignLevelAndSeqnoForIngestedFile", + &overlap_with_db); + file_to_ingest->picked_level = target_level; + if (overlap_with_db && *assigned_seqno == 0) { + *assigned_seqno = last_seqno + 1; + } + return status; +} + +Status ExternalSstFileIngestionJob::CheckLevelForIngestedBehindFile( + IngestedFileInfo* file_to_ingest) { + auto* vstorage = cfd_->current()->storage_info(); + // first check if new files fit in the bottommost level + int bottom_lvl = cfd_->NumberLevels() - 1; + if(!IngestedFileFitInLevel(file_to_ingest, bottom_lvl)) { + return Status::InvalidArgument( + "Can't ingest_behind file as it doesn't fit " + "at the bottommost level!"); + } + + // second check if despite allow_ingest_behind=true we still have 0 seqnums + // at some upper level + for (int lvl = 0; lvl < cfd_->NumberLevels() - 1; lvl++) { + for (auto file : vstorage->LevelFiles(lvl)) { + if (file->smallest_seqno == 0) { + return Status::InvalidArgument( + "Can't ingest_behind file as despite allow_ingest_behind=true " + "there are files with 0 seqno in database at upper levels!"); + } + } + } + + file_to_ingest->picked_level = bottom_lvl; + return Status::OK(); +} + +Status ExternalSstFileIngestionJob::AssignGlobalSeqnoForIngestedFile( + IngestedFileInfo* file_to_ingest, SequenceNumber seqno) { + if (file_to_ingest->original_seqno == seqno) { + // This file already have the correct global seqno + return Status::OK(); + } else if (!ingestion_options_.allow_global_seqno) { + return Status::InvalidArgument("Global seqno is required, but disabled"); + } else if (file_to_ingest->global_seqno_offset == 0) { + return Status::InvalidArgument( + "Trying to set global seqno for a file that dont have a global seqno " + "field"); + } + + std::unique_ptr rwfile; + Status status = env_->NewRandomRWFile(file_to_ingest->internal_file_path, + &rwfile, env_options_); + if (!status.ok()) { + return status; + } + + // Write the new seqno in the global sequence number field in the file + std::string seqno_val; + PutFixed64(&seqno_val, seqno); + status = rwfile->Write(file_to_ingest->global_seqno_offset, seqno_val); + if (status.ok()) { + file_to_ingest->assigned_seqno = seqno; + } + return status; +} + +Status ExternalSstFileIngestionJob::IngestedFileOverlapWithIteratorRange( + const IngestedFileInfo* file_to_ingest, InternalIterator* iter, + bool* overlap) { + auto* vstorage = cfd_->current()->storage_info(); + auto* ucmp = vstorage->InternalComparator()->user_comparator(); + InternalKey range_start(file_to_ingest->smallest_user_key, kMaxSequenceNumber, + kValueTypeForSeek); + iter->Seek(range_start.Encode()); + if (!iter->status().ok()) { + return iter->status(); + } + + *overlap = false; + if (iter->Valid()) { + ParsedInternalKey seek_result; + if (!ParseInternalKey(iter->key(), &seek_result)) { + return Status::Corruption("DB have corrupted keys"); + } + + if (ucmp->Compare(seek_result.user_key, file_to_ingest->largest_user_key) <= + 0) { + *overlap = true; + } + } + + return iter->status(); +} + +Status ExternalSstFileIngestionJob::IngestedFileOverlapWithRangeDeletions( + const IngestedFileInfo* file_to_ingest, InternalIterator* range_del_iter, + bool* overlap) { + auto* vstorage = cfd_->current()->storage_info(); + auto* ucmp = vstorage->InternalComparator()->user_comparator(); + + *overlap = false; + if (range_del_iter != nullptr) { + for (range_del_iter->SeekToFirst(); range_del_iter->Valid(); + range_del_iter->Next()) { + ParsedInternalKey parsed_key; + if (!ParseInternalKey(range_del_iter->key(), &parsed_key)) { + return Status::Corruption("corrupted range deletion key: " + + range_del_iter->key().ToString()); + } + RangeTombstone range_del(parsed_key, range_del_iter->value()); + if (ucmp->Compare(range_del.start_key_, + file_to_ingest->largest_user_key) <= 0 && + ucmp->Compare(file_to_ingest->smallest_user_key, + range_del.end_key_) <= 0) { + *overlap = true; + break; + } + } + } + return Status::OK(); +} + +bool ExternalSstFileIngestionJob::IngestedFileFitInLevel( + const IngestedFileInfo* file_to_ingest, int level) { + if (level == 0) { + // Files can always fit in L0 + return true; + } + + auto* vstorage = cfd_->current()->storage_info(); + Slice file_smallest_user_key(file_to_ingest->smallest_user_key); + Slice file_largest_user_key(file_to_ingest->largest_user_key); + + if (vstorage->OverlapInLevel(level, &file_smallest_user_key, + &file_largest_user_key)) { + // File overlap with another files in this level, we cannot + // add it to this level + return false; + } + if (cfd_->RangeOverlapWithCompaction(file_smallest_user_key, + file_largest_user_key, level)) { + // File overlap with a running compaction output that will be stored + // in this level, we cannot add this file to this level + return false; + } + + // File did not overlap with level files, our compaction output + return true; +} + +Status ExternalSstFileIngestionJob::IngestedFileOverlapWithLevel( + SuperVersion* sv, IngestedFileInfo* file_to_ingest, int lvl, + bool* overlap_with_level) { + Arena arena; + ReadOptions ro; + ro.total_order_seek = true; + MergeIteratorBuilder merge_iter_builder(&cfd_->internal_comparator(), + &arena); + sv->current->AddIteratorsForLevel(ro, env_options_, &merge_iter_builder, lvl, + nullptr /* range_del_agg */); + ScopedArenaIterator level_iter(merge_iter_builder.Finish()); + + std::vector level_range_del_iters; + sv->current->AddRangeDelIteratorsForLevel(ro, env_options_, lvl, + &level_range_del_iters); + std::unique_ptr level_range_del_iter(NewMergingIterator( + &cfd_->internal_comparator(), + level_range_del_iters.empty() ? nullptr : &level_range_del_iters[0], + static_cast(level_range_del_iters.size()))); + + Status status = IngestedFileOverlapWithIteratorRange( + file_to_ingest, level_iter.get(), overlap_with_level); + if (status.ok() && *overlap_with_level == false) { + status = IngestedFileOverlapWithRangeDeletions( + file_to_ingest, level_range_del_iter.get(), overlap_with_level); + } + return status; +} + +} // namespace rocksdb + +#endif // !ROCKSDB_LITE diff --git a/db/external_sst_file_ingestion_job.h b/db/external_sst_file_ingestion_job.h new file mode 100644 index 00000000000..2d0fadeed79 --- /dev/null +++ b/db/external_sst_file_ingestion_job.h @@ -0,0 +1,171 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#include +#include +#include + +#include "db/column_family.h" +#include "db/dbformat.h" +#include "db/internal_stats.h" +#include "db/snapshot_impl.h" +#include "options/db_options.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/sst_file_writer.h" +#include "util/autovector.h" + +namespace rocksdb { + +struct IngestedFileInfo { + // External file path + std::string external_file_path; + // Smallest user key in external file + std::string smallest_user_key; + // Largest user key in external file + std::string largest_user_key; + // Sequence number for keys in external file + SequenceNumber original_seqno; + // Offset of the global sequence number field in the file, will + // be zero if version is 1 (global seqno is not supported) + size_t global_seqno_offset; + // External file size + uint64_t file_size; + // total number of keys in external file + uint64_t num_entries; + // Id of column family this file shoule be ingested into + uint32_t cf_id; + // TableProperties read from external file + TableProperties table_properties; + // Version of external file + int version; + + // FileDescriptor for the file inside the DB + FileDescriptor fd; + // file path that we picked for file inside the DB + std::string internal_file_path = ""; + // Global sequence number that we picked for the file inside the DB + SequenceNumber assigned_seqno = 0; + // Level inside the DB we picked for the external file. + int picked_level = 0; + + InternalKey smallest_internal_key() const { + return InternalKey(smallest_user_key, assigned_seqno, + ValueType::kTypeValue); + } + + InternalKey largest_internal_key() const { + return InternalKey(largest_user_key, assigned_seqno, ValueType::kTypeValue); + } +}; + +class ExternalSstFileIngestionJob { + public: + ExternalSstFileIngestionJob( + Env* env, VersionSet* versions, ColumnFamilyData* cfd, + const ImmutableDBOptions& db_options, const EnvOptions& env_options, + SnapshotList* db_snapshots, + const IngestExternalFileOptions& ingestion_options) + : env_(env), + versions_(versions), + cfd_(cfd), + db_options_(db_options), + env_options_(env_options), + db_snapshots_(db_snapshots), + ingestion_options_(ingestion_options), + job_start_time_(env_->NowMicros()) {} + + // Prepare the job by copying external files into the DB. + Status Prepare(const std::vector& external_files_paths); + + // Check if we need to flush the memtable before running the ingestion job + // This will be true if the files we are ingesting are overlapping with any + // key range in the memtable. + // REQUIRES: Mutex held + Status NeedsFlush(bool* flush_needed); + + // Will execute the ingestion job and prepare edit() to be applied. + // REQUIRES: Mutex held + Status Run(); + + // Update column family stats. + // REQUIRES: Mutex held + void UpdateStats(); + + // Cleanup after successful/failed job + void Cleanup(const Status& status); + + VersionEdit* edit() { return &edit_; } + + const autovector& files_to_ingest() const { + return files_to_ingest_; + } + + private: + // Open the external file and populate `file_to_ingest` with all the + // external information we need to ingest this file. + Status GetIngestedFileInfo(const std::string& external_file, + IngestedFileInfo* file_to_ingest); + + // Check if the files we are ingesting overlap with any memtable. + // REQUIRES: Mutex held + Status IngestedFilesOverlapWithMemtables(SuperVersion* sv, bool* overlap); + + // Assign `file_to_ingest` the appropriate sequence number and the lowest + // possible level that it can be ingested to according to compaction_style. + // REQUIRES: Mutex held + Status AssignLevelAndSeqnoForIngestedFile(SuperVersion* sv, + bool force_global_seqno, + CompactionStyle compaction_style, + IngestedFileInfo* file_to_ingest, + SequenceNumber* assigned_seqno); + + // File that we want to ingest behind always goes to the lowest level; + // we just check that it fits in the level, that DB allows ingest_behind, + // and that we don't have 0 seqnums at the upper levels. + // REQUIRES: Mutex held + Status CheckLevelForIngestedBehindFile(IngestedFileInfo* file_to_ingest); + + // Set the file global sequence number to `seqno` + Status AssignGlobalSeqnoForIngestedFile(IngestedFileInfo* file_to_ingest, + SequenceNumber seqno); + + // Check if `file_to_ingest` key range overlap with the range `iter` represent + // REQUIRES: Mutex held + Status IngestedFileOverlapWithIteratorRange( + const IngestedFileInfo* file_to_ingest, InternalIterator* iter, + bool* overlap); + + // Check if `file_to_ingest` key range overlaps with any range deletions + // specified by `iter`. + // REQUIRES: Mutex held + Status IngestedFileOverlapWithRangeDeletions( + const IngestedFileInfo* file_to_ingest, InternalIterator* range_del_iter, + bool* overlap); + + // Check if `file_to_ingest` key range overlap with level + // REQUIRES: Mutex held + Status IngestedFileOverlapWithLevel(SuperVersion* sv, + IngestedFileInfo* file_to_ingest, int lvl, bool* overlap_with_level); + + // Check if `file_to_ingest` can fit in level `level` + // REQUIRES: Mutex held + bool IngestedFileFitInLevel(const IngestedFileInfo* file_to_ingest, + int level); + + Env* env_; + VersionSet* versions_; + ColumnFamilyData* cfd_; + const ImmutableDBOptions& db_options_; + const EnvOptions& env_options_; + SnapshotList* db_snapshots_; + autovector files_to_ingest_; + const IngestExternalFileOptions& ingestion_options_; + VersionEdit edit_; + uint64_t job_start_time_; +}; + +} // namespace rocksdb diff --git a/db/external_sst_file_test.cc b/db/external_sst_file_test.cc new file mode 100644 index 00000000000..4a4e82e792d --- /dev/null +++ b/db/external_sst_file_test.cc @@ -0,0 +1,1961 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include +#include "db/db_test_util.h" +#include "port/port.h" +#include "port/stack_trace.h" +#include "rocksdb/sst_file_writer.h" +#include "util/testutil.h" + +namespace rocksdb { + +class ExternalSSTFileTest : public DBTestBase { + public: + ExternalSSTFileTest() : DBTestBase("/external_sst_file_test") { + sst_files_dir_ = dbname_ + "/sst_files/"; + DestroyAndRecreateExternalSSTFilesDir(); + } + + void DestroyAndRecreateExternalSSTFilesDir() { + test::DestroyDir(env_, sst_files_dir_); + env_->CreateDir(sst_files_dir_); + } + + Status GenerateAndAddExternalFile( + const Options options, + std::vector> data, int file_id = -1, + bool allow_global_seqno = false, bool sort_data = false, + std::map* true_data = nullptr, + ColumnFamilyHandle* cfh = nullptr) { + // Generate a file id if not provided + if (file_id == -1) { + file_id = last_file_id_ + 1; + last_file_id_++; + } + + // Sort data if asked to do so + if (sort_data) { + std::sort(data.begin(), data.end(), + [&](const std::pair& e1, + const std::pair& e2) { + return options.comparator->Compare(e1.first, e2.first) < 0; + }); + auto uniq_iter = std::unique( + data.begin(), data.end(), + [&](const std::pair& e1, + const std::pair& e2) { + return options.comparator->Compare(e1.first, e2.first) == 0; + }); + data.resize(uniq_iter - data.begin()); + } + std::string file_path = sst_files_dir_ + ToString(file_id); + SstFileWriter sst_file_writer(EnvOptions(), options, cfh); + + Status s = sst_file_writer.Open(file_path); + if (!s.ok()) { + return s; + } + for (auto& entry : data) { + s = sst_file_writer.Put(entry.first, entry.second); + if (!s.ok()) { + sst_file_writer.Finish(); + return s; + } + } + s = sst_file_writer.Finish(); + + if (s.ok()) { + IngestExternalFileOptions ifo; + ifo.allow_global_seqno = allow_global_seqno; + if (cfh) { + s = db_->IngestExternalFile(cfh, {file_path}, ifo); + } else { + s = db_->IngestExternalFile({file_path}, ifo); + } + } + + if (s.ok() && true_data) { + for (auto& entry : data) { + (*true_data)[entry.first] = entry.second; + } + } + + return s; + } + + Status GenerateAndAddExternalFileIngestBehind( + const Options options, const IngestExternalFileOptions ifo, + std::vector> data, int file_id = -1, + bool sort_data = false, + std::map* true_data = nullptr, + ColumnFamilyHandle* cfh = nullptr) { + // Generate a file id if not provided + if (file_id == -1) { + file_id = last_file_id_ + 1; + last_file_id_++; + } + + // Sort data if asked to do so + if (sort_data) { + std::sort(data.begin(), data.end(), + [&](const std::pair& e1, + const std::pair& e2) { + return options.comparator->Compare(e1.first, e2.first) < 0; + }); + auto uniq_iter = std::unique( + data.begin(), data.end(), + [&](const std::pair& e1, + const std::pair& e2) { + return options.comparator->Compare(e1.first, e2.first) == 0; + }); + data.resize(uniq_iter - data.begin()); + } + std::string file_path = sst_files_dir_ + ToString(file_id); + SstFileWriter sst_file_writer(EnvOptions(), options, cfh); + + Status s = sst_file_writer.Open(file_path); + if (!s.ok()) { + return s; + } + for (auto& entry : data) { + s = sst_file_writer.Put(entry.first, entry.second); + if (!s.ok()) { + sst_file_writer.Finish(); + return s; + } + } + s = sst_file_writer.Finish(); + + if (s.ok()) { + if (cfh) { + s = db_->IngestExternalFile(cfh, {file_path}, ifo); + } else { + s = db_->IngestExternalFile({file_path}, ifo); + } + } + + if (s.ok() && true_data) { + for (auto& entry : data) { + (*true_data)[entry.first] = entry.second; + } + } + + return s; + } + + + + Status GenerateAndAddExternalFile( + const Options options, std::vector> data, + int file_id = -1, bool allow_global_seqno = false, bool sort_data = false, + std::map* true_data = nullptr, + ColumnFamilyHandle* cfh = nullptr) { + std::vector> file_data; + for (auto& entry : data) { + file_data.emplace_back(Key(entry.first), entry.second); + } + return GenerateAndAddExternalFile(options, file_data, file_id, + allow_global_seqno, sort_data, true_data, + cfh); + } + + Status GenerateAndAddExternalFile( + const Options options, std::vector keys, int file_id = -1, + bool allow_global_seqno = false, bool sort_data = false, + std::map* true_data = nullptr, + ColumnFamilyHandle* cfh = nullptr) { + std::vector> file_data; + for (auto& k : keys) { + file_data.emplace_back(Key(k), Key(k) + ToString(file_id)); + } + return GenerateAndAddExternalFile(options, file_data, file_id, + allow_global_seqno, sort_data, true_data, + cfh); + } + + Status DeprecatedAddFile(const std::vector& files, + bool move_files = false, + bool skip_snapshot_check = false) { + IngestExternalFileOptions opts; + opts.move_files = move_files; + opts.snapshot_consistency = !skip_snapshot_check; + opts.allow_global_seqno = false; + opts.allow_blocking_flush = false; + return db_->IngestExternalFile(files, opts); + } + + ~ExternalSSTFileTest() { test::DestroyDir(env_, sst_files_dir_); } + + protected: + int last_file_id_ = 0; + std::string sst_files_dir_; +}; + +TEST_F(ExternalSSTFileTest, Basic) { + do { + Options options = CurrentOptions(); + + SstFileWriter sst_file_writer(EnvOptions(), options); + + // Current file size should be 0 after sst_file_writer init and before open a file. + ASSERT_EQ(sst_file_writer.FileSize(), 0); + + // file1.sst (0 => 99) + std::string file1 = sst_files_dir_ + "file1.sst"; + ASSERT_OK(sst_file_writer.Open(file1)); + for (int k = 0; k < 100; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + ExternalSstFileInfo file1_info; + Status s = sst_file_writer.Finish(&file1_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + + // Current file size should be non-zero after success write. + ASSERT_GT(sst_file_writer.FileSize(), 0); + + ASSERT_EQ(file1_info.file_path, file1); + ASSERT_EQ(file1_info.num_entries, 100); + ASSERT_EQ(file1_info.smallest_key, Key(0)); + ASSERT_EQ(file1_info.largest_key, Key(99)); + // sst_file_writer already finished, cannot add this value + s = sst_file_writer.Put(Key(100), "bad_val"); + ASSERT_FALSE(s.ok()) << s.ToString(); + + // file2.sst (100 => 199) + std::string file2 = sst_files_dir_ + "file2.sst"; + ASSERT_OK(sst_file_writer.Open(file2)); + for (int k = 100; k < 200; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + // Cannot add this key because it's not after last added key + s = sst_file_writer.Put(Key(99), "bad_val"); + ASSERT_FALSE(s.ok()) << s.ToString(); + ExternalSstFileInfo file2_info; + s = sst_file_writer.Finish(&file2_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file2_info.file_path, file2); + ASSERT_EQ(file2_info.num_entries, 100); + ASSERT_EQ(file2_info.smallest_key, Key(100)); + ASSERT_EQ(file2_info.largest_key, Key(199)); + + // file3.sst (195 => 299) + // This file values overlap with file2 values + std::string file3 = sst_files_dir_ + "file3.sst"; + ASSERT_OK(sst_file_writer.Open(file3)); + for (int k = 195; k < 300; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val_overlap")); + } + ExternalSstFileInfo file3_info; + s = sst_file_writer.Finish(&file3_info); + + ASSERT_TRUE(s.ok()) << s.ToString(); + // Current file size should be non-zero after success finish. + ASSERT_GT(sst_file_writer.FileSize(), 0); + ASSERT_EQ(file3_info.file_path, file3); + ASSERT_EQ(file3_info.num_entries, 105); + ASSERT_EQ(file3_info.smallest_key, Key(195)); + ASSERT_EQ(file3_info.largest_key, Key(299)); + + // file4.sst (30 => 39) + // This file values overlap with file1 values + std::string file4 = sst_files_dir_ + "file4.sst"; + ASSERT_OK(sst_file_writer.Open(file4)); + for (int k = 30; k < 40; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val_overlap")); + } + ExternalSstFileInfo file4_info; + s = sst_file_writer.Finish(&file4_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file4_info.file_path, file4); + ASSERT_EQ(file4_info.num_entries, 10); + ASSERT_EQ(file4_info.smallest_key, Key(30)); + ASSERT_EQ(file4_info.largest_key, Key(39)); + + // file5.sst (400 => 499) + std::string file5 = sst_files_dir_ + "file5.sst"; + ASSERT_OK(sst_file_writer.Open(file5)); + for (int k = 400; k < 500; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + ExternalSstFileInfo file5_info; + s = sst_file_writer.Finish(&file5_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file5_info.file_path, file5); + ASSERT_EQ(file5_info.num_entries, 100); + ASSERT_EQ(file5_info.smallest_key, Key(400)); + ASSERT_EQ(file5_info.largest_key, Key(499)); + + // Cannot create an empty sst file + std::string file_empty = sst_files_dir_ + "file_empty.sst"; + ExternalSstFileInfo file_empty_info; + s = sst_file_writer.Finish(&file_empty_info); + ASSERT_NOK(s); + + DestroyAndReopen(options); + // Add file using file path + s = DeprecatedAddFile({file1}); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(db_->GetLatestSequenceNumber(), 0U); + for (int k = 0; k < 100; k++) { + ASSERT_EQ(Get(Key(k)), Key(k) + "_val"); + } + + // Add file while holding a snapshot will fail + const Snapshot* s1 = db_->GetSnapshot(); + if (s1 != nullptr) { + ASSERT_NOK(DeprecatedAddFile({file2})); + db_->ReleaseSnapshot(s1); + } + // We can add the file after releaseing the snapshot + ASSERT_OK(DeprecatedAddFile({file2})); + + ASSERT_EQ(db_->GetLatestSequenceNumber(), 0U); + for (int k = 0; k < 200; k++) { + ASSERT_EQ(Get(Key(k)), Key(k) + "_val"); + } + + // This file has overlapping values with the existing data + s = DeprecatedAddFile({file3}); + ASSERT_FALSE(s.ok()) << s.ToString(); + + // This file has overlapping values with the existing data + s = DeprecatedAddFile({file4}); + ASSERT_FALSE(s.ok()) << s.ToString(); + + // Overwrite values of keys divisible by 5 + for (int k = 0; k < 200; k += 5) { + ASSERT_OK(Put(Key(k), Key(k) + "_val_new")); + } + ASSERT_NE(db_->GetLatestSequenceNumber(), 0U); + + // Key range of file5 (400 => 499) dont overlap with any keys in DB + ASSERT_OK(DeprecatedAddFile({file5})); + + // Make sure values are correct before and after flush/compaction + for (int i = 0; i < 2; i++) { + for (int k = 0; k < 200; k++) { + std::string value = Key(k) + "_val"; + if (k % 5 == 0) { + value += "_new"; + } + ASSERT_EQ(Get(Key(k)), value); + } + for (int k = 400; k < 500; k++) { + std::string value = Key(k) + "_val"; + ASSERT_EQ(Get(Key(k)), value); + } + ASSERT_OK(Flush()); + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + } + + Close(); + options.disable_auto_compactions = true; + Reopen(options); + + // Delete keys in range (400 => 499) + for (int k = 400; k < 500; k++) { + ASSERT_OK(Delete(Key(k))); + } + // We deleted range (400 => 499) but cannot add file5 because + // of the range tombstones + ASSERT_NOK(DeprecatedAddFile({file5})); + + // Compacting the DB will remove the tombstones + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + + // Now we can add the file + ASSERT_OK(DeprecatedAddFile({file5})); + + // Verify values of file5 in DB + for (int k = 400; k < 500; k++) { + std::string value = Key(k) + "_val"; + ASSERT_EQ(Get(Key(k)), value); + } + DestroyAndRecreateExternalSSTFilesDir(); + } while (ChangeOptions(kSkipPlainTable | kSkipFIFOCompaction)); +} +class SstFileWriterCollector : public TablePropertiesCollector { + public: + explicit SstFileWriterCollector(const std::string prefix) : prefix_(prefix) { + name_ = prefix_ + "_SstFileWriterCollector"; + } + + const char* Name() const override { return name_.c_str(); } + + Status Finish(UserCollectedProperties* properties) override { + *properties = UserCollectedProperties{ + {prefix_ + "_SstFileWriterCollector", "YES"}, + {prefix_ + "_Count", std::to_string(count_)}, + }; + return Status::OK(); + } + + Status AddUserKey(const Slice& user_key, const Slice& value, EntryType type, + SequenceNumber seq, uint64_t file_size) override { + ++count_; + return Status::OK(); + } + + virtual UserCollectedProperties GetReadableProperties() const override { + return UserCollectedProperties{}; + } + + private: + uint32_t count_ = 0; + std::string prefix_; + std::string name_; +}; + +class SstFileWriterCollectorFactory : public TablePropertiesCollectorFactory { + public: + explicit SstFileWriterCollectorFactory(std::string prefix) + : prefix_(prefix), num_created_(0) {} + virtual TablePropertiesCollector* CreateTablePropertiesCollector( + TablePropertiesCollectorFactory::Context context) override { + num_created_++; + return new SstFileWriterCollector(prefix_); + } + const char* Name() const override { return "SstFileWriterCollectorFactory"; } + + std::string prefix_; + uint32_t num_created_; +}; + +TEST_F(ExternalSSTFileTest, AddList) { + do { + Options options = CurrentOptions(); + + auto abc_collector = std::make_shared("abc"); + auto xyz_collector = std::make_shared("xyz"); + + options.table_properties_collector_factories.emplace_back(abc_collector); + options.table_properties_collector_factories.emplace_back(xyz_collector); + + SstFileWriter sst_file_writer(EnvOptions(), options); + + // file1.sst (0 => 99) + std::string file1 = sst_files_dir_ + "file1.sst"; + ASSERT_OK(sst_file_writer.Open(file1)); + for (int k = 0; k < 100; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + ExternalSstFileInfo file1_info; + Status s = sst_file_writer.Finish(&file1_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file1_info.file_path, file1); + ASSERT_EQ(file1_info.num_entries, 100); + ASSERT_EQ(file1_info.smallest_key, Key(0)); + ASSERT_EQ(file1_info.largest_key, Key(99)); + // sst_file_writer already finished, cannot add this value + s = sst_file_writer.Put(Key(100), "bad_val"); + ASSERT_FALSE(s.ok()) << s.ToString(); + + // file2.sst (100 => 199) + std::string file2 = sst_files_dir_ + "file2.sst"; + ASSERT_OK(sst_file_writer.Open(file2)); + for (int k = 100; k < 200; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + // Cannot add this key because it's not after last added key + s = sst_file_writer.Put(Key(99), "bad_val"); + ASSERT_FALSE(s.ok()) << s.ToString(); + ExternalSstFileInfo file2_info; + s = sst_file_writer.Finish(&file2_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file2_info.file_path, file2); + ASSERT_EQ(file2_info.num_entries, 100); + ASSERT_EQ(file2_info.smallest_key, Key(100)); + ASSERT_EQ(file2_info.largest_key, Key(199)); + + // file3.sst (195 => 199) + // This file values overlap with file2 values + std::string file3 = sst_files_dir_ + "file3.sst"; + ASSERT_OK(sst_file_writer.Open(file3)); + for (int k = 195; k < 200; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val_overlap")); + } + ExternalSstFileInfo file3_info; + s = sst_file_writer.Finish(&file3_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file3_info.file_path, file3); + ASSERT_EQ(file3_info.num_entries, 5); + ASSERT_EQ(file3_info.smallest_key, Key(195)); + ASSERT_EQ(file3_info.largest_key, Key(199)); + + // file4.sst (30 => 39) + // This file values overlap with file1 values + std::string file4 = sst_files_dir_ + "file4.sst"; + ASSERT_OK(sst_file_writer.Open(file4)); + for (int k = 30; k < 40; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val_overlap")); + } + ExternalSstFileInfo file4_info; + s = sst_file_writer.Finish(&file4_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file4_info.file_path, file4); + ASSERT_EQ(file4_info.num_entries, 10); + ASSERT_EQ(file4_info.smallest_key, Key(30)); + ASSERT_EQ(file4_info.largest_key, Key(39)); + + // file5.sst (200 => 299) + std::string file5 = sst_files_dir_ + "file5.sst"; + ASSERT_OK(sst_file_writer.Open(file5)); + for (int k = 200; k < 300; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + ExternalSstFileInfo file5_info; + s = sst_file_writer.Finish(&file5_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file5_info.file_path, file5); + ASSERT_EQ(file5_info.num_entries, 100); + ASSERT_EQ(file5_info.smallest_key, Key(200)); + ASSERT_EQ(file5_info.largest_key, Key(299)); + + // list 1 has internal key range conflict + std::vector file_list0({file1, file2}); + std::vector file_list1({file3, file2, file1}); + std::vector file_list2({file5}); + std::vector file_list3({file3, file4}); + + DestroyAndReopen(options); + + // This list of files have key ranges are overlapping with each other + s = DeprecatedAddFile(file_list1); + ASSERT_FALSE(s.ok()) << s.ToString(); + + // Add files using file path list + s = DeprecatedAddFile(file_list0); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(db_->GetLatestSequenceNumber(), 0U); + for (int k = 0; k < 200; k++) { + ASSERT_EQ(Get(Key(k)), Key(k) + "_val"); + } + + TablePropertiesCollection props; + ASSERT_OK(db_->GetPropertiesOfAllTables(&props)); + ASSERT_EQ(props.size(), 2); + for (auto file_props : props) { + auto user_props = file_props.second->user_collected_properties; + ASSERT_EQ(user_props["abc_SstFileWriterCollector"], "YES"); + ASSERT_EQ(user_props["xyz_SstFileWriterCollector"], "YES"); + ASSERT_EQ(user_props["abc_Count"], "100"); + ASSERT_EQ(user_props["xyz_Count"], "100"); + } + + // Add file while holding a snapshot will fail + const Snapshot* s1 = db_->GetSnapshot(); + if (s1 != nullptr) { + ASSERT_NOK(DeprecatedAddFile(file_list2)); + db_->ReleaseSnapshot(s1); + } + // We can add the file after releaseing the snapshot + ASSERT_OK(DeprecatedAddFile(file_list2)); + ASSERT_EQ(db_->GetLatestSequenceNumber(), 0U); + for (int k = 0; k < 300; k++) { + ASSERT_EQ(Get(Key(k)), Key(k) + "_val"); + } + + ASSERT_OK(db_->GetPropertiesOfAllTables(&props)); + ASSERT_EQ(props.size(), 3); + for (auto file_props : props) { + auto user_props = file_props.second->user_collected_properties; + ASSERT_EQ(user_props["abc_SstFileWriterCollector"], "YES"); + ASSERT_EQ(user_props["xyz_SstFileWriterCollector"], "YES"); + ASSERT_EQ(user_props["abc_Count"], "100"); + ASSERT_EQ(user_props["xyz_Count"], "100"); + } + + // This file list has overlapping values with the existing data + s = DeprecatedAddFile(file_list3); + ASSERT_FALSE(s.ok()) << s.ToString(); + + // Overwrite values of keys divisible by 5 + for (int k = 0; k < 200; k += 5) { + ASSERT_OK(Put(Key(k), Key(k) + "_val_new")); + } + ASSERT_NE(db_->GetLatestSequenceNumber(), 0U); + + // Make sure values are correct before and after flush/compaction + for (int i = 0; i < 2; i++) { + for (int k = 0; k < 200; k++) { + std::string value = Key(k) + "_val"; + if (k % 5 == 0) { + value += "_new"; + } + ASSERT_EQ(Get(Key(k)), value); + } + for (int k = 200; k < 300; k++) { + std::string value = Key(k) + "_val"; + ASSERT_EQ(Get(Key(k)), value); + } + ASSERT_OK(Flush()); + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + } + + // Delete keys in range (200 => 299) + for (int k = 200; k < 300; k++) { + ASSERT_OK(Delete(Key(k))); + } + // We deleted range (200 => 299) but cannot add file5 because + // of the range tombstones + ASSERT_NOK(DeprecatedAddFile(file_list2)); + + // Compacting the DB will remove the tombstones + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + + // Now we can add the file + ASSERT_OK(DeprecatedAddFile(file_list2)); + + // Verify values of file5 in DB + for (int k = 200; k < 300; k++) { + std::string value = Key(k) + "_val"; + ASSERT_EQ(Get(Key(k)), value); + } + DestroyAndRecreateExternalSSTFilesDir(); + } while (ChangeOptions(kSkipPlainTable | kSkipFIFOCompaction)); +} + +TEST_F(ExternalSSTFileTest, AddListAtomicity) { + do { + Options options = CurrentOptions(); + + SstFileWriter sst_file_writer(EnvOptions(), options); + + // files[0].sst (0 => 99) + // files[1].sst (100 => 199) + // ... + // file[8].sst (800 => 899) + int n = 9; + std::vector files(n); + std::vector files_info(n); + for (int i = 0; i < n; i++) { + files[i] = sst_files_dir_ + "file" + std::to_string(i) + ".sst"; + ASSERT_OK(sst_file_writer.Open(files[i])); + for (int k = i * 100; k < (i + 1) * 100; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + Status s = sst_file_writer.Finish(&files_info[i]); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(files_info[i].file_path, files[i]); + ASSERT_EQ(files_info[i].num_entries, 100); + ASSERT_EQ(files_info[i].smallest_key, Key(i * 100)); + ASSERT_EQ(files_info[i].largest_key, Key((i + 1) * 100 - 1)); + } + files.push_back(sst_files_dir_ + "file" + std::to_string(n) + ".sst"); + auto s = DeprecatedAddFile(files); + ASSERT_NOK(s) << s.ToString(); + for (int k = 0; k < n * 100; k++) { + ASSERT_EQ("NOT_FOUND", Get(Key(k))); + } + files.pop_back(); + ASSERT_OK(DeprecatedAddFile(files)); + for (int k = 0; k < n * 100; k++) { + std::string value = Key(k) + "_val"; + ASSERT_EQ(Get(Key(k)), value); + } + DestroyAndRecreateExternalSSTFilesDir(); + } while (ChangeOptions(kSkipPlainTable | kSkipFIFOCompaction)); +} +// This test reporduce a bug that can happen in some cases if the DB started +// purging obsolete files when we are adding an external sst file. +// This situation may result in deleting the file while it's being added. +TEST_F(ExternalSSTFileTest, PurgeObsoleteFilesBug) { + Options options = CurrentOptions(); + SstFileWriter sst_file_writer(EnvOptions(), options); + + // file1.sst (0 => 500) + std::string sst_file_path = sst_files_dir_ + "file1.sst"; + Status s = sst_file_writer.Open(sst_file_path); + ASSERT_OK(s); + for (int i = 0; i < 500; i++) { + std::string k = Key(i); + s = sst_file_writer.Put(k, k + "_val"); + ASSERT_OK(s); + } + + ExternalSstFileInfo sst_file_info; + s = sst_file_writer.Finish(&sst_file_info); + ASSERT_OK(s); + + options.delete_obsolete_files_period_micros = 0; + options.disable_auto_compactions = true; + DestroyAndReopen(options); + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::AddFile:FileCopied", [&](void* arg) { + ASSERT_OK(Put("aaa", "bbb")); + ASSERT_OK(Flush()); + ASSERT_OK(Put("aaa", "xxx")); + ASSERT_OK(Flush()); + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + s = DeprecatedAddFile({sst_file_path}); + ASSERT_OK(s); + + for (int i = 0; i < 500; i++) { + std::string k = Key(i); + std::string v = k + "_val"; + ASSERT_EQ(Get(k), v); + } + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(ExternalSSTFileTest, SkipSnapshot) { + Options options = CurrentOptions(); + + SstFileWriter sst_file_writer(EnvOptions(), options); + + // file1.sst (0 => 99) + std::string file1 = sst_files_dir_ + "file1.sst"; + ASSERT_OK(sst_file_writer.Open(file1)); + for (int k = 0; k < 100; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + ExternalSstFileInfo file1_info; + Status s = sst_file_writer.Finish(&file1_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file1_info.file_path, file1); + ASSERT_EQ(file1_info.num_entries, 100); + ASSERT_EQ(file1_info.smallest_key, Key(0)); + ASSERT_EQ(file1_info.largest_key, Key(99)); + + // file2.sst (100 => 299) + std::string file2 = sst_files_dir_ + "file2.sst"; + ASSERT_OK(sst_file_writer.Open(file2)); + for (int k = 100; k < 300; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + ExternalSstFileInfo file2_info; + s = sst_file_writer.Finish(&file2_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file2_info.file_path, file2); + ASSERT_EQ(file2_info.num_entries, 200); + ASSERT_EQ(file2_info.smallest_key, Key(100)); + ASSERT_EQ(file2_info.largest_key, Key(299)); + + ASSERT_OK(DeprecatedAddFile({file1})); + + // Add file will fail when holding snapshot and use the default + // skip_snapshot_check to false + const Snapshot* s1 = db_->GetSnapshot(); + if (s1 != nullptr) { + ASSERT_NOK(DeprecatedAddFile({file2})); + } + + // Add file will success when set skip_snapshot_check to true even db holding + // snapshot + if (s1 != nullptr) { + ASSERT_OK(DeprecatedAddFile({file2}, false, true)); + db_->ReleaseSnapshot(s1); + } + + // file3.sst (300 => 399) + std::string file3 = sst_files_dir_ + "file3.sst"; + ASSERT_OK(sst_file_writer.Open(file3)); + for (int k = 300; k < 400; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k) + "_val")); + } + ExternalSstFileInfo file3_info; + s = sst_file_writer.Finish(&file3_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_EQ(file3_info.file_path, file3); + ASSERT_EQ(file3_info.num_entries, 100); + ASSERT_EQ(file3_info.smallest_key, Key(300)); + ASSERT_EQ(file3_info.largest_key, Key(399)); + + // check that we have change the old key + ASSERT_EQ(Get(Key(300)), "NOT_FOUND"); + const Snapshot* s2 = db_->GetSnapshot(); + ASSERT_OK(DeprecatedAddFile({file3}, false, true)); + ASSERT_EQ(Get(Key(300)), Key(300) + ("_val")); + ASSERT_EQ(Get(Key(300), s2), Key(300) + ("_val")); + + db_->ReleaseSnapshot(s2); +} + +TEST_F(ExternalSSTFileTest, MultiThreaded) { + // Bulk load 10 files every file contain 1000 keys + int num_files = 10; + int keys_per_file = 1000; + + // Generate file names + std::vector file_names; + for (int i = 0; i < num_files; i++) { + std::string file_name = "file_" + ToString(i) + ".sst"; + file_names.push_back(sst_files_dir_ + file_name); + } + + do { + Options options = CurrentOptions(); + + std::atomic thread_num(0); + std::function write_file_func = [&]() { + int file_idx = thread_num.fetch_add(1); + int range_start = file_idx * keys_per_file; + int range_end = range_start + keys_per_file; + + SstFileWriter sst_file_writer(EnvOptions(), options); + + ASSERT_OK(sst_file_writer.Open(file_names[file_idx])); + + for (int k = range_start; k < range_end; k++) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k))); + } + + Status s = sst_file_writer.Finish(); + ASSERT_TRUE(s.ok()) << s.ToString(); + }; + // Write num_files files in parallel + std::vector sst_writer_threads; + for (int i = 0; i < num_files; ++i) { + sst_writer_threads.emplace_back(write_file_func); + } + + for (auto& t : sst_writer_threads) { + t.join(); + } + + fprintf(stderr, "Wrote %d files (%d keys)\n", num_files, + num_files * keys_per_file); + + thread_num.store(0); + std::atomic files_added(0); + // Thread 0 -> Load {f0,f1} + // Thread 1 -> Load {f0,f1} + // Thread 2 -> Load {f2,f3} + // Thread 3 -> Load {f2,f3} + // Thread 4 -> Load {f4,f5} + // Thread 5 -> Load {f4,f5} + // ... + std::function load_file_func = [&]() { + // We intentionally add every file twice, and assert that it was added + // only once and the other add failed + int thread_id = thread_num.fetch_add(1); + int file_idx = (thread_id / 2) * 2; + // sometimes we use copy, sometimes link .. the result should be the same + bool move_file = (thread_id % 3 == 0); + + std::vector files_to_add; + + files_to_add = {file_names[file_idx]}; + if (static_cast(file_idx + 1) < file_names.size()) { + files_to_add.push_back(file_names[file_idx + 1]); + } + + Status s = DeprecatedAddFile(files_to_add, move_file); + if (s.ok()) { + files_added += static_cast(files_to_add.size()); + } + }; + + // Bulk load num_files files in parallel + std::vector add_file_threads; + DestroyAndReopen(options); + for (int i = 0; i < num_files; ++i) { + add_file_threads.emplace_back(load_file_func); + } + + for (auto& t : add_file_threads) { + t.join(); + } + ASSERT_EQ(files_added.load(), num_files); + fprintf(stderr, "Loaded %d files (%d keys)\n", num_files, + num_files * keys_per_file); + + // Overwrite values of keys divisible by 100 + for (int k = 0; k < num_files * keys_per_file; k += 100) { + std::string key = Key(k); + Status s = Put(key, key + "_new"); + ASSERT_TRUE(s.ok()); + } + + for (int i = 0; i < 2; i++) { + // Make sure the values are correct before and after flush/compaction + for (int k = 0; k < num_files * keys_per_file; ++k) { + std::string key = Key(k); + std::string value = (k % 100 == 0) ? (key + "_new") : key; + ASSERT_EQ(Get(key), value); + } + ASSERT_OK(Flush()); + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + } + + fprintf(stderr, "Verified %d values\n", num_files * keys_per_file); + DestroyAndRecreateExternalSSTFilesDir(); + } while (ChangeOptions(kSkipPlainTable | kSkipFIFOCompaction)); +} + +TEST_F(ExternalSSTFileTest, OverlappingRanges) { + Random rnd(301); + int picked_level = 0; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "ExternalSstFileIngestionJob::Run", [&picked_level](void* arg) { + ASSERT_TRUE(arg != nullptr); + picked_level = *(static_cast(arg)); + }); + bool need_flush = false; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::IngestExternalFile:NeedFlush", [&need_flush](void* arg) { + ASSERT_TRUE(arg != nullptr); + need_flush = *(static_cast(arg)); + }); + bool overlap_with_db = false; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "ExternalSstFileIngestionJob::AssignLevelAndSeqnoForIngestedFile", + [&overlap_with_db](void* arg) { + ASSERT_TRUE(arg != nullptr); + overlap_with_db = *(static_cast(arg)); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + do { + Options options = CurrentOptions(); + DestroyAndReopen(options); + + SstFileWriter sst_file_writer(EnvOptions(), options); + + printf("Option config = %d\n", option_config_); + std::vector> key_ranges; + for (int i = 0; i < 500; i++) { + int range_start = rnd.Uniform(20000); + int keys_per_range = 10 + rnd.Uniform(41); + + key_ranges.emplace_back(range_start, range_start + keys_per_range); + } + + int memtable_add = 0; + int success_add_file = 0; + int failed_add_file = 0; + std::map true_data; + for (size_t i = 0; i < key_ranges.size(); i++) { + int range_start = key_ranges[i].first; + int range_end = key_ranges[i].second; + + Status s; + std::string range_val = "range_" + ToString(i); + + // For 20% of ranges we use DB::Put, for 80% we use DB::AddFile + if (i && i % 5 == 0) { + // Use DB::Put to insert range (insert into memtable) + range_val += "_put"; + for (int k = range_start; k <= range_end; k++) { + s = Put(Key(k), range_val); + ASSERT_OK(s); + } + memtable_add++; + } else { + // Use DB::AddFile to insert range + range_val += "_add_file"; + + // Generate the file containing the range + std::string file_name = sst_files_dir_ + env_->GenerateUniqueId(); + ASSERT_OK(sst_file_writer.Open(file_name)); + for (int k = range_start; k <= range_end; k++) { + s = sst_file_writer.Put(Key(k), range_val); + ASSERT_OK(s); + } + ExternalSstFileInfo file_info; + s = sst_file_writer.Finish(&file_info); + ASSERT_OK(s); + + // Insert the generated file + s = DeprecatedAddFile({file_name}); + auto it = true_data.lower_bound(Key(range_start)); + if (option_config_ != kUniversalCompaction && + option_config_ != kUniversalCompactionMultiLevel) { + if (it != true_data.end() && it->first <= Key(range_end)) { + // This range overlap with data already exist in DB + ASSERT_NOK(s); + failed_add_file++; + } else { + ASSERT_OK(s); + success_add_file++; + } + } else { + if ((it != true_data.end() && it->first <= Key(range_end)) || + need_flush || picked_level > 0 || overlap_with_db) { + // This range overlap with data already exist in DB + ASSERT_NOK(s); + failed_add_file++; + } else { + ASSERT_OK(s); + success_add_file++; + } + } + } + + if (s.ok()) { + // Update true_data map to include the new inserted data + for (int k = range_start; k <= range_end; k++) { + true_data[Key(k)] = range_val; + } + } + + // Flush / Compact the DB + if (i && i % 50 == 0) { + Flush(); + } + if (i && i % 75 == 0) { + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + } + } + + printf("Total: %" ROCKSDB_PRIszt + " ranges\n" + "AddFile()|Success: %d ranges\n" + "AddFile()|RangeConflict: %d ranges\n" + "Put(): %d ranges\n", + key_ranges.size(), success_add_file, failed_add_file, memtable_add); + + // Verify the correctness of the data + for (const auto& kv : true_data) { + ASSERT_EQ(Get(kv.first), kv.second); + } + printf("keys/values verified\n"); + DestroyAndRecreateExternalSSTFilesDir(); + } while (ChangeOptions(kSkipPlainTable | kSkipFIFOCompaction)); +} + +TEST_F(ExternalSSTFileTest, PickedLevel) { + Options options = CurrentOptions(); + options.disable_auto_compactions = false; + options.level0_file_num_compaction_trigger = 4; + options.num_levels = 4; + DestroyAndReopen(options); + + std::map true_data; + + // File 0 will go to last level (L3) + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 10}, -1, false, false, + &true_data)); + EXPECT_EQ(FilesPerLevel(), "0,0,0,1"); + + // File 1 will go to level L2 (since it overlap with file 0 in L3) + ASSERT_OK(GenerateAndAddExternalFile(options, {2, 9}, -1, false, false, + &true_data)); + EXPECT_EQ(FilesPerLevel(), "0,0,1,1"); + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"ExternalSSTFileTest::PickedLevel:0", "BackgroundCallCompaction:0"}, + {"DBImpl::BackgroundCompaction:Start", + "ExternalSSTFileTest::PickedLevel:1"}, + {"ExternalSSTFileTest::PickedLevel:2", + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Flush 4 files containing the same keys + for (int i = 0; i < 4; i++) { + ASSERT_OK(Put(Key(3), Key(3) + "put")); + ASSERT_OK(Put(Key(8), Key(8) + "put")); + true_data[Key(3)] = Key(3) + "put"; + true_data[Key(8)] = Key(8) + "put"; + ASSERT_OK(Flush()); + } + + // Wait for BackgroundCompaction() to be called + TEST_SYNC_POINT("ExternalSSTFileTest::PickedLevel:0"); + TEST_SYNC_POINT("ExternalSSTFileTest::PickedLevel:1"); + + EXPECT_EQ(FilesPerLevel(), "4,0,1,1"); + + // This file overlaps with file 0 (L3), file 1 (L2) and the + // output of compaction going to L1 + ASSERT_OK(GenerateAndAddExternalFile(options, {4, 7}, -1, false, false, + &true_data)); + EXPECT_EQ(FilesPerLevel(), "5,0,1,1"); + + // This file does not overlap with any file or with the running compaction + ASSERT_OK(GenerateAndAddExternalFile(options, {9000, 9001}, -1, false, false, + &true_data)); + EXPECT_EQ(FilesPerLevel(), "5,0,1,2"); + + // Hold compaction from finishing + TEST_SYNC_POINT("ExternalSSTFileTest::PickedLevel:2"); + + dbfull()->TEST_WaitForCompact(); + EXPECT_EQ(FilesPerLevel(), "1,1,1,2"); + + size_t kcnt = 0; + VerifyDBFromMap(true_data, &kcnt, false); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(ExternalSSTFileTest, PickedLevelBug) { + Options options = CurrentOptions(); + options.disable_auto_compactions = false; + options.level0_file_num_compaction_trigger = 3; + options.num_levels = 2; + DestroyAndReopen(options); + + std::vector file_keys; + + // file #1 in L0 + file_keys = {0, 5, 7}; + for (int k : file_keys) { + ASSERT_OK(Put(Key(k), Key(k))); + } + ASSERT_OK(Flush()); + + // file #2 in L0 + file_keys = {4, 6, 8, 9}; + for (int k : file_keys) { + ASSERT_OK(Put(Key(k), Key(k))); + } + ASSERT_OK(Flush()); + + // We have 2 overlapping files in L0 + EXPECT_EQ(FilesPerLevel(), "2"); + + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::AddFile:MutexLock", "ExternalSSTFileTest::PickedLevelBug:0"}, + {"ExternalSSTFileTest::PickedLevelBug:1", "DBImpl::AddFile:MutexUnlock"}, + {"ExternalSSTFileTest::PickedLevelBug:2", + "DBImpl::RunManualCompaction:0"}, + {"ExternalSSTFileTest::PickedLevelBug:3", + "DBImpl::RunManualCompaction:1"}}); + + std::atomic bg_compact_started(false); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BackgroundCompaction:Start", + [&](void* arg) { bg_compact_started.store(true); }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // While writing the MANIFEST start a thread that will ask for compaction + rocksdb::port::Thread bg_compact([&]() { + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + }); + TEST_SYNC_POINT("ExternalSSTFileTest::PickedLevelBug:2"); + + // Start a thread that will ingest a new file + rocksdb::port::Thread bg_addfile([&]() { + file_keys = {1, 2, 3}; + ASSERT_OK(GenerateAndAddExternalFile(options, file_keys, 1)); + }); + + // Wait for AddFile to start picking levels and writing MANIFEST + TEST_SYNC_POINT("ExternalSSTFileTest::PickedLevelBug:0"); + + TEST_SYNC_POINT("ExternalSSTFileTest::PickedLevelBug:3"); + + // We need to verify that no compactions can run while AddFile is + // ingesting the files into the levels it find suitable. So we will + // wait for 2 seconds to give a chance for compactions to run during + // this period, and then make sure that no compactions where able to run + env_->SleepForMicroseconds(1000000 * 2); + ASSERT_FALSE(bg_compact_started.load()); + + // Hold AddFile from finishing writing the MANIFEST + TEST_SYNC_POINT("ExternalSSTFileTest::PickedLevelBug:1"); + + bg_addfile.join(); + bg_compact.join(); + + dbfull()->TEST_WaitForCompact(); + + int total_keys = 0; + Iterator* iter = db_->NewIterator(ReadOptions()); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_OK(iter->status()); + total_keys++; + } + ASSERT_EQ(total_keys, 10); + + delete iter; + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(ExternalSSTFileTest, CompactDuringAddFileRandom) { + Options options = CurrentOptions(); + options.disable_auto_compactions = false; + options.level0_file_num_compaction_trigger = 2; + options.num_levels = 2; + DestroyAndReopen(options); + + std::function bg_compact = [&]() { + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + }; + + int range_id = 0; + std::vector file_keys; + std::function bg_addfile = [&]() { + ASSERT_OK(GenerateAndAddExternalFile(options, file_keys, range_id)); + }; + + std::vector threads; + while (range_id < 5000) { + int range_start = range_id * 10; + int range_end = range_start + 10; + + file_keys.clear(); + for (int k = range_start + 1; k < range_end; k++) { + file_keys.push_back(k); + } + ASSERT_OK(Put(Key(range_start), Key(range_start))); + ASSERT_OK(Put(Key(range_end), Key(range_end))); + ASSERT_OK(Flush()); + + if (range_id % 10 == 0) { + threads.emplace_back(bg_compact); + } + threads.emplace_back(bg_addfile); + + for (auto& t : threads) { + t.join(); + } + threads.clear(); + + range_id++; + } + + for (int rid = 0; rid < 5000; rid++) { + int range_start = rid * 10; + int range_end = range_start + 10; + + ASSERT_EQ(Get(Key(range_start)), Key(range_start)) << rid; + ASSERT_EQ(Get(Key(range_end)), Key(range_end)) << rid; + for (int k = range_start + 1; k < range_end; k++) { + std::string v = Key(k) + ToString(rid); + ASSERT_EQ(Get(Key(k)), v) << rid; + } + } +} + +TEST_F(ExternalSSTFileTest, PickedLevelDynamic) { + Options options = CurrentOptions(); + options.disable_auto_compactions = false; + options.level0_file_num_compaction_trigger = 4; + options.level_compaction_dynamic_level_bytes = true; + options.num_levels = 4; + DestroyAndReopen(options); + std::map true_data; + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"ExternalSSTFileTest::PickedLevelDynamic:0", + "BackgroundCallCompaction:0"}, + {"DBImpl::BackgroundCompaction:Start", + "ExternalSSTFileTest::PickedLevelDynamic:1"}, + {"ExternalSSTFileTest::PickedLevelDynamic:2", + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Flush 4 files containing the same keys + for (int i = 0; i < 4; i++) { + for (int k = 20; k <= 30; k++) { + ASSERT_OK(Put(Key(k), Key(k) + "put")); + true_data[Key(k)] = Key(k) + "put"; + } + for (int k = 50; k <= 60; k++) { + ASSERT_OK(Put(Key(k), Key(k) + "put")); + true_data[Key(k)] = Key(k) + "put"; + } + ASSERT_OK(Flush()); + } + + // Wait for BackgroundCompaction() to be called + TEST_SYNC_POINT("ExternalSSTFileTest::PickedLevelDynamic:0"); + TEST_SYNC_POINT("ExternalSSTFileTest::PickedLevelDynamic:1"); + + // This file overlaps with the output of the compaction (going to L3) + // so the file will be added to L0 since L3 is the base level + ASSERT_OK(GenerateAndAddExternalFile(options, {31, 32, 33, 34}, -1, false, + false, &true_data)); + EXPECT_EQ(FilesPerLevel(), "5"); + + // This file does not overlap with the current running compactiong + ASSERT_OK(GenerateAndAddExternalFile(options, {9000, 9001}, -1, false, false, + &true_data)); + EXPECT_EQ(FilesPerLevel(), "5,0,0,1"); + + // Hold compaction from finishing + TEST_SYNC_POINT("ExternalSSTFileTest::PickedLevelDynamic:2"); + + // Output of the compaction will go to L3 + dbfull()->TEST_WaitForCompact(); + EXPECT_EQ(FilesPerLevel(), "1,0,0,2"); + + Close(); + options.disable_auto_compactions = true; + Reopen(options); + + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 15, 19}, -1, false, false, + &true_data)); + ASSERT_EQ(FilesPerLevel(), "1,0,0,3"); + + ASSERT_OK(GenerateAndAddExternalFile(options, {1000, 1001, 1002}, -1, false, + false, &true_data)); + ASSERT_EQ(FilesPerLevel(), "1,0,0,4"); + + ASSERT_OK(GenerateAndAddExternalFile(options, {500, 600, 700}, -1, false, + false, &true_data)); + ASSERT_EQ(FilesPerLevel(), "1,0,0,5"); + + // File 5 overlaps with file 2 (L3 / base level) + ASSERT_OK(GenerateAndAddExternalFile(options, {2, 10}, -1, false, false, + &true_data)); + ASSERT_EQ(FilesPerLevel(), "2,0,0,5"); + + // File 6 overlaps with file 2 (L3 / base level) and file 5 (L0) + ASSERT_OK(GenerateAndAddExternalFile(options, {3, 9}, -1, false, false, + &true_data)); + ASSERT_EQ(FilesPerLevel(), "3,0,0,5"); + + // Verify data in files + size_t kcnt = 0; + VerifyDBFromMap(true_data, &kcnt, false); + + // Write range [5 => 10] to L0 + for (int i = 5; i <= 10; i++) { + std::string k = Key(i); + std::string v = k + "put"; + ASSERT_OK(Put(k, v)); + true_data[k] = v; + } + ASSERT_OK(Flush()); + ASSERT_EQ(FilesPerLevel(), "4,0,0,5"); + + // File 7 overlaps with file 4 (L3) + ASSERT_OK(GenerateAndAddExternalFile(options, {650, 651, 652}, -1, false, + false, &true_data)); + ASSERT_EQ(FilesPerLevel(), "5,0,0,5"); + + VerifyDBFromMap(true_data, &kcnt, false); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(ExternalSSTFileTest, AddExternalSstFileWithCustomCompartor) { + Options options = CurrentOptions(); + options.comparator = ReverseBytewiseComparator(); + DestroyAndReopen(options); + + SstFileWriter sst_file_writer(EnvOptions(), options); + + // Generate files with these key ranges + // {14 -> 0} + // {24 -> 10} + // {34 -> 20} + // {44 -> 30} + // .. + std::vector generated_files; + for (int i = 0; i < 10; i++) { + std::string file_name = sst_files_dir_ + env_->GenerateUniqueId(); + ASSERT_OK(sst_file_writer.Open(file_name)); + + int range_end = i * 10; + int range_start = range_end + 15; + for (int k = (range_start - 1); k >= range_end; k--) { + ASSERT_OK(sst_file_writer.Put(Key(k), Key(k))); + } + ExternalSstFileInfo file_info; + ASSERT_OK(sst_file_writer.Finish(&file_info)); + generated_files.push_back(file_name); + } + + std::vector in_files; + + // These 2nd and 3rd files overlap with each other + in_files = {generated_files[0], generated_files[4], generated_files[5], + generated_files[7]}; + ASSERT_NOK(DeprecatedAddFile(in_files)); + + // These 2 files dont overlap with each other + in_files = {generated_files[0], generated_files[2]}; + ASSERT_OK(DeprecatedAddFile(in_files)); + + // These 2 files dont overlap with each other but overlap with keys in DB + in_files = {generated_files[3], generated_files[7]}; + ASSERT_NOK(DeprecatedAddFile(in_files)); + + // Files dont overlap and dont overlap with DB key range + in_files = {generated_files[4], generated_files[6], generated_files[8]}; + ASSERT_OK(DeprecatedAddFile(in_files)); + + for (int i = 0; i < 100; i++) { + if (i % 20 <= 14) { + ASSERT_EQ(Get(Key(i)), Key(i)); + } else { + ASSERT_EQ(Get(Key(i)), "NOT_FOUND"); + } + } +} + +TEST_F(ExternalSSTFileTest, AddFileTrivialMoveBug) { + Options options = CurrentOptions(); + options.num_levels = 3; + options.IncreaseParallelism(20); + DestroyAndReopen(options); + + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 4}, 1)); // L3 + ASSERT_OK(GenerateAndAddExternalFile(options, {2, 3}, 2)); // L2 + + ASSERT_OK(GenerateAndAddExternalFile(options, {10, 14}, 3)); // L3 + ASSERT_OK(GenerateAndAddExternalFile(options, {12, 13}, 4)); // L2 + + ASSERT_OK(GenerateAndAddExternalFile(options, {20, 24}, 5)); // L3 + ASSERT_OK(GenerateAndAddExternalFile(options, {22, 23}, 6)); // L2 + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "CompactionJob::Run():Start", [&](void* arg) { + // fit in L3 but will overlap with compaction so will be added + // to L2 but a compaction will trivially move it to L3 + // and break LSM consistency + ASSERT_OK(dbfull()->SetOptions({{"max_bytes_for_level_base", "1"}})); + ASSERT_OK(GenerateAndAddExternalFile(options, {15, 16}, 7)); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + CompactRangeOptions cro; + cro.exclusive_manual_compaction = false; + ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr)); + + dbfull()->TEST_WaitForCompact(); + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); +} + +TEST_F(ExternalSSTFileTest, CompactAddedFiles) { + Options options = CurrentOptions(); + options.num_levels = 3; + DestroyAndReopen(options); + + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 10}, 1)); // L3 + ASSERT_OK(GenerateAndAddExternalFile(options, {2, 9}, 2)); // L2 + ASSERT_OK(GenerateAndAddExternalFile(options, {3, 8}, 3)); // L1 + ASSERT_OK(GenerateAndAddExternalFile(options, {4, 7}, 4)); // L0 + + ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr)); +} + +TEST_F(ExternalSSTFileTest, SstFileWriterNonSharedKeys) { + Options options = CurrentOptions(); + DestroyAndReopen(options); + std::string file_path = sst_files_dir_ + "/not_shared"; + SstFileWriter sst_file_writer(EnvOptions(), options); + + std::string suffix(100, 'X'); + ASSERT_OK(sst_file_writer.Open(file_path)); + ASSERT_OK(sst_file_writer.Put("A" + suffix, "VAL")); + ASSERT_OK(sst_file_writer.Put("BB" + suffix, "VAL")); + ASSERT_OK(sst_file_writer.Put("CC" + suffix, "VAL")); + ASSERT_OK(sst_file_writer.Put("CXD" + suffix, "VAL")); + ASSERT_OK(sst_file_writer.Put("CZZZ" + suffix, "VAL")); + ASSERT_OK(sst_file_writer.Put("ZAAAX" + suffix, "VAL")); + + ASSERT_OK(sst_file_writer.Finish()); + ASSERT_OK(DeprecatedAddFile({file_path})); +} + +TEST_F(ExternalSSTFileTest, IngestFileWithGlobalSeqnoRandomized) { + Options options = CurrentOptions(); + options.IncreaseParallelism(20); + options.level0_slowdown_writes_trigger = 256; + options.level0_stop_writes_trigger = 256; + + for (int iter = 0; iter < 2; iter++) { + bool write_to_memtable = (iter == 0); + DestroyAndReopen(options); + + Random rnd(301); + std::map true_data; + for (int i = 0; i < 2000; i++) { + std::vector> random_data; + for (int j = 0; j < 100; j++) { + std::string k; + std::string v; + test::RandomString(&rnd, rnd.Next() % 20, &k); + test::RandomString(&rnd, rnd.Next() % 50, &v); + random_data.emplace_back(k, v); + } + + if (write_to_memtable && rnd.OneIn(4)) { + // 25% of writes go through memtable + for (auto& entry : random_data) { + ASSERT_OK(Put(entry.first, entry.second)); + true_data[entry.first] = entry.second; + } + } else { + ASSERT_OK(GenerateAndAddExternalFile(options, random_data, -1, true, + true, &true_data)); + } + } + size_t kcnt = 0; + VerifyDBFromMap(true_data, &kcnt, false); + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + VerifyDBFromMap(true_data, &kcnt, false); + } +} + +TEST_F(ExternalSSTFileTest, IngestFileWithGlobalSeqnoAssignedLevel) { + Options options = CurrentOptions(); + options.num_levels = 5; + options.disable_auto_compactions = true; + DestroyAndReopen(options); + std::vector> file_data; + std::map true_data; + + // Insert 100 -> 200 into the memtable + for (int i = 100; i <= 200; i++) { + ASSERT_OK(Put(Key(i), "memtable")); + true_data[Key(i)] = "memtable"; + } + + // Insert 0 -> 20 using AddFile + file_data.clear(); + for (int i = 0; i <= 20; i++) { + file_data.emplace_back(Key(i), "L4"); + } + ASSERT_OK(GenerateAndAddExternalFile(options, file_data, -1, true, false, + &true_data)); + + // This file dont overlap with anything in the DB, will go to L4 + ASSERT_EQ("0,0,0,0,1", FilesPerLevel()); + + // Insert 80 -> 130 using AddFile + file_data.clear(); + for (int i = 80; i <= 130; i++) { + file_data.emplace_back(Key(i), "L0"); + } + ASSERT_OK(GenerateAndAddExternalFile(options, file_data, -1, true, false, + &true_data)); + + // This file overlap with the memtable, so it will flush it and add + // it self to L0 + ASSERT_EQ("2,0,0,0,1", FilesPerLevel()); + + // Insert 30 -> 50 using AddFile + file_data.clear(); + for (int i = 30; i <= 50; i++) { + file_data.emplace_back(Key(i), "L4"); + } + ASSERT_OK(GenerateAndAddExternalFile(options, file_data, -1, true, false, + &true_data)); + + // This file dont overlap with anything in the DB and fit in L4 as well + ASSERT_EQ("2,0,0,0,2", FilesPerLevel()); + + // Insert 10 -> 40 using AddFile + file_data.clear(); + for (int i = 10; i <= 40; i++) { + file_data.emplace_back(Key(i), "L3"); + } + ASSERT_OK(GenerateAndAddExternalFile(options, file_data, -1, true, false, + &true_data)); + + // This file overlap with files in L4, we will ingest it in L3 + ASSERT_EQ("2,0,0,1,2", FilesPerLevel()); + + size_t kcnt = 0; + VerifyDBFromMap(true_data, &kcnt, false); +} + +TEST_F(ExternalSSTFileTest, IngestFileWithGlobalSeqnoMemtableFlush) { + Options options = CurrentOptions(); + DestroyAndReopen(options); + uint64_t entries_in_memtable; + std::map true_data; + + for (int k : {10, 20, 40, 80}) { + ASSERT_OK(Put(Key(k), "memtable")); + true_data[Key(k)] = "memtable"; + } + db_->GetIntProperty(DB::Properties::kNumEntriesActiveMemTable, + &entries_in_memtable); + ASSERT_GE(entries_in_memtable, 1); + + // No need for flush + ASSERT_OK(GenerateAndAddExternalFile(options, {90, 100, 110}, -1, true, false, + &true_data)); + db_->GetIntProperty(DB::Properties::kNumEntriesActiveMemTable, + &entries_in_memtable); + ASSERT_GE(entries_in_memtable, 1); + + // This file will flush the memtable + ASSERT_OK(GenerateAndAddExternalFile(options, {19, 20, 21}, -1, true, false, + &true_data)); + db_->GetIntProperty(DB::Properties::kNumEntriesActiveMemTable, + &entries_in_memtable); + ASSERT_EQ(entries_in_memtable, 0); + + for (int k : {200, 201, 205, 206}) { + ASSERT_OK(Put(Key(k), "memtable")); + true_data[Key(k)] = "memtable"; + } + db_->GetIntProperty(DB::Properties::kNumEntriesActiveMemTable, + &entries_in_memtable); + ASSERT_GE(entries_in_memtable, 1); + + // No need for flush, this file keys fit between the memtable keys + ASSERT_OK(GenerateAndAddExternalFile(options, {202, 203, 204}, -1, true, + false, &true_data)); + db_->GetIntProperty(DB::Properties::kNumEntriesActiveMemTable, + &entries_in_memtable); + ASSERT_GE(entries_in_memtable, 1); + + // This file will flush the memtable + ASSERT_OK(GenerateAndAddExternalFile(options, {206, 207}, -1, true, false, + &true_data)); + db_->GetIntProperty(DB::Properties::kNumEntriesActiveMemTable, + &entries_in_memtable); + ASSERT_EQ(entries_in_memtable, 0); + + size_t kcnt = 0; + VerifyDBFromMap(true_data, &kcnt, false); +} + +TEST_F(ExternalSSTFileTest, L0SortingIssue) { + Options options = CurrentOptions(); + options.num_levels = 2; + DestroyAndReopen(options); + std::map true_data; + + ASSERT_OK(Put(Key(1), "memtable")); + ASSERT_OK(Put(Key(10), "memtable")); + + // No Flush needed, No global seqno needed, Ingest in L1 + ASSERT_OK(GenerateAndAddExternalFile(options, {7, 8}, -1, true, false)); + // No Flush needed, but need a global seqno, Ingest in L0 + ASSERT_OK(GenerateAndAddExternalFile(options, {7, 8}, -1, true, false)); + printf("%s\n", FilesPerLevel().c_str()); + + // Overwrite what we added using external files + ASSERT_OK(Put(Key(7), "memtable")); + ASSERT_OK(Put(Key(8), "memtable")); + + // Read values from memtable + ASSERT_EQ(Get(Key(7)), "memtable"); + ASSERT_EQ(Get(Key(8)), "memtable"); + + // Flush and read from L0 + ASSERT_OK(Flush()); + printf("%s\n", FilesPerLevel().c_str()); + ASSERT_EQ(Get(Key(7)), "memtable"); + ASSERT_EQ(Get(Key(8)), "memtable"); +} + +TEST_F(ExternalSSTFileTest, CompactionDeadlock) { + Options options = CurrentOptions(); + options.num_levels = 2; + options.level0_file_num_compaction_trigger = 4; + options.level0_slowdown_writes_trigger = 4; + options.level0_stop_writes_trigger = 4; + DestroyAndReopen(options); + + // atomic conter of currently running bg threads + std::atomic running_threads(0); + + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"DBImpl::DelayWrite:Wait", "ExternalSSTFileTest::DeadLock:0"}, + {"ExternalSSTFileTest::DeadLock:1", "DBImpl::AddFile:Start"}, + {"DBImpl::AddFile:MutexLock", "ExternalSSTFileTest::DeadLock:2"}, + {"ExternalSSTFileTest::DeadLock:3", "BackgroundCallCompaction:0"}, + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // Start ingesting and extrnal file in the background + rocksdb::port::Thread bg_ingest_file([&]() { + running_threads += 1; + ASSERT_OK(GenerateAndAddExternalFile(options, {5, 6})); + running_threads -= 1; + }); + + ASSERT_OK(Put(Key(1), "memtable")); + ASSERT_OK(Flush()); + + ASSERT_OK(Put(Key(2), "memtable")); + ASSERT_OK(Flush()); + + ASSERT_OK(Put(Key(3), "memtable")); + ASSERT_OK(Flush()); + + ASSERT_OK(Put(Key(4), "memtable")); + ASSERT_OK(Flush()); + + // This thread will try to insert into the memtable but since we have 4 L0 + // files this thread will be blocked and hold the writer thread + rocksdb::port::Thread bg_block_put([&]() { + running_threads += 1; + ASSERT_OK(Put(Key(10), "memtable")); + running_threads -= 1; + }); + + // Make sure DelayWrite is called first + TEST_SYNC_POINT("ExternalSSTFileTest::DeadLock:0"); + + // `DBImpl::AddFile:Start` will wait until we be here + TEST_SYNC_POINT("ExternalSSTFileTest::DeadLock:1"); + + // Wait for IngestExternalFile() to start and aquire mutex + TEST_SYNC_POINT("ExternalSSTFileTest::DeadLock:2"); + + // Now let compaction start + TEST_SYNC_POINT("ExternalSSTFileTest::DeadLock:3"); + + // Wait for max 5 seconds, if we did not finish all bg threads + // then we hit the deadlock bug + for (int i = 0; i < 10; i++) { + if (running_threads.load() == 0) { + break; + } + env_->SleepForMicroseconds(500000); + } + + ASSERT_EQ(running_threads.load(), 0); + + bg_ingest_file.join(); + bg_block_put.join(); +} + +TEST_F(ExternalSSTFileTest, DirtyExit) { + Options options = CurrentOptions(); + DestroyAndReopen(options); + std::string file_path = sst_files_dir_ + "/dirty_exit"; + std::unique_ptr sst_file_writer; + + // Destruct SstFileWriter without calling Finish() + sst_file_writer.reset(new SstFileWriter(EnvOptions(), options)); + ASSERT_OK(sst_file_writer->Open(file_path)); + sst_file_writer.reset(); + + // Destruct SstFileWriter with a failing Finish + sst_file_writer.reset(new SstFileWriter(EnvOptions(), options)); + ASSERT_OK(sst_file_writer->Open(file_path)); + ASSERT_NOK(sst_file_writer->Finish()); +} + +TEST_F(ExternalSSTFileTest, FileWithCFInfo) { + Options options = CurrentOptions(); + CreateAndReopenWithCF({"koko", "toto"}, options); + + SstFileWriter sfw_default(EnvOptions(), options, handles_[0]); + SstFileWriter sfw_cf1(EnvOptions(), options, handles_[1]); + SstFileWriter sfw_cf2(EnvOptions(), options, handles_[2]); + SstFileWriter sfw_unknown(EnvOptions(), options); + + // default_cf.sst + const std::string cf_default_sst = sst_files_dir_ + "/default_cf.sst"; + ASSERT_OK(sfw_default.Open(cf_default_sst)); + ASSERT_OK(sfw_default.Put("K1", "V1")); + ASSERT_OK(sfw_default.Put("K2", "V2")); + ASSERT_OK(sfw_default.Finish()); + + // cf1.sst + const std::string cf1_sst = sst_files_dir_ + "/cf1.sst"; + ASSERT_OK(sfw_cf1.Open(cf1_sst)); + ASSERT_OK(sfw_cf1.Put("K3", "V1")); + ASSERT_OK(sfw_cf1.Put("K4", "V2")); + ASSERT_OK(sfw_cf1.Finish()); + + // cf_unknown.sst + const std::string unknown_sst = sst_files_dir_ + "/cf_unknown.sst"; + ASSERT_OK(sfw_unknown.Open(unknown_sst)); + ASSERT_OK(sfw_unknown.Put("K5", "V1")); + ASSERT_OK(sfw_unknown.Put("K6", "V2")); + ASSERT_OK(sfw_unknown.Finish()); + + IngestExternalFileOptions ifo; + + // SST CF dont match + ASSERT_NOK(db_->IngestExternalFile(handles_[0], {cf1_sst}, ifo)); + // SST CF dont match + ASSERT_NOK(db_->IngestExternalFile(handles_[2], {cf1_sst}, ifo)); + // SST CF match + ASSERT_OK(db_->IngestExternalFile(handles_[1], {cf1_sst}, ifo)); + + // SST CF dont match + ASSERT_NOK(db_->IngestExternalFile(handles_[1], {cf_default_sst}, ifo)); + // SST CF dont match + ASSERT_NOK(db_->IngestExternalFile(handles_[2], {cf_default_sst}, ifo)); + // SST CF match + ASSERT_OK(db_->IngestExternalFile(handles_[0], {cf_default_sst}, ifo)); + + // SST CF unknown + ASSERT_OK(db_->IngestExternalFile(handles_[1], {unknown_sst}, ifo)); + // SST CF unknown + ASSERT_OK(db_->IngestExternalFile(handles_[2], {unknown_sst}, ifo)); + // SST CF unknown + ASSERT_OK(db_->IngestExternalFile(handles_[0], {unknown_sst}, ifo)); + + // Cannot ingest a file into a dropped CF + ASSERT_OK(db_->DropColumnFamily(handles_[1])); + ASSERT_NOK(db_->IngestExternalFile(handles_[1], {unknown_sst}, ifo)); + + // CF was not dropped, ok to Ingest + ASSERT_OK(db_->IngestExternalFile(handles_[2], {unknown_sst}, ifo)); +} + +class TestIngestExternalFileListener : public EventListener { + public: + void OnExternalFileIngested(DB* db, + const ExternalFileIngestionInfo& info) override { + ingested_files.push_back(info); + } + + std::vector ingested_files; +}; + +TEST_F(ExternalSSTFileTest, IngestionListener) { + Options options = CurrentOptions(); + TestIngestExternalFileListener* listener = + new TestIngestExternalFileListener(); + options.listeners.emplace_back(listener); + CreateAndReopenWithCF({"koko", "toto"}, options); + + // Ingest into default cf + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 2}, -1, true, true, nullptr, + handles_[0])); + ASSERT_EQ(listener->ingested_files.size(), 1); + ASSERT_EQ(listener->ingested_files.back().cf_name, "default"); + ASSERT_EQ(listener->ingested_files.back().global_seqno, 0); + ASSERT_EQ(listener->ingested_files.back().table_properties.column_family_id, + 0); + ASSERT_EQ(listener->ingested_files.back().table_properties.column_family_name, + "default"); + + // Ingest into cf1 + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 2}, -1, true, true, nullptr, + handles_[1])); + ASSERT_EQ(listener->ingested_files.size(), 2); + ASSERT_EQ(listener->ingested_files.back().cf_name, "koko"); + ASSERT_EQ(listener->ingested_files.back().global_seqno, 0); + ASSERT_EQ(listener->ingested_files.back().table_properties.column_family_id, + 1); + ASSERT_EQ(listener->ingested_files.back().table_properties.column_family_name, + "koko"); + + // Ingest into cf2 + ASSERT_OK(GenerateAndAddExternalFile(options, {1, 2}, -1, true, true, nullptr, + handles_[2])); + ASSERT_EQ(listener->ingested_files.size(), 3); + ASSERT_EQ(listener->ingested_files.back().cf_name, "toto"); + ASSERT_EQ(listener->ingested_files.back().global_seqno, 0); + ASSERT_EQ(listener->ingested_files.back().table_properties.column_family_id, + 2); + ASSERT_EQ(listener->ingested_files.back().table_properties.column_family_name, + "toto"); +} + +TEST_F(ExternalSSTFileTest, SnapshotInconsistencyBug) { + Options options = CurrentOptions(); + DestroyAndReopen(options); + const int kNumKeys = 10000; + + // Insert keys using normal path and take a snapshot + for (int i = 0; i < kNumKeys; i++) { + ASSERT_OK(Put(Key(i), Key(i) + "_V1")); + } + const Snapshot* snap = db_->GetSnapshot(); + + // Overwrite all keys using IngestExternalFile + std::string sst_file_path = sst_files_dir_ + "file1.sst"; + SstFileWriter sst_file_writer(EnvOptions(), options); + ASSERT_OK(sst_file_writer.Open(sst_file_path)); + for (int i = 0; i < kNumKeys; i++) { + ASSERT_OK(sst_file_writer.Put(Key(i), Key(i) + "_V2")); + } + ASSERT_OK(sst_file_writer.Finish()); + + IngestExternalFileOptions ifo; + ifo.move_files = true; + ASSERT_OK(db_->IngestExternalFile({sst_file_path}, ifo)); + + for (int i = 0; i < kNumKeys; i++) { + ASSERT_EQ(Get(Key(i), snap), Key(i) + "_V1"); + ASSERT_EQ(Get(Key(i)), Key(i) + "_V2"); + } + + db_->ReleaseSnapshot(snap); +} + +TEST_F(ExternalSSTFileTest, IngestBehind) { + Options options = CurrentOptions(); + options.compaction_style = kCompactionStyleUniversal; + options.num_levels = 3; + options.disable_auto_compactions = false; + DestroyAndReopen(options); + std::vector> file_data; + std::map true_data; + + // Insert 100 -> 200 into the memtable + for (int i = 100; i <= 200; i++) { + ASSERT_OK(Put(Key(i), "memtable")); + true_data[Key(i)] = "memtable"; + } + + // Insert 100 -> 200 using IngestExternalFile + file_data.clear(); + for (int i = 0; i <= 20; i++) { + file_data.emplace_back(Key(i), "ingest_behind"); + } + + IngestExternalFileOptions ifo; + ifo.allow_global_seqno = true; + ifo.ingest_behind = true; + + // Can't ingest behind since allow_ingest_behind isn't set to true + ASSERT_NOK(GenerateAndAddExternalFileIngestBehind(options, ifo, + file_data, -1, false, + &true_data)); + + options.allow_ingest_behind = true; + // check that we still can open the DB, as num_levels should be + // sanitized to 3 + options.num_levels = 2; + DestroyAndReopen(options); + + options.num_levels = 3; + DestroyAndReopen(options); + // Insert 100 -> 200 into the memtable + for (int i = 100; i <= 200; i++) { + ASSERT_OK(Put(Key(i), "memtable")); + true_data[Key(i)] = "memtable"; + } + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + // Universal picker should go at second from the bottom level + ASSERT_EQ("0,1", FilesPerLevel()); + ASSERT_OK(GenerateAndAddExternalFileIngestBehind(options, ifo, + file_data, -1, false, + &true_data)); + ASSERT_EQ("0,1,1", FilesPerLevel()); + // this time ingest should fail as the file doesn't fit to the bottom level + ASSERT_NOK(GenerateAndAddExternalFileIngestBehind(options, ifo, + file_data, -1, false, + &true_data)); + ASSERT_EQ("0,1,1", FilesPerLevel()); + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + // bottom level should be empty + ASSERT_EQ("0,1", FilesPerLevel()); + + size_t kcnt = 0; + VerifyDBFromMap(true_data, &kcnt, false); +} +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, + "SKIPPED as External SST File Writer and Ingestion are not supported " + "in ROCKSDB_LITE\n"); + return 0; +} + +#endif // !ROCKSDB_LITE diff --git a/db/fault_injection_test.cc b/db/fault_injection_test.cc new file mode 100644 index 00000000000..adfcb4db5a7 --- /dev/null +++ b/db/fault_injection_test.cc @@ -0,0 +1,547 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright 2014 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +// This test uses a custom Env to keep track of the state of a filesystem as of +// the last "sync". It then checks for data loss errors by purposely dropping +// file data (or entire files) not protected by a "sync". + +#include "db/db_impl.h" +#include "db/log_format.h" +#include "db/version_set.h" +#include "env/mock_env.h" +#include "rocksdb/cache.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/table.h" +#include "rocksdb/write_batch.h" +#include "util/fault_injection_test_env.h" +#include "util/filename.h" +#include "util/logging.h" +#include "util/mutexlock.h" +#include "util/sync_point.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +static const int kValueSize = 1000; +static const int kMaxNumValues = 2000; +static const size_t kNumIterations = 3; + +class FaultInjectionTest : public testing::Test, + public testing::WithParamInterface { + protected: + enum OptionConfig { + kDefault, + kDifferentDataDir, + kWalDir, + kSyncWal, + kWalDirSyncWal, + kMultiLevels, + kEnd, + }; + int option_config_; + // When need to make sure data is persistent, sync WAL + bool sync_use_wal_; + // When need to make sure data is persistent, call DB::CompactRange() + bool sync_use_compact_; + + bool sequential_order_; + + protected: + public: + enum ExpectedVerifResult { kValExpectFound, kValExpectNoError }; + enum ResetMethod { + kResetDropUnsyncedData, + kResetDropRandomUnsyncedData, + kResetDeleteUnsyncedFiles, + kResetDropAndDeleteUnsynced + }; + + std::unique_ptr base_env_; + FaultInjectionTestEnv* env_; + std::string dbname_; + shared_ptr tiny_cache_; + Options options_; + DB* db_; + + FaultInjectionTest() + : option_config_(kDefault), + sync_use_wal_(false), + sync_use_compact_(true), + base_env_(nullptr), + env_(NULL), + db_(NULL) { + } + + ~FaultInjectionTest() { + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); + } + + bool ChangeOptions() { + option_config_++; + if (option_config_ >= kEnd) { + return false; + } else { + if (option_config_ == kMultiLevels) { + base_env_.reset(new MockEnv(Env::Default())); + } + return true; + } + } + + // Return the current option configuration. + Options CurrentOptions() { + sync_use_wal_ = false; + sync_use_compact_ = true; + Options options; + switch (option_config_) { + case kWalDir: + options.wal_dir = test::TmpDir(env_) + "/fault_test_wal"; + break; + case kDifferentDataDir: + options.db_paths.emplace_back(test::TmpDir(env_) + "/fault_test_data", + 1000000U); + break; + case kSyncWal: + sync_use_wal_ = true; + sync_use_compact_ = false; + break; + case kWalDirSyncWal: + options.wal_dir = test::TmpDir(env_) + "/fault_test_wal"; + sync_use_wal_ = true; + sync_use_compact_ = false; + break; + case kMultiLevels: + options.write_buffer_size = 64 * 1024; + options.target_file_size_base = 64 * 1024; + options.level0_file_num_compaction_trigger = 2; + options.level0_slowdown_writes_trigger = 2; + options.level0_stop_writes_trigger = 4; + options.max_bytes_for_level_base = 128 * 1024; + options.max_write_buffer_number = 2; + options.max_background_compactions = 8; + options.max_background_flushes = 8; + sync_use_wal_ = true; + sync_use_compact_ = false; + break; + default: + break; + } + return options; + } + + Status NewDB() { + assert(db_ == NULL); + assert(tiny_cache_ == nullptr); + assert(env_ == NULL); + + env_ = + new FaultInjectionTestEnv(base_env_ ? base_env_.get() : Env::Default()); + + options_ = CurrentOptions(); + options_.env = env_; + options_.paranoid_checks = true; + + BlockBasedTableOptions table_options; + tiny_cache_ = NewLRUCache(100); + table_options.block_cache = tiny_cache_; + options_.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + dbname_ = test::TmpDir() + "/fault_test"; + + EXPECT_OK(DestroyDB(dbname_, options_)); + + options_.create_if_missing = true; + Status s = OpenDB(); + options_.create_if_missing = false; + return s; + } + + void SetUp() override { + sequential_order_ = GetParam(); + ASSERT_OK(NewDB()); + } + + void TearDown() override { + CloseDB(); + + Status s = DestroyDB(dbname_, options_); + + delete env_; + env_ = NULL; + + tiny_cache_.reset(); + + ASSERT_OK(s); + } + + void Build(const WriteOptions& write_options, int start_idx, int num_vals) { + std::string key_space, value_space; + WriteBatch batch; + for (int i = start_idx; i < start_idx + num_vals; i++) { + Slice key = Key(i, &key_space); + batch.Clear(); + batch.Put(key, Value(i, &value_space)); + ASSERT_OK(db_->Write(write_options, &batch)); + } + } + + Status ReadValue(int i, std::string* val) const { + std::string key_space, value_space; + Slice key = Key(i, &key_space); + Value(i, &value_space); + ReadOptions options; + return db_->Get(options, key, val); + } + + Status Verify(int start_idx, int num_vals, + ExpectedVerifResult expected) const { + std::string val; + std::string value_space; + Status s; + for (int i = start_idx; i < start_idx + num_vals && s.ok(); i++) { + Value(i, &value_space); + s = ReadValue(i, &val); + if (s.ok()) { + EXPECT_EQ(value_space, val); + } + if (expected == kValExpectFound) { + if (!s.ok()) { + fprintf(stderr, "Error when read %dth record (expect found): %s\n", i, + s.ToString().c_str()); + return s; + } + } else if (!s.ok() && !s.IsNotFound()) { + fprintf(stderr, "Error when read %dth record: %s\n", i, + s.ToString().c_str()); + return s; + } + } + return Status::OK(); + } + +#ifdef ROCKSDB_UBSAN_RUN +#if defined(__clang__) +__attribute__((__no_sanitize__("shift"), no_sanitize("signed-integer-overflow"))) +#elif defined(__GNUC__) +__attribute__((__no_sanitize_undefined__)) +#endif +#endif + // Return the ith key + Slice Key(int i, std::string* storage) const { + int num = i; + if (!sequential_order_) { + // random transfer + const int m = 0x5bd1e995; + num *= m; + num ^= num << 24; + } + char buf[100]; + snprintf(buf, sizeof(buf), "%016d", num); + storage->assign(buf, strlen(buf)); + return Slice(*storage); + } + + // Return the value to associate with the specified key + Slice Value(int k, std::string* storage) const { + Random r(k); + return test::RandomString(&r, kValueSize, storage); + } + + void CloseDB() { + delete db_; + db_ = nullptr; + } + + Status OpenDB() { + CloseDB(); + env_->ResetState(); + Status s = DB::Open(options_, dbname_, &db_); + assert(db_ != nullptr); + return s; + } + + void DeleteAllData() { + Iterator* iter = db_->NewIterator(ReadOptions()); + WriteOptions options; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ASSERT_OK(db_->Delete(WriteOptions(), iter->key())); + } + + delete iter; + + FlushOptions flush_options; + flush_options.wait = true; + db_->Flush(flush_options); + } + + // rnd cannot be null for kResetDropRandomUnsyncedData + void ResetDBState(ResetMethod reset_method, Random* rnd = nullptr) { + env_->AssertNoOpenFile(); + switch (reset_method) { + case kResetDropUnsyncedData: + ASSERT_OK(env_->DropUnsyncedFileData()); + break; + case kResetDropRandomUnsyncedData: + ASSERT_OK(env_->DropRandomUnsyncedFileData(rnd)); + break; + case kResetDeleteUnsyncedFiles: + ASSERT_OK(env_->DeleteFilesCreatedAfterLastDirSync()); + break; + case kResetDropAndDeleteUnsynced: + ASSERT_OK(env_->DropUnsyncedFileData()); + ASSERT_OK(env_->DeleteFilesCreatedAfterLastDirSync()); + break; + default: + assert(false); + } + } + + void PartialCompactTestPreFault(int num_pre_sync, int num_post_sync) { + DeleteAllData(); + + WriteOptions write_options; + write_options.sync = sync_use_wal_; + + Build(write_options, 0, num_pre_sync); + if (sync_use_compact_) { + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + } + write_options.sync = false; + Build(write_options, num_pre_sync, num_post_sync); + } + + void PartialCompactTestReopenWithFault(ResetMethod reset_method, + int num_pre_sync, int num_post_sync, + Random* rnd = nullptr) { + env_->SetFilesystemActive(false); + CloseDB(); + ResetDBState(reset_method, rnd); + ASSERT_OK(OpenDB()); + ASSERT_OK(Verify(0, num_pre_sync, FaultInjectionTest::kValExpectFound)); + ASSERT_OK(Verify(num_pre_sync, num_post_sync, + FaultInjectionTest::kValExpectNoError)); + WaitCompactionFinish(); + ASSERT_OK(Verify(0, num_pre_sync, FaultInjectionTest::kValExpectFound)); + ASSERT_OK(Verify(num_pre_sync, num_post_sync, + FaultInjectionTest::kValExpectNoError)); + } + + void NoWriteTestPreFault() { + } + + void NoWriteTestReopenWithFault(ResetMethod reset_method) { + CloseDB(); + ResetDBState(reset_method); + ASSERT_OK(OpenDB()); + } + + void WaitCompactionFinish() { + static_cast(db_->GetRootDB())->TEST_WaitForCompact(); + ASSERT_OK(db_->Put(WriteOptions(), "", "")); + } +}; + +TEST_P(FaultInjectionTest, FaultTest) { + do { + Random rnd(301); + + for (size_t idx = 0; idx < kNumIterations; idx++) { + int num_pre_sync = rnd.Uniform(kMaxNumValues); + int num_post_sync = rnd.Uniform(kMaxNumValues); + + PartialCompactTestPreFault(num_pre_sync, num_post_sync); + PartialCompactTestReopenWithFault(kResetDropUnsyncedData, num_pre_sync, + num_post_sync); + NoWriteTestPreFault(); + NoWriteTestReopenWithFault(kResetDropUnsyncedData); + + PartialCompactTestPreFault(num_pre_sync, num_post_sync); + PartialCompactTestReopenWithFault(kResetDropRandomUnsyncedData, + num_pre_sync, num_post_sync, &rnd); + NoWriteTestPreFault(); + NoWriteTestReopenWithFault(kResetDropUnsyncedData); + + // Setting a separate data path won't pass the test as we don't sync + // it after creating new files, + PartialCompactTestPreFault(num_pre_sync, num_post_sync); + PartialCompactTestReopenWithFault(kResetDropAndDeleteUnsynced, + num_pre_sync, num_post_sync); + NoWriteTestPreFault(); + NoWriteTestReopenWithFault(kResetDropAndDeleteUnsynced); + + PartialCompactTestPreFault(num_pre_sync, num_post_sync); + // No new files created so we expect all values since no files will be + // dropped. + PartialCompactTestReopenWithFault(kResetDeleteUnsyncedFiles, num_pre_sync, + num_post_sync); + NoWriteTestPreFault(); + NoWriteTestReopenWithFault(kResetDeleteUnsyncedFiles); + } + } while (ChangeOptions()); +} + +// Previous log file is not fsynced if sync is forced after log rolling. +TEST_P(FaultInjectionTest, WriteOptionSyncTest) { + test::SleepingBackgroundTask sleeping_task_low; + env_->SetBackgroundThreads(1, Env::HIGH); + // Block the job queue to prevent flush job from running. + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::HIGH); + sleeping_task_low.WaitUntilSleeping(); + + WriteOptions write_options; + write_options.sync = false; + + std::string key_space, value_space; + ASSERT_OK( + db_->Put(write_options, Key(1, &key_space), Value(1, &value_space))); + FlushOptions flush_options; + flush_options.wait = false; + ASSERT_OK(db_->Flush(flush_options)); + write_options.sync = true; + ASSERT_OK( + db_->Put(write_options, Key(2, &key_space), Value(2, &value_space))); + db_->FlushWAL(false); + + env_->SetFilesystemActive(false); + NoWriteTestReopenWithFault(kResetDropAndDeleteUnsynced); + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilDone(); + + ASSERT_OK(OpenDB()); + std::string val; + Value(2, &value_space); + ASSERT_OK(ReadValue(2, &val)); + ASSERT_EQ(value_space, val); + + Value(1, &value_space); + ASSERT_OK(ReadValue(1, &val)); + ASSERT_EQ(value_space, val); +} + +TEST_P(FaultInjectionTest, UninstalledCompaction) { + options_.target_file_size_base = 32 * 1024; + options_.write_buffer_size = 100 << 10; // 100KB + options_.level0_file_num_compaction_trigger = 6; + options_.level0_stop_writes_trigger = 1 << 10; + options_.level0_slowdown_writes_trigger = 1 << 10; + options_.max_background_compactions = 1; + OpenDB(); + + if (!sequential_order_) { + rocksdb::SyncPoint::GetInstance()->LoadDependency({ + {"FaultInjectionTest::FaultTest:0", "DBImpl::BGWorkCompaction"}, + {"CompactionJob::Run():End", "FaultInjectionTest::FaultTest:1"}, + {"FaultInjectionTest::FaultTest:2", + "DBImpl::BackgroundCompaction:NonTrivial:AfterRun"}, + }); + } + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + int kNumKeys = 1000; + Build(WriteOptions(), 0, kNumKeys); + FlushOptions flush_options; + flush_options.wait = true; + db_->Flush(flush_options); + ASSERT_OK(db_->Put(WriteOptions(), "", "")); + TEST_SYNC_POINT("FaultInjectionTest::FaultTest:0"); + TEST_SYNC_POINT("FaultInjectionTest::FaultTest:1"); + env_->SetFilesystemActive(false); + TEST_SYNC_POINT("FaultInjectionTest::FaultTest:2"); + CloseDB(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + ResetDBState(kResetDropUnsyncedData); + + std::atomic opened(false); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::Open:Opened", [&](void* arg) { opened.store(true); }); + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "DBImpl::BGWorkCompaction", + [&](void* arg) { ASSERT_TRUE(opened.load()); }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + ASSERT_OK(OpenDB()); + ASSERT_OK(Verify(0, kNumKeys, FaultInjectionTest::kValExpectFound)); + WaitCompactionFinish(); + ASSERT_OK(Verify(0, kNumKeys, FaultInjectionTest::kValExpectFound)); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + rocksdb::SyncPoint::GetInstance()->ClearAllCallBacks(); +} + +TEST_P(FaultInjectionTest, ManualLogSyncTest) { + test::SleepingBackgroundTask sleeping_task_low; + env_->SetBackgroundThreads(1, Env::HIGH); + // Block the job queue to prevent flush job from running. + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task_low, + Env::Priority::HIGH); + sleeping_task_low.WaitUntilSleeping(); + + WriteOptions write_options; + write_options.sync = false; + + std::string key_space, value_space; + ASSERT_OK( + db_->Put(write_options, Key(1, &key_space), Value(1, &value_space))); + FlushOptions flush_options; + flush_options.wait = false; + ASSERT_OK(db_->Flush(flush_options)); + ASSERT_OK( + db_->Put(write_options, Key(2, &key_space), Value(2, &value_space))); + ASSERT_OK(db_->FlushWAL(true)); + + env_->SetFilesystemActive(false); + NoWriteTestReopenWithFault(kResetDropAndDeleteUnsynced); + sleeping_task_low.WakeUp(); + sleeping_task_low.WaitUntilDone(); + + ASSERT_OK(OpenDB()); + std::string val; + Value(2, &value_space); + ASSERT_OK(ReadValue(2, &val)); + ASSERT_EQ(value_space, val); + + Value(1, &value_space); + ASSERT_OK(ReadValue(1, &val)); + ASSERT_EQ(value_space, val); +} + +TEST_P(FaultInjectionTest, WriteBatchWalTerminationTest) { + ReadOptions ro; + Options options = CurrentOptions(); + options.env = env_; + + WriteOptions wo; + wo.sync = true; + wo.disableWAL = false; + WriteBatch batch; + batch.Put("cats", "dogs"); + batch.MarkWalTerminationPoint(); + batch.Put("boys", "girls"); + ASSERT_OK(db_->Write(wo, &batch)); + + env_->SetFilesystemActive(false); + NoWriteTestReopenWithFault(kResetDropAndDeleteUnsynced); + ASSERT_OK(OpenDB()); + + std::string val; + ASSERT_OK(db_->Get(ro, "cats", &val)); + ASSERT_EQ("dogs", val); + ASSERT_EQ(db_->Get(ro, "boys", &val), Status::NotFound()); +} + +INSTANTIATE_TEST_CASE_P(FaultTest, FaultInjectionTest, ::testing::Bool()); + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/file_indexer.cc b/db/file_indexer.cc index 56691bde5f1..abfa7cf4c68 100644 --- a/db/file_indexer.cc +++ b/db/file_indexer.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,25 +9,28 @@ #include "db/file_indexer.h" #include -#include "rocksdb/comparator.h" +#include #include "db/version_edit.h" +#include "rocksdb/comparator.h" namespace rocksdb { FileIndexer::FileIndexer(const Comparator* ucmp) : num_levels_(0), ucmp_(ucmp), level_rb_(nullptr) {} -uint32_t FileIndexer::NumLevelIndex() { - return next_level_index_.size(); -} +size_t FileIndexer::NumLevelIndex() const { return next_level_index_.size(); } -uint32_t FileIndexer::LevelIndexSize(uint32_t level) { +size_t FileIndexer::LevelIndexSize(size_t level) const { + if (level >= next_level_index_.size()) { + return 0; + } return next_level_index_[level].num_index; } -void FileIndexer::GetNextLevelIndex( - const uint32_t level, const uint32_t file_index, const int cmp_smallest, - const int cmp_largest, int32_t* left_bound, int32_t* right_bound) { +void FileIndexer::GetNextLevelIndex(const size_t level, const size_t file_index, + const int cmp_smallest, + const int cmp_largest, int32_t* left_bound, + int32_t* right_bound) const { assert(level > 0); // Last level, no hint @@ -69,7 +72,7 @@ void FileIndexer::GetNextLevelIndex( assert(*right_bound <= level_rb_[level + 1]); } -void FileIndexer::UpdateIndex(Arena* arena, const uint32_t num_levels, +void FileIndexer::UpdateIndex(Arena* arena, const size_t num_levels, std::vector* const files) { if (files == nullptr) { return; @@ -90,17 +93,17 @@ void FileIndexer::UpdateIndex(Arena* arena, const uint32_t num_levels, } // L1 - Ln-1 - for (uint32_t level = 1; level < num_levels_ - 1; ++level) { + for (size_t level = 1; level < num_levels_ - 1; ++level) { const auto& upper_files = files[level]; - const int32_t upper_size = upper_files.size(); + const int32_t upper_size = static_cast(upper_files.size()); const auto& lower_files = files[level + 1]; - level_rb_[level] = upper_files.size() - 1; + level_rb_[level] = static_cast(upper_files.size()) - 1; if (upper_size == 0) { continue; } IndexLevel& index_level = next_level_index_[level]; index_level.num_index = upper_size; - char* mem = arena->AllocateAligned(upper_size * sizeof(IndexUnit)); + mem = arena->AllocateAligned(upper_size * sizeof(IndexUnit)); index_level.index_units = new (mem) IndexUnit[upper_size]; CalculateLB( @@ -129,7 +132,8 @@ void FileIndexer::UpdateIndex(Arena* arena, const uint32_t num_levels, [](IndexUnit* index, int32_t f_idx) { index->largest_rb = f_idx; }); } - level_rb_[num_levels_ - 1] = files[num_levels_ - 1].size() - 1; + level_rb_[num_levels_ - 1] = + static_cast(files[num_levels_ - 1].size()) - 1; } void FileIndexer::CalculateLB( @@ -137,8 +141,8 @@ void FileIndexer::CalculateLB( const std::vector& lower_files, IndexLevel* index_level, std::function cmp_op, std::function set_index) { - const int32_t upper_size = upper_files.size(); - const int32_t lower_size = lower_files.size(); + const int32_t upper_size = static_cast(upper_files.size()); + const int32_t lower_size = static_cast(lower_files.size()); int32_t upper_idx = 0; int32_t lower_idx = 0; @@ -175,8 +179,8 @@ void FileIndexer::CalculateRB( const std::vector& lower_files, IndexLevel* index_level, std::function cmp_op, std::function set_index) { - const int32_t upper_size = upper_files.size(); - const int32_t lower_size = lower_files.size(); + const int32_t upper_size = static_cast(upper_files.size()); + const int32_t lower_size = static_cast(lower_files.size()); int32_t upper_idx = upper_size - 1; int32_t lower_idx = lower_size - 1; diff --git a/db/file_indexer.h b/db/file_indexer.h index 127b3ee466a..1bef3aab0ca 100644 --- a/db/file_indexer.h +++ b/db/file_indexer.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -12,6 +12,7 @@ #include #include #include +#include "port/port.h" #include "util/arena.h" #include "util/autovector.h" @@ -25,9 +26,9 @@ struct FileLevel; // The file tree structure in Version is prebuilt and the range of each file // is known. On Version::Get(), it uses binary search to find a potential file // and then check if a target key can be found in the file by comparing the key -// to each file's smallest and largest key. The results of these comparisions +// to each file's smallest and largest key. The results of these comparisons // can be reused beyond checking if a key falls into a file's range. -// With some pre-calculated knowledge, each key comparision that has been done +// With some pre-calculated knowledge, each key comparison that has been done // can serve as a hint to narrow down further searches: if a key compared to // be smaller than a file's smallest or largest, that comparison can be used // to find out the right bound of next binary search. Similarly, if a key @@ -42,27 +43,28 @@ class FileIndexer { public: explicit FileIndexer(const Comparator* ucmp); - uint32_t NumLevelIndex(); + size_t NumLevelIndex() const; - uint32_t LevelIndexSize(uint32_t level); + size_t LevelIndexSize(size_t level) const; // Return a file index range in the next level to search for a key based on - // smallest and largest key comparision for the current file specified by + // smallest and largest key comparison for the current file specified by // level and file_index. When *left_index < *right_index, both index should // be valid and fit in the vector size. - void GetNextLevelIndex( - const uint32_t level, const uint32_t file_index, const int cmp_smallest, - const int cmp_largest, int32_t* left_bound, int32_t* right_bound); + void GetNextLevelIndex(const size_t level, const size_t file_index, + const int cmp_smallest, const int cmp_largest, + int32_t* left_bound, int32_t* right_bound) const; - void UpdateIndex(Arena* arena, const uint32_t num_levels, + void UpdateIndex(Arena* arena, const size_t num_levels, std::vector* const files); enum { - kLevelMaxIndex = std::numeric_limits::max() + // MSVC version 1800 still does not have constexpr for ::max() + kLevelMaxIndex = rocksdb::port::kMaxInt32 }; private: - uint32_t num_levels_; + size_t num_levels_; const Comparator* ucmp_; struct IndexUnit { diff --git a/db/file_indexer_test.cc b/db/file_indexer_test.cc index 673d85a5c29..5cd8c2d2cf6 100644 --- a/db/file_indexer_test.cc +++ b/db/file_indexer_test.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -11,6 +11,7 @@ #include "db/file_indexer.h" #include "db/dbformat.h" #include "db/version_edit.h" +#include "port/stack_trace.h" #include "rocksdb/comparator.h" #include "util/testharness.h" #include "util/testutil.h" @@ -19,23 +20,29 @@ namespace rocksdb { class IntComparator : public Comparator { public: - int Compare(const Slice& a, const Slice& b) const { + int Compare(const Slice& a, const Slice& b) const override { assert(a.size() == 8); assert(b.size() == 8); - return *reinterpret_cast(a.data()) - - *reinterpret_cast(b.data()); + int64_t diff = *reinterpret_cast(a.data()) - + *reinterpret_cast(b.data()); + if (diff < 0) { + return -1; + } else if (diff == 0) { + return 0; + } else { + return 1; + } } - const char* Name() const { - return "IntComparator"; - } + const char* Name() const override { return "IntComparator"; } - void FindShortestSeparator(std::string* start, const Slice& limit) const {} + void FindShortestSeparator(std::string* start, + const Slice& limit) const override {} - void FindShortSuccessor(std::string* key) const {} + void FindShortSuccessor(std::string* key) const override {} }; -struct FileIndexerTest { +class FileIndexerTest : public testing::Test { public: FileIndexerTest() : kNumLevels(4), files(new std::vector[kNumLevels]) {} @@ -84,7 +91,7 @@ struct FileIndexerTest { }; // Case 0: Empty -TEST(FileIndexerTest, Empty) { +TEST_F(FileIndexerTest, Empty) { Arena arena; indexer = new FileIndexer(&ucmp); indexer->UpdateIndex(&arena, 0, files); @@ -92,9 +99,8 @@ TEST(FileIndexerTest, Empty) { } // Case 1: no overlap, files are on the left of next level files -TEST(FileIndexerTest, no_overlap_left) { +TEST_F(FileIndexerTest, no_overlap_left) { Arena arena; - uint32_t kNumLevels = 4; indexer = new FileIndexer(&ucmp); // level 1 AddFile(1, 100, 200); @@ -133,9 +139,8 @@ TEST(FileIndexerTest, no_overlap_left) { } // Case 2: no overlap, files are on the right of next level files -TEST(FileIndexerTest, no_overlap_right) { +TEST_F(FileIndexerTest, no_overlap_right) { Arena arena; - uint32_t kNumLevels = 4; indexer = new FileIndexer(&ucmp); // level 1 AddFile(1, 2100, 2200); @@ -176,9 +181,8 @@ TEST(FileIndexerTest, no_overlap_right) { } // Case 3: empty L2 -TEST(FileIndexerTest, empty_L2) { +TEST_F(FileIndexerTest, empty_L2) { Arena arena; - uint32_t kNumLevels = 4; indexer = new FileIndexer(&ucmp); for (uint32_t i = 1; i < kNumLevels; ++i) { ASSERT_EQ(0U, indexer->LevelIndexSize(i)); @@ -217,7 +221,7 @@ TEST(FileIndexerTest, empty_L2) { } // Case 4: mixed -TEST(FileIndexerTest, mixed) { +TEST_F(FileIndexerTest, mixed) { Arena arena; indexer = new FileIndexer(&ucmp); // level 1 @@ -340,5 +344,7 @@ TEST(FileIndexerTest, mixed) { } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/filename_test.cc b/db/filename_test.cc index 5a5f880e4d8..d6bde52834e 100644 --- a/db/filename_test.cc +++ b/db/filename_test.cc @@ -1,13 +1,13 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. -#include "db/filename.h" +#include "util/filename.h" #include "db/dbformat.h" #include "port/port.h" @@ -16,9 +16,9 @@ namespace rocksdb { -class FileNameTest { }; +class FileNameTest : public testing::Test {}; -TEST(FileNameTest, Parse) { +TEST_F(FileNameTest, Parse) { Slice db; FileType type; uint64_t number; @@ -105,7 +105,7 @@ TEST(FileNameTest, Parse) { }; } -TEST(FileNameTest, InfoLogFileName) { +TEST_F(FileNameTest, InfoLogFileName) { std::string dbname = ("/data/rocksdb"); std::string db_absolute_path; Env::Default()->GetAbsolutePath(dbname, &db_absolute_path); @@ -121,7 +121,7 @@ TEST(FileNameTest, InfoLogFileName) { OldInfoLogFileName(dbname, 666u, db_absolute_path, "/data/rocksdb_log")); } -TEST(FileNameTest, Construction) { +TEST_F(FileNameTest, Construction) { uint64_t number; FileType type; std::string fname; @@ -175,5 +175,6 @@ TEST(FileNameTest, Construction) { } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/flush_job.cc b/db/flush_job.cc new file mode 100644 index 00000000000..778c9eca124 --- /dev/null +++ b/db/flush_job.cc @@ -0,0 +1,362 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/flush_job.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include + +#include +#include + +#include "db/builder.h" +#include "db/db_iter.h" +#include "db/dbformat.h" +#include "db/event_helpers.h" +#include "db/log_reader.h" +#include "db/log_writer.h" +#include "db/memtable_list.h" +#include "db/merge_context.h" +#include "db/version_set.h" +#include "monitoring/iostats_context_imp.h" +#include "monitoring/perf_context_imp.h" +#include "monitoring/thread_status_util.h" +#include "port/likely.h" +#include "port/port.h" +#include "db/memtable.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/statistics.h" +#include "rocksdb/status.h" +#include "rocksdb/table.h" +#include "table/block.h" +#include "table/block_based_table_factory.h" +#include "table/merging_iterator.h" +#include "table/table_builder.h" +#include "table/two_level_iterator.h" +#include "util/coding.h" +#include "util/event_logger.h" +#include "util/file_util.h" +#include "util/filename.h" +#include "util/log_buffer.h" +#include "util/logging.h" +#include "util/mutexlock.h" +#include "util/stop_watch.h" +#include "util/sync_point.h" + +namespace rocksdb { + +FlushJob::FlushJob(const std::string& dbname, ColumnFamilyData* cfd, + const ImmutableDBOptions& db_options, + const MutableCFOptions& mutable_cf_options, + const EnvOptions& env_options, VersionSet* versions, + InstrumentedMutex* db_mutex, + std::atomic* shutting_down, + std::vector existing_snapshots, + SequenceNumber earliest_write_conflict_snapshot, + JobContext* job_context, LogBuffer* log_buffer, + Directory* db_directory, Directory* output_file_directory, + CompressionType output_compression, Statistics* stats, + EventLogger* event_logger, bool measure_io_stats) + : dbname_(dbname), + cfd_(cfd), + db_options_(db_options), + mutable_cf_options_(mutable_cf_options), + env_options_(env_options), + versions_(versions), + db_mutex_(db_mutex), + shutting_down_(shutting_down), + existing_snapshots_(std::move(existing_snapshots)), + earliest_write_conflict_snapshot_(earliest_write_conflict_snapshot), + job_context_(job_context), + log_buffer_(log_buffer), + db_directory_(db_directory), + output_file_directory_(output_file_directory), + output_compression_(output_compression), + stats_(stats), + event_logger_(event_logger), + measure_io_stats_(measure_io_stats), + pick_memtable_called(false) { + // Update the thread status to indicate flush. + ReportStartedFlush(); + TEST_SYNC_POINT("FlushJob::FlushJob()"); +} + +FlushJob::~FlushJob() { + ThreadStatusUtil::ResetThreadStatus(); +} + +void FlushJob::ReportStartedFlush() { + ThreadStatusUtil::SetColumnFamily(cfd_, cfd_->ioptions()->env, + db_options_.enable_thread_tracking); + ThreadStatusUtil::SetThreadOperation(ThreadStatus::OP_FLUSH); + ThreadStatusUtil::SetThreadOperationProperty( + ThreadStatus::COMPACTION_JOB_ID, + job_context_->job_id); + IOSTATS_RESET(bytes_written); +} + +void FlushJob::ReportFlushInputSize(const autovector& mems) { + uint64_t input_size = 0; + for (auto* mem : mems) { + input_size += mem->ApproximateMemoryUsage(); + } + ThreadStatusUtil::IncreaseThreadOperationProperty( + ThreadStatus::FLUSH_BYTES_MEMTABLES, + input_size); +} + +void FlushJob::RecordFlushIOStats() { + RecordTick(stats_, FLUSH_WRITE_BYTES, IOSTATS(bytes_written)); + ThreadStatusUtil::IncreaseThreadOperationProperty( + ThreadStatus::FLUSH_BYTES_WRITTEN, IOSTATS(bytes_written)); + IOSTATS_RESET(bytes_written); +} + +void FlushJob::PickMemTable() { + db_mutex_->AssertHeld(); + assert(!pick_memtable_called); + pick_memtable_called = true; + // Save the contents of the earliest memtable as a new Table + cfd_->imm()->PickMemtablesToFlush(&mems_); + if (mems_.empty()) { + return; + } + + ReportFlushInputSize(mems_); + + // entries mems are (implicitly) sorted in ascending order by their created + // time. We will use the first memtable's `edit` to keep the meta info for + // this flush. + MemTable* m = mems_[0]; + edit_ = m->GetEdits(); + edit_->SetPrevLogNumber(0); + // SetLogNumber(log_num) indicates logs with number smaller than log_num + // will no longer be picked up for recovery. + edit_->SetLogNumber(mems_.back()->GetNextLogNumber()); + edit_->SetColumnFamily(cfd_->GetID()); + + // path 0 for level 0 file. + meta_.fd = FileDescriptor(versions_->NewFileNumber(), 0, 0); + + base_ = cfd_->current(); + base_->Ref(); // it is likely that we do not need this reference +} + +Status FlushJob::Run(FileMetaData* file_meta) { + db_mutex_->AssertHeld(); + assert(pick_memtable_called); + AutoThreadOperationStageUpdater stage_run( + ThreadStatus::STAGE_FLUSH_RUN); + if (mems_.empty()) { + ROCKS_LOG_BUFFER(log_buffer_, "[%s] Nothing in memtable to flush", + cfd_->GetName().c_str()); + return Status::OK(); + } + + // I/O measurement variables + PerfLevel prev_perf_level = PerfLevel::kEnableTime; + uint64_t prev_write_nanos = 0; + uint64_t prev_fsync_nanos = 0; + uint64_t prev_range_sync_nanos = 0; + uint64_t prev_prepare_write_nanos = 0; + if (measure_io_stats_) { + prev_perf_level = GetPerfLevel(); + SetPerfLevel(PerfLevel::kEnableTime); + prev_write_nanos = IOSTATS(write_nanos); + prev_fsync_nanos = IOSTATS(fsync_nanos); + prev_range_sync_nanos = IOSTATS(range_sync_nanos); + prev_prepare_write_nanos = IOSTATS(prepare_write_nanos); + } + + // This will release and re-acquire the mutex. + Status s = WriteLevel0Table(); + + if (s.ok() && + (shutting_down_->load(std::memory_order_acquire) || cfd_->IsDropped())) { + s = Status::ShutdownInProgress( + "Database shutdown or Column family drop during flush"); + } + + if (!s.ok()) { + cfd_->imm()->RollbackMemtableFlush(mems_, meta_.fd.GetNumber()); + } else { + TEST_SYNC_POINT("FlushJob::InstallResults"); + // Replace immutable memtable with the generated Table + s = cfd_->imm()->InstallMemtableFlushResults( + cfd_, mutable_cf_options_, mems_, versions_, db_mutex_, + meta_.fd.GetNumber(), &job_context_->memtables_to_free, db_directory_, + log_buffer_); + } + + if (s.ok() && file_meta != nullptr) { + *file_meta = meta_; + } + RecordFlushIOStats(); + + auto stream = event_logger_->LogToBuffer(log_buffer_); + stream << "job" << job_context_->job_id << "event" + << "flush_finished"; + stream << "lsm_state"; + stream.StartArray(); + auto vstorage = cfd_->current()->storage_info(); + for (int level = 0; level < vstorage->num_levels(); ++level) { + stream << vstorage->NumLevelFiles(level); + } + stream.EndArray(); + stream << "immutable_memtables" << cfd_->imm()->NumNotFlushed(); + + if (measure_io_stats_) { + if (prev_perf_level != PerfLevel::kEnableTime) { + SetPerfLevel(prev_perf_level); + } + stream << "file_write_nanos" << (IOSTATS(write_nanos) - prev_write_nanos); + stream << "file_range_sync_nanos" + << (IOSTATS(range_sync_nanos) - prev_range_sync_nanos); + stream << "file_fsync_nanos" << (IOSTATS(fsync_nanos) - prev_fsync_nanos); + stream << "file_prepare_write_nanos" + << (IOSTATS(prepare_write_nanos) - prev_prepare_write_nanos); + } + + return s; +} + +void FlushJob::Cancel() { + db_mutex_->AssertHeld(); + assert(base_ != nullptr); + base_->Unref(); +} + +Status FlushJob::WriteLevel0Table() { + AutoThreadOperationStageUpdater stage_updater( + ThreadStatus::STAGE_FLUSH_WRITE_L0); + db_mutex_->AssertHeld(); + const uint64_t start_micros = db_options_.env->NowMicros(); + Status s; + { + db_mutex_->Unlock(); + if (log_buffer_) { + log_buffer_->FlushBufferToLog(); + } + // memtables and range_del_iters store internal iterators over each data + // memtable and its associated range deletion memtable, respectively, at + // corresponding indexes. + std::vector memtables; + std::vector range_del_iters; + ReadOptions ro; + ro.total_order_seek = true; + Arena arena; + uint64_t total_num_entries = 0, total_num_deletes = 0; + size_t total_memory_usage = 0; + for (MemTable* m : mems_) { + ROCKS_LOG_INFO( + db_options_.info_log, + "[%s] [JOB %d] Flushing memtable with next log file: %" PRIu64 "\n", + cfd_->GetName().c_str(), job_context_->job_id, m->GetNextLogNumber()); + memtables.push_back(m->NewIterator(ro, &arena)); + auto* range_del_iter = m->NewRangeTombstoneIterator(ro); + if (range_del_iter != nullptr) { + range_del_iters.push_back(range_del_iter); + } + total_num_entries += m->num_entries(); + total_num_deletes += m->num_deletes(); + total_memory_usage += m->ApproximateMemoryUsage(); + } + + event_logger_->Log() << "job" << job_context_->job_id << "event" + << "flush_started" + << "num_memtables" << mems_.size() << "num_entries" + << total_num_entries << "num_deletes" + << total_num_deletes << "memory_usage" + << total_memory_usage; + + { + ScopedArenaIterator iter( + NewMergingIterator(&cfd_->internal_comparator(), &memtables[0], + static_cast(memtables.size()), &arena)); + std::unique_ptr range_del_iter(NewMergingIterator( + &cfd_->internal_comparator(), + range_del_iters.empty() ? nullptr : &range_del_iters[0], + static_cast(range_del_iters.size()))); + ROCKS_LOG_INFO(db_options_.info_log, + "[%s] [JOB %d] Level-0 flush table #%" PRIu64 ": started", + cfd_->GetName().c_str(), job_context_->job_id, + meta_.fd.GetNumber()); + + TEST_SYNC_POINT_CALLBACK("FlushJob::WriteLevel0Table:output_compression", + &output_compression_); + EnvOptions optimized_env_options = + db_options_.env->OptimizeForCompactionTableWrite(env_options_, db_options_); + + int64_t _current_time = 0; + db_options_.env->GetCurrentTime(&_current_time); // ignore error + const uint64_t current_time = static_cast(_current_time); + + uint64_t oldest_key_time = mems_.front()->ApproximateOldestKeyTime(); + + s = BuildTable( + dbname_, db_options_.env, *cfd_->ioptions(), mutable_cf_options_, + optimized_env_options, cfd_->table_cache(), iter.get(), + std::move(range_del_iter), &meta_, cfd_->internal_comparator(), + cfd_->int_tbl_prop_collector_factories(), cfd_->GetID(), + cfd_->GetName(), existing_snapshots_, + earliest_write_conflict_snapshot_, output_compression_, + cfd_->ioptions()->compression_opts, + mutable_cf_options_.paranoid_file_checks, cfd_->internal_stats(), + TableFileCreationReason::kFlush, event_logger_, job_context_->job_id, + Env::IO_HIGH, &table_properties_, 0 /* level */, current_time, + oldest_key_time); + LogFlush(db_options_.info_log); + } + ROCKS_LOG_INFO(db_options_.info_log, + "[%s] [JOB %d] Level-0 flush table #%" PRIu64 ": %" PRIu64 + " bytes %s" + "%s", + cfd_->GetName().c_str(), job_context_->job_id, + meta_.fd.GetNumber(), meta_.fd.GetFileSize(), + s.ToString().c_str(), + meta_.marked_for_compaction ? " (needs compaction)" : ""); + + if (output_file_directory_ != nullptr) { + output_file_directory_->Fsync(); + } + TEST_SYNC_POINT("FlushJob::WriteLevel0Table"); + db_mutex_->Lock(); + } + base_->Unref(); + + // Note that if file_size is zero, the file has been deleted and + // should not be added to the manifest. + if (s.ok() && meta_.fd.GetFileSize() > 0) { + // if we have more than 1 background thread, then we cannot + // insert files directly into higher levels because some other + // threads could be concurrently producing compacted files for + // that key range. + // Add file to L0 + edit_->AddFile(0 /* level */, meta_.fd.GetNumber(), meta_.fd.GetPathId(), + meta_.fd.GetFileSize(), meta_.smallest, meta_.largest, + meta_.smallest_seqno, meta_.largest_seqno, + meta_.marked_for_compaction); + } + + // Note that here we treat flush as level 0 compaction in internal stats + InternalStats::CompactionStats stats(1); + stats.micros = db_options_.env->NowMicros() - start_micros; + stats.bytes_written = meta_.fd.GetFileSize(); + cfd_->internal_stats()->AddCompactionStats(0 /* level */, stats); + cfd_->internal_stats()->AddCFStats(InternalStats::BYTES_FLUSHED, + meta_.fd.GetFileSize()); + RecordFlushIOStats(); + return s; +} + +} // namespace rocksdb diff --git a/db/flush_job.h b/db/flush_job.h new file mode 100644 index 00000000000..4698ae7b03d --- /dev/null +++ b/db/flush_job.h @@ -0,0 +1,110 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#pragma once + +#include +#include +#include +#include +#include +#include +#include + +#include "db/column_family.h" +#include "db/dbformat.h" +#include "db/flush_scheduler.h" +#include "db/internal_stats.h" +#include "db/job_context.h" +#include "db/log_writer.h" +#include "db/memtable_list.h" +#include "db/snapshot_impl.h" +#include "db/version_edit.h" +#include "db/write_controller.h" +#include "db/write_thread.h" +#include "monitoring/instrumented_mutex.h" +#include "options/db_options.h" +#include "port/port.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/transaction_log.h" +#include "table/scoped_arena_iterator.h" +#include "util/autovector.h" +#include "util/event_logger.h" +#include "util/stop_watch.h" +#include "util/thread_local.h" + +namespace rocksdb { + +class MemTable; +class TableCache; +class Version; +class VersionEdit; +class VersionSet; +class Arena; + +class FlushJob { + public: + // TODO(icanadi) make effort to reduce number of parameters here + // IMPORTANT: mutable_cf_options needs to be alive while FlushJob is alive + FlushJob(const std::string& dbname, ColumnFamilyData* cfd, + const ImmutableDBOptions& db_options, + const MutableCFOptions& mutable_cf_options, + const EnvOptions& env_options, VersionSet* versions, + InstrumentedMutex* db_mutex, std::atomic* shutting_down, + std::vector existing_snapshots, + SequenceNumber earliest_write_conflict_snapshot, + JobContext* job_context, LogBuffer* log_buffer, + Directory* db_directory, Directory* output_file_directory, + CompressionType output_compression, Statistics* stats, + EventLogger* event_logger, bool measure_io_stats); + + ~FlushJob(); + + // Require db_mutex held. + // Once PickMemTable() is called, either Run() or Cancel() has to be called. + void PickMemTable(); + Status Run(FileMetaData* file_meta = nullptr); + void Cancel(); + TableProperties GetTableProperties() const { return table_properties_; } + + private: + void ReportStartedFlush(); + void ReportFlushInputSize(const autovector& mems); + void RecordFlushIOStats(); + Status WriteLevel0Table(); + const std::string& dbname_; + ColumnFamilyData* cfd_; + const ImmutableDBOptions& db_options_; + const MutableCFOptions& mutable_cf_options_; + const EnvOptions& env_options_; + VersionSet* versions_; + InstrumentedMutex* db_mutex_; + std::atomic* shutting_down_; + std::vector existing_snapshots_; + SequenceNumber earliest_write_conflict_snapshot_; + JobContext* job_context_; + LogBuffer* log_buffer_; + Directory* db_directory_; + Directory* output_file_directory_; + CompressionType output_compression_; + Statistics* stats_; + EventLogger* event_logger_; + TableProperties table_properties_; + bool measure_io_stats_; + + // Variables below are set by PickMemTable(): + FileMetaData meta_; + autovector mems_; + VersionEdit* edit_; + Version* base_; + bool pick_memtable_called; +}; + +} // namespace rocksdb diff --git a/db/flush_job_test.cc b/db/flush_job_test.cc new file mode 100644 index 00000000000..34a3c983c33 --- /dev/null +++ b/db/flush_job_test.cc @@ -0,0 +1,223 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include +#include +#include + +#include "db/column_family.h" +#include "db/flush_job.h" +#include "db/version_set.h" +#include "rocksdb/cache.h" +#include "rocksdb/write_buffer_manager.h" +#include "table/mock_table.h" +#include "util/file_reader_writer.h" +#include "util/string_util.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +// TODO(icanadi) Mock out everything else: +// 1. VersionSet +// 2. Memtable +class FlushJobTest : public testing::Test { + public: + FlushJobTest() + : env_(Env::Default()), + dbname_(test::TmpDir() + "/flush_job_test"), + options_(), + db_options_(options_), + table_cache_(NewLRUCache(50000, 16)), + write_buffer_manager_(db_options_.db_write_buffer_size), + versions_(new VersionSet(dbname_, &db_options_, env_options_, + table_cache_.get(), &write_buffer_manager_, + &write_controller_)), + shutting_down_(false), + mock_table_factory_(new mock::MockTableFactory()) { + EXPECT_OK(env_->CreateDirIfMissing(dbname_)); + db_options_.db_paths.emplace_back(dbname_, + std::numeric_limits::max()); + // TODO(icanadi) Remove this once we mock out VersionSet + NewDB(); + std::vector column_families; + cf_options_.table_factory = mock_table_factory_; + column_families.emplace_back(kDefaultColumnFamilyName, cf_options_); + + EXPECT_OK(versions_->Recover(column_families, false)); + } + + void NewDB() { + VersionEdit new_db; + new_db.SetLogNumber(0); + new_db.SetNextFile(2); + new_db.SetLastSequence(0); + + const std::string manifest = DescriptorFileName(dbname_, 1); + unique_ptr file; + Status s = env_->NewWritableFile( + manifest, &file, env_->OptimizeForManifestWrite(env_options_)); + ASSERT_OK(s); + unique_ptr file_writer( + new WritableFileWriter(std::move(file), EnvOptions())); + { + log::Writer log(std::move(file_writer), 0, false); + std::string record; + new_db.EncodeTo(&record); + s = log.AddRecord(record); + } + ASSERT_OK(s); + // Make "CURRENT" file that points to the new manifest file. + s = SetCurrentFile(env_, dbname_, 1, nullptr); + } + + Env* env_; + std::string dbname_; + EnvOptions env_options_; + Options options_; + ImmutableDBOptions db_options_; + std::shared_ptr table_cache_; + WriteController write_controller_; + WriteBufferManager write_buffer_manager_; + ColumnFamilyOptions cf_options_; + std::unique_ptr versions_; + InstrumentedMutex mutex_; + std::atomic shutting_down_; + std::shared_ptr mock_table_factory_; +}; + +TEST_F(FlushJobTest, Empty) { + JobContext job_context(0); + auto cfd = versions_->GetColumnFamilySet()->GetDefault(); + EventLogger event_logger(db_options_.info_log.get()); + FlushJob flush_job(dbname_, versions_->GetColumnFamilySet()->GetDefault(), + db_options_, *cfd->GetLatestMutableCFOptions(), + env_options_, versions_.get(), &mutex_, &shutting_down_, + {}, kMaxSequenceNumber, &job_context, nullptr, nullptr, + nullptr, kNoCompression, nullptr, &event_logger, false); + { + InstrumentedMutexLock l(&mutex_); + flush_job.PickMemTable(); + ASSERT_OK(flush_job.Run()); + } + job_context.Clean(); +} + +TEST_F(FlushJobTest, NonEmpty) { + JobContext job_context(0); + auto cfd = versions_->GetColumnFamilySet()->GetDefault(); + auto new_mem = cfd->ConstructNewMemtable(*cfd->GetLatestMutableCFOptions(), + kMaxSequenceNumber); + new_mem->Ref(); + auto inserted_keys = mock::MakeMockFile(); + // Test data: + // seqno [ 1, 2 ... 8998, 8999, 9000, 9001, 9002 ... 9999 ] + // key [ 1001, 1002 ... 9998, 9999, 0, 1, 2 ... 999 ] + // range-delete "9995" -> "9999" at seqno 10000 + for (int i = 1; i < 10000; ++i) { + std::string key(ToString((i + 1000) % 10000)); + std::string value("value" + key); + new_mem->Add(SequenceNumber(i), kTypeValue, key, value); + if ((i + 1000) % 10000 < 9995) { + InternalKey internal_key(key, SequenceNumber(i), kTypeValue); + inserted_keys.insert({internal_key.Encode().ToString(), value}); + } + } + new_mem->Add(SequenceNumber(10000), kTypeRangeDeletion, "9995", "9999a"); + InternalKey internal_key("9995", SequenceNumber(10000), kTypeRangeDeletion); + inserted_keys.insert({internal_key.Encode().ToString(), "9999a"}); + + autovector to_delete; + cfd->imm()->Add(new_mem, &to_delete); + for (auto& m : to_delete) { + delete m; + } + + EventLogger event_logger(db_options_.info_log.get()); + FlushJob flush_job(dbname_, versions_->GetColumnFamilySet()->GetDefault(), + db_options_, *cfd->GetLatestMutableCFOptions(), + env_options_, versions_.get(), &mutex_, &shutting_down_, + {}, kMaxSequenceNumber, &job_context, nullptr, nullptr, + nullptr, kNoCompression, nullptr, &event_logger, true); + FileMetaData fd; + mutex_.Lock(); + flush_job.PickMemTable(); + ASSERT_OK(flush_job.Run(&fd)); + mutex_.Unlock(); + ASSERT_EQ(ToString(0), fd.smallest.user_key().ToString()); + ASSERT_EQ("9999a", + fd.largest.user_key().ToString()); // range tombstone end key + ASSERT_EQ(1, fd.smallest_seqno); + ASSERT_EQ(10000, fd.largest_seqno); // range tombstone seqnum 10000 + mock_table_factory_->AssertSingleFile(inserted_keys); + job_context.Clean(); +} + +TEST_F(FlushJobTest, Snapshots) { + JobContext job_context(0); + auto cfd = versions_->GetColumnFamilySet()->GetDefault(); + auto new_mem = cfd->ConstructNewMemtable(*cfd->GetLatestMutableCFOptions(), + kMaxSequenceNumber); + + std::vector snapshots; + std::set snapshots_set; + int keys = 10000; + int max_inserts_per_keys = 8; + + Random rnd(301); + for (int i = 0; i < keys / 2; ++i) { + snapshots.push_back(rnd.Uniform(keys * (max_inserts_per_keys / 2)) + 1); + snapshots_set.insert(snapshots.back()); + } + std::sort(snapshots.begin(), snapshots.end()); + + new_mem->Ref(); + SequenceNumber current_seqno = 0; + auto inserted_keys = mock::MakeMockFile(); + for (int i = 1; i < keys; ++i) { + std::string key(ToString(i)); + int insertions = rnd.Uniform(max_inserts_per_keys); + for (int j = 0; j < insertions; ++j) { + std::string value(test::RandomHumanReadableString(&rnd, 10)); + auto seqno = ++current_seqno; + new_mem->Add(SequenceNumber(seqno), kTypeValue, key, value); + // a key is visible only if: + // 1. it's the last one written (j == insertions - 1) + // 2. there's a snapshot pointing at it + bool visible = (j == insertions - 1) || + (snapshots_set.find(seqno) != snapshots_set.end()); + if (visible) { + InternalKey internal_key(key, seqno, kTypeValue); + inserted_keys.insert({internal_key.Encode().ToString(), value}); + } + } + } + + autovector to_delete; + cfd->imm()->Add(new_mem, &to_delete); + for (auto& m : to_delete) { + delete m; + } + + EventLogger event_logger(db_options_.info_log.get()); + FlushJob flush_job( + dbname_, versions_->GetColumnFamilySet()->GetDefault(), db_options_, + *cfd->GetLatestMutableCFOptions(), env_options_, versions_.get(), &mutex_, + &shutting_down_, snapshots, kMaxSequenceNumber, &job_context, nullptr, + nullptr, nullptr, kNoCompression, nullptr, &event_logger, true); + mutex_.Lock(); + flush_job.PickMemTable(); + ASSERT_OK(flush_job.Run()); + mutex_.Unlock(); + mock_table_factory_->AssertSingleFile(inserted_keys); + job_context.Clean(); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/flush_scheduler.cc b/db/flush_scheduler.cc new file mode 100644 index 00000000000..8735a6b369b --- /dev/null +++ b/db/flush_scheduler.cc @@ -0,0 +1,88 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/flush_scheduler.h" + +#include + +#include "db/column_family.h" + +namespace rocksdb { + +void FlushScheduler::ScheduleFlush(ColumnFamilyData* cfd) { +#ifndef NDEBUG + std::lock_guard lock(checking_mutex_); + assert(checking_set_.count(cfd) == 0); + checking_set_.insert(cfd); +#endif // NDEBUG + cfd->Ref(); +// Suppress false positive clang analyzer warnings. +#ifndef __clang_analyzer__ + Node* node = new Node{cfd, head_.load(std::memory_order_relaxed)}; + while (!head_.compare_exchange_strong( + node->next, node, std::memory_order_relaxed, std::memory_order_relaxed)) { + // failing CAS updates the first param, so we are already set for + // retry. TakeNextColumnFamily won't happen until after another + // inter-thread synchronization, so we don't even need release + // semantics for this CAS + } +#endif // __clang_analyzer__ +} + +ColumnFamilyData* FlushScheduler::TakeNextColumnFamily() { +#ifndef NDEBUG + std::lock_guard lock(checking_mutex_); +#endif // NDEBUG + while (true) { + if (head_.load(std::memory_order_relaxed) == nullptr) { + return nullptr; + } + + // dequeue the head + Node* node = head_.load(std::memory_order_relaxed); + head_.store(node->next, std::memory_order_relaxed); + ColumnFamilyData* cfd = node->column_family; + delete node; + +#ifndef NDEBUG + auto iter = checking_set_.find(cfd); + assert(iter != checking_set_.end()); + checking_set_.erase(iter); +#endif // NDEBUG + + if (!cfd->IsDropped()) { + // success + return cfd; + } + + // no longer relevant, retry + if (cfd->Unref()) { + delete cfd; + } + } +} + +bool FlushScheduler::Empty() { +#ifndef NDEBUG + std::lock_guard lock(checking_mutex_); +#endif // NDEBUG + auto rv = head_.load(std::memory_order_relaxed) == nullptr; +#ifndef NDEBUG + assert(rv == checking_set_.empty()); +#endif // NDEBUG + return rv; +} + +void FlushScheduler::Clear() { + ColumnFamilyData* cfd; + while ((cfd = TakeNextColumnFamily()) != nullptr) { + if (cfd->Unref()) { + delete cfd; + } + } + assert(head_.load(std::memory_order_relaxed) == nullptr); +} + +} // namespace rocksdb diff --git a/db/flush_scheduler.h b/db/flush_scheduler.h new file mode 100644 index 00000000000..cd3575861a8 --- /dev/null +++ b/db/flush_scheduler.h @@ -0,0 +1,48 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include +#include +#include + +namespace rocksdb { + +class ColumnFamilyData; + +// Unless otherwise noted, all methods on FlushScheduler should be called +// only with the DB mutex held or from a single-threaded recovery context. +class FlushScheduler { + public: + FlushScheduler() : head_(nullptr) {} + + // May be called from multiple threads at once, but not concurrent with + // any other method calls on this instance + void ScheduleFlush(ColumnFamilyData* cfd); + + // Removes and returns Ref()-ed column family. Client needs to Unref(). + // Filters column families that have been dropped. + ColumnFamilyData* TakeNextColumnFamily(); + + bool Empty(); + + void Clear(); + + private: + struct Node { + ColumnFamilyData* column_family; + Node* next; + }; + + std::atomic head_; +#ifndef NDEBUG + std::mutex checking_mutex_; + std::set checking_set_; +#endif // NDEBUG +}; + +} // namespace rocksdb diff --git a/db/forward_iterator.cc b/db/forward_iterator.cc index 74e6dd24924..65fff95956d 100644 --- a/db/forward_iterator.cc +++ b/db/forward_iterator.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef ROCKSDB_LITE #include "db/forward_iterator.h" @@ -10,14 +10,17 @@ #include #include +#include "db/column_family.h" #include "db/db_impl.h" #include "db/db_iter.h" -#include "db/column_family.h" +#include "db/dbformat.h" +#include "db/job_context.h" #include "rocksdb/env.h" #include "rocksdb/slice.h" #include "rocksdb/slice_transform.h" -#include "table/merger.h" -#include "db/dbformat.h" +#include "table/merging_iterator.h" +#include "util/string_util.h" +#include "util/sync_point.h" namespace rocksdb { @@ -26,13 +29,27 @@ namespace rocksdb { // iter.SetFileIndex(file_index); // iter.Seek(target); // iter.Next() -class LevelIterator : public Iterator { +class LevelIterator : public InternalIterator { public: LevelIterator(const ColumnFamilyData* const cfd, - const ReadOptions& read_options, - const std::vector& files) - : cfd_(cfd), read_options_(read_options), files_(files), valid_(false), - file_index_(std::numeric_limits::max()) {} + const ReadOptions& read_options, + const std::vector& files) + : cfd_(cfd), + read_options_(read_options), + files_(files), + valid_(false), + file_index_(std::numeric_limits::max()), + file_iter_(nullptr), + pinned_iters_mgr_(nullptr) {} + + ~LevelIterator() { + // Reset current pointer + if (pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled()) { + pinned_iters_mgr_->PinIterator(file_iter_); + } else { + delete file_iter_; + } + } void SetFileIndex(uint32_t file_index) { assert(file_index < files_.size()); @@ -44,15 +61,33 @@ class LevelIterator : public Iterator { } void Reset() { assert(file_index_ < files_.size()); - file_iter_.reset(cfd_->table_cache()->NewIterator( + + // Reset current pointer + if (pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled()) { + pinned_iters_mgr_->PinIterator(file_iter_); + } else { + delete file_iter_; + } + + RangeDelAggregator range_del_agg( + cfd_->internal_comparator(), {} /* snapshots */); + file_iter_ = cfd_->table_cache()->NewIterator( read_options_, *(cfd_->soptions()), cfd_->internal_comparator(), - files_[file_index_]->fd, nullptr /* table_reader_ptr */, false)); + files_[file_index_]->fd, + read_options_.ignore_range_deletions ? nullptr : &range_del_agg, + nullptr /* table_reader_ptr */, nullptr, false); + file_iter_->SetPinnedItersMgr(pinned_iters_mgr_); + if (!range_del_agg.IsEmpty()) { + status_ = Status::NotSupported( + "Range tombstones unsupported with ForwardIterator"); + valid_ = false; + } } void SeekToLast() override { status_ = Status::NotSupported("LevelIterator::SeekToLast()"); valid_ = false; } - void Prev() { + void Prev() override { status_ = Status::NotSupported("LevelIterator::Prev()"); valid_ = false; } @@ -69,6 +104,10 @@ class LevelIterator : public Iterator { file_iter_->Seek(internal_key); valid_ = file_iter_->Valid(); } + void SeekForPrev(const Slice& internal_key) override { + status_ = Status::NotSupported("LevelIterator::SeekForPrev()"); + valid_ = false; + } void Next() override { assert(valid_); file_iter_->Next(); @@ -101,6 +140,20 @@ class LevelIterator : public Iterator { } return Status::OK(); } + bool IsKeyPinned() const override { + return pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled() && + file_iter_->IsKeyPinned(); + } + bool IsValuePinned() const override { + return pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled() && + file_iter_->IsValuePinned(); + } + void SetPinnedItersMgr(PinnedIteratorsManager* pinned_iters_mgr) override { + pinned_iters_mgr_ = pinned_iters_mgr; + if (file_iter_) { + file_iter_->SetPinnedItersMgr(pinned_iters_mgr_); + } + } private: const ColumnFamilyData* const cfd_; @@ -110,74 +163,128 @@ class LevelIterator : public Iterator { bool valid_; uint32_t file_index_; Status status_; - std::unique_ptr file_iter_; + InternalIterator* file_iter_; + PinnedIteratorsManager* pinned_iters_mgr_; }; ForwardIterator::ForwardIterator(DBImpl* db, const ReadOptions& read_options, - ColumnFamilyData* cfd) + ColumnFamilyData* cfd, + SuperVersion* current_sv) : db_(db), read_options_(read_options), cfd_(cfd), - prefix_extractor_(cfd->options()->prefix_extractor.get()), + prefix_extractor_(cfd->ioptions()->prefix_extractor), user_comparator_(cfd->user_comparator()), immutable_min_heap_(MinIterComparator(&cfd_->internal_comparator())), - sv_(nullptr), + sv_(current_sv), mutable_iter_(nullptr), current_(nullptr), valid_(false), - is_prev_set_(false) {} + status_(Status::OK()), + immutable_status_(Status::OK()), + has_iter_trimmed_for_upper_bound_(false), + current_over_upper_bound_(false), + is_prev_set_(false), + is_prev_inclusive_(false), + pinned_iters_mgr_(nullptr) { + if (sv_) { + RebuildIterators(false); + } +} ForwardIterator::~ForwardIterator() { - Cleanup(); + Cleanup(true); +} + +namespace { +// Used in PinnedIteratorsManager to release pinned SuperVersion +static void ReleaseSuperVersionFunc(void* sv) { + delete reinterpret_cast(sv); +} +} // namespace + +void ForwardIterator::SVCleanup() { + if (sv_ != nullptr && sv_->Unref()) { + // Job id == 0 means that this is not our background process, but rather + // user thread + JobContext job_context(0); + db_->mutex_.Lock(); + sv_->Cleanup(); + db_->FindObsoleteFiles(&job_context, false, true); + if (read_options_.background_purge_on_iterator_cleanup) { + db_->ScheduleBgLogWriterClose(&job_context); + } + db_->mutex_.Unlock(); + if (pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled()) { + pinned_iters_mgr_->PinPtr(sv_, &ReleaseSuperVersionFunc); + } else { + delete sv_; + } + if (job_context.HaveSomethingToDelete()) { + db_->PurgeObsoleteFiles( + job_context, read_options_.background_purge_on_iterator_cleanup); + } + job_context.Clean(); + } } -void ForwardIterator::Cleanup() { - delete mutable_iter_; +void ForwardIterator::Cleanup(bool release_sv) { + if (mutable_iter_ != nullptr) { + DeleteIterator(mutable_iter_, true /* is_arena */); + } + for (auto* m : imm_iters_) { - delete m; + DeleteIterator(m, true /* is_arena */); } imm_iters_.clear(); + for (auto* f : l0_iters_) { - delete f; + DeleteIterator(f); } l0_iters_.clear(); + for (auto* l : level_iters_) { - delete l; + DeleteIterator(l); } level_iters_.clear(); - if (sv_ != nullptr && sv_->Unref()) { - DBImpl::DeletionState deletion_state; - db_->mutex_.Lock(); - sv_->Cleanup(); - db_->FindObsoleteFiles(deletion_state, false, true); - db_->mutex_.Unlock(); - delete sv_; - if (deletion_state.HaveSomethingToDelete()) { - db_->PurgeObsoleteFiles(deletion_state); - } + if (release_sv) { + SVCleanup(); } } bool ForwardIterator::Valid() const { - return valid_; + // See UpdateCurrent(). + return valid_ ? !current_over_upper_bound_ : false; } void ForwardIterator::SeekToFirst() { - if (sv_ == nullptr || - sv_ ->version_number != cfd_->GetSuperVersionNumber()) { - RebuildIterators(); - } else if (status_.IsIncomplete()) { + if (sv_ == nullptr) { + RebuildIterators(true); + } else if (sv_->version_number != cfd_->GetSuperVersionNumber()) { + RenewIterators(); + } else if (immutable_status_.IsIncomplete()) { ResetIncompleteIterators(); } SeekInternal(Slice(), true); } +bool ForwardIterator::IsOverUpperBound(const Slice& internal_key) const { + return !(read_options_.iterate_upper_bound == nullptr || + cfd_->internal_comparator().user_comparator()->Compare( + ExtractUserKey(internal_key), + *read_options_.iterate_upper_bound) < 0); +} + void ForwardIterator::Seek(const Slice& internal_key) { - if (sv_ == nullptr || - sv_ ->version_number != cfd_->GetSuperVersionNumber()) { - RebuildIterators(); - } else if (status_.IsIncomplete()) { + if (IsOverUpperBound(internal_key)) { + valid_ = false; + } + if (sv_ == nullptr) { + RebuildIterators(true); + } else if (sv_->version_number != cfd_->GetSuperVersionNumber()) { + RenewIterators(); + } else if (immutable_status_.IsIncomplete()) { ResetIncompleteIterators(); } SeekInternal(internal_key, false); @@ -185,6 +292,7 @@ void ForwardIterator::Seek(const Slice& internal_key) { void ForwardIterator::SeekInternal(const Slice& internal_key, bool seek_to_first) { + assert(mutable_iter_); // mutable seek_to_first ? mutable_iter_->SeekToFirst() : mutable_iter_->Seek(internal_key); @@ -194,13 +302,32 @@ void ForwardIterator::SeekInternal(const Slice& internal_key, // if it turns to need to seek immutable often. We probably want to have // an option to turn it off. if (seek_to_first || NeedToSeekImmutable(internal_key)) { + immutable_status_ = Status::OK(); + if (has_iter_trimmed_for_upper_bound_ && + ( + // prev_ is not set yet + is_prev_set_ == false || + // We are doing SeekToFirst() and internal_key.size() = 0 + seek_to_first || + // prev_key_ > internal_key + cfd_->internal_comparator().InternalKeyComparator::Compare( + prev_key_.GetInternalKey(), internal_key) > 0)) { + // Some iterators are trimmed. Need to rebuild. + RebuildIterators(true); + // Already seeked mutable iter, so seek again + seek_to_first ? mutable_iter_->SeekToFirst() + : mutable_iter_->Seek(internal_key); + } { auto tmp = MinIterHeap(MinIterComparator(&cfd_->internal_comparator())); immutable_min_heap_.swap(tmp); } - for (auto* m : imm_iters_) { + for (size_t i = 0; i < imm_iters_.size(); i++) { + auto* m = imm_iters_[i]; seek_to_first ? m->SeekToFirst() : m->Seek(internal_key); - if (m->Valid()) { + if (!m->status().ok()) { + immutable_status_ = m->status(); + } else if (m->Valid()) { immutable_min_heap_.push(m); } } @@ -209,155 +336,153 @@ void ForwardIterator::SeekInternal(const Slice& internal_key, if (!seek_to_first) { user_key = ExtractUserKey(internal_key); } - auto* files = sv_->current->files_; - for (uint32_t i = 0; i < files[0].size(); ++i) { + const VersionStorageInfo* vstorage = sv_->current->storage_info(); + const std::vector& l0 = vstorage->LevelFiles(0); + for (size_t i = 0; i < l0.size(); ++i) { + if (!l0_iters_[i]) { + continue; + } if (seek_to_first) { l0_iters_[i]->SeekToFirst(); } else { // If the target key passes over the larget key, we are sure Next() // won't go over this file. if (user_comparator_->Compare(user_key, - files[0][i]->largest.user_key()) > 0) { + l0[i]->largest.user_key()) > 0) { + if (read_options_.iterate_upper_bound != nullptr) { + has_iter_trimmed_for_upper_bound_ = true; + DeleteIterator(l0_iters_[i]); + l0_iters_[i] = nullptr; + } continue; } l0_iters_[i]->Seek(internal_key); } - if (l0_iters_[i]->status().IsIncomplete()) { - // if any of the immutable iterators is incomplete (no-io option was - // used), we are unable to reliably find the smallest key - assert(read_options_.read_tier == kBlockCacheTier); - status_ = l0_iters_[i]->status(); - valid_ = false; - return; + if (!l0_iters_[i]->status().ok()) { + immutable_status_ = l0_iters_[i]->status(); } else if (l0_iters_[i]->Valid()) { - immutable_min_heap_.push(l0_iters_[i]); + if (!IsOverUpperBound(l0_iters_[i]->key())) { + immutable_min_heap_.push(l0_iters_[i]); + } else { + has_iter_trimmed_for_upper_bound_ = true; + DeleteIterator(l0_iters_[i]); + l0_iters_[i] = nullptr; + } } } - int32_t search_left_bound = 0; - int32_t search_right_bound = FileIndexer::kLevelMaxIndex; - for (int32_t level = 1; level < sv_->current->NumberLevels(); ++level) { - if (files[level].empty()) { - search_left_bound = 0; - search_right_bound = FileIndexer::kLevelMaxIndex; + for (int32_t level = 1; level < vstorage->num_levels(); ++level) { + const std::vector& level_files = + vstorage->LevelFiles(level); + if (level_files.empty()) { + continue; + } + if (level_iters_[level - 1] == nullptr) { continue; } - assert(level_iters_[level - 1] != nullptr); uint32_t f_idx = 0; if (!seek_to_first) { - // TODO(ljin): remove before committing - // f_idx = FindFileInRange( - // files[level], internal_key, 0, files[level].size()); - - if (search_left_bound == search_right_bound) { - f_idx = search_left_bound; - } else if (search_left_bound < search_right_bound) { - f_idx = FindFileInRange( - files[level], internal_key, search_left_bound, - search_right_bound == FileIndexer::kLevelMaxIndex ? - files[level].size() : search_right_bound); - } else { - // search_left_bound > search_right_bound - // There are only 2 cases this can happen: - // (1) target key is smaller than left most file - // (2) target key is larger than right most file - assert(search_left_bound == (int32_t)files[level].size() || - search_right_bound == -1); - if (search_right_bound == -1) { - assert(search_left_bound == 0); - f_idx = 0; - } else { - sv_->current->file_indexer_.GetNextLevelIndex( - level, files[level].size() - 1, - 1, 1, &search_left_bound, &search_right_bound); - continue; - } - } - - // Prepare hints for the next level - if (f_idx < files[level].size()) { - int cmp_smallest = user_comparator_->Compare( - user_key, files[level][f_idx]->smallest.user_key()); - int cmp_largest = -1; - if (cmp_smallest >= 0) { - cmp_smallest = user_comparator_->Compare( - user_key, files[level][f_idx]->smallest.user_key()); - } - sv_->current->file_indexer_.GetNextLevelIndex(level, f_idx, - cmp_smallest, cmp_largest, - &search_left_bound, &search_right_bound); - } else { - sv_->current->file_indexer_.GetNextLevelIndex( - level, files[level].size() - 1, - 1, 1, &search_left_bound, &search_right_bound); - } + f_idx = FindFileInRange(level_files, internal_key, 0, + static_cast(level_files.size())); } // Seek - if (f_idx < files[level].size()) { + if (f_idx < level_files.size()) { level_iters_[level - 1]->SetFileIndex(f_idx); seek_to_first ? level_iters_[level - 1]->SeekToFirst() : level_iters_[level - 1]->Seek(internal_key); - if (level_iters_[level - 1]->status().IsIncomplete()) { - // see above - assert(read_options_.read_tier == kBlockCacheTier); - status_ = level_iters_[level - 1]->status(); - valid_ = false; - return; + if (!level_iters_[level - 1]->status().ok()) { + immutable_status_ = level_iters_[level - 1]->status(); } else if (level_iters_[level - 1]->Valid()) { - immutable_min_heap_.push(level_iters_[level - 1]); + if (!IsOverUpperBound(level_iters_[level - 1]->key())) { + immutable_min_heap_.push(level_iters_[level - 1]); + } else { + // Nothing in this level is interesting. Remove. + has_iter_trimmed_for_upper_bound_ = true; + DeleteIterator(level_iters_[level - 1]); + level_iters_[level - 1] = nullptr; + } } } } - if (seek_to_first || immutable_min_heap_.empty()) { + if (seek_to_first) { is_prev_set_ = false; } else { - prev_key_.SetKey(internal_key); + prev_key_.SetInternalKey(internal_key); is_prev_set_ = true; + is_prev_inclusive_ = true; } + + TEST_SYNC_POINT_CALLBACK("ForwardIterator::SeekInternal:Immutable", this); } else if (current_ && current_ != mutable_iter_) { // current_ is one of immutable iterators, push it back to the heap immutable_min_heap_.push(current_); } UpdateCurrent(); + TEST_SYNC_POINT_CALLBACK("ForwardIterator::SeekInternal:Return", this); } void ForwardIterator::Next() { assert(valid_); + bool update_prev_key = false; if (sv_ == nullptr || sv_->version_number != cfd_->GetSuperVersionNumber()) { std::string current_key = key().ToString(); Slice old_key(current_key.data(), current_key.size()); - RebuildIterators(); + if (sv_ == nullptr) { + RebuildIterators(true); + } else { + RenewIterators(); + } SeekInternal(old_key, false); if (!valid_ || key().compare(old_key) != 0) { return; } } else if (current_ != mutable_iter_) { // It is going to advance immutable iterator - prev_key_.SetKey(current_->key()); - is_prev_set_ = true; + + if (is_prev_set_ && prefix_extractor_) { + // advance prev_key_ to current_ only if they share the same prefix + update_prev_key = + prefix_extractor_->Transform(prev_key_.GetUserKey()) + .compare(prefix_extractor_->Transform(current_->key())) == 0; + } else { + update_prev_key = true; + } + + + if (update_prev_key) { + prev_key_.SetInternalKey(current_->key()); + is_prev_set_ = true; + is_prev_inclusive_ = false; + } } current_->Next(); if (current_ != mutable_iter_) { - if (current_->status().IsIncomplete()) { - assert(read_options_.read_tier == kBlockCacheTier); - status_ = current_->status(); - valid_ = false; - return; - } else if (current_->Valid()) { + if (!current_->status().ok()) { + immutable_status_ = current_->status(); + } else if ((current_->Valid()) && (!IsOverUpperBound(current_->key()))) { immutable_min_heap_.push(current_); + } else { + if ((current_->Valid()) && (IsOverUpperBound(current_->key()))) { + // remove the current iterator + DeleteCurrentIter(); + current_ = nullptr; + } + if (update_prev_key) { + mutable_iter_->Seek(prev_key_.GetInternalKey()); + } } } - UpdateCurrent(); + TEST_SYNC_POINT_CALLBACK("ForwardIterator::Next:Return", this); } Slice ForwardIterator::key() const { @@ -377,63 +502,224 @@ Status ForwardIterator::status() const { return mutable_iter_->status(); } - for (auto *it : imm_iters_) { - if (it && !it->status().ok()) { - return it->status(); + return immutable_status_; +} + +Status ForwardIterator::GetProperty(std::string prop_name, std::string* prop) { + assert(prop != nullptr); + if (prop_name == "rocksdb.iterator.super-version-number") { + *prop = ToString(sv_->version_number); + return Status::OK(); + } + return Status::InvalidArgument(); +} + +void ForwardIterator::SetPinnedItersMgr( + PinnedIteratorsManager* pinned_iters_mgr) { + pinned_iters_mgr_ = pinned_iters_mgr; + UpdateChildrenPinnedItersMgr(); +} + +void ForwardIterator::UpdateChildrenPinnedItersMgr() { + // Set PinnedIteratorsManager for mutable memtable iterator. + if (mutable_iter_) { + mutable_iter_->SetPinnedItersMgr(pinned_iters_mgr_); + } + + // Set PinnedIteratorsManager for immutable memtable iterators. + for (InternalIterator* child_iter : imm_iters_) { + if (child_iter) { + child_iter->SetPinnedItersMgr(pinned_iters_mgr_); } } - for (auto *it : l0_iters_) { - if (it && !it->status().ok()) { - return it->status(); + + // Set PinnedIteratorsManager for L0 files iterators. + for (InternalIterator* child_iter : l0_iters_) { + if (child_iter) { + child_iter->SetPinnedItersMgr(pinned_iters_mgr_); } } - for (auto *it : level_iters_) { - if (it && !it->status().ok()) { - return it->status(); + + // Set PinnedIteratorsManager for L1+ levels iterators. + for (LevelIterator* child_iter : level_iters_) { + if (child_iter) { + child_iter->SetPinnedItersMgr(pinned_iters_mgr_); } } +} + +bool ForwardIterator::IsKeyPinned() const { + return pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled() && + current_->IsKeyPinned(); +} - return Status::OK(); +bool ForwardIterator::IsValuePinned() const { + return pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled() && + current_->IsValuePinned(); } -void ForwardIterator::RebuildIterators() { +void ForwardIterator::RebuildIterators(bool refresh_sv) { // Clean up - Cleanup(); - // New - sv_ = cfd_->GetReferencedSuperVersion(&(db_->mutex_)); - mutable_iter_ = sv_->mem->NewIterator(read_options_); - sv_->imm->AddIterators(read_options_, &imm_iters_); - const auto& l0_files = sv_->current->files_[0]; + Cleanup(refresh_sv); + if (refresh_sv) { + // New + sv_ = cfd_->GetReferencedSuperVersion(&(db_->mutex_)); + } + RangeDelAggregator range_del_agg( + InternalKeyComparator(cfd_->internal_comparator()), {} /* snapshots */); + mutable_iter_ = sv_->mem->NewIterator(read_options_, &arena_); + sv_->imm->AddIterators(read_options_, &imm_iters_, &arena_); + if (!read_options_.ignore_range_deletions) { + std::unique_ptr range_del_iter( + sv_->mem->NewRangeTombstoneIterator(read_options_)); + range_del_agg.AddTombstones(std::move(range_del_iter)); + sv_->imm->AddRangeTombstoneIterators(read_options_, &arena_, + &range_del_agg); + } + has_iter_trimmed_for_upper_bound_ = false; + + const auto* vstorage = sv_->current->storage_info(); + const auto& l0_files = vstorage->LevelFiles(0); l0_iters_.reserve(l0_files.size()); for (const auto* l0 : l0_files) { + if ((read_options_.iterate_upper_bound != nullptr) && + cfd_->internal_comparator().user_comparator()->Compare( + l0->smallest.user_key(), *read_options_.iterate_upper_bound) > 0) { + has_iter_trimmed_for_upper_bound_ = true; + l0_iters_.push_back(nullptr); + continue; + } l0_iters_.push_back(cfd_->table_cache()->NewIterator( - read_options_, *cfd_->soptions(), cfd_->internal_comparator(), l0->fd)); + read_options_, *cfd_->soptions(), cfd_->internal_comparator(), l0->fd, + read_options_.ignore_range_deletions ? nullptr : &range_del_agg)); } - level_iters_.reserve(sv_->current->NumberLevels() - 1); - for (int32_t level = 1; level < sv_->current->NumberLevels(); ++level) { - if (sv_->current->files_[level].empty()) { - level_iters_.push_back(nullptr); - } else { - level_iters_.push_back(new LevelIterator(cfd_, read_options_, - sv_->current->files_[level])); + BuildLevelIterators(vstorage); + current_ = nullptr; + is_prev_set_ = false; + + UpdateChildrenPinnedItersMgr(); + if (!range_del_agg.IsEmpty()) { + status_ = Status::NotSupported( + "Range tombstones unsupported with ForwardIterator"); + valid_ = false; + } +} + +void ForwardIterator::RenewIterators() { + SuperVersion* svnew; + assert(sv_); + svnew = cfd_->GetReferencedSuperVersion(&(db_->mutex_)); + + if (mutable_iter_ != nullptr) { + DeleteIterator(mutable_iter_, true /* is_arena */); + } + for (auto* m : imm_iters_) { + DeleteIterator(m, true /* is_arena */); + } + imm_iters_.clear(); + + mutable_iter_ = svnew->mem->NewIterator(read_options_, &arena_); + svnew->imm->AddIterators(read_options_, &imm_iters_, &arena_); + RangeDelAggregator range_del_agg( + InternalKeyComparator(cfd_->internal_comparator()), {} /* snapshots */); + if (!read_options_.ignore_range_deletions) { + std::unique_ptr range_del_iter( + svnew->mem->NewRangeTombstoneIterator(read_options_)); + range_del_agg.AddTombstones(std::move(range_del_iter)); + sv_->imm->AddRangeTombstoneIterators(read_options_, &arena_, + &range_del_agg); + } + + const auto* vstorage = sv_->current->storage_info(); + const auto& l0_files = vstorage->LevelFiles(0); + const auto* vstorage_new = svnew->current->storage_info(); + const auto& l0_files_new = vstorage_new->LevelFiles(0); + size_t iold, inew; + bool found; + std::vector l0_iters_new; + l0_iters_new.reserve(l0_files_new.size()); + + for (inew = 0; inew < l0_files_new.size(); inew++) { + found = false; + for (iold = 0; iold < l0_files.size(); iold++) { + if (l0_files[iold] == l0_files_new[inew]) { + found = true; + break; + } + } + if (found) { + if (l0_iters_[iold] == nullptr) { + l0_iters_new.push_back(nullptr); + TEST_SYNC_POINT_CALLBACK("ForwardIterator::RenewIterators:Null", this); + } else { + l0_iters_new.push_back(l0_iters_[iold]); + l0_iters_[iold] = nullptr; + TEST_SYNC_POINT_CALLBACK("ForwardIterator::RenewIterators:Copy", this); + } + continue; } + l0_iters_new.push_back(cfd_->table_cache()->NewIterator( + read_options_, *cfd_->soptions(), cfd_->internal_comparator(), + l0_files_new[inew]->fd, + read_options_.ignore_range_deletions ? nullptr : &range_del_agg)); + } + + for (auto* f : l0_iters_) { + DeleteIterator(f); } + l0_iters_.clear(); + l0_iters_ = l0_iters_new; + for (auto* l : level_iters_) { + DeleteIterator(l); + } + level_iters_.clear(); + BuildLevelIterators(vstorage_new); current_ = nullptr; is_prev_set_ = false; + SVCleanup(); + sv_ = svnew; + + UpdateChildrenPinnedItersMgr(); + if (!range_del_agg.IsEmpty()) { + status_ = Status::NotSupported( + "Range tombstones unsupported with ForwardIterator"); + valid_ = false; + } +} + +void ForwardIterator::BuildLevelIterators(const VersionStorageInfo* vstorage) { + level_iters_.reserve(vstorage->num_levels() - 1); + for (int32_t level = 1; level < vstorage->num_levels(); ++level) { + const auto& level_files = vstorage->LevelFiles(level); + if ((level_files.empty()) || + ((read_options_.iterate_upper_bound != nullptr) && + (user_comparator_->Compare(*read_options_.iterate_upper_bound, + level_files[0]->smallest.user_key()) < + 0))) { + level_iters_.push_back(nullptr); + if (!level_files.empty()) { + has_iter_trimmed_for_upper_bound_ = true; + } + } else { + level_iters_.push_back( + new LevelIterator(cfd_, read_options_, level_files)); + } + } } void ForwardIterator::ResetIncompleteIterators() { - const auto& l0_files = sv_->current->files_[0]; - for (uint32_t i = 0; i < l0_iters_.size(); ++i) { + const auto& l0_files = sv_->current->storage_info()->LevelFiles(0); + for (size_t i = 0; i < l0_iters_.size(); ++i) { assert(i < l0_files.size()); - if (!l0_iters_[i]->status().IsIncomplete()) { + if (!l0_iters_[i] || !l0_iters_[i]->status().IsIncomplete()) { continue; } - delete l0_iters_[i]; + DeleteIterator(l0_iters_[i]); l0_iters_[i] = cfd_->table_cache()->NewIterator( read_options_, *cfd_->soptions(), cfd_->internal_comparator(), - l0_files[i]->fd); + l0_files[i]->fd, nullptr /* range_del_agg */); + l0_iters_[i]->SetPinnedItersMgr(pinned_iters_mgr_); } for (auto* level_iter : level_iters_) { @@ -471,30 +757,113 @@ void ForwardIterator::UpdateCurrent() { if (!status_.ok()) { status_ = Status::OK(); } + + // Upper bound doesn't apply to the memtable iterator. We want Valid() to + // return false when all iterators are over iterate_upper_bound, but can't + // just set valid_ to false, as that would effectively disable the tailing + // optimization (Seek() would be called on all immutable iterators regardless + // of whether the target key is greater than prev_key_). + current_over_upper_bound_ = valid_ && IsOverUpperBound(current_->key()); } bool ForwardIterator::NeedToSeekImmutable(const Slice& target) { - if (!valid_ || !is_prev_set_) { + // We maintain the interval (prev_key_, immutable_min_heap_.top()->key()) + // such that there are no records with keys within that range in + // immutable_min_heap_. Since immutable structures (SST files and immutable + // memtables) can't change in this version, we don't need to do a seek if + // 'target' belongs to that interval (immutable_min_heap_.top() is already + // at the correct position). + + if (!valid_ || !current_ || !is_prev_set_ || !immutable_status_.ok()) { return true; } - Slice prev_key = prev_key_.GetKey(); + Slice prev_key = prev_key_.GetInternalKey(); if (prefix_extractor_ && prefix_extractor_->Transform(target).compare( prefix_extractor_->Transform(prev_key)) != 0) { return true; } if (cfd_->internal_comparator().InternalKeyComparator::Compare( - prev_key, target) >= 0) { + prev_key, target) >= (is_prev_inclusive_ ? 1 : 0)) { return true; } - if (immutable_min_heap_.empty() || - cfd_->internal_comparator().InternalKeyComparator::Compare( - target, current_ == mutable_iter_ ? immutable_min_heap_.top()->key() - : current_->key()) > 0) { + + if (immutable_min_heap_.empty() && current_ == mutable_iter_) { + // Nothing to seek on. + return false; + } + if (cfd_->internal_comparator().InternalKeyComparator::Compare( + target, current_ == mutable_iter_ ? immutable_min_heap_.top()->key() + : current_->key()) > 0) { return true; } return false; } +void ForwardIterator::DeleteCurrentIter() { + const VersionStorageInfo* vstorage = sv_->current->storage_info(); + const std::vector& l0 = vstorage->LevelFiles(0); + for (size_t i = 0; i < l0.size(); ++i) { + if (!l0_iters_[i]) { + continue; + } + if (l0_iters_[i] == current_) { + has_iter_trimmed_for_upper_bound_ = true; + DeleteIterator(l0_iters_[i]); + l0_iters_[i] = nullptr; + return; + } + } + + for (int32_t level = 1; level < vstorage->num_levels(); ++level) { + if (level_iters_[level - 1] == nullptr) { + continue; + } + if (level_iters_[level - 1] == current_) { + has_iter_trimmed_for_upper_bound_ = true; + DeleteIterator(level_iters_[level - 1]); + level_iters_[level - 1] = nullptr; + } + } +} + +bool ForwardIterator::TEST_CheckDeletedIters(int* pdeleted_iters, + int* pnum_iters) { + bool retval = false; + int deleted_iters = 0; + int num_iters = 0; + + const VersionStorageInfo* vstorage = sv_->current->storage_info(); + const std::vector& l0 = vstorage->LevelFiles(0); + for (size_t i = 0; i < l0.size(); ++i) { + if (!l0_iters_[i]) { + retval = true; + deleted_iters++; + } else { + num_iters++; + } + } + + for (int32_t level = 1; level < vstorage->num_levels(); ++level) { + if ((level_iters_[level - 1] == nullptr) && + (!vstorage->LevelFiles(level).empty())) { + retval = true; + deleted_iters++; + } else if (!vstorage->LevelFiles(level).empty()) { + num_iters++; + } + } + if ((!retval) && num_iters <= 1) { + retval = true; + } + if (pdeleted_iters) { + *pdeleted_iters = deleted_iters; + } + if (pnum_iters) { + *pnum_iters = num_iters; + } + return retval; +} + uint32_t ForwardIterator::FindFileInRange( const std::vector& files, const Slice& internal_key, uint32_t left, uint32_t right) { @@ -515,6 +884,22 @@ uint32_t ForwardIterator::FindFileInRange( return right; } +void ForwardIterator::DeleteIterator(InternalIterator* iter, bool is_arena) { + if (iter == nullptr) { + return; + } + + if (pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled()) { + pinned_iters_mgr_->PinIterator(iter, is_arena); + } else { + if (is_arena) { + iter->~InternalIterator(); + } else { + delete iter; + } + } +} + } // namespace rocksdb #endif // ROCKSDB_LITE diff --git a/db/forward_iterator.h b/db/forward_iterator.h index bbf423a507b..d4f32cba9fa 100644 --- a/db/forward_iterator.h +++ b/db/forward_iterator.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #ifndef ROCKSDB_LITE @@ -14,6 +14,8 @@ #include "rocksdb/iterator.h" #include "rocksdb/options.h" #include "db/dbformat.h" +#include "table/internal_iterator.h" +#include "util/arena.h" namespace rocksdb { @@ -22,6 +24,7 @@ class Env; struct SuperVersion; class ColumnFamilyData; class LevelIterator; +class VersionStorageInfo; struct FileMetaData; class MinIterComparator { @@ -29,16 +32,15 @@ class MinIterComparator { explicit MinIterComparator(const Comparator* comparator) : comparator_(comparator) {} - bool operator()(Iterator* a, Iterator* b) { + bool operator()(InternalIterator* a, InternalIterator* b) { return comparator_->Compare(a->key(), b->key()) > 0; } private: const Comparator* comparator_; }; -typedef std::priority_queue, - MinIterComparator> MinIterHeap; +typedef std::priority_queue, + MinIterComparator> MinIterHeap; /** * ForwardIterator is a special type of iterator that only supports Seek() @@ -47,17 +49,21 @@ typedef std::priority_queue& files, const Slice& internal_key, uint32_t left, uint32_t right); + bool IsOverUpperBound(const Slice& internal_key) const; + + // Set PinnedIteratorsManager for all children Iterators, this function should + // be called whenever we update children Iterators or pinned_iters_mgr_. + void UpdateChildrenPinnedItersMgr(); + + // A helper function that will release iter in the proper manner, or pass it + // to pinned_iters_mgr_ to release it later if pinning is enabled. + void DeleteIterator(InternalIterator* iter, bool is_arena = false); + DBImpl* const db_; const ReadOptions read_options_; ColumnFamilyData* const cfd_; @@ -89,17 +116,37 @@ class ForwardIterator : public Iterator { MinIterHeap immutable_min_heap_; SuperVersion* sv_; - Iterator* mutable_iter_; - std::vector imm_iters_; - std::vector l0_iters_; + InternalIterator* mutable_iter_; + std::vector imm_iters_; + std::vector l0_iters_; std::vector level_iters_; - Iterator* current_; - // internal iterator status - Status status_; + InternalIterator* current_; bool valid_; + // Internal iterator status; set only by one of the unsupported methods. + Status status_; + // Status of immutable iterators, maintained here to avoid iterating over + // all of them in status(). + Status immutable_status_; + // Indicates that at least one of the immutable iterators pointed to a key + // larger than iterate_upper_bound and was therefore destroyed. Seek() may + // need to rebuild such iterators. + bool has_iter_trimmed_for_upper_bound_; + // Is current key larger than iterate_upper_bound? If so, makes Valid() + // return false. + bool current_over_upper_bound_; + + // Left endpoint of the range of keys that immutable iterators currently + // cover. When Seek() is called with a key that's within that range, immutable + // iterators don't need to be moved; see NeedToSeekImmutable(). This key is + // included in the range after a Seek(), but excluded when advancing the + // iterator using Next(). IterKey prev_key_; bool is_prev_set_; + bool is_prev_inclusive_; + + PinnedIteratorsManager* pinned_iters_mgr_; + Arena arena_; }; } // namespace rocksdb diff --git a/db/forward_iterator_bench.cc b/db/forward_iterator_bench.cc new file mode 100644 index 00000000000..e9ae770cfaf --- /dev/null +++ b/db/forward_iterator_bench.cc @@ -0,0 +1,375 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#if !defined(GFLAGS) || defined(ROCKSDB_LITE) +#include +int main() { + fprintf(stderr, "Please install gflags to run rocksdb tools\n"); + return 1; +} +#elif defined(OS_MACOSX) || defined(OS_WIN) +// Block forward_iterator_bench under MAC and Windows +int main() { return 0; } +#else +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rocksdb/cache.h" +#include "rocksdb/db.h" +#include "rocksdb/status.h" +#include "rocksdb/table.h" +#include "port/port.h" +#include "util/testharness.h" + +const int MAX_SHARDS = 100000; + +DEFINE_int32(writers, 8, ""); +DEFINE_int32(readers, 8, ""); +DEFINE_int64(rate, 100000, ""); +DEFINE_int64(value_size, 300, ""); +DEFINE_int64(shards, 1000, ""); +DEFINE_int64(memtable_size, 500000000, ""); +DEFINE_int64(block_cache_size, 300000000, ""); +DEFINE_int64(block_size, 65536, ""); +DEFINE_double(runtime, 300.0, ""); +DEFINE_bool(cache_only_first, true, ""); +DEFINE_bool(iterate_upper_bound, true, ""); + +struct Stats { + char pad1[128] __attribute__((__unused__)); + std::atomic written{0}; + char pad2[128] __attribute__((__unused__)); + std::atomic read{0}; + std::atomic cache_misses{0}; + char pad3[128] __attribute__((__unused__)); +} stats; + +struct Key { + Key() {} + Key(uint64_t shard_in, uint64_t seqno_in) + : shard_be(htobe64(shard_in)), seqno_be(htobe64(seqno_in)) {} + + uint64_t shard() const { return be64toh(shard_be); } + uint64_t seqno() const { return be64toh(seqno_be); } + + private: + uint64_t shard_be; + uint64_t seqno_be; +} __attribute__((__packed__)); + +struct Reader; +struct Writer; + +struct ShardState { + char pad1[128] __attribute__((__unused__)); + std::atomic last_written{0}; + Writer* writer; + Reader* reader; + char pad2[128] __attribute__((__unused__)); + std::atomic last_read{0}; + std::unique_ptr it; + std::unique_ptr it_cacheonly; + Key upper_bound; + rocksdb::Slice upper_bound_slice; + char pad3[128] __attribute__((__unused__)); +}; + +struct Reader { + public: + explicit Reader(std::vector* shard_states, rocksdb::DB* db) + : shard_states_(shard_states), db_(db) { + sem_init(&sem_, 0, 0); + thread_ = port::Thread(&Reader::run, this); + } + + void run() { + while (1) { + sem_wait(&sem_); + if (done_.load()) { + break; + } + + uint64_t shard; + { + std::lock_guard guard(queue_mutex_); + assert(!shards_pending_queue_.empty()); + shard = shards_pending_queue_.front(); + shards_pending_queue_.pop(); + shards_pending_set_.reset(shard); + } + readOnceFromShard(shard); + } + } + + void readOnceFromShard(uint64_t shard) { + ShardState& state = (*shard_states_)[shard]; + if (!state.it) { + // Initialize iterators + rocksdb::ReadOptions options; + options.tailing = true; + if (FLAGS_iterate_upper_bound) { + state.upper_bound = Key(shard, std::numeric_limits::max()); + state.upper_bound_slice = rocksdb::Slice( + (const char*)&state.upper_bound, sizeof(state.upper_bound)); + options.iterate_upper_bound = &state.upper_bound_slice; + } + + state.it.reset(db_->NewIterator(options)); + + if (FLAGS_cache_only_first) { + options.read_tier = rocksdb::ReadTier::kBlockCacheTier; + state.it_cacheonly.reset(db_->NewIterator(options)); + } + } + + const uint64_t upto = state.last_written.load(); + for (rocksdb::Iterator* it : {state.it_cacheonly.get(), state.it.get()}) { + if (it == nullptr) { + continue; + } + if (state.last_read.load() >= upto) { + break; + } + bool need_seek = true; + for (uint64_t seq = state.last_read.load() + 1; seq <= upto; ++seq) { + if (need_seek) { + Key from(shard, state.last_read.load() + 1); + it->Seek(rocksdb::Slice((const char*)&from, sizeof(from))); + need_seek = false; + } else { + it->Next(); + } + if (it->status().IsIncomplete()) { + ++::stats.cache_misses; + break; + } + assert(it->Valid()); + assert(it->key().size() == sizeof(Key)); + Key key; + memcpy(&key, it->key().data(), it->key().size()); + // fprintf(stderr, "Expecting (%ld, %ld) read (%ld, %ld)\n", + // shard, seq, key.shard(), key.seqno()); + assert(key.shard() == shard); + assert(key.seqno() == seq); + state.last_read.store(seq); + ++::stats.read; + } + } + } + + void onWrite(uint64_t shard) { + { + std::lock_guard guard(queue_mutex_); + if (!shards_pending_set_.test(shard)) { + shards_pending_queue_.push(shard); + shards_pending_set_.set(shard); + sem_post(&sem_); + } + } + } + + ~Reader() { + done_.store(true); + sem_post(&sem_); + thread_.join(); + } + + private: + char pad1[128] __attribute__((__unused__)); + std::vector* shard_states_; + rocksdb::DB* db_; + rocksdb::port::Thread thread_; + sem_t sem_; + std::mutex queue_mutex_; + std::bitset shards_pending_set_; + std::queue shards_pending_queue_; + std::atomic done_{false}; + char pad2[128] __attribute__((__unused__)); +}; + +struct Writer { + explicit Writer(std::vector* shard_states, rocksdb::DB* db) + : shard_states_(shard_states), db_(db) {} + + void start() { thread_ = port::Thread(&Writer::run, this); } + + void run() { + std::queue workq; + std::chrono::steady_clock::time_point deadline( + std::chrono::steady_clock::now() + + std::chrono::nanoseconds((uint64_t)(1000000000 * FLAGS_runtime))); + std::vector my_shards; + for (int i = 1; i <= FLAGS_shards; ++i) { + if ((*shard_states_)[i].writer == this) { + my_shards.push_back(i); + } + } + + std::mt19937 rng{std::random_device()()}; + std::uniform_int_distribution shard_dist( + 0, static_cast(my_shards.size()) - 1); + std::string value(FLAGS_value_size, '*'); + + while (1) { + auto now = std::chrono::steady_clock::now(); + if (FLAGS_runtime >= 0 && now >= deadline) { + break; + } + if (workq.empty()) { + for (int i = 0; i < FLAGS_rate; i += FLAGS_writers) { + std::chrono::nanoseconds offset(1000000000LL * i / FLAGS_rate); + workq.push(now + offset); + } + } + while (!workq.empty() && workq.front() < now) { + workq.pop(); + uint64_t shard = my_shards[shard_dist(rng)]; + ShardState& state = (*shard_states_)[shard]; + uint64_t seqno = state.last_written.load() + 1; + Key key(shard, seqno); + // fprintf(stderr, "Writing (%ld, %ld)\n", shard, seqno); + rocksdb::Status status = + db_->Put(rocksdb::WriteOptions(), + rocksdb::Slice((const char*)&key, sizeof(key)), + rocksdb::Slice(value)); + assert(status.ok()); + state.last_written.store(seqno); + state.reader->onWrite(shard); + ++::stats.written; + } + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + // fprintf(stderr, "Writer done\n"); + } + + ~Writer() { thread_.join(); } + + private: + char pad1[128] __attribute__((__unused__)); + std::vector* shard_states_; + rocksdb::DB* db_; + rocksdb::port::Thread thread_; + char pad2[128] __attribute__((__unused__)); +}; + +struct StatsThread { + explicit StatsThread(rocksdb::DB* db) + : db_(db), thread_(&StatsThread::run, this) {} + + void run() { + // using namespace std::chrono; + auto tstart = std::chrono::steady_clock::now(), tlast = tstart; + uint64_t wlast = 0, rlast = 0; + while (!done_.load()) { + { + std::unique_lock lock(cvm_); + cv_.wait_for(lock, std::chrono::seconds(1)); + } + auto now = std::chrono::steady_clock::now(); + double elapsed = + std::chrono::duration_cast >( + now - tlast).count(); + uint64_t w = ::stats.written.load(); + uint64_t r = ::stats.read.load(); + fprintf(stderr, + "%s elapsed %4lds | written %10ld | w/s %10.0f | read %10ld | " + "r/s %10.0f | cache misses %10ld\n", + db_->GetEnv()->TimeToString(time(nullptr)).c_str(), + std::chrono::duration_cast(now - tstart) + .count(), + w, (w - wlast) / elapsed, r, (r - rlast) / elapsed, + ::stats.cache_misses.load()); + wlast = w; + rlast = r; + tlast = now; + } + } + + ~StatsThread() { + { + std::lock_guard guard(cvm_); + done_.store(true); + } + cv_.notify_all(); + thread_.join(); + } + + private: + rocksdb::DB* db_; + std::mutex cvm_; + std::condition_variable cv_; + rocksdb::port::Thread thread_; + std::atomic done_{false}; +}; + +int main(int argc, char** argv) { + GFLAGS::ParseCommandLineFlags(&argc, &argv, true); + + std::mt19937 rng{std::random_device()()}; + rocksdb::Status status; + std::string path = rocksdb::test::TmpDir() + "/forward_iterator_test"; + fprintf(stderr, "db path is %s\n", path.c_str()); + rocksdb::Options options; + options.create_if_missing = true; + options.compression = rocksdb::CompressionType::kNoCompression; + options.compaction_style = rocksdb::CompactionStyle::kCompactionStyleNone; + options.level0_slowdown_writes_trigger = 99999; + options.level0_stop_writes_trigger = 99999; + options.use_direct_io_for_flush_and_compaction = true; + options.write_buffer_size = FLAGS_memtable_size; + rocksdb::BlockBasedTableOptions table_options; + table_options.block_cache = rocksdb::NewLRUCache(FLAGS_block_cache_size); + table_options.block_size = FLAGS_block_size; + options.table_factory.reset( + rocksdb::NewBlockBasedTableFactory(table_options)); + + status = rocksdb::DestroyDB(path, options); + assert(status.ok()); + rocksdb::DB* db_raw; + status = rocksdb::DB::Open(options, path, &db_raw); + assert(status.ok()); + std::unique_ptr db(db_raw); + + std::vector shard_states(FLAGS_shards + 1); + std::deque readers; + while (static_cast(readers.size()) < FLAGS_readers) { + readers.emplace_back(&shard_states, db_raw); + } + std::deque writers; + while (static_cast(writers.size()) < FLAGS_writers) { + writers.emplace_back(&shard_states, db_raw); + } + + // Each shard gets a random reader and random writer assigned to it + for (int i = 1; i <= FLAGS_shards; ++i) { + std::uniform_int_distribution reader_dist(0, FLAGS_readers - 1); + std::uniform_int_distribution writer_dist(0, FLAGS_writers - 1); + shard_states[i].reader = &readers[reader_dist(rng)]; + shard_states[i].writer = &writers[writer_dist(rng)]; + } + + StatsThread stats_thread(db_raw); + for (Writer& w : writers) { + w.start(); + } + + writers.clear(); + readers.clear(); +} +#endif // !defined(GFLAGS) || defined(ROCKSDB_LITE) diff --git a/db/internal_stats.cc b/db/internal_stats.cc index 3142d13b301..e98bd98cf77 100644 --- a/db/internal_stats.cc +++ b/db/internal_stats.cc @@ -1,316 +1,891 @@ -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #include "db/internal_stats.h" + +#ifndef __STDC_FORMAT_MACROS #define __STDC_FORMAT_MACROS +#endif + #include +#include +#include +#include +#include #include #include "db/column_family.h" + #include "db/db_impl.h" +#include "util/string_util.h" namespace rocksdb { +#ifndef ROCKSDB_LITE + +const std::map InternalStats::compaction_level_stats = + { + {LevelStatType::NUM_FILES, LevelStat{"NumFiles", "Files"}}, + {LevelStatType::COMPACTED_FILES, + LevelStat{"CompactedFiles", "CompactedFiles"}}, + {LevelStatType::SIZE_BYTES, LevelStat{"SizeBytes", "Size"}}, + {LevelStatType::SCORE, LevelStat{"Score", "Score"}}, + {LevelStatType::READ_GB, LevelStat{"ReadGB", "Read(GB)"}}, + {LevelStatType::RN_GB, LevelStat{"RnGB", "Rn(GB)"}}, + {LevelStatType::RNP1_GB, LevelStat{"Rnp1GB", "Rnp1(GB)"}}, + {LevelStatType::WRITE_GB, LevelStat{"WriteGB", "Write(GB)"}}, + {LevelStatType::W_NEW_GB, LevelStat{"WnewGB", "Wnew(GB)"}}, + {LevelStatType::MOVED_GB, LevelStat{"MovedGB", "Moved(GB)"}}, + {LevelStatType::WRITE_AMP, LevelStat{"WriteAmp", "W-Amp"}}, + {LevelStatType::READ_MBPS, LevelStat{"ReadMBps", "Rd(MB/s)"}}, + {LevelStatType::WRITE_MBPS, LevelStat{"WriteMBps", "Wr(MB/s)"}}, + {LevelStatType::COMP_SEC, LevelStat{"CompSec", "Comp(sec)"}}, + {LevelStatType::COMP_COUNT, LevelStat{"CompCount", "Comp(cnt)"}}, + {LevelStatType::AVG_SEC, LevelStat{"AvgSec", "Avg(sec)"}}, + {LevelStatType::KEY_IN, LevelStat{"KeyIn", "KeyIn"}}, + {LevelStatType::KEY_DROP, LevelStat{"KeyDrop", "KeyDrop"}}, +}; + namespace { const double kMB = 1048576.0; const double kGB = kMB * 1024; +const double kMicrosInSec = 1000000.0; void PrintLevelStatsHeader(char* buf, size_t len, const std::string& cf_name) { - snprintf( - buf, len, - "\n** Compaction Stats [%s] **\n" - "Level Files Size(MB) Score Read(GB) Rn(GB) Rnp1(GB) " - "Write(GB) Wnew(GB) RW-Amp W-Amp Rd(MB/s) Wr(MB/s) Rn(cnt) " - "Rnp1(cnt) Wnp1(cnt) Wnew(cnt) Comp(sec) Comp(cnt) Avg(sec) " - "Stall(sec) Stall(cnt) Avg(ms)\n" - "--------------------------------------------------------------------" - "--------------------------------------------------------------------" - "--------------------------------------------------------------------\n", - cf_name.c_str()); + int written_size = + snprintf(buf, len, "\n** Compaction Stats [%s] **\n", cf_name.c_str()); + auto hdr = [](LevelStatType t) { + return InternalStats::compaction_level_stats.at(t).header_name.c_str(); + }; + int line_size = snprintf( + buf + written_size, len - written_size, + "Level %s %s %s %s %s %s %s %s %s %s %s %s %s %s %s %s %s\n", + // Note that we skip COMPACTED_FILES and merge it with Files column + hdr(LevelStatType::NUM_FILES), hdr(LevelStatType::SIZE_BYTES), + hdr(LevelStatType::SCORE), hdr(LevelStatType::READ_GB), + hdr(LevelStatType::RN_GB), hdr(LevelStatType::RNP1_GB), + hdr(LevelStatType::WRITE_GB), hdr(LevelStatType::W_NEW_GB), + hdr(LevelStatType::MOVED_GB), hdr(LevelStatType::WRITE_AMP), + hdr(LevelStatType::READ_MBPS), hdr(LevelStatType::WRITE_MBPS), + hdr(LevelStatType::COMP_SEC), hdr(LevelStatType::COMP_COUNT), + hdr(LevelStatType::AVG_SEC), hdr(LevelStatType::KEY_IN), + hdr(LevelStatType::KEY_DROP)); + + written_size += line_size; + snprintf(buf + written_size, len - written_size, "%s\n", + std::string(line_size, '-').c_str()); } -void PrintLevelStats(char* buf, size_t len, const std::string& name, - int num_files, int being_compacted, double total_file_size, double score, - double rw_amp, double w_amp, double stall_us, uint64_t stalls, - const InternalStats::CompactionStats& stats) { - uint64_t bytes_read = stats.bytes_readn + stats.bytes_readnp1; - uint64_t bytes_new = stats.bytes_written - stats.bytes_readnp1; - double elapsed = (stats.micros + 1) / 1000000.0; +void PrepareLevelStats(std::map* level_stats, + int num_files, int being_compacted, + double total_file_size, double score, double w_amp, + const InternalStats::CompactionStats& stats) { + uint64_t bytes_read = + stats.bytes_read_non_output_levels + stats.bytes_read_output_level; + int64_t bytes_new = + stats.bytes_written - stats.bytes_read_output_level; + double elapsed = (stats.micros + 1) / kMicrosInSec; + + (*level_stats)[LevelStatType::NUM_FILES] = num_files; + (*level_stats)[LevelStatType::COMPACTED_FILES] = being_compacted; + (*level_stats)[LevelStatType::SIZE_BYTES] = total_file_size; + (*level_stats)[LevelStatType::SCORE] = score; + (*level_stats)[LevelStatType::READ_GB] = bytes_read / kGB; + (*level_stats)[LevelStatType::RN_GB] = + stats.bytes_read_non_output_levels / kGB; + (*level_stats)[LevelStatType::RNP1_GB] = stats.bytes_read_output_level / kGB; + (*level_stats)[LevelStatType::WRITE_GB] = stats.bytes_written / kGB; + (*level_stats)[LevelStatType::W_NEW_GB] = bytes_new / kGB; + (*level_stats)[LevelStatType::MOVED_GB] = stats.bytes_moved / kGB; + (*level_stats)[LevelStatType::WRITE_AMP] = w_amp; + (*level_stats)[LevelStatType::READ_MBPS] = bytes_read / kMB / elapsed; + (*level_stats)[LevelStatType::WRITE_MBPS] = + stats.bytes_written / kMB / elapsed; + (*level_stats)[LevelStatType::COMP_SEC] = stats.micros / kMicrosInSec; + (*level_stats)[LevelStatType::COMP_COUNT] = stats.count; + (*level_stats)[LevelStatType::AVG_SEC] = + stats.count == 0 ? 0 : stats.micros / kMicrosInSec / stats.count; + (*level_stats)[LevelStatType::KEY_IN] = + static_cast(stats.num_input_records); + (*level_stats)[LevelStatType::KEY_DROP] = + static_cast(stats.num_dropped_records); +} +void PrintLevelStats(char* buf, size_t len, const std::string& name, + const std::map& stat_value) { snprintf(buf, len, - "%4s %5d/%-3d %8.0f %5.1f " /* Level, Files, Size(MB), Score */ - "%8.1f " /* Read(GB) */ - "%7.1f " /* Rn(GB) */ - "%8.1f " /* Rnp1(GB) */ - "%9.1f " /* Write(GB) */ - "%8.1f " /* Wnew(GB) */ - "%6.1f " /* RW-Amp */ - "%5.1f " /* W-Amp */ - "%8.1f " /* Rd(MB/s) */ - "%8.1f " /* Wr(MB/s) */ - "%8d " /* Rn(cnt) */ - "%9d " /* Rnp1(cnt) */ - "%9d " /* Wnp1(cnt) */ - "%9d " /* Wnew(cnt) */ - "%10.0f " /* Comp(sec) */ - "%9d " /* Comp(cnt) */ - "%8.3f " /* Avg(sec) */ - "%10.2f " /* Stall(sec) */ - "%10" PRIu64 " " /* Stall(cnt) */ - "%7.2f\n" /* Avg(ms) */, - name.c_str(), num_files, being_compacted, total_file_size / kMB, score, - bytes_read / kGB, - stats.bytes_readn / kGB, - stats.bytes_readnp1 / kGB, - stats.bytes_written / kGB, - bytes_new / kGB, - rw_amp, - w_amp, - bytes_read / kMB / elapsed, - stats.bytes_written / kMB / elapsed, - stats.files_in_leveln, - stats.files_in_levelnp1, - stats.files_out_levelnp1, - stats.files_out_levelnp1 - stats.files_in_levelnp1, - stats.micros / 1000000.0, - stats.count, - stats.count == 0 ? 0 : stats.micros / 1000000.0 / stats.count, - stall_us / 1000000.0, - stalls, - stalls == 0 ? 0 : stall_us / 1000.0 / stalls); -} - - -} - -DBPropertyType GetPropertyType(const Slice& property, bool* is_int_property, - bool* need_out_of_mutex) { - assert(is_int_property != nullptr); - assert(need_out_of_mutex != nullptr); - Slice in = property; - Slice prefix("rocksdb."); - *need_out_of_mutex = false; - *is_int_property = false; - if (!in.starts_with(prefix)) { - return kUnknown; - } - in.remove_prefix(prefix.size()); - - if (in.starts_with("num-files-at-level")) { - return kNumFilesAtLevel; - } else if (in == "levelstats") { - return kLevelStats; - } else if (in == "stats") { - return kStats; - } else if (in == "cfstats") { - return kCFStats; - } else if (in == "dbstats") { - return kDBStats; - } else if (in == "sstables") { - return kSsTables; + "%4s " /* Level */ + "%6d/%-3d " /* Files */ + "%8s " /* Size */ + "%5.1f " /* Score */ + "%8.1f " /* Read(GB) */ + "%7.1f " /* Rn(GB) */ + "%8.1f " /* Rnp1(GB) */ + "%9.1f " /* Write(GB) */ + "%8.1f " /* Wnew(GB) */ + "%9.1f " /* Moved(GB) */ + "%5.1f " /* W-Amp */ + "%8.1f " /* Rd(MB/s) */ + "%8.1f " /* Wr(MB/s) */ + "%9.0f " /* Comp(sec) */ + "%9d " /* Comp(cnt) */ + "%8.3f " /* Avg(sec) */ + "%7s " /* KeyIn */ + "%6s\n", /* KeyDrop */ + name.c_str(), + static_cast(stat_value.at(LevelStatType::NUM_FILES)), + static_cast(stat_value.at(LevelStatType::COMPACTED_FILES)), + BytesToHumanString( + static_cast(stat_value.at(LevelStatType::SIZE_BYTES))) + .c_str(), + stat_value.at(LevelStatType::SCORE), + stat_value.at(LevelStatType::READ_GB), + stat_value.at(LevelStatType::RN_GB), + stat_value.at(LevelStatType::RNP1_GB), + stat_value.at(LevelStatType::WRITE_GB), + stat_value.at(LevelStatType::W_NEW_GB), + stat_value.at(LevelStatType::MOVED_GB), + stat_value.at(LevelStatType::WRITE_AMP), + stat_value.at(LevelStatType::READ_MBPS), + stat_value.at(LevelStatType::WRITE_MBPS), + stat_value.at(LevelStatType::COMP_SEC), + static_cast(stat_value.at(LevelStatType::COMP_COUNT)), + stat_value.at(LevelStatType::AVG_SEC), + NumberToHumanString( + static_cast(stat_value.at(LevelStatType::KEY_IN))) + .c_str(), + NumberToHumanString(static_cast( + stat_value.at(LevelStatType::KEY_DROP))) + .c_str()); +} + +void PrintLevelStats(char* buf, size_t len, const std::string& name, + int num_files, int being_compacted, double total_file_size, + double score, double w_amp, + const InternalStats::CompactionStats& stats) { + std::map level_stats; + PrepareLevelStats(&level_stats, num_files, being_compacted, total_file_size, + score, w_amp, stats); + PrintLevelStats(buf, len, name, level_stats); +} + +// Assumes that trailing numbers represent an optional argument. This requires +// property names to not end with numbers. +std::pair GetPropertyNameAndArg(const Slice& property) { + Slice name = property, arg = property; + size_t sfx_len = 0; + while (sfx_len < property.size() && + isdigit(property[property.size() - sfx_len - 1])) { + ++sfx_len; } + name.remove_suffix(sfx_len); + arg.remove_prefix(property.size() - sfx_len); + return {name, arg}; +} +} // anonymous namespace + +static const std::string rocksdb_prefix = "rocksdb."; - *is_int_property = true; - if (in == "num-immutable-mem-table") { - return kNumImmutableMemTable; - } else if (in == "mem-table-flush-pending") { - return kMemtableFlushPending; - } else if (in == "compaction-pending") { - return kCompactionPending; - } else if (in == "background-errors") { - return kBackgroundErrors; - } else if (in == "cur-size-active-mem-table") { - return kCurSizeActiveMemTable; - } else if (in == "num-entries-active-mem-table") { - return kNumEntriesInMutableMemtable; - } else if (in == "num-entries-imm-mem-tables") { - return kNumEntriesInImmutableMemtable; - } else if (in == "estimate-num-keys") { - return kEstimatedNumKeys; - } else if (in == "estimate-table-readers-mem") { - *need_out_of_mutex = true; - return kEstimatedUsageByTableReaders; - } else if (in == "is-file-deletions-enabled") { - return kIsFileDeletionEnabled; +static const std::string num_files_at_level_prefix = "num-files-at-level"; +static const std::string compression_ratio_at_level_prefix = + "compression-ratio-at-level"; +static const std::string allstats = "stats"; +static const std::string sstables = "sstables"; +static const std::string cfstats = "cfstats"; +static const std::string cfstats_no_file_histogram = + "cfstats-no-file-histogram"; +static const std::string cf_file_histogram = "cf-file-histogram"; +static const std::string dbstats = "dbstats"; +static const std::string levelstats = "levelstats"; +static const std::string num_immutable_mem_table = "num-immutable-mem-table"; +static const std::string num_immutable_mem_table_flushed = + "num-immutable-mem-table-flushed"; +static const std::string mem_table_flush_pending = "mem-table-flush-pending"; +static const std::string compaction_pending = "compaction-pending"; +static const std::string background_errors = "background-errors"; +static const std::string cur_size_active_mem_table = + "cur-size-active-mem-table"; +static const std::string cur_size_all_mem_tables = "cur-size-all-mem-tables"; +static const std::string size_all_mem_tables = "size-all-mem-tables"; +static const std::string num_entries_active_mem_table = + "num-entries-active-mem-table"; +static const std::string num_entries_imm_mem_tables = + "num-entries-imm-mem-tables"; +static const std::string num_deletes_active_mem_table = + "num-deletes-active-mem-table"; +static const std::string num_deletes_imm_mem_tables = + "num-deletes-imm-mem-tables"; +static const std::string estimate_num_keys = "estimate-num-keys"; +static const std::string estimate_table_readers_mem = + "estimate-table-readers-mem"; +static const std::string is_file_deletions_enabled = + "is-file-deletions-enabled"; +static const std::string num_snapshots = "num-snapshots"; +static const std::string oldest_snapshot_time = "oldest-snapshot-time"; +static const std::string num_live_versions = "num-live-versions"; +static const std::string current_version_number = + "current-super-version-number"; +static const std::string estimate_live_data_size = "estimate-live-data-size"; +static const std::string min_log_number_to_keep = "min-log-number-to-keep"; +static const std::string base_level = "base-level"; +static const std::string total_sst_files_size = "total-sst-files-size"; +static const std::string estimate_pending_comp_bytes = + "estimate-pending-compaction-bytes"; +static const std::string aggregated_table_properties = + "aggregated-table-properties"; +static const std::string aggregated_table_properties_at_level = + aggregated_table_properties + "-at-level"; +static const std::string num_running_compactions = "num-running-compactions"; +static const std::string num_running_flushes = "num-running-flushes"; +static const std::string actual_delayed_write_rate = + "actual-delayed-write-rate"; +static const std::string is_write_stopped = "is-write-stopped"; +static const std::string estimate_oldest_key_time = "estimate-oldest-key-time"; + +const std::string DB::Properties::kNumFilesAtLevelPrefix = + rocksdb_prefix + num_files_at_level_prefix; +const std::string DB::Properties::kCompressionRatioAtLevelPrefix = + rocksdb_prefix + compression_ratio_at_level_prefix; +const std::string DB::Properties::kStats = rocksdb_prefix + allstats; +const std::string DB::Properties::kSSTables = rocksdb_prefix + sstables; +const std::string DB::Properties::kCFStats = rocksdb_prefix + cfstats; +const std::string DB::Properties::kCFStatsNoFileHistogram = + rocksdb_prefix + cfstats_no_file_histogram; +const std::string DB::Properties::kCFFileHistogram = + rocksdb_prefix + cf_file_histogram; +const std::string DB::Properties::kDBStats = rocksdb_prefix + dbstats; +const std::string DB::Properties::kLevelStats = rocksdb_prefix + levelstats; +const std::string DB::Properties::kNumImmutableMemTable = + rocksdb_prefix + num_immutable_mem_table; +const std::string DB::Properties::kNumImmutableMemTableFlushed = + rocksdb_prefix + num_immutable_mem_table_flushed; +const std::string DB::Properties::kMemTableFlushPending = + rocksdb_prefix + mem_table_flush_pending; +const std::string DB::Properties::kCompactionPending = + rocksdb_prefix + compaction_pending; +const std::string DB::Properties::kNumRunningCompactions = + rocksdb_prefix + num_running_compactions; +const std::string DB::Properties::kNumRunningFlushes = + rocksdb_prefix + num_running_flushes; +const std::string DB::Properties::kBackgroundErrors = + rocksdb_prefix + background_errors; +const std::string DB::Properties::kCurSizeActiveMemTable = + rocksdb_prefix + cur_size_active_mem_table; +const std::string DB::Properties::kCurSizeAllMemTables = + rocksdb_prefix + cur_size_all_mem_tables; +const std::string DB::Properties::kSizeAllMemTables = + rocksdb_prefix + size_all_mem_tables; +const std::string DB::Properties::kNumEntriesActiveMemTable = + rocksdb_prefix + num_entries_active_mem_table; +const std::string DB::Properties::kNumEntriesImmMemTables = + rocksdb_prefix + num_entries_imm_mem_tables; +const std::string DB::Properties::kNumDeletesActiveMemTable = + rocksdb_prefix + num_deletes_active_mem_table; +const std::string DB::Properties::kNumDeletesImmMemTables = + rocksdb_prefix + num_deletes_imm_mem_tables; +const std::string DB::Properties::kEstimateNumKeys = + rocksdb_prefix + estimate_num_keys; +const std::string DB::Properties::kEstimateTableReadersMem = + rocksdb_prefix + estimate_table_readers_mem; +const std::string DB::Properties::kIsFileDeletionsEnabled = + rocksdb_prefix + is_file_deletions_enabled; +const std::string DB::Properties::kNumSnapshots = + rocksdb_prefix + num_snapshots; +const std::string DB::Properties::kOldestSnapshotTime = + rocksdb_prefix + oldest_snapshot_time; +const std::string DB::Properties::kNumLiveVersions = + rocksdb_prefix + num_live_versions; +const std::string DB::Properties::kCurrentSuperVersionNumber = + rocksdb_prefix + current_version_number; +const std::string DB::Properties::kEstimateLiveDataSize = + rocksdb_prefix + estimate_live_data_size; +const std::string DB::Properties::kMinLogNumberToKeep = + rocksdb_prefix + min_log_number_to_keep; +const std::string DB::Properties::kTotalSstFilesSize = + rocksdb_prefix + total_sst_files_size; +const std::string DB::Properties::kBaseLevel = rocksdb_prefix + base_level; +const std::string DB::Properties::kEstimatePendingCompactionBytes = + rocksdb_prefix + estimate_pending_comp_bytes; +const std::string DB::Properties::kAggregatedTableProperties = + rocksdb_prefix + aggregated_table_properties; +const std::string DB::Properties::kAggregatedTablePropertiesAtLevel = + rocksdb_prefix + aggregated_table_properties_at_level; +const std::string DB::Properties::kActualDelayedWriteRate = + rocksdb_prefix + actual_delayed_write_rate; +const std::string DB::Properties::kIsWriteStopped = + rocksdb_prefix + is_write_stopped; +const std::string DB::Properties::kEstimateOldestKeyTime = + rocksdb_prefix + estimate_oldest_key_time; + +const std::unordered_map + InternalStats::ppt_name_to_info = { + {DB::Properties::kNumFilesAtLevelPrefix, + {false, &InternalStats::HandleNumFilesAtLevel, nullptr, nullptr}}, + {DB::Properties::kCompressionRatioAtLevelPrefix, + {false, &InternalStats::HandleCompressionRatioAtLevelPrefix, nullptr, + nullptr}}, + {DB::Properties::kLevelStats, + {false, &InternalStats::HandleLevelStats, nullptr, nullptr}}, + {DB::Properties::kStats, + {false, &InternalStats::HandleStats, nullptr, nullptr}}, + {DB::Properties::kCFStats, + {false, &InternalStats::HandleCFStats, nullptr, + &InternalStats::HandleCFMapStats}}, + {DB::Properties::kCFStatsNoFileHistogram, + {false, &InternalStats::HandleCFStatsNoFileHistogram, nullptr, + nullptr}}, + {DB::Properties::kCFFileHistogram, + {false, &InternalStats::HandleCFFileHistogram, nullptr, nullptr}}, + {DB::Properties::kDBStats, + {false, &InternalStats::HandleDBStats, nullptr, nullptr}}, + {DB::Properties::kSSTables, + {false, &InternalStats::HandleSsTables, nullptr, nullptr}}, + {DB::Properties::kAggregatedTableProperties, + {false, &InternalStats::HandleAggregatedTableProperties, nullptr, + nullptr}}, + {DB::Properties::kAggregatedTablePropertiesAtLevel, + {false, &InternalStats::HandleAggregatedTablePropertiesAtLevel, + nullptr, nullptr}}, + {DB::Properties::kNumImmutableMemTable, + {false, nullptr, &InternalStats::HandleNumImmutableMemTable, nullptr}}, + {DB::Properties::kNumImmutableMemTableFlushed, + {false, nullptr, &InternalStats::HandleNumImmutableMemTableFlushed, + nullptr}}, + {DB::Properties::kMemTableFlushPending, + {false, nullptr, &InternalStats::HandleMemTableFlushPending, nullptr}}, + {DB::Properties::kCompactionPending, + {false, nullptr, &InternalStats::HandleCompactionPending, nullptr}}, + {DB::Properties::kBackgroundErrors, + {false, nullptr, &InternalStats::HandleBackgroundErrors, nullptr}}, + {DB::Properties::kCurSizeActiveMemTable, + {false, nullptr, &InternalStats::HandleCurSizeActiveMemTable, + nullptr}}, + {DB::Properties::kCurSizeAllMemTables, + {false, nullptr, &InternalStats::HandleCurSizeAllMemTables, nullptr}}, + {DB::Properties::kSizeAllMemTables, + {false, nullptr, &InternalStats::HandleSizeAllMemTables, nullptr}}, + {DB::Properties::kNumEntriesActiveMemTable, + {false, nullptr, &InternalStats::HandleNumEntriesActiveMemTable, + nullptr}}, + {DB::Properties::kNumEntriesImmMemTables, + {false, nullptr, &InternalStats::HandleNumEntriesImmMemTables, + nullptr}}, + {DB::Properties::kNumDeletesActiveMemTable, + {false, nullptr, &InternalStats::HandleNumDeletesActiveMemTable, + nullptr}}, + {DB::Properties::kNumDeletesImmMemTables, + {false, nullptr, &InternalStats::HandleNumDeletesImmMemTables, + nullptr}}, + {DB::Properties::kEstimateNumKeys, + {false, nullptr, &InternalStats::HandleEstimateNumKeys, nullptr}}, + {DB::Properties::kEstimateTableReadersMem, + {true, nullptr, &InternalStats::HandleEstimateTableReadersMem, + nullptr}}, + {DB::Properties::kIsFileDeletionsEnabled, + {false, nullptr, &InternalStats::HandleIsFileDeletionsEnabled, + nullptr}}, + {DB::Properties::kNumSnapshots, + {false, nullptr, &InternalStats::HandleNumSnapshots, nullptr}}, + {DB::Properties::kOldestSnapshotTime, + {false, nullptr, &InternalStats::HandleOldestSnapshotTime, nullptr}}, + {DB::Properties::kNumLiveVersions, + {false, nullptr, &InternalStats::HandleNumLiveVersions, nullptr}}, + {DB::Properties::kCurrentSuperVersionNumber, + {false, nullptr, &InternalStats::HandleCurrentSuperVersionNumber, + nullptr}}, + {DB::Properties::kEstimateLiveDataSize, + {true, nullptr, &InternalStats::HandleEstimateLiveDataSize, nullptr}}, + {DB::Properties::kMinLogNumberToKeep, + {false, nullptr, &InternalStats::HandleMinLogNumberToKeep, nullptr}}, + {DB::Properties::kBaseLevel, + {false, nullptr, &InternalStats::HandleBaseLevel, nullptr}}, + {DB::Properties::kTotalSstFilesSize, + {false, nullptr, &InternalStats::HandleTotalSstFilesSize, nullptr}}, + {DB::Properties::kEstimatePendingCompactionBytes, + {false, nullptr, &InternalStats::HandleEstimatePendingCompactionBytes, + nullptr}}, + {DB::Properties::kNumRunningFlushes, + {false, nullptr, &InternalStats::HandleNumRunningFlushes, nullptr}}, + {DB::Properties::kNumRunningCompactions, + {false, nullptr, &InternalStats::HandleNumRunningCompactions, + nullptr}}, + {DB::Properties::kActualDelayedWriteRate, + {false, nullptr, &InternalStats::HandleActualDelayedWriteRate, + nullptr}}, + {DB::Properties::kIsWriteStopped, + {false, nullptr, &InternalStats::HandleIsWriteStopped, nullptr}}, + {DB::Properties::kEstimateOldestKeyTime, + {false, nullptr, &InternalStats::HandleEstimateOldestKeyTime, + nullptr}}, +}; + +const DBPropertyInfo* GetPropertyInfo(const Slice& property) { + std::string ppt_name = GetPropertyNameAndArg(property).first.ToString(); + auto ppt_info_iter = InternalStats::ppt_name_to_info.find(ppt_name); + if (ppt_info_iter == InternalStats::ppt_name_to_info.end()) { + return nullptr; } - return kUnknown; + return &ppt_info_iter->second; +} + +bool InternalStats::GetStringProperty(const DBPropertyInfo& property_info, + const Slice& property, + std::string* value) { + assert(value != nullptr); + assert(property_info.handle_string != nullptr); + Slice arg = GetPropertyNameAndArg(property).second; + return (this->*(property_info.handle_string))(value, arg); } -bool InternalStats::GetIntPropertyOutOfMutex(DBPropertyType property_type, - Version* version, - uint64_t* value) const { +bool InternalStats::GetMapProperty(const DBPropertyInfo& property_info, + const Slice& property, + std::map* value) { assert(value != nullptr); - if (property_type != kEstimatedUsageByTableReaders) { + assert(property_info.handle_map != nullptr); + return (this->*(property_info.handle_map))(value); +} + +bool InternalStats::GetIntProperty(const DBPropertyInfo& property_info, + uint64_t* value, DBImpl* db) { + assert(value != nullptr); + assert(property_info.handle_int != nullptr && + !property_info.need_out_of_mutex); + db->mutex_.AssertHeld(); + return (this->*(property_info.handle_int))(value, db, nullptr /* version */); +} + +bool InternalStats::GetIntPropertyOutOfMutex( + const DBPropertyInfo& property_info, Version* version, uint64_t* value) { + assert(value != nullptr); + assert(property_info.handle_int != nullptr && + property_info.need_out_of_mutex); + return (this->*(property_info.handle_int))(value, nullptr /* db */, version); +} + +bool InternalStats::HandleNumFilesAtLevel(std::string* value, Slice suffix) { + uint64_t level; + const auto* vstorage = cfd_->current()->storage_info(); + bool ok = ConsumeDecimalNumber(&suffix, &level) && suffix.empty(); + if (!ok || static_cast(level) >= number_levels_) { + return false; + } else { + char buf[100]; + snprintf(buf, sizeof(buf), "%d", + vstorage->NumLevelFiles(static_cast(level))); + *value = buf; + return true; + } +} + +bool InternalStats::HandleCompressionRatioAtLevelPrefix(std::string* value, + Slice suffix) { + uint64_t level; + const auto* vstorage = cfd_->current()->storage_info(); + bool ok = ConsumeDecimalNumber(&suffix, &level) && suffix.empty(); + if (!ok || level >= static_cast(number_levels_)) { + return false; + } + *value = ToString( + vstorage->GetEstimatedCompressionRatioAtLevel(static_cast(level))); + return true; +} + +bool InternalStats::HandleLevelStats(std::string* value, Slice suffix) { + char buf[1000]; + const auto* vstorage = cfd_->current()->storage_info(); + snprintf(buf, sizeof(buf), + "Level Files Size(MB)\n" + "--------------------\n"); + value->append(buf); + + for (int level = 0; level < number_levels_; level++) { + snprintf(buf, sizeof(buf), "%3d %8d %8.0f\n", level, + vstorage->NumLevelFiles(level), + vstorage->NumLevelBytes(level) / kMB); + value->append(buf); + } + return true; +} + +bool InternalStats::HandleStats(std::string* value, Slice suffix) { + if (!HandleCFStats(value, suffix)) { + return false; + } + if (!HandleDBStats(value, suffix)) { return false; } - if (version == nullptr) { + return true; +} + +bool InternalStats::HandleCFMapStats(std::map* cf_stats) { + DumpCFMapStats(cf_stats); + return true; +} + +bool InternalStats::HandleCFStats(std::string* value, Slice suffix) { + DumpCFStats(value); + return true; +} + +bool InternalStats::HandleCFStatsNoFileHistogram(std::string* value, + Slice suffix) { + DumpCFStatsNoFileHistogram(value); + return true; +} + +bool InternalStats::HandleCFFileHistogram(std::string* value, Slice suffix) { + DumpCFFileHistogram(value); + return true; +} + +bool InternalStats::HandleDBStats(std::string* value, Slice suffix) { + DumpDBStats(value); + return true; +} + +bool InternalStats::HandleSsTables(std::string* value, Slice suffix) { + auto* current = cfd_->current(); + *value = current->DebugString(true, true); + return true; +} + +bool InternalStats::HandleAggregatedTableProperties(std::string* value, + Slice suffix) { + std::shared_ptr tp; + auto s = cfd_->current()->GetAggregatedTableProperties(&tp); + if (!s.ok()) { + return false; + } + *value = tp->ToString(); + return true; +} + +bool InternalStats::HandleAggregatedTablePropertiesAtLevel(std::string* value, + Slice suffix) { + uint64_t level; + bool ok = ConsumeDecimalNumber(&suffix, &level) && suffix.empty(); + if (!ok || static_cast(level) >= number_levels_) { + return false; + } + std::shared_ptr tp; + auto s = cfd_->current()->GetAggregatedTableProperties( + &tp, static_cast(level)); + if (!s.ok()) { + return false; + } + *value = tp->ToString(); + return true; +} + +bool InternalStats::HandleNumImmutableMemTable(uint64_t* value, DBImpl* db, + Version* version) { + *value = cfd_->imm()->NumNotFlushed(); + return true; +} + +bool InternalStats::HandleNumImmutableMemTableFlushed(uint64_t* value, + DBImpl* db, + Version* version) { + *value = cfd_->imm()->NumFlushed(); + return true; +} + +bool InternalStats::HandleMemTableFlushPending(uint64_t* value, DBImpl* db, + Version* version) { + // Return number of mem tables that are ready to flush (made immutable) + *value = (cfd_->imm()->IsFlushPending() ? 1 : 0); + return true; +} + +bool InternalStats::HandleNumRunningFlushes(uint64_t* value, DBImpl* db, + Version* version) { + *value = db->num_running_flushes(); + return true; +} + +bool InternalStats::HandleCompactionPending(uint64_t* value, DBImpl* db, + Version* version) { + // 1 if the system already determines at least one compaction is needed. + // 0 otherwise, + const auto* vstorage = cfd_->current()->storage_info(); + *value = (cfd_->compaction_picker()->NeedsCompaction(vstorage) ? 1 : 0); + return true; +} + +bool InternalStats::HandleNumRunningCompactions(uint64_t* value, DBImpl* db, + Version* version) { + *value = db->num_running_compactions_; + return true; +} + +bool InternalStats::HandleBackgroundErrors(uint64_t* value, DBImpl* db, + Version* version) { + // Accumulated number of errors in background flushes or compactions. + *value = GetBackgroundErrorCount(); + return true; +} + +bool InternalStats::HandleCurSizeActiveMemTable(uint64_t* value, DBImpl* db, + Version* version) { + // Current size of the active memtable + *value = cfd_->mem()->ApproximateMemoryUsage(); + return true; +} + +bool InternalStats::HandleCurSizeAllMemTables(uint64_t* value, DBImpl* db, + Version* version) { + // Current size of the active memtable + immutable memtables + *value = cfd_->mem()->ApproximateMemoryUsage() + + cfd_->imm()->ApproximateUnflushedMemTablesMemoryUsage(); + return true; +} + +bool InternalStats::HandleSizeAllMemTables(uint64_t* value, DBImpl* db, + Version* version) { + *value = cfd_->mem()->ApproximateMemoryUsage() + + cfd_->imm()->ApproximateMemoryUsage(); + return true; +} + +bool InternalStats::HandleNumEntriesActiveMemTable(uint64_t* value, DBImpl* db, + Version* version) { + // Current number of entires in the active memtable + *value = cfd_->mem()->num_entries(); + return true; +} + +bool InternalStats::HandleNumEntriesImmMemTables(uint64_t* value, DBImpl* db, + Version* version) { + // Current number of entries in the immutable memtables + *value = cfd_->imm()->current()->GetTotalNumEntries(); + return true; +} + +bool InternalStats::HandleNumDeletesActiveMemTable(uint64_t* value, DBImpl* db, + Version* version) { + // Current number of entires in the active memtable + *value = cfd_->mem()->num_deletes(); + return true; +} + +bool InternalStats::HandleNumDeletesImmMemTables(uint64_t* value, DBImpl* db, + Version* version) { + // Current number of entries in the immutable memtables + *value = cfd_->imm()->current()->GetTotalNumDeletes(); + return true; +} + +bool InternalStats::HandleEstimateNumKeys(uint64_t* value, DBImpl* db, + Version* version) { + // Estimate number of entries in the column family: + // Use estimated entries in tables + total entries in memtables. + const auto* vstorage = cfd_->current()->storage_info(); + uint64_t estimate_keys = cfd_->mem()->num_entries() + + cfd_->imm()->current()->GetTotalNumEntries() + + vstorage->GetEstimatedActiveKeys(); + uint64_t estimate_deletes = + cfd_->mem()->num_deletes() + cfd_->imm()->current()->GetTotalNumDeletes(); + *value = estimate_keys > estimate_deletes * 2 + ? estimate_keys - (estimate_deletes * 2) + : 0; + return true; +} + +bool InternalStats::HandleNumSnapshots(uint64_t* value, DBImpl* db, + Version* version) { + *value = db->snapshots().count(); + return true; +} + +bool InternalStats::HandleOldestSnapshotTime(uint64_t* value, DBImpl* db, + Version* version) { + *value = static_cast(db->snapshots().GetOldestSnapshotTime()); + return true; +} + +bool InternalStats::HandleNumLiveVersions(uint64_t* value, DBImpl* db, + Version* version) { + *value = cfd_->GetNumLiveVersions(); + return true; +} + +bool InternalStats::HandleCurrentSuperVersionNumber(uint64_t* value, DBImpl* db, + Version* version) { + *value = cfd_->GetSuperVersionNumber(); + return true; +} + +bool InternalStats::HandleIsFileDeletionsEnabled(uint64_t* value, DBImpl* db, + Version* version) { + *value = db->IsFileDeletionsEnabled(); + return true; +} + +bool InternalStats::HandleBaseLevel(uint64_t* value, DBImpl* db, + Version* version) { + const auto* vstorage = cfd_->current()->storage_info(); + *value = vstorage->base_level(); + return true; +} + +bool InternalStats::HandleTotalSstFilesSize(uint64_t* value, DBImpl* db, + Version* version) { + *value = cfd_->GetTotalSstFilesSize(); + return true; +} + +bool InternalStats::HandleEstimatePendingCompactionBytes(uint64_t* value, + DBImpl* db, + Version* version) { + const auto* vstorage = cfd_->current()->storage_info(); + *value = vstorage->estimated_compaction_needed_bytes(); + return true; +} + +bool InternalStats::HandleEstimateTableReadersMem(uint64_t* value, DBImpl* db, + Version* version) { + *value = (version == nullptr) ? 0 : version->GetMemoryUsageByTableReaders(); + return true; +} + +bool InternalStats::HandleEstimateLiveDataSize(uint64_t* value, DBImpl* db, + Version* version) { + const auto* vstorage = cfd_->current()->storage_info(); + *value = vstorage->EstimateLiveDataSize(); + return true; +} + +bool InternalStats::HandleMinLogNumberToKeep(uint64_t* value, DBImpl* db, + Version* version) { + *value = db->MinLogNumberToKeep(); + return true; +} + +bool InternalStats::HandleActualDelayedWriteRate(uint64_t* value, DBImpl* db, + Version* version) { + const WriteController& wc = db->write_controller(); + if (!wc.NeedsDelay()) { *value = 0; } else { - *value = version->GetMemoryUsageByTableReaders(); + *value = wc.delayed_write_rate(); } return true; } -bool InternalStats::GetStringProperty(DBPropertyType property_type, - const Slice& property, - std::string* value) { - assert(value != nullptr); - Version* current = cfd_->current(); - Slice in = property; - - switch (property_type) { - case kNumFilesAtLevel: { - in.remove_prefix(strlen("rocksdb.num-files-at-level")); - uint64_t level; - bool ok = ConsumeDecimalNumber(&in, &level) && in.empty(); - if (!ok || (int)level >= number_levels_) { - return false; - } else { - char buf[100]; - snprintf(buf, sizeof(buf), "%d", - current->NumLevelFiles(static_cast(level))); - *value = buf; - return true; - } - } - case kLevelStats: { - char buf[1000]; - snprintf(buf, sizeof(buf), - "Level Files Size(MB)\n" - "--------------------\n"); - value->append(buf); +bool InternalStats::HandleIsWriteStopped(uint64_t* value, DBImpl* db, + Version* version) { + *value = db->write_controller().IsStopped() ? 1 : 0; + return true; +} - for (int level = 0; level < number_levels_; level++) { - snprintf(buf, sizeof(buf), "%3d %8d %8.0f\n", level, - current->NumLevelFiles(level), - current->NumLevelBytes(level) / kMB); - value->append(buf); - } - return true; - } - case kStats: { - if (!GetStringProperty(kCFStats, "rocksdb.cfstats", value)) { - return false; - } - if (!GetStringProperty(kDBStats, "rocksdb.dbstats", value)) { - return false; - } - return true; - } - case kCFStats: { - DumpCFStats(value); - return true; - } - case kDBStats: { - DumpDBStats(value); - return true; - } - case kSsTables: - *value = current->DebugString(); - return true; - default: - return false; +bool InternalStats::HandleEstimateOldestKeyTime(uint64_t* value, DBImpl* /*db*/, + Version* /*version*/) { + // TODO(yiwu): The property is currently available for fifo compaction + // with allow_compaction = false. This is because we don't propagate + // oldest_key_time on compaction. + if (cfd_->ioptions()->compaction_style != kCompactionStyleFIFO || + cfd_->ioptions()->compaction_options_fifo.allow_compaction) { + return false; } -} -bool InternalStats::GetIntProperty(DBPropertyType property_type, - uint64_t* value, DBImpl* db) const { - Version* current = cfd_->current(); - - switch (property_type) { - case kNumImmutableMemTable: - *value = cfd_->imm()->size(); - return true; - case kMemtableFlushPending: - // Return number of mem tables that are ready to flush (made immutable) - *value = (cfd_->imm()->IsFlushPending() ? 1 : 0); - return true; - case kCompactionPending: - // 1 if the system already determines at least one compacdtion is needed. - // 0 otherwise, - *value = (current->NeedsCompaction() ? 1 : 0); - return true; - case kBackgroundErrors: - // Accumulated number of errors in background flushes or compactions. - *value = GetBackgroundErrorCount(); - return true; - case kCurSizeActiveMemTable: - // Current size of the active memtable - *value = cfd_->mem()->ApproximateMemoryUsage(); - return true; - case kNumEntriesInMutableMemtable: - // Current size of the active memtable - *value = cfd_->mem()->GetNumEntries(); - return true; - case kNumEntriesInImmutableMemtable: - // Current size of the active memtable - *value = cfd_->imm()->current()->GetTotalNumEntries(); - return true; - case kEstimatedNumKeys: - // Estimate number of entries in the column family: - // Use estimated entries in tables + total entries in memtables. - *value = cfd_->mem()->GetNumEntries() + - cfd_->imm()->current()->GetTotalNumEntries() + - current->GetEstimatedActiveKeys(); - return true; -#ifndef ROCKSDB_LITE - case kIsFileDeletionEnabled: - *value = db->IsFileDeletionsEnabled(); - return true; -#endif - default: - return false; + TablePropertiesCollection collection; + auto s = cfd_->current()->GetPropertiesOfAllTables(&collection); + if (!s.ok()) { + return false; + } + *value = std::numeric_limits::max(); + for (auto& p : collection) { + *value = std::min(*value, p.second->oldest_key_time); + if (*value == 0) { + break; + } + } + if (*value > 0) { + *value = std::min({cfd_->mem()->ApproximateOldestKeyTime(), + cfd_->imm()->ApproximateOldestKeyTime(), *value}); } + return *value > 0 && *value < std::numeric_limits::max(); } void InternalStats::DumpDBStats(std::string* value) { char buf[1000]; // DB-level stats, only available from default column family - double seconds_up = (env_->NowMicros() - started_at_ + 1) / 1000000.0; + double seconds_up = (env_->NowMicros() - started_at_ + 1) / kMicrosInSec; double interval_seconds_up = seconds_up - db_stats_snapshot_.seconds_up; snprintf(buf, sizeof(buf), "\n** DB Stats **\nUptime(secs): %.1f total, %.1f interval\n", seconds_up, interval_seconds_up); value->append(buf); // Cumulative - uint64_t user_bytes_written = db_stats_[InternalStats::BYTES_WRITTEN]; - uint64_t write_other = db_stats_[InternalStats::WRITE_DONE_BY_OTHER]; - uint64_t write_self = db_stats_[InternalStats::WRITE_DONE_BY_SELF]; - uint64_t wal_bytes = db_stats_[InternalStats::WAL_FILE_BYTES]; - uint64_t wal_synced = db_stats_[InternalStats::WAL_FILE_SYNCED]; - uint64_t write_with_wal = db_stats_[InternalStats::WRITE_WITH_WAL]; + uint64_t user_bytes_written = GetDBStats(InternalStats::BYTES_WRITTEN); + uint64_t num_keys_written = GetDBStats(InternalStats::NUMBER_KEYS_WRITTEN); + uint64_t write_other = GetDBStats(InternalStats::WRITE_DONE_BY_OTHER); + uint64_t write_self = GetDBStats(InternalStats::WRITE_DONE_BY_SELF); + uint64_t wal_bytes = GetDBStats(InternalStats::WAL_FILE_BYTES); + uint64_t wal_synced = GetDBStats(InternalStats::WAL_FILE_SYNCED); + uint64_t write_with_wal = GetDBStats(InternalStats::WRITE_WITH_WAL); + uint64_t write_stall_micros = GetDBStats(InternalStats::WRITE_STALL_MICROS); + + const int kHumanMicrosLen = 32; + char human_micros[kHumanMicrosLen]; + // Data + // writes: total number of write requests. + // keys: total number of key updates issued by all the write requests + // commit groups: number of group commits issued to the DB. Each group can + // contain one or more writes. + // so writes/keys is the average number of put in multi-put or put + // writes/groups is the average group commit size. + // + // The format is the same for interval stats. snprintf(buf, sizeof(buf), - "Cumulative writes: %" PRIu64 " writes, %" PRIu64 " batches, " - "%.1f writes per batch, %.2f GB user ingest\n", - write_other + write_self, write_self, + "Cumulative writes: %s writes, %s keys, %s commit groups, " + "%.1f writes per commit group, ingest: %.2f GB, %.2f MB/s\n", + NumberToHumanString(write_other + write_self).c_str(), + NumberToHumanString(num_keys_written).c_str(), + NumberToHumanString(write_self).c_str(), (write_other + write_self) / static_cast(write_self + 1), - user_bytes_written / kGB); + user_bytes_written / kGB, user_bytes_written / kMB / seconds_up); value->append(buf); // WAL snprintf(buf, sizeof(buf), - "Cumulative WAL: %" PRIu64 " writes, %" PRIu64 " syncs, " - "%.2f writes per sync, %.2f GB written\n", - write_with_wal, wal_synced, + "Cumulative WAL: %s writes, %s syncs, " + "%.2f writes per sync, written: %.2f GB, %.2f MB/s\n", + NumberToHumanString(write_with_wal).c_str(), + NumberToHumanString(wal_synced).c_str(), write_with_wal / static_cast(wal_synced + 1), - wal_bytes / kGB); + wal_bytes / kGB, wal_bytes / kMB / seconds_up); + value->append(buf); + // Stall + AppendHumanMicros(write_stall_micros, human_micros, kHumanMicrosLen, true); + snprintf(buf, sizeof(buf), + "Cumulative stall: %s, %.1f percent\n", + human_micros, + // 10000 = divide by 1M to get secs, then multiply by 100 for pct + write_stall_micros / 10000.0 / std::max(seconds_up, 0.001)); value->append(buf); // Interval uint64_t interval_write_other = write_other - db_stats_snapshot_.write_other; uint64_t interval_write_self = write_self - db_stats_snapshot_.write_self; + uint64_t interval_num_keys_written = + num_keys_written - db_stats_snapshot_.num_keys_written; snprintf(buf, sizeof(buf), - "Interval writes: %" PRIu64 " writes, %" PRIu64 " batches, " - "%.1f writes per batch, %.1f MB user ingest\n", - interval_write_other + interval_write_self, - interval_write_self, + "Interval writes: %s writes, %s keys, %s commit groups, " + "%.1f writes per commit group, ingest: %.2f MB, %.2f MB/s\n", + NumberToHumanString( + interval_write_other + interval_write_self).c_str(), + NumberToHumanString(interval_num_keys_written).c_str(), + NumberToHumanString(interval_write_self).c_str(), static_cast(interval_write_other + interval_write_self) / (interval_write_self + 1), - (user_bytes_written - db_stats_snapshot_.ingest_bytes) / kMB); + (user_bytes_written - db_stats_snapshot_.ingest_bytes) / kMB, + (user_bytes_written - db_stats_snapshot_.ingest_bytes) / kMB / + std::max(interval_seconds_up, 0.001)), value->append(buf); uint64_t interval_write_with_wal = @@ -319,160 +894,317 @@ void InternalStats::DumpDBStats(std::string* value) { uint64_t interval_wal_bytes = wal_bytes - db_stats_snapshot_.wal_bytes; snprintf(buf, sizeof(buf), - "Interval WAL: %" PRIu64 " writes, %" PRIu64 " syncs, " - "%.2f writes per sync, %.2f MB written\n", - interval_write_with_wal, - interval_wal_synced, + "Interval WAL: %s writes, %s syncs, " + "%.2f writes per sync, written: %.2f MB, %.2f MB/s\n", + NumberToHumanString(interval_write_with_wal).c_str(), + NumberToHumanString(interval_wal_synced).c_str(), interval_write_with_wal / static_cast(interval_wal_synced + 1), - interval_wal_bytes / kGB); + interval_wal_bytes / kGB, + interval_wal_bytes / kMB / std::max(interval_seconds_up, 0.001)); + value->append(buf); + + // Stall + AppendHumanMicros( + write_stall_micros - db_stats_snapshot_.write_stall_micros, + human_micros, kHumanMicrosLen, true); + snprintf(buf, sizeof(buf), + "Interval stall: %s, %.1f percent\n", + human_micros, + // 10000 = divide by 1M to get secs, then multiply by 100 for pct + (write_stall_micros - db_stats_snapshot_.write_stall_micros) / + 10000.0 / std::max(interval_seconds_up, 0.001)); value->append(buf); db_stats_snapshot_.seconds_up = seconds_up; db_stats_snapshot_.ingest_bytes = user_bytes_written; db_stats_snapshot_.write_other = write_other; db_stats_snapshot_.write_self = write_self; + db_stats_snapshot_.num_keys_written = num_keys_written; db_stats_snapshot_.wal_bytes = wal_bytes; db_stats_snapshot_.wal_synced = wal_synced; db_stats_snapshot_.write_with_wal = write_with_wal; + db_stats_snapshot_.write_stall_micros = write_stall_micros; } -void InternalStats::DumpCFStats(std::string* value) { - Version* current = cfd_->current(); +/** + * Dump Compaction Level stats to a map of stat name to value in double. + * The level in stat name is represented with a prefix "Lx" where "x" + * is the level number. A special level "Sum" represents the sum of a stat + * for all levels. + */ +void InternalStats::DumpCFMapStats(std::map* cf_stats) { + CompactionStats compaction_stats_sum(0); + std::map> levels_stats; + DumpCFMapStats(&levels_stats, &compaction_stats_sum); + for (auto const& level_ent : levels_stats) { + auto level_str = + level_ent.first == -1 ? "Sum" : "L" + ToString(level_ent.first); + for (auto const& stat_ent : level_ent.second) { + auto stat_type = stat_ent.first; + auto key_str = + level_str + "." + + InternalStats::compaction_level_stats.at(stat_type).property_name; + (*cf_stats)[key_str] = stat_ent.second; + } + } +} + +void InternalStats::DumpCFMapStats( + std::map>* levels_stats, + CompactionStats* compaction_stats_sum) { + const VersionStorageInfo* vstorage = cfd_->current()->storage_info(); int num_levels_to_check = - (cfd_->options()->compaction_style != kCompactionStyleUniversal && - cfd_->options()->compaction_style != kCompactionStyleFIFO) - ? current->NumberLevels() - 1 + (cfd_->ioptions()->compaction_style != kCompactionStyleFIFO) + ? vstorage->num_levels() - 1 : 1; - // Compaction scores are sorted base on its value. Restore them to the + + // Compaction scores are sorted based on its value. Restore them to the // level order std::vector compaction_score(number_levels_, 0); for (int i = 0; i < num_levels_to_check; ++i) { - compaction_score[current->compaction_level_[i]] = - current->compaction_score_[i]; + compaction_score[vstorage->CompactionScoreLevel(i)] = + vstorage->CompactionScore(i); } // Count # of files being compacted for each level std::vector files_being_compacted(number_levels_, 0); - for (int level = 0; level < num_levels_to_check; ++level) { - for (auto* f : current->files_[level]) { + for (int level = 0; level < number_levels_; ++level) { + for (auto* f : vstorage->LevelFiles(level)) { if (f->being_compacted) { ++files_being_compacted[level]; } } } - char buf[1000]; - // Per-ColumnFamily stats - PrintLevelStatsHeader(buf, sizeof(buf), cfd_->GetName()); - value->append(buf); - - CompactionStats stats_sum(0); int total_files = 0; int total_files_being_compacted = 0; double total_file_size = 0; - uint64_t total_slowdown_soft = 0; - uint64_t total_slowdown_count_soft = 0; - uint64_t total_slowdown_hard = 0; - uint64_t total_slowdown_count_hard = 0; - uint64_t total_stall_count = 0; - double total_stall_us = 0; + uint64_t flush_ingest = cf_stats_value_[BYTES_FLUSHED]; + uint64_t add_file_ingest = cf_stats_value_[BYTES_INGESTED_ADD_FILE]; + uint64_t curr_ingest = flush_ingest + add_file_ingest; for (int level = 0; level < number_levels_; level++) { - int files = current->NumLevelFiles(level); + int files = vstorage->NumLevelFiles(level); total_files += files; total_files_being_compacted += files_being_compacted[level]; if (comp_stats_[level].micros > 0 || files > 0) { - uint64_t stalls = level == 0 ? - (cf_stats_count_[LEVEL0_SLOWDOWN] + - cf_stats_count_[LEVEL0_NUM_FILES] + - cf_stats_count_[MEMTABLE_COMPACTION]) - : (stall_leveln_slowdown_count_soft_[level] + - stall_leveln_slowdown_count_hard_[level]); - - double stall_us = level == 0 ? - (cf_stats_value_[LEVEL0_SLOWDOWN] + - cf_stats_value_[LEVEL0_NUM_FILES] + - cf_stats_value_[MEMTABLE_COMPACTION]) - : (stall_leveln_slowdown_soft_[level] + - stall_leveln_slowdown_hard_[level]); - - stats_sum.Add(comp_stats_[level]); - total_file_size += current->NumLevelBytes(level); - total_stall_us += stall_us; - total_stall_count += stalls; - total_slowdown_soft += stall_leveln_slowdown_soft_[level]; - total_slowdown_count_soft += stall_leveln_slowdown_count_soft_[level]; - total_slowdown_hard += stall_leveln_slowdown_hard_[level]; - total_slowdown_count_hard += stall_leveln_slowdown_count_hard_[level]; - int64_t bytes_read = comp_stats_[level].bytes_readn + - comp_stats_[level].bytes_readnp1; - double rw_amp = (comp_stats_[level].bytes_readn == 0) ? 0.0 - : (comp_stats_[level].bytes_written + bytes_read) / - static_cast(comp_stats_[level].bytes_readn); - double w_amp = (comp_stats_[level].bytes_readn == 0) ? 0.0 - : comp_stats_[level].bytes_written / - static_cast(comp_stats_[level].bytes_readn); - PrintLevelStats(buf, sizeof(buf), "L" + std::to_string(level), - files, files_being_compacted[level], current->NumLevelBytes(level), - compaction_score[level], rw_amp, w_amp, stall_us, stalls, - comp_stats_[level]); - value->append(buf); + compaction_stats_sum->Add(comp_stats_[level]); + total_file_size += vstorage->NumLevelBytes(level); + uint64_t input_bytes; + if (level == 0) { + input_bytes = curr_ingest; + } else { + input_bytes = comp_stats_[level].bytes_read_non_output_levels; + } + double w_amp = + (input_bytes == 0) + ? 0.0 + : static_cast(comp_stats_[level].bytes_written) / + input_bytes; + std::map level_stats; + PrepareLevelStats(&level_stats, files, files_being_compacted[level], + static_cast(vstorage->NumLevelBytes(level)), + compaction_score[level], w_amp, comp_stats_[level]); + (*levels_stats)[level] = level_stats; } } - uint64_t curr_ingest = cf_stats_value_[BYTES_FLUSHED]; // Cumulative summary - double rw_amp = (stats_sum.bytes_written + stats_sum.bytes_readn + - stats_sum.bytes_readnp1) / static_cast(curr_ingest + 1); - double w_amp = stats_sum.bytes_written / static_cast(curr_ingest + 1); + double w_amp = compaction_stats_sum->bytes_written / + static_cast(curr_ingest + 1); // Stats summary across levels - PrintLevelStats(buf, sizeof(buf), "Sum", total_files, - total_files_being_compacted, total_file_size, 0, rw_amp, w_amp, - total_stall_us, total_stall_count, stats_sum); + std::map sum_stats; + PrepareLevelStats(&sum_stats, total_files, total_files_being_compacted, + total_file_size, 0, w_amp, *compaction_stats_sum); + (*levels_stats)[-1] = sum_stats; // -1 is for the Sum level +} + +void InternalStats::DumpCFStats(std::string* value) { + DumpCFStatsNoFileHistogram(value); + DumpCFFileHistogram(value); +} + +void InternalStats::DumpCFStatsNoFileHistogram(std::string* value) { + char buf[2000]; + // Per-ColumnFamily stats + PrintLevelStatsHeader(buf, sizeof(buf), cfd_->GetName()); + value->append(buf); + + // Print stats for each level + std::map> levels_stats; + CompactionStats compaction_stats_sum(0); + DumpCFMapStats(&levels_stats, &compaction_stats_sum); + for (int l = 0; l < number_levels_; ++l) { + if (levels_stats.find(l) != levels_stats.end()) { + PrintLevelStats(buf, sizeof(buf), "L" + ToString(l), levels_stats[l]); + value->append(buf); + } + } + + // Print sum of level stats + PrintLevelStats(buf, sizeof(buf), "Sum", levels_stats[-1]); value->append(buf); + + uint64_t flush_ingest = cf_stats_value_[BYTES_FLUSHED]; + uint64_t add_file_ingest = cf_stats_value_[BYTES_INGESTED_ADD_FILE]; + uint64_t ingest_files_addfile = cf_stats_value_[INGESTED_NUM_FILES_TOTAL]; + uint64_t ingest_l0_files_addfile = + cf_stats_value_[INGESTED_LEVEL0_NUM_FILES_TOTAL]; + uint64_t ingest_keys_addfile = cf_stats_value_[INGESTED_NUM_KEYS_TOTAL]; + // Cumulative summary + uint64_t total_stall_count = + cf_stats_count_[LEVEL0_SLOWDOWN_TOTAL] + + cf_stats_count_[LEVEL0_NUM_FILES_TOTAL] + + cf_stats_count_[SOFT_PENDING_COMPACTION_BYTES_LIMIT] + + cf_stats_count_[HARD_PENDING_COMPACTION_BYTES_LIMIT] + + cf_stats_count_[MEMTABLE_COMPACTION] + cf_stats_count_[MEMTABLE_SLOWDOWN]; // Interval summary + uint64_t interval_flush_ingest = + flush_ingest - cf_stats_snapshot_.ingest_bytes_flush; + uint64_t interval_add_file_inget = + add_file_ingest - cf_stats_snapshot_.ingest_bytes_addfile; uint64_t interval_ingest = - curr_ingest - cf_stats_snapshot_.ingest_bytes + 1; - CompactionStats interval_stats(stats_sum); + interval_flush_ingest + interval_add_file_inget + 1; + CompactionStats interval_stats(compaction_stats_sum); interval_stats.Subtract(cf_stats_snapshot_.comp_stats); - rw_amp = (interval_stats.bytes_written + - interval_stats.bytes_readn + interval_stats.bytes_readnp1) / - static_cast(interval_ingest); - w_amp = interval_stats.bytes_written / static_cast(interval_ingest); - PrintLevelStats(buf, sizeof(buf), "Int", 0, 0, 0, 0, - rw_amp, w_amp, total_stall_us - cf_stats_snapshot_.stall_us, - total_stall_count - cf_stats_snapshot_.stall_count, interval_stats); + double w_amp = + interval_stats.bytes_written / static_cast(interval_ingest); + PrintLevelStats(buf, sizeof(buf), "Int", 0, 0, 0, 0, w_amp, interval_stats); + value->append(buf); + + double seconds_up = (env_->NowMicros() - started_at_ + 1) / kMicrosInSec; + double interval_seconds_up = seconds_up - cf_stats_snapshot_.seconds_up; + snprintf(buf, sizeof(buf), "Uptime(secs): %.1f total, %.1f interval\n", + seconds_up, interval_seconds_up); + value->append(buf); + snprintf(buf, sizeof(buf), "Flush(GB): cumulative %.3f, interval %.3f\n", + flush_ingest / kGB, interval_flush_ingest / kGB); + value->append(buf); + snprintf(buf, sizeof(buf), "AddFile(GB): cumulative %.3f, interval %.3f\n", + add_file_ingest / kGB, interval_add_file_inget / kGB); + value->append(buf); + + uint64_t interval_ingest_files_addfile = + ingest_files_addfile - cf_stats_snapshot_.ingest_files_addfile; + snprintf(buf, sizeof(buf), "AddFile(Total Files): cumulative %" PRIu64 + ", interval %" PRIu64 "\n", + ingest_files_addfile, interval_ingest_files_addfile); value->append(buf); + uint64_t interval_ingest_l0_files_addfile = + ingest_l0_files_addfile - cf_stats_snapshot_.ingest_l0_files_addfile; snprintf(buf, sizeof(buf), - "Flush(GB): accumulative %.3f, interval %.3f\n", - curr_ingest / kGB, interval_ingest / kGB); + "AddFile(L0 Files): cumulative %" PRIu64 ", interval %" PRIu64 "\n", + ingest_l0_files_addfile, interval_ingest_l0_files_addfile); value->append(buf); + + uint64_t interval_ingest_keys_addfile = + ingest_keys_addfile - cf_stats_snapshot_.ingest_keys_addfile; snprintf(buf, sizeof(buf), - "Stalls(secs): %.3f level0_slowdown, %.3f level0_numfiles, " - "%.3f memtable_compaction, %.3f leveln_slowdown_soft, " - "%.3f leveln_slowdown_hard\n", - cf_stats_value_[LEVEL0_SLOWDOWN] / 1000000.0, - cf_stats_value_[LEVEL0_NUM_FILES] / 1000000.0, - cf_stats_value_[MEMTABLE_COMPACTION] / 1000000.0, - total_slowdown_soft / 1000000.0, - total_slowdown_hard / 1000000.0); + "AddFile(Keys): cumulative %" PRIu64 ", interval %" PRIu64 "\n", + ingest_keys_addfile, interval_ingest_keys_addfile); value->append(buf); + // Compact + uint64_t compact_bytes_read = 0; + uint64_t compact_bytes_write = 0; + uint64_t compact_micros = 0; + for (int level = 0; level < number_levels_; level++) { + compact_bytes_read += comp_stats_[level].bytes_read_output_level + + comp_stats_[level].bytes_read_non_output_levels; + compact_bytes_write += comp_stats_[level].bytes_written; + compact_micros += comp_stats_[level].micros; + } + snprintf(buf, sizeof(buf), - "Stalls(count): %" PRIu64 " level0_slowdown, " - "%" PRIu64 " level0_numfiles, %" PRIu64 " memtable_compaction, " - "%" PRIu64 " leveln_slowdown_soft, " - "%" PRIu64 " leveln_slowdown_hard\n", - cf_stats_count_[LEVEL0_SLOWDOWN], - cf_stats_count_[LEVEL0_NUM_FILES], + "Cumulative compaction: %.2f GB write, %.2f MB/s write, " + "%.2f GB read, %.2f MB/s read, %.1f seconds\n", + compact_bytes_write / kGB, compact_bytes_write / kMB / seconds_up, + compact_bytes_read / kGB, compact_bytes_read / kMB / seconds_up, + compact_micros / kMicrosInSec); + value->append(buf); + + // Compaction interval + uint64_t interval_compact_bytes_write = + compact_bytes_write - cf_stats_snapshot_.compact_bytes_write; + uint64_t interval_compact_bytes_read = + compact_bytes_read - cf_stats_snapshot_.compact_bytes_read; + uint64_t interval_compact_micros = + compact_micros - cf_stats_snapshot_.compact_micros; + + snprintf( + buf, sizeof(buf), + "Interval compaction: %.2f GB write, %.2f MB/s write, " + "%.2f GB read, %.2f MB/s read, %.1f seconds\n", + interval_compact_bytes_write / kGB, + interval_compact_bytes_write / kMB / std::max(interval_seconds_up, 0.001), + interval_compact_bytes_read / kGB, + interval_compact_bytes_read / kMB / std::max(interval_seconds_up, 0.001), + interval_compact_micros / kMicrosInSec); + value->append(buf); + cf_stats_snapshot_.compact_bytes_write = compact_bytes_write; + cf_stats_snapshot_.compact_bytes_read = compact_bytes_read; + cf_stats_snapshot_.compact_micros = compact_micros; + + snprintf(buf, sizeof(buf), "Stalls(count): %" PRIu64 + " level0_slowdown, " + "%" PRIu64 + " level0_slowdown_with_compaction, " + "%" PRIu64 + " level0_numfiles, " + "%" PRIu64 + " level0_numfiles_with_compaction, " + "%" PRIu64 + " stop for pending_compaction_bytes, " + "%" PRIu64 + " slowdown for pending_compaction_bytes, " + "%" PRIu64 + " memtable_compaction, " + "%" PRIu64 + " memtable_slowdown, " + "interval %" PRIu64 " total count\n", + cf_stats_count_[LEVEL0_SLOWDOWN_TOTAL], + cf_stats_count_[LEVEL0_SLOWDOWN_WITH_COMPACTION], + cf_stats_count_[LEVEL0_NUM_FILES_TOTAL], + cf_stats_count_[LEVEL0_NUM_FILES_WITH_COMPACTION], + cf_stats_count_[HARD_PENDING_COMPACTION_BYTES_LIMIT], + cf_stats_count_[SOFT_PENDING_COMPACTION_BYTES_LIMIT], cf_stats_count_[MEMTABLE_COMPACTION], - total_slowdown_count_soft, total_slowdown_count_hard); + cf_stats_count_[MEMTABLE_SLOWDOWN], + total_stall_count - cf_stats_snapshot_.stall_count); value->append(buf); - cf_stats_snapshot_.ingest_bytes = curr_ingest; - cf_stats_snapshot_.comp_stats = stats_sum; - cf_stats_snapshot_.stall_us = total_stall_us; + cf_stats_snapshot_.seconds_up = seconds_up; + cf_stats_snapshot_.ingest_bytes_flush = flush_ingest; + cf_stats_snapshot_.ingest_bytes_addfile = add_file_ingest; + cf_stats_snapshot_.ingest_files_addfile = ingest_files_addfile; + cf_stats_snapshot_.ingest_l0_files_addfile = ingest_l0_files_addfile; + cf_stats_snapshot_.ingest_keys_addfile = ingest_keys_addfile; + cf_stats_snapshot_.comp_stats = compaction_stats_sum; cf_stats_snapshot_.stall_count = total_stall_count; } +void InternalStats::DumpCFFileHistogram(std::string* value) { + char buf[2000]; + snprintf(buf, sizeof(buf), + "\n** File Read Latency Histogram By Level [%s] **\n", + cfd_->GetName().c_str()); + value->append(buf); + + for (int level = 0; level < number_levels_; level++) { + if (!file_read_latency_[level].Empty()) { + char buf2[5000]; + snprintf(buf2, sizeof(buf2), + "** Level %d read latency histogram (micros):\n%s\n", level, + file_read_latency_[level].ToString().c_str()); + value->append(buf2); + } + } +} + +#else + +const DBPropertyInfo* GetPropertyInfo(const Slice& property) { return nullptr; } + +#endif // !ROCKSDB_LITE + } // namespace rocksdb diff --git a/db/internal_stats.h b/db/internal_stats.h index 2e04f24e719..a0b8a902718 100644 --- a/db/internal_stats.h +++ b/db/internal_stats.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,10 +9,11 @@ // #pragma once -#include "db/version_set.h" - -#include +#include #include +#include + +#include "db/version_set.h" class ColumnFamilyData; @@ -21,43 +22,83 @@ namespace rocksdb { class MemTableList; class DBImpl; -enum DBPropertyType : uint32_t { - kUnknown, - kNumFilesAtLevel, // Number of files at a specific level - kLevelStats, // Return number of files and total sizes of each level - kCFStats, // Return general statitistics of CF - kDBStats, // Return general statitistics of DB - kStats, // Return general statitistics of both DB and CF - kSsTables, // Return a human readable string of current SST files - kStartIntTypes, // ---- Dummy value to indicate the start of integer values - kNumImmutableMemTable, // Return number of immutable mem tables - kMemtableFlushPending, // Return 1 if mem table flushing is pending, - // otherwise 0. - kCompactionPending, // Return 1 if a compaction is pending. Otherwise 0. - kBackgroundErrors, // Return accumulated background errors encountered. - kCurSizeActiveMemTable, // Return current size of the active memtable - kNumEntriesInMutableMemtable, // Return number of entries in the mutable - // memtable. - kNumEntriesInImmutableMemtable, // Return sum of number of entries in all - // the immutable mem tables. - kEstimatedNumKeys, // Estimated total number of keys in the database. - kEstimatedUsageByTableReaders, // Estimated memory by table readers. - kIsFileDeletionEnabled, // Equals disable_delete_obsolete_files_, - // 0 means file deletions enabled +// Config for retrieving a property's value. +struct DBPropertyInfo { + bool need_out_of_mutex; + + // gcc had an internal error for initializing union of pointer-to-member- + // functions. Workaround is to populate exactly one of the following function + // pointers with a non-nullptr value. + + // @param value Value-result argument for storing the property's string value + // @param suffix Argument portion of the property. For example, suffix would + // be "5" for the property "rocksdb.num-files-at-level5". So far, only + // certain string properties take an argument. + bool (InternalStats::*handle_string)(std::string* value, Slice suffix); + + // @param value Value-result argument for storing the property's uint64 value + // @param db Many of the int properties rely on DBImpl methods. + // @param version Version is needed in case the property is retrieved without + // holding db mutex, which is only supported for int properties. + bool (InternalStats::*handle_int)(uint64_t* value, DBImpl* db, + Version* version); + bool (InternalStats::*handle_map)( + std::map* compaction_stats); }; -extern DBPropertyType GetPropertyType(const Slice& property, - bool* is_int_property, - bool* need_out_of_mutex); +extern const DBPropertyInfo* GetPropertyInfo(const Slice& property); + +#ifndef ROCKSDB_LITE +#undef SCORE +enum class LevelStatType { + INVALID = 0, + NUM_FILES, + COMPACTED_FILES, + SIZE_BYTES, + SCORE, + READ_GB, + RN_GB, + RNP1_GB, + WRITE_GB, + W_NEW_GB, + MOVED_GB, + WRITE_AMP, + READ_MBPS, + WRITE_MBPS, + COMP_SEC, + COMP_COUNT, + AVG_SEC, + KEY_IN, + KEY_DROP, + TOTAL // total number of types +}; + +struct LevelStat { + // This what will be L?.property_name in the flat map returned to the user + std::string property_name; + // This will be what we will print in the header in the cli + std::string header_name; +}; class InternalStats { public: + static const std::map compaction_level_stats; + enum InternalCFStatsType { - LEVEL0_SLOWDOWN, + LEVEL0_SLOWDOWN_TOTAL, + LEVEL0_SLOWDOWN_WITH_COMPACTION, MEMTABLE_COMPACTION, - LEVEL0_NUM_FILES, + MEMTABLE_SLOWDOWN, + LEVEL0_NUM_FILES_TOTAL, + LEVEL0_NUM_FILES_WITH_COMPACTION, + SOFT_PENDING_COMPACTION_BYTES_LIMIT, + HARD_PENDING_COMPACTION_BYTES_LIMIT, WRITE_STALLS_ENUM_MAX, BYTES_FLUSHED, + BYTES_INGESTED_ADD_FILE, + INGESTED_NUM_FILES_TOTAL, + INGESTED_LEVEL0_NUM_FILES_TOTAL, + INGESTED_NUM_KEYS_TOTAL, INTERNAL_CF_STATS_ENUM_MAX, }; @@ -65,122 +106,163 @@ class InternalStats { WAL_FILE_BYTES, WAL_FILE_SYNCED, BYTES_WRITTEN, + NUMBER_KEYS_WRITTEN, WRITE_DONE_BY_OTHER, WRITE_DONE_BY_SELF, WRITE_WITH_WAL, + WRITE_STALL_MICROS, INTERNAL_DB_STATS_ENUM_MAX, }; InternalStats(int num_levels, Env* env, ColumnFamilyData* cfd) - : db_stats_(INTERNAL_DB_STATS_ENUM_MAX), - cf_stats_value_(INTERNAL_CF_STATS_ENUM_MAX), - cf_stats_count_(INTERNAL_CF_STATS_ENUM_MAX), + : db_stats_{}, + cf_stats_value_{}, + cf_stats_count_{}, comp_stats_(num_levels), - stall_leveln_slowdown_hard_(num_levels), - stall_leveln_slowdown_count_hard_(num_levels), - stall_leveln_slowdown_soft_(num_levels), - stall_leveln_slowdown_count_soft_(num_levels), + file_read_latency_(num_levels), bg_error_count_(0), number_levels_(num_levels), env_(env), cfd_(cfd), - started_at_(env->NowMicros()) { - for (int i = 0; i< INTERNAL_DB_STATS_ENUM_MAX; ++i) { - db_stats_[i] = 0; - } - for (int i = 0; i< INTERNAL_CF_STATS_ENUM_MAX; ++i) { - cf_stats_value_[i] = 0; - cf_stats_count_[i] = 0; - } - for (int i = 0; i < num_levels; ++i) { - stall_leveln_slowdown_hard_[i] = 0; - stall_leveln_slowdown_count_hard_[i] = 0; - stall_leveln_slowdown_soft_[i] = 0; - stall_leveln_slowdown_count_soft_[i] = 0; - } - } + started_at_(env->NowMicros()) {} // Per level compaction stats. comp_stats_[level] stores the stats for // compactions that produced data for the specified "level". struct CompactionStats { uint64_t micros; - // Bytes read from level N during compaction between levels N and N+1 - uint64_t bytes_readn; + // The number of bytes read from all non-output levels + uint64_t bytes_read_non_output_levels; - // Bytes read from level N+1 during compaction between levels N and N+1 - uint64_t bytes_readnp1; + // The number of bytes read from the compaction output level. + uint64_t bytes_read_output_level; - // Total bytes written during compaction between levels N and N+1 + // Total number of bytes written during compaction uint64_t bytes_written; - // Files read from level N during compaction between levels N and N+1 - int files_in_leveln; + // Total number of bytes moved to the output level + uint64_t bytes_moved; + + // The number of compaction input files in all non-output levels. + int num_input_files_in_non_output_levels; + + // The number of compaction input files in the output level. + int num_input_files_in_output_level; - // Files read from level N+1 during compaction between levels N and N+1 - int files_in_levelnp1; + // The number of compaction output files. + int num_output_files; - // Files written during compaction between levels N and N+1 - int files_out_levelnp1; + // Total incoming entries during compaction between levels N and N+1 + uint64_t num_input_records; + + // Accumulated diff number of entries + // (num input entries - num output entires) for compaction levels N and N+1 + uint64_t num_dropped_records; // Number of compactions done int count; - explicit CompactionStats(int count = 0) + explicit CompactionStats(int _count = 0) : micros(0), - bytes_readn(0), - bytes_readnp1(0), + bytes_read_non_output_levels(0), + bytes_read_output_level(0), bytes_written(0), - files_in_leveln(0), - files_in_levelnp1(0), - files_out_levelnp1(0), - count(count) {} + bytes_moved(0), + num_input_files_in_non_output_levels(0), + num_input_files_in_output_level(0), + num_output_files(0), + num_input_records(0), + num_dropped_records(0), + count(_count) {} explicit CompactionStats(const CompactionStats& c) : micros(c.micros), - bytes_readn(c.bytes_readn), - bytes_readnp1(c.bytes_readnp1), + bytes_read_non_output_levels(c.bytes_read_non_output_levels), + bytes_read_output_level(c.bytes_read_output_level), bytes_written(c.bytes_written), - files_in_leveln(c.files_in_leveln), - files_in_levelnp1(c.files_in_levelnp1), - files_out_levelnp1(c.files_out_levelnp1), + bytes_moved(c.bytes_moved), + num_input_files_in_non_output_levels( + c.num_input_files_in_non_output_levels), + num_input_files_in_output_level( + c.num_input_files_in_output_level), + num_output_files(c.num_output_files), + num_input_records(c.num_input_records), + num_dropped_records(c.num_dropped_records), count(c.count) {} + void Clear() { + this->micros = 0; + this->bytes_read_non_output_levels = 0; + this->bytes_read_output_level = 0; + this->bytes_written = 0; + this->bytes_moved = 0; + this->num_input_files_in_non_output_levels = 0; + this->num_input_files_in_output_level = 0; + this->num_output_files = 0; + this->num_input_records = 0; + this->num_dropped_records = 0; + this->count = 0; + } + void Add(const CompactionStats& c) { this->micros += c.micros; - this->bytes_readn += c.bytes_readn; - this->bytes_readnp1 += c.bytes_readnp1; + this->bytes_read_non_output_levels += c.bytes_read_non_output_levels; + this->bytes_read_output_level += c.bytes_read_output_level; this->bytes_written += c.bytes_written; - this->files_in_leveln += c.files_in_leveln; - this->files_in_levelnp1 += c.files_in_levelnp1; - this->files_out_levelnp1 += c.files_out_levelnp1; + this->bytes_moved += c.bytes_moved; + this->num_input_files_in_non_output_levels += + c.num_input_files_in_non_output_levels; + this->num_input_files_in_output_level += + c.num_input_files_in_output_level; + this->num_output_files += c.num_output_files; + this->num_input_records += c.num_input_records; + this->num_dropped_records += c.num_dropped_records; this->count += c.count; } void Subtract(const CompactionStats& c) { this->micros -= c.micros; - this->bytes_readn -= c.bytes_readn; - this->bytes_readnp1 -= c.bytes_readnp1; + this->bytes_read_non_output_levels -= c.bytes_read_non_output_levels; + this->bytes_read_output_level -= c.bytes_read_output_level; this->bytes_written -= c.bytes_written; - this->files_in_leveln -= c.files_in_leveln; - this->files_in_levelnp1 -= c.files_in_levelnp1; - this->files_out_levelnp1 -= c.files_out_levelnp1; + this->bytes_moved -= c.bytes_moved; + this->num_input_files_in_non_output_levels -= + c.num_input_files_in_non_output_levels; + this->num_input_files_in_output_level -= + c.num_input_files_in_output_level; + this->num_output_files -= c.num_output_files; + this->num_input_records -= c.num_input_records; + this->num_dropped_records -= c.num_dropped_records; this->count -= c.count; } }; + void Clear() { + for (int i = 0; i < INTERNAL_DB_STATS_ENUM_MAX; i++) { + db_stats_[i].store(0); + } + for (int i = 0; i < INTERNAL_CF_STATS_ENUM_MAX; i++) { + cf_stats_count_[i] = 0; + cf_stats_value_[i] = 0; + } + for (auto& comp_stat : comp_stats_) { + comp_stat.Clear(); + } + for (auto& h : file_read_latency_) { + h.Clear(); + } + cf_stats_snapshot_.Clear(); + db_stats_snapshot_.Clear(); + bg_error_count_ = 0; + started_at_ = env_->NowMicros(); + } + void AddCompactionStats(int level, const CompactionStats& stats) { comp_stats_[level].Add(stats); } - void RecordLevelNSlowdown(int level, uint64_t micros, bool soft) { - if (soft) { - stall_leveln_slowdown_soft_[level] += micros; - ++stall_leveln_slowdown_count_soft_[level]; - } else { - stall_leveln_slowdown_hard_[level] += micros; - ++stall_leveln_slowdown_count_hard_[level]; - } + void IncBytesMoved(int level, uint64_t amount) { + comp_stats_[level].bytes_moved += amount; } void AddCFStats(InternalCFStatsType type, uint64_t value) { @@ -188,53 +270,109 @@ class InternalStats { ++cf_stats_count_[type]; } - void AddDBStats(InternalDBStatsType type, uint64_t value) { - db_stats_[type] += value; + void AddDBStats(InternalDBStatsType type, uint64_t value, + bool concurrent = false) { + auto& v = db_stats_[type]; + if (concurrent) { + v.fetch_add(value, std::memory_order_relaxed); + } else { + v.store(v.load(std::memory_order_relaxed) + value, + std::memory_order_relaxed); + } + } + + uint64_t GetDBStats(InternalDBStatsType type) { + return db_stats_[type].load(std::memory_order_relaxed); + } + + HistogramImpl* GetFileReadHist(int level) { + return &file_read_latency_[level]; } uint64_t GetBackgroundErrorCount() const { return bg_error_count_; } uint64_t BumpAndGetBackgroundErrorCount() { return ++bg_error_count_; } - bool GetStringProperty(DBPropertyType property_type, const Slice& property, - std::string* value); + bool GetStringProperty(const DBPropertyInfo& property_info, + const Slice& property, std::string* value); + + bool GetMapProperty(const DBPropertyInfo& property_info, + const Slice& property, + std::map* value); - bool GetIntProperty(DBPropertyType property_type, uint64_t* value, - DBImpl* db) const; + bool GetIntProperty(const DBPropertyInfo& property_info, uint64_t* value, + DBImpl* db); - bool GetIntPropertyOutOfMutex(DBPropertyType property_type, Version* version, - uint64_t* value) const; + bool GetIntPropertyOutOfMutex(const DBPropertyInfo& property_info, + Version* version, uint64_t* value); + + // Store a mapping from the user-facing DB::Properties string to our + // DBPropertyInfo struct used internally for retrieving properties. + static const std::unordered_map ppt_name_to_info; private: void DumpDBStats(std::string* value); + void DumpCFMapStats(std::map* cf_stats); + void DumpCFMapStats( + std::map>* level_stats, + CompactionStats* compaction_stats_sum); void DumpCFStats(std::string* value); + void DumpCFStatsNoFileHistogram(std::string* value); + void DumpCFFileHistogram(std::string* value); // Per-DB stats - std::vector db_stats_; + std::atomic db_stats_[INTERNAL_DB_STATS_ENUM_MAX]; // Per-ColumnFamily stats - std::vector cf_stats_value_; - std::vector cf_stats_count_; + uint64_t cf_stats_value_[INTERNAL_CF_STATS_ENUM_MAX]; + uint64_t cf_stats_count_[INTERNAL_CF_STATS_ENUM_MAX]; // Per-ColumnFamily/level compaction stats std::vector comp_stats_; - // These count the number of microseconds for which MakeRoomForWrite stalls. - std::vector stall_leveln_slowdown_hard_; - std::vector stall_leveln_slowdown_count_hard_; - std::vector stall_leveln_slowdown_soft_; - std::vector stall_leveln_slowdown_count_soft_; + std::vector file_read_latency_; // Used to compute per-interval statistics struct CFStatsSnapshot { // ColumnFamily-level stats CompactionStats comp_stats; - uint64_t ingest_bytes; // Bytes written to L0 - uint64_t stall_us; // Stall time in micro-seconds + uint64_t ingest_bytes_flush; // Bytes written to L0 (Flush) uint64_t stall_count; // Stall count + // Stats from compaction jobs - bytes written, bytes read, duration. + uint64_t compact_bytes_write; + uint64_t compact_bytes_read; + uint64_t compact_micros; + double seconds_up; + + // AddFile specific stats + uint64_t ingest_bytes_addfile; // Total Bytes ingested + uint64_t ingest_files_addfile; // Total number of files ingested + uint64_t ingest_l0_files_addfile; // Total number of files ingested to L0 + uint64_t ingest_keys_addfile; // Total number of keys ingested CFStatsSnapshot() : comp_stats(0), - ingest_bytes(0), - stall_us(0), - stall_count(0) {} + ingest_bytes_flush(0), + stall_count(0), + compact_bytes_write(0), + compact_bytes_read(0), + compact_micros(0), + seconds_up(0), + ingest_bytes_addfile(0), + ingest_files_addfile(0), + ingest_l0_files_addfile(0), + ingest_keys_addfile(0) {} + + void Clear() { + comp_stats.Clear(); + ingest_bytes_flush = 0; + stall_count = 0; + compact_bytes_write = 0; + compact_bytes_read = 0; + compact_micros = 0; + seconds_up = 0; + ingest_bytes_addfile = 0; + ingest_files_addfile = 0; + ingest_l0_files_addfile = 0; + ingest_keys_addfile = 0; + } } cf_stats_snapshot_; struct DBStatsSnapshot { @@ -247,6 +385,13 @@ class InternalStats { // another thread. uint64_t write_other; uint64_t write_self; + // Total number of keys written. write_self and write_other measure number + // of write requests written, Each of the write request can contain updates + // to multiple keys. num_keys_written is total number of keys updated by all + // those writes. + uint64_t num_keys_written; + // Total time writes delayed by stalls. + uint64_t write_stall_micros; double seconds_up; DBStatsSnapshot() @@ -256,9 +401,83 @@ class InternalStats { write_with_wal(0), write_other(0), write_self(0), + num_keys_written(0), + write_stall_micros(0), seconds_up(0) {} + + void Clear() { + ingest_bytes = 0; + wal_bytes = 0; + wal_synced = 0; + write_with_wal = 0; + write_other = 0; + write_self = 0; + num_keys_written = 0; + write_stall_micros = 0; + seconds_up = 0; + } } db_stats_snapshot_; + // Handler functions for getting property values. They use "value" as a value- + // result argument, and return true upon successfully setting "value". + bool HandleNumFilesAtLevel(std::string* value, Slice suffix); + bool HandleCompressionRatioAtLevelPrefix(std::string* value, Slice suffix); + bool HandleLevelStats(std::string* value, Slice suffix); + bool HandleStats(std::string* value, Slice suffix); + bool HandleCFMapStats(std::map* compaction_stats); + bool HandleCFStats(std::string* value, Slice suffix); + bool HandleCFStatsNoFileHistogram(std::string* value, Slice suffix); + bool HandleCFFileHistogram(std::string* value, Slice suffix); + bool HandleDBStats(std::string* value, Slice suffix); + bool HandleSsTables(std::string* value, Slice suffix); + bool HandleAggregatedTableProperties(std::string* value, Slice suffix); + bool HandleAggregatedTablePropertiesAtLevel(std::string* value, Slice suffix); + bool HandleNumImmutableMemTable(uint64_t* value, DBImpl* db, + Version* version); + bool HandleNumImmutableMemTableFlushed(uint64_t* value, DBImpl* db, + Version* version); + bool HandleMemTableFlushPending(uint64_t* value, DBImpl* db, + Version* version); + bool HandleNumRunningFlushes(uint64_t* value, DBImpl* db, Version* version); + bool HandleCompactionPending(uint64_t* value, DBImpl* db, Version* version); + bool HandleNumRunningCompactions(uint64_t* value, DBImpl* db, + Version* version); + bool HandleBackgroundErrors(uint64_t* value, DBImpl* db, Version* version); + bool HandleCurSizeActiveMemTable(uint64_t* value, DBImpl* db, + Version* version); + bool HandleCurSizeAllMemTables(uint64_t* value, DBImpl* db, Version* version); + bool HandleSizeAllMemTables(uint64_t* value, DBImpl* db, Version* version); + bool HandleNumEntriesActiveMemTable(uint64_t* value, DBImpl* db, + Version* version); + bool HandleNumEntriesImmMemTables(uint64_t* value, DBImpl* db, + Version* version); + bool HandleNumDeletesActiveMemTable(uint64_t* value, DBImpl* db, + Version* version); + bool HandleNumDeletesImmMemTables(uint64_t* value, DBImpl* db, + Version* version); + bool HandleEstimateNumKeys(uint64_t* value, DBImpl* db, Version* version); + bool HandleNumSnapshots(uint64_t* value, DBImpl* db, Version* version); + bool HandleOldestSnapshotTime(uint64_t* value, DBImpl* db, Version* version); + bool HandleNumLiveVersions(uint64_t* value, DBImpl* db, Version* version); + bool HandleCurrentSuperVersionNumber(uint64_t* value, DBImpl* db, + Version* version); + bool HandleIsFileDeletionsEnabled(uint64_t* value, DBImpl* db, + Version* version); + bool HandleBaseLevel(uint64_t* value, DBImpl* db, Version* version); + bool HandleTotalSstFilesSize(uint64_t* value, DBImpl* db, Version* version); + bool HandleEstimatePendingCompactionBytes(uint64_t* value, DBImpl* db, + Version* version); + bool HandleEstimateTableReadersMem(uint64_t* value, DBImpl* db, + Version* version); + bool HandleEstimateLiveDataSize(uint64_t* value, DBImpl* db, + Version* version); + bool HandleMinLogNumberToKeep(uint64_t* value, DBImpl* db, Version* version); + bool HandleActualDelayedWriteRate(uint64_t* value, DBImpl* db, + Version* version); + bool HandleIsWriteStopped(uint64_t* value, DBImpl* db, Version* version); + bool HandleEstimateOldestKeyTime(uint64_t* value, DBImpl* db, + Version* version); + // Total number of background errors encountered. Every time a flush task // or compaction task fails, this counter is incremented. The failure can // be caused by any possible reason, including file system errors, out of @@ -269,7 +488,103 @@ class InternalStats { const int number_levels_; Env* env_; ColumnFamilyData* cfd_; - const uint64_t started_at_; + uint64_t started_at_; +}; + +#else + +class InternalStats { + public: + enum InternalCFStatsType { + LEVEL0_SLOWDOWN_TOTAL, + LEVEL0_SLOWDOWN_WITH_COMPACTION, + MEMTABLE_COMPACTION, + MEMTABLE_SLOWDOWN, + LEVEL0_NUM_FILES_TOTAL, + LEVEL0_NUM_FILES_WITH_COMPACTION, + SOFT_PENDING_COMPACTION_BYTES_LIMIT, + HARD_PENDING_COMPACTION_BYTES_LIMIT, + WRITE_STALLS_ENUM_MAX, + BYTES_FLUSHED, + BYTES_INGESTED_ADD_FILE, + INGESTED_NUM_FILES_TOTAL, + INGESTED_LEVEL0_NUM_FILES_TOTAL, + INGESTED_NUM_KEYS_TOTAL, + INTERNAL_CF_STATS_ENUM_MAX, + }; + + enum InternalDBStatsType { + WAL_FILE_BYTES, + WAL_FILE_SYNCED, + BYTES_WRITTEN, + NUMBER_KEYS_WRITTEN, + WRITE_DONE_BY_OTHER, + WRITE_DONE_BY_SELF, + WRITE_WITH_WAL, + WRITE_STALL_MICROS, + INTERNAL_DB_STATS_ENUM_MAX, + }; + + InternalStats(int num_levels, Env* env, ColumnFamilyData* cfd) {} + + struct CompactionStats { + uint64_t micros; + uint64_t bytes_read_non_output_levels; + uint64_t bytes_read_output_level; + uint64_t bytes_written; + uint64_t bytes_moved; + int num_input_files_in_non_output_levels; + int num_input_files_in_output_level; + int num_output_files; + uint64_t num_input_records; + uint64_t num_dropped_records; + int count; + + explicit CompactionStats(int _count = 0) {} + + explicit CompactionStats(const CompactionStats& c) {} + + void Add(const CompactionStats& c) {} + + void Subtract(const CompactionStats& c) {} + }; + + void AddCompactionStats(int level, const CompactionStats& stats) {} + + void IncBytesMoved(int level, uint64_t amount) {} + + void AddCFStats(InternalCFStatsType type, uint64_t value) {} + + void AddDBStats(InternalDBStatsType type, uint64_t value, + bool concurrent = false) {} + + HistogramImpl* GetFileReadHist(int level) { return nullptr; } + + uint64_t GetBackgroundErrorCount() const { return 0; } + + uint64_t BumpAndGetBackgroundErrorCount() { return 0; } + + bool GetStringProperty(const DBPropertyInfo& property_info, + const Slice& property, std::string* value) { + return false; + } + + bool GetMapProperty(const DBPropertyInfo& property_info, + const Slice& property, + std::map* value) { + return false; + } + + bool GetIntProperty(const DBPropertyInfo& property_info, uint64_t* value, + DBImpl* db) const { + return false; + } + + bool GetIntPropertyOutOfMutex(const DBPropertyInfo& property_info, + Version* version, uint64_t* value) const { + return false; + } }; +#endif // !ROCKSDB_LITE } // namespace rocksdb diff --git a/db/job_context.h b/db/job_context.h new file mode 100644 index 00000000000..950a3a667db --- /dev/null +++ b/db/job_context.h @@ -0,0 +1,129 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include +#include + +#include "db/log_writer.h" + +namespace rocksdb { + +class MemTable; + +struct JobContext { + inline bool HaveSomethingToDelete() const { + return full_scan_candidate_files.size() || sst_delete_files.size() || + log_delete_files.size() || manifest_delete_files.size() || + new_superversion != nullptr || superversions_to_free.size() > 0 || + memtables_to_free.size() > 0 || logs_to_free.size() > 0; + } + + // Structure to store information for candidate files to delete. + struct CandidateFileInfo { + std::string file_name; + uint32_t path_id; + CandidateFileInfo(std::string name, uint32_t path) + : file_name(std::move(name)), path_id(path) {} + bool operator==(const CandidateFileInfo& other) const { + return file_name == other.file_name && path_id == other.path_id; + } + }; + + // Unique job id + int job_id; + + // a list of all files that we'll consider deleting + // (every once in a while this is filled up with all files + // in the DB directory) + // (filled only if we're doing full scan) + std::vector full_scan_candidate_files; + + // the list of all live sst files that cannot be deleted + std::vector sst_live; + + // a list of sst files that we need to delete + std::vector sst_delete_files; + + // a list of log files that we need to delete + std::vector log_delete_files; + + // a list of log files that we need to preserve during full purge since they + // will be reused later + std::vector log_recycle_files; + + // a list of manifest files that we need to delete + std::vector manifest_delete_files; + + // a list of memtables to be free + autovector memtables_to_free; + + autovector superversions_to_free; + + autovector logs_to_free; + + SuperVersion* new_superversion; // if nullptr no new superversion + + // the current manifest_file_number, log_number and prev_log_number + // that corresponds to the set of files in 'live'. + uint64_t manifest_file_number; + uint64_t pending_manifest_file_number; + uint64_t log_number; + uint64_t prev_log_number; + + uint64_t min_pending_output = 0; + uint64_t prev_total_log_size = 0; + size_t num_alive_log_files = 0; + uint64_t size_log_to_delete = 0; + + explicit JobContext(int _job_id, bool create_superversion = false) { + job_id = _job_id; + manifest_file_number = 0; + pending_manifest_file_number = 0; + log_number = 0; + prev_log_number = 0; + new_superversion = create_superversion ? new SuperVersion() : nullptr; + } + + // For non-empty JobContext Clean() has to be called at least once before + // before destruction (see asserts in ~JobContext()). Should be called with + // unlocked DB mutex. Destructor doesn't call Clean() to avoid accidentally + // doing potentially slow Clean() with locked DB mutex. + void Clean() { + // free pending memtables + for (auto m : memtables_to_free) { + delete m; + } + // free superversions + for (auto s : superversions_to_free) { + delete s; + } + for (auto l : logs_to_free) { + delete l; + } + // if new_superversion was not used, it will be non-nullptr and needs + // to be freed here + delete new_superversion; + + memtables_to_free.clear(); + superversions_to_free.clear(); + logs_to_free.clear(); + new_superversion = nullptr; + } + + ~JobContext() { + assert(memtables_to_free.size() == 0); + assert(superversions_to_free.size() == 0); + assert(new_superversion == nullptr); + assert(logs_to_free.size() == 0); + } +}; + +} // namespace rocksdb diff --git a/db/listener_test.cc b/db/listener_test.cc new file mode 100644 index 00000000000..5b5f2266b31 --- /dev/null +++ b/db/listener_test.cc @@ -0,0 +1,895 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/db_impl.h" +#include "db/db_test_util.h" +#include "db/dbformat.h" +#include "db/version_set.h" +#include "db/write_batch_internal.h" +#include "memtable/hash_linklist_rep.h" +#include "monitoring/statistics.h" +#include "rocksdb/cache.h" +#include "rocksdb/compaction_filter.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/filter_policy.h" +#include "rocksdb/options.h" +#include "rocksdb/perf_context.h" +#include "rocksdb/slice.h" +#include "rocksdb/slice_transform.h" +#include "rocksdb/table.h" +#include "rocksdb/table_properties.h" +#include "table/block_based_table_factory.h" +#include "table/plain_table_factory.h" +#include "util/filename.h" +#include "util/hash.h" +#include "util/logging.h" +#include "util/mutexlock.h" +#include "util/rate_limiter.h" +#include "util/string_util.h" +#include "util/sync_point.h" +#include "util/testharness.h" +#include "util/testutil.h" +#include "utilities/merge_operators.h" + +#ifndef ROCKSDB_LITE + +namespace rocksdb { + +class EventListenerTest : public DBTestBase { + public: + EventListenerTest() : DBTestBase("/listener_test") {} + + const size_t k110KB = 110 << 10; +}; + +struct TestPropertiesCollector : public rocksdb::TablePropertiesCollector { + virtual rocksdb::Status AddUserKey(const rocksdb::Slice& key, + const rocksdb::Slice& value, + rocksdb::EntryType type, + rocksdb::SequenceNumber seq, + uint64_t file_size) override { + return Status::OK(); + } + virtual rocksdb::Status Finish( + rocksdb::UserCollectedProperties* properties) override { + properties->insert({"0", "1"}); + return Status::OK(); + } + + virtual const char* Name() const override { + return "TestTablePropertiesCollector"; + } + + rocksdb::UserCollectedProperties GetReadableProperties() const override { + rocksdb::UserCollectedProperties ret; + ret["2"] = "3"; + return ret; + } +}; + +class TestPropertiesCollectorFactory : public TablePropertiesCollectorFactory { + public: + virtual TablePropertiesCollector* CreateTablePropertiesCollector( + TablePropertiesCollectorFactory::Context context) override { + return new TestPropertiesCollector; + } + const char* Name() const override { return "TestTablePropertiesCollector"; } +}; + +class TestCompactionListener : public EventListener { + public: + void OnCompactionCompleted(DB *db, const CompactionJobInfo& ci) override { + std::lock_guard lock(mutex_); + compacted_dbs_.push_back(db); + ASSERT_GT(ci.input_files.size(), 0U); + ASSERT_GT(ci.output_files.size(), 0U); + ASSERT_EQ(db->GetEnv()->GetThreadID(), ci.thread_id); + ASSERT_GT(ci.thread_id, 0U); + + for (auto fl : {ci.input_files, ci.output_files}) { + for (auto fn : fl) { + auto it = ci.table_properties.find(fn); + ASSERT_NE(it, ci.table_properties.end()); + auto tp = it->second; + ASSERT_TRUE(tp != nullptr); + ASSERT_EQ(tp->user_collected_properties.find("0")->second, "1"); + } + } + } + + std::vector compacted_dbs_; + std::mutex mutex_; +}; + +TEST_F(EventListenerTest, OnSingleDBCompactionTest) { + const int kTestKeySize = 16; + const int kTestValueSize = 984; + const int kEntrySize = kTestKeySize + kTestValueSize; + const int kEntriesPerBuffer = 100; + const int kNumL0Files = 4; + + Options options; + options.env = CurrentOptions().env; + options.create_if_missing = true; + options.write_buffer_size = kEntrySize * kEntriesPerBuffer; + options.compaction_style = kCompactionStyleLevel; + options.target_file_size_base = options.write_buffer_size; + options.max_bytes_for_level_base = options.target_file_size_base * 2; + options.max_bytes_for_level_multiplier = 2; + options.compression = kNoCompression; +#ifdef ROCKSDB_USING_THREAD_STATUS + options.enable_thread_tracking = true; +#endif // ROCKSDB_USING_THREAD_STATUS + options.level0_file_num_compaction_trigger = kNumL0Files; + options.table_properties_collector_factories.push_back( + std::make_shared()); + + TestCompactionListener* listener = new TestCompactionListener(); + options.listeners.emplace_back(listener); + std::vector cf_names = { + "pikachu", "ilya", "muromec", "dobrynia", + "nikitich", "alyosha", "popovich"}; + CreateAndReopenWithCF(cf_names, options); + ASSERT_OK(Put(1, "pikachu", std::string(90000, 'p'))); + ASSERT_OK(Put(2, "ilya", std::string(90000, 'i'))); + ASSERT_OK(Put(3, "muromec", std::string(90000, 'm'))); + ASSERT_OK(Put(4, "dobrynia", std::string(90000, 'd'))); + ASSERT_OK(Put(5, "nikitich", std::string(90000, 'n'))); + ASSERT_OK(Put(6, "alyosha", std::string(90000, 'a'))); + ASSERT_OK(Put(7, "popovich", std::string(90000, 'p'))); + for (int i = 1; i < 8; ++i) { + ASSERT_OK(Flush(i)); + const Slice kRangeStart = "a"; + const Slice kRangeEnd = "z"; + ASSERT_OK(dbfull()->CompactRange(CompactRangeOptions(), handles_[i], + &kRangeStart, &kRangeEnd)); + dbfull()->TEST_WaitForFlushMemTable(); + dbfull()->TEST_WaitForCompact(); + } + + ASSERT_EQ(listener->compacted_dbs_.size(), cf_names.size()); + for (size_t i = 0; i < cf_names.size(); ++i) { + ASSERT_EQ(listener->compacted_dbs_[i], db_); + } +} + +// This simple Listener can only handle one flush at a time. +class TestFlushListener : public EventListener { + public: + explicit TestFlushListener(Env* env) + : slowdown_count(0), stop_count(0), db_closed(), env_(env) { + db_closed = false; + } + void OnTableFileCreated( + const TableFileCreationInfo& info) override { + // remember the info for later checking the FlushJobInfo. + prev_fc_info_ = info; + ASSERT_GT(info.db_name.size(), 0U); + ASSERT_GT(info.cf_name.size(), 0U); + ASSERT_GT(info.file_path.size(), 0U); + ASSERT_GT(info.job_id, 0); + ASSERT_GT(info.table_properties.data_size, 0U); + ASSERT_GT(info.table_properties.raw_key_size, 0U); + ASSERT_GT(info.table_properties.raw_value_size, 0U); + ASSERT_GT(info.table_properties.num_data_blocks, 0U); + ASSERT_GT(info.table_properties.num_entries, 0U); + +#ifdef ROCKSDB_USING_THREAD_STATUS + // Verify the id of the current thread that created this table + // file matches the id of any active flush or compaction thread. + uint64_t thread_id = env_->GetThreadID(); + std::vector thread_list; + ASSERT_OK(env_->GetThreadList(&thread_list)); + bool found_match = false; + for (auto thread_status : thread_list) { + if (thread_status.operation_type == ThreadStatus::OP_FLUSH || + thread_status.operation_type == ThreadStatus::OP_COMPACTION) { + if (thread_id == thread_status.thread_id) { + found_match = true; + break; + } + } + } + ASSERT_TRUE(found_match); +#endif // ROCKSDB_USING_THREAD_STATUS + } + + void OnFlushCompleted( + DB* db, const FlushJobInfo& info) override { + flushed_dbs_.push_back(db); + flushed_column_family_names_.push_back(info.cf_name); + if (info.triggered_writes_slowdown) { + slowdown_count++; + } + if (info.triggered_writes_stop) { + stop_count++; + } + // verify whether the previously created file matches the flushed file. + ASSERT_EQ(prev_fc_info_.db_name, db->GetName()); + ASSERT_EQ(prev_fc_info_.cf_name, info.cf_name); + ASSERT_EQ(prev_fc_info_.job_id, info.job_id); + ASSERT_EQ(prev_fc_info_.file_path, info.file_path); + ASSERT_EQ(db->GetEnv()->GetThreadID(), info.thread_id); + ASSERT_GT(info.thread_id, 0U); + ASSERT_EQ(info.table_properties.user_collected_properties.find("0")->second, + "1"); + } + + std::vector flushed_column_family_names_; + std::vector flushed_dbs_; + int slowdown_count; + int stop_count; + bool db_closing; + std::atomic_bool db_closed; + TableFileCreationInfo prev_fc_info_; + + protected: + Env* env_; +}; + +TEST_F(EventListenerTest, OnSingleDBFlushTest) { + Options options; + options.env = CurrentOptions().env; + options.write_buffer_size = k110KB; +#ifdef ROCKSDB_USING_THREAD_STATUS + options.enable_thread_tracking = true; +#endif // ROCKSDB_USING_THREAD_STATUS + TestFlushListener* listener = new TestFlushListener(options.env); + options.listeners.emplace_back(listener); + std::vector cf_names = { + "pikachu", "ilya", "muromec", "dobrynia", + "nikitich", "alyosha", "popovich"}; + options.table_properties_collector_factories.push_back( + std::make_shared()); + CreateAndReopenWithCF(cf_names, options); + + ASSERT_OK(Put(1, "pikachu", std::string(90000, 'p'))); + ASSERT_OK(Put(2, "ilya", std::string(90000, 'i'))); + ASSERT_OK(Put(3, "muromec", std::string(90000, 'm'))); + ASSERT_OK(Put(4, "dobrynia", std::string(90000, 'd'))); + ASSERT_OK(Put(5, "nikitich", std::string(90000, 'n'))); + ASSERT_OK(Put(6, "alyosha", std::string(90000, 'a'))); + ASSERT_OK(Put(7, "popovich", std::string(90000, 'p'))); + for (int i = 1; i < 8; ++i) { + ASSERT_OK(Flush(i)); + dbfull()->TEST_WaitForFlushMemTable(); + ASSERT_EQ(listener->flushed_dbs_.size(), i); + ASSERT_EQ(listener->flushed_column_family_names_.size(), i); + } + + // make sure call-back functions are called in the right order + for (size_t i = 0; i < cf_names.size(); ++i) { + ASSERT_EQ(listener->flushed_dbs_[i], db_); + ASSERT_EQ(listener->flushed_column_family_names_[i], cf_names[i]); + } +} + +TEST_F(EventListenerTest, MultiCF) { + Options options; + options.env = CurrentOptions().env; + options.write_buffer_size = k110KB; +#ifdef ROCKSDB_USING_THREAD_STATUS + options.enable_thread_tracking = true; +#endif // ROCKSDB_USING_THREAD_STATUS + TestFlushListener* listener = new TestFlushListener(options.env); + options.listeners.emplace_back(listener); + options.table_properties_collector_factories.push_back( + std::make_shared()); + std::vector cf_names = { + "pikachu", "ilya", "muromec", "dobrynia", + "nikitich", "alyosha", "popovich"}; + CreateAndReopenWithCF(cf_names, options); + + ASSERT_OK(Put(1, "pikachu", std::string(90000, 'p'))); + ASSERT_OK(Put(2, "ilya", std::string(90000, 'i'))); + ASSERT_OK(Put(3, "muromec", std::string(90000, 'm'))); + ASSERT_OK(Put(4, "dobrynia", std::string(90000, 'd'))); + ASSERT_OK(Put(5, "nikitich", std::string(90000, 'n'))); + ASSERT_OK(Put(6, "alyosha", std::string(90000, 'a'))); + ASSERT_OK(Put(7, "popovich", std::string(90000, 'p'))); + for (int i = 1; i < 8; ++i) { + ASSERT_OK(Flush(i)); + ASSERT_EQ(listener->flushed_dbs_.size(), i); + ASSERT_EQ(listener->flushed_column_family_names_.size(), i); + } + + // make sure call-back functions are called in the right order + for (size_t i = 0; i < cf_names.size(); i++) { + ASSERT_EQ(listener->flushed_dbs_[i], db_); + ASSERT_EQ(listener->flushed_column_family_names_[i], cf_names[i]); + } +} + +TEST_F(EventListenerTest, MultiDBMultiListeners) { + Options options; + options.env = CurrentOptions().env; +#ifdef ROCKSDB_USING_THREAD_STATUS + options.enable_thread_tracking = true; +#endif // ROCKSDB_USING_THREAD_STATUS + options.table_properties_collector_factories.push_back( + std::make_shared()); + std::vector listeners; + const int kNumDBs = 5; + const int kNumListeners = 10; + for (int i = 0; i < kNumListeners; ++i) { + listeners.emplace_back(new TestFlushListener(options.env)); + } + + std::vector cf_names = { + "pikachu", "ilya", "muromec", "dobrynia", + "nikitich", "alyosha", "popovich"}; + + options.create_if_missing = true; + for (int i = 0; i < kNumListeners; ++i) { + options.listeners.emplace_back(listeners[i]); + } + DBOptions db_opts(options); + ColumnFamilyOptions cf_opts(options); + + std::vector dbs; + std::vector> vec_handles; + + for (int d = 0; d < kNumDBs; ++d) { + ASSERT_OK(DestroyDB(dbname_ + ToString(d), options)); + DB* db; + std::vector handles; + ASSERT_OK(DB::Open(options, dbname_ + ToString(d), &db)); + for (size_t c = 0; c < cf_names.size(); ++c) { + ColumnFamilyHandle* handle; + db->CreateColumnFamily(cf_opts, cf_names[c], &handle); + handles.push_back(handle); + } + + vec_handles.push_back(std::move(handles)); + dbs.push_back(db); + } + + for (int d = 0; d < kNumDBs; ++d) { + for (size_t c = 0; c < cf_names.size(); ++c) { + ASSERT_OK(dbs[d]->Put(WriteOptions(), vec_handles[d][c], + cf_names[c], cf_names[c])); + } + } + + for (size_t c = 0; c < cf_names.size(); ++c) { + for (int d = 0; d < kNumDBs; ++d) { + ASSERT_OK(dbs[d]->Flush(FlushOptions(), vec_handles[d][c])); + reinterpret_cast(dbs[d])->TEST_WaitForFlushMemTable(); + } + } + + for (auto* listener : listeners) { + int pos = 0; + for (size_t c = 0; c < cf_names.size(); ++c) { + for (int d = 0; d < kNumDBs; ++d) { + ASSERT_EQ(listener->flushed_dbs_[pos], dbs[d]); + ASSERT_EQ(listener->flushed_column_family_names_[pos], cf_names[c]); + pos++; + } + } + } + + + for (auto handles : vec_handles) { + for (auto h : handles) { + delete h; + } + handles.clear(); + } + vec_handles.clear(); + + for (auto db : dbs) { + delete db; + } +} + +TEST_F(EventListenerTest, DisableBGCompaction) { + Options options; + options.env = CurrentOptions().env; +#ifdef ROCKSDB_USING_THREAD_STATUS + options.enable_thread_tracking = true; +#endif // ROCKSDB_USING_THREAD_STATUS + TestFlushListener* listener = new TestFlushListener(options.env); + const int kCompactionTrigger = 1; + const int kSlowdownTrigger = 5; + const int kStopTrigger = 100; + options.level0_file_num_compaction_trigger = kCompactionTrigger; + options.level0_slowdown_writes_trigger = kSlowdownTrigger; + options.level0_stop_writes_trigger = kStopTrigger; + options.max_write_buffer_number = 10; + options.listeners.emplace_back(listener); + // BG compaction is disabled. Number of L0 files will simply keeps + // increasing in this test. + options.compaction_style = kCompactionStyleNone; + options.compression = kNoCompression; + options.write_buffer_size = 100000; // Small write buffer + options.table_properties_collector_factories.push_back( + std::make_shared()); + + CreateAndReopenWithCF({"pikachu"}, options); + ColumnFamilyMetaData cf_meta; + db_->GetColumnFamilyMetaData(handles_[1], &cf_meta); + + // keep writing until writes are forced to stop. + for (int i = 0; static_cast(cf_meta.file_count) < kSlowdownTrigger * 10; + ++i) { + Put(1, ToString(i), std::string(10000, 'x'), WriteOptions()); + db_->Flush(FlushOptions(), handles_[1]); + db_->GetColumnFamilyMetaData(handles_[1], &cf_meta); + } + ASSERT_GE(listener->slowdown_count, kSlowdownTrigger * 9); +} + +class TestCompactionReasonListener : public EventListener { + public: + void OnCompactionCompleted(DB* db, const CompactionJobInfo& ci) override { + std::lock_guard lock(mutex_); + compaction_reasons_.push_back(ci.compaction_reason); + } + + std::vector compaction_reasons_; + std::mutex mutex_; +}; + +TEST_F(EventListenerTest, CompactionReasonLevel) { + Options options; + options.env = CurrentOptions().env; + options.create_if_missing = true; + options.memtable_factory.reset( + new SpecialSkipListFactory(DBTestBase::kNumKeysByGenerateNewRandomFile)); + + TestCompactionReasonListener* listener = new TestCompactionReasonListener(); + options.listeners.emplace_back(listener); + + options.level0_file_num_compaction_trigger = 4; + options.compaction_style = kCompactionStyleLevel; + + DestroyAndReopen(options); + Random rnd(301); + + // Write 4 files in L0 + for (int i = 0; i < 4; i++) { + GenerateNewRandomFile(&rnd); + } + dbfull()->TEST_WaitForCompact(); + + ASSERT_EQ(listener->compaction_reasons_.size(), 1); + ASSERT_EQ(listener->compaction_reasons_[0], + CompactionReason::kLevelL0FilesNum); + + DestroyAndReopen(options); + + // Write 3 non-overlapping files in L0 + for (int k = 1; k <= 30; k++) { + ASSERT_OK(Put(Key(k), Key(k))); + if (k % 10 == 0) { + Flush(); + } + } + + // Do a trivial move from L0 -> L1 + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + + options.max_bytes_for_level_base = 1; + Close(); + listener->compaction_reasons_.clear(); + Reopen(options); + + dbfull()->TEST_WaitForCompact(); + ASSERT_GT(listener->compaction_reasons_.size(), 1); + + for (auto compaction_reason : listener->compaction_reasons_) { + ASSERT_EQ(compaction_reason, CompactionReason::kLevelMaxLevelSize); + } + + options.disable_auto_compactions = true; + Close(); + listener->compaction_reasons_.clear(); + Reopen(options); + + Put("key", "value"); + CompactRangeOptions cro; + cro.bottommost_level_compaction = BottommostLevelCompaction::kForce; + ASSERT_OK(db_->CompactRange(cro, nullptr, nullptr)); + ASSERT_GT(listener->compaction_reasons_.size(), 0); + for (auto compaction_reason : listener->compaction_reasons_) { + ASSERT_EQ(compaction_reason, CompactionReason::kManualCompaction); + } +} + +TEST_F(EventListenerTest, CompactionReasonUniversal) { + Options options; + options.env = CurrentOptions().env; + options.create_if_missing = true; + options.memtable_factory.reset( + new SpecialSkipListFactory(DBTestBase::kNumKeysByGenerateNewRandomFile)); + + TestCompactionReasonListener* listener = new TestCompactionReasonListener(); + options.listeners.emplace_back(listener); + + options.compaction_style = kCompactionStyleUniversal; + + Random rnd(301); + + options.level0_file_num_compaction_trigger = 8; + options.compaction_options_universal.max_size_amplification_percent = 100000; + options.compaction_options_universal.size_ratio = 100000; + DestroyAndReopen(options); + listener->compaction_reasons_.clear(); + + // Write 8 files in L0 + for (int i = 0; i < 8; i++) { + GenerateNewRandomFile(&rnd); + } + dbfull()->TEST_WaitForCompact(); + + ASSERT_GT(listener->compaction_reasons_.size(), 0); + for (auto compaction_reason : listener->compaction_reasons_) { + ASSERT_EQ(compaction_reason, CompactionReason::kUniversalSortedRunNum); + } + + options.level0_file_num_compaction_trigger = 8; + options.compaction_options_universal.max_size_amplification_percent = 1; + options.compaction_options_universal.size_ratio = 100000; + + DestroyAndReopen(options); + listener->compaction_reasons_.clear(); + + // Write 8 files in L0 + for (int i = 0; i < 8; i++) { + GenerateNewRandomFile(&rnd); + } + dbfull()->TEST_WaitForCompact(); + + ASSERT_GT(listener->compaction_reasons_.size(), 0); + for (auto compaction_reason : listener->compaction_reasons_) { + ASSERT_EQ(compaction_reason, CompactionReason::kUniversalSizeAmplification); + } + + options.disable_auto_compactions = true; + Close(); + listener->compaction_reasons_.clear(); + Reopen(options); + + db_->CompactRange(CompactRangeOptions(), nullptr, nullptr); + + ASSERT_GT(listener->compaction_reasons_.size(), 0); + for (auto compaction_reason : listener->compaction_reasons_) { + ASSERT_EQ(compaction_reason, CompactionReason::kManualCompaction); + } +} + +TEST_F(EventListenerTest, CompactionReasonFIFO) { + Options options; + options.env = CurrentOptions().env; + options.create_if_missing = true; + options.memtable_factory.reset( + new SpecialSkipListFactory(DBTestBase::kNumKeysByGenerateNewRandomFile)); + + TestCompactionReasonListener* listener = new TestCompactionReasonListener(); + options.listeners.emplace_back(listener); + + options.level0_file_num_compaction_trigger = 4; + options.compaction_style = kCompactionStyleFIFO; + options.compaction_options_fifo.max_table_files_size = 1; + + DestroyAndReopen(options); + Random rnd(301); + + // Write 4 files in L0 + for (int i = 0; i < 4; i++) { + GenerateNewRandomFile(&rnd); + } + dbfull()->TEST_WaitForCompact(); + + ASSERT_GT(listener->compaction_reasons_.size(), 0); + for (auto compaction_reason : listener->compaction_reasons_) { + ASSERT_EQ(compaction_reason, CompactionReason::kFIFOMaxSize); + } +} + +class TableFileCreationListener : public EventListener { + public: + class TestEnv : public EnvWrapper { + public: + TestEnv() : EnvWrapper(Env::Default()) {} + + void SetStatus(Status s) { status_ = s; } + + Status NewWritableFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) { + if (fname.size() > 4 && fname.substr(fname.size() - 4) == ".sst") { + if (!status_.ok()) { + return status_; + } + } + return Env::Default()->NewWritableFile(fname, result, options); + } + + private: + Status status_; + }; + + TableFileCreationListener() { + for (int i = 0; i < 2; i++) { + started_[i] = finished_[i] = failure_[i] = 0; + } + } + + int Index(TableFileCreationReason reason) { + int idx; + switch (reason) { + case TableFileCreationReason::kFlush: + idx = 0; + break; + case TableFileCreationReason::kCompaction: + idx = 1; + break; + default: + idx = -1; + } + return idx; + } + + void CheckAndResetCounters(int flush_started, int flush_finished, + int flush_failure, int compaction_started, + int compaction_finished, int compaction_failure) { + ASSERT_EQ(started_[0], flush_started); + ASSERT_EQ(finished_[0], flush_finished); + ASSERT_EQ(failure_[0], flush_failure); + ASSERT_EQ(started_[1], compaction_started); + ASSERT_EQ(finished_[1], compaction_finished); + ASSERT_EQ(failure_[1], compaction_failure); + for (int i = 0; i < 2; i++) { + started_[i] = finished_[i] = failure_[i] = 0; + } + } + + void OnTableFileCreationStarted( + const TableFileCreationBriefInfo& info) override { + int idx = Index(info.reason); + if (idx >= 0) { + started_[idx]++; + } + ASSERT_GT(info.db_name.size(), 0U); + ASSERT_GT(info.cf_name.size(), 0U); + ASSERT_GT(info.file_path.size(), 0U); + ASSERT_GT(info.job_id, 0); + } + + void OnTableFileCreated(const TableFileCreationInfo& info) override { + int idx = Index(info.reason); + if (idx >= 0) { + finished_[idx]++; + } + ASSERT_GT(info.db_name.size(), 0U); + ASSERT_GT(info.cf_name.size(), 0U); + ASSERT_GT(info.file_path.size(), 0U); + ASSERT_GT(info.job_id, 0); + if (info.status.ok()) { + ASSERT_GT(info.table_properties.data_size, 0U); + ASSERT_GT(info.table_properties.raw_key_size, 0U); + ASSERT_GT(info.table_properties.raw_value_size, 0U); + ASSERT_GT(info.table_properties.num_data_blocks, 0U); + ASSERT_GT(info.table_properties.num_entries, 0U); + } else { + if (idx >= 0) { + failure_[idx]++; + } + } + } + + TestEnv test_env; + int started_[2]; + int finished_[2]; + int failure_[2]; +}; + +TEST_F(EventListenerTest, TableFileCreationListenersTest) { + auto listener = std::make_shared(); + Options options; + options.create_if_missing = true; + options.listeners.push_back(listener); + options.env = &listener->test_env; + DestroyAndReopen(options); + + ASSERT_OK(Put("foo", "aaa")); + ASSERT_OK(Put("bar", "bbb")); + ASSERT_OK(Flush()); + dbfull()->TEST_WaitForFlushMemTable(); + listener->CheckAndResetCounters(1, 1, 0, 0, 0, 0); + + ASSERT_OK(Put("foo", "aaa1")); + ASSERT_OK(Put("bar", "bbb1")); + listener->test_env.SetStatus(Status::NotSupported("not supported")); + ASSERT_NOK(Flush()); + listener->CheckAndResetCounters(1, 1, 1, 0, 0, 0); + listener->test_env.SetStatus(Status::OK()); + + Reopen(options); + ASSERT_OK(Put("foo", "aaa2")); + ASSERT_OK(Put("bar", "bbb2")); + ASSERT_OK(Flush()); + dbfull()->TEST_WaitForFlushMemTable(); + listener->CheckAndResetCounters(1, 1, 0, 0, 0, 0); + + const Slice kRangeStart = "a"; + const Slice kRangeEnd = "z"; + dbfull()->CompactRange(CompactRangeOptions(), &kRangeStart, &kRangeEnd); + dbfull()->TEST_WaitForCompact(); + listener->CheckAndResetCounters(0, 0, 0, 1, 1, 0); + + ASSERT_OK(Put("foo", "aaa3")); + ASSERT_OK(Put("bar", "bbb3")); + ASSERT_OK(Flush()); + listener->test_env.SetStatus(Status::NotSupported("not supported")); + dbfull()->CompactRange(CompactRangeOptions(), &kRangeStart, &kRangeEnd); + dbfull()->TEST_WaitForCompact(); + listener->CheckAndResetCounters(1, 1, 0, 1, 1, 1); +} + +class MemTableSealedListener : public EventListener { +private: + SequenceNumber latest_seq_number_; +public: + MemTableSealedListener() {} + void OnMemTableSealed(const MemTableInfo& info) override { + latest_seq_number_ = info.first_seqno; + } + + void OnFlushCompleted(DB* /*db*/, + const FlushJobInfo& flush_job_info) override { + ASSERT_LE(flush_job_info.smallest_seqno, latest_seq_number_); + } +}; + +TEST_F(EventListenerTest, MemTableSealedListenerTest) { + auto listener = std::make_shared(); + Options options; + options.create_if_missing = true; + options.listeners.push_back(listener); + DestroyAndReopen(options); + + for (unsigned int i = 0; i < 10; i++) { + std::string tag = std::to_string(i); + ASSERT_OK(Put("foo"+tag, "aaa")); + ASSERT_OK(Put("bar"+tag, "bbb")); + + ASSERT_OK(Flush()); + } +} + +class ColumnFamilyHandleDeletionStartedListener : public EventListener { + private: + std::vector cfs_; + int counter; + + public: + explicit ColumnFamilyHandleDeletionStartedListener( + const std::vector& cfs) + : cfs_(cfs), counter(0) { + cfs_.insert(cfs_.begin(), kDefaultColumnFamilyName); + } + void OnColumnFamilyHandleDeletionStarted( + ColumnFamilyHandle* handle) override { + ASSERT_EQ(cfs_[handle->GetID()], handle->GetName()); + counter++; + } + int getCounter() { return counter; } +}; + +TEST_F(EventListenerTest, ColumnFamilyHandleDeletionStartedListenerTest) { + std::vector cfs{"pikachu", "eevee", "Mewtwo"}; + auto listener = + std::make_shared(cfs); + Options options; + options.env = CurrentOptions().env; + options.create_if_missing = true; + options.listeners.push_back(listener); + CreateAndReopenWithCF(cfs, options); + ASSERT_EQ(handles_.size(), 4); + delete handles_[3]; + delete handles_[2]; + delete handles_[1]; + handles_.resize(1); + ASSERT_EQ(listener->getCounter(), 3); +} + +class BackgroundErrorListener : public EventListener { + private: + SpecialEnv* env_; + int counter_; + + public: + BackgroundErrorListener(SpecialEnv* env) : env_(env), counter_(0) {} + + void OnBackgroundError(BackgroundErrorReason reason, Status* bg_error) override { + if (counter_ == 0) { + // suppress the first error and disable write-dropping such that a retry + // can succeed. + *bg_error = Status::OK(); + env_->drop_writes_.store(false, std::memory_order_release); + env_->no_slowdown_ = false; + } + ++counter_; + } + + int counter() { return counter_; } +}; + +TEST_F(EventListenerTest, BackgroundErrorListenerFailedFlushTest) { + auto listener = std::make_shared(env_); + Options options; + options.create_if_missing = true; + options.env = env_; + options.listeners.push_back(listener); + options.memtable_factory.reset(new SpecialSkipListFactory(1)); + options.paranoid_checks = true; + DestroyAndReopen(options); + + // the usual TEST_WaitForFlushMemTable() doesn't work for failed flushes, so + // forge a custom one for the failed flush case. + rocksdb::SyncPoint::GetInstance()->LoadDependency( + {{"DBImpl::BGWorkFlush:done", + "EventListenerTest:BackgroundErrorListenerFailedFlushTest:1"}}); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + env_->drop_writes_.store(true, std::memory_order_release); + env_->no_slowdown_ = true; + + ASSERT_OK(Put("key0", "val")); + ASSERT_OK(Put("key1", "val")); + TEST_SYNC_POINT("EventListenerTest:BackgroundErrorListenerFailedFlushTest:1"); + ASSERT_EQ(1, listener->counter()); + ASSERT_OK(Put("key2", "val")); + ASSERT_OK(dbfull()->TEST_WaitForFlushMemTable()); + ASSERT_EQ(1, NumTableFilesAtLevel(0)); +} + +TEST_F(EventListenerTest, BackgroundErrorListenerFailedCompactionTest) { + auto listener = std::make_shared(env_); + Options options; + options.create_if_missing = true; + options.disable_auto_compactions = true; + options.env = env_; + options.level0_file_num_compaction_trigger = 2; + options.listeners.push_back(listener); + options.memtable_factory.reset(new SpecialSkipListFactory(2)); + options.paranoid_checks = true; + DestroyAndReopen(options); + + // third iteration triggers the second memtable's flush + for (int i = 0; i < 3; ++i) { + ASSERT_OK(Put("key0", "val")); + if (i > 0) { + ASSERT_OK(dbfull()->TEST_WaitForFlushMemTable()); + } + ASSERT_OK(Put("key1", "val")); + } + ASSERT_EQ(2, NumTableFilesAtLevel(0)); + + env_->drop_writes_.store(true, std::memory_order_release); + env_->no_slowdown_ = true; + ASSERT_OK(dbfull()->SetOptions({{"disable_auto_compactions", "false"}})); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + ASSERT_EQ(1, listener->counter()); + + // trigger flush so compaction is triggered again; this time it succeeds + ASSERT_OK(Put("key0", "val")); + ASSERT_OK(dbfull()->TEST_WaitForFlushMemTable()); + ASSERT_OK(dbfull()->TEST_WaitForCompact()); + ASSERT_EQ(0, NumTableFilesAtLevel(0)); +} + +} // namespace rocksdb + +#endif // ROCKSDB_LITE + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/log_and_apply_bench.cc b/db/log_and_apply_bench.cc deleted file mode 100644 index a5aa950173f..00000000000 --- a/db/log_and_apply_bench.cc +++ /dev/null @@ -1,84 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - - -#include - -#include "util/testharness.h" -#include "util/benchharness.h" -#include "db/version_set.h" -#include "util/mutexlock.h" - -namespace rocksdb { - -std::string MakeKey(unsigned int num) { - char buf[30]; - snprintf(buf, sizeof(buf), "%016u", num); - return std::string(buf); -} - -void BM_LogAndApply(int iters, int num_base_files) { - VersionSet* vset; - ColumnFamilyData* default_cfd; - uint64_t fnum = 1; - port::Mutex mu; - MutexLock l(&mu); - - BENCHMARK_SUSPEND { - std::string dbname = test::TmpDir() + "/rocksdb_test_benchmark"; - ASSERT_OK(DestroyDB(dbname, Options())); - - DB* db = nullptr; - Options opts; - opts.create_if_missing = true; - Status s = DB::Open(opts, dbname, &db); - ASSERT_OK(s); - ASSERT_TRUE(db != nullptr); - - delete db; - db = nullptr; - - Options options; - EnvOptions sopt; - // Notice we are using the default options not through SanitizeOptions(). - // We might want to initialize some options manually if needed. - options.db_paths.emplace_back(dbname, 0); - // The parameter of table cache is passed in as null, so any file I/O - // operation is likely to fail. - vset = new VersionSet(dbname, &options, sopt, nullptr); - std::vector dummy; - dummy.push_back(ColumnFamilyDescriptor()); - ASSERT_OK(vset->Recover(dummy)); - default_cfd = vset->GetColumnFamilySet()->GetDefault(); - VersionEdit vbase; - for (int i = 0; i < num_base_files; i++) { - InternalKey start(MakeKey(2 * fnum), 1, kTypeValue); - InternalKey limit(MakeKey(2 * fnum + 1), 1, kTypeDeletion); - vbase.AddFile(2, ++fnum, 0, 1 /* file size */, start, limit, 1, 1); - } - ASSERT_OK(vset->LogAndApply(default_cfd, &vbase, &mu)); - } - - for (int i = 0; i < iters; i++) { - VersionEdit vedit; - vedit.DeleteFile(2, fnum); - InternalKey start(MakeKey(2 * fnum), 1, kTypeValue); - InternalKey limit(MakeKey(2 * fnum + 1), 1, kTypeDeletion); - vedit.AddFile(2, ++fnum, 0, 1 /* file size */, start, limit, 1, 1); - vset->LogAndApply(default_cfd, &vedit, &mu); - } -} - -BENCHMARK_NAMED_PARAM(BM_LogAndApply, 1000_iters_1_file, 1000, 1) -BENCHMARK_NAMED_PARAM(BM_LogAndApply, 1000_iters_100_files, 1000, 100) -BENCHMARK_NAMED_PARAM(BM_LogAndApply, 1000_iters_10000_files, 1000, 10000) -BENCHMARK_NAMED_PARAM(BM_LogAndApply, 100_iters_100000_files, 100, 100000) - -} // namespace rocksdb - -int main(int argc, char** argv) { - rocksdb::benchmark::RunBenchmarks(); - return 0; -} diff --git a/db/log_format.h b/db/log_format.h index 919c087e244..be22201af0a 100644 --- a/db/log_format.h +++ b/db/log_format.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -22,14 +22,24 @@ enum RecordType { // For fragments kFirstType = 2, kMiddleType = 3, - kLastType = 4 + kLastType = 4, + + // For recycled log files + kRecyclableFullType = 5, + kRecyclableFirstType = 6, + kRecyclableMiddleType = 7, + kRecyclableLastType = 8, }; -static const int kMaxRecordType = kLastType; +static const int kMaxRecordType = kRecyclableLastType; static const unsigned int kBlockSize = 32768; -// Header is checksum (4 bytes), type (1 byte), length (2 bytes). -static const int kHeaderSize = 4 + 1 + 2; +// Header is checksum (4 bytes), length (2 bytes), type (1 byte) +static const int kHeaderSize = 4 + 2 + 1; + +// Recyclable header is checksum (4 bytes), type (1 byte), log number +// (4 bytes), length (2 bytes). +static const int kRecyclableHeaderSize = 4 + 1 + 4 + 2; } // namespace log } // namespace rocksdb diff --git a/db/log_reader.cc b/db/log_reader.cc index be1fb8ceb6f..cae5d8ea087 100644 --- a/db/log_reader.cc +++ b/db/log_reader.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -13,6 +13,7 @@ #include "rocksdb/env.h" #include "util/coding.h" #include "util/crc32c.h" +#include "util/file_reader_writer.h" namespace rocksdb { namespace log { @@ -20,9 +21,11 @@ namespace log { Reader::Reporter::~Reporter() { } -Reader::Reader(unique_ptr&& file, Reporter* reporter, - bool checksum, uint64_t initial_offset) - : file_(std::move(file)), +Reader::Reader(std::shared_ptr info_log, + unique_ptr&& _file, Reporter* reporter, + bool checksum, uint64_t initial_offset, uint64_t log_num) + : info_log_(info_log), + file_(std::move(_file)), reporter_(reporter), checksum_(checksum), backing_store_(new char[kBlockSize]), @@ -32,20 +35,20 @@ Reader::Reader(unique_ptr&& file, Reporter* reporter, eof_offset_(0), last_record_offset_(0), end_of_buffer_offset_(0), - initial_offset_(initial_offset) { -} + initial_offset_(initial_offset), + log_number_(log_num), + recycled_(false) {} Reader::~Reader() { delete[] backing_store_; } bool Reader::SkipToInitialBlock() { - size_t offset_in_block = initial_offset_ % kBlockSize; - uint64_t block_start_location = initial_offset_ - offset_in_block; + size_t initial_offset_in_block = initial_offset_ % kBlockSize; + uint64_t block_start_location = initial_offset_ - initial_offset_in_block; // Don't search a block if we'd be in the trailer - if (offset_in_block > kBlockSize - 6) { - offset_in_block = 0; + if (initial_offset_in_block > kBlockSize - 6) { block_start_location += kBlockSize; } @@ -55,7 +58,7 @@ bool Reader::SkipToInitialBlock() { if (block_start_location > 0) { Status skip_status = file_->Skip(block_start_location); if (!skip_status.ok()) { - ReportDrop(block_start_location, skip_status); + ReportDrop(static_cast(block_start_location), skip_status); return false; } } @@ -63,7 +66,15 @@ bool Reader::SkipToInitialBlock() { return true; } -bool Reader::ReadRecord(Slice* record, std::string* scratch) { +// For kAbsoluteConsistency, on clean shutdown we don't expect any error +// in the log files. For other modes, we can ignore only incomplete records +// in the last log file, which are presumably due to a write in progress +// during restart (or from log recycling). +// +// TODO krad: Evaluate if we need to move to a more strict mode where we +// restrict the inconsistency to only the last log +bool Reader::ReadRecord(Slice* record, std::string* scratch, + WALRecoveryMode wal_recovery_mode) { if (last_record_offset_ < initial_offset_) { if (!SkipToInitialBlock()) { return false; @@ -80,19 +91,17 @@ bool Reader::ReadRecord(Slice* record, std::string* scratch) { Slice fragment; while (true) { uint64_t physical_record_offset = end_of_buffer_offset_ - buffer_.size(); - const unsigned int record_type = ReadPhysicalRecord(&fragment); + size_t drop_size = 0; + const unsigned int record_type = ReadPhysicalRecord(&fragment, &drop_size); switch (record_type) { case kFullType: - if (in_fragmented_record) { + case kRecyclableFullType: + if (in_fragmented_record && !scratch->empty()) { // Handle bug in earlier versions of log::Writer where // it could emit an empty kFirstType record at the tail end // of a block followed by a kFullType or kFirstType record // at the beginning of the next block. - if (scratch->empty()) { - in_fragmented_record = false; - } else { - ReportCorruption(scratch->size(), "partial record without end(1)"); - } + ReportCorruption(scratch->size(), "partial record without end(1)"); } prospective_record_offset = physical_record_offset; scratch->clear(); @@ -101,16 +110,13 @@ bool Reader::ReadRecord(Slice* record, std::string* scratch) { return true; case kFirstType: - if (in_fragmented_record) { + case kRecyclableFirstType: + if (in_fragmented_record && !scratch->empty()) { // Handle bug in earlier versions of log::Writer where // it could emit an empty kFirstType record at the tail end // of a block followed by a kFullType or kFirstType record // at the beginning of the next block. - if (scratch->empty()) { - in_fragmented_record = false; - } else { - ReportCorruption(scratch->size(), "partial record without end(2)"); - } + ReportCorruption(scratch->size(), "partial record without end(2)"); } prospective_record_offset = physical_record_offset; scratch->assign(fragment.data(), fragment.size()); @@ -118,6 +124,7 @@ bool Reader::ReadRecord(Slice* record, std::string* scratch) { break; case kMiddleType: + case kRecyclableMiddleType: if (!in_fragmented_record) { ReportCorruption(fragment.size(), "missing start of fragmented record(1)"); @@ -127,6 +134,7 @@ bool Reader::ReadRecord(Slice* record, std::string* scratch) { break; case kLastType: + case kRecyclableLastType: if (!in_fragmented_record) { ReportCorruption(fragment.size(), "missing start of fragmented record(2)"); @@ -138,8 +146,19 @@ bool Reader::ReadRecord(Slice* record, std::string* scratch) { } break; + case kBadHeader: + if (wal_recovery_mode == WALRecoveryMode::kAbsoluteConsistency) { + // in clean shutdown we don't expect any error in the log files + ReportCorruption(drop_size, "truncated header"); + } + // fall-thru + case kEof: if (in_fragmented_record) { + if (wal_recovery_mode == WALRecoveryMode::kAbsoluteConsistency) { + // in clean shutdown we don't expect any error in the log files + ReportCorruption(scratch->size(), "error reading trailing data"); + } // This can be caused by the writer dying immediately after // writing a physical record but before completing the next; don't // treat it as a corruption, just ignore the entire logical record. @@ -147,6 +166,23 @@ bool Reader::ReadRecord(Slice* record, std::string* scratch) { } return false; + case kOldRecord: + if (wal_recovery_mode != WALRecoveryMode::kSkipAnyCorruptedRecords) { + // Treat a record from a previous instance of the log as EOF. + if (in_fragmented_record) { + if (wal_recovery_mode == WALRecoveryMode::kAbsoluteConsistency) { + // in clean shutdown we don't expect any error in the log files + ReportCorruption(scratch->size(), "error reading trailing data"); + } + // This can be caused by the writer dying immediately after + // writing a physical record but before completing the next; don't + // treat it as a corruption, just ignore the entire logical record. + scratch->clear(); + } + return false; + } + // fall-thru + case kBadRecord: if (in_fragmented_record) { ReportCorruption(scratch->size(), "error in middle of record"); @@ -155,6 +191,26 @@ bool Reader::ReadRecord(Slice* record, std::string* scratch) { } break; + case kBadRecordLen: + case kBadRecordChecksum: + if (recycled_ && + wal_recovery_mode == + WALRecoveryMode::kTolerateCorruptedTailRecords) { + scratch->clear(); + return false; + } + if (record_type == kBadRecordLen) { + ReportCorruption(drop_size, "bad record length"); + } else { + ReportCorruption(drop_size, "checksum mismatch"); + } + if (in_fragmented_record) { + ReportCorruption(scratch->size(), "error in middle of record"); + in_fragmented_record = false; + scratch->clear(); + } + break; + default: { char buf[40]; snprintf(buf, sizeof(buf), "unknown record type %u", record_type); @@ -248,32 +304,49 @@ void Reader::ReportDrop(size_t bytes, const Status& reason) { } } -unsigned int Reader::ReadPhysicalRecord(Slice* result) { +bool Reader::ReadMore(size_t* drop_size, int *error) { + if (!eof_ && !read_error_) { + // Last read was a full read, so this is a trailer to skip + buffer_.clear(); + Status status = file_->Read(kBlockSize, &buffer_, backing_store_); + end_of_buffer_offset_ += buffer_.size(); + if (!status.ok()) { + buffer_.clear(); + ReportDrop(kBlockSize, status); + read_error_ = true; + *error = kEof; + return false; + } else if (buffer_.size() < (size_t)kBlockSize) { + eof_ = true; + eof_offset_ = buffer_.size(); + } + return true; + } else { + // Note that if buffer_ is non-empty, we have a truncated header at the + // end of the file, which can be caused by the writer crashing in the + // middle of writing the header. Unless explicitly requested we don't + // considering this an error, just report EOF. + if (buffer_.size()) { + *drop_size = buffer_.size(); + buffer_.clear(); + *error = kBadHeader; + return false; + } + buffer_.clear(); + *error = kEof; + return false; + } +} + +unsigned int Reader::ReadPhysicalRecord(Slice* result, size_t* drop_size) { while (true) { + // We need at least the minimum header size if (buffer_.size() < (size_t)kHeaderSize) { - if (!eof_ && !read_error_) { - // Last read was a full read, so this is a trailer to skip - buffer_.clear(); - Status status = file_->Read(kBlockSize, &buffer_, backing_store_); - end_of_buffer_offset_ += buffer_.size(); - if (!status.ok()) { - buffer_.clear(); - ReportDrop(kBlockSize, status); - read_error_ = true; - return kEof; - } else if (buffer_.size() < (size_t)kBlockSize) { - eof_ = true; - eof_offset_ = buffer_.size(); - } - continue; - } else { - // Note that if buffer_ is non-empty, we have a truncated header at the - // end of the file, which can be caused by the writer crashing in the - // middle of writing the header. Instead of considering this an error, - // just report EOF. - buffer_.clear(); - return kEof; + int r; + if (!ReadMore(drop_size, &r)) { + return r; } + continue; } // Parse the header @@ -282,16 +355,37 @@ unsigned int Reader::ReadPhysicalRecord(Slice* result) { const uint32_t b = static_cast(header[5]) & 0xff; const unsigned int type = header[6]; const uint32_t length = a | (b << 8); - if (kHeaderSize + length > buffer_.size()) { - size_t drop_size = buffer_.size(); + int header_size = kHeaderSize; + if (type >= kRecyclableFullType && type <= kRecyclableLastType) { + if (end_of_buffer_offset_ - buffer_.size() == 0) { + recycled_ = true; + } + header_size = kRecyclableHeaderSize; + // We need enough for the larger header + if (buffer_.size() < (size_t)kRecyclableHeaderSize) { + int r; + if (!ReadMore(drop_size, &r)) { + return r; + } + continue; + } + const uint32_t log_num = DecodeFixed32(header + 7); + if (log_num != log_number_) { + return kOldRecord; + } + } + if (header_size + length > buffer_.size()) { + *drop_size = buffer_.size(); buffer_.clear(); if (!eof_) { - ReportCorruption(drop_size, "bad record length"); - return kBadRecord; + return kBadRecordLen; } // If the end of the file has been reached without reading |length| bytes // of payload, assume the writer died in the middle of writing the record. - // Don't report a corruption. + // Don't report a corruption unless requested. + if (*drop_size) { + return kBadHeader; + } return kEof; } @@ -308,29 +402,28 @@ unsigned int Reader::ReadPhysicalRecord(Slice* result) { // Check crc if (checksum_) { uint32_t expected_crc = crc32c::Unmask(DecodeFixed32(header)); - uint32_t actual_crc = crc32c::Value(header + 6, 1 + length); + uint32_t actual_crc = crc32c::Value(header + 6, length + header_size - 6); if (actual_crc != expected_crc) { // Drop the rest of the buffer since "length" itself may have // been corrupted and if we trust it, we could find some // fragment of a real log record that just happens to look // like a valid log record. - size_t drop_size = buffer_.size(); + *drop_size = buffer_.size(); buffer_.clear(); - ReportCorruption(drop_size, "checksum mismatch"); - return kBadRecord; + return kBadRecordChecksum; } } - buffer_.remove_prefix(kHeaderSize + length); + buffer_.remove_prefix(header_size + length); // Skip physical record that started before initial_offset_ - if (end_of_buffer_offset_ - buffer_.size() - kHeaderSize - length < + if (end_of_buffer_offset_ - buffer_.size() - header_size - length < initial_offset_) { result->clear(); return kBadRecord; } - *result = Slice(header + kHeaderSize, length); + *result = Slice(header + header_size, length); return type; } } diff --git a/db/log_reader.h b/db/log_reader.h index 81d334da295..c6a471cda44 100644 --- a/db/log_reader.h +++ b/db/log_reader.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -14,14 +14,22 @@ #include "db/log_format.h" #include "rocksdb/slice.h" #include "rocksdb/status.h" +#include "rocksdb/options.h" namespace rocksdb { -class SequentialFile; +class SequentialFileReader; +class Logger; using std::unique_ptr; namespace log { +/** + * Reader is a general purpose log stream reader implementation. The actual job + * of reading from the device is implemented by the SequentialFile interface. + * + * Please see Writer for details on the file and record layout. + */ class Reader { public: // Interface for reporting errors. @@ -45,8 +53,10 @@ class Reader { // // The Reader will start reading at the first record located at physical // position >= initial_offset within the file. - Reader(unique_ptr&& file, Reporter* reporter, - bool checksum, uint64_t initial_offset); + Reader(std::shared_ptr info_log, + unique_ptr&& file, + Reporter* reporter, bool checksum, uint64_t initial_offset, + uint64_t log_num); ~Reader(); @@ -55,7 +65,9 @@ class Reader { // "*scratch" as temporary storage. The contents filled in *record // will only be valid until the next mutating operation on this // reader or the next mutation to *scratch. - bool ReadRecord(Slice* record, std::string* scratch); + bool ReadRecord(Slice* record, std::string* scratch, + WALRecoveryMode wal_recovery_mode = + WALRecoveryMode::kTolerateCorruptedTailRecords); // Returns the physical offset of the last record returned by ReadRecord. // @@ -74,10 +86,11 @@ class Reader { // block that was partially read. void UnmarkEOF(); - SequentialFile* file() { return file_.get(); } + SequentialFileReader* file() { return file_.get(); } private: - const unique_ptr file_; + std::shared_ptr info_log_; + const unique_ptr file_; Reporter* const reporter_; bool const checksum_; char* const backing_store_; @@ -97,6 +110,12 @@ class Reader { // Offset at which to start looking for the first record to return uint64_t const initial_offset_; + // which log number this is + uint64_t const log_number_; + + // Whether this is a recycled log file + bool recycled_; + // Extend record types with the following special values enum { kEof = kMaxRecordType + 1, @@ -105,7 +124,15 @@ class Reader { // * The record has an invalid CRC (ReadPhysicalRecord reports a drop) // * The record is a 0-length record (No drop is reported) // * The record is below constructor's initial_offset (No drop is reported) - kBadRecord = kMaxRecordType + 2 + kBadRecord = kMaxRecordType + 2, + // Returned when we fail to read a valid header. + kBadHeader = kMaxRecordType + 3, + // Returned when we read an old record from a previous user of the log. + kOldRecord = kMaxRecordType + 4, + // Returned when we get a bad record length + kBadRecordLen = kMaxRecordType + 5, + // Returned when we get a bad record checksum + kBadRecordChecksum = kMaxRecordType + 6, }; // Skips all blocks that are completely before "initial_offset_". @@ -114,7 +141,10 @@ class Reader { bool SkipToInitialBlock(); // Return type, or one of the preceding special values - unsigned int ReadPhysicalRecord(Slice* result); + unsigned int ReadPhysicalRecord(Slice* result, size_t* drop_size); + + // Read some more + bool ReadMore(size_t* drop_size, int *error); // Reports dropped bytes to the reporter. // buffer_ must be updated to remove the dropped bytes prior to invocation. diff --git a/db/log_test.cc b/db/log_test.cc index 6577a6a9cb4..651a1d0eeee 100644 --- a/db/log_test.cc +++ b/db/log_test.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -12,8 +12,10 @@ #include "rocksdb/env.h" #include "util/coding.h" #include "util/crc32c.h" +#include "util/file_reader_writer.h" #include "util/random.h" #include "util/testharness.h" +#include "util/testutil.h" namespace rocksdb { namespace log { @@ -41,48 +43,8 @@ static std::string RandomSkewedString(int i, Random* rnd) { return BigString(NumberString(i), rnd->Skewed(17)); } -class LogTest { +class LogTest : public ::testing::TestWithParam { private: - class StringDest : public WritableFile { - public: - std::string contents_; - - explicit StringDest(Slice& reader_contents) : - WritableFile(), - contents_(""), - reader_contents_(reader_contents), - last_flush_(0) { - reader_contents_ = Slice(contents_.data(), 0); - }; - - virtual Status Close() { return Status::OK(); } - virtual Status Flush() { - ASSERT_TRUE(reader_contents_.size() <= last_flush_); - size_t offset = last_flush_ - reader_contents_.size(); - reader_contents_ = Slice( - contents_.data() + offset, - contents_.size() - offset); - last_flush_ = contents_.size(); - - return Status::OK(); - } - virtual Status Sync() { return Status::OK(); } - virtual Status Append(const Slice& slice) { - contents_.append(slice.data(), slice.size()); - return Status::OK(); - } - void Drop(size_t bytes) { - contents_.resize(contents_.size() - bytes); - reader_contents_ = Slice( - reader_contents_.data(), reader_contents_.size() - bytes); - last_flush_ = contents_.size(); - } - - private: - Slice& reader_contents_; - size_t last_flush_; - }; - class StringSource : public SequentialFile { public: Slice& contents_; @@ -99,8 +61,8 @@ class LogTest { force_eof_position_(0), returned_partial_(false) { } - virtual Status Read(size_t n, Slice* result, char* scratch) { - ASSERT_TRUE(!returned_partial_) << "must not Read() after eof/error"; + virtual Status Read(size_t n, Slice* result, char* scratch) override { + EXPECT_TRUE(!returned_partial_) << "must not Read() after eof/error"; if (force_error_) { if (force_error_position_ >= n) { @@ -138,7 +100,7 @@ class LogTest { return Status::OK(); } - virtual Status Skip(uint64_t n) { + virtual Status Skip(uint64_t n) override { if (n > contents_.size()) { contents_.clear(); return Status::NotFound("in-memory file skipepd past end"); @@ -156,50 +118,64 @@ class LogTest { std::string message_; ReportCollector() : dropped_bytes_(0) { } - virtual void Corruption(size_t bytes, const Status& status) { + virtual void Corruption(size_t bytes, const Status& status) override { dropped_bytes_ += bytes; message_.append(status.ToString()); } }; std::string& dest_contents() { - auto dest = dynamic_cast(writer_.file()); + auto dest = + dynamic_cast(writer_.file()->writable_file()); assert(dest); return dest->contents_; } const std::string& dest_contents() const { - auto dest = dynamic_cast(writer_.file()); + auto dest = + dynamic_cast(writer_.file()->writable_file()); assert(dest); return dest->contents_; } void reset_source_contents() { - auto src = dynamic_cast(reader_.file()); + auto src = dynamic_cast(reader_.file()->file()); assert(src); src->contents_ = dest_contents(); } Slice reader_contents_; - unique_ptr dest_holder_; - unique_ptr source_holder_; + unique_ptr dest_holder_; + unique_ptr source_holder_; ReportCollector report_; Writer writer_; Reader reader_; // Record metadata for testing initial offset functionality static size_t initial_offset_record_sizes_[]; - static uint64_t initial_offset_last_record_offsets_[]; + uint64_t initial_offset_last_record_offsets_[4]; public: - LogTest() : reader_contents_(), - dest_holder_(new StringDest(reader_contents_)), - source_holder_(new StringSource(reader_contents_)), - writer_(std::move(dest_holder_)), - reader_(std::move(source_holder_), &report_, true/*checksum*/, - 0/*initial_offset*/) { + LogTest() + : reader_contents_(), + dest_holder_(test::GetWritableFileWriter( + new test::StringSink(&reader_contents_))), + source_holder_( + test::GetSequentialFileReader(new StringSource(reader_contents_))), + writer_(std::move(dest_holder_), 123, GetParam()), + reader_(NULL, std::move(source_holder_), &report_, true /*checksum*/, + 0 /*initial_offset*/, 123) { + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + initial_offset_last_record_offsets_[0] = 0; + initial_offset_last_record_offsets_[1] = header_size + 10000; + initial_offset_last_record_offsets_[2] = 2 * (header_size + 10000); + initial_offset_last_record_offsets_[3] = 2 * (header_size + 10000) + + (2 * log::kBlockSize - 1000) + + 3 * header_size; } + Slice* get_reader_contents() { return &reader_contents_; } + void Write(const std::string& msg) { writer_.AddRecord(Slice(msg)); } @@ -208,10 +184,11 @@ class LogTest { return dest_contents().size(); } - std::string Read() { + std::string Read(const WALRecoveryMode wal_recovery_mode = + WALRecoveryMode::kTolerateCorruptedTailRecords) { std::string scratch; Slice record; - if (reader_.ReadRecord(&record, &scratch)) { + if (reader_.ReadRecord(&record, &scratch, wal_recovery_mode)) { return record.ToString(); } else { return "EOF"; @@ -227,20 +204,23 @@ class LogTest { } void ShrinkSize(int bytes) { - auto dest = dynamic_cast(writer_.file()); + auto dest = + dynamic_cast(writer_.file()->writable_file()); assert(dest); dest->Drop(bytes); } - void FixChecksum(int header_offset, int len) { + void FixChecksum(int header_offset, int len, bool recyclable) { // Compute crc of type/len/data - uint32_t crc = crc32c::Value(&dest_contents()[header_offset+6], 1 + len); + int header_size = recyclable ? kRecyclableHeaderSize : kHeaderSize; + uint32_t crc = crc32c::Value(&dest_contents()[header_offset + 6], + header_size - 6 + len); crc = crc32c::Mask(crc); EncodeFixed32(&dest_contents()[header_offset], crc); } void ForceError(size_t position = 0) { - auto src = dynamic_cast(reader_.file()); + auto src = dynamic_cast(reader_.file()->file()); src->force_error_ = true; src->force_error_position_ = position; } @@ -254,13 +234,13 @@ class LogTest { } void ForceEOF(size_t position = 0) { - auto src = dynamic_cast(reader_.file()); + auto src = dynamic_cast(reader_.file()->file()); src->force_eof_ = true; src->force_eof_position_ = position; } void UnmarkEOF() { - auto src = dynamic_cast(reader_.file()); + auto src = dynamic_cast(reader_.file()->file()); src->returned_partial_ = false; reader_.UnmarkEOF(); } @@ -288,10 +268,11 @@ class LogTest { void CheckOffsetPastEndReturnsNoRecords(uint64_t offset_past_end) { WriteInitialOffsetLog(); - unique_ptr source(new StringSource(reader_contents_)); + unique_ptr file_reader( + test::GetSequentialFileReader(new StringSource(reader_contents_))); unique_ptr offset_reader( - new Reader(std::move(source), &report_, true/*checksum*/, - WrittenBytes() + offset_past_end)); + new Reader(NULL, std::move(file_reader), &report_, + true /*checksum*/, WrittenBytes() + offset_past_end, 123)); Slice record; std::string scratch; ASSERT_TRUE(!offset_reader->ReadRecord(&record, &scratch)); @@ -300,10 +281,11 @@ class LogTest { void CheckInitialOffsetRecord(uint64_t initial_offset, int expected_record_offset) { WriteInitialOffsetLog(); - unique_ptr source(new StringSource(reader_contents_)); + unique_ptr file_reader( + test::GetSequentialFileReader(new StringSource(reader_contents_))); unique_ptr offset_reader( - new Reader(std::move(source), &report_, true/*checksum*/, - initial_offset)); + new Reader(NULL, std::move(file_reader), &report_, + true /*checksum*/, initial_offset, 123)); Slice record; std::string scratch; ASSERT_TRUE(offset_reader->ReadRecord(&record, &scratch)); @@ -322,19 +304,9 @@ size_t LogTest::initial_offset_record_sizes_[] = 2 * log::kBlockSize - 1000, // Span three blocks 1}; -uint64_t LogTest::initial_offset_last_record_offsets_[] = - {0, - kHeaderSize + 10000, - 2 * (kHeaderSize + 10000), - 2 * (kHeaderSize + 10000) + - (2 * log::kBlockSize - 1000) + 3 * kHeaderSize}; - - -TEST(LogTest, Empty) { - ASSERT_EQ("EOF", Read()); -} +TEST_P(LogTest, Empty) { ASSERT_EQ("EOF", Read()); } -TEST(LogTest, ReadWrite) { +TEST_P(LogTest, ReadWrite) { Write("foo"); Write("bar"); Write(""); @@ -347,7 +319,7 @@ TEST(LogTest, ReadWrite) { ASSERT_EQ("EOF", Read()); // Make sure reads at eof work } -TEST(LogTest, ManyBlocks) { +TEST_P(LogTest, ManyBlocks) { for (int i = 0; i < 100000; i++) { Write(NumberString(i)); } @@ -357,7 +329,7 @@ TEST(LogTest, ManyBlocks) { ASSERT_EQ("EOF", Read()); } -TEST(LogTest, Fragmentation) { +TEST_P(LogTest, Fragmentation) { Write("small"); Write(BigString("medium", 50000)); Write(BigString("large", 100000)); @@ -367,11 +339,12 @@ TEST(LogTest, Fragmentation) { ASSERT_EQ("EOF", Read()); } -TEST(LogTest, MarginalTrailer) { +TEST_P(LogTest, MarginalTrailer) { // Make a trailer that is exactly the same length as an empty record. - const int n = kBlockSize - 2*kHeaderSize; + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + const int n = kBlockSize - 2 * header_size; Write(BigString("foo", n)); - ASSERT_EQ((unsigned int)(kBlockSize - kHeaderSize), WrittenBytes()); + ASSERT_EQ((unsigned int)(kBlockSize - header_size), WrittenBytes()); Write(""); Write("bar"); ASSERT_EQ(BigString("foo", n), Read()); @@ -380,11 +353,12 @@ TEST(LogTest, MarginalTrailer) { ASSERT_EQ("EOF", Read()); } -TEST(LogTest, MarginalTrailer2) { +TEST_P(LogTest, MarginalTrailer2) { // Make a trailer that is exactly the same length as an empty record. - const int n = kBlockSize - 2*kHeaderSize; + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + const int n = kBlockSize - 2 * header_size; Write(BigString("foo", n)); - ASSERT_EQ((unsigned int)(kBlockSize - kHeaderSize), WrittenBytes()); + ASSERT_EQ((unsigned int)(kBlockSize - header_size), WrittenBytes()); Write("bar"); ASSERT_EQ(BigString("foo", n), Read()); ASSERT_EQ("bar", Read()); @@ -393,10 +367,11 @@ TEST(LogTest, MarginalTrailer2) { ASSERT_EQ("", ReportMessage()); } -TEST(LogTest, ShortTrailer) { - const int n = kBlockSize - 2*kHeaderSize + 4; +TEST_P(LogTest, ShortTrailer) { + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + const int n = kBlockSize - 2 * header_size + 4; Write(BigString("foo", n)); - ASSERT_EQ((unsigned int)(kBlockSize - kHeaderSize + 4), WrittenBytes()); + ASSERT_EQ((unsigned int)(kBlockSize - header_size + 4), WrittenBytes()); Write(""); Write("bar"); ASSERT_EQ(BigString("foo", n), Read()); @@ -405,15 +380,16 @@ TEST(LogTest, ShortTrailer) { ASSERT_EQ("EOF", Read()); } -TEST(LogTest, AlignedEof) { - const int n = kBlockSize - 2*kHeaderSize + 4; +TEST_P(LogTest, AlignedEof) { + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + const int n = kBlockSize - 2 * header_size + 4; Write(BigString("foo", n)); - ASSERT_EQ((unsigned int)(kBlockSize - kHeaderSize + 4), WrittenBytes()); + ASSERT_EQ((unsigned int)(kBlockSize - header_size + 4), WrittenBytes()); ASSERT_EQ(BigString("foo", n), Read()); ASSERT_EQ("EOF", Read()); } -TEST(LogTest, RandomRead) { +TEST_P(LogTest, RandomRead) { const int N = 500; Random write_rnd(301); for (int i = 0; i < N; i++) { @@ -428,7 +404,7 @@ TEST(LogTest, RandomRead) { // Tests of all the error paths in log_reader.cc follow: -TEST(LogTest, ReadError) { +TEST_P(LogTest, ReadError) { Write("foo"); ForceError(); ASSERT_EQ("EOF", Read()); @@ -436,17 +412,17 @@ TEST(LogTest, ReadError) { ASSERT_EQ("OK", MatchError("read error")); } -TEST(LogTest, BadRecordType) { +TEST_P(LogTest, BadRecordType) { Write("foo"); // Type is stored in header[6] IncrementByte(6, 100); - FixChecksum(0, 3); + FixChecksum(0, 3, false); ASSERT_EQ("EOF", Read()); ASSERT_EQ(3U, DroppedBytes()); ASSERT_EQ("OK", MatchError("unknown record type")); } -TEST(LogTest, TruncatedTrailingRecordIsIgnored) { +TEST_P(LogTest, TruncatedTrailingRecordIsIgnored) { Write("foo"); ShrinkSize(4); // Drop all payload as well as a header byte ASSERT_EQ("EOF", Read()); @@ -455,18 +431,32 @@ TEST(LogTest, TruncatedTrailingRecordIsIgnored) { ASSERT_EQ("", ReportMessage()); } -TEST(LogTest, BadLength) { - const int kPayloadSize = kBlockSize - kHeaderSize; +TEST_P(LogTest, TruncatedTrailingRecordIsNotIgnored) { + Write("foo"); + ShrinkSize(4); // Drop all payload as well as a header byte + ASSERT_EQ("EOF", Read(WALRecoveryMode::kAbsoluteConsistency)); + // Truncated last record is ignored, not treated as an error + ASSERT_GT(DroppedBytes(), 0U); + ASSERT_EQ("OK", MatchError("Corruption: truncated header")); +} + +TEST_P(LogTest, BadLength) { + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + const int kPayloadSize = kBlockSize - header_size; Write(BigString("bar", kPayloadSize)); Write("foo"); // Least significant size byte is stored in header[4]. IncrementByte(4, 1); - ASSERT_EQ("foo", Read()); - ASSERT_EQ(kBlockSize, DroppedBytes()); - ASSERT_EQ("OK", MatchError("bad record length")); + if (!GetParam()) { + ASSERT_EQ("foo", Read()); + ASSERT_EQ(kBlockSize, DroppedBytes()); + ASSERT_EQ("OK", MatchError("bad record length")); + } else { + ASSERT_EQ("EOF", Read()); + } } -TEST(LogTest, BadLengthAtEndIsIgnored) { +TEST_P(LogTest, BadLengthAtEndIsIgnored) { Write("foo"); ShrinkSize(1); ASSERT_EQ("EOF", Read()); @@ -474,55 +464,68 @@ TEST(LogTest, BadLengthAtEndIsIgnored) { ASSERT_EQ("", ReportMessage()); } -TEST(LogTest, ChecksumMismatch) { +TEST_P(LogTest, BadLengthAtEndIsNotIgnored) { Write("foo"); - IncrementByte(0, 10); + ShrinkSize(1); + ASSERT_EQ("EOF", Read(WALRecoveryMode::kAbsoluteConsistency)); + ASSERT_GT(DroppedBytes(), 0U); + ASSERT_EQ("OK", MatchError("Corruption: truncated header")); +} + +TEST_P(LogTest, ChecksumMismatch) { + Write("foooooo"); + IncrementByte(0, 14); ASSERT_EQ("EOF", Read()); - ASSERT_EQ(10U, DroppedBytes()); - ASSERT_EQ("OK", MatchError("checksum mismatch")); + if (!GetParam()) { + ASSERT_EQ(14U, DroppedBytes()); + ASSERT_EQ("OK", MatchError("checksum mismatch")); + } else { + ASSERT_EQ(0U, DroppedBytes()); + ASSERT_EQ("", ReportMessage()); + } } -TEST(LogTest, UnexpectedMiddleType) { +TEST_P(LogTest, UnexpectedMiddleType) { Write("foo"); - SetByte(6, kMiddleType); - FixChecksum(0, 3); + SetByte(6, GetParam() ? kRecyclableMiddleType : kMiddleType); + FixChecksum(0, 3, !!GetParam()); ASSERT_EQ("EOF", Read()); ASSERT_EQ(3U, DroppedBytes()); ASSERT_EQ("OK", MatchError("missing start")); } -TEST(LogTest, UnexpectedLastType) { +TEST_P(LogTest, UnexpectedLastType) { Write("foo"); - SetByte(6, kLastType); - FixChecksum(0, 3); + SetByte(6, GetParam() ? kRecyclableLastType : kLastType); + FixChecksum(0, 3, !!GetParam()); ASSERT_EQ("EOF", Read()); ASSERT_EQ(3U, DroppedBytes()); ASSERT_EQ("OK", MatchError("missing start")); } -TEST(LogTest, UnexpectedFullType) { +TEST_P(LogTest, UnexpectedFullType) { Write("foo"); Write("bar"); - SetByte(6, kFirstType); - FixChecksum(0, 3); + SetByte(6, GetParam() ? kRecyclableFirstType : kFirstType); + FixChecksum(0, 3, !!GetParam()); ASSERT_EQ("bar", Read()); ASSERT_EQ("EOF", Read()); ASSERT_EQ(3U, DroppedBytes()); ASSERT_EQ("OK", MatchError("partial record without end")); } -TEST(LogTest, UnexpectedFirstType) { +TEST_P(LogTest, UnexpectedFirstType) { Write("foo"); Write(BigString("bar", 100000)); - SetByte(6, kFirstType); - FixChecksum(0, 3); + SetByte(6, GetParam() ? kRecyclableFirstType : kFirstType); + FixChecksum(0, 3, !!GetParam()); ASSERT_EQ(BigString("bar", 100000), Read()); ASSERT_EQ("EOF", Read()); ASSERT_EQ(3U, DroppedBytes()); ASSERT_EQ("OK", MatchError("partial record without end")); } -TEST(LogTest, MissingLastIsIgnored) { +TEST_P(LogTest, MissingLastIsIgnored) { Write(BigString("bar", kBlockSize)); // Remove the LAST block, including header. ShrinkSize(14); @@ -531,7 +534,16 @@ TEST(LogTest, MissingLastIsIgnored) { ASSERT_EQ(0U, DroppedBytes()); } -TEST(LogTest, PartialLastIsIgnored) { +TEST_P(LogTest, MissingLastIsNotIgnored) { + Write(BigString("bar", kBlockSize)); + // Remove the LAST block, including header. + ShrinkSize(14); + ASSERT_EQ("EOF", Read(WALRecoveryMode::kAbsoluteConsistency)); + ASSERT_GT(DroppedBytes(), 0U); + ASSERT_EQ("OK", MatchError("Corruption: error reading trailing data")); +} + +TEST_P(LogTest, PartialLastIsIgnored) { Write(BigString("bar", kBlockSize)); // Cause a bad record length in the LAST block. ShrinkSize(1); @@ -540,7 +552,18 @@ TEST(LogTest, PartialLastIsIgnored) { ASSERT_EQ(0U, DroppedBytes()); } -TEST(LogTest, ErrorJoinsRecords) { +TEST_P(LogTest, PartialLastIsNotIgnored) { + Write(BigString("bar", kBlockSize)); + // Cause a bad record length in the LAST block. + ShrinkSize(1); + ASSERT_EQ("EOF", Read(WALRecoveryMode::kAbsoluteConsistency)); + ASSERT_GT(DroppedBytes(), 0U); + ASSERT_EQ("OK", MatchError( + "Corruption: truncated headerCorruption: " + "error reading trailing data")); +} + +TEST_P(LogTest, ErrorJoinsRecords) { // Consider two fragmented records: // first(R1) last(R1) first(R2) last(R2) // where the middle two fragments disappear. We do not want @@ -556,71 +579,71 @@ TEST(LogTest, ErrorJoinsRecords) { SetByte(offset, 'x'); } - ASSERT_EQ("correct", Read()); - ASSERT_EQ("EOF", Read()); - const unsigned int dropped = DroppedBytes(); - ASSERT_LE(dropped, 2*kBlockSize + 100); - ASSERT_GE(dropped, 2*kBlockSize); + if (!GetParam()) { + ASSERT_EQ("correct", Read()); + ASSERT_EQ("EOF", Read()); + size_t dropped = DroppedBytes(); + ASSERT_LE(dropped, 2 * kBlockSize + 100); + ASSERT_GE(dropped, 2 * kBlockSize); + } else { + ASSERT_EQ("EOF", Read()); + } } -TEST(LogTest, ReadStart) { - CheckInitialOffsetRecord(0, 0); -} +TEST_P(LogTest, ReadStart) { CheckInitialOffsetRecord(0, 0); } -TEST(LogTest, ReadSecondOneOff) { - CheckInitialOffsetRecord(1, 1); -} +TEST_P(LogTest, ReadSecondOneOff) { CheckInitialOffsetRecord(1, 1); } -TEST(LogTest, ReadSecondTenThousand) { - CheckInitialOffsetRecord(10000, 1); -} +TEST_P(LogTest, ReadSecondTenThousand) { CheckInitialOffsetRecord(10000, 1); } -TEST(LogTest, ReadSecondStart) { - CheckInitialOffsetRecord(10007, 1); +TEST_P(LogTest, ReadSecondStart) { + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + CheckInitialOffsetRecord(10000 + header_size, 1); } -TEST(LogTest, ReadThirdOneOff) { - CheckInitialOffsetRecord(10008, 2); +TEST_P(LogTest, ReadThirdOneOff) { + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + CheckInitialOffsetRecord(10000 + header_size + 1, 2); } -TEST(LogTest, ReadThirdStart) { - CheckInitialOffsetRecord(20014, 2); +TEST_P(LogTest, ReadThirdStart) { + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + CheckInitialOffsetRecord(20000 + 2 * header_size, 2); } -TEST(LogTest, ReadFourthOneOff) { - CheckInitialOffsetRecord(20015, 3); +TEST_P(LogTest, ReadFourthOneOff) { + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + CheckInitialOffsetRecord(20000 + 2 * header_size + 1, 3); } -TEST(LogTest, ReadFourthFirstBlockTrailer) { +TEST_P(LogTest, ReadFourthFirstBlockTrailer) { CheckInitialOffsetRecord(log::kBlockSize - 4, 3); } -TEST(LogTest, ReadFourthMiddleBlock) { +TEST_P(LogTest, ReadFourthMiddleBlock) { CheckInitialOffsetRecord(log::kBlockSize + 1, 3); } -TEST(LogTest, ReadFourthLastBlock) { +TEST_P(LogTest, ReadFourthLastBlock) { CheckInitialOffsetRecord(2 * log::kBlockSize + 1, 3); } -TEST(LogTest, ReadFourthStart) { +TEST_P(LogTest, ReadFourthStart) { + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; CheckInitialOffsetRecord( - 2 * (kHeaderSize + 1000) + (2 * log::kBlockSize - 1000) + 3 * kHeaderSize, + 2 * (header_size + 1000) + (2 * log::kBlockSize - 1000) + 3 * header_size, 3); } -TEST(LogTest, ReadEnd) { - CheckOffsetPastEndReturnsNoRecords(0); -} +TEST_P(LogTest, ReadEnd) { CheckOffsetPastEndReturnsNoRecords(0); } -TEST(LogTest, ReadPastEnd) { - CheckOffsetPastEndReturnsNoRecords(5); -} +TEST_P(LogTest, ReadPastEnd) { CheckOffsetPastEndReturnsNoRecords(5); } -TEST(LogTest, ClearEofSingleBlock) { +TEST_P(LogTest, ClearEofSingleBlock) { Write("foo"); Write("bar"); - ForceEOF(3 + kHeaderSize + 2); + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + ForceEOF(3 + header_size + 2); ASSERT_EQ("foo", Read()); UnmarkEOF(); ASSERT_EQ("bar", Read()); @@ -632,12 +655,13 @@ TEST(LogTest, ClearEofSingleBlock) { ASSERT_TRUE(IsEOF()); } -TEST(LogTest, ClearEofMultiBlock) { +TEST_P(LogTest, ClearEofMultiBlock) { size_t num_full_blocks = 5; - size_t n = (kBlockSize - kHeaderSize) * num_full_blocks + 25; + int header_size = GetParam() ? kRecyclableHeaderSize : kHeaderSize; + size_t n = (kBlockSize - header_size) * num_full_blocks + 25; Write(BigString("foo", n)); Write(BigString("bar", n)); - ForceEOF(n + num_full_blocks * kHeaderSize + 10); + ForceEOF(n + num_full_blocks * header_size + header_size + 3); ASSERT_EQ(BigString("foo", n), Read()); ASSERT_TRUE(IsEOF()); UnmarkEOF(); @@ -649,7 +673,7 @@ TEST(LogTest, ClearEofMultiBlock) { ASSERT_TRUE(IsEOF()); } -TEST(LogTest, ClearEofError) { +TEST_P(LogTest, ClearEofError) { // If an error occurs during Read() in UnmarkEOF(), the records contained // in the buffer should be returned on subsequent calls of ReadRecord() // until no more full records are left, whereafter ReadRecord() should return @@ -667,7 +691,7 @@ TEST(LogTest, ClearEofError) { ASSERT_EQ("EOF", Read()); } -TEST(LogTest, ClearEofError2) { +TEST_P(LogTest, ClearEofError2) { Write("foo"); Write("bar"); UnmarkEOF(); @@ -681,9 +705,35 @@ TEST(LogTest, ClearEofError2) { ASSERT_EQ("OK", MatchError("read error")); } +TEST_P(LogTest, Recycle) { + if (!GetParam()) { + return; // test is only valid for recycled logs + } + Write("foo"); + Write("bar"); + Write("baz"); + Write("bif"); + Write("blitz"); + while (get_reader_contents()->size() < log::kBlockSize * 2) { + Write("xxxxxxxxxxxxxxxx"); + } + unique_ptr dest_holder(test::GetWritableFileWriter( + new test::OverwritingStringSink(get_reader_contents()))); + Writer recycle_writer(std::move(dest_holder), 123, true); + recycle_writer.AddRecord(Slice("foooo")); + recycle_writer.AddRecord(Slice("bar")); + ASSERT_GE(get_reader_contents()->size(), log::kBlockSize * 2); + ASSERT_EQ("foooo", Read()); + ASSERT_EQ("bar", Read()); + ASSERT_EQ("EOF", Read()); +} + +INSTANTIATE_TEST_CASE_P(bool, LogTest, ::testing::Values(0, 2)); + } // namespace log } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/log_writer.cc b/db/log_writer.cc index df601a47064..b02eec89dd9 100644 --- a/db/log_writer.cc +++ b/db/log_writer.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -13,60 +13,72 @@ #include "rocksdb/env.h" #include "util/coding.h" #include "util/crc32c.h" +#include "util/file_reader_writer.h" namespace rocksdb { namespace log { -Writer::Writer(unique_ptr&& dest) +Writer::Writer(unique_ptr&& dest, uint64_t log_number, + bool recycle_log_files, bool manual_flush) : dest_(std::move(dest)), - block_offset_(0) { + block_offset_(0), + log_number_(log_number), + recycle_log_files_(recycle_log_files), + manual_flush_(manual_flush) { for (int i = 0; i <= kMaxRecordType; i++) { char t = static_cast(i); type_crc_[i] = crc32c::Value(&t, 1); } } -Writer::~Writer() { -} +Writer::~Writer() { WriteBuffer(); } + +Status Writer::WriteBuffer() { return dest_->Flush(); } Status Writer::AddRecord(const Slice& slice) { const char* ptr = slice.data(); size_t left = slice.size(); + // Header size varies depending on whether we are recycling or not. + const int header_size = + recycle_log_files_ ? kRecyclableHeaderSize : kHeaderSize; + // Fragment the record if necessary and emit it. Note that if slice // is empty, we still want to iterate once to emit a single // zero-length record Status s; bool begin = true; do { - const int leftover = kBlockSize - block_offset_; + const int64_t leftover = kBlockSize - block_offset_; assert(leftover >= 0); - if (leftover < kHeaderSize) { + if (leftover < header_size) { // Switch to a new block if (leftover > 0) { - // Fill the trailer (literal below relies on kHeaderSize being 7) - assert(kHeaderSize == 7); - dest_->Append(Slice("\x00\x00\x00\x00\x00\x00", leftover)); + // Fill the trailer (literal below relies on kHeaderSize and + // kRecyclableHeaderSize being <= 11) + assert(header_size <= 11); + dest_->Append( + Slice("\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00", leftover)); } block_offset_ = 0; } - // Invariant: we never leave < kHeaderSize bytes in a block. - assert(kBlockSize - block_offset_ - kHeaderSize >= 0); + // Invariant: we never leave < header_size bytes in a block. + assert(static_cast(kBlockSize - block_offset_) >= header_size); - const size_t avail = kBlockSize - block_offset_ - kHeaderSize; + const size_t avail = kBlockSize - block_offset_ - header_size; const size_t fragment_length = (left < avail) ? left : avail; RecordType type; const bool end = (left == fragment_length); if (begin && end) { - type = kFullType; + type = recycle_log_files_ ? kRecyclableFullType : kFullType; } else if (begin) { - type = kFirstType; + type = recycle_log_files_ ? kRecyclableFirstType : kFirstType; } else if (end) { - type = kLastType; + type = recycle_log_files_ ? kRecyclableLastType : kLastType; } else { - type = kMiddleType; + type = recycle_log_files_ ? kRecyclableMiddleType : kMiddleType; } s = EmitPhysicalRecord(type, ptr, fragment_length); @@ -79,28 +91,50 @@ Status Writer::AddRecord(const Slice& slice) { Status Writer::EmitPhysicalRecord(RecordType t, const char* ptr, size_t n) { assert(n <= 0xffff); // Must fit in two bytes - assert(block_offset_ + kHeaderSize + n <= kBlockSize); + + size_t header_size; + char buf[kRecyclableHeaderSize]; // Format the header - char buf[kHeaderSize]; buf[4] = static_cast(n & 0xff); buf[5] = static_cast(n >> 8); buf[6] = static_cast(t); + uint32_t crc = type_crc_[t]; + if (t < kRecyclableFullType) { + // Legacy record format + assert(block_offset_ + kHeaderSize + n <= kBlockSize); + header_size = kHeaderSize; + } else { + // Recyclable record format + assert(block_offset_ + kRecyclableHeaderSize + n <= kBlockSize); + header_size = kRecyclableHeaderSize; + + // Only encode low 32-bits of the 64-bit log number. This means + // we will fail to detect an old record if we recycled a log from + // ~4 billion logs ago, but that is effectively impossible, and + // even if it were we'dbe far more likely to see a false positive + // on the 32-bit CRC. + EncodeFixed32(buf + 7, static_cast(log_number_)); + crc = crc32c::Extend(crc, buf + 7, 4); + } + // Compute the crc of the record type and the payload. - uint32_t crc = crc32c::Extend(type_crc_[t], ptr, n); - crc = crc32c::Mask(crc); // Adjust for storage + crc = crc32c::Extend(crc, ptr, n); + crc = crc32c::Mask(crc); // Adjust for storage EncodeFixed32(buf, crc); // Write the header and the payload - Status s = dest_->Append(Slice(buf, kHeaderSize)); + Status s = dest_->Append(Slice(buf, header_size)); if (s.ok()) { s = dest_->Append(Slice(ptr, n)); if (s.ok()) { - s = dest_->Flush(); + if (!manual_flush_) { + s = dest_->Flush(); + } } } - block_offset_ += kHeaderSize + n; + block_offset_ += header_size + n; return s; } diff --git a/db/log_writer.h b/db/log_writer.h index d7b7afff094..a3a879924e9 100644 --- a/db/log_writer.h +++ b/db/log_writer.h @@ -1,43 +1,95 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. - #pragma once -#include + #include + +#include + #include "db/log_format.h" #include "rocksdb/slice.h" #include "rocksdb/status.h" namespace rocksdb { -class WritableFile; +class WritableFileWriter; using std::unique_ptr; namespace log { +/** + * Writer is a general purpose log stream writer. It provides an append-only + * abstraction for writing data. The details of the how the data is written is + * handled by the WriteableFile sub-class implementation. + * + * File format: + * + * File is broken down into variable sized records. The format of each record + * is described below. + * +-----+-------------+--+----+----------+------+-- ... ----+ + * File | r0 | r1 |P | r2 | r3 | r4 | | + * +-----+-------------+--+----+----------+------+-- ... ----+ + * <--- kBlockSize ------>|<-- kBlockSize ------>| + * rn = variable size records + * P = Padding + * + * Data is written out in kBlockSize chunks. If next record does not fit + * into the space left, the leftover space will be padded with \0. + * + * Legacy record format: + * + * +---------+-----------+-----------+--- ... ---+ + * |CRC (4B) | Size (2B) | Type (1B) | Payload | + * +---------+-----------+-----------+--- ... ---+ + * + * CRC = 32bit hash computed over the payload using CRC + * Size = Length of the payload data + * Type = Type of record + * (kZeroType, kFullType, kFirstType, kLastType, kMiddleType ) + * The type is used to group a bunch of records together to represent + * blocks that are larger than kBlockSize + * Payload = Byte stream as long as specified by the payload size + * + * Recyclable record format: + * + * +---------+-----------+-----------+----------------+--- ... ---+ + * |CRC (4B) | Size (2B) | Type (1B) | Log number (4B)| Payload | + * +---------+-----------+-----------+----------------+--- ... ---+ + * + * Same as above, with the addition of + * Log number = 32bit log file number, so that we can distinguish between + * records written by the most recent log writer vs a previous one. + */ class Writer { public: // Create a writer that will append data to "*dest". // "*dest" must be initially empty. // "*dest" must remain live while this Writer is in use. - explicit Writer(unique_ptr&& dest); + explicit Writer(unique_ptr&& dest, uint64_t log_number, + bool recycle_log_files, bool manual_flush = false); ~Writer(); Status AddRecord(const Slice& slice); - WritableFile* file() { return dest_.get(); } - const WritableFile* file() const { return dest_.get(); } + WritableFileWriter* file() { return dest_.get(); } + const WritableFileWriter* file() const { return dest_.get(); } + + uint64_t get_log_number() const { return log_number_; } + + Status WriteBuffer(); private: - unique_ptr dest_; - int block_offset_; // Current offset in block + unique_ptr dest_; + size_t block_offset_; // Current offset in block + uint64_t log_number_; + bool recycle_log_files_; // crc32c values for all supported record types. These are // pre-computed to reduce the overhead of computing the crc of the @@ -46,6 +98,10 @@ class Writer { Status EmitPhysicalRecord(RecordType type, const char* ptr, size_t length); + // If true, it does not flush after each write. Instead it relies on the upper + // layer to manually does the flush by calling ::WriteBuffer() + bool manual_flush_; + // No copying allowed Writer(const Writer&); void operator=(const Writer&); diff --git a/db/malloc_stats.cc b/db/malloc_stats.cc new file mode 100644 index 00000000000..7acca65123e --- /dev/null +++ b/db/malloc_stats.cc @@ -0,0 +1,52 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/malloc_stats.h" + +#ifndef ROCKSDB_LITE +#include +#include + +namespace rocksdb { + +#ifdef ROCKSDB_JEMALLOC +#include "jemalloc/jemalloc.h" + +typedef struct { + char* cur; + char* end; +} MallocStatus; + +static void GetJemallocStatus(void* mstat_arg, const char* status) { + MallocStatus* mstat = reinterpret_cast(mstat_arg); + size_t status_len = status ? strlen(status) : 0; + size_t buf_size = (size_t)(mstat->end - mstat->cur); + if (!status_len || status_len > buf_size) { + return; + } + + snprintf(mstat->cur, buf_size, "%s", status); + mstat->cur += status_len; +} +#endif // ROCKSDB_JEMALLOC + +void DumpMallocStats(std::string* stats) { +#ifdef ROCKSDB_JEMALLOC + MallocStatus mstat; + const unsigned int kMallocStatusLen = 1000000; + std::unique_ptr buf{new char[kMallocStatusLen + 1]}; + mstat.cur = buf.get(); + mstat.end = buf.get() + kMallocStatusLen; + je_malloc_stats_print(GetJemallocStatus, &mstat, ""); + stats->append(buf.get()); +#endif // ROCKSDB_JEMALLOC +} + +} +#endif // !ROCKSDB_LITE diff --git a/db/malloc_stats.h b/db/malloc_stats.h new file mode 100644 index 00000000000..a2f324ff18d --- /dev/null +++ b/db/malloc_stats.h @@ -0,0 +1,22 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#ifndef ROCKSDB_LITE + +#include + +namespace rocksdb { + +void DumpMallocStats(std::string*); + +} + +#endif // !ROCKSDB_LITE diff --git a/db/managed_iterator.cc b/db/managed_iterator.cc new file mode 100644 index 00000000000..c393eb5a6fd --- /dev/null +++ b/db/managed_iterator.cc @@ -0,0 +1,262 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include "db/managed_iterator.h" + +#include +#include +#include + +#include "db/column_family.h" +#include "db/db_impl.h" +#include "db/db_iter.h" +#include "db/dbformat.h" +#include "rocksdb/env.h" +#include "rocksdb/slice.h" +#include "rocksdb/slice_transform.h" +#include "table/merging_iterator.h" + +namespace rocksdb { + +namespace { +// Helper class that locks a mutex on construction and unlocks the mutex when +// the destructor of the MutexLock object is invoked. +// +// Typical usage: +// +// void MyClass::MyMethod() { +// MILock l(&mu_); // mu_ is an instance variable +// ... some complex code, possibly with multiple return paths ... +// } + +class MILock { + public: + explicit MILock(std::mutex* mu, ManagedIterator* mi) : mu_(mu), mi_(mi) { + this->mu_->lock(); + } + ~MILock() { + this->mu_->unlock(); + } + ManagedIterator* GetManagedIterator() { return mi_; } + + private: + std::mutex* const mu_; + ManagedIterator* mi_; + // No copying allowed + MILock(const MILock&) = delete; + void operator=(const MILock&) = delete; +}; +} // anonymous namespace + +// +// Synchronization between modifiers, releasers, creators +// If iterator operation, wait till (!in_use), set in_use, do op, reset in_use +// if modifying mutable_iter, atomically exchange in_use: +// return if in_use set / otherwise set in use, +// atomically replace new iter with old , reset in use +// The releaser is the new operation and it holds a lock for a very short time +// The existing non-const iterator operations are supposed to be single +// threaded and hold the lock for the duration of the operation +// The existing const iterator operations use the cached key/values +// and don't do any locking. +ManagedIterator::ManagedIterator(DBImpl* db, const ReadOptions& read_options, + ColumnFamilyData* cfd) + : db_(db), + read_options_(read_options), + cfd_(cfd), + svnum_(cfd->GetSuperVersionNumber()), + mutable_iter_(nullptr), + valid_(false), + snapshot_created_(false), + release_supported_(true) { + read_options_.managed = false; + if ((!read_options_.tailing) && (read_options_.snapshot == nullptr)) { + assert(nullptr != (read_options_.snapshot = db_->GetSnapshot())); + snapshot_created_ = true; + } + cfh_.SetCFD(cfd); + mutable_iter_ = unique_ptr(db->NewIterator(read_options_, &cfh_)); +} + +ManagedIterator::~ManagedIterator() { + Lock(); + if (snapshot_created_) { + db_->ReleaseSnapshot(read_options_.snapshot); + snapshot_created_ = false; + read_options_.snapshot = nullptr; + } + UnLock(); +} + +bool ManagedIterator::Valid() const { return valid_; } + +void ManagedIterator::SeekToLast() { + MILock l(&in_use_, this); + if (NeedToRebuild()) { + RebuildIterator(); + } + assert(mutable_iter_ != nullptr); + mutable_iter_->SeekToLast(); + if (mutable_iter_->status().ok()) { + UpdateCurrent(); + } +} + +void ManagedIterator::SeekToFirst() { + MILock l(&in_use_, this); + SeekInternal(Slice(), true); +} + +void ManagedIterator::Seek(const Slice& user_key) { + MILock l(&in_use_, this); + SeekInternal(user_key, false); +} + +void ManagedIterator::SeekForPrev(const Slice& user_key) { + MILock l(&in_use_, this); + if (NeedToRebuild()) { + RebuildIterator(); + } + assert(mutable_iter_ != nullptr); + mutable_iter_->SeekForPrev(user_key); + UpdateCurrent(); +} + +void ManagedIterator::SeekInternal(const Slice& user_key, bool seek_to_first) { + if (NeedToRebuild()) { + RebuildIterator(); + } + assert(mutable_iter_ != nullptr); + if (seek_to_first) { + mutable_iter_->SeekToFirst(); + } else { + mutable_iter_->Seek(user_key); + } + UpdateCurrent(); +} + +void ManagedIterator::Prev() { + if (!valid_) { + status_ = Status::InvalidArgument("Iterator value invalid"); + return; + } + MILock l(&in_use_, this); + if (NeedToRebuild()) { + std::string current_key = key().ToString(); + Slice old_key(current_key); + RebuildIterator(); + SeekInternal(old_key, false); + UpdateCurrent(); + if (!valid_) { + return; + } + if (key().compare(old_key) != 0) { + valid_ = false; + status_ = Status::Incomplete("Cannot do Prev now"); + return; + } + } + mutable_iter_->Prev(); + if (mutable_iter_->status().ok()) { + UpdateCurrent(); + status_ = Status::OK(); + } else { + status_ = mutable_iter_->status(); + } +} + +void ManagedIterator::Next() { + if (!valid_) { + status_ = Status::InvalidArgument("Iterator value invalid"); + return; + } + MILock l(&in_use_, this); + if (NeedToRebuild()) { + std::string current_key = key().ToString(); + Slice old_key(current_key.data(), cached_key_.Size()); + RebuildIterator(); + SeekInternal(old_key, false); + UpdateCurrent(); + if (!valid_) { + return; + } + if (key().compare(old_key) != 0) { + valid_ = false; + status_ = Status::Incomplete("Cannot do Next now"); + return; + } + } + mutable_iter_->Next(); + UpdateCurrent(); +} + +Slice ManagedIterator::key() const { + assert(valid_); + return cached_key_.GetUserKey(); +} + +Slice ManagedIterator::value() const { + assert(valid_); + return cached_value_.GetUserKey(); +} + +Status ManagedIterator::status() const { return status_; } + +void ManagedIterator::RebuildIterator() { + svnum_ = cfd_->GetSuperVersionNumber(); + mutable_iter_ = unique_ptr(db_->NewIterator(read_options_, &cfh_)); +} + +void ManagedIterator::UpdateCurrent() { + assert(mutable_iter_ != nullptr); + + valid_ = mutable_iter_->Valid(); + if (!valid_) { + status_ = mutable_iter_->status(); + return; + } + + status_ = Status::OK(); + cached_key_.SetUserKey(mutable_iter_->key()); + cached_value_.SetUserKey(mutable_iter_->value()); +} + +void ManagedIterator::ReleaseIter(bool only_old) { + if ((mutable_iter_ == nullptr) || (!release_supported_)) { + return; + } + if (svnum_ != cfd_->GetSuperVersionNumber() || !only_old) { + if (!TryLock()) { // Don't release iter if in use + return; + } + mutable_iter_ = nullptr; // in_use for a very short time + UnLock(); + } +} + +bool ManagedIterator::NeedToRebuild() { + if ((mutable_iter_ == nullptr) || (status_.IsIncomplete()) || + (!only_drop_old_ && (svnum_ != cfd_->GetSuperVersionNumber()))) { + return true; + } + return false; +} + +void ManagedIterator::Lock() { + in_use_.lock(); + return; +} + +bool ManagedIterator::TryLock() { return in_use_.try_lock(); } + +void ManagedIterator::UnLock() { + in_use_.unlock(); +} + +} // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/db/managed_iterator.h b/db/managed_iterator.h new file mode 100644 index 00000000000..8e962f781a2 --- /dev/null +++ b/db/managed_iterator.h @@ -0,0 +1,85 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once + +#ifndef ROCKSDB_LITE + +#include +#include +#include +#include + +#include "db/column_family.h" +#include "rocksdb/db.h" +#include "rocksdb/iterator.h" +#include "rocksdb/options.h" +#include "util/arena.h" + +namespace rocksdb { + +class DBImpl; +struct SuperVersion; +class ColumnFamilyData; + +/** + * ManagedIterator is a special type of iterator that supports freeing the + * underlying iterator and still being able to access the current key/value + * pair. This is done by copying the key/value pair so that clients can + * continue to access the data without getting a SIGSEGV. + * The underlying iterator can be freed manually through the call to + * ReleaseIter or automatically (as needed on space pressure or age.) + * The iterator is recreated using the saved original arguments. + */ +class ManagedIterator : public Iterator { + public: + ManagedIterator(DBImpl* db, const ReadOptions& read_options, + ColumnFamilyData* cfd); + virtual ~ManagedIterator(); + + virtual void SeekToLast() override; + virtual void Prev() override; + virtual bool Valid() const override; + void SeekToFirst() override; + virtual void Seek(const Slice& target) override; + virtual void SeekForPrev(const Slice& target) override; + virtual void Next() override; + virtual Slice key() const override; + virtual Slice value() const override; + virtual Status status() const override; + void ReleaseIter(bool only_old); + void SetDropOld(bool only_old) { + only_drop_old_ = read_options_.tailing || only_old; + } + + private: + void RebuildIterator(); + void UpdateCurrent(); + void SeekInternal(const Slice& user_key, bool seek_to_first); + bool NeedToRebuild(); + void Lock(); + bool TryLock(); + void UnLock(); + DBImpl* const db_; + ReadOptions read_options_; + ColumnFamilyData* const cfd_; + ColumnFamilyHandleInternal cfh_; + + uint64_t svnum_; + std::unique_ptr mutable_iter_; + // internal iterator status + Status status_; + bool valid_; + + IterKey cached_key_; + IterKey cached_value_; + + bool only_drop_old_ = true; + bool snapshot_created_; + bool release_supported_; + std::mutex in_use_; // is managed iterator in use +}; + +} // namespace rocksdb +#endif // !ROCKSDB_LITE diff --git a/util/manual_compaction_test.cc b/db/manual_compaction_test.cc similarity index 82% rename from util/manual_compaction_test.cc rename to db/manual_compaction_test.cc index dd615f05704..039b9080ed3 100644 --- a/util/manual_compaction_test.cc +++ b/db/manual_compaction_test.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Test for issue 178: a manual compaction causes deleted data to reappear. #include @@ -13,6 +13,7 @@ #include "rocksdb/slice.h" #include "rocksdb/write_batch.h" #include "util/testharness.h" +#include "port/port.h" using namespace rocksdb; @@ -30,7 +31,7 @@ std::string Key2(int i) { return Key1(i) + "_xxx"; } -class ManualCompactionTest { +class ManualCompactionTest : public testing::Test { public: ManualCompactionTest() { // Get rid of any state from an old run. @@ -45,20 +46,18 @@ class DestroyAllCompactionFilter : public CompactionFilter { public: DestroyAllCompactionFilter() {} - virtual bool Filter(int level, - const Slice& key, - const Slice& existing_value, + virtual bool Filter(int level, const Slice& key, const Slice& existing_value, std::string* new_value, - bool* value_changed) const { + bool* value_changed) const override { return existing_value.ToString() == "destroy"; } - virtual const char* Name() const { + virtual const char* Name() const override { return "DestroyAllCompactionFilter"; } }; -TEST(ManualCompactionTest, CompactTouchesAllKeys) { +TEST_F(ManualCompactionTest, CompactTouchesAllKeys) { for (int iter = 0; iter < 2; ++iter) { DB* db; Options options; @@ -79,7 +78,7 @@ TEST(ManualCompactionTest, CompactTouchesAllKeys) { db->Put(WriteOptions(), Slice("key4"), Slice("destroy")); Slice key4("key4"); - db->CompactRange(nullptr, &key4); + db->CompactRange(CompactRangeOptions(), nullptr, &key4); Iterator* itr = db->NewIterator(ReadOptions()); itr->SeekToFirst(); ASSERT_TRUE(itr->Valid()); @@ -94,8 +93,7 @@ TEST(ManualCompactionTest, CompactTouchesAllKeys) { } } -TEST(ManualCompactionTest, Test) { - +TEST_F(ManualCompactionTest, Test) { // Open database. Disable compression since it affects the creation // of layers and the code below is trying to test against a very // specific scenario. @@ -133,7 +131,7 @@ TEST(ManualCompactionTest, Test) { rocksdb::Slice greatest(end_key.data(), end_key.size()); // commenting out the line below causes the example to work correctly - db->CompactRange(&least, &greatest); + db->CompactRange(CompactRangeOptions(), &least, &greatest); // count the keys rocksdb::Iterator* iter = db->NewIterator(rocksdb::ReadOptions()); @@ -152,5 +150,6 @@ TEST(ManualCompactionTest, Test) { } // anonymous namespace int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/memtable.cc b/db/memtable.cc index e9e7051c75f..d51b2618734 100644 --- a/db/memtable.cc +++ b/db/memtable.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,102 +9,159 @@ #include "db/memtable.h" -#include #include #include +#include #include "db/dbformat.h" #include "db/merge_context.h" +#include "db/merge_helper.h" +#include "db/pinned_iterators_manager.h" +#include "monitoring/perf_context_imp.h" +#include "monitoring/statistics.h" +#include "port/port.h" #include "rocksdb/comparator.h" #include "rocksdb/env.h" #include "rocksdb/iterator.h" #include "rocksdb/merge_operator.h" #include "rocksdb/slice_transform.h" -#include "table/merger.h" +#include "rocksdb/write_buffer_manager.h" +#include "table/internal_iterator.h" +#include "table/iterator_wrapper.h" +#include "table/merging_iterator.h" #include "util/arena.h" +#include "util/autovector.h" #include "util/coding.h" +#include "util/memory_usage.h" #include "util/murmurhash.h" #include "util/mutexlock.h" -#include "util/perf_context_imp.h" -#include "util/statistics.h" -#include "util/stop_watch.h" namespace rocksdb { -MemTable::MemTable(const InternalKeyComparator& cmp, const Options& options) +ImmutableMemTableOptions::ImmutableMemTableOptions( + const ImmutableCFOptions& ioptions, + const MutableCFOptions& mutable_cf_options) + : arena_block_size(mutable_cf_options.arena_block_size), + memtable_prefix_bloom_bits( + static_cast( + static_cast(mutable_cf_options.write_buffer_size) * + mutable_cf_options.memtable_prefix_bloom_size_ratio) * + 8u), + memtable_huge_page_size(mutable_cf_options.memtable_huge_page_size), + inplace_update_support(ioptions.inplace_update_support), + inplace_update_num_locks(mutable_cf_options.inplace_update_num_locks), + inplace_callback(ioptions.inplace_callback), + max_successive_merges(mutable_cf_options.max_successive_merges), + statistics(ioptions.statistics), + merge_operator(ioptions.merge_operator), + info_log(ioptions.info_log) {} + +MemTable::MemTable(const InternalKeyComparator& cmp, + const ImmutableCFOptions& ioptions, + const MutableCFOptions& mutable_cf_options, + WriteBufferManager* write_buffer_manager, + SequenceNumber latest_seq, uint32_t column_family_id) : comparator_(cmp), + moptions_(ioptions, mutable_cf_options), refs_(0), - kArenaBlockSize(OptimizeBlockSize(options.arena_block_size)), - kWriteBufferSize(options.write_buffer_size), - arena_(options.arena_block_size), - table_(options.memtable_factory->CreateMemTableRep( - comparator_, &arena_, options.prefix_extractor.get(), - options.info_log.get())), + kArenaBlockSize(OptimizeBlockSize(moptions_.arena_block_size)), + mem_tracker_(write_buffer_manager), + arena_( + moptions_.arena_block_size, + (write_buffer_manager != nullptr && write_buffer_manager->enabled()) + ? &mem_tracker_ + : nullptr, + mutable_cf_options.memtable_huge_page_size), + table_(ioptions.memtable_factory->CreateMemTableRep( + comparator_, &arena_, ioptions.prefix_extractor, ioptions.info_log, + column_family_id)), + range_del_table_(SkipListFactory().CreateMemTableRep( + comparator_, &arena_, nullptr /* transform */, ioptions.info_log, + column_family_id)), + is_range_del_table_empty_(true), + data_size_(0), num_entries_(0), + num_deletes_(0), + write_buffer_size_(mutable_cf_options.write_buffer_size), flush_in_progress_(false), flush_completed_(false), file_number_(0), first_seqno_(0), + earliest_seqno_(latest_seq), + creation_seq_(latest_seq), mem_next_logfile_number_(0), - locks_(options.inplace_update_support ? options.inplace_update_num_locks - : 0), - prefix_extractor_(options.prefix_extractor.get()), - should_flush_(ShouldFlushNow()) { - // if should_flush_ == true without an entry inserted, something must have - // gone wrong already. - assert(!should_flush_); - if (prefix_extractor_ && options.memtable_prefix_bloom_bits > 0) { + min_prep_log_referenced_(0), + locks_(moptions_.inplace_update_support + ? moptions_.inplace_update_num_locks + : 0), + prefix_extractor_(ioptions.prefix_extractor), + flush_state_(FLUSH_NOT_REQUESTED), + env_(ioptions.env), + insert_with_hint_prefix_extractor_( + ioptions.memtable_insert_with_hint_prefix_extractor), + oldest_key_time_(std::numeric_limits::max()) { + UpdateFlushState(); + // something went wrong if we need to flush before inserting anything + assert(!ShouldScheduleFlush()); + + if (prefix_extractor_ && moptions_.memtable_prefix_bloom_bits > 0) { prefix_bloom_.reset(new DynamicBloom( - &arena_, - options.memtable_prefix_bloom_bits, options.bloom_locality, - options.memtable_prefix_bloom_probes, nullptr, - options.memtable_prefix_bloom_huge_page_tlb_size, - options.info_log.get())); + &arena_, moptions_.memtable_prefix_bloom_bits, ioptions.bloom_locality, + 6 /* hard coded 6 probes */, nullptr, moptions_.memtable_huge_page_size, + ioptions.info_log)); } } MemTable::~MemTable() { + mem_tracker_.FreeMem(); assert(refs_ == 0); } size_t MemTable::ApproximateMemoryUsage() { - size_t arena_usage = arena_.ApproximateMemoryUsage(); - size_t table_usage = table_->ApproximateMemoryUsage(); - // let MAX_USAGE = std::numeric_limits::max() - // then if arena_usage + total_usage >= MAX_USAGE, return MAX_USAGE. - // the following variation is to avoid numeric overflow. - if (arena_usage >= std::numeric_limits::max() - table_usage) { - return std::numeric_limits::max(); + autovector usages = {arena_.ApproximateMemoryUsage(), + table_->ApproximateMemoryUsage(), + range_del_table_->ApproximateMemoryUsage(), + rocksdb::ApproximateMemoryUsage(insert_hints_)}; + size_t total_usage = 0; + for (size_t usage : usages) { + // If usage + total_usage >= kMaxSizet, return kMaxSizet. + // the following variation is to avoid numeric overflow. + if (usage >= port::kMaxSizet - total_usage) { + return port::kMaxSizet; + } + total_usage += usage; } // otherwise, return the actual usage - return arena_usage + table_usage; + return total_usage; } bool MemTable::ShouldFlushNow() const { + size_t write_buffer_size = write_buffer_size_.load(std::memory_order_relaxed); // In a lot of times, we cannot allocate arena blocks that exactly matches the // buffer size. Thus we have to decide if we should over-allocate or // under-allocate. - // This constant avariable can be interpreted as: if we still have more than + // This constant variable can be interpreted as: if we still have more than // "kAllowOverAllocationRatio * kArenaBlockSize" space left, we'd try to over // allocate one more block. const double kAllowOverAllocationRatio = 0.6; // If arena still have room for new block allocation, we can safely say it // shouldn't flush. - auto allocated_memory = - table_->ApproximateMemoryUsage() + arena_.MemoryAllocatedBytes(); + auto allocated_memory = table_->ApproximateMemoryUsage() + + range_del_table_->ApproximateMemoryUsage() + + arena_.MemoryAllocatedBytes(); // if we can still allocate one more block without exceeding the // over-allocation ratio, then we should not flush. if (allocated_memory + kArenaBlockSize < - kWriteBufferSize + kArenaBlockSize * kAllowOverAllocationRatio) { + write_buffer_size + kArenaBlockSize * kAllowOverAllocationRatio) { return false; } - // if user keeps adding entries that exceeds kWriteBufferSize, we need to + // if user keeps adding entries that exceeds write_buffer_size, we need to // flush earlier even though we still have much available memory left. if (allocated_memory > - kWriteBufferSize + kArenaBlockSize * kAllowOverAllocationRatio) { + write_buffer_size + kArenaBlockSize * kAllowOverAllocationRatio) { return true; } @@ -136,6 +193,32 @@ bool MemTable::ShouldFlushNow() const { return arena_.AllocatedAndUnused() < kArenaBlockSize / 4; } +void MemTable::UpdateFlushState() { + auto state = flush_state_.load(std::memory_order_relaxed); + if (state == FLUSH_NOT_REQUESTED && ShouldFlushNow()) { + // ignore CAS failure, because that means somebody else requested + // a flush + flush_state_.compare_exchange_strong(state, FLUSH_REQUESTED, + std::memory_order_relaxed, + std::memory_order_relaxed); + } +} + +void MemTable::UpdateOldestKeyTime() { + uint64_t oldest_key_time = oldest_key_time_.load(std::memory_order_relaxed); + if (oldest_key_time == std::numeric_limits::max()) { + int64_t current_time = 0; + auto s = env_->GetCurrentTime(¤t_time); + if (s.ok()) { + assert(current_time >= 0); + // If fail, the timestamp is already set. + oldest_key_time_.compare_exchange_strong( + oldest_key_time, static_cast(current_time), + std::memory_order_relaxed, std::memory_order_relaxed); + } + } +} + int MemTable::KeyComparator::operator()(const char* prefix_len_key1, const char* prefix_len_key2) const { // Internal keys are encoded as length-prefixed strings. @@ -158,7 +241,7 @@ Slice MemTableRep::UserKey(const char* key) const { } KeyHandle MemTableRep::Allocate(const size_t len, char** buf) { - *buf = arena_->Allocate(len); + *buf = allocator_->Allocate(len); return static_cast(*buf); } @@ -167,20 +250,25 @@ KeyHandle MemTableRep::Allocate(const size_t len, char** buf) { // into this scratch space. const char* EncodeKey(std::string* scratch, const Slice& target) { scratch->clear(); - PutVarint32(scratch, target.size()); + PutVarint32(scratch, static_cast(target.size())); scratch->append(target.data(), target.size()); return scratch->data(); } -class MemTableIterator: public Iterator { +class MemTableIterator : public InternalIterator { public: - MemTableIterator( - const MemTable& mem, const ReadOptions& options, Arena* arena) + MemTableIterator(const MemTable& mem, const ReadOptions& read_options, + Arena* arena, bool use_range_del_table = false) : bloom_(nullptr), prefix_extractor_(mem.prefix_extractor_), + comparator_(mem.comparator_), valid_(false), - arena_mode_(arena != nullptr) { - if (prefix_extractor_ != nullptr && !options.total_order_seek) { + arena_mode_(arena != nullptr), + value_pinned_( + !mem.GetImmutableMemTableOptions()->inplace_update_support) { + if (use_range_del_table) { + iter_ = mem.range_del_table_->GetIterator(arena); + } else if (prefix_extractor_ != nullptr && !read_options.total_order_seek) { bloom_ = mem.prefix_bloom_.get(); iter_ = mem.table_->GetDynamicPrefixIterator(arena); } else { @@ -189,6 +277,12 @@ class MemTableIterator: public Iterator { } ~MemTableIterator() { +#ifndef NDEBUG + // Assert that the MemTableIterator is never deleted while + // Pinning is Enabled. + assert(!pinned_iters_mgr_ || + (pinned_iters_mgr_ && !pinned_iters_mgr_->PinningEnabled())); +#endif if (arena_mode_) { iter_->~Iterator(); } else { @@ -196,66 +290,123 @@ class MemTableIterator: public Iterator { } } - virtual bool Valid() const { return valid_; } - virtual void Seek(const Slice& k) { - if (bloom_ != nullptr && - !bloom_->MayContain(prefix_extractor_->Transform(ExtractUserKey(k)))) { - valid_ = false; - return; +#ifndef NDEBUG + virtual void SetPinnedItersMgr( + PinnedIteratorsManager* pinned_iters_mgr) override { + pinned_iters_mgr_ = pinned_iters_mgr; + } + PinnedIteratorsManager* pinned_iters_mgr_ = nullptr; +#endif + + virtual bool Valid() const override { return valid_; } + virtual void Seek(const Slice& k) override { + PERF_TIMER_GUARD(seek_on_memtable_time); + PERF_COUNTER_ADD(seek_on_memtable_count, 1); + if (bloom_ != nullptr) { + if (!bloom_->MayContain( + prefix_extractor_->Transform(ExtractUserKey(k)))) { + PERF_COUNTER_ADD(bloom_memtable_miss_count, 1); + valid_ = false; + return; + } else { + PERF_COUNTER_ADD(bloom_memtable_hit_count, 1); + } + } + iter_->Seek(k, nullptr); + valid_ = iter_->Valid(); + } + virtual void SeekForPrev(const Slice& k) override { + PERF_TIMER_GUARD(seek_on_memtable_time); + PERF_COUNTER_ADD(seek_on_memtable_count, 1); + if (bloom_ != nullptr) { + if (!bloom_->MayContain( + prefix_extractor_->Transform(ExtractUserKey(k)))) { + PERF_COUNTER_ADD(bloom_memtable_miss_count, 1); + valid_ = false; + return; + } else { + PERF_COUNTER_ADD(bloom_memtable_hit_count, 1); + } } iter_->Seek(k, nullptr); valid_ = iter_->Valid(); + if (!Valid()) { + SeekToLast(); + } + while (Valid() && comparator_.comparator.Compare(k, key()) < 0) { + Prev(); + } } - virtual void SeekToFirst() { + virtual void SeekToFirst() override { iter_->SeekToFirst(); valid_ = iter_->Valid(); } - virtual void SeekToLast() { + virtual void SeekToLast() override { iter_->SeekToLast(); valid_ = iter_->Valid(); } - virtual void Next() { + virtual void Next() override { + PERF_COUNTER_ADD(next_on_memtable_count, 1); assert(Valid()); iter_->Next(); valid_ = iter_->Valid(); } - virtual void Prev() { + virtual void Prev() override { + PERF_COUNTER_ADD(prev_on_memtable_count, 1); assert(Valid()); iter_->Prev(); valid_ = iter_->Valid(); } - virtual Slice key() const { + virtual Slice key() const override { assert(Valid()); return GetLengthPrefixedSlice(iter_->key()); } - virtual Slice value() const { + virtual Slice value() const override { assert(Valid()); Slice key_slice = GetLengthPrefixedSlice(iter_->key()); return GetLengthPrefixedSlice(key_slice.data() + key_slice.size()); } - virtual Status status() const { return Status::OK(); } + virtual Status status() const override { return Status::OK(); } + + virtual bool IsKeyPinned() const override { + // memtable data is always pinned + return true; + } + + virtual bool IsValuePinned() const override { + // memtable value is always pinned, except if we allow inplace update. + return value_pinned_; + } private: DynamicBloom* bloom_; const SliceTransform* const prefix_extractor_; + const MemTable::KeyComparator comparator_; MemTableRep::Iterator* iter_; bool valid_; bool arena_mode_; + bool value_pinned_; // No copying allowed MemTableIterator(const MemTableIterator&); void operator=(const MemTableIterator&); }; -Iterator* MemTable::NewIterator(const ReadOptions& options, Arena* arena) { - if (arena == nullptr) { - return new MemTableIterator(*this, options, nullptr); - } else { - auto mem = arena->AllocateAligned(sizeof(MemTableIterator)); - return new (mem) - MemTableIterator(*this, options, arena); +InternalIterator* MemTable::NewIterator(const ReadOptions& read_options, + Arena* arena) { + assert(arena != nullptr); + auto mem = arena->AllocateAligned(sizeof(MemTableIterator)); + return new (mem) MemTableIterator(*this, read_options, arena); +} + +InternalIterator* MemTable::NewRangeTombstoneIterator( + const ReadOptions& read_options) { + if (read_options.ignore_range_deletions || is_range_del_table_empty_) { + return nullptr; } + return new MemTableIterator(*this, read_options, nullptr /* arena */, + true /* use_range_del_table */); } port::RWMutex* MemTable::GetLock(const Slice& key) { @@ -263,46 +414,127 @@ port::RWMutex* MemTable::GetLock(const Slice& key) { return &locks_[hash(key) % locks_.size()]; } +MemTable::MemTableStats MemTable::ApproximateStats(const Slice& start_ikey, + const Slice& end_ikey) { + uint64_t entry_count = table_->ApproximateNumEntries(start_ikey, end_ikey); + entry_count += range_del_table_->ApproximateNumEntries(start_ikey, end_ikey); + if (entry_count == 0) { + return {0, 0}; + } + uint64_t n = num_entries_.load(std::memory_order_relaxed); + if (n == 0) { + return {0, 0}; + } + if (entry_count > n) { + // (range_del_)table_->ApproximateNumEntries() is just an estimate so it can + // be larger than actual entries we have. Cap it to entries we have to limit + // the inaccuracy. + entry_count = n; + } + uint64_t data_size = data_size_.load(std::memory_order_relaxed); + return {entry_count * (data_size / n), entry_count}; +} + void MemTable::Add(SequenceNumber s, ValueType type, const Slice& key, /* user key */ - const Slice& value) { + const Slice& value, bool allow_concurrent, + MemTablePostProcessInfo* post_process_info) { // Format of an entry is concatenation of: // key_size : varint32 of internal_key.size() // key bytes : char[internal_key.size()] // value_size : varint32 of value.size() // value bytes : char[value.size()] - size_t key_size = key.size(); - size_t val_size = value.size(); - size_t internal_key_size = key_size + 8; - const size_t encoded_len = - VarintLength(internal_key_size) + internal_key_size + - VarintLength(val_size) + val_size; + uint32_t key_size = static_cast(key.size()); + uint32_t val_size = static_cast(value.size()); + uint32_t internal_key_size = key_size + 8; + const uint32_t encoded_len = VarintLength(internal_key_size) + + internal_key_size + VarintLength(val_size) + + val_size; char* buf = nullptr; - KeyHandle handle = table_->Allocate(encoded_len, &buf); - assert(buf != nullptr); + std::unique_ptr& table = + type == kTypeRangeDeletion ? range_del_table_ : table_; + KeyHandle handle = table->Allocate(encoded_len, &buf); + char* p = EncodeVarint32(buf, internal_key_size); memcpy(p, key.data(), key_size); + Slice key_slice(p, key_size); p += key_size; - EncodeFixed64(p, (s << 8) | type); + uint64_t packed = PackSequenceAndType(s, type); + EncodeFixed64(p, packed); p += 8; p = EncodeVarint32(p, val_size); memcpy(p, value.data(), val_size); assert((unsigned)(p + val_size - buf) == (unsigned)encoded_len); - table_->Insert(handle); - num_entries_++; + if (!allow_concurrent) { + // Extract prefix for insert with hint. + if (insert_with_hint_prefix_extractor_ != nullptr && + insert_with_hint_prefix_extractor_->InDomain(key_slice)) { + Slice prefix = insert_with_hint_prefix_extractor_->Transform(key_slice); + table->InsertWithHint(handle, &insert_hints_[prefix]); + } else { + table->Insert(handle); + } - if (prefix_bloom_) { - assert(prefix_extractor_); - prefix_bloom_->Add(prefix_extractor_->Transform(key)); - } + // this is a bit ugly, but is the way to avoid locked instructions + // when incrementing an atomic + num_entries_.store(num_entries_.load(std::memory_order_relaxed) + 1, + std::memory_order_relaxed); + data_size_.store(data_size_.load(std::memory_order_relaxed) + encoded_len, + std::memory_order_relaxed); + if (type == kTypeDeletion) { + num_deletes_.store(num_deletes_.load(std::memory_order_relaxed) + 1, + std::memory_order_relaxed); + } - // The first sequence number inserted into the memtable - assert(first_seqno_ == 0 || s > first_seqno_); - if (first_seqno_ == 0) { - first_seqno_ = s; - } + if (prefix_bloom_) { + assert(prefix_extractor_); + prefix_bloom_->Add(prefix_extractor_->Transform(key)); + } + + // The first sequence number inserted into the memtable + assert(first_seqno_ == 0 || s > first_seqno_); + if (first_seqno_ == 0) { + first_seqno_.store(s, std::memory_order_relaxed); - should_flush_ = ShouldFlushNow(); + if (earliest_seqno_ == kMaxSequenceNumber) { + earliest_seqno_.store(GetFirstSequenceNumber(), + std::memory_order_relaxed); + } + assert(first_seqno_.load() >= earliest_seqno_.load()); + } + assert(post_process_info == nullptr); + UpdateFlushState(); + } else { + table->InsertConcurrently(handle); + + assert(post_process_info != nullptr); + post_process_info->num_entries++; + post_process_info->data_size += encoded_len; + if (type == kTypeDeletion) { + post_process_info->num_deletes++; + } + + if (prefix_bloom_) { + assert(prefix_extractor_); + prefix_bloom_->AddConcurrently(prefix_extractor_->Transform(key)); + } + + // atomically update first_seqno_ and earliest_seqno_. + uint64_t cur_seq_num = first_seqno_.load(std::memory_order_relaxed); + while ((cur_seq_num == 0 || s < cur_seq_num) && + !first_seqno_.compare_exchange_weak(cur_seq_num, s)) { + } + uint64_t cur_earliest_seqno = + earliest_seqno_.load(std::memory_order_relaxed); + while ( + (cur_earliest_seqno == kMaxSequenceNumber || s < cur_earliest_seqno) && + !first_seqno_.compare_exchange_weak(cur_earliest_seqno, s)) { + } + } + if (is_range_del_table_empty_ && type == kTypeRangeDeletion) { + is_range_del_table_empty_ = false; + } + UpdateOldestKeyTime(); } // Callback from MemTable::Get() @@ -314,22 +546,27 @@ struct Saver { bool* found_final_value; // Is value set correctly? Used by KeyMayExist bool* merge_in_progress; std::string* value; + SequenceNumber seq; const MergeOperator* merge_operator; // the merge operations encountered; MergeContext* merge_context; + RangeDelAggregator* range_del_agg; MemTable* mem; Logger* logger; Statistics* statistics; bool inplace_update_support; + Env* env_; + bool* is_blob_index; }; } // namespace static bool SaveValue(void* arg, const char* entry) { Saver* s = reinterpret_cast(arg); MergeContext* merge_context = s->merge_context; + RangeDelAggregator* range_del_agg = s->range_del_agg; const MergeOperator* merge_operator = s->merge_operator; - assert(s != nullptr && merge_context != nullptr); + assert(s != nullptr && merge_context != nullptr && range_del_agg != nullptr); // entry format is: // klength varint32 @@ -342,11 +579,33 @@ static bool SaveValue(void* arg, const char* entry) { // all entries with overly large sequence numbers. uint32_t key_length; const char* key_ptr = GetVarint32Ptr(entry, entry + 5, &key_length); - if (s->mem->GetInternalKeyComparator().user_comparator()->Compare( - Slice(key_ptr, key_length - 8), s->key->user_key()) == 0) { + if (s->mem->GetInternalKeyComparator().user_comparator()->Equal( + Slice(key_ptr, key_length - 8), s->key->user_key())) { // Correct user key const uint64_t tag = DecodeFixed64(key_ptr + key_length - 8); - switch (static_cast(tag & 0xff)) { + ValueType type; + UnPackSequenceAndType(tag, &s->seq, &type); + + if ((type == kTypeValue || type == kTypeMerge || type == kTypeBlobIndex) && + range_del_agg->ShouldDelete(Slice(key_ptr, key_length))) { + type = kTypeRangeDeletion; + } + switch (type) { + case kTypeBlobIndex: + if (s->is_blob_index == nullptr) { + ROCKS_LOG_ERROR(s->logger, "Encounter unexpected blob index."); + *(s->status) = Status::NotSupported( + "Encounter unsupported blob value. Please open DB with " + "rocksdb::blob_db::BlobDB instead."); + } else if (*(s->merge_in_progress)) { + *(s->status) = + Status::NotSupported("Blob DB does not support merge operator."); + } + if (!s->status->ok()) { + *(s->found_final_value) = true; + return false; + } + // intentional fallthrough case kTypeValue: { if (s->inplace_update_support) { s->mem->GetLock(s->key->user_key())->ReadLock(); @@ -354,34 +613,30 @@ static bool SaveValue(void* arg, const char* entry) { Slice v = GetLengthPrefixedSlice(key_ptr + key_length); *(s->status) = Status::OK(); if (*(s->merge_in_progress)) { - assert(merge_operator); - if (!merge_operator->FullMerge(s->key->user_key(), &v, - merge_context->GetOperands(), s->value, - s->logger)) { - RecordTick(s->statistics, NUMBER_MERGE_FAILURES); - *(s->status) = - Status::Corruption("Error: Could not perform merge."); - } - } else { + *(s->status) = MergeHelper::TimedFullMerge( + merge_operator, s->key->user_key(), &v, + merge_context->GetOperands(), s->value, s->logger, s->statistics, + s->env_, nullptr /* result_operand */, true); + } else if (s->value != nullptr) { s->value->assign(v.data(), v.size()); } if (s->inplace_update_support) { s->mem->GetLock(s->key->user_key())->ReadUnlock(); } *(s->found_final_value) = true; + if (s->is_blob_index != nullptr) { + *(s->is_blob_index) = (type == kTypeBlobIndex); + } return false; } - case kTypeDeletion: { + case kTypeDeletion: + case kTypeSingleDeletion: + case kTypeRangeDeletion: { if (*(s->merge_in_progress)) { - assert(merge_operator); - *(s->status) = Status::OK(); - if (!merge_operator->FullMerge(s->key->user_key(), nullptr, - merge_context->GetOperands(), s->value, - s->logger)) { - RecordTick(s->statistics, NUMBER_MERGE_FAILURES); - *(s->status) = - Status::Corruption("Error: Could not perform merge."); - } + *(s->status) = MergeHelper::TimedFullMerge( + merge_operator, s->key->user_key(), nullptr, + merge_context->GetOperands(), s->value, s->logger, s->statistics, + s->env_, nullptr /* result_operand */, true); } else { *(s->status) = Status::NotFound(); } @@ -399,10 +654,10 @@ static bool SaveValue(void* arg, const char* entry) { *(s->found_final_value) = true; return false; } - std::string merge_result; // temporary area for merge results later Slice v = GetLengthPrefixedSlice(key_ptr + key_length); *(s->merge_in_progress) = true; - merge_context->PushOperand(v); + merge_context->PushOperand( + v, s->inplace_update_support == false /* operand_pinned */); return true; } default: @@ -416,41 +671,63 @@ static bool SaveValue(void* arg, const char* entry) { } bool MemTable::Get(const LookupKey& key, std::string* value, Status* s, - MergeContext& merge_context, const Options& options) { + MergeContext* merge_context, + RangeDelAggregator* range_del_agg, SequenceNumber* seq, + const ReadOptions& read_opts, bool* is_blob_index) { // The sequence number is updated synchronously in version_set.h - if (first_seqno_ == 0) { + if (IsEmpty()) { // Avoiding recording stats for speed. return false; } PERF_TIMER_GUARD(get_from_memtable_time); + std::unique_ptr range_del_iter( + NewRangeTombstoneIterator(read_opts)); + Status status = range_del_agg->AddTombstones(std::move(range_del_iter)); + if (!status.ok()) { + *s = status; + return false; + } + Slice user_key = key.user_key(); bool found_final_value = false; bool merge_in_progress = s->IsMergeInProgress(); - - if (prefix_bloom_ && - !prefix_bloom_->MayContain(prefix_extractor_->Transform(user_key))) { + bool const may_contain = + nullptr == prefix_bloom_ + ? false + : prefix_bloom_->MayContain(prefix_extractor_->Transform(user_key)); + if (prefix_bloom_ && !may_contain) { // iter is null if prefix bloom says the key does not exist + PERF_COUNTER_ADD(bloom_memtable_miss_count, 1); + *seq = kMaxSequenceNumber; } else { + if (prefix_bloom_) { + PERF_COUNTER_ADD(bloom_memtable_hit_count, 1); + } Saver saver; saver.status = s; saver.found_final_value = &found_final_value; saver.merge_in_progress = &merge_in_progress; saver.key = &key; saver.value = value; - saver.status = s; + saver.seq = kMaxSequenceNumber; saver.mem = this; - saver.merge_context = &merge_context; - saver.merge_operator = options.merge_operator.get(); - saver.logger = options.info_log.get(); - saver.inplace_update_support = options.inplace_update_support; - saver.statistics = options.statistics.get(); + saver.merge_context = merge_context; + saver.range_del_agg = range_del_agg; + saver.merge_operator = moptions_.merge_operator; + saver.logger = moptions_.info_log; + saver.inplace_update_support = moptions_.inplace_update_support; + saver.statistics = moptions_.statistics; + saver.env_ = env_; + saver.is_blob_index = is_blob_index; table_->Get(key, &saver, SaveValue); + + *seq = saver.seq; } // No change to value, since we have not yet found a Put/Delete if (!found_final_value && merge_in_progress) { - *s = Status::MergeInProgress(""); + *s = Status::MergeInProgress(); } PERF_COUNTER_ADD(get_from_memtable_count, 1); return found_final_value; @@ -479,33 +756,29 @@ void MemTable::Update(SequenceNumber seq, const char* entry = iter->key(); uint32_t key_length = 0; const char* key_ptr = GetVarint32Ptr(entry, entry + 5, &key_length); - if (comparator_.comparator.user_comparator()->Compare( - Slice(key_ptr, key_length - 8), lkey.user_key()) == 0) { + if (comparator_.comparator.user_comparator()->Equal( + Slice(key_ptr, key_length - 8), lkey.user_key())) { // Correct user key const uint64_t tag = DecodeFixed64(key_ptr + key_length - 8); - switch (static_cast(tag & 0xff)) { - case kTypeValue: { - Slice prev_value = GetLengthPrefixedSlice(key_ptr + key_length); - uint32_t prev_size = prev_value.size(); - uint32_t new_size = value.size(); - - // Update value, if new value size <= previous value size - if (new_size <= prev_size ) { - char* p = EncodeVarint32(const_cast(key_ptr) + key_length, - new_size); - WriteLock wl(GetLock(lkey.user_key())); - memcpy(p, value.data(), value.size()); - assert((unsigned)((p + value.size()) - entry) == - (unsigned)(VarintLength(key_length) + key_length + - VarintLength(value.size()) + value.size())); - return; - } + ValueType type; + SequenceNumber unused; + UnPackSequenceAndType(tag, &unused, &type); + if (type == kTypeValue) { + Slice prev_value = GetLengthPrefixedSlice(key_ptr + key_length); + uint32_t prev_size = static_cast(prev_value.size()); + uint32_t new_size = static_cast(value.size()); + + // Update value, if new value size <= previous value size + if (new_size <= prev_size) { + char* p = + EncodeVarint32(const_cast(key_ptr) + key_length, new_size); + WriteLock wl(GetLock(lkey.user_key())); + memcpy(p, value.data(), value.size()); + assert((unsigned)((p + value.size()) - entry) == + (unsigned)(VarintLength(key_length) + key_length + + VarintLength(value.size()) + value.size())); + return; } - default: - // If the latest value is kTypeDeletion, kTypeMerge or kTypeLogData - // we don't have enough space for update inplace - Add(seq, kTypeValue, key, value); - return; } } } @@ -516,8 +789,7 @@ void MemTable::Update(SequenceNumber seq, bool MemTable::UpdateCallback(SequenceNumber seq, const Slice& key, - const Slice& delta, - const Options& options) { + const Slice& delta) { LookupKey lkey(key, seq); Slice memkey = lkey.memtable_key(); @@ -538,22 +810,25 @@ bool MemTable::UpdateCallback(SequenceNumber seq, const char* entry = iter->key(); uint32_t key_length = 0; const char* key_ptr = GetVarint32Ptr(entry, entry + 5, &key_length); - if (comparator_.comparator.user_comparator()->Compare( - Slice(key_ptr, key_length - 8), lkey.user_key()) == 0) { + if (comparator_.comparator.user_comparator()->Equal( + Slice(key_ptr, key_length - 8), lkey.user_key())) { // Correct user key const uint64_t tag = DecodeFixed64(key_ptr + key_length - 8); - switch (static_cast(tag & 0xff)) { + ValueType type; + uint64_t unused; + UnPackSequenceAndType(tag, &unused, &type); + switch (type) { case kTypeValue: { Slice prev_value = GetLengthPrefixedSlice(key_ptr + key_length); - uint32_t prev_size = prev_value.size(); + uint32_t prev_size = static_cast(prev_value.size()); char* prev_buffer = const_cast(prev_value.data()); - uint32_t new_prev_size = prev_size; + uint32_t new_prev_size = prev_size; std::string str_value; WriteLock wl(GetLock(lkey.user_key())); - auto status = options.inplace_callback(prev_buffer, &new_prev_size, - delta, &str_value); + auto status = moptions_.inplace_callback(prev_buffer, &new_prev_size, + delta, &str_value); if (status == UpdateStatus::UPDATED_INPLACE) { // Value already updated by callback. assert(new_prev_size <= prev_size); @@ -566,17 +841,17 @@ bool MemTable::UpdateCallback(SequenceNumber seq, memcpy(p, prev_buffer, new_prev_size); } } - RecordTick(options.statistics.get(), NUMBER_KEYS_UPDATED); - should_flush_ = ShouldFlushNow(); + RecordTick(moptions_.statistics, NUMBER_KEYS_UPDATED); + UpdateFlushState(); return true; } else if (status == UpdateStatus::UPDATED) { Add(seq, kTypeValue, key, Slice(str_value)); - RecordTick(options.statistics.get(), NUMBER_KEYS_WRITTEN); - should_flush_ = ShouldFlushNow(); + RecordTick(moptions_.statistics, NUMBER_KEYS_WRITTEN); + UpdateFlushState(); return true; } else if (status == UpdateStatus::UPDATE_FAILED) { // No action required. Return. - should_flush_ = ShouldFlushNow(); + UpdateFlushState(); return true; } } @@ -606,13 +881,16 @@ size_t MemTable::CountSuccessiveMergeEntries(const LookupKey& key) { const char* entry = iter->key(); uint32_t key_length = 0; const char* iter_key_ptr = GetVarint32Ptr(entry, entry + 5, &key_length); - if (comparator_.comparator.user_comparator()->Compare( - Slice(iter_key_ptr, key_length - 8), key.user_key()) != 0) { + if (!comparator_.comparator.user_comparator()->Equal( + Slice(iter_key_ptr, key_length - 8), key.user_key())) { break; } const uint64_t tag = DecodeFixed64(iter_key_ptr + key_length - 8); - if (static_cast(tag & 0xff) != kTypeMerge) { + ValueType type; + uint64_t unused; + UnPackSequenceAndType(tag, &unused, &type); + if (type != kTypeMerge) { break; } @@ -631,4 +909,17 @@ void MemTableRep::Get(const LookupKey& k, void* callback_args, } } +void MemTable::RefLogContainingPrepSection(uint64_t log) { + assert(log > 0); + auto cur = min_prep_log_referenced_.load(); + while ((log < cur || cur == 0) && + !min_prep_log_referenced_.compare_exchange_strong(cur, log)) { + cur = min_prep_log_referenced_.load(); + } +} + +uint64_t MemTable::GetMinLogContainingPrepSection() { + return min_prep_log_referenced_.load(); +} + } // namespace rocksdb diff --git a/db/memtable.h b/db/memtable.h index 8bc281c6cf9..4f63818eee8 100644 --- a/db/memtable.h +++ b/db/memtable.h @@ -1,54 +1,118 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #pragma once -#include -#include +#include #include +#include +#include +#include +#include +#include #include "db/dbformat.h" -#include "db/skiplist.h" +#include "db/range_del_aggregator.h" #include "db/version_edit.h" +#include "monitoring/instrumented_mutex.h" +#include "options/cf_options.h" #include "rocksdb/db.h" +#include "rocksdb/env.h" #include "rocksdb/memtablerep.h" -#include "util/arena.h" +#include "util/allocator.h" +#include "util/concurrent_arena.h" #include "util/dynamic_bloom.h" +#include "util/hash.h" namespace rocksdb { -class Arena; class Mutex; class MemTableIterator; class MergeContext; +class InternalIterator; + +struct ImmutableMemTableOptions { + explicit ImmutableMemTableOptions(const ImmutableCFOptions& ioptions, + const MutableCFOptions& mutable_cf_options); + size_t arena_block_size; + uint32_t memtable_prefix_bloom_bits; + size_t memtable_huge_page_size; + bool inplace_update_support; + size_t inplace_update_num_locks; + UpdateStatus (*inplace_callback)(char* existing_value, + uint32_t* existing_value_size, + Slice delta_value, + std::string* merged_value); + size_t max_successive_merges; + Statistics* statistics; + MergeOperator* merge_operator; + Logger* info_log; +}; +// Batched counters to updated when inserting keys in one write batch. +// In post process of the write batch, these can be updated together. +// Only used in concurrent memtable insert case. +struct MemTablePostProcessInfo { + uint64_t data_size = 0; + uint64_t num_entries = 0; + uint64_t num_deletes = 0; +}; + +// Note: Many of the methods in this class have comments indicating that +// external synchromization is required as these methods are not thread-safe. +// It is up to higher layers of code to decide how to prevent concurrent +// invokation of these methods. This is usually done by acquiring either +// the db mutex or the single writer thread. +// +// Some of these methods are documented to only require external +// synchronization if this memtable is immutable. Calling MarkImmutable() is +// not sufficient to guarantee immutability. It is up to higher layers of +// code to determine if this MemTable can still be modified by other threads. +// Eg: The Superversion stores a pointer to the current MemTable (that can +// be modified) and a separate list of the MemTables that can no longer be +// written to (aka the 'immutable memtables'). class MemTable { public: struct KeyComparator : public MemTableRep::KeyComparator { const InternalKeyComparator comparator; explicit KeyComparator(const InternalKeyComparator& c) : comparator(c) { } virtual int operator()(const char* prefix_len_key1, - const char* prefix_len_key2) const; + const char* prefix_len_key2) const override; virtual int operator()(const char* prefix_len_key, const Slice& key) const override; }; // MemTables are reference counted. The initial reference count // is zero and the caller must call Ref() at least once. + // + // earliest_seq should be the current SequenceNumber in the db such that any + // key inserted into this memtable will have an equal or larger seq number. + // (When a db is first created, the earliest sequence number will be 0). + // If the earliest sequence number is not known, kMaxSequenceNumber may be + // used, but this may prevent some transactions from succeeding until the + // first key is inserted into the memtable. explicit MemTable(const InternalKeyComparator& comparator, - const Options& options); + const ImmutableCFOptions& ioptions, + const MutableCFOptions& mutable_cf_options, + WriteBufferManager* write_buffer_manager, + SequenceNumber earliest_seq, uint32_t column_family_id); + // Do not delete this MemTable unless Unref() indicates it not in use. ~MemTable(); // Increase reference count. + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable. void Ref() { ++refs_; } // Drop reference count. - // If the refcount goes to zero return this memtable, otherwise return null + // If the refcount goes to zero return this memtable, otherwise return null. + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable. MemTable* Unref() { --refs_; assert(refs_ >= 0); @@ -62,12 +126,23 @@ class MemTable { // data structure. // // REQUIRES: external synchronization to prevent simultaneous - // operations on the same MemTable. + // operations on the same MemTable (unless this Memtable is immutable). size_t ApproximateMemoryUsage(); // This method heuristically determines if the memtable should continue to // host more data. - bool ShouldFlush() const { return should_flush_; } + bool ShouldScheduleFlush() const { + return flush_state_.load(std::memory_order_relaxed) == FLUSH_REQUESTED; + } + + // Returns true if a flush should be scheduled and the caller should + // be the one to schedule it + bool MarkFlushScheduled() { + auto before = FLUSH_REQUESTED; + return flush_state_.compare_exchange_strong(before, FLUSH_SCHEDULED, + std::memory_order_relaxed, + std::memory_order_relaxed); + } // Return an iterator that yields the contents of the memtable. // @@ -81,15 +156,19 @@ class MemTable { // arena: If not null, the arena needs to be used to allocate the Iterator. // Calling ~Iterator of the iterator will destroy all the states but // those allocated in arena. - Iterator* NewIterator(const ReadOptions& options, - Arena* arena = nullptr); + InternalIterator* NewIterator(const ReadOptions& read_options, Arena* arena); + + InternalIterator* NewRangeTombstoneIterator(const ReadOptions& read_options); // Add an entry into memtable that maps key to value at the // specified sequence number and with the specified type. // Typically value will be empty if type==kTypeDeletion. - void Add(SequenceNumber seq, ValueType type, - const Slice& key, - const Slice& value); + // + // REQUIRES: if allow_concurrent = false, external synchronization to prevent + // simultaneous operations on the same MemTable. + void Add(SequenceNumber seq, ValueType type, const Slice& key, + const Slice& value, bool allow_concurrent = false, + MemTablePostProcessInfo* post_process_info = nullptr); // If memtable contains a value for key, store it in *value and return true. // If memtable contains a deletion for key, store a NotFound() error @@ -99,8 +178,23 @@ class MemTable { // prepend the current merge operand to *operands. // store MergeInProgress in s, and return false. // Else, return false. + // If any operation was found, its most recent sequence number + // will be stored in *seq on success (regardless of whether true/false is + // returned). Otherwise, *seq will be set to kMaxSequenceNumber. + // On success, *s may be set to OK, NotFound, or MergeInProgress. Any other + // status returned indicates a corruption or other unexpected error. bool Get(const LookupKey& key, std::string* value, Status* s, - MergeContext& merge_context, const Options& options); + MergeContext* merge_context, RangeDelAggregator* range_del_agg, + SequenceNumber* seq, const ReadOptions& read_opts, + bool* is_blob_index = nullptr); + + bool Get(const LookupKey& key, std::string* value, Status* s, + MergeContext* merge_context, RangeDelAggregator* range_del_agg, + const ReadOptions& read_opts, bool* is_blob_index = nullptr) { + SequenceNumber seq; + return Get(key, value, s, merge_context, range_del_agg, &seq, read_opts, + is_blob_index); + } // Attempts to update the new_value inplace, else does normal Add // Pseudocode @@ -109,11 +203,14 @@ class MemTable { // update inplace // else add(key, new_value) // else add(key, new_value) + // + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable. void Update(SequenceNumber seq, const Slice& key, const Slice& value); - // If prev_value for key exits, attempts to update it inplace. + // If prev_value for key exists, attempts to update it inplace. // else returns false // Pseudocode // if key exists in current memtable && prev_value is of type kTypeValue @@ -122,36 +219,118 @@ class MemTable { // update inplace // else add(key, new_value) // else return false + // + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable. bool UpdateCallback(SequenceNumber seq, const Slice& key, - const Slice& delta, - const Options& options); + const Slice& delta); // Returns the number of successive merge entries starting from the newest // entry for the key up to the last non-merge entry or last entry for the // key in the memtable. size_t CountSuccessiveMergeEntries(const LookupKey& key); + // Update counters and flush status after inserting a whole write batch + // Used in concurrent memtable inserts. + void BatchPostProcess(const MemTablePostProcessInfo& update_counters) { + num_entries_.fetch_add(update_counters.num_entries, + std::memory_order_relaxed); + data_size_.fetch_add(update_counters.data_size, std::memory_order_relaxed); + if (update_counters.num_deletes != 0) { + num_deletes_.fetch_add(update_counters.num_deletes, + std::memory_order_relaxed); + } + UpdateFlushState(); + } + // Get total number of entries in the mem table. - uint64_t GetNumEntries() const { return num_entries_; } + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable (unless this Memtable is immutable). + uint64_t num_entries() const { + return num_entries_.load(std::memory_order_relaxed); + } + + // Get total number of deletes in the mem table. + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable (unless this Memtable is immutable). + uint64_t num_deletes() const { + return num_deletes_.load(std::memory_order_relaxed); + } + + // Dynamically change the memtable's capacity. If set below the current usage, + // the next key added will trigger a flush. Can only increase size when + // memtable prefix bloom is disabled, since we can't easily allocate more + // space. + void UpdateWriteBufferSize(size_t new_write_buffer_size) { + if (prefix_bloom_ == nullptr || + new_write_buffer_size < write_buffer_size_) { + write_buffer_size_.store(new_write_buffer_size, + std::memory_order_relaxed); + } + } // Returns the edits area that is needed for flushing the memtable VersionEdit* GetEdits() { return &edit_; } + // Returns if there is no entry inserted to the mem table. + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable (unless this Memtable is immutable). + bool IsEmpty() const { return first_seqno_ == 0; } + // Returns the sequence number of the first element that was inserted - // into the memtable - SequenceNumber GetFirstSequenceNumber() { return first_seqno_; } + // into the memtable. + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable (unless this Memtable is immutable). + SequenceNumber GetFirstSequenceNumber() { + return first_seqno_.load(std::memory_order_relaxed); + } + + // Returns the sequence number that is guaranteed to be smaller than or equal + // to the sequence number of any key that could be inserted into this + // memtable. It can then be assumed that any write with a larger(or equal) + // sequence number will be present in this memtable or a later memtable. + // + // If the earliest sequence number could not be determined, + // kMaxSequenceNumber will be returned. + SequenceNumber GetEarliestSequenceNumber() { + return earliest_seqno_.load(std::memory_order_relaxed); + } + + // DB's latest sequence ID when the memtable is created. This number + // may be updated to a more recent one before any key is inserted. + SequenceNumber GetCreationSeq() const { return creation_seq_; } + + void SetCreationSeq(SequenceNumber sn) { creation_seq_ = sn; } // Returns the next active logfile number when this memtable is about to // be flushed to storage + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable. uint64_t GetNextLogNumber() { return mem_next_logfile_number_; } // Sets the next active logfile number when this memtable is about to // be flushed to storage + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable. void SetNextLogNumber(uint64_t num) { mem_next_logfile_number_ = num; } - // Notify the underlying storage that no more items will be added - void MarkImmutable() { table_->MarkReadOnly(); } + // if this memtable contains data from a committed + // two phase transaction we must take note of the + // log which contains that data so we can know + // when to relese that log + void RefLogContainingPrepSection(uint64_t log); + uint64_t GetMinLogContainingPrepSection(); + + // Notify the underlying storage that no more items will be added. + // REQUIRES: external synchronization to prevent simultaneous + // operations on the same MemTable. + // After MarkImmutable() is called, you should not attempt to + // write anything to this MemTable(). (Ie. do not call Add() or Update()). + void MarkImmutable() { + table_->MarkReadOnly(); + mem_tracker_.DoneAllocating(); + } // return true if the current MemTableRep supports merge operator. bool IsMergeOperatorSupported() const { @@ -159,7 +338,18 @@ class MemTable { } // return true if the current MemTableRep supports snapshots. - bool IsSnapshotSupported() const { return table_->IsSnapshotSupported(); } + // inplace update prevents snapshots, + bool IsSnapshotSupported() const { + return table_->IsSnapshotSupported() && !moptions_.inplace_update_support; + } + + struct MemTableStats { + uint64_t size; + uint64_t count; + }; + + MemTableStats ApproximateStats(const Slice& start_ikey, + const Slice& end_ikey); // Get the lock associated for the key port::RWMutex* GetLock(const Slice& key); @@ -168,24 +358,38 @@ class MemTable { return comparator_.comparator; } - const Arena& TEST_GetArena() const { return arena_; } + const ImmutableMemTableOptions* GetImmutableMemTableOptions() const { + return &moptions_; + } + + uint64_t ApproximateOldestKeyTime() const { + return oldest_key_time_.load(std::memory_order_relaxed); + } private: - // Dynamically check if we can add more incoming entries. - bool ShouldFlushNow() const; + enum FlushStateEnum { FLUSH_NOT_REQUESTED, FLUSH_REQUESTED, FLUSH_SCHEDULED }; friend class MemTableIterator; friend class MemTableBackwardIterator; friend class MemTableList; KeyComparator comparator_; + const ImmutableMemTableOptions moptions_; int refs_; const size_t kArenaBlockSize; - const size_t kWriteBufferSize; - Arena arena_; + AllocTracker mem_tracker_; + ConcurrentArena arena_; unique_ptr table_; + unique_ptr range_del_table_; + bool is_range_del_table_empty_; + + // Total data size of all data inserted + std::atomic data_size_; + std::atomic num_entries_; + std::atomic num_deletes_; - uint64_t num_entries_; + // Dynamically changeable memtable option + std::atomic write_buffer_size_; // These are used to manage memtable flushes to storage bool flush_in_progress_; // started the flush @@ -197,23 +401,51 @@ class MemTable { VersionEdit edit_; // The sequence number of the kv that was inserted first - SequenceNumber first_seqno_; + std::atomic first_seqno_; + + // The db sequence number at the time of creation or kMaxSequenceNumber + // if not set. + std::atomic earliest_seqno_; + + SequenceNumber creation_seq_; // The log files earlier than this number can be deleted. uint64_t mem_next_logfile_number_; + // the earliest log containing a prepared section + // which has been inserted into this memtable. + std::atomic min_prep_log_referenced_; + // rw locks for inplace updates std::vector locks_; - // No copying allowed - MemTable(const MemTable&); - void operator=(const MemTable&); - const SliceTransform* const prefix_extractor_; std::unique_ptr prefix_bloom_; - // a flag indicating if a memtable has met the criteria to flush - bool should_flush_; + std::atomic flush_state_; + + Env* env_; + + // Extract sequential insert prefixes. + const SliceTransform* insert_with_hint_prefix_extractor_; + + // Insert hints for each prefix. + std::unordered_map insert_hints_; + + // Timestamp of oldest key + std::atomic oldest_key_time_; + + // Returns a heuristic flush decision + bool ShouldFlushNow() const; + + // Updates flush_state_ using ShouldFlushNow() + void UpdateFlushState(); + + void UpdateOldestKeyTime(); + + // No copying allowed + MemTable(const MemTable&); + MemTable& operator=(const MemTable&); }; extern const char* EncodeKey(std::string* scratch, const Slice& target); diff --git a/db/memtable_list.cc b/db/memtable_list.cc index d3fc1356b27..5921a50b351 100644 --- a/db/memtable_list.cc +++ b/db/memtable_list.cc @@ -1,19 +1,27 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #include "db/memtable_list.h" +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include #include -#include "rocksdb/db.h" #include "db/memtable.h" #include "db/version_set.h" +#include "monitoring/thread_status_util.h" +#include "rocksdb/db.h" #include "rocksdb/env.h" #include "rocksdb/iterator.h" -#include "table/merger.h" +#include "table/merging_iterator.h" #include "util/coding.h" #include "util/log_buffer.h" +#include "util/sync_point.h" namespace rocksdb { @@ -21,18 +29,48 @@ class InternalKeyComparator; class Mutex; class VersionSet; -MemTableListVersion::MemTableListVersion(MemTableListVersion* old) { +void MemTableListVersion::AddMemTable(MemTable* m) { + memlist_.push_front(m); + *parent_memtable_list_memory_usage_ += m->ApproximateMemoryUsage(); +} + +void MemTableListVersion::UnrefMemTable(autovector* to_delete, + MemTable* m) { + if (m->Unref()) { + to_delete->push_back(m); + assert(*parent_memtable_list_memory_usage_ >= m->ApproximateMemoryUsage()); + *parent_memtable_list_memory_usage_ -= m->ApproximateMemoryUsage(); + } else { + } +} + +MemTableListVersion::MemTableListVersion( + size_t* parent_memtable_list_memory_usage, MemTableListVersion* old) + : max_write_buffer_number_to_maintain_( + old->max_write_buffer_number_to_maintain_), + parent_memtable_list_memory_usage_(parent_memtable_list_memory_usage) { if (old != nullptr) { memlist_ = old->memlist_; - size_ = old->size_; for (auto& m : memlist_) { m->Ref(); } + + memlist_history_ = old->memlist_history_; + for (auto& m : memlist_history_) { + m->Ref(); + } } } +MemTableListVersion::MemTableListVersion( + size_t* parent_memtable_list_memory_usage, + int max_write_buffer_number_to_maintain) + : max_write_buffer_number_to_maintain_(max_write_buffer_number_to_maintain), + parent_memtable_list_memory_usage_(parent_memtable_list_memory_usage) {} + void MemTableListVersion::Ref() { ++refs_; } +// called by superversion::clean() void MemTableListVersion::Unref(autovector* to_delete) { assert(refs_ >= 1); --refs_; @@ -41,41 +79,107 @@ void MemTableListVersion::Unref(autovector* to_delete) { // that refs_ will not be zero assert(to_delete != nullptr); for (const auto& m : memlist_) { - MemTable* x = m->Unref(); - if (x != nullptr) { - to_delete->push_back(x); - } + UnrefMemTable(to_delete, m); + } + for (const auto& m : memlist_history_) { + UnrefMemTable(to_delete, m); } delete this; } } -int MemTableListVersion::size() const { return size_; } +int MemTableList::NumNotFlushed() const { + int size = static_cast(current_->memlist_.size()); + assert(num_flush_not_started_ <= size); + return size; +} -// Returns the total number of memtables in the list -int MemTableList::size() const { - assert(num_flush_not_started_ <= current_->size_); - return current_->size_; +int MemTableList::NumFlushed() const { + return static_cast(current_->memlist_history_.size()); } // Search all the memtables starting from the most recent one. // Return the most recent value found, if any. // Operands stores the list of merge operations to apply, so far. bool MemTableListVersion::Get(const LookupKey& key, std::string* value, - Status* s, MergeContext& merge_context, - const Options& options) { - for (auto& memtable : memlist_) { - if (memtable->Get(key, value, s, merge_context, options)) { + Status* s, MergeContext* merge_context, + RangeDelAggregator* range_del_agg, + SequenceNumber* seq, const ReadOptions& read_opts, + bool* is_blob_index) { + return GetFromList(&memlist_, key, value, s, merge_context, range_del_agg, + seq, read_opts, is_blob_index); +} + +bool MemTableListVersion::GetFromHistory( + const LookupKey& key, std::string* value, Status* s, + MergeContext* merge_context, RangeDelAggregator* range_del_agg, + SequenceNumber* seq, const ReadOptions& read_opts, bool* is_blob_index) { + return GetFromList(&memlist_history_, key, value, s, merge_context, + range_del_agg, seq, read_opts, is_blob_index); +} + +bool MemTableListVersion::GetFromList( + std::list* list, const LookupKey& key, std::string* value, + Status* s, MergeContext* merge_context, RangeDelAggregator* range_del_agg, + SequenceNumber* seq, const ReadOptions& read_opts, bool* is_blob_index) { + *seq = kMaxSequenceNumber; + + for (auto& memtable : *list) { + SequenceNumber current_seq = kMaxSequenceNumber; + + bool done = memtable->Get(key, value, s, merge_context, range_del_agg, + ¤t_seq, read_opts, is_blob_index); + if (*seq == kMaxSequenceNumber) { + // Store the most recent sequence number of any operation on this key. + // Since we only care about the most recent change, we only need to + // return the first operation found when searching memtables in + // reverse-chronological order. + *seq = current_seq; + } + + if (done) { + assert(*seq != kMaxSequenceNumber); return true; } + if (!done && !s->ok() && !s->IsMergeInProgress() && !s->IsNotFound()) { + return false; + } } return false; } -void MemTableListVersion::AddIterators(const ReadOptions& options, - std::vector* iterator_list) { +Status MemTableListVersion::AddRangeTombstoneIterators( + const ReadOptions& read_opts, Arena* arena, + RangeDelAggregator* range_del_agg) { + assert(range_del_agg != nullptr); for (auto& m : memlist_) { - iterator_list->push_back(m->NewIterator(options)); + std::unique_ptr range_del_iter( + m->NewRangeTombstoneIterator(read_opts)); + Status s = range_del_agg->AddTombstones(std::move(range_del_iter)); + if (!s.ok()) { + return s; + } + } + return Status::OK(); +} + +Status MemTableListVersion::AddRangeTombstoneIterators( + const ReadOptions& read_opts, + std::vector* range_del_iters) { + for (auto& m : memlist_) { + auto* range_del_iter = m->NewRangeTombstoneIterator(read_opts); + if (range_del_iter != nullptr) { + range_del_iters->push_back(range_del_iter); + } + } + return Status::OK(); +} + +void MemTableListVersion::AddIterators( + const ReadOptions& options, std::vector* iterator_list, + Arena* arena) { + for (auto& m : memlist_) { + iterator_list->push_back(m->NewIterator(options, arena)); } } @@ -90,23 +194,73 @@ void MemTableListVersion::AddIterators( uint64_t MemTableListVersion::GetTotalNumEntries() const { uint64_t total_num = 0; for (auto& m : memlist_) { - total_num += m->GetNumEntries(); + total_num += m->num_entries(); + } + return total_num; +} + +MemTable::MemTableStats MemTableListVersion::ApproximateStats( + const Slice& start_ikey, const Slice& end_ikey) { + MemTable::MemTableStats total_stats = {0, 0}; + for (auto& m : memlist_) { + auto mStats = m->ApproximateStats(start_ikey, end_ikey); + total_stats.size += mStats.size; + total_stats.count += mStats.count; + } + return total_stats; +} + +uint64_t MemTableListVersion::GetTotalNumDeletes() const { + uint64_t total_num = 0; + for (auto& m : memlist_) { + total_num += m->num_deletes(); } return total_num; } +SequenceNumber MemTableListVersion::GetEarliestSequenceNumber( + bool include_history) const { + if (include_history && !memlist_history_.empty()) { + return memlist_history_.back()->GetEarliestSequenceNumber(); + } else if (!memlist_.empty()) { + return memlist_.back()->GetEarliestSequenceNumber(); + } else { + return kMaxSequenceNumber; + } +} + // caller is responsible for referencing m -void MemTableListVersion::Add(MemTable* m) { +void MemTableListVersion::Add(MemTable* m, autovector* to_delete) { assert(refs_ == 1); // only when refs_ == 1 is MemTableListVersion mutable - memlist_.push_front(m); - ++size_; + AddMemTable(m); + + TrimHistory(to_delete); } -// caller is responsible for unreferencing m -void MemTableListVersion::Remove(MemTable* m) { +// Removes m from list of memtables not flushed. Caller should NOT Unref m. +void MemTableListVersion::Remove(MemTable* m, + autovector* to_delete) { assert(refs_ == 1); // only when refs_ == 1 is MemTableListVersion mutable memlist_.remove(m); - --size_; + + if (max_write_buffer_number_to_maintain_ > 0) { + memlist_history_.push_front(m); + TrimHistory(to_delete); + } else { + UnrefMemTable(to_delete, m); + } +} + +// Make sure we don't use up too much space in history +void MemTableListVersion::TrimHistory(autovector* to_delete) { + while (memlist_.size() + memlist_history_.size() > + static_cast(max_write_buffer_number_to_maintain_) && + !memlist_history_.empty()) { + MemTable* x = memlist_history_.back(); + memlist_history_.pop_back(); + + UnrefMemTable(to_delete, x); + } } // Returns true if there is at least one memtable on which flush has @@ -114,7 +268,7 @@ void MemTableListVersion::Remove(MemTable* m) { bool MemTableList::IsFlushPending() const { if ((flush_requested_ && num_flush_not_started_ >= 1) || (num_flush_not_started_ >= min_write_buffer_number_to_merge_)) { - assert(imm_flush_needed.NoBarrier_Load() != nullptr); + assert(imm_flush_needed.load(std::memory_order_relaxed)); return true; } return false; @@ -122,6 +276,8 @@ bool MemTableList::IsFlushPending() const { // Returns the memtables that need to be flushed. void MemTableList::PickMemtablesToFlush(autovector* ret) { + AutoThreadOperationStageUpdater stage_updater( + ThreadStatus::STAGE_PICK_MEMTABLES_TO_FLUSH); const auto& memlist = current_->memlist_; for (auto it = memlist.rbegin(); it != memlist.rend(); ++it) { MemTable* m = *it; @@ -129,7 +285,7 @@ void MemTableList::PickMemtablesToFlush(autovector* ret) { assert(!m->flush_completed_); num_flush_not_started_--; if (num_flush_not_started_ == 0) { - imm_flush_needed.Release_Store(nullptr); + imm_flush_needed.store(false, std::memory_order_release); } m->flush_in_progress_ = true; // flushing will start very soon ret->push_back(m); @@ -139,12 +295,13 @@ void MemTableList::PickMemtablesToFlush(autovector* ret) { } void MemTableList::RollbackMemtableFlush(const autovector& mems, - uint64_t file_number, - FileNumToPathIdMap* pending_outputs) { + uint64_t file_number) { + AutoThreadOperationStageUpdater stage_updater( + ThreadStatus::STAGE_MEMTABLE_ROLLBACK); assert(!mems.empty()); // If the flush was not successful, then just reset state. - // Maybe a suceeding attempt to flush will be successful. + // Maybe a succeeding attempt to flush will be successful. for (MemTable* m : mems) { assert(m->flush_in_progress_); assert(m->file_number_ == 0); @@ -154,19 +311,20 @@ void MemTableList::RollbackMemtableFlush(const autovector& mems, m->edit_.Clear(); num_flush_not_started_++; } - pending_outputs->erase(file_number); - imm_flush_needed.Release_Store(reinterpret_cast(1)); + imm_flush_needed.store(true, std::memory_order_release); } // Record a successful flush in the manifest file Status MemTableList::InstallMemtableFlushResults( - ColumnFamilyData* cfd, const autovector& mems, VersionSet* vset, - port::Mutex* mu, Logger* info_log, uint64_t file_number, - FileNumToPathIdMap* pending_outputs, autovector* to_delete, + ColumnFamilyData* cfd, const MutableCFOptions& mutable_cf_options, + const autovector& mems, VersionSet* vset, InstrumentedMutex* mu, + uint64_t file_number, autovector* to_delete, Directory* db_directory, LogBuffer* log_buffer) { + AutoThreadOperationStageUpdater stage_updater( + ThreadStatus::STAGE_MEMTABLE_INSTALL_FLUSH_RESULTS); mu->AssertHeld(); - // flush was sucessful + // flush was successful for (size_t i = 0; i < mems.size(); ++i) { // All the edits are associated with the first memtable of this batch. assert(i == 0 || mems[i]->GetEdits()->NumEntries() == 0); @@ -175,100 +333,122 @@ Status MemTableList::InstallMemtableFlushResults( mems[i]->file_number_ = file_number; } - // if some other thread is already commiting, then return + // if some other thread is already committing, then return Status s; if (commit_in_progress_) { + TEST_SYNC_POINT("MemTableList::InstallMemtableFlushResults:InProgress"); return s; } // Only a single thread can be executing this piece of code commit_in_progress_ = true; - // scan all memtables from the earliest, and commit those - // (in that order) that have finished flushing. Memetables - // are always committed in the order that they were created. - while (!current_->memlist_.empty() && s.ok()) { - MemTable* m = current_->memlist_.back(); // get the last element - if (!m->flush_completed_) { + // Retry until all completed flushes are committed. New flushes can finish + // while the current thread is writing manifest where mutex is released. + while (s.ok()) { + auto& memlist = current_->memlist_; + if (memlist.empty() || !memlist.back()->flush_completed_) { break; } + // scan all memtables from the earliest, and commit those + // (in that order) that have finished flushing. Memetables + // are always committed in the order that they were created. + uint64_t batch_file_number = 0; + size_t batch_count = 0; + autovector edit_list; + // enumerate from the last (earliest) element to see how many batch finished + for (auto it = memlist.rbegin(); it != memlist.rend(); ++it) { + MemTable* m = *it; + if (!m->flush_completed_) { + break; + } + if (it == memlist.rbegin() || batch_file_number != m->file_number_) { + batch_file_number = m->file_number_; + ROCKS_LOG_BUFFER(log_buffer, + "[%s] Level-0 commit table #%" PRIu64 " started", + cfd->GetName().c_str(), m->file_number_); + edit_list.push_back(&m->edit_); + } + batch_count++; + } - LogToBuffer(log_buffer, "[%s] Level-0 commit table #%lu started", - cfd->GetName().c_str(), (unsigned long)m->file_number_); - - // this can release and reacquire the mutex. - s = vset->LogAndApply(cfd, &m->edit_, mu, db_directory); - - // we will be changing the version in the next code path, - // so we better create a new one, since versions are immutable - InstallNewVersion(); - - // All the later memtables that have the same filenum - // are part of the same batch. They can be committed now. - uint64_t mem_id = 1; // how many memtables has been flushed. - do { - if (s.ok()) { // commit new state - LogToBuffer(log_buffer, - "[%s] Level-0 commit table #%lu: memtable #%lu done", - cfd->GetName().c_str(), (unsigned long)m->file_number_, - (unsigned long)mem_id); - current_->Remove(m); - assert(m->file_number_ > 0); - - // pending_outputs can be cleared only after the newly created file - // has been written to a committed version so that other concurrently - // executing compaction threads do not mistakenly assume that this - // file is not live. - pending_outputs->erase(m->file_number_); - if (m->Unref() != nullptr) { - to_delete->push_back(m); + if (batch_count > 0) { + // this can release and reacquire the mutex. + s = vset->LogAndApply(cfd, mutable_cf_options, edit_list, mu, + db_directory); + + // we will be changing the version in the next code path, + // so we better create a new one, since versions are immutable + InstallNewVersion(); + + // All the later memtables that have the same filenum + // are part of the same batch. They can be committed now. + uint64_t mem_id = 1; // how many memtables have been flushed. + if (s.ok()) { // commit new state + while (batch_count-- > 0) { + MemTable* m = current_->memlist_.back(); + ROCKS_LOG_BUFFER(log_buffer, "[%s] Level-0 commit table #%" PRIu64 + ": memtable #%" PRIu64 " done", + cfd->GetName().c_str(), m->file_number_, mem_id); + assert(m->file_number_ > 0); + current_->Remove(m, to_delete); + ++mem_id; } } else { - //commit failed. setup state so that we can flush again. - Log(info_log, - "Level-0 commit table #%lu: memtable #%lu failed", - (unsigned long)m->file_number_, - (unsigned long)mem_id); - m->flush_completed_ = false; - m->flush_in_progress_ = false; - m->edit_.Clear(); - num_flush_not_started_++; - pending_outputs->erase(m->file_number_); - m->file_number_ = 0; - imm_flush_needed.Release_Store((void *)1); + for (auto it = current_->memlist_.rbegin(); batch_count-- > 0; it++) { + MemTable* m = *it; + // commit failed. setup state so that we can flush again. + ROCKS_LOG_BUFFER(log_buffer, "Level-0 commit table #%" PRIu64 + ": memtable #%" PRIu64 " failed", + m->file_number_, mem_id); + m->flush_completed_ = false; + m->flush_in_progress_ = false; + m->edit_.Clear(); + num_flush_not_started_++; + m->file_number_ = 0; + imm_flush_needed.store(true, std::memory_order_release); + ++mem_id; + } } - ++mem_id; - } while (!current_->memlist_.empty() && (m = current_->memlist_.back()) && - m->file_number_ == file_number); + } } commit_in_progress_ = false; return s; } // New memtables are inserted at the front of the list. -void MemTableList::Add(MemTable* m) { - assert(current_->size_ >= num_flush_not_started_); +void MemTableList::Add(MemTable* m, autovector* to_delete) { + assert(static_cast(current_->memlist_.size()) >= num_flush_not_started_); InstallNewVersion(); // this method is used to move mutable memtable into an immutable list. // since mutable memtable is already refcounted by the DBImpl, // and when moving to the imutable list we don't unref it, // we don't have to ref the memtable here. we just take over the // reference from the DBImpl. - current_->Add(m); + current_->Add(m, to_delete); m->MarkImmutable(); num_flush_not_started_++; if (num_flush_not_started_ == 1) { - imm_flush_needed.Release_Store((void *)1); + imm_flush_needed.store(true, std::memory_order_release); } } // Returns an estimate of the number of bytes of data in use. -size_t MemTableList::ApproximateMemoryUsage() { - size_t size = 0; +size_t MemTableList::ApproximateUnflushedMemTablesMemoryUsage() { + size_t total_size = 0; for (auto& memtable : current_->memlist_) { - size += memtable->ApproximateMemoryUsage(); + total_size += memtable->ApproximateMemoryUsage(); } - return size; + return total_size; +} + +size_t MemTableList::ApproximateMemoryUsage() { return current_memory_usage_; } + +uint64_t MemTableList::ApproximateOldestKeyTime() const { + if (!current_->memlist_.empty()) { + return current_->memlist_.back()->ApproximateOldestKeyTime(); + } + return std::numeric_limits::max(); } void MemTableList::InstallNewVersion() { @@ -277,10 +457,30 @@ void MemTableList::InstallNewVersion() { } else { // somebody else holds the current version, we need to create new one MemTableListVersion* version = current_; - current_ = new MemTableListVersion(current_); + current_ = new MemTableListVersion(¤t_memory_usage_, current_); current_->Ref(); version->Unref(); } } +uint64_t MemTableList::GetMinLogContainingPrepSection() { + uint64_t min_log = 0; + + for (auto& m : current_->memlist_) { + // this mem has been flushed it no longer + // needs to hold on the its prep section + if (m->flush_completed_) { + continue; + } + + auto log = m->GetMinLogContainingPrepSection(); + + if (log > 0 && (min_log == 0 || log < min_log)) { + min_log = log; + } + } + + return min_log; +} + } // namespace rocksdb diff --git a/db/memtable_list.h b/db/memtable_list.h index f4923e831a4..69038af5004 100644 --- a/db/memtable_list.h +++ b/db/memtable_list.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #pragma once @@ -10,62 +10,139 @@ #include #include #include -#include "rocksdb/db.h" -#include "rocksdb/options.h" -#include "rocksdb/iterator.h" #include "db/dbformat.h" -#include "db/filename.h" -#include "db/skiplist.h" #include "db/memtable.h" +#include "db/range_del_aggregator.h" +#include "monitoring/instrumented_mutex.h" #include "rocksdb/db.h" #include "rocksdb/iterator.h" #include "rocksdb/options.h" +#include "rocksdb/types.h" #include "util/autovector.h" +#include "util/filename.h" #include "util/log_buffer.h" namespace rocksdb { class ColumnFamilyData; class InternalKeyComparator; -class Mutex; +class InstrumentedMutex; class MergeIteratorBuilder; // keeps a list of immutable memtables in a vector. the list is immutable // if refcount is bigger than one. It is used as a state for Get() and // Iterator code paths +// +// This class is not thread-safe. External synchronization is required +// (such as holding the db mutex or being on the write thread). class MemTableListVersion { public: - explicit MemTableListVersion(MemTableListVersion* old = nullptr); + explicit MemTableListVersion(size_t* parent_memtable_list_memory_usage, + MemTableListVersion* old = nullptr); + explicit MemTableListVersion(size_t* parent_memtable_list_memory_usage, + int max_write_buffer_number_to_maintain); void Ref(); void Unref(autovector* to_delete = nullptr); - int size() const; - // Search all the memtables starting from the most recent one. // Return the most recent value found, if any. + // + // If any operation was found for this key, its most recent sequence number + // will be stored in *seq on success (regardless of whether true/false is + // returned). Otherwise, *seq will be set to kMaxSequenceNumber. + bool Get(const LookupKey& key, std::string* value, Status* s, + MergeContext* merge_context, RangeDelAggregator* range_del_agg, + SequenceNumber* seq, const ReadOptions& read_opts, + bool* is_blob_index = nullptr); + bool Get(const LookupKey& key, std::string* value, Status* s, - MergeContext& merge_context, const Options& options); + MergeContext* merge_context, RangeDelAggregator* range_del_agg, + const ReadOptions& read_opts, bool* is_blob_index = nullptr) { + SequenceNumber seq; + return Get(key, value, s, merge_context, range_del_agg, &seq, read_opts, + is_blob_index); + } + + // Similar to Get(), but searches the Memtable history of memtables that + // have already been flushed. Should only be used from in-memory only + // queries (such as Transaction validation) as the history may contain + // writes that are also present in the SST files. + bool GetFromHistory(const LookupKey& key, std::string* value, Status* s, + MergeContext* merge_context, + RangeDelAggregator* range_del_agg, SequenceNumber* seq, + const ReadOptions& read_opts, + bool* is_blob_index = nullptr); + bool GetFromHistory(const LookupKey& key, std::string* value, Status* s, + MergeContext* merge_context, + RangeDelAggregator* range_del_agg, + const ReadOptions& read_opts, + bool* is_blob_index = nullptr) { + SequenceNumber seq; + return GetFromHistory(key, value, s, merge_context, range_del_agg, &seq, + read_opts, is_blob_index); + } + + Status AddRangeTombstoneIterators(const ReadOptions& read_opts, Arena* arena, + RangeDelAggregator* range_del_agg); + Status AddRangeTombstoneIterators( + const ReadOptions& read_opts, + std::vector* range_del_iters); void AddIterators(const ReadOptions& options, - std::vector* iterator_list); + std::vector* iterator_list, + Arena* arena); void AddIterators(const ReadOptions& options, MergeIteratorBuilder* merge_iter_builder); uint64_t GetTotalNumEntries() const; + uint64_t GetTotalNumDeletes() const; + + MemTable::MemTableStats ApproximateStats(const Slice& start_ikey, + const Slice& end_ikey); + + // Returns the value of MemTable::GetEarliestSequenceNumber() on the most + // recent MemTable in this list or kMaxSequenceNumber if the list is empty. + // If include_history=true, will also search Memtables in MemTableList + // History. + SequenceNumber GetEarliestSequenceNumber(bool include_history = false) const; + private: - // REQUIRE: m is mutable memtable - void Add(MemTable* m); - // REQUIRE: m is mutable memtable - void Remove(MemTable* m); + // REQUIRE: m is an immutable memtable + void Add(MemTable* m, autovector* to_delete); + // REQUIRE: m is an immutable memtable + void Remove(MemTable* m, autovector* to_delete); + + void TrimHistory(autovector* to_delete); + + bool GetFromList(std::list* list, const LookupKey& key, + std::string* value, Status* s, MergeContext* merge_context, + RangeDelAggregator* range_del_agg, SequenceNumber* seq, + const ReadOptions& read_opts, bool* is_blob_index = nullptr); + + void AddMemTable(MemTable* m); + + void UnrefMemTable(autovector* to_delete, MemTable* m); friend class MemTableList; + + // Immutable MemTables that have not yet been flushed. std::list memlist_; - int size_ = 0; + + // MemTables that have already been flushed + // (used during Transaction validation) + std::list memlist_history_; + + // Maximum number of MemTables to keep in memory (including both flushed + // and not-yet-flushed tables). + const int max_write_buffer_number_to_maintain_; + int refs_ = 0; + + size_t* parent_memtable_list_memory_usage_; }; // This class stores references to all the immutable memtables. @@ -74,28 +151,44 @@ class MemTableListVersion { // flushes can occur concurrently. However, they are 'committed' // to the manifest in FIFO order to maintain correctness and // recoverability from a crash. +// +// +// Other than imm_flush_needed, this class is not thread-safe and requires +// external synchronization (such as holding the db mutex or being on the +// write thread.) class MemTableList { public: // A list of memtables. - explicit MemTableList(int min_write_buffer_number_to_merge) - : min_write_buffer_number_to_merge_(min_write_buffer_number_to_merge), - current_(new MemTableListVersion()), + explicit MemTableList(int min_write_buffer_number_to_merge, + int max_write_buffer_number_to_maintain) + : imm_flush_needed(false), + min_write_buffer_number_to_merge_(min_write_buffer_number_to_merge), + current_(new MemTableListVersion(¤t_memory_usage_, + max_write_buffer_number_to_maintain)), num_flush_not_started_(0), commit_in_progress_(false), flush_requested_(false) { - imm_flush_needed.Release_Store(nullptr); current_->Ref(); + current_memory_usage_ = 0; } + + // Should not delete MemTableList without making sure MemTableList::current() + // is Unref()'d. ~MemTableList() {} MemTableListVersion* current() { return current_; } // so that background threads can detect non-nullptr pointer to // determine whether there is anything more to start flushing. - port::AtomicPointer imm_flush_needed; + std::atomic imm_flush_needed; - // Returns the total number of memtables in the list - int size() const; + // Returns the total number of memtables in the list that haven't yet + // been flushed and logged. + int NumNotFlushed() const; + + // Returns total number of memtables in the list that have been + // completely flushed and logged. + int NumFlushed() const; // Returns true if there is at least one memtable on which flush has // not yet started. @@ -108,35 +201,51 @@ class MemTableList { // Reset status of the given memtable list back to pending state so that // they can get picked up again on the next round of flush. void RollbackMemtableFlush(const autovector& mems, - uint64_t file_number, - FileNumToPathIdMap* pending_outputs); + uint64_t file_number); // Commit a successful flush in the manifest file Status InstallMemtableFlushResults( - ColumnFamilyData* cfd, const autovector& m, VersionSet* vset, - port::Mutex* mu, Logger* info_log, uint64_t file_number, - FileNumToPathIdMap* pending_outputs, autovector* to_delete, + ColumnFamilyData* cfd, const MutableCFOptions& mutable_cf_options, + const autovector& m, VersionSet* vset, InstrumentedMutex* mu, + uint64_t file_number, autovector* to_delete, Directory* db_directory, LogBuffer* log_buffer); // New memtables are inserted at the front of the list. // Takes ownership of the referenced held on *m by the caller of Add(). - void Add(MemTable* m); + void Add(MemTable* m, autovector* to_delete); // Returns an estimate of the number of bytes of data in use. size_t ApproximateMemoryUsage(); - // Request a flush of all existing memtables to storage + // Returns an estimate of the number of bytes of data used by + // the unflushed mem-tables. + size_t ApproximateUnflushedMemTablesMemoryUsage(); + + // Returns an estimate of the timestamp of the earliest key. + uint64_t ApproximateOldestKeyTime() const; + + // Request a flush of all existing memtables to storage. This will + // cause future calls to IsFlushPending() to return true if this list is + // non-empty (regardless of the min_write_buffer_number_to_merge + // parameter). This flush request will persist until the next time + // PickMemtablesToFlush() is called. void FlushRequested() { flush_requested_ = true; } + bool HasFlushRequested() { return flush_requested_; } + // Copying allowed // MemTableList(const MemTableList&); // void operator=(const MemTableList&); + size_t* current_memory_usage() { return ¤t_memory_usage_; } + + uint64_t GetMinLogContainingPrepSection(); + private: // DB mutex held void InstallNewVersion(); - int min_write_buffer_number_to_merge_; + const int min_write_buffer_number_to_merge_; MemTableListVersion* current_; @@ -149,6 +258,8 @@ class MemTableList { // Requested a flush of all memtables to storage bool flush_requested_; + // The current memory usage. + size_t current_memory_usage_; }; } // namespace rocksdb diff --git a/db/memtable_list_test.cc b/db/memtable_list_test.cc new file mode 100644 index 00000000000..30e51666372 --- /dev/null +++ b/db/memtable_list_test.cc @@ -0,0 +1,615 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/memtable_list.h" +#include +#include +#include +#include "db/merge_context.h" +#include "db/range_del_aggregator.h" +#include "db/version_set.h" +#include "db/write_controller.h" +#include "rocksdb/db.h" +#include "rocksdb/status.h" +#include "rocksdb/write_buffer_manager.h" +#include "util/string_util.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +class MemTableListTest : public testing::Test { + public: + std::string dbname; + DB* db; + Options options; + + MemTableListTest() : db(nullptr) { + dbname = test::TmpDir() + "/memtable_list_test"; + } + + // Create a test db if not yet created + void CreateDB() { + if (db == nullptr) { + options.create_if_missing = true; + DestroyDB(dbname, options); + Status s = DB::Open(options, dbname, &db); + EXPECT_OK(s); + } + } + + ~MemTableListTest() { + if (db) { + delete db; + DestroyDB(dbname, options); + } + } + + // Calls MemTableList::InstallMemtableFlushResults() and sets up all + // structures needed to call this function. + Status Mock_InstallMemtableFlushResults( + MemTableList* list, const MutableCFOptions& mutable_cf_options, + const autovector& m, autovector* to_delete) { + // Create a mock Logger + test::NullLogger logger; + LogBuffer log_buffer(DEBUG_LEVEL, &logger); + + // Create a mock VersionSet + DBOptions db_options; + ImmutableDBOptions immutable_db_options(db_options); + EnvOptions env_options; + shared_ptr table_cache(NewLRUCache(50000, 16)); + WriteBufferManager write_buffer_manager(db_options.db_write_buffer_size); + WriteController write_controller(10000000u); + + CreateDB(); + VersionSet versions(dbname, &immutable_db_options, env_options, + table_cache.get(), &write_buffer_manager, + &write_controller); + + // Create mock default ColumnFamilyData + ColumnFamilyOptions cf_options; + std::vector column_families; + column_families.emplace_back(kDefaultColumnFamilyName, cf_options); + EXPECT_OK(versions.Recover(column_families, false)); + + auto column_family_set = versions.GetColumnFamilySet(); + auto cfd = column_family_set->GetColumnFamily(0); + EXPECT_TRUE(cfd != nullptr); + + // Create dummy mutex. + InstrumentedMutex mutex; + InstrumentedMutexLock l(&mutex); + + return list->InstallMemtableFlushResults(cfd, mutable_cf_options, m, + &versions, &mutex, 1, to_delete, + nullptr, &log_buffer); + } +}; + +TEST_F(MemTableListTest, Empty) { + // Create an empty MemTableList and validate basic functions. + MemTableList list(1, 0); + + ASSERT_EQ(0, list.NumNotFlushed()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + ASSERT_FALSE(list.IsFlushPending()); + + autovector mems; + list.PickMemtablesToFlush(&mems); + ASSERT_EQ(0, mems.size()); + + autovector to_delete; + list.current()->Unref(&to_delete); + ASSERT_EQ(0, to_delete.size()); +} + +TEST_F(MemTableListTest, GetTest) { + // Create MemTableList + int min_write_buffer_number_to_merge = 2; + int max_write_buffer_number_to_maintain = 0; + MemTableList list(min_write_buffer_number_to_merge, + max_write_buffer_number_to_maintain); + + SequenceNumber seq = 1; + std::string value; + Status s; + MergeContext merge_context; + InternalKeyComparator ikey_cmp(options.comparator); + RangeDelAggregator range_del_agg(ikey_cmp, {} /* snapshots */); + autovector to_delete; + + LookupKey lkey("key1", seq); + bool found = list.current()->Get(lkey, &value, &s, &merge_context, + &range_del_agg, ReadOptions()); + ASSERT_FALSE(found); + + // Create a MemTable + InternalKeyComparator cmp(BytewiseComparator()); + auto factory = std::make_shared(); + options.memtable_factory = factory; + ImmutableCFOptions ioptions(options); + + WriteBufferManager wb(options.db_write_buffer_size); + MemTable* mem = new MemTable(cmp, ioptions, MutableCFOptions(options), &wb, + kMaxSequenceNumber, 0 /* column_family_id */); + mem->Ref(); + + // Write some keys to this memtable. + mem->Add(++seq, kTypeDeletion, "key1", ""); + mem->Add(++seq, kTypeValue, "key2", "value2"); + mem->Add(++seq, kTypeValue, "key1", "value1"); + mem->Add(++seq, kTypeValue, "key2", "value2.2"); + + // Fetch the newly written keys + merge_context.Clear(); + found = mem->Get(LookupKey("key1", seq), &value, &s, &merge_context, + &range_del_agg, ReadOptions()); + ASSERT_TRUE(s.ok() && found); + ASSERT_EQ(value, "value1"); + + merge_context.Clear(); + found = mem->Get(LookupKey("key1", 2), &value, &s, &merge_context, + &range_del_agg, ReadOptions()); + // MemTable found out that this key is *not* found (at this sequence#) + ASSERT_TRUE(found && s.IsNotFound()); + + merge_context.Clear(); + found = mem->Get(LookupKey("key2", seq), &value, &s, &merge_context, + &range_del_agg, ReadOptions()); + ASSERT_TRUE(s.ok() && found); + ASSERT_EQ(value, "value2.2"); + + ASSERT_EQ(4, mem->num_entries()); + ASSERT_EQ(1, mem->num_deletes()); + + // Add memtable to list + list.Add(mem, &to_delete); + + SequenceNumber saved_seq = seq; + + // Create another memtable and write some keys to it + WriteBufferManager wb2(options.db_write_buffer_size); + MemTable* mem2 = new MemTable(cmp, ioptions, MutableCFOptions(options), &wb2, + kMaxSequenceNumber, 0 /* column_family_id */); + mem2->Ref(); + + mem2->Add(++seq, kTypeDeletion, "key1", ""); + mem2->Add(++seq, kTypeValue, "key2", "value2.3"); + + // Add second memtable to list + list.Add(mem2, &to_delete); + + // Fetch keys via MemTableList + merge_context.Clear(); + found = list.current()->Get(LookupKey("key1", seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_TRUE(found && s.IsNotFound()); + + merge_context.Clear(); + found = list.current()->Get(LookupKey("key1", saved_seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_TRUE(s.ok() && found); + ASSERT_EQ("value1", value); + + merge_context.Clear(); + found = list.current()->Get(LookupKey("key2", seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_TRUE(s.ok() && found); + ASSERT_EQ(value, "value2.3"); + + merge_context.Clear(); + found = list.current()->Get(LookupKey("key2", 1), &value, &s, &merge_context, + &range_del_agg, ReadOptions()); + ASSERT_FALSE(found); + + ASSERT_EQ(2, list.NumNotFlushed()); + + list.current()->Unref(&to_delete); + for (MemTable* m : to_delete) { + delete m; + } +} + +TEST_F(MemTableListTest, GetFromHistoryTest) { + // Create MemTableList + int min_write_buffer_number_to_merge = 2; + int max_write_buffer_number_to_maintain = 2; + MemTableList list(min_write_buffer_number_to_merge, + max_write_buffer_number_to_maintain); + + SequenceNumber seq = 1; + std::string value; + Status s; + MergeContext merge_context; + InternalKeyComparator ikey_cmp(options.comparator); + RangeDelAggregator range_del_agg(ikey_cmp, {} /* snapshots */); + autovector to_delete; + + LookupKey lkey("key1", seq); + bool found = list.current()->Get(lkey, &value, &s, &merge_context, + &range_del_agg, ReadOptions()); + ASSERT_FALSE(found); + + // Create a MemTable + InternalKeyComparator cmp(BytewiseComparator()); + auto factory = std::make_shared(); + options.memtable_factory = factory; + ImmutableCFOptions ioptions(options); + + WriteBufferManager wb(options.db_write_buffer_size); + MemTable* mem = new MemTable(cmp, ioptions, MutableCFOptions(options), &wb, + kMaxSequenceNumber, 0 /* column_family_id */); + mem->Ref(); + + // Write some keys to this memtable. + mem->Add(++seq, kTypeDeletion, "key1", ""); + mem->Add(++seq, kTypeValue, "key2", "value2"); + mem->Add(++seq, kTypeValue, "key2", "value2.2"); + + // Fetch the newly written keys + merge_context.Clear(); + found = mem->Get(LookupKey("key1", seq), &value, &s, &merge_context, + &range_del_agg, ReadOptions()); + // MemTable found out that this key is *not* found (at this sequence#) + ASSERT_TRUE(found && s.IsNotFound()); + + merge_context.Clear(); + found = mem->Get(LookupKey("key2", seq), &value, &s, &merge_context, + &range_del_agg, ReadOptions()); + ASSERT_TRUE(s.ok() && found); + ASSERT_EQ(value, "value2.2"); + + // Add memtable to list + list.Add(mem, &to_delete); + ASSERT_EQ(0, to_delete.size()); + + // Fetch keys via MemTableList + merge_context.Clear(); + found = list.current()->Get(LookupKey("key1", seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_TRUE(found && s.IsNotFound()); + + merge_context.Clear(); + found = list.current()->Get(LookupKey("key2", seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_TRUE(s.ok() && found); + ASSERT_EQ("value2.2", value); + + // Flush this memtable from the list. + // (It will then be a part of the memtable history). + autovector to_flush; + list.PickMemtablesToFlush(&to_flush); + ASSERT_EQ(1, to_flush.size()); + + s = Mock_InstallMemtableFlushResults(&list, MutableCFOptions(options), + to_flush, &to_delete); + ASSERT_OK(s); + ASSERT_EQ(0, list.NumNotFlushed()); + ASSERT_EQ(1, list.NumFlushed()); + ASSERT_EQ(0, to_delete.size()); + + // Verify keys are no longer in MemTableList + merge_context.Clear(); + found = list.current()->Get(LookupKey("key1", seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_FALSE(found); + + merge_context.Clear(); + found = list.current()->Get(LookupKey("key2", seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_FALSE(found); + + // Verify keys are present in history + merge_context.Clear(); + found = list.current()->GetFromHistory(LookupKey("key1", seq), &value, &s, + &merge_context, &range_del_agg, + ReadOptions()); + ASSERT_TRUE(found && s.IsNotFound()); + + merge_context.Clear(); + found = list.current()->GetFromHistory(LookupKey("key2", seq), &value, &s, + &merge_context, &range_del_agg, + ReadOptions()); + ASSERT_TRUE(found); + ASSERT_EQ("value2.2", value); + + // Create another memtable and write some keys to it + WriteBufferManager wb2(options.db_write_buffer_size); + MemTable* mem2 = new MemTable(cmp, ioptions, MutableCFOptions(options), &wb2, + kMaxSequenceNumber, 0 /* column_family_id */); + mem2->Ref(); + + mem2->Add(++seq, kTypeDeletion, "key1", ""); + mem2->Add(++seq, kTypeValue, "key3", "value3"); + + // Add second memtable to list + list.Add(mem2, &to_delete); + ASSERT_EQ(0, to_delete.size()); + + to_flush.clear(); + list.PickMemtablesToFlush(&to_flush); + ASSERT_EQ(1, to_flush.size()); + + // Flush second memtable + s = Mock_InstallMemtableFlushResults(&list, MutableCFOptions(options), + to_flush, &to_delete); + ASSERT_OK(s); + ASSERT_EQ(0, list.NumNotFlushed()); + ASSERT_EQ(2, list.NumFlushed()); + ASSERT_EQ(0, to_delete.size()); + + // Add a third memtable to push the first memtable out of the history + WriteBufferManager wb3(options.db_write_buffer_size); + MemTable* mem3 = new MemTable(cmp, ioptions, MutableCFOptions(options), &wb3, + kMaxSequenceNumber, 0 /* column_family_id */); + mem3->Ref(); + list.Add(mem3, &to_delete); + ASSERT_EQ(1, list.NumNotFlushed()); + ASSERT_EQ(1, list.NumFlushed()); + ASSERT_EQ(1, to_delete.size()); + + // Verify keys are no longer in MemTableList + merge_context.Clear(); + found = list.current()->Get(LookupKey("key1", seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_FALSE(found); + + merge_context.Clear(); + found = list.current()->Get(LookupKey("key2", seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_FALSE(found); + + merge_context.Clear(); + found = list.current()->Get(LookupKey("key3", seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_FALSE(found); + + // Verify that the second memtable's keys are in the history + merge_context.Clear(); + found = list.current()->GetFromHistory(LookupKey("key1", seq), &value, &s, + &merge_context, &range_del_agg, + ReadOptions()); + ASSERT_TRUE(found && s.IsNotFound()); + + merge_context.Clear(); + found = list.current()->GetFromHistory(LookupKey("key3", seq), &value, &s, + &merge_context, &range_del_agg, + ReadOptions()); + ASSERT_TRUE(found); + ASSERT_EQ("value3", value); + + // Verify that key2 from the first memtable is no longer in the history + merge_context.Clear(); + found = list.current()->Get(LookupKey("key2", seq), &value, &s, + &merge_context, &range_del_agg, ReadOptions()); + ASSERT_FALSE(found); + + // Cleanup + list.current()->Unref(&to_delete); + ASSERT_EQ(3, to_delete.size()); + for (MemTable* m : to_delete) { + delete m; + } +} + +TEST_F(MemTableListTest, FlushPendingTest) { + const int num_tables = 5; + SequenceNumber seq = 1; + Status s; + + auto factory = std::make_shared(); + options.memtable_factory = factory; + ImmutableCFOptions ioptions(options); + InternalKeyComparator cmp(BytewiseComparator()); + WriteBufferManager wb(options.db_write_buffer_size); + autovector to_delete; + + // Create MemTableList + int min_write_buffer_number_to_merge = 3; + int max_write_buffer_number_to_maintain = 7; + MemTableList list(min_write_buffer_number_to_merge, + max_write_buffer_number_to_maintain); + + // Create some MemTables + std::vector tables; + MutableCFOptions mutable_cf_options(options); + for (int i = 0; i < num_tables; i++) { + MemTable* mem = new MemTable(cmp, ioptions, mutable_cf_options, &wb, + kMaxSequenceNumber, 0 /* column_family_id */); + mem->Ref(); + + std::string value; + MergeContext merge_context; + + mem->Add(++seq, kTypeValue, "key1", ToString(i)); + mem->Add(++seq, kTypeValue, "keyN" + ToString(i), "valueN"); + mem->Add(++seq, kTypeValue, "keyX" + ToString(i), "value"); + mem->Add(++seq, kTypeValue, "keyM" + ToString(i), "valueM"); + mem->Add(++seq, kTypeDeletion, "keyX" + ToString(i), ""); + + tables.push_back(mem); + } + + // Nothing to flush + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + autovector to_flush; + list.PickMemtablesToFlush(&to_flush); + ASSERT_EQ(0, to_flush.size()); + + // Request a flush even though there is nothing to flush + list.FlushRequested(); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Attempt to 'flush' to clear request for flush + list.PickMemtablesToFlush(&to_flush); + ASSERT_EQ(0, to_flush.size()); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Request a flush again + list.FlushRequested(); + // No flush pending since the list is empty. + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Add 2 tables + list.Add(tables[0], &to_delete); + list.Add(tables[1], &to_delete); + ASSERT_EQ(2, list.NumNotFlushed()); + ASSERT_EQ(0, to_delete.size()); + + // Even though we have less than the minimum to flush, a flush is + // pending since we had previously requested a flush and never called + // PickMemtablesToFlush() to clear the flush. + ASSERT_TRUE(list.IsFlushPending()); + ASSERT_TRUE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Pick tables to flush + list.PickMemtablesToFlush(&to_flush); + ASSERT_EQ(2, to_flush.size()); + ASSERT_EQ(2, list.NumNotFlushed()); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Revert flush + list.RollbackMemtableFlush(to_flush, 0); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_TRUE(list.imm_flush_needed.load(std::memory_order_acquire)); + to_flush.clear(); + + // Add another table + list.Add(tables[2], &to_delete); + // We now have the minimum to flush regardles of whether FlushRequested() + // was called. + ASSERT_TRUE(list.IsFlushPending()); + ASSERT_TRUE(list.imm_flush_needed.load(std::memory_order_acquire)); + ASSERT_EQ(0, to_delete.size()); + + // Pick tables to flush + list.PickMemtablesToFlush(&to_flush); + ASSERT_EQ(3, to_flush.size()); + ASSERT_EQ(3, list.NumNotFlushed()); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Pick tables to flush again + autovector to_flush2; + list.PickMemtablesToFlush(&to_flush2); + ASSERT_EQ(0, to_flush2.size()); + ASSERT_EQ(3, list.NumNotFlushed()); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Add another table + list.Add(tables[3], &to_delete); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_TRUE(list.imm_flush_needed.load(std::memory_order_acquire)); + ASSERT_EQ(0, to_delete.size()); + + // Request a flush again + list.FlushRequested(); + ASSERT_TRUE(list.IsFlushPending()); + ASSERT_TRUE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Pick tables to flush again + list.PickMemtablesToFlush(&to_flush2); + ASSERT_EQ(1, to_flush2.size()); + ASSERT_EQ(4, list.NumNotFlushed()); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Rollback first pick of tables + list.RollbackMemtableFlush(to_flush, 0); + ASSERT_TRUE(list.IsFlushPending()); + ASSERT_TRUE(list.imm_flush_needed.load(std::memory_order_acquire)); + to_flush.clear(); + + // Add another tables + list.Add(tables[4], &to_delete); + ASSERT_EQ(5, list.NumNotFlushed()); + // We now have the minimum to flush regardles of whether FlushRequested() + ASSERT_TRUE(list.IsFlushPending()); + ASSERT_TRUE(list.imm_flush_needed.load(std::memory_order_acquire)); + ASSERT_EQ(0, to_delete.size()); + + // Pick tables to flush + list.PickMemtablesToFlush(&to_flush); + // Should pick 4 of 5 since 1 table has been picked in to_flush2 + ASSERT_EQ(4, to_flush.size()); + ASSERT_EQ(5, list.NumNotFlushed()); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Pick tables to flush again + autovector to_flush3; + ASSERT_EQ(0, to_flush3.size()); // nothing not in progress of being flushed + ASSERT_EQ(5, list.NumNotFlushed()); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Flush the 4 memtables that were picked in to_flush + s = Mock_InstallMemtableFlushResults(&list, MutableCFOptions(options), + to_flush, &to_delete); + ASSERT_OK(s); + + // Note: now to_flush contains tables[0,1,2,4]. to_flush2 contains + // tables[3]. + // Current implementation will only commit memtables in the order they were + // created. So InstallMemtableFlushResults will install the first 3 tables + // in to_flush and stop when it encounters a table not yet flushed. + ASSERT_EQ(2, list.NumNotFlushed()); + int num_in_history = std::min(3, max_write_buffer_number_to_maintain); + ASSERT_EQ(num_in_history, list.NumFlushed()); + ASSERT_EQ(5 - list.NumNotFlushed() - num_in_history, to_delete.size()); + + // Request a flush again. Should be nothing to flush + list.FlushRequested(); + ASSERT_FALSE(list.IsFlushPending()); + ASSERT_FALSE(list.imm_flush_needed.load(std::memory_order_acquire)); + + // Flush the 1 memtable that was picked in to_flush2 + s = MemTableListTest::Mock_InstallMemtableFlushResults( + &list, MutableCFOptions(options), to_flush2, &to_delete); + ASSERT_OK(s); + + // This will actually install 2 tables. The 1 we told it to flush, and also + // tables[4] which has been waiting for tables[3] to commit. + ASSERT_EQ(0, list.NumNotFlushed()); + num_in_history = std::min(5, max_write_buffer_number_to_maintain); + ASSERT_EQ(num_in_history, list.NumFlushed()); + ASSERT_EQ(5 - list.NumNotFlushed() - num_in_history, to_delete.size()); + + for (const auto& m : to_delete) { + // Refcount should be 0 after calling InstallMemtableFlushResults. + // Verify this, by Ref'ing then UnRef'ing: + m->Ref(); + ASSERT_EQ(m, m->Unref()); + delete m; + } + to_delete.clear(); + + list.current()->Unref(&to_delete); + int to_delete_size = std::min(5, max_write_buffer_number_to_maintain); + ASSERT_EQ(to_delete_size, to_delete.size()); + + for (const auto& m : to_delete) { + // Refcount should be 0 after calling InstallMemtableFlushResults. + // Verify this, by Ref'ing then UnRef'ing: + m->Ref(); + ASSERT_EQ(m, m->Unref()); + delete m; + } + to_delete.clear(); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/merge_context.h b/db/merge_context.h index bf483a8275b..5e75e099731 100644 --- a/db/merge_context.h +++ b/db/merge_context.h @@ -1,69 +1,116 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #pragma once +#include +#include #include "db/dbformat.h" #include "rocksdb/slice.h" -#include -#include namespace rocksdb { -const std::deque empty_operand_list; +const std::vector empty_operand_list; // The merge context for merging a user key. // When doing a Get(), DB will create such a class and pass it when // issuing Get() operation to memtables and version_set. The operands // will be fetched from the context when issuing partial of full merge. class MergeContext { -public: + public: // Clear all the operands void Clear() { - if (operand_list) { - operand_list->clear(); + if (operand_list_) { + operand_list_->clear(); + copied_operands_->clear(); } } - // Replace all operands with merge_result, which are expected to be the - // merge result of them. - void PushPartialMergeResult(std::string& merge_result) { - assert (operand_list); - operand_list->clear(); - operand_list->push_front(std::move(merge_result)); - } + // Push a merge operand - void PushOperand(const Slice& operand_slice) { + void PushOperand(const Slice& operand_slice, bool operand_pinned = false) { Initialize(); - operand_list->push_front(operand_slice.ToString()); + SetDirectionBackward(); + + if (operand_pinned) { + operand_list_->push_back(operand_slice); + } else { + // We need to have our own copy of the operand since it's not pinned + copied_operands_->emplace_back( + new std::string(operand_slice.data(), operand_slice.size())); + operand_list_->push_back(*copied_operands_->back()); + } } + + // Push back a merge operand + void PushOperandBack(const Slice& operand_slice, + bool operand_pinned = false) { + Initialize(); + SetDirectionForward(); + + if (operand_pinned) { + operand_list_->push_back(operand_slice); + } else { + // We need to have our own copy of the operand since it's not pinned + copied_operands_->emplace_back( + new std::string(operand_slice.data(), operand_slice.size())); + operand_list_->push_back(*copied_operands_->back()); + } + } + // return total number of operands in the list size_t GetNumOperands() const { - if (!operand_list) { + if (!operand_list_) { return 0; } - return operand_list->size(); + return operand_list_->size(); } + // Get the operand at the index. - Slice GetOperand(int index) const { - assert (operand_list); - return (*operand_list)[index]; + Slice GetOperand(int index) { + assert(operand_list_); + + SetDirectionForward(); + return (*operand_list_)[index]; } + // Return all the operands. - const std::deque& GetOperands() const { - if (!operand_list) { + const std::vector& GetOperands() { + if (!operand_list_) { return empty_operand_list; } - return *operand_list; + + SetDirectionForward(); + return *operand_list_; } -private: + + private: void Initialize() { - if (!operand_list) { - operand_list.reset(new std::deque()); + if (!operand_list_) { + operand_list_.reset(new std::vector()); + copied_operands_.reset(new std::vector>()); + } + } + + void SetDirectionForward() { + if (operands_reversed_ == true) { + std::reverse(operand_list_->begin(), operand_list_->end()); + operands_reversed_ = false; + } + } + + void SetDirectionBackward() { + if (operands_reversed_ == false) { + std::reverse(operand_list_->begin(), operand_list_->end()); + operands_reversed_ = true; } } - std::unique_ptr> operand_list; + + // List of operands + std::unique_ptr> operand_list_; + // Copy of operands that are not pinned. + std::unique_ptr>> copied_operands_; + bool operands_reversed_ = true; }; } // namespace rocksdb - diff --git a/db/merge_helper.cc b/db/merge_helper.cc index 7bde824ab0f..55f8254cf0d 100644 --- a/db/merge_helper.cc +++ b/db/merge_helper.cc @@ -1,155 +1,277 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -#include "merge_helper.h" +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/merge_helper.h" + +#include +#include + #include "db/dbformat.h" +#include "monitoring/perf_context_imp.h" +#include "monitoring/statistics.h" #include "rocksdb/comparator.h" #include "rocksdb/db.h" #include "rocksdb/merge_operator.h" -#include "util/statistics.h" -#include -#include +#include "table/internal_iterator.h" namespace rocksdb { +MergeHelper::MergeHelper(Env* env, const Comparator* user_comparator, + const MergeOperator* user_merge_operator, + const CompactionFilter* compaction_filter, + Logger* logger, bool assert_valid_internal_key, + SequenceNumber latest_snapshot, int level, + Statistics* stats, + const std::atomic* shutting_down) + : env_(env), + user_comparator_(user_comparator), + user_merge_operator_(user_merge_operator), + compaction_filter_(compaction_filter), + shutting_down_(shutting_down), + logger_(logger), + assert_valid_internal_key_(assert_valid_internal_key), + allow_single_operand_(false), + latest_snapshot_(latest_snapshot), + level_(level), + keys_(), + filter_timer_(env_), + total_filter_time_(0U), + stats_(stats) { + assert(user_comparator_ != nullptr); + if (user_merge_operator_) { + allow_single_operand_ = user_merge_operator_->AllowSingleOperand(); + } +} + +Status MergeHelper::TimedFullMerge(const MergeOperator* merge_operator, + const Slice& key, const Slice* value, + const std::vector& operands, + std::string* result, Logger* logger, + Statistics* statistics, Env* env, + Slice* result_operand, + bool update_num_ops_stats) { + assert(merge_operator != nullptr); + + if (operands.size() == 0) { + assert(value != nullptr && result != nullptr); + result->assign(value->data(), value->size()); + return Status::OK(); + } + + if (update_num_ops_stats) { + MeasureTime(statistics, READ_NUM_MERGE_OPERANDS, + static_cast(operands.size())); + } + + bool success; + Slice tmp_result_operand(nullptr, 0); + const MergeOperator::MergeOperationInput merge_in(key, value, operands, + logger); + MergeOperator::MergeOperationOutput merge_out(*result, tmp_result_operand); + { + // Setup to time the merge + StopWatchNano timer(env, statistics != nullptr); + PERF_TIMER_GUARD(merge_operator_time_nanos); + + // Do the merge + success = merge_operator->FullMergeV2(merge_in, &merge_out); + + if (tmp_result_operand.data()) { + // FullMergeV2 result is an existing operand + if (result_operand != nullptr) { + *result_operand = tmp_result_operand; + } else { + result->assign(tmp_result_operand.data(), tmp_result_operand.size()); + } + } else if (result_operand) { + *result_operand = Slice(nullptr, 0); + } + + RecordTick(statistics, MERGE_OPERATION_TOTAL_TIME, + statistics ? timer.ElapsedNanos() : 0); + } + + if (!success) { + RecordTick(statistics, NUMBER_MERGE_FAILURES); + return Status::Corruption("Error: Could not perform merge."); + } + + return Status::OK(); +} + // PRE: iter points to the first merge type entry // POST: iter points to the first entry beyond the merge process (or the end) // keys_, operands_ are updated to reflect the merge result. // keys_ stores the list of keys encountered while merging. // operands_ stores the list of merge operands encountered while merging. // keys_[i] corresponds to operands_[i] for each i. -void MergeHelper::MergeUntil(Iterator* iter, SequenceNumber stop_before, - bool at_bottom, Statistics* stats, int* steps) { +Status MergeHelper::MergeUntil(InternalIterator* iter, + RangeDelAggregator* range_del_agg, + const SequenceNumber stop_before, + const bool at_bottom) { // Get a copy of the internal key, before it's invalidated by iter->Next() // Also maintain the list of merge operands seen. assert(HasOperator()); keys_.clear(); - operands_.clear(); - keys_.push_front(iter->key().ToString()); - operands_.push_front(iter->value().ToString()); + merge_context_.Clear(); + has_compaction_filter_skip_until_ = false; assert(user_merge_operator_); - - success_ = false; // Will become true if we hit Put/Delete or bottom + bool first_key = true; // We need to parse the internal key again as the parsed key is // backed by the internal key! // Assume no internal key corruption as it has been successfully parsed // by the caller. - // Invariant: keys_.back() will not change. Hence, orig_ikey is always valid. + // original_key_is_iter variable is just caching the information: + // original_key_is_iter == (iter->key().ToString() == original_key) + bool original_key_is_iter = true; + std::string original_key = iter->key().ToString(); + // Important: + // orig_ikey is backed by original_key if keys_.empty() + // orig_ikey is backed by keys_.back() if !keys_.empty() ParsedInternalKey orig_ikey; - ParseInternalKey(keys_.back(), &orig_ikey); + ParseInternalKey(original_key, &orig_ikey); + Status s; bool hit_the_next_user_key = false; - std::string merge_result; // Temporary value for merge results - if (steps) { - ++(*steps); - } - for (iter->Next(); iter->Valid(); iter->Next()) { + for (; iter->Valid(); iter->Next(), original_key_is_iter = false) { + if (IsShuttingDown()) { + return Status::ShutdownInProgress(); + } + ParsedInternalKey ikey; - assert(operands_.size() >= 1); // Should be invariants! - assert(keys_.size() == operands_.size()); + assert(keys_.size() == merge_context_.GetNumOperands()); if (!ParseInternalKey(iter->key(), &ikey)) { // stop at corrupted key if (assert_valid_internal_key_) { - assert(!"corrupted internal key is not expected"); + assert(!"Corrupted internal key not expected."); + return Status::Corruption("Corrupted internal key not expected."); } break; - } - - if (user_comparator_->Compare(ikey.user_key, orig_ikey.user_key) != 0) { + } else if (first_key) { + assert(user_comparator_->Equal(ikey.user_key, orig_ikey.user_key)); + first_key = false; + } else if (!user_comparator_->Equal(ikey.user_key, orig_ikey.user_key)) { // hit a different user key, stop right here hit_the_next_user_key = true; break; - } - - if (stop_before && ikey.sequence <= stop_before) { + } else if (stop_before && ikey.sequence <= stop_before) { // hit an entry that's visible by the previous snapshot, can't touch that break; } // At this point we are guaranteed that we need to process this key. - if (kTypeDeletion == ikey.type) { - // hit a delete - // => merge nullptr with operands_ + assert(IsValueType(ikey.type)); + if (ikey.type != kTypeMerge) { + + // hit a put/delete/single delete + // => merge the put value or a nullptr with operands_ // => store result in operands_.back() (and update keys_.back()) // => change the entry type to kTypeValue for keys_.back() - // We are done! Return a success if the merge passes. - success_ = user_merge_operator_->FullMerge(ikey.user_key, nullptr, - operands_, &merge_result, - logger_); - - // We store the result in keys_.back() and operands_.back() - // if nothing went wrong (i.e.: no operand corruption on disk) - if (success_) { - std::string& key = keys_.back(); // The original key encountered - orig_ikey.type = kTypeValue; - UpdateInternalKey(&key[0], key.size(), - orig_ikey.sequence, orig_ikey.type); - swap(operands_.back(), merge_result); - } else { - RecordTick(stats, NUMBER_MERGE_FAILURES); - } + // We are done! Success! - // move iter to the next entry (before doing anything else) - iter->Next(); - if (steps) { - ++(*steps); + // If there are no operands, just return the Status::OK(). That will cause + // the compaction iterator to write out the key we're currently at, which + // is the put/delete we just encountered. + if (keys_.empty()) { + return Status::OK(); } - return; - } - if (kTypeValue == ikey.type) { - // hit a put - // => merge the put value with operands_ - // => store result in operands_.back() (and update keys_.back()) - // => change the entry type to kTypeValue for keys_.back() - // We are done! Success! - const Slice value = iter->value(); - success_ = user_merge_operator_->FullMerge(ikey.user_key, &value, - operands_, &merge_result, - logger_); + // TODO(noetzli) If the merge operator returns false, we are currently + // (almost) silently dropping the put/delete. That's probably not what we + // want. Also if we're in compaction and it's a put, it would be nice to + // run compaction filter on it. + const Slice val = iter->value(); + const Slice* val_ptr = (kTypeValue == ikey.type) ? &val : nullptr; + std::string merge_result; + s = TimedFullMerge(user_merge_operator_, ikey.user_key, val_ptr, + merge_context_.GetOperands(), &merge_result, logger_, + stats_, env_); // We store the result in keys_.back() and operands_.back() // if nothing went wrong (i.e.: no operand corruption on disk) - if (success_) { - std::string& key = keys_.back(); // The original key encountered + if (s.ok()) { + // The original key encountered + original_key = std::move(keys_.back()); orig_ikey.type = kTypeValue; - UpdateInternalKey(&key[0], key.size(), - orig_ikey.sequence, orig_ikey.type); - swap(operands_.back(), merge_result); - } else { - RecordTick(stats, NUMBER_MERGE_FAILURES); + UpdateInternalKey(&original_key, orig_ikey.sequence, orig_ikey.type); + keys_.clear(); + merge_context_.Clear(); + keys_.emplace_front(std::move(original_key)); + merge_context_.PushOperand(merge_result); } // move iter to the next entry iter->Next(); - if (steps) { - ++(*steps); - } - return; - } - - if (kTypeMerge == ikey.type) { + return s; + } else { // hit a merge + // => if there is a compaction filter, apply it. + // => check for range tombstones covering the operand // => merge the operand into the front of the operands_ list - // => use the user's associative merge function to determine how. + // if not filtered // => then continue because we haven't yet seen a Put/Delete. - assert(!operands_.empty()); // Should have at least one element in it - - // keep queuing keys and operands until we either meet a put / delete + // + // Keep queuing keys and operands until we either meet a put / delete // request or later did a partial merge. - keys_.push_front(iter->key().ToString()); - operands_.push_front(iter->value().ToString()); - if (steps) { - ++(*steps); + + Slice value_slice = iter->value(); + // add an operand to the list if: + // 1) it's included in one of the snapshots. in that case we *must* write + // it out, no matter what compaction filter says + // 2) it's not filtered by a compaction filter + CompactionFilter::Decision filter = + ikey.sequence <= latest_snapshot_ + ? CompactionFilter::Decision::kKeep + : FilterMerge(orig_ikey.user_key, value_slice); + if (filter != CompactionFilter::Decision::kRemoveAndSkipUntil && + range_del_agg != nullptr && + range_del_agg->ShouldDelete( + iter->key(), + RangeDelAggregator::RangePositioningMode::kForwardTraversal)) { + filter = CompactionFilter::Decision::kRemove; + } + if (filter == CompactionFilter::Decision::kKeep || + filter == CompactionFilter::Decision::kChangeValue) { + if (original_key_is_iter) { + // this is just an optimization that saves us one memcpy + keys_.push_front(std::move(original_key)); + } else { + keys_.push_front(iter->key().ToString()); + } + if (keys_.size() == 1) { + // we need to re-anchor the orig_ikey because it was anchored by + // original_key before + ParseInternalKey(keys_.back(), &orig_ikey); + } + if (filter == CompactionFilter::Decision::kKeep) { + merge_context_.PushOperand( + value_slice, iter->IsValuePinned() /* operand_pinned */); + } else { // kChangeValue + // Compaction filter asked us to change the operand from value_slice + // to compaction_filter_value_. + merge_context_.PushOperand(compaction_filter_value_, false); + } + } else if (filter == CompactionFilter::Decision::kRemoveAndSkipUntil) { + // Compaction filter asked us to remove this key altogether + // (not just this operand), along with some keys following it. + keys_.clear(); + merge_context_.Clear(); + has_compaction_filter_skip_until_ = true; + return Status::OK(); } } } + if (merge_context_.GetNumOperands() == 0) { + // we filtered out all the merge operands + return Status::OK(); + } + // We are sure we have seen this key's entire history if we are at the // last level and exhausted all internal keys of this user key. // NOTE: !iter->Valid() does not necessarily mean we hit the @@ -170,42 +292,102 @@ void MergeHelper::MergeUntil(Iterator* iter, SequenceNumber stop_before, // do a final merge with nullptr as the existing value and say // bye to the merge type (it's now converted to a Put) assert(kTypeMerge == orig_ikey.type); - assert(operands_.size() >= 1); - assert(operands_.size() == keys_.size()); - success_ = user_merge_operator_->FullMerge(orig_ikey.user_key, nullptr, - operands_, &merge_result, - logger_); - - if (success_) { - std::string& key = keys_.back(); // The original key encountered + assert(merge_context_.GetNumOperands() >= 1); + assert(merge_context_.GetNumOperands() == keys_.size()); + std::string merge_result; + s = TimedFullMerge(user_merge_operator_, orig_ikey.user_key, nullptr, + merge_context_.GetOperands(), &merge_result, logger_, + stats_, env_); + if (s.ok()) { + // The original key encountered + // We are certain that keys_ is not empty here (see assertions couple of + // lines before). + original_key = std::move(keys_.back()); orig_ikey.type = kTypeValue; - UpdateInternalKey(&key[0], key.size(), - orig_ikey.sequence, orig_ikey.type); - - // The final value() is always stored in operands_.back() - swap(operands_.back(),merge_result); - } else { - RecordTick(stats, NUMBER_MERGE_FAILURES); - // Do nothing if not success_. Leave keys() and operands() as they are. + UpdateInternalKey(&original_key, orig_ikey.sequence, orig_ikey.type); + keys_.clear(); + merge_context_.Clear(); + keys_.emplace_front(std::move(original_key)); + merge_context_.PushOperand(merge_result); } } else { // We haven't seen the beginning of the key nor a Put/Delete. // Attempt to use the user's associative merge function to // merge the stacked merge operands into a single operand. - - if (operands_.size() >= 2 && - operands_.size() >= min_partial_merge_operands_ && - user_merge_operator_->PartialMergeMulti( + s = Status::MergeInProgress(); + if (merge_context_.GetNumOperands() >= 2 || + (allow_single_operand_ && merge_context_.GetNumOperands() == 1)) { + bool merge_success = false; + std::string merge_result; + { + StopWatchNano timer(env_, stats_ != nullptr); + PERF_TIMER_GUARD(merge_operator_time_nanos); + merge_success = user_merge_operator_->PartialMergeMulti( orig_ikey.user_key, - std::deque(operands_.begin(), operands_.end()), - &merge_result, logger_)) { - // Merging of operands (associative merge) was successful. - // Replace operands with the merge result - operands_.clear(); - operands_.push_front(std::move(merge_result)); - keys_.erase(keys_.begin(), keys_.end() - 1); + std::deque(merge_context_.GetOperands().begin(), + merge_context_.GetOperands().end()), + &merge_result, logger_); + RecordTick(stats_, MERGE_OPERATION_TOTAL_TIME, + stats_ ? timer.ElapsedNanosSafe() : 0); + } + if (merge_success) { + // Merging of operands (associative merge) was successful. + // Replace operands with the merge result + merge_context_.Clear(); + merge_context_.PushOperand(merge_result); + keys_.erase(keys_.begin(), keys_.end() - 1); + } + } + } + + return s; +} + +MergeOutputIterator::MergeOutputIterator(const MergeHelper* merge_helper) + : merge_helper_(merge_helper) { + it_keys_ = merge_helper_->keys().rend(); + it_values_ = merge_helper_->values().rend(); +} + +void MergeOutputIterator::SeekToFirst() { + const auto& keys = merge_helper_->keys(); + const auto& values = merge_helper_->values(); + assert(keys.size() == values.size()); + it_keys_ = keys.rbegin(); + it_values_ = values.rbegin(); +} + +void MergeOutputIterator::Next() { + ++it_keys_; + ++it_values_; +} + +CompactionFilter::Decision MergeHelper::FilterMerge(const Slice& user_key, + const Slice& value_slice) { + if (compaction_filter_ == nullptr) { + return CompactionFilter::Decision::kKeep; + } + if (stats_ != nullptr) { + filter_timer_.Start(); + } + compaction_filter_value_.clear(); + compaction_filter_skip_until_.Clear(); + auto ret = compaction_filter_->FilterV2( + level_, user_key, CompactionFilter::ValueType::kMergeOperand, value_slice, + &compaction_filter_value_, compaction_filter_skip_until_.rep()); + if (ret == CompactionFilter::Decision::kRemoveAndSkipUntil) { + if (user_comparator_->Compare(*compaction_filter_skip_until_.rep(), + user_key) <= 0) { + // Invalid skip_until returned from compaction filter. + // Keep the key as per FilterV2 documentation. + ret = CompactionFilter::Decision::kKeep; + } else { + compaction_filter_skip_until_.ConvertFromUserKey(kMaxSequenceNumber, + kValueTypeForSeek); } } + total_filter_time_ += filter_timer_.ElapsedNanosSafe(); + return ret; } } // namespace rocksdb diff --git a/db/merge_helper.h b/db/merge_helper.h index 7188d02a4d5..b9ef12a4cff 100644 --- a/db/merge_helper.h +++ b/db/merge_helper.h @@ -1,15 +1,22 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #ifndef MERGE_HELPER_H #define MERGE_HELPER_H +#include +#include +#include + #include "db/dbformat.h" +#include "db/merge_context.h" +#include "db/range_del_aggregator.h" +#include "rocksdb/compaction_filter.h" +#include "rocksdb/env.h" #include "rocksdb/slice.h" -#include -#include +#include "util/stop_watch.h" namespace rocksdb { @@ -18,46 +25,77 @@ class Iterator; class Logger; class MergeOperator; class Statistics; +class InternalIterator; class MergeHelper { public: - MergeHelper(const Comparator* user_comparator, - const MergeOperator* user_merge_operator, Logger* logger, - unsigned min_partial_merge_operands, - bool assert_valid_internal_key) - : user_comparator_(user_comparator), - user_merge_operator_(user_merge_operator), - logger_(logger), - min_partial_merge_operands_(min_partial_merge_operands), - assert_valid_internal_key_(assert_valid_internal_key), - keys_(), - operands_(), - success_(false) {} + MergeHelper(Env* env, const Comparator* user_comparator, + const MergeOperator* user_merge_operator, + const CompactionFilter* compaction_filter, Logger* logger, + bool assert_valid_internal_key, SequenceNumber latest_snapshot, + int level = 0, Statistics* stats = nullptr, + const std::atomic* shutting_down = nullptr); + + // Wrapper around MergeOperator::FullMergeV2() that records perf statistics. + // Result of merge will be written to result if status returned is OK. + // If operands is empty, the value will simply be copied to result. + // Set `update_num_ops_stats` to true if it is from a user read, so that + // the latency is sensitive. + // Returns one of the following statuses: + // - OK: Entries were successfully merged. + // - Corruption: Merge operator reported unsuccessful merge. + static Status TimedFullMerge(const MergeOperator* merge_operator, + const Slice& key, const Slice* value, + const std::vector& operands, + std::string* result, Logger* logger, + Statistics* statistics, Env* env, + Slice* result_operand = nullptr, + bool update_num_ops_stats = false); // Merge entries until we hit // - a corrupted key // - a Put/Delete, // - a different user key, // - a specific sequence number (snapshot boundary), + // - REMOVE_AND_SKIP_UNTIL returned from compaction filter, // or - the end of iteration // iter: (IN) points to the first merge type entry // (OUT) points to the first entry not included in the merge process + // range_del_agg: (IN) filters merge operands covered by range tombstones. // stop_before: (IN) a sequence number that merge should not cross. // 0 means no restriction // at_bottom: (IN) true if the iterator covers the bottem level, which means // we could reach the start of the history of this user key. - void MergeUntil(Iterator* iter, SequenceNumber stop_before = 0, - bool at_bottom = false, Statistics* stats = nullptr, - int* steps = nullptr); + // + // Returns one of the following statuses: + // - OK: Entries were successfully merged. + // - MergeInProgress: Put/Delete not encountered, and didn't reach the start + // of key's history. Output consists of merge operands only. + // - Corruption: Merge operator reported unsuccessful merge or a corrupted + // key has been encountered and not expected (applies only when compiling + // with asserts removed). + // - ShutdownInProgress: interrupted by shutdown (*shutting_down == true). + // + // REQUIRED: The first key in the input is not corrupted. + Status MergeUntil(InternalIterator* iter, + RangeDelAggregator* range_del_agg = nullptr, + const SequenceNumber stop_before = 0, + const bool at_bottom = false); + + // Filters a merge operand using the compaction filter specified + // in the constructor. Returns the decision that the filter made. + // Uses compaction_filter_value_ and compaction_filter_skip_until_ for the + // optional outputs of compaction filter. + CompactionFilter::Decision FilterMerge(const Slice& user_key, + const Slice& value_slice); // Query the merge result // These are valid until the next MergeUntil call // If the merging was successful: - // - IsSuccess() will be true - // - key() will have the latest sequence number of the merges. - // The type will be Put or Merge. See IMPORTANT 1 note, below. - // - value() will be the result of merging all the operands together - // - The user should ignore keys() and values(). + // - keys() contains a single element with the latest sequence number of + // the merges. The type will be Put or Merge. See IMPORTANT 1 note, below. + // - values() contains a single element with the result of merging all the + // operands together // // IMPORTANT 1: the key type could change after the MergeUntil call. // Put/Delete + Merge + ... + Merge => Put @@ -65,7 +103,6 @@ class MergeHelper { // // If the merge operator is not associative, and if a Put/Delete is not found // then the merging will be unsuccessful. In this case: - // - IsSuccess() will be false // - keys() contains the list of internal keys seen in order of iteration. // - values() contains the list of values (merges) seen in the same order. // values() is parallel to keys() so that the first entry in @@ -73,34 +110,84 @@ class MergeHelper { // and so on. These lists will be the same length. // All of these pairs will be merges over the same user key. // See IMPORTANT 2 note below. - // - The user should ignore key() and value(). // // IMPORTANT 2: The entries were traversed in order from BACK to FRONT. // So keys().back() was the first key seen by iterator. // TODO: Re-style this comment to be like the first one - bool IsSuccess() const { return success_; } - Slice key() const { assert(success_); return Slice(keys_.back()); } - Slice value() const { assert(success_); return Slice(operands_.back()); } - const std::deque& keys() const { - assert(!success_); return keys_; - } - const std::deque& values() const { - assert(!success_); return operands_; + const std::deque& keys() const { return keys_; } + const std::vector& values() const { + return merge_context_.GetOperands(); } + uint64_t TotalFilterTime() const { return total_filter_time_; } bool HasOperator() const { return user_merge_operator_ != nullptr; } + // If compaction filter returned REMOVE_AND_SKIP_UNTIL, this method will + // return true and fill *until with the key to which we should skip. + // If true, keys() and values() are empty. + bool FilteredUntil(Slice* skip_until) const { + if (!has_compaction_filter_skip_until_) { + return false; + } + assert(compaction_filter_ != nullptr); + assert(skip_until != nullptr); + assert(compaction_filter_skip_until_.Valid()); + *skip_until = compaction_filter_skip_until_.Encode(); + return true; + } + private: + Env* env_; const Comparator* user_comparator_; const MergeOperator* user_merge_operator_; + const CompactionFilter* compaction_filter_; + const std::atomic* shutting_down_; Logger* logger_; - unsigned min_partial_merge_operands_; bool assert_valid_internal_key_; // enforce no internal key corruption? + bool allow_single_operand_; + SequenceNumber latest_snapshot_; + int level_; // the scratch area that holds the result of MergeUntil // valid up to the next MergeUntil call - std::deque keys_; // Keeps track of the sequence of keys seen - std::deque operands_; // Parallel with keys_; stores the values - bool success_; + + // Keeps track of the sequence of keys seen + std::deque keys_; + // Parallel with keys_; stores the operands + mutable MergeContext merge_context_; + + StopWatchNano filter_timer_; + uint64_t total_filter_time_; + Statistics* stats_; + + bool has_compaction_filter_skip_until_ = false; + std::string compaction_filter_value_; + InternalKey compaction_filter_skip_until_; + + bool IsShuttingDown() { + // This is a best-effort facility, so memory_order_relaxed is sufficient. + return shutting_down_ && shutting_down_->load(std::memory_order_relaxed); + } +}; + +// MergeOutputIterator can be used to iterate over the result of a merge. +class MergeOutputIterator { + public: + // The MergeOutputIterator is bound to a MergeHelper instance. + explicit MergeOutputIterator(const MergeHelper* merge_helper); + + // Seeks to the first record in the output. + void SeekToFirst(); + // Advances to the next record in the output. + void Next(); + + Slice key() { return Slice(*it_keys_); } + Slice value() { return Slice(*it_values_); } + bool Valid() { return it_keys_ != merge_helper_->keys().rend(); } + + private: + const MergeHelper* merge_helper_; + std::deque::const_reverse_iterator it_keys_; + std::vector::const_reverse_iterator it_values_; }; } // namespace rocksdb diff --git a/db/merge_helper_test.cc b/db/merge_helper_test.cc new file mode 100644 index 00000000000..dc43db0d105 --- /dev/null +++ b/db/merge_helper_test.cc @@ -0,0 +1,290 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include +#include +#include + +#include "db/merge_helper.h" +#include "rocksdb/comparator.h" +#include "util/coding.h" +#include "util/testharness.h" +#include "util/testutil.h" +#include "utilities/merge_operators.h" + +namespace rocksdb { + +class MergeHelperTest : public testing::Test { + public: + MergeHelperTest() { env_ = Env::Default(); } + + ~MergeHelperTest() = default; + + Status Run(SequenceNumber stop_before, bool at_bottom, + SequenceNumber latest_snapshot = 0) { + iter_.reset(new test::VectorIterator(ks_, vs_)); + iter_->SeekToFirst(); + merge_helper_.reset(new MergeHelper(env_, BytewiseComparator(), + merge_op_.get(), filter_.get(), nullptr, + false, latest_snapshot)); + return merge_helper_->MergeUntil(iter_.get(), nullptr /* range_del_agg */, + stop_before, at_bottom); + } + + void AddKeyVal(const std::string& user_key, const SequenceNumber& seq, + const ValueType& t, const std::string& val, + bool corrupt = false) { + InternalKey ikey(user_key, seq, t); + if (corrupt) { + test::CorruptKeyType(&ikey); + } + ks_.push_back(ikey.Encode().ToString()); + vs_.push_back(val); + } + + Env* env_; + std::unique_ptr iter_; + std::shared_ptr merge_op_; + std::unique_ptr merge_helper_; + std::vector ks_; + std::vector vs_; + std::unique_ptr filter_; +}; + +// If MergeHelper encounters a new key on the last level, we know that +// the key has no more history and it can merge keys. +TEST_F(MergeHelperTest, MergeAtBottomSuccess) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + + AddKeyVal("a", 20, kTypeMerge, test::EncodeInt(1U)); + AddKeyVal("a", 10, kTypeMerge, test::EncodeInt(3U)); + AddKeyVal("b", 10, kTypeMerge, test::EncodeInt(4U)); // <- iter_ after merge + + ASSERT_TRUE(Run(0, true).ok()); + ASSERT_EQ(ks_[2], iter_->key()); + ASSERT_EQ(test::KeyStr("a", 20, kTypeValue), merge_helper_->keys()[0]); + ASSERT_EQ(test::EncodeInt(4U), merge_helper_->values()[0]); + ASSERT_EQ(1U, merge_helper_->keys().size()); + ASSERT_EQ(1U, merge_helper_->values().size()); +} + +// Merging with a value results in a successful merge. +TEST_F(MergeHelperTest, MergeValue) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + + AddKeyVal("a", 40, kTypeMerge, test::EncodeInt(1U)); + AddKeyVal("a", 30, kTypeMerge, test::EncodeInt(3U)); + AddKeyVal("a", 20, kTypeValue, test::EncodeInt(4U)); // <- iter_ after merge + AddKeyVal("a", 10, kTypeMerge, test::EncodeInt(1U)); + + ASSERT_TRUE(Run(0, false).ok()); + ASSERT_EQ(ks_[3], iter_->key()); + ASSERT_EQ(test::KeyStr("a", 40, kTypeValue), merge_helper_->keys()[0]); + ASSERT_EQ(test::EncodeInt(8U), merge_helper_->values()[0]); + ASSERT_EQ(1U, merge_helper_->keys().size()); + ASSERT_EQ(1U, merge_helper_->values().size()); +} + +// Merging stops before a snapshot. +TEST_F(MergeHelperTest, SnapshotBeforeValue) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + + AddKeyVal("a", 50, kTypeMerge, test::EncodeInt(1U)); + AddKeyVal("a", 40, kTypeMerge, test::EncodeInt(3U)); // <- iter_ after merge + AddKeyVal("a", 30, kTypeMerge, test::EncodeInt(1U)); + AddKeyVal("a", 20, kTypeValue, test::EncodeInt(4U)); + AddKeyVal("a", 10, kTypeMerge, test::EncodeInt(1U)); + + ASSERT_TRUE(Run(31, true).IsMergeInProgress()); + ASSERT_EQ(ks_[2], iter_->key()); + ASSERT_EQ(test::KeyStr("a", 50, kTypeMerge), merge_helper_->keys()[0]); + ASSERT_EQ(test::EncodeInt(4U), merge_helper_->values()[0]); + ASSERT_EQ(1U, merge_helper_->keys().size()); + ASSERT_EQ(1U, merge_helper_->values().size()); +} + +// MergeHelper preserves the operand stack for merge operators that +// cannot do a partial merge. +TEST_F(MergeHelperTest, NoPartialMerge) { + merge_op_ = MergeOperators::CreateStringAppendTESTOperator(); + + AddKeyVal("a", 50, kTypeMerge, "v2"); + AddKeyVal("a", 40, kTypeMerge, "v"); // <- iter_ after merge + AddKeyVal("a", 30, kTypeMerge, "v"); + + ASSERT_TRUE(Run(31, true).IsMergeInProgress()); + ASSERT_EQ(ks_[2], iter_->key()); + ASSERT_EQ(test::KeyStr("a", 40, kTypeMerge), merge_helper_->keys()[0]); + ASSERT_EQ("v", merge_helper_->values()[0]); + ASSERT_EQ(test::KeyStr("a", 50, kTypeMerge), merge_helper_->keys()[1]); + ASSERT_EQ("v2", merge_helper_->values()[1]); + ASSERT_EQ(2U, merge_helper_->keys().size()); + ASSERT_EQ(2U, merge_helper_->values().size()); +} + +// A single operand can not be merged. +TEST_F(MergeHelperTest, SingleOperand) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + + AddKeyVal("a", 50, kTypeMerge, test::EncodeInt(1U)); + + ASSERT_TRUE(Run(31, true).IsMergeInProgress()); + ASSERT_FALSE(iter_->Valid()); + ASSERT_EQ(test::KeyStr("a", 50, kTypeMerge), merge_helper_->keys()[0]); + ASSERT_EQ(test::EncodeInt(1U), merge_helper_->values()[0]); + ASSERT_EQ(1U, merge_helper_->keys().size()); + ASSERT_EQ(1U, merge_helper_->values().size()); +} + +// Merging with a deletion turns the deletion into a value +TEST_F(MergeHelperTest, MergeDeletion) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + + AddKeyVal("a", 30, kTypeMerge, test::EncodeInt(3U)); + AddKeyVal("a", 20, kTypeDeletion, ""); + + ASSERT_TRUE(Run(15, false).ok()); + ASSERT_FALSE(iter_->Valid()); + ASSERT_EQ(test::KeyStr("a", 30, kTypeValue), merge_helper_->keys()[0]); + ASSERT_EQ(test::EncodeInt(3U), merge_helper_->values()[0]); + ASSERT_EQ(1U, merge_helper_->keys().size()); + ASSERT_EQ(1U, merge_helper_->values().size()); +} + +// The merge helper stops upon encountering a corrupt key +TEST_F(MergeHelperTest, CorruptKey) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + + AddKeyVal("a", 30, kTypeMerge, test::EncodeInt(3U)); + AddKeyVal("a", 25, kTypeMerge, test::EncodeInt(1U)); + // Corrupt key + AddKeyVal("a", 20, kTypeDeletion, "", true); // <- iter_ after merge + + ASSERT_TRUE(Run(15, false).IsMergeInProgress()); + ASSERT_EQ(ks_[2], iter_->key()); + ASSERT_EQ(test::KeyStr("a", 30, kTypeMerge), merge_helper_->keys()[0]); + ASSERT_EQ(test::EncodeInt(4U), merge_helper_->values()[0]); + ASSERT_EQ(1U, merge_helper_->keys().size()); + ASSERT_EQ(1U, merge_helper_->values().size()); +} + +// The compaction filter is called on every merge operand +TEST_F(MergeHelperTest, FilterMergeOperands) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + filter_.reset(new test::FilterNumber(5U)); + + AddKeyVal("a", 30, kTypeMerge, test::EncodeInt(3U)); + AddKeyVal("a", 29, kTypeMerge, test::EncodeInt(5U)); // Filtered + AddKeyVal("a", 28, kTypeMerge, test::EncodeInt(3U)); + AddKeyVal("a", 27, kTypeMerge, test::EncodeInt(1U)); + AddKeyVal("a", 26, kTypeMerge, test::EncodeInt(5U)); // Filtered + AddKeyVal("a", 25, kTypeValue, test::EncodeInt(1U)); + + ASSERT_TRUE(Run(15, false).ok()); + ASSERT_FALSE(iter_->Valid()); + MergeOutputIterator merge_output_iter(merge_helper_.get()); + merge_output_iter.SeekToFirst(); + ASSERT_EQ(test::KeyStr("a", 30, kTypeValue), + merge_output_iter.key().ToString()); + ASSERT_EQ(test::EncodeInt(8U), merge_output_iter.value().ToString()); + merge_output_iter.Next(); + ASSERT_FALSE(merge_output_iter.Valid()); +} + +TEST_F(MergeHelperTest, FilterAllMergeOperands) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + filter_.reset(new test::FilterNumber(5U)); + + AddKeyVal("a", 30, kTypeMerge, test::EncodeInt(5U)); + AddKeyVal("a", 29, kTypeMerge, test::EncodeInt(5U)); + AddKeyVal("a", 28, kTypeMerge, test::EncodeInt(5U)); + AddKeyVal("a", 27, kTypeMerge, test::EncodeInt(5U)); + AddKeyVal("a", 26, kTypeMerge, test::EncodeInt(5U)); + AddKeyVal("a", 25, kTypeMerge, test::EncodeInt(5U)); + + // filtered out all + ASSERT_TRUE(Run(15, false).ok()); + ASSERT_FALSE(iter_->Valid()); + MergeOutputIterator merge_output_iter(merge_helper_.get()); + merge_output_iter.SeekToFirst(); + ASSERT_FALSE(merge_output_iter.Valid()); + + // we have one operand that will survive because it's a delete + AddKeyVal("a", 24, kTypeDeletion, test::EncodeInt(5U)); + AddKeyVal("b", 23, kTypeValue, test::EncodeInt(5U)); + ASSERT_TRUE(Run(15, true).ok()); + merge_output_iter = MergeOutputIterator(merge_helper_.get()); + ASSERT_TRUE(iter_->Valid()); + merge_output_iter.SeekToFirst(); + ASSERT_FALSE(merge_output_iter.Valid()); + + // when all merge operands are filtered out, we leave the iterator pointing to + // the Put/Delete that survived + ASSERT_EQ(test::KeyStr("a", 24, kTypeDeletion), iter_->key().ToString()); + ASSERT_EQ(test::EncodeInt(5U), iter_->value().ToString()); +} + +// Make sure that merge operands are filtered at the beginning +TEST_F(MergeHelperTest, FilterFirstMergeOperand) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + filter_.reset(new test::FilterNumber(5U)); + + AddKeyVal("a", 31, kTypeMerge, test::EncodeInt(5U)); // Filtered + AddKeyVal("a", 30, kTypeMerge, test::EncodeInt(5U)); // Filtered + AddKeyVal("a", 29, kTypeMerge, test::EncodeInt(2U)); + AddKeyVal("a", 28, kTypeMerge, test::EncodeInt(1U)); + AddKeyVal("a", 27, kTypeMerge, test::EncodeInt(3U)); + AddKeyVal("a", 26, kTypeMerge, test::EncodeInt(5U)); // Filtered + AddKeyVal("a", 25, kTypeMerge, test::EncodeInt(5U)); // Filtered + AddKeyVal("b", 24, kTypeValue, test::EncodeInt(5U)); // next user key + + ASSERT_OK(Run(15, true)); + ASSERT_TRUE(iter_->Valid()); + MergeOutputIterator merge_output_iter(merge_helper_.get()); + merge_output_iter.SeekToFirst(); + // sequence number is 29 here, because the first merge operand got filtered + // out + ASSERT_EQ(test::KeyStr("a", 29, kTypeValue), + merge_output_iter.key().ToString()); + ASSERT_EQ(test::EncodeInt(6U), merge_output_iter.value().ToString()); + merge_output_iter.Next(); + ASSERT_FALSE(merge_output_iter.Valid()); + + // make sure that we're passing user keys into the filter + ASSERT_EQ("a", filter_->last_merge_operand_key()); +} + +// Make sure that merge operands are not filtered out if there's a snapshot +// pointing at them +TEST_F(MergeHelperTest, DontFilterMergeOperandsBeforeSnapshotTest) { + merge_op_ = MergeOperators::CreateUInt64AddOperator(); + filter_.reset(new test::FilterNumber(5U)); + + AddKeyVal("a", 31, kTypeMerge, test::EncodeInt(5U)); + AddKeyVal("a", 30, kTypeMerge, test::EncodeInt(5U)); + AddKeyVal("a", 29, kTypeMerge, test::EncodeInt(2U)); + AddKeyVal("a", 28, kTypeMerge, test::EncodeInt(1U)); + AddKeyVal("a", 27, kTypeMerge, test::EncodeInt(3U)); + AddKeyVal("a", 26, kTypeMerge, test::EncodeInt(5U)); + AddKeyVal("a", 25, kTypeMerge, test::EncodeInt(5U)); + AddKeyVal("b", 24, kTypeValue, test::EncodeInt(5U)); + + ASSERT_OK(Run(15, true, 32)); + ASSERT_TRUE(iter_->Valid()); + MergeOutputIterator merge_output_iter(merge_helper_.get()); + merge_output_iter.SeekToFirst(); + ASSERT_EQ(test::KeyStr("a", 31, kTypeValue), + merge_output_iter.key().ToString()); + ASSERT_EQ(test::EncodeInt(26U), merge_output_iter.value().ToString()); + merge_output_iter.Next(); + ASSERT_FALSE(merge_output_iter.Valid()); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/merge_operator.cc b/db/merge_operator.cc index a14df8a8714..1981e65ba4e 100644 --- a/db/merge_operator.cc +++ b/db/merge_operator.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // /** * Back-end implementation details specific to the Merge Operator. @@ -11,6 +11,18 @@ namespace rocksdb { +bool MergeOperator::FullMergeV2(const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const { + // If FullMergeV2 is not implemented, we convert the operand_list to + // std::deque and pass it to FullMerge + std::deque operand_list_str; + for (auto& op : merge_in.operand_list) { + operand_list_str.emplace_back(op.data(), op.size()); + } + return FullMerge(merge_in.key, merge_in.existing_value, operand_list_str, + &merge_out->new_value, merge_in.logger); +} + // The default implementation of PartialMergeMulti, which invokes // PartialMerge multiple times internally and merges two operands at // a time. @@ -20,11 +32,11 @@ bool MergeOperator::PartialMergeMulti(const Slice& key, Logger* logger) const { assert(operand_list.size() >= 2); // Simply loop through the operands - std::string temp_value; Slice temp_slice(operand_list[0]); for (size_t i = 1; i < operand_list.size(); ++i) { auto& operand = operand_list[i]; + std::string temp_value; if (!PartialMerge(key, temp_slice, operand, &temp_value, logger)) { return false; } @@ -39,23 +51,20 @@ bool MergeOperator::PartialMergeMulti(const Slice& key, // Given a "real" merge from the library, call the user's // associative merge function one-by-one on each of the operands. // NOTE: It is assumed that the client's merge-operator will handle any errors. -bool AssociativeMergeOperator::FullMerge( - const Slice& key, - const Slice* existing_value, - const std::deque& operand_list, - std::string* new_value, - Logger* logger) const { - +bool AssociativeMergeOperator::FullMergeV2( + const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const { // Simply loop through the operands Slice temp_existing; - std::string temp_value; - for (const auto& operand : operand_list) { - Slice value(operand); - if (!Merge(key, existing_value, value, &temp_value, logger)) { + const Slice* existing_value = merge_in.existing_value; + for (const auto& operand : merge_in.operand_list) { + std::string temp_value; + if (!Merge(merge_in.key, existing_value, operand, &temp_value, + merge_in.logger)) { return false; } - swap(temp_value, *new_value); - temp_existing = Slice(*new_value); + swap(temp_value, merge_out->new_value); + temp_existing = Slice(merge_out->new_value); existing_value = &temp_existing; } diff --git a/db/merge_test.cc b/db/merge_test.cc index 7e71ccf861e..b6582b7a596 100644 --- a/db/merge_test.cc +++ b/db/merge_test.cc @@ -1,12 +1,13 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #include #include #include +#include "port/stack_trace.h" #include "rocksdb/cache.h" #include "rocksdb/comparator.h" #include "rocksdb/db.h" @@ -19,19 +20,14 @@ #include "utilities/merge_operators.h" #include "util/testharness.h" -using namespace std; using namespace rocksdb; namespace { - int numMergeOperatorCalls; - void resetNumMergeOperatorCalls() { - numMergeOperatorCalls = 0; - } +size_t num_merge_operator_calls; +void resetNumMergeOperatorCalls() { num_merge_operator_calls = 0; } - int num_partial_merge_calls; - void resetNumPartialMergeCalls() { - num_partial_merge_calls = 0; - } +size_t num_partial_merge_calls; +void resetNumPartialMergeCalls() { num_partial_merge_calls = 0; } } class CountMergeOperator : public AssociativeMergeOperator { @@ -45,7 +41,8 @@ class CountMergeOperator : public AssociativeMergeOperator { const Slice& value, std::string* new_value, Logger* logger) const override { - ++numMergeOperatorCalls; + assert(new_value->empty()); + ++num_merge_operator_calls; if (existing_value == nullptr) { new_value->assign(value.data(), value.size()); return true; @@ -61,7 +58,9 @@ class CountMergeOperator : public AssociativeMergeOperator { virtual bool PartialMergeMulti(const Slice& key, const std::deque& operand_list, - std::string* new_value, Logger* logger) const { + std::string* new_value, + Logger* logger) const override { + assert(new_value->empty()); ++num_partial_merge_calls; return mergeOperator_->PartialMergeMulti(key, operand_list, new_value, logger); @@ -76,27 +75,31 @@ class CountMergeOperator : public AssociativeMergeOperator { }; namespace { -std::shared_ptr OpenDb(const string& dbname, const bool ttl = false, - const size_t max_successive_merges = 0, - const uint32_t min_partial_merge_operands = 2) { +std::shared_ptr OpenDb(const std::string& dbname, const bool ttl = false, + const size_t max_successive_merges = 0) { DB* db; Options options; options.create_if_missing = true; options.merge_operator = std::make_shared(); options.max_successive_merges = max_successive_merges; - options.min_partial_merge_operands = min_partial_merge_operands; Status s; DestroyDB(dbname, Options()); +// DBWithTTL is not supported in ROCKSDB_LITE +#ifndef ROCKSDB_LITE if (ttl) { - cout << "Opening database with TTL\n"; + std::cout << "Opening database with TTL\n"; DBWithTTL* db_with_ttl; s = DBWithTTL::Open(options, dbname, &db_with_ttl); db = db_with_ttl; } else { s = DB::Open(options, dbname, &db); } +#else + assert(!ttl); + s = DB::Open(options, dbname, &db); +#endif // !ROCKSDB_LITE if (!s.ok()) { - cerr << s.ToString() << endl; + std::cerr << s.ToString() << std::endl; assert(false); } return std::shared_ptr(db); @@ -136,34 +139,36 @@ class Counters { // if the underlying level db operation failed. // mapped to a levedb Put - bool set(const string& key, uint64_t value) { + bool set(const std::string& key, uint64_t value) { // just treat the internal rep of int64 as the string - Slice slice((char *)&value, sizeof(value)); + char buf[sizeof(value)]; + EncodeFixed64(buf, value); + Slice slice(buf, sizeof(value)); auto s = db_->Put(put_option_, key, slice); if (s.ok()) { return true; } else { - cerr << s.ToString() << endl; + std::cerr << s.ToString() << std::endl; return false; } } // mapped to a rocksdb Delete - bool remove(const string& key) { + bool remove(const std::string& key) { auto s = db_->Delete(delete_option_, key); if (s.ok()) { return true; } else { - cerr << s.ToString() << std::endl; + std::cerr << s.ToString() << std::endl; return false; } } // mapped to a rocksdb Get - bool get(const string& key, uint64_t *value) { - string str; + bool get(const std::string& key, uint64_t* value) { + std::string str; auto s = db_->Get(get_option_, key, &str); if (s.IsNotFound()) { @@ -173,35 +178,33 @@ class Counters { } else if (s.ok()) { // deserialization if (str.size() != sizeof(uint64_t)) { - cerr << "value corruption\n"; + std::cerr << "value corruption\n"; return false; } *value = DecodeFixed64(&str[0]); return true; } else { - cerr << s.ToString() << std::endl; + std::cerr << s.ToString() << std::endl; return false; } } // 'add' is implemented as get -> modify -> set // An alternative is a single merge operation, see MergeBasedCounters - virtual bool add(const string& key, uint64_t value) { + virtual bool add(const std::string& key, uint64_t value) { uint64_t base = default_; return get(key, &base) && set(key, base + value); } // convenience functions for testing - void assert_set(const string& key, uint64_t value) { + void assert_set(const std::string& key, uint64_t value) { assert(set(key, value)); } - void assert_remove(const string& key) { - assert(remove(key)); - } + void assert_remove(const std::string& key) { assert(remove(key)); } - uint64_t assert_get(const string& key) { + uint64_t assert_get(const std::string& key) { uint64_t value = default_; int result = get(key, &value); assert(result); @@ -209,7 +212,7 @@ class Counters { return value; } - void assert_add(const string& key, uint64_t value) { + void assert_add(const std::string& key, uint64_t value) { int result = add(key, value); assert(result); if (result == 0) exit(1); // Disable unused variable warning. @@ -228,7 +231,7 @@ class MergeBasedCounters : public Counters { } // mapped to a rocksdb Merge operation - virtual bool add(const string& key, uint64_t value) override { + virtual bool add(const std::string& key, uint64_t value) override { char encoded[sizeof(uint64_t)]; EncodeFixed64(encoded, value); Slice slice(encoded, sizeof(uint64_t)); @@ -237,7 +240,7 @@ class MergeBasedCounters : public Counters { if (s.ok()) { return true; } else { - cerr << s.ToString() << endl; + std::cerr << s.ToString() << std::endl; return false; } } @@ -248,7 +251,7 @@ void dumpDb(DB* db) { auto it = unique_ptr(db->NewIterator(ReadOptions())); for (it->SeekToFirst(); it->Valid(); it->Next()) { uint64_t value = DecodeFixed64(it->value().data()); - cout << it->key().ToString() << ": " << value << endl; + std::cout << it->key().ToString() << ": " << value << std::endl; } assert(it->status().ok()); // Check for any errors found during the scan } @@ -296,9 +299,9 @@ void testCounters(Counters& counters, DB* db, bool test_compaction) { if (test_compaction) { db->Flush(o); - cout << "Compaction started ...\n"; - db->CompactRange(nullptr, nullptr); - cout << "Compaction ended\n"; + std::cout << "Compaction started ...\n"; + db->CompactRange(CompactRangeOptions(), nullptr, nullptr); + std::cout << "Compaction ended\n"; dumpDb(db); @@ -307,31 +310,31 @@ void testCounters(Counters& counters, DB* db, bool test_compaction) { } } -void testSuccessiveMerge( - Counters& counters, int max_num_merges, int num_merges) { +void testSuccessiveMerge(Counters& counters, size_t max_num_merges, + size_t num_merges) { counters.assert_remove("z"); uint64_t sum = 0; - for (int i = 1; i <= num_merges; ++i) { + for (size_t i = 1; i <= num_merges; ++i) { resetNumMergeOperatorCalls(); counters.assert_add("z", i); sum += i; if (i % (max_num_merges + 1) == 0) { - assert(numMergeOperatorCalls == max_num_merges + 1); + assert(num_merge_operator_calls == max_num_merges + 1); } else { - assert(numMergeOperatorCalls == 0); + assert(num_merge_operator_calls == 0); } resetNumMergeOperatorCalls(); assert(counters.assert_get("z") == sum); - assert(numMergeOperatorCalls == i % (max_num_merges + 1)); + assert(num_merge_operator_calls == i % (max_num_merges + 1)); } } -void testPartialMerge(Counters* counters, DB* db, int max_merge, int min_merge, - int count) { +void testPartialMerge(Counters* counters, DB* db, size_t max_merge, + size_t min_merge, size_t count) { FlushOptions o; o.wait = true; @@ -339,16 +342,16 @@ void testPartialMerge(Counters* counters, DB* db, int max_merge, int min_merge, // operands exceeds the threshold. uint64_t tmp_sum = 0; resetNumPartialMergeCalls(); - for (int i = 1; i <= count; i++) { + for (size_t i = 1; i <= count; i++) { counters->assert_add("b", i); tmp_sum += i; } db->Flush(o); - db->CompactRange(nullptr, nullptr); + db->CompactRange(CompactRangeOptions(), nullptr, nullptr); ASSERT_EQ(tmp_sum, counters->assert_get("b")); if (count > max_merge) { // in this case, FullMerge should be called instead. - ASSERT_EQ(num_partial_merge_calls, 0); + ASSERT_EQ(num_partial_merge_calls, 0U); } else { // if count >= min_merge, then partial merge should be called once. ASSERT_EQ((count >= min_merge), (num_partial_merge_calls == 1)); @@ -358,29 +361,29 @@ void testPartialMerge(Counters* counters, DB* db, int max_merge, int min_merge, resetNumPartialMergeCalls(); tmp_sum = 0; db->Put(rocksdb::WriteOptions(), "c", "10"); - for (int i = 1; i <= count; i++) { + for (size_t i = 1; i <= count; i++) { counters->assert_add("c", i); tmp_sum += i; } db->Flush(o); - db->CompactRange(nullptr, nullptr); + db->CompactRange(CompactRangeOptions(), nullptr, nullptr); ASSERT_EQ(tmp_sum, counters->assert_get("c")); - ASSERT_EQ(num_partial_merge_calls, 0); + ASSERT_EQ(num_partial_merge_calls, 0U); } -void testSingleBatchSuccessiveMerge( - DB* db, - int max_num_merges, - int num_merges) { +void testSingleBatchSuccessiveMerge(DB* db, size_t max_num_merges, + size_t num_merges) { assert(num_merges > max_num_merges); Slice key("BatchSuccessiveMerge"); uint64_t merge_value = 1; - Slice merge_value_slice((char *)&merge_value, sizeof(merge_value)); + char buf[sizeof(merge_value)]; + EncodeFixed64(buf, merge_value); + Slice merge_value_slice(buf, sizeof(merge_value)); // Create the batch WriteBatch batch; - for (int i = 0; i < num_merges; ++i) { + for (size_t i = 0; i < num_merges; ++i) { batch.Merge(key, merge_value_slice); } @@ -390,12 +393,13 @@ void testSingleBatchSuccessiveMerge( Status s = db->Write(WriteOptions(), &batch); assert(s.ok()); } - assert(numMergeOperatorCalls == - num_merges - (num_merges % (max_num_merges + 1))); + ASSERT_EQ( + num_merge_operator_calls, + static_cast(num_merges - (num_merges % (max_num_merges + 1)))); // Get the value resetNumMergeOperatorCalls(); - string get_value_str; + std::string get_value_str; { Status s = db->Get(ReadOptions(), key, &get_value_str); assert(s.ok()); @@ -403,35 +407,37 @@ void testSingleBatchSuccessiveMerge( assert(get_value_str.size() == sizeof(uint64_t)); uint64_t get_value = DecodeFixed64(&get_value_str[0]); ASSERT_EQ(get_value, num_merges * merge_value); - ASSERT_EQ(numMergeOperatorCalls, (num_merges % (max_num_merges + 1))); + ASSERT_EQ(num_merge_operator_calls, + static_cast((num_merges % (max_num_merges + 1)))); } -void runTest(int argc, const string& dbname, const bool use_ttl = false) { - auto db = OpenDb(dbname, use_ttl); - - { - cout << "Test read-modify-write counters... \n"; - Counters counters(db, 0); - testCounters(counters, db.get(), true); - } - +void runTest(int argc, const std::string& dbname, const bool use_ttl = false) { bool compact = false; if (argc > 1) { compact = true; - cout << "Turn on Compaction\n"; + std::cout << "Turn on Compaction\n"; } { - cout << "Test merge-based counters... \n"; - MergeBasedCounters counters(db, 0); - testCounters(counters, db.get(), compact); + auto db = OpenDb(dbname, use_ttl); + + { + std::cout << "Test read-modify-write counters... \n"; + Counters counters(db, 0); + testCounters(counters, db.get(), true); + } + + { + std::cout << "Test merge-based counters... \n"; + MergeBasedCounters counters(db, 0); + testCounters(counters, db.get(), compact); + } } DestroyDB(dbname, Options()); - db.reset(); { - cout << "Test merge in memtable... \n"; + std::cout << "Test merge in memtable... \n"; size_t max_merge = 5; auto db = OpenDb(dbname, use_ttl, max_merge); MergeBasedCounters counters(db, 0); @@ -442,34 +448,34 @@ void runTest(int argc, const string& dbname, const bool use_ttl = false) { } { - cout << "Test Partial-Merge\n"; + std::cout << "Test Partial-Merge\n"; size_t max_merge = 100; - for (uint32_t min_merge = 5; min_merge < 25; min_merge += 5) { - for (uint32_t count = min_merge - 1; count <= min_merge + 1; count++) { - auto db = OpenDb(dbname, use_ttl, max_merge, min_merge); - MergeBasedCounters counters(db, 0); - testPartialMerge(&counters, db.get(), max_merge, min_merge, count); - DestroyDB(dbname, Options()); - } - { - auto db = OpenDb(dbname, use_ttl, max_merge, min_merge); - MergeBasedCounters counters(db, 0); - testPartialMerge(&counters, db.get(), max_merge, min_merge, - min_merge * 10); - DestroyDB(dbname, Options()); - } + // Min merge is hard-coded to 2. + uint32_t min_merge = 2; + for (uint32_t count = min_merge - 1; count <= min_merge + 1; count++) { + auto db = OpenDb(dbname, use_ttl, max_merge); + MergeBasedCounters counters(db, 0); + testPartialMerge(&counters, db.get(), max_merge, min_merge, count); + DestroyDB(dbname, Options()); + } + { + auto db = OpenDb(dbname, use_ttl, max_merge); + MergeBasedCounters counters(db, 0); + testPartialMerge(&counters, db.get(), max_merge, min_merge, + min_merge * 10); + DestroyDB(dbname, Options()); } } { - cout << "Test merge-operator not set after reopen\n"; + std::cout << "Test merge-operator not set after reopen\n"; { auto db = OpenDb(dbname); MergeBasedCounters counters(db, 0); counters.add("test-key", 1); counters.add("test-key", 1); counters.add("test-key", 1); - db->CompactRange(nullptr, nullptr); + db->CompactRange(CompactRangeOptions(), nullptr, nullptr); } DB* reopen_db; @@ -482,7 +488,7 @@ void runTest(int argc, const string& dbname, const bool use_ttl = false) { /* Temporary remove this test { - cout << "Test merge-operator not set after reopen (recovery case)\n"; + std::cout << "Test merge-operator not set after reopen (recovery case)\n"; { auto db = OpenDb(dbname); MergeBasedCounters counters(db, 0); @@ -500,8 +506,12 @@ void runTest(int argc, const string& dbname, const bool use_ttl = false) { int main(int argc, char *argv[]) { //TODO: Make this test like a general rocksdb unit-test + rocksdb::port::InstallStackTraceHandler(); runTest(argc, test::TmpDir() + "/merge_testdb"); +// DBWithTTL is not supported in ROCKSDB_LITE +#ifndef ROCKSDB_LITE runTest(argc, test::TmpDir() + "/merge_testdbttl", true); // Run test on TTL database +#endif // !ROCKSDB_LITE printf("Passed all tests!\n"); return 0; } diff --git a/db/options_file_test.cc b/db/options_file_test.cc new file mode 100644 index 00000000000..fc62840eb47 --- /dev/null +++ b/db/options_file_test.cc @@ -0,0 +1,119 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE +#include + +#include "db/db_impl.h" +#include "db/db_test_util.h" +#include "rocksdb/options.h" +#include "rocksdb/table.h" +#include "util/testharness.h" + +namespace rocksdb { +class OptionsFileTest : public testing::Test { + public: + OptionsFileTest() : dbname_(test::TmpDir() + "/options_file_test") {} + + std::string dbname_; +}; + +namespace { +void UpdateOptionsFiles(DB* db, + std::unordered_set* filename_history, + int* options_files_count) { + std::vector filenames; + db->GetEnv()->GetChildren(db->GetName(), &filenames); + uint64_t number; + FileType type; + *options_files_count = 0; + for (auto filename : filenames) { + if (ParseFileName(filename, &number, &type) && type == kOptionsFile) { + filename_history->insert(filename); + (*options_files_count)++; + } + } +} + +// Verify whether the current Options Files are the latest ones. +void VerifyOptionsFileName( + DB* db, const std::unordered_set& past_filenames) { + std::vector filenames; + std::unordered_set current_filenames; + db->GetEnv()->GetChildren(db->GetName(), &filenames); + uint64_t number; + FileType type; + for (auto filename : filenames) { + if (ParseFileName(filename, &number, &type) && type == kOptionsFile) { + current_filenames.insert(filename); + } + } + for (auto past_filename : past_filenames) { + if (current_filenames.find(past_filename) != current_filenames.end()) { + continue; + } + for (auto filename : current_filenames) { + ASSERT_GT(filename, past_filename); + } + } +} +} // namespace + +TEST_F(OptionsFileTest, NumberOfOptionsFiles) { + const int kReopenCount = 20; + Options opt; + opt.create_if_missing = true; + DestroyDB(dbname_, opt); + std::unordered_set filename_history; + DB* db; + for (int i = 0; i < kReopenCount; ++i) { + ASSERT_OK(DB::Open(opt, dbname_, &db)); + int num_options_files = 0; + UpdateOptionsFiles(db, &filename_history, &num_options_files); + ASSERT_GT(num_options_files, 0); + ASSERT_LE(num_options_files, 2); + // Make sure we always keep the latest option files. + VerifyOptionsFileName(db, filename_history); + delete db; + } +} + +TEST_F(OptionsFileTest, OptionsFileName) { + const uint64_t kOptionsFileNum = 12345; + uint64_t number; + FileType type; + + auto options_file_name = OptionsFileName("", kOptionsFileNum); + ASSERT_TRUE(ParseFileName(options_file_name, &number, &type, nullptr)); + ASSERT_EQ(type, kOptionsFile); + ASSERT_EQ(number, kOptionsFileNum); + + const uint64_t kTempOptionsFileNum = 54352; + auto temp_options_file_name = TempOptionsFileName("", kTempOptionsFileNum); + ASSERT_TRUE(ParseFileName(temp_options_file_name, &number, &type, nullptr)); + ASSERT_NE(temp_options_file_name.find(kTempFileNameSuffix), + std::string::npos); + ASSERT_EQ(type, kTempFile); + ASSERT_EQ(number, kTempOptionsFileNum); +} +} // namespace rocksdb + +int main(int argc, char** argv) { +#if !(defined NDEBUG) || !defined(OS_WIN) + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +#else + return 0; +#endif // !(defined NDEBUG) || !defined(OS_WIN) +} +#else + +#include + +int main(int argc, char** argv) { + printf("Skipped as Options file is not supported in RocksDBLite.\n"); + return 0; +} +#endif // !ROCKSDB_LITE diff --git a/db/perf_context_test.cc b/db/perf_context_test.cc index a182fb52143..d06843a8303 100644 --- a/db/perf_context_test.cc +++ b/db/perf_context_test.cc @@ -1,21 +1,25 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #include #include +#include #include -#include "/usr/include/valgrind/callgrind.h" +#include "monitoring/histogram.h" +#include "monitoring/instrumented_mutex.h" +#include "monitoring/thread_status_util.h" +#include "port/port.h" #include "rocksdb/db.h" +#include "rocksdb/memtablerep.h" #include "rocksdb/perf_context.h" #include "rocksdb/slice_transform.h" -#include "rocksdb/memtablerep.h" -#include "util/histogram.h" #include "util/stop_watch.h" +#include "util/string_util.h" #include "util/testharness.h" - +#include "utilities/merge_operators.h" bool FLAGS_random_key = false; bool FLAGS_use_set_based_memetable = false; @@ -23,112 +27,137 @@ int FLAGS_total_keys = 100; int FLAGS_write_buffer_size = 1000000000; int FLAGS_max_write_buffer_number = 8; int FLAGS_min_write_buffer_number_to_merge = 7; +bool FLAGS_verbose = false; // Path to the database on file system const std::string kDbName = rocksdb::test::TmpDir() + "/perf_context_test"; namespace rocksdb { -std::shared_ptr OpenDb() { +std::shared_ptr OpenDb(bool read_only = false) { DB* db; Options options; options.create_if_missing = true; + options.max_open_files = -1; options.write_buffer_size = FLAGS_write_buffer_size; options.max_write_buffer_number = FLAGS_max_write_buffer_number; options.min_write_buffer_number_to_merge = FLAGS_min_write_buffer_number_to_merge; if (FLAGS_use_set_based_memetable) { - auto prefix_extractor = rocksdb::NewFixedPrefixTransform(0); - options.memtable_factory.reset( - NewHashSkipListRepFactory(prefix_extractor)); +#ifndef ROCKSDB_LITE + options.prefix_extractor.reset(rocksdb::NewFixedPrefixTransform(0)); + options.memtable_factory.reset(NewHashSkipListRepFactory()); +#endif // ROCKSDB_LITE } - Status s = DB::Open(options, kDbName, &db); - ASSERT_OK(s); + Status s; + if (!read_only) { + s = DB::Open(options, kDbName, &db); + } else { + s = DB::OpenForReadOnly(options, kDbName, &db); + } + EXPECT_OK(s); return std::shared_ptr(db); } -class PerfContextTest { }; +class PerfContextTest : public testing::Test {}; -TEST(PerfContextTest, SeekIntoDeletion) { +TEST_F(PerfContextTest, SeekIntoDeletion) { DestroyDB(kDbName, Options()); auto db = OpenDb(); WriteOptions write_options; ReadOptions read_options; for (int i = 0; i < FLAGS_total_keys; ++i) { - std::string key = "k" + std::to_string(i); - std::string value = "v" + std::to_string(i); + std::string key = "k" + ToString(i); + std::string value = "v" + ToString(i); db->Put(write_options, key, value); } for (int i = 0; i < FLAGS_total_keys -1 ; ++i) { - std::string key = "k" + std::to_string(i); + std::string key = "k" + ToString(i); db->Delete(write_options, key); } HistogramImpl hist_get; HistogramImpl hist_get_time; for (int i = 0; i < FLAGS_total_keys - 1; ++i) { - std::string key = "k" + std::to_string(i); + std::string key = "k" + ToString(i); std::string value; - perf_context.Reset(); - StopWatchNano timer(Env::Default(), true); + get_perf_context()->Reset(); + StopWatchNano timer(Env::Default()); + timer.Start(); auto status = db->Get(read_options, key, &value); auto elapsed_nanos = timer.ElapsedNanos(); ASSERT_TRUE(status.IsNotFound()); - hist_get.Add(perf_context.user_key_comparison_count); + hist_get.Add(get_perf_context()->user_key_comparison_count); hist_get_time.Add(elapsed_nanos); } - std::cout << "Get uesr key comparison: \n" << hist_get.ToString() - << "Get time: \n" << hist_get_time.ToString(); + if (FLAGS_verbose) { + std::cout << "Get user key comparison: \n" << hist_get.ToString() + << "Get time: \n" << hist_get_time.ToString(); + } - HistogramImpl hist_seek_to_first; - std::unique_ptr iter(db->NewIterator(read_options)); + { + HistogramImpl hist_seek_to_first; + std::unique_ptr iter(db->NewIterator(read_options)); - perf_context.Reset(); - StopWatchNano timer(Env::Default(), true); - iter->SeekToFirst(); - hist_seek_to_first.Add(perf_context.user_key_comparison_count); - auto elapsed_nanos = timer.ElapsedNanos(); + get_perf_context()->Reset(); + StopWatchNano timer(Env::Default(), true); + iter->SeekToFirst(); + hist_seek_to_first.Add(get_perf_context()->user_key_comparison_count); + auto elapsed_nanos = timer.ElapsedNanos(); - std::cout << "SeekToFirst uesr key comparison: \n" << hist_seek_to_first.ToString() - << "ikey skipped: " << perf_context.internal_key_skipped_count << "\n" - << "idelete skipped: " << perf_context.internal_delete_skipped_count << "\n" - << "elapsed: " << elapsed_nanos << "\n"; + if (FLAGS_verbose) { + std::cout << "SeekToFirst uesr key comparison: \n" + << hist_seek_to_first.ToString() + << "ikey skipped: " << get_perf_context()->internal_key_skipped_count + << "\n" + << "idelete skipped: " + << get_perf_context()->internal_delete_skipped_count << "\n" + << "elapsed: " << elapsed_nanos << "\n"; + } + } HistogramImpl hist_seek; for (int i = 0; i < FLAGS_total_keys; ++i) { std::unique_ptr iter(db->NewIterator(read_options)); - std::string key = "k" + std::to_string(i); + std::string key = "k" + ToString(i); - perf_context.Reset(); + get_perf_context()->Reset(); StopWatchNano timer(Env::Default(), true); iter->Seek(key); auto elapsed_nanos = timer.ElapsedNanos(); - hist_seek.Add(perf_context.user_key_comparison_count); - std::cout << "seek cmp: " << perf_context.user_key_comparison_count - << " ikey skipped " << perf_context.internal_key_skipped_count - << " idelete skipped " << perf_context.internal_delete_skipped_count - << " elapsed: " << elapsed_nanos << "ns\n"; + hist_seek.Add(get_perf_context()->user_key_comparison_count); + if (FLAGS_verbose) { + std::cout << "seek cmp: " << get_perf_context()->user_key_comparison_count + << " ikey skipped " << get_perf_context()->internal_key_skipped_count + << " idelete skipped " + << get_perf_context()->internal_delete_skipped_count + << " elapsed: " << elapsed_nanos << "ns\n"; + } - perf_context.Reset(); + get_perf_context()->Reset(); ASSERT_TRUE(iter->Valid()); StopWatchNano timer2(Env::Default(), true); iter->Next(); auto elapsed_nanos2 = timer2.ElapsedNanos(); - std::cout << "next cmp: " << perf_context.user_key_comparison_count - << "elapsed: " << elapsed_nanos2 << "ns\n"; + if (FLAGS_verbose) { + std::cout << "next cmp: " << get_perf_context()->user_key_comparison_count + << "elapsed: " << elapsed_nanos2 << "ns\n"; + } } - std::cout << "Seek uesr key comparison: \n" << hist_seek.ToString(); + if (FLAGS_verbose) { + std::cout << "Seek uesr key comparison: \n" << hist_seek.ToString(); + } } -TEST(PerfContextTest, StopWatchNanoOverhead) { +TEST_F(PerfContextTest, StopWatchNanoOverhead) { // profile the timer cost by itself! const int kTotalIterations = 1000000; std::vector timings(kTotalIterations); @@ -143,17 +172,20 @@ TEST(PerfContextTest, StopWatchNanoOverhead) { histogram.Add(timing); } - std::cout << histogram.ToString(); + if (FLAGS_verbose) { + std::cout << histogram.ToString(); + } } -TEST(PerfContextTest, StopWatchOverhead) { +TEST_F(PerfContextTest, StopWatchOverhead) { // profile the timer cost by itself! const int kTotalIterations = 1000000; + uint64_t elapsed = 0; std::vector timings(kTotalIterations); - StopWatch timer(Env::Default()); + StopWatch timer(Env::Default(), nullptr, 0, &elapsed); for (auto& timing : timings) { - timing = timer.ElapsedMicros(); + timing = elapsed; } HistogramImpl histogram; @@ -163,10 +195,12 @@ TEST(PerfContextTest, StopWatchOverhead) { prev_timing = timing; } - std::cout << histogram.ToString(); + if (FLAGS_verbose) { + std::cout << histogram.ToString(); + } } -void ProfileKeyComparison() { +void ProfileQueries(bool enabled_time = false) { DestroyDB(kDbName, Options()); // Start this test with a fresh DB auto db = OpenDb(); @@ -175,75 +209,267 @@ void ProfileKeyComparison() { ReadOptions read_options; HistogramImpl hist_put; + HistogramImpl hist_get; HistogramImpl hist_get_snapshot; HistogramImpl hist_get_memtable; + HistogramImpl hist_get_files; HistogramImpl hist_get_post_process; HistogramImpl hist_num_memtable_checked; + + HistogramImpl hist_mget; + HistogramImpl hist_mget_snapshot; + HistogramImpl hist_mget_memtable; + HistogramImpl hist_mget_files; + HistogramImpl hist_mget_post_process; + HistogramImpl hist_mget_num_memtable_checked; + HistogramImpl hist_write_pre_post; HistogramImpl hist_write_wal_time; HistogramImpl hist_write_memtable_time; - std::cout << "Inserting " << FLAGS_total_keys << " key/value pairs\n...\n"; + uint64_t total_db_mutex_nanos = 0; + + if (FLAGS_verbose) { + std::cout << "Inserting " << FLAGS_total_keys << " key/value pairs\n...\n"; + } std::vector keys; + const int kFlushFlag = -1; for (int i = 0; i < FLAGS_total_keys; ++i) { keys.push_back(i); + if (i == FLAGS_total_keys / 2) { + // Issuing a flush in the middle. + keys.push_back(kFlushFlag); + } } if (FLAGS_random_key) { std::random_shuffle(keys.begin(), keys.end()); } - +#ifndef NDEBUG + ThreadStatusUtil::TEST_SetStateDelay(ThreadStatus::STATE_MUTEX_WAIT, 1U); +#endif + int num_mutex_waited = 0; for (const int i : keys) { - std::string key = "k" + std::to_string(i); - std::string value = "v" + std::to_string(i); + if (i == kFlushFlag) { + FlushOptions fo; + db->Flush(fo); + continue; + } + + std::string key = "k" + ToString(i); + std::string value = "v" + ToString(i); - perf_context.Reset(); + std::vector values; + + get_perf_context()->Reset(); db->Put(write_options, key, value); - hist_write_pre_post.Add(perf_context.write_pre_and_post_process_time); - hist_write_wal_time.Add(perf_context.write_wal_time); - hist_write_memtable_time.Add(perf_context.write_memtable_time); - hist_put.Add(perf_context.user_key_comparison_count); - - perf_context.Reset(); - db->Get(read_options, key, &value); - hist_get_snapshot.Add(perf_context.get_snapshot_time); - hist_get_memtable.Add(perf_context.get_from_memtable_time); - hist_num_memtable_checked.Add(perf_context.get_from_memtable_count); - hist_get_post_process.Add(perf_context.get_post_process_time); - hist_get.Add(perf_context.user_key_comparison_count); - } - - std::cout << "Put uesr key comparison: \n" << hist_put.ToString() - << "Get uesr key comparison: \n" << hist_get.ToString(); - std::cout << "Put(): Pre and Post Process Time: \n" - << hist_write_pre_post.ToString() - << " Writing WAL time: \n" - << hist_write_wal_time.ToString() << "\n" - << " Writing Mem Table time: \n" - << hist_write_memtable_time.ToString() << "\n"; - - std::cout << "Get(): Time to get snapshot: \n" - << hist_get_snapshot.ToString() - << " Time to get value from memtables: \n" - << hist_get_memtable.ToString() << "\n" - << " Number of memtables checked: \n" - << hist_num_memtable_checked.ToString() << "\n" - << " Time to post process: \n" - << hist_get_post_process.ToString() << "\n"; + if (++num_mutex_waited > 3) { +#ifndef NDEBUG + ThreadStatusUtil::TEST_SetStateDelay(ThreadStatus::STATE_MUTEX_WAIT, 0U); +#endif + } + hist_write_pre_post.Add(get_perf_context()->write_pre_and_post_process_time); + hist_write_wal_time.Add(get_perf_context()->write_wal_time); + hist_write_memtable_time.Add(get_perf_context()->write_memtable_time); + hist_put.Add(get_perf_context()->user_key_comparison_count); + total_db_mutex_nanos += get_perf_context()->db_mutex_lock_nanos; + } +#ifndef NDEBUG + ThreadStatusUtil::TEST_SetStateDelay(ThreadStatus::STATE_MUTEX_WAIT, 0U); +#endif + + for (const int i : keys) { + if (i == kFlushFlag) { + continue; + } + std::string key = "k" + ToString(i); + std::string expected_value = "v" + ToString(i); + std::string value; + + std::vector multiget_keys = {Slice(key)}; + std::vector values; + + get_perf_context()->Reset(); + ASSERT_OK(db->Get(read_options, key, &value)); + ASSERT_EQ(expected_value, value); + hist_get_snapshot.Add(get_perf_context()->get_snapshot_time); + hist_get_memtable.Add(get_perf_context()->get_from_memtable_time); + hist_get_files.Add(get_perf_context()->get_from_output_files_time); + hist_num_memtable_checked.Add(get_perf_context()->get_from_memtable_count); + hist_get_post_process.Add(get_perf_context()->get_post_process_time); + hist_get.Add(get_perf_context()->user_key_comparison_count); + + get_perf_context()->Reset(); + db->MultiGet(read_options, multiget_keys, &values); + hist_mget_snapshot.Add(get_perf_context()->get_snapshot_time); + hist_mget_memtable.Add(get_perf_context()->get_from_memtable_time); + hist_mget_files.Add(get_perf_context()->get_from_output_files_time); + hist_mget_num_memtable_checked.Add(get_perf_context()->get_from_memtable_count); + hist_mget_post_process.Add(get_perf_context()->get_post_process_time); + hist_mget.Add(get_perf_context()->user_key_comparison_count); + } + + if (FLAGS_verbose) { + std::cout << "Put uesr key comparison: \n" << hist_put.ToString() + << "Get uesr key comparison: \n" << hist_get.ToString() + << "MultiGet uesr key comparison: \n" << hist_get.ToString(); + std::cout << "Put(): Pre and Post Process Time: \n" + << hist_write_pre_post.ToString() << " Writing WAL time: \n" + << hist_write_wal_time.ToString() << "\n" + << " Writing Mem Table time: \n" + << hist_write_memtable_time.ToString() << "\n" + << " Total DB mutex nanos: \n" << total_db_mutex_nanos << "\n"; + + std::cout << "Get(): Time to get snapshot: \n" + << hist_get_snapshot.ToString() + << " Time to get value from memtables: \n" + << hist_get_memtable.ToString() << "\n" + << " Time to get value from output files: \n" + << hist_get_files.ToString() << "\n" + << " Number of memtables checked: \n" + << hist_num_memtable_checked.ToString() << "\n" + << " Time to post process: \n" << hist_get_post_process.ToString() + << "\n"; + + std::cout << "MultiGet(): Time to get snapshot: \n" + << hist_mget_snapshot.ToString() + << " Time to get value from memtables: \n" + << hist_mget_memtable.ToString() << "\n" + << " Time to get value from output files: \n" + << hist_mget_files.ToString() << "\n" + << " Number of memtables checked: \n" + << hist_mget_num_memtable_checked.ToString() << "\n" + << " Time to post process: \n" + << hist_mget_post_process.ToString() << "\n"; + } + + if (enabled_time) { + ASSERT_GT(hist_get.Average(), 0); + ASSERT_GT(hist_get_snapshot.Average(), 0); + ASSERT_GT(hist_get_memtable.Average(), 0); + ASSERT_GT(hist_get_files.Average(), 0); + ASSERT_GT(hist_get_post_process.Average(), 0); + ASSERT_GT(hist_num_memtable_checked.Average(), 0); + + ASSERT_GT(hist_mget.Average(), 0); + ASSERT_GT(hist_mget_snapshot.Average(), 0); + ASSERT_GT(hist_mget_memtable.Average(), 0); + ASSERT_GT(hist_mget_files.Average(), 0); + ASSERT_GT(hist_mget_post_process.Average(), 0); + ASSERT_GT(hist_mget_num_memtable_checked.Average(), 0); +#ifndef NDEBUG + ASSERT_GT(total_db_mutex_nanos, 2000U); +#endif + } + + db.reset(); + db = OpenDb(true); + + hist_get.Clear(); + hist_get_snapshot.Clear(); + hist_get_memtable.Clear(); + hist_get_files.Clear(); + hist_get_post_process.Clear(); + hist_num_memtable_checked.Clear(); + + hist_mget.Clear(); + hist_mget_snapshot.Clear(); + hist_mget_memtable.Clear(); + hist_mget_files.Clear(); + hist_mget_post_process.Clear(); + hist_mget_num_memtable_checked.Clear(); + + for (const int i : keys) { + if (i == kFlushFlag) { + continue; + } + std::string key = "k" + ToString(i); + std::string expected_value = "v" + ToString(i); + std::string value; + + std::vector multiget_keys = {Slice(key)}; + std::vector values; + + get_perf_context()->Reset(); + ASSERT_OK(db->Get(read_options, key, &value)); + ASSERT_EQ(expected_value, value); + hist_get_snapshot.Add(get_perf_context()->get_snapshot_time); + hist_get_memtable.Add(get_perf_context()->get_from_memtable_time); + hist_get_files.Add(get_perf_context()->get_from_output_files_time); + hist_num_memtable_checked.Add(get_perf_context()->get_from_memtable_count); + hist_get_post_process.Add(get_perf_context()->get_post_process_time); + hist_get.Add(get_perf_context()->user_key_comparison_count); + + get_perf_context()->Reset(); + db->MultiGet(read_options, multiget_keys, &values); + hist_mget_snapshot.Add(get_perf_context()->get_snapshot_time); + hist_mget_memtable.Add(get_perf_context()->get_from_memtable_time); + hist_mget_files.Add(get_perf_context()->get_from_output_files_time); + hist_mget_num_memtable_checked.Add(get_perf_context()->get_from_memtable_count); + hist_mget_post_process.Add(get_perf_context()->get_post_process_time); + hist_mget.Add(get_perf_context()->user_key_comparison_count); + } + + if (FLAGS_verbose) { + std::cout << "ReadOnly Get uesr key comparison: \n" << hist_get.ToString() + << "ReadOnly MultiGet uesr key comparison: \n" + << hist_mget.ToString(); + + std::cout << "ReadOnly Get(): Time to get snapshot: \n" + << hist_get_snapshot.ToString() + << " Time to get value from memtables: \n" + << hist_get_memtable.ToString() << "\n" + << " Time to get value from output files: \n" + << hist_get_files.ToString() << "\n" + << " Number of memtables checked: \n" + << hist_num_memtable_checked.ToString() << "\n" + << " Time to post process: \n" << hist_get_post_process.ToString() + << "\n"; + + std::cout << "ReadOnly MultiGet(): Time to get snapshot: \n" + << hist_mget_snapshot.ToString() + << " Time to get value from memtables: \n" + << hist_mget_memtable.ToString() << "\n" + << " Time to get value from output files: \n" + << hist_mget_files.ToString() << "\n" + << " Number of memtables checked: \n" + << hist_mget_num_memtable_checked.ToString() << "\n" + << " Time to post process: \n" + << hist_mget_post_process.ToString() << "\n"; + } + + if (enabled_time) { + ASSERT_GT(hist_get.Average(), 0); + ASSERT_GT(hist_get_memtable.Average(), 0); + ASSERT_GT(hist_get_files.Average(), 0); + ASSERT_GT(hist_num_memtable_checked.Average(), 0); + // In read-only mode Get(), no super version operation is needed + ASSERT_EQ(hist_get_post_process.Average(), 0); + ASSERT_EQ(hist_get_snapshot.Average(), 0); + + ASSERT_GT(hist_mget.Average(), 0); + ASSERT_GT(hist_mget_snapshot.Average(), 0); + ASSERT_GT(hist_mget_memtable.Average(), 0); + ASSERT_GT(hist_mget_files.Average(), 0); + ASSERT_GT(hist_mget_post_process.Average(), 0); + ASSERT_GT(hist_mget_num_memtable_checked.Average(), 0); + } } -TEST(PerfContextTest, KeyComparisonCount) { +#ifndef ROCKSDB_LITE +TEST_F(PerfContextTest, KeyComparisonCount) { SetPerfLevel(kEnableCount); - ProfileKeyComparison(); + ProfileQueries(); SetPerfLevel(kDisable); - ProfileKeyComparison(); + ProfileQueries(); SetPerfLevel(kEnableTime); - ProfileKeyComparison(); + ProfileQueries(true); } +#endif // ROCKSDB_LITE // make perf_context_test // export ROCKSDB_TESTS=PerfContextTest.SeekKeyComparison @@ -257,13 +483,15 @@ TEST(PerfContextTest, KeyComparisonCount) { // memtable. When there are two memtables, even the avg Seek Key comparison // starts to become linear to the input size. -TEST(PerfContextTest, SeekKeyComparison) { +TEST_F(PerfContextTest, SeekKeyComparison) { DestroyDB(kDbName, Options()); auto db = OpenDb(); WriteOptions write_options; ReadOptions read_options; - std::cout << "Inserting " << FLAGS_total_keys << " key/value pairs\n...\n"; + if (FLAGS_verbose) { + std::cout << "Inserting " << FLAGS_total_keys << " key/value pairs\n...\n"; + } std::vector keys; for (int i = 0; i < FLAGS_total_keys; ++i) { @@ -281,51 +509,165 @@ TEST(PerfContextTest, SeekKeyComparison) { SetPerfLevel(kEnableTime); StopWatchNano timer(Env::Default()); for (const int i : keys) { - std::string key = "k" + std::to_string(i); - std::string value = "v" + std::to_string(i); + std::string key = "k" + ToString(i); + std::string value = "v" + ToString(i); - perf_context.Reset(); + get_perf_context()->Reset(); timer.Start(); db->Put(write_options, key, value); auto put_time = timer.ElapsedNanos(); hist_put_time.Add(put_time); - hist_wal_time.Add(perf_context.write_wal_time); - hist_time_diff.Add(put_time - perf_context.write_wal_time); + hist_wal_time.Add(get_perf_context()->write_wal_time); + hist_time_diff.Add(put_time - get_perf_context()->write_wal_time); } - std::cout << "Put time:\n" << hist_put_time.ToString() - << "WAL time:\n" << hist_wal_time.ToString() - << "time diff:\n" << hist_time_diff.ToString(); + if (FLAGS_verbose) { + std::cout << "Put time:\n" << hist_put_time.ToString() << "WAL time:\n" + << hist_wal_time.ToString() << "time diff:\n" + << hist_time_diff.ToString(); + } HistogramImpl hist_seek; HistogramImpl hist_next; for (int i = 0; i < FLAGS_total_keys; ++i) { - std::string key = "k" + std::to_string(i); - std::string value = "v" + std::to_string(i); + std::string key = "k" + ToString(i); + std::string value = "v" + ToString(i); std::unique_ptr iter(db->NewIterator(read_options)); - perf_context.Reset(); + get_perf_context()->Reset(); iter->Seek(key); ASSERT_TRUE(iter->Valid()); ASSERT_EQ(iter->value().ToString(), value); - hist_seek.Add(perf_context.user_key_comparison_count); + hist_seek.Add(get_perf_context()->user_key_comparison_count); } std::unique_ptr iter(db->NewIterator(read_options)); for (iter->SeekToFirst(); iter->Valid();) { - perf_context.Reset(); + get_perf_context()->Reset(); iter->Next(); - hist_next.Add(perf_context.user_key_comparison_count); + hist_next.Add(get_perf_context()->user_key_comparison_count); + } + + if (FLAGS_verbose) { + std::cout << "Seek:\n" << hist_seek.ToString() << "Next:\n" + << hist_next.ToString(); + } +} + +TEST_F(PerfContextTest, DBMutexLockCounter) { + int stats_code[] = {0, static_cast(DB_MUTEX_WAIT_MICROS)}; + for (PerfLevel perf_level : + {PerfLevel::kEnableTimeExceptForMutex, PerfLevel::kEnableTime}) { + for (int c = 0; c < 2; ++c) { + InstrumentedMutex mutex(nullptr, Env::Default(), stats_code[c]); + mutex.Lock(); + rocksdb::port::Thread child_thread([&] { + SetPerfLevel(perf_level); + get_perf_context()->Reset(); + ASSERT_EQ(get_perf_context()->db_mutex_lock_nanos, 0); + mutex.Lock(); + mutex.Unlock(); + if (perf_level == PerfLevel::kEnableTimeExceptForMutex || + stats_code[c] != DB_MUTEX_WAIT_MICROS) { + ASSERT_EQ(get_perf_context()->db_mutex_lock_nanos, 0); + } else { + // increment the counter only when it's a DB Mutex + ASSERT_GT(get_perf_context()->db_mutex_lock_nanos, 0); + } + }); + Env::Default()->SleepForMicroseconds(100); + mutex.Unlock(); + child_thread.join(); + } + } +} + +TEST_F(PerfContextTest, FalseDBMutexWait) { + SetPerfLevel(kEnableTime); + int stats_code[] = {0, static_cast(DB_MUTEX_WAIT_MICROS)}; + for (int c = 0; c < 2; ++c) { + InstrumentedMutex mutex(nullptr, Env::Default(), stats_code[c]); + InstrumentedCondVar lock(&mutex); + get_perf_context()->Reset(); + mutex.Lock(); + lock.TimedWait(100); + mutex.Unlock(); + if (stats_code[c] == static_cast(DB_MUTEX_WAIT_MICROS)) { + // increment the counter only when it's a DB Mutex + ASSERT_GT(get_perf_context()->db_condition_wait_nanos, 0); + } else { + ASSERT_EQ(get_perf_context()->db_condition_wait_nanos, 0); + } } +} + +TEST_F(PerfContextTest, ToString) { + get_perf_context()->Reset(); + get_perf_context()->block_read_count = 12345; + + std::string zero_included = get_perf_context()->ToString(); + ASSERT_NE(std::string::npos, zero_included.find("= 0")); + ASSERT_NE(std::string::npos, zero_included.find("= 12345")); - std::cout << "Seek:\n" << hist_seek.ToString() - << "Next:\n" << hist_next.ToString(); + std::string zero_excluded = get_perf_context()->ToString(true); + ASSERT_EQ(std::string::npos, zero_excluded.find("= 0")); + ASSERT_NE(std::string::npos, zero_excluded.find("= 12345")); } +TEST_F(PerfContextTest, MergeOperatorTime) { + DestroyDB(kDbName, Options()); + DB* db; + Options options; + options.create_if_missing = true; + options.merge_operator = MergeOperators::CreateStringAppendOperator(); + Status s = DB::Open(options, kDbName, &db); + EXPECT_OK(s); + + std::string val; + ASSERT_OK(db->Merge(WriteOptions(), "k1", "val1")); + ASSERT_OK(db->Merge(WriteOptions(), "k1", "val2")); + ASSERT_OK(db->Merge(WriteOptions(), "k1", "val3")); + ASSERT_OK(db->Merge(WriteOptions(), "k1", "val4")); + + SetPerfLevel(kEnableTime); + get_perf_context()->Reset(); + ASSERT_OK(db->Get(ReadOptions(), "k1", &val)); +#ifdef OS_SOLARIS + for (int i = 0; i < 100; i++) { + ASSERT_OK(db->Get(ReadOptions(), "k1", &val)); + } +#endif + EXPECT_GT(get_perf_context()->merge_operator_time_nanos, 0); + + ASSERT_OK(db->Flush(FlushOptions())); + + get_perf_context()->Reset(); + ASSERT_OK(db->Get(ReadOptions(), "k1", &val)); +#ifdef OS_SOLARIS + for (int i = 0; i < 100; i++) { + ASSERT_OK(db->Get(ReadOptions(), "k1", &val)); + } +#endif + EXPECT_GT(get_perf_context()->merge_operator_time_nanos, 0); + + ASSERT_OK(db->CompactRange(CompactRangeOptions(), nullptr, nullptr)); + + get_perf_context()->Reset(); + ASSERT_OK(db->Get(ReadOptions(), "k1", &val)); +#ifdef OS_SOLARIS + for (int i = 0; i < 100; i++) { + ASSERT_OK(db->Get(ReadOptions(), "k1", &val)); + } +#endif + EXPECT_GT(get_perf_context()->merge_operator_time_nanos, 0); + + delete db; +} } int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); for (int i = 1; i < argc; i++) { int n; @@ -349,10 +691,15 @@ int main(int argc, char** argv) { FLAGS_use_set_based_memetable = n; } + if (sscanf(argv[i], "--verbose=%d%c", &n, &junk) == 1 && + (n == 0 || n == 1)) { + FLAGS_verbose = n; + } } - std::cout << kDbName << "\n"; + if (FLAGS_verbose) { + std::cout << kDbName << "\n"; + } - rocksdb::test::RunAllTests(); - return 0; + return RUN_ALL_TESTS(); } diff --git a/db/pinned_iterators_manager.h b/db/pinned_iterators_manager.h new file mode 100644 index 00000000000..7874eef6d77 --- /dev/null +++ b/db/pinned_iterators_manager.h @@ -0,0 +1,87 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#pragma once +#include +#include +#include +#include + +#include "table/internal_iterator.h" + +namespace rocksdb { + +// PinnedIteratorsManager will be notified whenever we need to pin an Iterator +// and it will be responsible for deleting pinned Iterators when they are +// not needed anymore. +class PinnedIteratorsManager : public Cleanable { + public: + PinnedIteratorsManager() : pinning_enabled(false) {} + ~PinnedIteratorsManager() { + if (pinning_enabled) { + ReleasePinnedData(); + } + } + + // Enable Iterators pinning + void StartPinning() { + assert(pinning_enabled == false); + pinning_enabled = true; + } + + // Is pinning enabled ? + bool PinningEnabled() { return pinning_enabled; } + + // Take ownership of iter and delete it when ReleasePinnedData() is called + void PinIterator(InternalIterator* iter, bool arena = false) { + if (arena) { + PinPtr(iter, &PinnedIteratorsManager::ReleaseArenaInternalIterator); + } else { + PinPtr(iter, &PinnedIteratorsManager::ReleaseInternalIterator); + } + } + + typedef void (*ReleaseFunction)(void* arg1); + void PinPtr(void* ptr, ReleaseFunction release_func) { + assert(pinning_enabled); + if (ptr == nullptr) { + return; + } + pinned_ptrs_.emplace_back(ptr, release_func); + } + + // Release pinned Iterators + inline void ReleasePinnedData() { + assert(pinning_enabled == true); + pinning_enabled = false; + + // Remove duplicate pointers + std::sort(pinned_ptrs_.begin(), pinned_ptrs_.end()); + auto unique_end = std::unique(pinned_ptrs_.begin(), pinned_ptrs_.end()); + + for (auto i = pinned_ptrs_.begin(); i != unique_end; ++i) { + void* ptr = i->first; + ReleaseFunction release_func = i->second; + release_func(ptr); + } + pinned_ptrs_.clear(); + // Also do cleanups from the base Cleanable + Cleanable::Reset(); + } + + private: + static void ReleaseInternalIterator(void* ptr) { + delete reinterpret_cast(ptr); + } + + static void ReleaseArenaInternalIterator(void* ptr) { + reinterpret_cast(ptr)->~InternalIterator(); + } + + bool pinning_enabled; + std::vector> pinned_ptrs_; +}; + +} // namespace rocksdb diff --git a/db/plain_table_db_test.cc b/db/plain_table_db_test.cc index bb0f96f1584..0b60332e53a 100644 --- a/db/plain_table_db_test.cc +++ b/db/plain_table_db_test.cc @@ -1,18 +1,18 @@ -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef ROCKSDB_LITE + #include #include #include "db/db_impl.h" -#include "db/filename.h" #include "db/version_set.h" #include "db/write_batch_internal.h" #include "rocksdb/cache.h" @@ -22,13 +22,17 @@ #include "rocksdb/filter_policy.h" #include "rocksdb/slice_transform.h" #include "rocksdb/table.h" -#include "table/meta_blocks.h" #include "table/bloom_block.h" +#include "table/meta_blocks.h" #include "table/plain_table_factory.h" +#include "table/plain_table_key_coding.h" #include "table/plain_table_reader.h" +#include "table/table_builder.h" +#include "util/filename.h" #include "util/hash.h" #include "util/logging.h" #include "util/mutexlock.h" +#include "util/string_util.h" #include "util/testharness.h" #include "util/testutil.h" #include "utilities/merge_operators.h" @@ -36,27 +40,85 @@ using std::unique_ptr; namespace rocksdb { +class PlainTableKeyDecoderTest : public testing::Test {}; + +TEST_F(PlainTableKeyDecoderTest, ReadNonMmap) { + std::string tmp; + Random rnd(301); + const uint32_t kLength = 2222; + Slice contents = test::RandomString(&rnd, kLength, &tmp); + test::StringSource* string_source = + new test::StringSource(contents, 0, false); + + unique_ptr file_reader( + test::GetRandomAccessFileReader(string_source)); + unique_ptr file_info(new PlainTableReaderFileInfo( + std::move(file_reader), EnvOptions(), kLength)); + + { + PlainTableFileReader reader(file_info.get()); + + const uint32_t kReadSize = 77; + for (uint32_t pos = 0; pos < kLength; pos += kReadSize) { + uint32_t read_size = std::min(kLength - pos, kReadSize); + Slice out; + ASSERT_TRUE(reader.Read(pos, read_size, &out)); + ASSERT_EQ(0, out.compare(tmp.substr(pos, read_size))); + } + + ASSERT_LT(uint32_t(string_source->total_reads()), kLength / kReadSize / 2); + } + + std::vector>> reads = { + {{600, 30}, {590, 30}, {600, 20}, {600, 40}}, + {{800, 20}, {100, 20}, {500, 20}, {1500, 20}, {100, 20}, {80, 20}}, + {{1000, 20}, {500, 20}, {1000, 50}}, + {{1000, 20}, {500, 20}, {500, 20}}, + {{1000, 20}, {500, 20}, {200, 20}, {500, 20}}, + {{1000, 20}, {500, 20}, {200, 20}, {1000, 50}}, + {{600, 500}, {610, 20}, {100, 20}}, + {{500, 100}, {490, 100}, {550, 50}}, + }; + + std::vector num_file_reads = {2, 6, 2, 2, 4, 3, 2, 2}; + + for (size_t i = 0; i < reads.size(); i++) { + string_source->set_total_reads(0); + PlainTableFileReader reader(file_info.get()); + for (auto p : reads[i]) { + Slice out; + ASSERT_TRUE(reader.Read(p.first, p.second, &out)); + ASSERT_EQ(0, out.compare(tmp.substr(p.first, p.second))); + } + ASSERT_EQ(num_file_reads[i], string_source->total_reads()); + } +} -class PlainTableDBTest { +class PlainTableDBTest : public testing::Test, + public testing::WithParamInterface { protected: private: std::string dbname_; Env* env_; DB* db_; + bool mmap_mode_; Options last_options_; public: - PlainTableDBTest() : env_(Env::Default()) { - dbname_ = test::TmpDir() + "/plain_table_db_test"; - ASSERT_OK(DestroyDB(dbname_, Options())); - db_ = nullptr; - Reopen(); - } + PlainTableDBTest() : env_(Env::Default()) {} ~PlainTableDBTest() { delete db_; - ASSERT_OK(DestroyDB(dbname_, Options())); + EXPECT_OK(DestroyDB(dbname_, Options())); + } + + void SetUp() override { + mmap_mode_ = GetParam(); + dbname_ = test::TmpDir() + "/plain_table_db_test"; + EXPECT_OK(DestroyDB(dbname_, Options())); + db_ = nullptr; + Reopen(); } // Return the current option configuration. @@ -77,7 +139,8 @@ class PlainTableDBTest { options.memtable_factory.reset(NewHashLinkListRepFactory(4, 0, 3, true)); options.prefix_extractor.reset(NewFixedPrefixTransform(8)); - options.allow_mmap_reads = true; + options.allow_mmap_reads = mmap_mode_; + options.allow_concurrent_memtable_write = false; return options; } @@ -149,16 +212,15 @@ class PlainTableDBTest { int NumTableFilesAtLevel(int level) { std::string property; - ASSERT_TRUE( - db_->GetProperty("rocksdb.num-files-at-level" + NumberToString(level), - &property)); + EXPECT_TRUE(db_->GetProperty( + "rocksdb.num-files-at-level" + NumberToString(level), &property)); return atoi(property.c_str()); } // Return spread of files per level std::string FilesPerLevel() { std::string result; - int last_non_zero_offset = 0; + size_t last_non_zero_offset = 0; for (int level = 0; level < db_->NumberLevels(); level++) { int f = NumTableFilesAtLevel(level); char buf[100]; @@ -183,7 +245,7 @@ class PlainTableDBTest { } }; -TEST(PlainTableDBTest, Empty) { +TEST_P(PlainTableDBTest, Empty) { ASSERT_TRUE(dbfull() != nullptr); ASSERT_EQ("NOT_FOUND", Get("0000000000000foo")); } @@ -192,36 +254,40 @@ extern const uint64_t kPlainTableMagicNumber; class TestPlainTableReader : public PlainTableReader { public: - TestPlainTableReader(const EnvOptions& storage_options, + TestPlainTableReader(const EnvOptions& env_options, const InternalKeyComparator& icomparator, EncodingType encoding_type, uint64_t file_size, int bloom_bits_per_key, double hash_table_ratio, size_t index_sparseness, const TableProperties* table_properties, - unique_ptr&& file, - const Options& options, bool* expect_bloom_not_match, - bool store_index_in_file) - : PlainTableReader(options, std::move(file), storage_options, icomparator, + unique_ptr&& file, + const ImmutableCFOptions& ioptions, + bool* expect_bloom_not_match, bool store_index_in_file, + uint32_t column_family_id, + const std::string& column_family_name) + : PlainTableReader(ioptions, std::move(file), env_options, icomparator, encoding_type, file_size, table_properties), expect_bloom_not_match_(expect_bloom_not_match) { - Status s = MmapDataFile(); - ASSERT_TRUE(s.ok()); + Status s = MmapDataIfNeeded(); + EXPECT_TRUE(s.ok()); s = PopulateIndex(const_cast(table_properties), bloom_bits_per_key, hash_table_ratio, index_sparseness, 2 * 1024 * 1024); - ASSERT_TRUE(s.ok()); + EXPECT_TRUE(s.ok()); TableProperties* props = const_cast(table_properties); + EXPECT_EQ(column_family_id, static_cast(props->column_family_id)); + EXPECT_EQ(column_family_name, props->column_family_name); if (store_index_in_file) { auto bloom_version_ptr = props->user_collected_properties.find( PlainTablePropertyNames::kBloomVersion); - ASSERT_TRUE(bloom_version_ptr != props->user_collected_properties.end()); - ASSERT_EQ(bloom_version_ptr->second, std::string("1")); - if (options.bloom_locality > 0) { + EXPECT_TRUE(bloom_version_ptr != props->user_collected_properties.end()); + EXPECT_EQ(bloom_version_ptr->second, std::string("1")); + if (ioptions.bloom_locality > 0) { auto num_blocks_ptr = props->user_collected_properties.find( PlainTablePropertyNames::kNumBloomBlocks); - ASSERT_TRUE(num_blocks_ptr != props->user_collected_properties.end()); + EXPECT_TRUE(num_blocks_ptr != props->user_collected_properties.end()); } } } @@ -232,9 +298,9 @@ class TestPlainTableReader : public PlainTableReader { virtual bool MatchBloom(uint32_t hash) const override { bool ret = PlainTableReader::MatchBloom(hash); if (*expect_bloom_not_match_) { - ASSERT_TRUE(!ret); + EXPECT_TRUE(!ret); } else { - ASSERT_TRUE(ret); + EXPECT_TRUE(ret); } return ret; } @@ -245,35 +311,42 @@ extern const uint64_t kPlainTableMagicNumber; class TestPlainTableFactory : public PlainTableFactory { public: explicit TestPlainTableFactory(bool* expect_bloom_not_match, - const PlainTableOptions& options) + const PlainTableOptions& options, + uint32_t column_family_id, + std::string column_family_name) : PlainTableFactory(options), bloom_bits_per_key_(options.bloom_bits_per_key), hash_table_ratio_(options.hash_table_ratio), index_sparseness_(options.index_sparseness), store_index_in_file_(options.store_index_in_file), - expect_bloom_not_match_(expect_bloom_not_match) {} - - Status NewTableReader(const Options& options, const EnvOptions& soptions, - const InternalKeyComparator& internal_comparator, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table) const override { + expect_bloom_not_match_(expect_bloom_not_match), + column_family_id_(column_family_id), + column_family_name_(std::move(column_family_name)) {} + + Status NewTableReader( + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + unique_ptr* table, + bool prefetch_index_and_filter_in_cache) const override { TableProperties* props = nullptr; - auto s = ReadTableProperties(file.get(), file_size, kPlainTableMagicNumber, - options.env, options.info_log.get(), &props); - ASSERT_TRUE(s.ok()); + auto s = + ReadTableProperties(file.get(), file_size, kPlainTableMagicNumber, + table_reader_options.ioptions, &props); + EXPECT_TRUE(s.ok()); if (store_index_in_file_) { BlockHandle bloom_block_handle; s = FindMetaBlock(file.get(), file_size, kPlainTableMagicNumber, - options.env, BloomBlockBuilder::kBloomBlock, - &bloom_block_handle); - ASSERT_TRUE(s.ok()); + table_reader_options.ioptions, + BloomBlockBuilder::kBloomBlock, &bloom_block_handle); + EXPECT_TRUE(s.ok()); BlockHandle index_block_handle; - s = FindMetaBlock( - file.get(), file_size, kPlainTableMagicNumber, options.env, - PlainTableIndexBuilder::kPlainTableIndexBlock, &index_block_handle); - ASSERT_TRUE(s.ok()); + s = FindMetaBlock(file.get(), file_size, kPlainTableMagicNumber, + table_reader_options.ioptions, + PlainTableIndexBuilder::kPlainTableIndexBlock, + &index_block_handle); + EXPECT_TRUE(s.ok()); } auto& user_props = props->user_collected_properties; @@ -284,10 +357,11 @@ class TestPlainTableFactory : public PlainTableFactory { DecodeFixed32(encoding_type_prop->second.c_str())); std::unique_ptr new_reader(new TestPlainTableReader( - soptions, internal_comparator, encoding_type, file_size, + table_reader_options.env_options, + table_reader_options.internal_comparator, encoding_type, file_size, bloom_bits_per_key_, hash_table_ratio_, index_sparseness_, props, - std::move(file), options, expect_bloom_not_match_, - store_index_in_file_)); + std::move(file), table_reader_options.ioptions, expect_bloom_not_match_, + store_index_in_file_, column_family_id_, column_family_name_)); *table = std::move(new_reader); return s; @@ -299,9 +373,11 @@ class TestPlainTableFactory : public PlainTableFactory { size_t index_sparseness_; bool store_index_in_file_; bool* expect_bloom_not_match_; + const uint32_t column_family_id_; + const std::string column_family_name_; }; -TEST(PlainTableDBTest, Flush) { +TEST_P(PlainTableDBTest, Flush) { for (size_t huge_page_tlb_size = 0; huge_page_tlb_size <= 2 * 1024 * 1024; huge_page_tlb_size += 2 * 1024 * 1024) { for (EncodingType encoding_type : {kPlain, kPrefix}) { @@ -309,10 +385,6 @@ TEST(PlainTableDBTest, Flush) { for (int total_order = 0; total_order <= 1; total_order++) { for (int store_index_in_file = 0; store_index_in_file <= 1; ++store_index_in_file) { - if (!bloom_bits && store_index_in_file) { - continue; - } - Options options = CurrentOptions(); options.create_if_missing = true; // Set only one bucket to force bucket conflict. @@ -388,7 +460,7 @@ TEST(PlainTableDBTest, Flush) { } } -TEST(PlainTableDBTest, Flush2) { +TEST_P(PlainTableDBTest, Flush2) { for (size_t huge_page_tlb_size = 0; huge_page_tlb_size <= 2 * 1024 * 1024; huge_page_tlb_size += 2 * 1024 * 1024) { for (EncodingType encoding_type : {kPlain, kPrefix}) { @@ -425,7 +497,8 @@ TEST(PlainTableDBTest, Flush2) { plain_table_options.encoding_type = encoding_type; plain_table_options.store_index_in_file = store_index_in_file; options.table_factory.reset(new TestPlainTableFactory( - &expect_bloom_not_match, plain_table_options)); + &expect_bloom_not_match, plain_table_options, + 0 /* column_family_id */, kDefaultColumnFamilyName)); DestroyAndReopen(&options); ASSERT_OK(Put("0000000000000bar", "b")); @@ -468,7 +541,7 @@ TEST(PlainTableDBTest, Flush2) { } } -TEST(PlainTableDBTest, Iterator) { +TEST_P(PlainTableDBTest, Iterator) { for (size_t huge_page_tlb_size = 0; huge_page_tlb_size <= 2 * 1024 * 1024; huge_page_tlb_size += 2 * 1024 * 1024) { for (EncodingType encoding_type : {kPlain, kPrefix}) { @@ -494,7 +567,8 @@ TEST(PlainTableDBTest, Iterator) { plain_table_options.encoding_type = encoding_type; options.table_factory.reset(new TestPlainTableFactory( - &expect_bloom_not_match, plain_table_options)); + &expect_bloom_not_match, plain_table_options, + 0 /* column_family_id */, kDefaultColumnFamilyName)); } else { PlainTableOptions plain_table_options; plain_table_options.user_key_len = 16; @@ -505,7 +579,8 @@ TEST(PlainTableDBTest, Iterator) { plain_table_options.encoding_type = encoding_type; options.table_factory.reset(new TestPlainTableFactory( - &expect_bloom_not_match, plain_table_options)); + &expect_bloom_not_match, plain_table_options, + 0 /* column_family_id */, kDefaultColumnFamilyName)); } DestroyAndReopen(&options); @@ -602,7 +677,7 @@ std::string MakeLongKey(size_t length, char c) { } } // namespace -TEST(PlainTableDBTest, IteratorLargeKeys) { +TEST_P(PlainTableDBTest, IteratorLargeKeys) { Options options = CurrentOptions(); PlainTableOptions plain_table_options; @@ -626,7 +701,7 @@ TEST(PlainTableDBTest, IteratorLargeKeys) { }; for (size_t i = 0; i < 7; i++) { - ASSERT_OK(Put(key_list[i], std::to_string(i))); + ASSERT_OK(Put(key_list[i], ToString(i))); } dbfull()->TEST_FlushMemTable(); @@ -637,7 +712,7 @@ TEST(PlainTableDBTest, IteratorLargeKeys) { for (size_t i = 0; i < 7; i++) { ASSERT_TRUE(iter->Valid()); ASSERT_EQ(key_list[i], iter->key().ToString()); - ASSERT_EQ(std::to_string(i), iter->value().ToString()); + ASSERT_EQ(ToString(i), iter->value().ToString()); iter->Next(); } @@ -652,7 +727,7 @@ std::string MakeLongKeyWithPrefix(size_t length, char c) { } } // namespace -TEST(PlainTableDBTest, IteratorLargeKeysWithPrefix) { +TEST_P(PlainTableDBTest, IteratorLargeKeysWithPrefix) { Options options = CurrentOptions(); PlainTableOptions plain_table_options; @@ -674,7 +749,7 @@ TEST(PlainTableDBTest, IteratorLargeKeysWithPrefix) { MakeLongKeyWithPrefix(26, '6')}; for (size_t i = 0; i < 7; i++) { - ASSERT_OK(Put(key_list[i], std::to_string(i))); + ASSERT_OK(Put(key_list[i], ToString(i))); } dbfull()->TEST_FlushMemTable(); @@ -685,7 +760,7 @@ TEST(PlainTableDBTest, IteratorLargeKeysWithPrefix) { for (size_t i = 0; i < 7; i++) { ASSERT_TRUE(iter->Valid()); ASSERT_EQ(key_list[i], iter->key().ToString()); - ASSERT_EQ(std::to_string(i), iter->value().ToString()); + ASSERT_EQ(ToString(i), iter->value().ToString()); iter->Next(); } @@ -694,40 +769,12 @@ TEST(PlainTableDBTest, IteratorLargeKeysWithPrefix) { delete iter; } -// A test comparator which compare two strings in this way: -// (1) first compare prefix of 8 bytes in alphabet order, -// (2) if two strings share the same prefix, sort the other part of the string -// in the reverse alphabet order. -class SimpleSuffixReverseComparator : public Comparator { - public: - SimpleSuffixReverseComparator() {} - - virtual const char* Name() const { return "SimpleSuffixReverseComparator"; } - - virtual int Compare(const Slice& a, const Slice& b) const { - Slice prefix_a = Slice(a.data(), 8); - Slice prefix_b = Slice(b.data(), 8); - int prefix_comp = prefix_a.compare(prefix_b); - if (prefix_comp != 0) { - return prefix_comp; - } else { - Slice suffix_a = Slice(a.data() + 8, a.size() - 8); - Slice suffix_b = Slice(b.data() + 8, b.size() - 8); - return -(suffix_a.compare(suffix_b)); - } - } - virtual void FindShortestSeparator(std::string* start, - const Slice& limit) const {} - - virtual void FindShortSuccessor(std::string* key) const {} -}; - -TEST(PlainTableDBTest, IteratorReverseSuffixComparator) { +TEST_P(PlainTableDBTest, IteratorReverseSuffixComparator) { Options options = CurrentOptions(); options.create_if_missing = true; // Set only one bucket to force bucket conflict. // Test index interval for the same prefix to be 1, 2 and 4 - SimpleSuffixReverseComparator comp; + test::SimpleSuffixReverseComparator comp; options.comparator = ∁ DestroyAndReopen(&options); @@ -791,7 +838,7 @@ TEST(PlainTableDBTest, IteratorReverseSuffixComparator) { delete iter; } -TEST(PlainTableDBTest, HashBucketConflict) { +TEST_P(PlainTableDBTest, HashBucketConflict) { for (size_t huge_page_tlb_size = 0; huge_page_tlb_size <= 2 * 1024 * 1024; huge_page_tlb_size += 2 * 1024 * 1024) { for (unsigned char i = 1; i <= 3; i++) { @@ -884,13 +931,13 @@ TEST(PlainTableDBTest, HashBucketConflict) { } } -TEST(PlainTableDBTest, HashBucketConflictReverseSuffixComparator) { +TEST_P(PlainTableDBTest, HashBucketConflictReverseSuffixComparator) { for (size_t huge_page_tlb_size = 0; huge_page_tlb_size <= 2 * 1024 * 1024; huge_page_tlb_size += 2 * 1024 * 1024) { for (unsigned char i = 1; i <= 3; i++) { Options options = CurrentOptions(); options.create_if_missing = true; - SimpleSuffixReverseComparator comp; + test::SimpleSuffixReverseComparator comp; options.comparator = ∁ // Set only one bucket to force bucket conflict. // Test index interval for the same prefix to be 1, 2 and 4 @@ -977,7 +1024,7 @@ TEST(PlainTableDBTest, HashBucketConflictReverseSuffixComparator) { } } -TEST(PlainTableDBTest, NonExistingKeyToNonEmptyBucket) { +TEST_P(PlainTableDBTest, NonExistingKeyToNonEmptyBucket) { Options options = CurrentOptions(); options.create_if_missing = true; // Set only one bucket to force bucket conflict. @@ -1033,11 +1080,10 @@ static std::string RandomString(Random* rnd, int len) { return r; } -TEST(PlainTableDBTest, CompactionTrigger) { +TEST_P(PlainTableDBTest, CompactionTrigger) { Options options = CurrentOptions(); - options.write_buffer_size = 100 << 10; //100KB + options.write_buffer_size = 120 << 10; // 100KB options.num_levels = 3; - options.max_mem_compaction_level = 0; options.level0_file_num_compaction_trigger = 3; Reopen(&options); @@ -1046,11 +1092,12 @@ TEST(PlainTableDBTest, CompactionTrigger) { for (int num = 0; num < options.level0_file_num_compaction_trigger - 1; num++) { std::vector values; - // Write 120KB (12 values, each 10K) - for (int i = 0; i < 12; i++) { - values.push_back(RandomString(&rnd, 10000)); + // Write 120KB (10 values, each 12K) + for (int i = 0; i < 10; i++) { + values.push_back(RandomString(&rnd, 12000)); ASSERT_OK(Put(Key(i), values[i])); } + ASSERT_OK(Put(Key(999), "")); dbfull()->TEST_WaitForFlushMemTable(); ASSERT_EQ(NumTableFilesAtLevel(0), num + 1); } @@ -1061,13 +1108,14 @@ TEST(PlainTableDBTest, CompactionTrigger) { values.push_back(RandomString(&rnd, 10000)); ASSERT_OK(Put(Key(i), values[i])); } + ASSERT_OK(Put(Key(999), "")); dbfull()->TEST_WaitForCompact(); ASSERT_EQ(NumTableFilesAtLevel(0), 0); ASSERT_EQ(NumTableFilesAtLevel(1), 1); } -TEST(PlainTableDBTest, AdaptiveTable) { +TEST_P(PlainTableDBTest, AdaptiveTable) { Options options = CurrentOptions(); options.create_if_missing = true; @@ -1110,8 +1158,21 @@ TEST(PlainTableDBTest, AdaptiveTable) { ASSERT_NE("v5", Get("3000000000000bar")); } +INSTANTIATE_TEST_CASE_P(PlainTableDBTest, PlainTableDBTest, ::testing::Bool()); + } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, "SKIPPED as plain table is not supported in ROCKSDB_LITE\n"); + return 0; } + +#endif // !ROCKSDB_LITE diff --git a/db/prefix_test.cc b/db/prefix_test.cc index a69dda2b4b1..a4ed201dad1 100644 --- a/db/prefix_test.cc +++ b/db/prefix_test.cc @@ -1,13 +1,15 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE #ifndef GFLAGS #include int main() { - fprintf(stderr, "Please install gflags to run rocksdb tools\n"); - return 1; + fprintf(stderr, "Please install gflags to run this test... Skipping...\n"); + return 0; } #else @@ -16,32 +18,39 @@ int main() { #include #include +#include "db/db_impl.h" +#include "monitoring/histogram.h" #include "rocksdb/comparator.h" #include "rocksdb/db.h" +#include "rocksdb/filter_policy.h" +#include "rocksdb/memtablerep.h" #include "rocksdb/perf_context.h" #include "rocksdb/slice_transform.h" -#include "rocksdb/memtablerep.h" -#include "util/histogram.h" +#include "rocksdb/table.h" +#include "util/random.h" #include "util/stop_watch.h" +#include "util/string_util.h" #include "util/testharness.h" +#include "utilities/merge_operators.h" +#include "util/coding.h" using GFLAGS::ParseCommandLineFlags; DEFINE_bool(trigger_deadlock, false, "issue delete in range scan to trigger PrefixHashMap deadlock"); -DEFINE_uint64(bucket_count, 100000, "number of buckets"); +DEFINE_int32(bucket_count, 100000, "number of buckets"); DEFINE_uint64(num_locks, 10001, "number of locks"); DEFINE_bool(random_prefix, false, "randomize prefix"); DEFINE_uint64(total_prefixes, 100000, "total number of prefixes"); DEFINE_uint64(items_per_prefix, 1, "total number of values per prefix"); DEFINE_int64(write_buffer_size, 33554432, ""); -DEFINE_int64(max_write_buffer_number, 2, ""); -DEFINE_int64(min_write_buffer_number_to_merge, 1, ""); +DEFINE_int32(max_write_buffer_number, 2, ""); +DEFINE_int32(min_write_buffer_number_to_merge, 1, ""); DEFINE_int32(skiplist_height, 4, ""); -DEFINE_int32(memtable_prefix_bloom_bits, 10000000, ""); -DEFINE_int32(memtable_prefix_bloom_probes, 10, ""); -DEFINE_int32(memtable_prefix_bloom_huge_page_tlb_size, 2 * 1024 * 1024, ""); +DEFINE_double(memtable_prefix_bloom_size_ratio, 0.1, ""); +DEFINE_int32(memtable_huge_page_size, 2 * 1024 * 1024, ""); DEFINE_int32(value_size, 40, ""); +DEFINE_bool(enable_print, false, "Print options generated to console."); // Path to the database on file system const std::string kDbName = rocksdb::test::TmpDir() + "/prefix_test"; @@ -52,16 +61,21 @@ struct TestKey { uint64_t prefix; uint64_t sorted; - TestKey(uint64_t prefix, uint64_t sorted) : prefix(prefix), sorted(sorted) {} + TestKey(uint64_t _prefix, uint64_t _sorted) + : prefix(_prefix), sorted(_sorted) {} }; // return a slice backed by test_key -inline Slice TestKeyToSlice(const TestKey& test_key) { - return Slice((const char*)&test_key, sizeof(test_key)); +inline Slice TestKeyToSlice(std::string &s, const TestKey& test_key) { + s.clear(); + PutFixed64(&s, test_key.prefix); + PutFixed64(&s, test_key.sorted); + return Slice(s.c_str(), s.size()); } -inline const TestKey* SliceToTestKey(const Slice& slice) { - return (const TestKey*)slice.data(); +inline const TestKey SliceToTestKey(const Slice& slice) { + return TestKey(DecodeFixed64(slice.data()), + DecodeFixed64(slice.data() + 8)); } class TestKeyComparator : public Comparator { @@ -69,20 +83,22 @@ class TestKeyComparator : public Comparator { // Compare needs to be aware of the possibility of a and/or b is // prefix only - virtual int Compare(const Slice& a, const Slice& b) const { - const TestKey* key_a = SliceToTestKey(a); - const TestKey* key_b = SliceToTestKey(b); + virtual int Compare(const Slice& a, const Slice& b) const override { + const TestKey kkey_a = SliceToTestKey(a); + const TestKey kkey_b = SliceToTestKey(b); + const TestKey *key_a = &kkey_a; + const TestKey *key_b = &kkey_b; if (key_a->prefix != key_b->prefix) { if (key_a->prefix < key_b->prefix) return -1; if (key_a->prefix > key_b->prefix) return 1; } else { - ASSERT_TRUE(key_a->prefix == key_b->prefix); + EXPECT_TRUE(key_a->prefix == key_b->prefix); // note, both a and b could be prefix only if (a.size() != b.size()) { // one of them is prefix - ASSERT_TRUE( - (a.size() == sizeof(uint64_t) && b.size() == sizeof(TestKey)) || - (b.size() == sizeof(uint64_t) && a.size() == sizeof(TestKey))); + EXPECT_TRUE( + (a.size() == sizeof(uint64_t) && b.size() == sizeof(TestKey)) || + (b.size() == sizeof(uint64_t) && a.size() == sizeof(TestKey))); if (a.size() < b.size()) return -1; if (a.size() > b.size()) return 1; } else { @@ -92,7 +108,7 @@ class TestKeyComparator : public Comparator { } // both a and b are whole key - ASSERT_TRUE(a.size() == sizeof(TestKey) && b.size() == sizeof(TestKey)); + EXPECT_TRUE(a.size() == sizeof(TestKey) && b.size() == sizeof(TestKey)); if (key_a->sorted < key_b->sorted) return -1; if (key_a->sorted > key_b->sorted) return 1; if (key_a->sorted == key_b->sorted) return 0; @@ -101,30 +117,54 @@ class TestKeyComparator : public Comparator { return 0; } - virtual const char* Name() const override { - return "TestKeyComparator"; + bool operator()(const TestKey& a, const TestKey& b) const { + std::string sa, sb; + return Compare(TestKeyToSlice(sa, a), TestKeyToSlice(sb, b)) < 0; } - virtual void FindShortestSeparator( - std::string* start, - const Slice& limit) const { + virtual const char* Name() const override { + return "TestKeyComparator"; } - virtual void FindShortSuccessor(std::string* key) const {} + virtual void FindShortestSeparator(std::string* start, + const Slice& limit) const override {} + virtual void FindShortSuccessor(std::string* key) const override {} }; namespace { void PutKey(DB* db, WriteOptions write_options, uint64_t prefix, uint64_t suffix, const Slice& value) { TestKey test_key(prefix, suffix); - Slice key = TestKeyToSlice(test_key); + std::string s; + Slice key = TestKeyToSlice(s, test_key); ASSERT_OK(db->Put(write_options, key, value)); } +void PutKey(DB* db, WriteOptions write_options, const TestKey& test_key, + const Slice& value) { + std::string s; + Slice key = TestKeyToSlice(s, test_key); + ASSERT_OK(db->Put(write_options, key, value)); +} + +void MergeKey(DB* db, WriteOptions write_options, const TestKey& test_key, + const Slice& value) { + std::string s; + Slice key = TestKeyToSlice(s, test_key); + ASSERT_OK(db->Merge(write_options, key, value)); +} + +void DeleteKey(DB* db, WriteOptions write_options, const TestKey& test_key) { + std::string s; + Slice key = TestKeyToSlice(s, test_key); + ASSERT_OK(db->Delete(write_options, key)); +} + void SeekIterator(Iterator* iter, uint64_t prefix, uint64_t suffix) { TestKey test_key(prefix, suffix); - Slice key = TestKeyToSlice(test_key); + std::string s; + Slice key = TestKeyToSlice(s, test_key); iter->Seek(key); } @@ -133,7 +173,8 @@ const std::string kNotFoundResult = "NOT_FOUND"; std::string Get(DB* db, const ReadOptions& read_options, uint64_t prefix, uint64_t suffix) { TestKey test_key(prefix, suffix); - Slice key = TestKeyToSlice(test_key); + std::string s2; + Slice key = TestKeyToSlice(s2, test_key); std::string result; Status s = db->Get(read_options, key, &result); @@ -144,9 +185,38 @@ std::string Get(DB* db, const ReadOptions& read_options, uint64_t prefix, } return result; } + +class SamePrefixTransform : public SliceTransform { + private: + const Slice prefix_; + std::string name_; + + public: + explicit SamePrefixTransform(const Slice& prefix) + : prefix_(prefix), name_("rocksdb.SamePrefix." + prefix.ToString()) {} + + virtual const char* Name() const override { return name_.c_str(); } + + virtual Slice Transform(const Slice& src) const override { + assert(InDomain(src)); + return prefix_; + } + + virtual bool InDomain(const Slice& src) const override { + if (src.size() >= prefix_.size()) { + return Slice(src.data(), prefix_.size()) == prefix_; + } + return false; + } + + virtual bool InRange(const Slice& dst) const override { + return dst == prefix_; + } +}; + } // namespace -class PrefixTest { +class PrefixTest : public testing::Test { public: std::shared_ptr OpenDb() { DB* db; @@ -157,13 +227,19 @@ class PrefixTest { options.min_write_buffer_number_to_merge = FLAGS_min_write_buffer_number_to_merge; - options.memtable_prefix_bloom_bits = FLAGS_memtable_prefix_bloom_bits; - options.memtable_prefix_bloom_probes = FLAGS_memtable_prefix_bloom_probes; - options.memtable_prefix_bloom_huge_page_tlb_size = - FLAGS_memtable_prefix_bloom_huge_page_tlb_size; + options.memtable_prefix_bloom_size_ratio = + FLAGS_memtable_prefix_bloom_size_ratio; + options.memtable_huge_page_size = FLAGS_memtable_huge_page_size; + + options.prefix_extractor.reset(NewFixedPrefixTransform(8)); + BlockBasedTableOptions bbto; + bbto.filter_policy.reset(NewBloomFilterPolicy(10, false)); + bbto.whole_key_filtering = false; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + options.allow_concurrent_memtable_write = false; Status s = DB::Open(options, kDbName, &db); - ASSERT_OK(s); + EXPECT_OK(s); return std::shared_ptr(db); } @@ -219,7 +295,57 @@ class PrefixTest { Options options; }; -TEST(PrefixTest, TestResult) { +TEST(SamePrefixTest, InDomainTest) { + DB* db; + Options options; + options.create_if_missing = true; + options.prefix_extractor.reset(new SamePrefixTransform("HHKB")); + BlockBasedTableOptions bbto; + bbto.filter_policy.reset(NewBloomFilterPolicy(10, false)); + bbto.whole_key_filtering = false; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + WriteOptions write_options; + ReadOptions read_options; + { + ASSERT_OK(DestroyDB(kDbName, Options())); + ASSERT_OK(DB::Open(options, kDbName, &db)); + ASSERT_OK(db->Put(write_options, "HHKB pro2", "Mar 24, 2006")); + ASSERT_OK(db->Put(write_options, "HHKB pro2 Type-S", "June 29, 2011")); + ASSERT_OK(db->Put(write_options, "Realforce 87u", "idk")); + db->Flush(FlushOptions()); + std::string result; + auto db_iter = db->NewIterator(ReadOptions()); + + db_iter->Seek("Realforce 87u"); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_OK(db_iter->status()); + ASSERT_EQ(db_iter->key(), "Realforce 87u"); + ASSERT_EQ(db_iter->value(), "idk"); + + delete db_iter; + delete db; + ASSERT_OK(DestroyDB(kDbName, Options())); + } + + { + ASSERT_OK(DB::Open(options, kDbName, &db)); + ASSERT_OK(db->Put(write_options, "pikachu", "1")); + ASSERT_OK(db->Put(write_options, "Meowth", "1")); + ASSERT_OK(db->Put(write_options, "Mewtwo", "idk")); + db->Flush(FlushOptions()); + std::string result; + auto db_iter = db->NewIterator(ReadOptions()); + + db_iter->Seek("Mewtwo"); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_OK(db_iter->status()); + delete db_iter; + delete db; + ASSERT_OK(DestroyDB(kDbName, Options())); + } +} + +TEST_F(PrefixTest, TestResult) { for (int num_buckets = 1; num_buckets <= 2; num_buckets++) { FirstOption(); while (NextOptions(num_buckets)) { @@ -392,7 +518,62 @@ TEST(PrefixTest, TestResult) { } } -TEST(PrefixTest, DynamicPrefixIterator) { +// Show results in prefix +TEST_F(PrefixTest, PrefixValid) { + for (int num_buckets = 1; num_buckets <= 2; num_buckets++) { + FirstOption(); + while (NextOptions(num_buckets)) { + std::cout << "*** Mem table: " << options.memtable_factory->Name() + << " number of buckets: " << num_buckets << std::endl; + DestroyDB(kDbName, Options()); + auto db = OpenDb(); + WriteOptions write_options; + ReadOptions read_options; + + // Insert keys with common prefix and one key with different + Slice v16("v16"); + Slice v17("v17"); + Slice v18("v18"); + Slice v19("v19"); + PutKey(db.get(), write_options, 12345, 6, v16); + PutKey(db.get(), write_options, 12345, 7, v17); + PutKey(db.get(), write_options, 12345, 8, v18); + PutKey(db.get(), write_options, 12345, 9, v19); + PutKey(db.get(), write_options, 12346, 8, v16); + db->Flush(FlushOptions()); + TestKey test_key(12346, 8); + std::string s; + db->Delete(write_options, TestKeyToSlice(s, test_key)); + db->Flush(FlushOptions()); + read_options.prefix_same_as_start = true; + std::unique_ptr iter(db->NewIterator(read_options)); + SeekIterator(iter.get(), 12345, 6); + ASSERT_TRUE(iter->Valid()); + ASSERT_TRUE(v16 == iter->value()); + + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_TRUE(v17 == iter->value()); + + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_TRUE(v18 == iter->value()); + + iter->Next(); + ASSERT_TRUE(iter->Valid()); + ASSERT_TRUE(v19 == iter->value()); + iter->Next(); + ASSERT_FALSE(iter->Valid()); + ASSERT_EQ(kNotFoundResult, Get(db.get(), read_options, 12346, 8)); + + // Verify seeking past the prefix won't return a result. + SeekIterator(iter.get(), 12345, 10); + ASSERT_TRUE(!iter->Valid()); + } + } +} + +TEST_F(PrefixTest, DynamicPrefixIterator) { while (NextOptions(FLAGS_bucket_count)) { std::cout << "*** Mem table: " << options.memtable_factory->Name() << std::endl; @@ -418,14 +599,15 @@ TEST(PrefixTest, DynamicPrefixIterator) { for (uint64_t sorted = 0; sorted < FLAGS_items_per_prefix; sorted++) { TestKey test_key(prefix, sorted); - Slice key = TestKeyToSlice(test_key); + std::string s; + Slice key = TestKeyToSlice(s, test_key); std::string value(FLAGS_value_size, 0); - perf_context.Reset(); + get_perf_context()->Reset(); StopWatchNano timer(Env::Default(), true); ASSERT_OK(db->Put(write_options, key, value)); hist_put_time.Add(timer.ElapsedNanos()); - hist_put_comparison.Add(perf_context.user_key_comparison_count); + hist_put_comparison.Add(get_perf_context()->user_key_comparison_count); } } @@ -440,10 +622,11 @@ TEST(PrefixTest, DynamicPrefixIterator) { for (auto prefix : prefixes) { TestKey test_key(prefix, FLAGS_items_per_prefix / 2); - Slice key = TestKeyToSlice(test_key); - std::string value = "v" + std::to_string(0); + std::string s; + Slice key = TestKeyToSlice(s, test_key); + std::string value = "v" + ToString(0); - perf_context.Reset(); + get_perf_context()->Reset(); StopWatchNano timer(Env::Default(), true); auto key_prefix = options.prefix_extractor->Transform(key); uint64_t total_keys = 0; @@ -457,7 +640,7 @@ TEST(PrefixTest, DynamicPrefixIterator) { total_keys++; } hist_seek_time.Add(timer.ElapsedNanos()); - hist_seek_comparison.Add(perf_context.user_key_comparison_count); + hist_seek_comparison.Add(get_perf_context()->user_key_comparison_count); ASSERT_EQ(total_keys, FLAGS_items_per_prefix - FLAGS_items_per_prefix/2); } @@ -474,13 +657,14 @@ TEST(PrefixTest, DynamicPrefixIterator) { prefix < FLAGS_total_prefixes + 10000; prefix++) { TestKey test_key(prefix, 0); - Slice key = TestKeyToSlice(test_key); + std::string s; + Slice key = TestKeyToSlice(s, test_key); - perf_context.Reset(); + get_perf_context()->Reset(); StopWatchNano timer(Env::Default(), true); iter->Seek(key); hist_no_seek_time.Add(timer.ElapsedNanos()); - hist_no_seek_comparison.Add(perf_context.user_key_comparison_count); + hist_no_seek_comparison.Add(get_perf_context()->user_key_comparison_count); ASSERT_TRUE(!iter->Valid()); } @@ -491,14 +675,225 @@ TEST(PrefixTest, DynamicPrefixIterator) { } } +TEST_F(PrefixTest, PrefixSeekModePrev) { + // Only for SkipListFactory + options.memtable_factory.reset(new SkipListFactory); + options.merge_operator = MergeOperators::CreatePutOperator(); + options.write_buffer_size = 1024 * 1024; + Random rnd(1); + for (size_t m = 1; m < 100; m++) { + std::cout << "[" + std::to_string(m) + "]" + "*** Mem table: " + << options.memtable_factory->Name() << std::endl; + DestroyDB(kDbName, Options()); + auto db = OpenDb(); + WriteOptions write_options; + ReadOptions read_options; + std::map entry_maps[3], whole_map; + for (uint64_t i = 0; i < 10; i++) { + int div = i % 3 + 1; + for (uint64_t j = 0; j < 10; j++) { + whole_map[TestKey(i, j)] = entry_maps[rnd.Uniform(div)][TestKey(i, j)] = + 'v' + std::to_string(i) + std::to_string(j); + } + } + + std::map type_map; + for (size_t i = 0; i < 3; i++) { + for (auto& kv : entry_maps[i]) { + if (rnd.OneIn(3)) { + PutKey(db.get(), write_options, kv.first, kv.second); + type_map[kv.first] = "value"; + } else { + MergeKey(db.get(), write_options, kv.first, kv.second); + type_map[kv.first] = "merge"; + } + } + if (i < 2) { + db->Flush(FlushOptions()); + } + } + + for (size_t i = 0; i < 2; i++) { + for (auto& kv : entry_maps[i]) { + if (rnd.OneIn(10)) { + whole_map.erase(kv.first); + DeleteKey(db.get(), write_options, kv.first); + entry_maps[2][kv.first] = "delete"; + } + } + } + + if (FLAGS_enable_print) { + for (size_t i = 0; i < 3; i++) { + for (auto& kv : entry_maps[i]) { + std::cout << "[" << i << "]" << kv.first.prefix << kv.first.sorted + << " " << kv.second + " " + type_map[kv.first] << std::endl; + } + } + } + + std::unique_ptr iter(db->NewIterator(read_options)); + for (uint64_t prefix = 0; prefix < 10; prefix++) { + uint64_t start_suffix = rnd.Uniform(9); + SeekIterator(iter.get(), prefix, start_suffix); + auto it = whole_map.find(TestKey(prefix, start_suffix)); + if (it == whole_map.end()) { + continue; + } + ASSERT_NE(it, whole_map.end()); + ASSERT_TRUE(iter->Valid()); + if (FLAGS_enable_print) { + std::cout << "round " << prefix + << " iter: " << SliceToTestKey(iter->key()).prefix + << SliceToTestKey(iter->key()).sorted + << " | map: " << it->first.prefix << it->first.sorted << " | " + << iter->value().ToString() << " " << it->second << std::endl; + } + ASSERT_EQ(iter->value(), it->second); + uint64_t stored_prefix = prefix; + for (size_t k = 0; k < 9; k++) { + if (rnd.OneIn(2) || it == whole_map.begin()) { + iter->Next(); + it++; + if (FLAGS_enable_print) { + std::cout << "Next >> "; + } + } else { + iter->Prev(); + it--; + if (FLAGS_enable_print) { + std::cout << "Prev >> "; + } + } + if (!iter->Valid() || + SliceToTestKey(iter->key()).prefix != stored_prefix) { + break; + } + stored_prefix = SliceToTestKey(iter->key()).prefix; + ASSERT_TRUE(iter->Valid()); + ASSERT_NE(it, whole_map.end()); + ASSERT_EQ(iter->value(), it->second); + if (FLAGS_enable_print) { + std::cout << "iter: " << SliceToTestKey(iter->key()).prefix + << SliceToTestKey(iter->key()).sorted + << " | map: " << it->first.prefix << it->first.sorted + << " | " << iter->value().ToString() << " " << it->second + << std::endl; + } + } + } + } +} + +TEST_F(PrefixTest, PrefixSeekModePrev2) { + // Only for SkipListFactory + // test the case + // iter1 iter2 + // | prefix | suffix | | prefix | suffix | + // | 1 | 1 | | 1 | 2 | + // | 1 | 3 | | 1 | 4 | + // | 2 | 1 | | 3 | 3 | + // | 2 | 2 | | 3 | 4 | + // after seek(15), iter1 will be at 21 and iter2 will be 33. + // Then if call Prev() in prefix mode where SeekForPrev(21) gets called, + // iter2 should turn to invalid state because of bloom filter. + options.memtable_factory.reset(new SkipListFactory); + options.write_buffer_size = 1024 * 1024; + std::string v13("v13"); + DestroyDB(kDbName, Options()); + auto db = OpenDb(); + WriteOptions write_options; + ReadOptions read_options; + PutKey(db.get(), write_options, TestKey(1, 2), "v12"); + PutKey(db.get(), write_options, TestKey(1, 4), "v14"); + PutKey(db.get(), write_options, TestKey(3, 3), "v33"); + PutKey(db.get(), write_options, TestKey(3, 4), "v34"); + db->Flush(FlushOptions()); + reinterpret_cast(db.get())->TEST_WaitForFlushMemTable(); + PutKey(db.get(), write_options, TestKey(1, 1), "v11"); + PutKey(db.get(), write_options, TestKey(1, 3), "v13"); + PutKey(db.get(), write_options, TestKey(2, 1), "v21"); + PutKey(db.get(), write_options, TestKey(2, 2), "v22"); + db->Flush(FlushOptions()); + reinterpret_cast(db.get())->TEST_WaitForFlushMemTable(); + std::unique_ptr iter(db->NewIterator(read_options)); + SeekIterator(iter.get(), 1, 5); + iter->Prev(); + ASSERT_EQ(iter->value(), v13); } +TEST_F(PrefixTest, PrefixSeekModePrev3) { + // Only for SkipListFactory + // test SeekToLast() with iterate_upper_bound_ in prefix_seek_mode + options.memtable_factory.reset(new SkipListFactory); + options.write_buffer_size = 1024 * 1024; + std::string v14("v14"); + TestKey upper_bound_key = TestKey(1, 5); + std::string s; + Slice upper_bound = TestKeyToSlice(s, upper_bound_key); + + { + DestroyDB(kDbName, Options()); + auto db = OpenDb(); + WriteOptions write_options; + ReadOptions read_options; + read_options.iterate_upper_bound = &upper_bound; + PutKey(db.get(), write_options, TestKey(1, 2), "v12"); + PutKey(db.get(), write_options, TestKey(1, 4), "v14"); + db->Flush(FlushOptions()); + reinterpret_cast(db.get())->TEST_WaitForFlushMemTable(); + PutKey(db.get(), write_options, TestKey(1, 1), "v11"); + PutKey(db.get(), write_options, TestKey(1, 3), "v13"); + PutKey(db.get(), write_options, TestKey(2, 1), "v21"); + PutKey(db.get(), write_options, TestKey(2, 2), "v22"); + db->Flush(FlushOptions()); + reinterpret_cast(db.get())->TEST_WaitForFlushMemTable(); + std::unique_ptr iter(db->NewIterator(read_options)); + iter->SeekToLast(); + ASSERT_EQ(iter->value(), v14); + } + { + DestroyDB(kDbName, Options()); + auto db = OpenDb(); + WriteOptions write_options; + ReadOptions read_options; + read_options.iterate_upper_bound = &upper_bound; + PutKey(db.get(), write_options, TestKey(1, 2), "v12"); + PutKey(db.get(), write_options, TestKey(1, 4), "v14"); + PutKey(db.get(), write_options, TestKey(3, 3), "v33"); + PutKey(db.get(), write_options, TestKey(3, 4), "v34"); + db->Flush(FlushOptions()); + reinterpret_cast(db.get())->TEST_WaitForFlushMemTable(); + PutKey(db.get(), write_options, TestKey(1, 1), "v11"); + PutKey(db.get(), write_options, TestKey(1, 3), "v13"); + db->Flush(FlushOptions()); + reinterpret_cast(db.get())->TEST_WaitForFlushMemTable(); + std::unique_ptr iter(db->NewIterator(read_options)); + iter->SeekToLast(); + ASSERT_EQ(iter->value(), v14); + } +} + +} // end namespace rocksdb + int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); ParseCommandLineFlags(&argc, &argv, true); std::cout << kDbName << "\n"; - rocksdb::test::RunAllTests(); - return 0; + return RUN_ALL_TESTS(); } #endif // GFLAGS + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, + "SKIPPED as HashSkipList and HashLinkList are not supported in " + "ROCKSDB_LITE\n"); + return 0; +} + +#endif // !ROCKSDB_LITE diff --git a/db/range_del_aggregator.cc b/db/range_del_aggregator.cc new file mode 100644 index 00000000000..c83f5a88cd8 --- /dev/null +++ b/db/range_del_aggregator.cc @@ -0,0 +1,520 @@ +// Copyright (c) 2016-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/range_del_aggregator.h" + +#include + +namespace rocksdb { + +RangeDelAggregator::RangeDelAggregator( + const InternalKeyComparator& icmp, + const std::vector& snapshots, + bool collapse_deletions /* = true */) + : upper_bound_(kMaxSequenceNumber), + icmp_(icmp), + collapse_deletions_(collapse_deletions) { + InitRep(snapshots); +} + +RangeDelAggregator::RangeDelAggregator(const InternalKeyComparator& icmp, + SequenceNumber snapshot, + bool collapse_deletions /* = false */) + : upper_bound_(snapshot), + icmp_(icmp), + collapse_deletions_(collapse_deletions) {} + +void RangeDelAggregator::InitRep(const std::vector& snapshots) { + assert(rep_ == nullptr); + rep_.reset(new Rep()); + for (auto snapshot : snapshots) { + rep_->stripe_map_.emplace( + snapshot, + PositionalTombstoneMap(TombstoneMap( + stl_wrappers::LessOfComparator(icmp_.user_comparator())))); + } + // Data newer than any snapshot falls in this catch-all stripe + rep_->stripe_map_.emplace( + kMaxSequenceNumber, + PositionalTombstoneMap(TombstoneMap( + stl_wrappers::LessOfComparator(icmp_.user_comparator())))); + rep_->pinned_iters_mgr_.StartPinning(); +} + +bool RangeDelAggregator::ShouldDelete( + const Slice& internal_key, RangeDelAggregator::RangePositioningMode mode) { + if (rep_ == nullptr) { + return false; + } + ParsedInternalKey parsed; + if (!ParseInternalKey(internal_key, &parsed)) { + assert(false); + } + return ShouldDelete(parsed, mode); +} + +bool RangeDelAggregator::ShouldDelete( + const ParsedInternalKey& parsed, + RangeDelAggregator::RangePositioningMode mode) { + assert(IsValueType(parsed.type)); + if (rep_ == nullptr) { + return false; + } + auto& positional_tombstone_map = GetPositionalTombstoneMap(parsed.sequence); + const auto& tombstone_map = positional_tombstone_map.raw_map; + if (tombstone_map.empty()) { + return false; + } + auto& tombstone_map_iter = positional_tombstone_map.iter; + if (tombstone_map_iter == tombstone_map.end() && + (mode == kForwardTraversal || mode == kBackwardTraversal)) { + // invalid (e.g., if AddTombstones() changed the deletions), so need to + // reseek + mode = kBinarySearch; + } + switch (mode) { + case kFullScan: + assert(!collapse_deletions_); + // The maintained state (PositionalTombstoneMap::iter) isn't useful when + // we linear scan from the beginning each time, but we maintain it anyways + // for consistency. + tombstone_map_iter = tombstone_map.begin(); + while (tombstone_map_iter != tombstone_map.end()) { + const auto& tombstone = tombstone_map_iter->second; + if (icmp_.user_comparator()->Compare(parsed.user_key, + tombstone.start_key_) < 0) { + break; + } + if (parsed.sequence < tombstone.seq_ && + icmp_.user_comparator()->Compare(parsed.user_key, + tombstone.end_key_) < 0) { + return true; + } + ++tombstone_map_iter; + } + return false; + case kForwardTraversal: + assert(collapse_deletions_ && tombstone_map_iter != tombstone_map.end()); + if (tombstone_map_iter == tombstone_map.begin() && + icmp_.user_comparator()->Compare(parsed.user_key, + tombstone_map_iter->first) < 0) { + // before start of deletion intervals + return false; + } + while (std::next(tombstone_map_iter) != tombstone_map.end() && + icmp_.user_comparator()->Compare( + std::next(tombstone_map_iter)->first, parsed.user_key) <= 0) { + ++tombstone_map_iter; + } + break; + case kBackwardTraversal: + assert(collapse_deletions_ && tombstone_map_iter != tombstone_map.end()); + while (tombstone_map_iter != tombstone_map.begin() && + icmp_.user_comparator()->Compare(parsed.user_key, + tombstone_map_iter->first) < 0) { + --tombstone_map_iter; + } + if (tombstone_map_iter == tombstone_map.begin() && + icmp_.user_comparator()->Compare(parsed.user_key, + tombstone_map_iter->first) < 0) { + // before start of deletion intervals + return false; + } + break; + case kBinarySearch: + assert(collapse_deletions_); + tombstone_map_iter = + tombstone_map.upper_bound(parsed.user_key); + if (tombstone_map_iter == tombstone_map.begin()) { + // before start of deletion intervals + return false; + } + --tombstone_map_iter; + break; + } + assert(mode != kFullScan); + assert(tombstone_map_iter != tombstone_map.end() && + icmp_.user_comparator()->Compare(tombstone_map_iter->first, + parsed.user_key) <= 0); + assert(std::next(tombstone_map_iter) == tombstone_map.end() || + icmp_.user_comparator()->Compare( + parsed.user_key, std::next(tombstone_map_iter)->first) < 0); + return parsed.sequence < tombstone_map_iter->second.seq_; +} + +bool RangeDelAggregator::ShouldAddTombstones( + bool bottommost_level /* = false */) { + // TODO(andrewkr): can we just open a file and throw it away if it ends up + // empty after AddToBuilder()? This function doesn't take into subcompaction + // boundaries so isn't completely accurate. + if (rep_ == nullptr) { + return false; + } + auto stripe_map_iter = rep_->stripe_map_.begin(); + assert(stripe_map_iter != rep_->stripe_map_.end()); + if (bottommost_level) { + // For the bottommost level, keys covered by tombstones in the first + // (oldest) stripe have been compacted away, so the tombstones are obsolete. + ++stripe_map_iter; + } + while (stripe_map_iter != rep_->stripe_map_.end()) { + if (!stripe_map_iter->second.raw_map.empty()) { + return true; + } + ++stripe_map_iter; + } + return false; +} + +Status RangeDelAggregator::AddTombstones( + std::unique_ptr input) { + if (input == nullptr) { + return Status::OK(); + } + input->SeekToFirst(); + bool first_iter = true; + while (input->Valid()) { + if (first_iter) { + if (rep_ == nullptr) { + InitRep({upper_bound_}); + } else { + InvalidateTombstoneMapPositions(); + } + first_iter = false; + } + ParsedInternalKey parsed_key; + if (!ParseInternalKey(input->key(), &parsed_key)) { + return Status::Corruption("Unable to parse range tombstone InternalKey"); + } + RangeTombstone tombstone(parsed_key, input->value()); + AddTombstone(std::move(tombstone)); + input->Next(); + } + if (!first_iter) { + rep_->pinned_iters_mgr_.PinIterator(input.release(), false /* arena */); + } + return Status::OK(); +} + +void RangeDelAggregator::InvalidateTombstoneMapPositions() { + if (rep_ == nullptr) { + return; + } + for (auto stripe_map_iter = rep_->stripe_map_.begin(); + stripe_map_iter != rep_->stripe_map_.end(); ++stripe_map_iter) { + stripe_map_iter->second.iter = stripe_map_iter->second.raw_map.end(); + } +} + +Status RangeDelAggregator::AddTombstone(RangeTombstone tombstone) { + auto& positional_tombstone_map = GetPositionalTombstoneMap(tombstone.seq_); + auto& tombstone_map = positional_tombstone_map.raw_map; + if (collapse_deletions_) { + // In collapsed mode, we only fill the seq_ field in the TombstoneMap's + // values. The end_key is unneeded because we assume the tombstone extends + // until the next tombstone starts. For gaps between real tombstones and + // for the last real tombstone, we denote end keys by inserting fake + // tombstones with sequence number zero. + std::vector new_range_dels{ + tombstone, RangeTombstone(tombstone.end_key_, Slice(), 0)}; + auto new_range_dels_iter = new_range_dels.begin(); + // Position at the first overlapping existing tombstone; if none exists, + // insert until we find an existing one overlapping a new point + const Slice* tombstone_map_begin = nullptr; + if (!tombstone_map.empty()) { + tombstone_map_begin = &tombstone_map.begin()->first; + } + auto last_range_dels_iter = new_range_dels_iter; + while (new_range_dels_iter != new_range_dels.end() && + (tombstone_map_begin == nullptr || + icmp_.user_comparator()->Compare(new_range_dels_iter->start_key_, + *tombstone_map_begin) < 0)) { + tombstone_map.emplace( + new_range_dels_iter->start_key_, + RangeTombstone(Slice(), Slice(), new_range_dels_iter->seq_)); + last_range_dels_iter = new_range_dels_iter; + ++new_range_dels_iter; + } + if (new_range_dels_iter == new_range_dels.end()) { + return Status::OK(); + } + // above loop advances one too far + new_range_dels_iter = last_range_dels_iter; + auto tombstone_map_iter = + tombstone_map.upper_bound(new_range_dels_iter->start_key_); + // if nothing overlapped we would've already inserted all the new points + // and returned early + assert(tombstone_map_iter != tombstone_map.begin()); + tombstone_map_iter--; + + // untermed_seq is non-kMaxSequenceNumber when we covered an existing point + // but haven't seen its corresponding endpoint. It's used for (1) deciding + // whether to forcibly insert the new interval's endpoint; and (2) possibly + // raising the seqnum for the to-be-inserted element (we insert the max + // seqnum between the next new interval and the unterminated interval). + SequenceNumber untermed_seq = kMaxSequenceNumber; + while (tombstone_map_iter != tombstone_map.end() && + new_range_dels_iter != new_range_dels.end()) { + const Slice *tombstone_map_iter_end = nullptr, + *new_range_dels_iter_end = nullptr; + if (tombstone_map_iter != tombstone_map.end()) { + auto next_tombstone_map_iter = std::next(tombstone_map_iter); + if (next_tombstone_map_iter != tombstone_map.end()) { + tombstone_map_iter_end = &next_tombstone_map_iter->first; + } + } + if (new_range_dels_iter != new_range_dels.end()) { + auto next_new_range_dels_iter = std::next(new_range_dels_iter); + if (next_new_range_dels_iter != new_range_dels.end()) { + new_range_dels_iter_end = &next_new_range_dels_iter->start_key_; + } + } + + // our positions in existing/new tombstone collections should always + // overlap. The non-overlapping cases are handled above and below this + // loop. + assert(new_range_dels_iter_end == nullptr || + icmp_.user_comparator()->Compare(tombstone_map_iter->first, + *new_range_dels_iter_end) < 0); + assert(tombstone_map_iter_end == nullptr || + icmp_.user_comparator()->Compare(new_range_dels_iter->start_key_, + *tombstone_map_iter_end) < 0); + + int new_to_old_start_cmp = icmp_.user_comparator()->Compare( + new_range_dels_iter->start_key_, tombstone_map_iter->first); + // nullptr end means extends infinitely rightwards, set new_to_old_end_cmp + // accordingly so we can use common code paths later. + int new_to_old_end_cmp; + if (new_range_dels_iter_end == nullptr && + tombstone_map_iter_end == nullptr) { + new_to_old_end_cmp = 0; + } else if (new_range_dels_iter_end == nullptr) { + new_to_old_end_cmp = 1; + } else if (tombstone_map_iter_end == nullptr) { + new_to_old_end_cmp = -1; + } else { + new_to_old_end_cmp = icmp_.user_comparator()->Compare( + *new_range_dels_iter_end, *tombstone_map_iter_end); + } + + if (new_to_old_start_cmp < 0) { + // the existing one's left endpoint comes after, so raise/delete it if + // it's covered. + if (tombstone_map_iter->second.seq_ < new_range_dels_iter->seq_) { + untermed_seq = tombstone_map_iter->second.seq_; + if (tombstone_map_iter != tombstone_map.begin() && + std::prev(tombstone_map_iter)->second.seq_ == + new_range_dels_iter->seq_) { + tombstone_map_iter = tombstone_map.erase(tombstone_map_iter); + --tombstone_map_iter; + } else { + tombstone_map_iter->second.seq_ = new_range_dels_iter->seq_; + } + } + } else if (new_to_old_start_cmp > 0) { + if (untermed_seq != kMaxSequenceNumber || + tombstone_map_iter->second.seq_ < new_range_dels_iter->seq_) { + auto seq = tombstone_map_iter->second.seq_; + // need to adjust this element if not intended to span beyond the new + // element (i.e., was_tombstone_map_iter_raised == true), or if it + // can be raised + tombstone_map_iter = tombstone_map.emplace( + new_range_dels_iter->start_key_, + RangeTombstone( + Slice(), Slice(), + std::max( + untermed_seq == kMaxSequenceNumber ? 0 : untermed_seq, + new_range_dels_iter->seq_))); + untermed_seq = seq; + } + } else { + // their left endpoints coincide, so raise the existing one if needed + if (tombstone_map_iter->second.seq_ < new_range_dels_iter->seq_) { + untermed_seq = tombstone_map_iter->second.seq_; + tombstone_map_iter->second.seq_ = new_range_dels_iter->seq_; + } + } + + // advance whichever one ends earlier, or both if their right endpoints + // coincide + if (new_to_old_end_cmp < 0) { + ++new_range_dels_iter; + } else if (new_to_old_end_cmp > 0) { + ++tombstone_map_iter; + untermed_seq = kMaxSequenceNumber; + } else { + ++new_range_dels_iter; + ++tombstone_map_iter; + untermed_seq = kMaxSequenceNumber; + } + } + while (new_range_dels_iter != new_range_dels.end()) { + tombstone_map.emplace( + new_range_dels_iter->start_key_, + RangeTombstone(Slice(), Slice(), new_range_dels_iter->seq_)); + ++new_range_dels_iter; + } + } else { + auto start_key = tombstone.start_key_; + tombstone_map.emplace(start_key, std::move(tombstone)); + } + return Status::OK(); +} + +RangeDelAggregator::PositionalTombstoneMap& +RangeDelAggregator::GetPositionalTombstoneMap(SequenceNumber seq) { + assert(rep_ != nullptr); + // The stripe includes seqnum for the snapshot above and excludes seqnum for + // the snapshot below. + StripeMap::iterator iter; + if (seq > 0) { + // upper_bound() checks strict inequality so need to subtract one + iter = rep_->stripe_map_.upper_bound(seq - 1); + } else { + iter = rep_->stripe_map_.begin(); + } + // catch-all stripe justifies this assertion in either of above cases + assert(iter != rep_->stripe_map_.end()); + return iter->second; +} + +// TODO(andrewkr): We should implement an iterator over range tombstones in our +// map. It'd enable compaction to open tables on-demand, i.e., only once range +// tombstones are known to be available, without the code duplication we have +// in ShouldAddTombstones(). It'll also allow us to move the table-modifying +// code into more coherent places: CompactionJob and BuildTable(). +void RangeDelAggregator::AddToBuilder( + TableBuilder* builder, const Slice* lower_bound, const Slice* upper_bound, + FileMetaData* meta, + CompactionIterationStats* range_del_out_stats /* = nullptr */, + bool bottommost_level /* = false */) { + if (rep_ == nullptr) { + return; + } + auto stripe_map_iter = rep_->stripe_map_.begin(); + assert(stripe_map_iter != rep_->stripe_map_.end()); + if (bottommost_level) { + // TODO(andrewkr): these are counted for each compaction output file, so + // lots of double-counting. + if (!stripe_map_iter->second.raw_map.empty()) { + range_del_out_stats->num_range_del_drop_obsolete += + static_cast(stripe_map_iter->second.raw_map.size()) - + (collapse_deletions_ ? 1 : 0); + range_del_out_stats->num_record_drop_obsolete += + static_cast(stripe_map_iter->second.raw_map.size()) - + (collapse_deletions_ ? 1 : 0); + } + // For the bottommost level, keys covered by tombstones in the first + // (oldest) stripe have been compacted away, so the tombstones are obsolete. + ++stripe_map_iter; + } + + // Note the order in which tombstones are stored is insignificant since we + // insert them into a std::map on the read path. + while (stripe_map_iter != rep_->stripe_map_.end()) { + bool first_added = false; + for (auto tombstone_map_iter = stripe_map_iter->second.raw_map.begin(); + tombstone_map_iter != stripe_map_iter->second.raw_map.end(); + ++tombstone_map_iter) { + RangeTombstone tombstone; + if (collapse_deletions_) { + auto next_tombstone_map_iter = std::next(tombstone_map_iter); + if (next_tombstone_map_iter == stripe_map_iter->second.raw_map.end() || + tombstone_map_iter->second.seq_ == 0) { + // it's a sentinel tombstone + continue; + } + tombstone.start_key_ = tombstone_map_iter->first; + tombstone.end_key_ = next_tombstone_map_iter->first; + tombstone.seq_ = tombstone_map_iter->second.seq_; + } else { + tombstone = tombstone_map_iter->second; + } + if (upper_bound != nullptr && + icmp_.user_comparator()->Compare(*upper_bound, + tombstone.start_key_) <= 0) { + // Tombstones starting at upper_bound or later only need to be included + // in the next table. Break because subsequent tombstones will start + // even later. + break; + } + if (lower_bound != nullptr && + icmp_.user_comparator()->Compare(tombstone.end_key_, + *lower_bound) <= 0) { + // Tombstones ending before or at lower_bound only need to be included + // in the prev table. Continue because subsequent tombstones may still + // overlap [lower_bound, upper_bound). + continue; + } + + auto ikey_and_end_key = tombstone.Serialize(); + builder->Add(ikey_and_end_key.first.Encode(), ikey_and_end_key.second); + if (!first_added) { + first_added = true; + InternalKey smallest_candidate = std::move(ikey_and_end_key.first); + if (lower_bound != nullptr && + icmp_.user_comparator()->Compare(smallest_candidate.user_key(), + *lower_bound) <= 0) { + // Pretend the smallest key has the same user key as lower_bound + // (the max key in the previous table or subcompaction) in order for + // files to appear key-space partitioned. + // + // Choose lowest seqnum so this file's smallest internal key comes + // after the previous file's/subcompaction's largest. The fake seqnum + // is OK because the read path's file-picking code only considers user + // key. + smallest_candidate = InternalKey(*lower_bound, 0, kTypeRangeDeletion); + } + if (meta->smallest.size() == 0 || + icmp_.Compare(smallest_candidate, meta->smallest) < 0) { + meta->smallest = std::move(smallest_candidate); + } + } + InternalKey largest_candidate = tombstone.SerializeEndKey(); + if (upper_bound != nullptr && + icmp_.user_comparator()->Compare(*upper_bound, + largest_candidate.user_key()) <= 0) { + // Pretend the largest key has the same user key as upper_bound (the + // min key in the following table or subcompaction) in order for files + // to appear key-space partitioned. + // + // Choose highest seqnum so this file's largest internal key comes + // before the next file's/subcompaction's smallest. The fake seqnum is + // OK because the read path's file-picking code only considers the user + // key portion. + // + // Note Seek() also creates InternalKey with (user_key, + // kMaxSequenceNumber), but with kTypeDeletion (0x7) instead of + // kTypeRangeDeletion (0xF), so the range tombstone comes before the + // Seek() key in InternalKey's ordering. So Seek() will look in the + // next file for the user key. + largest_candidate = InternalKey(*upper_bound, kMaxSequenceNumber, + kTypeRangeDeletion); + } + if (meta->largest.size() == 0 || + icmp_.Compare(meta->largest, largest_candidate) < 0) { + meta->largest = std::move(largest_candidate); + } + meta->smallest_seqno = std::min(meta->smallest_seqno, tombstone.seq_); + meta->largest_seqno = std::max(meta->largest_seqno, tombstone.seq_); + } + ++stripe_map_iter; + } +} + +bool RangeDelAggregator::IsEmpty() { + if (rep_ == nullptr) { + return true; + } + for (auto stripe_map_iter = rep_->stripe_map_.begin(); + stripe_map_iter != rep_->stripe_map_.end(); ++stripe_map_iter) { + if (!stripe_map_iter->second.raw_map.empty()) { + return false; + } + } + return true; +} + +} // namespace rocksdb diff --git a/db/range_del_aggregator.h b/db/range_del_aggregator.h new file mode 100644 index 00000000000..9d4b8ca1683 --- /dev/null +++ b/db/range_del_aggregator.h @@ -0,0 +1,161 @@ +// Copyright (c) 2016-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include +#include + +#include "db/compaction_iteration_stats.h" +#include "db/dbformat.h" +#include "db/pinned_iterators_manager.h" +#include "db/version_edit.h" +#include "include/rocksdb/comparator.h" +#include "include/rocksdb/types.h" +#include "table/internal_iterator.h" +#include "table/scoped_arena_iterator.h" +#include "table/table_builder.h" +#include "util/kv_map.h" + +namespace rocksdb { + +// A RangeDelAggregator aggregates range deletion tombstones as they are +// encountered in memtables/SST files. It provides methods that check whether a +// key is covered by range tombstones or write the relevant tombstones to a new +// SST file. +class RangeDelAggregator { + public: + // @param snapshots These are used to organize the tombstones into snapshot + // stripes, which is the seqnum range between consecutive snapshots, + // including the higher snapshot and excluding the lower one. Currently, + // this is used by ShouldDelete() to prevent deletion of keys that are + // covered by range tombstones in other snapshot stripes. This constructor + // is used for writes (flush/compaction). All DB snapshots are provided + // such that no keys are removed that are uncovered according to any DB + // snapshot. + // Note this overload does not lazily initialize Rep. + RangeDelAggregator(const InternalKeyComparator& icmp, + const std::vector& snapshots, + bool collapse_deletions = true); + + // @param upper_bound Similar to snapshots above, except with a single + // snapshot, which allows us to store the snapshot on the stack and defer + // initialization of heap-allocating members (in Rep) until the first range + // deletion is encountered. This constructor is used in case of reads (get/ + // iterator), for which only the user snapshot (upper_bound) is provided + // such that the seqnum space is divided into two stripes. Only the older + // stripe will be used by ShouldDelete(). + RangeDelAggregator(const InternalKeyComparator& icmp, + SequenceNumber upper_bound, + bool collapse_deletions = false); + + // We maintain position in the tombstone map across calls to ShouldDelete. The + // caller may wish to specify a mode to optimize positioning the iterator + // during the next call to ShouldDelete. The non-kFullScan modes are only + // available when deletion collapsing is enabled. + // + // For example, if we invoke Next() on an iterator, kForwardTraversal should + // be specified to advance one-by-one through deletions until one is found + // with its interval containing the key. This will typically be faster than + // doing a full binary search (kBinarySearch). + enum RangePositioningMode { + kFullScan, // used iff collapse_deletions_ == false + kForwardTraversal, + kBackwardTraversal, + kBinarySearch, + }; + + // Returns whether the key should be deleted, which is the case when it is + // covered by a range tombstone residing in the same snapshot stripe. + // @param mode If collapse_deletions_ is true, this dictates how we will find + // the deletion whose interval contains this key. Otherwise, its + // value must be kFullScan indicating linear scan from beginning.. + bool ShouldDelete(const ParsedInternalKey& parsed, + RangePositioningMode mode = kFullScan); + bool ShouldDelete(const Slice& internal_key, + RangePositioningMode mode = kFullScan); + bool ShouldAddTombstones(bool bottommost_level = false); + + // Adds tombstones to the tombstone aggregation structure maintained by this + // object. + // @return non-OK status if any of the tombstone keys are corrupted. + Status AddTombstones(std::unique_ptr input); + + // Resets iterators maintained across calls to ShouldDelete(). This may be + // called when the tombstones change, or the owner may call explicitly, e.g., + // if it's an iterator that just seeked to an arbitrary position. The effect + // of invalidation is that the following call to ShouldDelete() will binary + // search for its tombstone. + void InvalidateTombstoneMapPositions(); + + // Writes tombstones covering a range to a table builder. + // @param extend_before_min_key If true, the range of tombstones to be added + // to the TableBuilder starts from the beginning of the key-range; + // otherwise, it starts from meta->smallest. + // @param lower_bound/upper_bound Any range deletion with [start_key, end_key) + // that overlaps the target range [*lower_bound, *upper_bound) is added to + // the builder. If lower_bound is nullptr, the target range extends + // infinitely to the left. If upper_bound is nullptr, the target range + // extends infinitely to the right. If both are nullptr, the target range + // extends infinitely in both directions, i.e., all range deletions are + // added to the builder. + // @param meta The file's metadata. We modify the begin and end keys according + // to the range tombstones added to this file such that the read path does + // not miss range tombstones that cover gaps before/after/between files in + // a level. lower_bound/upper_bound above constrain how far file boundaries + // can be extended. + // @param bottommost_level If true, we will filter out any tombstones + // belonging to the oldest snapshot stripe, because all keys potentially + // covered by this tombstone are guaranteed to have been deleted by + // compaction. + void AddToBuilder(TableBuilder* builder, const Slice* lower_bound, + const Slice* upper_bound, FileMetaData* meta, + CompactionIterationStats* range_del_out_stats = nullptr, + bool bottommost_level = false); + bool IsEmpty(); + + private: + // Maps tombstone user start key -> tombstone object + typedef std::multimap + TombstoneMap; + // Also maintains position in TombstoneMap last seen by ShouldDelete(). The + // end iterator indicates invalidation (e.g., if AddTombstones() changes the + // underlying map). End iterator cannot be invalidated. + struct PositionalTombstoneMap { + explicit PositionalTombstoneMap(TombstoneMap _raw_map) + : raw_map(std::move(_raw_map)), iter(raw_map.end()) {} + PositionalTombstoneMap(const PositionalTombstoneMap&) = delete; + PositionalTombstoneMap(PositionalTombstoneMap&& other) + : raw_map(std::move(other.raw_map)), iter(raw_map.end()) {} + + TombstoneMap raw_map; + TombstoneMap::const_iterator iter; + }; + + // Maps snapshot seqnum -> map of tombstones that fall in that stripe, i.e., + // their seqnums are greater than the next smaller snapshot's seqnum. + typedef std::map StripeMap; + + struct Rep { + StripeMap stripe_map_; + PinnedIteratorsManager pinned_iters_mgr_; + }; + // Initializes rep_ lazily. This aggregator object is constructed for every + // read, so expensive members should only be created when necessary, i.e., + // once the first range deletion is encountered. + void InitRep(const std::vector& snapshots); + + PositionalTombstoneMap& GetPositionalTombstoneMap(SequenceNumber seq); + Status AddTombstone(RangeTombstone tombstone); + + SequenceNumber upper_bound_; + std::unique_ptr rep_; + const InternalKeyComparator& icmp_; + // collapse range deletions so they're binary searchable + const bool collapse_deletions_; +}; + +} // namespace rocksdb diff --git a/db/range_del_aggregator_test.cc b/db/range_del_aggregator_test.cc new file mode 100644 index 00000000000..39029bd2a2e --- /dev/null +++ b/db/range_del_aggregator_test.cc @@ -0,0 +1,157 @@ +// Copyright (c) 2016-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include + +#include "db/db_test_util.h" +#include "db/range_del_aggregator.h" +#include "rocksdb/comparator.h" +#include "util/testutil.h" + +namespace rocksdb { + +class RangeDelAggregatorTest : public testing::Test {}; + +namespace { + +struct ExpectedPoint { + Slice begin; + SequenceNumber seq; +}; + +enum Direction { + kForward, + kReverse, +}; + +void VerifyRangeDels(const std::vector& range_dels, + const std::vector& expected_points) { + // Test same result regardless of which order the range deletions are added. + for (Direction dir : {kForward, kReverse}) { + auto icmp = InternalKeyComparator(BytewiseComparator()); + RangeDelAggregator range_del_agg(icmp, {} /* snapshots */, true); + std::vector keys, values; + for (const auto& range_del : range_dels) { + auto key_and_value = range_del.Serialize(); + keys.push_back(key_and_value.first.Encode().ToString()); + values.push_back(key_and_value.second.ToString()); + } + if (dir == kReverse) { + std::reverse(keys.begin(), keys.end()); + std::reverse(values.begin(), values.end()); + } + std::unique_ptr range_del_iter( + new test::VectorIterator(keys, values)); + range_del_agg.AddTombstones(std::move(range_del_iter)); + + for (const auto expected_point : expected_points) { + ParsedInternalKey parsed_key; + parsed_key.user_key = expected_point.begin; + parsed_key.sequence = expected_point.seq; + parsed_key.type = kTypeValue; + ASSERT_FALSE(range_del_agg.ShouldDelete( + parsed_key, + RangeDelAggregator::RangePositioningMode::kForwardTraversal)); + if (parsed_key.sequence > 0) { + --parsed_key.sequence; + ASSERT_TRUE(range_del_agg.ShouldDelete( + parsed_key, + RangeDelAggregator::RangePositioningMode::kForwardTraversal)); + } + } + } +} + +} // anonymous namespace + +TEST_F(RangeDelAggregatorTest, Empty) { VerifyRangeDels({}, {{"a", 0}}); } + +TEST_F(RangeDelAggregatorTest, SameStartAndEnd) { + VerifyRangeDels({{"a", "a", 5}}, {{" ", 0}, {"a", 0}, {"b", 0}}); +} + +TEST_F(RangeDelAggregatorTest, Single) { + VerifyRangeDels({{"a", "b", 10}}, {{" ", 0}, {"a", 10}, {"b", 0}}); +} + +TEST_F(RangeDelAggregatorTest, OverlapAboveLeft) { + VerifyRangeDels({{"a", "c", 10}, {"b", "d", 5}}, + {{" ", 0}, {"a", 10}, {"c", 5}, {"d", 0}}); +} + +TEST_F(RangeDelAggregatorTest, OverlapAboveRight) { + VerifyRangeDels({{"a", "c", 5}, {"b", "d", 10}}, + {{" ", 0}, {"a", 5}, {"b", 10}, {"d", 0}}); +} + +TEST_F(RangeDelAggregatorTest, OverlapAboveMiddle) { + VerifyRangeDels({{"a", "d", 5}, {"b", "c", 10}}, + {{" ", 0}, {"a", 5}, {"b", 10}, {"c", 5}, {"d", 0}}); +} + +TEST_F(RangeDelAggregatorTest, OverlapFully) { + VerifyRangeDels({{"a", "d", 10}, {"b", "c", 5}}, + {{" ", 0}, {"a", 10}, {"d", 0}}); +} + +TEST_F(RangeDelAggregatorTest, OverlapPoint) { + VerifyRangeDels({{"a", "b", 5}, {"b", "c", 10}}, + {{" ", 0}, {"a", 5}, {"b", 10}, {"c", 0}}); +} + +TEST_F(RangeDelAggregatorTest, SameStartKey) { + VerifyRangeDels({{"a", "c", 5}, {"a", "b", 10}}, + {{" ", 0}, {"a", 10}, {"b", 5}, {"c", 0}}); +} + +TEST_F(RangeDelAggregatorTest, SameEndKey) { + VerifyRangeDels({{"a", "d", 5}, {"b", "d", 10}}, + {{" ", 0}, {"a", 5}, {"b", 10}, {"d", 0}}); +} + +TEST_F(RangeDelAggregatorTest, GapsBetweenRanges) { + VerifyRangeDels( + {{"a", "b", 5}, {"c", "d", 10}, {"e", "f", 15}}, + {{" ", 0}, {"a", 5}, {"b", 0}, {"c", 10}, {"d", 0}, {"e", 15}, {"f", 0}}); +} + +// Note the Cover* tests also test cases where tombstones are inserted under a +// larger one when VerifyRangeDels() runs them in reverse +TEST_F(RangeDelAggregatorTest, CoverMultipleFromLeft) { + VerifyRangeDels( + {{"b", "d", 5}, {"c", "f", 10}, {"e", "g", 15}, {"a", "f", 20}}, + {{" ", 0}, {"a", 20}, {"f", 15}, {"g", 0}}); +} + +TEST_F(RangeDelAggregatorTest, CoverMultipleFromRight) { + VerifyRangeDels( + {{"b", "d", 5}, {"c", "f", 10}, {"e", "g", 15}, {"c", "h", 20}}, + {{" ", 0}, {"b", 5}, {"c", 20}, {"h", 0}}); +} + +TEST_F(RangeDelAggregatorTest, CoverMultipleFully) { + VerifyRangeDels( + {{"b", "d", 5}, {"c", "f", 10}, {"e", "g", 15}, {"a", "h", 20}}, + {{" ", 0}, {"a", 20}, {"h", 0}}); +} + +TEST_F(RangeDelAggregatorTest, AlternateMultipleAboveBelow) { + VerifyRangeDels( + {{"b", "d", 15}, {"c", "f", 10}, {"e", "g", 20}, {"a", "h", 5}}, + {{" ", 0}, + {"a", 5}, + {"b", 15}, + {"d", 10}, + {"e", 20}, + {"g", 5}, + {"h", 0}}); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/repair.cc b/db/repair.cc index 820cc19243f..9ed326032cb 100644 --- a/db/repair.cc +++ b/db/repair.cc @@ -1,51 +1,89 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. // -// We recover the contents of the descriptor from the other files we find. -// (1) Any log files are first converted to tables -// (2) We scan every table to compute -// (a) smallest/largest for the table -// (b) largest sequence number in the table -// (3) We generate descriptor contents: -// - log number is set to zero -// - next-file-number is set to 1 + largest file number we found -// - last-sequence-number is set to largest sequence# found across -// all tables (see 2c) -// - compaction pointers are cleared -// - every table file is added at level 0 +// Repairer does best effort recovery to recover as much data as possible after +// a disaster without compromising consistency. It does not guarantee bringing +// the database to a time consistent state. +// +// Repair process is broken into 4 phases: +// (a) Find files +// (b) Convert logs to tables +// (c) Extract metadata +// (d) Write Descriptor +// +// (a) Find files +// +// The repairer goes through all the files in the directory, and classifies them +// based on their file name. Any file that cannot be identified by name will be +// ignored. +// +// (b) Convert logs to table +// +// Every log file that is active is replayed. All sections of the file where the +// checksum does not match is skipped over. We intentionally give preference to +// data consistency. +// +// (c) Extract metadata +// +// We scan every table to compute +// (1) smallest/largest for the table +// (2) largest sequence number in the table +// +// If we are unable to scan the file, then we ignore the table. +// +// (d) Write Descriptor +// +// We generate descriptor contents: +// - log number is set to zero +// - next-file-number is set to 1 + largest file number we found +// - last-sequence-number is set to largest sequence# found across +// all tables (see 2c) +// - compaction pointers are cleared +// - every table file is added at level 0 // // Possible optimization 1: // (a) Compute total size and use to pick appropriate max-level M // (b) Sort tables by largest sequence# in the table // (c) For each table: if it overlaps earlier table, place in level-0, // else place in level-M. +// (d) We can provide options for time consistent recovery and unsafe recovery +// (ignore checksum failure when applicable) // Possible optimization 2: // Store per-table metadata (smallest, largest, largest-seq#, ...) // in the table's meta section to speed up ScanTable. #ifndef ROCKSDB_LITE +#ifndef __STDC_FORMAT_MACROS #define __STDC_FORMAT_MACROS +#endif + #include #include "db/builder.h" #include "db/db_impl.h" #include "db/dbformat.h" -#include "db/filename.h" #include "db/log_reader.h" #include "db/log_writer.h" #include "db/memtable.h" #include "db/table_cache.h" #include "db/version_edit.h" #include "db/write_batch_internal.h" +#include "options/cf_options.h" #include "rocksdb/comparator.h" #include "rocksdb/db.h" #include "rocksdb/env.h" +#include "rocksdb/options.h" +#include "rocksdb/write_buffer_manager.h" +#include "table/scoped_arena_iterator.h" +#include "util/file_reader_writer.h" +#include "util/filename.h" +#include "util/string_util.h" namespace rocksdb { @@ -53,47 +91,122 @@ namespace { class Repairer { public: - Repairer(const std::string& dbname, const Options& options) + Repairer(const std::string& dbname, const DBOptions& db_options, + const std::vector& column_families, + const ColumnFamilyOptions& default_cf_opts, + const ColumnFamilyOptions& unknown_cf_opts, bool create_unknown_cfs) : dbname_(dbname), - env_(options.env), - icmp_(options.comparator), - options_(SanitizeOptions(dbname, &icmp_, options)), + env_(db_options.env), + env_options_(), + db_options_(SanitizeOptions(dbname_, db_options)), + immutable_db_options_(db_options_), + icmp_(default_cf_opts.comparator), + default_cf_opts_(default_cf_opts), + default_cf_iopts_( + ImmutableCFOptions(immutable_db_options_, default_cf_opts)), + unknown_cf_opts_(unknown_cf_opts), + create_unknown_cfs_(create_unknown_cfs), raw_table_cache_( // TableCache can be small since we expect each table to be opened // once. - NewLRUCache(10, options_.table_cache_numshardbits, - options_.table_cache_remove_scan_count_limit)), + NewLRUCache(10, db_options_.table_cache_numshardbits)), + table_cache_(new TableCache(default_cf_iopts_, env_options_, + raw_table_cache_.get())), + wb_(db_options_.db_write_buffer_size), + wc_(db_options_.delayed_write_rate), + vset_(dbname_, &immutable_db_options_, env_options_, + raw_table_cache_.get(), &wb_, &wc_), next_file_number_(1) { - table_cache_ = - new TableCache(&options_, storage_options_, raw_table_cache_.get()); - edit_ = new VersionEdit(); + for (const auto& cfd : column_families) { + cf_name_to_opts_[cfd.name] = cfd.options; + } + } + + const ColumnFamilyOptions* GetColumnFamilyOptions( + const std::string& cf_name) { + if (cf_name_to_opts_.find(cf_name) == cf_name_to_opts_.end()) { + if (create_unknown_cfs_) { + return &unknown_cf_opts_; + } + return nullptr; + } + return &cf_name_to_opts_[cf_name]; + } + + // Adds a column family to the VersionSet with cf_options_ and updates + // manifest. + Status AddColumnFamily(const std::string& cf_name, uint32_t cf_id) { + const auto* cf_opts = GetColumnFamilyOptions(cf_name); + if (cf_opts == nullptr) { + return Status::Corruption("Encountered unknown column family with name=" + + cf_name + ", id=" + ToString(cf_id)); + } + Options opts(db_options_, *cf_opts); + MutableCFOptions mut_cf_opts(opts); + + VersionEdit edit; + edit.SetComparatorName(opts.comparator->Name()); + edit.SetLogNumber(0); + edit.SetColumnFamily(cf_id); + ColumnFamilyData* cfd; + cfd = nullptr; + edit.AddColumnFamily(cf_name); + + mutex_.Lock(); + Status status = vset_.LogAndApply(cfd, mut_cf_opts, &edit, &mutex_, + nullptr /* db_directory */, + false /* new_descriptor_log */, cf_opts); + mutex_.Unlock(); + return status; } ~Repairer() { delete table_cache_; - raw_table_cache_.reset(); - delete edit_; } Status Run() { Status status = FindFiles(); if (status.ok()) { + // Discard older manifests and start a fresh one + for (size_t i = 0; i < manifests_.size(); i++) { + ArchiveFile(dbname_ + "/" + manifests_[i]); + } + // Just create a DBImpl temporarily so we can reuse NewDB() + DBImpl* db_impl = new DBImpl(db_options_, dbname_); + status = db_impl->NewDB(); + delete db_impl; + } + + if (status.ok()) { + // Recover using the fresh manifest created by NewDB() + status = + vset_.Recover({{kDefaultColumnFamilyName, default_cf_opts_}}, false); + } + if (status.ok()) { + // Need to scan existing SST files first so the column families are + // created before we process WAL files + ExtractMetaData(); + + // ExtractMetaData() uses table_fds_ to know which SST files' metadata to + // extract -- we need to clear it here since metadata for existing SST + // files has been extracted already + table_fds_.clear(); ConvertLogFilesToTables(); ExtractMetaData(); - status = WriteDescriptor(); + status = AddTables(); } if (status.ok()) { uint64_t bytes = 0; for (size_t i = 0; i < tables_.size(); i++) { bytes += tables_[i].meta.fd.GetFileSize(); } - Log(options_.info_log, - "**** Repaired rocksdb %s; " - "recovered %zu files; %" PRIu64 - "bytes. " - "Some data may have been lost. " - "****", - dbname_.c_str(), tables_.size(), bytes); + ROCKS_LOG_WARN(db_options_.info_log, + "**** Repaired rocksdb %s; " + "recovered %" ROCKSDB_PRIszt " files; %" PRIu64 + "bytes. " + "Some data may have been lost. " + "****", + dbname_.c_str(), tables_.size(), bytes); } return status; } @@ -101,31 +214,54 @@ class Repairer { private: struct TableInfo { FileMetaData meta; + uint32_t column_family_id; + std::string column_family_name; SequenceNumber min_sequence; SequenceNumber max_sequence; }; std::string const dbname_; Env* const env_; - InternalKeyComparator const icmp_; - Options const options_; + const EnvOptions env_options_; + const DBOptions db_options_; + const ImmutableDBOptions immutable_db_options_; + const InternalKeyComparator icmp_; + const ColumnFamilyOptions default_cf_opts_; + const ImmutableCFOptions default_cf_iopts_; // table_cache_ holds reference + const ColumnFamilyOptions unknown_cf_opts_; + const bool create_unknown_cfs_; std::shared_ptr raw_table_cache_; TableCache* table_cache_; - VersionEdit* edit_; + WriteBufferManager wb_; + WriteController wc_; + VersionSet vset_; + std::unordered_map cf_name_to_opts_; + InstrumentedMutex mutex_; std::vector manifests_; std::vector table_fds_; std::vector logs_; std::vector tables_; uint64_t next_file_number_; - const EnvOptions storage_options_; Status FindFiles() { std::vector filenames; bool found_file = false; - for (uint32_t path_id = 0; path_id < options_.db_paths.size(); path_id++) { + std::vector to_search_paths; + + for (size_t path_id = 0; path_id < db_options_.db_paths.size(); path_id++) { + to_search_paths.push_back(db_options_.db_paths[path_id].path); + } + + // search wal_dir if user uses a customize wal_dir + if (!db_options_.wal_dir.empty() && + db_options_.wal_dir != dbname_) { + to_search_paths.push_back(db_options_.wal_dir); + } + + for (size_t path_id = 0; path_id < to_search_paths.size(); path_id++) { Status status = - env_->GetChildren(options_.db_paths[path_id].path, &filenames); + env_->GetChildren(to_search_paths[path_id], &filenames); if (!status.ok()) { return status; } @@ -138,17 +274,16 @@ class Repairer { for (size_t i = 0; i < filenames.size(); i++) { if (ParseFileName(filenames[i], &number, &type)) { if (type == kDescriptorFile) { - assert(path_id == 0); manifests_.push_back(filenames[i]); } else { if (number + 1 > next_file_number_) { next_file_number_ = number + 1; } if (type == kLogFile) { - assert(path_id == 0); logs_.push_back(number); } else if (type == kTableFile) { - table_fds_.emplace_back(number, path_id, 0); + table_fds_.emplace_back(number, static_cast(path_id), + 0); } else { // Ignore other files } @@ -164,12 +299,13 @@ class Repairer { void ConvertLogFilesToTables() { for (size_t i = 0; i < logs_.size(); i++) { - std::string logname = LogFileName(dbname_, logs_[i]); + // we should use LogFileName(wal_dir, logs_[i]) here. user might uses wal_dir option. + std::string logname = LogFileName(db_options_.wal_dir, logs_[i]); Status status = ConvertLogToTable(logs_[i]); if (!status.ok()) { - Log(options_.info_log, - "Log #%" PRIu64 ": ignoring conversion error: %s", logs_[i], - status.ToString().c_str()); + ROCKS_LOG_WARN(db_options_.info_log, + "Log #%" PRIu64 ": ignoring conversion error: %s", + logs_[i], status.ToString().c_str()); } ArchiveFile(logname); } @@ -180,79 +316,110 @@ class Repairer { Env* env; std::shared_ptr info_log; uint64_t lognum; - virtual void Corruption(size_t bytes, const Status& s) { + virtual void Corruption(size_t bytes, const Status& s) override { // We print error messages for corruption, but continue repairing. - Log(info_log, "Log #%" PRIu64 ": dropping %d bytes; %s", lognum, - static_cast(bytes), s.ToString().c_str()); + ROCKS_LOG_ERROR(info_log, "Log #%" PRIu64 ": dropping %d bytes; %s", + lognum, static_cast(bytes), s.ToString().c_str()); } }; // Open the log file - std::string logname = LogFileName(dbname_, log); + std::string logname = LogFileName(db_options_.wal_dir, log); unique_ptr lfile; - Status status = env_->NewSequentialFile(logname, &lfile, storage_options_); + Status status = env_->NewSequentialFile( + logname, &lfile, env_->OptimizeForLogRead(env_options_)); if (!status.ok()) { return status; } + unique_ptr lfile_reader( + new SequentialFileReader(std::move(lfile))); // Create the log reader. LogReporter reporter; reporter.env = env_; - reporter.info_log = options_.info_log; + reporter.info_log = db_options_.info_log; reporter.lognum = log; - // We intentially make log::Reader do checksumming so that + // We intentionally make log::Reader do checksumming so that // corruptions cause entire commits to be skipped instead of // propagating bad information (like overly large sequence // numbers). - log::Reader reader(std::move(lfile), &reporter, false/*do not checksum*/, - 0/*initial_offset*/); + log::Reader reader(db_options_.info_log, std::move(lfile_reader), &reporter, + true /*enable checksum*/, 0 /*initial_offset*/, log); + + // Initialize per-column family memtables + for (auto* cfd : *vset_.GetColumnFamilySet()) { + cfd->CreateNewMemtable(*cfd->GetLatestMutableCFOptions(), + kMaxSequenceNumber); + } + auto cf_mems = new ColumnFamilyMemTablesImpl(vset_.GetColumnFamilySet()); // Read all the records and add to a memtable std::string scratch; Slice record; WriteBatch batch; - MemTable* mem = new MemTable(icmp_, options_); - auto cf_mems_default = new ColumnFamilyMemTablesDefault(mem, &options_); - mem->Ref(); int counter = 0; while (reader.ReadRecord(&record, &scratch)) { - if (record.size() < 12) { + if (record.size() < WriteBatchInternal::kHeader) { reporter.Corruption( record.size(), Status::Corruption("log record too small")); continue; } WriteBatchInternal::SetContents(&batch, record); - status = WriteBatchInternal::InsertInto(&batch, cf_mems_default); + status = WriteBatchInternal::InsertInto(&batch, cf_mems, nullptr); if (status.ok()) { counter += WriteBatchInternal::Count(&batch); } else { - Log(options_.info_log, "Log #%" PRIu64 ": ignoring %s", log, - status.ToString().c_str()); + ROCKS_LOG_WARN(db_options_.info_log, "Log #%" PRIu64 ": ignoring %s", + log, status.ToString().c_str()); status = Status::OK(); // Keep going with rest of file } } - // Do not record a version edit for this conversion to a Table - // since ExtractMetaData() will also generate edits. - FileMetaData meta; - meta.fd = FileDescriptor(next_file_number_++, 0, 0); - ReadOptions ro; - ro.total_order_seek = true; - Iterator* iter = mem->NewIterator(ro); - status = BuildTable(dbname_, env_, options_, storage_options_, table_cache_, - iter, &meta, icmp_, 0, 0, kNoCompression); - delete iter; - delete mem->Unref(); - delete cf_mems_default; - mem = nullptr; - if (status.ok()) { - if (meta.fd.GetFileSize() > 0) { - table_fds_.push_back(meta.fd); + // Dump a table for each column family with entries in this log file. + for (auto* cfd : *vset_.GetColumnFamilySet()) { + // Do not record a version edit for this conversion to a Table + // since ExtractMetaData() will also generate edits. + MemTable* mem = cfd->mem(); + if (mem->IsEmpty()) { + continue; + } + + FileMetaData meta; + meta.fd = FileDescriptor(next_file_number_++, 0, 0); + ReadOptions ro; + ro.total_order_seek = true; + Arena arena; + ScopedArenaIterator iter(mem->NewIterator(ro, &arena)); + EnvOptions optimized_env_options = + env_->OptimizeForCompactionTableWrite(env_options_, immutable_db_options_); + + int64_t _current_time = 0; + status = env_->GetCurrentTime(&_current_time); // ignore error + const uint64_t current_time = static_cast(_current_time); + + status = BuildTable( + dbname_, env_, *cfd->ioptions(), *cfd->GetLatestMutableCFOptions(), + optimized_env_options, table_cache_, iter.get(), + std::unique_ptr(mem->NewRangeTombstoneIterator(ro)), + &meta, cfd->internal_comparator(), + cfd->int_tbl_prop_collector_factories(), cfd->GetID(), cfd->GetName(), + {}, kMaxSequenceNumber, kNoCompression, CompressionOptions(), false, + nullptr /* internal_stats */, TableFileCreationReason::kRecovery, + nullptr /* event_logger */, 0 /* job_id */, Env::IO_HIGH, + nullptr /* table_properties */, -1 /* level */, current_time); + ROCKS_LOG_INFO(db_options_.info_log, + "Log #%" PRIu64 ": %d ops saved to Table #%" PRIu64 " %s", + log, counter, meta.fd.GetNumber(), + status.ToString().c_str()); + if (status.ok()) { + if (meta.fd.GetFileSize() > 0) { + table_fds_.push_back(meta.fd); + } + } else { + break; } } - Log(options_.info_log, - "Log #%" PRIu64 ": %d ops saved to Table #%" PRIu64 " %s", log, counter, - meta.fd.GetNumber(), status.ToString().c_str()); + delete cf_mems; return status; } @@ -263,12 +430,12 @@ class Repairer { Status status = ScanTable(&t); if (!status.ok()) { std::string fname = TableFileName( - options_.db_paths, t.meta.fd.GetNumber(), t.meta.fd.GetPathId()); + db_options_.db_paths, t.meta.fd.GetNumber(), t.meta.fd.GetPathId()); char file_num_buf[kFormatFileNumberBufSize]; FormatFileNumber(t.meta.fd.GetNumber(), t.meta.fd.GetPathId(), file_num_buf, sizeof(file_num_buf)); - Log(options_.info_log, "Table #%s: ignoring %s", file_num_buf, - status.ToString().c_str()); + ROCKS_LOG_WARN(db_options_.info_log, "Table #%s: ignoring %s", + file_num_buf, status.ToString().c_str()); ArchiveFile(fname); } else { tables_.push_back(t); @@ -277,16 +444,55 @@ class Repairer { } Status ScanTable(TableInfo* t) { - std::string fname = TableFileName(options_.db_paths, t->meta.fd.GetNumber(), - t->meta.fd.GetPathId()); + std::string fname = TableFileName( + db_options_.db_paths, t->meta.fd.GetNumber(), t->meta.fd.GetPathId()); int counter = 0; uint64_t file_size; Status status = env_->GetFileSize(fname, &file_size); t->meta.fd = FileDescriptor(t->meta.fd.GetNumber(), t->meta.fd.GetPathId(), file_size); + std::shared_ptr props; + if (status.ok()) { + status = table_cache_->GetTableProperties(env_options_, icmp_, t->meta.fd, + &props); + } if (status.ok()) { - Iterator* iter = table_cache_->NewIterator( - ReadOptions(), storage_options_, icmp_, t->meta.fd); + t->column_family_id = static_cast(props->column_family_id); + if (t->column_family_id == + TablePropertiesCollectorFactory::Context::kUnknownColumnFamily) { + ROCKS_LOG_WARN( + db_options_.info_log, + "Table #%" PRIu64 + ": column family unknown (probably due to legacy format); " + "adding to default column family id 0.", + t->meta.fd.GetNumber()); + t->column_family_id = 0; + } + + if (vset_.GetColumnFamilySet()->GetColumnFamily(t->column_family_id) == + nullptr) { + status = + AddColumnFamily(props->column_family_name, t->column_family_id); + } + } + ColumnFamilyData* cfd = nullptr; + if (status.ok()) { + cfd = vset_.GetColumnFamilySet()->GetColumnFamily(t->column_family_id); + if (cfd->GetName() != props->column_family_name) { + ROCKS_LOG_ERROR( + db_options_.info_log, + "Table #%" PRIu64 + ": inconsistent column family name '%s'; expected '%s' for column " + "family id %" PRIu32 ".", + t->meta.fd.GetNumber(), props->column_family_name.c_str(), + cfd->GetName().c_str(), t->column_family_id); + status = Status::Corruption(dbname_, "inconsistent column family name"); + } + } + if (status.ok()) { + InternalIterator* iter = table_cache_->NewIterator( + ReadOptions(), env_options_, cfd->internal_comparator(), t->meta.fd, + nullptr /* range_del_agg */); bool empty = true; ParsedInternalKey parsed; t->min_sequence = 0; @@ -294,8 +500,9 @@ class Repairer { for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { Slice key = iter->key(); if (!ParseInternalKey(key, &parsed)) { - Log(options_.info_log, "Table #%" PRIu64 ": unparsable key %s", - t->meta.fd.GetNumber(), EscapeString(key).c_str()); + ROCKS_LOG_ERROR(db_options_.info_log, + "Table #%" PRIu64 ": unparsable key %s", + t->meta.fd.GetNumber(), EscapeString(key).c_str()); continue; } @@ -303,6 +510,7 @@ class Repairer { if (empty) { empty = false; t->meta.smallest.DecodeFrom(key); + t->min_sequence = parsed.sequence; } t->meta.largest.DecodeFrom(key); if (parsed.sequence < t->min_sequence) { @@ -316,66 +524,52 @@ class Repairer { status = iter->status(); } delete iter; + + ROCKS_LOG_INFO(db_options_.info_log, "Table #%" PRIu64 ": %d entries %s", + t->meta.fd.GetNumber(), counter, + status.ToString().c_str()); } - Log(options_.info_log, "Table #%" PRIu64 ": %d entries %s", - t->meta.fd.GetNumber(), counter, status.ToString().c_str()); return status; } - Status WriteDescriptor() { - std::string tmp = TempFileName(dbname_, 1); - unique_ptr file; - Status status = env_->NewWritableFile( - tmp, &file, env_->OptimizeForManifestWrite(storage_options_)); - if (!status.ok()) { - return status; - } - + Status AddTables() { + std::unordered_map> cf_id_to_tables; SequenceNumber max_sequence = 0; for (size_t i = 0; i < tables_.size(); i++) { + cf_id_to_tables[tables_[i].column_family_id].push_back(&tables_[i]); if (max_sequence < tables_[i].max_sequence) { max_sequence = tables_[i].max_sequence; } } + vset_.SetLastToBeWrittenSequence(max_sequence); + vset_.SetLastSequence(max_sequence); + + for (const auto& cf_id_and_tables : cf_id_to_tables) { + auto* cfd = + vset_.GetColumnFamilySet()->GetColumnFamily(cf_id_and_tables.first); + VersionEdit edit; + edit.SetComparatorName(cfd->user_comparator()->Name()); + edit.SetLogNumber(0); + edit.SetNextFile(next_file_number_); + edit.SetColumnFamily(cfd->GetID()); - edit_->SetComparatorName(icmp_.user_comparator()->Name()); - edit_->SetLogNumber(0); - edit_->SetNextFile(next_file_number_); - edit_->SetLastSequence(max_sequence); - - for (size_t i = 0; i < tables_.size(); i++) { // TODO(opt): separate out into multiple levels - const TableInfo& t = tables_[i]; - edit_->AddFile(0, t.meta.fd.GetNumber(), t.meta.fd.GetPathId(), - t.meta.fd.GetFileSize(), t.meta.smallest, t.meta.largest, - t.min_sequence, t.max_sequence); - } - - //fprintf(stderr, "NewDescriptor:\n%s\n", edit_.DebugString().c_str()); - { - log::Writer log(std::move(file)); - std::string record; - edit_->EncodeTo(&record); - status = log.AddRecord(record); - } - - if (!status.ok()) { - env_->DeleteFile(tmp); - } else { - // Discard older manifests - for (size_t i = 0; i < manifests_.size(); i++) { - ArchiveFile(dbname_ + "/" + manifests_[i]); + for (const auto* table : cf_id_and_tables.second) { + edit.AddFile(0, table->meta.fd.GetNumber(), table->meta.fd.GetPathId(), + table->meta.fd.GetFileSize(), table->meta.smallest, + table->meta.largest, table->min_sequence, + table->max_sequence, table->meta.marked_for_compaction); } - - // Install new manifest - status = env_->RenameFile(tmp, DescriptorFileName(dbname_, 1)); - if (status.ok()) { - status = SetCurrentFile(env_, dbname_, 1, nullptr); - } else { - env_->DeleteFile(tmp); + mutex_.Lock(); + Status status = vset_.LogAndApply( + cfd, *cfd->GetLatestMutableCFOptions(), &edit, &mutex_, + nullptr /* db_directory */, false /* new_descriptor_log */); + mutex_.Unlock(); + if (!status.ok()) { + return status; } } - return status; + return Status::OK(); } void ArchiveFile(const std::string& fname) { @@ -394,14 +588,60 @@ class Repairer { new_file.append("/"); new_file.append((slash == nullptr) ? fname.c_str() : slash + 1); Status s = env_->RenameFile(fname, new_file); - Log(options_.info_log, "Archiving %s: %s\n", - fname.c_str(), s.ToString().c_str()); + ROCKS_LOG_INFO(db_options_.info_log, "Archiving %s: %s\n", fname.c_str(), + s.ToString().c_str()); } }; -} // namespace + +Status GetDefaultCFOptions( + const std::vector& column_families, + ColumnFamilyOptions* res) { + assert(res != nullptr); + auto iter = std::find_if(column_families.begin(), column_families.end(), + [](const ColumnFamilyDescriptor& cfd) { + return cfd.name == kDefaultColumnFamilyName; + }); + if (iter == column_families.end()) { + return Status::InvalidArgument( + "column_families", "Must contain entry for default column family"); + } + *res = iter->options; + return Status::OK(); +} +} // anonymous namespace + +Status RepairDB(const std::string& dbname, const DBOptions& db_options, + const std::vector& column_families) { + ColumnFamilyOptions default_cf_opts; + Status status = GetDefaultCFOptions(column_families, &default_cf_opts); + if (status.ok()) { + Repairer repairer(dbname, db_options, column_families, default_cf_opts, + ColumnFamilyOptions() /* unknown_cf_opts */, + false /* create_unknown_cfs */); + status = repairer.Run(); + } + return status; +} + +Status RepairDB(const std::string& dbname, const DBOptions& db_options, + const std::vector& column_families, + const ColumnFamilyOptions& unknown_cf_opts) { + ColumnFamilyOptions default_cf_opts; + Status status = GetDefaultCFOptions(column_families, &default_cf_opts); + if (status.ok()) { + Repairer repairer(dbname, db_options, column_families, default_cf_opts, + unknown_cf_opts, true /* create_unknown_cfs */); + status = repairer.Run(); + } + return status; +} Status RepairDB(const std::string& dbname, const Options& options) { - Repairer repairer(dbname, options); + DBOptions db_options(options); + ColumnFamilyOptions cf_options(options); + Repairer repairer(dbname, db_options, {}, cf_options /* default_cf_opts */, + cf_options /* unknown_cf_opts */, + true /* create_unknown_cfs */); return repairer.Run(); } diff --git a/db/repair_test.cc b/db/repair_test.cc new file mode 100644 index 00000000000..b267c6d1683 --- /dev/null +++ b/db/repair_test.cc @@ -0,0 +1,328 @@ +// Copyright (c) 2016-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include +#include +#include + +#include "db/db_impl.h" +#include "db/db_test_util.h" +#include "rocksdb/comparator.h" +#include "rocksdb/db.h" +#include "rocksdb/transaction_log.h" +#include "util/file_util.h" +#include "util/string_util.h" + +namespace rocksdb { + +#ifndef ROCKSDB_LITE +class RepairTest : public DBTestBase { + public: + RepairTest() : DBTestBase("/repair_test") {} + + std::string GetFirstSstPath() { + uint64_t manifest_size; + std::vector files; + db_->GetLiveFiles(files, &manifest_size); + auto sst_iter = + std::find_if(files.begin(), files.end(), [](const std::string& file) { + uint64_t number; + FileType type; + bool ok = ParseFileName(file, &number, &type); + return ok && type == kTableFile; + }); + return sst_iter == files.end() ? "" : dbname_ + *sst_iter; + } +}; + +TEST_F(RepairTest, LostManifest) { + // Add a couple SST files, delete the manifest, and verify RepairDB() saves + // the day. + Put("key", "val"); + Flush(); + Put("key2", "val2"); + Flush(); + // Need to get path before Close() deletes db_, but delete it after Close() to + // ensure Close() didn't change the manifest. + std::string manifest_path = + DescriptorFileName(dbname_, dbfull()->TEST_Current_Manifest_FileNo()); + + Close(); + ASSERT_OK(env_->FileExists(manifest_path)); + ASSERT_OK(env_->DeleteFile(manifest_path)); + ASSERT_OK(RepairDB(dbname_, CurrentOptions())); + Reopen(CurrentOptions()); + + ASSERT_EQ(Get("key"), "val"); + ASSERT_EQ(Get("key2"), "val2"); +} + +TEST_F(RepairTest, CorruptManifest) { + // Manifest is in an invalid format. Expect a full recovery. + Put("key", "val"); + Flush(); + Put("key2", "val2"); + Flush(); + // Need to get path before Close() deletes db_, but overwrite it after Close() + // to ensure Close() didn't change the manifest. + std::string manifest_path = + DescriptorFileName(dbname_, dbfull()->TEST_Current_Manifest_FileNo()); + + Close(); + ASSERT_OK(env_->FileExists(manifest_path)); + CreateFile(env_, manifest_path, "blah"); + ASSERT_OK(RepairDB(dbname_, CurrentOptions())); + Reopen(CurrentOptions()); + + ASSERT_EQ(Get("key"), "val"); + ASSERT_EQ(Get("key2"), "val2"); +} + +TEST_F(RepairTest, IncompleteManifest) { + // In this case, the manifest is valid but does not reference all of the SST + // files. Expect a full recovery. + Put("key", "val"); + Flush(); + std::string orig_manifest_path = + DescriptorFileName(dbname_, dbfull()->TEST_Current_Manifest_FileNo()); + CopyFile(orig_manifest_path, orig_manifest_path + ".tmp"); + Put("key2", "val2"); + Flush(); + // Need to get path before Close() deletes db_, but overwrite it after Close() + // to ensure Close() didn't change the manifest. + std::string new_manifest_path = + DescriptorFileName(dbname_, dbfull()->TEST_Current_Manifest_FileNo()); + + Close(); + ASSERT_OK(env_->FileExists(new_manifest_path)); + // Replace the manifest with one that is only aware of the first SST file. + CopyFile(orig_manifest_path + ".tmp", new_manifest_path); + ASSERT_OK(RepairDB(dbname_, CurrentOptions())); + Reopen(CurrentOptions()); + + ASSERT_EQ(Get("key"), "val"); + ASSERT_EQ(Get("key2"), "val2"); +} + +TEST_F(RepairTest, LostSst) { + // Delete one of the SST files but preserve the manifest that refers to it, + // then verify the DB is still usable for the intact SST. + Put("key", "val"); + Flush(); + Put("key2", "val2"); + Flush(); + auto sst_path = GetFirstSstPath(); + ASSERT_FALSE(sst_path.empty()); + ASSERT_OK(env_->DeleteFile(sst_path)); + + Close(); + ASSERT_OK(RepairDB(dbname_, CurrentOptions())); + Reopen(CurrentOptions()); + + // Exactly one of the key-value pairs should be in the DB now. + ASSERT_TRUE((Get("key") == "val") != (Get("key2") == "val2")); +} + +TEST_F(RepairTest, CorruptSst) { + // Corrupt one of the SST files but preserve the manifest that refers to it, + // then verify the DB is still usable for the intact SST. + Put("key", "val"); + Flush(); + Put("key2", "val2"); + Flush(); + auto sst_path = GetFirstSstPath(); + ASSERT_FALSE(sst_path.empty()); + CreateFile(env_, sst_path, "blah"); + + Close(); + ASSERT_OK(RepairDB(dbname_, CurrentOptions())); + Reopen(CurrentOptions()); + + // Exactly one of the key-value pairs should be in the DB now. + ASSERT_TRUE((Get("key") == "val") != (Get("key2") == "val2")); +} + +TEST_F(RepairTest, UnflushedSst) { + // This test case invokes repair while some data is unflushed, then verifies + // that data is in the db. + Put("key", "val"); + VectorLogPtr wal_files; + ASSERT_OK(dbfull()->GetSortedWalFiles(wal_files)); + ASSERT_EQ(wal_files.size(), 1); + uint64_t total_ssts_size; + GetAllSSTFiles(&total_ssts_size); + ASSERT_EQ(total_ssts_size, 0); + // Need to get path before Close() deletes db_, but delete it after Close() to + // ensure Close() didn't change the manifest. + std::string manifest_path = + DescriptorFileName(dbname_, dbfull()->TEST_Current_Manifest_FileNo()); + + Close(); + ASSERT_OK(env_->FileExists(manifest_path)); + ASSERT_OK(env_->DeleteFile(manifest_path)); + ASSERT_OK(RepairDB(dbname_, CurrentOptions())); + Reopen(CurrentOptions()); + + ASSERT_OK(dbfull()->GetSortedWalFiles(wal_files)); + ASSERT_EQ(wal_files.size(), 0); + GetAllSSTFiles(&total_ssts_size); + ASSERT_GT(total_ssts_size, 0); + ASSERT_EQ(Get("key"), "val"); +} + +TEST_F(RepairTest, SeparateWalDir) { + do { + Options options = CurrentOptions(); + DestroyAndReopen(options); + Put("key", "val"); + Put("foo", "bar"); + VectorLogPtr wal_files; + ASSERT_OK(dbfull()->GetSortedWalFiles(wal_files)); + ASSERT_EQ(wal_files.size(), 1); + uint64_t total_ssts_size; + GetAllSSTFiles(&total_ssts_size); + ASSERT_EQ(total_ssts_size, 0); + std::string manifest_path = + DescriptorFileName(dbname_, dbfull()->TEST_Current_Manifest_FileNo()); + + Close(); + ASSERT_OK(env_->FileExists(manifest_path)); + ASSERT_OK(env_->DeleteFile(manifest_path)); + ASSERT_OK(RepairDB(dbname_, options)); + + // make sure that all WALs are converted to SSTables. + options.wal_dir = ""; + + Reopen(options); + ASSERT_OK(dbfull()->GetSortedWalFiles(wal_files)); + ASSERT_EQ(wal_files.size(), 0); + GetAllSSTFiles(&total_ssts_size); + ASSERT_GT(total_ssts_size, 0); + ASSERT_EQ(Get("key"), "val"); + ASSERT_EQ(Get("foo"), "bar"); + + } while(ChangeWalOptions()); +} + +TEST_F(RepairTest, RepairMultipleColumnFamilies) { + // Verify repair logic associates SST files with their original column + // families. + const int kNumCfs = 3; + const int kEntriesPerCf = 2; + DestroyAndReopen(CurrentOptions()); + CreateAndReopenWithCF({"pikachu1", "pikachu2"}, CurrentOptions()); + for (int i = 0; i < kNumCfs; ++i) { + for (int j = 0; j < kEntriesPerCf; ++j) { + Put(i, "key" + ToString(j), "val" + ToString(j)); + if (j == kEntriesPerCf - 1 && i == kNumCfs - 1) { + // Leave one unflushed so we can verify WAL entries are properly + // associated with column families. + continue; + } + Flush(i); + } + } + + // Need to get path before Close() deletes db_, but delete it after Close() to + // ensure Close() doesn't re-create the manifest. + std::string manifest_path = + DescriptorFileName(dbname_, dbfull()->TEST_Current_Manifest_FileNo()); + Close(); + ASSERT_OK(env_->FileExists(manifest_path)); + ASSERT_OK(env_->DeleteFile(manifest_path)); + + ASSERT_OK(RepairDB(dbname_, CurrentOptions())); + + ReopenWithColumnFamilies({"default", "pikachu1", "pikachu2"}, + CurrentOptions()); + for (int i = 0; i < kNumCfs; ++i) { + for (int j = 0; j < kEntriesPerCf; ++j) { + ASSERT_EQ(Get(i, "key" + ToString(j)), "val" + ToString(j)); + } + } +} + +TEST_F(RepairTest, RepairColumnFamilyOptions) { + // Verify repair logic uses correct ColumnFamilyOptions when repairing a + // database with different options for column families. + const int kNumCfs = 2; + const int kEntriesPerCf = 2; + + Options opts(CurrentOptions()), rev_opts(CurrentOptions()); + opts.comparator = BytewiseComparator(); + rev_opts.comparator = ReverseBytewiseComparator(); + + DestroyAndReopen(opts); + CreateColumnFamilies({"reverse"}, rev_opts); + ReopenWithColumnFamilies({"default", "reverse"}, + std::vector{opts, rev_opts}); + for (int i = 0; i < kNumCfs; ++i) { + for (int j = 0; j < kEntriesPerCf; ++j) { + Put(i, "key" + ToString(j), "val" + ToString(j)); + if (i == kNumCfs - 1 && j == kEntriesPerCf - 1) { + // Leave one unflushed so we can verify RepairDB's flush logic + continue; + } + Flush(i); + } + } + Close(); + + // RepairDB() records the comparator in the manifest, and DB::Open would fail + // if a different comparator were used. + ASSERT_OK(RepairDB(dbname_, opts, {{"default", opts}, {"reverse", rev_opts}}, + opts /* unknown_cf_opts */)); + ASSERT_OK(TryReopenWithColumnFamilies({"default", "reverse"}, + std::vector{opts, rev_opts})); + for (int i = 0; i < kNumCfs; ++i) { + for (int j = 0; j < kEntriesPerCf; ++j) { + ASSERT_EQ(Get(i, "key" + ToString(j)), "val" + ToString(j)); + } + } + + // Examine table properties to verify RepairDB() used the right options when + // converting WAL->SST + TablePropertiesCollection fname_to_props; + db_->GetPropertiesOfAllTables(handles_[1], &fname_to_props); + ASSERT_EQ(fname_to_props.size(), 2U); + for (const auto& fname_and_props : fname_to_props) { + std::string comparator_name ( + InternalKeyComparator(rev_opts.comparator).Name()); + comparator_name = comparator_name.substr(comparator_name.find(':') + 1); + ASSERT_EQ(comparator_name, + fname_and_props.second->comparator_name); + } + + // Also check comparator when it's provided via "unknown" CF options + ASSERT_OK(RepairDB(dbname_, opts, {{"default", opts}}, + rev_opts /* unknown_cf_opts */)); + ASSERT_OK(TryReopenWithColumnFamilies({"default", "reverse"}, + std::vector{opts, rev_opts})); + for (int i = 0; i < kNumCfs; ++i) { + for (int j = 0; j < kEntriesPerCf; ++j) { + ASSERT_EQ(Get(i, "key" + ToString(j)), "val" + ToString(j)); + } + } +} + +#endif // ROCKSDB_LITE +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, "SKIPPED as RepairDB is not supported in ROCKSDB_LITE\n"); + return 0; +} + +#endif // ROCKSDB_LITE diff --git a/db/simple_table_db_test.cc b/db/simple_table_db_test.cc deleted file mode 100644 index e88485070e5..00000000000 --- a/db/simple_table_db_test.cc +++ /dev/null @@ -1,810 +0,0 @@ -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. -#include -#include - -#include "rocksdb/db.h" -#include "rocksdb/filter_policy.h" -#include "db/db_impl.h" -#include "db/filename.h" -#include "db/version_set.h" -#include "db/write_batch_internal.h" -#include "rocksdb/statistics.h" -#include "rocksdb/cache.h" -#include "rocksdb/compaction_filter.h" -#include "rocksdb/env.h" -#include "rocksdb/table.h" -#include "rocksdb/table_properties.h" -#include "table/table_builder.h" -#include "util/hash.h" -#include "util/logging.h" -#include "util/mutexlock.h" -#include "util/testharness.h" -#include "util/testutil.h" -#include "utilities/merge_operators.h" - -using std::unique_ptr; - -// IS THIS FILE STILL NEEDED? -namespace rocksdb { - -// SimpleTable is a simple table format for UNIT TEST ONLY. It is not built -// as production quality. -// SimpleTable requires the input key size to be fixed 16 bytes, value cannot -// be longer than 150000 bytes and stored data on disk in this format: -// +--------------------------------------------+ <= key1 offset -// | key1 | value_size (4 bytes) | | -// +----------------------------------------+ | -// | value1 | -// | | -// +----------------------------------------+---+ <= key2 offset -// | key2 | value_size (4 bytes) | | -// +----------------------------------------+ | -// | value2 | -// | | -// | ...... | -// +-----------------+--------------------------+ <= index_block_offset -// | key1 | key1 offset (8 bytes) | -// +-----------------+--------------------------+ -// | key2 | key2 offset (8 bytes) | -// +-----------------+--------------------------+ -// | key3 | key3 offset (8 bytes) | -// +-----------------+--------------------------+ -// | ...... | -// +-----------------+------------+-------------+ -// | index_block_offset (8 bytes) | -// +------------------------------+ - -// SimpleTable is a simple table format for UNIT TEST ONLY. It is not built -// as production quality. -class SimpleTableReader: public TableReader { -public: - // Attempt to open the table that is stored in bytes [0..file_size) - // of "file", and read the metadata entries necessary to allow - // retrieving data from the table. - // - // If successful, returns ok and sets "*table" to the newly opened - // table. The client should delete "*table" when no longer needed. - // If there was an error while initializing the table, sets "*table" - // to nullptr and returns a non-ok status. Does not take ownership of - // "*source", but the client must ensure that "source" remains live - // for the duration of the returned table's lifetime. - // - // *file must remain live while this Table is in use. - static Status Open(const Options& options, const EnvOptions& soptions, - unique_ptr && file, uint64_t file_size, - unique_ptr* table_reader); - - Iterator* NewIterator(const ReadOptions&, Arena* arena) override; - - Status Get(const ReadOptions&, const Slice& key, void* arg, - bool (*handle_result)(void* arg, const ParsedInternalKey& k, - const Slice& v), - void (*mark_key_may_exist)(void*) = nullptr) override; - - uint64_t ApproximateOffsetOf(const Slice& key) override; - - virtual size_t ApproximateMemoryUsage() const override { return 0; } - - void SetupForCompaction() override; - - std::shared_ptr GetTableProperties() const override; - - ~SimpleTableReader(); - -private: - struct Rep; - Rep* rep_; - - explicit SimpleTableReader(Rep* rep) { - rep_ = rep; - } - friend class TableCache; - friend class SimpleTableIterator; - - Status GetOffset(const Slice& target, uint64_t* offset); - - // No copying allowed - explicit SimpleTableReader(const TableReader&) = delete; - void operator=(const TableReader&) = delete; -}; - -// Iterator to iterate SimpleTable -class SimpleTableIterator: public Iterator { -public: - explicit SimpleTableIterator(SimpleTableReader* table); - ~SimpleTableIterator(); - - bool Valid() const; - - void SeekToFirst(); - - void SeekToLast(); - - void Seek(const Slice& target); - - void Next(); - - void Prev(); - - Slice key() const; - - Slice value() const; - - Status status() const; - -private: - SimpleTableReader* table_; - uint64_t offset_; - uint64_t next_offset_; - Slice key_; - Slice value_; - char tmp_str_[4]; - char* key_str_; - char* value_str_; - int value_str_len_; - Status status_; - // No copying allowed - SimpleTableIterator(const SimpleTableIterator&) = delete; - void operator=(const Iterator&) = delete; -}; - -struct SimpleTableReader::Rep { - ~Rep() { - } - Rep(const EnvOptions& storage_options, uint64_t index_start_offset, - int num_entries) : - soptions(storage_options), index_start_offset(index_start_offset), - num_entries(num_entries) { - } - - Options options; - const EnvOptions& soptions; - Status status; - unique_ptr file; - uint64_t index_start_offset; - int num_entries; - std::shared_ptr table_properties; - - const static int user_key_size = 16; - const static int offset_length = 8; - const static int key_footer_len = 8; - - static int GetInternalKeyLength() { - return user_key_size + key_footer_len; - } -}; - -SimpleTableReader::~SimpleTableReader() { - delete rep_; -} - -Status SimpleTableReader::Open(const Options& options, - const EnvOptions& soptions, - unique_ptr && file, - uint64_t size, - unique_ptr* table_reader) { - char footer_space[Rep::offset_length]; - Slice footer_input; - Status s = file->Read(size - Rep::offset_length, Rep::offset_length, - &footer_input, footer_space); - if (s.ok()) { - uint64_t index_start_offset = DecodeFixed64(footer_space); - - int num_entries = (size - Rep::offset_length - index_start_offset) - / (Rep::GetInternalKeyLength() + Rep::offset_length); - SimpleTableReader::Rep* rep = new SimpleTableReader::Rep(soptions, - index_start_offset, - num_entries); - - rep->file = std::move(file); - rep->options = options; - table_reader->reset(new SimpleTableReader(rep)); - } - return s; -} - -void SimpleTableReader::SetupForCompaction() { -} - -std::shared_ptr SimpleTableReader::GetTableProperties() - const { - return rep_->table_properties; -} - -Iterator* SimpleTableReader::NewIterator(const ReadOptions& options, - Arena* arena) { - if (arena == nullptr) { - return new SimpleTableIterator(this); - } else { - auto mem = arena->AllocateAligned(sizeof(SimpleTableIterator)); - return new (mem) SimpleTableIterator(this); - } -} - -Status SimpleTableReader::GetOffset(const Slice& target, uint64_t* offset) { - uint32_t left = 0; - uint32_t right = rep_->num_entries - 1; - char key_chars[Rep::GetInternalKeyLength()]; - Slice tmp_slice; - - uint32_t target_offset = 0; - while (left <= right) { - uint32_t mid = (left + right + 1) / 2; - - uint64_t offset_to_read = rep_->index_start_offset - + (Rep::GetInternalKeyLength() + Rep::offset_length) * mid; - Status s = rep_->file->Read(offset_to_read, Rep::GetInternalKeyLength(), - &tmp_slice, key_chars); - if (!s.ok()) { - return s; - } - - InternalKeyComparator ikc(rep_->options.comparator); - int compare_result = ikc.Compare(tmp_slice, target); - - if (compare_result < 0) { - if (left == right) { - target_offset = right + 1; - break; - } - left = mid; - } else { - if (left == right) { - target_offset = left; - break; - } - right = mid - 1; - } - } - - if (target_offset >= (uint32_t) rep_->num_entries) { - *offset = rep_->index_start_offset; - return Status::OK(); - } - - char value_offset_chars[Rep::offset_length]; - - int64_t offset_for_value_offset = rep_->index_start_offset - + (Rep::GetInternalKeyLength() + Rep::offset_length) * target_offset - + Rep::GetInternalKeyLength(); - Status s = rep_->file->Read(offset_for_value_offset, Rep::offset_length, - &tmp_slice, value_offset_chars); - if (s.ok()) { - *offset = DecodeFixed64(value_offset_chars); - } - return s; -} - -Status SimpleTableReader::Get(const ReadOptions& options, const Slice& k, - void* arg, - bool (*saver)(void*, const ParsedInternalKey&, - const Slice&), - void (*mark_key_may_exist)(void*)) { - Status s; - SimpleTableIterator* iter = new SimpleTableIterator(this); - for (iter->Seek(k); iter->Valid(); iter->Next()) { - ParsedInternalKey parsed_key; - if (!ParseInternalKey(iter->key(), &parsed_key)) { - return Status::Corruption(Slice()); - } - - if (!(*saver)(arg, parsed_key, iter->value())) { - break; - } - } - s = iter->status(); - delete iter; - return s; -} - -uint64_t SimpleTableReader::ApproximateOffsetOf(const Slice& key) { - return 0; -} - -SimpleTableIterator::SimpleTableIterator(SimpleTableReader* table) : - table_(table) { - key_str_ = new char[SimpleTableReader::Rep::GetInternalKeyLength()]; - value_str_len_ = -1; - SeekToFirst(); -} - -SimpleTableIterator::~SimpleTableIterator() { - delete[] key_str_; - if (value_str_len_ >= 0) { - delete[] value_str_; - } -} - -bool SimpleTableIterator::Valid() const { - return offset_ < table_->rep_->index_start_offset; -} - -void SimpleTableIterator::SeekToFirst() { - next_offset_ = 0; - Next(); -} - -void SimpleTableIterator::SeekToLast() { - assert(false); -} - -void SimpleTableIterator::Seek(const Slice& target) { - Status s = table_->GetOffset(target, &next_offset_); - if (!s.ok()) { - status_ = s; - } - Next(); -} - -void SimpleTableIterator::Next() { - offset_ = next_offset_; - if (offset_ >= table_->rep_->index_start_offset) { - return; - } - Slice result; - int internal_key_size = SimpleTableReader::Rep::GetInternalKeyLength(); - - Status s = table_->rep_->file->Read(next_offset_, internal_key_size, &result, - key_str_); - next_offset_ += internal_key_size; - key_ = result; - - Slice value_size_slice; - s = table_->rep_->file->Read(next_offset_, 4, &value_size_slice, tmp_str_); - next_offset_ += 4; - uint32_t value_size = DecodeFixed32(tmp_str_); - - Slice value_slice; - if ((int) value_size > value_str_len_) { - if (value_str_len_ >= 0) { - delete[] value_str_; - } - value_str_ = new char[value_size]; - value_str_len_ = value_size; - } - s = table_->rep_->file->Read(next_offset_, value_size, &value_slice, - value_str_); - next_offset_ += value_size; - value_ = value_slice; -} - -void SimpleTableIterator::Prev() { - assert(false); -} - -Slice SimpleTableIterator::key() const { - Log(table_->rep_->options.info_log, "key!!!!"); - return key_; -} - -Slice SimpleTableIterator::value() const { - return value_; -} - -Status SimpleTableIterator::status() const { - return status_; -} - -class SimpleTableBuilder: public TableBuilder { -public: - // Create a builder that will store the contents of the table it is - // building in *file. Does not close the file. It is up to the - // caller to close the file after calling Finish(). The output file - // will be part of level specified by 'level'. A value of -1 means - // that the caller does not know which level the output file will reside. - SimpleTableBuilder(const Options& options, WritableFile* file, - CompressionType compression_type); - - // REQUIRES: Either Finish() or Abandon() has been called. - ~SimpleTableBuilder(); - - // Add key,value to the table being constructed. - // REQUIRES: key is after any previously added key according to comparator. - // REQUIRES: Finish(), Abandon() have not been called - void Add(const Slice& key, const Slice& value) override; - - // Return non-ok iff some error has been detected. - Status status() const override; - - // Finish building the table. Stops using the file passed to the - // constructor after this function returns. - // REQUIRES: Finish(), Abandon() have not been called - Status Finish() override; - - // Indicate that the contents of this builder should be abandoned. Stops - // using the file passed to the constructor after this function returns. - // If the caller is not going to call Finish(), it must call Abandon() - // before destroying this builder. - // REQUIRES: Finish(), Abandon() have not been called - void Abandon() override; - - // Number of calls to Add() so far. - uint64_t NumEntries() const override; - - // Size of the file generated so far. If invoked after a successful - // Finish() call, returns the size of the final generated file. - uint64_t FileSize() const override; - -private: - struct Rep; - Rep* rep_; - - // No copying allowed - SimpleTableBuilder(const SimpleTableBuilder&) = delete; - void operator=(const SimpleTableBuilder&) = delete; -}; - -struct SimpleTableBuilder::Rep { - Options options; - WritableFile* file; - uint64_t offset = 0; - Status status; - - uint64_t num_entries = 0; - - bool closed = false; // Either Finish() or Abandon() has been called. - - const static int user_key_size = 16; - const static int offset_length = 8; - const static int key_footer_len = 8; - - static int GetInternalKeyLength() { - return user_key_size + key_footer_len; - } - - std::string index; - - Rep(const Options& opt, WritableFile* f) : - options(opt), file(f) { - } - ~Rep() { - } -}; - -SimpleTableBuilder::SimpleTableBuilder(const Options& options, - WritableFile* file, - CompressionType compression_type) : - rep_(new SimpleTableBuilder::Rep(options, file)) { -} - -SimpleTableBuilder::~SimpleTableBuilder() { - delete (rep_); -} - -void SimpleTableBuilder::Add(const Slice& key, const Slice& value) { - assert((int ) key.size() == Rep::GetInternalKeyLength()); - - // Update index - rep_->index.append(key.data(), key.size()); - PutFixed64(&(rep_->index), rep_->offset); - - // Write key-value pair - rep_->file->Append(key); - rep_->offset += Rep::GetInternalKeyLength(); - - std::string size; - int value_size = value.size(); - PutFixed32(&size, value_size); - Slice sizeSlice(size); - rep_->file->Append(sizeSlice); - rep_->file->Append(value); - rep_->offset += value_size + 4; - - rep_->num_entries++; -} - -Status SimpleTableBuilder::status() const { - return Status::OK(); -} - -Status SimpleTableBuilder::Finish() { - Rep* r = rep_; - assert(!r->closed); - r->closed = true; - - uint64_t index_offset = rep_->offset; - Slice index_slice(rep_->index); - rep_->file->Append(index_slice); - rep_->offset += index_slice.size(); - - std::string index_offset_str; - PutFixed64(&index_offset_str, index_offset); - Slice foot_slice(index_offset_str); - rep_->file->Append(foot_slice); - rep_->offset += foot_slice.size(); - - return Status::OK(); -} - -void SimpleTableBuilder::Abandon() { - rep_->closed = true; -} - -uint64_t SimpleTableBuilder::NumEntries() const { - return rep_->num_entries; -} - -uint64_t SimpleTableBuilder::FileSize() const { - return rep_->offset; -} - -class SimpleTableFactory: public TableFactory { -public: - ~SimpleTableFactory() { - } - SimpleTableFactory() { - } - const char* Name() const override { - return "SimpleTable"; - } - Status NewTableReader(const Options& options, const EnvOptions& soptions, - const InternalKeyComparator& internal_key, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table_reader) const; - - TableBuilder* NewTableBuilder(const Options& options, - const InternalKeyComparator& internal_key, - WritableFile* file, - CompressionType compression_type) const; - - virtual Status SanitizeDBOptions(const DBOptions* db_opts) const override { - return Status::OK(); - } - - virtual std::string GetPrintableTableOptions() const override { - return std::string(); - } -}; - -Status SimpleTableFactory::NewTableReader( - const Options& options, const EnvOptions& soptions, - const InternalKeyComparator& internal_key, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table_reader) const { - - return SimpleTableReader::Open(options, soptions, std::move(file), file_size, - table_reader); -} - -TableBuilder* SimpleTableFactory::NewTableBuilder( - const Options& options, const InternalKeyComparator& internal_key, - WritableFile* file, CompressionType compression_type) const { - return new SimpleTableBuilder(options, file, compression_type); -} - -class SimpleTableDBTest { -protected: -public: - std::string dbname_; - Env* env_; - DB* db_; - - Options last_options_; - - SimpleTableDBTest() : - env_(Env::Default()) { - dbname_ = test::TmpDir() + "/simple_table_db_test"; - ASSERT_OK(DestroyDB(dbname_, Options())); - db_ = nullptr; - Reopen(); - } - - ~SimpleTableDBTest() { - delete db_; - ASSERT_OK(DestroyDB(dbname_, Options())); - } - - // Return the current option configuration. - Options CurrentOptions() { - Options options; - options.table_factory.reset(new SimpleTableFactory()); - return options; - } - - DBImpl* dbfull() { - return reinterpret_cast(db_); - } - - void Reopen(Options* options = nullptr) { - ASSERT_OK(TryReopen(options)); - } - - void Close() { - delete db_; - db_ = nullptr; - } - - void DestroyAndReopen(Options* options = nullptr) { - //Destroy using last options - Destroy(&last_options_); - ASSERT_OK(TryReopen(options)); - } - - void Destroy(Options* options) { - delete db_; - db_ = nullptr; - ASSERT_OK(DestroyDB(dbname_, *options)); - } - - Status PureReopen(Options* options, DB** db) { - return DB::Open(*options, dbname_, db); - } - - Status TryReopen(Options* options = nullptr) { - delete db_; - db_ = nullptr; - Options opts; - if (options != nullptr) { - opts = *options; - } else { - opts = CurrentOptions(); - opts.create_if_missing = true; - } - last_options_ = opts; - - return DB::Open(opts, dbname_, &db_); - } - - Status Put(const Slice& k, const Slice& v) { - return db_->Put(WriteOptions(), k, v); - } - - Status Delete(const std::string& k) { - return db_->Delete(WriteOptions(), k); - } - - std::string Get(const std::string& k, const Snapshot* snapshot = nullptr) { - ReadOptions options; - options.snapshot = snapshot; - std::string result; - Status s = db_->Get(options, k, &result); - if (s.IsNotFound()) { - result = "NOT_FOUND"; - } else if (!s.ok()) { - result = s.ToString(); - } - return result; - } - - - int NumTableFilesAtLevel(int level) { - std::string property; - ASSERT_TRUE( - db_->GetProperty("rocksdb.num-files-at-level" + NumberToString(level), - &property)); - return atoi(property.c_str()); - } - - // Return spread of files per level - std::string FilesPerLevel() { - std::string result; - int last_non_zero_offset = 0; - for (int level = 0; level < db_->NumberLevels(); level++) { - int f = NumTableFilesAtLevel(level); - char buf[100]; - snprintf(buf, sizeof(buf), "%s%d", (level ? "," : ""), f); - result += buf; - if (f > 0) { - last_non_zero_offset = result.size(); - } - } - result.resize(last_non_zero_offset); - return result; - } - - std::string IterStatus(Iterator* iter) { - std::string result; - if (iter->Valid()) { - result = iter->key().ToString() + "->" + iter->value().ToString(); - } else { - result = "(invalid)"; - } - return result; - } -}; - -TEST(SimpleTableDBTest, Empty) { - ASSERT_TRUE(db_ != nullptr); - ASSERT_EQ("NOT_FOUND", Get("0000000000000foo")); -} - -TEST(SimpleTableDBTest, ReadWrite) { - ASSERT_OK(Put("0000000000000foo", "v1")); - ASSERT_EQ("v1", Get("0000000000000foo")); - ASSERT_OK(Put("0000000000000bar", "v2")); - ASSERT_OK(Put("0000000000000foo", "v3")); - ASSERT_EQ("v3", Get("0000000000000foo")); - ASSERT_EQ("v2", Get("0000000000000bar")); -} - -TEST(SimpleTableDBTest, Flush) { - ASSERT_OK(Put("0000000000000foo", "v1")); - ASSERT_OK(Put("0000000000000bar", "v2")); - ASSERT_OK(Put("0000000000000foo", "v3")); - dbfull()->TEST_FlushMemTable(); - ASSERT_EQ("v3", Get("0000000000000foo")); - ASSERT_EQ("v2", Get("0000000000000bar")); -} - -TEST(SimpleTableDBTest, Flush2) { - ASSERT_OK(Put("0000000000000bar", "b")); - ASSERT_OK(Put("0000000000000foo", "v1")); - dbfull()->TEST_FlushMemTable(); - - ASSERT_OK(Put("0000000000000foo", "v2")); - dbfull()->TEST_FlushMemTable(); - ASSERT_EQ("v2", Get("0000000000000foo")); - - ASSERT_OK(Put("0000000000000eee", "v3")); - dbfull()->TEST_FlushMemTable(); - ASSERT_EQ("v3", Get("0000000000000eee")); - - ASSERT_OK(Delete("0000000000000bar")); - dbfull()->TEST_FlushMemTable(); - ASSERT_EQ("NOT_FOUND", Get("0000000000000bar")); - - ASSERT_OK(Put("0000000000000eee", "v5")); - dbfull()->TEST_FlushMemTable(); - ASSERT_EQ("v5", Get("0000000000000eee")); -} - -static std::string Key(int i) { - char buf[100]; - snprintf(buf, sizeof(buf), "key_______%06d", i); - return std::string(buf); -} - -static std::string RandomString(Random* rnd, int len) { - std::string r; - test::RandomString(rnd, len, &r); - return r; -} - -TEST(SimpleTableDBTest, CompactionTrigger) { - Options options = CurrentOptions(); - options.write_buffer_size = 100 << 10; //100KB - options.num_levels = 3; - options.max_mem_compaction_level = 0; - options.level0_file_num_compaction_trigger = 3; - Reopen(&options); - - Random rnd(301); - - for (int num = 0; num < options.level0_file_num_compaction_trigger - 1; - num++) { - std::vector values; - // Write 120KB (12 values, each 10K) - for (int i = 0; i < 12; i++) { - values.push_back(RandomString(&rnd, 10000)); - ASSERT_OK(Put(Key(i), values[i])); - } - dbfull()->TEST_WaitForFlushMemTable(); - ASSERT_EQ(NumTableFilesAtLevel(0), num + 1); - } - - //generate one more file in level-0, and should trigger level-0 compaction - std::vector values; - for (int i = 0; i < 12; i++) { - values.push_back(RandomString(&rnd, 10000)); - ASSERT_OK(Put(Key(i), values[i])); - } - dbfull()->TEST_WaitForCompact(); - - ASSERT_EQ(NumTableFilesAtLevel(0), 0); - ASSERT_EQ(NumTableFilesAtLevel(1), 1); -} - -} // namespace rocksdb - -int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); -} diff --git a/db/snapshot.h b/db/snapshot.h deleted file mode 100644 index 2c2e3eac803..00000000000 --- a/db/snapshot.h +++ /dev/null @@ -1,86 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#pragma once -#include "rocksdb/db.h" - -namespace rocksdb { - -class SnapshotList; - -// Snapshots are kept in a doubly-linked list in the DB. -// Each SnapshotImpl corresponds to a particular sequence number. -class SnapshotImpl : public Snapshot { - public: - SequenceNumber number_; // const after creation - - private: - friend class SnapshotList; - - // SnapshotImpl is kept in a doubly-linked circular list - SnapshotImpl* prev_; - SnapshotImpl* next_; - - SnapshotList* list_; // just for sanity checks -}; - -class SnapshotList { - public: - SnapshotList() { - list_.prev_ = &list_; - list_.next_ = &list_; - list_.number_ = 0xFFFFFFFFL; // placeholder marker, for debugging - } - - bool empty() const { return list_.next_ == &list_; } - SnapshotImpl* oldest() const { assert(!empty()); return list_.next_; } - SnapshotImpl* newest() const { assert(!empty()); return list_.prev_; } - - const SnapshotImpl* New(SequenceNumber seq) { - SnapshotImpl* s = new SnapshotImpl; - s->number_ = seq; - s->list_ = this; - s->next_ = &list_; - s->prev_ = list_.prev_; - s->prev_->next_ = s; - s->next_->prev_ = s; - return s; - } - - void Delete(const SnapshotImpl* s) { - assert(s->list_ == this); - s->prev_->next_ = s->next_; - s->next_->prev_ = s->prev_; - delete s; - } - - // retrieve all snapshot numbers. They are sorted in ascending order. - void getAll(std::vector& ret) { - if (empty()) return; - SnapshotImpl* s = &list_; - while (s->next_ != &list_) { - ret.push_back(s->next_->number_); - s = s ->next_; - } - } - - // get the sequence number of the most recent snapshot - const SequenceNumber GetNewest() { - if (empty()) { - return 0; - } - return newest()->number_; - } - - private: - // Dummy head of doubly-linked list of snapshots - SnapshotImpl list_; -}; - -} // namespace rocksdb diff --git a/db/snapshot_impl.cc b/db/snapshot_impl.cc new file mode 100644 index 00000000000..032ef398aa5 --- /dev/null +++ b/db/snapshot_impl.cc @@ -0,0 +1,26 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "rocksdb/snapshot.h" + +#include "rocksdb/db.h" + +namespace rocksdb { + +ManagedSnapshot::ManagedSnapshot(DB* db) : db_(db), + snapshot_(db->GetSnapshot()) {} + +ManagedSnapshot::ManagedSnapshot(DB* db, const Snapshot* _snapshot) + : db_(db), snapshot_(_snapshot) {} + +ManagedSnapshot::~ManagedSnapshot() { + if (snapshot_) { + db_->ReleaseSnapshot(snapshot_); + } +} + +const Snapshot* ManagedSnapshot::snapshot() { return snapshot_;} + +} // namespace rocksdb diff --git a/db/snapshot_impl.h b/db/snapshot_impl.h new file mode 100644 index 00000000000..7dc405931cb --- /dev/null +++ b/db/snapshot_impl.h @@ -0,0 +1,151 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once +#include + +#include "rocksdb/db.h" + +namespace rocksdb { + +class SnapshotList; + +// Snapshots are kept in a doubly-linked list in the DB. +// Each SnapshotImpl corresponds to a particular sequence number. +class SnapshotImpl : public Snapshot { + public: + SequenceNumber number_; // const after creation + + virtual SequenceNumber GetSequenceNumber() const override { return number_; } + + private: + friend class SnapshotList; + + // SnapshotImpl is kept in a doubly-linked circular list + SnapshotImpl* prev_; + SnapshotImpl* next_; + + SnapshotList* list_; // just for sanity checks + + int64_t unix_time_; + + // Will this snapshot be used by a Transaction to do write-conflict checking? + bool is_write_conflict_boundary_; +}; + +class SnapshotList { + public: + SnapshotList() { + list_.prev_ = &list_; + list_.next_ = &list_; + list_.number_ = 0xFFFFFFFFL; // placeholder marker, for debugging + count_ = 0; + } + + bool empty() const { return list_.next_ == &list_; } + SnapshotImpl* oldest() const { assert(!empty()); return list_.next_; } + SnapshotImpl* newest() const { assert(!empty()); return list_.prev_; } + + const SnapshotImpl* New(SnapshotImpl* s, SequenceNumber seq, + uint64_t unix_time, bool is_write_conflict_boundary) { + s->number_ = seq; + s->unix_time_ = unix_time; + s->is_write_conflict_boundary_ = is_write_conflict_boundary; + s->list_ = this; + s->next_ = &list_; + s->prev_ = list_.prev_; + s->prev_->next_ = s; + s->next_->prev_ = s; + count_++; + return s; + } + + // Do not responsible to free the object. + void Delete(const SnapshotImpl* s) { + assert(s->list_ == this); + s->prev_->next_ = s->next_; + s->next_->prev_ = s->prev_; + count_--; + } + + // retrieve all snapshot numbers up until max_seq. They are sorted in + // ascending order. + std::vector GetAll( + SequenceNumber* oldest_write_conflict_snapshot = nullptr, + const SequenceNumber& max_seq = kMaxSequenceNumber) const { + std::vector ret; + + if (oldest_write_conflict_snapshot != nullptr) { + *oldest_write_conflict_snapshot = kMaxSequenceNumber; + } + + if (empty()) { + return ret; + } + const SnapshotImpl* s = &list_; + while (s->next_ != &list_) { + if (s->next_->number_ > max_seq) { + break; + } + ret.push_back(s->next_->number_); + + if (oldest_write_conflict_snapshot != nullptr && + *oldest_write_conflict_snapshot == kMaxSequenceNumber && + s->next_->is_write_conflict_boundary_) { + // If this is the first write-conflict boundary snapshot in the list, + // it is the oldest + *oldest_write_conflict_snapshot = s->next_->number_; + } + + s = s->next_; + } + return ret; + } + + // Whether there is an active snapshot in range [lower_bound, upper_bound). + bool HasSnapshotInRange(SequenceNumber lower_bound, + SequenceNumber upper_bound) { + if (empty()) { + return false; + } + const SnapshotImpl* s = &list_; + while (s->next_ != &list_) { + if (s->next_->number_ >= lower_bound) { + return s->next_->number_ < upper_bound; + } + s = s->next_; + } + return false; + } + + // get the sequence number of the most recent snapshot + SequenceNumber GetNewest() { + if (empty()) { + return 0; + } + return newest()->number_; + } + + int64_t GetOldestSnapshotTime() const { + if (empty()) { + return 0; + } else { + return oldest()->unix_time_; + } + } + + uint64_t count() const { return count_; } + + private: + // Dummy head of doubly-linked list of snapshots + SnapshotImpl list_; + uint64_t count_; +}; + +} // namespace rocksdb diff --git a/db/table_cache.cc b/db/table_cache.cc index c362499a685..b4d5cc1bb71 100644 --- a/db/table_cache.cc +++ b/db/table_cache.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,20 +9,30 @@ #include "db/table_cache.h" -#include "db/filename.h" +#include "db/dbformat.h" #include "db/version_edit.h" +#include "util/filename.h" +#include "monitoring/perf_context_imp.h" #include "rocksdb/statistics.h" +#include "table/get_context.h" +#include "table/internal_iterator.h" #include "table/iterator_wrapper.h" +#include "table/table_builder.h" #include "table/table_reader.h" #include "util/coding.h" +#include "util/file_reader_writer.h" #include "util/stop_watch.h" +#include "util/sync_point.h" namespace rocksdb { +namespace { + +template static void DeleteEntry(const Slice& key, void* value) { - TableReader* table_reader = reinterpret_cast(value); - delete table_reader; + T* typed_value = reinterpret_cast(value); + delete typed_value; } static void UnrefEntry(void* arg1, void* arg2) { @@ -31,18 +41,37 @@ static void UnrefEntry(void* arg1, void* arg2) { cache->Release(h); } +static void DeleteTableReader(void* arg1, void* arg2) { + TableReader* table_reader = reinterpret_cast(arg1); + delete table_reader; +} + static Slice GetSliceForFileNumber(const uint64_t* file_number) { return Slice(reinterpret_cast(file_number), sizeof(*file_number)); } -TableCache::TableCache(const Options* options, - const EnvOptions& storage_options, Cache* const cache) - : env_(options->env), - db_paths_(options->db_paths), - options_(options), - storage_options_(storage_options), - cache_(cache) {} +#ifndef ROCKSDB_LITE + +void AppendVarint64(IterKey* key, uint64_t v) { + char buf[10]; + auto ptr = EncodeVarint64(buf, v); + key->TrimAppend(key->Size(), buf, ptr - buf); +} + +#endif // ROCKSDB_LITE + +} // namespace + +TableCache::TableCache(const ImmutableCFOptions& ioptions, + const EnvOptions& env_options, Cache* const cache) + : ioptions_(ioptions), env_options_(env_options), cache_(cache) { + if (ioptions_.row_cache) { + // If the same cache is shared by multiple instances, we need to + // disambiguate its entries. + PutVarint64(&row_cache_id_, ioptions_.row_cache->NewId()); + } +} TableCache::~TableCache() { } @@ -55,113 +84,341 @@ void TableCache::ReleaseHandle(Cache::Handle* handle) { cache_->Release(handle); } -Status TableCache::FindTable(const EnvOptions& toptions, +Status TableCache::GetTableReader( + const EnvOptions& env_options, + const InternalKeyComparator& internal_comparator, const FileDescriptor& fd, + bool sequential_mode, size_t readahead, bool record_read_stats, + HistogramImpl* file_read_hist, unique_ptr* table_reader, + bool skip_filters, int level, bool prefetch_index_and_filter_in_cache, + bool for_compaction) { + std::string fname = + TableFileName(ioptions_.db_paths, fd.GetNumber(), fd.GetPathId()); + unique_ptr file; + Status s = ioptions_.env->NewRandomAccessFile(fname, &file, env_options); + + RecordTick(ioptions_.statistics, NO_FILE_OPENS); + if (s.ok()) { + if (readahead > 0) { + file = NewReadaheadRandomAccessFile(std::move(file), readahead); + } + if (!sequential_mode && ioptions_.advise_random_on_open) { + file->Hint(RandomAccessFile::RANDOM); + } + StopWatch sw(ioptions_.env, ioptions_.statistics, TABLE_OPEN_IO_MICROS); + std::unique_ptr file_reader( + new RandomAccessFileReader( + std::move(file), fname, ioptions_.env, + record_read_stats ? ioptions_.statistics : nullptr, SST_READ_MICROS, + file_read_hist, ioptions_.rate_limiter, for_compaction)); + s = ioptions_.table_factory->NewTableReader( + TableReaderOptions(ioptions_, env_options, internal_comparator, + skip_filters, level), + std::move(file_reader), fd.GetFileSize(), table_reader, + prefetch_index_and_filter_in_cache); + TEST_SYNC_POINT("TableCache::GetTableReader:0"); + } + return s; +} + +void TableCache::EraseHandle(const FileDescriptor& fd, Cache::Handle* handle) { + ReleaseHandle(handle); + uint64_t number = fd.GetNumber(); + Slice key = GetSliceForFileNumber(&number); + cache_->Erase(key); +} + +Status TableCache::FindTable(const EnvOptions& env_options, const InternalKeyComparator& internal_comparator, const FileDescriptor& fd, Cache::Handle** handle, - const bool no_io) { + const bool no_io, bool record_read_stats, + HistogramImpl* file_read_hist, bool skip_filters, + int level, + bool prefetch_index_and_filter_in_cache) { + PERF_TIMER_GUARD(find_table_nanos); Status s; uint64_t number = fd.GetNumber(); Slice key = GetSliceForFileNumber(&number); *handle = cache_->Lookup(key); + TEST_SYNC_POINT_CALLBACK("TableCache::FindTable:0", + const_cast(&no_io)); + if (*handle == nullptr) { - if (no_io) { // Dont do IO and return a not-found status + if (no_io) { // Don't do IO and return a not-found status return Status::Incomplete("Table not found in table_cache, no_io is set"); } - std::string fname = - TableFileName(db_paths_, fd.GetNumber(), fd.GetPathId()); - unique_ptr file; unique_ptr table_reader; - s = env_->NewRandomAccessFile(fname, &file, toptions); - RecordTick(options_->statistics.get(), NO_FILE_OPENS); - if (s.ok()) { - if (options_->advise_random_on_open) { - file->Hint(RandomAccessFile::RANDOM); - } - StopWatch sw(env_, options_->statistics.get(), TABLE_OPEN_IO_MICROS); - s = options_->table_factory->NewTableReader( - *options_, toptions, internal_comparator, std::move(file), - fd.GetFileSize(), &table_reader); - } - + s = GetTableReader(env_options, internal_comparator, fd, + false /* sequential mode */, 0 /* readahead */, + record_read_stats, file_read_hist, &table_reader, + skip_filters, level, prefetch_index_and_filter_in_cache); if (!s.ok()) { assert(table_reader == nullptr); - RecordTick(options_->statistics.get(), NO_FILE_ERRORS); + RecordTick(ioptions_.statistics, NO_FILE_ERRORS); // We do not cache error results so that if the error is transient, // or somebody repairs the file, we recover automatically. } else { - assert(file.get() == nullptr); - *handle = cache_->Insert(key, table_reader.release(), 1, &DeleteEntry); + s = cache_->Insert(key, table_reader.get(), 1, &DeleteEntry, + handle); + if (s.ok()) { + // Release ownership of table reader. + table_reader.release(); + } } } return s; } -Iterator* TableCache::NewIterator(const ReadOptions& options, - const EnvOptions& toptions, - const InternalKeyComparator& icomparator, - const FileDescriptor& fd, - TableReader** table_reader_ptr, - bool for_compaction, Arena* arena) { - if (table_reader_ptr != nullptr) { - *table_reader_ptr = nullptr; - } - TableReader* table_reader = fd.table_reader; - Cache::Handle* handle = nullptr; +InternalIterator* TableCache::NewIterator( + const ReadOptions& options, const EnvOptions& env_options, + const InternalKeyComparator& icomparator, const FileDescriptor& fd, + RangeDelAggregator* range_del_agg, TableReader** table_reader_ptr, + HistogramImpl* file_read_hist, bool for_compaction, Arena* arena, + bool skip_filters, int level) { + PERF_TIMER_GUARD(new_table_iterator_nanos); + Status s; - if (table_reader == nullptr) { - s = FindTable(toptions, icomparator, fd, &handle, - options.read_tier == kBlockCacheTier); - if (!s.ok()) { - return NewErrorIterator(s, arena); + bool create_new_table_reader = false; + TableReader* table_reader = nullptr; + Cache::Handle* handle = nullptr; + if (s.ok()) { + if (table_reader_ptr != nullptr) { + *table_reader_ptr = nullptr; + } + size_t readahead = 0; + if (for_compaction) { +#ifndef NDEBUG + bool use_direct_reads_for_compaction = env_options.use_direct_reads; + TEST_SYNC_POINT_CALLBACK("TableCache::NewIterator:for_compaction", + &use_direct_reads_for_compaction); +#endif // !NDEBUG + if (ioptions_.new_table_reader_for_compaction_inputs) { + readahead = ioptions_.compaction_readahead_size; + create_new_table_reader = true; + } + } else { + readahead = options.readahead_size; + create_new_table_reader = readahead > 0; + } + + if (create_new_table_reader) { + unique_ptr table_reader_unique_ptr; + s = GetTableReader( + env_options, icomparator, fd, true /* sequential_mode */, readahead, + !for_compaction /* record stats */, nullptr, &table_reader_unique_ptr, + false /* skip_filters */, level, + true /* prefetch_index_and_filter_in_cache */, for_compaction); + if (s.ok()) { + table_reader = table_reader_unique_ptr.release(); + } + } else { + table_reader = fd.table_reader; + if (table_reader == nullptr) { + s = FindTable(env_options, icomparator, fd, &handle, + options.read_tier == kBlockCacheTier /* no_io */, + !for_compaction /* record read_stats */, file_read_hist, + skip_filters, level); + if (s.ok()) { + table_reader = GetTableReaderFromHandle(handle); + } + } } - table_reader = GetTableReaderFromHandle(handle); } + InternalIterator* result = nullptr; + if (s.ok()) { + result = table_reader->NewIterator(options, arena, skip_filters); + if (create_new_table_reader) { + assert(handle == nullptr); + result->RegisterCleanup(&DeleteTableReader, table_reader, nullptr); + } else if (handle != nullptr) { + result->RegisterCleanup(&UnrefEntry, cache_, handle); + handle = nullptr; // prevent from releasing below + } - Iterator* result = table_reader->NewIterator(options, arena); - if (handle != nullptr) { - result->RegisterCleanup(&UnrefEntry, cache_, handle); + if (for_compaction) { + table_reader->SetupForCompaction(); + } + if (table_reader_ptr != nullptr) { + *table_reader_ptr = table_reader; + } } - if (table_reader_ptr != nullptr) { - *table_reader_ptr = table_reader; + if (s.ok() && range_del_agg != nullptr && !options.ignore_range_deletions) { + std::unique_ptr range_del_iter( + table_reader->NewRangeTombstoneIterator(options)); + if (range_del_iter != nullptr) { + s = range_del_iter->status(); + } + if (s.ok()) { + s = range_del_agg->AddTombstones(std::move(range_del_iter)); + } } - if (for_compaction) { - table_reader->SetupForCompaction(); + if (handle != nullptr) { + ReleaseHandle(handle); } + if (!s.ok()) { + assert(result == nullptr); + result = NewErrorInternalIterator(s, arena); + } + return result; +} +InternalIterator* TableCache::NewRangeTombstoneIterator( + const ReadOptions& options, const EnvOptions& env_options, + const InternalKeyComparator& icomparator, const FileDescriptor& fd, + HistogramImpl* file_read_hist, bool skip_filters, int level) { + Status s; + TableReader* table_reader = nullptr; + Cache::Handle* handle = nullptr; + table_reader = fd.table_reader; + if (table_reader == nullptr) { + s = FindTable(env_options, icomparator, fd, &handle, + options.read_tier == kBlockCacheTier /* no_io */, + true /* record read_stats */, file_read_hist, skip_filters, + level); + if (s.ok()) { + table_reader = GetTableReaderFromHandle(handle); + } + } + InternalIterator* result = nullptr; + if (s.ok()) { + result = table_reader->NewRangeTombstoneIterator(options); + if (result != nullptr) { + if (handle != nullptr) { + result->RegisterCleanup(&UnrefEntry, cache_, handle); + } + } + } + if (result == nullptr && handle != nullptr) { + // the range deletion block didn't exist, or there was a failure between + // getting handle and getting iterator. + ReleaseHandle(handle); + } + if (!s.ok()) { + assert(result == nullptr); + result = NewErrorInternalIterator(s); + } return result; } Status TableCache::Get(const ReadOptions& options, const InternalKeyComparator& internal_comparator, - const FileDescriptor& fd, const Slice& k, void* arg, - bool (*saver)(void*, const ParsedInternalKey&, - const Slice&), - void (*mark_key_may_exist)(void*)) { - TableReader* t = fd.table_reader; + const FileDescriptor& fd, const Slice& k, + GetContext* get_context, HistogramImpl* file_read_hist, + bool skip_filters, int level) { + std::string* row_cache_entry = nullptr; + bool done = false; +#ifndef ROCKSDB_LITE + IterKey row_cache_key; + std::string row_cache_entry_buffer; + + // Check row cache if enabled. Since row cache does not currently store + // sequence numbers, we cannot use it if we need to fetch the sequence. + if (ioptions_.row_cache && !get_context->NeedToReadSequence()) { + uint64_t fd_number = fd.GetNumber(); + auto user_key = ExtractUserKey(k); + // We use the user key as cache key instead of the internal key, + // otherwise the whole cache would be invalidated every time the + // sequence key increases. However, to support caching snapshot + // reads, we append the sequence number (incremented by 1 to + // distinguish from 0) only in this case. + uint64_t seq_no = + options.snapshot == nullptr ? 0 : 1 + GetInternalKeySeqno(k); + + // Compute row cache key. + row_cache_key.TrimAppend(row_cache_key.Size(), row_cache_id_.data(), + row_cache_id_.size()); + AppendVarint64(&row_cache_key, fd_number); + AppendVarint64(&row_cache_key, seq_no); + row_cache_key.TrimAppend(row_cache_key.Size(), user_key.data(), + user_key.size()); + + if (auto row_handle = + ioptions_.row_cache->Lookup(row_cache_key.GetUserKey())) { + // Cleanable routine to release the cache entry + Cleanable value_pinner; + auto release_cache_entry_func = [](void* cache_to_clean, + void* cache_handle) { + ((Cache*)cache_to_clean)->Release((Cache::Handle*)cache_handle); + }; + auto found_row_cache_entry = static_cast( + ioptions_.row_cache->Value(row_handle)); + // If it comes here value is located on the cache. + // found_row_cache_entry points to the value on cache, + // and value_pinner has cleanup procedure for the cached entry. + // After replayGetContextLog() returns, get_context.pinnable_slice_ + // will point to cache entry buffer (or a copy based on that) and + // cleanup routine under value_pinner will be delegated to + // get_context.pinnable_slice_. Cache entry is released when + // get_context.pinnable_slice_ is reset. + value_pinner.RegisterCleanup(release_cache_entry_func, + ioptions_.row_cache.get(), row_handle); + replayGetContextLog(*found_row_cache_entry, user_key, get_context, + &value_pinner); + RecordTick(ioptions_.statistics, ROW_CACHE_HIT); + done = true; + } else { + // Not found, setting up the replay log. + RecordTick(ioptions_.statistics, ROW_CACHE_MISS); + row_cache_entry = &row_cache_entry_buffer; + } + } +#endif // ROCKSDB_LITE Status s; + TableReader* t = fd.table_reader; Cache::Handle* handle = nullptr; - if (!t) { - s = FindTable(storage_options_, internal_comparator, fd, &handle, - options.read_tier == kBlockCacheTier); + if (!done && s.ok()) { + if (t == nullptr) { + s = FindTable(env_options_, internal_comparator, fd, &handle, + options.read_tier == kBlockCacheTier /* no_io */, + true /* record_read_stats */, file_read_hist, skip_filters, + level); + if (s.ok()) { + t = GetTableReaderFromHandle(handle); + } + } + if (s.ok() && get_context->range_del_agg() != nullptr && + !options.ignore_range_deletions) { + std::unique_ptr range_del_iter( + t->NewRangeTombstoneIterator(options)); + if (range_del_iter != nullptr) { + s = range_del_iter->status(); + } + if (s.ok()) { + s = get_context->range_del_agg()->AddTombstones( + std::move(range_del_iter)); + } + } if (s.ok()) { - t = GetTableReaderFromHandle(handle); + get_context->SetReplayLog(row_cache_entry); // nullptr if no cache. + s = t->Get(options, k, get_context, skip_filters); + get_context->SetReplayLog(nullptr); + } else if (options.read_tier == kBlockCacheTier && s.IsIncomplete()) { + // Couldn't find Table in cache but treat as kFound if no_io set + get_context->MarkKeyMayExist(); + s = Status::OK(); + done = true; } } - if (s.ok()) { - s = t->Get(options, k, arg, saver, mark_key_may_exist); - if (handle != nullptr) { - ReleaseHandle(handle); - } - } else if (options.read_tier && s.IsIncomplete()) { - // Couldnt find Table in cache but treat as kFound if no_io set - (*mark_key_may_exist)(arg); - return Status::OK(); + +#ifndef ROCKSDB_LITE + // Put the replay log in row cache only if something was found. + if (!done && s.ok() && row_cache_entry && !row_cache_entry->empty()) { + size_t charge = + row_cache_key.Size() + row_cache_entry->size() + sizeof(std::string); + void* row_ptr = new std::string(std::move(*row_cache_entry)); + ioptions_.row_cache->Insert(row_cache_key.GetUserKey(), row_ptr, charge, + &DeleteEntry); + } +#endif // ROCKSDB_LITE + + if (handle != nullptr) { + ReleaseHandle(handle); } return s; } + Status TableCache::GetTableProperties( - const EnvOptions& toptions, + const EnvOptions& env_options, const InternalKeyComparator& internal_comparator, const FileDescriptor& fd, std::shared_ptr* properties, bool no_io) { Status s; @@ -174,7 +431,7 @@ Status TableCache::GetTableProperties( } Cache::Handle* table_handle = nullptr; - s = FindTable(toptions, internal_comparator, fd, &table_handle, no_io); + s = FindTable(env_options, internal_comparator, fd, &table_handle, no_io); if (!s.ok()) { return s; } @@ -186,7 +443,7 @@ Status TableCache::GetTableProperties( } size_t TableCache::GetMemoryUsageByTableReader( - const EnvOptions& toptions, + const EnvOptions& env_options, const InternalKeyComparator& internal_comparator, const FileDescriptor& fd) { Status s; @@ -197,7 +454,7 @@ size_t TableCache::GetMemoryUsageByTableReader( } Cache::Handle* table_handle = nullptr; - s = FindTable(toptions, internal_comparator, fd, &table_handle, true); + s = FindTable(env_options, internal_comparator, fd, &table_handle, true); if (!s.ok()) { return 0; } diff --git a/db/table_cache.h b/db/table_cache.h index 79090e0649b..8b65bafa3ef 100644 --- a/db/table_cache.h +++ b/db/table_cache.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -15,9 +15,12 @@ #include #include "db/dbformat.h" +#include "db/range_del_aggregator.h" +#include "options/cf_options.h" #include "port/port.h" #include "rocksdb/cache.h" #include "rocksdb/env.h" +#include "rocksdb/options.h" #include "rocksdb/table.h" #include "table/table_reader.h" @@ -26,11 +29,14 @@ namespace rocksdb { class Env; class Arena; struct FileDescriptor; +class GetContext; +class HistogramImpl; +class InternalIterator; class TableCache { public: - TableCache(const Options* options, const EnvOptions& storage_options, - Cache* cache); + TableCache(const ImmutableCFOptions& ioptions, + const EnvOptions& storage_options, Cache* cache); ~TableCache(); // Return an iterator for the specified file number (the corresponding @@ -40,30 +46,55 @@ class TableCache { // the returned iterator. The returned "*tableptr" object is owned by // the cache and should not be deleted, and is valid for as long as the // returned iterator is live. - Iterator* NewIterator(const ReadOptions& options, const EnvOptions& toptions, - const InternalKeyComparator& internal_comparator, - const FileDescriptor& file_fd, - TableReader** table_reader_ptr = nullptr, - bool for_compaction = false, Arena* arena = nullptr); + // @param range_del_agg If non-nullptr, adds range deletions to the + // aggregator. If an error occurs, returns it in a NewErrorInternalIterator + // @param skip_filters Disables loading/accessing the filter block + // @param level The level this table is at, -1 for "not set / don't know" + InternalIterator* NewIterator( + const ReadOptions& options, const EnvOptions& toptions, + const InternalKeyComparator& internal_comparator, + const FileDescriptor& file_fd, RangeDelAggregator* range_del_agg, + TableReader** table_reader_ptr = nullptr, + HistogramImpl* file_read_hist = nullptr, bool for_compaction = false, + Arena* arena = nullptr, bool skip_filters = false, int level = -1); + + InternalIterator* NewRangeTombstoneIterator( + const ReadOptions& options, const EnvOptions& toptions, + const InternalKeyComparator& internal_comparator, + const FileDescriptor& file_fd, HistogramImpl* file_read_hist, + bool skip_filters, int level); // If a seek to internal key "k" in specified file finds an entry, // call (*handle_result)(arg, found_key, found_value) repeatedly until // it returns false. + // @param get_context State for get operation. If its range_del_agg() returns + // non-nullptr, adds range deletions to the aggregator. If an error occurs, + // returns non-ok status. + // @param skip_filters Disables loading/accessing the filter block + // @param level The level this table is at, -1 for "not set / don't know" Status Get(const ReadOptions& options, const InternalKeyComparator& internal_comparator, - const FileDescriptor& file_fd, const Slice& k, void* arg, - bool (*handle_result)(void*, const ParsedInternalKey&, - const Slice&), - void (*mark_key_may_exist)(void*) = nullptr); + const FileDescriptor& file_fd, const Slice& k, + GetContext* get_context, HistogramImpl* file_read_hist = nullptr, + bool skip_filters = false, int level = -1); // Evict any entry for the specified file number static void Evict(Cache* cache, uint64_t file_number); + // Clean table handle and erase it from the table cache + // Used in DB close, or the file is not live anymore. + void EraseHandle(const FileDescriptor& fd, Cache::Handle* handle); + // Find table reader + // @param skip_filters Disables loading/accessing the filter block + // @param level == -1 means not specified Status FindTable(const EnvOptions& toptions, const InternalKeyComparator& internal_comparator, const FileDescriptor& file_fd, Cache::Handle**, - const bool no_io = false); + const bool no_io = false, bool record_read_stats = true, + HistogramImpl* file_read_hist = nullptr, + bool skip_filters = false, int level = -1, + bool prefetch_index_and_filter_in_cache = true); // Get TableReader from a cache handle. TableReader* GetTableReaderFromHandle(Cache::Handle* handle); @@ -81,7 +112,7 @@ class TableCache { bool no_io = false); // Return total memory usage of the table reader of the file. - // 0 of table reader of the file is not loaded. + // 0 if table reader of the file is not loaded. size_t GetMemoryUsageByTableReader( const EnvOptions& toptions, const InternalKeyComparator& internal_comparator, @@ -90,12 +121,26 @@ class TableCache { // Release the handle from a cache void ReleaseHandle(Cache::Handle* handle); + // Capacity of the backing Cache that indicates inifinite TableCache capacity. + // For example when max_open_files is -1 we set the backing Cache to this. + static const int kInfiniteCapacity = 0x400000; + private: - Env* const env_; - const std::vector db_paths_; - const Options* options_; - const EnvOptions& storage_options_; + // Build a table reader + Status GetTableReader(const EnvOptions& env_options, + const InternalKeyComparator& internal_comparator, + const FileDescriptor& fd, bool sequential_mode, + size_t readahead, bool record_read_stats, + HistogramImpl* file_read_hist, + unique_ptr* table_reader, + bool skip_filters = false, int level = -1, + bool prefetch_index_and_filter_in_cache = true, + bool for_compaction = false); + + const ImmutableCFOptions& ioptions_; + const EnvOptions& env_options_; Cache* const cache_; + std::string row_cache_id_; }; } // namespace rocksdb diff --git a/db/table_properties_collector.cc b/db/table_properties_collector.cc index 25bd7003627..a1f4dba97bb 100644 --- a/db/table_properties_collector.cc +++ b/db/table_properties_collector.cc @@ -1,24 +1,30 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #include "db/table_properties_collector.h" #include "db/dbformat.h" #include "util/coding.h" +#include "util/string_util.h" namespace rocksdb { -Status InternalKeyPropertiesCollector::Add( - const Slice& key, const Slice& value) { +Status InternalKeyPropertiesCollector::InternalAdd(const Slice& key, + const Slice& value, + uint64_t file_size) { ParsedInternalKey ikey; if (!ParseInternalKey(key, &ikey)) { return Status::InvalidArgument("Invalid internal key"); } - if (ikey.type == ValueType::kTypeDeletion) { + // Note: We count both, deletions and single deletions here. + if (ikey.type == ValueType::kTypeDeletion || + ikey.type == ValueType::kTypeSingleDeletion) { ++deleted_keys_; + } else if (ikey.type == ValueType::kTypeMerge) { + ++merge_operands_; } return Status::OK(); @@ -29,30 +35,71 @@ Status InternalKeyPropertiesCollector::Finish( assert(properties); assert(properties->find( InternalKeyTablePropertiesNames::kDeletedKeys) == properties->end()); - std::string val; + assert(properties->find(InternalKeyTablePropertiesNames::kMergeOperands) == + properties->end()); - PutVarint64(&val, deleted_keys_); - properties->insert({ InternalKeyTablePropertiesNames::kDeletedKeys, val }); + std::string val_deleted_keys; + PutVarint64(&val_deleted_keys, deleted_keys_); + properties->insert( + {InternalKeyTablePropertiesNames::kDeletedKeys, val_deleted_keys}); + + std::string val_merge_operands; + PutVarint64(&val_merge_operands, merge_operands_); + properties->insert( + {InternalKeyTablePropertiesNames::kMergeOperands, val_merge_operands}); return Status::OK(); } UserCollectedProperties InternalKeyPropertiesCollector::GetReadableProperties() const { - return { - { "kDeletedKeys", std::to_string(deleted_keys_) } - }; + return {{"kDeletedKeys", ToString(deleted_keys_)}, + {"kMergeOperands", ToString(merge_operands_)}}; +} + +namespace { + +EntryType GetEntryType(ValueType value_type) { + switch (value_type) { + case kTypeValue: + return kEntryPut; + case kTypeDeletion: + return kEntryDelete; + case kTypeSingleDeletion: + return kEntrySingleDelete; + case kTypeMerge: + return kEntryMerge; + default: + return kEntryOther; + } +} + +uint64_t GetUint64Property(const UserCollectedProperties& props, + const std::string property_name, + bool* property_present) { + auto pos = props.find(property_name); + if (pos == props.end()) { + *property_present = false; + return 0; + } + Slice raw = pos->second; + uint64_t val = 0; + *property_present = true; + return GetVarint64(&raw, &val) ? val : 0; } +} // namespace -Status UserKeyTablePropertiesCollector::Add( - const Slice& key, const Slice& value) { +Status UserKeyTablePropertiesCollector::InternalAdd(const Slice& key, + const Slice& value, + uint64_t file_size) { ParsedInternalKey ikey; if (!ParseInternalKey(key, &ikey)) { return Status::InvalidArgument("Invalid internal key"); } - return collector_->Add(ikey.user_key, value); + return collector_->AddUserKey(ikey.user_key, value, GetEntryType(ikey.type), + ikey.sequence, file_size); } Status UserKeyTablePropertiesCollector::Finish( @@ -68,16 +115,20 @@ UserKeyTablePropertiesCollector::GetReadableProperties() const { const std::string InternalKeyTablePropertiesNames::kDeletedKeys = "rocksdb.deleted.keys"; +const std::string InternalKeyTablePropertiesNames::kMergeOperands = + "rocksdb.merge.operands"; uint64_t GetDeletedKeys( const UserCollectedProperties& props) { - auto pos = props.find(InternalKeyTablePropertiesNames::kDeletedKeys); - if (pos == props.end()) { - return 0; - } - Slice raw = pos->second; - uint64_t val = 0; - return GetVarint64(&raw, &val) ? val : 0; + bool property_present_ignored; + return GetUint64Property(props, InternalKeyTablePropertiesNames::kDeletedKeys, + &property_present_ignored); +} + +uint64_t GetMergeOperands(const UserCollectedProperties& props, + bool* property_present) { + return GetUint64Property( + props, InternalKeyTablePropertiesNames::kMergeOperands, property_present); } } // namespace rocksdb diff --git a/db/table_properties_collector.h b/db/table_properties_collector.h index aafcb520218..d8cd75689d5 100644 --- a/db/table_properties_collector.h +++ b/db/table_properties_collector.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file defines a collection of statistics collectors. #pragma once @@ -16,13 +16,45 @@ namespace rocksdb { struct InternalKeyTablePropertiesNames { static const std::string kDeletedKeys; + static const std::string kMergeOperands; +}; + +// Base class for internal table properties collector. +class IntTblPropCollector { + public: + virtual ~IntTblPropCollector() {} + virtual Status Finish(UserCollectedProperties* properties) = 0; + + virtual const char* Name() const = 0; + + // @params key the user key that is inserted into the table. + // @params value the value that is inserted into the table. + virtual Status InternalAdd(const Slice& key, const Slice& value, + uint64_t file_size) = 0; + + virtual UserCollectedProperties GetReadableProperties() const = 0; + + virtual bool NeedCompact() const { return false; } +}; + +// Factory for internal table properties collector. +class IntTblPropCollectorFactory { + public: + virtual ~IntTblPropCollectorFactory() {} + // has to be thread-safe + virtual IntTblPropCollector* CreateIntTblPropCollector( + uint32_t column_family_id) = 0; + + // The name of the properties collector can be used for debugging purpose. + virtual const char* Name() const = 0; }; // Collecting the statistics for internal keys. Visible only by internal // rocksdb modules. -class InternalKeyPropertiesCollector : public TablePropertiesCollector { +class InternalKeyPropertiesCollector : public IntTblPropCollector { public: - virtual Status Add(const Slice& key, const Slice& value) override; + virtual Status InternalAdd(const Slice& key, const Slice& value, + uint64_t file_size) override; virtual Status Finish(UserCollectedProperties* properties) override; @@ -34,12 +66,14 @@ class InternalKeyPropertiesCollector : public TablePropertiesCollector { private: uint64_t deleted_keys_ = 0; + uint64_t merge_operands_ = 0; }; class InternalKeyPropertiesCollectorFactory - : public TablePropertiesCollectorFactory { + : public IntTblPropCollectorFactory { public: - virtual TablePropertiesCollector* CreateTablePropertiesCollector() { + virtual IntTblPropCollector* CreateIntTblPropCollector( + uint32_t column_family_id) override { return new InternalKeyPropertiesCollector(); } @@ -53,7 +87,7 @@ class InternalKeyPropertiesCollectorFactory // // This class extracts user key from the encoded internal key when Add() is // invoked. -class UserKeyTablePropertiesCollector : public TablePropertiesCollector { +class UserKeyTablePropertiesCollector : public IntTblPropCollector { public: // transfer of ownership explicit UserKeyTablePropertiesCollector(TablePropertiesCollector* collector) @@ -61,7 +95,8 @@ class UserKeyTablePropertiesCollector : public TablePropertiesCollector { virtual ~UserKeyTablePropertiesCollector() {} - virtual Status Add(const Slice& key, const Slice& value) override; + virtual Status InternalAdd(const Slice& key, const Slice& value, + uint64_t file_size) override; virtual Status Finish(UserCollectedProperties* properties) override; @@ -69,19 +104,26 @@ class UserKeyTablePropertiesCollector : public TablePropertiesCollector { UserCollectedProperties GetReadableProperties() const override; + virtual bool NeedCompact() const override { + return collector_->NeedCompact(); + } + protected: std::unique_ptr collector_; }; class UserKeyTablePropertiesCollectorFactory - : public TablePropertiesCollectorFactory { + : public IntTblPropCollectorFactory { public: explicit UserKeyTablePropertiesCollectorFactory( std::shared_ptr user_collector_factory) : user_collector_factory_(user_collector_factory) {} - virtual TablePropertiesCollector* CreateTablePropertiesCollector() { + virtual IntTblPropCollector* CreateIntTblPropCollector( + uint32_t column_family_id) override { + TablePropertiesCollectorFactory::Context context; + context.column_family_id = column_family_id; return new UserKeyTablePropertiesCollector( - user_collector_factory_->CreateTablePropertiesCollector()); + user_collector_factory_->CreateTablePropertiesCollector(context)); } virtual const char* Name() const override { diff --git a/db/table_properties_collector_test.cc b/db/table_properties_collector_test.cc index 638b259f2a5..66c66c02531 100644 --- a/db/table_properties_collector_test.cc +++ b/db/table_properties_collector_test.cc @@ -1,123 +1,176 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #include #include #include +#include +#include #include "db/db_impl.h" #include "db/dbformat.h" #include "db/table_properties_collector.h" +#include "options/cf_options.h" #include "rocksdb/table.h" #include "table/block_based_table_factory.h" #include "table/meta_blocks.h" #include "table/plain_table_factory.h" #include "table/table_builder.h" #include "util/coding.h" +#include "util/file_reader_writer.h" #include "util/testharness.h" #include "util/testutil.h" namespace rocksdb { -class TablePropertiesTest { +class TablePropertiesTest : public testing::Test, + public testing::WithParamInterface { + public: + virtual void SetUp() override { backward_mode_ = GetParam(); } + + bool backward_mode_; }; -// TODO(kailiu) the following classes should be moved to some more general -// places, so that other tests can also make use of them. -// `FakeWritableFile` and `FakeRandomeAccessFile` bypass the real file system -// and therefore enable us to quickly setup the tests. -class FakeWritableFile : public WritableFile { +// Utilities test functions +namespace { +static const uint32_t kTestColumnFamilyId = 66; +static const std::string kTestColumnFamilyName = "test_column_fam"; + +void MakeBuilder(const Options& options, const ImmutableCFOptions& ioptions, + const InternalKeyComparator& internal_comparator, + const std::vector>* + int_tbl_prop_collector_factories, + std::unique_ptr* writable, + std::unique_ptr* builder) { + unique_ptr wf(new test::StringSink); + writable->reset(new WritableFileWriter(std::move(wf), EnvOptions())); + int unknown_level = -1; + builder->reset(NewTableBuilder( + ioptions, internal_comparator, int_tbl_prop_collector_factories, + kTestColumnFamilyId, kTestColumnFamilyName, + writable->get(), options.compression, options.compression_opts, + unknown_level)); +} +} // namespace + +// Collects keys that starts with "A" in a table. +class RegularKeysStartWithA: public TablePropertiesCollector { public: - ~FakeWritableFile() { } + const char* Name() const override { return "RegularKeysStartWithA"; } - const std::string& contents() const { return contents_; } + Status Finish(UserCollectedProperties* properties) override { + std::string encoded; + std::string encoded_num_puts; + std::string encoded_num_deletes; + std::string encoded_num_single_deletes; + std::string encoded_num_size_changes; + PutVarint32(&encoded, count_); + PutVarint32(&encoded_num_puts, num_puts_); + PutVarint32(&encoded_num_deletes, num_deletes_); + PutVarint32(&encoded_num_single_deletes, num_single_deletes_); + PutVarint32(&encoded_num_size_changes, num_size_changes_); + *properties = UserCollectedProperties{ + {"TablePropertiesTest", message_}, + {"Count", encoded}, + {"NumPuts", encoded_num_puts}, + {"NumDeletes", encoded_num_deletes}, + {"NumSingleDeletes", encoded_num_single_deletes}, + {"NumSizeChanges", encoded_num_size_changes}, + }; + return Status::OK(); + } - virtual Status Close() { return Status::OK(); } - virtual Status Flush() { return Status::OK(); } - virtual Status Sync() { return Status::OK(); } + Status AddUserKey(const Slice& user_key, const Slice& value, EntryType type, + SequenceNumber seq, uint64_t file_size) override { + // simply asssume all user keys are not empty. + if (user_key.data()[0] == 'A') { + ++count_; + } + if (type == kEntryPut) { + num_puts_++; + } else if (type == kEntryDelete) { + num_deletes_++; + } else if (type == kEntrySingleDelete) { + num_single_deletes_++; + } + if (file_size < file_size_) { + message_ = "File size should not decrease."; + } else if (file_size != file_size_) { + num_size_changes_++; + } - virtual Status Append(const Slice& data) { - contents_.append(data.data(), data.size()); return Status::OK(); } + virtual UserCollectedProperties GetReadableProperties() const override { + return UserCollectedProperties{}; + } + private: - std::string contents_; + std::string message_ = "Rocksdb"; + uint32_t count_ = 0; + uint32_t num_puts_ = 0; + uint32_t num_deletes_ = 0; + uint32_t num_single_deletes_ = 0; + uint32_t num_size_changes_ = 0; + uint64_t file_size_ = 0; }; - -class FakeRandomeAccessFile : public RandomAccessFile { +// Collects keys that starts with "A" in a table. Backward compatible mode +// It is also used to test internal key table property collector +class RegularKeysStartWithABackwardCompatible + : public TablePropertiesCollector { public: - explicit FakeRandomeAccessFile(const Slice& contents) - : contents_(contents.data(), contents.size()) { - } + const char* Name() const override { return "RegularKeysStartWithA"; } - virtual ~FakeRandomeAccessFile() { } - - uint64_t Size() const { return contents_.size(); } + Status Finish(UserCollectedProperties* properties) override { + std::string encoded; + PutVarint32(&encoded, count_); + *properties = UserCollectedProperties{{"TablePropertiesTest", "Rocksdb"}, + {"Count", encoded}}; + return Status::OK(); + } - virtual Status Read(uint64_t offset, size_t n, Slice* result, - char* scratch) const { - if (offset > contents_.size()) { - return Status::InvalidArgument("invalid Read offset"); - } - if (offset + n > contents_.size()) { - n = contents_.size() - offset; + Status Add(const Slice& user_key, const Slice& value) override { + // simply asssume all user keys are not empty. + if (user_key.data()[0] == 'A') { + ++count_; } - memcpy(scratch, &contents_[offset], n); - *result = Slice(scratch, n); return Status::OK(); } + virtual UserCollectedProperties GetReadableProperties() const override { + return UserCollectedProperties{}; + } + private: - std::string contents_; + uint32_t count_ = 0; }; - -class DumbLogger : public Logger { +class RegularKeysStartWithAInternal : public IntTblPropCollector { public: - virtual void Logv(const char* format, va_list ap) { } - virtual size_t GetLogFileSize() const { return 0; } -}; + const char* Name() const override { return "RegularKeysStartWithA"; } -// Utilities test functions -namespace { -void MakeBuilder(const Options& options, - const InternalKeyComparator& internal_comparator, - std::unique_ptr* writable, - std::unique_ptr* builder) { - writable->reset(new FakeWritableFile); - builder->reset(options.table_factory->NewTableBuilder( - options, internal_comparator, writable->get(), options.compression)); -} -} // namespace - -// Collects keys that starts with "A" in a table. -class RegularKeysStartWithA: public TablePropertiesCollector { - public: - const char* Name() const { return "RegularKeysStartWithA"; } - - Status Finish(UserCollectedProperties* properties) { - std::string encoded; - PutVarint32(&encoded, count_); - *properties = UserCollectedProperties { - { "TablePropertiesTest", "Rocksdb" }, - { "Count", encoded } - }; - return Status::OK(); - } + Status Finish(UserCollectedProperties* properties) override { + std::string encoded; + PutVarint32(&encoded, count_); + *properties = UserCollectedProperties{{"TablePropertiesTest", "Rocksdb"}, + {"Count", encoded}}; + return Status::OK(); + } - Status Add(const Slice& user_key, const Slice& value) { - // simply asssume all user keys are not empty. - if (user_key.data()[0] == 'A') { - ++count_; - } - return Status::OK(); - } + Status InternalAdd(const Slice& user_key, const Slice& value, + uint64_t file_size) override { + // simply asssume all user keys are not empty. + if (user_key.data()[0] == 'A') { + ++count_; + } + return Status::OK(); + } - virtual UserCollectedProperties GetReadableProperties() const { + virtual UserCollectedProperties GetReadableProperties() const override { return UserCollectedProperties{}; } @@ -125,200 +178,322 @@ class RegularKeysStartWithA: public TablePropertiesCollector { uint32_t count_ = 0; }; -class RegularKeysStartWithAFactory : public TablePropertiesCollectorFactory { +class RegularKeysStartWithAFactory : public IntTblPropCollectorFactory, + public TablePropertiesCollectorFactory { + public: + explicit RegularKeysStartWithAFactory(bool backward_mode) + : backward_mode_(backward_mode) {} + virtual TablePropertiesCollector* CreateTablePropertiesCollector( + TablePropertiesCollectorFactory::Context context) override { + EXPECT_EQ(kTestColumnFamilyId, context.column_family_id); + if (!backward_mode_) { + return new RegularKeysStartWithA(); + } else { + return new RegularKeysStartWithABackwardCompatible(); + } + } + virtual IntTblPropCollector* CreateIntTblPropCollector( + uint32_t column_family_id) override { + return new RegularKeysStartWithAInternal(); + } + const char* Name() const override { return "RegularKeysStartWithA"; } + + bool backward_mode_; +}; + +class FlushBlockEveryThreePolicy : public FlushBlockPolicy { public: - virtual TablePropertiesCollector* CreateTablePropertiesCollector() { - return new RegularKeysStartWithA(); + virtual bool Update(const Slice& key, const Slice& value) override { + return (++count_ % 3U == 0); } - const char* Name() const { return "RegularKeysStartWithA"; } + + private: + uint64_t count_ = 0; }; -extern uint64_t kBlockBasedTableMagicNumber; -extern uint64_t kPlainTableMagicNumber; +class FlushBlockEveryThreePolicyFactory : public FlushBlockPolicyFactory { + public: + explicit FlushBlockEveryThreePolicyFactory() {} + + const char* Name() const override { + return "FlushBlockEveryThreePolicyFactory"; + } + + FlushBlockPolicy* NewFlushBlockPolicy( + const BlockBasedTableOptions& table_options, + const BlockBuilder& data_block_builder) const override { + return new FlushBlockEveryThreePolicy; + } +}; + +extern const uint64_t kBlockBasedTableMagicNumber; +extern const uint64_t kPlainTableMagicNumber; namespace { void TestCustomizedTablePropertiesCollector( - uint64_t magic_number, bool encode_as_internal, const Options& options, - const InternalKeyComparator& internal_comparator) { + bool backward_mode, uint64_t magic_number, bool test_int_tbl_prop_collector, + const Options& options, const InternalKeyComparator& internal_comparator) { // make sure the entries will be inserted with order. - std::map kvs = { - {"About ", "val5"}, // starts with 'A' - {"Abstract", "val2"}, // starts with 'A' - {"Around ", "val7"}, // starts with 'A' - {"Beyond ", "val3"}, - {"Builder ", "val1"}, - {"Cancel ", "val4"}, - {"Find ", "val6"}, + std::map, std::string> kvs = { + {{"About ", kTypeValue}, "val5"}, // starts with 'A' + {{"Abstract", kTypeValue}, "val2"}, // starts with 'A' + {{"Around ", kTypeValue}, "val7"}, // starts with 'A' + {{"Beyond ", kTypeValue}, "val3"}, + {{"Builder ", kTypeValue}, "val1"}, + {{"Love ", kTypeDeletion}, ""}, + {{"Cancel ", kTypeValue}, "val4"}, + {{"Find ", kTypeValue}, "val6"}, + {{"Rocks ", kTypeDeletion}, ""}, + {{"Foo ", kTypeSingleDeletion}, ""}, }; // -- Step 1: build table std::unique_ptr builder; - std::unique_ptr writable; - MakeBuilder(options, internal_comparator, &writable, &builder); + std::unique_ptr writer; + const ImmutableCFOptions ioptions(options); + std::vector> + int_tbl_prop_collector_factories; + if (test_int_tbl_prop_collector) { + int_tbl_prop_collector_factories.emplace_back( + new RegularKeysStartWithAFactory(backward_mode)); + } else { + GetIntTblPropCollectorFactory(ioptions, &int_tbl_prop_collector_factories); + } + MakeBuilder(options, ioptions, internal_comparator, + &int_tbl_prop_collector_factories, &writer, &builder); + SequenceNumber seqNum = 0U; for (const auto& kv : kvs) { - if (encode_as_internal) { - InternalKey ikey(kv.first, 0, ValueType::kTypeValue); - builder->Add(ikey.Encode(), kv.second); - } else { - builder->Add(kv.first, kv.second); - } + InternalKey ikey(kv.first.first, seqNum++, kv.first.second); + builder->Add(ikey.Encode(), kv.second); } ASSERT_OK(builder->Finish()); + writer->Flush(); // -- Step 2: Read properties - FakeRandomeAccessFile readable(writable->contents()); + test::StringSink* fwf = + static_cast(writer->writable_file()); + std::unique_ptr fake_file_reader( + test::GetRandomAccessFileReader( + new test::StringSource(fwf->contents()))); TableProperties* props; - Status s = ReadTableProperties( - &readable, - writable->contents().size(), - magic_number, - Env::Default(), - nullptr, - &props - ); + Status s = ReadTableProperties(fake_file_reader.get(), fwf->contents().size(), + magic_number, ioptions, &props); std::unique_ptr props_guard(props); ASSERT_OK(s); auto user_collected = props->user_collected_properties; + ASSERT_NE(user_collected.find("TablePropertiesTest"), user_collected.end()); ASSERT_EQ("Rocksdb", user_collected.at("TablePropertiesTest")); uint32_t starts_with_A = 0; + ASSERT_NE(user_collected.find("Count"), user_collected.end()); Slice key(user_collected.at("Count")); ASSERT_TRUE(GetVarint32(&key, &starts_with_A)); ASSERT_EQ(3u, starts_with_A); + + if (!backward_mode && !test_int_tbl_prop_collector) { + uint32_t num_puts; + ASSERT_NE(user_collected.find("NumPuts"), user_collected.end()); + Slice key_puts(user_collected.at("NumPuts")); + ASSERT_TRUE(GetVarint32(&key_puts, &num_puts)); + ASSERT_EQ(7u, num_puts); + + uint32_t num_deletes; + ASSERT_NE(user_collected.find("NumDeletes"), user_collected.end()); + Slice key_deletes(user_collected.at("NumDeletes")); + ASSERT_TRUE(GetVarint32(&key_deletes, &num_deletes)); + ASSERT_EQ(2u, num_deletes); + + uint32_t num_single_deletes; + ASSERT_NE(user_collected.find("NumSingleDeletes"), user_collected.end()); + Slice key_single_deletes(user_collected.at("NumSingleDeletes")); + ASSERT_TRUE(GetVarint32(&key_single_deletes, &num_single_deletes)); + ASSERT_EQ(1u, num_single_deletes); + + uint32_t num_size_changes; + ASSERT_NE(user_collected.find("NumSizeChanges"), user_collected.end()); + Slice key_size_changes(user_collected.at("NumSizeChanges")); + ASSERT_TRUE(GetVarint32(&key_size_changes, &num_size_changes)); + ASSERT_GE(num_size_changes, 2u); + } } } // namespace -TEST(TablePropertiesTest, CustomizedTablePropertiesCollector) { +TEST_P(TablePropertiesTest, CustomizedTablePropertiesCollector) { // Test properties collectors with internal keys or regular keys // for block based table for (bool encode_as_internal : { true, false }) { Options options; - std::shared_ptr collector_factory( - new RegularKeysStartWithAFactory()); - if (encode_as_internal) { - options.table_properties_collector_factories.emplace_back( - new UserKeyTablePropertiesCollectorFactory(collector_factory)); - } else { - options.table_properties_collector_factories.resize(1); - options.table_properties_collector_factories[0] = collector_factory; - } - test::PlainInternalKeyComparator ikc(options.comparator); - TestCustomizedTablePropertiesCollector(kBlockBasedTableMagicNumber, - encode_as_internal, options, ikc); - } + BlockBasedTableOptions table_options; + table_options.flush_block_policy_factory = + std::make_shared(); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - // test plain table - Options options; - options.table_properties_collector_factories.emplace_back( - new RegularKeysStartWithAFactory()); + test::PlainInternalKeyComparator ikc(options.comparator); + std::shared_ptr collector_factory( + new RegularKeysStartWithAFactory(backward_mode_)); + options.table_properties_collector_factories.resize(1); + options.table_properties_collector_factories[0] = collector_factory; - PlainTableOptions plain_table_options; - plain_table_options.user_key_len = 8; - plain_table_options.bloom_bits_per_key = 8; - plain_table_options.hash_table_ratio = 0; + TestCustomizedTablePropertiesCollector(backward_mode_, + kBlockBasedTableMagicNumber, + encode_as_internal, options, ikc); - options.table_factory = - std::make_shared(plain_table_options); - test::PlainInternalKeyComparator ikc(options.comparator); - TestCustomizedTablePropertiesCollector(kPlainTableMagicNumber, true, options, - ikc); +#ifndef ROCKSDB_LITE // PlainTable is not supported in Lite + // test plain table + PlainTableOptions plain_table_options; + plain_table_options.user_key_len = 8; + plain_table_options.bloom_bits_per_key = 8; + plain_table_options.hash_table_ratio = 0; + + options.table_factory = + std::make_shared(plain_table_options); + TestCustomizedTablePropertiesCollector(backward_mode_, + kPlainTableMagicNumber, + encode_as_internal, options, ikc); +#endif // !ROCKSDB_LITE + } } namespace { void TestInternalKeyPropertiesCollector( - uint64_t magic_number, - bool sanitized, + bool backward_mode, uint64_t magic_number, bool sanitized, std::shared_ptr table_factory) { InternalKey keys[] = { - InternalKey("A ", 0, ValueType::kTypeValue), - InternalKey("B ", 0, ValueType::kTypeValue), - InternalKey("C ", 0, ValueType::kTypeValue), - InternalKey("W ", 0, ValueType::kTypeDeletion), - InternalKey("X ", 0, ValueType::kTypeDeletion), - InternalKey("Y ", 0, ValueType::kTypeDeletion), - InternalKey("Z ", 0, ValueType::kTypeDeletion), + InternalKey("A ", 0, ValueType::kTypeValue), + InternalKey("B ", 1, ValueType::kTypeValue), + InternalKey("C ", 2, ValueType::kTypeValue), + InternalKey("W ", 3, ValueType::kTypeDeletion), + InternalKey("X ", 4, ValueType::kTypeDeletion), + InternalKey("Y ", 5, ValueType::kTypeDeletion), + InternalKey("Z ", 6, ValueType::kTypeDeletion), + InternalKey("a ", 7, ValueType::kTypeSingleDeletion), + InternalKey("b ", 8, ValueType::kTypeMerge), + InternalKey("c ", 9, ValueType::kTypeMerge), }; std::unique_ptr builder; - std::unique_ptr writable; + std::unique_ptr writable; Options options; test::PlainInternalKeyComparator pikc(options.comparator); + std::vector> + int_tbl_prop_collector_factories; options.table_factory = table_factory; if (sanitized) { options.table_properties_collector_factories.emplace_back( - new RegularKeysStartWithAFactory()); + new RegularKeysStartWithAFactory(backward_mode)); // with sanitization, even regular properties collector will be able to // handle internal keys. auto comparator = options.comparator; // HACK: Set options.info_log to avoid writing log in // SanitizeOptions(). - options.info_log = std::make_shared(); + options.info_log = std::make_shared(); options = SanitizeOptions("db", // just a place holder - &pikc, options); + ImmutableCFOptions ioptions(options); + GetIntTblPropCollectorFactory(ioptions, &int_tbl_prop_collector_factories); options.comparator = comparator; } else { - options.table_properties_collector_factories = { - std::make_shared()}; + int_tbl_prop_collector_factories.emplace_back( + new InternalKeyPropertiesCollectorFactory); } + const ImmutableCFOptions ioptions(options); for (int iter = 0; iter < 2; ++iter) { - MakeBuilder(options, pikc, &writable, &builder); + MakeBuilder(options, ioptions, pikc, &int_tbl_prop_collector_factories, + &writable, &builder); for (const auto& k : keys) { builder->Add(k.Encode(), "val"); } ASSERT_OK(builder->Finish()); + writable->Flush(); - FakeRandomeAccessFile readable(writable->contents()); + test::StringSink* fwf = + static_cast(writable->writable_file()); + unique_ptr reader(test::GetRandomAccessFileReader( + new test::StringSource(fwf->contents()))); TableProperties* props; Status s = - ReadTableProperties(&readable, writable->contents().size(), - magic_number, Env::Default(), nullptr, &props); + ReadTableProperties(reader.get(), fwf->contents().size(), magic_number, + ioptions, &props); ASSERT_OK(s); std::unique_ptr props_guard(props); auto user_collected = props->user_collected_properties; uint64_t deleted = GetDeletedKeys(user_collected); - ASSERT_EQ(4u, deleted); + ASSERT_EQ(5u, deleted); // deletes + single-deletes + + bool property_present; + uint64_t merges = GetMergeOperands(user_collected, &property_present); + ASSERT_TRUE(property_present); + ASSERT_EQ(2u, merges); if (sanitized) { uint32_t starts_with_A = 0; + ASSERT_NE(user_collected.find("Count"), user_collected.end()); Slice key(user_collected.at("Count")); ASSERT_TRUE(GetVarint32(&key, &starts_with_A)); ASSERT_EQ(1u, starts_with_A); + + if (!backward_mode) { + uint32_t num_puts; + ASSERT_NE(user_collected.find("NumPuts"), user_collected.end()); + Slice key_puts(user_collected.at("NumPuts")); + ASSERT_TRUE(GetVarint32(&key_puts, &num_puts)); + ASSERT_EQ(3u, num_puts); + + uint32_t num_deletes; + ASSERT_NE(user_collected.find("NumDeletes"), user_collected.end()); + Slice key_deletes(user_collected.at("NumDeletes")); + ASSERT_TRUE(GetVarint32(&key_deletes, &num_deletes)); + ASSERT_EQ(4u, num_deletes); + + uint32_t num_single_deletes; + ASSERT_NE(user_collected.find("NumSingleDeletes"), + user_collected.end()); + Slice key_single_deletes(user_collected.at("NumSingleDeletes")); + ASSERT_TRUE(GetVarint32(&key_single_deletes, &num_single_deletes)); + ASSERT_EQ(1u, num_single_deletes); + } } } } } // namespace -TEST(TablePropertiesTest, InternalKeyPropertiesCollector) { - TestInternalKeyPropertiesCollector( - kBlockBasedTableMagicNumber, - true /* sanitize */, - std::make_shared() - ); +TEST_P(TablePropertiesTest, InternalKeyPropertiesCollector) { TestInternalKeyPropertiesCollector( - kBlockBasedTableMagicNumber, - true /* not sanitize */, - std::make_shared() - ); + backward_mode_, kBlockBasedTableMagicNumber, true /* sanitize */, + std::make_shared()); + if (backward_mode_) { + TestInternalKeyPropertiesCollector( + backward_mode_, kBlockBasedTableMagicNumber, false /* not sanitize */, + std::make_shared()); + } +#ifndef ROCKSDB_LITE // PlainTable is not supported in Lite PlainTableOptions plain_table_options; plain_table_options.user_key_len = 8; plain_table_options.bloom_bits_per_key = 8; plain_table_options.hash_table_ratio = 0; TestInternalKeyPropertiesCollector( - kPlainTableMagicNumber, false /* not sanitize */, + backward_mode_, kPlainTableMagicNumber, false /* not sanitize */, std::make_shared(plain_table_options)); +#endif // !ROCKSDB_LITE } +INSTANTIATE_TEST_CASE_P(InternalKeyPropertiesCollector, TablePropertiesTest, + ::testing::Bool()); + +INSTANTIATE_TEST_CASE_P(CustomizedTablePropertiesCollector, TablePropertiesTest, + ::testing::Bool()); + } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/transaction_log_impl.cc b/db/transaction_log_impl.cc index bfcf7b3281c..e22c0c4af05 100644 --- a/db/transaction_log_impl.cc +++ b/db/transaction_log_impl.cc @@ -1,19 +1,25 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef ROCKSDB_LITE +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + #include "db/transaction_log_impl.h" +#include #include "db/write_batch_internal.h" +#include "util/file_reader_writer.h" namespace rocksdb { TransactionLogIteratorImpl::TransactionLogIteratorImpl( - const std::string& dir, const DBOptions* options, + const std::string& dir, const ImmutableDBOptions* options, const TransactionLogIterator::ReadOptions& read_options, const EnvOptions& soptions, const SequenceNumber seq, - std::unique_ptr files, DBImpl const* const dbimpl) + std::unique_ptr files, VersionSet const* const versions) : dir_(dir), options_(options), read_options_(read_options), @@ -25,9 +31,9 @@ TransactionLogIteratorImpl::TransactionLogIteratorImpl( currentFileIndex_(0), currentBatchSeq_(0), currentLastSeq_(0), - dbimpl_(dbimpl) { + versions_(versions) { assert(files_ != nullptr); - assert(dbimpl_ != nullptr); + assert(versions_ != nullptr); reporter_.env = options_->env; reporter_.info_log = options_->info_log.get(); @@ -35,23 +41,28 @@ TransactionLogIteratorImpl::TransactionLogIteratorImpl( } Status TransactionLogIteratorImpl::OpenLogFile( - const LogFile* logFile, - unique_ptr* file) { + const LogFile* logFile, unique_ptr* file_reader) { Env* env = options_->env; + unique_ptr file; + Status s; + EnvOptions optimized_env_options = env->OptimizeForLogRead(soptions_); if (logFile->Type() == kArchivedLogFile) { std::string fname = ArchivedLogFileName(dir_, logFile->LogNumber()); - return env->NewSequentialFile(fname, file, soptions_); + s = env->NewSequentialFile(fname, &file, optimized_env_options); } else { std::string fname = LogFileName(dir_, logFile->LogNumber()); - Status status = env->NewSequentialFile(fname, file, soptions_); - if (!status.ok()) { + s = env->NewSequentialFile(fname, &file, optimized_env_options); + if (!s.ok()) { // If cannot open file in DB directory. // Try the archive dir, as it could have moved in the meanwhile. fname = ArchivedLogFileName(dir_, logFile->LogNumber()); - status = env->NewSequentialFile(fname, file, soptions_); + s = env->NewSequentialFile(fname, &file, optimized_env_options); } - return status; } + if (s.ok()) { + file_reader->reset(new SequentialFileReader(std::move(file))); + } + return s; } BatchResult TransactionLogIteratorImpl::GetBatch() { @@ -74,7 +85,7 @@ bool TransactionLogIteratorImpl::RestrictedRead( Slice* record, std::string* scratch) { // Don't read if no more complete entries to read from logs - if (currentLastSeq_ >= dbimpl_->GetLatestSequenceNumber()) { + if (currentLastSeq_ >= versions_->LastSequence()) { return false; } return currentLogReader_->ReadRecord(record, scratch); @@ -97,7 +108,7 @@ void TransactionLogIteratorImpl::SeekToStartSequence( return; } while (RestrictedRead(&record, &scratch)) { - if (record.size() < 12) { + if (record.size() < WriteBatchInternal::kHeader) { reporter_.Corruption( record.size(), Status::Corruption("very small log record")); continue; @@ -157,7 +168,7 @@ void TransactionLogIteratorImpl::NextImpl(bool internal) { currentLogReader_->UnmarkEOF(); } while (RestrictedRead(&record, &scratch)) { - if (record.size() < 12) { + if (record.size() < WriteBatchInternal::kHeader) { reporter_.Corruption( record.size(), Status::Corruption("very small log record")); continue; @@ -177,15 +188,15 @@ void TransactionLogIteratorImpl::NextImpl(bool internal) { // Open the next file if (currentFileIndex_ < files_->size() - 1) { ++currentFileIndex_; - Status status =OpenLogReader(files_->at(currentFileIndex_).get()); - if (!status.ok()) { + Status s = OpenLogReader(files_->at(currentFileIndex_).get()); + if (!s.ok()) { isValid_ = false; - currentStatus_ = status; + currentStatus_ = s; return; } } else { isValid_ = false; - if (currentLastSeq_ == dbimpl_->GetLatestSequenceNumber()) { + if (currentLastSeq_ == versions_->LastSequence()) { currentStatus_ = Status::OK(); } else { currentStatus_ = Status::Corruption("NO MORE DATA LEFT"); @@ -203,12 +214,10 @@ bool TransactionLogIteratorImpl::IsBatchExpected( if (batchSeq != expectedSeq) { char buf[200]; snprintf(buf, sizeof(buf), - "Discontinuity in log records. Got seq=%lu, Expected seq=%lu, " - "Last flushed seq=%lu.Log iterator will reseek the correct " - "batch.", - (unsigned long)batchSeq, - (unsigned long)expectedSeq, - (unsigned long)dbimpl_->GetLatestSequenceNumber()); + "Discontinuity in log records. Got seq=%" PRIu64 + ", Expected seq=%" PRIu64 ", Last flushed seq=%" PRIu64 + ".Log iterator will reseek the correct batch.", + batchSeq, expectedSeq, versions_->LastSequence()); reporter_.Info(buf); return false; } @@ -240,22 +249,23 @@ void TransactionLogIteratorImpl::UpdateCurrentWriteBatch(const Slice& record) { currentLastSeq_ = currentBatchSeq_ + WriteBatchInternal::Count(batch.get()) - 1; // currentBatchSeq_ can only change here - assert(currentLastSeq_ <= dbimpl_->GetLatestSequenceNumber()); + assert(currentLastSeq_ <= versions_->LastSequence()); - currentBatch_ = move(batch); + currentBatch_ = std::move(batch); isValid_ = true; currentStatus_ = Status::OK(); } Status TransactionLogIteratorImpl::OpenLogReader(const LogFile* logFile) { - unique_ptr file; - Status status = OpenLogFile(logFile, &file); - if (!status.ok()) { - return status; + unique_ptr file; + Status s = OpenLogFile(logFile, &file); + if (!s.ok()) { + return s; } assert(file); - currentLogReader_.reset(new log::Reader(std::move(file), &reporter_, - read_options_.verify_checksums_, 0)); + currentLogReader_.reset(new log::Reader( + options_->info_log, std::move(file), &reporter_, + read_options_.verify_checksums_, 0, logFile->LogNumber())); return Status::OK(); } } // namespace rocksdb diff --git a/db/transaction_log_impl.h b/db/transaction_log_impl.h index 319b01cb131..769d8339bd9 100644 --- a/db/transaction_log_impl.h +++ b/db/transaction_log_impl.h @@ -1,33 +1,24 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once #ifndef ROCKSDB_LITE -#pragma once #include +#include "db/log_reader.h" +#include "db/version_set.h" +#include "options/db_options.h" +#include "port/port.h" #include "rocksdb/env.h" #include "rocksdb/options.h" -#include "rocksdb/types.h" #include "rocksdb/transaction_log.h" -#include "db/db_impl.h" -#include "db/log_reader.h" -#include "db/filename.h" +#include "rocksdb/types.h" +#include "util/filename.h" namespace rocksdb { -struct LogReporter : public log::Reader::Reporter { - Env* env; - Logger* info_log; - virtual void Corruption(size_t bytes, const Status& s) { - Log(info_log, "dropping %zu bytes; %s", bytes, s.ToString().c_str()); - } - virtual void Info(const char* s) { - Log(info_log, "%s", s); - } -}; - class LogFileImpl : public LogFile { public: LogFileImpl(uint64_t logNum, WalFileType logType, SequenceNumber startSeq, @@ -38,20 +29,20 @@ class LogFileImpl : public LogFile { sizeFileBytes_(sizeBytes) { } - std::string PathName() const { + std::string PathName() const override { if (type_ == kArchivedLogFile) { return ArchivedLogFileName("", logNumber_); } return LogFileName("", logNumber_); } - uint64_t LogNumber() const { return logNumber_; } + uint64_t LogNumber() const override { return logNumber_; } - WalFileType Type() const { return type_; } + WalFileType Type() const override { return type_; } - SequenceNumber StartSequence() const { return startSequence_; } + SequenceNumber StartSequence() const override { return startSequence_; } - uint64_t SizeFileBytes() const { return sizeFileBytes_; } + uint64_t SizeFileBytes() const override { return sizeFileBytes_; } bool operator < (const LogFile& that) const { return LogNumber() < that.LogNumber(); @@ -68,22 +59,22 @@ class LogFileImpl : public LogFile { class TransactionLogIteratorImpl : public TransactionLogIterator { public: TransactionLogIteratorImpl( - const std::string& dir, const DBOptions* options, + const std::string& dir, const ImmutableDBOptions* options, const TransactionLogIterator::ReadOptions& read_options, const EnvOptions& soptions, const SequenceNumber seqNum, - std::unique_ptr files, DBImpl const* const dbimpl); + std::unique_ptr files, VersionSet const* const versions); - virtual bool Valid(); + virtual bool Valid() override; - virtual void Next(); + virtual void Next() override; - virtual Status status(); + virtual Status status() override; - virtual BatchResult GetBatch(); + virtual BatchResult GetBatch() override; private: const std::string& dir_; - const DBOptions* options_; + const ImmutableDBOptions* options_; const TransactionLogIterator::ReadOptions read_options_; const EnvOptions& soptions_; SequenceNumber startingSequenceNumber_; @@ -94,11 +85,24 @@ class TransactionLogIteratorImpl : public TransactionLogIterator { size_t currentFileIndex_; std::unique_ptr currentBatch_; unique_ptr currentLogReader_; - Status OpenLogFile(const LogFile* logFile, unique_ptr* file); - LogReporter reporter_; + Status OpenLogFile(const LogFile* logFile, + unique_ptr* file); + + struct LogReporter : public log::Reader::Reporter { + Env* env; + Logger* info_log; + virtual void Corruption(size_t bytes, const Status& s) override { + ROCKS_LOG_ERROR(info_log, "dropping %" ROCKSDB_PRIszt " bytes; %s", bytes, + s.ToString().c_str()); + } + virtual void Info(const char* s) { ROCKS_LOG_INFO(info_log, "%s", s); } + } reporter_; + SequenceNumber currentBatchSeq_; // sequence number at start of current batch SequenceNumber currentLastSeq_; // last sequence in the current batch - DBImpl const * const dbimpl_; // The db on whose log files this iterates + // Used only to get latest seq. num + // TODO(icanadi) can this be just a callback? + VersionSet const* const versions_; // Reads from transaction log only if the writebatch record has been written bool RestrictedRead(Slice* record, std::string* scratch); diff --git a/db/version_builder.cc b/db/version_builder.cc new file mode 100644 index 00000000000..e8db67527ea --- /dev/null +++ b/db/version_builder.cc @@ -0,0 +1,467 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/version_builder.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "db/dbformat.h" +#include "db/internal_stats.h" +#include "db/table_cache.h" +#include "db/version_set.h" +#include "port/port.h" +#include "table/table_reader.h" + +namespace rocksdb { + +bool NewestFirstBySeqNo(FileMetaData* a, FileMetaData* b) { + if (a->largest_seqno != b->largest_seqno) { + return a->largest_seqno > b->largest_seqno; + } + if (a->smallest_seqno != b->smallest_seqno) { + return a->smallest_seqno > b->smallest_seqno; + } + // Break ties by file number + return a->fd.GetNumber() > b->fd.GetNumber(); +} + +namespace { +bool BySmallestKey(FileMetaData* a, FileMetaData* b, + const InternalKeyComparator* cmp) { + int r = cmp->Compare(a->smallest, b->smallest); + if (r != 0) { + return (r < 0); + } + // Break ties by file number + return (a->fd.GetNumber() < b->fd.GetNumber()); +} +} // namespace + +class VersionBuilder::Rep { + private: + // Helper to sort files_ in v + // kLevel0 -- NewestFirstBySeqNo + // kLevelNon0 -- BySmallestKey + struct FileComparator { + enum SortMethod { kLevel0 = 0, kLevelNon0 = 1, } sort_method; + const InternalKeyComparator* internal_comparator; + + bool operator()(FileMetaData* f1, FileMetaData* f2) const { + switch (sort_method) { + case kLevel0: + return NewestFirstBySeqNo(f1, f2); + case kLevelNon0: + return BySmallestKey(f1, f2, internal_comparator); + } + assert(false); + return false; + } + }; + + struct LevelState { + std::unordered_set deleted_files; + // Map from file number to file meta data. + std::unordered_map added_files; + }; + + const EnvOptions& env_options_; + Logger* info_log_; + TableCache* table_cache_; + VersionStorageInfo* base_vstorage_; + int num_levels_; + LevelState* levels_; + // Store states of levels larger than num_levels_. We do this instead of + // storing them in levels_ to avoid regression in case there are no files + // on invalid levels. The version is not consistent if in the end the files + // on invalid levels don't cancel out. + std::map> invalid_levels_; + // Whether there are invalid new files or invalid deletion on levels larger + // than num_levels_. + bool has_invalid_levels_; + FileComparator level_zero_cmp_; + FileComparator level_nonzero_cmp_; + + public: + Rep(const EnvOptions& env_options, Logger* info_log, TableCache* table_cache, + VersionStorageInfo* base_vstorage) + : env_options_(env_options), + info_log_(info_log), + table_cache_(table_cache), + base_vstorage_(base_vstorage), + num_levels_(base_vstorage->num_levels()), + has_invalid_levels_(false) { + levels_ = new LevelState[num_levels_]; + level_zero_cmp_.sort_method = FileComparator::kLevel0; + level_nonzero_cmp_.sort_method = FileComparator::kLevelNon0; + level_nonzero_cmp_.internal_comparator = + base_vstorage_->InternalComparator(); + } + + ~Rep() { + for (int level = 0; level < num_levels_; level++) { + const auto& added = levels_[level].added_files; + for (auto& pair : added) { + UnrefFile(pair.second); + } + } + + delete[] levels_; + } + + void UnrefFile(FileMetaData* f) { + f->refs--; + if (f->refs <= 0) { + if (f->table_reader_handle) { + assert(table_cache_ != nullptr); + table_cache_->ReleaseHandle(f->table_reader_handle); + f->table_reader_handle = nullptr; + } + delete f; + } + } + + void CheckConsistency(VersionStorageInfo* vstorage) { +#ifdef NDEBUG + if (!vstorage->force_consistency_checks()) { + // Dont run consistency checks in release mode except if + // explicitly asked to + return; + } +#endif + // make sure the files are sorted correctly + for (int level = 0; level < num_levels_; level++) { + auto& level_files = vstorage->LevelFiles(level); + for (size_t i = 1; i < level_files.size(); i++) { + auto f1 = level_files[i - 1]; + auto f2 = level_files[i]; + if (level == 0) { + if (!level_zero_cmp_(f1, f2)) { + fprintf(stderr, "L0 files are not sorted properly"); + abort(); + } + + if (f2->smallest_seqno == f2->largest_seqno) { + // This is an external file that we ingested + SequenceNumber external_file_seqno = f2->smallest_seqno; + if (!(external_file_seqno < f1->largest_seqno || + external_file_seqno == 0)) { + fprintf(stderr, "L0 file with seqno %" PRIu64 " %" PRIu64 + " vs. file with global_seqno %" PRIu64 "\n", + f1->smallest_seqno, f1->largest_seqno, + external_file_seqno); + abort(); + } + } else if (f1->smallest_seqno <= f2->smallest_seqno) { + fprintf(stderr, "L0 files seqno %" PRIu64 " %" PRIu64 + " vs. %" PRIu64 " %" PRIu64 "\n", + f1->smallest_seqno, f1->largest_seqno, f2->smallest_seqno, + f2->largest_seqno); + abort(); + } + } else { + if (!level_nonzero_cmp_(f1, f2)) { + fprintf(stderr, "L%d files are not sorted properly", level); + abort(); + } + + // Make sure there is no overlap in levels > 0 + if (vstorage->InternalComparator()->Compare(f1->largest, + f2->smallest) >= 0) { + fprintf(stderr, "L%d have overlapping ranges %s vs. %s\n", level, + (f1->largest).DebugString(true).c_str(), + (f2->smallest).DebugString(true).c_str()); + abort(); + } + } + } + } + } + + void CheckConsistencyForDeletes(VersionEdit* edit, uint64_t number, + int level) { +#ifdef NDEBUG + if (!base_vstorage_->force_consistency_checks()) { + // Dont run consistency checks in release mode except if + // explicitly asked to + return; + } +#endif + // a file to be deleted better exist in the previous version + bool found = false; + for (int l = 0; !found && l < num_levels_; l++) { + const std::vector& base_files = + base_vstorage_->LevelFiles(l); + for (size_t i = 0; i < base_files.size(); i++) { + FileMetaData* f = base_files[i]; + if (f->fd.GetNumber() == number) { + found = true; + break; + } + } + } + // if the file did not exist in the previous version, then it + // is possibly moved from lower level to higher level in current + // version + for (int l = level + 1; !found && l < num_levels_; l++) { + auto& level_added = levels_[l].added_files; + auto got = level_added.find(number); + if (got != level_added.end()) { + found = true; + break; + } + } + + // maybe this file was added in a previous edit that was Applied + if (!found) { + auto& level_added = levels_[level].added_files; + auto got = level_added.find(number); + if (got != level_added.end()) { + found = true; + } + } + if (!found) { + fprintf(stderr, "not found %" PRIu64 "\n", number); + abort(); + } + } + + bool CheckConsistencyForNumLevels() { + // Make sure there are no files on or beyond num_levels(). + if (has_invalid_levels_) { + return false; + } + for (auto& level : invalid_levels_) { + if (level.second.size() > 0) { + return false; + } + } + return true; + } + + // Apply all of the edits in *edit to the current state. + void Apply(VersionEdit* edit) { + CheckConsistency(base_vstorage_); + + // Delete files + const VersionEdit::DeletedFileSet& del = edit->GetDeletedFiles(); + for (const auto& del_file : del) { + const auto level = del_file.first; + const auto number = del_file.second; + if (level < num_levels_) { + levels_[level].deleted_files.insert(number); + CheckConsistencyForDeletes(edit, number, level); + + auto exising = levels_[level].added_files.find(number); + if (exising != levels_[level].added_files.end()) { + UnrefFile(exising->second); + levels_[level].added_files.erase(number); + } + } else { + if (invalid_levels_[level].count(number) > 0) { + invalid_levels_[level].erase(number); + } else { + // Deleting an non-existing file on invalid level. + has_invalid_levels_ = true; + } + } + } + + // Add new files + for (const auto& new_file : edit->GetNewFiles()) { + const int level = new_file.first; + if (level < num_levels_) { + FileMetaData* f = new FileMetaData(new_file.second); + f->refs = 1; + + assert(levels_[level].added_files.find(f->fd.GetNumber()) == + levels_[level].added_files.end()); + levels_[level].deleted_files.erase(f->fd.GetNumber()); + levels_[level].added_files[f->fd.GetNumber()] = f; + } else { + uint64_t number = new_file.second.fd.GetNumber(); + if (invalid_levels_[level].count(number) == 0) { + invalid_levels_[level].insert(number); + } else { + // Creating an already existing file on invalid level. + has_invalid_levels_ = true; + } + } + } + } + + // Save the current state in *v. + void SaveTo(VersionStorageInfo* vstorage) { + CheckConsistency(base_vstorage_); + CheckConsistency(vstorage); + + for (int level = 0; level < num_levels_; level++) { + const auto& cmp = (level == 0) ? level_zero_cmp_ : level_nonzero_cmp_; + // Merge the set of added files with the set of pre-existing files. + // Drop any deleted files. Store the result in *v. + const auto& base_files = base_vstorage_->LevelFiles(level); + auto base_iter = base_files.begin(); + auto base_end = base_files.end(); + const auto& unordered_added_files = levels_[level].added_files; + vstorage->Reserve(level, + base_files.size() + unordered_added_files.size()); + + // Sort added files for the level. + std::vector added_files; + added_files.reserve(unordered_added_files.size()); + for (const auto& pair : unordered_added_files) { + added_files.push_back(pair.second); + } + std::sort(added_files.begin(), added_files.end(), cmp); + +#ifndef NDEBUG + FileMetaData* prev_file = nullptr; +#endif + + for (const auto& added : added_files) { +#ifndef NDEBUG + if (level > 0 && prev_file != nullptr) { + assert(base_vstorage_->InternalComparator()->Compare( + prev_file->smallest, added->smallest) <= 0); + } + prev_file = added; +#endif + + // Add all smaller files listed in base_ + for (auto bpos = std::upper_bound(base_iter, base_end, added, cmp); + base_iter != bpos; ++base_iter) { + MaybeAddFile(vstorage, level, *base_iter); + } + + MaybeAddFile(vstorage, level, added); + } + + // Add remaining base files + for (; base_iter != base_end; ++base_iter) { + MaybeAddFile(vstorage, level, *base_iter); + } + } + + CheckConsistency(vstorage); + } + + void LoadTableHandlers(InternalStats* internal_stats, int max_threads, + bool prefetch_index_and_filter_in_cache) { + assert(table_cache_ != nullptr); + // + std::vector> files_meta; + for (int level = 0; level < num_levels_; level++) { + for (auto& file_meta_pair : levels_[level].added_files) { + auto* file_meta = file_meta_pair.second; + assert(!file_meta->table_reader_handle); + files_meta.emplace_back(file_meta, level); + } + } + + std::atomic next_file_meta_idx(0); + std::function load_handlers_func = [&]() { + while (true) { + size_t file_idx = next_file_meta_idx.fetch_add(1); + if (file_idx >= files_meta.size()) { + break; + } + + auto* file_meta = files_meta[file_idx].first; + int level = files_meta[file_idx].second; + table_cache_->FindTable(env_options_, + *(base_vstorage_->InternalComparator()), + file_meta->fd, &file_meta->table_reader_handle, + false /*no_io */, true /* record_read_stats */, + internal_stats->GetFileReadHist(level), false, + level, prefetch_index_and_filter_in_cache); + if (file_meta->table_reader_handle != nullptr) { + // Load table_reader + file_meta->fd.table_reader = table_cache_->GetTableReaderFromHandle( + file_meta->table_reader_handle); + } + } + }; + + if (max_threads <= 1) { + load_handlers_func(); + } else { + std::vector threads; + for (int i = 0; i < max_threads; i++) { + threads.emplace_back(load_handlers_func); + } + + for (auto& t : threads) { + t.join(); + } + } + } + + void MaybeAddFile(VersionStorageInfo* vstorage, int level, FileMetaData* f) { + if (levels_[level].deleted_files.count(f->fd.GetNumber()) > 0) { + // f is to-be-delected table file + vstorage->RemoveCurrentStats(f); + } else { + vstorage->AddFile(level, f, info_log_); + } + } +}; + +VersionBuilder::VersionBuilder(const EnvOptions& env_options, + TableCache* table_cache, + VersionStorageInfo* base_vstorage, + Logger* info_log) + : rep_(new Rep(env_options, info_log, table_cache, base_vstorage)) {} + +VersionBuilder::~VersionBuilder() { delete rep_; } + +void VersionBuilder::CheckConsistency(VersionStorageInfo* vstorage) { + rep_->CheckConsistency(vstorage); +} + +void VersionBuilder::CheckConsistencyForDeletes(VersionEdit* edit, + uint64_t number, int level) { + rep_->CheckConsistencyForDeletes(edit, number, level); +} + +bool VersionBuilder::CheckConsistencyForNumLevels() { + return rep_->CheckConsistencyForNumLevels(); +} + +void VersionBuilder::Apply(VersionEdit* edit) { rep_->Apply(edit); } + +void VersionBuilder::SaveTo(VersionStorageInfo* vstorage) { + rep_->SaveTo(vstorage); +} + +void VersionBuilder::LoadTableHandlers( + InternalStats* internal_stats, int max_threads, + bool prefetch_index_and_filter_in_cache) { + rep_->LoadTableHandlers(internal_stats, max_threads, + prefetch_index_and_filter_in_cache); +} + +void VersionBuilder::MaybeAddFile(VersionStorageInfo* vstorage, int level, + FileMetaData* f) { + rep_->MaybeAddFile(vstorage, level, f); +} + +} // namespace rocksdb diff --git a/db/version_builder.h b/db/version_builder.h new file mode 100644 index 00000000000..440d4eaf6ba --- /dev/null +++ b/db/version_builder.h @@ -0,0 +1,45 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +#pragma once +#include "rocksdb/env.h" + +namespace rocksdb { + +class TableCache; +class VersionStorageInfo; +class VersionEdit; +struct FileMetaData; +class InternalStats; + +// A helper class so we can efficiently apply a whole sequence +// of edits to a particular state without creating intermediate +// Versions that contain full copies of the intermediate state. +class VersionBuilder { + public: + VersionBuilder(const EnvOptions& env_options, TableCache* table_cache, + VersionStorageInfo* base_vstorage, Logger* info_log = nullptr); + ~VersionBuilder(); + void CheckConsistency(VersionStorageInfo* vstorage); + void CheckConsistencyForDeletes(VersionEdit* edit, uint64_t number, + int level); + bool CheckConsistencyForNumLevels(); + void Apply(VersionEdit* edit); + void SaveTo(VersionStorageInfo* vstorage); + void LoadTableHandlers(InternalStats* internal_stats, int max_threads, + bool prefetch_index_and_filter_in_cache); + void MaybeAddFile(VersionStorageInfo* vstorage, int level, FileMetaData* f); + + private: + class Rep; + Rep* rep_; +}; + +extern bool NewestFirstBySeqNo(FileMetaData* a, FileMetaData* b); +} // namespace rocksdb diff --git a/db/version_builder_test.cc b/db/version_builder_test.cc new file mode 100644 index 00000000000..304df2a0455 --- /dev/null +++ b/db/version_builder_test.cc @@ -0,0 +1,305 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include +#include "db/version_edit.h" +#include "db/version_set.h" +#include "util/logging.h" +#include "util/string_util.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +class VersionBuilderTest : public testing::Test { + public: + const Comparator* ucmp_; + InternalKeyComparator icmp_; + Options options_; + ImmutableCFOptions ioptions_; + MutableCFOptions mutable_cf_options_; + VersionStorageInfo vstorage_; + uint32_t file_num_; + CompactionOptionsFIFO fifo_options_; + std::vector size_being_compacted_; + + VersionBuilderTest() + : ucmp_(BytewiseComparator()), + icmp_(ucmp_), + ioptions_(options_), + mutable_cf_options_(options_), + vstorage_(&icmp_, ucmp_, options_.num_levels, kCompactionStyleLevel, + nullptr, false), + file_num_(1) { + mutable_cf_options_.RefreshDerivedOptions(ioptions_); + size_being_compacted_.resize(options_.num_levels); + } + + ~VersionBuilderTest() { + for (int i = 0; i < vstorage_.num_levels(); i++) { + for (auto* f : vstorage_.LevelFiles(i)) { + if (--f->refs == 0) { + delete f; + } + } + } + } + + InternalKey GetInternalKey(const char* ukey, + SequenceNumber smallest_seq = 100) { + return InternalKey(ukey, smallest_seq, kTypeValue); + } + + void Add(int level, uint32_t file_number, const char* smallest, + const char* largest, uint64_t file_size = 0, uint32_t path_id = 0, + SequenceNumber smallest_seq = 100, SequenceNumber largest_seq = 100, + uint64_t num_entries = 0, uint64_t num_deletions = 0, + bool sampled = false, SequenceNumber smallest_seqno = 0, + SequenceNumber largest_seqno = 0) { + assert(level < vstorage_.num_levels()); + FileMetaData* f = new FileMetaData; + f->fd = FileDescriptor(file_number, path_id, file_size); + f->smallest = GetInternalKey(smallest, smallest_seq); + f->largest = GetInternalKey(largest, largest_seq); + f->smallest_seqno = smallest_seqno; + f->largest_seqno = largest_seqno; + f->compensated_file_size = file_size; + f->refs = 0; + f->num_entries = num_entries; + f->num_deletions = num_deletions; + vstorage_.AddFile(level, f); + if (sampled) { + f->init_stats_from_file = true; + vstorage_.UpdateAccumulatedStats(f); + } + } + + void UpdateVersionStorageInfo() { + vstorage_.UpdateFilesByCompactionPri(ioptions_.compaction_pri); + vstorage_.UpdateNumNonEmptyLevels(); + vstorage_.GenerateFileIndexer(); + vstorage_.GenerateLevelFilesBrief(); + vstorage_.CalculateBaseBytes(ioptions_, mutable_cf_options_); + vstorage_.GenerateLevel0NonOverlapping(); + vstorage_.SetFinalized(); + } +}; + +void UnrefFilesInVersion(VersionStorageInfo* new_vstorage) { + for (int i = 0; i < new_vstorage->num_levels(); i++) { + for (auto* f : new_vstorage->LevelFiles(i)) { + if (--f->refs == 0) { + delete f; + } + } + } +} + +TEST_F(VersionBuilderTest, ApplyAndSaveTo) { + Add(0, 1U, "150", "200", 100U); + + Add(1, 66U, "150", "200", 100U); + Add(1, 88U, "201", "300", 100U); + + Add(2, 6U, "150", "179", 100U); + Add(2, 7U, "180", "220", 100U); + Add(2, 8U, "221", "300", 100U); + + Add(3, 26U, "150", "170", 100U); + Add(3, 27U, "171", "179", 100U); + Add(3, 28U, "191", "220", 100U); + Add(3, 29U, "221", "300", 100U); + UpdateVersionStorageInfo(); + + VersionEdit version_edit; + version_edit.AddFile(2, 666, 0, 100U, GetInternalKey("301"), + GetInternalKey("350"), 200, 200, false); + version_edit.DeleteFile(3, 27U); + + EnvOptions env_options; + + VersionBuilder version_builder(env_options, nullptr, &vstorage_); + + VersionStorageInfo new_vstorage(&icmp_, ucmp_, options_.num_levels, + kCompactionStyleLevel, nullptr, false); + version_builder.Apply(&version_edit); + version_builder.SaveTo(&new_vstorage); + + ASSERT_EQ(400U, new_vstorage.NumLevelBytes(2)); + ASSERT_EQ(300U, new_vstorage.NumLevelBytes(3)); + + UnrefFilesInVersion(&new_vstorage); +} + +TEST_F(VersionBuilderTest, ApplyAndSaveToDynamic) { + ioptions_.level_compaction_dynamic_level_bytes = true; + + Add(0, 1U, "150", "200", 100U, 0, 200U, 200U, 0, 0, false, 200U, 200U); + Add(0, 88U, "201", "300", 100U, 0, 100U, 100U, 0, 0, false, 100U, 100U); + + Add(4, 6U, "150", "179", 100U); + Add(4, 7U, "180", "220", 100U); + Add(4, 8U, "221", "300", 100U); + + Add(5, 26U, "150", "170", 100U); + Add(5, 27U, "171", "179", 100U); + UpdateVersionStorageInfo(); + + VersionEdit version_edit; + version_edit.AddFile(3, 666, 0, 100U, GetInternalKey("301"), + GetInternalKey("350"), 200, 200, false); + version_edit.DeleteFile(0, 1U); + version_edit.DeleteFile(0, 88U); + + EnvOptions env_options; + + VersionBuilder version_builder(env_options, nullptr, &vstorage_); + + VersionStorageInfo new_vstorage(&icmp_, ucmp_, options_.num_levels, + kCompactionStyleLevel, nullptr, false); + version_builder.Apply(&version_edit); + version_builder.SaveTo(&new_vstorage); + + ASSERT_EQ(0U, new_vstorage.NumLevelBytes(0)); + ASSERT_EQ(100U, new_vstorage.NumLevelBytes(3)); + ASSERT_EQ(300U, new_vstorage.NumLevelBytes(4)); + ASSERT_EQ(200U, new_vstorage.NumLevelBytes(5)); + + UnrefFilesInVersion(&new_vstorage); +} + +TEST_F(VersionBuilderTest, ApplyAndSaveToDynamic2) { + ioptions_.level_compaction_dynamic_level_bytes = true; + + Add(0, 1U, "150", "200", 100U, 0, 200U, 200U, 0, 0, false, 200U, 200U); + Add(0, 88U, "201", "300", 100U, 0, 100U, 100U, 0, 0, false, 100U, 100U); + + Add(4, 6U, "150", "179", 100U); + Add(4, 7U, "180", "220", 100U); + Add(4, 8U, "221", "300", 100U); + + Add(5, 26U, "150", "170", 100U); + Add(5, 27U, "171", "179", 100U); + UpdateVersionStorageInfo(); + + VersionEdit version_edit; + version_edit.AddFile(4, 666, 0, 100U, GetInternalKey("301"), + GetInternalKey("350"), 200, 200, false); + version_edit.DeleteFile(0, 1U); + version_edit.DeleteFile(0, 88U); + version_edit.DeleteFile(4, 6U); + version_edit.DeleteFile(4, 7U); + version_edit.DeleteFile(4, 8U); + + EnvOptions env_options; + + VersionBuilder version_builder(env_options, nullptr, &vstorage_); + + VersionStorageInfo new_vstorage(&icmp_, ucmp_, options_.num_levels, + kCompactionStyleLevel, nullptr, false); + version_builder.Apply(&version_edit); + version_builder.SaveTo(&new_vstorage); + + ASSERT_EQ(0U, new_vstorage.NumLevelBytes(0)); + ASSERT_EQ(100U, new_vstorage.NumLevelBytes(4)); + ASSERT_EQ(200U, new_vstorage.NumLevelBytes(5)); + + UnrefFilesInVersion(&new_vstorage); +} + +TEST_F(VersionBuilderTest, ApplyMultipleAndSaveTo) { + UpdateVersionStorageInfo(); + + VersionEdit version_edit; + version_edit.AddFile(2, 666, 0, 100U, GetInternalKey("301"), + GetInternalKey("350"), 200, 200, false); + version_edit.AddFile(2, 676, 0, 100U, GetInternalKey("401"), + GetInternalKey("450"), 200, 200, false); + version_edit.AddFile(2, 636, 0, 100U, GetInternalKey("601"), + GetInternalKey("650"), 200, 200, false); + version_edit.AddFile(2, 616, 0, 100U, GetInternalKey("501"), + GetInternalKey("550"), 200, 200, false); + version_edit.AddFile(2, 606, 0, 100U, GetInternalKey("701"), + GetInternalKey("750"), 200, 200, false); + + EnvOptions env_options; + + VersionBuilder version_builder(env_options, nullptr, &vstorage_); + + VersionStorageInfo new_vstorage(&icmp_, ucmp_, options_.num_levels, + kCompactionStyleLevel, nullptr, false); + version_builder.Apply(&version_edit); + version_builder.SaveTo(&new_vstorage); + + ASSERT_EQ(500U, new_vstorage.NumLevelBytes(2)); + + UnrefFilesInVersion(&new_vstorage); +} + +TEST_F(VersionBuilderTest, ApplyDeleteAndSaveTo) { + UpdateVersionStorageInfo(); + + EnvOptions env_options; + VersionBuilder version_builder(env_options, nullptr, &vstorage_); + VersionStorageInfo new_vstorage(&icmp_, ucmp_, options_.num_levels, + kCompactionStyleLevel, nullptr, false); + + VersionEdit version_edit; + version_edit.AddFile(2, 666, 0, 100U, GetInternalKey("301"), + GetInternalKey("350"), 200, 200, false); + version_edit.AddFile(2, 676, 0, 100U, GetInternalKey("401"), + GetInternalKey("450"), 200, 200, false); + version_edit.AddFile(2, 636, 0, 100U, GetInternalKey("601"), + GetInternalKey("650"), 200, 200, false); + version_edit.AddFile(2, 616, 0, 100U, GetInternalKey("501"), + GetInternalKey("550"), 200, 200, false); + version_edit.AddFile(2, 606, 0, 100U, GetInternalKey("701"), + GetInternalKey("750"), 200, 200, false); + version_builder.Apply(&version_edit); + + VersionEdit version_edit2; + version_edit.AddFile(2, 808, 0, 100U, GetInternalKey("901"), + GetInternalKey("950"), 200, 200, false); + version_edit2.DeleteFile(2, 616); + version_edit2.DeleteFile(2, 636); + version_edit.AddFile(2, 806, 0, 100U, GetInternalKey("801"), + GetInternalKey("850"), 200, 200, false); + version_builder.Apply(&version_edit2); + + version_builder.SaveTo(&new_vstorage); + + ASSERT_EQ(300U, new_vstorage.NumLevelBytes(2)); + + UnrefFilesInVersion(&new_vstorage); +} + +TEST_F(VersionBuilderTest, EstimatedActiveKeys) { + const uint32_t kTotalSamples = 20; + const uint32_t kNumLevels = 5; + const uint32_t kFilesPerLevel = 8; + const uint32_t kNumFiles = kNumLevels * kFilesPerLevel; + const uint32_t kEntriesPerFile = 1000; + const uint32_t kDeletionsPerFile = 100; + for (uint32_t i = 0; i < kNumFiles; ++i) { + Add(static_cast(i / kFilesPerLevel), i + 1, + ToString((i + 100) * 1000).c_str(), + ToString((i + 100) * 1000 + 999).c_str(), + 100U, 0, 100, 100, + kEntriesPerFile, kDeletionsPerFile, + (i < kTotalSamples)); + } + // minus 2X for the number of deletion entries because: + // 1x for deletion entry does not count as a data entry. + // 1x for each deletion entry will actually remove one data entry. + ASSERT_EQ(vstorage_.GetEstimatedActiveKeys(), + (kEntriesPerFile - 2 * kDeletionsPerFile) * kNumFiles); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/version_edit.cc b/db/version_edit.cc index 4e2cf8f5b40..b01f7bbdf70 100644 --- a/db/version_edit.cc +++ b/db/version_edit.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,8 +10,11 @@ #include "db/version_edit.h" #include "db/version_set.h" -#include "util/coding.h" #include "rocksdb/slice.h" +#include "util/coding.h" +#include "util/event_logger.h" +#include "util/string_util.h" +#include "util/sync_point.h" namespace rocksdb { @@ -31,12 +34,22 @@ enum Tag { // these are new formats divergent from open source leveldb kNewFile2 = 100, kNewFile3 = 102, + kNewFile4 = 103, // 4th (the latest) format version of adding files kColumnFamily = 200, // specify column family for version edit kColumnFamilyAdd = 201, kColumnFamilyDrop = 202, kMaxColumnFamily = 203, }; +enum CustomTag { + kTerminate = 1, // The end of customized fields + kNeedCompaction = 2, + kPathId = 65, +}; +// If this bit for the custom tag is set, opening DB should fail if +// we don't know this field. +uint32_t kCustomTagNonSafeIgnoreMask = 1 << 6; + uint64_t PackFileNumberAndPathId(uint64_t number, uint64_t path_id) { assert(number <= kFileNumberMask); return number | (path_id * (kFileNumberMask + 1)); @@ -64,63 +77,104 @@ void VersionEdit::Clear() { column_family_name_.clear(); } -void VersionEdit::EncodeTo(std::string* dst) const { +bool VersionEdit::EncodeTo(std::string* dst) const { if (has_comparator_) { PutVarint32(dst, kComparator); PutLengthPrefixedSlice(dst, comparator_); } if (has_log_number_) { - PutVarint32(dst, kLogNumber); - PutVarint64(dst, log_number_); + PutVarint32Varint64(dst, kLogNumber, log_number_); } if (has_prev_log_number_) { - PutVarint32(dst, kPrevLogNumber); - PutVarint64(dst, prev_log_number_); + PutVarint32Varint64(dst, kPrevLogNumber, prev_log_number_); } if (has_next_file_number_) { - PutVarint32(dst, kNextFileNumber); - PutVarint64(dst, next_file_number_); + PutVarint32Varint64(dst, kNextFileNumber, next_file_number_); } if (has_last_sequence_) { - PutVarint32(dst, kLastSequence); - PutVarint64(dst, last_sequence_); + PutVarint32Varint64(dst, kLastSequence, last_sequence_); } if (has_max_column_family_) { - PutVarint32(dst, kMaxColumnFamily); - PutVarint32(dst, max_column_family_); + PutVarint32Varint32(dst, kMaxColumnFamily, max_column_family_); } for (const auto& deleted : deleted_files_) { - PutVarint32(dst, kDeletedFile); - PutVarint32(dst, deleted.first /* level */); - PutVarint64(dst, deleted.second /* file number */); + PutVarint32Varint32Varint64(dst, kDeletedFile, deleted.first /* level */, + deleted.second /* file number */); } for (size_t i = 0; i < new_files_.size(); i++) { const FileMetaData& f = new_files_[i].second; - if (f.fd.GetPathId() == 0) { + if (!f.smallest.Valid() || !f.largest.Valid()) { + return false; + } + bool has_customized_fields = false; + if (f.marked_for_compaction) { + PutVarint32(dst, kNewFile4); + has_customized_fields = true; + } else if (f.fd.GetPathId() == 0) { // Use older format to make sure user can roll back the build if they // don't config multiple DB paths. PutVarint32(dst, kNewFile2); } else { PutVarint32(dst, kNewFile3); } - PutVarint32(dst, new_files_[i].first); // level - PutVarint64(dst, f.fd.GetNumber()); - if (f.fd.GetPathId() != 0) { + PutVarint32Varint64(dst, new_files_[i].first /* level */, f.fd.GetNumber()); + if (f.fd.GetPathId() != 0 && !has_customized_fields) { + // kNewFile3 PutVarint32(dst, f.fd.GetPathId()); } PutVarint64(dst, f.fd.GetFileSize()); PutLengthPrefixedSlice(dst, f.smallest.Encode()); PutLengthPrefixedSlice(dst, f.largest.Encode()); - PutVarint64(dst, f.smallest_seqno); - PutVarint64(dst, f.largest_seqno); + PutVarint64Varint64(dst, f.smallest_seqno, f.largest_seqno); + if (has_customized_fields) { + // Customized fields' format: + // +-----------------------------+ + // | 1st field's tag (varint32) | + // +-----------------------------+ + // | 1st field's size (varint32) | + // +-----------------------------+ + // | bytes for 1st field | + // | (based on size decoded) | + // +-----------------------------+ + // | | + // | ...... | + // | | + // +-----------------------------+ + // | last field's size (varint32)| + // +-----------------------------+ + // | bytes for last field | + // | (based on size decoded) | + // +-----------------------------+ + // | terminating tag (varint32) | + // +-----------------------------+ + // + // Customized encoding for fields: + // tag kPathId: 1 byte as path_id + // tag kNeedCompaction: + // now only can take one char value 1 indicating need-compaction + // + if (f.fd.GetPathId() != 0) { + PutVarint32(dst, CustomTag::kPathId); + char p = static_cast(f.fd.GetPathId()); + PutLengthPrefixedSlice(dst, Slice(&p, 1)); + } + if (f.marked_for_compaction) { + PutVarint32(dst, CustomTag::kNeedCompaction); + char p = static_cast(1); + PutLengthPrefixedSlice(dst, Slice(&p, 1)); + } + TEST_SYNC_POINT_CALLBACK("VersionEdit::EncodeTo:NewFile4:CustomizeFields", + dst); + + PutVarint32(dst, CustomTag::kTerminate); + } } // 0 is default and does not need to be explicitly written if (column_family_ != 0) { - PutVarint32(dst, kColumnFamily); - PutVarint32(dst, column_family_); + PutVarint32Varint32(dst, kColumnFamily, column_family_); } if (is_column_family_add_) { @@ -131,6 +185,7 @@ void VersionEdit::EncodeTo(std::string* dst) const { if (is_column_family_drop_) { PutVarint32(dst, kColumnFamilyDrop); } + return true; } static bool GetInternalKey(Slice* input, InternalKey* dst) { @@ -156,6 +211,63 @@ bool VersionEdit::GetLevel(Slice* input, int* level, const char** msg) { } } +const char* VersionEdit::DecodeNewFile4From(Slice* input) { + const char* msg = nullptr; + int level; + FileMetaData f; + uint64_t number; + uint32_t path_id = 0; + uint64_t file_size; + if (GetLevel(input, &level, &msg) && GetVarint64(input, &number) && + GetVarint64(input, &file_size) && GetInternalKey(input, &f.smallest) && + GetInternalKey(input, &f.largest) && + GetVarint64(input, &f.smallest_seqno) && + GetVarint64(input, &f.largest_seqno)) { + // See comments in VersionEdit::EncodeTo() for format of customized fields + while (true) { + uint32_t custom_tag; + Slice field; + if (!GetVarint32(input, &custom_tag)) { + return "new-file4 custom field"; + } + if (custom_tag == kTerminate) { + break; + } + if (!GetLengthPrefixedSlice(input, &field)) { + return "new-file4 custom field lenth prefixed slice error"; + } + switch (custom_tag) { + case kPathId: + if (field.size() != 1) { + return "path_id field wrong size"; + } + path_id = field[0]; + if (path_id > 3) { + return "path_id wrong vaue"; + } + break; + case kNeedCompaction: + if (field.size() != 1) { + return "need_compaction field wrong size"; + } + f.marked_for_compaction = (field[0] == 1); + break; + default: + if ((custom_tag & kCustomTagNonSafeIgnoreMask) != 0) { + // Should not proceed if cannot understand it + return "new-file4 custom field not supported"; + } + break; + } + } + } else { + return "new-file4 entry"; + } + f.fd = FileDescriptor(number, path_id, file_size); + new_files_.push_back(std::make_pair(level, f)); + return nullptr; +} + Status VersionEdit::DecodeFrom(const Slice& src) { Clear(); Slice input = src; @@ -164,7 +276,6 @@ Status VersionEdit::DecodeFrom(const Slice& src) { // Temporary storage for parsing int level; - uint64_t number; FileMetaData f; Slice str; InternalKey key; @@ -233,9 +344,9 @@ Status VersionEdit::DecodeFrom(const Slice& src) { } break; - case kDeletedFile: - if (GetLevel(&input, &level, &msg) && - GetVarint64(&input, &number)) { + case kDeletedFile: { + uint64_t number; + if (GetLevel(&input, &level, &msg) && GetVarint64(&input, &number)) { deleted_files_.insert(std::make_pair(level, number)); } else { if (!msg) { @@ -243,6 +354,7 @@ Status VersionEdit::DecodeFrom(const Slice& src) { } } break; + } case kNewFile: { uint64_t number; @@ -293,12 +405,17 @@ Status VersionEdit::DecodeFrom(const Slice& src) { new_files_.push_back(std::make_pair(level, f)); } else { if (!msg) { - msg = "new-file2 entry"; + msg = "new-file3 entry"; } } break; } + case kNewFile4: { + msg = DecodeNewFile4From(&input); + break; + } + case kColumnFamily: if (!GetVarint32(&input, &column_family_)) { if (!msg) { @@ -355,7 +472,7 @@ std::string VersionEdit::DebugString(bool hex_key) const { AppendNumberTo(&r, prev_log_number_); } if (has_next_file_number_) { - r.append("\n NextFile: "); + r.append("\n NextFileNumber: "); AppendNumberTo(&r, next_file_number_); } if (has_last_sequence_) { @@ -400,4 +517,75 @@ std::string VersionEdit::DebugString(bool hex_key) const { return r; } +std::string VersionEdit::DebugJSON(int edit_num, bool hex_key) const { + JSONWriter jw; + jw << "EditNumber" << edit_num; + + if (has_comparator_) { + jw << "Comparator" << comparator_; + } + if (has_log_number_) { + jw << "LogNumber" << log_number_; + } + if (has_prev_log_number_) { + jw << "PrevLogNumber" << prev_log_number_; + } + if (has_next_file_number_) { + jw << "NextFileNumber" << next_file_number_; + } + if (has_last_sequence_) { + jw << "LastSeq" << last_sequence_; + } + + if (!deleted_files_.empty()) { + jw << "DeletedFiles"; + jw.StartArray(); + + for (DeletedFileSet::const_iterator iter = deleted_files_.begin(); + iter != deleted_files_.end(); + ++iter) { + jw.StartArrayedObject(); + jw << "Level" << iter->first; + jw << "FileNumber" << iter->second; + jw.EndArrayedObject(); + } + + jw.EndArray(); + } + + if (!new_files_.empty()) { + jw << "AddedFiles"; + jw.StartArray(); + + for (size_t i = 0; i < new_files_.size(); i++) { + jw.StartArrayedObject(); + jw << "Level" << new_files_[i].first; + const FileMetaData& f = new_files_[i].second; + jw << "FileNumber" << f.fd.GetNumber(); + jw << "FileSize" << f.fd.GetFileSize(); + jw << "SmallestIKey" << f.smallest.DebugString(hex_key); + jw << "LargestIKey" << f.largest.DebugString(hex_key); + jw.EndArrayedObject(); + } + + jw.EndArray(); + } + + jw << "ColumnFamily" << column_family_; + + if (is_column_family_add_) { + jw << "ColumnFamilyAdd" << column_family_name_; + } + if (is_column_family_drop_) { + jw << "ColumnFamilyDrop" << column_family_name_; + } + if (has_max_column_family_) { + jw << "MaxColumnFamily" << max_column_family_; + } + + jw.EndObject(); + + return jw.Get(); +} + } // namespace rocksdb diff --git a/db/version_edit.h b/db/version_edit.h index 58edfed4515..47ebf5b1c76 100644 --- a/db/version_edit.h +++ b/db/version_edit.h @@ -1,13 +1,14 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #pragma once +#include #include #include #include @@ -38,10 +39,10 @@ struct FileDescriptor { FileDescriptor() : FileDescriptor(0, 0, 0) {} - FileDescriptor(uint64_t number, uint32_t path_id, uint64_t file_size) + FileDescriptor(uint64_t number, uint32_t path_id, uint64_t _file_size) : table_reader(nullptr), packed_number_and_path_id(PackFileNumberAndPathId(number, path_id)), - file_size(file_size) {} + file_size(_file_size) {} FileDescriptor& operator=(const FileDescriptor& fd) { table_reader = fd.table_reader; @@ -54,52 +55,90 @@ struct FileDescriptor { return packed_number_and_path_id & kFileNumberMask; } uint32_t GetPathId() const { - return packed_number_and_path_id / (kFileNumberMask + 1); + return static_cast( + packed_number_and_path_id / (kFileNumberMask + 1)); } uint64_t GetFileSize() const { return file_size; } }; +struct FileSampledStats { + FileSampledStats() : num_reads_sampled(0) {} + FileSampledStats(const FileSampledStats& other) { *this = other; } + FileSampledStats& operator=(const FileSampledStats& other) { + num_reads_sampled = other.num_reads_sampled.load(); + return *this; + } + + // number of user reads to this file. + mutable std::atomic num_reads_sampled; +}; + struct FileMetaData { - int refs; FileDescriptor fd; InternalKey smallest; // Smallest internal key served by table InternalKey largest; // Largest internal key served by table - bool being_compacted; // Is this file undergoing compaction? SequenceNumber smallest_seqno; // The smallest seqno in this file SequenceNumber largest_seqno; // The largest seqno in this file // Needs to be disposed when refs becomes 0. Cache::Handle* table_reader_handle; + FileSampledStats stats; + // Stats for compensating deletion entries during compaction // File size compensated by deletion entry. - // This is updated in Version::UpdateTemporaryStats() first time when the - // file is created or loaded. After it is updated, it is immutable. + // This is updated in Version::UpdateAccumulatedStats() first time when the + // file is created or loaded. After it is updated (!= 0), it is immutable. uint64_t compensated_file_size; + // These values can mutate, but they can only be read or written from + // single-threaded LogAndApply thread uint64_t num_entries; // the number of entries. uint64_t num_deletions; // the number of deletion entries. uint64_t raw_key_size; // total uncompressed key size. uint64_t raw_value_size; // total uncompressed value size. + + int refs; // Reference count + + bool being_compacted; // Is this file undergoing compaction? bool init_stats_from_file; // true if the data-entry stats of this file // has initialized from file. + bool marked_for_compaction; // True if client asked us nicely to compact this + // file. + FileMetaData() - : refs(0), - being_compacted(false), + : smallest_seqno(kMaxSequenceNumber), + largest_seqno(0), table_reader_handle(nullptr), compensated_file_size(0), num_entries(0), num_deletions(0), raw_key_size(0), raw_value_size(0), - init_stats_from_file(false) {} + refs(0), + being_compacted(false), + init_stats_from_file(false), + marked_for_compaction(false) {} + + // REQUIRED: Keys must be given to the function in sorted order (it expects + // the last key to be the largest). + void UpdateBoundaries(const Slice& key, SequenceNumber seqno) { + if (smallest.size() == 0) { + smallest.DecodeFrom(key); + } + largest.DecodeFrom(key); + smallest_seqno = std::min(smallest_seqno, seqno); + largest_seqno = std::max(largest_seqno, seqno); + } }; -// A compressed copy of file meta data that just contain -// smallest and largest key's slice +// A compressed copy of file meta data that just contain minimum data needed +// to server read operations, while still keeping the pointer to full metadata +// of the file in case it is needed. struct FdWithKeyRange { FileDescriptor fd; + FileMetaData* file_metadata; // Point to all metadata Slice smallest_key; // slice that contain smallest key Slice largest_key; // slice that contain largest key @@ -109,20 +148,20 @@ struct FdWithKeyRange { largest_key() { } - FdWithKeyRange(FileDescriptor fd, - Slice smallest_key, Slice largest_key) - : fd(fd), - smallest_key(smallest_key), - largest_key(largest_key) { - } + FdWithKeyRange(FileDescriptor _fd, Slice _smallest_key, Slice _largest_key, + FileMetaData* _file_metadata) + : fd(_fd), + file_metadata(_file_metadata), + smallest_key(_smallest_key), + largest_key(_largest_key) {} }; // Data structure to store an array of FdWithKeyRange in one level // Actual data is guaranteed to be stored closely -struct FileLevel { +struct LevelFilesBrief { size_t num_files; FdWithKeyRange* files; - FileLevel() { + LevelFilesBrief() { num_files = 0; files = nullptr; } @@ -163,18 +202,25 @@ class VersionEdit { // Add the specified file at the specified number. // REQUIRES: This version has not been saved (see VersionSet::SaveTo) // REQUIRES: "smallest" and "largest" are smallest and largest keys in file - void AddFile(int level, uint64_t file, uint64_t file_size, - uint64_t file_path_id, const InternalKey& smallest, + void AddFile(int level, uint64_t file, uint32_t file_path_id, + uint64_t file_size, const InternalKey& smallest, const InternalKey& largest, const SequenceNumber& smallest_seqno, - const SequenceNumber& largest_seqno) { + const SequenceNumber& largest_seqno, + bool marked_for_compaction) { assert(smallest_seqno <= largest_seqno); FileMetaData f; - f.fd = FileDescriptor(file, file_size, file_path_id); + f.fd = FileDescriptor(file, file_path_id, file_size); f.smallest = smallest; f.largest = largest; f.smallest_seqno = smallest_seqno; f.largest_seqno = largest_seqno; - new_files_.push_back(std::make_pair(level, f)); + f.marked_for_compaction = marked_for_compaction; + new_files_.emplace_back(level, std::move(f)); + } + + void AddFile(int level, const FileMetaData& f) { + assert(f.smallest_seqno <= f.largest_seqno); + new_files_.emplace_back(level, f); } // Delete the specified "file" from the specified "level". @@ -183,9 +229,7 @@ class VersionEdit { } // Number of edits - int NumEntries() { - return new_files_.size() + deleted_files_.size(); - } + size_t NumEntries() { return new_files_.size() + deleted_files_.size(); } bool IsColumnFamilyManipulation() { return is_column_family_add_ || is_column_family_drop_; @@ -212,17 +256,26 @@ class VersionEdit { is_column_family_drop_ = true; } - void EncodeTo(std::string* dst) const; + // return true on success. + bool EncodeTo(std::string* dst) const; Status DecodeFrom(const Slice& src); + const char* DecodeNewFile4From(Slice* input); + + typedef std::set> DeletedFileSet; + + const DeletedFileSet& GetDeletedFiles() { return deleted_files_; } + const std::vector>& GetNewFiles() { + return new_files_; + } + std::string DebugString(bool hex_key = false) const; + std::string DebugJSON(int edit_num, bool hex_key = false) const; private: friend class VersionSet; friend class Version; - typedef std::set< std::pair> DeletedFileSet; - bool GetLevel(Slice* input, int* level, const char** msg); int max_level_; diff --git a/db/version_edit_test.cc b/db/version_edit_test.cc index 850f242c1f1..338bb36f605 100644 --- a/db/version_edit_test.cc +++ b/db/version_edit_test.cc @@ -1,13 +1,14 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #include "db/version_edit.h" +#include "util/sync_point.h" #include "util/testharness.h" namespace rocksdb { @@ -22,18 +23,19 @@ static void TestEncodeDecode(const VersionEdit& edit) { ASSERT_EQ(encoded, encoded2); } -class VersionEditTest { }; +class VersionEditTest : public testing::Test {}; -TEST(VersionEditTest, EncodeDecode) { +TEST_F(VersionEditTest, EncodeDecode) { static const uint64_t kBig = 1ull << 50; + static const uint32_t kBig32Bit = 1ull << 30; VersionEdit edit; for (int i = 0; i < 4; i++) { TestEncodeDecode(edit); - edit.AddFile(3, kBig + 300 + i, kBig + 400 + i, 0, + edit.AddFile(3, kBig + 300 + i, kBig32Bit + 400 + i, 0, InternalKey("foo", kBig + 500 + i, kTypeValue), InternalKey("zoo", kBig + 600 + i, kTypeDeletion), - kBig + 500 + i, kBig + 600 + i); + kBig + 500 + i, kBig + 600 + i, false); edit.DeleteFile(4, kBig + 700 + i); } @@ -44,7 +46,129 @@ TEST(VersionEditTest, EncodeDecode) { TestEncodeDecode(edit); } -TEST(VersionEditTest, ColumnFamilyTest) { +TEST_F(VersionEditTest, EncodeDecodeNewFile4) { + static const uint64_t kBig = 1ull << 50; + + VersionEdit edit; + edit.AddFile(3, 300, 3, 100, InternalKey("foo", kBig + 500, kTypeValue), + InternalKey("zoo", kBig + 600, kTypeDeletion), kBig + 500, + kBig + 600, true); + edit.AddFile(4, 301, 3, 100, InternalKey("foo", kBig + 501, kTypeValue), + InternalKey("zoo", kBig + 601, kTypeDeletion), kBig + 501, + kBig + 601, false); + edit.AddFile(5, 302, 0, 100, InternalKey("foo", kBig + 502, kTypeValue), + InternalKey("zoo", kBig + 602, kTypeDeletion), kBig + 502, + kBig + 602, true); + + edit.DeleteFile(4, 700); + + edit.SetComparatorName("foo"); + edit.SetLogNumber(kBig + 100); + edit.SetNextFile(kBig + 200); + edit.SetLastSequence(kBig + 1000); + TestEncodeDecode(edit); + + std::string encoded, encoded2; + edit.EncodeTo(&encoded); + VersionEdit parsed; + Status s = parsed.DecodeFrom(encoded); + ASSERT_TRUE(s.ok()) << s.ToString(); + auto& new_files = parsed.GetNewFiles(); + ASSERT_TRUE(new_files[0].second.marked_for_compaction); + ASSERT_TRUE(!new_files[1].second.marked_for_compaction); + ASSERT_TRUE(new_files[2].second.marked_for_compaction); + ASSERT_EQ(3, new_files[0].second.fd.GetPathId()); + ASSERT_EQ(3, new_files[1].second.fd.GetPathId()); + ASSERT_EQ(0, new_files[2].second.fd.GetPathId()); +} + +TEST_F(VersionEditTest, ForwardCompatibleNewFile4) { + static const uint64_t kBig = 1ull << 50; + VersionEdit edit; + edit.AddFile(3, 300, 3, 100, InternalKey("foo", kBig + 500, kTypeValue), + InternalKey("zoo", kBig + 600, kTypeDeletion), kBig + 500, + kBig + 600, true); + edit.AddFile(4, 301, 3, 100, InternalKey("foo", kBig + 501, kTypeValue), + InternalKey("zoo", kBig + 601, kTypeDeletion), kBig + 501, + kBig + 601, false); + edit.DeleteFile(4, 700); + + edit.SetComparatorName("foo"); + edit.SetLogNumber(kBig + 100); + edit.SetNextFile(kBig + 200); + edit.SetLastSequence(kBig + 1000); + + std::string encoded; + + // Call back function to add extra customized builds. + bool first = true; + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "VersionEdit::EncodeTo:NewFile4:CustomizeFields", [&](void* arg) { + std::string* str = reinterpret_cast(arg); + PutVarint32(str, 33); + const std::string str1 = "random_string"; + PutLengthPrefixedSlice(str, str1); + if (first) { + first = false; + PutVarint32(str, 22); + const std::string str2 = "s"; + PutLengthPrefixedSlice(str, str2); + } + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + edit.EncodeTo(&encoded); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + VersionEdit parsed; + Status s = parsed.DecodeFrom(encoded); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_TRUE(!first); + auto& new_files = parsed.GetNewFiles(); + ASSERT_TRUE(new_files[0].second.marked_for_compaction); + ASSERT_TRUE(!new_files[1].second.marked_for_compaction); + ASSERT_EQ(3, new_files[0].second.fd.GetPathId()); + ASSERT_EQ(3, new_files[1].second.fd.GetPathId()); + ASSERT_EQ(1u, parsed.GetDeletedFiles().size()); +} + +TEST_F(VersionEditTest, NewFile4NotSupportedField) { + static const uint64_t kBig = 1ull << 50; + VersionEdit edit; + edit.AddFile(3, 300, 3, 100, InternalKey("foo", kBig + 500, kTypeValue), + InternalKey("zoo", kBig + 600, kTypeDeletion), kBig + 500, + kBig + 600, true); + + edit.SetComparatorName("foo"); + edit.SetLogNumber(kBig + 100); + edit.SetNextFile(kBig + 200); + edit.SetLastSequence(kBig + 1000); + + std::string encoded; + + // Call back function to add extra customized builds. + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "VersionEdit::EncodeTo:NewFile4:CustomizeFields", [&](void* arg) { + std::string* str = reinterpret_cast(arg); + const std::string str1 = "s"; + PutLengthPrefixedSlice(str, str1); + }); + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + edit.EncodeTo(&encoded); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + VersionEdit parsed; + Status s = parsed.DecodeFrom(encoded); + ASSERT_NOK(s); +} + +TEST_F(VersionEditTest, EncodeEmptyFile) { + VersionEdit edit; + edit.AddFile(0, 0, 0, 0, InternalKey(), InternalKey(), 0, 0, false); + std::string buffer; + ASSERT_TRUE(!edit.EncodeTo(&buffer)); +} + +TEST_F(VersionEditTest, ColumnFamilyTest) { VersionEdit edit; edit.SetColumnFamily(2); edit.AddColumnFamily("column_family"); @@ -60,5 +184,6 @@ TEST(VersionEditTest, ColumnFamilyTest) { } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/version_set.cc b/db/version_set.cc index 3a15458532e..782ebc263eb 100644 --- a/db/version_set.cc +++ b/db/version_set.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,43 +9,57 @@ #include "db/version_set.h" +#ifndef __STDC_FORMAT_MACROS #define __STDC_FORMAT_MACROS +#endif + #include +#include #include +#include #include #include -#include +#include #include #include -#include - -#include "db/filename.h" +#include "db/compaction.h" +#include "db/internal_stats.h" #include "db/log_reader.h" #include "db/log_writer.h" #include "db/memtable.h" #include "db/merge_context.h" +#include "db/merge_helper.h" +#include "db/pinned_iterators_manager.h" #include "db/table_cache.h" -#include "db/compaction.h" +#include "db/version_builder.h" +#include "monitoring/file_read_sample.h" +#include "monitoring/perf_context_imp.h" #include "rocksdb/env.h" #include "rocksdb/merge_operator.h" -#include "table/table_reader.h" -#include "table/merger.h" -#include "table/two_level_iterator.h" +#include "rocksdb/write_buffer_manager.h" #include "table/format.h" -#include "table/plain_table_factory.h" +#include "table/get_context.h" +#include "table/internal_iterator.h" +#include "table/merging_iterator.h" #include "table/meta_blocks.h" +#include "table/plain_table_factory.h" +#include "table/table_reader.h" +#include "table/two_level_iterator.h" #include "util/coding.h" -#include "util/logging.h" +#include "util/file_reader_writer.h" +#include "util/filename.h" #include "util/stop_watch.h" +#include "util/string_util.h" +#include "util/sync_point.h" namespace rocksdb { namespace { -// Find File in FileLevel data structure +// Find File in LevelFilesBrief data structure // Within an index range defined by left and right int FindFileInRange(const InternalKeyComparator& icmp, - const FileLevel& file_level, + const LevelFilesBrief& file_level, const Slice& key, uint32_t left, uint32_t right) { @@ -65,27 +79,6 @@ int FindFileInRange(const InternalKeyComparator& icmp, return right; } -bool NewestFirstBySeqNo(FileMetaData* a, FileMetaData* b) { - if (a->smallest_seqno != b->smallest_seqno) { - return a->smallest_seqno > b->smallest_seqno; - } - if (a->largest_seqno != b->largest_seqno) { - return a->largest_seqno > b->largest_seqno; - } - // Break ties by file number - return a->fd.GetNumber() > b->fd.GetNumber(); -} - -bool BySmallestKey(FileMetaData* a, FileMetaData* b, - const InternalKeyComparator* cmp) { - int r = cmp->Compare(a->smallest, b->smallest); - if (r != 0) { - return (r < 0); - } - // Break ties by file number - return (a->fd.GetNumber() < b->fd.GetNumber()); -} - // Class to help choose the next file to search for the particular key. // Searches and returns files level by level. // We can search level-by-level since entries never hop across @@ -94,23 +87,22 @@ bool BySmallestKey(FileMetaData* a, FileMetaData* b, // are MergeInProgress). class FilePicker { public: - FilePicker( - std::vector* files, - const Slice& user_key, - const Slice& ikey, - autovector* file_levels, - unsigned int num_levels, - FileIndexer* file_indexer, - const Comparator* user_comparator, - const InternalKeyComparator* internal_comparator) + FilePicker(std::vector* files, const Slice& user_key, + const Slice& ikey, autovector* file_levels, + unsigned int num_levels, FileIndexer* file_indexer, + const Comparator* user_comparator, + const InternalKeyComparator* internal_comparator) : num_levels_(num_levels), - curr_level_(-1), + curr_level_(static_cast(-1)), + returned_file_level_(static_cast(-1)), + hit_file_level_(static_cast(-1)), search_left_bound_(0), search_right_bound_(FileIndexer::kLevelMaxIndex), #ifndef NDEBUG files_(files), #endif - file_levels_(file_levels), + level_files_brief_(file_levels), + is_hit_file_last_in_level_(false), user_key_(user_key), ikey_(ikey), file_indexer_(file_indexer), @@ -120,8 +112,8 @@ class FilePicker { search_ended_ = !PrepareNextLevel(); if (!search_ended_) { // Prefetch Level 0 table data to avoid cache miss if possible. - for (unsigned int i = 0; i < (*file_levels_)[0].num_files; ++i) { - auto* r = (*file_levels_)[0].files[i].fd.table_reader; + for (unsigned int i = 0; i < (*level_files_brief_)[0].num_files; ++i) { + auto* r = (*level_files_brief_)[0].files[i].fd.table_reader; if (r) { r->Prepare(ikey); } @@ -129,17 +121,22 @@ class FilePicker { } } + int GetCurrentLevel() const { return curr_level_; } + FdWithKeyRange* GetNextFile() { while (!search_ended_) { // Loops over different levels. while (curr_index_in_curr_level_ < curr_file_level_->num_files) { // Loops over all files in current level. FdWithKeyRange* f = &curr_file_level_->files[curr_index_in_curr_level_]; + hit_file_level_ = curr_level_; + is_hit_file_last_in_level_ = + curr_index_in_curr_level_ == curr_file_level_->num_files - 1; int cmp_largest = -1; // Do key range filtering of files or/and fractional cascading if: // (1) not all the files are in level 0, or - // (2) there are more than 3 Level 0 files - // If there are only 3 or less level 0 files in the system, we skip + // (2) there are more than 3 current level files + // If there are only 3 or less current level files in the system, we skip // the key range filtering. In this case, more likely, the system is // highly tuned to minimize number of tables queried by each query, // so it is unlikely that key range filtering is more efficient than @@ -198,6 +195,7 @@ class FilePicker { } prev_file_ = f; #endif + returned_file_level_ = curr_level_; if (curr_level_ > 0 && cmp_largest < 0) { // No more files to search in this level. search_ended_ = !PrepareNextLevel(); @@ -213,17 +211,28 @@ class FilePicker { return nullptr; } + // getter for current file level + // for GET_HIT_L0, GET_HIT_L1 & GET_HIT_L2_AND_UP counts + unsigned int GetHitFileLevel() { return hit_file_level_; } + + // Returns true if the most recent "hit file" (i.e., one returned by + // GetNextFile()) is at the last index in its level. + bool IsHitFileLastInLevel() { return is_hit_file_last_in_level_; } + private: unsigned int num_levels_; unsigned int curr_level_; - int search_left_bound_; - int search_right_bound_; + unsigned int returned_file_level_; + unsigned int hit_file_level_; + int32_t search_left_bound_; + int32_t search_right_bound_; #ifndef NDEBUG std::vector* files_; #endif - autovector* file_levels_; + autovector* level_files_brief_; bool search_ended_; - FileLevel* curr_file_level_; + bool is_hit_file_last_in_level_; + LevelFilesBrief* curr_file_level_; unsigned int curr_index_in_curr_level_; unsigned int start_index_in_curr_level_; Slice user_key_; @@ -240,7 +249,7 @@ class FilePicker { bool PrepareNextLevel() { curr_level_++; while (curr_level_ < num_levels_) { - curr_file_level_ = &(*file_levels_)[curr_level_]; + curr_file_level_ = &(*level_files_brief_)[curr_level_]; if (curr_file_level_->num_files == 0) { // When current level is empty, the search bound generated from upper // level must be [0, -1] or [0, FileIndexer::kLevelMaxIndex] if it is @@ -273,14 +282,16 @@ class FilePicker { start_index = search_left_bound_; } else if (search_left_bound_ < search_right_bound_) { if (search_right_bound_ == FileIndexer::kLevelMaxIndex) { - search_right_bound_ = curr_file_level_->num_files - 1; + search_right_bound_ = + static_cast(curr_file_level_->num_files) - 1; } - start_index = FindFileInRange(*internal_comparator_, - *curr_file_level_, ikey_, - search_left_bound_, search_right_bound_); + start_index = + FindFileInRange(*internal_comparator_, *curr_file_level_, ikey_, + static_cast(search_left_bound_), + static_cast(search_right_bound_)); } else { // search_left_bound > search_right_bound, key does not exist in - // this level. Since no comparision is done in this level, it will + // this level. Since no comparison is done in this level, it will // need to search all files in the next level. search_left_bound_ = 0; search_right_bound_ = FileIndexer::kLevelMaxIndex; @@ -301,6 +312,8 @@ class FilePicker { }; } // anonymous namespace +VersionStorageInfo::~VersionStorageInfo() { delete[] files_; } + Version::~Version() { assert(refs_ == 0); @@ -309,34 +322,29 @@ Version::~Version() { next_->prev_ = prev_; // Drop references to files - for (int level = 0; level < num_levels_; level++) { - for (size_t i = 0; i < files_[level].size(); i++) { - FileMetaData* f = files_[level][i]; + for (int level = 0; level < storage_info_.num_levels_; level++) { + for (size_t i = 0; i < storage_info_.files_[level].size(); i++) { + FileMetaData* f = storage_info_.files_[level][i]; assert(f->refs > 0); f->refs--; if (f->refs <= 0) { - if (f->table_reader_handle) { - cfd_->table_cache()->ReleaseHandle(f->table_reader_handle); - f->table_reader_handle = nullptr; - } vset_->obsolete_files_.push_back(f); } } } - delete[] files_; } int FindFile(const InternalKeyComparator& icmp, - const FileLevel& file_level, + const LevelFilesBrief& file_level, const Slice& key) { - return FindFileInRange(icmp, file_level, key, 0, file_level.num_files); + return FindFileInRange(icmp, file_level, key, 0, + static_cast(file_level.num_files)); } -void DoGenerateFileLevel(FileLevel* file_level, +void DoGenerateLevelFilesBrief(LevelFilesBrief* file_level, const std::vector& files, Arena* arena) { assert(file_level); - assert(files.size() >= 0); assert(arena); size_t num = files.size(); @@ -357,6 +365,7 @@ void DoGenerateFileLevel(FileLevel* file_level, FdWithKeyRange& f = file_level->files[i]; f.fd = files[i]->fd; + f.file_metadata = files[i]; f.smallest_key = Slice(mem, smallest_size); f.largest_key = Slice(mem + smallest_size, largest_size); } @@ -379,7 +388,7 @@ static bool BeforeFile(const Comparator* ucmp, bool SomeFileOverlapsRange( const InternalKeyComparator& icmp, bool disjoint_sorted_files, - const FileLevel& file_level, + const LevelFilesBrief& file_level, const Slice* smallest_user_key, const Slice* largest_user_key) { const Comparator* ucmp = icmp.user_comparator(); @@ -401,7 +410,8 @@ bool SomeFileOverlapsRange( uint32_t index = 0; if (smallest_user_key != nullptr) { // Find the earliest possible internal key for smallest_user_key - InternalKey small(*smallest_user_key, kMaxSequenceNumber,kValueTypeForSeek); + InternalKey small; + small.SetMaxPossibleForUserKey(*smallest_user_key); index = FindFile(icmp, file_level, small.Encode()); } @@ -413,105 +423,161 @@ bool SomeFileOverlapsRange( return !BeforeFile(ucmp, largest_user_key, &file_level.files[index]); } +namespace { + // An internal iterator. For a given version/level pair, yields // information about the files in the level. For a given entry, key() // is the largest key that occurs in the file, and value() is an // 16-byte value containing the file number and file size, both // encoded using EncodeFixed64. -class Version::LevelFileNumIterator : public Iterator { +class LevelFileNumIterator : public InternalIterator { public: LevelFileNumIterator(const InternalKeyComparator& icmp, - const FileLevel* flevel) + const LevelFilesBrief* flevel, bool should_sample) : icmp_(icmp), flevel_(flevel), - index_(flevel->num_files), - current_value_(0, 0, 0) { // Marks as invalid - } - virtual bool Valid() const { - return index_ < flevel_->num_files; - } - virtual void Seek(const Slice& target) { + index_(static_cast(flevel->num_files)), + current_value_(0, 0, 0), // Marks as invalid + should_sample_(should_sample) {} + virtual bool Valid() const override { return index_ < flevel_->num_files; } + virtual void Seek(const Slice& target) override { index_ = FindFile(icmp_, *flevel_, target); } - virtual void SeekToFirst() { index_ = 0; } - virtual void SeekToLast() { - index_ = (flevel_->num_files == 0) ? 0 : flevel_->num_files - 1; + virtual void SeekForPrev(const Slice& target) override { + SeekForPrevImpl(target, &icmp_); } - virtual void Next() { + + virtual void SeekToFirst() override { index_ = 0; } + virtual void SeekToLast() override { + index_ = (flevel_->num_files == 0) + ? 0 + : static_cast(flevel_->num_files) - 1; + } + virtual void Next() override { assert(Valid()); index_++; } - virtual void Prev() { + virtual void Prev() override { assert(Valid()); if (index_ == 0) { - index_ = flevel_->num_files; // Marks as invalid + index_ = static_cast(flevel_->num_files); // Marks as invalid } else { index_--; } } - Slice key() const { + Slice key() const override { assert(Valid()); return flevel_->files[index_].largest_key; } - Slice value() const { + Slice value() const override { assert(Valid()); auto file_meta = flevel_->files[index_]; + if (should_sample_) { + sample_file_read_inc(file_meta.file_metadata); + } current_value_ = file_meta.fd; return Slice(reinterpret_cast(¤t_value_), sizeof(FileDescriptor)); } - virtual Status status() const { return Status::OK(); } + virtual Status status() const override { return Status::OK(); } + private: const InternalKeyComparator icmp_; - const FileLevel* flevel_; + const LevelFilesBrief* flevel_; uint32_t index_; mutable FileDescriptor current_value_; + bool should_sample_; }; -class Version::LevelFileIteratorState : public TwoLevelIteratorState { +class LevelFileIteratorState : public TwoLevelIteratorState { public: + // @param skip_filters Disables loading/accessing the filter block LevelFileIteratorState(TableCache* table_cache, - const ReadOptions& read_options, const EnvOptions& env_options, - const InternalKeyComparator& icomparator, bool for_compaction, - bool prefix_enabled) - : TwoLevelIteratorState(prefix_enabled), - table_cache_(table_cache), read_options_(read_options), - env_options_(env_options), icomparator_(icomparator), - for_compaction_(for_compaction) {} - - Iterator* NewSecondaryIterator(const Slice& meta_handle) override { + const ReadOptions& read_options, + const EnvOptions& env_options, + const InternalKeyComparator& icomparator, + HistogramImpl* file_read_hist, bool for_compaction, + bool prefix_enabled, bool skip_filters, int level, + RangeDelAggregator* range_del_agg) + : TwoLevelIteratorState(prefix_enabled), + table_cache_(table_cache), + read_options_(read_options), + env_options_(env_options), + icomparator_(icomparator), + file_read_hist_(file_read_hist), + for_compaction_(for_compaction), + skip_filters_(skip_filters), + level_(level), + range_del_agg_(range_del_agg) {} + + InternalIterator* NewSecondaryIterator(const Slice& meta_handle) override { if (meta_handle.size() != sizeof(FileDescriptor)) { - return NewErrorIterator( + return NewErrorInternalIterator( Status::Corruption("FileReader invoked with unexpected value")); - } else { - const FileDescriptor* fd = - reinterpret_cast(meta_handle.data()); - return table_cache_->NewIterator( - read_options_, env_options_, icomparator_, *fd, - nullptr /* don't need reference to table*/, for_compaction_); } + const FileDescriptor* fd = + reinterpret_cast(meta_handle.data()); + return table_cache_->NewIterator( + read_options_, env_options_, icomparator_, *fd, range_del_agg_, + nullptr /* don't need reference to table */, file_read_hist_, + for_compaction_, nullptr /* arena */, skip_filters_, level_); } bool PrefixMayMatch(const Slice& internal_key) override { return true; } + bool KeyReachedUpperBound(const Slice& internal_key) override { + return read_options_.iterate_upper_bound != nullptr && + icomparator_.user_comparator()->Compare( + ExtractUserKey(internal_key), + *read_options_.iterate_upper_bound) >= 0; + } + private: TableCache* table_cache_; const ReadOptions read_options_; const EnvOptions& env_options_; const InternalKeyComparator& icomparator_; + HistogramImpl* file_read_hist_; bool for_compaction_; + bool skip_filters_; + int level_; + RangeDelAggregator* range_del_agg_; +}; + +// A wrapper of version builder which references the current version in +// constructor and unref it in the destructor. +// Both of the constructor and destructor need to be called inside DB Mutex. +class BaseReferencedVersionBuilder { + public: + explicit BaseReferencedVersionBuilder(ColumnFamilyData* cfd) + : version_builder_(new VersionBuilder( + cfd->current()->version_set()->env_options(), cfd->table_cache(), + cfd->current()->storage_info(), cfd->ioptions()->info_log)), + version_(cfd->current()) { + version_->Ref(); + } + ~BaseReferencedVersionBuilder() { + delete version_builder_; + version_->Unref(); + } + VersionBuilder* version_builder() { return version_builder_; } + + private: + VersionBuilder* version_builder_; + Version* version_; }; +} // anonymous namespace Status Version::GetTableProperties(std::shared_ptr* tp, const FileMetaData* file_meta, - const std::string* fname) { + const std::string* fname) const { auto table_cache = cfd_->table_cache(); - auto options = cfd_->options(); + auto ioptions = cfd_->ioptions(); Status s = table_cache->GetTableProperties( - vset_->storage_options_, cfd_->internal_comparator(), file_meta->fd, + vset_->env_options_, cfd_->internal_comparator(), file_meta->fd, tp, true /* no io */); if (s.ok()) { return s; @@ -526,15 +592,15 @@ Status Version::GetTableProperties(std::shared_ptr* tp, // 2. Table is not present in table cache, we'll read the table properties // directly from the properties block in the file. std::unique_ptr file; + std::string file_name; if (fname != nullptr) { - s = options->env->NewRandomAccessFile( - *fname, &file, vset_->storage_options_); + file_name = *fname; } else { - s = options->env->NewRandomAccessFile( - TableFileName(vset_->options_->db_paths, file_meta->fd.GetNumber(), - file_meta->fd.GetPathId()), - &file, vset_->storage_options_); + file_name = + TableFileName(vset_->db_options_->db_paths, file_meta->fd.GetNumber(), + file_meta->fd.GetPathId()); } + s = ioptions->env->NewRandomAccessFile(file_name, &file, vset_->env_options_); if (!s.ok()) { return s; } @@ -542,33 +608,77 @@ Status Version::GetTableProperties(std::shared_ptr* tp, TableProperties* raw_table_properties; // By setting the magic number to kInvalidTableMagicNumber, we can by // pass the magic number check in the footer. + std::unique_ptr file_reader( + new RandomAccessFileReader(std::move(file), file_name)); s = ReadTableProperties( - file.get(), file_meta->fd.GetFileSize(), - Footer::kInvalidTableMagicNumber /* table's magic number */, - vset_->env_, options->info_log.get(), &raw_table_properties); + file_reader.get(), file_meta->fd.GetFileSize(), + Footer::kInvalidTableMagicNumber /* table's magic number */, *ioptions, &raw_table_properties); if (!s.ok()) { return s; } - RecordTick(options->statistics.get(), NUMBER_DIRECT_LOAD_TABLE_PROPERTIES); + RecordTick(ioptions->statistics, NUMBER_DIRECT_LOAD_TABLE_PROPERTIES); *tp = std::shared_ptr(raw_table_properties); return s; } Status Version::GetPropertiesOfAllTables(TablePropertiesCollection* props) { - for (int level = 0; level < num_levels_; level++) { - for (const auto& file_meta : files_[level]) { - auto fname = - TableFileName(vset_->options_->db_paths, file_meta->fd.GetNumber(), - file_meta->fd.GetPathId()); - // 1. If the table is already present in table cache, load table - // properties from there. - std::shared_ptr table_properties; - Status s = GetTableProperties(&table_properties, file_meta, &fname); - if (s.ok()) { - props->insert({fname, table_properties}); - } else { - return s; + Status s; + for (int level = 0; level < storage_info_.num_levels_; level++) { + s = GetPropertiesOfAllTables(props, level); + if (!s.ok()) { + return s; + } + } + + return Status::OK(); +} + +Status Version::GetPropertiesOfAllTables(TablePropertiesCollection* props, + int level) { + for (const auto& file_meta : storage_info_.files_[level]) { + auto fname = + TableFileName(vset_->db_options_->db_paths, file_meta->fd.GetNumber(), + file_meta->fd.GetPathId()); + // 1. If the table is already present in table cache, load table + // properties from there. + std::shared_ptr table_properties; + Status s = GetTableProperties(&table_properties, file_meta, &fname); + if (s.ok()) { + props->insert({fname, table_properties}); + } else { + return s; + } + } + + return Status::OK(); +} + +Status Version::GetPropertiesOfTablesInRange( + const Range* range, std::size_t n, TablePropertiesCollection* props) const { + for (int level = 0; level < storage_info_.num_non_empty_levels(); level++) { + for (decltype(n) i = 0; i < n; i++) { + // Convert user_key into a corresponding internal key. + InternalKey k1(range[i].start, kMaxSequenceNumber, kValueTypeForSeek); + InternalKey k2(range[i].limit, kMaxSequenceNumber, kValueTypeForSeek); + std::vector files; + storage_info_.GetOverlappingInputs(level, &k1, &k2, &files, -1, nullptr, + false); + for (const auto& file_meta : files) { + auto fname = + TableFileName(vset_->db_options_->db_paths, + file_meta->fd.GetNumber(), file_meta->fd.GetPathId()); + if (props->count(fname) == 0) { + // 1. If the table is already present in table cache, load table + // properties from there. + std::shared_ptr table_properties; + Status s = GetTableProperties(&table_properties, file_meta, &fname); + if (s.ok()) { + props->insert({fname, table_properties}); + } else { + return s; + } + } } } } @@ -576,271 +686,361 @@ Status Version::GetPropertiesOfAllTables(TablePropertiesCollection* props) { return Status::OK(); } +Status Version::GetAggregatedTableProperties( + std::shared_ptr* tp, int level) { + TablePropertiesCollection props; + Status s; + if (level < 0) { + s = GetPropertiesOfAllTables(&props); + } else { + s = GetPropertiesOfAllTables(&props, level); + } + if (!s.ok()) { + return s; + } + + auto* new_tp = new TableProperties(); + for (const auto& item : props) { + new_tp->Add(*item.second); + } + tp->reset(new_tp); + return Status::OK(); +} + size_t Version::GetMemoryUsageByTableReaders() { size_t total_usage = 0; - for (auto& file_level : file_levels_) { + for (auto& file_level : storage_info_.level_files_brief_) { for (size_t i = 0; i < file_level.num_files; i++) { total_usage += cfd_->table_cache()->GetMemoryUsageByTableReader( - vset_->storage_options_, cfd_->internal_comparator(), + vset_->env_options_, cfd_->internal_comparator(), file_level.files[i].fd); } } return total_usage; } -uint64_t Version::GetEstimatedActiveKeys() { - // Estimation will be not accurate when: - // (1) there is merge keys +void Version::GetColumnFamilyMetaData(ColumnFamilyMetaData* cf_meta) { + assert(cf_meta); + assert(cfd_); + + cf_meta->name = cfd_->GetName(); + cf_meta->size = 0; + cf_meta->file_count = 0; + cf_meta->levels.clear(); + + auto* ioptions = cfd_->ioptions(); + auto* vstorage = storage_info(); + + for (int level = 0; level < cfd_->NumberLevels(); level++) { + uint64_t level_size = 0; + cf_meta->file_count += vstorage->LevelFiles(level).size(); + std::vector files; + for (const auto& file : vstorage->LevelFiles(level)) { + uint32_t path_id = file->fd.GetPathId(); + std::string file_path; + if (path_id < ioptions->db_paths.size()) { + file_path = ioptions->db_paths[path_id].path; + } else { + assert(!ioptions->db_paths.empty()); + file_path = ioptions->db_paths.back().path; + } + files.emplace_back( + MakeTableFileName("", file->fd.GetNumber()), file_path, + file->fd.GetFileSize(), file->smallest_seqno, file->largest_seqno, + file->smallest.user_key().ToString(), + file->largest.user_key().ToString(), + file->stats.num_reads_sampled.load(std::memory_order_relaxed), + file->being_compacted); + level_size += file->fd.GetFileSize(); + } + cf_meta->levels.emplace_back( + level, level_size, std::move(files)); + cf_meta->size += level_size; + } +} + + +uint64_t VersionStorageInfo::GetEstimatedActiveKeys() const { + // Estimation will be inaccurate when: + // (1) there exist merge keys // (2) keys are directly overwritten // (3) deletion on non-existing keys - return num_non_deletions_ - num_deletions_; -} + // (4) low number of samples + if (current_num_samples_ == 0) { + return 0; + } -void Version::AddIterators(const ReadOptions& read_options, - const EnvOptions& soptions, - std::vector* iters) { - // Merge all level zero files together since they may overlap - for (size_t i = 0; i < file_levels_[0].num_files; i++) { - const auto& file = file_levels_[0].files[i]; - iters->push_back(cfd_->table_cache()->NewIterator( - read_options, soptions, cfd_->internal_comparator(), file.fd)); + if (current_num_non_deletions_ <= current_num_deletions_) { + return 0; } - // For levels > 0, we can use a concatenating iterator that sequentially - // walks through the non-overlapping files in the level, opening them - // lazily. - for (int level = 1; level < num_levels_; level++) { - if (file_levels_[level].num_files != 0) { - iters->push_back(NewTwoLevelIterator(new LevelFileIteratorState( - cfd_->table_cache(), read_options, soptions, - cfd_->internal_comparator(), false /* for_compaction */, - cfd_->options()->prefix_extractor != nullptr), - new LevelFileNumIterator(cfd_->internal_comparator(), - &file_levels_[level]))); - } + uint64_t est = current_num_non_deletions_ - current_num_deletions_; + + uint64_t file_count = 0; + for (int level = 0; level < num_levels_; ++level) { + file_count += files_[level].size(); + } + + if (current_num_samples_ < file_count) { + // casting to avoid overflowing + return + static_cast( + (est * static_cast(file_count) / current_num_samples_) + ); + } else { + return est; } } +double VersionStorageInfo::GetEstimatedCompressionRatioAtLevel( + int level) const { + assert(level < num_levels_); + uint64_t sum_file_size_bytes = 0; + uint64_t sum_data_size_bytes = 0; + for (auto* file_meta : files_[level]) { + sum_file_size_bytes += file_meta->fd.GetFileSize(); + sum_data_size_bytes += file_meta->raw_key_size + file_meta->raw_value_size; + } + if (sum_file_size_bytes == 0) { + return -1.0; + } + return static_cast(sum_data_size_bytes) / sum_file_size_bytes; +} + void Version::AddIterators(const ReadOptions& read_options, const EnvOptions& soptions, - MergeIteratorBuilder* merge_iter_builder) { - // Merge all level zero files together since they may overlap - for (size_t i = 0; i < file_levels_[0].num_files; i++) { - const auto& file = file_levels_[0].files[i]; - merge_iter_builder->AddIterator(cfd_->table_cache()->NewIterator( - read_options, soptions, cfd_->internal_comparator(), file.fd, nullptr, - false, merge_iter_builder->GetArena())); - } - - // For levels > 0, we can use a concatenating iterator that sequentially - // walks through the non-overlapping files in the level, opening them - // lazily. - for (int level = 1; level < num_levels_; level++) { - if (file_levels_[level].num_files != 0) { - merge_iter_builder->AddIterator(NewTwoLevelIterator( - new LevelFileIteratorState( - cfd_->table_cache(), read_options, soptions, - cfd_->internal_comparator(), false /* for_compaction */, - cfd_->options()->prefix_extractor != nullptr), - new LevelFileNumIterator(cfd_->internal_comparator(), - &file_levels_[level]), merge_iter_builder->GetArena())); - } - } -} - -// Callback from TableCache::Get() -enum SaverState { - kNotFound, - kFound, - kDeleted, - kCorrupt, - kMerge // saver contains the current merge result (the operands) -}; + MergeIteratorBuilder* merge_iter_builder, + RangeDelAggregator* range_del_agg) { + assert(storage_info_.finalized_); -namespace version_set { -struct Saver { - SaverState state; - const Comparator* ucmp; - Slice user_key; - bool* value_found; // Is value set correctly? Used by KeyMayExist - std::string* value; - const MergeOperator* merge_operator; - // the merge operations encountered; - MergeContext* merge_context; - Logger* logger; - Statistics* statistics; -}; -} // namespace version_set - -// Called from TableCache::Get and Table::Get when file/block in which -// key may exist are not there in TableCache/BlockCache respectively. In this -// case we can't guarantee that key does not exist and are not permitted to do -// IO to be certain.Set the status=kFound and value_found=false to let the -// caller know that key may exist but is not there in memory -static void MarkKeyMayExist(void* arg) { - version_set::Saver* s = reinterpret_cast(arg); - s->state = kFound; - if (s->value_found != nullptr) { - *(s->value_found) = false; - } -} - -static bool SaveValue(void* arg, const ParsedInternalKey& parsed_key, - const Slice& v) { - version_set::Saver* s = reinterpret_cast(arg); - MergeContext* merge_contex = s->merge_context; - std::string merge_result; // temporary area for merge results later - - assert(s != nullptr && merge_contex != nullptr); - - // TODO: Merge? - if (s->ucmp->Compare(parsed_key.user_key, s->user_key) == 0) { - // Key matches. Process it - switch (parsed_key.type) { - case kTypeValue: - if (kNotFound == s->state) { - s->state = kFound; - s->value->assign(v.data(), v.size()); - } else if (kMerge == s->state) { - assert(s->merge_operator != nullptr); - s->state = kFound; - if (!s->merge_operator->FullMerge(s->user_key, &v, - merge_contex->GetOperands(), - s->value, s->logger)) { - RecordTick(s->statistics, NUMBER_MERGE_FAILURES); - s->state = kCorrupt; - } - } else { - assert(false); - } - return false; - - case kTypeDeletion: - if (kNotFound == s->state) { - s->state = kDeleted; - } else if (kMerge == s->state) { - s->state = kFound; - if (!s->merge_operator->FullMerge(s->user_key, nullptr, - merge_contex->GetOperands(), - s->value, s->logger)) { - RecordTick(s->statistics, NUMBER_MERGE_FAILURES); - s->state = kCorrupt; - } - } else { - assert(false); - } - return false; + for (int level = 0; level < storage_info_.num_non_empty_levels(); level++) { + AddIteratorsForLevel(read_options, soptions, merge_iter_builder, level, + range_del_agg); + } +} + +void Version::AddIteratorsForLevel(const ReadOptions& read_options, + const EnvOptions& soptions, + MergeIteratorBuilder* merge_iter_builder, + int level, + RangeDelAggregator* range_del_agg) { + assert(storage_info_.finalized_); + if (level >= storage_info_.num_non_empty_levels()) { + // This is an empty level + return; + } else if (storage_info_.LevelFilesBrief(level).num_files == 0) { + // No files in this level + return; + } - case kTypeMerge: - assert(s->state == kNotFound || s->state == kMerge); - s->state = kMerge; - merge_contex->PushOperand(v); - return true; + bool should_sample = should_sample_file_read(); - default: - assert(false); - break; + auto* arena = merge_iter_builder->GetArena(); + if (level == 0) { + // Merge all level zero files together since they may overlap + for (size_t i = 0; i < storage_info_.LevelFilesBrief(0).num_files; i++) { + const auto& file = storage_info_.LevelFilesBrief(0).files[i]; + merge_iter_builder->AddIterator(cfd_->table_cache()->NewIterator( + read_options, soptions, cfd_->internal_comparator(), file.fd, + range_del_agg, nullptr, cfd_->internal_stats()->GetFileReadHist(0), + false, arena, false /* skip_filters */, 0 /* level */)); + } + if (should_sample) { + // Count ones for every L0 files. This is done per iterator creation + // rather than Seek(), while files in other levels are recored per seek. + // If users execute one range query per iterator, there may be some + // discrepancy here. + for (FileMetaData* meta : storage_info_.LevelFiles(0)) { + sample_file_read_inc(meta); + } } + } else { + // For levels > 0, we can use a concatenating iterator that sequentially + // walks through the non-overlapping files in the level, opening them + // lazily. + auto* mem = arena->AllocateAligned(sizeof(LevelFileIteratorState)); + auto* state = new (mem) + LevelFileIteratorState(cfd_->table_cache(), read_options, soptions, + cfd_->internal_comparator(), + cfd_->internal_stats()->GetFileReadHist(level), + false /* for_compaction */, + cfd_->ioptions()->prefix_extractor != nullptr, + IsFilterSkipped(level), level, range_del_agg); + mem = arena->AllocateAligned(sizeof(LevelFileNumIterator)); + auto* first_level_iter = new (mem) LevelFileNumIterator( + cfd_->internal_comparator(), &storage_info_.LevelFilesBrief(level), + should_sample_file_read()); + merge_iter_builder->AddIterator( + NewTwoLevelIterator(state, first_level_iter, arena, false)); } +} - // s->state could be Corrupt, merge or notfound +void Version::AddRangeDelIteratorsForLevel( + const ReadOptions& read_options, const EnvOptions& soptions, int level, + std::vector* range_del_iters) { + range_del_iters->clear(); + for (size_t i = 0; i < storage_info_.LevelFilesBrief(level).num_files; i++) { + const auto& file = storage_info_.LevelFilesBrief(level).files[i]; + auto* range_del_iter = cfd_->table_cache()->NewRangeTombstoneIterator( + read_options, soptions, cfd_->internal_comparator(), file.fd, + cfd_->internal_stats()->GetFileReadHist(level), + false /* skip_filters */, level); + if (range_del_iter != nullptr) { + range_del_iters->push_back(range_del_iter); + } + } +} - return false; +VersionStorageInfo::VersionStorageInfo( + const InternalKeyComparator* internal_comparator, + const Comparator* user_comparator, int levels, + CompactionStyle compaction_style, VersionStorageInfo* ref_vstorage, + bool _force_consistency_checks) + : internal_comparator_(internal_comparator), + user_comparator_(user_comparator), + // cfd is nullptr if Version is dummy + num_levels_(levels), + num_non_empty_levels_(0), + file_indexer_(user_comparator), + compaction_style_(compaction_style), + files_(new std::vector[num_levels_]), + base_level_(num_levels_ == 1 ? -1 : 1), + files_by_compaction_pri_(num_levels_), + level0_non_overlapping_(false), + next_file_to_compact_by_size_(num_levels_), + compaction_score_(num_levels_), + compaction_level_(num_levels_), + l0_delay_trigger_count_(0), + accumulated_file_size_(0), + accumulated_raw_key_size_(0), + accumulated_raw_value_size_(0), + accumulated_num_non_deletions_(0), + accumulated_num_deletions_(0), + current_num_non_deletions_(0), + current_num_deletions_(0), + current_num_samples_(0), + estimated_compaction_needed_bytes_(0), + finalized_(false), + force_consistency_checks_(_force_consistency_checks) { + if (ref_vstorage != nullptr) { + accumulated_file_size_ = ref_vstorage->accumulated_file_size_; + accumulated_raw_key_size_ = ref_vstorage->accumulated_raw_key_size_; + accumulated_raw_value_size_ = ref_vstorage->accumulated_raw_value_size_; + accumulated_num_non_deletions_ = + ref_vstorage->accumulated_num_non_deletions_; + accumulated_num_deletions_ = ref_vstorage->accumulated_num_deletions_; + current_num_non_deletions_ = ref_vstorage->current_num_non_deletions_; + current_num_deletions_ = ref_vstorage->current_num_deletions_; + current_num_samples_ = ref_vstorage->current_num_samples_; + } } -Version::Version(ColumnFamilyData* cfd, VersionSet* vset, +Version::Version(ColumnFamilyData* column_family_data, VersionSet* vset, uint64_t version_number) - : cfd_(cfd), - internal_comparator_((cfd == nullptr) ? nullptr - : &cfd->internal_comparator()), - user_comparator_( - (cfd == nullptr) ? nullptr : internal_comparator_->user_comparator()), - table_cache_((cfd == nullptr) ? nullptr : cfd->table_cache()), - merge_operator_((cfd == nullptr) ? nullptr - : cfd->options()->merge_operator.get()), - info_log_((cfd == nullptr) ? nullptr : cfd->options()->info_log.get()), - db_statistics_((cfd == nullptr) ? nullptr - : cfd->options()->statistics.get()), - // cfd is nullptr if Version is dummy - num_levels_(cfd == nullptr ? 0 : cfd->NumberLevels()), - num_non_empty_levels_(num_levels_), - file_indexer_(cfd == nullptr - ? nullptr - : cfd->internal_comparator().user_comparator()), + : env_(vset->env_), + cfd_(column_family_data), + info_log_((cfd_ == nullptr) ? nullptr : cfd_->ioptions()->info_log), + db_statistics_((cfd_ == nullptr) ? nullptr + : cfd_->ioptions()->statistics), + table_cache_((cfd_ == nullptr) ? nullptr : cfd_->table_cache()), + merge_operator_((cfd_ == nullptr) ? nullptr + : cfd_->ioptions()->merge_operator), + storage_info_( + (cfd_ == nullptr) ? nullptr : &cfd_->internal_comparator(), + (cfd_ == nullptr) ? nullptr : cfd_->user_comparator(), + cfd_ == nullptr ? 0 : cfd_->NumberLevels(), + cfd_ == nullptr ? kCompactionStyleLevel + : cfd_->ioptions()->compaction_style, + (cfd_ == nullptr || cfd_->current() == nullptr) + ? nullptr + : cfd_->current()->storage_info(), + cfd_ == nullptr ? false : cfd_->ioptions()->force_consistency_checks), vset_(vset), next_(this), prev_(this), refs_(0), - files_(new std::vector[num_levels_]), - files_by_size_(num_levels_), - next_file_to_compact_by_size_(num_levels_), - compaction_score_(num_levels_), - compaction_level_(num_levels_), - version_number_(version_number), - total_file_size_(0), - total_raw_key_size_(0), - total_raw_value_size_(0), - num_non_deletions_(0), - num_deletions_(0) { - if (cfd != nullptr && cfd->current() != nullptr) { - total_file_size_ = cfd->current()->total_file_size_; - total_raw_key_size_ = cfd->current()->total_raw_key_size_; - total_raw_value_size_ = cfd->current()->total_raw_value_size_; - num_non_deletions_ = cfd->current()->num_non_deletions_; - num_deletions_ = cfd->current()->num_deletions_; - } -} - -void Version::Get(const ReadOptions& options, - const LookupKey& k, - std::string* value, - Status* status, + version_number_(version_number) {} + +void Version::Get(const ReadOptions& read_options, const LookupKey& k, + PinnableSlice* value, Status* status, MergeContext* merge_context, - bool* value_found) { + RangeDelAggregator* range_del_agg, bool* value_found, + bool* key_exists, SequenceNumber* seq, bool* is_blob) { Slice ikey = k.internal_key(); Slice user_key = k.user_key(); assert(status->ok() || status->IsMergeInProgress()); - version_set::Saver saver; - saver.state = status->ok()? kNotFound : kMerge; - saver.ucmp = user_comparator_; - saver.user_key = user_key; - saver.value_found = value_found; - saver.value = value; - saver.merge_operator = merge_operator_; - saver.merge_context = merge_context; - saver.logger = info_log_; - saver.statistics = db_statistics_; - - FilePicker fp(files_, user_key, ikey, &file_levels_, num_non_empty_levels_, - &file_indexer_, user_comparator_, internal_comparator_); + + if (key_exists != nullptr) { + // will falsify below if not found + *key_exists = true; + } + + PinnedIteratorsManager pinned_iters_mgr; + GetContext get_context( + user_comparator(), merge_operator_, info_log_, db_statistics_, + status->ok() ? GetContext::kNotFound : GetContext::kMerge, user_key, + value, value_found, merge_context, range_del_agg, this->env_, seq, + merge_operator_ ? &pinned_iters_mgr : nullptr, is_blob); + + // Pin blocks that we read to hold merge operands + if (merge_operator_) { + pinned_iters_mgr.StartPinning(); + } + + FilePicker fp( + storage_info_.files_, user_key, ikey, &storage_info_.level_files_brief_, + storage_info_.num_non_empty_levels_, &storage_info_.file_indexer_, + user_comparator(), internal_comparator()); FdWithKeyRange* f = fp.GetNextFile(); while (f != nullptr) { - *status = table_cache_->Get(options, *internal_comparator_, f->fd, ikey, - &saver, SaveValue, MarkKeyMayExist); + if (get_context.sample()) { + sample_file_read_inc(f->file_metadata); + } + *status = table_cache_->Get( + read_options, *internal_comparator(), f->fd, ikey, &get_context, + cfd_->internal_stats()->GetFileReadHist(fp.GetHitFileLevel()), + IsFilterSkipped(static_cast(fp.GetHitFileLevel()), + fp.IsHitFileLastInLevel()), + fp.GetCurrentLevel()); // TODO: examine the behavior for corrupted key if (!status->ok()) { return; } - switch (saver.state) { - case kNotFound: - break; // Keep searching in other files - case kFound: + switch (get_context.State()) { + case GetContext::kNotFound: + // Keep searching in other files + break; + case GetContext::kFound: + if (fp.GetHitFileLevel() == 0) { + RecordTick(db_statistics_, GET_HIT_L0); + } else if (fp.GetHitFileLevel() == 1) { + RecordTick(db_statistics_, GET_HIT_L1); + } else if (fp.GetHitFileLevel() >= 2) { + RecordTick(db_statistics_, GET_HIT_L2_AND_UP); + } return; - case kDeleted: - *status = Status::NotFound(); // Use empty error message for speed + case GetContext::kDeleted: + // Use empty error message for speed + *status = Status::NotFound(); return; - case kCorrupt: + case GetContext::kCorrupt: *status = Status::Corruption("corrupted key for ", user_key); return; - case kMerge: + case GetContext::kMerge: break; + case GetContext::kBlobIndex: + ROCKS_LOG_ERROR(info_log_, "Encounter unexpected blob index."); + *status = Status::NotSupported( + "Encounter unexpected blob index. Please open DB with " + "rocksdb::blob_db::BlobDB instead."); + return; } f = fp.GetNextFile(); } - if (kMerge == saver.state) { + if (GetContext::kMerge == get_context.State()) { if (!merge_operator_) { *status = Status::InvalidArgument( "merge_operator is not properly initialized."); @@ -848,47 +1048,63 @@ void Version::Get(const ReadOptions& options, } // merge_operands are in saver and we hit the beginning of the key history // do a final merge of nullptr and operands; - if (merge_operator_->FullMerge(user_key, nullptr, - saver.merge_context->GetOperands(), value, - info_log_)) { - *status = Status::OK(); - } else { - RecordTick(db_statistics_, NUMBER_MERGE_FAILURES); - *status = Status::Corruption("could not perform end-of-key merge for ", - user_key); + std::string* str_value = value != nullptr ? value->GetSelf() : nullptr; + *status = MergeHelper::TimedFullMerge( + merge_operator_, user_key, nullptr, merge_context->GetOperands(), + str_value, info_log_, db_statistics_, env_, + nullptr /* result_operand */, true); + if (LIKELY(value != nullptr)) { + value->PinSelf(); } } else { + if (key_exists != nullptr) { + *key_exists = false; + } *status = Status::NotFound(); // Use an empty error message for speed } } -void Version::GenerateFileLevels() { - file_levels_.resize(num_non_empty_levels_); +bool Version::IsFilterSkipped(int level, bool is_file_last_in_level) { + // Reaching the bottom level implies misses at all upper levels, so we'll + // skip checking the filters when we predict a hit. + return cfd_->ioptions()->optimize_filters_for_hits && + (level > 0 || is_file_last_in_level) && + level == storage_info_.num_non_empty_levels() - 1; +} + +void VersionStorageInfo::GenerateLevelFilesBrief() { + level_files_brief_.resize(num_non_empty_levels_); for (int level = 0; level < num_non_empty_levels_; level++) { - DoGenerateFileLevel(&file_levels_[level], files_[level], &arena_); + DoGenerateLevelFilesBrief( + &level_files_brief_[level], files_[level], &arena_); } } -void Version::PrepareApply(std::vector& size_being_compacted) { - UpdateTemporaryStats(); - ComputeCompactionScore(size_being_compacted); - UpdateFilesBySize(); - UpdateNumNonEmptyLevels(); - file_indexer_.UpdateIndex(&arena_, num_non_empty_levels_, files_); - GenerateFileLevels(); +void Version::PrepareApply( + const MutableCFOptions& mutable_cf_options, + bool update_stats) { + UpdateAccumulatedStats(update_stats); + storage_info_.UpdateNumNonEmptyLevels(); + storage_info_.CalculateBaseBytes(*cfd_->ioptions(), mutable_cf_options); + storage_info_.UpdateFilesByCompactionPri(cfd_->ioptions()->compaction_pri); + storage_info_.GenerateFileIndexer(); + storage_info_.GenerateLevelFilesBrief(); + storage_info_.GenerateLevel0NonOverlapping(); } bool Version::MaybeInitializeFileMetaData(FileMetaData* file_meta) { - if (file_meta->init_stats_from_file) { + if (file_meta->init_stats_from_file || + file_meta->compensated_file_size > 0) { return false; } std::shared_ptr tp; Status s = GetTableProperties(&tp, file_meta); file_meta->init_stats_from_file = true; if (!s.ok()) { - Log(vset_->options_->info_log, - "Unable to load table properties for file %" PRIu64 " --- %s\n", - file_meta->fd.GetNumber(), s.ToString().c_str()); + ROCKS_LOG_ERROR(vset_->db_options_->info_log, + "Unable to load table properties for file %" PRIu64 + " --- %s\n", + file_meta->fd.GetNumber(), s.ToString().c_str()); return false; } if (tp.get() == nullptr) return false; @@ -900,54 +1116,246 @@ bool Version::MaybeInitializeFileMetaData(FileMetaData* file_meta) { return true; } -void Version::UpdateTemporaryStats() { - static const int kDeletionWeightOnCompaction = 2; +void VersionStorageInfo::UpdateAccumulatedStats(FileMetaData* file_meta) { + assert(file_meta->init_stats_from_file); + accumulated_file_size_ += file_meta->fd.GetFileSize(); + accumulated_raw_key_size_ += file_meta->raw_key_size; + accumulated_raw_value_size_ += file_meta->raw_value_size; + accumulated_num_non_deletions_ += + file_meta->num_entries - file_meta->num_deletions; + accumulated_num_deletions_ += file_meta->num_deletions; + + current_num_non_deletions_ += + file_meta->num_entries - file_meta->num_deletions; + current_num_deletions_ += file_meta->num_deletions; + current_num_samples_++; +} - // incrementally update the average value size by - // including newly added files into the global stats - int init_count = 0; - int total_count = 0; - for (int level = 0; level < num_levels_; level++) { - for (auto* file_meta : files_[level]) { - if (MaybeInitializeFileMetaData(file_meta)) { - // each FileMeta will be initialized only once. - total_file_size_ += file_meta->fd.GetFileSize(); - total_raw_key_size_ += file_meta->raw_key_size; - total_raw_value_size_ += file_meta->raw_value_size; - num_non_deletions_ += - file_meta->num_entries - file_meta->num_deletions; - num_deletions_ += file_meta->num_deletions; - init_count++; +void VersionStorageInfo::RemoveCurrentStats(FileMetaData* file_meta) { + if (file_meta->init_stats_from_file) { + current_num_non_deletions_ -= + file_meta->num_entries - file_meta->num_deletions; + current_num_deletions_ -= file_meta->num_deletions; + current_num_samples_--; + } +} + +void Version::UpdateAccumulatedStats(bool update_stats) { + if (update_stats) { + // maximum number of table properties loaded from files. + const int kMaxInitCount = 20; + int init_count = 0; + // here only the first kMaxInitCount files which haven't been + // initialized from file will be updated with num_deletions. + // The motivation here is to cap the maximum I/O per Version creation. + // The reason for choosing files from lower-level instead of higher-level + // is that such design is able to propagate the initialization from + // lower-level to higher-level: When the num_deletions of lower-level + // files are updated, it will make the lower-level files have accurate + // compensated_file_size, making lower-level to higher-level compaction + // will be triggered, which creates higher-level files whose num_deletions + // will be updated here. + for (int level = 0; + level < storage_info_.num_levels_ && init_count < kMaxInitCount; + ++level) { + for (auto* file_meta : storage_info_.files_[level]) { + if (MaybeInitializeFileMetaData(file_meta)) { + // each FileMeta will be initialized only once. + storage_info_.UpdateAccumulatedStats(file_meta); + // when option "max_open_files" is -1, all the file metadata has + // already been read, so MaybeInitializeFileMetaData() won't incur + // any I/O cost. "max_open_files=-1" means that the table cache passed + // to the VersionSet and then to the ColumnFamilySet has a size of + // TableCache::kInfiniteCapacity + if (vset_->GetColumnFamilySet()->get_table_cache()->GetCapacity() == + TableCache::kInfiniteCapacity) { + continue; + } + if (++init_count >= kMaxInitCount) { + break; + } + } + } + } + // In case all sampled-files contain only deletion entries, then we + // load the table-property of a file in higher-level to initialize + // that value. + for (int level = storage_info_.num_levels_ - 1; + storage_info_.accumulated_raw_value_size_ == 0 && level >= 0; + --level) { + for (int i = static_cast(storage_info_.files_[level].size()) - 1; + storage_info_.accumulated_raw_value_size_ == 0 && i >= 0; --i) { + if (MaybeInitializeFileMetaData(storage_info_.files_[level][i])) { + storage_info_.UpdateAccumulatedStats(storage_info_.files_[level][i]); + } } - total_count++; } } + storage_info_.ComputeCompensatedSizes(); +} + +void VersionStorageInfo::ComputeCompensatedSizes() { + static const int kDeletionWeightOnCompaction = 2; uint64_t average_value_size = GetAverageValueSize(); // compute the compensated size for (int level = 0; level < num_levels_; level++) { for (auto* file_meta : files_[level]) { // Here we only compute compensated_file_size for those file_meta - // which compensated_file_size is uninitialized (== 0). + // which compensated_file_size is uninitialized (== 0). This is true only + // for files that have been created right now and no other thread has + // access to them. That's why we can safely mutate compensated_file_size. if (file_meta->compensated_file_size == 0) { - file_meta->compensated_file_size = file_meta->fd.GetFileSize() + - file_meta->num_deletions * average_value_size * - kDeletionWeightOnCompaction; + file_meta->compensated_file_size = file_meta->fd.GetFileSize(); + // Here we only boost the size of deletion entries of a file only + // when the number of deletion entries is greater than the number of + // non-deletion entries in the file. The motivation here is that in + // a stable workload, the number of deletion entries should be roughly + // equal to the number of non-deletion entries. If we compensate the + // size of deletion entries in a stable workload, the deletion + // compensation logic might introduce unwanted effet which changes the + // shape of LSM tree. + if (file_meta->num_deletions * 2 >= file_meta->num_entries) { + file_meta->compensated_file_size += + (file_meta->num_deletions * 2 - file_meta->num_entries) * + average_value_size * kDeletionWeightOnCompaction; + } } } } } -void Version::ComputeCompactionScore( - std::vector& size_being_compacted) { - double max_score = 0; - int max_score_level = 0; +int VersionStorageInfo::MaxInputLevel() const { + if (compaction_style_ == kCompactionStyleLevel) { + return num_levels() - 2; + } + return 0; +} + +int VersionStorageInfo::MaxOutputLevel(bool allow_ingest_behind) const { + if (allow_ingest_behind) { + assert(num_levels() > 1); + return num_levels() - 2; + } + return num_levels() - 1; +} + +void VersionStorageInfo::EstimateCompactionBytesNeeded( + const MutableCFOptions& mutable_cf_options) { + // Only implemented for level-based compaction + if (compaction_style_ != kCompactionStyleLevel) { + estimated_compaction_needed_bytes_ = 0; + return; + } + + // Start from Level 0, if level 0 qualifies compaction to level 1, + // we estimate the size of compaction. + // Then we move on to the next level and see whether it qualifies compaction + // to the next level. The size of the level is estimated as the actual size + // on the level plus the input bytes from the previous level if there is any. + // If it exceeds, take the exceeded bytes as compaction input and add the size + // of the compaction size to tatal size. + // We keep doing it to Level 2, 3, etc, until the last level and return the + // accumulated bytes. + + uint64_t bytes_compact_to_next_level = 0; + uint64_t level_size = 0; + for (auto* f : files_[0]) { + level_size += f->fd.GetFileSize(); + } + // Level 0 + bool level0_compact_triggered = false; + if (static_cast(files_[0].size()) >= + mutable_cf_options.level0_file_num_compaction_trigger || + level_size >= mutable_cf_options.max_bytes_for_level_base) { + level0_compact_triggered = true; + estimated_compaction_needed_bytes_ = level_size; + bytes_compact_to_next_level = level_size; + } else { + estimated_compaction_needed_bytes_ = 0; + } + + // Level 1 and up. + uint64_t bytes_next_level = 0; + for (int level = base_level(); level <= MaxInputLevel(); level++) { + level_size = 0; + if (bytes_next_level > 0) { +#ifndef NDEBUG + uint64_t level_size2 = 0; + for (auto* f : files_[level]) { + level_size2 += f->fd.GetFileSize(); + } + assert(level_size2 == bytes_next_level); +#endif + level_size = bytes_next_level; + bytes_next_level = 0; + } else { + for (auto* f : files_[level]) { + level_size += f->fd.GetFileSize(); + } + } + if (level == base_level() && level0_compact_triggered) { + // Add base level size to compaction if level0 compaction triggered. + estimated_compaction_needed_bytes_ += level_size; + } + // Add size added by previous compaction + level_size += bytes_compact_to_next_level; + bytes_compact_to_next_level = 0; + uint64_t level_target = MaxBytesForLevel(level); + if (level_size > level_target) { + bytes_compact_to_next_level = level_size - level_target; + // Estimate the actual compaction fan-out ratio as size ratio between + // the two levels. + + assert(bytes_next_level == 0); + if (level + 1 < num_levels_) { + for (auto* f : files_[level + 1]) { + bytes_next_level += f->fd.GetFileSize(); + } + } + if (bytes_next_level > 0) { + assert(level_size > 0); + estimated_compaction_needed_bytes_ += static_cast( + static_cast(bytes_compact_to_next_level) * + (static_cast(bytes_next_level) / + static_cast(level_size) + + 1)); + } + } + } +} - int max_input_level = - cfd_->compaction_picker()->MaxInputLevel(NumberLevels()); +namespace { +uint32_t GetExpiredTtlFilesCount(const ImmutableCFOptions& ioptions, + const std::vector& files) { + uint32_t ttl_expired_files_count = 0; + + int64_t _current_time; + auto status = ioptions.env->GetCurrentTime(&_current_time); + if (status.ok()) { + const uint64_t current_time = static_cast(_current_time); + for (auto f : files) { + if (!f->being_compacted && f->fd.table_reader != nullptr && + f->fd.table_reader->GetTableProperties() != nullptr) { + auto creation_time = + f->fd.table_reader->GetTableProperties()->creation_time; + if (creation_time > 0 && + creation_time < + (current_time - ioptions.compaction_options_fifo.ttl)) { + ttl_expired_files_count++; + } + } + } + } + return ttl_expired_files_count; +} +} // anonymous namespace - for (int level = 0; level <= max_input_level; level++) { +void VersionStorageInfo::ComputeCompactionScore( + const ImmutableCFOptions& immutable_cf_options, + const MutableCFOptions& mutable_cf_options) { + for (int level = 0; level <= MaxInputLevel(); level++) { double score; if (level == 0) { // We treat level-0 specially by bounding the number of files @@ -961,49 +1369,72 @@ void Version::ComputeCompactionScore( // file size is small (perhaps because of a small write-buffer // setting, or very high compression ratios, or lots of // overwrites/deletions). - int numfiles = 0; + int num_sorted_runs = 0; uint64_t total_size = 0; - for (unsigned int i = 0; i < files_[level].size(); i++) { - if (!files_[level][i]->being_compacted) { - total_size += files_[level][i]->compensated_file_size; - numfiles++; + for (auto* f : files_[level]) { + if (!f->being_compacted) { + total_size += f->compensated_file_size; + num_sorted_runs++; } } - if (cfd_->options()->compaction_style == kCompactionStyleFIFO) { - score = static_cast(total_size) / - cfd_->options()->compaction_options_fifo.max_table_files_size; - } else if (numfiles >= cfd_->options()->level0_stop_writes_trigger) { - // If we are slowing down writes, then we better compact that first - score = 1000000; - } else if (numfiles >= cfd_->options()->level0_slowdown_writes_trigger) { - score = 10000; + if (compaction_style_ == kCompactionStyleUniversal) { + // For universal compaction, we use level0 score to indicate + // compaction score for the whole DB. Adding other levels as if + // they are L0 files. + for (int i = 1; i < num_levels(); i++) { + if (!files_[i].empty() && !files_[i][0]->being_compacted) { + num_sorted_runs++; + } + } + } + + if (compaction_style_ == kCompactionStyleFIFO) { + score = + static_cast(total_size) / + immutable_cf_options.compaction_options_fifo.max_table_files_size; + if (immutable_cf_options.compaction_options_fifo.allow_compaction) { + score = std::max( + static_cast(num_sorted_runs) / + mutable_cf_options.level0_file_num_compaction_trigger, + score); + } + if (immutable_cf_options.compaction_options_fifo.ttl > 0) { + score = std::max(static_cast(GetExpiredTtlFilesCount( + immutable_cf_options, files_[level])), + score); + } + } else { - score = static_cast(numfiles) / - cfd_->options()->level0_file_num_compaction_trigger; + score = static_cast(num_sorted_runs) / + mutable_cf_options.level0_file_num_compaction_trigger; + if (compaction_style_ == kCompactionStyleLevel && num_levels() > 1) { + // Level-based involves L0->L0 compactions that can lead to oversized + // L0 files. Take into account size as well to avoid later giant + // compactions to the base level. + score = std::max( + score, static_cast(total_size) / + mutable_cf_options.max_bytes_for_level_base); + } } } else { // Compute the ratio of current size to size limit. - const uint64_t level_bytes = - TotalCompensatedFileSize(files_[level]) - size_being_compacted[level]; - score = static_cast(level_bytes) / - cfd_->compaction_picker()->MaxBytesForLevel(level); - if (max_score < score) { - max_score = score; - max_score_level = level; + uint64_t level_bytes_no_compacting = 0; + for (auto f : files_[level]) { + if (!f->being_compacted) { + level_bytes_no_compacting += f->compensated_file_size; + } } + score = static_cast(level_bytes_no_compacting) / + MaxBytesForLevel(level); } compaction_level_[level] = level; compaction_score_[level] = score; } - // update the max compaction score in levels 1 to n-1 - max_compaction_score_ = max_score; - max_compaction_score_level_ = max_score_level; - // sort all the levels based on their score. Higher scores get listed // first. Use bubble sort because the number of entries are small. - for (int i = 0; i < NumberLevels() - 2; i++) { - for (int j = i + 1; j < NumberLevels() - 1; j++) { + for (int i = 0; i < num_levels() - 2; i++) { + for (int j = i + 1; j < num_levels() - 1; j++) { if (compaction_score_[i] < compaction_score_[j]) { double score = compaction_score_[i]; int level = compaction_level_[i]; @@ -1014,19 +1445,120 @@ void Version::ComputeCompactionScore( } } } + ComputeFilesMarkedForCompaction(); + EstimateCompactionBytesNeeded(mutable_cf_options); +} + +void VersionStorageInfo::ComputeFilesMarkedForCompaction() { + files_marked_for_compaction_.clear(); + int last_qualify_level = 0; + + // Do not include files from the last level with data + // If table properties collector suggests a file on the last level, + // we should not move it to a new level. + for (int level = num_levels() - 1; level >= 1; level--) { + if (!files_[level].empty()) { + last_qualify_level = level - 1; + break; + } + } + + for (int level = 0; level <= last_qualify_level; level++) { + for (auto* f : files_[level]) { + if (!f->being_compacted && f->marked_for_compaction) { + files_marked_for_compaction_.emplace_back(level, f); + } + } + } } namespace { + +// used to sort files by size +struct Fsize { + size_t index; + FileMetaData* file; +}; + // Compator that is used to sort files based on their size // In normal mode: descending size -bool CompareCompensatedSizeDescending(const Version::Fsize& first, - const Version::Fsize& second) { +bool CompareCompensatedSizeDescending(const Fsize& first, const Fsize& second) { return (first.file->compensated_file_size > second.file->compensated_file_size); } } // anonymous namespace -void Version::UpdateNumNonEmptyLevels() { +void VersionStorageInfo::AddFile(int level, FileMetaData* f, Logger* info_log) { + auto* level_files = &files_[level]; + // Must not overlap +#ifndef NDEBUG + if (level > 0 && !level_files->empty() && + internal_comparator_->Compare( + (*level_files)[level_files->size() - 1]->largest, f->smallest) >= 0) { + auto* f2 = (*level_files)[level_files->size() - 1]; + if (info_log != nullptr) { + Error(info_log, "Adding new file %" PRIu64 + " range (%s, %s) to level %d but overlapping " + "with existing file %" PRIu64 " %s %s", + f->fd.GetNumber(), f->smallest.DebugString(true).c_str(), + f->largest.DebugString(true).c_str(), level, f2->fd.GetNumber(), + f2->smallest.DebugString(true).c_str(), + f2->largest.DebugString(true).c_str()); + LogFlush(info_log); + } + assert(false); + } +#endif + f->refs++; + level_files->push_back(f); +} + +// Version::PrepareApply() need to be called before calling the function, or +// following functions called: +// 1. UpdateNumNonEmptyLevels(); +// 2. CalculateBaseBytes(); +// 3. UpdateFilesByCompactionPri(); +// 4. GenerateFileIndexer(); +// 5. GenerateLevelFilesBrief(); +// 6. GenerateLevel0NonOverlapping(); +void VersionStorageInfo::SetFinalized() { + finalized_ = true; +#ifndef NDEBUG + if (compaction_style_ != kCompactionStyleLevel) { + // Not level based compaction. + return; + } + assert(base_level_ < 0 || num_levels() == 1 || + (base_level_ >= 1 && base_level_ < num_levels())); + // Verify all levels newer than base_level are empty except L0 + for (int level = 1; level < base_level(); level++) { + assert(NumLevelBytes(level) == 0); + } + uint64_t max_bytes_prev_level = 0; + for (int level = base_level(); level < num_levels() - 1; level++) { + if (LevelFiles(level).size() == 0) { + continue; + } + assert(MaxBytesForLevel(level) >= max_bytes_prev_level); + max_bytes_prev_level = MaxBytesForLevel(level); + } + int num_empty_non_l0_level = 0; + for (int level = 0; level < num_levels(); level++) { + assert(LevelFiles(level).size() == 0 || + LevelFiles(level).size() == LevelFilesBrief(level).num_files); + if (level > 0 && NumLevelBytes(level) > 0) { + num_empty_non_l0_level++; + } + if (LevelFiles(level).size() > 0) { + assert(level < num_non_empty_levels()); + } + } + assert(compaction_level_.size() > 0); + assert(compaction_level_.size() == compaction_score_.size()); +#endif +} + +void VersionStorageInfo::UpdateNumNonEmptyLevels() { num_non_empty_levels_ = num_levels_; for (int i = num_levels_ - 1; i >= 0; i--) { if (files_[i].size() != 0) { @@ -1037,40 +1569,131 @@ void Version::UpdateNumNonEmptyLevels() { } } -void Version::UpdateFilesBySize() { - if (cfd_->options()->compaction_style == kCompactionStyleFIFO || - cfd_->options()->compaction_style == kCompactionStyleUniversal) { +namespace { +// Sort `temp` based on ratio of overlapping size over file size +void SortFileByOverlappingRatio( + const InternalKeyComparator& icmp, const std::vector& files, + const std::vector& next_level_files, + std::vector* temp) { + std::unordered_map file_to_order; + auto next_level_it = next_level_files.begin(); + + for (auto& file : files) { + uint64_t overlapping_bytes = 0; + // Skip files in next level that is smaller than current file + while (next_level_it != next_level_files.end() && + icmp.Compare((*next_level_it)->largest, file->smallest) < 0) { + next_level_it++; + } + + while (next_level_it != next_level_files.end() && + icmp.Compare((*next_level_it)->smallest, file->largest) < 0) { + overlapping_bytes += (*next_level_it)->fd.file_size; + + if (icmp.Compare((*next_level_it)->largest, file->largest) > 0) { + // next level file cross large boundary of current file. + break; + } + next_level_it++; + } + + assert(file->fd.file_size != 0); + file_to_order[file->fd.GetNumber()] = + overlapping_bytes * 1024u / file->fd.file_size; + } + + std::sort(temp->begin(), temp->end(), + [&](const Fsize& f1, const Fsize& f2) -> bool { + return file_to_order[f1.file->fd.GetNumber()] < + file_to_order[f2.file->fd.GetNumber()]; + }); +} +} // namespace + +void VersionStorageInfo::UpdateFilesByCompactionPri( + CompactionPri compaction_pri) { + if (compaction_style_ == kCompactionStyleFIFO || + compaction_style_ == kCompactionStyleUniversal) { // don't need this return; } // No need to sort the highest level because it is never compacted. - for (int level = 0; level < NumberLevels() - 1; level++) { + for (int level = 0; level < num_levels() - 1; level++) { const std::vector& files = files_[level]; - auto& files_by_size = files_by_size_[level]; - assert(files_by_size.size() == 0); + auto& files_by_compaction_pri = files_by_compaction_pri_[level]; + assert(files_by_compaction_pri.size() == 0); // populate a temp vector for sorting based on size std::vector temp(files.size()); - for (unsigned int i = 0; i < files.size(); i++) { + for (size_t i = 0; i < files.size(); i++) { temp[i].index = i; temp[i].file = files[i]; } // sort the top number_of_files_to_sort_ based on file size - size_t num = Version::number_of_files_to_sort_; + size_t num = VersionStorageInfo::kNumberFilesToSort; if (num > temp.size()) { num = temp.size(); } - std::partial_sort(temp.begin(), temp.begin() + num, temp.end(), - CompareCompensatedSizeDescending); + switch (compaction_pri) { + case kByCompensatedSize: + std::partial_sort(temp.begin(), temp.begin() + num, temp.end(), + CompareCompensatedSizeDescending); + break; + case kOldestLargestSeqFirst: + std::sort(temp.begin(), temp.end(), + [](const Fsize& f1, const Fsize& f2) -> bool { + return f1.file->largest_seqno < f2.file->largest_seqno; + }); + break; + case kOldestSmallestSeqFirst: + std::sort(temp.begin(), temp.end(), + [](const Fsize& f1, const Fsize& f2) -> bool { + return f1.file->smallest_seqno < f2.file->smallest_seqno; + }); + break; + case kMinOverlappingRatio: + SortFileByOverlappingRatio(*internal_comparator_, files_[level], + files_[level + 1], &temp); + break; + default: + assert(false); + } assert(temp.size() == files.size()); - // initialize files_by_size_ - for (unsigned int i = 0; i < temp.size(); i++) { - files_by_size.push_back(temp[i].index); + // initialize files_by_compaction_pri_ + for (size_t i = 0; i < temp.size(); i++) { + files_by_compaction_pri.push_back(static_cast(temp[i].index)); } next_file_to_compact_by_size_[level] = 0; - assert(files_[level].size() == files_by_size_[level].size()); + assert(files_[level].size() == files_by_compaction_pri_[level].size()); + } +} + +void VersionStorageInfo::GenerateLevel0NonOverlapping() { + assert(!finalized_); + level0_non_overlapping_ = true; + if (level_files_brief_.size() == 0) { + return; + } + + // A copy of L0 files sorted by smallest key + std::vector level0_sorted_file( + level_files_brief_[0].files, + level_files_brief_[0].files + level_files_brief_[0].num_files); + std::sort(level0_sorted_file.begin(), level0_sorted_file.end(), + [this](const FdWithKeyRange& f1, const FdWithKeyRange& f2) -> bool { + return (internal_comparator_->Compare(f1.smallest_key, + f2.smallest_key) < 0); + }); + + for (size_t i = 1; i < level0_sorted_file.size(); ++i) { + FdWithKeyRange& f = level0_sorted_file[i]; + FdWithKeyRange& prev = level0_sorted_file[i - 1]; + if (internal_comparator_->Compare(prev.largest_key, f.smallest_key) >= 0) { + level0_non_overlapping_ = false; + break; + } } } @@ -1088,74 +1711,31 @@ bool Version::Unref() { return false; } -bool Version::NeedsCompaction() const { - // In universal compaction case, this check doesn't really - // check the compaction condition, but checks num of files threshold - // only. We are not going to miss any compaction opportunity - // but it's likely that more compactions are scheduled but - // ending up with nothing to do. We can improve it later. - // TODO(sdong): improve this function to be accurate for universal - // compactions. - int max_input_level = - cfd_->compaction_picker()->MaxInputLevel(NumberLevels()); - - for (int i = 0; i <= max_input_level; i++) { - if (compaction_score_[i] >= 1) { - return true; - } +bool VersionStorageInfo::OverlapInLevel(int level, + const Slice* smallest_user_key, + const Slice* largest_user_key) { + if (level >= num_non_empty_levels_) { + // empty level, no overlap + return false; } - return false; -} - -bool Version::OverlapInLevel(int level, - const Slice* smallest_user_key, - const Slice* largest_user_key) { - return SomeFileOverlapsRange(cfd_->internal_comparator(), (level > 0), - file_levels_[level], smallest_user_key, + return SomeFileOverlapsRange(*internal_comparator_, (level > 0), + level_files_brief_[level], smallest_user_key, largest_user_key); } -int Version::PickLevelForMemTableOutput( - const Slice& smallest_user_key, - const Slice& largest_user_key) { - int level = 0; - if (!OverlapInLevel(0, &smallest_user_key, &largest_user_key)) { - // Push to next level if there is no overlap in next level, - // and the #bytes overlapping in the level after that are limited. - InternalKey start(smallest_user_key, kMaxSequenceNumber, kValueTypeForSeek); - InternalKey limit(largest_user_key, 0, static_cast(0)); - std::vector overlaps; - int max_mem_compact_level = cfd_->options()->max_mem_compaction_level; - while (max_mem_compact_level > 0 && level < max_mem_compact_level) { - if (OverlapInLevel(level + 1, &smallest_user_key, &largest_user_key)) { - break; - } - if (level + 2 >= num_levels_) { - level++; - break; - } - GetOverlappingInputs(level + 2, &start, &limit, &overlaps); - const uint64_t sum = TotalFileSize(overlaps); - if (sum > cfd_->compaction_picker()->MaxGrandParentOverlapBytes(level)) { - break; - } - level++; - } - } - - return level; -} - // Store in "*inputs" all files in "level" that overlap [begin,end] // If hint_index is specified, then it points to a file in the // overlapping range. // The file_index returns a pointer to any file in an overlapping range. -void Version::GetOverlappingInputs(int level, - const InternalKey* begin, - const InternalKey* end, - std::vector* inputs, - int hint_index, - int* file_index) { +void VersionStorageInfo::GetOverlappingInputs( + int level, const InternalKey* begin, const InternalKey* end, + std::vector* inputs, int hint_index, int* file_index, + bool expand_range) const { + if (level >= num_non_empty_levels_) { + // this level is empty, no overlapping inputs + return; + } + inputs->clear(); Slice user_begin, user_end; if (begin != nullptr) { @@ -1167,14 +1747,15 @@ void Version::GetOverlappingInputs(int level, if (file_index) { *file_index = -1; } - const Comparator* user_cmp = cfd_->internal_comparator().user_comparator(); + const Comparator* user_cmp = user_comparator_; if (begin != nullptr && end != nullptr && level > 0) { - GetOverlappingInputsBinarySearch(level, user_begin, user_end, inputs, - hint_index, file_index); + GetOverlappingInputsRangeBinarySearch(level, user_begin, user_end, inputs, + hint_index, file_index); return; } - for (size_t i = 0; i < file_levels_[level].num_files; ) { - FdWithKeyRange* f = &(file_levels_[level].files[i++]); + + for (size_t i = 0; i < level_files_brief_[level].num_files; ) { + FdWithKeyRange* f = &(level_files_brief_[level].files[i++]); const Slice file_start = ExtractUserKey(f->smallest_key); const Slice file_limit = ExtractUserKey(f->largest_key); if (begin != nullptr && user_cmp->Compare(file_limit, user_begin) < 0) { @@ -1183,7 +1764,7 @@ void Version::GetOverlappingInputs(int level, // "f" is completely after specified range; skip it } else { inputs->push_back(files_[level][i-1]); - if (level == 0) { + if (level == 0 && expand_range) { // Level-0 files may overlap each other. So check if the newly // added file has expanded the range. If so, restart search. if (begin != nullptr && user_cmp->Compare(file_start, user_begin) < 0) { @@ -1197,29 +1778,61 @@ void Version::GetOverlappingInputs(int level, i = 0; } } else if (file_index) { - *file_index = i-1; + *file_index = static_cast(i) - 1; } } } } +// Store in "*inputs" files in "level" that within range [begin,end] +// Guarantee a "clean cut" boundary between the files in inputs +// and the surrounding files and the maxinum number of files. +// This will ensure that no parts of a key are lost during compaction. +// If hint_index is specified, then it points to a file in the range. +// The file_index returns a pointer to any file in an overlapping range. +void VersionStorageInfo::GetCleanInputsWithinInterval( + int level, const InternalKey* begin, const InternalKey* end, + std::vector* inputs, int hint_index, int* file_index) const { + if (level >= num_non_empty_levels_) { + // this level is empty, no inputs within range + return; + } + + inputs->clear(); + Slice user_begin, user_end; + if (begin != nullptr) { + user_begin = begin->user_key(); + } + if (end != nullptr) { + user_end = end->user_key(); + } + if (file_index) { + *file_index = -1; + } + if (begin != nullptr && end != nullptr && level > 0) { + GetOverlappingInputsRangeBinarySearch(level, user_begin, user_end, inputs, + hint_index, file_index, + true /* within_interval */); + } +} + // Store in "*inputs" all files in "level" that overlap [begin,end] // Employ binary search to find at least one file that overlaps the // specified range. From that file, iterate backwards and // forwards to find all overlapping files. -void Version::GetOverlappingInputsBinarySearch( - int level, - const Slice& user_begin, - const Slice& user_end, - std::vector* inputs, - int hint_index, - int* file_index) { +// if within_range is set, then only store the maximum clean inputs +// within range [begin, end]. "clean" means there is a boudnary +// between the files in "*inputs" and the surrounding files +void VersionStorageInfo::GetOverlappingInputsRangeBinarySearch( + int level, const Slice& user_begin, const Slice& user_end, + std::vector* inputs, int hint_index, int* file_index, + bool within_interval) const { assert(level > 0); int min = 0; int mid = 0; - int max = files_[level].size() -1; + int max = static_cast(files_[level].size()) - 1; bool foundOverlap = false; - const Comparator* user_cmp = cfd_->internal_comparator().user_comparator(); + const Comparator* user_cmp = user_comparator_; // if the caller already knows the index of a file that has overlap, // then we can skip the binary search. @@ -1230,12 +1843,16 @@ void Version::GetOverlappingInputsBinarySearch( while (!foundOverlap && min <= max) { mid = (min + max)/2; - FdWithKeyRange* f = &(file_levels_[level].files[mid]); + FdWithKeyRange* f = &(level_files_brief_[level].files[mid]); const Slice file_start = ExtractUserKey(f->smallest_key); const Slice file_limit = ExtractUserKey(f->largest_key); - if (user_cmp->Compare(file_limit, user_begin) < 0) { + if ((!within_interval && user_cmp->Compare(file_limit, user_begin) < 0) || + (within_interval && user_cmp->Compare(file_start, user_begin) < 0)) { min = mid + 1; - } else if (user_cmp->Compare(user_end, file_start) < 0) { + } else if ((!within_interval && + user_cmp->Compare(user_end, file_start) < 0) || + (within_interval && + user_cmp->Compare(user_end, file_limit) < 0)) { max = mid - 1; } else { foundOverlap = true; @@ -1251,28 +1868,38 @@ void Version::GetOverlappingInputsBinarySearch( if (file_index) { *file_index = mid; } - ExtendOverlappingInputs(level, user_begin, user_end, inputs, mid); + + int start_index, end_index; + if (within_interval) { + ExtendFileRangeWithinInterval(level, user_begin, user_end, mid, &start_index, + &end_index); + } else { + ExtendFileRangeOverlappingInterval(level, user_begin, user_end, mid, + &start_index, &end_index); + } + assert(end_index >= start_index); + // insert overlapping files into vector + for (int i = start_index; i <= end_index; i++) { + inputs->push_back(files_[level][i]); + } } -// Store in "*inputs" all files in "level" that overlap [begin,end] -// The midIndex specifies the index of at least one file that +// Store in *start_index and *end_index the range of all files in +// "level" that overlap [begin,end] +// The mid_index specifies the index of at least one file that // overlaps the specified range. From that file, iterate backward // and forward to find all overlapping files. // Use FileLevel in searching, make it faster -void Version::ExtendOverlappingInputs( - int level, - const Slice& user_begin, - const Slice& user_end, - std::vector* inputs, - unsigned int midIndex) { - - const Comparator* user_cmp = cfd_->internal_comparator().user_comparator(); - const FdWithKeyRange* files = file_levels_[level].files; +void VersionStorageInfo::ExtendFileRangeOverlappingInterval( + int level, const Slice& user_begin, const Slice& user_end, + unsigned int mid_index, int* start_index, int* end_index) const { + const Comparator* user_cmp = user_comparator_; + const FdWithKeyRange* files = level_files_brief_[level].files; #ifndef NDEBUG { - // assert that the file at midIndex overlaps with the range - assert(midIndex < file_levels_[level].num_files); - const FdWithKeyRange* f = &files[midIndex]; + // assert that the file at mid_index overlaps with the range + assert(mid_index < level_files_brief_[level].num_files); + const FdWithKeyRange* f = &files[mid_index]; const Slice fstart = ExtractUserKey(f->smallest_key); const Slice flimit = ExtractUserKey(f->largest_key); if (user_cmp->Compare(fstart, user_begin) >= 0) { @@ -1282,102 +1909,125 @@ void Version::ExtendOverlappingInputs( } } #endif - int startIndex = midIndex + 1; - int endIndex = midIndex; + *start_index = mid_index + 1; + *end_index = mid_index; int count __attribute__((unused)) = 0; // check backwards from 'mid' to lower indices - for (int i = midIndex; i >= 0 ; i--) { + for (int i = mid_index; i >= 0 ; i--) { const FdWithKeyRange* f = &files[i]; const Slice file_limit = ExtractUserKey(f->largest_key); if (user_cmp->Compare(file_limit, user_begin) >= 0) { - startIndex = i; + *start_index = i; assert((count++, true)); } else { break; } } // check forward from 'mid+1' to higher indices - for (unsigned int i = midIndex+1; i < file_levels_[level].num_files; i++) { + for (unsigned int i = mid_index+1; + i < level_files_brief_[level].num_files; i++) { const FdWithKeyRange* f = &files[i]; const Slice file_start = ExtractUserKey(f->smallest_key); if (user_cmp->Compare(file_start, user_end) <= 0) { assert((count++, true)); - endIndex = i; + *end_index = i; } else { break; } } - assert(count == endIndex - startIndex + 1); - - // insert overlapping files into vector - for (int i = startIndex; i <= endIndex; i++) { - FileMetaData* f = files_[level][i]; - inputs->push_back(f); - } + assert(count == *end_index - *start_index + 1); } -// Returns true iff the first or last file in inputs contains -// an overlapping user key to the file "just outside" of it (i.e. -// just after the last file, or just before the first file) -// REQUIRES: "*inputs" is a sorted list of non-overlapping files -bool Version::HasOverlappingUserKey( - const std::vector* inputs, - int level) { - - // If inputs empty, there is no overlap. - // If level == 0, it is assumed that all needed files were already included. - if (inputs->empty() || level == 0){ - return false; +// Store in *start_index and *end_index the clean range of all files in +// "level" within [begin,end] +// The mid_index specifies the index of at least one file within +// the specified range. From that file, iterate backward +// and forward to find all overlapping files and then "shrink" to +// the clean range required. +// Use FileLevel in searching, make it faster +void VersionStorageInfo::ExtendFileRangeWithinInterval( + int level, const Slice& user_begin, const Slice& user_end, + unsigned int mid_index, int* start_index, int* end_index) const { + assert(level != 0); + const Comparator* user_cmp = user_comparator_; + const FdWithKeyRange* files = level_files_brief_[level].files; +#ifndef NDEBUG + { + // assert that the file at mid_index is within the range + assert(mid_index < level_files_brief_[level].num_files); + const FdWithKeyRange* f = &files[mid_index]; + const Slice fstart = ExtractUserKey(f->smallest_key); + const Slice flimit = ExtractUserKey(f->largest_key); + assert(user_cmp->Compare(fstart, user_begin) >= 0 && + user_cmp->Compare(flimit, user_end) <= 0); } - - const Comparator* user_cmp = cfd_->internal_comparator().user_comparator(); - const FileLevel& file_level = file_levels_[level]; - const FdWithKeyRange* files = file_levels_[level].files; - const size_t kNumFiles = file_level.num_files; - - // Check the last file in inputs against the file after it - size_t last_file = FindFile(cfd_->internal_comparator(), file_level, - inputs->back()->largest.Encode()); - assert(0 <= last_file && last_file < kNumFiles); // File should exist! - if (last_file < kNumFiles-1) { // If not the last file - const Slice last_key_in_input = ExtractUserKey( - files[last_file].largest_key); - const Slice first_key_after = ExtractUserKey( - files[last_file+1].smallest_key); - if (user_cmp->Compare(last_key_in_input, first_key_after) == 0) { - // The last user key in input overlaps with the next file's first key - return true; +#endif + ExtendFileRangeOverlappingInterval(level, user_begin, user_end, mid_index, + start_index, end_index); + int left = *start_index; + int right = *end_index; + // shrink from left to right + while (left <= right) { + const Slice& first_key_in_range = ExtractUserKey(files[left].smallest_key); + if (user_cmp->Compare(first_key_in_range, user_begin) < 0) { + left++; + continue; + } + if (left > 0) { // If not first file + const Slice& last_key_before = + ExtractUserKey(files[left - 1].largest_key); + if (user_cmp->Equal(first_key_in_range, last_key_before)) { + // The first user key in range overlaps with the previous file's last + // key + left++; + continue; + } } + break; } - - // Check the first file in inputs against the file just before it - size_t first_file = FindFile(cfd_->internal_comparator(), file_level, - inputs->front()->smallest.Encode()); - assert(0 <= first_file && first_file <= last_file); // File should exist! - if (first_file > 0) { // If not first file - const Slice& first_key_in_input = ExtractUserKey( - files[first_file].smallest_key); - const Slice& last_key_before = ExtractUserKey( - files[first_file-1].largest_key); - if (user_cmp->Compare(first_key_in_input, last_key_before) == 0) { - // The first user key in input overlaps with the previous file's last key - return true; + // shrink from right to left + while (left <= right) { + const Slice last_key_in_range = ExtractUserKey(files[right].largest_key); + if (user_cmp->Compare(last_key_in_range, user_end) > 0) { + right--; + continue; + } + if (right < static_cast(level_files_brief_[level].num_files) - + 1) { // If not the last file + const Slice first_key_after = + ExtractUserKey(files[right + 1].smallest_key); + if (user_cmp->Equal(last_key_in_range, first_key_after)) { + // The last user key in range overlaps with the next file's first key + right--; + continue; + } } + break; } - return false; + *start_index = left; + *end_index = right; } -int64_t Version::NumLevelBytes(int level) const { +uint64_t VersionStorageInfo::NumLevelBytes(int level) const { assert(level >= 0); - assert(level < NumberLevels()); + assert(level < num_levels()); return TotalFileSize(files_[level]); } -const char* Version::LevelSummary(LevelSummaryStorage* scratch) const { - int len = snprintf(scratch->buffer, sizeof(scratch->buffer), "files["); - for (int i = 0; i < NumberLevels(); i++) { +const char* VersionStorageInfo::LevelSummary( + LevelSummaryStorage* scratch) const { + int len = 0; + if (compaction_style_ == kCompactionStyleLevel && num_levels() > 1) { + assert(base_level_ < static_cast(level_max_bytes_.size())); + len = snprintf(scratch->buffer, sizeof(scratch->buffer), + "base level %d max bytes base %" PRIu64 " ", base_level_, + level_max_bytes_[base_level_]); + } + len += + snprintf(scratch->buffer + len, sizeof(scratch->buffer) - len, "files["); + for (int i = 0; i < num_levels(); i++) { int sz = sizeof(scratch->buffer) - len; int ret = snprintf(scratch->buffer + len, sz, "%d ", int(files_[i].size())); if (ret < 0 || ret >= sz) break; @@ -1387,12 +2037,20 @@ const char* Version::LevelSummary(LevelSummaryStorage* scratch) const { // overwrite the last space --len; } - snprintf(scratch->buffer + len, sizeof(scratch->buffer) - len, "]"); + len += snprintf(scratch->buffer + len, sizeof(scratch->buffer) - len, + "] max score %.2f", compaction_score_[0]); + + if (!files_marked_for_compaction_.empty()) { + snprintf(scratch->buffer + len, sizeof(scratch->buffer) - len, + " (%" ROCKSDB_PRIszt " files need compaction)", + files_marked_for_compaction_.size()); + } + return scratch->buffer; } -const char* Version::LevelFileSummary(FileSummaryStorage* scratch, - int level) const { +const char* VersionStorageInfo::LevelFileSummary(FileSummaryStorage* scratch, + int level) const { int len = snprintf(scratch->buffer, sizeof(scratch->buffer), "files_size["); for (const auto& f : files_[level]) { int sz = sizeof(scratch->buffer) - len; @@ -1414,10 +2072,10 @@ const char* Version::LevelFileSummary(FileSummaryStorage* scratch, return scratch->buffer; } -int64_t Version::MaxNextLevelOverlappingBytes() { +int64_t VersionStorageInfo::MaxNextLevelOverlappingBytes() { uint64_t result = 0; std::vector overlaps; - for (int level = 1; level < NumberLevels() - 1; level++) { + for (int level = 1; level < num_levels() - 1; level++) { for (const auto& f : files_[level]) { GetOverlappingInputs(level + 1, &f->smallest, &f->largest, &overlaps); const uint64_t sum = TotalFileSize(overlaps); @@ -1429,28 +2087,194 @@ int64_t Version::MaxNextLevelOverlappingBytes() { return result; } +uint64_t VersionStorageInfo::MaxBytesForLevel(int level) const { + // Note: the result for level zero is not really used since we set + // the level-0 compaction threshold based on number of files. + assert(level >= 0); + assert(level < static_cast(level_max_bytes_.size())); + return level_max_bytes_[level]; +} + +void VersionStorageInfo::CalculateBaseBytes(const ImmutableCFOptions& ioptions, + const MutableCFOptions& options) { + // Special logic to set number of sorted runs. + // It is to match the previous behavior when all files are in L0. + int num_l0_count = static_cast(files_[0].size()); + if (compaction_style_ == kCompactionStyleUniversal) { + // For universal compaction, we use level0 score to indicate + // compaction score for the whole DB. Adding other levels as if + // they are L0 files. + for (int i = 1; i < num_levels(); i++) { + if (!files_[i].empty()) { + num_l0_count++; + } + } + } + set_l0_delay_trigger_count(num_l0_count); + + level_max_bytes_.resize(ioptions.num_levels); + if (!ioptions.level_compaction_dynamic_level_bytes) { + base_level_ = (ioptions.compaction_style == kCompactionStyleLevel) ? 1 : -1; + + // Calculate for static bytes base case + for (int i = 0; i < ioptions.num_levels; ++i) { + if (i == 0 && ioptions.compaction_style == kCompactionStyleUniversal) { + level_max_bytes_[i] = options.max_bytes_for_level_base; + } else if (i > 1) { + level_max_bytes_[i] = MultiplyCheckOverflow( + MultiplyCheckOverflow(level_max_bytes_[i - 1], + options.max_bytes_for_level_multiplier), + options.MaxBytesMultiplerAdditional(i - 1)); + } else { + level_max_bytes_[i] = options.max_bytes_for_level_base; + } + } + } else { + uint64_t max_level_size = 0; + + int first_non_empty_level = -1; + // Find size of non-L0 level of most data. + // Cannot use the size of the last level because it can be empty or less + // than previous levels after compaction. + for (int i = 1; i < num_levels_; i++) { + uint64_t total_size = 0; + for (const auto& f : files_[i]) { + total_size += f->fd.GetFileSize(); + } + if (total_size > 0 && first_non_empty_level == -1) { + first_non_empty_level = i; + } + if (total_size > max_level_size) { + max_level_size = total_size; + } + } + + // Prefill every level's max bytes to disallow compaction from there. + for (int i = 0; i < num_levels_; i++) { + level_max_bytes_[i] = std::numeric_limits::max(); + } + + if (max_level_size == 0) { + // No data for L1 and up. L0 compacts to last level directly. + // No compaction from L1+ needs to be scheduled. + base_level_ = num_levels_ - 1; + } else { + uint64_t base_bytes_max = options.max_bytes_for_level_base; + uint64_t base_bytes_min = static_cast( + base_bytes_max / options.max_bytes_for_level_multiplier); + + // Try whether we can make last level's target size to be max_level_size + uint64_t cur_level_size = max_level_size; + for (int i = num_levels_ - 2; i >= first_non_empty_level; i--) { + // Round up after dividing + cur_level_size = static_cast( + cur_level_size / options.max_bytes_for_level_multiplier); + } + + // Calculate base level and its size. + uint64_t base_level_size; + if (cur_level_size <= base_bytes_min) { + // Case 1. If we make target size of last level to be max_level_size, + // target size of the first non-empty level would be smaller than + // base_bytes_min. We set it be base_bytes_min. + base_level_size = base_bytes_min + 1U; + base_level_ = first_non_empty_level; + ROCKS_LOG_WARN(ioptions.info_log, + "More existing levels in DB than needed. " + "max_bytes_for_level_multiplier may not be guaranteed."); + } else { + // Find base level (where L0 data is compacted to). + base_level_ = first_non_empty_level; + while (base_level_ > 1 && cur_level_size > base_bytes_max) { + --base_level_; + cur_level_size = static_cast( + cur_level_size / options.max_bytes_for_level_multiplier); + } + if (cur_level_size > base_bytes_max) { + // Even L1 will be too large + assert(base_level_ == 1); + base_level_size = base_bytes_max; + } else { + base_level_size = cur_level_size; + } + } + + uint64_t level_size = base_level_size; + for (int i = base_level_; i < num_levels_; i++) { + if (i > base_level_) { + level_size = MultiplyCheckOverflow( + level_size, options.max_bytes_for_level_multiplier); + } + // Don't set any level below base_bytes_max. Otherwise, the LSM can + // assume an hourglass shape where L1+ sizes are smaller than L0. This + // causes compaction scoring, which depends on level sizes, to favor L1+ + // at the expense of L0, which may fill up and stall. + level_max_bytes_[i] = std::max(level_size, base_bytes_max); + } + } + } +} + +uint64_t VersionStorageInfo::EstimateLiveDataSize() const { + // Estimate the live data size by adding up the size of the last level for all + // key ranges. Note: Estimate depends on the ordering of files in level 0 + // because files in level 0 can be overlapping. + uint64_t size = 0; + + auto ikey_lt = [this](InternalKey* x, InternalKey* y) { + return internal_comparator_->Compare(*x, *y) < 0; + }; + // (Ordered) map of largest keys in non-overlapping files + std::map ranges(ikey_lt); + + for (int l = num_levels_ - 1; l >= 0; l--) { + bool found_end = false; + for (auto file : files_[l]) { + // Find the first file where the largest key is larger than the smallest + // key of the current file. If this file does not overlap with the + // current file, none of the files in the map does. If there is + // no potential overlap, we can safely insert the rest of this level + // (if the level is not 0) into the map without checking again because + // the elements in the level are sorted and non-overlapping. + auto lb = (found_end && l != 0) ? + ranges.end() : ranges.lower_bound(&file->smallest); + found_end = (lb == ranges.end()); + if (found_end || internal_comparator_->Compare( + file->largest, (*lb).second->smallest) < 0) { + ranges.emplace_hint(lb, &file->largest, file); + size += file->fd.file_size; + } + } + } + return size; +} + + void Version::AddLiveFiles(std::vector* live) { - for (int level = 0; level < NumberLevels(); level++) { - const std::vector& files = files_[level]; + for (int level = 0; level < storage_info_.num_levels(); level++) { + const std::vector& files = storage_info_.files_[level]; for (const auto& file : files) { live->push_back(file->fd); } } } -std::string Version::DebugString(bool hex) const { +std::string Version::DebugString(bool hex, bool print_stats) const { std::string r; - for (int level = 0; level < num_levels_; level++) { + for (int level = 0; level < storage_info_.num_levels_; level++) { // E.g., // --- level 1 --- // 17:123['a' .. 'd'] // 20:43['e' .. 'g'] + // + // if print_stats=true: + // 17:123['a' .. 'd'](4096) r.append("--- level "); AppendNumberTo(&r, level); r.append(" --- version# "); AppendNumberTo(&r, version_number_); r.append(" ---\n"); - const std::vector& files = files_[level]; + const std::vector& files = storage_info_.files_[level]; for (size_t i = 0; i < files.size(); i++) { r.push_back(' '); AppendNumberTo(&r, files[i]->fd.GetNumber()); @@ -1460,7 +2284,14 @@ std::string Version::DebugString(bool hex) const { r.append(files[i]->smallest.DebugString(hex)); r.append(" .. "); r.append(files[i]->largest.DebugString(hex)); - r.append("]\n"); + r.append("]"); + if (print_stats) { + r.append("("); + r.append(ToString( + files[i]->stats.num_reads_sampled.load(std::memory_order_relaxed))); + r.append(")"); + } + r.append("\n"); } } return r; @@ -1470,285 +2301,54 @@ std::string Version::DebugString(bool hex) const { struct VersionSet::ManifestWriter { Status status; bool done; - port::CondVar cv; + InstrumentedCondVar cv; ColumnFamilyData* cfd; - VersionEdit* edit; - - explicit ManifestWriter(port::Mutex* mu, ColumnFamilyData* cfd, - VersionEdit* e) - : done(false), cv(mu), cfd(cfd), edit(e) {} -}; - -// A helper class so we can efficiently apply a whole sequence -// of edits to a particular state without creating intermediate -// Versions that contain full copies of the intermediate state. -class VersionSet::Builder { - private: - // Helper to sort v->files_ - // kLevel0 -- NewestFirstBySeqNo - // kLevelNon0 -- BySmallestKey - struct FileComparator { - enum SortMethod { - kLevel0 = 0, - kLevelNon0 = 1, - } sort_method; - const InternalKeyComparator* internal_comparator; - - bool operator()(FileMetaData* f1, FileMetaData* f2) const { - switch (sort_method) { - case kLevel0: - return NewestFirstBySeqNo(f1, f2); - case kLevelNon0: - return BySmallestKey(f1, f2, internal_comparator); - } - assert(false); - return false; - } - }; - - typedef std::set FileSet; - struct LevelState { - std::set deleted_files; - FileSet* added_files; - }; - - ColumnFamilyData* cfd_; - Version* base_; - LevelState* levels_; - FileComparator level_zero_cmp_; - FileComparator level_nonzero_cmp_; - - public: - Builder(ColumnFamilyData* cfd) : cfd_(cfd), base_(cfd->current()) { - base_->Ref(); - levels_ = new LevelState[base_->NumberLevels()]; - level_zero_cmp_.sort_method = FileComparator::kLevel0; - level_nonzero_cmp_.sort_method = FileComparator::kLevelNon0; - level_nonzero_cmp_.internal_comparator = &cfd->internal_comparator(); - - levels_[0].added_files = new FileSet(level_zero_cmp_); - for (int level = 1; level < base_->NumberLevels(); level++) { - levels_[level].added_files = new FileSet(level_nonzero_cmp_); - } - } - - ~Builder() { - for (int level = 0; level < base_->NumberLevels(); level++) { - const FileSet* added = levels_[level].added_files; - std::vector to_unref; - to_unref.reserve(added->size()); - for (FileSet::const_iterator it = added->begin(); - it != added->end(); ++it) { - to_unref.push_back(*it); - } - delete added; - for (uint32_t i = 0; i < to_unref.size(); i++) { - FileMetaData* f = to_unref[i]; - f->refs--; - if (f->refs <= 0) { - if (f->table_reader_handle) { - cfd_->table_cache()->ReleaseHandle(f->table_reader_handle); - f->table_reader_handle = nullptr; - } - delete f; - } - } - } - - delete[] levels_; - base_->Unref(); - } - - void CheckConsistency(Version* v) { -#ifndef NDEBUG - // make sure the files are sorted correctly - for (int level = 0; level < v->NumberLevels(); level++) { - for (size_t i = 1; i < v->files_[level].size(); i++) { - auto f1 = v->files_[level][i - 1]; - auto f2 = v->files_[level][i]; - if (level == 0) { - assert(level_zero_cmp_(f1, f2)); - assert(f1->largest_seqno > f2->largest_seqno); - } else { - assert(level_nonzero_cmp_(f1, f2)); - - // Make sure there is no overlap in levels > 0 - if (cfd_->internal_comparator().Compare(f1->largest, f2->smallest) >= - 0) { - fprintf(stderr, "overlapping ranges in same level %s vs. %s\n", - (f1->largest).DebugString().c_str(), - (f2->smallest).DebugString().c_str()); - abort(); - } - } - } - } -#endif - } - - void CheckConsistencyForDeletes(VersionEdit* edit, uint64_t number, - int level) { -#ifndef NDEBUG - // a file to be deleted better exist in the previous version - bool found = false; - for (int l = 0; !found && l < base_->NumberLevels(); l++) { - const std::vector& base_files = base_->files_[l]; - for (unsigned int i = 0; i < base_files.size(); i++) { - FileMetaData* f = base_files[i]; - if (f->fd.GetNumber() == number) { - found = true; - break; - } - } - } - // if the file did not exist in the previous version, then it - // is possibly moved from lower level to higher level in current - // version - for (int l = level+1; !found && l < base_->NumberLevels(); l++) { - const FileSet* added = levels_[l].added_files; - for (FileSet::const_iterator added_iter = added->begin(); - added_iter != added->end(); ++added_iter) { - FileMetaData* f = *added_iter; - if (f->fd.GetNumber() == number) { - found = true; - break; - } - } - } - - // maybe this file was added in a previous edit that was Applied - if (!found) { - const FileSet* added = levels_[level].added_files; - for (FileSet::const_iterator added_iter = added->begin(); - added_iter != added->end(); ++added_iter) { - FileMetaData* f = *added_iter; - if (f->fd.GetNumber() == number) { - found = true; - break; - } - } - } - if (!found) { - fprintf(stderr, "not found %" PRIu64 "\n", number); - } - assert(found); -#endif - } - - // Apply all of the edits in *edit to the current state. - void Apply(VersionEdit* edit) { - CheckConsistency(base_); - - // Delete files - const VersionEdit::DeletedFileSet& del = edit->deleted_files_; - for (const auto& del_file : del) { - const auto level = del_file.first; - const auto number = del_file.second; - levels_[level].deleted_files.insert(number); - CheckConsistencyForDeletes(edit, number, level); - } - - // Add new files - for (const auto& new_file : edit->new_files_) { - const int level = new_file.first; - FileMetaData* f = new FileMetaData(new_file.second); - f->refs = 1; - - levels_[level].deleted_files.erase(f->fd.GetNumber()); - levels_[level].added_files->insert(f); - } - } - - // Save the current state in *v. - void SaveTo(Version* v) { - CheckConsistency(base_); - CheckConsistency(v); - - for (int level = 0; level < base_->NumberLevels(); level++) { - const auto& cmp = (level == 0) ? level_zero_cmp_ : level_nonzero_cmp_; - // Merge the set of added files with the set of pre-existing files. - // Drop any deleted files. Store the result in *v. - const auto& base_files = base_->files_[level]; - auto base_iter = base_files.begin(); - auto base_end = base_files.end(); - const auto& added_files = *levels_[level].added_files; - v->files_[level].reserve(base_files.size() + added_files.size()); - - for (const auto& added : added_files) { - // Add all smaller files listed in base_ - for (auto bpos = std::upper_bound(base_iter, base_end, added, cmp); - base_iter != bpos; - ++base_iter) { - MaybeAddFile(v, level, *base_iter); - } - - MaybeAddFile(v, level, added); - } - - // Add remaining base files - for (; base_iter != base_end; ++base_iter) { - MaybeAddFile(v, level, *base_iter); - } - } - - CheckConsistency(v); - } - - void LoadTableHandlers() { - for (int level = 0; level < cfd_->NumberLevels(); level++) { - for (auto& file_meta : *(levels_[level].added_files)) { - assert (!file_meta->table_reader_handle); - cfd_->table_cache()->FindTable( - base_->vset_->storage_options_, cfd_->internal_comparator(), - file_meta->fd, &file_meta->table_reader_handle, false); - if (file_meta->table_reader_handle != nullptr) { - // Load table_reader - file_meta->fd.table_reader = - cfd_->table_cache()->GetTableReaderFromHandle( - file_meta->table_reader_handle); - } - } - } - } + const autovector& edit_list; - void MaybeAddFile(Version* v, int level, FileMetaData* f) { - if (levels_[level].deleted_files.count(f->fd.GetNumber()) > 0) { - // File is deleted: do nothing - } else { - auto* files = &v->files_[level]; - if (level > 0 && !files->empty()) { - // Must not overlap - assert(cfd_->internal_comparator().Compare( - (*files)[files->size() - 1]->largest, f->smallest) < 0); - } - f->refs++; - files->push_back(f); - } - } + explicit ManifestWriter(InstrumentedMutex* mu, ColumnFamilyData* _cfd, + const autovector& e) + : done(false), cv(mu), cfd(_cfd), edit_list(e) {} }; -VersionSet::VersionSet(const std::string& dbname, const DBOptions* options, - const EnvOptions& storage_options, Cache* table_cache) - : column_family_set_(new ColumnFamilySet(dbname, options, storage_options, - table_cache)), - env_(options->env), +VersionSet::VersionSet(const std::string& dbname, + const ImmutableDBOptions* db_options, + const EnvOptions& storage_options, Cache* table_cache, + WriteBufferManager* write_buffer_manager, + WriteController* write_controller) + : column_family_set_( + new ColumnFamilySet(dbname, db_options, storage_options, table_cache, + write_buffer_manager, write_controller)), + env_(db_options->env), dbname_(dbname), - options_(options), + db_options_(db_options), next_file_number_(2), manifest_file_number_(0), // Filled by Recover() pending_manifest_file_number_(0), last_sequence_(0), + last_to_be_written_sequence_(0), prev_log_number_(0), current_version_number_(0), manifest_file_size_(0), - storage_options_(storage_options), - storage_options_compactions_(storage_options_) {} + env_options_(storage_options), + env_options_compactions_( + env_->OptimizeForCompactionTableRead(env_options_, *db_options_)) {} + +void CloseTables(void* ptr, size_t) { + TableReader* table_reader = reinterpret_cast(ptr); + table_reader->Close(); +} VersionSet::~VersionSet() { // we need to delete column_family_set_ because its destructor depends on // VersionSet + Cache* table_cache = column_family_set_->get_table_cache(); + table_cache->ApplyToAllCacheEntries(&CloseTables, false /* thread_safe */); column_family_set_.reset(); for (auto file : obsolete_files_) { + if (file->table_reader_handle) { + table_cache->Release(file->table_reader_handle); + TableCache::Evict(table_cache, file->fd.GetNumber()); + } delete file; } obsolete_files_.clear(); @@ -1756,6 +2356,14 @@ VersionSet::~VersionSet() { void VersionSet::AppendVersion(ColumnFamilyData* column_family_data, Version* v) { + // compute new compaction score + v->storage_info()->ComputeCompactionScore( + *column_family_data->ioptions(), + *column_family_data->GetLatestMutableCFOptions()); + + // Mark v finalized + v->storage_info_.SetFinalized(); + // Make "v" current assert(v->refs_ == 0); Version* current = column_family_data->current(); @@ -1775,20 +2383,35 @@ void VersionSet::AppendVersion(ColumnFamilyData* column_family_data, } Status VersionSet::LogAndApply(ColumnFamilyData* column_family_data, - VersionEdit* edit, port::Mutex* mu, - Directory* db_directory, bool new_descriptor_log, - const ColumnFamilyOptions* options) { + const MutableCFOptions& mutable_cf_options, + const autovector& edit_list, + InstrumentedMutex* mu, Directory* db_directory, + bool new_descriptor_log, + const ColumnFamilyOptions* new_cf_options) { mu->AssertHeld(); + // num of edits + auto num_edits = edit_list.size(); + if (num_edits == 0) { + return Status::OK(); + } else if (num_edits > 1) { +#ifndef NDEBUG + // no group commits for column family add or drop + for (auto& edit : edit_list) { + assert(!edit->IsColumnFamilyManipulation()); + } +#endif + } // column_family_data can be nullptr only if this is column_family_add. // in that case, we also need to specify ColumnFamilyOptions if (column_family_data == nullptr) { - assert(edit->is_column_family_add_); - assert(options != nullptr); + assert(num_edits == 1); + assert(edit_list[0]->is_column_family_add_); + assert(new_cf_options != nullptr); } // queue our request - ManifestWriter w(mu, column_family_data, edit); + ManifestWriter w(mu, column_family_data, edit_list); manifest_writers_.push_back(&w); while (!w.done && &w != manifest_writers_.front()) { w.cv.Wait(); @@ -1804,37 +2427,40 @@ Status VersionSet::LogAndApply(ColumnFamilyData* column_family_data, if (!manifest_writers_.empty()) { manifest_writers_.front()->cv.Signal(); } - return Status::OK(); + // we steal this code to also inform about cf-drop + return Status::ShutdownInProgress(); } - std::vector batch_edits; + autovector batch_edits; Version* v = nullptr; - std::unique_ptr builder(nullptr); + std::unique_ptr builder_guard(nullptr); // process all requests in the queue ManifestWriter* last_writer = &w; assert(!manifest_writers_.empty()); assert(manifest_writers_.front() == &w); - if (edit->IsColumnFamilyManipulation()) { + if (w.edit_list.front()->IsColumnFamilyManipulation()) { // no group commits for column family add or drop - LogAndApplyCFHelper(edit); - batch_edits.push_back(edit); + LogAndApplyCFHelper(w.edit_list.front()); + batch_edits.push_back(w.edit_list.front()); } else { v = new Version(column_family_data, this, current_version_number_++); - builder.reset(new Builder(column_family_data)); + builder_guard.reset(new BaseReferencedVersionBuilder(column_family_data)); + auto* builder = builder_guard->version_builder(); for (const auto& writer : manifest_writers_) { - if (writer->edit->IsColumnFamilyManipulation() || + if (writer->edit_list.front()->IsColumnFamilyManipulation() || writer->cfd->GetID() != column_family_data->GetID()) { // no group commits for column family add or drop // also, group commits across column families are not supported break; } last_writer = writer; - LogAndApplyHelper(column_family_data, builder.get(), v, last_writer->edit, - mu); - batch_edits.push_back(last_writer->edit); + for (const auto& edit : writer->edit_list) { + LogAndApplyHelper(column_family_data, builder, v, edit, mu); + batch_edits.push_back(edit); + } } - builder->SaveTo(v); + builder->SaveTo(v->storage_info()); } // Initialize new descriptor log file if necessary by creating @@ -1844,9 +2470,9 @@ Status VersionSet::LogAndApply(ColumnFamilyData* column_family_data, assert(pending_manifest_file_number_ == 0); if (!descriptor_log_ || - manifest_file_size_ > options_->max_manifest_file_size) { + manifest_file_size_ > db_options_->max_manifest_file_size) { pending_manifest_file_number_ = NewFileNumber(); - batch_edits.back()->SetNextFile(next_file_number_); + batch_edits.back()->SetNextFile(next_file_number_.load()); new_descriptor_log = true; } else { pending_manifest_file_number_ = manifest_file_number_; @@ -1855,91 +2481,79 @@ Status VersionSet::LogAndApply(ColumnFamilyData* column_family_data, if (new_descriptor_log) { // if we're writing out new snapshot make sure to persist max column family if (column_family_set_->GetMaxColumnFamily() > 0) { - edit->SetMaxColumnFamily(column_family_set_->GetMaxColumnFamily()); + w.edit_list.front()->SetMaxColumnFamily( + column_family_set_->GetMaxColumnFamily()); } } // Unlock during expensive operations. New writes cannot get here // because &w is ensuring that all new writes get queued. { - std::vector size_being_compacted; - if (!edit->IsColumnFamilyManipulation()) { - size_being_compacted.resize(v->NumberLevels() - 1); - // calculate the amount of data being compacted at every level - column_family_data->compaction_picker()->SizeBeingCompacted( - size_being_compacted); - } mu->Unlock(); - if (!edit->IsColumnFamilyManipulation() && options_->max_open_files == -1) { + TEST_SYNC_POINT("VersionSet::LogAndApply:WriteManifest"); + if (!w.edit_list.front()->IsColumnFamilyManipulation() && + this->GetColumnFamilySet()->get_table_cache()->GetCapacity() == + TableCache::kInfiniteCapacity) { // unlimited table cache. Pre-load table handle now. // Need to do it out of the mutex. - builder->LoadTableHandlers(); + builder_guard->version_builder()->LoadTableHandlers( + column_family_data->internal_stats(), + column_family_data->ioptions()->optimize_filters_for_hits, + true /* prefetch_index_and_filter_in_cache */); } // This is fine because everything inside of this block is serialized -- // only one thread can be here at the same time if (new_descriptor_log) { // create manifest file - Log(options_->info_log, - "Creating manifest %" PRIu64 "\n", pending_manifest_file_number_); + ROCKS_LOG_INFO(db_options_->info_log, "Creating manifest %" PRIu64 "\n", + pending_manifest_file_number_); unique_ptr descriptor_file; - s = env_->NewWritableFile( - DescriptorFileName(dbname_, pending_manifest_file_number_), - &descriptor_file, env_->OptimizeForManifestWrite(storage_options_)); + EnvOptions opt_env_opts = env_->OptimizeForManifestWrite(env_options_); + s = NewWritableFile( + env_, DescriptorFileName(dbname_, pending_manifest_file_number_), + &descriptor_file, opt_env_opts); if (s.ok()) { descriptor_file->SetPreallocationBlockSize( - options_->manifest_preallocation_size); - descriptor_log_.reset(new log::Writer(std::move(descriptor_file))); + db_options_->manifest_preallocation_size); + + unique_ptr file_writer( + new WritableFileWriter(std::move(descriptor_file), opt_env_opts)); + descriptor_log_.reset( + new log::Writer(std::move(file_writer), 0, false)); s = WriteSnapshot(descriptor_log_.get()); } } - if (!edit->IsColumnFamilyManipulation()) { + if (!w.edit_list.front()->IsColumnFamilyManipulation()) { // This is cpu-heavy operations, which should be called outside mutex. - v->PrepareApply(size_being_compacted); + v->PrepareApply(mutable_cf_options, true); } // Write new record to MANIFEST log if (s.ok()) { for (auto& e : batch_edits) { std::string record; - e->EncodeTo(&record); + if (!e->EncodeTo(&record)) { + s = Status::Corruption( + "Unable to Encode VersionEdit:" + e->DebugString(true)); + break; + } + TEST_KILL_RANDOM("VersionSet::LogAndApply:BeforeAddRecord", + rocksdb_kill_odds * REDUCE_ODDS2); s = descriptor_log_->AddRecord(record); if (!s.ok()) { break; } } if (s.ok()) { - if (options_->use_fsync) { - StopWatch sw(env_, options_->statistics.get(), - MANIFEST_FILE_SYNC_MICROS); - s = descriptor_log_->file()->Fsync(); - } else { - StopWatch sw(env_, options_->statistics.get(), - MANIFEST_FILE_SYNC_MICROS); - s = descriptor_log_->file()->Sync(); - } + s = SyncManifest(env_, db_options_, descriptor_log_->file()); } if (!s.ok()) { - Log(options_->info_log, "MANIFEST write: %s\n", s.ToString().c_str()); - bool all_records_in = true; - for (auto& e : batch_edits) { - std::string record; - e->EncodeTo(&record); - if (!ManifestContains(pending_manifest_file_number_, record)) { - all_records_in = false; - break; - } - } - if (all_records_in) { - Log(options_->info_log, - "MANIFEST contains log record despite error; advancing to new " - "version to prevent mismatch between in-memory and logged state" - " If paranoid is set, then the db is now in readonly mode."); - s = Status::OK(); - } + ROCKS_LOG_ERROR(db_options_->info_log, "MANIFEST write: %s\n", + s.ToString().c_str()); } } @@ -1948,15 +2562,6 @@ Status VersionSet::LogAndApply(ColumnFamilyData* column_family_data, if (s.ok() && new_descriptor_log) { s = SetCurrentFile(env_, dbname_, pending_manifest_file_number_, db_directory); - if (s.ok() && pending_manifest_file_number_ > manifest_file_number_) { - // delete old manifest file - Log(options_->info_log, - "Deleting manifest %" PRIu64 " current manifest %" PRIu64 "\n", - manifest_file_number_, pending_manifest_file_number_); - // we don't care about an error here, PurgeObsoleteFiles will take care - // of it later - env_->DeleteFile(DescriptorFileName(dbname_, manifest_file_number_)); - } } if (s.ok()) { @@ -1964,18 +2569,32 @@ Status VersionSet::LogAndApply(ColumnFamilyData* column_family_data, new_manifest_file_size = descriptor_log_->file()->GetFileSize(); } - LogFlush(options_->info_log); + if (w.edit_list.front()->is_column_family_drop_) { + TEST_SYNC_POINT("VersionSet::LogAndApply::ColumnFamilyDrop:0"); + TEST_SYNC_POINT("VersionSet::LogAndApply::ColumnFamilyDrop:1"); + TEST_SYNC_POINT("VersionSet::LogAndApply::ColumnFamilyDrop:2"); + } + + LogFlush(db_options_->info_log); + TEST_SYNC_POINT("VersionSet::LogAndApply:WriteManifestDone"); mu->Lock(); } + // Append the old mainfest file to the obsolete_manifests_ list to be deleted + // by PurgeObsoleteFiles later. + if (s.ok() && new_descriptor_log) { + obsolete_manifests_.emplace_back( + DescriptorFileName("", manifest_file_number_)); + } + // Install the new version if (s.ok()) { - if (edit->is_column_family_add_) { + if (w.edit_list.front()->is_column_family_add_) { // no group commit on column family add assert(batch_edits.size() == 1); - assert(options != nullptr); - CreateColumnFamily(*options, edit); - } else if (edit->is_column_family_drop_) { + assert(new_cf_options != nullptr); + CreateColumnFamily(*new_cf_options, w.edit_list.front()); + } else if (w.edit_list.front()->is_column_family_drop_) { assert(batch_edits.size() == 1); column_family_data->SetDropped(); if (column_family_data->Unref()) { @@ -1998,16 +2617,22 @@ Status VersionSet::LogAndApply(ColumnFamilyData* column_family_data, manifest_file_number_ = pending_manifest_file_number_; manifest_file_size_ = new_manifest_file_size; - prev_log_number_ = edit->prev_log_number_; + prev_log_number_ = w.edit_list.front()->prev_log_number_; } else { - Log(options_->info_log, "Error in committing version %lu to [%s]", - (unsigned long)v->GetVersionNumber(), - column_family_data->GetName().c_str()); + std::string version_edits; + for (auto& e : batch_edits) { + version_edits = version_edits + "\n" + e->DebugString(true); + } + ROCKS_LOG_ERROR( + db_options_->info_log, + "[%s] Error in committing version edit to MANIFEST: %s", + column_family_data ? column_family_data->GetName().c_str() : "", + version_edits.c_str()); delete v; if (new_descriptor_log) { - Log(options_->info_log, - "Deleting manifest %" PRIu64 " current manifest %" PRIu64 "\n", - manifest_file_number_, pending_manifest_file_number_); + ROCKS_LOG_INFO(db_options_->info_log, "Deleting manifest %" PRIu64 + " current manifest %" PRIu64 "\n", + manifest_file_number_, pending_manifest_file_number_); descriptor_log_.reset(); env_->DeleteFile( DescriptorFileName(dbname_, pending_manifest_file_number_)); @@ -2035,7 +2660,7 @@ Status VersionSet::LogAndApply(ColumnFamilyData* column_family_data, void VersionSet::LogAndApplyCFHelper(VersionEdit* edit) { assert(edit->IsColumnFamilyManipulation()); - edit->SetNextFile(next_file_number_); + edit->SetNextFile(next_file_number_.load()); edit->SetLastSequence(last_sequence_); if (edit->is_column_family_drop_) { // if we drop column family, we have to make sure to save max column family, @@ -2044,21 +2669,21 @@ void VersionSet::LogAndApplyCFHelper(VersionEdit* edit) { } } -void VersionSet::LogAndApplyHelper(ColumnFamilyData* cfd, Builder* builder, - Version* v, VersionEdit* edit, - port::Mutex* mu) { +void VersionSet::LogAndApplyHelper(ColumnFamilyData* cfd, + VersionBuilder* builder, Version* v, + VersionEdit* edit, InstrumentedMutex* mu) { mu->AssertHeld(); assert(!edit->IsColumnFamilyManipulation()); if (edit->has_log_number_) { assert(edit->log_number_ >= cfd->GetLogNumber()); - assert(edit->log_number_ < next_file_number_); + assert(edit->log_number_ < next_file_number_.load()); } if (!edit->has_prev_log_number_) { edit->SetPrevLogNumber(prev_log_number_); } - edit->SetNextFile(next_file_number_); + edit->SetNextFile(next_file_number_.load()); edit->SetLastSequence(last_sequence_); builder->Apply(edit); @@ -2097,18 +2722,23 @@ Status VersionSet::Recover( return Status::Corruption("CURRENT file corrupted"); } - Log(options_->info_log, "Recovering from manifest file: %s\n", - manifest_filename.c_str()); + ROCKS_LOG_INFO(db_options_->info_log, "Recovering from manifest file: %s\n", + manifest_filename.c_str()); manifest_filename = dbname_ + "/" + manifest_filename; - unique_ptr manifest_file; - s = env_->NewSequentialFile(manifest_filename, &manifest_file, - storage_options_); - if (!s.ok()) { - return s; + unique_ptr manifest_file_reader; + { + unique_ptr manifest_file; + s = env_->NewSequentialFile(manifest_filename, &manifest_file, + env_->OptimizeForManifestRead(env_options_)); + if (!s.ok()) { + return s; + } + manifest_file_reader.reset( + new SequentialFileReader(std::move(manifest_file))); } - uint64_t manifest_file_size; - s = env_->GetFileSize(manifest_filename, &manifest_file_size); + uint64_t current_manifest_file_size; + s = env_->GetFileSize(manifest_filename, ¤t_manifest_file_size); if (!s.ok()) { return s; } @@ -2120,9 +2750,9 @@ Status VersionSet::Recover( uint64_t next_file = 0; uint64_t last_sequence = 0; uint64_t log_number = 0; - uint64_t prev_log_number = 0; + uint64_t previous_log_number = 0; uint32_t max_column_family = 0; - std::unordered_map builders; + std::unordered_map builders; // add default column family auto default_cf_iter = cf_name_to_options.find(kDefaultColumnFamilyName); @@ -2134,13 +2764,16 @@ Status VersionSet::Recover( default_cf_edit.SetColumnFamily(0); ColumnFamilyData* default_cfd = CreateColumnFamily(default_cf_iter->second, &default_cf_edit); - builders.insert({0, new Builder(default_cfd)}); + // In recovery, nobody else can access it, so it's fine to set it to be + // initialized earlier. + default_cfd->set_initialized(); + builders.insert({0, new BaseReferencedVersionBuilder(default_cfd)}); { VersionSet::LogReporter reporter; reporter.status = &s; - log::Reader reader(std::move(manifest_file), &reporter, true /*checksum*/, - 0 /*initial_offset*/); + log::Reader reader(NULL, std::move(manifest_file_reader), &reporter, + true /*checksum*/, 0 /*initial_offset*/, 0); Slice record; std::string scratch; while (reader.ReadRecord(&record, &scratch) && s.ok()) { @@ -2180,7 +2813,9 @@ Status VersionSet::Recover( {edit.column_family_, edit.column_family_name_}); } else { cfd = CreateColumnFamily(cf_options->second, &edit); - builders.insert({edit.column_family_, new Builder(cfd)}); + cfd->set_initialized(); + builders.insert( + {edit.column_family_, new BaseReferencedVersionBuilder(cfd)}); } } else if (edit.is_column_family_drop_) { if (cf_in_builders) { @@ -2213,24 +2848,20 @@ Status VersionSet::Recover( cfd = column_family_set_->GetColumnFamily(edit.column_family_); // this should never happen since cf_in_builders is true assert(cfd != nullptr); - if (edit.max_level_ >= cfd->current()->NumberLevels()) { - s = Status::InvalidArgument( - "db has more levels than options.num_levels"); - break; - } // if it is not column family add or column family drop, // then it's a file add/delete, which should be forwarded // to builder auto builder = builders.find(edit.column_family_); assert(builder != builders.end()); - builder->second->Apply(&edit); + builder->second->version_builder()->Apply(&edit); } if (cfd != nullptr) { if (edit.has_log_number_) { if (cfd->GetLogNumber() > edit.log_number_) { - Log(options_->info_log, + ROCKS_LOG_WARN( + db_options_->info_log, "MANIFEST corruption detected, but ignored - Log numbers in " "records NOT monotonically increasing"); } else { @@ -2248,7 +2879,7 @@ Status VersionSet::Recover( } if (edit.has_prev_log_number_) { - prev_log_number = edit.prev_log_number_; + previous_log_number = edit.prev_log_number_; have_prev_log_number = true; } @@ -2278,18 +2909,18 @@ Status VersionSet::Recover( } if (!have_prev_log_number) { - prev_log_number = 0; + previous_log_number = 0; } column_family_set_->UpdateMaxColumnFamily(max_column_family); - MarkFileNumberUsed(prev_log_number); - MarkFileNumberUsed(log_number); + MarkFileNumberUsedDuringRecovery(previous_log_number); + MarkFileNumberUsedDuringRecovery(log_number); } // there were some column families in the MANIFEST that weren't specified // in the argument. This is OK in read_only mode - if (read_only == false && column_families_not_found.size() > 0) { + if (read_only == false && !column_families_not_found.empty()) { std::string list_of_not_found; for (const auto& cf : column_families_not_found) { list_of_not_found += ", " + cf.second; @@ -2302,50 +2933,73 @@ Status VersionSet::Recover( if (s.ok()) { for (auto cfd : *column_family_set_) { + assert(builders.count(cfd->GetID()) > 0); + auto* builder = builders[cfd->GetID()]->version_builder(); + if (!builder->CheckConsistencyForNumLevels()) { + s = Status::InvalidArgument( + "db has more levels than options.num_levels"); + break; + } + } + } + + if (s.ok()) { + for (auto cfd : *column_family_set_) { + if (cfd->IsDropped()) { + continue; + } + assert(cfd->initialized()); auto builders_iter = builders.find(cfd->GetID()); assert(builders_iter != builders.end()); - auto builder = builders_iter->second; + auto* builder = builders_iter->second->version_builder(); - if (options_->max_open_files == -1) { - // unlimited table cache. Pre-load table handle now. - // Need to do it out of the mutex. - builder->LoadTableHandlers(); + if (GetColumnFamilySet()->get_table_cache()->GetCapacity() == + TableCache::kInfiniteCapacity) { + // unlimited table cache. Pre-load table handle now. + // Need to do it out of the mutex. + builder->LoadTableHandlers( + cfd->internal_stats(), db_options_->max_file_opening_threads, + false /* prefetch_index_and_filter_in_cache */); } Version* v = new Version(cfd, this, current_version_number_++); - builder->SaveTo(v); + builder->SaveTo(v->storage_info()); // Install recovered version - std::vector size_being_compacted(v->NumberLevels() - 1); - cfd->compaction_picker()->SizeBeingCompacted(size_being_compacted); - v->PrepareApply(size_being_compacted); + v->PrepareApply(*cfd->GetLatestMutableCFOptions(), + !(db_options_->skip_stats_update_on_db_open)); AppendVersion(cfd, v); } - manifest_file_size_ = manifest_file_size; - next_file_number_ = next_file + 1; + manifest_file_size_ = current_manifest_file_size; + next_file_number_.store(next_file + 1); + last_to_be_written_sequence_ = last_sequence; last_sequence_ = last_sequence; - prev_log_number_ = prev_log_number; + prev_log_number_ = previous_log_number; - Log(options_->info_log, + ROCKS_LOG_INFO( + db_options_->info_log, "Recovered from manifest file:%s succeeded," "manifest_file_number is %lu, next_file_number is %lu, " "last_sequence is %lu, log_number is %lu," "prev_log_number is %lu," "max_column_family is %u\n", manifest_filename.c_str(), (unsigned long)manifest_file_number_, - (unsigned long)next_file_number_, (unsigned long)last_sequence_, + (unsigned long)next_file_number_.load(), (unsigned long)last_sequence_, (unsigned long)log_number, (unsigned long)prev_log_number_, column_family_set_->GetMaxColumnFamily()); for (auto cfd : *column_family_set_) { - Log(options_->info_log, - "Column family [%s] (ID %u), log number is %" PRIu64 "\n", - cfd->GetName().c_str(), cfd->GetID(), cfd->GetLogNumber()); + if (cfd->IsDropped()) { + continue; + } + ROCKS_LOG_INFO(db_options_->info_log, + "Column family [%s] (ID %u), log number is %" PRIu64 "\n", + cfd->GetName().c_str(), cfd->GetID(), cfd->GetLogNumber()); } } - for (auto builder : builders) { + for (auto& builder : builders) { delete builder.second; } @@ -2369,19 +3023,24 @@ Status VersionSet::ListColumnFamilies(std::vector* column_families, current.resize(current.size() - 1); std::string dscname = dbname + "/" + current; + + unique_ptr file_reader; + { unique_ptr file; s = env->NewSequentialFile(dscname, &file, soptions); if (!s.ok()) { return s; } + file_reader.reset(new SequentialFileReader(std::move(file))); + } std::map column_family_names; // default column family is always implicitly there column_family_names.insert({0, kDefaultColumnFamilyName}); VersionSet::LogReporter reporter; reporter.status = &s; - log::Reader reader(std::move(file), &reporter, true /*checksum*/, - 0 /*initial_offset*/); + log::Reader reader(NULL, std::move(file_reader), &reporter, true /*checksum*/, + 0 /*initial_offset*/, 0); Slice record; std::string scratch; while (reader.ReadRecord(&record, &scratch) && s.ok()) { @@ -2422,18 +3081,20 @@ Status VersionSet::ListColumnFamilies(std::vector* column_families, #ifndef ROCKSDB_LITE Status VersionSet::ReduceNumberOfLevels(const std::string& dbname, const Options* options, - const EnvOptions& storage_options, + const EnvOptions& env_options, int new_levels) { if (new_levels <= 1) { return Status::InvalidArgument( "Number of levels needs to be bigger than 1"); } + ImmutableDBOptions db_options(*options); ColumnFamilyOptions cf_options(*options); - std::shared_ptr tc(NewLRUCache( - options->max_open_files - 10, options->table_cache_numshardbits, - options->table_cache_remove_scan_count_limit)); - VersionSet versions(dbname, options, storage_options, tc.get()); + std::shared_ptr tc(NewLRUCache(options->max_open_files - 10, + options->table_cache_numshardbits)); + WriteController wc(options->delayed_write_rate); + WriteBufferManager wb(options->db_write_buffer_size); + VersionSet versions(dbname, &db_options, env_options, tc.get(), &wb, &wc); Status status; std::vector dummy; @@ -2447,7 +3108,8 @@ Status VersionSet::ReduceNumberOfLevels(const std::string& dbname, Version* current_version = versions.GetColumnFamilySet()->GetDefault()->current(); - int current_levels = current_version->NumberLevels(); + auto* vstorage = current_version->storage_info(); + int current_levels = vstorage->num_levels(); if (current_levels <= new_levels) { return Status::OK(); @@ -2458,7 +3120,7 @@ Status VersionSet::ReduceNumberOfLevels(const std::string& dbname, int first_nonempty_level = -1; int first_nonempty_level_filenum = 0; for (int i = new_levels - 1; i < current_levels; i++) { - int file_num = current_version->NumLevelFiles(i); + int file_num = vstorage->NumLevelFiles(i); if (file_num != 0) { if (first_nonempty_level < 0) { first_nonempty_level = i; @@ -2475,38 +3137,45 @@ Status VersionSet::ReduceNumberOfLevels(const std::string& dbname, } } - std::vector* old_files_list = current_version->files_; // we need to allocate an array with the old number of levels size to // avoid SIGSEGV in WriteSnapshot() // however, all levels bigger or equal to new_levels will be empty std::vector* new_files_list = new std::vector[current_levels]; for (int i = 0; i < new_levels - 1; i++) { - new_files_list[i] = old_files_list[i]; + new_files_list[i] = vstorage->LevelFiles(i); } if (first_nonempty_level > 0) { - new_files_list[new_levels - 1] = old_files_list[first_nonempty_level]; + new_files_list[new_levels - 1] = vstorage->LevelFiles(first_nonempty_level); } - delete[] current_version->files_; - current_version->files_ = new_files_list; - current_version->num_levels_ = new_levels; + delete[] vstorage -> files_; + vstorage->files_ = new_files_list; + vstorage->num_levels_ = new_levels; + MutableCFOptions mutable_cf_options(*options); VersionEdit ve; - port::Mutex dummy_mutex; - MutexLock l(&dummy_mutex); - return versions.LogAndApply(versions.GetColumnFamilySet()->GetDefault(), &ve, - &dummy_mutex, nullptr, true); + InstrumentedMutex dummy_mutex; + InstrumentedMutexLock l(&dummy_mutex); + return versions.LogAndApply( + versions.GetColumnFamilySet()->GetDefault(), + mutable_cf_options, &ve, &dummy_mutex, nullptr, true); } Status VersionSet::DumpManifest(Options& options, std::string& dscname, - bool verbose, bool hex) { + bool verbose, bool hex, bool json) { // Open the specified manifest file. - unique_ptr file; - Status s = options.env->NewSequentialFile(dscname, &file, storage_options_); - if (!s.ok()) { - return s; + unique_ptr file_reader; + Status s; + { + unique_ptr file; + s = options.env->NewSequentialFile( + dscname, &file, env_->OptimizeForManifestRead(env_options_)); + if (!s.ok()) { + return s; + } + file_reader.reset(new SequentialFileReader(std::move(file))); } bool have_prev_log_number = false; @@ -2514,10 +3183,10 @@ Status VersionSet::DumpManifest(Options& options, std::string& dscname, bool have_last_sequence = false; uint64_t next_file = 0; uint64_t last_sequence = 0; - uint64_t prev_log_number = 0; + uint64_t previous_log_number = 0; int count = 0; std::unordered_map comparators; - std::unordered_map builders; + std::unordered_map builders; // add default column family VersionEdit default_cf_edit; @@ -2525,13 +3194,13 @@ Status VersionSet::DumpManifest(Options& options, std::string& dscname, default_cf_edit.SetColumnFamily(0); ColumnFamilyData* default_cfd = CreateColumnFamily(ColumnFamilyOptions(options), &default_cf_edit); - builders.insert({0, new Builder(default_cfd)}); + builders.insert({0, new BaseReferencedVersionBuilder(default_cfd)}); { VersionSet::LogReporter reporter; reporter.status = &s; - log::Reader reader(std::move(file), &reporter, true/*checksum*/, - 0/*initial_offset*/); + log::Reader reader(NULL, std::move(file_reader), &reporter, + true /*checksum*/, 0 /*initial_offset*/, 0); Slice record; std::string scratch; while (reader.ReadRecord(&record, &scratch) && s.ok()) { @@ -2542,9 +3211,10 @@ Status VersionSet::DumpManifest(Options& options, std::string& dscname, } // Write out each individual edit - if (verbose) { - printf("*************************Edit[%d] = %s\n", - count, edit.DebugString(hex).c_str()); + if (verbose && !json) { + printf("%s\n", edit.DebugString(hex).c_str()); + } else if (json) { + printf("%s\n", edit.DebugJSON(count, hex).c_str()); } count++; @@ -2564,7 +3234,9 @@ Status VersionSet::DumpManifest(Options& options, std::string& dscname, break; } cfd = CreateColumnFamily(ColumnFamilyOptions(options), &edit); - builders.insert({edit.column_family_, new Builder(cfd)}); + cfd->set_initialized(); + builders.insert( + {edit.column_family_, new BaseReferencedVersionBuilder(cfd)}); } else if (edit.is_column_family_drop_) { if (!cf_in_builders) { s = Status::Corruption( @@ -2596,7 +3268,7 @@ Status VersionSet::DumpManifest(Options& options, std::string& dscname, // to builder auto builder = builders.find(edit.column_family_); assert(builder != builders.end()); - builder->second->Apply(&edit); + builder->second->version_builder()->Apply(&edit); } if (cfd != nullptr && edit.has_log_number_) { @@ -2604,7 +3276,7 @@ Status VersionSet::DumpManifest(Options& options, std::string& dscname, } if (edit.has_prev_log_number_) { - prev_log_number = edit.prev_log_number_; + previous_log_number = edit.prev_log_number_; have_prev_log_number = true; } @@ -2623,7 +3295,7 @@ Status VersionSet::DumpManifest(Options& options, std::string& dscname, } } } - file.reset(); + file_reader.reset(); if (s.ok()) { if (!have_next_file) { @@ -2635,22 +3307,22 @@ Status VersionSet::DumpManifest(Options& options, std::string& dscname, } if (!have_prev_log_number) { - prev_log_number = 0; + previous_log_number = 0; } } if (s.ok()) { for (auto cfd : *column_family_set_) { + if (cfd->IsDropped()) { + continue; + } auto builders_iter = builders.find(cfd->GetID()); assert(builders_iter != builders.end()); - auto builder = builders_iter->second; + auto builder = builders_iter->second->version_builder(); Version* v = new Version(cfd, this, current_version_number_++); - builder->SaveTo(v); - std::vector size_being_compacted(v->NumberLevels() - 1); - cfd->compaction_picker()->SizeBeingCompacted(size_being_compacted); - v->PrepareApply(size_being_compacted); - delete builder; + builder->SaveTo(v->storage_info()); + v->PrepareApply(*cfd->GetLatestMutableCFOptions(), false); printf("--------------- Column family \"%s\" (ID %u) --------------\n", cfd->GetName().c_str(), (unsigned int)cfd->GetID()); @@ -2665,15 +3337,21 @@ Status VersionSet::DumpManifest(Options& options, std::string& dscname, delete v; } - next_file_number_ = next_file + 1; + // Free builders + for (auto& builder : builders) { + delete builder.second; + } + + next_file_number_.store(next_file + 1); + last_to_be_written_sequence_ = last_sequence; last_sequence_ = last_sequence; - prev_log_number_ = prev_log_number; + prev_log_number_ = previous_log_number; printf( "next_file_number %lu last_sequence " "%lu prev_log_number %lu max_column_family %u\n", - (unsigned long)next_file_number_, (unsigned long)last_sequence, - (unsigned long)prev_log_number, + (unsigned long)next_file_number_.load(), (unsigned long)last_sequence, + (unsigned long)previous_log_number, column_family_set_->GetMaxColumnFamily()); } @@ -2681,9 +3359,11 @@ Status VersionSet::DumpManifest(Options& options, std::string& dscname, } #endif // ROCKSDB_LITE -void VersionSet::MarkFileNumberUsed(uint64_t number) { - if (next_file_number_ <= number) { - next_file_number_ = number + 1; +void VersionSet::MarkFileNumberUsedDuringRecovery(uint64_t number) { + // only called during recovery which is single threaded, so this works because + // there can't be concurrent calls + if (next_file_number_.load(std::memory_order_relaxed) <= number) { + next_file_number_.store(number + 1, std::memory_order_relaxed); } } @@ -2696,6 +3376,10 @@ Status VersionSet::WriteSnapshot(log::Writer* log) { // LogAndApply. Column family manipulations can only happen within LogAndApply // (the same single thread), so we're safe to iterate. for (auto cfd : *column_family_set_) { + if (cfd->IsDropped()) { + continue; + } + assert(cfd->initialized()); { // Store column family info VersionEdit edit; @@ -2708,7 +3392,10 @@ Status VersionSet::WriteSnapshot(log::Writer* log) { edit.SetComparatorName( cfd->internal_comparator().user_comparator()->Name()); std::string record; - edit.EncodeTo(&record); + if (!edit.EncodeTo(&record)) { + return Status::Corruption( + "Unable to Encode VersionEdit:" + edit.DebugString(true)); + } Status s = log->AddRecord(record); if (!s.ok()) { return s; @@ -2721,15 +3408,20 @@ Status VersionSet::WriteSnapshot(log::Writer* log) { edit.SetColumnFamily(cfd->GetID()); for (int level = 0; level < cfd->NumberLevels(); level++) { - for (const auto& f : cfd->current()->files_[level]) { + for (const auto& f : + cfd->current()->storage_info()->LevelFiles(level)) { edit.AddFile(level, f->fd.GetNumber(), f->fd.GetPathId(), f->fd.GetFileSize(), f->smallest, f->largest, - f->smallest_seqno, f->largest_seqno); + f->smallest_seqno, f->largest_seqno, + f->marked_for_compaction); } } edit.SetLogNumber(cfd->GetLogNumber()); std::string record; - edit.EncodeTo(&record); + if (!edit.EncodeTo(&record)) { + return Status::Corruption( + "Unable to Encode VersionEdit:" + edit.DebugString(true)); + } Status s = log->AddRecord(record); if (!s.ok()) { return s; @@ -2740,68 +3432,112 @@ Status VersionSet::WriteSnapshot(log::Writer* log) { return Status::OK(); } -// Opens the mainfest file and reads all records -// till it finds the record we are looking for. -bool VersionSet::ManifestContains(uint64_t manifest_file_number, - const std::string& record) const { - std::string fname = - DescriptorFileName(dbname_, manifest_file_number); - Log(options_->info_log, "ManifestContains: checking %s\n", fname.c_str()); - unique_ptr file; - Status s = env_->NewSequentialFile(fname, &file, storage_options_); - if (!s.ok()) { - Log(options_->info_log, "ManifestContains: %s\n", s.ToString().c_str()); - Log(options_->info_log, - "ManifestContains: is unable to reopen the manifest file %s", - fname.c_str()); - return false; - } - log::Reader reader(std::move(file), nullptr, true/*checksum*/, 0); - Slice r; - std::string scratch; - bool result = false; - while (reader.ReadRecord(&r, &scratch)) { - if (r == Slice(record)) { - result = true; - break; +// TODO(aekmekji): in CompactionJob::GenSubcompactionBoundaries(), this +// function is called repeatedly with consecutive pairs of slices. For example +// if the slice list is [a, b, c, d] this function is called with arguments +// (a,b) then (b,c) then (c,d). Knowing this, an optimization is possible where +// we avoid doing binary search for the keys b and c twice and instead somehow +// maintain state of where they first appear in the files. +uint64_t VersionSet::ApproximateSize(Version* v, const Slice& start, + const Slice& end, int start_level, + int end_level) { + // pre-condition + assert(v->cfd_->internal_comparator().Compare(start, end) <= 0); + + uint64_t size = 0; + const auto* vstorage = v->storage_info(); + end_level = end_level == -1 + ? vstorage->num_non_empty_levels() + : std::min(end_level, vstorage->num_non_empty_levels()); + + assert(start_level <= end_level); + + for (int level = start_level; level < end_level; level++) { + const LevelFilesBrief& files_brief = vstorage->LevelFilesBrief(level); + if (!files_brief.num_files) { + // empty level, skip exploration + continue; + } + + if (!level) { + // level 0 data is sorted order, handle the use case explicitly + size += ApproximateSizeLevel0(v, files_brief, start, end); + continue; + } + + assert(level > 0); + assert(files_brief.num_files > 0); + + // identify the file position for starting key + const uint64_t idx_start = FindFileInRange( + v->cfd_->internal_comparator(), files_brief, start, + /*start=*/0, static_cast(files_brief.num_files - 1)); + assert(idx_start < files_brief.num_files); + + // scan all files from the starting position until the ending position + // inferred from the sorted order + for (uint64_t i = idx_start; i < files_brief.num_files; i++) { + uint64_t val; + val = ApproximateSize(v, files_brief.files[i], end); + if (!val) { + // the files after this will not have the range + break; + } + + size += val; + + if (i == idx_start) { + // subtract the bytes needed to be scanned to get to the starting + // key + val = ApproximateSize(v, files_brief.files[i], start); + assert(size >= val); + size -= val; + } } } - Log(options_->info_log, "ManifestContains: result = %d\n", result ? 1 : 0); - return result; + + return size; +} + +uint64_t VersionSet::ApproximateSizeLevel0(Version* v, + const LevelFilesBrief& files_brief, + const Slice& key_start, + const Slice& key_end) { + // level 0 files are not in sorted order, we need to iterate through + // the list to compute the total bytes that require scanning + uint64_t size = 0; + for (size_t i = 0; i < files_brief.num_files; i++) { + const uint64_t start = ApproximateSize(v, files_brief.files[i], key_start); + const uint64_t end = ApproximateSize(v, files_brief.files[i], key_end); + assert(end >= start); + size += end - start; + } + return size; } +uint64_t VersionSet::ApproximateSize(Version* v, const FdWithKeyRange& f, + const Slice& key) { + // pre-condition + assert(v); -uint64_t VersionSet::ApproximateOffsetOf(Version* v, const InternalKey& ikey) { uint64_t result = 0; - for (int level = 0; level < v->NumberLevels(); level++) { - const std::vector& files = v->files_[level]; - for (size_t i = 0; i < files.size(); i++) { - if (v->cfd_->internal_comparator().Compare(files[i]->largest, ikey) <= - 0) { - // Entire file is before "ikey", so just add the file size - result += files[i]->fd.GetFileSize(); - } else if (v->cfd_->internal_comparator().Compare(files[i]->smallest, - ikey) > 0) { - // Entire file is after "ikey", so ignore - if (level > 0) { - // Files other than level 0 are sorted by meta->smallest, so - // no further files in this level will contain data for - // "ikey". - break; - } - } else { - // "ikey" falls in the range for this table. Add the - // approximate offset of "ikey" within the table. - TableReader* table_reader_ptr; - Iterator* iter = v->cfd_->table_cache()->NewIterator( - ReadOptions(), storage_options_, v->cfd_->internal_comparator(), - files[i]->fd, &table_reader_ptr); - if (table_reader_ptr != nullptr) { - result += table_reader_ptr->ApproximateOffsetOf(ikey.Encode()); - } - delete iter; - } + if (v->cfd_->internal_comparator().Compare(f.largest_key, key) <= 0) { + // Entire file is before "key", so just add the file size + result = f.fd.GetFileSize(); + } else if (v->cfd_->internal_comparator().Compare(f.smallest_key, key) > 0) { + // Entire file is after "key", so ignore + result = 0; + } else { + // "key" falls in the range for this table. Add the + // approximate offset of "key" within the table. + TableReader* table_reader_ptr; + InternalIterator* iter = v->cfd_->table_cache()->NewIterator( + ReadOptions(), env_options_, v->cfd_->internal_comparator(), f.fd, + nullptr /* range_del_agg */, &table_reader_ptr); + if (table_reader_ptr != nullptr) { + result = table_reader_ptr->ApproximateOffsetOf(key); } + delete iter; } return result; } @@ -2810,70 +3546,97 @@ void VersionSet::AddLiveFiles(std::vector* live_list) { // pre-calculate space requirement int64_t total_files = 0; for (auto cfd : *column_family_set_) { + if (!cfd->initialized()) { + continue; + } Version* dummy_versions = cfd->dummy_versions(); for (Version* v = dummy_versions->next_; v != dummy_versions; v = v->next_) { - for (int level = 0; level < v->NumberLevels(); level++) { - total_files += v->files_[level].size(); + const auto* vstorage = v->storage_info(); + for (int level = 0; level < vstorage->num_levels(); level++) { + total_files += vstorage->LevelFiles(level).size(); } } } // just one time extension to the right size - live_list->reserve(live_list->size() + total_files); + live_list->reserve(live_list->size() + static_cast(total_files)); for (auto cfd : *column_family_set_) { + if (!cfd->initialized()) { + continue; + } + auto* current = cfd->current(); + bool found_current = false; Version* dummy_versions = cfd->dummy_versions(); for (Version* v = dummy_versions->next_; v != dummy_versions; v = v->next_) { - for (int level = 0; level < v->NumberLevels(); level++) { - for (const auto& f : v->files_[level]) { - live_list->push_back(f->fd); - } + v->AddLiveFiles(live_list); + if (v == current) { + found_current = true; } } + if (!found_current && current != nullptr) { + // Should never happen unless it is a bug. + assert(false); + current->AddLiveFiles(live_list); + } } } -Iterator* VersionSet::MakeInputIterator(Compaction* c) { +InternalIterator* VersionSet::MakeInputIterator( + const Compaction* c, RangeDelAggregator* range_del_agg) { auto cfd = c->column_family_data(); ReadOptions read_options; - read_options.verify_checksums = - cfd->options()->verify_checksums_in_compaction; + read_options.verify_checksums = true; read_options.fill_cache = false; + // Compaction iterators shouldn't be confined to a single prefix. + // Compactions use Seek() for + // (a) concurrent compactions, + // (b) CompactionFilter::Decision::kRemoveAndSkipUntil. + read_options.total_order_seek = true; // Level-0 files have to be merged together. For other levels, // we will make a concatenating iterator per level. // TODO(opt): use concatenating iterator for level-0 if there is no overlap - const int space = (c->level() == 0 ? - c->input_levels(0)->num_files + c->num_input_levels() - 1: - c->num_input_levels()); - Iterator** list = new Iterator*[space]; - int num = 0; - for (int which = 0; which < c->num_input_levels(); which++) { + const size_t space = (c->level() == 0 ? c->input_levels(0)->num_files + + c->num_input_levels() - 1 + : c->num_input_levels()); + InternalIterator** list = new InternalIterator* [space]; + size_t num = 0; + for (size_t which = 0; which < c->num_input_levels(); which++) { if (c->input_levels(which)->num_files != 0) { if (c->level(which) == 0) { - const FileLevel* flevel = c->input_levels(which); + const LevelFilesBrief* flevel = c->input_levels(which); for (size_t i = 0; i < flevel->num_files; i++) { list[num++] = cfd->table_cache()->NewIterator( - read_options, storage_options_compactions_, - cfd->internal_comparator(), flevel->files[i].fd, nullptr, - true /* for compaction */); + read_options, env_options_compactions_, + cfd->internal_comparator(), flevel->files[i].fd, range_del_agg, + nullptr /* table_reader_ptr */, + nullptr /* no per level latency histogram */, + true /* for_compaction */, nullptr /* arena */, + false /* skip_filters */, (int)which /* level */); } } else { // Create concatenating iterator for the files from this level - list[num++] = NewTwoLevelIterator(new Version::LevelFileIteratorState( - cfd->table_cache(), read_options, storage_options_, - cfd->internal_comparator(), true /* for_compaction */, - false /* prefix enabled */), - new Version::LevelFileNumIterator(cfd->internal_comparator(), - c->input_levels(which))); + list[num++] = NewTwoLevelIterator( + new LevelFileIteratorState( + cfd->table_cache(), read_options, env_options_compactions_, + cfd->internal_comparator(), + nullptr /* no per level latency histogram */, + true /* for_compaction */, false /* prefix enabled */, + false /* skip_filters */, (int)which /* level */, + range_del_agg), + new LevelFileNumIterator(cfd->internal_comparator(), + c->input_levels(which), + false /* don't sample compaction */)); } } } assert(num <= space); - Iterator* result = NewMergingIterator( - &c->column_family_data()->internal_comparator(), list, num); + InternalIterator* result = + NewMergingIterator(&c->column_family_data()->internal_comparator(), list, + static_cast(num)); delete[] list; return result; } @@ -2883,46 +3646,44 @@ Iterator* VersionSet::MakeInputIterator(Compaction* c) { bool VersionSet::VerifyCompactionFileConsistency(Compaction* c) { #ifndef NDEBUG Version* version = c->column_family_data()->current(); + const VersionStorageInfo* vstorage = version->storage_info(); if (c->input_version() != version) { - Log(options_->info_log, - "[%s] VerifyCompactionFileConsistency version mismatch", + ROCKS_LOG_INFO( + db_options_->info_log, + "[%s] compaction output being applied to a different base version from" + " input version", c->column_family_data()->GetName().c_str()); - } - - // verify files in level - int level = c->level(); - for (int i = 0; i < c->num_input_files(0); i++) { - uint64_t number = c->input(0, i)->fd.GetNumber(); - // look for this file in the current version - bool found = false; - for (unsigned int j = 0; j < version->files_[level].size(); j++) { - FileMetaData* f = version->files_[level][j]; - if (f->fd.GetNumber() == number) { - found = true; - break; + if (vstorage->compaction_style_ == kCompactionStyleLevel && + c->start_level() == 0 && c->num_input_levels() > 2U) { + // We are doing a L0->base_level compaction. The assumption is if + // base level is not L1, levels from L1 to base_level - 1 is empty. + // This is ensured by having one compaction from L0 going on at the + // same time in level-based compaction. So that during the time, no + // compaction/flush can put files to those levels. + for (int l = c->start_level() + 1; l < c->output_level(); l++) { + if (vstorage->NumLevelFiles(l) != 0) { + return false; + } } } - if (!found) { - return false; // input files non existant in current version - } } - // verify level+1 files - level++; - for (int i = 0; i < c->num_input_files(1); i++) { - uint64_t number = c->input(1, i)->fd.GetNumber(); - // look for this file in the current version - bool found = false; - for (unsigned int j = 0; j < version->files_[level].size(); j++) { - FileMetaData* f = version->files_[level][j]; - if (f->fd.GetNumber() == number) { - found = true; - break; + for (size_t input = 0; input < c->num_input_levels(); ++input) { + int level = c->level(input); + for (size_t i = 0; i < c->num_input_files(input); ++i) { + uint64_t number = c->input(input, i)->fd.GetNumber(); + bool found = false; + for (size_t j = 0; j < vstorage->files_[level].size(); j++) { + FileMetaData* f = vstorage->files_[level][j]; + if (f->fd.GetNumber() == number) { + found = true; + break; + } + } + if (!found) { + return false; // input files non existent in current version } - } - if (!found) { - return false; // input files non existant in current version } } #endif @@ -2933,9 +3694,13 @@ Status VersionSet::GetMetadataForFile(uint64_t number, int* filelevel, FileMetaData** meta, ColumnFamilyData** cfd) { for (auto cfd_iter : *column_family_set_) { + if (!cfd_iter->initialized()) { + continue; + } Version* version = cfd_iter->current(); - for (int level = 0; level < version->NumberLevels(); level++) { - for (const auto& file : version->files_[level]) { + const auto* vstorage = version->storage_info(); + for (int level = 0; level < vstorage->num_levels(); level++) { + for (const auto& file : vstorage->LevelFiles(level)) { if (file->fd.GetNumber() == number) { *meta = file; *filelevel = level; @@ -2950,16 +3715,20 @@ Status VersionSet::GetMetadataForFile(uint64_t number, int* filelevel, void VersionSet::GetLiveFilesMetaData(std::vector* metadata) { for (auto cfd : *column_family_set_) { + if (cfd->IsDropped() || !cfd->initialized()) { + continue; + } for (int level = 0; level < cfd->NumberLevels(); level++) { - for (const auto& file : cfd->current()->files_[level]) { + for (const auto& file : + cfd->current()->storage_info()->LevelFiles(level)) { LiveFileMetaData filemetadata; filemetadata.column_family_name = cfd->GetName(); uint32_t path_id = file->fd.GetPathId(); - if (path_id < options_->db_paths.size()) { - filemetadata.db_path = options_->db_paths[path_id].path; + if (path_id < db_options_->db_paths.size()) { + filemetadata.db_path = db_options_->db_paths[path_id].path; } else { - assert(!options_->db_paths.empty()); - filemetadata.db_path = options_->db_paths.back().path; + assert(!db_options_->db_paths.empty()); + filemetadata.db_path = db_options_->db_paths.back().path; } filemetadata.name = MakeTableFileName("", file->fd.GetNumber()); filemetadata.level = level; @@ -2974,25 +3743,72 @@ void VersionSet::GetLiveFilesMetaData(std::vector* metadata) { } } -void VersionSet::GetObsoleteFiles(std::vector* files) { - files->insert(files->end(), obsolete_files_.begin(), obsolete_files_.end()); - obsolete_files_.clear(); +void VersionSet::GetObsoleteFiles(std::vector* files, + std::vector* manifest_filenames, + uint64_t min_pending_output) { + assert(manifest_filenames->empty()); + obsolete_manifests_.swap(*manifest_filenames); + std::vector pending_files; + for (auto f : obsolete_files_) { + if (f->fd.GetNumber() < min_pending_output) { + files->push_back(f); + } else { + pending_files.push_back(f); + } + } + obsolete_files_.swap(pending_files); } ColumnFamilyData* VersionSet::CreateColumnFamily( - const ColumnFamilyOptions& options, VersionEdit* edit) { + const ColumnFamilyOptions& cf_options, VersionEdit* edit) { assert(edit->is_column_family_add_); Version* dummy_versions = new Version(nullptr, this); + // Ref() dummy version once so that later we can call Unref() to delete it + // by avoiding calling "delete" explicitly (~Version is private) + dummy_versions->Ref(); auto new_cfd = column_family_set_->CreateColumnFamily( - edit->column_family_name_, edit->column_family_, dummy_versions, options); + edit->column_family_name_, edit->column_family_, dummy_versions, + cf_options); Version* v = new Version(new_cfd, this, current_version_number_++); + // Fill level target base information. + v->storage_info()->CalculateBaseBytes(*new_cfd->ioptions(), + *new_cfd->GetLatestMutableCFOptions()); AppendVersion(new_cfd, v); - new_cfd->CreateNewMemtable(); + // GetLatestMutableCFOptions() is safe here without mutex since the + // cfd is not available to client + new_cfd->CreateNewMemtable(*new_cfd->GetLatestMutableCFOptions(), + LastSequence()); new_cfd->SetLogNumber(edit->log_number_); return new_cfd; } +uint64_t VersionSet::GetNumLiveVersions(Version* dummy_versions) { + uint64_t count = 0; + for (Version* v = dummy_versions->next_; v != dummy_versions; v = v->next_) { + count++; + } + return count; +} + +uint64_t VersionSet::GetTotalSstFilesSize(Version* dummy_versions) { + std::unordered_set unique_files; + uint64_t total_files_size = 0; + for (Version* v = dummy_versions->next_; v != dummy_versions; v = v->next_) { + VersionStorageInfo* storage_info = v->storage_info(); + for (int level = 0; level < storage_info->num_levels_; level++) { + for (const auto& file_meta : storage_info->LevelFiles(level)) { + if (unique_files.find(file_meta->fd.packed_number_and_path_id) == + unique_files.end()) { + unique_files.insert(file_meta->fd.packed_number_and_path_id); + total_files_size += file_meta->fd.GetFileSize(); + } + } + } + } + return total_files_size; +} + } // namespace rocksdb diff --git a/db/version_set.h b/db/version_set.h index 2f6d477a1db..5862dea3350 100644 --- a/db/version_set.h +++ b/db/version_set.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -18,37 +18,47 @@ // synchronization on all accesses. #pragma once +#include +#include +#include #include #include #include +#include +#include #include -#include -#include -#include -#include "db/dbformat.h" -#include "db/version_edit.h" -#include "port/port.h" -#include "db/table_cache.h" + +#include "db/column_family.h" #include "db/compaction.h" #include "db/compaction_picker.h" -#include "db/column_family.h" -#include "db/log_reader.h" +#include "db/dbformat.h" #include "db/file_indexer.h" +#include "db/log_reader.h" +#include "db/range_del_aggregator.h" +#include "db/table_cache.h" +#include "db/version_builder.h" +#include "db/version_edit.h" +#include "db/write_controller.h" +#include "monitoring/instrumented_mutex.h" +#include "options/db_options.h" +#include "port/port.h" +#include "rocksdb/env.h" namespace rocksdb { -namespace log { class Writer; } +namespace log { +class Writer; +} class Compaction; -class CompactionPicker; -class Iterator; +class InternalIterator; class LogBuffer; class LookupKey; class MemTable; class Version; class VersionSet; +class WriteBufferManager; class MergeContext; -class ColumnFamilyData; class ColumnFamilySet; class TableCache; class MergeIteratorBuilder; @@ -58,8 +68,7 @@ class MergeIteratorBuilder; // REQUIRES: "file_level.files" contains a sorted list of // non-overlapping files. extern int FindFile(const InternalKeyComparator& icmp, - const FileLevel& file_level, - const Slice& key); + const LevelFilesBrief& file_level, const Slice& key); // Returns true iff some file in "files" overlaps the user key range // [*smallest,*largest]. @@ -67,97 +76,135 @@ extern int FindFile(const InternalKeyComparator& icmp, // largest==nullptr represents a key largest than all keys in the DB. // REQUIRES: If disjoint_sorted_files, file_level.files[] // contains disjoint ranges in sorted order. -extern bool SomeFileOverlapsRange( - const InternalKeyComparator& icmp, - bool disjoint_sorted_files, - const FileLevel& file_level, - const Slice* smallest_user_key, - const Slice* largest_user_key); - -// Generate FileLevel from vector +extern bool SomeFileOverlapsRange(const InternalKeyComparator& icmp, + bool disjoint_sorted_files, + const LevelFilesBrief& file_level, + const Slice* smallest_user_key, + const Slice* largest_user_key); + +// Generate LevelFilesBrief from vector // Would copy smallest_key and largest_key data to sequential memory // arena: Arena used to allocate the memory -extern void DoGenerateFileLevel(FileLevel* file_level, - const std::vector& files, - Arena* arena); +extern void DoGenerateLevelFilesBrief(LevelFilesBrief* file_level, + const std::vector& files, + Arena* arena); -class Version { +class VersionStorageInfo { public: - // Append to *iters a sequence of iterators that will - // yield the contents of this Version when merged together. - // REQUIRES: This version has been saved (see VersionSet::SaveTo) - void AddIterators(const ReadOptions&, const EnvOptions& soptions, - std::vector* iters); + VersionStorageInfo(const InternalKeyComparator* internal_comparator, + const Comparator* user_comparator, int num_levels, + CompactionStyle compaction_style, + VersionStorageInfo* src_vstorage, + bool _force_consistency_checks); + ~VersionStorageInfo(); - void AddIterators(const ReadOptions&, const EnvOptions& soptions, - MergeIteratorBuilder* merger_iter_builder); + void Reserve(int level, size_t size) { files_[level].reserve(size); } - // Lookup the value for key. If found, store it in *val and - // return OK. Else return a non-OK status. - // Uses *operands to store merge_operator operations to apply later - // REQUIRES: lock is not held - void Get(const ReadOptions&, const LookupKey& key, std::string* val, - Status* status, MergeContext* merge_context, - bool* value_found = nullptr); + void AddFile(int level, FileMetaData* f, Logger* info_log = nullptr); + + void SetFinalized(); + + // Update num_non_empty_levels_. + void UpdateNumNonEmptyLevels(); + + void GenerateFileIndexer() { + file_indexer_.UpdateIndex(&arena_, num_non_empty_levels_, files_); + } + + // Update the accumulated stats from a file-meta. + void UpdateAccumulatedStats(FileMetaData* file_meta); + + // Decrease the current stat form a to-be-delected file-meta + void RemoveCurrentStats(FileMetaData* file_meta); + + void ComputeCompensatedSizes(); // Updates internal structures that keep track of compaction scores // We use compaction scores to figure out which compaction to do next - // REQUIRES: If Version is not yet saved to current_, it can be called without - // a lock. Once a version is saved to current_, call only with mutex held - void ComputeCompactionScore(std::vector& size_being_compacted); + // REQUIRES: db_mutex held!! + // TODO find a better way to pass compaction_options_fifo. + void ComputeCompactionScore(const ImmutableCFOptions& immutable_cf_options, + const MutableCFOptions& mutable_cf_options); - // Generate file_levels_ from files_ - void GenerateFileLevels(); + // Estimate est_comp_needed_bytes_ + void EstimateCompactionBytesNeeded( + const MutableCFOptions& mutable_cf_options); - // Update scores, pre-calculated variables. It needs to be called before - // applying the version to the version set. - void PrepareApply(std::vector& size_being_compacted); + // This computes files_marked_for_compaction_ and is called by + // ComputeCompactionScore() + void ComputeFilesMarkedForCompaction(); - // Reference count management (so Versions do not disappear out from - // under live iterators) - void Ref(); - // Decrease reference count. Delete the object if no reference left - // and return true. Otherwise, return false. - bool Unref(); + // Generate level_files_brief_ from files_ + void GenerateLevelFilesBrief(); + // Sort all files for this version based on their file size and + // record results in files_by_compaction_pri_. The largest files are listed + // first. + void UpdateFilesByCompactionPri(CompactionPri compaction_pri); + + void GenerateLevel0NonOverlapping(); + bool level0_non_overlapping() const { + return level0_non_overlapping_; + } - // Returns true iff some level needs a compaction. - bool NeedsCompaction() const; + int MaxInputLevel() const; + int MaxOutputLevel(bool allow_ingest_behind) const; - // Returns the maxmimum compaction score for levels 1 to max - double MaxCompactionScore() const { return max_compaction_score_; } + // Return level number that has idx'th highest score + int CompactionScoreLevel(int idx) const { return compaction_level_[idx]; } - // See field declaration - int MaxCompactionScoreLevel() const { return max_compaction_score_level_; } + // Return idx'th highest score + double CompactionScore(int idx) const { return compaction_score_[idx]; } void GetOverlappingInputs( - int level, - const InternalKey* begin, // nullptr means before all keys - const InternalKey* end, // nullptr means after all keys + int level, const InternalKey* begin, // nullptr means before all keys + const InternalKey* end, // nullptr means after all keys std::vector* inputs, - int hint_index = -1, // index of overlap file - int* file_index = nullptr); // return index of overlap file - - void GetOverlappingInputsBinarySearch( - int level, - const Slice& begin, // nullptr means before all keys - const Slice& end, // nullptr means after all keys + int hint_index = -1, // index of overlap file + int* file_index = nullptr, // return index of overlap file + bool expand_range = true) // if set, returns files which overlap the + const; // range and overlap each other. If false, + // then just files intersecting the range + void GetCleanInputsWithinInterval( + int level, const InternalKey* begin, // nullptr means before all keys + const InternalKey* end, // nullptr means after all keys + std::vector* inputs, + int hint_index = -1, // index of overlap file + int* file_index = nullptr) // return index of overlap file + const; + + void GetOverlappingInputsRangeBinarySearch( + int level, // level > 0 + const Slice& begin, // nullptr means before all keys + const Slice& end, // nullptr means after all keys std::vector* inputs, - int hint_index, // index of overlap file - int* file_index); // return index of overlap file + int hint_index, // index of overlap file + int* file_index, // return index of overlap file + bool within_interval = false) // if set, force the inputs within interval + const; - void ExtendOverlappingInputs( + void ExtendFileRangeOverlappingInterval( int level, - const Slice& begin, // nullptr means before all keys - const Slice& end, // nullptr means after all keys - std::vector* inputs, - unsigned int index); // start extending from this index + const Slice& begin, // nullptr means before all keys + const Slice& end, // nullptr means after all keys + unsigned int index, // start extending from this index + int* startIndex, // return the startIndex of input range + int* endIndex) // return the endIndex of input range + const; + + void ExtendFileRangeWithinInterval( + int level, + const Slice& begin, // nullptr means before all keys + const Slice& end, // nullptr means after all keys + unsigned int index, // start extending from this index + int* startIndex, // return the startIndex of input range + int* endIndex) // return the endIndex of input range + const; // Returns true iff some file in the specified level overlaps // some part of [*smallest_user_key,*largest_user_key]. // smallest_user_key==NULL represents a key smaller than all keys in the DB. // largest_user_key==NULL represents a key largest than all keys in the DB. - bool OverlapInLevel(int level, - const Slice* smallest_user_key, + bool OverlapInLevel(int level, const Slice* smallest_user_key, const Slice* largest_user_key); // Returns true iff the first or last file in inputs contains @@ -167,27 +214,89 @@ class Version { bool HasOverlappingUserKey(const std::vector* inputs, int level); + int num_levels() const { return num_levels_; } + + // REQUIRES: This version has been saved (see VersionSet::SaveTo) + int num_non_empty_levels() const { + assert(finalized_); + return num_non_empty_levels_; + } - // Return the level at which we should place a new memtable compaction - // result that covers the range [smallest_user_key,largest_user_key]. - int PickLevelForMemTableOutput(const Slice& smallest_user_key, - const Slice& largest_user_key); + // REQUIRES: This version has been finalized. + // (CalculateBaseBytes() is called) + // This may or may not return number of level files. It is to keep backward + // compatible behavior in universal compaction. + int l0_delay_trigger_count() const { return l0_delay_trigger_count_; } - int NumberLevels() const { return num_levels_; } + void set_l0_delay_trigger_count(int v) { l0_delay_trigger_count_ = v; } - // REQUIRES: lock is held - int NumLevelFiles(int level) const { return files_[level].size(); } + // REQUIRES: This version has been saved (see VersionSet::SaveTo) + int NumLevelFiles(int level) const { + assert(finalized_); + return static_cast(files_[level].size()); + } // Return the combined file size of all files at the specified level. - int64_t NumLevelBytes(int level) const; + uint64_t NumLevelBytes(int level) const; + + // REQUIRES: This version has been saved (see VersionSet::SaveTo) + const std::vector& LevelFiles(int level) const { + return files_[level]; + } + + const rocksdb::LevelFilesBrief& LevelFilesBrief(int level) const { + assert(level < static_cast(level_files_brief_.size())); + return level_files_brief_[level]; + } + + // REQUIRES: This version has been saved (see VersionSet::SaveTo) + const std::vector& FilesByCompactionPri(int level) const { + assert(finalized_); + return files_by_compaction_pri_[level]; + } + + // REQUIRES: This version has been saved (see VersionSet::SaveTo) + // REQUIRES: DB mutex held during access + const autovector>& FilesMarkedForCompaction() + const { + assert(finalized_); + return files_marked_for_compaction_; + } + + int base_level() const { return base_level_; } + + // REQUIRES: lock is held + // Set the index that is used to offset into files_by_compaction_pri_ to find + // the next compaction candidate file. + void SetNextCompactionIndex(int level, int index) { + next_file_to_compact_by_size_[level] = index; + } + + // REQUIRES: lock is held + int NextCompactionIndex(int level) const { + return next_file_to_compact_by_size_[level]; + } + + // REQUIRES: This version has been saved (see VersionSet::SaveTo) + const FileIndexer& file_indexer() const { + assert(finalized_); + return file_indexer_; + } + + // Only the first few entries of files_by_compaction_pri_ are sorted. + // There is no need to sort all the files because it is likely + // that on a running system, we need to look at only the first + // few largest files because a new version is created every few + // seconds/minutes (because of concurrent compactions). + static const size_t kNumberFilesToSort = 50; // Return a human-readable short (single-line) summary of the number // of files per level. Uses *scratch as backing store. struct LevelSummaryStorage { - char buffer[100]; + char buffer[1000]; }; struct FileSummaryStorage { - char buffer[1000]; + char buffer[3000]; }; const char* LevelSummary(LevelSummaryStorage* scratch) const; // Return a human-readable short (single-line) summary of files @@ -198,124 +307,104 @@ class Version { // file at a level >= 1. int64_t MaxNextLevelOverlappingBytes(); - // Add all files listed in the current version to *live. - void AddLiveFiles(std::vector* live); - // Return a human readable string that describes this version's contents. std::string DebugString(bool hex = false) const; - // Returns the version nuber of this version - uint64_t GetVersionNumber() const { return version_number_; } - uint64_t GetAverageValueSize() const { - if (num_non_deletions_ == 0) { + if (accumulated_num_non_deletions_ == 0) { return 0; } - assert(total_raw_key_size_ + total_raw_value_size_ > 0); - assert(total_file_size_ > 0); - return total_raw_value_size_ / num_non_deletions_ * total_file_size_ / - (total_raw_key_size_ + total_raw_value_size_); + assert(accumulated_raw_key_size_ + accumulated_raw_value_size_ > 0); + assert(accumulated_file_size_ > 0); + return accumulated_raw_value_size_ / accumulated_num_non_deletions_ * + accumulated_file_size_ / + (accumulated_raw_key_size_ + accumulated_raw_value_size_); } - // REQUIRES: lock is held - // On success, "tp" will contains the table properties of the file - // specified in "file_meta". If the file name of "file_meta" is - // known ahread, passing it by a non-null "fname" can save a - // file-name conversion. - Status GetTableProperties(std::shared_ptr* tp, - const FileMetaData* file_meta, - const std::string* fname = nullptr); + uint64_t GetEstimatedActiveKeys() const; - // REQUIRES: lock is held - // On success, *props will be populated with all SSTables' table properties. - // The keys of `props` are the sst file name, the values of `props` are the - // tables' propertis, represented as shared_ptr. - Status GetPropertiesOfAllTables(TablePropertiesCollection* props); + double GetEstimatedCompressionRatioAtLevel(int level) const; - uint64_t GetEstimatedActiveKeys(); + // re-initializes the index that is used to offset into + // files_by_compaction_pri_ + // to find the next compaction candidate file. + void ResetNextCompactionIndex(int level) { + next_file_to_compact_by_size_[level] = 0; + } - size_t GetMemoryUsageByTableReaders(); + const InternalKeyComparator* InternalComparator() { + return internal_comparator_; + } - // used to sort files by size - struct Fsize { - int index; - FileMetaData* file; - }; + // Returns maximum total bytes of data on a given level. + uint64_t MaxBytesForLevel(int level) const; - private: - friend class Compaction; - friend class VersionSet; - friend class DBImpl; - friend class ColumnFamilyData; - friend class CompactionPicker; - friend class LevelCompactionPicker; - friend class UniversalCompactionPicker; - friend class FIFOCompactionPicker; - friend class ForwardIterator; - friend class InternalStats; - - class LevelFileNumIterator; - class LevelFileIteratorState; - - bool PrefixMayMatch(const ReadOptions& options, Iterator* level_iter, - const Slice& internal_prefix) const; + // Must be called after any change to MutableCFOptions. + void CalculateBaseBytes(const ImmutableCFOptions& ioptions, + const MutableCFOptions& options); - // Update num_non_empty_levels_. - void UpdateNumNonEmptyLevels(); + // Returns an estimate of the amount of live data in bytes. + uint64_t EstimateLiveDataSize() const; - // The helper function of UpdateTemporaryStats, which may fill the missing - // fields of file_mata from its associated TableProperties. - // Returns true if it does initialize FileMetaData. - bool MaybeInitializeFileMetaData(FileMetaData* file_meta); + uint64_t estimated_compaction_needed_bytes() const { + return estimated_compaction_needed_bytes_; + } - // Update the temporary stats associated with the current version. - // This temporary stats will be used in compaction. - void UpdateTemporaryStats(); + void TEST_set_estimated_compaction_needed_bytes(uint64_t v) { + estimated_compaction_needed_bytes_ = v; + } - // Sort all files for this version based on their file size and - // record results in files_by_size_. The largest files are listed first. - void UpdateFilesBySize(); + bool force_consistency_checks() const { return force_consistency_checks_; } - ColumnFamilyData* cfd_; // ColumnFamilyData to which this Version belongs + private: const InternalKeyComparator* internal_comparator_; const Comparator* user_comparator_; - TableCache* table_cache_; - const MergeOperator* merge_operator_; - - autovector file_levels_; // A copy of list of files per level - Logger* info_log_; - Statistics* db_statistics_; - int num_levels_; // Number of levels - int num_non_empty_levels_; // Number of levels. Any level larger than it - // is guaranteed to be empty. + int num_levels_; // Number of levels + int num_non_empty_levels_; // Number of levels. Any level larger than it + // is guaranteed to be empty. + // Per-level max bytes + std::vector level_max_bytes_; + + // A short brief metadata of files per level + autovector level_files_brief_; FileIndexer file_indexer_; - VersionSet* vset_; // VersionSet to which this Version belongs - Arena arena_; // Used to allocate space for file_levels_ - Version* next_; // Next version in linked list - Version* prev_; // Previous version in linked list - int refs_; // Number of live refs to this version + Arena arena_; // Used to allocate space for file_levels_ + + CompactionStyle compaction_style_; // List of files per level, files in each level are arranged // in increasing order of keys std::vector* files_; + // Level that L0 data should be compacted to. All levels < base_level_ should + // be empty. -1 if it is not level-compaction so it's not applicable. + int base_level_; + // A list for the same set of files that are stored in files_, // but files in each level are now sorted based on file // size. The file with the largest size is at the front. // This vector stores the index of the file from files_. - std::vector> files_by_size_; + std::vector> files_by_compaction_pri_; - // An index into files_by_size_ that specifies the first + // If true, means that files in L0 have keys with non overlapping ranges + bool level0_non_overlapping_; + + // An index into files_by_compaction_pri_ that specifies the first // file that is not yet compacted std::vector next_file_to_compact_by_size_; - // Only the first few entries of files_by_size_ are sorted. + // Only the first few entries of files_by_compaction_pri_ are sorted. // There is no need to sort all the files because it is likely // that on a running system, we need to look at only the first // few largest files because a new version is created every few // seconds/minutes (because of concurrent compactions). static const size_t number_of_files_to_sort_ = 50; + // This vector contains list of files marked for compaction and also not + // currently being compacted. It is protected by DB mutex. It is calculated in + // ComputeCompactionScore() + autovector> files_marked_for_compaction_; + // Level that should be compacted next and its compaction score. // Score < 1 means compaction is not strictly needed. These fields // are initialized by Finalize(). @@ -323,8 +412,194 @@ class Version { // These are used to pick the best compaction level std::vector compaction_score_; std::vector compaction_level_; - double max_compaction_score_; // max score in l1 to ln-1 - int max_compaction_score_level_; // level on which max score occurs + int l0_delay_trigger_count_ = 0; // Count used to trigger slow down and stop + // for number of L0 files. + + // the following are the sampled temporary stats. + // the current accumulated size of sampled files. + uint64_t accumulated_file_size_; + // the current accumulated size of all raw keys based on the sampled files. + uint64_t accumulated_raw_key_size_; + // the current accumulated size of all raw keys based on the sampled files. + uint64_t accumulated_raw_value_size_; + // total number of non-deletion entries + uint64_t accumulated_num_non_deletions_; + // total number of deletion entries + uint64_t accumulated_num_deletions_; + // current number of non_deletion entries + uint64_t current_num_non_deletions_; + // current number of delection entries + uint64_t current_num_deletions_; + // current number of file samples + uint64_t current_num_samples_; + // Estimated bytes needed to be compacted until all levels' size is down to + // target sizes. + uint64_t estimated_compaction_needed_bytes_; + + bool finalized_; + + // If set to true, we will run consistency checks even if RocksDB + // is compiled in release mode + bool force_consistency_checks_; + + friend class Version; + friend class VersionSet; + // No copying allowed + VersionStorageInfo(const VersionStorageInfo&) = delete; + void operator=(const VersionStorageInfo&) = delete; +}; + +class Version { + public: + // Append to *iters a sequence of iterators that will + // yield the contents of this Version when merged together. + // REQUIRES: This version has been saved (see VersionSet::SaveTo) + void AddIterators(const ReadOptions&, const EnvOptions& soptions, + MergeIteratorBuilder* merger_iter_builder, + RangeDelAggregator* range_del_agg); + + void AddIteratorsForLevel(const ReadOptions&, const EnvOptions& soptions, + MergeIteratorBuilder* merger_iter_builder, + int level, RangeDelAggregator* range_del_agg); + + void AddRangeDelIteratorsForLevel( + const ReadOptions& read_options, const EnvOptions& soptions, int level, + std::vector* range_del_iters); + + // Lookup the value for key. If found, store it in *val and + // return OK. Else return a non-OK status. + // Uses *operands to store merge_operator operations to apply later. + // + // If the ReadOptions.read_tier is set to do a read-only fetch, then + // *value_found will be set to false if it cannot be determined whether + // this value exists without doing IO. + // + // If the key is Deleted, *status will be set to NotFound and + // *key_exists will be set to true. + // If no key was found, *status will be set to NotFound and + // *key_exists will be set to false. + // If seq is non-null, *seq will be set to the sequence number found + // for the key if a key was found. + // + // REQUIRES: lock is not held + void Get(const ReadOptions&, const LookupKey& key, PinnableSlice* value, + Status* status, MergeContext* merge_context, + RangeDelAggregator* range_del_agg, bool* value_found = nullptr, + bool* key_exists = nullptr, SequenceNumber* seq = nullptr, + bool* is_blob = nullptr); + + // Loads some stats information from files. Call without mutex held. It needs + // to be called before applying the version to the version set. + void PrepareApply(const MutableCFOptions& mutable_cf_options, + bool update_stats); + + // Reference count management (so Versions do not disappear out from + // under live iterators) + void Ref(); + // Decrease reference count. Delete the object if no reference left + // and return true. Otherwise, return false. + bool Unref(); + + // Add all files listed in the current version to *live. + void AddLiveFiles(std::vector* live); + + // Return a human readable string that describes this version's contents. + std::string DebugString(bool hex = false, bool print_stats = false) const; + + // Returns the version nuber of this version + uint64_t GetVersionNumber() const { return version_number_; } + + // REQUIRES: lock is held + // On success, "tp" will contains the table properties of the file + // specified in "file_meta". If the file name of "file_meta" is + // known ahread, passing it by a non-null "fname" can save a + // file-name conversion. + Status GetTableProperties(std::shared_ptr* tp, + const FileMetaData* file_meta, + const std::string* fname = nullptr) const; + + // REQUIRES: lock is held + // On success, *props will be populated with all SSTables' table properties. + // The keys of `props` are the sst file name, the values of `props` are the + // tables' propertis, represented as shared_ptr. + Status GetPropertiesOfAllTables(TablePropertiesCollection* props); + Status GetPropertiesOfAllTables(TablePropertiesCollection* props, int level); + Status GetPropertiesOfTablesInRange(const Range* range, std::size_t n, + TablePropertiesCollection* props) const; + + // REQUIRES: lock is held + // On success, "tp" will contains the aggregated table property amoug + // the table properties of all sst files in this version. + Status GetAggregatedTableProperties( + std::shared_ptr* tp, int level = -1); + + uint64_t GetEstimatedActiveKeys() { + return storage_info_.GetEstimatedActiveKeys(); + } + + size_t GetMemoryUsageByTableReaders(); + + ColumnFamilyData* cfd() const { return cfd_; } + + // Return the next Version in the linked list. Used for debug only + Version* TEST_Next() const { + return next_; + } + + int TEST_refs() const { return refs_; } + + VersionStorageInfo* storage_info() { return &storage_info_; } + + VersionSet* version_set() { return vset_; } + + void GetColumnFamilyMetaData(ColumnFamilyMetaData* cf_meta); + + private: + Env* env_; + friend class VersionSet; + + const InternalKeyComparator* internal_comparator() const { + return storage_info_.internal_comparator_; + } + const Comparator* user_comparator() const { + return storage_info_.user_comparator_; + } + + bool PrefixMayMatch(const ReadOptions& read_options, + InternalIterator* level_iter, + const Slice& internal_prefix) const; + + // Returns true if the filter blocks in the specified level will not be + // checked during read operations. In certain cases (trivial move or preload), + // the filter block may already be cached, but we still do not access it such + // that it eventually expires from the cache. + bool IsFilterSkipped(int level, bool is_file_last_in_level = false); + + // The helper function of UpdateAccumulatedStats, which may fill the missing + // fields of file_mata from its associated TableProperties. + // Returns true if it does initialize FileMetaData. + bool MaybeInitializeFileMetaData(FileMetaData* file_meta); + + // Update the accumulated stats associated with the current version. + // This accumulated stats will be used in compaction. + void UpdateAccumulatedStats(bool update_stats); + + // Sort all files for this version based on their file size and + // record results in files_by_compaction_pri_. The largest files are listed + // first. + void UpdateFilesByCompactionPri(); + + ColumnFamilyData* cfd_; // ColumnFamilyData to which this Version belongs + Logger* info_log_; + Statistics* db_statistics_; + TableCache* table_cache_; + const MergeOperator* merge_operator_; + + VersionStorageInfo storage_info_; + VersionSet* vset_; // VersionSet to which this Version belongs + Version* next_; // Next version in linked list + Version* prev_; // Previous version in linked list + int refs_; // Number of live refs to this version // A version number that uniquely represents this version. This is // used for debugging and logging purposes only. @@ -332,25 +607,8 @@ class Version { Version(ColumnFamilyData* cfd, VersionSet* vset, uint64_t version_number = 0); - // total file size - uint64_t total_file_size_; - // the total size of all raw keys. - uint64_t total_raw_key_size_; - // the total size of all raw values. - uint64_t total_raw_value_size_; - // total number of non-deletion entries - uint64_t num_non_deletions_; - // total number of deletion entries - uint64_t num_deletions_; - ~Version(); - // re-initializes the index that is used to offset into files_by_size_ - // to find the next compaction candidate file. - void ResetNextCompactionIndex(int level) { - next_file_to_compact_by_size_[level] = 0; - } - // No copying allowed Version(const Version&); void operator=(const Version&); @@ -358,8 +616,10 @@ class Version { class VersionSet { public: - VersionSet(const std::string& dbname, const DBOptions* options, - const EnvOptions& storage_options, Cache* table_cache); + VersionSet(const std::string& dbname, const ImmutableDBOptions* db_options, + const EnvOptions& env_options, Cache* table_cache, + WriteBufferManager* write_buffer_manager, + WriteController* write_controller); ~VersionSet(); // Apply *edit to the current version to form a new descriptor that @@ -368,11 +628,25 @@ class VersionSet { // column_family_options has to be set if edit is column family add // REQUIRES: *mu is held on entry. // REQUIRES: no other thread concurrently calls LogAndApply() - Status LogAndApply(ColumnFamilyData* column_family_data, VersionEdit* edit, - port::Mutex* mu, Directory* db_directory = nullptr, - bool new_descriptor_log = false, - const ColumnFamilyOptions* column_family_options = - nullptr); + Status LogAndApply( + ColumnFamilyData* column_family_data, + const MutableCFOptions& mutable_cf_options, VersionEdit* edit, + InstrumentedMutex* mu, Directory* db_directory = nullptr, + bool new_descriptor_log = false, + const ColumnFamilyOptions* column_family_options = nullptr) { + autovector edit_list; + edit_list.push_back(edit); + return LogAndApply(column_family_data, mutable_cf_options, edit_list, mu, + db_directory, new_descriptor_log, column_family_options); + } + // The batch version. If edit_list.size() > 1, caller must ensure that + // no edit in the list column family add or drop + Status LogAndApply( + ColumnFamilyData* column_family_data, + const MutableCFOptions& mutable_cf_options, + const autovector& edit_list, InstrumentedMutex* mu, + Directory* db_directory = nullptr, bool new_descriptor_log = false, + const ColumnFamilyOptions* column_family_options = nullptr); // Recover the last saved descriptor from persistent storage. // If read_only == true, Recover() will not complain if some column families @@ -397,58 +671,75 @@ class VersionSet { // among [4-6] contains files. static Status ReduceNumberOfLevels(const std::string& dbname, const Options* options, - const EnvOptions& storage_options, + const EnvOptions& env_options, int new_levels); // printf contents (for debugging) Status DumpManifest(Options& options, std::string& manifestFileName, - bool verbose, bool hex = false); + bool verbose, bool hex = false, bool json = false); #endif // ROCKSDB_LITE // Return the current manifest file number - uint64_t ManifestFileNumber() const { return manifest_file_number_; } + uint64_t manifest_file_number() const { return manifest_file_number_; } - uint64_t PendingManifestFileNumber() const { + uint64_t options_file_number() const { return options_file_number_; } + + uint64_t pending_manifest_file_number() const { return pending_manifest_file_number_; } + uint64_t current_next_file_number() const { return next_file_number_.load(); } + // Allocate and return a new file number - uint64_t NewFileNumber() { return next_file_number_++; } - - // Arrange to reuse "file_number" unless a newer file number has - // already been allocated. - // REQUIRES: "file_number" was returned by a call to NewFileNumber(). - void ReuseLogFileNumber(uint64_t file_number) { - if (next_file_number_ == file_number + 1) { - next_file_number_ = file_number; - } - } + uint64_t NewFileNumber() { return next_file_number_.fetch_add(1); } // Return the last sequence number. uint64_t LastSequence() const { return last_sequence_.load(std::memory_order_acquire); } + // Note: memory_order_acquire must be sufficient. + uint64_t LastToBeWrittenSequence() const { + return last_to_be_written_sequence_.load(std::memory_order_seq_cst); + } + // Set the last sequence number to s. void SetLastSequence(uint64_t s) { assert(s >= last_sequence_); + // Last visible seqeunce must always be less than last written seq + assert(!db_options_->concurrent_prepare || + s <= last_to_be_written_sequence_); last_sequence_.store(s, std::memory_order_release); } + // Note: memory_order_release must be sufficient + void SetLastToBeWrittenSequence(uint64_t s) { + assert(s >= last_to_be_written_sequence_); + last_to_be_written_sequence_.store(s, std::memory_order_seq_cst); + } + + // Note: memory_order_release must be sufficient + uint64_t FetchAddLastToBeWrittenSequence(uint64_t s) { + return last_to_be_written_sequence_.fetch_add(s, std::memory_order_seq_cst); + } + // Mark the specified file number as used. - void MarkFileNumberUsed(uint64_t number); + // REQUIRED: this is only called during single-threaded recovery + void MarkFileNumberUsedDuringRecovery(uint64_t number); // Return the log file number for the log file that is currently // being compacted, or zero if there is no such log file. - uint64_t PrevLogNumber() const { return prev_log_number_; } + uint64_t prev_log_number() const { return prev_log_number_; } // Returns the minimum log number such that all // log numbers less than or equal to it can be deleted uint64_t MinLogNumber() const { uint64_t min_log_num = std::numeric_limits::max(); for (auto cfd : *column_family_set_) { - if (min_log_num > cfd->GetLogNumber()) { + // It's safe to ignore dropped column families here: + // cfd->IsDropped() becomes true after the drop is persisted in MANIFEST. + if (min_log_num > cfd->GetLogNumber() && !cfd->IsDropped()) { min_log_num = cfd->GetLogNumber(); } } @@ -457,17 +748,20 @@ class VersionSet { // Create an iterator that reads over the compaction inputs for "*c". // The caller should delete the iterator when no longer needed. - Iterator* MakeInputIterator(Compaction* c); + InternalIterator* MakeInputIterator(const Compaction* c, + RangeDelAggregator* range_del_agg); // Add all files listed in any live version to *live. void AddLiveFiles(std::vector* live_list); - // Return the approximate offset in the database of the data for - // "key" as of version "v". - uint64_t ApproximateOffsetOf(Version* v, const InternalKey& key); + // Return the approximate size of data to be scanned for range [start, end) + // in levels [start_level, end_level). If end_level == 0 it will search + // through all non-empty levels + uint64_t ApproximateSize(Version* v, const Slice& start, const Slice& end, + int start_level = 0, int end_level = -1); // Return the size of the current manifest file - uint64_t ManifestFileSize() const { return manifest_file_size_; } + uint64_t manifest_file_size() const { return manifest_file_size_; } // verify that the files that we started with for a compaction // still exist in the current version and in the same original level. @@ -478,46 +772,61 @@ class VersionSet { Status GetMetadataForFile(uint64_t number, int* filelevel, FileMetaData** metadata, ColumnFamilyData** cfd); - void GetLiveFilesMetaData( - std::vector *metadata); + // This function doesn't support leveldb SST filenames + void GetLiveFilesMetaData(std::vector *metadata); - void GetObsoleteFiles(std::vector* files); + void GetObsoleteFiles(std::vector* files, + std::vector* manifest_filenames, + uint64_t min_pending_output); ColumnFamilySet* GetColumnFamilySet() { return column_family_set_.get(); } + const EnvOptions& env_options() { return env_options_; } + + static uint64_t GetNumLiveVersions(Version* dummy_versions); + + static uint64_t GetTotalSstFilesSize(Version* dummy_versions); private: - class Builder; struct ManifestWriter; friend class Version; + friend class DBImpl; struct LogReporter : public log::Reader::Reporter { Status* status; - virtual void Corruption(size_t bytes, const Status& s) { + virtual void Corruption(size_t bytes, const Status& s) override { if (this->status->ok()) *this->status = s; } }; + // ApproximateSize helper + uint64_t ApproximateSizeLevel0(Version* v, const LevelFilesBrief& files_brief, + const Slice& start, const Slice& end); + + uint64_t ApproximateSize(Version* v, const FdWithKeyRange& f, + const Slice& key); + // Save current contents to *log Status WriteSnapshot(log::Writer* log); void AppendVersion(ColumnFamilyData* column_family_data, Version* v); - bool ManifestContains(uint64_t manifest_file_number, - const std::string& record) const; - - ColumnFamilyData* CreateColumnFamily(const ColumnFamilyOptions& options, + ColumnFamilyData* CreateColumnFamily(const ColumnFamilyOptions& cf_options, VersionEdit* edit); std::unique_ptr column_family_set_; Env* const env_; const std::string dbname_; - const DBOptions* const options_; - uint64_t next_file_number_; + const ImmutableDBOptions* const db_options_; + std::atomic next_file_number_; uint64_t manifest_file_number_; + uint64_t options_file_number_; uint64_t pending_manifest_file_number_; + // The last seq visible to reads std::atomic last_sequence_; + // The last seq with which a writer has written/will write. + std::atomic last_to_be_written_sequence_; uint64_t prev_log_number_; // 0 or backing store for memtable being compacted // Opened lazily @@ -533,21 +842,22 @@ class VersionSet { uint64_t manifest_file_size_; std::vector obsolete_files_; + std::vector obsolete_manifests_; - // storage options for all reads and writes except compactions - const EnvOptions& storage_options_; + // env options for all reads and writes except compactions + const EnvOptions& env_options_; - // storage options used for compactions. This is a copy of - // storage_options_ but with readaheads set to readahead_compactions_. - const EnvOptions storage_options_compactions_; + // env options used for compactions. This is a copy of + // env_options_ but with readaheads set to readahead_compactions_. + const EnvOptions env_options_compactions_; // No copying allowed VersionSet(const VersionSet&); void operator=(const VersionSet&); void LogAndApplyCFHelper(VersionEdit* edit); - void LogAndApplyHelper(ColumnFamilyData* cfd, Builder* b, Version* v, - VersionEdit* edit, port::Mutex* mu); + void LogAndApplyHelper(ColumnFamilyData* cfd, VersionBuilder* b, Version* v, + VersionEdit* edit, InstrumentedMutex* mu); }; } // namespace rocksdb diff --git a/db/version_set_test.cc b/db/version_set_test.cc index 402762efa11..625d4592264 100644 --- a/db/version_set_test.cc +++ b/db/version_set_test.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -14,16 +14,16 @@ namespace rocksdb { -class GenerateFileLevelTest { +class GenerateLevelFilesBriefTest : public testing::Test { public: std::vector files_; - FileLevel file_level_; + LevelFilesBrief file_level_; Arena arena_; - GenerateFileLevelTest() { } + GenerateLevelFilesBriefTest() { } - ~GenerateFileLevelTest() { - for (unsigned int i = 0; i < files_.size(); i++) { + ~GenerateLevelFilesBriefTest() { + for (size_t i = 0; i < files_.size(); i++) { delete files_[i]; } } @@ -49,33 +49,217 @@ class GenerateFileLevelTest { } }; -TEST(GenerateFileLevelTest, Empty) { - DoGenerateFileLevel(&file_level_, files_, &arena_); +TEST_F(GenerateLevelFilesBriefTest, Empty) { + DoGenerateLevelFilesBrief(&file_level_, files_, &arena_); ASSERT_EQ(0u, file_level_.num_files); ASSERT_EQ(0, Compare()); } -TEST(GenerateFileLevelTest, Single) { +TEST_F(GenerateLevelFilesBriefTest, Single) { Add("p", "q"); - DoGenerateFileLevel(&file_level_, files_, &arena_); + DoGenerateLevelFilesBrief(&file_level_, files_, &arena_); ASSERT_EQ(1u, file_level_.num_files); ASSERT_EQ(0, Compare()); } - -TEST(GenerateFileLevelTest, Multiple) { +TEST_F(GenerateLevelFilesBriefTest, Multiple) { Add("150", "200"); Add("200", "250"); Add("300", "350"); Add("400", "450"); - DoGenerateFileLevel(&file_level_, files_, &arena_); + DoGenerateLevelFilesBrief(&file_level_, files_, &arena_); ASSERT_EQ(4u, file_level_.num_files); ASSERT_EQ(0, Compare()); } -class FindLevelFileTest { +class CountingLogger : public Logger { + public: + CountingLogger() : log_count(0) {} + using Logger::Logv; + virtual void Logv(const char* format, va_list ap) override { log_count++; } + int log_count; +}; + +Options GetOptionsWithNumLevels(int num_levels, + std::shared_ptr logger) { + Options opt; + opt.num_levels = num_levels; + opt.info_log = logger; + return opt; +} + +class VersionStorageInfoTest : public testing::Test { + public: + const Comparator* ucmp_; + InternalKeyComparator icmp_; + std::shared_ptr logger_; + Options options_; + ImmutableCFOptions ioptions_; + MutableCFOptions mutable_cf_options_; + VersionStorageInfo vstorage_; + + InternalKey GetInternalKey(const char* ukey, + SequenceNumber smallest_seq = 100) { + return InternalKey(ukey, smallest_seq, kTypeValue); + } + + VersionStorageInfoTest() + : ucmp_(BytewiseComparator()), + icmp_(ucmp_), + logger_(new CountingLogger()), + options_(GetOptionsWithNumLevels(6, logger_)), + ioptions_(options_), + mutable_cf_options_(options_), + vstorage_(&icmp_, ucmp_, 6, kCompactionStyleLevel, nullptr, false) {} + + ~VersionStorageInfoTest() { + for (int i = 0; i < vstorage_.num_levels(); i++) { + for (auto* f : vstorage_.LevelFiles(i)) { + if (--f->refs == 0) { + delete f; + } + } + } + } + + void Add(int level, uint32_t file_number, const char* smallest, + const char* largest, uint64_t file_size = 0) { + assert(level < vstorage_.num_levels()); + FileMetaData* f = new FileMetaData; + f->fd = FileDescriptor(file_number, 0, file_size); + f->smallest = GetInternalKey(smallest, 0); + f->largest = GetInternalKey(largest, 0); + f->compensated_file_size = file_size; + f->refs = 0; + f->num_entries = 0; + f->num_deletions = 0; + vstorage_.AddFile(level, f); + } +}; + +TEST_F(VersionStorageInfoTest, MaxBytesForLevelStatic) { + ioptions_.level_compaction_dynamic_level_bytes = false; + mutable_cf_options_.max_bytes_for_level_base = 10; + mutable_cf_options_.max_bytes_for_level_multiplier = 5; + Add(4, 100U, "1", "2"); + Add(5, 101U, "1", "2"); + + vstorage_.CalculateBaseBytes(ioptions_, mutable_cf_options_); + ASSERT_EQ(vstorage_.MaxBytesForLevel(1), 10U); + ASSERT_EQ(vstorage_.MaxBytesForLevel(2), 50U); + ASSERT_EQ(vstorage_.MaxBytesForLevel(3), 250U); + ASSERT_EQ(vstorage_.MaxBytesForLevel(4), 1250U); + + ASSERT_EQ(0, logger_->log_count); +} + +TEST_F(VersionStorageInfoTest, MaxBytesForLevelDynamic) { + ioptions_.level_compaction_dynamic_level_bytes = true; + mutable_cf_options_.max_bytes_for_level_base = 1000; + mutable_cf_options_.max_bytes_for_level_multiplier = 5; + Add(5, 1U, "1", "2", 500U); + + vstorage_.CalculateBaseBytes(ioptions_, mutable_cf_options_); + ASSERT_EQ(0, logger_->log_count); + ASSERT_EQ(vstorage_.base_level(), 5); + + Add(5, 2U, "3", "4", 550U); + vstorage_.CalculateBaseBytes(ioptions_, mutable_cf_options_); + ASSERT_EQ(0, logger_->log_count); + ASSERT_EQ(vstorage_.MaxBytesForLevel(4), 1000U); + ASSERT_EQ(vstorage_.base_level(), 4); + + Add(4, 3U, "3", "4", 550U); + vstorage_.CalculateBaseBytes(ioptions_, mutable_cf_options_); + ASSERT_EQ(0, logger_->log_count); + ASSERT_EQ(vstorage_.MaxBytesForLevel(4), 1000U); + ASSERT_EQ(vstorage_.base_level(), 4); + + Add(3, 4U, "3", "4", 250U); + Add(3, 5U, "5", "7", 300U); + vstorage_.CalculateBaseBytes(ioptions_, mutable_cf_options_); + ASSERT_EQ(1, logger_->log_count); + ASSERT_EQ(vstorage_.MaxBytesForLevel(4), 1005U); + ASSERT_EQ(vstorage_.MaxBytesForLevel(3), 1000U); + ASSERT_EQ(vstorage_.base_level(), 3); + + Add(1, 6U, "3", "4", 5U); + Add(1, 7U, "8", "9", 5U); + logger_->log_count = 0; + vstorage_.CalculateBaseBytes(ioptions_, mutable_cf_options_); + ASSERT_EQ(1, logger_->log_count); + ASSERT_GT(vstorage_.MaxBytesForLevel(4), 1005U); + ASSERT_GT(vstorage_.MaxBytesForLevel(3), 1005U); + ASSERT_EQ(vstorage_.MaxBytesForLevel(2), 1005U); + ASSERT_EQ(vstorage_.MaxBytesForLevel(1), 1000U); + ASSERT_EQ(vstorage_.base_level(), 1); +} + +TEST_F(VersionStorageInfoTest, MaxBytesForLevelDynamicLotsOfData) { + ioptions_.level_compaction_dynamic_level_bytes = true; + mutable_cf_options_.max_bytes_for_level_base = 100; + mutable_cf_options_.max_bytes_for_level_multiplier = 2; + Add(0, 1U, "1", "2", 50U); + Add(1, 2U, "1", "2", 50U); + Add(2, 3U, "1", "2", 500U); + Add(3, 4U, "1", "2", 500U); + Add(4, 5U, "1", "2", 1700U); + Add(5, 6U, "1", "2", 500U); + + vstorage_.CalculateBaseBytes(ioptions_, mutable_cf_options_); + ASSERT_EQ(vstorage_.MaxBytesForLevel(4), 800U); + ASSERT_EQ(vstorage_.MaxBytesForLevel(3), 400U); + ASSERT_EQ(vstorage_.MaxBytesForLevel(2), 200U); + ASSERT_EQ(vstorage_.MaxBytesForLevel(1), 100U); + ASSERT_EQ(vstorage_.base_level(), 1); + ASSERT_EQ(0, logger_->log_count); +} + +TEST_F(VersionStorageInfoTest, MaxBytesForLevelDynamicLargeLevel) { + uint64_t kOneGB = 1000U * 1000U * 1000U; + ioptions_.level_compaction_dynamic_level_bytes = true; + mutable_cf_options_.max_bytes_for_level_base = 10U * kOneGB; + mutable_cf_options_.max_bytes_for_level_multiplier = 10; + Add(0, 1U, "1", "2", 50U); + Add(3, 4U, "1", "2", 32U * kOneGB); + Add(4, 5U, "1", "2", 500U * kOneGB); + Add(5, 6U, "1", "2", 3000U * kOneGB); + + vstorage_.CalculateBaseBytes(ioptions_, mutable_cf_options_); + ASSERT_EQ(vstorage_.MaxBytesForLevel(5), 3000U * kOneGB); + ASSERT_EQ(vstorage_.MaxBytesForLevel(4), 300U * kOneGB); + ASSERT_EQ(vstorage_.MaxBytesForLevel(3), 30U * kOneGB); + ASSERT_EQ(vstorage_.MaxBytesForLevel(2), 10U * kOneGB); + ASSERT_EQ(vstorage_.base_level(), 2); + ASSERT_EQ(0, logger_->log_count); +} + +TEST_F(VersionStorageInfoTest, EstimateLiveDataSize) { + // Test whether the overlaps are detected as expected + Add(1, 1U, "4", "7", 1U); // Perfect overlap with last level + Add(2, 2U, "3", "5", 1U); // Partial overlap with last level + Add(2, 3U, "6", "8", 1U); // Partial overlap with last level + Add(3, 4U, "1", "9", 1U); // Contains range of last level + Add(4, 5U, "4", "5", 1U); // Inside range of last level + Add(4, 5U, "6", "7", 1U); // Inside range of last level + Add(5, 6U, "4", "7", 10U); + ASSERT_EQ(10U, vstorage_.EstimateLiveDataSize()); +} + +TEST_F(VersionStorageInfoTest, EstimateLiveDataSize2) { + Add(0, 1U, "9", "9", 1U); // Level 0 is not ordered + Add(0, 1U, "5", "6", 1U); // Ignored because of [5,6] in l1 + Add(1, 1U, "1", "2", 1U); // Ignored because of [2,3] in l2 + Add(1, 2U, "3", "4", 1U); // Ignored because of [2,3] in l2 + Add(1, 3U, "5", "6", 1U); + Add(2, 4U, "2", "3", 1U); + Add(3, 5U, "7", "8", 1U); + ASSERT_EQ(4U, vstorage_.EstimateLiveDataSize()); +} + +class FindLevelFileTest : public testing::Test { public: - FileLevel file_level_; + LevelFilesBrief file_level_; bool disjoint_sorted_files_; Arena arena_; @@ -131,7 +315,7 @@ class FindLevelFileTest { } }; -TEST(FindLevelFileTest, LevelEmpty) { +TEST_F(FindLevelFileTest, LevelEmpty) { LevelFileInit(0); ASSERT_EQ(0, Find("foo")); @@ -141,7 +325,7 @@ TEST(FindLevelFileTest, LevelEmpty) { ASSERT_TRUE(! Overlaps(nullptr, nullptr)); } -TEST(FindLevelFileTest, LevelSingle) { +TEST_F(FindLevelFileTest, LevelSingle) { LevelFileInit(1); Add("p", "q"); @@ -173,7 +357,7 @@ TEST(FindLevelFileTest, LevelSingle) { ASSERT_TRUE(Overlaps(nullptr, nullptr)); } -TEST(FindLevelFileTest, LevelMultiple) { +TEST_F(FindLevelFileTest, LevelMultiple) { LevelFileInit(4); Add("150", "200"); @@ -213,7 +397,7 @@ TEST(FindLevelFileTest, LevelMultiple) { ASSERT_TRUE(Overlaps("450", "500")); } -TEST(FindLevelFileTest, LevelMultipleNullBoundaries) { +TEST_F(FindLevelFileTest, LevelMultipleNullBoundaries) { LevelFileInit(4); Add("150", "200"); @@ -235,7 +419,7 @@ TEST(FindLevelFileTest, LevelMultipleNullBoundaries) { ASSERT_TRUE(Overlaps("450", nullptr)); } -TEST(FindLevelFileTest, LevelOverlapSequenceChecks) { +TEST_F(FindLevelFileTest, LevelOverlapSequenceChecks) { LevelFileInit(1); Add("200", "200", 5000, 3000); @@ -246,7 +430,7 @@ TEST(FindLevelFileTest, LevelOverlapSequenceChecks) { ASSERT_TRUE(Overlaps("200", "210")); } -TEST(FindLevelFileTest, LevelOverlappingFiles) { +TEST_F(FindLevelFileTest, LevelOverlappingFiles) { LevelFileInit(2); Add("150", "600"); @@ -269,5 +453,6 @@ TEST(FindLevelFileTest, LevelOverlappingFiles) { } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/wal_manager.cc b/db/wal_manager.cc new file mode 100644 index 00000000000..4a9ecbfdd8e --- /dev/null +++ b/db/wal_manager.cc @@ -0,0 +1,476 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "db/wal_manager.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include +#include +#include + +#include "db/log_reader.h" +#include "db/log_writer.h" +#include "db/transaction_log_impl.h" +#include "db/write_batch_internal.h" +#include "port/port.h" +#include "rocksdb/env.h" +#include "rocksdb/options.h" +#include "rocksdb/write_batch.h" +#include "util/cast_util.h" +#include "util/coding.h" +#include "util/file_reader_writer.h" +#include "util/filename.h" +#include "util/logging.h" +#include "util/mutexlock.h" +#include "util/string_util.h" +#include "util/sync_point.h" + +namespace rocksdb { + +#ifndef ROCKSDB_LITE + +Status WalManager::GetSortedWalFiles(VectorLogPtr& files) { + // First get sorted files in db dir, then get sorted files from archived + // dir, to avoid a race condition where a log file is moved to archived + // dir in between. + Status s; + // list wal files in main db dir. + VectorLogPtr logs; + s = GetSortedWalsOfType(db_options_.wal_dir, logs, kAliveLogFile); + if (!s.ok()) { + return s; + } + + // Reproduce the race condition where a log file is moved + // to archived dir, between these two sync points, used in + // (DBTest,TransactionLogIteratorRace) + TEST_SYNC_POINT("WalManager::GetSortedWalFiles:1"); + TEST_SYNC_POINT("WalManager::GetSortedWalFiles:2"); + + files.clear(); + // list wal files in archive dir. + std::string archivedir = ArchivalDirectory(db_options_.wal_dir); + Status exists = env_->FileExists(archivedir); + if (exists.ok()) { + s = GetSortedWalsOfType(archivedir, files, kArchivedLogFile); + if (!s.ok()) { + return s; + } + } else if (!exists.IsNotFound()) { + assert(s.IsIOError()); + return s; + } + + uint64_t latest_archived_log_number = 0; + if (!files.empty()) { + latest_archived_log_number = files.back()->LogNumber(); + ROCKS_LOG_INFO(db_options_.info_log, "Latest Archived log: %" PRIu64, + latest_archived_log_number); + } + + files.reserve(files.size() + logs.size()); + for (auto& log : logs) { + if (log->LogNumber() > latest_archived_log_number) { + files.push_back(std::move(log)); + } else { + // When the race condition happens, we could see the + // same log in both db dir and archived dir. Simply + // ignore the one in db dir. Note that, if we read + // archived dir first, we would have missed the log file. + ROCKS_LOG_WARN(db_options_.info_log, "%s already moved to archive", + log->PathName().c_str()); + } + } + + return s; +} + +Status WalManager::GetUpdatesSince( + SequenceNumber seq, std::unique_ptr* iter, + const TransactionLogIterator::ReadOptions& read_options, + VersionSet* version_set) { + + // Get all sorted Wal Files. + // Do binary search and open files and find the seq number. + + std::unique_ptr wal_files(new VectorLogPtr); + Status s = GetSortedWalFiles(*wal_files); + if (!s.ok()) { + return s; + } + + s = RetainProbableWalFiles(*wal_files, seq); + if (!s.ok()) { + return s; + } + iter->reset(new TransactionLogIteratorImpl( + db_options_.wal_dir, &db_options_, read_options, env_options_, seq, + std::move(wal_files), version_set)); + return (*iter)->status(); +} + +// 1. Go through all archived files and +// a. if ttl is enabled, delete outdated files +// b. if archive size limit is enabled, delete empty files, +// compute file number and size. +// 2. If size limit is enabled: +// a. compute how many files should be deleted +// b. get sorted non-empty archived logs +// c. delete what should be deleted +void WalManager::PurgeObsoleteWALFiles() { + bool const ttl_enabled = db_options_.wal_ttl_seconds > 0; + bool const size_limit_enabled = db_options_.wal_size_limit_mb > 0; + if (!ttl_enabled && !size_limit_enabled) { + return; + } + + int64_t current_time; + Status s = env_->GetCurrentTime(¤t_time); + if (!s.ok()) { + ROCKS_LOG_ERROR(db_options_.info_log, "Can't get current time: %s", + s.ToString().c_str()); + assert(false); + return; + } + uint64_t const now_seconds = static_cast(current_time); + uint64_t const time_to_check = (ttl_enabled && !size_limit_enabled) + ? db_options_.wal_ttl_seconds / 2 + : kDefaultIntervalToDeleteObsoleteWAL; + + if (purge_wal_files_last_run_ + time_to_check > now_seconds) { + return; + } + + purge_wal_files_last_run_ = now_seconds; + + std::string archival_dir = ArchivalDirectory(db_options_.wal_dir); + std::vector files; + s = env_->GetChildren(archival_dir, &files); + if (!s.ok()) { + ROCKS_LOG_ERROR(db_options_.info_log, "Can't get archive files: %s", + s.ToString().c_str()); + assert(false); + return; + } + + size_t log_files_num = 0; + uint64_t log_file_size = 0; + + for (auto& f : files) { + uint64_t number; + FileType type; + if (ParseFileName(f, &number, &type) && type == kLogFile) { + std::string const file_path = archival_dir + "/" + f; + if (ttl_enabled) { + uint64_t file_m_time; + s = env_->GetFileModificationTime(file_path, &file_m_time); + if (!s.ok()) { + ROCKS_LOG_WARN(db_options_.info_log, + "Can't get file mod time: %s: %s", file_path.c_str(), + s.ToString().c_str()); + continue; + } + if (now_seconds - file_m_time > db_options_.wal_ttl_seconds) { + s = env_->DeleteFile(file_path); + if (!s.ok()) { + ROCKS_LOG_WARN(db_options_.info_log, "Can't delete file: %s: %s", + file_path.c_str(), s.ToString().c_str()); + continue; + } else { + MutexLock l(&read_first_record_cache_mutex_); + read_first_record_cache_.erase(number); + } + continue; + } + } + + if (size_limit_enabled) { + uint64_t file_size; + s = env_->GetFileSize(file_path, &file_size); + if (!s.ok()) { + ROCKS_LOG_ERROR(db_options_.info_log, + "Unable to get file size: %s: %s", file_path.c_str(), + s.ToString().c_str()); + return; + } else { + if (file_size > 0) { + log_file_size = std::max(log_file_size, file_size); + ++log_files_num; + } else { + s = env_->DeleteFile(file_path); + if (!s.ok()) { + ROCKS_LOG_WARN(db_options_.info_log, + "Unable to delete file: %s: %s", file_path.c_str(), + s.ToString().c_str()); + continue; + } else { + MutexLock l(&read_first_record_cache_mutex_); + read_first_record_cache_.erase(number); + } + } + } + } + } + } + + if (0 == log_files_num || !size_limit_enabled) { + return; + } + + size_t const files_keep_num = + db_options_.wal_size_limit_mb * 1024 * 1024 / log_file_size; + if (log_files_num <= files_keep_num) { + return; + } + + size_t files_del_num = log_files_num - files_keep_num; + VectorLogPtr archived_logs; + GetSortedWalsOfType(archival_dir, archived_logs, kArchivedLogFile); + + if (files_del_num > archived_logs.size()) { + ROCKS_LOG_WARN(db_options_.info_log, + "Trying to delete more archived log files than " + "exist. Deleting all"); + files_del_num = archived_logs.size(); + } + + for (size_t i = 0; i < files_del_num; ++i) { + std::string const file_path = archived_logs[i]->PathName(); + s = env_->DeleteFile(db_options_.wal_dir + "/" + file_path); + if (!s.ok()) { + ROCKS_LOG_WARN(db_options_.info_log, "Unable to delete file: %s: %s", + file_path.c_str(), s.ToString().c_str()); + continue; + } else { + MutexLock l(&read_first_record_cache_mutex_); + read_first_record_cache_.erase(archived_logs[i]->LogNumber()); + } + } +} + +void WalManager::ArchiveWALFile(const std::string& fname, uint64_t number) { + auto archived_log_name = ArchivedLogFileName(db_options_.wal_dir, number); + // The sync point below is used in (DBTest,TransactionLogIteratorRace) + TEST_SYNC_POINT("WalManager::PurgeObsoleteFiles:1"); + Status s = env_->RenameFile(fname, archived_log_name); + // The sync point below is used in (DBTest,TransactionLogIteratorRace) + TEST_SYNC_POINT("WalManager::PurgeObsoleteFiles:2"); + ROCKS_LOG_INFO(db_options_.info_log, "Move log file %s to %s -- %s\n", + fname.c_str(), archived_log_name.c_str(), + s.ToString().c_str()); +} + +namespace { +struct CompareLogByPointer { + bool operator()(const std::unique_ptr& a, + const std::unique_ptr& b) { + LogFileImpl* a_impl = static_cast_with_check(a.get()); + LogFileImpl* b_impl = static_cast_with_check(b.get()); + return *a_impl < *b_impl; + } +}; +} + +Status WalManager::GetSortedWalsOfType(const std::string& path, + VectorLogPtr& log_files, + WalFileType log_type) { + std::vector all_files; + const Status status = env_->GetChildren(path, &all_files); + if (!status.ok()) { + return status; + } + log_files.reserve(all_files.size()); + for (const auto& f : all_files) { + uint64_t number; + FileType type; + if (ParseFileName(f, &number, &type) && type == kLogFile) { + SequenceNumber sequence; + Status s = ReadFirstRecord(log_type, number, &sequence); + if (!s.ok()) { + return s; + } + if (sequence == 0) { + // empty file + continue; + } + + // Reproduce the race condition where a log file is moved + // to archived dir, between these two sync points, used in + // (DBTest,TransactionLogIteratorRace) + TEST_SYNC_POINT("WalManager::GetSortedWalsOfType:1"); + TEST_SYNC_POINT("WalManager::GetSortedWalsOfType:2"); + + uint64_t size_bytes; + s = env_->GetFileSize(LogFileName(path, number), &size_bytes); + // re-try in case the alive log file has been moved to archive. + std::string archived_file = ArchivedLogFileName(path, number); + if (!s.ok() && log_type == kAliveLogFile && + env_->FileExists(archived_file).ok()) { + s = env_->GetFileSize(archived_file, &size_bytes); + if (!s.ok() && env_->FileExists(archived_file).IsNotFound()) { + // oops, the file just got deleted from archived dir! move on + s = Status::OK(); + continue; + } + } + if (!s.ok()) { + return s; + } + + log_files.push_back(std::unique_ptr( + new LogFileImpl(number, log_type, sequence, size_bytes))); + } + } + CompareLogByPointer compare_log_files; + std::sort(log_files.begin(), log_files.end(), compare_log_files); + return status; +} + +Status WalManager::RetainProbableWalFiles(VectorLogPtr& all_logs, + const SequenceNumber target) { + int64_t start = 0; // signed to avoid overflow when target is < first file. + int64_t end = static_cast(all_logs.size()) - 1; + // Binary Search. avoid opening all files. + while (end >= start) { + int64_t mid = start + (end - start) / 2; // Avoid overflow. + SequenceNumber current_seq_num = all_logs.at(mid)->StartSequence(); + if (current_seq_num == target) { + end = mid; + break; + } else if (current_seq_num < target) { + start = mid + 1; + } else { + end = mid - 1; + } + } + // end could be -ve. + size_t start_index = std::max(static_cast(0), end); + // The last wal file is always included + all_logs.erase(all_logs.begin(), all_logs.begin() + start_index); + return Status::OK(); +} + +Status WalManager::ReadFirstRecord(const WalFileType type, + const uint64_t number, + SequenceNumber* sequence) { + *sequence = 0; + if (type != kAliveLogFile && type != kArchivedLogFile) { + ROCKS_LOG_ERROR(db_options_.info_log, "[WalManger] Unknown file type %s", + ToString(type).c_str()); + return Status::NotSupported( + "File Type Not Known " + ToString(type)); + } + { + MutexLock l(&read_first_record_cache_mutex_); + auto itr = read_first_record_cache_.find(number); + if (itr != read_first_record_cache_.end()) { + *sequence = itr->second; + return Status::OK(); + } + } + Status s; + if (type == kAliveLogFile) { + std::string fname = LogFileName(db_options_.wal_dir, number); + s = ReadFirstLine(fname, number, sequence); + if (env_->FileExists(fname).ok() && !s.ok()) { + // return any error that is not caused by non-existing file + return s; + } + } + + if (type == kArchivedLogFile || !s.ok()) { + // check if the file got moved to archive. + std::string archived_file = + ArchivedLogFileName(db_options_.wal_dir, number); + s = ReadFirstLine(archived_file, number, sequence); + // maybe the file was deleted from archive dir. If that's the case, return + // Status::OK(). The caller with identify this as empty file because + // *sequence == 0 + if (!s.ok() && env_->FileExists(archived_file).IsNotFound()) { + return Status::OK(); + } + } + + if (s.ok() && *sequence != 0) { + MutexLock l(&read_first_record_cache_mutex_); + read_first_record_cache_.insert({number, *sequence}); + } + return s; +} + +// the function returns status.ok() and sequence == 0 if the file exists, but is +// empty +Status WalManager::ReadFirstLine(const std::string& fname, + const uint64_t number, + SequenceNumber* sequence) { + struct LogReporter : public log::Reader::Reporter { + Env* env; + Logger* info_log; + const char* fname; + + Status* status; + bool ignore_error; // true if db_options_.paranoid_checks==false + virtual void Corruption(size_t bytes, const Status& s) override { + ROCKS_LOG_WARN(info_log, "[WalManager] %s%s: dropping %d bytes; %s", + (this->ignore_error ? "(ignoring error) " : ""), fname, + static_cast(bytes), s.ToString().c_str()); + if (this->status->ok()) { + // only keep the first error + *this->status = s; + } + } + }; + + std::unique_ptr file; + Status status = env_->NewSequentialFile( + fname, &file, env_->OptimizeForLogRead(env_options_)); + unique_ptr file_reader( + new SequentialFileReader(std::move(file))); + + if (!status.ok()) { + return status; + } + + LogReporter reporter; + reporter.env = env_; + reporter.info_log = db_options_.info_log.get(); + reporter.fname = fname.c_str(); + reporter.status = &status; + reporter.ignore_error = !db_options_.paranoid_checks; + log::Reader reader(db_options_.info_log, std::move(file_reader), &reporter, + true /*checksum*/, 0 /*initial_offset*/, number); + std::string scratch; + Slice record; + + if (reader.ReadRecord(&record, &scratch) && + (status.ok() || !db_options_.paranoid_checks)) { + if (record.size() < WriteBatchInternal::kHeader) { + reporter.Corruption(record.size(), + Status::Corruption("log record too small")); + // TODO read record's till the first no corrupt entry? + } else { + WriteBatch batch; + WriteBatchInternal::SetContents(&batch, record); + *sequence = WriteBatchInternal::Sequence(&batch); + return Status::OK(); + } + } + + // ReadRecord returns false on EOF, which means that the log file is empty. we + // return status.ok() in that case and set sequence number to 0 + *sequence = 0; + return status; +} + +#endif // ROCKSDB_LITE +} // namespace rocksdb diff --git a/db/wal_manager.h b/db/wal_manager.h new file mode 100644 index 00000000000..aa62d793bc7 --- /dev/null +++ b/db/wal_manager.h @@ -0,0 +1,95 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "db/version_set.h" +#include "options/db_options.h" +#include "port/port.h" +#include "rocksdb/env.h" +#include "rocksdb/status.h" +#include "rocksdb/transaction_log.h" +#include "rocksdb/types.h" + +namespace rocksdb { + +#ifndef ROCKSDB_LITE +class WalManager { + public: + WalManager(const ImmutableDBOptions& db_options, + const EnvOptions& env_options) + : db_options_(db_options), + env_options_(env_options), + env_(db_options.env), + purge_wal_files_last_run_(0) {} + + Status GetSortedWalFiles(VectorLogPtr& files); + + Status GetUpdatesSince( + SequenceNumber seq_number, std::unique_ptr* iter, + const TransactionLogIterator::ReadOptions& read_options, + VersionSet* version_set); + + void PurgeObsoleteWALFiles(); + + void ArchiveWALFile(const std::string& fname, uint64_t number); + + Status TEST_ReadFirstRecord(const WalFileType type, const uint64_t number, + SequenceNumber* sequence) { + return ReadFirstRecord(type, number, sequence); + } + + Status TEST_ReadFirstLine(const std::string& fname, const uint64_t number, + SequenceNumber* sequence) { + return ReadFirstLine(fname, number, sequence); + } + + private: + Status GetSortedWalsOfType(const std::string& path, VectorLogPtr& log_files, + WalFileType type); + // Requires: all_logs should be sorted with earliest log file first + // Retains all log files in all_logs which contain updates with seq no. + // Greater Than or Equal to the requested SequenceNumber. + Status RetainProbableWalFiles(VectorLogPtr& all_logs, + const SequenceNumber target); + + Status ReadFirstRecord(const WalFileType type, const uint64_t number, + SequenceNumber* sequence); + + Status ReadFirstLine(const std::string& fname, const uint64_t number, + SequenceNumber* sequence); + + // ------- state from DBImpl ------ + const ImmutableDBOptions& db_options_; + const EnvOptions& env_options_; + Env* env_; + + // ------- WalManager state ------- + // cache for ReadFirstRecord() calls + std::unordered_map read_first_record_cache_; + port::Mutex read_first_record_cache_mutex_; + + // last time when PurgeObsoleteWALFiles ran. + uint64_t purge_wal_files_last_run_; + + // obsolete files will be deleted every this seconds if ttl deletion is + // enabled and archive size_limit is disabled. + static const uint64_t kDefaultIntervalToDeleteObsoleteWAL = 600; +}; + +#endif // ROCKSDB_LITE +} // namespace rocksdb diff --git a/db/wal_manager_test.cc b/db/wal_manager_test.cc new file mode 100644 index 00000000000..9f5cf273d24 --- /dev/null +++ b/db/wal_manager_test.cc @@ -0,0 +1,310 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include +#include + +#include "rocksdb/cache.h" +#include "rocksdb/write_batch.h" +#include "rocksdb/write_buffer_manager.h" + +#include "db/column_family.h" +#include "db/db_impl.h" +#include "db/log_writer.h" +#include "db/version_set.h" +#include "db/wal_manager.h" +#include "env/mock_env.h" +#include "table/mock_table.h" +#include "util/file_reader_writer.h" +#include "util/string_util.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +// TODO(icanadi) mock out VersionSet +// TODO(icanadi) move other WalManager-specific tests from db_test here +class WalManagerTest : public testing::Test { + public: + WalManagerTest() + : env_(new MockEnv(Env::Default())), + dbname_(test::TmpDir() + "/wal_manager_test"), + db_options_(), + table_cache_(NewLRUCache(50000, 16)), + write_buffer_manager_(db_options_.db_write_buffer_size), + current_log_number_(0) { + DestroyDB(dbname_, Options()); + } + + void Init() { + ASSERT_OK(env_->CreateDirIfMissing(dbname_)); + ASSERT_OK(env_->CreateDirIfMissing(ArchivalDirectory(dbname_))); + db_options_.db_paths.emplace_back(dbname_, + std::numeric_limits::max()); + db_options_.wal_dir = dbname_; + db_options_.env = env_.get(); + + versions_.reset(new VersionSet(dbname_, &db_options_, env_options_, + table_cache_.get(), &write_buffer_manager_, + &write_controller_)); + + wal_manager_.reset(new WalManager(db_options_, env_options_)); + } + + void Reopen() { + wal_manager_.reset(new WalManager(db_options_, env_options_)); + } + + // NOT thread safe + void Put(const std::string& key, const std::string& value) { + assert(current_log_writer_.get() != nullptr); + uint64_t seq = versions_->LastSequence() + 1; + WriteBatch batch; + batch.Put(key, value); + WriteBatchInternal::SetSequence(&batch, seq); + current_log_writer_->AddRecord(WriteBatchInternal::Contents(&batch)); + versions_->SetLastToBeWrittenSequence(seq); + versions_->SetLastSequence(seq); + } + + // NOT thread safe + void RollTheLog(bool archived) { + current_log_number_++; + std::string fname = ArchivedLogFileName(dbname_, current_log_number_); + unique_ptr file; + ASSERT_OK(env_->NewWritableFile(fname, &file, env_options_)); + unique_ptr file_writer( + new WritableFileWriter(std::move(file), env_options_)); + current_log_writer_.reset(new log::Writer(std::move(file_writer), 0, false)); + } + + void CreateArchiveLogs(int num_logs, int entries_per_log) { + for (int i = 1; i <= num_logs; ++i) { + RollTheLog(true); + for (int k = 0; k < entries_per_log; ++k) { + Put(ToString(k), std::string(1024, 'a')); + } + } + } + + std::unique_ptr OpenTransactionLogIter( + const SequenceNumber seq) { + unique_ptr iter; + Status status = wal_manager_->GetUpdatesSince( + seq, &iter, TransactionLogIterator::ReadOptions(), versions_.get()); + EXPECT_OK(status); + return iter; + } + + std::unique_ptr env_; + std::string dbname_; + ImmutableDBOptions db_options_; + WriteController write_controller_; + EnvOptions env_options_; + std::shared_ptr table_cache_; + WriteBufferManager write_buffer_manager_; + std::unique_ptr versions_; + std::unique_ptr wal_manager_; + + std::unique_ptr current_log_writer_; + uint64_t current_log_number_; +}; + +TEST_F(WalManagerTest, ReadFirstRecordCache) { + Init(); + std::string path = dbname_ + "/000001.log"; + unique_ptr file; + ASSERT_OK(env_->NewWritableFile(path, &file, EnvOptions())); + + SequenceNumber s; + ASSERT_OK(wal_manager_->TEST_ReadFirstLine(path, 1 /* number */, &s)); + ASSERT_EQ(s, 0U); + + ASSERT_OK( + wal_manager_->TEST_ReadFirstRecord(kAliveLogFile, 1 /* number */, &s)); + ASSERT_EQ(s, 0U); + + unique_ptr file_writer( + new WritableFileWriter(std::move(file), EnvOptions())); + log::Writer writer(std::move(file_writer), 1, + db_options_.recycle_log_file_num > 0); + WriteBatch batch; + batch.Put("foo", "bar"); + WriteBatchInternal::SetSequence(&batch, 10); + writer.AddRecord(WriteBatchInternal::Contents(&batch)); + + // TODO(icanadi) move SpecialEnv outside of db_test, so we can reuse it here. + // Waiting for lei to finish with db_test + // env_->count_sequential_reads_ = true; + // sequential_read_counter_ sanity test + // ASSERT_EQ(env_->sequential_read_counter_.Read(), 0); + + ASSERT_OK(wal_manager_->TEST_ReadFirstRecord(kAliveLogFile, 1, &s)); + ASSERT_EQ(s, 10U); + // did a read + // TODO(icanadi) move SpecialEnv outside of db_test, so we can reuse it here + // ASSERT_EQ(env_->sequential_read_counter_.Read(), 1); + + ASSERT_OK(wal_manager_->TEST_ReadFirstRecord(kAliveLogFile, 1, &s)); + ASSERT_EQ(s, 10U); + // no new reads since the value is cached + // TODO(icanadi) move SpecialEnv outside of db_test, so we can reuse it here + // ASSERT_EQ(env_->sequential_read_counter_.Read(), 1); +} + +namespace { +uint64_t GetLogDirSize(std::string dir_path, Env* env) { + uint64_t dir_size = 0; + std::vector files; + env->GetChildren(dir_path, &files); + for (auto& f : files) { + uint64_t number; + FileType type; + if (ParseFileName(f, &number, &type) && type == kLogFile) { + std::string const file_path = dir_path + "/" + f; + uint64_t file_size; + env->GetFileSize(file_path, &file_size); + dir_size += file_size; + } + } + return dir_size; +} +std::vector ListSpecificFiles( + Env* env, const std::string& path, const FileType expected_file_type) { + std::vector files; + std::vector file_numbers; + env->GetChildren(path, &files); + uint64_t number; + FileType type; + for (size_t i = 0; i < files.size(); ++i) { + if (ParseFileName(files[i], &number, &type)) { + if (type == expected_file_type) { + file_numbers.push_back(number); + } + } + } + return file_numbers; +} + +int CountRecords(TransactionLogIterator* iter) { + int count = 0; + SequenceNumber lastSequence = 0; + BatchResult res; + while (iter->Valid()) { + res = iter->GetBatch(); + EXPECT_TRUE(res.sequence > lastSequence); + ++count; + lastSequence = res.sequence; + EXPECT_OK(iter->status()); + iter->Next(); + } + return count; +} +} // namespace + +TEST_F(WalManagerTest, WALArchivalSizeLimit) { + db_options_.wal_ttl_seconds = 0; + db_options_.wal_size_limit_mb = 1000; + Init(); + + // TEST : Create WalManager with huge size limit and no ttl. + // Create some archived files and call PurgeObsoleteWALFiles(). + // Count the archived log files that survived. + // Assert that all of them did. + // Change size limit. Re-open WalManager. + // Assert that archive is not greater than wal_size_limit_mb after + // PurgeObsoleteWALFiles() + // Set ttl and time_to_check_ to small values. Re-open db. + // Assert that there are no archived logs left. + + std::string archive_dir = ArchivalDirectory(dbname_); + CreateArchiveLogs(20, 5000); + + std::vector log_files = + ListSpecificFiles(env_.get(), archive_dir, kLogFile); + ASSERT_EQ(log_files.size(), 20U); + + db_options_.wal_size_limit_mb = 8; + Reopen(); + wal_manager_->PurgeObsoleteWALFiles(); + + uint64_t archive_size = GetLogDirSize(archive_dir, env_.get()); + ASSERT_TRUE(archive_size <= db_options_.wal_size_limit_mb * 1024 * 1024); + + db_options_.wal_ttl_seconds = 1; + env_->FakeSleepForMicroseconds(2 * 1000 * 1000); + Reopen(); + wal_manager_->PurgeObsoleteWALFiles(); + + log_files = ListSpecificFiles(env_.get(), archive_dir, kLogFile); + ASSERT_TRUE(log_files.empty()); +} + +TEST_F(WalManagerTest, WALArchivalTtl) { + db_options_.wal_ttl_seconds = 1000; + Init(); + + // TEST : Create WalManager with a ttl and no size limit. + // Create some archived log files and call PurgeObsoleteWALFiles(). + // Assert that files are not deleted + // Reopen db with small ttl. + // Assert that all archived logs was removed. + + std::string archive_dir = ArchivalDirectory(dbname_); + CreateArchiveLogs(20, 5000); + + std::vector log_files = + ListSpecificFiles(env_.get(), archive_dir, kLogFile); + ASSERT_GT(log_files.size(), 0U); + + db_options_.wal_ttl_seconds = 1; + env_->FakeSleepForMicroseconds(3 * 1000 * 1000); + Reopen(); + wal_manager_->PurgeObsoleteWALFiles(); + + log_files = ListSpecificFiles(env_.get(), archive_dir, kLogFile); + ASSERT_TRUE(log_files.empty()); +} + +TEST_F(WalManagerTest, TransactionLogIteratorMoveOverZeroFiles) { + Init(); + RollTheLog(false); + Put("key1", std::string(1024, 'a')); + // Create a zero record WAL file. + RollTheLog(false); + RollTheLog(false); + + Put("key2", std::string(1024, 'a')); + + auto iter = OpenTransactionLogIter(0); + ASSERT_EQ(2, CountRecords(iter.get())); +} + +TEST_F(WalManagerTest, TransactionLogIteratorJustEmptyFile) { + Init(); + RollTheLog(false); + auto iter = OpenTransactionLogIter(0); + // Check that an empty iterator is returned + ASSERT_TRUE(!iter->Valid()); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, "SKIPPED as WalManager is not supported in ROCKSDB_LITE\n"); + return 0; +} + +#endif // !ROCKSDB_LITE diff --git a/db/write_batch.cc b/db/write_batch.cc index bfa5e3f6f27..76fc94844a7 100644 --- a/db/write_batch.cc +++ b/db/write_batch.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -13,56 +13,170 @@ // data: record[count] // record := // kTypeValue varstring varstring -// kTypeMerge varstring varstring // kTypeDeletion varstring +// kTypeSingleDeletion varstring +// kTypeMerge varstring varstring // kTypeColumnFamilyValue varint32 varstring varstring -// kTypeColumnFamilyMerge varint32 varstring varstring // kTypeColumnFamilyDeletion varint32 varstring varstring +// kTypeColumnFamilySingleDeletion varint32 varstring varstring +// kTypeColumnFamilyMerge varint32 varstring varstring +// kTypeBeginPrepareXID varstring +// kTypeEndPrepareXID +// kTypeCommitXID varstring +// kTypeRollbackXID varstring +// kTypeNoop // varstring := // len: varint32 // data: uint8[len] #include "rocksdb/write_batch.h" -#include "rocksdb/options.h" -#include "rocksdb/merge_operator.h" -#include "db/dbformat.h" -#include "db/db_impl.h" + +#include +#include +#include +#include +#include + #include "db/column_family.h" +#include "db/db_impl.h" +#include "db/dbformat.h" +#include "db/flush_scheduler.h" #include "db/memtable.h" -#include "db/snapshot.h" +#include "db/merge_context.h" +#include "db/snapshot_impl.h" #include "db/write_batch_internal.h" +#include "monitoring/perf_context_imp.h" +#include "monitoring/statistics.h" +#include "rocksdb/merge_operator.h" #include "util/coding.h" -#include "util/statistics.h" -#include +#include "util/string_util.h" namespace rocksdb { -// WriteBatch header has an 8-byte sequence number followed by a 4-byte count. -static const size_t kHeader = 12; +// anon namespace for file-local types +namespace { -WriteBatch::WriteBatch(size_t reserved_bytes) { - rep_.reserve((reserved_bytes > kHeader) ? reserved_bytes : kHeader); - Clear(); -} +enum ContentFlags : uint32_t { + DEFERRED = 1 << 0, + HAS_PUT = 1 << 1, + HAS_DELETE = 1 << 2, + HAS_SINGLE_DELETE = 1 << 3, + HAS_MERGE = 1 << 4, + HAS_BEGIN_PREPARE = 1 << 5, + HAS_END_PREPARE = 1 << 6, + HAS_COMMIT = 1 << 7, + HAS_ROLLBACK = 1 << 8, + HAS_DELETE_RANGE = 1 << 9, + HAS_BLOB_INDEX = 1 << 10, +}; -WriteBatch::~WriteBatch() { } +struct BatchContentClassifier : public WriteBatch::Handler { + uint32_t content_flags = 0; -WriteBatch::Handler::~Handler() { } + Status PutCF(uint32_t, const Slice&, const Slice&) override { + content_flags |= ContentFlags::HAS_PUT; + return Status::OK(); + } + + Status DeleteCF(uint32_t, const Slice&) override { + content_flags |= ContentFlags::HAS_DELETE; + return Status::OK(); + } + + Status SingleDeleteCF(uint32_t, const Slice&) override { + content_flags |= ContentFlags::HAS_SINGLE_DELETE; + return Status::OK(); + } + + Status DeleteRangeCF(uint32_t, const Slice&, const Slice&) override { + content_flags |= ContentFlags::HAS_DELETE_RANGE; + return Status::OK(); + } + + Status MergeCF(uint32_t, const Slice&, const Slice&) override { + content_flags |= ContentFlags::HAS_MERGE; + return Status::OK(); + } + + Status PutBlobIndexCF(uint32_t, const Slice&, const Slice&) override { + content_flags |= ContentFlags::HAS_BLOB_INDEX; + return Status::OK(); + } + + Status MarkBeginPrepare() override { + content_flags |= ContentFlags::HAS_BEGIN_PREPARE; + return Status::OK(); + } + + Status MarkEndPrepare(const Slice&) override { + content_flags |= ContentFlags::HAS_END_PREPARE; + return Status::OK(); + } + + Status MarkCommit(const Slice&) override { + content_flags |= ContentFlags::HAS_COMMIT; + return Status::OK(); + } + + Status MarkRollback(const Slice&) override { + content_flags |= ContentFlags::HAS_ROLLBACK; + return Status::OK(); + } +}; + +} // anon namespace -void WriteBatch::Handler::Put(const Slice& key, const Slice& value) { - // you need to either implement Put or PutCF - throw std::runtime_error("Handler::Put not implemented!"); +struct SavePoints { + std::stack stack; +}; + +WriteBatch::WriteBatch(size_t reserved_bytes, size_t max_bytes) + : save_points_(nullptr), content_flags_(0), max_bytes_(max_bytes), rep_() { + rep_.reserve((reserved_bytes > WriteBatchInternal::kHeader) ? + reserved_bytes : WriteBatchInternal::kHeader); + rep_.resize(WriteBatchInternal::kHeader); } -void WriteBatch::Handler::Merge(const Slice& key, const Slice& value) { - throw std::runtime_error("Handler::Merge not implemented!"); +WriteBatch::WriteBatch(const std::string& rep) + : save_points_(nullptr), + content_flags_(ContentFlags::DEFERRED), + max_bytes_(0), + rep_(rep) {} + +WriteBatch::WriteBatch(const WriteBatch& src) + : save_points_(src.save_points_), + wal_term_point_(src.wal_term_point_), + content_flags_(src.content_flags_.load(std::memory_order_relaxed)), + max_bytes_(src.max_bytes_), + rep_(src.rep_) {} + +WriteBatch::WriteBatch(WriteBatch&& src) + : save_points_(std::move(src.save_points_)), + wal_term_point_(std::move(src.wal_term_point_)), + content_flags_(src.content_flags_.load(std::memory_order_relaxed)), + max_bytes_(src.max_bytes_), + rep_(std::move(src.rep_)) {} + +WriteBatch& WriteBatch::operator=(const WriteBatch& src) { + if (&src != this) { + this->~WriteBatch(); + new (this) WriteBatch(src); + } + return *this; } -void WriteBatch::Handler::Delete(const Slice& key) { - // you need to either implement Delete or DeleteCF - throw std::runtime_error("Handler::Delete not implemented!"); +WriteBatch& WriteBatch::operator=(WriteBatch&& src) { + if (&src != this) { + this->~WriteBatch(); + new (this) WriteBatch(std::move(src)); + } + return *this; } +WriteBatch::~WriteBatch() { delete save_points_; } + +WriteBatch::Handler::~Handler() { } + void WriteBatch::Handler::LogData(const Slice& blob) { // If the user has not specified something to do with blobs, then we ignore // them. @@ -74,16 +188,101 @@ bool WriteBatch::Handler::Continue() { void WriteBatch::Clear() { rep_.clear(); - rep_.resize(kHeader); + rep_.resize(WriteBatchInternal::kHeader); + + content_flags_.store(0, std::memory_order_relaxed); + + if (save_points_ != nullptr) { + while (!save_points_->stack.empty()) { + save_points_->stack.pop(); + } + } + + wal_term_point_.clear(); } int WriteBatch::Count() const { return WriteBatchInternal::Count(this); } +uint32_t WriteBatch::ComputeContentFlags() const { + auto rv = content_flags_.load(std::memory_order_relaxed); + if ((rv & ContentFlags::DEFERRED) != 0) { + BatchContentClassifier classifier; + Iterate(&classifier); + rv = classifier.content_flags; + + // this method is conceptually const, because it is performing a lazy + // computation that doesn't affect the abstract state of the batch. + // content_flags_ is marked mutable so that we can perform the + // following assignment + content_flags_.store(rv, std::memory_order_relaxed); + } + return rv; +} + +void WriteBatch::MarkWalTerminationPoint() { + wal_term_point_.size = GetDataSize(); + wal_term_point_.count = Count(); + wal_term_point_.content_flags = content_flags_; +} + +bool WriteBatch::HasPut() const { + return (ComputeContentFlags() & ContentFlags::HAS_PUT) != 0; +} + +bool WriteBatch::HasDelete() const { + return (ComputeContentFlags() & ContentFlags::HAS_DELETE) != 0; +} + +bool WriteBatch::HasSingleDelete() const { + return (ComputeContentFlags() & ContentFlags::HAS_SINGLE_DELETE) != 0; +} + +bool WriteBatch::HasDeleteRange() const { + return (ComputeContentFlags() & ContentFlags::HAS_DELETE_RANGE) != 0; +} + +bool WriteBatch::HasMerge() const { + return (ComputeContentFlags() & ContentFlags::HAS_MERGE) != 0; +} + +bool ReadKeyFromWriteBatchEntry(Slice* input, Slice* key, bool cf_record) { + assert(input != nullptr && key != nullptr); + // Skip tag byte + input->remove_prefix(1); + + if (cf_record) { + // Skip column_family bytes + uint32_t cf; + if (!GetVarint32(input, &cf)) { + return false; + } + } + + // Extract key + return GetLengthPrefixedSlice(input, key); +} + +bool WriteBatch::HasBeginPrepare() const { + return (ComputeContentFlags() & ContentFlags::HAS_BEGIN_PREPARE) != 0; +} + +bool WriteBatch::HasEndPrepare() const { + return (ComputeContentFlags() & ContentFlags::HAS_END_PREPARE) != 0; +} + +bool WriteBatch::HasCommit() const { + return (ComputeContentFlags() & ContentFlags::HAS_COMMIT) != 0; +} + +bool WriteBatch::HasRollback() const { + return (ComputeContentFlags() & ContentFlags::HAS_ROLLBACK) != 0; +} + Status ReadRecordFromWriteBatch(Slice* input, char* tag, uint32_t* column_family, Slice* key, - Slice* value, Slice* blob) { + Slice* value, Slice* blob, Slice* xid) { assert(key != nullptr && value != nullptr); *tag = (*input)[0]; input->remove_prefix(1); @@ -101,15 +300,29 @@ Status ReadRecordFromWriteBatch(Slice* input, char* tag, } break; case kTypeColumnFamilyDeletion: + case kTypeColumnFamilySingleDeletion: if (!GetVarint32(input, column_family)) { return Status::Corruption("bad WriteBatch Delete"); } // intentional fallthrough case kTypeDeletion: + case kTypeSingleDeletion: if (!GetLengthPrefixedSlice(input, key)) { return Status::Corruption("bad WriteBatch Delete"); } break; + case kTypeColumnFamilyRangeDeletion: + if (!GetVarint32(input, column_family)) { + return Status::Corruption("bad WriteBatch DeleteRange"); + } + // intentional fallthrough + case kTypeRangeDeletion: + // for range delete, "key" is begin_key, "value" is end_key + if (!GetLengthPrefixedSlice(input, key) || + !GetLengthPrefixedSlice(input, value)) { + return Status::Corruption("bad WriteBatch DeleteRange"); + } + break; case kTypeColumnFamilyMerge: if (!GetVarint32(input, column_family)) { return Status::Corruption("bad WriteBatch Merge"); @@ -121,12 +334,41 @@ Status ReadRecordFromWriteBatch(Slice* input, char* tag, return Status::Corruption("bad WriteBatch Merge"); } break; + case kTypeColumnFamilyBlobIndex: + if (!GetVarint32(input, column_family)) { + return Status::Corruption("bad WriteBatch BlobIndex"); + } + // intentional fallthrough + case kTypeBlobIndex: + if (!GetLengthPrefixedSlice(input, key) || + !GetLengthPrefixedSlice(input, value)) { + return Status::Corruption("bad WriteBatch BlobIndex"); + } + break; case kTypeLogData: assert(blob != nullptr); if (!GetLengthPrefixedSlice(input, blob)) { return Status::Corruption("bad WriteBatch Blob"); } break; + case kTypeNoop: + case kTypeBeginPrepareXID: + break; + case kTypeEndPrepareXID: + if (!GetLengthPrefixedSlice(input, xid)) { + return Status::Corruption("bad EndPrepare XID"); + } + break; + case kTypeCommitXID: + if (!GetLengthPrefixedSlice(input, xid)) { + return Status::Corruption("bad Commit XID"); + } + break; + case kTypeRollbackXID: + if (!GetLengthPrefixedSlice(input, xid)) { + return Status::Corruption("bad Rollback XID"); + } + break; default: return Status::Corruption("unknown WriteBatch tag"); } @@ -135,12 +377,12 @@ Status ReadRecordFromWriteBatch(Slice* input, char* tag, Status WriteBatch::Iterate(Handler* handler) const { Slice input(rep_); - if (input.size() < kHeader) { + if (input.size() < WriteBatchInternal::kHeader) { return Status::Corruption("malformed WriteBatch (too small)"); } - input.remove_prefix(kHeader); - Slice key, value, blob; + input.remove_prefix(WriteBatchInternal::kHeader); + Slice key, value, blob, xid; int found = 0; Status s; while (s.ok() && !input.empty() && handler->Continue()) { @@ -148,7 +390,7 @@ Status WriteBatch::Iterate(Handler* handler) const { uint32_t column_family = 0; // default s = ReadRecordFromWriteBatch(&input, &tag, &column_family, &key, &value, - &blob); + &blob, &xid); if (!s.ok()) { return s; } @@ -156,22 +398,71 @@ Status WriteBatch::Iterate(Handler* handler) const { switch (tag) { case kTypeColumnFamilyValue: case kTypeValue: + assert(content_flags_.load(std::memory_order_relaxed) & + (ContentFlags::DEFERRED | ContentFlags::HAS_PUT)); s = handler->PutCF(column_family, key, value); found++; break; case kTypeColumnFamilyDeletion: case kTypeDeletion: + assert(content_flags_.load(std::memory_order_relaxed) & + (ContentFlags::DEFERRED | ContentFlags::HAS_DELETE)); s = handler->DeleteCF(column_family, key); found++; break; + case kTypeColumnFamilySingleDeletion: + case kTypeSingleDeletion: + assert(content_flags_.load(std::memory_order_relaxed) & + (ContentFlags::DEFERRED | ContentFlags::HAS_SINGLE_DELETE)); + s = handler->SingleDeleteCF(column_family, key); + found++; + break; + case kTypeColumnFamilyRangeDeletion: + case kTypeRangeDeletion: + assert(content_flags_.load(std::memory_order_relaxed) & + (ContentFlags::DEFERRED | ContentFlags::HAS_DELETE_RANGE)); + s = handler->DeleteRangeCF(column_family, key, value); + found++; + break; case kTypeColumnFamilyMerge: case kTypeMerge: + assert(content_flags_.load(std::memory_order_relaxed) & + (ContentFlags::DEFERRED | ContentFlags::HAS_MERGE)); s = handler->MergeCF(column_family, key, value); found++; break; + case kTypeColumnFamilyBlobIndex: + case kTypeBlobIndex: + assert(content_flags_.load(std::memory_order_relaxed) & + (ContentFlags::DEFERRED | ContentFlags::HAS_BLOB_INDEX)); + s = handler->PutBlobIndexCF(column_family, key, value); + found++; + break; case kTypeLogData: handler->LogData(blob); break; + case kTypeBeginPrepareXID: + assert(content_flags_.load(std::memory_order_relaxed) & + (ContentFlags::DEFERRED | ContentFlags::HAS_BEGIN_PREPARE)); + handler->MarkBeginPrepare(); + break; + case kTypeEndPrepareXID: + assert(content_flags_.load(std::memory_order_relaxed) & + (ContentFlags::DEFERRED | ContentFlags::HAS_END_PREPARE)); + handler->MarkEndPrepare(xid); + break; + case kTypeCommitXID: + assert(content_flags_.load(std::memory_order_relaxed) & + (ContentFlags::DEFERRED | ContentFlags::HAS_COMMIT)); + handler->MarkCommit(xid); + break; + case kTypeRollbackXID: + assert(content_flags_.load(std::memory_order_relaxed) & + (ContentFlags::DEFERRED | ContentFlags::HAS_ROLLBACK)); + handler->MarkRollback(xid); + break; + case kTypeNoop: + break; default: return Status::Corruption("unknown WriteBatch tag"); } @@ -202,8 +493,13 @@ void WriteBatchInternal::SetSequence(WriteBatch* b, SequenceNumber seq) { EncodeFixed64(&b->rep_[0], seq); } -void WriteBatchInternal::Put(WriteBatch* b, uint32_t column_family_id, - const Slice& key, const Slice& value) { +size_t WriteBatchInternal::GetFirstOffset(WriteBatch* b) { + return WriteBatchInternal::kHeader; +} + +Status WriteBatchInternal::Put(WriteBatch* b, uint32_t column_family_id, + const Slice& key, const Slice& value) { + LocalSavePoint save(b); WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); if (column_family_id == 0) { b->rep_.push_back(static_cast(kTypeValue)); @@ -213,15 +509,21 @@ void WriteBatchInternal::Put(WriteBatch* b, uint32_t column_family_id, } PutLengthPrefixedSlice(&b->rep_, key); PutLengthPrefixedSlice(&b->rep_, value); + b->content_flags_.store( + b->content_flags_.load(std::memory_order_relaxed) | ContentFlags::HAS_PUT, + std::memory_order_relaxed); + return save.commit(); } -void WriteBatch::Put(ColumnFamilyHandle* column_family, const Slice& key, - const Slice& value) { - WriteBatchInternal::Put(this, GetColumnFamilyID(column_family), key, value); +Status WriteBatch::Put(ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) { + return WriteBatchInternal::Put(this, GetColumnFamilyID(column_family), key, + value); } -void WriteBatchInternal::Put(WriteBatch* b, uint32_t column_family_id, - const SliceParts& key, const SliceParts& value) { +Status WriteBatchInternal::Put(WriteBatch* b, uint32_t column_family_id, + const SliceParts& key, const SliceParts& value) { + LocalSavePoint save(b); WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); if (column_family_id == 0) { b->rep_.push_back(static_cast(kTypeValue)); @@ -231,15 +533,66 @@ void WriteBatchInternal::Put(WriteBatch* b, uint32_t column_family_id, } PutLengthPrefixedSliceParts(&b->rep_, key); PutLengthPrefixedSliceParts(&b->rep_, value); + b->content_flags_.store( + b->content_flags_.load(std::memory_order_relaxed) | ContentFlags::HAS_PUT, + std::memory_order_relaxed); + return save.commit(); } -void WriteBatch::Put(ColumnFamilyHandle* column_family, const SliceParts& key, - const SliceParts& value) { - WriteBatchInternal::Put(this, GetColumnFamilyID(column_family), key, value); +Status WriteBatch::Put(ColumnFamilyHandle* column_family, const SliceParts& key, + const SliceParts& value) { + return WriteBatchInternal::Put(this, GetColumnFamilyID(column_family), key, + value); } -void WriteBatchInternal::Delete(WriteBatch* b, uint32_t column_family_id, - const Slice& key) { +Status WriteBatchInternal::InsertNoop(WriteBatch* b) { + b->rep_.push_back(static_cast(kTypeNoop)); + return Status::OK(); +} + +Status WriteBatchInternal::MarkEndPrepare(WriteBatch* b, const Slice& xid) { + // a manually constructed batch can only contain one prepare section + assert(b->rep_[12] == static_cast(kTypeNoop)); + + // all savepoints up to this point are cleared + if (b->save_points_ != nullptr) { + while (!b->save_points_->stack.empty()) { + b->save_points_->stack.pop(); + } + } + + // rewrite noop as begin marker + b->rep_[12] = static_cast(kTypeBeginPrepareXID); + b->rep_.push_back(static_cast(kTypeEndPrepareXID)); + PutLengthPrefixedSlice(&b->rep_, xid); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_END_PREPARE | + ContentFlags::HAS_BEGIN_PREPARE, + std::memory_order_relaxed); + return Status::OK(); +} + +Status WriteBatchInternal::MarkCommit(WriteBatch* b, const Slice& xid) { + b->rep_.push_back(static_cast(kTypeCommitXID)); + PutLengthPrefixedSlice(&b->rep_, xid); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_COMMIT, + std::memory_order_relaxed); + return Status::OK(); +} + +Status WriteBatchInternal::MarkRollback(WriteBatch* b, const Slice& xid) { + b->rep_.push_back(static_cast(kTypeRollbackXID)); + PutLengthPrefixedSlice(&b->rep_, xid); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_ROLLBACK, + std::memory_order_relaxed); + return Status::OK(); +} + +Status WriteBatchInternal::Delete(WriteBatch* b, uint32_t column_family_id, + const Slice& key) { + LocalSavePoint save(b); WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); if (column_family_id == 0) { b->rep_.push_back(static_cast(kTypeDeletion)); @@ -248,14 +601,20 @@ void WriteBatchInternal::Delete(WriteBatch* b, uint32_t column_family_id, PutVarint32(&b->rep_, column_family_id); } PutLengthPrefixedSlice(&b->rep_, key); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_DELETE, + std::memory_order_relaxed); + return save.commit(); } -void WriteBatch::Delete(ColumnFamilyHandle* column_family, const Slice& key) { - WriteBatchInternal::Delete(this, GetColumnFamilyID(column_family), key); +Status WriteBatch::Delete(ColumnFamilyHandle* column_family, const Slice& key) { + return WriteBatchInternal::Delete(this, GetColumnFamilyID(column_family), + key); } -void WriteBatchInternal::Delete(WriteBatch* b, uint32_t column_family_id, - const SliceParts& key) { +Status WriteBatchInternal::Delete(WriteBatch* b, uint32_t column_family_id, + const SliceParts& key) { + LocalSavePoint save(b); WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); if (column_family_id == 0) { b->rep_.push_back(static_cast(kTypeDeletion)); @@ -264,15 +623,120 @@ void WriteBatchInternal::Delete(WriteBatch* b, uint32_t column_family_id, PutVarint32(&b->rep_, column_family_id); } PutLengthPrefixedSliceParts(&b->rep_, key); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_DELETE, + std::memory_order_relaxed); + return save.commit(); } -void WriteBatch::Delete(ColumnFamilyHandle* column_family, - const SliceParts& key) { - WriteBatchInternal::Delete(this, GetColumnFamilyID(column_family), key); +Status WriteBatch::Delete(ColumnFamilyHandle* column_family, + const SliceParts& key) { + return WriteBatchInternal::Delete(this, GetColumnFamilyID(column_family), + key); } -void WriteBatchInternal::Merge(WriteBatch* b, uint32_t column_family_id, - const Slice& key, const Slice& value) { +Status WriteBatchInternal::SingleDelete(WriteBatch* b, + uint32_t column_family_id, + const Slice& key) { + LocalSavePoint save(b); + WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); + if (column_family_id == 0) { + b->rep_.push_back(static_cast(kTypeSingleDeletion)); + } else { + b->rep_.push_back(static_cast(kTypeColumnFamilySingleDeletion)); + PutVarint32(&b->rep_, column_family_id); + } + PutLengthPrefixedSlice(&b->rep_, key); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_SINGLE_DELETE, + std::memory_order_relaxed); + return save.commit(); +} + +Status WriteBatch::SingleDelete(ColumnFamilyHandle* column_family, + const Slice& key) { + return WriteBatchInternal::SingleDelete( + this, GetColumnFamilyID(column_family), key); +} + +Status WriteBatchInternal::SingleDelete(WriteBatch* b, + uint32_t column_family_id, + const SliceParts& key) { + LocalSavePoint save(b); + WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); + if (column_family_id == 0) { + b->rep_.push_back(static_cast(kTypeSingleDeletion)); + } else { + b->rep_.push_back(static_cast(kTypeColumnFamilySingleDeletion)); + PutVarint32(&b->rep_, column_family_id); + } + PutLengthPrefixedSliceParts(&b->rep_, key); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_SINGLE_DELETE, + std::memory_order_relaxed); + return save.commit(); +} + +Status WriteBatch::SingleDelete(ColumnFamilyHandle* column_family, + const SliceParts& key) { + return WriteBatchInternal::SingleDelete( + this, GetColumnFamilyID(column_family), key); +} + +Status WriteBatchInternal::DeleteRange(WriteBatch* b, uint32_t column_family_id, + const Slice& begin_key, + const Slice& end_key) { + LocalSavePoint save(b); + WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); + if (column_family_id == 0) { + b->rep_.push_back(static_cast(kTypeRangeDeletion)); + } else { + b->rep_.push_back(static_cast(kTypeColumnFamilyRangeDeletion)); + PutVarint32(&b->rep_, column_family_id); + } + PutLengthPrefixedSlice(&b->rep_, begin_key); + PutLengthPrefixedSlice(&b->rep_, end_key); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_DELETE_RANGE, + std::memory_order_relaxed); + return save.commit(); +} + +Status WriteBatch::DeleteRange(ColumnFamilyHandle* column_family, + const Slice& begin_key, const Slice& end_key) { + return WriteBatchInternal::DeleteRange(this, GetColumnFamilyID(column_family), + begin_key, end_key); +} + +Status WriteBatchInternal::DeleteRange(WriteBatch* b, uint32_t column_family_id, + const SliceParts& begin_key, + const SliceParts& end_key) { + LocalSavePoint save(b); + WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); + if (column_family_id == 0) { + b->rep_.push_back(static_cast(kTypeRangeDeletion)); + } else { + b->rep_.push_back(static_cast(kTypeColumnFamilyRangeDeletion)); + PutVarint32(&b->rep_, column_family_id); + } + PutLengthPrefixedSliceParts(&b->rep_, begin_key); + PutLengthPrefixedSliceParts(&b->rep_, end_key); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_DELETE_RANGE, + std::memory_order_relaxed); + return save.commit(); +} + +Status WriteBatch::DeleteRange(ColumnFamilyHandle* column_family, + const SliceParts& begin_key, + const SliceParts& end_key) { + return WriteBatchInternal::DeleteRange(this, GetColumnFamilyID(column_family), + begin_key, end_key); +} + +Status WriteBatchInternal::Merge(WriteBatch* b, uint32_t column_family_id, + const Slice& key, const Slice& value) { + LocalSavePoint save(b); WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); if (column_family_id == 0) { b->rep_.push_back(static_cast(kTypeMerge)); @@ -282,44 +746,203 @@ void WriteBatchInternal::Merge(WriteBatch* b, uint32_t column_family_id, } PutLengthPrefixedSlice(&b->rep_, key); PutLengthPrefixedSlice(&b->rep_, value); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_MERGE, + std::memory_order_relaxed); + return save.commit(); } -void WriteBatch::Merge(ColumnFamilyHandle* column_family, const Slice& key, - const Slice& value) { - WriteBatchInternal::Merge(this, GetColumnFamilyID(column_family), key, value); +Status WriteBatch::Merge(ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) { + return WriteBatchInternal::Merge(this, GetColumnFamilyID(column_family), key, + value); } -void WriteBatch::PutLogData(const Slice& blob) { +Status WriteBatchInternal::Merge(WriteBatch* b, uint32_t column_family_id, + const SliceParts& key, + const SliceParts& value) { + LocalSavePoint save(b); + WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); + if (column_family_id == 0) { + b->rep_.push_back(static_cast(kTypeMerge)); + } else { + b->rep_.push_back(static_cast(kTypeColumnFamilyMerge)); + PutVarint32(&b->rep_, column_family_id); + } + PutLengthPrefixedSliceParts(&b->rep_, key); + PutLengthPrefixedSliceParts(&b->rep_, value); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_MERGE, + std::memory_order_relaxed); + return save.commit(); +} + +Status WriteBatch::Merge(ColumnFamilyHandle* column_family, + const SliceParts& key, const SliceParts& value) { + return WriteBatchInternal::Merge(this, GetColumnFamilyID(column_family), key, + value); +} + +Status WriteBatchInternal::PutBlobIndex(WriteBatch* b, + uint32_t column_family_id, + const Slice& key, const Slice& value) { + LocalSavePoint save(b); + WriteBatchInternal::SetCount(b, WriteBatchInternal::Count(b) + 1); + if (column_family_id == 0) { + b->rep_.push_back(static_cast(kTypeBlobIndex)); + } else { + b->rep_.push_back(static_cast(kTypeColumnFamilyBlobIndex)); + PutVarint32(&b->rep_, column_family_id); + } + PutLengthPrefixedSlice(&b->rep_, key); + PutLengthPrefixedSlice(&b->rep_, value); + b->content_flags_.store(b->content_flags_.load(std::memory_order_relaxed) | + ContentFlags::HAS_BLOB_INDEX, + std::memory_order_relaxed); + return save.commit(); +} + +Status WriteBatch::PutLogData(const Slice& blob) { + LocalSavePoint save(this); rep_.push_back(static_cast(kTypeLogData)); PutLengthPrefixedSlice(&rep_, blob); + return save.commit(); +} + +void WriteBatch::SetSavePoint() { + if (save_points_ == nullptr) { + save_points_ = new SavePoints(); + } + // Record length and count of current batch of writes. + save_points_->stack.push(SavePoint( + GetDataSize(), Count(), content_flags_.load(std::memory_order_relaxed))); +} + +Status WriteBatch::RollbackToSavePoint() { + if (save_points_ == nullptr || save_points_->stack.size() == 0) { + return Status::NotFound(); + } + + // Pop the most recent savepoint off the stack + SavePoint savepoint = save_points_->stack.top(); + save_points_->stack.pop(); + + assert(savepoint.size <= rep_.size()); + assert(savepoint.count <= Count()); + + if (savepoint.size == rep_.size()) { + // No changes to rollback + } else if (savepoint.size == 0) { + // Rollback everything + Clear(); + } else { + rep_.resize(savepoint.size); + WriteBatchInternal::SetCount(this, savepoint.count); + content_flags_.store(savepoint.content_flags, std::memory_order_relaxed); + } + + return Status::OK(); +} + +Status WriteBatch::PopSavePoint() { + if (save_points_ == nullptr || save_points_->stack.size() == 0) { + return Status::NotFound(); + } + + // Pop the most recent savepoint off the stack + save_points_->stack.pop(); + + return Status::OK(); } -namespace { class MemTableInserter : public WriteBatch::Handler { - public: + SequenceNumber sequence_; - ColumnFamilyMemTables* cf_mems_; - bool ignore_missing_column_families_; - uint64_t log_number_; + ColumnFamilyMemTables* const cf_mems_; + FlushScheduler* const flush_scheduler_; + const bool ignore_missing_column_families_; + const uint64_t recovering_log_number_; + // log number that all Memtables inserted into should reference + uint64_t log_number_ref_; DBImpl* db_; - const bool dont_filter_deletes_; + const bool concurrent_memtable_writes_; + bool post_info_created_; + + bool* has_valid_writes_; + // On some (!) platforms just default creating + // a map is too expensive in the Write() path as they + // cause memory allocations though unused. + // Make creation optional but do not incur + // unique_ptr additional allocation + using + MemPostInfoMap = std::map; + using + PostMapType = std::aligned_storage::type; + PostMapType mem_post_info_map_; + // current recovered transaction we are rebuilding (recovery) + WriteBatch* rebuilding_trx_; + + MemPostInfoMap& GetPostMap() { + assert(concurrent_memtable_writes_); + if(!post_info_created_) { + new (&mem_post_info_map_) MemPostInfoMap(); + post_info_created_ = true; + } + return *reinterpret_cast(&mem_post_info_map_); + } + +public: + // cf_mems should not be shared with concurrent inserters + MemTableInserter(SequenceNumber _sequence, ColumnFamilyMemTables* cf_mems, + FlushScheduler* flush_scheduler, + bool ignore_missing_column_families, + uint64_t recovering_log_number, DB* db, + bool concurrent_memtable_writes, + bool* has_valid_writes = nullptr) + : sequence_(_sequence), + cf_mems_(cf_mems), + flush_scheduler_(flush_scheduler), + ignore_missing_column_families_(ignore_missing_column_families), + recovering_log_number_(recovering_log_number), + log_number_ref_(0), + db_(reinterpret_cast(db)), + concurrent_memtable_writes_(concurrent_memtable_writes), + post_info_created_(false), + has_valid_writes_(has_valid_writes), + rebuilding_trx_(nullptr) { + assert(cf_mems_); + } - MemTableInserter(SequenceNumber sequence, ColumnFamilyMemTables* cf_mems, - bool ignore_missing_column_families, uint64_t log_number, - DB* db, const bool dont_filter_deletes) - : sequence_(sequence), - cf_mems_(cf_mems), - ignore_missing_column_families_(ignore_missing_column_families), - log_number_(log_number), - db_(reinterpret_cast(db)), - dont_filter_deletes_(dont_filter_deletes) { - assert(cf_mems); - if (!dont_filter_deletes_) { - assert(db_); + ~MemTableInserter() { + if (post_info_created_) { + reinterpret_cast + (&mem_post_info_map_)->~MemPostInfoMap(); + } + } + + MemTableInserter(const MemTableInserter&) = delete; + MemTableInserter& operator=(const MemTableInserter&) = delete; + + void set_log_number_ref(uint64_t log) { log_number_ref_ = log; } + + SequenceNumber sequence() const { return sequence_; } + + void PostProcess() { + assert(concurrent_memtable_writes_); + // If post info was not created there is nothing + // to process and no need to create on demand + if(post_info_created_) { + for (auto& pair : GetPostMap()) { + pair.first->BatchPostProcess(pair.second); + } } } bool SeekToColumnFamily(uint32_t column_family_id, Status* s) { + // If we are in a concurrent mode, it is the caller's responsibility + // to clone the original ColumnFamilyMemTables so that each thread + // has its own instance. Otherwise, it must be guaranteed that there + // is no concurrent access bool found = cf_mems_->Seek(column_family_id); if (!found) { if (ignore_missing_column_families_) { @@ -330,34 +953,56 @@ class MemTableInserter : public WriteBatch::Handler { } return false; } - if (log_number_ != 0 && log_number_ < cf_mems_->GetLogNumber()) { - // This is true only in recovery environment (log_number_ is always 0 in + if (recovering_log_number_ != 0 && + recovering_log_number_ < cf_mems_->GetLogNumber()) { + // This is true only in recovery environment (recovering_log_number_ is + // always 0 in // non-recovery, regular write code-path) - // * If log_number_ < cf_mems_->GetLogNumber(), this means that column + // * If recovering_log_number_ < cf_mems_->GetLogNumber(), this means that + // column // family already contains updates from this log. We can't apply updates // twice because of update-in-place or merge workloads -- ignore the // update *s = Status::OK(); return false; } + + if (has_valid_writes_ != nullptr) { + *has_valid_writes_ = true; + } + + if (log_number_ref_ > 0) { + cf_mems_->GetMemTable()->RefLogContainingPrepSection(log_number_ref_); + } + return true; } - virtual Status PutCF(uint32_t column_family_id, const Slice& key, - const Slice& value) { + + Status PutCFImpl(uint32_t column_family_id, const Slice& key, + const Slice& value, ValueType value_type) { + if (rebuilding_trx_ != nullptr) { + WriteBatchInternal::Put(rebuilding_trx_, column_family_id, key, value); + return Status::OK(); + } + Status seek_status; if (!SeekToColumnFamily(column_family_id, &seek_status)) { ++sequence_; return seek_status; } + MemTable* mem = cf_mems_->GetMemTable(); - const Options* options = cf_mems_->GetOptions(); - if (!options->inplace_update_support) { - mem->Add(sequence_, kTypeValue, key, value); - } else if (options->inplace_callback == nullptr) { + auto* moptions = mem->GetImmutableMemTableOptions(); + if (!moptions->inplace_update_support) { + mem->Add(sequence_, value_type, key, value, concurrent_memtable_writes_, + get_post_process_info(mem)); + } else if (moptions->inplace_callback == nullptr) { + assert(!concurrent_memtable_writes_); mem->Update(sequence_, key, value); - RecordTick(options->statistics.get(), NUMBER_KEYS_UPDATED); + RecordTick(moptions->statistics, NUMBER_KEYS_UPDATED); } else { - if (mem->UpdateCallback(sequence_, key, value, *options)) { + assert(!concurrent_memtable_writes_); + if (mem->UpdateCallback(sequence_, key, value)) { } else { // key not found in memtable. Do sst get, update, add SnapshotImpl read_from_snapshot; @@ -369,24 +1014,27 @@ class MemTableInserter : public WriteBatch::Handler { std::string merged_value; auto cf_handle = cf_mems_->GetColumnFamilyHandle(); - if (cf_handle == nullptr) { - cf_handle = db_->DefaultColumnFamily(); + Status s = Status::NotSupported(); + if (db_ != nullptr && recovering_log_number_ == 0) { + if (cf_handle == nullptr) { + cf_handle = db_->DefaultColumnFamily(); + } + s = db_->Get(ropts, cf_handle, key, &prev_value); } - Status s = db_->Get(ropts, cf_handle, key, &prev_value); char* prev_buffer = const_cast(prev_value.c_str()); - uint32_t prev_size = prev_value.size(); - auto status = options->inplace_callback(s.ok() ? prev_buffer : nullptr, - s.ok() ? &prev_size : nullptr, - value, &merged_value); + uint32_t prev_size = static_cast(prev_value.size()); + auto status = moptions->inplace_callback(s.ok() ? prev_buffer : nullptr, + s.ok() ? &prev_size : nullptr, + value, &merged_value); if (status == UpdateStatus::UPDATED_INPLACE) { // prev_value is updated in-place with final value. - mem->Add(sequence_, kTypeValue, key, Slice(prev_buffer, prev_size)); - RecordTick(options->statistics.get(), NUMBER_KEYS_WRITTEN); + mem->Add(sequence_, value_type, key, Slice(prev_buffer, prev_size)); + RecordTick(moptions->statistics, NUMBER_KEYS_WRITTEN); } else if (status == UpdateStatus::UPDATED) { // merged_value contains the final value. - mem->Add(sequence_, kTypeValue, key, Slice(merged_value)); - RecordTick(options->statistics.get(), NUMBER_KEYS_WRITTEN); + mem->Add(sequence_, value_type, key, Slice(merged_value)); + RecordTick(moptions->statistics, NUMBER_KEYS_WRITTEN); } } } @@ -394,28 +1042,119 @@ class MemTableInserter : public WriteBatch::Handler { // sequence number. Even if the update eventually fails and does not result // in memtable add/update. sequence_++; + CheckMemtableFull(); + return Status::OK(); + } + + virtual Status PutCF(uint32_t column_family_id, const Slice& key, + const Slice& value) override { + return PutCFImpl(column_family_id, key, value, kTypeValue); + } + + Status DeleteImpl(uint32_t column_family_id, const Slice& key, + const Slice& value, ValueType delete_type) { + MemTable* mem = cf_mems_->GetMemTable(); + mem->Add(sequence_, delete_type, key, value, concurrent_memtable_writes_, + get_post_process_info(mem)); + sequence_++; + CheckMemtableFull(); return Status::OK(); } + virtual Status DeleteCF(uint32_t column_family_id, + const Slice& key) override { + if (rebuilding_trx_ != nullptr) { + WriteBatchInternal::Delete(rebuilding_trx_, column_family_id, key); + return Status::OK(); + } + + Status seek_status; + if (!SeekToColumnFamily(column_family_id, &seek_status)) { + ++sequence_; + return seek_status; + } + + return DeleteImpl(column_family_id, key, Slice(), kTypeDeletion); + } + + virtual Status SingleDeleteCF(uint32_t column_family_id, + const Slice& key) override { + if (rebuilding_trx_ != nullptr) { + WriteBatchInternal::SingleDelete(rebuilding_trx_, column_family_id, key); + return Status::OK(); + } + + Status seek_status; + if (!SeekToColumnFamily(column_family_id, &seek_status)) { + ++sequence_; + return seek_status; + } + + return DeleteImpl(column_family_id, key, Slice(), kTypeSingleDeletion); + } + + virtual Status DeleteRangeCF(uint32_t column_family_id, + const Slice& begin_key, + const Slice& end_key) override { + if (rebuilding_trx_ != nullptr) { + WriteBatchInternal::DeleteRange(rebuilding_trx_, column_family_id, + begin_key, end_key); + return Status::OK(); + } + + Status seek_status; + if (!SeekToColumnFamily(column_family_id, &seek_status)) { + ++sequence_; + return seek_status; + } + if (db_ != nullptr) { + auto cf_handle = cf_mems_->GetColumnFamilyHandle(); + if (cf_handle == nullptr) { + cf_handle = db_->DefaultColumnFamily(); + } + auto* cfd = reinterpret_cast(cf_handle)->cfd(); + if (!cfd->is_delete_range_supported()) { + return Status::NotSupported( + std::string("DeleteRange not supported for table type ") + + cfd->ioptions()->table_factory->Name() + " in CF " + + cfd->GetName()); + } + } + + return DeleteImpl(column_family_id, begin_key, end_key, kTypeRangeDeletion); + } + virtual Status MergeCF(uint32_t column_family_id, const Slice& key, - const Slice& value) { + const Slice& value) override { + assert(!concurrent_memtable_writes_); + if (rebuilding_trx_ != nullptr) { + WriteBatchInternal::Merge(rebuilding_trx_, column_family_id, key, value); + return Status::OK(); + } + Status seek_status; if (!SeekToColumnFamily(column_family_id, &seek_status)) { ++sequence_; return seek_status; } + MemTable* mem = cf_mems_->GetMemTable(); - const Options* options = cf_mems_->GetOptions(); + auto* moptions = mem->GetImmutableMemTableOptions(); bool perform_merge = false; - if (options->max_successive_merges > 0 && db_ != nullptr) { + // If we pass DB through and options.max_successive_merges is hit + // during recovery, Get() will be issued which will try to acquire + // DB mutex and cause deadlock, as DB mutex is already held. + // So we disable merge in recovery + if (moptions->max_successive_merges > 0 && db_ != nullptr && + recovering_log_number_ == 0) { LookupKey lkey(key, sequence_); // Count the number of successive merges at the head // of the key in the memtable size_t num_merges = mem->CountSuccessiveMergeEntries(lkey); - if (num_merges >= options->max_successive_merges) { + if (num_merges >= moptions->max_successive_merges) { perform_merge = true; } } @@ -439,17 +1178,17 @@ class MemTableInserter : public WriteBatch::Handler { Slice get_value_slice = Slice(get_value); // 2) Apply this merge - auto merge_operator = options->merge_operator.get(); + auto merge_operator = moptions->merge_operator; assert(merge_operator); - std::deque operands; - operands.push_front(value.ToString()); std::string new_value; - if (!merge_operator->FullMerge(key, &get_value_slice, operands, - &new_value, options->info_log.get())) { - // Failed to merge! - RecordTick(options->statistics.get(), NUMBER_MERGE_FAILURES); + Status merge_status = MergeHelper::TimedFullMerge( + merge_operator, key, &get_value_slice, {value}, &new_value, + moptions->info_log, moptions->statistics, Env::Default()); + + if (!merge_status.ok()) { + // Failed to merge! // Store the delta in memtable perform_merge = false; } else { @@ -464,59 +1203,249 @@ class MemTableInserter : public WriteBatch::Handler { } sequence_++; + CheckMemtableFull(); return Status::OK(); } - virtual Status DeleteCF(uint32_t column_family_id, const Slice& key) { - Status seek_status; - if (!SeekToColumnFamily(column_family_id, &seek_status)) { - ++sequence_; - return seek_status; + virtual Status PutBlobIndexCF(uint32_t column_family_id, const Slice& key, + const Slice& value) override { + // Same as PutCF except for value type. + return PutCFImpl(column_family_id, key, value, kTypeBlobIndex); + } + + void CheckMemtableFull() { + if (flush_scheduler_ != nullptr) { + auto* cfd = cf_mems_->current(); + assert(cfd != nullptr); + if (cfd->mem()->ShouldScheduleFlush() && + cfd->mem()->MarkFlushScheduled()) { + // MarkFlushScheduled only returns true if we are the one that + // should take action, so no need to dedup further + flush_scheduler_->ScheduleFlush(cfd); + } } - MemTable* mem = cf_mems_->GetMemTable(); - const Options* options = cf_mems_->GetOptions(); - if (!dont_filter_deletes_ && options->filter_deletes) { - SnapshotImpl read_from_snapshot; - read_from_snapshot.number_ = sequence_; - ReadOptions ropts; - ropts.snapshot = &read_from_snapshot; - std::string value; - auto cf_handle = cf_mems_->GetColumnFamilyHandle(); - if (cf_handle == nullptr) { - cf_handle = db_->DefaultColumnFamily(); + } + + Status MarkBeginPrepare() override { + assert(rebuilding_trx_ == nullptr); + assert(db_); + + if (recovering_log_number_ != 0) { + // during recovery we rebuild a hollow transaction + // from all encountered prepare sections of the wal + if (db_->allow_2pc() == false) { + return Status::NotSupported( + "WAL contains prepared transactions. Open with " + "TransactionDB::Open()."); } - if (!db_->KeyMayExist(ropts, cf_handle, key, &value)) { - RecordTick(options->statistics.get(), NUMBER_FILTERED_DELETES); - return Status::OK(); + + // we are now iterating through a prepared section + rebuilding_trx_ = new WriteBatch(); + if (has_valid_writes_ != nullptr) { + *has_valid_writes_ = true; } + } else { + // in non-recovery we ignore prepare markers + // and insert the values directly. making sure we have a + // log for each insertion to reference. + assert(log_number_ref_ > 0); } - mem->Add(sequence_, kTypeDeletion, key, Slice()); - sequence_++; + + return Status::OK(); + } + + Status MarkEndPrepare(const Slice& name) override { + assert(db_); + assert((rebuilding_trx_ != nullptr) == (recovering_log_number_ != 0)); + + if (recovering_log_number_ != 0) { + assert(db_->allow_2pc()); + db_->InsertRecoveredTransaction(recovering_log_number_, name.ToString(), + rebuilding_trx_); + rebuilding_trx_ = nullptr; + } else { + assert(rebuilding_trx_ == nullptr); + assert(log_number_ref_ > 0); + } + + return Status::OK(); + } + + Status MarkCommit(const Slice& name) override { + assert(db_); + + Status s; + + if (recovering_log_number_ != 0) { + // in recovery when we encounter a commit marker + // we lookup this transaction in our set of rebuilt transactions + // and commit. + auto trx = db_->GetRecoveredTransaction(name.ToString()); + + // the log contaiting the prepared section may have + // been released in the last incarnation because the + // data was flushed to L0 + if (trx != nullptr) { + // at this point individual CF lognumbers will prevent + // duplicate re-insertion of values. + assert(log_number_ref_ == 0); + // all insertes must reference this trx log number + log_number_ref_ = trx->log_number_; + s = trx->batch_->Iterate(this); + log_number_ref_ = 0; + + if (s.ok()) { + db_->DeleteRecoveredTransaction(name.ToString()); + } + if (has_valid_writes_ != nullptr) { + *has_valid_writes_ = true; + } + } + } else { + // in non recovery we simply ignore this tag + } + + return s; + } + + Status MarkRollback(const Slice& name) override { + assert(db_); + + if (recovering_log_number_ != 0) { + auto trx = db_->GetRecoveredTransaction(name.ToString()); + + // the log containing the transactions prep section + // may have been released in the previous incarnation + // because we knew it had been rolled back + if (trx != nullptr) { + db_->DeleteRecoveredTransaction(name.ToString()); + } + } else { + // in non recovery we simply ignore this tag + } + return Status::OK(); } + + private: + MemTablePostProcessInfo* get_post_process_info(MemTable* mem) { + if (!concurrent_memtable_writes_) { + // No need to batch counters locally if we don't use concurrent mode. + return nullptr; + } + return &GetPostMap()[mem]; + } }; -} // namespace -Status WriteBatchInternal::InsertInto(const WriteBatch* b, +// This function can only be called in these conditions: +// 1) During Recovery() +// 2) During Write(), in a single-threaded write thread +// 3) During Write(), in a concurrent context where memtables has been cloned +// The reason is that it calls memtables->Seek(), which has a stateful cache +Status WriteBatchInternal::InsertInto(WriteThread::WriteGroup& write_group, + SequenceNumber sequence, + ColumnFamilyMemTables* memtables, + FlushScheduler* flush_scheduler, + bool ignore_missing_column_families, + uint64_t recovery_log_number, DB* db, + bool concurrent_memtable_writes) { + MemTableInserter inserter(sequence, memtables, flush_scheduler, + ignore_missing_column_families, recovery_log_number, + db, concurrent_memtable_writes); + for (auto w : write_group) { + if (!w->ShouldWriteToMemtable()) { + continue; + } + SetSequence(w->batch, inserter.sequence()); + w->sequence = inserter.sequence(); + inserter.set_log_number_ref(w->log_ref); + w->status = w->batch->Iterate(&inserter); + if (!w->status.ok()) { + return w->status; + } + } + return Status::OK(); +} + +Status WriteBatchInternal::InsertInto(WriteThread::Writer* writer, + SequenceNumber sequence, ColumnFamilyMemTables* memtables, + FlushScheduler* flush_scheduler, bool ignore_missing_column_families, uint64_t log_number, DB* db, - const bool dont_filter_deletes) { - MemTableInserter inserter(WriteBatchInternal::Sequence(b), memtables, + bool concurrent_memtable_writes) { + assert(writer->ShouldWriteToMemtable()); + MemTableInserter inserter(sequence, memtables, flush_scheduler, + ignore_missing_column_families, log_number, db, + concurrent_memtable_writes); + SetSequence(writer->batch, sequence); + inserter.set_log_number_ref(writer->log_ref); + Status s = writer->batch->Iterate(&inserter); + if (concurrent_memtable_writes) { + inserter.PostProcess(); + } + return s; +} + +Status WriteBatchInternal::InsertInto( + const WriteBatch* batch, ColumnFamilyMemTables* memtables, + FlushScheduler* flush_scheduler, bool ignore_missing_column_families, + uint64_t log_number, DB* db, bool concurrent_memtable_writes, + SequenceNumber* last_seq_used, bool* has_valid_writes) { + MemTableInserter inserter(Sequence(batch), memtables, flush_scheduler, ignore_missing_column_families, log_number, db, - dont_filter_deletes); - return b->Iterate(&inserter); + concurrent_memtable_writes, has_valid_writes); + Status s = batch->Iterate(&inserter); + if (last_seq_used != nullptr) { + *last_seq_used = inserter.sequence(); + } + if (concurrent_memtable_writes) { + inserter.PostProcess(); + } + return s; } -void WriteBatchInternal::SetContents(WriteBatch* b, const Slice& contents) { - assert(contents.size() >= kHeader); +Status WriteBatchInternal::SetContents(WriteBatch* b, const Slice& contents) { + assert(contents.size() >= WriteBatchInternal::kHeader); b->rep_.assign(contents.data(), contents.size()); + b->content_flags_.store(ContentFlags::DEFERRED, std::memory_order_relaxed); + return Status::OK(); +} + +Status WriteBatchInternal::Append(WriteBatch* dst, const WriteBatch* src, + const bool wal_only) { + size_t src_len; + int src_count; + uint32_t src_flags; + + const SavePoint& batch_end = src->GetWalTerminationPoint(); + + if (wal_only && !batch_end.is_cleared()) { + src_len = batch_end.size - WriteBatchInternal::kHeader; + src_count = batch_end.count; + src_flags = batch_end.content_flags; + } else { + src_len = src->rep_.size() - WriteBatchInternal::kHeader; + src_count = Count(src); + src_flags = src->content_flags_.load(std::memory_order_relaxed); + } + + SetCount(dst, Count(dst) + src_count); + assert(src->rep_.size() >= WriteBatchInternal::kHeader); + dst->rep_.append(src->rep_.data() + WriteBatchInternal::kHeader, src_len); + dst->content_flags_.store( + dst->content_flags_.load(std::memory_order_relaxed) | src_flags, + std::memory_order_relaxed); + return Status::OK(); } -void WriteBatchInternal::Append(WriteBatch* dst, const WriteBatch* src) { - SetCount(dst, Count(dst) + Count(src)); - assert(src->rep_.size() >= kHeader); - dst->rep_.append(src->rep_.data() + kHeader, src->rep_.size() - kHeader); +size_t WriteBatchInternal::AppendedByteSize(size_t leftByteSize, + size_t rightByteSize) { + if (leftByteSize == 0 || rightByteSize == 0) { + return leftByteSize + rightByteSize; + } else { + return leftByteSize + rightByteSize - WriteBatchInternal::kHeader; + } } } // namespace rocksdb diff --git a/db/write_batch_base.cc b/db/write_batch_base.cc new file mode 100644 index 00000000000..5522c1ff77f --- /dev/null +++ b/db/write_batch_base.cc @@ -0,0 +1,94 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "rocksdb/write_batch_base.h" + +#include + +#include "rocksdb/slice.h" +#include "rocksdb/status.h" + +namespace rocksdb { + +// Simple implementation of SlicePart variants of Put(). Child classes +// can override these method with more performant solutions if they choose. +Status WriteBatchBase::Put(ColumnFamilyHandle* column_family, + const SliceParts& key, const SliceParts& value) { + std::string key_buf, value_buf; + Slice key_slice(key, &key_buf); + Slice value_slice(value, &value_buf); + + return Put(column_family, key_slice, value_slice); +} + +Status WriteBatchBase::Put(const SliceParts& key, const SliceParts& value) { + std::string key_buf, value_buf; + Slice key_slice(key, &key_buf); + Slice value_slice(value, &value_buf); + + return Put(key_slice, value_slice); +} + +Status WriteBatchBase::Delete(ColumnFamilyHandle* column_family, + const SliceParts& key) { + std::string key_buf; + Slice key_slice(key, &key_buf); + return Delete(column_family, key_slice); +} + +Status WriteBatchBase::Delete(const SliceParts& key) { + std::string key_buf; + Slice key_slice(key, &key_buf); + return Delete(key_slice); +} + +Status WriteBatchBase::SingleDelete(ColumnFamilyHandle* column_family, + const SliceParts& key) { + std::string key_buf; + Slice key_slice(key, &key_buf); + return SingleDelete(column_family, key_slice); +} + +Status WriteBatchBase::SingleDelete(const SliceParts& key) { + std::string key_buf; + Slice key_slice(key, &key_buf); + return SingleDelete(key_slice); +} + +Status WriteBatchBase::DeleteRange(ColumnFamilyHandle* column_family, + const SliceParts& begin_key, + const SliceParts& end_key) { + std::string begin_key_buf, end_key_buf; + Slice begin_key_slice(begin_key, &begin_key_buf); + Slice end_key_slice(end_key, &end_key_buf); + return DeleteRange(column_family, begin_key_slice, end_key_slice); +} + +Status WriteBatchBase::DeleteRange(const SliceParts& begin_key, + const SliceParts& end_key) { + std::string begin_key_buf, end_key_buf; + Slice begin_key_slice(begin_key, &begin_key_buf); + Slice end_key_slice(end_key, &end_key_buf); + return DeleteRange(begin_key_slice, end_key_slice); +} + +Status WriteBatchBase::Merge(ColumnFamilyHandle* column_family, + const SliceParts& key, const SliceParts& value) { + std::string key_buf, value_buf; + Slice key_slice(key, &key_buf); + Slice value_slice(value, &value_buf); + + return Merge(column_family, key_slice, value_slice); +} + +Status WriteBatchBase::Merge(const SliceParts& key, const SliceParts& value) { + std::string key_buf, value_buf; + Slice key_slice(key, &key_buf); + Slice value_slice(value, &value_buf); + + return Merge(key_slice, value_slice); +} + +} // namespace rocksdb diff --git a/db/write_batch_internal.h b/db/write_batch_internal.h index 615a47f5ebd..2408686f12b 100644 --- a/db/write_batch_internal.h +++ b/db/write_batch_internal.h @@ -1,21 +1,26 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #pragma once +#include +#include "db/write_thread.h" #include "rocksdb/types.h" #include "rocksdb/write_batch.h" #include "rocksdb/db.h" #include "rocksdb/options.h" +#include "util/autovector.h" namespace rocksdb { class MemTable; +class FlushScheduler; +class ColumnFamilyData; class ColumnFamilyMemTables { public: @@ -26,14 +31,14 @@ class ColumnFamilyMemTables { // been processed) virtual uint64_t GetLogNumber() const = 0; virtual MemTable* GetMemTable() const = 0; - virtual const Options* GetOptions() const = 0; virtual ColumnFamilyHandle* GetColumnFamilyHandle() = 0; + virtual ColumnFamilyData* current() { return nullptr; } }; class ColumnFamilyMemTablesDefault : public ColumnFamilyMemTables { public: - ColumnFamilyMemTablesDefault(MemTable* mem, const Options* options) - : ok_(false), mem_(mem), options_(options) {} + explicit ColumnFamilyMemTablesDefault(MemTable* mem) + : ok_(false), mem_(mem) {} bool Seek(uint32_t column_family_id) override { ok_ = (column_family_id == 0); @@ -47,38 +52,63 @@ class ColumnFamilyMemTablesDefault : public ColumnFamilyMemTables { return mem_; } - const Options* GetOptions() const override { - assert(ok_); - return options_; - } - ColumnFamilyHandle* GetColumnFamilyHandle() override { return nullptr; } private: bool ok_; MemTable* mem_; - const Options* const options_; }; // WriteBatchInternal provides static methods for manipulating a // WriteBatch that we don't want in the public WriteBatch interface. class WriteBatchInternal { public: + + // WriteBatch header has an 8-byte sequence number followed by a 4-byte count. + static const size_t kHeader = 12; + // WriteBatch methods with column_family_id instead of ColumnFamilyHandle* - static void Put(WriteBatch* batch, uint32_t column_family_id, - const Slice& key, const Slice& value); + static Status Put(WriteBatch* batch, uint32_t column_family_id, + const Slice& key, const Slice& value); - static void Put(WriteBatch* batch, uint32_t column_family_id, - const SliceParts& key, const SliceParts& value); + static Status Put(WriteBatch* batch, uint32_t column_family_id, + const SliceParts& key, const SliceParts& value); - static void Delete(WriteBatch* batch, uint32_t column_family_id, - const SliceParts& key); + static Status Delete(WriteBatch* batch, uint32_t column_family_id, + const SliceParts& key); - static void Delete(WriteBatch* batch, uint32_t column_family_id, - const Slice& key); + static Status Delete(WriteBatch* batch, uint32_t column_family_id, + const Slice& key); - static void Merge(WriteBatch* batch, uint32_t column_family_id, - const Slice& key, const Slice& value); + static Status SingleDelete(WriteBatch* batch, uint32_t column_family_id, + const SliceParts& key); + + static Status SingleDelete(WriteBatch* batch, uint32_t column_family_id, + const Slice& key); + + static Status DeleteRange(WriteBatch* b, uint32_t column_family_id, + const Slice& begin_key, const Slice& end_key); + + static Status DeleteRange(WriteBatch* b, uint32_t column_family_id, + const SliceParts& begin_key, + const SliceParts& end_key); + + static Status Merge(WriteBatch* batch, uint32_t column_family_id, + const Slice& key, const Slice& value); + + static Status Merge(WriteBatch* batch, uint32_t column_family_id, + const SliceParts& key, const SliceParts& value); + + static Status PutBlobIndex(WriteBatch* batch, uint32_t column_family_id, + const Slice& key, const Slice& value); + + static Status MarkEndPrepare(WriteBatch* batch, const Slice& xid); + + static Status MarkRollback(WriteBatch* batch, const Slice& xid); + + static Status MarkCommit(WriteBatch* batch, const Slice& xid); + + static Status InsertNoop(WriteBatch* batch); // Return the number of entries in the batch. static int Count(const WriteBatch* batch); @@ -93,6 +123,10 @@ class WriteBatchInternal { // this batch. static void SetSequence(WriteBatch* batch, SequenceNumber seq); + // Returns the offset of the first entry in the batch. + // This offset is only valid if the batch is not empty. + static size_t GetFirstOffset(WriteBatch* batch); + static Slice Contents(const WriteBatch* batch) { return Slice(batch->rep_); } @@ -101,26 +135,96 @@ class WriteBatchInternal { return batch->rep_.size(); } - static void SetContents(WriteBatch* batch, const Slice& contents); + static Status SetContents(WriteBatch* batch, const Slice& contents); - // Inserts batch entries into memtable - // If dont_filter_deletes is false AND options.filter_deletes is true, - // then --> Drops deletes in batch if db->KeyMayExist returns false - // If ignore_missing_column_families == true. WriteBatch referencing - // non-existing column family should be ignored. - // However, if ignore_missing_column_families == false, any WriteBatch - // referencing non-existing column family will return a InvalidArgument() - // failure. + // Inserts batches[i] into memtable, for i in 0..num_batches-1 inclusive. + // + // If ignore_missing_column_families == true. WriteBatch + // referencing non-existing column family will be ignored. + // If ignore_missing_column_families == false, processing of the + // batches will be stopped if a reference is found to a non-existing + // column family and InvalidArgument() will be returned. The writes + // in batches may be only partially applied at that point. // // If log_number is non-zero, the memtable will be updated only if - // memtables->GetLogNumber() >= log_number + // memtables->GetLogNumber() >= log_number. + // + // If flush_scheduler is non-null, it will be invoked if the memtable + // should be flushed. + // + // Under concurrent use, the caller is responsible for making sure that + // the memtables object itself is thread-local. + static Status InsertInto(WriteThread::WriteGroup& write_group, + SequenceNumber sequence, + ColumnFamilyMemTables* memtables, + FlushScheduler* flush_scheduler, + bool ignore_missing_column_families = false, + uint64_t log_number = 0, DB* db = nullptr, + bool concurrent_memtable_writes = false); + + // Convenience form of InsertInto when you have only one batch + // last_seq_used returns the last sequnce number used in a MemTable insert static Status InsertInto(const WriteBatch* batch, ColumnFamilyMemTables* memtables, + FlushScheduler* flush_scheduler, + bool ignore_missing_column_families = false, + uint64_t log_number = 0, DB* db = nullptr, + bool concurrent_memtable_writes = false, + SequenceNumber* last_seq_used = nullptr, + bool* has_valid_writes = nullptr); + + static Status InsertInto(WriteThread::Writer* writer, SequenceNumber sequence, + ColumnFamilyMemTables* memtables, + FlushScheduler* flush_scheduler, bool ignore_missing_column_families = false, uint64_t log_number = 0, DB* db = nullptr, - const bool dont_filter_deletes = true); + bool concurrent_memtable_writes = false); - static void Append(WriteBatch* dst, const WriteBatch* src); + static Status Append(WriteBatch* dst, const WriteBatch* src, + const bool WAL_only = false); + + // Returns the byte size of appending a WriteBatch with ByteSize + // leftByteSize and a WriteBatch with ByteSize rightByteSize + static size_t AppendedByteSize(size_t leftByteSize, size_t rightByteSize); +}; + +// LocalSavePoint is similar to a scope guard +class LocalSavePoint { + public: + explicit LocalSavePoint(WriteBatch* batch) + : batch_(batch), + savepoint_(batch->GetDataSize(), batch->Count(), + batch->content_flags_.load(std::memory_order_relaxed)) +#ifndef NDEBUG + , + committed_(false) +#endif + { + } + +#ifndef NDEBUG + ~LocalSavePoint() { assert(committed_); } +#endif + Status commit() { +#ifndef NDEBUG + committed_ = true; +#endif + if (batch_->max_bytes_ && batch_->rep_.size() > batch_->max_bytes_) { + batch_->rep_.resize(savepoint_.size); + WriteBatchInternal::SetCount(batch_, savepoint_.count); + batch_->content_flags_.store(savepoint_.content_flags, + std::memory_order_relaxed); + return Status::MemoryLimit(); + } + return Status::OK(); + } + + private: + WriteBatch* batch_; + SavePoint savepoint_; +#ifndef NDEBUG + bool committed_; +#endif }; } // namespace rocksdb diff --git a/db/write_batch_test.cc b/db/write_batch_test.cc index 1d30552b31b..4584793abe1 100644 --- a/db/write_batch_test.cc +++ b/db/write_batch_test.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,13 +10,16 @@ #include "rocksdb/db.h" #include -#include "db/memtable.h" #include "db/column_family.h" +#include "db/memtable.h" #include "db/write_batch_internal.h" #include "rocksdb/env.h" #include "rocksdb/memtablerep.h" #include "rocksdb/utilities/write_batch_with_index.h" +#include "rocksdb/write_buffer_manager.h" +#include "table/scoped_arena_iterator.h" #include "util/logging.h" +#include "util/string_util.h" #include "util/testharness.h" namespace rocksdb { @@ -26,48 +29,94 @@ static std::string PrintContents(WriteBatch* b) { auto factory = std::make_shared(); Options options; options.memtable_factory = factory; - MemTable* mem = new MemTable(cmp, options); + ImmutableCFOptions ioptions(options); + WriteBufferManager wb(options.db_write_buffer_size); + MemTable* mem = new MemTable(cmp, ioptions, MutableCFOptions(options), &wb, + kMaxSequenceNumber, 0 /* column_family_id */); mem->Ref(); std::string state; - ColumnFamilyMemTablesDefault cf_mems_default(mem, &options); - Status s = WriteBatchInternal::InsertInto(b, &cf_mems_default); + ColumnFamilyMemTablesDefault cf_mems_default(mem); + Status s = WriteBatchInternal::InsertInto(b, &cf_mems_default, nullptr); int count = 0; - Iterator* iter = mem->NewIterator(ReadOptions()); - for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { - ParsedInternalKey ikey; - memset((void *)&ikey, 0, sizeof(ikey)); - ASSERT_TRUE(ParseInternalKey(iter->key(), &ikey)); - switch (ikey.type) { - case kTypeValue: - state.append("Put("); - state.append(ikey.user_key.ToString()); - state.append(", "); - state.append(iter->value().ToString()); - state.append(")"); - count++; - break; - case kTypeMerge: - state.append("Merge("); - state.append(ikey.user_key.ToString()); - state.append(", "); - state.append(iter->value().ToString()); - state.append(")"); - count++; - break; - case kTypeDeletion: - state.append("Delete("); - state.append(ikey.user_key.ToString()); - state.append(")"); - count++; - break; - default: - assert(false); - break; + int put_count = 0; + int delete_count = 0; + int single_delete_count = 0; + int delete_range_count = 0; + int merge_count = 0; + for (int i = 0; i < 2; ++i) { + Arena arena; + ScopedArenaIterator arena_iter_guard; + std::unique_ptr iter_guard; + InternalIterator* iter; + if (i == 0) { + iter = mem->NewIterator(ReadOptions(), &arena); + arena_iter_guard.set(iter); + } else { + iter = mem->NewRangeTombstoneIterator(ReadOptions()); + iter_guard.reset(iter); + } + if (iter == nullptr) { + continue; + } + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ParsedInternalKey ikey; + ikey.clear(); + EXPECT_TRUE(ParseInternalKey(iter->key(), &ikey)); + switch (ikey.type) { + case kTypeValue: + state.append("Put("); + state.append(ikey.user_key.ToString()); + state.append(", "); + state.append(iter->value().ToString()); + state.append(")"); + count++; + put_count++; + break; + case kTypeDeletion: + state.append("Delete("); + state.append(ikey.user_key.ToString()); + state.append(")"); + count++; + delete_count++; + break; + case kTypeSingleDeletion: + state.append("SingleDelete("); + state.append(ikey.user_key.ToString()); + state.append(")"); + count++; + single_delete_count++; + break; + case kTypeRangeDeletion: + state.append("DeleteRange("); + state.append(ikey.user_key.ToString()); + state.append(", "); + state.append(iter->value().ToString()); + state.append(")"); + count++; + delete_range_count++; + break; + case kTypeMerge: + state.append("Merge("); + state.append(ikey.user_key.ToString()); + state.append(", "); + state.append(iter->value().ToString()); + state.append(")"); + count++; + merge_count++; + break; + default: + assert(false); + break; + } + state.append("@"); + state.append(NumberToString(ikey.sequence)); } - state.append("@"); - state.append(NumberToString(ikey.sequence)); } - delete iter; + EXPECT_EQ(b->HasPut(), put_count > 0); + EXPECT_EQ(b->HasDelete(), delete_count > 0); + EXPECT_EQ(b->HasSingleDelete(), single_delete_count > 0); + EXPECT_EQ(b->HasDeleteRange(), delete_range_count > 0); + EXPECT_EQ(b->HasMerge(), merge_count > 0); if (!s.ok()) { state.append(s.ToString()); } else if (count != WriteBatchInternal::Count(b)) { @@ -77,31 +126,34 @@ static std::string PrintContents(WriteBatch* b) { return state; } -class WriteBatchTest { }; +class WriteBatchTest : public testing::Test {}; -TEST(WriteBatchTest, Empty) { +TEST_F(WriteBatchTest, Empty) { WriteBatch batch; ASSERT_EQ("", PrintContents(&batch)); ASSERT_EQ(0, WriteBatchInternal::Count(&batch)); ASSERT_EQ(0, batch.Count()); } -TEST(WriteBatchTest, Multiple) { +TEST_F(WriteBatchTest, Multiple) { WriteBatch batch; batch.Put(Slice("foo"), Slice("bar")); batch.Delete(Slice("box")); + batch.DeleteRange(Slice("bar"), Slice("foo")); batch.Put(Slice("baz"), Slice("boo")); WriteBatchInternal::SetSequence(&batch, 100); ASSERT_EQ(100U, WriteBatchInternal::Sequence(&batch)); - ASSERT_EQ(3, WriteBatchInternal::Count(&batch)); - ASSERT_EQ("Put(baz, boo)@102" - "Delete(box)@101" - "Put(foo, bar)@100", - PrintContents(&batch)); - ASSERT_EQ(3, batch.Count()); + ASSERT_EQ(4, WriteBatchInternal::Count(&batch)); + ASSERT_EQ( + "Put(baz, boo)@103" + "Delete(box)@101" + "Put(foo, bar)@100" + "DeleteRange(bar, foo)@102", + PrintContents(&batch)); + ASSERT_EQ(4, batch.Count()); } -TEST(WriteBatchTest, Corruption) { +TEST_F(WriteBatchTest, Corruption) { WriteBatch batch; batch.Put(Slice("foo"), Slice("bar")); batch.Delete(Slice("box")); @@ -114,7 +166,7 @@ TEST(WriteBatchTest, Corruption) { PrintContents(&batch)); } -TEST(WriteBatchTest, Append) { +TEST_F(WriteBatchTest, Append) { WriteBatch b1, b2; WriteBatchInternal::SetSequence(&b1, 200); WriteBatchInternal::SetSequence(&b2, 300); @@ -142,118 +194,385 @@ TEST(WriteBatchTest, Append) { "Delete(foo)@203", PrintContents(&b1)); ASSERT_EQ(4, b1.Count()); + b2.Clear(); + b2.Put("c", "cc"); + b2.Put("d", "dd"); + b2.MarkWalTerminationPoint(); + b2.Put("e", "ee"); + WriteBatchInternal::Append(&b1, &b2, /*wal only*/ true); + ASSERT_EQ( + "Put(a, va)@200" + "Put(b, vb)@202" + "Put(b, vb)@201" + "Put(c, cc)@204" + "Put(d, dd)@205" + "Delete(foo)@203", + PrintContents(&b1)); + ASSERT_EQ(6, b1.Count()); + ASSERT_EQ( + "Put(c, cc)@0" + "Put(d, dd)@1" + "Put(e, ee)@2", + PrintContents(&b2)); + ASSERT_EQ(3, b2.Count()); +} + +TEST_F(WriteBatchTest, SingleDeletion) { + WriteBatch batch; + WriteBatchInternal::SetSequence(&batch, 100); + ASSERT_EQ("", PrintContents(&batch)); + ASSERT_EQ(0, batch.Count()); + batch.Put("a", "va"); + ASSERT_EQ("Put(a, va)@100", PrintContents(&batch)); + ASSERT_EQ(1, batch.Count()); + batch.SingleDelete("a"); + ASSERT_EQ( + "SingleDelete(a)@101" + "Put(a, va)@100", + PrintContents(&batch)); + ASSERT_EQ(2, batch.Count()); } namespace { struct TestHandler : public WriteBatch::Handler { std::string seen; virtual Status PutCF(uint32_t column_family_id, const Slice& key, - const Slice& value) { + const Slice& value) override { if (column_family_id == 0) { seen += "Put(" + key.ToString() + ", " + value.ToString() + ")"; } else { - seen += "PutCF(" + std::to_string(column_family_id) + ", " + + seen += "PutCF(" + ToString(column_family_id) + ", " + key.ToString() + ", " + value.ToString() + ")"; } return Status::OK(); } + virtual Status DeleteCF(uint32_t column_family_id, + const Slice& key) override { + if (column_family_id == 0) { + seen += "Delete(" + key.ToString() + ")"; + } else { + seen += "DeleteCF(" + ToString(column_family_id) + ", " + + key.ToString() + ")"; + } + return Status::OK(); + } + virtual Status SingleDeleteCF(uint32_t column_family_id, + const Slice& key) override { + if (column_family_id == 0) { + seen += "SingleDelete(" + key.ToString() + ")"; + } else { + seen += "SingleDeleteCF(" + ToString(column_family_id) + ", " + + key.ToString() + ")"; + } + return Status::OK(); + } + virtual Status DeleteRangeCF(uint32_t column_family_id, + const Slice& begin_key, + const Slice& end_key) override { + if (column_family_id == 0) { + seen += "DeleteRange(" + begin_key.ToString() + ", " + + end_key.ToString() + ")"; + } else { + seen += "DeleteRangeCF(" + ToString(column_family_id) + ", " + + begin_key.ToString() + ", " + end_key.ToString() + ")"; + } + return Status::OK(); + } virtual Status MergeCF(uint32_t column_family_id, const Slice& key, - const Slice& value) { + const Slice& value) override { if (column_family_id == 0) { seen += "Merge(" + key.ToString() + ", " + value.ToString() + ")"; } else { - seen += "MergeCF(" + std::to_string(column_family_id) + ", " + + seen += "MergeCF(" + ToString(column_family_id) + ", " + key.ToString() + ", " + value.ToString() + ")"; } return Status::OK(); } - virtual void LogData(const Slice& blob) { + virtual void LogData(const Slice& blob) override { seen += "LogData(" + blob.ToString() + ")"; } - virtual Status DeleteCF(uint32_t column_family_id, const Slice& key) { - if (column_family_id == 0) { - seen += "Delete(" + key.ToString() + ")"; - } else { - seen += "DeleteCF(" + std::to_string(column_family_id) + ", " + - key.ToString() + ")"; - } + virtual Status MarkBeginPrepare() override { + seen += "MarkBeginPrepare()"; + return Status::OK(); + } + virtual Status MarkEndPrepare(const Slice& xid) override { + seen += "MarkEndPrepare(" + xid.ToString() + ")"; + return Status::OK(); + } + virtual Status MarkCommit(const Slice& xid) override { + seen += "MarkCommit(" + xid.ToString() + ")"; + return Status::OK(); + } + virtual Status MarkRollback(const Slice& xid) override { + seen += "MarkRollback(" + xid.ToString() + ")"; return Status::OK(); } }; } -TEST(WriteBatchTest, Blob) { +TEST_F(WriteBatchTest, PutNotImplemented) { + WriteBatch batch; + batch.Put(Slice("k1"), Slice("v1")); + ASSERT_EQ(1, batch.Count()); + ASSERT_EQ("Put(k1, v1)@0", PrintContents(&batch)); + + WriteBatch::Handler handler; + ASSERT_OK(batch.Iterate(&handler)); +} + +TEST_F(WriteBatchTest, DeleteNotImplemented) { + WriteBatch batch; + batch.Delete(Slice("k2")); + ASSERT_EQ(1, batch.Count()); + ASSERT_EQ("Delete(k2)@0", PrintContents(&batch)); + + WriteBatch::Handler handler; + ASSERT_OK(batch.Iterate(&handler)); +} + +TEST_F(WriteBatchTest, SingleDeleteNotImplemented) { + WriteBatch batch; + batch.SingleDelete(Slice("k2")); + ASSERT_EQ(1, batch.Count()); + ASSERT_EQ("SingleDelete(k2)@0", PrintContents(&batch)); + + WriteBatch::Handler handler; + ASSERT_OK(batch.Iterate(&handler)); +} + +TEST_F(WriteBatchTest, MergeNotImplemented) { + WriteBatch batch; + batch.Merge(Slice("foo"), Slice("bar")); + ASSERT_EQ(1, batch.Count()); + ASSERT_EQ("Merge(foo, bar)@0", PrintContents(&batch)); + + WriteBatch::Handler handler; + ASSERT_OK(batch.Iterate(&handler)); +} + +TEST_F(WriteBatchTest, Blob) { WriteBatch batch; batch.Put(Slice("k1"), Slice("v1")); batch.Put(Slice("k2"), Slice("v2")); batch.Put(Slice("k3"), Slice("v3")); batch.PutLogData(Slice("blob1")); batch.Delete(Slice("k2")); + batch.SingleDelete(Slice("k3")); batch.PutLogData(Slice("blob2")); batch.Merge(Slice("foo"), Slice("bar")); - ASSERT_EQ(5, batch.Count()); - ASSERT_EQ("Merge(foo, bar)@4" - "Put(k1, v1)@0" - "Delete(k2)@3" - "Put(k2, v2)@1" - "Put(k3, v3)@2", - PrintContents(&batch)); + ASSERT_EQ(6, batch.Count()); + ASSERT_EQ( + "Merge(foo, bar)@5" + "Put(k1, v1)@0" + "Delete(k2)@3" + "Put(k2, v2)@1" + "SingleDelete(k3)@4" + "Put(k3, v3)@2", + PrintContents(&batch)); TestHandler handler; batch.Iterate(&handler); ASSERT_EQ( - "Put(k1, v1)" - "Put(k2, v2)" - "Put(k3, v3)" - "LogData(blob1)" - "Delete(k2)" - "LogData(blob2)" - "Merge(foo, bar)", - handler.seen); + "Put(k1, v1)" + "Put(k2, v2)" + "Put(k3, v3)" + "LogData(blob1)" + "Delete(k2)" + "SingleDelete(k3)" + "LogData(blob2)" + "Merge(foo, bar)", + handler.seen); } -TEST(WriteBatchTest, Continue) { +TEST_F(WriteBatchTest, PrepareCommit) { + WriteBatch batch; + WriteBatchInternal::InsertNoop(&batch); + batch.Put(Slice("k1"), Slice("v1")); + batch.Put(Slice("k2"), Slice("v2")); + batch.SetSavePoint(); + WriteBatchInternal::MarkEndPrepare(&batch, Slice("xid1")); + Status s = batch.RollbackToSavePoint(); + ASSERT_EQ(s, Status::NotFound()); + WriteBatchInternal::MarkCommit(&batch, Slice("xid1")); + WriteBatchInternal::MarkRollback(&batch, Slice("xid1")); + ASSERT_EQ(2, batch.Count()); + + TestHandler handler; + batch.Iterate(&handler); + ASSERT_EQ( + "MarkBeginPrepare()" + "Put(k1, v1)" + "Put(k2, v2)" + "MarkEndPrepare(xid1)" + "MarkCommit(xid1)" + "MarkRollback(xid1)", + handler.seen); +} + +// It requires more than 30GB of memory to run the test. With single memory +// allocation of more than 30GB. +// Not all platform can run it. Also it runs a long time. So disable it. +TEST_F(WriteBatchTest, DISABLED_ManyUpdates) { + // Insert key and value of 3GB and push total batch size to 12GB. + static const size_t kKeyValueSize = 4u; + static const uint32_t kNumUpdates = 3 << 30; + std::string raw(kKeyValueSize, 'A'); + WriteBatch batch(kNumUpdates * (4 + kKeyValueSize * 2) + 1024u); + char c = 'A'; + for (uint32_t i = 0; i < kNumUpdates; i++) { + if (c > 'Z') { + c = 'A'; + } + raw[0] = c; + raw[raw.length() - 1] = c; + c++; + batch.Put(raw, raw); + } + + ASSERT_EQ(kNumUpdates, batch.Count()); + + struct NoopHandler : public WriteBatch::Handler { + uint32_t num_seen = 0; + char expected_char = 'A'; + virtual Status PutCF(uint32_t column_family_id, const Slice& key, + const Slice& value) override { + EXPECT_EQ(kKeyValueSize, key.size()); + EXPECT_EQ(kKeyValueSize, value.size()); + EXPECT_EQ(expected_char, key[0]); + EXPECT_EQ(expected_char, value[0]); + EXPECT_EQ(expected_char, key[kKeyValueSize - 1]); + EXPECT_EQ(expected_char, value[kKeyValueSize - 1]); + expected_char++; + if (expected_char > 'Z') { + expected_char = 'A'; + } + ++num_seen; + return Status::OK(); + } + virtual Status DeleteCF(uint32_t column_family_id, + const Slice& key) override { + ADD_FAILURE(); + return Status::OK(); + } + virtual Status SingleDeleteCF(uint32_t column_family_id, + const Slice& key) override { + ADD_FAILURE(); + return Status::OK(); + } + virtual Status MergeCF(uint32_t column_family_id, const Slice& key, + const Slice& value) override { + ADD_FAILURE(); + return Status::OK(); + } + virtual void LogData(const Slice& blob) override { ADD_FAILURE(); } + virtual bool Continue() override { return num_seen < kNumUpdates; } + } handler; + + batch.Iterate(&handler); + ASSERT_EQ(kNumUpdates, handler.num_seen); +} + +// The test requires more than 18GB memory to run it, with single memory +// allocation of more than 12GB. Not all the platform can run it. So disable it. +TEST_F(WriteBatchTest, DISABLED_LargeKeyValue) { + // Insert key and value of 3GB and push total batch size to 12GB. + static const size_t kKeyValueSize = 3221225472u; + std::string raw(kKeyValueSize, 'A'); + WriteBatch batch(size_t(12884901888ull + 1024u)); + for (char i = 0; i < 2; i++) { + raw[0] = 'A' + i; + raw[raw.length() - 1] = 'A' - i; + batch.Put(raw, raw); + } + + ASSERT_EQ(2, batch.Count()); + + struct NoopHandler : public WriteBatch::Handler { + int num_seen = 0; + virtual Status PutCF(uint32_t column_family_id, const Slice& key, + const Slice& value) override { + EXPECT_EQ(kKeyValueSize, key.size()); + EXPECT_EQ(kKeyValueSize, value.size()); + EXPECT_EQ('A' + num_seen, key[0]); + EXPECT_EQ('A' + num_seen, value[0]); + EXPECT_EQ('A' - num_seen, key[kKeyValueSize - 1]); + EXPECT_EQ('A' - num_seen, value[kKeyValueSize - 1]); + ++num_seen; + return Status::OK(); + } + virtual Status DeleteCF(uint32_t column_family_id, + const Slice& key) override { + ADD_FAILURE(); + return Status::OK(); + } + virtual Status SingleDeleteCF(uint32_t column_family_id, + const Slice& key) override { + ADD_FAILURE(); + return Status::OK(); + } + virtual Status MergeCF(uint32_t column_family_id, const Slice& key, + const Slice& value) override { + ADD_FAILURE(); + return Status::OK(); + } + virtual void LogData(const Slice& blob) override { ADD_FAILURE(); } + virtual bool Continue() override { return num_seen < 2; } + } handler; + + batch.Iterate(&handler); + ASSERT_EQ(2, handler.num_seen); +} + +TEST_F(WriteBatchTest, Continue) { WriteBatch batch; struct Handler : public TestHandler { int num_seen = 0; virtual Status PutCF(uint32_t column_family_id, const Slice& key, - const Slice& value) { + const Slice& value) override { ++num_seen; return TestHandler::PutCF(column_family_id, key, value); } - virtual Status MergeCF(uint32_t column_family_id, const Slice& key, - const Slice& value) { + virtual Status DeleteCF(uint32_t column_family_id, + const Slice& key) override { ++num_seen; - return TestHandler::MergeCF(column_family_id, key, value); + return TestHandler::DeleteCF(column_family_id, key); } - virtual void LogData(const Slice& blob) { + virtual Status SingleDeleteCF(uint32_t column_family_id, + const Slice& key) override { ++num_seen; - TestHandler::LogData(blob); + return TestHandler::SingleDeleteCF(column_family_id, key); } - virtual Status DeleteCF(uint32_t column_family_id, const Slice& key) { + virtual Status MergeCF(uint32_t column_family_id, const Slice& key, + const Slice& value) override { ++num_seen; - return TestHandler::DeleteCF(column_family_id, key); + return TestHandler::MergeCF(column_family_id, key, value); } - virtual bool Continue() override { - return num_seen < 3; + virtual void LogData(const Slice& blob) override { + ++num_seen; + TestHandler::LogData(blob); } + virtual bool Continue() override { return num_seen < 5; } } handler; batch.Put(Slice("k1"), Slice("v1")); + batch.Put(Slice("k2"), Slice("v2")); batch.PutLogData(Slice("blob1")); batch.Delete(Slice("k1")); + batch.SingleDelete(Slice("k2")); batch.PutLogData(Slice("blob2")); batch.Merge(Slice("foo"), Slice("bar")); batch.Iterate(&handler); ASSERT_EQ( - "Put(k1, v1)" - "LogData(blob1)" - "Delete(k1)", - handler.seen); + "Put(k1, v1)" + "Put(k2, v2)" + "LogData(blob1)" + "Delete(k1)" + "SingleDelete(k2)", + handler.seen); } -TEST(WriteBatchTest, PutGatherSlices) { +TEST_F(WriteBatchTest, PutGatherSlices) { WriteBatch batch; batch.Put(Slice("foo"), Slice("bar")); @@ -287,19 +606,24 @@ class ColumnFamilyHandleImplDummy : public ColumnFamilyHandleImpl { explicit ColumnFamilyHandleImplDummy(int id) : ColumnFamilyHandleImpl(nullptr, nullptr, nullptr), id_(id) {} uint32_t GetID() const override { return id_; } + const Comparator* GetComparator() const override { + return BytewiseComparator(); + } private: uint32_t id_; }; } // namespace anonymous -TEST(WriteBatchTest, ColumnFamiliesBatchTest) { +TEST_F(WriteBatchTest, ColumnFamiliesBatchTest) { WriteBatch batch; ColumnFamilyHandleImplDummy zero(0), two(2), three(3), eight(8); batch.Put(&zero, Slice("foo"), Slice("bar")); batch.Put(&two, Slice("twofoo"), Slice("bar2")); batch.Put(&eight, Slice("eightfoo"), Slice("bar8")); batch.Delete(&eight, Slice("eightfoo")); + batch.SingleDelete(&two, Slice("twofoo")); + batch.DeleteRange(&two, Slice("3foo"), Slice("4foo")); batch.Merge(&three, Slice("threethree"), Slice("3three")); batch.Put(&zero, Slice("foo"), Slice("bar")); batch.Merge(Slice("omom"), Slice("nom")); @@ -311,19 +635,24 @@ TEST(WriteBatchTest, ColumnFamiliesBatchTest) { "PutCF(2, twofoo, bar2)" "PutCF(8, eightfoo, bar8)" "DeleteCF(8, eightfoo)" + "SingleDeleteCF(2, twofoo)" + "DeleteRangeCF(2, 3foo, 4foo)" "MergeCF(3, threethree, 3three)" "Put(foo, bar)" "Merge(omom, nom)", handler.seen); } -TEST(WriteBatchTest, ColumnFamiliesBatchWithIndexTest) { - WriteBatchWithIndex batch(BytewiseComparator(), 20); +#ifndef ROCKSDB_LITE +TEST_F(WriteBatchTest, ColumnFamiliesBatchWithIndexTest) { + WriteBatchWithIndex batch; ColumnFamilyHandleImplDummy zero(0), two(2), three(3), eight(8); batch.Put(&zero, Slice("foo"), Slice("bar")); batch.Put(&two, Slice("twofoo"), Slice("bar2")); batch.Put(&eight, Slice("eightfoo"), Slice("bar8")); batch.Delete(&eight, Slice("eightfoo")); + batch.SingleDelete(&two, Slice("twofoo")); + batch.DeleteRange(&two, Slice("twofoo"), Slice("threefoo")); batch.Merge(&three, Slice("threethree"), Slice("3three")); batch.Put(&zero, Slice("foo"), Slice("bar")); batch.Merge(Slice("omom"), Slice("nom")); @@ -348,6 +677,31 @@ TEST(WriteBatchTest, ColumnFamiliesBatchWithIndexTest) { ASSERT_OK(iter->status()); ASSERT_TRUE(!iter->Valid()); + iter.reset(batch.NewIterator(&two)); + iter->Seek("twofoo"); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(WriteType::kPutRecord, iter->Entry().type); + ASSERT_EQ("twofoo", iter->Entry().key.ToString()); + ASSERT_EQ("bar2", iter->Entry().value.ToString()); + + iter->Next(); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(WriteType::kSingleDeleteRecord, iter->Entry().type); + ASSERT_EQ("twofoo", iter->Entry().key.ToString()); + + iter->Next(); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ(WriteType::kDeleteRangeRecord, iter->Entry().type); + ASSERT_EQ("twofoo", iter->Entry().key.ToString()); + ASSERT_EQ("threefoo", iter->Entry().value.ToString()); + + iter->Next(); + ASSERT_OK(iter->status()); + ASSERT_TRUE(!iter->Valid()); + iter.reset(batch.NewIterator()); iter->Seek("gggg"); ASSERT_OK(iter->status()); @@ -393,14 +747,148 @@ TEST(WriteBatchTest, ColumnFamiliesBatchWithIndexTest) { "PutCF(2, twofoo, bar2)" "PutCF(8, eightfoo, bar8)" "DeleteCF(8, eightfoo)" + "SingleDeleteCF(2, twofoo)" + "DeleteRangeCF(2, twofoo, threefoo)" "MergeCF(3, threethree, 3three)" "Put(foo, bar)" "Merge(omom, nom)", handler.seen); } +#endif // !ROCKSDB_LITE + +TEST_F(WriteBatchTest, SavePointTest) { + Status s; + WriteBatch batch; + batch.SetSavePoint(); + + batch.Put("A", "a"); + batch.Put("B", "b"); + batch.SetSavePoint(); + + batch.Put("C", "c"); + batch.Delete("A"); + batch.SetSavePoint(); + batch.SetSavePoint(); + + ASSERT_OK(batch.RollbackToSavePoint()); + ASSERT_EQ( + "Delete(A)@3" + "Put(A, a)@0" + "Put(B, b)@1" + "Put(C, c)@2", + PrintContents(&batch)); + + ASSERT_OK(batch.RollbackToSavePoint()); + ASSERT_OK(batch.RollbackToSavePoint()); + ASSERT_EQ( + "Put(A, a)@0" + "Put(B, b)@1", + PrintContents(&batch)); + + batch.Delete("A"); + batch.Put("B", "bb"); + + ASSERT_OK(batch.RollbackToSavePoint()); + ASSERT_EQ("", PrintContents(&batch)); + + s = batch.RollbackToSavePoint(); + ASSERT_TRUE(s.IsNotFound()); + ASSERT_EQ("", PrintContents(&batch)); + + batch.Put("D", "d"); + batch.Delete("A"); + + batch.SetSavePoint(); + + batch.Put("A", "aaa"); + + ASSERT_OK(batch.RollbackToSavePoint()); + ASSERT_EQ( + "Delete(A)@1" + "Put(D, d)@0", + PrintContents(&batch)); + + batch.SetSavePoint(); + + batch.Put("D", "d"); + batch.Delete("A"); + + ASSERT_OK(batch.RollbackToSavePoint()); + ASSERT_EQ( + "Delete(A)@1" + "Put(D, d)@0", + PrintContents(&batch)); + + s = batch.RollbackToSavePoint(); + ASSERT_TRUE(s.IsNotFound()); + ASSERT_EQ( + "Delete(A)@1" + "Put(D, d)@0", + PrintContents(&batch)); + + WriteBatch batch2; + + s = batch2.RollbackToSavePoint(); + ASSERT_TRUE(s.IsNotFound()); + ASSERT_EQ("", PrintContents(&batch2)); + + batch2.Delete("A"); + batch2.SetSavePoint(); + + s = batch2.RollbackToSavePoint(); + ASSERT_OK(s); + ASSERT_EQ("Delete(A)@0", PrintContents(&batch2)); + + batch2.Clear(); + ASSERT_EQ("", PrintContents(&batch2)); + + batch2.SetSavePoint(); + + batch2.Delete("B"); + ASSERT_EQ("Delete(B)@0", PrintContents(&batch2)); + + batch2.SetSavePoint(); + s = batch2.RollbackToSavePoint(); + ASSERT_OK(s); + ASSERT_EQ("Delete(B)@0", PrintContents(&batch2)); + + s = batch2.RollbackToSavePoint(); + ASSERT_OK(s); + ASSERT_EQ("", PrintContents(&batch2)); + + s = batch2.RollbackToSavePoint(); + ASSERT_TRUE(s.IsNotFound()); + ASSERT_EQ("", PrintContents(&batch2)); + + WriteBatch batch3; + + s = batch3.PopSavePoint(); + ASSERT_TRUE(s.IsNotFound()); + ASSERT_EQ("", PrintContents(&batch3)); + + batch3.SetSavePoint(); + batch3.Delete("A"); + + s = batch3.PopSavePoint(); + ASSERT_OK(s); + ASSERT_EQ("Delete(A)@0", PrintContents(&batch3)); +} + +TEST_F(WriteBatchTest, MemoryLimitTest) { + Status s; + // The header size is 12 bytes. The two Puts take 8 bytes which gives total + // of 12 + 8 * 2 = 28 bytes. + WriteBatch batch(0, 28); + + ASSERT_OK(batch.Put("a", "....")); + ASSERT_OK(batch.Put("b", "....")); + s = batch.Put("c", "...."); + ASSERT_TRUE(s.IsMemoryLimit()); +} } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/db/write_callback.h b/db/write_callback.h new file mode 100644 index 00000000000..6517a7c3aa1 --- /dev/null +++ b/db/write_callback.h @@ -0,0 +1,27 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include "rocksdb/status.h" + +namespace rocksdb { + +class DB; + +class WriteCallback { + public: + virtual ~WriteCallback() {} + + // Will be called while on the write thread before the write executes. If + // this function returns a non-OK status, the write will be aborted and this + // status will be returned to the caller of DB::Write(). + virtual Status Callback(DB* db) = 0; + + // return true if writes with this callback can be batched with other writes + virtual bool AllowWriteBatching() = 0; +}; + +} // namespace rocksdb diff --git a/db/write_callback_test.cc b/db/write_callback_test.cc new file mode 100644 index 00000000000..d2bf30a0930 --- /dev/null +++ b/db/write_callback_test.cc @@ -0,0 +1,397 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include +#include +#include +#include +#include + +#include "db/db_impl.h" +#include "db/write_callback.h" +#include "rocksdb/db.h" +#include "rocksdb/write_batch.h" +#include "port/port.h" +#include "util/random.h" +#include "util/sync_point.h" +#include "util/testharness.h" + +using std::string; + +namespace rocksdb { + +class WriteCallbackTest : public testing::Test { + public: + string dbname; + + WriteCallbackTest() { + dbname = test::TmpDir() + "/write_callback_testdb"; + } +}; + +class WriteCallbackTestWriteCallback1 : public WriteCallback { + public: + bool was_called = false; + + Status Callback(DB *db) override { + was_called = true; + + // Make sure db is a DBImpl + DBImpl* db_impl = dynamic_cast (db); + if (db_impl == nullptr) { + return Status::InvalidArgument(""); + } + + return Status::OK(); + } + + bool AllowWriteBatching() override { return true; } +}; + +class WriteCallbackTestWriteCallback2 : public WriteCallback { + public: + Status Callback(DB *db) override { + return Status::Busy(); + } + bool AllowWriteBatching() override { return true; } +}; + +class MockWriteCallback : public WriteCallback { + public: + bool should_fail_ = false; + bool allow_batching_ = false; + std::atomic was_called_{false}; + + MockWriteCallback() {} + + MockWriteCallback(const MockWriteCallback& other) { + should_fail_ = other.should_fail_; + allow_batching_ = other.allow_batching_; + was_called_.store(other.was_called_.load()); + } + + Status Callback(DB* db) override { + was_called_.store(true); + if (should_fail_) { + return Status::Busy(); + } else { + return Status::OK(); + } + } + + bool AllowWriteBatching() override { return allow_batching_; } +}; + +TEST_F(WriteCallbackTest, WriteWithCallbackTest) { + struct WriteOP { + WriteOP(bool should_fail = false) { callback_.should_fail_ = should_fail; } + + void Put(const string& key, const string& val) { + kvs_.push_back(std::make_pair(key, val)); + write_batch_.Put(key, val); + } + + void Clear() { + kvs_.clear(); + write_batch_.Clear(); + callback_.was_called_.store(false); + } + + MockWriteCallback callback_; + WriteBatch write_batch_; + std::vector> kvs_; + }; + + // In each scenario we'll launch multiple threads to write. + // The size of each array equals to number of threads, and + // each boolean in it denote whether callback of corresponding + // thread should succeed or fail. + std::vector> write_scenarios = { + {true}, + {false}, + {false, false}, + {true, true}, + {true, false}, + {false, true}, + {false, false, false}, + {true, true, true}, + {false, true, false}, + {true, false, true}, + {true, false, false, false, false}, + {false, false, false, false, true}, + {false, false, true, false, true}, + }; + + for (auto& two_queues : {true, false}) { + for (auto& allow_parallel : {true, false}) { + for (auto& allow_batching : {true, false}) { + for (auto& enable_WAL : {true, false}) { + for (auto& enable_pipelined_write : {true, false}) { + for (auto& write_group : write_scenarios) { + Options options; + options.create_if_missing = true; + options.allow_concurrent_memtable_write = allow_parallel; + options.enable_pipelined_write = enable_pipelined_write; + options.concurrent_prepare = two_queues; + + ReadOptions read_options; + DB* db; + DBImpl* db_impl; + + DestroyDB(dbname, options); + ASSERT_OK(DB::Open(options, dbname, &db)); + + db_impl = dynamic_cast(db); + ASSERT_TRUE(db_impl); + + // Writers that have called JoinBatchGroup. + std::atomic threads_joining(0); + // Writers that have linked to the queue + std::atomic threads_linked(0); + // Writers that pass WriteThread::JoinBatchGroup:Wait sync-point. + std::atomic threads_verified(0); + + std::atomic seq(db_impl->GetLatestSequenceNumber()); + ASSERT_EQ(db_impl->GetLatestSequenceNumber(), 0); + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "WriteThread::JoinBatchGroup:Start", [&](void*) { + uint64_t cur_threads_joining = threads_joining.fetch_add(1); + // Wait for the last joined writer to link to the queue. + // In this way the writers link to the queue one by one. + // This allows us to confidently detect the first writer + // who increases threads_linked as the leader. + while (threads_linked.load() < cur_threads_joining) { + } + }); + + // Verification once writers call JoinBatchGroup. + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "WriteThread::JoinBatchGroup:Wait", [&](void* arg) { + uint64_t cur_threads_linked = threads_linked.fetch_add(1); + bool is_leader = false; + bool is_last = false; + + // who am i + is_leader = (cur_threads_linked == 0); + is_last = (cur_threads_linked == write_group.size() - 1); + + // check my state + auto* writer = reinterpret_cast(arg); + + if (is_leader) { + ASSERT_TRUE(writer->state == + WriteThread::State::STATE_GROUP_LEADER); + } else { + ASSERT_TRUE(writer->state == + WriteThread::State::STATE_INIT); + } + + // (meta test) the first WriteOP should indeed be the first + // and the last should be the last (all others can be out of + // order) + if (is_leader) { + ASSERT_TRUE(writer->callback->Callback(nullptr).ok() == + !write_group.front().callback_.should_fail_); + } else if (is_last) { + ASSERT_TRUE(writer->callback->Callback(nullptr).ok() == + !write_group.back().callback_.should_fail_); + } + + threads_verified.fetch_add(1); + // Wait here until all verification in this sync-point + // callback finish for all writers. + while (threads_verified.load() < write_group.size()) { + } + }); + + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "WriteThread::JoinBatchGroup:DoneWaiting", [&](void* arg) { + // check my state + auto* writer = reinterpret_cast(arg); + + if (!allow_batching) { + // no batching so everyone should be a leader + ASSERT_TRUE(writer->state == + WriteThread::State::STATE_GROUP_LEADER); + } else if (!allow_parallel) { + ASSERT_TRUE(writer->state == + WriteThread::State::STATE_COMPLETED || + (enable_pipelined_write && + writer->state == + WriteThread::State:: + STATE_MEMTABLE_WRITER_LEADER)); + } + }); + + std::atomic thread_num(0); + std::atomic dummy_key(0); + + // Each write thread create a random write batch and write to DB + // with a write callback. + std::function write_with_callback_func = [&]() { + uint32_t i = thread_num.fetch_add(1); + Random rnd(i); + + // leaders gotta lead + while (i > 0 && threads_verified.load() < 1) { + } + + // loser has to lose + while (i == write_group.size() - 1 && + threads_verified.load() < write_group.size() - 1) { + } + + auto& write_op = write_group.at(i); + write_op.Clear(); + write_op.callback_.allow_batching_ = allow_batching; + + // insert some keys + for (uint32_t j = 0; j < rnd.Next() % 50; j++) { + // grab unique key + char my_key = dummy_key.fetch_add(1); + + string skey(5, my_key); + string sval(10, my_key); + write_op.Put(skey, sval); + + if (!write_op.callback_.should_fail_) { + seq.fetch_add(1); + } + } + + WriteOptions woptions; + woptions.disableWAL = !enable_WAL; + woptions.sync = enable_WAL; + Status s = db_impl->WriteWithCallback( + woptions, &write_op.write_batch_, &write_op.callback_); + + if (write_op.callback_.should_fail_) { + ASSERT_TRUE(s.IsBusy()); + } else { + ASSERT_OK(s); + } + }; + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + + // do all the writes + std::vector threads; + for (uint32_t i = 0; i < write_group.size(); i++) { + threads.emplace_back(write_with_callback_func); + } + for (auto& t : threads) { + t.join(); + } + + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + + // check for keys + string value; + for (auto& w : write_group) { + ASSERT_TRUE(w.callback_.was_called_.load()); + for (auto& kvp : w.kvs_) { + if (w.callback_.should_fail_) { + ASSERT_TRUE( + db->Get(read_options, kvp.first, &value).IsNotFound()); + } else { + ASSERT_OK(db->Get(read_options, kvp.first, &value)); + ASSERT_EQ(value, kvp.second); + } + } + } + + ASSERT_EQ(seq.load(), db_impl->GetLatestSequenceNumber()); + + delete db; + DestroyDB(dbname, options); + } + } + } + } + } +} +} + +TEST_F(WriteCallbackTest, WriteCallBackTest) { + Options options; + WriteOptions write_options; + ReadOptions read_options; + string value; + DB* db; + DBImpl* db_impl; + + DestroyDB(dbname, options); + + options.create_if_missing = true; + Status s = DB::Open(options, dbname, &db); + ASSERT_OK(s); + + db_impl = dynamic_cast (db); + ASSERT_TRUE(db_impl); + + WriteBatch wb; + + wb.Put("a", "value.a"); + wb.Delete("x"); + + // Test a simple Write + s = db->Write(write_options, &wb); + ASSERT_OK(s); + + s = db->Get(read_options, "a", &value); + ASSERT_OK(s); + ASSERT_EQ("value.a", value); + + // Test WriteWithCallback + WriteCallbackTestWriteCallback1 callback1; + WriteBatch wb2; + + wb2.Put("a", "value.a2"); + + s = db_impl->WriteWithCallback(write_options, &wb2, &callback1); + ASSERT_OK(s); + ASSERT_TRUE(callback1.was_called); + + s = db->Get(read_options, "a", &value); + ASSERT_OK(s); + ASSERT_EQ("value.a2", value); + + // Test WriteWithCallback for a callback that fails + WriteCallbackTestWriteCallback2 callback2; + WriteBatch wb3; + + wb3.Put("a", "value.a3"); + + s = db_impl->WriteWithCallback(write_options, &wb3, &callback2); + ASSERT_NOK(s); + + s = db->Get(read_options, "a", &value); + ASSERT_OK(s); + ASSERT_EQ("value.a2", value); + + delete db; + DestroyDB(dbname, options); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, + "SKIPPED as WriteWithCallback is not supported in ROCKSDB_LITE\n"); + return 0; +} + +#endif // !ROCKSDB_LITE diff --git a/db/write_controller.cc b/db/write_controller.cc new file mode 100644 index 00000000000..558aa721923 --- /dev/null +++ b/db/write_controller.cc @@ -0,0 +1,128 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/write_controller.h" + +#include +#include +#include +#include "rocksdb/env.h" + +namespace rocksdb { + +std::unique_ptr WriteController::GetStopToken() { + ++total_stopped_; + return std::unique_ptr(new StopWriteToken(this)); +} + +std::unique_ptr WriteController::GetDelayToken( + uint64_t write_rate) { + total_delayed_++; + // Reset counters. + last_refill_time_ = 0; + bytes_left_ = 0; + set_delayed_write_rate(write_rate); + return std::unique_ptr(new DelayWriteToken(this)); +} + +std::unique_ptr +WriteController::GetCompactionPressureToken() { + ++total_compaction_pressure_; + return std::unique_ptr( + new CompactionPressureToken(this)); +} + +bool WriteController::IsStopped() const { + return total_stopped_.load(std::memory_order_relaxed) > 0; +} +// This is inside DB mutex, so we can't sleep and need to minimize +// frequency to get time. +// If it turns out to be a performance issue, we can redesign the thread +// synchronization model here. +// The function trust caller will sleep micros returned. +uint64_t WriteController::GetDelay(Env* env, uint64_t num_bytes) { + if (total_stopped_.load(std::memory_order_relaxed) > 0) { + return 0; + } + if (total_delayed_.load(std::memory_order_relaxed) == 0) { + return 0; + } + + const uint64_t kMicrosPerSecond = 1000000; + const uint64_t kRefillInterval = 1024U; + + if (bytes_left_ >= num_bytes) { + bytes_left_ -= num_bytes; + return 0; + } + // The frequency to get time inside DB mutex is less than one per refill + // interval. + auto time_now = NowMicrosMonotonic(env); + + uint64_t sleep_debt = 0; + uint64_t time_since_last_refill = 0; + if (last_refill_time_ != 0) { + if (last_refill_time_ > time_now) { + sleep_debt = last_refill_time_ - time_now; + } else { + time_since_last_refill = time_now - last_refill_time_; + bytes_left_ += + static_cast(static_cast(time_since_last_refill) / + kMicrosPerSecond * delayed_write_rate_); + if (time_since_last_refill >= kRefillInterval && + bytes_left_ > num_bytes) { + // If refill interval already passed and we have enough bytes + // return without extra sleeping. + last_refill_time_ = time_now; + bytes_left_ -= num_bytes; + return 0; + } + } + } + + uint64_t single_refill_amount = + delayed_write_rate_ * kRefillInterval / kMicrosPerSecond; + if (bytes_left_ + single_refill_amount >= num_bytes) { + // Wait until a refill interval + // Never trigger expire for less than one refill interval to avoid to get + // time. + bytes_left_ = bytes_left_ + single_refill_amount - num_bytes; + last_refill_time_ = time_now + kRefillInterval; + return kRefillInterval + sleep_debt; + } + + // Need to refill more than one interval. Need to sleep longer. Check + // whether expiration will hit + + // Sleep just until `num_bytes` is allowed. + uint64_t sleep_amount = + static_cast(num_bytes / + static_cast(delayed_write_rate_) * + kMicrosPerSecond) + + sleep_debt; + last_refill_time_ = time_now + sleep_amount; + return sleep_amount; +} + +uint64_t WriteController::NowMicrosMonotonic(Env* env) { + return env->NowNanos() / std::milli::den; +} + +StopWriteToken::~StopWriteToken() { + assert(controller_->total_stopped_ >= 1); + --controller_->total_stopped_; +} + +DelayWriteToken::~DelayWriteToken() { + controller_->total_delayed_--; + assert(controller_->total_delayed_.load() >= 0); +} + +CompactionPressureToken::~CompactionPressureToken() { + controller_->total_compaction_pressure_--; + assert(controller_->total_compaction_pressure_ >= 0); +} + +} // namespace rocksdb diff --git a/db/write_controller.h b/db/write_controller.h new file mode 100644 index 00000000000..7c301ce7d27 --- /dev/null +++ b/db/write_controller.h @@ -0,0 +1,144 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include + +#include +#include +#include "rocksdb/rate_limiter.h" + +namespace rocksdb { + +class Env; +class WriteControllerToken; + +// WriteController is controlling write stalls in our write code-path. Write +// stalls happen when compaction can't keep up with write rate. +// All of the methods here (including WriteControllerToken's destructors) need +// to be called while holding DB mutex +class WriteController { + public: + explicit WriteController(uint64_t _delayed_write_rate = 1024u * 1024u * 32u, + int64_t low_pri_rate_bytes_per_sec = 1024 * 1024) + : total_stopped_(0), + total_delayed_(0), + total_compaction_pressure_(0), + bytes_left_(0), + last_refill_time_(0), + low_pri_rate_limiter_( + NewGenericRateLimiter(low_pri_rate_bytes_per_sec)) { + set_max_delayed_write_rate(_delayed_write_rate); + } + ~WriteController() = default; + + // When an actor (column family) requests a stop token, all writes will be + // stopped until the stop token is released (deleted) + std::unique_ptr GetStopToken(); + // When an actor (column family) requests a delay token, total delay for all + // writes to the DB will be controlled under the delayed write rate. Every + // write needs to call GetDelay() with number of bytes writing to the DB, + // which returns number of microseconds to sleep. + std::unique_ptr GetDelayToken( + uint64_t delayed_write_rate); + // When an actor (column family) requests a moderate token, compaction + // threads will be increased + std::unique_ptr GetCompactionPressureToken(); + + // these three metods are querying the state of the WriteController + bool IsStopped() const; + bool NeedsDelay() const { return total_delayed_.load() > 0; } + bool NeedSpeedupCompaction() const { + return IsStopped() || NeedsDelay() || total_compaction_pressure_ > 0; + } + // return how many microseconds the caller needs to sleep after the call + // num_bytes: how many number of bytes to put into the DB. + // Prerequisite: DB mutex held. + uint64_t GetDelay(Env* env, uint64_t num_bytes); + void set_delayed_write_rate(uint64_t write_rate) { + // avoid divide 0 + if (write_rate == 0) { + write_rate = 1u; + } else if (write_rate > max_delayed_write_rate()) { + write_rate = max_delayed_write_rate(); + } + delayed_write_rate_ = write_rate; + } + + void set_max_delayed_write_rate(uint64_t write_rate) { + // avoid divide 0 + if (write_rate == 0) { + write_rate = 1u; + } + max_delayed_write_rate_ = write_rate; + // update delayed_write_rate_ as well + delayed_write_rate_ = write_rate; + } + + uint64_t delayed_write_rate() const { return delayed_write_rate_; } + + uint64_t max_delayed_write_rate() const { return max_delayed_write_rate_; } + + RateLimiter* low_pri_rate_limiter() { return low_pri_rate_limiter_.get(); } + + private: + uint64_t NowMicrosMonotonic(Env* env); + + friend class WriteControllerToken; + friend class StopWriteToken; + friend class DelayWriteToken; + friend class CompactionPressureToken; + + std::atomic total_stopped_; + std::atomic total_delayed_; + std::atomic total_compaction_pressure_; + uint64_t bytes_left_; + uint64_t last_refill_time_; + // write rate set when initialization or by `DBImpl::SetDBOptions` + uint64_t max_delayed_write_rate_; + // current write rate + uint64_t delayed_write_rate_; + + std::unique_ptr low_pri_rate_limiter_; +}; + +class WriteControllerToken { + public: + explicit WriteControllerToken(WriteController* controller) + : controller_(controller) {} + virtual ~WriteControllerToken() {} + + protected: + WriteController* controller_; + + private: + // no copying allowed + WriteControllerToken(const WriteControllerToken&) = delete; + void operator=(const WriteControllerToken&) = delete; +}; + +class StopWriteToken : public WriteControllerToken { + public: + explicit StopWriteToken(WriteController* controller) + : WriteControllerToken(controller) {} + virtual ~StopWriteToken(); +}; + +class DelayWriteToken : public WriteControllerToken { + public: + explicit DelayWriteToken(WriteController* controller) + : WriteControllerToken(controller) {} + virtual ~DelayWriteToken(); +}; + +class CompactionPressureToken : public WriteControllerToken { + public: + explicit CompactionPressureToken(WriteController* controller) + : WriteControllerToken(controller) {} + virtual ~CompactionPressureToken(); +}; + +} // namespace rocksdb diff --git a/db/write_controller_test.cc b/db/write_controller_test.cc new file mode 100644 index 00000000000..a1fe3fa27ea --- /dev/null +++ b/db/write_controller_test.cc @@ -0,0 +1,135 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#include + +#include "db/write_controller.h" + +#include "rocksdb/env.h" +#include "util/testharness.h" + +namespace rocksdb { + +class WriteControllerTest : public testing::Test {}; + +class TimeSetEnv : public EnvWrapper { + public: + explicit TimeSetEnv() : EnvWrapper(nullptr) {} + uint64_t now_micros_ = 6666; + virtual uint64_t NowNanos() override { return now_micros_ * std::milli::den; } +}; + +TEST_F(WriteControllerTest, ChangeDelayRateTest) { + TimeSetEnv env; + WriteController controller(40000000u); // also set max delayed rate + controller.set_delayed_write_rate(10000000u); + auto delay_token_0 = + controller.GetDelayToken(controller.delayed_write_rate()); + ASSERT_EQ(static_cast(2000000), + controller.GetDelay(&env, 20000000u)); + auto delay_token_1 = controller.GetDelayToken(2000000u); + ASSERT_EQ(static_cast(10000000), + controller.GetDelay(&env, 20000000u)); + auto delay_token_2 = controller.GetDelayToken(1000000u); + ASSERT_EQ(static_cast(20000000), + controller.GetDelay(&env, 20000000u)); + auto delay_token_3 = controller.GetDelayToken(20000000u); + ASSERT_EQ(static_cast(1000000), + controller.GetDelay(&env, 20000000u)); + // This is more than max rate. Max delayed rate will be used. + auto delay_token_4 = + controller.GetDelayToken(controller.delayed_write_rate() * 3); + ASSERT_EQ(static_cast(500000), + controller.GetDelay(&env, 20000000u)); +} + +TEST_F(WriteControllerTest, SanityTest) { + WriteController controller(10000000u); + auto stop_token_1 = controller.GetStopToken(); + auto stop_token_2 = controller.GetStopToken(); + + ASSERT_TRUE(controller.IsStopped()); + stop_token_1.reset(); + ASSERT_TRUE(controller.IsStopped()); + stop_token_2.reset(); + ASSERT_FALSE(controller.IsStopped()); + + TimeSetEnv env; + + auto delay_token_1 = controller.GetDelayToken(10000000u); + ASSERT_EQ(static_cast(2000000), + controller.GetDelay(&env, 20000000u)); + + env.now_micros_ += 1999900u; // sleep debt 1000 + + auto delay_token_2 = controller.GetDelayToken(10000000u); + // Rate reset after changing the token. + ASSERT_EQ(static_cast(2000000), + controller.GetDelay(&env, 20000000u)); + + env.now_micros_ += 1999900u; // sleep debt 1000 + + // One refill: 10240 bytes allowed, 1000 used, 9240 left + ASSERT_EQ(static_cast(1124), controller.GetDelay(&env, 1000u)); + env.now_micros_ += 1124u; // sleep debt 0 + + delay_token_2.reset(); + // 1000 used, 8240 left + ASSERT_EQ(static_cast(0), controller.GetDelay(&env, 1000u)); + + env.now_micros_ += 100u; // sleep credit 100 + // 1000 used, 7240 left + ASSERT_EQ(static_cast(0), controller.GetDelay(&env, 1000u)); + + env.now_micros_ += 100u; // sleep credit 200 + // One refill: 10240 fileed, sleep credit generates 2000. 8000 used + // 7240 + 10240 + 2000 - 8000 = 11480 left + ASSERT_EQ(static_cast(1024u), controller.GetDelay(&env, 8000u)); + + env.now_micros_ += 200u; // sleep debt 824 + // 1000 used, 10480 left. + ASSERT_EQ(static_cast(0), controller.GetDelay(&env, 1000u)); + + env.now_micros_ += 200u; // sleep debt 624 + // Out of bound sleep, still 10480 left + ASSERT_EQ(static_cast(3000624u), + controller.GetDelay(&env, 30000000u)); + + env.now_micros_ += 3000724u; // sleep credit 100 + // 6000 used, 4480 left. + ASSERT_EQ(static_cast(0), controller.GetDelay(&env, 6000u)); + + env.now_micros_ += 200u; // sleep credit 300 + // One refill, credit 4480 balance + 3000 credit + 10240 refill + // Use 8000, 9720 left + ASSERT_EQ(static_cast(1024u), controller.GetDelay(&env, 8000u)); + + env.now_micros_ += 3024u; // sleep credit 2000 + + // 1720 left + ASSERT_EQ(static_cast(0u), controller.GetDelay(&env, 8000u)); + + // 1720 balance + 20000 credit = 20170 left + // Use 8000, 12170 left + ASSERT_EQ(static_cast(0u), controller.GetDelay(&env, 8000u)); + + // 4170 left + ASSERT_EQ(static_cast(0u), controller.GetDelay(&env, 8000u)); + + // Need a refill + ASSERT_EQ(static_cast(1024u), controller.GetDelay(&env, 9000u)); + + delay_token_1.reset(); + ASSERT_EQ(static_cast(0), controller.GetDelay(&env, 30000000u)); + delay_token_1.reset(); + ASSERT_FALSE(controller.IsStopped()); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/db/write_thread.cc b/db/write_thread.cc new file mode 100644 index 00000000000..afe2f27978e --- /dev/null +++ b/db/write_thread.cc @@ -0,0 +1,661 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "db/write_thread.h" +#include +#include +#include "db/column_family.h" +#include "port/port.h" +#include "util/random.h" +#include "util/sync_point.h" + +namespace rocksdb { + +WriteThread::WriteThread(const ImmutableDBOptions& db_options) + : max_yield_usec_(db_options.enable_write_thread_adaptive_yield + ? db_options.write_thread_max_yield_usec + : 0), + slow_yield_usec_(db_options.write_thread_slow_yield_usec), + allow_concurrent_memtable_write_( + db_options.allow_concurrent_memtable_write), + enable_pipelined_write_(db_options.enable_pipelined_write), + newest_writer_(nullptr), + newest_memtable_writer_(nullptr), + last_sequence_(0) {} + +uint8_t WriteThread::BlockingAwaitState(Writer* w, uint8_t goal_mask) { + // We're going to block. Lazily create the mutex. We guarantee + // propagation of this construction to the waker via the + // STATE_LOCKED_WAITING state. The waker won't try to touch the mutex + // or the condvar unless they CAS away the STATE_LOCKED_WAITING that + // we install below. + w->CreateMutex(); + + auto state = w->state.load(std::memory_order_acquire); + assert(state != STATE_LOCKED_WAITING); + if ((state & goal_mask) == 0 && + w->state.compare_exchange_strong(state, STATE_LOCKED_WAITING)) { + // we have permission (and an obligation) to use StateMutex + std::unique_lock guard(w->StateMutex()); + w->StateCV().wait(guard, [w] { + return w->state.load(std::memory_order_relaxed) != STATE_LOCKED_WAITING; + }); + state = w->state.load(std::memory_order_relaxed); + } + // else tricky. Goal is met or CAS failed. In the latter case the waker + // must have changed the state, and compare_exchange_strong has updated + // our local variable with the new one. At the moment WriteThread never + // waits for a transition across intermediate states, so we know that + // since a state change has occurred the goal must have been met. + assert((state & goal_mask) != 0); + return state; +} + +uint8_t WriteThread::AwaitState(Writer* w, uint8_t goal_mask, + AdaptationContext* ctx) { + uint8_t state; + + // 1. Busy loop using "pause" for 1 micro sec + // 2. Else SOMETIMES busy loop using "yield" for 100 micro sec (default) + // 3. Else blocking wait + + // On a modern Xeon each loop takes about 7 nanoseconds (most of which + // is the effect of the pause instruction), so 200 iterations is a bit + // more than a microsecond. This is long enough that waits longer than + // this can amortize the cost of accessing the clock and yielding. + for (uint32_t tries = 0; tries < 200; ++tries) { + state = w->state.load(std::memory_order_acquire); + if ((state & goal_mask) != 0) { + return state; + } + port::AsmVolatilePause(); + } + + // If we're only going to end up waiting a short period of time, + // it can be a lot more efficient to call std::this_thread::yield() + // in a loop than to block in StateMutex(). For reference, on my 4.0 + // SELinux test server with support for syscall auditing enabled, the + // minimum latency between FUTEX_WAKE to returning from FUTEX_WAIT is + // 2.7 usec, and the average is more like 10 usec. That can be a big + // drag on RockDB's single-writer design. Of course, spinning is a + // bad idea if other threads are waiting to run or if we're going to + // wait for a long time. How do we decide? + // + // We break waiting into 3 categories: short-uncontended, + // short-contended, and long. If we had an oracle, then we would always + // spin for short-uncontended, always block for long, and our choice for + // short-contended might depend on whether we were trying to optimize + // RocksDB throughput or avoid being greedy with system resources. + // + // Bucketing into short or long is easy by measuring elapsed time. + // Differentiating short-uncontended from short-contended is a bit + // trickier, but not too bad. We could look for involuntary context + // switches using getrusage(RUSAGE_THREAD, ..), but it's less work + // (portability code and CPU) to just look for yield calls that take + // longer than we expect. sched_yield() doesn't actually result in any + // context switch overhead if there are no other runnable processes + // on the current core, in which case it usually takes less than + // a microsecond. + // + // There are two primary tunables here: the threshold between "short" + // and "long" waits, and the threshold at which we suspect that a yield + // is slow enough to indicate we should probably block. If these + // thresholds are chosen well then CPU-bound workloads that don't + // have more threads than cores will experience few context switches + // (voluntary or involuntary), and the total number of context switches + // (voluntary and involuntary) will not be dramatically larger (maybe + // 2x) than the number of voluntary context switches that occur when + // --max_yield_wait_micros=0. + // + // There's another constant, which is the number of slow yields we will + // tolerate before reversing our previous decision. Solitary slow + // yields are pretty common (low-priority small jobs ready to run), + // so this should be at least 2. We set this conservatively to 3 so + // that we can also immediately schedule a ctx adaptation, rather than + // waiting for the next update_ctx. + + const size_t kMaxSlowYieldsWhileSpinning = 3; + + // Whether the yield approach has any credit in this context. The credit is + // added by yield being succesfull before timing out, and decreased otherwise. + auto& yield_credit = ctx->value; + // Update the yield_credit based on sample runs or right after a hard failure + bool update_ctx = false; + // Should we reinforce the yield credit + bool would_spin_again = false; + // The samling base for updating the yeild credit. The sampling rate would be + // 1/sampling_base. + const int sampling_base = 256; + + if (max_yield_usec_ > 0) { + update_ctx = Random::GetTLSInstance()->OneIn(sampling_base); + + if (update_ctx || yield_credit.load(std::memory_order_relaxed) >= 0) { + // we're updating the adaptation statistics, or spinning has > + // 50% chance of being shorter than max_yield_usec_ and causing no + // involuntary context switches + auto spin_begin = std::chrono::steady_clock::now(); + + // this variable doesn't include the final yield (if any) that + // causes the goal to be met + size_t slow_yield_count = 0; + + auto iter_begin = spin_begin; + while ((iter_begin - spin_begin) <= + std::chrono::microseconds(max_yield_usec_)) { + std::this_thread::yield(); + + state = w->state.load(std::memory_order_acquire); + if ((state & goal_mask) != 0) { + // success + would_spin_again = true; + break; + } + + auto now = std::chrono::steady_clock::now(); + if (now == iter_begin || + now - iter_begin >= std::chrono::microseconds(slow_yield_usec_)) { + // conservatively count it as a slow yield if our clock isn't + // accurate enough to measure the yield duration + ++slow_yield_count; + if (slow_yield_count >= kMaxSlowYieldsWhileSpinning) { + // Not just one ivcsw, but several. Immediately update yield_credit + // and fall back to blocking + update_ctx = true; + break; + } + } + iter_begin = now; + } + } + } + + if ((state & goal_mask) == 0) { + state = BlockingAwaitState(w, goal_mask); + } + + if (update_ctx) { + // Since our update is sample based, it is ok if a thread overwrites the + // updates by other threads. Thus the update does not have to be atomic. + auto v = yield_credit.load(std::memory_order_relaxed); + // fixed point exponential decay with decay constant 1/1024, with +1 + // and -1 scaled to avoid overflow for int32_t + // + // On each update the positive credit is decayed by a facor of 1/1024 (i.e., + // 0.1%). If the sampled yield was successful, the credit is also increased + // by X. Setting X=2^17 ensures that the credit never exceeds + // 2^17*2^10=2^27, which is lower than 2^31 the upperbound of int32_t. Same + // logic applies to negative credits. + v = v - (v / 1024) + (would_spin_again ? 1 : -1) * 131072; + yield_credit.store(v, std::memory_order_relaxed); + } + + assert((state & goal_mask) != 0); + return state; +} + +void WriteThread::SetState(Writer* w, uint8_t new_state) { + auto state = w->state.load(std::memory_order_acquire); + if (state == STATE_LOCKED_WAITING || + !w->state.compare_exchange_strong(state, new_state)) { + assert(state == STATE_LOCKED_WAITING); + + std::lock_guard guard(w->StateMutex()); + assert(w->state.load(std::memory_order_relaxed) != new_state); + w->state.store(new_state, std::memory_order_relaxed); + w->StateCV().notify_one(); + } +} + +bool WriteThread::LinkOne(Writer* w, std::atomic* newest_writer) { + assert(newest_writer != nullptr); + assert(w->state == STATE_INIT); + Writer* writers = newest_writer->load(std::memory_order_relaxed); + while (true) { + w->link_older = writers; + if (newest_writer->compare_exchange_weak(writers, w)) { + return (writers == nullptr); + } + } +} + +bool WriteThread::LinkGroup(WriteGroup& write_group, + std::atomic* newest_writer) { + assert(newest_writer != nullptr); + Writer* leader = write_group.leader; + Writer* last_writer = write_group.last_writer; + Writer* w = last_writer; + while (true) { + // Unset link_newer pointers to make sure when we call + // CreateMissingNewerLinks later it create all missing links. + w->link_newer = nullptr; + w->write_group = nullptr; + if (w == leader) { + break; + } + w = w->link_older; + } + Writer* newest = newest_writer->load(std::memory_order_relaxed); + while (true) { + leader->link_older = newest; + if (newest_writer->compare_exchange_weak(newest, last_writer)) { + return (newest == nullptr); + } + } +} + +void WriteThread::CreateMissingNewerLinks(Writer* head) { + while (true) { + Writer* next = head->link_older; + if (next == nullptr || next->link_newer != nullptr) { + assert(next == nullptr || next->link_newer == head); + break; + } + next->link_newer = head; + head = next; + } +} + +void WriteThread::CompleteLeader(WriteGroup& write_group) { + assert(write_group.size > 0); + Writer* leader = write_group.leader; + if (write_group.size == 1) { + write_group.leader = nullptr; + write_group.last_writer = nullptr; + } else { + assert(leader->link_newer != nullptr); + leader->link_newer->link_older = nullptr; + write_group.leader = leader->link_newer; + } + write_group.size -= 1; + SetState(leader, STATE_COMPLETED); +} + +void WriteThread::CompleteFollower(Writer* w, WriteGroup& write_group) { + assert(write_group.size > 1); + assert(w != write_group.leader); + if (w == write_group.last_writer) { + w->link_older->link_newer = nullptr; + write_group.last_writer = w->link_older; + } else { + w->link_older->link_newer = w->link_newer; + w->link_newer->link_older = w->link_older; + } + write_group.size -= 1; + SetState(w, STATE_COMPLETED); +} + +static WriteThread::AdaptationContext jbg_ctx("JoinBatchGroup"); +void WriteThread::JoinBatchGroup(Writer* w) { + TEST_SYNC_POINT_CALLBACK("WriteThread::JoinBatchGroup:Start", w); + assert(w->batch != nullptr); + + bool linked_as_leader = LinkOne(w, &newest_writer_); + if (linked_as_leader) { + SetState(w, STATE_GROUP_LEADER); + } + + TEST_SYNC_POINT_CALLBACK("WriteThread::JoinBatchGroup:Wait", w); + + if (!linked_as_leader) { + /** + * Wait util: + * 1) An existing leader pick us as the new leader when it finishes + * 2) An existing leader pick us as its follewer and + * 2.1) finishes the memtable writes on our behalf + * 2.2) Or tell us to finish the memtable writes in pralallel + * 3) (pipelined write) An existing leader pick us as its follower and + * finish book-keeping and WAL write for us, enqueue us as pending + * memtable writer, and + * 3.1) we become memtable writer group leader, or + * 3.2) an existing memtable writer group leader tell us to finish memtable + * writes in parallel. + */ + AwaitState(w, STATE_GROUP_LEADER | STATE_MEMTABLE_WRITER_LEADER | + STATE_PARALLEL_MEMTABLE_WRITER | STATE_COMPLETED, + &jbg_ctx); + TEST_SYNC_POINT_CALLBACK("WriteThread::JoinBatchGroup:DoneWaiting", w); + } +} + +size_t WriteThread::EnterAsBatchGroupLeader(Writer* leader, + WriteGroup* write_group) { + assert(leader->link_older == nullptr); + assert(leader->batch != nullptr); + assert(write_group != nullptr); + + size_t size = WriteBatchInternal::ByteSize(leader->batch); + + // Allow the group to grow up to a maximum size, but if the + // original write is small, limit the growth so we do not slow + // down the small write too much. + size_t max_size = 1 << 20; + if (size <= (128 << 10)) { + max_size = size + (128 << 10); + } + + leader->write_group = write_group; + write_group->leader = leader; + write_group->last_writer = leader; + write_group->size = 1; + Writer* newest_writer = newest_writer_.load(std::memory_order_acquire); + + // This is safe regardless of any db mutex status of the caller. Previous + // calls to ExitAsGroupLeader either didn't call CreateMissingNewerLinks + // (they emptied the list and then we added ourself as leader) or had to + // explicitly wake us up (the list was non-empty when we added ourself, + // so we have already received our MarkJoined). + CreateMissingNewerLinks(newest_writer); + + // Tricky. Iteration start (leader) is exclusive and finish + // (newest_writer) is inclusive. Iteration goes from old to new. + Writer* w = leader; + while (w != newest_writer) { + w = w->link_newer; + + if (w->sync && !leader->sync) { + // Do not include a sync write into a batch handled by a non-sync write. + break; + } + + if (w->no_slowdown != leader->no_slowdown) { + // Do not mix writes that are ok with delays with the ones that + // request fail on delays. + break; + } + + if (!w->disable_wal && leader->disable_wal) { + // Do not include a write that needs WAL into a batch that has + // WAL disabled. + break; + } + + if (w->batch == nullptr) { + // Do not include those writes with nullptr batch. Those are not writes, + // those are something else. They want to be alone + break; + } + + if (w->callback != nullptr && !w->callback->AllowWriteBatching()) { + // dont batch writes that don't want to be batched + break; + } + + auto batch_size = WriteBatchInternal::ByteSize(w->batch); + if (size + batch_size > max_size) { + // Do not make batch too big + break; + } + + w->write_group = write_group; + size += batch_size; + write_group->last_writer = w; + write_group->size++; + } + return size; +} + +void WriteThread::EnterAsMemTableWriter(Writer* leader, + WriteGroup* write_group) { + assert(leader != nullptr); + assert(leader->link_older == nullptr); + assert(leader->batch != nullptr); + assert(write_group != nullptr); + + size_t size = WriteBatchInternal::ByteSize(leader->batch); + + // Allow the group to grow up to a maximum size, but if the + // original write is small, limit the growth so we do not slow + // down the small write too much. + size_t max_size = 1 << 20; + if (size <= (128 << 10)) { + max_size = size + (128 << 10); + } + + leader->write_group = write_group; + write_group->leader = leader; + write_group->size = 1; + Writer* last_writer = leader; + + if (!allow_concurrent_memtable_write_ || !leader->batch->HasMerge()) { + Writer* newest_writer = newest_memtable_writer_.load(); + CreateMissingNewerLinks(newest_writer); + + Writer* w = leader; + while (w != newest_writer) { + w = w->link_newer; + + if (w->batch == nullptr) { + break; + } + + if (w->batch->HasMerge()) { + break; + } + + if (!allow_concurrent_memtable_write_) { + auto batch_size = WriteBatchInternal::ByteSize(w->batch); + if (size + batch_size > max_size) { + // Do not make batch too big + break; + } + size += batch_size; + } + + w->write_group = write_group; + last_writer = w; + write_group->size++; + } + } + + write_group->last_writer = last_writer; + write_group->last_sequence = + last_writer->sequence + WriteBatchInternal::Count(last_writer->batch) - 1; +} + +void WriteThread::ExitAsMemTableWriter(Writer* self, WriteGroup& write_group) { + Writer* leader = write_group.leader; + Writer* last_writer = write_group.last_writer; + + Writer* newest_writer = last_writer; + if (!newest_memtable_writer_.compare_exchange_strong(newest_writer, + nullptr)) { + CreateMissingNewerLinks(newest_writer); + Writer* next_leader = last_writer->link_newer; + assert(next_leader != nullptr); + next_leader->link_older = nullptr; + SetState(next_leader, STATE_MEMTABLE_WRITER_LEADER); + } + Writer* w = leader; + while (true) { + if (!write_group.status.ok()) { + w->status = write_group.status; + } + Writer* next = w->link_newer; + if (w != leader) { + SetState(w, STATE_COMPLETED); + } + if (w == last_writer) { + break; + } + w = next; + } + // Note that leader has to exit last, since it owns the write group. + SetState(leader, STATE_COMPLETED); +} + +void WriteThread::LaunchParallelMemTableWriters(WriteGroup* write_group) { + assert(write_group != nullptr); + write_group->running.store(write_group->size); + for (auto w : *write_group) { + SetState(w, STATE_PARALLEL_MEMTABLE_WRITER); + } +} + +static WriteThread::AdaptationContext cpmtw_ctx("CompleteParallelMemTableWriter"); +// This method is called by both the leader and parallel followers +bool WriteThread::CompleteParallelMemTableWriter(Writer* w) { + + auto* write_group = w->write_group; + if (!w->status.ok()) { + std::lock_guard guard(write_group->leader->StateMutex()); + write_group->status = w->status; + } + + if (write_group->running-- > 1) { + // we're not the last one + AwaitState(w, STATE_COMPLETED, &cpmtw_ctx); + return false; + } + // else we're the last parallel worker and should perform exit duties. + w->status = write_group->status; + return true; +} + +void WriteThread::ExitAsBatchGroupFollower(Writer* w) { + auto* write_group = w->write_group; + + assert(w->state == STATE_PARALLEL_MEMTABLE_WRITER); + assert(write_group->status.ok()); + ExitAsBatchGroupLeader(*write_group, write_group->status); + assert(w->status.ok()); + assert(w->state == STATE_COMPLETED); + SetState(write_group->leader, STATE_COMPLETED); +} + +static WriteThread::AdaptationContext eabgl_ctx("ExitAsBatchGroupLeader"); +void WriteThread::ExitAsBatchGroupLeader(WriteGroup& write_group, + Status status) { + Writer* leader = write_group.leader; + Writer* last_writer = write_group.last_writer; + assert(leader->link_older == nullptr); + + // Propagate memtable write error to the whole group. + if (status.ok() && !write_group.status.ok()) { + status = write_group.status; + } + + if (enable_pipelined_write_) { + // Notify writers don't write to memtable to exit. + for (Writer* w = last_writer; w != leader;) { + Writer* next = w->link_older; + w->status = status; + if (!w->ShouldWriteToMemtable()) { + CompleteFollower(w, write_group); + } + w = next; + } + if (!leader->ShouldWriteToMemtable()) { + CompleteLeader(write_group); + } + // Link the ramaining of the group to memtable writer list. + if (write_group.size > 0) { + if (LinkGroup(write_group, &newest_memtable_writer_)) { + // The leader can now be different from current writer. + SetState(write_group.leader, STATE_MEMTABLE_WRITER_LEADER); + } + } + // Reset newest_writer_ and wake up the next leader. + Writer* newest_writer = last_writer; + if (!newest_writer_.compare_exchange_strong(newest_writer, nullptr)) { + Writer* next_leader = newest_writer; + while (next_leader->link_older != last_writer) { + next_leader = next_leader->link_older; + assert(next_leader != nullptr); + } + next_leader->link_older = nullptr; + SetState(next_leader, STATE_GROUP_LEADER); + } + AwaitState(leader, STATE_MEMTABLE_WRITER_LEADER | + STATE_PARALLEL_MEMTABLE_WRITER | STATE_COMPLETED, + &eabgl_ctx); + } else { + Writer* head = newest_writer_.load(std::memory_order_acquire); + if (head != last_writer || + !newest_writer_.compare_exchange_strong(head, nullptr)) { + // Either w wasn't the head during the load(), or it was the head + // during the load() but somebody else pushed onto the list before + // we did the compare_exchange_strong (causing it to fail). In the + // latter case compare_exchange_strong has the effect of re-reading + // its first param (head). No need to retry a failing CAS, because + // only a departing leader (which we are at the moment) can remove + // nodes from the list. + assert(head != last_writer); + + // After walking link_older starting from head (if not already done) + // we will be able to traverse w->link_newer below. This function + // can only be called from an active leader, only a leader can + // clear newest_writer_, we didn't, and only a clear newest_writer_ + // could cause the next leader to start their work without a call + // to MarkJoined, so we can definitely conclude that no other leader + // work is going on here (with or without db mutex). + CreateMissingNewerLinks(head); + assert(last_writer->link_newer->link_older == last_writer); + last_writer->link_newer->link_older = nullptr; + + // Next leader didn't self-identify, because newest_writer_ wasn't + // nullptr when they enqueued (we were definitely enqueued before them + // and are still in the list). That means leader handoff occurs when + // we call MarkJoined + SetState(last_writer->link_newer, STATE_GROUP_LEADER); + } + // else nobody else was waiting, although there might already be a new + // leader now + + while (last_writer != leader) { + last_writer->status = status; + // we need to read link_older before calling SetState, because as soon + // as it is marked committed the other thread's Await may return and + // deallocate the Writer. + auto next = last_writer->link_older; + SetState(last_writer, STATE_COMPLETED); + + last_writer = next; + } + } +} + +static WriteThread::AdaptationContext eu_ctx("EnterUnbatched"); +void WriteThread::EnterUnbatched(Writer* w, InstrumentedMutex* mu) { + assert(w != nullptr && w->batch == nullptr); + mu->Unlock(); + bool linked_as_leader = LinkOne(w, &newest_writer_); + if (!linked_as_leader) { + TEST_SYNC_POINT("WriteThread::EnterUnbatched:Wait"); + // Last leader will not pick us as a follower since our batch is nullptr + AwaitState(w, STATE_GROUP_LEADER, &eu_ctx); + } + if (enable_pipelined_write_) { + WaitForMemTableWriters(); + } + mu->Lock(); +} + +void WriteThread::ExitUnbatched(Writer* w) { + assert(w != nullptr); + Writer* newest_writer = w; + if (!newest_writer_.compare_exchange_strong(newest_writer, nullptr)) { + CreateMissingNewerLinks(newest_writer); + Writer* next_leader = w->link_newer; + assert(next_leader != nullptr); + next_leader->link_older = nullptr; + SetState(next_leader, STATE_GROUP_LEADER); + } +} + +static WriteThread::AdaptationContext wfmw_ctx("WaitForMemTableWriters"); +void WriteThread::WaitForMemTableWriters() { + assert(enable_pipelined_write_); + if (newest_memtable_writer_.load() == nullptr) { + return; + } + Writer w; + if (!LinkOne(&w, &newest_memtable_writer_)) { + AwaitState(&w, STATE_MEMTABLE_WRITER_LEADER, &wfmw_ctx); + } + newest_memtable_writer_.store(nullptr); +} + +} // namespace rocksdb diff --git a/db/write_thread.h b/db/write_thread.h new file mode 100644 index 00000000000..57ce71e08f3 --- /dev/null +++ b/db/write_thread.h @@ -0,0 +1,391 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "db/write_callback.h" +#include "monitoring/instrumented_mutex.h" +#include "rocksdb/options.h" +#include "rocksdb/status.h" +#include "rocksdb/types.h" +#include "rocksdb/write_batch.h" +#include "util/autovector.h" + +namespace rocksdb { + +class WriteThread { + public: + enum State : uint8_t { + // The initial state of a writer. This is a Writer that is + // waiting in JoinBatchGroup. This state can be left when another + // thread informs the waiter that it has become a group leader + // (-> STATE_GROUP_LEADER), when a leader that has chosen to be + // non-parallel informs a follower that its writes have been committed + // (-> STATE_COMPLETED), or when a leader that has chosen to perform + // updates in parallel and needs this Writer to apply its batch (-> + // STATE_PARALLEL_FOLLOWER). + STATE_INIT = 1, + + // The state used to inform a waiting Writer that it has become the + // leader, and it should now build a write batch group. Tricky: + // this state is not used if newest_writer_ is empty when a writer + // enqueues itself, because there is no need to wait (or even to + // create the mutex and condvar used to wait) in that case. This is + // a terminal state unless the leader chooses to make this a parallel + // batch, in which case the last parallel worker to finish will move + // the leader to STATE_COMPLETED. + STATE_GROUP_LEADER = 2, + + // The state used to inform a waiting writer that it has become the + // leader of memtable writer group. The leader will either write + // memtable for the whole group, or launch a parallel group write + // to memtable by calling LaunchParallelMemTableWrite. + STATE_MEMTABLE_WRITER_LEADER = 4, + + // The state used to inform a waiting writer that it has become a + // parallel memtable writer. It can be the group leader who launch the + // parallel writer group, or one of the followers. The writer should then + // apply its batch to the memtable concurrently and call + // CompleteParallelMemTableWriter. + STATE_PARALLEL_MEMTABLE_WRITER = 8, + + // A follower whose writes have been applied, or a parallel leader + // whose followers have all finished their work. This is a terminal + // state. + STATE_COMPLETED = 16, + + // A state indicating that the thread may be waiting using StateMutex() + // and StateCondVar() + STATE_LOCKED_WAITING = 32, + }; + + struct Writer; + + struct WriteGroup { + Writer* leader = nullptr; + Writer* last_writer = nullptr; + SequenceNumber last_sequence; + // before running goes to zero, status needs leader->StateMutex() + Status status; + std::atomic running; + size_t size = 0; + + struct Iterator { + Writer* writer; + Writer* last_writer; + + explicit Iterator(Writer* w, Writer* last) + : writer(w), last_writer(last) {} + + Writer* operator*() const { return writer; } + + Iterator& operator++() { + assert(writer != nullptr); + if (writer == last_writer) { + writer = nullptr; + } else { + writer = writer->link_newer; + } + return *this; + } + + bool operator!=(const Iterator& other) const { + return writer != other.writer; + } + }; + + Iterator begin() const { return Iterator(leader, last_writer); } + Iterator end() const { return Iterator(nullptr, nullptr); } + }; + + // Information kept for every waiting writer. + struct Writer { + WriteBatch* batch; + bool sync; + bool no_slowdown; + bool disable_wal; + bool disable_memtable; + uint64_t log_used; // log number that this batch was inserted into + uint64_t log_ref; // log number that memtable insert should reference + WriteCallback* callback; + bool made_waitable; // records lazy construction of mutex and cv + std::atomic state; // write under StateMutex() or pre-link + WriteGroup* write_group; + SequenceNumber sequence; // the sequence number to use for the first key + Status status; // status of memtable inserter + Status callback_status; // status returned by callback->Callback() + std::aligned_storage::type state_mutex_bytes; + std::aligned_storage::type state_cv_bytes; + Writer* link_older; // read/write only before linking, or as leader + Writer* link_newer; // lazy, read/write only before linking, or as leader + + Writer() + : batch(nullptr), + sync(false), + no_slowdown(false), + disable_wal(false), + disable_memtable(false), + log_used(0), + log_ref(0), + callback(nullptr), + made_waitable(false), + state(STATE_INIT), + write_group(nullptr), + link_older(nullptr), + link_newer(nullptr) {} + + Writer(const WriteOptions& write_options, WriteBatch* _batch, + WriteCallback* _callback, uint64_t _log_ref, bool _disable_memtable) + : batch(_batch), + sync(write_options.sync), + no_slowdown(write_options.no_slowdown), + disable_wal(write_options.disableWAL), + disable_memtable(_disable_memtable), + log_used(0), + log_ref(_log_ref), + callback(_callback), + made_waitable(false), + state(STATE_INIT), + write_group(nullptr), + link_older(nullptr), + link_newer(nullptr) {} + + ~Writer() { + if (made_waitable) { + StateMutex().~mutex(); + StateCV().~condition_variable(); + } + } + + bool CheckCallback(DB* db) { + if (callback != nullptr) { + callback_status = callback->Callback(db); + } + return callback_status.ok(); + } + + void CreateMutex() { + if (!made_waitable) { + // Note that made_waitable is tracked separately from state + // transitions, because we can't atomically create the mutex and + // link into the list. + made_waitable = true; + new (&state_mutex_bytes) std::mutex; + new (&state_cv_bytes) std::condition_variable; + } + } + + // returns the aggregate status of this Writer + Status FinalStatus() { + if (!status.ok()) { + // a non-ok memtable write status takes presidence + assert(callback == nullptr || callback_status.ok()); + return status; + } else if (!callback_status.ok()) { + // if the callback failed then that is the status we want + // because a memtable insert should not have been attempted + assert(callback != nullptr); + assert(status.ok()); + return callback_status; + } else { + // if there is no callback then we only care about + // the memtable insert status + assert(callback == nullptr || callback_status.ok()); + return status; + } + } + + bool CallbackFailed() { + return (callback != nullptr) && !callback_status.ok(); + } + + bool ShouldWriteToMemtable() { + return status.ok() && !CallbackFailed() && !disable_memtable; + } + + bool ShouldWriteToWAL() { + return status.ok() && !CallbackFailed() && !disable_wal; + } + + // No other mutexes may be acquired while holding StateMutex(), it is + // always last in the order + std::mutex& StateMutex() { + assert(made_waitable); + return *static_cast(static_cast(&state_mutex_bytes)); + } + + std::condition_variable& StateCV() { + assert(made_waitable); + return *static_cast( + static_cast(&state_cv_bytes)); + } + }; + + struct AdaptationContext { + const char* name; + std::atomic value; + + explicit AdaptationContext(const char* name0) : name(name0), value(0) {} + }; + + explicit WriteThread(const ImmutableDBOptions& db_options); + + virtual ~WriteThread() = default; + + // IMPORTANT: None of the methods in this class rely on the db mutex + // for correctness. All of the methods except JoinBatchGroup and + // EnterUnbatched may be called either with or without the db mutex held. + // Correctness is maintained by ensuring that only a single thread is + // a leader at a time. + + // Registers w as ready to become part of a batch group, waits until the + // caller should perform some work, and returns the current state of the + // writer. If w has become the leader of a write batch group, returns + // STATE_GROUP_LEADER. If w has been made part of a sequential batch + // group and the leader has performed the write, returns STATE_DONE. + // If w has been made part of a parallel batch group and is responsible + // for updating the memtable, returns STATE_PARALLEL_FOLLOWER. + // + // The db mutex SHOULD NOT be held when calling this function, because + // it will block. + // + // Writer* w: Writer to be executed as part of a batch group + void JoinBatchGroup(Writer* w); + + // Constructs a write batch group led by leader, which should be a + // Writer passed to JoinBatchGroup on the current thread. + // + // Writer* leader: Writer that is STATE_GROUP_LEADER + // WriteGroup* write_group: Out-param of group members + // returns: Total batch group byte size + size_t EnterAsBatchGroupLeader(Writer* leader, WriteGroup* write_group); + + // Unlinks the Writer-s in a batch group, wakes up the non-leaders, + // and wakes up the next leader (if any). + // + // WriteGroup* write_group: the write group + // Status status: Status of write operation + void ExitAsBatchGroupLeader(WriteGroup& write_group, Status status); + + // Exit batch group on behalf of batch group leader. + void ExitAsBatchGroupFollower(Writer* w); + + // Constructs a write batch group led by leader from newest_memtable_writers_ + // list. The leader should either write memtable for the whole group and + // call ExitAsMemTableWriter, or launch parallel memtable write through + // LaunchParallelMemTableWriters. + void EnterAsMemTableWriter(Writer* leader, WriteGroup* write_grup); + + // Memtable writer group leader, or the last finished writer in a parallel + // write group, exit from the newest_memtable_writers_ list, and wake up + // the next leader if needed. + void ExitAsMemTableWriter(Writer* self, WriteGroup& write_group); + + // Causes JoinBatchGroup to return STATE_PARALLEL_FOLLOWER for all of the + // non-leader members of this write batch group. Sets Writer::sequence + // before waking them up. + // + // WriteGroup* write_group: Extra state used to coordinate the parallel add + void LaunchParallelMemTableWriters(WriteGroup* write_group); + + // Reports the completion of w's batch to the parallel group leader, and + // waits for the rest of the parallel batch to complete. Returns true + // if this thread is the last to complete, and hence should advance + // the sequence number and then call EarlyExitParallelGroup, false if + // someone else has already taken responsibility for that. + bool CompleteParallelMemTableWriter(Writer* w); + + // Waits for all preceding writers (unlocking mu while waiting), then + // registers w as the currently proceeding writer. + // + // Writer* w: A Writer not eligible for batching + // InstrumentedMutex* mu: The db mutex, to unlock while waiting + // REQUIRES: db mutex held + void EnterUnbatched(Writer* w, InstrumentedMutex* mu); + + // Completes a Writer begun with EnterUnbatched, unblocking subsequent + // writers. + void ExitUnbatched(Writer* w); + + // Wait for all parallel memtable writers to finish, in case pipelined + // write is enabled. + void WaitForMemTableWriters(); + + SequenceNumber UpdateLastSequence(SequenceNumber sequence) { + if (sequence > last_sequence_) { + last_sequence_ = sequence; + } + return last_sequence_; + } + + private: + // See AwaitState. + const uint64_t max_yield_usec_; + const uint64_t slow_yield_usec_; + + // Allow multiple writers write to memtable concurrently. + const bool allow_concurrent_memtable_write_; + + // Enable pipelined write to WAL and memtable. + const bool enable_pipelined_write_; + + // Points to the newest pending writer. Only leader can remove + // elements, adding can be done lock-free by anybody. + std::atomic newest_writer_; + + // Points to the newest pending memtable writer. Used only when pipelined + // write is enabled. + std::atomic newest_memtable_writer_; + + // The last sequence that have been consumed by a writer. The sequence + // is not necessary visible to reads because the writer can be ongoing. + SequenceNumber last_sequence_; + + // Waits for w->state & goal_mask using w->StateMutex(). Returns + // the state that satisfies goal_mask. + uint8_t BlockingAwaitState(Writer* w, uint8_t goal_mask); + + // Blocks until w->state & goal_mask, returning the state value + // that satisfied the predicate. Uses ctx to adaptively use + // std::this_thread::yield() to avoid mutex overheads. ctx should be + // a context-dependent static. + uint8_t AwaitState(Writer* w, uint8_t goal_mask, AdaptationContext* ctx); + + // Set writer state and wake the writer up if it is waiting. + void SetState(Writer* w, uint8_t new_state); + + // Links w into the newest_writer list. Return true if w was linked directly + // into the leader position. Safe to call from multiple threads without + // external locking. + bool LinkOne(Writer* w, std::atomic* newest_writer); + + // Link write group into the newest_writer list as a whole, while keeping the + // order of the writers unchanged. Return true if the group was linked + // directly into the leader position. + bool LinkGroup(WriteGroup& write_group, std::atomic* newest_writer); + + // Computes any missing link_newer links. Should not be called + // concurrently with itself. + void CreateMissingNewerLinks(Writer* head); + + // Set the leader in write_group to completed state and remove it from the + // write group. + void CompleteLeader(WriteGroup& write_group); + + // Set a follower in write_group to completed state and remove it from the + // write group. + void CompleteFollower(Writer* w, WriteGroup& write_group); +}; + +} // namespace rocksdb diff --git a/doc/doc.css b/doc/doc.css deleted file mode 100644 index 700c564e433..00000000000 --- a/doc/doc.css +++ /dev/null @@ -1,89 +0,0 @@ -body { - margin-left: 0.5in; - margin-right: 0.5in; - background: white; - color: black; -} - -h1 { - margin-left: -0.2in; - font-size: 14pt; -} -h2 { - margin-left: -0in; - font-size: 12pt; -} -h3 { - margin-left: -0in; -} -h4 { - margin-left: -0in; -} -hr { - margin-left: -0in; -} - -/* Definition lists: definition term bold */ -dt { - font-weight: bold; -} - -address { - text-align: center; -} -code,samp,var { - color: blue; -} -kbd { - color: #600000; -} -div.note p { - float: right; - width: 3in; - margin-right: 0%; - padding: 1px; - border: 2px solid #6060a0; - background-color: #fffff0; -} - -ul { - margin-top: -0em; - margin-bottom: -0em; -} - -ol { - margin-top: -0em; - margin-bottom: -0em; -} - -UL.nobullets { - list-style-type: none; - list-style-image: none; - margin-left: -1em; -} - -p { - margin: 1em 0 1em 0; - padding: 0 0 0 0; -} - -pre { - line-height: 1.3em; - padding: 0.4em 0 0.8em 0; - margin: 0 0 0 0; - border: 0 0 0 0; - color: blue; -} - -.datatable { - margin-left: auto; - margin-right: auto; - margin-top: 2em; - margin-bottom: 2em; - border: 1px solid; -} - -.datatable td,th { - padding: 0 0.5em 0 0.5em; - text-align: right; -} diff --git a/doc/index.html b/doc/index.html deleted file mode 100644 index 94f7cb888dd..00000000000 --- a/doc/index.html +++ /dev/null @@ -1,827 +0,0 @@ - - - - -RocksDB - - - -

RocksDB

-
The Facebook Database Engineering Team
-
Build on earlier work on leveldb by Sanjay Ghemawat - (sanjay@google.com) and Jeff Dean (jeff@google.com)
-

-The rocksdb library provides a persistent key value store. Keys and -values are arbitrary byte arrays. The keys are ordered within the key -value store according to a user-specified comparator function. - -

-

Opening A Database

-

-A rocksdb database has a name which corresponds to a file system -directory. All of the contents of database are stored in this -directory. The following example shows how to open a database, -creating it if necessary: -

-

-  #include <assert>
-  #include "rocksdb/db.h"
-
-  rocksdb::DB* db;
-  rocksdb::Options options;
-  options.create_if_missing = true;
-  rocksdb::Status status = rocksdb::DB::Open(options, "/tmp/testdb", &db);
-  assert(status.ok());
-  ...
-
-If you want to raise an error if the database already exists, add -the following line before the rocksdb::DB::Open call: -
-  options.error_if_exists = true;
-
-

Status

-

-You may have noticed the rocksdb::Status type above. Values of this -type are returned by most functions in rocksdb that may encounter an -error. You can check if such a result is ok, and also print an -associated error message: -

-

-   rocksdb::Status s = ...;
-   if (!s.ok()) cerr << s.ToString() << endl;
-
-

Closing A Database

-

-When you are done with a database, just delete the database object. -Example: -

-

-  ... open the db as described above ...
-  ... do something with db ...
-  delete db;
-
-

Reads And Writes

-

-The database provides Put, Delete, and Get methods to -modify/query the database. For example, the following code -moves the value stored under key1 to key2. -

-  std::string value;
-  rocksdb::Status s = db->Get(rocksdb::ReadOptions(), key1, &value);
-  if (s.ok()) s = db->Put(rocksdb::WriteOptions(), key2, value);
-  if (s.ok()) s = db->Delete(rocksdb::WriteOptions(), key1);
-
- -

Atomic Updates

-

-Note that if the process dies after the Put of key2 but before the -delete of key1, the same value may be left stored under multiple keys. -Such problems can be avoided by using the WriteBatch class to -atomically apply a set of updates: -

-

-  #include "rocksdb/write_batch.h"
-  ...
-  std::string value;
-  rocksdb::Status s = db->Get(rocksdb::ReadOptions(), key1, &value);
-  if (s.ok()) {
-    rocksdb::WriteBatch batch;
-    batch.Delete(key1);
-    batch.Put(key2, value);
-    s = db->Write(rocksdb::WriteOptions(), &batch);
-  }
-
-The WriteBatch holds a sequence of edits to be made to the database, -and these edits within the batch are applied in order. Note that we -called Delete before Put so that if key1 is identical to key2, -we do not end up erroneously dropping the value entirely. -

-Apart from its atomicity benefits, WriteBatch may also be used to -speed up bulk updates by placing lots of individual mutations into the -same batch. - -

Synchronous Writes

-By default, each write to leveldb is asynchronous: it -returns after pushing the write from the process into the operating -system. The transfer from operating system memory to the underlying -persistent storage happens asynchronously. The sync flag -can be turned on for a particular write to make the write operation -not return until the data being written has been pushed all the way to -persistent storage. (On Posix systems, this is implemented by calling -either fsync(...) or fdatasync(...) or -msync(..., MS_SYNC) before the write operation returns.) -
-  rocksdb::WriteOptions write_options;
-  write_options.sync = true;
-  db->Put(write_options, ...);
-
-Asynchronous writes are often more than a thousand times as fast as -synchronous writes. The downside of asynchronous writes is that a -crash of the machine may cause the last few updates to be lost. Note -that a crash of just the writing process (i.e., not a reboot) will not -cause any loss since even when sync is false, an update -is pushed from the process memory into the operating system before it -is considered done. - -

-Asynchronous writes can often be used safely. For example, when -loading a large amount of data into the database you can handle lost -updates by restarting the bulk load after a crash. A hybrid scheme is -also possible where every Nth write is synchronous, and in the event -of a crash, the bulk load is restarted just after the last synchronous -write finished by the previous run. (The synchronous write can update -a marker that describes where to restart on a crash.) - -

-WriteBatch provides an alternative to asynchronous writes. -Multiple updates may be placed in the same WriteBatch and -applied together using a synchronous write (i.e., -write_options.sync is set to true). The extra cost of -the synchronous write will be amortized across all of the writes in -the batch. - -

-We also provide a way to completely disable Write Ahead Log for a -particular write. If you set write_option.disableWAL to true, the -write will not go to the log at all and may be lost in an event of -process crash. - -

-When opening a DB, you can disable syncing of data files by setting -Options::disableDataSync to true. This can be useful when doing -bulk-loading or big idempotent operations. Once the operation is -finished, you can manually call sync() to flush all dirty buffers -to stable storage. - -

-RocksDB by default uses faster fdatasync() to sync files. If you want -to use fsync(), you can set Options::use_fsync to true. You should set -this to true on filesystems like ext3 that can lose files after a -reboot. - -

-

Concurrency

-

-A database may only be opened by one process at a time. -The rocksdb implementation acquires a lock from the -operating system to prevent misuse. Within a single process, the -same rocksdb::DB object may be safely shared by multiple -concurrent threads. I.e., different threads may write into or fetch -iterators or call Get on the same database without any -external synchronization (the leveldb implementation will -automatically do the required synchronization). However other objects -(like Iterator and WriteBatch) may require external synchronization. -If two threads share such an object, they must protect access to it -using their own locking protocol. More details are available in -the public header files. - -

-

Merge operators

-

-Merge operators provide efficient support for read-modify-write operation. -More on the interface and implementation can be found on: -

- - Merge Operator -

- - Merge Operator Implementation - -

-

Iteration

-

-The following example demonstrates how to print all key,value pairs -in a database. -

-

-  rocksdb::Iterator* it = db->NewIterator(rocksdb::ReadOptions());
-  for (it->SeekToFirst(); it->Valid(); it->Next()) {
-    cout << it->key().ToString() << ": "  << it->value().ToString() << endl;
-  }
-  assert(it->status().ok());  // Check for any errors found during the scan
-  delete it;
-
-The following variation shows how to process just the keys in the -range [start,limit): -

-

-  for (it->Seek(start);
-       it->Valid() && it->key().ToString() < limit;
-       it->Next()) {
-    ...
-  }
-
-You can also process entries in reverse order. (Caveat: reverse -iteration may be somewhat slower than forward iteration.) -

-

-  for (it->SeekToLast(); it->Valid(); it->Prev()) {
-    ...
-  }
-
-

Snapshots

-

-Snapshots provide consistent read-only views over the entire state of -the key-value store. ReadOptions::snapshot may be non-NULL to indicate -that a read should operate on a particular version of the DB state. -If ReadOptions::snapshot is NULL, the read will operate on an -implicit snapshot of the current state. -

-Snapshots are created by the DB::GetSnapshot() method: -

-

-  rocksdb::ReadOptions options;
-  options.snapshot = db->GetSnapshot();
-  ... apply some updates to db ...
-  rocksdb::Iterator* iter = db->NewIterator(options);
-  ... read using iter to view the state when the snapshot was created ...
-  delete iter;
-  db->ReleaseSnapshot(options.snapshot);
-
-Note that when a snapshot is no longer needed, it should be released -using the DB::ReleaseSnapshot interface. This allows the -implementation to get rid of state that was being maintained just to -support reading as of that snapshot. -

Slice

-

-The return value of the it->key() and it->value() calls above -are instances of the rocksdb::Slice type. Slice is a simple -structure that contains a length and a pointer to an external byte -array. Returning a Slice is a cheaper alternative to returning a -std::string since we do not need to copy potentially large keys and -values. In addition, rocksdb methods do not return null-terminated -C-style strings since rocksdb keys and values are allowed to -contain '\0' bytes. -

-C++ strings and null-terminated C-style strings can be easily converted -to a Slice: -

-

-   rocksdb::Slice s1 = "hello";
-
-   std::string str("world");
-   rocksdb::Slice s2 = str;
-
-A Slice can be easily converted back to a C++ string: -
-   std::string str = s1.ToString();
-   assert(str == std::string("hello"));
-
-Be careful when using Slices since it is up to the caller to ensure that -the external byte array into which the Slice points remains live while -the Slice is in use. For example, the following is buggy: -

-

-   rocksdb::Slice slice;
-   if (...) {
-     std::string str = ...;
-     slice = str;
-   }
-   Use(slice);
-
-When the if statement goes out of scope, str will be destroyed and the -backing storage for slice will disappear. -

-

Comparators

-

-The preceding examples used the default ordering function for key, -which orders bytes lexicographically. You can however supply a custom -comparator when opening a database. For example, suppose each -database key consists of two numbers and we should sort by the first -number, breaking ties by the second number. First, define a proper -subclass of rocksdb::Comparator that expresses these rules: -

-

-  class TwoPartComparator : public rocksdb::Comparator {
-   public:
-    // Three-way comparison function:
-    //   if a < b: negative result
-    //   if a > b: positive result
-    //   else: zero result
-    int Compare(const rocksdb::Slice& a, const rocksdb::Slice& b) const {
-      int a1, a2, b1, b2;
-      ParseKey(a, &a1, &a2);
-      ParseKey(b, &b1, &b2);
-      if (a1 < b1) return -1;
-      if (a1 > b1) return +1;
-      if (a2 < b2) return -1;
-      if (a2 > b2) return +1;
-      return 0;
-    }
-
-    // Ignore the following methods for now:
-    const char* Name() const { return "TwoPartComparator"; }
-    void FindShortestSeparator(std::string*, const rocksdb::Slice&) const { }
-    void FindShortSuccessor(std::string*) const { }
-  };
-
-Now create a database using this custom comparator: -

-

-  TwoPartComparator cmp;
-  rocksdb::DB* db;
-  rocksdb::Options options;
-  options.create_if_missing = true;
-  options.comparator = &cmp;
-  rocksdb::Status status = rocksdb::DB::Open(options, "/tmp/testdb", &db);
-  ...
-
-

Backwards compatibility

-

-The result of the comparator's Name method is attached to the -database when it is created, and is checked on every subsequent -database open. If the name changes, the rocksdb::DB::Open call will -fail. Therefore, change the name if and only if the new key format -and comparison function are incompatible with existing databases, and -it is ok to discard the contents of all existing databases. -

-You can however still gradually evolve your key format over time with -a little bit of pre-planning. For example, you could store a version -number at the end of each key (one byte should suffice for most uses). -When you wish to switch to a new key format (e.g., adding an optional -third part to the keys processed by TwoPartComparator), -(a) keep the same comparator name (b) increment the version number -for new keys (c) change the comparator function so it uses the -version numbers found in the keys to decide how to interpret them. - - -

-

MemTable and Table factories

-

-By default, we keep the data in memory in skiplist memtable and the data -on disk in a table format described here: - - RocksDB Table Format. -

-Since one of the goals of RocksDB is to have -different parts of the system easily pluggable, we support different -implementations of both memtable and table format. You can supply -your own memtable factory by setting Options::memtable_factory -and your own table factory by setting Options::table_factory. -For available memtable factories, please refer to -rocksdb/memtablerep.h and for table factores to -rocksdb/table.h. These features are both in active development -and please be wary of any API changes that might break your application -going forward. -

-You can also read more about memtables here: - -Memtables wiki - - -

-

Performance

-

-Performance can be tuned by changing the default values of the -types defined in include/rocksdb/options.h. - -

-

Block size

-

-rocksdb groups adjacent keys together into the same block and such a -block is the unit of transfer to and from persistent storage. The -default block size is approximately 4096 uncompressed bytes. -Applications that mostly do bulk scans over the contents of the -database may wish to increase this size. Applications that do a lot -of point reads of small values may wish to switch to a smaller block -size if performance measurements indicate an improvement. There isn't -much benefit in using blocks smaller than one kilobyte, or larger than -a few megabytes. Also note that compression will be more effective -with larger block sizes. To change block size parameter, use -Options::block_size. -

-

Write buffer

-

-Options::write_buffer_size specifies the amount of data -to build up in memory before converting to a sorted on-disk file. -Larger values increase performance, especially during bulk loads. -Up to max_write_buffer_number write buffers may be held in memory -at the same time, -so you may wish to adjust this parameter to control memory usage. -Also, a larger write buffer will result in a longer recovery time -the next time the database is opened. -Related option is -Options::max_write_buffer_number, which is maximum number -of write buffers that are built up in memory. The default is 2, so that -when 1 write buffer is being flushed to storage, new writes can continue -to the other write buffer. -Options::min_write_buffer_number_to_merge is the minimum number -of write buffers that will be merged together before writing to storage. -If set to 1, then all write buffers are flushed to L0 as individual files and -this increases read amplification because a get request has to check in all -of these files. Also, an in-memory merge may result in writing lesser -data to storage if there are duplicate records in each of these -individual write buffers. Default: 1 -

-

Compression

-

-Each block is individually compressed before being written to -persistent storage. Compression is on by default since the default -compression method is very fast, and is automatically disabled for -uncompressible data. In rare cases, applications may want to disable -compression entirely, but should only do so if benchmarks show a -performance improvement: -

-

-  rocksdb::Options options;
-  options.compression = rocksdb::kNoCompression;
-  ... rocksdb::DB::Open(options, name, ...) ....
-
-

Cache

-

-The contents of the database are stored in a set of files in the -filesystem and each file stores a sequence of compressed blocks. If -options.block_cache is non-NULL, it is used to cache frequently -used uncompressed block contents. If options.block_cache_compressed -is non-NULL, it is used to cache frequently used compressed blocks. Compressed -cache is an alternative to OS cache, which also caches compressed blocks. If -compressed cache is used, the OS cache will be disabled automatically by setting -options.allow_os_buffer to false. -

-

-  #include "rocksdb/cache.h"
-
-  rocksdb::Options options;
-  options.block_cache = rocksdb::NewLRUCache(100 * 1048576);  // 100MB uncompressed cache
-  options.block_cache_compressed = rocksdb::NewLRUCache(100 * 1048576);  // 100MB compressed cache
-  rocksdb::DB* db;
-  rocksdb::DB::Open(options, name, &db);
-  ... use the db ...
-  delete db
-  delete options.block_cache;
-  delete options.block_cache_compressed;
-
-

-When performing a bulk read, the application may wish to disable -caching so that the data processed by the bulk read does not end up -displacing most of the cached contents. A per-iterator option can be -used to achieve this: -

-

-  rocksdb::ReadOptions options;
-  options.fill_cache = false;
-  rocksdb::Iterator* it = db->NewIterator(options);
-  for (it->SeekToFirst(); it->Valid(); it->Next()) {
-    ...
-  }
-
-

-You can also disable block cache by setting options.no_block_cache -to true. -

Key Layout

-

-Note that the unit of disk transfer and caching is a block. Adjacent -keys (according to the database sort order) will usually be placed in -the same block. Therefore the application can improve its performance -by placing keys that are accessed together near each other and placing -infrequently used keys in a separate region of the key space. -

-For example, suppose we are implementing a simple file system on top -of rocksdb. The types of entries we might wish to store are: -

-

-   filename -> permission-bits, length, list of file_block_ids
-   file_block_id -> data
-
-We might want to prefix filename keys with one letter (say '/') and the -file_block_id keys with a different letter (say '0') so that scans -over just the metadata do not force us to fetch and cache bulky file -contents. -

-

Filters

-

-Because of the way rocksdb data is organized on disk, -a single Get() call may involve multiple reads from disk. -The optional FilterPolicy mechanism can be used to reduce -the number of disk reads substantially. -

-   rocksdb::Options options;
-   options.filter_policy = NewBloomFilter(10);
-   rocksdb::DB* db;
-   rocksdb::DB::Open(options, "/tmp/testdb", &db);
-   ... use the database ...
-   delete db;
-   delete options.filter_policy;
-
-The preceding code associates a -Bloom filter -based filtering policy with the database. Bloom filter based -filtering relies on keeping some number of bits of data in memory per -key (in this case 10 bits per key since that is the argument we passed -to NewBloomFilter). This filter will reduce the number of unnecessary -disk reads needed for Get() calls by a factor of -approximately a 100. Increasing the bits per key will lead to a -larger reduction at the cost of more memory usage. We recommend that -applications whose working set does not fit in memory and that do a -lot of random reads set a filter policy. -

-If you are using a custom comparator, you should ensure that the filter -policy you are using is compatible with your comparator. For example, -consider a comparator that ignores trailing spaces when comparing keys. -NewBloomFilter must not be used with such a comparator. -Instead, the application should provide a custom filter policy that -also ignores trailing spaces. For example: -

-  class CustomFilterPolicy : public rocksdb::FilterPolicy {
-   private:
-    FilterPolicy* builtin_policy_;
-   public:
-    CustomFilterPolicy() : builtin_policy_(NewBloomFilter(10)) { }
-    ~CustomFilterPolicy() { delete builtin_policy_; }
-
-    const char* Name() const { return "IgnoreTrailingSpacesFilter"; }
-
-    void CreateFilter(const Slice* keys, int n, std::string* dst) const {
-      // Use builtin bloom filter code after removing trailing spaces
-      std::vector<Slice> trimmed(n);
-      for (int i = 0; i < n; i++) {
-        trimmed[i] = RemoveTrailingSpaces(keys[i]);
-      }
-      return builtin_policy_->CreateFilter(&trimmed[i], n, dst);
-    }
-
-    bool KeyMayMatch(const Slice& key, const Slice& filter) const {
-      // Use builtin bloom filter code after removing trailing spaces
-      return builtin_policy_->KeyMayMatch(RemoveTrailingSpaces(key), filter);
-    }
-  };
-
-

-Advanced applications may provide a filter policy that does not use -a bloom filter but uses some other mechanism for summarizing a set -of keys. See rocksdb/filter_policy.h for detail. -

-

Checksums

-

-rocksdb associates checksums with all data it stores in the file system. -There are two separate controls provided over how aggressively these -checksums are verified: -

-

    -
  • ReadOptions::verify_checksums may be set to true to force - checksum verification of all data that is read from the file system on - behalf of a particular read. By default, no such verification is - done. -

    -

  • Options::paranoid_checks may be set to true before opening a - database to make the database implementation raise an error as soon as - it detects an internal corruption. Depending on which portion of the - database has been corrupted, the error may be raised when the database - is opened, or later by another database operation. By default, - paranoid checking is off so that the database can be used even if - parts of its persistent storage have been corrupted. -

    - If a database is corrupted (perhaps it cannot be opened when - paranoid checking is turned on), the rocksdb::RepairDB function - may be used to recover as much of the data as possible. -

    -

- -

-

Compaction

-

-You can read more on Compactions here: - - Multi-threaded compactions - -

-Here we give overview of the options that impact behavior of Compactions: -

    -

    -

  • Options::compaction_style - RocksDB currently supports two -compaction algorithms - Universal style and Level style. This option switches -between the two. Can be kCompactionStyleUniversal or kCompactionStyleLevel. -If this is kCompactionStyleUniversal, then you can configure universal style -parameters with Options::compaction_options_universal. -

    -

  • Options::disable_auto_compactions - Disable automatic compactions. -Manual compactions can still be issued on this database. -

    -

  • Options::compaction_filter - Allows an application to modify/delete -a key-value during background compaction. The client must provide -compaction_filter_factory if it requires a new compaction filter to be used -for different compaction processes. Client should specify only one of filter -or factory. -

    -

  • Options::compaction_filter_factory - a factory that provides -compaction filter objects which allow an application to modify/delete a -key-value during background compaction. -
-

-Other options impacting performance of compactions and when they get triggered -are: -

    -

    -

  • Options::access_hint_on_compaction_start - Specify the file access -pattern once a compaction is started. It will be applied to all input files of a compaction. Default: NORMAL -

    -

  • Options::level0_file_num_compaction_trigger - Number of files to trigger level-0 compaction. -A negative value means that level-0 compaction will not be triggered by number of files at all. -

    -

  • Options::max_mem_compaction_level - Maximum level to which a new compacted memtable is pushed if it -does not create overlap. We try to push to level 2 to avoid the relatively expensive level 0=>1 compactions and to avoid some -expensive manifest file operations. We do not push all the way to the largest level since that can generate a lot of wasted disk -space if the same key space is being repeatedly overwritten. -

    -

  • Options::target_file_size_base and Options::target_file_size_multiplier - -Target file size for compaction. target_file_size_base is per-file size for level-1. -Target file size for level L can be calculated by target_file_size_base * (target_file_size_multiplier ^ (L-1)) -For example, if target_file_size_base is 2MB and target_file_size_multiplier is 10, then each file on level-1 will -be 2MB, and each file on level 2 will be 20MB, and each file on level-3 will be 200MB. Default target_file_size_base is 2MB -and default target_file_size_multiplier is 1. -

    -

  • Options::expanded_compaction_factor - Maximum number of bytes in all compacted files. We avoid expanding -the lower level file set of a compaction if it would make the total compaction cover more than -(expanded_compaction_factor * targetFileSizeLevel()) many bytes. -

    -

  • Options::source_compaction_factor - Maximum number of bytes in all source files to be compacted in a -single compaction run. We avoid picking too many files in the source level so that we do not exceed the total source bytes -for compaction to exceed (source_compaction_factor * targetFileSizeLevel()) many bytes. -Default:1, i.e. pick maxfilesize amount of data as the source of a compaction. -

    -

  • Options::max_grandparent_overlap_factor - Control maximum bytes of overlaps in grandparent (i.e., level+2) before we -stop building a single file in a level->level+1 compaction. -

    -

  • Options::max_background_compactions - Maximum number of concurrent background jobs, submitted to -the default LOW priority thread pool -
- -

-You can learn more about all of those options in rocksdb/options.h - -

Universal style compaction specific settings

-

-If you're using Universal style compaction, there is an object CompactionOptionsUniversal -that hold all the different options for that compaction. The exact definition is in -rocksdb/universal_compaction.h and you can set it in Options::compaction_options_universal. -Here we give short overview of options in CompactionOptionsUniversal: -

    -

    -

  • CompactionOptionsUniversal::size_ratio - Percentage flexibility while comparing file size. If the candidate file(s) - size is 1% smaller than the next file's size, then include next file into - this candidate set. Default: 1 -

    -

  • CompactionOptionsUniversal::min_merge_width - The minimum number of files in a single compaction run. Default: 2 -

    -

  • CompactionOptionsUniversal::max_merge_width - The maximum number of files in a single compaction run. Default: UINT_MAX -

    -

  • CompactionOptionsUniversal::max_size_amplification_percent - The size amplification is defined as the amount (in percentage) of -additional storage needed to store a single byte of data in the database. For example, a size amplification of 2% means that a database that -contains 100 bytes of user-data may occupy upto 102 bytes of physical storage. By this definition, a fully compacted database has -a size amplification of 0%. Rocksdb uses the following heuristic to calculate size amplification: it assumes that all files excluding -the earliest file contribute to the size amplification. Default: 200, which means that a 100 byte database could require upto -300 bytes of storage. -

    -

  • CompactionOptionsUniversal::compression_size_percent - If this option is set to be -1 (the default value), all the output files -will follow compression type specified. If this option is not negative, we will try to make sure compressed -size is just above this value. In normal cases, at least this percentage -of data will be compressed. -When we are compacting to a new file, here is the criteria whether -it needs to be compressed: assuming here are the list of files sorted -by generation time: [ A1...An B1...Bm C1...Ct ], -where A1 is the newest and Ct is the oldest, and we are going to compact -B1...Bm, we calculate the total size of all the files as total_size, as -well as the total size of C1...Ct as total_C, the compaction output file -will be compressed iff total_C / total_size < this percentage -

    -

  • CompactionOptionsUniversal::stop_style - The algorithm used to stop picking files into a single compaction run. -Can be kCompactionStopStyleSimilarSize (pick files of similar size) or kCompactionStopStyleTotalSize (total size of picked files > next file). -Default: kCompactionStopStyleTotalSize -
- -

Thread pools

-

-A thread pool is associated with Env environment object. The client has to create a thread pool by setting the number of background -threads using method Env::SetBackgroundThreads() defined in rocksdb/env.h. -We use the thread pool for compactions and memtable flushes. -Since memtable flushes are in critical code path (stalling memtable flush can stall writes, increasing p99), we suggest -having two thread pools - with priorities HIGH and LOW. Memtable flushes can be set up to be scheduled on HIGH thread pool. -There are two options available for configuration of background compactions and flushes: -

    -

    -

  • Options::max_background_compactions - Maximum number of concurrent background jobs, -submitted to the default LOW priority thread pool -

    -

  • Options::max_background_flushes - Maximum number of concurrent background memtable flush jobs, submitted to -the HIGH priority thread pool. By default, all background jobs (major compaction and memtable flush) go -to the LOW priority pool. If this option is set to a positive number, memtable flush jobs will be submitted to the HIGH priority pool. -It is important when the same Env is shared by multiple db instances. Without a separate pool, long running major compaction jobs could -potentially block memtable flush jobs of other db instances, leading to unnecessary Put stalls. -
-

-

-  #include "rocksdb/env.h"
-  #include "rocksdb/db.h"
-
-  auto env = rocksdb::Env::Default();
-  env->SetBackgroundThreads(2, rocksdb::Env::LOW);
-  env->SetBackgroundThreads(1, rocksdb::Env::HIGH);
-  rocksdb::DB* db;
-  rocksdb::Options options;
-  options.env = env;
-  options.max_background_compactions = 2;
-  options.max_background_flushes = 1;
-  rocksdb::Status status = rocksdb::DB::Open(options, "/tmp/testdb", &db);
-  assert(status.ok());
-  ...
-
-

Approximate Sizes

-

-The GetApproximateSizes method can used to get the approximate -number of bytes of file system space used by one or more key ranges. -

-

-   rocksdb::Range ranges[2];
-   ranges[0] = rocksdb::Range("a", "c");
-   ranges[1] = rocksdb::Range("x", "z");
-   uint64_t sizes[2];
-   rocksdb::Status s = db->GetApproximateSizes(ranges, 2, sizes);
-
-The preceding call will set sizes[0] to the approximate number of -bytes of file system space used by the key range [a..c) and -sizes[1] to the approximate number of bytes used by the key range -[x..z). -

-

Environment

-

-All file operations (and other operating system calls) issued by the -rocksdb implementation are routed through a rocksdb::Env object. -Sophisticated clients may wish to provide their own Env -implementation to get better control. For example, an application may -introduce artificial delays in the file IO paths to limit the impact -of rocksdb on other activities in the system. -

-

-  class SlowEnv : public rocksdb::Env {
-    .. implementation of the Env interface ...
-  };
-
-  SlowEnv env;
-  rocksdb::Options options;
-  options.env = &env;
-  Status s = rocksdb::DB::Open(options, ...);
-
-

Porting

-

-rocksdb may be ported to a new platform by providing platform -specific implementations of the types/methods/functions exported by -rocksdb/port/port.h. See rocksdb/port/port_example.h for more -details. -

-In addition, the new platform may need a new default rocksdb::Env -implementation. See rocksdb/util/env_posix.h for an example. - -

Statistics

-

-To be able to efficiently tune your application, it is always helpful if you -have access to usage statistics. You can collect those statistics by setting -Options::table_properties_collectors or -Options::statistics. For more information, refer to -rocksdb/table_properties.h and rocksdb/statistics.h. -These should not add significant overhead to your application and we -recommend exporting them to other monitoring tools. - -

Purging WAL files

-

-By default, old write-ahead logs are deleted automatically when they fall out -of scope and application doesn't need them anymore. There are options that -enable the user to archive the logs and then delete them lazily, either in -TTL fashion or based on size limit. - -The options are Options::WAL_ttl_seconds and -Options::WAL_size_limit_MB. Here is how they can be used: -

    -
  • -

    -If both set to 0, logs will be deleted asap and will never get into the archive. -

  • -

    -If WAL_ttl_seconds is 0 and WAL_size_limit_MB is not 0, WAL -files will be checked every 10 min and if total size is greater then -WAL_size_limit_MB, they will be deleted starting with the -earliest until size_limit is met. All empty files will be deleted. -

  • -

    -If WAL_ttl_seconds is not 0 and WAL_size_limit_MB is 0, then -WAL files will be checked every WAL_ttl_seconds / 2 and those -that are older than WAL_ttl_seconds will be deleted. -

  • -

    -If both are not 0, WAL files will be checked every 10 min and both -checks will be performed with ttl being first. -

- -

Other Information

-

-Details about the rocksdb implementation may be found in -the following documents: -

- - - diff --git a/doc/log_format.txt b/doc/log_format.txt deleted file mode 100644 index 3a0414b65ac..00000000000 --- a/doc/log_format.txt +++ /dev/null @@ -1,75 +0,0 @@ -The log file contents are a sequence of 32KB blocks. The only -exception is that the tail of the file may contain a partial block. - -Each block consists of a sequence of records: - block := record* trailer? - record := - checksum: uint32 // crc32c of type and data[] - length: uint16 - type: uint8 // One of FULL, FIRST, MIDDLE, LAST - data: uint8[length] - -A record never starts within the last six bytes of a block (since it -won't fit). Any leftover bytes here form the trailer, which must -consist entirely of zero bytes and must be skipped by readers. - -Aside: if exactly seven bytes are left in the current block, and a new -non-zero length record is added, the writer must emit a FIRST record -(which contains zero bytes of user data) to fill up the trailing seven -bytes of the block and then emit all of the user data in subsequent -blocks. - -More types may be added in the future. Some Readers may skip record -types they do not understand, others may report that some data was -skipped. - -FULL == 1 -FIRST == 2 -MIDDLE == 3 -LAST == 4 - -The FULL record contains the contents of an entire user record. - -FIRST, MIDDLE, LAST are types used for user records that have been -split into multiple fragments (typically because of block boundaries). -FIRST is the type of the first fragment of a user record, LAST is the -type of the last fragment of a user record, and MID is the type of all -interior fragments of a user record. - -Example: consider a sequence of user records: - A: length 1000 - B: length 97270 - C: length 8000 -A will be stored as a FULL record in the first block. - -B will be split into three fragments: first fragment occupies the rest -of the first block, second fragment occupies the entirety of the -second block, and the third fragment occupies a prefix of the third -block. This will leave six bytes free in the third block, which will -be left empty as the trailer. - -C will be stored as a FULL record in the fourth block. - -=================== - -Some benefits over the recordio format: - -(1) We do not need any heuristics for resyncing - just go to next -block boundary and scan. If there is a corruption, skip to the next -block. As a side-benefit, we do not get confused when part of the -contents of one log file are embedded as a record inside another log -file. - -(2) Splitting at approximate boundaries (e.g., for mapreduce) is -simple: find the next block boundary and skip records until we -hit a FULL or FIRST record. - -(3) We do not need extra buffering for large records. - -Some downsides compared to recordio format: - -(1) No packing of tiny records. This could be fixed by adding a new -record type, so it is a shortcoming of the current implementation, -not necessarily the format. - -(2) No compression. Again, this could be fixed by adding new record types. diff --git a/doc/rockslogo.jpg b/doc/rockslogo.jpg deleted file mode 100644 index 363905af5c269839cf2ebeab95ebc6414e97a88d..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 137232 zcmeEv2UHcymUbU<&N(X?BudVTWRV~_NDc=i=O7>mN)8evBN<6b1__cgNRGlG3J7=r z!9)I=d*8eF&b&7>Z{C`x)(yLQ9%>@^G^P?lGc2Otm#U<&>K z*Q+>sO0u#h>Y8fuN)P403jlzl_z>#oghT)U4vy}wnhN(B^z;oF&_@6y024q1_yEAn z!p&LrvAh-lZc1{p3~pee8~OL`WEueP1OO9Uis}puzsLVW5}}2&t2+Qd)WOtuEv?)v zK=}nIKlO5VzKMr}GKsnUjf{kHBfEkH1Z9dF+2&XI&M#?xm1S;ZO9w|wFwISwoh=)WZP^$_P*B z-JQjFc)Yy4xS>`S+&2~a+xA~3_$}psEd1el+&9nrt?n2eSXrBSJauQdsZks9)qiJPr!K1v5f8Pw4RY;H$Os5{ijkpb%XAGPp5 zne7i{xDkKFH7F40UjZa0+yKES34pve1fURO1IR{M;1$Sk?e-8;2e^6jbm_nTihEE7 zum5uVHwUCx@K+=^s13tSw5+BUgN28y=Zy@$6E_D`01LneNB~NJ4qyb>04{(Z5C+5n z89)wD2Gjs8Kp%JlSOB(w1KqAPk5E;(#O|4af%affAqsr~>MNW}qGD0S1B3 zz!b0mz<>?l8vqARfOD`TLxfk;8*AP*s$5PgU##0KIB@qqY4 zLLiZl1V|d>Eu;kU0n!L*hx9|nAajrv$Ts8*J~B@Lw*r4FSF zWddad1&(rsii=8v%84q5s)VYCYJ=*98ityTT8LVU+KoDex`BFvhK5Fl#)>A4riiAG zW{2j77KN6HR*CirZ47MP;xpsl!+(PB zi=T{NjX#RNOMpqhLLf_EMi4-dPS8j&LvTz;Ovp#5M(98oMfi@epKyx^orr}wYe z1yLSRC(#-)GBG2uEU^V~2ys4fH}M7u8VM_j5{Vs2BuP2R2nn2&h*W@7ht!)iowS7% zMutqrLZ(Falq{C4ifo$fjGUJI9=R2HIC%y6IQdTsY6=+&D~bq;N{T6pGfDb$Fy{`3bf9& zskEK6yL4o9GIVxyiFB=W+w{crQuI*zMEW-R9R^Z{dkjw*-Y|4Az;DsqQoQAH>+P+P zTj#e~Z)@KUzFm3y3nLbz2%|M)B4Y<5oQa-EmC2u}lxdzBlUan>mN|vFm-#0PD~m46 zOO|?;EmjIvC01Y7Qr1N_TsCPo7q(osNp>`L5q5j_O!m(lNF0J3wjAjkqnt>bLYz>} zOwO@8sCPu~INr&i(zcjfN}-fg&hAiybLC6FaBD@ZJ;D)>^cL-0~aSjbhV zLTFo(9VA8jw&7^asS7jJwEM*F1zTRWMXLqmc-u`{w`>ywE@Bfq) zlYJ)p=>hTsg$EH2M&yX(wB^#|mgH~C+sK#7!xaP-{1iSZqAIE=#w*S!(JEOel_TA`-N6e2LAJwY?YD#La)xM~+s=KIv)Iifv z*GSjc)a29j*X(^v{Mh92yT>P5_qAfR7PQ&4J+wP?@O6xIN_2keKG03jh3WC=J<}W1 zr_zV&HyEHB=ol0l92?3RCK+xR2^xhNO&hZsdmHyZp?dQ4Ns9@-iJ3{YDXOWi={wU4 zGc~hZvtx55^9*yig{(!2#WzbC%S6j)PhI$qV@gx-$cLp~fn;XYq|<$VkMQ2Z?XI{a_> zKlfjGcJJBS07!s&Kt~{B;ETYGAcdgf=UC6}pAQG~1}6mn2+<8`dO`al=*3#7LTG6i zUYKjx%uDf?*{@JuL0^r8-wjUMMJOpd&WvWOap=8Jw4eH~*RGZHHln-zx^ z=NLB~FCAZ;K$zg0u=ZN@bzLGuVnpJPB$K56WP#-D6s#1F6xf@GZyHjWQsYuD(rnYF z((k8NX3%7WXZ*~x%pA{>$*Rbv&5q1I%Yo+1zEyZzm&=lyoQIa@nYWp*liyn)QczMz zRTx=#S>#v*E7mOTDiJOzc}MduwiKzsO)q3gYt$7u8OynWR;Qcf%hKocRrYY zn66T(>Zlf}uBc(I$*3i+ji`gv`P9MdZR=qT`VC`^%8eaO;!U+5xjz;)-)c^8A#I6m z#cF-gdfn#RcKpft({8&>`&x%d$3mxG=VX^g*GRWYcVCZuPiODF-nKr;zUF?h{>A~3 zf%-wA!MY*Aq1s`=;o1?Qk-Aag(T2~WpPR-c##+av$J-|!O!Q1DO%6^yni`wdo}QgC znt{z)&Th>;ojaKKoIhU(T0~up_(Jd{d5LBzcbR>;5+(#|S$VKBwEB2;Va;Og+q&EO z#m0+G+|A^#^j}N1__vz3<+nfY818I*bN+U|8@5Ndm$A>bU-Mn&`w(0YzH#7saD5nc zM0r&7L*PfpvHJ1SiNneH&jlGkM z&KqhC0II402gn9608|JOfDB5I8#V!<0{OwOyh0g5{hO=_Vf#xOBmj5`{e?)}a1W5* z-)zj!!2`(G|8hpof&6~Q9HhUdMi0tC{;P~4a+B|R0gwfdAds6K3EaUA85Q|vLqkD9 zMny*h2TyQxMaRI#M909wKtsdC#l*tK#=*fs$Hc|M#lZtb?3d_mR8m_wop4acMnf5Zy(=~7olM< zUxh~`yiQC?PI;4>mYbJfP*_x4@~*17rnauWp|Po>v#YzOx37O-d}4BHdS-TRer0uS zePi?M*7nZ9;n9!dlb?vwvm3p@_W4z<-!%JodJ%$pA%Sgxf_|eH1j*}0aY7W-TYP9l z_cYMWT!?S;KgS@sAOH44J0{~@O*pB!>lhXplfVk|!HsIaX!egO7W_|X_M2kA>opJH zAVa{$LnZ{Ifyc@FxcT#K4~z z_!9$vV&G2<{E2}-G4Lk_{=~qa82DGm0J}_oo-uP`Co=0*fzvE}|5{srp*$2tZ5^Tg`{E(a=TY0wb8 ztUUhPqA|5O2Fj2v+x8yse+u69Npj$ z##RY?h4|z^Hb(<4xMsrp|CjE+zSwk`>`>wYE)Q8th#ZObZ;k%f#{c7a`k{N_Wa9M0 zA7wUb?;tfAa^_$b_5Uw|n#sq9AQsA@j!0v2qz$Mf8Zy5G6zd2!Bz|6>+$>!$10j6h z_HnPzXc}s>l3TNq+kT?J0Q)(5i(nWoD>|(nrW}dZe_DTktjquF@{es5#ca>yrngl< zZUjdVTW$pKp9OiN34B^RWsFgS5%<$t83j3!qJmo}9%V)bizr}zng1;Bzrr;R89t54 zq+0GA5)HYhe;4tpE=!sBC`tK^20kgsk?6Ah2B8}$Ne1lkXSI3O7BpbXk?6p{n*hIr zn8s}tIhMvGuxoW=XN^z|Ys{eX0}|Ec#l|#$dy35Zn6g-ey)*55zp?jz<737W@w-S* z>3)EFjYdD+Czkf_^6lSM}iZ&-W-A)8McJAth1h`xv9wphMl|29p1>km68| z$uBSQ0CnIr{f`C|f@0P*AuI!S*9x$8<=erwEq#w`xfbD|^w6@~N|RO{>}zxwNRH2I z27E8)r;}k^;D8P~*R^=Qnw$145sB9N8C=3!(Td~!v6p* zHyNWE3$+qwo~%}tV1pS*H1q%vCkcrjB-1b{5Iyhz2Cw!pKXy2*=&Vc`bCbewncW0$ zARYIQzTf}7bnp0~$N9I0i*c}2xjP@_)_??dSKz6@4%>+fP==HxLo8Zx--swM1YjoW%7*;+SKysm%hPXAlpL;1x!%JV7e zAj<0g(Jl#6*07lru*XHqL2_sRhROUbjQ=vhqgX>r$XdIdYBWXq29<4xon1jgvmWw8 zL(b!G`qp3G7O;O*QuWy9ldq%L-3D34ZEvA|^$;J$MapYX(X*5YS4WRT1J?O4Z=^Mn{HNa%ky?lb&d<~4@ z8|f#r=t(f!!I$_FlTMmccidrHy@aodlns@Yq|-mZ)nZ$3f0HIYZZGlPJldNnCF)>0 zOyG$llIdWh)jNvl%55#Zd=qF*kLLXi){|0(7*t8oXq$ZE<>l9CK^U5}hE+<<*29RN zk@4Xx1@Z-A#gS}SLVjpxWlQH#J!c8Kc4Dnkx6r)iYlnXK)9mTgO`qqm3ldK4!YT1$ zwstN7Cpqu@ved%@U-YmxI}i!`mT!|WxC7ufAkILg z!ewro`S}+{U9s49TE2aO?CSljen*_wGCO!{6mE@EL7F7w=3PO2K~m3JtI{8~q%CNM z8Tv(f*%m}-u^otIkXlIuV=#n_mkJJeM|Ng-DD!7369s&i6n(^(w)`bwW7AOFedjeR za_oReCoWY4Zz=7mbsj7ES8_k;ZK2o}X&bc|^$T~nXiEZHxu6O;fr(*WFew}(9!&*f zIm2HHUinT4v0YEgI! zsV=2vp`j*W?y>SCrHXHg1KaM?CsI4Pt)$mLSRW$(8jz*Gdb}5im{RaMH@T2+sZFMA zR$@?!%cI!cTXV+f9T=prl7a<`>o)S_>*DDz;6J$rR*J5HK{Y;a*$fkd$u>AUSS0i> zQ$yE4$2Q#Q61SD|8fed_zc3E)Mf9E&H+@#wZX=LlT-2xT!#g*Y?Lw25Vj0NDPr&VS z?kvQ^s1Uv)uMd{qa)no3;g(+mh^B)NtrBbBiq$v=SXO_!ZhGxGYCB%4Dd4VHlcguR zG$2a2ae)UOr!TJo?1gsv zAEMs6VAS+AaA9l$U+${J^X@s_^JlEwN-H07Nj2aU-zxHuk-4Ky5ywApcVE(esB~>4 z@IGF_!vePQg_Cfe%$Qkk{Uoh~D89Y=G1qG#^~?~wb5VOyQKbLfYam2X8qs8eFuKCO z2HsY8njr3mTmzAnBO@*B%6kjV@4jz%5nw;NI}N^Oh&LC#S3zK=!jVAYh@IRiZ3$EV=`D|J5(+nKDjmM z9r3z6d6iLzGeGxU%id6Z^zNQ}MS#Vd4T_;$yDa|KA%4cTmvp1+vtXySo8^4dVQQy~ zd+1p;wA?p~#-O|gl z`eZFz&84)*MX7YEn*^+{oCVV+sk&Ah9YS6rLh((8kQS+ z2r}lXw}u>LVNaYRk1v>5?6Aw+)YzJ^V;j#>ofLP6ubOY(;sL$*^90g+46cUpk7wDd z8a$jN6V~qT2l!Vhmn`|Tdoe|2Bp8ZF%JBS=Q~gn%HacXlWfq)iu>dLgjfIHdZwS)= zRnwtwMuZhd27S2^vo-QoZp8LKI+*^K=?V!$)CwEE3MOfLI8)0z+vdJUdQ|gRqqogcV3(n9)I4JsjduSG-rxjW&!%U)~|37X_Dg9aUqZ>4HZWX)- zM^S@6++T`BIOS7V;)Lk!FpO_NYhF{s;Dn|G|^!iVFt#{8s%pzY{JoY*C9?Bn<(L+x?W9u5TDi zA#os|#JqZVCaQveSsAriUwZSdSNgkorEC(!N!STP zhD@6+kLH!ElNm{0D{Vo)3h0McJyRYS-NQcWCuyfrJQTU# z8>EQrUAa@Xc6$&J96NQyL0F2ipP00JUQzO+sV>FvF1)!-pnF45vJS8GMO~9;)9(4j z*^WQ-;N!{N87WHr9fRJm!bsiE4)oqwY6}c6DS0?hvF^Ly$v-%9(8z4PYmzq~_O{47 zLcE;&o>0kCha)el4_PeHA>2J*vUz!N$Q5_`IKf`sO7TJ~6SOGn~CP8I>^m@l3Dzer0E^#Uq~8Pz6rb%8p6Py{hWtHreA5x&y1v z`P-ZD#PzyFqK^x!ss+!!%q#||_)A^`!5;o@;PIvRSM>v&x|u8rxL#`m)w8b=`g>#) zFXAhc9IW4FW)E_vXcLOxUny9^_4AAd$E}4YqlGn@={$gu;@{e$S?sPKG~9ihU_`Ds{eQYPdqs7pjsmSO|TB5+G0<<8Y``wIL-eLXnx z1y^5zRe;WF{&7os?Zijjg~PHze(yy&gOd7TmdSzxLqijty4s(0{j(PZh|6mLMFSCd z&V{g+G+g_>$6SDLoc6!WWQC_Ix%s?|rtYhYIW5SBCQB|E2z^>eM^5|EHph5{p$&IB z2|`F)tY(U@zKpuCH!B+!-#$x@x1b9}?j(MbX)wRfp1)YtDP>d@EVSaiINd6FHMj&^ zMqhA|_wXP%SR;?sVoZpRkp^>Z$;r9!(j^CKywK*Ci2G~9Y*O6B%HK!8c;>vCUD~!r zU+l*rq!lJ4D07rnXi1;K2)2nZxHhY!&DGyG)Z4^Ru(&MJ^|PwDc;i6E>4huNa4H(g4}_Hl!nI9^y$gRnfG_32dtn`!x5;C#ScJq75u5A!{XE|j7sK;~ zh4mqC`98%pn4V|_O0dB@99 zH_Td3AE{2ox!qr-L%Q~5j0Aqg!j5m?E5S@_ZFSt%0`V{H{QEld-xz71zem$54P`~K zZp*vG>cK|7qvLGYDRCsQJ5XlaX+a%6NXC%^EDMp)K%z|S-z*u%W8$vAN8uSF$zE9$ zVe%s$r8{naV~MykpIz0X5v@2^s?1^dz0Iu@Gi}wG|La>}@+GeF*!E}CS6I3}taBCz z_+bpSTpQ(6J(<9Z`%OB|t>k%6SXft_s&*q`X6tz$K21abMl{tV`HFn&PT zUn`6T)v7V2e}g-Yr1*)ZBG6`jUXG_lajt)q^suexJL2}sPqDE?#wPO!$F)!O6Zo~N zJ}adK?;{6eh@RutM4Wt(caG}5Eq~`@r2pv0!=Z&wtrWfOWrCvWDc(vg{*$LQhdLA1 z^-{}_)i9T^gp)&UawzjPP3t1xx;W(yjP>kM3+rOJZXWsxH z?RBE`mNL}&ETK{+pRBZT4<&_5u-TT})`aC@A+UvY@I1;?cR;#Ec(?JO`U=y&?2%Kh z6YT|0!h-}_%{gvrwP3*}%~nr&_tW*I5T{{kfmxk5Zx<@z-%gI;JkbyM?2n8Qo`-7e zLQB0(Xz>WsY8Dwe$tU+H+R>3S0+6#2hTRgx+Ear@JMH4{5`7YDYAZ47nI>i)S=Mh# zYY#NcFs>^5zp15Ax=c&9FW+M+`!QiCJPUp@7j-3tc)8lWUt4q`(-3jFJH~yhks$w3 zPG$jC4weEC5*1aa)VJ_m`!pTK6JAd?Z8C3yeIYC0IrC0;&%pQ;Q$&X$`ap;R4aEZ% zVvK*mJqpeyK7Uyayr5FM2D)cvowH+>XDJ;d(46GU?clDD9F@N>Ij|0YH+HCKq8H8f zHoh#4&foAkKhW#Q&y0BU1GHQU zx|HMO^H@=<-nsO+w12(b=CtB?>utA>HqHz#Ifa&*LKC#1wkXZ>nNHEB{-%cR?OyFq z)6fB2yE-qW%J#gRm#R1iMv8HRYX~ofM=M)@rtS_TW_3s##+}yo&M7_4tJ!{#~QYA2a?Rks%DCB$~HY zlbcreD?n@e9-Cx%yebX&^h<*QB z3r_fjO5nC6DVMK>7F$s<7Bi8 zmVq7$no4WWN9+ZgRf>###JRbTJ)peth7TUPWvn^ty?^d4k!%r!YPse6aK0if*t^mL zUHqp?NzXT$B4;F7Vh8MAPWogStN<&nw5e<0h3lr`TBPR3G%dNqX_5sR7GC;~}o~C50Sk>;r5|LWuj|(potBhEiWUX&#QXvt7 zVk=S357#(aVdcwzit}>i+k>ZceW(0#p?pyHw*1?u3!|FzVLNGef5SPBT>4Xu#L4$ljMLbxX*3X4s|Lnr;AdqMoQOZUB90LSOI z$u?PjddJuzU7OU8Z+}pFq8{RW4H#lz@KfAc^wuzu4vCK+vER7{W@k>k4(K~g2-{v= zSkcyIP{q7C*|r^7wtrO!+F>$+17HB(L74gueG{Gzj8tc2M^J#KLqiCEw5 z_i-KvFeql}9GjG$OBv9F&I~bT=kVK=-H5 zMT2&_nqTsp#l9DdnG<5YM}2-sN{o(o57%g$EGeIKP%>JjF~u8t%0_o?ikgjDY>&6` zNjsPz#1>ATU8dJMnB2R}-W6Pqx?-Ggp;9Xl)AN3|X_y>g8&kk$^Xx;Oa@E4ZD9+ne zY!=E?DTiy|)kNUV{55cNEAZ#LAZhoKnG`~UCy^?8OMVG5;z?om=-VkyN8K9hDj)gj zt}MT3gZ;=9Ga=oMT*IGRy#`dr4fb#Od0DKvH#gOWcpou^p--&p=W8zxQmasy1KdIT z^zZ3010=n@PErt@g%|e`l$G79+4zV>>T)Vo_EfI#qtBT*qG;)e7?bZ3KN#GM zPK5Mbm>|~R$$|5opPm4Sjl%7-Z+5Ldq0!uWJDrwgMxrucGrd;>pa}4MSyIe|5Xsg@ym5Xh+r)AVX@=v9+A5^iaYdt*K?^OON7Mt>st(1N^%yNOL1s9tC zWm)!HAc7@(H&NO`dXBzxiF9&>01<|uwJ?EHkzHA-j`>$9W;aIPs{cZXaU~Cjxamp5 zHR!{aapddar7H@1qOj6RO6W4##e+JsR;?_>@q*~wXl|5u+Z=7_UQU#Ub81Pc zDcflW*%$%Cy7XjHRQ|#Zmrd>Jc|X1<#b%!EBSUV zS-kYr-UFU2b{z;=B4x-q|?!lBKVle=Mx_F@GVsc<=??N2XRz=u`7<#Pbl&WXAnynU;xB zuAPq*`Smdrb^AQ$Er-0LOYKj|wuDYL+eF%6c_wfL9z5^TmHKcAkMlcQx&}_s0v=)3 z@R4woz`3(B3=t}^ zevn9T>rs^&7;%t(()9A_M7OJYc_+8TsWy^SSN|<(PYiQFVoicd)u?#5{R{m__U>!= zQTH`aBi&)o!OHQ4cbF%Zr@=)LY5`rL)CJwc}O^B(mT zZX0};j`m|@mo%whX}SK-%e2j9xd?1;*BsVVC_6ytlsL$$Oa_bb7_Y8CJMpqd;qd))&Rn~8HV>_p-n3GMLXsEy*P(l z4M|CLcboE`XPWdYXb#9dt27}3Ik;pqaK(fel&a?+;A$4N9igR_Iod>H(s_JDU~xB# zFF1tt!?|5sbaOzr8OH$*n;9%|Mb)kLqpPEmv>;jLQe}GI)k}D5w6~RJiH>TIrhmza z63a}fs()-zAS`3epNrZ|SBnU&U0T<~Xd$OP!gxA1&Zg=29_L(Cjf-SdALyBRf<@toBPoYp9m0TIr zO}x;+@1@!b<%-*qRRM-kI{6>F7AP37<)HDl z=NG;2E7TQO8@Mru{!2L6(!5ObkjKMSG935S16Pd8Wx#?1{}Tc9Pv{nlNR?C4L*rR0 zn($mHnG%?Ked8x&`)^!R{^)9`%KjS6qZyG5<9$n##EfE7ZoTb^B*xw?$zB^783O|| zGyG4O;z>a^orjK=xk)umtYHwe!=9^V;JNb6poA4PAUo34Xg`7RTe`+L7|Ai>I5ILW zFc2mbx0k*y zu(wenaMj13D_>y)>WjLDsh{$-p;2}0!-@7_F>-~HS4S zKo>pmbVk1JVDwd#m+<4}kcZ(a5IfQ=R2kb;yX?;Dtt~51G~K6F&~4pQRd2 zNgT@45lx#Z9Li@kd8AgJDNVH|E)FT0?xKe1g=jubp(gD$|!1Py+z4kFsog+6J0r;rihE zUN*rMc3?ZN0BA9w^Cv{yvr8)qce>!=NvDtGW`rCbwzj*`xG|MDSUGK`7uEW`x%x~- z&$uOx;A*=dAve6N4-kUVzc0EVRxv(T>%ynBbA>L6PQxbiY9{VD+zz(+IBwN2KXPuN zko>(*Jc(<#Vs1%4C-#T#3DV`OW=7D)dVHC5!2|DIAA8AC(x_*zEqF4j?*cw@fx3Zk2i=Re z3b6XiUg;l~#WRQDCKId|R3_(gfgQ7;ZPk3yoAYtg=TLAld6al*JNd-3DzLCTbj}b_ z!M(=a<&z?8@B+%9C2we+Ll!+76p-_mX*}cEQ z9f?zVwKS-O#iqF4WmH1%kf*uTP?lL`kyVqh@K16}wcDvwQ=~@nuVnh~tRG9@q1V}^ z1UCy8`iqUZf0RTU=#_;pzjW80s9C@kA;Y#OLE(@^Aq@Hu6j5B9gQ^bvc-P9?`dTG| zlmAGi%ud#-inxvw$s0ck_MYqG>+BTl5E(j`f~&r4eCEsCy(`{1Z|*67|Mq+bI|sY( zo>D_k5j6_DIG?bhDgqNZ_2pBC{A{;oH~D}m zXLDTYs%}gUTL11CeTPmYikazo>kNT&ioDmB31xOwiUjP5&!fBW4K5@>D6aXd9iT^E z15V3Ch`_G2@~P?Y6^uLOKm1s3nZ4KFb*Nxr%K?nnxje5uH+SjQx~o%BTct;HbQC)#U(=cqG+FUfm3+(q!ci(8UG^gXWm zC%fX8iA~#Efrgfow>o_!+03b;pgo69mP=I^hLUSnm{WeVP2PQr^jx~TmeroufXSo> za#p^1Yi7hIXL4Xw-)jz`FI!%W&iDggDBLD(GGcOOk}ha2Gxo0V5WISF4h|(I-_RC! z4y!*rpF=amP`s0q%J;3oQ4^i+t&`*Dhccg{*xLIB>qVIyDE2;IA=6v~PZ3V><_-2? z@KjGeW*scmg7U^9_JRqm{K~Ixr=?MD%AzAKuX@bB_p_7-vK3$1tKIfx%WZ|7#{E~O zuzai0M>RE?P{~jKx4uqc*Rni%G|RDu&g_oNR#S)#X_~i-KL_Box<1oxZK7GC{ z4YZs6!7k{8?k%)5sBRyrP}9~7)lZ(w5K|(`0m#+0_7`QCN{oWh2Wl@yS7#!%ou}y3 z;>_>N2-uAX?XIqnt>qauwttZ%R=s<&jE_Jg&8eoDPMWALBL8_LkX3)B&neg>{^jvW z+<{a*{B1LR>Eb=N!fF=l1b#>V8q-0ti#1y-^g{0*mu_b*t@(g#smegyiWHoBVPS5@ z2)4=UzAhNY^M0n2%FRVjo0t(B)zzJW{5(>OFLjgJd!!qsRY%e`9kBOv?$*Wk?kP2j zbFwV$UH0n2bzv3d2W{PvMF=1NbUo*v^%eQOHk6WTKi|MYA8x#o1guRy(3Y9#H<>|`btoo2iHU@#fPTc!4I!*^ZvmHO$^fk zMTVfxNf<2Sat_{U(0IVpX@YMj))~%yjLSd2>4ili-mE<2EJDcy=RAwt`l-vAr$QQ9 z@W`Gz@7;O|S7KU}Mfp^vk1@-#PJ+q-KH1yo5;4-RBF#ddl|FvSUr0{&+P402?6nB` z(447X*dAmDwZYW_LjtSmdwJ;mRS=9YKV0hGY+~-hj|XHmKXrw@bYlfO7l#;yi5lu+ zl(L7sVT5@ku^j#lCub?q5^lNv!tmjd+5=*}@gKNUgHH22$J7mb4mpb}-rG(2`c45s7kVEjC-_)CBKX zkhtbsV@~W&U~I?@GE!KCQ}4IUm7WYEoX+*VZPZ#3cUPmiPtH!dmIrhDJ&5eiTsdDY z8qK5g(DhTQ*I^M2Xs7DyCwOtFyUe_WW!KSP5Xhd`TvlFD&LZruNUuWSA9uwve6vp~ zJNK}a#{|Wr<5lVBHnMk}n$V8w-@y3~_B`GvCsmX9b->|#H`#ED6MI363fSk6LI#Lh-U|o`4Hn!QT5}%M>#$Wgy*z_Je zd)L|SYhf%0ZF+Du_R&+%flH!qEw20P;vk9?R9^35c<-=;5f+FL%sUA@aJ@i5q@F8+ zULTTSAIq*L#J<^$-*d9Pl|WwAI+@qdkdd2j&jbSCfh-ZqgBX5_R2&z6{K>P!?{lDU z4Cwr4-S*<83?lphzgwEvBlUYhOT)BB|9Zl`297e`vINDk2PIUQVG4sa(Q(vv2ibMK zo!Rr^y<$txS$D+`(A%@{Cdy%{9*Tm`BqV+kB^$?7wX_UWq2eH+yee5#QEGXhZ-OFi zB^~3d*6oCf_;?b3nN!iMi~bVUVE-Lwdx<_$bz*-bYbw{$r+huiC|{G*d&w5_a_N4L z(Ta)YUg6gX!NZpqWZzFfZWji6fd-pjXdPI`7e5Q>^38hVFS@e%QtxW8h5YB&-G0H0 z9CGMhUV{Cn_=^n(nlKrh@Z)CzLV@kWXEF8XMptv$)#ST=FzR9R4mFA$7w^|b$y~R^ zzuD{Uv-)A%!!(*V&uIw0Jst|`B?zBNoh{+5!lwhD=p5Ld@9VV33YT7q?dF-+UBkV6 z`O@F%ZTl+rFm`7ee?b@7PtQ&J2eFSEyfoO99G2vSqV8_W=LfM=R$bu-wwGpC(RWg3 zYF}vNB7Ux_*3P|>Ie8%R``x~i|bcIcNMF}e2wX_eKE^`P`q{W*XS%0SUQt4O(HDU$f9C98V^_aj7 zBrVo{#@_MA1_^sT!o_QCnT)yn%a@b4t0EX5R21*g+rFsnvhwe-ysbZkLQG@Me;AYv zMbJ(3&~d>lUbFUCPdxQds0*#Nci`=V8yjJy4BuK8?_x1Nw~#nBMPH92-G3Yzv$<4I zxqt{jno&C$)~CE3Hh@V`k=45|*OsJDv&K)P#tN9Hu6&_gFlcLSdpIA_Y>)16&5&6 z!cm1M;PR|I+e~5YswBQhCOe`RQf2x462DkLT)B?+flFDac1ChnB+e^CPZIO>-B@t0 z8Qcy|ICM49llO3Ia=s|l3d1TP*nQ{O@ ze=gvicanT??|?jpn`cITvSPDwQZT)wd7)h1_%!jY>l1JeU(Jcsh^J(2{BmCv`#~*y z`Idfot*sk=x;t;5bTkMPHHA>>L7_Fyy-;C)}PRE0DUn@Z$51L^|Ram z&9eabnPGAMYC!`IBQ2Ce`F_y!4eK3XSB_WGuXVVw*WTDBf!~5=C`$%0t=$?r0|lDz z<6xU|v4}BEV;Px-g4~EP&YK1Bzq8=~?wi<`l$Qc?JpQsi_GUJC68-zkdU+4y(DYv+ zGd(}P-R!p7yVtgojRF11>o5KKJW$v8Kwg8w;Qo$Fq%xG9*;;^QRe zVOB%_uAa=K-TBSHn{GY9rZj2Y`(h`zC^(P(n|(pOPS{tZTg98MoMQx+>y zmIGyX-?tKM)O;O&QnUWiw)K+wyzH53AZRuq{-AXYgnNZnY&>;OIJKZvaE@M({#r!n zcmB+KdVj<=ry9Oj-4_)p;5nZ1Dn4j{PP36#{vM4hzVZPUC+y&g>%iLd^1?mu3YEg? zilPXyJ>EO9;xFCDn~g^*eITlGVU+BmooVi?zO$+lo*7#=KgrqLshwo!V&uE9=j?xnn&eW*id9WV*B$esWB$8=%?C zH>E>*PP$8)WOeNO=U)4jOKoQBNx}qKJB@tRa!e0v<~8JvzRr*=3|f65h1gY``yE!3 zR+>e&<*)68ovcXan==zy78E=AhIv!v$Y1qnslW7^n)}FIJ}1sxI>(vu{nPRbVQFfA zKe|W@mlKf}{6`edcYzsA@@7Ti{@9ogxd%(qLc`jRQAFi^;f44s>p+|r< z=ad9MDyAh3v_g2TJi&fp+esMx!p?$cc(in**mf%nTD=lEn7`tN6s zX~i|B7{?i76@w{@^q)``;B@F)7^gcd5>aF%t(DLHS(CQbe9 zOAX&7+j5(Ga2)hSVKwn5V8z`-36YtqEa~xwTown=UrlCFzScpYu9ow3CBL`W!$hv} z67umO^go9p4Wg})r2BUnuXu4Q2)5hO!&RMLWMNzu{J@WB!@iKd1~6h@{h+T2%-tHQ zo45L&Y3C5=1FgyTW3*xHJFMCtI`6OK8k2@aOHm@E&lSDhdar>hujq@%@YNZ$)?f#d z`(aSWy;^(4g8KNfX$yYkcx6+I+e0&0-94oDQKhXlmtE5}5c9?xZsEq#;y5X`=}+7g|g%*GN8x*gOr`#Epw3c0*X{A%#)sokC~}4{7RyNr?SvB zdc;%kix=@gwMGO~2(*Wu3n9q9sZBZNO;0zMP@P8P^{rYNpNzu5~ zSfn_16q|DBJ*~tbPf?fRLY%Lrgw^nn2J~M8ME;V6_6FtUp_`>H%wLlVW$z5cI-mcT zH2zRZE}kk+vi^MdS*aW+e_~R(X-nMff-GKvo0GZInmO8IH#SWZerc??N_`p`u4)YU zeQL8}BH-Bj9UKju1gykd3kBWe`?8mWi&Vtt!xK?^s=(?$h1&YF!Sa z52Z&zu`u4{Y-`@(<1%$oFtMw;ycL`99{MdWj~<_H|5T7zeg`R*>g-i}Hc^0xcbg_6 zdNq~~@p3INJl%Mbd%rwqLgLkn`n&Qm$!|54E(X69CTI~pIF(6;hu%tMZSl84928v8 zBivTY0}jWJ^kX|cg?*!sD-ugF<{XGsnN*#3g%yc}PLa9s(b_}L3jA;gR=h=vyGw9)3KVxJ z9y~xu&)0p<-gmEi&RT1qd(Xc6{t@9xo_yaI-n?_nF~*$p;Bw8^9fI3gsK^x3%$t|Y zQ@v#_fZtsyZXP!H!5ZsnoN(pq%}uh``Vb@F9CvpwozdhdC=#}+NgFdG!;DJgw&gak zp15!Dt7PiEf!<=yTMxsfS;-I_vDZe)N}OODA2Jj;%Wo7gJxf1ZL4-JZ(TWA{n^x8h z$W^LT`%Eft78WV9B*rdqEF2C;q@>RlUb6TV%aHhxBBS>KZu+~l6cXAwrxSeUDVkX| z`<&%8O)mDT3cQq}9{bhWjOU=5^1W?ART^Z0tqi@yt zx{d=O#hqCLp?2b|jW$_$>oYRm+aPuX1=bUu59u-VAnrI{6ZP#zp}6*f@kyhd-uh{# z8`53I@Ii<>oX$*E(`Y!BtmnJJc9ypa2hqskxBxlH!eG$xeQ?X=$9T_Hsw@L~~s9 zzA=-d2v0xE$~5aX#WwL7>-51+vBoU#;^esA7uk+EpiY1+vZB-O&I>{=aeR8UPqAOA zvB&|&9Rz6CW!VTz=+7))ebYj;Q&r;9;JX{qFXo&)oAC@M@lx5?(obI5*T*zkG60&HRCMYAFLvZLqEmU+1F}@-vwb`k&1fOjwFD-I@`P>f%$v zj7w%&KFBbZ^4TQZlTh4;+acgTetj=XNB@CUI1{exJWRyG;ps|hTUiTBVzKv3IwZdi z_T_9OIPk1swM%n4ODF_T;(>bS&5XV}tUMw-9r_rNuc8r-@vyw)MzruZX&f|ipYlyi z7`;5?;9-V)Z9CCB_2g#Av14fxE3&TNsg>^}A*+z`kt$7tO@2n8ReXp@a#Vbj?4wjT z`MF9knvlYxRSGV?DIxL}>)hX<29j~$17lmMJrkUoIvaoWw+(*=N;%i7f#9aO#5LQ5 zAEcNo8qcZh6u{b-%x_>l@Rqu%+sR3Bcs(y;Ahh!4h}-4jiw8&V!K)<|k4fk= zb!(f6XYXUaiV!Ck>hHOe8vUAiYBV>}l$}CzH@5GYmVY~wQU>BhM7B1Ung>L4G&bJQ zAR-}!S~E-u9#d+=rd%GZrUd-aT?FK`|J`^E^l7OMzqGb3mHOjK`SQ1EuFfz5;N|SW z`*uny{3}Isi^4J=#KcZtY{@qfkRE)Ip|d^})Kh|!M-DjqS%%dkcvcqJr%$;OAqw5L zoHEUYyB96r?;Z?wftX18i2ajhXf2=m{~7{1?r*#KpU=8XKQ|9cWb04Q_|g<4^rawjLHHa+BWc zZ1&2*Y-P22OjgOHq3UitcZ?|!4U-t!pQUw9Se}s^@cn9HQ0F7V z;F%AtsH$oXru#xR@}wTFu;}riyY{X)cYs(FBwuDW*ROY(_$;|37ec~wGEebta&h!g zY!V({(8W_$=Un%UFTE5;8;p3zqF#zRX^-WIhb7@t`BNO6VD$aOOYKJK=9#XcB%ah< z8l{|sqrPg$aDTV&Or6H6B==d9+&E9OL|(7u>YlQ}eQ}q~b_cRM#~ zoc0Jx;Tt<0!yVz;8M4khg`fWVUUB7hMWHOjY2ogyIP9%OaV?{@%1&rb(NZJLm62E@ zf57}3)H29jY9D&0GHc^w0q!C^eVV_&4NWpwV%%uqr;~E4GvadwhBUITLPF+JjZxrN z6_rg^69P}w55o$|?RXT`<(F7+MtS3gUETG4Xw4Y4Q}2o%Q(23?_NQ_Yu=qf{nfxVV zBW@`#tMw8u$;9~=m#4l!K7(t7rgmcsnL*1E`cdt2qh}f~;ydUspsX<}a4+;@p10N0P*KxKI{Y6Ho7?AAjSW zU*+xz;!}&atleahK8{VpkYjDaDq#D-zmd@YM#R%$b$lAJ`dEhHMhE>yruPC;?KPue z2rE2gvR{^RSdy5}5b~cud!f4X{6?9ueJqc$d&;4UR!g{JWzDZ(L~xcx6$fkkOEMmS zJ)a&KyrSy-;Fyo7hC-Sk}&+wEPIk8I%_kr&(+ zx>IU2zd<^-r(zf)t=l4Lmg5i%esZplFfx8-Z*4hnMn@a-sJ6bAlX3Xf+t%jl4Ez_| ztoD&SW@eHt&8ab*ksG&yU*%5#dlC)>aQXF=d zDf;B3jo%<@#7ZrqCgcEb^$S1R_eKs? zrQA5im$)>kf;=l(Z7SNRm$$aGm?&8ZwptS%E1)8C&~R>5dTKo@(+XUDIcz1}?+KFX zz|jwkB2k`kE5k+Iyr~1KMWRl)t?lj55-^L}Y5a>&mQS*_caKE^y0?@@SvxieLE+GN zFtVsw1@MOpVIGvB)lai z4#9pTMYxi3izyo(MTrh#@PTZPkn@HmKgv%GrE|ly&eY}42CT!2pv;7VU$m;5qH``5 z@+X&9FAufm2~)SCIOE(3?QU75K@h);Gpy5)1M?dJ8yExZ!|3cxm%3HvTq|Is@h!HI z?CzZ>BPloZqCC-+85J;#yXBe_`b;SEMtn}yFFLCd?Xs>Sw2BC*i6-@&*xuI{^*0sG zhB<9EXUH}SRKsPua8mCTJARFe0I6WxHL(QTmcwlM2{AoT9EWC;V=m~`j%}{KT@vO= z66ST)E88u3`V2_&hTv6c6IH=J$vQOngEZjR!(l-!7y2mQ=A46TtjkAuTZgf~tZPBi zdF)IV-cbHxy_fZBBKxApFs`~7+JT&vqRZc zD8bIsJ>+>_+^VudvFjc+y51zhF|dF&Ly;A(3kU8LpR`8bCL>JCP`b1qfb70oz|A&S zrB<0+*n!Vrs!wV|e4wlIktt!@Aw4ZtZUx&~d(0-@1lA z@LbZ9N3(7^BwF)X^GZ-D^p4CUDvJ~AHp2!67z79C>_n+c4i3iOhjGL(zYcp!q3H+= z2EoP_21&A|2nCFShafB#w}k* zmiH4755@ow%74Kmy%isgo3|*E_LPI3|B3-yJ~7f9|}K0-R$ z4z;pkmMg`%-b_$C1neWUb5hHn`ydJY~H(}=)&`7kk;k0r~98(3Ka46{(`$~16 z)b}X4K&m46*=%iH;j_EGPmDTfMyy%Cn^;gz#PvNbwN?8ZWZhsdwv3^gf0{<;ASaNu9dN*=M6G?ONj_ zDR1Y@BFC9vT|*-NlI;=+UN5hbd%4pp_86bqrES?Y*c^Y)635J84Sus?u^wIi67PN) zTV35}SAy80cOVc$|E?lyT-VIxG9K*RcRtte{H=n9#+~k?^*gmk)VduFYl@n+$X#-D zkOE%mM{iiLAC9M32dRD}cBiDUx}G>3f1_5fl;z!gpx0;;E92){Ib2DC6Twkj@v?9h zdJCt&E^LfXwCM9SmZp}#<}Gt}ic(MFT3QGl{+gwHxx~buJ8Ji=E1N;s_|m}DTEtG{ z=ve;KZ;+(}jnC_r+UQqfWvg|KvdIp7Yqs7{&akx?ET8C20!_pOXdcn)RCx+v@K*!v z_`CQ?EsuP&W*~vPRAwizHpL;oudt7;X;}!PK55+5p~A_Yg7@sVTkDLO7QE02RmEjZ zKUfMXKh3mdLF9XHztQapVhL`U=_7Re#fJ%5cyWJ}#QFqMFn>7f?IDn;R-Y00SsiOW z0dSqLvtARC>+B^PtNHhH@Q6ZR3Kv|2DMD?V?9ZlJpxQpfQct_Kw0vv%iu3tS?9d4p z#>&qD(q>$|<6nPC4P~TgT4a8>9WB!XPY;E3161o!0rLsTz%DkQ0!=$jX=2!B_lo0B zl%!{hTAH^aCx1hp!>fk|_xU~wIr&anWAKu}`)J&n;7t8eu{py?5#gq zYE8MxT83$EqPV3aiSNZmAp+96&rwof|7bLHLkl2utkucu4LRV3BRQUgWy>8EcK zQ+7C<`@LrOs>Uv#Hbc`_cDbsL6X>AB>})`yJ+OnG4(Xvi+NWDjXMtGn)4iTzRmY39 zbsc=n_;ui~W>`zgNf{}MT_5QY2QZ}Wv4z%9>%~pq%4r4Z#rnlMPnw?15lOR;Gdpai zsNU7XkYn7zzT?_BlY*@5o?y>c(vRDSHW5n^X(Y)C(%2a|ei~g&@@Z8M6$jbNxZHTN zQnvy<44Pbc`Iqxjmk&=esRXJmvcITngm@My`iXv=uQT%Yp+X|oEkk7eTzl05K5oWtTw08rh5FW#Hh&4q>QN*S`#pfT!LsFC+=QLm`d8JrbYnrQp5RTXcrzw92W3@OwF!#yYA?h#eF zu;8kIKBqabU=<*!d}q?CeWi=rHX;xJ<4piRkme4Wgm?DV6spf0ZO-2`Ugu#}A)isZ z+LuO_d-yw-&|@@1>pqgHYY}_RFJ;CLdy3bMI_~!UH*R1YI9@UxXa^n{*=9x<5w=)1 z%s^$>g51idI0`8}Earban)Ix>ggi9nMJMN7GcuXaGtFqq+XlCA5^^3e&7tf?oO70t z6Gb6wtwUz(>v#65s;dE>I3Fn-raa|$us#hfN{f0Z|D3Yp^W0OOT0DILh14bcdJ!Vxt|K{Di57YJ+2>@@QN;(g`2C4&siEqKFLUN0UtXW?8xRqaLV7tl5wZOOF zyl*&_N`u#jm%cB<5&>6;>IHyhXaH9T!yhcuG_dbTq3Nv13c}OBJPKwi7cDBA4UZ0; znfS%eFg;xrC0@$t`CMJkLQ(0Pgd2@UWJCK1_w?!r=`gCYC_BMqQQ7`XExI^8t2yED zAj^{2sf)(%m5OGs_S@D$_gm)D<z5FPiASoq zMtyHpiE*7B9mU13rB-13{N~?oL$py%{PSgTAC_mW$md@!i}ee+Y@2c@(CKi!Sjx}* zTD>Sq#H3nRJ+Y?1TVkqf5{gSH{1(t(KY<8R5G9`SZOuuK(gY+vXFRFInA=z{?R>Xc zti8--z1&otMw5V#!Ggo4T;i|4Px{uR8awa!-u_6LJ1PKh*unOe$^R(E!Ztfl$yc09 zHJp-p0vH`M|Bdm|1gy*kfR!1j{{KUKgT$Ff=|i1PlX1uE;|}U}B7TF?v?**x6b$wZ<5AfG(s|L8Z z!LoYHep!bquqZP#iygd4gm+t5dM*zz5KjPfY@=k|YRZ)* z=054o{T&2S-Qnk}9%Fn-i|k_0IE9+bF2;wHbMfvxDd#Q4Yg77)8>II(1Y4Ry>u06P z1?m+wPM0qLA}}xs{p<#)-;s9Xy^kTlcUc>@y$C}=ojg{5H9z~Vzeoh_#--A#vz^bc zA=Q(XldDY|Ix!?QXC0~>d=%-=RPh@`A;bJ@`7VYQ71#|hc@D&>tIPMTbv`Z?xIc!& zD)r{NCLx?rY?gN4i|4hJMen_ZP@G2ZXRkwl?qLY*))p<-j`ojpd{~Yz!-q?fIY@(% zbY1$JIkhPtD)p`P8Pc1b2gyHNp5%B)Ry#gxhA()%G}l|#Ibktzpq)luqG^+FAU@Bg z&VVopw1sL{VbPsDdek_e;zVzLup*_bdL=IPqo*bjV|ZtoTs%#@ zmMI3x-tfv{8Br2;`p2f1dZA1ltQj&Po6#t)`~$aR%ZpbiOT1`=CI3B?r6$rhUSoF8 zL;qJyP4JH0w;K38mWE*MQ??YX#o9M^>h4jHRgVMfV;F`2D6Y7xiK@t5K-;4h*NL8c zw$821%l$ZNej{y^yBkh0r|dYcx0|IMppde54?WAf=97H)L!WNlB;%hj>F=Wp+^^`P zX;3;OAi3HDfB8X+9H}EtXLI$zf)9o zesgi}Hd};bZt&tYQYRX%mp{LgQrcw4W&8p=DSRYg&A^&LP_Xa-%Q5YMq|U|E35^7f z)#0~5wOk|CV*vSts9GE;pp47jtY&eR>7x+4@WC+TJP7Na7bi}P@Ai0_4F zcj!G~5vvJr2oDd@ z%?e}XJ87cF(V~Ydg0QBu%8J3Jkl7x9$3Yu&paOGqQqsFB7Ods;C^f^dX&c`Z=6u?y zxLFq5SHF{i=<*TjoXrnynE$*hq3V#5t7w$d)WmK%abLDlTN$!LzbDp)G}h1i8gF{o zK`@6%z3Sh+{8;s~mDGp75(Qq7dPXA441GkEu|9ZjM!jw+_7fVgCvtu7?{@v62K_~O zALkWs=VZs~*KAdMIZjtaJ(er*!_cCL63O%2VF}Us!=+VVb5;BFucE>K>hZ0r8!=C+}r$ECQ?*Ak8u>VmF_8-0Pk1Mdv(RyV= z`z#~z&zZgbNALbW{#w}eH9%C@rYl+fAu4hc^$Y=`LQwAu5(I)nl-63s(6DhoykD*E z)v|}#Vh^m5LjfLrzNG`I+PvcleAw?U*uwzhiN^uU+Hx3`7-yl)W#YAArIKldf0t8^ zs%PYoO{*gaD`^y_kvS;7*By}`klHCV)iu8oS;Dw*vh%AH4gYZyMYSRgBV*IkCx<1U z{{|84?d~^UGd3LyVF>Ve^wUVu%#*wuHJVSnY7|mkBOq9061X7j0#^7(Li6VQHRyrz zI#XDR-C^b$-Mp@A(QSr#WJ@8BsfjkVI}!WR^JSmY*vYUp>1V5Z%apdYDGnheOS{+C zTEW!upNvA=HjsD)tRM+if@{-N1RCHHAq~i!hdvrVM8rkBT0H_ftUkLBJnx`1vpbX+ z#I9qv<-MU&@N@{xhb@bmmTrO9!}oJ4#wBo9$r2It82> z5Ca|A6Hxye%uOq^R6qCg-HSd3%vY4j%`;D(Ria-_NUhHm(3-MYw#LLQMMl>FxSsNd zyOo?$n%*Gs(m8~}Dl~hVHUVtgMUj?w5hP|+rYD^0r^wn=wO+MMxu===L<5ot9u{%G zC39Dvs=Z;^hJ|F76DOHxWuujTlx|NcSC~(3Xw}JL9uUSJn)WDmocDr*g8C2^WB;#Rk_y_&9;|Q$w-O+mhU$Hv!a7wnolzGGNwP${1`w=vSD$lm91nuw@Qjfy%s_?9w$@^EDU?v{?$ zdx;TE(L3;;*dmuA+S9_RzA(kcimA1SbH*J=2n{Pf>5n2ldFcRoiYs{vlvL+}ysAd_ zi?=0~)*97RV)7ihmZjH5dFL%DH$&lF((~+Trs_pSs=K=$adifwht=~2M!dY<7zdPP z*CmE<=D-bde^J>SW`;2nlJn9SLwY_6c%c#V;!yxT#TlYA8pVN|>9DHo>BqTqoio=3 zW{vy#iZp8TIdx2tMCQ$; zdOb_7XGjlp7LcB>G5ze}WYcRSr<=-6V%CqzTyAQ43bo3`Kyvp+@aKMFyt0#yM?brC zX?>&Gr|M=q8uzHlwX}VfB&2xuk*VZIH?o(hC!VX{Tc^_RK-TuAZnw_8$w=*Dreirw z5(;3ad!5B7m7E6(2XvjB5J_BZmJeaY?TVnMG&jZhEv{}?xmmWC7@8m2N1YyZAARTU zP}*9jaFUY-U;`Paj`ix%Tk$fEva4X8?=64RA1^&TX2#R$rHl0%9Wwwgo}I9J;wO`3>@C z*_Zssn=C573tS7h7l>@m%_kl9g+FWFGpVkOjEz!Uc+);F?ca{=->dHF3tJZp_jy}q zUFMl&SRah4$iVh~WDDzg!2|BlPuIn){}!G^=JQa)v)Y#^FVedU|a4fW3X z_yv5gimOCg#Y)!87&4j#4VP&+h8>ixfC+}i^^KQo1$((=nE@M0fZe*I7XeGDJPu9O z52s--oi0F6A!_%RD`d$Pj5giH2?`}x;UMf1w-mActMy0^^{9!#j+9+xKKGjL?0LDT_*;rE zv(66XR6!eI=v0QaZQ5g{1o}IU5iRp-dGj-`<6Cw(-JPtLZJE4+R*32P<_f}WRBS)S zDOyi2?dzYb7`X5Qb#QKgm-!mLh&t75{9*p)%4UPdaf3s~(B{q)cRqK2xS+EwK?_h{ z4@N>VNq1A3dh~mebGlivE?$;+2s_jJ9n5T~3eK9l)6kpIM|+?&ZUj(})d$IEw(>vh zHxtaMKK>PD{cS!VOYp&6tl<4m_~QNIhN>zn%M?zkY-=vBcQ|c1r+g+q!g#Mz=hrsR z-MIyhshInI6^e6f+R9RM%laG)B;}4K`%(DyWbmMZTZCgGHHhI-kB5PG8v3g>F^`n*U z)b~~Znv%4dWDf~@EBK#?p(zY5giyXWvM5{GW0Ia{&|X^OZ#lg(3Ar-S(k^qo!hL3X zPpZ;BzbhdbdT*5zW2fy*Da5n|4U)z?w#DG@90x<0(AdqG@yW9n?8)w7cl9Iv^ zJs~P+fBeain}b~jXZ2?*7eE|{VxF*vVW<~!7h|*+J?I6M&AIG30uGo>@Y zd^um(A*=Pkc(7^g_7IsFcovQ2Y11o*U3JS@>wqlqfDQbMNHG!??Ye8DtPtVj2<0mzuDT%4>IlYo*f-wWHg=T_dyssp~_uEFnXv9T86Z>sR0;$g)c#?QlNb!mFmm!s-y$&hE01M|9BX zv2&PNz`ZdVLZ?se@sSl52>)g}Q{-s}t$XNdAG(QG5GO#oFxZFK=;G7;ia`O8`In6I z4!^$Tdy}i+_7(}Z`}ayjTC@c`zPC<46nH;OtJsXI9$Q)P22_HeheO7IFx+~7@OY-@ zubb&+QwRm0gw?(YER&V!hgN?0@NQz?q;}FIY-w(hk|QhfWsX%g zV9x|3VNl-ANAuRuJOe#zH6F^^Se~an2=_nqLgs z#jHMe)>VrEm|!h~v)f6IeQw(QvmbQ{{Al+Y@yC^O6SwtGb=XvP8#htaDGC|NKXZ?^ zfZX%HY$D!2Ja+!$_HNNI>ysvbHuA7p@|qv%Q*FQ{{&$l9?+E1oJ7W6Zf3Ge^66K$b z79OM1IPOO4`@$Q07&-+!?8Cl)tzygvF=9Q#Vxd=k5=9(ewD#oNy`l!i9JoR4Qm9vz zlCaHtc3RD>P4+BBqqu(B389tuz&ifbAtG*RZOc_Wz$E1z3T26 zBqDXqVYQ}=e*yKXQ^6#wVs^rCP)lXDn{C=b^Earo(qQRA1Cj zai6scuDxBk5r)~_ol%j{$>;?mn^rLXNDn!0F!t;fqhF_!bcC^Oo>rTsDJngi;JzrR zzceFk1#NJnf?{cFyzNO^Yq2gqI#8W8v84;+=d=du0tPjzDU=~ou zwZX@LaKGj1)fuI2e8{>-&^gKdWt&#UX&yJ-v)gCq5_Sjd z@(mBN93=iKH-CmpXO9nBYo3Lze9a)V0PVCsc?;37!^OkMeeZRum5uxE>%xFD>Oti? zsq<^yEw9_e(^hzhzO^hUZnM&6F5kXjuXMW+C_+6C3HbZJkAGM8`9FS#PBM>g)1>j& zY}o?Uk)_eJETLG{pY&y(vm}MslJ;LI_P&b3anc{%f4JhSp$_-vtXY&~@0s`Z&aql= z!w%%JN5^L9o~S;3idQoZ<^JL*Mq<&pR!*LCb0Q0FVLjONW;}ctdg?V6cx0<`5W|d7v=)= zp|8937)AUNgoRu&QULY#*GS9TyJ!IG`aggjr+`w=oK+N3nPMb%L;71LsVg~ICEX7fMC640Df4J-ZZ8W&byZ{)0fMPV) zZLF(awy*w7PX^b+h1n;8-f>GedpqgnB39li%Va`13^!|=j_y4kO7~!>u;h%%7r@C ztzVJ;1;4abB1#gBe=VT-vB3`Z7Qw>$|ym2CD=hB=$4Y#$Ms69YW?9E;Wk*h&F z+0RzD8khHdk@H=&Idw@G%IYY+iUzyVI#f8@e+*A2(?C9#4;pExt(%fv8K;4q#` znB~_e(cfs*{^`z+6Kci7^6akoFf~UE!4<_170a4`l}+q;QUrCYZ46fdG`{zu2tbak2x&1~&ZL$uVwm&8 zzJ!TLlv8tQuG`wA!hF1h=aqd%V_lC>>@#7xh)_3Q(@vxT+;vZ8XzyB?#`S zsJb`jT%{+RsqEKG&+mB88myll@($HUJq;GjZ(@LJ>-a)F=5AaUqlp?5zvSa)LcQ@_ zShfPfIEw!1k0P&sE&wGQF<-hiU#6f{p5H}j02CKYrMMmAlyQv$W zb$<>;9V>j=Pu+vIqQ;#a-K<7+g%R{$L@*=iuUEE-rYNUgg zSRdq2Z461=ruT_H&Hj-;`dfyR$3xGSh$y4LoGH4p#SV1Q&6IgnO=((jk}Oc{=rJe` zV*@fV;^Qx&`RdE`CBr*7lipG`*G=^N9S(t#mpzeA`^{AL5#=1-;mKceDBI4K+87l_ z$nU7)_h7N?Eq_8(PDdkUy!aDW&+RN^1$-+N(BH@{X*8pqu-W{MSugtTZZ{xx{Dq@K zB*}JK@yi1tmD>E9y>O!_EQ|NbLhrB=Sy!-rYPY~%c>1nj3_3{j!+Y2asB5-n5=m}C z=N-1-TdhS&rYH71$MFVe?Q3@XsGA^JUti*6bCHo;VeN~HUlW5o zrKHTVG(1C|fD4g5*M;dJMigbcN)#K{(-^O2DCiJ20$ws4?KJ&h8iOzQ_Tcj{N@ev;8@qb}2lmR27?JxR@X(;>Ua zrz~yWI^Q%dJjxRHH1I!OP?qr)SKAW>oaV$cuz=l<9tTu6HVpPlH!n~M6SDThOlVb; z`a~K-qS6jFEIk>M>I!xiciej-r}3vPL!)ei)o_LG^q-qw0B`ZRh)bt2E#f$;#Wv5= zV%lK-X69rySYR-71CYg3f!lM6bB{Klq_ner!n$#}%S-=8`!BD2h}D|6!u`i|tart} z3>Z9)|3pn_Ndq24eZV9498YeRRWYHmn98|mO4W9DaJiWKSsdLNL6%L# zIyJH6;=XvHI=@4HVMLPI%yJ`y+Uu1Rv|*%2o=?sW2GYkVGt{NzzQTRM&1wGFpAbvq zC(({UIYcpP& zpH13aOMm&{TStUWzuCd&f$-tz$m%-oyP{JF2_B?!OT7+NIy0KI;b(@+<)m!>~m`Q=F(2ctv7UvykF#^{71knh zkogQI^@T;>RO3FcGg_po*e5kkm)I8BJQJ4ri6ZeO{)W`(aMo2J|J5j4ESv4vCi(jN zUhJZtp2GeMyj-`0b#Qs~Ul!A?cf)OSLp2nBuvdTn`l#|j9NDaN`lR?eJyx%}ZwjKS zo?jr!HfF`iNqZ~z{*?%M%FSd2db8_`y#KJ1CX?deVav=4fO~xf)_{Qj|Dp+t7Rdut z{u^Whd;wLaRzAtQ*rm+7*zq!AgN$4#2EQXN+Ej0_sWf(T7GB_6pBQvtCU^%kXHl+_wz2NWYJ>d1!Dy9AEwD`@jWWZ51gt9kd#*Mo)etvpY^HbF<`E5AsoUh(maV^nn1J&} z@0I+}VuM7TVS(+y)K&2X*bBKAXr)YijJoDhsp-qDDLFQ<^xkpHEv;iNFAr3%2%dQp z%x!CHHO-L~dWTgZo1ifJPYx{kJ~vm3ul+4q*9z8;84aZO>a)up2;++^D>y5bf8yF| z@Mdk0!UBrP^8`e1R({tX4o-XO$)jo;8cat*C0WA?H!|=;g!>iWdOQrx{*=u%m6Lss zyKa^l(XkvXnc2Ne{VEHk`MQoQyKiE+`PCh+jp~T7?ib>3oi3B*KVB!TIP9C|`G_@n z{{}G(Td&demrM&du%*=0RGI6A9re58I5Oe4OBj0|VTjRG*m;3-YptF@@qZ&lKN(+{JVNfv9~5k7PVIV)$#}`y=1^&wSO9W!557)l*BUPkw{a z>zleh7~83P`n0N2`_>vxoVFfAvQ#gY-2vQGHXl9o@?ZVRIe5(3`7EYk6KTb5Zscl4 zpnsM^K0OzX&o9U+l>;1d>i-mh{cURU|1D|tm7bR4D?VU5c{P`Aq?ROD4^&}j0%!gA zTipLYdW|AzeKfQYuaiN<9_>}AzcVi5Q{$|!SFXv#CxWw#Gb8n)O+uVvG^~59Z6d)( zMV7#6YCwfG$$b%il`{5CheAy~n~L)pT&3<;V)0@r_Im!o!z?u6q^3>&L1RmYSK^$H zL(PWax=Zc7NS6$SZ9LG*dYj)<%;Lx1@cpIU)%4~irdrEbvnD~FB#oXITG*Ri?04as zRoHd5gbYgrhZ7Cf_!g#c#r`LVG}LnMf+(#}hvSCV)>KPzLN?Xy&l)e0(#mQLo$B!$ zJVKO~epM8UxDVjP%e1~3MXRfh8z;H%QM$n%V#GX?*3NhSfP>2UqNA`8{>8cWK>!>C zJa>P4kkpG4X^H~Wrn+dioIh&QqK#NIpf&~7a=vRyJCntUr+iOt3Fn@ISswj7)kLL@ z+4wy(C}HWWYTtgrOio3WEN&F$tOg5r;b*ZQdu7jL;K{#w5sExgxnU1*9egHAMh=F5 z@p3=o+um~N-wc(WX-7tXZdhjgp;zmruek#;wqBe{Iildrx>nLR(9Mf37)9OitXlcL z;>o8Ok<=G`8@@lcc&S_(ndbg3CJdn5RGebpB!_}9S<1wIo8ZMC8F?KGGzzW$T^_n+Lzl(s*X|7 zAY||=qL#OmP^YfUF20&$zl@A0knko6r#4~ElQpd@&TkA_V_$asWjN1T$rVA&)1B?F z_;B%i&!l5pjN^-U;U@ud9u(STPsh(}M<478=>4Kh5}R=`lgDXR;eo#+b1w`~1+o9) zzCv^{(c5ehd+888iEml9Db3|R%S%Xen1C$#lDS*%8?vv)KTWIO-yJUT&4}VgJ6K{E zn(8XVLeY@#o}CfLwVF6cRm$XR zFHoj>vis|TL_*V1!SbJjL48D@x?c=?L-$W_9AC4rX>Q8Omj<32?j>STM2yP(tcr;VAw{J*yta5GVmBAtX*>NakBzGqQS9qDd zgEI9n(=idV!o4_D{9?|G;AyrEy6_JA>$(JcMOGN*`{2nwpGHU&6`tbDG*`8-O6ZJE z)2<&)c#lr+RD#dzg}h6W_Du|un2w2 zPhWr5e&gKGOgNnV1zAEX*8F|I(f4v2`WbE{eB|Qru$%-h<41uV7e~MELfLWn&Fvi^ zZ2W|b{#)4i*YNSb^Vkf&z*MFjt>mU9mJf0n*dPamm?=pghnW#}`HnY?JA0QibbAMR z;T`m@k!JhcX{xnCcNQ(Zu%h#mZWc>#qP{}>Sf}^hzi3BAlbNXI-?vaIE^dj>9OlkF zb%zS=c#3L7c?CM^_#EG)R>9cZeU@(?FTcvijQi2SF;-vYyw-?Vf^e3WG-dk_hX%fS zuwqTO9+i!Z1`4qudPoQq9d~2%sw{?v9Lez~Su;8iulG)VgR&DjN1ybV93ZCvO`z=? zKtR@JI;`Um8}^bIv`G0*H~{?*2;}I3cqwj)r$+nwMj5RBuc>xGC{+Mr&i{-_DZ2f} zWkei6K2*oJ*&il&duJn?iO82ULs@g=N+Tq2l#kXklj`(Qg@YFvGp!7}+8?(geT)FU z+1xtm?Wl87;|WgSCCCNBw&?wP?-mMAJ;(p3b*xa@>#V`y%Np(WI~rla*l>od>t6 zH`)VqquhSKLEJ-;Q2Twtb$22EOr;-Xla43(p}DkU9uaWsD-pNW{GO?NE};a8BH=l| z?-reMosg`$3xy$zPG~F?sSc&|OvMnW-6+}^)1h3l=6+%BqbLd?Uw8Fe4t^**U+R#3 z*^GH^PQ*nyY4kpd&7$kWi6GuQ)$%e@4LlEfa~lk^ycvDqFmWVp09bgkP%a8h3B4CV zJ)0ztb$iyyIWVRtpH^9xBDrYM+qa>ujAfq?mbZe+sk*C(ne-avP(mjfBc?kPr)nP5 z=p1?7#Z*&ePAui}vXN%hb)0q@;bYC3GO>OYYoMVbCt#;iP4TTm;i+j-)cP~&RH6&1^Sn!mh!71((N}&+kinmB{ zQrw}q6)5f&+!LS>^gFYA@3YR{Ypt`-*>~JA?mhSZ!5~b=AagQj-uHQ)pB+JMZ2?ks z2NDEdy4OUNzr{qQuiK-B$hCwn9;i@Tn(h4j*#p8!2i*b90c*&eP8827&rq|Ml$vV4 z>du_Qe(gfD{o2f=G1Uqm$!>gUv?x&s3amGU(u8 z%#XN3W-9c8{15x{iP8x#Bvznp{P@dDr^u^^uZTfO-kHNhK=g4swUz%!Hy={P8#CHHwxFExP{ky6NcYIBAOygLstUx6N zP)$%h{m&}K|8uIvKb2~JaO|r;5GGtr*Bxj4wY=-!R3ZP2()?dP=DF1K7ibPw8LQ$f zm$>2y)x+Su?BRQ*&r%_c%PVbm7RuN9i0S7&Zd^HmfKGjn`=(41q9ui62DXiU*+~`t z5LpTzj$sv^eqpagAmu@Oe&c4Xv9`j*u`I2!==!Ui_#9wMMmcp^YB^B`cFQ+V?)1xk zg($z*r;YEtMRKo>8ej`V`T7-KK`^*GnU7nNE->RPRzLK{tq1|Z^SeLl@A&cf*VuvX z)cj6n(%~X(`~IQhfe@3fjuFsB@c>`0Am~^BpRme|O<>qhHD}@(-tUI4E~cqZ`P?ZG zHuNa6JZa#*u#vsSlF-}wo5!djfx#w|A-Khi9s2_sWy*v7mTDV=oo!hUji^tx9)erx zp)5Z<1CrJ0BIDfbz0}vy&rki+q-2e&R8q0VX!=kP;T`BF#=%?|ZN@V+ z)jU$#qk8?ck1Z2~n?BVWP_?=5!P#kT`Ad%BI(t|N@>$ip@(HAnqdo8OefzKj-(vIc zvq~nL-9-!Z?e4KdB(!yOmr)b{Lw^g%{d&Tr%1z|;j@{3|L;6=sH^8R+r zHfVi0dwLkvTJ0W!*1Wuw=uOs}yefZK{Zam9Ah{4n%%&-i3WzES#0KhD?*&u(3ws8y zQh%@Ymuwad^SI~ZInGAi|5I)J8J1`;=^>QQi8mdPBWN~-FR6w9kt2|GQQ8=AYKV!5RrgM`KPw3WZS-Qbl>qgAfr<2e?i1L;?C+&U3nJnA@D_N3d{5xZJe3XU<9VHEWqEmb1bWSn0M0>K)f)VuyG14B?a39$wA@5;dUt zJuml%Px(6I-^UwNU*oTAen<(pqd;4tTC@tz+Uv;sl9d#t4qdAPv!f2=x-blR&m*@1 zua4xBrnPHtewws#U@Cv@iv3&biQ`7jH7jB?BlKZkd znpEpdXurvfCHb9`;^0-*^5PRC=C038!F1rr{>M33-mRbM$%M}ni zsA;_9i8O{-bg$Po`dD{^7!=90O)>)e`efo^Y0dzJ^fNKYl^eyaoTprCBkZgjUQ^N7 z(C8kO3t#EyA2)+7K8!N5;gWLmsICr(WTpEqH2@OSGk53$kuPoKx1gt2}*mr}g zSB9@IQd=kh-20r}h56$2FVKq}N!}a9Ixr?@HT;?qE9Js|*e*yo=*2o9)*zeH*hp<< zMo12kiBtE!+C{duT4vGcd-f~`gtCuhc}&}7f6utb?U@&XXGeR>Ipb@fB)h)a9ozG> zt@W@EqDJ;Kyvo{RD$RMwg@Dd$t8PNgBz&UR{FJkf)l<7>4FI|S;^fAk&{oh{E&JPR zO@sH0lx0G2-jrhLKRaOH-)?Q*tRMGt?+Gp`&|=kBh}Z?Mj!GRfr~l#-mD!YdsYJ&E zqKVSMypOw$-5<@9X{0kB(g%t}(n5QO35b%292nW~yn)ya2i zfEm|Z`!E}6WfO-yZ{t0N=R5j$pHsaylq7VO37q&^qhUp$=88&<3hak(03K<>KoB}J!rtvR8o%3*L2eH%6Qe0d{+LQIHSUbN`2`U>qIly zV@>1SqZ;ma{yOwD>v*l*m?Z{NNeaP>4GsaV_$rpFcgeOh1$mA3(yppA;Jg(RT4Wb3 z(iC#^$9^qhtaz`LvYOs`85RbsKJ2syqXhTU$_BjUCiwUBKjekKF#Yy&>nm$G1;?E8 zWu1y*bxNnorl_=>tXY={&sY=L%5Jk^SF$r;vm|^TuljHW|JjeVI3=8?e?G`DTo`D0 z)ghjz_n|JvVqW)&8^RypC3!yk*AKhKUA90RHbU&w>DYC?*dd}4e!%40vKAP#c=V&tumpD}2OWW+5~Je>_=5C5DEB(XM>R9Q#nt~Y zuKANx^#6cM%zy1Y@e6zNzCCQqpVDUk`DeNM@(;`L-#PsM@BO}va}}@^V_tR$td)Co zY}MQdA&N}5nQ3&LC0Gh7!2g^ZKPP!#frQv1W#yeO|0{>b*&$rK{i-tpSw8L%enU9h zoTBc%+c06|{l^c8wUR#ZFI|aDG{;SBIkna}?&C^y$lhYaZL8Isvz;xvYo0>7F*|;s z*?UfJZmIbN5rfN|uqTdb_Hb|Di~!rHhtqh}=w{ipj$5f$4aL&#^FbXbe!d~@(vHTJ zvYhGOkh#`DXPqjdksV4^1I3q0)qXyz?g=*JUSW)9!U{+C5rG9fIBn}M?lWce*8NPD z!*oa%e+FbP9`A$KKjgbnQI4LyMGk7JEGCxHm+7ne)L#DuB1y6Er?dFgu5*SFfyPCa ztveu9p2<@E-pT$gW4KJSr|z?AYc=jK!0|OlPsqHs8aBTu{Snuv`0I3*FUYAU_(z*# z$w}mc;f@PeU2me&>cm`@u;toMW>GS6Y7-iuwfLS={p{8}Ns0F&Ed(7t4EF_J=M73= zlou{}_keLTm<}b!-Qz6o2oF!pZlOhf1aDl{VsBewE4!b--itxpm}y~#@KcT86b@ep z7K&;`-M2XH1#GKezfcNpaeG zI$OzZSa+7FeD$46JiAwi8q(HZ8innz;_XL?5BKMAM&-r<&Q<>EoiG|F8OlWx!bCfm z-qOOyRH^EzDyocX2kzp_tA{^m%K#59>#_wI@Ufx3m%RJU@#qEH@7gc^BU#mM0CAz< zWAc|afy5Q;eC2cV1%AwZWwBg9YpH}F$9<<_>VEDKdL`?J8nj$eLr>(MO--rRaG5?I z+N6^q{&5Sk1p=a}99MeUmdLJj#!Hvc?kR3!6zXfTyD!^E7Xt4un%B+NX%fp}dO_q* z!M471*BMRHXi4L~747=^rlxcA#zqUT55E5U%0(eYh?qMNKqR<&4p8wVCBkHke_Etk zx;@Bq&3j!8w>*a{^gr>kuCLFSoBHj$8di}-g0{sC3UcF{f0Ob1P{QZ>qETO;UnsyT zdPYkHEA(hW-2FlFrw5GDQc^+V=?A2@z90$-jq6z$?C_h%b=) zRZW(9|F`|Y+|FYDaSc2bRhj;>T(7TQ_2yoMG5KB)P(~RYxBuN@Lz3R=nDqf!v{ln0 zGM1Z|^-8I};{o7C8G*D|!x=yG;+ili)8em|;h8dQ=WM8HPsJ+x71%=ap%H$L;E$Z-s|!8h@9bdMbUM^(OF++IGUhr>w4xT*#5mkeRE%6AMn z@0-2-S|iDj6F874$tn{*fs6#3B0oLI492`&>iGU@6#is6G4Fkpu9PN+GY|34mp*Dw zfgM6yMcV#F#e@}fqF92Nmlvd#+ji#hjp9;kWoYPE^ze@?!Z|&$vbb%`Li|I#!_Qtr zlDL7RLQFr*Gppxxt%7`|0O_PhuvjGP(53pRPQqhm`TgS2^i+4cu;m z%|CD}f(RyXG2?%0lTAqA!T3$0T8Sp`ec6Nv1qveJO0 zO?ShnO#O(whNM>h3%(@oa0$(a!;*bW`}MQ@m!rSgLL9d1>m_L1zBZHwk=0L8Q#% zM>3TWmFgGg`4cw4(9wh8i-Np@0ue;j?NDDka^iEunZY?2tQtSzRpz4bnLxeOqFS7(J8Oq50E zZn=wW&xWk`ngZN3|k zK7-_4Z&F;CA-=m|nnVeHMecoBtMx6kbyTH4dUZXR-2zy?1ikiO1f%C#D}fdK!|3V| z_WcU8z*wgtU&}C6&*xFU%njMxug7MeYD4Bek-t(1Gh6r#*j(njoM%J2X0iYlbems` zaNi1SMo}Kk9BO{-mi6hu{9{y?y$KN2!LK~{@Kb`Rkx_<>ne=C+=fTHR=Zor8@TJP= zAEgg(*cLh?zHZk{3b05I&To^#1VGaQdSAPpf3VjCDoq0C7V4R~yLxs-l{wYnovoZI z*sjX7Xi=MVp%Cts8zqgb>LL$hjp_5SMu>QnW+%NB3hrrL&ZR*sqJ0#q@C=% z9xY2}w+UI^8xJ(bk*yUhKyDmTN#3-c_=XE%oixKyn?gk??&{p!gkdbJq}UIk23(M}giLR(@Qmj)Xg7!;dz2Bb+hLctfFk9>y$Lzb1bEPRsVeKcagBvY zLb}+q;Sxyq#nNWOiN>t74mv}rpJseU^XY5x_B#jrM@K61zxMd}SOA$)_1MGk+1Xj~ zaIWm9pUAn|iwc$jxRm&FA<)YXra=1YAq$n4IP6sB{$YPS6aGVQQ6cwW3(7=;b@>8M zEqLvg53f0`ylvml_(;c&X_e*J;P=gEi-dJWoR9h9a4vcAnS$aRY^6}T&D&x%f$e@( zzduu--T&c!_jjB3fA1V_69ubmXjvC&*}h@icn!SD-)Eo4_e$_enB_W+JHTz~m+7~s z^nrNbELt#GJPw!5Z49J{|M#v;f8XW&zwKwOw%%`AF~13R*4C=J+ufG`mPc2!ml6h7Tb6u~rU7|QP@&naVBk}fsHYmG8D-DciHPsi^BkNnv z7&De79|R8^8LCrG6zNp&T@;)z8S-`+8aB>}{_w{glcUF4&MBkws{RlSihh=7%XSp; z&{=SS;Iv!=W~tNJX?(@^?T$sVS@?eT;5ZL(G_2j={>CvyEK+A`^u;u+Hw2emFsHva$yQ?U?2thQ5#xcnB-i-dlDhEs?0z0Hs0 z)Zscpy=AEN_Pa{66rJvXG>eN`Up*Y7n$%J9dtOETzuQViG z>Ii5B-$sf+By|8YRQ2)HYI&+_Q?+!s{Byz)1GXxNK-d+NX2RM!r#nzj6|?Z{8Z$WwcxQ@2{a9ZT;`uA#OJfKxdv1U@aK!&V1`iByN$dawj$JRjMm5 z^$DCzv_Y%hnbs1)z_=s2{!)0ColLp?s*Gu+jPn+Y#;pqj9OTm@%TyPCM0Tt}z($K3eQ+ zJ;?7bJ6Jl&A~E-5COMih6Rt2B9mq61I6=5J*O)j$9n$fGd=>u`w}_KSSl^%Gc26Qx z_Zlc}AGIt@1I4WsKHWC#YbFRY5!%^`ueX~?3pW_hc)rE18udX~SC^@%zu!q|HAvwu zR&W*EE)DGUEmCO;Q|95zloWGDia+W@J`rBNQ)Vc>mH$;MNP+eOxhIUN5xl#5G8KNX zZxf+ox%~T`%4sW+Cy!QwD8kh)sR}iD0e_r2XV_6XB{Dk2I}e08Ietuys>O(af$#)@ z=yK8UAc7sx+r;~|~f1i-(aUv};IkdL=ato_Dx-GUef(HkNBra0w=NnO>KARUm(J zRE*^PF$>{99ns7*;0);l-SFphy3HgxU|!J1XC$7_nO{O* z>04?tkXv#Bl$7u`BdFF#=fd%LS9VNKz@-=OLDg@ladenJ;g#(~3xQ|38wrU-hV5hW z)4u7jLstB+H@o-akcdLg0>08S<_^aQZ~QL52lT^tY)I9HNt0AGcLeO;DRq_yDaSmc z+j5RkV#x};VezE?3$&7ta6AE1l&JXR*C`6ioj7TWRAedW>e7LZd+}KmmzU^lU?f)% z`(muZ=3Z*#w?iHnzxhnPvPA!KVSgpDA|Sn8CZVWgCf*(9@6mGUC^XPsjQ=)PbuKvN z(>5c;kD7BlE#qA-VWt_4k{}X)FeHIU?hv=_NnLncl{jn66n4#)T^}6`G;}2`!k&b~ zMkDdP$Hxq(8}c62iE4DT(QO}}{y}w|jb#HImYu6a`DA5G9{7%L0L~QJ6Jx+YXR7{c zB)mhXZ8C#!qzW}_TWR=bMy!BK=Y;}-KH=5{+M^x-Zprf>xaH?*rx#ci+UjzQ`+M^s zb+NgfN{*ug!h(?L`vp4Fy?V>*zg)WI-XO=McWK?Exl5}7y<}~jLbzV;(`_E1HeIbY zJaPhYbn8Ap`oZxEKH^|=tZFw*M%c@_Ry`w^9LYu7t;C*;_uUxFN1By}H+3W!mC3B# z_BOB6lWQ?ip*dIE|E}SxyY6~o^Tz&Rv*Ek3Gr;FWvW7PG9%pC&SdedI>nGUClwnUc z^b9IbvD|yILiE0lU$hKfcL`odVz*2m@yLOJy-j>(5U_wTMJ>$S416uFD#1rPlvxchv2;6b%SCX{Aul5hP(!;8w zU^4+IokAL@ZsqJyZG&97`97{xWH4L#lZ+AN$f_9PH9Jxzp1}*w;6)BOu8sgpOE>7` zN~QOc*u$BAFx+9P`D)O&HH?9VA-ke`<>BQgholj|#+j47lRY2xy<|*&=Teo#z8fll z*4K2rwMIs<#M)}#Bfu(rY3OKnl=6C>tDr7Q%mh#rNwK9`?GIdnJsZ1B#OpAYk?S@M z?B7yV3anTB1h}*tgJ^@f*mGmZlFn0{Zj~LHU&s4X$)jtR%c2=K5^eK$8$K|@$Jd{+ zWUi+ypeXJV>#Rn-L}Z8P@buhH$+&PtoKozx;y3BretW?`I5-sPMfv(=$RgC5fu|@6 zP>z9iQ?HVxFx1+fr~2RYvRCuy`;3((UMD0CJ=sb1;de+=T(p?yD01p_((4|Z%*)A9 zn}Qq$h)rJF#LImLwiU6P(b=AT(My`y1<#{<+T*^QMd~YK;?+&m-;Lxcod!vg?Hwtd z4Rf81dO(phw1JQP?MNa6mMc@&lS%<>P91CIm}g=zKb*InDqg+jtXy^)m!Z7zn&AsW z&gYF!r!~YgWDP=``G8%ii`#(Ued{bnaY6D$gY8-Oj60DoZf-6-9k;8L8scT*q)7|H zzr=M0g4m{pl1v^0x#yL@`DJ?cbPoi;%eJ|TOm-eaDQ3^*{628*)!cq0Tu2<7_WWK^ z>%p1u($obqqWt7&KAQehaTQ$k;<9XDW3R<(CFKseUdd$01E2byV_4kyMN}hm-~*zK zW=wL&3&qO%ChF9T(!n3$tSld?G9;9+%sdE0e9xo0oeShC4p#qhe6Ecg-^9u%g@T|l zb!TpYw@^pP1O|{XD=t{#U@nl@5Np`s&rsaO@Zw|p^BrN|Bip_12MZUC@?L!c`6t7) zo1}drsneevrud*9{QO5y|4&&8<8JJVuq}C$&?6y$&K34g8H@jn#rS{lF%tYoNuNuD z)yqErD~Ij>vOxtE)$fxh770^N5=IQ?=5kc{D(YT-qBO4*UIyW?WhTI2FTUxkx)iEX zH8(eiRSfWcP>NOfw+O3k7?-axwz15A(nK=4t@X}oqAR=KsL~9=ID}Pn$T=ZIjqocS zC!D4JB`k7PK%@hm#Kg2){Hf?1t{L4WSR2w#J_Y8;3F370w>G9i#DUdvq%a zjs+F^`K*h|l3W^i^@;2Er5QSMrw(>2{ycG8x$vopT8v6*as%aSqn@sS@C$W-0by(- z$J$U08NJIiI|0MhlE8n>lVkLH+UP9$@(sEY}%k(>gHGB7;>-DLwQVRKf` z_8E(PJ;uM##0FSyoqo0!J4xuxe)Xo0^^K#05OE<<%slwehyZ zn*DvWpZ8WdzuX)%+!{ud<9;D5R$R$%&+tG^HYG6;R>UnI>ZC3gk{a9@dI0UFG}XJv z$mIgOmqalno%21>n1JaSK8xSFdpT5x6F`{n(Qm+hqBgHIAs`#881y}zJM?DcXX@#; zan1FtZN;|#(Yd0kiF&AvL;&jg-JB|Vuw<%s%D#f@tx;vV`Sl+wP|z(k5V9I<$UJw=~3Ovt=;qe={k^ZG*~ zQ2bXCfdb%HewRhp&8GbWyKYH>8ReY)3|qB*ZH2ME>vn%h0t=EeTSa$n3jr{JvdCSB zLzqY!CVinoQ3Q9USMkr1MM&LyXb*buw{SQ?AaR zql)L0_D?ZTp(PGOwFBmNr}#51svR zoY~dEVv6TE62){MxJ4~hnhHSsIb7L{Xr zZKUmVX_k8B>z_E;KlRCO`?9X6Fymq^2b9qByp9&8q~UC!`>Vx|@%eiA)Zam5Qb&+m zS;R6DZD=N%GzjWja#hyq0Mm%E_h9bg$m^?6gl7*~m-s&VSHzha)%p zVBT;-QC^_*I4M^0{#|_hjk>IeqfC;H9 zB>bK@e$?-TLl5C}f~Ckk1j5Y}yR=9#8|5y3l#<-U)Lbu!rUSS^Kwg$K(ex0pXcX^f z`}#W!^psYI_Qy?cLm?Q!sqNf47-pEanK`rzRXurg3I=lI`-p%2$|9QC0gWWiNJ#z zi%7EMRIg0kZ37xk&AV>U0d-zOrD$z_x(iFlX9uI z0K@jr!*;^laX_5kCM7F#%SMN}S&H2O4j$zhF;PB+g5$82VOE{>A29afMA?n<{z#9F zJpkeHt3Q7B4>$aO==boWZYE)siNoEckH=o4)04_~E?$ZhwPQfDL2-FzqUQERD0xYKlLLgCFTFxLhy_2?)~F zZD=x19JTMgei3x_vM^@sBR)No&E2i3u6vEG7~bYuV|obd3M@2$+V=%I`*dS<|Osl{%-Z8E9>j<-$2!CLot6PRa-YDW6W6F+-1KE!hMJ_$|-;&f6{S}Kk zn-tyls5G>38gO)R*pPagNGM}ROjwcm43G2~vdxL%S=SB4+ezw*1UToF)VFPj@ltFv z)o-9-t9ylO-VKz|wH;Wj-Aezk$DIw@lR`A$b!9 zSTfOz$g$)T>W9dOB1^|l^7G-f_N{4&OY(k#=QN~P+mvDP>W2J~gY4$-&FUg_IWj_9o-#oMEQM`&q)%CZQ!XDZCPYUXXlepN zmweZ;39-pY@k+2vp_+E?ia4DAfw$O{0`PoVKmGTkCta&Z8CHwcB|@7A5@A|?nK`&G zIH*)O6p0((<0PykEUyI07ORE#HxqiMzRS^-K*X!crrHdGwc*MoTSMa9jS#W1y~&?g z-qp>$^my!@;4K)o8+J#R-2P7@&L2L&?SnTaRE2k zN4siMZuh3k*2YgRYd(b+DOD(#z8PT_&Xlksq<7UH9-28}y!WyiJrrv7Z7bG`+Q9d& zy(1H9(a8F%&b!X|K*02V>Tr?Ovpf{l!rWmoEy1qo?v{{aZ#KF;lP~C{Zi zxKkWwnON@mps%dU3f!y4T;*$P(EQZhJ= zh}3H$tmz6f_Gt^Fb*gAykxIV4{QV^>BZZ?7CbOtJ9&#Ifpg-&6@nL9IWeU1^v%_AE zGHUd&IUW1fz!% z4z-){!Ohk0&zHiCjENt*I$;6kP)aMtMVWVCMOk_2Clw>)_0+z&H*+e`=;{3f^Y7n3 z(End~t(92W_H9Im?%f|-!oW4nDNT>@{|-(5SHF|cU6mN)l`6rH_&!vuS>V<)@6TR^_3gs*iyLHAHg zQbS!+Kdn5w1qL`)`ZG(FW{Gp7I1;qtH?&cSsoz@qlMek1tK+Rtcb)GD)EKMt>PFJm z%!XH{Ih>{K+9%ENF66dnx3q-5sz}|`Oy%|QafMqXK?!0XA(AX*Kp?jG(ejpuzA7Vf z)LViD*u@K8NqmCbgCzH5`$&8NFUBXtY9Nl5r`Op5Hh?L=GNYh;$O)2!Eu<;qJoOBJ z4IZ|P6WA^sLm#IX!rHmoO8%|||0}zJjb{?_0b~b=dExV2+_@@jCQ%Q;N;7s&Xyr2* zNSu8;H6FdOT5GXqioj@{-q;dp`hY)N*yoK4_1@Z{3F%jBRNMa^ndkpjs4E(D)r2|r z_*WFgq<%t1%R?N{7#}9VN%;Ig41kpGU)&3CgWFq3CVZWXBAD^F&%6_7=n?K18m{_# z_2p;U)Eaur_YtWV$Q6=3?riKEeH&)-d_ObIkj_%;I%;JnAw;f2GEsco^IA}uXx91; zrDRXH?~BchEd^-f8Ta!G>*UVSv3=#aW(PpVSqu&JmC8Bjo?t7@Z`Ra3xBynF4*I<+ zIX^7trR?q!{cqvxP_mj))6eoA1-xDg)F_(J^jrR0;n$=EelvLbmMa@G$%q3 zcl($ALk;!>0n0j;@;eZcpKH{1=F3*NxZ!gZ6``t?e{?o~?${rl4H>J(zokvyR)LK% zF=I`Wo!B`w1!qwf7;QNGdYa$MY)NAaDjkd|f!}0J+i+5SpINiGH1g^y&o$KcZxGsmpt# zyl}4j=53dU4_j|(ki>0w76x+B16!unRBZD0y%aN=iau)|VOxEbrHsQ<=#NI9k;Fqk zl9_jocZEDXhEsIh7;k7V%WJb{{Z5?EGk|ni^;xs7R+_!_N{sBD&~{vJ-1WyvG6t4{ zALmiVAN}n_$r|w)?mGbxV$eoPjl`2OjNZj);p^Y=_H#bGdx_rsb`Hj(g2!bJFk zJv;8w0`rhKtrgBIN{Q1*;Htc4W<3KsMZ1cvT@3cds)oroHLI(&$rQ3@JuUTF$#Rv0 za0HWrp8I~+6A#dI>Iw7Wqy}3Bbqd=41(I7o7*{oJb-U6vsHM-FIK5JrUcTyKGx}ZV zll8NC%gd(be5NJzz>NwcxYq%+MH5&kX7W;XRPNFC#c<%2e89tq{<^OH_r87QX7vdC zxBamQG4A7DDi1R^zhVnN;(QCDp)@fl^%+4$7oQpN;+DW6%b;@5C!{0}UN^X1BcyV+Z8YfREUzby7H2S@7Qkg8yv(Rc4D) zYzLEEK}R{NFqs*S(tTv%(s_~c6SY!zxSf<^oSc8101+hH)@4SP_eegEBr#K|s--8U zo5!nqeRB-$oM;=5w4rcLoevfA8PAY``v4iN9h_1Iya_w8JQU6_GqW@ZU>}H$RwIf0ZH@?~IpJql@l6-#wL@i7+FeOCbPl^6maw zra+uaM75B6Ca>g{v`CIPo}57SJ{&Ubdbk_6>00y)|B$&OOY*hnIMbrtSEX%2wuX~P zHzIm2@*H~I8SLDW=3;amfS+YP|ANFyp^68#*9Zgb#}l(mdVawwTlXs;R~OD~!Dye; z9pqZ0CX(%J^uwm(6^f`7R0MwixU7dY{XWU_*Sb;z%!#iap+mZBR}G^djglmiJr$Yn zeKtp_5?en+!T?PFL-3+1rSeK1PQ`T4WH&b1L_#qyzGJlarFNw5H5jyQ{OXCW#*#dN;3X063{9 zRr?Hx6ChWyC@rm-RJs<*1}4%kE}eD*NMyhq8LRiCWYmf-OE)u5dSsriRWv5JunZ9l zv&P0f4!P*~!Sv)i(2nmix^xin>e~<);}w?w`pDDYF`=)oc9+2kIGrZ6Ram_QeMd5` z=0t*5o*JZ3-ehie6k1d6Fa@o22lJxjNPl`-8ci3Nq`4!;RAmKsG8)8c+#8^e0g}#T z<@(=hkrw9biW>&VvU;S%l>6z8ZU*u%Q2t!Qp#_{+W?BEEgjZU60o(1^C+0Y3x4i`S zBhkqk*kbh1yr{TUH-(cynJVYQD0AvdY2UQ8Ew6n67v^gNY}!->e%% z=&-~!x{toromk}Gs)f7Dt#-^~%u|{eA?#||Yl#bSErq8edBs~Z7$PeE{e)TjwaE*F z=G?qteUx6VW;8u6K>v>c1|c>)H}?{Z3=v5MH~Fd%V>_x{=JNcGP|Q+y8gFAnfQ0%l z@YtsNVi6nj&Qpt3kCf{6I2^K8BU0bYme0Vl=59L~yWpAi$}L3jo1Q@I$G<=#*5eJ| zW%daX+XxUOVJ+s)QVY74zhh%wE3gBVs?$0$60ytg4cCSlY zDSY6|A9AJBKNLX!OKsP;;U|XAkBl067;N+xqNtv(#i>7oR3%Qc`;qP(gQJ(KCL_^W zDxUU>OWGLRjM(jMp`TJXwZn+dQ?}foqZ!&&z81wx`J35TY_u{U2m5(egX!-y+F@r} zG~!k0wa+jT39jyN)KHQ#PrvUer<_;IGD!9O&$w-v2Ro30Ia7LkQmVrk~EpSuGT;c4lM`+tF8 zz_fyDi{T$J0AM6+LU*a=KZUzu@>SsdGS{Rd=~IpQK4%Am#RaCvcHE11V=OrsL`emE); zJGw|uEQXXh6bVD)qpds`-KsCE{G(Tp#HPo4ZV=S!pjxWh0z=y9jjp;{D~AIRZ1znC zhA~rm;t}qKrz#a*IcNC1oUJ4`#wa@FqmndaE>Q~+X#4R$O@M3(zD{a2HhuU4e z#t<(XZ`Z!H2v!SgvN)}ecCFyXGfnK(`3q#8w3ELT5V~|*h96y_Z;ob)v^Nu54*gs? zz^#QSh zh(0o*auc~~30c20_F7qJqdC>lr`TrP__WpWv)uigPJ_efH{pQ#^a!%DAX(4!GNyJe zkMIZc!uMN33TV2DcVG5+o`E&4B(YK}!y&q)9m@*>lXq7``+T}14R)NA_%~D{d-!mA z7HDZfAbfTC00nfw$9m2?&s6TJ0B(qd{N~qBRd;@XyFd3Ir+TsDkkz72`C$F6@=a9{f_ohSs2{ISuZBDFfiw1TZS zD<4Pw1xl2IyJH^?^1_=-8ZvK;+h=O*_Cq=#W&7`X0Cizv&Qbm0?{zS9fK4;a^6%2q zth|ODCaw=WJ?5`Fosy0Fk6J*pIQTXhJv@?l6;~@5iqlR^Z6~j^C-o6Op|pnxMXp%Q zuyFUCGZE?^n}JWI`tWg?%>4Z13cdLMg~&@$?!}%s(1h?#GV9|UpIVWzIC&qUEYjh$ z|2X&LG~SS-dOh-nZQqKwl)2#`?`n`yf6$h&ZKwhG@QNjgO}1gCYbQcne?-_YN{?(uVFtue_HsqH(?}G>c`_76Ef!f z?X#*)(I?*%RuvH1ri|pueS%K`&c6&1TC{74{H?w({goqO40hD}oPY?rGkRzFjs!(q0JQh&J_-QME+)8Ewr8)#;v7d51TRT90j@KZgr z;@+@ae^}5z4Y%D`XG?4rxQhPlk9RZB(AuLau&ObJ9Gp$jF4v}>;A__1Vd?4>vE=9>d>f`5Xq zX%9eFz+VA}yPq+m6Rl5M?dr}WPY511-Y3njP0isCRG>N&#AR8A4z-d4e(h;c@Ac~I zvP7@!UGG<=^?cPzUWw1N5lQU?Y@LW^h6heMDL=D77_$OLKr1Xv>J%6pPH;1eAPcY?dbx?A-Ebu678@W;kL8Q^*v_h)6-aO_I~d%o^K#@Gqkpm{=XEO) z#s?~dFFwfh4_Uj;2&KI^A0>0&sSI<(24MhW~ zB6)2ReaQ@|0$rk0i12W^2{ywTQF*_vjU%?;53sICP;Wp3AnIkK!a~|WxQyDPfZy}w z^efgYtu|?!tqx1os7xDs%O916#RI#*7vcW_LaXmN;x^RvfOETG-DJpAR(tZhUq)G3 zngN4gu8IJ_TvE>E3(};*!%KWdR}2dSG$t@cSh!Os*f{h@0fe(|T7kn}(oiL(BW3ob z9bvBwyHsq7!{gfW_~!ynjFNC-{>z?g4Zu|jf=fZCo=%vW{Ml+Do$dIX} z@!t!Ni;nKLiXzMAKY#_ZR)IpXv=*tX)HA|V`^#4DJhB`0^?dj|_oPk&9X9(#e)9ZA zNS=&gCcEDy2Jb!f{c^dl9zY_QlX}J|;b3+o161{sQ{uhrJN zN2Ii>j{mvnM1(aUNZ>4X3M>;$!;!B(t4_R+eX6Xf&P6J@G&;KA_G~HeCv91-t$_mL z#CkPyE2}Wa`VItu{}}M(4G4JUe!ip59kV$T@=phMI5VFl3UqL-Y67~C)S{9_TS63cGG){NNDLWiJd?jr{InKB9ZNZt{AoH%V=@Bd zPqoC~aOCGaN0!1BWT2iL|0~(rJSg9j5Yz@8MAZ9X49geuS^fKo4YMVB_tn+;aC}7PWC))#kKyA2yfhytBoCQX`c`Qr|Nq#yd zJ*=dOD!iA{yMfjUJ0e;&RqbVp3}C?6S!PSbzi5eiBTZ=Kr1pN$cgyhB(|hE8Vrwy+ z$Wl0wWXIXlWQ)rUhBLSQi~?DAuJl@X_9>*L=j9oq{uC-*5dPD`;m4h36EAo)Z!h}e z=#_mQXhMs%7zkC8V+4>w-l2*}J0c*W&*T&9R|h$rqDTZe*l1s}TqC@q&?t|?lPM`a z9vrKnh--keVkka@%4?gtZOhQI2=YCVehu6B{#$BMgc|`b=|eXBHo%2 zOEBP7^ZV#pzkc|RjN?TraZ^;D)BWm8qpByFEd1w;J_FrzQ0i+5PwK`T?P4%$vVk{^Gsu+_sxOvK?}qPD&}OKQ8~HXkrTsgg>~M%ofFf1?=-b z)~pz*M&7HLVyqOLrq#&(&h7EuCRcRl;|^dKNW_bSFps zBZtKAt|lk_)M5r}_lm_6hDPpM3nK zEWcFOWKgzX=ONC~Bcn_g|>iZ0M+ZQeb z89wRC`4;z_nb0C24Qqp}wwQKeAIv)`_*A92-f$NCA*B}ladCzA5@Czni7_S8;qFuP zY*p4NvPJOku`D>+L+VXu4>wSt8yaB3?MJSCgx;HFtVL`{2lfxSl&`Q8CX$GHIgW8P z8S`KI@eQLGB}HQtN%`Lvaq=Lsa8GDk>(dg6nxD70xp42f;jbr;qWXMHn=BSc={VxeP~hEE z3Y0c!q=-o0dv2cr{>o~v8PPKnFTcWZY(SP(Q+XJzfDoUgz0HJXo|(?c39zU`dzAh1 z{}fsIKl6XQrXZ(zO_(-9Gvze3zaO~`0;a-z|DC7xe=oBA?{Z$6bHZ8a+vJf&>e)NJ z#>s6|DwJ6dbNdVTH*N4#=#(6?#eTV|<59`DHZ3Zcsv1V=19)+eTcSGW{s{8-$ELyf zJY@=9KaR?|lJzkLJLv6k&M64k-??Z`nb?lY!RoX2^h2YsL?qu6e?Pb+LjS;(Iw=U4 z4dnT-7*#;T2-dz?b4+~;ppMB^JpOS27;oNg%h{u3d%<-lQGyQdG(82o9-G*CnWW4y zKBpA3e1f-@nZ|%59v~5s_g!wu!V&ewTrFo-4?*5U7c#6^Wi~UhY~@dsEW4&EiyE9)&G%z+#_@QB4ciaclW=L zx!l?A6X@m{Vor?FvIFh1u=>b;UL#$19hHuHUY`-P6@o~(@V-IEYEdcZMM9qUxZW;a z(<@L1+sO_BIbrtvzV`dT3=}q}GYa&q|J!w73fT=H+gb?9h-I1etYCEawZD~59)DfJ z;Tdk#LHMws%aNVPNp>5dzz@i77#_-Qu|gkv4dhbM?}Dd!5+nVVX-6}XI)X_Ar=eU* z2^3y&va{hgLo9b2Q5SuJbVDZ9_BaK0&XHkj(q3Bj4|;69l8kcoY)J3h@~+^%Y_au^ zu3!dpHv#dR+az8AWFOxqZ~g%fu8p)eL(*`hYaV1ZRu4T_hW}RVvfXtig_N`K^xbDY?2T;c~097 z9@vX=1i|G^yWRT7!Fsh5^#+_tD=^1vA^Yzi&yF)(*hfdRJ$*qBr55IR?6}w`pu7J% zulFZ#zB>T+tY9;56hC?m85?YoAWU2b=Rm004trfjENN5`c?>tP1@#z*{#6!86Cw3aTwZ7yfg=x?qhXl3^jh1JRp<`#u^jcQ$r!yV^1x98Q$fK+c0jAqw^ zn6l#X8k2t~pL{k=`8V>(D9r{32@A*}mkq4h0lEL7-E7SXfnNyl?0y9%deztCncb$aLH{=4@+n-n{2dh{uf zgNNb#m{wdhpN|l8&%o>2osU8kT7R_FHD*4Q3Ht3qr+;SR42xI#C(?}bu76k$Y4c1z z<}h1lU&){DwOGwc?zpnN3)%RZNia6RAH^KwK1G7`DeHQd6DA!ThV=E9?X3VE$>(kB zMRBUvO&Uzg@EdXa$y0(kHUzq)2F_k-3wgPOwGKA>zDt^(Tv1JIKd$(34hw|f{}W3} z{|YG|1;moJYP=x@VoA$KyK7nS2Dm>HJ;VDAAfpSr%!^m|R739aCb61`9GOk_}E{o__puW#{s8tulSqRBj!hB;oc9^{a~PD^pu&0K^0kJv1L0_%%&- z0?TyF#!08@oJ-$h29axBL=OQvJomW`-eFBPT60ss(48k`Clnk(*1wfg`Ta?XI( zfzRU#>@3+TAonN|I4PV;_tIX7;j+HXU_B zvX=bHfG*`J?_7(JYw1kOpj-+9^Z01)*yD94vcb=&Gumz~15~d?#*PiFyZI+0ZcUgzv{MVr=1 z^O)Iys|Lqdh9_Eqbc9X+^gRM5 zRmtoxJDJ2Cn2mV%!BD#g-;AprH6$K!&EDL~m>uO^85~&(AUh+X1re?pXG}{E*k9Da zw}G}}cMKD*cvFnHF!uoj|9Mh%tA;qp$^sl#NYtLQ!Kz z&cqapN@Op|4{q~EIy)y=S>c-^XDo1UxKOe7$+;nhRT}%UB118$8i=LV(Q5}PV_$#C zqq*(lv60N9UZQuX!`GH(P+4i{M=E zhE?TSJYXF7_=fy|;rXVq;@Tf@ts0Id2MfUKZd70c&KPg{7{y)wH87UA38-!~0bNaP zu7B~Az$Cy7MC+fU0QUUmaVlxRug;uhfV{Bz7w?T&&eiRlRcIaH7~4gBh`r)fymbDE zz(%TJ6(Z5L0K7i&FJ1~70QmRUUkU=DXX~gw+-(Ry5MnCDinWOSUw&n`?wa!zgWmPZ zCDB!Q=3l&DKy4#9FZy5p`k$C^z~(T4%ZAYAs(7wd1b|VHQ|-t0^6W3NdDnHzUzIz* zxPN$CIgYu()Ft04c36iT7;;Fqsd+*UfZ6uF-4w}AQ3Fl!l%t`|_cWeg@uT}>W1l@4 zRdZa0VYqGr^H&%lp0JM(hV359xh(u_u>DD8WF?#RO^RTPM@?=sNUp7`7fSt86?~L= zYWj${^-7ipcdF(@4^(Zc*<%^K16~!AX>i7|S}PV{Ed@uNShXfBpe--~_GI_RnAD0f zxgU~uB8PD6O#C))s4u^m-;A^IWflv6!4#pBfv5O0)&y5}PX7FcZZ^tOb-bR8-ByxX zqA)JLyw`R^vRHYjdhjR#Ec)uPlNaCJE1Ul*!gB^gc|&}@hG91abp(*XJT?8k&5NgN z@@J_o{dAzpHQkFFi2iByKI&8jGAsujKxK}1zDH4~gKEAgGQCaI934jIgnV0+C@^d1 z>1$0w7*;@SI#|gNHaYZIlO`a$?#pwPR!NXtzaRRR3$D zoXd)kSe&~V9Y5o`@K?!#FI_k_a=`^B(T3K>kFkx-cZ)pzS+cWuTw+PgJP-Uo)a<5; z!ZSFJg|9zpFBc8K?_mNeeQpjrRx5xniJpgtI(eJG9J=cA1IoOKWwCJcoU0{RP0&V{-p+DhAB^&6l;%9!%mzCmz@CP2iIs=Ys&OTLTv=bg@ai%~WRTy=D8yKF4sP2n5$QC~byR9^ zA}=Wge+X0s0_ZItev_1g?vf2f`VNjeoVC;Et~B?bSU=X6e51sLq$$2VxVFHjeHTO8 z115*)0h&z!u1L6CK?LJ{L+woL6V&VDasADibMpJlV}{e?_wnB4QmUZf!kF7jl8{?|z;wne%OW$mVzJ7^TujYrW^gp)PZsgbzG4i5s*~Eva z_o?Yz#^Ju8vslq`e(lK|evpH24J_q3E~_xwD2bTqQ_6v5d~UoM7vG>81M@w9r3%XV z#vAg(ajZ$j@&Rj&aJV3ZFP4y>D9qC%dwp1gK_I4aVKW^tihZ%&R(NmTtp7Q7OK}c3 z8d!niYXfW`PGDJag&s%GwWh@v3ML&@I*-$P^#Ym6K#yNCi!yBK4%H_Gjn~u!@cD$S zI-|8&3a#S?b)!6!xy#;|G--M}Z+)6aI|3Lm&oyDH8A}6cxe-q8-1Kp&adijSk{vYs zBXk%#rbu6|AcU2vX{v5H^Wn;rI!Jly6XKvkVK`QoXm&Vj>TuZt*3YZK9>g|*a3;Ma-P|)Pi#IZ5pPc>8^`$wIIw{(V?*PNQ~OiCsE03uPf2 z>8PKn#EE@VG-t((FOVbhBoX_J*GyD|VtNN*oAW!{pU=D$OHrG!K4gBaowl8ta;q#0 zMK7CA#9XN9q$srdTPIsWP;y5NkunQv%%0i5!z&>6NS)n$N;F#uErH$AK+s^MzD*Tw z3^5wX7V+s*O-r{69Zeg*Y1N+3JHM)PQ@mg0l65>&_O?{{N_+<4RsBiApYiIAwI4;1 z4f$x+zk3%|1e_jn{CDr7CO8X04(GGaAwJ(d!P5H)HdDaFZ-+Ow60iwW2N6Vx;V(LI zzLb{u)7Jr8NL5E!7R2^Qul<633(?G9blA4|iRYe3`DTc!$$&F85d4ez(Z&a@ZXxj} zfAJ2dv%}SwDJQ(trRLSWv}#Wk9wCQB2dT?KaOV_O{6WO1SQCV6%KD__zJ(AH~uMuuDP zo?L|FIxG1u3d%)WTW+e6v^p8hGfy+?S0*EQea6e_k(Guv(qWidyAmx*SzBWqFd79V z-DvL#Yn#Z=%LIggiRu)CKT-e22rVU!I-V_RJlAnib#4F5hzF>2_=&yh+{J)X5KBQs znesZ6o>)scQ$NX~JmJqx<+*|$ZA3+M8Q=N#mdwY6=_+6tU;Hn~nt$eE7q8lq0R5l#CrsoxdsCOT9f&ewlHQJA2#1 z#no8KdyBTr*(uv^7t#l;v9Dmpy6Ha^g&*g)A=={x#4CL*f8jj^2w?c1fgiX1d4Z$y z0!f4Rn&0_auch!Zuf_04Kq!ZO-!cRa^dKPXRx{0llZAxgF00oXAY0(I8dr=;^lzUcs$r`3_&D9an4JFyM1br`0)F zxaN2mc5LplvR2O+lZ>33`!l&z}Ez)WT zF6@L^X32R<-@)RewJZnwqNov&px~6mb}9yutZ0G|9EkpFCLuap zoDrPSY>|G_(a$zNn^qa7oO3mAurqYhalTEs<3qT?{f9N#PZ$8=v4gRoq()SfKn?dS`D z@G=u<=wN=x*`EkTAGpbz?-(1S6xk^%CD{dW$x~j_jhD|Cv)4C{T5V3SNXL40!x*1LsH?7WPe+Qk-4_f z@h$?o4zCuL>XI@Qp};t~R+CQ5?vw(*>xBfY(ure;~uMVbHe)s^4W zJ?SfR1t`zsv|J^j6G{2)?QLc&;r^Ufhq$6}Y4Yqm$4@z3l)v@5%*fm6p%Trh-HrF* zyLluhQ&hg#3ruP740b&f_{5?pbV#qt4CG8X(gF!@nwg=eH8j5#QFpEO0J9 z(d4XbUne%VdDct+w6JqfXGzv{mc91=uXLOo&`9`UK6EIs?qBuSM_%iiDjh}-FuIcT zQGa2)&T|Jss@B}I@>s_pN=Ii1YB#I^r>*(k+1$lNW`$FQTZJEZ_WZDZa>z%R)>z}T zOvqG09Xz=lAhxRk+|_8cN7X?fJu5aFW?MlF&9p57G}51BqpOUQV%vOXKU>Tt_}Gs=Ox>j z6b`!VG~`r5)lbF(6y0>r)H+rgX0aLc;fSNzzjzvJFktE!>?~(jvf{(P-WTRn>AgE| ziq0sWU%0fDD~tSgY?*cu*mSHZYaaBH#q8Pb3UF=Dt3=$DDo~` zXtjD(6-(=Z1StS7Jf|^)?!+a_sjX#X@RuFAbrO39P;jJi5eD4DITd}$bk$2Zpd2d)8H-Kx14b(?7|)+(+!Hx*>9f$> z0vk}O>ebtr)YI()cx=g)lFDl&;;T8zIn?6_q-CVLqE{br%$bC@pTOqAaO;`0c_#IF z**UINqny7J*p2U17r!nlRZJ^Y{$~Rp2ow9ELSyjd-z@O1tn~0(wqQD}VsVGBjk!(f z3g7K%-BGEQ@VFAaw$I(>%JEH*{HTbd-Ad*mfzBNU-?b-oQ+osAlV@78p4ub(<+G}u z?@{%qSL3kI3}Bf-V)Nt-WFk))1s8;Nv#?_V3URHS^sLCl`NHWGS-%gObrvuCztAn` zZEYDg4nyDFn%-zBjP=enm3uH!F_Mp}R*msVmoy+Eyw}ARJO$eG|KKy!(c;`qNxrzp zBGhngmgU_z5;^R1arFM+>=BvBU{UZq;?_DiZ`3q8>A9me5}NusTG{+xS!J!lyYk+zuEtQZMPPqb#1i-b`=}Uum{8GbaQfsa(1=owPTV$<2vJ$rOE|;d9p!?v{qf zC$u1(tUsgy`v-Df%7BUpeMHEysO4I z#4DbV#K$jrSb6CSsyJ+^JsVaFlT!-SP3&5JqP3(!sWRtodS!g0_UjMe4b6o73vA*) zV?Ox*+I#-pERkhf_@lq)Y0LU$@;{gU|JP`?fAtZ*m3zLPCDv)6>Av2s+V!f@`@TS< zH!+D=+{@RaK)i*ve6)>0Igip*2}f?D#yya8?sV#Y*jI>Np^wIZ69iQJ z+b-pJoipWl?UAY*F$rTo;hK71H4G0^at)og^LC%Za!zhc{fs926g}>OIb_ei!B>bM%kYU-%ddvW%WzC1|&^gZF5as9Wq-vHNN0*>tb*|kOU4Y*E% zS*R|&R$o>!tE4R8)AB_s=lbltqSo38X)W34G$2Pb7tI|=0bA?190hv`Y*8yj4~KwB z*z;B>P}@!HS!a=aq_^hy#)emQ&pck;Nz~}(9g>&{Lezc3jX-51JWG9Y_e~bAXV18R zNN<+lz@K<{x1xY%1rM)Gfdb=K2~lbn@!?lTOf8wRyAyIIP;G{su^#tQP!4xf5Rct~ z{{UDmgtqDUb~ijrSx_VMXM*NufH0??j10ar#egthr)(| zC+(`Mg9KBH&chY&V44HCqF3jv8^j|zujB#ZC zS(}9p=!TU@``({gl*H*84J-_* zv_c}PlJV7v-58&3yx&A0Lh)`!TBJH;8IF&-XP^E-Z!?uO#s$_#h^bZw*YQ}H`O2}l zxSA!J0kAIxvgd4>Vy})TVKzfLKAro^7wQ@#WyS-C3sa{}v)y(ds#Ayr+w8zbhg&xO zPYXQvuNK||xsS+^u&6(z6~aAq)n>oVic-ZzDHuGNW0VMQF}x4LC%i`v&P9CZn+WmI zNH#1{RWD3$vD=;0@wm!6jqv2u&~Wien3l-X@>J^s7^sJqie2(lTFz@rJShwlOd#z$7hH6EN{kq{=-Z$!*>Qw05Vnrru?xr(|!|!K)K$G|$UZV_oJdX}2=fS$yP6Jxwc9 z^%w@C;Hod`dc^ z2<>Hk9R+b_)U_)clx>naYh!XVnAkJLw7uz&5bgJOC&U#p&Xp8q$L3dH)cIvEyuiP1 z(;~qg0T^kiHSQS2T1TkJZZ*KA$?@jY{`Q%r))(ptD3o4l&uTc~@s^@ZwC#@@s}whN7XH z`a2Azwd{P~^dHcsRJRUBdTgDgsG^6v(>d&hY+QT|&xmb$yLp=1*IUxmI{Ya4b;+}P zKiJA1_OS{3T?+Pj~!Tm=IwFX+@eLJS$2L#aHgOVW5Q}9RN0Gm0T^@pG4I8LonJ;L5D(%&y*`PwJl zM|+-g*naKOCMlHwAwrQiSoh} zxSI1jsGY!ba^001DyD-G)e{gB_Lb^*!MG{K!rE;z@@wdv!JDsMct#yWc*3K9@u&k< z7c;J9uZMEJAgCGO zCXKEL>*1s+(2~hd$swf5tgN%sSegW>tspNL+E^aQ=tJb6)|oh#z)isy`HTA5v!JtP z^vM@MY5R*uvOnO+8u15b^0F!UYg)3w_A)*IAbRP=(Cf;gB)!9^5WIFWulO^YtHwTB zs>fK)HpcStv8(MR9t)LYxf=U6!9!Ia2}~<0?}j)Yb%5q=XhP+Ll-HA1&}&7FxEJr~ z=8f2s5XR5%&r1a?2IV)dXjP~9=S4*RWJT)QKA??UPYo+jr;`+PFOW88KfZhxWWc04 zIdH}lKS{K;(fy&#J?V7_gxPLmufAe$h|*YS=#!Ji?DAsUJeJpRImM)lF+^Rjf_JUR zUr@pzV>Ukln%Z8sRD0nSfgxBWI$A34`LTP3;qIo(=0x_y{J!ajG+dH|o*HYx9kfdi zLM)RmmEMup{{fVIgyYz4OqEGbg_iq|r6IaYx;-%xY3A?A?%5a)p=XEM{4;AO^fk=K zI+PI^owEkov=!tY?FD}F9>YebvUhA!P-++M8& zm0Ud4H8NR7&nP%)V88i~b7(l|zuMma^M1G@@N@?+_5+fMN3pS*NMBy>en`yphxGR^ zAL2g(0lO`G;P;@u9OP^8rNF!mF3#z?9)F3BI9S9i|JE+?QM11aG&-PzK49wsm>h8o%w>yA28VuX@a2&9s+*+euT+jB z)|}%f*9$-b>u=!W5r39$ih$5e73d7q;yvUICJ$)igAkPGU|PuAX-5wo%H24o$f2Pi zuNTe0)k)xZSYMxp&dXZ=_E#d*2Ko|sb2+9DeS~sM4MbRfg{ZF~?kxQXJe+ysT;dbB z`L(>v$0DxOgikzj^MnWFk&t*=t^jo7M7N-mzp%={MnMRa3@exOx~&a)G>cAY{gCo3 zKehHbLv;){`>-z^;|{0ZrlOrvisxgwvAMHVuPh}0PlGFH4w>59=4g(o^uD`^dn+0j9KY>{vJKJ`a+Pg6cIodEZo7s%Fwx6UNsvZqHpXQ{g{$8~Fv3-MXDB zC_OiDK5Nys+|o`pYnqb|9r=q_O$H=LwAmN=0CuTOka2XB`|aA=FPD7B6%1T9UDBs> zF_OZ=Y8=8Ne*{0Bbi#g*^(a!;kqVlrmy1VBo_FYZfR89CS=(QBt_BhFF{XNnb;k99 z(QsVmALrnCv*;3^`%aG=Tau3|Ez>Qz$D;d(;`{JBc4}5VC9t*KY4r521{5%w9TD7; zbnmFTN|T26V!WndU-}U+bzC;iX|!gnR7zO4+w_IBU&@$`4DfQo#GJkDjXVu%&Gi)><1aaRPO6zl?Gb^sV zb+~oMF!(n86Y8^Wbwa$M_)KQz`~9?6-2WjgB;dIZQ!nLLH&E(qKzo;4-%N(t#@WYF z;Q{{rvHmvgwezHHut`3`+xYZP7Vadm#eN#i35=J}U1zUz|A$Zpm@KCGTEqvmLB2RU zzg5@psja#^z^UFp#tS#G?*{7|N)W5Vj}MG}k=wR7%l&ibn|H`CPj7+Lgt`Rupcdry zam2&s1CdDgRkbL&|Peg8g~U%WyrtdLM#;dr8WhkOT(~t2{^Z{w>wJ z2VlS?giDeb3rbO(SN5rorXIeZQEX}DLrieTcGT-uck!cdR%CI81X$o)`(jv0UnU3D z*VgNpKPfBN#O?5^!6eF466Q~?*n`@h{m7b(p=Xs~RvoIt&V+MMrd%Dj3QN|HS31`% zw6?y)_6vA14ps?LRnIe2dOI$K2uA-%!)I#=HYndq?^%u%h|OOdahO#7)oQ8;l=8;ENpv&vSU!f^+b&%XI9rwH*W3H+K@hHyBevGlKXm9NXy43}?|B!s zU{|zb9gm<0v;gKWo0J|K#3^v?y z1wy~W!Gw?N7r^9L!={9DXERwpQuMna=CHffBY~-8uGd{DWZLUWgV2jL%803|y1h-K z?&ZpqXYX40?MI1?_S*6sm+BK%G|L(-_pX3R zZwX~5Q2+nMG+4m6dv6M-!Z-v1Q( zP+!a0v<5koDtPg=@~GTl6>mVkJ4kwx13$$wFyR?Xe&D;6-hgOh#ST_dAOO}UZMb$+ z!`P@-EyHVrvG)^Czd&DL{8TdFXYV2WF9lYdJuGW=3z_aRrp!=%|J*?=6OdN3 z!@bSU_MUPODEc6wRV(DU*(}ZU6!nCZu?#%26ijWJ*nAQwnQ8p z)SP`{Rl(}v+WxRo^uEVfI?dxIdv)eHC~w!1DG@wMlKaL+m1oO_yc|_9K&3{V3x8$d zXr9KJx})2bPnstqf6#9|qcPW4BdFJ01&3XYb+`L^j1m0SEHRSl>Et^Xk_%5AfASx@4a-d9j^amj!d#=BG>t*>Jq?d} z^cgu4KNE2z8kAiCd8gc*{Y!G%7R!&@9L^`w6TG!lJ{~n9cvX!XY5jzU#QZAM%i~q~G zU3!pI^^u}M#Mn5LC<8i)U{G9h!M<*3nQ#`W6ql8IUf9QX`7yHoc}tPwF{w4&qE)H) zSKcqh_mud1*&Haz_R<4@_r7Sm=g?3lZuj`b@$ro<88H=apjQ<;Q^ss8-hViiKq%eZ zg@}Z$5p`)vz4-EC_6c`}ufTf?--T4ig|}5iTe_-VrNI-3GziBa3gX_A?2h#>c;KYZ z?J@73{_#G@X;6{Id+W7k^|cPGWfk^GTo+&G9&Z7|++1<*sIHgsicS4u#<6G88Uo1g9}v0)2P+-N!!D7d1q z1(`In%wG4!iI?PCC6cCADxo{Hg8|?4TY^ZtD;kI{kh5HCQ13;T#1w8A@$hk*<(jsiqi>Rq-x2QVYEnpbMHDe*bfA zW!Syy^Usj^#Z8eu#pZVVWK29#uS=26BFbk&BWT~Y;jrmdpG&Efxm-_|!{x#jH=|6i zi(0o%mp0!3($nRV%U=jo$EW6mF7X|t_l9m7v>ue4i)=U~vVIMDH^dJz8Kk;~k*a7m zwyo?!0d8#PCs1&?*=*<24><)U>W`9hYa3tQ&-_eE`wU-NpApTH+=n&ArO7=)6+xEy zCgEcx3X1ENRxhN~jw#cs40Muj#Xd2y9bNQB8h~PSZh2*l@5;L`N zWDc)ivE%;BF=gLABJT9|)sx>uT`e!9G&kfmPK@k=e1)l)p5rq$dU%ny^3N77U&8uo zR)~Er2-5r@ipTq=Hx~=4V>2ToB}V+x2cE+T?TBYRz@?eg!UZ}CLp)UUK3v&P(gU?m zwLd(BzVrdFd-slEI9DPZkKuHfbR)SZ0Ui@f2IQ{!AIaNEIeW@bWT!wno2aURRz@=C zTJ=@dHB&%`kpk__Z(j34!{wnh9mb^=K!AL|pHIyFt((KSW*EVNI;MJ@So6G=OMLMM z;1`c#t@8dV=$X-~cx&%qU59d6`E6v{cZ1(QLnGeC6+@!_FtGi3$bW{|h{aL|IX*o_ z2Ym4s?QkV%N>!bzfid0M>|@Tix~I&hPE=yYkA9WdJ?y3X7q8Z*GqMYW5U|&v{5@4W zGh!qtSeh8q2tkly|Kf?QE^=AHo8#W$@6Jl5fEIB}FajLl6irtI zmbM&xix8NcHS{DFNZeCk8{ec$4`0}dU6!1Qaj)V0%9x<_^`%x3A=LL~2v93c0wQv1 zwT`paU<(AwQ*x%crK(ihr*t*574L^#!zlLYj`SaAuD|9;P?N0$Jn_v?-~{ZjLfD){ z{l%NB3g`r!k#`GAN_-K!3hecMtTl(r0sfw60N+w;==v;c5CJ)a&F>i$ga?%n^K#c} zA8d||@VzbzO^iy^xmv^sqlHc^=mZ5IUspUu_b%vQ76=IbKSsYTttd zc6{Kz+eW#>hXl`qZiusxHfR`1@&^oSjLbfv-luy!P@h3McO<0ukv7Jnc%9e!*<1c0 z=FG?ZcT`9eG$W;nxHq2iWm{^Wfyt(mHsBEk)BFf8jW_$#=L*4()(-U4e|C6N;Z|8o0mCzM6e}*42Nev^i|K@g(nsKK+j_8VgeQ_&~7G%>)AJtQg)#zn1mf0G9>< z^mpQdi51FPTY7Z7;X7P>W?*D*riwr@iPK_S+2_NiN0!bxSQS*!M$ut-MMwV41PjCT zX!I7aJfCU3fC$_3gD(%x=2 zassvO9BRH2WO+Iu9eBFq!*k|13Yhc=*27!aWt-8;T){_1H(~{6KDdUyG(F zj;j9e%*<3pe1mhkstn%!59Oh1(7_y}E~#vVm!TX%=^{qcq&MSoo?9-6UzHt4dUL`5%PK*eBZ9{c{vCe_O;(y z^5yt8%{>9W zCRTso=;{2*F?*o-I?r4EFW-paQh?_2qZKo2^RRqe13mxZ$*bbEC~fPs&MH=qBM%3b zVM2hl{;dyna6T7fkiO+`LUMwJ zRz1I)tSnbj#x&XsSJb6IH|%1F*_JL9qRC|CcQkF7&`iM&X}0SwjIa^~bf`hBab+lB zm2tL&iI;VL)Uub3X_o0;mkHWFmybg}|CEND-t9YzJ(LA}=K8?yPInglU%Z!(U_007 zRMbrt>iI@!dj;lAL85kR0!PTER;Us)?|sLjIAUd07Z=rMG6YxnxUhiDK{zFVrA!2@ zVS9uU*NS0tn*A6*g5acg)(>3Ww2VqLmlNz;lCs9eFkQ^G0<-tOMpb6cYdK6@Sj-Nv zPucHKVu1O~5yJWUK6K>5I40&AYq{<$rGY$D7kI4c7fen4X?&0gV(=G_{oef!?@-Bk zjz#zeN@_g}nDt^%99X8z1d>5;g6Nay%}H6y;ZKNU|r7hHXF-8fw`@Y zYfLzukT>Y+6gg^`bl9;v6R(gIV6?J)Kx<^}c#^VAyB#BVSR4wE6dguhnvyg{oN0)! zZwY$7QEy!s9T9|8)EmgXL*Rp04QNVw8o3ACmeavIFy}PkLkVQWMIQ@S8zu)ua-rWc zWqkFaKHE2KbE+Vo!z677Fcvo3`M6vP)i~{q?QtJEUQ4T>ZsFQAHxAUnYSnT6p^P+l z>xq?QBdxabl=0Y^jADo%nd+q946W4mV-nW|rMl)hqzs@B$5&%y)7CF<`kcZW#L0G2 z90Tuovs8qVOkbeG^Gtl)RqG}JhkS(o9iM3WS#32pLz9tAk}?? z18r9(j(daYAvqVRw=_RJ7dRq)&L!MLW}I zS1lZwRv+8_H}>8;uE}*v8;+GCib$6dl@0<*ZxRJ50U^?*Mk&$+r1u0tdXp|SN=Hhh zOAWmv(xvy_TL=*1_v~}#ymQXpduGnO^Obq$_x%wd2~YCeZIx@SbuFqC(ZsM7kk1X! zrEV|hGYMSZT!qhgPFR_^%^Qjl(L|1rSM;`5=NO8gt6p7r7k$;?v6KojOdH ztwPyZ97@Wi0RWVDAsXo@ZJgt;%Q(Nl>~xlAjmySd+L^hI7(`s$gQuP+&xZ?uS!{PI z#7nLgAYUi8wk&vU=)ImQ~EHpO3uN9TkG?|KqRp2Mldu**tDm6g1|K;((1 z_HqcPF)#R@ZKB!risbo73ZQLSdMlNhmBziNApj4#$Zs*=HNzP{j17hfr`plz?)a+k zUb+JP5O25ClP6$=h`fVax45x=%Dy9Z%yoc+twlwaJ|IuU)u2@8pcK|Agi2EAHIcg4 zlc$hOu1D#e3IrgvaOW!%PCDHZqIfFp)9Vt#d*uFwwnc4Toi;m3#1k=LEG$uNMAPDT z(gff2yLl2w8RpKC+3l!TC0gS;9H@DjB{R9+`RewVGOmQ@d|F8%FK=?|Y@%yRFYe;a zAAg==vp5mSd(N<5b?^}&K=UJa*{w8E55;sU0X;GE4%vVfK|#pbbPu!gb+o^fAkx$J zIgMvcS^3)&@e69EmrWSAnP7~jKY)J@{{o(!dN+x%Nw!EG!@ObZo#&SL zkDup8DKtZ(n|7w3!?qo&4wqze*D6k1!z!xlYDKjN^PsO1FSXxG`y9KgP>fDO&MhF1 zfNZxVz+x=ZNq4)3?c3Ff6b>q2iP0G3+kPD~J;u+ATv%GP`NS3U&4AIgO6`E$%f)P( zb2rUZV}0o1uFaZd-V@0g*t9&?jI{^vI7XV^S8%g}c|4m&TW^|IX=GztWc2VAR@CjUTob}rr}r$ITu4mL2O~Z58WhN9=f5MyR`cZQls}FvG+w;X~=39Pkbbq z#W-6~A1~k_;RxD+rosG{M1wbIkFQW&qwX{=cf7UXNf{KCi?eAPM01+rR=jytVcx<^ zNdCaV1VeI_^A8XU7hHa3WguRuKsc4TpV2@F%>!>^Fi~uMH{Sg4QaoE|G>HDvct^V) zehFu{5E>aUEj3U+C9K?;J$XO-L8I5=>fZL91qT{~*##HtL!voM#FjLN8?DNy({t_X z@=Rwp94URZenek&sV-Te6YH1_IYVB-ZSAQV?N2%@&*!33)>dQF;%oJnyb(0p6Xrc` zQB9$&tp%@XXg+IfhaK5?NILQ->9H7xXg_nxM30SIJ03g$Fe!l8mFj=wNuw&{|of&o41izlD=}F{}`EI zbk+x7c}+%7Qmhp^Z%w;;DrUl3WaQXoz{E>Ko>jiFqw8y9BZxpfXD_0m^Qfzsj+ELv zWnC_d*U9_PlE9G{y)YNIkURgSIqiX~{dT=8M8rAO_PE6(HlhA*Z8fGgnc4LCR#{0s z=+gb)+Qyt-Fa&C0*3Km0c&E|s9y}wME@}+Xjde^#% zMD$}l)$Y$2*6rJ-^eZzqPBboW*RWCgKjj#2KK2z`2$#vUx_U8}5fH-{;O`dU3gyaR zMs3GXEz2BNy6lu`!eMf?%8m`(x#~3Aa(=X|shZw}X)sw^-A0e6lMQ1VTKaPZL5sih zLz>)TUl(_~L;pO(+Thu?Y0^^MTl${_4vQw=ULOJLY=Ctjvhn5H6C2`JUt@>3+U4jn zqYsr~5RJV}RrU&jf_$Ci*y*jB%7a&O?vbth<1n)2OvzWk)xwxE>1vcX+9w}wM+FaY|mCVmS z)V4WZtvynhv=g|)Hujjw!8!y;2(~1@g!5GVR*PyGm+P7cJJGJ>k{V!~z5JB50)|NK zyu^-y3@bk%B>{LAJk~ICP+T9uA4~}(A{QYE_b?p!hrYd~bi#2;w&WyddKU>(^%=uFD z3q^?lz+iqLeSDcBrUj@1H}z8d1^Nsa09;rM-m1`IrdXlnk%oH2wJr8h)!>8b`T0|t z2&s$i6oMF56Mw0w6tbx8`b+2U@l12Q!U=6Ct3B#Pl!s2#mdP`kUmzVuRpJir8_Tvj zEcEwkpJBN7AmI5&pL~jVfH-AJhLEy9;g_I{1bKQEb_e(J1*uP`Q_Cl zf4qaFJjV;Hem?g$_n-)c>chrYLpL&Y#wA2IQ+E$!hB9hOc6;-?XzI6YyM_9EwliyYQwMot&DiwN?&nbfh6cNQMGvA)vFR>_vnQEYY7m!9Qxj)%Q9jo&qg$anVkLFoK1<&llB1x@J4O=kRjdq?4 z#b2-J7j9O0y?;^F?8faU-hloxt}AwMnw2P3W~?_7^hbWl`BWro|K&nUYoyq zNi48 z;8IAoWQ@EY%JTFMUyroS#O?aN^90Q7Ec~L|DQ5xFis>Hw!i=nSwTR&2jI3D~a+{<~c(Lj4xL5-zF;mTI(;{^_U@c^6Isi0~1c7 zJGaEGa2pH!cuIL#=(P7J{tkhoUpUVdGEv}^7}M)myPd?FY}#GFY|B*fu9o!hLH*r5 zB695<-}%~VH9M;^9Vye8$~x*^>+Q{2gkbfl>>o_^&fEW1jJUFdGuDPOM9DrYGFh`Azl$@ z^2rOn8`tK)C#Y0DB9x3dsu=Xu`gUiZnB`7M3bWQnM&Bp+J26;8T+W$*+W_R27igHR zKB-GIT5?LNKvd0N%lP)RjB9E`tfl~CTcN*AdVTJBC?J>WBu4$qSAc5f19RRXwGQ4v z_~7Ai0QS3y_$}}HVbXm0p!1jErfOHEs&u>g*K{EiW(II2q*lIaIZb@<)P`r?V5~-@ zvZK}Zc3e5Gr=7Lklrl7*q$%AlD?Yw7Y_P0)dcxIP7&7}AvMXyN!o}}pcE<87=~4Cw z>iLhNl)9`>7nGIlIQXLGOkc|`-|V^QugZdpX@<}tI1ru8_d?p|uM~Z@&ylikh?hv| z6RwJQ6i5@pLnV_thmh&2@iAk24E~W8o%&U%qiW3i|X|OuMXDEO(AeRny*M|NHiU_2y7S zy>;SmZEq@EW_DFE6E`5^o^vhNTuVswZcX^I}z?o$O)6P50efP^oArYcVVEBU;5f$Iyt{+_A5O zcr-SPv(1@>=#ly60>5KwA!F>D)%1)Tdm_mpl0HNPppzB6r zN0b!P?93Sd+}w>}T;zTaz2--PLB97KT*wQ0o-Sp$r`K=erJt|(=5b?rr5c%lLk?el zIF38h_B-(cy>LrEyo1qjKESheFvVKOY4y>g6s8Q@fpUQ2WNYID{KK8-@{;>y6K6J3c#EQTDF{kBD3-lyzJe|ig>eN40FTWbMu9s^FWhTl zm#ldfBDWSkc|A(9k(}$XcT3Qu@a1DkRC&)97t|91jm`V1RCIw%ez)P>>Jo} z6b&jjG@rBcKSI;=n7Y;tEm{=5!ME-o(2kp1%-!Pdrnt#tOZ?t zB75Z+@~c^NP;0eqH1h}5-UEJj&pe?CxNuvodqC5x^_O2Mm-BgAJ0IPhF)EqjdBj{< zUAiwB>J&hCTYNq-JFF)C>eWkC3D&mHNtyyql*t~f0HB%3c^qc`CjsDA0LpF5kgbmZ z{QTJ0sc#Fu1v!Lm_OUndOKvg0mO}HJ*v^F4sM`C%NfF)f-KP+8LT}PI{xN`;`?|F^ zCP;t-&2RC2|Ay!;Z8;wJe{eNc8me zomsSpW0k^LMXy9`SfsmX*%y;orX;dZi0vRv`^MyFnmAy)k~z0$l~v2t@+TE|bx2 zlfC8k$H$w_XMzTJNPCDEk~Sva@Ui;$F2%Mp&I$UceSwm%3l@!9udjDg! zt~faud=b!f7U$AVXzpbwG{la5v3IpKI?(gPl*~F*Gczv536fcdwss1u*qPg=p7m}f zRHs#+q8gF!wZT`U$Z5hqLwOtd%&Wh}_&Ew*Av<%BA>;PR_yi>Ozc`9jgAclqT?O+r zYc##O^I4UIGpj`TbZY(B0~IvCr&TebJwf=!{v~F=jy6RoCz`*e8`7f zk^vlth|wuc9d?e@^JeO;PVOI-``;x_4`lS+^?RlxL+Ph@4&+nnjDf7q22%{>L1a7T zdihL%#JA$T=~QXd+l{(Qai$;ayCk9u!@_1$n8}3$;n}ydrd+|hwt{$ zMT<6GHhXc?9dI>W|UsZa4-&O*m zukhpa4Q<$SEYx@xjmi)0KjrvxSaSrC%td2r<<5k0;1&<+;rD5b1DmZ~)Rg1;tb$FR z0%G)+Z<4?LODwzYOb*d?8rbCR_JBF11K
  • $E15kc5WxyWN-vTJhPLOxMD0n6MJDl?sR2trsbS`K0Bb2 zK#U35zE3V7wG}^ku-_3FC&gULXnIQ={oM*PhXQKn1>xnxQvA-mapy57AA0GRV6u%s ze&7|=CzpMRS#``W;QEr@rbkdRrmd;$=x#9iYM`9ZPANO{ zS?`gCPH2C2D$1#E<(W1EWsLEXd`^va@+#I^>vd!==C+&saf8wQAtAe)+1aoe%b{Q3sU!toAdVhhGXE$Ef%66P^Khjgu?r%$nPPxOi`g6>O z`3FO+**yU2hITmhHiFZOlVn~JU2Zilv^?O>`cZ07!Kc4t0Tkw|!NC^x49c3~(TcP+ z)olX@3#`+}GtA~wvw4p?;RmZ-1~!y4PI831t(NGzzDWRiGYqiO&CwXWbP28{R}IB> zy`5M_m=sWV{0V@5{%1bYa_%K675eFrID7>I7M|jtW}s0$w|oFFpZ>@Fn_j!sv?alO z%FKtP%yRYCMl7S2pUt^`RiAk7>~i;o|MM3MKGu!W0)%OvxxSmd*?5kUuR{aFSZJ<+ zq!cAZY7?tWmXBCV^4GU-iJQU6UM73j3~+v)*hmJe(bMGo+8~aPB{BxS(|X=N8yikE zFEr52>dZ-*EL3Pj9n6(v9fhowdus`ey4$f~nlma}ht3rOz4+Ugy~>OG#?)P0rfnen zrGV#dRRh)qOus5T*C(8V4$fI}BdH)no4B{^Ya^w7K^c8PNfJV?X9%BGLGfL4OVWjdV*H;&8!n$;K)=Qz&q&;I zTCn3bJHgxcg*|qq8{#-L!9_FymIJL z3b++(*=R$TMSaY}vR|vpS1*#0AC#Pe(iNGR29S}W(v;s*s{DJ;PudsyHbrlgvL@v; z+F#G=E{g`Z57;|Q7vA1ve09TZyoIAj{BbS80onK*aoG@O(j;@xa$887aCdiUlESEm zP2SChO-3eO``(jfgZE3LH->-{h^(cv|M+CD30{k=;q<00|N0@;gp*nfuv^lySEWs;>B!zh4wftMQg4> z>+RVK{VY$?a_E72=tG5IDXuJBTceGaZjP>%OUJxc$oiEy>V`WH$Zp&qp+Dpp!9Xyx z)BIr`fuV@2ZZ(bLZ8>aHmGAi%lG^$g8omas^f*xOb-1g$OvL!bysJU=-#$+uzx|>> zGDAY!x7J-xyUZ{?SvNCj?RDh$y&Kug7%6jiNu{+77c|D%6Cy~|G;<9v4iq%=)!t3F zOtR_u(y>1=QDEDpXk2wk(&5uW&c({y^vdQ}oG(9bemxvw8Rud<=Dp!DygqN%r>&~K zJz=-^JCfzUm%KcwdpRwacdrk}Y*dIISXP#tT-_eH`t9s~;6>qR)~mA154j7zG}~*5 zO_D6P&aci|N(?V`U$L$a*&g*ya5+@g9#Bbs-aOQ*qLGQ_C|SoRxQ+o`byKOS)C}40 z%Kg@qi|0%h@lzW@S_FpfKGhzt{|4e)mWv)3ys^H<-afPJ`|%BpwT;IGA|NB&_ zocqVtLy?kcX`A}?29=oFNi$dMo1_{KME+dg)mH~cQV3DK=Ly`A?ySj}=<1`9Gi%TP zxCpj@oZ2m4byp#<=*W)7uoXZ7+Lu*?0^c(cU5bX@xdMt-1l@XENQ&FQF5(hITQMQG zVeh1wCaiA9wIw=Z9j_$Q12I3l?4Nm5&)cK-@3KF{>n}QyK0H7Q?~0Ndiyccw+;U9d zsa@}L$*6Jl>{7Sl+Q_d_VMp|}Bu??scBi_v8tOfW=&TP_4YnSXvk?-(+X2Z6Kt4od zZat%$^%=^%dI(5gaH=>1%HC%m8^dJ@y>9$1=*Toa9LB`PdXHQ;;dw>bhi)nvy1B=T zCiX9eWp4^x5O33`>+AypoyFw%TP*VmTO4~f_X&b-uB{MqDW#sq>Ak?ai2= z`_WF!rs(o_UII_0Uw4i%VhS*&02r4%3xqo2T?p?mAKrwPefBX3P>lf1^ zajYNWD&1*xr5bIEd2AaEKR+gn&h6a5Ap`XCe46Am;kG&{gXKk8Jm02lGrDRcSMD!_ z2}+HL8CzGzj6Cv%UPR4Q&uyhB!c^C}eN z`GM!4hCm9$@~99jm^4pPS}_upS97{n<+Gqz?6@rk_p$s&s>bW^l>Su%Lp=q zb@EILT4ivMbgP6iO`X7^ZIoYC)V%C}dH+SJXab>SrW7c-xxNW+1ydBHN9xyyU1M;6elq^%DN}37 z+Y&|M%O(z-jdyBt&tO=u^TcwyaET$obqYhvX+~I@^>P?`oa&(lQ`M_LW36|#S`VO| zO_lpV$90|1JtEg<TxcO* z`EP+1c9bdFny-)nUXru9kc*pXukgqT7wXa!Y~~acV8cmQ&0af5&7w*iHQV5>b>tuhDgQcBOXz^aS+!rlpV|7nH@B ztSIxPFZpGP@p^^x0*sj*?#XQ=rPrefsF1tiGN!V{qT1c^>I|9xNQ#A{u+O14q6 zV|FiZU29E2u`*?>TdJxo&kdk{ns^QFO6Wn-1-5|$v@0bFm&)6!Or=txEHK`v$_EN& zYX|88UHGcWs}D52X?;?hkH+}}?H!J1Q}U$u;8e~Y<(`=d%{|lYQS&)u-bU#F+x%2a z86E|-DIKdgu5`rImjr5*Bu%IX@PvB4x5x_7yb+y2Mr|eqdsR5r7uYMxJb&)u z`M+!HCwm)ZqS&;$5>!2NXhu6*k)g$lkdm3U_c*~lZpm;fOO2;PF`J`vS}Awdqu*aP z&2VkL8GZ*819$a{iu!JSz+8WJo-EB(+1lqXq8Gj)IHM0Pp01L6!rJjg8It6`T;+z*C-S0>21B8)-A;;tR5~V5AZEyb2#9N_RNdpmt#a|%o=t=etKF1T- zoJSIl!kPxe-{;eEh;foPr*b0jKh*yQwFjIYF%GA6KI;qB>kC*O z<_ickUvzh|k7Bf`rZbg(`CO_GwajpA^XMU*vVGD<)@Q%%2gUWp@yIYiwr1`go95 zvCwHFXz6a8W1_(=7K1H%aa#xKulD;u%J?{uA#Nr=VU$&*+Em@HVsPH!RbVdJ%#7Eq zYab+1t`JeTua2|r1RZg1;vZQ$Bi%*3zVXgmnt5GRrGCNzqCC7>2cMnUw48?o+c@)w zw2t~0Y!RM%cC#PkAa2mGbGrDrJ3Rgl>g%mM(HCD}=G!w)Z+MwoE)Sc{$<$ZcNy{iX zC`*bMH|ja>Tj|w^+PQEO5}fMqM!K+4j*nxY zrFXIiFP$d6W9DE@TY_`sel~|lvTO=z_a%tOvO36;A;$Nu*eVF*R1}Rdo3*hUN$`O|lRiZwx${+Fbi(16Rbr4-BgL?eePy#lJsA9doiu2hh+ygj3n z4_LBgcm6%Ut}p@iv;O)vhQXNqipP^b&)Oe_ zQ+J0o%$43&4d9Ell2zzt%`8eCELE{MGEA%oQ|l8!tYvI(?a&O4Dty^~3%Q6i+qr<~ z8KK2uE_#Vu&UM?Uo7g)u)s|FuEhUx)y0nSj1T~OS%TsIKI1S-ZqVsrRv9IPEW;+!P zHLR&L8?KnxKDt+E6}Qx#~!DU5dXgvSeS3{~&$QM|HR`UTR(KGa-*D#xU9c3F18p2nn3 z_m@DY0Fsd(DS|Te5aKg88;sT^uyiGnj2F zpggB_@c8z5-FOqYU?G=M=5mFqO_hGu#6&;XStpJ#3@LB-0|o-QXt)lh5blM*hKxo4 zdNj-Zw)w|}U#ZneF6T3S3V0cDQ;-*}UrmFEC}G+RUvKF_8Dzzp37Yn6_d!)J^4w{Ihi zHC|s3C|ct6U91w=EsoPrJ@)|!RP>I6j+E_gZj4|5o>7?1UNSg#u|4~(?&~V^Ce|Fw zp9S&*lAZ6s^@L`~?z2MfM=z}EsY+;Xdx3~QEO7jKbyD_W z!=*#fIaCwVr8(x0*y*dqG}8|0eLUjswo2R0lFT`L?O_+vKy=h66*})*_gYDwcY9|s zUV+WebBi8@toJF3Bt5Sw$11Hqgnq*mx6cDrX=*U)P7P{u-&l35Q0hoYsY*B6Wpi<& z)QIT}f@t2!nI7MO+}37yV@ecyRiVjBQJfw1BL4>6iOC&&P4G!{eRN4l;Ioo|36%gI zPfw>8=^?DdQTM{GX#FK0Y#=Kg1DjL5-t|rJ+L1v;9C{DBAuhd-OjqJbS7=(Pm5Z2{ zAj~jGVnYTV8b+TRPOSd~)a(&Qz<*>h|MosL(^86re`dw;o`MwR=Lp5c=biVWd<)2b zYa%lgs~)QvqhcX6?yArPE{8Soko&=@B2aVgY}VH{wCV%6#6>=*4z+(v`Fd60iPWr* z!xNJ}FE(6dKK{<0R{iI>nq+^!@8M|PTZY9m@;R|@pD_B)ijWw#L|AWP##yk4(1q>D z9R2z{_BfGbLB$f{%j@gizNFh4DgJj>if30?=E>=ClFxUEg0~NmR5D-TV-IxI3=PP9 z8Uoaxblf2b<3;D;Bqq2^kKD5jiImB?reo*Q!wOxQ80<@{p<_@EGt&V=@@x+fL~eE0 zYnWfm(Ul)&;nmkEiSIfd*fPu>o*K71yf&B9`iORZG|*(Imfj_Xg;SuRC*4yW%3$^w zDjWC4MXZC>QD34h#tywEsx46~_JszGsG5U!V(Bq;~ zEYJ!VMULPA!oA*ETT+VmN?*){Gn}*ZT;y3<#f{xoz3sfum(o<;)D9NQ$p6BKPgo-bO1Tubk*E&s42qy?}Q zHpH8J1xG?SJ$wezV#lL_zf0m+u@@z{S|4#`niQJOFZ@=vHh3w_ z7jefedPQf?B@|mH|66VO1V9#Y_}AicS52L(CUGxIL#!gC-*Y!gW{I^yC51hq z!k!}Nx@3u6H@qrbRO6cV^BC<~)A_COwL&31cgFVS9Y1MWoX#@p!svLs4EYlRt9=?Upj7AaNk3W2|X+-ch?mB)25 z0Lq>5rvpg;la#>!%4?s(j?%xDqe7(VcbBu+P(faDpI;csMhaXD#ihpyT%gx}DgT6M zRnBqVUh_fJN{gmi=)gWu>NzwLPWmm_VIVYSUxgswPbb8)Q0UIl#y`OVbVk`Uf{=C6 zh6wBOnkJ1i@WhJLza;90ZqvHP(+7}fMa5p6An#(>=fOsv10bqJB#zUl%6txe`==q! zPA-r}CrLZe(ly8OI{V_c1G|=@WrgQ=oONE4Qj=NGcOp|GX-tcp7szZNlbI_!JaUhm zJ-sL196d5?PE5^%&Mn5Nux!_P)2yt!ZUME1S7dOIGqANzHAl;o2VfYsYCn*DdwL_v zVpJ#D>{Xdy>er`dN$*y%cf`)H|%Y)Kl)ZC1<#K@RwS8JO#fltss@94P>)?Az>t4s2N2B-c1`J{S0(K;O<>P6ugjFA@mu2BR zznn8o)qvSoF}pCJW%=XSjTu@mEfecJYdd>qj}oEri0sO%Mn+4?Z*0xFJL}j#?2Ce0 z7RIF+5p6~nagNIvtPb|5rHa}1>#YsrR19}}nfwEtyA|6fA8i|3y)W%H&n5&T_@{&L z)Y;v<)%BfrZ0A#Dj76@BZmdMwoK;>9RHL#MTF$$F(8UWg7kN#^n=LzLwpRk0`drS&+|2gHd$~E2a3VRaRU(iF>FK38qRqPn zR+V|~duHSf4|~@cPO>}n#W{q&CMavIFpGj>AdX!kl+zW0`|>O6-xhn#k34+5p76uB9z6|*`>M# zq}6Hvr+P-{bVGKZ+Xa%C<*hMu4Za-1_WXn^)==xoF??nvPyCqsfkqtk)GtEtNZ1if zFGUO3()1g8NUV2Jh8tERnQr;1O#(Z|i?M6FVb@-Y$O{hXwBUl9)#hvcD19n1wic3x zc?w!)MW^j5dNyF?Ch_hLM?Eogsi$&I4KLHV5F!`9js6BPfd@kucKk$K5(M(Cn89r* z0W(>5D~Q8{!d;hF4v)L#&)-@v+6DarUByFpV>XeyU!7()0f!Gr7X$Ka6hq*Gucv0# z$BM%@MwzTfAhT)Jy**$`=Q?Swhvv>%$-yS$(v~do!m=r*FIDa|h)GthKE&tEU4<|- z)fy^a0FW+;yulRK6w{)rPm^Y^%b`=}@mro7XXqd?0};;;Flrh5=yfKTgr684qs|&II(+H48IG|P;_HyJe6r9}zw?@K zFz?0Gwr{-mLfK0V$j@&u#4tM_gEbI9?%BQ#0dQX`QRm!AbrQ7@?8W+|o!P8KR}E!u zeX9dG(wcaRyCJmPpU(TMroKuRKkD`LM>NhYvl-(|c$2itZv*_Q4aOO`@s*WYL%AQ{ z_loTZMo_?2QNauWcEcIV3ZPB?n!)fF1L6WHsZ2nsoNNxwCDN#o=*&!>wDKDXFX`pP zaz}yxLWkur(<4mO~nj*|yE*PQWWRa)efmqc4k z60^u2+_`hJtC`6SU6P9qS*R!d+@9zqfS1$NP)ka(ht}kFiHv4t_GQl5vgJ4ZtTJBH z(6HqbaUCl=^#G^N>}KMDb13s$Gm2dufHIH5aK|6=3dP`-@uadowJeo$l5ft&`+nYA zzlx(56PD8N7hOSna4R)Ip6_K;7V7W8-e&tl zC-4T;E-Z9(D6yWyTDy3hV=7en{(kp|DI?8`QcQu{_3{ecjp&EC9Y8~qoTp)4BExlz zF6~wN;7#(~0-Jy6mnC3Ny;Bz*67NCdn){prkv*9oLj68`qdv|v7@ZBo)3QY3xI}^a z$Bj{yU!Vax9xVBd+Iw?e7^Y2@_v*)He3qUA)%N8nXlwXp*6|c7-py}b8hO9|MO=_& z4z*e4n?nyC_=@B920--LJZqMDPZCDrr5+-@nwPbyvP+5zxR>zzJN?jL>q=|V-&z`P5=YqJqjyoi8|F_ zLf~=y_uo5(4))+_r0*;rMAv4Zk6~+kKy^n4Y|8g<$MQesI`w-G~_{WpVj zcOes6C7t0sN1mD*X=}Sh>Rt3!)GrXn-Fe8F3$O?;thc}5bg&)2KrtWoz~?o{wfn-m zNJZ2M`!3M_lK?rj%{!953FJ-Aqt87O2nm;_kh3WGzPvN!>^hKRpMD4Jj0OSNorkr^ zAYO3f5SQ7zRFs1SGy&qpVled$d%qYmKhUDhN^T!!bPA`VE-VJ)d+TZtKUi%eyl>Ul zbAk{L`;0OkimTh{Wv@R3Tm^)W*T>)2C~8W??OS>{xAr!eFxA@dTv?a#ReglZ%a?2g zBr(@I>S&Fh;A6$z*>o#x;K^6v=iuY99;W^sb_?lX|PCG@*~hz z%RGW^T%MTD=Y_x6<#%bBI*WaH2FqU{wuN~JV0r;7@D82>ub{;z>9mrcQI*5bS>l1F zNN*}4bRw}-luIQZ8&#({M6crx3T~l;t&E=3?s))22`0@t9p3Fy6PG{}*V~kmtm3bX^N! z*)rzMh<=zC@Od|j0dBYe6p_&&@|)aKy%z50gtU}@pNtQgfM((D05ZH$`oq74od36e zy#NvnzjX-qtG_fX1>2LqPb-6?*8_t;&c0ncw zDi_e9qod#w0|1+T9tSewaEGnI{HPDB+Vn3V#2zUnySTYKp`@cXaXX#bou$Tkl zTaRa#hyb=cQw2y=|MfD+92Ql!^S?f~5Hs`)dC_=lQjF^Ch@7;wTGbu);}G^`9CuX?U{9U$jRJs_b*TsIWAb&9DkN~ ztN_Rd2r{<-8JJXXy5l1xIi(E~SwDbPrkRh^dX`!BynS#LDs_X$xL%}CRLQ|_O47T1 zM(s=fV7TJu0MboO&I?$GD=ff(qi}ZEtGKsf6(e4r1x@X8{mOe&h(X7)r90~68Hx>G z7RWj^IW=1~!@1RJ(}MXoc0Aegf27FYi@z0{uN@tBuV`ZJ93jcF=E1Xj!w$!b*%D_h z=-lHnA%l|EGYx#LZx+cUm<}TUaciQ{z!tRs1?nTOhvS2HBU@66B|RnnWo<(6z{&)U z1I%-W8j$^|0p!W8NSt@h79jED1b=~er#OI_;Q9s1{rC&C5;((8cnbgD+1=oOds@*A z;6HpJ=O6Crf9~n{r&a)%5VeQ@vV=j%7Nu^;$`q^`f=@gYj;^wU&oPsA91T#!Sk0kF|C$ZS>ha^T)goGg}l z#HOIIOnl54a?3mVS|gIP>!VGylheNDioh1z&+G3wr#QMXzmpO=!!N% zkd@&&I?OWMiR-(PIOMSfO7)>)|VYFQ_uO<_feOS+R207r=+;o4JDP2Z?F}G{n^^0f5LF=3T=^;h##0@a$hO=sTBM&|V}eozaC`qKFEj#xDB z4G~5I1AdtnhbVB}aq?1wZ5SU-kw}3>fA}I6KB~I8+-AZ#%h^p4lyILV-|lp#WU7t= ztMbn%S{)#g^{;y#XZJf@``=P2OyLXxJ)V4?C8e+@+FXq&-DCM)iW%lDT!|!5ob$T2 z?SJ{&QGVIwlI&3x#@C1=jL1^3mh;jyW#DXgK39x8lz1M`&w z4@lf5*cn4v-#J%AZNyWH)cEIGv9rDpo*Wn2f!ubqaPWn2mX@?D)n_-#KO43pP3)Ig z>!0?2b@GlNt)ROUaG7~_bW6~n-_jY9cJG8Wc@?R)PAFT)19C3#Ct(ovzYK|3WaArv z5u|@`{%>Zl&f?zevsOFx->gJRY~TORqND#GG*th8S^o!1j9Z@{$2+qnZ0Si>ikA;r zVsUl}fRkIkVgoJaQ&n8|`>_S|8)UNhs|r5Y{}c0nIpk!4Kr#oM9P(Yv)(-Hfei*2< zhlsmfRGBW6==oa8{l`keyGZ%94OxE?@p3dD- zMjz6Twf+Ktn^h42WAh(*L()TV`2^A`;HwM=F~&PLdgsXBQ|ehEshq_GzC4ZxlRsSu zUeWk4@WGLhV{Fo9yu|6%gE;%)44E)Hj&nn-a=v)+$1X8affi%7ZsKjA!|cRr<(o6@ zdav^=q^SygezYZ{$c@?i_JCyai-C9 z4b*05PG@{i15D@F4!C%fpxNlxD83|-I;P09PN~<;ZwVY5FT^x8O7+@!bx#~mSg?** z)pL|H`W<8fKr#~y))FA&81w!5cGk$s?gLlb>Cnid&@l7QXyboMY*xE@Hd)m^Oc>~X zAQ+N$ZG1llahQ@;TQ~8rx7Mg}#jR?=UTCF{$EZIT>bHx2^YK-<+!EuO>|=5?)voHNeRUC;6Jc-2P|Kh;^ zH=F~2-A4v6e-81_cXe7fWY6?Az`KcV0%tc3gMgkO`h}t#1nIkHQfen3x5HDy7OrAU zYe^RB!1JztWivOuq=yQHy=isc*}mD~MR{qoO%X#r6Mc9>bXmtQ4q)gHShR+m(x zv-L#}UnhZ*i`wJ67DN@#RL;3WuPf==UsII4HZrO+0vw?h>Hl{106gULIxTfXvpYUV zFiO1YFt3v#k(|$@k&%JOEh0Sje8MyBGUv@BgU7RqLO-p3 z{jD2PK$DDL#AN9$nJ>Bbdt`i|7cs=h*=#_UXn1>K_x4tP3W3WlW^M>sc3bLch%j%p zDU5G7h*>IfSS=P&bozMqWfl2D3~LiyPWh9i0FV^Yyh{L&50Cs`?R^P6 z)a(BLXwxo5Buu43M2fUvS`MOv5RPvdp(qWJ?rEn6hS_%D#?WWQj1@ciCs` zvwr=b>733v#kse8@9*Aw{^y?8t1>f<@AG`0@AKI|@Asq8<*SFPmB7cutKi7m5JfaE zm+nBB9O2_=PO)_7lkQ7N6n(~Vqep1e{v5w_&7o<(+5St=kUUZK=le5N?KvC(K zLyQL#hch556`9NHavYohs1OBG{e%g*IL~Pe86bHJ5p*ch;aBtkWDD6%2zZ|| zV-F5g!D&JeH$?U)%prQDU2HkP0t(GOr@4v2E++voE!2BchkJ-XE`&A%QjVX7QvGSD zImBw=aRPk51Jq%*<3ay$71FSXrBj##3NJiCwpt6+$1zkv9Ysg{{Gp%)CPdsbhnSo| zIdZ(bK=@QoAoIY=0h(vQdd(YXq4`W#{wHr;^!sleUkVN_!huD8^m&n?qLt3SJ_?Z2 z1doDfHSf-+Ovuk*6^t7+!fg&7oL1J#B8su;ZucTfj-wFQ!F(f+w8B9K1PWB!8_* z(GiQIb26?pBy7p5%j0y`$V|=jx#V^wN6dM=!h3H*!nUGC%eOpOE+ac{Kgx{B87&zZ zlFUU)zdG<#EKVyu;%Bi#0k30931-RNjxTbzfzmOD_`$Lp53dK}#JC*vW#=O6r7)92 z7|JTZH(v1)4+PWg+wb~P&d<9ftsej=WCX{p@q|tk^Q^->#P)#h5BYK_L$n%8pj z9y5=pz3McbWN0$AQ&U+`eH8aNiLw42yz*HjwpD>sgTwi_lbUe%jYrFdvlR;!6Ozu1lD7-kBfkEC%3~UToIaK9RWq%g)p&oX9w@*&N(*1$(8nF`Q;I!Ix zb9iNz>Fopj#9oN8kpo@@k~}L48Wc9b<+K=vih?<29mTCtgSz5|lK~nzQS|brGObz% zvVR0hD@%b7eh1{O3$5%l?#?0JU}NVH>otOj#R^+fAHm0r!WtXO7_Wu|HR0p{Ok3|e zsxQW(kRe&~&Zo3Yt%rw7sC90l&Tg^}YM{R2UExGmJsC4={HblK$^5nAb<|pg^$z$R z!Q}m+WUnH@t>d~cELUU%5|uwh=LH`qSZUq@D{ z<#WD0(4S}EEyWerSG_0r+VExHluq&8jt5gNX+1j`Xb|m-nnSqWhk8q{(J^yDPwy)1*%9oEE?iEt*sxDC3LSd<|r#4ytnDB6$J8f&3mV2_7e z2~#hTnjmvdT|s~KYoCoXFgHsL!CfYqcD#n%FO#9d6^V8z&qE=bbhe!2a1XaGp2RCX zc74*VP~CH_;ZekjyYUa=`M88`qJ~IsjP$C!OEvQ~1BEyx&M4b6uXQM91HdtXWwlwYEqhYuRk%~Ok1X;t*^C8dUm~3xyg;~uN9)7bweA56lBH)X7Jtg;Y1Cc z`cF>Jva_H5m~2?ln|wIirc!15qtoUySG~4H`PaQ^jG#o(B+8)zlTLoM;)V>j^od&E zB4op?<=9EZhBuxASbpd2h$b(Vw1Dx>lDvz2%~2Yz3 z4v&tc_&zIoC*SvmgKFA-BA-gNk{#oiHIyt7Ti$cK^kgf2FJw|ahp-{ZTQjx>5UH?x5-v-62Ag>2 ztu?bT4Q_lwWhW`xZYrf*R`*xRKNmYAv}6&=1@uap-mJgieDdKvB*rUt7Iw5|{j#4eQ z3cXbKw3t$z0e+b5qRC`)E!C_PDdThxlHQRFXU!UcuGu8pmtS{bvAW3aBtZpeY0^hR zQL+Q7Ye{1ku0%k6b=<=p?nSi+x^EIARF76#BTcN+&B|-<7V4Dg_^jm2!A;vYNhJKX zh?WhfTZXs2`iHJZvHKn|SN33Nyrd9|?VAZ+q@+}Nw07v&wXv~Fd#h~4{42*s-$?-u zUEQCL3siSA@bYp$)9ooOF>#n5U>*Hc5NwTv%WvQ8(%UVz+YIH22_Fe!Y{aq^x2hT3 z7lU^5GNhQ88*p!mk%8yu5c+%E;atoYf7k)@hjxhnL)P_=7YY1lZkz==ZqcAhA~^w- z5gR-FcqQ`?^A(}DE>8sd7!?PMGk+>|7~m}C5ZIctC_pL_Q#Om?m0Q!pu~NVN%EJ=N z{mBuAAIlDV6zuZZD{_JT0Dm{(7~!fURT}S!oDnr6(3E#$QKfcU(bD~{YBpuYvyvR5 z>JJQf>%BBW#}YrnSFVJ6G9wwwMUsNL(hKms&X)<9sT5R~#(GN9P_@+V;zJ5g<)&pBIp(E_%=w-KOXI}*A5EHW{!rMMkQrN$+WYMHQ%c;FHW1p{vO-&X zR%jdXb4H49f#`C9jMTTk?wi#0JHh2QnJ?4=>9L=O{%w5svxh%#n0fbog*Ns(G4*$( z;S{oeKcxO|+;sdffwH59!M*tm(~WT;J`)#i`n z^HGbsG5Ht)XCBWgSc)Q961G9@NLF)Na-`vzRsK&}h4yVDwPf0Q_gK~F_YRrAFY3R( zLp5CME+LVQF9>kQdsMvUOjOG_{UAPqj#F;9TFX>+*ihz(Dkia;=53+4#Oj;TB=Cz` zo7xQ&C^0@+>!4#CTkfn~o@mAZ-NSc)`&pow>if~MRP2Sr;1~#!9dqdTtB*-Rs(u-= z`}1dEh714NN;Z5OYLNp;nG)JLDqD=?NEK7VC^mE&JUjf(q=Rcb{TRo<(cX1O3%Dt` zbyiHn2Y5$~uAl+N973LWn&8|Owlkub3kpoYioo&j#x;~{bInA;^Biba3m0G)D$4%b zKU>_7p@`cvBFww(nU2lSpj|Gckjh*N6&MrxHe)srh8sTszSYEG>NLKA0kVL$co@Y@ z6Yhb9p`7!CBFUBki!(J$V{pnt^d1t6PU z^QsEQtNFM(KRme>EqI719VEooNdyNg@Pnk6uw`croht$aF7JYwPql*?mq7&N3!KwY z#-=&Mg~t$0&laOoxR#oTq&v+aR7~Dkr!(%4pM41EkRGDsUIIOq06U;!@SxxZpFrJ# zM%aw!5Rv0VBSYU2R$6O2vH&%Mcnt<{7*4`}8iD9lnDLX7Cewm?@F!)46yqU?cHZ6q zki8>cw+eyx`z@!{rlg0ib__imFhA6>B$v1G)5M4F8Y*8ZibfBbLmZdE=?j9&fM2{e zsB~sRQ~UI$nVXa;)>Fz8?U&$L1BMo(X~vha?l7#i74$uSE@xgqJ}%*wtX`ks|meEXgH6++{XTQho>bX zyUJ>jzWo@I9ic?B3ilioKCa0!!i2q`5zS|y{yAzs<3`{ibhcVj1@~E0hqHFbdaq%7*CQwKGE0$2R~{< zcPnCBz*Z|pOVstmIpn&_HV(ERF$#3+ zSt38B0{?l81F^ZvzV5JjL}{D-QqUf%m)k@ss}51Ycngy9>s-7Osd!1ZwMXNAWV zB6={K9cHj(URM3Ik-B|mc6R3DcQTbpC--Hiix>N=NIT$)X}&(r$}aJgJle2!gWbXQ z{;{!sZE@n>O!4Y5P|s4?-oVFvw(V`~#gv59m#`?->kVk&qp)wVmbtbyk&$9dE}M2~ zq5Id`UB&;v41z9o@W8|lo~v@3$=CARhIpWyjEqNO7BNSbb68t-3iu@{$8NuOO7E8N zNH!hNHqPWxvd?#RxZYSsI+e}rD=B_zU~vd>#L3GYG2&w*ud;pDok}()lj%-4Pg;&G zBrFdncN96?Ymc%|voO&M-jjYPrPHN<_>IiFCHL2_+faV_vT4mEO>eivGzLV~RP647 ztdtBBH*@PXmVFUOPYnI4Q#rHTjJ6Y7)bVQeF6unyuBL-!SI#k#R(8w&v@tXRIHH+i3IG9YM@@hEO6)d?`G%LyaV7#qv4_ztWVT>Ai{bii5e=6Qkj zn8w7hmWJFjK(KGu{gz%ph}nEe9v?U?tipEa<01~G2seEx#uqdsxPyiS59F7AgaUN09N6tbl5{(4 z%or>Py^Vp}g9cbq=P|qWw^E0>4W!x7m#V^wH-=l(z}Dv#n=K?P1}AcgOCzQM zDUaNPZnV;1_&}rOSZf7ImPXY;jVn(oC@0}MV^Eu%cg!Kq$D^2n0hMSCvIcxo_AWFc zV={+`s3TnPE%wbfa~6kjx3@i38CHFl^+f3qaq?t;G^DqFaoFS2*I&FH){RSguHbp3 zv#!x2<`r1YA3>-%k5qdZsvUpeTi14gG#g)y`r2Eybs;>x4$BNd($e7gbX8or`=IbV zDAX%%3QMCeYlg}lDBzCHEk{i~M~^JUN|m7|q@gYkI>8+`Aqs%EQ8X|k48P0U--+$n zs5j>j?7KlRBpHAL^8m1FQ6o5wYuX)NYXG#0Nf!sx1$fa*6R{fTQNuL=!Qu&ZhK%4%5n+Db)G8W5NgT(J?)23e;)iGy_n= zaD$ZHgE_=w3ujOf#v5obCDQSq8VP7JiCIkmPiE#L&X*DjkGp}YDnXi?HVGcpN2r{S z0=DD%cn#*d2I?9N9>v=Hes~YU3?FFRsRhSP@j`ELoS^px#68vuCU-zx7+fJ8BtKhg z>asu!&MMT4J79GX{Cp4jJs*Vlxaw;Mgx<0PB)^E?3is#NM8gTu`qLG1ZKruG^09VeVyMxXJJe#^~edY^%35g=R8P?cPLYNV$0;_)`m3|!ac)X0^Ed(#6 zCeLU9vFb&mXkh0|J-G+OmCuFpJL39<&ykI6&)`1}P-bFL(8m^_tRiQ1D1MkUpcCi& z9k26oV;klWo56f8U`4x47~U^|ZcGI;sT7;ynArnDiXqUvuobSt@=<^~SeXv5M^cpi z?O-6PcY{W7S0;u1EvGTYE*X>inLQyU-rWS02>xZ+Ig!h!b06%KtIfKRtskG{qmV!e z$=+AoCzJR3E>3);VI=H9)xZZMP8tw$PHCtCOyV8zXfjx>QNgGo4{#zYh+JE_sc6NU z$e}=O(2W62m_ifE*>H{tGzCP#AAH>prGx6;=}QZTs1DHVQsnpsWu_*Ik%OY&A?zB6 zU}{xBpK4$=N#OJny-DyL)C2&m3J(&B36z7Vv*SFsrcWZ>P^(6rW$;G%7m^z!0* z2TC|hDpXqtt>3e$c~jb0!PZ;5AekJEZWra!JfXY>2FArMb|@=^WNUA4D;jA2)G*M* z{+gc0c7HP;?t#v|&VstV(^%~gaq-W=Ck`R9dBCSch~yj zy(PfBXFCBPsxA3Ae9saTp~YW;9Wunw4bQXWF3jM+q=!zd?Ar~J2zdXd-JH`O6=W)H zXi`O8RO96%nNZ@{(P@)Wjd{P1>K4M?K~9OWY8VBVfQKlI?oQ>dfus6zqh>=GB795Hk@{=G57piMRtVLESL& zdplJ7x^wUAg3iuP^rWAWNdfYKf+#Zt-eKAs_!8mH9Oz#tmOuy;!stEOD?P(3x|pPc@PG#sNQi;Afb>+hflb1=A^ ziZJOlMcUhuCbbN^2ylL+*ZC~b7#e)>U5}X``8WG^iPYG)?pfm z*J=A^8jfm<*6tYlAht657I3c~0yFzn*t8Zs9dr76 zAEe*ReQ1vWAd}4J{)9{uW&~@LAb$oh=L+WZ%oKximI?(Zt_y+jK#XlZD1M+2V;MEz zn#_&RPGn@qd>^TY^f@{O;ArJ9;OJL6R1Q$*dixG;!^D1L{kMH>{n4Ee;240lK)sYU z^8uQ2TgfEj^I6=4%8W=>x6<@zG|=m-xu-*bzzSjLHkc3%ubw#sbwdq9lTqV0977QX z(ClZ0s2RRwV7UejOXk&4W<;D%FqXv>)(kE43$okI&FE~HMJ3#?S>~x-Uxx;~?LARk zrxv@?>Zq>mPZjlJw=ZTCkd&Xu4?fnO@n?h2%|=uTPF09}-)n^M&YVv(g=V>Ync~QWa6{O*SuPOv$!k?T~HjKA*4Z{aX zxqu~gePVb*ql4Jnz_l$y+RQV(KXYPZ&q4+7pb2rtRv@Q|=sCoZR6IBs=R64O1A#c} zQI5|cFrYK{wmIfe10B5>dK-?T!~m_FOvZ7ddVv>EfUFmzZ3F80G*ls{zy*Eef(Dwz zsBH`RTS;fZto^Q|hP#29ziiQ!&hRmVh?&sfDTO>dVf=Ga{w21di^(OH?0mMUyM> z(R3pjNX<@8VrPD1^$mebSR^i;sb>(_K{FIi{igB{!?8YPk)4ausnj}l%HFB740oxYFe$i#u&=1LKU z^EaYs8Ul6=47vjZui8ylaWFXg=7Y54eOU29sHH$+4uEHxrGVaYe#t-kTrbO!k4h4B@HOq4oo-71$TL8dW_A8`E zSots9z}M*Be+@nwT?NPHxe-8+_%$XR#zcf;#qKZDkY z9(os`qMxwu3YW%B7(V(qrpr5UgmQ3Ati)JkUUr~UbccAgQ48YYBY!LT!K(?;^`!G@ zAI=AMcXu_4gnO*YNh)L0QQq!<1Yw777%VB>CC)JB(U%#nj~9q+p|{l}qdISQO!%z< zJ2J~1_T;9=e|GyR8{p-Gz=4m+UN5w0fM2p|u>0!BrdN-e6Zba&# z%{uGZH_9E6*Omyn5d?dFwBRq>WVSxb`=-I5Q~6+!3R(8w^l<$jxT6(9dC4gp7E$~T-37|CMieV)})s=1pKB~GVaqvw^+D#YK-Ah18| zjB+nNc1>N49xHR%e|X2o@03I`^ed9sQ|}59<*rNYBukC<)pJWObRJE(XPcTym6pU z2j;XdTDV7UqrJx&{dK_s&lpPe6z3rd zaRuK&qS#68y`hrxYK z90lil62Z)Hk6}O+!``{(dseU(&h13`Lx2rt3kdnm!AjRuKmY=~n7quB3C~}xg;w+| z8Ci}3j291~Vwp-%=k7xRZ1ru*Vnys60xI&KLtK`m?S!b-W*v+mOdyV|2Q}dekZ*iP zO)qufZs`lptaX=lx3Fw)84cpmd)l9&>oWP4^wwwDz6XbF%(HiQC+sx6|3L8Z#Yj?? zZY?JAX@5V1`$74uHzmXnew{R%yri}ZgngehTrrz+PSR1%s&zR=WM7?(+UZe2%>MRn zAvluSPVRaKOLR%at_H^ zePUYVgziuRSPeBto{@-gAtQ9`r5}Gf{MyY>~2cBmoA-S z9l@EDGbt1mTH6~Kz*8jAu2BwePD-DoINcv6k3yzvObMRy45F!oZ&FPJ=EMH3VWqBN zO=7P-gBuO6ya^7Hw_v8rX?JnlX_tjd$YY^PAGI_?#-_6hdj5@~{M1JV*&)eNeGm4P zaZ?(#ZzYCwDR^k?F{0 z+pj5(UNd7*zfv5z@Mo|~JtCxh79KwZHfWI04=@{w$>5d$<9zH zn+f@fbBcqqJ81O4A!nf1ekRYV<7ayfBQbo?+K*zGibl8joXiVw`$lV_;zIrk$Aq)!dvcPqYM^gcZbk%`ZVA zfj_*1&>6gVW&_2=KGFGjCLL3ZTK>!!Og$IzZCmzC^7zD-XK7ZE(u zt*Dv8&iiPHw^ka+Iz14>J=fw(&wQc2yKDJu$R=#H`dKp_O8*pHeSvD);Q& zin{I<``wzh?YVSdi|ZY>f+grtzZ<5w>SV8EuU0tb`vBqukS_X##mIvWeI4qV#72IG zvVt83fx_C)v}k~4>?^GT^5eIku#j(mp=tl5Yp_rXub4po0cvUKt{J^KZeH4KGRoOa zpq#jR-DjIv{0Ty`_^4bg-?^xCgTi|mJxlyBFWRNZfw&&h@}XcJuiY>)%3Rm~WBbwE zh{+cLD-Vqw+hmiVbaaWy9kG7fAKxUY-KXVV4oL_LO3RBy8u@*a-WRuu`>4$376Xii z(i~zR)gn9(^dQ~=Rl9<{GNg}(%W#(Ou?>mN2O49PN}JVyEu!__c;)X%4e>?wv`i=N}BfrH*eUi;!<_pbXo&moUDe7#La`_ z=c6Z8==-rm2sqIf<(;%ZCfhOiHYU zS6-O(HZInDeEMo&Vu!|wW?hQ*vKSd$m;QUi0$dDT&SRcJ)%0cbQA~h5*rcV;N^eM+T4U zS}V(yxtQHClZ$m~)L_X6=uldq><qia|vB}L_u>BZ)QJ3=$J`>TvT z%3r)F&)9c-N?`f9dc5{im86O)^L|qJDBhfjc5Aw}^@5pkh=oQ#iNa`wr23!%x#NECW&TBDCl`e!_kZ^F*S+MaROPVX2~#$6Y9iua9+1` zZ&r#)8s=%7iMKG0(a&wDD&To2wSwzh@_8MPe7QwV=q>_d&)0Bg8CsA##q+dPK!ZvY zY1-7~3C^GL$UXjSp1=oJ>~NE8{X`t|CfFWVQN2FSYT(4~h8Xz18KVVYUA}GdC1^pN z9r!z-t92h4*S#aX6uE*DR2amQm6{SbZhpvchQsR4#-)Cj6~);){mv3McE7BzFfZMp zM$eXEj4N2W?y(y*Ib*6G;&0fT0-{#s0U6-@c;E=Xl*bMlypFC_-`i)uNL7@pSBt1+ zj|PPaQBXts3Uk6XuEF58hgT@aV4GlpXJA(?KcD(RLB#ao`7?f83R^wc-z^!fy@a@M z(D%*3medHw;c=9k405T%1Nw<^7oWzbaf6R?Q`@2{l|&Pxj_LT-X6i0kcN9N##hX}y zB!KT;*XRU(MRuH_~8`AZ97tYL%QN)w)=hZ2b2%rNP&!sAayeYP;dKvec{~f_(FRG6jBvBr3fT%iSRT!@c*zmuc;O~JWap=n zy{7_pIEy+L`5xO=K=ZVm4~VA4m3Fl^2+c-}+Xr=rj|_m^!avCd@E45YzYD0Qi;1Ql z{%Dc%@lr-chMlKK_%V{$>wF!}^6d{10_HQe@QMvA1eg1a-J92x0J`d#66>DfyXyxX z4WA!!u;aBFJG9xKgPA;_aP=Wnunp?;+XGjp>7v#XOi%JA$+=kFF_Jv_m|bIS#9D!} z<2E1}ljg!+<&Qc#+<73v^C0z=!$H%O$SkU)1S#nqknmHT?Q44bD)%iP&e=x0@i++k zE}CQnW<@xR(h)O1B!jy@<&}sk-k(Nn(v|Mvy^fv$9-qg(POCC-{-n9))6dvuAgv|;1%iNWw05qvJ_=8Hy>l~CM_m6ZE-q^83~&9mNe+)34? z^0Ggs1k&io9`|{x@t+Uv9DZZSZ)`r98cDUEk4p_&8ggE0%2+@2herd=Y6?}*$+#>O zqVmn^8+!Kx1*7(lCaa_nm2kNBG{>;-=F%|2eBt8g{EN%upH*Cc7#b5DI@oMWHi7Q3 z>!3VZnB@yUmcbrgfIKE!n|If1KP}j<1$wax?6DgiaG;Vu zaw&QcT9I=Q?+~HwTRfA##x&A&qP_a{eTk#C{Hm2)8&9cA`OBM+gTBgd@$a8Hy%l$z z*%VtPTs;e1g3vIbu6T5%sVV^RS6zKZx0Q<{Jh!9>aWEna(FiK`9ub$@7b6e8sw~nco%RL&>tmzFr+^VYoPk zFvX<@;xaYuayoeRL=rIYWYbP5L;S4PXSyC_tsQm+ zSW9bAdDxz}AGC)U$$n-7ZLeUQ0s&_kwgShc0FovJN1V?mFk~o}J*0I5v-@d|MJ~|} z*s-x@tu(9nrGW=3(sJ%H#hVOr3 zu@jMgrVC-^XclBA`r2oe1{6K>CiG?W`Q$|3jzcb-H=Ez5WtuVh;+n3$rrK z8|eSr+t$*<7NGd;R((9@+DR%|db7<~TlgO7!V#7HR?<2h{s|R?-Oh!;g!{=c_he84 z4#7oAWQ%+4vGHb)qEe%xQj}DZF4^Auf#u{#VKPx)0Bv~ zqaNFroLqZ*!vl-amKP6QJ$4A(R6J~R(VoS`vf^YUw%C!+A*S0Zr7>^k&?yW5uG4aS zTvrLw8_Tjo9!^@FprTi0Lrs}gSnJ}})32RB%5mjZMeUf$37T~xq^1{*WAOz!o2JqH z*z-up;V3h#E@(DeGC6xTu>v&Ekxd2*zH}J><|q3n9tJlZUa<(bsk=T;6n>{~7UR86 zraJRQ%g$w4?7OY6MMM#A9EG!AGC$Ee%1pzZZt5Rw@OvL{>HM8YyCpYdnA&L8C1$}T z2z)}|64-Z#z$H$}lzzE{e3mcOaqKOMXomy8}_pz_b4d{h?& z0>_N6)$)Zt+A|;B2QfC&>GBT1hps8bA={kKkx|}qqUJZlk8|fVm-^Tm^N$HHiIaZ&;y!<; zttayozAU3La#z3lDbpmKw3tx_o9cC&Bz-^A$&+^1N|D8Qw=9lQNw%RdUP@?G^RWKf zRJQb-L*CUTxX?=3zzto-Mb`0gz;+CGe%*5((!>f-Z!kblp^PkA6r}K-` zosh&7`XlS)IR!t%6qo&v^-DTw`>2{DditTtA5As%tYaXl#}%yW)wz-@AZjKHabbdX&;G&`1ICXR6u9a(rLBiX- z>xu70-U=8XZ0S$QJe`vF2$ym9vP@j_I86Oa9VG&v{J>v+7?&%KZ|F5HH>M#F)CWS{ zae7856F2_=J&OkAD(VbI7Bpb%6*76^S0XW(0i!+aw6(xHaXn^^$@z0(%csOu3*VS*v1 zTArG!<-7esb8E5;_~)zR0{@8b5mshLY&Aawye=_S*S*08f`;o~5HP>|)H?7sDm+@V zUw6?S*0+DWRPaL0n7v{o--XLAJPYyf%m6FXQ!1PQlJY#-irZir4oq~s(bG1~n_W{S z!?CW{v5|O$OEbf<7Uw8&OC-W-Izp(?>-kENR{1M6xjaYqtJgQ%hD=2hukQ<{vrJbA;`vbpOXXm00E}|Z36X|^(m^a(+Z3B&*TLu*c0Ej zH2*KWXOA~Y+^w{BjU2ycdk86l?Mk?3{kQjx@b>if=JWRp+Lj!R(~|jraI}`Fhdc0g?QYHU9=&^0z);nds?oa8R=r&G%r7ilQ_s@c{o+V*TT^ z`D_Hj0#_oTY?W-WTSesnBJl9rMfsn3A7H>@ZfI&U+M`h_2)#(4rP4VSg|BlV^W#g>n7*^#402?ty&F9;JK57MU zZ&AALl4tITo81$Y6$Kk^@BFEQtX4Vo97~%?RZK@~9*=8gA)J50i+@wG`9+`v6h{64e)J>P3OtR4I2#>-gptIc3j?)khC z{{h4C*Z03haDMlC$gfr0d&{~m?sHL_mAW3v&0omvRNmBU_d;E9w=^S0A(f-ep5@P; z`R-AE_euJ^|6q^Q))bw%MH7AI8yjIgEu+u?tkX}grJYVM9(-VPrPW1ESpR8#zb^0t z#8~jQKfLq*XPM(qp@CmEUt`E<1nZe!o2MdRAfc!(XYV79cred_blfZt_`Ip=);;>o_O5eg(qx&3q`t#kD-_6u;)*m$+`~xJL$zfm-g4nu94vO~_pM`OTeZlwd_dK;@E!!8|FseRSetPAs;9p9 zXHutun8Wd`z~K2j8DDZyQ*kH$`5Jc9f*7s6V=vY^$Z*bL#8}y^_X{R!f|)>n3~Oq4 zvB@I!WpVndVz{TXZ1%X%uSF*nIwdBD)pc7%A8UIzJKo;Gb^Fc1s8)IN7b+m{@i`-K zB?}SmQ`89l{G7A|pJuV)8=t9u+UcJeGtSD7L@qe=ABe&0Pd~q3b63$3CW(i#!l?nQ z;Dcr7fAKm#zxF=tsV}Nx3nEaF#ayRYyrlELLmXC*Cg^+OmahRp!jnJ!+Wi^!X(ao$ zh$wAW^a`6-ND%d(n5Q`Yh8Q;m_x~qfcKRyQ)NGHH=}^l(OLhC& zTPd1~X7kbszgMkN1F7hGtey$-R-qZ9 z2oYvbk=c}xEJK*dzKm^*G2ds-`Fwu={=Tp8-yg46c|4DKp69;q`@XLC^}gPhyG{;g zwr%-s3jhGy&f5Or3;>%F003gLc_Vn{XGh~H@S6~a$*Db*0t*)#AU4A2au^8p5(cbZMnlfHnVU_#WWE-=M z+Sb-aWw)Wk>yo6RT-*%`_^Q$`J%LeY#=nOE(?mw^-@_Ajn-gUIJ@l{v@$@C^Y_`S&PK9YX*22so++NX-R)U4{PdmjAOQ@V`X_{BQRF|8pAv@IQYE z{4Y=d|GyFP($H9usyDUepCJ4qBx(pIs||dCt_sR~D%FW5Yjyfti%F`uB8|;;bw%b@1QN&qLS@ zewaQ4{1-_im=@~)23+0wFC5V08*bVLyt(u5tMC$CZceyB<$*W1o>@sBmbq1P!p^lw z*AK|s`yWuD3_mFcR5<>NMP`3f(*yK+|8uEGQ5L*p_wOaT7Gwy}yZ@g{B_}t_0~M$L zbIEMSgdO0w<3G^mLW@nH6Zp`I{{S4cn%X}3-_>3P`vI9z|012ttqvRX<*7Gi{Jxux zO94+pP59FW!bPWT@<2!Mf2T<0Zu~AHKcMPSmlR0ZyEc%vOsZ_)ie>zWFa9-#A3!8* zObGfN*y#5@9tr{Pvm_9!0)haH#2to!4ER!xDpw^-@vy!%|3G!JfWvcpaXhA z&QS0e0B#>ro4pkME1+Tyn(*?EopkV=p7jp1TWA?T4=U@I@WkBR_s4xz@HBHbUl#D| z`;QGCJXTZzKAr{S>6yYWo4^n4B2Cz3LilokpBMNm_9s?lWC6gh&f#l@-eDU*KxGs7 z0NASw;QRuB;7`zi&oursj0Xhph0cMt(RiO>30}?gHPs#L1AqsK(t*T(EoJTe=QN7F z4WPd|s}BAO;HRSv-Z)hHNqgxhnHd(~+5P-Y`X>I{fLrt2U;ReX$L+c_ARuH^vE({t zZSXX^C4c~Wn`QLqt-{1@06;ek*Lv*!1p$!*#2zuLV`tQ1z=6tdyA~=8N&i|s^i|D1 zKUZPScA$Q?!Wy`c;`fWCR^;0Go+)0MLe9m!g7ITgy$?8ceUfZormf3LGdWv`~p!C6jWEdqk~ z%lZH>`43s?*qhdZaoFwPGp`tO8S%z)emGlo;Pl6gt>8@y@7@NoqS>8*o~ohDHa-0Y zr#JUCo*8`<4Mr~I4`nW6+i(y--Uik_Yu3^Qyvf+&2mHLdAC|dMs6CH?03ly(uuzk# zjd?8at%1u!?`5S!oAch5u=ERv!ymvMv|rSMkIB>lm+C)TY=N0^^7h5o2H!GNQn8!C z#Zv}N+>4U?cgyIdkW*#BVPCJ?MxRsy50L!?Z&=KDmz3<&6mJslQ8#b zt~xyi$n=rkG`eLn_kb&x&Sa=%)05LCVc@Hp2EC?0Z^`)8?a0BcelCdJlaL>gz~(*Qlb1GWdCTSyb7H-G0P zS6<=TO7H-E&HaV$&9U+|5osm@kaqwr(eB}}=T&G^Nb&@0(oiiyRN8|u9|8{b zMBIpZ&I|^qdm7R9gP8=OC+^MLX>|m&K-9=e z|1k@o2wYEeuepN9t+>bI;ozG5eo|4MJ^`)}z3BME2ftXqDQtAV&jHt2CY%+GYgqsU z_Mi>WjEs{?fcc&)N2%|@cK`yO7~lZC?lrB@M8{WMZv2GT+aE11M?S*(e;X3_QNHH6hCG$bMmoZzzyg zAJ3M1Bemm&JHhS3(Stzo=RVC7ai-I9nZCGLxppJzJ@wsKAe9^AT&O)zrmk^>g#cN>Mg#XxW!03oe6H%&aT zntb5u6xsG(XSF`uVQ& zNZqP{-+t8U&Dbk)1kf=o*s6j~^}wNgR_qk4Rg{Ny_}cvZTTx-MRTB_ElDxdU$m~8R zoM$b*S3EkcHF{wAD>zBKOjW-wX_z#k-)$16oeO8(tmp&RH(<|veX1aSZto(5{>pR{ zpmz<11-&EEN!~qjWhmbBr~H!4Z%Z41%wQz&awB2R z3^RYl6t@=BQxWLcQ-?vD#;=6eEPSzJ$SGJj={8PYmwp^vf^=Yhl(UmAOy9546et7O zS7P0|4(P?g(`LSWaW=j2(+RWG6Ug}S<5CX|kMF4p^^DwGaqc;ww>92F4mg$P_)SUu6@%{4u9&Z? z_tBxvbO>857dIE5eL>p7MYBzd#zn?AX9OH;-wjP=4XeSO@(ZQj@2DqCx-A7v10*t> zm=qSqD2bR<(0d4u?ZEIKTLdLS*`HWF)d&+ziMd(C5~XMGpdx*8mKHezp-+%111LpD zEplb~+0j1AvxQC39ffdzPL3sVC2n@GQELI-`Yv*!s~js@ium#VZg9-}C(YvUmDP!| zGTCA4^Ts`r~2}*Xp>r!&f<=gC*&DLgFQ$ z_gv&<47jyzUG9P%HNGRz0leE>5S72?RUTI()DK~Td#WLz6~u7gRhXZxSLetUyUT5bMfoK&QUbS(rrTv^qy+mOxp8Q)V-+h9QMt(opD zTenw^(D-T(`xGj84|7a4t{;OVRmknjA9nhdXeGj8i^i9S?x>a~4&2wNeX#}WB16{V z8n8{Cjc+wMMf~d#)2^bOsc)?%o4Et@dcbY^PHfjDE|8G#{>`~9%dHz`|E1>L&bA*s zXCJe2L{!)jf44xBsPfc1Io?`#lGC+RaEpH&?o}04mXAt=>ElM=90Xxj$BK&QjBH<# z@{H3OC~NRc(kpY00d~DGmy6gCXq$veceJL#k>O2)hGH^cf~1e>nkav$(GAFw&V3&A zYQmH2^sP+N`*aT3q}XcH za}FLuA=|?^BsuJaE2^KtmTke6)y=n2olO`T zD(q0$Mm}yIX5`5~&Y=3DtLf6FOhaYR*~;HNW!KeVskQ}>Z|t=i*mgf|+k-JiPWD#+ z-gIZ9==;2e z4H4>U?5~oy{H%{#c16;l39w&_#(o;A^=ntO_gu|t(H0EUd(=&g;ka+C zC?q>nO9sSvpoNX^Z5ixj+GfUxP-`bFjXa{`2bafRSB8H7D!%KX6*pXGu%qq%os71Q zO&M+Mm;H(+GGRKypJyT+pXF4<<&!7-c!NiR)@yoyy&*dYsq-{j#?0Ct?lrGI`!nw6 zPiIry(>RhrRoZ-gbAL-bub?2J6SJr=5}@7cxFMVx;~O)Sq*VgztchOn?1~WfDKl5Ln?jP+5%zeqr#xC`Y=HJ;z)yYCW63b|S zNLEkZ;W;1(PW4^DR8iLj_EtjG#8c{YQ+8IKM3QlMnJ#Zvj^!;^Y{^Wpb+txdFrr>7u zieeX1|6a-Gr?++TIj)<@)B!>zS3DhEd~R^7Zr!WkCgJQ*GctihH-?)U!_6f_s_PeP zR>zAOrhKx?q5`-dAq|vYaUG-q#6xa07duKVE~Tyl?5Pg>8>=!7$HR$_+4{_|$%BiN=I0dv`%TiY;3X6Ozyy+0IcR)w_=%hEQO}*` zT-MGVRHxUEJdaSESm&=Lws}=kDwvd@yg(n zg84o>JAVc*>_hnUOzY=@h>;iy^{XuF-wQr=l-9-7YEocAW!L%X%G>`74-i+L~s`GKX7 zuf-@^XdGgPK8oapj`f|n%^6te)SB$CJ3JJ=y5_B=ORGg}XiTnug+Q+4(Bks9+*63$ zescnOh@bn~DGH^F1o|+U<&gp#%=Bnqsd=`L)9?m?ga-EHz)gO2@U4eltH>^Am2c}< zLswm=#=C6Tla&#W*Fiv6VQrRdxed^Sjt4iwQU*=BM=Q_1-4nEG+SwT7vHIl9KFY@| zB*+t-1#`v+WSAD+;NZFj=X5be&`)zf)zq)LnXKz(#FO{yB z_6+(vY+S?EUVVI5Ao)CQty#6SaQNHYFcR1W-BTSMM$+nR^@l0BHHHV8-3Yr7x1*+ z!(U}iD6R9#gEpEqH)i;j@{UG@<&F~l(Kg2!k}sR^w0D<|JP+bO(>#Zn|4KQ+Nb$1M+&@fVXDgXxhnj(UF+v9k6D54mT9%VGK_po!o|T> zd6Bl^Br9BpX1L8F;XP%NRcEJL|K|HNbP}c#C?wD*JJ{v!IzyH2S=ZYw+_u%D+aJT3_xi?M}Z>NQd5m0VwZZY>jTlXeB79 zdTVXko)OWQl|2@^b@U1UT_+uyfW|LdMwag}b#3e1X^#iUv&I~e6n9iDt7mvQBQN2X z-*K@=`&j<6Lc$!J*0$68#(Csn&?)HBb`0S8=3I&oiSQvSSFN`mdZ%LnJ$)oOL#_Jk z3w)1%M)ntiamB%z7hy;%U>K}ns@#vyvZJK|f=ge0XR}v=cB<*oM^k1OpR$%l9#eeq4f94K1HU$x>~{0VO}2jE* z#QVI1tU^f@eaj8Y_2b9=rNC869_oRp&2QgB{jZF`;9SkN%m^F5gineT2 z9%Wz9TDEwNh(eJb0xNxJ&P!q>4osGc`DtzAAsj;nm#Y3;c+|H|7*HqUFXo%7a@h^zc z9;Ys}_%v|?`YRq$OjxlRZ{R0HT-@5Qzl-cgHu~&aB41!FY0^a~7g1JO^G|pLvEynY<-)QvE~5jxs&g)AxnGwm1 zYjXR}RIzR;SY1Fafu=X?&z?Z5x*I7sLLV69bQ=v*%m&^?qw1rWIS~5q(1i0Kr}O~c z@Oe@6a=TypL$h{tQ!Q3%BEDp-6|M(3AOmD;5EnEQ;eYzue1;T2PZrlO=#RTXyZ$ZLHc z%VhSN@2X#Q?w~bx@=DDToV2Qt&@xHx-#R+CGrjEyUMz82Y$gpFT@1@{tMUg!X9+2o zYr(OQ)a_8PTyU!Saqo!FR(^%0^I3AQ5!|w<&A2R-H)@oXp?c(?lYE)Sf;FB(Cv4*9 zF)#RNa8ns*91M)Ee$ehvmtp;GXs^<`hM`?#NYiIR1N2L|y-hVCG%L4P{r$DDTs8PS zZ(U$QA+uIaFTmCUvVWgQ9n_m3HUw7WxYG9!T-iv)5CFsNthIV98 zE85{|rsA<`N5Kb*HY-%=52O5$Fp@var?BaGGD+Ax8;xIUDgLd+&AiNU@>@ea*#uX& zB~hRvMc-}BoTF*QY|?rdoc+DhDF+O~RiTIdmmRG-dial!BJUCXUfteO=m04+Q*3xK zoLX9et*9sQvIWEuqoC7fjG}WUWsW{HZIW26T*0`^Zr_pR+-_@`8%uKp22;Of9e5_) zR_2-)5-?Aindg^hMhq%JmPig59>R}ZSa9A@cN*w}JcBk66RM!5um>8DYs6S~9YXL? z${$prxIX<>C$Fa4{gvKpsOEjU(EaCV2XwwXKCxaS!46+n)1X?ML_2be+ZWw+tkLf_ zApyF*Vj5&cFpFM-KBX1$v)olm`CEUxNYbfnf5|6~*b8*RBcWQhf_`T>HFnRq#rj^nIJs@U7VJUavD8EEMkW?8M>w{WJ@>TmI}UbFnzD*r3ELV?E>)wcD_P$JeqV1N>_?eud{DY+i#p!Mrnm7WI7$qzaU#dc(yB9JGcD?W+hAGh6_Ba zSJN<*I2M{%U3ker=9+=-a<1s%!rXTP4Jul#w3uYAw9Ojh=38(jwM+p!_TU=pvQPZn zhfcJEK|97r!s4viD-(H|w{;`C=5i&v61&Ac9ja3#?iw$+o)nr9ni879&zLn{CWU?u zDefGJg6<>vD>?C&cBf6HE!sX7GJIETIs>nVj58VG5IS56=7c>HON5sq8Cs7`8C-(LLL~|_8G$k z0oeKYFWP8o=$+9S5G7UOYG&~3G%*VKDkJjT1<-7RX39D4)e87Qxz0MCspLlkwsZA# zmofPox{*W72AGi7$>Ln@*66qBHld+6pgNiBtLAumKAB2kRxI& ziD+Um&G%)?5Zi3Bq?vPJz2lRig5^VY-)kqIp7=1#D0sEr72fd!imqUhutzg?{VqDM z1(FqlQ-pxH08f79-N|mKh=eui9rvDzb&Krn!%vq{Hgcj^Rw<&kI*ZdwE&<*Hz3@B3 zfIP@{>;2+T4ha4Ln5V(w!D)`9y~BOiivTmVuMu!R6=hY7ItCN;O#!iHcFYycO zlDRnvd!1n;O=9=N9v`O2Pu2ZaZRyeEc9xx zG1sdqtk>0&BN1^V@yD$vzLvOAtviZPO$~<6;swcTwv|4C(S*w&0jow0^hvN7|}g?9Ca|$F`V4O zxa==+{6-SB;huCGE@y{ki=J+a-vp>^1QXUjOg_mnQ^-;;G*__gC7iR--ozaytF2Ng9LT4AO;u>LtOltW%QUXyLTLZnPY|qwO<0|MPU{ zR{!hh!^>8fa&pDB{9xHt`Vl*Sigv1-^$kbmUkfq4gw&cDizwL-kWdWS0%uc3sB))C zIY%lHKA0Ymf)+t8x#tF0qP?5i2v53w;zEyGCqKbpd}@*j&oNBI@0vI4Dp59-kY_bv zDl(p{L-k|W89Vc)(W`unza?&nD-mh6+O-l}*ywYu8GQ2NnJ(I#F*P8CL%r45N&MR< z2qS5Bx8ls*)y#P@ZBXWz_+ymyizdN-f7XP7b#FZ-1azXe0z3oEM^<@vWZb!ZMrDra zpCaM=#Vuge>x^PvHe#Ha7SVo|40o}o%HyZe80FdELN)n;x{11o*^gxi(fOrj3%-S9 z@=TAhK{h!1DPt}B;!3t$ufS`><0m}19`(lMVl=kY({UgUqu?*%NAGC$4<@L(u+t5T^oeD;s*-t_sAn+00(R?*!y%8}1& z=T|;C8*$fITJoSnYx(q8?`We3j68Z>e`Fo@)L*C_&9PiYGzQ7GScLJw2(gkmhL{V$ z!kSJ`8#kiTIcA#jEU~&5BeXn3_8UXzU5N)y(Om+;V91{Mg)_yo#=Wx{R-&Hq>qIAW zjydD_geb*&4%Rv4BHUe1puM&L9`)%Yr|>hT{$9Mg`WTEmn9q>vpmSzrk+N;t8N9m& zV1%@KGFSJpMrVRN*XK48`N?WdM3ab+dRbQY(BAtFRD%+YEcO(B93_59Rf@S5xI3HO zrude-erwbAC)K&0ZSZKfXPAfOjy%&ps)Br)19k`0)#xxzSK$rIgkx8-CMRlg*G%@8 zA$?LH?U+dm*Q&@SF@gE!;fxHhTOHV#siNtTfcYH=P;^)W9z2rdK2fAV0T<|Zgr>&+ zP-1#$VhLj9bm9x`6u%=f$b$)Ob%b~dktHsSsM?Z>7ZT^k?Ko)}k8a%JRzx4Y5$4%8 zJp~eS(qsLT zVPRaz758dLCdP`jD9uEVXOA=Q>o&ZNWnN^MJ=ZerRqHh!^6pYU>62sEb@U`xn!|c) z*mEWx^QaP`wJ6Y^Q$Ei2ksG|QOZH4yZntJ~N@VLWn9D>lqwa;w*j~*J&01tEwJn|9 zoF&X+`}-vGmY#I~Uj1p}$zuanD8EM2i*qB9J9_f^h*brAeaw2jrE}qJ$RT;39YC_3 zPw}L2PgM_})LjCdt8m1F*@KRhSI2Js-n2gwoxYj=Z_4`%2|#E-O{j9tBUH9t#|%CB7u6uM=!bsa6(gR89AekZ+67^sa% zqjRy{5_hDM<_UH!7EJ8H6bN!33MU1(5bix$w0o7wPt!Bz96goz(O&Zqqc))a_hG2J zzzcR&GjDHUAHK?M+ITs4iVT=fq((9qcX5`Ov4OUu5j-xKYjqkqfBq>RC+^DMGjmsxBFesG=?Rx_`TPRW7=yN;qCJbp%mydeLyC_hf#RWf@8j% zHkER2asgB4an(|5_m%|X$Q*9VbeZNy%|_Hx^4e9kUZb*8Ra2uGzCQJ?+|{mX>Z@#{ zc}u|W0GLU_4!i((pAXUML|&mZM1XKrR?kZykQg8n;e#XwBn3LDbns8&Nr%UzS>2(l zvi0_S{hKa{UkNeLX{8_P4_G&3=>!+b1y(s!*QK81q?IiA(GZC8-g^_JL*NB=NdASJxu@$(zxwtI5n zcf&^XK3(OitZtAdaY!I9w#D*lMlsLb`BI*L**L^~CcsiD$B5+@=oz>>aIex2|Ll1$ zrE!0AUX9g!>jC5mawul8(`|WgUujXUs4dpNA661*9sshIq^Zmk#!~cgCdUo7nJ+YE zT>w2liErhioWm!b>#Wo1S&ooE69saFfZZQX-x@Rzo0#-s^sQ$tlG3KlAvq^DgkQ;Z zNV`$dub+cG*ci{VTp7HDg%`&eQOTuRCxuW;ZoeHzd z;%Y~?#-Yo}maZ>P$u<)nVfNK9YeyP7szbWUpo#_!@!m@J9%}vcJ`6JcV<9b&w>?$v z+z@_95rN`Pv&gl$1Br#3kVDfKIc6YH^Hr23nWvS^)D&}Pjb+B$dbJghj?s8`$)ub+ zpFne$l%Na2VoyEkgvn4`j$6e#rh$d#2-+qhKPbDU-q{!XqBhyEqaPyZNuKx6_{yBB z*b6#wx;P}GbGmE>cq>?A998Is4X25*(9GApvd1}3SH}yi(RE=)n!(~pO zYNr~&a^Zy=BIK=yJ$<_%Vt}l5(?hoQbW<}S0-B`sv{31I2Dmog`r95NQu`qoN-5kn zW502uvsMh+TT9+$j8ur}*mGUWpyLk|QHwpFB{8Q7%#oYZ5{&YF)9qGB{uK^T8DCS6 zHngs8np|1x{MY+-qjLnBr`be(X#V7P*MCYuyd>wvt?`Q+wvMiD-P!iUx9%179-L0f zJHmNQF@v`f3zN{^Bf=|tmG%VSnp&B5c_yW^#y111&{ghKuLJq9T}K+ds@!D1gghhg zRVC*5A`h#ZajW|rUS(+ULRvy-h0y7T!HMj&n}OCBgSsrsSrs$SpBk5kR!sLa?25oP z{iw6*$t~YD|DA6-Qfr9K76qp>^r2XEkD*+)e+8FK2<3l0Fr3l7f2MI#gel`iwl<`7 ztIZt$9wI&$1q|4NViCGZLE`Czmb}SRK%LG>C)p}O82nJG(@~`nUM(a9n(X|f2k~5E z7Jev;t~2D7*g<$q>8hJ)5K*aL3#k?3jA}GS8s61FGdc_|mqynYZ_$QbDXcX(c{Q2-!$cK@` zEH}Me?|*!N(=9cUS}*O%RHj&`SI=5IZwe%p#9-0MKGn)T0RxizCNc>@k53pWQPm;m zl!IVK8n%p>rl*CgZzwyMkH-fQV;W>eB#2k=2o_@=nx*ukAl7JT`p=!-h@^<+fjE@1 zpGHaDIJKDC(t5ctXb^&Rw;GyfNJKsUbB^nxh($~5^GlOIm&Brn;GW(PsGWL*bB%w@ zNtP2*49UkFb>5pzH_rEhJ<_NG=oWUzQPg$)gZ6enmR3LEB*g+OHwG35e=W2zxOiUnsXX)n zY}QIRl7f6eIyVT3?Dj&#JL}LywJ_@KiV&?hv@aBAV13fZhKo=w>>2gq9s82HuJALbAy{#BdbnUA#LX6DOcbI_0UnJ-~b=u7j_kMV#!)Of%gurko=gD zh$BF4L+A2*?MY&2ju9pY?!pn~pR77#Gt0Dd8Q$dx>*>K|Wr*KEO0?46M1O)Au9eEFkY$`;Y*ZBREzi>T+jcnj}{)Iq{y=zdk)%8-_q9gGA0HZY%F zji^3biPFLzX!K9~wD2b>AP=UcUJ{~p+~7)UobgV> z#H&w6CR@HH~H-=_)!Jj3o9vzFzyIyGVK_< zWppugRS4e@pi@@A^j)jOGJNoaa+`Xdy&lNU-VBrFE-ZsAsdmcxO_$F^x;|s{7+C$E zw$ArXtVL8gQ^zL=_0ab&N!blyTp>H!wbx)MF0sAN2tTL0{$X`T@$Tmykw4Q)Pd(17 z4$`p@1uj)gTKDAY>xu%XIKkyaglr(!F2QR1?%;LP!WXic7~?h76yOaglK~Q-u)$-8 zM2d6^+wk;p5JU&r76Lk)6*+vM@{nt11=YUpm5tr1q>gI(-I?q3_crI0SoWVS?f zD1@gws)r{I5z-$b0+#%_%8RWbpQ?7<_=`Z6>Mg1~K~2MkC@l|gWtdJqB#y4*59Fh8 z+K&gTp?CwsZhtOIL2>?6qTw&us(THsoiaVN60&{+!NsCfIi4viYXSIS;l5O`l(VjJ z!md3txQ!`V?WXx`sPFd0V=Lor`fY8=6E3ewkEd$*ImO^xY(QPG;RdLrtFBKuhhd+a zKWJ^6jdH5VCCof=vg~XK+F^E5JhR50NtISzIwQs7nrqp&c{!JbRiRZV+?&U<#-flb zqAS+yb+Fzg3cu|oq01>CZqCi;#f*dExr8iGwwJjD6wN1KG#~+rOR5Q&#~Qh?5hHO? z;mlXULugC~3I&Q_PLVYtU9dzoxKh!V<<~9yifyV5>*K%;hY)ub+PtN(nKy0+m?fw9 zqqACKn~^L(VV6N zh50rI;HPxKg0*OT4RdtAgJ5J=dHibGbmwgne!DW=UKvRTu7MS6`=sGTb>P+Y`{4hf zX@$38@#+`s^uW@UTDXqE^RYg}y2dwnCF(Ze11he!w=Gv9z&DI{7ltO3bRX$JADpQ|jfAZ2n2+X*X|wI@=-?4MWc#~F^kIG! zPrcauoBMPcn^veD-NjCdn0)VyS%|Y6t)o@U;jW90JsOOz!BL%xKD?+tUv+lrbt^7Q zd?-I`iG?`FKeI)6-e#$CvIvS6+`2VcOi^M-)Oar~CQH6Lrm+@raZk~?pwALlcj`as z4alm~omUYy+E53q=X@pIor{cZ2$??2WCWvm5n{iUD+9PX%Yc40_0w5x+DO2&HANtA z%K2>8Y*mn}@hNaKCw_-iWo67EwrKupXOgZ47V}#Dg%1L27GUIY210?ae@b?!&r!?6 zdhyvjc{~%M<&1f+fcIKL(tPMn_v1wAvSyRT}+y6_E#FE4k)^j?Ck8&Z+4-#LnGz_WOk z%++@gFG`qw+VDRD#dpvkFw8)FgN%#?;FiAW(e#6Wg7!FLdI-%>4yd27QhIxwHEo0> zV#s|ujWeJmjEckhnjpYbA!ELupFy291~V|9L!+&)o%hAWQsSqEQji_(S-AnZs@PRO zVT!ddMeAVw%8&aKf%@@U_&9^RQ>k<#a1%=y>z#y}o@Jt{`W0Wx;RgoPdjs}?s?3<% z0X>tsJ-QXTfm~*?YJ@L{{Rej9CwQb{)dL}8lM)KV>PwG=L5p)i%EW)pNt$1|zx&13 z@3E&XWjmP3w`4$pvAy!97Xa$(L|HZVcKP4F>vH6QNr}XMe>yN3#})Y~_i0?PQ;GVd zo`JZDk{yE7Q|cZrR1;@SKq_xzpifu_AsEVahG*SlEp)_=;n$8JeQ7NYI%&;iQR~g* zL;lNE3nI@pHd~{wGBm#~{3?aq;m&)<$<^jh$JUq6zjroP(E+vJa80sZC-0EG*;;nm zmcg^3MLT_B@?*snj)hsg$y@(a$C{HSX+64M`eJmz2jkK|DX8e@sL7y~de@xPI^Xx8 zDwxvx+mxgyTA;FPeXhmY2PtbT@|x!Rt`(F;724H=J64~ohu(y3E~>@41p2a)cbL74 zReu+@G{odgApisO_?-Zc1QtrEn{@%#ZU)FEp#Np6dH2#NXt!VAXrB29=kB(8d%~7?* z-yIe9jV;0JD-E>6##~DSX#)-4Fx(qEerkJ62_CY&Q+IBUPVc$y;Jz~GR;+3}B2Hao z5A*bspU4#6QB=v4QWTT}bl^h&SVNs??TVtqb{PRRTVn0fN%GTfbZ;t$AVD*ZgJt2? zF7U9FsCeXJpOByHyP*|+wQfw%-Q9Sh`jY;TdstQjzB+X5cEGM=go_m#WF0l#N}Pwx ze*iK5FM0^Chvm&T(67DY=ft&b*y-hj9I*mrP{_rwrAk6Q{4WxAQL39*YMCsCN1}*m zwpG?>g(LsY7FP03qS#QhXuz>>(W(9Ul=@VBM)KXacxkLJ`Z@IH2lj$4c6PaKh<(v< zX*4t&Ylfr+HLbOOE}*tqqVBtybqA_)l7I8DtU@F;vEux$i>4zc3*)FFmPor1-DNr< znnK~;s7w;nw*6&VG?V%a|Z3ZoX^&b(v2BBu4s(v ztgnVzU4WlRSB#lRpAKJ$sm|(~v3Q5T4YDQ-rz}9N<;&UXW2JQT4%xx81lrt76t1Wz?qHraT!%dWFP*1dXV*hEua1x;f>SM58ZN2(= zycSNmIwQ|Y-iwPp_OHDEk@sHzYDOOi$K-+ne;Ht)WXEV3G(C zQt^7tnsr09*F1~nt7C!A2hmDK2D#En$1*N|(z3`L%VtlHlFXW^Lo;z!{o8PwD(fFh z5Im5^XovAM_QS7|R8aJyi|Z z{c2gf*m`wAY2AoGAEvBh2XAf~f8Aa1iNE%0zuF||MkF4R251;qFb%r>fL~jb6H6sF zfB>@XS+bQ!QVTQ@O(K-R1so_oV-tNBZd?=h)a(+wo$#7uF6dYjhRf|{ReikCQDVcZ_*E0`=^vW&-9;-2tTSYVr6 z%CPk@05CXI^v2JtoTd&e4a+6e%oQwm9@u@OKNL10h_-Y;ZENuxD3Fa7@{DQjI<&R< zSoQ=Fl+RX<8ApR+U?dU+647s2k)Xgoq_a;jP#;&kj_Z)WL3DT6+;_hhRbRO;?wPsO zY~10ErC@&5!SOUoi&(pISSr{uoo#c*thYO3=IPGTxZNYEXD=}N=SJRAR{NvYJ==ba z;w$&8UGps_7CWS^A>F)@FY$$-#=q8-qdOrF>hh%d(XQu*uWG2Pz1pvGHRa8TuZ+h5 zfgPq-Z>Njera%j!EMK3TS^2|KKf7y7?53BG_nt|bI{h%Xf4Y8o08F0dHIb6)f7k=f z>2vriGL zz3M2z-`6I+w~kT;?^s!ZtBIgk+cX<<$7gC%7u0JNUCQjX<*IgKwAL}px!JK{I)hq%n8DEq0E$)@Y~EM-0bUT|Q}86aKO0#W!?CH{4hFxA$RSd=8ONM+pI2?YO{xqq?En= zagRFFYse+5<%d!ITxxo4(?VoB5T7*qSc_mesSY8pX%TAEL1+fq(01Yp)3gt_mWoKc zTv3hQw38J`=vKHE9IV|l_X6Pb_! zpX3K0Uo~@2PY3Qx9K0CI%L=5I!OC70>~OMSv!O$ zn<#KAS!ObUertP4U@EfY;YDy?U*MdzGvH7NaA0jD#nCl?N-8i%(i=dP7MwunrX^5B9n7WPlLxPsjNEjwq3l&rr+^dIEe~ z-~8%WaZ=N++k>TeL~_aOhv?MunS0ZEFRI1Ou~#fkBVQN)7842$$pzKj)G;1HT$nsS zctiOL|3=zxG8CqD8Md&`G~ziiOe?9vo_f>eC^?ioyZpsY_hr+N;WBw>u0^#>mbV}# z-TUFwkDGY8_>Hm}wk~Y;RkgG30_azd{%)|dg!3IFNJDPp} z{{2yD=%M0Z@_`l?!&YZmix{wZLE1Lc)Q4EMntKoP7w}j}I;mIF)*+8Px-`S})oEDy zp>@Im)*kfR>W!`~xZT)IbYom;>ec~w#}m6vG&b>4z$RfJ;fbx@&?bMqcf{paLl2wZ zEMNvsC4oH!m+zdz2tPaL)D)-c)|WLr9X?XseV{@uKiELeL&1729PB7LLp|5G@R<~D zbV>{3^5I<|Cx=?YgVaz~;fLZIH|#{Pj5IepoFMN9L$DDv0yb~8(0;0OLD|jg{83PL zpDivwAeQhNBHkGu&1OL8MFtDA#$c;t`xfcqNqBPZf@}!0ZSXo-uHqRq(Gj*$b2Dye zAnZ?&m1&`Q`8J+@@cZL^QHwKUSNdJD!IwyTm*tQY2AkI?fp>-x(GGv2Y`#motf;e= zGj#&0qOrjFHA!T<0i`4DjtRz1AL-cTm`4@~EAUGk*UtA~?+=$YcEF^X5j%2BN!wUv zKGtF9de~ouxiWdj!Jg|^8k$%j;i5U*L`Hi!CO&o)@#XdO&o@2Mp_clmZ7SN{96!JJ z2np{~P)&c9T$|QB8J_Oxs5>-sn~I-ey!m?= z9>XWr@NTc+J8Y`B? z%(vIiwmj%CyrCtPjBYL3#n;YAFg``ZIIM`iT~0<0T?d)QbkSq=k-r{v{+D%j6&RFhTX3)ijrZ{5VTB!a0;$UX z_%NAnj08I+8c|KXgeoZBMHnaTA7h=+Wd6fSzcQ(2mDlwDBO?_Jgg0;NT!S>$NVG|X z&tdy%i!V_xZTM0}Yw$iZwfQ_Fgm^3P=AK5LOxMVJZ9p%W`1~1bk7ITBuSPQU5{dR` zwB)4av5f{GZs>eE{L?e~J2AnhSjZ>5frcvVv$%j9*I0vJAmteeI;v44s|Y?%U%6qI z668!Gw-sCHVFI`DMYE10|04TGyzBENK4QljszneJ*_RJ`v8QrCe-QyzB;Bg*X!MD( z%K7S<+2v=;>E)O+mNKCuF}pJzL0o>^6#V;L5AMTU-XvZADb(|qnSD#bS&42t4=fkN zUaCE+uAzZ7dactgz-T6)>Tp)*ASY` zBf|29VcAF94Q5Xv=d|Fvc-B1N{OLs8+#mbG+J31z3Uq^vO_4PzOyolc}qma!+< zi4fUl21VIr%Q|E?m?f1N$bH4BU{d4-~^!Cp4-p{>U_jO-4-hEUvgqdf%Fm8SC zyx>_qGM>$Hqt{gX=cPcHuRCEVDHD!RnSE%w@Q8&RT^x!CEKi7PDt_*S((LA(e+3Q= zGzrdTr%c7#Cdg)_(T81nyPdU^!W@02N2apej{4vBdA9djdZht^8e+0&1LXm!V zE>2jmSL(7a88C-dxdLpJj4)CFm>0@$rl&te9u@YO>>uz`xyP7^*R!8E zo9*XKg9v=vIAmJ!1F2&! zo)`B!`&!MOy49^LCysC2enKDBu)AG|B+qxX^B<^<^(o!!+<&VmTH@5fq@`sdf_19subsud zilE_l>m*%(U;TKdXthys+x&jIzv!pBvcCAx#*Hm43)B7jQ0KVMI3HZDRy}`pF8}>}_sDJH}4qi#~K;97{jcnboWOX}1NZsU?)2M-n_8AuPa|x{?A2s0`?g+8% z38aH6V))&i!54Fa?D;>`oxD>0yBk)=YOI?`aQ)L|L})BL!1xYmikds$j0P)21qBv< z*Y#V-u#B%L?5EqQ(KD^#4vAozBJ7kjpOzbTFYCSI|n%R8DfRq#>diJ(oHk9 zZ7tdI(NmGwTI07opxDjxyHga(oR4R?S$?H{n{Jt#eJV5;J5k?}cYc6A%w!O7hsg&7 zH{ZETkhh)Vlh%D*MG_!P?Tm=%zjqKZG(=*_^5TX%kIU6AMmgW%RK>e?B_qzvrRWvO zt>doF!pQ7FzSq(5P zAf$p67yHNDXD)S*0>^;YepvDBgGTrL8zI#d=ejJh9bR@T_TFm^hPvx*6F)OQ5JLtd zUp8{$XZX;7x6rFjr^_}NZ*&E1trVuZ+S}8|0PmuQ{ZkCT)H~4PHr1h6SaLLg;J;k$ zKD0|m?f$$B3)CTL}u?7E>DYbC#69dK^ zgEX>?*~_1(5S!Pggkmf zB!>?@TuPI8yd%cyIR6~_&wct24fkAtKV4PSu>i({cl$*CuGh1TzqGJ1`;&R2J=61i zGSgZIae2QuO>JHs&>D8??H6muVWk0F?MAmeG+FECIv9hH4>RveD_6@#oE7?!C-`ev z2Vq-uzh90F&lU>oBwE{_f*EJR%XvpN`U{UU03pkqv5-Zrt*G>)wkp!v%VHaTsJBWo zqJe3F9OhqF8i;C?+yUM}e{7cMP?Q$D|3OXUVSn>#r9Yi0n&-Rh#34ogBg3BUfikVa zP;yk%I*yTsHje{iRH{DW8a}VVe0oL@T!{`gJbRWOy7r4)_O4^~+o4DAObgP;>Vr)w z$pSkIjIVm9w5kzhYlx2dE(3yMbeYTX59F{Hu_NckG`4yYVe< ziXA6vy3vQ;%SYR(a%3p&_Rv!F%pd!&1(`4GMdD^E_xRrn(w?8AmF^rA0g_R|@b#ak zWVPC0zLs&UYy!cxTT(LOEECX0;01DWfd_}av9Jn!eJ{Ag)N!3@ECjUy?*5TIa z96UTHA5NgWLk=1x{hHtt#&XB^^OO5>3_I>1CoH2rBf!XG5f#?6D3;A;cE{6gLW-xY zua$Wagy*&hJK?Aw{yF@iS{BUTlf{Xw8&^4$l=9P+*r3JshcBITMy4rmObcg)8JWul z4yA^pXPoBk!spA2!rM-TlK>gghNE^LxW3l6AO2yvRF5Q0`tHsp4;%UC_7Fh%44H6{ z;nt{y(@#LV*tQ(R6`t%~O^U%eKimu>AAv-8f1Lp*#kJ9aWfYu2!`ESh5O!0!w!BSu z$yA@oKVPa+wI@h@PF8HDMXVoRdQ5H->%Xm203Q!)sVbb&K6QnO{Y6&JoPX^MIA4Yo zl3iKL7q%${bX5Y9*@0r6kK=K41QVCp{^=sirn;;^c!YqjW{2i8gj`-FVuR~1a5OZj zhhh_ww~=i~(yG0Sq^FpB7HFl>AAW#*hghcuPdns)1e}zP?hLIC_?EybAK(gD#eq(* z1*H9wY(1#>NF$Tuzf=MD;Q+fAkXbq_*WKFj@T9`I!3JJ`Q+p%1`sDPbg?Jeiol|#1EtAa|J(ghl)i!hzR9IJN+%*20jId@iX^sGJ(^B@{{qWx(rn8 zo&$jWztN-zU%o^TT7w&q)>fk?=FoWALx37b;gl;Nn;qT`uCVZ33+AIkz{-bSmBn*w zoV%)hbfmi3!6PPX#fr{@rXo8UDO@>2Nn~UKsQ&P83!vzshu_?st*EN6f%hoK24`pw zN!*VR_p7+xKcFI}Ui}v#aek&zyR=Jh+f#fH=e!@?Kd@_z92UYCf|dzvJNloWszYIn z+@l(8H`9twp%T4@_&OUSv0ae)gZmpjC0;9uxMl~)k>l5gKfd2(w~Q4&Dn>#1V%$t; z?FPUNd}nu2PqJ%gL3?NtJt(0^)<1>YZIKTf?q&Zw!lHheS)(q)i&kCb6Pw*R^l!L& z#W^f40xSp;o4GW$C+=FE=`k6%yO@=or=4@eAt5%R`45&F6e?^Zc7HTy1$N#P@9CCIqQ1UefWAZ(Avq zgCsjY6$R|;TY~~b262s%b##%-koI#ja!NKU3a4}o4(^*QeC6`fiV%M4o4U`%gW+;# zwmOGRR@K+K{pAD1SVM>y79J`ActK7FBH;6RFigxu=7YSmi5-&swUEO(^6W&1lL zqrgD4PK1A#@gJnkhBD;psHHtnO{nap`#Ia#u=H$?Jph1!97?aZ&A z-N%0J-?T&)JXRr9AYc-kblxS)HWj%x@#CFY=%QclN^M?d-{bymtS%7R0h7Yk?QkNf zhM2rLq>CQ@mG@tEbBdij@Ns}T9)j?ZZ)=CobewhK$F zZ_A*y4W~(uM&COd0wBt-JrCsM*QWT4jF$EDl3*8c9JZ7qZYb=Otp3Qyp)8SZ>wcH- zHi@GvKApNkd6W$gd0pOh-HZj7ach+j5)mk&WCprea->iFlZ5m-F!zQ#U>>icKT1A~ zaHX^80{<>a*kWy|Wu4TAgs8x>GF+;g81`g4qf5%qUpXAR%7BgO;?mgi>dr}d* zb2kvE(R5_$1Qq4I7UI(b*v}luIo~DQ5zcYq`wxRCCz6-=KT-GS-iLLcy{j)xKnB0b1WJWd>Z#q^6u3u(1;eAjw^Umg>~WYt>5Q;=2`#I~ zYvZ<2&BT;y=%eLy+0>C%eF&tnAwd2JYIzxUe-8gBb#K_kY2*bqe3)O((Xw*WHem|s z((K^!HQOMj$R$f?-J5`Itm|xmv5yUYag7^_nv$>h5Qg_(w>yB*w40zP%t-}C_e#j{ z+U_8c$n7%9&wP9;=v}E?p0PQ%xVvPX`*Bo~ri;`wcm7bC)oi+(rxojub6Dp_*H+y^7iy6&t;AxNWf&bW{Vy_?S8nf3j^ezjz6?uY0^hqI!;Vln|)2IkCX8Ga7+io{7|YmTiNDkL^>n>m+OpWMIVB}FBf17eB5yT zHdxa&JaStW>Yh6t{Mo-#_gtmL&#}7RHg1Z!@iz6m@r8Ur4JSZZx~?9=P;**>0cKcI zU>!c(qpfGN)Qj>k)u@9c`$wLoG|LIr4C@l6(NB(5sx*|xurnE2|-EC{O2 z%vH8(!7lWhllN2|Jdp5pR+0P&d(%t&IWy7p zYon2$fE|NdUofxUdJx){H@3>xn<+7Dp1e*Psm%A7+=F46OwS>`#O@`c#xxQaEy?g+ zUXhWi$7MpTZU6T`O`OfLzk--QQc;}HVw$x65|L1`OPN0^8VB7BdPY zNl-qWIKz)l=I?J+rs}(#+KID@#sxY^5TBc+#clh;u>9rbZ;?$e1weVnI>i^SyXQ98 zu)aS3+S+OX*gfcDAz`&LVX})hi%(_`OljJ03+HGbQ@6oPYJyx=aK&u^v{6b;5sTA^ zbY{01V|fAG%3yZHp%srI}dq5*w}p6cXX4(+$-wS-IJKnwSd}5dFHc)r7aN2?eC>kG~QUp>i8@;&Ak)EybLz}%+`u?_yO49K4{gt!< zqx3-1S7!HK`2X#9-uSpc=67sw@L)!(OSVMi_Whw|LhTN%Bj_p=Tr`4m$RC-t*vng1 zZ*o5?#40DCODw4@l6@s4kngM3;y--Hdcs@K5?^QoI5a?3=6B5&xL4k-$E3-F3e_*O zU&vJZW>mv3WssHS(?a`zBGHWo|BfVYVUjx+I@|02p46k@-FdUjT{{4M@}^y@oy$f& zN$~|2rk47O^65|GS@xZEGICex>hqDHyxN6R^O1XnF2sC(t8mNprV=UE*Ge>6Uk0VG*Bh z{aQyrBNXs~tj><_lnj@H06Hiw_X^0;#;KfsDClSM2rIu=@;%9@YnU(>!e;tI z;l1;gYM0q((rxr%9uPdXCWnp5X&_=3$e!$a;W5Hq1B6P7)TX^=>#hXU-@Z6``c=`7 zaYU*|cgQ?LXy)aWC4w?^ZA7fl|8!kb%b%Wq=IlWkc0_g zl{?w}qJ4TC4dj>bbZ04s4+W#8)uX((AK+n~vOr|>e_vPT+(TABMyx_?+7*FuKIOGz zAkKN3h@YBcQiW!`DnbQx_jv~5xWI|0fefac+s?dn?21q7F^%EhEI@~BDe`?EdmHzC zF~tV*MIEamYWwe9A(xt`uIQ&vRlB?FRdk#!o-dOFUAjp?v3fLgss;u+i}cTDBl67e zjNi`s36BPL^mWhNOJTip-Lt#}ih8o2p$}L$P|~9EdoA zOk95%wEXm;`^Bkw&z~QkIJT`M+7@laZ3ovSG?L2X=I>0(u}+zH%~Q`0zUQ_|Yafv< z=n9xuH@@H+`s9bjkgTI|0SfLqbf|r^@O$!oyqep|Pbx4LkcxzQ^S;iX>u*@!qF>9x zac2wJfPg`K3}ir(C+d;b8=@V<1lfMQCyRq|24Cw2sIY_ijyTkEE;kwD!r{ zF6Tqo)dfF+*^?XTN@_FQ&xIK8_@2d_p$kg^&}5w9tY`K7(NKuB>qPb0vsOJ$=B{_F z0D|`&*bq09J)YG7q84m z6C4_`nHc37ZJYMyB-E2ZZV?mJF%mHn&;3Yf6!Zbz(gF)1dwY5vL!)x@^6g*kVN% zDD3LhD`p(mJECt|cW|Rc^U_zmqc>ljzYaJC2>>clsi5inE)NDb1-gdku2YvAq$`gy zTEET!8B4J_>Wha)0;#Xngt3K5#wy)fk5>vMd~3J(El; zdjMJvgb~y`YOs{j^$b#QzBXw049>v>y3#gyCJeitSeBqiYtcKsO+ceVo(<9ynVWj0 zL+aCrJ*np(KxG?N4Mg)wb67Sz*MK!5P(5A*J5Pyp1c$D!6N-eVOSXAy}?QJ=zyts7*%ln8Y6BcA$%7FRY zn`dWHrFw)r^^Yxu@!VEGNG#fdWuZa}E>Ga-Z)utSLc_8Z_7kcZWR2}C-KRpF4rn^X za%bE^*!2n;r_jh$r04pU`00U=ZLHho(WTpVr3kHuLYzuN4hSh-1w2;)$bc6VqmIjQ z$3;9Z626aXN`7x>OgK}2&9>9$a?Mv)OytddO)~tesb!rld{1r#bcSvNaAP|pFG$lf z5*H!};rbplmJRRMC-$BjtSMy20)>_$Xltq|!ZO#Vc{uXZ%caO#kYmC9{rS=bF~Ux( zr|>84SBf8}p2+WnEM~{F!_%}|DUes3V!fc{PgxvO?YUWYtxpu zJt?tLmAs(~`k}Kv6)NZG4z5xTl09=M6ICq|5y%v^qNa)0iY@0_JuE*eP|RL7Bz=&q z>C7jwMpIc>4H{&4lKx!dMvL{|?1%$BGi9##Sju}8@&M!vfu70>@JFkM zVAGep&)$P1z`b|X=jGdIJXihdCwKcB!m)tp=Oa-#p&zsKm6Qq6kH+YxM$Y%#;p&1= z{5Mq3!~zIfk@eXB=dttM#n)^-SYIo<%cG-kRejf^@o~Y2@3~>` zmTg-G9gQEU#PYHd3STVW68>1}x`lhjufQewmM6essN}5V+mVG^QfoBZ;)T1I6aU6z z@JH;x9lcxa*=^SX=fMFV6#PpGdEUV^d3lOB`_n9xvbVOQlv}&`k`Z>?;k_2o-j+z* zN)WFkY^Rv5G4AOoOZ!nY7b1h}yS^iT3Y9(5bw(7(nfU(_!mP%cH}`|mK7r_6ET`rj zcth)XUY5Ezgj?v(T?s+!u=KYw#eM}|5PwAqA6m3=hfe$FESgi3_GuBKYX*dJfX01s z$Xw=>GWCRfaB}^6p4sHC-eam_7OzQjPrNAg?BtBAXPJ|ab8qkC?<{1{Dp^CzWIcV} zk3liEq2AR3AkV${x!}nw_Tz=AU#H$4PnW;23QBq%K!{G2ZImk$a+ zUscFA52LFTcrok~f3dSEFrV3W@zs($2mq z{$QTC)jppkbo4+ra~iNKt^1eB!%zLKRGeAJPptt&BDVZAZs`;|<9kl-%Ey8P++HoJ zJD@}uLY$=S*Bf%q+tK5IQ~YY$q>AJ?XYWbJcE2@noOC;@yOD0;dZR1; zB1%VMcoQ%rQ;*)J@5E%u&kZvw*LK2CJWmi>y43;!i-Wjn%=45VXkqxxHKmoH&HET( ze<=oP!Qp2nidBSLnmqeK_t4iy^x7Eum;3|V^$ITY# z)i?GHD3VC_YL7{!hC?;>J7OUmc;_V<$;&`C5WMWXK2|%fJYE@ci*R5zsb{*0N6H_> zG`J&<2U>M?E!?{3kZpS4bL|$`* zY=+C99GBy9s^bceabD9MOV@DiUa7B^6wSQQGI|qquk~nBY~Vxt0a(vi&j7-EMvES3 zoDs^sjl8A9a(%zF4`TlO;*}2wtm53}pXMhzvkzLbwpFn=i4~nc45?K9a%N7?Vfk$H zg1x9+89v>?@z7A!{0k<0Uk;9UWOyR>aX;C20BO1~L?ymVOgD8J0)svR zP)qvHT#hhJwa#`uX`ZUF)7Cub9b!fcLx;kif9*5r=i15M9NLZfI#k2wAwINNcfOL* zKks>9_Zur-{HPViZ_YTkfoe`+hX9ca(d_VAs)n#vLwv7nK`=>$Fjinl>PCAWSP}z` zx-$`jGHlldN#~+)NcsaER(zrFkD%RSw5ZS%J9i`*)uV}0%hI&{;YWdRNbi3zfXXIyT0`51*0-8>54nUhqd*$!Wa9O1ALNz|qY$2?cXH8= zc2ue`#5Rz=80}=E0AXhgarW)+4JZQ%yJV>$8HEQxzBcc(nTaIvd2-YmEn9$9WtHZ@ z4u8GQBZnb751T|Bf*OE?9RJcN+@DkKvZK|Nh%*2^GOh&+j0?k>6+ZC=US8WDx=8iwI)=};PN1eKx#jSrT4Q|U}?tq&wDcXB+fvjPy z%Zp~o!$0kQ@#?$Dy_SOOKty{=z2y(4t&a2P?rl~mFOA0$6BPP^2OnEcQpp{se?{Ge z`bUQq@#6%$6_sH)x(k)!Xi^;D_gDlwu{d6y zpj75+(>|l_vK6^-f&+pul5OWB`h}Hv%!L`3S*N;g83A$(OkkjN+WtuA=?A-MfZZ4D zRIuV>e-%+CLqcpfD7;Fa)A8U)70GwJk541s@;*PKb!`MtA4pURYX0;?lcTo9!n1~0ROj{4vP{m(AJWsP3@cSwcy0nOgh95Y~2}a zye>=*L@3U+hDR<<9?nZFp;vNEqrTel$eByfN@Uo~u^^o7< zeX&9mr_1yC*Vzu{q73Y+Yh8;SHkcU8iRaC5dauPJ16s8IJOy}5bGLt$L;0q3vd-Qo zQHf8LMXVe{&ev}ztv8iDZsWk~E99~Q1`r;F+8^S)(xpT2MV#VI13;y(ZWJs$2#e^H z$2ydwO8@rYNB1)rS=5<17m3Grel2`v!OB*=^cyY1MhbT% zm>FxewFMHn0+56*f}J8Z(kwnt9KrZz<-p3uNBPhg;J7+Z%i*1B3lD_$6G+qf*G2-J zbaJ}b@v*63Q^%&OIoLt~^~*RdmxH_^13Gj3UU=(LGl3l^#ETa5I|$k**MTA{UOwG| z#qmk>L0Tp_@MOlqjQiGZN?GFi9QYZc(uelBq{^QJBI_9ei$Px<&dxP5i!w1sx^t8~ z7~3qrejcZTJ%kB!0^MC6b)0X|DVX?-Q3hXPUh`l#%}xRQRdSka)%HF3au{Ghyvq3Djyc^zeACSa)vYyO(5^<=!QMy z#JR6&cIonaIGLYLj7<*H(I!FI^UiqJ(Zadc-%|DuII&Zl+|{%JE?Mi8m>F=Ix+H0XUvB^ zdWOg}>1etkkO+@}L^Sq;lM(rwy&pudw=oWk5ijMp&Ukqmy_IPOJFawP_``OxvnchlfzeYwM?xayOW5Sbl6SA>fviRs zjcotQ?io8$=vzJKu%X^V=T<+sZ1kY#^NBB&1IBu|&_VO!Jdjc#tK)0KHsFXBZ}tbT zm&1wM?{|^tiiB5e3_|@{wfCadJAcmC23(!N1x4tYZn3)LmU8!uya6GaER@H85zJUr zu;caP!O|0Gv8g;QSkjuqYlw>N84zL%Xp>uh$zA)?kHUp-ILNB9?;DAP^&~!(9%*O| zG8`m9k$~Yc3p|7F&N?gJov(fwnchTF8lnQCGvR#M!V~0>41_;@>%;EC%}r{3V31Bl zC!pe>BIEod!5WNMct5CxtyJ$mT){ra{VhD9ZKAJ38Ko^+PgBWAVkYV?&*nPnx`EC9 zs<9>AXP37$PdswkDm~1Y{25LF1n9*TqEX;ahFOxykjnYPVD6!#hRDr1qs`MY#XJhp zLAz-4D~lXdEOL1#O~CPX_-r3rE^UwhfFi96Woo`NR_uzkiifU$DG$}&-R7}hT}x+ZhOdG2 ztOToPIwef$mRvy}_Xe9DT))s>NJ1it;59CUHt{Ic2)=puK-NcQ z^5@f^Q3WkL#fr5L>xx6E6yO97hE=z_^C#jvlipM4VkAZ_>aEO$^1Lj=MH}E?lu+QJ zM}U{|TM*OJYdEILvg}_oC?xZExMxVlfIO+Sd=Lto7Bku@e!9R7RxYtXm*FL7c;M%{ zl=x;!GL0SSKb?w_j)OxUxtg5@Y-&AjT!4(gI)e~-w zPL}l6j~54QZ!g?jysyPo9ey*lu1A{gQ}JyxIKPoTbc^`jZ0&nZnbz+m&Z+&_o(A>S z7R^Iy?Lmh?gB=1RoL(*3OB(NlXhvTNR9!2gk)6bM19wqIYvL`USD& z;|U~w=MaRF)oyfAzP#Xi#RAO(_Hekg&El^{y31I($pO%o`Bf4r-!0sc zxV%XH7rc9Vcxg^GgL2K)vCepI;e}L1b-OFUuKlYs<&(V05>f4gz^%F*^Vx*?Ec6@! z`P}PWQGT@B#CFQ|ryci~VW9!>ti7;b7ub>D2Fs+r?m9)`G;3iM88G;9S**nY8afu> zp4bJ@D_ifqW`g?8I50El*TN zZH=A~miE(43mJ4WTW)&5hs-w5%K<9(-+8%oy5-x)z4F$KD+DR`G5jkc{`Eow` zgAR+;-Z$#vXgk+?Ogr)}Dq1tbIJ-b?gU?^?Pk#FJ4}Ed9j1wYnP8x z8K9sQ8>(~7CxynI#UeSI{&Mt5sC;z=HG2+J(5C

    Q7SsZ!v{Cw5xw)n*Y}C^|Pvt zw<8Vx?&8qD5Wqt0Fp5qG>#WKK-}?7#zh9$v#DA1)3lu_4>+tBKK{Fhx`dkhQZ}D-z z)qoa4?##ps{T;3IK;G%;RtvY1`?CxuKV-tG_r0Qme-9eq+IThdQr-YXBrlaskEA`7 zJclPhkZcDW{-3+Y*_6y>4`#}K#BMA%?d`>t)RT`m)`^E!NT#R)xp{V#K<*G~(3wer zq{*FH)V+r2!C4is_6d}3P%meM1yikQ2~3nb3D8)t=XX_(SGk+}x6jV1VvJk((3VVA zs8Nr2eRo|u5z!Blo?itP8--nG$-ka=Fx(0ARX5B!=YCG#mHP-OX+%7; z#bk+zBw$eww1<{SNQ-HN!*)G51zlaS)cGaTf#i<4#wyx5L|)ox)~gJsQggflq9!v` z%E5@IW#-=_a{gOU4B{KSqo)-#jW8KkYf<7tvxxBOxk+R`=y18BKj6*nmVca<`ktvDnUf&!VB6eDygq=h#`CSIrtW)+(TTQ;_^S1;}-IBFSPcE2Nt2z2vRl zN6OH3uyavFW*;456FVWY+k1!dMdOVBP+*u#QF8sC@A0s(%nh=>YqkA3mh#gA|%$Qf7fs}{nL1QcmL{m-OX?r_mGunN=) zbgCtnUiPGm%F!ee00Sq?3EqQuy*Qm+YV*?0TH_4=O*DO9O#ea|`N(|RQ<*1vE}hy; zN9mrBx$;_-g4oFdK=&$_oK~zfz5sGJK%d7z;HT#M6I{U%Tm$g?!Q^7fCk)&^wV!k_ zeSkLvjhh(US(?PYF^v0q@$j%rudR_n{ImKJ z>X((nk^mm}{(>vvM*AdCDsV1;)KEX!6I`14o`w5_IzsV22uhqXHQ$~4OXO8V6IX6+ zDY^`4TQsO;leD#mmB$(^cRJDqS=sO!AmQSEi03aaa$=X}HY318BiGj&>Lf>w4Hmq5 ziEp$9tg3iuGkaJnwX;NRU(Q9s%Mm~0S~`_&=j=G#Mo{^*(E331?H_-ofT{EJ z?mF$5YW9&CKPr>BO0hwBGe3}u+pK^TP|Iq~q+L&VwYf-dS}rhRlpMq<+u5Xx$AfAd zmlqr(Evy)kj=}x(n{jZp8aBnKxe*4SQq^a|l^w+)QMjDPb&OC;`tLYj`!D4TpAr=a zE-Sz+uG*CsspK_@tOB{#6T~3UT7J(0H-tDyC2|-|@EIh-S-P(K%dyo{rcrNKU8LDf zddDKMeEq19&38u_YL+-+9}_HKqyw*Su8&j1i;JD3Nz`0mON$)jy|v7pcR6)tZ7><} z&JaSW>iaKcSP#nZ<$>$adz-Rn68&mmHaUG_G|#F1MZgK2y};88lCqehim&VBC1C6=pVIcw&~Ql2}2rw6?P`sNTO zA&%Y6lC0Glx>xj?1jMB2U8rJOhWD$2ldho>Ae1_J^R0`%l9CRYSs-#_LO{m{x(x2* z^`|*N+{W`LQeVA<$F%9k%HsA1Gmg&7jSV=-aLHN2D=NZ}sljKj{$UbL)DT@8VQ~oH z8vpET#h=&|O#dzRmTv4Ta5by60_WT(uy`Zg$N)e_#CPsShq$ z4Ki!IB4DK{*j)=brlT|ibqaY3H{XMlj+j=HT;Xr3)6`kJR*ibg4_Y}Ju7z3$h@zV+ zt^7u#({SP8Ra1)<-`7OCmgC`JyU_!d)v^MAc;AAB?aYwp4p9E4Kd6cSe1Y3_)MNv91sDEMYN0?t!t{f+TRZiV#upUfnK+CBt0U+S8u^@m zNq$tA0IGmC^~IqWquM#?X+m>(13!}XMJao26ZLH3@dD`@X7LYtN7M&os;n6Ka6ov&AHmQ| zJJn1WZkQ3Ge3@z;IkYK3-5PWjD&KhX>lDDslDq*k@IDcFyOEnpLten0UsSj+1*0S^ zZ0(Fq;1`9&HrrJwT&0g^7Sy(8-lXt=Cd!Yqk2$AFr@&pVEYrSi%J*(qK%;|Fv0?{^ zsgsfr6b-PF>+gYe;$y#DUF^^JGt**@G?69VebqM+x2k4-fu|pFMQ>4SyJt|QGs?)j z#Bi^-9X#TG?_l%t9^tGzCn|@|!s`vxvsB^Jc3Pa9-8j+GPt`RCzwzx8PnTy<%A84q zUA(>&JYf4RKV^@0)skeFI+c8LsYY)qw1v6pHO&uEQ1Kk3l`YEGV2=>4(<>QOiPEd( z3VwdfE~T6G+n|PqI)#AkvH{yQUw$N)A}36f)ahN_K;C#|hjy&i0Wjb_eZ3uC3YQ(2sJlMg;mQ23lm^4DgL_&+g+RMbt=~*s`Xz=85q-bPthd4G$)zqf;p5voPDt$ z2~=;AK@-Ok8(;AxgCmy$;p8$vyYnuYoXA?cw|=69aDh8?p-toBON$&8y*ynv=Xp2Q zbo*SvfCv<~uQg}&%!8r8AI4YMAf;CzG33*{>A-sZisE!@Ni>c(qvhf>iS~Vw;R~wk zEAaq~181-P7eEG5zz;pcOyi_86X<@2s3-en67gU$DBY&?Xn;-hBn`M9KaeYS3V=hC zZp_rk{PeV%ZJR&TjB0=cD1iZmy;S`RV;!qFD5I0Q$50pjD|d$SMPOpjXQAgJQvCH1 z?r;R{woYFbw&(#J1E!2c!L4`c{JJOTcpU}wF%B|9e@gtbaYJk~(=@pP+=?edV~bab zv@5Si+Ncx587C1qaKR0u-;TCL;!2canU%WnqXOKgf;zl74S4o_%kE6m@o0b!GUmSyo|Zw9|?d9NBxXJ#z&RtY4i#z_|3! zXsI68QxJX}cnB32c4I!)k_3I{)E0tbBceRd{|Hi3Sp#wTAU+hfT))EMo~t6&7ntZX z?l%ht*CTRQWY|{ZUKO!R+A(p@e}ZNl$evW(3bE`W;fdj`c5;2P{=QyJ#IJPm6KBY5 zCMvR4>B^Ow$xzsazsWdBE1e0J?23_&!AG#S%NM)f>z+zR(3D)F@J}E~*X}(u46#8u9TX}E

    ~vEA84+_ zs=zn1;ijrb8>4V-x`{A{FMrTexx+bO?c%QYuwfsh4=1IaHTI^8sOj9jUjd++C zSHUvkFi9^&u_lRHa{b*?s53kyq~tPEmc}CMSTKl#pd0WJUl&$VdYvd2G|Y3&Rh8CA znKs+P;xZk*PfIAy{YtUzzT<}@8uVyiUQZktCwQ3zTLhaZpNYh;&M`sZ=66{NkB{v? zqNEfyCx`O8TRyBGFcAHa^}MkE#l+o8X3e$TiL7^Wdz;nq$e6h0`#68r(5=|>0`*fF zx#OS|VpW^^^b&6SqV8T%JM&mkZ%cps?=6wH2%5A_snpb9ERIPavw1>cY5@J})Nn8h z-)|@CQFRI|tUON3(9;tlX)ML|%tLK(sn2XP2W6Mkzs7>6x0$L{r0{P*VgluWK zzv(H^ywg$6P&>q!Z#x%yj|0O04m_$OBmL|EhC)`CV@c5yD|Pd2#tg>F(4w5Kas_up zP(Hl?@P;>$r^@i+loQj<@d}Cu-Gj}REcDVFj(3Nk05$-3vK9fvuu2oCm-C_*`Y&&1 zfd;f;wEl~6$r&!NGvK?cNM6*Li4c!NR7*SSFVwrc_y=Nh@tl`eON0S(0hL^o!s|a< zV+_(cE7uoz8YB0NDxMP$S=j+fBw1B5m*i{q~@y=D* zj1cR}jb=cNezyiT&6j}LUpG+@807PmX?M=sw`W*`(y9(LzyIqVH;s>82}pKh?59mE2=R}XF&C5>>KiW!zgB~l}n4^X3wS0&J9lj2rPnsVvsp=hJXVIgttns zRBe8y@=0*Euv-d;nJ6DVuvXaa4;3>xTltdi=F~4x9Em}-?39SAW!<^{Op$QAUB#hT z?(!#wZd%^YO267UDyVsX(G*~`IQ9Zu!;W48--&nk2hXGH&O2IsCMW$CEGbu{&eq=C z>A05>VxYrnek|NJwNK){->l|=tCpu?+w1jIKfy&@HL%_)%r!^MEztXUe4usn?7`vZmPU6*24JC)X_fNWXzW@;_sTKP?&54^E_89N}o!t?UUF=NE00eox0n8@1 zWOqJvrAY)Bq(n|Dis_un2(@;{0rbKtwS^aBwJVh-%|skQ4yqV_3IaD&1WdZ=cNDTl z*Z%G`jgbE<`lj)@ujJGvSR(}ff^ST>vbMB*&|u2Pfb!{31^@5QV3-J`pqU~nrJ>im zt2j5Ecf~D%&|t4Dz~<+wA92P>;BRB9$w(17fhgHem$!2P-zA|mXqns2%Z}|CS@W{k z^_HN{Ab0I=J&bRlBls5_92VhPl&vT%o013O?&aSm$8;wpj;S5h=HeaoN;M$d_h!aP z6`X}*a9w1Isxe73i^$uNRFZl6(cR?M%i9Xn8(PLe z<~<7Mrb)g}99<^CFqzi%)SlfwI7R7Y1rgi?3}L=bw8|_H8-7~x-=~hAg4yeY)=x1@r6XnhJcDiPIRMzts+gb@Oq zr9)eD?0kfeCXso2u9#^A7aGu%7ry^RSm<`!8T&`fC_*%$XN(V^n3>AbXBe>YfC^7!5T_uUfOjQ<(M3Fa-}Byx(;$zD*|^IhH8@ zlFfgq&+l2lI_lq-^qrpUafi=<8TV(oK$FL`DK@own+7`Fx?B1>P433)6tNEGwh#pZ z6RP0yYRW>KsB^Ke(153z04gg}{Whg7g=x}*lo>Lz78~Ix+*H+6)ySkX7rWlt*Sv4k zl?Rm_yji{zur|t{({YyVB*r5ERG5^nb1oU}wWETet0{qS)%*=vlEgM62|>DdexV$_eVhAIu+sbH-qJUN<~=ip zX+pn5qjK&KjqOt>@1EjvSqq-q0%8e(tMMBR5tw7ei3cDmG#?I$=nE^AkHhxKcN~ZT ze0BZPDNFnDlTfwmzn>HNV87L*RVIIb>7GtsJvL&BpJtEzM15lW;ZegzXl-5Q_mA_9 z8>2yYa~hH--FmTm>K-S`qJSAW+Jn0#f6?)^vq*ScyF-!?qdQA&A1>~~R*P=6sh(5$ zF^tvHhTpxppg*{@S=~Q7i<}AAl+lyzjOU2~OpJdLW)OxtKRBfi_*Epdo@B*G8N<$6 zr^({zWg<(Zqezy&41YLGa6xMd%$V>+H21X&)PMBFW+#P zi_gzfBs}9)?t}8LW0~J^dBZ1JneHkpbQ18`Sn?L?je5wwrW)<T~g}4r;E{QnUCb31wm4jNq+QK-~ZlWf5fPztZW#{;er>&<*{S zUtz*@(_$t88~x)i#L1-v1vWssvG{O>539dpa*KIS8%BD0vMTk;M%5qmwuEJiK;G z+Lb&)8ad%a7Cg@r&`awd9YDe%h#$}0tEuvq?xF*(W-qAz)9JJW2C298jCl{RaWrxF zG{~h0Hb!IPmSgyGXeH~DHe(tL!}Q10g@H*CJFt%0(FIuF&cv#YJs(qtwft;(jO{nX)R@M72Z{Rvwb?FzE_^9tPPyNb7x|u=HPWWl z@)e+W&;WQnq9AL_bs^gCf8I>LQlr=P{Gr7o<8S{` zO$}DjKYAoclf|ktu;5zT0BO#ts&4}$d21(?CZhendD$yi2TYW2<^IRi$Jpg)JL&D$ zrBTeP*9ztAmjAs4cl}$q!o`{$QK}_V$dl5m7Z)%;cqA#km1<6K$cG0e@hqF&^F z^MjE3=nm5_fq4tov0#D1J^(vKRrHD;uFt`5iVj^>W08yt-9AO_UfEeu%}sw0h<>`& zHeCACUebkYT!2$WxJ!H6v6PN}Bkhx8oaK07wK~&LK5m(vc8**-Sl_?0uZ1Tv3culQ^@L!9aLD=#`q-D$Q}Z$kzwOVMHEf5V z^7AZ8`aSG;CCmRwH(m%t&4J9dy{xf5)26MIAIZ?zJ)aRhaDsMjJx~=R*@AJh;^K`O zbH`*Ood2FiJed7yUEh$z6wT(TCSd7%n1pM?25RnynO z81Jat+{22rZ~SLBn439#;d+~c-CMiN8YJ?4+U@O-@2<7=CD%H#Ess=jTV7whmcVW+ zd^eBXjq9MIh;+h@L`Sb{Z=Gc1zw>q+v1G6Ec?O*eKfL;fCwoXX>VgHwlNSjqN*8ZD zW9i~Gx>6B#m(xeWvW&}9`ANe?iPZPGzmM1G;zGKob*Y4f`!1P*nkpN+P>jmP0+zO| zx6zrdN!ZQUU8-V%pkZ(qj~;|t78ztj*_iSg;76>{yQ)~^xWYP)P!|V`E&b6%@@sblDet3U@3I9!qLro||xK)1{x@5%pT>bnD>{{Q&j4(Du=y;YQXQubCftj19?PCi*> zg~+-yLS~zim8_FdlE3>A#gLZfe^U84jRaQ<(!vSRsHc2X!2J@;LT%$3SdIEimv)r{$ zJWbwSwz3%*OD#-h<%0nz2joK=-ZVzfjYSr7P(s$H2QZeUgSpBKzFJf%kYrLhB=*WyTWxwbIxCtIjK)IGpO{}xuQ{~?=QlR~tgT#Ck&|JO z4`q}DU~T}YZZIj0wAaS!(SmR8-72Tmc0@$XtYe*6W*=qJ z>6Hiq51I))LLvU0Iq5m~)E6ZDQ_5-UT0-Ob$u6XShTf{}`i*w>J?yyk*~nLY!?32O zBIputv(Jawnk5~Gi^YVLlO1$LaPVWLfemWEnfL}5w118YgP`*OJJtO16Q$Os7_gAu zto-!3Sl;qmxlfB$g3)6go;4U|!mWYHD_O=uoypql5t6#LsOr%y^^~gFEm0YWD=>&=p3;;BY#$kYNZygW4d-j2BWlZ-4yv%6QoJTO4&f(IYRn$a|};JfG9Ocv-4M zs@%YVGoX6lI*|?gvp8?+D$+iy+nCtY&CZh2_{f;_W}xRXV)SKkE(0V0vi*He^ckAS z4Rd%mZlvI-L$=@HViWmtxP^@D^P((>KUB3-(C8|S59<5$u%iYUY;VSbH_2r26myG9 z$ybJvdJi*kB}cUXUskTwFH3C!fb3Numc;-9W9Ffj6F8!W3fvd`4i6yJJK=Dc_2R0x zaXjTtF!Zxmc`go8qxd`%;;lJVvAXv9nT6Es+Ze$5ff+DJi#1JQNpq#;Ulnkz- zxEr6S=eVfo+oKG&Jcox&nB5GhZ>W1)8n}h^Yu`-`Jedh8;xFRYF1$=jIebt;dOevy ziQB*37_q66q@%Z*>wmNJ_3aV8JSIZ!8c(pt?9ZmM%<8?T18={rG_gQvA@J31L}c!V zx@a3W3}^tkdeeh3Q`?Ejw%S%>hPjPiSKKY;@LIEiL=zV^v1C%p~O)kTV3>l9T~ggiIr2o*)lEMn#^8`PQK(RKR@5v7gQ<=LSmqfDaXGc zDt7UI2f!t@y0IE35v=b;fMj;}esjN0W@J(xT&$m*6XMP)!CmvV65SiNDJ_7d8zpZ% zXdCUCP#Yl)9R-=9CE?yYnL~*gv^Fb0aI|6sSYUbX+dEKpvzkn_RmE?fzLXSoO?e-eJMK$I*7{J@VXA z!@@VF^fT2V=;-}pISkrC$CR8WxHxr9$)YIGaiqQX@NVPptg3g0`Sh8dWl zjc0cRUlO|)hhMb1)7I*jjD?ZlD@=n>pg9P7!!gn;%Raod9fO_K)dIx%_^0)rZFAv^zY{6->SN z$$$Dn+3Adenso>{qjkK?rYLXRiq&wxV7=lJae3=g&EQ}WN5nk-EZAL+>Hh7t@}nrm zxwbA3RyA8U?C+&i)PPEak{991q|v?D6?nt^EmZjAh=)r{#9}xqa3anlPgG9ftq4mF zGfP9<2`;SMt`D>98&^bK$-B+D;&dI~enyQ|k+?X=bE2a~r}0G!z%r8JhOq4**=607 zAIW||`f9@<#cRdEcH7%V=oBCAV$3Cvp9HZMBH#<%-LV-`|4Z(ydRhYng?s+_5_j5e zBeA_sVs;(~(Ya(0w{&0LX8EylF&TXfo+5KNkwVocHw5h*=f3b>fBwOrbEt62`Nf6u zwCwXvi^L3RU#`0vBTCif`;w4#O4rRm-A{K%^cK})5Ron+b6s%AP|lozvsdSm5#zz! z&pVI{Prw&=FlC}Cfq0;R^Y`OpIgw}6=MVPge!I!?GCEbwF#EdPrPCm7rvMy`ptSD? zu-{GyH|&xgd)zn|iz)CYl6J}%c4`mMV20f0d;kyy7mViw0aJJe5LHDxHvqx5gC&ls~%l}eUxH2nMcEiQ7r$I#s$^q^$K}z}ck+nNoN|B%KZlecV zp>iG}tcw?WB&9JHtsqTaJ;xfAw>OvCC&&N|ody7Wrr*_jG6Ul4;lFe|Zd|Vu>K_;D z^-`_5sqGG3iGLeSBDdi~!lfF@86d?c4?VF)AM1tVFb0Pp|MztTr^q9LTN&?#MnO8T zd2hYJbDIJ)e)6YR*J5aXt;L`WSG1T4GOaHcs2~Z<;xf-cWFKyTkN|PmNJ;3ZlkAuS& zeM!8Iqa{Mf9yG7%YD;ax?9zt~qAD%kSvoOjOmdZR|J3nI!yv5*Vf~=G=}_qtH(qYG zuJ6Txn29`<%L6Rg=1vM{K-qO9SSbhwB^hgKVhX<>m?JK{3^6|wLYeZ&NBT6_pqv^s zys835^gQxSREtD^|E3hGEP=x4yYu&_TEnpSibB0K1CoC()YkX0H3F*=S2uNNQQu_! zeK+k2Rq`GWuoKmS7IwF7U-LkDNDFdUTmHr$TxL@p^ZM%&WT`W1QR;uPH3j(*Upf<^ zxt`Ttm=&R2`gG+V$ z!Q6L4;jk@tgYJ8Mu63`Mmw@-~O6*X{aapvA!KN(KuR=-VT^$n6%eY zwF;t1yT~9CUG*5+dQHKEj6rqCW+?apDd;haUwhvTr)8jVA#^*Sciw#|q%nHt>=1k7 z^H6BC4eyhLv29%mb(=#*^3)6#pVeTM!+wwzfVd@SuH0GI@Cw<^fdhl{zj}B93D$mz z(}%$Js%fqr*}Z4k203{z2}I|*+>7PhA9sk}<~8l-ymG#}e7N>m*VOnbH^)D!6F$My zk_G(mfzwV$%I;_GYK=34cot~VjJ@z2E`l!CXDojk_1$E6H{<+H^@%MFF}Abk!!&#c z83Lz#^ZH!`kD7I@kw5s8AIy%0&-OV+6PK1dKov~3xoFa4dXwWg#$jC$Mc*EvPSKZF zn@;siK=!j=+Hlln+j`;75!sf?@35*ZjShTuIcBX_WH*ePlWeAmOiLg+l-cNM6bod1 z^@CNkN=^7|;c0PIE$%7vj~mxNQeOm*=*_d6S+j!aCV}aGo2Eb|Oz@(J^*tPQve6^Y zBp*Muk_;?{YJ&&ro(av}JNO6cYy(n7Z}_k{__zrQSy4Q*9RynSIL!9^m~AeCbM9|i zp#}`7{UMDh88#G`?5#V)=z|V=LkIU7m8lQuOe_qT_~67$71Nc9CY;E5?)Fzy%JZOt zba$WDa`sV-b;&w_Y)jFFeJ|^+Z}w`BR%3sHrTs%`VwN zVlf1W@Dbnw3BKAVXJ(U8eB@Tk^PJ%tZR_`lyY|CH&R4wbK8+!d`fjf_91E~Tghjo%sb`QG8Xtk{}AShBFTa#mF7m#|aaf)j9azhrP@%#~Jy4P^p-EC8HrGo^WeB zNMl3oYP}0+!>9L0}1}shQ0dV@lF*Zd*r)Hn@Nz(5$$r9X3W4b!m(LINDi*+8DTgJp` zTaYbq7x`zn8n+OnKdQLoeMvfvm<%%4ogBw_Lo``ki%C%M;N9BI+S29sj*xD+#ZHpO z`eQozZqv#4iY++n&xu|^8>LNE#Rt7X)q;{`ILWY+USec+Ooevl=nHk%oDQBEpKzwb z7Oe|EF6Ddw89$$1C^9?fF#AIlM(PH=We0!MwEqZPD8Pl#eZGWmueoa%jcR8li*UC1 zEOonJvOdMmMd`~pgH=iIQnRM$J}5i_SE+_jZ+q6c%&8aNZp zj~P%SP~ofNRo_~GGdfo0WC@IwY5E=>fsxV^E*$CJMMPfBk!^N$@Mp&@(2}0<^uE`1 zd;xEn(esvZlth6I) z>kw`zRq#ap2^;hTq~=?Tp$Kb-Ay=am;VUC<#S=bPzM$86J1k3cHsi4Y{ZhgCXHS#A z8TraGMb6{=^u>=u<3U_H!QBg!_WkSB$z9vB@SS&-kl661Mp+3@>W&VC{kuB>fVmHG z^B^Lh*B*j3G>-3m1|Gm_l`3B##^$_ejQ$^?VG;KL{`fL6wy2#f{geHy(e=9;__I>5 zK0lIu-J-qvKal;R?!ds^!uj`Y+hz7Un>WTF&8pu%yDgG$yyv|wM2&bjpeLX&hV^R% z#>ejDC8Hj*hW!)DwJ@~nTt;>YE-mYfsQ8N=(#^5Rj(jd-{FX0jk5uB3r+6fW;I}JA z)i<6ss<37%m>`^V{RGu)^!G^aw=aB{Q}W1Tw164R*4_R+l4=1X{f`?&LPkPM79v5g zIX;Nd$-W#09RxQlS}Otqx`?>D&U{})ZERCsC#SGt%2g3#OS30)WTHqpt)z;r1hAlbNaeVMt8Fj-gOwWEdHMy2*H9&w2wZO?~yU46-peiLb)Cm0iFg?(0M}8^ zwZFuT-OH6U6Nec!9pN(7{lYq@Octcy2E0UnGwFp~G) zkDaPF6y(ejtF*QZfKfuHYO@WBSK_wVTiYY5qh$((ggmDJm_548@iv?za8@DiXt*<=_ zf!XoD@wJ|kb=S>K*tAq?YkzA=sw1LLO7o@|0<(UPjv-Ys9GoD$cYOc_Z|L3H3FV5z z>*-(l9_ZxC0g8-0{gcF&`skw0@616c;lmAN#{I+v;!t^uldKH9g(`$}xV|;=#Aw{r zGPCw=TaPax+a;5{{uaMlg4)PiDJ-Q`ar!O93=Mkdc*!DfCifii`EE9@K>2#-WQsrC zK8XZ(k9*t@tTbSNrmPt_kgRStU1~aO4+|qx&Q};Iv;F8WoV;k#rEYMZ!MD0;Cm$E` z6<_a&S)JH;m$HMmMwLf+HPXsy6*$KT@5UyxF;d=Qke9^w0{NPOgWpmdga#vTS`zsQ zZ5Dekm6W6cmAz-TS!@RP0BrTbbMK8xJ{&>c*hP-L(~)0hI!BP~Um0%LBktQ@@jOiAPN2 zUUL+SP<#I1;Ma`F2x%?k-kuNx8MMZ_Vh1&R1)>M9v%i#@ukw;d&0W!<&n16zm2OO< zHGw_zI=aZ9_abN{@k#a5QJMFsc(e$iKd&r976}K(Pwzi)ET!`-Xu@4MD?{aQY$Ae385j%_?E_iik-_K8*qtJ6d9}d1~f|0^! z@4sp#nj`F+Sb9(cO4WtQ44elb7GT8pf{ItdFR;r@acevpI~wJ-&hExvr`A=`ic%SI zaB1J8`UuO8kg*m21CiY3XUzsP^kJmWark=O|13@GB-NCRW@C~R0uI%I6w&m2Zr~+mqRQ)r%XpcpHFLErc{OCQA&v`$}9j->N^$WB)8S zCEthB6N z_{3kND*Qjdy8Y9Px=?q7S(t#REz^FX;GhBpWe2c+{4B- zWgtiUt!rryYMiRa`c?v!Z`MibY&U`GUFbKkLu4)Z$qn{)?-851EFE0>-8XtS|C8$N zXP6vg4i)lxUH(RK8oPvA*$)_AXl~!m@1_JgocXmzxX5{s&3nhZ-OYE6&vlA^L;r!16WXL-j{0 z;~6%l*U1?UP207&koo&n-|X{{HRbcUR=c8H;a0|I0Us<>^SM@9Cm@A~c2RHN5Y|EQvU99fdzt&;jko)rqinAaq`aLbR#bkmhB^;$@=(1!LmY<$Y0GR8t>a{k zt)!lwhuq2=%fBhM11$S#IPJY>>-V_cZ;J&{%dQ(CA!|_jj27^=JdYK>UiY(n*KKiw z9b<#SP`gl87>V_5K5HOCPfcu@H6`$o$kYB+uM ztzh+)Jp4KurM3(+0Gm7@?l-Ez8d-LaATHh!YD^M^9m*D;xkb8}MoxQWAF?<6Mroh8 zy>a~I-fGpm;~zVf93I1tuudkk2DgI#=%HH_0y?ZY7hb}v}2 zyoL;bRNwj zFd5Jj0G=)m^3*;iiEp{WOP<#WWak@2kxYnbJre#%^7}Wb2{e#?$&L}eOw2A+PW`4% zyy=t83nZH_z^>4Dz_gaKZ!mim=oJjI+c$1jxB~h*v(UUV;Kr=jRYZ+Cg|1xsj^1lmZum@lB#ph z)ZBtNSj7p#KN;G^bbJ+8x6cl6^4{rLZ(aspfz~E14mk7-#L1A?z!sVY%x~bRbmeAs zJ!gBM95o;c{&oy9n-EL<*=aAIrbAI*s zQyPmomK>zReY>|>1%!c!p{ibD#i>*8=Xf;LPM&}i>0A86oLP_CuUKG4O*))A1w#hb zHbCDP-Iw0`pa%9qKF+|`o_v+1>XS^~!s^YhH^Rbi9@uj22oum@2eod;pZs6nX`+SA1*+ zI`9vhprwsn#zse?%$B(J?pVIxR_KE4gi7mu7bOWz%^!j*zh~9Ps$S6@=cl=C(UTHL zIJOq}IGrzo_B!JzkQzgH9DarNx;QKMd?E2&1V+8$n*9hp95ZLi6*dCLI}F+EjcD$2 z?ynkHDj**qB8i*hRYsw!UyiE9G7w%qhl!v!Wi;0tde$B6-uEq{j&{}d^k<#>@X*Z3 zanCFe#xeP&E%WJ_5t#fXL$y#n-TwX?L)Sj`_cYgIgwMAI4f~)v>#^tO)?NsRKr^N? z5HeIXu);08K*_uP9{!*V{JrG|(1A+m0UxaeJt-cuTKm<@q{JFgV|i@Cr;v*1$tlb5 z(W414o}g}sU+VK8j-x^Ls6{BAOd zGDN1#EfB*p@01oC^iv!^2ri4VzkDLKyQ&vL$^ix3#v@^Q4KzKId-wZ1{D&=hhg8eQ z_!-BQBP6+RY8T18T}sJ+x*WhuyTiL1Ygrf(`)HJwTV<>v!z94ONazPzO2CAlT-C<8 zHNAm*kR<(!tw_IxD7H0zJCq7TYK7|w!Ca`-5ywjdriRGXB_xf212o6*E9`9RxrHe@ z`t=P3T?_s|G+~c{?NUtm^35ZWqSXpfIqq;sv?|(je#i>99gY%?zexmPY@CmRbWqFVAl2Wn^IybV4?fIdO?)B4dFQi z#@SK1oXTGwuHZ$81unRa?Bw(8vkFu5+Us3+?%|VbW7U|sb@35xXRbL|G7?WDrQBQq zg^X1fUhPJZ`fRHq>>v7S<;IVEJ<-Tp-WJBn0`g8QGE9!Z-#b${;nTUm%N2Lfb{zWIQRhZ}oth4W*@H!}TFhIz1B?S4Z+9S&<%!ZciT;FsVa1RN-2WM_^4bL_Cf=;7Kgh>-*3Of@6iIWVQ>D*zFQbM zie=@NdH0I5B$R#t*yOL;PVvMkgdJ_0;jE%4UTCfDxl9+L$tgloaMIBcSmeW`Ls72DX+ovHf73ZtS^rQ^QwdJFAU#z9bR?hxL;EnL$A(S4WI5|{ z?KDzH>_S(7cytRCAP;-s(8_a&xtXQxxmOgi*JG=GI#hQ(99)W!s2={SOkd7ofB9#) z{?tiR$J5M^32?6!9hnHKds&oE&wQ>i`;MMsC;#k4kwrP!7<&_geU*mK9%{_`r{jHP zdIHgRGE`T4H!#_)Sh>*=m5s3)cF%d-)46G@u(S^11eW)G&Wify$qPYLko1YZ@w!=E zL|Yh|ZSF&0{o_?Q92G%a?s26O6i+jcF!X~$TXJL>e0`dI{!9HREBmpX6@&LBs2N;B z*-jb_$JPa}g!fSx@c%{p+r@z8ngdo3+1%bzU{W6WoH1q3EY|0KH*M|J`@4lh%74L8 zVNe&@T;P{`|uj&#ck6 zEzDhqc!O&WHdI$UnZ?)%G8Gv75N^V!7ehEHS&#s2#`*@vCkz+vfA^d4L6)#A*-S}4 zh)&TGJi!_%c(qM(FeYUs6hb>U9Oc3CO`dh{fm@g{;XxUL#wR${RSq8iDo~s@zE?vQ z#KJx*%I<*nw#4~fddOz8UfDi;XAjiF>Tk~Fy>B6IjRX3u?623&^R?*O-?ybHTD-(h z){%7KkQeBaLt;X`o*q6)7T!_ZwB_LV2;sstL@neQUjTyz1+peUu;-k$+^^&&OP0Nt z|FZ;7ZutIAs;EvDjUw)bw^0ycKTb3I{+MZgP_Vaq+J?brVM|&Hx&m@@Ai0mfc-(%J z9CRgG_;4-f8TZ#?-SQTpt^S2I@fDr=wOm{sMFT|Yat;TwY*SDigT1oQ@=@%(;`&(C zmM_9Cey}9Q#qFoZ+V;&gPr>XpeqjCg9|0`Ljw0O9J_vQ}8{9@dc^4oI83dQ%yleHm zAj8iuILOsYJN=wk9fbWbyeVTd7H@=P>d^W?9Gg)aT3-qK&ogQN-d0lbO!K5V)DP@e zbRr@g_3+tjFWOxK8hZm1n6$F9WgvIpFQ5vEkJO=@01Jf%yBLBV3n4nD$%xE8=B}nd zIBQ{$>P4mMuU$LQ=nCpqz#$%zE{IAYk=F;5&RJ91)!{}er7yxY?m93LK%S>;$>*Gi z)*wdw`YBZSn!krjT&&So)m)#_ilE0lh&sx@K=(^^JNLEPTf;=kPofDxIx>KHu=KFx z*!7TnM2xU)rs(iv6hR&<*M;SQ5(ob=#5XId(<5d0y~wJ!r;~4b3Zj~F{kBBy*K$K$ zKqkl8Y@!<;WS>GHy9)}v#r2ByK!+{GRDDIGdGD7n2MoRXg6mgxI#*q!3%C!+dF5=` zb}vY6C-Fjphb7|Ge{mAtfn%g;$nUGQz=dtsPjni|VT(OJAk1+b!9-X}mV{MvM(D7H zsSSn+nU3&mrT4gx;rs69@2yv!D- zv80eCIJ_T10c5T6Iysh!z{tzQSRn?40Z<2(i6wVj2f+QT9wJajxk07~M0nzGuC(}$ zv~LEfdaXC;aW6}(|D5>5PSxHUHmn(si|DuOZy2E&aP_UB#a(je<=!a|YSo`r6ewk$ zj-N;jE_b=KdM>Gz8_5NZ=1Ypy?q_y1y~xSA)7mlXUX!6u6t(2&Q0h`K1>K3J1Hw?+$P4P_EOzSWLBQE)#zrU6i4yC z%z!N}8Po__`i6h_W8v%VIlW$)KuI)rs zAIjU6uukA_$Z@-28qZ6UFjCbGst9E2LM^eUbKJg=;`rIPu63M)3x zQ^$7S6!d;1*pCjGc^;r`N>H1A17i#1p)#hw{P2{3!`7bT;UH+zUwu+?&O%Q2rHi-N z8m+FzQw#dA0=^iWw|%)MoCPl#_71nab!HfnJr9|?#WaHqFNlM{`|5zENbB6Bt7;Uf ziwBf(Qzb&zzg2x0k?PZN`yT*Gc+HBPzlRjUMgVqpKW6)+y}yMQ7FvC4|N8K6gwM$E zsM}qtH;B*^PGus_N$eP4^h?KzC}fJKRHatKD-3g`G*n<4g@&lkRT9`|ld?zPwK8V{ zw1nH=I(s#}O$vV&{ZI5`h`@9b@lOaR=1}}ywWRwIC~v~8Up?Ed(cK@Thie%OgFw!Q zZw%sq3syn?+#x1)@c_JXsoicvD&R!}xgjUWFml*tV*)S@ELdO zFqZ>)9nqH#H55$tTPzAhu8d(8?ds(X$cP8QLQK$6kkEwX`iQ;4)O6v(3&u7Vh08yc zBg*`?yPbDbrG2Z0+%lT=AVSNjxt&iMKfF}JWg*by0kA{B;iylDNdM`T8SSxz?7%9Q z5*J_s!$PB$lTvpr2W2O96w{yD$u-7Q++R;qJvO%i1lws3Eu(xZgk zcDbgSy63cCyCCi%nts1<|DxSb>2b;N{#*mKs%?K^aPRxrLSgo;KNa7cDSGR^8%ulD z1*PHCnu4*)x`J@RNlr8jY<2Hj-BRDOQ6Adnp@_1Kv{jdg?z?P05EiyOy{}zbMgAsu zry)bl;3cB>A>umaB)EMBFz1{#)BW040seN*6W8%6!o8h4OPvEs5cXT+&f}5RC)&S% z)PI+}eNk62ZKoK5pfzBY<+$)3#la*@%U~(l{%@-Hv^rK&!$E)`$QUH3plN`T>r(I^ zGS;1R)v@z9nz&0FSC{r(l8nxU2p3P`JDz|Je1tIi$BnXVG~qv#%j`zJTwb`J?Gy7` zi^*F)!e>d%NDrzw_zjal!+}C`2pusx5A`1EW8*#qM@~J;VjNYE4CTPKuhds~Zujsa z7$N^4*^^*jwn1pT_c5^O7(KTg2u4Li*$CeeOSg0d>=b1ds zXyY6+UU}1zmoKgq4W(|?nkH80m1{i3SONEzqQM9$p5}N6qUS9RA$s72CHmssE?Dq7 zzw)@ZlNL=BtI^<$TzfL6N-j=>pk7Yy9}MDjcm?ETc$)?>%w3I1@T>jo{7;|tBiG4U z(dZMg@0GC$~7<{%r zSHU1(vAkM{L#m*$zRI&eK*BTz|83GuB8{q!BCPJbnnnsi*s0S*APtwuz2`lcArc6z zKyaR1pUaPD=5y3mOct$TnzTN0T9}Z8(vygMJGUz9d3HJ@7$MCRAo(DAXGm=1FVDF2 zXPUMQ)y3xd&x^+ODjmHPS1P+0z6`l-ib8}{%gsta>pCFZ(T26K;L4~I1;+{`dHdV* z^{pmGWowed1-$%yzk@~2fl-IZbNl|U8pFKy9Mia;yur{!-mrVM~ z5fOQR>XL`t?y{|Pg4ZCS_V(z8!s~bB)Q&Siae%Lcu8h9%ws{beeG5N1n~9n6`{|>0 zg=fMKtcJi1CS@4(Dm(uGeFfl}4s2u_!6xQDuY7x5VcL$M!Bow!*{>0O*>@$9lD_=j z4RsNg_RVq>0kmQT8Qb6&jBmsgr&_}w`HJ$HORi-&{rH>YA*b3KT36suj?OpY@eww# zRpZ5Lvrc}t3}t78&gQOQYrG zhSjcvlpKaxzPccy33qw-F}iR_n(b|*c1R-CZ|Fcv6Rwf9s*nk4Z`>bKP5qdNdGSHk za*JK;@UGW#_rj{2!Non#;V73b+=>z7vL1GOyK%OuqOPFv*$3<+u-qY|t*@g(*ohZc zIX<#sB0<^dZ`R29XCP~LG!$Ap!W8QPHmTwsISqWIHCa}VM`4$l>i|t+1&mFssjgG; zYJM9-I6pDYm~J4Aq{sOQtSwG6?GQ)3#ce5@zZ3c0>1AN6r@&NTVoa*_Utv&;;}QBo z_W`5-9oD%|B;G;1uf++iGT}2DLkKWO=-Pt0PV>S{YJtc({O^Dp7A{ghR5`_Dc7G@H zq^F!FPTj|3si!mSoj}*d5Ugdv@lp8q$*!m3Fr9)(lSy6a-QQ2av*DYc_fZI4cB%(t zj6O<7{#Rr0oyiPB!wPClcgz{%G_5#H5#}8+1Q48Kf4~}jbzft&wk%VY{Hzleq=#5@0q6ZXE9SW^F38xz&|?~_T(Zu@o{9bnAEOj>Op_P_5*WIKNyOhV*WF(JH|B+; z5@~Wh$Z5Ai*lU9a>%fD`bmZW_2YrVA_u$LyMk|DQVOH3mwShi^GolASZVTodlSU>1 z`=&zZ-x;(D%l8%d;jzH&mA0w+KYdzPSP|cscP-R0xS?5tpIlDeD;?$Otk7)BgOj& z(%pQQHd1!>`J|DVys*?}uqIFhBm_=4`q{r5`NiKJv6K$y^2o%eXsl}k-X#8TSX;R5b%JODdjCV0zCgZ(d0tGEciHuz}cl-|~X zN(%AChjw4r0p+scF57CM*6IFNBfV9NzIO&DGx0ue?8f6(-TLox`x-DJf&M;U;zE`C zF7WmErs}{R5i^|I;4h$1k3`^FY##oj0To29+r`jqx5`q|#M z?97y=&4JH59@!eh;09pzj)Ug(obL4ans2pLo?5${EyRhBJ=$?i7WLd26FyDO$x;R) zP=)O74fAhimn5Oq_?~M`DzLvnYAmZ(qi05Qt`W}pG~#7F*W2;K*q&%HUZz?3@c+vOOz476)z&iovfc&x(zT z{(_(h6>#!fD<)}2?ZP&7e{nzopumIh@t5dSO-!wpgd!VXnD%Er4KA~u}Cgo4pkJYh7PO0%pSFjv| z{)Ph0uZ(WZNNK`Tg&bp=-b%i>bL=|4O^W2okAym(lLft>L_13(&+}6B0Ire=9AyaH z7r}^hUhVgfNC>%hnxQqGN7Hs+y%?6_{mFIfCW#vh(ym*)d1151yf)^a*Vi~L+|*QO zX>LJ7pkk2lYdT{jIKd99nIcAQ*B|1D=Dr^;$Plqhy^`vEmV^47O9ZRMOEcEvg_${7 zBRUTRoD^SIjzU9MK#?G%#`}~pFxm|C`5ggx-zP@CyYh2ejRnzgv~mo-*F1(G0srUN zq=b8s)X*#&^bCY72^=3ey&70FTTE795>{Lo|B}UDIdRRbpi}!6`|s&vxz4Oqu1FKz zHyN$ye~XPePXXi$p?>8=N*7t9AQ~K%YfF{(%x{2STMS|>et&kb8;e&Hq>V|6EX@V> zELbfNen9f^LbDmJ8gHjQGx&1+IIVH$$6~ybXf%(Orp;$g!ev%WuIM+(bw~8G!^>`Q zVz}hzX|LMCg&?E`*htsI#2R2o6J7azGkG*O9VE!{8$)r@@I44E50=%4D2gE{o*p$x zRy}laG9XW>Ir{S%EPeGH=FC_ypQ+ zPyl#;ab01}BnlZMx02ZY$RMbe0lGvt-eP68lI5NrLa2ui)W~dkY4Zmx;r}JicTO>y z{`Md)+iIl3{G081o!~R6<@+E#Ud@>tyxYm(K0~i_5E5odPd311ZS}KfavT0X!C4?f zJgSiyo9+#WS1nXj()at*gb!2u{<}1zjU5SUKCqJbJQ8or%M}%4%meo+8Ent|I0Yk= zGs)Hez^CH`_jcwKEOX18mzr2Mo;H-9GwAskUB8e-?Ypxtdg#^7lw*JK^EY$x4k~d3 zUfnujajRv|U-K^)V+j)@dg6WX&S#*p(%1p0uW{a?keM)bzxNo3ZmV@AorZM9kF1o` z6lRVr`u&+zUj_jVH)mwM87GeqcjVo-BPy$anRfegl!Vx$DW%P6Cf7kG66x!-Xib51|h&xar z@(>}lxEsPAIRNW3U*FykXMm*WVzgF>NxR2ub$|6>-@M=CQiX`z*a4-rz@UKG z5A?MRJP%2y{mRBD`rlB82oZ(d;b#%%jNt4-iXcpnehxH{;;G9F8EkbNwSOqzai?uh z9p#3^z;~Y`BBvOvA?S+4-=P={zfJxr6hYhbQrjawIsA>(r}L!GOd45pV_{GQu(2S4 z!^XrS*rsL0?O)+Akd(obekXbiCJEatHf{LSbqwg$1ugIbBABy? zcSYxb$3QtS_Fh794s*)XS84Km$+b;Z$T|&tLxzdV7!7JFK(0<+->Jl2 z4O<3A0M#-T)CTjMcmX^e?Dp~RYy{)!$n2VfBD=y)WlY_iP>zH&@;AuRfYRws89RTp z-Ua5Dg^@1yTIxJ%@dfe>xc6=ZFeSc{{sr^~~ zQI+zP#aSJLvRs9~#*hCmtJkI4lj*caE-zl~EfhwShU~iawL6YU_Wb`qNCq=?7JqGT z{W0;J=vWak`*)!Mr$gQC6*Jr#KFn?S)itWelk%TJ$%`U4u63s$;;>>yLJh~V$UPI~ zU;0+32*{^kYMSeomA zY-K^)-xtFoznw;iIXuG-0r;zaE8%^2+(^c z?T|64uS9@R)!Rub^LwpaykiI)R$J_?K&25Z zRns#{ERUIh167BVj}*4V|NRd3wn+eUM2n>B4R?Ol%hkuM!eb07oC3ccz%+0dk+Mvq1 zgO~Qq5{&O>V1WVaporQ%aWeD5HcxYZ_7F!p@y2juyKf+0t% zzB^w1cPBny?j<<45R3Z&DMzjPvdDs8uCI$gBKMm|D@z2P7UDwNt4rB5YdrF6kN>;q zOY5RwO21?9{<;57Nozvo32?xmyERPg@IkX|6#wsr&7X1K}R*>1@EZ9JH^c zFifl{vPM1!#1fO;y>}b%e9lu{z$54G5vE2?@(1|#eev#N!#o}V+50vK4!*-oa8O|C z8%RsrZ>(wq8UOTGm%yv^5KYc|p-+u9j`XGiln+1Og5`Z#fu*4JH_vMJkJa8C__H!n_4_8| zdohC~Tc#&whHe+r(x^!EmjH4Xn9!PbF)^#kBr)1d??b;+=#1zQS%*yal4mTa>yoBe z#a-qc75JY>ugOu;E_a?BBf{!(pjm}V zFQ{vFr)!*HI;YOQdUSc(UHC;n;2)`LOoXeK0~eJUdIcG{E394~dMO7%?%naSARZ5Q z?=G_~XFnip#x{u=>&_q_00eU zN}vc4fFI{9m||;Hn-I4%nEue8ig0;g0Yr27MP`nE{-=a);eT;d?D#2A7);<5V~gxa zYb9!<2v?WtyX3+9j}2MeWx9nyp%YZ|5O7QgfAn!)8cssVBq01YrO>iSbVsEiitDa5 zZ4$@2&M=nu5zc5OhE(NhbN!tg<@M7EbD$I5T4w!x9P z=t#Gy#6>0uO>dgm_|~PrVF)iwwSFJen=UA)mWMVSC0LpH+|b`tJ}CNgt_xR` zFmOKdPR?^>ip8W7xuI|%=q%dyV*ObF77q3dZLNv`<$IF{Zi!6zfoNM`)aXzUP(=gy zjA#$A#U}ALt-QCi3>~^b4bJ%hdJ@eODF#B0jA28AC$IiYag4B8K=i9NTKtbwAVkG; zj?4B{`P)V3{9tI|*8{^{YM9rk>&VJawC8RL)BvSZ~Y$uc1pRt}gD> z^r@xlE{`D@#Ea<#YMF_tE|61e~{RC+?N)JeL9hgMFqgc0U$AMt1~dFI!a zV;hjVU;~?_Kp={#>n`Bj1UM5aIBZM;MWY?h4iObh8KEKoWI}E6y-2u&kQF^^!t1-m zm+0fGp)S3OEn#Dd;00OgJ3prbpv7mwmrsYGgNL%JQs;A`3zs|ijlA72J_VVqiwQw z#FP?#`^-}Pkk9I(i?yOH4=5yt(jU(eXA(&65cC9~1JLP2GV+By>GrAg492nW>)F#h z``5#q^Sf6-r|4qPA^v|)$u0QvSMbz`1>&$Sn6^-?iTSiR2z`OzQwD!C6?8Op0V>0A z^$+2Gan9FAtz+STni8GrwSYYgDH^Ru=2-!1KLF%FvTA8wX4chvyK)`RN(PP%&YoA1 z0}cs+ivb{6Y3?To-me#{frEW5p#Bw1Mok>FUpfyoA~XlOJwdXET3X$4*slkq$+AS!Uxv7h4;z9D!H-8x}?T*XK`8fti^9Nm@eJg=s&)$br z3%FAXrieLgV&)l34|EH+qkB2(j z-WjH*2@xi1DU`}YO;HWi?vNj0NR9EMhrZd^&<|}k61!hkQMTH*p{d2rmJmJ7E?N0G zW!qk_A3KqyEKweXnOT z?J00gLr4#JZ8eUb_HkJ`WmP+unt%PkjxPk_ViT(R$6g(~YWOCjZUbGRyfjeJe;xY} z8`(!_2Cs+~4VIG-UPN(qWo<%>CoeH)GrWPv4oBgo2^-#Z{?Fz4iTmtc-x`kVwdbaF zbYmOU&EmIBW_#x-Kk?oY9e%p|VuITs2o?b|tiuc)e(zFUyouc@LA|U&lrgvQn2n_6 z1}G~)jw^tF(@#J9PB7<00z@CY5{2?BbtCUnh`g4^1@R(E)8+2jplTrQ0?H{grJrq| zk}dKrbKs`EGQ!Gjfxzq4=C-GSJ1<@kj6bXJ0(ORrC};ww6{*hCd%!n7a}pQf(Z?U2 z%kTZYnZaiEIdbhk;DkM1DM(3;-aL>v9=(Z+O;&JC!rse`Z&L~CWUoLqE;X+inax=3 z{_hcCXWI^^*)_kZ@ar+S)@K7F;wk5NhhpxVbF;0)MBHdK_QYm4omUdGbSjEmWA;P&=;HYi{d!wc@ zp`djOBl% z-<5!T$H1%{6LAH%xS*VSyMJV0oWiX!1eB;H8z%Pe?f}Xx^62I7Uw_zGa%!;^POC1u zvzCOFl-t~DH<+Gr!_53Xyk@#^yd_oyLWht(Z(#Yb^I+|Gc+1__`v;TkNQ~)(Qb0k> zU-a>N=LB^T-4*Ow{c%uIe?Gd;2X=yU0fzxrSl@bmF@!;1Ta|f65GLhlXLNeQ*iv`` zN^*t${Z|#aejDARFWoK~DCK9InYIvTW)pD9DJMdmM80#^nQ>1fbO&oE5+F2xusFdE zI-rqHLJ(1>k~W>@N<7&a(r+V70rN_{yHbA>K2Y-T9Iok;xJFL>(?T%)=!#e{r(FsS zGR_Djy(Z@c7OH<#Wy5?j$hb<@oo@5BA98^+DyLaZQCEO`xV^41euJ;2`)OkUvNZFZ zFA7Zh^b#BE=0g|F7_b3r3Vxv}O^V|M&$q4w9s0>R6|w+x9e~hG+c5D(10M)&KK!N! z(he)YX9DBUreU2RZone+{$}M>OcV>_B=#I*FNO_8xt!kART? z#Bu>@Fxgf%-YDK97F4pW_Y}Dz2Kc8Yd8re8mf3_~oo&JepHdnu*9emhK=H@FM*Pur z{okO^ZTSJZba-AtkR+PY_rE!c{%6#RQYxo=Kb^YO@r zawfO*2Hm#)TU?Auxvhu|;6yg3JRN?r|KosuLmc1_GKN^@*eJj8B58Y@{wxJMNw7<` zKJwkIGe7RFt-a9U4;#ObZQ58h*n9Qv58)rp8j1Z-EIskC z=@ImjPD9OQSZE-0kn1-ia~^(E@a4qlGb&qeLTWIw%{@nA2uW0O92yn7D1HkOze3=? z9|rDby_C)gd?yhp3cgAMQu8?dSbO3JqHQ{4(-|b0@ojOEqJnTh4D=&6kpi@`3*vfG zmS_-a_7aE&;)CJjg=z1;pY^ir$4XKJG6dQdX21aEYglR|W}B}+TAG*BfL0uO7yDxYsYzhD<|K3v5^ zxrtvG?7FIGXpB0`l(jDd$@%vjpcRO9uFZ?K)cnjM19ooREbiay(BfBT*sc3?NafI7 zt7A?2B!H;h&BfZ~5M2ddoQGeWiyXi_xtscXMLBoHft6Z#g9nWUh_o?$23dU$gk$$E zX=h;beiR;q=o=s`$CETps8I)Mr@}m9(or&y8>dOe32A0L_wMEVb!wQH>x*l#NY4-oGU`=34 z5Bx6TIivLgBhjw1xDIHliHh;ie~EQL%WzqCoWzO15ROkGay`j|XGG-Fn>AMk{vL>& z&I+qaAkjArfoO6PM9k09JyG@OfWr9z-GX_jfXqAq8_t*yD^;Pd(a|Rlo)6(jz$QYY zvZ?UBksRL!tu^jy#+hjJIvG5q)0qcB^AW)U z&%!A>JtM`WaT0-xF%*#GA-fanJ*fEJ(EvAMh zdhi;B0-cUPo1PV-_&yyF|Y^uK-u9!DLoY>FZyLe-VQciC=WyV%Z-jaWSuX z1KNs^Ygi{%uZ;|nr{c1tXzDTE!N&qyPuBc&HLn*tX$ycwlyBLNxO^9^&4zn5tURS# zReRi}BRX``3g>x}!lUU3C=7#NS`np#?MRZD1>Y;2G!FL)#InzA66*NV;6J#AR21^L)oUCd*H1X&btGF<$Che zKi`Ry15=O>&FIJKE!* zLce_Q6EqOX5Q_Nk7DA}k6;IV*c-@dB_|3jOo0ct+8|<^r{e9XC7_XY)y@tacs8<%q zd}LFba%~Q6ln9}o-HlS5XL=W^^{i#QTV|4PaZ=1x0K7Sou4&-+Ur)-0vLeS#DjU zInwss(xEz}&q27#BXxcf=QSqe%W%jV0VNd054`ib7!`G483$stKs0mH{Yo%GiePL6 z=?e&L`+O+p*Jo9W|D`&z(X|tYy}{K+JbhT(rf>pTeKH$)EHM(Qnog@b%QWIyr3q^~b2Fi~hfKth{%uecn6- zZ<`Ml-qV{f*ckB6{k-{|E=txJ7a^ z2;v4rgSUoMOLWrmCfZLbVaJ*Fq;DGb>!dSNj4Xcf-0Y%)*R%Tuf7=ABRt|!FJbT~gI6|#7qXxDW_S)_FU zlUn}K9JNj@D_5YrX!L!F48_JdU*Qog{D!DKuaXTIz#XeeLKaN5)i17k9mss2yoc`0 zpc8<6TA2BW@}@(gKL1}7Dtn!J2tY!nnsd}cs(y(F108k3hmLSY1XyH=7|HE>6j2Pk zz9t%`aR!}o5U0l9Z*CjDTTv86_kkK|%>+TVMSQFeB^y6LODt+Qu|%QK_ClsUsM4%} z-N$Z#1eH2-k_KkbuIx!9$fxrw*bC=FGY-PbV5r`XwxdM{=vs6Uzz)C@H4$&nDVacI zQ8O?QYIMihE@X~{2a|OM?ls(IW$Xs^HJG4)ynx!8(b&Ik|{VMrV43p>&bzESMng%>+Pjk$5>W?KBb zftqrT6k9XF!84@FNDVAQLqhWnVrTxq#!cK;mpTwn1=q{@L{kFEBjoO17+L)+scP_DH~1eSIOk_t{L$b_C4K>Np*S3Tq3!OFL z%P82&cd_;vaQxm?$0;i5mQ?1KliTg?;6vQ`Scv*lx$+E{o}hgoPB7taj=CVo?9ZyETqB-?1HE!=x>jZMr@mu~ zgE4^{iCxk%T@uRG>(uQ)Zl7+2T`QIG{xwr+ h4?&7hN9-#r{+hG!;&7=ZJKh5;e7yoat2~&@{{hW=Jz@X= diff --git a/docs/.gitignore b/docs/.gitignore new file mode 100644 index 00000000000..e48dc98be89 --- /dev/null +++ b/docs/.gitignore @@ -0,0 +1,9 @@ +.DS_STORE +_site/ +*.swo +*.swp +_site +.sass-cache +*.psd +*~ + diff --git a/docs/CNAME b/docs/CNAME new file mode 100644 index 00000000000..827d1c0ed1e --- /dev/null +++ b/docs/CNAME @@ -0,0 +1 @@ +rocksdb.org \ No newline at end of file diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md new file mode 100644 index 00000000000..2c5842fb456 --- /dev/null +++ b/docs/CONTRIBUTING.md @@ -0,0 +1,115 @@ +This provides guidance on how to contribute various content to `rocksdb.org`. + +## Getting started + +You should only have to do these one time. + +- Rename this file to `CONTRIBUTING.md`. +- Rename `EXAMPLE-README-FOR-RUNNING-DOCS.md` to `README.md` (replacing the existing `README.md` that came with the template). +- Rename `EXAMPLE-LICENSE` to `LICENSE`. +- Review the [template information](./TEMPLATE-INFORMATION.md). +- Review `./_config.yml`. +- Make sure you update `title`, `description`, `tagline` and `gacode` (Google Analytics) in `./_config.yml`. + +## Basic Structure + +Most content is written in markdown. You name the file `something.md`, then have a header that looks like this: + +``` +--- +docid: getting-started +title: Getting started with ProjectName +layout: docs +permalink: /docs/getting-started.html +--- +``` + +Customize these values for each document, blog post, etc. + +> The filename of the `.md` file doesn't actually matter; what is important is the `docid` being unique and the `permalink` correct and unique too). + +## Landing page + +Modify `index.md` with your new or updated content. + +If you want a `GridBlock` as part of your content, you can do so directly with HTML: + +``` +

     +``` + +or with a combination of changing `./_data/features.yml` and adding some Liquid to `index.md`, such as: + +``` +{% include content/gridblocks.html data_source=site.data.features imagealign="bottom"%} +``` + +## Blog + +To modify a blog post, edit the appopriate markdown file in `./_posts/`. + +Adding a new blog post is a four-step process. + +> Some posts have a `permalink` and `comments` in the blog post YAML header. You will not need these for new blog posts. These are an artifact of migrating the blog from Wordpress to gh-pages. + +1. Create your blog post in `./_posts/` in markdown (file extension `.md` or `.markdown`). See current posts in that folder or `./doc-type-examples/2016-04-07-blog-post-example.md` for an example of the YAML format. **If the `./_posts` directory does not exist, create it**. + - You can add a `` tag in the middle of your post such that you show only the excerpt above that tag in the main `/blog` index on your page. +1. If you have not authored a blog post before, modify the `./_data/authors.yml` file with the `author` id you used in your blog post, along with your full name and Facebook ID to get your profile picture. +1. [Run the site locally](./README.md) to test your changes. It will be at `http://127.0.0.1/blog/your-new-blog-post-title.html` +1. Push your changes to GitHub. + +## Docs + +To modify docs, edit the appropriate markdown file in `./_docs/`. + +To add docs to the site.... + +1. Add your markdown file to the `./_docs/` folder. See `./doc-type-examples/docs-hello-world.md` for an example of the YAML header format. **If the `./_docs/` directory does not exist, create it**. + - You can use folders in the `./_docs/` directory to organize your content if you want. +1. Update `_data/nav_docs.yml` to add your new document to the navigation bar. Use the `docid` you put in your doc markdown in as the `id` in the `_data/nav_docs.yml` file. +1. [Run the site locally](./README.md) to test your changes. It will be at `http://127.0.0.1/docs/your-new-doc-permalink.html` +1. Push your changes to GitHub. + +## Header Bar + +To modify the header bar, change `./_data/nav.yml`. + +## Top Level Page + +To modify a top-level page, edit the appropriate markdown file in `./top-level/` + +If you want a top-level page (e.g., http://your-site.com/top-level.html) -- not in `/blog/` or `/docs/`.... + +1. Create a markdown file in the root `./top-level/`. See `./doc-type-examples/top-level-example.md` for more information. +1. If you want a visible link to that file, update `_data/nav.yml` to add a link to your new top-level document in the header bar. + + > This is not necessary if you just want to have a page that is linked to from another page, but not exposed as direct link to the user. + +1. [Run the site locally](./README.md) to test your changes. It will be at `http://127.0.0.1/your-top-level-page-permalink.html` +1. Push your changes to GitHub. + +## Other Changes + +- CSS: `./css/main.css` or `./_sass/*.scss`. +- Images: `./static/images/[docs | posts]/....` +- Main Blog post HTML: `./_includes/post.html` +- Main Docs HTML: `./_includes/doc.html` diff --git a/docs/Gemfile b/docs/Gemfile new file mode 100644 index 00000000000..93dc8b0d7f6 --- /dev/null +++ b/docs/Gemfile @@ -0,0 +1,2 @@ +source 'https://rubygems.org' +gem 'github-pages', '~> 104' diff --git a/docs/Gemfile.lock b/docs/Gemfile.lock new file mode 100644 index 00000000000..346fc6c1e16 --- /dev/null +++ b/docs/Gemfile.lock @@ -0,0 +1,145 @@ +GEM + remote: https://rubygems.org/ + specs: + activesupport (4.2.7) + i18n (~> 0.7) + json (~> 1.7, >= 1.7.7) + minitest (~> 5.1) + thread_safe (~> 0.3, >= 0.3.4) + tzinfo (~> 1.1) + addressable (2.4.0) + coffee-script (2.4.1) + coffee-script-source + execjs + coffee-script-source (1.10.0) + colorator (1.1.0) + ethon (0.9.1) + ffi (>= 1.3.0) + execjs (2.7.0) + faraday (0.9.2) + multipart-post (>= 1.2, < 3) + ffi (1.9.14) + forwardable-extended (2.6.0) + gemoji (2.1.0) + github-pages (104) + activesupport (= 4.2.7) + github-pages-health-check (= 1.2.0) + jekyll (= 3.3.0) + jekyll-avatar (= 0.4.2) + jekyll-coffeescript (= 1.0.1) + jekyll-feed (= 0.8.0) + jekyll-gist (= 1.4.0) + jekyll-github-metadata (= 2.2.0) + jekyll-mentions (= 1.2.0) + jekyll-paginate (= 1.1.0) + jekyll-redirect-from (= 0.11.0) + jekyll-sass-converter (= 1.3.0) + jekyll-seo-tag (= 2.1.0) + jekyll-sitemap (= 0.12.0) + jekyll-swiss (= 0.4.0) + jemoji (= 0.7.0) + kramdown (= 1.11.1) + liquid (= 3.0.6) + listen (= 3.0.6) + mercenary (~> 0.3) + minima (= 2.0.0) + rouge (= 1.11.1) + terminal-table (~> 1.4) + github-pages-health-check (1.2.0) + addressable (~> 2.3) + net-dns (~> 0.8) + octokit (~> 4.0) + public_suffix (~> 1.4) + typhoeus (~> 0.7) + html-pipeline (2.4.2) + activesupport (>= 2) + nokogiri (>= 1.4) + i18n (0.7.0) + jekyll (3.3.0) + addressable (~> 2.4) + colorator (~> 1.0) + jekyll-sass-converter (~> 1.0) + jekyll-watch (~> 1.1) + kramdown (~> 1.3) + liquid (~> 3.0) + mercenary (~> 0.3.3) + pathutil (~> 0.9) + rouge (~> 1.7) + safe_yaml (~> 1.0) + jekyll-avatar (0.4.2) + jekyll (~> 3.0) + jekyll-coffeescript (1.0.1) + coffee-script (~> 2.2) + jekyll-feed (0.8.0) + jekyll (~> 3.3) + jekyll-gist (1.4.0) + octokit (~> 4.2) + jekyll-github-metadata (2.2.0) + jekyll (~> 3.1) + octokit (~> 4.0, != 4.4.0) + jekyll-mentions (1.2.0) + activesupport (~> 4.0) + html-pipeline (~> 2.3) + jekyll (~> 3.0) + jekyll-paginate (1.1.0) + jekyll-redirect-from (0.11.0) + jekyll (>= 2.0) + jekyll-sass-converter (1.3.0) + sass (~> 3.2) + jekyll-seo-tag (2.1.0) + jekyll (~> 3.3) + jekyll-sitemap (0.12.0) + jekyll (~> 3.3) + jekyll-swiss (0.4.0) + jekyll-watch (1.5.0) + listen (~> 3.0, < 3.1) + jemoji (0.7.0) + activesupport (~> 4.0) + gemoji (~> 2.0) + html-pipeline (~> 2.2) + jekyll (>= 3.0) + json (1.8.3) + kramdown (1.11.1) + liquid (3.0.6) + listen (3.0.6) + rb-fsevent (>= 0.9.3) + rb-inotify (>= 0.9.7) + mercenary (0.3.6) + mini_portile2 (2.1.0) + minima (2.0.0) + minitest (5.9.1) + multipart-post (2.0.0) + net-dns (0.8.0) + nokogiri (1.6.8.1) + mini_portile2 (~> 2.1.0) + octokit (4.4.1) + sawyer (~> 0.7.0, >= 0.5.3) + pathutil (0.14.0) + forwardable-extended (~> 2.6) + public_suffix (1.5.3) + rb-fsevent (0.9.8) + rb-inotify (0.9.7) + ffi (>= 0.5.0) + rouge (1.11.1) + safe_yaml (1.0.4) + sass (3.4.22) + sawyer (0.7.0) + addressable (>= 2.3.5, < 2.5) + faraday (~> 0.8, < 0.10) + terminal-table (1.7.3) + unicode-display_width (~> 1.1.1) + thread_safe (0.3.5) + typhoeus (0.8.0) + ethon (>= 0.8.0) + tzinfo (1.2.2) + thread_safe (~> 0.1) + unicode-display_width (1.1.1) + +PLATFORMS + ruby + +DEPENDENCIES + github-pages (~> 104) + +BUNDLED WITH + 1.13.1 diff --git a/docs/LICENSE-DOCUMENTATION b/docs/LICENSE-DOCUMENTATION new file mode 100644 index 00000000000..1f255c9f373 --- /dev/null +++ b/docs/LICENSE-DOCUMENTATION @@ -0,0 +1,385 @@ +Attribution 4.0 International + +======================================================================= + +Creative Commons Corporation ("Creative Commons") is not a law firm and +does not provide legal services or legal advice. Distribution of +Creative Commons public licenses does not create a lawyer-client or +other relationship. Creative Commons makes its licenses and related +information available on an "as-is" basis. Creative Commons gives no +warranties regarding its licenses, any material licensed under their +terms and conditions, or any related information. Creative Commons +disclaims all liability for damages resulting from their use to the +fullest extent possible. + +Using Creative Commons Public Licenses + +Creative Commons public licenses provide a standard set of terms and +conditions that creators and other rights holders may use to share +original works of authorship and other material subject to copyright +and certain other rights specified in the public license below. The +following considerations are for informational purposes only, are not +exhaustive, and do not form part of our licenses. + + Considerations for licensors: Our public licenses are + intended for use by those authorized to give the public + permission to use material in ways otherwise restricted by + copyright and certain other rights. Our licenses are + irrevocable. Licensors should read and understand the terms + and conditions of the license they choose before applying it. + Licensors should also secure all rights necessary before + applying our licenses so that the public can reuse the + material as expected. Licensors should clearly mark any + material not subject to the license. This includes other CC- + licensed material, or material used under an exception or + limitation to copyright. More considerations for licensors: + wiki.creativecommons.org/Considerations_for_licensors + + Considerations for the public: By using one of our public + licenses, a licensor grants the public permission to use the + licensed material under specified terms and conditions. If + the licensor's permission is not necessary for any reason--for + example, because of any applicable exception or limitation to + copyright--then that use is not regulated by the license. Our + licenses grant only permissions under copyright and certain + other rights that a licensor has authority to grant. Use of + the licensed material may still be restricted for other + reasons, including because others have copyright or other + rights in the material. A licensor may make special requests, + such as asking that all changes be marked or described. + Although not required by our licenses, you are encouraged to + respect those requests where reasonable. More_considerations + for the public: + wiki.creativecommons.org/Considerations_for_licensees + +======================================================================= + +Creative Commons Attribution 4.0 International Public License + +By exercising the Licensed Rights (defined below), You accept and agree +to be bound by the terms and conditions of this Creative Commons +Attribution 4.0 International Public License ("Public License"). To the +extent this Public License may be interpreted as a contract, You are +granted the Licensed Rights in consideration of Your acceptance of +these terms and conditions, and the Licensor grants You such rights in +consideration of benefits the Licensor receives from making the +Licensed Material available under these terms and conditions. + +Section 1 -- Definitions. + + a. Adapted Material means material subject to Copyright and Similar + Rights that is derived from or based upon the Licensed Material + and in which the Licensed Material is translated, altered, + arranged, transformed, or otherwise modified in a manner requiring + permission under the Copyright and Similar Rights held by the + Licensor. For purposes of this Public License, where the Licensed + Material is a musical work, performance, or sound recording, + Adapted Material is always produced where the Licensed Material is + synched in timed relation with a moving image. + +b. Adapter's License means the license You apply to Your Copyright + and Similar Rights in Your contributions to Adapted Material in + accordance with the terms and conditions of this Public License. + +c. Copyright and Similar Rights means copyright and/or similar rights + closely related to copyright including, without limitation, + performance, broadcast, sound recording, and Sui Generis Database + Rights, without regard to how the rights are labeled or + categorized. For purposes of this Public License, the rights + specified in Section 2(b)(1)-(2) are not Copyright and Similar + Rights. + +d. Effective Technological Measures means those measures that, in the + absence of proper authority, may not be circumvented under laws + fulfilling obligations under Article 11 of the WIPO Copyright + Treaty adopted on December 20, 1996, and/or similar international + agreements. + +e. Exceptions and Limitations means fair use, fair dealing, and/or + any other exception or limitation to Copyright and Similar Rights + that applies to Your use of the Licensed Material. + +f. Licensed Material means the artistic or literary work, database, + or other material to which the Licensor applied this Public + License. + +g. Licensed Rights means the rights granted to You subject to the + terms and conditions of this Public License, which are limited to + all Copyright and Similar Rights that apply to Your use of the + Licensed Material and that the Licensor has authority to license. + +h. Licensor means the individual(s) or entity(ies) granting rights + under this Public License. + +i. Share means to provide material to the public by any means or + process that requires permission under the Licensed Rights, such + as reproduction, public display, public performance, distribution, + dissemination, communication, or importation, and to make material + available to the public including in ways that members of the + public may access the material from a place and at a time + individually chosen by them. + +j. Sui Generis Database Rights means rights other than copyright + resulting from Directive 96/9/EC of the European Parliament and of + the Council of 11 March 1996 on the legal protection of databases, + as amended and/or succeeded, as well as other essentially + equivalent rights anywhere in the world. + +k. You means the individual or entity exercising the Licensed Rights + under this Public License. Your has a corresponding meaning. + +Section 2 -- Scope. + +a. License grant. + + 1. Subject to the terms and conditions of this Public License, + the Licensor hereby grants You a worldwide, royalty-free, + non-sublicensable, non-exclusive, irrevocable license to + exercise the Licensed Rights in the Licensed Material to: + + a. reproduce and Share the Licensed Material, in whole or + in part; and + + b. produce, reproduce, and Share Adapted Material. + + 2. Exceptions and Limitations. For the avoidance of doubt, where + Exceptions and Limitations apply to Your use, this Public + License does not apply, and You do not need to comply with + its terms and conditions. + + 3. Term. The term of this Public License is specified in Section + 6(a). + + 4. Media and formats; technical modifications allowed. The + Licensor authorizes You to exercise the Licensed Rights in + all media and formats whether now known or hereafter created, + and to make technical modifications necessary to do so. The + Licensor waives and/or agrees not to assert any right or + authority to forbid You from making technical modifications + necessary to exercise the Licensed Rights, including + technical modifications necessary to circumvent Effective + Technological Measures. For purposes of this Public License, + simply making modifications authorized by this Section 2(a) + (4) never produces Adapted Material. + + 5. Downstream recipients. + + a. Offer from the Licensor -- Licensed Material. Every + recipient of the Licensed Material automatically + receives an offer from the Licensor to exercise the + Licensed Rights under the terms and conditions of this + Public License. + + b. No downstream restrictions. You may not offer or impose + any additional or different terms or conditions on, or + apply any Effective Technological Measures to, the + Licensed Material if doing so restricts exercise of the + Licensed Rights by any recipient of the Licensed + Material. + + 6. No endorsement. Nothing in this Public License constitutes or + may be construed as permission to assert or imply that You + are, or that Your use of the Licensed Material is, connected + with, or sponsored, endorsed, or granted official status by, + the Licensor or others designated to receive attribution as + provided in Section 3(a)(1)(A)(i). + +b. Other rights. + + 1. Moral rights, such as the right of integrity, are not + licensed under this Public License, nor are publicity, + privacy, and/or other similar personality rights; however, to + the extent possible, the Licensor waives and/or agrees not to + assert any such rights held by the Licensor to the limited + extent necessary to allow You to exercise the Licensed + Rights, but not otherwise. + + 2. Patent and trademark rights are not licensed under this + Public License. + + 3. To the extent possible, the Licensor waives any right to + collect royalties from You for the exercise of the Licensed + Rights, whether directly or through a collecting society + under any voluntary or waivable statutory or compulsory + licensing scheme. In all other cases the Licensor expressly + reserves any right to collect such royalties. + +Section 3 -- License Conditions. + +Your exercise of the Licensed Rights is expressly made subject to the +following conditions. + +a. Attribution. + + 1. If You Share the Licensed Material (including in modified + form), You must: + + a. retain the following if it is supplied by the Licensor + with the Licensed Material: + + i. identification of the creator(s) of the Licensed + Material and any others designated to receive + attribution, in any reasonable manner requested by + the Licensor (including by pseudonym if + designated); + + ii. a copyright notice; + + iii. a notice that refers to this Public License; + + iv. a notice that refers to the disclaimer of + warranties; + + v. a URI or hyperlink to the Licensed Material to the + extent reasonably practicable; + + b. indicate if You modified the Licensed Material and + retain an indication of any previous modifications; and + + c. indicate the Licensed Material is licensed under this + Public License, and include the text of, or the URI or + hyperlink to, this Public License. + + 2. You may satisfy the conditions in Section 3(a)(1) in any + reasonable manner based on the medium, means, and context in + which You Share the Licensed Material. For example, it may be + reasonable to satisfy the conditions by providing a URI or + hyperlink to a resource that includes the required + information. + + 3. If requested by the Licensor, You must remove any of the + information required by Section 3(a)(1)(A) to the extent + reasonably practicable. + + 4. If You Share Adapted Material You produce, the Adapter's + License You apply must not prevent recipients of the Adapted + Material from complying with this Public License. + +Section 4 -- Sui Generis Database Rights. + +Where the Licensed Rights include Sui Generis Database Rights that +apply to Your use of the Licensed Material: + +a. for the avoidance of doubt, Section 2(a)(1) grants You the right + to extract, reuse, reproduce, and Share all or a substantial + portion of the contents of the database; + +b. if You include all or a substantial portion of the database + contents in a database in which You have Sui Generis Database + Rights, then the database in which You have Sui Generis Database + Rights (but not its individual contents) is Adapted Material; and + +c. You must comply with the conditions in Section 3(a) if You Share + all or a substantial portion of the contents of the database. + +For the avoidance of doubt, this Section 4 supplements and does not +replace Your obligations under this Public License where the Licensed +Rights include other Copyright and Similar Rights. + +Section 5 -- Disclaimer of Warranties and Limitation of Liability. + +a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE + EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS + AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF + ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, + IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, + WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR + PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, + ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT + KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT + ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. + +b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE + TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, + NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, + INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, + COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR + USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN + ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR + DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR + IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. + +c. The disclaimer of warranties and limitation of liability provided + above shall be interpreted in a manner that, to the extent + possible, most closely approximates an absolute disclaimer and + waiver of all liability. + +Section 6 -- Term and Termination. + +a. This Public License applies for the term of the Copyright and + Similar Rights licensed here. However, if You fail to comply with + this Public License, then Your rights under this Public License + terminate automatically. + +b. Where Your right to use the Licensed Material has terminated under + Section 6(a), it reinstates: + + 1. automatically as of the date the violation is cured, provided + it is cured within 30 days of Your discovery of the + violation; or + + 2. upon express reinstatement by the Licensor. + + For the avoidance of doubt, this Section 6(b) does not affect any + right the Licensor may have to seek remedies for Your violations + of this Public License. + +c. For the avoidance of doubt, the Licensor may also offer the + Licensed Material under separate terms or conditions or stop + distributing the Licensed Material at any time; however, doing so + will not terminate this Public License. + +d. Sections 1, 5, 6, 7, and 8 survive termination of this Public + License. + +Section 7 -- Other Terms and Conditions. + +a. The Licensor shall not be bound by any additional or different + terms or conditions communicated by You unless expressly agreed. + +b. Any arrangements, understandings, or agreements regarding the + Licensed Material not stated herein are separate from and + independent of the terms and conditions of this Public License. + +Section 8 -- Interpretation. + +a. For the avoidance of doubt, this Public License does not, and + shall not be interpreted to, reduce, limit, restrict, or impose + conditions on any use of the Licensed Material that could lawfully + be made without permission under this Public License. + +b. To the extent possible, if any provision of this Public License is + deemed unenforceable, it shall be automatically reformed to the + minimum extent necessary to make it enforceable. If the provision + cannot be reformed, it shall be severed from this Public License + without affecting the enforceability of the remaining terms and + conditions. + +c. No term or condition of this Public License will be waived and no + failure to comply consented to unless expressly agreed to by the + Licensor. + +d. Nothing in this Public License constitutes or may be interpreted + as a limitation upon, or waiver of, any privileges and immunities + that apply to the Licensor or You, including from the legal + processes of any jurisdiction or authority. + +======================================================================= + +Creative Commons is not a party to its public licenses. +Notwithstanding, Creative Commons may elect to apply one of its public +licenses to material it publishes and in those instances will be +considered the "Licensor." Except for the limited purpose of indicating +that material is shared under a Creative Commons public license or as +otherwise permitted by the Creative Commons policies published at +creativecommons.org/policies, Creative Commons does not authorize the +use of the trademark "Creative Commons" or any other trademark or logo +of Creative Commons without its prior written consent including, +without limitation, in connection with any unauthorized modifications +to any of its public licenses or any other arrangements, +understandings, or agreements concerning use of licensed material. For +the avoidance of doubt, this paragraph does not form part of the public +licenses. + +Creative Commons may be contacted at creativecommons.org. + diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000000..0ae8978bcbb --- /dev/null +++ b/docs/README.md @@ -0,0 +1,80 @@ +## User Documentation for rocksdb.org + +This directory will contain the user and feature documentation for RocksDB. The documentation will be hosted on GitHub pages. + +### Contributing + +See [CONTRIBUTING.md](./CONTRIBUTING.md) for details on how to add or modify content. + +### Run the Site Locally + +The requirements for running a GitHub pages site locally is described in [GitHub help](https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/#requirements). The steps below summarize these steps. + +> If you have run the site before, you can start with step 1 and then move on to step 5. + +1. Ensure that you are in the `/docs` directory in your local RocksDB clone (i.e., the same directory where this `README.md` exists). The below RubyGems commands, etc. must be run from there. + +1. Make sure you have Ruby and [RubyGems](https://rubygems.org/) installed. + + > Ruby >= 2.2 is required for the gems. On the latest versions of Mac OS X, Ruby 2.0 is the + > default. Use `brew install ruby` (or your preferred upgrade mechanism) to install a newer + > version of Ruby for your Mac OS X system. + +1. Make sure you have [Bundler](http://bundler.io/) installed. + + ``` + # may require sudo + gem install bundler + ``` +1. Install the project's dependencies + + ``` + # run this in the 'docs' directory + bundle install + ``` + + > If you get an error when installing `nokogiri`, you may be running into the problem described + > in [this nokogiri issue](https://github.com/sparklemotion/nokogiri/issues/1483). You can + > either `brew uninstall xz` (and then `brew install xz` after the bundle is installed) or + > `xcode-select --install` (although this may not work if you have already installed command + > line tools). + +1. Run Jekyll's server. + + - On first runs or for structural changes to the documentation (e.g., new sidebar menu item), do a full build. + + ``` + bundle exec jekyll serve + ``` + + - For content changes only, you can use `--incremental` for faster builds. + + ``` + bundle exec jekyll serve --incremental + ``` + + > We use `bundle exec` instead of running straight `jekyll` because `bundle exec` will always use the version of Jekyll from our `Gemfile`. Just running `jekyll` will use the system version and may not necessarily be compatible. + + - To run using an actual IP address, you can use `--host=0.0.0.0` + + ``` + bundle exec jekyll serve --host=0.0.0.0 + ``` + + This will allow you to use the IP address associated with your machine in the URL. That way you could share it with other people. + + e.g., on a Mac, you can your IP address with something like `ifconfig | grep "inet " | grep -v 127.0.0.1`. + +1. Either of commands in the previous step will serve up the site on your local device at http://127.0.0.1:4000/ or http://localhost:4000. + +### Updating the Bundle + +The site depends on Github Pages and the installed bundle is based on the `github-pages` gem. +Occasionally that gem might get updated with new or changed functionality. If that is the case, +you can run: + +``` +bundle update +``` + +to get the latest packages for the installation. diff --git a/docs/TEMPLATE-INFORMATION.md b/docs/TEMPLATE-INFORMATION.md new file mode 100644 index 00000000000..9175bc0c292 --- /dev/null +++ b/docs/TEMPLATE-INFORMATION.md @@ -0,0 +1,17 @@ +## Template Details + +First, go through `_config.yml` and adjust the available settings to your project's standard. When you make changes here, you'll have to kill the `jekyll serve` instance and restart it to see those changes, but that's only the case with the config file. + +Next, update some image assets - you'll want to update `favicon.png`, `logo.svg`, and `og_image.png` (used for Like button stories and Shares on Facbeook) in the `static` folder with your own logos. + +Next, if you're going to have docs on your site, keep the `_docs` and `docs` folders, if not, you can safely remove them (or you can safely leave them and not include them in your navigation - Jekyll renders all of this before a client views the site anyway, so there's no performance hit from just leaving it there for a future expansion). + +Same thing with a blog section, either keep or delete the `_posts` and `blog` folders. + +You can customize your homepage in three parts - the first in the homepage header, which is mostly automatically derived from the elements you insert into your config file. However, you can also specify a series of 'promotional' elements in `_data/promo.yml`. You can read that file for more information. + +The second place for your homepage is in `index.md` which contains the bulk of the main content below the header. This is all markdown if you want, but you can use HTML and Jekyll's template tags (called Liquid) in there too. Checkout this folder's index.md for an example of one common template tag that we use on our sites called gridblocks. + +The third and last place is in the `_data/powered_by.yml` and `_data/powered_by_highlight.yml` files. Both these files combine to create a section on the homepage that is intended to show a list of companies or apps that are using your project. The `powered_by_highlight` file is a list of curated companies/apps that you want to show as a highlight at the top of this section, including their logos in whatever format you want. The `powered_by` file is a more open list that is just text links to the companies/apps and can be updated via Pull Request by the community. If you don't want these sections on your homepage, just empty out both files and leave them blank. + +The last thing you'll want to do is setup your top level navigation bar. You can do this by editing `nav.yml` and keeping the existing title/href/category structure used there. Although the nav is responsive and fairly flexible design-wise, no more than 5 or 6 nav items is recommended. diff --git a/docs/_config.yml b/docs/_config.yml new file mode 100644 index 00000000000..2e5cee097fb --- /dev/null +++ b/docs/_config.yml @@ -0,0 +1,85 @@ +# Site settings +permalink: /blog/:year/:month/:day/:title.html +title: RocksDB +tagline: A persistent key-value store for fast storage environments +description: > + RocksDB is an embeddable persistent key-value store for fast storage. +fbappid: "1615782811974223" +gacode: "UA-49459723-1" +# baseurl determines the subpath of your site. For example if you're using an +# organisation.github.io/reponame/ basic site URL, then baseurl would be set +# as "/reponame" but leave blank if you have a top-level domain URL as it is +# now set to "" by default as discussed in: +# http://jekyllrb.com/news/2016/10/06/jekyll-3-3-is-here/ +baseurl: "" + +# the base hostname & protocol for your site +# If baseurl is set, then the absolute url for your site would be url/baseurl +# This was also be set to the right thing automatically for local development +# https://github.com/blog/2277-what-s-new-in-github-pages-with-jekyll-3-3 +# http://jekyllrb.com/news/2016/10/06/jekyll-3-3-is-here/ +url: "http://rocksdb.org" + +# Note: There are new filters in Jekyll 3.3 to help with absolute and relative urls +# absolute_url +# relative_url +# So you will see these used throughout the Jekyll code in this template. +# no more need for | prepend: site.url | prepend: site.baseurl +# http://jekyllrb.com/news/2016/10/06/jekyll-3-3-is-here/ +#https://github.com/blog/2277-what-s-new-in-github-pages-with-jekyll-3-3 + +# The GitHub repo for your project +ghrepo: "facebook/rocksdb" + +# Use these color settings to determine your colour scheme for the site. +color: + # primary should be a vivid color that reflects the project's brand + primary: "#2a2a2a" + # secondary should be a subtle light or dark color used on page backgrounds + secondary: "#f9f9f9" + # Use the following to specify whether the previous two colours are 'light' + # or 'dark' and therefore what colors can be overlaid on them + primary-overlay: "dark" + secondary-overlay: "light" + +#Uncomment this if you want to enable Algolia doc search with your own values +#searchconfig: +# apikey: "" +# indexname: "" + +# Blog posts are builtin to Jekyll by default, with the `_posts` directory. +# Here you can specify other types of documentation. The names here are `docs` +# and `top-level`. This means their content will be in `_docs` and `_top-level`. +# The permalink format is also given. +# http://ben.balter.com/2015/02/20/jekyll-collections/ +collections: + docs: + output: true + permalink: /docs/:name/ + top-level: + output: true + permalink: :name.html + +# DO NOT ADJUST BELOW THIS LINE UNLESS YOU KNOW WHAT YOU ARE CHANGING + +markdown: kramdown +kramdown: + input: GFM + syntax_highlighter: rouge + + syntax_highlighter_opts: + css_class: 'rougeHighlight' + span: + line_numbers: false + block: + line_numbers: true + start_line: 1 + +sass: + style: :compressed + +redcarpet: + extensions: [with_toc_data] + +gems: + - jekyll-redirect-from diff --git a/docs/_data/authors.yml b/docs/_data/authors.yml new file mode 100644 index 00000000000..b3ed49e5398 --- /dev/null +++ b/docs/_data/authors.yml @@ -0,0 +1,62 @@ +icanadi: + full_name: Igor Canadi + fbid: 706165749 + +xjin: + full_name: Xing Jin + fbid: 100000739847320 + +leijin: + full_name: Lei Jin + fbid: 634570164 + +yhciang: + full_name: Yueh-Hsuan Chiang + fbid: 1619020986 + +radheshyam: + full_name: Radheshyam Balasundaram + fbid: 800837305 + +zagfox: + full_name: Feng Zhu + fbid: 100006493823622 + +lgalanis: + full_name: Leonidas Galanis + fbid: 8649950 + +sdong: + full_name: Siying Dong + fbid: 9805119 + +dmitrism: + full_name: Dmitri Smirnov + +rven2: + full_name: Venkatesh Radhakrishnan + fbid: 100008352697325 + +yiwu: + full_name: Yi Wu + fbid: 100000476362039 + +maysamyabandeh: + full_name: Maysam Yabandeh + fbid: 100003482360101 + +IslamAbdelRahman: + full_name: Islam AbdelRahman + fbid: 642759407 + +ajkr: + full_name: Andrew Kryczka + fbid: 100010829806660 + +sagar0: + full_name: Sagar Vemuri + fbid: 2419111 + +lightmark: + full_name: Aaron Gao + fbid: 1351549072 diff --git a/docs/_data/features.yml b/docs/_data/features.yml new file mode 100644 index 00000000000..d692c1849d1 --- /dev/null +++ b/docs/_data/features.yml @@ -0,0 +1,19 @@ +- title: High Performance + text: | + RocksDB uses a log structured database engine, written entirely in C++, for maximum performance. Keys and values are just arbitrarily-sized byte streams. + image: images/promo-performance.svg + +- title: Optimized for Fast Storage + text: | + RocksDB is optimized for fast, low latency storage such as flash drives and high-speed disk drives. RocksDB exploits the full potential of high read/write rates offered by flash or RAM. + image: images/promo-flash.svg + +- title: Adaptable + text: | + RocksDB is adaptable to different workloads. From database storage engines such as [MyRocks](https://github.com/facebook/mysql-5.6) to [application data caching](http://techblog.netflix.com/2016/05/application-data-caching-using-ssds.html) to embedded workloads, RocksDB can be used for a variety of data needs. + image: images/promo-adapt.svg + +- title: Basic and Advanced Database Operations + text: | + RocksDB provides basic operations such as opening and closing a database, reading and writing to more advanced operations such as merging and compaction filters. + image: images/promo-operations.svg diff --git a/docs/_data/nav.yml b/docs/_data/nav.yml new file mode 100644 index 00000000000..108de025453 --- /dev/null +++ b/docs/_data/nav.yml @@ -0,0 +1,30 @@ +- title: Docs + href: /docs/ + category: docs + +- title: GitHub + href: https://github.com/facebook/rocksdb/ + category: external + +- title: API (C++) + href: https://github.com/facebook/rocksdb/tree/master/include/rocksdb + category: external + +- title: API (Java) + href: https://github.com/facebook/rocksdb/tree/master/java/src/main/java/org/rocksdb + category: external + +- title: Support + href: /support.html + category: support + +- title: Blog + href: /blog/ + category: blog + +- title: Facebook + href: https://www.facebook.com/groups/rocksdb.dev/ + category: external + +# Use external for external links not associated with the paths of the current site. +# If a category is external, site urls, for example, are not prepended to the href, etc.. diff --git a/docs/_data/nav_docs.yml b/docs/_data/nav_docs.yml new file mode 100644 index 00000000000..8cdfd2d04d9 --- /dev/null +++ b/docs/_data/nav_docs.yml @@ -0,0 +1,3 @@ +- title: Quick Start + items: + - id: getting-started diff --git a/docs/_data/powered_by.yml b/docs/_data/powered_by.yml new file mode 100644 index 00000000000..a780cfe4012 --- /dev/null +++ b/docs/_data/powered_by.yml @@ -0,0 +1 @@ +# Fill in later if desired diff --git a/docs/_data/powered_by_highlight.yml b/docs/_data/powered_by_highlight.yml new file mode 100644 index 00000000000..a780cfe4012 --- /dev/null +++ b/docs/_data/powered_by_highlight.yml @@ -0,0 +1 @@ +# Fill in later if desired diff --git a/docs/_data/promo.yml b/docs/_data/promo.yml new file mode 100644 index 00000000000..9a72aa844c0 --- /dev/null +++ b/docs/_data/promo.yml @@ -0,0 +1,6 @@ +# This file determines the list of promotional elements added to the header of \ +# your site's homepage. Full list of plugins are shown + +- type: button + href: docs/getting-started.html + text: Get Started diff --git a/docs/_docs/faq.md b/docs/_docs/faq.md new file mode 100644 index 00000000000..0887a0987f7 --- /dev/null +++ b/docs/_docs/faq.md @@ -0,0 +1,48 @@ +--- +docid: support-faq +title: FAQ +layout: docs +permalink: /docs/support/faq.html +--- + +Here is an ever-growing list of frequently asked questions around RocksDB + +## What is RocksDB? + +RocksDB is an embeddable persistent key-value store for fast storage. RocksDB can also be the foundation for a client-server database but our current focus is on embedded workloads. + +RocksDB builds on [LevelDB](https://code.google.com/p/leveldb/) to be scalable to run on servers with many CPU cores, to efficiently use fast storage, to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation. + +For the latest details, watch [Mark Callaghan’s and Igor Canadi’s talk at CMU on 10/2015](https://scs.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=f4e0eb37-ae18-468f-9248-cb73edad3e56). [Dhruba Borthakur’s introductory talk](https://github.com/facebook/rocksdb/blob/gh-pages-old/intro.pdf?raw=true) from the Data @ Scale 2013 conference provides some perspective about how RocksDB has evolved. + +## How does performance compare? + +We benchmarked LevelDB and found that it was unsuitable for our server workloads. The [benchmark results](http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html) look awesome at first sight, but we quickly realized that those results were for a database whose size was smaller than the size of RAM on the test machine – where the entire database could fit in the OS page cache. When we performed the same benchmarks on a database that was at least 5 times larger than main memory, the performance results were dismal. + +By contrast, we’ve published the [RocksDB benchmark results](https://github.com/facebook/rocksdb/wiki/Performance-Benchmarks) for server side workloads on Flash. We also measured the performance of LevelDB on these server-workload benchmarks and found that RocksDB solidly outperforms LevelDB for these IO bound workloads. We found that LevelDB’s single-threaded compaction process was insufficient to drive server workloads. We saw frequent write-stalls with LevelDB that caused 99-percentile latency to be tremendously large. We found that mmap-ing a file into the OS cache introduced performance bottlenecks for reads. We could not make LevelDB consume all the IOs offered by the underlying Flash storage. + +## What is RocksDB suitable for? + +RocksDB can be used by applications that need low latency database accesses. Possibilities include: + +* A user-facing application that stores the viewing history and state of users of a website. +* A spam detection application that needs fast access to big data sets. +* A graph-search query that needs to scan a data set in realtime. +* A cache data from Hadoop, thereby allowing applications to query Hadoop data in realtime. +* A message-queue that supports a high number of inserts and deletes. + +## How big is RocksDB adoption? + +RocksDB is an embedded storage engine that is used in a number of backend systems at Facebook. In the Facebook newsfeed’s backend, it replaced another internal storage engine called Centrifuge and is one of the many components used. ZippyDB, a distributed key value store service used by Facebook products relies RocksDB. Details on ZippyDB are in [Muthu Annamalai’s talk at Data@Scale in Seattle](https://youtu.be/DfiN7pG0D0k). Dragon, a distributed graph query engine part of the social graph infrastructure, is using RocksDB to store data. Parse has been running [MongoDB on RocksDB in production](http://blog.parse.com/announcements/mongodb-rocksdb-parse/) since early 2015. + +RocksDB is proving to be a useful component for a lot of other groups in the industry. For a list of projects currently using RocksDB, take a look at our USERS.md list on github. + +## How good is RocksDB as a database storage engine? + +Our engineering team at Facebook firmly believes that RocksDB has great potential as storage engine for databases. It has been proven in production with MongoDB: [MongoRocks](https://github.com/mongodb-partners/mongo-rocks) is the RocksDB based storage engine for MongoDB. + +[MyRocks](https://code.facebook.com/posts/190251048047090/myrocks-a-space-and-write-optimized-mysql-database/) is the RocksDB based storage engine for MySQL. Using RocksDB we have managed to achieve 2x better compression and 10x less write amplification for our benchmarks compared to our existing MySQL setup. Given our current results, work is currently underway to develop MyRocks into a production ready solution for web-scale MySQL workloads. Follow along on [GitHub](https://github.com/facebook/mysql-5.6)! + +## Why is RocksDB open sourced? + +We are open sourcing this project on [GitHub](http://github.com/facebook/rocksdb) because we think it will be useful beyond Facebook. We are hoping that software programmers and database developers will use, enhance, and customize RocksDB for their use-cases. We would also like to engage with the academic community on topics related to efficiency for modern database algorithms. diff --git a/docs/_docs/getting-started.md b/docs/_docs/getting-started.md new file mode 100644 index 00000000000..8b01dfefd45 --- /dev/null +++ b/docs/_docs/getting-started.md @@ -0,0 +1,78 @@ +--- +docid: getting-started +title: Getting started +layout: docs +permalink: /docs/getting-started.html +--- + +## Overview + +The RocksDB library provides a persistent key value store. Keys and values are arbitrary byte arrays. The keys are ordered within the key value store according to a user-specified comparator function. + +The library is maintained by the Facebook Database Engineering Team, and is based on [LevelDB](https://github.com/google/leveldb), by Sanjay Ghemawat and Jeff Dean at Google. + +This overview gives some simple examples of how RocksDB is used. For the story of why RocksDB was created in the first place, see [Dhruba Borthakur’s introductory talk](https://github.com/facebook/rocksdb/blob/gh-pages-old/intro.pdf?raw=true) from the Data @ Scale 2013 conference. + +## Opening A Database + +A rocksdb database has a name which corresponds to a file system directory. All of the contents of database are stored in this directory. The following example shows how to open a database, creating it if necessary: + +```c++ +#include +#include "rocksdb/db.h" + +rocksdb::DB* db; +rocksdb::Options options; +options.create_if_missing = true; +rocksdb::Status status = + rocksdb::DB::Open(options, "/tmp/testdb", &db); +assert(status.ok()); +... +``` + +If you want to raise an error if the database already exists, add the following line before the rocksdb::DB::Open call: + +```c++ +options.error_if_exists = true; +``` + +## Status + +You may have noticed the `rocksdb::Status` type above. Values of this type are returned by most functions in RocksDB that may encounter +an error. You can check if such a result is ok, and also print an associated error message: + +```c++ +rocksdb::Status s = ...; +if (!s.ok()) cerr << s.ToString() << endl; +``` + +## Closing A Database + +When you are done with a database, just delete the database object. For example: + +```c++ +/* open the db as described above */ +/* do something with db */ +delete db; +``` + +## Reads And Writes + +The database provides Put, Delete, and Get methods to modify/query the database. For example, the following code moves the value stored under `key1` to `key2`. + +```c++ +std::string value; +rocksdb::Status s = db->Get(rocksdb::ReadOptions(), key1, &value); +if (s.ok()) s = db->Put(rocksdb::WriteOptions(), key2, value); +if (s.ok()) s = db->Delete(rocksdb::WriteOptions(), key1); +``` + +## Further documentation + +These are just simple examples of how RocksDB is used. The full documentation is currently on the [GitHub wiki](https://github.com/facebook/rocksdb/wiki). + +Here are some specific details about the RocksDB implementation: + +- [Architecture Guide](https://github.com/facebook/rocksdb/wiki/Rocksdb-Architecture-Guide) +- [Format of an immutable Table file](https://github.com/facebook/rocksdb/wiki/Rocksdb-Table-Format) +- [Format of a log file](https://github.com/facebook/rocksdb/wiki/Write-Ahead-Log-File-Format) diff --git a/docs/_includes/blog_pagination.html b/docs/_includes/blog_pagination.html new file mode 100644 index 00000000000..6a1f33436e9 --- /dev/null +++ b/docs/_includes/blog_pagination.html @@ -0,0 +1,28 @@ + +{% if paginator.total_pages > 1 %} +
    + +
    +{% endif %} diff --git a/docs/_includes/content/gridblocks.html b/docs/_includes/content/gridblocks.html new file mode 100644 index 00000000000..49c5e5917d3 --- /dev/null +++ b/docs/_includes/content/gridblocks.html @@ -0,0 +1,5 @@ +
    +{% for item in {{include.data_source}} %} + {% include content/items/gridblock.html item=item layout=include.layout imagealign=include.imagealign align=include.align %} +{% endfor %} +
    \ No newline at end of file diff --git a/docs/_includes/content/items/gridblock.html b/docs/_includes/content/items/gridblock.html new file mode 100644 index 00000000000..58c9e7fdafb --- /dev/null +++ b/docs/_includes/content/items/gridblock.html @@ -0,0 +1,37 @@ +{% if include.layout == "fourColumn" %} + {% assign layout = "fourByGridBlock" %} +{% else %} + {% assign layout = "twoByGridBlock" %} +{% endif %} + +{% if include.imagealign == "side" %} + {% assign imagealign = "imageAlignSide" %} +{% else %} + {% if item.image %} + {% assign imagealign = "imageAlignTop" %} + {% else %} + {% assign imagealign = "" %} + {% endif %} +{% endif %} + +{% if include.align == "right" %} + {% assign align = "alignRight" %} +{% elsif include.align == "center" %} + {% assign align = "alignCenter" %} +{% else %} + {% assign align = "alignLeft" %} +{% endif %} + +
    + {% if item.image %} +
    + {{ item.title }} +
    + {% endif %} +
    +

    {{ item.title }}

    + {% if item.text %} + {{ item.text | markdownify }} + {% endif %} +
    +
    diff --git a/docs/_includes/doc.html b/docs/_includes/doc.html new file mode 100644 index 00000000000..a7950004ecc --- /dev/null +++ b/docs/_includes/doc.html @@ -0,0 +1,25 @@ +
    +
    +

    {% if include.truncate %}{{ page.title }}{% else %}{{ page.title }}{% endif %}

    +
    + +
    + {% if include.truncate %} + {% if page.content contains '' %} + {{ page.content | split:'' | first }} +
    + + ...Read More + +
    + {% else %} + {{ page.content }} + {% endif %} + {% else %} + {{ content }} + +

    Edit on GitHub

    + {% endif %} +
    + {% include doc_paging.html %} +
    diff --git a/docs/_includes/doc_paging.html b/docs/_includes/doc_paging.html new file mode 100644 index 00000000000..e69de29bb2d diff --git a/docs/_includes/footer.html b/docs/_includes/footer.html new file mode 100644 index 00000000000..dd9494aeb5d --- /dev/null +++ b/docs/_includes/footer.html @@ -0,0 +1,33 @@ +
    + +
    + diff --git a/docs/_includes/head.html b/docs/_includes/head.html new file mode 100644 index 00000000000..10845ec1d57 --- /dev/null +++ b/docs/_includes/head.html @@ -0,0 +1,23 @@ + + + + + + + + + + + + + + {% if site.searchconfig %} + + {% endif %} + + {% if page.title %}{{ page.title }} | {{ site.title }}{% else %}{{ site.title }}{% endif %} + + + + + diff --git a/docs/_includes/header.html b/docs/_includes/header.html new file mode 100644 index 00000000000..8108d222be9 --- /dev/null +++ b/docs/_includes/header.html @@ -0,0 +1,19 @@ +
    +
    +
    + +

    {{ site.title }}

    +

    {{ site.tagline }}

    + +
    +

    {% if page.excerpt %}{{ page.excerpt | strip_html }}{% else %}{{ site.description }}{% endif %}

    +
    +
    + {% for promo in site.data.promo %} + {% include plugins/{{promo.type}}.html button_href=promo.href button_text=promo.text %} +
    + {% endfor %} +
    +
    +
    +
    diff --git a/docs/_includes/hero.html b/docs/_includes/hero.html new file mode 100644 index 00000000000..e69de29bb2d diff --git a/docs/_includes/home_header.html b/docs/_includes/home_header.html new file mode 100644 index 00000000000..90880d17cfd --- /dev/null +++ b/docs/_includes/home_header.html @@ -0,0 +1,22 @@ +
    +
    +
    +
    +

    {{ site.tagline }}

    +
    +

    {% if page.excerpt %}{{ page.excerpt | strip_html }}{% else %}{{ site.description }}{% endif %}

    +
    +
    + {% for promo in site.data.promo %} +
    + {% include plugins/{{promo.type}}.html href=promo.href text=promo.text children=promo.children %} +
    + {% endfor %} +
    +
    + +
    +
    +
    diff --git a/docs/_includes/katex_import.html b/docs/_includes/katex_import.html new file mode 100644 index 00000000000..6d6b7cf44a3 --- /dev/null +++ b/docs/_includes/katex_import.html @@ -0,0 +1,3 @@ + + + diff --git a/docs/_includes/katex_render.html b/docs/_includes/katex_render.html new file mode 100644 index 00000000000..56e2e89743d --- /dev/null +++ b/docs/_includes/katex_render.html @@ -0,0 +1,210 @@ + diff --git a/docs/_includes/nav.html b/docs/_includes/nav.html new file mode 100644 index 00000000000..9c6fed06b10 --- /dev/null +++ b/docs/_includes/nav.html @@ -0,0 +1,37 @@ +
    +
    +
    + + +

    {{ site.title }}

    +     + + + +
    +
    +
    diff --git a/docs/_includes/nav/collection_nav.html b/docs/_includes/nav/collection_nav.html new file mode 100644 index 00000000000..a3c7a2dd35b --- /dev/null +++ b/docs/_includes/nav/collection_nav.html @@ -0,0 +1,64 @@ +
    + +
    + diff --git a/docs/_includes/nav/collection_nav_group.html b/docs/_includes/nav/collection_nav_group.html new file mode 100644 index 00000000000..b236ac5e3ff --- /dev/null +++ b/docs/_includes/nav/collection_nav_group.html @@ -0,0 +1,19 @@ +
    +

    +{{ grouptitle }}

    +
      + {% if include.data_collection %} + {% for item in group.items %} + {% for collectionitem in include.data_collection %} + {% if collectionitem.docid == item.id %} + {% assign groupitem = collectionitem %} + {% include nav/collection_nav_group_item.html %} + {% endif %} + {% endfor %} + {% endfor %} + {% else %} + {% for groupitem in groupitems %} + {% include nav/collection_nav_group_item.html %} + {% endfor %} + {% endif %} +
    +
    \ No newline at end of file diff --git a/docs/_includes/nav/collection_nav_group_item.html b/docs/_includes/nav/collection_nav_group_item.html new file mode 100644 index 00000000000..fbb063deb73 --- /dev/null +++ b/docs/_includes/nav/collection_nav_group_item.html @@ -0,0 +1 @@ +
    +
    +
    +

    Your Features

    +     +
    +
    + +
    +
    +

    More information

    +

    + Stuff here +

    +
    +
    +
  • {{ groupitem.title }}
    • diff --git a/docs/_includes/nav/header_nav.html b/docs/_includes/nav/header_nav.html new file mode 100644 index 00000000000..0fe945cdcd2 --- /dev/null +++ b/docs/_includes/nav/header_nav.html @@ -0,0 +1,30 @@ +
    +
    + +
    + +
    + \ No newline at end of file diff --git a/docs/_includes/nav_search.html b/docs/_includes/nav_search.html new file mode 100644 index 00000000000..84956b9f786 --- /dev/null +++ b/docs/_includes/nav_search.html @@ -0,0 +1,15 @@ +
  • + +
    • + + \ No newline at end of file diff --git a/docs/_includes/plugins/all_share.html b/docs/_includes/plugins/all_share.html new file mode 100644 index 00000000000..59b00d615fd --- /dev/null +++ b/docs/_includes/plugins/all_share.html @@ -0,0 +1,3 @@ +
    + {% include plugins/like_button.html %}{% include plugins/twitter_share.html %}{% include plugins/google_share.html %} +
    \ No newline at end of file diff --git a/docs/_includes/plugins/ascii_cinema.html b/docs/_includes/plugins/ascii_cinema.html new file mode 100644 index 00000000000..7d3f971480d --- /dev/null +++ b/docs/_includes/plugins/ascii_cinema.html @@ -0,0 +1,2 @@ +
    + \ No newline at end of file diff --git a/docs/_includes/plugins/button.html b/docs/_includes/plugins/button.html new file mode 100644 index 00000000000..9e499fe3f39 --- /dev/null +++ b/docs/_includes/plugins/button.html @@ -0,0 +1,6 @@ +
    + {{ include.text }} +
    \ No newline at end of file diff --git a/docs/_includes/plugins/github_star.html b/docs/_includes/plugins/github_star.html new file mode 100644 index 00000000000..6aea70fc737 --- /dev/null +++ b/docs/_includes/plugins/github_star.html @@ -0,0 +1,4 @@ +
    + Star +
    + \ No newline at end of file diff --git a/docs/_includes/plugins/github_watch.html b/docs/_includes/plugins/github_watch.html new file mode 100644 index 00000000000..64233b57b68 --- /dev/null +++ b/docs/_includes/plugins/github_watch.html @@ -0,0 +1,4 @@ +
    + Watch +
    + \ No newline at end of file diff --git a/docs/_includes/plugins/google_share.html b/docs/_includes/plugins/google_share.html new file mode 100644 index 00000000000..1b557db86c5 --- /dev/null +++ b/docs/_includes/plugins/google_share.html @@ -0,0 +1,5 @@ +
    +
    +
    + + diff --git a/docs/_includes/plugins/iframe.html b/docs/_includes/plugins/iframe.html new file mode 100644 index 00000000000..525b59f2274 --- /dev/null +++ b/docs/_includes/plugins/iframe.html @@ -0,0 +1,6 @@ +
    + +
    +
    + {% include plugins/button.html href=include.href text=include.text %} +
    \ No newline at end of file diff --git a/docs/_includes/plugins/like_button.html b/docs/_includes/plugins/like_button.html new file mode 100644 index 00000000000..bcb8a7beef0 --- /dev/null +++ b/docs/_includes/plugins/like_button.html @@ -0,0 +1,18 @@ +
    + \ No newline at end of file diff --git a/docs/_includes/plugins/plugin_row.html b/docs/_includes/plugins/plugin_row.html new file mode 100644 index 00000000000..800f50b8214 --- /dev/null +++ b/docs/_includes/plugins/plugin_row.html @@ -0,0 +1,5 @@ +
    +{% for child in include.children %} + {% include plugins/{{child.type}}.html href=child.href text=child.text %} +{% endfor %} +
    \ No newline at end of file diff --git a/docs/_includes/plugins/post_social_plugins.html b/docs/_includes/plugins/post_social_plugins.html new file mode 100644 index 00000000000..cb4d6e2ece0 --- /dev/null +++ b/docs/_includes/plugins/post_social_plugins.html @@ -0,0 +1,34 @@ +
    + +
    +
    + + + diff --git a/docs/_includes/plugins/slideshow.html b/docs/_includes/plugins/slideshow.html new file mode 100644 index 00000000000..69fa2b300e4 --- /dev/null +++ b/docs/_includes/plugins/slideshow.html @@ -0,0 +1,88 @@ +
    + + + \ No newline at end of file diff --git a/docs/_includes/plugins/twitter_follow.html b/docs/_includes/plugins/twitter_follow.html new file mode 100644 index 00000000000..9443db3dcbe --- /dev/null +++ b/docs/_includes/plugins/twitter_follow.html @@ -0,0 +1,5 @@ +
    + +
    + + \ No newline at end of file diff --git a/docs/_includes/plugins/twitter_share.html b/docs/_includes/plugins/twitter_share.html new file mode 100644 index 00000000000..67f1d62f98a --- /dev/null +++ b/docs/_includes/plugins/twitter_share.html @@ -0,0 +1,4 @@ +
    + +
    + \ No newline at end of file diff --git a/docs/_includes/post.html b/docs/_includes/post.html new file mode 100644 index 00000000000..4c4b5b1acee --- /dev/null +++ b/docs/_includes/post.html @@ -0,0 +1,35 @@ +
    + {% assign author = site.data.authors[page.author] %} +
    + {% if author.fbid %} +
    + {{ author.fullname }} +
    + {% endif %} + {% if author.full_name %} +

    {{ author.full_name }}

    + {% endif %} +

    {% if include.truncate %}{{ page.title }}{% else %}{{ page.title }}{% endif %}

    +

    Posted {{ page.date | date: '%B %d, %Y' }}{% if page.meta %} • {{ page.meta }}{% endif %}

    +
    + +
    + {% if include.truncate %} + {% if page.content contains '' %} + {{ page.content | split:'' | first | markdownify }} +
    + + Read More + +
    + {% else %} + {{ page.content | markdownify }} + {% endif %} + {% else %} + {{ content }} + {% endif %} + {% unless include.truncate %} + {% include plugins/like_button.html %} + {% endunless %} +
    +
    diff --git a/docs/_includes/powered_by.html b/docs/_includes/powered_by.html new file mode 100644 index 00000000000..c629429cd0f --- /dev/null +++ b/docs/_includes/powered_by.html @@ -0,0 +1,28 @@ +{% if site.data.powered_by.first.items or site.data.powered_by_highlight.first.items %} +
    +
    + {% if site.data.powered_by_highlight.first.title %} +

    {{ site.data.powered_by_highlight.first.title }}

    + {% else %} +

    {{ site.data.powered_by.first.title }}

    + {% endif %} + {% if site.data.powered_by_highlight.first.items %} +
    + {% for item in site.data.powered_by_highlight.first.items %} +
    + {{ item.name }} +
    + {% endfor %} +
    + {% endif %} +
    + {% for item in site.data.powered_by.first.items %} +
    + {{ item.name }} +
    + {% endfor %} +
    +
    Does your app use {{ site.title }}? Add it to this list with a pull request!
    +
    +
    +{% endif %} diff --git a/docs/_includes/social_plugins.html b/docs/_includes/social_plugins.html new file mode 100644 index 00000000000..db4f1aecdd2 --- /dev/null +++ b/docs/_includes/social_plugins.html @@ -0,0 +1,24 @@ + +
    + +
    + + + diff --git a/docs/_includes/ui/button.html b/docs/_includes/ui/button.html new file mode 100644 index 00000000000..729ccc33b93 --- /dev/null +++ b/docs/_includes/ui/button.html @@ -0,0 +1 @@ +{{ include.button_text }} \ No newline at end of file diff --git a/docs/_layouts/basic.html b/docs/_layouts/basic.html new file mode 100644 index 00000000000..65bd21060c9 --- /dev/null +++ b/docs/_layouts/basic.html @@ -0,0 +1,12 @@ +--- +layout: doc_default +--- + +
    +
    +
    + {{ content }} +
    +
    +
    + diff --git a/docs/_layouts/blog.html b/docs/_layouts/blog.html new file mode 100644 index 00000000000..1b0da41359e --- /dev/null +++ b/docs/_layouts/blog.html @@ -0,0 +1,11 @@ +--- +category: blog +layout: blog_default +--- + +
    +
    + {{ content }} +
    +
    + diff --git a/docs/_layouts/blog_default.html b/docs/_layouts/blog_default.html new file mode 100644 index 00000000000..a29d58d3dda --- /dev/null +++ b/docs/_layouts/blog_default.html @@ -0,0 +1,14 @@ + + + {% include head.html %} + + {% include nav.html alwayson=true %} +
    +
    + {% include nav/collection_nav.html navdata=site.posts type="blog" sectionpath="/blog/" sectiontitle="Blog" %} + {{ content }} +
    + {% include footer.html %} +
    + + diff --git a/docs/_layouts/default.html b/docs/_layouts/default.html new file mode 100644 index 00000000000..0167d9fd91f --- /dev/null +++ b/docs/_layouts/default.html @@ -0,0 +1,12 @@ + + + {% include head.html %} + + {% include nav.html alwayson=true %} +
    + {{ content }} + {% include footer.html %} +
    + + + diff --git a/docs/_layouts/doc_default.html b/docs/_layouts/doc_default.html new file mode 100644 index 00000000000..4a4139247c9 --- /dev/null +++ b/docs/_layouts/doc_default.html @@ -0,0 +1,14 @@ + + + {% include head.html %} + + {% include nav.html alwayson=true %} +
    +
    + {% include nav/collection_nav.html navdata=site.data.nav_docs type="docs" sectionpath="/docs/" sectiontitle="Docs" data_collection=site.docs %} + {{ content }} +
    + {% include footer.html %} +
    + + diff --git a/docs/_layouts/doc_page.html b/docs/_layouts/doc_page.html new file mode 100644 index 00000000000..dba761e7d70 --- /dev/null +++ b/docs/_layouts/doc_page.html @@ -0,0 +1,10 @@ +--- +layout: doc_default +--- + +
    +
    + {{ content }} +
    +
    + diff --git a/docs/_layouts/docs.html b/docs/_layouts/docs.html new file mode 100644 index 00000000000..749dafabbe4 --- /dev/null +++ b/docs/_layouts/docs.html @@ -0,0 +1,5 @@ +--- +layout: doc_page +--- + +{% include doc.html %} \ No newline at end of file diff --git a/docs/_layouts/home.html b/docs/_layouts/home.html new file mode 100644 index 00000000000..e3c320f55c1 --- /dev/null +++ b/docs/_layouts/home.html @@ -0,0 +1,17 @@ + + + {% include head.html %} + + {% include nav.html alwayson=true %} +
    + {% include home_header.html %} +
    +
    + {{ content }} +
    + {% include powered_by.html %} +
    + {% include footer.html %} +
    + + diff --git a/docs/_layouts/page.html b/docs/_layouts/page.html new file mode 100644 index 00000000000..bec36805b22 --- /dev/null +++ b/docs/_layouts/page.html @@ -0,0 +1,3 @@ +--- +layout: blog +--- diff --git a/docs/_layouts/plain.html b/docs/_layouts/plain.html new file mode 100644 index 00000000000..fccc02ce17e --- /dev/null +++ b/docs/_layouts/plain.html @@ -0,0 +1,10 @@ +--- +layout: default +--- + +
    +
    + {{ content }} +
    +
    + diff --git a/docs/_layouts/post.html b/docs/_layouts/post.html new file mode 100644 index 00000000000..4c92cf214c4 --- /dev/null +++ b/docs/_layouts/post.html @@ -0,0 +1,8 @@ +--- +collection: blog +layout: blog +--- + +
    +{% include post.html %} +
    \ No newline at end of file diff --git a/docs/_layouts/redirect.html b/docs/_layouts/redirect.html new file mode 100644 index 00000000000..c24f8174849 --- /dev/null +++ b/docs/_layouts/redirect.html @@ -0,0 +1,6 @@ + + + + + + diff --git a/docs/_layouts/top-level.html b/docs/_layouts/top-level.html new file mode 100644 index 00000000000..fccc02ce17e --- /dev/null +++ b/docs/_layouts/top-level.html @@ -0,0 +1,10 @@ +--- +layout: default +--- + +
    +
    + {{ content }} +
    +
    + diff --git a/docs/_posts/2014-03-27-how-to-backup-rocksdb.markdown b/docs/_posts/2014-03-27-how-to-backup-rocksdb.markdown new file mode 100644 index 00000000000..f9e4a54447a --- /dev/null +++ b/docs/_posts/2014-03-27-how-to-backup-rocksdb.markdown @@ -0,0 +1,135 @@ +--- +title: How to backup RocksDB? +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/191/how-to-backup-rocksdb/ +--- + +In RocksDB, we have implemented an easy way to backup your DB. Here is a simple example: + + + + #include "rocksdb/db.h" + #include "utilities/backupable_db.h" + using namespace rocksdb; + + DB* db; + DB::Open(Options(), "/tmp/rocksdb", &db); + BackupableDB* backupable_db = new BackupableDB(db, BackupableDBOptions("/tmp/rocksdb_backup")); + backupable_db->Put(...); // do your thing + backupable_db->CreateNewBackup(); + delete backupable_db; // no need to also delete db + + + + +This simple example will create a backup of your DB in "/tmp/rocksdb_backup". Creating new BackupableDB consumes DB* and you should be calling all the DB methods on object `backupable_db` going forward. + +Restoring is also easy: + + + + RestoreBackupableDB* restore = new RestoreBackupableDB(Env::Default(), BackupableDBOptions("/tmp/rocksdb_backup")); + restore->RestoreDBFromLatestBackup("/tmp/rocksdb", "/tmp/rocksdb"); + delete restore; + + + + +This code will restore the backup back to "/tmp/rocksdb". The second parameter is the location of log files (In some DBs they are different from DB directory, but usually they are the same. See Options::wal_dir for more info). + +An alternative API for backups is to use BackupEngine directly: + + + + #include "rocksdb/db.h" + #include "utilities/backupable_db.h" + using namespace rocksdb; + + DB* db; + DB::Open(Options(), "/tmp/rocksdb", &db); + db->Put(...); // do your thing + BackupEngine* backup_engine = BackupEngine::NewBackupEngine(Env::Default(), BackupableDBOptions("/tmp/rocksdb_backup")); + backup_engine->CreateNewBackup(db); + delete db; + delete backup_engine; + + + + +Restoring with BackupEngine is similar to RestoreBackupableDB: + + + + BackupEngine* backup_engine = BackupEngine::NewBackupEngine(Env::Default(), BackupableDBOptions("/tmp/rocksdb_backup")); + backup_engine->RestoreDBFromLatestBackup("/tmp/rocksdb", "/tmp/rocksdb"); + delete backup_engine; + + + + +Backups are incremental. You can create a new backup with `CreateNewBackup()` and only the new data will be copied to backup directory (for more details on what gets copied, see "Under the hood"). Checksum is always calculated for any backuped file (including sst, log, and etc). It is used to make sure files are kept sound in the file system. Checksum is also verified for files from the previous backups even though they do not need to be copied. A checksum mismatch aborts the current backup (see "Under the hood" for more details). Once you have more backups saved, you can issue `GetBackupInfo()` call to get a list of all backups together with information on timestamp of the backup and the size (please note that sum of all backups' sizes is bigger than the actual size of the backup directory because some data is shared by multiple backups). Backups are identified by their always-increasing IDs. `GetBackupInfo()` is available both in `BackupableDB` and `RestoreBackupableDB`. + +You probably want to keep around only small number of backups. To delete old backups, just call `PurgeOldBackups(N)`, where N is how many backups you'd like to keep. All backups except the N newest ones will be deleted. You can also choose to delete arbitrary backup with call `DeleteBackup(id)`. + +`RestoreDBFromLatestBackup()` will restore the DB from the latest consistent backup. An alternative is `RestoreDBFromBackup()` which takes a backup ID and restores that particular backup. Checksum is calculated for any restored file and compared against the one stored during the backup time. If a checksum mismatch is detected, the restore process is aborted and `Status::Corruption` is returned. Very important thing to note here: Let's say you have backups 1, 2, 3, 4. If you restore from backup 2 and start writing more data to your database, newly created backup will delete old backups 3 and 4 and create new backup 3 on top of 2. + + + +## Advanced usage + + +Let's say you want to backup your DB to HDFS. There is an option in `BackupableDBOptions` to set `backup_env`, which will be used for all file I/O related to backup dir (writes when backuping, reads when restoring). If you set it to HDFS Env, all the backups will be stored in HDFS. + +`BackupableDBOptions::info_log` is a Logger object that is used to print out LOG messages if not-nullptr. + +If `BackupableDBOptions::sync` is true, we will sync data to disk after every file write, guaranteeing that backups will be consistent after a reboot or if machine crashes. Setting it to false will speed things up a bit, but some (newer) backups might be inconsistent. In most cases, everything should be fine, though. + +If you set `BackupableDBOptions::destroy_old_data` to true, creating new `BackupableDB` will delete all the old backups in the backup directory. + +`BackupableDB::CreateNewBackup()` method takes a parameter `flush_before_backup`, which is false by default. When `flush_before_backup` is true, `BackupableDB` will first issue a memtable flush and only then copy the DB files to the backup directory. Doing so will prevent log files from being copied to the backup directory (since flush will delete them). If `flush_before_backup` is false, backup will not issue flush before starting the backup. In that case, the backup will also include log files corresponding to live memtables. Backup will be consistent with current state of the database regardless of `flush_before_backup` parameter. + + + +## Under the hood + + +`BackupableDB` implements `DB` interface and adds four methods to it: `CreateNewBackup()`, `GetBackupInfo()`, `PurgeOldBackups()`, `DeleteBackup()`. Any `DB` interface calls will get forwarded to underlying `DB` object. + +When you call `BackupableDB::CreateNewBackup()`, it does the following: + + + + + + 1. Disable file deletions + + + + 2. Get live files (this includes table files, current and manifest file). + + + + 3. Copy live files to the backup directory. Since table files are immutable and filenames unique, we don't copy a table file that is already present in the backup directory. For example, if there is a file `00050.sst` already backed up and `GetLiveFiles()` returns `00050.sst`, we will not copy that file to the backup directory. However, checksum is calculated for all files regardless if a file needs to be copied or not. If a file is already present, the calculated checksum is compared against previously calculated checksum to make sure nothing crazy happened between backups. If a mismatch is detected, backup is aborted and the system is restored back to the state before `BackupableDB::CreateNewBackup()` is called. One thing to note is that a backup abortion could mean a corruption from a file in backup directory or the corresponding live file in current DB. Both manifest and current files are copied, since they are not immutable. + + + + 4. If `flush_before_backup` was set to false, we also need to copy log files to the backup directory. We call `GetSortedWalFiles()` and copy all live files to the backup directory. + + + + 5. Enable file deletions + + + + +Backup IDs are always increasing and we have a file `LATEST_BACKUP` that contains the ID of the latest backup. If we crash in middle of backing up, on a restart we will detect that there are newer backup files than `LATEST_BACKUP` claims there are. In that case, we will delete any backup newer than `LATEST_BACKUP` and clean up all the files since some of the table files might be corrupted. Having corrupted table files in the backup directory is dangerous because of our deduplication strategy. + + + +## Further reading + + +For the API details, see `include/utilities/backupable_db.h`. For the implementation, see `utilities/backupable/backupable_db.cc`. diff --git a/docs/_posts/2014-03-27-how-to-persist-in-memory-rocksdb-database.markdown b/docs/_posts/2014-03-27-how-to-persist-in-memory-rocksdb-database.markdown new file mode 100644 index 00000000000..89ffb2d97e2 --- /dev/null +++ b/docs/_posts/2014-03-27-how-to-persist-in-memory-rocksdb-database.markdown @@ -0,0 +1,54 @@ +--- +title: How to persist in-memory RocksDB database? +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/245/how-to-persist-in-memory-rocksdb-database/ +--- + +In recent months, we have focused on optimizing RocksDB for in-memory workloads. With growing RAM sizes and strict low-latency requirements, lots of applications decide to keep their entire data in memory. Running in-memory database with RocksDB is easy -- just mount your RocksDB directory on tmpfs or ramfs [1]. Even if the process crashes, RocksDB can recover all of your data from in-memory filesystem. However, what happens if the machine reboots? + + + +In this article we will explain how you can recover your in-memory RocksDB database even after a machine reboot. + +Every update to RocksDB is written to two places - one is an in-memory data structure called memtable and second is write-ahead log. Write-ahead log can be used to completely recover the data in memtable. By default, when we flush the memtable to table file, we also delete the current log, since we don't need it anymore for recovery (the data from the log is "persisted" in the table file -- we say that the log file is obsolete). However, if your table file is stored in in-memory file system, you may need the obsolete write-ahead log to recover the data after the machine reboots. Here's how you can do that. + +Options::wal_dir is the directory where RocksDB stores write-ahead log files. If you configure this directory to be on flash or disk, you will not lose current log file on machine reboot. +Options::WAL_ttl_seconds is the timeout when we delete the archived log files. If the timeout is non-zero, obsolete log files will be moved to `archive/` directory under Options::wal_dir. Those archived log files will only be deleted after the specified timeout. + +Let's assume Options::wal_dir is a directory on persistent storage and Options::WAL_ttl_seconds is set to one day. To fully recover the DB, we also need to backup the current snapshot of the database (containing table and metadata files) with a frequency of less than one day. RocksDB provides an utility that enables you to easily backup the snapshot of your database. You can learn more about it here: [How to backup RocksDB?](https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB%3F) + +You should configure the backup process to avoid backing up log files, since they are already stored in persistent storage. To do that, set BackupableDBOptions::backup_log_files to false. + +Restore process by default cleans up entire DB and WAL directory. Since we didn't include log files in the backup, we need to make sure that restoring the database doesn't delete log files in WAL directory. When restoring, configure RestoreOptions::keep_log_file to true. That option will also move any archived log files back to WAL directory, enabling RocksDB to replay all archived log files and rebuild the in-memory database state. + +To reiterate, here's what you have to do: + + + + + * Set DB directory to tmpfs or ramfs mounted drive + + + + * Set Options::wal_log to a directory on persistent storage + + + + * Set Options::WAL_ttl_seconds to T seconds + + + + * Backup RocksDB every T/2 seconds, with BackupableDBOptions::backup_log_files = false + + + + * When you lose data, restore from backup with RestoreOptions::keep_log_file = true + + + + + +[1] You might also want to consider using [PlainTable format](https://github.com/facebook/rocksdb/wiki/PlainTable-Format) for table files diff --git a/docs/_posts/2014-04-02-the-1st-rocksdb-local-meetup-held-on-march-27-2014.markdown b/docs/_posts/2014-04-02-the-1st-rocksdb-local-meetup-held-on-march-27-2014.markdown new file mode 100644 index 00000000000..7ccbdbaada4 --- /dev/null +++ b/docs/_posts/2014-04-02-the-1st-rocksdb-local-meetup-held-on-march-27-2014.markdown @@ -0,0 +1,53 @@ +--- +title: The 1st RocksDB Local Meetup Held on March 27, 2014 +layout: post +author: xjin +category: blog +redirect_from: + - /blog/323/the-1st-rocksdb-local-meetup-held-on-march-27-2014/ +--- + +On Mar 27, 2014, RocksDB team @ Facebook held the 1st RocksDB local meetup in FB HQ (Menlo Park, California). We invited around 80 guests from 20+ local companies, including LinkedIn, Twitter, Dropbox, Square, Pinterest, MapR, Microsoft and IBM. Finally around 50 guests showed up, totaling around 60% show-up rate. + + + +[![Resize of 20140327_200754](/static/images/Resize-of-20140327_200754-300x225.jpg)](/static/images/Resize-of-20140327_200754-300x225.jpg) + +RocksDB team @ Facebook gave four talks about the latest progress and experience on RocksDB: + + + + + * [Supporting a 1PB In-Memory Workload](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Haobo-RocksDB-In-Memory.pdf) + + + + + * [Column Families in RocksDB](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Igor-Column-Families.pdf) + + + + + * ["Lockless" Get() in RocksDB?](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Lei-Lockless-Get.pdf) + + + + + * [Prefix Hashing in RocksDB](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Siying-Prefix-Hash.pdf) + + +A very interesting question asked by a massive number of guests is: does RocksDB plan to provide replication functionality? Obviously, many applications need a resilient and distributed storage solution, not just single-node storage. We are considering how to approach this issue. + +When will be the next meetup? We haven't decided yet. We will see whether the community is interested in it and how it can help RocksDB grow. + +If you have any questions or feedback for the meetup or RocksDB, please let us know in [our Facebook group](https://www.facebook.com/groups/rocksdb.dev/). + +### Comments + +**[Rajiv](geetasen@gmail.com)** + +Have any of these talks been recorded and if so will they be published? + +**[Igor Canadi](icanadi@fb.com)** + +Yes, I think we plan to publish them soon. diff --git a/docs/_posts/2014-04-07-rocksdb-2-8-release.markdown b/docs/_posts/2014-04-07-rocksdb-2-8-release.markdown new file mode 100644 index 00000000000..7be7842a5f6 --- /dev/null +++ b/docs/_posts/2014-04-07-rocksdb-2-8-release.markdown @@ -0,0 +1,40 @@ +--- +title: RocksDB 2.8 release +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/371/rocksdb-2-8-release/ +--- + +Check out the new RocksDB 2.8 release on [Github](https://github.com/facebook/rocksdb/releases/tag/2.8.fb). + +RocksDB 2.8. is mostly focused on improving performance for in-memory workloads. We are seeing read QPS as high as 5M (we will write a separate blog post on this). + + + +Here is the summary of new features: + + * Added a new table format called PlainTable, which is optimized for RAM storage (ramfs or tmpfs). You can read more details about it on [our wiki](https://github.com/facebook/rocksdb/wiki/PlainTable-Format). + + + * New prefixed memtable format HashLinkedList, which is optimized for cases where there are only a few keys for each prefix. + + + * Merge operator supports a new function PartialMergeMulti() that allows users to do partial merges against multiple operands. This function enables big speedups for workloads that use merge operators. + + + * Added a V2 compaction filter interface. It buffers the kv-pairs sharing the same key prefix, process them in batches, and return the batched results back to DB. + + + * Geo-spatial support for locations and radial-search. + + + * Improved read performance using thread local cache for frequently accessed data. + + + * Stability improvements -- we're now ignoring partially written tailing record to MANIFEST or WAL files. + + + +We have also introduced small incompatible API changes (mostly for advanced users). You can see full release notes in our [HISTORY.my](https://github.com/facebook/rocksdb/blob/2.8.fb/HISTORY.md) file. diff --git a/docs/_posts/2014-04-21-indexing-sst-files-for-better-lookup-performance.markdown b/docs/_posts/2014-04-21-indexing-sst-files-for-better-lookup-performance.markdown new file mode 100644 index 00000000000..368055d2c23 --- /dev/null +++ b/docs/_posts/2014-04-21-indexing-sst-files-for-better-lookup-performance.markdown @@ -0,0 +1,28 @@ +--- +title: Indexing SST Files for Better Lookup Performance +layout: post +author: leijin +category: blog +redirect_from: + - /blog/431/indexing-sst-files-for-better-lookup-performance/ +--- + +For a `Get()` request, RocksDB goes through mutable memtable, list of immutable memtables, and SST files to look up the target key. SST files are organized in levels. + +On level 0, files are sorted based on the time they are flushed. Their key range (as defined by FileMetaData.smallest and FileMetaData.largest) are mostly overlapped with each other. So it needs to look up every L0 file. + + + +Compaction is scheduled periodically to pick up files from an upper level and merges them with files from lower level. As a result, key/values are moved from L0 down the LSM tree gradually. Compaction sorts key/values and split them into files. From level 1 and below, SST files are sorted based on key. Their key range are mutually exclusive. Instead of scanning through each SST file and checking if a key falls into its range, RocksDB performs a binary search based on FileMetaData.largest to locate a candidate file that can potentially contain the target key. This reduces complexity from O(N) to O(log(N)). However, log(N) can still be large for bottom levels. For a fan-out ratio of 10, level 3 can have 1000 files. That requires 10 comparisons to locate a candidate file. This is a significant cost for an in-memory database when you can do [several million gets per second](https://github.com/facebook/rocksdb/wiki/RocksDB-In-Memory-Workload-Performance-Benchmarks). + +One observation to this problem is that: after the LSM tree is built, an SST file's position in its level is fixed. Furthermore, its order relative to files from the next level is also fixed. Based on this idea, we can perform [fractional cascading](http://en.wikipedia.org/wiki/Fractional_cascading) kind of optimization to narrow down the binary search range. Here is an example: + +[![tree_example](/static/images/tree_example1.png)](/static/images/tree_example1.png) + +Level 1 has 2 files and level 2 has 8 files. Now, we want to look up key 80. A binary search based FileMetaData.largest tells you file 1 is the candidate. Then key 80 is compared with its FileMetaData.smallest and FileMetaData.largest to decide if it falls into the range. The comparison shows 80 is less than FileMetaData.smallest (100), so file 1 does not possibly contain key 80. We to proceed to check level 2. Usually, we need to do binary search among all 8 files on level 2. But since we already know target key 80 is less than 100 and only file 1 to file 3 can contain key less than 100, we can safely exclude other files from the search. As a result we cut down the search space from 8 files to 3 files. + +Let's look at another example. We want to get key 230. A binary search on level 1 locates to file 2 (this also implies key 230 is larger than file 1's FileMetaData.largest 200). A comparison with file 2's range shows the target key is smaller than file 2's FileMetaData.smallest 300. Even though, we couldn't find key on level 1, we have derived hints that target key is in range between 200 and 300. Any files on level 2 that cannot overlap with [200, 300] can be safely excluded. As a result, we only need to look at file 5 and file 6 on level 2. + +Inspired by this concept, we pre-build pointers at compaction time on level 1 files that point to a range of files on level 2. For example, file 1 on level 1 points to file 3 (on level 2) on the left and file 4 on the right. File 2 will point to level 2 files 6 and 7. At query time, these pointers are used to determine the actual binary search range based on comparison result. + +Our benchmark shows that this optimization improves lookup QPS by ~5% for similar setup mentioned [here](https://github.com/facebook/rocksdb/wiki/RocksDB-In-Memory-Workload-Performance-Benchmarks). diff --git a/docs/_posts/2014-05-14-lock.markdown b/docs/_posts/2014-05-14-lock.markdown new file mode 100644 index 00000000000..12009cc88c1 --- /dev/null +++ b/docs/_posts/2014-05-14-lock.markdown @@ -0,0 +1,88 @@ +--- +title: Reducing Lock Contention in RocksDB +layout: post +author: sdong +category: blog +redirect_from: + - /blog/521/lock/ +--- + +In this post, we briefly introduce the recent improvements we did to RocksDB to improve the issue of lock contention costs. + +RocksDB has a simple thread synchronization mechanism (See [RocksDB Architecture Guide](https://github.com/facebook/rocksdb/wiki/Rocksdb-Architecture-Guide)  to understand terms used below, like SST tables or mem tables). SST tables are immutable after being written and mem tables are lock-free data structures supporting single writer and multiple readers. There is only one single major lock, the DB mutex (DBImpl.mutex_) protecting all the meta operations, including: + + + + * Increase or decrease reference counters of mem tables and SST tables + + + * Change and check meta data structures, before and after finishing compactions, flushes and new mem table creations + + + * Coordinating writers + + +This DB mutex used to be scalability bottleneck preventing us from scaling to more than 16 threads. To address the issue, we improved RocksDB in several ways. + +1. Consolidate reference counters and introduce "super version". For every read operation, mutex was acquired, and reference counters for each mem table and each SST table were increased. One such operation is not expensive but if you are building a high throughput server with lots of reads, the lock contention will become the bottleneck. This is especially true if you store all your data in RAM. + +To solve this problem, we created a meta-meta data structure called “[super version](https://reviews.facebook.net/rROCKSDB1fdb3f7dc60e96394e3e5b69a46ede5d67fb976c)”, which holds reference counters to all those mem table and SST tables, so that readers only need to increase the reference counters for this single data structure. In RocksDB, list of live mem tables and SST tables only changes infrequently, which would happen when new mem tables are created or flush/compaction happens. Now, at those times, a new super version is created with their reference counters increased. A super version lists live mem tables and SST tables so a reader only needs acquire the lock in order to find the latest super version and increase its reference counter. From the super version, the reader can find all the mem and SST tables which are safety accessible as long as the reader holds the reference count for the super version. + +2. We replace some reference counters to stc::atomic objects, so that decreasing reference count of an object usually doesn’t need to be inside the mutex any more. + +3. Make fetching super version and reference counting lock-free in read queries. After consolidating reference counting to one single super version and removing the locking for decreasing reference counts, in read case, we only acquire mutex for one thing: fetch the latest super version and increase the reference count for that (dereference the counter is done in an atomic decrease). We designed and implemented a (mostly) lock-free approach to do it. See [details](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Lei-Lockless-Get.pdf). We will write a separate blog post for that. + +4. Avoid disk I/O inside the mutex. As we know, each disk I/O to hard drives takes several milliseconds. It can be even longer if file system journal is involved or I/Os are queued. Even occasional disk I/O within mutex can cause huge performance outliers. +We identified in two situations, we might do disk I/O inside mutex and we removed them: +(1) Opening and closing transactional log files. We moved those operations out of the mutex. +(2) Information logging. In multiple places we write to logs within mutex. There is a chance that file write will wait for disk I/O to finish before finishing, even if fsync() is not issued, especially in EXT systems. We occasionally see 100+ milliseconds write() latency on EXT. Instead of removing those logging, we came up with a solution of delay logging. When inside mutex, instead of directly writing to the log file, we write to a log buffer, with the timing information. As soon as mutex is released, we flush the log buffer to log files. + +5. Reduce object creation inside the mutex. +Object creation can be slow because it involves malloc (in our case). Malloc sometimes is slow because it needs to lock some shared data structures. Allocating can also be slow because we sometimes do expensive operations in some of our classes' constructors. For these reasons, we try to reduce object creations inside the mutex. Here are two examples: + +(1) std::vector uses malloc inside. We introduced “[autovector](https://reviews.facebook.net/rROCKSDBc01676e46d3be08c3c140361ef1f5884f47d3b3c)” data structure, in which memory for first a few elements are pre-allocated as members of the autovector class. When an autovector is used as a stack variable, no malloc will be needed unless the pre-allocated buffer is used up. This autovector is quite useful for manipulating those meta data structures. Those meta operations are often locked inside DB mutex. + +(2) When building an iterator, we used to creating iterator of every live men table and SST table within the mutex and a merging iterator on top of them. Besides malloc, some of those iterators can be quite expensive to create, like sorting. Now, instead of doing that, we simply increase the reference counters of them, and release the mutex before creating any iterator. + +6. Deal with mutexes in LRU caches. +When I said there was only one single major lock, I was lying. In RocksDB, all LRU caches had exclusive mutexes within to protect writes to the LRU lists, which are done in both of read and write operations. LRU caches are used in block cache and table cache. Both of them are accessed more frequently than DB data structures. Lock contention of these two locks are as intense as the DB mutex. Even if LRU cache is sharded into ShardedLRUCache, we can still see lock contentions, especially table caches. We further address this issue in two way: +(1) Bypassing table caches. A table cache maintains list of SST table’s read handlers. Those handlers contain SST files’ descriptors, table metadata, and possibly data indexes, as well as bloom filters. When the table handler needs to be evicted based on LRU, those information is cleared. When the SST table needs to be read and its table handler is not in LRU cache, the table is opened and those metadata is loaded. In some cases, users want to tune the system in a way that table handler evictions should never happen. It is common for high-throughput, low-latency servers. We introduce a mode where table cache is bypassed in read queries. In this mode, all table handlers are cached and accessed directly, so there is no need to query and adjust table caches for reading the database. It is the users’ responsibility to reserve enough resource for it. This mode can be turned on by setting options.max_open_files=-1. + +(2) [New PlainTable format](//github.com/facebook/rocksdb/wiki/PlainTable-Format) (optimized for SST in ramfs/tmpfs) does not organize data by blocks. Data are located by memory addresses so no block cache is needed. + +With all of those improvements, lock contention is not a bottleneck anymore, which is shown in our [memory-only benchmark](https://github.com/facebook/rocksdb/wiki/RocksDB-In-Memory-Workload-Performance-Benchmarks) . Furthermore, lock contentions are not causing some huge (50 milliseconds+) latency outliers they used to cause. + +### Comments + +**[Lee Hounshell](lee@apsalar.com)** + +Please post an example of reading the same rocksdb concurrently. + +We are using the latest 3.0 rocksdb; however, when two separate processes +try and open the same rocksdb for reading, only one of the open requests +succeed. The other open always fails with “db/LOCK: Resource temporarily unavailable” So far we have not found an option that allows sharing the rocksdb for reads. An example would be most appreciated. + +**[Siying Dong](siying.d@fb.com)** + +Sorry for the delay. We don’t have feature support for this scenario yet. Here is an example you can work around this problem. You can build a snapshot of the DB by doing this: + +1. create a separate directory on the same host for a snapshot of the DB. +1. call `DB::DisableFileDeletions()` +1. call `DB::GetLiveFiles()` to get a full list of the files. +1. for all the files except manifest, add a hardlink file in your new directory pointing to the original file +1. copy the manifest file and truncate the size (you can read the comments of `DB::GetLiveFiles()` for more information) +1. call `DB::EnableFileDeletions()` +1. now you can open the snapshot directory in another process to access those files. Please remember to delete the directory after reading the data to allow those files to be recycled. + +By the way, the best way to ask those questions is in our [facebook group](https://www.facebook.com/groups/rocksdb.dev/). Let us know if you need any further help. + +**[Darshan](darshan.ghumare@gmail.com)** + +Will this consistency problem of RocksDB all occurs in case of single put/write? +What all ACID properties is supported by RocksDB, only durability irrespective of single or batch write? + +**[Siying Dong](siying.d@fb.com)** + +We recently [introduced optimistic transaction](https://reviews.facebook.net/D33435) which can help you ensure all of ACID. + +This blog post is mainly about optimizations in implementation. The RocksDB consistency semantic is not changed. diff --git a/docs/_posts/2014-05-19-rocksdb-3-0-release.markdown b/docs/_posts/2014-05-19-rocksdb-3-0-release.markdown new file mode 100644 index 00000000000..61c90dc936f --- /dev/null +++ b/docs/_posts/2014-05-19-rocksdb-3-0-release.markdown @@ -0,0 +1,24 @@ +--- +title: RocksDB 3.0 release +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/557/rocksdb-3-0-release/ +--- + +Check out new RocksDB release on [Github](https://github.com/facebook/rocksdb/releases/tag/3.0.fb)! + +New features in RocksDB 3.0: + + * [Column Family support](https://github.com/facebook/rocksdb/wiki/Column-Families) + + + * [Ability to chose different checksum function](https://github.com/facebook/rocksdb/commit/0afc8bc29a5800e3212388c327c750d32e31f3d6) + + + * Deprecated ReadOptions::prefix_seek and ReadOptions::prefix + + + +Check out the full [change log](https://github.com/facebook/rocksdb/blob/3.0.fb/HISTORY.md). diff --git a/docs/_posts/2014-05-22-rocksdb-3-1-release.markdown b/docs/_posts/2014-05-22-rocksdb-3-1-release.markdown new file mode 100644 index 00000000000..30156742b25 --- /dev/null +++ b/docs/_posts/2014-05-22-rocksdb-3-1-release.markdown @@ -0,0 +1,20 @@ +--- +title: RocksDB 3.1 release +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/575/rocksdb-3-1-release/ +--- + +Check out the new release on [Github](https://github.com/facebook/rocksdb/releases/tag/rocksdb-3.1)! + +New features in RocksDB 3.1: + + * [Materialized hash index](https://github.com/facebook/rocksdb/commit/0b3d03d026a7248e438341264b4c6df339edc1d7) + + + * [FIFO compaction style](https://github.com/facebook/rocksdb/wiki/FIFO-compaction-style) + + +We released 3.1 so fast after 3.0 because one of our internal customers needed materialized hash index. diff --git a/docs/_posts/2014-06-23-plaintable-a-new-file-format.markdown b/docs/_posts/2014-06-23-plaintable-a-new-file-format.markdown new file mode 100644 index 00000000000..6a641f23353 --- /dev/null +++ b/docs/_posts/2014-06-23-plaintable-a-new-file-format.markdown @@ -0,0 +1,47 @@ +--- +title: PlainTable — A New File Format +layout: post +author: sdong +category: blog +redirect_from: + - /blog/599/plaintable-a-new-file-format/ +--- + +In this post, we are introducing "PlainTable" -- a file format we designed for RocksDB, initially to satisfy a production use case at Facebook. + +Design goals: + +1. All data stored in memory, in files stored in tmpfs/ramfs. Support DBs larger than 100GB (may be sharded across multiple RocksDB instance). +1. Optimize for [prefix hashing](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Siying-Prefix-Hash.pdf) +1. Less than or around 1 micro-second average latency for single Get() or Seek(). +1. Minimize memory consumption. +1. Queries efficiently return empty results + + + +Notice that our priority was not to maximize query performance, but to strike a balance between query performance and memory consumption. PlainTable query performance is not as good as you would see with a nicely-designed hash table, but they are of the same order of magnitude, while keeping memory overhead to a minimum. + +Since we are targeting micro-second latency, it is on the level of the number of CPU cache misses (if they cannot be parallellized, which are usually the case for index look-ups). On our target hardware with Intel CPUs of multiple sockets with NUMA, we can only allow 4-5 CPU cache misses (including costs of data TLB). + +To meet our requirements, given that only hash prefix iterating is needed, we made two decisions: + +1. to use a hash index, which is +1. directly addressed to rows, with no block structure. + +Having addressed our latency goal, the next task was to design a very compact hash index to minimize memory consumption. Some tricks we used to meet this goal: + +1. We only use 32-bit integers for data and index offsets.The first bit serves as a flag, so we can avoid using 8-byte pointers. +1. We never copy keys or parts of keys to index search structures. We store only offsets from which keys can be retrieved, to make comparisons with search keys. +1. Since our file is immutable, we can accurately estimate the number of hash buckets needed. + +To make sure the format works efficiently with empty queries, we added a bloom filter check before the query. This adds only one cache miss for non-empty cases [1], but avoids multiple cache misses for most empty results queries. This is a good trade-off for use cases with a large percentage of empty results. + +These are the design goals and basic ideas of PlainTable file format. For detailed information, see [this wiki page](https://github.com/facebook/rocksdb/wiki/PlainTable-Format). + +[1] Bloom filter checks typically require multiple memory access. However, because they are independent, they usually do not make the CPU pipeline stale. In any case, we improved the bloom filter to improve data locality - we may cover this further in a future blog post. + +### Comments + +**[Siying Dong](siying.d@fb.com)** + +Does [http://rocksdb.org/feed/](http://rocksdb.org/feed/) work? diff --git a/docs/_posts/2014-06-27-avoid-expensive-locks-in-get.markdown b/docs/_posts/2014-06-27-avoid-expensive-locks-in-get.markdown new file mode 100644 index 00000000000..4411c7ae313 --- /dev/null +++ b/docs/_posts/2014-06-27-avoid-expensive-locks-in-get.markdown @@ -0,0 +1,89 @@ +--- +title: Avoid Expensive Locks in Get() +layout: post +author: leijin +category: blog +redirect_from: + - /blog/677/avoid-expensive-locks-in-get/ +--- + +As promised in the previous [blog post](blog/2014/05/14/lock.html)! + +RocksDB employs a multiversion concurrency control strategy. Before reading data, it needs to grab the current version, which is encapsulated in a data structure called [SuperVersion](https://reviews.facebook.net/rROCKSDB1fdb3f7dc60e96394e3e5b69a46ede5d67fb976c). + + + +At the beginning of `GetImpl()`, it used to do this: + + + mutex_.Lock(); + auto* s = super_version_->Ref(); + mutex_.Unlock(); + + +The lock is necessary because pointer super_version_ may be updated, the corresponding SuperVersion may be deleted while Ref() is in progress. + + +`Ref()` simply increases the reference counter and returns “this” pointer. However, this simple operation posed big challenges for in-memory workload and stopped RocksDB from scaling read throughput beyond 8 cores. Running 32 read threads on a 32-core CPU leads to [70% system CPU usage](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Lei-Lockless-Get.pdf). This is outrageous! + + + + +Luckily, we found a way to circumvent this problem by using [thread local storage](http://en.wikipedia.org/wiki/Thread-local_storage). Version change is a rare event comparable to millions of read requests. On the very first Get() request, each thread pays the mutex cost to acquire a reference to the new super version. Instead of releasing the reference after use, the reference is cached in thread’s local storage. An atomic variable is used to track global super version number. Subsequent reads simply compare the local super version number against the global super version number. If they are the same, the cached super version reference may be used directly, at no cost. If a version change is detected, mutex must be acquired to update the reference. The cost of mutex lock is amortized among millions of reads and becomes negligible. + + + + +The code looks something like this: + + + + + + SuperVersion* s = thread_local_->Get(); + if (s->version_number != super_version_number_.load()) { + // slow path, cleanup of current super version is omitted + mutex_.Lock(); + s = super_version_->Ref(); + mutex_.Unlock(); + } + + + + +The result is quite amazing. RocksDB can nicely [scale to 32 cores](https://github.com/facebook/rocksdb/raw/gh-pages/talks/2014-03-27-RocksDB-Meetup-Lei-Lockless-Get.pdf) and most CPU time is spent in user land. + + + + +Daryl Grove gives a pretty good [comparison between mutex and atomic](https://blogs.oracle.com/d/entry/the_cost_of_mutexes). However, the real cost difference lies beyond what is shown in the assembly code. Mutex can keep threads spinning on CPU or even trigger thread context switches in which all readers compete to access the critical area. Our approach prevents mutual competition by directing threads to check against a global version which does not change at high frequency, and is therefore much more cache-friendly. + + + + +The new approach entails one issue: a thread can visit GetImpl() once but can never come back again. SuperVersion is referenced and cached in its thread local storage. All resources (e.g., memtables, files) which belong to that version are frozen. A “supervisor” is required to visit each thread’s local storage and free its resources without incurring a lock. We designed a lockless sweep using CAS (compare and switch instruction). Here is how it works: + + + + +(1) A reader thread uses CAS to acquire SuperVersion from its local storage and to put in a special flag (SuperVersion::kSVInUse). + + + + +(2) Upon completion of GetImpl(), the reader thread tries to return SuperVersion to local storage by CAS, expecting the special flag (SuperVersion::kSVInUse) in its local storage. If it does not see SuperVersion::kSVInUse, that means a “sweep” was done and the reader thread is responsible for cleanup (this is expensive, but does not happen often on the hot path). + + + + +(3) After any flush/compaction, the background thread performs a sweep (CAS) across all threads’ local storage and frees encountered SuperVersion. A reader thread must re-acquire a new SuperVersion reference on its next visit. + +### Comments + +**[David Barbour](dmbarbour@gmail.com)** + +Please post an example of reading the same rocksdb concurrently. + +We are using the latest 3.0 rocksdb; however, when two separate processes +try and open the same rocksdb for reading, only one of the open requests +succeed. The other open always fails with “db/LOCK: Resource temporarily unavailable” So far we have not found an option that allows sharing the rocksdb for reads. An example would be most appreciated. diff --git a/docs/_posts/2014-06-27-rocksdb-3-2-release.markdown b/docs/_posts/2014-06-27-rocksdb-3-2-release.markdown new file mode 100644 index 00000000000..e4eba6af4be --- /dev/null +++ b/docs/_posts/2014-06-27-rocksdb-3-2-release.markdown @@ -0,0 +1,30 @@ +--- +title: RocksDB 3.2 release +layout: post +author: leijin +category: blog +redirect_from: + - /blog/647/rocksdb-3-2-release/ +--- + +Check out new RocksDB release on [GitHub](https://github.com/facebook/rocksdb/releases/tag/rocksdb-3.2)! + +New Features in RocksDB 3.2: + + * PlainTable now supports a new key encoding: for keys of the same prefix, the prefix is only written once. It can be enabled through encoding_type paramter of NewPlainTableFactory() + + + * Add AdaptiveTableFactory, which is used to convert from a DB of PlainTable to BlockBasedTabe, or vise versa. It can be created using NewAdaptiveTableFactory() + + + +Public API changes: + + + * We removed seek compaction as a concept from RocksDB + + + * Add two paramters to NewHashLinkListRepFactory() for logging on too many entries in a hash bucket when flushing + + + * Added new option BlockBasedTableOptions::hash_index_allow_collision. When enabled, prefix hash index for block-based table will not store prefix and allow hash collision, reducing memory consumption diff --git a/docs/_posts/2014-07-29-rocksdb-3-3-release.markdown b/docs/_posts/2014-07-29-rocksdb-3-3-release.markdown new file mode 100644 index 00000000000..d858e4fafe4 --- /dev/null +++ b/docs/_posts/2014-07-29-rocksdb-3-3-release.markdown @@ -0,0 +1,34 @@ +--- +title: RocksDB 3.3 Release +layout: post +author: yhciang +category: blog +redirect_from: + - /blog/1301/rocksdb-3-3-release/ +--- + +Check out new RocksDB release on [GitHub](https://github.com/facebook/rocksdb/releases/tag/rocksdb-3.3)! + +New Features in RocksDB 3.3: + + * **JSON API prototype**. + + + * **Performance improvement on HashLinkList**: We addressed performance outlier of HashLinkList caused by skewed bucket by switching data in the bucket from linked list to skip list. Add parameter threshold_use_skiplist in NewHashLinkListRepFactory(). + + + + * **More effective on storage space reclaim**: RocksDB is now able to reclaim storage space more effectively during the compaction process. This is done by compensating the size of each deletion entry by the 2X average value size, which makes compaction to be triggerred by deletion entries more easily. + + + * **TimeOut API to write**: Now WriteOptions have a variable called timeout_hint_us. With timeout_hint_us set to non-zero, any write associated with this timeout_hint_us may be aborted when it runs longer than the specified timeout_hint_us, and it is guaranteed that any write completes earlier than the specified time-out will not be aborted due to the time-out condition. + + + * **rate_limiter option**: We added an option that controls total throughput of flush and compaction. The throughput is specified in bytes/sec. Flush always has precedence over compaction when available bandwidth is constrained. + + + +Public API changes: + + + * Removed NewTotalOrderPlainTableFactory because it is not used and implemented semantically incorrect. diff --git a/docs/_posts/2014-09-12-cuckoo.markdown b/docs/_posts/2014-09-12-cuckoo.markdown new file mode 100644 index 00000000000..22178f7cac5 --- /dev/null +++ b/docs/_posts/2014-09-12-cuckoo.markdown @@ -0,0 +1,74 @@ +--- +title: Cuckoo Hashing Table Format +layout: post +author: radheshyam +category: blog +redirect_from: + - /blog/1427/new-bloom-filter-format/ +--- + +## Introduction + +We recently introduced a new [Cuckoo Hashing](http://en.wikipedia.org/wiki/Cuckoo_hashing) based SST file format which is optimized for fast point lookups. The new format was built for applications which require very high point lookup rates (~4Mqps) in read only mode but do not use operations like range scan, merge operator, etc. But, the existing RocksDB file formats were built to support range scan and other operations and the current best point lookup in RocksDB is 1.2 Mqps given by [PlainTable](https://github.com/facebook/rocksdb/wiki/PlainTable-Format)[ format](https://github.com/facebook/rocksdb/wiki/PlainTable-Format). This prompted a hashing based file format, which we present here. The new table format uses a cache friendly version of Cuckoo Hashing algorithm with only 1 or 2 memory accesses per lookup. + + + +Goals: + + * Reduce memory accesses per lookup to 1 or 2 + + + * Get an end to end point lookup rate of at least 4 Mqps + + + * Minimize database size + + +Assumptions: + + * Key length and value length are fixed + + + * The database is operated in read only mode + + +Non-goals: + + + * While optimizing the performance of Get() operation was our primary goal, compaction and build times were secondary. We may work on improving them in future. + + +Details for setting up the table format can be found in [GitHub](https://github.com/facebook/rocksdb/wiki/CuckooTable-Format). + + +## Cuckoo Hashing Algorithm + +In order to achieve high lookup speeds, we did multiple optimizations, including a cache friendly cuckoo hash algorithm. Cuckoo Hashing uses multiple hash functions, _h1, ..., __hn._ + +### Original Cuckoo Hashing + +To insert any new key _k_, we compute hashes of the key _h1(k), ..., __hn__(k)_. We insert the key in the first hash location that is free. If all the locations are blocked, we try to move one of the colliding keys to a different location by trying to re-insert it. + +Finding smallest set of keys to displace in order to accommodate the new key is naturally a shortest path problem in a directed graph where nodes are buckets of hash table and there is an edge from bucket _A_ to bucket _B_ if the element stored in bucket _A_ can be accommodated in bucket _B_ using one of the hash functions. The source nodes are the possible hash locations for the given key _k_ and destination is any one of the empty buckets. We use this algorithm to handle collision. + +To retrieve a key _k_, we compute hashes, _h1(k), ..., __hn__(k)_ and the key must be present in one of these locations. + +Our goal is to minimize average (and maximum) number of hash functions required and hence the number of memory accesses. In our experiments, with a hash utilization of 90%, we found that the average number of lookups is 1.8 and maximum is 3. Around 44% of keys are accommodated in first hash location and 33% in second location. + + +### Cache Friendly Cuckoo Hashing + +We noticed the following two sub-optimal properties in original Cuckoo implementation: + + + * If the key is not present in first hash location, we jump to second hash location which may not be in cache. This results in many cache misses. + + + * Because only 44% of keys are located in first cuckoo block, we couldn't have an optimal prefetching strategy - prefetching all hash locations for a key is wasteful. But prefetching only the first hash location helps only 44% of cases. + + + +The solution is to insert more keys near first location. In case of collision in the first hash location - _h1(k)_, we try to insert it in next few buckets, _h1(k)+1, _h1(k)+2, _..., h1(k)+t-1_. If all of these _t_ locations are occupied, we skip over to next hash function _h2_ and repeat the process. We call the set of _t_ buckets as a _Cuckoo Block_. We chose _t_ such that size of a block is not bigger than a cache line and we prefetch the first cuckoo block. + + +With the new algorithm, for 90% hash utilization, we found that 85% of keys are accommodated in first Cuckoo Block. Prefetching the first cuckoo block yields best results. For a database of 100 million keys with key length 8 and value length 4, the hash algorithm alone can achieve 9.6 Mqps and we are working on improving it further. End to end RocksDB performance results can be found [here](https://github.com/facebook/rocksdb/wiki/CuckooTable-Format). diff --git a/docs/_posts/2014-09-12-new-bloom-filter-format.markdown b/docs/_posts/2014-09-12-new-bloom-filter-format.markdown new file mode 100644 index 00000000000..96fa50a401f --- /dev/null +++ b/docs/_posts/2014-09-12-new-bloom-filter-format.markdown @@ -0,0 +1,52 @@ +--- +title: New Bloom Filter Format +layout: post +author: zagfox +category: blog +redirect_from: + - /blog/1367/cuckoo/ +--- + +## Introduction + +In this post, we are introducing "full filter block" --- a new bloom filter format for [block based table](https://github.com/facebook/rocksdb/wiki/Rocksdb-BlockBasedTable-Format). This could bring about 40% of improvement for key query under in-memory (all data stored in memory, files stored in tmpfs/ramfs, an [example](https://github.com/facebook/rocksdb/wiki/RocksDB-In-Memory-Workload-Performance-Benchmarks) workload. The main idea behind is to generate a big filter that covers all the keys in SST file to avoid lots of unnecessary memory look ups. + + + + +## What is Bloom Filter + +In brief, [bloom filter](https://github.com/facebook/rocksdb/wiki/RocksDB-Bloom-Filter) is a bits array generated for a set of keys that could tell if an arbitrary key may exist in that set. + +In RocksDB, we generate such a bloom filter for each SST file. When we conduct a query for a key, we first goes to the bloom filter block of SST file. If key may exist in filter, we goes into data block in SST file to search for the key. If not, we would return directly. So it could help speed up point look up operation a lot. + +## Original Bloom Filter Format + +Original bloom filter creates filters for each individual data block in SST file. It has complex structure (ref [here](https://github.com/facebook/rocksdb/wiki/Rocksdb-BlockBasedTable-Format#filter-meta-block)) which results in a lot of non-adjacent memory look ups. + +Here's the work flow for checking original bloom filter in block based table: + +1. Given the target key, we goes to the index block to get the "data block ID" where this key may reside. +1. Using the "data block ID", we goes to the filter block and get the correct "offset of filter". +1. Using the "offset of filter", we goes to the actual filter and do the checking. + +## New Bloom Filter Format + +New bloom filter creates filter for all keys in SST file and we name it "full filter". The data structure of full filter is very simple, there is just one big filter: + +    [ full filter ] + +In this way, the work flow of bloom filter checking is much simplified. + +(1) Given the target key, we goes directly to the filter block and conduct the filter checking. + +To be specific, there would be no checking for index block and no address jumping inside of filter block. + +Though it is a big filter, the total filter size would be the same as the original filter. + +One little draw back is that the new bloom filter introduces more memory consumption when building SST file because we need to buffer keys (or their hashes) before generating filter. Original filter just creates a bunch of small filters so it just buffer a small amount of keys. For full filter, we buffer hashes of all keys, which would take more memory when SST file size increases. + + +## Usage & Customization + +You can refer to the document here for [usage](https://github.com/facebook/rocksdb/wiki/RocksDB-Bloom-Filter#usage-of-new-bloom-filter) and [customization](https://github.com/facebook/rocksdb/wiki/RocksDB-Bloom-Filter#customize-your-own-filterpolicy). diff --git a/docs/_posts/2014-09-15-rocksdb-3-5-release.markdown b/docs/_posts/2014-09-15-rocksdb-3-5-release.markdown new file mode 100644 index 00000000000..1878a5a5673 --- /dev/null +++ b/docs/_posts/2014-09-15-rocksdb-3-5-release.markdown @@ -0,0 +1,38 @@ +--- +title: RocksDB 3.5 Release! +layout: post +author: leijin +category: blog +redirect_from: + - /blog/1547/rocksdb-3-5-release/ +--- + +New RocksDB release - 3.5! + + +**New Features** + + + 1. Add include/utilities/write_batch_with_index.h, providing a utility class to query data out of WriteBatch when building it. + + + 2. new ReadOptions.total_order_seek to force total order seek when block-based table is built with hash index. + + + +**Public API changes** + + + 1. The Prefix Extractor used with V2 compaction filters is now passed user key to SliceTransform::Transform instead of unparsed RocksDB key. + + + 2. Move BlockBasedTable related options to BlockBasedTableOptions from Options. Change corresponding JNI interface. Options affected include: no_block_cache, block_cache, block_cache_compressed, block_size, block_size_deviation, block_restart_interval, filter_policy, whole_key_filtering. filter_policy is changed to shared_ptr from a raw pointer. + + + 3. Remove deprecated options: disable_seek_compaction and db_stats_log_interval + + + 4. OptimizeForPointLookup() takes one parameter for block cache size. It now builds hash index, bloom filter, and block cache. + + +[https://github.com/facebook/rocksdb/releases/tag/v3.5](https://github.com/facebook/rocksdb/releases/tag/rocksdb-3.5) diff --git a/docs/_posts/2015-01-16-migrating-from-leveldb-to-rocksdb-2.markdown b/docs/_posts/2015-01-16-migrating-from-leveldb-to-rocksdb-2.markdown new file mode 100644 index 00000000000..f18de0bbc39 --- /dev/null +++ b/docs/_posts/2015-01-16-migrating-from-leveldb-to-rocksdb-2.markdown @@ -0,0 +1,112 @@ +--- +title: Migrating from LevelDB to RocksDB +layout: post +author: lgalanis +category: blog +redirect_from: + - /blog/1811/migrating-from-leveldb-to-rocksdb-2/ +--- + +If you have an existing application that uses LevelDB and would like to migrate to using RocksDB, one problem you need to overcome is to map the options for LevelDB to proper options for RocksDB. As of release 3.9 this can be automatically done by using our option conversion utility found in rocksdb/utilities/leveldb_options.h. What is needed, is to first replace `leveldb::Options` with `rocksdb::LevelDBOptions`. Then, use `rocksdb::ConvertOptions( )` to convert the `LevelDBOptions` struct into appropriate RocksDB options. Here is an example: + + + +LevelDB code: + +```c++ +#include +#include "leveldb/db.h" + +using namespace leveldb; + +int main(int argc, char** argv) { + DB *db; + + Options opt; + opt.create_if_missing = true; + opt.max_open_files = 1000; + opt.block_size = 4096; + + Status s = DB::Open(opt, "/tmp/mydb", &db); + + delete db; +} +``` + +RocksDB code: + +```c++ +#include +#include "rocksdb/db.h" +#include "rocksdb/utilities/leveldb_options.h" + +using namespace rocksdb; + +int main(int argc, char** argv) { + DB *db; + + LevelDBOptions opt; + opt.create_if_missing = true; + opt.max_open_files = 1000; + opt.block_size = 4096; + + Options rocksdb_options = ConvertOptions(opt); + // add rocksdb specific options here + + Status s = DB::Open(rocksdb_options, "/tmp/mydb_rocks", &db); + + delete db; +} +``` + +The difference is: + +```diff +-#include "leveldb/db.h" ++#include "rocksdb/db.h" ++#include "rocksdb/utilities/leveldb_options.h" + +-using namespace leveldb; ++using namespace rocksdb; + +- Options opt; ++ LevelDBOptions opt; + +- Status s = DB::Open(opt, "/tmp/mydb", &db); ++ Options rocksdb_options = ConvertOptions(opt); ++ // add rockdb specific options here ++ ++ Status s = DB::Open(rocksdb_options, "/tmp/mydb_rocks", &db); +``` + +Once you get up and running with RocksDB you can then focus on tuning RocksDB further by modifying the converted options struct. + +The reason why ConvertOptions is handy is because a lot of individual options in RocksDB have moved to other structures in different components. For example, block_size is not available in struct rocksdb::Options. It resides in struct rocksdb::BlockBasedTableOptions, which is used to create a TableFactory object that RocksDB uses internally to create the proper TableBuilder objects. If you were to write your application from scratch it would look like this: + +RocksDB code from scratch: + +```c++ +#include +#include "rocksdb/db.h" +#include "rocksdb/table.h" + +using namespace rocksdb; + +int main(int argc, char** argv) { + DB *db; + + Options opt; + opt.create_if_missing = true; + opt.max_open_files = 1000; + + BlockBasedTableOptions topt; + topt.block_size = 4096; + opt.table_factory.reset(NewBlockBasedTableFactory(topt)); + + Status s = DB::Open(opt, "/tmp/mydb_rocks", &db); + + delete db; +} +``` + +The LevelDBOptions utility can ease migration to RocksDB from LevelDB and allows us to break down the various options across classes as it is needed. diff --git a/docs/_posts/2015-02-24-reading-rocksdb-options-from-a-file.markdown b/docs/_posts/2015-02-24-reading-rocksdb-options-from-a-file.markdown new file mode 100644 index 00000000000..cddc0dd01fb --- /dev/null +++ b/docs/_posts/2015-02-24-reading-rocksdb-options-from-a-file.markdown @@ -0,0 +1,41 @@ +--- +title: Reading RocksDB options from a file +layout: post +author: lgalanis +category: blog +redirect_from: + - /blog/1883/reading-rocksdb-options-from-a-file/ +--- + +RocksDB options can be provided using a file or any string to RocksDB. The format is straightforward: `write_buffer_size=1024;max_write_buffer_number=2`. Any whitespace around `=` and `;` is OK. Moreover, options can be nested as necessary. For example `BlockBasedTableOptions` can be nested as follows: `write_buffer_size=1024; max_write_buffer_number=2; block_based_table_factory={block_size=4k};`. Similarly any white space around `{` or `}` is ok. Here is what it looks like in code: + + + +```c++ +#include +#include "rocksdb/db.h" +#include "rocksdb/table.h" +#include "rocksdb/utilities/convenience.h" + +using namespace rocksdb; + +int main(int argc, char** argv) { + DB *db; + + Options opt; + + std::string options_string = + "create_if_missing=true;max_open_files=1000;" + "block_based_table_factory={block_size=4096}"; + + Status s = GetDBOptionsFromString(opt, options_string, &opt); + + s = DB::Open(opt, "/tmp/mydb_rocks", &db); + + // use db + + delete db; +} +``` + +Using `GetDBOptionsFromString` is a convenient way of changing options for your RocksDB application without needing to resort to recompilation or tedious command line parsing. diff --git a/docs/_posts/2015-02-27-write-batch-with-index.markdown b/docs/_posts/2015-02-27-write-batch-with-index.markdown new file mode 100644 index 00000000000..7f9f7765365 --- /dev/null +++ b/docs/_posts/2015-02-27-write-batch-with-index.markdown @@ -0,0 +1,20 @@ +--- +title: 'WriteBatchWithIndex: Utility for Implementing Read-Your-Own-Writes' +layout: post +author: sdong +category: blog +redirect_from: + - /blog/1901/write-batch-with-index/ +--- + +RocksDB can be used as a storage engine of a higher level database. In fact, we are currently plugging RocksDB into MySQL and MongoDB as one of their storage engines. RocksDB can help with guaranteeing some of the ACID properties: durability is guaranteed by RocksDB by design; while consistency and isolation need to be enforced by concurrency controls on top of RocksDB; Atomicity can be implemented by committing a transaction's writes with one write batch to RocksDB in the end. + + + +However, if we enforce atomicity by only committing all writes in the end of the transaction in one batch, you cannot get the updated value from RocksDB previously written by the same transaction (read-your-own-write). To read the updated value, the databases on top of RocksDB need to maintain an internal buffer for all the written keys, and when a read happens they need to merge the result from RocksDB and from this buffer. This is a problem we faced when building the RocksDB storage engine in MongoDB. We solved it by creating a utility class, WriteBatchWithIndex (a write batch with a searchable index) and made it part of public API so that the community can also benefit from it. + +Before talking about the index part, let me introduce write batch first. The write batch class, `WriteBatch`, is a RocksDB data structure for atomic writes of multiple keys. Users can buffer their updates to a `WriteBatch` by calling `write_batch.Put("key1", "value1")` or `write_batch.Delete("key2")`, similar as calling RocksDB's functions of the same names. In the end, they call `db->Write(write_batch)` to atomically update all those batched operations to the DB. It is how a database can guarantee atomicity, as shown above. Adding a searchable index to `WriteBatch`, we now have `WriteBatchWithIndex`. Users can put updates to WriteBatchIndex in the same way as to `WriteBatch`. In the end, users can get a `WriteBatch` object from it and issue `db->Write()`. Additionally, users can create an iterator of a WriteBatchWithIndex, seek to any key location and iterate from there. + +To implement read-your-own-write using `WriteBatchWithIndex`, every time the user creates a transaction, we create a `WriteBatchWithIndex` attached to it. All the writes of the transaction go to the `WriteBatchWithIndex` first. When we commit the transaction, we atomically write the batch to RocksDB. When the user wants to call `Get()`, we first check if the value exists in the `WriteBatchWithIndex` and return the value if existing, by seeking and reading from an iterator of the write batch, before checking data in RocksDB. For example, here is the we implement it in MongoDB's RocksDB storage engine: [link](https://github.com/mongodb/mongo/blob/a31cc114a89a3645e97645805ba77db32c433dce/src/mongo/db/storage/rocks/rocks_recovery_unit.cpp#L245-L260). If a range query comes, we pass a DB's iterator to `WriteBatchWithIndex`, which creates a super iterator which combines the results from the DB iterator with the batch's iterator. Using this super iterator, we can iterate the DB with the transaction's own writes. Here is the iterator creation codes in MongoDB's RocksDB storage engine: [link](https://github.com/mongodb/mongo/blob/a31cc114a89a3645e97645805ba77db32c433dce/src/mongo/db/storage/rocks/rocks_recovery_unit.cpp#L266-L269). In this way, the database can solve the read-your-own-write problem by using RocksDB to handle a transaction's uncommitted writes. + +Using `WriteBatchWithIndex`, we successfully implemented read-your-own-writes in the RocksDB storage engine of MongoDB. If you also have a read-your-own-write problem, `WriteBatchWithIndex` can help you implement it quickly and correctly. diff --git a/docs/_posts/2015-04-22-integrating-rocksdb-with-mongodb-2.markdown b/docs/_posts/2015-04-22-integrating-rocksdb-with-mongodb-2.markdown new file mode 100644 index 00000000000..1ffe2c532ec --- /dev/null +++ b/docs/_posts/2015-04-22-integrating-rocksdb-with-mongodb-2.markdown @@ -0,0 +1,16 @@ +--- +title: Integrating RocksDB with MongoDB +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/1967/integrating-rocksdb-with-mongodb-2/ +--- + +Over the last couple of years, we have been busy integrating RocksDB with various services here at Facebook that needed to store key-value pairs locally. We have also seen other companies using RocksDB as local storage components of their distributed systems. + + + +The next big challenge for us is to bring RocksDB storage engine to general purpose databases. Today we have an exciting milestone to share with our community! We're running MongoDB with RocksDB in production and seeing great results! You can read more about it here: [http://blog.parse.com/announcements/mongodb-rocksdb-parse/](http://blog.parse.com/announcements/mongodb-rocksdb-parse/) + +Keep tuned for benchmarks and more stability and performance improvements. diff --git a/docs/_posts/2015-06-12-rocksdb-in-osquery.markdown b/docs/_posts/2015-06-12-rocksdb-in-osquery.markdown new file mode 100644 index 00000000000..f3a55faae17 --- /dev/null +++ b/docs/_posts/2015-06-12-rocksdb-in-osquery.markdown @@ -0,0 +1,10 @@ +--- +title: RocksDB in osquery +layout: post +author: icanadi +category: lgalanis +redirect_from: + - /blog/1997/rocksdb-in-osquery/ +--- + +Check out [this](https://code.facebook.com/posts/1411870269134471/how-rocksdb-is-used-in-osquery/) blog post by [Mike Arpaia](https://www.facebook.com/mike.arpaia) and [Ted Reed](https://www.facebook.com/treeded) about how osquery leverages RocksDB to build an embedded pub-sub system. This article is a great read and contains insights on how to properly use RocksDB. diff --git a/docs/_posts/2015-07-15-rocksdb-2015-h2-roadmap.markdown b/docs/_posts/2015-07-15-rocksdb-2015-h2-roadmap.markdown new file mode 100644 index 00000000000..b3e2703fc61 --- /dev/null +++ b/docs/_posts/2015-07-15-rocksdb-2015-h2-roadmap.markdown @@ -0,0 +1,92 @@ +--- +title: RocksDB 2015 H2 roadmap +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/2015/rocksdb-2015-h2-roadmap/ +--- + +Every 6 months, RocksDB team gets together to prioritize the work ahead of us. We just went through this exercise and we wanted to share the results with the community. Here's what RocksDB team will be focusing on for the next 6 months: + + + +**MyRocks** + +As you might know, we're working hard to integrate RocksDB as a storage engine for MySQL. This project is pretty important for us because we're heavy users of MySQL. We're already getting pretty good performance results, but there is more work to be done. We need to focus on both performance and stability. The most high priority items on are list are: + + + + + 1. Reduce CPU costs of RocksDB as a MySQL storage engine + + + 2. Implement pessimistic concurrency control to support repeatable read isolation level in MyRocks + + + 3. Reduce P99 read latency, which is high mostly because of lingering tombstones + + + 4. Port ZSTD compression + + +**MongoRocks** + +Another database that we're working on is MongoDB. The project of integrating MongoDB with RocksDB storage engine is called MongoRocks. It's already running in production at Parse [1] and we're seeing surprisingly few issues. Our plans for the next half: + + + + + 1. Keep improving performance and stability, possibly reuse work done on MyRocks (workloads are pretty similar). + + + 2. Increase internal and external adoption. + + + 3. Support new MongoDB 3.2. + + +**RocksDB on cheaper storage media** + +Up to now, our mission was to build the best key-value store “for fast storage” (flash and in-memory). However, there are some use-cases at Facebook that don't need expensive high-end storage. In the next six months, we plan to deploy RocksDB on cheaper storage media. We will optimize performance to RocksDB on either or both: + + + + + 1. Hard drive storage array. + + + 2. Tiered Storage. + + +**Quality of Service** + +When talking to our customers, there are couple of issues that keep reoccurring. We need to fix them to make our customers happy. We will improve RocksDB to provide better assurance of performance and resource usage. Non-exhaustive list includes: + + + + + 1. Iterate P99 can be high due to the presence of tombstones. + + + 2. Write stalls can happen during high write loads. + + + 3. Better control of memory and disk usage. + + + 4. Service quality and performance of backup engine. + + +**Operation's user experience** + +As we increase deployment of RocksDB, engineers are spending more time on debugging RocksDB issues. We plan to improve user experience when running RocksDB. The goal is to reduce TTD (time-to-debug). The work includes monitoring, visualizations and documentations. + +[1]( http://blog.parse.com/announcements/mongodb-rocksdb-parse/](http://blog.parse.com/announcements/mongodb-rocksdb-parse/) + + +### Comments + +**[Mike](allspace2012@outlook.com)** + +What’s the status of this roadmap? “RocksDB on cheaper storage media”, has this been implemented? diff --git a/docs/_posts/2015-07-17-spatial-indexing-in-rocksdb.markdown b/docs/_posts/2015-07-17-spatial-indexing-in-rocksdb.markdown new file mode 100644 index 00000000000..fe7b7b26819 --- /dev/null +++ b/docs/_posts/2015-07-17-spatial-indexing-in-rocksdb.markdown @@ -0,0 +1,78 @@ +--- +title: Spatial indexing in RocksDB +layout: post +author: icanadi +category: blog +redirect_from: + - /blog/2039/spatial-indexing-in-rocksdb/ +--- + +About a year ago, there was a need to develop a spatial database at Facebook. We needed to store and index Earth's map data. Before building our own, we looked at the existing spatial databases. They were all very good technology, but also general purpose. We could sacrifice a general-purpose API, so we thought we could build a more performant database, since it would be specifically designed for our use-case. Furthermore, we decided to build the spatial database on top of RocksDB, because we have a lot of operational experience with running and tuning RocksDB at a large scale. + + + +When we started looking at this project, the first thing that surprised us was that our planet is not that big. Earth's entire map data can fit in memory on a reasonably high-end machine. Thus, we also decided to build a spatial database optimized for memory-resident dataset. + +The first use-case of our spatial database was an experimental map renderer. As part of our project, we successfully loaded [Open Street Maps](https://www.openstreetmap.org/) dataset and hooked it up with [Mapnik](http://mapnik.org/), a map rendering engine. + +The usual Mapnik workflow is to load the map data into a SQL-based database and then define map layers with SQL statements. To render a tile, Mapnik needs to execute a couple of SQL queries. The benefit of this approach is that you don't need to reload your database when you change your map style. You can just change your SQL query and Mapnik picks it up. In our model, we decided to precompute the features we need for each tile. We need to know the map style before we create the database. However, when rendering the map tile, we only fetch the features that we need to render. + +We haven't open sourced the RocksDB Mapnik plugin or the database loading pipeline. However, the spatial indexing is available in RocksDB under a name [SpatialDB](https://github.com/facebook/rocksdb/blob/master/include/rocksdb/utilities/spatial_db.h). The API is focused on map rendering use-case, but we hope that it can also be used for other spatial-based applications. + +Let's take a tour of the API. When you create a spatial database, you specify the spatial indexes that need to be built. Each spatial index is defined by a bounding box and granularity. For map rendering, we create a spatial index for each zoom levels. Higher zoom levels have more granularity. + + + + SpatialDB::Create( + SpatialDBOptions(), + "/data/map", { + SpatialIndexOptions("zoom10", BoundingBox(0, 0, 100, 100), 10), + SpatialIndexOptions("zoom16", BoundingBox(0, 0, 100, 100), 16) + } + ); + + + + +When you insert a feature (building, street, country border) into SpatialDB, you need to specify the list of spatial indexes that will index the feature. In the loading phase we process the map style to determine the list of zoom levels on which we'll render the feature. For example, we will not render the building on zoom level that shows an entire country. Building will only be indexed on higher zoom level's index. Country borders will be indexes on all zoom levels. + + + + FeatureSet feature; + feature.Set("type", "building"); + feature.Set("height", 6); + db->Insert(WriteOptions(), BoundingBox(5, 5, 10, 10), + well_known_binary_blob, feature, {"zoom16"}); + + + + +The indexing part is pretty simple. For each feature, we first find a list of index tiles that it intersects. Then, we add a link from the tile's [quad key](https://msdn.microsoft.com/en-us/library/bb259689.aspx) to the feature's primary key. Using quad keys improves data locality, i.e. features closer together geographically will have similar quad keys. Even though we're optimizing for a memory-resident dataset, data locality is still very important due to different caching effects. + +After you're done inserting all the features, you can call an API Compact() that will compact the dataset and speed up read queries. + + + + db->Compact(); + + + + +SpatialDB's query specifies: 1) bounding box we're interested in, and 2) a zoom level. We find all tiles that intersect with the query's bounding box and return all features in those tiles. + + + + + Cursor* c = db_->Query(ReadOptions(), BoundingBox(1, 1, 7, 7), "zoom16"); + for (c->Valid(); c->Next()) { + Render(c->blob(), c->feature_set()); + } + + + + +Note: `Render()` function is not part of RocksDB. You will need to use one of many open source map renderers, for example check out [Mapnik](http://mapnik.org/). + +TL;DR If you need an embedded spatial database, check out RocksDB's SpatialDB. [Let us know](https://www.facebook.com/groups/rocksdb.dev/) how we can make it better. + +If you're interested in learning more, check out this [talk](https://www.youtube.com/watch?v=T1jWsDMONM8). diff --git a/docs/_posts/2015-07-22-rocksdb-is-now-available-in-windows-platform.markdown b/docs/_posts/2015-07-22-rocksdb-is-now-available-in-windows-platform.markdown new file mode 100644 index 00000000000..b6bb47d5332 --- /dev/null +++ b/docs/_posts/2015-07-22-rocksdb-is-now-available-in-windows-platform.markdown @@ -0,0 +1,30 @@ +--- +title: RocksDB is now available in Windows Platform +layout: post +author: dmitrism +category: blog +redirect_from: + - /blog/2033/rocksdb-is-now-available-in-windows-platform/ +--- + +Over the past 6 months we have seen a number of use cases where RocksDB is successfully used by the community and various companies to achieve high throughput and volume in a modern server environment. + +We at Microsoft Bing could not be left behind. As a result we are happy to [announce](http://bit.ly/1OmWBT9) the availability of the Windows Port created here at Microsoft which we intend to use as a storage option for one of our key/value data stores. + + + +We are happy to make this available for the community. Keep tuned for more announcements to come. + +### Comments + +**[Siying Dong](siying.d@fb.com)** + +Appreciate your contributions to RocksDB project! I believe it will benefits many users! + +**[empresas sevilla](oxofkx@gmail.com)** + +Magnifico artículo|, un placer leer el blog + +**[jak usunac](tomogedac@o2.pl)** + +I believe it will benefits too diff --git a/docs/_posts/2015-07-23-dynamic-level.markdown b/docs/_posts/2015-07-23-dynamic-level.markdown new file mode 100644 index 00000000000..0ff3a0542f8 --- /dev/null +++ b/docs/_posts/2015-07-23-dynamic-level.markdown @@ -0,0 +1,29 @@ +--- +title: Dynamic Level Size for Level-Based Compaction +layout: post +author: sdong +category: blog +redirect_from: + - /blog/2207/dynamic-level/ +--- + +In this article, we follow up on the first part of an answer to one of the questions in our [AMA](https://www.reddit.com/r/IAmA/comments/3de3cv/we_are_rocksdb_engineering_team_ask_us_anything/ct4a8tb), the dynamic level size in level-based compaction. + + + +Level-based compaction is the original LevelDB compaction style and one of the two major compaction styles in RocksDB (See [our wiki](https://github.com/facebook/rocksdb/wiki/RocksDB-Basics#multi-threaded-compactions)). In RocksDB we introduced parallelism and more configurable options to it but the main algorithm stayed the same, until we recently introduced the dynamic level size mode. + + +In level-based compaction, we organize data to different sorted runs, called levels. Each level has a target size.  Usually target size of levels increases by the same size multiplier. For example, you can set target size of level 1 to be 1GB, and size multiplier to be 10, and the target size of level 1, 2, 3, 4 will be 1GB, 10GB, 100GB and 1000GB. Before level 1, there will be some staging file flushed from mem tables, called Level 0 files, which will later be merged to level 1. Compactions will be triggered as soon as actual size of a level exceeds its target size. We will merge a subset of data of that level to next level, to reduce size of the level. More compactions will be triggered until sizes of all the levels are lower than their target sizes. In a steady state, the size of each level will be around the same size of the size of level targets. + + +Level-based compaction’s advantage is its good space efficiency. We usually use the metric space amplification to measure the space efficiency. In this article ignore the effects of data compression so space amplification= size_on_file_system / size_of_user_data. + + +How do we estimate space amplification of level-based compaction? We focus specifically on the databases in steady state, which means database size is stable or grows slowly over time. This means updates will add roughly the same or little more data than what is removed by deletes. Given that, if we compact all the data all to the last level, the size of level will be equal as the size of last level before the compaction. On the other hand, the size of user data will be approximately the size of DB if we compact all the levels down to the last level. So the size of the last level will be a good estimation of user data size. So total size of the DB divided by the size of the last level will be a good estimation of space amplification. + + +Applying the equation, if we have four non-zero levels, their sizes are 1GB, 10GB, 100GB, 1000GB, the size amplification will be approximately (1000GB + 100GB + 10GB + 1GB) / 1000GB = 1.111, which is a very good number. However, there is a catch here: how to make sure the last level’s size is 1000GB, the same as the level’s size target? A user has to fine tune level sizes to achieve this number and will need to re-tune if DB size changes. The theoretic number 1.11 is hard to achieve in practice. In a worse case, if you have the target size of last level to be 1000GB but the user data is only 200GB, then the actual space amplification will be (200GB + 100GB + 10GB + 1GB) / 200GB = 1.555, a much worse number. + + +To solve this problem, my colleague Igor Kabiljo came up with a solution of dynamic level size target mode. You can enable it by setting options.level_compaction_dynamic_level_bytes=true. In this mode, size target of levels are changed dynamically based on size of the last level. Suppose the level size multiplier to be 10, and the DB size is 200GB. The target size of the last level is automatically set to be the actual size of the level, which is 200GB, the second to last level’s size target will be automatically set to be size_last_level / 10 = 20GB, the third last level’s will be size_last_level/100 = 2GB, and next level to be size_last_level/1000 = 200MB. We stop here because 200MB is within the range of the first level. In this way, we can achieve the 1.111 space amplification, without fine tuning of the level size targets. More details can be found in [code comments of the option](https://github.com/facebook/rocksdb/blob/v3.11/include/rocksdb/options.h#L366-L423) in the header file. diff --git a/docs/_posts/2015-10-27-getthreadlist.markdown b/docs/_posts/2015-10-27-getthreadlist.markdown new file mode 100644 index 00000000000..332a29f0200 --- /dev/null +++ b/docs/_posts/2015-10-27-getthreadlist.markdown @@ -0,0 +1,193 @@ +--- +title: GetThreadList +layout: post +author: yhciang +category: blog +redirect_from: + - /blog/2261/getthreadlist/ +--- + +We recently added a new API, called `GetThreadList()`, that exposes the RocksDB background thread activity. With this feature, developers will be able to obtain the real-time information about the currently running compactions and flushes such as the input / output size, elapsed time, the number of bytes it has written. Below is an example output of `GetThreadList`. To better illustrate the example, we have put a sample output of `GetThreadList` into a table where each column represents a thread status: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    ThreadID +140716395198208 +140716416169728 +
    DB +db1 +db2 +
    CF +default +picachu +
    ThreadType +High Pri +Low Pri +
    Operation +Flush +Compaction +
    ElapsedTime +143.459 ms +607.538 ms +
    Stage +FlushJob::WriteLevel0Table +CompactionJob::Install +
    OperationProperties + +BytesMemtables 4092938 +BytesWritten 1050701 + +BaseInputLevel 1 +BytesRead 4876417 +BytesWritten 4140109 +IsDeletion 0 +IsManual 0 +IsTrivialMove 0 +JobID 146 +OutputLevel 2 +TotalInputBytes 4883044 +
    + +In the above output, we can see `GetThreadList()` reports the activity of two threads: one thread running flush job (middle column) and the other thread running a compaction job (right-most column). In each thread status, it shows basic information about the thread such as thread id, it's target db / column family, and the job it is currently doing and the current status of the job. For instance, we can see thread 140716416169728 is doing compaction on the `picachu` column family in database `db2`. In addition, we can see the compaction has been running for 600 ms, and it has read 4876417 bytes out of 4883044 bytes. This indicates the compaction is about to complete. The stage property indicates which code block the thread is currently executing. For instance, thread 140716416169728 is currently running `CompactionJob::Install`, which further indicates the compaction job is almost done. + +Below we briefly describe its API. + + +## How to Enable it? + + +To enable thread-tracking of a rocksdb instance, simply set `enable_thread_tracking` to true in its DBOptions: + +```c++ +// If true, then the status of the threads involved in this DB will +// be tracked and available via GetThreadList() API. +// +// Default: false +bool enable_thread_tracking; +``` + + + +## The API + + +The GetThreadList API is defined in [include/rocksdb/env.h](https://github.com/facebook/rocksdb/blob/master/include/rocksdb/env.h#L317-L318), which is an Env +function: + +```c++ +virtual Status GetThreadList(std::vector* thread_list) +``` + +Since an Env can be shared across multiple rocksdb instances, the output of +`GetThreadList()` include the background activity of all the rocksdb instances +that using the same Env. + +The `GetThreadList()` API simply returns a vector of `ThreadStatus`, each describes +the current status of a thread. The `ThreadStatus` structure, defined in +[include/rocksdb/thread_status.h](https://github.com/facebook/rocksdb/blob/master/include/rocksdb/thread_status.h), contains the following information: + +```c++ +// An unique ID for the thread. +const uint64_t thread_id; + +// The type of the thread, it could be HIGH_PRIORITY, +// LOW_PRIORITY, and USER +const ThreadType thread_type; + +// The name of the DB instance where the thread is currently +// involved with. It would be set to empty string if the thread +// does not involve in any DB operation. +const std::string db_name; + +// The name of the column family where the thread is currently +// It would be set to empty string if the thread does not involve +// in any column family. +const std::string cf_name; + +// The operation (high-level action) that the current thread is involved. +const OperationType operation_type; + +// The elapsed time in micros of the current thread operation. +const uint64_t op_elapsed_micros; + +// An integer showing the current stage where the thread is involved +// in the current operation. +const OperationStage operation_stage; + +// A list of properties that describe some details about the current +// operation. Same field in op_properties[] might have different +// meanings for different operations. +uint64_t op_properties[kNumOperationProperties]; + +// The state (lower-level action) that the current thread is involved. +const StateType state_type; +``` + +If you are interested in the background thread activity of your RocksDB application, please feel free to give `GetThreadList()` a try :) diff --git a/docs/_posts/2015-11-10-use-checkpoints-for-efficient-snapshots.markdown b/docs/_posts/2015-11-10-use-checkpoints-for-efficient-snapshots.markdown new file mode 100644 index 00000000000..6852b8ffa3a --- /dev/null +++ b/docs/_posts/2015-11-10-use-checkpoints-for-efficient-snapshots.markdown @@ -0,0 +1,45 @@ +--- +title: Use Checkpoints for Efficient Snapshots +layout: post +author: rven2 +category: blog +redirect_from: + - /blog/2609/use-checkpoints-for-efficient-snapshots/ +--- + +**Checkpoint** is a feature in RocksDB which provides the ability to take a snapshot of a running RocksDB database in a separate directory. Checkpoints can be used as a point in time snapshot, which can be opened Read-only to query rows as of the point in time or as a Writeable snapshot by opening it Read-Write. Checkpoints can be used for both full and incremental backups. + + + + +The Checkpoint feature enables RocksDB to create a consistent snapshot of a given RocksDB database in the specified directory. If the snapshot is on the same filesystem as the original database, the SST files will be hard-linked, otherwise SST files will be copied. The manifest and CURRENT files will be copied. In addition, if there are multiple column families, log files will be copied for the period covering the start and end of the checkpoint, in order to provide a consistent snapshot across column families. + + + + +A Checkpoint object needs to be created for a database before checkpoints are created. The API is as follows: + + + + +`Status Create(DB* db, Checkpoint** checkpoint_ptr);` + + + + +Given a checkpoint object and a directory, the CreateCheckpoint function creates a consistent snapshot of the database in the given directory. + + + + +`Status CreateCheckpoint(const std::string& checkpoint_dir);` + + + + +The directory should not already exist and will be created by this API. The directory will be an absolute path. The checkpoint can be used as a ​read-only copy of the DB or can be opened as a standalone DB. When opened read/write, the SST files continue to be hard links and these links are removed when the files are obsoleted. When the user is done with the snapshot, the user can delete the directory to remove the snapshot. + + + + +Checkpoints are used for online backup in ​MyRocks. which is MySQL using RocksDB as the storage engine . ([MySQL on RocksDB](https://github.com/facebook/mysql-5.6)) ​ diff --git a/docs/_posts/2015-11-16-analysis-file-read-latency-by-level.markdown b/docs/_posts/2015-11-16-analysis-file-read-latency-by-level.markdown new file mode 100644 index 00000000000..b21b04fe386 --- /dev/null +++ b/docs/_posts/2015-11-16-analysis-file-read-latency-by-level.markdown @@ -0,0 +1,244 @@ +--- +title: Analysis File Read Latency by Level +layout: post +author: sdong +category: blog +redirect_from: + - /blog/2537/analysis-file-read-latency-by-level/ +--- + +In many use cases of RocksDB, people rely on OS page cache for caching compressed data. With this approach, verifying effective of the OS page caching is challenging, because file system is a black box to users. + +As an example, a user can tune the DB as following: use level-based compaction, with L1 - L4 sizes to be 1GB, 10GB, 100GB and 1TB. And they reserve about 20GB memory as OS page cache, expecting level 0, 1 and 2 are mostly cached in memory, leaving only reads from level 3 and 4 requiring disk I/Os. However, in practice, it's not easy to verify whether OS page cache does exactly what we expect. For example, if we end up with doing 4 instead of 2 I/Os per query, it's not easy for users to figure out whether the it's because of efficiency of OS page cache or reading multiple blocks for a level. Analysis like it is especially important if users run RocksDB on hard drive disks, for the gap of latency between hard drives and memory is much higher than flash-based SSDs. + + + +In order to make tuning easier, we added new instrumentation to help users analysis latency distribution of file reads in different levels. If users turn DB statistics on, we always keep track of distribution of file read latency for each level. Users can retrieve the information by querying DB property “rocksdb.stats” ( [https://github.com/facebook/rocksdb/blob/v3.13.1/include/rocksdb/db.h#L315-L316](https://github.com/facebook/rocksdb/blob/v3.13.1/include/rocksdb/db.h#L315-L316) ). It will also printed out as a part of compaction summary in info logs periodically. + +The output looks like this: + + +``` +** Level 0 read latency histogram (micros): +Count: 696 Average: 489.8118 StdDev: 222.40 +Min: 3.0000 Median: 452.3077 Max: 1896.0000 +Percentiles: P50: 452.31 P75: 641.30 P99: 1068.00 P99.9: 1860.80 P99.99: 1896.00 +------------------------------------------------------ +[ 2, 3 ) 1 0.144% 0.144% +[ 18, 20 ) 1 0.144% 0.287% +[ 45, 50 ) 5 0.718% 1.006% +[ 50, 60 ) 26 3.736% 4.741% # +[ 60, 70 ) 6 0.862% 5.603% +[ 90, 100 ) 1 0.144% 5.747% +[ 120, 140 ) 2 0.287% 6.034% +[ 140, 160 ) 1 0.144% 6.178% +[ 160, 180 ) 1 0.144% 6.322% +[ 200, 250 ) 9 1.293% 7.615% +[ 250, 300 ) 45 6.466% 14.080% # +[ 300, 350 ) 88 12.644% 26.724% ### +[ 350, 400 ) 88 12.644% 39.368% ### +[ 400, 450 ) 71 10.201% 49.569% ## +[ 450, 500 ) 65 9.339% 58.908% ## +[ 500, 600 ) 74 10.632% 69.540% ## +[ 600, 700 ) 92 13.218% 82.759% ### +[ 700, 800 ) 64 9.195% 91.954% ## +[ 800, 900 ) 35 5.029% 96.983% # +[ 900, 1000 ) 12 1.724% 98.707% +[ 1000, 1200 ) 6 0.862% 99.569% +[ 1200, 1400 ) 2 0.287% 99.856% +[ 1800, 2000 ) 1 0.144% 100.000% + +** Level 1 read latency histogram (micros): +(......not pasted.....) + +** Level 2 read latency histogram (micros): +(......not pasted.....) + +** Level 3 read latency histogram (micros): +(......not pasted.....) + +** Level 4 read latency histogram (micros): +(......not pasted.....) + +** Level 5 read latency histogram (micros): +Count: 25583746 Average: 421.1326 StdDev: 385.11 +Min: 1.0000 Median: 376.0011 Max: 202444.0000 +Percentiles: P50: 376.00 P75: 438.00 P99: 1421.68 P99.9: 4164.43 P99.99: 9056.52 +------------------------------------------------------ +[ 0, 1 ) 2351 0.009% 0.009% +[ 1, 2 ) 6077 0.024% 0.033% +[ 2, 3 ) 8471 0.033% 0.066% +[ 3, 4 ) 788 0.003% 0.069% +[ 4, 5 ) 393 0.002% 0.071% +[ 5, 6 ) 786 0.003% 0.074% +[ 6, 7 ) 1709 0.007% 0.080% +[ 7, 8 ) 1769 0.007% 0.087% +[ 8, 9 ) 1573 0.006% 0.093% +[ 9, 10 ) 1495 0.006% 0.099% +[ 10, 12 ) 3043 0.012% 0.111% +[ 12, 14 ) 2259 0.009% 0.120% +[ 14, 16 ) 1233 0.005% 0.125% +[ 16, 18 ) 762 0.003% 0.128% +[ 18, 20 ) 451 0.002% 0.130% +[ 20, 25 ) 794 0.003% 0.133% +[ 25, 30 ) 1279 0.005% 0.138% +[ 30, 35 ) 1172 0.005% 0.142% +[ 35, 40 ) 1363 0.005% 0.148% +[ 40, 45 ) 409 0.002% 0.149% +[ 45, 50 ) 105 0.000% 0.150% +[ 50, 60 ) 80 0.000% 0.150% +[ 60, 70 ) 280 0.001% 0.151% +[ 70, 80 ) 1583 0.006% 0.157% +[ 80, 90 ) 4245 0.017% 0.174% +[ 90, 100 ) 6572 0.026% 0.200% +[ 100, 120 ) 9724 0.038% 0.238% +[ 120, 140 ) 3713 0.015% 0.252% +[ 140, 160 ) 2383 0.009% 0.261% +[ 160, 180 ) 18344 0.072% 0.333% +[ 180, 200 ) 51873 0.203% 0.536% +[ 200, 250 ) 631722 2.469% 3.005% +[ 250, 300 ) 2721970 10.639% 13.644% ## +[ 300, 350 ) 5909249 23.098% 36.742% ##### +[ 350, 400 ) 6522507 25.495% 62.237% ##### +[ 400, 450 ) 4296332 16.793% 79.030% ### +[ 450, 500 ) 2130323 8.327% 87.357% ## +[ 500, 600 ) 1553208 6.071% 93.428% # +[ 600, 700 ) 642129 2.510% 95.938% # +[ 700, 800 ) 372428 1.456% 97.394% +[ 800, 900 ) 187561 0.733% 98.127% +[ 900, 1000 ) 85858 0.336% 98.462% +[ 1000, 1200 ) 82730 0.323% 98.786% +[ 1200, 1400 ) 50691 0.198% 98.984% +[ 1400, 1600 ) 38026 0.149% 99.133% +[ 1600, 1800 ) 32991 0.129% 99.261% +[ 1800, 2000 ) 30200 0.118% 99.380% +[ 2000, 2500 ) 62195 0.243% 99.623% +[ 2500, 3000 ) 36684 0.143% 99.766% +[ 3000, 3500 ) 21317 0.083% 99.849% +[ 3500, 4000 ) 10216 0.040% 99.889% +[ 4000, 4500 ) 8351 0.033% 99.922% +[ 4500, 5000 ) 4152 0.016% 99.938% +[ 5000, 6000 ) 6328 0.025% 99.963% +[ 6000, 7000 ) 3253 0.013% 99.976% +[ 7000, 8000 ) 2082 0.008% 99.984% +[ 8000, 9000 ) 1546 0.006% 99.990% +[ 9000, 10000 ) 1055 0.004% 99.994% +[ 10000, 12000 ) 1566 0.006% 100.000% +[ 12000, 14000 ) 761 0.003% 100.003% +[ 14000, 16000 ) 462 0.002% 100.005% +[ 16000, 18000 ) 226 0.001% 100.006% +[ 18000, 20000 ) 126 0.000% 100.006% +[ 20000, 25000 ) 107 0.000% 100.007% +[ 25000, 30000 ) 43 0.000% 100.007% +[ 30000, 35000 ) 15 0.000% 100.007% +[ 35000, 40000 ) 14 0.000% 100.007% +[ 40000, 45000 ) 16 0.000% 100.007% +[ 45000, 50000 ) 1 0.000% 100.007% +[ 50000, 60000 ) 22 0.000% 100.007% +[ 60000, 70000 ) 10 0.000% 100.007% +[ 70000, 80000 ) 5 0.000% 100.007% +[ 80000, 90000 ) 14 0.000% 100.007% +[ 90000, 100000 ) 11 0.000% 100.007% +[ 100000, 120000 ) 33 0.000% 100.007% +[ 120000, 140000 ) 6 0.000% 100.007% +[ 140000, 160000 ) 3 0.000% 100.007% +[ 160000, 180000 ) 7 0.000% 100.007% +[ 200000, 250000 ) 2 0.000% 100.007% +``` + + +In this example, you can see we only issued 696 reads from level 0 while issued 25 million reads from level 5. The latency distribution is also clearly shown among those reads. This will be helpful for users to analysis OS page cache efficiency. + +Currently the read latency per level includes reads from data blocks, index blocks, as well as bloom filter blocks. We are also working on a feature to break down those three type of blocks. + +### Comments + +**[Tao Feng](fengtao04@gmail.com)** + +Is this feature also included in RocksJava? + +**[Siying Dong](siying.d@fb.com)** + +Should be. As long as you enable statistics, you should be able to get the value from `RocksDB.getProperty()` with property `rocksdb.dbstats`. Let me know if you can’t find it. + +**[chiddu](cnbscience@gmail.com)** + +> In this example, you can see we only issued 696 reads from level 0 while issued 256K reads from level 5. + +Isn’t it 2.5 M of reads instead of 256K ? . + +Also could anyone please provide more description on the histogram ? especially + +> Count: 25583746 Average: 421.1326 StdDev: 385.11 +> Min: 1.0000 Median: 376.0011 Max: 202444.0000 +> Percentiles: P50: 376.00 P75: 438.00 P99: 1421.68 P99.9: 4164.43 P99.99: 9056.52 + +and + +> [ 0, 1 ) 2351 0.009% 0.009% +> [ 1, 2 ) 6077 0.024% 0.033% +> [ 2, 3 ) 8471 0.033% 0.066% +> [ 3, 4 ) 788 0.003% 0.069%” + +thanks in advance + +**[Siying Dong](siying.d@fb.com)** + +Thank you for pointing out the mistake. I fixed it now. + +In this output, there are 2.5 million samples, average latency is 421 micro seconds, with standard deviation 385. Median is 376, max value is 202 milliseconds. 0.009% has value of 1, 0.024% has value of 1, 0.033% has value of 2. Accumulated value from 0 to 2 is 0.066%. + +Hope it helps. + +**[chiddu](cnbscience@gmail.com)** + +Thank you Siying for the quick reply, I was running couple of benchmark testing to check the performance of rocksdb on SSD. One of the test is similar to what is mentioned in the wiki, TEST 4 : Random read , except the key_size is 10 and value_size is 20. I am inserting 1 billion hashes and reading 1 billion hashes with 32 threads. The histogram shows something like this + +``` +Level 5 read latency histogram (micros): +Count: 7133903059 Average: 480.4357 StdDev: 309.18 +Min: 0.0000 Median: 551.1491 Max: 224142.0000 +Percentiles: P50: 551.15 P75: 651.44 P99: 996.52 P99.9: 2073.07 P99.99: 3196.32 +—————————————————— +[ 0, 1 ) 28587385 0.401% 0.401% +[ 1, 2 ) 686572516 9.624% 10.025% ## +[ 2, 3 ) 567317522 7.952% 17.977% ## +[ 3, 4 ) 44979472 0.631% 18.608% +[ 4, 5 ) 50379685 0.706% 19.314% +[ 5, 6 ) 64930061 0.910% 20.224% +[ 6, 7 ) 22613561 0.317% 20.541% +…………more…………. +``` + +If I understand your previous comment correctly, + +1. How is it that the count is around 7 billion when I have only inserted 1 billion hashes ? is the stat broken ? +1. What does the percentiles and the numbers signify ? +1. 0, 1 ) 28587385 0.401% 0.401% what does this “28587385” stand for in the histogram row ? + +**[Siying Dong](siying.d@fb.com)** + +If I remember correctly, with db_bench, if you specify –num=1000000000 –threads=32, it is every thread reading one billion keys, total of 32 billions. Is it the case you ran into? + +28,587,385 means that number of data points take the value [0,1) +28,587,385 / 7,133,903,058 = 0.401% provides percentage. + +**[chiddu](cnbscience@gmail.com)** + +I do have `num=1000000000` and `t=32`. The script says reading 1 billion hashes and not 32 billion hashes. + +this is the script on which I have used + +``` +echo “Load 1B keys sequentially into database…..” +bpl=10485760;overlap=10;mcz=2;del=300000000;levels=6;ctrig=4; delay=8; stop=12; wbn=3; mbc=20; mb=67108864;wbs=134217728; dds=1; sync=0; r=1000000000; t=1; vs=20; bs=4096; cs=1048576; of=500000; si=1000000; ./db_bench –benchmarks=fillseq –disable_seek_compaction=1 –mmap_read=0 –statistics=1 –histogram=1 –num=$r –threads=$t –value_size=$vs –block_size=$bs –cache_size=$cs –bloom_bits=10 –cache_numshardbits=6 –open_files=$of –verify_checksum=1 –db=/data/mysql/leveldb/test –sync=$sync –disable_wal=1 –compression_type=none –stats_interval=$si –compression_ratio=0.5 –disable_data_sync=$dds –write_buffer_size=$wbs –target_file_size_base=$mb –max_write_buffer_number=$wbn –max_background_compactions=$mbc –level0_file_num_compaction_trigger=$ctrig –level0_slowdown_writes_trigger=$delay –level0_stop_writes_trigger=$stop –num_levels=$levels –delete_obsolete_files_period_micros=$del –min_level_to_compress=$mcz –max_grandparent_overlap_factor=$overlap –stats_per_interval=1 –max_bytes_for_level_base=$bpl –use_existing_db=0 –key_size=10 + +echo “Reading 1B keys in database in random order….” +bpl=10485760;overlap=10;mcz=2;del=300000000;levels=6;ctrig=4; delay=8; stop=12; wbn=3; mbc=20; mb=67108864;wbs=134217728; dds=0; sync=0; r=1000000000; t=32; vs=20; bs=4096; cs=1048576; of=500000; si=1000000; ./db_bench –benchmarks=readrandom –disable_seek_compaction=1 –mmap_read=0 –statistics=1 –histogram=1 –num=$r –threads=$t –value_size=$vs –block_size=$bs –cache_size=$cs –bloom_bits=10 –cache_numshardbits=6 –open_files=$of –verify_checksum=1 –db=/some_data_base –sync=$sync –disable_wal=1 –compression_type=none –stats_interval=$si –compression_ratio=0.5 –disable_data_sync=$dds –write_buffer_size=$wbs –target_file_size_base=$mb –max_write_buffer_number=$wbn –max_background_compactions=$mbc –level0_file_num_compaction_trigger=$ctrig –level0_slowdown_writes_trigger=$delay –level0_stop_writes_trigger=$stop –num_levels=$levels –delete_obsolete_files_period_micros=$del –min_level_to_compress=$mcz –max_grandparent_overlap_factor=$overlap –stats_per_interval=1 –max_bytes_for_level_base=$bpl –use_existing_db=1 –key_size=10 +``` + +After running this script, there were no issues wrt to loading billion hashes , but when it came to reading part, its been almost 4 days and still I have only read 7 billion hashes and have read 200 million hashes in 2 and half days. Is there something which is missing in db_bench or something which I am missing ? + +**[Siying Dong](siying.d@fb.com)** + +It’s a printing error then. If you have `num=1000000000` and `t=32`, it will be 32 threads, and each reads 1 billion keys. diff --git a/docs/_posts/2016-01-29-compaction_pri.markdown b/docs/_posts/2016-01-29-compaction_pri.markdown new file mode 100644 index 00000000000..ba9ee627c91 --- /dev/null +++ b/docs/_posts/2016-01-29-compaction_pri.markdown @@ -0,0 +1,51 @@ +--- +title: Option of Compaction Priority +layout: post +author: sdong +category: blog +redirect_from: + - /blog/2921/compaction_pri/ +--- + +The most popular compaction style of RocksDB is level-based compaction, which is an improved version of LevelDB's compaction algorithm. Page 9- 16 of this [slides](https://github.com/facebook/rocksdb/blob/gh-pages/talks/2015-09-29-HPTS-Siying-RocksDB.pdf) gives an illustrated introduction of this compaction style. The basic idea that: data is organized by multiple levels with exponential increasing target size. Except a special level 0, every level is key-range partitioned into many files. When size of a level exceeds its target size, we pick one or more of its files, and merge the file into the next level. + + + +Which file to pick to compact is an interesting question. LevelDB only uses one thread for compaction and it always picks files in round robin manner. We implemented multi-thread compaction in RocksDB by picking multiple files from the same level and compact them in parallel. We had to move away from LevelDB's file picking approach. Recently, we created an option [options.compaction_pri](https://github.com/facebook/rocksdb/blob/d6c838f1e130d8860407bc771fa6d4ac238859ba/include/rocksdb/options.h#L83-L93), which indicated three different algorithms to pick files to compact. + +Why do we need to multiple algorithms to choose from? Because there are different factors to consider when picking the files, and we now don't yet know how to balance them automatically, so we expose it to users to choose. Here are factors to consider: + +**Write amplification** + +When we estimate write amplification, we usually simplify the problem by assuming keys are uniformly distributed inside each level. In reality, it is not the case, even if user updates are uniformly distributed across the whole key range. For instance, when we compact one file of a level to the next level, it creates a hole. Over time, incoming compaction will fill data to the hole, but the density will still be lower for a while. Picking a file with keys least densely populated is more expensive to get the file to the next level, because there will be more overlapping files in the next level so we need to rewrite more data. For example, assume a file is 100MB, if an L2 file overlaps with 8 L3 files, we need to rewrite about 800MB of data to get the file to L3. If the file overlaps with 12 L3 files, we'll need to rewrite about 1200MB to get a file of the same size out of L2. It uses 50% more writes. (This analysis ignores the key density of the next level, because the range covers N times of files in that level so one hole only impacts write amplification by 1/N) + +If all the updates are uniformly distributed, LevelDB's approach optimizes write amplification, because a file being picked covers a range whose last compaction time to the next level is the oldest, so the range will accumulated keys from incoming compactions for the longest and the density is the highest. + +We created a compaction priority **kOldestSmallestSeqFirst** for the same effect. With this mode, we always pick the file covers the oldest updates in the level, which usually is contains the densest key range. If you have a use case where writes are uniformly distributed across the key space and you want to reduce write amplification, you should set options.compaction_pri=kOldestSmallestSeqFirst. + +**Optimize for small working set** + +We are assuming updates are uniformly distributed across the whole key space in previous analysis. However, in many use cases, there are subset of keys that are frequently updated while other key ranges are very cold. In this case, keeping hot key ranges from compacting to deeper levels will benefit write amplification, as well as space amplification. For example, if in a DB only key 150-160 are updated and other keys are seldom updated. If level 1 contains 20 keys, we want to keep 150-160 all stay in level 1. Because when next level 0 -> 1 compaction comes, it will simply overwrite existing keys so size level 1 doesn't increase, so no need to schedule further compaction for level 1->2. On the other hand, if we compact key 150-155 to level2, when a new Level 1->2 compaction comes, it increases the size of level 1, making size of level 1 exceed target size and more compactions will be needed, which generates more writes. + +The compaction priority **kOldestLargestSeqFirst** optimizes this use case. In this mode, we will pick a file whose latest update is the oldest. It means there is no incoming data for the range for the longest. Usually it is the coldest range. By compacting coldest range first, we leave the hot ranges in the level. If your use case is to overwrite existing keys in a small range, try options.compaction_pri=kOldestLargestSeqFirst**.** + +**Drop delete marker sooner** + +If one file contains a lot of delete markers, it may slow down iterating over this area, because we still need to iterate those deleted keys just to ignore them. Furthermore, the sooner we compact delete keys into the last level, the sooner the disk space is reclaimed, so it is good for space efficiency. + +Our default compaction priority **kByCompensatedSize** considers the case. If number of deletes in a file exceeds number of inserts, it is more likely to be picked for compaction. The more number of deletes exceed inserts, the more likely it is being compacted. The optimization is added to avoid the worst performance of space efficiency and query performance when a large percentage of the DB is deleted. + +**Efficiency of compaction filter** + +Usually people use [compaction filters](https://github.com/facebook/rocksdb/blob/v4.1/include/rocksdb/options.h#L201-L226) to clean up old data to free up space. Picking files to compact may impact space efficiency. We don't yet have a a compaction priority to optimize this case. In some of our use cases, we solved the problem in a different way: we have an external service checking modify time of all SST files. If any of the files is too old, we force the single file to compaction by calling DB::CompactFiles() using the single file. In this way, we can provide a time bound of data passing through compaction filters. + + +In all, there three choices of compaction priority modes optimizing different scenarios. if you have a new use case, we suggest you start with `options.compaction_pri=kOldestSmallestSeqFirst` (note it is not the default one for backward compatible reason). If you want to further optimize your use case, you can try other two use cases if your use cases apply. + +If you have good ideas about better compaction picker approach, you are welcome to implement and benchmark it. We'll be glad to review and merge your a pull requests. + +### Comments + +**[Mark Callaghan](mdcallag@gmail.com)** + +Performance results for compaction_pri values and linkbench are explained at [http://smalldatum.blogspot.com/2016/02/compaction-priority-in-rocksdb.html](http://smalldatum.blogspot.com/2016/02/compaction-priority-in-rocksdb.html) diff --git a/docs/_posts/2016-02-24-rocksdb-4-2-release.markdown b/docs/_posts/2016-02-24-rocksdb-4-2-release.markdown new file mode 100644 index 00000000000..409015cc8c8 --- /dev/null +++ b/docs/_posts/2016-02-24-rocksdb-4-2-release.markdown @@ -0,0 +1,41 @@ +--- +title: RocksDB 4.2 Release! +layout: post +author: sdong +category: blog +redirect_from: + - /blog/3017/rocksdb-4-2-release/ +--- + +New RocksDB release - 4.2! + + +**New Features** + + 1. Introduce CreateLoggerFromOptions(), this function create a Logger for provided DBOptions. + + + 2. Add GetAggregatedIntProperty(), which returns the sum of the GetIntProperty of all the column families. + + + 3. Add MemoryUtil in rocksdb/utilities/memory.h. It currently offers a way to get the memory usage by type from a list rocksdb instances. + + + + + +**Public API changes** + + 1. CompactionFilter::Context includes information of Column Family ID + + + 2. The need-compaction hint given by TablePropertiesCollector::NeedCompact() will be persistent and recoverable after DB recovery. This introduces a breaking format change. If you use this experimental feature, including NewCompactOnDeletionCollectorFactory() in the new version, you may not be able to directly downgrade the DB back to version 4.0 or lower. + + + 3. TablePropertiesCollectorFactory::CreateTablePropertiesCollector() now takes an option Context, containing the information of column family ID for the file being written. + + + 4. Remove DefaultCompactionFilterFactory. + + +[https://github.com/facebook/rocksdb/releases/tag/v4.2](https://github.com/facebook/rocksdb/releases/tag/v4.2) diff --git a/docs/_posts/2016-02-25-rocksdb-ama.markdown b/docs/_posts/2016-02-25-rocksdb-ama.markdown new file mode 100644 index 00000000000..2ba04f39a18 --- /dev/null +++ b/docs/_posts/2016-02-25-rocksdb-ama.markdown @@ -0,0 +1,20 @@ +--- +title: RocksDB AMA +layout: post +author: yhchiang +category: blog +redirect_from: + - /blog/3065/rocksdb-ama/ +--- + +RocksDB developers are doing a Reddit Ask-Me-Anything now at 10AM – 11AM PDT! We welcome you to stop by and ask any RocksDB related questions, including existing / upcoming features, tuning tips, or database design. + +Here are some enhancements that we'd like to focus on over the next six months: + +* 2-Phase Commit +* Lua support in some custom functions +* Backup and repair tools +* Direct I/O to bypass OS cache +* RocksDB Java API + +[https://www.reddit.com/r/IAmA/comments/47k1si/we_are_rocksdb_developers_ask_us_anything/](https://www.reddit.com/r/IAmA/comments/47k1si/we_are_rocksdb_developers_ask_us_anything/) diff --git a/docs/_posts/2016-03-07-rocksdb-options-file.markdown b/docs/_posts/2016-03-07-rocksdb-options-file.markdown new file mode 100644 index 00000000000..703449b01ae --- /dev/null +++ b/docs/_posts/2016-03-07-rocksdb-options-file.markdown @@ -0,0 +1,24 @@ +--- +title: RocksDB Options File +layout: post +author: yhciang +category: blog +redirect_from: + - /blog/3089/rocksdb-options-file/ +--- + +In RocksDB 4.3, we added a new set of features that makes managing RocksDB options easier. Specifically: + + * **Persisting Options Automatically**: Each RocksDB database will now automatically persist its current set of options into an INI file on every successful call of DB::Open(), SetOptions(), and CreateColumnFamily() / DropColumnFamily(). + + + + * **Load Options from File**: We added [LoadLatestOptions() / LoadOptionsFromFile()](https://github.com/facebook/rocksdb/blob/4.3.fb/include/rocksdb/utilities/options_util.h#L48-L58) that enables developers to construct RocksDB options object from an options file. + + + + * **Sanity Check Options**: We added [CheckOptionsCompatibility](https://github.com/facebook/rocksdb/blob/4.3.fb/include/rocksdb/utilities/options_util.h#L64-L77) that performs compatibility check on two sets of RocksDB options. + + + +Want to know more about how to use this new features? Check out the [RocksDB Options File wiki page](https://github.com/facebook/rocksdb/wiki/RocksDB-Options-File) and start using this new feature today! diff --git a/docs/_posts/2016-04-26-rocksdb-4-5-1-released.markdown b/docs/_posts/2016-04-26-rocksdb-4-5-1-released.markdown new file mode 100644 index 00000000000..247768d307b --- /dev/null +++ b/docs/_posts/2016-04-26-rocksdb-4-5-1-released.markdown @@ -0,0 +1,60 @@ +--- +title: RocksDB 4.5.1 Released! +layout: post +author: sdong +category: blog +redirect_from: + - /blog/3179/rocksdb-4-5-1-released/ +--- + +## 4.5.1 (3/25/2016) + +### Bug Fixes + + *  Fix failures caused by the destorying order of singleton objects. + +
    + +## 4.5.0 (2/5/2016) + +### Public API Changes + + * Add a new perf context level between kEnableCount and kEnableTime. Level 2 now does not include timers for mutexes. + * Statistics of mutex operation durations will not be measured by default. If you want to have them enabled, you need to set Statistics::stats_level_ to kAll. + * DBOptions::delete_scheduler and NewDeleteScheduler() are removed, please use DBOptions::sst_file_manager and NewSstFileManager() instead + +### New Features + * ldb tool now supports operations to non-default column families. + * Add kPersistedTier to ReadTier. This option allows Get and MultiGet to read only the persited data and skip mem-tables if writes were done with disableWAL = true. + * Add DBOptions::sst_file_manager. Use NewSstFileManager() in include/rocksdb/sst_file_manager.h to create a SstFileManager that can be used to track the total size of SST files and control the SST files deletion rate. + +
    + + + +## 4.4.0 (1/14/2016) + +### Public API Changes + + * Change names in CompactionPri and add a new one. + * Deprecate options.soft_rate_limit and add options.soft_pending_compaction_bytes_limit. + * If options.max_write_buffer_number > 3, writes will be slowed down when writing to the last write buffer to delay a full stop. + * Introduce CompactionJobInfo::compaction_reason, this field include the reason to trigger the compaction. + * After slow down is triggered, if estimated pending compaction bytes keep increasing, slowdown more. + * Increase default options.delayed_write_rate to 2MB/s. + * Added a new parameter --path to ldb tool. --path accepts the name of either MANIFEST, SST or a WAL file. Either --db or --path can be used when calling ldb. + +
    + +## 4.3.0 (12/8/2015) + +### New Features + + * CompactionFilter has new member function called IgnoreSnapshots which allows CompactionFilter to be called even if there are snapshots later than the key. + * RocksDB will now persist options under the same directory as the RocksDB database on successful DB::Open, CreateColumnFamily, DropColumnFamily, and SetOptions. + * Introduce LoadLatestOptions() in rocksdb/utilities/options_util.h. This function can construct the latest DBOptions / ColumnFamilyOptions used by the specified RocksDB intance. + * Introduce CheckOptionsCompatibility() in rocksdb/utilities/options_util.h. This function checks whether the input set of options is able to open the specified DB successfully. + +### Public API Changes + + * When options.db_write_buffer_size triggers, only the column family with the largest column family size will be flushed, not all the column families. diff --git a/docs/_posts/2016-07-26-rocksdb-4-8-released.markdown b/docs/_posts/2016-07-26-rocksdb-4-8-released.markdown new file mode 100644 index 00000000000..b42a66e301c --- /dev/null +++ b/docs/_posts/2016-07-26-rocksdb-4-8-released.markdown @@ -0,0 +1,48 @@ +--- +title: RocksDB 4.8 Released! +layout: post +author: yiwu +category: blog +redirect_from: + - /blog/3239/rocksdb-4-8-released/ +--- + +## 4.8.0 (5/2/2016) + +### [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#public-api-change-1)Public API Change + + * Allow preset compression dictionary for improved compression of block-based tables. This is supported for zlib, zstd, and lz4. The compression dictionary's size is configurable via CompressionOptions::max_dict_bytes. + * Delete deprecated classes for creating backups (BackupableDB) and restoring from backups (RestoreBackupableDB). Now, BackupEngine should be used for creating backups, and BackupEngineReadOnly should be used for restorations. For more details, see [https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB%3F](https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB%3F) + * Expose estimate of per-level compression ratio via DB property: "rocksdb.compression-ratio-at-levelN". + * Added EventListener::OnTableFileCreationStarted. EventListener::OnTableFileCreated will be called on failure case. User can check creation status via TableFileCreationInfo::status. + +### [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#new-features-2)New Features + + * Add ReadOptions::readahead_size. If non-zero, NewIterator will create a new table reader which performs reads of the given size. + +
    + + + +## [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#470-482016)4.7.0 (4/8/2016) + +### [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#public-api-change-2)Public API Change + + * rename options compaction_measure_io_stats to report_bg_io_stats and include flush too. + * Change some default options. Now default options will optimize for server-workloads. Also enable slowdown and full stop triggers for pending compaction bytes. These changes may cause sub-optimal performance or significant increase of resource usage. To avoid these risks, users can open existing RocksDB with options extracted from RocksDB option files. See [https://github.com/facebook/rocksdb/wiki/RocksDB-Options-File](https://github.com/facebook/rocksdb/wiki/RocksDB-Options-File) for how to use RocksDB option files. Or you can call Options.OldDefaults() to recover old defaults. DEFAULT_OPTIONS_HISTORY.md will track change history of default options. + +
    + +## [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#460-3102016)4.6.0 (3/10/2016) + +### [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#public-api-changes-1)Public API Changes + + * Change default of BlockBasedTableOptions.format_version to 2. It means default DB created by 4.6 or up cannot be opened by RocksDB version 3.9 or earlier + * Added strict_capacity_limit option to NewLRUCache. If the flag is set to true, insert to cache will fail if no enough capacity can be free. Signature of Cache::Insert() is updated accordingly. + * Tickers [NUMBER_DB_NEXT, NUMBER_DB_PREV, NUMBER_DB_NEXT_FOUND, NUMBER_DB_PREV_FOUND, ITER_BYTES_READ] are not updated immediately. The are updated when the Iterator is deleted. + * Add monotonically increasing counter (DB property "rocksdb.current-super-version-number") that increments upon any change to the LSM tree. + +### [](https://github.com/facebook/rocksdb/blob/master/HISTORY.md#new-features-3)New Features + + * Add CompactionPri::kMinOverlappingRatio, a compaction picking mode friendly to write amplification. + * Deprecate Iterator::IsKeyPinned() and replace it with Iterator::GetProperty() with prop_name="rocksdb.iterator.is.key.pinned" diff --git a/docs/_posts/2016-09-28-rocksdb-4-11-2-released.markdown b/docs/_posts/2016-09-28-rocksdb-4-11-2-released.markdown new file mode 100644 index 00000000000..87c20eb47d4 --- /dev/null +++ b/docs/_posts/2016-09-28-rocksdb-4-11-2-released.markdown @@ -0,0 +1,49 @@ +--- +title: RocksDB 4.11.2 Released! +layout: post +author: sdong +category: blog +--- +We abandoned release candidates 4.10.x and directly go to 4.11.2 from 4.9, to make sure the latest release is stable. In 4.11.2, we fixed several data corruption related bugs introduced in 4.9.0. + +## 4.11.2 (9/15/2016) + +### Bug fixes + + * Segfault when failing to open an SST file for read-ahead iterators. + * WAL without data for all CFs is not deleted after recovery. + + + +## 4.11.1 (8/30/2016) + +### Bug Fixes + + * Mitigate the regression bug of deadlock condition during recovery when options.max_successive_merges hits. + * Fix data race condition related to hash index in block based table when putting indexes in the block cache. + +## 4.11.0 (8/1/2016) + +### Public API Change + + * options.memtable_prefix_bloom_huge_page_tlb_size => memtable_huge_page_size. When it is set, RocksDB will try to allocate memory from huge page for memtable too, rather than just memtable bloom filter. + +### New Features + + * A tool to migrate DB after options change. See include/rocksdb/utilities/option_change_migration.h. + * Add ReadOptions.background_purge_on_iterator_cleanup. If true, we avoid file deletion when destorying iterators. + +## 4.10.0 (7/5/2016) + +### Public API Change + + * options.memtable_prefix_bloom_bits changes to options.memtable_prefix_bloom_bits_ratio and deprecate options.memtable_prefix_bloom_probes + * enum type CompressionType and PerfLevel changes from char to unsigned char. Value of all PerfLevel shift by one. + * Deprecate options.filter_deletes. + +### New Features + + * Add avoid_flush_during_recovery option. + * Add a read option background_purge_on_iterator_cleanup to avoid deleting files in foreground when destroying iterators. Instead, a job is scheduled in high priority queue and would be executed in a separate background thread. + * RepairDB support for column families. RepairDB now associates data with non-default column families using information embedded in the SST/WAL files (4.7 or later). For data written by 4.6 or earlier, RepairDB associates it with the default column family. + * Add options.write_buffer_manager which allows users to control total memtable sizes across multiple DB instances. diff --git a/docs/_posts/2017-01-06-rocksdb-5-0-1-released.markdown b/docs/_posts/2017-01-06-rocksdb-5-0-1-released.markdown new file mode 100644 index 00000000000..fb0413055db --- /dev/null +++ b/docs/_posts/2017-01-06-rocksdb-5-0-1-released.markdown @@ -0,0 +1,26 @@ +--- +title: RocksDB 5.0.1 Released! +layout: post +author: yiwu +category: blog +--- + +### Public API Change + + * Options::max_bytes_for_level_multiplier is now a double along with all getters and setters. + * Support dynamically change `delayed_write_rate` and `max_total_wal_size` options via SetDBOptions(). + * Introduce DB::DeleteRange for optimized deletion of large ranges of contiguous keys. + * Support dynamically change `delayed_write_rate` option via SetDBOptions(). + * Options::allow_concurrent_memtable_write and Options::enable_write_thread_adaptive_yield are now true by default. + * Remove Tickers::SEQUENCE_NUMBER to avoid confusion if statistics object is shared among RocksDB instance. Alternatively DB::GetLatestSequenceNumber() can be used to get the same value. + * Options.level0_stop_writes_trigger default value changes from 24 to 32. + * New compaction filter API: CompactionFilter::FilterV2(). Allows to drop ranges of keys. + * Removed flashcache support. + * DB::AddFile() is deprecated and is replaced with DB::IngestExternalFile(). DB::IngestExternalFile() remove all the restrictions that existed for DB::AddFile. + +### New Features + + * Add avoid_flush_during_shutdown option, which speeds up DB shutdown by not flushing unpersisted data (i.e. with disableWAL = true). Unpersisted data will be lost. The options is dynamically changeable via SetDBOptions(). + * Add memtable_insert_with_hint_prefix_extractor option. The option is mean to reduce CPU usage for inserting keys into memtable, if keys can be group by prefix and insert for each prefix are sequential or almost sequential. See include/rocksdb/options.h for more details. + * Add LuaCompactionFilter in utilities. This allows developers to write compaction filters in Lua. To use this feature, LUA_PATH needs to be set to the root directory of Lua. + * No longer populate "LATEST_BACKUP" file in backup directory, which formerly contained the number of the latest backup. The latest backup can be determined by finding the highest numbered file in the "meta/" subdirectory. diff --git a/docs/_posts/2017-02-07-rocksdb-5-1-2-released.markdown b/docs/_posts/2017-02-07-rocksdb-5-1-2-released.markdown new file mode 100644 index 00000000000..35bafb219cb --- /dev/null +++ b/docs/_posts/2017-02-07-rocksdb-5-1-2-released.markdown @@ -0,0 +1,15 @@ +--- +title: RocksDB 5.1.2 Released! +layout: post +author: maysamyabandeh +category: blog +--- + +### Public API Change +* Support dynamically change `delete_obsolete_files_period_micros` option via SetDBOptions(). +* Added EventListener::OnExternalFileIngested which will be called when IngestExternalFile() add a file successfully. +* BackupEngine::Open and BackupEngineReadOnly::Open now always return error statuses matching those of the backup Env. + +### Bug Fixes +* Fix the bug that if 2PC is enabled, checkpoints may loss some recent transactions. +* When file copying is needed when creating checkpoints or bulk loading files, fsync the file after the file copying. diff --git a/docs/_posts/2017-02-17-bulkoad-ingest-sst-file.markdown b/docs/_posts/2017-02-17-bulkoad-ingest-sst-file.markdown new file mode 100644 index 00000000000..9a43a846a4d --- /dev/null +++ b/docs/_posts/2017-02-17-bulkoad-ingest-sst-file.markdown @@ -0,0 +1,50 @@ +--- +title: Bulkloading by ingesting external SST files +layout: post +author: IslamAbdelRahman +category: blog +--- + +## Introduction + +One of the basic operations of RocksDB is writing to RocksDB, Writes happen when user call (DB::Put, DB::Write, DB::Delete ... ), but what happens when you write to RocksDB ? .. this is a brief description of what happens. +- User insert a new key/value by calling DB::Put() (or DB::Write()) +- We create a new entry for the new key/value in our in-memory structure (memtable / SkipList by default) and we assign it a new sequence number. +- When the memtable exceeds a specific size (64 MB for example), we convert this memtable to a SST file, and put this file in level 0 of our LSM-Tree +- Later, compaction will kick in and move data from level 0 to level 1, and then from level 1 to level 2 .. and so on + +But what if we can skip these steps and add data to the lowest possible level directly ? This is what bulk-loading does + +## Bulkloading + +- Write all of our keys and values into SST file outside of the DB +- Add the SST file into the LSM directly + +This is bulk-loading, and in specific use-cases it allow users to achieve faster data loading and better write-amplification. + +and doing it is as simple as +```cpp +Options options; +SstFileWriter sst_file_writer(EnvOptions(), options, options.comparator); +Status s = sst_file_writer.Open(file_path); +assert(s.ok()); + +// Insert rows into the SST file, note that inserted keys must be +// strictly increasing (based on options.comparator) +for (...) { + s = sst_file_writer.Add(key, value); + assert(s.ok()); +} + +// Ingest the external SST file into the DB +s = db_->IngestExternalFile({"/home/usr/file1.sst"}, IngestExternalFileOptions()); +assert(s.ok()); +``` + +You can find more details about how to generate SST files and ingesting them into RocksDB in this [wiki page](https://github.com/facebook/rocksdb/wiki/Creating-and-Ingesting-SST-files) + +## Use cases +There are multiple use cases where bulkloading could be useful, for example +- Generating SST files in offline jobs in Hadoop, then downloading and ingesting the SST files into RocksDB +- Migrating shards between machines by dumping key-range in SST File and loading the file in a different machine +- Migrating from a different storage (InnoDB to RocksDB migration in MyRocks) diff --git a/docs/_posts/2017-03-02-rocksdb-5-2-1-released.markdown b/docs/_posts/2017-03-02-rocksdb-5-2-1-released.markdown new file mode 100644 index 00000000000..c6ce27d64db --- /dev/null +++ b/docs/_posts/2017-03-02-rocksdb-5-2-1-released.markdown @@ -0,0 +1,22 @@ +--- +title: RocksDB 5.2.1 Released! +layout: post +author: sdong +category: blog +--- + +### Public API Change +* NewLRUCache() will determine number of shard bits automatically based on capacity, if the user doesn't pass one. This also impacts the default block cache when the user doesn't explict provide one. +* Change the default of delayed slowdown value to 16MB/s and further increase the L0 stop condition to 36 files. + +### New Features +* Added new overloaded function GetApproximateSizes that allows to specify if memtable stats should be computed only without computing SST files' stats approximations. +* Added new function GetApproximateMemTableStats that approximates both number of records and size of memtables. +* (Experimental) Two-level indexing that partition the index and creates a 2nd level index on the partitions. The feature can be enabled by setting kTwoLevelIndexSearch as IndexType and configuring index_per_partition. + +### Bug Fixes +* RangeSync() should work if ROCKSDB_FALLOCATE_PRESENT is not set +* Fix wrong results in a data race case in Get() +* Some fixes related to 2PC. +* Fix several bugs in Direct I/O supports. +* Fix a regression bug which can cause Seek() to miss some keys if the return key has been updated many times after the snapshot which is used by the iterator. diff --git a/docs/_posts/2017-05-12-partitioned-index-filter.markdown b/docs/_posts/2017-05-12-partitioned-index-filter.markdown new file mode 100644 index 00000000000..a537feb0c7e --- /dev/null +++ b/docs/_posts/2017-05-12-partitioned-index-filter.markdown @@ -0,0 +1,34 @@ +--- +title: Partitioned Index/Filters +layout: post +author: maysamyabandeh +category: blog +--- + +As DB/mem ratio gets larger, the memory footprint of filter/index blocks becomes non-trivial. Although `cache_index_and_filter_blocks` allows storing only a subset of them in block cache, their relatively large size negatively affects the performance by i) occupying the block cache space that could otherwise be used for caching data, ii) increasing the load on the disk storage by loading them into the cache after a miss. Here we illustrate these problems in more detail and explain how partitioning index/filters alleviates the overhead. + +### How large are the index/filter blocks? + +RocksDB has by default one index/filter block per SST file. The size of the index/filter varies based on the configuration but for a SST of size 256MB the index/filter block of size 0.5/5MB is typical, which is much larger than the typical data block size of 4-32KB. That is fine when all index/filters fit perfectly into memory and hence are read once per SST lifetime, not so much when they compete with data blocks for the block cache space and are also likely to be re-read many times from the disk. + +### What is the big deal with large index/filter blocks? + +When index/filter blocks are stored in block cache they are effectively competing with data blocks (as well as with each other) on this scarce resource. A filter of size 5MB is occupying the space that could otherwise be used to cache 1000s of data blocks (of size 4KB). This would result in more cache misses for data blocks. The large index/filters also kick each other out of the block cache more often and exacerbate their own cache miss rate too. This is while only a small part of the index/filter block might have been actually used during its lifetime in the cache. + +After the cache miss of an index/filter, it has to be reloaded from the disk, and its large size is not helping in reducing the IO cost. While a simple point lookup might need at most a couple of data block reads (of size 4KB) one from each layer of LSM, it might end up also loading multiple megabytes of index/filter blocks. If that happens often then the disk is spending more time serving index/filters rather than the actual data blocks. + +## What is partitioned index/filters? + +With partitioning, the index/filter of a SST file is partitioned into smaller blocks with an additional top-level index on them. When reading an index/filter, only top-level index is loaded into memory. The partitioned index/filter then uses the top-level index to load on demand into the block cache the partitions that are required to perform the index/filter query. The top-level index, which has much smaller memory footprint, can be stored in heap or block cache depending on the `cache_index_and_filter_blocks` setting. + +### Success stories + +#### HDD, 100TB DB + +In this example we have a DB of size 86G on HDD and emulate the small memory that is present to a node with 100TB of data by using direct IO (skipping OS file cache) and a very small block cache of size 60MB. Partitioning improves throughput by 11x from 5 op/s to 55 op/s. + +#### SSD, Linkbench + +In this example we have a DB of size 300G on SSD and emulate the small memory that would be available in presence of other DBs on the same node by by using direct IO (skipping OS file cache) and block cache of size 6G and 2G. Without partitioning the linkbench throughput drops from 38k tps to 23k when reducing block cache size from 6G to 2G. With partitioning the throughput drops from 38k to only 30k. + +Learn more [here](https://github.com/facebook/rocksdb/wiki/Partitioned-Index-Filters). diff --git a/docs/_posts/2017-05-14-core-local-stats.markdown b/docs/_posts/2017-05-14-core-local-stats.markdown new file mode 100644 index 00000000000..a806541fcad --- /dev/null +++ b/docs/_posts/2017-05-14-core-local-stats.markdown @@ -0,0 +1,106 @@ +--- +title: Core-local Statistics +layout: post +author: ajkr +category: blog +--- + +## Origins: Global Atomics + +Until RocksDB 4.12, ticker/histogram statistics were implemented with std::atomic values shared across the entire program. A ticker consists of a single atomic, while a histogram consists of several atomics to represent things like min/max/per-bucket counters. These statistics could be updated by all user/background threads. + +For concurrent/high-throughput workloads, cache line bouncing of atomics caused high CPU utilization. For example, we have tickers that count block cache hits and misses. Almost every user read increments these tickers a few times. Many concurrent user reads would cause the cache lines containing these atomics to bounce between cores. + +### Performance + +Here are perf results for 32 reader threads where most reads (99%+) are served by uncompressed block cache. Such a scenario stresses the statistics code heavily. + +Benchmark command: `TEST_TMPDIR=/dev/shm/ perf record -g ./db_bench -statistics -use_existing_db=true -benchmarks=readrandom -threads=32 -cache_size=1048576000 -num=1000000 -reads=1000000 && perf report -g --children` + +Perf snippet for "cycles" event: + +``` + Children Self Command Shared Object Symbol ++ 30.33% 30.17% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick ++ 3.65% 0.98% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +Perf snippet for "cache-misses" event: + +``` + Children Self Command Shared Object Symbol ++ 19.54% 19.50% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick ++ 3.44% 0.57% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +The high CPU overhead for updating tickers and histograms corresponds well to the high cache misses. + +## Thread-locals: Faster Updates + +Since RocksDB 4.12, ticker/histogram statistics use thread-local storage. Each thread has a local set of atomic values that no other thread can update. This prevents the cache line bouncing problem described above. Even though updates to a given value are always made by the same thread, atomics are still useful to synchronize with aggregations for querying statistics. + +Implementing this approach involved a couple challenges. First, each query for a statistic's global value must aggregate all threads' local values. This adds some overhead, which may pass unnoticed if statistics are queried infrequently. Second, exited threads' local values are still needed to provide accurate statistics. We handle this by merging a thread's local values into process-wide variables upon thread exit. + +### Performance + +Update benchmark setup is same as before. CPU overhead improved 7.8x compared to global atomics, corresponding to a 17.8x reduction in cache-misses overhead. + +Perf snippet for "cycles" event: + +``` + Children Self Command Shared Object Symbol ++ 2.96% 0.87% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick ++ 1.37% 0.10% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +Perf snippet for "cache-misses" event: + +``` + Children Self Command Shared Object Symbol ++ 1.21% 0.65% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick + 0.08% 0.00% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +To measure statistics query latency, we ran sysbench with 4K OLTP clients concurrently with one client that queries statistics repeatedly. Times shown are in milliseconds. + +``` + min: 18.45 + avg: 27.91 + max: 231.65 + 95th percentile: 55.82 +``` + +## Core-locals: Faster Querying + +The thread-local approach is working well for applications calling RocksDB from only a few threads, or polling statistics infrequently. Eventually, though, we found use cases where those assumptions do not hold. For example, one application has per-connection threads and typically runs into performance issues when connection count grows very high. For debugging such issues, they want high-frequency statistics polling to correlate issues in their application with changes in RocksDB's state. + +Once [PR #2258](https://github.com/facebook/rocksdb/pull/2258) lands, ticker/histogram statistics will be local to each CPU core. Similarly to thread-local, each core updates only its local values, thus avoiding cache line bouncing. Local values are still atomics to make aggregation possible. With this change, query work depends only on number of cores, not the number of threads. So, applications with many more threads than cores can no longer impact statistics query latency. + +### Performance + +Update benchmark setup is same as before. CPU overhead worsened ~23% compared to thread-local, while cache performance was unchanged. + +Perf snippet for "cycles" event: + +``` + Children Self Command Shared Object Symbol ++ 2.96% 0.87% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick ++ 1.37% 0.10% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +Perf snippet for "cache-misses" event: + +``` + Children Self Command Shared Object Symbol ++ 1.21% 0.65% db_bench db_bench [.] rocksdb::StatisticsImpl::recordTick + 0.08% 0.00% db_bench db_bench [.] rocksdb::StatisticsImpl::measureTime +``` + +Query latency is measured same as before with times in milliseconds. Average latency improved by 6.3x compared to thread-local. + +``` + min: 2.47 + avg: 4.45 + max: 91.13 + 95th percentile: 7.56 +``` diff --git a/docs/_posts/2017-05-26-rocksdb-5-4-5-released.markdown b/docs/_posts/2017-05-26-rocksdb-5-4-5-released.markdown new file mode 100644 index 00000000000..561dab4c207 --- /dev/null +++ b/docs/_posts/2017-05-26-rocksdb-5-4-5-released.markdown @@ -0,0 +1,39 @@ +--- +title: RocksDB 5.4.5 Released! +layout: post +author: sagar0 +category: blog +--- + +### Public API Change +* Support dynamically changing `stats_dump_period_sec` option via SetDBOptions(). +* Added ReadOptions::max_skippable_internal_keys to set a threshold to fail a request as incomplete when too many keys are being skipped while using iterators. +* DB::Get in place of std::string accepts PinnableSlice, which avoids the extra memcpy of value to std::string in most of cases. + * PinnableSlice releases the pinned resources that contain the value when it is destructed or when ::Reset() is called on it. + * The old API that accepts std::string, although discouraged, is still supported. +* Replace Options::use_direct_writes with Options::use_direct_io_for_flush_and_compaction. See Direct IO wiki for details. + +### New Features +* Memtable flush can be avoided during checkpoint creation if total log file size is smaller than a threshold specified by the user. +* Introduce level-based L0->L0 compactions to reduce file count, so write delays are incurred less often. +* (Experimental) Partitioning filters which creates an index on the partitions. The feature can be enabled by setting partition_filters when using kFullFilter. Currently the feature also requires two-level indexing to be enabled. Number of partitions is the same as the number of partitions for indexes, which is controlled by metadata_block_size. +* DB::ResetStats() to reset internal stats. +* Added CompactionEventListener and EventListener::OnFlushBegin interfaces. +* Added DB::CreateColumnFamilie() and DB::DropColumnFamilies() to bulk create/drop column families. +* Facility for cross-building RocksJava using Docker. + +### Bug Fixes +* Fix WriteBatchWithIndex address use after scope error. +* Fix WritableFile buffer size in direct IO. +* Add prefetch to PosixRandomAccessFile in buffered io. +* Fix PinnableSlice access invalid address when row cache is enabled. +* Fix huge fallocate calls fail and make XFS unhappy. +* Fix memory alignment with logical sector size. +* Fix alignment in ReadaheadRandomAccessFile. +* Fix bias with read amplification stats (READ_AMP_ESTIMATE_USEFUL_BYTES and READ_AMP_TOTAL_READ_BYTES). +* Fix a manual / auto compaction data race. +* Fix CentOS 5 cross-building of RocksJava. +* Build and link with ZStd when creating the static RocksJava build. +* Fix snprintf's usage to be cross-platform. +* Fix build errors with blob DB. +* Fix readamp test type inconsistency. diff --git a/docs/_posts/2017-06-26-17-level-based-changes.markdown b/docs/_posts/2017-06-26-17-level-based-changes.markdown new file mode 100644 index 00000000000..9e838eb7f2b --- /dev/null +++ b/docs/_posts/2017-06-26-17-level-based-changes.markdown @@ -0,0 +1,60 @@ +--- +title: Level-based Compaction Changes +layout: post +author: ajkr +category: blog +--- + +### Introduction + +RocksDB provides an option to limit the number of L0 files, which bounds read-amplification. Since L0 files (unlike files at lower levels) can span the entire key-range, a key might be in any file, thus reads need to check them one-by-one. Users often wish to configure a low limit to improve their read latency. + +Although, the mechanism with which we enforce L0's file count limit may be unappealing. When the limit is reached, RocksDB intentionally delays user writes. This slows down accumulation of files in L0, and frees up resources for compacting files down to lower levels. But adding delays will significantly increase user-visible write latency jitter. + +Also, due to how L0 files can span the entire key-range, compaction parallelization is limited. Files at L0 or L1 may be locked due to involvement in pending L0->L1 or L1->L2 compactions. We can only schedule a parallel L0->L1 compaction if it does not require any of the locked files, which is typically not the case. + +To handle these constraints better, we added a new type of compaction, L0->L0. It quickly reduces file count in L0 and can be scheduled even when L1 files are locked, unlike L0->L1. We also changed the L0->L1 picking algorithm to increase opportunities for parallelism. + +### Old L0->L1 Picking Logic + +Previously, our logic for picking which L0 file to compact was the same as every other level: pick the largest file in the level. One special property of L0->L1 compaction is that files can overlap in the input level, so those overlapping files must be pulled in as well. For example, a compaction may look like this: + +![full-range.png](/static/images/compaction/full-range.png) + +This compaction pulls in every L0 and L1 file. This happens regardless of which L0 file is initially chosen as each file overlaps with every other file. + +Users may insert their data less uniformly in the key-range. For example, a database may look like this during L0->L1 compaction: + +![part-range-old.png](/static/images/compaction/part-range-old.png) + +Let's say the third file from the top is the largest, and let's say the top two files are created after the compaction started. When the compaction is picked, the fourth L0 file and six rightmost L1 files are pulled in due to overlap. Notice this leaves the database in a state where we might not be able to schedule parallel compactions. For example, if the sixth file from the top is the next largest, we can't compact it because it overlaps with the top two files, which overlap with the locked L0 files. + +We can now see the high-level problems with this approach more clearly. First, locked files in L0 or L1 prevent us from parallelizing compactions. When locked files block L0->L1 compaction, there is nothing we can do to eliminate L0 files. Second, L0->L1 compactions are relatively slow. As we saw, when keys are uniformly distributed, L0->L1 compacts two entire levels. While this is happening, new files are being flushed to L0, advancing towards the file count limit. + +### New L0->L0 Algorithm + +We introduced compaction within L0 to improve both parallelization and speed of reducing L0 file count. An L0->L0 compaction may look like this: + +![l1-l2-contend.png](/static/images/compaction/l1-l2-contend.png) + +Say the L1->L2 compaction started first. Now L0->L1 is prevented by the locked L1 file. In this case, we compact files within L0. This allows us to start the work for eliminating L0 files earlier. It also lets us do less work since we don't pull in any L1 files, whereas L0->L1 compaction would've pulled in all of them. This lets us quickly reduce L0 file count to keep read-amp low while sustaining large bursts of writes (i.e., fast accumulation of L0 files). + +The tradeoff is this increases total compaction work, as we're now compacting files without contributing towards our eventual goal of moving them towards lower levels. Our benchmarks, though, consistently show less compaction stalls and improved write throughput. One justification is that L0 file data is highly likely in page cache and/or block cache due to it being recently written and frequently accessed. So, this type of compaction is relatively cheap compared to compactions at lower levels. + +This feature is available since RocksDB 5.4. + +### New L0->L1 Picking Logic + +Recall how the old L0->L1 picking algorithm chose the largest L0 file for compaction. This didn't fit well with L0->L0 compaction, which operates on a span of files. That span begins at the newest L0 file, and expands towards older files as long as they're not being compacted. Since the largest file may be anywhere, the old L0->L1 picking logic could arbitrarily prevent us from getting a long span of files. See the second illustration in this post for a scenario where this would happen. + +So, we changed the L0->L1 picking algorithm to start from the oldest file and expand towards newer files as long as they're not being compacted. For example: + +![l0-l1-contend.png](/static/images/compaction/l0-l1-contend.png) + +Now, there can never be L0 files unreachable for L0->L0 due to L0->L1 selecting files in the middle. When longer spans of files are available for L0->L0, we perform less compaction work per deleted L0 file, thus improving efficiency. + +This feature will be available in RocksDB 5.7. + +### Performance Changes + +Mark Callaghan did the most extensive benchmarking of this feature's impact on MyRocks. See his results [here](http://smalldatum.blogspot.com/2017/05/innodb-myrocks-and-tokudb-on-insert.html). Note the primary change between his March 17 and April 14 builds is the latter performs L0->L0 compaction. diff --git a/docs/_posts/2017-06-29-rocksdb-5-5-1-released.markdown b/docs/_posts/2017-06-29-rocksdb-5-5-1-released.markdown new file mode 100644 index 00000000000..d7856088bf8 --- /dev/null +++ b/docs/_posts/2017-06-29-rocksdb-5-5-1-released.markdown @@ -0,0 +1,22 @@ +--- +title: RocksDB 5.5.1 Released! +layout: post +author: lightmark +category: blog +--- + +### New Features +* FIFO compaction to support Intra L0 compaction too with CompactionOptionsFIFO.allow_compaction=true. +* Statistics::Reset() to reset user stats. +* ldb add option --try_load_options, which will open DB with its own option file. +* Introduce WriteBatch::PopSavePoint to pop the most recent save point explicitly. +* Support dynamically change `max_open_files` option via SetDBOptions() +* Added DB::CreateColumnFamilie() and DB::DropColumnFamilies() to bulk create/drop column families. +* Add debugging function `GetAllKeyVersions` to see internal versions of a range of keys. +* Support file ingestion with universal compaction style +* Support file ingestion behind with option `allow_ingest_behind` +* New option enable_pipelined_write which may improve write throughput in case writing from multiple threads and WAL enabled. + +### Bug Fixes +* Fix the bug that Direct I/O uses direct reads for non-SST file +* Fix the bug that flush doesn't respond to fsync result diff --git a/docs/_posts/2017-07-25-rocksdb-5-6-1-released.markdown b/docs/_posts/2017-07-25-rocksdb-5-6-1-released.markdown new file mode 100644 index 00000000000..3b54ffd5ad8 --- /dev/null +++ b/docs/_posts/2017-07-25-rocksdb-5-6-1-released.markdown @@ -0,0 +1,22 @@ +--- +title: RocksDB 5.6.1 Released! +layout: post +author: yiwu +category: blog +--- + +### Public API Change +* Scheduling flushes and compactions in the same thread pool is no longer supported by setting `max_background_flushes=0`. Instead, users can achieve this by configuring their high-pri thread pool to have zero threads. See https://github.com/facebook/rocksdb/wiki/Thread-Pool for more details. +* Replace `Options::max_background_flushes`, `Options::max_background_compactions`, and `Options::base_background_compactions` all with `Options::max_background_jobs`, which automatically decides how many threads to allocate towards flush/compaction. +* options.delayed_write_rate by default take the value of options.rate_limiter rate. +* Replace global variable `IOStatsContext iostats_context` with `IOStatsContext* get_iostats_context()`; replace global variable `PerfContext perf_context` with `PerfContext* get_perf_context()`. + +### New Features +* Change ticker/histogram statistics implementations to use core-local storage. This improves aggregation speed compared to our previous thread-local approach, particularly for applications with many threads. See http://rocksdb.org/blog/2017/05/14/core-local-stats.html for more details. +* Users can pass a cache object to write buffer manager, so that they can cap memory usage for memtable and block cache using one single limit. +* Flush will be triggered when 7/8 of the limit introduced by write_buffer_manager or db_write_buffer_size is triggered, so that the hard threshold is hard to hit. See https://github.com/facebook/rocksdb/wiki/Write-Buffer-Manager for more details. +* Introduce WriteOptions.low_pri. If it is true, low priority writes will be throttled if the compaction is behind. See https://github.com/facebook/rocksdb/wiki/Low-Priority-Write for more details. +* `DB::IngestExternalFile()` now supports ingesting files into a database containing range deletions. + +### Bug Fixes +* Shouldn't ignore return value of fsync() in flush. diff --git a/docs/_posts/2017-08-24-pinnableslice.markdown b/docs/_posts/2017-08-24-pinnableslice.markdown new file mode 100644 index 00000000000..7ac2fec34be --- /dev/null +++ b/docs/_posts/2017-08-24-pinnableslice.markdown @@ -0,0 +1,37 @@ +--- +title: PinnableSlice; less memcpy with point lookups +layout: post +author: maysamyabandeh +category: blog +--- + +The classic API for [DB::Get](https://github.com/facebook/rocksdb/blob/9e583711144f580390ce21a49a8ceacca338fcd5/include/rocksdb/db.h#L310) receives a std::string as argument to which it will copy the value. The memcpy overhead could be non-trivial when the value is large. The [new API](https://github.com/facebook/rocksdb/blob/9e583711144f580390ce21a49a8ceacca338fcd5/include/rocksdb/db.h#L322) receives a PinnableSlice instead, which avoids memcpy in most of the cases. + +### What is PinnableSlice? + +Similarly to Slice, PinnableSlice refers to some in-memory data so it does not incur the memcpy cost. To ensure that the data will not be erased while it is being processed by the user, PinnableSlice, as its name suggests, has the data pinned in memory. The pinned data are released when PinnableSlice object is destructed or when ::Reset is invoked explicitly on it. + +### How good is it? + +Here are the improvements in throughput for an [in-memory benchmark](https://github.com/facebook/rocksdb/pull/1756#issuecomment-286201693): +* value 1k byte: 14% +* value 10k byte: 34% + +### Any limitations? + +PinnableSlice tries to avoid memcpy as much as possible. The primary gain is when reading large values from the block cache. There are however cases that it would still have to copy the data into its internal buffer. The reason is mainly the complexity of implementation and if there is enough motivation on the application side. the scope of PinnableSlice could be extended to such cases too. These include: +* Merged values +* Reads from memtables + +### How to use it? + +```cpp +PinnableSlice pinnable_val; +while (!stopped) { + auto s = db->Get(opt, cf, key, &pinnable_val); + // ... use it + pinnable_val.Reset(); // then release it immediately +} +``` + +You can also [initialize the internal buffer](https://github.com/facebook/rocksdb/blob/9e583711144f580390ce21a49a8ceacca338fcd5/include/rocksdb/db.h#L314) of PinnableSlice by passing your own string in the constructor. [simple_example.cc](https://github.com/facebook/rocksdb/blob/master/examples/simple_example.cc) demonstrates that with more examples. diff --git a/docs/_posts/2017-08-25-flushwal.markdown b/docs/_posts/2017-08-25-flushwal.markdown new file mode 100644 index 00000000000..2dc5626ad48 --- /dev/null +++ b/docs/_posts/2017-08-25-flushwal.markdown @@ -0,0 +1,26 @@ +--- +title: FlushWAL; less fwrite, faster writes +layout: post +author: maysamyabandeh +category: blog +--- + +When `DB::Put` is called, the data is written to both memtable (to be flushed to SST files later) and the WAL (write-ahead log) if it is enabled. In the case of a crash, RocksDB can recover as much as the memtable state that is reflected into the WAL. By default RocksDB automatically flushes the WAL from the application memory to the OS buffer after each `::Put`. It however can be configured to perform the flush manually after an explicit call to `::FlushWAL`. Not doing fwrite syscall after each `::Put` offers a tradeoff between reliability and write latency for the general case. As we explain below, some applications such as MyRocks benefit from this API to gain higher write throughput with however no compromise in reliability. + +### How much is the gain? + +Using `::FlushWAL` API along with setting `DBOptions.concurrent_prepare`, MyRocks achieves 40% higher throughput in Sysbench's [update-nonindex](https://github.com/akopytov/sysbench/blob/master/src/lua/oltp_update_non_index.lua) benchmark. + +### Write, Flush, and Sync + +The write to the WAL is first written to the application memory buffer. The buffer in the next step is "flushed" to OS buffer by calling fwrite syscall. The OS buffer is later "synced" to the persistent storage. The data in the OS buffer, although not persisted yet, will survive the application crash. By default, the flush occurs automatically upon each call to `DB::Put` or `DB::Write`. The user can additionally request sync after each write by setting `WriteOptions::sync`. + +### FlushWAL API + +The user can turn off the automatic flush of the WAL by setting `DBOptions::manual_wal_flush`. In that case, the WAL buffer is flushed when it is either full or `DB::FlushWAL` is called by the user. The API also accepts a boolean argument should we want to sync right after the flush: `::FlushWAL(true)`. + +### Success story: MyRocks + +Some applications that use RocksDB, already have other machinsims in place to provide reliability. MySQL for example uses 2PC (two-phase commit) to write to both binlog as well as the storage engine such as InnoDB and MyRocks. The group commit logic in MySQL allows the 1st phase (Prepare) to be run in parallel but after a commit group is formed performs the 2nd phase (Commit) in a serial manner. This makes low commit latency in the storage engine essential for acheiving high throughput. The commit in MyRocks includes writing to the RocksDB WAL, which as explaiend above, by default incures the latency of flushing the WAL new appends to the OS buffer. + +Since binlog helps in recovering from some failure scenarios, MySQL can provide reliability without however needing a storage WAL flush after each individual commit. MyRocks benefits from this property, disables automatic WAL flush in RocksDB, and manually calls `::FlushWAL` when requested by MySQL. diff --git a/docs/_sass/_base.scss b/docs/_sass/_base.scss new file mode 100644 index 00000000000..6d26d9feba6 --- /dev/null +++ b/docs/_sass/_base.scss @@ -0,0 +1,492 @@ +body { + background: $secondary-bg; + color: $text; + font: normal #{$base-font-size}/#{$base-line-height} $base-font-family; + height: 100vh; + text-align: left; + text-rendering: optimizeLegibility; +} + +img { + max-width: 100%; +} + +article { + p { + img { + max-width: 100%; + display:block; + margin-left: auto; + margin-right: auto; + } + } +} + +a { + border-bottom: 1px dotted $primary-bg; + color: $text; + text-decoration: none; + -webkit-transition: background 0.3s, color 0.3s; + transition: background 0.3s, color 0.3s; +} + +blockquote { + padding: 15px 30px 15px 15px; + margin: 20px 0 0 10px; + background-color: rgba(204, 122, 111, 0.1); + border-left: 10px solid rgba(191, 87, 73, 0.2); +} + +#fb_oss a { + border: 0; +} + +h1, h2, h3, h4 { + font-family: $header-font-family; + font-weight: 900; +} + +.navPusher { + border-top: $header-height + $header-ptop + $header-pbot solid $primary-bg; + height: 100%; + left: 0; + position: relative; + z-index: 99; +} + +.homeContainer { + background: $primary-bg; + color: $primary-overlay; + + a { + color: $primary-overlay; + } + + .homeSplashFade { + color: white; + } + + .homeWrapper { + padding: 2em 10px; + text-align: left; + + .wrapper { + margin: 0px auto; + max-width: $content-width; + padding: 0 20px; + } + + .projectLogo { + img { + height: 100px; + margin-bottom: 0px; + } + } + + h1#project_title { + font-family: $header-font-family; + font-size: 300%; + letter-spacing: -0.08em; + line-height: 1em; + margin-bottom: 80px; + } + + h2#project_tagline { + font-family: $header-font-family; + font-size: 200%; + letter-spacing: -0.04em; + line-height: 1em; + } + } +} + +.wrapper { + margin: 0px auto; + max-width: $content-width; + padding: 0 10px; +} + +.projectLogo { + display: none; + + img { + height: 100px; + margin-bottom: 0px; + } +} + +section#intro { + margin: 40px 0; +} + +.fbossFontLight { + font-family: $base-font-family; + font-weight: 300; + font-style: normal; +} + +.fb-like { + display: block; + margin-bottom: 20px; + width: 100%; +} + +.center { + display: block; + text-align: center; +} + +.mainContainer { + background: $secondary-bg; + overflow: auto; + + .mainWrapper { + padding: 4vh 10px; + text-align: left; + + .allShareBlock { + padding: 10px 0; + + .pluginBlock { + margin: 12px 0; + padding: 0; + } + } + + a { + &:hover, + &:focus { + background: $primary-bg; + color: $primary-overlay; + } + } + + em, i { + font-style: italic; + } + + strong, b { + font-weight: bold; + } + + h1 { + font-size: 300%; + line-height: 1em; + padding: 1.4em 0 1em; + text-align: left; + } + + h2 { + font-size: 250%; + line-height: 1em; + margin-bottom: 20px; + padding: 1.4em 0 20px; + text-align: left; + + & { + border-bottom: 1px solid darken($primary-bg, 10%); + color: darken($primary-bg, 10%); + font-size: 22px; + padding: 10px 0; + } + + &.blockHeader { + border-bottom: 1px solid white; + color: white; + font-size: 22px; + margin-bottom: 20px; + padding: 10px 0; + } + } + + h3 { + font-size: 150%; + line-height: 1.2em; + padding: 1em 0 0.8em; + } + + h4 { + font-size: 130%; + line-height: 1.2em; + padding: 1em 0 0.8em; + } + + p { + padding: 0.8em 0; + } + + ul { + list-style: disc; + } + + ol, ul { + padding-left: 24px; + li { + padding-bottom: 4px; + padding-left: 6px; + } + } + + strong { + font-weight: bold; + } + + .post { + position: relative; + + .katex { + font-weight: 700; + } + + &.basicPost { + margin-top: 30px; + } + + a { + color: $primary-bg; + + &:hover, + &:focus { + color: #fff; + } + } + + h2 { + border-bottom: 4px solid $primary-bg; + font-size: 130%; + } + + h3 { + border-bottom: 1px solid $primary-bg; + font-size: 110%; + } + + ol { + list-style: decimal outside none; + } + + .post-header { + padding: 1em 0; + + h1 { + font-size: 150%; + line-height: 1em; + padding: 0.4em 0 0; + + a { + border: none; + } + } + + .post-meta { + color: $primary-bg; + font-family: $header-font-family; + text-align: center; + } + } + + .postSocialPlugins { + padding-top: 1em; + } + + .docPagination { + background: $primary-bg; + bottom: 0px; + left: 0px; + position: absolute; + right: 0px; + + .pager { + display: inline-block; + width: 50%; + } + + .pagingNext { + float: right; + text-align: right; + } + + a { + border: none; + color: $primary-overlay; + display: block; + padding: 4px 12px; + + &:hover { + background-color: $secondary-bg; + color: $text; + } + + .pagerLabel { + display: inline; + } + + .pagerTitle { + display: none; + } + } + } + } + + .posts { + .post { + margin-bottom: 6vh; + } + } + } +} + +#integrations_title { + font-size: 250%; + margin: 80px 0; +} + +.ytVideo { + height: 0; + overflow: hidden; + padding-bottom: 53.4%; /* 16:9 */ + padding-top: 25px; + position: relative; +} + +.ytVideo iframe, +.ytVideo object, +.ytVideo embed { + height: 100%; + left: 0; + position: absolute; + top: 0; + width: 100%; +} + +@media only screen and (min-width: 480px) { + h1#project_title { + font-size: 500%; + } + + h2#project_tagline { + font-size: 250%; + } + + .projectLogo { + img { + margin-bottom: 10px; + height: 200px; + } + } + + .homeContainer .homeWrapper { + padding-left: 10px; + padding-right: 10px; + } + + .mainContainer { + .mainWrapper { + .post { + h2 { + font-size: 180%; + } + + h3 { + font-size: 120%; + } + + .docPagination { + a { + .pagerLabel { + display: none; + } + .pagerTitle { + display: inline; + } + } + } + } + } + } +} + +@media only screen and (min-width: 900px) { + .homeContainer { + .homeWrapper { + position: relative; + + #inner { + box-sizing: border-box; + max-width: 600px; + padding-right: 40px; + } + + .projectLogo { + align-items: center; + bottom: 0; + display: flex; + justify-content: flex-end; + left: 0; + padding: 2em 20px 4em; + position: absolute; + right: 20px; + top: 0; + + img { + height: 100%; + max-height: 250px; + } + } + } + } +} + +@media only screen and (min-width: 1024px) { + .mainContainer { + .mainWrapper { + .post { + box-sizing: border-box; + display: block; + + .post-header { + h1 { + font-size: 250%; + } + } + } + + .posts { + .post { + margin-bottom: 4vh; + width: 100%; + } + } + } + } +} + +@media only screen and (min-width: 1200px) { + .homeContainer { + .homeWrapper { + #inner { + max-width: 750px; + } + } + } + + .wrapper { + max-width: 1100px; + } +} + +@media only screen and (min-width: 1500px) { + .homeContainer { + .homeWrapper { + #inner { + max-width: 1100px; + padding-bottom: 40px; + padding-top: 40px; + } + } + } + + .wrapper { + max-width: 1400px; + } +} diff --git a/docs/_sass/_blog.scss b/docs/_sass/_blog.scss new file mode 100644 index 00000000000..74335d10b41 --- /dev/null +++ b/docs/_sass/_blog.scss @@ -0,0 +1,45 @@ +.blogContainer { + .posts { + margin-top: 60px; + + .post { + border: 1px solid $primary-bg; + border-radius: 3px; + padding: 10px 20px 20px; + } + } + + .lonePost { + margin-top: 60px; + + .post { + padding: 10px 0px 0px; + } + } + + .post-header { + h1 { + text-align: center; + } + + .post-authorName { + color: rgba($text, 0.7); + font-size: 14px; + font-weight: 900; + margin-top: 0; + padding: 0; + text-align: center; + } + + .authorPhoto { + border-radius: 50%; + height: 50px; + left: 50%; + margin-left: -25px; + overflow: hidden; + position: absolute; + top: -25px; + width: 50px; + } + } +} \ No newline at end of file diff --git a/docs/_sass/_buttons.scss b/docs/_sass/_buttons.scss new file mode 100644 index 00000000000..a0371618fc4 --- /dev/null +++ b/docs/_sass/_buttons.scss @@ -0,0 +1,47 @@ +.button { + border: 1px solid $primary-bg; + border-radius: 3px; + color: $primary-bg; + display: inline-block; + font-size: 14px; + font-weight: 900; + line-height: 1.2em; + padding: 10px; + text-transform: uppercase; + transition: background 0.3s, color 0.3s; + + &:hover { + background: $primary-bg; + color: $primary-overlay; + } +} + +.homeContainer { + .button { + border-color: $primary-overlay; + border-width: 1px; + color: $primary-overlay; + + &:hover { + background: $primary-overlay; + color: $primary-bg; + } + } +} + +.blockButton { + display: block; +} + +.edit-page-link { + float: right; + font-size: 14px; + font-weight: normal; + line-height: 20px; + opacity: 0.6; + transition: opacity 0.5s; +} + +.edit-page-link:hover { + opacity: 1; +} diff --git a/docs/_sass/_footer.scss b/docs/_sass/_footer.scss new file mode 100644 index 00000000000..5b743951799 --- /dev/null +++ b/docs/_sass/_footer.scss @@ -0,0 +1,82 @@ +.footerContainer { + background: $secondary-bg; + color: $primary-bg; + overflow: hidden; + padding: 0 10px; + text-align: left; + + .footerWrapper { + border-top: 1px solid $primary-bg; + padding: 0; + + .footerBlocks { + align-items: center; + align-content: center; + display: flex; + flex-flow: row wrap; + margin: 0 -20px; + padding: 10px 0; + } + + .footerSection { + box-sizing: border-box; + flex: 1 1 25%; + font-size: 14px; + min-width: 275px; + padding: 0px 20px; + + a { + border: 0; + color: inherit; + display: inline-block; + line-height: 1.2em; + } + + .footerLink { + padding-right: 20px; + } + } + + .fbOpenSourceFooter { + align-items: center; + display: flex; + flex-flow: row nowrap; + max-width: 25%; + + .facebookOSSLogoSvg { + flex: 0 0 31px; + height: 30px; + margin-right: 10px; + width: 31px; + + path { + fill: $primary-bg; + } + + .middleRing { + opacity: 0.7; + } + + .innerRing { + opacity: 0.45; + } + } + + h2 { + display: block; + font-weight: 900; + line-height: 1em; + } + } + } +} + +@media only screen and (min-width: 900px) { + .footerSection { + &.rightAlign { + margin-left: auto; + max-width: 25%; + text-align: right; + } + } +} \ No newline at end of file diff --git a/docs/_sass/_gridBlock.scss b/docs/_sass/_gridBlock.scss new file mode 100644 index 00000000000..679b31c14c5 --- /dev/null +++ b/docs/_sass/_gridBlock.scss @@ -0,0 +1,115 @@ +.gridBlock { + margin: -5px 0; + padding: 0; + padding-bottom: 20px; + + .blockElement { + padding: 5px 0; + + img { + max-width: 100%; + } + + h3 { + border-bottom: 1px solid rgba($primary-bg, 0.5); + color: $primary-bg; + font-size: 18px; + margin: 0; + padding: 10px 0; + } + } + + .gridClear { + clear: both; + } + +} + +.gridBlock .alignCenter { + text-align: center; +} +.gridBlock .alignRight { + text-align: right; +} +.gridBlock .imageAlignSide { + align-items: center; + display: flex; + flex-flow: row wrap; +} +.blockImage { + max-width: 150px; + width: 50%; +} +.imageAlignTop .blockImage { + margin-bottom: 20px; +} +.imageAlignTop.alignCenter .blockImage { + margin-left: auto; + margin-right: auto; +} +.imageAlignSide .blockImage { + flex: 0 1 100px; + margin-right: 20px; +} +.imageAlignSide .blockContent { + flex: 1 1; +} + +@media only screen and (max-width: 1023px) { + .responsiveList .blockContent { + position: relative; + } + .responsiveList .blockContent > div { + padding-left: 20px; + } + .responsiveList .blockContent::before { + content: "\2022"; + position: absolute; + } +} + +@media only screen and (min-width: 1024px) { + .gridBlock { + display: flex; + flex-direction: row; + flex-wrap: wrap; + margin: -10px -10px 10px -10px; + + .twoByGridBlock { + box-sizing: border-box; + flex: 1 0 50%; + padding: 10px; + } + + .fourByGridBlock { + box-sizing: border-box; + flex: 1 0 25%; + padding: 10px; + } + } + + h2 + .gridBlock { + padding-top: 20px; + } +} + +@media only screen and (min-width: 1400px) { + .gridBlock { + display: flex; + flex-direction: row; + flex-wrap: wrap; + margin: -10px -20px 10px -20px; + + .twoByGridBlock { + box-sizing: border-box; + flex: 1 0 50%; + padding: 10px 20px; + } + + .fourByGridBlock { + box-sizing: border-box; + flex: 1 0 25%; + padding: 10px 20px; + } + } +} \ No newline at end of file diff --git a/docs/_sass/_header.scss b/docs/_sass/_header.scss new file mode 100644 index 00000000000..b4cd0711397 --- /dev/null +++ b/docs/_sass/_header.scss @@ -0,0 +1,138 @@ +.fixedHeaderContainer { + background: $primary-bg; + color: $primary-overlay; + height: $header-height; + padding: $header-ptop 0 $header-pbot; + position: fixed; + width: 100%; + z-index: 9999; + + a { + align-items: center; + border: 0; + color: $primary-overlay; + display: flex; + flex-flow: row nowrap; + height: $header-height; + } + + header { + display: flex; + flex-flow: row nowrap; + position: relative; + text-align: left; + + img { + height: 24px; + margin-right: 10px; + } + + h2 { + display: block; + font-family: $header-font-family; + font-weight: 900; + line-height: 18px; + position: relative; + } + } +} + +.navigationFull { + height: 34px; + margin-left: auto; + + nav { + position: relative; + + ul { + display: flex; + flex-flow: row nowrap; + margin: 0 -10px; + + li { + padding: 0 10px; + display: block; + + a { + border: 0; + color: $primary-overlay-special; + font-size: 16px; + font-weight: 400; + line-height: 1.2em; + + &:hover { + border-bottom: 2px solid $primary-overlay; + color: $primary-overlay; + } + } + + &.navItemActive { + a { + color: $primary-overlay; + } + } + } + } + } +} + +/* 900px + + + .fixedHeaderContainer { + .navigationWrapper { + nav { + padding: 0 1em; + position: relative; + top: -9px; + + ul { + margin: 0 -0.4em; + li { + display: inline-block; + + a { + padding: 14px 0.4em; + border: 0; + color: $primary-overlay-special; + display: inline-block; + + &:hover { + color: $primary-overlay; + } + } + + &.navItemActive { + a { + color: $primary-overlay; + } + } + } + } + } + + &.navigationFull { + display: inline-block; + } + + &.navigationSlider { + display: none; + } + } + } + + 1200px + + .fixedHeaderContainer { + header { + max-width: 1100px; + } + } + + 1500px + .fixedHeaderContainer { + header { + max-width: 1400px; + } + } + */ \ No newline at end of file diff --git a/docs/_sass/_poweredby.scss b/docs/_sass/_poweredby.scss new file mode 100644 index 00000000000..4155b605361 --- /dev/null +++ b/docs/_sass/_poweredby.scss @@ -0,0 +1,69 @@ +.poweredByContainer { + background: $primary-bg; + color: $primary-overlay; + margin-bottom: 20px; + + a { + color: $primary-overlay; + } + + .poweredByWrapper { + h2 { + border-color: $primary-overlay-special; + color: $primary-overlay-special; + } + } + + .poweredByMessage { + color: $primary-overlay-special; + font-size: 14px; + padding-top: 20px; + } +} + +.poweredByItems { + display: flex; + flex-flow: row wrap; + margin: 0 -10px; +} + +.poweredByItem { + box-sizing: border-box; + flex: 1 0 50%; + line-height: 1.1em; + padding: 5px 10px; + + &.itemLarge { + flex-basis: 100%; + padding: 10px; + text-align: center; + + &:nth-child(4) { + padding-bottom: 20px; + } + + img { + max-height: 30px; + } + } +} + +@media only screen and (min-width: 480px) { + .itemLarge { + flex-basis: 50%; + max-width: 50%; + } +} + +@media only screen and (min-width: 1024px) { + .poweredByItem { + flex-basis: 25%; + max-width: 25%; + + &.itemLarge { + padding-bottom: 20px; + text-align: left; + } + } +} + diff --git a/docs/_sass/_promo.scss b/docs/_sass/_promo.scss new file mode 100644 index 00000000000..8c9a809dcb1 --- /dev/null +++ b/docs/_sass/_promo.scss @@ -0,0 +1,55 @@ +.promoSection { + display: flex; + flex-flow: column wrap; + font-size: 125%; + line-height: 1.6em; + margin: -10px 0; + position: relative; + z-index: 99; + + .promoRow { + padding: 10px 0; + + .pluginWrapper { + display: block; + + &.ghWatchWrapper, &.ghStarWrapper { + height: 28px; + } + } + + .pluginRowBlock { + display: flex; + flex-flow: row wrap; + margin: 0 -2px; + + .pluginWrapper { + padding: 0 2px; + } + } + } +} + +iframe.pluginIframe { + height: 500px; + margin-top: 20px; + width: 100%; +} + +.iframeContent { + display: none; +} + +.iframePreview { + display: inline-block; + margin-top: 20px; +} + +@media only screen and (min-width: 1024px) { + .iframeContent { + display: block; + } + .iframePreview { + display: none; + } +} \ No newline at end of file diff --git a/docs/_sass/_react_docs_nav.scss b/docs/_sass/_react_docs_nav.scss new file mode 100644 index 00000000000..f0a651e7faf --- /dev/null +++ b/docs/_sass/_react_docs_nav.scss @@ -0,0 +1,332 @@ +.docsNavContainer { + background: $sidenav; + height: 35px; + left: 0; + position: fixed; + width: 100%; + z-index: 100; +} + +.docMainWrapper { + .wrapper { + &.mainWrapper { + padding-left: 0; + padding-right: 0; + padding-top: 10px; + } + } +} + +.docsSliderActive { + .docsNavContainer { + box-sizing: border-box; + height: 100%; + overflow-y: auto; + -webkit-overflow-scrolling: touch; + padding-bottom: 50px; + } + + .mainContainer { + display: none; + } +} + +.navBreadcrumb { + box-sizing: border-box; + display: flex; + flex-flow: row nowrap; + font-size: 12px; + height: 35px; + overflow: hidden; + padding: 5px 10px; + + a, span { + border: 0; + color: $sidenav-text; + } + + i { + padding: 0 3px; + } +} + +nav.toc { + position: relative; + + section { + padding: 0px; + position: relative; + + .navGroups { + display: none; + padding: 40px 10px 10px; + } + } + + .toggleNav { + background: $sidenav; + color: $sidenav-text; + position: relative; + transition: background-color 0.3s, color 0.3s; + + .navToggle { + cursor: pointer; + height: 24px; + margin-right: 10px; + position: relative; + text-align: left; + width: 18px; + + &::before, &::after { + content: ""; + position: absolute; + top: 50%; + left: 0; + left: 8px; + width: 3px; + height: 6px; + border: 5px solid $sidenav-text; + border-width: 5px 0; + margin-top: -8px; + transform: rotate(45deg); + z-index: 1; + } + + &::after { + transform: rotate(-45deg); + } + + i { + &::before, &::after { + content: ""; + position: absolute; + top: 50%; + left: 2px; + background: transparent; + border-width: 0 5px 5px; + border-style: solid; + border-color: transparent $sidenav-text; + height: 0; + margin-top: -7px; + opacity: 1; + width: 5px; + z-index: 10; + } + + &::after { + border-width: 5px 5px 0; + margin-top: 2px; + } + } + } + + .navGroup { + background: $sidenav-overlay; + margin: 1px 0; + + ul { + display: none; + } + + h3 { + background: $sidenav-overlay; + color: $sidenav-text; + cursor: pointer; + font-size: 14px; + font-weight: 400; + line-height: 1.2em; + padding: 10px; + transition: color 0.2s; + + i:not(:empty) { + width: 16px; + height: 16px; + display: inline-block; + box-sizing: border-box; + text-align: center; + color: rgba($sidenav-text, 0.5); + margin-right: 10px; + transition: color 0.2s; + } + + &:hover { + color: $primary-bg; + + i:not(:empty) { + color: $primary-bg; + } + } + } + + &.navGroupActive { + background: $sidenav-active; + color: $sidenav-text; + + ul { + display: block; + padding-bottom: 10px; + padding-top: 10px; + } + + h3 { + background: $primary-bg; + color: $primary-overlay; + + i { + display: none; + } + } + } + } + + ul { + padding-left: 0; + padding-right: 24px; + + li { + list-style-type: none; + padding-bottom: 0; + padding-left: 0; + + a { + border: none; + color: $sidenav-text; + display: inline-block; + font-size: 14px; + line-height: 1.1em; + margin: 2px 10px 5px; + padding: 5px 0 2px; + transition: color 0.3s; + + &:hover, + &:focus { + color: $primary-bg; + } + + &.navItemActive { + color: $primary-bg; + font-weight: 900; + } + } + } + } + } + + .toggleNavActive { + .navBreadcrumb { + background: $sidenav; + margin-bottom: 20px; + position: fixed; + width: 100%; + } + + section { + .navGroups { + display: block; + } + } + + + .navToggle { + &::before, &::after { + border-width: 6px 0; + height: 0px; + margin-top: -6px; + } + + i { + opacity: 0; + } + } + } +} + +.docsNavVisible { + .navPusher { + .mainContainer { + padding-top: 35px; + } + } +} + +@media only screen and (min-width: 900px) { + .navBreadcrumb { + padding: 5px 0; + } + + nav.toc { + section { + .navGroups { + padding: 40px 0 0; + } + } + } +} + +@media only screen and (min-width: 1024px) { + .navToggle { + display: none; + } + + .docsSliderActive { + .mainContainer { + display: block; + } + } + + .docsNavVisible { + .navPusher { + .mainContainer { + padding-top: 0; + } + } + } + + .docsNavContainer { + background: none; + box-sizing: border-box; + height: auto; + margin: 40px 40px 0 0; + overflow-y: auto; + position: relative; + width: 300px; + } + + nav.toc { + section { + .navGroups { + display: block; + padding-top: 0px; + } + } + + .toggleNavActive { + .navBreadcrumb { + margin-bottom: 0; + position: relative; + } + } + } + + .docMainWrapper { + display: flex; + flex-flow: row nowrap; + margin-bottom: 40px; + + .wrapper { + padding-left: 0; + padding-right: 0; + + &.mainWrapper { + padding-top: 0; + } + } + } + + .navBreadcrumb { + display: none; + h2 { + padding: 0 10px; + } + } +} \ No newline at end of file diff --git a/docs/_sass/_react_header_nav.scss b/docs/_sass/_react_header_nav.scss new file mode 100644 index 00000000000..13c0e562b76 --- /dev/null +++ b/docs/_sass/_react_header_nav.scss @@ -0,0 +1,141 @@ +.navigationFull { + display: none; +} + +.navigationSlider { + position: absolute; + right: 0px; + + .navSlideout { + cursor: pointer; + padding-top: 4px; + position: absolute; + right: 10px; + top: 0; + transition: top 0.3s; + z-index: 101; + } + + .slidingNav { + background: $secondary-bg; + box-sizing: border-box; + height: 0px; + overflow-x: hidden; + padding: 0; + position: absolute; + right: 0px; + top: 0; + transition: height 0.3s cubic-bezier(0.68, -0.55, 0.265, 1.55), width 0.3s cubic-bezier(0.68, -0.55, 0.265, 1.55); + width: 0; + + ul { + flex-flow: column nowrap; + list-style: none; + padding: 10px; + + li { + margin: 0; + padding: 2px 0; + + a { + color: $primary-bg; + display: inline; + margin: 3px 5px; + padding: 2px 0px; + transition: background-color 0.3s; + + &:focus, + &:hover { + border-bottom: 2px solid $primary-bg; + } + } + } + } + } + + .navSlideoutActive { + .slidingNav { + height: auto; + padding-top: $header-height + $header-pbot; + width: 300px; + } + + .navSlideout { + top: -2px; + .menuExpand { + span:nth-child(1) { + background-color: $text; + top: 16px; + transform: rotate(45deg); + } + span:nth-child(2) { + opacity: 0; + } + span:nth-child(3) { + background-color: $text; + transform: rotate(-45deg); + } + } + } + } +} + +.menuExpand { + display: flex; + flex-flow: column nowrap; + height: 20px; + justify-content: space-between; + + span { + background: $primary-overlay; + border-radius: 3px; + display: block; + flex: 0 0 4px; + height: 4px; + position: relative; + top: 0; + transition: background-color 0.3s, top 0.3s, opacity 0.3s, transform 0.3s; + width: 20px; + } +} + +.navPusher { + border-top: $header-height + $header-ptop + $header-pbot solid $primary-bg; + position: relative; + left: 0; + z-index: 99; + height: 100%; + + &::after { + position: absolute; + top: 0; + right: 0; + width: 0; + height: 0; + background: rgba(0,0,0,0.4); + content: ''; + opacity: 0; + -webkit-transition: opacity 0.5s, width 0.1s 0.5s, height 0.1s 0.5s; + transition: opacity 0.5s, width 0.1s 0.5s, height 0.1s 0.5s; + } + + .sliderActive &::after { + width: 100%; + height: 100%; + opacity: 1; + -webkit-transition: opacity 0.5s; + transition: opacity 0.5s; + z-index: 100; + } +} + + +@media only screen and (min-width: 1024px) { + .navigationFull { + display: block; + } + + .navigationSlider { + display: none; + } +} \ No newline at end of file diff --git a/docs/_sass/_reset.scss b/docs/_sass/_reset.scss new file mode 100644 index 00000000000..0e5f2e0c1d3 --- /dev/null +++ b/docs/_sass/_reset.scss @@ -0,0 +1,43 @@ +html, body, div, span, applet, object, iframe, +h1, h2, h3, h4, h5, h6, p, blockquote, pre, +a, abbr, acronym, address, big, cite, code, +del, dfn, em, img, ins, kbd, q, s, samp, +small, strike, strong, sub, sup, tt, var, +b, u, i, center, +dl, dt, dd, ol, ul, li, +fieldset, form, label, legend, +table, caption, tbody, tfoot, thead, tr, th, td, +article, aside, canvas, details, embed, +figure, figcaption, footer, header, hgroup, +menu, nav, output, ruby, section, summary, +time, mark, audio, video { + margin: 0; + padding: 0; + border: 0; + font-size: 100%; + font: inherit; + vertical-align: baseline; +} +/* HTML5 display-role reset for older browsers */ +article, aside, details, figcaption, figure, +footer, header, hgroup, menu, nav, section { + display: block; +} +body { + line-height: 1; +} +ol, ul { + list-style: none; +} +blockquote, q { + quotes: none; +} +blockquote:before, blockquote:after, +q:before, q:after { + content: ''; + content: none; +} +table { + border-collapse: collapse; + border-spacing: 0; +} diff --git a/docs/_sass/_search.scss b/docs/_sass/_search.scss new file mode 100644 index 00000000000..eadfa11d1e5 --- /dev/null +++ b/docs/_sass/_search.scss @@ -0,0 +1,142 @@ +input[type="search"] { + -moz-appearance: none; + -webkit-appearance: none; +} + +.navSearchWrapper { + align-self: center; + position: relative; + + &::before { + border: 3px solid $primary-overlay-special; + border-radius: 50%; + content: " "; + display: block; + height: 6px; + left: 15px; + width: 6px; + position: absolute; + top: 4px; + z-index: 1; + } + + &::after { + background: $primary-overlay-special; + content: " "; + height: 7px; + left: 24px; + position: absolute; + transform: rotate(-45deg); + top: 12px; + width: 3px; + z-index: 1; + } + + .aa-dropdown-menu { + background: $secondary-bg; + border: 3px solid rgba($text, 0.25); + color: $text; + font-size: 14px; + left: auto !important; + line-height: 1.2em; + right: 0 !important; + + .algolia-docsearch-suggestion--category-header { + background: $primary-overlay-special; + color: $primary-bg; + + .algolia-docsearch-suggestion--highlight { + background-color: $primary-bg; + color: $primary-overlay; + } + } + + .algolia-docsearch-suggestion--title .algolia-docsearch-suggestion--highlight, + .algolia-docsearch-suggestion--subcategory-column .algolia-docsearch-suggestion--highlight { + color: $primary-bg; + } + + .algolia-docsearch-suggestion__secondary, + .algolia-docsearch-suggestion--subcategory-column { + border-color: rgba($text, 0.3); + } + } +} + +input#search_input { + padding-left: 25px; + font-size: 14px; + line-height: 20px; + border-radius: 20px; + background-color: rgba($primary-overlay-special, 0.25); + border: none; + color: rgba($primary-overlay-special, 0); + outline: none; + position: relative; + transition: background-color .2s cubic-bezier(0.68, -0.55, 0.265, 1.55), width .2s cubic-bezier(0.68, -0.55, 0.265, 1.55), color .2s ease; + width: 60px; + + &:focus, &:active { + background-color: $secondary-bg; + color: $text; + width: 240px; + } +} + +.navigationSlider { + .navSearchWrapper { + &::before { + left: 6px; + top: 6px; + } + + &::after { + left: 15px; + top: 14px; + } + } + + input#search_input_react { + box-sizing: border-box; + padding-left: 25px; + font-size: 14px; + line-height: 20px; + border-radius: 20px; + background-color: rgba($primary-overlay-special, 0.25); + border: none; + color: $text; + outline: none; + position: relative; + transition: background-color .2s cubic-bezier(0.68, -0.55, 0.265, 1.55), width .2s cubic-bezier(0.68, -0.55, 0.265, 1.55), color .2s ease; + width: 100%; + + &:focus, &:active { + background-color: $primary-bg; + color: $primary-overlay; + } + } + + .algolia-docsearch-suggestion--subcategory-inline { + display: none; + } + + & > span { + width: 100%; + } + + .aa-dropdown-menu { + background: $secondary-bg; + border: 0px solid $secondary-bg; + color: $text; + font-size: 12px; + line-height: 2em; + max-height: 140px; + min-width: auto; + overflow-y: scroll; + -webkit-overflow-scrolling: touch; + padding: 0; + border-radius: 0; + position: relative !important; + width: 100%; + } +} \ No newline at end of file diff --git a/docs/_sass/_slideshow.scss b/docs/_sass/_slideshow.scss new file mode 100644 index 00000000000..cd98a6cdbaf --- /dev/null +++ b/docs/_sass/_slideshow.scss @@ -0,0 +1,48 @@ +.slideshow { + position: relative; + + .slide { + display: none; + + img { + display: block; + margin: 0 auto; + } + + &.slideActive { + display: block; + } + + a { + border: none; + display: block; + } + } + + .pagination { + display: block; + margin: -10px; + padding: 1em 0; + text-align: center; + width: 100%; + + .pager { + background: transparent; + border: 2px solid rgba(255, 255, 255, 0.5); + border-radius: 50%; + cursor: pointer; + display: inline-block; + height: 12px; + margin: 10px; + transition: background-color 0.3s, border-color 0.3s; + width: 12px; + + &.pagerActive { + background: rgba(255, 255, 255, 0.5); + border-width: 4px; + height: 8px; + width: 8px; + } + } + } +} diff --git a/docs/_sass/_syntax-highlighting.scss b/docs/_sass/_syntax-highlighting.scss new file mode 100644 index 00000000000..e55c88a2eab --- /dev/null +++ b/docs/_sass/_syntax-highlighting.scss @@ -0,0 +1,129 @@ + + +.rougeHighlight { background-color: $code-bg; color: #93a1a1 } +.rougeHighlight .c { color: #586e75 } /* Comment */ +.rougeHighlight .err { color: #93a1a1 } /* Error */ +.rougeHighlight .g { color: #93a1a1 } /* Generic */ +.rougeHighlight .k { color: #859900 } /* Keyword */ +.rougeHighlight .l { color: #93a1a1 } /* Literal */ +.rougeHighlight .n { color: #93a1a1 } /* Name */ +.rougeHighlight .o { color: #859900 } /* Operator */ +.rougeHighlight .x { color: #cb4b16 } /* Other */ +.rougeHighlight .p { color: #93a1a1 } /* Punctuation */ +.rougeHighlight .cm { color: #586e75 } /* Comment.Multiline */ +.rougeHighlight .cp { color: #859900 } /* Comment.Preproc */ +.rougeHighlight .c1 { color: #72c02c; } /* Comment.Single */ +.rougeHighlight .cs { color: #859900 } /* Comment.Special */ +.rougeHighlight .gd { color: #2aa198 } /* Generic.Deleted */ +.rougeHighlight .ge { color: #93a1a1; font-style: italic } /* Generic.Emph */ +.rougeHighlight .gr { color: #dc322f } /* Generic.Error */ +.rougeHighlight .gh { color: #cb4b16 } /* Generic.Heading */ +.rougeHighlight .gi { color: #859900 } /* Generic.Inserted */ +.rougeHighlight .go { color: #93a1a1 } /* Generic.Output */ +.rougeHighlight .gp { color: #93a1a1 } /* Generic.Prompt */ +.rougeHighlight .gs { color: #93a1a1; font-weight: bold } /* Generic.Strong */ +.rougeHighlight .gu { color: #cb4b16 } /* Generic.Subheading */ +.rougeHighlight .gt { color: #93a1a1 } /* Generic.Traceback */ +.rougeHighlight .kc { color: #cb4b16 } /* Keyword.Constant */ +.rougeHighlight .kd { color: #268bd2 } /* Keyword.Declaration */ +.rougeHighlight .kn { color: #859900 } /* Keyword.Namespace */ +.rougeHighlight .kp { color: #859900 } /* Keyword.Pseudo */ +.rougeHighlight .kr { color: #268bd2 } /* Keyword.Reserved */ +.rougeHighlight .kt { color: #dc322f } /* Keyword.Type */ +.rougeHighlight .ld { color: #93a1a1 } /* Literal.Date */ +.rougeHighlight .m { color: #2aa198 } /* Literal.Number */ +.rougeHighlight .s { color: #2aa198 } /* Literal.String */ +.rougeHighlight .na { color: #93a1a1 } /* Name.Attribute */ +.rougeHighlight .nb { color: #B58900 } /* Name.Builtin */ +.rougeHighlight .nc { color: #268bd2 } /* Name.Class */ +.rougeHighlight .no { color: #cb4b16 } /* Name.Constant */ +.rougeHighlight .nd { color: #268bd2 } /* Name.Decorator */ +.rougeHighlight .ni { color: #cb4b16 } /* Name.Entity */ +.rougeHighlight .ne { color: #cb4b16 } /* Name.Exception */ +.rougeHighlight .nf { color: #268bd2 } /* Name.Function */ +.rougeHighlight .nl { color: #93a1a1 } /* Name.Label */ +.rougeHighlight .nn { color: #93a1a1 } /* Name.Namespace */ +.rougeHighlight .nx { color: #93a1a1 } /* Name.Other */ +.rougeHighlight .py { color: #93a1a1 } /* Name.Property */ +.rougeHighlight .nt { color: #268bd2 } /* Name.Tag */ +.rougeHighlight .nv { color: #268bd2 } /* Name.Variable */ +.rougeHighlight .ow { color: #859900 } /* Operator.Word */ +.rougeHighlight .w { color: #93a1a1 } /* Text.Whitespace */ +.rougeHighlight .mf { color: #2aa198 } /* Literal.Number.Float */ +.rougeHighlight .mh { color: #2aa198 } /* Literal.Number.Hex */ +.rougeHighlight .mi { color: #2aa198 } /* Literal.Number.Integer */ +.rougeHighlight .mo { color: #2aa198 } /* Literal.Number.Oct */ +.rougeHighlight .sb { color: #586e75 } /* Literal.String.Backtick */ +.rougeHighlight .sc { color: #2aa198 } /* Literal.String.Char */ +.rougeHighlight .sd { color: #93a1a1 } /* Literal.String.Doc */ +.rougeHighlight .s2 { color: #2aa198 } /* Literal.String.Double */ +.rougeHighlight .se { color: #cb4b16 } /* Literal.String.Escape */ +.rougeHighlight .sh { color: #93a1a1 } /* Literal.String.Heredoc */ +.rougeHighlight .si { color: #2aa198 } /* Literal.String.Interpol */ +.rougeHighlight .sx { color: #2aa198 } /* Literal.String.Other */ +.rougeHighlight .sr { color: #dc322f } /* Literal.String.Regex */ +.rougeHighlight .s1 { color: #2aa198 } /* Literal.String.Single */ +.rougeHighlight .ss { color: #2aa198 } /* Literal.String.Symbol */ +.rougeHighlight .bp { color: #268bd2 } /* Name.Builtin.Pseudo */ +.rougeHighlight .vc { color: #268bd2 } /* Name.Variable.Class */ +.rougeHighlight .vg { color: #268bd2 } /* Name.Variable.Global */ +.rougeHighlight .vi { color: #268bd2 } /* Name.Variable.Instance */ +.rougeHighlight .il { color: #2aa198 } /* Literal.Number.Integer.Long */ + +.highlighter-rouge { + color: darken(#72c02c, 8%); + font: 800 12px/1.5em Hack, monospace; + max-width: 100%; + + .rougeHighlight { + border-radius: 3px; + margin: 20px 0; + padding: 0px; + overflow-x: scroll; + -webkit-overflow-scrolling: touch; + + table { + background: none; + border: none; + + tbody { + tr { + background: none; + display: flex; + flex-flow: row nowrap; + + td { + display: block; + flex: 1 1; + + &.gutter { + border-right: 1px solid lighten($code-bg, 10%); + color: lighten($code-bg, 15%); + margin-right: 10px; + max-width: 40px; + padding-right: 10px; + + pre { + max-width: 20px; + } + } + } + } + } + } + } +} + +p > .highlighter-rouge, +li > .highlighter-rouge, +a > .highlighter-rouge { + font-size: 16px; + font-weight: 400; + line-height: inherit; +} + +a:hover { + .highlighter-rouge { + color: white; + } +} \ No newline at end of file diff --git a/docs/_sass/_tables.scss b/docs/_sass/_tables.scss new file mode 100644 index 00000000000..f847c701370 --- /dev/null +++ b/docs/_sass/_tables.scss @@ -0,0 +1,47 @@ +table { + background: $lightergrey; + border: 1px solid $lightgrey; + border-collapse: collapse; + display:table; + margin: 20px 0; + + thead { + border-bottom: 1px solid $lightgrey; + display: table-header-group; + } + tbody { + display: table-row-group; + } + tr { + display: table-row; + &:nth-of-type(odd) { + background: $greyish; + } + + th, td { + border-right: 1px dotted $lightgrey; + display: table-cell; + font-size: 14px; + line-height: 1.3em; + padding: 10px; + text-align: left; + + &:last-of-type { + border-right: 0; + } + + code { + color: $green; + display: inline-block; + font-size: 12px; + } + } + + th { + color: #000000; + font-weight: bold; + font-family: $header-font-family; + text-transform: uppercase; + } + } +} \ No newline at end of file diff --git a/docs/_top-level/support.md b/docs/_top-level/support.md new file mode 100644 index 00000000000..64165751fe2 --- /dev/null +++ b/docs/_top-level/support.md @@ -0,0 +1,22 @@ +--- +layout: top-level +title: Support +id: support +category: support +--- + +## Need help? + +Do not hesitate to ask questions if you are having trouble with RocksDB. + +### GitHub issues + +Use [GitHub issues](https://github.com/facebook/rocksdb/issues) to report bugs, issues and feature requests for the RocksDB codebase. + +### Facebook Group + +Use the [RocksDB Facebook group](https://www.facebook.com/groups/rocksdb.dev/) for general questions and discussion about RocksDB. + +### FAQ + +Check out a list of [commonly asked questions](/docs/support/faq) about RocksDB. diff --git a/docs/blog/all.html b/docs/blog/all.html new file mode 100644 index 00000000000..3be2d3bff20 --- /dev/null +++ b/docs/blog/all.html @@ -0,0 +1,20 @@ +--- +id: all +layout: blog +category: blog +--- + +
    +
    +

    All Posts

    + {% for post in site.posts %} + {% assign author = site.data.authors[post.author] %} +

    + + {{ post.title }} + + on {{ post.date | date: "%B %e, %Y" }} by {{ author.display_name }} +

    + {% endfor %} +
    +
    diff --git a/docs/blog/index.html b/docs/blog/index.html new file mode 100644 index 00000000000..9f6b25d03cd --- /dev/null +++ b/docs/blog/index.html @@ -0,0 +1,12 @@ +--- +id: blog +title: Blog +layout: blog +category: blog +--- + +
    + {% for page in site.posts %} + {% include post.html truncate=true %} + {% endfor %} +
    diff --git a/docs/css/main.scss b/docs/css/main.scss new file mode 100644 index 00000000000..48a3e14ef97 --- /dev/null +++ b/docs/css/main.scss @@ -0,0 +1,149 @@ +--- +# Only the main Sass file needs front matter (the dashes are enough) +--- +@charset "utf-8"; + +@font-face { + font-family: 'Lato'; + src: url("{{ '/static/fonts/LatoLatin-Italic.woff2' }}") format('woff2'), + url("{{ '/static/fonts/LatoLatin-Italic.woff' }}") format('woff'); + font-weight: normal; + font-style: italic; +} + +@font-face { + font-family: 'Lato'; + src: url("{{ '/static/fonts/LatoLatin-Black.woff2' }}") format('woff2'), + url("{{ '/static/fonts/LatoLatin-Black.woff' }}") format('woff'); + font-weight: 900; + font-style: normal; +} + +@font-face { + font-family: 'Lato'; + src: url("{{ '/static/fonts/LatoLatin-BlackItalic.woff2' }}") format('woff2'), + url("{{ '/static/fonts/LatoLatin-BlackItalic.woff' }}") format('woff'); + font-weight: 900; + font-style: italic; +} + +@font-face { + font-family: 'Lato'; + src: url("{{ '/static/fonts/LatoLatin-Light.woff2' }}") format('woff2'), + url("{{ '/static/fonts/LatoLatin-Light.woff' }}") format('woff'); + font-weight: 300; + font-style: normal; +} + +@font-face { + font-family: 'Lato'; + src: url("{{ '/static/fonts/LatoLatin-Regular.woff2' }}") format('woff2'), + url("{{ '/static/fonts/LatoLatin-Regular.woff' }}") format('woff'); + font-weight: normal; + font-style: normal; +} + +// Our variables +$base-font-family: 'Lato', Calibri, Arial, sans-serif; +$header-font-family: 'Lato', 'Helvetica Neue', Arial, sans-serif; +$base-font-size: 18px; +$small-font-size: $base-font-size * 0.875; +$base-line-height: 1.4em; + +$spacing-unit: 12px; + +// Two configured colors (see _config.yml) +$primary-bg: {{ site.color.primary }}; +$secondary-bg: {{ site.color.secondary }}; + +// $primary-bg overlays +{% if site.color.primary-overlay == 'light' %} +$primary-overlay: darken($primary-bg, 70%); +$primary-overlay-special: darken($primary-bg, 40%); +{% else %} +$primary-overlay: #fff; +$primary-overlay-special: lighten($primary-bg, 30%); +{% endif %} + +// $secondary-bg overlays +{% if site.color.secondary-overlay == 'light' %} +$text: #393939; +$sidenav: darken($secondary-bg, 20%); +$sidenav-text: $text; +$sidenav-overlay: darken($sidenav, 10%); +$sidenav-active: lighten($sidenav, 10%); +{% else %} +$text: #fff; +$sidenav: lighten($secondary-bg, 20%); +$sidenav-text: $text; +$sidenav-overlay: lighten($sidenav, 10%); +$sidenav-active: darken($sidenav, 10%); +{% endif %} + +$code-bg: #002b36; + +$header-height: 34px; +$header-ptop: 10px; +$header-pbot: 8px; + +// Width of the content area +$content-width: 900px; + +// Table setting variables +$lightergrey: #F8F8F8; +$greyish: #E8E8E8; +$lightgrey: #B0B0B0; +$green: #2db04b; + +// Using media queries with like this: +// @include media-query($on-palm) { +// .wrapper { +// padding-right: $spacing-unit / 2; +// padding-left: $spacing-unit / 2; +// } +// } +@mixin media-query($device) { + @media screen and (max-width: $device) { + @content; + } +} + + + +// Import partials from `sass_dir` (defaults to `_sass`) +@import + "reset", + "base", + "header", + "search", + "syntax-highlighting", + "promo", + "buttons", + "gridBlock", + "poweredby", + "footer", + "react_header_nav", + "react_docs_nav", + "tables", + "blog" +; + +// Anchor links +// http://ben.balter.com/2014/03/13/pages-anchor-links/ +.header-link { + position: absolute; + margin-left: 0.2em; + opacity: 0; + + -webkit-transition: opacity 0.2s ease-in-out 0.1s; + -moz-transition: opacity 0.2s ease-in-out 0.1s; + -ms-transition: opacity 0.2s ease-in-out 0.1s; +} + +h2:hover .header-link, +h3:hover .header-link, +h4:hover .header-link, +h5:hover .header-link, +h6:hover .header-link { + opacity: 1; +} diff --git a/docs/doc-type-examples/2016-04-07-blog-post-example.md b/docs/doc-type-examples/2016-04-07-blog-post-example.md new file mode 100644 index 00000000000..ef954d63a7c --- /dev/null +++ b/docs/doc-type-examples/2016-04-07-blog-post-example.md @@ -0,0 +1,21 @@ +--- +title: Blog Post Example +layout: post +author: exampleauthor +category: blog +--- + +Any local blog posts would go in the `_posts` directory. + +This is an example blog post introduction, try to keep it short and about a paragraph long, to encourage people to click through to read the entire post. + + + +Everything below the `` tag will only show on the actual blog post page, not on the `/blog/` index. + +Author is defined in `_data/authors.yml` + + +## No posts? + +If you have no blog for your site, you can remove the entire `_posts` folder. Otherwise add markdown files in here. See CONTRIBUTING.md for details. diff --git a/docs/doc-type-examples/docs-hello-world.md b/docs/doc-type-examples/docs-hello-world.md new file mode 100644 index 00000000000..c7094ba5af4 --- /dev/null +++ b/docs/doc-type-examples/docs-hello-world.md @@ -0,0 +1,12 @@ +--- +docid: hello-world +title: Hello, World! +layout: docs +permalink: /docs/hello-world.html +--- + +Any local docs would go in the `_docs` directory. + +## No documentation? + +If you have no documentation for your site, you can remove the entire `_docs` folder. Otherwise add markdown files in here. See CONTRIBUTING.md for details. diff --git a/docs/doc-type-examples/top-level-example.md b/docs/doc-type-examples/top-level-example.md new file mode 100644 index 00000000000..67b1fa71101 --- /dev/null +++ b/docs/doc-type-examples/top-level-example.md @@ -0,0 +1,8 @@ +--- +layout: top-level +title: Support Example +id: top-level-example +category: top-level +--- + +This is a static page disconnected from the blog or docs collections that can be added at a top-level (i.e., the same level as `index.md`). diff --git a/docs/docs/index.html b/docs/docs/index.html new file mode 100644 index 00000000000..fa6ec8b5a6f --- /dev/null +++ b/docs/docs/index.html @@ -0,0 +1,6 @@ +--- +id: docs +title: Docs +layout: redirect +destination: getting-started.html +--- diff --git a/docs/feed.xml b/docs/feed.xml new file mode 100644 index 00000000000..590cbd9d76e --- /dev/null +++ b/docs/feed.xml @@ -0,0 +1,30 @@ +--- +layout: null +--- + + + + {{ site.title | xml_escape }} + {{ site.description | xml_escape }} + {{ absolute_url }}/ + + {{ site.time | date_to_rfc822 }} + {{ site.time | date_to_rfc822 }} + Jekyll v{{ jekyll.version }} + {% for post in site.posts limit:10 %} + + {{ post.title | xml_escape }} + {{ post.content | xml_escape }} + {{ post.date | date_to_rfc822 }} + {{ post.url | absolute_url }} + {{ post.url | absolute_url }} + {% for tag in post.tags %} + {{ tag | xml_escape }} + {% endfor %} + {% for cat in post.categories %} + {{ cat | xml_escape }} + {% endfor %} + + {% endfor %} + + diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000000..2b9570d2305 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,9 @@ +--- +layout: home +title: RocksDB | A persistent key-value store +id: home +--- + +## Features + +{% include content/gridblocks.html data_source=site.data.features align="center" %} diff --git a/docs/static/favicon.png b/docs/static/favicon.png new file mode 100644 index 0000000000000000000000000000000000000000..7f668f38f7a5084a442278e780e43de74501fcb7 GIT binary patch literal 3927 zcmV-d52)~oP)X+uL$Nkc;* zP;zf(X>4Tx07wm;mUmQB*%pV-y*Itk5+Wca^cs2zAksTX6$DXM^`x7XQc?|s+0 z08spb1j2M!0f022SQPH-!CVp(%f$Br7!UytSOLJ{W@ZFO_(THK{JlMynW#v{v-a*T zfMmPdEWc1DbJqWVks>!kBnAKqMb$PuekK>?0+ds;#ThdH1j_W4DKdsJG8Ul;qO2n0 z#IJ1jr{*iW$(WZWsE0n`c;fQ!l&-AnmjxZO1uWyz`0VP>&nP`#itsL#`S=Q!g`M=rU9)45( zJ;-|dRq-b5&z?byo>|{)?5r=n76A4nTALlSzLiw~v~31J<>9PP?;rs31pu_(obw)r zY+jPY;tVGXi|p)da{-@gE-UCa`=5eu%D;v=_nFJ?`&K)q7e9d`Nfk3?MdhZarb|T3 z%nS~f&t(1g5dY)AIcd$w!z`Siz!&j_=v7hZlnI21XuE|xfmo0(WD10T)!}~_HYW!e zew}L+XmwuzeT6wtxJd`dZ#@7*BLgIEKY9Xv>st^p3dp{^Xswa2bB{85{^$B13tWnB z;Y>jyQ|9&zk7RNsqAVGs--K+z0uqo1bf5|}fi5rtEMN^BfHQCd-XH*kfJhJnmIE$G z0%<@5vOzxB0181d*a3EfYH$G5fqKvcPJ%XY23!PJzzuK<41h;K3WmW;Fah3yX$XSw z5EY_9s*o0>51B&N5F1(uc|$=^I1~fLLy3?Ol0f;;Ca4%HgQ}rJP(Ab`bQ-z{U4#0d z2hboi2K@njgb|nm(_szR0JebHusa+GN5aeCM0gdP2N%HG;Yzp`J`T6S7vUT504#-H z!jlL<$Or?`Mpy_N@kBz9SR?@vA#0H$qyni$nvf2p8@Y{0k#Xb$28W?xm>3qu8RLgp zjNxKdVb)?wFx8l2m{v>|<~C*!GlBVnrDD~wrdTJeKXwT=5u1%I#8zOBU|X=4u>;s) z>^mF|$G{ol9B_WP7+f-LHLe7=57&&lfa}8z;U@8Tyei%l?}87(bMRt(A-)QK9Dg3) zj~~XrCy)tR1Z#p1A(kK{Y$Q|=8VKhI{e%(1G*N-5Pjn)N5P8I0VkxnX*g?EW941ba z6iJ387g8iCnY4jaNopcpCOsy-A(P2EWJhusSwLP-t|XrzUnLKcKTwn?CKOLf97RIe zPB}`sKzTrUL#0v;sBY9)s+hW+T2H-1eM)^VN0T#`^Oxhvt&^*fYnAJldnHel*Ozyf zUoM{~Um<@={-*r60#U(0!Bc^wuvVc);k3d%g-J!4qLpHZVwz%!VuRu}#Ze`^l7W)9 z5>Kf>>9Eozr6C$Z)1`URxU@~QI@)F0FdauXr2Es8>BaOP=)Lp_WhG@>R;lZ?BJkMlIuMhw8ApiF&yDYW2hFJ?fJhni{?u z85&g@mo&yT8JcdI$(rSw=QPK(Xj%)k1X|@<=e1rim6`6$RAwc!i#egKuI;BS(LSWz zt39n_sIypSqfWEV6J3%nTQ@-4i zi$R;gsG*9XzhRzXqv2yCs*$VFDx+GXJH|L;wsDH_KI2;^u!)^Xl1YupO;gy^-c(?^ z&$Q1BYvyPsG^;hc$D**@Sy`+`)}T4VJji^bd7Jqw3q6Zii=7tT7GEswEK@D(EFW1Z zSp`^awCb?>!`j4}Yh7b~$A)U-W3$et-R8BesV(1jzwLcHnq9En7Q0Tn&-M=XBKs!$ zF$X<|c!#|X_tWYh)GZit z(Q)Cp9CDE^WG;+fcyOWARoj*0TI>4EP1lX*cEoMO-Pk?Z{kZ!p4@(b`M~lalr<3Oz z&kJ6Nm#vN_+kA5{dW4@^Vjg_`q%qU1ULk& z3Fr!>1V#i_2R;ij2@(Z$1jE4r!MlPVFVbHmT+|iPIq0wy5aS{>yK?9ZAjVh%SOwMWgFjair&;wpi!{CU}&@N=Eg#~ zLQ&zpEzVmGY{hI9Z0+4-0xS$$Xe-OToc?Y*V;rTcf_ zb_jRe-RZjXSeas3UfIyD;9afd%<`i0x4T#DzE)vdabOQ=k7SRuGN`h>O0Q~1)u-yD z>VX=Mn&!Rgd$;YK+Q-}1zu#?t(*cbG#Ronf6db&N$oEidtwC+YVcg-Y!_VuY>bk#Y ze_ww@?MU&F&qswvrN_dLb=5o6*Egs)ls3YRlE$&)amR1{;Ppd$6RYV^Go!iq1UMl% z@#4q$AMc(FJlT1QeX8jv{h#)>&{~RGq1N2iiMFIRX?sk2-|2wUogK~{EkB$8eDsX= znVPf8XG_nK&J~=SIiGia@9y}|z3FhX{g&gcj=lwb=lWgyFW&aLedUh- zof`v-2Kw$UzI*>(+&$@i-u=-BsSjR1%z8NeX#HdC`Hh-Z(6xI-`hmHDqv!v)W&&nrf>M(RhcN6(D;jNN*%^u_SYjF;2ng}*8Ow)d6M ztDk;%`@Lsk$;9w$(d(H%O5UixIr`T2ZRcd@{Q`z;1W>YM66OX$1py%eVw5QfMgw9nn)qXy38F?^ zP(z4Fj6WA6>+@W0U%S%Ih9Th}PV##1 zyZ794zUSQUIj=E*A9vk&Rd_ zuyihLd=cKO%BI&BgEPLHqb>(sJ+WB*G`EPSX?gHrF+5uowAC*F#o|?u!qHElS7_SR z9XzCFF_Mz`cWSbgJrC~gIG&z$5_gw{v+Z#dYnzmuoZN_-y9t?37*jKt!^Dy?ki{aaiv8E*F(zxZ#Ov76hV|?{?s?lAA z;rRxmwQ`WrEOzD;VoV+%YQyv?aRkdS-UdIcgHf5Vdb6af5B(Nc{4Je(Cp6LKwsKC0H^kn_oE1>A0E+H{L2^wOE<&SRry95Z2x~soM*TOW zUy7ze&M?Sb0!?kETLF{-rHeHgKl=V=x#v^jxC=*I?~^+RXYycyi{niRj81cN{0w~Y6O?U-Iw{%yBt8xbZ`GBM zn+*@VrZq!2pq3_y((;9nl%UZb`wPBa9V=y($ZaFf)6i_EOiNO{5XTKZ$aNQ`|LKtD~(FUJ1mmbEXcQR#c8$0)h8|u5F8qUWrM0c*vgk_+wp}~og zjXNNJ9PF>NlP1jHd&tN3pMzSe(~TRUJEiKDX@j9+8Pbk}I{!6`2tai?X-4!20pEE_ zA1}!Hn+>~QS(&~L)2QllD%ycc7y#*ntd}}Ymq?m~EWJKyCM)#^r9-&TR}|1^=J#E> z@_7J&!>*f+N_XZEcK24WZ_< zL}&U0{mhmU8;GddHzfc-wM|o0>8Njl0^@4ctLkbtEQbZt^q0`o9muGz)sz4Lb3u(` zhwG2VIsu?0*#X;1b<}$0>V7wV8$7SF5opEcvg-MYV5$m~WC$m&1S0Erm z!VuMU1#x9nAs`?&?@xdFv-7VqM$a&;yI0^%9{tYQ8PE--$IDpPwin@{ia`N2zpfMC*AP=8*U8906R6)O6y zf%-=vnpu08etK0vKvF0`Kruj6wTCU{CI-epK+(OQHEe$bj0neu`KS5mJ$~jypYavp z1@g|^#@YR|4=xZ8XeAI3q>QniMW>aCqb(3n{O4Hgsy|;p$os-0xV4>;!KePL1x@&j zvQes2EgJ)Odmvyi*w5!8{3AfIfEjEJY)n2qn$O_{0RdqQ<&Y1u+uJ!g0|CPVK3^*$ z5YU%lQQ;Icdq^F&OPfa%fne^Xz9L^2}KCxOEBdZU=T4>WW|0$ zVuM5vhzu}e1^OgpBv7bOqHuL^QIf-tj^TjS_GpC1xF?r7DF8_nr%wV86O8W zY8KoTTFje8-Z$Xqu!}pR05>8@eJMPvtu={R_GQ(5iSo~wpWhEBduG(-Z(fRX`!RU9 z(-J49=ew3f7Qua?wdA35G4WOq#yt{;$i7 z5EfP&E)n5QW9TwSj%0QMVc%HhW!Ck}GtK=&a<@FZ0m-cJSPbqU9ixfCQgtEax#9St zMsJ)u7Qz?qm`%cJGs2dDci4^0QukIW77q8q$-Af)7ynJu8J_`uXI^O;P21IBx-FxT;r>V`dIRIWgpB0mUV5&f zu;I=&@^;B!*H3Hd24c{w{3{pM0&RF3&N@}Dkwmo|hOQQ=&v_ z<>HKEOsje~jn2tirom zbEzlTKbSI*LMkpqzaheDZJ9wYH5=UdbJ03x-0@<}TOVo7I~?gz;Qv zaZcRi+QcaN5YP`q0qw*>0vzVhh5|Ia(~bMgtqE)?tR_?(~6O#ydtu=p(1Y;zKBN@T19q&i9dQDqdnf;uxg^n+9rj z=EChW43BvD20BcGTXgy&Yv1TkE9>*QV;2R1>l0~!r^ltRR z_Hr0wyCYA6$_G4cTExSnBdWsm5nsY;z^3|}2|{Qryf!#mSFe8H>0Ye9$=O&K9;gIx4^-9bCQyaTh%erRvnT*A9!bV9NP%IY)Q z)c2tS&^B5E&Nh7jK{Aafryw1DkopKKFnnMyz2f@(kC?bXx&HEfg7#3j5S+JnSdX~4 zz3{zilZxENErUlf3~bD& zSrzpbwv%QeMr%iH_I8^cnUqM}`C-`DgHPZ#pBa#Znnw%w+>FOi>y07ncXqZ%dukHV zE{Jf}Wp}Y?kw^1C|ESvbT$?cPC_2w8ep^PbfWlXZ?k_42QL-wc1}fdLwfXV>1*E35 z@HJ_XGtz?l0`L%XK_V@MGUgpLrawXB8`9g8co`-5Y;U9-q+%-bxOXx zShxuFIr&3C#5WXeu_4L~kSe=o+GeM{ft37t%xZ+&P+}xoR@!F~MvFnYpTu(?b4kd^;g_QJH*9=K#4n zmIqK_4eeK}UlPCiPQAwtNJjH#lBjdZA^B!xND=7=#zbH08evu4aa33Qi%AP4@KA=R zAY*6zH)eV-Brm-fU%L@}5dxOqw(j&fvcYndx18)Af1#}N!P|Kuek%;`dwaa1pq@pb ze@3y05YfVT^~B|C4vu>09e}^Af$csEwDXF%qS;zle=ht9OsJunPp4WYjdrXQ(ysEC z9A|-A>Gemo&(LKaM=hnVib?p8q-47=NYglQfL8|He-c^B!Tym-8)AAl~1g)QJGfYyi# z_@w-8!2gsG(HgKM=mX(H42Fzz0-=mcX8txHZG)EuaH8cLJFfg_TQ7RDx{#QV*;v73Mu~~ z%gWP~la>)vU~y;Aky@*!c#ltu?rTZa;KG-hbZYoWsN>_|-3oU{RTRS`X2T;5xVNLh z_U;j_8ht!w%?_ERGo+;xtR<3c7k8w}HjS<@m&0BzSR=gW&$VQ8$cO3sv-5YYK^@g()C9@^)IYLq12scIdmbu$=-cF~jY?OE;SPK_5>D@hHj*rpl{ z>ESh~s$xJOzrhh0t8?UYx^7QcrJ84rA&=bfPEjd`XNENWhLej^jqUJa;cCZoIi3!y zf910h_4}l1<&_!syY)bB^&@k@YK7N;wasu3`w?&0y3^$9n#8S}{%{WQvp9nlb?x`2 z;tFOYKL1s1{poeFVdOaWa(5?!^+{64TtdSbimoK6vtb%BhA4zX%}K?$;|!KzjlNUX z-W|KEd!{a40XzP~uBi$8EwolZYKE8Uukr`3CWUvYcN^C1LrW7|cClU)D0q9#yy z`sd_f9Kt=@zXRvIT>nD2g#_&tV1R;!5bjlAfQ5x5>6KuBj)p)8WYovyi-tt$6&4v3 zVfZ2gfe=KQl?zc4v`PraGv(4@IJon^rtuLv4zS)RHUb(B0bfysAyMN83x*2frKvVSQN3x*tGiSKhV=V zrZ+m>bJ$~&MU-h{9LKByh-d82D1@kT0&`Ld%kT^Kuo$jFFj*>PI_b)|tDBs+sTcY6 zR_YbrDv1E+f&jss4LC|3lPLt@?>mg#)W(YDDz22Jd~MRYjHHgha9<2)Jr~e%BdkF{ z8o&!Tkc4$34CO`~aEWMwAJl?;`kG;a)?H=#)ykN#JNd^vy7$;{TEI1{W*FwN0IUmP zDA%Zs$|I>E1oxKlwh=BS0$KPIpcMF&D9*rwX$IkjCuGJubpU`HY7()>Kof@41(zOx zOC$YC5VLmx=tP>Zs_V?IlNofx{G&+h{*<~U$v}Ej=NnCT;1IfowSktkfkx3X`t37d z^`zsrJxLc~)(X9=Tc-Bsp#iL>GsxYLv>|TwdcG=pb-nrB*J>v7rW7@j@&oelqsBGC zO1{QS`N}kl6v0ZDM(k*HN_RB56Y^WG1;*uk;I(X;HD=+njQsoC8_I^c^{kltNii=| zlB`wp8S6$fJnSteSgXyrY-G>X=xo`QqZ#Z4Q?fN?l(US8`zk^HQ{IwgLJ$}vjqu+7 z`AG?3)1rJ%61XYWaE59zeb#=RAgC%a25mU3-v|5FF@4umGr4U1e_)rE`7jasJ7*wxFMidtt(ABL%Nl}(LW zVwLRPevBk(HsmGJnA(KV4gG0Kt&Z@q-UF*8{d%~HxhiIi{A;fQIHkUO`>GM9N-P)T zQRmO9gr~o&sKYbWO@9!lX!b}f$rIy{)GTOpzNh3TKc>Z}x>mQ za`=;9F+%9}EcfuIpabHfVe@ROX_>qg3mraqTlDZ-l-dVz!!Oz0Z(pzJ3LMHC<%D*W z=32k9(0z%K7TQVNCBCBKN{gH(&3BR7*%d)Z^D{%Q}wT`l0J|K20A7~qD zuwm7JMz-T>o|}swuG6{;em)L}9q^4N2y68d3*AYL0Hmug#Nj8WI&S{*uyh)eRMEd7 zu+PN?ew(-Az_}G}*|PkK3GJA(Xa66X+&2WP79W#xFJ`yDrsz5nzqyijoJzT_rkvJO zubQaWPt+KveH;6$q5T7`<7C3z4{gnEGV+SOz_D>XW~pP&lU?gV?But37f!WX;hpWD z=Bm%TMAth>#|*)z8il_yJHE;-AMrB{dt~MwlyVJEIYp#iB~q&wt}+hO7=y1%#4{xQ zzcm2SH-L6d!e26R_f9NeqWuNqsW1`lm9^=~^s~w6)mW&ObS&l|s14a*Kfz!-vVH{Q zt;$hHBh6LyxeH@PH8X$%Puw3Xl5x!czQ%)-fou@O)`$f$F4&+RqtPCf&>n;CXtiXQ z`S^1U5^Cn6SfLHKS)leRQ(q4oVcxuQx_socOtZOkr{TeeQ5QA|#qU!PV_3lLQ)no` z8ixvH3e4w{H=BF^Tym^28a5Tdcq0A9kp@qUz28`ucn441xR!zKo99PKPcDINC(IEb~<7gLsE9_0O=(%mOzJDy&F8q$horj(Arp@I#pE2|L^v;36+; z!8#HYbI(HJ!rDTw>WR5)se5PR+e#th?r$6LUp*=v$v-V8?GJ8(0##rxGJ&&G8up)15~iHjW^6NnP4% zJQq5V+;^+;+48Wev7Sna>yK4CW_^3aRSAGoD-abdaH-`7s+Q@a zQE7xgSBWW@ETY*gy3WCW4d~Fgx!<%UKER&^E-LdWAUXJa6B+7YS}~l#9|CMQeP{8C z0`c?655YZy_%?xOF>XQM`hXY^VL^iW5E)QmK?nN&b`>OTEN3D|02~+ixkGSi1(E*6 z^lJU#VNXZJ4zi*JbdmZ4W1eQ1A^j$~+$ zpIcUwsy|KsWuKyV@gK9gbZ-M8C=K{TuOk4&m?`#;eTqoyOP)t=ci>JDw0mf;v8Gcb zJv6tmJ1X^V;B~MMAe`2B+DkTf$WEAb(7R*xHh_1~y91x4yZqV>U31>$`S+Y#UF znQc1RgLwlx_N4)lCaJmTXBEo;3AR@wS3kGU| z{M04%S{>7MtzXuuU)HK$275FJBXIp!qChzW_6*d`V?PAN2l5f*pCy0=0_`JYz=Q=B z?qguUjRqp=qhP@1fg1KF@8j8ipkyH6f&0-10zjkzneiVY1hu6hElI2FGG3)MUik#v zCk9)yFS5bn7JFR_*TcE8B5}1Dtch7^Lh4{M zToyZB3;%{wXGQWtNAwS@jdi=$*Jhtuk-z+_v>NOF51@>8|ABB-?mA0ywX;EazN;kp z*Ig9;r~jrDn;|VcNSt#TJ8Uz)?fMH`Gw1ng8N<_7TwaSnh+lLePdHlAMkz@g$HTLj zh)bjYDVc1I{sbphOIEbD+;CI81Fi8dlz&2qsU{@ail{(GBsnVF5lNAbcwAJZEt~=a zX^;4SW(Hzpm!n^l4tM3sSP{)@jW(j!pMGIF_4q`=wCDn7I2WYi0pT&ONOn+}6QT=z z!L;C?Cf1T*s0*TxS$H=z@swy$TR00w!ang29g)l^38w^EJP~cIIW@7y6eJsyzsW-* zSR{3^!4xD16A>M(zsc2l#(yHqT2ro>hFm)}iFRTvb~Ubu{6xT zmq>@iAm2ptCSFgZe4N)FVGI(V&a~Vd$KQ;_U^3=}OdI2|X#Xw;ZjAp4k%l01SHvm! zf>l8l?r>Acxl_VSy#Lyahr=@#hFqN$WOM$TL_9+i@dBUI6>L_z^A#5pSX;7_7w73kg{Cm{$x48yG%3Z?3Oj zcV!(yGY2JQw~NT?`eO}O6k)j|4a6BwDMImv0EL*Yg9EYdD+vhP*9PKzr=tOQoPZsu zQw|XB*xuM3012e)ZZAB}DWLEFBgHlWj0jILCB7+%{1af>j{4H@)?@i}l!1)mWzv!Q zQ;Zr;uuCiYF=$H~2LFy^fvYI}S)TwU!Oy$69xpVkQk1OY9+t^B3lyXa6yyfUaAK?a z|3YXAxg6#4SqtQH$0+1?5lPR3l0Mr-e2nKsRG|(-3^ka;XalW=nCh@Qkp?=9tf#0N z?3(NJ=PADxrvkEY{=V3fxt#wo;ga^01ds!BfxlcE=<%ISg!w!H=ppa-db@E~(m-Dv z_4VLSCjxvp|880XW*}ay0d^3R%H7r6>)c)I~2}BK}ZvtFJj!AK|3(Eu0rPJD&4*Il@nGjd zPd|e?%B?Se@%-x`k4D?%@|Zc>{~uRWdfi}6AAL1~^gVw~_`fIxroqubjc(Ghx2lj~ z0=VIGJiwm`4bEhJ-@l_O;&fH82D$z!CXKydRTID%@OSG!2=`wpQa=i>dBxyA(FE8E z6MVC?KkXG@2wwa2KU47+YYPzV%-&R?6(uNbXai{?Ysz5S5I`|vGiraT&PpPTwxfYG zo%L7MEUty{*!DK^(4WHX0;6zE^{FSTPS!@p<01KogW z;{t|&wW0N;fvqY1X~6+Q5Zc!NZ7POpM(s-_UQq(mj?|Y%vZf5A6$KQ-HYfL`GOZ{< zXlDV2zI@U@oN61;*+SOCIcyE)P@k&?KtDLO&L30-=!d4hHW1~jVmR-Z2t!lTFfjSh zoQCh>L^5A|kemc_dVF2WuEkmH7^-C5UH&&Xy~r^!`BqgGmEuxYPp7stG~TYEi9vm0 zXi{BOm6Fm_U$3UNG(N7SsYy+3W%9qxdyBo(`SI1#lm>Zf_6xJa!Q*=a7nDiv`tJ>y zYHIW2mRg#!)Rh({tkqQ|DT|Hk?rvq>M*ZVuy2w9OtQ-WoNFxt_w^n+_V1|acYM7gy zL+k(;)V>1Juw_XZw7`yERJ7{#w8R%X(|nZJUWx_^Vh%@iDV2XwW=5E7>)zv*t~zC< zUKuw2>xYEqW9tTZUSTe$cuRHk9FyIf3@?ELr!eph^rOal@nhYD%F{?wyE~g-tb?}& z!SPi#`#5PXO4Fb{%kwZ?e{|-8n^wN5JoIf#JE-8Il#1k2AP)&a^wd^n8Z!zPMa8|H zJU?t<9`z`fJ4cbUVF(|K$xrp ztffmNx7tV!!MK6~^bs*(L;dshF)`uA{I4vjnAuY@zK`e3_y1Qotub?5X6`u2_IMIIx!?Djr}aCe+s&pe{;#Q^4i9^Fg!6xzG2G(qy%Rj{ z8Ibfhn$FNoqyyBsY2V#o_14sXk9S;e8Y(&r3J09l0q^&l6!9n7_p3*O#)z0;Fh#-3;a1 zOFEBzJUvzNAQN<(&r`u(wz==W_+s7(kkbZ5Heb3A65r+4H&wIDNe?1PZn@N^QIz+T zZ&~zc;^m`SOxjzO>4M#&2DKgQMw4d>IIZx9q{)R_gUwB<{1z4!01=4ga`*whKx}+Gr8@!xvQ4v83$`>9a_4055(apqu$B+o!pA%h*itumRK@f~bM?}13DcSt zrajC@zg2+oECc6V2*kG(L}<$U|6^Be6IHs8QsLK+s~&SBWb{jyFr3Rjr)Hj6neK7a zxu$wf33u)1>)t5TdXz2mELh`gYrPDtne`99P1l+US@{z4Ath7^ zgE(~rx5<9Qy-x2xH+@w01U+p$M#b}Y)4%!c!(V>*@joB7s*P&!t4rOVEdMA zn^Aqz0nO`aNG`3&FF3a=SB8u2Gldysnj(cMw( z-r0&+b$`WTC|?;;>3&t;06x7bc!t;W(ep)3Ed<~eYrY|>kp)WXGr42RY`wBD;7vkU z5GsG&irJEs=aC#tI(JVDC;xqAh^i}NBxlM%K7Sedt@$v3o%hlddR=5o7B$64d8vu= zlhw*7t$wO_wWc1Z_gN7}xqjXOvBJ_>9oBMp*_`uEyDhAd1w|GxwrRi+ax>rS%z#rA zATtMT_p)J zMH+l1S-M!StqU9o!2fVP-L>O*QX+DyQL_P=fs4jfZ2gu@YB&Z>jA|YFP26f1BbMS- z!v?JImA{>xv3Uo72&3};l0~SyosGpeJj%!8y)6pcx@8E(51kjr@subDGxo%+@Cx5^ z(KfB3WnN0fvYI$;G;v*H;Qh4blZY#kQnvxKxG+0) z4EQh+lQP<-CDmA}Hdr3oV(a(l4}0%dc;|lS>*F}unv-g-%wf-3A$o4hyS8Gj;vdsa z7&l7^bIykA&kT0Iu@q6n7J`v}*ItxFw{DFy)W9;=TKA=l>8vJDUu=&<)kM$2PLQd^ zRDQhv&RodaE#NuUfXP&Vw)G)6Xp)DXlFwSqU1;se2#;p*B`Usps_I+; zSChpDB$ca<1YGF{YDC-BJ)&DZe?o2JwwkYdk=>^<&hGAt7M)sLc7x2&f;F|-=t!34 zZ0HBcqZ+}+@s767l0Jo1{3v$tR^P@Q-fhE?0f9Qg{xPXeS5Bq6!yUH{ZzgrSgD6Y& zd;~tF*LYm?$Hh=j#Cift2-4daMH`b~H)Y(T%guL}-*uCAT^`<5aY{Y4obTm@^fCvy zB(CQyg%>#=g2n9b-AU2K#x$fxM?k5GveIK3Qo~A^sfl&TGPk14BeLN<5yiwe%M=#| z<@5>aFn_f(B|HG3e5B zE$&8t?TgQ--rMitsrM55vr~(m50EmxmzRgv_ooi;Z_jia8|x1*BNe;i))#4|c@hy9 z=fQb*5XynBkQZcY+EIQd{5%-r#Z;>{&@Pf5w{3O{^3SUr4@u7nJ6?N5Q|iZ%L%3J# zXI38b_!C$idDY9_ej!)b7xWE3@3Nnkyx;fTUe};vgL0b?Zi}bO8((Ko&UcE}n0rH@HFv&>Y;(HSA1nx%}3(&HxHp|u#+ z8GUV8b$TSmktb(fGjxW`5S|O=TKnR{_S%7At8L?sC*R^6ac+DX)3=_3J0@aw(9oIQ z0Z~gBr~6o~iuJuD?3EVHE*8UHidE+i#vDEkDf;RePP?!7fhoPhb34SYp+$w&ke4i#!Fl#x&S);Dj53J78`w;5;lXBIh^O$hsgC z9@~Q{!35uUfTcJxycAJrEiU`H*a++)O4%$1-^i}wER{epTVbS#U+dTzVMN_3hRx?Q@>o@C>~U0%Q^-^s0}M%HDY5`q{~Rgy^I5A5m1PSuQ?Xs(jJdOlHSGQ(i0eBFe>s1xm=ZH34(Jf zE2&O!M)v?4;aRkx(9FD7}J@GM|SL>DKiU^ZH23L}x>nVys2A=+Am$MYSv{gU#@D)qMl!srH`i;;mx4TEj&=Y1}EMZr5e-7 z*!%e%%MU5>WRHIZ%{uBMukEg!ZH?52R2(VTj!E%DB-i3>B2dU8<~REbDGj zf^Mn2T;E?+1i3w_zzbchYl@`5tF9mKaGPruIP4b8)N8uBL(VRm4828dZ6qL`V|6@J zvo4o0h2Dg4%!I>bbqG=v7#u)9OaQj&sXB>5oUe-`rPTcez>f#X1ZZG}TeXn%t+z z?_hL0BK$>6uek51Zq#dX@=92jx>Q6z-aC=VHyY&3${ewe*9^Zj{gT}O=zR8npkw)6 z1|6x;CcXUaVZzSSQ*b7CrbpRxxSB_|5*fW>`A*79BdlxrL0^qwOyhZJmc3eagSnux z>f_sDwQuM2n%(|<)luxphx&-qVpKRltQ0`fbjZ zxE^gS|IK1T<(<)3ZA0F2q3J7X{!{l|o1{n_3ueI+pYDcCceCe~s z8HQMwhGtjP^^{)by=|^<*W&ALf78>A2Z`;kw#R%w$+9$oF0leSgaY|$g_5?zH6E3s z@6&g}qjxcb46Dh}{5oc@3C71MhNzre(lVqZxyVi~p})oAvJV*}XsjrT7NCzx(SGPr z8v{YFV#OR2pYP4r;0A2-&B2Mb>%oo$Aw41m5*S@hD$Eg97c@0SLKNmJKP@OsYf&7) zqI!{u106>sn(x6bk85gEn_(KI%k#@Sfkisk`QxKIE+9p5fDAF4TL6}RTyB_K@KzTm zXi592c8c*}HzYf$l zek)T0s8wHLSD=-vQcKNP(C*v$YtCTa>4evqy>jY>nH5%CEwg@RHrRPqvhFr29PhiE zrk!lZxPIU2LJlp-Rn)ofELE%x+Skg9E3@stoq-)$k_)MGJ9whu9IIMasn_M#I2GyU zYkV7ES!meMnDJu%Rw?nVQsg+Jy3lM%e#Y8|x1p~)V^LK~`CIYAlB$${jgsXF6{&za zrL1$x4~b>+7-tjYLh2;UZy8|6O$qJT2~29ky6Q2H%2uwM(TYir$|m=_O^O*TdE>af zR)u_5%PA(|HA+G*DN40sCYsXfl!EP25wogNSv5-7$5f)ncd?6=KNV`Nn7@UI95>z0 zTVb7QON;r%y<1sf%n7USuumA67{Aump5cYee)K$;q72ssxiDLV z@%y3le9-qk_H^%=43P97_H^IQZzI$`DSxb7q2;r29xE^6Ek=sYjeBHY+8w=o>w3wF zt!Y;Katcge!8`nZMSr)Y{8+r-k@4)XL7=-gA3{Ir#vP#i0k_4W+V|Dq74H}=W#Sm` zBb#sfWW;UKjgOSUZKuT3*Y=dnuM&R6QBgC;(2iSU**Gfr>fw9OwVTA35w`)#AjOxm zhX>^Yf;;=wmXp`JbwUBB8NCZQ5Ef8SR!~qMFVMC7<)<;d5!3i- z`K^6}-6Ab6oRAZq-Ji5e%Wb6r)4Xsl=jDY?@+8?_$H!GDRSk0f(_8_IYmA}R#>RN% z4@y&w=q&SC6ap6onMXXRWvY1@(_+3{I73I)sD%wa^VqB1x-r?xjnk4b20u+5ixV=0 zoJ}K_8&Hawl{<3^DR{*VJYy|SI3pLGRJ3Olt8xkzc*XNQV?XZ{h@97?lCu(5&Mtm+ zNrhg1qSP@f(c&V1a&5hH!MDu2QNYU%J3fX*tzuU2pB8Z70+p<0nsUa6o~4?XF(cv6 zjxlWPW?AGc&zY8-<05BWdj1iOm6^vp&r=t$tmwy9R5r81t6+qca$bFGH3l1vTo}Vk z?=?DHAK@)`u@fC_l9lGkPILFJgq?fym^ED`MqU0vAV|{O?`*yJ0~)i8ghql|f2zk2 z?ARP;_A@u4`^@uVI|-}(1GwH_U0x;e{F+=|pN<}37OyZ0n&>q~1i#@y-Z|qg%sC*| z?BvD92HE35CV0k>G(LkJyTHw#U>7Zt1D@@0c|B~Z9Q@UC1-gC+YWaZNdA^+7rFwR~ z-Fk+Z|NipsXUH7Mb_?c?x#0o_9hfOh zB(xxnSTxQ`D@#TOO__#f#3uRunXP}M&k*4q#7GG-1lTk9|9*KZF*c^NiYWQ)uE z>BS~hDtywAG5BT%0Ye@pVUSC+S_G1dwAjPbSyXfAr?5454>@rz&~*1oEOW>mktQx~ zZ7%J!?w}3zgBK$tKcSOJ}DM;v9(6FUf=8p#!3{2x+6C;yz)6 zCb48{Bwhn3?MO8MdL0u&9U}&wvG+?<;Vla`ZgqBE)6D99s5jFbUef?!Q$-uoh=Khw zwM=qO)6_+aoC7Gz+@67~S(k=>xdy3PO`|kU$z0AMTPLlv{b<|FmPLd0IjS2ZYVSDN zb@Cg%5b8<$8CKt3qkWydIPZPxwCuc7*xG*1n&2B9IE-7ax{F&fojLE;iifSt^-%IV zI-X*LEepD@O|wbvZ&zhQLuEt7qpvKEeA14VY-8mZl5)KAj+S&|oRfPfAd$K%4#pNJ6__x8d8d27JeyJsyXhdHUJ zVh;HokOu35O2N@1K#?K}k^2WmMn(pL7X=e{xglyFDh@)4iHN{bbZH0-iCqvSD~OVO zS2cjZK~ENh8nn{`VUB);bT^yM?s_quUT@iW!Xt#$lcN$0?RyqtBp2 z84R3KMln)3Ffo}du|Bzw2dzv5m2S>mg*;Sip#;8e!2~k5A?eL}-{Tlb9u#{n8m#?C z4E}9NTt8N?00eV1QnWGZj)WnTa{4$+Nkf$U>J+fo#Bqfr(c<#kaTyXaUl>W$b`a@C)=+*cOkF9=7fc-<-4-4$Upqz~xq z*Tp{-)%lASt$+;-aR1Da!1?H2K}|fZR?e1^V>XLRI?sasj?+wKZM3pfhGk3HQl&i? zz@)HPbVbLKrp77}p}yOv@qYuw(}b#8OhnV`*Te!)eMw zr|e;qd^66?mYJ{P*#oujN3{=}wm7KruUGb=mpuzg(O_K$SC zrmJt&(I!r5ZHmO(w0ACBPg@`;0P0qaX7p5k$ql4X#$>8TYBsg+5oM-J5r7}Z7C zg^Q|{3&K{5*;fntfjJ7v&z9exEjpg9!XV7aSu=|I)(Q>QO1H4{->{2=u**TCRoSc6 z3YIk3i+0%uh^->PvCF<;7yZUA=<{S2xgKm7E&W{6)t z26|Qr`G7D;kl;T1RI5EX&5NO?dViG@icJhcztgC5$XugNQi6hzfl4&N_%+a`^+k$$ zNY0aBmT6#PvuA&-k@=SWz0udsQm|atE&kuwXa+;M-fZEa|6fHxLW3|=64o2 z)c#^;vM?V8B-e<97vm_Ke#B40pO<=%Yl=nYe#wFCD?gkCg-1ECZJr3w-<_2jM>o-tSCz@W=Azx99q`;* zn@gLfI8*SmWIBOtCOrcfbF)+@$_$s<>bRMU1yF!xU!7l{0R4co$W;HSA#1cZOH-3E zPlOaM0O@Hl93?`J%o3Ex$x^D-A((iwG+>g+7cYkRIO*vn z$iyA>Iz%^>+9sjcZtECIFz+7U7A#{Nbw;?g*{9tSc3Z!eNPD%0sC~rQ6S#JejhMq? z;D^#dUwWJjMi$CQk3<_69n)=*$)$cXE)gBwgEL#@93rnwF)m3FtAix?rsw2T!f4cL zOD{DYn{wo^+V0Ol+uQO~FC0b1s7XgOO*2#eb>dn!WmluSIFV=Kh6>BbqTD!Z?By3$3LN-lrVE93Qr=wN7S1gsy+Smh_tv(r$u*YO= ztQj0bqpnXf@nAqQp}I1vM5#DmyiZY<9Thx6=FWv@PB3(*rns9tCP@BG1c`l!G9%tW z^&0a<9YSg>ze&+^lC)hz>UzI`Kp(a)hyXuk@@hv~N@wog7cA$9Ks^wiTaohuN7@hj zRed9VaW}CIDvoZz57WE8i{iBPcot2SQ{Bn@t*4BlAO2810okaWf~sI_DoSDf=q|H9 z!WMyHi4lEn?xdux73fV>CWDj(>YX)&tn*s1mN=yB&VIQGW27K)U6AwFm(;xQ7au2T z_e>;B1Kz1BZj^zq zu%@d5@T4p(n;5}MfiH3j#Pb$u!a*eW9;hnsE7h$Dyi}GXm)51q=OJ`OuUsPn`s6)K z;Q>`dEbO;zXtO**EXGMG4$YQa#;3i61;e%qNUYvCtJ{4g`r4sH1k{Eu2Zzh-vI~-E zJfqROOx>J-UYruL?U|cOI(X%_{Gu}Z1#Ha8CP1RfBsj}v^_=U=;r;%^DGFx`n(73( z?lD!n+PNSj222(2w4fH13h?6R*D)E|b##!6bp?}(mKPaW7zXqTQNMqv-_`HW>+3Oe1^kF`4=zrL7LiP5>B}~Y+cBZqDe|~B?@C5x$jNaFK{2gyn$H^e4253Xo z8LePQ@dinKuD8@0vs4~6jid>PU)5(4%YOz)1|qkzyWDLJ$kQx}ldqS}3GW>f(!AGF|bL!&*0vh1<||w*$U@`De~FMJOaLA1|J0P zgz5ZQATW2Qn2z8fh$FCJ=?rto{Y_rO+VHBY0Egrrp>y_>fYtC`9-HybOtcB<4nx=c zxb`z)k4b-4n43bnL!xK7bNfuf#AJM`a^!ZI1_y=wE_poj#Z zBqaz}QPF=8T005# zX_tlHfyC+b}%MINEOTS$;Eh(3(mv5OB&FFidotZh&N(WHYN^VkM&N`@!Fzexgv0 z+a`9OvgX)So0&4QWNGaX9D|~k#9PF!Xfm!pNC&x|=PGZ%v?Hj@hdQ5QwZ{oN zx(a4Lepw>gEFwIrR-n;~{XPmr8Jy;0cVB1Oy9`Y-gCa*n5ZeRia8R;PGI zkl}SuJQcDQ_{GGlW%;z#UHjt+IH{*UOAQ2V6aJd=S;0{&g}(F$(Gh~&>P7}@bGeKO zU%TaNm-K`p0gzl@FIr9H3o$v4@2FZ42)+rMrZvDfVk2-tA!3lVjj|})F$13&g91b) zP9UY#1NLo0KD+3{so857oUpalWg`>o2nP8MJEAW!VPhAk$;dI!{(0Ke%||mwHxGPb z2gQw6hv&nEu=~N6(AOq*T6>3sMum);b$qv=IBqK2&C)`#^H-Imgpl`~IHMjMz$RD# z?Dne>BAZw5#Z;)}W*C_fLTkkLdT43twu;c6UkUM=*0Rm&cc?|`YmGeB64dI{&{n@+ zx;xDlsxRpSk$d<(3=&oK0UgSdeJmb2zue*U63gOAZPBYe!R^mmBQ&leaE+DAF51ji zyAOb{fPdmzs<9Qv+t)#dBOUx|XwWPCc(?!14*oF6^R{90Vd8k3Et=uabvE~mf84}; zo1bWCGu!VJ`EY)yQl5~05lf@v53#O5_+J1wK*+z+z6fb2bBw4*Ls}*F%FSXaszT1z zXZCr;#QTUEru*ZMpW|xSN>82y6r?yC(WbYbqc>5H;kt0W#M*Pj3fM9Vy z`w%mKz+b>yw4Z42YbPJ39%FSMt3CP+eFFc+JouYP!%iKv{zy{eA`IEh+I#=h2=Za` zoy%HwxIUK0cwFBghu_me`cX_^4}GowO{O?3p9}4w1L~azelUs=GefPx?j{E{%pdR} zcF9_TH?M#Dlk^9_Uoh)UpgTSG-YxT--O5@=TJ?xlZ`Yxjc_oYQ?@`@3ANmis zju^B9eNK(z4s)_vWmY7VuXtqoq9<3Cs$?IV)GYtpP2n~U|KV<;AMHO)KMF<>&B%Qb z)A|o_&-C9zOoIBA#NqIC@HZq!iKLJlt!DOTu+x3;GyLnN3j0qJ&ET`0456Jn5h*Zg zCzWvi{I!20dQ`&me~15ZI4mFFFPJ7L9Wny`g6c*t1b#Q2K0(C8_Zs+N(0LE(StSwA zcHB%bDkq{p=|6?LEYb}_y0U>Q#Mh868hJv-kL?+INU@Rlx^U(BHMw&OB&5YC&NgS& zW~x9dk-XxKJxv1Pt7=1{rg%yn`B11GPe59F_`#bU0FaIm@StZ^!m-Rv*nt~l;06Yt z`s<9KMpvE3T&1{j?OHXO4*@hcrpf9xy&sKko-f zJB^_YI|O!327pZn;Anz4Lh20PD~kYZ4bsc|fxb*q^lC|=-EA+F(5vBx%YaZA)$#fs3Q z-hCK(1<1fY@Pab)$NV4zw>oe2gNy zr_aZ%PI#y;*V3E0@%1I^F~c*SX--ep=xv%ufkh~UT+F_I-hLcUd8bu#_!am=~d z)a&YBHh$zQW$ z#ev55!B%Z=JODjE9+RmLpe`lgF&RY{c-WM(AsF@z#GZ|d^o8R~PM<#V@t%sRy&vCs znm$E*wPt>HYrcWbCVpGH?$qSjPj9HDQy`^Cnwr=&k)ZOJRH4~lyPhr_MgIZm=F$J; zEy3-FMk!XWLT$)|=DZ-Zb;G{3XUtEH(_*e9qbN$`Kv2&ma25BnQ&trdt9?wP*( z7Qc0Lx>R0Zi3nLUCphA}@4dV>#0SG49_PYya0!PK4=2J#=nKOGH}G;F>OV)n(|-;m z!wE1@%WPtySq4;a8jQwI?wNE$-;dtqWo65~q(F=1qd1(nMkY2u^q6@!OIQjC`vcM< zkPpx&1*!!tSHv~aMf5xLcX2{KUl>P!1K$?`tusIVjd)4CMEz}izM~h`|BYG_Cysw3 zt^ij~ZjpPnTCcosIrmJOu|Lkz=AKYX{H%rZ?*6!{4p)bMLbBtENUxEsrcd=>0S>2E{Epg@ zEV8RF@?{#c95@L_|7F5KZb>)LW3op(GDc^ChmzFtBqk<-{8I;6eU?EPkUFE_3+ews zv8aC(@$|K$$wJ~(|17GE8&GE}`aE>s2Bk|%UIG;3M@<7)$WxRN#{53yDM+k*e%#r} zeTwnAh>*le4t+r&ym(}@6YIt=tJ35J3p(BXzn}2sZ4>UC9^CD8i4^vlhS9$2MJ>+Y z_HQ1qs|-{Hax4a?5S*k`7PrQi;qVYsN46BsKG@N}Ol8Y8W}C&gz%aad!u&_g3I-Vl+p)n^HNe1qIDUhyOcC>k2od^UM5dH8 z)x}loVv8^-B;(*jgXCCX43`FFN`l(ZF(?%@YAYU8d3+@kQepGQvf0s#EvAIXWWo!0 z#0Ew@M@^2N*e|x1dd74m5t3!&%A34ab%u0Q=IjOYbG*IdT8xH{8OuuUJvCI`iz^n~ zk&*Y%oN>n%mhwbuIrt4%la{6S`%;2q%U-cI6s66b{^ZhGDy>R0HBf06$z4^MnN=?N z(OtnMJ>`^gY^TPk!GbqdmNitZKRe&-R3~6Pn-0g#YVHS|WE??9XXK7bFW56?%-$)1lEufnJtK;|s>C>p{vF^4YMa4Wx3aTwc4caI z=G(cIHJhGYwC0`LTO>xeCdZ(Hl|ozbLOp+u(&5@e;~@$H713?YjCTqp*&|e|lva0< zJ9ngCy>PO>*lE>TrJb2G=FanVJ$73|`E4(*U2$#%sT9i;8ik{Ae)S#qYK&UVT+}W# z+um3=@8vzMjoAJdL6` z=~Wp8R#U*8E(87xc@kA>MblVc%l6*flEn}7w9Ii4|7}k-)h%mlomZ((Qdw2h;j8wX zCZpdjui5(QnnkbgYjxX+Bo3T=LLRug@!ZqP%st6g%v%uhb{zBOjlr8a26H}UVZuv% zb{&r|?Tp3bhaUp}PU1bAL0hq)t#w9`3UNxBt{r1IEnEM>nnf>euXo#@J@>*3h$j!^ zshoQf&b3RKIY(fSFTljdkqp5xk#D#8i_zfdCg8jr4h;dNg2yMku8}2aq;6)Le`HW+ z2-btv*{iFeZp}WwwV{0FqtiNf<;vRYFXc&+^=?_3LS~dCT4ub|cD6_?QeQGQZJAO$ z(gI`J(%y0t`N>rJ&o3YR`+XzJ*PUD3d1B3ofVXu`_Ns|GZ?ck?#N#PV9{p5tN!HA6 zb+(Wh8?vDs6x7=|4jRfCV- zBda$(w{+?wD=T@)8aZ*BwJKnm-1X#!#cG3^v773h?{1%VerIdRiZk=D=YTxGF@WKr zVi+EVkC<-Cn5m&9Dsd~JEDKm4733o%-_@prf9jGZu}LKp3#$$PGMzY?LuQZm|`3pr+m$sG^J zm$>*iaIG}bb%*4Me-Fyj!s(9HVyq~}2>x$DW*BotOBvPLM3x%0HJCLh)y>*~KE^E^ zA~_?Py<^{$Kz#h2hLq&Aa<4utY5!|CZkRydo6+MKt2^AK+|STD%gtJ1h&CaSW6Kyr z#gI6$*>4=8brmeD3~zis>?hx#)}f_88){wn=Egcl?F-&Ew(Yh~H~m&sg}1{&zcs9W zk&kEQ=TG0))<2uLy}m8O+ONHlLNReM|6VZeR zSJuSG-<^_dD4tNrz4jAR%U~TQc7KkI{X>K!`;k<%_9(ZWHanC}?ei)$41KomcKx0E zx!3WNRM|wDi^k@$6+~8!>>TR$1nwm`ch9 zXQ|9`kBOCU!8;O@xDJhGNxTM;MRNHYU}5W>-g4X6EMDCGN^xQB42C0WZ=!f4PZ$>` zq3y;Kv>E8Qf4J@>R7f-(<+m zP^&X?4W@jT8q7&?=9`Rp&Xg2qp3#)=Ou2SjOc}W%Loi*XN)H-B`|m?OUwwCYiCOu? zSiS>;^0BxX;|C+;hB^Vq-$$TM+|UONx27fux-WqIKjCv>rKF@l8ifDK@DPzue0V1C0G% z$8g2=OMCrI{?xdm$lk{$HcTm06Z=DK3^#JXk8wgp68(3na$HJwT~@`lpIANP{{Z{T z#A(KMwS)RwNdNfnM&Ns?uR?amzsTC%gj=z@TWfy(REk4RFFLC_0 zuJe)oeG_$$>dRl)QK;ST!eeOW4gF>04l}cmJdB%c+cDKv3IUbH;?^V%>mYxqT|X%p zAAi4GCeVA+$Xs?jr|-i4tL7K#!1dhyg08=g!pbHa~&|ID;O#Zir~-_`TkWtJ05)18Z}1y}Q_p9t>sUdyGh%^M)? zz^&-AH@ddr8w|0~dA#LKmW3NN2<9pgM6wl*n+L9Ne+}pOg?JB#b)Je;cAe1|(><}1 zHDkb!+D1;@ZZ?XQVo`!n96zDpVDpIi4QceAXb%;hvVmPig5Yrg$UuL;#lw&1CkbzN zxiXsORT8SOtGaH*U?7uz?-GaQH?A7;+xCm*l_6G*!-;syQ_c|o^gwK(hwFmaBE@V2 z)^Cs;8}!EtDwUH2wX&1l8MiT8p#)k^vZL{iSuTWc@X}IfaW-mOm)UN>i}1{gH(LpQ zY^bbkXs8rh0zgqVj*Ze`JixtuO`0EncyBT(Dx}g;gug zEDko;*qq7c981fDvV}*-Zk$3CHWU`u)D-Kpt@8f6twtfqlVznUENW4*)=oL*tMzeW zg*&Bj)o2e%$<=CQJejD@n^ae|d0Mf%uG?2RPOGfRReH0W;*4Dr8ds0@WaC^ZXpeI4 z2Pj)7)*Rgk?io691^yWG?GIp_F|Z!d;#*;y(TUr2=fG^dxAi&1(b}*sb{%XInA`}C z4XtbQX5dGuM&wRhV|)JrVphW_SYtWWd+xdC&M|9{Fn$5AaX)4A%g}6aXvB9CvfZxF zskGb5ygHq?%x165(Njs+exQ@jS(`A)fZDrY7ZFU($vx+_ezJfHFoQ5Gwn9||1 z%0OIX6|l;)Y?j$%;MYb(Jk;WdDs6~`tKDoAgZPTdJDU&xd5}s%xo%jx2kEyM-Z{j; zY5FcKJCpBo-LQOpOAHx%V>Zjbq6S$?6tj)Mz!_LK;!7H{<_Cw=6oW3Se^XiAxZ&sq z?zzc;#r8FSTg-@i3vWY1)0poSZC+R`^7|aO-i2wpBWT9Rjv1~Nckj{b(&l2;dur@m^GYHU=ZGZ{d=O~Qj~Wg6kfMkLL@W{` zi;)hF1PY>h$xAyL8g{(2Wa-Ph8X9)JymaQ|$us9nnL_DCY=3p-s@Hea)$MqF)yh}5 zkLY{m&Lc+-9z1g7PG)>b8F-x9M6KZPp*DCq94oV%hqK{$C>bs^oY>(wngIsK4x9)F zKFcVWxm>u&5uCF^NOYG~Ar`jGmWVU5ysCNkl~y5EHqCH}Gj+99_Lt_k{K_J@v)@lq5+W@!z=q@Qy3-_SJ{M`Mm1TervMck&>Rn z3Hc=A8stOGj_rf{(yY~@RM#Dut6Qp^GOkh5a z89k%)H3F+nlaXESwPfpsj*8KpqbnTVDMzLTr;n@Gkq&9HR3UYx>1;Z+A)~mnxM9PD z9GS_ZSs+uYv?`I>AWluwr5Ie5z13w)JAE=smJahuAH}@>!aO?gOGI#tTG;)AC_R(F%9YHnyR^8s;ao-*> zXO%bDoR?vD6lcmlo@3mLh$p>Qt>VYc+ z&CNq&J{Qk)U|$!4Vi0pK1U^Djm@z8TiNaVDoMdKtlB!>B=pQ zNi6-p@ruBWOk?d0Rse3lkaNpx$=k9J}bnvzv2!eM1JK^gAvXy?-7< zzv&n&Izz+{X+GB+smrlCWcV%`7_^9x4e*a)8<_Jyuz_1_hG_HPJvYQW&a>t*jqd)R zvlRkmx(3w$Z>tfM1M&;E~ zQ`rtrj8P%*$2xx(i-o9_5wr%;2j`}FkRS12E_yg++)L%rbLg+=Ie;*_;`T~)i3sn{QI;rzOb5~3BKcqE!VbOfB20dOL1I3b(OMh}1V^1A5f z`()a<1BqfjUz~VQa|1+$0T9Pa-NhjU_tKAMit`g}4aSGuamJ5}Z;F>GU<|=c4 ze-6WvuO~aZ#|M6|OogA^zFu#3uOD*=2L2n$TFb0o;GA@P5_~Wz9r97koFk-a>C47+ z!82T8+#!^c{w)|sKMowmRKYVmN#aJ7lCGs6Hz+`nSf51GG83T!KZ#}Z+Y+yxAQz)2NMJ>evLs;vLAFwOb(8+132SnTl+t3-(k!I5~khDy5nRjLj= zK*Cz+(!vr$Nh){_H#2N!IOiv4LAv3bbUB-o?lY3J6cV}#{I2h!Oa`8!i%Ao?&rl%k zf90z+(!3O+{43iO?$ulIX^qLG6{dE4`k@*uVtkd+V zNdl$A={1aE>SDx0{S=nIx=38On(n}iHQK}8s{pt zm(()#Fnx~#ds0jHDIMnXTg7e<_eJ_YO6gBJwokv{iO9K3J+M3pb)|^(l5^?j1PYZ# z{P}YNrN*q}*U^7X6z~WjOqRwI_;)f`kAOK#5+jF)6vW8ZxjrWc(Sh|V_}1a*C_ajWuQt%D^(KnYR#`saZ zB=o~k@kR;v4E-rECdQ5BiQ?l#{INWAUqAxOk0L}zPYe=*Eu;96{^-Zg;*~08LO#P* zUV=)cjOSbFZ-4^ug-IQJBsG6bl8Dddto=;QCkkNfFB;S?=I|sTNS4Kud}W4NoNb>| z#{F|#l8_gt&zn?ll!6)fw?XPC@eTK>(8)h9tC>h2&7rm7pH=#2%|)ZKtR(@LSfzd5 zQZy>lTIzRkXQreVPp-BpwMq?qQ)7p35KRSlfK>5KjJCrm>>yD%qY6w>9Q$o-C#@O8IXZxy1h4>(d%905J54O~odNu42H3WiG4ZC$=^eU+8)pRU_@iN?wqwOM|DR_U}_J8H9| zR2Q+CCt-Zw&^u!SGIrGis@u`DvN4%iot0^6m7Q6cW6IJ_+S4r#hb7(4^=G%_rKaY! zWM?%6Q&WRYS!S2ZY;!sh{a z>3nr=XKjvFn^WtIpuY&({SoZ%P%Dxlk7s0kAL%9ZZ^&HcHH$=<9-;!f!TqNbq$x%v za)c>j=m@bt6e^PA5WN}D{{H~&KOXOn@Wb@Dp?8GmcPfWz|9WwOhnPkGl#*>e`INHA z!?*rJru;I#_O}Iv!R>xFuxU02Zh_ad+VRflw=k4=O6~PB@BRz$D^v`bROkn?Ou_lXi8jR0ghqPCyXn9Sh4V z=R5nmyC#kw@0gfH9foqluHiYL&gJ7hA|9Lu6p@if&y`{$nnmFAALyh4Kxe0!%>q7( zIUR-G{x9;THF*kW>}hS;Go#ScG%c_H3vc1$xl41~clQS7ybKbL9t8<6&k0Q4)eb=* zIO)m-^W*B49BS{`H_`2$xUZ-E(2}}1dQYzFWMZ6g-u_=t+W!6HQ>Q-u{q{+}-apS6 zm&mU3!dUPkjM<>ETf>WesHg0Dpu-&&8TrFE)J1h>Wrm}vEGId8Wy{1;Vf>NOY83BR zbf|&f>wa&k)S?rsGHbINM>$>H8Sj-8P#^X`*f62Cj9A_u%+F=&Egtwc^${h&_Xx;f z9Iof)gUGNdU=(th)hxX7ZN^G%q&~`W%>eu)sXA4fqt7n4)Xu0hB?yzb^v{&i=CAJX zNi(b}xhd$-@Lmz=9U3ZwTxShzeQ8spL$Fk3O-afcx31Yee)h&ukJ3*qDof!fDMWAN zZ@P4Nt=O6>gSxO_;Ct|Ue2)v>O@v{r8C~c=4!tewX@%tX5`{_QDY02gGBuja605Dm zqv4+Ee`VZL6DB-0jwrbLt)D-&qdzy{al6C z(Y*`HC)_zB*HO|?>E&xLDdH@}jTN4zN?*0tCe4sfxsDlUDjbfo$KTdmKex5Oo1MDz zzO34OUtwKNLA64ouwMJ;PzIsy(jS1kKn=>NRbpc;B8MEO;m=eE69fi>hHna{DO9dP zo2$$zM_F9|Em0>Z;{|3%wwBbo^38@ox+>LIY0I9!V2*2GfYS?Uze4`NF>xMBM{Asq zaIC~55buC({tU(+Q8T!jcm(Un_uwAL3$)6>kk%+3D(l@tMS*nT@45ekzlU_8&Y3xn zP)wfWBSae3K|i?M3UmT%zu#t@+}@p(0OI*#yF{F(kn72P zZD-FMFUyOQ$)$WrUGLIjb9N%m#VjS~frC6Tq<#j*H!ouh(kK1}>6(Fl?Pj>n-unIKd(F${8$D9?RDXR_oA_L@VN*lvfqFRwesWgx;(AfSo z;)^ebdNhG>IHLJ;yCBG{3JGC+q)YE)u&TkB}F#1JW>Yp1?dl0jA-0 zg0#sw7=IGuPa+(j=e(QT!TkW;Rb)j|5`gCIpd-qq4e z(vO18JUNcBykhggD_3~(pYLR1!o&1gunm95XTFugzlX0}L1p^h{<-h%==!9{Y2&3J zyLY3%RYn9isNlF!+i5gm^Rb z-NR%~-$mk0CO2y?_=)?+!SCjBr(8V}`mTxoJ=rsG0DZ@W(`vG(@7q}~y~Oy^HqKni zOXXla9%3hLjhmmGGWE$Vjg4ELoI2&n&5cBJ@9B;8^&3z3_MF~OU%%lrvNN!f0HizC z?~BcaEgsLvLbJJWq{q`zXeLIvn)6Mj{AQP{3B5MK-{;X!5G2%dG+#i&8`4`5V7vyj zDT2orlAucIDo#(L66HzTYp0c^rj|{s-JT>*q>|E$T}om>&l@j2GfHLE%Ho>mef`g$ z*RA{cpI^^!j+1Gvs!`9p{05fOLq7q2hV^I=pRfmzWfTL2M4U`OGa-<#`c~;GNn>i+ zw|@OTriQc zW5#v|DjWGQz}@Wa+SuYA*V*YTsC`Uk&eAzMJIA?OHcjv{dX@$GE@t^2?+1vP^?AEgJa|%rWRCfhOq8Gl#ItF}R+p>Qm`3-C&3>D< zw4&NtEQ}#sr^^>Mds(06p%@De*hh4x7#CuO64*wAo>uR1uw#;PF_!X{NO7>h6lh zH8%U4hQX`^4!%n>jr%N&QPFG#Co00fp?RN?`|R)O?_Z(6pY{m4i{+7NC#d+rbnE&MxGO7^(QbXU3}!iUK-PMrO9oF3isp8vSWTe^t8J+qt4<)pP60CO-J-+Izo0-XhkR#A2a#mNDp1 zF8&bsFHC>y@q@DqoTFCU^_v+ppWD@%r1O}{g0AK{rMXLHPpWqC1yTW%lUFnFJrCV| zmCmW;jAhr-qf^`Xk*)!^jdid+D0H!FVDPFle90^Mvp^9y#sBMtlkZ&ar6%> zR{rr=XL{X?4F_9S{NZQ^p%=tYuWP7X`*8P^M^;xh7tD^6OB9k#3kt>+8N@Qlvs0fx zu@zC-dgA%1Q^N0`o4WR+`^S&J|D&~SN4Lyv%$RWElW4gGoI>mk!#A?Z0d|TJx79NH zdND3a0}*Y2l!4S|2DE&>%9Az<{HLycka;A|qBk2q2*}4Jrl<_`k6>c&zVS|3dZ|li z7t@!oZvosg#DGYfp;6m4!avjgmdmTdBCJJ1wbt7ICKkakcax=}x5UvLVr-d_lqoa)-i| z_JNGOW7kXclS$l{nKk9>*Du40gpJV~?B3MSu(>A~?AhGVu&FzES&aN>p$HR~tl=1yE+Wz5Y`sWNhnrhK%Losa75jQ9 zy_!E@4=9-tJD|jaAdjykDh$l}v;j@gRp|jR=b&S(H?zZj7L8dtK3?^pH-DH5T^a}?RDuwLe!Gn(R-mK2_LkHkzrR+}jbJ`(he9#SiPtAs7 zbTyhgF?r1qnS@NlVdTklZ17>k-iwOZa<`*n1O*^q)-6k!w*TDp-M=fIBhac;MrT>> zgiWd7qCk-TPr`?v-G(g9n$&Tpoe^V$%|V@%pbo% zDA!x{n>LI?w`mErLNk7|Z>LzLSBc{im(O3^wEK+}C(Q6G#6wAlE1DyD%3S7}}ENW02+h?Oa3s{kyQ`N!>^*>rGO18=U@^@^<1 zJ2yosk}Kn}{`NrGT+}}~JgiH2VOGgS7Zow`3?l={^d!;R(^*zbDD53xJy}Vl)4V`F z`rzE+@{KR95u#7Z`qlnsfm$Wf8f?mfj9iyZDbdA~IkKaxXYMxDJBy&guhFcDM6(YTSPnS9qt3B`_e2>wlQ1(a3~}P>*1IL7{asv}%TCSrOy4&CZ_pVpWI< z-o5N6UlCad%o?wERNm4jR*FV z$lqZcgK*&VflbcyF@L<#x4uEvNXdWjZG)HSx0u)e(8~eg^b1caQWF!jiU$Ni2cU?x zC&rxk_+$7FnQb@GqkSDtXJ-!h6}nxI!>n6V>+xO5==7PH?`8HQ_^@k4z>GupfYeOl zU0di9ih(y^#?cH;bRfM_)PQ`Fc;ICENrI<}FDxrgpXEtQOn$9uuiToFz|B07#no6{ zs_j){5d3)h>!?sgPJC_V?y8ej+f{arfXX48zXh+X@nn1(D4R$xmYJL>AesJ)E?q4E zJ0SG!8n+WzJwAFV*p{rZD(H_9?`Z>9sEHBx*+yT&st4EIXG=|N-S_&UMX&B{Y1#Yg zB9yL7U$7N7r>8d+TCIgm>FLeIHsXU9=&#RpcAf*O7hV9Wa~&P$=&xV6^Tj#o>2qGZ z^U#I4>FIMXFn(z=Y&#A2akL+7MID81E{3g#CV_CaLuvY60xE(y2?HQKzZ<*;IG=s? z8S$w$SGQfv=SvedfYsOBI2MO=3ZNWL9BYJj4A6Q%KHwmP^umk01Rw7;znh)c$5H9Q z1ODb5^~ko$E{VLN_<`iHTTb=PdTd3ht8r=`{nx-OH&6v!Pxk7{G~hvhcUPBh^!QqT zz>=VweSDU$c-~#(o42l9m`{Hsmf(3DY|lJq&p{edp<-~-<5wsnLPKUXf#uOYhN=Cb zMFrYGOV0PBcAcL*?f9B5^LGC@nQL-IPJ>sSRIq&3*m;Gr5W|!h@liN!TYcs92A`zu zr$vu0$aQrcm=cuBEFVc^yZkfeT62gG)){i_a%Qa# z%Gd;Xkum!}eC$inOu~pK(^QjMK=b4cC7%;@o{73~0mwUi2kg>|K^%@moDJQv;1|Ppwz;vrwHlZgKVQ zg*CNXo?W!${QAoF8u~SPo1gxi{PDx3J-!yd)|l5s^TdYaEbp{qvwBW%Y^d4r#H>A^ z_tIs$G$;=`P*X*Ght_zJt$4*DO9^Y8UDVP_&4+X3D!o6BW#3>v=I^NK0Y~|2h*P?RoLpBf9)WUBw1>Aefug*qxO#x+YgK(N-i7+9Y~Y zqAtstmZgc48B{6pabme%K?o2R=G-cC(P*`_Dz|JxfmZB+Zb~Rrs%2(fiawdUGeM%2 zsLc|INR|L7*xv+ft9h`k8T&xagwNQA-0WlQ!z%{UNbX5+QXxq@M}Ma?a3!GTGO*89 z7a6~rDN@~WM~ZYDc#lRKZGdip^j>A|^|y!D%5P6LS{%I1j4mb9aM! z5V)KEE%4qA2gkE0{%YgwgFO8jp99Tc>HtF`ZSl}`Z}O8q6DS(GOEmN|~vHJx00^yLP@Gmj?6Qbc$+!mZ8R3f#X$qs-;_~roYxBzU%fS2!; z6JOSY1$4^dj~4$Hl+@FU2r%=bnP-TBWnm!n(GD5#&{}kCTc|$)L%* zl39pDX3w`Xsw*7^C)K86uZ3rl2aBsrzL`@dI?N?L3(q0XEw8oV&o}v=JeDk1@Ltf^ zREau^PNY!9ozpus0{EWZEs`tv7c^G*jTwHUj60`8-~9o7aAq;>2kUQ{xInXqIFmZY2m(V<1|Bk`5E?=h}9PPpekd{ zAU_GKv-=Zr%%}8=p!ieJOrOTTKh9|;FH--2zn8PW$45yc#W_wo`YwZy@~bVSI*?>n zBcW34h!|fkijxSICp(nyfOX5k>UWeGB1rRDNb{>Gnyhrti?iZAi)f<1YCA~Mm0GIv z>F@0+h^Bryz3v^QBYC+%5+{-`7G)^kL2b1Js3{8C9T(dw6py0ltNqK!-9QbLpD;0E z%fR1xO73$U0lI4outNuQ!J+Hi85N3H6jBjPtJmUs9l zU9OJ$YwencCs{{N==C~%{U_qYi9Ij>`rw=*&&YYDJ*Vzmm(#Ougw`CWFy^=I=ndp7 z`puR&0iRu0;auiUA>YJtYE$%Vo3AI0_=3;yb6O&EL+#NwWGahDazN=YN@a!&h1!{B zkw`3Q&i>zq->Tu4GMUjqET~%36%2N*sq%N$6=~E(bz}WuWNclLT2oZl3EmJIa%{F7 zgHUeC*6Fe>^3ZDtCqS@wN9)M#lY?q^f!SQ(RtG0h@@-8`3?(GMStn|ip< zbHpf;wML4gAVXs$5xsyGk{{m#EzKBNnz)_qj9uq4Cog&P_EG*U%g9wNX)={es>3<2*XYya-+7bqj3Km?fSKi!I4jtW^11Li$zw^? zvv#-HI?|H~o>-^Kt#Qh@YlT%T2KXV2dT09K$(dr=8j&(NUgjuuU7J7s2n6H^h=Tx* z@nIZCeZ)I~^6le+F|;KT&LG7J%G5{wpU_wTTB~#zq+p)W6_KuzA+;0C^E3m)oxf@6wsXSJz5JVj`#iqF6*-pq}o3cufDVu1-~W zWWOu-$$f{v@m(fOX$VK@3?QXH;fOS#ffz$ zJo-8k;?0P2HnGlyM_f1(b&BSTp-02X(#Ts;m^OqlAT?(2nK*d5&fHmnPVzgz)AR_S zpueFXJ$jVfKs^RSk-nEt&_9916BNLYaDp_9kj7jXn-}A~GrUtAbMe9F^_hLcE?SVc z(;LC6Ok0d)h;!%8W!k!9ty>Tuj31-Rv<+Q~SaxE3ht4x&=)%QJ#+M;I-j3HwMd%C} zjx3CFvC)mIB~vwC5W5^fw@dmzqDO&_Y~TE2!IaVZDWZY1uN+z2()7 z)p4td#}uCGtmi;?R+URmAD?_|b}n5^Ν%2WK4ZWpd2mKlgAy2;XyY54zOhsvRN( zrwIC=PP~`py8an0Qh#Jj=gVOqr$m%xjw~=63)*r#jkyN3zm-?j-gIO|UHy(%maced zOZ7=mqZ}OsGP?OoOL1Fva7=;0Sk&(Kw-*_Sqw1VlC+1B#yRErl`-O#n`ULz&=jiWa z+NS{WQqQhQ)WzT?Vl@Kc(onACb(=^~35bfE`Yff)-H@Bx;Fc+~>U^G(bcxVbmXlLv z6H3xczbqY}udUm5VbP)s+v>FW<4b$GtJXX=b?Rels=7Zk1nM$8je%4+iSuOC1q>8d zpH*VA)dWqZV2#aGlBK6ploey9=Xc*bzo=;bz1{iK$5bd?_snbCH>0p{#=f?B_rR#2 zV&FSo1@{FkPZp<=*(Ht65#ifUq3$QJFz}p zrApH#Cu`GGs&u`WI45%2Kfm_0$YuDN``S&5Q_amAyK-{6Ha3skGSTOoxMifbJS`dB z{*zf^OHQ_xfF7~#>Pl$}Xm8$x3T_G)%w`H^vjw4E&8E+Sw<#Ty$0tSW3|{n*rlIqO zJvfbzes<`@fIdrz#yvfL{L|z5|3&?Vod{Uii2>Skbdej0sk{|XYrf);`M-p#R@qb?w#DnIcCn^i6uBQ?MqHP@d_;SHEBnx2+I_~G8@@4 zEo_+tT;?m%z?Lb4Wgg(3$7MF*GJoT!SvvR_KWv9e`ayCbrDyUDx#5_EDh6Jq0-f;= zUvM*|WypoL_Mr)fe3WqLZmP_w!3l@7e`vG4f)WlhJJvPV&lr*IbfxaN*I%8To>`j3 zBpi~@TtBsf-U#Yxd}frM&gQKD|Ea6i|8rd?Km0#b_spA_!@$76Md~8O-zQb%>!NB54nGZv+XQSEgHAEcrTofb_3MFkFM5dGJ=(%Q;~LG zF&qKCWDytVUj&e6On|c{VpZF16H9Gkg*_)#kUyc^Qnmee+rWiWUSGUHEX!U{G;v*{ zp?6s?AWm(EB}>whnUY;)xFq*axo1E6@t+^HwJq(eFbM*Sc04-o2%LdwzHjf(p&zz6 z0eM-baCdHO2Tu2gf*<>?O~`%v&p&?mVSD@X&PtOYxM=62bB@rzrN8{XJ$ygoCA1DT z{C%;*T+bMFGvshrGe+MD8RKHc(7lY8&^-&>*mo`dg6k~#f|xfh-pb{cM9ljZZ{ey- z^xOq%3GwwUpWj(=^BY!g>HNVFgB--~(59O_&aY@1dj^N6+ki2Urv7y1$BwB1J_& zq^XEVZ%c1$L6j;Yy+lAjdI>FrA|fC~5Rf8OML`6D^xk`oiVz?`r~!eH0Fnd}$bVq> ztnQw@_nxzN_nv#6`+uH!e)G;dU%pKF=6n0heeTtrsnzd#4-N*g0Q3$IoW8&CSSaKs;QD6kJuQ zW9H%aQQ;a~ZUHPSORtXjK@V~^Do7Bu`axFHHCe7^ryCSccKEkWdS6SZc(f{?g^RTy zJNJX!*rshiO#6o9_ov;pWR5avCZ_8UJ4FMeWtu(&l|RU^W^2YCvVQ4Xl`86gLOoD+ z=(+Q-1SJGXtxQTS+0bg2nyS3Eb{>bJReHdgzji*mWm2BY0>?5HmQ>^{K~!Q6JRIOd zU%?iP>YsMq(#UVcv`>1!Cvr`Nk>1zR2Ji-h%Po{tLsx6KUb0r*m5*vG7i@RG86I*v ziO%#KsL4-lWJj^>Sr`Ui z_Gg_K5G(TFWSFtn=xD`b9mH10&#^~y*gLXZEa!SocLEj?y{#o=la@btaUg~V*AA2_?!M4yDA_#I{poq?*PzxJgt!Xr-6YO=AE%6 zS&V}%P{r25GbW+a$X7TC3qRFUJ9kc_*g`UCW=L7MDhqO>=vRj8(+W3FEB}z$v?_rV z%``HEIi-=#@3!s`CBiO|OGE>nwXLofh;Hv|uMl-ZcoNGmku^@E#AxdUXq1`S393`z z7GjXa0((@m^066JBX-4-EEd!@Tp{Zon+{|1%#W=S`^$Aep*@ttOL}a1^Bbo-yf4i7 z$|nba+QP%Re)?4wuA_zMFF2urQVHnUJ~ch#R>ydhZHPks!V1b<0Y>Z_fB8^s=VYXiLZkT0Va&T`b_viRZ)6q5RuB>)=R6-|Aq7&&cdKj z9F#gS6g~Njy+=M_u%hGy@wq5)zJdHvSh}gFTpn0^U_Jp0zNUP>uMsGhH*ah8pIu`5 z7I^Gs$W4*U{HM+-7nVCG zsWKoTMVZ^lgAzRWgkbL8=t@Hdzp(X?*2kMlP)IK35d#7}sLDVrR0gzS|MrmdGg0OGn#nLOJM3f=_u z;oyDrGZLbid8|9stCj6X{7ep-Kl&m;j6P!eWYcy*;QRt5YSXrx>+z++HnIe9;)uez zhdY;9BhiEY`m~Ru4en~1YKxD`L1oq9HDZQPRG}@An$Xc& zp)zILOpLxn1_m&EvbRnzx!=iAo;{`!{%Dm$hnOo`4-8;MJ$2FaCs85`vyWDdb|?3U zy&-ubx__-jY-5vcg3WD`Wt{EfzPbKQ&=_0ndZ3R>>*e8c%BBk8I(a@Y(B|9SHhY&iai1I5-G#Fu%*H06U)Y(0La^s4_n~Nr|oPFLu z8>WtbV-v3ZBuVM$R%7tv)uv-=xR>FkwJNFiOC?|5*U-Qv4&Q2Cl^+-ZG98`QdMLeV zCAR&`8SA+LpM$yPkNwCc-LR0-deWohwZny>uG~XzB$?6V1#O4+(7V zjcK!fyt-z0XEAA*>&>Z01rw3%x+jJ^u(w`w-IRojA$+hmOBX#Il_Qd|5})7NeO}@c zot{ZKuXxM1kz5du<|kP#O68Z0@g6tG@>{X~p-1Fnuh$ihms!KTUh!&o<9PzEjmbaq zsS2y?FH(H$Gol$lu)hmCh=*haaq~olwtKoyfFRqv$PQw@#A8)Xv?t?G9((AW{-}>k zIw8WNeD7BC--=@+?7`PDR?6Wq@3nkRyQ9iysAl{s+{V7_wE8|nFr)MYapS7D9CBIN zPnpIXd{r;r`gQ5?;spn`rK8ul`L5MPK8COypQ|PL9|gB?JkZCWL4%g)xtl*ku)emY z<9t?@Rx@OJuTBemhxK;9lLWl_`MXPM6}nuWI-zIo$7$(CXS}pad5N`C8t-2O;)4|_u*@YIK3-{D^A$i(i|PVh*f!5F3a^IdZ~aB`cX4@ zjlJ6LRzQT!iIzu(kq9y#EAw_uX3lH!sUJhSqw>Yk&3w)KZ!U_p7g-ITEhf_y4_|_Q zXb~fre7akaSLw38@WOoF|7$n>*TtBR=!_q8p!-Tl4*V3jdMMcW2$WTBQsRg z0-qFYNh@5?won}r4U8E?$BlI=A|Jq}jgH{kN&};J3>6sh3WoM9aCt&TABQqn|8{V> zi6H#dNk^rS5%P%^^ggitqYd`-90t3f`k8I3-qdCxR%SYDL@}*En~l-O+u0!{Lhqls z2)wUiu~{$dgg3o9%b>oxF7`S(lBZ9|CUqN5UZznWT^8qU;|TDoQ?sR#@CY|w1q(im zYDjYL5wtP7DK*gQxw3t$T&SoDefNp8bw)uW$#L6+!QAM+bXEN7 z_1?e_O5>Lo*d0pBN-z9)xa;BrFI#r`R)>=vHq@yqrz;*^W>pZ6G<+gtF?}K4dTnyz zoe81I-!*SXuVN)@a~w|CrYKM9nH3Cu!MfKsfVHF-@d_fdkd=noTVIzw;iOSCrGq4a z6(eBX4=Jq(ca^PVXn-kv;IUV(v6h;5ja;<%)S4y2-N>H87ts3OH(*5&x2VN@FCN<3?1u{Ek&74jjz@7 zV12AVm6trkD6BBmD0iRCZ&X`uxqzQBXKz0tQOXebfV8@gvsH(^#VxJzW?HkI!Yor4 zqP8KSPypR=)nyVnq1@S`fIh<6X63`KJ$X|hM8fCtjx^>Rs4E$I)?_TRmXgi+rQG)( z{dCV5h1>5CC_&q&K1oDqiMMm>XmP+5H*^<*3CS2yVqX7+sk0)XjZuLFENUWGkw#yv zEtyB942t^_9Kk$DQ`YX1%+NKl2(ht8Oorilm2?h8^h-ti*@}*)ezY{d*((qZ;Snpm zfKF(wjZ3DOu9safn4DR8Wx7=VRCa|y96p}3xk3GyJoI$@OtHyTFNe}%r)U0ST9(QyZ5E~Mh^I4W+)b}WJ9N6y z$->TcNX}M<{sAUvAfoO97g+h30M3mNWx90uUJzRnThZ~zaXX`jZ&v(`Lgy;GwoUbM z18Vu7j0uv3XQXHJqpS{?jGY)kqpg#oV?gWGMbyxum;#<6F6zoaUU*F;vS$3K>?61X zYA2;441;a6U~kePAB<-rQWkgHeq2S2P=QWK^{qbH9M&##n>m8m}e&7k)iv4_n>Zqn38 zgw?wAWEy&nqRP7wbtg9u*JC6*z4J$3f+lN-oSZg2s<=y>Ei*m`w5RzL`dsR;&YcXE z{_Vaja*^{mg?>XsX-CSjJ0#uOP{iutiB^@`kW7=ntDwn~N}s@rxt<${iFo6I-u>*) z0;fhkVwOhN`<@s+ZhO~-D4S1xV;s}~Q$z`NP&Q4N$fYMu#*&I-kT;h*(iH-vW4M~@ zF;JzHOg<4AYmQr)0DyFKA8uQ17Udi$x0%`8m-!-yLgO zk<4H$1vg~hMto9Aldm__kXYmCXq>nFIzxxBz}jbk(^F{ChEK~Xh%`(oPQiK-h!(i$1zM6T4j*k6E7DsWY~% zf64irTjm`Tnr$&w7FIImvcMj2U8^dyOPxGqO}@rN3axk$Q@R+?LQ2T(U-b4&d}Hd7 zXnkEQH1+)ffg+j;3!xzkMX9^oQWg>`C8nkG$-lW%3bS;b6}8tP$>p^4zjvnVO-aE*KPdiph z;Y*@a5@N@|rdLA6;7&=K&Y4kF*lvDlfjl<}=Qf#(t_&-lwzI&HsU+2>xr%i1Ap}A0 zsk01bRph5gmQ`+zDV~Byfp7fVYZtw`v^`!E1u(Mc4ayeR8cZNw9d<`}#vmJi-uiW2 zXOtzaw8bHPOD5+QoU=dn{ZRekCk^JD0;grtn+*F$fiitJVc6f`Eqypc?Y}v;{zjxpReGO0{@->l*ecWFAZtIL=e$vqVV1KrA_6$F1C~6{WD`h_VjN!Wvs}8e!!fx) zsqe(U(X*Nim8#VJqk~IwFmr_4Ms;)^!t-2>yOX+2i(#C8)Y zn?FnFtvK4Zx1F8VO8B~Kq+$|(7A8*|{i4^Bza+U^Fk3t_cb9xD^wSz(T@;J_(qp+* zoLn!ME%FbOIZW_*aS#bU5OGh7xkmfm(Zsa@alsDF71X^$$_3X<)-#8SdViUDy_hyj z@rE^Z@dcQv94G|8D=c?)xlfX#19B11lQhlz7Xqy9@s?Z{i%{-5+32$?CG&%v4l5m0 zKF3wvXgk~v@Ve3Dtcr3rW%7}ED&PdwTHQ??Aq*y^$ovIVZiiQgkcnlYJ zj1w(noj>uXWGRjX7>DbvI01(^g4TEL5DOAoad}-BC~KbT5N8Iv?q$UQTn?m$r82!M zMJvOfrNWzW{njXqdfU7Eg~(DhAMaOg0%45_dG$)+^f?jXK+E*O6Yo{)SGME*T1ssF272OMAgDMOXnBaY zW8snQ1E@9v?tM?9st+uynz`qr@x6+eCF8TI4zc4(;W@Y;8aC=l>Es7U+fG70=~l*o zte4h;0-<+!;!U^&%fj;avKh(wcHMPP!j((gyZ2EOWy`jkx925Uz0y=C6U9wDQ2$uOByE(d z7tlKWI?n3^_Ia-=Dm=l?Q2Vr>H)!Yu`*14H=jTaXDgzmo`i?f&&4H~C$45+SgoAi{RcGzqv$_dB8$qB$UPUc-vkhS8~| z8MAK7oioF+IaVE#f15qMmOBTExCN1E1;S=4PnX+)OQ}@;yfuJ*awlDZBfG(-eUqN$i4e@cFIgBkY{k5}N3Ur~c*UWy@vuhkaTs)=Z;GOD!I z^g?QT(mH-`#ZtU9r;yH-q7yZB5pW{pHw%^XE|5PsBP=pJ+yw2QGG6?->SoJzy_2 zu?kaR%v0K7pYR?S^4j%7VcxSeex|CQ{0bNX-&0P6YEK(;(5;s#4P*8UqFch7#*DpJ z1c!{&>W2=N+#y-)uGpok6pJ5b>RTq%ocq~^c&}n>xoJ$N)h+CO?I)pQqW-hc@dkv> z(&J1l-OGZ*Kodt2)RQy0aIg`O-ric#Qq$TsKHu$2N`kNB10) zS&HtGt(_I_j2Tz_N!k=92!R9|13M4-y|wE`(BM=>K=lRmFB`?v%S7lA8(5N2c6o(yFdJ!(QD=0*f$`4gn zPjhxK`kd6BvI|(EQm`V$T^v=&vASH39(9Hfo4>A-Vp%}{N-j(=RR?>oa!z!>ar*Am{M|ZAKQF6_-G5u&*=^rSyqg<+O-D1&C^~hF$*4(Nph%5XMX~^GGZx{d zahEGb!@exs{v-ykRbC#JxiWZ4Rnh?sJVQ(Fv>l$~EQ9&!3A(dDnTQI6AoLEhd0Tjr zo4~s%pu7GGHRr?f`^ycMx!+p`GxAMKc~k%1TEN6Lxf$c>xOdW8U7SAYT*CE&uvgR7lLPhsDRLZA8OAXZsEeN2v!*kJM z>Czdo_&|Cqa%M|%^cgdEdXXxQHLC)+7-rLJceY6v!RWP~2L9LM^Lx3#FB1tdV-Lwp zqYx_%@XNOz>^jt*>|(#zL6CCIzF5|_)6UH_`K|n8mZS&SUB8}O3O_)yMjpz$-XM8t zbg*E~$<-%A&8pbFTHISDdsqa$IY7%fr2OC%wSj&%s}!{wr}-yhns3HAugbeb+41Mi zULa_UC}jihP7^_9WX<#_ZPHYLRDgY7h?Y8aQ~n0^x$7;-Xzv@Nd68lqx?VlrRok4S zMwS|2CljTJ4qPSjthOuZHAx;4fy~(~V8BITJms>7G3#R)=L(NiPWsN~c;L^Ncd+>| zvaB*sl-KF#y8JZFXlT}L?o+O)dsfn#9DUIXX|6N&LUy}Ouxa;9)190@D5U(PP5Uc| z14d66iLm(2J{aA}-sGl;J=|X`b>E)ThwZbGqItR`VP?O=<1Crb8JVDXnUDdWPF-d> zBs|{52hh~M$V$Cc#SeV5a8C6brY$~%sCT5(vFA#=zMg1|^`%l>q*qlSS zxUu)?_|}%~-`+4%c-BKqa2Ncf_U6X3NXuKofKAUF?k=`#S28EKi&4AfsGJ@Rsj)6v z+cIHoS0HfP3FJg4u4KO6)<#$s%m+S8T75^lX3Q1Iw6173E~5V^5EsFeS`LKF_CXyT zBKmgS=tw+Ms_Z58$FCpVQ2QfrXMSpA%}M?h5>(rD9($~c%H(55V+~Jclaylz{qn3B zd>SnC7(P>}KO^pPMYUSoyG#4i3;7Z zoj2ab<{N3XtXp7+z*kQQZ>ieEMgVtGEkzyhO1;!56fAAFJhIKRGyG~$&T8ji!2xSy z@G_o_c(Fd@&c&~~0R&qjxYnua(aNM*Z5TnQ@PjRJU&9BlZbfL>+p&^xp?TJxk+q9E zYWL%}NN-BL+oe}2&!lo28dmuY#s1_a{C(UJf&U(;*>+OSae+(R55WeR{1fV6d{z_Q zvlmrvzbR3I&PG|nK0VKnjRebJSA6v3t`0DJgxY-b9}j~=wlGFlsAL|8)jDrhBhI35 z)w8LiChtlSWt|mukNsv77Pp3{3oq{2|2c3z28tLLsgr?1)9MtP6psQ$1lp1KOef%p zs7|!|xU-HGM|*YUrGzkhf|T_LefyxxVvF@7VKM@xm*4HeU`QIY!krk?P*}DP)a_k41K2csQzyr=VKCG%y7;;=*^u33MW5 zzmgUFq%d5?0*E=Z0+2%q& zt%R)a$N$z9&)NQMB7U~~%F^skeq@BFPLVMGkV;$iTSFG6rT=NBVxCopon78aNq#lF zcu^@wq7K`A9!q(uC&vrX>YxVyS9#hi0@?6+a}qo;_yKgY=c{Zw z@O`fdnz?TImx2%ksbMaT0CHz-ph@tBf9buoy3qS!3|q$&=(xJHU;|7^&hRfcY%5TW z(yKzRGvt&@g~+sl(G9rkbFx@ULWPiOxuu`0?a{+BuI)=bksKe^XbyI#GyDoK~3A5hr zNr#-*yJxi_EI5P#*7z*swXx^jv%{{js&^8a}N3#)>Y-ax5TX zYcLyMmhb4~o{3N$6s1}&diyL1n2$>Hf|X$OTa|JP$3{|6EP$Y0RRWxd_u|IlXEQDk zE>!V$Dc)SU>8>3I{Ts`Ow@dnF_DylkIJ-EF_7bP3<|x>>9qXBM{{cQ5He z&40jxWeQXVz$Ul4@^%UF#d(^)ajG6J*@7^?+Ws#$iWhhz1By>i>gDb?PdM|hTU{<@ zyHN8N0W!CC@!n~$B#V3*e+r};HkQk--I^ZN=3NGxXOBpkhgO>`WdGg6-2`rdZpOm(B13da+YZp!4H)Kq_@e7>8d@8HT z&_STT^R%~R^YiiRb3AECRM5_~@}vsisw+H22kZ*?5T3{KdW+5 zyQ*B+V~fY9L;q>WKqOMmx_nuvRkf{Uo0GVoDO935AAvrHjrP-=FM)a_`smb8jX!nq z&6zWilONA2X-%3VnMeA)Uhl2z78Y(v&M6tH>t3mkvMa_fLMXEsz1gkY^U}c6)N#pu zFFx4trnB#|Y%e}+WJeUqWkYIl3^cUe5pt(4e-#siIQv_TQC-fNZmG7;C}46sOCd{b zm)11@EW7i*ABowWn!ncb7W(qeX=3wh-(=F}rOeNnm6y+M=b5$_E{n-oA;4dv#!I(o zbu%p%B{cj)bK#;Jtl3H}pKX)3Xescmx8B`1>Lfrdh*dg2MpSdfYL3DT95?Tyr5bW# zQQ=Cu>f7}ro}4fHxxczSl2LsO{yliM0sbqNsQVp{y?zCyAVmQowAeH|cmF>D2sz3v zC3JrmQ{@mQUe>vWHmhKLCSnsIz8Kw^487_+nPI6oQ)%!pwKOya$}|RR1#(A}GdwB6 zwqv|&2l}Vk`)gS3R3EubpjZv4%QQJooJx&8^4+ioGQ#--1_yu`u%*6r?{4CW_3Q5B z;L3QC_muOP?P^%S-=*e6edk1wZCaXOT>MB^bpTPF*~w1u2TYiPb%5hy{W(2BjF3{= z@F;y!#DbToK73RUnbrbHx6{Tc^*mZ6%e!3h9K!3_{^A{~Y&)-$&J?`p2xP&;zU{Z! zjTODRoeqq*E(fFd+C<5&oGw5zsETI`Y1?CcFR6bp?@5_0%fAV%KJVb4=TGJ&3S1K@DgW0`)r*&p`V^A1}K+)ftvZrIdh4kGc;6J8*+Hp08 zHI_LWj#8dHUu#azafs)G?||l>;lr}T(G&4(pXc~?4hKdwG-ck_qg?2D&yJyFXs%Xl z>XMyyP6ehax1rG5yPNOcDX!)BOJ|pQn?wN=o_*}C>H7Qo%Bt8fx`(ofcY^qo z^zKGzGbK;|=bK=vY~t7hJ|*e95pcTX>0fW6PmA@5SsHK4aj%c_u!7+|c_|BagNul@ zXxQ0oSvd``M>2Xrl%rTXEtP&_FOpY&{7mCz$Zf>6Fl(B^>Z@xoQi!R0VUIOLjX=?{ zwYM#uPaB#EXP8Z6N7c{?%`zXGm>`BR%Ztoo1U4b%Ji& ztPMNL*2yc7?G;vRZGZt`$-8eKsVZPt(BJC30deYc8O`K5rJUu#EGG7z*M_BP%Q(|&95*3uXDfulB(@V9+o#){6}`cUeR z^0^Wf3}tHfe7QpFF)DK?2r4nG_K{!DPEu4q|GI}3IoZAT?O?Cpn7=B_GpqPV2ds(< zH?w7t$DUUE(C=Jo(-eV_r)ZWX0<{43az@^0xm(@o}j%tmL6 zcxeE76e{~Y^c_8KghV9D3-nSWi7Si%mnF%q&Z z(?WNi8l{o#0TwaqM5}JWj%P;to!^#%r=!LCH^9d+$>M+CK=xHS_voJhb+7M}tcUA_ zFawh!tLE!GD)Liiw`sR>Ek&tH2@e4lj%VUneJivn=~S>{0Px&&qzs|D%c`S^hVV}p zTUJ2U8L$SWp%T-lIt53$ma4ou-O|AUy(p=G{BA!Ukwt}BE4_Bp=^blb@JlCtDJR{c z=lk))fD*LlOuz_8fsbR6cs=U%pjr(MVcoEv&r{-_mH#?;38at>`ksX}taXy)_JHKQ7FAPOhwSS-H`XGBS$~Y)RConZ zin%|*rk8`R6DwUJKwOMB(hxddI@X@``^?u3td#p@I`vZwvZc9vyEzLbO%pzzasK8i zGTbN)&9gLa=~Jv^33EFLYGiTCw9b(n8;haFosFJ89X)eUYx1l!i}1O5wLs<|Uq?uO7Rz@m=g7(-Q}5J<$ukDvqk-v+7UM z8w33EAb;_#psKEP^ zd9G|jzl8m)^5*jNnrU&Yk+Ppdzqb+eSqp^rM1=jDvNrZs*1QMRjuYiHKl>F6E*b~+ zQm-)<$Wr(a)mEQ&BFKyzH|B3I_DI@culv6mTvN9zR0w>v`H4FreH<_LBh+Pqae{5@ z5g}A*s_vz>&)g=XIb`RNK?aLY`=JcCAY0Q()_<6+T~S;L;LDV_Du5GN1^k_mtI##< z>~1*M;;@Y+GxPm~nwJS9f~}U`s);ZBPj1jDg>N0syykyZV7+)x-C5M@@^UFzZd6*) z26Ad=y)p5nF1VYc(h4NMaJC5`gXB{5OaFc(UBUC}0)G=Xdz4hx=@w1HpajDZtt z@Eh=s3deTe-v_L{=Z>#fkf2?@r-9@O#iv#>TIP(^&c>ggH*&q+P8fRCPhB5}=ih$i zO{2Y@Oc-aGc!js$hDBrFy~8^Vs*%cni2#wy5quOaR*_ICXrMqt7B}J*9zwGFmL5uP zG&?%~?P2=}qhYDyH=TXw8k^Uw@*i(t)^Bcx_OmmW!CjuYKq!|`Wkg{TPE!XU#JQA9-vA@`&US++T`|2xHv@jx|bU8J3B=0=G4A1 zsxOcg|2ajvj;qIOW4avOxjen|t8|vG66`eIS?>$5Ypg{PDya!{hCVVXf3~6y2&bEl z4Y6-)Y0Ur;SAXbhv`Wm!*Kg0f^!p_XHNYfq$w{-DzG_4u30c|j-AK$IO*sK(0Mf(R zn^`7 zr$8~~7uyuV=G{VtO&bsSu6`egAL`rF&~RY#fuvb84hQpiAhoN)4CZN7Y)D!+!(eQ3 zhhEi;BKzEdf`HwGXpgA+yt>cMyhU8zYn}1c8P$1tqKc8Q=I>KYu5Ax`eFptD2T;W1 zQT019=z`$S+Bob>L@VHEH#4;p|pUm7SZM9VtS~b#E_Zu4c z&1RfEtWyuHSYz+}n}_^k{n7h;UsDj5KnOS6o1RiAyz#6BC-`8@w3tFPFYZD{T+e67 zb2$;+dF#bmjr644b#YJg?Ee2PO!5b4uX-mFv2tiVY{FQ?1bsh(%NK9p!{C{|QD52+ z3SCgC3yc#Z#Mxpol!8+4b6jh}vn{t0op`_g@J)segTJV0!rur;O8iL5s}3XUjbGh) zw{k8}k8sPt-q+kL_j&M|I$Wt?LA?z;Lb|OiN$vW;QlGj&i$18$0bdMU ziRuMss-?4bCzBxe-<+N?wZ#`Dr@ntm6F$HGZPI}s zQtnN7Ir2Mafx=Z<+g0|A4%Vd?{bId`wl=fYId&L|F!RV4M5^@1N_7sTxrp9mjYGmWdEOUC>`q_j8lK&-L(gV&c z|3<*0!I|rvvU{OPW}XA=&$LGHdn57*7I8CmEO-RrgU7wR)WGZ ziYfcg`vWQj$&5q^D&~-X&sc*q@}sXQ>d0u@34HOBr*V|RKs!QEavb#CvRy%04|6;d zv7_`BW7`u9#?{wgP|bk3>t*Os?o4#7_IhUyWOhJd&5m>PhZs{hH)6eWL&06~7g9V` z)n>iZk@I;UcM0U4R7@{AR#rhgOcjmmWaoU|!VUJhM;}AG-YNQPFy(yS!wrVq6OQSf zi>*=+Pg1?^bMMiw;I)6^m&_}8qipog>eq(~?T!-ck4Hyr15&uN{U7>vDxDknWj_AC zD!{K#DP&-xf^KA9V<>DHeDT;7{Ly%obEIX}$f<|Dyh(oUUEe{ZUvVeze+w=dUF$lp z#;jBPI8No6;46;{YQj41;c@EEOkd5OSE(BEKL_^sVdO3)Qw{APIdK;yFKD&$I&E;Pe78-14XQJBtX$6{ z*dOb9AAt9M-xt_HFwn3!F@Skff~t#4tH<^;u|^AvJD+4xdXP#9ot)|670vs znd%|ZZeMBc%JE6Wf*#;=3<;)D+D}PC-9jada(+tZIH}OO*sOuR598>X{t$ zJUK*G_*_f!e0h&wg?9Uxw{=2l!p|oKR6SNp?4Hb+KS-;RW%ijvqDYnIS{TeUpc4~G zJduryt2rk!qgSCGFdcA-1-BKr%>Y3pbnzJQC#?z$)aMzX{{xF(ud5zn1VjtUie1bT zcO`vZhW~o;u8kk=Z%|n4Bw2)AOxTOi^N)CMbW<0HIuCv5J5z0RRoY>}hHgCnWvZk{ z$-g5LEk*LWQ+0sLo3BbA@e;X=)Ok~9+3-XF<%Guo1wvXq5HqB6NrQTV_gLVg@)7Ab z-8pW5tNEyG%V?|T)7v`z!ff7oM$VfJQH8ARAGbk07JBGe?sY4*AN~zP^UoJb5;Qb#9!3rLCZBc zbJhbuNBFzDiJ2rJ7Sf8{USS;u*z=o=aE6%^3P574Oh#k84~^~P`2)g}-(_=SA*}`(;w5D5}EcPLi&1UV{tj ztuVFgaogos{FB8srI<}xu1?Ig;`AHkj9p&mN3<>=!|%z*I`YQ-gUr(7H;={s5$K{n zKXUnk{NsDnvCC)UIIa~3-|#yAk1~xh$RZ%$1hP93Qv@;N6Z#Vvr#WCns&}l4MzSV{ z8yQZ%a^H*F38=sSyj>>P;y(6Tpi;kp0SFZ2ySwz*kM)9`=DYyWKfQB(%RCzR{vNgn zTN*b$99%ctAr(IyLp9bAZ;NWfjJE=>BdFjdnj!#2kg^T))iycP}_?U7|Za^`UG1tse-wb0eS_f zI^-x>xI&hZ#4)#ccgXdD)f=Of;(+$0-TJ}rN-_KKnI8k*%zp~7{^nsdR2VHh6^s}7 z=0fxvnf|Ni_tz4%| zihtxf5Ay9jwuD`#vBl1Xn&B(bfW|;SDLmHaLO1mT!B7$T)|PFh(sk(usWt_$~C9MMul=Gw|{IOb$vK9}q=yS(^Ew0MxY_v8~CbK)>}iqqBpCzu>} zii6d>ZgtV~k6h-Fe|wKQVfjVezmoYnF8xk%yqeeTE-n6%n>_Nl_jnVQ|8(X&1@$pq zslzkF;DAf37Ek7y#T`jq-FOYi1)Xdm`PGx^Ypsk>@ppW-7yHrfCq=rO&3&+Kb<51s zH!nOI=Z`PtQ7^x@|E$-6gXg-}PImC|48J`5LFmfS=KPHA2KV`@29ccV2ayEDc?Ey^ ztu-|tSx9KSt;~N3o?1h)M)UX5K?-!?~gS4+KbMKdB^0$cOo& zXqoY0YKxVn`DG-&S~vQz5*<8AnZEY!cH|uDu=&HJDOU)u>%r(m%4r#92{|@GKW`AD zxprb+ki(qiB-`ATNi$-LyAnCyjpQ!Eupa?oX|BgdYD{O%ta*3 zQP^bI2t7qEwM&-5tFS!WW#Vza(-{wA8X1}g2Cp4fI;c#`6%y0bF~x`*z;)RdQ#H(N zuuNi`QNaxilzltr$mjj4Q`~Fw1Zww*G^^vp{SGS~%gnPmL6~7O54nni$k3`zUSbcP zaXp~b>~aSibVD@+F-1o#Sk%a_Jjsu{=398sE znw%-xTDxr{8S43|1E!DEU0EmAbn|dt8|D$MdOkVJhhWg{>l?xmP#lf42AI*g$*Gdc zdx~=`40{I)3ldg%5QaN~zspr`Qb)y&VB&0vznjqMc^(mN%Ilu~FCfY`h(c*Myw8t^ zd+>fr|Gz}=kLP;id`egI{udD4p8SYh(;WBo|4YP0#B)=e7E!gl);Y5Bxwqs#Kk}kS<{~sX$C!HxQ3_Zo*7&Mg*F^e-p3_~I_h^M zpp*;`YM=?E3#cV)2W39U6r?yz7Pe?Nb#znMh?Q8S{mN$br--aXBO-*zLsZ4}%<&$e z_$g_4mj;;^ruXiexpQApI89Lv?LM38IvE1Bf1SVv5 z;fCWf2UL5Ks5(q#S8^_P8&L7~NCY`pbK|tCm@CaE=>4)B4^MuwzPbN(nOb&b;4F1TlN*^BZ7H7Gc+o(SGrm<&^g(@?*R8{#7${&_;_v_UVgS_x- z%6AGgMNEQP%wrlqx?gh5Cwoj*7HZtV7A=G=Zb-lL*81844now`?{Cl}*-Sl>}g@g=2Q=`)S2@KO1O?JnFQD~O~58R zKOP-i@+?HXSP~w)1V8MYvdladD(}j&(&SIn+Hj7NdW+k~*_^eQ&gyi_VpqhdXVFd2 zDd4Mp)iNA{P1$Bg&ms<2Np+T8hS9k@J^lZz{}or_Qt9QFbTm%4?gw9CJ2m>^!{HFA z|Ihm0aV63s({@ZIc_(LZM~Pq7>KfsV@Md_6NR#rzTmcE0BO=7DwzW&RleM&}Rw(dF z7F@-Ci)!Ao*v(&~ygASHh!R8C)DiX&D_3@{#+$?fwe*8o=RHOTc7q*{P`;l{Z$mX`JJUSQXzQCzO9y;6y&s0R)s*Dr$YZ{3C~Kz zlzAAGW92U-KLz1p)Hd{tfUTK0N zE>rD6(dX}^41ou40+KA(buw=-feI9?-BdETL3LEa;xKg{(_weRgXTpDCaBqO;3ztf zu@c02SiiUM;Cv>XY10%9y@6}x!xkoX*re;BFaepLSZrrDSmnb zjOU>F@4;34nPSA#y3l3GdxhtqbR!O(=2yScf0Vz5JzjTJhIUHyQOj}qUlBMzSDy>G zQwkiJs0;>r11hd3s10}rX%K`D)XgfsNTU=>d{s;lQq>{D+C~b2614LS*>w%V|w7jB! zh!9I_bv@Z~la}`n5H1H>P)8LGX(zcH40uaMHmde`O(N&I8l^3_iO%_;DP5eSr>3W+ z`du0c8N=;6`5=oEK7Ct={J?(z#uU(?a*Kl4udFcz&Wo? zujRFd1E_pGUID)bd{tvb?M@KF1yO&h(!}B53Zl3XhM+D(9Z`U7>rX;)0T_)jHU+)B zejB1L(G~}v0yP#Qv<}tZA}I$|Sj{C_p-}XG$Lcx!%%kf91YDIB{ABU(aAz-WxG!&1 zH4-MblRJB;Q29eq(0CfqBatST7K2IeNpFWiqZ1;ZGmg}nK?QMru3r*{E1JhEvmiWP zWYfD9_>soo(U~BELj$XFU=6;j5sL_r#8KlguFLYPx`C%FL8SQI!(7!OY2nC*dxuHL zTc@(;q9fCg`N?M#sMy{gJ?6t%e+w#w>l+15a?-6}%Y+KJ<}Vj&#O5l+vrVUVuXtUf z0e1}O7%=8kv&{?oLm2!@$u^T6Jq--xgXDXgB0&xG%7?fjpp^_IlXu&A|UB$ z45U>!#C5c;5TSHLJ1M=9Apv#NuT+7{vj&tYhftoqgA+L+Fn%dSii~)>D6X}K~RVmU`2)&AQ z0qIqWD1!752q3*fC?W!(C>;WX-VuWIDoB^!d*~qT#(n;rbMD>yp0V%VcZ_7NXT5V} z&L3-L)-k@<`GgunglxHPi>4AY+$HGe`ZYXo1Ibn{$aEd8 z#B<$7;7-C0(GNOkDK;`WUYbJ&_8p9eX|LvE8xT@LeEMK*H(MPp1kA6j2M+|%L-AlJ zMq%YM?ya)&{y~Yq1)R(Kzey;Z(QW-bpnfK{wO`&pB=NTZVL9(WqtqE->+gZ5XUtpS z<-CK9e+wv=^L}fTJG;I0_khZoz*cuT?@;3(g43&=>)YrH?0l)<9j!^WSK^V?M8a#+ z{|GL0uzNi3NIXqX4X}GWxJ&U(!7aXqb58 zQ974Qn8>|gU$;5ZIryx4w-D}3)LFHmW6qJ>toPPCAY|dej?NEpFUagj6EJ@YcnR{n zC4t8nlrzhM@yB$AS)Y&-wWQrl^BXn=HY^qUPw$ON+n;WvSE18p=`Nq^HLdT&Ok=&u zHk;zA+fC;>Z>sA+OVM$ZsYr}vAP1M2{_jcXI3TXu!letFn7bIt@C<5pxO4pv9{;04 z<2ee%^p`q0)pguUyW+G{k$aZEClziPd=i0NS^NlAIR2*B(1$sGycRByWU~ zbN>o%4%PlG^mN;^4!~eh;UCR&Y7S6d)VxFcw%exn?99OGSv$C zvmkDJjGAhJkWT(-D4A-9Fiv{e9{&R_%+Az(Df_*|j(2~wJ2U?8u%B{bbinJo!{r8; zA!WR0O5|C6zR;wFaXDxA;1IrYo(=Av!#sXd)6Fq>OQ(yjeo+16Skq+V8G; z8=9k8MastiCqXoQ+ou+Hoh0`!gy>q?b&@>M%57oit)k2YwAASxxI-Bqj4x z{BkmAqilw+5LtAk_<>KpbM!>zA1_lF+J=!%~$Bt6?bY}kJ;CT zBZ@s1J%?FV3K+oul12U;j!5@d#2aSe$YlUaA9fJrE2upack;98r-Lnz3AU$#k9tb1 zqr{yz6D|HlvJuzXNHpmG?vXtrSdr>}*i&F_BEG+#sM}xfu{mdAwJ8^K4tCH7HX z&YS;#Qsc({aGiu{18BdN{^ZkMyD$913T^@H)U74!9SAyVo$uzA?eXg(Yi~sB@!Kk= zAdR?zlFIjZ)2aW#KPBgetKzV!veExJp`Y9sGig5(sT?h7h}v?sEw+IDp2U%QlnR}I z&A;9W&1v|9o9W;`$ovn>_{U?K;#(J2llld#zjJA{6%&W8>mj;}+u5ERx^CRwFU zD5r+K5tT&IUXb=k{%(F`{zg8G&>|6a?%iIWT-k*TA2F(^@JJWY_OdEKNgJl|*ws~v zj}t=@#BCVM`^Vijb&sqE6(KSEi*9?d=Su@3Mo|py9PLpo8&HS=qOdR1xvyqktcZWq zWz;jTCjDu?f0%}&%?E?eB)R0AvQvY1lG0gRBBF*`T*6GUhYt9?q-O@;2ZRL* z0%t1OBf81?y|q0>8oGU|)hyej+m6Rb(Uru#`jN=tt+v>c2IWt_#OJolH}eMQIpg=GZ7S(FyUhILsetdWNBwU zS`{F#f6Ys0XTyE-eUqouHl&9EIxXuM6{wIWjRdhZ+HKBLjy$yDedR|31kXMXwXKh8 zDdDCyByP$IJ|=h++i=#;coaY6fT!SO@&GMgqgIpvT6|J{|1Df}xvz$6n>qnkARV85 z7EhwdPiFxCrxYNH(PSf=hAf4Y16QQ@?%xs|UK4pF6G3KtxqUZ??<4W+UqC_k2 z?wj++SMic_g6G6nIEt%*WHf;(C~6WknK1s!tbkFlCk)XT_)+)R-v=DQbhLko>SaU%&u(Y_9x6LN5x+wvu{5uuKbdI$=w z`Pgdqx8bI(0^CewuX$2?x?jN(+O?gQ{}QtSs?0_B#zb);rQVIaU5~*uui$0I3hT1Sp5U`i;Hs&irCV{2483+ORZqj%(MkV zDB3%vAH8^BnMk;;Y1GY@daowaoKjg+gYzs__NA}BnawpKF+n}3dxf%_qz@z0ikROF zQ`HYAr6~76mt!JGN#?zzP}02T)vfXtFMsZA6od5IXm%Q2iZ5Sz2fX7w9s@e8EV+$| zoGC}oSdpE*iMB_X9*_8Y#?c>aYkp47zYy-(J+7`} zz*ccN1(f#wa{ROC<{mu+Q}FKkDfMOs-0+OCv=^=bcYP$p*-JMQ<#uvuCcN1)8UkPW z(QCSD^qxR6F9)5v%iH%}rSEx9mEcN#Fbt+TB=^?j+oyXYtrX>$`l#{|U*#{toygSHtqfMsn+nlwcNZO;s)Yw~aDzY2&J_&#UrIj=Im-C9 z+KfREGene2LoIIf=SrT`OP*{-yYY2My91gH0dokU`lT+X<`l9}?tnxm@fwOBtEG|q z@=TfV+Zq_%NWRc0!6*ZGxmJ>-mR3rga9mjyj0;CMW5AV-gZYCe^Ue5vcjiJRvj|OrFZ-r?Agv^cAoG5Vz?iWW4J^?@T?ucJ_Jmi|{ zz-86^r6?Zv^4dvOVUx{TB{c5Zl+A|LoejKd62(2oEvRf#XsPyt<>~u+YTEamxYL~= z+MuVgGPOD>ilALx9CZZY8*QQ$t_&`_bcU>{rP;|m8b$n){`cAoMwKBW+H9)a(MNen ziU!?T&VwrD_mRBsZwFbUDQXdCrHcacmdxIZPg~40z}h~PL_^D(Y6;r?6{&kND%VWG zS_HFFN>X?f0U3|yBT>{RK5{;Iiz@RCMMFgVY7g}|D;CI8R%(5t(^)I{EomCb;X(0% z>mXxt-`i^eT|B$*Vm=>&<6zAkn#+0`!hrGzMwNunf*%&G^7gB)GqTemC@iJ%Ve z_-H)4hCEHIqS=ey=fAIy5jHQhH(0{MjNG16mg}GJ5WF! z(y+UdWl8v zCAJnW3hxHaTB!#sQ(gj7r4#H++f~ZKdJ|uSWpP_XIaci!un4N z6E|$BxC1K539+!1)c`w%2VV!(yTiG2IUDaq5@mw@kz(IP;NMNKuZp07|V&6 z&qY;xbHBV3L+D#e&$$xP(f+kFL4$XKqs44EZ zMO7B}|5%;4qw9)}9a2Dr^(;M$3z}DHQdfZ$7Av6UZ@gHn0d&)ZW@z2^m{T1$96Xx( zOr_6K#SuX3J<)62wcb=W zPVArcXGJ77=R75^yIXzx z`4#bG%)@}lv;}bGP-00Er!Tz=qd;?i4#gWa^%CyqW#U?zcC1!x)b=Ib>*X92gxb5#o_@+YF75Q>G3zUD_ieb}$O1+!1z4=Rw)ZS12-Qn{tez*+ zIBLiZp9^)BJMRoJ6;P=g&Z}A6-Af4zwg#A3e-ip?&Yj;BIFvyyro+y?bv49xG`Xi@LukXG1#>dA!K^G88z~0$?z%Z#tGbSJ)-t+s>#GHHr0ciSXF%IM;SZvp4b===SiNTU7gzSm(gxw!}oT{Psr*%t4)` z`Q~ZJi-D$#5(WhfPJ|`?bWRwHBgLZ#v}-jeW0$fmoj_s1t;d|E_dD&2 z_xx@8ow)9|NZa%n{2XcE77pC(vz$hecNoIqx^sFMQClrWcwh4ASO%7A0`dY=-b37pAs0qlL@dvNpf|Ajppg z#a4+P8M{7d4%_U|pzGzgMYpWNi+70`zDgnBpb|nzD}MDY>KAw6F9nwuG>t+*4&VnZ zx4F^%blr}xzCy;XC7WGqY&kxmlAH&g2W;%Jw(rw0bQ2&ts2CLd3snT<kU4j?rH5)O1Y$89 zyOybFE%!SUP?3>Y`T79Sm}OeAI6>h!>s{Nvd0WpWXXu0e1v+F4mtOb&x4s%g`)R!o z^&YcHk1__#+bUa6#JKCE;uqMtDix%QQL^0SC}xH}ic;XPHSrfKUUIEZC9IAXav<|w zimVS6$_N2uW2jtqb*9@U-GMuc#`VeJ{PD+TBH43zhyZeT-kpUDPajezf`q-ho*lt8 z5#6UZ7?*AqEn0XY)G^p~N6slx+7f5cva=^c&a5ROZ0H1Fx~S}#_~zv1s1tqTtOxhf zbq1YY@XtJJqDD0yn6AUs&~ z>0X2kgrq`w6@>F2(sY_PALI;oJ$1{EI1q8xJ$#D#)<3cOa%*(Vq$iuWZY+jB4BzU# z0qf5EM=eU%07E!_q+sb2hvi> z8t3} zlTX) z61%B4jjZcVg`z%GkUh{uY20V@aAjimuimVx#%%6nv&*EEPx&@++B#{Lq5JzVHeh5 mAAA;HGeKS;3w(s6?Wz!@=OHA(Uu|ot>Ua$&iJmPE=f42Br04_y literal 0 HcmV?d00001 diff --git a/docs/static/fonts/LatoLatin-Black.woff2 b/docs/static/fonts/LatoLatin-Black.woff2 new file mode 100644 index 0000000000000000000000000000000000000000..4127b4d0b9f0977d1a366a5ec6537ded003fcac0 GIT binary patch literal 43456 zcmZ^}bC4)a5InfHZQJ(lTidp6+qP}nwrzX&t!;bneLr2?#a;c=(-ScfU0FF9SrzRj zC&~l>2=Ff|`T-FCEkMl1|DH7ffPmNjd;9+m?0{hcoDgj=eSjDM2rkGpbVLZKz)XOE zDe z1oNbhKhHNzkdkosi%<}stexk{1<61mHkI$!zy}RI`OdL6j&}sjX(pz>|9{~_u2=kd zo0LRmW9I!VOxu6Ek~Yy^h5!`FCy|&OZTK`m(L^yDBVrViqH+PPnzdeT+M6H-MN`ov z&N$^wF>W|cp~xITBOypIri)gz82#3K^m0;(7q@dwZ34~GlLNQsMMrEV|STo#1;cmWO zUO9@yvYQ#dabot+6QWz%>pp}8-!F1;VD$GnLTUQ;wipCR45ol`Dr52#IE z>yRUM+)6r?ZpQth(^X;q45M0%KT=WvJaY-cY!)XC?vc6@q-ztrpvQvTOM8EhiyKmwyQB* z30Bu$?sK_39|+PmtMKy!AEI#6#4_t4>;u1HY%;Ie0SwGOrp6Vc0-I0eZ=Wf9f?Tcy z6Y3n^RT1`wodnoUbi1?#7N6R$pd(2I#R#to^FV(9jU`l~i_Rn5%d;N`%QybB-{PMNYo!NFysitEEoz_AU*X?`1Kzh| zf9X|Vr~Bx7b>!ExIk9G}ulpiM=Jc-aVw}HXW3`1-k*ULfDtug5T z`KiPT>~Kj&;&dFqpdQ6&Apx_Nv94YZPrxqHG_OFAXRHTe5MV$IqCkc&;{W`n_-26g zSJW+~aBZ8*TBuq~M!Hk+sj8R^I|m}!-RY__t^E9Y)pN#)NQIaxKLW~Ch`~^i| z`2Ab@?fa#+SRGy_*L;00F`@)*mFsgWTsd5gR8UYbAIgE`gDuH>M)MZV#5_H1;*Z2A zp8v2NSNhkL&Bv7eT_hbiTjWHtVqec<&N#jat$+b67)T2X-p6R*XZq%97uQ`WibU{W zfiS{cJXn-MxH~dU^aWwYZq)6uYW+zLW6QV1Oc_B&cQ6J9nk3t2z$AE+?Wqg)bL00n zYgJlkHCn-gy(dmBBC-1YO^=(bibSd4Wdmva<BI@qrXExdNtzrmBewIsY_si)gR;? zI=01*N=r+{hU%+KiF0NrtJl@+11jly9b7t*)dTg-M;jIYmmq_nc)T&15SfMp6a?`! z7zq<)?6=VN_K)zAxlF&&WkU)fsKD=I&^!(!<^}({Kk$j=+xet)#u6yg)FJhImphup zXOc1vQ&?1xnHxL5QJICgwOp9q>#a_7 zIERtYLdvNyjJ`}OtX1_iP@3Qm zUm3>7X~o;Ndt)PakgcgPE+H}sZRrY<>&%bEPRvev`o`AE_Owa5pTR67Y^Jg@;{$${ zQlA9;ESbFNYtDLG-~)W}dCNK%sIB>hiktBy#gv8E+XR;Wqm>lKZzrZTk zT`#M#xU2N(Whe1Jq}4N#AbpS75yU$OgAa{jap}>k6c&`#q(g$h%0>AZ>MB-QZGOJE+tA%D zVG*E_WlG3O>LX4(%R$23Aks7Q!%C7e3MtFV$|7pPA?lT-wI|rx+8_4?riBNF#;GT! zE}o-i{;K8XshN`ORk!r@P-t+6Qf0E2d1Y|CE{)s{b%GG`<@Ztxq@|Wg1RR0}*p7UC zMoP;nE-+)~y|%+64)@*TlG}T(+;D?+sn^$nf8$ADsi-&LM>+;NqAc4WDJL--%u^ zSB_k11}j=}-_=-kBoxLK*0eJdWj`5PuG}Q+tU`PC{QyY*L{h{j(jY8S64WIuZh(|` zg{w&b1aCMix>4%b%EZ?yn2i-5xsHffJkKh5Qfrk)V=~9MO!9c_JW8inm($xJD9ZLb zOWl2M`fZ;3c>NjQAwjY!)D0$zzv!L-v6Ybt*Zm3nM9xK^Phco`P(XA{ATvnA2dBVE zi0_9Ak$N{r#_qGTTvU=2K@G_VShY8q#*7=2d$rp&G^-Wtk)iL>)F9U>qs&PKi~n1Y z3?@z>O)>G?_foU#c3L-~B~6?%pDIhFEcxzqTDR>nKHfP)N0fTz43{9prln*<;u(zh zVX*XICWir5f^gGcsfyQ9tfJNx#fAu#0{Kr4Oy0pW$h6x!urhrLnrh6A88F6%?v!_o zj%73?0pN#ETynk!U>73B@Aupey?`Y-0J47IWV1>%_E}or{EU65Vq6k`=5MlL|Fjv;9KHiz?Xgp zq1^XP(0FQ9eRP3VMc@g=;{~x0E&i$L_tBTn!Y}ssZ`(`vZQm!hm}p`Hsf6c5g7uCb zh-+`p!Bgj=jzt?U9w7#CG=d-mVes9@ebW@r*K%8&6X>O-qzY9u;vj@UFryI0LvJ4} zW0?fwYS@L88eQrT24Sc`cu)MGPzxE;tEY;30g+%i%|522i_Wc28R}pf3NFD$V1heI z#%9Uk?KP}X)96E|jJ&fnBcR3UAGtoV3FE!oN^MjHRF7Z?6`%RkIoJGz#2*#F*tCi` z9M+>9+DM>!a_UG&fS(N_9X5i#WW_}AKujS1x^a{+u&f1$g4cHOf4_9v z1I&kGqO+?@o=@3yu8UNB8}-fPrpfGnW1|~d?&I&VlT?tLk{F>c$f#E9QXLy7B5hL! zZd$QgJPmCqjG9>|OeardK4&S62QDJWZE9?eA4ywNT5VeUwvN|fH)va-eV29r?r!#u z>q7nE*rV$}@6ie4>zc;PcC|c*Q^U0#;~yG0E~B%PX3zNvw9VaU<5erqkC9&*E8EYd zYr5O)2K2aa^e7?z_~UOX1&A0OE|5&bMhr(lhTw+SNA%;$<|<0mAP(!8H|0NNx<%e?Po#L`X`3+MumGJpmN6*F*f zS3(efwnjoQFANczFlF3uDD32MB6GAwoNE^hTTYnu3Y+6%(~ics5kHxlZo}zXy-J-r z)7EK++Zt#cB5QXeP>CxCO0$nSy||}w6aD$kJeCZwf3Ig1w7P| znFhr*zTGMx(2X7RC!j9~Z~!7Q#mk9&z1b9l*UDAU{N$Na=&GW5efzIbK~+Rl=oG)Nb?_db%3TdiFqq!_)Zz zGW!S$fRH1&5JNT|1vjZ#n+KvWc^VbhBcpA{VU72d_immzj`x0v-y$O#?R>)9e(?IMA9YEmSlmtAj}xU+Wq1-4I)tEFFFeKxMoL<8%S8 z<2D1cQ6)~-U-i2VEr3Z8pVzY+Zt-9fWoaJBb>lQmRgu0=Rsb)pdM@SaNutnU+-e3l zlW&=+4QKTPCuvwAuDm8Isy3MIJ!BJ>G@RFCTU~-`c|pWH!y(DxogsG$ykpMEsP{II zzTws&w{TgV&gZigwOWTAr{CTnVUmAHCNc0(rZm4vr+vE$0^7_*-H=rIcRk5Uz@sk( zdOtzEM4Pw)E*np=Jub)uJ#JGG#y!adp+Gta)Mch?!D%H5MQ&UF%3S?-*; z&EhE{Dd0YfNXu;JA-$AcPp+pu>JFqHT$fDSSczIRwYIJ=OWC$=tJOJwtjub(9UAup~jtSqf9s3@r^s!HeajEa(ytfG9*$Y$KQ?m*?_ zp~28Zb66X;b$3C&m(oe< zu{9D`sVkpehnty%WSrJ26_8=F5W2tQ=SL@{HB`HAJF$b-XUw|z2^V;Sb^@*fPksJ8 z*AAEQ0I;czk5v^!38uDz{8J4-k}8}NRV{gbay_=VuWGwTd1BiDmes`ie0{Ys=*zic zj?#I{?$7TWJ*1x}VbGzPpMJykuKl#*3u%VmoubU&k05X2)cg7@Y1E+x0(83T z0X(WsKO#v_=dvyUOSxhB(o@8T8F&e?y*s%!rF^yz;HU^djG&Ncfv9XiaCCUS$o@#B z^H`zwh{D4}65=h&A1JS|~i6z2^rOI3#k*%~~Dq%Ibd0;cq3@+Tamld4^f;CHv#03e1?NHieO*x=~!dH(X{4Esdk;UbCgi3v*Kg~dp;qW1a;ZV=rD{Ae56fiCnzijlm_YoMe9YgM`jqSV3IskCn%#iREu$P z-o`Es^M<8ejf|M9mLtb@AO%cFB~#`U4Odi)hDPH?;*L7p1a7e%*hXL=3dt%EA)^ip z5+WT5t`0~dogv+D#yE4BU{pvooN?2it~VUUJqjJ!(HkDxT2Rc>I9x;nTm>Xj1w$ip zHa0eePO>%sZ=924Lowm-GES#VSu0CNQ7t9$z7R$u`##o+0#yM|W)2#qTCt(8>$LkI zmpmd-|K0kA(ixh8gUqaU_H4j(bvLT@pl86IrBSSH$rWS7os9rK@zFsUQGUOYIxqE6 zN_Oul#%Wf zevKmK3`bNdl?qj-&x_%5^ojss;uh1n=5(gLM4??w=c)K#x9t8qa(pRd5M?s4mr9+K9-wM2Jh7fsqMUY&oK)?X zoE(2xuC0D@k{H|mgp0`q3Jww#8Xh1eDlRfLIx=dn|1)9jol{-PuWsz3qV|iA4MF{| z(w;r$4NW;BTSc{Gv{CZfAe=`Myx2n?qKmnRQgFKJao3NJrIfi7t>B#57qI6Q7Z}?% zK};Oym;I&QRUgQFEg3mIK}AVTQB_&odjx~QOu7i9{}~{L8bxSm)fq2-TJ#bI(*_Qn zfPaRh{Eu}wGi?J+Be5Vx)I^R>Sz{P71qMg3DXr$xYSlvRP$$j>xWhW4m{YYGx9xm9 za%c?4dGgXM$7Qy{X-vnSp{ZBJf4{5cSm&w^Jy+;7LBT|or3ZaXFS|qRX;IqXP%Vwa zEw-tV{Yuq?1}L%UU?QcmrZbMv0^^A*x{Mj;bgR$=R_u~C)Ep>Z;Ps>`G%tD&8hB#c z;!xw1qvXQ}cG}9Y%t*08oQ(C*3$wcQ%<7pnYu8?wmYCd<-^g$Er~;-39kc!tsNB#L zrDSWuD5&L8{s|ciU=tflarJw}1V^x(v*iFB)m1CwP%M(tJy_(?JjhITGY!G`4-lHn9~(6j(S#DU`1u7p8dB=;Ha%DGQNLOPXN_OTRU#WGLqh4a3;DTTb1QkJ7sSZQpOM{^4 zDlZj9)I*OVn{64rM%5Wt?4KxpuatLxlPg;SZom2A<~0i#kYpaBh*4$PfTIhN_bLr( zLXx-kUF!+JI}n9r6kb?j{7|obss`#w7fwc1S&AWR8nFb%;Rt6ah(;HdP*s*;m#*?= zVSL3LPwRbfLGd(6p>`7}GKs(6b!!pN7KD0gU68O0ys>c88wS>e7wCo-CQ1+|K^hBy zpiRi1$Do13?@KrM&yciS7MsEThgHN~S@rLg6i|^+f;v&k1n`*dms&Z~7E1?Xl8gXl?5JI zKvy(3=%ypBkO7WmWnY8)ejK;``T02YGrmBfqMa`z{l|I$Y2an4I2?-Vv9vLPj zDJeEFJ~j+y(zxQF?Z|1XZqUn_Hnt*<{3#z#x3Tz$i$g z=9%74PL^QuuYoC=ux{A&2l;;-nM!XkBIPNtb>isq&6AWYn1E0!GkU<#;eQ%Pizg4@ zF%;Fcs!$jtbLl*L|80-*aV&P#4f#ihsf<#zYHiPxBL|L}$sK#kD zm)$fdx{l*UJ7iL+bk>CRSSIuBWS85(ncnR#+B}H6RG(-V?vmwvy^Abn%WgV!Yx~ik zWvjtro2O{A;c&g3ehcN=q0fNBu=Ja|@PSFB|2S)_-s33t!vBvRG+pXr_JD&3BeG-m z4ch0RpstLNr2h;zwv6dNw@qS?VDO*SPOL8f=d)Gc4L_xn(RxbIY}azXaq5BhD1pKp z2AUAR#3ivp0FC}vf<=%^?Cc=v;%EW1lLFjCoT;8xmY{GbZ|qt+%>cdq(j|Fb(u z^Bw+Q(0P7)XQE5`HuR?r-UhUN?4cu|AjuLfOH}X!89<)(W%qb~a9UY^cETeA)q=cCM&wT!WJ;=uM3L}mvMj^j5 z4EAxV#oIj1Vfs16G%1|T6P*}9#4(yH8-A#)2`O?esTf;?okGY z+z(m=GbARC8p{9gBb(cQbORb3YygC%?GARvKxM=JZ~DCXPXBKVn%)oeC~ii~=|eVy z&QbY%0R;&6Ndj|{p+KO8@)>|2VXJjv32kNL(+Z62_g-`; zex#L_?=?k&+yMKOj+!;dRxCX`6CLtD5z#IyhgVesGuV!?ifXxn0M# z)?-VrAiqO==(2bzSx=NPI>-b-73!c*qVV3vP+t>b1ucl((FUu45oarKP$-Zeg6c4p zE3do{1q%!fK38)z5Zu`^A|>v*H5pGyy|JShyM5#k@l~6z4^@vFlttd$Uu0K$qkd0R z(8=kQ?L^_2N6uT1C7s1;H$dCAQ_%KdF4k1SZA|IrstzCEr&GvCVudz6>EXp%Z9_GK z>X*AES~?#}5=4Klh zxJ+~;udLVN?h@7PrW2k8hUxN+#7tA=&tj{6vx>=9wfW~_4d+2p^!0c#i&Fv(Y18Vs zDydfxr)2XGKQhU6V=u4zHL0zltP;=`c~_7}S7Dn~u}<1%jLXQ3pAW>bV{Z@+ZF?~0IFZ5HAn~f1&uJ%bT~U*ys8n?I zF_Lj1$^X$fjQ@wmp*>SgQrW1k8g8i`SKnV|dbDEVgXlznWzzMk<;H*n&nbm5`u*MI;arU*qQ5#e^Yh&JHdHEh{%^1T@>pt)x zh6#Dx#d4A4InKt@?(g0^0*#}K*2VBOtghBAa&jcz$fu3IlJ-kC>D@8d`nP)siV zg$(ce6-;(q%QX>RS;YIu8_Rl7ckG!KNGzGLyU2d(f<6u+q$?j;QpQCREk6pz*zXiP zVfl^tlT6J78yK*VhxpCnAk4pggyQxbct3zrwtQewiOhifFOE8VZrw|!q6cq~YNJ>9 zPEE^N%9VbysBpJ6U9EioAOX=4|6aBl(Tq_Ch?`MxjczxU$V&WKBY+>13wY=iR1;Yn zb|YQO(#ooE#3eiJrJWN^bJnohp!~g0iM6fibTLP~E8QT^!~M|0+L)z0KRP=Qd^C*3cKWx~>gEzpb}1VCM#3D7^;J|=s${ArYwx5=pk^ZhsGXCL|qI6t;S>jg!ZkBuxevW+5(R zkYsib%2292pM0SwktY#r;;9o@M~q}rA<36nH^!2gg}_F&1}l(No0aXO;1f7%N%F{d zt9H9~S6+~(prgSubiAsYU->ld8Xc31NwvC)W|$Zl7>NOd|CYhy(d;H@sOK}fTp1-M z2IC+LG*~5kYwUk6w3dXImlw}RO;79r>}e&doLP)NYOOJI_Qc+Gg41wF;UnANs?%aP zyk&O63;`kj#uYLgsijj87r~5$pe&Fj{*FI3NSORUg{e*z*I~1Lr}Y>S6tDvY>B9KD zlL;&As#Pn6jr;~5ek3fXJs&StjX0obI8x0~*7Yh8w+n%TJi|wfF$oh{www%N^uV%28bzo)TgZ?Oyq2yLGwpAr^0v#wXNTbtd9ec`#m5X%p*#U>hl zmd@S8TlF`=gSBZlS^VcWb&o7>WPWjxE3LViP=9gpW}Ot-;)I~ty*J#jGCU_%RO(aT zX30=9KGkKay1(8@?#666$E(u(Wx`j7VYKyzCN~z;K?+$1l~&Q0qP={6vO1ImDMS3o zB4K7vagvFjXsVYvDYVwwJQi6S%*uGt5z0|72&A_a9V_uK3ie{<6JgFd#XFble)cv^|JVsLm)r^WL&5}b$6s~Gu zX$ds@_E+jHPAO>^NjUg7`V3+8padye+K zHXSt{BU6IUw1&|6v|h*9rTyq8npu`@mC`3$-t~{iDGWcKGH=E*IA!a^LH&J2=n1#0 zP4LKlkQYOAzVsfLOgpqBT9GMUg>gLdcWJs_2u4*THJd9(v6&IO+}U2G+dqR#p~KB_ zz|$RzpTufolk**?^fyayk#9tHP6c#lv3*7&u8i2?8xJdo@U)Vc8QmH!2Vn>HhDB*Q zz3T%#Da>cRAAo;%BsLp)*I_eaH%j|;R>;C*8TSN!UJ3|r(*JQkMY6M7N&X^#UbXpa8y8s7a8|jUC$+o$sj}V8JYlS zH=8E8mic`^cH z!ur#c1OF^QM9fD>Y+afhVW5adD>7A(23f7Cvuww2d+0wNg$!wtq{SWGr+E7+_UDDH z*--crzC%OM{u_aN?ki{xA~X=!ghy*Ce;YLyjtJAM4lYZQPKGO#Ua23C$5rzyTt%vk zRfzg@W3iByYgh?}^H5k1Fk_QL?rf_Ej6&VppkjimB3ONS_eqR#h=ze98(`95t*P&HbCS?{xwViPRXlUL8@BvuR=7GanEFI>5go% zBwmayJM;^d)nV9SE7~MlZ#`HSwdB^GtJ==?uoR1rE#wS!{z^FsPD++_aLOFTw4GgS zPwtv#a+46OGR3(+#d3cOT2Fn|l4Sa-DzET3CbFhsq}LF+kf2&vJvs=08vvN9bm>|( z^=V`~8>ngX?MMg1BC{p|7b!BrG(UKYX-jICy9AF42F9(Nix`?gVqG&)SL>W<=iV{f z3BqH%aF8Qp${w+Jh<$YK_PXbi;+GM1hkW=7erwZ{w&hKGo4MO{Gjcvi@|!VLQo7E! zP;x~IbD9j<@hSZ&0K1B3Q4$+`C#=!k2=ZdSU?G%OQO!HvaELHFq197=J`^8OyYb_R zsPlzkyN+6qyYRg?w!7^dk#Ex;zTMYB{v@Rk>69_~GR=E@T|QEOFg;+jx#o??+Q8G6 zaWL+kZ&3_rt8`5P2=PZ?`(DZvHKdSnCOl;Pw)d>Odrp%|>*mqYe(5;^5A027LL{Z1 zJ0^N`43Ynofu-jAol;M(lLBeFGyW;kZ2SXau-Q)Bz(}z*!vHBz-K}yIDjikDqI3@0 z<40-w$Vvs9#@=M+dH{}^QI)gYQT(utCqN|=NOw!@p2Y9t3iA`h9-X-YYE|+X`)tl` zaDS=Tm{vgdWSY#a>wYX}e+t|t{n#1QR6KoFrU;#Zs1{~W9Bl+%le3dHU_cn!aG~-( zKBOd1h)Gavu3Mbr^R{Pr56U~p-!4j$lYM-@BI97Gx4`RuIa7UR3QlnXhdQoLYq^W7 zbxl60f$47W@_T#;OCul%@H#F$t}r0f!p4>3S$+4O@X*rUYP7z+N5E2^8m`;D%zy)Nui4=StJ&_Si(M@FXG4TuOL3MQ1|FKmkd8BP+O6Qu)b(4omVXSWp6_ z6oDN;ZfTU1;=n2r$qwqnpVcFD{)4$!9O;*RclHbkgW znp2nev*ThDFgovmQKjeS|a`hh@MM@rEq*8&|R;hsY?+~TpPzgs? zAanb$*vQ*1h?q(l2Ao?-k*~O~xMwuF6ABOrh{YQ&pomM}kZ1&oc`6lGi#!pt8lnXx zRKY{~_6n~Dh56L&z*Ob@L=VSzkQ>|D2W9}dX(btUyI&D4!AN#Nc0E`F50fj@#^O{cp z2EuE(6*UfsqfJAC#-!`nWXWSMoeb>V-2eg?R1{o>1tnP6noWJr0tgh_CP;f!LTLDK zOd#+1;eOX~>UT$n^6rTvsOJgI3;C1i9S6qHy(fgfZ9_;Q9>=_Yw$w|&8SZ%}PovyU zL{0eJ2QRR{2LUHg$3$P@m{Pf7*+UDa9-YN<-TqkHbv2rp_jx1*a3er=gs9|r zDkWuy8KgeJq~*Doyc!wvSYqFxOqVcQ0@)!=mNsQ`mAr!lcCfClXI-hv+0aonX%Hpc zDruN?MB7CgFLCmhc#t-19357H2CPsROV}b1!GwumGAXz`5qKekXBbN(o}s=nFT6pQ ztSSRjUAR8d7DJidymG6yBJcub7qh2o zyi{kT^FWfzi^AyCHG`{fYP&=ymF}w1)gi5B3Vy352;x@!VC93u9D8}xeq{mSUfq3c zh#Pe`;!bu_)@i`~r4+eUW(JR=-pdX&V0NW1$BzoXAix1CR(J;^GC!&RR<0sXnfY$@ zQ|}L-4mg}m^4qTg=ZSSw8eafCKx@(+a(_Hi1vVE;Ev%{z1n6s=5*RLkq4!1{`)#6) zNtjTs$B6=nyX` z$Gu~@?WWtrBD$_?g|2LWnJ-LVvpNM&SQGe`+=L=02+8V=E5W^@8^Q@?J`J zGQmsC$VfuL(sdoAH#-k2{MEFO@i{lZZ(35FeCQ8GBtM*=cPv3R&p1}qq>^Cj6(*HL zRWnQv30k5%d`YiQL#(&i0Q{^z=xho|a&E)4T~E3*KJwO`IW>CZV4{nSu*y=vY2l|; zp3Yy%t<9|szyj%+Ps^uYgJ4yRQ9LY*5J@YnvWNeD`^}@K z25?WM_^ce7*ghh5AwQ`avFy;K{c3*cvQ}g&H=gtKMDU*lHmd;o`6YI}TWs_DVRm~{ z+8Zm-)PM%tPeON$>mBjVyK4BdxcZ1{I3{q%iA^Eze+|nU`2zq>EJX#-+cC8+>v8ug za{SnuN}u@W-JAbPv3H#PeJS@FvQo>252(fH=I8s;x657Fhxc&1Zwx$l653%`h*yZ6 z`;Z;I+qy|@3RgJDnL+)n6Vl7jpU=!l5RD{DomCCXmNfh>5J;*jb#@UZHbS?1n5Op| zrNReT;Ha@sTZpP8aufQ3h6807PNy@W0l?jJzBcg1+~*F6PVn;F9zQ6kT3C%7%LVMes=<$|`~O92Uj0o>$X{^OlGm1QNie0OK zb10f>_}YX~Ax6)|aJy#L$1k6E8ZsoAe8+9Y{U8tl@b}$>@o%tEIv%cWwY9|V8VO0& zRu8o&JJ9l>1z`Wimf2io&XqK+-03;@tF~U)PM>TMQGF7+i021GCkww4>AKQM#IBc$|pCXQes4i}KkNMGYS3+h88TGmL`9H;u(=r{GM zos<4o4f6Nc>Q7stC?TvCRNHb20TwboEZNkbL_mi=bhUI_r=B8Y<4tFpOASxf7GX-< zXhoRn(;!70*a-@Ok9t(eI;Ke^E~(NnbakAFjpF6j3JIhHWthJQ1{&7$qXEkVF*-kE z<}tlp#xGoJYCCtO#?+{TZ$i5BIYP;+Dx6SHolFA3pyzb#qlu8kOaAFNgOoJo4Mk#L zbf&8(dD2*r;F$C!+j^%(Etu&~RS{j{X+rs43`p*)BnB*@0hGmRw4=^3Kmm#H1|*#R>v_dy*zQ_fn}{U=W3^_>l8FKzZr$qRkb8nhOHnn3(czw!il~dC6(S*ChuMq z#@62@4D!OG(#7Y)lh6rd5z8W&hcymx9CF+FCs0?c63yg$3JxeTE7Y3Q88D0s6xVjG zEahPw+O;{8bD4<*BS+(SZvU1j_ z*8MOd2`M4|(qH}ca-6emRo(5oa+a@4skxQXw`@Omjv)6e6C)R}F&8^8ia4^DRiKWq zU3tWh>vLKS5$KSjZ{U|wqCfpEO3U%}hd-Ya?w}Oa;l!{adoIdu4q)m#;2|m|4fW@+ zySyH|eUNi#@mraBX89y1B}E3akY=elagq(-_5UsfzOh`T&+^g9K)z z3dsPXS;43Qtq`ZpuGkzwix^il-Lbtuy(U{8;YpC=v=6a@ewc8nMs*r^p z;d3HGi0D1Ewyd`ZgqOI9>~Pu*J(o5Q`g<&1%29Tm(gvdg~54}^|S zv@f0z2g{MrA19Y!UpjsYrY*jABt@c4`Ou=^3mi13F?MndpS2)&EgN0&*;`yU!oj1B z6Uc%e0JywF4)_dn2B#|+I|r zhzjErpR+n~egsd9>IGlPY}m!^_o2&tjn{*$TmK4fNxiDZsypM6dcWIPOX|7ks}gmw z-Ol02Mb-IdwUP$7FBS|y#tSWBqTh%Jq()68SLUYY-AJd!H*5Aksv z04C*3=Y9e(VP;8*$ry`H%%h~IwTIw*v2F>#P<43~xmI08BvsKK`%rO&w=P zeAp6Cj_eW|sGq_0TXwq6isB~2D!SW(Ri0r)IIys1&obkDhl^#`0`(?tAFbCO95TUM zqiygI&uyK5rDdWe?7o8vuActK7D(a<IYs05r0v=F~%1T!3RO*>%01J4E-5Y+xu`^MOg@XOID?60EY0_qI}&6_;EAMb>)mkfs*V5|rZ5lVDyUMsMyEG0J#xN@A*r)9dPDWZ)cKc@arjE z-GmGog48hs&APSwP$xa3a*toq@g)x{;#V>UQyykthd75->%FVx>*vu@TCHnO;E9)c zM&o>`JaD!!Do3O5^-%e-VYVt&z^Kbx$6ruw{MSwf*VAowu}QdqZiS1t9VBDTi2YcV zrb8@Ejyx5LFR!9CCYS`eG4Gm?|EBQd+R%hIX=QRZ!Vro&Cf|pfT;w@xN|j`M+sXz- z0VCW+nLErhb~Q5zz?%nzjnvk6ni(TQLglEPgAC`IzQE4qr^W`0l4wz6g%p6L#iIaG zF7bjNL>tvwXLL^@EJI;O$akQ(f6+l-*y`f$g2s%vm%5gT1{lQFbs_yJQm~Z$UJ<{5Vf0Ff{3q2g|fA>mlL-9DrVOcij0^Kl%Xuk zOQ36p5@YMD8f(;z=aK!o(92T;V9r`y=)F>pd61MkhsKsDi60~ZybUY$^#9zZrKS&( zg@a8tg8Z1`kO~yaBt8EIf|ZkDtKQ&WvjXJymIlgYRHGe)Cu>ToI&&tKGSg}Nq9Z1p zh6X*P77~HM40TehGs%Rl4YQW6p(gh(w68NN&wbwq}=j0=%HKIu{3KIy;+BLatqgm#<;Wc*f$jW1=<)q7JhAOb+0i`L&|9lTTO* z80j@env}S)rqJfFVn(8PcHrBETLwr?ury(Vk`PbmSt4P6_vzPKsFek9&?fTSu1*3G zN(I<2Xvia~ybK7<~;aKS`W4o6(;cVN;pOl;RLl2?_x}D zLheP2fC&D|13$f&>FZwewMIb0Jji@CwR<#vS|U%)ZUIBA*ySr2Q7A`i^ny~epI!2{ zVzjv%g!X^~`v9U*H(=eLXd4XVs=2hcJ^Q((px&bdS(oc}J=o9N?_)Zh*FhN0=5 z)?A^`L#1pJASNkuZ7HEp4={wN-7 zoEFljbq?wD_jbWhIae4Lj%n*Kmbl^ItKj%l=w-@b@6H~4JE2e7!fFzOz~mII3ZG$J zp%HYaRhl$7;V0_JVzi_}EpQ$RFcV!pv+)hBrd-G5dLvP*>~xas$o8#~P2^2Jsl-BA zsIZFSz77`B4zMTZ#HC?$Z(lV zr-XS_zORHD_x(Epp@||M;!zoL98W3%9RDC#(z)gZ$_qm&+|v4@8_=*YALnYz?vy}u zLd>eVbWL(8aYeWqj3CN^Z^>4~yA$8m^iLQV+@$g+xh9Ke)&ky8Nj10>+A?5kk$qF2 z7oC`UMvSCaJ6ug7baqPkso`-VlKBQ~4*ERYEQ(yTHSN!OuE>h9OXbo_(|28Dkyxm$ zCG88?8|yc;a?>k2t&p4)pe^fez|eRO#l=^Jz%X~y5ZLG`+<;P#EZQfGN+aD}!u>J&OGO)4Y*au0W{nHFQVQtNOWLp!yVhUZQJ%9+qP|c zk8SUKH-EsAdo^tB{T@2dmUIMZCS|!UBpO0dSuQd%Y}$?a z;>V{VADDB*wmEq2>%=@)dAnxUJ;UQwl8Aa)d2$rQN77hALG=|R5Z9eWT%k{|(i6E^ z6KiM@f{h+vq8X7_l!EWhI+aFXG zn1*W)p$#6bpQp0^1=C_t3$SC>vmRU$N7H9tEr2bX>{Js}X5kL`vGm zhZ2xwdhUJnEgYT_c_*l(2!fdr+Hy`+C2axd+&yWOf`yd;Ai5&4>TX!3zq~fi-x^IO&b+q^n<2O}rZ(^uRxRWo?rso5g7_8^3e^3Qtp!6!!3O>0NVL19+oMK1Y2-3%1-vQX!{M0aJ z&A%OQYL;)JBR4o+=L7YkDDPpR(4?Sn#kdB>QA*5l!zc~vaVJtIg|HE)gMgm@Oj~qs zfxeghqXq0vVx);T8dXBV&n7s4t{yz#Ea@8bqw|j$3vxV0d&!7;$35e`hv)i)*I_5h zz&v}#gl#z*DFeZIhbX?E23hdtcu723`)0h4gt)iUntfW9FxZ01@%Pgx!3pnxeD^0$gG3<)1*sv4+vriei7wFGC__DCEOPyj2#6#ve7 z2zq&!*crCaze%Iv z?sI7d6f(hSlJXcu0tsDSxf;m|#LXD-K$sdd zPyrC|>So0$B4QDG2Z#`jqyk4!v7&yaU8rym-zgwM*r{^nloVD79&4dc)EHeVO2y&M z2(WF7OzDP7Up8gSTqfBT;^T{kwy&W7e(dSyB7qk({K-|wl>`o&BQAR*m_G??lyuH; z#u~ga+eSAv_j3me;ROYp>o+U>`n;UxikhW=7F2rNG~EiD0JcwD#&j3xB>qA&g2W<)Z7Nqc&du0MS8=&^h5YWCRj-kTK#}XGF%qO?E8e_qLxfEwp z-f6p&U*+cW$UvR4H_xM*S?^If-nLUI?Xp#n_}x{WRu$T-lUsh8)UIBG8HVn*ovly! z*QV^<=KwS(RU?qkEoSyw6Qna#iM4+w3RsY1y12pC8ooha>KivJ zj3sOoE@iBC0tVFdm>?OY6RPOxqISA$-8RAmC$1u(uhKZd^!%>Du5cO*kZ2$w8;{J> zE0v4>ep>*IKbM+04E^>b8c5lsVrriTRcL6w+W~W&zb)6~9TGo6AUP@DiAy`qWbFPG z=*^D`4`u66F?&F=K0dTMQ+A)QYcvDgNs@5t6RKXeCJVx)Q$YV98Ssz!QK&#qvDoD} z>EjE1n%)R=97!U>s7GCL;nB3E5*5SV*|f#^M=XG)gjG_EU;QiOCJznWxR4>jV z`mQ;FvU}(Kl4y(iolm!l54Un7d#Y^YQPG#I1X%}|YiEKIuvvJq<)jq0ewYri4Xj1& zkh&*6dYlx01Tt_X*+AHR?F+Sk>h=sFkl;gzFIFy@F@LTcgsQd+UsK*h*v*J3W_uDC0ZgT1(=YMih@ZNK_}5nff(QfJuQ#LzKQvoO_F>|N=G zJa_tJF77lMHYwl0L|{3UR&`r#ZUJ%XiFRN`H0KJq0u`mtMjoKkf-s9O0*iDJhph!? z5+Be5DzD=yowN-Sp@ z4TmfRRgW`(UFHsWIZGOp;5fz4bWtSv&*Qkc(~Yn?sKXf36t*X%JX{V{#x*$#vlt=p zuyDHCNGFnfvA&c|B5)7yu2Ym+UG<&wkSMOe3afqEN>nlNK(9R)Z^@YXRJV8^6fd@> zS4(c~3VVbL>7W(8_^PGs;(eemIR4VNGOJ;1qZ>3oC2NRW*Lv1CL6hpa^O-b|En0e| z1K^s{Gv|{$Io4Bs#+s%CXHk`L_0EG?;OO!1_}_;FU>reclTZm~AgWZ7i)H{3p~)7b z7f+mU%i<5EyXszlB0LGc6S)Sa!hC=!2L{MI19zI_n{KJzwW%D{zl~o%#9MtmOwz4Q zwuZH?_cCt3&PN91)DZA{UtLLAt?ZbDthdB3~p6D-Px9qz1IkW|}+KqY9R=d%f=W{AhDeP9xEU|teyYhmcgyFhEZwY^v7G8vv(_+vc z7Y%et4cWkpx~2&hWdaVFRRsd=KMw}l^tDZ|&iHM&u|4kM$Xxr9XgR7%B~k46k;v%^ zPnQF!HmKo1#p1cpWOXDMHJx~Nw0Vq3g%zxMsQ0Yn&rn>zO3qnBuL%y{Wo;_R=IU$J zk`=SRUgrTLlQi6B@G`yV%YT>rm5L=A&X%Q+I>Yb~vkbTbq@^dlL*Uw6`h!$C^uaQ< zRz0`W21m*BS*ZXIng5jA*D-f{(EION+?kp@fBrSyZX)7I_w>O+-_bo$-TbhE)c<_DOG2WsHcCiwm+VyFT)DPsI_W~G1t zTVJzrE%JFcyItv7P2Y=-eJ?#Xv*(z5cz!wDY>=@WSDEz5s^2~(3%c?$vzf>%T{*oU zw*@JrEvv2?BLo1hVd>E6p3M@fh<&1asTe^292I5eTr{@EZACpDmg2Osj*7(M ztGSjNpuS$+c&=+wzf>o8>y)(U-h-rZT4-|CxI>q-(o?9HBR#3*f8(YmghhH4f8#Nc z&@sRU-ji{EE@=Se-NqrAvsj?H-SgvfIh=#`p;fMJE15tZd3R)>ph1%NX+~lJdtx~? z-GC{IP>$Iw{$bDe4T+ES_ol>ple$}v+7}XRyjRF!1-KM$Mq8;o3rL4{MSml-(lI%sY3ppZir!in$At6{ zHBDrZ(Ts|PVrc^K3rl(Lgh#^xC2{GFm|tVsU}!8qyNlkRcL{VQNlO z5Kt|Cj&GsvKvR?>m*r#=?yFQC)yxvcFV#Q5++5!nmm974?8RVkmiFr2-9kyhH!1ep z8|%#*(ncT5@R5#&ZjRBH>ZZ{Q7x9}nf_UY#CPO5kgcSy|sSk*79VK|+<_GllI=+w1 z+3CqlyX03ydgSZb4oxLze9MNpCy$%Wj`}pg904(SOMtnC?w-lW#lI6jbiWqe`)H&2WKm0YPmD`3m zvbNU;x0P$y%1ly;8qz(Q<9cf9ahAw--+J6J=NE)e&(-&siZ>Z^M>~RrPk5{iS4(VL zd<@BetZ|9;>@DJ7Ik+<_bxLiXGBvc2nAGa`@?@wdp0XMMCR*wMKjAo%FSJ+FINfa> zwIN{Im6#oBn_PTT-7+aG_%ZJE4)+u1$PXUxW6O>sqdliWg)qg|~@ z%Q>zhBDfQdyfct|;@XqJeO+-h`)WnA{m`NI0y6qqHalx(W_kVD`rE8Ydu&~2ug7@B ziLw(Lg^xin>1qcC2Y-jMwrin~)+K>Dq?eZg_o*<(AJEoOXCBZLfz`qlkRfRwV92w8 zJXzPO!LELW7C1SCaonw#`xH7Ept%N^l3k2}6}k^M8QMv7SAIMOmL{5AX#*io{cte4 z3Nr&cF8e4;$v@F#{9fXz57wbR4sP&2_T2@U-<8OZ1s!RJ8K>iB+p$ZL5DdOM8-%I@_$q zTcZ56$#{|ni^$M#zFnM{$p@~&^LBVGO-wE-Q!O?c0RsY^+IDnIcNxld`u1^AxI&d!>&1j(7y3+;Paq z+~D|_@`Qv&Yzc46;V<*hWWe7E$&rd*CF-G zg+5@AABhVP6^o4XsZVCs;fc5B3ybIlVw(O+0$LKzZJ38JB0={kTj&1cwjV;@v)${EaUX z1q?8CdbLcJ1GkctV;x9d^p?1yJx~#aA}zk1G6rsE-%FiZ9YOx#FF5xLTCAkun{p6O_tM!TZ@z^;NylxrnHL2`@9Xqh>+S$ z+r8y+ZF7=}a)G8T>fy8MemF2{73@?3)JLR<_1t(I!d*7@l2~vxH8`)7U}i;g?+8r@ zH4lg{+#fN}cZmqd?1=cwbD||VdY8m$bOC-jRe|wM@<$i5%j|N;Q%=KfqH`qPCX+@J zU(qgOPD@DrGBh-2=wg_Y$@@LR&0A{_sJ{L^L~AuRoSWgG53UBle-^SRQDHkRYZ<(% zXhj3bpoC2yMWmp;j`y`#%(%5wIL1pM&Ln`3xg)q=Fdi^@xisbT*8ZUOenA-bx?19I zVmqCQ9+3#z$C+Y3faf$Q%dlEqg&84GVeVwAWE~POp-w8-W6gYZSBtO8H#s2V6VGnJ zqet=h_;YpMgs~Vg4ez)ds5idt^)^v~aas_Di&A{Cj!h?UHs8|-C$i1+(V|AuS<>VK zT+`XU&&}&RTu|9ml#6ICxI;U5A7A9BTHc>l|eCyd(Yj?4h_y(*Q)v%=1iGY7P0zBg0>xi*IHEB=9 zfVUARdMOPe7hBCvSo_6L)Y>F(@iKhjo#Zd$)(Bzp zhquehmNf@zO6rA@=-q&XVJ^iP!(a{1Zx$i?Su(NWgC>+wOPn!DQ`NYNa+%s>+r^xF z3!5E~@FngA@v#C@h)m}_(~27p?1)N4f{-Bu3?hr3eTXkRx3{y;Mj!@*VqyAQab5J5 ztgBF}!L`yF0$&IvOGcj(^|p`i5C>sAst?_yo%J^d;FY!q?I2`@NleBL0>ru(E` zt%{CgMd6awN5F9P=ogKV-#w`?!+1o`MsqMyPkwo<*Z{a*2(@xr;X$Ua4fx1o1?f2# zD!h+)@qU*p0`jN2eQ;@g&jKw3>UeWzfJ2(SIpnH4F8R)1Hb6ujJABx=Ek~|rCM2^y z^|b?<-pE-M>&EWA85MKt5AfD=ezI@OKwmwKvOT;;6_i;b<}epGmGUE75-M)n1dZg- zS4}TYb8idj~>-UVkrij(@IuxWYR_wq?R99@e<)Fc)K&D!>8__OeK$XdIf#wtz;^|6X$HIY;c46upreUg?%`ZP z&19p+JYMYbo88d#;(g&lQtqJz4YlO>S99)GoQ2_hW~MExK^LWhZUwgrSbku65bFk_ zOxksp*xhS{0L-VX5civ6l$2XY^8hxlVv=M-K7z;6bhW1YGmLiTMU9P>m$4BLUdHsy zn4NNiAM5M@JL>SQLvpO!Hl9a}W`2bnYs|D&^!Y>IakH+2;Wbk zV+_Hrc2AeY`qT<@vrKY#VRiN|p8bWB4>xDexE)YK^Xozlf2YUPH*-u5a<1#aa*Zp< z>5n}$Kf0ah%OWou_RBQmSKAmKv;DinF|M6|HtBxW8Xr$9b`GpK;VK7CV2~m4tu0S* zEHilP0r)ICEe8|Maq3O#?CJz{6gLOlrX?WFd*qXnfes(sE-=?SCF3Bc)J&IYkS<6M zzgKWS$!T#zo^MC7e5XIPU4=u4H5AE3HVe01AZ`7z`a>3#d@_4nT7q&Czs->a2YoDLvrb|qLLp$Myrx(#~Jp_dJx9h z=yOT%_Yrlq@@Rqv-b}R8=k=vNv{2fg(>g<6pjdQdC=<2W8W5yTfA@B{mT_k2$C?Zl zyTI_4A46$cCr#)5G@}6~A9u?Y&9YQU9_p|y^U^#l9H-Y9Zdq!yTB9uUX5a!=Rkh2x zX)-GGc?~1 zcEY{8e3Xi!3rLYTtr1#2%m-gsMMN*12ZEZe7lkaL^%s#bO{X4mT8VFUE959s6l07=fu|3vf!pJ*NK+wq!w*mS( z>Rry1GxXW!3+&Al2FgB?FHwHpWJ6&sJYVgxt6a`BIX@un5f_qLnM)0}zDrwc+oqOq zHt=kw1qVduiNkhyf)eaVE_nX3wZs0Dv%ZnFT?RBjj{{*SeE0s13`3(pKiRRqmra`v z!e}VG9-+y~V3>LtWy0!$Oe=!?0TD6PY^<7*8PEL*`T&&j+7wBh8>MXDJ~Vci^aGdh z3w;d)5U-X;FC(~#q{8e3;sQS9<5Y50B9Ka;;+xxary`>5Qm6epBDGYnd9HD0QF-k7 z*Iz$`T+>nZ{QQd6t;<#Cw9v!CPJ^HL zx0>3@=Z2prlIAR3`Uzg)G%`wGVG(7R$1U{dTv?n#3m02wr5D`ed5(=cAo@iK1Y%;4 zKbtttS?^?ZOE-QI6C5PM1LiPgBZs9k zz;?=x10nZ^VOSNwi3JXBR%4R9uRq72dedjO3p8s*YNh0(Ws*mFxklZix?hPIg%iS> zdgDyG^MYN50{D1m1`A}@Aq^Eov!pv2RkcFm>wC(5%z-s1F8W^o+3np2QfCI&US~}y zBIK=|7*GO*$sD#y|Gc%O>=5M(GH=kc(At!r0e0cHE_yOrkQ^CVR=%~7bN&2Hz)oXF ztS_;s-tfz#b>+_zYpzb2JhTi7ULVt50uUSCkFMTuR2{*)+R7aeHQk#=fb@{MfKp%a zp^j!LDH-a?C#KW^$UOnTGQ@UHtvQ4@$}m#-mJ8{~jje0IpjTSYkO*xKTPIqK53TFU zuDuesEEx=fw1m_@zmPs~X<;~s=% zF?f*W6G>Mqj$qckGOo7-x6%(TxgtGrx#66=$0S+bcK4siPmh*x(<>UxgPZH{U>E7O z)CFoZMtmdj>KRBNRblfGYMBR135As#1Ai0LN7#}Z1wICcNyCJ(i z5AlF^*Ql0Hq1W%$HjeK=DE#^9!J<>yt_diw9(rW88beLzcw`jL^KGfwfIKqU&dO>B zM4ow?tq*S|0*Vq%Z%wxbN|~jQo3J>D5S59UK_cekPElyvV)cw1 zmHDZdFwJ%IpkqX}prf-uZKOTtkc_$FlB|FUPV}g7#7!-K8kv1Wp*-9U#ZqplBSvR6 zQ9|nwgOZEkb@fc^1Jk9&l)X>c4OhqbYxZKhUSFnx`&9%tw9(wn61?FRu-(;gBxf~G z&eog*J#nc^0TDAp+v&VMHnz{QvBqCl2C3mkyuI(eSktV71|cmoaJ&PE)SmCFL*&r< zVrrHBw|HZ2*-^L8aqPW5Djrvd@gXj<-xqcm$six}%kQzz6(rbkT9uvmRWvLZR4h0q z2lg{3IFc|D!xxc`va%>HpuAma-q5IlY7JH<{%VS7^h$(~yv(R|Xm0GxAHWM&i>~pi z1hpJgXDuO)HreE#$dFix2W1U<+?n+TZry!2k5Ol$%FDmTTmU%XrPbl<$}v;7F6Z#k z7)UASg;ngdl~>(%O7+<~^$Ro47U#(H)1xADVUs{Zy2>QrMGJLh_? zAK?tV(#xCH68qrIOMuz~5F*lZg@m8E`S-*EDGgvkTII?%Mbd2~i_#j;WRfVFQBQ3h zOBdOf3R++E#3URq*5$>E9Zm2*9V(-BjD;2OGkd=Hz}=_oGham*ilI3Rxrux=4Ks2> z#y$z>azA?%#*=l~x9UTE1L)-uq5h-Ea0tGb3>mJKXWhe)Z|jpdH;PJZ+-Gz>MnSy4 zN-k|t>48^t=D2KE0`DOvD=AG3);q?vYOi*3N3nLJSo)W_!^Tjp8K(3-x4;601@VX< zJAuQVl^I1#nI56zCwI=lm-8Bv5kX0r9`)7;$TSi+(bf8!cmUbA#hcJOIVAUJI}pSC zK;-u`ABPb5V1MAx`?)WA3;oX|7`dQFhbyl=GhLBi3l>cvE(h78;>05E{BqRS(O%%! z?3P3b*<$YAos^$(QMY%sFmq?=E!X%lS31;J9F z`m?=d2x99yNM8-k2Z4U9x4{w@c9GtG0%VYkZ>_%v!HSjkCpYE zJhDKpS&C1mCtN5d{&pwW@I*hD77+eW(`}D$!4W_jHS)5}>AIaEo)d77r9-g0rLT{N z8Eh5G(!x?MhOMb?6yvyPZ#yTcUF3^=R>3-l1R53yhKf4NV7FJ@iDu-Kx?a9bE(QIU z*fYu-#$C$Ff?Pg@qP}If0;)`*sE0&P&TjjzL%LE+>E7``ehtBXmIl5N_MY}c@8sxw zWUETa8iyk^bfk9lWx63=M5Sz}((q!#uB58dD`Zc2YQ*MvZ2zSJ-U;%W_0zYITAA3h z+J2CoYorN=OAID3^u5KLyFMAb!ci~9@h{_#y*U${8drcz8VL3m^?LaoXzaH0d}1^6^tcj67@(R6!>MCX|!m1N@RIE>$i8 z9|S?=Dp5j4$^H`6?XU0Oihl^sARMV&E0d`RklBfN4Jc{;KBTm?eRBEfvao6=0lqc@ z8-vJW>w1=fQCWhBStS)W@={KR{MLlBU_SD1SF_Zy$k*UPWmfJq(YgN4Ypvy}vFBvV zty+S~S$}VG#TK>FiGtXG6qcQf*!*Rs=qY=OS3DEw;A+Z6P?p4CHcr-L0u%b_IBeR4 z^c_%A+kmhBTJ3`Lx4g1kF%KZg>!vSV3nAzXNGvjO9YTqD8Ct>$3&e(mdhzkjJ~->* zgBJ%SmN2cOI*N%GG>2!B@?LP>PWlPcpa|f3j?U#f{;*{(zGu#8Fepl7{p-Qeom?4n znktBVrW%Vl)rMey%PX}eeo;LrkZ5~|)jNM7yMotL8eOZQPS!cQPM1F{v$8-|50OsJ zWtpFvcrv;W5C6*tB(Ou1W+q}cA$N7Jx^Gm8U1ga>RY5KHEwu5_Ofz@>$22Os^eWGs zD?`R;<^?gRsLhw9+i$7jlVL<>hesz7nkGo=#m~&7BL3f+NKT90_~zE1bWc}m22G8p zP37*Mx#=1v0DqSA%MPvaG_7(sJ+gURU-mp1`Dx70y*NI=(X3)0%_gVGUE5$PZ?|LsM}tf@L!8bQU7b~qFh!U?yK3G+a2GIouwY> znm&ZrM0_=J_ zvEmz`1|!IpQ;G*}>_^cbOGe~D4dWjZ@aCFk7=P0Pvn!f(mV-Szdqyef&`ELOmdnKDN4wtVQ;oSwiMqP$32F|m#WY(;Y8(}n7V9wRe%Ai9 z+0%$0U7NY}1K8s$5o@b*279@0MB<0XeSaSnfUoR%dT|8TK&fGh>}~H{XwSMs8^CQO zOSvdCs_jO&mKkyOc#JrPpMZ%BFaxpfd2K)q2T4f{#x#f^L(IH7W87({1cfHm{}UM7 zkx%r=ENBt&M^hR#;=J@uGAU)} z5XEYG;7Lxd>7$kn3s7jQi%v%^&s4-4}YA?+Lpg|2^i!(#9K(PG+M|%Nsgo z5}Ct^2LcVQ(5;FA2;E3XZM#1yHjH7TExE1tYS@lmp9oR6gG}$q4ZU`nXZbkVmX&ZW z=1Ykzcyp7|E&P^$To+q!1mup*5^{pQoj<0|>!eKdy6bOkoha(kx@;A7sh~R_=|H%4 zL6P61Jf0(+lHjm*S9`3fNx^ISekK-Tof4aO{BOX{gx*^aA8@~RN3K=6KRzAz7k2(h z11~+Hxyyfmv$X~Nuu-p`ge(&4DuVKN!R$H;^%_@G>^erdU1XmDs?|mZ`xA;yU8c=Z z>e4NlDT1@Qt8h`mnrI|l7ugb}kdEY;}#Bkyn*`9B{9VrINUTq$$ zWerr5oRW(79Kz_K+gl1mWX8A46+ezpDs2Jl#SMA56f+!4TiAI*56=YurLDL zJh|FoHtBm~6q|8uGyR%f`DNICl)wv{)~8=T+7?(($=>u?W1ct9z^e=qd(%vp*cvw& zEe~~_iH(aC_Y~qj7lTCQ+)Wc6DrK_!p1EG0Z^jc_v&%Ai@kY+t6Tt4p@h9L_nU5&o z9N>n)B2RQ&iN{+8M|V#&u9cc59<;n5hQnC(N_Q*Znz zK>cGX-wbWj8U&32*4>sWyd~zFqH2*^=ve9TG9#{+@mff{>-dQ=tAJ{M4m}x)Juv$x zosrez1U(^Gc=UzNRhLT9Bdx40 z?TmK=?Hop#yD_fJfCGentM!Zua?;Swli|*?J zO~TrSH;_C5x&Yv;7}w1RRb#bpyQgVEuA;!v+_^3xTfsD!6}jRRBGz=D$E%W#;swVU zG-%9&#XWUKmXwa(cj`uAwB*@Ph6qlQtO)UV)rFHjI6ZvplFB(J-(Cesi!dGju>RtJ z7|>`lcFG&#OssBDtPq|y*(HEE(PmTQf}ow&SksYM{kxUCRUEXnszchHi>TUCbya5cV|F*AoY`XmKrdk2HOTTq+XE1N_scnq zJNwW@?3eo62lyL0(3kig{)$>JcJHg^pE+)&EsuZddFD;wYe_~WSFx)$m#s&R0k!}o zU&CV{p)Xhl>Sn`uzxc1%OicWtXU!~h>Cv?FScoA!&^nj3yx6~QD?~Tc_gV%m5BlAa z5=<(@u9K1j8EP4ZjfZ-y{Z1?<@8yS#<;;?C{2|1u@dSCE7e)Gc8F3E&=M}k!ykKdS zSbL3)^>_n0ZmKM1J3?XQ9BR2XF5+s}5$%*Nrjm~XRnTc*V#Ent%u~_Rk1+CKZXgUe zp>%FJ?R0-ORzE5cSJNFg5mojMp>zbYv4LO5)PVv(y)enp$Fnb%P4v~<0{zUWTp=Yf z&?r_l37W~+yEAGU-c|^FHxSGH>ny3^$m0^>*y9#rIs|kbKe0#4Hy8x2U##2!S5QxF z7FqkN4JY%en_6ya5O-e3J)Ya%U)#Y`Rd5~1L}AvK*I}x$dl*X1VZ!n!vinpe?+tCx zoB>52rsg=*E#O4VN~jkhOl|$cxbRdH<9aW#wc?{*mcq-Yl@)0c*#CQz!{V^Pn3fSY zhcEzKJU0;C!WFk0RH-=>hTItp;AE9Hq&`O*P&=~`1gbvqDyB7s^!J=BI!N)EqHIBV zyp0~g!vX@TGHW2L$%Oa@daXclisF@E?6$Rwfp~AOn_L){!fKSwpM)4D->M@v0Eap< z)MK>j0+$$A*7;`K`}1hxf1y|${nR}MDE#Q%&dz4dl;hr~{bx_=r3GI84k2C8W`$w&8CYB}SnRJT!KDdajtF zUH1NF$JuR#sBwC7)=%S9)KFongG?4c@ZN5^LTF7O-|y&unKzX6lX(QcCWRu8fm%pl zUPL+i29Tdqs<``}hDU8?p0vJZ=xRw9m}Uwfd!c?;+aPw_8o^|XDi8WzDW*OV8~##6 zeDHjac{$yZtNNf4NIK>FsldLo@9P0va^VVloqGO=g-xQd@Q(KtQoN;0{7DGIZ1<`` zB<6XnPMz@0!m}*J4Em@D_au9!{TTC>~0*aRQRGo~QQEfxQL0!ZxvOPPL@M z(cvU(k_ns;v)nuj0fpXvLE2JiaM*>bp_2#G>C^xo={;(VaomO=chbDR=&wV4_t&gR zlE`X?`Nqx47nM;nXS@6Th$K>5p3L%*HUV@k^tMJ%C3@`S(Uap2A$hKiH<=M=cq*a( z^7sLxXx27n=S>wf(8{mVBx1Sg1@fGJ{v?}2e>%1e_|DlQof1epvdoUz#}1XTUV+^q zowBfqyPDq28O=f4uJPZQ!WZDPS)cpnF+g;7N*xE|2P8{a$ZYPp;fjgZ?v6`KbV1RB z+%FNj=QABZ?F|Wo)M0|eJAJ3_W)CH>u_+xM;)hqKA5Ao6H;@(ZEC-aIv@mvBRa+Uo z^e73gC$CvAB==fjWWQ_RD*J@D4P_fUv*YP& z(;qw|zS6Fo)B6fYr;2E)C>-SX9MS5h5<4xG(N2i^Kc`p}kH0pPE5itkby&!D+J!Gm zF4UR_GS4jr9LHR28%%M1t;T7XL9=F!@?*6|O9V5zqx9j`PhlKY;JZ5z$tM)gOWj_Q z!;xeC5KIaL*dcbyYJwFxt!d3@^+xy`JE!=M;rKEw$x&M@1JUaGvMA>8 zl>2H7JQ3-qPDU2D<*hKMS9qtN9cP*Y2&`_84<2h@Jvp=tgu(yiVAYPwz&zN9$-p2# zKwnQu*B$rxG}BDfTU*m?xHX8^3??)s9^6G1O~sGnz2O7?pGok-tt2@EZx$i5EjG=F zR%P=xXXOF9s1uUNb3(wvKSrNpX>=_e#j5rQzu2Gc2*hxBg)jnMTd*=RN%-GJTo^ge z{A2_R2kXPM9APw_d~K=2>l2&dMexWox;^+KLX+_8*P3N`!DKp_#%MH*p9rX92uL{g z9UdPiWY6HtGB3ZHul{2^GT|w|Pp>2Bt555oB>ZbwUVp)m%K(9zC@!x`$7o=ZIe-TI zKlm*|{Z>H&iRAykCyn{@6zCoQ-*xvwan?13Y@fwZvRRU#&nXF?12TqCdk`aIlnmDr zCg?D@L2?qZ_?zYroTIYG=*cR%4M*37@6jNl&L4`19iE8!ddH!ORGN`0XI!^^rZVc4 zmd(u%wz7>am!1b0L^CSeV-bo2Ayw)Z%;G>w5+vy&CXawkWRPEG(;kTAoXR3^vhq1g zV7ctGTK4SYDDS8Y7Cj&}M4xOsTgZP3*tib1ZzSJ;un|Tmn#d@iE}X)OC9WyXY)?Bwwd}7_s-DVpq8;Ae65U&W{@#Fj4SxvjXb2E_< zo+0|cf9U#blqkO3Nd4^HyuT!`rS?MpM-)+xVKIn?ZSK=>TvGc=e<`w=Oxlx6s!!MO zx-c{hFOO;yRKlCgoy)Ld_oTSF*OU3i;n#@xnqv)eZ?bTJ0soI1RpAe43h_T9>V?rU zk!8~*v=Jw%-{{&ZDa#G9y83=WR+hBRY_8(js52&8K9~7?d`f>Rcn8u*IY7)x^>IU> z`|etmh6hq!8}y+Nr6h|vNGF#mWV4OR+{pe$&RaWaqOg>?p26vKacpkd7oS-@4Sf3& zb$Pvcx$WIIbWrf__dFx7A^m?8gC&~$Ji*e|62zorbauhze=&W{1*((7*QE`4ZIvDL zOlGx!y>mrQ9Qnq-tB|!hxH9lZw21dl5yF81+e&r(*WXpU(JE^GBg2#yIFXl)KzY|5 zMj=}OfGwYHbUpky5;RCjII-x^q`-mv!gwMB*t`2zYPDKdv4H&?ssCykA?H`qPP+JQ zLcYytq4A9~v&*=e2_$FJkh_O+1pm{tC4N58m66ha>dE`veox?DACjVBW5T=7`62m< zedA zj8p0UndEX0mX%O$ zEMQEo+jj;&_>iSQ4obp#e}Vt)wS0n=p!S?$jw^i|9FA?L@H_Kpc=ijlKPMBE+nq5z zhB!=#%)wbdl-%k}RL_|dQ(V<~UBDL(o%J}9bBWYNB-zWw@qn=4F zbFR2D4l9>b(_RPCfwU1Vb&&F6Bw2i+rM_7MS zxnF|%`}}uDr3(>1+&f92M$7WdT#et_(sG*^G#Y8D#{a3^pBGopVMmiS6hEa+U7vmq z9&I~ScV+k-ZJyrfjoN|P9t&9cllcq00QDVe!VJDcBA|%$jx{bp1VeCaD>H0Obk_m~ zr~>q>Xk{%hE+sGJSx;GL?OMAo zfB*mlaGR}MwS|yBH8$Cue+d8ssoALkBSB$-p}}nP`ME?Gn*~4C=^nT|x z{zRnX%B2M>tNsxoI{%Q+YnuM?)`8-DvKG5WMwV$}VDCHi`|GX1;;4)78M*a?t{W9BAOu2dbGZURbEn4^9CRB5 zO476S=4GbBLnSA#_v&ZM-_rd{Gt>luHdv~~mA^&RgGPYyoF$p5^98@EK$TLr6aJg5 z0zLGcpU(h;797ujTZsfVAavWUL0wO=#@YVtez>_WnREXHUx%-VjbrosdD`d`#cXi? z8LZpWU)i**d101&0r=Yq*3)_GqWy)2p9vC$f(;C7o;$4~_zlwT=}N zelu4WuE%SDB&zl)dSs@iQ6*+}C68;#s|l=8SW5;v73he53P9*?+)MFa$io`r$*p*w zcxkq^qHfNhOCZz#oEy1op-cf$=S8FelZ2YaC7spYhOAd+nl-JIsNV z&AQruy?OgVs8qpdqf$h055i?mW5MP!z-BcmrTMu>L}EdYL&-fBpBz2}vHM$O_uFFM z((-~t@_(5XeEb1~|KYT_NIo0)cGI?s5kN3dUTiG_)cYuLikHKT{mjo-~8 z=*{irb_}_XiA<<9r<_NKaf@nYWzou7=aJ4one?1_Wz0^gHS2%0V$z|_SyheWCbY78 zD#XmkqkB0OeEsk?h%gebuRd%$7rNl59XPvgyFob2lmC0KXNwt-m9teebL24p5(dUi zWdFa>gH?V*9k=6i`|@9B6DjZ{_ICIZrnl&8}T>&z87;KLeAc-37s= zifBsLaQE;vQJ!nJW7C!moDTS_B#Ie2#Y%Zv+@_{Q!R3m)TXt{0hVT>K!CBL=n{Bp{ z>luu^jZ5>?gWMYg#gp`Qk#qPhx+bsxJfN}i`ZZimRysFc@!1ux@Cm$^A8Hw>2s`ok zIA*Y(SN2d!5eiZjGd35rH60pK+)LJ^lIlGl(2H&??#Elq%4vlEspB7^Ol+K>09g^6 zddyA8l4(uQzIRfG1;!<`1+&)1c@%*d-YF{CE`Wg3RGwZWg0f5d%o^eT5*z_&IaHz< zuAWX1nN57Vy%;b`u6VCq`0qjy+IGYI1`J+l{{+Z}Kq>0ctoV>0Ir0+j!9;@7O!=z# zO^65AVUd@hO}AynGNLGp}Zzhh+< z!T+bNuYjs*Y5zVb-O?S>B~l_ChXw%&=`KOKyTL;@(%lG1NT+~wcXyX`!~Xzs?|awx z&D!hC&WUH{nWy$Xzi|ros#DiKOHzmyKi==x;>8Y0MzenqTu=@!91d>W+7DRiKsdgE zKH$5FcytT~B9SBL0e~Mi1BqI9pXI2C@=lJeOW#Hm2NJXEvMn!MdqXi>|@4;nnF(^~{qV z3V2iNa8<`1Juepap+r7@vaF=ae?Fm)iK(NW&W>%Qjjj0Jc?2=R$@fdd`IpHCVB#IR zAaQLe2iL0_b)w5VwYkK0)PJ@8Jxjy?Oy5D~Mn(ZiKw z%dKsDJ_qUjFCt)VohZbn5v{oDhr|83^Xc8G)7kA=-Tdag+TtcT9N@{Acn~aryuy}r zHP@WCx9-{KYO0svyv&m+rr|QbkK-9Y!#i91E%1~+vWKpa25X^kbWM3$0h6$ig7`?Z z2zDI3JNdv7%I$r!O+qbd2yUROpm&}w*UuSY6i9lRIQ|EqR@X5_!?#7#)k_~c3A>`#Itz?4KnlkkXoQPhAAByWGY~GhP z3f9T}8;f&JZKVXV3(IvP~kPc+DmMj(Ygi+?Qg#(Lf_%wf=5jCJT>jiHpR`Z0MZb;j@7 ziJ8_9X-4_d4-5))fmV?cR1QgvTat>wWMtmwMtnH4Z^^dP&ror9<Fx`ZA?mlnng zCc)G!A;-^8W1t!vTIPCMx&pLGl`=%Ytg!i3r`3-b9k2&M$3BF`HXY}mFlxzeVTVuz z7k;wEu00s8Lnab}%f8f{PSzov#HqHSahr5r&^QNsUMCOIBY{O|7hy&d!9>r@Z`bX3 z*)FOHaTg3qiIDw_i1n3GV41^;5<=p)6nG>s5=g0D-JA^QrHeMD5gJbzeQTSEJvz{EeqL(nuCa6bLxI%Z}R6^)PnPeOk2N z;sn>BL;^)ali%A__Se~a$kzGvrhnI={iSorqy%=KvgGB@QIzy!D5wgYBdT2NXc9U% zSQ6saCJbg{HXK;of{3_Z6%Y7uUJN2QI2}wn{}C|&ZT{FIymXEOF8gOLx1I?fbOe8f z^o~x4{^g3_*Pt-OB$d5~OQ_M{tLY|Wq&q0}BAS3*Ue*{2js^0K4+~4_+e#~SM(+8Pi zS7t(jeOr<0s1e{iysK*=E=;7GN}!WV#%PDI`NWrCSDZG13sr>y}_&LA@XEUEyd_b-3byuSqAiG*q_x9?*dpJn zs?z`-$e!a5M5{O_5#76B*G%#N)MS#}kg<`RJz7Axi`0L99woi_4@`Q$V; zu$Oz-vSuw*vYBG2&GLhzK@TjZhS2rhnD}-U zxY&%sDFBP$)|4>oPvQZQw4vFcA6cCLGEWxPD}VT@YR+~#k6IQrJWn56ID$eG&iv@E zc-h~6-#^XTz_t*eJ%C)*^OyY(F_IpLu`T7=;}mW|0&8AYNAqA9m}i<=twb+)!Kp;k zuAE*+jW@Z?Xgrz*&i2Ni!MF|=F5=h4h1tuVaKo1G36Q_GE7(?h=RpC9-Cx5_DsIo$jyad zM}y~A98$_4!d$<)`Vx0>IiXup)t5QmN$J&MAl)by9b?MaSTtE{7dH(tt$XP7>wO-J zR%nVR@nx3}%!8V&%Y7lx^vfFnkd>eT036kD>A_|vs<$p=)2KkT(ClTu&Aip3Cm4Ud zI$Q#ONAbN6{=nO(Y0~NI^>>=EF0=gU2Zh5Gu5Y-!8}Wa@U2AaX+uT`MF|R(fSxI;M zZiD?sqsCq@wba{4gcVHv$I8ISB|j<+IByMmD-~pR0!%BQs=f@|AzNM}0Z^v!-Z?r3 zEE30ybi@U8K)?AA$E%kkrH@6b#tJj{%DBV6(7g_cV`7_YLTOX)7Np;tMv6}PA>N$# z0|G^rvbN;X2An1s&aTA|76vX40nzK%BCI~O{R4a)I*6Rv4-W~H&0Jab-nnP4DtDrV zA9a?T?a5pH5azU0J=Q z;l&d~G0K#3wa^*KrlQkQcPX5=pF4fU<#Q4l8~v75gWsWO^4>wg!o$I|G^0`@hVqMe zJ-}~*EyhlaZKlGhv(;AvR+^m3$LAUFXq4B>kL zIa^HY@+`UBjc;BgWTuI2*7ey+l|Um880(9~QdH6Q0z=udaZ#~2*@dLuP{H`Xc8Yo~ z6DFX1Q2RV19IE}r$#C}qoW}q<;7=lGL7dMAyfi*RmF2#Q4yET|1R9o_ZU1%35l<&u zNjxYnn@WGCFxnUdBQ=61yHKYAlHp9z405@x6+#DnzOTbJrG$FPpPUqn+}K8Nvdo%% zAB|dnPs8ey@3HVDk}mblD$sb}ZTl_o-o7IHaN2dddR!&SS^}{y5Sk8ZCD6Du3atFA zV5q&Te5mp{@pqzd|H2!h9`)KhcRS`fM`huAPf~E-u+c$L1hXPYYxZ3U&1orKv%Knd z_4yfa?&&oP0Fd;qDR^n;h&0q-^~6kZuN)Rad%Mniyi9Lo_lEq)>sh>Ta?_U7aOrmp z5-zD3!K(v1z+Nxc;JiMwBN3ja&Jsb=wGZD<3?=UD_R z|1if-`^|1U{&PtcF<*18bb5N$1YC$N*f0Rs58eZ-eHCg|T-V%J4w~Zh=wATZz^(WR8kvzJ-E zLw|F>U;2xN*~P7mtw|Ul#RS%vn;kg8OQPvrt5+oUm4(u`c`#Xz>RSR31;HFaZfg#UvQ%^D2`xd(m?9bC#Ggu%7#_8}obe$HnCoIxre=PchIDnMJTYX`-Y0%?o2c z9WXa>m-AJa%~U>b&lOMu9A4}`jKe>}qM-Pc{7|3o$ybPZ9Gv0R)H<9N*WUu_U*n00 zJ_fMD@0kZHzh7>?fJ3J;U@Kkdp}9s+hvf$sQ-L(JKiD z*!Ulvx2;h!r8eEetcVaIcTMndoxtP|lr!vUv3cSSpW zI)hq(e_&`e;w0|xK3Mg0$zw{z6P1IW+H2G)n+@d9%XcZfCTa$O!eGRv>>MsN8y5LQjOX% z3?nr{ST!{^UpaZq8x0XnCg~~G92MK+3#{$fh$1}B*^!~*Q2gZGvNAdiD!ZtLndRc^ zpFCB&icJSqd^=TXAle65VigE~fKZ(5xMG6ZgWSy)5EK9FG@sZgt8FlF=}C2oI;Gyp z*b6MR_^;PIN8hA9JI1_-TZ^Jk>M9ZUj2S5jOSRIQASAaZ8HRsJo8NLZdkEylt%(M; zsxZjHRd%D^JD?gzuNwPlZG_J0RBqxiOgHTO;MsIvwm#sP(ClkGx{bT_JCCw(`46q1 z58l4){4n2#_c)_^5qlUAfc;(XPxe)zEvxXHvqKS@#b5}i=TuNIU$py$hb2y3;A;`#R<=gQ z^FmS}ivRbfcTah?A&JE2mSw; zMLF*4xhJR9QKmnr4XLTks1NH6=m=0TDPpvzr}N)+tM~5`j`chsPdV_3yqsY7#P@!!H zNhq}cr3{V2!nW-*uG+F|SSpE?-SEN(&FonoHrl%%a@j|;!vxZNvazA23>=-a$nZV} zKqi8S*&Y=mPF2=-8ZplEt){% z?dJRr&5#T%UZ1glOdVF=tyj#joxY>cJ%@&7V(ZnL5I;C|phpV&2+FDn-yHbC0z5&m_;0`^ z-hhosAo^2(`Jd!o(bEd@kEE>Rn4DRG#-qZIL2FErF8A&tQQ#&PP^r=T)qSTtaXt1h zHcupadc@8Z`RYR;U`z*7${7xdWC9I!P%hcMI>~m#S=U+221)NvukdQeWha}3_ICojUX+iu z^)h%+IExE@NMW|alVqRqpNUVYIafG>vzs5<#3)gHz1F=y6vG9&BD~QEN|5Pd7o#8AWh_Lj+w9A%Z++GE1n84c;px99I2F&_})BAKqJT zc9x>&_K!PR+zQjHRXV#E%$ylf>FRlA4T|`53u+)AJ8utG1k_%8Z-+8Q15!wNNd**2 zS4Q6t$qN>V#D#**dsdTzUR^G`wRIIOk72{^?9PXQX?}2B3^W6e1zw}+&H?1oOvK$e z4!D0?vZ4{Dl$M3cZjAbmP8Fz)w{9U_Te(G0 z(Sw6?-Tq|mM)#cst+P0;g~uT7WAXQSY2G}KQ|Mtf-Sb(#0C>)fG~$Egh7jI^9pk$; z#i-~V=F5f9{490@D zCv1PNeIHWg+tsXQ_i@-us^mv?GBgLdK<=x_o&K@#It0M;VZshWJBZeylPL){^z;8^ z*esO89S{oh|9ESIUgoN_43j9-gPm{I3%)V8DbB%WGdt&%Lga^cylOR!))#mdU>%VU zNj$Q`d1ez@5?s^^!1M{YcZ=(xIW~PG5SlmeQhQwcX%IiI^r1(#=ce_o7K%L)umOPT z{i~)ZudI1Cyb3>QzjlA@`x|(?4i4z%qeeI(@lOhUO3811jK>U-Rc48z$!VUv6nC~tv#BrD)AXDVK6J{^Pk&G^)?GUG`=XAQ8+Do3?J zk}mg$cksW(N(iwf|4wUI@ZcLw3~sW9LK&d`n2|ed+gh>9i6#S%8!>^qpAJWN9rj+O zc)$1I6^ln2J~DpN6w;8crML8u5;6v<*HvBz5Wlqt9LRuJqYL+A{PadVUwv(gGG`{i zeEyd;olD1?lu{VcsRM>P_(h{=`fd5B9Q+)E3Gqu`fjk^CYC0Kif$3q>|SZ-|@koeD`)@=*Rh z>JEYCpJOrrQe|B0x@w#ap*|aD1VG* zGqY6rsWYbf9~sbb;k#|zsG11AK^NE!0@`#u(O_&1rB|wL9MqKd>9tLTUXgjqrR9W4 zOoK#mL;ZPD)F&!F0p$C47yICMDN*h6Pn(FDh-Y#0hkt76bgX9(<`~k_!9{h!YROK^ zDLb*>@2m(Dc|tMzWoM31n=RDzikwBQzqnH0y*vClYJc3I+U78_e$B`r%EBsXGp8h< z)H-SafD?MD1nhCdzjnQZy#Lru^L;wR8$xyjl94aaiwoBG<9f0wuoU`T$y)j}NXRML zK8G-oOJ!Vv^pCV1s}NJv2eqv3uO<|74(~p?Ou>{yWmaD?wLoHRUD0Kq6f zd`j3=jPe(~_@^Sikc^I>AP>t78bia+PX@Q4b=J+MaF$cOTO={oJfGVy-DgZ!6+mUK z()YH`BG91S##D&$|LUtF0euh{uZx$7aLeyL<3FDz?hEBiH?fsGXA`*P1YcLdnY+dK zaU(r-sWWbQw|GJ7=Z6+TURSfYaNX;IYJRQ)7<^N*SX2Jd{*QzX-Rhh`tMhJWAH_du zlwGt=Bokb3&+1j7%mO3kpr<0G+9nMo_^7MLgI%mP*T4GARQ zD4aXYb>x!Me%dvK(Wi%=E{$c*O#T=zyE)hhftS%kVPygS_k>@Q;EyX2$M3PiF7tx4 zB@=Ej?L*eB)tg6O3-rHbQVq7m4;haAC1G=VQxb+99300EOU~)4JpEZMKG$fE>I3X; zG|ByY@&uH|8l^+?yII%!Y|IL1*1TD>l-04eAE@RMDj->oP5<#5>%5L0O(>+#(~3&t ziN)TepSxN%nh7C)wHeKSoQ9rEP^>GHJe8${Ju%<`H~7J5d>FJShzw_A{B)2YTf> zvJ$s>g_vSc`Q9$ZFzVGWG)i){43E9ZR`WGL>1B$mD>qOc&GE9tzD_qS#8XY)BcB}M z9o%w1ZqK8Li?tv^pCTR%i+4oDMBK+EMR564uWHqe03}dMcL61AT)&;_lz1eL-AgN5 zs>C8;;A^Si#xd4-Hj-r+ts2)k`o*>#K@c)aaWBC2#iz=S0dD;Ry|L6iUQ1P!1X{gD(+vS9xsJ1%n;VjwJ z=zLzfI${>6?Am87laZM_cCiIZ`TU2<#YQS1f0YBoQUSx9{BL7IW?VZjJ_Di>u@Xky zs2Rk9b6h+Gn=2&=P+?l2VkLIF00lx6BZZZU7?_=3n_C%Ny}MW_;f6}YGBlZlE#OVQ z>I!M*xeXSMB^OK)Dn{ZFC`7b*O$22T_Qug3P;l3)gwT4WyXPQLuJim#!1nM9w5(}) zr#aLdfTgYK7p=Ei<_X6k{lg4JMS`NltQP#98@upIc~l&*YnDZV;3IwMf|e#FCTYKB zi`k%g3T00=ZovmqXpEBLWL;|H8Fj~#`3bF`QDgc2(n`y5Uj9DZ*`SvaJf=&X2yDA# zk+gP<4D;_T^5ebI;qy6e$2i}OBHj^yhAe2HVZwo&uur06gf30l+az2Q8C&3gnKldb z@`I6ysdo)PeMggJ*y`nhfl4A99`;(__oG+nm{pKl5Uub00?rV>vMnm_Qbj_n${S|r zIE?mG++%F=1FQ>)&*Y#WB#vsw3k^P^V!<~;exYJJ$IRC75(PCRTTP0NSd4sRrCZ17 zqnOx?I+4!d@0T^H-sG)TxXotBWoOgPMWZvjQke{^&RvVT1ohA;!#2q3{9@qH%v!cq z<2@k1Dn8j!(y!uqtr}4p(=b%PYsv<+z>3Hh`E@d0OBT&;d(s5uh1AvTo^YDNo^`hanan|>((*w3T>=q85;+;0 zGTf>j0d)K7#qfom7i6@ta5hTI(adYp`F0~Y z18+AAoPew>MnhLbs!r?uHp%@l+4Els<3=KP?h_khYS;V|2b3M{+q)GDsj5ya=bVl4 zXA|Kpmf*HLg5WuID}NNBR37{>oatZDC>Vbkw6D}vy~NU=oXCqZ6PK#g*ECvVS?5sF z1+UtvQOR`>xzW*@CI}U{ujsmPi^5%PaSxHI#diZ@lZfUm*lnIZF(mz(^4@h-d`%`_ zAzdP5J2w^lkyqvbu}jNLB`ynQEzrYyQS`8PD_YRjBZlmCuL z+UubsR|&*2so$~+)n*Fh&IS@9-U_!;o}qfdLfE)OYAxLam6#Dq4%@T9E`>83N0v{a zgWtHT)uvZ0&4ui)3_h1dT49(R!dYgi!%4A_B)t32FX?r2_i6541#}t z>@rQwhst%L)o8ZxElzLz*rU6z-lJ?budP6aijx2FO+a;^{Z6p}{5LO1qr_jXDz@yY z)9sZz}7%P&z%soYsio(AFFOwv}|GRw4 z{YLFI!;WCPr7)wCBvBcW#zgVhoLsbNTn;Aov2~tlk%QXw{7$9Jlw8x;cuiQedl!-B z?tR@n%&%@0YnL_(-jCS{=*|)tsNz)^E2^Bqyh3L4am=}eif(l-H1B`Rc>Fa?-Cx0#IuVam<(cv*Jrd& zWL9T}=Uz?7Tx}nVrvx{1e2C>{>yuC)hy6l>JiQ)|%`&z#U7rVQE$?@T`LB-K8ekLQ F{{S|x{Ez?u literal 0 HcmV?d00001 diff --git a/docs/static/fonts/LatoLatin-BlackItalic.woff b/docs/static/fonts/LatoLatin-BlackItalic.woff new file mode 100644 index 0000000000000000000000000000000000000000..142c1c9c48d42989e9829318ea1024fd2e08c370 GIT binary patch literal 72372 zcmb@t1ymi~mM*$+2oAyBgS!O}?g6rKcXxMpcL?qp+#Q0uy9IaMxLp2o`gWi0drrT8 z_q`h5{MKCGoU3-#s6AG#nl;KrUQ7%C1^@uSaO(l6A1$~{_uu*-?tk6HzsoCqB>DmX zjLHCjTJIE?2EMow@EZWYwDh5me+*tLeotVPS7KxV09Z;s#%q3q<=^x6*U91%Sc-wLjv&3<4F5&sxvQ=tHacIJ|HG0P70hPrJ?5#=#K)_L=MB zJYE3+xS9(~n=@N`qmQw;9~!{p!@}<(Um098)-zmOe*o_<7Lhsf0O+@^~E2`-`^2h1TugW++VLZFpL>JO_CBa&@Ystd+DwDp8ujV zyUqdan}Uwd{VU&O2j{`#1h<3nMZ>-JN!v&kH?fq+@ra;g!ttvO9@F>qm@ICm_97k7XN(CuDila>Fl~*Gc%SN0el1|JO`HW8IsAtyN8(FH(#w|!?>2Op!)oZ_T zEhbhwN1!$7aF*IE(XWa&oY?F#sPN5qqQJS48>$cSMmyoh1iF6nE?=lrpEQ3JoJED|Hb zD|E25sjs>k8EA9$Q9As39)<8qQGq;IQhlYT)VHShPSy^5uxj6d&BlCFbk!628ES-@ zow5mshhdk#6B8+R0h=<*o zf@_|o(m?(jq(@PAc0+eLDG-ekixGexyTaD{4}yc8XHXzS5K^xQ>#XiZ$I`A zdDDT?P?$og2XGK0O{sw9#pm-gh#&%bTsyR60qA~82c?#_*t1yGYHRpw!AE42v0(WnP0%7pZoM@9i5jI-%X%YmY^qDOHk~6gjj= zjmqm2kcEQkJJ9H#3%l@rTT~L%;i-dIzP(pqN@9V^>Q$;Fm68~&=F!6ZI)KQoF;~@2UH5ScM=m!OZdM@er2s9pMuiNpOjLW zSQm6*^Q3P^Qbnse*g)J8ds>Hdw_Q)T8PCC5?BKo!^AZxUg+D~LhHCO#?IGBbv&Cx0 zLjc_Sdh~d0d0zrtQFtMtN$>S0Vr&V2KPpVb9PO~Ax*>rH_b|N%Ai$7yOI&hc_rFd* zd8m3T5BS?@3V;m+zzwCsDt41^$nW1!YYGg5l%9CpaAk}tOqw|nE%@QHAZB|In9#?- zCj+p$Bep2Kq>KZ&(I^}WOUF_q)Nt_Pn9YfSjfvl>uvyhDT7j-U+&4?;FaRYlN-s`7KZ z{OSrp2WFEOQ>csMD(zFX0e#`?(qq3{1;Ejza7AuMR`#3Uf>@~lv*`KCYr(e!;P97S zsp5cuSpskLNABUcL|sDShvw;)yM!BpZ0j!GVy(vWjOM%ZW$f0jMm~kn@rT$V=6}W* z0uS+3A_a1AA!t6;kAR5_5_;C^k5Ionu<>hTTK<%|CCCl9yg-;PO+6F|j16|ZQ*|_c zw87(TZJVRQ#OFZ$0=%c(08wrvp&Ue`9_)`~`Lu-D@Woc+!L@}kb4pvx=2vbAk+1mq zWDj(8Jh*fxvtUvq<52&+gs(_bDA)rg+(r(p*ZsLw6ODSEWtmxbyL$qj*ch<1R~ zPQEfKRDX%5Ar?8fUbg*njb~ay&@N5#tB}}>U(E<(#T^(uHBebM%y5AU@q`8;w(}lv z`O0(o8u##K>-l2+@QCs<_O0zcf1NPWmddS~Thl74$fjSO=r^Oo)!^dgAm~9b7@w21 zq-eHA@JYuwe}ah_&Jj#2O^9tzIY7*r#r;YTtoawwJ{$qQ~7B= z>o9+sVYW=IJmoS~!fAr!on5RK%kaM?{6p@4>rT!|Sw%0D8Ycf>k)}zbNE(axr%)WC zqy7+TM__BEXw;0psu{ITzp)Ob20Ld#P5K?_LZ`wR`yKSaZ;4*UYkyN#`<9tZ;{eiE z{hH)XA2r6Z+V}^n77^MfkiE&_ZAleGuv{pRIN6QLJ6LN?cJ_ZB%_Ley7ff(*-=!mV z7E<4=p6gcx`H<}+Cf?U~F5z7G+9}w7g6~ok5GM9kHJ5G^z%;I(_YU2fO!6l2LHLFw zA$q3iSyGtkA9!lA^pY-)cTybwT56d2qgL*d8dXq5!nXxUW=rqv8=7idPeHthbqnsQ z*;!h(&bSlVuK^VaLJPyhCx)?3EJI&$3^NgG<)GE5{ySOi+rmFV9@26DF>Ndq5?HZZ z{{CMeI~i-_jjmw*S&?9FK7OHFlH^u#T$7qnF+Fr|TEM~mpfdx$xh7w(#*wm_yC?H- z%U*dPRTQ|wT+EL@;wQWrTllpvfo#bHr_l}SXM0!{7I6bgDZ!KjuRCFAmW|*fa_#}< zp-es%qXX~o%Nklwi8%IrxHxlUYmkBp%dwf5*0_XdBwGI$Fmi#QT>}X@L3>*AO}|}N z@_Vrl?>%%Eq9|`OV$w?|PWFg9aACUsl$-$DhLfI<14loQP=xORPx~QQOU=e$Z5XJ$ z>dSqq+kjMnL0s9b4QiBDevK!Dh8OQ{zG{T#H;I-#<#y&M_tBUP1RChgKvb956jJhs z$c7B-By$`y)sY%w_KbCk&tyt`jVUmeIQA>}PR^cyyK8CTVz5VA)scT$CxA0Nv1^_^ zN3sYr1QLIo>nm;>CiYg}ogvMBd1J4v9iA;^B8zgd7!KwcV&)Ml+7La#SLZQ1%X8wW z>|*nzDFmshW6g|6M%b8M(bIS42Z@(4znfNI1UcajgT~Gx?R??|j}Yf? zLLY1qL`;w4Tp6Ioqq`>O{FXL@UmmEJgsEBx=l+RZyF+3>zj(p)-MjvseB~nM?n2Z? z7MQj6qLaLUXO@3fGK*~X$!uNMl7?r2jn|catjV>5D49^uyafiI1sETXQozQfBD-x# znjsrl7iN2B@Q?;8fbk*|m>qUWjml5an}2spV|2QHD`RrnADQ`kX>dz{lvgTHJj|T$ zbSUBKVdXmx;t5$A2-JmAd9^c9l6l6heET1~z4+bBtW zdsK3t8HR~^RL(p=y+fkh#FRZm8F|K>xH2mj$jEwaE7o|2Rc`7=KrcRHk4m4Nv-d-t zlF43LJ%*zdwsKAsyj4OF3$Bp?%~FdzCj!zj&X)mgN|eVvQ|R2}))TQMB6bz0rT>r= z?C;^}Wag~ycwr-&=^XAGb2+x}W4|D#1-A>w(~Kz5Fwib)RXRU~F%sL6pn0&)p3Fg) z#uTXS##j`PIh+pR^N>2xDteGKq|k$`rD<-?O()yPZqA3=c~~x@BNrm36;G!dnO6=X zlSfUa1TXCATL*X3g#Z&M{>*xV8F26f%Xa)3T}uEKlF>XkV_ z6?Mc0ji|i&&*}{(7~^K#{;#_Bdq_Jx=Z*&xgs;?I`G98Ccjh}E2?7O7IO^z1EYWhhnYhX3#6m53s@wRYl ze7OjuGLR_bLePrILT6EbAHn{)^`wiF(w6C4ti*3Zw=z18VHmY}p4!4nqUZCOo-t5Y zUw(kh-e2XJ)i*%4pmHBmHP76TI9;(CTm@JEthelXoY+u5S@1+XIiP%ZOA4ZU^)h`I zd)k0>_s!Zeo`S!H_7c>YLS6%Z3ef1W+H$sqX+~dzI){en=G)S?C2A((1rS0a`}u9@ zUBY5R5<W`*EEA1~iZGb3m+;UB>UjIytQjh5r|DXUPBGAS+Iaos}9g zqcBX+@c>@~-Ld!%Ek{=z|Nr0=!q5(bZ=U{;*dl$*a0 zw0;*{C*c?Smy6P?;r;ugl17K%ir?K>D@5p`ytMf1FY-e>S24gO>=q6fFPX0idS{Qs*c3aEnJ!svGuW!6@RRQbH~f4DU`RknXn@eC8JWB7Ugwistvhst#7!2@>{ z2H3!EA%eA&_;NyC1_Pc?E$^0>PwVQIO-$-0CycW*$N71FdO36at3ads5615?em%OO zZ|aCpeciT9;E|9q0SdZo*!i14gC4F+NbIV?QaD}G)Irbv04QB_kMi+uutwj?E#YcJ zPsqlA7{7+qH8_5-G+nHR&z{iE0RbyvR9~+|sIm@!qjPSB;a&blob$C>Ms5t)xjL2O z>D`Cy_=K1x>R~dy0ax$hkMYYeh*oV|%2?<3X+)|QY8|8gU3uy&jd7P0W*;|JhC|Ok z5?LXOB2C62rDTbqAvG{oc4mXYXuudlGwD(pQ+Z%5RA+auqcDDu!V zlWwMUm32I38H^Jj209LLaLJ~B36U6uq1OSgNPJ07ui3KDf>PAkl+7@zeX9nM%5=3O zD#m`fJ&_w(8mv#s>#GtM+EW7Sw>L7y3Y|ZzNB83=CqInlw``og3r{B6c4W2YlQg&s zA5|<$q8);_DlR)htZ$DsY<-QD>2T${w+6%ezAxRpAiSbpW?x*`zzkC9L^+(3m+7y z@KA87OK--ki~b$>%Lnjw7UZh2KH8g0E@Z-ko*iD-449bJpuw}=9p^NH1)qsoxh*rQ z%X)Oz6*yPzh~+tGL?O0NFnU4SDdn!Q;}o=rj^Fx>>aLjd?ID4iNh(_5pamqvfm zQ=0cFez}ENX#Xb{i@qx)c~f}wB6sh8>e{rP7neRptk0V5^nXrjrd;rdCb`>m`pepX zN1im#SvPz{DOlwJ?*RC?b~Tvlr{pVdXv`}ERDpjabY}Z7>=uzoO;Rx=RI-rJIO*R* z6#A-Nv|hr$he-5ybyt!0SBD0BAUEV8sdYb{6)80RpJkqb48(m~z7!e~sppz>qVmW8 z0fiB_h{S4=N-&_3r@+QbLk?5;@1hCWhY@p&#Nv}mAfb|nz{U$h4wLxrq6pcC{(~HP z^#wJb7BwFZm2AS(qAuf^RuEbhZUf>X0I18pQ~+P&OtQf6dnHJ<4aEXW|Lj|ZR%pY& z6jA%-MXt~lUPL|%{DW)xB^a^4Iv1PcGdi$W5A0b{jhOUe6y`K!aTwYyjo=yc?sj!4 zx1dgVnPPO8HM+?NxeeqZG9S(~sWl%z6e~JExqz^9sCFane8*lgYO(#ZK}Z+{QC|># zYUEl?Oolkwspw7?TuV^pr3UX%uBh#=qL*GjXsrzlA&xC|7JmPh^x$ELo8LYP~eceY}igH@*Lv;s<+&4D@zum{bn~gXI1zr9#2i>FQz9 zjrj+uRWre38u#?lss4i~-`$I+t4BvC6ByJnq{VNa60^2P=JKd(gjq4>pQKjJ0**dO@^FJ@BSz8n$5T;rb>R9 za^zV}rg5nMcO(YXQ};-9CX=BJHtT1q#4f_Hub--nFdMFy4pZc&`*%Vv zdelw-pAuG)n_l|AB9($Q(aH?5$_(gH!aQYfBut~;JQ>wQxQLAEA-IUUo;OpHNpsC^|Bn6Vm+Q>FGganK7Th(-=>JAZpwokoGE)WFYy}Knw^%II_*my5 z3T^rbVIll;LXtkPGGg~@!M&)vMS+VS-r`Qg{4Rg(pqHXBbSxsS(qd&H+OjZTQ?;HvCdH1GMX| zg?LqtJ4x2@`OCWFIlb(OqI@x@fpfC_Z2=zr(^g) zx`(Bjkf`1pEHX}S_ij7qW%bT!-{ot>4Xa^JIt6kGR%xSl-XOkNA?65=d0cCV0x3>m zsKP{<<9wvTTd})v5VKU4`_K0J49`bGH}j}dm27F32CR4PF>em};p~^-g-dgnvrWm$ zVl``hc696mLS@#X7c)#!`7GWqqK6v+O_75d13UBwSC}VZ$BUQAGIb_r6#=f&C^m2- z>`yMdMF=s}eFpxnGAJ={f9Q=|rPJ0z#n=8XAFkQu72g?zyT3Cb{O@;AP+|}#_rGg& z|GvgeegBvfD6hO(QhQgjc6l+DCwwR~{AW5-M80WF5UnP6!%&IW(*Hx0azGCoF=7%^9+P*BdeNd`M9oD{g zxohnz7jsoKKtUoEM4$T12BFXzS;Cz4Z?MZM`KQq1Gs={l8zMJ1L}O=RTwbiR&J&9* zK_xeb48s-r57$T}iX~4BMN$D*H$%?!U`2;|9?;J>WM#Ko=kHtSn!p zAun+yV5$N*yU{KwKNfO+P2H{CZl9tRr3AJY=RM$wPnx%|I0GO6tHYN|; zSCK30|LgVZa@QBFM+?<{M##3SCdMmHfw#sd8*h?J16Bc!o4-gSYFyc%ZvNfRr}#Si zO@-9wmGK9|J15}+l8!@YSy35k%Sl-)S{^P+N_nYRSBn0ae)_uXUEWt4-V--Bc#FVzY4sP6$s92hcCu3Dzo2GB z&Td?wWgw<#AcnR~8fLWM-~d(eH4Z4yQ+B~L8PGBO`tj;gwU3ZfKI`|lMF~1IgE;Jn z->T#TEC241PM1-%k=cDhyAb_@ClM*CPls0dwYGT0w|IYhQRRH-*WC;7vq1=p^Oyfj zC6>k^?NFzQeoV*SfBL}fS;or|OGMz34(+7J>)M%=k>0T21)A!l%${%M>( zHuXn4HOr(w^H=r4aBh&hc!md2(xQ-chAfUSGl zr*DZ;@8nmnkf+xnl3fsg2aOL^Ie0_R=|x6YFlK;)>N;(2Mh}_tj{n+g;z_C`FD-(} z_;v+%273r=orJM7GHY8*8ltW;M5+R#@mv^=lc5;g*q1mQzJW-0=6vljQ?3(alh^Gr zx~#N*);d>XOdEkwLOt=w4=={McUqq73C<+TY$kbHomrhOzn?{&RV*zk z_LxKAS#V*V+~;89xiL;T8{_GzxD;5=WMtvF(Z0=q`w7#xA1Up;XAxS2H#EvU6N)kM zJShLB?DffAA*ncr(dXKw-A$VlXLwjbPVCy6_u1O>()!uMckuMUDdB879)~{dFTz1? zFb~JV23d=Z>;J&Y$7DSYiEQ;(5HOiD6q7anL5nBDDgTE*Y?tZA=9_lX;vsvsw#}0K zj(XeEV|{zob?L6@KDyN;)rR*cru2=r(_+9y#r2Qt2w;|!oLC5A>}|W-A9sX~pVJn6 zeC|^Q8Lt)hQ@F9%s&{qvQenqD%BwD_4RW0u5L$qS|6`| z37Wo9FUw0+E!AyfMPCeGaVHixWKLEhs;+Q=gdM5WP9jQJg`&ykDC>VHs%;$!5ipnJj0j@vVi$EZN?hoio3UF-VE0nUVB`%Y7R zxdOKMp2K6zTOu_w&?WYg4sD-w8`{w%3KSW4J*Ok_xRh+qqgX_0M4@O*Y6PZeEheR+ z6rkzF`Enrd`<7rL@Xj9}(mf|XhiaN;;>`zVX8Fb$JJ8pMJ`8zvy?I$2aclEL*E)fJq9v_zx&VIw`XGDH_h(EV+JLCtUlcbEKDl)rDth^wT4=os-Zo5l+eT|s zWJrT1MK-P*siIEN@7gx2Xf6OZ&+AW-4v%!}TY=ui?R;&pC!pB0P44}>h_@fl*nRFk ziT7`x9+S|zcHSf%fHELUM_p}sU9yu|R#j~p!<13ZixHYChPaZHqpH%u%5t((;mW$B zDiu5{?-1oh9Icm}lw%c;hkNAPHyab@pP(Ig-ZrA?N87w_>Dlsf$uZrL6Dk{%bt>*? z>+j}f5j)^4KC^?WH$nDoTEKwf=|}G8W6z{#ld%(S0z*!HvwO~0h=c60^_zuffjASj zE>DI+!-dg3H3!;;J+Fuibkj!|u{S8bj2WW^)s*4U`I9A(x{BHll7+_{H>V>;H?2~F zr-br}osd|OLorqhI?vO{=rFGT0iP~h`jN4de!!45Sm$+mZvGUP)v(|Lcj7!C1CWq!jI!GO}s<%>Fc|l=vfqI`tSf2erW$c7K`Fe$ zn|mFC2egQMWakd+B6f3*m^m^}&Q}-Sj;`9gb}wCNM|+XYQk6UjJ+c@fCtv#}Y`R76 z42N zw^gjvFzF*c^jSW8R`AnDk5Q7f1hUR5QzSGTY(wo{tYkalQxm7|{~6NG-( z0y%xvlHXq^tG1~vgy&NzR*A#v79Zie!;A0%{wc-S>@vjnV5h80teXly0P2euRwTIF z?KPamJQ-I51r(h`eeO(1&x0b$O{^0Tu;jLLiz}A~@Y}AResylxKA^a~Z<$w_y@n9< zeWDd3l}Dc+2R>5qTRKm~Yfo@ER1wuVA8J4R!Qovx*g=doh|RO@09QMisO;-#6!F4 z$uSR8If(oE`HxC*Bu8yj0Yz(611{+xvl4tN0yMJJ=z#_f=w_7Ls-NU}A~zgcJ?&rs zPe9XF!DGm4z4K`+KFx}h(7Lg#{={y<4n0JeQ9CRbsXnsW#Qe+FMX0#?_eLp z`gtrjV;}yZYZcOqmhg>~rsi+@l7;OF?!*1S7_ijAiW^O(CW!h;EvUT*6HDr0%J9KIMy7x}$c zh9|ZB`+y`D?_}Dy)1jN6FKr;{>b#CX39?d>8qEgM9gdWfJ+l_EqG52!xOsfWA<9Qp z?L;kn#-9hHI?R(3^lV+qj2vCEU1hyMR!J2F8mZ(m=i^Yix=yX{e6jolN9+ZybGy}9 zopo6TRb)2nEBYc|~q!npQxhLn2sum>gdsQ69=VQ%Kk@ow|+reV4A@X^O(oXH!N zT*s0Y(aeOy|F zcw|oE@Cv5e(C3>&sPvqY$oyx+AFVW_8Fj~=knvS4f*~gIDA_F@c})5NF%fImAwuCe zlUoKyzG402yg8lKz_9DNd~Dd**P#3snfPIetlnwmtX)%j`P^3y(|7Iy*@Mtz92Ru$ zI<7yvZqcT6*Q74m@yoaAebaLL_Nwx_69(%y+<$5|z!$8RC!d`3(N8wU;=Ujuouri0 zYuY3sPxo>}lRt*$M_3+6^sW#&SS>%}8eCXJcC_JoxUk+?`*rlWMSYiTp8dQ!h_l+% ze|LqJ_q9w;QI?{_T2aeMWmr*3tIA!PeqJ1uv{IIROyx93U)D7TY%1ev_9<@FIWC2n z6P^)7{1H%)>;3~VSYv_#UiE!9^OJDXGNuvs>WosGR{IU_5ue}g_XYQ>Qprcl82KXG^Wl^ z+)gpXzqA3eBap_d9G!9IxavX+6 zHcd+gr9B)&xd4ZG#Din(i6ESbAiRlw?1_1%3=2K3MIjE@boMcL_OWt{R4nsUl|`ni zBt317zINweH!H-O6`cKf&arZvRFzq#sx&vuqO@%tD|8$4atVyEOuGGf1^OQvVJt5e z2=A?SoBG-Ysf!^eolI5iV{HKz5mr%%t1Ebo^DLI-CQC&KtGL*W>}-}1td`}_tNXoO zZcG)8da4C_sug;wN`2MFKx|As?1dSo3T}keC@R&pGJ|Fe>tAHI?~YI4V!@W5XoCCw=HYFE2qK+p2$=WgmrG8M-tywqSSPeI zXI?R>VQ*+lWd({WvCh}~vyy~=HX(Pw;8o?OV=@8K79peWET2|)X;Lp<8F;evhyGK* zRp33MGFW38<1Wh~hh@?%qRiRR^ZDLQCaIX$B$`6`p_g;vVFKCvI%4x>yNbiK-q@nf zsmqp9Dr3Z$M*VYw84`(VawgiN?s&r_-7&l=fWw{ScW=NNN@)Z@m`Y>@%mNOd=* zwN=nIl+*S*wrz~sB}mUD>2(AnUnSB(5Z_2p#zAw;R(WiNl6jEZZfvE0RP*!A0*Rcx zc7E?$&dX6u(=!=lX19*#Gr;ife{}uAZd6 zwVtk#u5P6+hMul&qVLpL8+Lr^7!G#ogIZc@s%mOlDi#w{1+wn6GKbcHA!zkP#2B@1uoHTmA-|CJ^O za_7PdoHgyW6N1Q7z7bfp=+$lKk{bZcIPY>$( zF++aO*4N8YkGAb0ILT6<{lea#@gY9Jv!8mEhB&SqTltF2DEUaI*Heoi7P_mMwH}s?0wt5v*56OD#x5O?`=)HjOIIr9778E0j#H zR7=mtx-CmKtx;_FQAI8Mvq4m2d0Zj-2FGJb2Gf9Rj3k;#sm4MmN5hV2-R*h%=i)f zww}w&36A@MC?D-pOiHubeI8~xR;tzIgN#oaD2*JJ(o50JX4W`O{&;5tEm=ReC|l^) z8_%UU*R`;8_%|ZtL#qzBwF6aFXb*t5c^zDg(-sGyE9f7aKFsfmuHK#5De=zPqhov{ zYjtCO@=Gn%k26nUdBDiEbazauRUVlIFK*Uk7eNpMxdC?FZpk-%ZDr_M?JL1^cM% z6Xtz>(GC<1B=O*9TmK2OgUBv^r5qnW`#iy*a)UR|TOavYgGa=@cXWH#Y`)}Y;pB6~ z zuNemp0WtfcDCG@+z#Z7)tTGFKwA)nkW4QULoib5_`J%A+$PnoCyx{Z^L^c&ZzDlw5 zqJi{ZlX5T1*`1Y-mSun<&~JW=n$Wc z(mLYaT9v9wGBJ-B=;u)La4Mc}ThABq+yzI!@#VlYXd=_fBAW#fPD?q>(3G*LXEvy> zHz;f2$U9M%G|`mCG#)8fspy=27dR_`PGJ9@hMQ+uXj*b-){JDDddI4)mH5kA-C&)m zxEe>Xk^-}=%h}A=*&_Dtf#CCTwv#t{Wz)oHeIq>kcfyp)+0F)!j;$t5+OC8Zy{Kzs zj#@|Kzs)~_Fp@R6lccZj5P`^m&xNSy zf19%^l^-2u#Xcu{lFNOIjt{^Dsn*z~uh1qa!a~cy#v5U+^|h$ONzwMpxf4yX^lff+ z?GM+p-g3Uxd)pZO6pVKcHlxhHKRYk>x=$GX?jPCFjT~gk$7Y{C=|6oml;WQ}V9|8W<~hr`?Rj=pYn&`*rUTW9 zzeGX{@sy0;W5;P?I@OpsH<-1OitlD{W$VtsTH*F)V{x7QTN@f$Mb!<3v}mx zNI0T5#-Vh*4_jrt%s{cVri-bV-xtWttMls)## zecOi!k;3J5EUD-r3)uid%ckh`4czPQv^8sVizRbd8oGgT)FxKP?Xg%Iss%>Tspyi8 z-s_Q#0+)sqsTF1m_rH{6Mg$I0x_%`vCF%z$E9@o?3sGr_pmO$8r^VU66u2+&q8NUYeB%{{ zlOYU~-QY!~bC%13_ij~#yxs9yyM8|N(t z#uV>2Hp7G@yGHY`hNs=cc>~tpQQ1B5mjCn=>1qU%5Yg&89ULxl%Fan*@D4@pvUGA+ zcjFaN{+YZfXFyVF$t@_cox{Z*Yp9O@F$Tr9RXOeae0aA%dWz27gaI6-(mtkXRX!JD z#)7Zln-Ef``2jYM2V7!m)zm~Q)D}u8Se$2OW9rk%|NQe^1yu9Mt7DEvwky$jW~-Y& z*@OMGsrzol4cpxv6E`Z`+@8uo)p>r{_xMRmjM3Y3_Q?aD%Ec z-Cb;nT`d1Og{%P}0Q4He@tp+7KxdbCmb$C}+)X05`MWtBuuqvndKf+$<^zsytTDYy z3I}Y+F+CKPL5Su-eCYeMzAf5)CfDu^=Tk588YVDIk$WY>&# zFBs1sz`M&el_EpRS&gEUF&UP$cp2FXxT6BjZ=cNOrpg_KP*6}Xz)-7od;CfDHEN!l za>{?{YdNjY&uhU)7-7VxTx{G^CVgL9-)yo@O}jTfr@qf{nB=}+yHSOtjqWF%846KT zR!&VC7}2jWOzXa2)&0~Givm{Omn8`a;an~nsz#5bMvNY@7D&X0%_Kdh_izu~f5X9$ zF3-dbSv1nK*aX@e-4#G~P%iAUnHR!c%QVo=o6E_DhDl1WYba@%A7hj7ylZjuH@&;- zTj?unD##l*v&`dID6maB1fo|srzjXjhDV1Jd|@u;nm0EI=f0M*{L0x9pIEI*XL*Sf z`s^0UlTv6{^YK|C63Nz9w|NbZEh_`0GAq*#MM5)6)(UM4GUpvn2QMc@8vU3pLT&Y+ z5L7Y%iQCx7csxA62(kTu35>%@?!=$0x2Cj@JcV$T)!mW=mk%2@Xb=FVO!9pw$(u+& z#{9EHN&{6(rIVu9n(I)`$}WQRE;wq8FwNDp>+NRMR7zp+0eMu8gLN&9O-3{8F6@7i+VhP{5F5D zVJ}*Bc^hqsh$XGl7G1+V?-vu|%#fX(iqAuf#oG_yT%8g9wL-Q2V8|k9Dcjyp0qeNY zpY$@}?<8`5#v_7Bw8=MHgXF?r!`1uKK|%%_kuwxGBQ6~sdRsR;0vdJZ zUi-1KLRP&xg&ed2QQSZ`0ZNWa8tLp+^&)nw<1_x&&d=XqDl&Rl4cQ*?N9A_}AH1Z< zbkQX){lRXheM`AazwtuG%LlFb>cs??EAGgAtx*+XFwLLXY}o72DjeqHA@nqo0~K+D zilD#~#PVhbpFXwI!(bh>(i-1Ovx!9oPPRC|{fQ>qv{VN@4)d`pMQ*c9Lgy-0%#{Ad zSbVoTo6}ks3Pf(m9w_$jC*^zSx4lHprGX^X?Z(ln;pruDoxXq<+UI7vs2x<<(?qMP zPze0d>CrHB&M7j8j7Rvd+P+&b%*oy?)uz2L1#(gk{Ci z{VMd-9(L0aS|R&|JGq4ye9nEhhS%HCWI7j?t({aYUAou5nx>Yx@I6TIH4rLCO{x9u z!vjL2{I)9ex)3y0oNTSX@>Rf))0V1q3W2ZTf}nwDgf&&G0i|~xw?Ha*DQ~>5I|5m~DFSL{&M)=Zv}+#~$;d&vPg>#1d)x`Bi&8XH64`{ojD}{e=x`t_E&EavR1cq zXyH)2(O$vsTvRkBG$_$Qtyzs3F$`DIQk_{BStAE1v!I0B+)mIMMgDP z28Y9AOYC~vuS1ow)~LesVb!N^^G`i}fK0U-nLvm>YH;>;cJ>rP2GHQ!<#Kw1FxfAy zLQNxpD6~xIo0hN$-}C9C&eW2~+>Su^yXZs2<57*;m7p>7Ny%W0AE~P!6tqu1((HZC z+)t`36WZV4i84$rO54PPl&P~03ERy+guDSB+s*c{kmE={JnM5v$B{?JmS*!t5M$`! zoI~cCHwxrpD$^O%0H2uO!c&@3-~;9RQy(py{~(LGy3^wi0tT<1djUxL(3YlFz;1-% zH87O|$n@o{i?E=Y95n%IRMV1JPSdMVLEonzCW}(sIGgMuXd?5HpEzX)83 zfwVG?2=%u8QX~U2<Mu;0lDJC+mvCbguiXkgds$ z>(`Lp!Mw9RlF+6@Eplur%~*U=@2y8=Bb*LjTKOKoW|GZ)^Bt1nT9_gmD`8E`BH1_V z1}^_nc&A1n;Gv)S!N{LUFu%tQU<$@LYAOmA8*`{kd2Zt4obi+wf?`{WCYro%B#((u z3-EJM$R>IR=BkW5`1YsRi(+zf!4+=i2;M}hm^)NJ=-2h@4|vxbT0CEn5mDyt{sGe+ zWB%z7xs;)3H(z0kW%KaxZ{; zy;JQ9izI$q>sr<}?`1!*DLz13E?UgBQmjEEwHW@g6Ns6pAWbgTNqD|tcUl=VG!oM0 zLg-;J9nFZmpfHGLo}A=Fkot4KLR~p}0g|_VH-PGXOzMRHX}v+3zhIN^_H|+(&$wXj z^lf%!Uu#yW%%!U@?=|_bhv56Ccc`<{j3gxH?<^DHSgrsFoN=Cx!)~{nAentI1aB!I z7rH!zs+2PyIBs1xKSf}*?G_%qG`5I^i;+>kOUvS0ORHZ`6X?z}MaHJy>>QkiRz&}- zeuhk)ulKmjn73`at^whXvQpt7l74#U=19&(xV-YAk>*fUdk<@d4RM!Y#qT(fU8s0p zxk{=t;})OUmt?WezffJ|m*>$NF>m)}J5W0@Al08;JJ)TSTuwBK!|O|1%Ikoe{JsPjZ8eh*BqguJY64`0%0at zze@fu08K!$zdT0!f5;~o<^4r`KHBhLiwoP*G^)g6-@de;*FUi|xp?FYPvQVQf((_`O4kFCGR*vxv}G&K7D)L+&;v5X%5RwHf7 z4B$r^K%rkp*|9gW=G@e+K4FjQ=%~cyJ3k-2Xc8<82 zrrR+g-N>@b5FSH@9jhsvmZ4Sq%L9wTuU_-?vT#E6k}`u&@-FpJd{}Tm5F1;Z2ibvx zgrfFfe4bZB&tG4s^A{%UktiKr zM`wpSHO?5DGL_7*7ibMN#Pz7XW;s$E_jnp4@QQJ&}hd6q^ ziZ}`|w5BYC-+c~#S2W6R4AVcsad!fcD5)Q6s6SLw-z$OCN&nhH9Hs7lWMC?G$0MK# zU=&5KO&!o)_rP&t7>Pt{1hbH!cWf95O5odrrIuM{fp!NZ_)5qQut>h}b8c@qt8EkU27?Tgj8T44wwY!s3bs9;80FhJb6bEQ;Dsee^A0IsIf0&c;PX zF>tb#hFE!-Gl@5k9YqK=0$?IkMmvoMzDE_%>jSF(>Ey!>Q~wXPT92nL7MCE5Acj9! zYuxT?_*tlt37{_B3Tf_!_~~f;kc~n3q0|MBVL*uSFY$GtmHNbF<-JdEN$R9LNl!hA z#ZoVVIFpw5Y?KhNL?#ue$7A~cZj!@jL%^mmy$X-Q?v}`@fJ_b!Ddj^v?uGDVj1R`Z zP$IfzG#FVS^<)8cEwDJf{A~u2%%k~Lk{l0&n0Mf9ECIj9VWKA0m)R$~!4n!c#Q$w5 z@ApRJo#rfD6cH1l&=a@dGQofen{GOPMgZSpe;Ihfgr6ZN%VQF(DOpuxwRxl2~g~TC!6TCAPe*TU(Q-S0?s8LG3Bo zQ?mLly+g&ri}DxW(&#Hpkoy~#L+20OPR24vj_V_6t;`tl1nh4$JWf_CG!@Zkg|k@|jD=VvT&M!C##Fs)9Zsgqur&p z`Ue(;Ci~S5Q@!~v#WhP48f!~ZVsXvX+8BL|LD^beRg{xe=ql?@i7lO!lyX>;?Y?CMRfx#`r3={Kcy@pTfp*5t8y8uH`S z&SX=*RH97a=%7v(u(*I9H{*?4n^bF z;~}6>dhtsv%foVPx)>&v36-_Y!HHMnto+%faS>8*#f3PwCKh_MgnfdQ_d0R$oE`NH zacC}ea4zfVxkQoEP+*{BM4RUAD9xkARfe6?Pl`;YsURms4(@%D*8WNNcc3}7=eN#Z zotM+&$i;CN};Ue)At#$r))tOD^MSI}{~??0k^=_Sv4azJr}JpV?8Hli9c>ysks(;Abcm;&f+=q#!h_ zP3{uW`0FA5_d)%NqBDCI_9zoOE?OC7NVCcp>Drg&s^Wl?F`fJ7X4ne4R^3!vv#!yT zy!D^Qu%?C)-KUB=kIH|C@Ii>WerB*M)D1o z3Hc<9n$a`whjIU~C1s0%p99WJ4atl0SFg*^$CBc!IsCFEw=`SR3_8-IDxV@?zoWy$ z{pTM{$ss&KuEo3}t!ZaZGB0YsMqJ&K^e&pqj6p8@6nPfLd?gUtL<@vx2$=OkTwtu? z$Hy)*MDGOa$3c6EwQ(MgcUp6*jYfE5irG$zuHi)$EIZOyxM{^Kr+n|&WeA@+eudF- zMaz1N-|hbm(UTO-HJVlzPu{(-ki+AG^N!K06QnD1kjwKzy6RXxS4x-pqP#{yocUq| zvtOH#;9{EK545M+8G@4eA#N@8_GPGKgH+xaDQy%G{5Sf4Lf8kGmynQcHZOLJAtI?_l;k43k=!1K^P4t2 zyC{ViVwQUJ_HtrN%q%=mn=`A(AgG{Ts~`!fNoCB>3QO~-*AB)OnlB@*QhZ}YNucY7 zwt+d=4bIe9kuAR?ZQx_lz>U7BjMDr@a;b)R*Q3}uf{Y)(G72s=D;zK|BtIqF*UWJi zU6#7vA({l2mLV>WJ8=KS$-V+vWMmG`9nnF4%8DJzA6$~Ieyn>n^;Qq{9%Je@B%5u7 z@EVL?wf&FRI-5-f(xsZVw$i~N_~(Cy)@~H%MXiUgD&+BQ)(DLmtyx2RQc*-7`3~|= zF9F+(d_+1nh{!k?#HKM=v?kH4N`n$1yf!*2+_bz@=TicJCpZKhx0`Iz*QA(L7H=EE zXUO_Od}dUwZ%!h)gi-&qOhsLPg^OnNM8CCL|+ua0|+x6UsA)D=xT&QU>+Q#aZILhoJ9MC*MTqTS9CoZ zUYD^P^a>J5&JCrq$Z?-9OCA1}$`Lf^{0^1M;nx|04i(5yIRXZQ-=S1G{02k7q3S<- zMHG`_ozJFF*nGO7ACLP?{rOT1faFE!7LTHPC!#xYJ!#bSUEl{swyDFkY(slWj^ClX zq)o*~ws7%>MmF&3E-&~1Du`pqPq}+|lkZZxz&ro*kXsVbi4K~U%S+!CO53Gni(YrQ z>=z-QvWM;97-ozhacSc)2y!Q4$h0-odkDpdeS7tC_Kkl5d+^Nuv-Dzg@il2^{bauf zc~?A+cUQQA4jCE;KD3y2VeK^X@3ztW-9Y`{WmnYvQD%r7GDD|Eo1x6hFR){b6JXsy z&l85|P|4UjqJe%z*N;bR zUC-~db-#;S?4?&=1RwezaPZ1Zqw8N!?ZnUoY)>* zq`t-a-61T_<-|DENypxWZfOi9F+yh!WXf1zpr6V3Wh^oXG$le&dyh4UC-_ELt6 zN|-I1OVL5JuH=Q=HaEF4;S_1irUads?4&-9%tRVOexk(>qWWX;x{hQX?+%4rVDQJ` z*$lgrX@tj3Pej)Cv0s8XR!%Hy7s2{JlCaK+X!`I!m#(v8NzJ(e$zo!q-(bv`~-An$FsRf*XH2*{scy$;};07`(cfp;r>u#)g=>+xe zzW7R8O&5=NdE5-y5xRVEE`#`<5%{TGyFG@J0TW-*1pAX zbi*)5gNUPRC*o-QSQO*OGAy7p1osI53>U4pGlC^sG-o^lTWPvLpC| ziPYO~P}AbQF+8AmaCn?(;SFxLIjb?5Be!Mvz)?oo!_i3D)5fGlXdQ{-yh?tK_IZI> z_IqgheXC46=xCye)&BkB#0IG+$LEC@swc{RrACd<3vmIKNWJ+6)qPQ3+|NvL(#2;w zP&JxE_~4V&twb!LXNg!wdhZt{^3s9BPG{d`$lx<+J(P>*KtZOrI?&dUVmQ5xD^<|> z7%p{UlJsDMry^_wJv$oXN{`yYu`&=!hA+|(wpPeXYGhlTR;@`33O77^RrjH}>AADF z?A#p0wf3IUwAnSjw8alBOrP!9txia{u?3_A)4H|(0JhbhQRc`gvZj{B=az!A9Vv2y zDK@*gvNk2Ypr|rXxnowAv#2wrboL~Nh!dsOgmX0pN9FRG@*5TwYPGaHs8}q5BtN9< zCML`>nZZ*RO%Mws0u#<2OL2&Q^|HW`(=mqetWh}k+A?Qq{uAdflaH8-a z1GI#Ct%M)zHRflBrJ2-go3GJatpXv{fS8>We<$8rQ5xum;BLfb*X7v@I?^eEnD+QH z&&-OAr}H-^_z#!-EK(PYstCqM0&lLQ-dqW0n*w=OYjzU!h_~-hhv_l3EH8^CySM`kMjdfM;PwOGO78?t?PQRMAdbQS0l%;^ z_%jIKJ{lj$Q?w89CJ~KllWFQ`Y8c$rpj-=K{bY9WcST^WC8DW~L$LfNiCa4Y3qk-aJDi}+5Go>4 zAs5kg(7^y*2aRoxTii&!S$a+EnhM~{!OuSE6(tThv6+G64p9KxIPiB=PXL272fgmTySS^&h&gOv=47qkhn-EGp6{R;=JKcsqhnIs@tE@Shrx#!r3DR!Xh$fHH8OW zVcuJ^rz8)&LEA<(oCh|OGwvl?TkxaMtVKFn+sX$&^ao_9oPvu|may%e1~`<4y~_)BneEy5weZh9`bsTQG? zAlFH`Obxbn@C+NxjRZpAweC^+la$BlK z_K~r+S&MU*9PYAaC+bsX+z3imE*CixylSIoU;)|?uWqPn%$%{MeE!Kji^?5^x!FZ= zMO_(%i>BmAXSWaUCL{a&1N9i87^)|;U9u)_-KBvS9(1;uSp z71>Tqs$Gx+>e~B;HYlA|Y6V%3WQ>7L*J3y?3tNcO`x)b4dkSuwG75GqCNn0)JpU>1 zjRn83x^)!zPh?iDgkxdqEtJ$_k(#zEs;3K&l&_iT^0lV)U3H@>)$Tc^k5#a-k$N>> z!{R((87M5N9Vtya_ln$Pldy(-FdPs{=qEuNfU9u(C^f&uUO%NRF*&8PCQqGAy@l)@ z9?okIa+K6x)I|2$F3KA*CSdS0bY2J2kv6KXiGA6UXegGS3Dt7B!<}L)NHd2DiV8yJ z%teQLliKSFm1RPkHYV1Y>dnnIhw_W^L+0R&{nMx45ybscu~a1uI<{?GF-8> zH$!gmY8S{=YMnYQ&KRdx>1@f>sYRP-WETaPy6w-X{WOmC6XE!KOQ9!3J#otVD)Gd1 z(Dq4Le(CR^s9K~TIB8VX zQZp_LBN_Vu@xZsyK2F+rALmNZ!l#XPgic;5l4B2tajpnuz!E{eoq>Eyj_{2>If-KS z(Rgyu--NiXfY=GERVG|~op>yg;(w^W7F>dx^DX-2Ylg8~^LYD297VFjCGUW;Gih8+ z$)&Oo2z89kSB%dzT>Yswk~~;X9lk`apz+`V+d3435o)VSbIm7gHj@MN=KSeKIAmx!lb5!vT|oiP3j51t_&g!1NP z&Zb@|k%X6e?}}&;1rZHbh$Dj!Sr7549b-2lP_HoOHR0Wb2rca{6NMfkQe9Wma!npcCU+Mi*b7rE90tlguf>C8ciN`#YQtQcWZAQsPY}Y`a zGu0}x<+Z0_w#fK!4MOAlF4_mcJ|RpqjxV`D_$sERT?7pfT0K^3k{9?CB@^Ox{zrrt zTo;IVh?kT(Q#R@-8H(_Tc#2D%>jx~8jm+l>2&psX1mC4)@`pwh6J6h0)T@ znmC}r`~z>8lpupI<5B;RTQDE^SS+W0R%`y`WFe_~jBr6X3IvCZ+|cnXbq%zevcLY0 z8U$>!ncd8e!lO09X9XTL5q<8}lYOlvh5RjL>+k%2`kv?8fUrU0{o-nhJGKebA%*2(lTUPWPSMXm>ziVP}I zv`C;9(tg@PJQMqke1Ron8IdnP)(Fc`aLMF&z-QUhOX9*E8_IHKl*T7_ZcwCo3vDhn zhkU_N)RR?rZM)x5G(D$wbE}`S_D-9n50toP!?}5p*okc?pQHPr4A*XfKMqx0y-hyH z6i{*CcW-Blu=hBnFs`66QORiQ2Ec0yqgobq`&$yVToOh7%qhirQZaQFu=#ud>0V^VNRslW0CYzzp+G9E_Tlid(bRir;gq&^Gv0gAX2RgvN}OJ>C;+fB;Wai#X&@iqx^k-BmgGdcSfwS8a@@Jq zE=!8nPR@#n&+49JSLsw*_@>AK-!S7oPY&XDaV(59Kw~&OFbwgAM!rjKkn^zz*p95) zfWIK_|uuWUKq#6C&y*T8!q8Y`0zJig68n2gRCA^tKazH&LSDY0sHhSsSS3gJH)v#S!} zIN!1P_$E#)J@3pov>kw>V1}Z_m9g9pNoRI;i2RU~-oBd6Hy6w-P6~yRa;6tMke&h) z(T4rNmPYiHnTSYUJj@J*vaxJnXS?(3T<*y!IyT!=Q15Y-hjj-Dqurc9kXEzZL56%) zsYYXJl`m;>veB44ImzmFTkS3vgfW%;4ktOQE(>ECnUGl?=PVAykfbxK(iu0&AG3{t zL0HVTc=j<*X;7~ZmU>*p0i7;T>>7r_`X0*vziC}n&^u5OZ8b;q7WZR&Pb*xeMVlmv z37C>r;Y}x_6l#+KDRGPhDlx6gLZz6C_#$SDh#HO!jwfTRbYwho=pFXfhw3=frk%3X z1T2mEAtuqZ_Zf9Y0>}1Gx#}aG>7NSaM;Z%b7ix&b*faFL+y#<}Ju`m4jf}rw7ZQ!J zlKdC;HD}6%V|+b+4EAALZjw%yluM7HPs(FtbuYCa3$R>Bx5A<)uX?=r zDLEz+^hd3Tqp%s}K}TO*or=#l`-^;L8Sp|LT}AB1PLjuHeY48^Sl^F3A0c)F9m>5Q zKs1v_SzL%G%QkZ09{$&cW$fZgxN@5qTYYXV_Bn|A4SUB_otRW(>JP)8(D=Y#k&Wat z^lEEHVV_w>Z%Pd9RH$$>%Key`18pF7cvk}d11OS{IX=pnpE{$`yJlV8np&qRrQDTE zIek9192ANQdh^XaPXcWJ#weRTMxI1~q?$(@X`SIL=Z$hry zmd4aC*j&2thvPG9cRW2~+KIjM(e~xwFVI&hf<8qoq>eMPQQ}atMRY$eIWO4U^Pzlc)PN~vXf>1;BH75<*JdnHy!5tXaoi%8Yi|r!j%loYKaC%xO_#F>-TiFiCYxj48}6E&eIK z))vlqcwK38qG+kw784y;+>?CtLEnX$219*xG6h1QwTkDtA;C8ghPAvYGZ!#(~}LXqeWQS+};% znUGsmo+)y@>77(P$s5Y`=K1VWm69*L02!w-6t7CGnw4I+xHcm-*R%b$KvjNvRWh~!sBeF$cOBBc8&MmdMhS4ZSDO=)$zqAbaAO0_$~?#^{7Oj)fN|CXx7 zasKQ$T<1=+n!@pFW3a&Po4=sXJva#C`47Qo_>U|L>rOk`$GeNQ0J{&uiXJ=*=0I4` zR)xnA96g?%TQ9&C^<+9rK287Lfil74$FN=I^93-lxXWJ|k4NlfJ|mv4z@Gx&usx_R zh|GgxWfj^zWpjj>5*-YFAU-O*w5g64B@{&`=rAK+Z@{0bz2~-@azdmmN+_3zvYO}T zsN(sk9(y`|BY2<9fz;p6x|eu?uF;0$<32dPg>?_TPkIk1W<|b>o8Wh?k?+FafV1c~ zP#>S6-Uc3!Pvc}j$8H^H4>uIf#Ow|z_~+yJ<8cD{5U1VQ3{GNE_ApLpENm$p%L3uB zXehvVom0ak_;b#}3E&Q)C{$oqJ*u|U^fcW46IF5z&AdTmPpp?E2$Hxuy^`rFlqv$&1(T9V#;D^aa zQ13w-a?=3{VgV33+={jgVbYS2U|h=cz%iX;F&XQ#jm$B z`RD`i-gf#Phq|Pz5Bn0`Zy;VZ; zkx=5fiZEJQSY_ZVtR4Gc$e)}Eelbx3?1QuPnh5@lzlyy$bniXvC(VEWd+`l)uNwTD zoIY}IHqoy+PcZie|4O}!y*;>{?yU(UP3&#;fK$ET$tUUGvE~wqL;&gP_)fjm71tc^ z={>%wqGHqW-k#&vRA7}|4{azb+wf3V=R@nu%GN)G>R!QBU=a6A^!p5JMwQoFlVP=H z)OfvB8CI;;U72RFq*c1zlhNyB`1=5L0;`8IjyyxOj4Dw!6`3XtbRP^DQYH)}N6B!c8rO2~kkdgII$G^o^$ z0z007ChA%22wO(SoM>G{5t#^> zWoB$Y?lVt)vIFhQQ7vDAJIUMh&U2;&!+cH+#PKosyS&KnP*NZb&W2}V5LLKv7*7xRVOBUfqTzjZrXw%? z8lYDMKekhhi4{r?mXgln5I9bfI=B=VJw_1><6MatS0RbQ#EDJoY8)Dq#^zSqJQ^MV zz8QDUXUo=$6w&PJqHxEQ2Dg1ux=X;f`_halb7uFYg|a!&ja-{Bxi2?+ao_B)JLPUu zR(m$zVzUSq99@QvSqwf6@pnM{v*FxCWC)?mp{r8p9&gBPutEnW6G#7NDpL>*e%iHo zuW>EJ%Am2jVNLbGgpXnVX}8Y@{* zJA9oK3oYUQc-|JyRDREB#&d%N0YQA7#FZ193aE|1h* zm?0h=5Ra_Uc;Fl}l2gbtoIywi*+@^N8TzQ>QTuibYIXxy&7kC8ktJr(Y{^|8VD5vJ z20OL>(kQ=urd{o^D1b>wWg^_cbI~TCuz1w%XMp!YBA^R7xD~D$PqRdkHwjq!QFjPe~z7&LD8Z`|!m;-H&>`JFzZHLjWUITs+>f{QCJ7lx^&61pH3#R4N zwi#^c3-6he)X`9^t`*w!F*aAGFSC%oIi=ZX56wO}BhXfrqrzS@CB@qn@_@md6lYV& zlfYLpu}m$K7~+%`Us9;JYf|G)UQg>@M;XmDNduOMWUHi-;{1|cK+yZj@ zxdnJArE7C#`vV)w;W*oH7SRmHkq+gz=-`Vh{nXw$YqgeLgG`_0+(9AvDY=LxM4bjf1W6RbWkC#wO2}xz2P36i znedL0=|Ka=G@+rjVRZ2x3Yza!C=y#sJee|w%`E&{5+yLG90dVGc*fq2t3Nr|T)6JU z{Nm+p1#(@SPC_bui#P49J3u|skv?bNlkYED`QhzTi&h_*rVF_(ysWha!N~y)3|dy- z+Um?rNz&jdJlwK8r$uN?r4iPc+9zBdv0?uW8Jb@l^(W0B4BD38wby_o#b=S zwieLmphmXMF6gtx9^#FSA)8>@$$E(JNSjf*6Fk8cD7ihMqYJz5Sddw?>Yk2;cloQi zYPnn=8+3K|Y7_I~yx~|)e^;<$^IcE(_C9m?+Kynb?=Z$@c(@tPykdhT0QP?vxEcYLy@ZvUpa6$!0}KOIZAfR%wNFh;tPDgbQ|N42y^4?IA+ zo&Y3fiFs7dQZR}KNEC2SVpzxFsJ(Hu;J4EH5#$j$i(akxFsx|d>eMFcC(zz?{ZyAc zKF6(hNT|~na)9OxHYn01Xf+P4@M9`eb$Zn>32P8h)y0m#ONG=s_{P}$u54#{PCz0| zGT~b}(Yp4Ol=gLn=9C1rIw8euNkfm}NJB8& zS$`k`#AXOHJTm*k{f8aQQqFONnW)ndRxt~BSjAytV-;3nN=8rRNH(E#f;v77`gS{~ z_!FBP`}XZo#i*3G?B5;V0zb#?r9Uh7VR(!y)ns9E^A;0)rJlJ_saDE&@80d4>Q8Ep z-*Y4Utdidx`5d>0@jJwWXNhBQ%`QN3SHz(he((aD!=&+`t)(3iGh@(3?|)mvei+nS z^A|Z*AAWIm;ga?e%`%}*Emo>D_K>6ZHaGZl6q#hFx^NN$+Xa|NZIn_U3KqBKOb*&( zgc9DOn(wwRerS0vXR*8sBZ*G@*z)H#7wbc%p1IL7kwT*oZM&vf?N%klG@>?~H{4`R zu(J7!=Pjzb{*|Q%V-nR&&Gs*Y3B(rmA{HBZALTG+MT$n3>B1$$7%Alx(Hl;q3OH3V zCc%?MN4H4rA_mEU)pd0igyKUb9(h6dNcP(9tkP|-tgtJI!w zzPmnqbysG|)|XetkUn!?!pt3s_3~J?$nFUmzZA`A2xcj<%^Z_E*%-`Mr^$Wg^Qya! zU)RiA9~~nT{E=tzrdl>ilp32|05;2#Dwj7m-@CDvJztmbg1j*>D@GwwsdBgz;SMojraXJIUPVE??Ukn7sSL zEhb#7(o6b$rHd1rg<6$RXSAs|d$N6TO1ltG=V!F8E~{IjFHA^}R}$-Vt~W)ci}p>u z>h$d$?48ju^60-w^87s#QIuH19oWy-Xywd)*5Kb*EaCuM>**R`C_QK&7wr$wq7d5*E6DWM8??H|?{aT0k@NUzqvNPpjz2zIGbGxr1`*Z9eeGsXytd(>m%+ zCGhg8x8xM{$+`GM-b#^-6vk=45T(ZVW2+7|9{lR7gN=u9pTk5I`|F*~1|N7#moRiT z?NJC9r~8;%Ieum@g+qtK)=1}ID^qelR6WUry)+(v8MB>@y>x%c16WF&Fej_9c%j=Y z)xElOo7^lTxHS*dbBzwS>YC+K(d9>qU$cv{b4r^;W!^2z?_Yk6GFHjqG~qQrgBO>% zoj(V1TBwCGbAlQusgHGWF??`cD68e?>* zq#pQuZt<>GL#hO;hFt;v^zS_%eDDGGizcYuD&}&eQLDhp^N!*3hgpyo z7wxOi=Sb<6!gO+)4M+8Y;lwq@p*M}MNlYS@{iOoJXadJ_T2i&?x#`U+ZDHCi`srKm zncA|WGZ1K6nn%5ozBmy?2V4)d>$0`rNcQaB>1lQc`TX%Lr;(b;m0qZ}PG8BjKHc>M|womN1YI*!utr*3OaG-kA9JU!*6XM0+A z&MH)F)B55K9CLLrsUoD4g{Mu{)b?e?WH4B#6H%e;&sRXjzdOdin+~g)SHv+g{JnM9f{2c-d~x!sI$s6+v3r= zgT=|YQ^M-0^-YbByg@#6Rc3iVhn+s-y2`ch-_l_4<=R%-O%7AAzAvR|n$3ry^ZZL7 zecK=o(xG}3IxvG|(py@@HZj*C95}A2%=`$e*M^J(CpEEn4EfNiIyko?`WaYq<*0Keh;i)E8<&iz(RDk2D64>zrf)3qlmzt$gqrB4>1hjE zlB;&Uw77HMjF3K5o=DxVY6wu@;{R9QVlQY^QPv$mu{@joV#yI z<#p>&};RAD5**>#{}sROOrpxoZG08WciAE^Ndo@in1`D1W{S-=gbZhVxIOd#TXp%_kP3F?oQ)0}dVt^MDHb5?CLi zKBu;A0ZXWF9|BU)vIV=1I({=KqE6fl%0b#5>J<>$L&r@Otec2J;y(IpasmSbjPuyX zZw}7HKhgEaf)&PKVVpTRPGt%e+R^@8->+B2hmDT>B%R6;8fX}I0r~D= z4dKI|jqFu054ke0M3gQm!8dFAmtb|Lv7GmrJ$=>z{NKb*dN19N7V38g7QIXC>=(ju z)`Lnyfd9hM4#fayqkp)X$HC|WezEJB=Io%I84eeS1{5;Wrn+}c#+@U;UvYPru zBf+)UFW5#S)dWY3-WMa{%A)rQ;gD1#-5;D)NlYQmLjH;((!d!J*CaIab|PO#ZHqe_ zM{U!9Mb5KM;CLUeI8FTlNNPYeCOyAUvcfBxY z<*7qUDxJi63V&qsss>MD)9UgB>R{#Ou4HwvHe+GYg30l-UI5~|W;}QIwH?9K-tBik z)j{JFJNOf^9md>D#luKnd&rSYa(Gc2LXeD{Fh=#M$?mp>Ns7iBW~Ms(Uv^f-3DcVv zZM(g>cy6h!^LYPWiQcI(Wdst0fMuM%`OYat%bS9PK1EObwRrWB4qHQOm){i}ILwvs zJ6|}uyD!sQGcTv}!JF3xI@gxytl@lfTHV&JaA46>o45iFldB1^PLmw`RVe>fRw=|b z#1!%wm?!l4LXmdY95#&HM@q6Bk;<${;F5_G0wPLnk-E$60*x_NC9@fo11m;-QV}Fy zWxD^zP=cm3XAW zj2&^)RY+&4LHNy|L-~js$_Q zXZO?Hm9wpQlrSzkPM;o5luC@k53JSKbXl_!b)K3<1>JqbRHHvuR=(@4tKX%LoLP?= z<1cyjhFWies%B+XoLnuJ>6w`EHtHdK2fLfDP#iix!XQ1|F#%_qqvuCYn!UZw-V`s! z*b==urN|{G*9Z$E*8pee8u8|gz1@it`D&3$%#%BF+~?=d*at5t&;>5MfSfSozmje2 zyHR}>FA{TQ0lMR%0R^pe(J&4=n2CQ3%mZ}-DhAtSI)5MbwcegVpGU0_iz?0cfGS zR&5|AHSX*V^LV?&3IQpJFAN0V7*J)5dbS#lahy7N{*{B&86ZALu+BqzzlP)7PCmoR zL%Fl?#d8@ngo`9V?AHr+^66Ygrf}k1Ml`rrPBjX^y^-YE5ptr8i0FE3SqI9HMbEu3 zyD|R|eM{gSn+;h5?Vq21ChbwU#?WdIC_VIYep`60hI&rz zf*xi}xFItWdvJw(%C0xA$xNo+oK+*Q@q-2Ho>-E+8oRG)<_ln&uh^@kZtL9NmrAAJ zeFwqq;psQEikVm&43Zxr-w6(mtC)5isQ(7Ws)2SZ1;$z;`pFz)dOgU-Mt<~05%Ua3 zOzEk~vbWW^C#PGhHXHNoxhtoo)!y{RvYvx;lC{Cf0nicC6arC{|If~no{Y9RaiueJ z8)svO`FzjR>slASv9~FA$sN-Q)~;Ha1D?^x4V1(1o{An==3HxWe{id!|dhX00vTt0ffntXj7r#ifnSs`nYfP7Nh? z-q_Xm$m)Vn*Vg*Nl|2n2vEl~5XGcLlK?c`3H!FwkAu+B- zmNU#{x*C-(_@SfROMVZJ%9(LZAv&Ot+O@eNdq!#8R|!_F%OHujYFs8Mc8|nu|LA

    %Gds;)y`W&~-oA8q&7z{Bw3u*XRzXM^s0TS>-MOnIFSTLyk0p1z=@ZE%oPgKxl|VlRWz@fTJfzJb2hi?765wK;GyilJ;5j9sFkdsHx-G#zQ?8UK}fF0O+um3#%O2c)? zi%v*q3;8(Bi#nPYU$ACHc=0cW7lYqYhp|(H0csz5;^)X?JK&nZG%*-@I{5e*#!k_7 zqm^qKJg)qz(loy7R6=%1c0x)#T{jvpqE3yU`IimhIJL84Zo%aFHEI5U<+|IwMd`uh z+=Pr&rB-FZPhKDl9Hdv=o-umW&074w(kh>JFMCk|^1!=8$v4vaycrAGL zL4S~ElE{4vGTYZyn7WpA0qnso@SE9jV&kZb-}K019iLuz72wt!_GUl{QV#IQQ(m2V7Izwl1!sprD{4 zy{V{FK?uF72q*|hmo6pr-U9(tL{v&ddKZxnK{}!L4v{83p@vTAq312w=j`p?XW#P9 z-RJ(_egDTVj5XJoStWC=Ip!GO_~u%#Z3~k<3iJoLBj=^}6~C5m`+WDgGEto1X)P+}gu}|tS3EyL8?II)Kv#*8a$5%WfX8{D#iqgs6=0`WzLpl# z>~&-i?_2wkJ_J%oKXau2q-^}vC47j9!r;<@9N0z|=7d2=PB%LvLvQ^kO<;+%VV0K= z%N%-wm(2Z|wlpXUDbi1^60tV%{k{X|7P1l(mipWNipZEXR-G4r%Kz0gC;SI zeeGLBF$raBn+LcYz1Bz>@`HZ-7vv)K+hr@d^?Io%wgFYf8LOSU@daBsZ@ z4@}#ZTd5B^ROCh~JTWB!(fe{PIXqkY5i5p}qX4rA3AHjyy}i{ce~L8t_DiW?t!S-+ z$#e?>tnky93h}*VE&e(@32G{}SM0~}939>Fg^A7D?{$r6Nd}kMGFDajZs9@}mqP0Y za?QLz3;TKX19m|ArEg0eC>X(ek)VO@iphHwUnt@sT@b$Ma3*o8&Zf-_(3$P1=6Kk( zv`1|gcgE?r%}d$&yS|bKP)xrTSUAwrL5--#jBemP%D2%PudPFkbB8%^K^>>57PN8_Q8$ zE`DCVVqsOJSROw{x*A)F^Yn3=6+GY~rgy_Xi$dij!wcv8uc00UtUO8iEyYeXM z6=CR&d$@5TSne{bQul&7rN#otr*CgO^=#{OQRwMn*&wNC8%4#vSvu_E(yRBh+&u2Z zN>MkQ>(bTNK15bwq*!PJDp(*inhM0L7(I#DX}w2Yu7OAwfSnF+lMe|#(iPej=rtfW zepPZ#`(Z;6D9XE3Ww=KRY*VS3<$vU&ziL(PUG{7t0GZY4U4ZG#?pmMlg-w?cJca$HkLy3J6#0&AK_QRwJ6^lhMwM;{{{)V)>6L|)C%HGX_ZQP0#TKxyeK;AgXbJ#D8=c;?Nd z2S=l*`0C9MOj9Dy=szJM=Gz!50_=5!+b4A!!r_{*?!1ibr5lPEd_=s)fV(|J6`v8F zK{wJ>sZ+FVJ|?ngI#o9qHRMF~RL940KkjQ?nKbU25{Uni%=?3OW|Jv8skBLuk*}Pm zEn_-{~6sfmP7v$}T%YZ3=v!qRp@ z#Uls-0^a^%f6RW$6XnI$r~ke&mj7xQIL5 z-+Nku__R2#d^3#p5m65MEWA8y0cLd3KvGM;*H_$#``YGN{%$8h=WC5-qR;s0y0SF< z31*QEcl;iInNz=`C+Z_`>n2}|zuO1>m`_(V_G&|fgddK2mUw-W9pTs`Bq9}P=-_{! z0SywNQPxV%$j;gM{@OZ#;PZ$HO*dOPq~!X3v!Ia<>LlmukPMmr-fYgHacJ}Ly`6Gx^duodW+v=Do!t4 zW%u!Jt(-7y6$V+UpdkxeWqogEiiSc!&ORR{0+hBy=go?^+QRe~$>pP6h!Vej6S^${ zYGnlzbAOE#j<_rOyjnm-P~5b|;{7<%&;OEM?{+40P(vt&wINhGLC5Za9+bTE@a|_T zzGxk?P`7w$!5ezBC7SW<1r1gJW{3jL6CWn&fTc%ftq0ri*FxWeeISQ5ddpI z0g@_W#IaaUqO>kdOfpHHiZ=3^X4#Fk6*lD`^YU2$HdpF{=$2>}XlZHAhf{uc*^P2h z4`-XwP;1XNZAwnP+>q@VC#ZZes<8S?)k;-lUmArwlDh{T3wl`jn$o237MbSLUbZ5} zPx^}=V>)R)IaCLo2p};|6*Y1$tCeSI@n;%7hX+>2RJ3P(EIUoa#TY<6CH}A=l=@aA z<{{rzW+mj3hoixr2*nYOCj^k}BNn1O=SLBj%vbE*l)}Onrr3~tvrTfRv$LQ2$YMMrmQjI{TT>*1QZmZy1_2szWT`wcCUgyK<{nzl`Ei2 zzTv|tp^fp9;xpx5e(HS_c1dZEE_u_1CPtOgduXg&-~u`BJZB>>Ng%h*}FLJvUUXP#k5o#JyqKeXtPLiEOxM~7jdD; zkwv~b=zXlP&Z?Dm!FKA*xLTkYvrLo6R$)4hj+Z1H^9pX#Y@UCDz6_v;m8 zF=t$+ZK5eu`fLdORJq`O=HuEeaZX0@udOEX5@j1`?yMuzJ+G9_Q+?9xH(ghjn#pP# z+a&Q7fyT)gK&6Ou^>3HjqZ+)PI_T?2=h|tGYown(kE->WuWR)2 zJg0nc8Fy|x%4$aGh1B{{jq)Q$nkHoO{;f%_a{_J~?6A-Z+w!_sY%aM?OEgw;nY>JM z>5tnb4^Ulg^)%qBJ9_q|L$(FE6_Sih9u*wM+ScwbTXA28KXIp+j!3Pp)Qr`DY zAJ$CkT~uik`O($!MBr^~Jpr_lFq|Q8w3_hny+kL1zTYXgIB`->`Yi1J&cjY)n!H#&Xw6J!?T>0z2}`AEuS{JY+O{`$*8Hx$GG=b7H8+{ z`ID^Uvh&jzc2bpCwdYOsGG06yo|I*}_%1lslyBJIl4qkWZoy1`Ly2)v`J=O@;+h_b zYj!kUA>xs#?v*V6qLwh1ii^cJMFhB=7QHEqNbON-NrmbZl~rExSKZKQ$@kymR4c6R z$_lp)unM&?9HfQ$;_jX!%9R4aptG2`af&m$AE2|v2KQa~(r&~-%ysLcH64hwHdNG) z%9zDGX8RN?3i>>dmlDbPWV&~pMYbaH$CjnUnOaYH_4_nJKZu4TLoQiez_q+)0{7U+TkhOE)uP7bine0SWAXGwDYDI%@CYEYyl#K;ed4@@yIBAf4*NghgKAGBeW)9VBT7wk|X~ zt4I}sr4r|fu`cRTM)$@VmamC^#^h-=o@}V4R|cQm<%P#-SqI!2$JbL3vR@;Xmee`aK9e#0j@=>^a;V6TPi1~~mnD3{4kVd-W!FS)+eGkwLz(go<}Z~QWLOY1*Q{-@ ztTEMkBJ2;c`EO1U8as}@_MPHzcPh|=*(~3S9Xs;LS5}dtK!cD1zNEJ{!^q9BuByHm2nI{u>vxwK|(I(`+*^zQ$iyh z$!SwB`t2U2F-F2G-Xl-Z@9@ z-l4T6c6#<~$B@HC6STYunakAvb=&i(?3%3Wf>$f#k8I}%JuFAsR-{m%7>VfzbJ&5P zEVA7hyeyL=7wJ&0_oi3NBK9$pfU-KYn4y z8G?a6;4O0oWMW&VNQk}znp53#>5!A<{Da;jjw6=~x

    5&YZdc{QvR?Y)4dlq;cxh z1!JK83Q+(3)Hc58Z?(YU^mq3-uRap7=cW)9~YC7Jfnz?0t4*YDa?lpxFi|Nx^*=HZla)2*fd356u`;)g$yXl;`^cOZM zxJ_~VJ{~4nJLX4L?bIc{FS)Y2oTc*4m&uIdGe?JFySSli7U*$$a(eNm&f~e9kC|_* zG~zbEy6F00{}iL!SB+9=BKt963I1=yWN0?Z#e&~>zV}?J8sn?0rg^}(PO+lSv?fxX zZ?$l~`J8FX#>F4$1Ksd}sSB1rD9k@V--T77DslE_Gf*ZRAQB9 z^f5bD`fiN5Bfch&UL+~5RiW(@%t$i{uVoT~Y_sfnq)IXAn&BBmeL>JPYT|}=R#zG2 zxjW0=4SmGyljRB^$ufESf=p07V^y_Kt((CMMV$rigj!X;*XN9)j+%vycD6K2g*mwbQF zY#Y+XrlW@5D@MZ<4^O3*L(&}*8|ovxFX`nI+l+1qDJWZ1$P5; zsy%Y++2~_0o?y)Dj7#GYnQGv5LZI%`nI~0N`y6t4go{`N_*7o2yuUc+znbnrP~dW@ zZAeg=Mo@e~=)orV+nROT(Vf&+ro;(X zZ9jq#XAz+wAIDW*Hftq;$oyTtBtEf$#VGZOQ4!3n`g4I2$MtoN%hS(=kWlYbg*H>Y zs7T1^Sg+HuuR_oBI=oKwL!8x`jZf^%y7&2=mP=8hPIaCP8@DLZG^&$(M5!x%9_Pr> zff=}E9Sn}f`ihabL$-<7t=}pL&64B{J&k>~Op!rc!E(@c$LvJ)zWcIw@EJOn7gkRhxHpP0P+=`E?QD_KvX__Mi+?8xveg)P86BbJd zI9+m!WXuIIa%dntpU`gXUMs$K)D>x)yIC1XcaH$XLsAgpwrx|8yyKeBQ0p2vtPHKB zu4i$*hPGQ}lW;4qpsJ`=?cjEA9(w|*V~B_xHJG7Vv0Hz;|4d?fEq6?Fy<}(fl6%&m zTR85mAjdL;#u{msyT3XrE|O!m8)ZX*Ep98H^_wBU8q7%P+hHp{xal{F=5Du*CW6m% zj2f8S9#Joq#ug7}HRbFO7aMZtzBgndUE8@AoNkm=sZ3uV`@yxS#AL5&X{#!6+*|A= z(b{FkCT5{va-sLC?rj_)SP#fLA1to6z?~F1b*|!FuaK4UF1)RJAaKw3B5QH=K*z3% zOjsk@3R;x2#&7H%*Q_)y>aOA~AAS&U(eSp0VVWkp=TNXA6UrIS9QjmT3YJcb`NMH` z;F{Ob8{XH^xP@t=7z(rscxedaPKlg=RJ^2e&`$j*PmK8b7YnG)ViVIeyKYYZ_=vWk zwWzG#s!h-ifz~)<`$JrwSSDi42Al7=TP*0SX|K33J%SQL)f3U;?5u8gms{NC5qv>t zYD8&jOln&n@9La$%T!cmc`nAn^eOztI{%}pw=DygcWG}LNKIS4gI)^B{jVtRX0>T+ zgcoJATWP8P$j{6Mt82Aq=0~W5k1KL1M!o^e%Wq`m(U8{EO9}z{_ zcdfK^e&iQtga4})E^5XM{m*6je<%n2S z^Mhlulm81vx6Sw4X1lLFHQ{yrwk6P3X>i`Y-$kPf5r}r)H?gCIqk%KeCluS|5_0P9 zu0&^_+~h`dGNZ_uMO#HzL{>hX=e-Y=6*?F9za=)~uJ-F#W1VJau9L2H)eB8}bI3Ye zY<4kSXRtd1D+BKqchOWOCyX(htrB-31ZTwbovK3<%k8FJim3+mofF$))f`(l4?`;p z_O!Ru3}XM*y?KaAv^u>^raeh;6W2*6?gfvt25a&2C3hUUpCX&QAr>fWPtRh1HEp#t z=(U7Wr`zhSgmw05UO&)~X{FJSzPlU$0Mttl@)FDH7Cb)l z8=e-Moq$F7M>9Z+9M=-_pItkn-(V4UdpK+Tb`G^q=oK&a%CA}`2{mr3Uk~88yKV-l z;{m0Nz4sVE-v+(z`aXXRUXcW!7er7XXxj6D$1UioqfRbU$lov1_IjJdXuI-T;MSGp z165}^@HE@&YI30aKNu>OIXSj9&hWS?>rbsMy7!8Q$?p`gZ|KF(%I940_z`I(ytjUj ziJvq@4`I9QCaPAOT z(LPU5(`&a2qZ5Im9K1A{_r1W$?qha`Bmd()|NcBOqjO`dFtN+#|K?e#gV3YCV}riN zJLID0QeMPP3NcZtx$#q7wcZ ze`|Ivq0$#_vz1U*%>UbPZ=4(NsvDF#6s#-?*I)$Q>3t_06!klyz=Msp@Ih)=O-sa+bj7H+AbS8v*uTk=v>9Fki-Anz9VtYNY2>nav?l zaJJoD;k@8lLTnyA$EdMZYk~S;0MAl=Wd#yEaKlWuyyCT!t$$pzbjf&SwpP-4evXby zQLmz-{Dz`@M>t73PfNpvUo?GWUTA9{_4gHeQ!n&7*?eH1+p6y$EV!Os!9feh$KP4- z2s%EQ>d36s+tNi3^tPX2j82h^GHA_ta|Q6B1=2IO_h`VC57hZ*`>5#D)H6I5wEBBT zU(!Uq*NO@fF6|7?;_A)o10VZw?=rv9)Juq#UZR=wwO%#Z5uVeQHC$cM5x?_~wJMC68I_@M`PqzM(qTA!B2>L*iSa^Zo zsi^CoQ=b)6n^B1}Vcywx-DiOR!V!LaM>)?#B}ZE$H6|$CG&H6(P#YA*z@W-pM%lh5 zQmMOJwiv1hmv=QLlgW$30Y1+5#*V&`XTY)-7blU!J2yHEE3@sx;7RJ>@*BdhLb=$} zR?`UnKY3|AYCo34JXFk@n~72Lpkg|ubSn)_Sj94cIJ>@@g5dgo*hBxj6|{^412$4md9iDipK=g80dr z+BEc^!`!v>ri)x7MKyZ5Gck=z#Zri{2r^9=LybU>#!Ms{DPr$deeD!PgNhS5GIh&1 zHTAZu4iC<2N6z#BFJH|c0AH5>_7oeD#RU>X=mSn%_wiRukFNk#WPfQm4(hk8VO{^u zu{EXLaXw$yWyoPDNSwM+KOyL+=3ilPyzKAZc~6!`c?Ge+>vXYsOxE!X%aB{!;Zw%j z3C1gLpgL2Z zeuENlnY#WV*tX_kM+0X^6Mf-P+g3=?-fw0@$h7}+wc~uQ?pCC@(eYOKHjBBsdmNER zZR$QlQ-EXh!vL|st^VW?obc%HH7O@s>~Uxwrj%t=cl^}aIn|E+2E|@i($-YB^G|lM z4W@JW`|g+v1b!@VZ&31A|K^#mCUp{RMed$wnj(*-11j0zgJMZ1dpXoqp2>)1Y>#wK z2u*_=i3-T`JX_>y zw0v8xvjF32f@h=t{l&LNt3%@TFMnLB)L(pyI1Rc-UT}}j>E1w7d!+wwZmrgI)w1=# zi@QGz#RXn43rxUl?(AsiU2?k~KHvdfseeFUXGmYGOkbef3#__TescHHJm^@*PByvHFMfuyZ4NC5uZ+> zWi-_eL^#d`^h~}21I2)EGl`;oJr}wdw@p@1ki>0(IwQ2Bc*|Et6Zr1 z{hd=x!*P8c;FI71llFtdsnIk;BQtmUxUVGyX(D=J{q{x#29FFg613>x-TAp zp(`UsrB?}@Tq5Nkbo~Puz}86%zmn}QpzCienE|uk1F(!>x7cQ@3XD_Gu*_Dj&9Qk) zol!uUT}ZUR*r_2G4*PRFm|U`cy#V&H;Av)VmmmZgkpnOqoRxLxu>1n&!A+eOd*{D; zkFnMvuI5Qx-IV(obZw}6+XxGKi+E-q#j#G{(kbK8rm-uZRV~tZKrLC@yX5)1*w*C& zw@zn28zvB5E|{rm?fFCIS-cVIOI?Wr4R&0w!XLMG8a|kwLDF#?-m-r;L#yt$szR>} z7}KU26H;2qP^2?=T;+5sF##gXSw~QWv0phe2c(#|q?9F}wq%lH@Po}@I|nd`%F;}( z85Sl|vyHRJ5bwMXIn!wtU~^^}SlJYIQ6E+cyk?FXo*Gz7?dm1wlULXWzTs{3^}drj za?g>6Y=h@ztUa*zbGFU2*^MIE&4ktscWzO2S~@Y2PF^ORIv+e4rGEcUhFb{3x)fOF zv#`a_;F-B3k4nw)5v#N)fBK;kh05RCtbu#w@Aa$X%zoPbc=@VvR9KdM5l1B2(V*oT z^ol_b4e(--G@G>JxHCd0Wa#>3-&*fM0X)i=t;Z;W)`uEef33vf8UH7%YXHM!Oo{?ph>_rN8^!pz)nL2jfNRQS*( z#fC{nREk4Qvx=R?-9%;J>v4^QEl4(zUwM~wgr&RdJF&}KSNUNxg(L~9>^fHsjdUb! zC#Ba9`j1QnhZhoxyVVRvL=yox0Co6S|Hb|n_-m5>f#O3pPrbT!fyGTN<6$i=U{l~m z#l(7`D^fVVKisMcTIWU8mb;T-rwE|tPAZjlf9vlaXy}6J3~{o>tf}9gTK@9vhs<~Q zMea%bUO->0Pk6Ys<^-i4!?g^-zjbQ>38CJexC|qkdbh6vI4Q&82RBzHs4@yx+Sxik z-rVYwf`n2p;ZFyYW*Ic#&8gNThoui-u2=BMdjJ0F*DGi)G_HzUX`|(*@Tq_j(bQ5Q zt@gWHaEVU_!wd~n;xq^f*;S+T6kahf;UXMGPRj_I6{s+)FX{D|ndiAl&3ZSY#HH0c z>~yYqB3;6a(267NRJE?7Hv4`gh@B=bq9N%s4&w;6D)h zUy|*@5m9LzsaLs~2;lOzYHpFNpPoVb1J?3~0;Fy+vU=NS{S!`A zc=7O&{tUw;0975;m9yEo<7ODJ5f!suY)tMYGg4S8$Hv%*C(soy=6Hs79K%6IO$*E-FEqr#3IVQ0>#Bb#;_ZP(1X8y=D*|Dz_g#^yaFLa0M#fE=a((GSf2N+^YEdDVc^b*^{A<^;&+$qI=ti~P1|IQr(n&pAk7rSwqF^Xk06!9qU+%!WVtjz=_+50@ft}e=2?#$Do^ewx z^|-r*;Q$PL>3T0I@AAJ=DDp1f>it)Wv6QE{7TrR-=)shwqw6FtD|zi48r9hK*8sTQ z<`xMb?jQ!|$!ppa>-hm$MF33KA1JiNuEUe{%$6rbtRmO@9ROPwT)sT_Vf4gWhykYg z1~z-q<0to}F*!@vNs_U~*Zn-%)D`;X3vobH8f3Q0nNYvsewurdPXZ@k;gv7qraa5F zFe)8ip6*2Dvht-%uMZLMe~8j;v0$mHklzQ^rMT=iKb%J(x~7ix+NG@uwsSX|iM7T~ z@|Qc1k&ZjWZ!LEZ&Fk-4i5JV7rD5B&u%9ay;zJ#Z$*p}R4kIg)HolAFeZ<@({>})#52@C5Py5bUSxq`EtQMxM-pAmzd*xxzr{SIHd=o4IBV~|1U z2?e1%c{7wcY&DAi`#+4l%Qt%el_KbqMcC716WZIotkE*9d~&JEynmglQiS_3zQa(I zia@*5BYa9Wzh*4lk4v7Ctyv$NzP`K_JkMH*a=@`b9){o?1(N}O*WKKO3b-c4x8<`V zf}^uNE2f)(KC+yqGSjyb|3M>4zrPl`5>criU(U{+*)7lSks*`2uAji(uzany+B7IB#s9!FS+=igLOj`+&LKdo&W+b;UkihpBI z)CinI%20E-DHEsyMuR#oXvTQyT5SJLdl75=_V3^OrJ4u1U)r-YC079B=#+r-{e1<3 zY8Jy9C*XkUs9CR$4G9c{X`=!LsQ)<|RSqV5K-&=kFt2rGAe~s?hDi@wLoe|^86h*1LLAMZzctYg|);)am+JB2=O0;gZ z$@8#s7ceAK`c4WO51Or|dtgpUlq9}TrNnbN@r#J;SXM93_by<^G z@mokI7_?n-78|TDpmZ_pZw_P0#t# zd;Q1C%Wl{FHzj$!7mlAExH#=EfC6;0$(afTj<)rZ<1lG-jy5GUc$aK5W5w7;CGPA>p7khl-Sr@l?&M z8E_=b9jDI)@+fBb5__de$wr1gww2F{4n!^=MAKDAAH+|RE?jeamR9O3k}^u+>p0LI zUVfafDsvPxHm@EKP8sby4GAkJmqmy=grU{NGNY7QjS?;aq>2m;*~-hqUhk9j0ou?n zxtk?0?sl-F!6fTHEZDUfG`nMXrWs1{_F?;EvWPz}`pl~PJ^OI~ZYJ+8EW@fQgxkCU zW}y+qji~2s=5Gb=Jjs?bIr7bF6HfW>yaIRtfTOm28N8prw;6>AgaDXg9PsPoH|VD} z_M*)2Sf8gLS=oV(pGY|cLj05;o*5e!8~5GpdmhD8$RjQ@K%Z(zxZJ5po>9=_ePRj1|NBAHD~+l0ZcazDG}d zdpdpzrwUP(t^TZKCgE(?N)aEb`+ev|U+fQkbW0X(yu~sY^&RoOaB?>uTEo2Ed#!hP z(JDGDF(0MBU6tb0-mXtv^=^^pQZEmIlXI7o79tk15Z_+*frr5M+3*k++@GWqY<{VF zDQJ>faY1Z3UogdKxVJV$ycwFxJrKOuUJt!Ee&?77^J*lI&?=8~ zG4C{8-UT;pE?PDq{N-fUqlwO)JdWgE*-dgSJrqS(b?k0~yGpp7s=TzOvMhHzc-|Ih zTtcfu&HGoq-osXTB=Yv=_PU|$k{e5AFiF*!NRyheP`aWcXsV^bri3b`hlbX1<0Pny zojBVAIHRXs=9`)(zx}FF2Oz)&9-Kt$1^NsTO?K~kOm|2V58AsDKP61H|-p8UjVX1dXZ>FRWZn@Oe29S87 z_jQ3Zn8!h8Qx~V~zJ5R!t*B=)F|$Z$b!Hw(n?57J>06l*$60KZX6Avov#Q~ieH|P^ zvjT093JhjucA-2=Br9+ETrrjUM%F?b$niqreCBGVj4q8uL+7t4*lkfcO|M5Oc>+JN zXSiss{vk53E$Vq5iyeO=HglK-9k2?+LMYa~a%H@y{bnq-#eeskw47?>CGNF#a-!ikl|~l34yyi-w{i5N($nA<%$6{MpOLR0a*(J-SQ(Xd_is<*jQbUSxe>bMF)iNak_jL=5(REJsc zpRWL$Eqn$sZDhoyt#7D#C$u3|cOmSON;9(=2V&ug|EDSW`<<#gT&TVt+P3reuT}$y zc~CB!qj*-!u?LD|CoBL`Yi*03Ja;4q_7FFnbb&83wY=O%jHYKDAuG@5iF?75mp-}r z%sXMzJ-L?MO15aSld}y<79uq4k0_V|lt(t$qJRDwpZbtSEdJ-M*zOO zUI(CMif1W30e@9`@@q=|^I^yA7g+M5YR_+UlEhB|P&uw&j&}3kTmjBQox@8m=ETYs zwc(Zp6TO%wxpR1{3?`a)E+k;Qx_)Cs9)wkT2Su*t_iH64VE~?{PTz(A*Ay^C9k(hW zQA6=`613~LH;L$w-Y;i2NayzECgHcMUPY}Iw+TA*9I6isYIr3D-_slw_?$a>)E_SE)dekEAmYp!H=>TeX>u^ps?3NBr0w8I-0cE?Bh z!R3ds73gs{l-;T*Rxqsbb(|w?nkRmFbaly1+m4mlNqS@XE$^qjxSqUCVKBz{)Gt}1 z(O~}P0WQyBmj>S{l=kL7YwU0>#A;R6;<8cWo*H(m;#q<_3H@|X$lyswP)-y z*Bxl=2c~6kMV#Mf#}xVxzHzFk3GQZmiq$4Mx|3UG(0&$PdOMo0f51a&0bs9=ekBzQ z4?(HYL>~i*o<3lcrtjXd*i;Q=r~6!2malgI6~g97X2iyhOr#fpsfCXoa%FKJ$^P2h z-;aXpd3~A{l>l{L@B%tad>07JNhu9L5hgOx?E*1HHbQq33|+vQ5MYq~bRRwpF^I``3;uWa{$Wn(vQk zXH|U|6rJ^n$wPXbnT{krIz3Ya#B)NV_KZJa$(%ID4x8s8E%yYNjyXO$3gmvy%;qa! zR;Ny2-LL#7=u5J-ANJM_nIA5{%-2TVUC(2J1;K)M}MF zc+@x9SJ7XL$+4OJxd|4gah1#Y!z69XV*0*C&;7RsUj_}nc?5@8{l;nd6JB=P$ZZ=G zYM`Pslu65m-wuDr$M1x#Q27tm>3fy(ZWSiyI86N^)9glJ$3y&c?yrG+?TSPMnG1v4 zL-@n}x2^snk~E=>UET|l$E1<|@g&n^MC;fuEG+M%X9e8}IY1cU-BKbe(I-|Y3*?eL zAxitP8f29ZD#l46WG;s(kDY(w=ZuS60s_8Zf$rqM>B` zW=Bjw1d;IDb#x!fOzIXiTm7Kh{aB$zy>z%AAkiq~eF@o9qG`BV7@}?XRfA8(ZQ7Jm zww$}N?s3jN6%N`AI}`4a6JLHrqO+AECH+tdv2C$*uB$b6k6w2z z;yZjvgRiaaVY2nF)S^+-=)HHc$=1JKh(cw3!!S+i844MQ-=7wxle71B6oE-F0Gn%O zUC%BSimqaMz#)|))~Wv!sIFf;Y}wFBQ-}?E$~*Z)My&N63(eXs<^+?5&_MT3SCsy5 zP)cs-e}{~L>;U2|3kIaeqW=l8Qhc#g%c6Q?zk_4dwdElWMAMzt1Hz_jGUsMukj{SQ z@KQ5gi@ly@aWC#E!j77!9d*_nwE*WM{yxE=f9ukiF zv<9uf05q-#_ZlUOT<|){RYfb@GB8HJwi->G4zR`(jCkuJnJuD%X`-e)z;IFa zjnM&KCW4i)(K-0elHKWMlRWufA~m_U_T-*60GYvF$>cV`m7~*t4<+}6+%@CBG;(_Me$tZ1`Qon-zN~6MlkV z-{yBUz1XUq_2#G8(=tW9?vLb8$$aH!*pcm@C;%nY0l$)!(ihDW4;4_kGqCl~e`Rvh}7x@tD&jppBiR57&unOnR)kvuwv6ABxaUUgG8$ zi=?9nKLn>(Bp%K%i(5(N%obEbO;(D{wz__WGE%Jq8C#}SzKTtdw#zT9Ib%s0gs8^ONq3-qO>6Qv#D zzUj9=6?gVc^X>jjQ0|-N+5MRi*f%Y(`!hkhkNIZzCnDD?6mJ0CRyZ;}nU%LtU^iT{ z46sfVI?4A5JuQ6j17_%3c7L?VSdViO`w=@#J4-r~i_=8ia1q#-rGq=?~zajqXlx!5N zDQ8mDK8F#9sk~BQeJ%%FHfx6z$Nef^n{l`o|4rbB_Uo)da1|Q6 zHCC(Yiy=Ad2{@l=0=rl&dWl5<{Al(u9wQojj5M>Ati{+=n8uG8DQxueDiS9CJB79VzFbXo&&q??ILZv8^4mHiV^ zExkkJ%rRHg^CpXZPkg8<7Z(f>uucmzXpUV;(yhx?&tn=iC|y0Q0fy#m?t$OMeap{( z3v{N9B!ifLLY>d`G5>LjxH%fUO@Q`C-GRjX-rx|A7?gf2N$>eNmb1AAHJCi6GFs~{ zaqG6Et^1?Uh_rQ`mFTCn|G7z<{(mxQv!Sz9nO>@V|4?eh8~MYdz|f1&rj?R`zd=M>V3TzCsb8aKVbGId%g36D6aFp=M2lPKc#vz|N5TZH5CUuWTX!r^ZHaf!G2P=jzZGXH*j{@77OnOLUN!18G9a93r<@nWTu4YQ0A z8^-5YmsG>K44Qt;$&3tIr7;W~m6ZWWAd7{Qq_e~s`k zg0o=$8bN4edmZ!F2pJ>WTbRE_+%dAfiTP^;6I*dLkT0JT!^?iKQKVe-*p@95;FJH> zA^s7t;72;gqXr0%nFxoz-{p2_I09SoHb2LlHiC?0Dv} zd4t|o6u|2`yoBCK2?a6>eK4kkN!WD$@ks`JKcO;0@-muhl)p|1m)$S4F%68NNj7at zkLI!bA6cYqO84WjGz`?HPW~TR+{-&``pBsU_Rh9Sk!-OX{gC^Dqj&Y|Z-n*uNfcbHQl+io zZO0f(862IVSYnoCA0Wz+<-md6B_7Ece0xQY4O@tQ`!Tm%VrZt!1mF-~^MoL&w+$s> z*xSV=gQ!{O#>GpMm$3pJ$}*#GfJwIWZc@NJMm;sf3FU#iusrMtTuMKcHxUuw*u)SC zOtfmAU5ijzM3{MhB6oZ2 zwdiHV{RHSNq^jE<&m-++H-phTi%#DtCM(yRzM{9h(#M|0(+_Bs0CvGZ_>to8r;#FL z*Y%XltgCQho(En^nweF9I+0Y=i`{Cb@PB_QY1J`i+{7Xv)u(__)y*54lpls}h#T)C zzI^x&zEZ~dFj9Ih|GXY$Qu|G3tv6f&|NmBo;B%8jz2J*gSW$0f(OwCQMDtXop1xso z)d(=m&2zFLT-0J&V6ji$p&?#uc9njVW9iKOr6-SiVy!uv9xk1IbUzF%8Y08!f9w3~ zQ_kGALM87P`7H2JBUp&|ZU@8F9dB;EhK zR~jMmrmAd`Qp12QccbjS6@*A{NsunmO9(ym9zsZ@mlx0d-Wcbech9@u zz1K0`{p0?z=br1g*Is+=wfCA?Ywe6Vu@&F~J;3jg4E6=s@h-rwhs(3)y=Pm7>|%z@ z(hDWWE+xAjbI+bMQVfoA{b$1!<@u0n*w0``J%H1g`Cw0)_i%-0K13Fq0baTba2!(_ z>`C?R`77i%*pu!(RYAyyC}82>B|U)a*zZvonB#8!!!gQ1$@kuk6?FMl(%9dl{)+fT zOSz?NC64ae$30q}l4+j(8UGKVoZ3_vk8U_#B!&~Kl@U+_uaZdQ4K;l=b+yqw@8yE! z0)QIr5&e=2+Og|0XX5@zqoI|Z?ey@uQ!Ih((4_3&cC#rFReerIzO^n z49rw~KuCcUqUVr3hL6?wr2Uc@pB+njy+>*kDB~jZ2h|c`%2HY<7>1x`2IoP({VK3p zo*F@(gxRIE2*AXXwmBbV9%zH~QVP+1;v{eR@V$VGfM+&V{D?W|wW?NDes7=SN<7LTGD4yr;T1YgIGaSE%Trtx0-=6NIqGRSXiv(3fzBR)hzxt%$U|t>~@~uf0RFIguAp? z#lGbVCWrk;`IAQQ5c`%Vm=b15`yb^`8li?)Y-2?9)P4FF*&q@;!yUs>(U{mmp}3lcs>9JPZGvZ3h_m0o50 zT8CW3hvb|p0qm=lRu?7yxj+%}t z!Ik^pVsk-^h(-LfeaI@Rst!3`%ktpjk;er@-HgES@vUXcAN}Wa1n8_beP2y|tm4vv ziAW8y)s5|y_gzaw1Eiv!`UVaLxb!O&Cf42d&l~Umm?FG_no}rBtR^LWlvSWUE_?iw zc{?loK{s|*npf@0)7grVpgW$1sTPIMK(s3L;`4|ca_ zuyf;o1@ONEM}Y@>+XU>=cyD!twVDug%fo~DebJ<;MbY_43^Z$*fCtJr$|{y`$W%AMYR z0`U~$wv`*D8HEg4aPe^7|KZ%IXGeh^3~w~9>$ z`j>gikeW_+UN__LQ(qAT6Y;cF2{u%CLTE6o=d%VrarHZ7NyNm%+DWkYH85-g2t9y= zPzP~x!3_gg;lL;D6_a3ZE1IcIhACZ^eZvKQ_}k2U$7OT>gvjSx5^Y z*5@4V!jumDR{$g5f=&GyNE0B^hZe^=B?kW$V9>V!uRjNA1HAO%#>q{A;lBdN`Z{j* zOhXz0;XY?@^iy2$UjfQ}9q~Q0kXAsv4?9j`${hYH0PGAdD>CzgD4<$1D?S;{{Q_YH zYxA4!ox4sl0ER!RQETQAKl?i(NWHVKNCb3kudoBChkUYSk_<}=M9u#X)IIMc0+QWJ z>pT zXuJ@?y#Ew?2Z}=-J1sxOszq*~kn9hRw2?S8|8OB@?I?zS&>P#}g)jwJ6$R%Ix%}T{ z76r)@;q5@D5Y{@jf}krso6%6$ejld>fqiRi4z#{ zLt*Xh2HgH(nIykoBpZbOv4+PJuzeiaRMxPdjNq`~GQwJ)L4U87DBSu0-VXXec?HZA zG#YRu!9Dk53^ceJQfj@^U`8s%aks7AF^>jeTuajPNVCa>)$3N2S3Y%EA$~s_Q#L?T z^X1tQY559Ryk6_~gOxQMNr7u_rmWCfFrxnt`>>pO#g`oGJw~BC`%&0F zyRi-4XWgwX?OiokU6u#Weq@wn4x&8k{PlsoE2~OYDg{MLUOzH`&Z&gBE1uS+_C4ZN zU9Aa)sgzeVmtOSwO^_@(>=z6hef7WVs{@LIJiV3C;eoJ1w=>>CTgP% zXss|;1n1D$nG+vQw`wq|^*Xw>e}4Z$P@Jk>7PNPOqln;(fTvqa;quX)^<25`uLX-M za#c{D!YT&NxU>Vk)0gG@U9!h-R>wYNnG~th^ZFx@NH>!&b+#RCDT}pd=@faV_b0$+ zw5cr8p01Pq-4zK7v#Kx7e*m*Ncr^3?k6LCgxngNiQ|0C|GuBiY=|qSA6EH;`r$()! zZs3jN*tc{9kn93bqk0zf&9&ulcld zlZ$2D_UWDSDK3)ZmQ}URo})iObSGm9pCrK2yJp>Ow6UhyiE56qjqe3u!lwr3hM7S& z2Spz+?(l8=1Pxpp7}}pzm)FQzTaQx3aM2ZKf`epvPioh?-D0b0Qgm7TKlyjMefmr7 zU2BX;S$&_;#%Pa3OO*0y(uUjZ9D zd%b&T0v0Bcb&|iMkf1VS{KETC>8nTR!aoT#C8Et3OL=XTT0CMFm^;tE`zJxd7hW5s zCXa{(>dv$8ge7j7{gc42VH-A!alldaTl{9!L^QInhu~!z|1v^@rW$t^3x+9sw@@`E zm}URek>b(mqYSWAz@hd9;?Z=m*H6O+_|33MY3#s*egqZbsGikOJ>cKUDe`}TQF@pmwzrrxy;Hw{L2ssF&jaeqPH&! zq=z^km+t*9N@6!DhaEv6{bb~@eNj#Rq;&6ZAeR$^f$ftEF1*FOc#m z3JD3#o6RQ7sW3*%?loK}FcXmw{ytp8Tap0i)N@O$q09_hnno-I&0;q_$jEM!Q%TdU zQ=}}cZl!JsBa>VgqxYx|Q)of0I$EW(CLh>tQl^I$=ze@%^1}G|@@MuETB$pId~~FI3Fe&sQv4zIx_3reht)>O)5%f~?Gwxm*?8vtVk3!cAxt#f3r@+Y9 z;akuAO0}l0_Luc}^v6{k5BsC@UG7vYjAOB>K40q%XlrM)!`IE0zdgGy#`isDD}Bqy z1oHh8w~>~y4l3btk^jBUp&I}DM<$3*swS8xx!jh7`^f{mPbAgJN9gvyx=WvN0PI1W zmr{C!PT#Qo$fJ9szK`6C9-`SRO=a4|+4a@Pjnz8BXp=wVdAS=o617$N1RLRPQeV*R zt!{-cf$JB>uVnr(MChsIfciv#^iJY8(3xhvI=xO(g9kZGtMNN*3@@W%X1?BAcf}=c zyY67Pw6xn4U9Zd0=xMXq(57eSSF8*WA1%J;^6Y|xRj;WrT{=Tgkwvw|USL0I_=Wbf zuDH!h^WI_u`L-(C@h_9~w&AIb(T%SmPOz%#F(B4rdkH$!x7e4dxkwEtiC$#aXxcYZiUbPtn=S?Iz&<%;e&b0xYm z+$d-Lw~{iR_5P2KSq1nGIC89SF#HxbmAsEEoQ#SrgzPjaD(WRNMKT{!$4w?}6lCPS z8CYQVQ{MKo8Fc@a-M;rQwvn#GhdiM(Sc>tPw|#Z*>`t!R9Lp(gaj8p@+h=LCMy9|CNQpP zA|CN^j9R6V+dh6dmTjQh;@y@ahq01;A(v@8stqCr$Iq(WFO|6BjItME@JXGQY8O2; zzj!A^nRjFkmRG8q63EGr!sy~`yx{!Egdd!|p%&m#Mkn7WcUbgV?Yq`>9>A-jUBNJk z1i{GUoQ2?q@{kB~euTfwYGSs!7c(EfDksuD<{;Tca7swSYeXP$SRl~91)wQ43d0;1W|Fm9W3e2%=8{NMt)kU|G#A9s-@ z@f}mLCT-}NuD+JUo-ylX(B{ntZ{OYH3k}sZ;-EC9G2~Cu*?XCMuiA4PCv1acZHvoS zJ8L%murtFxap{!Qg^0###7h@Bnw87x>RRoLf`$VgF^cTk`fI$|>C`tU!Ya9`O~BNx zV$XR>B@05OzCyOXUvhavu@G&k^ic zp+Ax?WDO-K2|q*%4joX=8XBU1$UrKBk?H^|UFDn+de{%3~PH zsR-Z=1hApq_-(Vqi$rRj&lVTyrkOpY$bBLSzLL_TBa6!ag1o28nMq@`>@m~%x|8)) z>~5(rL;RH`V{%jJyXkk7EKF3;{%v$J6oCtt7egt;$j{zmPT-DHXv&!d?%xCE4uiU*|SuFLd?NJFT0`a=E(Y#6E6#>Fx$)mTsz@ z)(mxDo!cn?bg@Tb>@)3Ve#_Mnl(taQd%6p2EvE%N__@2x&{c2PpInuUEmWM=4mj^P ztFoBJEynxpP6F;8Xy()1MYbq+zeMagyC1y1ZXwc~flY&f_wAEJO5PN{fxl98E_QvO z<=J>{=FW{CjlJh0@0jkRnV2zpUk@wTKbT6nLNb};OlyXtG zl3d@cu&3lj87`W~MdeC$Wo`D8rL%i7pM}zu zj0>QOESoHY)@y5LKPzhR)WW`A)t+5`6LG946>w(^89sE?Y5w5NX0z)1*N%_GwtY4D zqK%;Rh&l5wt7+W&9MF`&w^Wcz7Z1+#eD8C`<%fhQ*&uT#WLD1~gcPQ*7fU6JvnR zgsIner=W%uW^?^&Q_S}W> zm8yAg;Z@q+%=h}{V=u#lT69^OM2J+2@LK?(&#kOVQ+^XmtPNok9%c1C{oHcmal(xB zH=W)OW^9jpeq5s#(@gFpJL6rP!@Aeboh zxn8>rch&sY`{68cO0AQp-vp)%B_*%ibZ&}R@n|;Wxb&VU>Uy!rJ&kAQjL(wXk~_$=Q9thdAG zeBFmslhJoK2{f%~gGLf`DCgcc#ymRa9SiBjS;{W+g#_e701J{nqa1p0B&R+(GvKjqOH6FW%EP6+NK zhvUnP&$KIF@ots?y7t6+`6XnO$I*`C#m#f|vz{+($?9$7ydAl=KZL22a%{Iz(Py8! zd~l;#sfIxrsm*!zCZ39ZTkFNb#}?HQjsu&Jp{o(%%(wW;I6H&a6Q!gRDDL%k*Cq#&)eiu%U8X`$ot)9;#G4k%Plv@jiy%i{?A*>UeO>bvaNWFNTAFsvH@VU`-5uVk3~w#L;WP8}dgNQ*(}}ZZCSsmHZgf;6OP^z$h<%drorW4J&^CR8H#VnF zfPg=(->6~ScvI}^9$ep^yR}) z0lZ3Zo53z+XK7sb^*4ixkZ8Ocf1~hdQ_kbEw=^PP8>)=hbUeN39D7CCOR)=^9OI?W zjyz3=UmdwQm#;)PnKy4I(lv!IQ$SC`oBp4MP zSAEkFWl`BD=e9Ji7Xfw;EsIu8uOT7_lB6Nd`99Z^Y45&05>Xo=Hi*V!CL24__C74! zsW852d+IG>{K43Sjxk%`>84X>pMMNJFId{I%khGiyt2!_a^xzCGQyU+l{xM_tLCMs zd$t;}vB7tVc1*?s-*svM}s76hw8m{-E`d5#?Nc{{)7cHbS) zzXZk&nTnXa>v1qtx(J$}p0{%Npk*@IQfPYq1MTBebvMie&q*APQ<)^v)}RV)VrkcO zqf?p1eE82wpS~$2WO(*|EVZaD>zzUI<|#9im6xb%6v|iHdqh}FMO>9kuQE$1j^2nD zsl9eZh?z}#|3NK$(q^$ZSNfU>iUK??^HyKnaiM!QGhh4E$|HHH_&PHyNT;kZx!w&9 zW)m<{q#K>PmnH}DOvg-LeXB90FM8$VD(XxTd``@yIN@3+Es9uyfVVLyuozYuu1%(m z%@*IEZ8eeJFq+eP;s*y@m#eyI@4VwJQ!swmSCiSRiva2)U?w;)$|u<$mB{oG7rP@z z{5bGXgx0SUWGc=Qf5$!>Xzs4SOC!Hu_)-gX{1O{^_q!J@AGkaTC=&3d3QipD0#R>GURonuReaxnO>{JsU#?IUtT(X63ih> zK_N^bn$$g05#LDO6om literal 0 HcmV?d00001 diff --git a/docs/static/fonts/LatoLatin-BlackItalic.woff2 b/docs/static/fonts/LatoLatin-BlackItalic.woff2 new file mode 100644 index 0000000000000000000000000000000000000000..e9862e69094b4e51bcda9cc1727287dd5abb6935 GIT binary patch literal 44316 zcmZ^~V~{Q|(>8dIZQHhO+qQMaJY(CoZQHhO+jGX+=Xv-2wraO(uRp2kN+s#;E8R(@ z|B9Q%*_5B=AHvdriRWWh%5mirhNdL}g2=b#!VU(Se~U^+`q<^^LUJ4=VjeV?Iq)VES} zE5P~ND>k7d*`eHkW;<1?colQwy3Q9r>pvCD z%&VdUcKcx5)oA`rS`lDE(Q!1Xw&A!bjGx!&y*cWJHG!pIVI}b7l>y=t22aInyia$l z2{Q|2kkgmnP4j*jG9CbBduoDv)X;z(V}((h52oFQ2-O~(Mm$E?3yXp(((nP&3e$}c1@qObg~I47S^ z>711F8zJsN@2erE`(;1qe2WH&GP>?~KZ!px`wJH8?2=%`=%*p;?%)=%BpDdNehsWm zKBMIw>XSp6Dzv5G-hQ)a*iA8w9DE>%c3c4LvK{_jP?EzanKT@+-GELw+Z}7Pd_* zzi_cZI+P8Vq@=H4ccN#u5KwI=7KvW_0o~~Qk2{P+VaeINW2yTXE+1;+B;ZG$2#BPM zh|&t3shjn)-3Y^HbAA@jXWKez75Tz#XtJ;H_z6}D${0%)y8LuOqzp-=-&s0*gpj8cX_QZ-}b|7TIe!? z9ckx8%y5t))P)5|bG*M>muE-6Vm`LqIuqg-=2tx8@VM1*hb$m3xbH3i6J-2TlB5frDw4wCI=+#=y<5vfGZ+_-iErqh#r->L$ z=Z7dJbkv9;JKuffD$7f1VG$7hfQ}j(XF_UXVB$toC}?684xLisab+RXcJxzuI*pBH zC$`dlf4m_p6mYdD zW*DnZM)rMh-^-m=de;XIkSZOv!K?C77PQDizj%Emn16=;*%f1o;H!c-R&5oL|06i+ z%dd0)3-N==fMA)uEKWU_pch3xpbc*<(|qdL1hD=n)lgP0f?}D_jc}RNIkl(yZRK-T zA;ag|$m3Y&A1}&J&4hSsVX3#me;TN9n0s$6_D*FQ#)vJGe2MmbzvWr*Nd3lE))ge8RKWEj6&YAdhRcj%bT9LVoZ2GnU{5`!k7aN*KD9pA$ztEi~>H3xKp zSZ3xL7jL;)u|*Kg*_@d6;0J6ne}PjCCixc#(A)gA8r=b6-pLH2*G}p=CFAz`OMi?; zQg0|*m-=@0d!XQ!C)-AhKpF^OYxaZQ_fwJkWVF(Zz__o^fz??Zd8u})Gm|aqRYYA( zSoAm+FI6mIBhd9M|3>_04phh9;4AvRMN@pzMsXuk54ypsrXm(3l8#auB|>q`FJW11 zl~v5KC)p2`4`9b&X=Y{aQv0{!J%~C>7`pZk^$&|RGb)Br((C?rQkl(@c$}wR?SoDws2q z+vjKC=xLscf$;a7M*08-hb((?a>K5hu#86qpb_wG?1oOi?Vt0l_^XlCQb@*F=+<7B z{nC8r{tdk%Wr2>IXf#6ghS9EWKoVn``LG`DB{Jt%SxJ(Zus8tg7FC(nD~B{17BGV` zEXm#jKF^#;u?O65OluFoedsB(GgB&tc=Z|(8SA!k-}dV4)Yc6o0|Nf(n!JRDDd#Mq zCYr^WXV!AYscgpr>z*CZN&lDZAsEH+;QD&;V4cGz%RXl2*cYale^No8x~NPiaMse| zQcDUt_da%1By zTrKh-<9L-mL_*8i^{iRz4fN!kF@N%ca77lx@HU;P zCp1NvwY{TN#0?X3K!)DvmJRRZAS^8(Cn3XxNLIA1qc|-uV>kQrhhUK+LbMlP-g^;< zZZ)vx<13LhnN3bcL8Jv0`x1=2_!5e`QMjtPm6&e%629)`je4Gujz;1MnWc%IiAJS~ zgNujngu>F2h~oE7xrEQkHn*VKgc>8N*eADTN6S z1M!t8(O6blKe1N?pE;N5#0DRamP=~)^^QpYWjjfEtz}(zX;)-EU7Ha2#4`&iY z7tw}1xkJlR_?LEHxg!>gR+Y8X%`XudOM~doEoyUhJYBwqr1ToXtIkNVxHK`_PZrT( zjHG4P4dcYX@9$7vtxy1(<2y5+O~ABSo%`755&Lhpx-M@j7_Olq3~%hc&y^?v$cP6& zcfnz=h0MD@h=*NHtZpT(nU^3+z(3#$+QvzF(0EXE%&7j~tw%IJEJ6`=s(Nt^hLdxv z`e)sGwu>q;t1ch*7JW zGg8LhMbdFIM6K;bpOjzagT4%P&pz`e&m~ZMNC_nIouc$BKd~x`;If}jZPkh?Xdnub z`m*&YZohW$gBg1mWCQ;Wj+4-Z8AlFW2|UQh1Z1d)!X+KjxlEiM`8X|sO7v_g8*B{g5aj1Zk-Rxb$wl!W zk&z(>L=uVM#|ugK;JC0lss9j)l2PTXUxN?fSwcppB`gSmiV7SyA0Ci>(|BLJaH- z3l8Z;I+r8LrQir@=^y@exD6j?ocs+In|Cv}#Jyl27f+0XVAv$vD`9ylI+DWj>1ATZ zwgu62$w}Xs4~C~T^))eST~M@|7|LvKsHw$b#d(btWait#8QhDD%CNUYxX!~G=pSt0 zk>6x`T%_DOVh{B?Tqv>2I>XdI;7di=E6{nyF@ZYhYYv9&ybt(sAVbJZ3P2$hW1$^` zDI}$-6t7xZ3t&Pq2?D@wF2F5K-mEl*l%baV$OTD^$1^>mnHphlHY3u0OKWs+*KX{tJf8g35kt@g0Q?tQEO%O6GV_>y z$a!sMy&WZpRmxepgElr>)foOL88d3xjvhR|0T)WY3FFgY@k*24q!r$%nRo6^ zI2@ zN2Q1=-zQ$AwToDhl}DadzqTg-*+>6c^d_|T7DLqnT3ut-_tPWA<{8MM5cURlzlyK$ zKJ)Ti*U)-$i$!q=tjfy)g&&*w6ho=Kumy3dSwdcycT1>yO9rWmp%2p6BsROpfm%MHY8=Mx6fd z%I6Tl&d0;%9m2=2166@sVmN^)awTZ6vgUpzl2D~+K^UB>I$JEI92){6=eN*A2oVssbH0T?=O2x00D* zrg668j;q{Hd=E`&DziF7Zvmo=Qgb?FTR85^$NB&p_ZSx}944%z<@vvP&DGIC~#RS9Dt5-bWg8=t2!Z$EYM`$`EuQJWJl{Zg}^!Yu4avH4B&j zz>6mSG;ChOmrdabA_+4F8heD#kco|{T_p&!q9J&Wno$nD34EF3CSLv##!x0z3bwFpQ>B`uwB*K$+6Y4B*c1-UY}$~ zCX*Kd$uJkU&ARLPoF&;UKSim*1p?2;p^Mdzjl#*D^e3KjTm}*VnX6bw5SKjDHe-;b z=wVL*6~h)}YBaH9X#2E?tMPSu+d&fO+l5X{W+m|ZrJDOg!)y_qUf<5?0Pxjn^Hf>mC*S9 zf|V|&r(ITPRU_@q#k(;{A!p370;kDLrDqqx$t<55wwPN`7TVFf%+v~WI7YkwX1@F1iw;l&(3j8Wq%ayg{zR@PiXLU2^_~eyWX?Wn8JkQXu?!f^x-P zxw)?A};4;d2J5@{y8+;=+Y7p zQ?Y&u;eB58#<|4*aIyFMx2AcT`JSH_Q|Qq<`&XV667XL?a|$W54moubN;ei-H*@z4 zSTgJUd6{zOUE*18t$%rV=XQ&8A!=mu$3>OVldN()D((vo784zj&1A+SxVz^%6Y200 zu3xO46Zv78=E!`CJBci@iEwNcuctW^F&pw_1*q z)SQ61T4t%F9 zMEz_l3HGCuTT|5(HeD}O>JJg8&R`a@!9KI;?6^?r>T{=Fgm5CND)UXb%*$m4LJK}u zmnsAdL=RN4m4*x@jL3`_Y7c|Ff~i*zuSx^-J(}NNW7qTE2$%|oj`9VEU5wr>j@IOG zIp`}{am|nt!2>?d*Akrl5c=dWOiMO9;qCm~{Ku92KK4j-W{3f1(I-b$;^@&~=vH6c zZP#49#^)mxq+ngKD>~X{<`wvoC*6?9n>{^@E4Iqn^qkSKh%OmiASG2jBG>F`tw(L@ zKLyQsm0k5&%f+yFS}MjnA6TWl;Hedxo|Tl5Kl47wNO&kMSS=LKCF9Kqn_>KsaU#zV z3Ie}pElVsckq)^pFBwv;K%1&IO`bk&=dp)4-j>6}D*MXZ#Y`?xcp~IIE|#88H*VqD z!-;Sbg|-)^bN;9H!y-qb8~TgwUtzQ0*ujRNBMyPZmZ2ve|IHFBmad*b3#P5h&+F zvA>6Ix&LPfO_EQ-Zuubq=Jh(rm%b} zIs0N4hy(ioNcp-5Cb4QL38?Y8Q<)g11icIWixAaTH+eTF$(dWU1>Tk!3f@vw=8bTY z4(7!@qDqW+i1D6^yzEH{7?r}?ZzCEC#NU_E5Xjs(;Z^6aSi zJG|RBIpU3hXagMh|HoSE*GK(4++1DlogH2txi5BK0Pl-lg40IYAU{W6X-6}8STGcL zcpxlOs&ZoohlhKrigGJUvx!m|^~fzX7=O`XoKiEMfsc?zhr;-WoVO#6pSNGyjsJDU z|Jo5L9u;q{v+mR<)oq`OIy1zGqu)u~yZ%N+_XqpoQ@U`Hl>2(u5Eio{#+BC|W!{_z zFJxSnl9@Xeiu30;f{00x9)?5hzj)fZRqFNOF@yjJCX!l3BmEaPH+Xvg^P0dI?O1GW z@Wa}NShYgmCSQDYFs8E^VK^<|L)mrlQo>^YT)O9dYhB;M>}s}w$g*UDVCag59NgOlHtw9MA|KILIKHl&8pJ6#|huWdQj=W=5VDJs+>cVYDH~)b8XGwAk zx*a+q>Rrt+JokijL;6lTaTc1|?HSXneiPDUMMX`q8a8H8pQ?EOen{W zJZ9BsLJ6rk3GME;UFUodc>j$b9Ti~7+1awsS$2m{$w}*Im?JhhJwiuHPgYuAVrOb^ znw%Y4Eo+KO0;q|180og4*h!N^svxmKG0Ru~-warwqvFoBHJJH)>I~`tB)N8LL`bN! zG7vnV2(SydZrk8n2yGTtlZ63;1=aVzBTCgu3AtlorXsj&-n64+w1rGd6u7Z5LJXb! z--n~~gt3DoP^?uml)SB(_>?3KMO9^WHI=pb<;B&DJWmbb(*H58HxT3iWMsoS?(JIS z*B6sF%HAF-=jFQUwX?%t?*K(qG!>iw&1co@T|*a65c`CnB$T0H<1&%G4|gSSssA*H zv6ldxHhFF$p*3vQf#a+ZzVTo1Kz2W@b!v8ZU}55;)mYbK1ide_ zb(yvgAz~O4lQs!o} zX)4aBWz>a7AIE?gNnDCzWolWgC?2wgT_bPLm^qDJHFN9GxefF`#2kW3EIdm3eDt{{ zNNxPCM4jec6_)%T-gwsDh5pM?B*bLoB$nhy_592L1`dH1OMdtdHHbi{mjC>*s))1h z&%lDuoN`{{d*!>ICy5)9fU<^MYyJs|XK3JBmNor<^x=7pkoRM5SMl*vyVfqZv9z|j z60N(dwx1_x`QNPqj`Dxh`R*Uj37cj@yK1ln0Jg%(pa6h;N=1^`$}IO+j;um&1{h@My7rT*&PY~QSXnyr9(SRI zV=pD1N)aX8Ll1_%BTo^@WlXtnjWpIK+rvggq|G@~IVWPQk5RVba3A z#8Qtz&+%@nr=+)LrH@FarPNikgW^iQ@Nf$^>VGX|ZHc#^BdR zb9;3=S0=%C&LnYXi)UxEtCIEW7prh}42dJL$^0nE#O~QS@re#BE*`8>E4{L=vK?I2 zIcERe9OXUq7L@mzrM6G~4G-QWB2X9j7yu`OdpKh$1h9nfKwG_${VNB?)>0YDxP)`j z9kJqJFP;%6@a4T%d#PI@onpv{2@4)HVCeZDs8zbD#dXlIWeArnU|Es>bujl10y ze9QYduq34=rbdsK_bXv6x0_D;4$8N`Qr>3cS#)^KyYu$kYIW{Ib-t;`o4;Nx*3dHORjUF zP}5f8f}0{m*VgsITmf!Gg98+(Af&h=EQMsGxn?YesFb>DjCvUdrkuIVH8Un4+-0cw z4+P4j$FLBZ^#`JM2k^TI{7p@%^;}Mf)RfG8IQO4fGf~7^hS#shcKV^J?MEx z+6j*ZLF?zD9|5n8*M$75_rULan_^VLP#u1*{%*Is00R19B^YOj|9`>*{QrhZsmNH1`xCzJFe!T?NH0~+{qtGb#A(Ug6&j`YT57}!5l89q z$JkO)@Iy9MK<>;H$f-7gq;Q$wQ-OdfQAD$}m^xOf(ln&L3pR2;NI<~58|m}Q$&_u1 zy$L)R!u3)2TwG?*`=1!jdo+<)fk6sVzcPoVvol!43z(ohqSSG^F5L8o`Lr7vvVR%0 zy%gV)zBXSaKQTXLAa5F+8$i%ph8^C2`zQSOgc5$Q$c_UtaS+a0HlVm6x$(l`{=k2s z;lj5DVfcqJ;BXXeEb#XOvVo)fu?TIV#|e*z#6Y7V6;Ng0V>U$)x*qBi%ZbH^Rpe@6 zKO_>?{RoN=T|uk%%v}CKpyF2MI0@2|9+I+0OwR{l$uZBxJ1f1s!9@TIq!cM?nXqD| zYrg6DOmtc<_Z-?t)vk@P%}jz1E7j}vrMXq&R~0?WkYhDa@GWW=lPj1GMCaucI5;@LyBvfrZfW z8X0aY_CZe{4YN%`$Al?NnqcW1nI;D@yB~}WS&}q?kt0IvE?C7)GmaD!7_psA^}-7& zBma{E)ytCC5frA^mmZ@BunEZd^F(38v8YfDYo)PNpF0_WTV{h%EE=*dl?RzdxDPie zm%8#x;63>o_#1V)|8v{KqXNd_c!Y=<3h1hrX=IXtpT{*~ceAJHKkKg$*qZ)q9Q(v) zE8B5*>1o;3F;bwoXA$tM-;P=6ipqG5n*odKU?LzxDnvihbH{va>1+=z_~bqE{%tMT z;{cj(L+#O`k{>B7G|D$BG~x=RLn>>-?+^Wno|<(pgpyu$)SXM@pC6~@gbkoyBnUJf(X?mH(WRK1e5KW=$!Q1^#?QZ* z7&6cj45{#!xU_Hg{-1x$SrjScV1y$>3qC${B59938CK&jTf|pZZ^PMy1jIXqRAO)z zOgEf7;^1*K!Y0Dz6dfqY3-fuZ2{&l4EYD-F!5f+e3db^*Z6J?zvr1n5*RBuVU9`vV08B~hN5?#*?+1&u zz^m36qPE!V%ruCgYdqsBvFP{bmzwyCjk7_sJ}Mj$jT#3lDJp~<+0l*dZTagG+!_+B_y^WFF#ApH^(8*?9vcX} zi2WN$={jNqcgcsJ(=vUb@7Wv}-#0Y`dk3$p zwP)hgmY^pa*)7bkluRMBZ|m(2u{x5TfWOlL ztLpGe+v=|N*Dm@8-V@1UDL|C7;L9ArF#N{TOj}5ahtVjfp+{j49t(wYu-l=5xJ;x_ z_*dRR?T%8gxR9gQ9RzGp`0stgY?MJ{L7pfl7ZHeaI=m4>vUR=BIR+SXf+MGkJd=tl z(kIdDuRXp^z^~DcG?dbB6_v2zP!EzZkrz<3zVI`0z7;STG`RTRRDQckIB9+GLd!Xe z&wgLFHf$gIG&wjUl2`M-CX&=I`{qVk(n_-8%8}iaN!^u?uSn|Wt1|Z0foyE)4))bx znse?==g^+Y(Hi9_ody5$bQE_xUn<{|Jts}h80&vc+7L`v8|S=@wzQAYYw3hn*r_cB zGj*02amm#a9C0BOB^#3bxpQF~3UWrHtbPo;J+Sn{{??|4J2kLwyKESfE$^FdcNX*c zUe4wB0WxxPT5ku$J1`8yODA3S?K0t=kuTQrDrjhIPj&QH0iQvj? zty?PpFztfW)O?V2_u5lZ)LgeRnFwp0VyW|3ZwX4wFgh^i2n$PiY_P-h0;$jhYY9CP z?OalE9V9~OEMsyw^1|!2^U;v=1NjVonp4^_208r%kjR(E^gC4R3bWltXj{d0Cw7TLm$Pgi)1VS*d?5$<&q_E3_ zF*6#)*#9V`+eyRkJ+h_=SQr2fj%k^FaHH{xNv6vaR-S972X*L z^}F=ARGX2jRsHov>B-@k^~5-mvLZOaz(M4%rUg8D7rax6s1=9=^gLyl<3wjJSrV-Q zTm5xw=y2v}0q07^@A0bs`E$7#3fXH})I=DN)nyep@Ei{k=ipSZ&5J4A|EL|S5Z{W( zJy=x>sp%ajvXmNF!T?fC3;x*DJI}H_osggoI=aclA_Og)z;qPI1x;DI8i?bb$}3p( zQtk0*Jd?$M$w*Hw^UxgIoy8-4WJlLi6K_qm^6b753KyPESo3&rxLh-~ z6-9pM0=8Pl9s3y*ej{AC+F&8FLYs;=M}4mHqE%egvj(N@wSR+@?%PdjuD+*{`I1n4 z6fYr3N<%Y4Yi%51qmf4tQ{B=W0b* z=b)-3O%DUP>CTu0*Rzu;mTwEGU0)$r<&FoXAucCw*@9m5ha0Ousy3gg#2tk*snF*u zE+5b63vukn&r%eVll;#CQosl(@#c9DU>Ff5di3ETKY%(lpAmn#DG=UY(}b(lz&q9H zbj0Ls=&}hwEdX<{SG>wx1oY$8>|}G^EF}{zeR&gL0 ziKTd)vOci@IwOGyjcYO92J4Y|;1AY#8!!#6z3P?=b%k3pSw1n6npC-zxZb2tC?H=* zNYIZP^SmB2|HwE%UK3h%0yA8u1Vj&=dOoGm|8|^uK8HB~TOByhBOlHq@nG*D326@r zI~-IVLmX&H>{nR)G$6wj(9{H3ooQyM;K$(bOEx%M!Lw{%bh9*HFZHxkCeJBXk{inN z_Ierp2X1^}eO%6oS_z_o0@8~o+K0~2FU%pIga$$lB*km>O?~Bj-g;$%jp)I-kz*z6 zM()*oF&jsCuH`IJRWO&n%Vi$)`pjMl$m4sP^c(8u07H=8WFo8;OZxn8zYGIJ*O`CX zio+PjVxqr#zn}c}>95CqcgUzcL_auyF|EE1km-j7@G*ex5C8@Ev%mF~*wT+9` zA(1JRY6VhJ>b9G%oKnKKynws;O!*#m$x*dJ%?c+o>ZeOzS8Ub`DOLM#TPv@v&4pSw zx9TE1h0A0byY`FdqFHM94YK;%`krE{o2Itizjx|spNC<0FYlE}^)i3Q6%eSt_OlH1 zT^qEV#r@5WZQ;uLiNa#C7_FzB1wK`M(b7#Ed_;F`Pli4GB7r_!}yum{&Sze2Mz zd?qf$upniWx&x^nGGGoZb6EhuG8^LSS7_f);0B)l>+V$`$#0Y%(AT#c(f2E!Y;cdj z+_fh5O^=-seo1nBZv_f@rwj1?{M8(_AN;B3_C5o?uL|=ZcoXquR0X_e5cL)A(H?Vn z|2o~~jh($`{u)4Mz)ajBHy!#ue7*m$ymIzaet30XrUuOh+jcHhBcSWDpG#j8k~R{? z@?T+EYctx=+a52xk8+UEP}~Vsa~=Wr$-|tlSIgo;fyQH*j8e7q>YJF!1Ska-+EUN18vlWZw^gHZOS4w6dJ*?;EO|+h-!@2oamNasbn!20OvQ;c zszy^IE{_}%ABkw3IOL%koAcXcpE=>OZE=1?Q>1nhB2u%NWJca8$L}eqf5ZFuBQHp_ zMn#hoL4eAqtG|mS+}p)@)?I|~PVxK1;7516=YK?gcs_inIcu6b2y4Iky|w{z&ai6e z?`)#;R!4pJ$*)yq;IScGXDN<&vku{Pvw~^(3GyvkZYd$y0K_NX=yr`qZ z04MZEhdWAE9x=#W9lW8xOB0x~*wg=teGNIQ&O)!N8<;ZPMu{ChS{b%|2BHL%5;OtK z#Fj?0cd|ZnpEaVE)({*sU3PcV%`IM4hDDr)EIkTt*+-9xt@O(*f`6SYWm*61W3q*V zOvegIo&T6(m1@|*maZQdqE2?68RoQ+rf34x(mH;Ch!YfL6$DsxhO zkFFVc8hpzdtOtOgT$)Gzs`^RoWznnq0tA%PT-#na3!?8&b#RbW>u|N8is&@VNAw2n zjp*xaXcZU6YtPnQjZFjL!KT$8S9a%QA^zlqZuCb07>L&=C$G1rp7S@GVAB^7@AO^& zE=w<&7UV#U7?vyYf2}(@=R|t})xpvpgJF~gJ1}ZHuLv`R|tFKe97>byAbGHWwVRof;3AOt4eh*l&3^D-_s`D-ty?eYan|;= z+-|cRO#MvjNb4s1mL1QZGMnVyG#pVVmiaTahcff)jtm;z^nat9I-_oExltVNGc2t(-cOzTch*LY79eA9`_N$14Bk}*O2;` zv1^^#i`<>-p+-+GMrb(giH}v#`W@J4qFhX1Fjk~v@RlfIZYrBI)D~dYN(x^55*)b# zs<;a+bj2lAL@ho~E=Ayaa1#O5CTe+oF3e9gZ1MDTrzE+D1rkg-OgAO0OO?9rDS?TB zpMb0=tCu!uX>t$iKLfmZt<{Xz5=oW)m=|i0SF8Ddw#XCeJ3w^G^5Wkn&5>nIWPXi6 zFwzipRB%tVUB|QoYtYwXAZ=`skbwe;Y&lT?wQm|Ix&Q(VDB2|h1}>wl;jz%HfCp+M zeU%5xI02s5wm)s5mZb#yjav3rVW}ZS+|u$GEue=A=P!h9fk}JQfEicaojK z)ef#=Pi+p*)FD*5*6x$6>Vn~21{w`uno9V4>H+EL`x}d1=;Xk{KWvm^v_UZh0loB(V*GA7M9rON zH4Kk#bLaUgY3u!UFS1vlBKOi<$XL()queLtF7YWO(Rjl%`R(93$@Q&_VssKv=@3`y z!G$4o_Kmjb!?6A>TUAXRgvW3s^WF?p>A;41!9oFws*=OFWGaB_NK{C(5YbsBCb0fY z$^b<*$z@!&|A^zzAzed6=N1^j`Y$m9=2a!9vx`bFZuU6W{Ag#jKi4~&*t*=3JvVUi zo;;0mCsg#JN;=6ti$DDn;xrVVr&BLeXfOB-PtR6Ve40u=$Yj!fdIbKGzE59(F@e#) zW^wnhwk3IC@<6sb^AkY#QP~=#WQO)3U|Kmm<;~t3K-Aay(w%A@(_K&*sIK(~@K)nY zVw0YYVpBP!c202cNW2Dg5sAb&&9mz3r=5GmqIz_AwjS?u7m7Nl|2i>ZCbpQ`f!3ow zu3X&ca?t2t?+ORws)&WJrj<*xRjP*9*bXAAjyzb-tCEFXZup;M*d>CM8`sbZe_mrb z4YqMo#!)ftG7txN@W@=`Tw}U3?G7_KH9Fd^?4O3gD_gqtsIWC?gPTM(Yp~v~Ux)|9}kgbwih%4)}kFgnf+h#lE z$-+oTF>saNq$4!-y>4K0usWOcIeMdX-n<;V9$dbMpSZY?F)myIqha^Q2Bb9Tu0ofV zl$=sl8aEbJGN{^H-94cRs2|-_!7Ts#?gM?kKxcKVK=(m!kcL`N)AD8#LnmWq9+|Gq zCk%A8p*V$S?5;AbVFE6WD6pSisZ>rI)Bh&%897F7NP55RQk~UZ7ri!rA`XWyeAfqY z<1ASSuoo|;sq^jP6ZQwFs{}Yp5{9Rc8?+unHbe#43B^CQJxu?ntZi!Cb@d^xi`V)o zSC&d+oDLt0wPU`rqBfnxEG;}a!SqR)=A5_;tGaPQIktv&F@dizJ`;>bY0s2UOqDeQ zHiO(2O7eU0L&V}8j5Ko=??tV+DZ!iJYG^;E4A*3o`!vX29V zOjKFoD}$^U|ClI%tMwh;OL@a>70>`V5Kmc!4cUNMb4Y1$VKmG%Pa9v?!RviB%#Ik) zqyJ|m;U&>Q!$J!7GGRuUnoobNh0p(lD6lZ$-Wv(DX1~utPyqO@~L1@sl$VoL#^S%@t*goq*SE?XrhE!HZ=gve+s~f+b0bH zM^l)R0th!SL8F#CkuAl9c?1AiFcKJ=@{sC;ccy;1IQVA1KTz7A8wlbRKcQ}<;<Y zRJh;_Q`z3tn_4!NdJ*5iNPjema(fA1fkq>XuqBLrTHs!bv=)fOqe+CZ<6n&8A0#*I zdh`|8zgs(%1h(j$N~1rBw*$FB7pPkLvQ#{gG;z-XuD%>Z?wB16&_KN{sm6rkzxirYhB?SU^ z=q*(PRs+$lP0Ci(e5uoK$t4|a1eOmXB9mdFyN92r_IwN zFsUnJ>L6MLG@QY|>0>YR9^wTyjfK=h69JRS>W)Y)3&k2rVkB||1SRX}n0ehPq|L$v z251MSzJ}9+=%o;d=kc&bx&|_gbWozm*ojwF1?QUM< z`Vnnr`Dh7@zUPH)Pxcu!;qWJ=TTihWL0?C%aia_k%n% z=-l?U=^n~RRVr}751i+tH2&*xO5g+uNHgDi#!)8LQoiK;6=7WM5mK-Sb%SI#&_Id* z;K_=}Z3wLi#6gi?7VJ-m#464Cj8t-B8`|-`p=RKfN1vnRP+m>M_(%p&*_XsS$!JMt zTZV5JKr-bzc9b?ppDd3uGcq?5utSi)#qX{}73>48I}sj64wKHx8mj*M!;a;1iZ2tb z)wR4i>8eohN``Y$N96a8YMxt;m_(+{4g!2UnA)Pen>XLM+YnA7@j~HnDxzxEo!bJU ztDOhXv>1u_!eIkI)RHQR~bfl0*j+>^%Up35QPjUAQfT@ppra^6_Ua4 zoBySJrv}_MEtcakx8$*UtJb0GAS)_8+5-zRo1h8>&Z7#-yV2exhAld>l=*A6a|ocE zeaO?{c9@^zDsdd2F-d)$%c2Z8!RZN1o}!294e-g>xr%W|TEG|I8E6FeFQ-%;p{*3-)=nOj_wH_PP3a{>j$Nfxc zc7fqu8&IA11Vq4|9etH&kl(u3Qavn#qXw%X$5j^mOQ)F^P)N6IVjlU44#0s?@iAYn zHK51cGy!M{jbKlcDAF;nB`Z_3Wf|!p_{sBTWA3=X+R+TU0w3{m`v%{6dPpWft|LR}Q^fxfKl+P2P2kRsA8Wxn zb89oIX{10jh#S(=iEZIhlF6iIzZVGbj^A;sIKRrj(qVG!F= zt}2SLn{TNwA$LcON}hh_{$vnrh5WZ92J!($6&i9c955rT1vVR_jALtj{; zYuD-j??!L)yC0C1&r*%EkAmr2pQTHjl@io zOyua|(gTFU<(l%CGGCDsVw6L)p*-u&+ru^l)lea@1iSzu*AZ52LLRx9S}ILh7lAgF zqmzX_5Wh*C^Sw+Kkuov>-QqvLT8tK6AC+t^*?Q$~*y_A9!55XCFT0{wbep(p?w2Um zqba7choVV+wYG3Da-$Z}XsaEi8JJa=BwMXDbv|Q7J!$nqy^vfp#t~Tcyah+61SIk& z+X@BYT+QmMMU1y>RS7EMv?TG=!WAbc7%|TV#$;$k8_T=GA3>BITQ%BhKa$gY%DRC__;%%PL(N%0WPU6-JNT7I zGtVJ369w;AFbP?A;v2M=gfJ7LeEwcb22!%ugLZZ^fgyl6Pf;iO7N(Q?$q5+Mz2MF( zEyX|t2&rP}Mx)RQsg%i@D#rBjF#RDz15>@A7}tsuH8d*b>f3I1)HI#eBwmp<057W zMe{P@OnzA4Ry>%kk0psXu%3%Z8!FaUMYX9e_oLxqk0$o^9KMke zX(AXNQl{Esm^|syQ$S>{e@KCe73|#2=Jr9kcA$MeI+vP0ifvTFGw<%?I58MldG)I7 zgbt9SX_Yit=;@3yVmQqWW@WsDMinyJ2y=uqRBdp?c$q{;y;SC`(Sg8@K&(ul@Gf)3{`syI+nXleI z4P_QkrM`HADO|M?Lq{y{Y{-fH-l2^#nw-&6y3bbK;%v&lTEbZ3R&bxco6+owZr`K# z$gmSpCZ=1iDr%4<{Wdx0K_xgmzp39u582*9P=^GjOwCr)4VE{eB;CLz!B}v+yugIP z1HkxTJB>bsaY=XcPIyEZktN9q#s%UnI*!jm*_tPY>x0Id6dhzRKNJKA#BhdetnPPX z`rSSp6e#!0dL?CxD2=Nad;C5(yw+HGw&_>6F@GYc+yza6X;2R6eJbw;tLPIYPrYw2 zZ{Dk3sv4LbefWJ~U~hPxR$fvyAF7fL+NW^75y16gP!{#ymwO?p@j8Clc)(@}FAEG0 z@L9jP0om3S0(h%fgacf(%{#i_k^b=%7y>ObE6*wLw)Wz9T^`5A~P@4UdKH=76m^x>7K@AE;WGKOsdH&V4K) zasu2}Wg+sqV9bO%G^gDgMg3In3L*WUD{lNQ66Zda?&cbIPz><{e9S5|Ncc%puG$9> zEKiVxKov~t!cg=(UnO#{T6k6tdYF{c&SCmclyNJTL9I$?4#ud)o_`*Rl&5^8?jrY|7uSrlOnl=fmf`?WPHijs}bsEyfv^<;jIzyV2v z+U2$Pc&1_`+2>x-_IX|ore%6vP65S7n#>emM`9}G3UhtNnvqHn_NzIhAOO|Tf#)HO8V0~ zb}kG_1WOyxac~sCKfy@pN=F*fmq%TlwE5|P(ebfM%YtfhxB>eCl(2_#^PkgbEOf?r za-nzh>x0T)#CRSyscpu{FN(bww>f4z7*W6Y%cDh`nW5amE$S!T{ia=9c6Y*A!M*k@ z)pgs2+*7^CWXl`6GMtAiy?+AH%jgC1Y29eyEW0L42P~di71gV#0Gwk}A4}zQZRNG= zk(kDh{6Ha(w+)k1f)#VDB=2-h5*s!n$qfl)pT976dLup*%4EyTEg(i_B#h+w6&F9< zN;3>{*@>eL1or+3t>WS%q4Rq0MR;oVGVZj_o^(U_5+_?qw<^&f zP6FL-WLDqXfo4i`G4<^Mc~!w+?<&yA=5EH~b)uQ#b%%#b%rL=QDV;VDnltM_zN-}7Wv8CdtMQ!aD zMl!x0n628XR=LtzgpqdJ)litiCzP__v8@z;S`xifFE?Y3hyRMsYOu$Rk1EinzcWM-Gw}4bBj9D~f3$ zf?wW+QC8PK_)h@Vl^H438QdBee zQCOB1B%Ldh5&bkMz5dOsu;8J8GF0<|FNu2tV>fkWd`9Bm$0BICb>rTvz!Omo+d)fA z3im}e?S6vQxI4pMTj;VQlpi|UUp6pcGQn-Q zz*@_FZgNT74mj$~j8xZ$hbX=AS`uuPd}}Uw?|BhDE;5mWkP&t3L^tb%ovJ5M*~*+% z?Ep#JDBk4MKTY<(VH@85Ab=5f-l<$?#pOO8*!Td)suzVh6D^SQ$(4fw%M*}7Fk)lF zcEY7~y-Rm()&xeT+|ahu-hr7rOoGt(oU3(Cza_fVvzUa>Fe&D7+|qHl>~ynMn$RD6 z=G~WiN~=82Fqf9Ak7NnI;Q}_I-SD;rRdK(Twj+mIs#|u3#CxV#ngW8n3V7^OZtYSX zNZ7Z*4M){&0uxvu#$~gU)offdK;M@dyX25~v6INEAcGg5F7WA8;!I1AJB~@_ef*5h zuyjiKPTB7B-C?UUGt0kN8$?^D%_W&BLQb4ePyb<1wEZE1i=xu1uGGx<(e4c7u%Na^ zBv*tUd@cAj8~womR?FjY5~E^9AuvJdRB*P3kOUMK@+TNmB#4!D2AhJg@;4t=zY}Vt z_oWPgR__jR1!_dmd%y+FTL^A$9vs#Ki~$d6GSWTMag$pda}2Z|D`Y6WM-cnVL4uPX z`5lhr0C3q{e}u7eigpxVHD>$VTSkA6JTr1e{z%+)vA)WN;M80q3RMsX93IGI$4jl> zeT;uji%&xE3Yq)UtS7o#YLQCmNOnYsZc6eIGYzJbR);OVzb)1a6iBVQxUTU$43b{&L#xfGrslgBeTP93qtN>48}Gq*=7VT>r@Q&PcUm#^a-sxJWFCCNAUO8a4<+lBD;)jj&l^Izs95hm(xh$X4x?;Pz+A>DE+ zD~ZKc<@2)8I9um~Szt#(iEy>2w9K>I1PngH{fd)W=_E#e#b!z>+OU|H@do*daG;!t z0`HAM3WY2EhqT9wAX@tTGd674ZrD`NiYZ*mU2C?jyK#PVyJtiCO`8 zaq;jt;wEFFDa!8-w&PP?RTFV|=RH)n8Oq6Ra{Vy+R_N2e(5{o{tCSk58Bi{z-lyBj zgKTdMc2nuiaG-cvD`$_bSKZ-)E!@N^!Q`S*LdIQ+f;FlHJZbG*NA~V|4SFjZ1ua~S zmx($L#2d zrVY5n*+~K7i7;*EQhwf^2{!%%uY%85MI2ZI8@Qs4N+tqnU{BsKJptvc^&3FUK3Lfg zBC&#=6L_||uTvYlzQdfRIIs3*vAV7Wn0fxq_tL1OdK24;-qKJtOj(&cEtOTsyoNy@ ziuMcKP$reX`vd8g>8|)V@b>0f$^1L1JoacWL-=}lX--8rOVVGsJ9GM`tX091VW7sX zOZ)t*pFBnnIcWf^F1e0y!!#+QOxB7BiG$3K24dFxEQy6;+TcL7x z``!X>b|S!GK7p+G{ETIdV#Nij8>!%ZBIw3e2WolFjbd~M>~iDBwVSthfE z11b)VInJ{oKkuh`d80v%MK{8sfp(dRkB(sbieF&$-F$RP%KVv=jwKozARJo8lGfl! z4Ok^%u&&+*8rY=?yxqsOzL&m;z^S%1Y#%f>z5}nW^=7?M`*^(J<7;URltLvRz!xJf zK`wc+KhP#!wd`9{Xj925wVyX#g$7rO>`O{w%Aqdi#SYR{3_6R!*e#ZS@( zux-0~kTh~(c+IzIEw?%LOlW$%}q;J<7t)IR&Oq{f-5`LtVu`E-dd!w+$6 z`Nt(R?_%SY-2Jg1*U(b;XfH=S>>y#O$fZdi3xUc0=x1uxV{WgYbdIjna&GXP*AX%o zXuX^N!{objS4iuBII!K9ptM`QB}PKDi_)Q4aga?dTW34P2MNDh-n66QuZSJ#49Ye5bi5V0bocyN1M*G?OH18LI zj!oacmxs$pMb{4~i3G=@Vwbcs<@LQ8BK??@HIPV9%y(Q>5^ z@7_3e&n>jvjUr=evS(Tbahi?(&}Jer ze>_p0?we1-FoBOhHeQs6530SZkn1>q-owulfSXXsrnZ5P4s_C8ni*A7>nGXD1+f#@F~WZM^g^S=PBbSoV6S>K;HA*2AY|iq|Rt zw~*KVw#-25rt}%#C}bQtCOrm72yp&bMK-4w$3U@tz7471xk_30MI!9_gwR5c4w@0? z=VIoew|pF)99+e40(#r=i1Zd%inaW}{dpSbebd8VV`HuuqZt;B-x85mGi7*BHX5E$ zk`e(#iv;E1*Yz8Ih;d(GB3wiQQ}ClnHDuZ}<7Vxeju*{)+3pD%??feh|EmNDoz9wZnvFIt_7Ur6}8w}+FYimqWYz_ z#gu35pgXo-1^2;fE>%9xN#Oze%!PxMyX(A)@$qN5*>T$iJ7)y)84Yhu_rWGmznCA= z#=iRwGpVl9`Z*O+0~Hr$QJB=4p+E2?OUq?L!hK&lg=M%$1QaOKPNDR2I{suM2$~*6 zQ?=Db4o1^50UuyEnc6!m40lO6q&4F1w1w#UPhtR#NSiS25Q@ zEd!-J^PCy>ANjo+hsDRzs_`?}VLJ#sW-8!sAm-(a0FQ9NUl6ln0k6G;c7oE<`vKaw zB4P3s(Ed*E*3gdylpeGi37YurgobiAOQiO0qnC--2um$doD%=FQ!s|L6+Am?77qX| zf65AS2X-*xk2l8rHnka`i&ynEs1-lGiL<2tjS6z}en7*{%1Jzs06U+N`hjip3@aB& z`4xSE&e(_4W}e1gk2xu0M*=pIwMg5L9h#IP=1R=7$Ve-__~Ke)SRIX`AcicgZn`oP zLZnfMn2ee_0a~Ar?3k1-ZgpQ-s9;je`fDLC66}R*Y%kuMn}A6XhOEwm?koW+lAAqe zHAhNkUA9Z{SfV1#3o!RN6SW7rF-qu#|3~00pXBO%wA3Cghc~q1b}1r~BSF9Z-ojM3 zX_9uaMn3*N!J4HA?%Zaf%J&ldc% zH%N-1j!`)i-uc|c0r2bdbI{wv?$#T>f-G%4qN&r)={n15WoB{{3<35A`%0)UO3!cU z+l8>FkYL9vw3_nr_^PzT)vk%B_YCdN&~E!0a#-dK7jPq5B^3U*E?BK0irs?nNxxS1 zuqJqU(u!>rIYP1>agDet;uWmn{lA6EvX*pmorIF7dhR;izs(vT$zd*nO!;FxK|IWNn_1w zcXBw!uy%FpZYyw68!2JO>G1)m1t{<-51(a4odsqQxlKjdWgrs>=f0BtkUX8u6P!O{p%&_B690TBYQ&qk(sHA zDX>C7*M%S5Tp8f`dG_XPu-yxF7GFd1DPU8yKIzfIBLK97{@{;?yj@U zb)$85oNLQOQ6qz5B`RLcxMbEn8eCbLzvMy3h(=S~&f7E`m?UZ(h;%>jfsb@?_>A+2 z2^hJFReLX~DK%4NB0F&ufSxd^!>z|0rjpJAzrW3%_h)Pq@xBTJ?%mh|4&MwZ4KR_L zuN4xPqQdVo650V_2HI5mQ7U>w)z$?d{!v=^PvoJRT2zP*W*qb^0z=sJO50C)@+O-i z+{hg_NEuD{i5d6dIf)4vm+%wIS_HpJUI|(yGOT|WE61U~n7)ZI9`i62VV0dl!_uqBBXe&xI(v1@DgGQZX|EFwKn#hl)W*K#$#t-7r@Ob zLW%u?2Z$L?{JkOX;nB@bDl1zU<@)ZT*zQ`v9nPRz(*9B8w$nSwc*U(xT4jeq11!^3 znvZi#0nx$1q%#o?a~$xa5wd8t1Uo*7m1nF!BbZ=nYA^DwjrZG7#R)mbC(QrE;r3zeF6~^eX1uB296rJRpxCN#jlWwVp|O@L-W8NY zaDtX|%mQJ?yepDfR!t?2t{QL1OGW};&w##)bOp`<{poC(N|LID2e3*iI%jc$*Vgk+ zjA@7cC&mf97bc}_nQ4qtn5gY`zzOp3wC{e8uYTEK%9(GrNf3LPV#m4gmZ1YqSNdFEzm9PgY$}!Bmk4W7JoB%z(|mKNuwBjYpEg96-NiG}*Ym z^r0Hp)CtJ&UOy->lKGcDnS6V2NQ7;9Ske66R*LXth=KE6Wi9kJn*d@qZ7X7A=LM)J z=@!h#mEE>3{6C*;DGezboTu*r_F2^C{bo}TC%;f$gM|e1<=u8=(-aiS(nJ6UxWBU4 zvxzCtutvuQW`r&soN_&7DCrH-?J=z&IW1u#zqx|z5@{kyA%KbeV$nlKGAk#wXrf0KP+cGzt+Y-X-p0L*It_@BaazAS>x>qQ_!V}eL(CALZG>z zVo`t-uaHt6$9Q^>-t60Sb-+Seln218!E`&nwBI$rK3LGN;1d~h>xY&uv`G2gIlgQd zJn-~>b6DL9nV3rxMHYOgKNee20rWm(MgS=OwPu~O1wpL0Y5+Rbu`wDWTDu2VhzS-K zNQlBloOTeEM3R{SY&g8@`kmJrr@MF|bHN&U`QU4%Cyy`gI$O1q@~!h=fyh5SIGGo~ z*Ruus7iCLR5=tjq681)Ehm-i z>-BH1#bJC~*1V1#5*ibwCh)7AYS=nKPW-DlQOdsru!JQUW#JLxvKkU<7%k~t+*6P2 z2H>ZT3qJX(B#dHq`h)VKRzt&f15h?_m#79#bVWo;m<@2>=zX6z@x`Dg!#A-Fx-ZSL zn{YFoy`j&*ET9^g9fNLL+LWfmJN0O00G(JL^Alb+^w?Ewn8|%^GE^< z+3<9;`nSkb4nwi&di;yjP!iJH-Rb`{X?C;SW4Y;&^e zvjcQS_m`v_#EIvV)6HQg(QbVw#tr9}RnFxTZ);bvFIrKttgcmQ*N@70jL3DH zI4<#GMdrojRuL^VsJ-?xZJTz=yOALk+3+S?nvB1~A0gXYT`Vki6^Yi@_>ZX5it#sB zK2{wyf1UOp4cr#&Zt@o2kK>O>Xci@*1N{ZaGswpZbO~M?iWaLf>24Cv*-u*s_kG24 ztOr%A8x=%ez)j$e-P4B0(BO+}9K!4%+*T_Y+CG{=a}8#$@?W;{se3QwSh}iMAq1;E zY1v2!_l>=c`#8iw_Bb?7*m7epeB_mGa`i)e=R<&LII#5S>;?4iQnKPB5F^x2_L5U+ zwtxC1@5xl}y*utNRJ4%t;Whk9#dosxCE#O$p>=UGDb0~Jvjs}w!DV1tC>4Tza`bzd z(tR>5U{iJEdNVRzy2?Bbor%VRubqKMm8^YP_7FY`w>T@%3 zubWbUKmgnDvi=!p>F=Xos^!K5!bYGSK1d*+5JYpp({>z?{hBl|ZEzJ0yWB7p*BSz((Fwu#uS^V3g3 z(>?+D5RM)f}}=^xo09|6ZJw+qFFwck74O*EmWY^KYKd-U)c`*`1{ z6jgwTrG@9|HDOlR{-rOQGUTgUG#>wA*H=I&j8P{>@o0E>L{&GR9ba)GsqGx-CT)+h z3_ULhu7c|G=X4{w>qc$Ro3-sBePFtS7DJ-a8vo`D^*G(SNz4f8_LKV4H8gZ%`HACd zv|M2wSF+1iQSUpw zd-2!`wP&O-=o#D?w>_pvVmQlVDy=xEx1`BINp9)ea602rWiA4D zy}0HFxPjA{X(%R1v^eJu7kX~MIUgo~cY}PO?WEE4RzCDM+7ca~nhX4yS}lducrNzF`a-VPZmZ(7BjxeZthZ)?n?NaQD4Wf`2Up zX2SL*=|Kqzv(_}QpoYjgvyt7=)AJM6Jl<8&#Y)ClQONBLBZquOaZ525GyPXRDl**F z4lGdyk8VH5F6#Bw)&cu8B`NDDhU?^oYotB}c{QFOic5x66Zh1{;mOyXtEaqe@!@a+ zh7DR{(*~_-NAJ6S5xr4iSyhEclS_EDb*HHsyOkSgZMGM7$EgY%M}6x2x} zW)Yx;EzI|V@m2}XYmjI6y{aZ!;?yULuEI%wjK)wHd1006HCHF-e!J4?1}4YFR1D*E zJvilN%8_OEWpppNb3|`H6knx)`XCQ8fu%R6}cli z24y7q9x;AZ3JMWbDh$o#bgi#NkgL;M?DbSVf?kM`EpV~>%K80|V}!)}RGDCoLUTv? z$No;3T&E}Xstk=f8(+;U!dM1|u(}9>C^b-Kq0jvfswgO(wVDl@u}I}p8T>AwZD7xd z>uo`LV#f(dL8MPKlfLB^!a5LI+~*p~0X55G(fLT4Pa@*0oZsPJ5|Yh?U1!oZmU9-5 zR*T&)&|M8Kpt;7vpUPi`Ce9lTM~?isa#;_RWyfwb4EtRg0$qle-t1uq<5HH>V~SST zCoTG>chRK^BFN>ZM&mk>6W5UkVH=PkAGSwRKL^_9)vqNZERU3tv0Qa{d@hZ&gQ)wy z@Ut2(`h3nCE@|=7iId}~^^Jy`Kh?dbX{V8z*sr;lluexb!IVt`Un>{wy#9YU9z@rc zZaW@*i_iqxp^1ZGF24r!ps;T9I39s;1gQDMk=msL!Z>5L$RJZJft1q9iwR)l`8<8$ zkVJVwHXU2(&NvHgTAUOkjR;GfTTId?PXBzt+#}8`&A}^Syt`5LWiVY{?#>K@XQbD$ zM3t#JJyEfGNxy&wf@*$}NDch(f128F_MhsD9(z}qe>Vp9R_A#GX~BeKPn2tipO#6( z1vmX^H<*~b^M{&$-({s$gXhJ`-6{&bWlY%cq3n*jveeNtd^4DT|3p?y9iQ&~B~#Em z@KJ3%1kXUbdIGM8i??xZr*YyhnSYw7*?l`2y-;sq5Lwo{2rQm z#3i1x7j}|UR+p#t46z^?yfq+jKouDd8}McCzmE&uSM^(3e4XQy!khigecWC2P_hKK zXpsY}-gwEh{tfWB0S*`Gr^aLPIt8@TpU8ysY=LBYph}1!l2kaNP(M(rY;n>O-RdD^ ztSB(r<{w6_sn$c-J>Ppef8ZEQ=^1gU7w=nN1TY2kSL}Z|CD`=Jq2cQhsTgD}&3H_l z#a-1(Pj>ZkA1#kE_%sl8&$puAV##fV;eEA1;N>DqP0cI5KQiN@%h1pOnHANZtRDEn z)R#&qwP^pfz9p~A!RaM;cK()|`@XAAa%)N`$wep>%QF=VNM%T&>3i!=0~OD?3Tc}8 zr+vHr5KEd)w^PqPERWgCziQmw=`pOW*tr&IPkJV-h+=JLvh?ZzN-Li=dW;pX{C6dQ zFmYLGBh&5~HIKa!Yhgz??uVZkMdlqYBg7{+q(Oq)T)|>Ig2f|E?To6o95!f zx?Rtf@O}s|uh3jRknGHBO)e^& zn29j{ZI6zKSv$c?$bM$a@{|^|X+hNe()HD9a1P6P^ zf*=rvQEI{Lo)Hy@2Qqa=2=u9{+lVnxuFn&wxFSPliExk(cGAc9zr=!(0=b*5 zq*YhUSaP2Mk%_L;9QR(*c1(rD)T~x+bWZHp-V{^2i{3Aow*Yp`Y=URl;bzk5<;Ev< z@6~>8Ic$l|Zj}1=z#lT+0|Tv9!O{R>5V&ve6DG8G4z^7m4(O@p$}?Cw<^CkHG5nzK zUG7AFyP|hOPuZa~V&!V#YN#keW>&N#^7e??f26~0te7)eRiOAa!Ork8|*y%eve9{0!KEQjw`YR;__bl=!cQv;I zE*CQoYJNb%d;h>Z;~e)BIt@~}(?5^Y*RrhtMP72n<6P`laR=kMK)^vas7el}t25u@ z-qAy|DA8Rh$nZrH0zKHvuSIeB_}&9sSTW=iAwn&<`2zqI?Y*Al(p(;rpR_W3Np0WQ zR39T48nbw72LnNE?mUKAGhL^r$iviFZsWW%oTiR6_5f`HFo2F3{rt8kNsCzGZFojA0h<&_wy73q4l+an6iwv;kOaIidE&m+ z2x8!(GE|Gda`T()WC)_EMI+Z)c*O}J=qu#CS|Nc)#0-zxTmUf&aV7gXX(*Bn#Fk<}0 zY%tv#+}1f>RyPaa^!KX!>lEzq+L3! z@K@x|i0m<&AH`)$fic)?Y+2>%qfgDBrjpnc)!5=>;qI*$*>n=HtZoh-a^*A}GqBCl zey14ifu5>PL!)PlYQo?q~f#N%eWYUy zSQTyQ3rRGu!Iw={YrkrSy4BTL-l<%5LNyz>Yk;`BCtsU>JjUlHCzAC<6|-7q`i3%! ziZjM~5!HFVpGx*PLK{P56k@tMPZ-tISe<5Ov|0q^8DuNF2|}*+Yjtbv{KF$4=wv*% zWiL?qp$(t2|Nl-gGp>0Z6omyem%G>5<gUi0g3ei<%7~ z0L!jTQf!LqGwBp)$%7=0CC+pj^Cjiw+yoNpXvOpL9o6)RYvHH;&is=<>U0bbeOzVs zk?;<|5B$gHptS2eEm;_SJHi&Sn44Mo6Zx!BayH{!Me#5G6dV8w;ex=Afr?mbH6I&O z?-*=E+%dWATLraQQY5G6QR22U)~jC7C37OCCYK2y(pbK=K)OE@>6A#lVH&|wtIl&$ z8luhATrDaEt0a|cSgWJ7Z5;$TIq&l0->I|_(wSnBXoJ9drwcj##ALD#_0ols7l;JN zDN87d(VX;$&wxwNd`F}+9BOZccR25}b9m9BZbZWq@!VW3>nA91@UBZWarUPoI-j03 z_e;r;NF*uXACY}GeHVM@F)|@55tppP?dWhBeKoF&eBJ2uJ^pjg;81u*U4=h@?Y4~l znQ%}=&6-y_^Is1a(7-W=UVe(%ernWcuh) zo&T7A6(cQdo!lnX8LmsAg^vbDFHXP)e2b0gsQj)5h)-B4tM24u#L26dwKSoRVe!G7 z8_#g^*Vn(fx7Mrtki{aBV;Vl7BD}gYi#l)tI&YZy8Be)nxPHs`&Lv$X-o-(Scb0RW zzoY7Jphrc_j}u_7LAq0 zsfpse&2X~T*?HoVJIL;%lc+Epl)8WAU#G?!=kJg(HEbG{FHw#T<*aI!Gco3NE|@jv z2SByOr?!OBRF0fwo)#~%?81u>T`1Tn4d{%lraT+<+eN6{03MZdI9R|C(tb~Cr&$qx z-j6v=_~*lnhYJE|OqRG}urX$a*rK3X;`_I+l{JoSMPD90GRKUib{>aTU@4l~ z&;T6`Ao1G(H2UD@u`!k#YZyD{)Mo?uGTA4f%$PkDXuKO^y+Ma3|1IO-te?Fb`?5KP z7a(z80bps8y_r{o#IX$xK#oil_1Ok203}eclm99}&f1nUFoN%rw&I$hLIX%#{~f;P z*#gt(4rJEhF}@$6NY^OO_vHI7dN2$KIgQSiIo5M>zgZjx=Z{p>5PwA=-FIKK+PE7= z)DP(ak$6-uu6PC?HpOB7k{@5X^AsV9!No%$dUz$lDV@%Z`7Cz#GP)QS0?qlFGd#|n zg!MDIApA9wm7z7tSloV5Mm;A&p?;9X!cFM4`;p1{O>6C#Ppw3zGl?o| z3Wn~(%SUs~@2`3Itnz50)mkn4LiEzIy=QKkG`QiNJ666Ji%@!o#d1HQ!G}&)-2YEt z#zRG55pJUm?P=U6qd}!txa{A$nEAH?Zob9K4!&KYW9W-OesOw?={jlM zh5r+_Ww`Aa=!E2P0l3e~RwHtlZKa|PF6DQOaaS_XwbcD<^lf@vWx@;S{y9&c ziVo&3IAbMpS#Clp8$e}s`)hxM^97kuF&|Gapk*nC~RTFLf>AkC=ov+8|~5 zA|ENr{BCUpC;*L{jqLH#*4F+^_-MifkZCU-vqnHg83`67sIAPx)>e5olC1YKs7=me zU`RkklQIb|Q@7*oTW2WZb9?@RE=tNhM?I=$a_sVq^)_hUNlX_xd;OW>zmT<;$?cbg zEoBezGT2$yYPDf?YQ6^o866zg*Iv!;)eV%X(#PCCaKOC1kKUJU&#w!gXf1`OptrVK zUSIF`dxN=LlGC@j0DP*ok`9gqq%o$%M4hAN(M)%Ze^1-VCb?GK4xK#TfPHye++a-7 z)vkQ_t@&jhkCj-(jd40%~Cs+kAcqS zv2E!1Ys$ZM2>@?rw9QRZ>-!jIYQD4*K;8sTh__z*kK@ul?QQ68E5OElS)0w637(=7 z7l>_gNxph)mr8UntE5lijL?kF4B{I~O|$s92+nD8gat_?EVXbj+PX~gm*G(8h)||v*%a4SeXiTuF@6TfDPT#N zdZ4TnA3K*ix!DSOyP|_Bh)>h0W{4_gqZ4{c-Seu(Fe0@uNlfLjZk`d@HE};C>HTLb z-MmTy5*F0rYFz7;wb_WZ>68MQC3=cCbG`TFnlI)_##EThZ2CLn3Bs}B#T0mQc*(rV z(a@O&G?DBBppON`j@Pk5O`FX$o-Zy^2n!uxelxbFKqtq3j?HEi1>mcfSzPg^wxKq6 z@i!RU(h_6<7z6e_vG1R^*ZJ#zi5kC zf<}e7oa&7pcLH)~j1p~UzWyCjfrg~~B~;|QR?2ucV%zsT$5v7!x6JqsmFTyjf~DU+ zhFCG7!F?+svJ!Z{6{XpNis;BX`06Z5H^bXAtFr870*j-xPE`mMQlu&NlSM)$2b>B^ z1V*yVWeW9`C{3nGdIapvr!9y%%ETK`hM{l)8ykUTyTNChxh&yqke+z5?&%rh=Y;^0 zRE=(P(Z9AM2T^w>g5V;#sKLV0w@GXg?#)Q?&>%P8N7W%LTAWaLIh_4Bth&iqWMy6* zTZT35r!k0L{zW?=*MsF7z+4B-7~H+Is)6Q*v4qX@D>e36aprPb7l9e>maf-W@>*?* zPMzCxpY7liWc1f{%C=r>7kr~yur|Eomb*@XDLHT*a}W{7VMslc=PtLDeh+Rs2%I&N z4;)qmoxgJgodGDOdSNML>c%D#zobOhvCLPP$)fhhUH^kJ#=&g(x&+kfsGWJn4jF6 zIfUaeEI=%U?bn*IE-5}qfB2%CdQG+h-aJ{)YADkEZ_oJ&CD#i4u;Nw5kAO(EExns(pV{CYMGr03Q<0=I@+WPpN zRMOLIi;dzk?L~}T`tGW1>Cl^ir!=aj6IWxp-E8O6(!A!^1%J4AtSP{X50QDSHDI3G z@4KQTQHU|8gWLRId>E0SGjoCwGe)*s%JfQMymBXsvakZTu#>q{P&~P!g8c|MgBZMh zslb&vyswrMHm1^_hB*c@8=QiOHTXRtR5y%|a@S)2)eB9EL+JV7ZlyDs!FUmVCwqaQ zzaQNw#+dzYw!v?V5xq^PVN8W>V<|D5DGwj$nnQV{VM5cBKMQQy6)_N4)7J^B0K!K*dy$xtWUks;DnNoGw1b|8wzS3M}&9grZ)=VCHu7|5Sx97UgdO z@&E1_e*pmcQZ*eME|UbaK|dXF|JNSRYcp2zTtY&z7Nrn&1y)e-B0gbs!5jnFh{}k( zKc1`{aRTM}1*xO4=s$44xQt?6P5vUHbn71>Cg+q(D+hcn9p1>ilul=xgAp;Nmr2zGh3}N>eCXNq< zGsxJ&7z4HSucO5Od%Tp3Kqin}p(ZtV4}bxJ6&X^^(N;~ZkRnr zH!;&bG09tGv6hv-9=mA9SH>DGhm_28R6v{|4qaIXDqB$Z_jlb0diE$&hmaA%3r8gs za;E(=`Z>^acx2>C8jI1*)w5mgWKT<)o*}0?%ncoo-i1A5wgn`JX z>jlK&d_L!MP*Q7S>-T5ozetqNVq$4>d4i8FCTTp?;7i3JPuF4Z7IxYm4wxYP zlkC5q=To5)zL^FiM!bV9{bBzvut9c_H@8W>9hp;A#eH^+@n%`?b+m4wxTF_(>Rr9@aCe(W$ASKDv!9G={sj8}_Gk^zOp+8Q z)lQy4JVQW0zY-HJERs7#9n>e=E76DL?zI+3BP)>9wxFJsTue|ZEiaKxB9~cSNl&tD zGj=g4nsQ9Tj>)sBy1f$z2pUXl2wT1`%h%l~ZO|*kT6wSQ*$QEnn@e?sr?W096qz$@ z=Da$FzVgt%{kE8r!`<4W3H_J5VK=01W4QbGzr2=_(O5E2BXH$E)&mcs)E9wi3x?H? zGdr1zZSKBaU(MT~5RqEfcCJFHJ4736=O(-x)nn=?5D+k(F@6U3wWLmn79EFw8oR`} z7AJmhw<<(BogJgLTChj25&lsZmNe>Z8eM}do7r}9TwNzF;q+(L8oS_LPJ2=MfLZgb zS=EvI{89h{ikI;hB;@88`y`f@Ky<*&sGt@o;>rC0lk+8_J*tF7toen^+Rso2ol6=n zuKd`++84UskZIr(N=&M$%`L^naJG5%fIU+G!5smw6JplF8F-ky$r$VH@8>b)f|n?* zg`(IY6NKjd7z=v~TsH8zUr9@$8uHwy(*!(fWkI-RCT?sF;yW^ZYC1JDV&=r>Y?F z-s0W{dd^s+*<@UQBqvrE6Vd@PFc1zOZy|q%&Nlj=SqS{Uvk=8~39s@0I|==tw!Q+a zjb?2-MT)gJ1xj%#?heHriWYY$P~6?UIK|!FU5gf{xH}Xp1W0j*KcR2mp7Wi5u3VeR zZgzL(*=HWPXOf$LbXvWlc5vgz$;mkqBCI)yS@o>gY_^WGtfjzg?Rmbhc~C^S#d2dv zl2wBb$!S=#&ZH5DP#xwi83G#hJu z-P{*?18uv7__=8i*#IV(-sb9x7z)M_MO_VpSxRP?*7)1Tj`k2Xe3%1w+VxoWrgzFY z@M1H?TgK*r$MVu5|~IhFjJo<`2zNBQ5@AdAC!)7r$Vn0G5HOZ7EH;nfc z&>ruWVE3MY!I=r|sq1fSlPvRiB>(gf?$6K%lYr`@!$@Z*u80dvioBr_YkLy&*7AuaM*;%BhLws)ads)031$O7}9JO z0Xi`-L-GdSDVy5{nvA{panzV+)697~h=>J&NPQzyO8JxkS`{ipsQ=1|cC$zz z@GT{}Wc)`*B^{$?Ft<=_dVzqpp*2{@2^-1DDfxyC&XEbtzfQK15MNGD>q;*z;sa%h z==Jd^fN13mcn6jeq{?`gmIei>j@ZSzWtORy!NuAEZM$AS-L(=fFAN^6zt&)4{8l`s zw={1nvx@ROdbFICHQzsJo?{EV!=RC>4pLJr5v0S=&Nnzdawh(tm_PTB_}57+=aJ{+ zt1B}3y$oZPP`zSjkx=0K@^VtE_yg-37UY;Jn;u6@Fr{PwMQ!n3WW?lIcyr()b#j8b z-OMSHjsVyRl^@PoUPMxxsD5i{ITHQxtJy+qI5+Clu&vJje8MX{)E1^0l3s%hjQfiT3jkn!j63EDOTN<_I!^0>?yO*+Uk)45 z=`T7RZ1K_=7V{B-VGBrwFT70Ka0u%?5~?veLww1|aDh3h=U&7W zB9WmgsJy68gmAsM!2;^8`)Fl%UEd4Y)^WysvQnZSPQMJ|Q_NTDK9`}8pjDOSwXM<# zkJ9+g&<-O&lCMzos0@-?~7G$w&&HNS1=7;@EOM>UL)#hCJ}R{AEmVQupfT zUXf=ARXyv<$E`;^Ymtccr%9KboKF;XSOJip9D{q}exeu!2R5Pt-@atbinK!o)+>o7 z8j#2H(bUGzIZMLNP$;STr##5Vr||j%S)9i5`ciA>p~Tr~7RtBp>{fh19_Tljr%9UY zpVh5!!S*I;h81t-5p+6*H2y8@Ao+?nyiY+$002p10AOP+ARqv)0H7enuX?iR-&p^t zV(PwPtgECEZt~1_QKs;9T^jHsP19>%C?VRe`Z46|RoCTrPm!j^{4M>?YFL*Gs)wo? zL1-Ob^K{Z`Pw;zfp9k;N;;?}zG6aUNPgw*yK~(=Hc|N-btf~6Y;Jc&qyB<|))@zTKnojmpc!G1ld{4XaSlUn#6_&>0NsW@O%J^*;K_gHAw_2863k_j)}@I z8?IY0|cfcd0-0>p}5MkGo*7;$z8IKDT=HK#~SNEGEM@gd5k>RhV zQS^_9@nR-%#VmRyg5ZcIym!ZDgpRhn%+lhJFUy7HIYf?s%c5gn)nH@wDvaYL_=wM2 z=uamv1K@p8;h&9__NW~2rxDiOJ=bfQ?;+o0i(!hX`ZT0&7ySC~`Qg8!pFaFRd-e=K z0tFCp`%-~65c+-ICz>(mlR2A{H^=&O5z+$ul90E_&BuAL;SwQ^jR~795`el%!?}N_J#|KcStkp#5p%SAmU8jJ z8`K+44zEfmD%1a~$1KX8hfDnR4U8C5h+yjIi5J2x|HbCl;3q3znt$d0Tp00KSf9-Z zmnFgbaoB->Tbo-<`niKG>R5=V`1Kg{*UPX4hJ3Sa;7tK`c9uH{eB#x6BXw1Mc;pvP z&042nOrv0j^PYu2T!>1#oqM@#7)}qZF0q3E1rrf8-<_whjPT&zdIN4J>w9oJ z+ly&|l0r}-db^xlx;h-7%&}tNTfVMXA`$?J6jkkvi%fyy(07?)rjuh8u)a8khdjH2 z*YV4`N4Dw~dPdY*yDvf?ocx?u#dx|hYs$w#hMUOgN|0(?JCvFxi9q8u5ULo7R-#fmV^;4rb2;^xZ2U?QJ_AcdDb&@N?of{6YFjsv z)48R{A%yx=0!iKXtmOQ)Rm-*$3eO4dtS*-cI`7zND=HVw!JgMBflJ1&tJig_RpBQ@ zREoc(UX~!S%d7K=14ateqg_hfaVEcNvCX1U=-NKG{1VkPw@;=rPN+;F>Zzj!m*sP0 zm*6fjK)R$sx0z{bEJ3_dR+ZvHfU}7^c8Cc{MS|PBGxE2lm>h7`!bqFlmN}G-gFciA zdWxaa?}0O?`@0C&{^sxEgQvKA=98C$2H{km>g0KXSJUe-z7eUhov|CQEK`%G6^60g zHDCY$V`IQ~FJQ*pH=Brlhw!~;xO;XdHfd!#S$%K$l> zd9)54pdluq1&9(&?pJ%jDH?`?aSmT-0P=&d^IaCmsEpr6i-mLi=+okm`l_N+Q&v`J zS6A=1!n~Y&S1XVKWK`P6hkmP4Q1 zn7dftCa>RRaqzCtSCzH@dfHBFaGn`X<>sEpHO@8;c22gAJZAM8r1=pboPu1`ISM%I;1r_JAA$Y$Wz&cQqN^w?m-0l@-z#)WWEIKRi|ElTBurDyK-x zB|oC8KK|sn<)biLYOq|o_)6?<1^Z727lYBu{h_ibLm~bMQhZR;USQ+3fPJExi)=;= zrE)=9#g~6@Hp743B?tQR-+p|%Eu{<`hP_3LCmVbvq8$OI(7Mbvj9dR465CItW%sj8 ze;x!La+cJgIRsduAH+2y97@Bi6c(BMDuP}FD!;i@`WXxtY;vJ{uPvN z5BfW7?1o80YKPl84@jr!$9Po8B`5bE=9X~b6TjY->CbE>V?Qog?o{x$?RyfQz6IW% zHk(sM_}T-3r{?IaxKJj;co+6o@*qZ@v18kZ^;mJ!`_qp7rd{N+`_||#p{A{$6ICT+ z$?mhct)FcE@xz{nd}BKrWa+!y2&=gYbruQU54~T51EY<<&`Q6dzNwJo)SHGzbC#wX z558usT3e5qF-(Y@l>|E{j93YrqZ13hlP|mn!h%@-{$as#lA`a0X{6a#?6Y^FUw~cO z!KVTh(Ig_O6n(CCr@e-i(oK&-$8W1S5?~han`!~JLQNI`gkE(6V=5hUKU?)yCrFo# zaEUkc4_@a}Wn(2mhJtwSU)=HVN)z{O`9+>)x?L-?hVpw)J0KWL_*plwm+~yVn9GKq zL06!Pu{T8pZVo$XH_;Dd>;BmIVyH$`R!2}lvUyW`PXwC>x4xt-=nsCUdx7x8CYika z)Cw@ktJN0`LoKDZqZThaj%G!>d*i~TqlRh6o$YA;A=gj@E3y69t`O5W4R`Xvtm>0#SDW^RSbaa^?U&XhB`-gI!0N>UDQ#Pbj_u z7(63G55Z4`-xFsCX(GZJN#)8?383OY1$LoHd^mZ)h$Lr?K5La8?QnbF0)S6|BAQf| zA1_ua6FFCRe>_ho&GaU>=Iy}BZ`i?xo}!p*$!x#Iw0~iXAW3B^6e8(fsQw$&HGujz zFFWZ=^@QoRr*3{Y_sZFQ-_{AS@Us;JvIbrO{lLL7V+6+-$!k_za6k;FZz9P)w`1-v z1e)8_v$t#yNGQiDD(C9$Y?PdpoIH=qX!q3@)1pXLAcqR8wA|w0?|kzc$&vpN?V0e; zy+{4d{UCwwaAGGmJVslzzEhu^5e4Tk1!R}W!hK+(J4X)m?-2|c#A19Wo@SaCr1`R% zEH4aok}l&_v;##*0_LmJp`U2yU!`+#<+bp}_K?tAf~-xdT3Uv_My5L6o>B^B>P8=$ zk9e7KaOE8^N90kSG4J3E<|TzdkW9 zx57mFw;lgg={&<|0k#~eOxI~O9_DzVMMz&m0foF@43-0D^F1&|#Jo?b96D}ZQ3{b8 zFSrM;shVv-jiLI>n|~x$QRu(St&9@Wu0%kA>tW*1rGz|D?o$_Z`93%s-`tj?r2w| zSR~qnak>hgHu;B1*phN>;5H)N9Lhpx0ZN+29B--Ls`RyC)$d3WEWbY)f~drQW_qUm!7-RO=Hl}3-Z=g_|sDG`vDYyh4;^atdgU)tdg*jr$XHVCOIUN z)cB`2%s626fjB^xrz>@Sf&r`p{g+QZo;4N%ZLO)nQhgHWvfMl<5LMdx_=5OQMr$CP zmP&PJL<+aECKLE=Z8VAPGHI>UozTd01pKo0ER zCw!8CPuv4UTz|{>-*Nh9rw<=Q27iki{NMUNHJi-dXfO@4G$%JAx*^@H+Z!WZ3NTUR zqjY1#%@p#=+Nkvf31sqUg*%kQZ;9I0Noe}tNwXLWNHp;K^(RN1 zhJ!+d+>j`{CxK2NhVBw~w zYc+^faVp6HgT=BSUt-Fj;T>of8RSlXb$+t$?X>&usG8)7sh;x(?d5#O9o18=?s0dr z1yt^>biXa-Sq#{YP@9Hh%J0|MUHOg8r#+TJ>3=E3KeGBc=^oH0%W*o(+vhZV^yzGwse4BnqF}P`ws4M z@y)e23LUTBzjFPz^7G%?|5SXOB#h8Wk7hxz>Ulk8a*fTRk#i0ZklV%p3-aY?h}|cr zLjiA>c)o!32%mwh5WEwep4$)E)YHW!4PsrCW){cd4v^>GrI63v`X_j(vg`2Q<^Imsl$`|E-q=Wo_ zb+t43f4=h{JA8V2<+wJyLO<%r8>VuY2)2YT(k<4g*%grrhzs9i1O&Rj@D<56^x#Rb z{gBlSG00LiEmV~?T7PB)syXN;9EN{=ml>I4TH+BO)^fQ%fen0awqKcVc^K(P@gcG5 zW$Zba7v$vyd)}9lh+B51sgahjX@=Q`k%n*6^xdoW{AceHw;dOVyg?&0OFR_6C#{)x zK5lPeVaCTrB>{EnXGFAxm83Nb!vqqwoVM$IbRPnq262=wK`XRhOE7I20xo$0i}b|7 z`%!Xxa!}b_uU{bMo&x5H$OIn^;LG;j+&t88peNTIV;5E7+p1(V%p|jT9H(|GjIdeMSNf za6lRDL;gcKOwJ5)vb?gk?&x6Ylg+~r2TaF;tvP(lxqi0m0KRoBDX^EILWU_SJa88# zRQS@sWBSC^e;HxDM*ux0+;O7Rtpl|~WE88M$VDvLDk-ce5)xuwnzks zru)b}f=l7kvWj&`)m!6fO#Fh>AG08^FaV%-?}Y#pK@93$^RDIyDN#rcF}gH89CllA z%-IJwt|<-^Vl#RiW8}ddSCk9F6V}g(CH;aXp^>LZ`Yl9DC2l#@EHFQO`1{Qlq0bDt zKP}U?ZrG%#MswWG(CkUCSKtNb@;)f)OFH4pZ6!vF+bS5xEq;IvQ?MY5m}=TMj<-aU z;_HFCA1Wf-l(N%T%qnfArm_+DszeKO%3YB?Y!LQ$%GG^2 zJ4{4RHaDjcE%600T*hI`uM2xg;{(~uXWx)OIU-dBA@YbJ7QR8)8d+DtG9^hy5bUlI z5V%;$?%s%N;&hw1QYX^{*jCd(dqIrS>sw_5L-L$?whz1{qtKDn&$4?LAt9CDdN$(g zVVJ)?+m+MHxfRYf+D5+T40EY4Px)5=G~SahRI@;y`;8UN9%VifUw+-VPG)yMz8`7w zWp}RA*Jfp|V2!Uyc5qX}n3%SfrOc3UlaUj(q_Uo|uVt)-M?*uKa6oY`K6o=>&5-Pl zXBQ+vgaCj6#6WB{!uHc6f@}n0NHHM-Y=gJ$}KI1=LdH?wVT>tntAbAQ)4F6;C(C@F-xpDo8gY(vuyTYH9g_p{q zP2pygU8F5-zhOIr6|3&aDz9CRa`5c6|1rM=tMwEIXC!Q^w!Gkyu*GuXqfY(FceUHt z^>jrI!+)-q!3JQ&@nI06%KYLP0nz#ShZ~j&z=tXh);Ez&w4Q}SY1PvjnRrgM%<++~ zWoxd|C5saecSGbo=#?6-es5_dm6f5sCk-F~zJKp*uwUI=Xt3njO-r8^F`fDfDK}h@-O%{?v2^KWD zWH+M>SmegW#+;F39ysIu~YS_|xt%l0O<|xg*HdCu>KHnVq8wBrt z%+{A6cEJP#y2liBuGZ6 z{x2ts`$vf&?=4y?*LA7P4$0=~F~B3~_`&G?{n~y#S_c^;_0#8Ta(6IyXH#077>f&G$jSP+ z>O7ObxFbZ(K=LRur|;A|(_x&M2NQc4+q5u*+ZJizL~^)n1jgJJAXQRm?LM9}#VxJR zzz}>>H9=2lbBmn&Ss!E`X6t1>=6W$TmqtT-f}Olsvfu>rUvJeKXvc=1mhxM(QCK#Mm-m05)Sh)pSK<_^0+P%(G|e z&xE%KL&n6gh2i(9n2WfqpP|bpwGq)+M4xoEcNH*yB6XH*+RxwnfHbk;yiOC87cbCU z`VczoZfNcXQdj(f%?CVgJYYvQ=bs~Q&w=5Gr<&3F_64nTkz5pX1k?%O%ZL zPvU4KvvcRa1A;@8(^|sjcJlMJu{R+OdvxyUb%dGs_JfI7jzRCJlMp-8I580Py@-Vx z193%PayqWZzwG1BCA}#hDVyqt!)iUgW49{ko^52FZX45%K=VTpcM+|Dyq||JD(@8+ zRn)lZcljS(H!1s4(4@HC?r8BlVA)=&>Z-F}6+XLs;EhUqH%#hnzS&&J@#sCZFA^;3 z&cN$AQaWrL##HcHyU2xYk?}@)w$9p7N?dr4!q~2g2$65B&}K`r`EA%bv+Eo^&hV-BB5eF3 zz0s8zz~PO3^$jiI;QFGz%K#sbyQOdK0t)s;O{TLO-)Aclob|JZS3GSJfPKl3D20t^ zU7Vr$;2$%Ys+0hs5}vcb8jI8R6f4z$s^}64m!)t;EJ-RQ+tmGe5LBvSXmVNxfwHTi zguK{H$`;lOr>-&X4{s1}g1@~T!y(2n{I=0K`_fH(C$ulS-SBbFuC2|KUFbC-!Rr6P&zoi4*I2>=Rq&^(w7vspYnU>(SI#-BwC}L{6ze(A zHyrDgao@YG))n{c)0MUN(wGTNt@Bv3x=g-KQYAf1T7IP_zf<7^eBZFsCUm%Q#AQXF zUP*)iH&C(wv|XbO3q?Z#p=SKDib_9u$2G#e%LtuoZe<>{Gq77=DER0VJX)di z61l~<^MGi1ul&s+(TLX#i4vxpRzI!dYaN{&+j?1z9!jD*k!FnW{js0qIUZLizBf5V zAjBzhk?F=dVia2V*ByXIZ_Fax9Bqa>bpU}ty;zrk0NMHhtm2CmIW(j;+wn#hUS65j zG6UWhnkEU293EFW*S!pHr@u zv(y}iQC2Tkm4r>Xxx5VOZJM8cTDvqd%4j^}vzKv|2u>6E)LIqH0cLE1x6t3c(jviQks3ayWuEX%!Q7$>jZK07kq2e(5W;Tl2$@G>n3%z-P z?Z#L~1iG#bd4xX_CFer4HFD&1kLF=G`eLwb>E(jv%<5c>cTc?tV#se|0A!-Vb<9g! z_atWdhBUr;j|!9@Zc!UT2I;5ST*aqE54XtdI4($@yJ^plorC}F<>w`v=ZDlFz_EUy z?%=OkKC^u>vQRNbnBIvDjGLTF6T$XZmKbRz(ytg&!OJyE6i+G4N1o|FEG+Ss+Z&fo z4b0dEf`_;A68B2P?7>CAFuPjRH@Sm3{D$}pEB)JvD$o5iYfY=&%aG%Wi?if=e%*^N zp^_6q;;jQ1F*9jpg-p<$>#b9ba!1)A1xt3SUU{m5h|Yo;6($((SJn3yeh%X@RIm821{8ND6pb(zpb_J_WbtMJIKn;WA!21OIk%#td{-+F=4t8vC{a7WxAYSwpDd*$R;~24!`?ukr z`#f8l_F3m=a&NNT@$MagL>&dB%Zi7Ej|zZj7E*3p_)_ry;M5TXDQzJ^OjMOXi2{*77J%cdLJ+gIRSUs$v8C{{WkI8Uz3U literal 0 HcmV?d00001 diff --git a/docs/static/fonts/LatoLatin-Italic.woff b/docs/static/fonts/LatoLatin-Italic.woff new file mode 100644 index 0000000000000000000000000000000000000000..d8cf84c8b96e712c12d2af8efefd8bf240316ebd GIT binary patch literal 74708 zcmbrG1z23on&%rQxVwZ9+}$k@LU3!`-QC>@9^5@x;~LzZ#)7+haQ7wOy|Z&?=I-pB z-G1JB>sS9%RV~lyI$dwoX;(RMaR3AW0DvH$2B5#?kZNjwmEZ6`KfcJzDZN$n0{}QP z0RR&Xns$A>FG|2q008&n-V}uIDuxTI<^Z0KC!w z05mxO0PDAn8OCMk;`G)|@J$CWz9n9$k5mdKcBa;ES`GjJQv?7YCeLFnotf%8zSR+v zc&h{d_k?C@pJ?sxxYzPQ!5%{@)aAY(Xj`yGT-$SE+|40xg`VKbXqi_I5+t61&h%UqKw#UcO zbg~eSw%3j|jWvzz+AJ^)o9C(f?WN#&%Xn(wu)L&uDTGC}gZJ6SDwJ~6q+_9+LR7yT zDW7_M>cw=VIjGcO2d|)>HRk!UTH&Bts$c+^aI{@LRa8Q7@w4K|`lD~Ui!HuiTC3l@u`J@UnU%$czXsDyOR6KG)7O zF=vtG9_HUHC6osrCJ1tG6FV?62zkS-(U({otQ|}~?fXyI`ZuMXk|2&U_zL_Uu%mdU z4CX@iW%eRk%1dL+9kCF^Yo>2h7;UOEIoX5T3*jnnYTR8+d&e<8(<)Dwqmr*dZW&d2 zZBTAp-%*aKR_WR19zbp-G|NYRSKn0>TgmQJ{6!f%vrNlBU9dRd${suKQu)V=EXOqG zjM^zs=86fFx>}-f=x7o7A~`xiVL_Qva5;1N+cc0zkI@aH6&9v9(5WHS@`3AVGN<@m21B|Gg9Pa2cBUUH#3R!1YcB`9)?<_R3!3y94qMb zPLnDaE10EDmnx{=(7FNIdR*V2JUdn2i{KK3t!thUOBf)93bRVg5CU@?CeMd8`G5db zEJ%VwcTk{UW!|-7y&99ErBlda3F0(ko3wZSILJ-=2ga4uVX>^zz|}1+aGW{K!Do;` zlb|H4v9Ml-R?Qk#Rdc2CX@-kdqOa61z2UbA-Ir8M1T5lkDJsz;pum# ztSePX-vT0FH~b~OKFgjCd(yry;i+9a)#-Ne1_;*rssBDVgLLR#Ch{kc*;+Xt+xD67 z7;8$FW$VSCjS*D;@O`n8&73WYlg<2+e(PR)G5*}nILd~%5MU5Hra)S<5osBuQ&iaV zv%b0VK`UE<p7DSHDfzlHovZVKnnz-05QT7d z$m1;To{|p$ipa<#iupC?u$XlNxu%kp5yic!%-(6BAJ1N`!qyp+EcxGbT2+%*IzzhJ zOPkvL98=VuUxRACxYN10^|x8~IRm(%@k5iq#`vA&!0o^D@Lye5n1JJleFzv35t<-H z%31OM4OdMH_JM!Qtun-m$p-Xv#T%(&;X#N86m?STkRtc{RqGMqp;G#VY4v<$#+Tl> zS7wf~^i6wat9zwS8Qa>MAD>t-a_3afl1Va+-}#T&kQS=ZinNRQ9IO}nf}P65{NyJD z4-k2erwm8-9xLdGKR@?n#SHoc0FIc%se?Keft!H3f^HMGJ)_=-B7ySm!aEit%b*Uz z47STdOP2p8Mcqpgc*L3oJML%N<#a}r0$~!M&ivb*nGg)&*oC!Vu(Hb$3U#-QhlgU5n8X3Pe~bMzH*M&zVZxC6D{jMFe`{Zm&@j=6e)u)KIZdb zkej+wmTC;+65l1|cYKI%$@A5o?4Wn5L|f4;ZhyIDVQs-=D9aBXpG>zq1sNS&BRMMrp7dkxb#{%P5i?4%aW z<~T&NWn!!N%Ki|siezue&!)F#ZO{iZvEQVMRM0zctZz%@otrr= z_*XNM08M@Xw+xJZ8p0+!cCR_@w<%?`qLTNrN+KDB0rMtE>y7?;Xn05=XN!LgXfC_c zV;PI7N__S@pEFd^_kjII-oH{H3nBZXU9?2_DsqTbC!*=h2C(b45Eku&y_m??M%Eqh zBmd-sx2zX!k$*}546{cD_gF8QBDI=7F=~?~SA3T;c3UcMtPkV0V52q#Q+#l03NR#5 zF&4>$kWP~RwU#~uvGu6kbSaB}ns6?A0G#`Xs9n;iJvfQt0omR_B%~Rg+!kiq7J=Iq zipnq$nAF(_0>8d*)kh#t4VJ15az6$2*zLh5ZYVe$;Yk32llK(+PZ&gAFb|6a-rfM~ zHaKnTn2Oby3Kvn)u1Cs0~qNC0Jk6lpfB4=oBDTXBaYnfXfepxxV`^Y3P-kpXy zOl}#5kJYE+K7dDm4~{XCw^n~TcY*~cT|gB^(Osy4Ly{hv|C&yXoc&^PhHiw2)Jl;9ayq?4OByjXuz7{3 zCrY%?lJHww@`jPZGdZqJYG}2<=v1-Bm1ahm@qc9MOW9?|a4Gn$AWXJ>xl{GjX*og$DCZx0+ zD2ZuN1Fr`XsZ}@pU>+nXgo(Zgv0YWCfiM&uW%QbpgB;z!wJxHjaPp$gYF+LJ;(96f z>V|WD>n)T@*-pAz#bPMkU#Ud};K?KK=u+<^x`u#UYboX_+U81t+Z^R8!#7tz^~if2 zU6xt;e)n%#T6l7BHCEg&d;tXd4AQSeatpaXe<=yJ-AzUWCbdjBRP5x;l%*o&?mX-+ zh9*s!yJwfxw3DIw2Hh5IJe!YMG0>$I&??vGbUY^-uqGnuep$=95`f62r;rjdK$2q# zsl@APl~g_Ha=X#QPm~&IiYymYuZyPF7{KfJfH969Mk8jxOHT|h3fDF4%zW{cj&%U9 zC5GdMIc%6!0CyHS*gU4ICbTIUtHO|n-Yto`oJ=x(G*K;>=?AL@mw38pgzLoSY2Ch5 zgtPf%1dexVE=SD=fe=rk0dLuj@xIM*lCmp{0zvbWBIkA4Ba&SWUGT`WhaegFBT#FZ-TDq%UL*Q&T zjUN4wmG84ooe^+!ZtC)!A|p4WP_HO3&3p!phmxh-2K@8G0Q7Q5bCd7pjf96-7;vxE z139K6kv(}xR)6%9iPLUAc(H!g_e@xv5b0dN(q-GcZFPH|=EqU&Nr-|K04#byHDRYmYTL zK59#=-`>aYM6wu$csc;ZLIkcpC$PzhIT?SJ3h@BEGY;O7YmCH9w%=rxz3BPs!)=dG z#keI_o>^QU-)fhqm8?)Pa`wbJd`+nJioEk60~1>SQ?-ki>W1Y*Y}(M8QVs$qan)jX zTHQ}9b$liUgK1xX=i=A?3RX8CEMuq0Y8R)ai+we>yf!ayE6|CpGnvqkiC5nqF{m*> z{=GtxSat6{s_@~viSeL*wD-spdtws*wgS8tO5nKu84r?R2a;>gdIiK$_qq+97Gj{K zz8(09XqWuKH=2#K;PvH-RQC0>tD2b`2X>%4a$H{)bl5u>T6$v?#B-sap&rT>YPZWh zmNA@~&s`gWE?Qb*6oC81wMOvhpf^^O$AS3^%GO{K(Lz#wz;REo+IzYOp6^&rje#(w7+Se!ajfR{qTLZIS&h{fMCI{K~w_D$AZPz zN>qr>`y7ZvIJJpc$L)`kUD(G`r>=ick8~;9wtZ{rEmZv{vPdDZew+SQC{vED z$S>T67^aask@gvrfo#q8%A$9?TJ@X7FGRq={F9yH6QY^%U%7``pMlX<5$#TTFgop@ z=z@}y*LTKSl_F+(=N{f$-vNuG-G4LL@l0uKq@_k)*6?x(u?+4K&muhpR4s-Ak0mme zy|!gdC?YI%);3pqD{;?{K0kS|?CSK|zCS0Fc!G6!vD@VOes}%Br|p|A)8-uF^NmPs z+k`Cmj!*e->oSEBC^kd_CuqOsCe1R<@ zxOS-f2q!TA()0xY)%Bhk4+%n4{YcjIl_#W|-dBRO`)Ad&+l;Ecc#Yhz9 zT2wF*1Y1iF7*)^2lC=fBc1idCYeR*pK*-j*tC2GkNzX_^S--E99GnkLNwM#Xq-72(;bz zxW^H;9}|^Zm-$~lR+`v9xs=xDg`a)^pThYVW9`x%%yN;S)IDP2)r6>W?&E81Ak8-Q zmDxw|Txie!G*w^A+KFWQWeRDzfu#o(@{l07LT{^MtDSx@^3`Y>XK@wQ1v6b>{jhjV zp5#Gl=N$+#MYKG#(&C~K=NrITEb1bz`AJ;U2a4B1!z6Wid=Ma6rY z62C`?1dw+r>0#j^_wQ5mt1+WQ!eL%f8-7TE*3suj?D}waYix&spCJZI72EaI=*)%H zjSs&y^MP8CDfC^7ekTjwcT^}&^sWf8(+`t$A3Q#5Z8%ie@u)UtOzyEgc!_6^?_c6S zQ;K(lP1-8jcDHdi;=QB1WI)_vtm{Au#N?aO>fB5J ze#Jh)?r!_ks0x3Ju%Sb?Oc&%CY*&f)N9>kdj%f4uY`G3K+70!M4YapoMnUQC>f+|c zgM-^z*CQf|4E$%JVTcK{C|gP{z8y0J3T0;(GO%H2Z)Y($9z35}*1R?FvY*Pq3(aCb4EEA}dxt&wEOzRY~^UL|Jjt-d* zy~RkmEV|d{N(@GXxFR|^2sE?P-ADFJ3Ts-FsS6;uRL@&;eF9my)f7}GvuRb=3-?Oi zuUbma{6eam&@JW@k($srRkjw^Y*VlQ6N2!TaDJPq=usz*n$>y!E%ebU_)Z#D$xSfI zCClU$sr`aJB{Z)CNk?P`_kcgI2H^{LSFsO}??qq*eolx0{Xr?8vl!|yz#~V{4DdmO zt8F+%n1UN!8uCh?A2|zRJm6rRhZzbCU3(?F^-n?YW^n1 zIqLXkDzIsZ|5p@&PDbmZ5VL!imKbWXldJA-rb6-`@Jj+0+77aqJCwqzaDpkuPO6wY zCHfdyH;)*1DEuDU2GNHM(ICffb}0Xic*l$`C>?}~xzV$)49T15CreS~!=9*L{}P5o zFG9B5`JrIiLd67eMx$Q#IWTcWcGkXS+NNF_e^qz*S$|czZgi%HF=83kUEHIs&t>8< z@kqnbbjMvj3WlKY(f_vdk>B^R!Erg@dLlE2!VGz2$**C$=2+NMPo;|3y&Ti+6EJyP zIvX=pwQn>!J9v)pc@WY51A7a3^^XSpofxzr~_aMNbcl6Tu$!taUNtb&)=U_a5|!HcE(!a4Ywiw{4+XFlV)E9p*bDlhiO zY&!28sRh?$m}QnHNJ)Ww6;g_BF)`Fvt^oL#KrK;G;C}yLefZWp!ZuK)o<@G2-E+ z2Gj)JDkl_!+Q`~Fjeyx4WHZD?|C=1Bqo?TKeEw9*u6Qb_Au}yuOBCHzVt7erP1<=o zE?@NL74VW^dw%Ju5gYFpuBZm}pRk8m~oM*vvK?))E*B`JSfK)T_S<4pT%rveC z8Z>QxO>spErSB$-up9h9R0IDHXp$9_N~OR1HN_p{-!bkXpXd4uTYdz{cvIQA;+&2HDa;4*Pe$^c{y&ytFO!p?cS_W*yl%72Sy=-8gc@;M@PEpr? zBgS-X8dlxm(F?*x{3G^kZdKiJ(+i?RT`jmbxGUt-se{zn^UY^acWd)Lte~T(> zjl-fLat6rgvCTs^1X%0w%|o~ZFm*9j5%&5EtkYE`yWbJ~hI;IRI78-!-xPtw*86aW z;{_L&k}Ucj|3?lyrybspL!oKrc_QTWeErM(8We6CQ^OvmH6W;MOI%nTwz4&D_{qAYnR;9fzELsp zq^l(!w;EtuItRP(|8t}ZB`+h2u5#e+9^Uxom9N;&YY7eYlyWZxexfjW!C>l*Abv0PH6++%{D-kH1v#SVkUlJ|8!)Ff|D?ap`zTJ}|xVXCIx z0y^8^#V7pGp;;j>2|Yz_@J+C~#)51H-s9}=Lwi*d;Yg|DP4GC(QNL87v&hz0e` zjMpWbwv$P28Sae&!)d;TphUgb#J5gSi$LBS#$vd`K-l*fPFoM5@-rwfJ)L-<@xLvK z>?bO>(3QBT62zCN75bIDPm@+hdV$b^Nu2NJ_`(|S7WE!8{^7YpI387Y6uR$yRTqx|o}w7bSSfS{-S`emf`KrPcl|s<`?gUCJiFwbdAN?g zJK4X5pFJiB zA^tOKvYg9nDCyE@#Jy&x%wZ2fU_x4Mz0c6LZfc;_VjT{2(Jb@`FaZ%f3ZwOU@TPb> zqjewU{_YXxE{3phkYzxFm{1QM+axnb$n5)qa25+$q4!ahIoTOVdjwLhFoIF7(dIDL z8g*-HT82v!yP=G2`#P(=I%H(Wik~43OBLTWbjBQuBug0>U)j>rEd1FYUGL+vC&Kaz z%8^kGe=5CPg;S_=oO7r%L26bAF80>vJ~ssx9p%XXf|E3@`&iPl0}N!cSzb}Eu8&_{ZYdNDl3o0oCDyf$i)~LuBm!}PDa3sDR zpkGCa_}R00T;;|`pM@B)*S*=)ZO2?^-t7nop|Sl^aMdp+&Erg!AO}CKT+o2@V>dFdrE$RU6~U( zQKm;X;(X5FsUFXlq5ogRYpfR!1Yd#Rj;wB7xsA%AvpJ=^BbtBc0;>|bk51%Vvf>fx zTSn}u?@71$|LZ({s>@LiHszg(|6Z3BGaML($q%3OG%%pKXj{|ET{H{Huqf+t2D3 zk9qo$r|34)h$1!7(W+M z$7hUu++!VTl6GMHkJx+vn>(9$Zz=MUxbvDb-q&-m1MlLLo#&@thG$}irtnzXFxW<^ z<$Bx#Rbs+6*6{SrO7iRMk$xN+bDPjat4od<$2zKSYvJ%DfBrX=wHTl|<}M&Es@qC% zA)1JWTC^ADLopj@`pun8yjQcO<&rTH?YLmi$Twy@kQuTe+hbRx-#Dedd_aHVgyYH^ z!I#;WIkqWlYF||QM>a=1se^G{4B)fw%B0+s#k4O9} zpXwyD{2WPHGUmEJaVEcPh;@JD^X0Iht>XmYt%hW8*h1rph*v4Bz_WacVkTcMHW$OI zkKz^A&d*XUo3_P(Ys6RHW)osDgnpM$kkeKM&XN(EO$BgXj=WJMNh3j{0K zHxps8E8SF=^SzI4m-#CMw^AObeM}ArZ_W?1UXMEb9#4HdZ(hew1704OFkW8PV4q&? zZaWaM4Mh#Z?`>J`*m0upbK_==$%VCF9NE$cyrb0Dw~iVPgvA0` zwf9<1?AkUtGhK_C7^)kH=d4VZS5}sqVjOh0Nw$2@jH-DAwD`RQDBZn|7P=gNn3kC~ z?~P|Q>1em9xOsa1yvW*VJ8eGxV);0_>;*Y`Z2NfKGvn&hxRH6}oBQ47`O^l`7h}y1 z`Ns3&^w;Q(hx}h<`LbG$)4H$apP3UoNdIQ4r(bqXO@S{y%73@r=e4OYalFZWG4_-9 z1Og>@&&7Y)_RFsF%x7RxA3y^s=kKoc9}1~;Y7sc64YVhMb$G%N5OroMtIko*#m2O0 z9$R}VTrIBL8SQQROA+m(d|7|xe{<8Yv#f;vC66u2Z0s%g`146fRwatuYp{yvkfUCY zL&Q6RkwXzI?kto{{aB1`cT*8UZFdnKk&uR+YRuU5D#t*i+NvyaF6LRa8^VQCW#J=AwA&@aN~0f~&m)=~`M~j!FgO z0|viU$oNQ`c<|_el0C#d~DD?Yan_F^MW@#8#Gn=Q4icNKoiqu=(oGC!`(c>krn`TVR{27LMN}cB7_L zEX7B>heYXSYdu_EylkgJPkB3{8re3_&3pDoa{{N!n^UPOcJ6p?kVr+NR6s-4w(4th z8M-4^YIGG_^eY^Zzg$1(t_DfYT>lksn2v7Voch&6-nCQ~%N2o0CHfTs8L!RZS6nvU z*_FUHo+I3rz0o1JG%}Xrxy=z>y~Kw1k9AS`0+eNB6S2}r6P3?Nae8V z<*<_$Xn#3X{LYl*i3^FNIc`+VwLZqY>-QR{Z*F%wJ85YsBV@{<(HU-6!T|3p5_vfu z=g(dClq0DRa0h&xyd9pajrW_35w_#`LOs z*OAp!x_bB0C)y(D1CnYcegv1{c|*VAXqsF+vVux>Y~2|*gVU4nwz7E?BnD7I~Hjy)qc8Uqwl>X<}gQUv>`78aF=}*eSfm zH)62~edYph^X!h>)E^aJc|QtL}oJmBRI<(%qXk4M({mGXJYXnwg8d`{(m&>>BF7QFZTme}ws`|!BTla5qZ z<9yKk7l~Uc62E=rx$!bqE=fh+hb38AUlQEgz)q;a&J{5STK09<(9)dKEY0BpoAX)tv zaep=s1_ffK$|UsacCFB<@S9WKOzO;^yY$J6KUyi`m)1tUjZbMpJ|gZg4_I8wE1vJy z)xQSc$mP;Xa;vJbBxR2tnfOLlC#akq5iz{RzfAF*-j-hK9yAl|-(IP?UsaP(KXPr` z8nV@1;R$?QBgBF}<}Jm#aF8s179YB|72Wsfj@+&PWto^)d0aCd*faFeZ^sNA=Iu*A|j zn2EHdeN=nSoS8+a;gZXXE$?8ANq>44bKr}K^-}X8QhSHnF>$#s5NyiVbmAJ6?5?6I zw9nnvAMVI=szkqA0FU~!hL*`xZ^1dXUGJE#Lg@=@e+-39`Zct>ar@J;UbL347>;w& zUazX_?(pfPz}2G%<9PoOIp>;j@}_jg>m+J{2l4Zk{xyU*Gz1Bx;D^9 zoGq>5-z#b?G%8Y~_Ol`V$|{+RWe_X#TK4L$MBtvXt#dxx=$o#4Ep@82R|j1nrz`!2cCBZLAau1Xo z8*6Efi6DmW3hV_lCRSaM6+GBVqWLT{+L-wzAOm)`GlTr{Nq-L8FIzgaeUFhg)84m) zx3$B*oo?SPHp%k&DsPz|YEV3v_bZJVv1zo>QuhZJfAiAPnELE}$c5d9_Z z__K;B9cQr`nw9-`o1N3{j|i^WHMK2%nc^QgT@zW4$urr@TuzUGC%hVj>-`uc+PZOa zacY`MK3mn#@Lt2H)@PLy(5)hajp;QxRN4sLTfbVHoHSH@29;YO&u}(K)Mu9GE{^V& z-c;QWd&{0o7hEvm4t8j-+d?yoiilJD_C(q3(sX~rTIDdI1UDfuH?u_KvYJA$YLc*O z3bJa3fRnPpD!|lo*nW0HN~;uOdc7AN7Mw|kx^s{^+(=vWX;Nj!j;`2Ur&`zWhUk`3 z^b=UEXRE#+7gSiidWS%H%(00t7jRCT=oS}Swj7CN(Y`27d#$Oxvx%dpVrP@T?238X zeCYd3>T_N115l01&0Ea9-MS66>)%t@)WYh3ul)L@R8rpFD>(7@_1>b6S@hMdafAF;3=so`ilS2e3KL77|@2Q12WRIRmml_M;r{5p?O z#~dFVlVWxCkW7q$-->RV``ijFo@-H9k8}(yXWu37+<&h9)c)|BN$VB=!NU8(3&pX^HPnjS;p}9S2nlNQ& zYFgp7S66n+pn!E15|lNi3m>zv7&Vycq+fy@@t5k5Y! z-~RJkc9X{tyJvlt(xXxs&$Ugedwhq9_IbOF?f$;vOd^G; zGIOAbw4+bWcl%JT^9C$gSw77M=GE!fl~^kfJ3}`vkDw*~nac7S546Ac&~M@KDG?a8 zI1Eb6-jzHd8gTCT{2Huf9^1qiU&QF9Ddi=#p$T=?(Z#JK%736vWDU#s3t0EUQYR=O zVB<4h{t&i_DqMmK@;AF`v0xAPQWKy3J8p1l$mZJqqwS{?3|Pq*rZOEROr zGTNd&zdS7fqVuq?JRE?3U_+7XX9XTA^>U;w zPt8&X@=tx?pGvk&`OG(7+f=e(fx-YbzEonWYAV4!S42Jk1==il*(`wAESTw)g6fkB z{D?q;0fQPAkS&%=tB@i}!G=5M&{vOhQE{MgvZ+C`sgbp*s3(>QxRKGrHKHgHYUK zSli9=L*+7#cR>4)NT|fo>&Mu6BJY6AA(32(BS<21i@{kIIknF&wNHaJVncBQdIafS z>KcP_vXacclDuBBYMPb|W8r<6m2;R?j#vXoj8hntu&7wzCXu-L0T>>tOv9*H5G0wX zsB{>grkE-wVT?hR_?06yU9Utu;B9^k28nn7AYdtGT{;U_kH?!%zCE8Wlh!I>YPp;* zlS3+@Y5O5*pG?~)2f`vPQC$8ZS@PEFPh}d8ACm8qX(`lpmvl|Iw07&Gr^(1zlV^+> zONX^!?KqCNm5Rm=eM)+4lQM;>NhC_QS9UuVUrS&dH3wQ#XUf!v($DnRjyy4avmCE= z4?9+Tf%95g{G;eiV$G4Ui?_CqsLw6E2+v#snFE^YL(V;^i|fvXCxRnKj`GEG3$9O6 z?RlLNOppC?@+%MFfkABH){m&Gfq1=7aR$wZ>vafPqM zu7HBfenAwbbV|DB8S2xaaKmAto!Z$Y-*%t++tSKI(zm@j#41sh=^inmnf;#ImoWEU z6J>rcWHZ3LlP?sDGrW)ZRU}@B&8wWyyL&l=gIsJ&4OKYv{RDbk_$Ir_5%;I#RrqnS zAw*jr@YO%hB3HyE1C3VJX%`MC7YpEFxOXYz|>^8e+vK4&7STi=J7-i0Z;5Au%4T@7n4 zcgrc4h8HK57ALhde)2O@EjCkaG*hiIQ?;IC#I3nds_0Q>CRM0{#VAS3(?kBK!$F}l zL!l#rQ7Igz`n{Je)Z9MhE}M( z-@GULxie?|l0Q2Hr3#i)7#;UqK&YHG#^Gw45$2Fk8PPat(ENBLjJrGNwesB+Nj;lj z_wh-)mxI6|;og_moZ$%(>jj7jca4ej5E1zB4)?iFrq_@XT!t0JZ?2b9?j(o2#{+&5L z+$}x3icEffooRKKD&q{-_7cgN$42p{_9Sm-)t5~z&7S8E}rj~lku=6*tLdq&}Pt9iuTcD zWXLRxZ)zwA2)c(#{`T0p5fD0^wG$*A<9WTjdl@5zO5+{`=Dqmmeq_LGdU9^471^4$ zC>z&wFtVevzK$)n$=BapOe8uUlig$Vu5{Nm9lO>MBWGwYoz!sssZq4p|6mmWyZM9r zhtII;K($$nhYY7Iu4$u~3Rhbf)~&mAViCV-G=<7tH`nal7^=@j#M;wFC8t@PiDj*I zLywE*&pxZlYmICQtNFOAJ98FeRFa)KzH+4iw#dfX*~76|rj#^+l74RY@%K(o(c(HFp3+r;0Ox8iC z%sxe;NC*v)2tDE|j3PSR_@NJBpciC~Q*<25ZK&C+zC=4Vnd|v`N-*q|R?8&G6m*V= zqpp%DbeRY|p%&4>Pg+xqqNSxhrJ zJBK?vHM(fKJ3FZ%a#LMd#8^vMn3>OOSy@=Bnb}yF^z_t7J9F0i05BAZ5Q0ciNJt&( z^Q|vH7FoTu{zgfEe|=Vo=zF>&>GuVloyY)%1(ZRzjG2T`M{FQba7XPshybUF??F}S z?-S%dG3Djt1q5X31UP-F#7W45{Dp;%{*4;#eNK*^2!+$9PjKku;jR3p@?8PEv^mgQ zr`C{ck#}%zrem2+Eo^(ME1vC8K^r3_VenCEnnSu@xMOqS^3Fz`L-JmaMmc))VIJ=@ zm?=;QtIDJnnDGN?)gJVpwT7 zk>py@#?=*Ejgb3RxyPduq`FAA2fWH_<7S$)+`B%9RbKODc~x}tY0plMbIBe73k)yU zf}eYVw=}ytt5FVy+HTURSE(4|fD!$B$Gnh$iECZ%eNqC_|>7?@Fg%}JZPFf^3E2CYu1 zngV$*h^r4b_IT6-OShBgNMRT3zD7v^4X82Q6`6+Z9kScvVEFGnM%uX>@cCQj8ofU zUz(LFi!*T!7#XI~^YF_aubNNh2t9;HJ_+Q&)oY>B$)K7ClT1oF|Dpx4YGl@HtkkP$ zToglbFut>b4QGInCh;g?P3F%d(#`Y1nzxm=7ZdRKf{3;}J2cV}yPxv!lm8;x4 zPKjeBd3}`q6df0cbFEfwpSDOBuLut-4IgKWyWG>H@lKMiPu7EUg0*LDy<>N%j_r!; zrOwCJI9)i-CB&RE|MvK#$on>aNIoF4trIoaOn}X#L4&=)ktt^cr#xgbcmEhBfu`RV zH!@)IV9fA$1-uZP z)E(voqq@LAW+&=W=#wFAL%f9BMTngODZ^jE_hhMWoBb*DllLj_y;aY?f-s~=Pv$$!33Yu z?Tj14N3K5uo|vX*(=4t^Z+k*=7{^#k4k(>@!OO74(BrYOIGH>~eJt;5!FmMOKyh4U z#K&G9C*5CUqvdKQ?oSP!d46(n1o(w)o~F8_CS$lWL0^$NMTY)O7|z4IKSGQNgFP^Z zhrLLnMI`QMw#PV4@Vgw5hiH96z^rVu`NDkm+@mSb0PNu@!pax%)WdY9(U(5sroIEM2@G|dB^j)29zOtdU^eP7HLrlP|HXvz>4CUl0F z$f$PnG+x!qVTs7d4#J5tr(iiH>fx`{(OT#t&$^C|g)D}RHcXOZ(TNB43(bDa44psk zYeXYxS=1TH#^}dOK8{?-ByOv<7sT<8T+-qjnwJ=5>@3mJRC#@&z@!(F&-=6s6)H)< z?^ImYMIO2egq2Cw?diYO+iGdl>=aMpv@&vs;H*igirrzgGExtUqF2=;AGy^h9|6t} zD$*!S73@+MXGR1KP`dFFo00Zit0-(I42e)_i=lJ%(WHL22VUSks=`VR<<={BfGL{Q zBrkUJNcE7a14&7uz~@_1l3J5DKG0bQq^bcZd6MKUH6T4A- zqJ4t}uhH+$_j&y`^5`61gbSNpg?gGHWTbQk&U^cFTr$&NvH1rhw^`eHsyYb^DL2P2 zOBqp=nsR>>+f5VTf$OW{l)*6U>*bR!kNY>fBfl|ueqsYhsB{i#n^jIkSa1=`1ja-( zXq6$}?vbBoZr0MmD9{m!|1mei!p_{In~#&=r8527q)7YBAN!5O`-*b$iLMXgu z;e@!)r!A*uWpLh>M@W8y+B)%@l*8agj)3Lb^m9G>HLi}?VdYzaD=OtdesUE38l8## z+V!mvDlY9^mL`4*=I6 zyTVZfY&H^S^)<-|W4`7i8=ELNAti3!z7vAVKBYQ0B%R$lj`Qo1eA8F^36}{Q-Ile6 zZiJ&|=M{tl$!h0kp*@|2YGSwtZ_k%wOsVE(UZ#xOR;SfV9h|P3TD3bg_Y`A&ceSuG zWgMy4kd`f^6iFJ&kDXSvh3Hvsr3)YX!IbGARtfJ=d}<)`!TW))6>wC$34Fgyal{FJ z>m;XsXTZhKXc@;=Nx@4+%f*C7mOP*8;vtQygA1I+--ZUY{Q(vnD@4juWeT&tqBcYSZL#`6{;+Rh*gJ84BP z$n!6pULsz6;pjx@u5BmJ32!+@|$(`_3U4>AO(sB+KKdTA!$0_|U069R$zj7=}=})4# zzZKYm_TbI^ZZFa3cQ5ETy)+9#2CVjxuZZu+vn(eK zpK!;iL+MFyK=5*JI1)n*hq-`+Q>8Ru*y*h9GGC-lz!MUgt>2zp`|F*>i+&B{$4gdp zSBGbCl@fVGZeQKv*YB-Knt9J2b(k$o%^@qh7c`&#Wa|~+{a{cVmls{e7ezQDS|0oE z@VtwIS&$zP7wRaDOA(7jI`CVyJyw!;L5Zx?|4yWtl?_7s9cnEaRx;Fk^Cz zIG}}w0cZBP71;cxzqqH&*Np~F1I;4pR62MQ0xMteD)q$!l&tD;(c_f|;lM``2#Pz` zKTKi};uab>c62Vbo0%UduSjYHXvHEx@SnoS0aud2kb+{TVv7~SZ(hV*3X(2RX=&S>o4ITe?|+l|2eKfG=+pu9+S_4GU8G zKP$N!7z8`yKY)HfuLUw3%AXtF50^N8(<-nz@Nq(j`4kXsA6@~D=6DybzNc#4+XpHa z{u*dbPuo1N!IC10RI5EDJ;j5+-B+1B=TJ+0V?{xD#A|XThMo7nZ%eW%S|0w#wr7F+ z{pGs2oQN_NUo48WgtVUj?%;waR%HuSp-PaWlCVa3Pv(vC9?}8-xpAX{%)4~8!k=4l z_2SiX>`FOOW!OFbe)6z?CAJ>PNgMnf(z7^AgQ(&dE#|<_kz}AD+_;FzubQzd=I6}^3Goj<3|&LR65{zFg!&u$7r+VpmL8jBgy;an)eBdVc88Zo zzNgO7-|69ZQ_nRb0Dvhc*m>dVg(;`?`b;$Tdm~?Ae}`iu&5ULVuKhH?EPga8Zft?1 zc;NmHZ;Mle%awv)W3)cLxzH_Gas|9Q@bt+L4M@$E+vdnfB_)6t4-0%N+gf++_0T&;z7zET7NknvI<#QLV;3kCz z3k#9^?{&!dMxDPwrNB~xRsOj&b7)w2H*H&377m&Kyj$UyT)cAZ_;K>^5J}&|h4*ZP z_sE!gI6iMWfOlFrYz~Rz8{z#q&r31w>G~_BV4j2wrp~1tKaRz;tqBpzr70ODn4MXUDu>=74~rDTgUSe%Dnf~G zR0_wf0~HN_5?lxrt%w$lPUJs;(ST+ItA$uEhGT0H1eu&DP@1Jf?~#Xv%Asd9QF1~V zn1AXBnM)pGSrNa-2^ExE$oa`Y+&FZMc<$O|mG(TFj;j$0EvAxY5yyWJ5ZLNd7pDhn zL|7ld@l~hJIie~zwZ+Ti#cGjc_8cFPq_Hc9z9SDo1YQhwstDrx0W?P=Uz0na?oqI8 zqqbta_Mt7tIMqY$SnxVD8ynu*U*7*Kpm<=zTlbc~lrVFD%aUL0D)-I0w}rMIaZgik zyg%6b@TXg!2JXMCXgl-woqd;9+e$t|Q&opCC= zFE}}*_^r|^8sk|PavKD4g<3lOU8@aZa-@D=Qlldl;Wp!VDt=|!2ODVY-Qs4Hl>?i{(s`I_dLYn%-XaB0j!`NrHCdOkJK24EZdo?KtxY+P5}_tg4A zSL1p-B^mr6qc|zpe8uo*y2kGhEV;TqZ`LzIdk_9GISkjxx3)jwzpv-QikyxosiVD* zt;i+@7?19+3s*It{Ojgtf%l_T_pS!`VP?G%4Yb`4LFGKxtd~@UPvYxwuosVx>TawhAY4)GNKKFL+s9o^j?x~p1n7o8c+rW=*^|+HBUt^M~&3;n0Ce-HR~GeTerTD0~Gxa_j%&m zx7CHIfaf8%g1RO$SvC|w_I~858_4fl=zkY`#?`R8tnR>)YNgr+bvZ@^EBz0XwC^Aq z`3nDk-dohu?B9@pV%@MiHp7?yXEi$fXR#Eqp zTPnJ08w6sz&mL3Om*p#rvshh4bIR)L`t8{gim#(m(dWfa|9akiPig z*;6~o-nkc&ej;7dAKO&Jo62b(J{t67d`4m^)p-25`>1+3 zAs~i_2x9mcbMG2>?=g6a*V{rMiOPrFPE4>0; z(ZIg6zC(@p?BRohZyl|#Kl;`ne1E9%(A+e7T;ksNpc4CC35=3a66 z;Hma0oTlQGrB5w%jDoIYkRlHcAMCxlze_pUnM&@C#2SwNe)%Xy45kRLK4@PHAYO77 z3r6~k!n|S5$pU}ES?a)nU_Jg7S4_g-7ktLG1R>K(9@Z*`Sz)D7QKex7OQwTktz@w{ zFTk;)=&=GZK72EA&}1+X2ImF;6K7w+Na&)lDaVx-w-by zjTcg1Yz{ysjR!_8JdGFj16Y1Wsb*8}aRgL_C{+F&mPlPZtCVvAi{!Jx9Dz#mGENQ; zL1W)96Z;=iN-;lnpIGewgeT{N?<8VI4tpWIe?fQ(#%E%fsvaXBJCT1SpW1XfSjYLd zLLk)3zzTse=qQ%z{~4Byx58lMl|;AJqsP)@AvFG!&Y@(nT1f0ib|h}(8vY1@L;o@wjzXJxLZzg|Y%KDj z!)y)mg*@2T>S$HU{Pv2(up_apH3hoh@amSCzLfS{tG#;Bu2YSfgDK0J&B6FG61U~e zShPO(>(sve4b;CgW)(&_n>N-qK5}Svi(91~jtSQyJL9I#;g8br^CrVrOtlu^b?9&S zCZUF1n}k;V6iXv0COKRtR7`>JBy9a3PhKa+R9~Emt5^K?W-_4Pq=50_Qi2WrYAtIz z9g76;_5rAaIcYf5vg;VD&d1-X`@c}3tGd-ak+PPjWGJTb@b zi!X@{>hDjhO9&oVn!mOql}uT>%ou5ui+z<9(`%-+TCy74iF0avIA0++dQ_=5>yOqbsG&yvt6Ga{4oJTjnt-ap(=91(nRZW?zy}4X(wAM|ZPGe~qIF)qHfwJWK9bCa`l+lvdjwzv~GGJ!ZGCexj}yxF5m zsPtwxrA4b0lBW_&?DCYp!_9*qpP!YVlg;G^hXxm~yWE}Dm2Zo1M#vcW&=w>3`IBui zHYUvYZ7ecAW`QGbfkuq8wx%I&YCD`zw4l@$AOrW@grGORz^=j5r!cy!`PseqOk)UR zd_dpGHG&IyDn*+6MhL)2HZ91$QF*2XRe`guy}t5Q4X&DUS9?>r9+;nOF$XEEjUBx{ zdOqi_&MO{_&c%bg(XIQRX&6|U(z~|~zj!@8T>FQto|4GtpQt)@U~Q{6*^4lu&QnvFZh;a0ryiP=TVqYXza!6f(3A?HIj;OzM5`kj{pXkjylwv3^* z;-62?OZ0pn7Up>OU4&P{$T#@wU{7bNSiK{r-ZV8hrB>4LH zrontgY+_cp+!m(tCwPkEG)cXOTFZ{^SXvhslT&5Mn%9t}guck8owW9oA^e~%WALk- zgx{zicN;8`h2yx#?{7>*TS(imf|S~=)7`g5Jf?Vi<-}atBvOW&kUdN4j!5o3cpGYb zrA>-Sdm(^TIl{p{BG02tGr*Z8NXv)YhZMVp!!{wQl0erDZ0~XkqOG}`I^`;iw_7O4 z>{wlt8ZRRp=IFu%8&}|$;WLI`dNbTc)QL!?s-V4YbBjyatPDtGo9Tc!=WXp!sxj_9 z0Y7KK(e{~7=SUAtHc;4qZniu{Y)NEkUv~YP;TI8A^+HmiEUvFPFh5g{4e2LZE+os9 zBW&(TNM9rJv~NcridUE`#;JHN#;k$+e9T7TO_a0Mx1v0fp&am}l)u|a(@yo!6PLH8 zlVO4&W5%k2G`9>FtM#&|e6NKo1jiWGxH}pvOtJuTQX%I(t| z%JrF-PDV|ONNF!}a-lGPJ(HftZ5=8#Bnr;h{J@M!an+J#?)F_x{y}Wx(nYb2Ys&qf zkwqNwEyNd11%oqp1H`%h4mkgGab;_7K}=1=e?dNPN9|9O$5HuXeii zfH#b7pyfo|o#b&KFVr`@hPTFc+U_Kdp5f3NUc#t;hBP1uZe=y|kZzpJV#jqcxUMw^D)`Q(AUnKGR~$0fZLU(+z21Q&RazRk{*LfD*+Ya zFwJplnITlF;RDxUwVp>&Laok}eDR^En#j>u0*2~SMFqph1Ts)3Q&0=da$%^F5=xZd zewB#Q-3W%q=;7{#^t6tf%Vcm@G~8V{!2t)rapd(MY4^@-tv2CzrmikB`9^R-b18Q# zHM@(M<}M}8t|_MZLna*HvhPUz-2@j5ZwMRtI8#S6WgdfJ27CN`(#{-jFNdGicW6ez z+VrKDAA7*w8nbwhl>Nn|+44<;_zJ=GMlzh>@7wHK6OgYS2s2}BKGezJ2AsR)`Cf28 z#9;2B8KpaN+AlqJ!rmIaWRHZ6V(TF$l>!x9CljtsM+;}yTxtM7A{w zr(zqUkBRH#c{*M&Dv%V>JG(aolZ3~3So0QB4_VdS_>0v^fEfJg?|K+B?{`~N{SyDXAt77 z>0&Fwlbm3LS^dql>I-DO_)tGE$pM-!PA2&*tNWI8si!3DCuTtY44$AVjsZd`^=%rW z*u_xH<&hIeu4e-7P1Kw~fZxaMd!U|YO{{z-vI{-Hl1NpDGh{bEMbhJ zY&Y^}CT76F)86VtLqnOjDp6ktG<@!cmsX}Mo0rvB8V#m#Nb_oi#uQ=FDD-B%0%R>e z-&eP*%ja!gzIAslPoPj+TT_y{N@6_S2im>uHt@Q$bb4%kV@y%2yRnrrgxHlTfvY5@ zpv0V1MktfRh3fe9xQd*VIPYlm$Nyuo=cc=kh5g8Pk6{Xc%i{=#- zZ12m`YH7Wi44Lp0`7&M6IaN-^4N9g=tz5o5Wme>66l3VonCy31GFBvur9dL(b8F~u z!HRb6@hRcF$qNy#`}W)`>jlBwSp*x#0K_c-vo&na?|;uFXW(p%#D(ef6J0dx(y{!G7&-J{3`3IJ}XFx!^ zvH1gwVw=`hP#7_=;Lm@~h=nwx*zq#dhfHlOZVs6oO@O(}FD(B?Wwr-tVvI_qDMk}y zH>vP%hXwezAWdsGE0t!uR_idUlx8P$Zs7OSIeJVzE0)D_-@5WuF?PO#0r&2dg|Jo# zKMTPehr?73F2sm@9O9rIj}y&LMB`liTU4V<*B&5>7y=l4%c*<8V-R1a?r@TKH3dU9 zaU8_~cZmnxrTiSh#xGRnn4q9&gj{q`P>flLe>?QjILEKBlE|2XTY!d*a8pd-mre<3 zih&W0iBM))mfMa)Z5S}DozO{vaIRNWri1VFcb;lWGFk~!;u#QnH^kEV; zgmnnP@iaQc>tC|7^=7d3qk;>HV7+#P*I(SN-063ay;>L9WYj)JJfPv z+PUD9r5AGX-*p=WIes?wbkXb(Q6aY6|2myFfXu<3ruR@~lXpmD$*teUuj_I0( z@zXa~`zZ*Pri?+?#8ECjjEy0F=e&$JY&3#iUpzq(mdIn2YN~{M_-dQ^6SU zUzSyGYK`ZMrm1Cu&N-D>@dQ)?^%Kw0HqCtFBv7FAcH9C)QwV-hI8fu5H>+r1nqzJ~ z+JUXu@q8csy8VTf6&ugD?bunq0sh^!J#o%`oohZm+YVnlR)2i1gV|f>jl7EggRrpp zEF~)*LQ{{G`vyR2#)=e}O(|4*LnEN^(bArtP+BSTTP#)#;O?a$9$(QH(Pg*D!9FWvaWOnY(IbbASW(~yM?dUn(ybW&pt!B2YbsbBN) zx%PCsP7QHO9JxkbBS@$dOsu-VuC;euRqu`6RWNP`m8r3rV9Lj;Y6uQETehUy$sy#9 z5>MI6dZ*G+gcl}vSNMc`o|Zv-8oHaBy3Oegp2(8yq*#jx&Ap5|i~k*rc^` zf)EaI5C>F`CTBUu)q|;TQIO`qe01qoDqZ?jUgBzO!g;v1v3F}$=1YPeiGSjADjKa~Ffuao74|Kh$Hd9?Kg85E!^EhpVy|(fLb1`A zqXxS7bOU=_poFq3?YRW;#+?d|7) z7_D#Tj1Q)M)U<6l)l$E@!s^Ve%5_F(6lX-I&E8qoxGmPj)5v5dM}{jZ->q}xROC2f zi)xEvm5##rw7!}oelHU-^{F%j2mhe?;WfqH>Mm!_^a8It!D@C!MyJ+i)^2Tc=X$e* zVu*dHw=l|;?udvlNJy*CP0FZ9&?UPJ*^3U>3?HIHdJ(BqK!UFWE#|3xe zBf$|t+VoXLVTr*~I!KMm^@MZRFhT4bgw`=4yKZx{OTe#3X)A4FQ`XR5CxC*m3ZXnd zK&BzB;7g&;y`Z`!?6%SAcAUdbTOgKU-1|_V*#bjr3H2=Lx^0fF&8x-kPORKvvJ}cv;iQ@hF{Zpl)?qyIlTr%=3kW3CP#{@ArK8+}bIR1hT!|dx9tor& zUcQk`k)YH;VIoQ$j3$F_ojXt}a+Gw+<-y6x6pU3&og;ean5G5E%fvuS!MWYm#$Az) ze;E`*hh&;S)RQaek*MsJAe&L8Qwm5iE)Q*RmQ>ef$MsUSyJAH2&EXqmUp0Ij277LO zm<6yFHg~788cm%sOUYgzI=NYaQes2iM$Y-TqhjH(6J(XGnUR3<-t^e&in7pjKl z7E(F={}nDUG*`p8`%U~?I=27=3r`>+Odq>qq7McB=JNVXNyY0u8SSZRA<8c#mOI)T z%XR4}k(Tl7x{dDkdz!Hg{uKj@;+of2U?`5Lrq0ph#@+J#yLBp9ZX8dWLf~2o^iv{m zY58Os$yyL4iJAhXVF5#L9*c#)2l1Nvd>8&6MchkW->nQH4zRhnKq~`@T%DXr7^)@y zMb;v#rO1ZukgDKVWh@r?H2yv6ZG+BH+0Y3uHV2iK+~J59`7~9xIfy-o;g~=~y(bP? zcX=ZO*@6*3pcMlZsgo5c)L1{s#d#_r0Hj`qP%vQ^WT7&n(O+f4zg5coi_BSJVOeHu zw?wIw_~)B5jp!4N+sMn1&Qp+1?dTaVJaEPn z_?!Eca`ria_-|6}2`tS2p;7`4sN~dcu@WI>gHY7_)u|VW<%d#vh%` z&kz061bX%Q_oyHDsg>kOl8vF0VHsS#1poH>`zj^ahd-;5`q!zVRH|qdwpyxyxPJk0 z&w=Bj7?d;dfES*_$7BNkGvFJk-^T+|5mrp8AahC-X0hM8y+{-;!9Lglb>}ub9a~49 zWeMmr2>}VDvz3AT8A|O>bt=gQ{*%>|sErD9jkn{RhU|kn-fu-V#Tb9IdYX; z!lUyI2k;c^PvjZspBbm{&*SIyFbghCi6j7nbNM5ikDcPA1n$)+n0wC{4C2d6lp4VZCxx%3z97XQlwL>R9%4Hl8W@|- zilDF(5B-l=t7tzdF&fCT6eST|(?D4SXW4xGA!h7=oS=IjqMnomD;NDo9%kUAP#=rg zK(6M~YxEsr4=!SQKY-DSE!69VuGzvxqxDuKmtZQch&@d46v-!;noBsw_i&78<{T%~ zYdEc^X}L_N?FhD(k)>gN2DrS&`lcSGQy4Vj!4p8>nu+Qh2d?zt2`0CqVa~H zTIpi{gB8l`+uaxuh7hA;-mfwRORz37G&VaTGA-T_On;sU!DK~5rMn&M)iQm!Ji0tJ z+-Nh((c238Et0M4Y#Cn1zBbxN;SLq0@5;j>HR^~^xjZyNt%(eigB9}7NR1}aAeWOJ<1B1i|m&{XvsHGkWzJ@g^=MbLf2*xLg4XlZyObk~xdv{F~Sw+{r#Z z_MH=B3H>ANv50NS?k09b z7q(L%nKab&d4Fj)h`c;P4jGpt`LSw-*+-l*zg?ke&9Z0@?v zmc$5EWzp(wXgz{_q8EFG{0rn+XuwN_8X|DYnN6z=q#f9xgs>*$S}d6kHAz|$@~xJ1 zyLtv09TS;IlJ>~x1kxOp6BiN^mlGA4;|d9JC`4B(z&afqVqC7B@(KPs_C97AP25lKSvuFi!(MBIX2eA zf0hI*%N`U*8o(>mh*Ym=_=P;uz|npy7JZ`9Fn>pN#go{2Iv%}ABC++8cNa1p;Rtc~OJ>~LD+ci7IGh=gA*ZDOCQ6)VY@sv}q#|2=us$`%#rsvt~ zyOzIpPg&-`L*2(`gqXl3dC}4{^BcFdxw6(=U0%AZAxTV?n?hzhuAa7J-?Tl~9-UsY z<@x#jKR?nJ%*?HlI!maa&tReZ*U)|?y&+;#vuR^L>STHYZwM6^&q}oAc#L@F{`%D^ zfFw=zRttIT@p5iOebq*H_2njBYVwK#jV&m)#9cqV+E!trK8pw^S^hOM7q{1A5Bl|G z1vG5l$PYvx{yE|gZQjtVqe?H%4M{MvF=nx+qum7{_nA@p$VhbHM%^ZXLdp{cnKc$3 zo)_2Jtk8$csHcNGNg3G%R$(x70L2PjR!+A5H69QtLS*=5{9uqVwR8T?&ip!;v_Ky^ zxF&VQ-V5^{rG9vMU73;KiFwOP&jRVsiUk^_kU7&@Jn}U-5B02wImIT%k+p%o{kZ8u z=z;TjY_Uu!Tm*2E!_Iq2VYG*6?IyUg?Aj2!Q9&N|zhbYcZH|v`uC1|SY1jXtb)at@ z@LQ8YVX`Aw$49=#?NDBrBX7P#)zhs4Y(wSOx@` zr?{>-wyiRuD%mCyi8%^|5ve@}rq%&5(&*zOG*Lxe$tk z2siMBL&tAI5!G2xUjsesU&yX&m?>g2amkqQW{Se1A~*yuJ~Pv6aD+;OWAp6Sgi@i^ zlN$|kz0NGP&26yHXlu06HN|keqxd^;t(>x=J;sMveb^TePCVqh1Hw@-`JVCA#!n_? z?oEi|==WqC{d*hQo0&X@PoCHf7FA^uK%!TLPF zcLO?NYNLbBG|XzVdVM$J^RIE}1n1Y%=s*}~91Mfwm>>)m4M9W8PdRHXcnUwkl0tgf z9MsQ(EdjU>Xbgb!om46i%;$?VDNo}-6R#p-*28Wr*SM??u$E(y?kVpEFO;NaI=BwMbk^UbZW%SCGvF|v{w=($e%69l4=^y@- z`e^uXjD9{navD3!z+>8oq4j^dFf5u2X65VRxa1=y{q;l>IzGdikBPBjjJ9D4=r67V zugfVPc49C3?sZIpKZETU{q9fLhZ27nwqqspI|*5Mwf`TFvLJgis4OTh!$%`CpFZ_KC^*-ii)Bb}3SyLoR z$dYQI2LZ)NjA8#4heU-yGf^3pY?X)A+`BkmBvcgjmqmhSi|#*hB-;?icho-p%oEi+ z0E)0eA9V7=^!#IzmM_tZ7&;|Cq2P^OXe|KnSLGjp6n zeS|F|hmpTR+d&j{Nq|vd#QL#N<-uPZR%$RVb@l^LOFpaA{`gvmGMf_u@p}XQjtu2{c7j%!jo@X$-}S)XrHy*1CN_tyL>$8d z{bHf_g>o;FvC0Uhr>qw_FSMtE_3*GzzN~mUNnilSL+r^$Z@R&3$WD_Bf-!zvPIrze z%o>*95fv5UIPPdzUrP>`Dy0j1oDFr=5suQN2%fY#J*9u);`F>4xuP^8zdb2y>B0rc z@maG5R;WEbk9x+T**<#B{SWmK7-H}pEq_DqA;-i<{sPm!j3ysN1M${)=pqCvPw1v! zh$WxobZ47Ft)X78sIUlw2-R<>i4$NtAwvR=>Fh~HZw54c*=aIiFwP@~8I1ZFj8>`w z7$uY{EMJIW0zr2{CtoSnTVMU9O(#i|9UygbBiQ(KS=P08H8| zhTOqvms0`xK+ErBB%UoekWQnpKOzG3unBuq+U|TKM8;#PC^(oNWyHCl*BXhivPY%w z$~Q;45muaFfRK8e`dN~}i3M>ehd`hP|HBEwoQ5Qj3Q8y7h_S3A>!EIc2*$#-G?qx( zp)*X^53wiKH?TS50-?$M(7a=HPihU8oE+z^nHigxkkr@*JQ9t>=*)6Ci@o}s?t$){ z`sv|zZ}+~|8HZh{I6!WUEli9n_vs_EYU*pUB4>149P!h)*EQ{Mf;aTFp4c3(OK+}- zcVzqAIJBN28l5A}B=z}|$7VMl=t(Tt@XUNXiez2) zk*{C(;Vp5Et4bS>Eu!_wA8`etxsi7YkHy186r%sD)JDUrhxK|Y#q!PBZ{cT;dDLg5P$^zs z*B#rmuCjdR;^xq-HR)A*deWR#^D-2qSglL2>#f;M@e5W{v$A&P$<+)0G9A)XKO!Tq zkfl)WQ8%j594eCMbcp3>S5I2}0X9l-8bG>jwB6{99wieh>X^E#kIa#U>t-azxkFNt z5;U7b1j-m){ET8p&hoQeo40A8f z(Z<{*_toW95$T%>T=`CQSbEcv-P6*WBk%-Xe*dn*!~&(eE?ZJM!>hQR) z>`En^l3v`gc57Uw?Av!YX^mBmda2;nQ>N;2j{$$STZPex|eQ?6U0agiN+j9ip*MTUuPY ztSVZn(NSGjPHvnYA3uHL$tzu*R~}^ELkJD0b}g#5)6L8pHhq4ftLwt&EMkmyA(Z$e zTK|ppAC)QM)QrYM>20*S;We`ijH41MMa4KSpsB}QzEY=39-SY*VZ&pmdwOI+g1_xI zN`A%FYAcR{=!x_&G~P@+%@phtRYa22k`OMZeh1F1T;94qSYS|6mxF;rETWF%D~4X0 zK+YzHM6&s2;u)qunLmvel1X$<$4Yy>NdK8F@Q)8?`V!-#jeD7ndnon(ICx2PDrIT% zDPn(qk6reLOfM5Fi9yWXQ_w9HbNjg@DdIjakV?seU6clV*5$tp`U|PYLc}6ao=e?> zm2pJWZ#6n`kO;U%99p03#cjBrJjdjnCwsxZNqtoJy6cwYxIZNu?D-GVl zsHj4(Vf2-lmA#}cF0O7#cGluLx4Uj}CcGMo>cpaoe4(MfA_Q{_>qCM=OouR0=4EoP zRvV)rIW`}2(#7V}FsD*|8rtZwIT|q2oLc2@R;PxCr&c>1RjFny_fX8tq{Oc1BMk=) zCZqmPBmKEM=1?Q^89U*qM7LKuoK>LH-}v91Og}!9q1uC+FckRr8AMavq<&FNZDwf}$CbQCdt z8tfx7qJZ5^Kc}HAu$zXGeoPY3hW7h7v|nMvQCNX$I|5bHqdtX3!DI&0nTC%C3wrZS zp`L=s;G%^Uk*1`wn8V$>+%C0NZ1EPwwq&g8NXy%Bb+HKxQ)WaDTuD#mX~puGjFz14 zv?Z;HSt~Bj3&9Pl3{uXvXV*HT(qk=Be=;YoVpd8_rpqkBO~R_A487K5^hlltN1**DK4o!P-qAfnx|jE8 zJ`qRr#yoa`p>I4*`y02mxEA#QP320bn5`S1b@c% z%k(3x;+`lrr%)847SHF0ItapH=<@lw)ORR0hd4h) z`yuom5NE6%E&3l!-Y&P6TS<--t(T)Hxxj6`{Xg>Q}pi^yJ%oA3u3h&6j?;rCh5O^bai*xb&sl?hR23B){$b zOv$xbA6UY->&v!X-&Pu;=P$wgsVBi}`8qB2WLSics*er>E!2^412|)}`9LxJ0`%&` zsCsb5tfvk^e7m5oNTK(KnO^7<8hDTOLLYTu(tw;uX}$mL#r6AolUwe8Yf0U`y(yQh zSq*W8+vj9iGwb6Dcl2f9vp=FPzdxs7*B^I(1j^p;E77W6Fuw(KPB@g%0TdPj6;mD#9EbJql+d6==-~?moVT?oX%x==@3E}V# z4yIIq_p;{{MTB~bVq$Cay%I^3xjwjM)#2K*wXN~Kmcd+VUeq+}NuzdFvOGcx-ppFO zcv-%u+!GR(RN*pd*pl`G-SN(b)s@YMHm}a61_Y=sr5XCYN{Ca&_{{X!<7Ln)_j%PC z13qR-d`$clz}NO?27}d&v9^Nl^!f9vwm#85>z?L#QCXOW$JfT!=NB$4k22&fsh3+(GkS%jHTBXk_omK0VDHmAu=2mAUi3}z>GZ7B~G3AG{RMe&Kvi&7lrNyZh# zv90bhw>CfqlV^h(iO|u{OhC^fqMtipWTY6bBD6;iS%+e++&X_ohg3yv1cPc>J%0Z%Q=?Z0d7>i^(RGrv`N40rWDT%nTwqAY zF*XIE;}h{ds%MU;HM(VIi?eS3FXkrCsxJ&lRp}JbsWqvks~TLP>22w;bCPF0PaY}` zcTwv^0z~lYH};kXM|$-|L7|q=q*=Ra5}Gr^<%&@3C}S(1hP3uWd}O2bA9Muf_0gM5 zL_d#Ia{c8Jmvs<(D%9++m8mc>d8jljS~66kH4~>}&05(a#IvVqo}Gc?pe%+`J;6PI z931ZykBT4-s(W!yUVlY&SW2}!Yks92WN9K43G4E*7uUN>H$64GZd;pM?=6p`){El| z$MK&nSmSM4nwi*=9S);O%As=yN7-FX>#KX7*;1Z7drw`_s)2zlijZj_ZH*9GXO+TgS|50?cU@Xo9_N@K>b*<54&p6N;{fdLuO6C}*ZzsMTic4s=hS6Y9XRS}wwcvwJ$L?IqdiVdup3oG3F ziza}Sty1hMsB<}s<8@mTe6F&N_{0jIiI=FhuyKJl#%%IhlzFxMrfN@2Y=%r29E6GD zE8;`6_O$RFc3G}3r6t`M9GmY9;&NsDFoRlcacN^?8?e;07`rP{Da7e>oWt->GWMsy z`taRY9~`r9bbWwM1IFc0pMe)Sm_#&9-J_;{E0Y5f>|L4wlwT<9LDX}F0?DOIQehhS zErrg=QAm)Uhqi;qV%f&>jGEDA>u%gg zn5mhbogQko6dd&IhzEzUR;m(QSO*@Xs_EbD$6E0b$lE~P`V`(I9exP=5EXs0+F0)s zF+?32lZT$cW#a)K4`{Hnz}HQ^O3keRd#K;d1hHU!C3Zh`tOayZM_NEPXl$WQgN7E` zf9qtuM8pxR;5!xVoue9z2_Cw2;x?dl5&nX7I13z9TjF#<5qhb{>JD)w*WdUUr*kDk z1i=AaoK-D1L}>J};i@!$_k@qM-dY2mBjotD!2SU9z$D{tyg@qra1vfG8GaZ`T@QY@ zp60_>tQYXp#7cVa+6@hJ#7h6GD~OfDsHXiiSciXszrqTkZFn5LxqicawlNAzaf|D? zLzm#P3Iq{LQal}PO?E?~%fgpM4rIl5wl)#yJH8-dacX==TQl>WG;&Fnr?agIuM;VQ z1#?^uQ-n#Z(h6t0V~vqU>VilWB&@A=$Q|%&X0UPPzd6Gi>;EDW*_?_f-yUTsYMv7hw|Y57<3&YVI1Tmj73Q zge_2h0^Oz6)I2mN5v&R#gSfz=PkH%}G;nT+s}Ie6HW4JHy6rpcRF4E4bnbAzwI2Iq zJ#`Q0Uk?VUjc|+=a10Ky8`2pxj#71i&~zMCu@f8#rV62AG;twb8#oap!6g1jXo&E9 zoRkd@5xbX5b%Hl}5>lx6y+9{j4h9u+h1~zrW>mk&s=)sa$CuHSK>@10qeS!Q;_<(6 zhJOPg=)nIt61!=VR8GLSmbpe_Ay>(~Q92#t~S zWv{r{*L!(Q0siuEdf~QLSKfE6X@+a?$kq9eQJ?*M_MD#q@q_(8JHC9H9UH2`XLg)j zkyp6(!c5UuZI5r8HZYi%+gB2?;O9Va_$UzlazT9C#`~X|fpCc#`3nCL#@9mni5z3? z)#)T0IiY`gpngij)ElIt{2caO%D~Jr`JBsZ^OO8URHaSmE$>>kr6{E#QP*we-_Z!@;6*F|$vI0Mk+sZq=#N;v%r0=ZW zgTO2l^& z7EyTlu%fH>)0v%0SM4C)@8JI0Ial%UhgO~CE(eBNqOU3S+jG9YkqH@0)gh+P{b=pZ z|9SMJDmQyP<3+sR^FnJ$aS!Q0n-^*wFI{Kv9%Kr0pjp`=P1u*O=$ZVcfOpkn>SD7Q z4B&x1Z^F8qZ?-cv{7F7G&MVIY=k%)b84Z}T+AcnQ(msC4NjoJxle{1I;BaFR)OKb? z)~jJPJ8^zGW8n$Wjf6<&xUc2VwCf$S?Q{vI6ZsO_(u}Y5C`yS3bWMId z+L7`jYfg%1mz}&HH0J#Chu{bwbeUmjM@QeD6PpAq7RD{qD8Qf5Y?KPZuOOj9BUh%# zS1b;hiS%lQ*Ug80Gy4)57sgQg25-ajz@a+k>k}9X<}1g3X^|nISt8z^Lg6G@LR8~K zP-4`%My}>KUVs^VF6PlH1J%c(;3!UfEE#;eHA><=1<7Rd2-BZmN!1lib$jn;XO(`n ztE^YE`mQ-9fz!Kj{iTQLA}erzqJ;QxK5wRcAW|~%M*`Cv$JUC`_mCA;tEkmy4uyxZ z-rNKaL1qls!k1zV-G`EqRs#_uK$y;3n%ph`1Fm=R%VbI6yde>>Y0Q)WjNE9@MeUc@>R6FZx%nAcnjO2!2~<0$5Xt|Q5j8&!<-gn_Xv*oEo;ju z19FR9>96i^CA}208}VC|_bLu*iHLa%xN}5%ww?)qxpNT7oKD>`*ojA3Y%~CG_{rk4 zC&u;eE@8Ury%+x2>2-bdGfRS+tm`^`v=y)G9bAvrDYQC9%?Jsk1Kz19=%=>Q(U(7+ zNUz@_EO)Pzb;H=UqX9Q`j;)sX4A|F%z&$f(gRZ<=OC2w86ujOT?9dV8_XbV%Bn%zD zjhlsXq!(34wVUL3sS=nl-fRTImkgkjkkP+RX{XfqINudw!$A3ciq$YC~;QJ%sw-fl@v@E|WUsoy_B ze=>#BI1}fZg+Tf;4QpLV_u*)wB)a8#9n+TQksvY|!S@5Ei$FK=J^Q*+5p8MXOhNX% z)%s7FbCw?VpSYUY779I5EKfL@=MX~~`My?9XcvrE-c|o}_dN+uZa5jt;73K@@}}!u z>%8aXQ2a>oNs(95aegAI@%b7>opiYfUz7|15Hy6{3^-YhK!7R z$-d~B*T(z3a*PN-$)Qh98nkU_l=V0|-c2oMaew9#AzDT7KELAoYwP|xfWFq|lNw8X zfub&R55lKMD&c)Q4rj|M_Rl^~mQv4;%&=%D>Ky&-?AT`;+n*lZs*AitbnIuTnA!9t z(i*?nZfS|nZN*BoIL2ASrnXfd?;4mjXdXyyzcYpTW=0S@&%~;4d=A#z28j*CB?rfF?Un@~&IgwD^U3@Q7Mxv!d92Fg1#l(>q1xt19s9Y`QP0tcQfrcoh;8ZYqpJ>+&Eb<4ysqeH zCjElJxj3bEQoz?K0};OCf@bw0#LJs=Ydw7gJxp1G?8Olu;g))+EKwLVX5#hv<%-Bf z&?AbRi{9qckxWUM=zQzbo5{RrFZJV_X5wlrWt7j^{?g}L612>8;;S$EX!Jk7f3)6| zgxBs*cBR53F|^&ipPtL(UsWX4#8eWc`FuD3qT^H5ZXtwuDG}SS&%X4Tu_sSVet{>C z_|44p*RL|CHz3b_7gbH~%dCskyZ7#eQ_$v`v`BS6%L?f*CUxg3n&fh z%oWCuUBAssBZyZ3WYI0qIFN2j`*G1lN&0kwd#Gx6;msEtmb#eqJ`vgZV=2y?cn=fm z#h8yB5G}Es2RH@d@Z6M@+k?h;)=JI%XBJb1GWm*|D^HBDOTG6{P}BVR!YUY z0fW7$HO(YpB6{4mYd=*WsL!))iQ8&&ff=#2xL-orO>(Y-67pdnjm#gWWmzfnP+F7!Moy7aI9l z(G9#~&W-^cxv3l7=eo*QbY}P8tM=bjc4DL4(>?vE8ITa(G7FVu*G3Ttxm8+R-1Ko+ z!oY`+Saa-4<{pBl%B*`O-tPwbP1T?1ryZ z3oyzlPm(NG{Ee-KQ4iol?H3Njo@T3AcnWLBs7&GEmcxdkeWyzBOPgKaUEUO}(P)Kp znM(A7`6=Dgn&S`MEic@z`rB*TLwO_`nI53$ta-%~J|174UvJ z<2wh6>qE3;MIT32kI-?Q`LWRB(Tp3F=MNIbO*2Rk7t6sX(Xo>(aUq8ry0jnKu z#X{1a7@xzt?CroA9DUI5<;r;AV2*os?yhev4Q0Z=>@QlevVBgdYpKi~v$n^_?a7ZU zYN?O+RUG70vk!giNBV$jh+-#26P$8?j!S1EwR?gub{VCLLEJpi$LqsI z#YFD6!^qR@j&Th-QbO>asjep^DrlkX%k`c}vbk406;$C7@|mlt*%ncFl$Scr2>pA; z*y+SW`eQCK?bz^!Rlp(x*LqqlDTxzGs2#&or{Kkc5zB94O+S2B6n#6UO&fuDYk4kK8VBnAa!w2eU78a?gfNCpJ)Tyc@htd|v^1Z&v5| zT=|m8j_}5FKWtiKwM!pfS${#1ls`l*d~HfV)YGgQhC(NINY5Np9%Nx6xdu7i|8a)d zSr$uo>Q#4P86bV+yuvNsLME~(ZID~j;xUe@z__qaCZBj#{$W`66WaorT-fJQCZ3_S zq_?SvrE24V@T*K8l&Zkj`Ym}aXO0lLy%-w>XoX4>r&DG==@i=UjrwA_HSgKYjH;iYf%?lOS16y9%B*ocLvQ1*wXX&l*yrU|Q zrn}DIe(Gsvucm9LW$7)u*8OwN&a&eCaCs+xA(=e(gBxRkh5#8v<7*$&*I^@LO<%u@ zy)cr9W_&QX7C;H>YQ!0%Ma4@0Aamj)I)upCe~2q6WQh|GAD+>y+Z4 zf3?DM`hb##pId`hW)4rC0IJW2zDQ-!`MjVXgUxaXjnD=Q#n78`J+}JDVMYg53;&o^ zzSGLjnHai8$77B)@t9vU75QYq_gL|A_q@xr7(Ll3lg%r3?H-Y?iAXJz`C%PGk(N(e zN-0?XFrHPn4K)a;)dqazF+a8lc1*hg7e59zx$UA*U98kD9jocvg6G|KaS5yIE3t}a z_c^Vhh678&?%ke66?Jt7@9}P$y=?Qht>GL+f;-*%*_Xcc9w-YDx#RcEo=F}>;V9S1Vq2lpK&bX&n!=)1 z&C)EThPBb3=&)l+g;=3QfOvigI{Ik1UkgFgSWvD9Vxk8^+Qo20>*Gz_a@GR`&|n?_ zi7Og+E`oUk;s*!r(4FAW+IU@W`PYqsi6$%)W9svO`&x6>3= z*7itt)nr4sbIft=Vyug9qJ5A}i%-HGp0`8&M-r77Bf)JQ2-Jse*uj47VPd-%)a#;E zLqRZ0;CD-pG+{F;Dlo5m>Q$}ze)73 zEAR(Kx2)`irb#Y&g!xjQ>VLSqLO(P0;942o(P}wj<-s*>%S9Tl;Lsw?Rx96UE2Y4u54}3!dlsVWWas=udmXd@OiZtK}7T$RJ9IrlE z(+;y^egE-lJ3?)(_Yg;Hp{4)4g=4lzf@X*%+L+Ydky_@dBd$&Y zc_(QV(E$0w=R-0b$4>(!XI-$n%db4YX}v_~U5c|zIPzxA4=>oIP;8r8COvBWITRkA z-!d;gs0OKRpO~)^G%sF$^E6(WWq`T4fRoK^=W_481$zIZ%?{b5C)uN!qQXM$bE;Ms zliU0oc~2w)ru9qRXaSy@QtIAjYPRpAL|08^9v?Hl^X|2_n6@qwwEdz#)e3LZw!Qvn z2LeUHKqm#`O?U+kzId1IJ&YhF?J><{ScR`b9%C_yYPA$vAYvxIr8yHr1jad&S%9YMguipF`u~D|`P`I~H@8RMjZ=>jU82+S(G^3wx z?;#)6BKK(HqupBFX*=lb&xyj8zEYjA`BA~o6oG0>$*QLtMyV@c1>!2{?F+S?_3{n* zMXAFOg(QS-G<=|;yBhvAxMV(l@1yAH3N}MdP3P8Wu^f1^!tOhzf^)Kr^eTO|1lrG>~>kNUtq`&XXZcOOVk^SOFqS+h@H`LW1*b|&V6_p`2C;75r z$}L=j*4#`{t>A_VtQxI43ajE~i^>Ya7-ykyPZQP3tievGVoNGh^^R3!(<&$zH00b{ zD0!L^MQ~xv#H|zL62-(U+a$%h+Rk7lSluIWD%3kDH%R9$US8y%ySDdf@jX?(_xT5X zo$D*V=TSvn?s2(a4D)5j)Ymv0sir3#Dh1GUotU#Qs#@bp3%0Z!spWc~I?4s2Uc40Q zI!u&LZIc@w%E3oexz_rX$DgWJ*Jr41h`l1X>q=D>fz?y71|@$xpGUv&>M#)8 zgeVIibVRx6p@^$S>sOKiE1NkHNye>(Avb0?CF0gdz{=tHZLGC~{UaLoVA0h63EkdV9pML{)yvCNZ;=;@ zL)i5{Ao|qMv?s3nVP*dD)f+KqQ!-c)}8@(y#QU zs?p*F;TvOFg!r@5r5BzPxq7`z{OK6S5u`C8j1-u@vgRl7hnP|cb|A+$m(?2OM!MxC z0Qx{(p)SPJYiQf{L#l!M!q9Ic57Bh+?8i*rEeeYZiimLG>J_Y0-tma& z>W|{|3?4qQyiF#`7BlZy+gc^^xskcTR8W{;d_cx;qBQ2%GY^*0i9;fE{=njbv)1 zZ850j4%=+%O{dlIljeK1EhZknep6>QIQTmJqk>hco{nibH^)8v8}c9-YexIxy+v8M7vf=f3}6x*?%x=@?R zlBWCGGpIfUwI(^N;Q4{o`CLnpP|C!phVOhpoleUAdLX#1)ki z+p`Rh&aCr~CpX|n3NnYu^KT8Cu;Y9nh9Cjm9jw^-cU)~hoB|ig75rm)xI>i8!VY|Y6cTAM7bO3Bb?1h(Rua!s_|2(AlWy)ZPFKI29j$oKMQ z0u%2raE>pEHqWBw!=^gl*&Nz#)BAvprsousW$EwArcH`{$cQo)m_OtRZDl+wkL)m) zB;CqZ1yetZnC$i1ei0!Qu2s;pcwg?jOrr{O;wR=5bz#|I#V_N-t?S>=_H)QiiV3Ye z=yo;)jzuAm&}TGu)0h^VG0ZtRQyD~II_%Ly=)!o})th$Fg91b2>6M?36{f&@o4!nA zd2`z3A+^wpS^I5~nn9GB%KSvye6e|{hZk4EHCguxQ;l-NG+tpT)O?9XQzUfQl!PRT z;w=V-oT}r2CODg9_NSyW`BFU9h-#ats*=PHKN`icCM7UgWC;&7H$R&J6D@Bam)ekm zHt)vk5?77Z`u8p`^`71Hrv;5D1CkF^mFXsITG2T!{X&+9tt8E5d3IDdBzZH}crM&% zwZi;IkKOl#FT5B@D_foV?=Cx?e#+(!G{qXdu}$T*F>Fr}hYx4-UA~qF*}GwJ-8&1_ z-u95+)O@CW)S~XjmiS)V+>MPR-T2GXzTO|Y@3U=gfR=`=lE=~&sZ;_tYCKO{-!R7S zP#C9robK5h#kt%ce^~AJ!A5{&;_@lbK431u^?Vp+?U=VYBV-?sKBSUr^+*xrBp__Ws>FTUny0rX>c@l0dE=Hc+^WsEbnmfi-^ay{C2S4A3C~-nuIC z=&sKO*8G@{h&xnf88g#`uj$(YrP^1l0-;>l9@9l4kR@Nq3-3`C%>tC5XHV}t-wP%Lcd|X*)*i$4tA>`y1`VZ3-35DW;dU-57hEnGaO$9fB}h;^ zj;uXFXPz?!d(hx2whs0nSZ_g~VM!LtXDlL9K)w>kaus#?OxO=IyTJfl)UndoY}QCc zlF*bL>4XGGloyS=ccI#AEusto+W|}g8`t`NYvCw3vz~VsI5Asjtw|gRaAtuhrl@Y*oP-XA!#@U`6T)T-tMZ ztBtYx&dgWjDXtutdAIK~%Za;5*VQY&)-3$%kTQaA*V3YIuC%Y z&UA|e1RWu;`_y6P&eTdB<}e|#yUJbaFg`KV(Jq{1RoLT}S3l~(xGi{At29Cw2VLHS zA_yt~!EyvaArLYqvJ+Tze8ZUB*dD!6Iq%Qn10@K$$>Ri3dK8UEWS$l`O{Ip(c|#Up zG>CG8Fnv?Sw)E zT+O0pw98ryE0~L_7HxzLc7sPBPfM+9tz_(enilM3>ebE08Rz|~A2_;y&nASw~qns`l+#+K2sF>wx&PSj_-_oMuykT`wXKZSuve zvLy8wH6r7@`&9BP{e{~O(K;rzhjtIQ5NA^MOeN^FBON{|mpnxqO8@A5^DeAdv9BOzQlcAS%97p_GNI1m-~Pz+6Om7F?)2v!JE=@VTSg&88A5Aq=tN<3lO zFjT}NvEI7ARV=`Rtl)xG3IJ(9w-y~x87wS&My%9z3=1w|C#}j~07k#5|gedvdamFz=Is{ao`!2i| ze1mTid`;-~B2b<0<9C_@rs^~|;to2zdSvDy+GccvSFns6!>?JFxv7G-`mJ&clq*38 zU2dlVnjj&p(U=unv2_We?66T_k26+m#{=>Y32jA-aagGna~Fb1%`UYgbZtf69k(~g z6j27S2w?@s$p?S$WTQ&fewW_#DF|ssQ2i=<3n~O` zDCBf9Gv4=0wY?_v9zdx0Ny%mVuJJbQ_(}O&`_XcSE0~}qckm3ia&6g9H>jDn-Y`nd{uu zal2rjhP#fT>0rxQw0eD`rpHguD?w9O(6<#G+N*Xj55&DitOYRVBsYT5K7yul@+0UZ zj2lPpHM)0q0~s{rV)ZWIK5cNWGj2Q(3xG=-!32H3Ks2i2%$Ow zbpaS7oz2e6KF^>}mqXyIsH~^Le$3&{a+qGB|4;n4^Z~h7sEgcR2_2}=)>dN^^ zG4>>YX3)Ew(L;-tK#a@A`zz^ufUe+kfm<>e}LXBk_0!db{HA647|Cxdt$?Aw^s3`I?1~Lu*Cl)wB zmp4g>k1Z{Hm3JieMY?6uCXdeL4tG9jG-Eh|oBK~D$b7o}mCZ!#*H$_cTfI0psZ=t*Z{o=Dz_*xA&c@<`e!YYprN(zvqk~q1uHd*Ff)z^k9 zf>YpKA9n z(76sn6T?#OoP^E@f|M1 zs%%pnb@lWj!4+eRg(qoLaRfTYNK28-uUj*lhxC!K3ci0w=`O5HNe z#E~7s5B)leY3wdZYHa3>v+igy=Z1C2pGlR1KF@b6OfmLC4fi;Hb0-ZlkzHeuAoTq@ zVsW(R)`ELjYBIKy_bqX{bXoKOMW7#xx}2h&wsBit^8zCgMWUQaO{RxpEqnKR#9G!o zW3n6l*Z%M-0~nW`C5a1G%^NG&VPTdC9&hzr46P9 z#oabGsQGXJJYnr_GC`Bx;k)vyK|ZEZl0F?tq*J77xSfK&Qdf@IhPo1VfNr)%uw-~B z7i4>Q^|tkB@WR+HBg-T_Q+kwjB&=viE~i>q zmD0X-9+x(nv@#G-I^}HsHXNLn&lRoYdVOIMekQF|SEldUXOs^bs=7Fx+mU-ycsyZt;C6?(BjHKPU%Pt`OdC8++F)5%EcVA4=~!?k zT@G@Cab^^Zv(q)_@Cc6(lAhlAvWx7b_~%l)+mqyh=opWfBpjWx?N zFfR@BXmT`UArm9@I?j@S2lfO2;!i&89gDS;x|JpEh^th#3wZkHZz$)GJu!~J z_an=yCe&13XAtAkdZ&s`q50o0o~&S%PPM?lG~AKkKxa!@f5gqG$d;7LeODcNaT^(8 z^!$np3A%l5hY4SWmhCd{pF%^M)5571FM0b(Kb<1)(b>DArewZ@U5>NB^Q9NH5gQU- zI>?A<&-*@p;`USOpj;@u3vT?bSf$$WGF>;YNf!BGE4$^1TH8Fn9YQ9<3IMIqw609OnInV z!WQdaNr&GdC4?5S8mamEILoENzyNu5$?#@)H!3?_XvAV*WG;KcFIAhSw!`@=Xfy(^ zbi?^CH17qX1lmMfm-7bPkqqsd9d!YqWQ7)hZ2^ZxSG`Oc{&z6Ff2!y6H)8%)twNz^}e{00=GaKMv} z8U_rlTK!VJw+d90F@M#T22UGV)YIvc*!8Zx>m#CX^u%lN{(}3W!4ofXQ4^-UwM^Yv zJYSvmbV7LLB4i}y%2~s|P5#PBtTehztzPoMXpx{+(Lx|=Rz!+NhGCwgppmLmg<3XM z!J^d~o}pElT13o_?Y`nwDRyCY+@J}T0ls#Jn$Eo$y-=w&4N0os^xUB8mHi?Pyz>fn zM+}poQ&s35a%PFy)@n*p-A1o)l(-z4X6>R6n&KYc4u6H(wDCY97=swPGVtHjm=;%= z8q)%DMoDMdXSnmOc_%xm_5U7h(@!d8+W6sSTi<9lN%%8J;ABU{tp2hp`HmH8DU?J{ zH!OSj49awCPi8VHjsE8j+8d#A$J&Trm*`3)Q#LL z;nSv)`|O{P7kyBrVN-sslc92;X_I{YqNPhqdlk?1Q@HUCLZ1XN@ExvG?4DF`(lo%U z)gj+NIo+YBgJB&0dtBEqBX0j2{!og7H2r@k4E_j+-T2*dbLlwn>r*US?dTfW|>U#>@9aYL2#}PHYx_+{5FKb065n={ktEX*G3OKle2z5}j zt*&5)Lo{{^NsGeP%%OAp)DkDysqihKV{hPiOPM6wIJ2Q1lA;FvKFx!(x7qobuX>w{v zCK0q0Fnc8}wx$&A)OfWf3=c;X#ruQON{mTHfrC(fZi0%f2IDebqJ`04<~GkJ?UdomePVw7doK%4FLP5f>YZt8sFFe|`IpA|gQh@Jxk)8=9deCbA(?+-3 zCb`uGD9IsB@=BPetVS7IXT=NAcnvorjk4URCTTgM?ZVtBjzG)=VP&zAy4z5Dz=2e` z+OPMj1dzOuFuP+uECv!un>`ZV<)%@L{QyNh6dNQfZT{iDt~kTBEWJSipc?5Cb;e%~ zJDv`5ZTfW|bp!{tWi!gn-&tr>E>ClRPtIm8X3l1lIWu5kb-LEPg;nBDPAc!ti!U?7 z0lACoajpr11Ai3!PKh_F3z)OTWg2&>Hp-neM$XHS2-j!-aPke_?!pWYxeS%1oTI4> z!1KPy!j^|e-ew<_V{-iTh&Lv4n6pV`&T5zk3S?9zIHHeiRR6LGgi5l;ot*I7jBGLC zR^VD!Tx&$P0y_xd9CQc+$VT*yCI)}OGVJQx7JkgVC^Mi>RUI^O4{#Piny=Vl-TpL@ z)e`8@iDs?XJi< z17iQRbS`ms^MD9p>P+GjLI4976AC(_yAIp2o-l;=r)kN)WCLm&ig8#)=gcaVGn0Kt zGkX^}y^_YWLn%gh7QNa`O)iI`YDg0af2(Ofezk$9(#f0SDSF+`ChNiR?uT3G8?XaN zvS^Vm@>DVWt84ocBQxQ?BM;Ox3Sb$v2W$S1vrpJ?${!4vS}TfBmYD$qc=8s_J1L^=J%8M797SZo+Dk zty}Oh>`=tFVZH7CLDLQCeiKWc^$Dwdo(gN28VgIFxB|;M8%z^H=||Ic*Q-mQVbDy) zv)2=86**WftcId(Twi@qNHduD@t!e-ukmaD6GhH{ttyUB@Vt@)2{oY!L{@aofOmsP z5Al+{jm8JbA38)1rNP-fF?*{I28ELYX|{+fFc`ZRf|k}Y zpQQk_eaoq7yXIG6 zlDm|tF`fXuijlwjN)!mqY+at>Amfgq5@Z-mKSi0DYwIK;_nXUVV7Gp> zeMu>iX$&klwnMlF;AM|gtWKWs`-!xND??_SIqPK&%h)hHJaQ3qr%~}udfF{NKrfBr zet^g;M;)p9%MNHO1*nsC5g`t=)_F8^o?Te zg;a++mra~c{}xbb+N-m4%Wh!<+2StOqX{k6*`JPdDsl4nKm^8+eQd74zi9sYDTxaPx!jHt(^Q%3k{S#jooMk2F%E`cZqt;L%m z=~H^a-|ZEZp*B(Pf6M?zJ51HZ&DW_e2-q&jIxG}AP}Mr**8LBtgkoQ2j;wldv$YlR z)}Q1M=&#}3?m#03Za~%8-Z$|2BGFlMDc)Ez&MID0s@&!c6X=i9( zbgicY-hGQ{UVE54YoP@wUj`m7;`IF;^tPCCn+>EYrfxpTWx-|yk)bMiE{&z1P_pLBoJpX0!z>3Mm^IsO* zLsmxYZfrfS4HH+2NRGi##b84zT61TDvnTt0uB$qk^1>GO?15WzLoig0^X7@so9q;w zkt#n!?MXwC!P>Gt>?Q>Yl5X#fSrS0^If|dz!OU5LOLI?dORR)_ZJ~R)ogaD|wsA-W zP!Oy%d(TVvvN-&g{Tq}4J+=HY9BtSH6*!I7-T9Cw@xKswEqG;rzYPRwLqwjSTc7l+ zWwZa+gkyO~QzQP)rbuD>?&p8B;M4X`?ED{{z^9>o3V4-A3*|a-4lIheP!L}<-dVj? zBY>y;scGuqFUx=IigDumpMUf}I4ZK~y&>1!|0c2IxRFFUXqtBsX5F5!P!{VRP=p!C zv^;L#lI~m3%-^(JG#)2Wvtqix>7~ zbiFd8s6Yatao^-L#UE)6+WBgLRwB7Qu4yyZDTtmviGYp^XSwkEmIu|21&mkAK>(+J zQk~V=Zk$X?O$uw?fH_0rV`q8F{nhw$T4>=xqLFcor;}BJ?NLexts#(=%n(TVJNH|v z`D&v?p3;BT=B(JPmwm5pF)yWx>_PPiQ`~PseoyJ~tG^A%xB=8zZ<$tE0F+rlRy`6n zdPo>IN$sn0b@sSVsj|4|t5Q$JHT2~`ETb9R>AnM&bD=gk`kwk*iIW>+6#wGX)BI=( z|5@9MsUge1oH%n^J38jKkQiygj?+-|)thl;(*IV${h9olVg7*TUdRD@e3T^iRFovX z{ezX6#>>wm5ylS!A0YA&lh5QYg(}tj9#~`H7U3(J2^$ohIlT0Es%4u+p+*xxuhopY zM$#kjm5p)T!#aB#*M-deWuqOzz|s~_Yx>}?*hwnYMydrQ99u*Fv6-6N+u4oiJWVzH*vVUxEVDbzQzWGT1 zomuXkYW%~$-@x5VXxU9~XMqNCZ0nC{;C&>(+svEyaE-=evszI`S&tFP za`OQ^mkC2MWEd<0P1sE}%a}NW31z>GCxSG@Kb&umfA~8f#qq$Rfc#@7GFylAIWmpO zw7w^oKC((XLq#^>mv^vO}5g9uEjDP&+Vkv!XFDP;TKjE?u zBhN44NL4aHzk|L0SdbyPF_38R%a!~grhmW-5(!RO5R=>#$xQbvBm(G(eA+_UXo7RT zd#JH9ovM@d%?{pg5JD|!RB;)(hQqDnbx(64HSDx3ihc*ryaM^2;J{bx5lQn4`e|&} z6*~L)QC=U%_2;XNKv{7}&-7sY(&QKASvL{cTME#FoCcC|v35<0WD6RlKnpxhB_2xz zBcJW+#XE9u?}EVUL}}f_OR5Fv(;VWWr7wP;uFBxPWS1fYWwhaj{?9;F+H(t{NT;B4@{UMwQoJEGma?tGVU~V19 zTr}-2Y#l6Z;4Q=o9(1NLAQ0?`b0%8EO$U3%80e8B70}`CcZOtoPEC|GaK+-({#q$9VlkK!_G)^w%OSSMMHHY*nQsX4{-uROZ zMSoC(N%mN+9y@azpwL&MB^Sjo@<+SmrIWFlOenE9HP0 zZpBy2GV?V)K7J-uH#J+^7ezLIN`?6A#1Y4Ob`6SL0(}m$BAdUu_Gz_AH55dRKHI$Jjtq^u zQsl3+y-&z|kfn%Q62n8J$#6=w7l}`2#jB8clvZu9?}azd>1XG(l5a&ozAniOm-HUP zzLgVm+8Xg2xg11g+bA^`nt$LhHusL+pmNL>!lSdRYa~Z>6*g*9*g$`a#^~ThY{piMed9|JPI+Oz(+v{CUcZ#kZ#jUmH#D z`D-HwQ2BF%jb*ZZgs*NjoW!NepObc+q|;(};lvDop56bG?)f)Mn|@CvHKQ{Z!vi=1`0H#uR=9Hale9FVf6ojNQSH zd9%nbYXI^;R9upP+$waSN>NE71x;O=AQDjV!_C-e;{(-8S-N?WF_QAP<<96oKUsda zsT$fR9b6*lk@wOw$ny`#FNs=-0k!z(ShZxoN^FlN^DyL+>yi^mLA;KRO!KZ0j4OfL z8RQap%`I!({pHcL%%fMsg7%F+Q6k@6-2apHlhKpOk~cm{uPfefYx1(Ps&e5^pkdPeE#Xm%!$L@(uO-GY$d%GIf}f*~rK0HBAEYGw%^##>?=Mo4sK7mzcV2T}!vr^rv8%f2 zIt}~v)#x)5Rs=zvQ6Ia-7xQG7-?NOB-w}EY{l+VvYAmy44T$!rdlvv$4~UgFGS`Y) z)Wwp#q6?0c4*#O@W4vy0LcDG-a=yb_pBY@{X+A`-{lP3Q{660$%}NqhnP3?pbvZsH zjYr_52KbT65gLXxUC}3EB<{m_v}=G8=q*btwaFm1+wH@33!BLfUM!HR*3R>+!f9A z)n;2>KW&HbgW_Rp&f2TkCdn9cX9Ks*SRbRoG~T%B1>T>hxUJ&Pokqw0vrV)psp^tH zC&knpG!DbW>nYkG0zmkpB_eA9cf~Dz)z^{NZxngHzi8x(VeBzyr~5rU@I%^SFfU9X zBg_j--GT#Ibpne6j59O|Bnh)sl}F_L*Z*YRdl#L$k^Ca@Yv3O#4w*qi^8WgcZ_9#G zUWm?*xYlHDSc?vhdup|s9`ye~-m)K<&P{#7`w4tBjSD0c+G%^x_gS36RmCE7yun@(N>cftJRAvAPIo$OpZf?P91JUli~H<{MH*-w-%_nIHc$vi zeN##J9UnvZ|BcItF#0T2Lw~|9x1$?BRZU%F1@b-3`rD~A*2W}RLSO6hAS;3laDvRB zuXXDDjhD$!6Pdo{Gfw@#%po&)Ng-3fc9LwKMkH6Y@Fq#hI;kmQenHE*vW_T|Kg?rf5T0~hyWuRDsW1sRc}C0DOXpz#>H=p zwDo9s=K5Tvn;4a6ycKX{mvyrcN%`*!M8~+d@>XPBzP?Qj*`{_i61S=Goj(q`bxGax z`6{w%w5dK9G&)L*^|<~@1RKtB;Tz?R-}KPrAx)j(k2D+OkNjP$#_)b$ocOP)>lxPRrxU}#16{pHs={gLOL)xtOWA!XD1Xo9}zIR5rp(|Tmem17$zkQ4T4 zp72iM6!*O90ij&09Q6t9;Zm8LS|8E{Jk9n`_!`e)bsNb_uW?V4c4we*CS|b;D zcZu@=4VhB|&Kh5h6@~cw{Ah!;zknR}g1a9o{*Erla4N?6u1Yk}(Z0o4eI$h42NueB zD_6$YS-*O8AODJD@VduxlX!S-42E(p6&>wj)lUr@oGFj){f%!Ve3 zc9(J|$2wb8Vw!(S((0=9p{r&a;+TERg#LF431dkJjd-QenV6{Id zQ8_oq74EX%IuemhBRt6830P;(NNygV_xTy!gym!s4w}|BQy`>$C%>Iq{xoIUZ8p(z zKlObFR-Zq$9=ql~Rj-;5CoWkELso=L+Y!jpd46ldvFr&G89mEHup4Y$Gf0dc-FPO1 zoGPaU5

    vb?&@5vUw$rLDP{TjVNtSMJ8v*_I&IZnTrS7*+J;-=zH&BgizX;Pd>U zni^LkZ)Z4XYS@yq;Kerm`v4(%Qgi3UE8zweh=AU7W1JDFG49sMWHmAzcD;=%oXa$y zK?Ler@XI5pz_JZkj^T>V&7c=aNY)KxYu3(5W*qd3c7x$H5Y=@WYKfmD`~p-<3f%cD zA9=hB{CyZ{Dqr(M!!E!9fIGCI5nyKOkacGi+K}J#=Cg7oZX*NQY#N_E~N(D zL(D&lITip6)2>7O zv*q|&uRGhknUO#FYTmm#9sd8?I}fNPw{736h=53wE>Z;PO{GbV0@90s(tD)W&^ywR zDoT;wq)12Uy(1tHiu4dVL8J!)q2vYK_dWOBdynhc``mZW9`}q9X6E->-JZ$p-mI^6v}nU|A6+UcBwZyG}wdWdPB-^0GBVH zi42p5D0fstbQYL5xfh&mNXt=eWX(@L5gl_|@4TS8%4IU0V_hF>lkk2+wMwtIN`BIX zb^1Guut_N2xL2juRn<4)LOIO^oHntpKZiCr)A41$XEq$F^aiT>CS3%lxy+`Qt?N^4 z5=Q1S(bb_cpB4ziY~ED5v9p$4rg5YG8hg5^Uf^ zA_5KqyXuV-kCn@HdAit+na#`PlnhKzm#HiA7^c37yPzAjp)Z+?Yf3gq8q%lv3WdDqraQu8mQL$QZ9`_ZPbo~6weA$1< z=V;H13<9}ZIqliM=PS8>sJ^IA9Vih7O^Y-|CxR8x1}qjth*E7RKlEfvir;vr_|Wsx zn*$)~5^)9dbIKodbO+pb&Wk|*g_JG^l6Q4#uZCmyhab*-gNcm>@}Vb9!j72N18On! zw~xhAN`)sZNo6!5%anvKJD>7V|Iee>s~}8w8jU;OuV)&ZyFN+qS6=UzH^QX6es#B^ zgzopA41e<`Gfl7Y0rM#(`Y9~C9`ghz-5rhiJ0KOFs3g#6M7HTozQue>>HhncU4H{a zoS*oh|FR|1V;;XGoKb}g(@T7R2P8YHkY#$E?{$8nWdChTCKa9+Nq+}KVwwAxCn$+^ zR3X#!DxX?@qImz`0o~vm$xoE(=e5jL;fYML7*)tN{Tm?1$^y8+PH4%3#`@F6)$L5p z_Cme}?O#k~gLYqGW!~_E*G>9YU902 zOwz3;SHHVYMNk~Gey$iO3f(>yD@qg2>>`uc%82*zDr5J0wWW@f<|q+~KZH8okf5SO zOwdWQ$Cc+%;MH$Y;LW=4$)U$~f7p~SPyShDXQEo$xEZ=hIND<|3r24MX9aOdt zQi7|`R@p(be4)`Z&7qiwImsl=4(VaaGt~j#UZ;=K>X_W&Kf<%(Uw|lau96>eI%jO% zF-6hSj%wZLH~;I|fW)X)0i*LkUT)EfJ&-Kw&D8_$rG?+**8XZLcOG-(ycQZ{ZGJWG zAHz7mRMd@DEWoa@lf17_w9W+>@1UR{dV@^%Ph}|d8?6fPCmkg><$z+8P z#+|Y{q{(r!1xS_8Ycj3smcEN4dR>9BqtAT@qvS*1&iB-=7?jjs;fYB&QdlxM-m)l1 z5AK{L-$%)e5%mbsYf0Z;4}C@*Q^fGeeaZJ}GGoL&9`stux7S}h6OAeQNr(|+ zP4@~<^1Yu7KVi=m`pE~ksY0LqEPU~dA;$Xlt4PWBF*5wud!*qU8G*GWcZLc*YZB&Gf2Y@_nKVKXK1b!lcl*FJ%6VS%PlR zE_I@F8vtN#9fkkxVaQ2$)~21HZSTgT;QRpCD*ZbY7+{9g=;XK!c!(5LS-tuU7g|5rC;73L4a z|JBWwMb*iN8Y!M7#8+r($1h6>oC_0&KOKptdQwTx)f~&E*q@66SBX-+y_L;uJ)WA` zc3~AA-4r9KzJnx_g0{hQD7!q17T;|u3OZCbIp)O3k(GpZeHhBTXw50P$FM1%YblTsqn zp??U0Gd2KU`Lk^eh<;t+P%or`mUq<0z@Yl1}-~}skACX z3K!n8Qi%*E;f~4gX6!Io&i*)QQ4Hk`q4<`zu%Fgo?S)ONMSBHz%(=Cj1EBJRh6b@+ z6VfdzvN;bTi1u`wX^MKK!0PEB;Tx5(xqh&KA)=)a>TP$)*ow5u+{ zeHN7AqV5n!rN>mlC|_a1US+wyu%&E*}v}>+L$o_|suOc>eLN%opBDVib4%*4fgE)(5FKr=HD&$Di8$u+k`Z zZqokT!&fkE2qg?J-?bA57@Uc~ap77>-4lL;O-~TlB*#564kJYhc?UFU>G2$BuHkvSZc7Ne;&{$N%Fohzdm$3OIGrAC=-$5p9WWl zNU~TZ)k1R#6>lYp^9Gs!X)qF+OQgt7RdpvT=8<&(ekrbXc{8@6=TfM+^+lD3igj7uTXlnSCG|8hsFOpHof)ah)8Bq%%E^ zHmFI>N7yOvLeIl~K;ff~Y`5PxZg44vpYC!?`0AbkVajZg8U>(Ax5yj#9af`q z09o2RWw7!P^O_pG@N;1MN(A*&BwM3`LAC~L9ggQwX&f>BGHzf5(<72=n6XYl1IDJ8 zy$fO&Y6^1@qu;s=76I|4A^{`;R(Qi4-(DdzHwse zT!Cjv)6GhNhqQU=VH&j`B$S5-6O@B|{KVv&HP5EO+(WCqL~(Vy+#`^oawuCFn+QmDM%%xW}3;`598=RRd~oflyq!x_wzW!r}D%#Ghxlj zga;nB-mKq5j=a(+r;$=AhrY&R68Cgums=cVHF@>)!>%?=c=-v;>2M#M&m*Ww37Vl%=-s(jJ$$q)2lJmE@70E7P0KsgdT~}Dp`Xnb zk8~hvr=9!OBEE&wcEjt-nlQX-yJ5R)r|K|)MMS;U;KJm*9f7wPyqJx2M>k5e%dxhu z@sVAe0j`ak_3JA0`jb;|)HGxl5O^@-Go6jx(M^tAb#OjTWdrW$mN@0jT!T0tL)i4% zJ>f3yg77F#XwEU5Ml=JlcFX%Qqy1qh%?L5lD6VT>%|jhu93T!a>viFM;Cw8S+spT$ z2!rhUVyoXM?hqy<+luB?Pt%K#ygy655@f(+ZIoxR|^m(6=0JGX`7B}EV zaiM!Jaq8(b_C0x6x?yHx4`CjMF05&4i1U7sBf1~7FMBD0o(k0eQ7UK8$xlB!-020^ z2x*<%I85kN>g%bhbyXcCLvbq0W-3!}Dtk+vr@M)*(I)N3eoH_Y`@z$-yP}!2>Tk+}^uZ3*6oju=8%R}_P9H>?&)HRN7dPk6wtStfx${7bX0&7pF^`ZeFkDDW4a_)=uVa-kUT zs9;dD>30tC4T=V+mCWUMgme_|rHs%!4V*Zxr1PLsj#fVb%+3V&1`~b1q3|PR34-2c z98nIGScP#SZv0`X=1SQoqr~=mAtv~JH{%649RC1x--|Y(=@W@}=h)L+DgOjaJO{IR z#{*b>$`b6q#=kl5*WmqglZ$M0=0Lxa8dODdWCyXYTrt9{&|E3B| zdcM6fGX&n?K{9M5Q+lhB>@Db&uGlFeY6M3}HV?*|nn_-W6HqZ6YwA>$Bq+nqDNY{4 zMXx7W7wSZ`MnC-MZMe)|2l6rrFS%~%iPr@>;jCRBW_ZF20f{|2q((5 zd&7SnTpcET%PynwvV>0WVV%Dd=GtEeq-$)$Pu~{GXvCDz>piLqbNcHb*a>g##xUPo zE1B_^CAah>>i#;Au8VS_UK1L2e!DI+9#g`g_m@Ev<=bW|4?i*L(k$q;5)}W5sls(c z$L+W%uohyq0=sdud|dRPu3H~5u}p_8pE|#a`i`pNHlpGWv9t{TJFIYmuq`mp!qDVTL@qZ zOrad-dlUV+^~A8fbG_K%dpklK^%4ddgTvficFNG=CIBy75Fw&$kii|_oymPvYLF|) z31dWtrJop`knKS-X@_V>(!l{A_gsPGcV)k34HF6J^b^mHO2J*tyTSi7p1W95`On}s*^mUAkXdMc+8j9SwPJo9fN6$21?_^CrJcGW9dqs7x% zeR<#>A|uc+rNd*e>muKt&JYfY5Sca&bvzE(bur&xdtmp7>LeXpfHxb)f9RV@u;%Wp z>_(9n7=r!}hJV)DX(-{%p-e4SBCta@7y_SajRx=>ZQnSyr% zbC%%+aDoB}db7dphXR}G_7MEjkLYz2-S_v}4RU-{6-k}Ym6%XCxd5$^>xSJ=&3Nq; zrZht7QJ#gZ%%*4|WyYSh{dU-2z1Z2RMru71kBhHnbWSxu{{N3tJG!L$T3(Th?>{?d zyhkHT%C7;6|1_uKaAZL_9Kd<@|HC;NSy4^~*ql}T_d8C9BlF530P3^fIGm&Nsv+{! zT+TKBL?=OBkIT7sbXoN`j&cG(@673FWci=wL<2<6;D;lN%KzDUokf%2q5B2&t~=+O z6D0qc^gy)A;XjW5Z^1w2tZiial9qyJX^@&HWG4sU?ls`@GB~ey4U7w4bHh7usTsZI z%~W#YQsYE|JTmmEM_$c9=hLI-l0GXV#CpFQ2}2z;}WybcXUl!v?5h~a8*StQA+WhccMmfoSk z8g^MUN>8@8nHSQK=Wg>{5jCkDUAzkDT{P5TVUc zIQR<{4%SdSfR<)rzoi7rMYL?)PmkcMhv7Iz{eK@qrcSKMXOhG6fDBt5%?5Gm&{U+- z$??8EjAxlPuxWi2TF5K%BAq~)jXINCBRb4ii;X&a(aB$Fm{my84A%Jtxx+1=gO2@K zI~(8EwIHgq8L)A*O?I44nO>RUU#$|X?yt2O*f@0Evy(tT#O&*l)4>~u8oR6Q(=1gg z*4YzM<&nlzo%~@oB!2`RmIoMPb>fFnlL)i8JpP(7@vt0tMm@6m#-+PLI2UqvBg(>6 zp&mH($EZFR!naXk;i_0~HKjMQndCBFfkw+mC5f)?O?`WMX~IY*+O4R-)J7k|z`N&!K8>+&vp;o0pEZ z?J?QPbW_k#Rr5&AhRz$iW8su&zkk3`hmKiG*hl)O&<**m(^%#Q{8VRH(I26Q4g8)Y=xI02qYZR(48?7Hkj^zI0nj*Osqhf^f-XG{ z0lO|#j$x!bp0%u|075k$n`Htrwi0~FJszVMO$czyAwlu!vpz-2BJX?q`-F+fre_%x zn__3(7Y#-K>OOla6Eq;;E$F9(I8}w~rLk?y$QDry(B#2n1 z=(`&a~i>%04JW9gIQClEX7c;)xm%Ha^If>v|V<*aXv*Sbnr#=O#J?N zjb3ehP&jxCbtm+EtWTBMOca4WiMAQrJMW|rOkF46Y;Z>R?Y`w>zF*%J(!?;=o)NC* zUo!lw+ZLZPxbyZyjq}?U6Hc8RuJ&o%ZpcFo3sE=;gtbo1DG#PJl$s^{$Te*9uYpWq z3D@AwYY@sh9w#%{066s=kTpl1(Vu4&Xvd`85|!`>hF@Ew9OijzCNuc*C%`H0ntf&X zhr`qz@^)HGh+T+&hH`ftAk0_M=LS-t_NE7618pbkj=^HAoHV*kHd`=wDW{e+H2FmZ zl4ry(UM@IGzb+3!Mzao;6mv@R-U9pG>(U4r_!Pg{HTyRa4! zG}y@Pwdpd4m+_pMPE{15@{&mqY-tr$6{lQgxU%cm01f%dYYxLf3)NMvG z+Bg?37^?B`n3?v!6I@Lc+priIP1}si9Ymxdy&4f-Vi4>Yh9zu4N%{qB%3Gigit8aa z8R(w{Mi6y}hKAl+q#y!GO5QxBxCH{x)6)kBO3r}1XrsH6PoUSgDinvMs!qqo7IY5P zPR394&dMKfQxGp*ZpOCsXULSvzm5#jny=f=*PB&Bt3>U>BIuPW*n~F9oQqt)x_A+X zcc!2hIbrsJT^3?Ul}}B0<^N30VLI-M$G9_RP>$8`x^$v6Y2ssb^wMK_^;8Z4GhDu$ zz6^E;+n&!vgVG4D28rjn}A#~E4OLsMQ-1J%1>3jE?cbWM}GmM{*e)juP z@1dYr0L@1|Mr99~8!AeD$Q6VoAQn$aixuQ85-ORert?0IaHgsVywe$hh@uk+*NslI z&dx$eF{}>z>;<0VFI{oD!WAi(J$_gS8h3lva7XM-EK@0~jW(}MqOAhKa}xUtBxM(@ zFXgoaXxT}9a^DMc?>osm`m|8C#A(3M-}=$esO)}?kQ>H%Y`sP`sS#}_-V7b}`#*f3775#Eobo~oMODDoaSn?0LsLd_g`P0zbn z-?t5Joe!GlVO}q)b>nJYcO8FGxGV*EikJ^|xJ;vU(;YDU{O02uNq)UxwkIK*qF1A5 zdDgY|zPfWYuyJpu!qppHxgEHh+C(O zptlw@#SRMCGJta*Z*{-e;a)^s)XN~bkDr3c6;L~;TLw>e*rq#l-M)E|2>T)7)kH}j zanqNJge2nP=;f+lw?`kuu>|A}SvZmDJ>Uj!Q4qW~#+EXAt%<`9WY@%-?)Wlsk*l5W z{&V3SktXW+z!&yxBBZl;TS3w7X7hdaJZ;f$8Af6zKf1?8nr8J4eTKfw-^4Ok%E}L?>MHuk*fQ#{s>sC{t0g|U>wKtspt{HHYAR@XzD+9n0lpA$ z0qcU-h0EyORhcgEUvNe9m*_u?3m4r|HZ+v&X1Ykz6>n;(reqwc(QRZ>z;55tw$zAU zXc1ynvc}e&2wv={&i$~&Aqali{^?qRJ|x)+@DQLFweb#yzSUiYj}d?+hMEwjHGP#u zY?huZVV0T8i=P^Nn-QMTx?K~4S4gSKwh^KySne%Vb6g+o%J2@RS(tj`)u|$Dr8_JyO$9N%!!l)6HSK2P^5-D6-}1Nmu~j(GkAz$#wusGtc2> zJ9pLzmoRe2j^>Sr(Nlf5rsU0?_8~Wsz*3-?o_({MWkvoyF^hqq`nhximPEb$*e$Zz zaRdSWRF&dMR<4AsWp9A!EStf$*|4l|0;R7x+aPVY>Tc?bZByoX;<(=4R&_O#VFv@5 zv*pP^z&jICrEZMl6Dy09&vN3r;LWy*^uyNN7u`r8UZaT<@YJ|>5OV;M&=-c!fO9u3RqgRKs>==t=q*CmBh zu5y@Em~ijiev+DSy=_jowF^Hf&?$~R!^Mn-PIVX*dY@E%;0E51B4)1 zCvdwbFLIUB!nF6ZHiy4jSxTXtIY-8>t)32UvP7Mdo=n2_>ILP3AV;>Ky-L2y^2gD= z8;;u>Pp4YkL%mB}7Z-3o)^>P~ zEMXd_P?kNR95j=%8WERy{Hja)-ZiM!gZ3U(ykd*c2Z?ZT3+WSsT$uR4lKysL><#v- z($B|oxDPJcu3tAm6m||>>(I1ZT*qngQObOcXGSx3!2WE7Nq@{ z(t)KH1hJN}lJ1L1LGB2P3tZgWlUlkC1_T#l-AU!kZ>sCN6{Bt$IWA3B7^RXXD!S}! z#J1K$NfvM?SEVzPxaN%n-{oEVE81_YwCPeysXp4z#A+zF6sBQou$>TyxR%U5BY0!P zr*o%|q))3=WYIby$C{ud#JZzvn^J0xVl3A_zWp7h=hdVn9QE>R>P$sOtM;oCm^0q^ z&@kDFRx3IFp(-o`#5^PHc_m!-qblS>Y>P#^+_O~{hIKJci``2E-m$UOOh%_VuoG zhVz9V*6BBv_T^2ZKj56sV@yun@-#+W&O0w(?YQz#E* zElT(HDU00ND2|F+?ny1I#a`LnWtk|!NDRJc?K5xz-%cE`&Yk{Mjqb_n2nC$DINm^cIWaB`O*#nHc z2Y{$lli19iwO2%)+{*i*JM$>1 zpD1gpLq)RGacJDFK%1`wpWM@3Bz5Ik9U5?J23dVDYc1Z_4-8%5wF+a1ye4V&W+dgH z-Z6h2hgE*B_tF*H;%t(m4lQPLa-dF;Gw#&KgcMnGvcoijYA**P8k2||iIJG5x9?nP z$N3Io??5}76rhsnc zh9Lup!;%(k8RRnMZ+z@Ipsxb8!E5L9aY;Vm$)~9kvt$e56ojt#qxzmYSvl*QaI^5s z?tk&%)wBSb3{CkgRfiR6e+H|eGq7g(0-Yrioo;S^Sy*7bcyf(n*Q#jE(mFqcZP*^9 z{j~AjRAU^q%Cl3U(iLxw@f7-2uXg0JVAHgxzH`-8@wfU}JLzsw8~dh3nR13$kRXNa zy}O+SYpm)WmM&&&1(9!ymUeCU6wv$BfA%!6Z^SWpG*CNnYLt|*>9$slnjP}x4@yiA z$jvw+t?|8CNz>tKM%}2N*9Uy0-KNgHMXM4dUh1lu)=In{d#Fa({4B*~UCps`@v{zT z^x)-&0ia#zvPjA!13kTw`>nh8X;G;HONJ0adD}C5w-G3|0Ad>j6-MDTyPje8`(Bu& ze((w1)o@F(QNwa0Hlgq_NYa06RP*@Z!w$~S#`@a{Y*A4pVR3Aaaq~wRYICtHRJn05 zZ`Lr=)gWeda)}XFE`sJ3XFa#N{SeCo{Q2nRcu+N4-pyOS%_wPZ!$rFm1BSblsJ%sP z1Qgd~BFIEDRIi-s)dY3A5y6!*=7To-c;IEyuaPkmSX(@ya``jYZjjB>zz(k(NfLH$ z-I<`etDbT7*ccGec9xI+@zObmLIUBir6u#f0$z1iA0^fAT_FT|cU0>~(q5hekm z;EU+4tRW}f=3~mD3|S;dc|^^5ji-$TuS>JjyOvhk=sLpHJJFxP{&rMbT+P)=kqdk7 zbo_Io4Fdt1{Wr!I@W$pjyLRx`cbKFXZ5sOSTF+_He3H_Qx)7WR2sXr>iCWh5%h3#G z-XT$6AesrAE@RZ*Va4xQ(TrfsZ>1Eud<))+-T+Sx#EV>bCK5FQVk$7oy0i?7OaCm!g= ztRWPzzr?&CqU+CY&%o85Q$s5-PtLz*&uc_i)oofs16{=O3Q6B)@ab}POX5uF=t*f~ zaJwD7@T6+jJSnB?^6QE_6u9=xY3)2UxFZu?B-amLZ%oJ}j|gILx3qmwSaf%})6OBh z!ycX4esj!_RGlZ9Tx(9j1w>}%h#$QS5=mpuh;%Hcu`0)RW#}Y!zplK(DiW8s=VH%T`*~Ab2nf0;+m_~{rCu-I=FYqs}*;Lj=Qb9fnvl6ctz;t3S$EaKWu@sA+CDIWwuq=oqVf{V>UqW zO(YN`61CW<`-xGv*F<-K6upzqj*nOi))67?ikw)R^brN^DzVoq8rSJ z!inwal@{1aR>F1LD|NdQYG@30O+;EWws8!Wt!i$NJ4%$jEN6!plnEvGnif&E)(Bkb zQs7B745)s5gE;qfe{3huyB5hwyMq8ll;`_H2z5$PB7t_3q?O5SrpHgm#->)3vI?Fv z5t4;sOJZKT_##rh*6_*I{AI4f&++fL=tv}3f*)VYq!7Owc%4{I0S?TfC6?mQeInEM zBnw#n*t5;$dQenV1s^+@3#X6z`_2SpGQ_W^KGM=PT~`Hp=8m{^{39nh1xmK*)niomyG;m%C@5k3TQ~}3!k=MCmoB!=eec2DMD|E9jY(+mOkq` zkx~ase6B`NV|iUKm6heOvvyRfB2$sVtV7W=I(_^1e(H)2R@#zuY=-bE2lx8+Pdc(j zmIVb^!&N}GRP}-0Uupvf-LeA|;WBVUl-=Ms{7lv`Ko*wDX|3v*Fh0pu`myBI zfQz%Mi;Jw;s#|L6h>eSTYTQ6_X2fJ-|8lCeS^d5Mu(>=2-Sz{wXIt&*S&SJNfCfuf zx>9dmyLqh#lZtX8gB z`E3(K6y>d%43|EsVeGhoc<Zcank8bzwgX9^UeI2Rk!M*>YO@t z_FlEu+UI&Gh%*B~0RPY^3qbyN1J%?702)pJpun~NeEt6dH*i`TFH{%85D*K1;)YJg zM23P1+GY(Z_5ooBB?ZO_qJ{*3!iEzAzzD(X>>!hw-~wTErZS$s z*4n7=rOp|dZ$%04)vw*TWv79*j;!Ik#3Fpj4^Rt_0k>tG{iSV1Kh9pB8*$YMeyS(~ zoBH+r|AQNH+=th1HoI2H-oYUbp|I_O=JUu>E$Kwu_UYVhUWUmK6m_qi5-KwL(VB*^ zcGGbl5=?T4ZElfWma{aMyXHV!NuZ*#K4MW#V6juhL;*l6hBgVFIGfaTi8H!6ri->1 zr=xh9FE$uiMrF)}QIpoDqY0*1h((o~RY(s8McW3Jz33UNq>vqpHAk6$_uP;`X*60( zjL%92f-W>yx2){LnZL?KJ`H)t^Jgz5zd~-Lg)|WK9ay>B3mPS(o-rmrB zejHR4`^Km^b%mhS)&a;qUlVKsT;UX!F$H&F5Bq#!0kW{JrpyPXyc)6HBtEw(g7RFE zcv!NFz!NCaa8hzEBR1oj3nBdAis}N#9|G=%(u?APNg&`!Y)yz-1xN*;erO@M2dHIn z=p zly+tvLXcsxB;VuSGo<6K{?7URxV#9iZmqvGH(T`UgZ^-)rAxyuoIHuGS4@#c@}A6{ z>)d+%<{YMbl9(RtrXvhjnjwhgntLm7N^aZ_m6a384QTTKe**cS##&MGNiA_}0aSfE zbz9=EL#PX&sTcqX;WiAmKP{oSUUPD}2x0K^Io!83VI9Zq7j1Tb+`-GL56X^U_sBf^ zH7I=pzdHYh+g9z2x8P>}mMD~lhs{|u3Eq0r6aOS5V1;E9`h5N|@Bf`M(A)&Qi}Vz= zKa1LtQ2Hh~?{fB_!fH78#{VukRE$qvbxaPk8O8J7!ne{p)mQmb-;%%bH_=G> zez~IE`VdcO{`20lzwo!+m%)?&%D3FN$Ctp<_wu)rz?bs2M4!m>P~FSHj~0b*udllN zxbyGo@3PgS@O7ipCk0FD-u?XV&_q1!&Ob^Gf0Wu1$y-e4l8&!ma10PV0Bc0t?ufjB z-!myaMvQf@Kp|}+ONLy>TdIw+s^}y!|L#YQyUCZ z>-bk^;^p_AhDC}h388HnPfcE*R_of$CR3(Xn9X}~_>k7CZQK*eJ+(oS<3?1MK7tbOcX8+zEeP5hqGAlBbP%kUEc&HW>?bo`fQc;5|NKK6} zMk47=m^WWRLlf*^HIbsLjEFHI_+=e@GOm~jk#qen0N|rp^L%du{$8?1cT4l##%rqP$aDD67~}BMC1o z1IE6dor~|sXbPk@uUnVWm#-q=eLsY=4x+teM;^7U5SM$rUC#_ zt%>wW2pK(3(-3!OPd`x$;Qf>BYExgw{Dv&6Xce&IN5W_)qwv~{ADf=40A~c_!2tyP zeG~u?{*7N%uc&B~hD{+%m5i`pv5A`c7R|j=KrHv4b#=!_gX8Zghxl-xEJgIT#J}ct zLewAz%^GPkYen9o%}f$oS{f#nh>hP66#+M($ITTn3gk<$9a9A}X&?Jh#fkNYPl(W{ z8J-lR1d^$$>U` zjm?jF^^T#MLMR?A&_r(WxR$ZiKCZrb4Dk1=fREct`@q=Cq zN~|O87nO9kiolsTS1|Oy$Ka4OR|7DJjb6e;7%;RoSUA+`Z2+K22qA|Ymax&2NP)^mRaCLOVpKiD>L8hE zR`t}COiHL$PGKly8OJzQuM(auwuZ%;|4*_26E;sExu$SuxvFG^xJ-|zEYdTIN^){S zRLwk0W7kmhucjWjnRU3?bGSLA8KuR=)0^in^x@V&*VWoDI;n^k5E%~w)g>QiTCOpo zG9hG2gj93fj}F|`xk`)!_v{jSaC|R%^j$Xfzls*VQTOL!kXS&o7Xad*;Y6O;&^=AS?h#*wM2gpyK*xV-)H*}B?+uGLp`J=qnZvv;KIBwIGMq+70v z(!Teqz*9YzMi37CQ_J}p`aU;f?R!%X4T;i2^2-eXOy2wco?2cU5#EH^OoK=RMT1?R z?yEnZ_NjdI=q6j2xLja^1F-%Q zDI9M@Ucyf^YwuCVwuT&3{w+AJq*zexFDXwq7a$&;R4oKSzVBd zF$!f8!ZiAp&iPgPJjGJ}&S!$&N`pCN9D_VW7^A-_<5LJDaKEo_8gaa4l$}Xr<5hxA0Hi_r*owrIw9YOav40>g9nx@fbPq z)lE)@O(#|+Ttp1e50jJ#65r{o#N5!=UT4f@D4HgcYWs~L0C&RyIoUjf zIIp#SZo3)5X(@N}k@3U04>xbnf9(G7_~Kc`^@7=e@}W~i?E1X3H|BZfw)BzzmHZQ& zB6LYQlR!WYfR^&-s&zr(oJq7Yy`7cD=|sC5313GQ`|bAI?6+JX7-AGba^LDCWwmNS z*>CoheCiSm=NbJowqtz@qwa{^Svqk><6Kb!|3#n1vh#no&jmy66uuS;`}277~%>3Z^-|nl{VdI}!PYXfOa| zv?1uEBJ8vyaHiSb=xUwq_v8>>8yv;>%C#CnmBCRG3{kiIj>ayU2JD;Egl7&fRyX~u zD$C}Dkp+*jP(@4@mefXGf)!KyL|GCq2Aj_acHf^0ME4|1L|F*%QAm>O%l0V~5h991 zP$rIp!x%!Pxcm)-b_892WJ9NZ%^em(&rBLCtQ%*}Ae}P4BCncAIudIO(F&=an}ovB z+7jkgVr?pRo!d>W7NgS?rJ|A#+DpWN#|F9osv!<>tCAeroPcP?QCKFjNvxkKiQub& zajHpMvAknW3)=YzX~8U`6WlZ2_CXRxcrCyI45kbQob@Ba<6CU*gxZ{mnRD&R<;5W$ z;r^ul3##3WG7Lr{>?)jX1gLwlcJ97-@(NfmZPT=KhXf;`3_~Rq!!$_}b5F#SCY1}E zqqM-#O01QrWBmB-~E114&^N?Bo><=)w(tyN?ClWYsua#@sd&KH)pW!zYXHdgK}+pdwVO{;4s z?me-}7^DKU3aky{pYs$nmdk2C-l?+lam_@?tsbt3CsWSUEmW=T7O!5yjiJbtG`-q_X$L=<4jC!ozV;4wsiLt+xkQjRgyT6e!2>3axt)|^}gAvfVf>?~2+ z?hf1QF&Om}5n>YSO-nEfA;M@J!>(SorWIc7?pa-?%e<;Iu!N_-<;ZJi6}(ytCI zW6*D8dL?Rcoi5M*;@{RnbF^*qs5$2}qY_u7gIQG@LJOm3dLau9M=YDl2N59skUp7+ z2fcxOw7#ast z3}ys-3nanu_Edw2A3_#_Q55p($AAH-r~}c&O)4!!)KwXJA1+Uy8Y^cU-IlNoT6fN(DcsN41|+k{U!q%y zPVO~$Rm;n4tgS6>uEiYJ3Wyj(D-~S&4!Ie*xgQ4U?KghBhn;^<{d?Ig1XkmY9#o*M z)pc<@lFkOI-m07I}5N8`V_s(V=fa$i+W`}e78i(M;Bk>9l~5!(W-&AVVyS%p24n9fe8CCOIh-&l@lIP=&CQitW9>s z*Dd{E|B#%u5zHLIJ#7=RWO-o{u~dx6=Giqd>RMw^uO(>6U`AwFJts&80gIz>txG_j zHbsd9lb`aK8@nu)6V=U5Di_ba95Hz5ud5US+eklpd)lnaYBmJhvfxhIO}xm0 z`c1o|OKU5v$AWw6>f+1~pKJcsJ4{4PU@_Lg+NK6q9wlsy6joTLxw})&n#`Cfh98t= zQW&ZpCl+!kY+h`!di-$~rZo49raGJqZACwT7S{_SYgCoQK1J>cq!cm8WJU->Ja5$# z&+Tf7)sULc&bkZki7K-W#~7+}0*1z1y;$@CfZ&~VF_bA<6fP7pK|2=I7lMqM_x?tw zqHd}qD(LwuB8NrHmR~A~`hAIYfE)Y<4z600x$p&ws69X!t(26c;wM`xD9%h-$|7|M zNojkG5Os0cME>+kj6o!ZT3V@X^_1jjY{6qWzATeJwf4`pxjC!Z_UkGhkUna-0cQ>i$@aBv_OM z8VV`KBa97!5DpQ7lH*dDM40lnz0ozBT9M`@1gz)CsGjZ|@k|6gq@!ok5+t%t+E1U_ z8S|juRL(Z#5n;(U1B$91xMb4HL*6p4%ZsOxK#R2+W=J$gneC2Ohm>|r{+JV5+#l^y zs28esme>o)C>*S=AzO`9E>BpB7>5abB_mxFql~W0N@Nig!1FL!GwoL(LyRmr6t5~2 z)o6n!sEx7qz;z-(W=0=p$G1 zKF)X{!@8sH7L-g}hffo*JCI`FJ;|nA+_St1p=2pq_@?FAbVZVMNKm;qo93`Put^O4 z=3^h`-6_#>Y|AD8p+Y3A(cxhu@#jc{2y_wl6g?WAVqYs_kJ(JD4GnaVIEl<6YF!wm zM0!-tQGS5a#1>tGe41d%Ihu01G{qKdC3c0{(lwu(u{6wGptd=OAYTZ_giX$!WsBo) z8RU^{in39f`M`k`F_{qNgfMKf6~*T}?IueGp94crjE1An&CF0AD&kaDDy=3A=IMxp zX&Li4VR7M7RSn6AMV5NM4%bY@=62}k15rx=*4B#%@E1FM$FEfX{78G@(8C!y)Di(5 z{h&;|r4VVoCrX{tRZBPT;QzM{VWAy^ov}PQb{hG?(4#UMs+wgl>(J|pFyo|s*jiUa zD_UB5;%!H&N9zYt#tV&&AQe$gbpCXkZSN zk2XvRwQ$&QoF3KB)Qq>2_}J7U+W#t6hONrQaXjXK>|0%Up%-lQ3r%eYhO!<=py6^J zB(!qp;xE&k4$8hP3YT|siSpoT&Mfg>R$7g50vrm^oyD{u)Xo-`c*`m)r$yPr)i-O- z(EYUFu+ou&y+deU>+8VrZHV;f3~P*w5=6#m2AaQ(=9t+IFQ}GK6B`$oXf`)f9 z<}T`8szQINM5M}4xBgcgkvET6H}eo;2{GjmQe_!&={t7amIxV#amdw*UG{I9{}R^3m`W#tccj0hHrI~`tx(DPuCzZhSXE&}r}{O2 zFEkyQg|F@&5=Jo1+z^x-wY8Bm>v9KPhyNlF=zfVJl|+iaUdT{=mMbW5mg7P%g$Dox zWG0eIEy|Qg>mI^D0{}^+QiM|bj@J(@Tr!QJimd@2?&JzzdQDrt{604i5!wB-PZ<+F zsAGm?P_P#J;%&)|`Mb2UhCh$^TnoI*JtDO4kD$d+7 z6=Bp_JTXN*66atcwL>eUEivxyrDi$)T%Ix%U}wlHIuFdmKpclnCf5^UDyKxBoD{JZ zHYw)~{xEh689m!elGCC8V%tm|xjl{jHt2?efe6eNB4h-E3Fe5zmDuPyU;Oi+_HXq; z1BdrSw3t$+Fsb4ej9S-@a2>pV>0V)fl2BZkpS!~G8Che1cu}IrlrVcu7h;hZiXzSF^u#tcVIdrjT4oR*OuQd8WO9GK%fr({;xVl%Vcvf*<9^Fvhg z{!kX^R#d1NGbDg?;SGs17Zw%5<^d&GpRCM7HB*hw4l%B{&>ibx+htQ{s$f>Em#b9j z2c)Z4IoK+=1Z}7%t16AB6ycBbwH7wE9p4@>3FrSut+&#Gt~~)_Zn6K*%t#?pvsD|5 z?X&nkVGH;tp}?dT;HDmeuq^fuFEKn;f&nJ4Ex;J`;lE}2$AlI8+lsLJ_3k!ohLVur zt@b#m-a>W1Zs29yYi;um5vI^9^9OD2S2rSUbYQ3wqlUR8v@^Kq;;S=OovuEo^2Hv0 zGsE89C*H#gaQTDK^`sF-8dgtJx?9a6cbfmlkraw19zHlXD>IlSPrZ{+v$e=!V=7y9 zA)}h5&K$0PJkV{ovStd%xM8|tx?(ui!6y@vAAj| zd^We*x|IS_4fVPr?O1KMOEyg_wHk!gw|lw3ERmMu749p6{MG!K*&tA^&{kOa=&O{R zg-*wjemHeab`HFM^FeLXrks)Q+Y^t;39s|6|Aqqp50yh6+VOr!$AId&TK#pa{zoxr zi0C2I2vX)y>0*`)+Lz)R%r9|!hcp*5+=t##V;Q;s!jF;-3Jgb>OPln6M|2Q0=D!M( zP${YSUX_m9JHw#ke%Z8_VlaB_7p%tdnAzIh{p1i605Ct44nXGaYF!0wXpsKL;3RDx zueVO-p6zmK*2Fu&82M=iC4uI2XXKd|fz4{oe@`C)fPF!Bl>dwY05ok6UMG8)lqXzXr7CRj7UP#8OXH*>|~SwHpP)P@b5zECvL|FakFoc;9i zd-5t6lQFMIHnx?LrPS7wg_5+1lgCW$t}ckZ{f=qDVQ>nWf})Xj&}`bpUR`~k5Ynvr zxi0C5d$H`;CJ`(5uGZHfkY%7f>QH7XCG8liX2}V%AiSPO>SK?GKy1n^-TIM*0lG=? z3h$|eAg6lPZL_IF8iiia&>1VEHuW{QKM8>Ux<)rxbTOynne%2 zXBBz*?Xa@kRBOlDMC|+G{`tYGa}H4CgWlpeu3IoZV6n?=ud$I!kL#H06kSH(>VoC8Az3llWDr;MYgs+Rqc z?c@79vXf$nsyx?emFujSyQ{1Y`EKq@uo6Q$Rl9iQA{*#*EN>pYVjd5S{g0s{Ce>Il z>Y*ExAjC!PKbH-GHrr-2;X+a#QTN||^ZxKW6Hq#7{0}n*L&9qB{<8|%Dfcc2Q|`iT z)G7CEg_8|rdZ{D?^pv&F$jZs;*_a)QvS+j&x>y7HCCTHZ+5O|7NygCCrlFg6@((1wgv1HVGHj2Le=d@K=G1Yvn0 z<1EhyjvIq&58D`DL`C+dj4?bpya@QUdk-v583=<~f=IxGF&BuJ|F3@s1m$L824BZR zXg9z@M4cl}k`n$?M`2o{(8iqOlMqw2Q{D-D4n_ZqtSamad-hcv4jT@_CF(I{&w?k@ zVWn6$Qq(g1!!iO~RTJH}3*)6@Rsk_eMen2CU62C?aVh(Lap@lR7#uF@m4x z{+|X|I7^+xS?uGa-T&TdESuNS)!E&F1*%ySOpW#q4*_0*jqixN(%kOL(#p9X@?`k* z9yafB&@sdr4pe!4$Hbg@(Fexq4Z~z~UQ{xAo6Kpg*C(_dUQ-4GFhcZK!NA?7udpAu zQ;*Yyu^SbNq(&~^%5{5f8|>od<&D0$en>Mbqn##SJS?#ApDOi}cQpZ4Sb=K^#f7wg zt`3)bEN6<29-ur%HmKluR`Y979HzwZSu4acg-c`m*13@n+R0;b{qoVEt&123Jv3?e zKMBshb%+7&MK>_(Zl(JH^_eeTjr@CPG<%DkVb+mtBz#ZU8&#@uZ`>^>Tp?+ccd9C} zYYG~8Xz5)R_%6J6yPmJlKMohzUt&;$%q{79Tu*4RXT=j^9fd=f~~`_@s}1wnP5xV{3ISUwdYEe z)(f=GKtHs|SJri#2&%A*d1yUDwknDma-!P)xho<(a71JQ(@A1|G!a?qN8O}~!}ah1 zcYR_hB;-Dx58rtfQ7v_iXL!V=d&3$BT-K|W3G?nFP~57rEKK;iN-H`6?p?TRtYWLf z0a`o(|NWPg3F}35?ru~hw0{}ek6POf-2l1j#Vc)z8E*Z7Vd1Um(8+WuTU+{(p@M`o z3bxTi)LPc|Ey^QQ%xGwv$QT-DNUcqSYvJ9413tGW~#kr<$Dz}a0q zV71tmltun%w)Bmt2qyhayt#6uiQy)h3(Av9ZC6XlZt6@6BT90cW{q*%|3!mcLtULs z$kIqON}V_U4F3x{Djgm?2Ub#iUpERi-DYO^Qunr()a|F(E!|Z2o{HZGPo{ZuCdKC zOT}L+UM9p0KFT0&U`2S_TbXIsM_sM=D;rhMBFcqVQx%gNvb@v#HgMW|IlfPM>r<-y z?y%Q}70J5-&(+;28T&(#${-USxyG+4L_H{)%l9_ZHan-*B^&YQo#)-|QA{Re6Zdpc zeLd96GW4s(k+!Gh8*y`{7%^_Rq0#cRr^AgX4>XdSCh?&*ep?2NuU*b||*OG6Ko-9b)m{sk?j}Gm7BlT7(RA z5Y4;P#o-Z~t4~&yu?#ac$p~&O&g0!4I{W1Jnqgb-$>?fMYHhFSRj!L`)XX35=4?lgsbaX2k($)otTF8seaKtC z>J&LaR!Wc7EHD{(qH*4c4p&v;S)8%dP!yq@8BrHb7uGMh^GF$c5>{2AG5IErnXOR! zGO!b|2_spUm5U29Su2~RTxH!YRnZL!RI(coC~mmPuZlPPL?FA2?|g1nqFPat8%}Yw z>HxDv4d6sSY4{$qFC8<^+G5}=uWb{yd(KP;ljdf7Rw7{yoB`?0zf;bfHVsT}gnl1I zj}Xe6uETWItzZh!^+y{vb3w5doh-}*Opd!$lhDtU9Gk}1u=6)@SHZr;#@6^ZL6BPM ze8?%_i{P~EPvbl(vJr4QWVH08TuO_w>S~!JRijqhbLfUvutJZHRI*zn zl*Yx(>tTeAqf_HcWlxpitku_!9gpL}(`M=HNZZ=J1%P=r(bQhX=<`!7&Nt_Fl{UUQ zJghF2=r_-_51g7eZ=qbP5HVE#B|J~4-7WJ*5oSV2)Eq-)Dr27Qe?sHb0uwOoOgtlA zZs7XBIT4w@iu~?n4jKHAc~8Dw-WC6BARIv+mY23 zD{F&SNwE%l3!r{LloTSH*$QWKE4whB8qW-6I;?KO8o2)ajBO7H9csjgq_d8E>+l3W z`fPCo9W08JZ1tx|R_}4mj2uWfUT#uZVl2T$H^ZQrp;J)`5i&K6ygWY%Ep7liUNqSes{+#t~ zsr&s3;P1O=8^wR+e0^B<4A>UojsLsoc^w&7o(IkQO7xT%AIk59m2d1PzIOK3>sj;b zqxX)j@<+rU$;?w6Ru&^rrsy+0fPw)GA;gs%-PDjxpMjRhO7yO~4@C*NzEy$|EM{(}&Fci_tC)3!A3(`ps*YllmorH#V^9}J%|2nr7n zJqETqzfm_>CP8AmjzFRXf3_ zA-8nW)nBnQN#We3?r2>>{b5&I3QRnKM;@{6Wmng+F(~Ukq7&(nn6{}#1Tzt#lNZAE zz;0`>SB(v=_O-R{r#`_A;;KVbX8sE8Jx|N|kGD!dHqw;HO3)i6UPUTET>#AaKFamk zI?YE3<&Zz5bH(^X8Cn?}gMeY2H+88K9fIDaj^4YM%@Yt_u0C~0Yw%DuH??e0%uA^p}zBM*0^QWgp+8$q( z9?pfhmHmknyuSV_p$myhJ`cvpPT$(>ae*Zf39=2QJsTsC%gqd;W1X7z_Qb!gNf{c^ zFYgD+6eBWuI3|O4fMTn0!OUnpxsEu|i(ODWrs-%$vD+#+Bdsk?sCKR!Gg4j!{`;kT zY{B({=wF&k`*zPjH5$gnRzau|+q4DK+({FC?;Adymxp~`uN(NWv*HjjbwbnBqYjli z?9PkG(!5V&lDVVs954y{S+r^l!KLXIcGdyqU2DjF)ea5;w zUfDxl1)lx(MRdibqSBZ*Aoe+#j{CBTZ5F`SzToIJvGfVmgcChUqhW^;fD~3?dN6m_ z!5N!K6nKo#M3Z3qZC;0q22*qaX4K7l;1t0@!j`&Sj+UQ@E3Zm7zc^ENw&9BP!3N| zVXVuuH%2VaZ(JBBkl_jbHrD$t^F;rWd8w9xHSObMst?BYfT9#iR@!Rglw@Q4Dcc1d z*vF1U1h)1Mo`d zJ5*Q}+MVpfFF_flKBz*I!MB`1F<;cU$V$AjvJve7ga!VOAAOUg-xQfQC;7;4j$27yu0sCr z=$-<94O(4pt1qkhg(<1JD#1Com1A;#MeN?qf>RS(oM`a3W?RI0i(J=c|9ZuBWtAIRn zbhOr`;wYRU_mrV%gB6KIqs8J(N!rq)6scObY!&wQwTZM0O&w`Ko1oT>MKl+_KKd#Y z6fgKf=i%k7`R9_t_o z3GS1r`T_z`j1%uUs>%pI?ym#pe40 z=?U-Oz^AIn*lpN;1Hi2R7ErT5tpB}p-xIBy`VMRepF!mq83X`q%G|rT!oMhB>AJWq z*yjs2DLenN=?@_~vhTc!>%Q$20Q_ZxdgFx**7rU|?|d{A4uQd(@CuGb4xh#erQ<=! zneUXmyE9-%@F3{m#V;>(TQ$P(xqrJyB^8C}wIwz-=QyEh(XC`Yp{`vm(TY~LlUH2I zgjj|q>4{0EKYz!+z<~)K#ndmI$3ocszQ~Cj+cEHvo8iofS9bJl#Ce;qBY)y`UE`|nT<9EYGg)T; zG1H;7U1QhA0dzj*u2{Q^^@06lO`Cjb{<%^A`S{WOS=8%#+{UhNvcQ}TLe&Gz)pOq9 zZhlNF`+|mp844Xha^dV8;;CeT#*#xok^vO_fUw03`tC~)trPh6#mzbd{N8lMiGv9Qbm-cF)*W@9s5Sbk?(0RD`Np!R!G-oe=f&=0uA z_B?cf++p)A3wV1U=aH{?ymzHm0DVrb=3ZomNQ8(OF9m*^IRf#HS%(|X+($jfz%U0O z=tqD;`$%Bt)O^4=VOVHWJSa3U_yO9V10khMLV%Vyw?V1$oClD=Umx?}bSN6`&mS=4 zM5(!bxEJq2e(;gPCikHQ)XPp{pDX|aQboq%h~tv_9Ycbj9_630uXjb7Saojrn9$Oq z=uONMjTvJm`DWZ>AG|!W{Bjy6x<~qFJNw_g*=qJ&a}DPfuO)i70=<51vlV)4_IR?D zQRmt5y6W5#c8^1i?fQK;rVO9+gJhrn@@=1g_3#{dvK?QGe}}*F^4}j91U)Z{2&l!q z^cNQFyo|NCS}5dPzUj{SSo^_K4KOGt6ZDW&rr(X$oA=Nm;UVE&QDu#)Q;s{JV->BeI{Ypc!$$pzdnFU}J* zMj!yld0iCFcX|S8`JYv^4&Q8j-?+G4)I2_dhT0bz-)m8-gFmXYztU6zx;I=)E|;Qc z(Hx-Bg$*2%aBseA`bayYz5}d(WuMG6BWtIpq`ay7RhY*QJQI_~f*sEVZRSGfz9Q;D z$^zZ?z)}c89*!#?57&fxNsZo4VN;G+dO9CzO1NqK^Iw}hEs3r0hkFcK!>P1kkU0c~ zeacWGin7L!fU}r?uMHJE24j_8~SPv{yTDRmBNgNE717#j;EwF6`)do83eB(>ST;7{zH8 zk!kjw+(;4``4tuU5uMKvGbkIf~f@URg22(eMmP^%Q{0fr($ zu~6*f>mi0JgX_M#lu5G{jH6aI<*^_*Mj zH+EXo-E(|Xh2kpu2APCgo9%h%6?g1;sfP3c76pDa!i83MiPmyN!0NoF1wRh8=-MKg zwM+o~mvrKe&KzGg$b`^mKGZ7dadfnOp|o5$g|S?M$aAl70R&WeN6s2z_dEUH63l4K zmFsd%UTLBXs0w?47lK`qka=+xZ9o1m2s3^T-nKI#`pLErBj<-=n7gekFbsAne$#b5 z)QVrIXyzHMM$;mv(Z9oC_cnv60sRoAMzJ9Gqc@n2KV7#cn$`c}Rbuts_tbb@X?e_u z6$}VifRcitM(NelkMoww894KmEum%V4k=)nX1dmb^4j)>|D4yzA+pqW~Z0}Ssyd>Yoh#nS$$8|!I& zIhml-2Y?cqvncN0K142h*Q<)~Lg+pzoU!HK8ANTOOXv@rxL}k%*Q*6h%gyj948Jy} z9Gq%D!bF4A6+zkl}DDE#6sT z6{duG{YLJfZ%xT&r@LWT=+3D;G!5owuD6cv(F133_yD1=C%8$Xqt&yO zEiS9}FiI)#GGT-hQZvIyhJ$xrK$XeK?&@4}H)hi@Ze!6eID+Oq78P2|fBWa1({ISa zdmuH7P84sc;7he^)6oa1dzMMvy$}P1R5Exc?RJkBxll3Pi#IHRJhU^sM=@>w&KNPu z)u)++)xrS116ixqzB%pY)mwerxbSFLxvQEnB$~ez1}(VW$B`qTnHo=-VCTBMAC4DT z;1Fs)n036JMQxYb;Geg78UcPSr*)VK?}LS21&`<3UJZ9%u)P~ZsPWtdTqf1_s8cD2 z<_&j@bhzaNT!=xGt~ZBgX#>4#)%36^ZSphw%ZrQ91&=7EdAayu)1O+-D~;qZSQgRszQ{w9l|n-Y*xrArm9{Ca9UX*J{< z9RjKt?wzctH8^PT&U^8zpjQ{p5ZQv_NlSEmAl07F;JivP^TB7S*X9NX<`+NZIB!CJrROlEZESu@?8^BeHB;XbSsC_B7jy%%m*1z zgskxErZsORF72|!mm<+Q;Q&<=lu#?;JCP)9>Lr{Seu40+1VNPqF26t$3_^^Al_KUv z%eqamj!2qnax6Y&vaCaf6d=K5Vge`QXoVHOl%Jl+tRojr78|1p-;RS6u`rQ)>`Dm4 zNBxlaO4#~I@;a@Td7)h6ax}vv$^;dc;1ajNijF>W!S8%6&|KBn8D0T~FjUzQa5k|RrZJF{o zuiPShd~$TjVIld?$d0_OaOzBQ+Zz`rArdo zDeCq1KaNte^xL#QPb61s>u!yX#rsnju%!$#}MW4HNGhb)5HS^W?P9-Z=}%Pj&9>ANrDVQvtzBIdA35XQot(CAv8 z(<5Q0^FrB#k2Nj}Z^-|Ct9{1j6{Q1wnp=jNu*`6N^Abdw_g#RDSXOQC5Lcx|(h!e_ zNC#jaTlLi_CL`YuT8G68D!R0v4*yLADR{(zpT*A%omM?jjy)7!Ve55BsuKd;G#AQ` z3kYg=OWa5_Af4}2o^MTW(^j)7iiw1J`}f}C`)o~5$;ah?aZEAktVLZ%Ip(ZOk(Tg? z?n281?qULkp8+Zarfhxz?u3Iym{p)n{W!FYfMiA7f!!_0`2AFP!Q16ZaNX_so}sq( zK=b}{mtjUkj|&1m4pcNyGw8IeAR~BiIIc{cKrNj2Cy0E(k_ta`+od&0)p9FJhT=F_ zVFGqQwd*0BAbw)Mf<%j%x2+k(%lkd^6)H*M!^5mG8dw8g7?B#QN>*ysCmJq+5Yh4z@iFPc5y#hxAz6BlI|9nE$U(8epjd^Lkic(n429b?8W7cSjPxegUo zaO(0tz|m0XTHo^WR2-4e)m2Er#(hT-#HA7Z{?j457mHS-F^}8@A@GlZu`mde%R2W8 zT3@H5&EtawHgYO=XTeQ~bzwr(s90|tP)L~QOF(nP>P||p_4fy( zu!+?CN{NW%F+E<=7-oQpK|?sS;w0(z4BPM_ zW$593BDb8eG1`Pn@dOC=y?E?dp^jVVK{O#;Bn8Qw>3!XW7K^4+7L^8#eHBo@A~)CyLvt+=Scz9E{& z-1+l2hfmVzlg#2L&|{f^Qi+=(Zf# z?1(}r79LpwF&fK|@WqPMn7(7q2zEG%uD+ZoJVHx)zUonn5cVuvQt@Rvn(8U*M=0KW za6hv0c?|?bAk$e9X8D6ET<6Tp5E+5Y7R$#7hzyS(nT7BT+KE*u1Dv#I0$p#C-!Ip< zgDzH(XZh3x29bS@1>xa&`)#Q?8m}BuX!#8beSi%R;+Q{xU`8ASGU(+Kqe_Xcu39Xa zF}n8{AkHl>uMB!^kF2clX|ePb!{j3t%j&?GP9OaUagdrdI^ikJb}o`l@9{0880CFs z1JaC>hiKVj`+Sm=BM(6k9e)Br6IC(_S%Yg1OF|*@C2BL+_n4-~!Wca@aC!plSh3nc z^rj**Z<)Sp;Ywx#9R-IUNYf!+r1h2M(Br%uH6EGYhh>YqiUHpXJ7rsQWlHz`hx0 z1f4qVIqg;eqtaogQJc_q9+>llRmki)pR@id;rc&B+Dk_ucw~Yye8dM)qUSW_d5OZn zs7+G?6n%N48VvKVE6}YKrDhQ_cak_sSTAZUW7`sVdsU?ddS1k8mY0Qnmz|kd4RC(l zgREqe@7kavH9_-Ad+?=3=3EyZt<>p=eJkX053BRIqcMYZ-!*~XJ)`tGqkZ&m+!YB zqymd28BVpl45MfJk5eDK=muF{K6!ZLC-;Lsa3JF3=|PhFB$rxmZFUzT^16R_G3YQU zy<;Jy!%gdU4&U!#pYy?DJ+{b|mID08Axih_RB)?%Cu+8gEyHGW8MsDcP84DKP-*6( zWe?3$G>`m?HSwV zK-|Q#3-*)4=%m?|sp>{55P>WRh_Y(=TRpPEPqL89_rQKBm=ytD*-jxXMiG~Ka0S;U!1y!a|nAcLz2FE4Xu??CqO zz{uPb=#X~q!IhhIjYF_&l6=}QcsLlC3BV}T|9$Pstsm+mF0O5d<# zO)YPJArZmxIwNT3NN+a%&ysDBpz>z6$Rg1q?G<*lrPvS{^baFg=kI&wouev48o}mto9=cGUL;Igd$pH$( zL%s-6yI{mR3>kz$1d|PyhZ8cQvqCPR23B>1x3nJ zuBhS!z5bNf{&VdGgh84^0u(ZVsQJ#~QF1H8szzYn&GUzdsv9bZHOn8NDPQo+Ex(eT z@yeSXCBn5#$hm=HLMka_c&>jtHh*T!8L2^NM8FI?QFtIu`AyGTF&pIkkI@2fjChVk z$p#}EOwjamAb7MWD*@74_sl7au{l0p{b-ursy0f;0ibiZT7S+GWf^NDrdPR@0jE|h zWgn!c)zoqXx^K1O@V@(WIdFdETD@DiSzcARLyoOpzlxSpec8$-LsRz2T{i0Y;G+(a zA~fx@m7wDmi)8}RbUhtj&aM-70xy3Mr9!!H(%JGCzdLS*mX8>}R~wx zZgzhJHGw&-|^p+{H3N)RES4ke5n#XBNv5iK3kZw_HGW8?BpN^wbx z4BK~z32Z^3mU)|z)4L#ZLu^Q|46&?+Qx{qYJod!HdI$lTuI}+dSa=s3!&SyXz%sK1 z!KgLZ`cg>^xICMq3Pd0c){y76lL$d5WLhfHt$uFF_ko)rophH);}4dXaL6!*r zAVt%~jfNBwQBcHmp_<+J0O&SMcQ zFY1Z~6U(I_fdW=kEmGrshw>aIjOnm#nai9N6$cTYH|iH|K}x_iosS^%tdH_8isxXh z$_hcS0M2Bo+Y|?rpEdP~iBslS<5!{4NX6-?;0T&byB@ZnW%&+{VFr3R*%MO%=rqQe zL^ME{``AH>x{6GdftSwIR&-h7HKVa>y**3R*>TzeMWY2!aMogRHm|u5D~Av9MT>wy zb{S)_D#XT`fwpMTL{OuJg%>n^Y&guymJ|bBO0~u@3QT=V63Z8hTLfdz{`ts)~TjpzbT2alom zM8;YDOLIhfdw+GD5r~x9lo?J(lu0xKoX~`rrHYG3!VpksYz$dtZG?oM(l5xw3Cgx3 ztHIlmWk1!hVS~U$;Siiy=pjw?&5n+8O}(G}-ZRpnxCN>Y+jYO_-#@=*N#kPtY8p;L zRE;$Y5%M6^srr#E*~aEkwNvP)q<2$~f%T)0Nrssd*zXG?eF@!Za;#YTjjL#pm>Rx@ zuz`9$Geswv)OE?py8^x#Js$qDjd;!eP3;n@$PmRQ!sS9g9PsZ@()&@zq^mJCE(|4u zc-HsadJG4!9}|9f@foJH4Eg>IfZz1v&mM%~+vcwivk#O~cCv4c9>I?QlHF-deu@;I z3X#6Ox|GDfGhaXCS+?5!^Ig!Mp0ouKSw)Df#OyN9)-4KgGA~O-88& zVeEg1Dy5lY?^q)YF84ti?rL`*h2M`>#T+ke*~$Ylu=Td*^V<&!KI)DVMM?RFQ0hLZ z+-}pBT|LAuTw5UNb=bM;z&ops4F`V|U?|@9 zUd6_FS}E{u>C( zPTu1M0ZTdd9klT&CGvQ2QC;N~T&to_c_!`~fSqy;e+&dGiv4v=>6!1z9Ie2X&4?`Y z8uYLZleV4==JqU)UpI^eJ3Fm6k|sCUxRGs%>}i7#0&lNL4@hJ^!B3rxX=wc}Lv z^{ykIxaK`2B{O*im{{pSB8NO@c|$n{ zqwq-96l2N?y$52gIb@r6-~hCKB$rxt!+V)$8_cwMvu+BUE(O##!p~KQ-TlO|)RY9d!(V9dd!5%@wEfx)&t>Oq2#6%R;*ZnH6X9yH8919WM(x znuT`rSvQ+vU;jbe^=I5q)sDkt=g+fhe`_0RT-00X zE~Ku)#C^&x7NQO6OpbNe2)IQaqNg$>;M>|+cDaFVR@KOs`ZqmJ-td%1O$Q|aeE>9h z{ig2zr&2iJtJ)cMMLxSv3AX#z&5x4O-5t)+A_D?k3KbJ=OmPd6^R~>xq~$xrmb!3@ z_WXw3=`8on!%Sur5cE*l6{1!JHcBcs*KWBi@N_&^I}=i_j%ZgPtiim7Eg34pdvQeE z49f?*_AbGzSElB7n~V9lIG=#SC(dPhBytTtLLpPbjtCK_gd4q!e2rl#vXCNqq3Cq? zrVFX{op+5~B~RVbh9evkL>dB z9?y)HqQAUy6OPtzn1PFJo#z#pU0-2$EH@^Fg|(IR{xf6u=AR>WH~&%KCm>)}%s&xx zewYeB&6|t-H$mGO#G1RR!*K^tw<~=^d_g&P7|gO>C|NuV2-jkGKEC}Cg0`GT8peX3 z<65jIJ?&yJ%XwH<(HIY8ut#BDz_6u?#g>6dV;;c~y^P}I1?QWr&Zuk0f}Y}Vv}Vbx+dGZZ2_v2C;=u5ba_IY$4$*G>Rvt$QpxzZgtk1HnknPkh)crrSTr&j-Xv>5r2GX)26e8tS|d*nydn&L+TwZ!ja(iyI*!}{LdebVK@z9RJ5Q9F8(l~57X3UH zrsKfnk^yHm#W69XNGmP$iQ?o13$wj?RoQ`uGluKhw`nKCvw1DV3pTr#wagwG5W-k@ z3~^QP3#ohS*#i+aO9aP*G)2gZ(0a;ctV&zihrfj91xho* z8X#$x^&B%lyu&j_F8J*(BR8oaw35s%-dE#Y^RHTEh$^9kZP?yuk|cI+&;N#VcpZvs zN-||}V8+)!Q61CU3RP(*Qj050dJ-I$c~n=$De#o@{M4+N$g<7wH<2gT*b#8!&12=E zNra=}2t1bI3 z7u6JO5{C<-_(gUQZtgOd!V)#hHC+)W3lp4;QQ|c3NI|yGr?V8oOCWqR8{mcH%oC*q z4pciiwrAhxHfpuZu9}}*>b(_aLKhI4OHr2kN2Rvu)B_>svYO!5R8g^PZ0EooStlUn zH662#=D6&|%PPrFD)o)(-oA8NYHV=(3SIlfCAGv(3xj}EQuVt1CzQb1zj+Sp%OEt` zI3CP4C1)Wr@5wddl^;F|X~ZZ6DG;I+ow+z2K&a zR~e3!#~-q99P>`Ql<66r)EJju?BAa*a4+FVGrTFVC@oX>)AY0mx9QrQ((cSH(T38f zA$_^FT1#d-b2jC8wSRc}@DO5uFi{jfkKQc{v;!DV zdS;if(z9b)?8T#w7eWoube^ZpqzT6m_0;CO7!;)`%D|#Au?eWCxT7s<<2Twe2DDc$R~P z70e8fQkuN8Y`CEIUxyAL-s)YBSr3rCU=YFihUfdrl7q{w`5^K;uJIS^6}%u)@9@R8 z1L}MzB9UMt-wh&muwbydYK)s-bQUkLa=`Sy&ag)v$jc|5I)5mO-(#!o?Az8_8~DN1 z+9B&q-qFzYW*IQ7B!Z22H<$^k5;x{*+d`xXOb4x>CH*SiUrfsMRM~JL^&Si~qZh}@ zU!P^2y552G@O0yhKDDHU#LHhn_O0uz4FbXEsMif1Ew)#aj|rC$Y(=~6GUx)OXM0A2 z8=1hACy5~su!U)`3^_nTY4UnmIUqioq-eNmGV=1mIeDaQV0KdPsU*iW#nhz+=|k07 ziVhbA*XiV+?T0e4#ypb^{xhWnH05E!QMS#hq&#_uK;9&UUvwDjtM}ys)TCEYHoIRP zM^5H;TN}8@1$CYXn9WX5?-2D#9rk^8ol|LjYQ%;c>gPb-thtK4KF%k$ArMo zS7OF^-^|GsG`$XdfLwyndeV6GFsawtAmm^c=M1Inh`rpV;%hTbJ6w|K$?f<7V-&8H zOs;^nS*#oJxcy=r&_UY$2Hb(dmGSni1ZMO$(4^@)_O5yoX(tjXEy$Ih@82%6N9iDD zN(cSd0)ZbhF|HGUqkd&e^lS|*PD&{b6fmjHc)jn)t6aQTAOllvSo=Gz>m)4F>KOR>&XF7uI3fCLr1WlG`Nw*UZ$#^{$jItWB>D4 zf0ynK`E}IkaSvqg=?NE)*w-hp1@b|-lnZp{Qvqu3%e~GL8@*yg7OvOXWUTjqA;VPB z!U}Fv9WKI$M+F&-W7>uzE>%QXxe9g_+GX+v%d3XDdD2ekr3vyHng>bD8#=qdsvW!R zH>!S=!p(TOFCWHG@xUo&j4&S}8P=x;R4Z{C5y-x6J##caleHt(g0e-<)>* z(z`*Zxvx$xj8*U*a{RR*wiUzqBux*3*0LV@(b}&i*xKiM!hf}fCgxR@SQ)L?Ff(!z z{y$}Oia=StHV^Wm?W#EyQa{Lpe(?n2H@`f+Fpj`u1n4mKZBYCN)7#E%H%Ygx{+X*J z=P+spj*5D-^JIG5J58d~Lh3PW1ve^n2CcJqBQt`dlU?;;>WcfD{E_Ja(J~huD_f;= zdgVM(*F(GlwuZzd1xigxRs3>~gn#<19h?|2plePmwyY4v(-I14-rE}rL8oiQa0hMxTL|;MqATK{M zk6Ho{Pu43yN9Nsa&ar0rDG7_NGh`oSr*f8SZnz6>}$d5@C6KYn;)@k;_N z+2aOr;s@VcUm}o# zX|^!;2hw{lP$-k<+Z2Ve;E@pHqyp6I*Rnt8#`Oj`<-QLaa>HM+QUE=|o}T{ezq}Ca z!bc3yRQI?IM!kLy;MS*w?iYsz+CPtHo;@Thaqd4TO>RCej0F5N-A7~F4%6S0mGr@D z5Lf&QUL6qnjvhV zrCM>lP9$WqMz6Z0P<4nRThQ&Co?zcAcIdPX4>m97<|a3G62taUs|d|!Y4nPX&KWUQ zr^vL=;Q;FA#n`3TWw17=bSckM*$$=c zz@G^#it*yQQ~aw>6+RoU9joXdrfXjpkn?(DZnXV%UNyq*m^^(*aJI%f@4=Phk)b*NAl{m0CEVWGOj?=dN+V_flTzE_D zU~scvfm=D&LOI^+l{J1#ObSzSs4gzERI}#{mZKKrG1;8FV9}74xgKlYXpqgV zc5J;F=L$sM@qx9i1>!PX6a8=(>-Iv#w^7T-toKVafWX#F^e+s4{kH$Hl>Ari2 zMdw@QrHhHrl8Pfc3UhbFWakYCzX%ZYr{Ep_{$;OobeO_5&xP;J`dH^v=Any>M`Yaa zM!jZpq)OArApriPP%%ECdU^3e8j+m#1cgmc(tVNLps;D?ZUgLhf99mgnd#}6lb|V~8+VFO;OSh2dX$Gmrft_y5Nq;i=*`7w7bcE3&r;*N4n8SK#q3&;H^0i`~4O zrT(Fx3W8F71gs!QbV4X16Dnh5iJ_^{F0r3UT_Y08`EXs#h#lNlOP6BTpt>9*G=y7= z0sD6XP!+#XuW1XBGy1mJBUDS?HN3x&DrIKd@NCz)LhoF!F4MOM=(}u9V3g#<|5bxq zLqLhgiAm9AE7Dcp)woWG^Umwl*SYdpf{op#XfGJN>!z{<2$uG3VFAYkeR>TdZZ`+d zPul+)9fSls$sXd7*q0$u1>maxQ_DDH?%J0St`W0qCO??H);zFdY9YbBMwRviD;vQm zq4v^gfbLoYWc1hR>6^T&6Fx0a7OEXPx$0Oy;d5M@^<=(%(p6clE2GezbT)c+Jublc zOSMZ#a@lo7R#xzR!>t$cKjcQ?uLHO@@-f@z@98J|4l6ShK5w#|d^x$mQPNFmdcuRB?plEJp?(xk&Ew+Er*_@#E#lVn zYHWoxNj)c1h;5AK1ur-+EK8&4c1}&e0+9+k|M}W#iQi*?6zC+mDZu@c@167Cmn{y4 z_dAFU34>;(jm&F`DzlDW({b7n(A2%B?S8+BlHPL6A=qL6Rp$IN=WdAkX#=q?RBLuz z%d^*|3NWYM!~?O;7+(3b_6|7pnD46nf-xGD3P_ zZBv*uPUp!LdgO7^LwZzNQ<&YlY-DpoQXqkn<1z{3YBC#O8D4uGkI4`d{&+Kiz=PYdeQy zCJ38Z`@(R|=+YWJZc_{S#T0JxED6{;!ro0r83?d7K>|o=RL~XVAH5jK&-y6>5GhP$ zC@BB;bmzQv1HaE5B!~Xj_yW0`0>fwe`aO+&N+sbR9-v50Pj%s~o&QVkr-1832buhD zC)ug`2QryLd0XQ|?)pzqvhsfDY@zdiSGNDG%jS2}TMFQS5ISJ$@qpv!y#Q2f1vvZn zi#7h?Qe2*@Ayhr<VkR$zOZD)) zH&5n8MuzBkR8Qui`Bas6(zyVbv~&K~hi|~47gkS>SPRE~Uj68_?Qg4@qh!UEpVwcf z1O7F%58l1%u~r;-y}Z%sI`HlnNY1+Xns`8r7FjKuF8wD)S{yUeo7A*88}R`n@Zs}} zcd;J(zUGF*S(xF+{{W2X9gl-h4cjASaQa_LLYgPQ+_G_3BefW~H3-8Q9;;@#R9>Y! z(eQJSX!sBiMc>;F=F)0hpGKU>#BY-oH^t3XXGd(w&Dj`|Q83Tz=#VV7r1+VV)Fm%S}7f=9RTC0u3 zo97+lBOr4bNgiBeK(FqI%dlRT%5!&8drJg%E5L%H!=^B`UY*gK7mj2cIFSOUG#Tt<=}1R5&S8D_U~wHycP>PC<&T{`n78`>y*~8Ruqp} zA7CG3b;Fr<10PRw=HRd2+ro9LKw$R;86#(@d<;&};%YyI%+DAH2RBGm$^=F1lf%r> zCUBY_A(eAg@p5TscDy1dD?K{hb+Q7B{jkg`LX2+j7c|<7J)YBO&)+X{e||v3ydUrp zmWJ9zL3kz#lY}ybC}(KxCz&64(`r2 z{rclD-nQ*XR-l*)1G)KC-$ns$Fd;ho$v7s835h-w;Qm5dxSz~{x#FgbhEP= z0F_5%j03Gs8}3M4@Et(czI*tc_W+CeA+Gf$&~CnO-+B+YJBzPd!PWtfGE!nU8H^ia zlT$Xu8jZ1=Qc_|!8V#FbQ<68v8Vsigpg4_Qv`OMdZ_*UHTZ5_bzG20i3xMy>FTn1l z?eiO@@#jPSObhZ;#-8;$hhNC-r`y8V-EQ><#KHHjc*<&OK2Le;yAB{d36o*>x$N@= z);LQ4Gb7kn6??|>4u`Q!-1Pk%)$^<-SJJU89@H}ZzuyS47moTU*D@`K~!T@flOeKYLIPu-_SrfM$Yam(C>=O z$r};858?Hf;T<6>sdG%Y!X@8D=;vqVg}^?D9~{C<@J)|a=k^zP^Ma5B9;-p0)fa(i z%<*3UNPItm3hqtdl=n!7JJcTJZk={8tRySCEhSqr(NlMhU+oX@>ImKVy0#no+qC-g zTd(Td>WG^QH`D^ZOL!6(5bYFrRtZxJ0=R4=J;z(g@Zw1`0r3u>2QdcnA_^Gy8GuX~ zW-{Ez+Ml`6^4!*pc`|^7Ia zQIp%w8}{Wz_G3h0BtjFeR`Wlx9!MZZ`})bc#JM%m+hj3e5|;DAt|*lgeR#Y{_L|d`jM@vA8y-pE2(j-B|#SaN4NDp-|MB4b?N(e zz0EmtD(C5?h;&l1iqy)tVku+57@f|5W zltXVzsLw8HNZB3;Cu#_FfbiN!FOQg_dDGIm`~!m8=7d>efpD(bG{euY7~*dV;)?#p z;J*Nqonj@`fr?E3rlb^qQ&M_~e?o-FUVMb(@c#nV#him1dJpI?unGz;vL5SE}liApAZCG++oPm1voy`ep`^HZ%YIVIgC6tHdbP0FDA*Y|?jCTMB-CE7e4CX^-8dohd{ZWQTNN~Ex zBad|9cf)^PDs+#HF*1&hj;OPThCg;Miiu$q+#lYSl{Gwk*5T*N&(?J8gz`77kwM}fz^}?tvc5D> z%y(CY`2Wb2cy6cMlv&q?rKb88aRS}B`%*kE+wJx!fVQRk{iiK)vUC~4TKnz0RiXK^ z__9>J=x|T(Vn)5S`vLslM6Um(Dw(<;@Rxvj2#R1Yqr~~ zF@n=NlrOp_FSm9d<3FUKGu?KhP+uJ}mfN1Zy0ia@re}@2I{YERZ9Q^)(BXUXoZEl4 zo9vQ5SB9MCZKN*u3m)NbVX9)b?RDnsm_d^^BK&vc8ip;TQ%rR=9~V_*ws;}Dm~5gv zV0>~t6iN~kB6y|#X^QYF8zIB)L*QIhB$Thh6sz8*@!=lz=BkcIY9ztnvx9P5@Pf1R zT)_BdAcwP`O4$X3e5n1qRQ~tdvm`N^<+ZI;e%4t({M*}(;Z)k#@fz*lwFQ&^x~6QI z9dJ&Y`SMYz{2PK8E%83kP&)C%Rk3CEudGvw^Yw|UB0zAXLl} z+Dv{%C}IEiv4b&xu><61s<(pcsUTW3>2fs&DylEVM}>U+n9nP#m(}~2By2YYETaX6 zDe2-EcIw+#Lo5VG$Qy} zoM2alD5im-#-Yd8&Q5G$xu)=C)W9$m8(73|535Em;My8lmeVdx=#>yCpAI|j4Q?3< zY7r9~9ITNL=Dq_qgkFGmYXl=_B}NUEwI%Dofb9Ngtuc60YKQ94z{Z=3%D5s5;wxnz z<;Wy1gkv{hxA>H{7?5tB;Ebr)~jghDXd}PpY2a`EnpzH{@~bJx@Lqy4O-qrl{6n%75ZHZZq0GQU6@ zZP;0ms@+i#l2xb;&F)sGX@R=WE)%*g4-LAci%2e%nn%CexaMYhI)~09qvxqam!5#_ zoGkn9te7Wi+#HcWcw1W=bN7{2Ap8m9()z3VFj&^n5X52UyXW|1y9c>#eJ2}tDDG|y zj~xJ!|SU^{xZ1eR&6n zymFb)eTmoo<|-Wvtm)4K4&l0ogIoI72H@83z%|p84!?L@>^7yz-}kiTJ36j;wZ+lh zeTWm+JIK}pkxA=|SUhyy+r?*3pSNhb_CD^ryD2wCT<6|Td7s0i?+Skhr%#{Ex*`LA z4nr@1$CZ{QiL++#q9)Sc6)D?E_$5x%X=?_^$Mm2=eo(Gus}NQ3so-0<6uU13-nzS?onGfU~|DRSo~M5~K-F)7l_ zuBzP4QOUfb2#Lx2#y00axHvKpT~F29OA-*e`+R+*aK_0~csj_PG5ddpPSNsJV&n$AZNUldb5*S;x*+l1E9)=>rfCV5_zuay;v&qOL5M?h9-B& z(#$UR>^ms;%G0KQ`Jb#Sts@#PWe!6**joL)?VHajidEKa_D5p?UNhkMou^mEchIr#t)jFPgi}z8lGwB`61VFX2{t{-R?j)vB_s@s5 zum2^7ogV;-X+Ut|ra^a;habtX;krIN?EYfiXMgu3yIONIfatlG$fCxTbl%EFvWOtu zxWY?cX`B;M>LOYxhKR&rw*ok$&`uS zh+ZTIy|rbJbeg+icYIvH_?SFr zzr{`-QS#@l3EDB+3VtVSmEWyRIyVLUwSIJQlgq~cF>rx@d_rj5*1Es{gUTL<9My1N z$)#vhf8x56Hq4zUl#hlD#5Wuct;~t&%eHkuZ2-9A&;Qa|w@aGXs+`cS6pnjtJYVe@ zC25F;^&uV77`NP5G2+enzc+QQeT_$=v^7kli`axzDXwhkUrMXY-WHo*9x;-WNNr*_ zr?*E5H-Gag%54;|Hc4uNQ(^x`X@a+k7p0_Q!dtjCi&oKn?Q#^_} z{-k6ug&8F<2J=%HrEvk&WZz^4FS^vDdbqwc=}>Ruxx{kK?(iaQbWi$$iAMt*#7n&` zPc0a2s+r>EXZEnPSz#vehVIKi&_|OYW>jB-M)IPyzWij*O0j`GrY&ua5=ZNN`6-@N z8zkXGN&I>jj@Ugy=2n{?Mi6q=xw?rNY_VH)BYgM-xlN)($FC6E)7-ZjV#oBwEqII- znPuqqUR4_pAc*7FxneB5#v~eGgv6}|2uk`J3#AHN?_`%9JzrFX*xI|^R zE>u92HryM+-S-MT$l?1l3^7s1h5|OYC=YtTvtkgD9{bJ%-nIz%oW7DA(|tz&G51%m zA00kcDoKsrSuDY;#UWiCzmIvOdQ23th4%R8PUV#c$5Oom*mggyAzvrdkP>n`f>N^R zJ*q%M&__*>;ny`kkJ(NggrgB@eU$jC&kyz=s#GM?9VXm=IH!(hPN{@*<6@6oa2MG< z+fULMFrIy2aVR;1k4^NzyyviH(giLYd?F7U_F;Q>Antz0+?kh(VMI5UUEe*{h1cvz zrvA9c|EWwT*kcd89m&*BJM<%Up!+*S1v3oYFKIdW4ol-KZ=F^xX+ z$nq6xskbCVW$WT7ZX6? zCaIg4#}i)P;fV0tW(VC?YQJQ5d=Mb;yXDx&)hsZ6AAE~?VEU2`9Yb!{EIVS?pT8N+ zWMdggZ}@ZkP5Wl$@#{`Ds!`)s+n5hK^pyB5xaH*kgh-rd%vzkhp__4C=k#(m@64#< z1XNc%mxKH}lIY(!2?B{9404Nzx=X$-04A`TmAQ@#TY8)sdMe9@RS@NS_Dp=#L_{ zlsbm>sp6F8Mn@3eBtCiRt?MWYi7m-3cmJ2O#E8VO*UNLIYZkjc<#t7IZ@IgF+31#^ z#D?XW{8+-vHbYK{o(;9``RS;tvK4QG=;L=w2a%& zX^zhSIX&+}IUhC;Jp5;$lH+$Um+ilNz?I_o;rZDB9#BdLEjvSd>9V`uDC6n~v>(?- zB!hc9U;5HlrKbhVH)XNwi;`=Ge-2iSk%xj0zx#enr@Q|gK%>y!ug4jR+m3}7sa6yDu&}>#52ARf~!nBHH!EOWQm?;SryLXZ##>pmW67HJ&Oi<7fGbJ zt6ku*b7`C=ob@MHyXZU#YH7HHLjYOFy$)PXk%eqNI`sZ{vJNdp3qc&tNASrsI9W1h zi{Xsk9FNyj-7psbO7J@7vVe`>JB*KN9wZ1uxu+_@BV>(3}jPWqMWxcQA{~ zPTYIpzab6}EXY9-ZX5mvfatCA8~_LSyJ>P*XdQWp{Yo@*(KUwbt6iokJxjqA=b<_K zjt}4J#^F;Pt zVd9?GE^cFbO!op=WFQLcD)Zn{fg<6B3?^Rui-ea6dQ2}o$|3_%vh+)f!2a&mA^tI1 z=GU}?O56 z00sHCfxG|a?0JnD^|CM*>%}24_4;ap`()%+~bwEciY`;UH;3Zf} zvrgwOB7h&zcQCh0?msEdHuVn)ExA%V96F2SSwp}kfMJL}DgpHw+EoDCcxOup$ZsY6 zMFBKp9SK`QxmgY0?;b@A$UMBRB=!3=c@5G1E;i(|LnTWXG z)@LYSHf})8O(z0s33iz~-kbU}CyB9P!Av3uAa!eKWF?V+cvi|MjviSAka0Xk$Q`rF z(E1D=$`nSTxzB>35Iid+$JTJ>N^!q)PpkeBSduWmH}yjh!0gKb6>DJy09l?DVSy|y z1=o-8=;>QElhE=S!}H`FnvHTH3zE%Nk(ybdu%(*-lD z@NF{S`$H;!Dma)QHuG<_2>Nonx3uD_Hl70iJ-efq9z}_An(;v!P!DG6o96qb34S z7KjHx06?&tzahe35E8=wncz^D0Pw?8MV%Y?v#6loZL3=3GZ84j04M-J0Ps8Axd6-i z13dFkm-yU$7lz`P%^7oRr(&m9*Qfi|I6R_M(=*U0J&TU%e90q`;{v}Q=ACSDW4#uZ`4e}7$tFe zPeNT;1C@>CEppRsiMy?%gi)5AauTtYj`kQi*J^*{ut#dm3#wYX%d}lnJj&_T7lW`y z+v3J~Vqe$h{RU$nNqgi$LQ?$)~6U_R~!fM#6XMr8|nh7i%C-&8{pco|m(s`UX53wY7wKBEL%Vqsq z`{>pidi5|IW1DaCI+b!acYN{M58tczyTcWMy?85m_c8aG><^}D=8qP`Wi}u4yK$`) ztg8+TP~z_fQ59` z>qbP61dZcz3RSYD*gK0$t?7JULJ&9b(GJR6OF4^J!KS;*O1D4Oy1{=VTkcYbAUCvx z-X&^yu#ECX;StgveugtgK)CWeVv9}!K)fSb54nHGgqDa$_fGJ;CzKHPQ2?x` zTofYjCZ0^Vh`Ostywe2SS&@@eg!TnIx@hzr7I>FYA?~wqz+)}|ao^FL8U6p!=YRi& z80tRE_5@7P_Z-X(9#w&*sGpeoihz@F79O1IeSg8D3tmgsGQL;ttbh_~;1Na|*P*+N zMsS{C!SNMx;BnTGah(5M7m}y(Ad886kOuBU+EOj;Pu>QnriiY3BEE*z(HjttS5JHd z_Rs!(0tGPhbS)mZh|IG>1rcaOMztY{+!qga?00MbxNgGfg}?}k;iQo<1z{5F{V=aon?{~I#%FwB30%skBF99vme^LBNjJ@mvD$<)oawORhzVkEQLDABu$T5$2=`)NuMd%|9+~ys%L$5+cP2y zB^AYRf~2C!1(0DmUJxZ&Q8nE#Edm)^t}wL%C_q4V080g9~UONrwjB28v4k z4vJrdhoUe3(3K#BE|m!@uibD|an!4leON;9={kXTLTJ5LlX|OXdH17i;Rqll#<*bT3oi0Q)-(C z?w|!jXA!^v4dPbp*SOUYClFZUVD4sG%M;~0qc_&&ionm$XNrQ}fx!r>782hy zu&n{E$P?cRw6!nqRB~$GUf@4CvArLkh#*y6lT}t%R#RP?o?V<-nB!vZh*36>08?Yl zFOI!3c!)`>-$s<$;e%8s0AHsHCW*t5GZ2j`aTinFAdaHAKv`{AaMgCl0$4cG{ek}Z z{-r|BbPwxoQb1-<1DsOTPIFmS2rqUi4p!8F;Se(ZR$8nWz8kVU4=6%$KWmAQ-k6Lf~8uUx`N znqC~|5Eb3ZlGH?O9$Bm6J&frBqC8`|Vdm-3m$_zpzUzM5rd@TL0UMn7f?qUX4j`*l zr*h6IS&eoa@I~Hh|HBppI(U`SaSo`AI=y(aGS=WQYb2wej$#y;bl12+)m?`iGax*8i3E zX^5b8-0_w$H$xypV6#3)&P6@L>HA9H!)74_WKOf@E$ZJ4zZPF&vP zc5-;~EYwXw@SBcIkQmh+W%%$Y90FWIx^$R?2gS@31Y!u*a8(YcOg)P^w{gP9j&S@> ztVqYLzDqw`CE%@0^VxqdnoFVr(K~EcD1UE{7pibg3FN_pX^X!$LH_rMRb+PQ<(TR< zOdca}F<;O9vitDmG+Bg_?3I^PUM`(TxA~Wb=<_v#_fyz_UbP zr01|}8#sKU*b?OJIFjg`$1WzpDfQ#h4(6+j(T6~CGN0hDw;Zqp)OOw$0bJ;Hchl9CZ7;oE$|XMQhiy>=W1%_E z(&&X>#p3TeGiT?*t4qBT&C`ILpATQ0fI)8r;BX3Mm!H2r`S?HGNiM+0qC$gvq=QUM zT1#9_UT>EK+h^0-Mp9BkajWOtt?Gc{Eq>tOk9S>N@t(G`d{l}m3aP8J`YA8Bk1xY7ocmvYT&$rU`PKDL-$hBZ6U{Ea$ftf|#BGk1(lx%CZHn~1!eZ=LVXdrikax6I z5!g~#rOp0mOQib~4aDK)-7BsfALG1{NmqG#&*J#)B%;AOqz{2=1`@F8&@mzVkdP|! z=2hkVt-6-loR`(@3v_w01CcEjHEXBvT&p|}ru16!Bk*fIq*vPAHP0@$%Q!eb^5q~|nI+gqJN$c)xfF`jAZBo=*{eF6^O%#|;S~&#T3?B`iWQ zwC+Y9`wqKr9+RG+`z`%0i{u#Vz*NZ+2|HAfu-gSE=1Ya;fej}r=oM=2aZ~W)Zx=l8 z`7K>aC0ZLzKt>G*k&~I_=WTp9+~p_QN+UrptI91c)RX*#%6>Z_t0(X*9Y~dLt5kGq z*}pj4A2miL%GJ0+%4HE(6rANh8r00rZ!`OqecQQ--sPvv-X5l#Tzw9YRermmX>-V4W1Q3KTNQ*f8Qf6t>2oZ zCfSXxKP7SeX#S|3J;hr#C9++XOfnYVnAk3CZRjXgp#8Cd3dw!FGx>lGdbnKf>>(0W1d&DM$NU`8)B z_Z+b;LE*(DC-#@SUjxXLgR*8vf9~b61L<%-^ z>{e%9-TV5M+~cB0SNBwV!mH|gbgeUx&Hke1N_;OVfc%H+Znv^*Be8hC$tB?E&hz4K zLCFcQc@QhwEze$3N_#cm#M_{S^rYKTRf3Vut%A?iKBjZE6Oc=(fuXIzRkr3VIe*S9 zO9TI&8W+?nD_8q?}qG0oDbD;Q>AS zK8m%m!}&q{V`?a&1}?w1Z;#UU4sJ1@_~H1cJ^bB-G#H?*weaQRm&x{HVf0r@8cdoZ zjpx&j0iWAQ`xVFiY#Lx7qfa~0z|^6=gtGPs6dX;b&bD`0nU9I+y-PXTO%mm~$|ERC zu1gy5aPOGOD_h5nTk4dp;B0TcYFpH(uBPK7ov|QX#of~ z)IDk)M|3tfFRsF?ecwrTdhh?xnc5WZ{}Snhy@{rVjR242U&J62e0#GoQA$1YNA!zC zy6oW3nbZa;>9j&ymG5O6^rWAskBPzel#`PKZva$90RS*78ZFUomAT{gw=PwKEat)}p44akPBfN8j67?L?INM8q7S{AYY)!NLSrpn>l!;pxJp-n9 zvq&7iS4?Nfn+j&CJR64+RIMqEp?6^{-*HVM|Eu8ChkAWTaM3e0KUAnSGCWG(1J8$S z94zy(5ce&iNIV4HCk!a{NilC~2n<%4-+4#*WW)|J)glcJGjpo$2&`_7`zodB2gJ(I zU?xy;9f;wY*N!atW}iy69_XY8yVn|x_V+eDMlYXrds`W{T@WDwj%mzQBmlsYGGtUZ zEytXFVLiWqFRT@tJ5@Vsy`9y8CsCwJnDzk$h@e(ASwWny=<84YDIG_TS$&XF_$jTC zgRUu*_CLb&u%Vibx*dU4QBi|1JZuzsGPU^IHBf5QNF>XU`~2LixiFGSBD?&(R;?I53h zKs5%%q_OWA!9l4I2@w+b7%VySF*oJCJ9J@KSn4<7GG5vH+sR&nqLW{6wSi&T-v#sU z9C}5>H|AY#ewBra_EU3TA<0^bs0>GKE5OrB+58sgXi;SR4v#__9^CdJz+%{upi4Nk z+@%Ex0guuxsK>@GkFc%nd>xlY?B|yX``pL03L!;YavSm9AXX+yINz^y^9>~ncq=AL zy_3kp#={2p-t3P0rkDzE@F0>>peQP+z} z7K1q-Mj^$M!w@L|lk+WdXI2@87k$nGFR+uQN+cmPboFVkYJM$ZAjS?+7A?$)&2t~Zi67z2F zES6ct(wy$QU1ImN)SCL&F_E1WsLBB2#(w$CiugS}tveVM^GXYrPg|%07e|dM1R^w6 zOlU-SM+cI^G;tp54+9J$HF@9lv4U(DoCTA)L)K?Egn#1s|FVwcsqlvw_qr@&#klEo zidZDW&jFeb#}&;NwH2~fDl&u7s_qfZ+($anCcT>S$A=4LuQ?uZ$`67oF2-30nsjWh zV|{hTyy+#nWqmaIPnC6HZS@YyS1t7lvC&3f1o~1(;4zQJlkXtyS*P_6gAc@uM|Tvk z4~WMHJO9KgqmX(Hm1;C^tj6F}uC`q*M7$X5cT4wZ<^TZ9STtxPLZAi)2|r0c0Z9V3 zX&mI7CCKndw2+1cnU{0>L%>X%uVnJ~&?Q7|qP^Go9(16rC?xHM`g7vPgDs+5K+B|; zL6`UA$eGu5w>B`enfS-tH;$uEi$gj`1+u~M3&Z{_t<*i+SwW!jp=WOFTq`~k%VC6$ zSNgksleHz}e~2`9S)~e`=)P&M^0s~Y16vEH&ta0l4UyJiSifSD9o8M@aX$NHVAR*H z8?*|ee`B?Q^CzNY#{^T_wJhrSBW5`PZnAh>tV-Cx=bh0zBX0|l-?-WHRQYm5^I`5L z5KguVRmdIQQ@o4qIo!3hiS1G;v1pTY((f~-QcnT7*;{Y{aEHue4 zi*Gus06Z-(7;#Ft+g7gA7P>;+BDx5M?>FxX6=?f&kt;8&;MhIZ2>>F@|1uAI>tg@F zHOxTi3u3iTt%cL!DBT$#IDjUTjsyTmQKEH++L0`7K1!R$1`32VUaZT(B+eQT< zla7uQgEfPrGad>p(3SN;-;Bs>@>Utv@3^mC5M+muJjLc4!L0gI1#d`$&_u#Kn%T5+ zMM7CX`|#is;9?2)n{-hcUdVq=ksM1ae&)T zYY{ovtc@BAr8O*m7U_EsXL$ve8U$V4JuN#U6(XJ6#x!}k0$>&3{npK+I@7VQaME0px}S|l_|iirUL=;-JGfbT>t|95&tJ@}3BhpwU? zVI)XUZdg^iklO7uUMbOjmz~l06|3gzSsMz__=VBL%+AEdBD+8LjNTrgOK`rlzS!Cv zTAadVLckr<3r<6D#KPByb8t_v0Hi_30h>|gHX)#OkHMw@0PrQu_5h1Ll<|e+!|1QY zZ~32|zf_xKOupbZ7{Y*)=Jm1LPDFX=a1jebr~5}hw-q5iqC^x90wF*sun?gP5x9%)oqBN3htq_rXxBklQ795G|8EY?LDEG#Te1L%^3 z7nG4Z#=@A*-e0(*M9Y zww$!#&fkR97y3O%5^p00jcYZZVwE}SS=0M7HSQsE4;$|cM!!JboJ0Q)8MXUkEkV8< zC}+FI!3d{RNdsl;IyL<22pQ78mzSFdJ&&udv*}tOPThX5r@BwiwKc{Ujr6%B@r8tl z&K!}r(P0H*0)mq->o=`OKYqxXfq)C%p13>fcCQmQ8biD`;KpaTdpb_|S2AI=uP#de z;69Mu;{QhhonZ*d!S13us1FtYrj4=BwE>T)dUf@Ios+Vjgd-}^Fs;Pq(9vPg`FW)* z%iP=dlhAI=39F`wrGcM*_zLaV5;DEt3d)IU6R9o^<|%;efG~7h z0*+{dwH9{Jg0bHuEJVRWffe){JK7SH?{$V|1X13=j{>{Z#XZ@IMk^!JZ$Upawzk~IM`SziOES3Geaw$>eZV?)P_vowKP|j zd74~wab_kjLs(exxKbAr(iAmu=``q$+P+;b)nL~y(0(E^wP*;%>-x0A28HMgv4F@p zuSb#MK&dR=DBotsscy4kFO;Ufr-?;eavg;J(OPQW1lh{YOZ246dhxNks3xCyhghB? zPKp9!>+*9&lE91GRNyzzQMe?86LOiF<+Zd*Ly2)$)7lh_+KT?3)rQI>W|8$`go7dGHM((=$@ z?MB$^$j1B{T*K-Ig~6kf67F$FI};Ydj`#HV;o&lW=QfFC`_6Ojrpm>p%Jn8PAOG4C zs#o!X?*+>e+*~9O03}q=7NsFVxV#J`bQmct#YSO0=y5{xha+tp1t)cmf_PqT&@saxjXOB2z74=M5Y&eiH-&6s$jo z*X|*z()*EtVWOa(CDb@E*CSd9#N!tFX4%!Ifk4W*BWP<`leKT9lmkvJ93 zKt>fl{?Fr41{O7xuGwPDnM-h1V_N|}h3w1CGB94<)%ElhMt~@WHaf8OH@}h=R;m2W z>l+%}TyL(CkD%rok{b#F?feHfF1)y4uFEwYW{y$)GkD-q4 zs_JYMt0AZTdi{;=dxYujobiQu;g$MwkZF3+&l?EUd zBAA4bla+enaC~Bg-L@d3tF5>(hG~$yYn)jfM>R$ z5ETD_0W<(OkcCE>7t%*ouv$p#rWaqRX$a_RZAu(il7?xJIt5QgrcH9C4XH_fB zbb;W-f~YgO*b?E+5FDH2#wJ}IA{GJ1GV0rEo$G7rVE#c^7gdi2BOBA_?}g!G>CyH@ z-Y(cy3bSk(6Rx#|^9%>tJjAgJ+ImfL3RSN9UFAe$dP*4$%Ib#?WO{w48b2LiE{}eV z0V5$Uhd@Ib2x`g#epvXJ(=)GdjG%f=Q@cgKJslH$fHRkNya6%$x zQB%WGVYk4p9ovDqntOCqtFh? z8;2%2`Xe38hPd{Vg3L?lmE>M=6jC{wAspSel&w1%WeY?O!|S%)DbJ4eM~`KOJ5~oxwCG$H?%TWt2_w(HI0mQ-lwMT^RJ@KbV3m2*9#HX)_O*+^d-Gx#;C&X~6KI;T#NQjM za)%OZHN33F<$N1+E9KQy-S5q*mlo!-8!TBzVTi8BwRdlIBq8{32{c z^ms(Z6CRNSR$uXfz1frKtzJX)ZP~wIImYVFu4rxxw~7bq>Ev=~lb6*Zk1AK2ghu z7)|FOTemKW!zfQqPC()D&;%)7c$&BI=8A_9-HMHF^%D~3WD*F+#c(UPv0R*OZF0lD z5zjWEmyj>kcFgI9 z(YjyGFU?Lp2LkpBqS}K`-;aVI$L{K3I2fyH4c9#jK#LDF5(sAM+s67AbVMOLQ7x|| zBGMD^f$Hx{yRAI)2J)U6WC?+%pk995?UUyf%uzQdzE>@l2AY4Twos_c_?F4z{5-ka zyfsDSm+f&bQSCL>E%{v~M*lJSEY51J0{hRNmH$cS zB#cu|Q@Hm$K$02sBFFN38D}$*O8KgHMJ(MbzSVhCI^B0uvc@)tV=ZHGaxhp?y@o>* zQIb+(B{r7UR*Kr}cxISI6C~iEZie6%&&fP~4{=QA`j03y)4TMN?K!Tk=kvMu-Om?G zg1|c*7y47;Ed5QBW#?bXSUt~@P=F(ssR6PPTGFk(#4)pdHVxLbt`M+3frTXCXows; znS|%^a&ck7;%gc2o<{_b=z=HN{}pbOb3#UGC3D*J#9He+8F(1bQb)l=`DSS3lVWhwb6oRO zUA{XmQbTDj$&MiI^tzpJo4j8Vx(xI?opaaTQm?q#3)p_7H2yU!wbyGGyv|mFe*s=L zv+#Jo^W!O82AR%s1y#jyD`7gpqA(~6^aJHiiJ&jP+p+dl=sJg`Bb(YE;i1Qw0Tb3V zz*|qe-X6QiJh!^cCM$dj{W3}AEZ#xE)H{ic7q!xPcI{a&i>RvQHUcdqR77Ds;V7Sw zrF_l&?37E7eo!P4k!vJRX;ds+p%h;B#J3NPLOKG#5&*4K^%4lHaFGcuHB3_)aor08 zAAZaDiCDG*vw*y$g6-#B*bN5#qU~~>i6Bbq`W6h^*E|~V=jCV21m;x7`6$8Jnq;Z| z7k{NR4dP==GIoN$aaqY{NgDkMbKaLWnEAzS0$YQ7mt?o}XXvQT;H zHEmdA3rvDXmM0+ai(Khzx0H$SXXlaKW4H!WD#bb~gfJ6$RyVPpBilS4XSa)0T+Ku5=iEa7 z^p7Y1qyht)fhmyMCqi?zb)a#Y&GIxQ&T`eqmyL4 z$LMXMvIo}=6G8C2IA_;4DZqvD3ec-qfp5_bx<$JL7^?n{3}j_H1$-_XpZZ{im}fV` zxGyN_dz8z$JU?UaO77B@(s%&poW61l6m$kZTMk$Wz_xP(wiOl~@a}7LV5d`1wME!N zR!HH9Ta<<%<%w8ho*rI;J*_snp0w!8BqaDvH+ zcXX)LAku{SRSl4EYtCqZ7T=BR*!(G!nr=7QWb~wf zw*o1Xu8g`@G}AnL_-W*^h>zw0JG)~|DacYm^1P|8i&#j2@l+1{^5$uSCDee}%cm6f zHo&?QAA2OTe~VVtBiWy*u^XS)WIVjlwtZuFX8Ma80B~2t1Uc%ym&!WeONn$31fFpB zw`_Ap$TkE((vLmgb@_F($_XR;SpkOm z`+qbDq9Px7L73n@+T@0+$Y@Mzb#Q>%ig^_Rb3^*{i#UFT*kE$z$@2Y(+3I@|-a z9j9Y2bd(aVIerkioelvX_QwR8emU%69Rq=!29)9|RE^sCZ|x!F6U2G}u0~zU;~6(7 zVUey#PD{OTiCRgp{60{ydxp#sX?!dGmKx@AYif-qD)m0DY;MQdrdq1vktrp`q+bgO zSIN{qNPi|nw!o|_|KL?Ta9$$YzFjAamysI+L8pFZ8($~xdZ6NqLd+9cTJR7sm)A4irIk{u>FQ_vctO zwbt!B^)upVhLtp|C3w}C2r;Dw30Ryt-Zc(2)krhLdKeY%w!8}>uJ=SBmbuwWz}DPL z_}FI?yH8Xub(As(X76|&SmXBLoAfH%<1i5Lyz8oVS;Om{W@J`iH$07*O~NI3VTX50 zYgpbP_~KT6BFPC{jV(4MM$aHFRD=_6;_=mV*6}L^FrOt-XWQ3QspLXd)sfHp6<7Yk zwADo$L!$NEt)s#x@$m9?D>LRpqPT{zU3F~inyUE%t8I$=t@c9!9?yN!EVczPO-Op z5#Fv2&HP2Cz1Jux%eLh00`r?}_le&v=SXJE4DBB$*%->dLJudgqW#9S{G`tqL_0_x zfy_#6WsPs9Y}74!(zf0$r_H|5ebYmjtn-NWJ$XB}pZj|23{8WGFerx<{u#r*jEmW% zCZQk>ne07mqAD5(W9VKage6=fjxVViY%au?>S|{ZB-{W+!NU$Y8RUWicEd>v5rnWK9NQ}4;$7tG_3#5ZV z8oP)JISUOzpqD5#b5&L3R>rrh#U+Z*ZYu9yS~40mmFY>)kJEsej-)ldTo6`P?@m_sCqI?ZaQ;CV1uas*IJG{4f;*)-8M;wa9N$?` z>2XDF1Qe6gn(KV?j`a%siQ`dKV}CzEuLNr|o0F z%x~faz)}bNa$NS{OF~7G4I7&5R1#Bx-qe7I?K(7-&|1DyD18-(+f;TP2DrwOnGw8vRleNZT4V zrhcj-12k$<`-|rIMzHVr-6%#U|Rn~g8CSyGZxk1s^RzY^vUFdbymY~AM`_| z6xNnHL`s|nf1j}1Z#M{bh^p>^jS#cNa$^!zN1n{V&ANjZN`iBg!kX zpjDpDlu2tnLRSO+Sh#HG%BfZ&uv@xBon)HumbC7r9(He)s?JyT%4^J}QU=SyCDovc z%xgtR>CsFLrOGs<$Z$=baEGv{p?&&x|70_J7D&YjgqUttNOi{e=R!Q=kJAX zn8C!Z-vZAP0UcBN$4&gi?l5DR1^aZoi`@-PEwoJ|pvZi8gC>>H4aSMPQ2bInO#IlNQXAjd#g-xjHXh(~kkqeqF<4VJ zV3Tm=WWGy=l6dv8?)H$nJPoG{<<&zHRAe>91h#RRL?NVUO-#kPI{ry za_P;lw}~aT*A3dyI(3CQO?yQ2mS>`&zx{4Pc!t(&BPwB&8 z^Bl-^kKcb4pZ;N!=q4wm6pmxjv^+mgA>(Lhx3+iUh}9>@;C5h6V71!xr;f;6aAof; zAq6C?0nN$cQC0Vk@970e7XtLiXB^$DgC1|`TW6w=T^7!m0bGgY+@#N0&mNm@cLICr z8gQs1A4~sK*ox2|9;vvU`AD9nTm|vF;aAjMj_fX=E@p5)#I^>`3NKx(epc3~E&C%T zZ>Q%YYSJTM@MDz!9U_DN6>lbt-vg+dPXMKXC-tHrZZXVV|L4g2anw_n{Vq)gG>!NVpP*Y*$ae*I!R zpWtgzg)8@U5SzQb>2naV1+VeJqes<^PUoY3#97Q35%A}mnW6%JzMenN5im8vy~sBe zlp!58(<8=aF6zSrtA$H52OK*&m=q?o8hL@O1+M7N27 zX+mxAfZe&0dG>d8`+`9;O`+iFEP+FB!rE@)iTZmx_n-Q`Yqv^#=a_%QYM}1i;CQ`T zxZgIm@F@{|7$ob@nx<9&cJjiA3oo>_9uDgI2i&NA?t@2r!zD6=Y@9@yKn5XgUtzY5 z)tRMJ{qXfOFvs)Or0lbIDLwte@3*BXT!PZ9RdRRh*hPK%l~Urrq%9?*Q=6DyqNVsiR&zsZ{}4@2 WZ$y@6puWu(W|-GhiY^Fm1^hoW?zidy literal 0 HcmV?d00001 diff --git a/docs/static/fonts/LatoLatin-Light.woff b/docs/static/fonts/LatoLatin-Light.woff new file mode 100644 index 0000000000000000000000000000000000000000..e7d4278cce83e38c533b9d4ea283b4ae1bba7631 GIT binary patch literal 72604 zcmb@t1yo$kp6=Z^!8K?gxVr^+*T$X3HE7V_1cwmZ-QC?ixVr{-Z!E|q?|WwEocZqD zJL|i{dg`~I{Zv&idNsXw^}iZdMF|N21ONblAV>qCza1fg%zvG~+5h|_sidg#HqaLU z;64Qati4~1#B?N8)Ii@_jUfEK6AdT;m9GGgy+ z|EoGmJyO`+GvCDTiQgUqooP2|Ubl7gcF*$)H9$O>Vs_{7lyVFYJ@+++lA$$~fsM-L zHD2#~YUlN5S9#prmQv_p>hgH4+o(H<^g9RV^}*6_z}~?qp!`mKU|U1%LM^q?yu((h zmImSem$|ZMc|F3!yQUi1%sc@c{s0<|9jI*#F!Ow9P*7AzW8mg3i~{EX-f<)f!(9HP|M!wb*Yt~!03*Tr5^^J zwbYilOeaR0dCbkmx6;DW$eJGr6LUWpK9~D4rL&+ zt!@!-+U?Y-=bEx2AA~BWL?~#~Tbp-yY$Oycu2pN(uf!0_dC{GlafvUqRwWq!4Cb8E zuT(d;Xgtc&C^2l2;?MI@!gP*LsTi3^&!SL6u zHGNoxID_u3Uul9{MLdHdyOA*_$AuA?6~v|RXf%XN?S5aS)`|pY@9(Hr%&JPrKR!d^ z!7YkCS*rD@$Dz|%f&d;V(fk^(cdn8OCRD5FwW2DTtW_OT(S2jVc2cAiIvXp&&poTMk(4_f**yE(zXr%7|5{KZ zOr%&p*Xs621@_Y~R6ojE{+Whl%*5-oLC)557U%ANdWu|n$XxE)N7=N$ilhqLO`b>CoJ){-&{082{;g3r`#X+b3 z!K5hWy&sR-6f><@Ki;MIn{BdBLZ)St9od*25>BFjnHDT{!s~U+zdM2zK&*u}A_ZU3Q( zXcb@!&E=QcZU4dV7dkG{8;r|dhCzI=+bjbCaNW=sXi4ythn0~-AcQ1@0>9vHvQ6Lx zZW4-_Z}O(Y;{$XOjM*{&fRCS#FIBfn#JS}K5;Olf_xMBrH;AzdGZKfp6Ivg((pRI~ zdeiv=7To!69rC)%^MX?ounuvqyvKhbpeeN>2ng>oy2!pIAoimG(cI(GLydG(ZK_m3 zdcq38(RI^az)(YrgE%w;kmvoKLHvsVPJj6<+_-~Yv7i4gGO?e3Bthe{4m&Kw-4XjS z0x!qf$~|c`UdHq0BmD+cYwze>_i}6Ts-B-l1>AbZJKmr|%<0to}(t?+71G*h3mz_0y5QhA^C->uf>6m>n1Dfa~(37#}=3sz_e| z+P_(Q9_T{RztIA#u&`NJfV0qbi2q3ss?d7#3$nM)-^a8HzZsHGAN>D9Ey*VW|5}nJ z>>wS(|AhN%x~B*1mnQ@t5fcKRJ?W9;T}7eM+zumkuf4J`$^0wTxWXJh>|Gih#d1G2 zon``4B$7wMpj7v^_n>a03}dkpwhW8?VLgjQEcO$_8!H4Jc9>JV;7YN}XvFFVwwp}> ztVe9DM;H%gL{tV9-ki<=%9fQ&zDf5(FfzUo6v3!nR~ydOv%J!aE*(N1I*fKtkXhfz7bDa64y@taM2dVT za(sk+g!svRS9%sNdLd2jFa2a7 z-@LwwZ_m(#qKj$^;POxP{C`x#&u;=P5 z0nXD8u1q!aXmA$cIp;CrdOuBLdY`lk_URYi@5_j1w;*BIst_a)5ndN*n+c;nYvNy? zkF@)Pm_OeCRwnSatk7I|{;5da@AT07;A99@cQM~R_FAIF)ZfzJdz3$7Z}%WFpQ+v0 zYXh`Ft>;G^9-cet7ui*S@B7Z#q?BltCK!iYP&= z2XHKc7&PJL1CT*vi_l@hp&1IW%;`uV9=xtEtiHu20F@y?0Y-OA>Is>@GnGG>;(dQk z1Y*=$Fx*+MQ$q?JoS7+c^#P6%P^dnj)S4PEZ-@R#N99pmrn2Bh=;?&n`~7Bdp(=@k zb*_wZe;Lt9nx4HUTzw*Dr;9&Ae5jYX(TwGy6Fz6!_h(AuM#-uXtdpsf1T7r2>`DrBTywE~Gth3xYXr z>}mLUKU-`849NrWSp&4tI#t7o6Wlm);YL)p7=a1a~G z2@LGy7MNIBN-)gvrT5Ix?9CkMJ~qS%z_>Z`MQ_PxbRdoijLr~GRjwQ^P(GUOeTY?L zbqc(3Vkx)D#||FV6)oFT?)_TH6Gnl^t`iB zD>mVTD#pn6iM((%4CS%m}q*;f|GzZ9!f# zoo#LfWX_t1HhrHY=am?X>&fbp+?h_dHVsOYlqun%Ra4k@J?7&IcB?KlBkWoiMHX)p zmM$z~AsoMu;v}LjM8(ddT3Y4aDuFIk1cV*u|# zZ#oEo8{&fu8PJVpkApi5HUxdbMSi=_yW6%5tFu+@$XzrI18b>yb)1#XOWEc`=ENC* z`$hRjB4Cz%(=?4@wu?c-`|bLBW2UF-;N`M$4bup3-Xf*gh);7%KVh52Mm+3kqMSs> zBZZ77q4~z^sXk3WDi%9kCOAc!`*PP#ICv9{LO4ONeQuNTV)9EAe)nSg_v&;P7iANA zou8)5Gjqiq4y|>5ZTSV zm6M(yN4<^B1cR3*j60?y9Iuu-bwqolt)yb`m5(mmV8~XDS=LPwQh{eN6G<4wLuG-k zd$A=fPs3fEY!~N9Prn^KxuH-;yAs(hc9A@}Au>n0_E{qw>NryIGYrWmN?2}ot*m$g zI&_5$j)2=x_sXyxaehpuxWxA5gyj|rbeDA)if?elr+c?K!k%!Pymmpi67 zWI=;s5>R+>K2t)*ked0EBWstyvYD4bP;3OsvYCWoaK@33VUXZFT7^C*akm77E!4BhrJ> znA2u7C~~Qag@DugkCi{UKss_QLq@QmTBz}hOs-AUFPufT##)8979rG{HA5%?QLDrgj?@fj##9DAyen+O9zWS(bD>m}3E=_3UowLp4 zm0*pq;s|U&QT^CEP7A4GNE!KpS%Az~_AE$7Pt!VpG6C6<#yQ(GpgXR`PZ@luM%gZd zMnK-(BoYc=J$JWokr&20A!?GB^8TpbZj%&h*%&$NO~oqSHvnI2+s4cu~wt&KV5IuV`&ze z{^rlx-tRwF_8^}5KE!(JFV*DweAju)ma87H#U^e%XxcRQ3+JEXk`hlUi+H{GX$_E< zz{KeZK`DWxJFwSTtc10u;Ln zK`{47o+#@6YMY2k@L^q6N{}O`Aht!whONp!7mXxqHZDY+b$vMyvxfNZzvrsE{k4Rm`K z_NI(Ib`$;$?1Ha#xAUflJya9=Dzq)Emym%yViWew-)rLgI{*>^)sc5dy$6Ri*lPZ0 zu57MUt5l;_z0#~)e@weQxGS>Js-r&$jil?lhQFLP^{L#f2C8{KZM=i^^|NUbxIQ7LlGsUqz#V%5QSIv|FUm9lM1{h>+z zX=kv7*PCC6+9GrBz=vL59tN4~3yrJQXHl!p4z7|PBvgm@uNg-)T*4TW zc3m~=y@*C8lp>69EgVPb&vDd3;y2_ao+>=|S`8RX85)tWjx?HE6I(aBN^uO&N}2D} zGW5FO*D1my>KhkED~9F~z{gL!d2In#W>JFTg#u84%=Pft+ECxW-#f;aah3c>k)dvi zOigF0S-DxO@q^=3oYyV!jaCyU1?Td-{>7NmsBB^u@ zPhW+Jc#;=rSa-q4aMKHUTqQ*D{(g&zezOl!;lBY17Jpy_2#c($OX06a>VxF1{ue9p z$VcXt5iKwwm1&;$jB6<(ARETb2PM5EHTe@(cBlLYQ@Y} zMmenH&fEpkH~6?mHTZY~FA(PcQGH{pJPO7uP3RJO-!BD?|c3p3fn zG-2q%RKT`(|7&qZn0OLr719>wx$ARQr>9E2?wWjG>)(~WgN3)3b#-FH_~81jTy(%; zv4_uI3p<9ColL^c_RTD4auh8EPlQd>!VLbe5KJ9?#=;zHJ{o@w`5fjEoOI0yZZGr! z(!H(`pOy*pT2GGS*VV%QZ56I{Nr9M)D&f`4$?J3ZWasC zB^6^tCryNpmi_jVLSzSn#4!}@{}A^)Nqi2jeqRQ4+zspdf|n}9QgnS7_-k4?SNBy# zeZRPdmO~@2_Ll_A#CIx3`5o`9h-L zE-ya{5cLfP+`FN@B5!pHKXHY;!d`BCd?D!b3AndI+d$rG753x`IfuPm``AIy=NfQt z`@@Yd;2$C+CxQzd6y9?9;r6eViGt|grASHiM*|uG_f}{I$XkLCPgK6I&=;A27tAgn z$a_V=>$}bWG8Br~5dIGtC`SAdQ?(09^yWvp?}lZ#LA6JbP)nN>7px0gqY={l*>T!g z>ttV_guIGg!Xq5wJyHm(rP@p8?b|Bmo!YXIR+t~gKw(xG8v=&`b+yyyn{ERx(X!LE-N3dHM&y}Vv-g{bE@ zztRQ%AHz=xmfns+1?lowm84-B=>4T3TWVsL7&&h5fNkHX^~JF^#DtNA^*`da zwcy6z7cQA#OC{1JzlYF*u}4^z)cbgO)e2Ht4fOcZkYP2kU5p&ZcfjUvMYT<3mw-JzW%%TT4aT_yZgR;XqFEsGXU7cz`3wu_MC2m@^PFVYoOt@uc* zk+!3e;H|kI?Cec?qd2T+TbsIS#?>5&XcpHU!I}h=pxg=mU|N-n3r0p<6U6u&CAcuVB3ao+no+f2n4TNnOLtUB(y1@Ta*wuuA zt6`LsIc^Ie=%!RclCju8;S68^(P{6$?f6F1{`^az4UHp$#r*&;fyKscT)mP(9>FN* z=gY?%IH;(jI80a`!z@hLWy7oweq3>wgUF8m8rjAQYrgKfhh( zDni(V-Y!R6i(o&OKiFlSM1dMH%z}dD0cF9#wisp+{BL0!?6+dL>7oTQF>BDe9 z0s9C7VI$E8`?da==<%<38!mu*5b&?y&2P|)$oQh-&ddx2TTq3tJQo1Q*6aJQ<8@cpu3_J{wBgx(%UO-&TU z4^@*8e`Bp`5*hKoi^krb(<*yj`c)Lff4AN-idZhaJ!@6=T=Zq%0^_TyFy4FTRs;q* z8UE7MT!h<;3{qGLu=3q6|5K0NFhP#)WMUaIMB`>`HEK`(% zkYFajTMkT{;!c-2&2Ce=Z5E9B>^Mv*;W%Uc;d@)yxz#6(^sR^^XFWBoRsUqmphkGA z{wru2t5*$_fg-iK@oRgQv44a*JC#lFjq1VCXwLX?eE83p42Nmg{DZ6GWG1}1(j%xf z`EnD}ZBkTs{J>DJtrP{s!+9CbRB8XP|N;A5yvjiD92f@ znsDvC_=Da5A#(xiw1+Z>J%T&7+bIxIQ7m>byGP@P~?{#})AXitkLL|- z#!=fHc-l&ioP?KkVPE9dua=2s!tYT4+P%V zKpD7fj=044;XfUihZMId_G-*%jxp3;}9Q&sd9cd zdob|nAWqEp)`Bl=_%qa@C+2%<>x&YJ!B@HZ-m@c79uGh!!J}Qt_NV{H@CZevyX$S= zjsx${w(ujb^PlQ$>%&*v{kN^*^^HH9`UkInFZFYq1%oalcb}m=Fdd9y@h9rxEX^Tl z!T+h2vkBb0VnNf1Os3rrWcv!_=CU?NebdpNEPMb?u1`kiNh2J&05|!{GRZ3v=1O zJ3wKCN%R{LMr~g{7;468g4BIi0jUwNzNwlidzUTurxc>*6B^_Fx7xSHhvYyq! z)vW^}NbT5k8PF7cK5pvB;NeoblNd*FL@2ZxhPN%fmd$*;Iy2dba!Rbu zTX;Ze4F-R>)w(&xKfa^fK3r(OIVN2GEW4IIC-5-#r3NMI0%DfWmj_O1A8#6|=ezr_|2Y6x(ZUEs z97L&Y;ro$?1eX8$K1$!MLZH&>B+J)AiOJV!5LU=(E<$z_TzR`H$^CI)C-y}nP6;XM*lWcC%Ya2kL%EOe(oRvHv)P<*+$gTF_>iO=i8e|6F~rWq|5Q*-91(rAk%N6 zTc!%Z69PdPT@!hp`PI6cr;5ntAB)wWK_`FJv5hp~>>-^{moi_qk6vwUt>av-dSt=49U0DT^cdlh;qDnEiu?XS zZ|4947I%C1Sug?>sS*8G$l5W+!d_Eu6k%-nHqCX&xlhk!k@E>x)Hl<;%gz?mm-F*E z!MSlT`I2ZWkfPMcIi&`(GiqHrj<=sSxKxWuSAO4Wd1KWz)%!`WUdAOnPg&9FXz0~o zE&q4sY2x#1E5X=Wq>r7!+-v)q>}%z@cv!&HX#~q&CqeAlj}R`Y&)jGCLqFMh`~(I+ zU`AAKd1yJ5DXs#(XFL4f4yH7rDpOPq<;Os5UZJb*;rTwxACp3^m%z?Z%zRjyXYWXK z&@;vVaJ1t+M4+;q-xjZ&woK>jdDJ_gQLLE%635AYN_;4NsP~=QR;qJ?$86-%`?2Ik z&Bw`3ZB0+1mF6-X{QFAAI&lWo$Cy}k4Fg*}^h1p?g}Twk#$uwW*-o;_Q}R{$HGmll z>F>&PR2bCK-iy#fyPE5Za* zpE_gxZ4G-?{$5?2qU`>VegY27HHO>Ls4FOAA+xV=Kwl`tLuX*C=z&1^j$;x%52&FDfY9zESje5%FWBFS_dAG4pMb4$ic-0#4k|S%od|qSd4JXh;$5l`UvRyv6(d0u77fa-N zLAs3~Lc4r^WAjY|MEh8NZ04c0Z}x{A^5N`{vV_q6T_sU3tDm_+j|pR+XF<6^^UA%JgXJ4DiVfa{*w>9Z4{qwL z3=}iUDy5b3`IV)4l|ywuL=cfJvq>28<|=+0>#MakyoU3$7Op$o-nsA+8d&A44M&*j zSj(Cc)dm~Vfw- zUwu@hyfE>vI^hrHE`IG^4?SJI)aKR-tat742}{S8Jm)EI{ce0VzvOE2JU}6S8@W{G zWt@!hRT1ynYI|ODNRp}1o6YzoteS#ogD~pc=S^ITq zj${5&cSlaUb#j^J(-}E0OlnxvLL|cJi>>OX?@RLZW|@|!RL3T#RMwIs<{xTh%hMKZ z&lpA$^(-c)JNIW*GjRJbo!pJ!6?MgmgT#)Fx2>-`2@ImbqxZF0YDgH?WMtbd4h^;O zC1fLy2c&G2hzarxp`0c_bcS9kR-aM}k1ub;`PRM{24CDn@_qImi0{V?jT^_w46R(d zPPToJ8+i8>nMtr;zgO@#A@f4b*5y}Zf^h8Pvpbs+xqMcY;!#yvw)kPW{0yT`^AgYA z=ZGiLxmE1N-ev4hRW2H!wdah%bF?;Js)CC?p^>mXlK$s zGBUd{j%0mE{5TOEvEF6;GH_;`L8sv$U70*V&LGR_RN->ub0gTE`I21eU1OZzxx0p% zhH8=c{o*k^ZmF;o%+w-7JgdKcIxEnwLrfa{TFX;9`jzjQ&H7BR0`})v{EPOh?|PCC zaoobs%vlFtcl8_EQ7i4kjp7l{$1MF4(YAp{Fx`FPi;~^^#&LUG)lBUok#M~|cmJUOk@C5jsJl7%tgBw;%r15bB?){)oQxS20XD~4A_O9DKjR$h z)s8bISN104lk%Z=L1t@y9UNU?nt#x#-Vivs9+@g#JWE}qKKP~YzR$)0@qP041@ow8 zKt6oVDD>P=;xz4mX;-&A;AV_$o4b$iB)=|o+1p={s|YD5&fXV{HjI7r>|8O{E7edK6>_rA5ManUzaZw z{}5c6;G-7%E?GnH{p;fbP={?J9IS4(Ng!j>Em7d(ZbYQ2V;woF*mQgDd$=Mwn9Vh( z&otEtjFp|(cg`{mgj^fH!ENqT&Hzsm)V6!|Dp&Fpe;;Je(s~JES=Tni zS6Xd6%YM4ft@t=kQ|2V(*?XcBhi{cP%Gn;A(t9%_In=Ic4>ytC^|_XUTd=TlAFQQk zU_RgAB~|!io5m}*LzwCqy)AqO^-$LziwKc#;)u*OQyFZ$P$F#fcWzi z^vuJlCbBimkH5Mvq@!mPR_>kOM!X8FMOo*9tw&eucrHtID`S&s;@w=uDn9(OV{@$R zpGjvyoVz`i`pi_a1C~@}$l#;N6x1%NRkFBdE1hZfC^$J+6tas|Vs;wCH20pCIsBr^ zrK2V8LA&XYW4$rnFin4AAz*K#OY*mfa> zXZZ(>X?9p)Y-8j8Pm*(i0PTsLqQS6csQQflDc_fLVQ<%hDZi&J`dqdL5u9zvt7yc? zk)I4qTpmB{ZrD9y_`$3d9|PK^4?e%xZZswHRYf@<;@y=?_^?K$+kyhBlsmHV#tgw$c%sfVCfsd#V|oi zW(^8r4f=#9D(l>EM4g=Ta5&X`#J7B0Hg3LyQ-tWgp$0CjU*X1&E>F&+MJ8us`#CzQ zn0U4)MJCF|Hve_jtBp8Y;&S$x(9*^7nF(Yx=S;OxhcONMFz>yYwg<4mq3y1U&YOvhnGZtY@h z!tyzD!t&XD!g4iqNu8zRX?1Vs1e?BoR_md1CuGn5pKK`?%W@rL6I{zN;FS3>^zc1g z{wmW9O-4zZc?p{Y63%d(RMW7rtv)k`p)tymnF#3txwah#_T@Uh7Qk@3(&w-5~EEU zYR56#@fbzlr~FuqvBaNKA84jm_t-Ab+MpUKcAy!@M`;{sT77kfGobY>*&gv)l<_ol zzq_vB?j@^ltfgJ;y#LipJl&CCNTgvP$MLkjclHVDT0M8d{H0_nI3U9DeFLlc#;e`) z!nH&(*^N`;3)Qfp8KIqjI`ytM-@uSfzslCf{dk|9PM@M377m8@sP95gJJCO{b`k2^ zt+ne3Dh}+3SkF&SR5rD4S$Io6o1J}$U-V>|P|Y<7z6>F@Lo36|Ld2Od^Gnhqs(K-n1bI=y&rjlG53{?=E{7l8=F#SqU)*1zRAY*yO9I!LC^=>?Su+BtH zDWw>9IUcWmQIRIvjf@wNel>s=WVWh(wY@;U^z^|{#e6v`t;3QJ@vu-)W^X>UvgG9Tfebuab>cwfqLfCm(P&2(|ShFk$sPR z&b1GvfH_Ih7Y4B-s@d|s1Dv@vu^m#ggoL- zS(e)!#oCp_*=2ua$O@fh3nc@}tE&_l(nw#kvf0!jOyOG_K^7%d8TZ%CuPR*UT21x8OLQzZ@!f63_wpbk)k)0 z3ejTxE89e9Vg!SDO*;FKi{p`r4d;-HGR4YJoDG$9O*;LMOG+3v!JERd31l*iLjqHz zQD&6PI;%&&It{5I4XHVrnz5OfZJU@KkBm|dnyJ^sc@8O0Q^-pHpr_EQiGvb!b5G{SaRnN9j@7)95SQaO?ZgZP636y*ip2rmoSGvFBlF|J>Jp zo%`cgjL}!(Fm#r$Scy0bQ6AcvSIM0Z#K4Lex8_h>XKs%ft*AN0ni!q3iC5whAA#B% z7CNcIv!EjdabBFmA)o7|<>vahBj@AF-@DG&G?|HtGdC;sK6WvqCUs?d^FLEAtkrAP z%3{|f)h@{EJ?SbH7hV{evzs}0z-s(D<1SlZl$)JT?LsO&j}!yC*}r46oo9@9juh5T zRlO>&Gf6g}88N-RWNu&aw0zLhSn;r_Hu4zrBZh=Uhx^{aZJXx*_$U{V7%$>~!2-Nb zqE<}pa7z~}b-SD;4^`K~D6AnD$(OMijwy9S zcwAP1FOz140JTBkk}^dro}(t4ocoATu01R_mzX^KBe`ZsE`LOB$)sS~v|##;CIv<% z)4HS@PE)ZnsaHp3S&rnK90`kY^wl)NMk-FB@<9iR zh@ndFas(O375b&qhH-Ye-#*G!h?NFml&6GU)qygMUVFNGkG$5o?os(piW*^ur2Y3w zzLil7A6Gtsx2M!Xcf&1@)aNF2v1^CfeY`%&s(UzGF)iP2cQosBn!s2k-MObzR4>q* zzNi=G-lxhxG3ca{VdywQ{C5BZqo-}uD*$G5TkAv>!$ADbEw0wDX@)HtBsVbK=T#3u#+Y7?Vmo~(oADyrQ(3NL78`4#M9O4Tct7hMI+ z7q<9N_tRc=*Uz8eEpmb#a$fwhvBqK_da#!lLEB~J)dH2XKQ|SOz|lpvQ_S2B+!N~N ztxkzvcGn#W<6wO;vaycRMP1jzdKYVg<63EasUHOkkCq1MLQ_P?9EYSI(zYPETMyL= zn#R0J_>3|N>1#%DI;wk6y|2PIo($tV8-#N?X$sU&CvBZ+E836LGh^`8!x#k_Y?lHg zxevz|X}Yx}R=Gbr>`ONfo}hHYjKk1vC4E(QT@(M_DvQ)Ig49fxX0L(gvao2^y!C|u zlaTp4D`?byHfgtxr7($Q0&hD>J0kIu9evT5TV$=6%Zu=K*%@v)A#1_`TLQOD*`!^W zW1r>>=Qh+D+j#NZz0ykLghln&lUlEsSGA^5!l?2AMNQC9$y0Kr=Y9eOwPq048T-r` z65CnXI_ygNdULqPIpbk&YZ>ZUSi+i`m*YYckyZBo%cF{ZIDvk6YKD8aQwKaZn006N z_=ifRhyp`Unl=LWvt2FDBb*JVJ@C)#X$&%y2F)6ROhKS>5Vj!*6z4lR+KOA2e2ULS*5-|uvqi{ zPzG(n0yt(+9Np69;)0#9)d&MSYu{yyTFAUJtQ9)QPW*^KOPd3Np@AT6PgJsziY3KB zLq})%h_8(XgH4y6h-O7i8q&ts4^*Y6&V~28|C-u`MD_x)p6q0Pb~d{7^)BNaV0iCn zMJhrztbVnCt`I#575%ygD;ocLJm%bMi1+&;#+bGX;>fVn@H9bxPy!k5xe3Pa`ZX0}QH(%jLr^5@Tm%aZ@ErNSV#ybS@fE z7ceCFDrF7}wh$7oc?E$?Z>W$QE#m$M?5`xw7pzzi;p zNw7+9p=lACQ93;&L*5R)ZV`4~!%==zXyor>sv#)Ny+ouxb#>tdhweDOEk{^BtFq6+P!_iH6? zYlXknWKQQq`QwF{%vgfmN6pa@r3^{Jr5gy4P|t0gAU{7n8#OX_kMm6w@36bFrpfKA zEA4N7@wcwk+3uKEVl#2)p1{!RNYm~b^#XWpdyF3pdzL@i z-otU%(ZRdNC>|v6gBPn}HfSWdz=#{iv15cAxx22^4_N^j)<%KD8F7ACAibTjiI$h_ zqVSL&nlVKI8UcI0j?Q+5V>V`?_`#z12 z^I~T0%N;a#!iLgIl%s!Cq!tAb3RoWSs|!6LW5wI7=HFilQ1yDjR?Sq4-(p z_+@MOIoZp(zm^LPmq#p|r)Zs3Qk+E{z1ip+5;Tjvmn&YDv;CHLU6qP_<{#zj#(9`} zml6LiBp%>=J6#(7Tzk3z(n1Ggn`6u4U0p3d9$27yJ7G>% zUIMEXUSA=8zCOKPcY31{10a5(Exlc<%GDp7W+bo@JjoSAzQ+W-2WwV4q%70LD#OFd z!N-{5toAl*zmuWsSMVU7WbNJ9?A#lwW4qycsq?loO%;l939_Wjy*oWC^ty{3Qu2>z z??Mf<;Ab;y&}MINV#*qMUmi4-eQ*jBM>F7q6X8E~_|t2oVF-Tc!ld0UDF~+-B{@v& z%Yl+aXhqik=i&0zmar)zV_1Dy~8-B%eUV?DfYe%2}c3P__fb;!m2&YkKnzs z+#fVRJpwJI=S!O2>$MnugxhZ(-1o7E3}$$gZWo_0ycGx1;6KpxZdt@sfp#WkhTog& zDyqq)U-2+(GxU1=TAoTAr#@Bkv0^=jYoIu-GT~(}kC7WFwAFRBknp31PQN_6ItEN5 zTc&8NXv!P!PSRH-&yb*p2*G(+4up#_VXy~eakCd{w+hD`&-a?92>eha^boCY@Sj&` zv0PfrT=;FSW;E*YRhX4G{HdRDETLHn*kbD#L_Fj6qxqYhN%-l z%V;`>+@1c*`{`&I?1ny=txSbSanO`OEKKMOQ4x_HmMJ_M*TYf~5uF5+WnTgnRj7xh zslV%@3qKn;ITf%NH`y}DjDL?mbXaQfWoGDVe5er(r)AM%BpIilC?+4dl8@if?8uMd z8@Z;%Gqx-?N!wkarK$3iq`;&XRLT+Ag9?_x=lfDr)=d_?t_CZgXwW-w2ik6J(&>^& z;IuJuhv2MoXOv#E=mvoHc08l^TC3+AFK}C zi5n88(i21H>ZeKm;h=Vf`&$E6W+=NJ=rKysqA7E=mqQFfstzC~iX6S%mX*<+y7h+6 zI3(8aN8tr>pA*RX;6G_>{FvK`X@-wt?DfI*tZAn{X*!z3R^!%ivUu(+qV7RC(1^w` zE~ci<85#>$T0aD*)P&liGS9QTpPD)@Xl?>LtH~u(u*1J_1X8wL30IN^7J=;->Tw2h zV%CMZSKlRMh5Yt%qH{|@*D~ZED>Io0W}N7PEGV9}SrWQ)uWV56bE5umi^p^%Gwk7j zGl<93;7gVvT-kxyOO9l=B!oi2&c2EBZQlR4f)eSBWuj;R*{wU4>dRtzQ#3!V71@PN zp~_hxW8M?bpb&_vlQqPzjD(%*h7)^|PlVkhHr}DZipS)n>toJVD=K>Y!`iDkLeXO2NK+2gCXn_`5poR2?(gFcmJ}8ie z@Q1w4Wm%Hz>A#8<%6@OUC%GRuHK z1Ry$QQcuc&1ppiQ7W<)G#+M1%56SaWWlH+2v+^vZmemm@vk8KD)0i4;Q93i6tXu+) zv9rYr*c|*H!3$x}(L=cHWUsbmsC9dKKH>1{0~lqbMy<7_oGM=CtLIjMGr z&Xg9a6?%7EqEIKtg%+lTn?e+{koH1k^rPU2>f)%(Gz&xN+!$S3B{o`w{w+TF6Y+42$dVL6? zhjiN0O-4H>8H6AbpckW_A&y;m+19hOoxK;N&DznP@XW;ob=NGf~ z#*_O>=2gczAEQ2T{xq#|z_Dk4L&}UT&6xvjMNIJHyE^vv+0h69-tgErw17U$OG14| z0U2pDBmvm3G9N&x(+Qx|agwM}M-qtjj(kAD%y!BQ34>{B~u{ioTWvI_N8bCAr!j(SH$0&YT26FAsJ!S|y40#I`eE?mzV9 z=~i`Iwwa<4FCZ@LF^o$!#6?ERfs0@$XO0XsKWrRx%#I}DCKy8bpaluDhPO2acg_e) zEscw>3k~82(M(ueM0#dua*;W9_y0UJL`*MFOP&k28aYnO-&P{11nm*9rR2 z59epiZ9H`84t5qee#fOljl^qb*lW+vqDN*u&t5xy;GZX^U5}b};-7f#zz*mwux$lT zk46uP3)iGZ&j8eg#?_zEZexhc8!jDeY&>{r1KSG<*q?*@4u5}s)~xg2A9jxZ`Oi(T z&4Z|Q=L?0-vvfM{4fjkuUPv6}#s|r&5%wpBy$QH(&>v+aMxP|>ojPK^^FVX6w|v9j zu;-{Etm}AQvwHl;dV$ZZriwgM*;(&yllX$@@R`W$WzXz&_txw^bzQjXVZS2y0e57C z|AI{7F6WwlvYt+JKHltH4u9;P136IcQqZ#q$ruxouZ$QZbWp&|Nu-Z_F?xe3570St zzd7^}`|ZWqy)OX%nTBo4=Ol;A!sVH>?jG3w_xqZgPky$yVBXBmWcG?!PHbdF)%AE5 zoB`sC{aQ;wd}6SapO|7oLUZt|vu%n9OBh%u6J@%2O=~z_1B5&Lv%!C9P5f z(pJP-X9qpvoKGC!bZHFy&71HyB-U4GnJ~J3iUa`t9R5q4AKgwa2O;NCLr*;go9aJL zDcH*(;`%qJK}t~oGTArKp9>F=iQDAqAqpU~zaM@Aye{?}co4pe@)=izq|bPQ2R-Ad z>6c7Ch`y>9wjVn74Y3-wABtuTrv&l=p4r?W*cltKI(z8eS=Q;PfFua9OcY{BimX}P zVx0s2yyDrl1v0rTTqcqUgo2db!*j@|#%1^u*scM7z=H|$K@NZj*Dag9d^8cUNI|wH@H^{Hx_bTptfjvE=0Kms$ z;i4~~8S<;q>#qpJPxgb8LNU9fwUwysD-lcSF5(ks^jSG)fM9IiMN%=mqtHS*umX7yivO^h8 zkut*3L9)XplZ(iMBx+S8rrv_>wbLU4$#q9$7D4d@FB)UVrIuQ`6x~>)ks1KNtJq>=&Oo4QKyA|4ZzDP9OO4Ol#|z zFAp62;!I1+nJ?gY4UZj$eZK+cypERzWg!wvD6|2RK@#BlL}TSr>yQxwWEeS2>y1qT%}PUox`yDCh_{7Uy)gIxSU ztAwvb0rjaz^%Ea2y*N@;HF9z3l9xuRt4Cgsrhq?4^R8b>4Dmsb5+_yygrrqpwU3$!}I+I6r$;};iMCpp;0=z`d6A)|@VywDHketR`@@ZgMK74epJ+VZNy z`<#cI>2O`l7b`@Ra|1nc@wtXAvkS%Y5N_VLk9|Y_4Duome(<*N_&z2f<>C+@tv+Z1 zkQ+pdG=9opZSC;X_`JS5w%pN|m$&Sp`8$7I6E0DR!s9Zldh(`i?o4rgmoac;*3c6b z)cZ`V)s$G%=cwo^wkF!^hg;k4TUOHBk|+;0#v6leZL7+vXBH(Uq&KYUEZ;mkZ%PHi zkv)cS>WLw>#i;Yx zEDCJRs11q1t6$qziTa&`Fri0A4=(uGfiAH`Z%L2fj2Hv5Zo=&;@eF7?^y_7AbT~9Q zUb_8iCGvcYSb;;KaWBO*?R>t|`FsO(roqQTg@7*;fsdePi&jf|L@v2LSDO|Znx>@= zi4|OXP-_TYf$f>N_Dp8J05?0nzUKhnJ_N6@CDYk1A)hZ~XLPc~%@IN{TP%W}57-O4 z3P*kK6^LnCM1$F+hFkyN3JD7fxt>msgk2wwvxQpK)DE!%;xnvD3JKvparneR8F(7v z6NYR@q%R;oXvGq7q-MBmFY4*pG6l^RQ$cK%KrZPac&u)=OiF`ud=a>xmIuvZb&QbM zC6_zz7Ac575>H8l&VzC}Q6rK#n_)j{*v~%LkD9Y@{1}-7h%IxR-*(89d~i^z6$;f- zaFABXI*4-T3q(0tD3v&u>ns|LMMrEG%V2wDu)QwWo(x$40_5pKi_8I#sCUUm*OS|> zeqx)3a>!HoDVn#@P-VF6+Gt^s2 z=~DxzCWv=;zkOGs-ce!Bp4n7kbUs)2*n;|%Q{zmL0$F@R+nhXGPqnq;=#__83|Gva zk|-C;Xz&qj%x$u#RVHggtobp-s=9hx_ehg-W%!1;!o=Xn{F$W%^XpTnf$az5QsRPy zSf@9$=g5aJj*Y%Js=W4HCP6aULx;DS?}z!H@H7dDw}*-7$uMVJ#QFFHe&n!xG>M{f zY>5ob1STlLO|XyuK=Rt4Pxk?&F$m7GZb+qT6_tCDYX_XsBWQT66onPQo9Af;_>O?h9b zDM%&^)>U=PPT%-avbuTxDhH$^sp9B2UOXX+n@47(rgv>=%%|gv+wGZC(}EM5*G>h0 zsH?wok)u7|kh}Deg<8d;N0wyk&HA9o$n;q4_HDIuizA|n2Pz8}w`5Rd+YiU5#0R1M zt21L)$xrDgpd80~Ta!iuUto8QqHBQ?K!*T2JXH3M*7m#F6VDu&d%U3N+|rKSz3JHt zPxQ|{m?0p8vwQy1QdYR)k-5KlXHI8*+w9`C&n#W_`mWlB`ZXc27r+~U{SA;;d5JEc z(AQ9QGnHc}UhRr#kub=R-J0KW*YxDnjtzAcGYeuQ5@CpLN?U(s_kD{Ca+f_gf99b~ zvWpgkMTg}rIW>Fm)WU2-oFOPWGTo#p+w{_^B`NAENJ2aq7c8O%VwUS)H|`y=`;5_4LtMr}DegL#d$RCx@PYD{JWF;OqkqQra@5 zaQO7Bzr8zWI`#OqhS2*u=CbU9i#zJ8Dv3JeH^Tm?=dj;s>&`Nr-r8Q?X_nuL^x%9|9#I9l4_n(u=M1t zzr8i5liYlL4Z+m3_T@EgcePs`-Rr(87_3dmTJqS^ z&ihxCW-mO|Z=cuO5-qfpb~qk@%Z>Ps+$~iSl|bBiZt6WsnYN$|n((&S^=Y|t_cYGf z*Jq~=C9OIB%JBA&PPP`UdVGG*BX?AWWwqsY99}civ@bkJBOKBv$w{r+Aj?S|iO@3K z*rGY|G?dXGyxZi_N|Q|zPA`#N*2cx9wX54RM09%3)@jv4)rmn;L1^&QnF}-LJhZYX zZ|Q?`r|q4aCy@~?^>?%-r*y2Z%Tq0SWNEfOUXP|qthRjXFYcK4%%&7Z7@;R4qI z4wstAahc3mx>X!iS!MMWbz*F=pAn_fCj={R0e4-3AygnSSnQGBc2#;(JmW_Z;5i^s zTj(QbpTowOO?klPHO)EkjX#{v{uts9uxj zBzD{}H3s3)0})K_KfSE%j!K2MP9`(_)b3o7#5R3hbsxl#!@aNl(1JV$Tu&HR)jSvO z!Y?51Q+ZuCFsG?5Ux~b&Ux< z8#c{+xVUE$St4_$^`+0+-s;>;tlfEkZ<|v+sRVKO2w#Bs)ZM1UOpf%VY#f&nUpjo{ z<7N`E^M=%MY=(3}86fYX&rYr<_z|3M%X|c}%@{b)9bFzm5t888(BqF2<=t!;he-4E z{rwJcP>PTs=u>AKM{{u(KJ7z)D+*w;zA5cC>x`n0QEuP)JihJ8@e=~KLLbZY2Bg$^GTdq{x2CnK{L?G)M&Ju#&{IdLJO z5iK1YDPcu&*-^3hC|@ot`Aaj`&5@=x^=3i3m{@t&5w3gcZrNF>^ekWa9b_T28BqUi zuzw?OH+ThWm~w18Ab386tDwzU?B2T{UnywMXKayy8z4@KWx7ZO6QPqzbrFmrQYQm8 znHEAKwbB%WB2p_GJw1^eno>k)rBZDKd=C9Gl5u}Fq{2_vzL<0u* z{D2p&aGc@#`5heRoEyv0WDDqI@+IvMlKvtjD>5anq0tfEcb*zIuT3axz|$bWXCh$Y{svw>)&sk#7`Hy)AIAE?j4Y4I z^#S)8{0PeFUYRyZq2LsSLJ_5vfpi4%s168WzrIz5Os+s&pP>r59WJ?e^M)61_EB+e zy~2Oc3w{8>IlThyfU~^j+n-lA&^5O+--MH>Am+B_;iPJ5e4T`I6UqFKlK5K}^4l6P zmu_SC;59t?hPr+HKoS|k8TQFkdSp&}ZbXunCWDwzbwk}Giv0S;^tPhdAaRO_BODI=p#2^w|eymTt-}@3yZ0ebdh4xgg7 z!$bKJ;@S{Bl@KPkudgfO_=;pf3)|1WCE51x^9CM8df&mGvW?jl-5D!VFe*26Ap)Z3 z=7D4E8=9bx@N)6_k=N!3)~`3#S}G(1j9x>Y*Tkkp#H5>o1wvt{X}DnaV&}hl0-_!q zOi095kVCF>&WO)3DK#OQpo9e5qG948w+frELjh)sWbEd0m+l(-Mlc)V+j85!t{YqZ zBxFu1tK)949~If`FDDZI<>Uh~8D)W3!6_7ySIh1qe?fi=ZH#zcpZk=0d{Y;sCsF3{ zZGV+UJ-#(;mVxm@W$czXk^pxY2;2+;pBMI(#>AHO78LZB#l)2Lf}f-|&Kq8yZYoR) zwYIKrtl2cvF%W)Fc%*H{rm4+S5>oSe@7R1tZ(jcJ+5W-f^D+iH(()A1NfG5!ZC#t{ zix(z>=VObfXJ*bQj)^Irk(oKYIF_Z8v!bPZzDRCP%P}NYJE9HwEtFNSrNyQ$N5ybc zQfP2dv?WQy7bvX_CHYI*?3T)&jH!#GnW9Whk~K~dzqPYscv@0)1SbP$AY*<)KY=2F z0_`Q(0Ojjw_XOA)c!Ecay^PusFg!UrLvY#0X3or1*;D~r{s%N4#3yJ%c)wU9Ed8QU z=+_feLgEX2T6{sw2jbC$0NwJy7z_J758p-v#-H11hJp&nvhxd2>)jhR&w8l1`|R0n zkj}nErZjWS?k1M(yuYu_8AaV&@cZ8vVDD`UY|}yScg4xU)>QshsH}UdJUCjZj1HE` zxK}xOc{GB&3|f^@2o|MO>d~tnwU^1B19!sqRA~QP<+Cp;BUIj2m|t|<-@H439p3E; zmQ!{Md`vLvvR9ug;1n8H)HlLQ6|iR@_cJ*HG(0ZfC!l=bn6?`&wt%|X z@<~5&qc!mful^w3|Aqd@%WU(c{h<*;{iVQx3WJ?9@B=h?<)~>-U?34DkUZV_Ag_-w{*wqAK+6?xQ`;PT9Clf_=kT7f^Gg>Bh>QS?HxIV9%tzWx!WI!0u36Q0Jqz z;=J1O4-d2lY&)iWUU~c!QxN3}5uDgni+7kH@$`7NuJarCpKI=T_Q$!&qVijLh@^QB zFU!kY_VB!U4-e<%4L>{&ZEh#Rf8hVE*7m!|tCd?W4iCS$rJ`cXi^Ibgw^WW+=Pfze z*L!Mde*V%^y?rN__qk#M7e4f>>>!(7^EnPcAZWd=29Q1M6Ey+DAH) z;^N}1E2`T^I+K!9h?ZQ8V8fP-28n?&jg*ulJlpnUwun`#w2f2pmz?VBKAM%1kXW$v zbnn2KEaJtImzE=x<4p?~`7@=zT#xWhNm{0mLOch@uJQ-)nNlRLVf<7{;J+xxy3W^q z4iE#K#OwI#xQf}_VLRK}xTYySzG+Qk)7r-P_{O!=VSJ)K@`^S7wF9&6U67r<;CS!e zpIg+W-771qH}*J!WX}2M0KTNREM6=Tjvuf)NB#L+j84f8?b~smdnk@2X%o0od^5&XP?lYe{aR-7I2--*`LwV>bnP)OKe_G`DLrQI5e_w4% zQv1efgZJcWX{lHmo8MVd&{L>Ssp}o+txHL2UR~31rDzf`2^9`ytyMO zxxBGFS??&TEOY47I@Z_DzS}i4x$Olx(+l)DZFAakQYxoari2&FsVraDmC;>1e40LD zYv@bMpI)Bn$c~LkjfvF9=Qib(E@`xSMk%i`xuhZ8lv-VoT9Kb!+LRepS&(39+txK2 zLViB`_D+Th6B;r^J=$q&_LA$^P z`}z}(^;qWQ?g+4MpOLs5;$VTa+3bE8XaakL#wo{qFhO@lCOCV?i1TlH;qztz+wbGa zf#WbKZ63E57@#w6lqNZ+B^Nq>I{^U?ibU!JFBNSH1aPy*@eLdb0QIDeJxA4H z8yb4q;6^sT7cttv)c86D_e1=dt9Yo&PXBelNrYDf?vv=zrAM$TdK4S4bpqma5`L`bD&u%ajCvM1f zmplkye}#eO#ErKpCZ8@u*bO!%{&B;JG}$(U4}im4up7#82eN%84JNoL+7mhIdZGUr zP>8z&2);i>1rs?zq`DQ8kAw#poEwZ5#~N-<@SnzWV%d+G6E~kI0p|-};WuCQdAw+q z^LG>Hi}SM^&7pAaWhehO-0uKW=|+=_7&$dQCj7_{A84j$T)2tB}gUBp`FX5?uRUt(Sy zf%??|+e`JiUuX)@uFWSBnKIzf4>qTTNl4$`dIDWP%(7V+zH$N%ej6&>9lci}J_@f` zeXR~}1K7w{+3C~R>D4kN2`c3xkpfiFjD*{7k&8yZPPAAONk%M1?W93{_B#1bw9bm~ zl^$r&@dfBY9&S_R3R};}0cq|ScI4mEud`2=Gko?NT1e7C?3a9HDf@Hh$o59#ahgJFnojxZHpu!^|OKS%^Um22-rVv)HXd zaj7mQSgMT8GRG7lQ@CcTHbg6=A#Tgb9O68E5bD*aKzj}uoG%FH0617OSY4VIM5 zL$C8n2I2qa&YepiOqjZ`tZd=b1ox}+!#kHO-nen`k~=xRy+UmR-_eKheQ8f51c(d{ z7US@wKhuY0%C7`LP8?^==My|ZDtcChWES(__rW2OuZyINhZEj-%@|0No7a=3bx=$P&Odoaz3BFRI>lPppw*6PGZB8~2|B1h{&x@3jB0xw& zf(5DUM?wl{WO!cgr0ymT&?9)C45=MBolxjE7Y0EbK+z2(4Q+1H72$w_7Dzrt@kv)K zlUExGgrCujbSoCc0+0clc(3^{B?|GqB%gq-+>7!=eD|IBA+C*w(DiK4NNoatfi|@i z^A2By(9>hX=@x=F+qnw$JI;tc11sQf&BfoHp!O1XKsnO5tRI{Eq6r!Ru(!A~E3}M>FG@)%iet(`vO1}~=F*O| z*y6N^h_vF^w2o3UH-6jr0^*>6#4-&RKCfo@V9(}_HLAq&^z`yXb@iQm!P*Vh^3>4K z)N<>(JJ8sCLM7p1{ zTU;9idyJ^&Nzi#1J?Z9<4eXy}vnMIDE8xD(9wcwyvFC^asJ);bqJ6@KbRNr!Fu22j z!mXJRspFQ^RW7v0ke89jNXKm^jB^3I23#CFg7hNoqZc2Li}*5)0_nx%eyA2>GN;;Q zK>*G@&_%6={N9hxt&A!gc%ic{4}$RCyNSo6zK8Qmh&bucbGlvtbu~WCb^zNx^53jU zC>IFiLSijncs)r1xaQ}eybEa4*`7~& z?@{YCIMIh5u5s=gL#|Xj$RfL_W>~7{)Npavy>BmE`1ZYB#lxq1SdvkL-q5V@o`=_0 z%zy6RBP0KQZhpo3hkL@aLfg_6>Q(B+5B_20-GAL*UA_OWcdz`zgNq?ZkxtWLtDg99 z-TT`@QGDnG>vdP&7PokL3<{_g(H+5Gk5tPN&$l%0N0gOYzm`y)UJ4Q zLQQTQUHsJCzD-7oOi}B4q6=ph4?dLJ9hy{ZE;LIu!M3RxJqv0pENwCD^IA0>;@m%F zQCI z86N`#0aWJvgos*OO?s%3U~9;f%62Ogt_m&5O(g%RR+FEQXQOiZyIL*#&v!?f4Elzx z{p}$50z1Z@{(YxNT-^Q|$oj}E4dL$1oEWuToId`Vzwk};xY|iQ9_y@6lAs+o9HyNF)b%0zPYg`8pj|Whi$f! zuk#GNvyo^&@L?W9MnL#}$nP2qf0)m~`D6sdVHbo?fjGdm$>Ga{UA_d?`mqA(&gvm(@7ZP-| zL7)v)gplv#?bv=tijuFO69(qbO$--vap@b$&w!Es2W<5T9!YC347N*HVY>$2qqx6E zfzwWH0RWHT-^xTm(&u1H zgiOlb1%C@n8i7y)VqtTLX4pS*4F2B0dkW+A6mJak0=7#$is9!m{5*og{%Xb^Bhl?n zXl(1SISxJ3$Ar<0nHZ9rIiw8s{)eq+8&96^Ns-K^iJ6NAYOv{_!kd)NG zeS+U|wCw_4f>z==)P}#+v`Y~dqt=+h6^d|^MjaETAnHAzV#5@Quvqx}A@+a4Mfy8@ zcCJ>!i6IBvn&i+$0s)_RW!1_ckuafluqYy;Xs|XxC<9R6;qNd%8(}{H{-y%i$lP+xCCS4Ca0?ac-EC}P zE^N4z#q z8xLOx1_m?p25Kz{VhP5-7ScRA5q}H7^n2myx~z;oS^&4ya*yI23NY%6n*Nr^A*T7` zgb2!Lx3;>56IpF9-&yfU@0g_L+ zT0y)~HY?;*qYZHsHbcA^@{(LFfzq2!3&k4wiy<${!c^)|*-N1p8I?>ZgkT8-hrTEe zWt1UuD&9IIVi=$dW&ZdFCX@kkMzkPtNGK#Jp=iiD6r^U<5%4QzxQbDU7FZXEf(TL| z;^yBe_7M3Q{Q!<9V^n;!hBNru2SkL-K*uf+UnDTZpi~zYu9LExb-KE9jZ$5Bm`>Vw zu1*(Bu4Ne~^ec(~0bfTm;e?z7m!(oRf|L{COte=YR;b&t%tN+tT|t7eKnLzpPYi-} z!CY^sYt)+#teuDfyXmE9CJc7WMo6XLGVC~vi6&kOfDG2V%f{A`({MhaE}Tz@(&Ym? z4u%B&XU=?qms88=D62$<@&kvDB15y)?h4*pS|`VDW;M^v;2>B)y6|XU-x16klXdOt6GR zcjsBBIIQtzov9mqFAa$diLy^ICzU#qQXEreW_6<7?p{xdsrOxZ$lRIm+`gBddjF#q_W9=iGfVUGmY(S+eox!4N$IQC zq~xm1S$EaeZ=8;)?U0rl;Tc&N^;4F(NCqV~lqjOs` z)8`G$A{=2U#b!s>lsI#W+3PKC|0sEu_b;5QiKk+4W&=|)VE!c{WQwC*(M{+FN+de+w9}+{_#MhHi#(qd+%5z4L>Jfd4M2JHiO(Ni=y9C2~8) zXu}xs>WkueRdG38OEb&Y&7U62+R5&ArB42K;{5e{r&jMZ%y|pkV zZRW0yicJez!Ws^>Oj*&EmQ>fDDJI0Su+rT4*plgv#zQS^WyP^dsl4m;PDta*F(qG4 z+u*p)@QQQc=qvzykNBg(0q`58mO?I8hBgZDDc=l{?s?-I%S7b|+YBB1`?J(>IdL(y z<@uVLU?L{Zm^DzJJoCZ7t=smW7Zxmj1(5p={HIT%rsa#+t7jfyKmPsd@#A?T7jT)`Tpv-vb2ccf_Y8+=}QVUxeJfaI`F~1 zin%YbKiu~ed-a(arA0!%IJT;O@K+%A#Roy;r4>yNe0y@i(|14EcG4^kM81#cJymHKqj3}SP4tQ++vt&R+h(Z~xkbeZfqZ}OE<0Mh@g9L`06U%x| zEG;Zqake+TwW=r*YJQPEr?a$a>rA`7`_39ieW88SoZhwh#PfZ^p)+?A33n9HF@Bfe{bJ@<>`L#2%wEQQ;L?Kozr)AMX@|0K1?qTjS98T+S%H% zbEZumVgyMOTjF}yarK#ja&A`&tB#6C=cyIUL-Fa-NOQS1Iq#z37m8p z8Ro8e&xLn-(TH9IQ4)(yZN@8eIX-egny?CvVuf{zfd%Eu&F=L zq9A%C$2m$+InJMhpC+=K6$-G%!j=&Ql#o3jlnDefA($_ukUm*NmJ)B$_i!K%cf}w`wikQ4F1GhQsX8xe_Sq@IuHJ=z!f)*_)Z!TV=6S zq1Q?avGIs?s33oV^;qiBnE1Guqp3&H=OOEn)FZL+@i9k<`!i=%S}c{b+&^PuxChgY zS{I-n!Z+|iF1L$18KsOKI-<7D^dOZ)0{iA5G8Ezm^9MrsOHKfOFs%p|92 z%<4!9v-ZXHg~73w*kF+{soEY1l0uwMjs!G&gC8$BcY6a2x-jrtdlyNxtgQOMG9I5F)5rBeY&!?f$ShmM$hpj`GsqKvXuMOx-~r*j-p8KsL!-i&nYpB#Uk=yvT*T<-dV>M z77B_YQe$<*C8_cXzk1{IOBcO3QbW8gRedeYNz&5-T|^i?dWbNXqeWQG@}R=2hPn>z zZecGgz|ze*YkCIT>NdS+Ta+>8sh9`S`;X13S=O4bHOWK8YHNN=ZryNw9QZpUWeX|_ zv=$wy3RaJ17z)!Oens3PR|C_G6T>C3g)J#s0VR*hNeWA!v1uy%o>|PW@3gP7P*j{e zyhs|CU6&4~U>rLjj!95g>5(t!iaf)q1JQptmz{TFx2>Rq-3Nx)eLdU@{Mb<+CI%fM zDNRd+po3P)>WP8u<=N~;upYjUQ6bKLbqu34s)-|FR}NFC^(hC=ag0J0^`xFVFF?_# za`+Ky3WthILXVC*3xG{!bU^muSPN3Gf{zJE!pZ)z5hQ?}d_tpnzVRc4f~45%U2%+x zZf5`3OocKDUC%Odf_lG^r%GJ}xXYvm2UIF{3b>$Evx_j!l~89C+)kcas-S-R3xk8tZ=X7K`}2c?FKn;x zD_(f8vGKs7;^IXI8XFHTEGAyNztuT(|%f=pFmbg~z@6+s7WidaS(s z*i|@2&{`xvrFWs+jLa_*3&Ye@8klwN+uVis+ zmRO)ur%J1q9BOUYzqmMc+QK}xEV|6tp^|4B#5y7PJa5Ue)n#={t776SXXk2Vgs^k> zEL(E-o}T(`>sREnxk4e@TN(R?A|XDd-n+h@eEVq1Lc32FNqd$MyAQ7BdpQjDE+|N- zo|Aj;{Zm$-ow@4dK(^APFbZiV+EzQis$-+= z$l9@`t!&}6EP2Fp+gcCK&np_Zvrr-6t0lJlq6|asjHMYxiyD&42y5N^qUxavgNqs{ z2l0@X74$Y9jiM>wj7E71;iTpAvKqKSA+A92yqA1dD*NX&XRlmAiip>n!NOq^!) z?R02W%ms&};$-r%(RS>ojD17?1I`Du@9&)i2s6UVHN+>?W)-I-Kt{=mjD-&^&Z^yV zzAt;;jH!_kB9%0?cJ7qM-2*w+&K+G@8!GaKh;?BRERSquZS7|-sK8(N7o*_>sphh zTdHG<%CqJkoZfkCNm1eO={`qa-(aruLmjv7Z-K~}h~MD7U^tI#j6;B?kpqsHiQRIX z+8d;zwZF?wO-&`jW7i95e$?~MUiiCC_&YoCI~21Mo5oF_0?+i}bF;@YY;qhx5fRLq z_`)y3Q)4wcb9#7wos5<%OtQ5z66><`Aqkn0zg3yl#lZ=|LWSU8;h9;v(Z+NGQdIMKt&`n~27WeH*q_$JG-syTVd( z!0*LjOdJ^<6rxUuQcCGQMv2FDA@3O~l-h!G_q!9{+88|*$~M%ojo@4zc!F5X=73*R zf>+r*P8J>@R+AqSue$f9A0YEaUnO3x#NQp`eM|le+Onwbf(V#J06OA25g;$HJJ{{% zV3^%&2aCXv9j<>%zXQtZ|+sEHQoWO|L0Ua$B1UtJuZy~?Wf{)3M$=R-bepj0Bvt;h* zYv_+itOA=*_&@JK-tWlE)FSSLA3j1FRggsv_@}awdJXaD064)GF0WY5z6F{G*b?x; zaQQNzBg4L~Vy^&s)z?)(%>JhekMZl^4EYLqoEL)bUZT0}(797Ox^Ck{;%iv}z@#)Z z)uI=xBKtCJvwLSIWwdHE5xu$gIkUQxCjEv?kZ3|>*_Py(#29NtrZok=vafNlBo_>Q zhv+;9MC84c9qTR`5a0;62mtJyB5P9^)h0a=FPAu##8(nI@q$o7ix~DF0!CZ}HpYWH zi^K{X=4*j2@bJyIUDH>UOo9NH5L9| z!~I@0fuc8vubj$97?Z}LCrpeKQ6*(yTe}1h5e54V^dk$v$~dq*U#t|OHZQ?8f9Bn$ zn>us}8+qJ$;1X&R{Yjxt)LS$r#`&I70vp{H$8Ifveko8WM6^TzW69e+QsWT<aNmGa0T_>LHISPcYE}ZwGb05wkHTD&G9{R-+d{YVGft0uj zZ=KPem58r#9q;Sh`)(OsAis$^~3nSbmh zH3|6%wFk3mHRUr`l;(8gMFU_gkELto{nUAP!s0DQnx-{5`9ig1=&kd24`i8(o0Ix} za&&!iTH4CfvlF-OQ5UW5%C-$(+^$W~1Ti?L?iLVASCX5s%{kR~B^5-Hdf()n-f@a% z-;|cRe3CL-YP0nEtXfNAb#ZQFWNvY_^KsAH#Kf9xy*|5!IF~grH92|eK$fGkt|Tg| zw64?P1u2b+Dyi!P15w#6nU0q1C{tmR)jF-vWGb9ywKf%+Al#7&;Y66Feo1-x(t1l= zNo#U)Ye}4?erb96lJU=+j1P`|ONr^RnV9=NMy3O3LF}w{@qL@BaPHhP!&~m78B!Z0~I;)shqdhjyn{XT}Bz#B^c# zT`#Y+_12|S-tl-}_3}mwy%BBAtK~YmA|}J>^M3 zcOh7rz9?8gN;C!pOAeww?cLRpAd;Cyd@8zpal>_!?`N)WD81CX{O7oMULzL}qIYd4 zaS_tgyUt5f+5dc|(1C2ZlKibuMDl~cKWRq7f9{7$;=D>5DGzdfslD_C-(}B&9_vaFXpSX8XXs#l%_So_LemtaA1{=W7zw330^1(VvlpWFfx) zzMNe~3_%%732=vuyB{*~7BjwA_C~-=llU@o8M@3Y2(nqb`*%h>9Oye3>Yua4XRjo-#W9zdxYjQCVDz)=AtnXF-Bz+aV8wxl8-#iOfq>QTF~Fg4^aMQq1BoGDQB+0Uf- z?CjloToh*DSdZ(MRP(@O$SDhJOLU9xr#-S1i7Ve2PhMFQu;pE7nz)R7%pjGj@Lo&& zQf%U`a|1Y&4~LkAd%`?x;xuq%apNT=>>T>~ZN$ zPIne-J2fk-fyw;3rkd8)8X1>Zi7HcPedx3(4UN-*%V5FgmsrkSU2G?no&7HD#5^aQ zM3o+*4-fly`W=Zg)6tnD}v)@_uC2Lz_dVKKAv?*|{&@KRyM=EV)+%YQ4 zhX`ZW_80C|RN!YV@2gSzNj9fhi^rt-G&|w7IX43@v#oc75r9*Y-30&OQndh#1E%G;i zoj0jYTajn2P2mHl9AioE;(@yNyKIiS=skFSo?npOWmj&mxHos3{Po%6gR9$^pO~Qg z{fxRn92bO=DQE7vEYZRVw)jY--7pZU9og!cZ`g`?tLv%*@YmPah49So9=x`PO@shU zM#B5ub>sQvFXmU|xZGVlwKZRH7Aq?}+Q*Nam$AiKP!pMBPTf`UTJ?P-Yl<@LyhVdV zJ?^r$-;sdI==Ww^1)46>&MY=zrToW zRT?{+u!Wf~*|?|(WIWb=J5%1vN{h%#k6JD9aLeEZ$r{8;bV{zMUKe6lV* zWAG}7YhrseJ)>{pGQu^SKiRuA2KS3}ZiPHUqV64&Z=P@M<%(R*BoSv-hV>)gN^>*1 zu71_4+%5T-v7We|jk4SRE+7#sk@|I~@$uWfK5_2#8`W3p-?3pUJzMTEb-O%9uy^A< zo;@>)8L6GN7gY6{U?<8i2haYR@QrUQ(jQZ9Qo?F^7wM)RfF2xHJ^rGUyWgwiG2oSf zD)7X`nv?FKNfx;_F&)VQd&&NZHLRtZQ~~ETVbS`xLa$EyS^7)e5`~Oy>O6)Vwb)D7 zU$jIAZt+bv6*e6oUttqx4j>a3JsK5$!C)5>xOF|mecqW=PQsZqI$)uP^&0(*aVE|+ zFUgl%iwy0(^J0$zp5n6dd2w#(+&OZ=3MrybYTAEm%9@w34)4r#$++CXgfM%W_$hsd z!N7e;;%M9O09FO1SnF)^UGksVz?@n2-#UDW|B&c3jYwh&Q~HWXeVFxS(*|6T20u?X zJ=+IZ=5F$JrxTxrlur0oU2=({kN7r+b4do8rXx;E+`|1yj%QV{#*=WWLJp|94P)>t zn%~_GkYX9cwI{G`c!MM&R+}4X6+j8{EeKMccBYhwU`NTYeH~(XDe$qTb-PZUlkDE6 zCmAU4ZreJ(&qHO^>V%U~qD~alIimVlnWn@mOn>({&kw$VyP4~65mZOYj2NrkDe;xM zpQ`ZXesn54GF~<-b%(Mzac@=}!$<#U+p7-$EZoP&Wguj!epGO)$fK#s1BR%db(;>? z-APN>5{Q+9l0&0B z#o~NNJ}+HpoWr4epJ*(Vmw3Q#Xo>x-^eizGsW$QFw`Hi4Kn`)=+ZQjT+NnS^jl`eV zWjSnY(dhcmqGXRRm)L;dwW1=WwRohRlsGe60nDFnB=pjsrr}5JXgr2`EyAGfai&rb z@dJW=QG+s3aF|x;G5zI{^_AC!PhQ-!D^=(}D}SsbI_~?CKu><$H`zO0e?FJJVDXvX zAz#+QjZ%KNmW7__Y^ih_!@R98{Zg`1FDDy6J2B&&j&S^}ds`EQ!}o|fKZ;mn+y}f% zanXpJSB;nAn<_7?;cOAJ>K&E(Skj&tvlk`py+ypa=n7(xZ3 zNs^il%IAl#YoAtZaz8bCj#?~YvCF){f6Q9s?=s+LCf(H*ZaZIYJhlXLMKu0+L|O+e zdSdN#g?o+BU>F#icQJXof+x8RJ*F$%2i%2=dh&Mk73@B0ZCyhNBA>T;=0GvO5+^aS zmgFz%@0^LgoTr5(@C1ezX2X*fVOMygH9KnyRv~9K`*IqXJ-cLaOezecE49U^Ocpzd z6Vh=vApWN$H%RWLeta;Gvp?DkABRrSiAY74nK?unt)rIPnXjI1P`maU9fUy~=W=la zHyYkzA;SF}1wt3t+F9g!VUh>92h=Y!CZ7d-&EuJ$Yw>uwjGy}bt=C;@*@uirxu-Wn zpRQH7US$MJ@x|%BQ-o5Ox*vqt%Q<`@BJ6R9bAg4mpNVnb2{tWmSLR~nc2_do8`1^) z=2t24)T>KH@eb?WjGx|lX}{li=-E;-eY<)uPWXJ5a@4^Th)zS--1V}lciB_snfmF7 z8yK3dp+rP5b-#iaI@(KouGCK{Kzo=kc9rO|dwYfOZt-mD9VildZGg7GDcb5_IAgAQv^(q6g* zuGn9iED2vfUp`n9B3mH4O*Uo6Zr-=}?(3I(TWKNFQ&L_Qz<)hyG?j#V zOTkHqPW^~Z18{u4T!K3L@};Wr`BrU5!6&Orag4-YS zV;y3pixKW|uL>*k?f5)#XH2?ku$%8DR>N3!`>`)11SLm;(uZengGFw6JtGwnHu!`u z6<|F;xocN1w$Un6yYEqfe=}m7TcemoAz1um4F=v)AnYJ5=jvBWP8>dSB4hH!h3it{i&kMovl?h5c>#SbeW^|=2@fv&2 zte2V*N3$0WUyjngQ=t*39Fctn(a;4cz0N}UAY%hwbsVC4!??EG6*;~aB!*zeAn8^{ z+9o9yVTYbDId2$35{}S^bFVYrrf)Ha@vUA=H4zbG=78?jCMF)55>Pi_HXCjS*xz$t z@kpX5bw zGu^s#M8P#bxr@+)a}u8sE|TF$!q5<6VwkjGV1m|%fh(du{qN9gE8)-V!Zd@n&VhQ5R zvoPj9r=3EZSzQ*>yKkFcnem54;3+Dm-KB-4_gc+0S@#8w*agLh+dZ{!A&~Cz?8~q( zG!etrx9)o72e-xD{MN6zJvwUYh*K(pM|}gB2r%U`?9y~vl# zudah8)am^CW;E!tt#>4o<(EIae!d5NBWV0~MoNACzGo>z)Rwx$NHs{Ja1JJfJ0n9? zV}*0aO@L?%@~jx@fDE|H$V(YHND!tunFjWc8nsPG7PPDg8&3I}%Q*N!b)`|A{bexz zlCmnn#r}7E2Lu#BBNOUwlHZ6uQs@d>9P7dcEP22L&4&&Sdg8`(B(IJGQ+%|zMpRHTKoq%X$W z?agD%`3-o;E#47O-KWi+#u1~Gf_b-**ber0lWjR1J*rjh5TgfcmYD7Pd?aSXF z>N%QF&@+6V8Xs(mnlNbGr(h2Xm?aladRac-X12_E#eh*hV@ zfp58cZdUFTU0TV1PTa$#SMFX^RcidP(mC5v-piKSru%Hx4Atd&}%v3YtP@wj{)RowllbMGd*1TsixC zw~;)(QNmyQo~^5$qD> zfAY-Q;^xvM%@RdMe}{aAb`_|naND{j1c{OFMCzvJYT_>%ZV)oVE)ZmbDNbGz^R zDsz=qL)U<2c$GpVt477Tq{vm{o9^d<<2u<=G7@|0HWnp zDf9zdvj>sF3sPf#s@XnGJ*Z@i1JW2NMyRzaO#U9ST_+Z4Zd3H3|A-LTDIh)M5VW~TjQxWc2qfFucf``s<5A|e+&v*^KVn*DhG zvs5scdI6dEgr&!{in_^o8(($q=G#T`w{ur3!v%DXrK&pLSx_X8Uh5sO5%>ZvR-;Ih z>3&`yqjUwP3!?L`j1L@b@%HTJY&NK%3x5&9MjrT3A(FdXuN{=k@%?gZ&RfWmPYv?5Em3u@vF*=^-18Y-gu{hsB) zDV4pSh%DnJg0O7U1fOy|kMf4DZ5{QBF3&gxJpQIGrU*fd6!jAzq>Ey=%r10x6FppZ zRmpCL7oTDBzR%Qpu2EZ2-g!UkE9!CQvU=^3miXs3Px7JkXeUBr=7NfuPI~CX&0vpR zL93@%=p5}n>SVrV<#j0PS#mn%Z*=wSX|Q&U%7yj@NaDn$v^n)in(wxob-xH$f6oz^ z1#@JlbCO$p?qed$@SGtFAP4qD;W3adE!*2~Ln5U{lpYMhv zEI3(MOF6|`vW&}=4BAPCE3Xb<@bo5F3qk3n2X02&7K-|`KVrmre|U#N#K`GZVrR=8 ze1noPi|s4i+a2zeXLB?*caNq0-nJA1`x0~2AG?w-B_B*NOqAe!O-X86@#(qPhLQ8; zj@^Eiv+P#OefdA^$X&v)G{13%euOvaK^wvqkMV)IDoPF17HrEP+buj$z+#+^BG2j-~h|u9mKu%Bu zUYzilS{5mSRM5bjiK`}82+3ir*2|p0C%H*BxKIL}(%n1f zr^f<{Qe+RoysbBsa@x!>1dAz$s}UxWPIN{Y0$w-!urymVVb4dgYqf6UI~B)QY?y8O zA#~`2?l~lw$-+J-Qb8w$=SUDNVDimC>6eJvi*Ce>0U_6{@k=XVZ1jQiaB2xc<;Hh# zatVUo#xjRyjw=QM{HKnX(?L~67kio~;DkwK1+P2X-Q9?$%{Qsw{zta^({d)Q$at;H z6!=D?ko(C-qa8PDTF<&b(PQ$053-nY;S(s1y{jj9TyM=?Fy9z3lG&81Wsx}K|ZUp@S@6^S}` zdhWl%RKIT(_V5u&4*PY%2cNP+2g%=jxUQ=9*JOe|Qd5QfmC3C8hP4l`RHk>w?l>=9 zDu}MFtx!j=uvLZ%O&_E3MRSway|xgY?Icle`sTOyF?&udYq@E%}J^dWNH3>&c#Tg`_sM3y^T6OMUwO`_e9A%qns?{+!mr8!Ldh{Sil z#FFt;a9}Axiio!{+@ZNnO)P?+&FiOGR%gG-cBvf)t~h*5;f2eTw@)?5bfzdVcSa4o z-`eaHLy8!ZasO01|0N&z5h8x;^VZl4h{|J{W0oUn$hyDh>;+)T!#@|uT&6ppX_wYAMYOi%wo(V;sW)hi31k@&D~n5r!T{d^4%mWwOn% zgdlc?mO8fN$co|~@0-K?RTqVFd#D7lX@6bL)}FW057g(|@;c$XP})zIy){ovKP^&zzDM`#H?Ca*yut20Xo;4QcBTY^zd?q0t;$ zj6K{erbT2JYz@-^6BdTh!DPXBn`O3d5ba3#1qocDrWOb4s74hLZ0IbX8-0o&>nyt) zZHmA7Y*5apl4g;XxM@G(S#B-AGPqD3=8}^9hJ55MoUzV++)CgU@MQ8NSL%-PYi~&5 z82Tx~RJAY^jjOH6nH_>TLn(^C6_9AwfJ!=$Pn5uFCZt(|84Y<-&w*#K8KKoXi9PyR z65%=ojmVqjbl|rM)aQC1EIQgbesJ6MjwuX%;hw`NQkLF7!>&SP@uZEpP~Hp5(v|@y ztF%z{?ApSXG860Fqu3Z}$gBS@Jg1(O{k~xnyRCQXF$jD5WJ@}9m*=L`3twKIMkw1U zftQKXkiB~JNx|Mzh~Pi#7`w89KJAzNArsv>M&gqf4Fii>gkg!dFlPe9y9Z!AqZh@E z8B=Y8BRFb0Ldj76@<{jNx<(@>4zxA6RFuqTzBrG274my8~vZ%w?e zzI><<`YGst$$wuTBzy{zSNI2_C4Y>=hr!Km{FYQhwWc+8RneFw^IrMy)cl7p{@IUO z@|Kj+=i{;R%UjZ$KW^m1rK8$>gGEWTS)pU*7131Q9EMj!^GqY_0lqxmyO>*lig2>; zJe21<>HDSC^QM0#8mi?(3SlLfF3mhn+VXshX)Ng5o!|4notVkPXk3-`@-^bvE~SQ% zB!jr{^1p$$NcWF*)AddD(#wu`TNUA@JtlF>T++6^K@g`Ui=I5^ ztbz`n<~N1^pI3 z!|5Dnck|y7WEpYM%|DipvvoskEwMs!76l&Z{s*B^>UXFLxxzG_AB}W zPrLTo<0f%2d>{D^3tQE8t;_LE3&mIBT5xSDU1Fb9d=8@Z9yQzG_}EUrUP&7uS**!r z*2?zJjv=<~r914*_kMG<+%9l8hx7u{%@Jw9)OdR{l$%Bd<{h{Z3#_<(};}N20Y0hZ&+fE3C&f>X| zrUYcq(zsFfJeehMBbyGmB!X|>B0lFB83!Rtg#DtP8Hl7Al~cM@zqU*uC+Abv!xw>j7|-u_!;M9WpvOJRw5p(AaZhCw`b z?cu)KxjK>0-0R3tG`@E(aR<36^Ng?DoGe3+7AF(5ma|Dvl2T1j+>Z<&#nP-r&T%T) z4}hLv z16np8fUu%!2<8%Wd82B}-C7ICW?Tg4+(+0A9Ja!#rXa7#*j(@oTyR?1-GXgVUC!Z} z?^B+ieDIVzokV7cU^1<+rAxpo5Sd2_EA|l`S*!hQtPp_?8pYY$VEVWwM{0l0E_Mby zI$$1aVOkqA1jnyjJcP8$ zY^-zy-s39kmH4*2tq+3Wzrs0MNE6NMZ^+Kr@*$GnJd%*MeX*Kj-KL~|9KcTNkc4glVn7C%} zJsxjOx+4RG4Y6xXgi~@gYP-=*`#TLoV+`jd>TtGnebsy-iD2==hXID#_E)5yos{H; zv5gqMQm*FPwzQ~@VjF>wr3Uie>2$tdUAdB3j1|}X8R4%*V0!D~yIYs6jR1^BF2)+r zOd~fxjbd?+kSvU63D>5E+bT_nJ`Dn{txxWo27%BkWS=90 zN}(+ha~uM+04R$tDS(PTWenl89~?E8jNIZskt8zFIII43HF za;?3SyshWfK_oYlz6bb$=)0K>rEzsKn~nWCrS*(A**e=5z@+GThqLz4Of;QqVviO~ zSs1uBiw!2R3y50?E#<2C>KMc&_vHtnf)ItPv# z<*W!fDLAUEy`JK_F&E^i2uQtJ?>Rv1^@OEvYyMbs5UkryS>`b26~mnRZy9$o4-~94 zJz3@zi+?h6bqi8Nc{YW32l*ob`_}ady^+W3m1|jt+nNAK^oV9(DN)LzCdYhy&sZd1 z(mB&3>rr$wFiN98$gRFdw7I_bA!@Miu=X`Rz2Jizy{-AP*f?J{3d7LL#!yNXW2O|B z?NG=Dm7+Dvl1c2iVZv=;*dsT5N*Gri!B(TFlQshXeC0@-A2+0CYUJ(qW?6_kdc6Y8 zwaVPt5#MHK$jg6f5YfAA@ho%4#3{qd3@CuXD$aA$xFeYoSTXYF4?RXNR8FlZBb}d*ms|wSPkJEs+u8VorHwN!^~V|lL9m+ zZMQk<7H?v54%zQltybkeLJt2r>1~9}2JJIno)nZYZT@0xJEYpipA`JgC`X%C=uy6v zsIXeqP6P{qQ~NsqJl^F_hoD)-Ck2mE`MZbgBdI3sTf>!l)24qCws6JWhmw6q#8Sg2 znk!tUkG%*RF2pyUua-dwC8g#Vj~$MENmK58Q``M6TffNhFm>9z$X`uJ_29h0Lz|i( za3E(WFpkF6bJYDwL{@piOizbMTxIIgRY0*P3ENH^D;k#N$J|I94h zEK?YSM*IyyoV^8@q^}E}F!3|~ozn0);Wlkk1i#J7PyfatN^mmx++Oyu!s3@A&J$48 zCrka7pq=@q<0!)OftQAldJf>o_Y2n#0SEme6nf-x1+r*NZ_TCy^86gL80USOV>tE# z)81Hg=Okq4d`D898m-?vep62Nij zU5Z`-Qwpg>pOFG`yFYIn+y0d)d}&H(J7BNqoA&A>P;NRl!$yLd=Tk zWnq_{6y*7S?a;(-&_Y|*NMnTqgC3D^W26Jwx>HeDZ2^%(yG|yb#LZ3Tyta#)tc0uU zMGB_i?kV#IHp6zL7&KqC!R90l(cl)k(gL6XOrRjah@H*^gf1v)C1Y4v;H5qJjM<6g z5m}_DdQ+-3>S^Y--lnxg?<=NTNC1s`3)SvLt99SpEn-V82h8>A-QrfFkMg1`dULc8xHYF$Nj| zXlTQm9l%{&@6kIQwla5o>0JGKqVIiYO@-;!W(W>U#toP8Ws3t=)cM-e^fSQ3nt!+2 zH0k5pzms;h!IOMFEuL_xiTz7nS7kAmUQ|*L=#8WZ(1PbTLtQIgS~RA(hGVBJYn6O3 z%hk@(>Ga~BUU1dT`e0^q5dkUFO1lzQ3!wyn8$^cIMb0Tdu&SiYd#BDer|}vs7}?_E z8tL%V+sWo9HSH)ZBaP{{K|7gp?y#IGxj8b#1njvr-zaufV}V0r1>;^CQPsqUzzqi+ zFMAY%bFu0 zzaY<{tm^*nv5Hi9AHHJTn{^rG5wyk-hgrj&kFdSonN0_X8xu`p&KSSXa49E0s$dAm z=~*#+1S5;Y5cL0gC-8`Uz&*>?bScb#O~>$&kx;R09of-GH{m|69vrK-5Pt{-73^JO z!<|QP-C+&5I6O7K4JO@^+L+qx8!R0t9Re{r_6>mqt@8Pcxo?|;G)$lBH@3HniSFEZ z((9)FUq^K3K3MO+KK*+q%0xNYA%g?=R#&d=@4fc$cql3PDTPrLV4K(F{^qc{-zmsy z1V8HjC&|QOBX@FHRg(NI4J6cMaH%A9CuU&gmGY!Lrh!9Z!L?-> zrp@yG&)BAD-KO&SkWtqwDZ!&bm|CDR!Jj=TPNuC?YjLp}eM~TsV=OLcll^jI@J})8 zXxudF_qrRM;FrKyJi#o*}Z&hFn(Jgn7<%mfnjy$5=yroB6h%!F+#+A7Qb%sho_- zkv3G0l@L74jTKHbr=PWi>B}m0C7%+#pxrUbj&iPe&ag(iF&&>QHm4peHYa$7bw+;1 zc}jCCdV0xse+wQDzY9Z=K)y}tNBGoIBVG7THca*=+Lo)r`|YVRW_V;F|H6f$Kw7cS zAOCgoj#ez^sw;AGG)KB^=#*6xYkN>q5>i=OGs@$~em-bo5-^Y>1) z1T`-Pk&z-f9zbae$bzC;mteD>1_(q@;awK@EX1l-{b2#oN#P6a)tUaS4>e2vs7YzR5p zz#X6-f|lucOXLkvjowT~xY6cCXP2A+g)%g#^3+{)D0_FV7m&EOxgR%ib~vX-QAdtU zX3s5&scjCBWb~)JSe{ST@zz7E4rrHE-Zk(3)j}~Uvn>)Bj8nLW4%hI^jOD|rwVhaq zDWMuKGxCDNS@N`pDHkTp_1>2lB+AM zZ?~I%Secj+Fr~Wn@#sF{!<6bh>({h^X4zmWjEI_A7k4S!8twjv4^<~0b)HHKpu+*o z1Rv^}K3>n>@!{C|?;?`9vxolmVp$txzE-Hzf{ZcnQCXwH_L7*(G@Vy|#`47!Yerc0 z;zipL(~^#QCTbY__{EfT26Rzj0x>s!O#j^;=XgTq%;hzNVQ`$YO}jSl9`rkJIDnUs zT3o(Tx+pU&@#O=K_U=_{#%JC4^4&uPiAol)4NJ7n5N=o9JKZ4m6+b_1iFo>N9L(i{ zcXfXv7dVRmMGd|2CKh>qG;05!%v-P+3|eTh@Q^)Dt=_JcE<6gD*-PXOM2a7IxaSE7TJg{1Z_vCnRn4gRD|W)U|40(PvA_YN4azE zeO>dtk5h)ilwHmA<;=Z&Q4&y*9r`7)c99Xj_6iEu2YJ3p;&ISwQ+GYjLg`n6#gyX= zo%53=5q)$8ozqm=Gzff4d#!(JZ1#(O#Woo#GmD@6wrf{;#@ZOv-+41NbVOg&ISlI z(@f7tK)Bs?jp~iL&$T?`$e|@nxrDRU%vSj$_O6zYLV3Mg*qU0LO_#OsG@H@js*$7@ zMWMg-tQeTl&W~>PCb*j;UQU+DuIi+MNqEz*sA*)D4*n>?-@YliGShMRJYvDNHc4D_kUl$2tsv;TmF_D$h`JBT;A&wP5xQVUn_NyW*7Az<9LZD}Wv@vav6 z@Qj1KReYWK9aMZX4fiww-E`ZNai6F(u8cePfBg|uDBZT83_4aT@1A#LjNb((K^G)=mO!G>o$h)) zf5<|iG;d$zm2vu$)&E?h|6(m(>-)&SBYLlkJpPKtcwE5?n|^?awF#Y5a8=0vw){dn zV>vs03e)|4?E9)J!{rW1@jn934JWrhEj1-%_*~hxk9;N4)q;8oyi!za#m*zL^TKb5NCOv)xb_}H##g>Rgl_g?uvDaaW{Xy`%A&z z;zgQaiIxxMTD!kx+FzP>beel?3fBdIznNLyHjJ$kHh5w5c@YLe8aDksVAm(@FhXJ8;1Lz z%e)tiQh0}9c!`6p;3U{pkK7IV!2frPDqV87+6Vr>SIATk$8n6b`+#vagE1Y531vK; zbo_<0Dct@8Z8xkp!%`!4f;i|VD#t@>?L?^fatu&J7~A5*hwm*h_Y@^I#}Rny(Oo}7tNa>+}x zm)uCkt3vE>c$;YIz!&YCb&D^U&)T}-^pu>{I-orz-L4yg5$JdboEM*-0dc*Yu}%uz zBk7U#P7%L36uL3dop`t2?r{#^50@90S})-5dPVi&5|~;WvZRQUW_5(Zo!p? z{wqhe6mP-cQY6xWXc8X>1_Z+Zoj9}<`FEZ`hbvJnhqj6U`+VhCWc(@qIn-ozx81PC zX))lWZX2TxmvwYn(E;1m{-*#E7XpVUYd(TIG=MN@i$ldOCrE3ZhqDY?>^-(jUh;Y6z8Rk|GS);^@*}7@uB!gP zD~&0o|NTf^yY5ih_RlWeO0l7PPZK>te|72hZMq~Z*@d|OiGpFmdhLacy%$vjPBh=) zPs?PpQa6 zN@)`2Vg>6n9qWPeVIZcA3b1{9rto|^HVkdYT)*A@Ng*)@#T2Tjr4xwq@)$6i-4SLU&s&!=$ zj_(04WOZ-g!NP3-ioi*~g_f%Yzw+q_N9S`rf9bO(om26|-j#z!gHiMPhs=;)2d+cvao(F5FcYI6&g?ZeYQh#0k;_n+ z$SLa@QIqdWX@3-k?`0Il*_AW;GmIh>!x?Eq@76Cee%SXPLlfX5@T7g^jYknwum>W> z$R~1C?KB=!f9KAJ)S%Ovj#!Y#oIs3%2Eh!TSEo1gs)(G#mJ|i+B*ixAX{#h9>lE|0 zTGpGJQd@uc`dfqmD}qft-VaFp9DP3CVd9z8hIMpx6nwgw$@$BxYxlf z2QIg_Nt4QQzek;Wwk;R%kM5KgT|0H|Yj|mtKbT^cGv!HHZhyUw{loadP@ty_Vzp^L z&3o&P;&{Ee3}V_Jl+A5_nDPqk*cqc;pDs)D#+mI-tM^C6QG-n9#5b9@KOe~HH8vdLvf#CJ*j06D@Q(d_9v9t$qiS_{padb$NnRVQ8;2VFR z{U#*tgooz-HUX~O-UVxeTlk>!=ON!y+1xNj0{jAMAeOER38dp>fQj3J*?8`Qu7%;N z=!MSRC8tXNYp|X~;tzrU+JsLpboTBQDibhqOE5K0tHZUI_-VXL0y;Ra|vNSAm%w zgVS1qE_70CahR_vrFy@sz}!yKY3)728_`flJ#^f2xPjO)xpH$)bEZmI{MC=3-JaM6 zdwUmBI+&2piisGI4NSE;tl>M;z{{I5a@>FZjnoRWnRu#SA-Th`@pXd7bHB;p`mAmZ z)I-Ao|6u+k*}m~MGysBi!@56@N96ux$Vp&h+2*I8z%xA82SAu)&x!BVzt1sbvQXkL zd|A}kT}y_<3NCkW~`uqqniP9)Y2MqA}gN$yaos&TwUX zHRfhII$XOL5s40zw*qzThs)``JBm^albdC=>Hmb#{h_~sq{*b!iV)~VlAfQ`(2+~K zinr0YtwB&%dukJo`jGYD3rrbsVfd@=3SO>oP9Do8~ zTZ0oE-Rx?tf<4|H1TLMs_jo3%zO>O#nn#UN#kKuWlBT;Y(2{xI0d^gpeG^7faMdh> zh&i&Up_mID4cM?xjPAcXO7CI>N=Eg+BoJqw_uJ?py#Ine$I-;^jD-(H? zcWT&@cVKG>WM1c+mT7KS!{+i4fo@Pq@~!c5?_JHw=hkKRXJJz7r4?hszHLe1m%}4{ zC$u_-_tvZ}ya@`5KJE*aH~cY^yPzm@zoNV$Zg}L|iH?q;;+nOk_a8IGbPVOz)-1dk z3W_rJoy!}dhDZ8O{+PL6-jFgpGH~LdW2n5gX6Y?iP?Wv@$4oj4Z_EPww0(l|2LEB< zz7uSnnLBHLg8A)4R%hnn8l9y#bAkP*{mAl$H^ahz%uL^ZRNnAvSh)Y>rq0a$wL}YV zvI6@*WTM`9o9RX@f7z8gh_&ULwF{R|a5O|aNrzYHXXOr*@e*yBKMMJ>mKTalXX0)h&&9<{}UAI|`O75O4DI04j;fbn-gwAhTZ-*jogWrpr z3pp)yJx8?HxKW))KPv*8jfC1h{U*!u?~u}9g~}&iDLKaOvgm$(h97J5@h$h$Kg%q5 zS{kcRsrpreV@!cXH|H5=tWD(dAFJtpk&B>>n=Olsh`Kl|KgJnH;PF~Q)n zq7i#UO`C}zkp1DQ0Z4bx$7tlQr~cac{sA?1V+mgrOs|;U9ef};3m0~fb2NLnCv+jAGp1HA5{ZGDMqf|~ZS6!xJ``52LCvSsz2QW8%Uj^C) zi?rTGbTK2WD#0_st(d2deM#R!tp-B6*#QmEF3^s#@>nwwla;CzE)Qjk>RiA3c z-1ZlwNAm`wIRWdWVL0o1skS}KEMU8rb2Ph34!twQ=Z!kWS5%XP+t(pVS|}C9Nd6<| z|8ni1H-VALI;B0&*gr|!vG%$EDQ_@0?51S8jnqj_I*qem(QdTa?3l}$a0?6Cg zSx$7n9q%tK=CG$_&m1BfdIR5&-OP@M8#epmp;%7;`%ZXg4VxiBK%=B*ET@V`@$ml? zW8SbG65&gOx_xGL+IbYe&HJBW6u}IeAOXIYP?ygrPC1V>;r}VdoM9Uz%$EYiai)Hn zcci&p^dDl(8n!@Q`VydM&je4ck4E4{{~^Y_Arunj%YYI)b3R==8rd%T&oF$OP?u26 z&Cs30K%b;D*=fR)CugaFK2l6rNX(NE`g?xuED1;iy) z$Du?l?wfw-+l^APlr`Mj&2sYTosZvf?~7mIJhtr9IFk;jT=B(y>~_s&*Rjua2ZHC3 z+>odoaQ3<(eum(L`NZc8*+xYvts>#NeHb=PDJ{>Gm6{&ir#v9HB6?yYb5rC-A1)#L zp4rJ|@7M-e!)*IL{5HL&Yr%zm>NT<>E7-RELK}TN8~wugDFD|XJ$_~4Qf3?Hi+6VY zc_~BLm|~xp#j?o6GAB|5IdOS=BpWnt5gBm25Jqju97;6!Pe1YqpU!e!ma11IT9#{z z3OKy%eW$nin3*s_! z^g?nPQ#Ry{ujI1*VpZT7CDQif2_p`YAzE6-H(p9fI#rr-k7$`{nQFHe2T6*&?fq8j zL28d7-bn=1@S|_S7B-RtNsAOi?%r8EZFZIP;Xy6QD7v#)aP#6sS}2iuOpmF4=%+%~ zt!y7k_^_i?&liIIv?4K-^>M8Xdj4t{TprI#KD7^Y*&oR8rSS5RI`S1=etf3aH|JU2 z!@Isv70~d`S9sa^#4ZI3dCi;G9t&xI{Jf9J{nRvi#x*7NyS0pcU+N;TIB7Rf?QwZM z7lpugPoHgjeR;gh`iym*^I3LZ$V&Dy^WMOoibs@B*684z`HJZt=;VfXnAD@xkG+(1 zhNmx~GoNPCXR>BK`rbvwoY5c+&LY!zDHt8RH@v^jaG%d~qyWlNvuPt8*kp4B9@_yW$P1 z^lp{WKV`yT7g)pj};;VRrkM3Vb0YHp*_ zo@;q0$v2HNi`-tBOZn$W9*I2EtvGI`aW*eysfPxiJ!I9k6hfS3K~a889zIkc#bQ7* zTnnVRrTTxt|I|iEf-8ggw?>Kp{}=pEZ3JpL9JL_72tUzlPw>4hP2|>v4A}hQmvx4-ix{;(+$m+X>>mai%Yd=PuG=E>HSgWJ{6R1*6r4vi2}2 z#}|3~34lxPQZX*lh)-!oBVPfT=4Sz-@S>|Xr6y{zn33<|p>rFnjtVst*e&=^39{c& z74);gTF(1NTh%i~*Em((5LzK20O1ZAeV_ugc_KIi5k{~ocS?bid zij(nAxOO^Z?`vI8OJzVL0*#!8lwv(+`FH`FW__)@X{n5eTA-2R5Lc{c4_}qY=Iy@L z)ifPOL>bV?b*MbnbCR!0Y*VnWbu(?60g(YTavrjd^<3nu0&KeWwZhY;84)1hlEV;v ztY;UWy~rj}Un?YyfFN!&m++=%;kaIwky2tN*f-)i!Pz9}VT=iCOTFfV#ldhPgw&m) z{$%whn|lONqqogV*lFa5j!Rns z;&yfw%ts!|C1yN!stCKFzWKOZY>9_VJO98xsP{XXJS`j1acwI@XlGY#`)EVU#f&FU zIXBDGs$TyK{z1KeCR<1Xcwnxx)bdbe&AQFfXPDBuM3+Fbl?r)y&}PA{eE1A(4XK)# z>s&tFLm{pykoyu>0e#HDYI4>D=aBsay)?Kgc$IftuYX!C=&Ha2{`&B8&pK+k*k9(F zo4f6c&+zBes?*~mL98iKT{#n?fPW4!0>HK`>OaRUZ7?a)`vL@@*bXs_9lTF$)&EU_ z4NG-E2Mz{3-hWrXjirZIPik=oESnc1O%OnZ@e6?s{4_5{nm#}X!xn1}ub#}}*8y$- zF$N=+7@i9$j0G%{7bN}b0EFj424e-w=S4|V2k>ARV*Nsp%L|hx31GsI!1BUFlEUtQ ze;o*Sea|KFij_?9Cq*7QgbH^Rl}Fi!`gM-VNrgzNMF9l9(2f}VZMc2-U^vjoWix8T zXDppp3?Xmc1{3V&Z? zS+7YZYE;mk+ph%n=CBUQcP8?;i;K0VUNTx~{iQMltWzCp)&2vXU%j*3Xy$M2PZ)n`+-!bNf6!H^>eUXTvUsK zAcwqMHj*|M$SEPt74q4VuCu-u4^}>S0KNqA-_E8 z!7wbUhcb$2j&M$)FzzXHj3k|}z?lVd3|2VyYXQfC5KNQDCk=epixz`Uhbi#yzyu<5 zwUc)(P4F1=4NgB{JS_{q^xJnsXzA#G33ndLy$S6nOrT}ufAThbh&27*0x-OnH0BPS zoIt1r;TTN;-;=)tSzvH4UJMx>o4`|xXfRDa->-#Ui|Csy!hBDG(Y-V=d~`s8Ukq3= z_;eHk92Uypv^+i;U|{brh5|l$U}P_4%mX@2ff5Vlv9#a8*>@p%;I`(x0Qi0^?;qCq z-73EY8{Uf~htR!{xD9^_abD8*ND9E#X>+H$R?FdxYlXo^R%}6@C@~AGiCl%L6yfeD(p4wKqh}SEq-0 z9MxI6xOXs607fC&NzRfk-h$~vc&?1b5UcmLx{#VkvNOxG`w|Md+$99-T?PR@`J)qv z9+v>^d|Ka{G=XZ)`sYzCSvD<$1C4)O9Is|#$Mi%Y$-gL9o!RCR!{TFMX{r{EsHF78 z;a$78?))v8vx`IUw{F69E&r>@&fU2sbAE9FzUe0Xxutn=c+c*?OmKiV6y7iDzGq{X z={%n`zqud;Z4~`H+-y5y>-?*!m8$Vr?WUvbDIXB5;+Qt*X`wPwIlf7|cC7&>Ir7H? zqZbPGd*hUD^r} zjM@K&CQmCz++Ews5r*0O+dewbmdYgfwvT&$)6T@dED&4%UEO~rHK@1P^2BM*2+F0c z6d{qlZ~C(s`p4Xs&+jVy3GD}u-1K5M9l9UYO<%s(=@AHK$&Yw~76nBH& zM^~i^eXGS6J1Kgfzk~ja%3bv<=B*ZA?lkIso&;4)**4hy2?4AYHFje4+`oegrx+XH zA6ykG{)DWJD|{P{C+BqjDu9={>QuB?Eo$#n>bXyX%BKt){){jih`)=e??_kN1hN&m zxZGFeKg0iTzX^;yn2+LQq^KnG7YO#{Zu#C{#=99YX6HdI5@x+2i~Zg*xp=t`;im#F zysiExmSFcz<3w-BrCk1gDC*{alF!3rmq-1nT$WEsE{aa}GvFa8{Cq`^Ty@xQn#ePJ z{!rD<=%1&|HS}=lwIaWs52zVBF)Wd5N37y*0g{_w%z%Sk!b$)Fi}_o>gcM2uj~5fR zVwK(BfY>G&)8YLtzl4AjjG6Emm%b8!&|=M2tctr0=$BBKDJU2FQunIx=Gg$r1}~Pc zz_qZ<=vd#8G0vrdp9GBJR9_}a4V^g@b=rOllyq}`0U(PfVWz#Z%S8K7DMEg#v$ovb zFjNY&7Zj1-ew#UzR<7a^FrQ>XZA<}Hx>bl|@NI-~4$<7~jK2-jkrZzq3KaIVAW6O` zU4vVY6#O}9huzWl-iq_BMbN4ZdA#oZRW$5j)vL*$lX3Vxy7aBM;Mzx8wO>OQ_lJ&n z18**GZXQ@rP1j1@s;duT3f_`*T26`UJniCxT>L!dcIKFyD=!oNUTA!v&!OL@%E10o z_03lsUhZU^d|tLdqV302|Kabk&PE$2O1t@=7K^KI1aOjQdFFk?)Mol{B%O>~_Qy1& z-NUbQd(=Q%+*$kZw=;l?KU?xI>(s55tzP4Hy$t5K$t%a$squ^z9}P`?V!*(4^3&wN zL)Us*etu2<1`TtUthY}Y*=1Ak$P)0#BEsY2*;!d*-!7k1znmM~Gu-;}lHj|fVBwYgrTMb5ar3^nzbf0*8$q+- zR3@w8LSK=mT^~9%hBJ5bRLu84tTjz|)puvugSz$RHaZlE?UUICs)AUbh4_6}J1xC@ z;6LIIX+OH&xDJp>!$z1qe;CCLwARHxf42v8pJ5yM+9MKv??51@Pv?N>g08)b88hph z2pSGyfUJo`HRB2G5EgY{vWcJveJhD11y3N$YWG7vwTEQo-BgD;9=9Cj5=rSZ@P|V< z^Q}Hml_k9$mnVOdmBkJnU_W|2F0z%@&B2z0`#%1KI;mN{IOTH{jprG1QI_#S;Fom8 z54-|+#xs$`mkDk8*Cn|$JZz@AZ$s08RO#+RI-4oY<~d!`x>PU2G$95}q;~SF3A6n^ z>kpl1n`0N`B{Uw3-Bx*8H?{%OtM5K~B|1Nvp?iAxttvC5#(Z|pN&~VYb(Xei^z1~L?L36+H*qb*d z8oEc;p3blneIFUq4Eci>jake+j!x<-X#}r0Qqbgec@CcpvfnOOPmFvqo67KbhH+Qb z)ZJWyKT6O;MG++~&BOu~4_s#Xyx^o5>lxyL8p0rWK{?zVrp5V$Y#zXWiTW}9g`k&a z|BI=_=5L>e zCXxj;69l-%JM!LLn%ZM&AN8yIF~PF@dMt(b$Tr3HlCPhB_p6(?U(%)9`Ls-17tVY; z!4VbL=WB-bJ%E4vc{$6nS4FMIT&duZfyZw$Ft*`rR&ez5$ z47sBf`TN}0B>Y5Zh|doMSCg>aqf*7F%{u#jD9<=wZ8RqiG)JD=Z5sSIS;@1^=;!7X zqwE2R%&f&Y(Mlz~%xG3V(KHmhGN4S{j}DmtkTI9*rcgC*KTZg3Q9jZs1zr^-7E3V_ z?;@&-Y7|p_XoymYZEdtRY~E-yStc>HdWbDyW}47lY2$y3?@w3WF-Yse8y5*>=QX z@*(vCZ9>0n8twE1*VmZJxh7pK*|WQ3*#3)nNidPM4?jZO6cJFX5@%|2r#2^h>=i3Y zQ02N-ccPkn{t>oXk+vniV8HH27FP{vTb5b$^!LkR_>*3PS8_!I)1A6w$a@%rbT;XO zS%-uxA;FJMVn5deaojO9ol0{LZ8ThfbSL)=_cg>8C-< z%GYRO8_61T_i?|)H9IZ51&vVH6+b)?ZgA?5y7^x8ZC&m~V*AH=`nJA`6xnFgcb-dr zWVCogS7F%ni(i-#mi)Yhi&X#OfbSn|grWKJ8A2XL?qVI+E=L|l8 z+iV|Mj+<+(5(lZqKxIV{*yEDg&d;rzT~sqZ3WfX#dC_^GoF#$c&y zBT#y)I?!~^;M;83cJma!Q8?`i7nBp7;TeATyFiEUPg}{3$(iHv;@@Q{wNATc-06+L zQ7U%xDMJ_tlajwL5(*{ka$g;*u2uIfv}RRj%~o$_zv9)bmo#_cJ<eU(DwWW5F~ukLZ6B_O~iy`aY8diA7<#1E0cJqxLI<<>p}O|}qB z&w)e3Z7%WTf!5APmHRakuRrBrk}whD1gUvu@RkvdcPGC{P823r2>78=o`trg$LfRM z|LVRJxclhN_SqPu1wSo!iZ%J&6<%XbN70DPk5!(qCv@ph58t=d+cXe(_JMg#zjrRJmuAz(eUK>`EqPLsmjTfWZ+6T{z86@ff?O+DPr>gRF;zxJIx<51yEfmX)|EO`7J1|b*`b$b z?MX9WObI+AlMh*9&#IFt=is#nOEUZJ)uRfzYtG9;E{RuFi{%Z#|~qyzGIK?#S&eUb)JrTe26mwV#RaMw^3M_S#B7>!i*j7bxbWY2Xp zaZ!X4Y0r_$wL<$Si`iOeck`En;dg5?>+DTb5}1d;{LF|7*-tRvFpt!bt(UxqKTg#; z?8rAkRd*{VU2Mf9DLOwC*ic)iZy#Jpd#9Pt2SkdU8n#{DeJW!iu$M;?kNy^spy;&N5QKVG0EMdh)_?;J1JBuN@s~Ba~npZUvo<& zvV7}0jGxcm3n1ZA^JA^&ViiZsrH_-6PmhMVv7X#|Ci`ho;i^brLOjl~_cTp$n?&{g zCZ#)t`jO!aIn@%18hnb9K_Vr?JQF*bqxuYK{K@C)@;2sI#+3zzB(xGki6+W+l_!sF z_rh6sraT4r=RqZu_?tV^yrYMUBZ#twWHQ!C@)+h-(Z=cYnheeWMkMLuucM`u)~aeX z54GwfG+)@>HDgb;$M)N$&mVH>J_sR8s-gW4*%ysPMhB5BoGR`%I zPmgkmN0zM??t2Pt$}iNl6fq3RVAmY zeis>7P@ww&!wuWg4coiw-6C-R3ksMm%n!$L92v+yy>35WI!9eGy96LE1ku7E{!Dfa z{)XV&=e1lsohF1bbabAl@BjoDBm&WR5pR7D8HTna`bt z;Uf}F_fV^7@Zo@ku%IShTxYy6Cn@pnOVMat6UwIuX{KN?rf3~U0g|sYDG*E?KB_+T z9ub2WBAU1=K6IkAa482KIg6s+M{LoLd!3lRb$N)yZjhw~4~P)=tzj92Y31|Vu}w;nRnlPslMmdZM6{5 z;sAhm6nYS%*3cKCwP8UVK9LYu%uVigPQExGsWb^gM4HZA9ZwJy5Q}#=m$@;qS~@$`@m9vhuuf)$(nc8NFI> zRec(eDf94Q4iJ6RS!`Z3h+j0S^gt`8cS4kYI4g*{23x;|_(9HBeordskxfpQXI31E zC(q~_Moyg00lCp8wgn`_pg+okkNS|@=-c1{HgDfkqkuS1_L9zSfqK^A9)m!_tJN2Q zgAR}9Ob$8Mo3gl?5_J%CMuR&C^i@!zRx<2$zHH(AY+>iyItZ#th+gHI*U$>iP^WTM z;DiIG{f2F8w)=tVQcsH~iR9{+g?^RRI1~b1vOnzP5i_LFCpr#+FF8GkZiRWDa zjmfK9?%s+$l!Iw@SY=`jycM-wKZ5Hkv|1EO=S0O67idq{m;~YKN^6Xe05bVkPhFG& zg0{FAY=G2cz0cWtygC(`<&8n#i`}$Lw9H#G*(f!(kx%Z}dvqz9Me}R*Hf4*}>CG7L zn^m~i9)Gfb@a6?b+d^e0r1kMgaWq#!Fxxi|bD>99QtXo^%IRR*k2qWgPoHShEiejr zG|*+~u@j}FKmTDfQ}P=4ZM>9Tu~5IDrtS%s4ZXP2}mKho$f z0;ubseN|akC)UmLQE1##B7SVXwz)Og)|&4Meb=o<8SUmC67XQ@s{`x(*KBM+<8_?g zYz9IG!U1f0+Lk67(<$vhi;Anj~>;ExZDkUhs_pX!q1D#7B@xQdtE`V>m~e{ugT_(iX%E& K1|xeC+W!Mlu4Ui= literal 0 HcmV?d00001 diff --git a/docs/static/fonts/LatoLatin-Light.woff2 b/docs/static/fonts/LatoLatin-Light.woff2 new file mode 100644 index 0000000000000000000000000000000000000000..b6d028836e469fba8fb861d9186259e8a693054f GIT binary patch literal 43468 zcmaI6V{|56*De}$Y}>YNn|Ey6wr$(CZ95%1>DXo`=_I?K=Y79@&OT$DJ?lqdtr}G| z>zdcpa#s*%1_B29E0r@qNPl-AN{oMxO@Tl_*Z#Ts{}r6TRX^NN9dHAnSRhDlsB{b@ zNa&zp$e=hjV0I8v&^S?)kN^;va8e*J!XNcw5XqddfzaCU%!_QieWfz`1vBfB2eNrc z4c6;go3$${mUE)3wq(GP)$bP!O$8?rIiq8*9$K8sfN?}RB4ELtD|d&$>rEiOc6Q)+ zU_l&2WyykMtZke1 z5@$IkdY;s%4VzLEyP1dUDhVQPH7p`$&d>xk<(!Ij(!iWuJ@pvGlU2lGgYA4Jd7x%x z{84XED0uSLMx6}7OEygTaa(hCXa_5g%+VNNlhuR z^=*pyQrv>P_5hiF=>j|3*p-)m@D#iyt7#G2>_;0mKzEZ+#FCC+#nW`>omvCAuEGEm zCjiZ+b^+E78yhEn0*{+zuSOnc@QB<>Rns<%J=m!sDw9uMdD?$Cper3M`K(%x3^Of_3h_Z2_sNHS#@N$PIz83OcyAt_8m={vJh>Qwiv!VY( zIPjZ@n>P}@T9=He);n%q+@)m}<4P0;9U2ykPF$SMBP#VrDS z$fN8 zU{mrn{Oy9_OcvcGmJ9R}zBZCvS;0Bf279g)k4x&ewh(iC0_GhQ_p_>*%|aPR2c;=zu{t|%Kd&UUcu;vN>BA`9Iw8!fk`5Fr)|kwd&-(7_i2nzTpW zU-MBZGd5E{>hS^Cm+=7&xdi%c7g3G;sain$XmP)Yj1IvkdTjwb5>c=P<^ zVHnc%#mmABEOm)gr;@NntIaN5kZ*7@!edyNn|29R9kD)kPLiWp$JW>&n_`PoMAI=P zkC+FcFO56Odbso`co}hGb_-rvz!&=A2CG&Gg+YHnqXo{KGflU*fi5=?ePLh40B4xG zSr9@2C+6${#x4fR?e_!l6Iuq_{=C2Ti?hGVjisabyY9>T{rqdAHO}iP;MuS@x*USC z3{eFEqS@f*)Ek5a5#?djz|ihNe317ZS3_mnil$1ArNtC+PjP+7AEt>nP`d}a0qG*K z-?(QG^e^J~HVPDY^-KFGoS0`ES{8yjF@(S0SlE!U>rj`=_@vGdO%DLmsvo_2xhq_Wn#)UnzRQ+p?sU&-$q zI9esX&6#d2*Wc%(h*!b=yV+0H4HB?p&t`%XN=;=eq9{1wJsvri6kcE7dhZvp&WSOp zo<(mI=5Rmln_q(w#TpwL5K)WEiYjQ<_OaV6f1R5hEdqfR)^=2`H#*64>^O+DX#=_> z!|R*-C{Qyxlpu&wq9_0>XN(_rdVp6}pFq**`eUiy_5l!AF5afyJtR zBvPZ1jto`kWNBaObh#YcId?lKVMO<7=WIQ@sdj%e6>X4ve^v~lvPx5ZeRr;n?3$zO zc7Rz~%xquD#|rPOeL#Uj`Ga^!L0ee^%?#T=`qxkORyZ-BA19qU-q7_&(|Q+vu zE{HqQHySWABGMBR>I``APaPwdx?uBu2n-PHRn1+tdcmofgE_EM!B3oDWn1N+0y@uh z_;_w->Ts+5Q$_j`qQLV*L}Ms}Km!&nsRBB8!WGLh7!)BxB_y!bzTON!L4p@hIXD?k z2Qe^#5uD93%SZy19kU|yD!%`?8ZJ2QI@9|L=%#@Lp-}C@OOo!(8OTP;B-6fK43DO* zrwLPjnF+ABeJ}h-)|uOyGEu%t6igGouqD5C*a~$0o|F8w4(0d2Cr{J}k=h3GyF6iM zixqgL0pRvH+e@9yp;>OebwLznP!rvK7%vyX2fO!8(tmln3x5dBpE*@88p3dOBjeHO z05VY);n(knZizD%Ykg_4lH1x=|><;ft_qA2n;WrY;Vf5! zh~V)o)aT+w%w#76br&wHLb-iR#V;E0cQuwf3rIqPb~q($Dl5)C4DYvuVU=%NSzKKo zb7ed`~24>sc%HGpN(s8ke-!J7Ut8Y5CRT(K;I^eUBDCh2b=H>0%PRTCkBngA5 zELtJPUMY~W2F*1pt8UWppkjzf)~+x+4WwNOCa|!%>N#(YjUHY~(uP(InGUFAEv0?u zoNeoaF&Wxyu^Or&(!X}uB0J(O(Hq7m8DE}CFmrTl{Cs(2d1P|`Y$1zf^K?&aTrf2< zBsm~Jt%>zkOrYo2_vY;^#)!?Fjfb|HXoi5327suR4AgQycmzPofPqG#T)oO zG%%*sEEza-Pec?*TYf=N2_RL~+(^#~9+&nQ=-j?c1{-5zuo0<5OiRne6bXgd5MA35 z-J%iQW+APt*g@3_ns|GFvI4V>;7O)mKoEb|?Vetj(!CwZMpj;!C6Y>4@@pxJGA_&l zD%}GrBdf3wGjA!{#ZlWrOx@#$Mo6eeZDXkR{WF?@0pysOgSfeexJ78x80Fv!Ktzx)~Rx9=v^cB9yLk6#q|A5#B znPVlERHFpRXkzZR_^Pl81C{(22~THiQZQ!HGxsEj*=vN;j9L#OX9GN_M5nhXEj7?Z z-&Niv!!$!bWxvgh%)#4jy6PhLB@VlR^9KB%fZA|Qn)212+KB|M8gH|sgg&3i)AyUi z#z8t1<}|1>pUHHeNGzFx3OPYSL7=Z-u9Y7E-%sYDW%1uyj-WkE(&E>^>vUE}Wn~cG z_TK*W`3VY=A)=f$q{+!5npd(3M~VmGdUl?s^t0;w4w!GDt@#mdj;mm(n4~*%!};L8 z9r~N;mYO>lbI1vL0`pU4-}rLBrp zRi^FTGgYwE4>BRTTTz>uROsYRe|}J3uYSv`T*$D?cn~lY`Wl9wk#0QDZD|@SO6li0 zy@*D;&IlQ30@KolhCV-s%RxG-F_q1akg!HwUP`|6H(%w6O*}%<;e-e;V$n88QhR#o zlq{a%ZzZ6~6avGV*I}6QFEL`2UJ)ZgI{Dcfsx}jH=x_qjn>U&Hs}W!q9|1Y>rI@~g z0_mDH3=s3$Zy%0lf-cGqlJ+XF!})!*brANT9*g;~2_l$nSxd#_SvWnG3kcF!6HUya zK$`&J>rg`vu}U7~91i+MU7Wkqkpw}Gff>iJT+}&U@UML*?|$D(pYAqs<3CqK9m6yXVJfrqdJ$le zGPjQs{GK-Sv{nj5rGke^BuQOqq79DF@DVSUGRkNf_l1=g?sm}rQ~D*VwzzvPLp3YK ziK~rC6oNFyQy1UgjyFsaA#z~l)!_<9va{IBUzTpLV9;SQJ*6(&C^+9s*ciJwr(Q59 zP$M<=GlBxJT!EO@kfJEe<@hk7q&DXdWnf*5P?(OC!0-Kf)O_9Yew1B*|9(_XAeOu{ zXjGwIKYBfGjHDGpF|m+qpN$B??B3|5Y(=w|4@5bmVK@Lkg=NCGrg1ge2i^81>`8fL zxL=Y)50q;NW%qe7Gre}Qimc^{lQj@63zfT#;pSL>V{59lpf-A20$FV5&%{#hsvOAY_&YOhs3eqY%g69=~S&ayZn@jUVmM&O;X6Oms9&U&9jptxEOm@Eq^ILN0 zI<#5ws)PQdY-x?&th~naicaiy4t3Q!0q6A;LwlgAH~<@NqryhKxK2!Ni+Rggv>B#L zSY#VLzxl)NTfnL~@lcwgwWrp5XfE5i&K^#LssExFanW*ytf0ze;4Vh@q6wCeD+D>u)@Jfc;LV!fTUyp>?+Z z7*`Y~Zvvu(`sO&%YcImuAFGvC>|(kxlF(F+xkjpaXgc zs#(Ul^$jQm!q~$Jkh3r*&q^mhelt$1lI63GHz(pNGyi-7h^BEnR*z*TS4*&uVy7<+ zy@IcdFsa@xc$CZL{DlS;@mH*o%sv&#nT+_BlY6H-MpANP?CZmuA-=$UisMwg?ibuP z^>Kk^kN~=62)h6<+mOMv2*0*q8$qa@$iagOqCC|XPZ+Im%H%TQm-nLFXYnJ3~DF$)#Z-92g^tlCO?)+pq~ z-0Ck*LJMencnT+9$D&`yKP~4XH!lkln>qk)X84&N94Pc~En0`f#cIgJ6J8O0Gn9O% z&<<4CYZg1uM|zGLm3ZnsQK|0cPO>T0Ro!_r zM}~Sen;L7H-G$v-lsl2UBR5KUORtvS7$x>?Up#PT$mpWo9*c|E#nz3KGp+qf6Y*pe z%9>6HB+xLZQwJ6zC%C_dj^Ag@p~zDl4Rt70t>iYrY!U}RWhoDXC`+SX!AT_4R@uqG zHKAu4tmM)NAjTPP>*3XN7$UvAHC@*eBGc~ev|oiQmVv3L6mL;fiegIvqG<6wNjk{rUVVPn@IesiK+P}ol|!Yjj;?EM?}yh_&85;zE4&Gc?G+ajqgAVlqP?N+l_vD+ zXt7y~R!B>LW^=OXH~SQSz$df=?jNEj?&NCX))#5r&Bl;06K6h za6`Ksb3!>m_0KkdwuicHHQCkI8R+Bl>but7KUOn4IA-7k8B>5Mp8$BW#ru+r>sVGI z^DLiI1H_GPto4puHPX0No>^X=|4e=5?)Cja#R(Bynkh~Y&4KNnPn&MC7~AA_Z<|KX z)`k9;oAku#1w)`rbtFq;T}iM6ysI!AL#{aCC(9!=eefqWA>NUNx?>NczJ>OUpT%dL z7TvyVR0xngKT^P*2_f+(eUitJx;)}b=qmvW9J9Dd>C#yH(Xe#kb{s>`(S@9 zX8K9ob|uWA|^VRsF!} z0jGkF>oy0>Ug*xx-kmV4lUD#zre~=ZXCCvT$MnvHirf>M-Xo73um`5kRe8) zeEJ{q1LL=<>b3>VN{1wGVuQzoTaF=*+YZ^QXy$paD!?>CLOx3Gn7CHtFd?ybbxEql z5pXWcY(YqpmO_IWPFN61wg|(hXyFwIlZa`cQ^jw=joGVhaOvY25TizrD`o!@c^ zAk}cLcnmg6UZkB$S+1g_VO04xJSzrEzr@1UitGMK>J+bI>6` zjl--kM@YwHyjFo!t(uj=9?If2B+*y{k+Ir|Mnlb2wbPLs@GaydL9*5Xy;M?C?XPJb zrMt+apq$ctCF8kc-r<;R&Ld)b4;@Tb#aYDK`!8e2mM5n+9v7A$LX1I1oI=X1!%jVf z(oIEHU4~k1hhF^O{nDp=PReT8S13+lE3WLOet!gO_oYe5#7SCQ+f)9LWlyU#C$UAp_W9xEIdLMN3SH)^;SJm6P;5Zmg&Wb z-HE#$tw7a~Vd|-w##SqO*Y;a>VyXUCFMiNq`&Nl1ijWv~m=RVeA<_h2t70$!<4R;w z`N{}Blpodi;ObV_V2xxc!%(_881>^Tz?&uJ25z7&3+l6bq1}L~IvkEYm17v8NdgqY zvFf+Yiw|Zqd2a0pgx*3nt~`^kHEc(F3YJ3WT7CE@93)?E{{Sj_I0d3qVsGJJ&SVys zwb!tLPam%jP$W&0G<6Df(nf;j!-QQF`X_8KiC|}M1UIoQmDtA`hbbbZxgfAVQz7P= z(m+Sqm^peFKQ2ux=cb+YY0W5ZZSA!tKmg^8)MB)$v`*_Z^ix4>?If3|PDRCR5YECO z8qLs~$VlZvr#(HtNQzC#>2wkcz$*L`J8vjIGnA1rFtj-?jkL9#oh$yE6>e6m#cHWe zq?~@o>*-?2exRT*$28|(v^K|NcX%SHq{~UA0*E*7o)1udr4ey&U*F7@E@`>DTwP!@ zSw2zPz+YS)FFI_*E*C5$)$g_q5s>wC#8y>!RTnRazx$A4b30s^I35saVXiZ&76%&+ zb6=K4!d1!j(3s{me`Ep7C`JD_FR>evb8LuuNe&4MBA1m_Lh=4!ItFuis*o*#n@RZq zy$x=6vxIPl!}moV?|z3(iB6_tX!UWOyx**{HBxZT@QwKXgr$pNAXJ-(LWPGQ zL3vm-a0?ql1UV%sQ71Hf%iz3OHd4eiO{Wnun1~XQOU(|CPO*y2fe%%`R`P%iE>qjV zAhLjLE`YC71-f6Fk;eM%4rEy@SUfuRTJ5%GMfX*J(X3(ZzR4o?asMW~y!WhG*l?Df zb$tF%a_F2fY$s>Os&P1x?>1%nhTpjiU8_}Bo>%4?r`>h{D$}g3^Gbmy#I7$3-3VJ! z0wAEVh#DOn*n;?0oPWDf0Ziheezv!>AR7R<7n^BI2;w>z5bc+G!ZL+QaYbzPW69pC=x7Wu^sbi%e`9tHMaQ#ASO4$K)f z|1m=hNHL>G0J%D)sx8TE#mus7?i)MI^jp;oxTAEZ-c|K-P~@Hmq_`q1g(Nk1PUksU z)rpSs&he-EL>Kk?Bc5CYR<>kG6MF(Vu9{c~l&-&R9lR5BCQ|?;%G5i{m97xAZOwGM z!=v8^l)=B6)zd~K(^O*HVXXZl{Wm_h7=IpnH?G;Rleg`6A$C_phifD~Z~MnLEoVJ{ zIJnlGbt8O?v1*#~!3+N9Euzdc;{IZkqs{FFK2E;gMu+#7MX!L8OYp!#%Q*CfQiwX1pWgZbpH5FNQ8EMxYc?!${1N;9O zTK0=h1;qf`QTY|z@shkRSHoa;qD!@ zVAifNycX8u* zNR;ZJSGPi)b>R2VtV~oaN0w^h=L3Yq;uOlspz*lK`jFoJ6+}hqA9^FB6 z`EsscHR@9N9}B^OU~>?lv>=1qM~F=Lsa;6GqlZxq4in%OcS3|-76N$QzI&TGagQUyq8e*_T z{rhu&+B+Tr-9K(o;S#^>G#D$}99W}DqX7v!Nb>(rSO^tsHCu*cqA5IXc4C5y@5Fg? zlSC?YlYjOp@(=P2%*+-ks)ms>6uJ|Xg{ntU!PJ>O-g9-x)pSv7HbS+`;0*_yW4+n!I{5m8sF7sKnL34lR=V={T^tja z`F}902Q#fj_dyO5$?NS{z^si3o%Ub#7d^80^U5TU%&MRctR&D2b;DjlE2&S45;NFgv5vy+<$Le z)Y4|qC#cUC;7uZ(q@zzFJy(%g1y4ersh6eQd6?YBgT70;`r8Iyw}^`PM>tRU^@5Zh z2!Z}N0a8>Y`_I#BTUUoD7G)dS*khT>`<>J%)%%_Z`u`Te8RHMIZjrOm@(&)4*O+C7 z`bR)cv|$>95p$zE={v1_bwsd>Uoea`@R7(%i(KFZUr#n-$(Sc1@bZiB5^_9`q4_)hw zv2;8y6}a>BTZqis)m&RP3LcRtM9P4o=p4+(xA2Mn1HH-!Gf0b-FuQ4s6|DI(+k`W0 z$rzL-T3c0Q$W(#-I*+v&IeWKkw?=+01{=*%s+Pv055j`_bTXB2O7&m!3x)i zF=YcCdA<(FSS zoz>eSPJazt{35jP<|hS$1Q97vZ%g7SRTwsK^Z-c|y@9S4kQVvtFRlmLDyv~`RobAuSx(nyg1LL?sVe<0E?d+1dsSq&1wSXkp~Jy4V) zzk(B!S=My*AEIwj&E*%_wP>2+hW_l9s8{2k{vp991fT-sW`$bci{ z|5MEQp_GKCx&*h>gtzu6MT5wR_3zMw>V*m${5h)KHoxmAfQSCJ$B190j;y4$_4(yl zfEl0GE{@=yLT`M9URu}Msb#aweVpQ%ru}By`|fOGi20HfhzL`NDTnZHs>`J}af*Gl zTl=Em<{SGd#(4e+0#vqieCWM&eUDAkiyd%dt$~j4E+6cM#vZp*3@FMOr`g@(p-3Xk z74=Rcm2L_vnU~tCdh+B_QH3|+DxnX5eIkK%YUxi0q7g=#G~c}^l?X};%pIQ*tDp*# zJD0mSP5{ijP$#O0ajeLy7<cHt)T zc9^^-Jk&05{};ExIY4}X9Jxcc3Jzq;-BJ5XShGr&{1N+5s5{Hbta z`!2C|&ksA%Q7GpE``B)u565L7k0}=D1n&#Z@#|jh8c@3g4C*G|hFz5<=e=2P+cj_m zhRv*3$srKP(+P6byhj6lHY4U7twRz*w(EYV`Qte)pvmI;Y{?0w5Itoc) zAhT-K8B*2_4~o_-V-F-tveTMDYusg&_Bnb2PnpSRx*4hZM#{Lz-e<)|S7#uf__?GB z7Ts!HDlYA7&rp@@>6;m~@sbWIO)i`1W~9mw74x)vFBy`?iC!3Vc2iepAR#sAUnMTi zH!jAD4Hx=H4$j}XU1@aRffE-#+__>2BEvZaYqLhwx#jI8`io{o^eGQ$(Z>Ulezfp~>r+}o ztjn5t=j=_c4I&5M>_e6gb3UOBB$CT9IY^<0knP_5_#m=K+iPA1{YN2Xloy~?{2_!l zCpGj_-oU2&Q_CA!5Cu$f%FVO?F1n+8I^};|0LOg%#h8x$_g*yC09{y6^`>@w6jV0 z^oByRh~tbHwfk;&P7>d%D_Lslf|ClNQA+YPXC^eGq%T+~2X&is?Xf1x^F;=jSrT@; zg_(@=fQRsv@b3bauqro_7Q_?<6s(&>Nm2pGP^`4v+4L_~q&&ZTmG$XZ&^w8+CylgT z6NGv3&*c-qx(B51xHC32xn-=tE-H1Nf=av&#*@gF#YZUEMT_#5HB8z>zj{KicMP#( zEp&R;y%)KA&tctvHkIT7Su)s3b_Fx7qAUYLktPV<;6ss0$|!>^PCtelBdbFM`Q1ga zZVREF8t8;)r3A`yJCa;q5qijF$qteErpj%f@738#U3eDRB`giq2^ znh@KG4wILcl8PcNtWK7xgCvW~#UwIh2@}C3EI`FhM^b3q%vH7Swy05uuvGTzVKUr9 zJ63XIh+-ASc1Ji(2l|TTC%#oSLKvc>33bowPmYm{a zRCkmz4sO+(5>5jQx@QWXSGnw0FjV z1Nh5C?0iZU0CGH(jSNf@OA-|oa~VEjTxa+I!S6GkRW zr^R2-1+<5xrpWs^)hNkuums2*gg?J4Uv@dOwfEy(xLpA=xHr6HG2k6+C`jz+VQ`K+;Sp-0Vr7*~Ot+=VF7p{W+}VWNS#X?klNnxdBCT_KkyG zAzjZ-PdJF!E$S8{22)>Qi(PY^tR~pv;HiyTJKkOznFKI+;Z0+0gO1cSKHSMlV=vFN zLImWKI29th#CT9Kc%!1|s2c{yS1k(_k+~=ABCex)J5~)yt1ETEwel5R4)buT2=L&! zUzYUpx12abKW^kJE~+exm16cq;*NKy#t&843!5iuaLks9_(^%lnojKc7_H=N3LCd# z>!p2^kCL-!%O>UNeL5zM@r8~GD7CC>lsPcUz=%rUeoAqrZI0Kfq=Tnf&LGOhnTN>F zo!}`y zTg`fTt>0W2w%&mLHxV4hn5;r565IvFh9+Nr1N9hwG<&_rX!z zUdD%Y)`dY5=f~?GiyhcI-$((}GC3?43p_k-Sq5EOgGD)|iDwq(AjO}2qc>!smA+e$ zCNozOL=dFJ^|{%r$MSlQtf3OIo^@P6*#Kw4A`}eRtm- z#LGp^aeKG(z)17>E=ha(j9hNDFJYyv3-qWrML7Uh4fWkqM{R3IG$Yid#S((K+^DjL z85XzFp&F48UCgzeTfEE?T))rzID(B-#oB>bCH_k zdy5=RA~^WrI2MwxL^_D4lip?zKN`pd*yu_G{PJq+C&x+7!p$Cv9oj_|w-xAChRXuV zQ{N&7;0hT`2&QL@L!yEq|8bAmIGJZ!`=e?ze%+AVFs@e?V05%ckQ`zG@9Dg}i`#5J z;t1Mwb{lruJMn~kQkmWx4tQCj(dzchePNN@*F6S7pOzYC0t6-8NvNghgDq^MMhNl<2|1GJ|NYMe}0 zr9}@g$3qo_01b=8&=5;RizZ8=_5^PP&K6*bj)NMFO2oY{XPU zeG5zC1W5F=+Fwkk@jfo;|6)GgC}iRix)tE&=k^$DX3+XRg!!{=c`O!H$8#zV#xAUn z+`e1JwqKdFiUIuavAd)Z>}&j+{(u=+*5Ff|;&!%mX|ap5q4F)inRq>Wq=meau_5;l zw;rD7^y=A)N+7fomnmy9Jn&@j;?zqA1>d!1y|=cki9s(lM*5)1+h3kuV-0ou&x6sy4cKhvqJdIu{qM)_c2%!$724*3VVht{_ZMg!G_@iDIy86GQs3@lxVgJ^@rZ6 z*7vaY_}R57uXzU{JLM>?&oj`f7fWFB0gYC0+_0GBf_;KrG7ZE*R~x?nmDlbw4<1U4LG7~arO=)2?oYRdbANtK-?neL~PU(+gC zKUHA%E$Q~o>5j1I4l;L7HSc)G!+F;vzCXuiJfIlk89Pyoq)V4Vq@(N3i&Vs=P_^l# z)a2=>*2+Mx#Z=0J#F<*v(pS${ymmGVBybwr?5I;=)Zk5bvlkOAUeY{19b)d|c&+rz z@#$`%&RoGuADN_d#_0-%fgXXq4G1L|x?d2`mJ0-VEZFh+ye{h@dp@z5)VY0^eZST^ zq7Y<1ZVlAm`|qz7-$8$ZpY`v00#huN76-Y76;vJ)6+Fw^Y=MlNaIz%ozT_C_BWGzP zw*Z*v6Zf&nkDew9#Cz?QSQwd^BzR-ILW*|1^@I@q&Kua(45ue%Y+*K>%K1UVo@^A) z-D|>QSn$6-BlVEV@csiPm;7_ZKb;QzqL>wd%>4ZkDTDep>t)$s>%vn86S!ua} zLcniM*P*W2Or2V_QnjGywb^2|W?7qUN23y5q6gX}ZsW@tFozYUgmReSFlXx&aapqK z1w6_@WQdfNE|bN&-$!uQ5%?}A+o#DWl;Z!UP=(1Ox0geiRwCIr$5r>t3TQ^JFLhG4 z(HCL+>1?}nxm%6#p4L#XH-K&=Upg*ahR4FP2$wrECr)#Yj9Y>umU)9>ql$~Ak z7n5>>AOc&t7U3kEzqA!2(!Sh+QMTRxs+rDG6)^P2|- zh9-;cRX}!vAVtbZAHC!%Xi~?I)|6O4sa7f1ir0oI&q`OfIgJ*hgy5od_zl5{Q@n06 zL0+Ag##JQio)*9>`_}O$6(C6`lhPrx$;@$3y^+bK3%F)iQt$ywtppkj*aV+oGOsmw z`WpC(w(c6?(48+~E!JgZ8euhVW9n2}*65%x%bAh)uo)zVj&hJ~p=*})n|SRY3lkMnOii}(iyygBG{JZ{n{5bEL`xdwyXwwMNN+xE(O5_!B{c`-q^)JsZU-Zo*}+9iz26?R6=d+$tX z1t4=6f3h2Wt(#w?SNgEHvc{!JPAT)c6u8HG5L-tt2z+`B#X1zUF}+~H3e2je=m<$0 zh{X!dX<`U7+v`t?iVWYxI_=@FX;&*_aL^lA?J)`A#e^J)ks35ufD?d7qt%+#gRxHe zHef@FcAyC}?S>OLd!g*7&RQ>9J0NVZTAx8OCAhmSo_u3jHp%n&mV}UTR*}e@$^eQU zkQp*K-g8%|W;|9Cz-J&1VFO8ltC2fiI$4g59=*AaoF)##>G8nss!aEYHM7|Vp~LH$ zn%85Qm@DA&PR8fLIN^@(jQ~GG(q;}Mgx>84DvQTycpyP4XAHO{`WSTM=r87@EP? z7FHGts3A*RqeT5-2F7;oTp$o$j2?W*+1(dWDi9Tv8gvmW04t-(;TUm>MBg$9e9yGN zQ=Nm4uB&$*$qaXo{20bAl#aF!a16Kx90D!@r-0k#z2&nQas?VJ7?Ug|TcjTpOGUm- zn9+nY7O9)way%B!n??noi{g5aTD^uc8V{`w*m5wa<73eB^GH0Y_o`Y!ink}{CS*dY z6);^<&$KSa;*03;_-Ay*O@tZ0f3V;3X}e#G-y**&7eHXQ4!-w+8T2BZjXLfYzp>?*wn{irxCV+KqXH8;IrJ zG{pLlJOfnZiSGc_qNvEG2T8+0O2mTpwI)zYIFA@69L6=zdko9Ub}stNhf>XT0q(frXhUqwkKK3I#f

    Y%5Ws7? z)bYvgXldeiP=F9#Ks57<-BCndU+ z&iM6#3QLODn~h;9*UV>r*{EhRS7)G;TKmmAauOJQb;ZOKJ)@0r)h!;j)phHhB5CVv zKBgALU58o-#L=*Oqv$F7w^Cc5A*+NF|k@v<+D)Ee19A&mhw?Qt(g0 zP>2gv(dEISx@WMpX;vq0qKD-&OkX!T3@`M4*(1ge7tqTx+k}->WQy=?@EOm{G)h!0 zDrsaLM?@aWZAs?cM2e1N`PR}IQ$n4uqigyT<>2bzb{4c8)?Gt~GTmQSMLRmX* z)a#AL4ADH8^yR+ZpWDPA>R;&J`S0KK+rRx^=k(t7h5DDZ%?r%5V#Eqxs^ss6#o@60l#|-W;V;E*O>?GHRea|MJfD%f z36;M&-Fq&{4I4zbvl=q%;^;5Dq?B(Ir&$sBqM33LJ0!hxuu0pz%c%CBofO7J3 zH@>Ig2~5x!l4RI;Kd~^n%HuvQ;(X)nBRBefTAqU7hf7xo(Wxfmr{#67?!RV}|8poV zs`1P0E!Dk>^Xiv8zpt-fI!P0DNevf8Lfo&1c>Oyy4NxlUI?o z!CCv=rYb(f5D%Tn8zNWBv{N?+`KwNhPH|dxoXD+{d z`VhQMu;WOieC!7BTMh5{3{Hq@uTNb0%)m*Ld@)bFH0cOBxVLiTWQIngLO)&B$5ur8cj}6nRxAa4i z2p$1JOY+0a8|vQ6!O*$7r|BGtF8etzZ^jR<9DLolwxucZU1v^yb<*_e;1NFhH}(F3 zPdfp{|m)&S~8RoSJs>jl{_KHDJA$CfhAUxcXoOOy_* zvyNXF6ILw0MVSu$ziN^H+Ie&pzV!jUO3%){aADvjH7lCnXNoc8OW}knn`~dO{ayJL zq5BAlC>OMAwDM@w+{(KNMt)>GjCt}ZpuAZ@SURE~6RJL4J6(ek3MC$WWEAS=>e(n( zj7xqUIv)uf2Nlna7u$&Uo%4`m=+Vz$f3-DRKF1`t!83@NgwEawZM7NVVg4Z)MjW#N{S@u>OzqoArPA}+ds$K+jOnE+j#$wLIdnab82Nu>5-gEYKe>Q%%hYky z9l1(LU20jK;^H2!lfPh2YjvkB({B%kRXkaoctki?I!;wKUyYtbR@@xS!Yzn_;slPo zdsrcVBc!*D=1KWMvZv+mi}7r6oK0>hwHX~xU5@1_=@Al;zF`i8Y!}U2t^{1Wmbuzf z^zs1h9zA1tc48$t9{kB|0AdpC@*S86(5KxyGY-4lOQ-z^sVB0}@e+}fj3Xa7vt~P! zBKSE}Pa#2IdB2XTHc;0{ZMwj}Gj3O!7O(xi7r_4rnxgnlI!kEmeevmv*^)=SsLQ1)bQXnwrmBoRE^EHdEfPNZfTE z8bwE-qYOxjD!pUUreb=<-g|+%!7*Qm7`B*vm|3IiVNzT7*~1oURtZ9f_JPnZpcj*w zk8|PW_Jt2-wHTha`Q2=~jJiNIZ}VZ zYp2&}{+Yf$;ep4vR(6J{fyUPR!5@07en10*Gsh*^-r|6#mQjaR3Zc0ck_j_4D<@yy zQKck#AUjF2Yg7!Wz72YaDJx#xN}%}l!uK$R3Hm}8U&P8x)6P-lS9ewAHUg3+?}T&L zfALXzgj%Y(D>o{fe7)pe&*G)B2x^s(n>+h&G9X3QsL;9jif;?16RP7+O9TtEBc_|) zKxUn_6VEmf_hH3JTD0cN06WTy!|~=|O55@NkBv`t$NLPF>92V&cE5 z_aI{pPC_}w?RcTy?x)GqUCv3epRIHecOBWrY0NW_64JeYZu$RtId3wnwY&*+UUSD1 zXsKDKAfi~EWzn8iIg9^mgN~Fsr6oUWBL1TX?17S^h|-9#0QW_71$?gVwl>AtQESD! z2&$cz(pQ$tQx!8%A}rfLS;~PD!i<#!J<_1G#$|PPtF3yLd-=S|Rz05yI#&%m>xJSk z44>6M%RKCJvAJbvQ1^-OCjKGtVqwnbpqw8Z>Tkm$7tioma=)xPpXPet@uG7-P{Rm6 zy>0kn)yK=LP-wu$d~Rff1oj@wspYy0S9Nu$2N;dci@*}RgC4tGc^|F+@}*`iA;-L_ zf?gMO?d8^Nmyv=_(5i7%tc@RmcG9g_)!Rn*P@e&XL2|$XFAwZ^0rhiF{kvQKKW_Ga z{i&;aO6M5$81%C1K!j}FUlr;2W@S0xC$cvu!A+_k&V#p?pw~NWNS+<_jCEuT6@Vg5 zjEz0iTmlB?W)kjb6m`DY$jle_GNiI03f!h9J_a_eds9vxx092TCy|J5(wW^r>L z2dD`?Cc2omEhOb!iB0V_LINGFO9VsjQLp5?@~1~B_N`=+z1Fc!8)U)&fqIPrN}+#O zG$qzy+HD*kb3D-5IIzuia@NgOdej2%)2#+JEZZrxVxMvIs3Fn$eon#t5(D(s{n;~; zyxzZ4pPN=Z!PY}RW1}z@Zofs#j21Gawk?S1q}n@*HaumS(HeQhoCObgQ0X*~AuiGr zuGIbI_&5i{PerMW$3dv!9DV%>uG;{bA)NEqv=-T!4DyC)vCe{#qE1keNz%n=iM{)4 z=V1&R)XH@AAF?l!ellG9_nFc^9ooNt`A7Vhu4s;c+lFpA0FP?r;tQDx=)cjgH;JB( zvK7R(y1~2pQ{)c4i)z`6`lkB})d{h)8_Q*L{OLGjX?UH9G$2I+RK~fQqIP%pH_qeZ z3n^FE-Q}j0tfDZVc0s=hW*p?QO|V|_ zm#KfnFmD=e3t*MN-Ub2!+Qvd}J z(X$h1-rX)%BE)#@AksW0T-D>xQM-TQf**Z7pLRlU@mX%=J=E!x{`cap8*gmP+eA=H z{o}z@-IIFn_TbBQA2?0#y)APG4{VIwahvM`j(_I)m9z7p#JTYA4oUCn2rbJa&l~0^ zNi*a+yFX|Rvb^Fb zKc>n=&)FzRq41WkXE`@gJ}gbS3;!a%3w)Cdo!6Z@1Vaa1d+YM?Gi28;gSGA}XN7kk zB!%wuoHI;@vKuq+&cquj%ca}5j@+%82Fl?U+|^a&6sBQ=U#j;ew0wF*G@PzA!m%EQ zh3`Hf-8X{v84s)WDDPGamwjAxV1q@+JFeN)J9lfFt3CbSCM&5*Fud`cxu%eeiMVf{ zXRaAVB{cs!l>1jtpPF4ymWa0a(cfSVjoCLS$5VwY$|7Ue(?8V&W!|^7+u!s2TEVap zZO%e>)P&{qrzhTk5B z;hUhis=Lrr;Y#R5EZwlJ#kb$6m1NH$&OA@w{gRHe+U;_`PbWO`dUH>$^ZYi?+RvUN zo-Ea^?oB7xgjW^Ti8?5z)ST@$e(V>7XSYi^BW(_(bv!)b+v4oXpO2W@SS~RO&Q+dY zpG_e(f2q~&X$iz^0@ixvIQCKVK%FNlnyskp8*;{+fZ$e6gKbcW1K>BRLXiO`#pL66 zy9y#V6Re<+JJjqhtDo?b4cNElH)Q3Ms+dJ)(F| zL}aM%vt08NkiNm|UAUw8MLHWN7Rw&zs|I zkHwwoP}U0|V5vY|81zmp5wWoUgg%`n!C82}PZ>l>EDgj4Z%HZ2-{Q_`-uJslPPzXp zL_NG>K*e;ajA9#VpkxXRbfy9%#^s3tQkhn>?x9<)>p1t!|U0 zecZbz$Ymm1H(f7V8GQF@E@OTLk1rU7-|@^Qandzu1_#-sItMPe&g_}gh{x+U(KhXo zYoFOn+)e(^A_h@`>yXF6Lo(t}qzv{)L*R&sNFQC*RCWHHpZgB8R`T*iZ0}xF^e?s4 zH~9)a3Ih)_GVdI`4wx@@Tsv*M#^X?EQ}}GQ2&l+lU{Mt$|Sxb}F?wDO z=4??H%?I`RDe}C&{#}yn`lI8m3!^Mv|LP+D+y8}B-5ZX#Z!-TJc+q&Zbixp_SZ8O7 zIa5B&Q55mMEty|fFec-ON|2p?eeaYvxcg1!d=^5Z@|T|}>y^z+#W8;8(h@2cf4ooio>& zGWhjSJ?Ex%FOis}V}@duMr*05*46};;LTs?9i8-BN?-D69VvU(i^e}ZkU$RVxuz?Z zl|rp_`M5}|;_jjkp7Ey6%*6llYWRhaC0vBnjSkg$BhVtUOAvWZH!o;290ZD6B$sJ6Gt)%c0>Hj%~@kTiRy`oipVX2H`Nv}*Tvp3M4kO3G8S0qki& zFt23|s&P4b@x-$rDiQ*G(;X?PcCf_^ufOw3X8C@JHP|0sPp!#-cvE^|gO)Z+TXKxH zx(7#_f}wAitHU06NbC4zO@Y`|96NHGo)ioJ2QKW=-8UhxKd+yqePnKz^?ve>Nh!ze zg8k$(JHDUl$k&-)Xgy{$ zFd1{LDNj4}l74+&TI7MarD5B|B-E79`UGI|j`LQ?@XFdKu}@G?u!atN!KtOJdCytq ziUr@-iRsgQXBAQ__mWTV@5m{h`YoUCbzJUKGD)pFj7h4*5skcpjuta!_IpIz5oPax zeyb`Z*r^?xCNniFRbVC0t%u8ahg7!FiANYcs($4k=0%F}$O?2NeK!$FWsXYc2iOLh zG9-t!RRIamhq|#Uo5Pj0R?wUkT&|HRkI*dx#_Pq(&+lulH+_-8^l$Ivr-2tFq zzw#XwGN|zRDDoSJ_zQ=+vfBj}aq@eenVfYd9i@muZ*R~RrN&3Z)mxvCcYu(w>e4J}P z&6WERGhM_ls-ZCW6rBM#?mX-|Ebg@}j52e>Qtb7h!UGRmu~_W3olF3hU<$*^tZM$M z0of&g@`Ar9Bnn}h{gl&>5}4EtmJvVlCdBu+F6;TKudSCh;F<%=KR9>R!_$m2{cP)< znm?Ix&-S-gtornWkHhIx%MWH{i9FdSl~!cJSPj1(uGWUKq^tlQtDlu`m00R`5_bdyDFllY(Q_*TcV3rO z=tasc-7TMEuRX*_z{??&Dnu7MnkqNmOZCZu&b7BiKOVjMzsv#pm(v{{u{8dS9LSL! zxPiyj6ypUd2Vr*&HQ#=@f2KFOghj}ty|Ly&u&m_sLo@GS?;~SrewQGhwPx9S#_jED z>l%KNd4S#U#hA)AN{Cbqx!mNkPOlr^O81~nZ;!4>58ST)VQUi6S^(KtHPQ$vJaA+2 zJVd9W8Z1YuYh-0RB&(lm$d=d^>(v~WO4Du?f!}-EZ0KYXF`dq$6Aj6@DBtK)| z#%4yn)50Q}hf&1{F7I@7xXKYNdv?#CxW3w7I8A1J|yjZNee;vof`t5 zX9dMu$6hnqJMfAk9Ohw4Ig}d@cLJFC(2mX{(;BF2ujH`_ra{Z%G7n=9L0ra}hkVks zza0H^6^B|dWW&pgLzp00P&>399}@GbYixsSVU5$rlN&q2Kc1aA#4~?JBIhMHzkHf9 z1>NvB-i+5B3Rqxd3a8K`hD zmKTUuBEL13Fbar8>g;0!*V^j_th0?|^bzZyt}GpClM&xT^+6Fhp2}XzdHYo%k41b` zYg-WhRx+yM=f6BLf7SCoME=Yq%>3}%b~bwTyMOC#6mZ@67ufJu0q0SeNfhwBv{k+V z8J^zqKyojzzGY(pKYyA(@z}i?<%hXvDxT-_Sjgv~zvn}wUI_rqbkbDg-N4ytd>RSk z?8k^q@!<%3_{qtiHr>XDeT9PfjCL6{hr1_C6QQOq!TZ_!VeY{|$}Jwb0)?8nD*TU> zjn1iL)#Ft^IuJ6sILE`J9vwoFi?!CsUb2#g@2r8B+$}Ynw=VR0VK1ocG`DihjJccU zI2kl>d*F61B`EdPPg>N!+4_8^Po&=&W9($A)VRu@)o& z|Brq3HHzO=QejO8E$}yAr;TWx<{I6ONQd~L4Y0JGwYF>U^we(Vy|(?BOqA=|Lih5% zdg86lEiLup*YUsia+9Dd`&wBC#vLwCz8OZ3;~9l>d6Ew=4jb2{!+uaqn*E2<eIsq3=v{zX!yJ2UYZrcNUwNgjTbJXl=FI9O?tjKzl z{p6`7gIqm{iT-YFE1zE zf1rI)b^IbDpC>n$c6RV9->*87QCVx! zMftr!?a=*ddJkpfcSU#i1@wGmDp_0mys!_Olkg|^t3_EJvI_I2P5~ZMwGLCVbS`fm za`$_r=d4!q^EGfkT((+S9yVM3-rCNmVB%w68ZZ6$0Ofsk0W8P?aoMs6bkW{*$|fgE z*88h$uU6;9;In02f|qWDb3KqBa_iB>)C;z2hTnT(=5}bqTAzd(s|c>EY4}nxpTH_4 zm2bVWNg_wXTYMY>V7P$Ynl?v+Y)PN(IbxoTzgyz1Fw&P6Ymt+b7pO>Uck^-UYm3%NYVlex|>YrA1-uJlo*8hT4fHZ%?mQ@BO{`P=oc@&NHDb68vesnDO^EoGhK^ zAZL?b2_%MfZ+>(@G`xAZB|=$9WSDuWb|a)mL^s%coV7lPue|XH+X0x{U$3;96<-Kz zH~sc_{qm>WM9|?t++S?WKORvVMWR`IQim=l4>2VVF((hPCJ(VE4{;<9aY<7rh1}Pt znG`p$7()QSU`3J(Ta9$_ufChiF!Pp`4xK3D9P8od*bIHd_L`2!An0iY7qxN4#%E1| ze&Y|IdFodr!=C0lDD+q`pKH3idPQ$zuv7LNEqnB=N_+HMko%q_@n#q zm<8d(oKJ90W5=ViEN*p`o#I1zVwjnU9GYLJbkH$MNo?&z@0;3_%x2o0A*2wgT_6ve z&KP8(h|x!?tqtO$BO07%d`xPiqsL2h3A-p-m`;{MNwMve)>niwphPr#fvW&)?E38` zoHr<Ir%aSS-bC>V;F5{WnRA z|8F46Uycwl$_*SW2}G`$A2#LV4sV^KdgG#qFT}1gjtH)WJBNp49rTlu_7Skp_7caj z+YufCZL%lW@72I(?j^GMC}vVy)%6V z!A)yA@Jz&Jmp8QubUhAnKkx>8*P_~G-~*`1P7E_)2-wxt?my~k-;nbb0S52&KS&ii zhIUtk@5=HGu9#mlToxL~yDRQrDBV8)u4a8BYt~yXVBSVh@xeRWm(K7QF;%61#ShZ_mPe1htxGj*=NgQ3_T}ktrek>Swv$3b1>g`0={AfA*o` z{k&DZWsrLG&yOk~e3p#0TqXXkw4J_JmHkLWVl|pRqZ55Gl|D0BQZRD^Gv042mrV6v zv5>AYb$45Rj>iJArDE*0YT`Y)kCmAIkEydiUK*0-Kn#XEX&T9n>OIGTdn6ak0KVEq}?3MXIHaE2UMPuo+Cf)_=o`7 zh+Xv!iZR#bDy_bKYlDdRW50FQTSq1-fW!BGlrV^VXoe)FuvM>^Z)K?@FTj`zsh?JIGsQ*w&DhGoUXoGpk; zuA#vPq)mI?C#&_K1RvMqHj+ia7Lfn}I=TgRbQ%4`M|W{K8p6z4J?#7~nwDIeTpn|G zRAd3h?&RII$nxQo_jqiL9tfxx4e|#dvJs(RAjdK}sm(lHN!G@O_v?2(q3!5Aq$lqo zY?zRz*w*4O94t`Ie5}a!3_*#n2l{$b1Gun*0;Sn!3v*`PJ-O(n+%+`~#~a0mjNS#~ z@LL~}LzxZdb2cz9(GM(ofvr1~qjk~ltnpO1sP*vBz__xzt?X(d3a%>YXS2N_u(ywz z8dRuLdHd!2rMvKWtjP9}W@A36za4D%>I>VQKZ{0%>63V5?YTlfyHPg9H z{UvYnBjlZ3MLcJ73v13%0{E4WMX>Er?WHBXa9J4m2_f{-7z)*bPSKEbo_Sc-5_7>P zDqkOHo$qGoa-=i3W=vifGzrubF4DtizZL1v9!NCLxOjIpwMgW{O%h&Ehkmq`<&&X>~it-Ht0)n+#?+q<{v1$M_g?cD_nK-U${23t*X*Zr2oLpS! z$|F?7ez-{p^ z$FuKsw)?g_1r<^iNi5+B!$A zR)2_0rR11IfgVn=;T2C1M`L!!y)gH#ZGOv30kBH0AUdCi*2gz2-h+o&RpQEy^c=ES zgpJ9iq}W(Rd)pTUP#weN#|z+;HZZLq{MRpC_g@CvQ3}pvZDG)67!Pl z%f~z~$U=138Dgn!@PnMtf4IEA{&0)gH+c2IxA2u#{?%3n@{dKeBenrzvD}hi{xm7q z8NQo!YoFLv2xRxW6&mA8Vr|_8gNS`3&AG!RP{G8lerygN9^8|O0&wxaYNZo0swE+) ziHjrdz#^L|H~ZR)Ky`AEeS5{0_zH=^2dRXM0blac3O3mXL^d7+U`B1kl(ttz2c7H1 zIS=fJ{V|5ip2LPV`Kw)5v%z?VY_R-AZnZ?7$oXt02kd~TVsEBo5Fg8eZ0F#wEneyC zr!(1-F=yymIA3Bijz_^le(oim;D_Ms)NbFkFSik5-PeaO6(p-*!)#2nwA*%{-|95P zhmVt5r!_Ngul-iOBHto26R~*c#|@7MWoND$x9{1><-wBb(Fk%1+gOSmLbkfEPDRZy z3c(rA7ln!zWu8hlyXJ@x$2^3GldGudZ-aPb`Wu_jZCA;O>%2>W4Ifmuvsi2;vP>A( zfB4~7yfl&J#Km_cEzVHh&=t^2pZ5+{ZclqralgJ>TMbqF4O~&`yQcEIZGMl*lsv(O z*%gd*1F?LI#j~B$Ix7s&nLi$jSz}ya&6iJIy_%&%PgtQq)XCj0V~P>^@=v#zu2+I+ z+7EvC+d>9GW=vEl^%GLC!%gxld(Y&S(Qm$NOrChnMeFy+i1<9!`HRtLA_R??WV#|oxk5zCm0H|gpdn9hi@@V>OBbB^{?yu6EWxo5$e-t^rdvsoFXD?_jYqa5Z zsP-b7;-2qD`P|uebS}8&i-q2xspF6W=~pp2Kyup)lwg*G;*Rs#Yn3SOZ?hcc@BB4m zRzLK5*a#xbQEOow*U>i1(}<5K5@r4{S$fKzlezdRelVB$^hOC!&aJ2N><`C%2xFf2 z{Yof+u@fI?&TGnqkYB#nzqp+PtjhFE<9+g1S-?Lo+?Sf-tP1+pQ7pnu znYz~e*L63MdALFT`-tZ`iz@?}J}Wt5JB}T^pe^ky!b~cB$5jDs+dWLeAWoVvq2cH6 ziy5X&4?j7NN6;Sm{rcWhyBY(EZs>MvtxxbkET)uw?Ig#nzUs?v$9 zZ(I0(|MIcbdLyb;+-v{KQGj{3>pGQ{Zoqm0*H|YV)@+XopCTt)sYuC&ud)6k*^rcI`R6dqzyaaQ)0?Uo<#rymGppKUKw*^!HwbkNOkdwI#)&AjAbCWPdkks3}S zkMQhO1ZQ}U0lvc|8Fkg(ekovTv<)Dq{SI;d;0+#GUZv?>O|*D;Je@yW&HImOeqHLh z0Xw*%_iQfbj~9xAR1Vh7>zomQF?%Nt2$=_4oMMrLujhu(A$md`X6ufc6_c^k?4Aj* zV7eVgeZ78QrXls^mvJHz*Y;Pq6kzre;a9&gG1*#gfo;8%=@gq=-4<4gM1@&QidpjZ zJf@l#<8zk1;E?tEuY8|1L7c|YF0%|`)}`;q-lB|fkkoF>X903h9CFIECr>3l|H&r{ z%#Iop-psvW(Ym-o z1@Z!=`69?m#*v0m{@K&)@_jiO-L6k-N(-*K2zhj5bZT~rQTG$4k3|^=LMM&m@f_b^ z!bX+LKd7xFP!cShc3EDNo5Uz%p1z(p4c~QJM6Y&g$?7%lkXwS?#)_y_AMDYhR6l>F z&i{Dx>&zf+6<%BM7%XjQ?$t9Xwo*Gbn8o}maKLhh3eDs=TOj>Qz23~ffj^bFc5KxX zQRl}1bP$VSuek`C(QrR3;uXCmAoFnWV2ea_A|GaT7`pRyew2BUejX9%sNkhfqjmpHJqa|_#Il^Z_m-qfrf-pio2EhJpG@n&s`9j#W!28)Jmc; z`2AK%lhrzh8uDF`z0d&9rC9IhkS+gxZuhB?(tw@_S$ueJw9VdLsOgxWAs}ev@fKTm zb5|LpB~^%`vW<5Ltwl&5=lZ8V4}NZp^m2vDQr^_JEDgR=HPP>?U#isJ(&fyQcoBau zvTjO z@A2B1L4N9}@(cs*zQFExXXr1KMQ}LyH7(hee*iNOzvuVg9$JzTva33_RgkSP|CHMj zAb{ikB$zXW$@W?7=!U)6^dkiV`1pf@=bIz9=L|<&w~dPzO15%j+8k!#g_r6dV&ppy zR`mS+FSxKgn5(AtB;(0YY#eelQ7q(=<=JTY>=30)4W0`hA-w`#4E7cLnSq<4oXXaj zp*UL7{N1|6(--83JLYR*%ZAqE+Kl_vdSgm_jvbJj*x?9?UM94zEA;|=L;AteRGGovru33u`pJn^Nbdzz;}a+jM-7|drwPA zm*K-mVP$)yK}~OHl5w$;W!LQ1(|pB(+WP5AcLV&UF`>feVI;eY@g@VM?M*QXOjqyn zEldN;?ZUJ1pAEFvR0#QeJ_nYeFWj{+Wb|vMbpa3%yWQD`0y8O5%Bm;KGY_Dx#bPdW zXVuPTor*-x8ck=uz?W|&+?ZTE8r+oL|2}flZvv9S6*n7fr+?&E&cDm-Shg5u(%xRR z5!PxogE$LGWje6+9!&Gt0MA5?;SP7dF5()l&HLdQ6)Q+TIm*eF`_M5LGH@Ee)#`IHMXx|7hoUs}e7{wtW_f8S9(ji2$g(Ed-oN6X^UujHEZ9cR#Z zsuv!){YFiNo>fF%5-3BA)lYUeF$RDxD;P0L>i69V)s9XULFMvPWWSNiwSa)ON3ujA z`QdDrFUs5wma_vYmNr|oRyXV~0$z4A%71oO9WpUud%V?eE4=!0LO)Xyh@2L(O|zMF zNOQF|cQ#-83_~Nn3J)Abd&t-gl8>mJTLrU%EmynUKjV}J-k=rz6$h4%5VoSd|NDFV z)!P?kf9ZO=6l!9*DUvahFyr$lF`Lw>9 zUQ_E8H-RP1qNd+o-PG56@N-3HtihmQ*Mp1ln`^NC9LmfCNvFnIv5?xbw(T0jR=)1m z8i5L-p`2T+fRV-nRp|E%e71V|JC@|@4&N_Xz_CO(ZS6)!Zzt}79phr#-6e?Ad`6_f zDq=Jf$CFUZxtMrC>JMP!P=H#_>mWJ5nh8VU`G^uf!@awrTS$J@s*>tNu2M3et7 zA1q(d?^ZM&m`d}ljQOzuooslei_yt>EP!+j>EUo-sDLM zq;e%&h=(T_Gl=Z>O#708NmE5^gGZkb7$!pa|BE_}pkIMF@M*EyJkdNa@;RJqE%M z8;T_d-tSC=izcWyloI`$(ZEsz2%-rWVXnzTHYb?$36#2Is$O)V6eipfS1+UB_E-1g zf4@~sI@h<(gSd-6=(!^iW2Rg|gE@W%F+m2}#Qvsi(bd3}e6rEjBwm3BvU7kti@U0`^=i+iXH;jQpNnUgzs!_t%|!fcAEUph-NA zW1C(7)&OwWuFL;;16+69+1|dJO>h(;?w85f9P-P{pBsuW2Z1s02c}$e>RINZD*lgw z*gKMSoQnNl1Px$B6J+)Qkrd)V(#`nMQ#R`(HxBC2-C&TMlwmYek&Qd`FAF?ywqbyIfOVSMvHIC8 zHONF*l|}F&dSBS6c5`zu=2tZMk_PGJ}!MQXM}lZ)>H*UG&p09WNim6S_PlCem>&Fj~ZP>_0vu&s__T$0x0%&>XJw< z?xv+5pYb0*8-VQx0IGc9xU$|AUg1kH45+@7C^zxki7)A{$VHr8k; z)eN_mLnQF@(yW_jteq1)9NQbAQ}s?%cn)E-FaP`~kwNCUp$pB@J)Z+J&%M!3{&A5( z;CMPzA~|s^rl$%RNjRvFaleQH?{O8SnjTClx53ctq>Fqi?dI7xcvBcD_nzyWDSJNKpXD3>V~Ij~gmrmeWsi?};A)nl z6bGB`VFwOH&&3d<^pIuu!}bXk#)pa?0BP~Ak+Lt=O{&nY$jsZP97RmM{LUZ46H&qq zTSNS4E=I-qSvWL+nneUzOUEFQk)Zh1V6VQQB2P?n<*|lvcLTp`8B8yeO|>oE%kduZ z83=^JqQ9@ms}n?2=}E zr6^`|IUS``m|sp{HKDC#F3GuIOM8OIrRTM{a$NH0kT**aM8Yz|?d}=l zI{ef2M%WfRyc;M+0f(_aXQ=9wh7?mbMeiZtf4DEVm>W0BH0oC*6pYEg{Wjl(opykd z@Q0?n6pKrbeQl9L9^U77eqJ3dXR{vE(cCY-1nh$Y`_O*-LusJmS$Fc(K+r7q2>1JP zu&t+!gI+!oD=}1_`ptW!FSJpwE$Lg72_gsaiUd6Le>&^%q7m|O;jt;U@sQ#)Q(Fq# z+PYZvmD*&>1#8htN&wgH?-l9m5m%bU1wk_pZ5il_H)NtuI-9mMz9MXxx`)os%WLSw z*3ERgzau=DnN&LDrt6f&NnS5^G0nVo^FNA*u2)_RVdS{iCW~tK-ydSSslBnDzPI$$ zx|@k^j*XcrxykR45#^X~>-(7l9;qRH%c?slGSMtv(cEHPd0kIDZ~L+!sr_oiuOZ1} zL@aEo)ek>-xmKJ`KayU#L{jUl){W(~9S_X`uWO2AXw3g%@5|$%T*JRlqU;o*?5840 z2-%rZDcK_X8rk=KH_cI;td(SEl#qR2vX+ot$-X7~zVFMt_o%@+sWV5v-{G~VCIadY6ka!crJ4(0qi3duSfqon2@=JOAxhPXdgWOqBv zo#uVK`na%470p!4-$T4sB8;P#D#!$<3P$PJa!dY_F&>D1zm+o1nQ^iFv-dMBO9rwP(fivMcPo3 zB%*qp=MF04g??cI;Z-bYb8F+F4o>4{vIa;rMC!7uBUX)CP&E;OZ=UIy)#fSdTi&BR3ZX;N9X$yv)(D=fv^IQ zbcfzXPzDz1^!tnlw&lb&21d+DL4#?N1d9ceCVJ#&Io=IHH=b}8LlM50r@p%6k}Kq? z9K%;M-rS-=$SJ6$bJ@-0bqV_|T?Cc-I&ELx-Wx6pYzZ%fJE2Z4Yc4IECpu_|%rC@X zG#iNLc{H^l+kY&}q$W4pn##d)>(+p5YN-ffUPPNkuwiMs&WXEbxZ?z{Pj_sdAN|gl zpc$O{%Rnv+N2@k}i2U?5@7ZA`HQ=WmhN~EB@tQ5)Bg>#MMs#+>Cm(9tv*3m1O}~#w zobWy^7FmP5_I~L^-jXc>^OT!!Y!e&VKF!d#W%&(g+d+sbTDkeXAp*E4_Xpl%Ci6r& zNFe_LQxtS`pcMV*&0pil{^%ooz6kkJ#eZ@!KPUAdkT2nH7B_y4BPon0mns?!De!(l zETH^-56Jfb-TCc|@4i^;y8+;BuS}qgUJ?FBjN=mn`Tq6}&C2li@Zb6mI1=spA!@0F zJLi6I!H=H)SMv-z0OWi4UM?yV`qyvKl{uI=u7vj)aCtxYIVAppR5l2Z?{CC`|8UvQ zJtl1C6i@GAMceRpUzRj}B8vxv_sUzHnu_m)5Qs zaSH3#*1KY9pQ(4By9PS<_5p;FcSnGBSu|H&-_rGx^Dwhy*Ri$GTh4taBbTd*-R?H% zUf*BZjpRD~BCOmnZXZE24Y#~$`u1ni`=>=Lxq@`6kL9Zsf5ioT2OfVpI(HtB zeN;z}wsE}!^^6lkSPgk@~Qw7fxmat___oLS4VI1jj-u2_kCniqg~-4t{{)&9%Tb)C_4EPt zCj;tF@9t)3Ee3XEDW$j@g%A_+AUS?IXY)nMQ}?sgwzw(itN`7zFJ^gnHCKSnbrHC> zGk)orZTPjbNV}fwL3VRF&pe)Tmm_jLZOoLNosTzx-s-YF#3W3p#koo3e3i$h-0QNp z#nIZD+uOIt7I#LPZzw@6YW3MbDoFT!5ojgq4d76xNba(5$<YAk{5l&aegtYaWNyhrL*hzC;U(p^g-~J8Gwi0iv() z-@NYr`E7+@k8z(O4o-N*2s^dZU+<8KL`zFhCh67xT70LX&Gl(I=;_N3iimZ{;4hVt=EM+Z zyE{irbsg7UZ?4>_^mCcl7*lti^pcG)%$4_I6=Pagrdf(&f6(Sv$FzQ- zC6B9qtXZH%MdW%m0Mi{X!?7Whh4S~*(c zrm+$UrO35Esn6-p>pr4oA<_C+sHY=;ba6(2UHc?k6EgP6I>xQB&oyfnk!nER!99hlhqgoLdT8q5l%A7roHLhda+?TLr^3HiUS3czHA;f`$V8 zmR~0;w}{TE7ExWcs4TSW{P=#({;pk81vFs7uGvsQ$W^=KhGcGMJD9^DC(7+=u12t=1^~Xee8tqA8?|Q2v=6AB$C*dU!%H zUwvO;mtKm!_LL6!W=F&f?}&}EWQl-jy^~X9tkYtV=~UC(vg+4XM~4wL+Ahu4mFpI& ztv}@DCFQNm7lQJvYD136+FEmphD*owcDcacV7++wV{S&vU9a|=hAFW>!edqWKXR5qWn<%vh8Q)+w_TRYF@Vt~`XV4d*XC^v1mP<7`*3_Ldg%@6BcTME9tMgX) zN+Yh(G?-;6>+x8;ypveAL?bh{DrvXGTY(=-U;_YE1OV0Bv0qthUd9Ct6WQm=?922| z-5@-0_gZ)C9B4{cC~(ELQ6@+V$hUn~>6Ii;VXB(fVIUf^$f`Gg4mxVyK_OX~XEkJN z%c^eMt5{=EKQlnpN}{DVlmiych#-kxUeJ}y(h z#<*ZJJ_Q?Kf7)T>YW?Pq)1ui%=NK2}#Z@xx%SJo9Roa_$fo-7SP9obp-OT43=WQZV zf-HH{TCLyE_%kiLfS{J3@F4}EU({lgrCnQ;g1j5RDfajh(ReSuGW9;~6{h?7hOUhH z`7q~|Ug;RkZ2Ra(ruCICdI*`+0xvHsiT7@jdLA}PYgTy2$$Y>4P1YmAo9s2U5=Cd+ z?Zv$$rf{o8c;i6dp5LHXMARqBQx}Euh2}g}7e_BuEjjSNG{1?>)N)g}+NvPy*^uIW zd;7!(vF@asX(q_7iPe4oNZBgO_ z7^P%jt6Clr%hdy2z84;{eHx=R(0Rv1%ga9d{E?^Bg1YStH#j7qt9@%INk-Js zN)lHsf&`z)^rgJ-#s-$xGsNm@7BjR}Y+aKbk37X*P14+0@CfHiW|>ihhMZLxs0`NB z9UQEwe;6?|v>3^uh;X3sAe5Ky_A|g%%5~$MPgHL4uw-Uo{9`bnpdck>xnwnqd}EDH zLdex=vNJDSl-lki(*ki%K#OU2AC_BKKW3}1v{Yb(>nd-tMUz~hG4-h$6Aa9Dkj>5{ z_%mcD-1%3QnynivLT<`YPx7unk!2XuhQR$VOHM6`ud^@M(jG8sQD5$JUZYMK8Cm)4 zy;=jUbBQSOOJ(dFjq~WK=k^H4cNCMVf5RSAY@(xASI{xSYWbwUqa{g6Y4Uev2jkY% z-d?Iq2F6dxPLd|&kq;D|q>3VLDW%mO=n`bE3g<_}z0{RmzJ=>Dv;g{a@>$A;;7A8h zBTP9rQXZz5UqJR2QFkLs%2d9%6UTP-5_M!9dspq3r$^}6E`V6<xTnk6mI+g^9R>UFnj61{#gV*3>jdx?|kp+=boN!SO|JX>C)2%zHkTzBF|jk{N{T;qRJ~&VW;aTfkG@B+(-fsn@ZTxxf|753DU$La z=t#!lp6@zABj$G$u%1h>ZC%p4yUL?fGJR>W|5``qJWYC%ertnP63A2CDPQ)^%x4%_a-ZyS9_aX10_1d({lwoZ$l(2(SCHfR_3<&q%{pkCq%C1YQw>xao zan+q5F#9yjYNI`%XN>6Th%)Cy4#5N78;i>Ycq4zr$5>j97I)?k#9Uy?HtCs6b=`QW z;yI!7#i;dT0fNV9FmaVKwBpQZh-NNuWFE}1G~f1inpAjynUS;*5EDUff2S#c>ntp6 zcUuU0TgYQ;V-KsOXDilQ`2dbuz>?7O|LE31JGyoqbmx;M7aBLU7p%=oF^9%glrn1{ z!n9@;S~|^pgmX&0ZT&j?#+|%Iow>NE0QEG-CT@|(n=K871cH1v*6S9@O`PP3oh~+O z2VIJbK|EX(wbW2hjx1IOlQ{N#6gaB~FQ4{pXp~Lyl$E?!(ts7>`Wt>BFW0sYUd(~4 z_&7r?di35>Z!AqtMGd33`Gw(}mEPj*u15pT>wE5?U=6B0jfPG9V(t{eR$F#fIx8uV z{r>3PhTp09`X#lK94l%_FU}{cJ5)aEFpYX!O4PX!$V)I=!E&FFx&Ko{gvjMTQxf9C zJin}eP%RB>aB}E2n`f>qe{82OIGD&|bt{8H2jEZ&;BdP06Drf^eG|5Ba-%n8O6Bws zYt>}mOK9aPf)p`yF6*xd$gHS7`HH=4$mS{G7lwyiqM#CTHP@@Xh+XBIu6br1jMqB! ztl~Dm)#pN3%d6|OC+t(yT0nP3r+PGx;RLPpl;h3rHXGfHnL(~iKbw)q^_*S?qDp7q z3g6C>O?WAs=y_{P?qKuUB~E}cBe}3QF0yME{zI(ve9!2Fzv6g?Yx zb1N?gdT@qWvctP+vxQ#>&xbz&vsPh4h$zyJL$jvOda8w&hPr5`K$l3ona}Du+0~SI zAwwoE2}deL=%xCPeV6eW2@8sW2f0#NUX?QCkkU0|M@qz(F zm=QE_Y8>PKINM$RKw+?@ZlbpRAyb$@O6*X{R);agYe+>yLfm#nkBtUvb*^PR(Q%r& z&qBW(&0!}@GLUd|9PlQ0cg!UJfT?~>gb_>?LND}Pq37)j<06+)C@@3ClM%%9yovzfU(J?e&ZBsDcIOCk)9Db zp-$n(C5B$;$lwiGQI%P7{)VcQ66C;mgJRA3vP~1Yjs7d8^aK0@YQpu2h8Zcth|6-~ z{8e6?&vU3J#@osd9QNq(W|ud?&UM(DL2jn3oh#tb*Lk8R*yQTOSphe}ki7*Tb<;c> z62!4(p(t7~inIDjhPWv|I|)&`E*%%;^uq2fP~c`&;A6*1ZgcH=N0oaEy<8I8BuJrd zO)vGih8?<<P~>T~q7>PwMk?_@ha6U!F!pxLH{pyWues#e9MsA(~*~dny^a#+u@z zmLV(Iofx6xMP25Y7^H#C^b%^8mY02hAY2Sl=;+?mXIibvd07(|S?sv%WfG5ckZin| zvrzJNos3}P)jQY&QW<>(3PV%H7GPX1a-1rIbuQT2mH|EZK&(LJfotiO7?Nl=yqa+S zoVj6bsr7JW!LY}d6FbYSrk_P9T1uO&Hi~#l_>0c@H}+I>c`-JdBEq-MM<~2o1|57B z*_hm{jmw%O2Cr?4<<3kfmT%8{Vc)L@hMGbc3J0@~1_vIz3V$O4nmeiPok4!lIur+HPT`wQIqI|V)* zAGq6*tu|**sEbdw+J^Ar?$20s8E&EFWDoMk%jdvT2PWYJcjw~$V zJN9eFIdl;zr+&_oQZL^8$l_L(D+O#nGJ#fmlN06II&r}nC4L=g$0`1r$ox@2OrlaU zcZ0-f!#e^9d4eoR7?CA~oLsp$Rkc#%c+R&rA0ouh2#I(~LNE1x={3z*%GrlAX}P_d z=PwHsbULPw91#i5AJc9@AJZq}SrL#gv>D&kP7`W%QHpR~E_^YpNsH%s`pe%p(l5j$9ngVJ zSn!M`W)Rki7|~~O))`3oiQJ33j>)2ORx5<#KG}n#bkg*IQjF{D=n3=h5hASeJloe+ zE0;-wsAqLm4z8?qI6jR1yi<}_YF@eLrxVNx1!zau@Z>rlxjaw@Pa~V6EqHM2%)>O0 z1=%{S$nPz2MSP_r6zu8bTUUdRB8!~wE`*-iCqaz3=Tr;E?}vR^SqvM`;~8?GX-`&T ziENc!o9dsTQExMZe#q{xsVcIBbW5xSwq*n~R0TQ4QOg|Xj5s)Iq2&h^=s3~w6m+%o zoeu1Z-rp88CcVT!P*7?z-lkBPa{ZFG#pOYYf#~Ye_~z8OCOi4E%#4C(Z$(V1um~1{ zX6r&GZk#h2qEazoK4yE!q)Chl3??)5AGTp8ArV2@^nbO-gWM}2!K&kPDl;0(ny(Dg z=NWtW(xk{=2I|?H;80z2Q}$1OLw42HWE#tS{sfuRW3Lc!+l3Y}{*5SG&BA~ZGA$Y0 z!*aDL4|Eqbt|whILD(egVfZ7%mwpvl4pjW|v()(>s+cMT1M?&6e#DK3Eq@;#?ms@* zU*EZC!xWtoLtL~DNi%5-r1a5v=B8x5n7%^ayvUn;r%QaKJKlgaO=uM-#ROVTgu;Whlc;|L#Bl0hbPBqYkdk|s8xD83{@Sr z)Ebep9PN{m5$hp(wLIyz)KmmrXkw-{?0|=ITbs>&AvTN>ucOL8sgjvgOzP#NtzT5H zFPU2?G#_i{*M+Y#^-6vo0bkGu%6}U;j-;b9lKWdm%|Gw}mJ=DJ=o$YY+<1Q@vO`yq-zPYT)JL@%{fbqo zjFK)YF?%BLvj%$|L>q^)+SNn9xW15>xX;RcVu@9`pg{Nqevz>L`B0bTUa4xA?pAPe zln97&kyb;fr7Z3{QT-Dint$M`PL~Znlyp#@vpoN)z;h&A<5qGIpqB;bY5pJOq-6K2 z{9=e@j&yf8wLO3~%9NI3d(zX$$gxaaZx_h7@;gK$?&myq?^A#!#>nT4o4wJ<)Te28 zfx{4uoan%ObVP&!{Zfmkjn~2J4wfpj{l3Ozv}^nk{?%s2WG}pbgFR$znDyv;7a4#2 zU+Oi=Fa;@NI4Cr5b9O0Na*K+MAgF7vG^B@ywJPTE8+2D%O~0!57Vq^nS?XDcb?QFhGe_~UF`>OyVdpSg{&9h zf*sfCT}{BLec?mnZP`otDXyI-A({wW>+zn+g?*~>y%K%wbBd{-;RUh-9P#avOzWk{ za|sm6e4^L;BO;Ooo(0Nfu|EyJWS4c!E5$>aKSe2@{awqzY_Snpqi=eceW=|X6VaQ+ zhA*IcGa%J|)zuikI9v10lanpTvPO!bZ+DxLUN~32kaly0%=NwJ&I=>5#5D~;eLlQ> z%xv+WFuAwQ^1ltwu&VpOBEhoglJ_$A2qyJ$y#fJ+$XiWUB*iBZJ=-!0?E7KW$bmO| z?>mDLOG^UvE5Y@=Rv*%P3bZD!_H@s=^4F6uo`V{A&nQh8XBdxdI+?C)MtM+gXi7#4 zrYdz%bHo-h1xEIqi&4|r8Wg{u%gNX#)P6T9+vo!!@A&X15cY;U9B!qYW$H0{jJvDk2H!xkEKjz20_B)8ePdSdF*Cv9}vTeso5sSf2>XCUN)1Z;> z8dt?|;^~_a5q6%c-BpUSy8hsxP?yZwO=zMazaG>1kx3;-N4ZY(D)R6`>szMK59XuU zGK|N15%LvfdrxLx)%n9zj>r50C7q~!4=fz z=3koPAYzQG;&z!A8b}RU4Nsf$up>{01uxPG%u?o#VSCZkTKHwP@9^U5?v|_GQsHuA z%G}7%+~pY2LN{_YBR1ipl}mT+e5FVd|GVYiE1 zSmz42$gAMXL0nIlE3}*_$&>5z-IATg9aDRF*%t;pK5ar5Cl9|*j;|wG&YQA{^}iQH zd|FrcPd@Q5oHbEQi#IDLO|41rNKUc?^%mm&ABV2@aT+^CDFjx5NIT`Q8bD|TI4SW}H> z&Y9w!3hP-3BYX8}WicEvkW(%Dq@p#Cnxxi+olYD;srkOyY{_h*H?L(R_pE(R% zBzopUnsc0Q>2x>WJfC;hGOFDIS1o6z%J0D=ljE@t{opi9m*JFaLgrxOIFF6(Ch7O$ z*1v=RO2?jxefH%y2pHjZ|oP#DbpA+Z9XC_GH)wpy$7>HWb(hSt<*SpPPwGbx$$&Jkg z#qyMYF4%Y7KU9-mO~#_`*fQE=7G9zv5bBq^=*dEhA2nMvDCqC%&Fvax+qcB{QGpf% zQ&Iv#G5@Jxp5Z%-fFk>+{=0!4W8vI*h+f)AxKr{7B+rDS4y7X)2 zqg)TiWt@ALUy$dFx7*B>vb!T>I>*|Y=MU9d$tc>?za|JZ%Qx5-DcGm#Xr$a@CVn?!WIUtq8Re#vau3)E2J56ug!in~u1_SAj0#_iU7vVm#BIEi zpLmpk^s+>j;4w*jHNkRe8W4{XUphnR$4}rN6yDTIT6{nJQr&T+gG_#CF-&cu z^8ED!RS7cK^LC<3$2VNW6>x8Fo?8%fvhJvy%1C_8SF^UzMNh@EGXBZ*O9?A&zE(G0 zeS7$5$g&E1#`F_XIu|`5xzjGG7B9^rRyGxAE{2G9zGNHm<-C@WlOTDve|4Rc-)>T% zK({MEVSe#`N-ShJY$%d^BY4q9$ILAI!h^fekkNcW!U~ttrqxoVM4=Gs6ybo{SH+n$ zPOb?u4~vg!DLx}B;G6jH$8&bkoF?t{WnX1P(u>Hoh~Ho%B*cfpg>IfiE0zAe!kh^& zAt|j8vedP4-7K%u2$i-_x|nkG?V1Qz0aQJRw%Mdr;1R|j!jg9?L}i1_EwC*NaDR^w zib{(Zdn&Z%C7pt}6L;J7q|XA;)yO2>wyUpP((g2+>eWZQX-ZHierdP)R*(sM4!_pZo1doC>XaBi!79rwucEv5xtZ&m{Jefw3m{v z?e_0>4-v+WVx6##yFWSyYZ(k~`GinN$;`Aor{Ql7Z}XK6yC6K`b5r=zkwoLGnJCEirE3_&9-i8;#=F3vn*IfdB^zQpdu9f|Jq zx^>5DS2>M(3Z0fdM8QhUBrlbvaE~;&&Um}a46j;R9)rM5NZ1tV7(AlaMyXe%YWvC zy&b~MkKy*66f@%&v|D50c0Ok#UM(>-Lgji(s@G%ImpwF_O!HeA zAiw0=6RiS{2N2e7*SVqYOqUKHJ3lU0N0wo5I)u=y5|=zetA9zh5P`H$CfrMAJQlwf zU4XvOQC{7M70|_1iK*`*9bJ$c@p%zA0W2K_?=_8{hJwdEepbS4b#pg_yODsL;O;SB z_^?`H)~vm5e`2^b`Y`jVCcfXx!Zoi7T3+)@n^w0!$x(vxA8P}riO`1Z3G~{`zmI%j zwm9Q$R(<{klgurNJ@edx0acNA>DTYPPj6&Ylkz?+6F*ng%^`hnAA-q1Uq z4FLuSybP!z%b|+`-*b!3e_$7I$Ky>lVsE#A?h>UrSXBX5vBJaNFVdCg#mop?bjV7d zf1rO$ic2L1s{|X?*)-D4^&7Y+Ty^5qi|6tz7z*CGWVJv3apmx^%oPS`lQHp_&(t7F=sF(9j6G zoXaX{Q)E6bJ|dXrM(-sRE!5+uv@S1C)Ds_xjnJ#`rBo)wC4pSTXdA{=>`Id8hjk8+$iM4d8@T>2}kRYpu#1^b#tK;eM^ zri*Yin70<)I}?D8l(cc*NxPH<5uBvaT^Nr&S!8n)l-qIe>^X+-zp_SkWc)N@-6PJ0 z8)JYFa?|?#TfevF%e~Tq>(gmfB?s4xvS7zkGLEw1zk2k1>^fC$PMjLhZOn2I6i`c* zTkY2=@jU(kUM6carkEMmY!b(r9?6}yM(45$|Y?P#2S~k{Fj~6g4 zT3arK)07{w@Bt*jU^AHaFID)@?C+>1@OwhyKZ(eW1s5}A^5rajd@dwwpdu5#G&DjG z?G@r}@!VCQ`(w19E?6{gW;%Sn^FD|6yM9&YrBgguFD9ft(V#-p7Uf06 zE!;m`b4P}>*1z~tZ}@Rf#2!S#1$a8@!{yrhejKa$Il}%s4UTtSDrINtb%%uqs#;cd zzIBaluwQYPK(H5D&qF!au6tYP=h6s$OpHX%WRc=vyOE9P?|04MpD>&Zvnnp*YnaL3 zefGesPPM>#Nu}8=53bbne33gz1Mww`GNs@=UO;_}tHhj=(w{98i3`vKeQ_jS^cfl& z6W!1o4cklUpDNM0MOpllzEJv@NtpO;zHMPGg03@WO>#f8D@Bg)c;$PfP%5~3D`u3>H1(h zL8x#5uH(=ZPcdH5iYB&LJ|9f$e$I*R-E!kPvu-aW&LgrSF+KO_s9tP#hHTHgp5lqi zi&fS=^Ad2zA-|dHf_D|%l}(oWIe^I}4vGewu~WUyXuS?&j}04a7W-Xa@5UWTdl5(O z;c5nDImhZDW|2=SjLDo=S>nugttX5%_y|Ms-yX^l#msu0K%BC z#d~|M%LX-x?ynpPA~ZH141U{rVb=m0D69K4G}fY3mga0N!n}}1VZ1eap$=}!sD~J(>YuTIJ19SE7!x-ox@C0C= z@(uG@p6#(9Xaq1N#XJjPv%9ABfeKlJ++)CL-LtKR_ zzfJ-1DQ%^^o{l0s){PcMbBf~lpmi$&6z8!6>|Ix(c~&3UJF)2RvWgE`)W1(Zr<44} zpZzXTtT3(Q#K=UX(kh1#)mU#y!{^?V@%F5-SCWg;{0A_?8R_ub9h_pczDcGe2xBgz zXxdufqHd_iUiM@UjXv}BiFf3WEKIyfEY7;!^!voCf_X(d6C#j)PB1$ zj2-jY(s}B>SEAcb`uT^T*Z1ab$Z~U{jvxk_{R)|)yjrh^((BVlb91LUkIKCuMGr>0 z{@|7w2HZ*8M@+y;utw`lo<8h)b?`Ml0s{q{@MTXtmuG`FNB{aCYBH&#|ok?+!! z2@pq_L>304Xl*OU;|Oz%jkp_{85cU?z^7n3ch}VoLI)66yM>%MPzn1OHO+xe zyi$k4A@Q}`L+HhbaDJ@Wx$7%=f2H>rpf{zs4N0$mcIYVTEC8fsYdrf%M=96wP0aL@ z(cnhD}sB-so)v zO3ocsR8^w57+&;yVwe-y0vnSh)yV7!*t+N*oBq&L)G?HvmmhnNxNO^0&~l%<&c!Bo zY5CwC#Alb@O_IZJ-}F|Le&C_dTCG5DOcAn#bC(_um=~b&XUH&q5I$nKaUaR{I@P?6Ixvq^lV5>X?)59#Xn9CQwePw1rN);;c{tq_& zBTp`(g|WYZ1acVXHea0l%YBxezUAXyO9I0te*gcGEO{G{w0QzfxVn~&61~35A~mDsv`@svuvS#a2)0BJ4G24$jR#@{sumjlm zIscwL&3FujB*P1r@z>H%Pi;#3tB34BFpM-VtN^(m_yxJqN%H~0eKdLa$G#A-%@6@B zqCANowSF*D5(?i1^QoD`Z#Unpz1?6Wyzf@(=kdaV?cKZ|dI#FUCQ094CL7dqxpTwj zmST54r)sC2DG#(RUAnfwD~=!fdtwNRZ#X{Cah+ByE)-C873s<;7HRVNjRh?g>@!{U z-SxPs(G!vm*?Va4dgF}Z@`nbIdglq6R$4LzGIxpC6l~@pb&B+J%vLMAGAV4#Ah^;S z$W}&0lA6vi#xW_6F+wwf9^RSH%i)cvwGF*J# z#IlR_Iet5V|QT3#sQXz zqtBS^8%iK5SX$n_uS-?1E_J-dV?b+enav4v7NYOUz$6^{mw0Od+&YQ-A~Icrlrww+ z!^y9Uy2$Fwo9vdT-%zmJCCNQF(e;D;^No`*O4DevY4Q@1xPs8jbivw~9z{x2)t|}R zg#i5|V1JEVd{E6}`uQb=6|~rX%|uy+gjhe3?XIub)-Y!?Zmb+Xy*nmPLdLjBDIZW) z!i2n4*j}T{dwEnvu=+Dh?16A7ZHx ziu(g~%L}j<7O6Vgl;}DYK;iM9Qb?htGQ^E+zDJ4SCm%9YkU}ghmfwz#r3nGcmg8<` z(fIXGZmf-q?dDtgg&sXJesCi=6;PIe*28i?74r9D|J#4cNkLlFzf1%ziTr?+dk|jW z{HxQ5;&){Kx5uC*FmEQ1LSpbjH+iu4!(f(R@BK`hc4yK(Sy#>{8CVaSw&BWuk7n0jrB{g(-a3Pz^){K-Uj|hX6mU3*jiAn z!Et{Hb|B;Q7OtIczZ0ST=y^ZWkux5^i1RL9I<~(h{`M;vI|v~v?#`g~;@%&E=05|o z_euc+<5+de{hQr|J;W`r?OSLeTiE!lS30I%6dfmut&nuqv}J$0slWQhlKYtH=kjCX z`iiC=Jc1rmQ%Lf>THB760sD@4u~X7aw^+o`isyc88z@c#yNLU4AN!77`T)BCYEjP& zxIg{kq09f#vZtN+Z;kdgSnoZY{_v<6!-{R)%dU(2AGB6*DI%Ko+dOdm4gWLi!7YZ)|uz!3LD_OInViY?uKsWZG_G^h?O)L#BTFvc=!*`HbCE|3o_=+(?ERO;vs^^=kowJn9@B z-2K+Bw-osG4Zu*SZO6CNFDx0PrJ4iAvh~rk8@kuPRxsG&;`X`x5Ia1;+tQ}Wx3h&C zu#m;i^NKG|qi5+F&L7Zw+J4$h9rb+J-?SXgjP>j`xy`_9mhA-DPc0EF_=x#61KqIN zZTH^L1q9bsh>-tQ^00=L%yz$2OBK6`UhEE60~`LVo81N582B5^Aqu$2Y@R}L$8+>{ zNkPW})4JttJBRppu>i&CS~FYt`V0s;HnOM#%1o_T%vq=s&QBnQ{7P4uPB`>wUCEe){dZ^W zGxYiS5v?G_Q~k;)B-yLWLtpwW|&S?h}>(Eckw%V^b`UN7G@|M^Bblh`IxiCVPET* zh6)6O`!VmHSnGz*v}k|HOCb-f#&*jN=s`1!{N0x5D+FO$KtcZ_1*MNDbMl{5_AvJU zVP!8r^`BJ!H>EP4X=A%n$H^@(>de3e)(OzaefE-raKZH+4oi z93-Y+%hKT~1w}4AINfyhj>FVPbBmt5UeLgGnQQL+^U3vxsfMWF0d@hIz`jGi?ahMj zz$mm^oyI3q7v8J&eprn<9?@m}y=&R)KcIydp))I0q1`JgLL@ks<)jJyYY$Ii%|OD= zq3k7zpc)V!&|!EBWsqjDA`*qEllcp{oQK}I_1S&g4ljT1Yhgz~aWWjy>YwADe`u3k zwNO(QL>%pZ%I{8MWh%ozj^cQ4x8z9Ez@Rj%4)*mVk))U`_0Qt&?_mvg3w2B05K)}| z;`>6vZcihx?v4T3tU&zBp=ZK?@}MeMv)q5Uex8mkJy|N1nMJs>zCUnMUmw-bhOu%y zdi|V?E5o;T%B{zi)PQBG^UhF0ExO6EKiSo7x^{XXt;yXle^MId-m+gSY+;I$Rn&f4Zla9H=^`iKtfP0 z+prqF@I#XUS=%%VTN?!}g&geRyeL_LcHg9Xh%b}vNjc_Umj|wbIVrda0cHL;m1job zdx81;RjPqQTrV;Szt|m){=Ih=6p?|_8aWZ*qPK0;T*T#h!TFa)=BksRauQJ}jq?qA z@Y5K;1~fdeAV4$U3{x#Rep>kO4pYjkVBh1Yjap&M_Os^Z%QM+^W@w(-qnmS&LoCAy znzVhq>38KoWd?Ij!fa#X-HZG)okdc3@Mjbfix`isqgUUDXxicdA?7c&jBc;^f8{5X zf&Z%o`yVSmIhH$*?x&>8#PIL{cXlS{u#peiWk6I zoB($rz_Z(d{x)RM{{Z`lEUuW|Bt`FhNE8Qur$`^Tl#5>*)PHozKN~SIkX==N&2kGI zsQwE)!9lgSJ=J`>^S1p0b3AO>9OAanss0}^+>0U>nin<3+w&$gyGkQa_0=2w>w<_RfHn_ zJ521M;Vp0i3fg-9z2@I5IRk>@TXaR}?S}Wb!B&*_6j_@T&yPZNRPQhJ``@7jf(Ir~ zvovpupjXy}d{zP_zV%Z(I|BVe5WJ1~)S^~LKcQqjF(qj(6y3IKme1ZF$GbHWwhJ_W z-!Q?R;hcMQ-mtCp5SkTYuyuAlv=iHBssr&alPZwzrl%nbMu&VsjTY5cas!rNv9I>*L8cU(nN_+p_S5{2NIZdV4OTAjn60J>a~+hLK*HKJQl^oUzB?m zFf_QP|7(uh`!MVf0r0^eb_EM1lyMPlT`ucSl%71D9Zr*KR=Tj+Zy!A#bbU}>5 zcf~ZFI~?dmIeUp1Y5Ht|dj3btcMj}7x$NyT{*%jp za`{eK_Ht_9tkQpY`QOD0o1lDt+VO8IQt~I}=JckOXlA+&%2oZ7bAX?N%}M4bB)+Nz z2W5U+N6k2lZhXGmre42RzVyU7bBJZ|CHGIz@y}lYT0Ad#RGFaOD~qCWn^?3yK-hto zE8F)W_TL;Yz8r|ErSn+!m)Jvj?=uW=ip-$E?tfaDpE3rE!WAd7;X0D+{ ztf0Ra%?vOym{pCv*lC{nH!{<-#3Xf-luR5)zPbn#TYz!844mW0=l5l@XY;)r{h>0r z!}a}*9Fm-B2!a>3pZ(`G{|V=CTL1?ma?8UzyHk{0_tvvLpT=+lEaa)H)#~VK$zBw} z+5x_k)ZSD7(o{^)DS%9WQWmFQ+VB}N0!-P`+x^nqlI9cS(#vFy|I&spF8rQWL;HhH z*e}WU#?hdN=-T$}`J9r^Sd(1{NUl>)VQ#Cpw8IDimLDeyJ(wMpg115??h-FzI7yur$c5NM3ZsZjl=snytSsF_K3U}inh^4-5cYD zy&eAS3ubGi(#H zs!k!VUBlU%;StXJCh@lcCH8pln`V6dx{kaD-!(P!ZPJ7p;sC}3>hJq^yW%t9;&!5|FMy2-jmPb~zeGV`l~virh@3W^aLO~q^OOY_1v zD#CruJgm;IwaVRiVocpbZCji{(bp`Tzp0lp>3Tl`<>5uJ3^xKxMR*0iCZ7CR0Ty#8 znqN`Ox7~#<`%HCFZQkwCSa8R9zntj?A!NS3#F_r{2!l%%7bO&zpKc9=I-2Dm7yYI;*q;k_5I=&Z81Vv7&!rGF(5tj;b1CAWZeMpRQ8 z0IT|;pAwSrswU&2c=aBKOHZ6K*JyE_HkX-u@$$kscZwcCj9@c4$JU(OvpzY$$T%p- zH*fUBS$n9qWpg!(AcB~#;fChIX_fOP@A|_ET-QQ-OCv+NHZ|2F<8zWaQU`=j(>tua zwPLX9{iv4FF{QNJDF}N2-2ZFo==?Rlg5#MB#r7+Bew#&?8zx;(&HvsISi&45MObXr z>Ez@n5~n)h)vbx~0CTmn$(a5t0}AjSz}P@}#rPf~fE4Gohg9^qaY!g&4B*a(rdVcm zE(e7}7#hryb?4qAS0Ci`H~N!QS%ih;=^o+C`zN$fmFD`6*21alpWRQI>1R%vY)vVr z2Bek0v9taf`DosPl|Pp%7rsJ=z3J7dGm?y9EVvI;R^=+rH_D93iSmhOtH zAO;u};nV2}jXAK*+Xi?+LBGM2@l@wNg=YyBmy&>t@rO?4cIfS)^z*QIocR!^Lf?dt z$T?z)7znFFOLfqlh)QM5_%q;P3m0NMZR@UR24NKTtFdakJ1`g>iF*!7acCLKJ$r(W z1a=2!{)t_o_wiqjLp6t+Ai`NXp|s@IV}kjPYN$j$P8X&I)Jf)k<@84pvg^} z*X{#eK~Z(&3=k<{Khk%A7vEn+ij-SO{^TAa^&B)b3Ii(cm<-;cE6oXPlX;&u7cjsd zRBOvCx2)SVmM6eBwX1XDbvhE{jVTmRM1p9=6Awhqrn;Em8zN1rV|c>f{JSE`OyJBZ(d_yMV>=p$%ybihlPy|76ui+^_^SE@RR zfw}sX;Km9vg3}O97$k?c)ub4!F(3Q^!LT%kJc`*Vh&(z8r{Fv+iX7OhBDEi@sky;? zVy2F`lu0L9+Z6C2U_Z@Ij7>}t_4O`CXv#ifo@J5dJnXfY!WmI|gd!dCc%Av9-TR7E zDp(cX78T4yL-#P<&uy_mJfA;11}$00a#bosqAzn9U*rqFT$;-QgST2=VAW%iz{H*h@CSrWX&bp8rAviui85dFIGAggXWF7}!b-)RHmR-O@SEXifLBx8`%3Hq<0eud7}l)^5uH1LF@XL4jiqYsjup@>0ukTJ zi6Y=bHL)>_n$28o@O$g#>tBf|)*S`dz^?@F$NaasTyTG2WC}ctxu)-gKV%!#xOZi~| z*q=n;f$L*B;)0yvPfE-&U)R1dT_ezIk*AjyKESJ39K^r?uRMS+Y_OZ7+28(M&h`N&@I=Icb>ObU5}#{0>Qxuh6(Tk%*~|YAdv6&Q z)!P1nZV?p$0YOTn1W{U~8)2g&F)Gq2NSEYL1CEL)t#mgC2uL%uN=SEi4h=)+%vl3& z_kQ13aO?k^>pCC&HrLFWwVrj?bI0#~?#*?c=YO@Ud^ZgFpL+rK_$!L(0Afu;4Z5WP z^nk&Uv2zr1p?4JrF6nN;CDb~^|8e2BfBc&d4{ycX5B2B60+v&KPrI?9niu3Y>6$`d zgZo@p`WJ7?@MlH%&8zhH@}-_uJU**#Kbly>E~>NLMK9&T_A>WEhx^ZjpZ^hP2?G_- zh4W3<0mWot_+Z?b8dwwgq^`N9XRE+yxAi{dw2n;Fucj#Oi2?LI!J+{um>(ShFkr*k zs1G>`9LAqKwI-OrDD`QOmfqE$#HVc4?-u^wZ231t{`;DDrRZ1)g@xW?w4mIa;X^$k z9_8fXd$nWMSE9G(rZ|<5PlV=`#92i8{$-2y|1KpJ?QidJu&usGZ_7()IMpCbaKiE; z*7%sex=rg#YC{?B4VIy>RW7NiFb?Ec21NnNF0C71E1N$#=dT8&pXSgk3-HXQ)}_UH zM4WASlR6LOP}qy=w02(7C;|y9ojYS^jVPz3-xA;_wF?pIt~}Y!;;eI=k!N zJU;*%k6jFEG5G^yjO&j|&7_d1DB}y1!SuD=Z}kgxy63q!FHok5{y;RUs|OR~QlCsq@MLpQDT$l0NY|{bqW%G_w8b{>Ff8AKilxOrBj-t?YwiR`ksVOj)XqJHsaPIGX%$qAP72Ikl!ymNk_a2@# z80zZLe>=e6`Kmlj5JV`aQtG(QzBE+2TMXMj-4jCctW1zK3Zgd@Ff)X@mRZ`LFZD{k zz#!#*d1>S0MGp<$g{`~ZP4n-5(K2A+iqWKNt?x_#}M%~22Td_sMp@XIKO%NlzJo>(V;+#kjrE_X-V z-g+ovM(qLh_Eh)Mu8d~1oSFZHWI2NrX{Q})M7n;?HfKSD@P0e?%{f9__`f?#;y(-y z`aQ2J1Mc_p^@uOMs;9!*AI&bVr3V=VIy}u4TV3vXMJ<$J*%REAQ99q@(q&xzA$@#{ zjK^-CGX~GnK|PwJb*jvW_vJ(pFba%n+O}58OFq*ey}ED$k^9HiFs1!$tI|Tzb{~pR z-vFL!$(H<=vH#m*{JU?od(cTyR*_qCkq%R#^+4=rBbz2>9nyc-lR=9pm~}VPq}#&T zEOV|&xD?e#FNHVd1Nub4HuPaMU_j)gu?&s|FF0&=L_Np)#N1_n8LJ2qez|j0S3p;! zwj%dIl=f9k6a}tZ8~qlpMW@~E{^e;G|7)4PrS9!%(&$Nv-P@jvn zyuTCFpj#BiTTR!_Z=u1Nvwf~uzhJn%GsGdX4dQ%RV=m6Y7jOHb{|64p7B!v`*~s&d zO7EFoYtcr1M8Pw~D{`>w#D=Ck(T-NB4ikcQ47+yjy}dB(OSR|!m#+Wr8!}EnkabD? z9+|>By+6AMv&PLA5pb~<4M3K)SBHn~>hoM%ja&_)9Hz46H63;wa={Td*84YYR_4!1+bPGzGFhsQ*#|n5IBhO2CAq^AZ=A6rK!*(fPSpQX zKda?{#BGmT!oa!o$lf+bYxr)F>XoxvvY$RO+YQ+aMuR(nG1%axp_4Pwf^C`L_bTOj z)^sn1Wjw{T-tvmDP6^poc_xWQZK#BITRc)TNe?bvS)-hBK(2qv-`K65oPjV_#hB>x zl+spiWpAwaE$PUZ?+?t+$1i|Zmr^ez@A(`5BT)a(rTy@QpmG>S=}{mJ%mY2_!Rjv( zuy{MaS2NXaK+K4#!MBtVp3xh1DP%W`UaBnrgV`uVm7qw!@k;nm@v;MaH(}hNje4cN z&Ef90iiJ#7W8_9pgo)k2_#S^MdGhwvwQqpJpIjEO%X(l1JfKl}d~yd5q|Ky)D4tOh z@SV%mdHVw8Trb-{oMm9QQ7Z=#%v(b5+s~UX*~??qLLzr~E+Rf#IMh$xww>iIhjhz! z4}~xCq8bx9%D5oU;i=o>rxE+>oSg27t0i>c40?wrN(?%4Iapl!;)K=7W{%nN+D@9W!? zE*%}ktGk!_E=s@)BCBr=*iU~!MnwqOD$kNzdn$#~s#5?~xkw|W&T_U(O!QYJ6c}y= zpr-m3@AnozcHALt`#f1K$lA5nRdyAxOt(2o_{^DeSti|izLir0P6CbLmCLJ-^z~Ea z%eNP-cgw&Mj;Q%P1Dx_?w-0kb<|of;bQQTcp2SvdF&Ed2qmgiv7y0ohJppPBu60=)|eiRTwC|Go$WRO<_|6PM3z~sWt1Q|D&>`G zK6g*9tMud_=r8+?!MtUMU5L>aKuceH3dTuaO9~#$-M6sXZ7_N7 z7%@fJ`On104;AD;{k>~6fQ$J@SAw0e&g_TH1dD1RAq&)u+D#NZW1<%XI0b*Iwc(cc z8%5os+g6I)xzsj|sdDiZA)U1qz+3XsVKv1cIc>T4-K)I)z3B)Yma(C7dm$?v3oknB~z;q%6^o_Sc@D2Xltk9^{FbV1c zVHJ{~E1YH^$3mF>)C#qfdt|667i`i;FvCfxym?d2#_Z~ z(N*|5)~E+Byxw~KE!H!FoJrHWhh7-l4J`|Uva z>ePZ$QP{$^oUMoac~Zz(pkzgp?oe)w2y%5Sf@g{w?$_?BBc|vxRnP+*e^ONdfvzAs z%T;a2p8w#Pn9L{s$Rt^vJ;!8`=cXvT)!uSxoO{-3es2l1qBB&s;5p-v>cpKQl@vK6ge0aU<5aBD5z)h{SiBulv;r}<(mT&Lw$ zThHDnAx~!`Vi?b7B#5moVE3K5WnERv|KUt!GmLgKuz4zjU)RIxaRI8-)~(9Eb7~@e zS;I7f^tGPt&sqk^lkvjM7xqU7|E3!@6eW*y9iFW32Cz1AB(>l<^f4GBfxQ!?w4_{pY2TZmBVkX z>em|2+SUuba?j=(rdfKAS8Z15`-rlzRU%6!78TsSu$)#bigxK(ofiL#%;5@Mx}$IDIX*oVsKUD$j< zzdcuColuoY@u{KVp=Pw3*^HVZFROExOSF*7NXpVs8}j|ChQ@e@$}Rj~k?TWwGDKI} zmzzI6G`z&yS`+Reyi`sufus=7~3#VXA@ zf@F31SBEG9MC$_#f4R**TX%S-@|rEe zMND_v$i~;uyLWSyzRQW(PbZp(yZPm${p`PG%zqxLunKrOO`qTOb+5?=vWHD`a$l@> z8bmck3R2zG*)tv5BKs7E!(OtyTp}`TVPd+2+>|W4QmZxWIPSyF*4Bo&A35G}{x(qY z_<4(RNO>%eo6LI>1iJP*alYNAi%SNmEdF3rQWKf4x?N>fnqYbsxZn7*}`Uqa#txRXd%AS52EQq`=ehhNOOZ7H8=TeSJ2~Qe@z9T zMZn&gz6`2>>N}e{xUd>xNh!mu?RGPmZVd1W7Na`nc(sDd99ALxa6#9`Fxji~eYOj9 z(rZQuRSD_OTCa#oc}1f-=DG3NdSfLd2FE*~w>f`pCBVOo(V4zaq8QCD45&4z@S24m z;nw26SIskaL_3z&f^w$1)@oSt(j?ET$&x>>d=L>HT4J@U1oDoGiB$>7EL}muCOi^F zx0a|aX9t_^{G3#bUt2~rPc6+N-9M$~@Ho>?t?!I^E15T%bbo_G-D3v~7qQPKbyJf_ z8IbLP5sbGjossSL;h#yMmLaElceyGf+|F5VVTV;q4o5(%cw$?pkjqRhmyC<&VtCM{ zZsR+j;&-F$C%OhiC@)r}ob=QGxiu0od|0rC*+!+e=W43vq<+rIw=?YirhWJ5ptro6 zRBPfHq`dIK0`?K%$Wb{6156mQYlI9J^j~i7&KR^``NWxIA)`UAn{79`n!ihCUO4U} zxREE!>_R73)#&xp&f{O)7Bd4>Sy7~;)^*ONvL{3!Q$AYC+%zo|lB#W&Q?j>9)ToXJaU+;Q62OiW-!m=W#jj+;H>h&Fs2**IqZzuWuv=ViahGZbm z&p|%HPBw~~vCQ3DrTC1pY?Y*N8OaBR=5}4rRm{|cG<4L(yRe_FMft=c#0+`LEor2v z?DwdVND8Nod53&2VJ>)B|3!KEU>0t$DnaoqIxW!ya^2afQH2bx1l9XubClVpAW>^V z+mBtGHq*yCN(27_r~F0zf3N=U!hxK<$L&3Y>)tSGzv^n)d~NOr#FvtIIqI@KgSL%y z2gvEpE@hXFUGY~Gx_6|nt3}_UW07XBiQLM5yzr9Mx6)6@kA4(-m0CKZaJ9)K+A#wZ zjWXz-_YnlD411e1VWNeV59<|63}+XN`t#B^KKq!zmt|*G8xG#OWVi^jV`){~%Hktt z;QVQFf3xy==ckwJzVz`u+=!V8=fO{M-_=PznWPsg>e?zvTKWzhx2RbRczOGgJg3@u zoesjfn&cKI8tLLR8H0T@S}TP-?9py)T9Yebcgd6B%23iW_Kho2B01=|m8Gkrp3`zx zvXSiPRP)=~FoB94P%ttffG3b6r6aK6aB6NU!P1p!=5~?tsNu}=uFpPEJHbX-@T^FE zsjx;0fpx|{m+MC6YCz(B;qCZa`;TsD0y8zfRDBVD7r!`%VEv(GXXf9dTD zDWb~UoYs%wWo}NDq?DwtqA%^8;cpP~f?&|h+(Eq#$3KA)Pjm=MDDqbh70WvWC3JRK3;cI2_W+czJ{jRx z@#L`J`&rnn6YHVYAt(U?)Dy1@9hU!><+QJ0tbYY1;L9e{1DKWrP{QZlJO!){5CBSG zpGqfU#-j8uOm>TDuF{do*QGcVH!%kcrjvC!lxd;V#v(bakoZZkfEg2nd(T93fX}oB zC=V&x&8j6p2GLW}S$PKAszRpJS4Ya{;JU3HL|8^hbN!FQxBDYsADA>Vb@l)x0B0QqK!hg=u7$Zu?&52ECeetE9O5_F0bu88 z<`!w>`VIXL#XJ;HN1TJlq!lE!6I}yn5M&^@$O+KE^2LQ`K74-!9S%GDjP6KOUwV^n z*E~PZDaEG{Bi)8&+#u(4@K9X}+`3 zW&r%lUkWQn+Sj`#=~QeD%OKT85KPpv?oIcBLKEzBEISGhD zvoR0gD0&P1C783<4!{TL@rj&7*W?dM{c2NDjj}O57?LZQcZLO21vsHM08!CuMd~dM z4q(nQ9R49ZewFZ=+F^CS7k*t7DsTvyNb`@(AtXzBdg)Tq##VYFjm@7N_pt;n+t(j} z;-IZ(Wqi$NYSsB!>4l62cN$TS>UO9KpkP#bP?gaaEjBOP00Uv&N>Xl*rELw8BPwrx zSr26kl*5_9jf-2(HAqhzWuQ{V;Z1xS6>d|irJAtqcUy`u3hNA>%d+3g6QUuWjb=b? zD6&+h-;}3d{YJ(YGD{`^e1+Cm8@dAoD!MC|S7RNR3yuj@#Isv{lQwv5Mj}QC5MJ(n zXhb8u`dM79m^%PkaI~hz<9krKH|l2Ih|Lk1ra%C>;4zW*56IAJ%@#N#^d&QN6p(`g zjR@P>fv%l5~s$f#b#`P?sPVd6@3+J!1q&g^{-D;Yp4G%zzz;vA{)~ zk-$Y|jeKY;DxY?DnHF_t^Sr4RZn(HRB$%vtXMajGZd-WYdblWfS$hTA zZw}jF7w|U9i{|CDUb~O@pmnFFTr^LhD4e5t-H2hS54O5vVKZ1><5MtHI1U2#yp;lu z7fQsTmsu4D3s5PrIZv;dyv%RrjfLUKh6m>-YN7qBp4<+%o>mZGs`B?V-p;U#Pr!$+ z0Dz88(=&%)HfKO*n3UY9shJ7CU``mL!B{1yCT8txY8C;^nxs`8cQt@4D1-eSpw_*;NZ>-s-iFF6PqMIPiKVN=A~hsFAQM7e89%ZPV!iU9ql> zf~8Cq=-gO;GBE6Xnav`qPbNvL*N{FFS1PUIhGt0FxwR1Usiqw+|A_*-Msw8j`LaEe zm0FuXb#{KDXu`U0_g71E9Fog;3m1?nmfP0MMLD5h6W2GNc_^qDXEJAq}l}l z>&FLR{nh}vQ;IP}%UyMiF@WidgN1qfA3%#o0YH-)Oki(I`3-;!25!ue(Mf!dK-J2W z_*oFHspG+pe|-qU8!;3$7)ZN3Kq2BkwX-H03?dB`WyKWcUO)};5gR0b?iiHka4yY? zLpvIJkiP@HE+)dqbS z~KtIb?2nF#}< zYm^ugXS(uwaW%?kIA7Op++WC9w7jTXA$zbxjFS46(U<7#JO*%JxHn@0P0}OgCh3%% z?sL8rvjC8r#`$pvuNA=DVgX2`)wr%5M(9&@v~lz*c&bjhwADiita2Cte2)Ww?<8iZ z)v29EC77zYbAucW(KN48j>7>eGsYR}vZ9y~1lR_GF9&F3yl!KhYgm-`oQa>|VYP7` zI0V}0=6X37++Bf>$@&GN_9HX5+j(E`xU$mM`a@*9)<-5X-|F)@!5jNUJJ$UO1s0vg z-l$0sQZ4Au(!Y+LPtsC>%w9^)mO=MQK?Shid)0r=q+De?6>E!I#)+>Gv(7G{3hVkC zHhh5;y+Y!{o&zE1O~8h;vWw_{54eB&Ot!{{C%GBy;6-+~-3D0i4b=kaW2HSKmO!2?-99JWr`E%lat zvB+q5*k)eI%Dt#M<*{NrfbIieiZvigN*8$H=1bo9TdjSOgV3|M5;1{l>6c6XQff95frhzO+y4={`05SfIY4& z5a{qY9SRujWdsKuR7Tqm7S_mkfT^Fv!es)`()+9^RJa1q4ZIDNUgvYr=kaj=|2b2jb7fE_u)V`tm z$!5;xOOvP~&H>lueg(cDP6D!&3ZwR`ZmhQ>sGBq$sw*~VmCoEUx{Se&6#(#U;&9(0 zgJuK##A6WAr}a~}<4O7JEP(H{Z`5^m?E&C%#$rM;FEJHhw>v{bodZa89nc8ia)7TV zix49Rq~Hu79q4|KFP*05-Lp8k*y7DV7(=^Xx`MBO%00;!E8cQ~ChYLklZPN0#icuP zTz8-ar3=}*N|!_+=Bjv(E~*zCZpnO-7JtQlmgCY&YCQefU2z#I*ZP$(HX_{$=Lox+9RMiTC~Zi88{vwT=y~n zchm~f_$z5_8-jPT=(XXAo(=n&y+{POz;*>4t^PD0u5&?CNzuG5Lnj)zx$fb76}dn{ zK;BcH2;lzwO+(b7sU$D*v2Uswgb#e_`MRv@#%NvedAj~Db?2Y86(}iYsv#21qImAie!d6-D^kal{+m8ap5pR-KYHM*Y*{?*#7T~^_cJjjje0{?r8 zYFA>UKWhQH>y+=jok%@zk(fM5{A{CA|Lft%A6`7=ea?1b{2O}xCzR#7DpU$00r0#P zIiP%q3wn4T0FM&K&&vEX$HihUWB8+qnR{8(-UsME02o{?1HiCCPF30+8l|6NIn)j+ z#fYVV)knqq^}}~J%AB=NB!4yqMsm}nD%AN z>O3@~|J>W^p+9)55JNx;i6}+F|JC&Pf6CwqjCDe?0<<2l?`&HiPJz76gqG>I1#CY> zvBQg7{3{ACykNpyV01qHRp*3(CXs~`jlY7X)6T_+X9Kt((6^;_C?Oqac=6!n#lMHD z{cBMHBqW{*xS8r3YMPouqe(;$#1GG&>81VaoBVCW#fmFF;D{4H%ik^jirIMywC&jN zH7Wkx$n&3+KJFoR3CR@E_U|o*;nz`|G4X&k_4!laXSTGz1bv+Co*nvbmZ-BI4l!c) z%mAKt`ee5M+}?{O6SNO2KmjehJfh-why66wifDl2%j)XC9(dfV{ns_nIE?3Lo~m+T z?0ns~^#BQBmFSH4iM#wIniW9kfnJ==W7^#7hhKgW({}t=s`k~!0T+lMdkOL(Y(A#@ zmTFs(r1Svx#nrlOtzdhFqhoN3yH#*Lg;Km9`q^FH|0 z(#jHc-+d;ltNFxdCob0z^h@4jTDfBiq=F=UX!AW|9!|XJQKa|tyd+^tdNbuqd}>wx z$hY-{>Q-=f6L&H%!b#)m~LJ`8!3%? zSibJcEI@kQ=KKZU|KiGD^gQm4P-CnMtZ2@87YZH@L4Owp$Y`o$hr_SjjR$Iepg&o# zuWwp?N88bBLx=fL48%w=fPU`AIlDzrC2!eY<%FyQB=={ecb1`r2}>BB?VDKc0oz*A zVZu!B2fbK5OuyCwj(~s)BKuHG2+DvHD3#zG1N;5dFs4eN`wm~~ClF5eq@lSqYqPmD zTr4bT`qWdOaX)m)iLSJSyc_HT`q6$~wqx+2iZ=~a_h8)N;tk&* z|0^eFkQIt;ylk-<3vn41SUx55m)l470pR0_=V6C6R4+}MUZS^4s7wWK7tz6`5p*p* z7d+<+V)AoL$dW-l9HGl!xOt)9&X;NMwOL?XuM0_*+2*^p$R0(;umywjKmCKl$mmuL ztyEbH(&oleTXeg^Op}_c^Meyv>5IP59*+fjtSwaS!u_`e3sk_AzkVjkoP8FZ{>A%= zf?4SgxURxk02igUH$C#JPizZ7TU9rs|A3=+LulREN`Kdkb(|AK+tMch9NrzTR`wH* zMFU*-0+-$FR{{gK&dBP7ORh&}{C&&*=F9oteb&GI0lcrb;-s^8H!t)T ze8GQ;+YV4&>HC_l{P+9Ow?!O&0CJcYo(H`$`8H(hpHKL&Ymfd(tQ-h8c8R%u5=06` zMVC7HW~VU`VU}&ecxhjAHIrO67Q3JoQyMZdZLF5Qc>lti%PDaWPSF!n^I_Zd&lYxQ zrgM6m&5g93Kb2(A%NmB=`a&=T`*lcb7f*cLf^mC({D|(8iOG|a_oAS8BZZ3%@6)6> z%{o}8EiSQ1FADDp4kI^~9`4(A2mAh~MQn(Ds^7w28Tjvp${t0E>+&ZWV>pRzez|$< zz?x{|o#J}lM4Qp^^S0&RZy~?pe*9j~e+tcS8vere_gh4Ni_LGb`K<-NwcxiF{MLfs zTJT#7erv&RE%>bkzqR1E7W~$N{};4?l8!)dD&cEI=X*K4klPr-`_JK%Rv>d#tGm2- zS}?@EtS(894tTToc6U%$G_?f6ecLmGc-#K=ix8hXv{aWKY&l@f1YP*#E8$g;aAS|p zC#d+=EuN`4O&o=a;kt$>)1`_XY56;u_BKIWBu!LCxR8a`Zpg8xB#%^Jb>KJT+Z}VH zg&`Y(7K!e(*+rtyyem7;EKX~bIxc^Ca*vNaX?d1xTz-@L_2@^}6WlL}#9cP~UC-rx zNssj$9~we_L@B)F%7r)e=L3gEtRf*xU_BR{Ew#|a)wvnbO@78ioR#5D3zVr#ICIjK za>&xTqY`=tiPbaeVw}?_TrrOul-?6vVFZErYACAh@H&ja`vxXpYhaD2UC$v@-oQ8y z64Jx$L7M_Rpk0)+GnKJ>LEW8p#b70}YR;)O+Z4PcgJ~Pif{Q=E!RZmApbU`(6t|&V`oFMF|S> z>(O2pJ>|n^T;&Ekzc$?5JgaaVoF!%zlk2(N5GPpYNJSOf#a>(Qblf?^?wi;1#b1nC zkkoC;C>E5_Ij-cm+r6*Xw6vQI6AQ{oKc%>t4v|9c?s(?2J+M6Pb+LrRD}wudN0y;S zr{6}z&c0WNS=|R^nM#{?Id`ge`)$5+S1w5|a9wWK(9l?i4S`%R7B08+7b7d<6unT9 zRdy*-l*c~yA&%kcbRNP((7|_!eZ2-_{9I^F)RVS5=AgOA@pP(Ck?M9IKF2C|YKPXV z7&!7H#E|E{p!E-&FYLEI3B}4ESLjkM#6Sj)yX~4r9^Es1-2xL&)UnEGHRMx!^1UbI z4+S??Wnl3%~|PfPq_|nbT3qIA^?3 z=FuN$@Fob3c^;;~y_o3=PQe>T>#JBemJK)>VsmZmf;k+d}j~|aW zr|D3axpa1UYq2k()@vCPn}zmvkV7>Dz5U7tsPdVvyr;P3K*I z8Biv0zd@pV)w!)8%4LT?urUR{YL^{qcR?9uH5n&#er-(I(|f=aa!0W1-SNlyB{9U1 z+Zb!c9aHtvmx@vBK$*PC>f+KcA1OiXm}qr#=@WVu+b#61R+PpYOjWm0^PKY?ZK*ob zJ>12W)JWPqudz9>xCqtlE0LLOd{Ys6@+3%TgYhaP1u%4QLmZ=H9*0H1_Xof<{6C6V zkG?9Bt)<_o2|=5n^pHP5&wh7tu=sY`7(-5(ZcVZ|POQ2-F@=P)E_&lss>+*lyziR^ z@OZdK@Ot5712m7_uv?&$dlw^Zx2x}UMaFKL`&8;`V;A`ixOH!7dfk{Mp>2ir0>k6>K9C=26YCrxY*`|*lE@0t`m%f zIlhB1^18X0tcTfV3Xgkn2IV;o&W1--kQ+qo@0OmNWA~Osgbp+~PpoAZ9yaP6;f_M> zNLq0Fnn1VBj+8*?scRpsChH_&DhLII+syTAN(vn9Dxg?ra3>_|4n%=t)j(LJ_?m|= zn@huJ!TUw(y^&PpIw(psFD*AzOi8KxZ4~$1P`-Wd*qrqM1rx>l@GCjr+PuG4>l9N1 zOtsluO+4R`8&kz6B4tDXkyhC4XHhSQVstZk zaF;sTwB+cAqZ!#J5i>-k_wJZUkTLR@R-2>JX-ya96>{_yUQW$9%uY|Ko(EZw5bFvs zXKHn@=tpHtMzw6MsA91%El`(;@D6F(8NlG<(H2}T!l35%`+Av+YMXI!B%*%1_Mp(6 zkL{Dg>`Q{E8OxdCyb~ae=3{p}yo9Hx#T=Rzhy`s9eYSu2+>iUOMZ(WfaMdXw$slrV7q(#pH;3@)m-vFL*BGK=cyoAY14v}rbE)_h>MWSBB4NwRXia0!!XD7qIi-%a={6qc*cMw z*iw^LgissVj;eb^FUQfQ3OJ@#87M#FY_tjhhg2V-oH6^DI#|sasZ2G_u}u0rUDojf z8L2`X3Gjzixx zaTqVQTO8nzAATOBXvTTe(+>7Yh70dNg*wOhZci)T<0SZte_4o<)zk|u8+`0Q5R%jwXci?Eb!e$#d`+8lR3*QkMo zb#VYT)8zLlGd!`r(Z9Z^xLKq~r9nuUxxkFPkH17uV)LZ#jFcX>t3SmVi%6-#ZGNQ7 z`EfxK(oxjm+$m@z$#LY^+T3^F=;Z3yZr8H`Oso`Bc*xoc#eHVJ6&6~rm zeq*3?b1AcI#~=J~h~_V9{jbsj-#{JM0jFh1RQ$|$nDF%<4fxPFs$eCBFrB^o3Jsm9bPh>*{p_$ALbR1=-|34vU`XwkSn9MuamG*nwxIfGEp-K!R@ths^er7=%CIPYprz$+Z}xswCn6dQvGag498$+k%#yqNCvF5 z5S$!n_b2D{XUdGeBE*~;FMjFDVwMwiblpliZQ>ktnMmg*jCHM0`2s2NnJP7OguFDP zwY!uQCIO89IL|k#pH;I=cS?@Z}#fJby{HWb#NV_L_&V!tUg)Y9Eu+kS`NC z-qj1+`)5|0UDXB)Hmb4|BdF!4o0?NL?loLM71QJ6LKLv_l-4)!U2IvZq%Dx6U?d#v zjYxjy_70?TXte&j2?DiV+>=ZkJ)KR&+1(|Z)W5p z`_+4(iaQDoD;H8%NFs`lS9q1$^cdE;?F$T>lbfwcBj2dcH0zM;u!# zq!pp7tXa7nJ%{$^+eNp)oOP2wI1q^C9&`soI9T@>;Rlq@VyKpXG7@Je}M9b z)?#msIc7f?`))$$@V)iv3FOV!8gzNWOJ~=!f?Qk!jrE2yw;#6zmn05ly9X)|3N1F<06Y(96K}zPLn>9M>ZFggJSeBGp zal+;tVgk7vH%o!Dj=&Fvu0;ihyJ`hfZ+hHAG{AQqIzivFzTXTIR1pgZtlYANk9pf{ zv65nsSTY<|v(w(Rtm&zYiBDV=yw(Z6W`9U0wxbOu4N8^q-whx3y8SI2|Ml#KC;)`n zn-lQ*NT!R$q=?lu#=sOl)H1(R)w9<3?@undFisx^((S+%fX=5RXE-%{bg#+lMLgRx zjTXMylfa%#<{~ts4Lx3zr)dD!nR4v|<NC^l-wH zQGw_AMh3aFC3!n7af-g4Lv!RL9K1MS1Ak8+uK%`ke=h30gpepNG^1HIhEo7smq{>g zpsS*J!+EkgvYn;&=rH<#3(76PUP-z(ZhLfeDJFPUM(2j1XZTHBFX_}5E4g*T{;#(0 zo%i%l9n;r^)Y6Ws@Hd0?>AD|U+1T24`r6NXD8yosJ}wZAsw>^~Y-EiAyZMQ{5ihiP zy!w)Qp1qp;XdydT(-tp4EEl3RdBTymH!yKHfocI`&7JDT&OOW!N%8iA7Yl(NH-(>LnA0D0slCy-_NAHO= z*O@US-$mhGe4r}894^Zvb^qaFuy~1W_6*4b_2#gTqEyis7au4GUWGPIJ5>$*5#|K_ z-1IOg*o!M;%hW{_*?Vjj1XvUU+tI!N-RQc%$6ampzw=Q4DjeIH%2GTo z;sYM5eHmZ|QElOIlt{zSWklvGb}EN>5a$#bNoK3dO}cOd;*^o+7K4|2*LUns8Y+i} z+fmD}hcGGE)VR2?gHDvL`Q#dW&bSqyeYTJ(+V;4Y1e~TKKy~F%=bpG6d7YRirm^m5 z-#xBj(;JPUjhr%;+(`p~KCNfTR6R32Eu58UfjDcaVsZX6g%#2CYVx&$^5q)|YoS>L z?7jJVpWz#%BNx9f3*U$@0|L^XYXbwvg+-E33iGWdd>=^NYbRYVBA45NRHjW4I%DK7 z;AexF!wQ~FDYj;1QRsh=>>qTzH9X=W$x*__I~5h&?9$>$Isvun&^h8 z4WV_Lls)r&>Q>F@sBA$)ThN&svg)1X$SaaE7Oxm1sC3>p7KU8NsOM)Zo%6ApwvO(% zZ}wW1I?~hvi#E|B4OhEtBm3PJx23orR5=i_B^Lej&^ApmBz~`XL98e zpV`F@VdLV(tPX>W*6L2Ulpo zm~CS}`zmC;@9Z`TUP8{*K{a52azjoTn@8@+wLcwZ!LuMQdVIq&-D74^1AMoXiv7zv z{sd!;v_X-@-BBg}nwnccEJo?=EgEJdviXBn^okCkG$z+5&2`j@fJ2gTF|FiB^Tt0j zGHbFo%xS%U6^gK8`sB$ZO`ZvkDs|jFGyH}^Y?xP2FWF?3v-t7akRZ=uPuBQQZ!P1G z4qcZ~07&w%s`8js%Bay)uM`n@ByHA^60e|XR|-3wJE}weDYns# z*+b5V#jTJoaPkVBw>L`pyjkdxR6dk0RT{wcKJ_L4=opz`ix>LFsu}F$saO7`{U)_w ztQ2nZ-0ZAswn=w}Fi~~Gli0M(r4heR)?u=Yvb2Nw=^Sf~vM)H~}X7z}%kUb$v6*(bo3nnEN@YPF4`=bPuD=^?^T%cq>1cu>?@VjO&TD%r_l7 zZy#-h4%CAPCj?g@%bWPfZSfB%KOvcs>7Lcy(W$AK@lnqp%8idx9P(6DjfK z-SZg(e{MflA0`d2%3W2JnaLMAwFY@)GcWZy+_^hQf(3h`HL%_ZFuONA z84UtvU(RKgKHgj$jF*?Xq3E-3l>QFFT-`J8lot2OXmu9((SNCMJqaD9n=C!vJg9$; z%h^v!@*^Ali>@v}<`Tv*8=0YQ%LO+wD~ONIi2!C2*hjf`A3(Zz{r!%%t=b!yD+%>i zUyzCLIguL#&20LJBI6%Bhn*%Htb+MNCbhbT2RR?*u^50aq zbca>>W=8UV*f^E<&UwLxR9~23>A~y{0@(z$}?9GQ* zwT{=t*{M^}?YtK{yHZkO_rN?TvV%Es8N{pUI@o`enB~>c)Hu|#TxB3j@A=W}7^Pdj z16}AZ7|ttl9ZaFEW}g~rjHiYbp+F9D)6^5uh}UtCEEl(g7+7#(NyHK1%TgOpMaUv4 zZd*{Y%A*z;<+%476_FA@&}qym^^w@6&v0ititj4Y*T07xNc57A_rkvag|_Ca9-*Fd zK%)PuT>xHTjRBP0KcN?W!$A>OdTglVQSV)5#L3%Zi28jSv_p_fueQ! z;c!dDkDOD$_FQPe6>u+naGc{Ht3B6suG`1$j9e{|`0XS+ct@N(w%QWwlAvCH&^ghF zzR2UM`UjyAU}$I4hueIPc?(Pil0xIvsqj@}5NNosUH85^p^kG~Wvjfp)F`RmwPC(& zz40l=A;yRw3CX4Ffa>Fum3kkuNrj!o<;^JExjG5C`Z8cB;HrbBziz!gLbm(CR(G~c z=JsSdZ$=AN-^04g$M5u5xv^rVpThzUkGTDQ6sS!N4c=U56}`r$LaZx92f`5+N%$aC z{iG-Uq`b={`$rc-@jYX#nXuN~6pfWE#-TogvCitE3XG4G@Ngy9Rp#m5tvwbQQ?W+~B+kmCf{l;8wenL85Oa{Z zFKrnY2y^{tOeUI+K&7}!L`nSz84_p^_JmQ)?R*=F@b6J-8P#9Bx>c01a<9k9Ov+~g zm)`QJpC42#L_Abo)sQhH_QXlc%j8u}+My2|XU?!vvX!m{<+s2lHTxdyoS-Dr5qxd- zh@S&f;*O_LtQf7=Ie9f28S*cNZ~1%@OBbCYda@qR%+t+$dFvCsy2~%%ln$0{9&*~o zpZxpXLXXNp(l6si}{Bjj1DVRzAlx7MK;(z(*fVH_^8-NI@j>&H|I^1U+cpIaM+2&k zWx*jK_~tR`pHtZt3_Lql&3;7RZF(AV8<9a)>!X^DS3xoIE~KrDbB2>=@1|omiZz&e z3!aqj?bo|1%Gt#u5At@Y!M0h+;jv5io|Ef8Q==At<7gs78L@3Z!2Gx)g#ucNQNOe6 zE0C|gX00J9Q2J#%H=@l*vMyOtPo4DYE&ck^`*u!DD+{n|=C7lpsZ;7eaI4MajDbRg zlapZRNtDt1@xrlJ2FclT4sU#{6&qGxM0gTCcq8{kQiLIKEK0e#@jlqPX=W;5#alta z*=k})n0hFGVI_R3f{=mRZB;QHViTGcn8HFGXf49s1y+5ZhOmD0e$&0RcT=ZySQfUW z8Q7*|@qUqfuVZ8+L+ad(o68RSCc!}(+Yy&*avoIX5H`yQT}_@p*Ibp3Sk2$EmXO;f zQnWJ=PkZT!yPZAB;2&3~A%1~EH8dE`(L|pRc=Z|{huPco2JtvIdjM}}_ z?25=^B1T1}jtys`mgYn(7xKCnLpxvdk~;(7{^l>j2&##-HG3+#SJBdAks=~d?pIpp zw+gjxR~45rCy)6A$d8wn33oZ)4ybZ^BT{Pccw)T&G_P}fEkTZIC!`^)*3%b(4b zH1bV*Kj~82^?7UwP9*DU6}pzcDy1WxNw6QT>tQ$dSfLn)6mtya&3UHoe8FEW?annp znT@Bn6CW+)6|^L;hUvDC*a3n6F|dEonGB1bpOHe*5h}9vgkR#>X1fe#abim_VJCXFJ2~PgU}*#O0V5IU0~j~ zoMCroMKosKklV7tiLHxbG9@bFGA{bK=w_P5^j{|smLfKi1roD zytM^@0`lE&oGk9|HMS(#T0~bQ&I`JZz}|3`GAttM?3;Z;Y0L-4de|3Vd!(gB6sW%e zYiQ!)`FK&hJEqzi6Msq-b;|WNr}B|g4_iDcRwhX)bGAYjJ;wqnv^*XMCFivz1%~Vr zX{_v~4hjz2UwbRA)g}65<$TK4I$zdW=bI4Dlt0V#=ZxrIR!hY#4aj- z3IfmEZoC|5Vf=i(6jtZT9{}EW5qukK%xaC;ln7@tqR(7JmAorQZPC?NE&1d{6qG+v z)moW)GgihcbLFF*>gQYIWo4dJDr+{Sv_q3x9_w7v;t^f<-jH29>(v>piHg?J0KWtM z8EN9wwJ%kvfx5xk!r(nVxMw4j(`-!mU@fC)EAED^nIQ?0TJKyKJr9pbUv5VH=Fnx- zP5QAi2bKyLbN8N&5?GZKw)y|G_nlEqrrW<~)ERZqaTEcSrhp(IARxWzC`}=#p-2fR z2#E9&0tAx7Sdc!UcNCCbq(dOVLXj>YEs#VBHIPI~2uUE^m-(M>cfz@Mt^4iDH`dEy zJ!O~Q-ut)r6EAG5R=*qyxjVPp?7Ys3H>to_azMy9%0Jv8@lhA=ATXxNx$r`cI{4OHpVjYr%h@ z`Rg;6+aXK06}06$u}aBvyPk>k59Wo$C0PN;=wi#{g!0{SquJ=2pA|Z7a7jmSY0jE$ zeTJbJ7#%l0*q+t3ygofzW)``yzN-*|-j&Unbcm$)-4|OV@bPz?UESUxxw+=hb1^#y zO?mmcGHK?$nhysv$hZZYmXCgbtt!_79Sj3Y{E5g5i+mNUl{p~Ov-VNe>Uikmj;K+o z{(88l?mkDUECo2bc@H|bhD6Oj9$1UNb{@sGMe!;q#FqExyZGHjl?UB>bk6={L951y?;XgWV+=RG&1hvuOcHM0?t zBdVvW5O1_nMWg9iO6h?W?Wzfiq=&nDsN!k$ArFjcCWdCin81w2ehmkX|c>~R+ zg&-G@Y|F47gMOlQ`zvlUIiM$JR2MZ#S@M~N4=y>Z7hjahVMQ9*BmG<5@ydRO@7WMA zkKcE(Sn`PWn2jnhHxUD(1r4dsW4n=#0~G_$xS^zQx1$@ImBACMQaK@}60tk1I?aqn zB%_>QsKl#ESr)rRax!50JHf*J@XYCUg_}8evpB&k;O8ak*cBN)9Gvq`*7eEDNS03H zZeH%cK5nn(-# zsrywe@{5i|%FpZ` z7qTnqY~-OXt3LDHScG^r-SwW8ji|9zg>Hm}{MvC10ho!iy<(57WlD5I`%(%|(J&%U z6;n3OF+h!6td2?7c;k~7-X-mQ|CjkG%kukS!|J18d2*7Dd(Xf8%fEeVV&sex+g{oI z)zHbyS-b(&nt_CDf2hg12GI2@CzzY_a(1zoIKe-{gfDl~YdM<$SvTkw+|( zvee%xVk=Oj%dsRf55ixlR>4UV=mz`LrI#JSt10(am!LZaA7*s)QEhzcyTS}I^`)L{ zv52UY|FLp+!*^LhskHj@%{)W#?IMzIZj%b^~`+QF&}_GOLV{Bra}+#7wX6EdO$t=HBsQZ{Q^Z@LkiO8#+8&frL!#qg&~mp1jF>B>Wl!PM_)bx)wEC}#%o%q2UnwDyy4Z~Et;2`y8=xcT6Af1y z_IV2>JXq#4%R6*o>3}KMxejTq*NEi$DN^bY1+QvniLyCZe9W;nQq=oi?EP^`P&EA3 zt+^?&;JIdKHGJ8hV8Iu>G-kToHg<3O;PnN%!`7FY6=tfzuyr(L^Jd_HmcfhY+YO%9 zjOS?8Sl%n}d(E0GePzT67~soTqYMKgu;e8UKQ z#^I^Q;*YiM^t`{_rl~3qDp=8_ek;nd=3iQQ$zCt^&N$Ls6?S<=PLXQ8I^7!JFnL|i zpYve4mqBQ$MZ2&!Z^J~)MXd`kI|6N&pwJzg2bkjL?M)a6a4`xdRYFB=h5Gd=rnZL8 z@>or6JIq>-+_D~6;2gaKHi}bm?m-GW#JgY6MOWFY)cNkT*hXewD)hbgjfCdkHJnaW zmWrCRLhO>##|`aq#1o#_4Pu_Q$oQD9tW~9sENsE53$F8-%>KL>Jeaec6hy>M<1XN{Siuw)%gxz zWxSW>ZKj5`-SgEiExk?aY0=w6y?;N|84ba1=Q;6$Yo}9=L9})z?G$+g*)It~|cO!XP_;;N~3cemZ*RISo^tGYVMDJW( zs>*)rs=|*y+Wl&iuqsLnN^OjbiP@$G9o-@|PS|F!EUT;qi+*jB-F%+{a*u1@@-JME zKYlemT4p~4aQ@CfxJDXRV$>DGLm}|PDM|X{4)v4BnvFzy*$@t^PE8$3qPpwy! z2-I>xg;Ni6CSja+>*SfuQO0Ha@AE0V*UPr&4aB~gpSmtc^2DP#^Xe5eKHqaQKl?Vj z5``pLXWca#7MBpaYjeG}x2WmGxyH#|OQpdQZLO$)@h4)B^LIOsmM{n^tyfQpN@cIG zzizM1H4=h$YNZm7;2uw@FDZlYF(XdvK~gVuCw2(lOAF_u9>hs!Rrh9<5ns;Ej+?2T ztW$jb`t@-FdAF+GnVMLZak@8Qke z*YDp8MyT4a)m(y)htKgM%sP6d%H3b*fj7(fQ#2L+Si$6bgIBT{?QHt(8-n9>Q>Bx2 zLwDm6!-;AM2028dW1{%=Ei2TsFzS}de&l`4>(`VXOHJ#$b4%uUo-*QDb1=9^y0MIo zo=s_sJ@hGDSS#?@c!z3P{IfQ1OMR?5tG9pV929kvX$w`1W$j^D@euZN^h9Txg~g7I zgc0XD#6y;jTB$85E?pPv_O=QO4Z(fLH!IF|3e%&XMBNNn_9?YL)7@yhoPQ#&iCo(2 zWfnXe|665=!qFLI%);!4AP}`8URT|5DutdKohZKh{cFb1#aQsV2v{x@w>X^M92GHt z{6^;3E!#$em3UZeTP-<5|3I+#NiDxWfQK%*9pWL)yL-nB;8SkNXz=fg6w#o-TXWYS zBcBYIk0lznpR|uk7Ji|R7RNvPe8Pa-lSy`TdMm)Fv{`?KYob=F>HldBgV>=)Xx{i% zj`tIi08kQO1OU$Ei%ZewSP8fKtF zz`JFsC3fMFO|JJWoJi=IaJE_<897({yb~2oC%o=oCJz7c*waY-!d2RUK$q=Ce#Y0F zoEneSI>M07Twb}(#Vf`8PmXGdOt4gj;Vz zox}O$jt(fsU}hqo5i@Hw+F{zD3mV-f8P%~}Y6DSU&qn0D;p=y_tZe&Hpf0%EGn6^|Jmfri)^83me&`nUUL&zFsLK zaIoX+SEE{NAsDv&sgmfibhS5$cqQAHjGIU0d3Hy)l+ElwNt7zHC2`vZJ3l;U*$olN zrqk$<0utG&P~B)1O*&vAA z3mG&iUwSv$ku!9)8}994aarT7X*5yGaqx+6g}-heL5szIJU*rs8qf`h-kuUK&MiZA zlJ%jtATj^*X!$=Ty2F{$K%9;ssMT`IvwQO0ulDO7Ho|F^1Q#(kZN1SPOutj1eI_Hu zqgac6`PAjyopKIGpEqXM=r?%xAt*aKKc3-*udCF(?um@imE1oXvsw6j%>X~1^q6MRbmfpyP-m}19p1(DIW}aKY6Su&aK&QEV zacF^#uwJ{>c@yen2t(TE2QuWMj4wWIIHhM7<54OIL(?tpZo?p=8p(!Z9WfQOzjWMJ zYUY>gF^qRwsnNn#zuB{r4@Tp*b`qgsb7I+d1oHRm*GK$n^h%!KUgwqB)OT7*DX6UD_# zRF6suj1uF=v3O^-c}S-C^QujW?#j|oL`xUqRhy?L6&NuZCFstMNwo6DFyU6?ou*>v zX`I#L_21m)6Rwjc;bxSrkt0nU+wrg~0?ZT_$jzGNLF~pGfre1l(w+L{H{u;s!`Raw z6=Dz$!nDt}jxp+Z732lZO3cczVAXWfw~_u%kK2#|V(D^GU+^>xZQTN>Rz`)l?&u`k zkXRwJz=)N1BU(s=aBRc8nHT7aV*44Q{A-NRtU3*@OmPa|ONkynx=vhcqpnNFNelXz zUCQpsZW!^CGo8MMoe{ioBb9SLB=+fV?z$yqswGMZ-#B4_ayyhRmB@|eo)ek|&~gTY zPoEWAC|o$xY`R=@Km7$GG$6`U$|D85`qb!_cJPOP7r%V;X3%i>yH9Oe_ZZ%{OBg1n zQ5}o`YbzU?H4ynpw2DAlYk9|n^m7QDQV$2|tsBQG+N(W0EDfa#O|H)rw}pO4J1G0h zVZ+KxQM8+PNrQSFxp#AG2kl4HV{1`gwQ6&do+Mr`Gv6=?JSAN1Y8p7?uR;KnZlGYB zmd}HO@gYyDa&$d7#%{yiqy1Xu`*hx1u>z_|1IbzsMYzE72Pyv(r5oK3^&9y{I{p5& zX)2<@%|uWp;ukeLKKxaV35(Y_N}<>@Df)Q5qUW)fhdPc+^O+`IcOndGm$f0^L!Uln z)O#+fXZ!c2^tMela70#$RZmKv^3q#>5Cq4nB;00OR^WwG3{57SqHyV3voi;+?0h?f zm51oXx@BZ>eQQwGv2*Vs&bm^Fx|Cy@rRz@>{!afMPHkwDC_+K>hqkQ}d`UgsG1EX! z=h2vUsuKlscsg6XJ7l?#PE>zYP~p()801P}Cq+N^6gGF|vemr|j=#Z!du{i#m1|W` za)ZRLJP$Yzfpc$K&nnb;zB{}4c(TSB?<+CqPexak=20S}`a4@Gg*0UI%l1AOMg3dR zBDR-VC54KK6($|fp#n;guEFrQTs^p(xm>RZK5E3QEV-KLm8o981|Fg0CvOHfJ%?Q` z5~v>@&x-1X@Ytl8S@9^cZvgpFTw7W&rvtd?Upl+9bK?^lXc)G^*^(%uUn+J?$YG@v z(|u~C@3Kw;!Ii7m^GGg8DBSu~@v!)^!GJWfTHZx^9Qn7g<$P9U?xVz%5B^>kx~2J)p;0dD1-KN;E6i8seB&VO z=&bFr!;zC9LSAI*wZ*(v_?m2CMQng(Mv!)u%N)B$8H)QtR zi`XO#eP3mKMgr+BBY3|JR;~!UypPN??kFBOF=hZiqA|sY7NM$x$6Hh zJ^T_f@iG@>JN6TN>`=;jy*FaPj&X}1rQpzJKyyNTsT(?-d5{I|(94q{a_*;?i9PzdCfU3%0- zN|vpRi^AEu;x&`(V>nQrC7U(2NlB%8s06Wm=~C#F;3ky%HgQDM-A^aWf8b}S0J#;7 ze50iyNQtJnlE0|HAl-VkT<1a$KV`XssxvL1hRIz=4rUNo3m|tmZ4rKw))7M$Xn~1Z z%MmhNbVBT`{&B7Ub3G9ez>YqWbZswk+tQu$35`$8ei5#8QGM_Hp}XD&C7xcR%2Ux= zF-paqP4$DqQRs}obbn2(+dBzalEiq6)MP>Iw~vha>RM}xj7Z6ikPlCKf?XvnRU!T$ zXP)jU)0VWiet+lK_@Z;p(T1JC>7t2#%rvpNuJ(0iTH4sq3DwIdg76uGjQ5B;rwjk} zFQq5OH?NzkQrP_B1G_q9VM4W2i5OTCGRr|bzx!4GFaGf19jfd0}`^VmUYarn(`I7o|R+#9WwE6|;z$0rr0dtnSrXe&(XCUmz`@2Hgoiq(C# zGF-8iEBCj5HUT%p7up&9BK-5rep5Fq0M^7*PYJgnhQR3EdlYFnafb@^XVulJAN4ne ze6W!pzF@jC#gJGz+YQ@D>aSh$dqV4eWigO&oq_+i+qU+XNC@1F&x2EZT4T;F_ZY6_ zNk6Yx$GVl+q`$>6fyhrPPeC7jTrz-Tc6YJ+EvyPK0rRVjzE6v%_7kH+IH0yu7nTyj zEqGf8Jpc*_hBr^mRjcr}F>vlxcrdSW*(l208eZcfQ)+Gem_n}9vrw4}4Y2(QtT8>t z`E6?8H_rpU5+MW&*e!2VbOeYVc$r2+F6Q|SiQZ&8+>-9^CGrOL|(*FM3@%DI*`9Q?HMPx;bAnbA2TUz5B3f%XH&N{i+`u1%)eck8>;J+uF? z?tqVN6a8YHLZp(9q5bH%#ttC|t6fB0LxK^9?TcC)|1j^4+Gci$S>*yVNY|^JO5Cr>F9vi+|f2GG{ndQVg;FM#yZN=-3V#e<|wt44^e%=D5e-IqA&JXrod7aifp(tie z-a_9THELiFO$1gh{QH__J1`rLxrOG{ZrO~f3s73@WgP~^xpHWd`Dc#4$~Du6I!nAq z_!Q%I-T8JzN@^cr6{jM*{7k28k~I&`#=!RDJOQgtqwO)q*uqzl|Lkbb zUoj0o9hn4$nB$puPx?b3(<`G3Q{;q2Kag2gEK}bRZ}!Z0{WX|lx&E*ES_z+?3&Lte zQlmODu#}H#bxK7)Lpr8N+3Ji-T1Q)Z-ilf>Vvs31AuThQj|e1nx3 z9rrH@^Q+G>FcG3Qc)t>)>w!d82U*)TbBSx~YsHZQyG%Pvex`IqfQqf6Z^kzUxf4|V z!nrp^cnx_;dF^YCy;`G~RQIcTqX^ye3BQR34`>Kv`CKxv$A&Ub0=VE=Ml}`V;`4nV zBQ4KY=5bF)6uQYEtTKu?RkM_^&}Ss}ctiAsFV1bx24 z<@h1j+)7Og`hi(|u3MHS&Fr6}Lvqf=R;BwLmQ#(1ST9$9MhF^-zE9fM8_91~@KPRu#IBUirc5A&s2EjxZ0jJsx0XrQ!sVJoR#>XX6R zJYR*01nB3T=eQ=j=_dDuczXvywf=vrqZk`#c|TAhE8AIW#TqKP{sss$9jHmcT;&WH zd-dnL0z@_LC{M#LTr}v~gSvMNEEF#=Gusczzujdwx83A#Ri*1};x?rK2TBPA=2?VR z(l&ihkv&W-PUi~Qs%~ojc3vClF==9->TI^oYvuG^eP?4?^qf$uujImOS_w~*#_skP z+eU)YsaNdfya>an3aFy2Wx2ZMsS;5OfmHuhwW&sLB1Awfo8?MpC`2`D7gff01JmoI z;g;FGp(TxsfmAK$9B!8ZL{{q%AYT{S_H>>b4Z2X;XOFBGIijvNUY@D`S{b_Vvw%vp zn<0d1tWZoT$~eDNdyaV%GG)}d&GcuDf067NrH|a&*+X$r=H(T!{Y}K`4-N7}TC5K4?Fano zGJnAFzS%=@m!dmcMzRHY$sRjU?kK)V( zq1-|qPzNtC>m&z%)#Y3=26OQCJ$d`-y67wzg`lz4z3K>LE;mW)k?U)#WP{}7jxsn< zDfBS6&QwT|vmMJu4-2}t^mk;#@ac`?ok!E0!#ARPQx)_aB_*pJJEHRaW$bOIU?Kz? zLEfKAisQU5dcFUXdUi+WTX30Gu87txcjLN}v;d&+uwv~*0}r?bc}%7)>pN90w7#NTO-1AFLKeu@VMV6RGYpDR{w=7Xs4>eK|p=5vU0)+$5dYOVs9}!{$niZV&&tM%#o8whxj3UZ}TW+wEmmWz;sdtjZn< zgJiKL`Cjtc58e;hxPe4Ux(s0!C-mZ7*HHbexXYBx7pe}d5*I}_xhKnOl7X2VtdZJ% zRLgQ3>$uRyBX3-yj;=>MA+UWlfy%@3yEh&F4GZsQ zvHLVq_yp~d)mv$8qg)_t_|AdmQclQj$4!TOtGq=zEMxDq;*iogSu6V>7m2vhoVX@e zGlXlcVx>^)hOFF+~%pT#bFB6BRPcD3J?%5&4t31%JWZBy+?H>RX{(n?j#UM2<} z&L^t>?l)s5aU8kSklDs)h!i&%pN3h{@s~=>giQ5mCRuBOv@e5(ehZ@UITq_>>dcL_ z`IDq^AWVY3BU9DCvYQ;>{DF-s7lNm+7v6tN%NEPxP6r17vCZkfj^V_TxQ&wZQ9m+^ zn(2ow$9$<5@2P36fsMWcSRhb(FQA@H1$K2Br^ItD1#HckruB*^KO6lXeFz;t+R>@W zp9+CYiAh(r$u}gkI!jP^$Xr{2m`;tn?mVMjhcUe=2=yf|q+FZz)NODb`ZFG=(woEM zGyAuvIng<1j6aIX-t%>=Z7MD^c(Gmv1L6of-o`doX&ErG`4e-5%_F&WYH)YNOm96B zsd$}k9t9?n!->%_^2b;m$9uvBGmFL@qQ6fcg^&Hz+f4a4jfCiTW-eEI2gciK^giDt zSYv;Bu@Bf=3Zbz0DKZ(d0i*gnm89h!K4q0OBdwNOQbCE;V2q zHpEk?G#KP6zd|NQ6?~id2BnC*Z9%NOtasDVP1SS@nTA+7!CmxrEWq9%tS;LqP$U437pFoqfo0T$rYVr2?_f`P6Ca62Ih=VuDB8Pl`oTyOE z01-@hxM7XGq;AEpececvhSBb5uAMMA1s@{5arem}s%zIoLRbfDP4m6wNMamAt~L1$ z;kt6$=IOR;7*QD@E1yyC_9eR|gf5K_e(zv->aj!2U3RxV7CQc^(xL3Q_W7?jo<3FE z+^lN80!^y2?Cu2G+uzXti>dgL(b>cAJA5BUyGlC%I+a zHji?f>?lZIohH7m#nH7Y+T&u+ib%^le70!S=n;sk&{{-aR_)RxD7U&ZL_MY&RwE9N zs!6{DLvw#GVqs-K2Oxs9wFJr6re*i!MJ(IdUwSx-_J(^zk}*E`M`Yn?ty>f+%-Tkm z_EOe}$a(=X>a!qe(C8L3&FHG02DR(c=fc~TUjh|x{Ip3p`{$?}_gEGf*!sNAmF?}3Gx*^?}z`E+B1EO-J8#|~xboGSJ8mX>n zx!O5-AgW2qV0hMAA2b+~8*O-8b4Y(cT-vG%r>nk|ANhx*3{kU*lS?|?six;+^X=wC zcCR?~B+7_R$dB)(y$lqM`%+$3n#Tf(NP87iE4-*StzR!_#Acb#2lz?E>*|g9+Lx1F zv$Y<7yPahjGCJ13yj%GE&V}`B=M&C5py22+AQageSMTumf8>MkEcm210bap+K5YK} zO+Q%_V4^01WfWez&Xugj)Hjr}bJ^o|%IZO`?#B%b!}e;r;?>Nmdv`gsA(g`pI)8oH z(F@_1yU!~%i>)Y*@@$EQ5WOVmv9X9f=5?j}Lxwa2@8(uME8w)Fr={w$0&Qw{%EgLS z7sXUeA%)NWwQxEb#JP4i6o#CXzydU2zCs80;K^#n&292w5@{*w&#xk0S?5CbU~2GN z>SshPL&U~A|DW3C{!^h9(got-6S;8DH(c=HE9Y=e+p)0bD&%?4m`SRBKJM9PgDzB5 zz#rQ|&JwqdODH(dOkh)a94kW5BFqq4Nh4XpmrRy@`CkcS4kbmJna}#sA8C- zL@UOnmF3%e&*n6EC6TU4anG zN($c;dwd1s+5Tmp&Y6FOA3pTkwu8c}e@N~BM_o2~jvcR@vWcBtpqVb>96}bcH~;wJ z-soiy>%MAO{zA0^?)fMgD<9VxBTO0KO2Q293=#Ft5*GoBVTFk_J)af zfcNTs7N^A>N;7YyM+toC_%aH6m%|tQuGwwaBl~(jwrnxPQPx*k$cdKy@By9+KKnbx zh}aXUHn>*@cai$L7_WU#M;0i-mdzQA9PjJMwy2b!0CEj5P-9sbMf}1B#liw}`BX(9 zRaWb?o?v&FMY6;1MYB6@z=L~FS$%5Zp9WeO5u=ssGt`}tm;oIDhqKaD_L92r+B)T2 z(U4V7vjS^sq*B**QA&h4b0VS?GPny={zb3O(9oFeTiZu^TynK3k3JjF*24yI&;>c9 z&>*f3L^pBH;pu(__in&6PAXT$affjKem4sgS3v_@C*T6Dd=1CFwyMb3nRMR|Ux{5u zGe0UH{oN_lPlAFV1Gl5r0MDv;DyS=D{ABQD`uDv=!#n$I_xu;U389xD#Fc zt9=!j^1)z5^`>13!ru^ILLK4fu^m3{dtfPl5E8aJD0E+Sl(rr`h{J}lsi-G|`VWIg z%*S)1uev5&7j%Ei0MS8D_8B|~1o}XYXGo;-pJJFju@3h=)UcPS0X&Gkj-7>B)QEgz z9Ip~4FRM*Z3dX_Rfe!jtO*E})jVyc2jM=JHu;)`umj|Ms)w}A0h+UF#S&pUy4E?su zQWW`TUon7g-dNcd9Hrx?+I8sM(bv9a4&XOrCaRcG(O{wgIw)64ese=mA$Ss~dJRmJ z)W?n}!6-tATSB0g3ps;nh)g{V2YgVGqi}D8Dugw;8WQhxov3Bc40nb!FE{?ynlr#) zwz@+0+F&DSlRFy*e%t29hJ|kb;#vNlpp~puvfrbrK(Pm(uK?k0Y9;w!@-na|>%+pz zM9t~nk}{jl3pSm76INoxtZ2k}aEv-|guZs`;5%;TMJ(t$PIv2rPDBHpl4w-S##bo;SZDV$wRy2HnS^9r%Qtw@ z7C)drQ|HW;7Gj7MWIIVLSpn1>WH0d|CkedzHfjAR>dH9%Yv0s(EdVA9f1OR5*YETF~ZdE)yp`uom+6&CrpHwCqcgr}A@8YJ|9x(zm*eaPUX% z+3Mu<$l1ZHVWa^>;{1Y{0KC+5uemU-xb21ob}wPjtKX&D($`TNrKIl%L)_A4-!|yi zk>oWV__(UOdkImpl^^8Y#=G1k2|9vhg-i6_P{B9EDT^}7_u51t=9fdfWTJWfdrm0$ zG2dR(kbFaap?cNvf5 z<;S`FBQKSUa{s@QmvHKUU5W*FU~uI3;*&KaO{Q$MxfUew@#L#U($4>9iE8cv0siDk#?)tf7$Br?pJb(83 z*s%*T$Bv!mV?0g&3Bdj&j{fPm>ucpF$KX9&i}Y{KIz88SJ$8)a(&5K(OIRQMPaYjp zdG=V#>-Y-xtdEhdujm?>_&joU;X2HK%@NX9Un+5-7!M|DXe{~jgz4mvDgO8P*{VPIz8^q`)yzHwYG%FSn7n9w- zois?UEqE27K!=oFSN!20I(cEYGC8CSG$jj2jwUD%!}ROEC-2;!0#@*Yx?{F~N22 z`0=)Q)ZD0}hT|C0y#zw1uAc>LXhgx#xY7v9}Bj>?hv(O#{dO`-nN_u)>#hneaN z7)pW*3dwQ-!!1tj5r0EiUled`Jt}bOzhALRiZ1=<-Vp)#;KV6*A-xCWN(!u4MtxI2 zsd~y^B~k7{cg&eN6>powJNnQWX(bA?3J;`jt2)Dg+AO+(1iGt3XpuR;i&KphmMZh{ z)|t2D#oIT~@2q_wV@m6};}Q`)&y+ta735!89@@ zh4o8jD(whjh2bEe3a&%lyOxF3Hr**_WCN{zcg}mWTNro70dJy91>WoL?Nkm%g@dP>z2Ll_ zX!(@!>l{n-&Zaau!ZxhUb5{_M3)rh5k!jurrjKGAmEI}=`;(5pe0pT_u+q8Vr%Y5l z;`dpFkA_OLc_Zt!JR8oHv*epnD?cSxV^!ne)fwOC9CLLq{>=1bc31#=rX1wkS`)?j zwn{djpscom9M00+iPQmYj2!Q9GO?0xkyse6@5RW;A40Aqk+w2fhpXOehFCJp_iXMX z*HuFy1uHf7+jHGXjtTV_&v5lsxy|O4O88=qbCadCV)mJnr1JHFB{W&IK4|7!n6-uvU}qu_TuwZ%{Y?&dF^ZJzV0DzB*ngekYpKaX(IEbsEgSXS>B8YEln z0wYI2j^>goG>`rujBkXE5*frk!Cz8YGyT&((@lGPOb}0@r1ok=D72B?Wa@~p&}GQ3 zy7Yb-kbp-j4-K_CqD!K}9dH!cCq2?E4tmtjwXp1Ibk7IDI7@45`}C^7mORdV?8w3& zF-qKbQD3*5fpV!jQ{`~|2PIlb!t9 z|I{ZIvSSb>s60!Ia-ST;s#XjSMXhFW7J{dL`VoebqCAes_eZG+1`j8@g6!VE_oKI` z&t<4=58vwUfuqW5%9*go3&W7Q3dI9)B*kN^<0l>_*@fJ|)Z8_G{i6RiR;3LlpsIVY zSl=oMv)WjTixoDi+>Zyfl?g`~lMik@k&hY?UwgbS9%|-d626WeVQs5J*ygl>1UJVk|}Ng&D+X~Dw>$0D!5!-9S6K? zEm|vLK{_K=I{md*M#E3@n1u25pE;QV$u6x))!sUF+ts{2*^U&_DzZ(it5TICu2aDM zJvW4No<^-ypObLUmQYXyhi{hMYKIOE#?jQtB@^oHPJ2TqPUNAmJ4JKg&3qTP#JD9t z4jX>Ocq9JV(qv5R4aSR{f+aRjSI0Obm=$3dPG=6K&0^ zMg5UcyBb^!2Zn;mB*taDr&$AJ1=UL+0DST$&5#r9*!NGN0sv2}{>Y#AyU-GYg(Bg5zBh$n|3LAr zhMdRqLC&<=sn7X%AhKFS-n^zQs-h0O*ukW_%Q_S^FDL*QmhpCQc7%S5?aM=U)QVok zdVYoVGjbjDLti>p(|TZMWF2aoY^Mfh_5YdgEKVBXUuIL!CM9eM<@+q_b%uW zIT8LewX~GnOcJ8LUpcqa?93(gv+s9va#GhlV_&l>pN`L`@5=ltb)BVZ%FJ$QDsIhb zo_k?9%DwEZf3kBg+NEls&tLgG`^^qu=+~)6R&~4|=a-D1QGQz+NlCI!2m8CL#+^EF zc;JPk97k<+J?M|}3P_IaNN8KRO;<18Z9Cs6A$u`4hIRkCFZ?uYUg?&H`Fx2s3YVI- z3$G+k^VICW#8D7u2R|v=>yMSUUEa7o+V<|74GL(MfQI~ z4Atnq=I&{_QS>@zb!n8GWfcU9rg?bDpx(Mda1eG3`?lu1<=*7v>`>qkM$uNS#7)sm zo#|ZMmJr0EYyL5F@QWA>GsXSV-{bN)L%+$RdgoEoCrp38z(41PlV6pbKvIlD=*y9( z7cf427dMNlG9%p!6ikOoW}`Hto>wR@OkOMCK_;y8>6O3PJUSJSOY`8;D%fa*bXOG#$mGC&#u~!WIy~-AXoUooTt&> zqNB8^Jd*GsZ}1GZljWLeB2->L6|PZQu7XV=e=V@Qq&YyZW8*co@{JMg?TXo7obZMB zV>`ryPw&;4Xc-OwFGCkG87}(04>#p%y&hM|reCKu9C-45Uk{U0fu!65Nk#(XKfSYT zrksh87Gl5I%*3~_zOT-?r_k42#iSaywUwN3ClmdweC>DRH0vqW8&x9hF+cT&$mWXD zGb%1mS8_d#66aYo`&Mdhb@TGL7OcJ)ngQ0ui$tWb1A&!4iCBIO*FTP%<>inI_{e~3 zcLzQ2>ZpSefzZ}pr!pej=Y0QB<1y6&lmJ_4iitD4=*{=w*Xn-S_p}|u#a^f&wmu5w zK(CWK(?2;oe_o5^1Q}t;*4|p-pbwT+eB8)W#AQ)tH zv3nI!*QlGwl{BX>9D=MY@(5+?`f6#m_ouH$Fn|QYPel+{H1V=LZL`2o%Je*fa+7B* zJ41h?-b;agtG)9a@`m_bHwPaUzDewgx(xJMPxxrdESqI;zvYu(4;ZfxO>ry|u;^-+ z$xvGsk8SbU1iz>5^OHH5kz><5Ve^FH!b;tFZdmtn=!6Orhr+z9U{v_>usrEJ3E_~p z`ml-^BYkB52tQuO~W{@^}1hDE>2u)L#k& zba}>iQ|DgA9}08nytVstX?sdk#8=JN_o;8v6Kj3J1nMo*%d!(R=}KoTN4ormQg>}N zQa1%wF)JEI$&?=SvJ+QUTxDYSBrDwau%TFpZ>QVzy}oD4{S(%RgDL8{-fb6jLLp)vI51{1 z2w}=;n_oOCWv@pu?X64A8>f7_rHbqQZI50r?tga@q;Qo8lNELHG5u~x=J!p{rjEk~48)aKRXmN!lF5~<^B zmQWu-%^*{wi*aL)A**8*#7LlFy0h))V(-#hyC|`t7?R-i?VC^DSaLbLjiHoXKfkzh z@UUWFJ+V@==N{KhG=z(CweGDP`S)+JGpBVR00q=uqswlAtLo-~a1W^W_klNY>_t0M z8Zl{mZyDhkr>h{|C^bK{T!!(*2?-8(pD<8r-Y8?||TI2`R9w5H& z8;2v_=C5bHGfXGCeSVVNQo1%WyHRYNNb?;@VlcI zZ%(PRbk5aDc~toN`ali6Qa{NvAh?5KYJ?jj76ZmR+%Vr(WNYyVy& zJ|6SeVS2VO3lDihZJ6AvgIhF4r1!vB;HH~DUl>%~V30sv$4Jk$g-jdCSt!Ruq@>ge zSg5!nHx#>UR~;Q|4z68hhBK4MzA}T`E5n_{!L&uiw@WI59sc&E<$PZ(M8-SZ0w`)S?rIY!fOx6X zq}w%1L3*z2VZTi9^Pi#VL1@eY%eq62IYizO6S8l$z@%OU(nL8iQxrZ_Q7Dh5$<-tw z%PAK|Y29%Lb<2VP$JP96t2o!m>`D`K2={VmQ^pydhaU{G#!A-~^6|9K6l&P30jr_}Ho-uTpzZMn+|`;xl5Xs$axlqk^7GM%uq z)xB(Z*&+E0dTR$TuR7(n>T6?OquT#^6l(mB68f>UJZhX$zteV2#7g32S0ODoZ*Y9% zmlE$$1nL9vlP9?>TiuUJ{CEQfPu-wAj`I!+l!}0a!Pw;?Kd~I0YCc9bS_%ZOd(y?` zc+s?Q+tT@%o_5@;vJ-Dt|HOBIPY;r}9mdM-D|=q5ODf-8pkQv9_705p%^=j7A0DLH zJ~7f5mIiOQYDTHx1-$)RZEO_qM0Q=($z=wF(paRX^3>U>NUB zT+$L|#U4F2dOhePX;8fv*1vREvuF9IZ;n1YCPUkU13X241v;{{`o4kSB%7L8UDP&k zTwOBa)M11A2OjcRc7L*X*x+?YwvY6PHmG%SvKq#Qks1TOfh{`Ye1HA9p{{dfP%SmJ zw#;%oh*t2#q+3_>tyS<}!N(yt*Z(qgN|ZTk%3Cerh#Jo2;csICYL7i(TQ?s1<@_P` zxoT-YrDyHZF-?fwWLj>$yC6h9a7eVMHFsMbx-wOwsbsaO-K%%Ca-3NIke6>}=NltK zU9kqaZcb)olh4a-qArNNHb*B}$Y1PFTXXM)-F@*V~C0v}J z;%K!H%9#hSHOrZ4ZJpPQO7K)$yd7cNO6ZQse=vFNaeak70kzL*9`#^-u)uH#8>b^! z=wR_#uCvu0Rn`rFvxg*)d9C>Ta;EKgAJVpuB?}K=Y3Y15`}t{!Iio-5lzXq#0qTT-)k1pBYmE!~d!0S^bCwS~G9zIBmTMa6EWtg zyKy#f`g-?0h9JjCgfHrq`pHN>&+qI(r#T~>ADKe!#>$+#&s?wq{tg zOty-8OepMs3c#Ln&ChWdEt}J2W$@Y6^W`7gkN~;A7Wtu6&Q@!xMUa<=H=A<~&hByp zh+GHRL6T>BWs96wAN5j8EO%aecX(;U%(gEoZWm+cEkkWu$6yz9&xF;Dff9k@t?F$6 z(o`YL1l>s2TwzzD5)<8_$d>iy2<>{5 zyS6O%c)r-U=odftwjOBCwCWc0U~s4bo6GHZ%(u``sgk8)=Z>uLZ1ftR-KhQG%-=hF z{56JR!O2+aa-F0~*zS|wN>KDf;V%m}+F{|~M>s7lkf^|UX&R|@Ig#=43Ph_Ewtn#GO;M7vojuY+RfyWY zsR@0cv)NGILfz3^pU0{|X)2Ov4(-jQCdcaaJrHXKz~1R8PJ9|2b_XEgU1TdWxnsQ} zq_(*-ePU27_h#XoBQIv4B+9VD0juBLufmD?a9RH333m^84Q)Ihzr?5pEL(E}ZB$`B zu^ZqdX~(+EB9G)#;?x8a{EkV9`Q63==767wM?Q4QRg+Ty7eC9yE@agiE$`nhMj=aN zwOdwf74AAy8RU-$+>iBYm+3IeT6e(v?<3S?(b;D*i%(&nDVT?PtOQfL-Eah_Co@cgrm--y+Ylh?*4m)&1VlKhm@c>s~z zTL4}ol^__sg@)N~l4RO~&4;QA+hAkwvKu(zMktahe^kPEGUx!Ph_?VK)PXY%gn7if zaW9$XucUx$sE6VV@6A?S7P|%Td=|4W?-w*KrHeS`H81^bZ;>QqWMty(;p5LcVO0Qb zY^VCd9JO4Fe5JLBOHoAa&H(Q-fWoJsefn6Pu@bViB%zKLeCQ|#f!9&zmKuNl{5ck8 zBhUxzhb>CRIFiZ4P)qWxqJni?;oV&8EZRYl#}5EmBVlFl*GF>?LNhFJp6>637* z7W*bp?ZR%4+c>&psB7pdREsFINi^*rWsR9RRH@#djN#!1$VmAx&o9meohn$)Z`D%Z zZo;r+W>_y)_9Y7_sTaU^*c!Ux_chasNb?`fZBF;{a0+Q`2&ywRg%t0Q*1OUSMyWu3 zmuF2+ZT#~tYbf}x?(c5uI^gPjJ)5RSCS<#zfvfsDJ6vj;`zK<2^$Qn?xL#~&u4vEZ zQoF*C78mM}krYz3MaON1&2d1)eD;6DavKjBVNmJ6Jou=k(+Pf}U29G*`e5+uvUDO7 zyNJ2ZUP*0$GICoqDP$BqqkCLhZ~%wjO}f2|JCc|-+nm+ zd)j;+K~x_+Tk?LVJgwMVxvol)9(+IO^wKrfHsuIAz@+a=hjgVGR`-IHqjS?++Gy0E zNAY1hMA@032>q-M`})agvs103W&PL!78xFp-9|HGJsY(tdaPoLY>azXUES0|usHMi zqB6&ijg_4Au{(RahtscYuX&rR5me{g33XNi*ATIu9?%MnWyMS%8U*3%CE`7@w58j# z69QMh_e>Dzg(Lio`jkdF9bdPxv?vzn3#qTtY?NNwU za?wWxA020%4PX9&6t`a=JW8-NxP6hhp6c(tN@d65|RV(|wuZrjhTy>EgI@m5E?tvfD}p1?kI3f1o9JEl6+ZVDaxBS|lf znmT$6QS9i@x5@H_VItLAJN%^}Sh(m=dYBG(!zzm@xrs(l7*07E)+0U!U0mnB>vNE) zAHeM&_9z`CK_1R5?sRS{aM`pess8fw`;l|6xEh@$4l;h(dGqg~)x>{z9xMWzU70VI zPeiIn`o8Q<-jtvzz3MJo(i9MQe-&wY;PCA!G%LnCDqMlaj6p)Qu6*x^*=lzl|4#Gm zr6<%Fpi@wEfuO$|DWr}Q_bw>d;bJF7IwxMZ*X9t(5m(~Y{Yt;BusW__&xc$R3qfSm z*!YFB>i-;}Bs!oQ(@Xh#)cVTIZbt`Jksxz^BS|4(GWDYbbX ztcUivbd)d;5Vabaw88P8EOuUP(C#X?+=C-M3Ym(7;OpTvzsi~L&pz;%kBu!1%hQfE zwAR^2*wDRce|`PO8N}bb?VvS!Ta*~G{k{En$@%XOwi~@cc+Yby;*~!-_%!57R6(ZV zccE`JUyYP4Mq$Yg$>HsD>YrUpY-Ycn<~l4RtdDboklL->u3OVBKawS`-8my>QBK#d zR|uJc)yN;DgQvoBUURZhmBNA_V39&|w$pcj%Qsr#;DtHnJjnN%-MH#g+YkA;I3hLG z`&PO$UvFUOO020ndCO!1Q@Rm&$gCR>ixS2ZYN3JpQ0QwnVLmTJ?XRp@s`Wpzk5_bh zif^0GE*7_ae+Um%Tf--EiptL|;QOQ{biq#ArK<^bA3E7k_q-2UTnQm=J-xGEetUSD zdJjV9-vCf>o!i^$WJ*tz(-5e4u$VmDT8cBoB*N&7gZBq; zTXI{=F0V{4E8e8ilnbn~Uz6cG!Ky+o(Z+=~_h?%~A^W2~ZI&o34rQBWt{IG{lL|$N z8QE73!iuc>9^4&YT8`F@;mr<=vDwyVY4Wn(*xP|}iNVFQ+hK=RnEpe6CdFbi z7K;@ZKRA!MOQwZFl+6^kAgQmq^?X4&_8-zMLG~3>c3xiFdTloRN5tMqi#j?Y>O!^Z zJpVf)3Vw8>J4S_BMx*r-pWVyK0KAjCOF&XgF|dtN<8xiysM+Rt{NuayFk6fC?8 z^w1(4EDRnTm{!biy|MaAuW(&`GsgpcFzwjbjrz@@9?zY+GiQ1%T@0qLJ*aasx4&L& zZj&QiW!LcOR224bAd3+z9obVu?Y1rh{ifS3$1hfxq#LWl1~Kl7*AC}2X%UKq{>U$d zPjDFT)yjRyb#@ccnr6Z2<iNr?T|o~IMFX_eV6$n+)VNhCe|#+ZNUx|mCM(09>HvO zR;k2g<5}$=N;h16)!c9z(FO5Z`G6-W*1C*t-JK8b7bEfIJ3MqFRWys^=m2?4CmHV) zB}Rwg#kR&%7b2}aLSO;b*N-a~dNUM4B+@c?-;acAhWB}WM6 zh>_0!3*(H6LBen6U~+t!)0wvALK>-z$w5k@pv><~+w16*KWoWvYD>TKZ0!RMqL2*er}K%MS&8QU1hn0+~({a$C`b0Q_Fq_`&be0j9cck7-n9lCd5=x&yl zpGcLIKP%Vd2qQ6JI)>dZg12PjXWLU0T@%}>>%Pb=Ctol_qx}zIy`r}cW~(wx7MtSK zjsthxSsKkwEe?vRM3TiKYzd16R7ivu2fAFT(C4j{)dES0ZSRQUe9S*V_{OA6X<;Ge zh`#49b}Cny+kGaUo1c?^L~-Ux7gld?9bvopS$YSXO=dp*pK_KYMJkAtmAj2YP`YlRX`Y`` z>n?33Wn5h28G@`HT4I>9GV|CNKz?s}A7IKPWXYhE%Wa;^U!jFP5*K`3B_U(=dl(3p zK#!+xjBay9-rC4@G;YXfymb0rLGp^7Pgc}crzf^fK>;UUV!uUs{*HWW*2UV5vVhi) zCPg}o5nZ9CC=j(ek8Hc?e+0vMG2WfwApNtxXS;Cx5trOjIawZrj4P;?|HQ@Yc?YoZ z`;9F%Snj1%3j7sR$MHo>t8CQs2I`HRnZl&d<&NA~+I4izJIe-7C2Ai3xD2#uX^?PM zEOXb>yo{)1vc{=mLhazLc#o@G%B=$|^SE6FoIWFp649WsAb|AT$maUyVHBU)uGp}? z5_d+Se@5O|zP3WCXk(+PawfrVsqv$E9M@>QnGc7BX}wrso%36KqV#I*NckaPp*tK+ znS!}G`^ z#AT;H-QAm+DDOSI2Z0uxFF1Kb`2YQ10O+vM>O#40N}6l*r)hW1ynrK)>SFx6Lv!`Y zr}zs;6kr)G2J2{LCo%i0PF-_|voh>PU$L02IoXKK2*gh7c*8;6?@ww(-T9K(S-r=f zIooKVH}E|Y-KfjTe&TXQadug8>$R#@K3}EQ@;O#t75m8zzosY8dFcD_|9V5Ze5n&4QwmK1v*ktR@ zSr5c=FEW&D%!!nC_TtsoB&Og3qDH zrufYiD$+8P)n?xyQLC^kW}iO*lR}FE<=0x=}8R3;Bv?b+;i&SX%E17_0tZxp1f(D7klUb;ntH z^M|QZZGqfVeXt;K09b6j1BPkdRq^SITdZ91! znE2a~M6ex46utTt_UADYbX)0uZuu;WgV9g&8!ENpHfTbo@<&!CUPE^FalgKxM~;u@ z%80wq8{6M8XlC6YUdc+jMt&%oNmQDx4jHgIjJp!#fIqD+u%44!L7f;Y`Os<1dTGz= z>i)=NP@VETGDtHFd)4&X7x-wTWwEvKW+CG>Vd9d3LIDvrrwg_cEVbDND_Vx9C<|QC z5WZgVm}Y)`t2-V(JiE2Jj{U{qFkaf@?=J|aZwle0iWh`bR1Y(Dy!V-XQ}oD&nq$aI zuu>)l6Jax5<+fFmP9*0S+Z=jQC83^Cym`dB^8KgeTJn)g74hA48_y49={4NwQRigF z_I%TnJq4(G0aTQ2_2Nx25Z%Akn)W3#wmg^(qlP&cB}A|)8H{)BZ`f=E`7`Ii{65gC z9VOoYClVRw*iUnrTNd3T1Bx0N+<{C#$aKdjH5u!<8eU|+9U+D`Of-UkCrtL5?%2jg zJe`c(Cx~K^F>k5q8#8o}JYO*+YGsVxo!v-g3`)UOvt|>XgA(^^5=N*q!8j1uD4ljc zH{$JP70d6rTEs8-mxF6GHOIyqIddby=ax&{+Vv#uxn0>?wnMs~FV}4Nv_+2H=Um>h zxpO48NT~l#nI#BqRu~9PJRl|J27!=c(ZqRAlTix=&n2Rbq2 zE@G%}GG4pkd?-7YFG$2lsF_$LSY%pi_O%Q64iDF^E2^wK(RWK@yjg4MgkO^8z;LTU zKC3W!ehcR&FP*WyFgGLo(Bse&Yy&X7gY*PGd=d$?w;29?uG=w7qkHG1LN5;ThRfN$ zDZS6F=LA}gRud#CyyWv7f#2VOcU$*5iNA|Kv>ov9^q9V`eB+QW#phmFS83VOf5C4` zoRTmRjy{CUSI}E%sod_EGIk%{Wt6Pd)sQIs=?Vv9Ps8-$?IWh3;|!(BZ}Uy1x}X@t zdq))U_!fp(8yEH74gSwgzZoJ~)j=;Z`g*>6?r`Pn+Aa}Np(&`AMy#%m=$7#J;hycR z2W1$EoF8E;u>NPSY0*^?-#Q9?qb06QF(Ao zd9LeiVr-&o(!aRAw+E3L6vNoNZPUsf+SMRgL&iG`krF67xBlPgb@FIC>s{Zz-{tdi zmb2g1$-AXW``UBNu%irpimxc}6g@)4w)puiFFRcbhyP1{^8gn>xA0KS*QuXOp) zyxxaSVY*WsHz4Bb-ga-{*Xp0E0LPS(N+FaKeStv&E-P0Xi^ZTnte#4!mH^F6x5#G? zdJq9uN{ve1w>xK6wbcMLH(*&Rq?inYS_e-kZH9hWE7f%G5*E9{#V%^%<2KIEB9a@G z!iYn-FAt_Dx}WK%&A3$Y&uwg_iQTCVz&8dmMa6@=s68rqi%rq5T})20@UqW8bf8Nd zii-2eRA(*RlR+u z2{8feHDr*Vpl~LY)e8J({NP6`j0s8ig`q0^{_!L5Xeb!UI3)HgE*Utxn5BGjX3`?o z`MQpadrCfh@J}!cvJyOdnG?fhXG%9HdgA>6I21p;Uv<-qtw8LB2?pV& zZFaSKLPmTy!NjrK9O1Z3MDZ6<+Lp$~*vWRnla<8S%*AP6-d69JdD}GomtY}I`y&wJ z+qiR#GCh4vjw450kb37Uail8$H=nV-mU9tRKaL~lZl3ieWWIX0?n;OZm-tvkfOf6E z((@$@*yCBg?w>SLAkz=t7z=Zv3A@?%VI#pWsYZLvUY;av-GZvpUTZ;n(3|6JDV{Sx zI|&Wdy^@v2J8cR5?~N}3=DMS>?NDBJmM^eOu_)V4j);+owAP>gb&f3+#Z&IT0;~P> zM?tD#8rJcw40NpxUS%SdX5Cdm*U2?oSFB!{CuYVx^b5-Z=>=b-+kFzeK2i;PZ;|BY z`t}19j}yG{T{D*6wUR@&g&CO6rESb8o*3gw3asy(eSMOTli-z<-M*xRUp$&>fzhd! z737@GQH6ta_Pp)}1~EBM7gRiv(cWYW2#dgALd$!4CNA^8C)EU*&+bO1RA^_dT!=(e z&0}UxFb@@Fi_6#96N(G(!U2_`k>;w9E5$MRFs1f9Oyf)4L@mv`*RuzKmKtLj2NTvR zn&<@QY?74tkLTkO+)m+15W~yn8aDOLbdxx$(nMJ*QTcO-pTSrI-H%=u!1}AEVqt-C zaN8|bSK_=QTg0nE6nHU|a216#L6ImY9ST;w?0p;|&ZeEiM4O+7R2j{0Wny|TVNXre zWcPsWaPMVz_-Cc_$J&DG3=b{KzV(@5sua}H9JO%^Wzip0M~uSe#U2R~3K(Mc!sbRo zZLmiK*V}5w^V@|~lP-7LUY}ASU*aL79=3F+57wh%$_tp6ypWOE&SqEvN80n@d2`KA zQz)nT1^3d^Hd{pOA-;!;C-kdFA;icJh4&u%X%@LxR{#7Z(%Fx@f*IH=CG9(R52zR2 zk=jizeaP@uS=!U5fhzIt>kCAQeREqeE^9I7klwh4+?+H7L`OCrb3R%9`O^hxHGz@7 z6Xqw}s7uqYn4E}|=segyG}kG*;}@eoE~}^*c;mbx6S@(wB_c|)K-sDSNiU8 z41yBdJz%-#7iLFzD(APsA}OTaFMqsENL|E1i({&-6)*8n?`S`j`6;@eY9$l#;E2-r zC;8FJG#MQjYrXMXoM$6sh;1;UG!;qdbLA`3xvDi%3xHjig4qbsxLZyxe#^|?nmRSR z@u~%Zhd+-}xJmt?1hB$NzuWtEIK&}|o5RFhYrm#pc}m5c3TcOGXh6;>wZzauvBS`b zm5^fog>ehp!WC08j@3@k<@cNgH*ctzz^tb&C{8Zjs;>CJ;OdC&Kd0_58lDT*>_5X3 z<8EhzA(PX>MC9xElccX8BcWPe4yPz5PfNkE$ulanx}b-S)8H5jl|>wp*>5N(cTA2S zw&<=qmTE48#d2UJqr^;$k%WCXe3qr7|NVOWDMah=m)*e*Y2&WT>mL8mx0*9-Wbw_=_&^H9;`Xj7ydym$qN& zN)MMfhD$6ahZ`&4p_E!ip*j!_l<0MNEP}iL*~mn+Tuv4dOZhp0uCzd}F6qm{K?9t& z@4@s^-_H8nclURB@r3Tid#ij8`-00?>73#f98O)4zqNL)>v9PQ_Ma(15fQP+AzC*x zQ>9#CANG|Ta~-qDY50~LX&{sT$KKZh)ja@aXX}f~CgV~P=lzfvhnRsx(4U(jdWo5-=alpJb z7GWl33(*uwZPS+n2sg~Kc|gO^mG7&IJ)1tpD@Hp77B;}W-cCn))vVLf;v~13qvhtO zE}vt?Tm!5wE&4vyx$j%!LB!--L&~a0@*vGu1{q)Y-tZ>yAjx%u672_RhZ}nog9`*> z`?gXc)iYs3KOpX#K^R@`@ewX;c>S6&^58B5u7VBIDhYn=(JW?~Vzq!gtdixRRn~D0 zUTrrKsyWknW@Be_Ni=O}xRz4UDo%{0TTEe95Zm?agdKXV+#Y7b*tm=g%ahBLD%WEp z_VC1w|8{|;REI824w`r~9W`Q-&z?WDwLX#OttrgxK}w`tMw(27bm<$pAvZEU5ZdkE^N46E}@!uPN#L;LDPc;WfuN(NlLrl!jq-Tq+gq`nh$sx!+nT(Jht=q^j7S4 zvSi*fpwJyr_8VH@j&i%IX=&Dv24h`ftS&~EALmGmnACU~-R$T!_Y^T)2Uvs0;%=#H zT*FH5xt{!rdsy8)-_CmVx&bY0Cvsm=JY6|;`6g>UcCo|KmQ6+} zE5B&ZqGJ`f$mD=T*{H-lhW}8D$2`n-q~Y2%(GXmzs}psqbtQrBU=`yq_~X@C2p_wG zi!i+Y9fP86?$57#f>zj9kJ?CPyeRWBb+!JE@Dc6kW4+y#VfXp&LL1=5=j>-6*UcgZ zdGCE2C2~w`{K+@RG{wR9Oa1o=QMakoSf#dsJrTm9Y=JOr6U}P366$)TZsQhhHL@cT zC?b^n=@cfV;=pe)#N1C{aBnhgF>wBA-|NL(IHNo>%X}g4b-l~TYBm(qScDg>Ho7k0 zyl=~QYse?HFcD?tw6?Jvigq$>ak^dgPQw~40&^wW7Ho>BN-0fl1kg>a2QUM@d<{%| zcQIfZRGf}gyDua%(~0<_N$$Z#_3n5Ft_wCtFJeO9W`g%a7v{QX8v~!j*tZE^REma5;|c> z@DD^qw&{hBjW-NWF84&F4!zXvTTd#>_WYGN4^zusMBi804tj4nOL8+=9M4aL^18D` zrJ{FP*FdlLlZUQ2Ky~|S_M!`@RFp%mUnraXM9pp-eI1UzPj}mtF+Izp`h50$5Y?4m z&)xWBoG7!6W-Ldwpss(I}195SKd3d@J9 z_QLP>pX z=;A$)!m}wauUVqpOaXf=hf4<*(!FGw_eH}W`jtYA0t7I!hU1)t) z)jN*mKeiq90fJzHHG0dJiE}>b#cfo+){8c7r>-E+LZ11Y??-4HbY9~*y|%I&+o}HD zQFNR^k>TkHU);)lvPoiGUuSyxgBd+e(MB0^7_VqY-zy3qGij z87s}sXMY>UI2I;tZv-gtEXZ4JzUp7~0rc8{W2$`aQ_9VYP|M~hV7TpCaIxUHe8Vq( zo3A&K(zh@fvpj#mG=WcVo=W4qsLOQnVV+6{$vE#P)+Y9rEpL(@il_tL4MKQScQzyS z{OJ4L!?O>l&RzIcJ$(DO5rBLnOj%NLuvXD|#WC7uo36~t-);4Vq?@Z+pR{RXb;^&B zJT&QgY7&>Og0<6&$;)9c4Shs#t}1<_Y^u>DQU<@ZCSljfbX!JrjPN4b!*PPXaYT-o z#QwZATgm&`6+^;Y>E>DT{cU>h%=+gJ>S*DQ$ez=pxxpOl zM8V%Exd%=y2#SLDQN|?3i&t+y*Z>M%#mw^mKl9rpt<)3vC{BAjr5^2z73aU`em+&~ z*-fP@EtkGZGKC6;B7;%`&P!f9kbV9>ptaydX7qPt`}G^u#rO4N)H5$hLO$>Ecc}s7 zPgIs%`jUQIBbkzW%|D-yioAc(Sr_Bb$GMK#tX>o+v-M3BVPIPV^>DfC*Le z7VrPnX~t}f{QvO}_Ld>?k~YojCYXf??AT!J<&mcjL2hU|LW)=sT) zn7;bC@PqAz=HZS*_KxVzJ*KP#J?}h?wT8?K^AEs0@u{&VaAVTGjXFD`~K0S(?0k5 zJz1^fF&?yGbK(mW6RPinp6P{5FkwlUbI5qB`Lbr$j$13=v*T?6n532iC+iSrTto0E zIpgAtSW~bNt%+1yI@Yo1xwmp(!Aq-@QJ1^Mb5Fk8qQ-kKJ@QCX)yYvB_C3#04fL45 z<;n8M{IOz48p!5c*m&*RTVne~*OKNPN6JVsOdj2*D*r5GF;}c6g#cj;Y8Aruv!f!t z=Q>&}_7@hi5I=&xeY+O$#$G_PxHmU5#QDQR35Axa$Ez=P*{IC;97%G?etICyu#EC{Oe1Mb}UY<=sCp>zTIAI9Eua}mUm6zI(G zzTUYAg^|mB$(o_Gt4UMc=}ALWLbB$nOJiL3a@}|6+V0>$X^L^s_UudVjaizCY~q`8 z2Lte~|MBi52Lqbcxg>1gD&L~YH=rj8#8I0w7H=Ju`G9`w2+op@!xIsv0a?3cG*uT{W+Q{NT8&(YZhNP@EPzB?WpO0LA-xheSYNPn(a z2Kkx2pJ})l%4z9pvee>E-~9pGNts9d-M2@t5KDqLCiQR=dC5nN1YLKF+?T7B`(j!| z#&C9ZB7x53JzkAQXPUiCOGow*Tg^2U4r7(h^G2G5eLI{Em1gLL)@|OAelD=0cq{OQ zEe`N?yKT+MyKKr)o}PXJ70_nc5ZkgjZmEWT^Qsa37W)`Q6YpXy)pvZ zS`tc1y7nY7zsOkN>%|wFyi;K(eVxzspa|u}5cRpb7Tb23GXG(*?!w_4qTbSJ1v5FO zdDC^{50}TQu^zjewQ?nWF5vL#d6{}gb(ogmvP?KPgu7yHF_YK8@O`6TC0LF!O;p1a;ZgR6J!<*0#HdD?ZA3T7a9RNFHZjNRJoE-fR()apIhU z)jT7RofUo6-oPG^7V*J^hSFX@bBw<=-S7+ju^S`>)hwODHEB&NniDiZC8saxpIDK; z=(kjxw>ptDsCF>izMMDVvT5#kns*FzR5NuT(t962)#~9E;!#Q7rH_d+VTN*eInhzw z(!^&c;|KYS=N1&i37TiGTUM5Kh~Hnj;o#eF1!qKT%u@aiX>XY0yHxcI4a<7yV(-NK zJ?iA;Q&g^>1bcI5KbWhUUtzq*`y7|K;+j`nadhEW-ccQEP0L98mKXb7M-!|U1p_}v zu`VXJy&j>jQB!wynGLBObq3qO3}PHRN&9=}UiL4?gxT-!C{?*rQe9cJ%texO>CtrC zrAkwWS6Q>gw2X{3yGf|;7d_kMb=E1CRmnZIVlX3dwK%IcZ&vp7^`(XKdd`(eMF$ zoU>FRleAUoXD!jp`b#aG8#( zmK)__MTb_LzwUKyquJjEB8=roV9F9z#+Ql@mdQ$D6kZm8asE_;-D$P1j?i~sNIo`) z4>$-cAFllnR~LG8@65}oJ5|9hG?ByXnNOOIew+W?@x1s`9658ekjzC_t$=}>EsK@H z*WYn@y>wh%JbR!Z+|c=WR&(9n@$95vzGjyy6=6fpF)5wb*PFlYJG8uac^;TG`T@S; z)FRT-4%-E(TfNCzmdXoxv!DBmU$%L>`h;Cv?CAGul0}>P2!!o&e=;V6?tv-sh{s`8 zwr5n-*uyEPS3A1A#l{{m*GyM9OnhSI6kO=5r%82X$a*eaY21lnX36& zQoF8s-}5qR^ZtX5LAqjdou#EO`VEgZ1-2#BrUVo$&5;k$dvUUyrlE2@Av)E?tocJZ z@zHKjX$6e-HG#d(=}q0Kl%glH?T1UUJx!Ae_zWJ?i+y|5&|C z%pe60nl)iyJTE{BzZ#=i=PS8msj!#RBJad@$LR zQ|8kmSzN5MjiO@sS2P_Qy5eeRGtnM60p@CuuEKrc=t=t7t&Ta?RSJh%DP@kJqC9R- z^N_*_EHCYJ!y5-4O=f?g*PY~U@e)Q;@dz3akkgslvJ*Le=+v3)$~%XeX^*@QT@3E1 zxKFKXJ4``Iscv@voRjW({Qen-6J9NK!*6x5b{{pyU_s~PZGMO`J%^Ab z)gpT_5f;D8%1J^@Jnye@e+j=7z+cHzY>s)G=z-)tQIbc8M7pd7ls(30e%jS<62s^w zeOS0su)4r>_q;!k{~SYWP13hIV%qy*ncaydx<4$3UIM40?{KNRgS(uowV9m1@_6wT z%v}dS5VI5z5~#b4hA6fBD!yTrw`c=GDf`?{n-x@YDg7+4GJuPl1ba%x)i`3AcmI_a7nr~APB zB*m;r;K=0Np8AnMyY4VCpyIaO@x=HYQpQSn`sEb z(iS4|Yd#!Yb&9@R?KeIn%kyEhlBdEPbD2yfYvR@m_rsnk_vC2Wj7KQkWnsEB$XEPF zwtyJ>4cen6er24k54>IU5}wJOnImpH@W{+6GpL4m$mNI62Q9nL3EGaGv~5{kGdS6v z!!|?ZiV0p$%Y(8jrf07&Rg##6jE~-QCroM1wqNFL`C;WzsDV|XW!+(R=9>)B=;82pFt}(0hI^=y8yJ? z_kIPau>pmk9yW~VeO3MCKqDpB`&8m{!Brmg^ewp~Wob`^W}ET7JU@IWc+6pLE@Yr2 zd&}eC2t_D`qa+38v%q0%560tug`X$uJjk94Fn@d{d}l7yO3XF$LC5ndT!~@|7gmwf zap`{MEAvv5Z@fyXmgYb-TObmgT8w@cCxop(9bg&C{2jf$_z6jqpBBE1L^Rim!@7uK zbnjVnS*^7Eg1ZBylg6q~5w&rqKqCglz*6yYJbMGu?! zlJX4ddMVE9kYV;FYE2eUogFtyqiD1~k9V$hy6|Hdv5kj^hc#^<16nhwLj~=)G1DZC zRs=AVK-KnqbnYm)_p*bw|C4n|dBzyY11CkQ#ds1usrjNQsoc7U;(i8|qGba1#c88l zGCbmwyPEin2~C{HitfAYi4va77cGBk@^v`?mw)C|H3=FOiYUgMoTE;sN;n$&OR}4< zi9m<2!09HlrTO|NkBK}T5q4MA$X-d-=MGD?uDMsq@k2;MSQpxmYr9g*yQ*h3X<+(n z^`!phO#!wHu!Gr7??}R*yX1pbdMD<%-n?4r`hcss&PE5ychy5%s$R@|uyi4Xz3Q-L z^gYbf2a^uBbQk95ax{q@U1RQ9T|_JY1<-4Qfm{vR`XLyLm zJsAIl9e1SoAw}n?hl!f4{@?<>g*16wQsGLf{wPKXg(*#2+M)-Di*?hn6$H?Uyow(h zY0CZV!Jk-+f%$Atu2F5!3xB@O(fd^@nj}|L3(DLN4&ia#eDK_Ux=wnqpk;Laiv)nr zRp3)gez?e%>=PbcsN-yN7)Tjbw7PWJHJoGM3GG@X6CgDI|&M>IaO#}zuZPKNaq zz3wf@ljm8{#cNq{VtFs$uO)KAxiz@eHrjDk`_jZr*u!P>57X*(t8KpiS5(m`${&^W z*Km#>HCu{Tlw=ljWR^P$RZO_Yny%%J-hHFDWoXj=e@4&0r~A+0GLooz$n46nUB~ll z+H7mNdbD1yZ%F7qQdOG7Fumg;6*gmCeIVgI+LWnTYHnh34lxTXmjyX*7ncgO+Kh%r z5IRixQZDav@we8vB}WU610R_Bd~u5aeS>_Dt6&@~dAAURFj5iHApG_Gup5aX+v2$) z2Sc~YBsFAy?P=&;umN5}z(^VWoOW|QLU^(6&?J0iPGDM-n1TkNsB_;NC$wQ28p!ND z11U9UNIZf7r=1L>a9j{S91T~|Gy}+NhX5|^;&npEsV~jVO*j0M^dAGmP1=Jdg2AO3 z#$1JyaE*x2jQuiyBD!N&H|G&(`5FnK1-q_t;={FG?gK4;2dY)e}3?!m(<|A4V! zLu~e}u@{mDg3ZBNQ*;}!&;ldE7Btj49D=9`H}e7Od#9Le(2fBesfBMaPkYr=3m{bX zI6oL%mObU4gNe;3JMA@sxD6G>#TA&r_=+w{*q|Kn{$oSkd~}D^l6BxLDgXsPi^mK_ zcZ$ewXCbAj9HC0gYc%OF08g>&3NN}!UbL}=;US=He&^Wy)}&wM3K+B!9Q1e-%pu0V z*~K&wp+%x{fg*g-q|XlqYzArcYd)LP1XaVw6|Bo%egvOs z>i`(tmL5W6kpp2x7iv^EJ8ns*0>H@fh$ZG)7ZJ&T+t$Om**i9pEP$j_pMnC`xa;#2 zFiB}xVejv>qeal#&@aNo=6g;BZe1gC0YyE-o=^W6t-)c4#J`9}Ftpl{@LISl*$u)z z0Uezh`2X8wRzw|WDBYx>6V8(stVbp|v}?tbjo326tCzJ++H}#c;{Q⁡PiFGpYE< zX#=yA$hWqis$a&K7A*LN-p+aO4BiP=JQIkGUz(`0!8pjCT^L-ss<1#%vi)8N`z`=g zV&-FpbIyYFdbe62y1n1%)gnR^G zZmMI0FLYEL1&|wXh=XsI}I*YR-jcXkI$(d*(pV@fu5JghdzR9)xTCx zofQC>Nq(MPYZOGh2M{JT33~}Aaofn701Z7geU1r#ArF)^&#?BM+^JC|?DGcZ*_GzY z=`|+C1_JdgqSDVo=U>o?q4W}@l&dU*_0ie`Rj^nn*h_b(?J&GD17pKC7{0jnCP>EO z3iwCt@&8>}5Ku#VAx1`MVZ{dj*$`CpsRS#rJwZ1FBpZPODbfSL&Yyq54tE&G#el<@ zpBMZWkUR(=iK}P{-=BT)Fd!-3MrFsw5SH2OM#^kLX@FO?9EuNu17f>u_i+dE=Ll86 zS%YLd0~p)w^F!!(fwo=fr9HTU{_(VAaPQX_0FYF`^acL^_FKG7OlUz9l7z5$?!e%^ zJwL_)_p{q?$r1#5A&h|?a41>e11RN)DeW}$#K6SC!xf~9?cj!MZZ*){rzYV=JJ#HL z0NuPdv)8^i_3!1D1LQVBtF*QTFSiSbjfzM0AzJyMF?zqLZLBvCnzKU^x_z5yb+DV6 zp*v7pR&gzB`0X6fe!e6n7BtPdUezveFhh8OGKb0 zQomF1mMmZ`rEx0loTe7+JJdOg0vFc|DN7z;Siia+bjJ*^x6fN>RWd*#joDYDvp z2||QJXb|wbk(blP=qO&8(dzj{0Yo1t@H1Wo!JQIRssee-)%Anb^H&G4V1tbAvP1IK zouW!X9`ZKzk!G6y+KUhvJUIji;4#@YhXs~Uj$+-Rmi)Tg9WK-K{%V-#jV2|72M6#T;2sq{jkK8WJ!u*E;6f7IgDddkn{@itFQSTrfgT9W z$svwtjY)-1fYdI(Za_?v(b}Wmdva%CXny-(jsrdmR$&3`zUym4Tk1>Q2R{p7r!piA z^mH@py7|Nd9vSK}Fti7rB7VAUvf!MS0y5;j5p@F2QJ~UhNGiR3D|-Ao;MghrVEA&E z+zBvFU*3-v@Rm^-j1uRd{3It18lvLU`~<&?Au;>APHjC*_GrMMs{=Xkg33fPJeXj42o(4N z)fGy_6WhFqee7nY>lu`!0LP{kz-G;-2Dp#dzOvjVBs8J96E6to2`Q+*NYtreM|X5F z+Od(65}G7l?_!4srpIxi9jBxWWqwlVlK_z?(f*XqXT4n$198T_id%Ar!kg|DkfIw@ z8a}kxAc%ZAwmRqbVY!BsDdqMz_@d7C9^;QzDjNBM&2*{n?}LjT>JQlkpBbUYD!`qZ2LQX81K%iNRy4GtiLeG+dEUbbHIXo?cmhZpy1jzu_ z?|X*Q6?j7{u-sY6SN7kY+ghE*LcrK8$wUznlLB)Xfqfq6HW#92Ta_Oeoa{{*ct-gf z$eyRVh$y4mZj9i{?-#p){s^$BPQs@!CjeN>U&UR7H@saY6&}LH<>$c6mV`GH4_or< z8BrD9DXKn$2tmb5R0VtO#fVF=*xWFn2;TB`O<#l+klRb6hv2I%zfcx`5Rl|4s(N_G zBmr-uQk<+p3@7OjAc^$GlWRLBNe_^8*ysTwuM&R*kd*GFx?|lV(}uh?A9vKjhJ*4m zm;ar*kYEqt=$-2{c15yBDM5IhG#1r@iqh_ZaXG3%d z0Fx{WPar1tI{LurJ@fR~>ChmssCCcBKkN*Q2!iWg+-C9*q?Goi4Tz>s;QZM1! z@R{kjMs$(NpI0?EGY?PBN3e`&lHHS8Z8GA>%RH0tngn4lrLa!E zos)k%p7h?+S&^3#j_pDWfKeQsIW%VzZbsbN=q@nar4C@%9QfpEvKv7h6igimp44~$ z;k`|NiVZCE{@~e^lCNh_q-3(5h9U(^Vu>mr*x`9(RDN&T)UV<1A_m@-BIzOAyJ8W* zz8l@}5i#?EGhizj1C#j{*<|WG#=b{BzrX?ais-n#kv4aBLibmepM2M{yY9Ip0Xy|E zTM-Kf-d5WDUpw0u+eShDcjrI+{D+_awBSE2`2SD~ddOHl=bcRW6@E?=+11#G=SH$T zDjQUso~kXTNLgAa-mk^A{9u6QYtxKYzj2O)jJiYcW&S3kuT3`IAsJJAF(w)Hk(tAE zg2w&EGA%b#&YCufSTt5Qg=Bx%a&OIW?0)kyQ_wmsk1Wr4w)BRispWva<1vS(k;gUu ztv|kfnNgd{Xkm;wmiHwtcsRXpV+R?I{H9qmyH=WJN6vV`rvR)+t75z3j|Kh2jvlMr zozA3Z{&kmC1?{^PBxS!02^?ki4l(S%epxp7|9`ivh~QCGVIuZD1CeS7JTQdc--gQQ zXko@^j`f$-5DTXS)-xRz%wl=OmT&XrnzFy1p#P~~lU)V5LSh=W1|CU7ebK*wq9Ryk zNOZtSg2(;a4la>QpMo5iQQwkI)B6`Ah0ojNtigOE-nME15v=Y!euH3xY1H2{gd*7i zqwZH|8_zTrJt=B`HglRolghh^U(1U6Qi84oEuHtn{Z11TG>^2FQ|-!^bEa)x2RH~H zecYDQ`lG())1xVyONR<<^S-3S_>uMxN1bsQbqLm$WoD~xy6|wRb(*grOhzdx(~oFA zZ_wC9BsB57Df8h8$NHuv2O-tRtuhz8XOlDMr>$Ne$_sr`I;v?pIoPjvcJmHc5$T6O zRZ#Jw&&c?baga8wN0EJ0bxr)OnUdGIzCPx}E4gz>^_Gk`jQhB zk(t+qF1pr5WiFeucdoTSjLA!F*$`x)k6W%2@70t^JL5_^NA$OIbR7;3`v^*0>H%n9aKwPH!$Pu7(8m@}aZx z8sq6k961(Ol7F!N!Un_(Gps#OSGjiQiHb1QrSgzMu9D<)EffXvf;jAOTdE zCMW*LU#O0p1!%*UzjR{9y|0fN00h%Z&_+@EFUAtl4ca(t>eB;HUH%Jt{&mF$5|ALD zfJu{VWhmQe%5>ebVqgk_1lx}WMc%&|i!ZDmCij&8&g)^;rCcGuVTzy^L0yVcye1jE7+{O%;q;@(e1Z_zzZc%||K?pd51)daEQty^?EWY% z!++f#O1i;fWuOGT;@Z6UU$KQ1R*xEIqsqSNNVi8%sW=FryhRb)s}Kfh2Nwz#K>&4W z_%*uEjcunXpZkSobk}0xySz4grSLtT)JOfd#%-EcN@{HHn0mL7(*@cw4Ly{-AGp%H zvFJ@s3wns8qw1>aT>tYAAIv4S=wvi!8prOvEV=jmeVXp)CSEp#M=wv zpR?E3 z6mGX<+3_QFm0a}oQsmgJ;ww)R-rZWCTPnyGT7FAt;5GU~21dmUIy&si;&gpB9FL#l zZ7x1M3x>cHnM}m=e)aVHU53u&-AJr2Y==$%Oo#9`Qir%hN`74=Ydn*FQOeHjs%;Tl z>c;JE&}Ud`Z5ZkK_=e8RD9wWfaT=UKBeW;8gMHSI@IO*~kf3h@Yx?7&YNEe0Q-=dH z$wzlk_<5bJIP&+J_ktaB=4KiCe_%@~q}|=IfO6%^QrKQrdH7zNb!!Xu?yI%!NzbRA zK#Hz(%s>=-JB=4!i@g3g!g@f8u#a5jzG*k{CUc6)}i?cGs+`XrZwtQ##}--U?j7P&#;NPZ4SYuR4mehJKgTyC3}bM-jy zXE@vF@~f?VZX>8r1rerwI<^f#F z+8Ej38!R`HMX16A#W_N!8pW2g(TqY=u|SLdGAi(Td$;Y(Yn%@>5QmSHojd}sGhFXA zHrZ|TL41?&`iC1>3V`mUaQ;A~D+K!^^bx%eYg7iy zrNewj#i9;?uYOlk1Qp-#6Oc?hJw#X%$o`SS?Bs>qa=VlH)8=fi%$y8H+8sw1^Ls5L z<-dgVIEPhQrdU-BI~gF1=@u6Z<{(7`-S z37Gb&bSg~$P2PwIOh2WZAfFM=d!uFB^3Gvcsl0%q$RCW^y4G&q8uTMDF2wXcf0ZVA z2upFuD9WNL;3hr-;Vd>>{09$IcKrtr6npYN1CK(vhtmDO9CH+_+42|Ln>Uo34Dt`~ zYrKwZIdCBJreQ%F+An{j$_IryHwXbNQ7Co66Q%)j9uP&v%yc9DrnCdo$~nXS(n&Ts z!~aUp^$s5QIs5d{9~}SYS`7~!U~2XVc4C2HNYF+r{5(Z3OnDLvd+N&;DIC*aB|*%@IV3E|5@-5w|^3M zamR{ZD&L+kn~~yB*{b3SoQA@%m&07LdghxZQ^aZ1kT95Ks*tW-S1c4*p;hkE)9r)m zP5R`6bal##iQ&(m9pfI#gzu$+fVm`euaVLTwAKu5yX&^y2eu&hW0%5)9KtrznK4U7+d9d`1$F6Pr zWPmVO0^e1R6T30hgm@b|vL}&RwI!wMlfy#RWl@H+iLSX~6k|=DkvDMH-`z17Bz6qh8ll8@l9}@8T&ru^fLtLbh z#)=Wgi&4Kunle?10IrUF&)&!|ge8PvU@9sW8ImQ&Xb^Fcb|6*^KmK1e zeh||g=UkV&Za6q0N-wN&1CQSj$A0jfHaInHaGDPeF2Ij$aL{anrnd4Q%J^B%v`0R- z>|&D?qgCFQ!WzYB{JV-^;VxxEmcPp`Q_3x?`XG=7GLR$qd#zpfY@#~rO|XbZtRyXK zt4(}r5E{2@wjaxHuw8;sdiHNffLED8dv-})os8UjFB0_f2dp!=`(}PrF6) zQ|%wL8oVGE?WWCr>4%0jQd$`Sr|n+&f?Xq04ksUN9M=YntPAD0pRU1j1cJ(%NPvf{ z?r%>bp#1v) zE^fNdENe;s#k+96omT@^vwxfg?RTQL!PqC@gdvsx-~pGy|Gn@a7Eo=4p9KKt1|0Ce zK>I%f6^e`5LPcnd0CYDVjj?nQqn~isS7L2WsXVGhFYl7ab$YuFzd>5z17r$_m^U>GLNP8?;6hXAajAkkGt!Zc6+)_hHXY;$m zmf2o@n{{&^G^_0tN|b0y@C{9B+>`Ka@7V3s#RenKK1Dwi!m8+@3t4F{D{nRd>3LXzWKmo!ngHPuxjsWG0 zRn$GNR2*_*uA(QhUUc9gg)(rYZAr1+(3!%ozX{#p@K|hbm6p8vMm$Bp4qGC4MFT(F z2F`Cd)GGW7@h_)@B}rxxAde51Fc7Si_j#Nl`27W3!oJ&ua`bg?e$A+^Cqfcx8>^G& z_l+i-ytT6*TpspH{e6vuMO>$CeEbB}SNOK~UzpK5*e=N9&UJLdj^!!=KyFfCCfo;Tqir4>P4uQ@@Ajs*<1ffX*vyL+w zEW)9Mcwv)^DhS~CZUW6R6xG_!o9oX+w1H4(bwB-e3W-N$SVA0W`4*a(8Gc zVS?pG=iHwB<_)*?F5DSi1eT87Jl`W}`+x@#Rpi(;40s}@>b z|6C(a{kxxv$O5k5OK4uj=97^KZ_F~B;)S=_sE0d7+yE2CU_SHZ4OINxIiV{PkDSt2 zpP2#bKw|6FE3Y@G8LOlCb_CM)LWUa>@jS3|PqWhM5yiUeZNWw|CEwooxYk*tr5Xhp zRXosHI|Adz(&-JGKuS-8Ovh$(>0j)=NUUPpi6dDO|FycYdOr%Ym{>qQLW}?Pk$0b; zY|Oq?Spjz#U4S`-X8G0mK%nxz66qI%i?1`zqVPIG6?1yBH`g#$ReEsaUJ(X-&@OxN z`l=@iO@u7+?c(@tl7h}FsCgImX~g9U@C~qV1=HKF3Y1S^yE3M6HDb$ z>Fvy39RKy)RKf)mEcGQpmUC6bZ0jn+H_MzI1q&k40;Fj6FhX>(Drhx`T=Tk#PkVVj zdwUL<7gf|yp=K2^)4Iyg&03CwwJb!5=B!K5G6#J=OaAL2PTkt7dCz{;7pR~RfiBFy zcBx1V=ifAAM@DfmOCTMMN~jR-F3G&^5c_sAsReQE8Cq5*}fM6s)e zT|mrTFtLc&wyS2xm!WFtPXYFY0Rhb3)I_)7Z^b0HI@T|7&h!B_xfXpBvu&|1bG}xf z1Mdtiy*FdhC$Hol_{Ud%X#|V5&M96ZpsSs)JMguqd5imE^c|ky{>!n%3({aQHN2JMW`AGADav@g}|_I{@-j=0vR*dF`ofY z%^e>Ai#$F7O^|Z5B5wOF?Tq}$)_*ZA1IlzW%na71X<4@#iy*lI~iH$&}Cj-0g99ujt8D5E1m zV7ZD0xe~o6z217z0=6CuXnz~$H+5kf5uQ+Rvbkx=PDde~ zdv!bMhiVdv^J(mgNbXQBsF1D1g)H1*c@&&YPo3xjCv+cKtgE?%v$uNN;ASgUa1Q~0 z97_n8#i8bMN+h0IvB<6V6cvJjS;KfA4#%eBDXzZoSjq|g?e zQmf8P=N%JcQ`sREvp&*NiHQ8(q*3S10u2No;1_9tA0c^98Br+K$+rH9rTYeg@Q)^e zy00+I={!^C{WmuW>5xptFW?4-f{!%~h0q|S1Pf`f!cJzh&CrS+zt0`I?)w?UpbM>; z`8XL!;@;36+)R|iwm{m*cp>G%02@GvQr^Li%mU>?5i=AeK@rnBuNbNvL5SNLtiJxO zHW12z%A@8$E#CFzxCk=4R*_fa8%x@&F^CtswH8scSM^e}d|;*<0T4u`aypGmJu!SM?i+K~dnJuj|af z$jh`qR9b>}Xz|`e)^<=5AW@g;=GdOw8{@F0sgB45?d45pP$3O>olZE&wzy*krbS=N zHWIsoJAJS^*ve1Gc7E0{5^cBw4*O7fzr#_9z>JGv7B-SH!GXK|b}=MUcGxY_5I{E( zy+9-)%$djtM15i#d}V0w<3+24+ZoFg?8KtpZZaf$wT*6n>k^A5Z}&Da^xK=i-hzg~ z78ygDZ|!*eNX@yqz1WUsp&wlptgtuCPu*u5`iZs`LbEBVonr|}e;vRoM1!Tp#(nZT zVKAnb)~>(dC8QF}1sr8eP9YJtlm3794UB~mT0U)uNB5>pb}S;&Gd{5Iy2{jC`uV|4r29x#XG>ebat0V!pHqgYbxS)_ zrf%>Zyug|_-_`>i1TeOJfd@V;#IEPFv9Nsy@t6 zJL3Qfkr3Dy>jkAHll~3b){qlH2#o#d-EoMI&}@yU4XpkT9;k}&A3RV1`F|EXC|INR zZzso#s@ITPnDlK8hU&Fpd8xxI2A$9hobQN@vT{+tygt^gCE2O&RTI9wRA)Y}K_1Bxf0hg-CYk=b}qu5Gbyws5nR zq2C1${SFR+tEl)SkMu+NDVNkj*Wb#IDieEfpSL)7(a zkkWD4Gqm*X^0<5Ws8P;Rq-E=Y(R_W)Cf$oo_H$M5=K5QeWnE~U2EPP`2oMo7=X*)h zdSc_B)3y34xVoA?*S(t8{=V%YHokFTuZR3iJzF3?{#P zD~JDCmkPubdw~9*wm0pKbA?Mrou6LhTwUz;Y7m^Z00}Dk`|5sQg~r1X;&6koVk#=6 zzM^_adb{lEG;*?fSKO1>-#>dBu^*GOVCqG~Wap%@^2)HKGS8uQ0rPINA*YdkyOk`1 zPS5fB%Ig!wbNvNNZ&p@mlXLA9H8K`Hzn$|X4$s;<|G1bNQ$PEIV~_IfNv-*(cU{*d z#{g+pzzE+R65r8=q*GH;5i8KKbjE=2rGf8}WJoE&^%28uV|yYlji3_?oui$4?%2}) zs}2hdA<5qj(>q^^aE2`f@t+Xm&|a>1Ytyno5yo#SQ*TKGCBe>Bz7+BZ`2*PJFM##? zmp5YX!kgXL(QkK{I9@r{SImtrj(RR|R);1H_jR9fTJ0~9y+dnTbWhrqsH`%l{ibH4 zHSUyjq2o)_DKeyLcLqi%)WcAb6$#%+!$%2S*9fX&hIW;Bmn^p}+Bl60uUaX6@Qlfr z{6XC{UTN#f9cDN5E_U_(3r}!FSSy>LjPw_z=B6aUd_sGo?(n*4{%tJ3clZofN>;XH zc4awiuEWb9N6Vb{rjyg+uJKUU`I6Y%SNSjn_t>R$qlWc9w}5NLy9~ZAy{<4wMEu?b z`02bt1OcVVa=mF$7!AhrDx_VwQr7o0bd^5)?VG>v6U%zH_hCNv~){E>C+_-%d{9x`^C(D8=L`&C?8 zvBx{a7Ts0s$Kx*-EN85S30Me(IWMGUw)6&LEgWmlt47`U0Sy7Z$TBPpDV*7Y6bLz% zR@>@i!+diMx7PdwuX1sEeMi`T)q8q>+#xFR>bPmPh2V`)XsdiW*MJdZNF{o{p?G{^XJbS zrb_sWI&D4DMV9)$ezF^QIBe{F!MQg-e1(aC)`@NUL~T7300BpZ09l_4n}7uJ_dPO2 zHm!d|Bs3Z?cfJ^EccU_yvZ-JskNT>anOSj{zGdMpZ2YcW^h-6A`>Tqo$VSQPWJvZ1UiXUwO=uzBvGVUv1FN%JwHhq3j1c~KnC#>WG>o@uq0 zdt^;}eo0D!8tjv=;mB18b!_D!-MgDRZ*VH*_CNI(aR`-A`i5d*oi~&E$5ZncG3ON^ zM$45Zi^=w?M|*oRn$7*p*`oJugH-Uyc@+jK&-i1(C+W0&8aeZp2JSSOHJLB;C}$lJ z5Ly)a1pOY;-e9w|DAtg$GWL;i;CoD%f?u>oM6{M!+(5l6q=~TAAQq5#kB%by?mA2V z!Bl0|xu)see;mPOm_N=5{*GRzN-qK4NOngMsT}#1)3AfB-t$3 zwkmj7O8%aQh2JBZKJ}7Nd+eD({&|6CpUEe){$IT!0j+o|Uedj)+zAMAD ziS>e)weQfJP8D7Z=M#&$0%#!q4xt>cPkJBd&J3@_@KX6!#2vrw7t zd-=ZKJLJ(o`0=`}b(?rRw;nVKU6hB+y2lP$_qDs!w|0|HUAldLudU~Wj90R4R~4I6 z3WS{cbjssf+j1mnMV;hG$!EPO{n)S{pLcJZ8gNfa>0m60y+N)OUm@Ci@|tS7`B%-7 zK%sjTb+Eqr%oH5m%itFcRYpvdeHIS1&Vvy7!= zSwrp{(g#%PNn+#o)S7ZH7@Is^ouhAwp^{EJG2zONBuT5gZc1vT{g}Lmj-Qvt^O9F4 zk&+MS4o|=g(rzBXb8Xz*Y(TlOXPjTnmV44>c=%$KB&(eBpi0nc>I-Y1TM5$~$o)ia zb=LM$UGc}s=r~;of`lZ~{OF1G_;bxotbjYnDY`!|+7A)Im2DT$XQ2&)L-czY*Lhuf z0i#z!z{PH^e&As=d9CI$dF<5$TNy9EQ^=M9F>W`AJ|0cOEYzT6VP>$G((jR*sB>P^ z_tke{3Ou#PtVzho;@mm&PHXZk>l&5_tJf9>)F;@sx-iD&=N8xoEVStEk56}X(vS|a zc^u@hVpW^j(8LvY&atmTgH2?rkozX`IL<3g?dOD_1>pYA9iUyHcmiS-9nB>|vuVf*g))R}Yy#S;m3KM@N@tl5 zrX=`_I=hPm3t4pg7)WJc38Wp9d}Vt6w{d3TV(ONZYF9?R)mMSiJOoiTwnI+-M)M^<&-pf(!LXZR5Pii(|5W3hHjK?|n9CI)A<~B9^bVS^EpaHkD zGZT%25|M;kI3*1*EJPBX-^qo{s&uRsKY0#JcxxxLHoTTMcCrs`oLvdKgVPDzQF(V? z?m17v9A0~iq3^vm{pBHhY(>Rwy461U>^jB2E?hzgg8>Lu%AbjUf~?(*cmes=VEE}P zvyh2C?K)x5L}IfMBeCy;mq@FW>ZU^p7*Tu>oDUEi+_?*l1DA(HUR-6BXo`_dHL(8j zr9?#6;vfS>x$@;`&E|#1G)DuY$gaA2{YK2)T z)+43{Bq0F<@p1dthk|l-o%H$?*XX zz_()Mx`Yq+iCjfHOCO65WW|IlaGqRrHfCg}} z#M0_o;j<*zEwZk2>joWcWWhf4n5-9v7^%>IX0}l(^B%Lyw@6w3_s5Y#I`ywEVPOR?8;KInD0Bm zhinb{2r!6fa>PUV)(DM|#IiX%YkfhO=hiyMfSb!LSvFQaUb7pacI&_XgDIj^_$;r| zXxz`NPf_AzkBh#U$GJn}3eP4}ofpUwD3vu{L~F0?NjDxOOPANTZqCwGf8S-J&!#aO z6Z37MH(aS#I!o?p^KrJ0Y>R6W$xk%|eLZ#|3JHUgD^_9&}lYnH0$_ zS7~B4x7h@o97ejgu2ogsSdsFhXWpHp=Vpz*R3__~VI1@NQ%GFdYj)n1=1)l}Fr(zj znWC%?CMz-_=Z94BbUhqw7u5y3{K_G3m3+VIY9sRv(k=xUSIKh)ZWp^BxXB-pIQRZI zl0A^Kuq|6Y){C(z({Q6?3o8???Csx8Sx5KP0QCV`GM47H5d);tE{WDXltSSjsFB%a)RVYPtUz zukX;?k#dF%D^_3(j^a}v7$cc$YkPY?H&61ZhU&ECFxhnZYtGhTB^gQthaon2R>Q8w zr80~TJB$1wGWb|dOq`9HAyU_GcHKxmR>pnLjT<{eH_N#xc%U|*F3vWyz*_$w^{aRw znF3J zI*Zf>l!lDq9oPN+9OB|gcaaqe38P446naCB2aZybJ+v~urOQ;^`8@{Wh9x1(w5MUZ zl^}h*$9f=%pD!Uf>|u@U>g%KTj1C8TkvXGue3KA#OzlPFjO=B$?wG+7)5SMqsXR9D zGsih+CWYM-c6(3w+e?=1nO-nT$vDC{vZ`a6c)YF25yW%CeCQ0>%C$Dyfj1A4A-bhX zKz8qex!4Ixniul22gSGr@^hArtvk=88ED4gQt&R^e5x(y^C+GuxvOwxznULa=1mvX z^bmo^W+E{*G$J;+K{g$GdG(tGWB8`~3bLG%ruuf3wI6^)sAGR-Ga7k9pfOap=SA(z zy=&=hmhR7CiHcHE=)~uF?k9Lojm9T4GCr{RA5-lj6Q{)deh8>j2F=lh9gZLf>(=~y z7_UM1UN4N%5{B@V54r<|`LiM=0gv45C&glRlkL55R?)onN>he|s0LTl~xFFM5l_?xn%NIst*SMWx4^4L(pZ41MC)-goZ| z(6eq2H+hH1m(u~wyY(msd5>zZ(cZqyxpEzE{Cp%WIayc2JN8RTek|Xogz<*2QHyOUtA%w@=a{p�ne@BBdOT--vfJIxPK4 z!5vxTB1*7Htz#dh0KzCB|BMz9^mqc?GxorGqT440?E0lzXzTU;%F+cmO3A zg*!(Iyg>?yIlSTRQPCX>kslQeV-$9P%`;ecdC>_DscOpa4}511vQPl{+OETjqz~WN z1=(Rw(I1cHjT;9Ob{6JOj}MDtBtM|F^8!lD*(8l1`NTG|Zu!k{?<2QIY@=R-Z0$BH!HFL#iICN?C=xOjjxn3>2yONJ+Cu zHtYVq?Iydxp$nnGTAC?*1=*))O%0_SaOdZ-4T0yZ{f5t>xV*6ce6}d7TxV(edwWzb z7CO5`xbjWvAz5LPZj6dr!<^BZ@? z%UxVz&o7irv9eESN=%L)nQn{@fF%^-KHMjmYppzH2If`qlbk`%!8EJsn8#}-Y>R2N z3%++-;NkO6B7m&`@&5zQ&>cVuyjXe+Lu}9O`vfDSzE~15v?)awPE4i=>~VeeW~gOS z%jgm;&H-~LU#t3%XY9<;`-eiBHmc%)NfY|(wMuPnD&k|W@&oTh;T@Y%*8g&f(hK$iypQv z3)H&xCQVD7la;Z(Ve^_78*P1uqAyJR_;{f=LFag&uTb2_aJO(vZ=<5{!$_f@0bU|* zTjr?@Ax<(ZhdUtEBGP%B`}|I{5Cf|jd-uinA@vijIiq%WqoSH+g+EAm$Xrm@RJ}0y zu`VkrKqvbu_F(kB}|0Dg$s9WHe0K50LL}X}iIUEW~m|uO0w;zi@`Jw9&0Do47V3 zN?!e3<@0s^!ViPHie(jgC9ifp;5p)}e@yP4Rfy4HaRRIszD-u8K`xDiXUSmSdAjY+9wDEGZdTRmN4<9>(!DIxI;*(t;emKs{jW>DqsQ7WwYh?1>n}st~k&ZJ^7UH^r0haR3{HQ^mFEf)blO#DjUk<;`9&k<$84bbFv@Sj`ykztiKlKLQ)`TH(4k|@X zA!^|<&=Qj(WHWWXJUQ&TXmZ_3hq?8aeQ8&VKdO&?3u=q^_a{!ynI06A^Q7Rd<2~Ys zkMBqd%L-rmDaM*i0l%D!ZAhM^lnd(X>#MbV-{V+qqXq}%Nyev2jOp@>p4E)qv-s>U z%IUu6t5e$FSoV57+?H?PC6>=eWAgdTBVV~#S=WRyVvLt1M|F#-l1f;OLxPW~Du2*p zFS*u)kyMWH;v3F{t@F2xFkYp}#;0Ze!=+JNUqXd53u+VQB&$cgB@37@x|VzzH}@(s zs+R3dSCu&AeN5Y>jrztwCPz|i7+v{ge(Gz|PcrU5sF%AoMfkE@M^NHW^@x_B?M&&S z2z?mijGFcPlb5s$^pt}Zp1d|>V>O{Sy#+P8b~2ze%f)|*mZ|qMgtCO;hF(O-sB<@s z;R!X`Z|8)2%P_bESeo?87ry&npPEg0FPRaQD1JeS$8pBkpojd2roPdjh2%5J>iWmU zeHa$yy9IQ)&hcI(UhKAWW*D74`clki(DA;VA z6S|3I?2?r}y3Y9___w<)^H%Toz7b-=F^mbR7w*w$J6b<6DQ$Dv&RSr}MTx#}@6gFT zUX*rneMCEpFj@|`mvh(^Db^>HxTd{N#}DQ=kz3xq_Y%4EYM@a__HV6r-+9y|L6K zd+y2d{bgejvI?%O=hopmRsOnPBNVzFr>!N!P1R)&jUN8QW?ESF_2nU3_qi5!V*={| z!S6HoN7aM7Efd;i>OIn*W z_MX)iwJV6dcY@Sz%xY^yjM}vou}6(iG4hRb|IYn6=Xam)eLm;=AJ>&DSFYFl^<0nf zdK0d{TFq+eWM+>r{PUPVN7h4k@TD^6eU7cdY)2S^XU z<+~yzIE*;VF>R_Hn0Zxfs=vlGSP|9t<%+V}6S`g;-ExSlKMMaTMkXLiE8`RA$p)|A zUi|Pq-o8UEW|I&_gxjHu^y$IIOcR&j()M;tuja`koNw{`zTO&q!9Tm%vERCA}&|{14W@fx1(-vlFMZ?4gg>3!}y9rdV+45v#a!KJ^Chao$Xw zWw*D0-iP73Zs$z;%Ctj*iWWg6!VWbPX=G{h9$lKWTcC@M%d~%r$Cev>$jt@XRs{FH zZl|olgL5RkWZXN70mzTX68EfO>$EGEE5xgXoJ)_PpWG+UKG7 z;fgDOl0{I0;jkPFXy05QNyul>J;bH)9;#G0>09v@kDKGe9_*=RN0GYuAb!If{@TYW zzvM8>pv+_+%r*ryw(~hW*5SYTJ9Tf}ZJ_E+0!WY%tHe#A-&f`SV*2w-%N89l=p@5y z`iFjl<5WFDzjm;qJSFG*3!vIjVKs?mgJw&Tp!czIjf&IOY&_h`V%W?B7q5@F8?}LhT8GTWWU*0(3Fy>r>oQrj} z+uOSa<%4qJ4M5KpVDEfG*OE|U)oHNu#_1!0@Ww+|mj*UPuBtbc)r`*C#{XFpu5>$u zGCEpySbZ%^{be*3B(d>D9i(LQJlqz8Tu@;I6Vr7KTN@_u;dV$- zb`MxVeex&yzK3m!n2{jluLv!(pjDwDw`XK=%h3zhGVfd0&Nyu{L6MT;H2Xo6#0S)X zD5iA7b9wgktygBAJsw&`6f$^ug080lcBh1~!7U%Q_w1jGtM}WbI{$K>qSL}!npRThl=?i@l>oZdP5DZz- z4aim2q+Dc&zxGJ$W+i?-DO@>%e)*XfZot$R=!Qh9P-~pdgfFp|Xueu6<%C;31YiFa zD6v0T;uX!JNT>~U-WczlgHQ2@3kB+|{w_~Ha^XEkv^Q1YO!vpSGU~f=OA`W3rGf(5 zDy=7u*V8VGV)l%e)9H3ZCW4NBi3>;&U4QjSjF-k48wK@y^X=3Oqmnqt!bW*C({(n# zX;Is6|50SxGd=X&dv9{o(J?p~L0T=+Xn44Jz9abU1=OxcvqsZ8myX+4I_U7`foE~7 zxknSLqJ_xwY$Ty zAI#8s)CeH_m?GSgkfdOtcWn+()LW;2%DeP#apt2~<6fe^{&>YKy*(C$6CX@)QoHp^IJ*+FC_f$ovQ50s?SWH$Y}-wdjL5Py~_$s*4B z!L`Omw;9N|l-PFhcV2QAc{?z;vNHA2xNa=abzXXVi+zRzenY%}ftvi5>>Ks_an9ae z{(gR{Nr|8<{ZvE>;4-FNR>tF+Sb3kig>jZ|gG&x=SY8S0FfSDlzSGA|;k>A7Iv2z% zjN%CV-Fz-^KV2cf*LnTjy%u=I@b7RfP*k;)G1c2XgrRUU2vTPifLw){YWaXXYD9?~ zz}~gq<6Ll;?v(_PaARJMV}BFuDCzr=HlMa5qfneEh`T!Ix-9U^;s-ear|0&{glH5( zB=Al+vfg8qjVM@c<=IBM-Yt@UOJ6~+eJSOb*NJ90fiDpe+w6WN$8>G0_e!=SySHsL zE@?8O@?@^JOVdewF@s3W6hzvSak{GXx$g6%51mt0HKq#|#!yS_ee%;JND;SapmPC{ zh?tgqpq+f+QdTN}$3dgqU9tykvUTcX#T?Mb^qmc2_-#$hSm5Xnha$&{(FYF%5gY$E z3jd$$-5Cbt9;*}>7+hbl?pfn-czt-tyW5aA9nfABB;rGH&xkhC7p2uJ80oipKGo&Gq;j&k&+6n#^sP5A$6%vL5RJUqa4-kz8hMrjC^MNYol4?h=>+^;;wC{ zIFHdsgWr%k4j4=AkwP{%`nBrWYZWV5+jz^lq$zi>+Fw*zH9#-e%(jp9v554+Kg$yyX z$nT%csP&LpIn<~K-ep?r_i{KaY7Ef#M++}QMrZYqSWP$rs};o|u%(I>|LTIbe;Dp& zohVWFU}#ujArIE~0RsZfSJNJAW;Kf#)3}qMnrMd+9$Gu5(?<(|n42gcj z!zSe(oF#77#>I=P^^^_bH)Jo;(C22@|V0GPr?26hUjiV{m)TnBCFywT0yL3~nGSDt&!^VUklobH1SK`WTygmmG5$5om#oe{Z_VlJ=59 zddIl9kM2g;9q z7VN*C+9OE({MDaXhFkutkVC!6i`nc}WVOYkByQ>b6;nZMG@X9S>A8B};_5IZT}m?n z;C6*Lx#WEhpj45KPl<{drU2R{n5on}p4cDAD?IdSxg2)j#U$&tcsvIVDAO`E2?Acv z?OhWrtb)NW>;^E>mKpqXuCChgt~Ex^E?wV}!t^7aW|E>+xD-vVH9I|t^3>!7BS6BR zw4rMq@K2u4qked%S)0wo8?SR{4GVhe&Irxjd+$o5&L^CjO4q5Gy)F1i7~6S`2^uM#5DEPK^N8NN zts#*NDTyf)TN>KZ`JwU$iN=CHjU+wN=K))*hJ}Dba*AQnl<-;Hb!{H%ycuxf5dLl2 z#6-uf6^XAAIf53LC(xhozT3akod~z;0d+lXA(w`|Lr;lnh)?~7gU{iHCFnk1)ss{$ z90k_NbY#p=+RH{=ca^b3O>LtzI>8AS*Y~BeDp5uF(>J}D_krdC z8_$s+JGa~p!>hk2kXe3m0tu)ciaKjj)s=LgTkuNp4_>XwgFslo^;p!&S<{9{&gxt% z!h?qC5-UzkSA~C#Yr#i(l8i&dZzafNXe`%g+2Y}^AI^B`#)|fJuKPhs(598mRrN9x zBieNf++*N5Igv8K6E3N`krV4eW#9rvbsExd9N^+KXkw!kS(AKDYo%pi^q12@|;@ZPZ+K1s$ATDvL)3iMg=Z0*00r&%n+H9w3QrF@Ry|PQ0=A|2_iRrvczMEFv z6JoJH%eK?GU~6X0{{1S(q}86B?_R!C5rN#WUI$Z-wE(cv6u-3#??Xfg5}i0yQBtw$ zQ7cQ61~2mu>rVDXo0{2L69@-QToa0lviiFl{_4GV6vIm>2F4|St`#LUpI+03DOXs4G*BU(q2M38x-K+2wJdh3jGHU8doO*(%^K>-wt)xDsq9boDx zJixCO5^pzW67Xq%RIL1#m2k@w@KpS}lGwyPRNS|!Jwex~yIqnqksOM0&8NZRK?!4b z*D~e_+MhgapLlO?-QqAh#^G^H0oxy@9;g+fj&H3T@T12lmh?f zyy3TG)gBB9Ati+R>8u6QWDy=_Nq~;ZN}plK=BKGGUwixEi#R^sdpM7{=Ems(_c`R# zzPQ>%#q0E2<&fG=pytU|?Sv)SJV_HH8fnSc2~hU>#xL76mE6MM6x6Z5r@1jBce8wo z1E!$Ac(X=C+UoFVHi4CkGd{0CMv98+;dvVGV5oN~8uk{=m={##$Qz_YJvYkPC!L<42( zo+sSg+}+>OE?6S}D!A*+eA90qgRCU(u6CgmOF}Pm;yKoJWR^>FkPD&44z0E=xjgAh zGCw0!pWop6;6tthDP9`O*T_F(R&Un(fd#~nT*d{mwlhM@%xV`{tW|N`*Sg$UzuV*i z79P^PnAVG_87tZ_&2S1_u!S?NbOi}R3{0zRdQzWFoXiJ^03V?zx&(hoNS2-QOcMjz zC&U&*aGb>_4U7T?pLkg0Ip{eZjB1x>056Bd;~u=A4ZKUB-&=jlHcT=G8~UxKI{z*< zEH(5AIDwnT6*2+OOZ1k8l}B2yY;~tgxBQUcQ+fIEzV}i|ilF7Yhl6aHA2*)i&)U66 zmUyK{^EsWP3y*)54yN?$!zsxF;QsOlK)^_rx1Lh zLD8VQWP?X!t6p!b>X(y2022aapcQ{bhm+>>VjDw;iCX%?gjEdi$tO=3;4Uzx%m0Ih z{5tAJxM(`nSSM&MR$nQ2nBO8@c48U=~EPV!_e}V@|kOV>gU&~nS4|srb zse)AkzP;MZ23j3r6SXb3#UD(My7-B$l~rxngH}OdfCuuJpgzwfX-5{Y0|JEFoY6Bd z2*?orE`qCCd15(=jEsyp=cRTSYti8jwn&grUG+2vT>IrG@Z|vJd|55qnXp@)_W|n!@!qK%Eg|%hQb8F`TCdi2@CQwIs_aiA3xt+ZPLm@6 zA`p>~miH_lTB*kNjrm3t?K8Bydi3GC&m-m6TB*0RfV#Qe7mw!BkyLIw`m?bErTCEt z))iFGoTFe*XwL4w`isxsMQ6grF*<>zPvwQYD$xS4X^Gk~bd7}WCaj|i{nCujj?hD}P6D3 z_lEA;gCy4vW==-df?E=t;ghNBg2@?#ikz3*U?Ao38vuo*yFs8sTK_7F{|2#wJX;sl z`kZ92BU!j?XSvH=+SyH3*RLLon+XJ^yGnah+GET6zK}&mobg|0jT(eq<(lWZlUG@9 z*yf0vI<+;Xa`LGM3H6%ODBhbT=0yIDD|p?Y)YiB<=32B(n#@xWOoXibfW9JJS$t}> zzEiJkI65j+ujjbDg}Akr)p>*l!Yx22mCEta>=?B|2Z*3w&F>OJi$J+Lhvr=r{a+u8 zt)i5`;7oquP4tB;?B!$)Qa&wRB!otau;9@4i_>(^UIV>9g{T5(2VIj2I6V-A_n@ zb`;(udV`hE$1ZC>LZUX{;Omtcug?O){jNYtvpXfuTAwyVr~}r`%x{bi?HnQwx)WT5 zr{a4nq%GlY+a7flHHa|>vN9&^4^!Y9ODRDDL478XLVl*g9!vcp-6G`@Hd(Q=!rz(P zGaq?HVfP57M}kA-Z$Y^V0>Zcos+u!pVxm>2Nw(ORQ{E04E!#Br-&*>EpPmW7i>TDp zoo?~7>_?^tT}z+7IUjvI^yO>en15|fj;B-9hrzTA`(&2yXl`+tRtEdg+(LH@+iN0*+}0&p2A)kKbwQBeRMJ7N~6RlF9X%$%@8} zdVn!KOL63>ZfyW+!*lvyEA{^;;`5LCqSuNP6Un4qW|p7?fQSMYjkZ_)mOT-Zlx3XC ztnb!4IUX3P!Cfbqfi*a5cJ;_5Ma874)*RQ)mgs_e6|g9gf_K4ve6O)mrawRvGhUp) zD|V= z=CUWBf-bSjEYe`Y8+L~Fdr!NPW=ZiSD3ZXG-W#tG$%+4mtKI2K0Xt177k%d1quEzq z6C+Zh&Yy7plWw0cXb-%V4B&co6xLn|PFyOgyNL_zk;$nY8Wr(q7kr|%VF9g@tdZa+ zlxTqt5bs(d7gAo0a$D@tj=mmZ93M#J&9A2tMGeV(SNsr!Nsx7!js)1KE!6?DcV#R& zJCtg1fOq!b>OdTdipBrCKp_mnZs&v3rGiOikncZxs+{w3GWTGS78*>Rs&Fh6uYNzf zjZ>g|i@+aB7e^oBRnKYk$=ZgFlX=A&-6O-n!N;2;WLw!Lhp9^FAC&etE*kle_(`l6 zKS{pq7T;>bZ10wfD$r&?)Klb-YNcCqXRBQG@W(PLOOXaAJ4XvAWj9U(mMC8RqtN27 zss(~IsjO-rrG&%Oi>&Ycuq)|∑h&`PBly!}9;4(X7d{}4P|AFI=_wuVsHVvT?E zh4>2|R=~QHxUXE7uh3fIrRaf?Lu~}UPB1xm@yXL~oZ7pRBCneP{+U&_RcDTp{Emxb zX)a^YI0Xgb@|*Ju;z=HvSer1`0db8c3vX%*01*shp~$uQqx|La#*5EeLcwJ4*mv z>f^oHm`WuT`uJSHUQx$8s|nkmmvgigfsT8h0(&YFe-@xU+8#woa+-)_F@EX19!pk& zZQETtQ_8-7oso1PeTctsa6Tw6)9&J^LZaG7VmsY66%MPv|0*nqK<5fAT*% zfbU?X%&!4eCy1ePs@xJSXZ>0Z{IGgn&joaHDgWpSP%L3g9KGXu-=v7Zw({fD@#wb! z`suyAJ;q8%`_hHySqkz-OvvW)TqIEyF+v)1DJ?N#j?7Y%Z;nsk=AK=AUBG-$L3=nsaCwI(>Ou7j ze{B>U_rZ>(D;RR;#Qi$8pY)NBwBHbWmACR#ZW$R5zoO=vvbMW#tA88oGQgWc@tx>> zJG&VK2JHryD%vFYJQ(<~vwYS|$y#QmPk%7a3PJRLZPesHWHTF04AT(cy=JPl+9g@l zBt*6_oLn3r+Zva3gGCt%r}?A=o=mYO-byLbmG_%3*-tlS!}Abcu6!-ko9I18iDa5$ zez7fG*yt9k{R^S;KR%+QN|AE1;oJc$F>`XwsOWY@68BYkO!Cs{Y_ZLC*qMnp%2dc) z*PjAn=VtT9lV=)jQcpjMvMI#bO^tno)CMq(t6kYm|DrFztMduRWQnS)u`Q}El{N0a zN)u!3lDrI{yl55ideUrad6+pR^Y|q?*{aLhv(s9Pu~z*}-XxQQ8>)J{$!h%N%-+S$ z2iv4MgWVctB(#^KTy4^V??1C`oL$FfWv}JVrH`EN7~0?eI=(Pw)5pTb<>|{HH7F`v zClg({-;l_wE9td48CaPW>kg357KkU9bj-y~E`R$me#tKS`B#zvO$hOlb_g@}k`F_T zHIJR;bM0}B`zl%=$_XdMKvWOF*Sy@l{#Ng+$O1-*s<)4JReg}n2PxN#pbuj|y@kjKG(XAW>7NcTtIF4ILAZa`=l_pOchNzdvLyA4 zZOQ@n!8|~dcQEzs> zE3))UarfpYmGZA2rE>zOzH2v#Fv8ru*va#?od90?ZC-8tZ#Qiy(Ag?n^|SmFp3Kzu zJs=8m6CHW^;HEd_k@JMf^Tw9>Z{uyKM}h|*WvoXdWd6;Q6w zXg56B6|!X7eyp3>&>|@mrVZs4+4ps6057Fw>FW=d-sd51o$0lntNWSj+mqw6>rkdU zp2xw<{!0#^i37g-rOK3S>r#3%ugcUYdrw=^fokcNk|uHFc6x5qY& zB#z*E4wX2BZmDEV0#GCDJa!t&!(RxlIto@%CD3Bi#<3DEM)kwS5J=Lyt`J8LEcI$U z!6Q;(iz$A!va%~xdZ+3Qt_@^-U%l~k=BucJ@zjWMx zP^i9G>`tK)OEU|)5lAMM9d>60M67(x$~8w8|0~c?>_x9yhHqGEb#Ic;ZUXJQE&&uXXI*U2 z#+hs&jL%^3_U(=ui6v~A!vwAS%2KxVP=(;>GnYIK(=n{Q#h`&AqixD-tEr?bY%(@^ z1!edh{2mJrALi)qy8sR&_8Y&;-pXBH^KZ~fx)KqDs0Rz7&3>Z>29Vb>+S(g>Bo}pn zjp?&QU5B5O7`GSS`sP2mmoln@iE&h>Gx$md1Lxh`5nuQe}MFzYFt&{B^)9&m9v=E?ZNU@!}JX(_sl~ z?+jh8e6tKA){jHTYPx`GSelkP=ZNn+UeNl&fE{f3#s%8C&it1_2z~4Ib@k77_>{(* z_xpvL+Vi?r=TW=oGstef?FY(zf#O9Dt%>C7u9c8x%w$Sn45R2g1wm2io(ha6 z-(XzzH@|1UUp^Y&U0C-ZWu>WV`(eR*DK})l1JckIKp?@;$@&SLim#kRR+m!p0w3{@ z=hC0AG@gYYyA5kYUylzGNc3GclKUwisQ-O zWP#V@a;wJ#YhVcf*2`T#`Z(q7VBUNEZ?GC<(>c{vr*gAN z<;(rlLy-j0NNGTL;z0pSNxyL>^0Tganls-hB~1*Ny5I>0`q&`M$iPkG1#xeswSuAW zSnc|*L%plrz_bE3*14LAuN;rh1Ng26a*ET}EGqrmwv0LQ_O2KU6YF&Nfh#%wO(4YSO__DxPb#__A4Z52J+>gcrMC+jp^cnk&#D(ekOhYml&?mcC z#?^f1#;PG3Uun|2vbFjUCDhAtqZV|sp*iL_xGnbE1U2tV^~xU^Cu)`(0@&D{ACS`x z-P0x`(UKp+kYDlm!e6?v^&{A%egO_q+Z(n=E;5!%#*H~iQ)_0ioOoxII&-imGyS0b zmVe#8`+cYM|1yCZ_tH)NfQI_!t|-^!SVMbb6Dj`e3>FGME4R8OxIz9()Ad@(lp+Q6 z>ouT1%D$m&{>Xnqo}$vyU(3u%x8q<5b$2vJU2NqdFGX*P^q|E3w{&E+S5!g@ZO}C> zX6_P~CWC#oyzVCi&s}wXwwU&+AlnKR({o~?hieNJ<^lnp;F)xJKTjPnI?xa*aGNG} z@d8i>^O4LQ!rXZFX^nh>xLmTfBMRgwD9ZMIHm&!(`;u>5=sc8{2^l85M?Q02zZQl$ zxm^_c_^{2O-_o`98^J1l^0<)~($x|tQtSc$QX|Fs@!%Odg*A1kX&1*GajbcE0*Jq= zo!~TM5hYP}^ejUF_gKPE>K!yg<3HoVY=D503HNCh=)| z5y;zU8%aeDUhYA)9$BY8z;FrO{W&i8mmVcV+VN_OMxKdjC11ff-cICEJ4vI6) ziQW(WKjl{P@K+JqGCOp-h*xe&s^!;OqY&M?hkDq`o3kjA%896jYIAz$Q>{$GEphEq z^REliG&qTDI$F*%*4=(liWh=x^grcGd2LrK*8$;aY5~_CZRTC_YF-aA3^*C^YRF!b zN7CSG>-cad+MV5bG~7iz6h1E3kM=vwfzIo}VF9_I-vNitdl{2_;qL_QUOSpNlLWR# zLeKI{KY1d(s!ep_UO|5on8rReq;MmZ8rMONe|`9#mVBgIn?B;Cz-#o zU%ORsYEMRmg~@{W9o>nId;VK;kY#{QyV*B1rk|}v(MBoijhVEjXlRm0nowDouZPL? z*L6N^`ITml5uV<|yufCzCUQTK&3o~NY16P5G05QLYpdTg*L51|bkYBQ0@8P90a7vt z6}=~df8P)BFQ<;bKLp#0c2(-vM|yuudcG|%VesQ{XoxhMzqlb-Cf-O#6UNdQA`=;p>i1azvsQ~H>KPUDb<%s?2+-J0lynPwZ8_y6V z3?emFS35ie1=ZHwOHen8d0N_5T%y@L zSnsiBViA(>&>%jR0^dJ2gO)gmGq2uh6=jaGV6!Cq3eNN)bdZLdgb1BD+0`@LPXPQT zJ<=SE!f&jSWjYc&(K}AAI`p0skmmGuroEmcNccUk$Tg$IY6+V!L)>DSB($aRkq5N7 zw1~U4LQp@T=GgW1TEaAZrD><`r?Z(}D1N}jAxN;&>v`c22mcas?T@Ikx`qq+pR|e9 zU#qml0=y#93S`b#8oY3yzDj_cYsqc)aSk82Ly2MAt4syjs-hR>3qPWpmn~y^%4xxb9qSU;~+< zfzYO@yMPv@flNYfD1Fo5TlteI{UU2jDNWl|V$DCq^)uPxL>Cp;IE$Z~y5;}K z4iba-BJu0kS?gh>^%n4;SqB_=beG}Msm_jxil2w!&Q_scn>I(w>v%L*mOL7p8E?5+ z=5gBTqQ|SwF@O`b)YW&6F@Ff6au19^u89s?nW?zN#%+ZL_m&`8h2r<7*HHGBs|RDz zhVAi_KPsj}>y>K?xqWl|TVb0aMOuknGs9i*UP}oGW}rII$3KR!3Rij{533 zw7+gSA>+FHu!jAmXLFLlEN?s;<0H;UW0|mQ&If$vHV})v*68Z@5dd=3~?$RWKK1| z!2F)E{gRsJcoBe%W+4Vh4)$xoFTVRu^MB_|Fcg|Bdwi!4Ad40)$TLenT?V0}HuS)v zDJH^IQi1JNPAVIYGRLQ+-6(Z2rA!yjB~XY)NeY7PD%x{i2#s#pVrRFg3HO zoxlfoFa^zl*z$paiSo-#R9PR=5=$Iwj1i~3-6^}wXmHcsM^m$jN-|}FK0_?Ql~cCW zQ`4JGBV5*nZSc@vbG#EUEmwSmcCj|{({K%|rKe=^VuzWf%#Fo4m(}~!vZ>Tc(+xdkKcl|hF?XTzF4lT^y^iXcS;Q>6w>~J|6Z_g|<+=e!=jzjf4#U+WfY(ib$GDfvJzF_7|q=vyFWEQzPgMq{$w6V^(fJo0BQc6IoV>{ zARcegRViywRv)BA9cH^B655ak&Rz#{c>oKAO!*JAh{!;@vXctuDT zNy0#bzcwwqdAf?jRDOeH@Hs`;!N!mKcFW0m@QXh>%IJKQ4}*B{UJY&e$nGwYQ#!1% z(d_YS*EI`cemalHE;W40Dn|wMoyt*O>W!Rc6k}VaNLqUucw8dQNyOY?;uUu@t1~2b;&UF6}Y}<2%uGisl>(g(mD9#@1L&l@?49bLwFoBY!CF3 z@{!OrLRhw8Y)!9-2;}}y$}_=Wo*!+>pA`bFLO|cQS^(I3K7-_R+W19%eMqzG;oOo% z8=#>{P~R=le?e2DHZ${6i>j}W7698Ln%3CZk`RaCa5}sngg$(>V1QkS%{R$8Q)IwY zGmP(;Gv^H(&S$vFjQcMKmJAU_sTZCgbMy1R?W%vTakIU#x+*T2foTOc?=>y#xa>rK z$O(4UGnyT%u^a83<&%PyFEtH9FgohnK3gS9{}e3$YtjF6BDv@u_u4)K(~S~^^fmi| z=1d%o+kv_6=C8`E5H;Oqnu|Je^TV=0IMHD(>OD9Zi0C6rwu}~cKAi6mXXdpB5%0un zt>Exe=j#?!Y`5$(w-aZ7_zyMu*g)@1TNq0nm5R>Trtfwiy8|5>l?8k4pF6I<>W~v} zsOl*huXXU*5Fi_xx&eAc>|D5VrQKHj8lXHUO{r1E7Z(rLNV6oRO#FgD0W1zXHeeDo zE|W>Y*uc|htwdUWOnct(Qpkla=Fy@iq}V@bDt7Q!1dZ?PK34cy+X9G-&Ok--Pg=+z z0)9WTVwG`pYAKJTzQmqUDFR4|bA9jMF!`f7njYFNH>hoRzq9DFrkFIpC4R1wGyI_@X?9&}%7J@JRLMrbl$;H=K zS8^A9^TYP_sY7G>&j3@`!V&L-r+2G&GG_UBNvxb6g@Rf6Rkb#uk8OcPeqIKi4rzdb zO(U{OBEw|K2}zZ!xQo;gdyDFNLTBUC#;pq$b$S#cE^T`Es4})AD)0|{sM40wPv(WY z;nF=wakWFGpt4rB7Tzf?gvfE0cg>xa$zP1zruA%a_4G{5yuWJeg%}Nku5lZ>qYOE%5Vb+645^B3 zWUH$tZ9(fE}PEit+=4s~ruGXw7j}AQ}FQdhRc33gBrI5+1w1nM= z$Re}rX4C@gzkTG6u1*&-VT=k^aJ&+}LqxRhl^x$4S-jIqljb4@WebH&A zbNbwNXSLA=^g>7ZbLR&#tA}fsUBLx-&O0fd{(oO(N>cRmqT~hnEFF=&kiOKZp*HrO zR60zIHxBFo4o~xLKd8;F)UY_kTEV+DY%t>SX)qq#WXh464OX#gN>+!0+0Oo|{$0mn z7iO=gkE~{rLN^N!WR^}d-;8V)uBPh^gNAGTdjmlAt8#rly}ONC!}r=JW={V?HUBeC z7LczODNe`SHV|g_Z6*(c@M9wF^lnSj65uD$nSGhkvL5WaU$JFtzY|AIGzywu-2VNn z6hpn`@>RYx@Lbu99oZsE=?fdNl#sZxzP)N~+Dm~}D4dG>To6cX8?@~`ZVM_@E^J;t zFzQLXakz10M>xl6r`d+PqkV&~S*t@sEvAd6vj)67Oh~&gW-3PYc%)B0tgiH0LGTN!`8+IM zZfR`rQP9X41I-e@1v6hdvEAIP zhGd#t80)hQb5`P0?9;!x$JaI2h_S&BNCnVOpr-aGo*y_E^;XU0gbwo{s3VfdZ1|Md z)xN$ji@lKF>X)^J_xnfb$2LfVs*3iokIzN#F-*J92&TkG@n;b;k18)P1uX{dl$V)o ze5_vM7&VOfG17wB|E|(xBhJ4h5@pW;woKiZ21S++DXnt)5-v+uwxh0W$TNJ;eFfdy zZRmpD2oUkNf04aVGFD}q1dRCFm8S5g7i7 zs-n2tVuk`z$T!x=r(=%OF1U5e=V0g|YF;iok#6Q-@L;=xvvl|zn6Qf4ijC}JRb1h0 zB({Bb1ocApzAD}SExS7%`SW3bDX_(eSSL|)%DJ(1C`&?|Keg#?3`wkr+8C*;+}Vr^ zehr#$QdsI4#%)2iB$;60*X*2Lox}t_UTstvgnBAxf86Wwelp{>WprN(4S>9z7zqI{ zDiyXA*2G3jdcKyK`=PhZ1Ph%siTRZD_!>VI%z4v{tNDK+u- z91Rd{s{FQn@u5ubek!B7v>gFV$tDK9*9dFI>Bv84%P9dmKhPzsea_d2}gK<0)eNfLbV+9={1=Ye%HiZCemBRtB_W;4AZ9S z8q)nQ+B#lih?&*k;4P4WP%4AR?X1ggi-P><^p@W=J>JI|9M@Gnu`64<`ad0^qYCa% zt3+|fYdRTi{5o}7NYjZ}K2$Bi|t?WN1pBPTT%J_Pj#aH8Y6KHBtr zJ=zoBkqT59ed9X+;B|xJW}~uzQ(DU`t9(153E8^Yi18Ave{`gHFL;)Vc*?P|B)7NC z^sD?7FMmOPz(@(zYhEU$KH5`Js%0h%__Y0csu5j9M8c7_ta=bKc{XsT5 zCu3{)ZK2orFup|Ew%K~BOg3nwpmiy7Y?s*#70-bmnX0e~uZ1dGkxa|Yc}Zd`LxVe) z9$t71A9$Tt2#aibw-WayCVunlxeI$H$FI+jBHtg2$1Gn!60T=IpMr?bTC~L8rxXMYCBFKa5x; zE`7JZs+yknJf~1`rCPu;b6kkGnN?6QBNr+^0UcPC?xnpjYUr?B(lmA0nwqE{a3kb8 zCI^WODghlA`5qrUIQXgf-BuQYD)DhH8f}!Rs|#Dmd6Rwo^uUMI>!)O3H@j1}@mwef z`o$unW81fU_!_T+aswJ_VpY+s+SNokK04PZ|Pygb!VO(*%(eqNL=kciYrvHS;I6c(#H4~Q&lt*jcwF>&4gs|5UiH#h_K2re6afh=NcMr z8O@AKDM8N%LcF?_>6|g7uf=(m^e2GXWIVI!S5IcYDWP+od<7J$Zu6sn)4&2wj%Y~B zePU0l$I`3>ltbobnO6SEMsx=UR)pm@DQ%vsChUL;u=(2ZGmW6@5wZVUi=}DVfwH_; zLa14Trq0a)|GJXd1fz~)J;I_~al@-z4Lf80U*FEZr+pWqBmrrN|{ z=dY?t8hvdh)NwS~k$3oA39f;TO7^BU*ntC^38|ihV{^;PBN?biXo*B!|8J=+cF`LT zwZdT=G~oeCCXWdQz+aQEVb4bbaPSrP4exJ7>0+b>$3<5(@oZ^f?c;5yQ>OUoF-=S)(#&ZC`>`pCPT^tV)vuL z{#})C#0$K?nZ%h-#m4TBH;$5%*Q}`pRZVKiLE&hz+0t=K7`57|*;|;%YJM%W45bCD zsHL8msFk7`U)^2Ljcb)-DLYYF^H-I<{t%Ltg&Zl-l9TGNcLwZHk86e5s^`2Obf5dh z-gXjJda~|&D2*|gA3A6oeLkv`w1yJ-f^JxkdYF3wV7PJ<4APcAhrdSMHcHNq{pCF~i`&0UFSDJ7*|t3ObWV2_m#q^rO1E9)(EHqf{8ods+UjcV`#bcyy}$h0 zyD?B?QCWF&kY)b9-BEcdl7Hl0U-U?zFDZ2Rielr}JgtMp^ajl-3bDDGJVv&unM$IQ z&lZX6b{teH@8kYafykCsyx}QSz-g7~>Ojp7T3| zc)+}_f1bdWr`TLJIy+&)*t>r9-h&VPwK5eMuJB zBs~!suajd;tS#(L#bP=5^z$tR;sAH+-|&_cX}87oV>rdygX*KU^H(^nA1>usQhNrJ zHCjaZDQky4HkhK24H|ZLn)e8$>&}#)NZ{Ldi7un?<>E*k=qP#Gg8B%PK4&KT)ZYEo zAuOS=&y$+6a<+lECTy4-F5 zOx!&hF*7lmhU!lZBbhTeASXY*eB#%*`$OA&>b9m`VGM|K^d_tK(2GUDiTu7uN|tEY zE%v35kq;VWCE)!}sm*H=)MeRTO@>*|HdL>y6YI;je?EhijN6Je-!WxEg9AxK{fzkC z^m{KNifg3UTR>YOeOY6vKjev-4Nmy4>P$nUqkaaq)$hn>NeFm1Q_uNjP2<`{{6VC^ z?PTFKx==v*Aryl*+|yFT_PR@;vOfOS`Al^I z7~hOZje&~iq(`eP&O<>;J{Ra2Fp`&deY5)F^1|hKE&ro=>i?S!l!$c6GbpDkV}`3+ zBSG-WF&Wn{p6jxP!Y!m3JBTmOm31ahdaYSs_zF-Jt+_@xi)u_9+V|`nwFnOn zXSS9IQ>%Kv9I=qrERd{Gw$mIDp731Qa`-=_ePuvY>l*GBPy_@-1e7)arMpW}kXGqd z>FykcFhCkbLb{P0YUq+0dg!4WhHe;`A@1_HgMIEf`|N%1zmbr&zVD6aecoEnT%*m^ zFGj?tr|@srofS?hFE^9f@!sjBthVi^>Imhkzm8Fmvk?j{;6CD@r+vYEtxo!0dypYxP0EATU4p;b)nuO@fcE zmt{_2-u7}xx~p!9aFVrGa>(ZP|3du#U$S@c=jhpRhE+K*%{GxN`>vDc@=ap)8fSYY zx5QxbcXEwaa~9=6P=~~%%GLoG&fXYhm^p(C#uN>jPi%3V6M}t(32vFci02s0)7(wt z8K0XjacKFR1#Lt^J-S_*B?9a3L3?|*_VZm^au!kN6^@dQr|1kOKC8pe;}5hSI!sst zuxxRd$s*40k!2KzR$hAtXkQeTdMI!AE>?VEc+ewJo%7WCt-nb{BAy`H>j=(RZU6RT zNvf$7*sE0CPd80XmT4$Y6D>cPYwMb?NJ;qxBjyu$y~U6fZBtmcZ=M2yi0Yp1ZCOTZ*e>ds<$7!r%CkZ z>An}Q=TE4D`}wP(90nQZvFUg<=!}N@_lD`Ut*9>9Wo~NI=ncVo-9_H>Oh+HK6y&pr z?@AO;(6ilYYUkQFADpfE7@vMp%6~9B;J8po4o1Vqu&d8TRqrVr9SJ9AZ@cexyDC3i zp*soj+ua9OT;E9gKVwjf~G<`6ew#$YwfZMq~BkFqD; z9T|J_`R#fBami)<%IoEo6;N{@w~{l9-EzKQbi9tln1{wp!0IZt@H_XCC3dvC{&O3I z0km3gnv~QqHZr^SVX-tz3VYwH`H0CdHdZUdWA)3EuCCsbgJ7A0dKW^gMPa2(2rxDf z&2UJ?*mBp`2gxKn7HnY=wecf5ajiPjNj>N*6XG%LVcpSi<|^bI6P!H_-P=j)jILcD zW0Eucd4-iC3!0dB`g9MNIE5o45G#PrI9OqYX4gE3XUWIeKYt$16gM4iObBm#RL!5i z)N1te4xVND;O!(i#Tr{;rnU{O2UxgyIVH*WYxPZT>g9Ojwyx2Ywa3#K{@t#neLo*{ z#}iGsj+W5YCJQgWOW4_!_HK00{d;o>L1&3e)o+CAMP2MrQtkM&Eh6PcbmawA=)%N# zxNW}ne(s6SL5R;{SFgcnWpln{M3|7d(xAJX)p5Ou#N_^RYY5f2C-p!Y``WPB^DJ!E z!R;y>QyR|d(iwKMKKa7#1LP4sPg zU(oKb@9Ci)-R**fL#^j&)}ik!j7dM)KjL=Jver3d{~Y<#?OcE7gNHdU-Xw9)T(LfW zJ&N^^@X4v|TdrWaze(gKBZ0_ftG*VQRuNWc(b9k3OS{x(IljXA6~l3!`N}1WW%Wbu zsySH+!QS2ii^mwp3WOo-I*c+^qVvERF&*!+@Ls)MB>7H~25-Y*yfsu5wY=-^%H&a% z8#R4yNHZYH{RWih(XO9h=GId_;ehaI{9Ln^pFHtR8mSX{ZFu^CHZSqCnnT*st<+3# zKjzSVx+osOrlVgzz5rFrjXIQDUU51)s0JF|(O$_F>lTE+=s&z9|E{p-W`dzRY9{jv z*SNEu|3SB>CZbBJ>6LQ%_#t{=-pRmb5J8obOI6BsCTTqQ>7GFR;VF~#DHEY{Vj7uZ z)(SKxXRYEH9db>$8+6_k7#}*2zI6&TT2T)|EiOwU*>vJ%fS&C`n+2W&Pe|ygPv{$3 zyvYiR#P&*yd~#KJ1A{DL6aRmS5{06!y^RQYIM6go{;zA zSaview#zpe0ZdqY6cj1DL!#JB99V_JmXT*&a7ji4u9a-Q{!PjZUu1B#?ej9tgpR|h zaOGOM@>(bjoz-o}o&}fk&x7c&&M#M}5B5aH787o_mOM_OSxIYCxc)4pVe?RG_jJBK z5NO7UK8N3|#oiXp&V3RrZ zV3$`MqFN;;ArFh8oBQ)6x6QVTl7MbOay$G$s*TKh%IX?%&{ar@>1O8ZE@h{~rJ|ZW z=B@)uF%JiGP@;-O0I8Lw9Z>=ilj~-yBTg#*NEFXE%g3URHU8xtKlm65gj;!xn&?0; zZyrnUMZQ#3sWnpL&#yLQi_6K?fhUwNU&;?ZVxqy^J*jzdzQfnN$U%d{OK<)-{DlCl zIME{y?X?$8F*gRBb<2lg0^F}RrYY@sjq{nL#cI>Mq3Joy4~B+PzJYHHa#rT#NJ`X` z*^KH~OT19oYe>XN`26~M+to`dm7nYkrFEGZ|CM#>4Z>Oq`z2J6q@hHwT3?{Auaot8rQohOR#Den$ud=DSw6 z`}41Cel^5d53Z$)uZ(G4-?{dIafvs$p9)K{e_g9ej&!jpOF1dA<&g$PUjU>M0%tB< zdb$P_3C}taJ_`*59N~CbMPk#2=0L0S?hUy#7kU25?$(+lW~s|f6X^Rn)^8IeUc!;u z4KG;aykfrXaS>sCrB7bmT+D(+zo_IQWhZz1*JzAo=S!pxywh<@n23u->3`>~J=kZS zZR3(k3iTIg6cBL(>Rwr!zQe;&%Qd zJ`TCGCBc^Kf~Ndwii{q{jn^ST)TjU3s6Fde!z+C|Pq|F$eetkNI*DiPx`KFE=6AP^ z_NaE1#>;rW5)S-KiPN86?m}fApBep8!2CaQ<^O^*^bwbtk!)z|#gnJgkUVWCOkQOW zdrfgpPKCxWsU7=)uxaU1H0dK$H;x?1#tcqr3(7|Zw;3;H z3(i5IcNaiv-d_aOr2i2K1oRE@$?%XH-?-V1fN^GxWXG4gg>8<{eJp7qw>(%~L_w67 z;%)M^c^hx6sh2yNu^Yo8hK67h=!G{__1|L-G$!4Lo|rQ_ZuClrWsnVvp&esG^MQK7ljS?>LtVpt*Tc*)zey0}ax5N%c~N?f_E z%quCj(he9}Z!nWTobS+EuyGo|iTzW`xD^^ay<@jt{AqR7|GrFaHOZhf=VRktJN35D zFSHj{J-}ydguSOg(Tw`5cDfazU9p~$zBDnWI6yFuja{2P`YZ!trR(}~l^M(9j7rqs zcn(?CKN&Th=93DdSLIzHmtf?J%uY6&&bIdEoz~4VJZ0n&LIgs%zmN0 zhnku}g1_bHn^OX8nf=pa7h>g>*{epu4J>Sl$MeBj1f)OvaOsDzJi{`z-Kkb!=By_986rQ1%7wCE1Zb!qVkUE+0|!jEZ@tdj{H z`b^N}4@?5{M`d6c0aLS#zo+&aSpH&J&frV)W?W>P+Eiw8R!l^|YNR(lSZL%9W=fLN zO{TQy;0o925Atjq@CAOP;{wy3uGyNy8mR{o(CJEe?W8vgqP<=Xa+p^@)PB;!;!Nbd z%pI0m9QaAY409z~T+D*GJg_c#OBsbJCq3&g0Xf|zk_NmMfB8m7@+i03&)e;n_u1;( zucGa`@|SVop8^eFuBG}V)3}d(Y-2AnQw<9roReFhOZ1`ljm`^S?HFjji3A1rnUo8& zh@uyaN2_45l#O7KfG@@xZW5%PTCEirFb|;p_bvN*pcX_eEg)!Y*rpHX_0-xjg**JF6wcu*Oyt#ebR6?8We@&2-{;nS!)S^nxhWx zEr~|v?eVZH)V?C&M$x%_aMHtn6>mnC}CuGB#y1(T5hVc^x$y zioG~3hZ8J^^W2wTeR(}MrYB_EC%^~UcTL0<=^@QkXPc#%z9;E!Vsw|fso>g;-oTF= z!jsXpnRzdoBog^7)g>TaV%7qYB>6sO4Le=k=_$^%QhZ;XE5${aUd*-*m=?E0F_D9Od-^{t0l-!iSKQh<%yqK1e($z9@0Xmgfd&rJ-(CJpK?30uEhzcsvRh|UPE?i zsTK%qf^=an@xx^i8(~$4DI~tEq-IXq#a2&Qf9@jm9mLxfixsx)`A56O!rgpLvR3eD zhFoYyi=~ld6nTl_;3}O*yofth0zg7=#->~HR0nlBX&@3hKwG>>F^SqkfBEGSjM|M! zuqU&j7?#+A20uwhL!)LLR;h-vuH1y+YRxCM*la$M^AyrQP2z0qGa zl@FQPnSeQn2+C0*9^xxhZ4?v`1tom=FQ>^)#3le9bK(_uj^#1H`MS3<9ur0aJPU!7lMr>2kY4 zSGsOpe^8~BL?OMFReu69LomfD6pN$)2ayslOVd@9K&H>sYmq)J@1!=UV^eAA<40A$ z=4_KN?d^V$a$yU#65cxvvtw*Ge57r^sBWRJR(@}y z(lrDUU%o6M_m-iWcj5?}h6N(;f0cvRf4@Vn`WQ(PJL0U}KuVIGcb$_#-8&;}x&qR= z3XEc>>%;epw*FRNW4A@+m^);foPjLA;!bBSrVao?+AXgl6lWv_-w;S~@E$g|j4fxu zcY_cE_VEgPL!CjcIolcG7Tf6ldm3sS;39{qxk<|RG-RjE!#*dUiw!I=0;#Bg{ER}| zGsr>w=|pHh$2^;GJ#qrw3ruwQyO90gf6gdEvU?X-e`tFa*hE*iX{x6^vf6p0ur3c~ z)%QkKr^VhS*59z5y=XDq@Uq{PX4(wvv{J~~m%iAb*i^Wlql{k9h9F^?CP7A(@Sw2- zN7dCM+zp3Y9Ndbp>L;Bbj9y`G-8vCXq@CC;oter>`U^t>QHPa#6c)Xm+UxVTM2yPM zmFZ6BR$p2QsIPT1TDYk+8J4d$lvfbijHh5G#z6h8&Z`0Xa$!)KqH}-L?fi4wAB7;Z zlkjR8z$qL7pNV%9Kp%AtJ?y6;cR1^@k3|GL(~D8gre**pIOKO=;2D zN78BeHb~&)HAwlsn)&)ll9m_ZeD2D-;1;)(%HD5(YPv(sWE6DjpBp`0&lYvO@W$Z$ zTjjTirqTSFIiC*{?ss?1i+-XniR*LpT zo$iNZg=H&bw?kH!VcOWdPt?h4_4pYby>yu612vS~0o6!!m=h5r@pj^hAh2{=EPanw>y4hK1|oLAP@%pesp z0k+Hoqln2;JHTSjswRo3z>(w~?Sp&_&Qsb?tI`+diR&|nPacxKG}pgKyP ze43iVPvz4RH=HR6twT!Qem?+86U0WRHcwu4cVyUl&=`){e#==kupm3nrac#fu5KY? z?C8diwH3SHiGWdICG%;-9H1hHv$=e2aTiZkDYIWnG7gNEziWmDlye{+>Q@fZW(|o- z4EAoWudbge&ulbUtK@wG<%iei8`n!QT-UBYvjq2DLuRmUT&h=V^FGsofx(PvzB^bwp9{8^C)+ypd5dlt1QA* ziGC(NjZT7)P@9LZTW!g7Jyg7G{zOj7Ak3*o=}Z|Zo`8fK3WXU9g^8aXy@M7}liC^- zsMI)QOBB>*AKLE8bu4{og41Rp#5Hstxob%^ZCTtsFGr(K`C+5>PhyF$mGAkF`p#%+ zZ|-27l4GjM5+Ije8^rn98cZIy`+G0u^8CY%VHYC{Usj3{RlUh7*#){E9H-=?hO-aP zUL`Ii;FH!`b^ELjhz@bf#ADROYMdNXVek(?&CODy+Gd%wN`U<)4Z)Jo4Npml7>3E% zw7S8DpJS-1vx>S|?;{wE8~7}T2jO4Kx1|ZMNo3icr+6u)rp!J>b4Jo-NJ{WF zb;F)Zyj&VED$^8~Mn82c>~>kQ4o>;JmR+)4|BtNd%6+@#2JvIJJy3xPrHEH*@GWz% zggI-Z?uuBYaYl9K(Xv4i-Ooj)WQrXBP>Rtp&`>mb)E_BcLOUULL@ zkV62l<+jLuQW41#m0u8FK0`?;R4~Ljv^}LA>Xm$6-D-o*v>LfCw@7VaK|7)%hf1N*WP{y{%rmoSuw4w?|c&#a0|;? zuZ<3W7w56=0bYsc@oC(y}c<5Pnog?lmnEFMCY+TK8dUqlgwwMWo;>%8A-j76fn=5ZZg*cid8>*Z0kqWEv&Oj zES7Vp)6i=;@8KHeWM5uANw-d^om;WBf<gqjAO z((=f)e*_ePxtqvCWuunj^DhDhpcju*!Xbvs{}xPL*P`dxwK-f$ZCGg-Oq$k)E^8cL zTBEwu`89=%h=ZgP|FA)dC!$B?P3QUK@k*wQfX&3n@VlbVcHh8|XWJq+*dlkwn80Nd zHA*YPKfT`Ki>{*LqgN$46 zj^(Jy6FYCku|t4VFGwEwCZp5y%~u&l{j5l0j$^-qK2e7^AOIW6QlL4Ob|Q0f0w3p^ zl^JXC@bQYOH#Lz_da$k$rTD&*nGVP7TOSvH+V=^+b8_cZbSV~}! z;dl}B=>`<2_zOgdQvZ3=_+Nu?`fyEOXG`_AM7<*|W=XCB#ls6B*P|PLo*tf`tvxN zHfCQ*^?bjU|MAGbb>RM$b&!<4`uscBJ$BNFzdI`W49 z`unG6#Fwx_SL|tGF8>qR{rv&|bhttck_}v0>;ggmFH;Er4dIL7ToufOIlm(RTj%tr z$N2Z(SolHz#8*wQ{@%Fy$L0OQPrm!t>1SeS5x!dg|3Vk8@5*n65i8zN%A`ojA$rILiLl~wpFPG$U{!1ThxXMut8M%&@|w_kES zX7&sDQg{fGOoU$W3f%Y<~k`fIei4 zytMevXyPc~^}8o|o7>+{iw!ZuDq9f)Nb%9<6tq=^-!udFUn_)cZGSQ}`c`KEtd-r< zAQ>OiZVJUkZAC^99%Runtz~5Iq~pB(zKxlxLE&DbPT#`JavLpgZ&$q?iz?8W4A340 z)?HBxdCu%3lVv3c6#G+&_eyH>HwKZ~h@qWP_Fm-D-s^(MN)u!3jsa))&upl?(#xUt zZz^53tO0~UlVtNr3Rh{GXWxsY&NbfZdY=To-6|#nGKGzKzx91d#AIQ)gw*9cWM{Yc z0#W9hNAe7Qo)gYj!}{m7&3`SY&uSY|-eHXPFt(k;Y;ldgS!O((zLnC1HZlIAhg*{e zS{kHpW1YUYI83DzG(9SWQ@-3!&&bdc_dI7CEmCY^tfDPn)!Ntq#=Q!PWR|-Rh|FdI zW~Xhrq_tg>@DdTFXla_zLy2;)8rdng#p7>=22Sm2+&d@PB;!&k&jmFNhihC>p&!9r zN~BJ_#Gk~H4&0BAvy2)Svbo(mcptK{VXW$>nHZamrdXCHWQb@!dc@#eiIv$Xx;HCi zo&F6R0icqTZ$z^2iGoGkqKHIJHoEBD%TnuAvP+3u?LibdUCf~R{;XYfz&?` zm=Ki#>~!!T+bpJdHZjufi#{bM#a8VSFlz|m4cWp>SF7WO9(;~jEenHq2~ykfKH3up zlWGj{4_Efm{o$m26>b2YP*x+T@}GoX2{;PR<|8SR`YaqFu48fWSs_eR3+*u4PqIjZ z`v_gv63wW{LQ)!t7$e+FaAP~=6pxB$Ofq!7O@@+|=Mn67`4l^_mO5aVFB=m^EljVR zpWD~*x{pP?Rm4{c7uCK>hU(0B-2Cj7aSFfHs>%jh4WQ(DUG>Bk?eh-3?XTH+k#_IXk;;E~>?$q3{r6px)m2-rPgRcs? z2%e+1gcF~hRf6jk>H&tL`fPq~(ZBv6qD`iR;`fsTiV>~~n(Pi4ZeB13|3etVORy9} zn2c-v1{D^oRBe9fekl~%5p5IC;47I7%Q^!Ho!z;TdZ61{Ckz&7>=PAb;%+tcZucpn zrBaM2Uz8MR0=4`}su=x=K{mT$l&g{y7WM&~e~-dT<*j?tc}}u>eO|shy}259_pu@w zkIL6wbexH+0+YFfVq{BnqCryWj>r|Stqnv*Mx|E4P>xlA+q7%&_>N=e5^AwBYnYKC zebk-^l9c&539qhf(k|DFG1JXCtWr#8Q;Um3RFSx?1!U`}k~W2+yH4t$aA>mm-BLrG z9;nU?|11uU09OUyI-g?FD+w|o95}FLQb1{TDT|Jo{}B+jN%|(9BQmb(@~Zst#lLov zKfcSYHPh=0-ffdQPuR6H^dB7#6ZpWwepvcuDk@Ao961b8bgw8fq{)n(@XMkj*7&}@ zqAlFIk2Zg-U$0YgH+!}-E6HPodKC;72y~?E~*j*J-`xVtDLsu6!I6ys8E|Un} z?ThJo@v>h7A0OXPI5VvT?|r_v=nfhAG#Id!E~l~jgaR&keJ8xWeE=3oa6LG32PDel z+9yWrg}GV;j+=KDs$TUyl?eOZ&RE)RsXCL%pW-Z6U-&s|1V}>rAGJ{SOC~ExSz4;I z2MOIADL2TD=R4G0G2rLaRqVj2)rJPna^;t~>e6MdP60^56Vj{NlY2{Cjva7nTZY1} z6D5|_R=t7FMK-5$n9k*ro=;vG#Lz6Rb@4G#y~ z;aJTeaHT?1B+jQhf-*c8A~EZ3mZ+|`j_~%`N5}x(M}xFttWe6RBIjVuvU2@%a0O*j zhH=qwz7~6Tw_<^R-0FMmPXBM#bPr#SA0E1JJ(4BGx7ZOZ!!%F6}aBS(8zA(s9S@MA^64u{&@-Ow%1xj}u4+W_CVd!=BIkNE*5rTa zFEK!P;A|Q^Th^p%#&-usW&3q)xZ71{X`&?W+mO)d`?s~spevuZg|@*Y!p>bSAc)r4 z9;Jyyp{*-Zz_K##7vd;!ZRbAE`ej_bdY#qul+E|Nyn`*&Po~OEv(`_Xa)WP+h@cVj4q8nUo{jRWjR?zmt(rj0ri}FKmCe zD>Of!(j&pROdqdqB}INpgGXJ$#&x!J09#{yEKimw z=eZ-U(b_!~_l*owu_my|FOGHP({8e%G-70L#0+}}L|Ws)p^2ctU~1>Gn>Q1+W_wjD zc#+`q1VocWTF8LAckPw83iR9S_DJWcgm5Hd+iKZY)WSZ;G_w?o^po`{gXhUf0V%Vv z3=Mr29dQqXX^ip*sd^IP<LKtrB%Zkz83jOH#=$hH)I z;`+c6W-fTxgqr@$7B`JLbwApV2wV6zMcgs;ElVeJqQo!c7Q^(>=JXTY%?^JrDxzR% ztHFe&Qzv-&8Mn6M*gAX&v?UWawUZ}jv8gf-k7I3;Nnm*Tw1oun6_41v=2-7|W-??J zWZ$er;!?MZF*O?0noqrAc8|J|{X(?oAO|o5N_sZr{~##)%3Ao8rTc66*CgxZLK26A z{oCAbZJZL5O<;wYwX$)F!0aBgw;D`Tt9oICAI1Pq8hMQ z64F~3pe4h4r=$PO*kLEb;5z9h@An&|>$@dldkbj}7(LXpY;{G2)+KVAnxo>}#mD)6 z<~fYg;|je4f-$gdk~Apg!aZ%hzG}(^5h{Za{`8oJKX$pl}1*zp|i{2eE_ebY5 zK1n%MFerL+lVe226Lkvx}Lf&GH9 zg0E!P=HUaulFtUN=Tf+bff{_ZJ+PYNRf7JJkzRL=L1eV9m!cbO5n%C&)ph%23aag+ zChYd84I)dlb#!=nD_%a{*huh~U3FA}_Ps~25BP-WR1{(ED{$@{ifjdFe=Px>KZC%9 zTt9aeXs_S4S@+7r!|Cp)&&$?ELN}(%$2Br!A@(O_1_v!v3ptiH+cY-oqBcOFwXmeM z*Q{EyKh}lZ=|&jruaZM2aSHhB&M@CLXb4X`ulPY;)BrBez6%aG>3)ORW1iYdfaovUtON98j zYpS|9dV8{`I#v9<5o!=6UvTth@KBRdA>6h#U+qofd;RggCR{MQ&yp2#*qAzJ=i|68 zL|RiNZ6yFxu0C1)Xss{8WacbSdcu1r$*t^;lz%5fwBEe?w`?1ih3!jluYhH{V~T=0*fExMEb?UmxSGGj!y-e5i+WhW==XX0@I{G%+QmS5k$`ZlBxPpgcHXT=iJt&;{Te76Ecp zq0L<3sQo>tmf0c})g}*@AVUbbTJ04q@gIE7^-xXW8EeMJCsa-_yusyAm_A;JW4!}Er^MT zPI!7jqyLg51iU6Ap49<~4dC^|Eg|&`vz=9d<;(CMm4w^7dalxra6ifh0__>0(Clk& zpRQDYF8azj^CVw)89D?YLTeoXkn`G$YCJ?t$7iugg_r{@Ldw=+J?{Z)dkvd)Pg+tO zYVdGHPxW1K36BN4=F@;NtslBxwdD7Nkb)NWo~nN49?r7B#JIb|fish6uJ+O*vhz#O z4XWJ@QU9WOCNx@CAW?Mmm~$FlGMl&vDl(t?1hPdCI03Qof-aqJbAe1{J-6D6Q!ChK zPmX;aT=az)B0LlbaR+vCIDc#l|F0tQZ{L_H1*k@VTT@pKV=1Up%3V1?ty5Uyk|y)` zUWZu23pNV$ro85j;j1CvUnS}a7)Ie>wPS|La_f$a4b*U_-S>j(LpAN zfIY=UP$+!txH?isMa7O-VR)LAkE>l}Wo7o>ow`KRPO(nJ8=hoRK!YgEJ6bfHCO`RT zfYFB$#g`Tn@A2^tx*w)sU*NSXAB5MO<9`W?&Aa`-HRku@cvjdjD}~9}F?zT5?ed4G z0Ezan0PNB4wu!UlGk*rI&Zw>60p^RKJf@aS? z1dGAE8wM;3CdZi!1oJTkJo;NFgCaf?$H@ufwGspY-!?~&<3A-vAJSbncZ~*bSm*2M zW86OxqF5!?y55UVfU5N~X75SMA1B|@b7fpsYuSDQ9Et#+$NzOG(Lq5b-4{&E1=0DB zN+X$hb$z6)#W9ERT4Z!|jR7uCU)^O^>yy(niJ25EG*H*zUbanwGC@);jj~S za2Ek7`MwIC&D=3Xg1r^DOR$J*YR!J5${bnEI1o2n8<12;EiY+JjO)FlSW%Ro4v81l z+7d3v4mW2OWOQIz89+Ez7^&OEhc~~1XpO{_z~n!v`#XWoX-Dd1v`hO_rkOt*R9Yv- z$k{l>}rh%!z)Mg2dLYrBYWQf+G4taYaI|F9|Q+~?PQ z@bbM&R}u=C)(>-_85!fW`WHD3AChNN-hGXKMC`u{#(!6(WEhr68A!ul-leUR^-V`h zbZ5iv_gVThJf2vYq`EU^+}Pw~gbDKxYIQJI=kb`Lk5waJCTq(LTxo0#J~Io3QiIFZ z+~>`C5sfS}f||KBSFn8D{K7mGyu2O_mWBG24Ugph;zGT4$aCvj#}OW~+Ihu3P6#H1 zOyHoa!AK7lHLC5pcrgRWlp3mxyy8eGC*Rpf{E26pz*?KH&-Q6g&wMMorgX6SZJPCX z5fRG|XXM^z?(xeHRWf<@!L?jdAo5baRg%xrZ-EJ5jFzq_ShebyE3N zp7y<$BCY+<#D_a6qu7dRet&?qYM86C-yJYD_%4b|8?U6W@Eh*EBauPXS|c9abMvN~f2~x? zVd&<`#hjh55>~*eibBGbrH2=+$Nzj;NjT2o=Gd%I0}yT{f;~#!W&ct>EyK`k^nPn% zSWn#S+ZBpQOPmc4OP+h~0wfCjsu+q@$po{K`yZixJd)vc}b3rsoH ze!(@L;f#zELeAWon(cjudwAbT8!V4ua;gdaeuCrrS#5(8F2m3e>dv@}1TvyGgmeds zfFv~2(-wvoV`)V8Kd->Ssw5s15#f(9* z>WN%le*VNSXO}P6cJGcJD;~5|kwP8zw;u_)y@F5!@=HWF#=*D{lh+MLicM~5wHM1C zRCpCUWuF8Q5Y)1;9xhNWX7!oYGp)^B%>8*t0nwoCH%Fzve@l`L{yumL?$)Ofx{)1g zC;A%zZKAK2UgP=EG|9;6mZo@{qo%|jiaK4ze{SlUAfwRCfcaNhZ|$*q1^ky4iC*H& zw*373&(4c4$r^xAq%rhyc-+GYg0yGuHHY7T3%?76g_^|_)q+szbv-~}_0spHU_}y0 z19+1?u&U;I`(jb^YWpnph4@uc38(_-TIqz({p6>BTiT<$N^|NBtY!R{Gr3;<%Q>BO zQJg)_f6=XQIbTM)OE4!$2Vy-9-}v*Hp~E*3XD2l$6`}IY zkG{X+Bxy=-xs^>6NPhIKX9>AS$)T@fnM>U#7t9n&HzboE-zzyEFb_*tN-Bp!TScrI z!}birH&J=-)Asv)0593NSGTtMo#n!*_saq|A*@HZ^xIk?d%Q229~}B+*U$Z5>XOdV z|NL3*=MA1=#eYuCUy#Ifh);dt^G9fQNz>uEw6})LEIZDv%-bY(RyF`4d={33Tl~Oi z#4wol+Vkf#G)f!aw@s;6nWz_$dmgO~FI?U~zB5|u7-uzoSVS#c$e|m;C5I>-+kiDF zYx3rkN2Wx~PK_Tl?);5szeW(%$+Q8Q7moV1`Jm0&cNQ(y@n5Tr!t1cS8Xq5DUO9!W z$DZO)gUkuN-bMx#%-rglM;zYR36OWzY=>4Vp{<_0Mx=l7m?X!pZX3w;FTLa>`SsL4YP@=LW`-5IGiOizLo6hF z2o!q%q8rS2|K?R_xjaBVt}VRPChQgMcb8ua4+(o^tYc~U*d6=5)s4B8EVRjEB`1&Y z6VENV>~6&!4Gi!11_6EWBYyAgGQrS@w(KmcJgZ96jBBjzOmkClMz8!Xc)4GE-{W1k zQ@F)|{p+Jk+T#6~nJ~;>+pm*_toeDhv&#TK9WuyP0RG|2EO*S^uA`=SYaS@zUBt3?flA)sw#MIVNZWniAcEG;-iBX< z%n&C1rC(WoDqSu6t2&8p)B~#jqJut?9{4wpzOkD<)?iGhxQf&7NFtou_`^{_%PKT- z_NnQ6h`f5@qLb*5%hXW{zGY?2>cji7xf*s4xZezF%_e>L@N{<8)w7D>MHXII1du&{pSnNyY61izNm98Ovq>rdzRPef zmFL$7O;3~T#cz9Chv?*3qm_lNaQzz6^sE1m{y{)uI8&agko2&$4GoXYu{;^k$z8vy zQkgKV?AEn$UhB#E)g%i0 z-hLY0!J}x~DPlcdY-7!O^InTGAZZp(dUdr8GNH>Iw}`LjM5tsb$`qqHRE zH*9Qd4`bTDW|^7W2s@gVi?y^cOdf7?)8Hs9_N&?eZ^OU<_EgMHNl{AlUcxf~iwF}^ z>fSL85zzp+WF^5tvmAy)CrjDKg@ORW7jssOp07k@c1FIUyh?P5iFRI6N?{AWe0BrBE_Hn!2D=`~~NzKl0o>a70#|1~5_j=t=t z8XT@%=D92!WGJ3*1WZFJ3DUpt(%IFk;&y zMNM;L`z%zPZ3sI$TMvXID^KsS3i~JXXuS=~E;kWqshft~V0$aC=U-kaP=%)_=;r&Y z(HOn^s*y^7%z)8=z3R!7#UwMN)bl~yxJ&z4?+bG$?dHC7*iV_11Bl}~h&)Y?A zt_wqbSi(a(jTJ8n{%^Vqd`)tg;2I5oqhpAS$}Kz53fr(jL{c~@?B03G=-g-V;_XC< zD9M?*IOp}r+9qcB_-^NCW;Xbjm8^YWxUoM-~Xv+$Tp@dDnuejO{c z2JcsBPoH~PA!fe6aj!8fPya#CpK_lUf9F@r2bIyA26CqTK*<;w z6DhqB@zHN7v(>Jm&pHoag!CFiXuvFfA^X<^ZN||l3ZmW?usgEn$L+8#DGGGC_WuJ1-ZKS zFtlQY0tH~K$}!hg8fpck*l!M8h$evyp7=SBmJreR-HX!yM>zc-KlwV5UcK^;+ijT> zZf!yu zfS*kSFZY75mc$GjYuhI%DOa20Vb*IJ+Dp0UZ_|~bK-O)14*y2Bia zXWS{4yBEuSEQFd;sPpK$NT^Pb0EDUtA~y0OHS&jx`%j-Fui#(XBO#jD{p6|%$&yEr zrUvsPQktZnjlGHDyv^#W8kaY0 z&ZCRV3$-&-aiTiX7YhV{VG01Ii)XlZ(V+gPhrjqWK*WLMW}@LR0CWKk?{J;M)_*Be z`Ug5+n+3>E)oVez*+t38&wGSyi`RBCcYwU&A(`ktbz8+xLx237704jCF#dkHV0&VzCVOqzxMftX#w5hr~>^?Z}X z3^nJtcK@af6be^b*fpA8;|S2`ZO6GqfKQMX%Dp(pl3M1>TgofYz6TfW+u?pFG3~Wy zEab54gB)Q@k4!R{84V+A)4AEmAPv6A|bgs-F*@#4jM`;XS|=tjk>RpVkyf6(O?LmA>u zAbR4w!@WK7tafg@od#9&B8g5=gJjUJk2rO*3 zMpt5_W_faXUfxN39_MO&c`Qvjx*XkNfMLa{e86BJQ=OAHRnOHhF`C~xN5?ki+V{X)oW&>*XkAgMljyjhI34xU z(i*{E5HBs{+Ji?4GWiDm!_rgiGN;@ zT^;XV)6wC`pEMwQ$Y*Gcb9MbC{kef|_4*RyB`ln)*MYbB<$rwGk8D`L&U!z|v`#Wy zxBIgF&Q>L6xDct2`Qkih*#5+p*JiqmZK5=l54GQF)m6*55f`@`gQ88+nTcUIh9v3H zN%M!~j0qRVuMO8xtFsz>u8v|o{ODT8WR%rgdBtDS#bZhl!9l=d0w+#`1#cXrmz?+5 z)OeZOB<9(z>um|K3)sjsj1*OGxM}Sk2S|ugPl;_bKiAOE;4bKIHBIyoodCl+0;{aY z%b=-8G~cBbc`RWb(q9A_#`4^QhMQBXbOO!o zkH`4n>{^u$#_+SpPssR9rT0!;2N4WGbLkZK?~I-wn48GE-82FtwS2Io5>=n_yyCE%N>9J}ZX+~bo}K-iz#;l~Xpo!3h7jCIU<-xMz=!DE1PG8s1qw;4?-Qa|}kz*D7IjkDSKBGfR+cRRV-7(4FZU z@=L|e z^V!JVp?=Zx+5YYtLXMpdMo*P7o8GVVtNssrUl|b9*8QzqP*PA*QWQaHF5L(hDG3Yd z7U}L`z)=Bdq@@uAmF^C~VFV4(BbVNKTMUM#+o> zHNxk`Se;iC$SGvCh_zWb$1YW{x1@BoTys!e@wlxxV z=(Zc*R=uDb^>p7mEw8DD=e?^lth$)di8U9<>z26Cf&rr1SW0!o5)w4+$#oyN$1Gg! zb9NOT&1wH0SNHY!Us9L0n^Ft2wg)mKt_E!thUVz(K`klP8zOS1O!toYx<&bBy|6G^ z_qUOLKB{;|*m~iCGCnPZg)8-CM&H$)u~~SCo`ZE-mT@>ybUCT@j!@4INMFg3RuCQU zMX1Q;UUquNa`I9O z9ks96yel!>8`=;#z;(5g%*_F%{_MRbY#bc$7UHdX@-P?|O#@_W%BWYy6LV(kAOF7v zwwf3DT@>psn+eQJ8EUr-Y(l-%>8zTAtBVJE#u2GX3(LX+x%K(dX8OYGotj|rjq&Kc z+foJGx&@~5-N4759>lLa4M}?*)s#B-=$)#I6WCwaKF89?adR&1d3xW+HB>-rjO8}B zVl>${GkCEl#`qW!q}0aqEY#Zfw)IUN$WR59!_;;EMiUJO9VS@M6A-*lJT}9eblE_y zmu{r&RU10gmYg)u=B~i_66jv+&hm;M^&ne3sX(l-`L0yXBy;J!Z&l4oxk8w*zi2MP zMxV#Hv8v+fK%YR;cKUe?0ly&#CeL=sKZa1BXesV2zO?KR|-?Ai_-faZZkD1 zTt50oIrrX8=bNXIIm0_tlw6R0hr|FSXOeM4GJ;4&XSwG^d5o&S*`>-v=4}(j)he_! zCT~}@l;xZUr2&sJ^kAnI5ngEV$~{e-zA1Yaj0Djx9|-RV+?;2ts|`zIb8@+uo|AM3 z)+80^d{?w*?Y(jivp`AShm~X_zPK^K+ zW}=4rx^&@iL@4XuMCU0<_R7!|M@txY*dYwN!4$r(vcC7}noXn(MdPfJT< zyg#?Do*Zi_r%V@VD;rL@M-HmGd^eW9z_8Ulb0x{Rn#ILVjIl!?F+t0E&TMpqug1nF zHu~>^%S3pleVOG6naiFqDva@{t}aIn<Fl5F=Z9c&UejC?*2t3z@X>oWOY}#K%!!DV`)~iKc-iZ z;Z~y;0hOn?*P>HX7B&BlWkG+9{dS-Id=w#@J~W!8Jd;nrX3xB8MhRWLF@#V~f3M%h z{;oB^q9YKX(#I;7$cHPzS8|$SV3dwSrpw(;a*ACl#PVMA8TI$fU2Dn0ke+5sMXRF} z4XcG?^U8MY;BM;EvuB%tI}RoQ4b{jW8=+F{j>D6`vu*Znvxt5YtXiyehv&JF>2`Nf z16Hy}^KL#|r9T|Fgs~cO(|>z?gThEVt)}bt0MEebWvY7n#t8nssE&(a10^XW>wq0C z7jbn_y5q|E_n9NaGJvvbJ;u7`Lx&whDKD};jdLgD_E@;3i6>OPyH=+=L8Wtzy%Fam z)rucwGHB6~&OOL9H zPg9j{oZ7tI^l0w7Z%X;@2b9sjlF55gY!(IKI~35cNG`KW4DxX6y-3ae_sehj+S=Pa z`fGd_FD*29adu|7a^{*2YAB{tTv2|%YDguNkaa8TP zgqn0sP{1cb5V#a&yUuR}S{}1@QjVXYbo6Zk2)K$GQKKBQTq2KMu-7TOY#-37F+M9b zX>tk{V`1dbltt%udTdaMj^w}HK$bfni>nZ z53ly7KMPK8fq4p#rs7g~ba8|rpbyC^F)na4?P~%Jwz6mG1clb$1!)@zpyhCPYVEzN z6Qc5S?gUkrRg|W2V^e@S2GRV4K%hJ~K73@{e~dW1U0^RtseQwDW4wIsX%aP9D6v+3 znTiJYjq4ME(8S?94gzk1Kv7jTDSNDV)^xlUDZ?{K&_r0xvy#8+y&C%DxZYj8WJor27>$zS29j9ILX-;*yx9xdlY`)^3G0I+AcXOznyQ_k>aM zCSP_6b^3eTr!19h3I%FHFlTL56)THWwcBJZgLN6Ch&uD=F{W`W)7Bs9IK-Ew7t&6= zz=9ToG2;u(qwKa%nrL5Do88D>p{)%MU!8f{fF-Zok-g!Uy@h@rI52^NNe;S}eBnQK z1fN|V>Xy9LA{dU#KIw8^&Ay#*IPXp;y!IbbGKohwg&gd{xBL=baSZ_G%Y=m6MYPaR z+_+o6%M8qy|Sb37ugD|U^o1rSDJ%lh3Jhttq z{jO^RF@L)_TeOzFX~zJ#tByf5ODXk6$2LdnNbSp3zdeyl1eg@B@!?$z1+CUi0#C?p z%2gs)axS}77o?&TarQ%!Id4tKn~k}&e`FYuoME4Uq_Q$o+U_)1O_1&tN~fyJuwWQ2 zYpQ0H&cVCr%2^g81VQhw8-lmGm8J?=i5h$vlJEHJOaa?2e~ogRW$^eLdt!F$g=g|_ zo|Ub5leMnRaG355IvNWsmabmq^cmPGRVsD3{?66GeiUYVZ7!XuBsP)oR;XRjRHkUF za1G;KV(yyg(pPC|8}{{K8Y?MgH_9DBJ-LQBLCKW1SJfqVCHhLH&ADt=&lA~pcoeTm zOh#D`gVIYDR?0Ue?76KiBP$h%9dT6Fnf*KKy}wSKVizht=R={|u?St9>PVQZJt^lIOBjm{i*cBVg!(E! zb9b-Fv=?}v)%Kw$WwvVOc2g+kwi6-wD<8DwBM zJg;yWX;I=rQ9Rl?!Iz09DI)l=BW$i&9&A^RLoo%N*#(bD$h$TGN40;^3SbS)8R^Mo8#f;GM!Q2Wf#TOXJ_4LHqGz)hPV$tH|(_pHfAx(a+bQGt1+O3 zZkP5zC_^Z7(MGNgDG@ZfX~!xwx{FuGWXCm_lkN&C`kGQDhM!ywZBu3OA_dqee| z(jWNd!!2fBBgXc`18(rQEc1G%< zh!+Wi1vj;Cal+nO`3{10V1~}dF|J&FwurjSneOCQeE-<#^uP8fy`bl2p{%Vd;G6Jj z^Wv;9B=s#Dzw%l@wI4Qvc`;Y2tEqYhT~Kt7!!|;g1i4qpunv(9AUwtGy6s=ZqBwv@ z37%SiE(&T3CA>}vL^0Yo$dgNEmoE5wqWLu0boN>9aj*BCZtN*2(z{mV`*4%p*k5q{ z6xB{warpA3-7fV*t8)~-+Gxu2A zWtxv{a?o|@*X#W-k5E;~@29F7Y%tN-$Vm)PquQ(GaOr%~UIE%IbqosP*v?FSXV|l4 zWx4t+go%l)&TCM-Ir}0p3vm16u55h#QtO6!^#!}rD-&r`Le>*eXqrlP;}-|Pt4YMD z=(lc*Ky+{OUcB&e!Jea;RgA0>hjf(KWvXaO@xhIP#CR$AtQQ;^< zh90i(aS5!B>1vPGd=RV&FNA4iWiCJR!zarbsVoOfUFzAKNsoK6F6RYSniqlOm*8nG z)0ySPH21kTg%?IK!aOJ(Z4A7iaq004nVO7h(d>KvRXgkb_>?X~_);dk|-ol;g zPR-Vp8!jg6r|VUS2HAAsRk$@37p!y{7kU#AZ)ft_o?j@ZaP{C@uh-x1`H-=U+K?KT zTgsw!F{!J^X?m4i-+3^?f=Jvw6y6Wh zf-cF5YM9}3LdsiWjxnVHMTT8wR*_kXPRiPFk?mUIkxm?&&A2K+ZNC1NsA zdfCL&KmCsG=wswY{H>sW|6`$`dIcgI`j0thefj!bd~&#~?JSo3?GF2Qor{_tvP z>EXH?ai_rG@@gz$o!#!!TCaE6D8QCLF9Qt6Iw{M`;X^5yUt$BcbM4JfqoAWtciAmk zs+yS;g|AlF)=xxF#K_V{y54M^Vzp{};TO9zmsVwb&?W!E5P(sQlT?HnVMJ>mr#yWJTd=sg=zoi4K)FFXT{ z+(}pul@20m%_z_)DumHp3at|aHEwsuE3A*+U{hA@8E`Q(u7BP+4z)Fn<>Zce5l)kA zTUODE;3y3=Gk3kGm|M`gqfyH1I%@<4TZ9dxt811j+h7gyqCN$;*T0N z)OCR?`cU9Il~$Q=11hwheK?%H;h=}e(U!GT?}{>;H&xle_!2as-%^jeeD*XgVU0&M zDI6E(&vQPCb>wYx_(nx%$|)x5uVSyZf3!wof<)oS(6{WVFvC)v7kPnM|C#Knn$)2r z7I_0_>+^)AT^G`+L8X&m{)`I@TRqu#Wf)hCFfUx?x*c$R$^x&;co8fH@3JKHhWee) z)yXOwoxF(HqBsagh3wQ+{GDH?P{^IBtKYZN!lvE6n`$Yp<`gSt7u?o$U>Oxdkj*CwteF2 zxW3cAbn(tLV?pwB1sQv7j0=GrVkUEcm-n^egU#U-8DtLEQlPs$Y1DEw6jZgLXA{ou zN^JFCEBfao&PA+*?%WmYPLw7n)BSFg`k4lp+;KrDoqhZfrg3bQ%w9jncDUhM0#) z44O>lo)}P4zQJvhZm6EUv7&LUqBlLVqhh1|`Y2q0JBG^TR>74r1D>2BN6~jXo}~na zSq|5k)>(;OYH)%Up2fP;L=E(-1qP5iT>0Awew$jYTipi%L?74U^xUkQZp?@uVg#a& zRoYz6wOz|Fh_P2Z%m?)?p|$$BTmNXK@zC{LUtB=QMS=?&%14V2f%@N3$`lpEAzPEN zoB^EY2EX|Hg43W1_Hb4yqv4Gg8yW9t42QS-N}+31t2$0Sv1$?Aff6!JPnKt}kNVQfbVr;p}JFh+adOm*MhiB{FsG<8_ zxceH6I#e6}4b%kJ#STibEGzmeRZX#m8^nTMWttT#Hn|n$7(rm`QGH_4aY!^D4wz+ z$7?T1fa1$7W(7@0;=|0HQ+O(3_o^YRbGP!PZ%CFWGC=ys9SJH*x1nFRegC4|woX@y z+4>b$-J3LD{`wt0Ss$V54ENrq7C3t+SkQ%(ieH^-cj>`g+Z%^U9p4O^S|$rd+W7cD zWex&N_LY$!j-7(i@PYin_bu6uK%C{R8^Z$Ux-+@kJ@BTg)*gmWE~8;;c%}fjN3c+w zLRf6Ign~(GrlymVlB#zQp3`AUg@cMdhRO4qEll#TGb}r8Ak$l|nNgx-scVk+;o=v9 zNt#iVpXkD)xe*Jq=fC{tceoJI8ba@L7`;e_(BU#{Dd#nhQWCgx9Y{vIEy~bJeZqtnRT{6Gp@>}_sR;PW{@f%=I@mAQtkr21Se8$)xBSmoCx16|W65^IRAk`n|b>Rr)G<=VC-&rxcy zEUC|P-dy=$DTHg^IS_=9B9&{L4Wq540z}+hUR*eyRVv!ZK|@!=)~$)+&cK%ceM19SR8AD z<8H7!4LQDXuiRrddpcUFa3X~7yRDj5q7)~>7QMyN#O2)ApjUhba@jO`$(Lz8CLe|L zN6`+KIhwiYfMSVw29Q|pC>3vliO==+i;k50N3)IUf|xaT6FB>!h4S}B?gIG@hSf8& zUtj>xkA`9oLy`C#>;a5HhG{JBY@bT4wB(sD%sX%!2iW<{J@!S&Y9peH()I#b(wkOH z+?O(5H%AP?i;u;@63~(I3nOJo6vhnWoJPU(tQgAF8nNnzYywZlxP=WmaO~Rg`h8OBRfS(lq->IF3Sy zDUiL>jR``h+g}ORLNw=-lzoVM@Si|KOjFCrdNsUXthHnm!_(uUn~hN*Q4&zMj!3F) zo3kk|K*fwcb)8Xt39cABQ%X;x9ou&ey4Kzo{_2F@=xx~W11x3|;$+3EMS9sF+pDR^ z&LZtmS>K6Yns%{4$%>D-b@08As)tp{$~ese@83+g}Kk~ zh{+GlO-u+zVgh?%21Ejl)QP+O*BEJ{f(zK8qW$pFaHD+Umdq^a55DotQll7;+b@Tv zPLS3&gb+m!DW!+-pp2tq*Wu1s5z)HuugkAIXlz~qB0 zWCVJaBn&0<2?H%G^J;t?3noMM+Mmzsrf_QTI`)WZUsLCW+CRZC*byYd`)|wX{}8l=vPy&GaC2q^75>5&EDe90B1-EJ&p8D=jZ{ z9B*J^%Furc6r6ZiXxwHTlcyOR>o3vT=CO>-kh?`@hwnpJ9n40EK;$*}xCq8|oXPnH6AZ9GHmvCT87@qwfwQk!yh< z0cBzS-DT^LyE0@9-YgW+?leTlLd#ncozOgR5NJNb8c^{#*S_3mEv0ltzWscwK%VgcJjhWDRf$CwGWcj=snw zDkvTlyNv;YYDiPd%7H56sIOv0EiXW&xVv3292$59824Nz7Q_K_$X-;^Zcx0n*Zlb~ zu_-j4MXlco_^{eQS3SQWA2rQ$8S)^S%fbB8@k-7-1kxU`AUglc+?pXH12>-w3{QlH zcIr2&cTqMJ3&Ys*eFs5{JEP(9ipFY=zz!@NN%ur~IN>3`@DtO2@F(|2mB~$Y%2!Dh zX%Wu5#=O^&20JqdXv5JBbUzz!qpmFxn@dvO;xrC*ExGxCzFbQ5Z)z5G0p4sIs7^8} zSqqcBpIl^%5<&OQOp8SuGSpZonv9u`F76l`zuwT`OYL{Z&-N+4(rN-vzU7uLST*)zchWrF(|81*Stlaq+;q; z)4!{VUc0Ttv*lDL8n4?*dyVg0G^hD{tmVW;x=|saB5l*HX)^OtJ>9&T5RXa=;=9LYU}RYzh>s>{|sHGhxtRizitys}+}`mb!YwG;0$ zvLtcOOHsd++I2XuqZ-vY?`ZA)++A~yCySjWGi)B&&e9(!`WVg6yg~fV!X$^ ztWfENNv_${z3aMS1*9RWrwT04csraQGWF2T6db`EN&U!V5`Xxf+BADLS z)Ks}IB57LJacAUmpexvA73TlWd)5%)AzZ3uGlzIyx@(zkXu!Z+>)k5%z1_nvAiUWh z1xll?-ne9ViCk4qOS3+Zq`O zn|oE%0z#Jm7G&M7*N?so0x9+sRBh<^>|J#M4z{YtFTmPWG{cqyv+x5KX{oyt>UY#j zWz$Vf%jUY(ISR$^i?ATEfFghAx1@NPWAI}DU(}vJntSP!w+Hxg9Ctl* zUXKhawy@)97f6rTzsb*6D;0#trc?6U>T-9$?kj6W?syIm<4@WjNNg z6|$^XhhOKpX!>py+92#RTQB&8J$>=9r(2Q?;FDDD-8Hut1Qh7cB8X~Rqa6$Xwmet9IJ&{sX ztMui6-fVSS6UCe!bv=V=E9^JhTM=dX>TJ$jolH4Jr~PeZ-k`BWpn77jpKK6HSm7Nv z7hJDwO}ry^1N|AY1ff0S|sbK@P2!NGwKfo(C0Yhnx4v z{R`G7nI19dUg^c{;WFP*QDh7!8@#MF(T67S1|LTlUtU5j_z`Q?&70yZK^$|05s|Z^ zGBO}>ia~TUAf*agUVF=~dwj$vdyB3%bxVQU7-$nrX${aIo7`1m!)byA(P;Y3cC$;h zfJGyKcEPpMjdIv*4_g?A=iU2dgOp zp1!!;R=Yj>u~FuttSb@1^)g?uriIEi;r1S<&6}GS9s?C@I%H49vn1el@!wcukqQj8 z*6fXG>=9d(dcMjh;vnxHNG^?)BXD;R!$Ok1mT{ zpC!Cify&HV+|QSPAM#68ACGR=2M1!&QYj z#EnZt!lyCac68HSFSi#kOUXT+7i6-)W>8G+avE@)RBvWAS|`jZTnxUUdpm<^=RHG8 z>~7neGF%_Ui|HKQD$nv#ljdk@kfuRatp9Enp5-m&XdQ*VlD|eaiYK^)6UOC;XIem>9#$#>cdA)8 zB;^CAbA6%q1!GWnzF_h0WD5gPlV*hrcwyQ{N)n_#6y9Jz9TKKN$Rv#C|3SjJ*##25|-!aP#W?%d47j6rUVRfx@NbcrMC5uaMB0u zQP$viypjMz47TGgC$OdqQMzv)=O;BTmN=FbMIr>t!3~_9CPrl=ff*+BW-&LewVJ>A zN7X4R^%jhylw^IOqM~$b+OWIn`b!05RYmh$!Gq#0u{z0L3C0gZ5m$`Dpr{5{c~__f zoaCrTVrD9FMN4dN4*u;+F!LNj-=79 znd=5F+qan>_qQW%4MNn1F_*Kz>nU9YfztpNFsXTkX|$yU9$Ln3l-k3r(LqU4O2q0h zGGf{EF+Lk60$Iy7?NBC$1p~wBe32|xz6KVHYc0f1e=k~Xr5YGIveoQt0&TGs?-e9> z3j-4G)qE&rSuj=S-6`?(zuHjw@7FRS^&^uyp&9!A){R9L>!%ZD_fqVUI87hyl%wBeRt$G~%tIkz}Eo+f$JmfQnH z_^du8iYSIJ_!-S#OR+i9e@u{|+Fcw%g{i+RJYuuqw^lQb3QfO3edA%kBjHEzB}r8_ zffXI?)dc(8T$6>>%Y-euVo{PWjm3&L2X4&w0PEHqeIEt3px5_%5I)yzj#B$hNs&jW z&8-=a3Xv!~4Sd%;#&?|(NL)y=(cDqb8yg`r<^@iz5-)^38tc2En`r2;LI)&?gT3Bg z1!_PGz@Ett*#Cda$IYwzm5M5k-uKWcBj0tt=KfSz4^nS0y*m_os&( z-DV@bEQI=808RJH$0zM80@nnA-tS)f%@qbkTdLO8Q2fZqNKtBME*7!W;%!6VJGa2p zHgo_L{AU3O0lyZ(K^>uz-86cc%dm(=4GJz60xPP#QU+3ZhpokbI&(46b701pA)d67 z=X!K}9ljns0*6(@l(ZY4n9^JjG8y9#iYbBq%n0|1V7b=R}{V z5~)BZ@E+bOgz{48{c3!c*ZmK7bzYEwrsIw>CQs>Hw2i{c#L=azQen7_x(y~t}k z@F0UOekqoXxnSZF#&l`Oi z8q|tMS%;IZEkf>kJ1ab<+@$t-Si1$uOQ77L42aCcT;1{Vh|gS=>5t}K$Cb<^LurEtA>htBBgSNiK$m5qjZd_ zfT()dP(_Vg-lk8*J1>zs)vIVPi=@RqPa|t%ag<~^6xs0EPW&RnMf&XT%?2D)3aLbH zU%ZQmqpcg=TswG*&CCJ`h%QR;#B+}V6-(Kyy=)zH?+g{I<%VTj=V_oNM8p)0fTyG5 z&d>WjejMj;o-SM{uJ5dQvll4X$iJuyYV$R6HmAmE6{QB7pQ)l=?t?B<(~PzREQHcX zrCyNML`J1T8D8_t!8J5O$tl1lqS$n{I5G@~kIHHBLx9hRq4c9)vY0zjWBO}uU>^z( z#fKt{UDj?y&H9=P3~htx#%mUG79O7SM9dMTU@7RT&2yg{#Pfs1P+#{G84i9thL0^V@R^@W}U$~?F1vX#E{-HI9y8*< zc@V(1Y*KKa)FEQPC~d4aV;n(jdD z^OU=G>+)`M#1)4cB;5l=;1t}5^hogJkC{G%x9_B>*MO@Uae}Jt>0H>zZ&aA$vQGh# z^Jk`jL;re%ra6m+-S+AStyTLR|LZ|y9OXlZ{riVsevceT5dW=Q-sU8RD`}`{F{fIQ z0B2!Fqu#exzq?N7v6Yxkbx+6Oi&+0P$^PJ#LxAG(6s|YT9oTYg zKFAz#Wu!zbX(N^GjeoarQ`pS1X@40S63@~c?1LVJIa;lagYX!YAU`cfVyS0vZF_K5 z%ylyDKq>a;{N~|B-u=PK7kQc6hh#i=BaY1w*)CT+npJ;{`+&9sWd9@}AZbByqb}p~ zJ}?}=5ksFy1XS{P5fkliCLccnz&&4Nr&%MW{i8BL;^{0_!UA70&pcyMmOD(Kr8yC1Eo zDL>aXdMdjB>^mSa+TAPkV8KIs&Qv&V{e1vD-#-qaO#FV88Sd3(wHyz|zhh2i_3%4QkuYr2G8+X{e%I0xDl%H)K zfcxfLKN=Av11NG9|$}b}NFxV@XkSbfcJ~;CSg8-XD zz&#S%J^V>m=YK5;WS9xKO~7HXqWvfBi~l)@%V~w7E|$3y`|Z((dHu#z{OC{BszB4q z(tvBq5nSyL3EDXi#59^+IMVy3`_~`b3jm`rT(+n9drc9a5VIu??7b>7$&exD?_v$x z9I`cB=6oQKDO`vQ*8ushO9NFMrPE5y%i;U1d2(94c;^(d0MU8X+8kxVAHL(n0IXrbgRMURBr;m2 zAzW{Ue3JfDRwY6h&SL1#k6Z1ROlhycGymvvnZ~9u)OQ#w!xG!q}d+-M!HT?*|s<*n$VTd*gE>`7@wj{p-2PV32*|48@OS z2VAkW(d_B5slI%?_$8g@#!g0(q0{?g(Hi2d(t%^oR9W`sI_YO}&Qow=RXx7rt z?Y-_XBHO}{s?JXJo(!n3n0)(hG?t1iqRoI}MNfQ!V;l>b*^a&~E~IqRvlU^r1s3X% z&)@!gTOXkd7BcHRp3{oyQRnHV-pSQB*xMVAz}GULpKm54DP~w5Oe>=(Mxox`G^4_~ z$4S8*9A<#b`R%W#w(W_)FLwd#|L4CRu*Htk@tmrbEztW&5UUz$PpeL$O|`s7y;a(N zd$krimfoOo&SC-c5w!^WA`st;8FHt&eOmf1si6T=e|Ju39)JJpL3ePa7Qx%{PW?gi zLowtb7XC#o9*HzE(a>V1^7*VzW>6)CDpl8X8TSF}CvxsD7CdxZe|NNu6hs}^-SmZc zfj*4@Y~36z5UpZ56hv3{jn+AQhqvph!srK+{ESL$HYD>VVoV;Nn0JsF+JuJ&sEqz=STc7)4^_C`Ne!XFX&q6jcs5%^^L zV*j0oDSE(cr956g+EVTF`_+R;12RDp)phKT2Kjo$4n+IaR6ml%=XEebo?DxEbK=Xg z{K{|lXDkLXXLIx;`@6~im~;PkhaVKI0sL{&asAG3_k*`)# zvyLC8ZhwX&K0wlu8G0|E$oB{Qw5b8yGA0K~hA(on%0l%tvWfPXnxOq{_Dny?=F#;7%*;gmqf~UXabx9u!D0>UKezUZAw_ixaH@h$}$wEpuaD8qY! ztZ77KO8-eL{fFhDPtaAsN16cbkLlW@vHve2ep8bCbvS*TH;xvG_bU3!-&d*c%=#UW zC`bd^dzs>^km)%D~XU9L;Cflf}1SlYEI9b6D!kmFKx`&;-;=gS_kgpoK~Ozkz3$Ndy8cT(KS^GZuM}DE^2mqtnep zN?0jzzr7>?*CJ2upMagG===_Zk0wo|a;uVrSX3miugMSV=omQttM`U!RGgYzzxnLq_pHD@uSdtHg;Xb?u{xUH4_(zEu<^03}H2U=->`GM^V zbq58tdt2?@1?TA|PAwnY=3e;>Gpd=7%Y<*{fyMAjj+l?-^-dkDBTm=G1aM=fz5H>L zUrCKJX|}z_>ERr;4i}5XnT&QCDQ(S>Q?q@a>Ms;b@OJmYvbUdusYB~Z=OaW6h;?qj z)zx9BQf&CiQ&`o2o~vYw<{`MApvDu!2CRg|$Q#RHpNn)Jfa;uWRB=)Xf1GyZn)#_C zlE_#L09$+z1qg5Nv(^4YFU zT?gV>e+Bm+wlf@s!ge6APHnvLsn&YnJC0;JzpxW00YvzMqGu00#V`H*)62K$0G#s4 zJUK86-(SQplYVas2qhW^4;=mOL9fXdJMgn#ee;e&G=Rqcf#k5n{s)r7(m<-!|4$$Z zYIi&HtsyapsJzy4YF%($tE+5pcc~KMQFVdc->s~6Hz2X16dg|~Tm+jf^>KZb6h6$Y zH7ts}QP~doHo~-__W3RTS|Zw6v!mJWtH7Rm^u=7T?bn-X$Z<>>sWS^*_VQiuwPgBrklI0P9?2cmLR=Di&RL7IN zcNdPD9C5z_u*o|2fb(mc1ff2?0ek50jw|G-zOnpnR4n^^{W;bc6y=A@Aga9ti+Q$1 z>=au6MXce{&RtQ>^`dW=>9BV?46%pESp1UV>bKJozm4g8QDpg+yLP2U`kXOGqtVJu zhQZFxAo3>H9T`}iO!iT1J}SCTjEM1Cs0s9&w?*1fqymw$YX~AsaVPYzL_y1@g{5-k zkp6Da_l1EiJY;k;+0i{JQ+!;TX|3%FF17PGp~wbn0d zyhfb~+S>v5jT51uo?wboJjrAx^fh)5k8*f|86qI1Sdh8#HJMs((I|PAngcSLSJ3g^ zqL!y9?wtD~?w>o+W>&yoWAG;&Y@3=>~Q~&U)~t z-LZ|Xw|N%Z@T()b?}2|SdB`CTY&BNlx2_^XR@OHtnibRB>6cXoI(LU!sZL#gsQcuJ z@zJ8$uL4aGc4UrSm+J08Z{&V}sufzfx0O;6`JpHlcd}yl`A**iOlk|k}?z_^BW8vK=^NwA_46=4qR&yR5I5IDpA6xa(!7G-3AV+jO5 z79(R|q>}e5M=+w&6^U^N-yMl%^tST=&qRvfoSmthLXXOsd-q;BHRKG5?(sannn)Jg zuch(>{lg&yY81`BAw*wGW9s%$>hokbd=(g+ua5@>mFUX3l=rEMhtAr4rMBM+wV4LF zG4GKr`uZ)15Gt(>1F||Lv|s}?faTSHkwh{@{b}J$$5k>gEtKCwPa&Q90JW z1o#LDA6j$gN=Q6vidusbPNZ>574mBgDhdV ziIL9AH)Er2;lGU`TF=rQq&S?h-9;(#J_cz)q09BdiS#Qzr&hQu*uA zPXwzpz>9=oLFzXbnqW(wGfpj@dfe&G&EzUuxL9(;$cSC`7Ii=*{hXhq zY5MxgLe73!{~zLWAE^I&AW;DfomdGc(VvJl0$AUyA`k=n*RcE7WIsbMYc8Nd9(Klz zetr9UaH8l(g3JF$@V?uZ8Dd!njr27pH1la|H$ z2V!kj1J<{pU-jvqAU+-d8c(jO?w>RQ#1)|gHOR_-UKtS<6mx+OjT-J}5A_sRKE_bP zA9--cxq)s0Q?q1Q5DPbVCP8tj{H9y z$Yuq6``dEAj=g=NfrbTdkHur~3;-eR!mFr10bu~vwNsm0p9R7pJ=EtL zTb^_&9v93NWk2I7enGTw=Gn+>`qxF#BNBZTS!=vB7s!eHwTLdNjt>>H^u$611~o!` zt$5~H;#BtUY!#0|p5PhyzM*G2T1Dpv#2N$x(z|M!toJ95A%6hMxa)v+kmW@O{@$&dpR9iZH4qDMz-#G`9)$dBfkYCJ zxd0>ni-(NIk;HI#q^n{8_zdQPZvPf^KY;>r6K3KzrXsmvZna zpC9Y=k6frPP)6^1F&~V5n8ja2E(s0*w1g~q#Xl+ikzNF1f$P1O&;9}8SxJ%Mv~Fzb zpC~YgG+>4k@^AkA2Z-0=K&6$?efmWpZ5Nk_U0tuJ$3qX@bD(t{Z9!#euCsLSj+n#V zdLY`#Gy7)z-w#?tDCT^VgqA}=5eJv5h)5lmw_wE32kO1QPi&?IuzFbB1fP$*{|G2Q zpWqU*fJ_rU{PVYdK6Ygl;JcXn6j1+w8IiceO1;`;_wTh|v!b9(mcfayYK4Rz?S9-! z9xT8LPxglaUq<*%& zq{uu9HQ8mHZ&~^Uf_}7Z{Ut}k@eiP9Vq7;!zY*;xI^Oaa{Mfy(5S_c)Oui;kS2eOQ9y)%IYjScKiG49Mo4HSbG z9Akge>-iuf0Zq1-*m@peASML=SIR0fxc)bS8kX%^muaxR^JEd&i|2nGG@w^Bb zo--t7K$@`+p=*n31%bY5H*pz!xTBIYRGw{wj03V;vY{?CwH9804 z^-9o^=5Qu3b@=!Mk1U+|j&%A|@Y@}nUdMCjOw~*`g#5`EMEtfBKy#%v@ON~GK9wnm z`0Kq?4Yy%WI|FIFPQtJEfL%v!pY#&vxTBUnwbE(MF?uWh47Au(HC;7F>&kqYlEG&f`xw{`&;*B#b2OgzA4s%BbUgHa> zHf*%Hi=SOLnpT&ssr5-No#W5eGW=pWyVqq()v+<6-St9z9ry>>mIvg|3qiyx8mTVw*P{UDd#fAY5P2`P z6BhUkV81J7*8^wwIOvf$;TgJZ3P35|e}O-A_4@itgY-?KSvy)a_V!cqa`(;gvnwK& zJr#ASJ%r(#Jritqfi$t8PIXl?jplW@QtI~GqNn06P_Hgi(S*q6i*!U^neRxIz3)UK zq#xoHJ<`OzMN;rc^X$@gmLOh{IAvaB?WzbrwZnxwapIJ~zUKSsUjX~M_to^A+>ZFn zk-vRo*lFsHV!8^|j`z`Q1_|u*nwQA|d**QpP_-Wj@xsgp3Qf0ad@4w|ql*YTqFz?2 z69EabqR0u}W`9uL{7W5qU|jWTPGqBLW|Lfc=$EUZfWv@H?{qWV#f;v4MjQyk^gI9^ zznTK1yy;RkXJuAs?!9X7Qq%xd%APW||ym(lQ#*Sc? zoF+|N)lOsamCL)%tD1GE-rkIl(`+_U&A|!VN5VC&Oj8Fg6o~3aXsyVL9E5?bGiGkx z=fc|txZrFPCLG||HmmfhjU7c&c^@B{u8}7gzRgVx;B{ED8RoR_5HLaj;FVL|8L0Sz z_7Qa5wkS%YtQO&}Q8tZh0@?sPiAX$~FXGNjw0pZzmUaXTK&czz$9N}DJVLWsEq(_b zo|KnyP<}vxcz*d?gE!OG#kMD(GoF3K;lBuJJF&$6ai4s3-GGvAbk!% zfQBwpkGMU_I)s-4sJj-#GA`dYk1$QQ%vAlLlvCockc>;%wqL z4DiOxl@E@l9a7{EW()L5GGT%}K)a1A)VNxuSM?M@e?W%_%KJgb-**n(_P=oFsdy$} zkJ5KZzYEU^QF5MB#`(KX>ov`0;&R@BXJLegn+a+Nkh&s!JSv@82oK;D+y+kY66c&W zE*BSD?`{=o4hvxfSbb-gM{`_r$Y|&ft@DY3*_FQm3q|5znkQzz3aHyUwGj)ek}*9t z5!>18GB44FBqrblpc>8h{PL$Jknhz@86vXH=atnWUGbXhF}Zr+0hX03m`woNTn<`3FbW|(|%swZlJK6z4| z7@9>slXnDsme29gsY_f8(Cq8#j_612T*skVWINajAc(y{L~l5*<2AcujTZ*Zt~T&rhmf zIJ91HYdQHly!!%C>VHBDEyqJy+qW`r$MrbNMmSuM+0-QbywPs>38FkD3L+w#;Rt$? zh+<#@`ZM2-?>@8Oplv}+84jqmko1q)XS0bup#SpO3^ZwJC0|+mQn8lHZ@)<@Ovwp!p)8s`UU> z{r$ij4*_)7c7GABkyk5mT1=mtuGx7H;U}Ass~0{yU<f?+%rHOcT*9^?=V^3?5~dF)lmrUNx}7eS!JOg>!KT z-g17TmHWkh(ja+6B5WgZF+l7@R+*4+n43X;fJGpQ;d4V7f`yLr+cfeD3b_pmfRlb$ zMo2r)4v8EzfL;I1K33w&xR4ky9;yk{9Cu)PkJ{ou`C?}MWutfd+?#(g z!9Uq>Kg9fL;(fuBQVdVzcr68;w!PW)%xb&Xi7hM_MtWuY&t^sn>-T!ja}2Ncn>S{3m%qmT51EEZHRZYmd)xJ#3`?gNK$~6WKf7pBuquh{PbKaS;*S8Tn;#HsN!%>9Hh-rfWh_Voa! z(+2BvopN0kJ#Ov*te^1PZ&vb*y}Ss?v#d_eLB=>4!tD4i$k={Z##|^Jed$lDMj7yK zM!&DLC`s$BC(q*2?TE9F{Z>UCZW)m%Tx{RrC1#{`$C>25rFLGC>H^=o^A7Ar>B+uAXr zmYGqQPI5J6wqthb_~PHfqgBG%!KH5;YUZjXyX_ zHzlErnB|(iKDCyiQxYp5T$IF3B=c+SDE@=bDjxy5`~;z^da?$eg8|}6E$Lcr>;<&N zsS~<+Ak_5Itcjc83{K>NFy$idU{v36HA@q(<(1qoE45M^DaU4!WO@;P0)_v$+S6H_;9@LUKXWjhjO`0_IMUz$gc4kUyFZ<9co=G_%Kf<(DW2q z)IH^)5JlIWQRW5*PshSA0<)2dNJ$LhngYAneERdkr2aGgJc}7-!q;x{Y|OTil>K&V zVDa0x_3l4azJFc=@qEq1hqWY~L#B#;INN>Rz)4Y2Y|z5IU|z`496T|SbF_t&aZ8~f zD_yk|(QM^W|7Ec5tgHJpf)=A#JT%5EU$+UL4W{a*bJe4SwMIk=p7*!Yc#w8s>r2ZA zY%6hnQ&CTidt~sN!2NdNC)({lKYBa;&QbfR;{9!8K^zsRu&}@3WzfL-W-Ps-qLO^E z|Kd6_yWhP*$7=D(^b)R)BAg!`u~IVelBn0jBRHEuNg$hD_~G6udOROj?_Hez`{Q^E|sH} zJn#7~4!Be&WQo9IEp38nhUk-%FBSU>ts$E*SSJwDW%%{@pved8 z*1_AMQ3Z>#al|f0U$%iGG>1QH7qx!`*aGJnB$GxD2yCDiT32R3it6k6z0*i+uH)j= zxa3Q1bs9#Q@^U*=7mPAMU_!rRV>U>o?02Q(5#Y|)Y@X(~9ecKssK}*#H^dCNRtH;> zN9d?sOZG975psK@6|KtZqF2BT&nS@W&a0>4Yc9!bB~MRU@wqni@XO*mlXoe60kxu( z(4yZ>*#ROl&X31w%$-OcSQK3ANa6Dk1Pru2Z9&_-P;w?#>C{VUpRt+?(K&{ub8p{p z+hq>g!n;b^nXiam+d#jL>qwC-cUkOXuRrUW0o&lVDkmh^cjqoL+}3whHE4V3SEUjF}y;zNif2=P9#rSV{lPUNDR)O7W>LwRZ4B%z7o3 z{cf_={WKuP!Rv+!bi3=U<7bmC7r8OSTb8#x`&4I%(?P)fG04UfbjISm$#$yJjgF5t zQzJ&fYxnH+{L*hrb%Tp|xxYyIU2sbdQ{M`==}Qqd+IP(V=l*{#0Cie`Cse&5rcjgN z|JX!vgzxC*pnL`dm)?d$s}7vKzzqY_f2J1$_CQQe3>4h95OqKlDaGmED!#^l>GLqG z&}Dwm;7ve>=qnrU(1X$CJ6zw=-$dgyr<>aCWAxmSvna(RXtvPKC5{wZm z+*sjc)5uy4?%j^b5QAh;W*?~^7Fbw+Oq`yWcah?ep?}ok=715ggnj8XOVrnsA7$`B z=Yuem+07qgftLtq#?uhg<70+>tg28;JL2vec$!9P@?FIX;4$hcj+x3jud6>vRxWe>~ z4boXfDs-O^aN5X-vP3?W6x~>LTtK{|rT_s=ct+;2ab;j=Sjl@*e2tfopL$8lNfb1YmU++K%&efl-pCyvrMZ#NNvWQvQ(MaQprK5>=CbMS z)R{iseATSZv18Aq?%r^@bYTO65c>)Z8W6&|47<3c`18QUD(qTdZmlB6v$kyV^*&?` z^<{uzM~7ZeD{*F0C;XPf%0+@}z5RfCTD`jA6>DHal&sw~`Za!zR6SGYJH{!ZR1|!w zDWCNBHSV@-!8+cwP;nmMJOmU{` z@@zGuy+6f8nQgh;nC|1^Ug>|W6cNEX)i%9l5k61qj#!4xwrp>pCw!iYv0`}Nqg2?K z{&~4)PsyJ@p2GQS0y0Q#JmOff#QeZ}i7-TRHhjr%AmMIf)+9QIW^Oo7cLSGy}WY2K?Yu{D=aMHYAX z`$eq^emS|(gu6p25_E?Sri%A2mVsPeJRq014^34~vvg)=(x7*Hae_W40?Bu!&bj8X zG@F*jwfebE+Smv!j6_*s$XK%L;(Fgr_qe%Unmgu+w9bO~wP>_F;LsHQ227fneQUpr5 zM_A473;gbI%6a<5-$ph1t#yR?>g&HlzkCyZ!?HrC1eT4asu%Dj1Fh%9m#(&5KZgph zk2@_PyxKN>6cQIfwC>AHdLjeWE802vWiNpIE3vD+?^Lz{qZH_h+1)oED?N*YYz9|X z)RHNPp=Xzcw^WjB!5-w}_tsTI^wk~I!E7;3JvpBAiOYGctWgL$^xMu>#JhU}+Xv*d z@8>oe6=>Ft+uB3^L`@5=qI#Nl?$n>D&QVKK2Um5fSz){DZd!1fPBy%f4x^@bxqEE3 zggeBNM8mNI6Jl8ke_Ak*3*J~NTl5zN!^|(0&O!cNp9;58MS>%%Qdm;O*w)XN0f@vZ)`xoH7i zR1T`6OQ0{(!#pnXQpfM6tU(V^UoPyks#Ig%4jO*u%wV*se^N@gKH zuK8IO#VTHXwO$tT*l1zt=E~MqO0dwqfz6zV>>#h^r>(Bb!G&`ltymO8sAIMi19@`Y49Xca1%U=6}F;hgoKjXbQN|Pxoy=EapWl4ci zWon5wY*H#fx^?gz!k$MAr6R3M{=14p@*)?nO6u%K{nfF;akb0{box2@&Nd3ZA?s}^ zFk+#1_}22;q}GsFxnXtY6oDHn@i@rse2-!-kI+Z2Sk=zyXxe{; zO~bca*qWb;$g&JjBIup_9~?jH!9|`US1+-3FP$XHpf@Rf%`7?4-Snm$+wa2ci5t!G zImVY!RJ3L}_EL!hJNkv9$7-C{5n{SjnTIG370twKQ?38Yj)_yX8IR zDxqs#shK^iPEzjg7da!s`R6_rPVm7oR-e-YOcPrVb8EP+fBkB}cb#rnf8<_&frp3N zqBrC@V|05HnaiQ6auh70C@n%Xy{<1sb#m>uy}dj56#OR|6dW$MvSya9GP9~?iWz-W zFXZ5~)$aN9!R7VXX=17xp1N*@p%7|k58>v-L7vG4J(t1REd4 z8c#JLxAO|XKwiyJU|m(xrD;0|akL}mC?|=AGng+&-ozQBt5qKApAv)m=s&pDz`4-7 z>D8zUdB8W#^;iEeE(qRnYNuvBAHyZ@O3E;+va3%b^gd(1Z0eP8dnACSu;!T2cM+&M zC)kjC{1?%fXHk)Tj&kJ)Gum^kNwC{b-Qz6z%O#&rvXQ3_)O2`{Z!w5h7peGhzK=gl=Ot)@+~_nmQQXs$Vt8UE^II6c&` zMUN*7o^vMW6pb#_uvIT-rJB{q0N<6ft!c~bQvL=jkTNn#Wju+_Yd0#SOb-x;?JkC} zBrrS=B<{BiTQ9*K`iGVS(8pdmotU0GVtsE*_p&8jrF$?gLxhmbSOS=TL9Sj+C71`A zS6hmcsR*Y&?UB@N)`-1*mIY97$&Ti(7H?GyT}2=cIby~*)cCFiH^tKFv^pr)jxx0M|jOu)eIPZ5%~n55w-DDsbFdukV|&z zEfIg}bs`u(Qmay95w`u}EHqM_$0)C_UhHW1k0`A)0cpAwf2plWBYoY*(CpOi2R!qC`(QB6qwy^3)RzI^JCO|J$Z7*3VO5keuB+*nP%@v8f}2{ z?&qv{vNGA>Z8Pt39#-`V6P_pg&OeSH?mbkD$Nt56!?Ff{-v2-lDFSJ(Bh3$3AX*Cp z#5Oac2cwx}PDoNN&N`j>O4O6nfB+vU3E-hy-W-yqK!xuuUAaXYTNZ705!mj_*;Tt* zf1M+|!aGbl!wHO>+0LVfM>c23PCj*E9EJ??1uL{DIU9LcjfTNfiH-W?uVhBZ$I0@T zjMdSWV+v;%qlU@=U;eN~K<~$;-Xtl{B(h^5pSqccDB~q%W&(ep&igVEx~fXK$|lJm zA@`x##Wp^?&KxxnPyf>B&YK}MWHSLco(zTZXaz4T#24T-$9CU@&@U}sX}WoTlR#4- zwNn%*++R}#SUA0)moWEA+k!@V%Xn*Ay6L=hya<4`)SSN9f+2iON@GM?*VJD$R*mn) z(by5vbQ2&x@RN18s53xgaH)|GEtteoaK|~}!s6Lrl!I=F5l6dBxinI-p9R~nJ?(%? zR2bD#oI5MwX8&yTtt4E5Wr3J`Ft*-}y>^X{n-~(jNU9ebJ7nAcI;-4CwNliftHT(7 zN||9033Zyl@;QPy?KYd7bO#z|T}8W@VW_C(SRw42X?SE%HW;uC;PclJ&E#oKpc=3tV;K=lMvwdMV=n)EDljV4M8Kt)(g z&`~H7FFm?*?wN;GI_&)`aXx%afy@?DKF6wX{5j%WTU9eA!5p0UyE3;mL^!B|3&af1 z-yt`iOzT1-=C+e6&?*qMQqP{080<%4NvfftkBrA1-U6f7C8gYU?;M#Jxp+^d=6(t# z$S9i17FuoTd>{1mo_DnC+&f|g!P(hB!8<3U1Dc-*S+T=VpTn-aLTSd{s?Wz?P}Xu2 z1X9f=@Qm0zbT=2=A@rCB*n+2SNukp55PZz`uE^$&Q?e&G6mJV1$2+Wh^PT6N}PFoTmx{_Ro4{Zs) znfeYBv|;fu>2%u_(pPbbsL+$AtUW9d|pcg{qs7<{X*F}@wIX}n%9T_LD~ zZIl9frWP~b5dn4PbjjL)uCp5Dfx)1MfDO+DtD1oXcG#qtx$PaiuZ+ZB{*m~-cygmo zSgJ=Tp}ii!ZRJ^~3GF3-_wpj24r>Wz(bh5C(TgVs+=O(iv{&NH_<}a7W-k2SmU#|H zo}(4SPacrG=K#o!sF7nTR<^mlRT!g1}hxxNaC#lV%G|iO>U@-yS+H+gMf1q)XSLjxHqw zO=Fx#(@r22z0s)K@mfZC`3k$+LE*&+XVMO1!z`HX2O|~pabbCq1OT47*P-LVxrn;CnX=9IRLjVNZs|5kq5)eEG#2S4 z(AR~QUE??g_$;)sZB=MR#h7?rOd|{9<=q zN`Bsa(l>+@ME0n(mg^P6YSS-QqAM5hTILowVo%NRLAL}`?;WiuK=dnmx*>J$m$v}w zA0~Esb$;ig>qmEy2*QIq7{+wgA*3mwk(F*SoqgyS54DjVlP|sX$`kx0a)T*8wDYUR zyRwsT35Z;LbsTWTl`6$&;&!->xP?oX-wh&FiWAwtsk9l7!j6llYy7MWGbs?0bNlqB zKaN6xmocp{i-_&@&JsXXWwBFdgYl-u#czv#U;W=+_v28FSUP2}-k*1Xw9RcsfaY$aF7(G(lAPA0Z|by2*nq;I$9E@ldMg5mU0!@g ziNlOIJ#h`qJcgR01k3$oF4Hx$yOgcdtm^=`ziTflKT!x=v0O-q6MVPBe$bC6FCSNU zzn7N&uOY?nc=TijAaCN7DYRa`YQ;7_*O$?c@GcG$zsCD=R}#eUe+2>t0et~lzj$Hz zeQChC)V{CoOP5;f-yYhJ5+N$o;@=p_H*0B}lE-mU`opL;V%v9>QQ z>;513x|^XbXf;T;`@-><)5d=^CH^d`A5V?PDGZaORhE^s>3=;|r;Dx$_uSz;phSt5 z?kO0OzNdSBl_Qb2?yFnk*il+cMQPVk9mWM@AMq*vT4%QXcL7|^c%HfdJK8LJOg-G4 zw1mWsldV|t1I7#dI@xwm33Qh^lK1JZ&3GH;mga7UnQ&^r>cgunmHtw&zq(1F>9_;* z1}m^l{yDL%u3f$rN(kfGEwVdtf5{SRXk9`4>C-)F!D>uPvq4&J=r zrqum%Uph*@fCk0L-7ynD5C_nrSTMuk12YH1G8c5~3V^lE4;IURHRzwo1~$rZnD#e{ z406x6&EqzjzVK3Xm#xHc(GHvk`s)n)4ZrqxSz=GL!(Zuy0=LtAWr$-$av?Dj;wQ!Ifost%Rvn#Qefva}gdsI?8w~gNF zBVJa|;rYdCF)ymjba|~hob1O)Ay=QsiCu?;p4E{*dL*adtoT9)>vH- zZDNHAQP=$G4$<=QniHEO3zKQF-WG-rqmk+bF4~Z$(PE^3y5x zP|IWXbC&>u4-Q-e{iOi>fpOmEDzZPwABCPI|AbA?qz>qL`QP2IVJsKjzFq*8;)J<4 zGjGF?ukmsh)fe#VZtB0H`a%DZ`!)Ew4mYo8uQ*vAFA>}5TvoFq@UTG<&`z7eqkl$a z=oaN$DuMFlQc^Fpo;EVT@ao-x9>37t)2?`YKQ3a&ZtnI2@q#lfM}}SJGC5W<4&t6B8v7SP;p~7GEqAI!{|h`na3$k!LrXuc zRFwwVjK8kFY$=R8itVft9|!0^M2F?4e+&ircLRh^n0wjU7-u!87#;p{xLq%Cb>&3J z|24>(5qEhoV5yZ^tfP;A8frmEWG_SZ7u_D1UR>G$qdzqhOaH%gDk6CT zm=9>rC7A=b+5dzTh!OybUK?d{{hy!wlr(?;^8dlc?qQt-m$*0Dr;kPM|{8G6n*24Uy-Z-edpjh_~5+ocOLy0#`)iue<)A?L(Bir@;}Y? zKPmX16#P#L@R7F}5Lyzed;S?v2{LYWNiQmMT)?6GW&3R)n_?@@0py)#Q|n_F$jW(I zh~2xLcvZ|VI-C<0PS(NAQo|&1tiUVCQijouv&LDjp=x6cW4^Jr7K$$F{&0++b#ZZF zE{~Q&fw63??QXj~zY~xTCHGWx%1wbw^`ML)vNKcIzWqF4k$`RAQUGM-SlFh{plP_k|As7ki@; zpf{+Uw`yt?0=7mn50Z1)5}_WD?OZL+B`8Ol#LoH zE>S8fS-Jb?DH>7eWMf#MmwGurCy{7rq0MHfnbfZZU#2Wa%1YalP+Sd5p(OUu%_^vj9lS(^}N<}InMp=5cFOU9%0l#hJM=B%! z9R<(zE_+?uJmklxWt(K!kV2pruN16_m=s--rrx?OH|%wj=6K~$hXGO8 zstP3J3liA|&w*xb7!!jf{IDGK^CtcQuk;>n)U8|(?p9pvL08Z&G6ht|6QTQRYF2z@ zSNh1R$&au*Z5`zJIfXlJhWSnJQzXU|GM;vt)xJkG>V2W_PB8VHftG|e&*w{EvpB)l(@zO zgj`OuXYHe#@MUX6%~tCtt5@JfPtvK)AZlh^L0l%59HKmtVl4&y_?t|e+&H&k(Wg8gqMqCWFoJWbJGl?|2u>OM6T-}<>cyDUA z7^K!Gk9o}{vVHz#Wv^Mc=xAba+xJFhYMceSZw+V* zQP^k3L^TS7XoFFwjfLKvipv=<5HUhxhy=21J0E?}Gb`P!v4L@)F=SrZH7d0Lb`9r_ zxx8SZ7TuC1HRN?g7YE(@oi)W_ud1smTCL7+C$$7x{`u>i^hg{F-2E?9h`c!YT@s+> z)Sv3=bgAKLgMTV-F4!Buspsmh_44DAKqX|=I_E=;^6br5s?@rDXE*Yr9oe2Zx2Rvn zI_S=9cHWy#gA8gI#pLAg8@*V=H!D*p{j!VyjQ%Ln`Xmw5_--_HpmO#bBU^a5o^2 zqAnTi;Tg_iJ@&a(n5m+8rO;I%F{r$4wx8Ave$UEAo>5yFC5i&OwJcKtwIl%Rq|#Lv z)OE}2=GZn*G{W<`E2Qik=@h5+7y>e5gkj<1is)%{SDn*jiL*)qOA7ZAn8i2c^>Y;E`DLI63Kt3Jg%KG{C+-u{tg`~RlSX_EIRPEX*TWX>0aThEZa>?)5B;RFbWmlWr7pV2V0c zbhUY*oH&*$tW?>=4DRasFL3R*KikYBN9r;Ciy0?qhwrHY2!ng_<>VMdJ2YZY zY8x)Lph8Ud6h%4ynLBg{tm5){V^!;{m@`hnoLeFwMa-!J1zZyEwEG6_Z9uX-8iW-N zwTR}-qH?7o$Op=YySJ^shC1h40JBL_iSB{KaEF0Q)MH3yYZ0oLP9M5di1cCCdq>}) zJOA}-zLt{gaE6$y+fqZGS)96Bx5?GX#(Hl9hwAKr ztSs9O$dOh^3zns^DpTHScx~u`&4+{ZR0ERfW-GKc=4bXfz0fb@w}`3G^>6N_P++eH ze&&pxN6y**k4PIaxl%U$@tt|bshs|fB=PjS;q^2Ea-38T_mg_^C*7{IS2236RwauU8`5(Tlbt% zB=d4Iwyz^UL}If{?8G zuJxgr)2Gvg=F{vTb{}76snydAlj*xDHEPrqeh?h$c??u;*rQ9>h^b*^^H2LJTk_cY z@&bSw-rE}aHiC=Ww}9mLV%`_feGmMRYxg%-%>6uIFBN$u_O;zS(c+1>(iPy&$j?|i24eaijb1nSz>20TQ_xbAFYRz1Hdbh;z;3z)4CYDE_(rhr zl>5mGIA>d(o?i@h!eF2G3KSExgopEA^4{zz=1I24*0V-grPFr2uG1;WA3UwJX;L=2 z`nWQQRW%x{T^4@GxG`fO%cc8kz-?Bbj6ktLX)~deX6fkSL=0>^0XQ`bRVi&k#?5%; zGXd|-xBs-_uqr{j2#eRoAfy$OB5yH_uJ@tUa`<|HX)iP?hxqtU%(Z;p^8&;J38;*G z6^SORWKs(b8G7A~A%5GkvEW)=O8`=A>hHfon8xS9+3(@lKc=M!xckJ4f|C*BTLaV+ zwLYAt0#9igxu3k6n%-` z!bQ4bMZk#Z3daeT1%B`t+9*VBl5fH*AehlRL9>~?ZgUOYU0d&c5fV);WGNVA5T>ou zi)>b{?|@lD*-ECDY#nP@Q`*Z}C6lh&IX_|Ed-%Wd$d9tnG^sAvI&0sZrfK(=;*K$I z^`))Qx~A))JTwm2<)ZEP4f$3RTNPVt9o<+AQ^1MoSF$UQ$;Rw7^^}FL)-BJ2lG-7s z^-4y09$eBEzSq09;#R6V)YR*WbRU2>ux6rVVuvW*qu+e}SJ=KgGl%d*0KE1nTL$kb zb)ag2qTObT9qK?Gl4=DRbS1*Ha1K znJK)4EiZia1l~`V8nxYi=LF|jy`0gypls5_?Um||jwr}LgER5~_76Zbze>FQ%;Dkv z%7Y7NYJN-I%@ZW=)ZA5|)v6S_#BoNb;-2k=V|1jBg>!)QDc46$VeL1X$^zvy5dDlv zv?naB5O%tAn>FfJ%2IPL4_z%cTxA+J>)Mvc0X;j&fsl*=XyZsr_xFSYs0P8AqL17) z&ckUjC2Jp4t}L>j5#(&4EyAmh{G4*-RX@BH_8GE03)Bp9KA&tS;P#^25`AUHeY>2q zh11dP9^s@KuDM2S1O+E%sh|Qwy;FKP!M|Tg|Etc#C&cIDM>6jFUOYFHB9w|CIcPn% zl3VWehe~cQ{%nGPql;>h8Rpw-UlGmYCfFu&y8@>vg4BX@Ux(23GDH=YPeGxnxxIbvvNB>c=}>n1BU zZSC+4LV}%Rs%w3+(*R~WN|B_uiBOM^#Oz4Y#G=jrPY zw)($nu=*s1O`)m&lKu#2vLGRwQqZnOf1t;Ip|A$#@hp|x+sigH)@7g^9toGQva)WH zePA^zp|3Qe{>a|JV4g1ps1a*S?=53~$Wkg;aT;z#Tx|S{QxgAj7u)gyrtH6TZO?lI zT9-WAIv((P`AGc`6@N0nogU$2F>TPWz`M$d$ zSxFSXRP-c~%`QLMsX|pVA4W$;xI4uL73kU?OsxVC^pP)9yZ6I8_DomhC&HQ#YR;){ z#xAhGfAiW}E6uy;NqA=kAer{@x-ETEWZ~hL1D_@tc`LbsWM|iCh=-#NkW@MF*h-ylFPjNIqiKXJN3ks!}3{mP) zqeu{WvSQaF#$K~Z3kRRki=e*Fq0&8*^3O&*KYjEMQi&v)f`HxwrRek)R&Uz}A0Nal z-R7(95x-&JcZ_a2NMw6$J&<2tLpw~UoO`%GE{-W}w4TKqcAbuGZ#2+XGaIjAn@=A3n~WAxdpgyF%#4LP^D+A!@XGQA1vuilp1VbG-CuV55~PqKV+plq@XGF(JD>ZzR4WJ*{QW5_WRC$ z-!;{LeL-^r&}vMDv!T0LvpC@5uS1UjB|A1yke`js7tK{If(>3W_>dBtNb2Pb@;aM?;NY; zWk6(f*F^SgBQ8I#w9=Iz2r`O!D8*U z)z~U#JNshtb18D!-{tOj2mJ?*<#!eIvn_01az@z;W2vlKR&VK3|1tgcU+5xk2gI(T z;HUOtaQ7b)7%G+KX{)e>$uc8OT~An&Zrz9oy9?fg9jt8Sd4nT+K+qL`WmYyKL-1=l*a`r<#DY)&{$I)uR zT-3$n(#k9k0liRJNsW=srsvPSQfiV?<+w{(IL(FAbs9<`R7%Ji`DfHRMWBwnvox(_ z2jT|&IpAU@tubL=?_r=$$Dhw*TZ=gc-E^3{P5?{y;@oBawDlPb*rQ|Q`FbNQe6>Cz zg0>`124=`_roUQ4z+EcJ$?I-Wj;PNSu{$PqoE!B?{)&$6WI*$Cd5v|V_iZL@b+sJ^SR{Ad#ysA%z_(&kOkx>~2f7XYk%>>IfvvA^1C?Jv!L z4QO?gCBrqk2ii?d2JdW-vFvCO>Ppek`=CdrdNj;8F!~%*fumrae2<@VnoOY-L(z&Th zN=>_bizg=+{8tv@FQPKu2;gtnNm>t)6liY)G$)~@cb1TI7atR3VuK1G84SsbQ!2+eugE7w!XA~W z0_{D@O5d(13I;nCiw@`bB8I{Bp|}e0{s?aWy_oaAYF*%o@jL)J{_gZJF_M80{$#Qa z-Dg1Bu}8Jl`Q9@5lQtsmEHS~vp~l#ja&2qC{pdK*ByFqdj=QEAIK@W3Dk~^zg;SI*yra; z#VbV+nel1AaB`Q7S@I1O#jI3>6u>gbJ5)QeY*tJd>yVIGnbDlB3tL{#Br&q1W*Q9~ zNRPgfb-7m!?vmx2jR&*K=b=alqw_N=Qsu+}*LS%v{Rx~#Rt(4+2_h2JohV{yOkwUAc;!A{ld*=$V> z_v2z?43nG(xN*bDO%9P?Goo$32_pPTeoLcRICPwLzP5&JC|keQXG6z*y%UI~ne33*^_V~wy3d*EvS z8SI6HIG1+dkW8*-0TgDoqAv37Bk?{&{|7O4=mhN|7#Qg(j)~pln4205jlRns%--$w zb4)ee>6#^@T7chIR^p*;Gr!sQxnQzDB36$gB55;U_!4F9x*6*0q?Vv2MPKH82rzbL zZ=wHqIM5PR=uqZ-8mtB~#Pm5U#_Pg~k57W0h{uy@#&amo+~JZte4^(0#1%)XMv^2+ zY;ICx8bXT{_Ba^{qw>HBxy#4R$U~JFpfBAoJ|9;XgrPIY znROGv8HubK#mIqa8wZ#RWLgA<5|}OzK+QDn7H;eV>Vs#3(LOB-B_cBKFZEDf2vsjp z)<-9d2XYo#H+zv6e@b9p&(P)x@>xSUxYuO7?6w@vc9heJIiTZ%A5i+}EL|X41Q@3I z7m}a?SID$TICQgYN4kv~)+$ix^TXglhwjC*t!ztEezQc@nRWQwrz&6amjhBZK` z2i)gxSCj$W14mUevmSna8Ozh++CneAotbB^aL&U_&cVkJsE2n`Go&p1c#tdmWSRJ0 zPDaHJGVQ%&^~wQ?XJ?5N*_Bq7BCeu9Kx@2AUz;OznKT2=lEV$xVb2B7VTSN$e>{|x zT0xLg$01%0VW8Mz3+q9ns7}7V7CnbqMW@LOr`mq%&Q8&G-R;@7Q2#i~bk$DtQ!`Is z4d$!7s^2j2Q@+{7R;4`=A-_Uc?);vDnYjZU9dyF#Zd8q{EfK%&Y~nW6eOc_&x|UA2 zY1^{pxw4fCw!x>C~eZ1EiAn&Sc(lql^q59J{v?P0H*2)FSo-7dH#;RqArreqhej)9q#LDx0n=XzTaz;pu3Vy!U zSvKp#MfUHg;+McfK_HuiIztBg zOyUea)FP@IV)5|dCS}p5g4@$>=La@Db2mnl)J|lSm!S94quiC38-M!*Ac?&@FbDpVV_Ye84S)yW`;n_iZx zx@oOrTBWwMRg@4?d+ZdgT~$IzqPAE<#1>@x-kurrbe{Klp6R@w-#_o~Q-9?X$({SY zuJb&v<2ugcJg-fsGKN&Sj|0QUEJQT?Z>>B)eGg;@!3r_S@sY@G zL7_tK4v`7oIiEDfkDO!(e=ZlsAlCn#=bY4MDrOO;f zLClu?cS|8k>;Pj3BI#!RlaoG)-2GG@q`_Hli4hc(G>|jA$^Nu>7e>YSvlj_1qoeN% zGE*A==9GOo>-0A*?z7qYk{SK_na>RD>!f@&wm$;f*B4*i;j24*^@6Wn@PEJyG(jf} zwtE45UC{r=&huZ0`mgBmzi>QX|Mk^c{%<;nQ!DqYgGzgM{`EzR}S{r&46 zcWXKv`!9z%1m-tH3kV{HD*T;Xa=Ux}XsPifzPV4J`0n_H$@I@^m$nQ#mHPw;6Ucrf zg;*FQW2N-m$+k1nK*K@=bqSL3s5U0RvP{$n3VXuiP9-|A5GSlF6Qi>=Mb{ zYft);JN@l54Fy}*O{(a6{^#o+`(b4j6c*aozwD+bBY#oJ?{@3oNhPM71vn|CX;+~? z=UF<9BTEpuJgwb$^Gy|4lDhy$*9(!9)g4gyO2MoJeP)m=ub>c=+rO-=vS>hDX&&>eXOY6t5-xa}$p1^cY=BzPxn@(yA_=EFg~ z?p*Q{V})xv+Q;@HVpwc65f7cgCdZo67G|8rBX3rUv!qk zukEO0gSQhDA@U`4L8)wsj3+T+_J%EY6%oTP=gZUky`(4Y^7GeM-kpc(mbauD%|+bDe6oeOQs<%mcf8E-qb7waV%N zcG>`_x8C*AP*RVqzm}|?mMpFFZNSpUaPq`h_j-sGvr0XenV0`Mh!D_0FCvtIap$`> zwXS)Q(@Qp|vrLX#J;@99(^4>b@Pu=+J_x9 z81$c!HNezqBYHr$um3SS;ND4jykKzZH|KWE)g9oR-LR5`E!b6uGuB|odRtx@hgiTl z`{L(q7kLyPEW3y)#QI_@Ib2%Kyoppp$5>8D$#MPfD!hn`lb817PfhU1+Xt9hwb$!f z+T7d0ad+?E183_w!yp1n)s**kdhm{h0{*ho4oSc)~ zj~9<_i?>w5qPb5J_)WT66F-gJ)htIb+N~k|5atM*-C~fvo7}4Pw6&z>bcD`cm5jIF zj4^&XQpn`9Km`FwMY^iN8A$M6LfPcUXT`3oTYx8T8sk~|y&N$UGj$OPVQKU}9YgaB*vm1f9` z1l#P2v})6K;3t)gx4sr$^(_$O({CQI=(ejM6Osk*-j&Qcpz+@tmi~DWntFa_uKp)& zF25_=TDsm_J35|%{70I`F$#|eBM~|~4k@_fm}1}iOz>I37GQ0GyF?vvx{FV;uF;U_ zl8H14&r>L0+Y)uvrg-%DIBC96KLVB?R6JMfyQ*;E;-W&^Ap=x?1+SM4+&VbVXJ8g5 zsJCH26JInrG&gb}!Jsu~(4RvX5H6|6-B8OdF?tS4y(n~bKYik+!udXWxqJsc&YZioqlu_NhPs&`jxz}6nz!s@6p%4 zTD~sN3G2+YDfYgW99`^kZz*55hUVyFtxC|e>Mm8DOLX$Y=^Fn~`u;tQP@|GLB&aGa z9ctwedg&QPAa!51wMz!kHgW*mkrN7-fuVjQ>CW+MMlttD|&FK4f`^w zJ+l7Ry%1g*wH@MzG^N%G?MsdbOh(~<^7ia}{$qtjTY3tcU9&8&kvH_jfq{=Pc?fn) zY+r&`ybq0#>GxSn;z4<=5GZ)FJ}h^xUM20aVzSD8Vd1PRf=@SI7tv9+%h)`jW%uyR zdXE9K@eL*?CUe23MTv1)5h2Z^x{wA$u7f=!Qf35ph=U*2?L8f)iVv0wE!c-Gc+?%9 zlc6ewBW7Q&cEPoh;(W(*@q*r-BSV2frccG~fb4x%M{<$I%^U)F{9ktu{~3++%K<=; zFoxPj+9|G)*>S(Lwkrdf;(gw+KaVMzg=}z-X6&7*n&+ZI&<4!+K~CWQ;P=YMsD?7J01{-oP! zH@KX)yvb!OJkM-J-82v#X0=qVg=CePOj!@sg|Zd35gzwcrTTkvwDL}|ej};NQ z@#Gpy58-c{g9kZQ_k4skeOhr&vktQI_Tk=EyJVUdyW-}I2}!(E^~`7e9jFz0IBTaa zEEvp)4{T2z&`FdW?zgvx^~^c4O7&t)80Nd)kPec{7No0oO3TeToTNSu4^P$!G2FyS ze{m>JISOcLvzKj7xixR6OFIun5{Uy+h|2X+umtxivR{3M`(BOoxum&hHX9XWUjdy= zjT|goh)EP>JPKOD`HSbNp(!%DeXvgr!nQiOM)%2d$E7T7LW{1x?LvVHBqUf>S{s=w z8}2{!<|fzf$wQi=LD1aFAPv^yV|?Jx0zK+z`$6s|;|(PLyAC}XnSar%V_6M*TQZC> zM;n0`gL+sma5Hvgp%CMl=ye?htx~T;x;+WgOHbF0*|~X@uDJy^X4eCL-b{}H`}j~B zUS^&Omtfazyw6ifsFy|9bk1!u0vHTIlhv}7FRy!b&A)wn{7S7->oyWjd3A==e%T=< z-VmpEasDCYjL?yS%a(z1l_zo(^F-(W5jbR)qO#Jmq%NK6>xZ?U3O^h~hFJxkl;f~! z3hFtJKsV{}c|^aGZ}+N_JMXt?(*)_`r-S6@WZGX;uGO&Ga%SDe4?gN8&gJHF^y~St z%)F0DSc6|)50H!7_3G5gW0{X)Wl>9;0+V+N(LA4+kJ5Fq>a|EL(>AdmpVIn~a!eIi z?$wFKKX%fAQ$uH;n5IgFy*9%Ze801^cprHT4-0~GYWyAW2Ta!cbLWEILT3!7pmKw1 zAY8z2Cpi2WP5)1jzV*(@{$26w3T@q|fBM5g1Mf8KMp|YhMx#vPRrd=uqJ(4YDGwW3 zR0AwV_Er-KdlT!%&ANM}uWe8yCdn6^-k@WgW6stGUI!wT$sR8UI`L5@s?vAVg1?k; zHGSAaZ9cJMf|;R4&9X2Jt9U1=N`5)FW^Qisuu5A!9&soj_~y^05n);`(S(hNENW9^ z?Rh(Ft&0{puDbk+fm|m~MQd!&YfaU2!F&hw*g>OAyAdSUOE%^CW@5_lSSa9a=?a;P`lTMugsfV6$C}k0*qN-phz6 ze>a(}h*LV$26mLrfR8;k=4l%npt*fmo}H^-DN{_=edJcL_JP53=tx(k0z%fm?#Mz{ zeYX8ba?C9IR8q4=ds~au`u8^z5l_s}KX->*AF1f4BZk|1gxb*r9&)Xe2g5F}2>DD7 zY^i>&)&Ckr)JsR5ykPJKkyWM>!Y*P-B^pAI^p%8rlNsi6>8$F%m@Al*kEhYni?0=@vN3 zNENNF883NoXc#_&G_%4L?ZP7MxMA~i5(~q-uZOlAXDMr&gNIfq?wu`rnwtlNQG*zE z03^FFGt>%Y1Il5#(bn!An9X9$#+^^Y!k;JM(H@61+W`*bQ~ld;b#Or{t4%yaYkBjX zEo*g4;q_>oR$0Dgi+1iBs?w~NuEXstj1WJhO(mou1W6S|r9nw(q{j6wTeQdMGR^b! zj!~DIrEojdLh!A^gIRaOql7hOpXPzjm!t+CAfD=T+OP+~T=p;>nBj{b(f0K4`fF%y zi-A_ga;8Z{nC~L0&m&o`dZKazC;(5m%1m?F6M=PC&JvAb*Q=P%v-_NVef96z8wh8) z{QJ=GZ^DO@%r1b2I%k3S{Tq>z$9oOeC()lKna{YdvzB_N_BaMptYdsOH@PCpkM|=^ z^W39QW$sAgc)c7LV=Tm6g#1=08wAd|lk{Ajz~J>^yUwF7HN)T`bJ&i0q9H3%xwaN* zBA;MLjaK9>-TN*E0~@owT<%><8RlB+pJiMEfS>&&Cy{=DI#ljO$SAb;*@%u5TnQ9J z3*QK}m0dORKnct16a8Cq_QdttUoIMqqcAJs$t}z1;)Q#yteBJ7o{s~fc%AP%f?cDb z)H-T86tIMVo);_?#t05N^VrN}5g_mq)}GsfI% zwlD*D*m!jfAU&I?(C0}?h-j_L+Jex^XvQ4FMbz__6@eRuP<{4I_+zyDmrJXZET5;N zC6!KYyL6-<&&nF_37Hs_uGjqAnood)Ds2GVd=Fhx{wi?NrZu-Gh<;F%YtNF4-r|QBtbK~P3$)u2uIE8~Im1sY2 z7lYP8QN^H*&aC;B^sVnh> zBIesxrLd(t;}|IsHuzpwyFJsA0FgMn*;QLD9`XJ{eFxVXr?0Wm?tw~SI7kG>iEL{N zLqsD)OA7BlFt#bNO_?n?j$RWBOLLwNJ6Y$bt6O>tL{bBOCf=Viy7?*b^is6`;KpdC z0cU^X*s_6--;=-GFK=%I0#UbbyO4=?O22b;z~(D8ne+3VP?LhR&|NBQiF4sQfesIc zP)IMV*pYG{OTD_&n;l)VhrIZIWX54q^%+}K^ubzIE(YJKjLhFm;%2cNKwh%*{7ysm z<^#L=?T0~*2#1rL!N(qL)+Cc~y318!Xt zxlqcjr`pt#7_jo?#vNb)P=E^~V5OtKMI#OS=N0hLqtZz`A^oxUTBIb3X&t6k!sV~$ zkbPoD+7d2^j6>%<#+{31KvkPVWe@_WUid}c0tO?IOByP_$wJ_;aam4tPglj`ZjZNH z8eh7(A*=_7TgjF^&dbYRnW8OZ22Zb+dk+;$GV8;qX*q}hT6VQ}Z5}QC%Is%UMfj7z%P*>EZyal)Oj-n z`2A~6y+Odupst>1$kgszpYB5-k8dRqgmtUKJMeOkXU6*ikWT_SuJ?v(#_bVIHV!1W z?ra<8-tl0dduKx2nvPh4xafTFaQx)hQ*AmQC%Jr8P}FOSYJv*EC{a_-7#U1C85UMU)fYnkWD%ERwTW7gVn^1ez2R39PKIY#blo zst&k8FRCb=b87cV>x9g9cS-6E?+AhAk5Ok)bZeNQI0*xpUrh+loJgy5HrIR0yXI!kTP0*DvQ?|U6!s$)x5y@Q^flh~2Yp_LWYR!)LsqVfqvqS9+xUay2AELIpf>%6tH zDE?~SDTP)9Lqaj61EA*ZhbgIEw>mb4A*?6r37Z+-K@L&{HeFMyY50O#E>fWknNK|I zRX9STX8qYFacSQy*!+t#F_GQ;(+lZ&Dc}Ddteh5pxNy=pghesXF?UpYADtfxmG6K* zIy0o|Zxdb{R=M8V+uI6HaBw+~Nwu2I-}8fh!yKY~i$hM$#D8E8H8v7tEeqMcrmq~q z`0Fwb*wXMjakql0QpbX4Q*tNGnXp~+nWtROp1$x6nTP+LsJEpOu^hHgMP}>8q`3u+ zJyQ>=VU=5Ru9lJOI&2Fvt`7Z!x{oc%9IkA&Yo#2{MrCHf9QMs8C)flyH3RbOwU_JO zA|A{QT^jj0PMuufO-UW(E;OE(&Bot$Ns7*-@xGMA7SN1CbFg-RzbQ#``Ww0MbZNGP zCo_DIne?G@XLFb4$L-@Ex8cKo9zoY~fUD@N)79VRzg`PeIt}D2N!+8m81cANRJ`aC z)+Ln!@b>&&VN|sDA$Th*dXyjwHfe8pfKOF9aADf}yrEwIZI@Vvo*SLsQyt;E_^BME z5o9`oSzYoeFRX9V%-@5%@Ngv{A-RGKk~|3{#Je(!wLCh-%u0=0WCg9fNp-Buyy?y7 zI>^~^J)3w@-`FwKDTJV|EsvVm$Bj?os4apwO5BT+n64irsJ9GPX^oR@3z!5zSE~~F znLT)3dvxJ#eDknM5vTQA1N>(Bi3e09OJ~wK!cHOoPX%WGc_g;@iRRs{t1+*Y!sH)p z4@b)59XI>T#D6@oB0mq$@kLCYw+dQOr=(}OOL=qCrXN~{;f zQ?aoXiUPPcX)jWS!ksJ7kGx08ML6u0SBEDVQvUcGJ&y>xbv1`{Y}Ix7Di%dMW(EpZ zl(RhiaWw~8`v=Z-xg5!FWJv-G2YTahGGCfuIM~!82JSv)}&h|ZZJ;{D)E`W>U z6wnZgr%OLg4YHQXbKG*q-e$8{ZKn&LeH(ZA>4}+{c}G^X0Me3k>DdRz((sF-#ja(9 z32q1g|AchdVw>1T13V)S3_M@$%g`^)NJkp~G`(=|Qp(P4hd1VVH8z3SFevuYr;ZNx zTj)l;e8+hMbm`KD1Z3P<>oGuZ&^GmG0&dK8e%z;{BA%1JmYx{1L|j`5yr_`)M$Vb% z{Y=+sRW+OsO0a9RnwQ5NpO}UE-K%+e zVIC-Lelqfg*Se&&I2{sISQ|xrqoq7Y%1X}5=USl3Wcp6)@okEb8l^YzqJjLMS6=>i z7j5`)FEDjyRhzGFPu;$K7vmbtyJX=dIin9%$ZxF8lcygR;H?rm4==Am6pKy?#NgG- z%|)tiyO>Vq;TzEz7Yf%_=QA`_i8v41#M~^Dcw*UC@<)cx<9h6Rs2a5<#&|ZB6yTcp zp}+bX#PyXCLZD}Lh*qGj880_TvLn!a;e?5ZZniSme;J|fwiR{+X|*XIR-Jn?vyk^0 zFPpg>aZ=kG!oP%xCzE&Y5+N@URH46p<+`s<*6 zEkpILWeMZ&)^5yeztdfNmhs{C_*uc_e^K-Nx4L_XVlMf2;OiEs-o=zs&Dse=s@5 z*Q-lU>bjb8?w0L%*yq$+fvJsCGWi(hdA&*hiXDlH1~g_zbwc zy{bd0&y|s&pA9hLU}|FBPSp0`Y#ZT!HrJ`jGhIcbZa>lf?Xs~>-$?BWl0!Z6x*^Y2 z^;i}sN>1WV2vii(KJz_9*{n}zMClrwV6LSmfWwj}b~&rIxE#YZuXh{k*$C^zg2a3d zIu_~mK|iH(EOTo7+MD9|-D6LVTS@Y;1TDG+N9^VuPM6*y$O1W7#_%btGEx_WVJ+%MSv}e?+|w7Wa5c;Xq45*spGVdBN2GP6GGF5xuG>XjwWMi- z+4V>E4d=znj#fGkq|)CCk*;xA?q0|E-5J*tmvBu}V19SzwM1kO32w5HjVyAYp8;!M zLFKz(3*IYsrk51fDv=$8w(;k7dUo4R{^#VRWC3m%u+o~Q@|&^ss_)2q$8UedXlMLn z)51Xj2@wubQ5~w5Er~;oijbt$#`Y(>MchWMWCTNOcgD{X8`03oS+*093|)8RzcKRo zTHTRurXi(!Jt;rG4pVX5XZ_tM1}^UC6GB{{okIfLNj_qd^167fJDzdl$zMW;ZG!6g zV<2j4ZY-q!-aY7~)GlX(%4!$Y^8Qq#Acq01Z-yu%Gwv~qV}vexZA>`rnjbSTcCr_I z;hX8Vn8lmB-g)b;Xd&EgpGln(6^RLRCK$(nUJF&5AW+mH5+R?rFjG6>RGoZcT0|8`yC#9G$%5#|SJcy&l5R`Tb? zkGE>Qje81V<_mIBKHfL4EMubStBtC%{d*TYYlUwbtwgY@jBVdzhit;?_+!l`qH*9YZx7##5b7ROpv=KZndET zNJwGFX!R{cD4xTmAu+r<4snt7-|JhTm&(4w{UIZi7$TT)TGYy++;hm3|3P?ASD6^b z)pwDYn)(tAb7^l`!eI1asAv)q6Ec4}_vaBIR@8Nh;Sf_m(;%KoN&+C7vfwlALM(A8 zpeXilkd^rjrJOs#x#v>18xf2?{C;9AAn_dL^~|c-COT){%adZKUaS}t2D{$Ap;>jj z?y(syLxSv+@S`AUCjSGp+{+G1evw=~ho!=A_Ko$6JyaX$iz?S%Q_tSrCZ|z|jv2QS z(^P5k5&Pae2k+~SqKvAwR5&*gdhB|anQ9xU-$Q%H9?^U=MKuqqaOf8Dbwoob8a6IG zR^4l1l@`E`c_BB)eva}!HLsq%FO^CXk~E^piT5CP0%7S2^nVqWS}XcnM@}kU-d+I; z_6p@i_h9#^3HAd!=gNK)J^W0;MeU4zkwvOWdYf6NlI%i5+k27?vNbm?3#+oHsk+B& z09_dB)8K9z9upEspAf_jl$Nm8r81XBrhOtn2v~HK_iJQd#Eo6~r<8&9dr<}q-Hbgm zwr2U=-fC5LE*@h)@`h12OtQ68eSc?AeKi&%?{7T53B3vIu_4DwvI%%iyM-ryU)+-* zvRgn1r|>dJ{_eAFb&6|I1HCSsCFm0MmSE)e%W_G#W7%w;5g6Y)IILNGJbNs53N%@? zB`1o)n(NVK&L@a&s;fj-FZQ~aZB(6rrBY)L3MPPL>e*XddXB%yW@hD?XUjcTm=tVJ z>{TB)Xm#--x)9~Mu{7DHueN>*0#O%)*Zl*-0}LSC56-zG)Ny%kQ;5nN4=mXORh>_R zfN)S{*7|Qw&-@P|_i+R zc9BO=TyRi*El}iLbQ*c9baf@&*A~=Ow{|+r=rZK-agC#0@Q`$e>btk@$-!C!p2cr% zYG<}KwHfcBZG+pm|1U%JnKmAPH)G1Foc+xJo#eSE%m*Iql-yMnnlIOW3iseR(mYlJ zo2X3(32U0_jCjVzJ$E%R^zpRpN0o1>;vi^Q(qj(>GP=H~(A4+m-R;f}xCV3H(bsR> z!zxla!!kH3aZqUdBGg-Yw35tb7Ure**VT0_jT|UaENa2DwFBwIP|s5d_3EB(__nk$ z5Gq8US0=NL4@;aW2|D`4q3_J7Z{$?{Jy>w9$BuAK5ZPv|YsSv;XDZ@mRyx4c-6Pd$E758^BfS)PKJ%+ycH zTbIqoTASMPJjih#`L@m2x`emZL_PNtVBSk1jYkk}Zw0%6{81<1`ONXk;<2@`hJ0>O?=|=?R+;N&?N7mp}LjNGtPc=-u`scgNA=AgoV` ziO__Ba!#|w1+dqqp_f1;i$vY9)qy$j*U0{e8@rvQx|IY)7;j0> zqD=!8)Yu7Q&$P;HNQqi?5ciJO9u?#e$#N>NjZN>0c&fc2m0D)UZyV zba7I4x4#9;X1>9UmX)i^s`5b=3cEFKC9DLlzs7?oYYScQp@KOe@gAY9@{$=`lB#cK zKX<3j0%?UCFnU!Boy_jJeA86;0W7eslWsI|s9Sxj9x-5)+(*<6 zH=%!@C93HVZJifGA)^FJ2v%2QE znTp38;CZ@67dKl8^od_e%{a2GO9c}Dh&fDLui3nOJY)0SBx6`TK_BstbeZ9#!I)Q< z{h@&wpK?I_($%V2^i1KW$l8TB6k<)-M!p$d5l>7)B1AP`^6J;4{YZmF+)WW1sNp(# zCR&!)J7k?bo1=D3{J|AHsyU_g5GfDws9{W8L@R#^;T+XIo(gO~@IZ#>+mdSXZF*MS z{4){(`^)@m!Dpjq!PH%!(*Im9@rQflut!p(U-WT}WH1bb$ z$c9?+raM~M;pzfsK5WFa@3xbm(C=XIFWc+w?Dp*j9{*mGj*XS@vbA*(Wd7lE8~9eb zGLFE)RJX&g=mGqdPAk2vmXEj!hCuLcPtV-oM+}uX5p{uzlWVYqv^XY{eK~XA*+G*7 zdDbHZ^7#D3Ps|)pb@WTeMgJY*2i5Q0?zl+eU zGZP0F@Q=?P7HEq{vTu)VIebbth^t0?)l0s^F%D7!``9WO){BST^SRye#Vs{}UPFN1 z(7>nW*6&}*kUn|3a)IhB*v5zLmd_4B&ySGwp3kWEEKm_vcDja*(L7cN*w|Qqk-{}w z6~pBtQA?9{6$PB>BV29F^{Zu!u|a_atRjz?&$P5zm2^`o)d4t>_XrlSzE>4xgrD4_ur%*iwTs0Ts{wYGr&Qot6Qpp^>t8 zid!Dfb|)OBFpB=6LV-k;TjdSCFdMGho!U^( zzr4@39^aS*tjhCg7Bb$N!9h{*X!4YD%mKujl^L2b^~p+@!w;seccp)MnB=}wu51S#8$TwGJ zaQ#i=3bS@)o2t}$q)^S~>?)r~FJan8axnO8)nXDzkJg6XM_M*e&|tcE55NiKH=C3B zeEe>db6@y{x`KMiNUci(>)2C7+4C=6#F2`Gfdo9byPL29@>{5al*O$DFZk3I;aO>g zAH{5yw659f6d{A%L=bTI;p>PF=gXbrcblu}Ok%ufT7<7P?D0-CW2VG~KA_qf9|s<* zu6cdwR)5Zun8z{eI%RJnVzH(Gz>5oM!7mm`w?toK82YGQdsE$%OEO55XP2bwjz!5zfU+UpxkhClnEfWrZ_1ar{EtYvwjuPr>lUjN?t~on{Rv!UVxv-UV0g9 znJM054K@p#sDdvm-@QvO4{B<(*y`CN-GqUsMlaQij`*+V8r*H?RJk?Q44@y+z@h=p zEO(HVp?`J0cNSN-HntBl1=TG>YRWx&#dwdsWGdLTmGhg}eo15gL_ zsI=9A7NZ1F;kGm&ozb)cmNU`BX1;b@4Gz}=`5mr|60X*CfD#}uai26I6hKm|LCRl@ zq)y_t$kxq$PVC=@92+ejzJDK1g!}5KOr%1L8AQ0yjb~{wG|T0a$$1+K|WjVXa~o z5!E`C5GO5CTYpN2BF`vmCuc83E(7Rd(>-|P>E6O#kZ*YaTx+8>NSujlC+VwL6TSx zV_5L*x>bS5py)E&nDTQ7IIIG(zptua;w-fDxLbV4@$TBJ*ZDZXn;{-6t87J{tQWF+ ziy?QuZPTv@5-WRv^jp%j3Acv(3!4JONS91xI+4Gp zd!S!&D%jY&yDD@BNNgVy@b-*_;6_JTdSdDd=bCiWdj};Kg{~6;f$YL$9AX?ee}S36 zm$1J_9O!ptVHz7&<^AJL83!9HGaT_-u;mTAfd0eo8E3OEQRBw8|Kmc=DjjajWJoI{wj8aKGA0-LTVMah1o!d+vW%ePv)g5kC-3eK`q6vca@Cjia`5|w zA{VK*pYlXJ?8o~hZ}lu!csbPP5pY8)m7iSrYeKKzQ!a9clCo9>EpZdt8ymyl;W*9i z^YL9`kv+V;3+%J2PLu~JoKx&l?bnAy9acxfn-!{@`*Wyeb~opqLU3=53lw5Z;T42% zalB*axC{D1jr;IaS-stlygKDBvjbK|@Wba|2%S=PLl4?)qW>QyNTR`>FR5U1xV}&-5J8U zp3uHl)PriOaD;=W1m;L)k5ELKqq-wu-MB^TDd5ZN8-1Y(^TN8P@wA-@kVzCtZE^Vq zv1dvdyd@Dn6ye*Gpt^&!Pw}GCiP6E7yJb)b=m+(`nETJ@+B32{2!;Fe{wODz{xNd! z;JmQnL1xTCw?pblo zwR*%P zEputh5kFz`W@0iI0SSUrB0?^8GSUYp-(%YE@C z=x}fJ8Ky(QHW7Hc!Tx}Co%3j0C?+PLd32J0Ez~fv|NU0sqE2uH)qX@+^Bv*Wn3W;! z(Xcm$FrAQ=93xVibWp9VLbyX>JMNWzjaE~9?u2=Iy1$O}IVVs;f+9(9xmt0`&qbsj z{cfZ6lQ{?msBt-T+99fI%o*x^B>csRj(SfOqKu%=V0p-VRBQgDVk(a$wES4+BgDhQ z_&n2!_2+KKcEqV?!{f@i2d2O=wFB`lUhGj#7$60TIs$MSJ$Yx`@!Hf_i{3Md;&g;y z5BufaR7Ha84D5(o-Oy-UAN)Mf#7ga-_h%yPA zelZsR4@IZSS|L*ML~s1<@fT*+n7~`~eFqdG{OQCuZZ{?xtIK^zVvWS1eJbEx;{8qc z+mb_wces!QXjqtUF|D&FwRq$O-9-eUd9&+K+Y`6IyfUahN$l>IX77QU-%?yojqphR z;b1^70+AhN{JtV>lSyC9QvPO*Coe_%|4P>U>?pTCtTd8?s}s#Sy#K=4pUfB8ZY#Wp zs)?h`y^-Gu=!Oc)eEDfDCwYX_x^H~BCi1gLvOQ@;P=7h-*bWu{A$PG2 zy1!oV>p=a};QxpMUtfH6kpBg!_-Zy^&E~7weD#8_UhvfmzIwq|FZk*OU%lY}mKQ8i bcR1Rw96pd1Sc?A!_`9TMd;xXN>CXQH*j*YD literal 0 HcmV?d00001 diff --git a/docs/static/images/compaction/l1-l2-contend.png b/docs/static/images/compaction/l1-l2-contend.png new file mode 100644 index 0000000000000000000000000000000000000000..6dafbbbf29e779119b1e5944a5688f1e3cb0448f GIT binary patch literal 230195 zcmeFZcUaQ>-#1*gEHm>gN4Y91&vNDI4waUgxmxa(S?;|Nk(OG{k~Bqya%L({T&S3u zW)5=TMp6_fkfNX>a^vrQp1-cVp8Ghi=fC^>!^8CzzI=GE*LeHpu8BVXk<&-^?AgO_ zaQoK%J$sI6?Add$h4&!$C*+{6b?&FV0r&N9>_PR2&vU;y?0efPV9y?rzji+Mx*+?w ze{yY)!L92LL-#Hb4u{)XKooz2X}1$*mJXji`!@3J{u|GDob#U@KI!{H!tk-@g{r4V z-WNxv46V%Z*RCP(@rVm`CJh-Yg5%?hL1fq*e3flWdFs>za%q0XOMP3 z*b38+UJUt>m|dRL`EU--us;XW3!idejY+b365`*k3ZA**w14Hiwz^@zg_`x<6qg}A zPL-bOCa=-MCT)#-n(czSVq9nBH8oa>=ePd}4sMzhKJKeEZqc;y=bQdN!r4u7Cy)DP zM2ZlS{wELe@16XYvi<+pyW^T3=~4CMzavbP&CisWV>3VWVLd9F>fsUSd)S?9_+$F~ z@0bujvUhL$%}-0>|J_x`-Zhc8JF$ZKU)cX2q)pR%;qQ5?OGXL*6JjUz9S=`}@=Ce& z$^UYc|K|^zI?8>J2dvFhiT}xi{NHsaU6RiOqvV6?IL97ewo5Ef`Or~koX=wa)A#gZ z@zd$LV|Z4YKNcf!3@@R~?ooa~U7nZmY1=Jun%_+vaibfQv~@8eqWFGy<9ioCNv%J7 z6KRsGDsNRjUE@hrDv1LDRml+VE@oGa4%)-dtE+-Bi<$rZTjtO~1;d;UYP z#bu7t0G0ukPyc!SSfy+41!j7GFXNRvTwBR-rM=x&Ds^^#*{?gHr1%03hM78vjbB_# z59N%VH-MfX(vWR+Isrr78!jbv?%kP(G$Y+0>qokoOb5;X_19lKL?H$bq(1bvLOrj0@$o(Fgan?4%c8ldXMyg{pow}-rTi$xSF4d z(5LC{^837!YeVH`QVC5DL%trV5-4(D*Svt&5e!pMbvf_JK3^=3f65Kv_nrfXWNsIE zxY3l)t(LDKKzZ#01!5H&6CVf6@LE#C8MK;n^R$vUt59As>DSB}24gi>aJcq~>1!9B z*&g~9X4x7^Fbubhjc~k!nVeyaNSD{tPkr*r4Ky1clO?fOAA3#Wb=sMM8jUn$ z%p!EdSbeJ!NF`zPhlbjGaOH_{-b9w>jUIIYZ%g*824wyeuIH(2ii?|@S9WdWw?eTk z!pPiDC05FNMMz-5K$TjfgMOUVm2xFx>S@$NbK4Ot)kZp9fXsdf%~PaOAx5v})P;-v zn7$DmKbzns!gzE!w$KHy-=_t2ZBQa1u#`Np67bZg22ce(fWgC)n4sIKTJL z{piubBRK|Z!{@tuP}s`)Dn7#Al30v|PT4>TLw65leBae{&(~-0^?%#lztew#V9<_~ zF|^p8Z)#ITx~}|ANtU;**-8bqSIQ^aF}Kg$&`umt{(XH*ImXf7K5iK|BG}%D_OPNw ze*yTg3SS<(nrF3iBz2&{q^xkXO!~U8B2^O&vA5vIcAX z7AObSw|7oztj2(mA1GcuHsR~I7&_qFGWrcJv5ee4rhe-evwTtlzzK78Wh=rn4P_VW z>QaDzyD2s(I5Q3@RZP9@)i8-L-J@c6`cpx!Ooo43dR>@#<650mc)kydLBX^ub=4S( zq;$#zJ&}Nl&2rc^#$=VCd=(u-a2&nzdmBrKwV9fr@|P5ys_0c+)-Gm8FhG9m^eLT)n?rqO zil@eFHY=nT&wU5TG>-G593(L>!s1W9{ymXhe*dqpy`$7pX>eN@l9*-z(=9%#ZW8Sh z_No9Yh)W+1)P(%DybK;29c7FYqvd{u`hhlDNJ(}l>7#(ieNb`-mQP*sdsgHwt(rTT zA7)Q^<;hK4{c{{VYkI;v5bPIYCq@*VY{TiNK5d?C+c%qnKt361m;U&>%=#y}X}iPi z%@txUJ>e2t>xMKf{`Yt7$u{B4%PU9e&2z~k<4#yYZ4q-ynBPXjNWOGrP)X)R)(IQa z{YMnPf0@rdVyg1x*Azm?^vxnOQm*@;HP~lK^Pc>(jVazo>@!rXy8PcC5>E$h8n8nB zOmS|457{4b8Yt0aC}0ezQe|8tY)bH@rmS%}6hO+LLw{OPAuDs+<`AlV)q)Vc90Yu= z+921g9$>9Xo$z}+UX$F1%F*6%e)V8TVSZjSz=vqGt_ed#qQRdSgcw;ybB&t(hJ?p> z`|Pd0K${TU7bYHIpgakeN>II3^IOo1(veTtB!Gn zOPJ@ibm(W_T(|Nf20^~an{hQ#2PQVJq>E%$=tO)R|>m!C=&C~&pJ;08Jj#8 zJ|cIyB>eZA{EY1l6+_O&azogOstutu1-pAX_)a$h0l zM}wj$A>=KxpI3alA92^Gj?;rYr0Lzz;xRceWA)E$XIc7)^rDb)J|k^at_Tw1!Ms7i z;1@iO6VpStE-2D8swj)-kG_TN$axZFgso`oG1H$%(j|A1f-DtqeK#lcgi;t$yNapZdPHW~6srvV%C*hW~hV@NbRJ6#+ABQ;nHb zQ{7dv>5)4&qY{f@R{J|Qll*H3`kw0_6+Zg|81r_rSD_#RShlh7>Q}Ga_{OT1+Ep@rsdBk!vwcZLr@2ivwG))oo36; zu*8pf8>kxQlw|$ZT_Ov8c<_a;m(^I+H-fCx(AKMb`$3Ul*UV7l5MI~A zs60^D(U|W@%BmdHxoh@1fAk#~o}U%;*Pr8ZFHgVywI;t&hZ}r9(RJ24eL6E%gqd-*&Zi!Uxg!BR^>QtaS(Ad#?>J2H8p*%$bLSrwrBw;kr-g%u2%Q3_9N-9T8AFswMU?>|llivT=Npl-1RYw2Qg8TUEcmH%|#35Ij?>&;k2o zImC3*Ri8HSzq#}!#1=j)VA{7-f4*BvGN71pXJ{JOn5qz;DGUVGe4-JgO#*)U(5tGT z$4Ch;PyIa@Oftr9kI2NdeLj$r&_Ra$V1`+;0Q$C#}P46d^q?IolN?7bM64@RU0V$dRKDHxv_b&)kfQKFeYT%WblIpB9KN?aH z29S-iJ>E@0H#S?q#nZ`F@~<&9rNJ>mU2k0+H-Glk%f4iQNR#ojB~y~7Wcv&-hBY;d zW}TJ%ou6a1(iE!0z17}X5p5gNMxb}Nz`0+9C1VZ5sh+seZ!ll zjxx~Rg_NtcG?@CX{UdINEqeC>pY`W9{_E|L>G{|xU>~q}LX)|4?fu2{e)k}P#6}M; zJPXUoD{~i7u#M;%&#n-0cH8{HdrLd82wpaNZhD!KV>c?NSWHCh%YyO_E8(R zj6Z|JT3l|S5yqh#>I)ez4!_F-Mq-F>GLI5RmU`Znp{ll|tv}QBq({oh^`P1N?3Ynh?*{(1&*u+X7 zLVx_ML&a6)ym#>H)%~DD&X#J}B4=&tvC(f$?_Xv=$c8sSz*P2mhZC9;9Q7JMf=IUZ zie7g^9mXOPSv{i|%gR+B4E0pj*QM|YhcI<62fMSJmDjS}`|e+IkobgbCL(n$1OD!i z^^qJerOIZviEb?ONdp6e?85Z=CbmBa+Cv=2qXYC9@oPHNOkKe{5dXp(1x}gSGhWKm zR>QTPcs>%SH@rp(SJFlQ0X#_{#R|*Aha#qzW2Yop>7pFRwJnm4PMN(!v356!nwncO z9LX%Ie~)jDkHnw{dGcHEKMvg7XR%c(WV;g)JBS)JoT)<1`k?0jW?QHi=(e{#+vN7S zYa{DY4_-rgr*6iRKPT>5Lr(ETVn1e{>a0An%jACr#3RV_S<2gJ#W>l0X3c_8!ZPVS zTIFpg0=RsE76YVF7p!V8cT_OjhxJcNh)kmCS1S@Cx!BAhwRA4EY*YGKP5t~`Qv_5g zm9w;r9=M461&(3$ z176*^xP7&HU>ROx+H*<#EDkEp5^a2>#r*zV>Ci!QC_o3h+3dej9AHS_mhS=ee%}tG zjos_oo7HBA7E91WJn4cpAit&5FstmkvP~)>a_MR8j|QJK*f;F~ zqbun5TXu2iM@7qdk8QGPfv;}lf2;UCGP6?V2IoM=`664(@{r&7m-YRC1HoXqaIcZ_ z8PwmS$Inh03v`kj)q<-bkZ^=e=*Jh@JZOn1SiO96{QS!!22&dkj6>|M3}ClqBGJYt zyXsvweYILfPahJeZ@oaFgakx)7>W;hf9Wl3OHH~ogDr#;<@B5Z~e7LdAzMO zOv9gR#Irrm&G~N4&7fZ2V1NC!*@$v(j?eBv3ZNY5pYB-Kp5al!p2n+_+h0%FYB}q} z;xjWFWSk8GFe|!U9uyy+`t4IE_)+`}CPZU!ZE3iZHkdW9`)I+y&JZjes-{^|qt{}f z3bmP`j`f@#A7)yV*-Ss8gh6t!wAu8{p{@43cn_T`r4ehg-o@0=yPg&sqp8fhz0_wS z%+-t(?(^)e(URKoT0`Z>$&71QKi-|?ToH-$ip7rI zOZOq#jV!M_x5P`gvx1=<)N_81iTlOVBCl}r(yV$Zn=>!I=DTbNBHw)6n0K-mAA-wt z2u!7|uXZomobb+ggIiq(%o%&$0059eM{w`=-&x z{8zWL+>1c?rnW{Pu^trA^0qgBljTFGj9BBuH@Q2$t$Z)Fj1pzO%9?LAV?+0eZQ7R> zV>t;~qgmUd$81^v!TQHlN+@B&zmuQN+w3$Tqn50znG^|%;s^%zG&5<`xV+F>!d^|K za$6+G64*@p(bUny;&eX+m=juQkWMPMw2O|Q`PoflnBG< zXr$CxV)_Dp$qP3&dOGSEW6-!4*}t$;vuC8I&W_&Nqrurk0YVf%NBRh~K^{Pfsx2*o z{zfZA@h%Nz_ONx+m0)dx@N=ee?BHo=ez^LG8K~{ouR5d@5Hjb$dxMzY2T{ak3^L}Qe2LxFjrVq(+OjQnYD+!Zu|aqd+`jcXTgiY=r+ehB~K zp?q0UrjP4fj94MS!?9qxt!>scF)hSs{zAM*8?_r&bmiBn>rK^ORP2_RQ{t87!D5>s zLW;R;iMR7BtT29w_gkTgqvbO(DX;!=(jP9dNOGruh0>>Y!M>-!%t>=UAm*_n~91!0%DRUNqCk zucNTgH^15xgC}&hUc4k62q-G>9<3x>2=at)SVE-7R#ZSi_Z2?qRS7jXJfv!CqgtKB z24^-T05tLK{3z8!y_%(dlxw~0a+me{VLw7mU^5-_x@%>GS(h01wlU-}i$k%EV<0$C zxy`s8z?dvS+T)tJnk({TA{-vnVqu$Tf&S;31z8HC3+TTx?+eP+1QuuT|q3$6&+H z*{#*dvmryYp<%edJM`FzC$1XSON>=8KVRF3Rk8WR)t;M2I*~RC$m5!oBEmvs`~oq2 zHTwh6_Mv+3Ht$~V1k3OFViHm!?6?c_^;A98Y!>;+H^76ohSjefwl-cSeu`>GTd8Rw zs+xYQTVjxeV)~JN3GoR?SK>~Je|ypR&*r-IoKo<{mem%`g=FnrJC|jcC}yXO2byAB z9N|JypGjGsZfmF#a0$t-nzI@IZg!I~fvaH8M$W8UasxBc)*F!zJl{cU{h7tMiOK?6 z^*404Mc1m!rsNxUf2*?FVQvGhxvt%^!93peU?;8h;Ms2+>_^oKr6ar6cp+|$&u=!n za_G+;KJ^q&ne4A*_F|)|LF~qj-WpKSM9F98K+aC#kVgA1%rV%&S#=JhZ4u6UeF^%o z0=Ys+MuS&u`H#K!%7|cg%=qiSOv=Md*>cbYP>M{t&c}{sX+sIcM2Taws+SXkY%!Ox zte5fb*1aDrU?D>Bu%{b^G46Vd_HiU|g>FfUhW&^t8?_k`c5IarivN&5@=jNkwU$59 z!`4Y$J8O!#Vov{B^^Lu5x;#rTKyVCHaUy2)g%+Q>C4#?vX2ygpghs0+M8h2-ESaFPd>xuwAw2b9@q_M~wN?wOc^=iq6ADc&em%x42WORZHv)n8O!u~u${VF|tOJn{fENF#SF`_JVX=`{2 z&Xye=g-vk><{~0jg&?q`M(RvozWm@IlI;a})T)J*4mM(0zrre*!WL{yPf=Y(CAEI7 zU*&g0RBDY|0?hX70v0p;9Rijn)*ExM*zyv`q=}=9lI`2C%p*oz*L_1*Uifc()>D;c z9V`SWd^r!0vNec8a$9Ju2c4nO9_=(#Thfjkdz}AO3}uUD7)mFmy=Q{9fx0ATkWM2w z*IHguxtn;GZ}wy+xSqWu&PbhYvk>r{1JCC1L!{o|h?!lqvCvudcboGJIa^zMzu@ra zQWLJl;N}iC1nKiIpcJD1khYLWGwXaAb_h^%6ST!0t8fc?mMRhSQxNsDn|&gkm6rIh^r#VwDs4l{+?hYy*!+X6rwhcC;ZwOuKv zXv2>)*jL)Ne17(SPJ}bQ!w34dfHEG(1W_BH$zivvjdv}MiUO476~xsGY|U*0-U{It zfMhEMQP6qI9${9}9ejbvl?*Lx0^}OdCGK#C{!u~L^o~mPWIs$&0-&K5&ObX(jyh1h zSk(4VN1_EUkelN+U(=T%qi0-v_kC*Mx&kvuK*};NkKo>s480PWp|ty4>LVz@?W?3AXFUdjftzmVT($I1((5x%&9E zJmQQd4hEO)v_q_?J(k)p=)Gl?ku@{fHd@(FC>GF=1bMEs@-_*vTaw1ASDALi$8~ir z->A-qULDsL`LME7kUe&Bb7N=v)$Da@53og;AK7OtNEIKVHE{!4r@69>@99@r)N;9U zk0xi~!pwxs$@e`IRBqvjJ7hd*QpLsB?E>F)i_O2hjGoeZKoQ5ue=z+{&Oycuu%%wBr| zEn}N3nyXgQZo$XJS%>2mKVX!-mIrqeY^^UG=k#t5JuWgBYe3yMKGI_0x7`;y2)jVR zT}I)s_O{B9%jZPtu&-aySkJY%{`=OVC*$^XSlB=&xJ0*gx`xAYy;vrxbaf_I(>No< zmKZ5oV%-hvgo{>JEzZ+Y?#zF{q^v~*cJxgMp&ywmw|A$&xd|N3rBFThKjv%1Ze((y zKZbGFw&h2U4OSQz^C&~RBvzqrbzQmwgp5-h%8oUc_(~CQV773mI>VX!V|$dx5cA6t ze}`}9TD_Eb9(yettHK=4E$j4aDfXwkm)E`d^nTa5SJ{Zm5=YtH?L7PE&}xz1IS&CD zdROk9#eE3|s*tlMbv(?Dy9$uYVGX z4;al#5`xXf)f43p0W#0GxKHb@2{d_jDC~gVv5@2IZ~gxiUru$tb&r(__R|9dns)DTLATSU{CMN zoA1FP4q=0c*(WbwO5YwAAyl|C`woiN{kh{lYbyH7HQlTbZMjQPyJEmoR?+JPQ&~kg zhETQ2uljfw;@%=`wH|M?r{MDs-pSz8WgM&Cjomi=TU@W+mmitaWAw50p z+j?K$#)}I3Ys)imN;g)a*1^r2Wu=6Te&4rXwJZ_lx<9ig(RT>cJ6OgXZmS^L;NeIv z;}G@~+@8_?Z9__{vd=@}ZjX4&p8y-V!vcW&e& zqAqNk`RzX=qUdX9x?u+V8YXRjo%6g~^OEEnWG9$jmCVfPsC7$0W|Bun4B*Jj+4&JA zbT&;)EjFp4gf&6&(tHSvjvC}V_6wn9F&J9`f>m1O7iNuN`fV|ir#^K6A+9Iy6s ztC7z-FHlG+(V8vjY5d&o$B5cF#)c|$yoi zJk)-N)!Y8z^koxD&;1K-R!x38G`LH~+UrPz6va2c`%avT5411Zp7`xD?g93$u30=y z42~A^bp1$Mw3{*epm!z!ViZU;Nh(f#SOmC{>1M$j*xh;cav6qHwJymuQY8uaj1Dlz z%ri*7iAr1;Uglvpc`U&#_%-sS+v&afC1E^evA;m=$needEDm(2P&1XmJ(a|!Ip5jx z2~|OLsFyaj)RbPuFwExA7VvT2H0!#p0>9y#dew3gVsv4q2@)F5>8FnNKI&ZLfS2Cpxktx{N(@PH84-NYC$)N2t159|?3A_C|NFj3? zj(i?FN1Z49M0;+)E>gmyU4L6-Q=5ira;;v{x?yuAo>_dd^Q2YJe z1ic(yIoq1wuZNAPH7~x>CK@3u>I=9VK2s*yt{CBasoSP%-{ZpB!LCqH&=f#JLGFWT z%p(%mNx#`g?=CHQncnK$qbcF8=3~ju{>l##x%GWn;G8cb-5@gn-t*?#T;KUi5H588 zf#N>~J6WZLRIPZC$CV%Fa zMny>pewV)YT$K^uM?tK8#h<#|N4|(=;7Gnjwo{9ZhBb~_m+eu1h%Z@n-y;Gr5~}EXuZ^;WN+o8 zUqAvAv-}$INy+B{UpnS-t>q#7C>q#3dSu+fwJIZ8) zx24A}w2c96kn`Zo759jd3FcZv%4m|%+NNF=PMaJN8@sI(>27VMy&_86<&E9*d_;!( z%2}vdq1_A}Pi9M?5Sz0%;lZs`OPH9gY)0#V;_KgSs}rCv))Guqik+baTQ`Ng_;~26 z)5V!1x*wG~U_QFv=e6-#4*niN?QaCb!p!A8Q)jpxY$k*M;D5?lx*DT6K~w7j5zXp4 zkK=^A`q=mvJw506Z^cZb6r*Xs~+wqF2OT`&Wp5b|ktDPZ)MjaiZcDdIo>-J;j`5Aw= zK-L`2Dv6QcJw^*aIAB5SxcRv{hosk>Zq-e7-K6egv(PWy#8 zf5=6^%J&Bz>}ajKYMH2e2y>r1ur|%a+h5sd3Z_faV-|mWN#>l!)jxLm8luN8lpfbW zv@8r#4=d$vgg8~w^z46&8Q8ObNK`)|)+~H!1T0DhiUJNEQp75la0JjmZf@j@zYDfa z&F#?ry1bNfNT+{V+fKWoTCa3vrKM&%En=bhnNy1RXp>{Oh_geJQc0uVBNAMl(lAoB z16a5YM~BvgJle>Z`K-%eii=dZf9+PP$0jRepx~4R-4mRb*fvrRqRSJs8`7sh!5wpr z4M>utHdc!ui(-T#_JNF>-jR*l}jL84L{|f*YJ6czw)y>KZNP009WLo%gm!mqK z`gq4&z4RjW*e(TF;~|e*lD@B!m#A+S8GTqo*!C>}ze?7Or^ln$+s40cH-3L%M6;MJ zPoA;7{z|N!6LW^tqtcB%p&y~F)yQSruxLO>BmLcWBQnvaGQ4MfL8fHWiOp7Tos%kJ z7f^!3whZ!tI`a4z2^|%ES|I zNe$oe*pxk(#3s1J2)b`s!ylG(!M3Dd;xjqPJF0oWmL(blmjDX2I56$UJRbL;fl$nj zX5kf(2nmBHpZlE(1ia9}Di4OmjJogrbbpXB`kBSfvSxG5T`;1sWOBZX&|qch87i&c zT4VgZ#N}YmV#+vEKV5CLp}OZhJur00aHUF<`1&nLFvy*u;^MX`V`un5;Mk6CpcmA= z+MVJOCLL?c_5eyu_93IdFTqNSwl%(!hFlh{E}*FF+@9L-B1VIk-x`*yH8P^XK$gLu zrx)%bkNtB`h63ToE=8|tN%%HQg=;I_`cgF`;BEVf`GC!?z5trgsQHkWOES@%x!!Lj z8(a_`vNRGD{=nlq499N~C8b)!)yQ{5GnE=g%8<37_jmwR3;4if)bkifYuK#YCCL5=0$Ny+Ms8xRkls(Mxzw$xw!(c4tm_h& zsV{TVME-R3b&k{N)$UZ(@XRV@nebV}d%U9O`EywmcTRTUAEC{;*ntlGNCY+&xXRFT@vweftwRGcD? z?siU&LbvG-!kIwBW1!M}*EN$TQ zGU6$x&PVkDa01Rt7CtEM}+FBa7vM9hRzI5&^>B)?RntUaVgUZ%-Jz zEVj7nesNc9k=FE{HA9u7SKIEAbH3~QSEf57Wla0^xv~EFBX_#1Lm|wFH2Xo;n^4Yn z4`N~{NWsR+e!OADZ%1~nnp1(P7}>j~Ip?~V_I1cWhKC!Lbq$toSOsSBo^5dPB`v6n z;3bTpuMGG?-o;5U7Y=zGBTvg;4c@T?y8~=4VnTpdZzKS{oQJ<3>GsJp>0aNj(@TUt z5cl(H$?gm6*@x5O)JLhxFNEJlleX4T-EO_U%8yfbYzLB(Jr4J49(Wh2oeL^yta16) ze=K5BnUyvXhueY7CvmGdwJ+VUO5t#+MXaW)=<7t%!D?^25S#3>wOyv5y*w5AkBS`B zx)6AqOS=^D)Nee=?*7KTE7Grgzw;yr8iVfS^!2=X^&{Y1*IK!p0Y}HMBA!ASu_%}B zBi>qHj>@rBI65L!?Do%Q%Y1+?scjVT;Gwb^%Zb@WRLrnJjF*zC95_~vgzLgN1`*D^ z$IIBGmFp72J4V3V77S7dz#!bM%$rlR4LRZT_o)YG*i)(9(x(k69nFGLG8C^2mr)&= zi~JW3N}jf)c$Fpdt_C4cNjtQhV0ho5>&9Bf&Jfc%goUZ!M`DW z->YV|TxQ-aGdC*qs^TM%7P5u<)f>A=asR0~RU?={S(#nptHS12K8H>zL4p_?fOB_; z?^&F~Nrmk=g}F{~;($UxQ2V91&y=5{0H4f}8ab>QcY#3#rK#1BOu*xwQVyimseq=Z zzG>$V2Gam1D{RZ3bok}gw$}qpSCDxI3|NkJgLepmGtE7$r8=&6i9lEq7gn?n4;+^k zQ23IV$xB2BEe>Yt1|90>O#9bLlUG)=lrGl2BsE9!C8mPA*gXb?^DRmD{q6S)ND0qg z_2x<(y1KI&uAHW(8Q^O>L*if0KZdBEmhPr2XJLb3x;>qRxjb%b^y{oCJ*PZ_&E`3% zO*Y(oRX0^5*wE|SZDpT0u`hs$Q1`*!Q@CdpVkm{pS#eM^QMX%4y~yLICRZTkPk~$; zGET+-!ygYWQ?CFQp7nsm%C&6%@?l24O4QN;d2c}?ZZ&L4q#VPYb*I$wMC^F;^OZ?zYl z;>IFa{XXYG5&FHuUUYJO<>x#z_DF`)xp5fkwJ*t`HI)PBy2{54b3IHoHdZn27{_4d zYD59I9BhJv$G$)Z84QwX2@#MbBe6a3+7!0J7zm^)JN>HFeX0K}ui>+^57fex=AhT) zYiPOUJE0^)IE9Idkd@D63%Y_IjBkHvL-Jv`zA$Wc#6J;KNfR9#4R3hQ{DQTgzhmYL zKfZ^wMo7T8a)|2HEB{0bU~0fKC&cjLo(^1tVhAY$z2`VHx!Ld8mV z?D6P8Ge^<72@U0qG<%QkGj+HCHD$ZCi4^=LFA%U5$v%16f9&;@W-ELZ7j1 zde1Yo$bDNcb@x)kNL|w|VGXK79wj;W&V@DnD2WCOU_>@`&fMRBPK+9wp5L*cN1ETA zYC&?TmL&|?e7C~EIDFJ%1&{tU&=^%Vk<-!Q0f>#q<729NdL~XtKVj5IIn8V&WLBHy zEgefh*UsXn_wf&v<|}JAdQrmCX8eIR?>j>+yuUtQR<4eKM z_0@B8yn}0j$rdU{eT3rgm0-d1F;r13#vaRH?e{KT3iS&2f%-Xg4%6H}?NDW$h^2|C z!L&^yMf;6yp`E>#jp0uvDo0y7Vni53|(}E>75$^H}Hps76{Mp)h$@kWkD6 z+l(cT#JCf739d(Vyhqnc*!EpP9=+ieauC`MqaM3(u@Bi};2;H=%jl$PDh&Z2))@Pa~i7EicWk3wk)PfIRcjW8_@s;T?*S)sF^coILbS;5%y z>mD#Jy{NX4QE`>$k-mCJ_*b^d!ybO-SoTBNu2`)nfDQQiGR3 z+rEm&*I{T@a`%ACW<;zf^`=(Dih!z|aliK7b~$4O?}s9j?QJBLtCqX_shkdQ{Zvc2 z=qtOF#-AD2mZq4v$Yi&b@5+)ZuZM&N<10R5SbYKKD$PY7jx+#}$0m{PavX_o`{&=~ z7OhSmm|3~^4NXGbPSiQe{-6hNC8$5%e77^iVQBPW+84KomSJsHHQ9i%&bF~;7F4+4 zIWdG`#Kh9;GU>T-XOEI42PRR_Q`Y}G*;zs=MoDHS!~>L~mF?PCwrwps;_=Uc%kwrz zqD}h`Ny36W-SJFjR;+?{qjb9ZNlXIbVW{^3*8YQPC_=`x0jDwQs?QV{=4>!er13`# z6?_@fl19!4S&V&j(jX8-2&#!n4rLhX778^Z@T339a_Rwe+whx>!5?;!pEm2-7kAS! z_B$6fG+~#g2x?9J1&a=~Y{z*DSRIs&%*Ed$wPk|zXjD?pD^P;Qq&BqP^Zjv1& zcIJwU(b`2%u|?Dxxn}=JfTSLoTz{e>0lg#KUwu8))m#TQ-*!sMDTIIK3Vbb4{4Klk z#e{{mdlu@$zftI8%UR-U*P1$z~r5an`;Cy7L3UFo)l-(4rlLCF713A*U` zO%cV$>FHYiYOmP0OxS7syqL`+W`bQw%v+MN6LVq+ezC(!ANjU{(wIUfcDK5O(F?I9 zD?ip2p{H?pQ{qrL2eWRHV*s8HS|*LN!Fu6$VBKhfx_)J-;cgi0_^|YuUvql99)D7O z(w4E{%~p|w@8wHBdY(3jh;UathF(Afe~*be*0-O;U&L?4N2)%^`32wJmH?@8#`5xD zc_PTrm3gFn3d`5t<>-IDEkv9mjHv$xZSTQwUWJ``iN zJK?luCr=y^#Ge9C7Um&0%`ZdhgK7AJQy5L-1WAn3(jf00$eA#zB!<~IsM4{WwX?Cu zvOY$&^J=ejE{&D;+lTjVYdI~Ayy0F+f>2#gIkpxGF|5|J5{m@ zL=hY}kunQZU_0?0<+t;5QpItGh8tMbZOXJ{uEi8t3-wsePh02K%C%u3yu1I%ur{+& zuVy(pev?F7`EM6kD$KdkKGnp^pnu+jDp8rsbdx9O;)lp{kNl;=%fG>NpZ7GfGU{w;a{u$a? zf*LzXjJG-c9*^bFw`P|^Kuf}u-8PbWk89lhNNKt#ze_uM*5r9|IB2%J#2vWuD*x8A zWyknIsY~xhX(AIVKZ~63Eh(t2isxI@#0zY8vR?bZrUG?dvsfNU#|e5a#RQyeTshC7 zB>YGKD#&HLIDpTr-j0}m?Gzz1xcMtaayM5_w9C0?DRskGGBf@KX)J%5FL$GR+uvS|HdU(S<}+pq zbn>Q8)omM()syV;BNYoYpU>lKifFruwl5NeUIrU|`YyRuSd*0}1{sCxg>PkuZ}#{H zJRd1D`Rp~$^HOYcLgXxUwr$30eBkS{k%!?OJ3^;9;W(~1u~nCSPt`Qylm)Ze4~Tr9 zx3MotOIyZ(uDc9sX0A=tPP?^s)iVx1?6Xspy%4Ys@Jt*Z(ilt+l|;Vf?qt-Rrm1MO zl_^uwblU#nOXo^t!6Do=$hCXxE6r2)4&cvQoiJW*iQR3^IOF?~JD5GcXuXRh&{TAC zc!$u5#ehGctMeU>J!?K=vwyKC{^QVHi@xRb%KVVe@L8nMmwDV}gSDt9E;Eck`}y%A zI6^8&An_w^L-04~{#M4&NpFNjU;SoMF`JF`{t^)*E&Av{FVAi0W`9M1xUY(iwbYny?0U*ov}2iiO5cRVhAO!*=fsiE(I&fjgKe{-ea-*4aWfFq9xlwR?q9oqZFgyt@Rxy zG)`x6XSF=s&3nu(w<<8=RkOYLB4}X_-ivg~|Hd1V3ydA@*JDyx8F{(72JQQ*&v&e` zMvqJkP|@7lVgD{CceuN9U1(^oHI)v`fOoU{d8;4_7uQI`2iBlp3^qx9zo}BR@4>_g z^Gw#q`!@bz1DTR^ZEe$X{Z31$*iyq!-m+7QE;H@nDSe&URad4h#T1)0n6XldKl)a64KBUVMsw0cpzbA0 z2r2M(wZ>RQev$B_Sl+Q%buU{$aY%6?rB(EK?NiWwYmvz+@{7!~#}O{AiNH9I-_d2V zU*P?;f7H!uIPTwONun-i-jXo00mes)G%>eoH<{?rU6SR{QU^Fle@uq1%4< zc0u2wc-0+~Zip#uK@@D3UB+CDy=xP$KnyVG8x=B4VlZ-~-At()z7A^|ieuC}Nx?o7 z+>Ik<{4c_%zlQ6A7JInNiM$&!WNVO@qQm}~p^lx(6GK6JxhucDq540y2q0IoGNh;6 zjRa@5_LO~St*ByfCG!}MG-uElR^GmT{oc)+H>-!_#j~CAwENpFA3OGww;MTe*O(zw zieI8dglV$hv07JrzoA(=VY?ZVc#n%-xUvG26~)g;|3CBFFHS`W@LG!5T4oatLQt0e ztf7!~_VY(Uj^m{t@B=SpV=#A`?nS|ma>UP<_O=$^$xV8XF*qtputZxu;Kqup+C6gO z6=llm(O*CAoKG+hd~7@X=-g%3e;$e6eR6HvNIH2(1?B610Zl z^k`ynv|yVxIPJeMhu2OVXlozz@-l6!))KrZ#&himwgI|G4!ZQjoMUmMEB7sVTCAVM z4Cb$ZpA=;mE_b|1=8_b^pR}u0iSgQ}r=|gYmh>YBv^5=$Xa(O;y5z7*mT6X^^Q*V5 z&p=?4)5~T296;IxW4cDU-oW%HARy|db_mdyNcfO{TAiU)F#R&~HSGz&w%n_KH2n_Z z$8YWZS548{SC)Tj&WB8X2+L4=dvhLv3FA8BMSv^cr7LtY%XsfKE&Hwo`<=0sd^{eq z{)wu6X+NlWPOy{n8B80m^54wVsg4l<7TUcsm1qnhYg2;j!^qhQyIQLD*Kr6>BzJ3{ zd!EbeuSzH$^Ltbdu|5(T9@2aJ@wy+NN^!W{jbaw7Hha!UrO~aWn01ka%iq&q9DBuX z=w#+cThDfZL?3G~NK`CcRRbTitRj|a9V2RycaxQ`X*-B|pLiI0Wdae}>C!$(=ueyO zJ!Hd4ZA61jMOjBQf^U2zQM5&5YS$Xt73W>!67#?_D_-eC_BMS-qcWqf&0d0%#D*F) z2?8{}`H%rEg-lYxa@nvq$9(e?dnWKYs}={LE^)4T336@Dy@c|o6!m_XBWFw{I&W-E zc9z`-uQmyJR0Yvwn;?q^=M#WBwJtX|L8%^=5kBM0?|1i+`tgr;MRa%NpO0{^nHR5F z^X~#o&XnHf8A|k8_Xay?s74lVKbCZiYBc{qUjOlAdA-9%y%_foZ|`9;g;)X5?kHow zJ9!#Xsh9(DIEfV;n-pnNFJZAk4x^+O^65j6v)Yq%;{q1|_&=f0@qr^V<7z^vbE61F zA5`(Fb7MPQ@?<32!*XPFx5oiE>^gr#PujBOk(UWR)*%LW%Qv+~ zvwU@bzv;11*A{uBGkL7Z6DKIzil%$(y7|Nw3JUvSXG=^?FF1MVSIHX zqc^`Nb=xj3@)>DW#c?*ohvT}*DTKl&f5R~SwCHnH%?>EbvZ*vvYDRl{Mnpa?dr8KG zJw7}PIeSvQXQpl-c9Glu89!}%f_9S@&-}FI{>v&d!+vI$IDBOOoVzKa{T3m|nAk^Q z6y6tgSu7IiV!jfkc~_5N5r|Y~o|l+9HG?)2bd?h&<~^+3=*K?9tdy}IPgqXuA7!SL zZ;Mh*6V;H?r zw+^dni~2^72m*qN3W5THN=XY+BCr(!rIl_JC8bN6ZK1T%jUwIMp(r6KNJw`|ci+j8 zjT>QeK;Qe__dcKh@GSOPYs@jn_>Guj#tIX$vC2vma|-g+S$ng@+f|}Vl*Lgsmw57W zOS)Qn*_~%n;SBwYACt_4-LO@bCCzL4RMIKmvIv#ywwi!sf-K?-NEBhFs%D1G0d<0n>g-b6_&$3w0(WJK|*-S!m zu3membUasyhr(IOpl4DzBPCu+IKQ_4NVRkF>gtH2k8grWn@TpqF`l}ALChMGdiM=% zMCC0s0y5XYzWIO@!(OnuBjC)3k_S9!YR}%XjwYP!&$xi{LHIss0Tr?_ja%j zc~@F5+KpNyrHJ>goi?QEO&V3a9&VRAqMu;PSFmJXE z-_+5m0#EcI2S?>uJ?gGw!}pjRjTO@$tTa@zueI8oxIWCH8k9dTGA3(Ps+OiLYf<~X zU)YA1HorDJUS~0Pa7~TVV)-Tc3dPD%i^^%Ws}nbC6q=h(T4h&Wj$1(heOrh%(x9#; z!ksezKo>A#SamC+y&Rt_2}M_W9)RQ&_(QPXCC(Cw^v9Hl^)`{ z*i2+O;pev-3m39faxQ&PqhmLiF2~kib}1c@;K3|@*U@&VbV^1{qox*_NTjp6b*N7MEgiTS@TVmw&{~XazHmLGznQ) zHdRyOxc!uc6Oqc2Tdp%Zf+dfD1gp66boCL0E2VzK;_A!L=^I%h&(B-32tH>&no>wO zmrgd?a)}wt&>xMRUvxcHGd!d2pOM<2UySB{aeVULH@Yyn>Gtx=eCFoefw?XzEW7tq z9K@-}^JxgCV~<07?7}(fTS~=67rH*zc`a)-net&yIGa#_V3;x+#&Ek* zCbchP`hJd~dBv*_D(qZb%xT#TY|h6iB!blyn9Jtu^VV_A8G$2@e7Zoo3My3;2XZ&kru3CuKT9Ji*aNF=eEF>kFI z4*r3F$+!LC6IDY@#XZp3Vhci3M}#fVO9V3++i4>N$Gg1rFtQ5I~P<5 zrfFvptOwn|1RiugS25HiaDa6(g;Z_sirTqiXF|5F^cuJGE`bjmS4OP#r>Of`1%t7n zI8?Nc6tno++qK5A^_G(riwO}<9ia)XFUMll*s@jeeQqX)%t>uBUyCcAn#}#m3eqOM{oK?XmcfNw`2IsRo%0wRSH%U zJcFICHk{3*gQbs9$(lQlsj*k-BRyGX=tH*yX)K+csQ*{3?|2R_V<4j+SoaeMDl`8; z2a$Qhnl%vS&p$kn@A>i8X5IFw&u+rjS`;59ay5lT5-e7ai`djh(pmJDrrRvdHjj=N z>%^<50jQKmD_MQEp}9V6^Uk(A#wMBO`f_!B#c=!}Q`5Er{;c90h!uFg|2Sbl5V>uWijv*oWK4BRo3XI|8 z*NU)A9Yut5E;%|0RT`h|KkhB0+`FJvy_h)j8sEn&@s8E4r>y_~qH*hE+Fe)ilPkxe#@SxV>RD zYYt0o_nTr8CZR`#8cM}41jW9cpJDTFnsUc>%$aH{<%s_VT<({t8>omKB_o<+WkS2o zfB6mKQH(WG>La>BB5~|@dIV}S>4dHoxhn})eH@l&H5ebtR6Bi4aiV7qkG%QK zN}jeKGa3u$%dU!G(0S>}m-ugSlw6%n)lI|@mN zClnD830<0{cMzBw(@gL7kz>)GpQdlPJL-sXZz=AXRKu#9*1lMXfFN4SjY5H)aG%A$n`mk_OUBdQBs&+ylTEvhsUTUQ}3Ccf}Qk`hC67}0P9Q!p5>@#dl3zYqmc zMe2`{HEgMoTrR|tjdnOsjc;Uj>8ZlG=SME3#J7O~Z~7sQya}{&V$m3RtJ06lV}luY z`vcmSd*yU8AFN?^HwhtXAIsYdM2nmZe}sdf^%@B%3H?Zmtrl+{W=o^J^(H|K_a2r? zqgz~i)6u)FX+m20u5IDzipZy=fcbj0yjpz%J=uR*3$vN~;o29^ieX*B{&4v?Tts?@ zDAR)k?TP+ng!{Aq^^ix7KPqOYeNzSF(hK3>4aXeASSLH6wRi5R@D+dQ`& z!!R2QFJWib%qa}c$Kg3lbBx-L=*>Zki55T1G$^gzxIsJ}vmP1c9<7tmY<^-6V8T9bjHB;I_Y{Bve&J`P2Ic-{XH7RjNHbnuuxGVrE zNt9WUq}g~u83O#s!N->|_eucP(K9r}J&gmiRIuYxt6gkuRk$?ccHYOm3w3BA%2R@4 zEQxk7_$l)O6FIFx0Am@};#SeZo`zE`)C>T!(ytt4L%Pr~P!?iNof?9vaj%phGDz2* zAC#VfjZ4{h{6I15O-Zp6uB*KBPEOa`|Bj%G%K>R>f6 ze()_3>eG$m`vy`908&VE{G$!vK~uUajo1x*Bs1c)Yj$ZXDG6kq}WWoN6i>CkpT z?Uq}YC4o^T#!9hn{Hh!U{EqnX=KU&%STqi3Co_*1HfZ^J15%Jk=9C08`yD{Pg^^8w z-zO#R@s?o6fekGO_y8dyriRT2_Igq90az!ljyf_7FHXSmXti7wfU&q&LXZ(aNUQxi z3haeN94x)>QkBO077{%kfWmv$GqA(UMHzv=@MS7DfaMQ+rRW+8r0uCK0sDeUFc?m# z&=GDJ!_I<*algeGDNg`5k6YCw2m{$+G&EMpV06zNEO|S$xBS8d5)UIPSSx41n!ZMF zNhRQ86E?|8%m-K^$c`f0ZwTuFPSsn0ybc!CS-}w^6Z9w8x2R-`P!Rwk_zrNY z9-4NaVB$UmG9gxqTivk#_e>FD1}OD@I|DnrkDdhd?c966eWz5&eHV#)SVic=VSYXe zY?PKBS(V=Z2XY7~iF+PJJfDLpP8MXa^;uS@MD`*YxM!e;fZmuB24N)#18}5pd1AEY zeqAqQ6v!ykRC=Sl3ONXzEq?WV()|u#hy{T8Q#VIZ{J>2x8wjZtgQ@)cw*DYs{ni{3 z$A1`lH6Bn|QHtyv1Ym)a$X3r#A`UYA+9Jx0rEet)kb+BdN23uQ#f!*bgs|psl}xw^ zQAjw!R3;BZWH|Wl%ARq!Vcpdap#fzdNPeD<=~|R?^i+6j~{N^ zNV4JmzupF&b5=-J-EZeEO975DbEVjRD96dd`c+dD#66#YPF5r4}2^RT;5KrTQK zgXp6(h)@I=RZc+q;PhnX@|g|%1~UMdduTj+hOiTem@U8tJY6JahPe?o5a;^73Z?Fs z5HS&;V`(0FVnf>nDS)Heq(+3Brx7;54a7}wJ}(l+kpqZM8PYWqB7_*QvjF;F2858v zuib@aqM@-=@|3b|+<;;NAH%5r_5@57zrKLTF}S|8)l(sJ5{C_Klxaaf4R0Il8=YwG zG{h~^X$cdse&T?`6l0{hsW%6NyXb$*ts_8S50z^@VC7&eJ)>iL77VPP1@tIS@$DjY zVe=RTSRmoGd!ehjZ|ju@*4xQqJNctuP*vbhn6uZS_M12tBmj3}N2&`uRSv=fG)7f} zR))y^AkrY@2_S_3^ri@B~X2Bkqm5lD;N{{h2anr*}rta5Rby1u2W0&P2NO^yGENVlQeymm#4O z1}Wo{6hmps!-y6eBa+xZ(D#S^tvku%UR+XSYrLJ!ds63nR{|{WB(6TLe)rcN?SWO& z!)Pp?FSOJ*00jvE02^-M(AqCBK;8N&esn8QG^{FxDFLcf5zA!zCX@!8=(jYd z9t>ZO7JZ>WC>KO}9(M=PSnO2WW_hEW7-ZB39b5NJsoD#;7o~@^uq^Yk2#7Ui98!pO z8za=i=Mz{u)B^Pxu)bjeX)`{Idc6llZDE=o8Tj>c!G5nduq~GzZ{4pM;`#z(GdFqV!1mf#ENTRO1ISQONjJB#3J%V7AZ z2S0q=<=w5ffbZ6U_uZl_fTb+sa&wp^?v*8aOTbd+T*2W@DgAc28wX2AUR=iwhRI2g zYm}#_Q^KrfuMwgk7?>L>jpkLD-97nUoc>oAmCpimn<_A%gjrF+cMv+0 z@!9rWHy6SM0dJ8o80Ntt=ztJu@EI6Jx9VmWzD}c~v8a9(428)eiiBP^^pi9Oj6;NA z{ibJ>yp;Ry=0TkKTNx8VADj+KFm3J~ zEjS1IYNx!*hDby%;TG{Td@pK+p;9NELz~Rq0m>BUWr9`kw&WGN2o713Y`j*l``?0R zvWFZkOkj`-5Tv#0>C5&Tav&N29HC#%Y$oR&dccATg}(1M(}PTbXz$lxxOvXlb87l4 zJqtqn9sVCc;y(3)2Yk%4w1zQF$!f6j9Nnks-6XPCGuX3;{}P0uYR_f}v*W*_`zyL% zUG!gUz?}YHZ2ZN>|7i~y+=RrF>ZSH$)+<;BNtbmkipt?_##9pnx;WW7wk$$bI1KR0)!gA(*Vl3HOUz&AX{L`kH!*xs=-ln&nWz!zi z7H<6>$D^CgVbE05?4R{b!J*B>q9-LZoy|Bo`&918Y>9?|uHlfXDUEsa$g5h(AFnF^0># zuiq5Z5L5E||Bjd-E^K&!RxYD#RH=65)o%n_Y1e{5T5r9ExR-D8(9+s>ea_GA z%O#2`F`(Yadn4Z1lC#z)Xn4ZPREYA_-IA;!T2OJBJ1UEK75RUsSAY>)l_P$trMors zANCPE3bJtHvEZ5g4uas=gPsA)H2z#5^s;kEF+Y}!^^oX&W&AK{M^qO1_F0`l%a@6# zkH#dVs2{p(%I7X#n0RVof?Fpoh$8tW9Y_88ErIIh*UD7%F>PSVmXzk2uE68if&_Cp zD4BZ-axM*e8dBkqHj1xk&57wWlE%ZB(f^Y#F;T;!5F``~L8w~KI;;;&p<(D(4P^um z@SV$eV#U?>t@EkqQ>tS+4GR`gq%FzWl{lCtBj)VVRg_L)=@LUl%D@RcnTe@UHuiyrc~h;9O1Hu^o#!u8{qNa^ab;Q*ftZ4o@|fW+(0p!g*N@oN{%>pa6}P9%;O(j zA3*zP5ubG)yOaOSo|Q<93$3+L>0JA^C>3Yb{sA=-ZQx&rj@RvhBUAa<^EUP;4;zaL z3$B#lYw{cZ`?74oNdo*<>ra2;0z20$uEnh}d**aIW`e%}pg8H|ckQ>-24^4zdZ27! z*R3P@2Pty3Ixs)pbft+u=!-Hd=u4qK3)Oae*z+&yDImPTt#sRB5PP&pdIHdw&I~_< z$LDP`lI`yAGFUsB_+>=u>Yh2@P881>EGW}!so>wP_C5as7HoFFFW7`r`<_LE36_!R zvyQ6nx2gr-fD>4`_|RwxXa9w$pMZJ})0e)rv&}T)hrQGO3 zKSiU5@r>6qzK`U(u>Y2;iFg7SB+R=1?+PYYC&+DVscA~pC6?<0mNVuGv$KuE35fTZ z?jjhdELw_YBEJJ{NzHaIyN6$rz&u-dRDk_{E9vmgYX%EUjPV7QUEmF(B`B|vAblJ>o18ctuoy}BN* zcM)~~cpQuOJ>cXF*TkN>Moa~QeN0m%-XA1CU@z0}O~Jx(2$4hzk&y`5l??|H`qRyw z^;V|O30>OH=B2KU>W%dryendP_2V?x_GR_Zp~Kj3(D+W3eW-FKAXLojdDfZoARaSb z&io$U#jEULSqjMvuwFyuK|nGbiP>${1^@9Em!4ORt$=TI4^(ubj-U$o6z;I6r5D|3 z6&h9C)uz`EwFsQgx_giBk>c>T&Mr0o(`~aU4R&uSKKr$l!0F5gjF?RH8+S7a`s9Mb zk>U?_L`_($ch}}rIZKbjSl>h~v7>LtfZ+aJ=Tu3m7w3C;sQlJk8eS)QnHwyRNTrJ7oeZL=ci`UXN( z=Y<5<&4^7v#Df z!_-i%-JVv9^AxDvkQjVwyy;YXz?v^_6{)&ic%;XIa!gD>9aXSkqTA#Yx_EfrF(NNF z*#LG6I<&stHlGjCrv}%1Mo$98>vC0PhHYMh<&2UV7uqGhRU01c`(;WskQ--4hu_ z9+?*tTh;u&=Qb>HE+zLsx)tMQt_JDSCP`M!?PqB6Q@Q#Rvr*jxrRFC>9ard8e#S!VS^J%fvKN%SP1qZyQ{(=J*RQ`qP zUvT^d2g0E77aV`V@fRF>N@D(J!b*W6pbY@$sg8|QCj zxc9L{^~zn}60WCg!DpaBmTQ?83ZJh`u(;%5J3a2EzYz30`3V~`Hi2P}I>we@^6Nlv zhb$5M;&?0(Ced(fa^l*Anlp1@rR*>r9^R~))#h24V4weEmCSG6h!0_ew@SE#s}94m zf4+M3UV8g#*vl0yYV-R}RaTqBM$tC~!5@5_HvdOJWY?TgoHn>a+RV0SdnIBIUEdfT zGJ!n1-rLt_m)gV*a@!nQu3tcUL00I2st!9{Xa|HsOVKZX5ghG`+umQ%XSSv{BtG81 z3DX^vALPqaM=Be4>AM?lV>Pw#D%t3Ev5BStdOXD{tef;cuhTnLJ--{hlZ{u>%?xl& zAHd$)R3wxVd344(!rmWqb6cQjIDkYzrq3=r^py`IvE!x*b<*l=Zb8~)RWzx9yY09K zfy=i|$x?#cJmxB-dzb2#ps?~-S6rz=i^q0VMPZEt4!z+xj07BpzCZ1^gDy6DraXz1 z-7xZk=82G7{www^3jOyb?2Fa8_(N^S**2L(!}4~tecXLtUSXFxR$^#@JmU*)Q%)b{ zlbdHF{3KyX03B@=opUaM6HCE;B&vt8iug4)WUM=LFhip_6p9g!Up8tpUpTJ232czB(6Us14nC>^T7(fCPmh z@AVDRcEIzdnq6;n#Jw+?U#v;ycfK{5vW;nFZ-|u6_cj#cv8P)oO9IMCd2ySEer*!m zya1F}7c9CwvV&agaMQ@YmF%!;f(U1zZN+HJHp}$dQ8-MK(b01)$YmVdg!lFfC;?Kd z$jDkz7~Q9dXx)#A0_0sSP8e*bl40kWZHisF2qg`Z`O_QrAjgBlb2n-)GsfMu`q^0R zNl`ujz|`Hw&Lmg^dGAL$2wgt*H`p+@<;>5E)0BC!vq@I`bYG z>8o8$%b==f$|HT~?NhbgU_YN2XO51!Dv01nAx3JwTm`3wxSX@`ikzo$o68uf?;8J% z2tXisJi$wMS@UH$<~kc}2uCLTt7Zjfuu$HdOmtUM?)t|wgs0*e0$YJQmS#P|2ZVPWe zJZ$X=Ji%e!Q3O3iMq4(6uE{QFTP`efQL?O>oy`~6;_lKfREN-b0}nh zzw0q7-yC0T^{j&^S^1e0&oJ7ZM|HpHZG9iAk8V`u(>kR(*D?qf3}CH`MuDIZS?Y}- z+SOpl>q>P~;pT_8cynrCz|R{66G7v88aYf&s)Du$+XG0Xz`c5U+aowERGAD}F?-T{ zUG9vD(WG0FQ$Tn$GudKwM?u^Mg7_$2j-aGH_%DEmdff)+Qi!)W$?s19yMk2ulES-F>+|u@*C@VWW*Uu_wxcea|&FCBp3AU&ZByG*i8k3YrrpSN830c#Teo`13o$rud`uwS`t6$Z>^ckRI@ zqYz4voQUho!Vae0+gqEj|9kQE9H zv#t~Ha6sTt5%mee3mm>BB}_*z3jNN05D=gAL#0X;7+Q4AUh46NjGx9De74 z)B8SS5l%MuvB8{ycT~WiNc1l_;Fb6n9Pr7`U(c~66aEX1|2x59j|)mc)YiNQw+!eY zXAe}pF-uB8-+~b^A|-YMo;~dH$n~8gWF6YW)6^1-(bu+&0)wpqwHY<_ZJ$W|kC11N zyI!B_QoBsNg^ix$;{?9s0)ATu$lwtMF!!U%UrP%YMc|$hbt<^vz#i@M%f}r=^9w=( zxrdK^IrQsm(5E1`K<7?9v8DA*itcm?bA2qHv`qMAL|>n7d08|UGPk_Y*cCT4mjEAH ze>?=?1cBbtrwMi}+%I_oC6;sp^+!-1lcJA2zEzGU(9s1$1xBT%Gkx2qe{Zg@TBH}J z+Brc3gC7y?N9WV3+U1H9whjoPlqgyKm7@!_aMN=zu7hs?UQdW!La_L+{AqzDBd(Id z;&o~!e6@Q@0$Wg2U<3$P{BALb@q%7eOW6F9P!%OxrC`@7_&tpUM#xzPDswsCI81js zeA%56q(PbteLH7#P@m)T(;rqQKKJE2jZDo|mRD|}{STN!G^+AyHB_C;*|Fr7RA56MXb4cr0|dNSy!uC}Fl_v$0f!jqTU z%bfHQDimIfPdCStW2$|pC!Ld){LvNH3N?* zr7Yc;hI}*QUI#D48p`hbsgXh29u;@gfe+WTNW~%hoi>Zuo@Z->1(W&`qAa zuW|C%8$Y^#^L#>_%nPgGwV}dz5-ORkrL`SXH$8`f4$t-p{4LV>#RPVI5&jt{I>@($ z@fv%#*vBuL*tApx7f?Bn>FP2~u>T1w1y2DHa?J0Oe?%k}C^_ACo^WR&!$&3rr~_S8 z{(v-fGB2-eL)kXxtJ6lxY$jdOm#eC*$Ywbbm}c82q3wGk_KPSs5?dazz!UTH_`umdE?>#J>r=1sM3XB z&P0hLt*!gfuc6IZtSr2{{Ii9=$&97EhF3`LLfQ6E=_(TZl8pJXL!z;c;dlAQQx>?U zpGJ+@@5N0RnoWgjUqEQu1?s@~L)h5hrf{H_&{MSWi{tZ-7t;p3Fl3>1+%Y(aWbY|dY7=J%na(^Y8TYxer7fULc!Y2 z>jL`hluO!C^sD2Cnj>euR+*!Xva1s{&CM8|v`OP3eb*z}brep_Pp zzjFI}p*mM!sO=*B;;dgAm;`aLud4C+Po~UIuA>+%=rPaE$KQb{Sn(w`)JC%#QG<1& zl?^6vL~$NXH8tyx7Clo`(=*&I+ePFoQ~uU$(qAUV;JJn9fXuRGq3P$h|Gzrk&gLWY9L`%DBL>kv5$jRy3yDnhF^AtC?>T2JH0$igi1uV1t|LhPsgI zg(|E{+f)+mkg9gHzP3UAg((;@0Eh*Ucuc81gv;1+(Nn(6n&`4@-xwMp2gH{(#R&V_ z$o8*03*OCR+>La z8-~E{pqLBfVm~wCeKG-*BM!V)WLL_*KL86Sk_fOeE793saC67UPGRRiL~;jIpu_BI z8U8?`!AC*DaK(V(PfiZ{3l4bQ{RPLCSpL5T90it9+SL1Z96acj6A*TzK+~$CQleE# z>&w$5M4PVT9auFWcse>1&aU_5Ohlg-bUx84|58+{I&4T6ukY!ySy?`D81uyFebOy5 z+!M{87~Fm+1=JVpX3;#`{(ZZr;5hs!{p*E1x$R?hn6D2b0hv6hdiu}847~#qzV9a) z`R2;)^i;6Mbu=(v&~7hT!5G}W$2wRQ8f)mMg@C`7;nq7Yi#kFGhY#M)Ag?y^Z-spU z7-Cx>co-DLnB6Sow(pjjnVuAQ*^Ae*VIR8%Aw&kdF;|E0;pvCIpXx@o5Aes)f3v1xSV4TWb>V(IW5)L4 z1;h$$VGA8?-K_#eoNl|uM`4DJhvtmL|@T zV8Vd6X+{)M!Bi|JW3GVhofe~SfExM@WD5RH{@Oo7bg(IOsekOH)TlM*S;fY-CF(<) zG&e6O1cEKpzUHzIwVm6o%B9MuY;7W$dJ!(8v%)aqvRcFU%fp7N#ZGg)UfOP((-a90 z5-zSQnm{_A+&64c|3v!sI?N_d1MA7gG2^+-<#ojLg$7Jp85@oQd=K}EPxFC zFF0Th;4e5}l30h!{~mCZdvU$pbR6Bddzk=KK7%B5Iq{8*qXpQQ%#G1}5A4`q&m_v3 ztjWGzUT)oGatW4=0LK7b1x3-NssiL6hFE}&(v;@ZE!F-)#Y^~O=9P*1$ZKH*H_9{qG$SOZ$aOdyAL&Y32p@s_1%@LcMOsj0=oizz5s)!NyisLk3K zGq05SQopl8GW^cU3vgegmVzR6^0S^mJ12Zkt8I;u%#KE2ia)96=A#*XF+2pN(^FRloSsX1~f zYQ)-|EjWvWMr`}f-u{T*LqcZtY(#&+w(*Xutg^#W*kYTO_Rwk`xfsjaLz`{?1GS>) zSV1d3mcA_9TE8S^+<9O|^DX0DqRQuQj{jiqC!>h3pv#9@7e)fxV6Q+wU_DA(Bb`|g z;v&sRq!&g_{z)qV?Ge}K&enYMxx|Ni@02Yh@-SA1bd;K>@wp#@C^&n7&aj(BDO*I8r5e7`2fX%}S1=%(;qbj*eGR$y4ZwMS=KZ3DQ(DbN3C4csXsm8_j60g-*D)VFf zxfkO|n_>(0M$RGT8?&y@j(@<7klgkg{2;wAzQZ08gStWLIa*7~ilVViyBqiC#%n)c zSZ`oSW-QP1$Yz^AZz8&)U^;+AATZ*Xbatc9jTf+3;xwDhn**&B;ou`0gVbgfYid*W ze2<7)7q~mvXyhDf#4LlcfkMpV8M^)5njzU{H&3&}DH1!x)>LV@wLNJ=PkU-36a;jY zivdFQk2fCYMjmaN7VZ*RhlTH#O!uzSn;>6QVZiykK_-|4`Rt&a(urZ^ z<8UlLLUJ*%DSNh80|U>Iu9JNkAnN&KgI^02MaTRsQ#&a*ve=mg7%lCm7Ea$^g!>gv z5j!>LN&zUpMO)fvVt#FFRs8f0jJxa{sG|?1sGQn2y|afI{aHCf{9VA4MWya0?lt%ut$zk zjx>DsR4y2k9{S0}VJTpqA2?XbG56l(pKoJPm_xO6CY=sds!n5vSh9cX66FKj9q z`;D#GQ$y9yd&k1&tG{AKvd6I0 ztKU-4x?7{T1J%>+Mq!wlTM%kjPtA+Y?KQ|}{<`i;LeFExDVXHkff}yVcWbgVwYH_z z#}8u(e1e??aKso58?5(n{$vp5P5!%mAtev4`YKyDPa{B;Dm;X)H_iR_=mBJY-U13M z$B>!+rGoQ5h{6y2i#xO&td0zVtFPFn8?Y>4hlln^`~MDcJ?3E) zBeUOA{6y?i+!Ji|TkQlPHto7Ei|TA9qQj3u@0|cz^{zZ+$g@Xz@2pzrYn%b+xD2IJ z{VOWm%7SRDD)&67PyD`CH(NOjHu4ugzCtJnm#F?GS!g0C=yo@k}|W6;Ozatjtl zb^CK`P=1Jp)xpYK$ryd(k94pumSFG|0zK-8+V}V2g3+ch_xBqfK8P&NPE+NSzip+& zHb_@&6vo^F9XQ(FxmNO>s~}av6-!j2d!oRvLGz@6Nj6@f*S!A?T7u$m!munYK_hzx=DyOxZ zOI>b~dw~&%`|`V~t=Q@-+ZsD671h(RY8SOtOXVLZ33;VH^h&Cb^{L>El0jq-YzZa4t|%Pcof(d zOQm(D@A=~)>JWdG``@2@)beFmBn~Zy8aPdtWzdzTCf`=nwH!X9KX!+`;z*)&vxAwh zSY=jhq>TAfbQIxNGdW6eGwG%6kp{7XuZ#sfu7#D%@^ktV^^Fx6)R`5dt|*S zgN9LajD?~bdT(&hHOx#I#B$M%^;%f8whj2IPxrp%grcyTEwrd^41}Q9kJHmB8oz46 zR=Darl{phx%WIf%n{YUYd9^vhxmX#_RrSbqcxp)XSfDNP?P>%R$Dy_D)^hJ-dV1m= zmlJ2$<5tQ`*e_zea`Rf9Jg-vz356lxxDYpz7wzhly@-t(Nx)MrKWp*>VoaXkY??F{ zn}Qm$2gY?>VWbDN^ZcTq$TSM4ht(G}?9It@GA2|eE!7I;roNsOOA(OGp%NWZZSbzt zskMl;=k30Q?|knKa+eh`c1_T0+W^vo*RFyyej3IvOolB#uq^iM8yy{ONE*1;c&^R? zKd(o9<y3~_v zn0{w8)@mgyJ3P4;dVJbrx?1mD`*mg7$5pLOY~ut>+J@7h^^XCPR=SDc@;G zK!&5vNUR#CW^_geP|4}?PmTLC`El8N`)L32{e#!WV{RwDPUS(J`LLL&$&y1|)(nb- zvZ&>o=jBX`J#C2egF{S6sDk;;0!gBq-n}RG@G!P}N|}OtgJk%Y)}<3%gDAC!v_2R+ zr5AfxbVkjlv7^rUj1CVqJx@U9wXz{;c+TQUFeYRfH_aras7P@jzhQcAblyZnWC>%i zlGJPDL+1m#NYM^0hGdb?hqF7go|grhEq8hv57*E18u)+A z>*G6*>dD^Q?l!@7&g+(ptQ7w}a_%|qnkqhb1I`nez7ATN_}Sw5t0rn{Q{3Es5AxOg z=T~J>@8sd;1g2i%ZaX&U$Xw5bkz3`cypSQ#+oXd{M^2dhk$IBy2>&-9gLxkVLM1%K z7NctqA50OUrn^Df7|aZp{JAd3T>Ewi~TV{G|862@2mZR2BG;66S6Vv@O zWr<#3E+3aaR_SnKRsfGvH``%oNcCp$P?ZG5{L`3WjLf4tg8muEr_a9em>QPht1`>E zQuFBEJ*&7`Elod8xlZw`v|5WP$o?L^QSGfOH}efH4kjFqL?vNOjg5*M9d9)b9Zz@X z#K_fpZ=qXynq8_=NVsdT=L)Cnd7*~-uS^QPR%9L@*xIGEN=nhcNx7-X|Gor?rI0W# z9MKKBV$~Ov8basUNJ=LrHOdm(FQC^pR@k}t#ISv#?JVkJH%&il>f_R7q-a#}?+s22 z7{0jCZ!F;2VpE1WNf`Z>R5@9pvWk5!jyYMXD1wl^-O=>Efjh1do;Z~KyGBp$=3 zyEvfsnH*(p2<32p)>>kXQeOV$BKr%&zn$hl&!38j)A2uJFirCsXPkRM@%hJ(1RiV~4&2q#U5;o&(C71!pKrM>2^pB^yrA{v!|Cz5t zs!N)4shgC<;a(izV_h}kPJiWE66lK!xqG0`Jdie$w0 zIx^NcPK0br%EvE#_w#w#6^y?bfAQ>JX~Nc0c+mS=13KAZN9r+qYqFO~to{;D&2f=S z{MJW4MR>V0RGkhD^zs|&cZ?2fuo`WOi<7*lwx5zDEIou;9xi`*<1+!D>8+vsdM01Qcsw)Fs)ovlU@=?xNM1saa{AALE?s z`DQCfl892pfrT0q42!3{yg_2&gz55iF@mAEWi+Y-m+`7WISYA#ehF!CmAgx??>U~A z?6NC7_Go&7Mt6yzmyy9oa(JxWI^3*3ASU`94t(#YGq>_}gU!-FoY5XxawdaiXwc|+rQCj#_qeJSMv`&e!LQz9Q zM;j=|EzFqB9;Z$OAL8{E^> zwcn=i-MT*0FU?mbQmYZ5gMSip)eFxnyqQx5?C>cPqT@0ZOUioxK+*Q=jqt8hm)NZJ zGDlLU*B(pu@v3~i(i`Q~b-*-a`naf0hcaKLUCGR(&fOUId5?n+t&V)v$_<*%jJi@Y z|0PbnOhJQCMFZRUE1fIb%1xxZG&FOPWlWQA<@@JuiuM#T1d;ciUZCO4lp`FO&TbNP zXA&}S_fbvprQ#!`YbcA!ArG9R+Z3I5=XT8;ez4y4}<5pz4jGh{?iP&>11{l|BzD5o+hh})xr=ipb-{8cUVMoG= zemYw&_^6j-DaO*dOuz3|CT-wc9O{B?P#mFgge7O5T-=vr?Vwo2mKQn4Bp*$jdpL-3 zfBKV;b$ZIW&nikQSiad)U=LXcog|f{Qxfd`X2~+L=;D`57+MwQZPcN_bdoqZCi5~9k5># zb4(^5Ps&PL<&b6PdR36>5~|MrPP*PW^L}DBla$Q&clpVmW@64gxu19;+;E{ORclGx z?rgWa(zydd8mKIvC2CsR#}u1tzYUSrPqy||2H^YZfAz0D$iL<%FE?e0FVI_%+q-qh7$Yrch^37wsQ}j48P9x)KYtc92)2y9sKpaN>5G<(0H_#U!_Nt!}Jq;Cz%I= z%cyBumYpA-;k-e2!~M);k~nSikkTg8IXoT~L==qK7QY-fbn-!c7|!H|UNy_+By z8Bn@ec(9k3n$IuryZTJ;Md5{S4w`2?V}~B*lJgPP9{G*pqKYi!y3id|7Rp*2)9!Xz z9EH_{BN`RHy>jm=JSNA&L1swyz<mechK?mWx6C9CWb+D;K!Zq1~jlZ;>bEY2rl$>y?Tg$O_D+ zi2Hq`_92gFXo@jZnER4(IOpu0J_@tICA?z&HaeM=AyNu$uCakf`I^FYWuudN?@G%O z=6hS0@@$;Ut0PvYXx(+Lw2bk&GLDFIz!?>zMnmv+!-zH}Uaz+}Zhtc)lX*FVUq*pf~H4#d&2d`*mXO^2>U2yTb zaPZnnzi-oX8!ygz%&JqbyI}!x9|frK2pC*+bq zQdi`U5Xho>!|L!_@|}#dLMv+6(rHkwMD?<<`wHO#wD}F%wQjbN zCqFpgla5n!`T2)f_xhG~H>VREm-{w3_L_#FFCc#SElrgvkK9^yYSN5Uz(_@qRqF_c zV2#9Tba{5MC!yYLg;vawh-st_$CtT7I_wwSI*M+*jC1ZgRf}um64U$YW7bD2bc2{U z#g_4fkCR4oR8+%9^&KZ4Ct0%il(jFJn+E!i098)mzx{F~aaNU%btWIxE8H>`6^TGh zGCazagr`3|s##2@yx|jju3>h6@EbZwKDqj0(;UT-(??=Y=aKght1vMMjJ$tWc#OvT z^}QwPmiCKl&KX4aQ7<3Uc9}aHq@&l}v3k~?*VM!$MRBO8BC>0$spE>G8aYHy&_st| zy*`9_eEiN+y8!#6=#b@)uJ%{IW4^;}s3c`db!w>aV{sA8F{2NIpT63g+Mt(c5;)q% z0?&YQz|2i-$<*}L$d*UaNN*ftiO@?XkP5zEEWTs(uB_ap=2p#lu>~153xV!+KPhJ9 zA(@DbveGIM#u=NcF%z$0>}z&Gs=noGW>W8PYin!XHQsdIka*KnmbGx?7}1XkKi-e>-txW;MPP|9w-q*B9}}NG>J!Up&LWl`PTE^P%ar zjCGEEn0wyMS}b)Yj-|ETCJlK(hrdqMG!y-hNb5=#+S;i)wY$dJh3viDm!1Amo@xlt zovoO9tjlL0ob{H*bY@P#HH0DF@g0#f=@M?kdU1~a-T}H2z4|X>-(M@?MxNGJ^DzpZ zS-x1Sx=P~4t?uwU^}AK1xR*)^QOPrB*CYq)iX>_Cxn22Z7?O+X*u|1B7!Af; z7Y;a38GLPpuqdW+n9=6c)Q2NqZq3u^W>h4W5T?$ZFgK5D9T**d9Qu`aZDIyg4WJmy z%_Q+mpA!!~;wSFtZT*5XM?Q9LsfJ0*Q}>I`vcTZ9G68wWFsZb@EiM*YTfS~>w^ERd zwpzb^u|l_Xv57k}U687lSj5bySs99YsiP$BlLtM5#nC$z63k~(+0gmj9TRMvs2VJk zZoQUHdx%of;T=m*yq-E{Ba?~vnfcsUpey<3hIP!1m0LQ?))!dP+F#)h)0GWV^)v`v z=~12jg1PvLsH!>$qk-9))=lO2#2tw&Xeh~&3$feblLoh&(VX|`W3!S#}i8E zvI_@FS%=W+BptS&8kR%CqqJx)!N18)(_DacW-GY8R?3#?5t%%yVLhv80lVU%2+%?u(_X?U>zYuu0Cpqs*J< zMUyk`zG=}|7I+L>mWs+%nE=8HA}8a^O2jPFxMfZ=9+Qh&lUfwUiL-s* zibl9(bPQxepXA*QAfcPTCDEGnc16OjN8s&mOrt@+;10MH+jtM(wlHr}a^;MDDN4{= zS#s|o-6Op>Bdlh93tG`3G{f2>NfBPs?6vpUD=#euJ=A81G;N)K;FBT}K63h13HD2F z_J;cUjL}aGO8NJaV-?b7P9QhZpd0H&Jzq5rLtrTDBqW6W{7T-75=nLuvJUl!H4>F>$b}bZzx^W$zFQhUZz= z#<1!+htXtF_IhGXwu{k*0W-#PW-YTtWw^JlH`tksO|dXSl~pS=oYu`eK`uDSp!9O@ zfa+L2nu%BFB2|8ke(txsmBT}tW)~&=T6?o=s@oGI!aDmWsi;S4mO8VCU$1y5okqJt znQ%{$#Gt21-AF5m+5LCMPPROdB&0ZdTAS(`%H|>Vo`P@IsQ97neNsU&aqC{=YPsdj zq^#Trs#DSU4NA$TT<2vOU`jEv~ z`0`g($}Qu<%RRy*prvHq9(~RcMzvKSbm#jC@a~>`O^f4{!jr{#s3SmD{5JolnYeh# zPBi*+mG3Z|r|qiSGe25hHOeWrrfP-QsrWQuNetNPbUM>QFLb2wniN^JE7jzo72UXT z9PhHA18lnh0;vBnR3vI<33hA2Gq?^XtpV5W$LfWc`>u;0Vs6C)-qEa>QaU&{{etn7 z19fb-p6Tf5?74TS81o2YY*}D8W|QvszRO!D)4|tgDfn5vAuR&@;1PYL z4vc#e-Ix7o<)C~6yJEUO|KT!2wT9>or{h)bwS2#>v-oX!WJj^~Q#v)xgpsfMRj7Iw zlcmVR^waN!VpQR`Sri6<8?4axkWGEGR=|+=Y}%`bK~y*jWmy{I+xi-UkzX0umvD2xeew0KK&)})u&jhsP8?;-)Q4T^esZfPER zSrtm9mx+mbB$*MW;uzMD6n<2G?w1qi3BJq_uoC4D31>rGo6uV>Nh=3|+F6QYY@5c? zRcZ=NIB=1;4qa4=i&|W6;xEBQ2|&X<=IJ+jvn;-Uoa{STO*5M7Tdh@&;S=LAkiZtQ)Q(6oTK@Q4 z44=v>D|7&#Bxfe83tH(7xbBo>uQ7-%eGoO<0YR6!Kl^rlZii~U&kCh{)zn3qR)U^6 zjm~!En69OFC%jGoqJ3?995(pPIrV}z#KNwQ2h^^i9>i4wBHc|8)yCM0w#~8e=7&;Y zB%s?1N#6=P=&f#JBT;r3;(KvJH zDOkH85VF(En=A0H4pk@Q=GA719=@MZChtoyt{zlsF%P>B)hbz`E~Jt#bua_`)=3fJMI~dq>i`kE2D>TKDev z1M4+Z^%p!p=l6>JzkV-_w9CC#jhwrC$Mk+C=91jXW}9Q>Q~0#AgO)ZzOiCu~)3YP1 zw0A>dW#jUTr}Wc1YH-5bsA0Aq*DFl)PrY}i)je|-MpNs`L;j1_W|7yKCTErFACEBNOV{-bF;{;oR(+vf z&p=l5eHy|4I|}P#bDj*AD#b{j#Rg&mEI#W>m~w zzQtKc4*C@6oDJxo9HlRvb#C*XJ}tN^GJ7`~8Mc+$=$H_mV;FDDhtbpJ5&f3G25+p* zp&QrS#g7qVuGt~(MKnpsK(=mMXPVB>`|OFNRp#!2$7UG&)nKDSki>cZxwV~e&->q9 z`PsTJtp|wQR8M-pMQ0X=HHzXFK5^b^sJp;|Ejqy-GOmGCVzO#ACfbLhwN6jfQglu4 zLnXbQc5)%h@$3hrK-U>QXlq}wxhJ?rUT2W9DqJw#btXDKN$(xAX6*`28{M+LPUe+F zfh(TV)0y(f@qQ=N2>M%eEBQOI+s;Omy>4|amYqa3ukSZ|GqIdhCd#^@P zcP`(W*T0h=q3|$R`pdPh{xMo;oxI=S762ft$vgRAisdd;EjW9c%HACm<%*K;SkV6) zcGdAQSmJW}_@&MOk)Fza7&!>*1iUPKRL;{P8SN&78ne~X)6o%B04Y6UP*A(qZ{8GA zt^bz3NeChoPYhpO2wHvh5Tl~IvX?;a+xe!GAvf9`JiPkY@>qkv8hdhmgIw{e@TQ`F z-h#DJNr{~?4QYy0w-(|yQafrnRCn*w-6o}b!?;ax<#zf`n9%I&DuWsNIUx3et`O&8 z@}ZMz%Jxh4*mqqy;T8k-@R+^mn!Q=#wl%F&7Cf!;T$E#()^AUUgR#)aOUkPMReK8Mm>)fbChsnQTg=*_e0PFYJ>09ub z+s6Qn10BJM_v3$7TJ4E@`#t!sKBE3VzUwYsz68kJgDNOAGXm~m8j8Rju`@VOql z!&SMRsKd3-Fv|I^#E9Ua_sXo#hM?M4JgimPrq$ImSXw+>q5^0cnZG<70S_H2-B^`Y zcs9jL7Nk(o4ginSvxHf_8{Rds&#!wx4n2a@YYIBZP4#;#Hj_mnI=X<*opX=yHe za(ZOaFDoc5Zma{}s^D8)Np^Qpeor;@WgSn?aPOKLsf>CGLCkaMfvH!(B}kw&`(DV^twt`sR=Zs@;*nBUkq0TCNOqLjDBkv#6P#Cn zYigFX^+SxCZu{>2R@bsCWnLtGX~MiBzVIkTUx5tnp-tOvBl}dn>XTBIva4QKXPst= z56Rh+%Zuf`GLBTblJSKXW?98IgD)?;I2&=REXLSzp;%%(IWaU3i($MZ#e(bS1dm?jKs8qBwl`9J*y1zHI zXHAraw?Dm9SEd9ZwY5N2@?dEKG6sg^qRjQ6=3v{d(Z3`dd9(SObuGUohiJW>EERMv z(#`4}{K^tj@41EM0(>lj7n}|ltBHJ$wvw)T#-6#Q;Xa<_S~)l`B1I|sZ5j_4_N~TK zUenj)lfB3vJ@u*y%g?F1Hn7bvTN}B(H%Xq^nXs}%CXl}TQB6@gHM3*Z@PKY6_SeT5 ziECZHj30atx8RBfB1IE( zuQxV@ym5#>`{AeuA(nM5j4y1{av%Q3_U)W-7l={WcYaX)sk93ocJ)UNlGpxf=fT@8 z{@EfbVeZ0fV5tWSOj!v*C<-U=XxJy|Ew4IesSUc7w0(}HQ2py4Jgd(9M#2yR#^dRn z?zO$c0WZf)xuRjJmcF0Q_!Q`ng||FAzT1(|l$LytB)H8qcH(0!ipdr9rQ5-igI#a5 zPXNmcAm=%$@D6dWD?<0;Eh*%FFb>{GjPaMiv#T189>(2j(rC!$#uZ2TO*n)2XGLtM zf4ZN+mn$Ik^$Yjwj>JdG)KCut?69|Q%*>B>=;|r2bwd*uON95T`i-3Y?p0NHt2CXb zH8Eic{OJwvtHTM|K;n=p>aM>~_dCFsPHHSSJu3fmvTD^c@9gDuJ^;+?*qcZ?&o-W2 zg&44DoKi5K&qx2r${i#(5nd`uyznBIDgz$s?w47am1UoTWg{>?>V$MbtPx*}OuhZ{ zJK#pDM{g+@mz)Ngm$pI14PHA)NCJ01e|UV~Kyw>Q5n_({8^ScjEc0zJv2l=r*3|Lj zwoRXg9T`n!Hw-owa%Q~7el_s~Rsc$RP4`Ntk6K3~19GMeBNGEFM^dkfV+#W|^tvd9 zL$#DqqWIw#s(j+3ZG}l(@t7cG2IlhIQgu-h+6)8j*2v`WEsV;MjUm3Lq}vqKcR!JUsP@VFB;JrHy(55Ra#Xw zC=>_pfmuj*UGEB;9RaFhI2eU9We?Mv;^X_?O2swtD!44o{E?G~uuzP=bBuiB zJhrFy#3``94u@~GW*KB^e8oNVxFV@z4&PNijN4f9KpB8^$=)v=rxOR*BP#Jo!WN}| z-9u-xuS+UIgmTeNtzR0cuenrZfZzMg1(&>sZYL(UPS?4=s04}aj#DO&i=-;WAJ`PD zukX45ux~GBYLiinbI?kir10iFdhIu(a@1pE{rVTIPa#Vyx2Wkmh7_2kUBQ&&x&{DW zR{o5`=_d5rHuvV7cx1aObU4u{^I!H+W&)e>YI`Dr+xYPuSg0P501A)BD+_W z4r&I+(t6L9=o73SF)&3!J#yE-zWG?}K>v{6fW)aUAYQtI(J(E1rHlCI62Z79)3(+? z+aWpjCkT4}~l$P^M7215Ma40xjr|6hrZ+geT z3s|H}T-cBGBM9yy1*4!>p6G+g%$t_UmL?C<+-9Z}Y6jJwqJ z6KUz%q6e7M&f^G3;`yEv?nB^Wx~e=EcE#8iPxDdl);{f3&U-8qePXu);qNh~cK1G0 zd6j*j3{r!^-q^Z5xEi!0R^#2B-V`)vRO@2#%H=F4#-8myLCmTLiH5IQ1!AJ7@uw*P z+iBVG?rkV^)b_#Wv0_+ul;jUMYdFO+Iy0_qx8tQA9-@jEk?@Ffe;8L0q1w)U7}mYs zEmDW_m1zb$gLw-)DSsd|CFY&Ll7u}J4BL-SZgw-DIOV1*cGG#c{<@{LZTA<>M&!wr z*df-VTEh-S+WB=))>}u2L$)#k5PaRw$i@a`{WhvnII+jKXFZ3?#geTce=5(m3fKc< zO+rwPwV2MnYbF4xsyCT*G_oZv4O0`gv);#PeCCn(Y#In0Q5y?8&3m0DdGx2wfM$ts z0Uv$HH zp^ML?#i!smUnDxsxGSYpdPXv{xx7;tf-Jb8XLTCM=w!Mdt}~tBr63OGqc^(W3m*js z%v}R9#)nhRi(FO1@TsEi9xfz%xnvVjdYzm!Bq2SZ=j|PWczEDfRQ786J$3XP^SX0p zX%lq>gm4v(N86kC&|7-?&XB7oHRloGPRX9f=}F6Z+n>L$yfah6?Ug`EgVgY2nz^e< zYx~tF0bi(Jrnud1VVvqiI%o;WYiMw+v9}+7giH+ir2>X2FP&ADs_<;Br5i5MXAoFrH)bZ+(%lmS^Tyswn~XAk%JB~jzN?4Bg7A?vqlHS^?PAzsB+ z__xLHS3q>vw(Fc+Xv@M$XcWAbSc1i@^!q;(GTyky*`RiVgCjM`6e;TqZ5(#1BQNQ! z-TX1pv#NAhEfPVH@_Nf?@yXdEm?Puz7oeWYqRucv=MU~}&kO3Vr!5cpST>kR$Wqkj zkKDF}51&-?&8+BFVs}aMjAwtiv1$Tj`wtc~V5LUmNks~fo+c|>#SW#lro>`Cr7ui) z<^F|ZO`Fqg zqNk)z>FsVK9g_6dq6Lo%L;)3W@}_XoYty?hHK0l4ekU9pUI+C;6b@9Mt?GU}}!y#6TR0^-am>c5GK5S&<{H^j{1HSICxlM{y zSTwYm$m2Zlb_W0o28~-I$KwVw4cwtly&}uwI@|Ym%E!{g=sh|G`5TF2eH= z=3iyo*3xeIEk88l$XH!5^d_ty60#89HCz*^jTx3ud#9(LUbaG4ZR;|+8a-s3;o7e| z-X;ZIEmw_$a()nB}OdvUJ;!auGaMqw79Jj}T^ zrmv+Pmuw7~B1?T_PfjveX)!`MuG_?yUwL8b5LjSkm_B|N&=e?-y5_AK^|B{7(mkMo zZxzK;y+k!9M3Ru}lS!GV4{q47asksH%nli+FNsV6mb(4l{o~|STYDveHYnmTsc2kq zJzgEY31PfZEQf{_Q!kYj0c^qpP|>Gb&V4Br=Kl;wmqNCQmODMPL+|BZzpkuP`9feU zVhN;mQJ4aMmOegrKAz-G&Hf)Q=AU0?r%%oLvJm}^=~AsB{$hl#XEO2wkiFn zENa3|E>pN^8RDrnI`@Vordx!M3h>&Fa{y8W&^U;Vzg=;L+p6}{v$mdj&`@!*68IJ0 z(Ge4MsGSu`OrcA32S$24vo-`)H@JDFPWtfdkJ7zz8_@c9W21nwjGUH?_xbqA94&2(<|gYF z5Abc15{u1hJ}^+;X2d{q&%_F1Ccas5^Ve=xC^Des(TZq>d+s@21Y1`SZ@zf69Csf| zj(??~?`Da9+W06}f|-nZ0u24Dcv^J+rYoAjHwJC>edIPL z>s6lE@*u0UwRyjdBf3d7#`(ty@F9Gz&nEy86Zf=hsWC_-{Tx3m9ZBt`&}Yxef0_1m zO1S6;lEo4%UPJ(bXiPEoQfHlgx<&wcOBL)nGgdl@v{=c0cueu`H|V4s1+)>p zjm6L0ra3}_F({P>`Wi^~!j_3L7070KMTA*ma;V=PF_>7WQ{#azpEX?h(c}}ePs$(E zl++hTjD*vUoSW+q{n|@m&*N?Lis&^V7-zZ*sAkQ7)q+~DcvmR=C~q)myZfrznGzY=p|`%Q+$ zNApfW*;BN_8*7wSCUHe6#-yD*MRupZiUwf*n$@&+zMeS;R;aFLHaXa+_9gCx70bAs zr{j)qzXv5Ipfa6NPl7Bu>9K3_%t1kryY=;kX2VX?y`&orw582pyPggFe55QvH83bp zS~ACewLi@kOL?3z{Y`P^Ny!_Z5bA&wq;Ag|JK#Mb;IVM%Qi)b%jMG)QI=iE$-6m8| zv>~X>znGBXH=O9-zG4n0>XTmOpE9?5hH={ZSu^VzMg4N4q0UetvHHifSG&fvU0Mc_ zLZIFKM_)@9u5Dc~RW;mv<%5^@`^gLweB}?>u@8Jb!DcSm-X0QfhDlwf%f}@60;(92 zL;(IBmI?7QWs6K<}&FSRr*3SmXB_ev(7mQHF%yO_y-mHi$l=`x`Y42F_ zai!kphK4D3#Oug_8tGpLuZs_%3U1IwWL8ZCwRC4Qf%BCoQhBkLLj)eiu^+a3LP|N8 z>Z#3wupj%OUXw6yh~rjbV{8qIkypm5vdH6Av{~BE`Wnd(%6hL+zXG4R;>B2^nCG9O zz{K4(y|gbnHBCfu(S2yA$ohnFp#GByK{ zuDPybW!qZ6;@>T?1;S>Xthn(qi{|^T_u~Og@(dpY3zRtZUMp@J85le%Ri$u}S4q8b%6^9&HW>`d!1UMV|zYyRzr zp+UI-a@0x~FC0!@?497bOBrn@wOz|amd@Yw5>19?nuR zXuOS%m)p94v$8Mb$Pj_mzoB)OzDkHkqdRj@bW`a?W2ygq3*Z4LJBSZohBrzWmGmCh zFuP|4NvG@N_P5?fYdz zV=uyAJDZWngT1Q(juG${#aG#`Mh$VW&aeIB-NYS{Kx`MUaJmRD<>VkvK_#N|!(XIKU_vX9a3ScQ#%cHl9)NyNu(kw**XSpC@y{Pd9;Au$9ASc{9v|fuC zr(dq`=?!{+2ad9k{Owr%ryWP>5JV84@@~Pa{l-fk&7lMx8p!EKN4cIEzJ*CTiyjEEoqZhRx(E> zYtx%AZER-0W4iT%O>D)a&JV|<|INd#PHX6qf!vKMkjgtSp|7uH9z*m;4eI@eCoB`Y zP0$cb^e(izTce4SK(7Xc%=MfPmYxg`m?AIpu+Hdjiwp<9*Y0{52DU1%+)AQaMKjrp z37|Y9_&hStCP`~`D+JKEuOIrJ?dHX4@P#{!y4r?Ui!Wu_jrvLat@!%+^OgFqfEsxp z_zReVJ)(3|9-9L5vJ;}Yc9Zl<6ZzZ39IjQ~XBnkI3YAlV@av3oJ6@Hcms^&bJHsYet1&Q_P4?E^~@OUBs2&$r7Rujg^N02Vdyv# zc}?g&U!2#ZU!21Nm>rKeT+~MsaP$~{=$j%0#LXL$97=vKWAyaGzhbc?`)cO(4`##z z-=!n$4&X{8`%(r z;!2a~j$z-PpSiK{9*QxqserfNYj>J#wjO5t-FDeovKz_L$bTyPzp91*dz3Qd>+L75 zp5^$a`Mtn^Yr4)@b#$a70Dv$x&muZ41%E3)nS^tU8(RUi?o;f3l=Y8xXikE8RGB{Y zg!fHR@fd%Sg}(BS`D3&~^v(f=ZbKnEYx@pO4bwbLL|N!3lD4Sz3$GySXnlB zmfMhxvT129=#7@fK{L!^y}PK4oZSXgT{=|Gl%O(mf{|MeZ|^=JnTG8_qhB?d1YeSC z(9o#)MF60<1LlQlU|EQ|tCBl>kiKl0$pZs-aj^v_Qp|}rW`x6bx z&gN6Yw|yo3IpgIXnf$KoIZ)ZOQYh+Y^}t4%^F)XHqFr>&2eGZ51a{_v zHTx}&DxMKVtSaGPvnU%SdnHtD__K{`n}Jr|pr5?vV!xVZhPoAG$9U53#d8apJv_g( zNptaD{YYvKj4hPoxwpRo>4<$+PF&I}9^)yGIxlU*kOcl3VQn*9s6xG>!7?82Cl)g! zk)tct@&enu^7LF9buk&CpjK z<1yi&m!N|tU6ya+`jxc@9(F&1pku zlDVs*kZF8^$(qn2*$l3WCoyN0Bj`slh5{r{N& zgqQmTERy5o9J2g0O0n3vQKmytVzjI|*R|~c_ao{TxAW4O2OF8TYD|4gW+PeZju};b zN6I7V&vQo{TV(Lbsl~Ff9)I8hZ}YZWn7{F*^V68Cv&p9D`7bq@-P6{7$~(hY7WD9m z)1tpssdJ5rp+wARNNc|ZiQnd)@Z9Exqsk+8u;w2a@#bMr;!?{)SEdc(H;j`SFzcTFEd~qPT$H_+y^~llobJW3M}Dr~7K3?p#|5#SVW-e6{}l1~ zY874Gy9Q`ewwq_rm(kkY`pBBM8aI?Kx%m-Pc3;imvJAm%=5#2328N|)_EcR@9IX0G z`W~+#7V27qdAez4c|`kZ4D1%WJ!2}U7S*Q>tQxlYvTYk^vX>djJb&222cK~v$KvU5 zKh>Z_Q%!S7yN%id4GNcI$x{|e+^a8A%y(M?EVj)I>XutGGaZEU?Bh!sgQoK4LOJrz z`otv-u+k~n71v!nqDIzF5RY!*I>y9<(%iQ1+_l`1$Tfew|C;w*Htu4+A~^pfDH6Oj z4RkLQBmmXD%;)`AgZ@_# zqAlg=l4#;->M9b)QSljJj}-A{fR~D^(s?GXy{C0C*PxU$`DVVc+5Soq$^V3!lH6o z)&o)^KMPNbh@e;nRs?vt9vxnMtQB(JobQ*>Ec|x&w2e?kPz&A)P=laaV^EyQyw#Sf}IYp;LVS!YOi9XAE%Nur2K+3X*1CIb^U)!E*e zU!RHXF1zQ^HB2n2{t3aGea{FlpPXU@m?o7->P$w&$=4~@vFczUht)GXJ8DL7X5E;& zBk(o#I~ZAh5wux_8?Q4iCfFD|Q=ZvJOn^5452Fi2X>w)gUa{E{Z) zFHS2R*nK(Y6-w`YA(VS&Ho}+Ve#o;~oOYL#GyPd>#ftA{&GlG$NQg?tTy;fhK_$#L zg@(l(0xM{B^Iex}YE_lFIcRXWcI0+llGk9NkubNzH};{p`D0vsUM@qGZG3AX{soM2 zyfj88j8~|=#!QGseZg8+IiC~@|t0fPrQNgg8Z*`B}I(lSTc@SJ1 ziw$G2oQ$(H(}4Ix+|t`}c;`eCgA;?{m?+T+}| z7nrP(`KY*5RIH`F!|uhP(_+ikPd$=x5V@@6^wrrN068dC(-hSm)S~=I$a{#tLua>< zR9MiZ)00oY(tvKm&iL)mQzLtEiGM4JZ{w4=QWCvz{&k}Gop`%`NTx={I8g9B&E zCu}mQ2Ql*KCTir0J@eXv^Xr4z{R>r8Wkx5S-CxXq5IQ_gZ-96}5JQ>ln=a7_{CFa~ zG%ss;UMm2ZrH_P~@a6?o_RH|EXV~DStmk7CTfDJ?P_|o-;1LE-_n!RtkoaVEVY~Ycn~JV zXJ09}={a7jBqp9(Mpw-zXYu)>F%(}~eMi~qG545U*y=-wRWbx+T9S<-yc=zJIX{dc z-Kx_lQizpHCs3j2NN2fTHc)sKndxUzwc_g>TbhlW<) zONMyT5>W+@vstsuOJM}`se@;qU+ziK94#hYFIn5OU&&I?(7dK6de<&!Qb(63?znq% z37C0!rD`~G7m2?@C$22KXN4NBKK920PbgldF_bvSQzHAgLnGt%yFaNde>Rf;>65g~ zwd0plnJ(T%x0el0aB=gQB?7qIN*dZFQ0?r&EdCjQz*JG*j5?c<4zj+jDX0R!Xnj(b z`x0zfbTW$XjOHkqxgF)=Xx&>AK5lI`j92C9cJHJ3L~Ye<=al^96u*9?O0shm9&!#QxeDbvCM}Wj0rs|LP}jT7 z{X9Vpc@1$SGY#Vq^_jwJq!!P1oye;&YA`Or0+0aF)S$+{vu1yd(&%F2r%j~Wn`>LeS2QS-u8V6 zsm^^Kv&Sbp0md`cOBM+EIX}efGo5Oc3cp?o>o2M)zW=z6Cl3j_CCfODMLgxri^PPU zYGsJ*iXhG_vUQ}7=;a>0UHKEE|6h&&^R+8@z|u6qE~uVg6PIn?ifWcH(##mKrm372 zTn#s|!3+`;e_Qd0BHVD)d!9qKCihSeY~n zd;A++ycAbQMGp&RMBP;S2HxnJhN`Vn1~iag~d>wg{J-m+WD# zI!1e@zcyO7aNOuT@Iyg^HdS;K~v}0Kk^*I{}mPxtR3RL>gfDlgtK2^YAY|lGY%<}BuQ2Ys$ zekx!9)chX@TK{#Q$k1Rb_W7mGs#3DqJ83}B2mzeWo3RT@ z->`_yOBGm&LGvrIH?i~MN^e0&SL#>?5(KxCJo72HIYP{Y{kyY=Bg!VWog!5!z&LG9 zV-Wq^2{b=q$MYlx5{a#Vc%`P^xKN)L*pzwv=dD9YN&RzGrqkll5d0%* z%Yg@t8t5zC)9Cc{qcq{#sqyK%(Rt+R9<0f@!P$(~Zi9q~MM*OqZ#Kp*S^XcNYmXNd zjinjcNGqY}y>mky)6S%B#6Dr7@hr5XLkLf3nUpqLJZ^Qf@Z>9Z{7AvFQ~z4H+g)VbVy74;XnLs zPdO+tz~#f_`NmexvEO)0I%r|i13j};*tHg>-HL2;gRX?@an&t<@l*5O_(XOtxl$rQ zw^(0MwIBC%XvbZ0DaxG{L^tB0sY{;EoULcex1Dqj^KV%e1)j9$2uMJ47Jmt{uh#HI zw)FKZB&`4zsFjj<=)hm|8QaT(BmwZM7VaulAAgQZ5`1?mw6D>dC9R6so&4z6{xF#W zLs+40fc&M1b8T`j6Wh9g9gsu=Z_U&=r&2COVfo3Ggc+Mz_qOYEWjV02oUa3TQ_j;G zHf@wFU82}H*sL1pIXPESTaI$oLeH9ISi5=}I*1WGnbp32@G)13YOa1Q4hGC0*7!2kD5+^+6D&pOg_-@W7W0$}^TJ|Z}7yr|WcX3w{;$Du;Kr2faTg`4;K z$1DilQ~BsnH@z(2#2jr+iO;JyfgCzN_oj9zhc$W%OYQL-QuYCrv5A>Hd7TLpBTM~> zKxsr`T(jDkXPD|1_5`;JX^RnqRKnv%H+5%!#6H?QH#aA` zw;Ay8Y)4@3dW?4|Z8Aah^Wck%rj!4yd4@X>G`nGzzW?3j&B!^$Q+KW3SqE59)2f=E zjLph$qaIf9#)Kt0J|E8m3_^@fCPq~ArWEqi^7TwShv1_glc3H>4Xr$7JIS+s%aa|h zFjo~5iy6yy?m=yF>`puA-M$s-a3$dd6f$|><9fAl-_=pbrtkf8N4pvQ+2hd@#az5X zDXUU=H{m1SoL#X$sPd$)MFuj*%`5I3rmnd;{O zH@@&`&DYXL5_Zl=hmG_CRvZCJ2y{n}T+cda_(yJ=-3)KZ3RKf>i?4FcELm0lERuS6 zXJLzdBdH8qe5OPks(s&p{p~jLqX81jsYI=Pz7lF`@+dINFI4DS28UcgK4E81Ijqx5 zJU|&|ui)>`2Uawh=@>0ss6`?ZkIyrVOOZm=Qd2{&3vHfsnskH|57b5LG+ZrTI3_4u z_+aaOQtk6w!98m$!bp@mo9OjLNt*w8CzJFa_k~{@^5o@}9`W;Lpt|a9hOewmnRso* zuaRuczR_=2RK!V{INM{|&f(=|YWbnM_jm}OQN7^^$9jujKFJj&T?xbb?hm(*uBkd{ zya^18vwlJ(6xv@Wr&MsDFQp%`fi|nvb2!*4;gmmwovjm70Vs zJ()e&Qq1+-DAOsN?66+w8(_O>9;IPa`_jzh6PatCNh z(Bj7;KGA3-JMo)qf}P1L)Z{{UWA4SWQy+CIjK#Fj$JWujZ@qczbavG&)Cf8w{?f*h z?vJGt=IdnIlER!>Svr=T?PU-3YKVc}2&WbXJNBBcjypp$MOD8KwC6>^ zm;MVTbkYg}wKp&vq^d$0GOWwF5j#)^`})&V2gu}bptK4%vXC|J`whkRiW(h^*92di zcTSnq_TFU2^V1QIxs<^hW+`Qnt=Dd?e_Z+I(}s9tRh{^cwNaaF>Zr4TlXc;+*Q;|< zMOj74|Lg9Xzod{S-v}q8v@Q7_+sC||CNZaF&@V_|lh409#4Pxnu8Hare#ie$<8KAY*QN?_4)SVaUt9cf-&E~75Jl2)J-2T+f zKes|OAX!;qqe2pI%kogjr4-3P^!vqiS2MLM`ZuhN)Mkh=Ln>)~G+W8_cYzO5M-PbnOtmT~@#Ob?5F>eim@IHr9G?gXbKD zIS?P4a&WFzt2Uyrw<;XHyUJB7(Z7(bv&!f9WBG-u{0?XF&*MTnO(8U315=3}+^faD zD4Wg?@nHQWP3Z0D|1HGEq39p(`iA6`fzc3SA@sZU`%5dt$NAbMc-K!y)q@MAWs z=kLuISQlhojtH`PkOq#oH1c(5e9C6?Qb>{stuJM<*6j3IQ6qO`ZW?xC0BlNX=HldlnTPpYdR{Z?uPW z-|9UWE{WCQbRlndTc~=PmVgz^c1z62)PknZzCOTU1)Cd*!oIdznyB4qw@bPGhn>H< zMv#?FQLe)J=Z?64rj`HJKnMtDu7P_OHnlTHcfP+#a0=zZ^i|pI88ru>+KU?$MvABK z0S;$Sq=ycl9^f-=^4NGDblOZ@Fin^8?AWz9glIV8Lxx?w)L@z$hv3g3Ec$0U%>XL+7;{0{R%$Gx-GbILtK zPdj=@-aMJaG0G@x@*`LHAI?sm9y~Y{QYl3FyPiL4X6`=i;n`ydNfqtA?qClv5c1p{ zwI}|nqf+j_6gBBSs3NG2tH5Aq%sOer8+sNZ8|F^Xz64i5kLJZ6Bsz#n3Z_jw^*ifC zr)7Hv9NaDpc6~i7Qf}NV!kd@BlTjW39Fyb|N0d|!&j_E}&d}a-{UM3B?$>^px9q; zD6ij)=YPpu_)^T}mVCQ>P<0b4yRFQ|&-VDp_WdoWC>aEhCWy-rU1)eZnC?3;bdk*+ zlwJGY71kaIuv-5VPg`a%H()dkn)igXDToyFsix_G3;P zSm!fVq%&3DKZLo88TD@<*r}_kT4w`PQcHkIEpF@p=N(@kC6nON;Znna%7ccoJ2Pj9 zr>hh8O~c^_CU!un5~}Rz-@Wy}uA_UBP9&CS>|@Cxc{W&;cVoN);_+>);l%@g?@f0- zv8c`6n@g7y#V)DSO+eq;ca+`v9Xo+o-j&Umj0d8kvu;JjHmSY&!yLeM+IGlW*zJ z_t4(0mJ zd!t0j&gFo{z?W0$sI}a^LHDDFy?o-l?LQkuc78Vv{51}SEHK}!32RX$PMevF7tA#* zveX<}q}|vG>L(2i;;;&hBy0bHXn$T?a40r|d&%J@A&&q34^~Q>sCYXdZjXaJCO?tw z?sNypzINOxc%)%VQ;G`HwA14192~F8InGP-Xtfy6@`PdRMF4zKmQy8HvK034-XWJ) zR6@hs_|;><*&$*_MR~=lM;oQH+3YODcQ;ODz4=+m<{)Y^pX2aFiQUGhS4la0{tVYbk${>G8U&b`Rojn2 z4Oc6VhhLdF;A_E*!eD1Vr+~z!YG37}XaV>d(9zEB;KY6D%on7?Ha`i?*oh3bt zf#GATgHN=TYb_Q#o5X`gb#B(SrInfd)`~~B;N!R(X5H(M(Pc2t%D z5cSn;4N+K?H9ac{G=`y*3UT?L#!d~6|)@FDGny=G|UZCn69wy%}fz;pDkI< z+pj!cbu6?t*oY3WcX#))$Gh{59Ku1)ktu++UisM@s>tHX(iMb(5^CSgQLIS)gIsQn z83$e8fZ~|(ZToKQb7j#E-u$m+y{0}hII|`1VTqPs@vaqIrmkZ;)TKKLS+gDR{MjCg zfR+Ij2dVL=Hb1s^+6P73@{?<9+xL&Op9VM-b>Y1Huu}j&{D5=f>{MoD^>T0M;;oI^ z&hZ?z@*DY3y?nxjRETzf|I+1?>GO@$w^#n&In}iSa))74&rgR_c8+)**}*L|&n>)) zyyRK)36%D@KRSb7+5SINR;e z@{NLX>N^hSw&TR2#xxE+wOQGXcPP4RHR$FrH32@0coXkhHe;o==m?it(>#Z*~h(~&-3p0 zip319b>G*0#rMkPG@|@ZFvU4YJZ3)NdRW59&5pPBovXnX7Xo?8UM8^hFvXn{(Qo1^< zb|^Cn?%cPI+FE#Aw43hKz8I^8Gqj=N?s+KA&Qdefb0BhP*qnxowxMkvbI)}Uiqm8D z?Qy9c7~70$^h;J*T+FW#Lxyg7Eh>z7%+>-{W(9~pyQy1+h&{-NU)Adx1-Fw-J)e19 zV>vWKhNxjGKzd)Ci2Mjva;MQkf*Kg!C@H}2pvL1fYA z!wl|kh;B!k`%KpyZs6K1;bjlP(l^H21R?}#xK#I9Y&_35s{fNy6Tg8; zEYN?x@bVpE!JTuKQ2+eLg$9Pr^g*367nP+V3=bVN=VrY%Eb$c5dDPK!J)vTP#^&Y| z_($CL3E%2EcJOH_(GB0L&@njN>!!y=wgxZFJ$+NDaMZ4e+EUkcXpy`T$!Zm(qho3# zmH~MJgjwkwGJ-06P}Q3YEkjGNc{7U@k+#d#0kx+lUY8bJXwS#4vC`MI{cPz z`LQO%$P0QJel9YgF^RWem_kVk9X4O^fvyCq=BQaON#=XcYl@2bPBLcGhtQ=)B|8SU zdr=44fQcWu`zH9j4wqnG&mg*Wv6T_%^k zmmM!~^(vB2t;`)RAu`0SA1y0J$>`0!KIY#k7OR=>y5A1p?vwF3<=F8w40$`xtd}Ap z*xJs0_bhwb+Fb7HT1B6LnTY_JDqB;oSC)%7OVlhj?y~@(> zYs+nMH}6L1iqJ_&7YLlRgh>u$N^8*NBS)du9f4otEhV#h`*MDBy7}+x{spN9SPqD_ z;80PT75jIqsOM>xvVo{&9JB9gORCg~;(rYu;>tS-`p7FF5$l8Y17!b13IW@3OEQU* zJHkBtpWyaCPNz=ss^YldJC_FSpMvshTRcM;AW90{M1p^q`=6=7Z;$@JmHZhF{2xvJ z|D%Z+1>I7K>^ITlyp3t_Dlv!ujcYCY;X)e$o^np3*+0_Oc$a~gR)aoS9^_=<6!*vH zB0ZOj`xp+23p_WGlIeAYF>m<)fw(_$!(y_dqCPMq`MbhS{uTqhH@4VQi7e#Y&U37d zRt;(ud6J~5XZ_<&lIr4+sV`%cJb!n~;tjZQ=$h=(pzv+Ejv`Jz)8xbs^8wjEz|NPa z7=D-Hitk+eC*IK?>Ik1bsjl#&z?ZH652{7{77ml0)h8Vu(|a*4cg6gFB>96mez5TWk>vkK^1orrzi|nKsA!Akk0gM|NfN-Z z=hw5HMDtaEEZ0)Deg{H$MRieDy|i6*Yu~ss$^N2e2>&WPTIdg`^*@z_nG;|2K)heP%#)7ChVku797i+gWYuKNr3H{r6zzwTFo* z!IjRla)0Q!Gnno_ua-hr9AemDn|$&6vKT`=NH9;8YEmSIs1z7GgYuX6OZ&wX*svGF zyYG)Dw}MP*g%DgOW+2$%eRWC%3i+gsJ&ipo=f4v=x+Do0v?rOAVW*qO%n|2e@vQ(S zstm-v{dOP78(JWTpX(V?`6sdYp;G>Q@tkhFfc&!LSk&oP;$s)(Zycqs?axLIfC_xS zefR{5^LI-ifYizpY{0m)x=(#CJ98%aT*MkDpl=hUJ(`j4PJepj4UdvG+%@(_b3MKP z-m2-7>g+#E2g{sNxmV|jwtm4{BQ^~XIC9m%J~69L1hG|s;7MX+c$eJYs5cRR!g-Nc zkRZLb@YFU#DsicXOMG^lM181r5r-R46&_dEppHc{)-cDTKAgT%_J<!*aTEM~wAmQH&1j)Z*gii+dXd9cl zUH9t%$PR+4;!K-FzK+VCv%4s%aMMwHj;4T3w;|8BT2ICKrQB zW%bAID!6kuwn*%4|MmmTrCW5L^sJd}nRS$zK|>jjF4sEpThCV0;twbPF$>^Mhuj2+ zi#SBua#Rd={b@<)K?dInnPh)-`nw(GeEBZ*j8akRq78g~%j(J$!*#<8Vmi03(te2t zEZM5jDC7Us|7UjnnzY z=Zxt@ruWR%B0d-j`y2|3j$KdGt!2(*(HT0>!p+l@v#enE-MD8-?UVVgztCN(fDERx zAwY(sUh8I9&Y1PW&bs+6F1&hVrLs0o*x91z(*=U~h<;ZS%|USY7GWmdT77q!7}#4b z9NgIJWL8<;BT6Pp9BUXSWcpE#>l2IND;c0p&Phs#@eWWi3?fw=A2j5?@t-L0gB1Vu zg5tNNyp>T&3CH!GtS>9DR0G(f?YPa&i!MXC5^8a!UDFNdWT0e#L(~qnf_oA%%1fA1 zY2&jVj(Z-XDdk0%XYC!xq7t-{&ldkcBa1_r%~^~sZ>qXX16Op^e&M1RTu79}p)B=% zeTqwM`O@a@R(`?Kfko4@yLHxwb)Sq1>Ym8I4QRi>Fig9m9nDTsbXXm7s8oqEh2AQtJ!&W;yW&={ z;%?|kyZGw$l}&}wni?_PuDF_UL!eAxwWJJ{ap;_N<>8W`u#vs7@XT85x00vhC|3oC z1d!>fmT6}Kx>!ODdUwSzF_03)X+t;Um1!`@Z+R>xYi)}F6LQ&Hq5aZ@;k-A|*X8&w z;n#Kk*%K3&!{9OP04s2XRPvB~IvO};w;J`5STL$zhavr`Rt;FwAEe7DcMXO>EZ@5ANb z=F;-=cHrpW_Rge$=KIUPu!>}-5qZBRi6L0ldOz%OlF&U~O9?%&i6EIB=uhchYp$VNc!CxiGU)- zH!v(NIJG2HAV^;aZwB8l_lrj7B$Nne3YU(WsF;&J%~P4QsKHXU2E;07Y}FGLQIJly z0$Lc5v)T!qjABt5F&`^IuDiUNNG38^BtyzsdZxbae@rkmol8VOz*WuPvEEKpRI@Ya z-rS**?Jny8WolH#U2@b)d+MmI-;U~GvR#xeD#@2BCW6KGS2DkAxTe#%vG^{9ytVNH zZnCFh%#LM`uDpz8yrMFkbwp&Q+5pH+yRx#eRVSy5KEooKF9meCZqLGle`H!Y2Qk9y zp^B{DX%X(9oYD+n_vj+fc~jlEJ_-A&1EoOZ1ZcnDZxoa93)xiuI=ZXrkv52uj}=0f zwsGj~#%fLih0t8!j|a+MbypgLxpc1>4xl@{$)SyUE1tqQ%P8xut=NU6_z*d2!R(JpkF8<1yIWD?PV;?CpZMK_LsP9-s86 zG7I}*;J{se^;78)zRi;xBk0w>($|{bj=`2*G^Zhtdi{w`k-3H z)j;IKi*LJ@NBa?7T?ssMUvh~$3(VulMH1L;JuMZ0C!~W_}q&LD619Opa9J6^}qZN2L3V_xeF2aT9ViWU7x1y20VqQa#zMB%eiYu4f z)D=?0|Fd-e7XvN+9GAD8-$&4B2(B7=Wvbty!f|I~Ci1Q(Uw%L^W#Y=A)6#Cb`NHky z=h@C3MazRHh;Cz}jbmT36bdTshyo$|?)czebXlLx>#`P^StLoz*m;-H50;U^wT+P1}uQfG7*gbI>puLZSlyD>qwU_Ve6(6G3{qpv*rwnv8N zhhG4JkePMd_q%{|t)8nRz__B*o!Y@+LlG;L zp7RWs&ESUioO*WCR`U9W44EFBAAGEng|!})Z5}u@oxK7!vjXrr_8p`M>PtymUR0x4 zLEHmqTZd>Z^rXTlQONQbjqkVI<0Y$F3bwM?Z&T5JT>64rl~=UV*lP}~p07Qm!KcAN z5y`Hz^*giDkr}g+j%FUXy`E7Le4q{_8&$uZJBSQ8cSj!3877{8E^fMvBO`FktGz$k zx+d$HPZ;g=v;ByZ6C>Q`#nf?r-w4fsjV=j;5!g=yEQin2R1M-aZiGbpK00%=KO0B4 zm*O&X=xNw%V?^y@c2Iw9s^bmYRE(M`rOL&NAv2s=&5|a5hhg}JkG~cgP22-@>PQ9z zFC`uxa802^czlyK-L52q4pu%d<917gx)djZt0i+ zg|4P{ZPNO*u|}xEQ*k6_wwfS{Jx%wFQX(2GuDhmV*}&NE2|66{&mk6AjsC3jhfarF zBFa3Jyb+EtXMv5Vc{l%cM2J%mW5Ecf;;DYLh^HfyEwxosN>yLuu3?}y$iiWQ+F>Q_ zgdf3I3k-$4%~9ZO53dfs0joRBxrhyu5fDxMIPzZ5A}MR$6`6 zPvUU1njEN<+|mXRTvx}|8Gg319v4j)PnDG_O+O(EN4pi=ZitRDfc`>oh0(9>f!n#73o%Q}HfL=8R!ala=!L;Xxk$ zuRqQWpH@+*qi~~YRe-^Zds#1T@Ei~Sy+01^&ti8Tqu~K9Ni7@h#Bs^{)~Wri!5QJ1 zckbcJ*Luze9eEmS*gy=0a63TJ_Z*Ws**HXl))AWq`@`OnStaJPaTjrw6BKs%20D*e zz+pIydWrIOxgO3 zI4p3^L!}{1-t?{rhpB1BQk6TvH5{(tB8zWK=cvHJgtje3tjO^)RhEaiG@})IKB5YY zg0rOTq=pmLM?#k?uwVTSp+2&sdMV!jBZDUXR2+dBVRL8>E#LfFOaC?YhsmoWBFjAN z)Y=;xI1NDk7a6a^B7V^rL7}FTOJ@T9+_JJVrTm+bJe`i`YGAjYYky2JhrNzgLd~7@ z91bBDWGqQ>+_v9q|HJLoqy0^1O?~{SnXW3aIVJ(f1Q2ww7hKKe`xzr4zlotWRc5!~ z$P)0`O7{7Wr2#I4iZ^PELgc`e=&#)f@~appC@Y+HlfwbYWYf2MiFEvQhQ@vAIyn4)ooNI8nV&futw@J+K zHZF$?>GKIRvr;9_sWG|Yn?a`V)4J1v;(2PBAi>Nkzf><85$oZD9j)5K$-Yzl*aaZk zopu@CGXex6+BIgjI}<=9m&SlSfL&m*qmmY}$@?LYf4*pt9?va?m)qJqZN6D3sgEQp z-T;2qs;9oMol>$#6fWp4;aYaEw1esL%k|pmV&lNSKOtrIbGzWu=m6zti&yFBk#>GI zBwF#+ufAOQ<3zc`)Z%rbsnn7IRUT5USY}7{@yDvtw;;nnIJn=2{_PJ|I0vD|87*Cz zpxz<+9?b+aVp8G7LxR!41Jj*9uk^l9LxR+;5n43)<4HqdCuNR0C^kgW|B&MrLRv7UYir(5e zcQXT8_%!xBp!3d$(^sM0{})1GMDiO$sw1B*F@|t*u?y62D|Px zF?7mRARr?l*PFZvZrB_kP`&<%P6&M*)7v!1@7qiRb~`X^*mu@%o0Zdd^l-8nq0rPq z;eb~_F3pCcI|Y|%9Nf%q6s!BlNx|u8$7_n_5Pwq*#fO#uyHBRZO!y z>rMmlWoF|-79gv@WHt2)3)t)}gIKno=TbM0WfiD($@EFe_ml(hWb)hRVHXImR_mW3 z3XmEY#J;;*M^o>G8r5*tIau)-f5BuoT5WC!ohs9+k1rpiNboX|D`)5uE#9HG*4W2g zX$Wz17n~g&@V}+&;>5(i7xpodmTwmy87^j%gleT>yZarpCXhvOf#HQ&R^QrvT(WiI zR0?`7U#=JE017w-b!I9_sAfaLF$`XGF?4B4I2r~fe^9XbqDa7G5)48oTUpxNQmR9j z8vJfkb?;XT4C)j~2TRr~*4h#a@Lb3Cs=oB~4drAtkvCjGjTIk#<+gBgYb-3lwiUIv zFtagVP!)~G!n=1K8J-MfnLYJKE~$?Lwe3fmxOs^(#k(8}G)HEiE2C8&xQQniR49%{ zkSTZVNZ@t#Wz}*K7e~p9=PDE$fd(F1=0A3MW|*foiPd}a%WJVrON}V-#t7(#TFP{lVlNb&RdIYP> zB;+d&Ry~$^g3q;pFt{FDUQ5x9r~4pzTrc?gU9Ve&k^mz$*eSF9ayElPAxWD(fJVnh zb@ylmwou+u0N#oIu-uJXPpRN0C3q}ni@Wf9tT&~M0mc#iculun(Z*?;W)rDdZ0WeI z1S#xwAmzba8)0^6e z`DTrXQ=5kbOOsr`jU)X-F`=?4hW9EKnud^5XkCGfH#!&gytashe zpd-On8D+7px|#`7u}c2K77+!od5Q?8f6`M_E72&i@@dVZBoNA5c3&~4$buHmsWTgm zzBOLo>D?5d^GBp21iY)*h;3je!zmJsgV;bp1=KerBXG{;=yB!febnE6lVF?oDpDXr zb3?_TEWEgst#64^ayazsjWj4Nwt^2BQUF?jWei44}tG8Ln295I2=vax&eFg9KcaB@jsDWk-7Fh%Vwa%1Ldo9kO?;1u2}d=G8`jIcxd zTK@c@2iDLh5O2+v8lKn!tE=}R00CiM=7u!LPf&%=*^z=K8fux&?<6V?4-_-06XX*- zb@KKWiBbN-GuQ*+50^|j6AIfY4)d1&7IBLeZa~ z0B`}cpG-MY))VIIJnCU46q(Fi>vT=LuGzdCPU-VRc&u{TKseJ^Sc5yY}IqiLwrDAZ`|_JmdgRCgo6pY!7Chy zy&dXATX+fA;cSH7YGH>Q#&fpu8ty@n@A+^}-izG}!8ExZ7~>)489ra8(A%c=hQ?7U zxsl*NBVqf(1B1q~7ch0^Jn8b)QQ&oL9L2da?Nt-rS@1BWxAL-ip;tP1`}ZS zU`#B$d~re_L4sAI;9pmd{?F?;ymuO2%fHR4eE0CtqzHUFA)Bj|m!FT%uctGHq$xO2 zk9`eJYnjn8dB>l`U2rtfWw*MZY6KbH80o9eKxM)6w@4?4XcZ`+87 z^PV{#<{Qv1rsv4#SYa$HFHf(izQrM>Aa6ICz`}y&6)^n_Pb5d2#-jO5FzI zmJX1ge}E?V;Qgz>f?N^e+?9_{IO^)U8pT+>lB&o`>NsVAZDxJr8vH1sZ1f9aY$NBx zdIf%ky(HgwE!c;K)aPJn;y~Tix0%}AE>pgz+JbmZ7f9TZpPnHEqod;*VUGEb#+)-> z<2K!VlGT=oGHMh9s$w-X8m8YGJHR31#jmy$2J~CZ4Drm>q-o0PjhK{l`G6ygo45G7 zR*wciM_UAIu;%&kW=atf+1hDx# z5g=fr285$*XSRMl9tqZ{`FskXE8d<* zefnR0yx9&9n-PQn#CL(RS`QX9fQ<)j3oo7`HBoG%h!v1~Gt7W;#XPYo@X61Smsg@!GY zu?8<-?QMx9IA{+C@WF01G@&FB4zJbw9Eez!PE2TWc}MD!ifBW2EJez{Ijkxk>GD$5 z)wUf`5Hg1AW+5{+_A@G)zG&#aCCdPGaRPwzmZKxhepRArzgR*7s1GG=5DlS%!UE|W z=cR6hpQ}qF; zFXN==F}#QXn{)ZD&cAq`e}!qq6)>qE0?{*_%QP;(>pn;vCKR4t8$I@c9(0aI-6wpE zS{`28q|qZIJ!&4P4=sQR-#zNR7F?UE&*y^-*DsAHJuN_5WdGnSx%cs1i4v9iQEUCH zJId+2>V%8#ORsh+kn$IU(1T$K3x{j=pqD;$Wk9HO!zmDW@1L$KijPFA)-ka@3aOAN z>LuaIRXUo;zv6ZI74=97cf72-1lzh-u=`0De@X4=HKCsUka!g{ox1=|gp&li@O+JS zoIP^H8hU0E<65h;%Itbv5O3=S2S^(Np!E@?Z9nwh4msxP z>mY=-ouM?a#}}4HhF=;J>^ZGXzrbX7(h7{v1&evzWLt>A7CGV2YDwj(`!Mwd2b<0lneU03 z)yrt9X;6fq`ie1N@Ek8N+aLcf0R@hJ3Kvuk3FN83qGk-~5{cJWbHc{&=jz5!66qkU+TLk!$GDkklk)5#bA zVvyId|1eDX5G9f9rlEuaO3&K>DuQ(YL zl~^Q^ne~CY!`f@G-yKe^RB`Vl@g1}HTJmq?y=kS!7cvq)zr>n&tuii1H1Jyh74>TM zN<_9KsA9ixEzt1t@_0nHV^-;UlzKJbSb6I}Q3)cF6*)QmHXJX(HVyl0Ih5YSYDbfd z6En@#%+>qOws@Cifg$hqHd~=jQ4i73)`#Cj*FRR=R2>RUBNvVB=M#3d_Oml*Hal>c zOJ_^WA;z6;LK1Tt?o=E0y!5A$M-#L>+fsNVJDimzrBF=1L%6cisK~s%NY)oZj|+>I z553DrLMxwRKF(@siPKmX4Gj38maOLPID$Lig{FnwPLy97;%J+M(3waDuqK4-qC9js z+UZTQ@IDZ#ySEk7FojT{Dz`Xlt64eZ(q8bekQP|W+>4HnU*`c*neXJ|+hHg3am3gy zkJ@_C$}Pz|&w3mVpGY;|Bb*MUeRd842fA;TgJXcoou3(9zhwF`t zc8?Bo+cqul>EX|O@RS*JLiD9|PgN;@8zmK#5=+3wZ>+5JXgokke`>@fbm+)gR-O5( zr(h#>C3qyLpg7k}%?#vr532^m46^aqY0=1=Lg~GkRXten11fL=Z;EM0OVK$WPKE1Q z`P8&^i18!0SVgX8C z0`kKUUKop^y7rJNt5{6q?~wdT7#4Imv@H=uec{G*w(Is_MVG4Iiy?2jDCj1k%$#nM?2Lh&a zdFWr=h(r!%{3c~)iVVL}xgK#^IPXFCA>DG540R_YD=ybfQEsd_x0I+}FTh}Tsh^Xf zJr3lf%aGapsX6H_BZOjzl37M3fSUM3fN={0|78B@Csj*P&{PAj zM|Xm&0~OJg7DR>#XL-d$)UtMM!{XbcRFk4+HbAwe8vM&7YljnIZU?$P7Uz_KQXxA@ef8lwdW+F zq8#C|XNZ<%aCek9fw#3qe;Q(Nl<$m=Mlo=3Lf^en2u99w#O4#JLsK~?p{?MJ5aKWLpF>gU_VLpr;)y4qeFL$yZo*8-1?d*`{b zByWg%a+b=a@xB-$<*1k(wi_rol7}8QX*Fy`J6_`5w%VZViiK;DdAp%q>s^t@BWtru z1-@nd{pYtArirv!YQKjC#Pu$&R-qEqd4`tB-fj->d#s)tc6BjnNQ4i#FDr{_+oQ?` zyWSk}+1?ft-b6*;uzPB_jQ)6;=B_6KxU12I?0$hH|GV*I5SVvlXb*|3LdC*J2Y8KJ z-)-%t$TB^AYN)L@+S4DhOy6<~m9xJ2B;pfq>`MEwM$wKQA&Xk-B^QF((O379G%t4B za`sg^Ed6?)_Ve3{4;Vgi88+j0TZvLJTvAC7Z@5gD3JBT$@bIu*|4>lFoGtp{Zg`dV zEmj>Ddkg-yl6%yHFoC<9)3+v8VQ846*pz9S4@Q3YafQ%nzYv})gvJxO+AJbcH1G-b z+arsh+r{GwMMFDnt1wh7VxHn0YCZ)a9D_+I8f$wvO&Almx5oi@zE{idmDvV&MUJ7T z9-Fq+6BQwr8rOcVkz1X9jX|D3T?Z@)-D*|v-$v$QTdEb{fl>u(o|&F zllsYYsV)NR@T49!Me*w&MzS z7*R(n>sUiUM)fN#-DKNoq2%IGzjKR58P!*dU|V^n9#jMeugKByI*E-QZ5FOxzO=uW z96)U$y}#Q)iG)tULNm?xk78tJ5w{bi@JbM&@rn3gVV(&N6moRK2edaGwnCHR`t%&+ zEe@6)>z#tmU-wL~{4lM9DZ&yS6-Gs*+z*ysS^e~KD$6=M`8O{w$;Z`O&qgf=I;|_j zoAH<654roCw!rRMBp1}l5?voX2GV%sUwMlVNBEJ~bXXdUGxAQmnBLtVp9p|? zF)i$|l#kjEBo6fT9**3uU)Q_sV{M=UlWH?5zYXyTbukh~uO}jF<|<0OcU;SIqaKfJ zsnZ$7fGuNzkvI+`)_R(2&uKhOy=<-b9S=X!jMfMXanu4)XAxLYCWDb^BBn^Ef&-_# zg13Cz5uUyS|22Z!>?1{V#UN~}(HJ8uPbbs4^?*dZ!uJE`8e?Xyu8v13aSx5%VcXX^ zS_>orzYX(2&WhCT>n&Mw!ybjTz+dOiO&Zx+9*mh|#SdBS6n?pWY_kcrJa(UH4Y$59 z-NPMuqeJH?H^0|iIoUH2 z&sPw@XIo7{aAOov3Jf!%0n}dg?y^Dy3&&lOPGM7N_MEBdJ3R-z&2|>cn%Cv4tWWdj zNlqAq`OvxeR1~6OCPM2tDcyWjm2b$-J+nodRKi7rfs(<6@Rbb4*l-wa9Y}f{k#IZ7 zUis=>lpEM|E`?@WdF}T^?ZU|i-B9%T@Qb1EU=v3uZ%ga$o3(*T7Dd}BR}&BEDP5** zIoF11@}Xd@?lytRrS#ZZ-dcO3F=y!)Q>Jf0S~1H@)3<@Q`*C-L9?eAEPMLf8x@A>n zxcW_pkzT8lf28n8eGm$P&U<^&n835tgO<}16H|~Nv8GJvR~<+4n-GbjZs4`Ks|8zq z{pq2ulWqml+Wrq5*8Fd9y}!d+HD6>0_lun6=Bf?H6t?lO)p`)(3kJ_?&ODaBWQb0qfjIaYva!espZoT*Q5%Rixn_v zq)hLJP^pJmn^`Q}-T8QnMeP&J>?`bYjdG0|GPYCJMM-XWqbU0AWkHwt`cUt!c&I4? zJ@1^o_&$g|J5j{-YwFH0XL|lEs^LfAN%7j{cNy|q{^uZc*B+n;Ued+JKz?0Hov}*F z%hT9X^VvZu1{RE7W{O(k;xl&b>VJ*Hxj&pOmQUo+OmIzNa_dEdAa0w}(Tk~&> zNO*z-iiB8oJ$`%ax$2kf-aCKntY&Olp_XkW$#&Xc^)9PU3KGh-`{<5%Qa+C?w)vG1 zzWC=9PhWZ1JRm}9(xQ-Z#vq#buv=;&#IcgbSq9Y64s|>FY~n5Pa8@^ z_m%V>Txt&4*{OBX*tV7mroI-T=Pd`F@6KsR$4C0|NvkzmBW)K^m4;>Y;T)iAx?X!j zcAb;P6T?fki>`Wo4(~0~G+nxdzvC>6tDpUci+S$gwmO;cXYf7(&@F`eBBtQZ=SkXM zj=M1>h)YR%Bt6kNf1M;qAcY9$dO>k)?(^JU1h(=e_UXB1xDnN_cQf6)%5J=b_g+1J}6;@7f^+XxQkxI$S-SO&E-9ItA`;poIBB05+}l zwC)8?KS|*H`H-3x!Cy~W1Ohnbi<6~W~(ldK+oltbWzWO-GJjJEoe${HdWN z!(pt^s}fjK5x05DNx_aD2H%lzoAfUOiKpTpG2(A5K%P05K94O&xbd=te-R-4KRM`3Hd?nr$JfuXi{poS`;09PWAvGo2=OO%Pn4Y7 zax;oCnu@-70g^c&2`tE5<-5P%phR0mhbS_<-#mO}TVTeL=XK>msT^9gg;{6pat-BHr%*ootENVy@aF*_R#6u;Ts1HRo=6BCpociam35qmi4--o<_7Q-AkN^P{(g%7nj;hT z5B7-5RaJdEM~04@^Rn#b!M;d{fgIaTW+A`Ak3A-iL*I}W|O^HE8yp~5Y`%ZZ>5lUf`zOUgHZwRU43 zwn@gk_V`sZP0J|`k$6^i3H01hN@_71r)3=K+}8AE*yCntfdOw6A6VDS%!>8C`q8Q( zx|Y!4w_s$h&W&MBt%I?ymTRV6@iY~_OmUL4g02idMphdhHT%Pb&XuV%>UG!kfP#+Bk zWi7GXf3CN;0mHdQkwKi2Bf9-{Xy`h>jTCz)Y$VLHy4E=+@_XsBa7 zyu+6*nP<7v#3H@lw{u8(Y&q3>gQp#vu9($5F{W_6cXw#{P3P;qhmQ+y9u=g=3Aa!J z8{A@jeat}!N}OZ*YxX+}WEGRdb-cVR+5lVK%in0F_&|K-#TB_ms-nq#VZgz9d@yS{ zkvHAvu=V-roS4g~uePEgo{g0m`LOJ}3%ar|h3)d*VX2j$Dq!(@xJFtH$N+ykueS~B zuO!Ic8X4XdF$cR`{T%g)!$efJ^)Yy3x!&-)LW2FGDgVgV>%!>m3n+N;L@$>uuqe)R zfT_7q=g9oa%V+wX)?`_RW#u{5_~XxRa<^i8(sjbON8pHeZT;lM{3gXi=L_tC$^HzG z;I@Y4ip?+tn}YX8?FH4k(r%XE?~3pVIqv9*;<2rr%2DU*HKA5WU?3wPxwm0d4H}6) z>ONvwb+^czeWHzlnXLeD)AmXh$*;m9@n~$~t0m64GxJHEm1*>#TN8FTc2*$!o7KfG z&g>u7>yB$Kz=HsW`(U?Tq}|jMRq^PKY+b2RRG8W+P3zXK?M?(%m5;eK-$*pezP=YW zZEK<}IEc|gbL=+#c&@PEZnZ&U!bj>6H%u&*asXf~!u4`~HQNf|CL$lY%j`@FKuXyY z4G;W$xMDNS`FR}b+Lf=ZI8|AWz*||d&SEgf2Z{}%a4qhdes!n2B*g4>%`R`NLk4J! zI$@_LZrGp3>*-xb?e~PDub|mm@QwqdJ=V6+$y|!>t;%l2V*5BDYFyECsX34g0h)Qw zI3N(vhmB>ITgg^R`>VnUlzU@eW6Y3WuseS~AS-?=`QsbOh4bNZw`9-<-Cj`+Ut?UH zoP2@&w@9x0QTPYP=Kl9vBCc=^=_gYgoZ!{N_gezmBKCQB1M%fHr_W4Mgz&MbmE>bj zaGV@J!X|U!Fo?)vRhCH#S9|yRV+;ICZS!%OLrd}8itzD+ z^(WpM>$kX*1r$y-5kkdo_aoW0nnx+k)7Hw^;ar(cOA)~n9k;cnBIJ;> z_}pwxpsAx`q{$z%02pN-7V+s)b?Q3VWER*mZSk()5ZUDsKTXB{F8E17bthidQA;x{ zxS-2xRh_Xf#0vU>fs!4EZwQx%E-u$zLuPS-Zk?`^nlk#OwK0;9bDdEL@*+Uthg&+?urvz;CkN-Bl4= zzmSlf^9pPxMcUAA`25GK6;@dVl)eTXO$pRSdV!JDM9G^mmxc1uE%% z7VI%?XK#J{e5mDR|I`HvcEIPZ*d+Ch3U}bdkh{z~#4I{gs?->3Y5wz(K*uWsZCxSd zi_kNl5i5)@?`vRC1vO_e{y{Pb4$+IQI-GyEZb=s7AV|;Z)%KdhB zb`;gsEA^0;me%-HuGUCrf9TT7~~v$Ki{?SFp^oj&~FG}NfS6;;?*QWB^g zWSMreyd(AJt^rqs0}|sMKXdz!HJm)0nAT2u>OF?0`S+yViBj;uWZ%Zq2F04wpV`F8 z7nm5|ykiaA&7IM&U*F(5w2zoQ4Ht2(27rncT-RWyKC0uxaGUSTum;BbgS549J@lB} zUf%$=4Qm*_Yu>w-`%#z9?)cJgNlreK=0ps(Sl`y9XpYf>b{tpO=-3NutLNKH53t-U zJ;BnDhgYF@_hwe@qi5}C!6k3gt@D2_ZJ(Yf(Tl@Ud$hnY{7Z?_WFfLL&1f2zTf;rQ(J%`Squ1lZknrew!PjPRCEIxNOT zSJHnwldXVU#RcN5Tpw1y(NtU=GFN+`a}X$o7h>cMwjyZO^g!QA<~q0W zY)0L;sp-l+S7Gm5`dpo2W?LB0k#OBKA--IWUX4G*3@0cejwT|`^^I&W$p=V&IL_d0 z<+H;v#IIl;rr*0lj1wN1R2DOfF)^I_5TS!w(N?~g!jxtk_ji;1~E zYppCDevMHMTPBi?5SNr4-`UyQx=3lKvsCGhh;afY&b8fx#A(}ddnxEnAjh#BgI-Pv z=7qgZ)gSJYVlBpTq{rTBaP&`jurS=V5{0ZV=+9@Ix%FzUYFkYY2#L{)gpWG^o?r z%>3#eTb>|Do#3;-UTwXvxz`>R>RVRUU!4(rLx8S5ici2c)*;`Df{R<*Zk=SJ4ZVTz z?;3TA9dB}g=VHvxYTh{~eeJB^t3UyKh=UOC`H5N-?_kXJb`ki1)?FK-oq3pzTW5d1 zUx>Cq+_NnBWR>c4|Hc+rTlZ*9$f^&smy{GVs!UWLmu~p=y7=)v-ka=d*mvzu<=64v z=4 z(XzYoYm_NAZ#xR91U|?8_!{Tx3k}LC9&69C@QOlO?%*F^oL)evF;9Z&zeQiwbDMp+ zwpv;4F`v99dE$oZdQU9$UQ3saz1}Gdwzm+$$HGg0YH7?_8vHM|URe*w7p(<6B<5OLSrfO(`sYP?kv0K=yU5j3KM#XQ;$d!@1)-OZrr#r(P-*Jp$6Mgf*&iW>3TbRsI`PyALUY= zA9%R0TcOq+Tyj>R#j`LEFWt{F+CG!471xFM%X;m8da>zHP>rwpv+wHcKb{B}VCQ}t z2jJhtUZNOWvf~StCXh4|y(}mnB`6dJ{z$Lyd>#^$qw6lZJbLU6UCv&rFIgy7BC7yQ z{@Njr7+}S0WUMJE<09J)_g@WNEbYrI!qjHjL$#H@MM$xowGj{50B-25-~U0(KQHCq zKB>Xd_FU-GZ>TJnz>|<+5h=IDC-g+d>QAQL`39rwKivVWq`kh8qobLS$89Q6C(h-g zC$5hH&K-ogDrmZ9EM@>4!CteWoFNL9OrpA|%`)?&x#~MlK>%Ao1x=>@H(oot@_+qG z9EM3jLDA4r?tSO(?sIT`VMNNs?JLj86TKD-pADz;Sx^j7iF!@td3B0L_ZN6zG?D@^ zSWCs{tomXigv4w_`D^@1G5_nMP7mmWDv*ggoZj`RYZV-xLa)pygfh?Fik0%yvZ-Qe z!4Ud??7ekdl-m-l@=r+WKAYHO_x5R>^)Y9Fx zG~Z{h!Gd0|_j`Z8*X#So=Reqe_B?0i%)DpLoS8YcvJE$gL5BdO?J`dDBVxnrEC3tU z{zb1{kk%D8t+5H!>`_frUOy56I_JZ!EH7fVr1S4-0O_M$3@t*GB`P&w!&Dqdh)CP1 zMLhXc+yoN4`TUl~fn%866fZ5g(XbL$jfuv7KAY{}c936!6;dJ|5PcAruR#1Au`{T+ zit^KQ#Ls^Dnx_D)v*(SS$ymTA~Z6TB2O;+Ryv;Ry)o2dPVbr zx16G01Sfe}K`MAKj8#eI=JxwFD=tSXh5`ZT+(E)%#6$uhjD48+R`c39L|5-LLL*9y zCL4-r;Vig{IBB8beOemsw3@lt5)jq7*H<_it*YN3f{yF4T?80+f1}5y6y!;5VFXE1 zu>|@pQTTLH;XOKh=!_bb`D>#T#q!sVN{kU}Ft9a%ok5M=az+#)SANC+Ykz9$uD^!| z*e$byADWt(;W9IeR=?0`xa}e$@|djSA_=LeM?^$Ku+%;c+X8?QHKxtXVaKC`FyVgx zDCy(fU%ds9>umg}b8~amVeI3#pR;-b!DT?b%RaqF6pYFMtZfb{k#f|pk$%LV|BACG zm)a{9(4x_TX?cW7`QwTIJyM)_#!D4Ab`CK6{Zqpezy30D&?r4srN#DQ5y9{78-F&) zWq{E<6DpdBK!GS$CyD<#%QUxvFb|ollWE=WuGUZ3t=Wh==jDv{eQk)5Kz_ys_W3q0 z_J_49S|$Ub*$FykkDFr7ex5kGcsyPA%K;eSY}Y%9^}D38vF4{enn{Vgu}BF$`d?Lu zaSN-%k4g>jN9G^M($?6rPbb>+rQa`!-XmJ5-1F{kHc>Vs!|P`TeV* z&pt{6l;g>e(zl!_X^h$yUpMm+%IY>hmt;SO)wm#(v2|nTOt~MK`qNKjO@KID+VO!h4{dwkA~uZ7VXVsl9u5CG@}pE? z=S*Kn!Ef_H493gvDY%Eo091ac{Mu6Z zZ(dd>0Lb-%SRMOUvA%}V12#!{X`_hpZLR(QF|2^6RW!c#asDT;NtFaB4>5T^`IYjD z^L-sTFe1v*<7l|-DAIrcRDK&LLh$=be)-1`p!c>!tKnay{yhmObm2v+l8@*22I3KRaVDfyRG8MiMBd>+IbY;q5Uf)#esc<5eeR;+PIp^2QWPUXo{71!r zL%P5Mk3RfT?~W(?O{fq)k+lTOH!VHa_euQU{Lv0~?(8na{;oUnGr~YY@K6hH&Y5E& zMVRLAvG@WIR;@oe(^XJ^RhP#Dowg2T}2VuYk2151H#=l*$ z{gd?-jw=D+Yw3#MfA9m&Ako4-E=z6Xf4?FK+&QUCKbz~%ivfs?|& zMBZF{1c)KEQpWDK*F+v|SHRZcEBb~(|M$8IoM6boF(h^I!|!+L_)&^w zF9Bl3s1w_A+;`=7X*kx+M(|XxqJJ`=)ep< z!aa;|vw+$ED?{>yO`sn0yuDzJ5dQ}xbP|4kJ!o6&{14-taL;P) z#U&(AmzJ-2uFN$w9GFmu`6+4SYjVo%;1sH}81){NU~m1j%1O*R z{{Hm*yJ-MbjZBuCAzkV;t(>-Vc}Z;CZ1njZ^pa^89Yo zTdq_dt_+u$(L32vk%?_FEkOfoyl;z&I++SsnBDcIAie8Lf?qEabrGIUU?BVWXs3Vi zb3V0kf8k-xa+9bIrGO=aoj8!o!*<2tPUj4tPR}6Qjh*tY?vUxM-TeIgDKfjoK0-eV z-u=<}nNjof$;a`rKIFt*YcRFm$ znTiPgJn@?t4~>DUwUuJ&9-Fm((|f}W^VCangLxL?^r7=>xAO<{J`xuCkbRlA)H!fm`3WA?y7nF3HcB$h3f8*``+iQsfl4zsEc*K)-};W1q60!d!o9E6vC zvI-qN%35S8+pi$IjY%0?BABqM)oIrGcxA{M9W-tT#aF3{wfit4kc21^Q(a z{+O=V7T}#g+@|MUf1UHo184utA z&6aHJfog*D?qqupwEviX<|$EAUDsS)i|3|+8Ll9Kr`P@ngCJsI?-UPElOr~IP10at?r~+!D4}I z4wYdp_QIv2WJ<`u6p(a-In8vF)hrV9NPfHwHn{%AdwVXW_6S^SWHdBjHi7gO5Peo! zG0AkNd=-mSxOH8DzGnxFxRw`==((vwNLUpq|zsou33Xh%L46@-T) zl|pAN(`TYN9lmOIOD&JLJdGqm=e!H}jE&<^Ds>E&9zQ{g|=B`mOD<|Aq`g~)g z30}vH@xg(qbYl<_MAe{Iq?t>6Phk!Z`NgpPVj@l7QobO1@+N0N*epvqyjn~(V4U`i z#=~4}+YB*T{g&)rl^2@=Aq7xJ0ic=^Vi}I=lq$nVh#UmuRCo@~x6o)V0;Oe`o15nh zlP90uRjAn2wx6tvg$)#!l`*GFZyB}Dg^ZSBu>v;;lq4~n5i|^K2x5vEl4#M~p3o6$ zL49eP#kgU$FzzeII<3ymF_oy`lB#hl!PBo-r6erJs{7-#Rj-w@rka_{hAqE-_Fw>t zTfG8yW10C*$mqhIX}WMqbZ#rXqMHG-|S35a0c-K%$*r%Qa+CT5$l51=ONf`JWDK9zU~XAJm1jxn^z?) z2--BOxD>NH>%SMB!L9*ix2HzSG8tTvF5C@{J-6P*Pp`4=+FOV1#aglQ==mAD=;-K! zC3@%!_5^6X9kdPASl#(^w^rAC_MM0~r7ZfGPB%Xngk@wa>H*0a)uZm{)xNP69GsIF z?{5a8?C)=McvnyP&ih=q9<1n9cV$`loX_FBTd#rRD@Ipa#l&u;(^77?(K!0{94Yc2 z+;q1j-l)2o@CY@@@)Facan)0$SHt2JL6E^JRjpYCaN_lA%xotwKodma*<~* zmRd|8SyD>zNe@7)Ssf-lB3+-8fY3nq40fZ*3zHXzDwf?>641*dvCLnArdktTzUc-h zuaY4^m8Ln$6Ep*2c)i(Q{Pt3((l6g>Uah#`YSyx-^`vz>J~rUew{PLepyy*Bj;}v) zHU%=1Nc|9#LjQUQbQ$mYEfR>}rlS2=pc{*rxlXTMKA8X>#m*?Krc-9W@co&~7O#-^ zV4mpZuvYjMLxOP7-jW$Oa4d~R@y{rncJo?P;g03!pAi!3E8}oJiapGHU*$_rl;hO9 z6*WC7f-XaGbJ3th89c`4rto2;J~P^EXieQ-vu@}zni-D#i%Ue5`&9yW&YAM_!ombr zJq=j~$R4q913_Ib{ zo*7xK;AXcnNy(grahq+g#8`i@Khx{JQ^p2#&HC=SCIb_Jh=rjJcY)s9y?R`LK0VZI0cjIr@OZYQ;!+_ z%~Qn&E`bS67>(a6Cm&Ya{1hiWmBFonB_xG(%_l@}H99}71{SPYB2Z$7J$F+&iiZN; zLc+4j&;6$P^}bRNmdYgRrpnLxwUXXPQ7BaPPr(mD!s?jx zlgM7-A+N=_%yk5zUW-<(nFG=YWdyfs-nuNyY&oq>7w4OK#PV;Ho##lxk$VxoPqDY^ z(sX0hQ7lQ88I~S2f_*C)kUyt=MF~sjk?r$aBZ(}l6#MPC<>X_O=(5ND)A{$tpyH1) z<)6Rl>@K}*WC}(zDWs`vx7R%n;mL0dIA+Oveb5kJfw!pZ#zBy7I%BQL0iTg2NFC6I z`KO0eJ@8@;qap(CuI2HN6dVG*%5_Zkb^0O?k(r~_Mx!GYAG zw6{D=;(9zgPvQMMN+4uWXWo@W@Y_~W&vW%2iL z&yVat{G?J7i%k4k4+S9kMN}E(uLOW>MDzOnyqZ1>{A$4cyNu?TPgkY97i8>&9;DDx z-1{b7p51B~?jROTvo1GtO6Lz1>*)kt$Xc%QO=x&^&Z>=v@f8IJOMzcfQecs0erSSs zaJq+g4H(AliMzGHe?$7R8Ql8FK2*VHDH{v+6FNz&UDq!nO z99cRyH(zsD=2U%1qnN}oo4`8R{U!zj>kU&gs+r*aNZIK2s*Zz9R!Ry6(CR|WII`w> zEtmeqSD+wvhM7M<77g*K!;>y~ecBksMrPn~jvBH0n#U1Yk})?0h;(T-yfhgd|<2O>ZE=5JFxun5A66#h}*7t>@!j#W?msC zCENR9fV=3Qw?tySS{0*KRGCCR0N&^W?^O}S>==fH=+2S~X+40Ymd5J*p{;QpqrMWj zrbq2HR%j$=;`iO?1`|9?qpJS8IV0Ab^BZIPi7*ktkZN`UQ?^ zpngzbNeP?6)nIXcw6#}ZYN<|uYi@L89m0P>Kh{D&>&yp%sIl`1%B%<9w7%l1dGt$g z`>xHz${-uDq$1)$`Fr6vUGxj*`bn#SZ6KCmj1fL^xuDyg!tIteV&p7fn-v8c1WH|2 z@)}%;fm-8=#(_*Dy@mA#+(A<9(8AjvTWI=dG80&Dz1FGs!y;m-xL!NbMy!a3T!ML3 zhJOt*y2a`u@JcL~F7)jLmTBoqP+GNv7#EOuol}5^ERZ1L+#1G9--NbvKvSaa$kiyv zs4;93R`HY<4e4~c`atbV$)Gx8X}mam4N~q>{Fv<~AEqyqC(=Bz!Q|ubtFHoW7SQN{Pu{qo(YzS`_OE)HBSn6LT&O25+WmII&{{ z^}DU*|cqa!7@yOnjA3_$}8iH|~Fc|*f z0xQ}@I7$av_dI#o8xr$-%S0txgaj0`U5r{Zsj5yCiyKqeBR@}oTp$SZWcBv)@;6wU zXDM`=1_y*nWzF+um9AI=Nzk7*WwVx9%Qxl*`FHUK`||UW0$WEP#um}4x{YU`>a{&F z5eP&%cSvRZp}=l2>D{tUPd#y|tr(tWFl%AX9*wr<*kZ-*raFg}tr@g)INlYQjLqy%I`*)3Olp%9};V9v~ZyvcP864;th&FXU4mc2cQRYI|} zdEg`#w=OIW?84EsWu>Q!VKL>THe!El+~_@cP&hg@fQ4zg;Ao5eQawz!v}_W~j=&i@ z9HyIxm^1WV5{ONP6q)INKXNz@u6+yAAbcVmh#KRxyLukk>(fUdbA0;Qt~R&=^cn~U zc+Hz`JKs{9Xg=SU?+%Y80K22I`FM4xhC^#|P2C%px4paNk7|X>NlnYCYc;DsXyD<)T6A-> zD{N@NU4ZZ506{`kqJrzSD7&ge6hCY{ z>tx*BzN%mDdnU;9bt`GFw=tK>y@w_U;>f@O0v%wfgfrJ$BYgTDIezj-J0*yGU>J9P z*p+&-OUY`=GcB|1a_z!{ItHt0*0ar6XRkZEt$Ss=LD5r0%GatPSePq{F}fW5uziUK zd-D?RVIiz&J_HZ6EA{@k?;+;qJu@`*<;B+n&SNyOg43vEjR@ zk5~lU9B%0}FvTXaOfw0x0<{~HcP`PP3Du1Ze|QzK%D1-g(0|jYMT+!OUQx6g(0}P; z`{#jLz(&-_^*SdN%ys$^S*pd_R*O;sep7d6|FzXHfK4T~t()B5DV_Q;uGz7%Nsusd z?VgDbmOzB5FV9I-nPU^dX1&zfb|;n`2iJfmNF3s69mI2NgA-=QE3zlHSTJ zR7+%H>NH5*TeRzR+Cc_u$93)64NET4>!X!YVJuv`^P=jhPf&}e;p9p@vR2J*zF{lc zBT9E8ZT%)tC~=FUDcEf%Zw!+wAVXx>!~Li%r<%u*xo$MQ|{^R_0}W0$lX{lPd3G>_^91x$a3RxZzra! z#yME}Ga{9#b3;xt(6~IJXwo(oDyr3uyfwF%O82p@?uk3IGU?*DnuW(eER1*W6~%lp#h_~C=I4ADWj1r;67$s^TBV1* z4zEaKDGHtUw{tR+FK(18FDaoigONyz01y&uW+`3RK)$ox#s9>b7-%5#L`Kw)Oxz^l zYeMT^fHFxhTkMv2ts+0-OUGpd5%>x4z#TL}x@-fOUpvqppux{hpuL^vN>rY`Qm@Dz z7q_kIjXxTI#v4G#R1*O|Qsfx0v136)aV9#@2}WKv;9jrQ?rwQ`{AGCEMoRJ`P>j)+ zVV)+(x3&oMH7=I{y7;Ve4$#+s@vDA zn3Wx?8rqoaKpYVDi!m;{IRS+B1&aq#T8?<~T-TnfZ@y#LT{ z-yJZLj~3J{kjbEzw&ojyU*GXfNkXUq$+1ZD=#E7E^!wL8{!yc@*;&7_V4jGx_A)XX zz>^xFfYb`~iam_IJVKNpl$Q++Gbtnt1L!E4V%Ylf0VQ%HA&Veuy6a+M>46FBeY6Gu zswc?{ghDJ>KOvFP(|^V>dh%qq(3$JiCoGlMxc|0$W?bPX+Ls~&vgoJ;+Jo6CE`4q7hL91 zSAIGU@Af4sD*F7EY27t;A8l+LX4@vz-i41)qpRig?21`*tFw|(j;x=D)6ir&xPhi& zeLUKK2wb#%(&8A-TFJhZyHjR^iq4bdt9-Bhu`;D+S1)0}<_$Ent;9#o0%@#9EYskn zvWmWF)%sC9fKtEq_5g~7p^gsM+@e^7rF;}1Y;pAH5i0ZGBZPBKehggcqZl`88(poXZWbQ-w5N^oC`n#*M;dh`i*LmcXXBS$CUhj@R*(DiK z1nhzMK*R9V{IFAT%fv)QNFvZUnk=zAesy9%-Obi^i&?X<*Y8_DB{+R-?}5`k^pF8f z1Y!})#p(5Y9bDN#?foE`!_d-kMeYebt#P*7a!h+qPlTOE7^?XLJae8fT0dR~aYyr+ z%xD##n;G|Y>Iw5Jf};uW3LmB#d)uJuBJuM=v=5{BZ5-K4=JT@vFI4N#&PMflow^tJ z4I(N=hszGbIA|%?^=W*S4gh~IXJtKB;K9cIw9yzj>(`g(+RI?p+w9|XQLb@~u_Qpp zfEIl|AO6&(Zwd-fLW~XCccU<3)7Qj4R@`4wh#d%A9l)GIEpv_P9&3Iu9v6e|x$)q1 zuY8g7G~5nzud=N~5C5nSO%Q#V%3-N%bsunt3_MXrHWDl1P5HuZ<*+}T>tWu>;{<;o zJEucR;uSI3wavaa29Hn-3zSr6Mh{d2qFfy_#CIFpl1c9Bgd;1=Nj}9Gtps{|OPgoN ziIlXHg9%61f3A6^pR)006hhWQWdm?b?mAd|;rD3qdLfJ*CeElM5fyUHWdFnNc!Pi9 z4WPAyw3x}mmuNs(MSKw)u#;^2De+dPEE-QJPi1oRb%{gUmOhJkt@FJc#ZOxz;YCV) z7UL4-TT95b!eHV?`+l;cD0?#R=u&z2%uvg zt1+7=V6n-gMOfO71wr-onBW^ws+ll>6?bivqDHR~YJ7&up@qzNa+ zrRa++-ST!96CSTZ-m8wEa8Xe)aw}opb?_v9y(f@1!ywWX4>*=9<#x$V+XzMWCsRx*1&f$eIm{djxim;^P|ofjEev72CYy|=+GfoaFF22fgmGr9JSXQI8g`P& zXHEAZF2$$VVAL4X-L;MRrRbr8GBbnZUVF#hz50Q5M_b2ThZ2}giLc>+Vrz#cG8b|^ zdi_-W6d{Zj<=$Bunvi!N`<_4ZxXE#agF5dzwObXV=-pZ(=<41yk=(WE3c+6r#Y_Cg zH%y#)g=hn^ow9xeWPC+<(3#6^ErBD2E_*#41t0c{AH1ch31wG)uQ}DSFwsSr8f$J8 zGx45(-R5@P!zf}33LJ>wfxZ3Os%W(YYdKpk$@s-PC3Dsk?)*(})o#3}SbSx$+*j!U z_1!c_*Ixpb%vk$lB^t%=)M*P2H+w6{u3-3WzEb^q?r@WljZC3th_A5MKchQ+Hopks-xL)M4^kLows`d!s*ZV6%+z|v@mlVnTqH^4L9C|uN3&XuVLGM2) z*Dd{+&GGmil}v8?tMELM)Vxm5f@v}Hs3i6Yv2-R!)CSNkN*~5CfxcPYME-SSBR>L# zy!ykrdWGr(*w7^kr;Vxr`diFt<d3Nc>6Gu>DtTDtm-5`={aB!sf zt&`ku7eZ?K*g5JwuzdE7Nzv8xtnz*#WbOw@fKiq zIQn2-zkq-H;vMn@-=rV(;@~Poc`(AUj+UF@Q=p5~JD0%W(pmJA#dEMv?b;ePT_QHoJ2dn}|BE8s-` z(88~>3+c$brmOc8qf90iqjb^8yv-4AZaOO3T*1h-Zz?NXfO*e+p{H@?R8#UWI#6$# znMcK-wNjzo^mu0%ax31928c6$Td3lYbu`HY=24+oc$4XU%e)3Q^$%su*20?8nbEw; zDSNzoWUzvYd#V}wTzGOx24vl2E_q|x`rAo~m73lP)npXIA-rw;VgbQ&xm$tg)j;QX zmW$Dnd`c>_lvyg^<-^fzYCnt+8s6Oop<>%yZK-I+W{$L!D!Wmsm@+lui)wREKxX@F z>jr@YP1M$R>Zw?o3YWeoR0-q4sjG^c{r<@pf93I;%DfVF%p=!MAul3tZg3jLCLUFQ z2$dAu;kd6GetGeVUCmk*E$@=7#}5up+$B&ruRM`taONkPGV#PulKA9B20aVB;fo8z zsTB|O^iib<8#|XqUR{#vSpwT)E(|Iye2h?XX;qfGQRcQ(T7SeaIAk|SVtQ1^JiBP} z`IATe9jAp$Tc4MoW-qTHi*DuVw^A6+r_fU{e`w9+&%?YEUjLEbDfxm%CxUHzEiU{_ zC#Bku4dFAuqQ^OatS9d_`L^yA1q*i|8NRi+Nq=PPFO(Cc(@{8wa0d8g`%_9AK*yZK zSgl8Exj4vvag8QdlE|1jJXL+qA*f+)2)dBZzleh)Zf@LD5Di=G>-db)z@Nt07R^@? zfB9%Xl+tr=xg;UpX^ucTmdNn>c(nkbw*X9;WJ|di(pdgEgmymv#^-#120Bs|)|mB$ z#k%*ih^0M43{+J)qzP|N_05i}L9!t821GABgp_X~lNU(|$uLOgCU*sR2KV-^LZ!(` zn&G6>VYNgE@NrbJOKr@}p){y6@|dzxu!{Qb;E}n!Vn)|%@R?9@$!nCJbjWH6e<~No4^4$z3h|;uzdaz`DHFL&C8t}p7siGri z$poz0Ug)s${Fs^vbbeC<88uJ!&i91mSL+N#@|m+ku4#*ih>*`T)zW!^%_8kB2g7-- zru_KEvq@1tg-Gxf!Rd+SV^w$qHg*u$pGcaw49Otqx5_H30Rgv-;PA|}2fnMf&aag( zi07gy2pJ9UPBkhRT>d=39N3PFfNG9vfH-A|${wBcdxd?FYAkN3f+v#*jVyp7xW$Y2 zre_-TD}=EB%y5fKiavP6eUfTG0*O%9XxCG z^tICRvmqkA(ED-gz^ZD}7xeSrJ8DAUV)E$f)RosY`lId^XKNHLhBPf@U%8b@!5XBX z?#Q0^5w(Q1g(*RJ8g;lYm)CdzoKJlYce$S+H^>|v4efT!L^&t>I-t6(-OKpWhvwTY zsXqNdgJHH*kkr?{bUdM>;USDX0`mlz(cLXR=QB0kDWh?CV-Z4f_d1;aPiRq z%<%T47gMuW z; zt~(Zw7XaATw;iW)4a^`O;-UzMFry9ldun84YNrfI^!Yv#T${I-On~lOZ=V+RUx^z) z_|+Hp|KV9|Y;2FRP?pHCIlkmxLFXerbQ8x-)q2cVhIvlQ91d5;hxeEu3*A-_{!8j^ z%uaWTT3%XSyr*F5JP{Ao4IMP7K)&A0z~+K9>y$LqRHgzhrTAn!XHvqf7-qEDQR*^e zwsUJUf^njM9U>TQgxdM!hEqsq2l=&T4x^1Q$DMowGvG2vD8jsE2mxw_@GuTe?N2&s z&U@QODEBa(LGnEtmgf8TD#9-93%Ea))Kv1Ds1&7alkPD}?|zZEPIF(nyEn0UE81GG zVKsw)qf%g8CVz5!mcG}^L4-|v8u-3n9Jg2Sbu_5c;xV;OX=b-8 zSgy<~Z~;F-aE;^KP-u9zWYT?K9~&qH(fmQy#8CWrcKgKI`~)`P-f{7CEV)x|Wy0bB z*vlZ+wbaxX8QEQHSuZ%Ni^pmz-FybP^QKIwY!EzTT04hMX^vz|5yGa4u z?3~f8zNUKi`o5G_i4Ts((-$3a{gy=&8e+q1) zU+95=-QktewNF0bHpybj9QrLSaCO2omJZrnDm^d&h{`fOu&~!@o+`G1g!8eAEevIu zhWFP2wz*5aXQ*QTQ*m*xN>Lb{%;C0ouSYbW-9&0qxPds}pC^EF@yH(b>Zji==*aAi zDO(JZV?~(#!4<$G(zs$0^?MXbUR_y&#qhwV513IL*szVtVs`dHk^;9Ex@EeyT}w_w zw~5tij6}8T9;Ir5q~j{wHkB2t>g$bTmaeVs0+-UP(bkY#Ev$|&+4f}sx1T$x*iTL_ zyz9QZ8=Lujw=!$_W9ej>*qG_XJK>X`kN{kT$SoyqE=rh}E-z_#+OSS&EZW!?7Im`& zAWuS`E}nbaq$iZ_vEBI8=KIW+`b_$u+C|y3zy(n{mcb(O$JcC4MO~C)i$Y zu)h((B6#3ge{=C3J*Cq-=Cm_Fw`Ba$tpU@Nfvs&HK3i;Gp^*7PCP;d}ra`w-G+180 zhH>7AwZs(gscbHHTt;v!9Pa_-;-X3@{3f}vm3u-f@hug%-_5{)iEapMaD0O>&ShObd&vIeNGmb%;opV*lD&kZ?QjO@d z|L83knyjo>d~YOQinONUu(_Z*v^A@sE6tj=%P6yu$#`J*uD$@}&BfsO2Te!p4N7^_ z9sqo`rgOVg9M1Jb!ucFP_cV#|&Ed%0vd6}yj>b?rUha}f2NnQ~(zv!bZIuSHmWu=w zkZj}y77Ld?58%kzAXxEsR-~PyNhs@;`codwDSP)#_=Pnv$t?Tsk3LU;2J>z`$tQ8$ zaA&L@DuDKHTr-yu+EFarS8*eXak=SqqfrD`=q8zYdwlpoDgn=YevQ`PU}hERWF8s%K*rFX^|0-RIMTiXdfGY&{8T z+-+SMl>JBaTuz(1RRS2}5V zZ~kzzWYkaN@I&ChFiVNf`yv9ePRx&ME#eJ9)8QeR0XxBwnMS)sk>Jj@*1Uq~;yFZ= zZb5R_qln7+%|fcU3!sMzQ5Sl$oLGzje_&|f6TyIK#bW46qeuk~{ZWsh))6vpYyKzI z;@zPPp`H6n1XP`Jt%BR5{=JLM+dhCdfNS`+HDBRyeOn@!BKd{7zrdO%&#F7)UA!cK z6}*N;9rT^G_0zE~*Y?+dH0g6$x>>?XC9sKyLL`7IVz}5|j*j{XwgSuAwdEOmuTz6T z@5a*tHMup#@JWu;ZD&2QnnOm9ej_QQHJVQ-%Kqca7DswU^O?~G558dCno%;wt4SE; zq7>`HD~G|Zj4hEB1G?Q(4okqMz{7i~g5CjbpAOpu%B5G7jx_)-W6!;&Ma?MZmzjqR zNw?-NVI9=j$WC%TXmp2pCdaChZmGGBH15%){;6dUM6+8@8Yyy7uA07Wci}Y*y!Xt& zti-b$!Dt98q0;b#-pH;0y^2ZU8l>5e&nPA2i2-ypn-C9hH0i82ugN#6UZAioSeVRA4REWS1S<-M^-N`oU%~U;j`IaG_=b*qU!_k=~nvS;@W3j`hY@68+POF#=6%9r1nL!v7hO9iAJNe5q~lnS~CJt)V&UA#N~ z3>8as7&dMu9ZcLg(e_5ek)40PvTU@D=E6-#D4lfDx_&D2qKKowq8$f3y5zH^80EA* zn>W2GN*N_9=(7CoMFw({cur@&!bHmmgw7C;i>8`X6Tws%ye@ zPo`z08n&+KE(#*+#4(l$iv1K)_bKZwm>}8h8vHg%RWeZJ{c|-eFv4AqHI`8tqHEmKdXO61vr|!5$ zDaGT`+aveZCVRNS6RGqrP^i6`L%i%nh4)IHW+b;?2^k}f|?k0+Ha@2)Bw$v$3opjLHE;FyZnXWdhViwTF!(E7>A)sWs28m^#h{(eO#Zg9&#e-F}b6ndgaVE%}iLnfEcTkw77x`-^OztOzys;N|EX!qJ2}Im5~~( zET>$|PLD#lCPjE8u!)7QR|Lu3o#U>&2$Qx=S(8`U5y;yMlD?}QRxuEX0SXQ=+yVf9 zLTOr!7zi}O^wSz$pn&vC#}HKDOY`j1=mcFeXbHQtB`t5AUQl+}pR_WD1?grC+EZo~ z%0oj#uQ4=;A)jiAe&bX+j#XYfI=PyhwAq(ur@V9HTdOQUEQS5gVsG6OviWi`+D|nX z!9hQR)Om#sxO;bHv{UfXu+^a0s6xRZ5C6_KwdWSK)PnfD;8Y*XU+mSj4}tG;J$(rM z)9VXdhpa{wDhg>!R|Y#hxdJm^+W{L@56%lLQpT^|zHi@^<8lGwMC0CqyD!WeM@79= zO!r;;_IHy)`GPb1?LVOlk-(!|q4TK@;P6u!Qq!l zsz3Y*-`Wg}2ICb(+QwmGzszokux-#lZs@uL>(&~F5u|JjXWC_H%zWHN!HX!^KrTUr zJ7Yv~zQn*xs`~3i9$)k{80(&S)L@+wg2W@J_Yb=K>cYL?M{V0(4wt3QJI{djs)1r? zieY=5^|0%{WK6?WVnKb|pg;oB&WTEC6T>S#O4FG*5P@}F`;Aek$8?PAb>pGp#M!A% zf^k1-`9l8v8J(IanoC{s>L8ck+HOeZ;lT>vrDOW@Z_l?szOEFpQ4Bpq_onuDkm1>G zr{)Vq;j`*VH}Ah>;5{zGe`C-M8a)bxHQ=yjd{fr&!CE6vP;FfqEc|9pPEX}!Z*l@x z*Jj+BOtpsvJ6-txUQ$`6l6Zlz_d`8vr=~wg32u3<%yB#MN13h)+K1FeOm8HU+b6V+?CF z(A$+c2QipZXm;`D_!bJFM!H1pQj3w+v7O8BGThg@S&i9d$DXkygm&;d$ zq#R{#`e#6QqlKQ71%-+}kVH*(JzOr2a)y+?3+qoR2%RW$YJyZ8cx!|Yt3QtZ8m-nl z!^pE`xv}x}{~x6Ir0RnV2^33vKLxm!Eqgp1UJ{KM38CY~IcI`vsI3!W-q}Pz0$sX? z%(^964}au9ry-Ee#ROU|Ahs5P52S~83j4m?a<2?3gJH}nn54<4QHwq9o?B_$qjfp5 z)5%rp&pN>6x1LDrfmz4WwJs8qkSH0lM;8DGBTF9-{g`fe!A#5*zGveLxyd%>ljSV=+rNQPI;xHBXp zk|7Dr?_T)NDpZ*0Ip~Mbn#Ppry?AGVz>olixS)bKOT@kFKk)CznStH{ksCtVl&_uC zPJrrf4@_0?6s2MC5hlVm>~v}e!nFV9t2-F?%$dX=2>kY2e==8LQ^5HvO1b!(l`KDB zpqF?7BZ{xzIsOV<|A`#XAQB##)x)c&L$Uw#yAum^kroKbFosN`rQwj`!5ZuP%HuS)gqNhb%Mmxv;Bv~mwgQg+FP3LHo zB9iRtifekz(+=AIOT-TWbioDc*(76xS0P&T(1^ZTh_m8p$qQK|b zZ?yB=Xuf-z_AqIH-<>I0CUAp2!v}iEvb{^YiFQDFc-473PH$7(!+bS-GppqH@0UfG z%?v7h%XpsF1x^C!)7d}oZ%jJ67ftqFey6bs_JO>bSF=5CqVZ4Sg&@-H_(&l!Gl|mw z2^SOLaFl9Jg8d&>c;+0O!VC0eKP(c$Y=6DvXMiT*|C`839{rCar^V<0PmY-2Kfm1t zE|!ygDr|u?^~56d$&ZK8KLg@8KETDX6(!sF-f{U0x=2P=4nFSrke{K2gtCSemi9fN z?4R&BjvId3bvJ6`RO_e@QktJD$H3hg+DIrb@%tK7nQgykr2eq8w=fjrd?T~@yP}AJ(^+AN&m%&FqX=NM&1%J1Pr|zJ0E%>sjrG8sJUfuS zf)SQhr<97|#3GJ>0Fm%nftJn(jP#LPv}2vzfwGw(HMH_GF=a4CfVXY{s{iWU%Aj~*U0o8T- zRiSBbtSXV|&Z^5Oy$x2;cmGaS96;Da?6g}a=Lf1sdCP$5mSb+S4Nve=CmZhTYZb?b zzp@N{i#Qxe%=slnGjbpwNXv`^ndysAms!S3iK^e@`3&*Gi5@!FPuxcZCmaKW8sadT zhULcn4nqz|H%ZcdTn1KfEzfKEA@ZT@*+*6d@A$ln|dkV;_( z)P3IVwfdgMQ(AiRy$JIAler1^!qXJ{+h>UX&96@Jn;{E;uK!I0VHy6%k<+sFKaTuA z%#lh>2XB4dFVG`Q0qVppzAD46hak_t1`$L!@WfQC%8`{0IcWs{CAc^M{=n4z2gl9) z5#R8e2+8mWaDL*WuKlJD^7A*WodqHm3Ge>P#*0e!eWhl6e#)y;Z(#7PY*V}Jya5q; z|EHM?ts9-;giUc)nRFIy7J5(?gG2OE=XJA+%#gH2=(VG*gQgu&L0g%Nj3d8T5&OaW zZtm`bgv<&xg^V_fJ&eWPC2|E*b1IKuKPD;rb3bN|pO(FE>nT|RAh#hd24I;M@p z(sl5ZjV6=tVVZ0oT5P^D@L~mW7{ct zdcINxu-gfXZXN9D6$Nyid+WQyJxx z+@bOE+1FPwGOsyoMpx2heSBSdQ(qJ3H0d}Jyx zzSKv{yT;9UO)tjJ54@mjle`$#M#BR&qpweB#Pzz=XjN-w` ziGKVbltWu3^qw@Ccb_!p)#gh%%n~&8ej>*75w9m-^@Y*~;Wq%opB!3`ml~}1VxrdK z`Qo3!xltDea{sg(9EpRXIynvB-Cr#10SMEPwv|OR`S|k z0y-?eF?c#vbi-Ge1|#!B4)~^lp@`9M)?Zo|0fVvN1}vS%@1e_77@2%y?SAS2zLTfU zo$_M;P6C9%z<9>gv_G&ztJ3Vg_#R6@sd9fi$!|KJYAhoO4r&xsL#Ol=rC5aOWpM(f zSJ%3rQq#^(X88-HAcwq~ zBu}exDKm-M(b!?vAraOXRU&zsA*l*uGYlHK0uP>@xm77+dNkS7Vj30BkZmY&Rt4GL(+Mgi>D(!67VTa(Y)Pk*K)s;<#moz+ne*ON- zSVDj8Q4=7WV@5gA#5YdsriK37$~2X?*)F0?RTfvIAhDlc*V+yQ^w~;K2?UqW;?==K zjD5HlLrLnuDp&413)O<{J55vW8~^vOn(+oqSG&H|od_$z0}ix7^$3^C{}q?_M0By1&UKq4+wr-$Q5n7^e#3K17W)Bas+Gjkcf z{kzDdb{Z&1@V6I(Zx?R2)GysH=0?%SuGL~YXzfuEOKCEt;k_~BALTT^ol#Q&No>fU zr_KsK5|byv#UnE!+4?)3J0+l`eL8?RDKd$j=ozoWvicFA$RBTi84Bmf>e}o<0!Rx+pgX8Pjd&bQd zTXS?S4CGMg-NYNXe#dG-vDP+SG~BNiEZITEN0>uCe?N&jhxs9AgWsM1FMh)J1ZFVm z*fie-IU0I{?=Rt)0(rQ30&&}#Vka~Bss;8V+Y_Emrwgg z49bA+cf9?vYz@IU{{64NzTrA>?xv4-^&g-60UHq(_9AdVl=Jxh=4m)ac;t9W%#Fsj z-+BT39hKoo_w_MQzk<{QtU>#M1d9~($L7ZaIzAd%LRUZAJGfyk@qb8r??9^i_kX-j z$DL6qGbB>>EGzTegh(iRD3CzbYg z2RtqXja)5!?{?gg+7uc}Dw36P6CWtQ=C{C}tGU;mQ` z3_(!k;xB33|Jy4k1F&XN*z&(_EB=$!IFhFZm0mk?pYZ?r~)ZSXW-!n_{1O4_eUbv7uz zblGiSTc_^=_elocx8-IMJJrqVuQ$KQ^OQQ?lM`@X+p*;KC^d-8HWb}>$b+8FPo28dn03B3<~#G^TFvL zM?yE57%!I@A}0gt76({5cLXbzVphs*rRZugf(m&AO4r5QDlf6%$oqHS`Cp=2O$7j> zKOSRE#-Jhrr{K^y=5uGTOy=f`(VFGsolpMX-ny*^Ok8}~;bZY2t$*UH@n^K5{0my; z**N+s0{A_mW3{N?r$iOGpSi~q+9zt?jE7ebeLA({8)A0)bWpU57D(8K45*ZA+vgs- ztiBps?$Cgu$-bl;@{Z-1vQgnUq9BS#Lg7RzC^`rbxIy~3j56;#xvvJnu7Mvdx=Z69 zSzgE9njmNxe!h%EB2BuCM9IxGbKF1p2Gj!5E}&^S7oCHBbPJkn*%XD?8crmKN0mD? zqI7m}9OCwMFgQusz6b)+NfCG`Lc@#m^QkSvqD*6v%L$LMYkaXQWU;GD-<57#5XjxR z0+u^IV?I=N!iL1sNnCQ-5v;|iY^!_y&J}Gw6wt+T&XQElTfjP#Qugg@3G&HNy^N)( zeL4aOZZ~$;Sk!ah%`~ND`_t$GJiS0?w7J53XMFnhb=x2i02f5veFKGClTNPcU8|jS zvlO~)076wT#4}et&%}7RUpVP8{|8*D8Py%p*cJ6y98?ZO#LXH}HN%Pg z@F-ZyghhtF+fVrJaRMESPqdi@l@agqf(g&12wqY{`WNq?Wlla*B4y1gpFsd?`} zVpjs63~p<|chF$CMviVqVy@FelHRG+%MiHLgzY{h zXX14&%?yw_d;9I&*1jYj%$?(J-54Xyc)1eXI z#>*WD-7QcuJ2j7nw$ByVwK}Xz_dvJ*9PHBP@s{l06na6s!}zsEmw?GtAY488+7e{Sy<|2pn(*ycJ+8;s&qJ9hSeM-p1tM%&AyC2-yqZKe)d=8u zMC6w=+ncugyVswYCj{g9#URHm&XherRx$^XX+r6k9f)EW`~XN_TDr~zNPljhtC50x zL=EE=+f{C$(&hXF?)Ok>C1&cc&zzqT%anWL)u_Ygdu+Qyyy{{}S67bHb-8hj#IniN#PfGgZ!vb(YeDb9d z)CNirDjN!|0U9&{qdPfy1AcbgLWF-nw}`*IP@Nt8g(%QT`|!BJ;VtZheXbzCeID~o zWzZxo(*TsxW9j-uKtK3FBOI508&CyURoWi_evC&2UcDJf!=Cw*%GUOlVrGKXP#9h9 z5RZ^Pb3qpJwe`DpwrA+GlCkzvy_UBxYSOme2!qW)8UeHF4>-R;)LS&dCrRYw6|wpb z3dwM{S@Xz++8mOeJOvPACY6-v!QE1)cWuhuW^P63*^rZ3Lg>O`LgixhTkm3}h{0lQO4IXt|(ed`TWhO8QF) zr!yV5cnpgt+O^#|J=7l?yixFQh-@+)CNj}h(D}(UQ}A^XC;S#PDiVpM zPv%IX03cHqkST$qh|n5<2D+Ic>~P5^OQ z(4y_oG>o4>PYVd!(sA2fa$}*)-UO9eKLSh zj%QB9<6NEn{b{i<3?aYBQChBnvTWJ?`NAH1%cqSx*}8Q^iX9oSYD_E0=h<`9NLPYY z1fUJsy1X}S(F^TnZ27cn`Nm3gJ533)$Jx@bYdIe0DJXa!B16c30Yk&4d;1JZ<^c?U zJRag|H>#832I)vaxbJ<5_Jj6@yz;6` z6A8d`yoJK=6OP6rawQ`6{S7ug;4OeL4RFx1iMm*V)$1+k2Xr>B*Lr|AbGJM z%@Dcz0Co`)*ia^0C)iH40UpbRU0gv%x+TxUL4?SE_Fxc99x7pn;8urMczUh?L4FS= zv>}hQ@$|5BDtHZ^#NJf%+$H<|hO5KE=nvPgw9$gPucCi>;n4B+woF+9QfJjc?-=_Q zy|o}lUU7(j+yDoiYi*I#74&MWAqo2IF}2q!E;BQYouSW1vL-lgb~Ke#4YkHGW`4Z6 zGJI{Qx@z}N_|($`jr}fdT|Anttvt}sEn={Dwgqu8g7q|CPqIl3fK|i9u_DLbZ5goJ z15Kn?;LAA~A`j-4Mo42rnd$tL=@Ad6maE^Mc>ymy2|Bx0kW{bV{5(3Kj~n>S9;P>5 zQ2f=G<$e?#rVClOx8h~5j}5S#9uR-g^Hp`Jy=$4kD$uu1h?VJVi!{V#!l?hAIKPAb zdC<>wr07L%jjw6o>RRqQai@4HpOSuVT6HFZAR9~PgM30i`JK^;9@)C(^RNUhxa_zg zlZpRaQa4a~2eg1r951%(;s7UlB&f>|VC?${q6dehk~19$g|9@F=rBY8kPeLZj}SG% z1%&2}Jq6e$5s7%O?YZ+O?LR+7GII#h^KMIZ%m@>I@FA6qIwy@*($MZCAibl=Kx}6$ z#Cq)k?A5^-yzeWSK=B{5Dur2F**fKSd%PM6ty`UU zwz4uVorNV(91KTyD#VR|G=lX2`cJt;rT>Pt2U|k9fjBdfWnfeqIFjgqJ|vTB6--x> zX#VlKjF8b4LoPEN6NFiMKAp#N`c&gqFk_l(!}*^`#3Oox;K?hjzMhL>@BPV)A6)s} z#cCFank9%m4@)ac9=9wAf-v;xc?;*B*i6si{+e<@r66gDdDuChMh3H@K2WlXYM<3lD7ie_;R;d8k!;!oYbqpknc} zuo|(lMuhH3Z)fu-iRb05M-1$q+&?c+OfIvuJ-W7iu5YQDRNaP1ba*DScsbiKGM59i zx(ZXA?=lj9^yo5z%s~e?p?1Y}(4ae{^545aC$EB$ zd7N6`eq@k!t9tNtQhl?Z)aqMOnRvyHRjsK0rKWEB&xq%b-dx5%@;Pbj^V2b#SOsDZ zs@jXgV1wpTXD7Ph1(wfx;f9V|ALtckSZ|3=Gr#ijScKpxtjQu z<}L5~Hz{4{L#J7nTzBZB7}y(+M{$L$-1<<!CE4;My9 z(fk8Tn{P5lLc+c5nzjsgYclUjV|D~O-8Z~7rmrnz)GP)6!QN`vk-fX3(=Ze9MxA55Mt7k%s5Huu)?x?ZwzD_w~LACEd zf^$>8duU8`RIc;)nE3&og9}(Z{`*or>3CSEM}(Q-yX}|u&1I7v3Tg$fQKL2T>Fn}_ zBh`=WO*6zNa;WxdDG(Zs@oyC8Rlzds)pb(eqceTa-U}fvSXQ{K)seg09jhY>|Hv4l zjcmFtP#+%WG7F!X4*gu%an-Jl=4zRqaz7Q18UiJmf6I9WOagb#PybjtJ$W3?s)XUb zW~obeUUCW8u>U@By#_Si>SU(kha`F5M>hvMZm(6Sc3;XoH)U(B_7r_9xGtur`O?J7 z(f4f{0)zvfiX{mV7wgH@7@|(a#-H=)&_5wSxFk8?*r_iXeK(tGx(bj zWpnPvBR4fY7IZO&S~;fqbR!oCfABgR|1zq>>pZDun#m9^@%MlKaKpd703nwShJS?L zK_{ddpU!J{>0;9*Or=e-yl?fF2xQo~6N{;LY61-7?9tOB%YA%VUluz~5BQBZcYfqZ zZ2CAPZDwd&v3%XEd;NlZe=#d!S-%Q3`=&vjeQHej+JzViVT!U7wL@jI&c+jrnw}eN zNCUUIT-_a`7M~v-WG#;%p`$#t?+dFbK-tM+((^wP-_i_1;T`_%6r@h>&SBrv_F!f5 z1+tZHDRS?UM3t@21@R?|A&pj>mh2TsN&8GYD>fd^mLTa<+19$CB_jhXMZEL%}{>T7xD2WHQc zJ{blS04I3fnS&bgTOR*vt0M&A73P-ORqf2S4Qc8Zsno{%BivuEJ(d-D?ryc_&H7R3 z9HPCCB5f#X2tK|2CJ3%_T1$ff_k3|$zbFkeA)x6#)0A#^8Vj_rGp-F&M;NzP&Q00KXl&hnUR}XU>NWAf zYiDI)r-Gz(&q8CXwKE>8wlCCK4gI9v-(KBnMTTq-*r}bO3ORt$(ZC{ zs%k`T9={v-!ON@;qN>MwH71$bz+4GfFI4o43cc%5hfm>u7Z8}5l4m_M7D!tU42S=) zxc{!>)nFwDXa9SIrI;Cts-1!o!>P$KvKTEqTew+QkJ$>+8w-(g>T~RD8lBl59a{qn zY=>n&yE1r%o)ei@UqGMxbLoM0(Q&l&jqw=Qu^PjxJ>8fVWA3XpE0RHFg6sLq-lO?k z1VJ2{`kSvvWp;w^+RR=juf}Xw30MjSiR)rg746a_Z{V(ljo^l-MrDzE1O;hV@Ev_| z_6IWm^`iVGL>?WPVF+z19eg-;{k%I|*$XW%;Ir!OI`_3d_d)o)JL6q?lNa~j9g9B;+l_DY|Dv|1*8P;2ht z^U43N-wYsrKS}2%aPRo~POrVBmlnBqPPVnbwEIfUnsN#U8)e?HTHtfdJRDikcWw-A zZ_mjQjdN3PmTa3a8(foku@m^ztTyf`d+T(TnU186ATP^Oz%=u1<}+)18bD8lZ7fZK zR1J=d;kc#A0GJvmnzRxI#S(;R;Qa80?lunWwQg;{6 z6zf_Sm~?LL=!%z4r`eQ{tNUVD5plNOjBIebC61&)SFYa-nv75L=!n8HkNnSbWd9WoumpK{ zhh*c|+f~*xG3BQa8gtLn`8RE{bpjBmtDcLGY|5h4GrMLq-50VEA&!hTX-;0`ua8KO zHHv>Qh+OK-EZv^I_W5CmW}u|xy08v7+;LesSiEz$wJwy3o50RpwvT3blS)(Hb1KoX zKA=!*T{w^?4Yx4i0EaPh2rJmmKD;vh-?1>M58}-d+M-_Zcp=`2;1?9fCOV23we3fS z5W{mAEu`X!&S8s9r1qK``%?Vv&`8@7nhQJhGOg1D8v-t`FI?Q#yD?x?(nHqnqhT}a zFN`{3kJHTz4~ zl#CRviMRYgCK+)I&=YJOOtp@(p>-O4D#>Hq)|GPseGkDqx4UFb3EV$qK%=cNsF> z-@Q@@scc=$^zsYI==Zp;rmer=8fSUr$ffBgRYJ>O)H&Lf?a~6faF)P>{k>RC@oJ}6 ziKS-##b)A^U17?=UCmA{nm9?&Jh?!I7ZOu0nq$eHdHKb|h{Tdv&)|ht3QhN&r>vJH zPkDLH+PT-IXzDw+e6>G*T86RdifzCM3v#zV=>z`_OUdfDC#XztZ2Fyem|CDtrdZ~e zJ=%+0ok$1{8~?@lBw$yF)}J0AtseT4&RBDORZAoCEV4|i5AEEc;BWUtNvYdvb1yRe z^bM!6k05TpBdfM0l{2!{fQOKzley_qvNOV}7o}%2sam>6nd0i1i39wWQ*edeMacqY z2Tmn4pYg8^Gyg;xKS#@U%BkqPT^^@TTzN4URBthfF<>+IOVtmD)X6Evdbd2zm^U_w;0tb6u{~KFy~r7rM=kqx1 zT;Fe*tS1nD&{)rL4Dx^4J!x|`ed?vkO z_ps|!l5_^DzwqWt$A_0;h-_GZSu#Uy9#@BEGlg*(;_fSx$Pc~u>PwGD=gl5tbX&_# zPvlWW4wPAsbP#QD>bdQ-ek6|5dv7-~%|Ckes^5H{ebsS8ZnhjTxA6}1OAk76zj%1cwyuRr?|6Li4>Yz|zrLKqEu}b1!nzXiEhvPbP;%ehdQ+m+n zsySnFGV{j0W25x$eKs*m$>y=vl&G;C;rh^$% z3+l5uqjjV5c3qh2dycAeBW*|5IFYJECqLuj=TCK%aZ(%J@6Gow(2CH3 zqstnFbNNFgvm9OzekK$5*O_~4P+Ml%86;{PR+6H>=b-U<+n=Y@`C8G1;Bi!@>*^L^ z^R3-pQR}ZuLlpNELI7VB?8tH8U}OYA#SBJ}Sh>^uLpur11N$?^WJB145ptwmnD(c&5qinaBC0KY(aoy<{;mA8WBLOZBuz%`m+ExxtW*zB(;hYc+L0M}-zBG1Y0hdC z$5!(S!#Ni)PW08Bdl_wyei6d2tiVN5OCr#4`XT05-LJ%oHXg4p1u}koXVF=i*B8PY zZ(X}NY51`p!$EgY&zNV1Ti9WvZYZLoFP^Wbr$jpzk{dvBmn0qt9&1*fM zZ!=1G&UNS7yKT(tTj|=~y0~H~mHOeBZ9}#dcMPdu(o~5{5?khgGpd`M!KGiprSZKO z$=l%mGaYZ8L<~&_}f8hMb(eUQ@|jfzM1bn(Y)_Qey3!$l6#- z#qysAWRCTTI_=K+T+@|&{*XMr{T{h?KId2l^OS^^3QH{d%>TS}~GPLJI(Gpe$bx4U+8`kmoaH-1^9`l}oIq1sH8 zse$=keIyzTHJn9=bpLd|jXe_+=1Y>XH@eV|tEoM2SNU)>>r2!+WRInB`qPG`oLLbe zvMF2Z&?TV@>DVKh`BD-|^W$}(9K!UB*r1^ev$pbNN~m#}4Y;@7U?-%r`1sOb9R$W%J5s*Mi+8}wBjA+@=`9Vh@tK*h=B0+5xW(l+_I1mvNl4nj z776Ej=0kbS8LGPUkvR0Iu|KOuV=ggt+<1_96o-d6xBO3X5=emIYW*rVkw$F5;4_P? zbCox{U!0%baBiZ^>E=+J!^siR?nXw_n|pR!+H=bzZDGtG7vG8#b@rF?zwMmpc&mA7 z4M)!=Wq<({Eyi##~cSx6c1-o<4?3(0=cex>KWZi zhI9Hm%i)He+eQ73dnGcF;iap7s#0Mk>FQf@Z6iJ16HalKu_7OC;54+s(RkUbx(3@G z9~%gQ_Gr@ndkMuo|xIHi1{?hH^psg-uvn2fTOn|7)sAH^EY$3BM&MHs9P79aMJe5-k zbHJH>JsfiOzsmwfyN`)vJMBcOD=cTZldrqyTb*P3vS5jJ;ZuAFZ}LIs zT_z5C#+cZLXS{^P-7M5A0_9xmr0Mfc56rZ^wAyRc-BM7#<-RfB*Rx>2h6v}M z`QjfXf_+_C64zB)gJcYp#HgINybMpFE{N->lG?v&ucv*d~TnssSJxv=V}6{-q+81&($R(m-1{7`eT$X z9dGdhZsSBi>{P)2&6xb3{of(@K2;T&-gk>`1sJkNTWHi9VEh6##S+u?kXt;SOX@61 zg_!JgPa4Z({##Ai_xi>MirWku9|eAbi_a2 z(+RO_(4L`+yf=;G4O=o`6fDBy+Ov7OV>mYd@67t;KehPad~_dU3&S@@j#iy{*~~5a z)&POoVq&2V3!l%ve1G{=msg|I`qyRGE>Wks!gU51OU>~+2gIa;RQ zVG?GdESZz%THBp#sTmqN$Bb!f)ZFvL@9H3)9u0I~(iCOwRDJHY;z7h%b5vJdiA4x~ zyUxu8v3pj$nbyWKd_XzQaavAUA4j|*6rYyU+h23lS4WZ2=?;JTg+Dmp{F~7fIY~PB z^+fw9RpH&6H@zB0Xhoys!%Bu5a+)#a0A3vx=Kf<`*3PI7L!wvfJoQ(7$la)sZnc$; z@zZatcudy0ImIGM9~QL~THkjiyeF37=$Y^FY$QkN?x+clWK|+zlvuY771qdOp!sh* zH(-tFMRs1H#FcrM?x~sR3B5w|`m*T)xF3h-7J$&o+=l_r)DTmrf&4uZA*=3fW3~r2 z`sQ1lZvDGaTfylA&SN~Y249!ZZOa_wS|6P%pacYHEpP{hB3(&$qfIKf@ll5;R+EFEF)Pu3=e1SLMHL zz_#y)BOJjgpBM9vtj~7Y51bD-SWDX~SbRl3>MX_5t_=6%iHvVH$iBG}hJ16V*{Mdh zwS^w(WUh+jZr|5cNpW{MK_BN8 zu!Z9iM{H2yi+e|DdnyU+ZN$VjiD?^vpFT~Q@g~AC?eTRcD1JjeY#WZzEOp9$=TH!Rd& zbTjg59p0EW^2k+6V24}Jqh2dZ9@RXLJPqk1adS z6Vqj-=!pT|PZICHMgIXx2^bsxAaKI+y!2o)tV2tXn+PbgY(n%7GCG^=h*1KR z`SQ0vE_hU(^|mRh;11z+*agB_c*;TaAsXA0*9%APlQZ8f$%Bs?a^kl{U8Go&(e(7hj^079PB z(b|LoeNiX(VIwBA3gLy_kMDd#JXPfaT---8V>?SlI63)!`n*~V<9sbe=7eGuv+t^n z#H=|^I|RN2nYMBT$V5l89-B7JVTkleXLsL?1UJVkM6YQfSAi4BA=3y~yEOlnj4qhz zIm~Df66={qd1PrzWO1zJFgy%0j|$_zfFBV0z|Ao|tjY-gJ^AFg0T<1Zz3o+nQ_w%S zTN?MK4Nh;~n|uJfibaGe(|lL*V-ENb07P@4rf->q>$w)}Gj}`eVvimEGsrkw31E6r zC*ZT~v2U|W>HrX_SL7=xA{O+%uCGb&%*(S}8ZV=`d35vN_ACz0+i+P}TyCPDxPg1MG7Zw)j9gsm& z5Sk7(qT5;f{!$nv%PeO^hbOXU#Z@!E;z+025@9M+ULwN5aW^i6d(a?um4Y9weF8g5 z#a%iv4$&&H+Ur2vs1lJNjysAuXR#ROw`&$WuSamqFd^ElqeNtPt(6I$3=`V0$O)Pm z#mB!#x1>X0O}7M@JV+KFQ^ul~*e2bT=2o2CRiIr>P7Urn8$w)uwR4XePQacbA?*ZK38o*iV*@j)@C%em*ocVT_7Yh#@RN@KJL1CJWtl6~=Z*Q9*~nhju$H^+F;3kzKyRk>v!oEG}4C3@1|xq4pHHl9*^(RRV)u3DsI^o&|kLLSiD z^YDq7#rL-N=T^0&vI|a9)53@QIp!;r=;u*NFJCgMWWczZx2<_im278euOERHl1`52 z9|&3#+y?~RyVgnz1pNUBI&Nl^00??H3JCf_{=sRS<&)UxYGiW42zrFv zd8?jBa_^&nJx{~*Q1ui7A^EJdY&)McyVjdCrm(n&x(fcR+U-W9auQFWCue@a zUzUV>5vRe{jtM8>IcT(7WxK>W$+pGJJd208`;f0BN|TlyZ17V6`w*WYQ!fFQqNu+B zMtUBVb}m;M9JD%*s@S%O1^+4OMaT?z5@%ZAy_Zg~ha1dmZdgo|Dx6GJNCnyBa}DhD=Nc>z0mCM68q15f$&O_*PmjxY%!+j>N%rEMgA9Nt@or zpFq!?LK3BXQJRjbD(M!8B!s$B<_}&SI77d&#ImMWmbyveHJJWx=S*q)gr%V$I~gn0 z4>4Aw zK$5mO7)ymLvBk+S+AL5bznGCW!gsTAWJ*R3=1VN9k-+wW3NDohl_>g+Y z!GP{TEkZ7;F?|m|SD96yI(tp#lJ>%jX&)%RfL=^8`0$xUf47Wqkk0qmuwbdHBb_u% z$k*gCaJ)WnJR>cBa3$A9<~Zpco|XkO%htl?&h?sm1Zs?f1{qSus+&Z5Ux zV^paw(Fv$(u24_+5yXE_l{)O{h|CiTyP7}bvx<26v1$8~7Cf<(`sY_uVoBGq=4y>bQfpZ}}`%okNaeoGm zeIXy+96@;Ux}#eaqdX_KLkN%QRrpwS+q0UB2B6XJ0^>dL2M+8ZWIFhzErXrX=78*P79HiMI>) zn)`Qs{@mb0%z3S;F7&#gb3C&tZ*8_}3#8uee@M%5^UFa$O@ss7s*S~H_h6<4^MWjt z&8=jWQ8FX$2k;ed43!ax((l6ZuO^J#=89_#N-FQlZLbOqcDd>NIc>s+6ETmZnnz{n zSwD(0f{$%X8+z?>4Dm?pCY745?oPy58E}st^ytW6MCQ>jlEtn|VJ0^Afmk zUJ|_EfA$0Og6@p1y z+I4aq$XNQYQdY_Vok_tg4>Klmf2)g1#1_p+9#Sxux@oRhAfV9T00|gyM+86a{YC3Q#z5uJtVq0 z*yED>A`>QsC(UcOm&J$5VD+VOw{0_h)es*123%>rvrX2Ob*}~ zLB1x-ovdSy$#}AkNXki1;<%{{vt@DofN3lr-f5*Zyf06kM==*3u;C*7dLLjzQ->TI z`4j@je@SQ8>K(Y;iP9?khyVd!YDKKVNk$GU85f(T#d(8jm{Bd9Af0^lG{{vz) z{Cf@Av&X9zT%=sddxaT^Od<|!+9sqlP}tpGe81BSo^{e0b2okggP)QW^}OQSr%WKe zl(a*GuVh&vr)j5@?HX6yKd;EO(Xh=uFHRcsn{W}H$YSx$X%c&z&ari%X}=|i_bi-B z-vPeh7hI0U3&(s$@zo4a|JgaxlQdYfq)*FqP!%_3;1&AAnDU%ei{~tODzkLTzYRdiTx^3d! zZ`o-F>;(NPZ}q<@3{I~7JKta_kc(uatod2(T(7W;4FKogT?YI*c})5L;bkaDJN+og z$IwbR^jUf)S_sX3<9Ry|VB2=**JJ3LAcq^y#S1aAw2oJ%9%lY zO4a7R4{3dK%vKMul&oD2wpA`J9LFhMV}`uQ~*eJn@9__7r9>x}qUH z|LMN~xjYrh%mrXOl6v3Ib6=3)tiH|c;nmEH$Y0*u50&s(bGM;D{d?ZCkRswGRFj2ZUxolx|0KgbMtnn?vz=`HJ9Pt8y)A+)H%p z?(1hJN2LS&&0T%{B-!Rv8b&IfRZDB9pZ-LW;(bK>$x-9&kZN0o4vVA40VFBeM``)q zSdPHBNRAR$Z&$6=48y4@H-s$S_EvSFrR`b7dL!yLJhpWQT!m*#3Mb9h>5tB=+(gY@ z>k0TbZ7&zbC;u}Dt7~nYOv5$-_rxC3%DqL}9E?O1(lx7yQvAzHjBen?m2CxMWH_iCvthE@`&-O0$$ku~KKmFz6G)GZeg0uqwuX zBer_=%ppI^MEoyeF=3~PJ2=~jlR79>-Ky7AEGAUwhhOQq#&*lYW2*(|)4gXy0*mc7 zX@~eKkdC(DBRq+XNUcKXF)TQM^ z)V0r&s*G6nl!eVVU5`xj)zh~xx!&$#o8lb|Im&;s@gJ;DKmqIo@F8@>mJz_3pU*gR zz}5t;+XOj-zxc%wj%VImOQ(CE$c9x{!`Jix0lLRQiXegNaQvf^)VIG+oJY{WZ{lG` z8O3FM_F$B+v29Xc?wtZh)c#ajQ#FP^SO~cdx2gZ28vObR`jUHNJqjp#BKZ@6_(zGW zoqs#4`v-wW$Dn_l2A0|G7-bjfKCsFIuq{xn-a`93{eO3HiZ368Ac71tIq5_svR=Q` z;eM8UR-qAuVju@YUBaUYd3yqfpuZxt{5H`scC4Acy+D+B&~6(7A{Dzf&6Z7EUF0Q# z@R5DZa*s6@%qOF>XA$O}*dGQH1?i;Z4bRx$w~^GyCro2a_Q?CfZ!wywYB|+qGyA?q zEFZpc3Vw?0i4(fjvi+rD2%0<1S6e;o!ufDC=uML434YC(W7H%mXI%Of>(9JZ z<(Q^4UwVE|MpQZ%DCo<)omVBnT=Fzu^X&_u$WoZY$Z+ZP8HwBO87Dz;o`PxmwDJXSkbgKFJ%q=FjQAY*c9TMqORFLA?divOF zE9PMrf+cve+x=hbUChwlu)9wrwfu(SbS|e zruce24096C4vbtOoi<{ySR3~{mh6#%)f|{6Br+tRX!%COZ-)v5zV`5JBi`3s=Z*78c*EeLh$zm{E+8{P=GBqadDa zggVsJm5yVvWa73mx+T_YbxBUCS24welEqF?dAy;GAj+=#y|MD5m|mu^dXDh(izpJg zOg;l!Vdu54+$A6~D8GUix4pUeo@8LFwv4y!Plv88mePp$r3z(J<)WSb*@j1fR+-Oh zU77SH?d60&L^tPgek3cL)v@~yv0r5gBz}r?QUDA7V^+kf2gxzAbGECNNHTs?>AwX1 z@Wc3GtF|Kic8W*n{8d@RgDA8drSw7`lJbbf6t3FVS2ed|9Afg)ty%}$$i#x@Ie}qdEmHSa8BJtYDJ$5_+(z zqgu+Scu<>EOelXoBF1bx>hpcSr~@MqOnCbyDv`(4EpGv4v*vRhF$Aq0iI}J(857G4 z+Mm8rXTsDMP}&W(;G?!PXj!kAr`T@>L}NcWCEZUDG?JHJ^LtxLeG$&|XOQLx z8lX)Y7upWs1;0+TDKeU>3B&v}ruf*L;lg4KDKWPO-?NJM*iq`wDA|h9%Mb#7)hH$u zZcGCz_;huve!ss`Kq8Frp1U;R?PAxhuTUXws-u*u z#12oWCZ??QOz>H3Nz9m81>`5Snak}O!a8h!&rxHJcVbwSedP}QJ}s4l?vC5&;`Hx; zoH7Gx4rlXrOzkT^@ro?`zJRQOuC=EZeNYHd3aeu*W>)|kB2K~!{FHjG#Xnk6mAZx7 zpZ4p-Tbh*hA)TS^Ry?`5$fWOpe04Z_)zT{wDxTI z5{f5qNOZ$@(HrU8@=03D>hUwG&efEBN^=)>O)FI3vRwFyqs{+N!|UBwXBR%r47)rx*u4}J_K=Utzj!#R&5xl` z_PN(+x4Xxt&YL7jnRliq9&dR}crOO5a74WUwOO#+h!I2`PdNFBJ0?iV7>B?~n8^ZrJX)lc3aok7*qF^R9#0OReYAV}Ci-Y7D|g z$`d=+IUxPO*vr!b8{=5O$yBfOY>4GeYvc+k!qb49q{avAd<$&1|9vI~h>n8Yy(2z_ z-3sL=C;ni5rgUEcGw|~>z|4+eS?I$_XsZ`ABY=DNZ*-xIRWrm+O#P>I??&6$_-r65>I4o67uiTxJX1VUTWyG@zr`?D z@zO*p`1~sfg09{a|1T?$B4~iwP|bPCjN5F_NpxGg!^wI0)&N(-GGT=1{x^jU5)4`i zf*5Z@=z=Fgac=($4{^D|i-(r^zs&*iRI3QVsv5bebeI)5Sv%E@St2ou`I#GnLkV80k~3?M^hBp~uotskqv@14$1`@?GhDW$$>FmscC zpJ1cO{py6>WbE|52C-A+PnE-{kbXQSN_xy_jB+HqAv(xMs}d1iyTQu_a$*2 zAR*^{_xNAuT)0E~DTEe(5MUU5`GRZW$|DHWPQqG0)mmxz90-zGH^HxYLUw+|CEE$! z`d?^$PX_w;zE=vqeGatD_#N`=B?uzR?)H(#&bLC&s?=wf`bJGCY0YD6^*_goV_0#A z3%oxEm?l|;*OcSl3kusA9T>v}sEWgU#<_h|PhL(is0Q1Az*-1$FZ`Dmld-p8gp61m zhX0cBgqxpC9fiuVT)x&G#!H1|FZB_)f8|6>w~a{)h0fNx7jnr?~< z8IV!b30v#a2X0}&hvtAmzyjjia#qqNs?vpLyW*^C`U|LV$IUB)HB)!^uHkI-3Gbpv zJ^SD+)u3<@?WYFpH-RseNIWWwxcM8Ez-2I;jTCfk;Q8Mn#N}D)Ip@kL6~Tj&TB=?- zTYYGV#b5j!?%hAp1vB!)8voW?QP?=~JLNeXDo7I#We}fwkXalsijGC)GY_L}-LhTg z$?v!Qxc`4#BufKTa7a(o#=!~i2m}RH9=(`(XM6vN1Xs9%CjneF3}Hykg`yn+PQ~o+4TU4;zr5=QT>)A~)*ttR z)iGUTKP=8?I?!aic^kb5FDu^f z;(ui?Xm!AFP&9hT;`1cqU@>ir_k-q1?sw#WnPvTN!bu}80XtI5>9S77HaXd5{35LE zQ)R8{i$k?z+y-Qmuml5t$uP(kTD$1=N^VVj^6p!FTlcA}NMRGy=iGRO!+|{YQtM1? z5ldf^>?K`4wY{hcqj!l?NMTtUZjK_WN{Y^N=fjhj4|MTtpZ7k!pxx~=@lDX6yCy@! zQeti^DrKO2@LQ0f#-Z}r%pneRA!qy-S4&Dw2BtD!cHov4BcsYuguiLCd?9IpJFVwtyrxLS0`8~1`=2WSkQVH}qB-eAW z<+vJ`N;M#3Cm_>R|2Tzb5i$BR`g65_j`o;KsN+(vnM*Y%$+N{Zh+eB5$@Qe}PNqfK~Lbosk=WQpkg ztCCAcXgd01omp{gZc7Sm4I|Py@jwkkN#$<{)Aft)qfVK{W0J|gY<4@`Fu};NW*mC1 z^vGR9q-$yW_{~O}v1sIGfw+*~2AslA9`tD(fy*{ju@HMQ_TC@7IF#c%kMAT{O17Sa zp{8%gxRBl$W=gZ#bjmEvvQ#c<}AD zOD66|Q_S@xKej|c9`zShJI@WIXnMXEl-O6uxn1ZyJI$2H=4CHi(Pzmx9Gx1 z#0Tq95%*S(#M|-XJ8KikmUwQ1@I|oVh0#EO0YuDzK zoqI0h?M;L%$i4~jY@X__%wh-S^&NWcE3WY|b9-MFX6I@>@lZ&G!J{@Fo?7;p&mICB zM4G!Q!aEJMba$(EMy?YoyOO4E?`2Ux6z{5%O1-C+FfDE_NuRX#<|sUE!`L2w-9)uA zmkL}b?R~9hPi-epb(IwC(Vud48tGr$-pM>}sIyz6(A9>2?D8O+eK1LxOLvE@#@=dN z&j%;BWYsJZIQzPFYsp?u?b+1s92mt;Xa8KJVYLJ_?Q4;T!+<&pRKfrBGPIsb6Y~D) z7FU@EeLK!ugU)m#X9c4}peQ+*@7dlFpXt3BMm~10^3Yd>XvOE|3ofZy3w|5ioV}_# zT8AMV_zgE6jO=Ym4+U<_;m3=uCQ+O_cVbd$uxmw`pv<#7bm(Qbe4o15jc(7`r`M7w zA=RF}t8?MrPNfq>TnY|h#m@alh7y~1RLs0X&$h@W3~YRQZN3n4_koGyc_YdD)k%wo zpmA&c6W+DGP6~!xm|c_R;}a6|&l%(any;r7gQ#Ys_Vp#Vdjx4I9t3Jx0L!Ieu(9(H z_1Rie`+ZIt0(JmSF0iK#kf+`PoYa}xE|4}nUUq*$H?}+fWYBv!5qZibbjOq=S*Z=W zICCf4sfbzil@#1_-J0*%&WFnC7PU^u-Yh_^r8NaR`J~x`hHyYzGE_JWXFGS^KOnh$ z)Q=dkKK~W~aPM-R4kdag^&u@$KKWZT84IeTc;P8#o#D!*Wfk1!82$L?cMp z8;Nu1lf$#CGF-$L;dzP2i8jv>00J!C)yo_dRAJD0q-&C1idRitPvNtcklhur;vPTW~ z1iH+(lg&1y;|(j@Y4c3KMe)9se9{SUMSDsorue^TIh}vK9Cyo>gfg_hU{Bwi_W^@}e zBWSo(nNFE`^~mIM34eclY1n2a^@!I~mtv8Rn}w?r12xqB6k_ z>b7Bg>QsCX!7=rO&5SP?bm<2l8$mh;5w*=zlj&botNYp4wNe0=RN2M~De{FRTJDDa zf9$<=K$C0#Kduv$MkSRJ6_FNEKw3D6k_yr(laA31J5#}+L|RfQk?syjX{2EcgwZ2M z$KZPpJc^#j=fiV;fBw#Y+ql`T>m9G_-50XaD65IV%+KsL2quN=;_T0E>KfzL)t)nh zGh08yp4atMEbT280fqZg6j|F*+u$#m!s?AFVx*ulEOkzqD@)_LApBdr=9&P&3I;xv z+0JTIZdXwP_S%1ps~8@WsKQS40W{n=W51qXDdBan$Tqy{185&E)g z3)fWcFht?ml|bhZ@(+X0d5IOcI#Po>e)KEC?vXFfb>Gvv3BDo*beN)eE!?Grf&1lM zY`_E>T11THHz75X?voLYkwqvkGpJ2(t3?Lf9+}TKm}?7%-h=L9YtDBY)1%-8i7QEU zPoGDVFYl;!s9so@vCC%pywoA1n>X$#$dN${Q$KTeUDpZeE{Ma|+W5p@FS<6A|CO-H zY+m+r3=rn5V1Tj8KW{|a@#da_5#SaBR4ZN;QD-bq^drk;0Lsy?U4{uYZO%Ai~-owVh98s0`abK0G%{Iu>uUP^`c|u*~df@g2XMgXAX9Mdm+A*CZ&jL)^hWA>HG2t7`5 zzkkaFR>DxR!0XU8WUR{V5;d=-D?FR@de~he*|1 z;0W}WNZM}~5Wi0iz_-f=q$_~K-w%#wm+c6D7%KVQ64sjlBAu-}l*fRCM*(PhrH;yo zu_XSfi?cNWSRi~UxPaY!Z=0l%L3J$O(-VU|9R(Q;xn@^x)Th#W>=k-lJ{|S|YoGX? z61^6e!#DScklZz`&o+&I4a|0KS{Wbcm1n&-M9h6}h%s+x^}lrQ53-`m z1zvY}(5|o=SFp4G3@p2-EFJAgOrkZu^tjE_bHSrZP>26F53o;-~6O<-$n}tt_jEy zd*lZMm)A71CN%)50K)YtXFrVEU1jE1$vZ!w&Ut^0UOi}Cq)4j;bsN9W|LL1Opl113H#P@ouBDJ8R(SSIL~Vkqk=)<4A2*jM5Cq@JH3XinGFkyf=O zi~yEhCUamT@B2Qm9za(#;hzERsk8c*#~ zxC)0DY)a9fpNZjuqUHi;rt3uDT629`?HdWpoSf; zB``-V;;b~d2MrBDy{nw}oP(YNZ&YW0WWifjNAW^D6xDA57YAs61&r{pHI}!-VfeXl z+c|Wd`FSs%cc3hl+lzlh%T3J^yPm~q96Hs6W`5*X$A1e1RYlvr$MD}WyLIMLi~ewR zv)i3@ix03?;2{QYl~O4O!(^ZW#sEEn-g%GQf&^9qUva%l&+Y zh;a?=g~kQ5V5TS_)z`RYsJ?smRK{{R<=E7>12`h$r9kE$?dDe7L#lO=_)1O8SeniK*>oj{8+5Jk6hPSRI1Sa@o%sRYwI3%<1I`zD5L zu0BisV>0z!os};uUf7h54`FMWl#JnYy+aSdF7>+51a-p8ngIe()}Lh@}Krk z1BqC3-2wk(|Gn_=J2y510AjUd z?*tHQ)$?%aK@O1YTL`B8Ibd0cEIwos)62(cB~H<$&z|MBFruqAa#$EFFR=(0;aLY7 z5OqIS`DGh+u=r7h1_vnU4dAYA`j2#$*MRvd99+}2hx+7d{=A}iLFJ6lh~0;mTHE)| z+4XPhsqe-&ck(j>3$6$rSEyt6wb8eGtSemftki%K@%2wlNO}F9^RYn<|*%rFuDA z@bsRs*Big{kCrChTx7((EmD%}gUy9rAlY?@w7K1hQi(-Gn}zmTh&Ga`-2+PW*plzk zFvAX%@U3^2Z25r_x*g}C{wUV7CrbACizt%BzQwRGM}BBefQ6^}S)xwX{zDl~{BI+y zVe;#mN#ll*ohxi-_Z5?ZSb_(Kgs>07`Bbert3JYp*!Zq6awb-DukZexE> z2)12&x**o0M0=H8SpL>{WX?cvz_&mFfKvkZiWeIA={`cli4R~Bi4&RaM!TQ5{jOBM zeIbP#IcpzO6-Wu{2U=Y1M|QNF)QR#4#VK1G#f1cG*m31*>uXmvpYCtSlp zoC~b;4Gwh8Y0-krQ66i~q49cYF+}QX*>*d_{k2Z?!Y8i@3+`MD#(Ni?-714W zNq0}0%S@DJ5?pGx+@larHZc?~p9e2;om^|9w0OAGxA-xwA_T-*kZ;C@tLb6Y+iC_a z*AD-<+yfavzp}eLvyf-O*EHnbN!+`u55G=l=(jCk&+ps7!Yk^iGtlOkbYZBTg85QD z^+th4Jn7|IzD6x0C7E(4Z|}U_3d#ONclB?3-@vOO5LxTwx=$U0e-3{?W)^n#c+3~2 z?AM8Zg^PY{^zX#Db{deO6OW<>@AN{7O(h7zwxEhTJ-C?W(Ywd`Jd@K1JC}8bGfhx8 zy|+nAcPAnsQ`?t|gzDD2&H8EJeq3J5J!zWjY_4mwMHv=F1UyPs$P^{9DJA&L@Vebk z4x3REa#aHM0-^U<9GB(bqS19OYo1g({Cm%mgy$k%QlSb0-78CZuZ2aTLRL1&H^Ow1 z$S9s~eQxDIrAOCiU}5UOfRlYL#$ex|^j%YTN!SVhhe?U0lgZGLs0*?BEO;7ku#}M+H=2a4)fpkdQQ_%Ot$a z=n=f06Op^hGrjTID3>*8OEAk}lqExUbFWM}oVBwzqhco2PIgEkS8ySbAG6@c<*|jp ztO(u%u3!_YFPg_T-e1LRnw|$Q89nxo87v%k?GVvby8fpG&{D@Rll+DW`PU>NqWiIL zI^WrRB0HgVQ4Skht4t+$Xx(TtG_M&_H2!|+cy*1Y{TVS2G%;DkG@Q`UFFy3q@e6j^N*Wx5ssLJj02>pe(eg4wf97=K zWDvPUIHJ$d%GJj@%K&!qk{1;HswJY$Of&I)GkIxILTYmD9Uc0hBJoRsrSfdDQi=wk z{-&mgS1~MHaX1H2pU0I#Z4Or_NZ~RZ`=R0lAX6?rY=+d-?7s{HGT%YanmL zZ^?E&erac;&NoS9$)wFqA*5US#PZ4O@$8X-o&&l-cYKOYkIXjT@XlI^IOPx`-I5@_ z*>vPt6FfNiA_S9|5=}3@<}W=^jG-g!b^jROY~3GNE?+NqSDXn%!>hIQ%+L&T|DDX=fI^L6 z!?0P6vq|sNYO^nm0)ImO1ZKLaX)iY-xmjR=V{ThW%jukTNi)j)Tfx%jn*xu3pX1QmrMEt=+z0Hp7QuVYR8b{z418 zD|II7Nd(C^yUD|1wtE?yf@Vq83`I@>DoGb^#4DU8kEl2fOQ)>ND|b|APsm{S;Y%aALJ?PySVh8@K4P`Ke}>=D47n9(IBcgssKHQtCMeyyp>$j zmd6#`O5RdcZ&|+SMIJQ!4S|9B^O0k(?m&6uD=Om_0xxXSzy|i&qF0QAW?yg>2U3DR z;vm0Cd|;^pGa&~A(KltlP@aa@vg|T&;fQ|3MSsKE#bjzqs6%fnYIovG&05?3c$Vq5 zwB)Wd6rTm+jUOJSjC^XN&>>Ip&rvc3A?Nuv#c<-eMvgYNErGq{w!wgj<&2O?y6s{I zNf+VQK;aKf#uhWno7o-kw&0p_NUdWjb{-U1310Cw(cV;4sR%)k`Z2@9c1s%hHrvIS zK7)(x(zoKUFcFxvycM|^w>wR>57q_Iiw6Y*qwEN+Xs0jEL>@s{fy^MwZZlW~M<>g9 z*;$=KtMiK+cyKk!ZFspD-%@L6em<(#();=OwpZL94$ed9UY5_+m@QT9)q(Np@eJ^^ zTgR0Op*)_~WjZNFnvHc~eCQ5tP7AO=ugNr#@Pe`)po_fmuH(T5tKb*9&Koeu-aV|l zv+piduHe^b;4=R)eq!^B^HyfdK|zZhFknkwCZBy0rye_W;P>eJ{ZBv&ollk7?Rd#K z58>#I=hHW8c^YSGsCQQqqu@G405LyhXB%u{%^zFqy>|?Rj2^|Q=4m0ARvBHFS{8%x zkcow%A`CQXDPQ!_^`>&>sN6yalQPz5UF;~#F8OV}oNv=3_%L!v0ec>(fhuKR=18M$ zPU@Nz+a!I&dGKyZ!H^3I23EorOk&AycpI>fb21wz_FLZque{#I#R<%RTz8bwW!L@y zR9B*g-as`}$7QoKn_Wwh*UJtKKebSVVynIP3bSXrc_|9vwhUirB&~tIJztvQzSqxU zCZAkUj2%FkbKis2K&2^NsN4|BGTEcb+?Hw~W+)(P%3u^umTO752C<6B0*RZ4*fp1euVU0Mfn{Iw0F>IQr ziI0+o^9jbLR5koQ5npL-!6=%R&AB8UtAWru=`96gXYQrCT*kJ$3pUYYu$b|=tDJ;xP`@iDk2+|DPy&IlH9%>p3OL>m_6#(__5SEk~e9w9|C=`Zs})8qMa_# zs$s00oLb5=_qa(t-I+zH=>=9{i(gIFpPy>ouym@rmGq2RWH)!QOo3lmXncpKsx#Wo zaHE`AlvfDD;!ij%u-Cd+8NMzj@O0@=ita!PzKiIg7itgmENdUWRAO3u<4ey;{`|`DoF8dL zo_07ZTAzORT1o+l75mnQ?om`n0`gdWHL?+SZZpW@X=OHh$@Sp0%avIU;slA!+4<`V z*UKgr`vi7k!qt*UILKZ9(5p_#-z_1R*xvFJ(%-0@ciYRbf2hLvjrFFulnO%BB%D1G zvdp*Y*h&3kM0qf7QJxxG@7}A3+HmM3iSC&#?ZsbD9t_C#*8&@NEhn#Xb05J{`LV~P zv&J;BH+3X#zSY51v;w1ak3-|X*tsAXo_l)U$#A#Rfb>@7EevKlH$_dy*V$@z-y6;< zHwMO|`S+J(F6yORZ|Vei4nM_($Q}11w{zM>mrkI%CS`gvhHWN*ac|MBsz z+;jON<#(z&vBH)STF87HU}@KnD8X3nyK+Msl+Vmpawfu|)@b)b1-c zEuHSs%qXdF50|nvGLE8;$~EG^Rm_~{O5~f|>&vd}@S!H1aCRF{BP1NwcsqORJFWkx z7cgtA@XfPzO>W2w+}yDWF)(1ZAM$9b$`{<4{fM05u30~|DqMj)=}+E#-W%dQ z{Uu!|q&wX_boic5+0bnoZmrIHnl0CJa^+iYm87%3mgmQ;PhalXE47)=O0M;2li1~P zrObDq?W4^&q~u33tG;1`PX&OGOS*C8cfz`JI>bJaUs_!%7nshrA-FvoTYRzXh=;=;juK}ttsP{a2ec@{koiprdN_?C=D!g#Oi-bzM zZWIVcITkzaZO%strlWF8Z4N;F2xh_0mU-PhAd1bOR(GuKP3_{7Y(sE}dseD!@aAwc z!>*z=TC0Q?{FzgFQ+uIJC|C3rca&jxi%}liz$$_hf|y`~-e{_Fo@=aOH<#J8m@FvW zO-<`WZ?Yc;*cLaNMtegMF4%-NC5DIzcQWkgI4cyFQOCM{#Jh+;vyTQ~t-W?kq|( zO~3q{s=$j@Jfx?F^Tk!*@#>mM`P4ii&P%ma8O|L_F`It5lY&E`zyu@Rx$=>L;RtTl z&ZE@R{9P)-uEUC0BK)Um*X7l*TX^beRt@RK`6p*Z6E~IXgz3sVYoW|!$^xrf7!+WhTD4W>6%?@MC~is8@TJR3l-nhgGMTup z4_Z}+hL+BcO*f}0F<;HOZ-^bg+9Lk}5xIsEoq1d?dCbCP$CjDT;e7(pcRKzrFDeTO zf!elpiW#sQswYkGrt-qqpw`)ywRc|bMDLZmv17e8qH8-fl_8vZx$J)70FlYoKxtR{%MrWm@JJk5@17jY>v|EvIc zW=uc%pp%8vIK@%Qr2kIs3)dz@hvNo;Z=(;8f+9X{?neWN)gB#CZYU-{-)Jn(UqK6``Tc$KMo z)q`tR8D}}IFT8l&P<6KO_=%H5eLmbF3G=&c!nil?ifcpD5P7AwoeK7tNkO=)2rN2L zG%7A87<#_jmNoR;&?r7H+BouCo)P)Np;cB5i#r4@i6o&`|!Z6TzdgKV{Hh2b;8La z&P~34nR?BLhF8AqhAv~VV2zW$lB?WE>w7$fNWk*I$kh`5Q*K=Vflp-1HwYA z&ONNmADi(o*YPRXy=2KMi^E5C_bOEYhm#DLDZb^_N{FFTor<3}J=lC>(twlF*PE3p z%s8mcjl~cdajmN>pCW+Kd%dOu1K5ATM@u1O*vgfW+op_@O+*NwH(+<0Vfv=Cbdns z%wI~d>rNm2>MfDR+iT8X{SObplMes|xe%=pXco;fs@-(Hm(S~U)hq2hi%vO`TlXLB zX{6A3^wlMsEfc@wGR<7{?#UQN|2aK9=g59qsR8%R>c!8vx-501-Z+h6*Gucg)9-{l6Y;; zn^V800pFcW)hTm%L8$GWLUXQ$x_&gf(E452YM`ddB&gvOSCi@9_W~FYd>2{pR4S?ro&qrjE~p zCeCx-sGCVPidv2J;yy9RUuIh`nm{hnZ$-$hRes+!^uB;9EUxJMvjglH?9rL~U-{XYop;>h z$&q}1!z;)nYs*>Dy^YRA_fSZ(xm&e9fqzt3pW9lNU>l!i`U`G3+U=bWZQMKKxtWD0My!^m5P2w(ZgGZp|4`<X@__&;!}iTX$NwcUI9>eqMN|OmGhcWXQ^U{<%7M-2;|(Eo#Dg?Tz@(XKQ+f z)vI1sUmlQp%~qc>%@bUf^WnQWpz`VwMj^MpkwSE7_VX#0o#pv~o2UHTy5`qgleLR> zrwSe+yPDXzON;xv1c`kHsciV;Aj9XPp zJ6VNG(|A^EtZ3_vOgjJEL_|hO3cc-8t!gx_zT1HKlBiSk5WKK#Q)RNNRJlTOB7!E# zhO>fu^8Pd%6>lOrazO!@)i}|icl@7nUkf1dYlcO{@Gd&3m4A8A)h27`rEk7KzG=U$ z?Y7zZTDz3A?Ab9Y&(nPMi`8#3rI~ytKPiYV58lW!LD6x?b0w0OJGE&~_XEQwF|LR( z$Krfs$V)jT-3D6`W7tw%Xv{-*7fZMfMAuo`;9^DE2WUm5St5U=1ABW98a&x`U0v5M ztk8U>SN!gR-==bYtkwWOjlN6I$|}4(25hi2aFis;e?T#!60wGEkLq&F$gpnt8Ef7J z^g^S<+msv+N8ly~=HOcU0iB9-+$!RnRyz%G7)+Gq@<{T+n^EV@MCE!m`y@-I14hIv(NYpsEzOzmx5?zWeY`v%wt%!-z zs&vxmznXk?H=8}Nd#(SW79vyOl>%@V9?+~E7`U0=v{7>>OA^nw-n7dZ#ygpNOg2V@ zUi4AF>#}dGQ7A`-&LFjt^>q4xUqq9B-alL7X=)|_ZZBC4l$gRD5`xO_=Xa~U%qX`* zH$Dd%G~pg!BsJwfyXz|Uh-TdM?AJF^*GidlH5C+LP~7+|jzO;mq|~WKt_5|nvT?Q! zbLn2?^t7wDh}r9{(D+b>n8a#s?E+SFz0nqNp*Q$!x1wkueY>y;eaU}hc>)bJbijJ! zta~=AJ}vg3)9T1R2G4AJSUr8@Z0CJuDZUdai1u|{XIr2e2CjL@(`E5D6MnxkQ7f&+ zATi!?=h7s)iS85$ZI}spr(uMxDVZgANm@kdXaGwMA1|MgOB+z7IMl6`dNB0kza3LQ zd%b@NZX8o=rS%`J+h(<&QkGku4zt27eU_f^u(CN~4^>HZbGA8bXNL@)wHcZFt0Mh1 z(zh?n*v8UFhcxa^M|%oPM$p7W!_N;jtCHI;Xn%kvwM8iji4CwiUMq5u5G3ew>ps!eJvNKqZ zW~XHky+gl2e|~L7+L_1KosO9_)Q7+9*G)kLihtl3Xr6sDaT--=W^u_i*z>gL%LkN5AINGJ5ISJgg<ebh)I@a zTb07rbj+@I`(qf`{Y>qgubcYy*(lhHDXo6fxID=wq$}29QiqB?@E0ITw>c zSy2BFb4N(ajgOP9m93+1Ggw+cG~VTlo>m}Xsl0{Z>}J@-gaY~A{4&U{?ZR-U>$1S! zhhS%YSw0}SVegpNk%Um2>`K>K*ishHn=J5Nvs}IN-&;`(`b{Eiuv%#?Ly`YP|KJh54U&WW2G}20b=TwA$fbo7lu!(Jw9Gk z^!aPz_c21$tN>t3*hjWNChBM$W=S5+t196Y@C~PI2NZoqBT6PnW0A&aK`L7I^uVgzi`Ghu9j-xfe3YoVPk4#^Ri&M+|<@xTd;mj77c8Nv&A+;l_be zZrdTlM!+UCdAQy4IhxMt9qjvSnx+c_!2vl&C^<}y;WnbvQr(Xgm^Zkyx0{p;oA2Cq zFBO!bF@R+cvYRm&2)gQOzzl72!ov~}p8ZnFvFvSi*5(k~-$onZRd3|MUBa&y{mIrXx z=2|58+?tu}3a*~FdhNzGElEf64ro-DxoqXI>cjN5KR1rmO8#kNpnu$33Ymh1JDX~{ z!I4_UIp?~=$HzWIyFuqQMb;*Zd}cn5A@|>wurr`i*497o3Y9$4R#um9tbZiDxjU$; zF4CHzq0=hkq)P7A2Hg zvDnC(zhh5Y(rG81Ccn(ZdO~oQ{du`-y&R*bp#QMTN@vihtG`EfVa%P)NJ`bMjht~% z{d2mCS|t$9miE^tOSaqYFMIeNgZ7GUw>H!>3eFzylaF&t<|lvow#&-Fw% zf2oSyUYipP5-(2Am63+>mD$24rI9YQ*FH{mfqN3Y+Hlj8ZcY=A_lIH`3=w zm>&x}0Ed$vNZ$5Pkijk0o}ti)><=)?t4&4OYp|ix+V!Y?d{{?#u^Y!8$apLtWXYYm zUAFutS*IgSPAu#IAYh0>&-DbcoXs9{EnUlw_b25K6v(ogaVXz!T4&{Oid(3Xhx zg<(6_u^sO#apj^ zB?TYPz8nQk=EZJ`<5GWASdZErV*WwD|)4mr-v*m(LElatik417~7h!8`6MyU{rx3tUXrQ|mk ztjJHUU(8d#WBY0vnT5x|Wmr|8m{c%qZJyeOpsdoJQ-4Dno`Uw~lou+%pV;|^!-Lpxw^RjY=8_kdAG|;|*?_1^Qoi>0 zCbBH{0->|^-x~8WpqkL@pCv^i&qSs7M+W&{dV9gUbk1}qf_O|h5NQeX0xwJ z>DpQ;AVd5N!&4Z)qLM1Ya!meW)JoZ%M+0~(55IOfi96MwtDvF3Z?Vgu7+fMOeD*0H zOS}Ah&-0K9IS?T#SxgWR9#@ykCn1WlQG*sXQ|X?kjj)b87^m)%ik*MO1gP|xvAj` z|K#od1UK5C!1KQLjrIF)4cIo-h65|4NE#bR<^?nx?rriHP6}ORgD^2nu3zSwWGY>k zjtvIzwM+mP*-DuE#(&wpaY$5fd3>?4WGL=)7Ll~80?v#i0n}+Ar>f0Iv~YZB+_LGh zfZc=Gz`-;wiCu@KYee1IV$1+NG$(rAd4rAh=&>-{8yK)`Nw7 zGvZcMuscP$I8qH0auqJmji78D*C;!uF0ojttgbBjb?Q>fr$@GT?T&+$WS8{cF= zb$)K(&9XO>7L(_#y=)u+sKdqE5s56vH~Yu~VaT7=`73*j4Y` zaR;=uhsi$w3vdS-0RGP@T^lL|SjazIFk9wvsWYX*R&JwecFy440KT2vS4QyUv)-uU zyo3lE|0u0_dbZJJeLTBMjwn=6g}1fF1Tcz+2H38@@~Sg8PH{kqcUJ2bEdec+%z)7M zfJqeIL&53PgMro;u5Eyn+MGwsPC}`C9W#YE3H>$JG8eO|WAN86Q<3oXbPT)_!-Vn; zS5_+gW0O|HP2wTgKTP0398M%c!62-%UQS?uv4=~$byR(81~=78~_2}g!<2} z0ywZSH-W_gx9Qb%cuW1JeM6Ho{sMyBn2&25%%|{4Z5cE-fppu;UMFK^t5#AKVU0Yk z`i!f2kba(!$3Q!m%vdwlW4oj4C6#Arl+Zc(kEy1vkuFk2@G~*390Ybejwq$3VMa##-6e*}deaGv)mwZhbgFX%yos{q(ZwIWT24liDA8g^A@Q{k z#D+v$8IE@?W-gubtY@#dAmp%FU7gME0E`Z&Z$!witO!R}>I3Gu<)&*X*oHndMziHGz-Kt?L^gb@mE`c93nq&%qww^6xnvjv@WrK@&445*UOz^9=nF!}no z`XS8M2LNP@bKk|}yKeUFznyjv>A8s!w#l{q*okli%HP3Bz*Cr#DF5ZR`_eB6LbL+X z-^wz@!__|L7Rk_iFk|Q-R8&TG|7)M1=B4kU8#{&AoZ-*&LjpeE2GFtxRDPh~Z{=xA zxafvntaz?TiqYIyX|@~38Q+!=%u`=yChuVa;H#lXK=A_hihXO zd*qp>Hb1u;*A{W`w4OSpp6Zq=(b(`#At~MbzPy9ymkZiwU;Ar6xgo2HGX_?rChlN@ z)i&2}y}<})%ben?sb)6>%p*qmcuoroZUE66uzT7T2YH8wqOIR4ExPIGVLz5%_^yLE zcy_vCFz~vZ@Aon8?bvueg74Dgv>zn0)qWgshyMNVfq>krSYP?t$oGe2 zF1905j6xn0UioKv^lxFhlX5@|BsMB+^@jreEr9p;)%^SkjPW#(4#0VCO!$3u|N9HQ zcAUDC7N2_lW4Ryf_E~-*9x4*LXkuGQf?EIbR}5}3ieV8=ZOhyKaE}elI^0IfqhfEG z`B}~awBv#WT;oVFi6FogVP`t~AKiP{8m`D%GS_p`;jO(qQL*_=T}~kd7+C6wz;{9Y znGAm&zWavmN*v((dk9gO{jmRm*3JmfaLM{2spWHl8Ojd~!>5%i&dp50<2>9Xx=1a3 zS*tG5I~cy;&MyKX4zw5s#$^p|-E++I@Q$PU>HB~4)38T?GVz-;n~EOdfKP6v=Fsw& z`&tAl_1A!+D9Im2Ls9KDXg9%t<{ANhF1O`-+r2$UmA2+C5QsV@(B63 zSgmN^Kg-;oPc=mdGfh~I2*51Z>e41i{(g<$fe_D59ni+m+r)=vj~s&~kOS*doQa6o z=NGMREE^v`<@Cl=*Q1Pp64Uz=mz{Nr zx7)_Ts6IHLk}|pd+We=F^v#vm@a#~kW?rAF=nsGOwM7t%SJ(6vk~Rl{C%XsoP#FN~ z_M2A`(T@+R^V=BHJOZVTgHCV(zS39!_LGOdeGv=`(_ca=%(?UN^y-L~eWMVP(8nIT zcvdSRk*uWuLkRcb-~?-Xnr_;;_BqYN_derz$;J}$@wYY;g}_41Z1^ z1#z}i07)FC_w_s+_m8!Q1&P=5@zY0Jxa$aiTd^nZN5mG3IT4bu?fs!iRfnbR@V#A2I3h^_9o zoLo&-Khdt;PmCaH_$$9)Z)@|`H<2d9f0#m;m%AZ!2iK3|4gOK45>7#Y!x7^yqVki1 z_TVT^;`;Fb&MObq!5k@RoPC-QAeod*{(DsOTd?8U{rz>r26Af;pKp3jxQbt25<0WbNx+Se7M*oc9XA27Hw35d;thqnhf;=phn%cF>jhjWc>zarCJytsZ{mCr{QR z!&0kV6x3}R+VkcKsGjq|d9Uwrkl(+E6$Q#r*6MGm;T~pp@#WK#0(75={h|nF4jRw= zC=XQ9he0@7;U~uY#}f_svXh#zniU#6eFQ3p+gxFcXI`*SL|Gc-BhLtE!w!^EuMnrM z*x@PI?kUnL!3*&lz&gCZJ35tIH9l?x#|~K#z~Q2ZXPZ?`Y_WHr2rk++azT5_qv{p8 z$RgDKQ7jYvs569cBxWzJQ%QYP&d%Az2yW0Wa=x{H?aL!OXO`Cdrr0?Ws=%1Ze6u8m z!@u>j?0+Yvm@WZ;W?cTY?+-uFo-P(?R*>XzKq9di!uV@LlWe&{N8s{t5C{HMEAy8! zKTT8J(!4NC_MtMX!vFKk{tt<5gSd-7Ww{Fr9|;BS11P{i|BC4+6wro|zX8=dEUg>_ zjWL|Bfwvx_mr&n>$cO0qYeK>pC@uiYXgxXn^X-$MFT{TC9^xSfdNn}x!nzdG>T*~W ze*K>BKQSeW)k()IDu*wDfRW&gnxBfY|7LP-saF#FVcZ0Iez|qzFzvrfV!Ib$5%IR$ z9qt_lytA6u6I=EcJ>N!Q*j+u1RC|7D>S(BxfmM@kYqg)sSOm!KHUK(;KXZT?TWY`> z-QD;@jd?^cZ8k6E(!IM`r?S1Oxl=HoF2a*BJ?Np? ztC(Kj&zax>nRLi6Th(s-5;dnd!@ylYL+f9;$Z-U*1&&G}BYdXXp$<=b#Ak2lPNJx*q6itTpA>C;jxY%Px!gWq!XZ}5yuP0vmen6V}Au1)d6Ixz)#?%UlWQ*5w$#NOCa zxyI|-BftUP=_xTlC{H;R9~QXS1sp}2IHh5_SGfDq51Yq-NYdhC54iy(cD@VC?_a2C z0CFjVf;b#jUOV**tX8wxA4U$8Q51hr*b%Ed0;kOmi`#$PIRFY%cY$?Ex&36=1M4*0 zvXrF!CjLNlCf>lh)a>cGpN9w^7L-Gat~^V4)k zv_w!R>vsjtrW|RR|B0N#IEsM#3^CFd-bc6C2+(>91p#ddzA5`*3jKy(FeZxoUkl@R zt{gt5#T;?cL z%{)f}*IemtzKl^*$Q_@{nmWHGG|mN+K4R+ zy+Mtv1`{PEkjs;93ik|4L^hR`N1;`HrdxwX3xJTN@}T1s1ehhZyN2|eVmzYHOR27B z9x%y6_sippmtY%RT0yxZQVc*R22spse+qtjFi+M}qf|mrbPQpM2bux$%A}TiM1-XD zPf@njOQjp5YzjKoisO7X1EL&W8+}^MPK^Ol+i{}AQai~Tp9*dXK$X&0-BlCuPydp&L-t;Ly`U-XL?5A(xS$0r11qQeTwBU!b9@wzZt;#uso8j+ zPR>W4gLI=A<9f(8Qv7n}V_O>BZk){DWG&!`IwB0jL;93-p8iTN1i2be)Zf79rD|s@ z3qCU?1+x!S6+-*)vrQy%ayo*6;{Q;=iSfnH|z&~qYg0q0mZ z9lCnufZ6-13gMAL?g5j$zlz%-`W`04C7`On8iy1SayV=ZWQbz%h1_jsU?%LdO;k-$ zyvAk)I-oQ>Sy_4EORDF}*=hyFCdED#tN2)ch~ z(_uMlR{$wQL<*TXGY!3gv5#i)n^ zO!Y2i+lYq4QGfK+a+R7r_`^e;w-95YoCmXRn)J|WucpKyCnPakM`W-0a3##oaQ5>; zi?P)_5pVxS}h08^#`vO;l+*vFQbnJL~M*pfzNy3IzqJX&IPv#IW|)3`UTe|bCe-T&5v-iza{-@D5TSD~Z&u|+idr@Y zL?%W~u7;j(4BLfc=~NALtnLkyR}(uAb--!CH%pn;H-nKRM+RJh(;w{GRsVLAwwNj| zcG%jIn=LY?#x#}1a+Wpi)7C@1)tPQ^@ax_9@No(g%a4Jhb(bdl?5l4=1qJ!mpPlZ| zmJ$&iTIa5*Pa7^YKs#APAC)R(fG9;xP5)ZX0>qKAqU1KT>AwM&_Ticth?Nj%PoT*c zGMpQbnVqu_dduD41vS$*}gDSu}-Te(Rj zhS}{lXOGfgUx3@rrgQ?Ds!5)Ijt#{pUuJ-)OCy~&=8vM1{5qS7exudvs$bopxKS`2 z1y7=>v+FAahI%cGjn>f_RF8=!nlUECnEMX~r*qEd~>yctIAC zn9h$$wG|~}3+9`6|552J-L-_pFK(U4?v^@#LoK1Svq+f~ZZ}JPWLFD{S6(}kj@*CE z?E1sez^0H8{VcZn`0PS?_+-^=bO*P-(_<5#gRAF%K)}yP^#dkyF94W@%l>OGH;^nv zF&79us!mMyuRop9qC1(#tLV~ZG5Z=}`%d6&CrPw(Eqi*xhLf0of zRQ0$yJhL$wDzCF?YrG|KA+ayBl6zQ9z;%+Y08vbxB5d`wQkGy^(|4j#GG}Qlu+2p* zj@sfkit+1WjU_893Wr{W{8}d%%K^*EGHu%XtjIaPWye~TlVsM*E)7%Fq@npBzx0f{ z8se*5?_s9j*){yE$f`SPxYS}DdnlbUrIE<)%QNE=<*Bi9iJKdXkgo`Cn=HS>jWBP* z4ITb$W~x@npKmjpDl|EVi0oB6+B$vG#hNO#_=_Jp{G36=!DD|TR(>yiqxz{MbeSs2 z3{K9sc1)P6RhUr)Jhzh5itz_jsN4p##X;;n;F5_W;pZ>RKOE*H#W^NNe674cHVq}` zES%DKGAb(Vh6Y64aqegfI?XQ)AH-42;_N+?S#TH>#mq|aU#;E7&8;z{Kv zI`xiJIOXj}QrUMl@=P_hin5c>$kCH-aij0cNBa#rmAzNGv#F1|Vq$m05B=FU z24T_oV{9Xq3X)52r|pJa#etC=IWN5)e+A%Uj|pP1{Pt#Og_GNOUyi;{1)ctmf(3-D z&R}(^t`0e&HMQ~bJ)SCJ9Rbm)fDD&QSokM{l|a~*(FR@J)=#S$Os>LML$63JfA(a3 zI52y|+a+b}4J_OdE3yItu(uQU`E)2Q)Ga#qML9Xk& z;{`VG2y|~muSKMbbU9|hx#;fPo1LQU`!VRrrBJru#i8J&!MzqNa+>AiLGugp=kn1W z7f-6|sWWM5$84w4}PU;2u{bk;;V(oBF71T54HGSMaU;n z#{08FMHadq#gLYa^^sqGO#!-(zY|A-^HU6)KOaf=_GP<)QF8XcHhr9Q-@aa0LGJ~gG z-Cztu`5TSE1hB=tN#PYEW}B{Mo2*eXKL$iiCv{5EeH-nYK4}p^d+c;jhv2;Qx^_;5 zN=eeoA@P&I#YUrzr)pnEQ;LgBpUjAq;=5_wbGWSI_s-ZcSP0+{=F29Y_~n`Qk75tp z7NSk9M|m6)BucP@Voae+X@s==R(e;UbIrnzIs(q))hWR+ikuOGlq6RZ|2@yXlmtEl|N3grjM} zP9tgwXhE`we2=Tmo+%F9>CRC43}dLQaH#lKlfXcuapxFKUPqAy|Al^A%$$MsQRg|d zzXFP1RB~$$S3>M>=%m`$5Y2oj7uR0YYK9D-xpQ;h#yTM#-$WbRK)*_P3-^SF0!jDc z*h^o{3TnkABa6D|rYwb6M}3 zjL9*6Kw^~D@>;2?T;LkCy z460&I0`782XO};!^EUwCY@gp-kwYvzI2#x#`|?l&8L|td6MLo0sDdb(S4Bq8$VCRf zIDq+6&Q_+Nv!f=yyGP}yrS3zx-6!^DA z90yf%WdcKB`b(Q=P`YN!e0$Am30v^Z?cvY0)=Bv}+tf%k&ajdXQRQ29Qn4O^hV;{D z!#hpN!0}N22)@@Tn@asRLHw2M2g;v&>FddQhpv^sG0`WiGJ3$TZT|nY_nuKrZR@}A z*;tV6RuoWFnwuh0q=@tqD~Pl$y@P-tHS`{`u_0Z02bCrrM0!^UQlkV2B?KY#5F#}s zq};_h_nd?6e|g6p@0UB|6N432=6vSw>1B@j^{F-MqXINz-c_zm1kuE-4jyQk*tIZT z0pF{`)>Y%J$x3fAs#|=1zh5Z;e5KTs=eHq&BN4A!ORC1&8u3NIo~50Lv#YDyjnCI2 z{}wN0zZHFLbe*JO0u_Dw<*Ds(TDU<=V0kTJ4d`i>WER~|1?cX6al56AX0nv;Aj%_B zJ`clX)dEZk&y@Eh$fitl@brJG8J{e7Te?LI`J0wqa465S_TqrV){w)blc4#38Uz0g zZ_-d5=o7R~C&?@VE3-Lb+T5qvB%TBc; zrsqtB31rj3?ez>7iHqCso9DE(wUFj3t&yCx7jXzBjz(>fY^41H8^w#<=A^BV?_G5-GN?+vi~?`WHQMMEzEOag1Iieakvp zF<@(kTcynWcc)sXp47k_b6&tQ+VkdA7cEb|yN{4|3V`aYKKzcazm@L)qRU4ffJ}-H zCBIWeFD4IN5J@9dk(?v{eBwof!J>jRPcZhUXwZe0C59bTs3H`Fy$ zU<%m=VC>sQ$p3(ps;1rXVHGe{G+;5n0nw|#9n8XLSnJYIQD?sl76t07kpf0bQrsDq zl5{R+UN#%2F52!q`%cvE!OR|S?y4wYnQmz#N4_Onu+mAxhI>o%ov0 z4!QXsgN;mEo?uAMG%&mP#RHw98s`~YEqrRh>g9QsQ<0{#g=a=@sE(z`^m%q2tn*!c zIPJdrko#GWR|9L=UpIBDZ3Q%7>SOWR0;ZwnEGK##Ms=Ux8h_)N?VM^mr%0eZ$?Lnk z-Za@8SUE>fd`v`pQ0;b_@BwSQTGW44b^cc_ej~t>&3*%h=3|R|zTxe^TN@eDEg}5}D7mTuKX-qfIXhy;`<<1wzBLp(M|GMLcc+f;(MSS?+oA12eNVH|?L0*~u zgtz}~f&Dj&RSG>99VuS~QhlO5d>KjZD=Dk4e2@zyUqlVKNZJBQNo=q~WgXAEBqk<< z;>wpYKiL)8Bfp#@l1s&BkJpghLiRmdQf!`%iG4mD_4hZh1Y+IW>#<8ISx-mWraK8y zknBh02mYMT@LBEZ(ORG^)^3Ai$Xm56G$d*A4<#?v*@-&MBpj-ONRe1jL5MIEbB%3S z=m>zLR^-%-FU^)dYw`8$OmwWja%exT8M}nA)ZZn~??sy5#=CMah}A>Z?;o|-zz&1S zFVSf7z=!+nVTf}Ey`C<)Ipf*UHMlY?sLs}=C&_j!Yir`D=nWD@S&=o(UD@y__@$%V=CMQNV8)vAmw zPSrx>a?AYipQ?o!Vc#W<{*Aq_$zl2Dk=w4OM||4YzkBZS=U)RQUV`ABoldHa=SG+) zyyH16x&CVUN1~8`7GGUOqy1a-PWWP~BP?Gj^`G}l?GcCn-q^RIA{n$xW;?LWcVtE# ziQibPA55AsU6~-gjEQOTka&GJkLLux^zkHJ>UmAD6FGjEf1Xz1xHCtaI3xV+oBwl? z-~4j)ud5F+bd3Ue--Q^GE3OKZAjqYq*fp3Lj|tUwm-mMc9Ts7yb`8qilTgxlbjtx_ z*gbozp%m4aE1>^X_561P`!8ir7yf;)^NGJ*Z}_)l5IYtO+*+t z^6ynvUO2%|f?U?zXjmRjYpDk;bwxt|*j2gAQ^C!cq$Y_dXL4y?ze!hQ-QYZ)eUyaG zGu-_CEkFE=;Q}CHSVXu0lfi?hjx7&0zuoTvq@rNQ5?C!3^`LgPFyL&{JEnnS8 zZ4;V2U3pb+lj7IESeqvqAFJezy=38Mq?7Ld?Z5owAcZdkJBVuxn|{aqz;;gsQ`^+C zSi2&%oK3Yfby0gAbpMxtic_ zUimD~vG4~^|IIynA0?mKwQ=4R0t2tUd(3<^9dyOhKOMcz%#0NLHu?44$N%+$)(W2U zZL1@7Nd50yz03b`u+V?BlC|}Udhk!c^Q~f87dhYNh6aDHNe&)iSEG~7pJ!LYR_Xs> z@!xOf;1Tr;z{@V5o&CWx+iT~U*ttMQCzkdrpTac(N&Kqe{)N2_!T-W!dGet1Gv zS^77=vnI-)9RzZ@P2_l$^wnK6_cAf(t7))fgupsh+y#gL1e*li4H4@+9t=Y zvw8GKH?>q9>@1LQ_}^EZ{~s*dR_@b4)a)5Jv*7z2r?66RaBw(W9Rh)-yVsu=ZC*?h z{}HKvK<#=Urc>b; zLdsY1$N%9!e()%mkLP29jlR_nBgZ+~Na@5(asck)oeWxqS<}AR=LtZxy1%4;a2Ip1 zEK{w+Oqf>tUnLelsB-$j$sRdj_>petVP;0xK5H?)V}yT_TTL5)b8KVD>3tTpes|ov z#5-NRn*)JBd-s>L{oy-~2&)3IR<*TiW0oI3|35I$zYfuGWH*ApMG=?ZCp-V>t>wIj zcx6Po#vS&Vrufl2u3FwZ3S`?x2%XD&17_uC4kmVIT}tYflk>B6=f75YLc#*m|4#e3 zXI*^OS;w@JeB~CE--T!Ga389wOOwWwWImls*w+Cs7`~{%+z$QJOuZZm ztG_~s>aMC=aVHqA$Hn_$ov8!qL$o2V|Pt*%>Ryf&w_aXTM}49tl`Pw0@w zon4EyTs?YlE#XG217nc+WpTqGtOciD5jiPO7y~fQ3bkeCbrplo}7|~4Nz9P za`{aSf;!$|CRS^&H>5M1#{|C)pAD$}O;u0NDjB6?~C^M&iR7tNB31TEp zjj%RXTC}6@Y6s~yg7ubJYi<(zR+}f-@ncCkW-SCzcM~2zOdZGVm5m6w~eoL{A zJbd7gN?>SC>m*l`d(U{(MJ3ldhoQn|3Ahr0tecy3vrWRfDT6lJl4F9^MeDCYz|BkD zLNw=B)bRw4+g9B@qJ@Qpl3Rt|6)MPt25c-<7+*#sEG!JCMU`fmtI{VLJe_sCutXuh zRD(83>xo9>d~Ewm!R=LYS2uSsfXhKAHZC>rWRu5w{)OkEqu1w4eWG-2eiYF$=zY=-Cq1yKG@s}tR5MpKeaCY3+F!UFqQvDZwVx_y~u@IkT ze-S1Q3o#k;0>+)xA!D@07vpp}cjm2kkt3fxEHoNg*TMz=z^Y6otxStM(3?yQDTR5XXH->-eBP?cY)F9}-RM1t!Y9!dFqeizx2?!L)p zR^D7QI2*0pgGPGuwB>(#lLO+K2=Wv9b3GXDK~JwG+iXm%HtadX$kM-{HY%RE5-{r- zt8910=?dqW+8ObJviXEn5t@2&zM0iT#M3aF&fDEZP(lxloYLn3ai6_DUfCRXW_-Oy zg3cKsgYIMOl-PMI^=AR}eQuI&otc&X5(w*+Ar z2^({LF#5JO%aa3J1rS)LRNq&k)4dOf_iK?~^IoDt0xYm4c<=!Eh&Y~aiA zL3i+Xju^}rk|xdGm}eG|9S;nJIhsIf*aD=MjJxb(P}`w~YDuwdCY8Oc`m?q0O-5(` z4yRtfS4wcMb8?EAaF8mbiN*w9hIjbeUaDJd{ar^+KC`7J)33jdh62f;q=+uBh?+G{ z0?S4Svz_aXKI$ugup+<;B3poJBd%q0cWZ3_oQIAcWr%d_J>ICsW#R5ooO#+;1Ge|7 zvAvdBu*>haU6=Fnn)iO7{PrRJn%FB+P9%`q<*DiqHx=$lC7opOjSgF`E8-?DT@u$c zpqbVOQ99<+z545*T$C0;(n{Kp*9?k{;gYCQX?NBV7iyciQdYmA8Ck+b4c7734-#e? z5wT~L&Zzh0q$1uVK)6ihWddP0aVGhR96na~PV40=<{@~Fuv1n8`EfI2b*!}A09m>XtA$3&j(Ncebs>W=OUe9w7^$3L@Uc7(vGy!bUdQp7s`8`GR>0Ke^G(y zx;c|-LK@<;B|Ji|zQv99RO7vy@kA$&&vHopCyP$VjOU#pou)F{yD3&LRGwP!Jfrzq z6QpSo%ww{WF5>0qDJ=_nS;h~w>`Rpl;p?u_lv~F2iF+%pFDRP{b}g#r>k65KmRC~k z`DPO0(PtQ%&HOm|(_DEcsqP8{Fn-trIwJ$f4(Y`7vM7SzY>bzOZviI z%HkhuF6;(}BZyQnSXLa!vFt2+ps&_YLqN3_%E=GbCmS%U$_O33Ha^^SU$Nj^WJMB( za9T-Li5+k3mbr`xrG^w~(??4WSWCRkyR;!&tR^U(5bKiF-)I%jT3V*R9HDb0Nh_aX z<4g*XqG$^e*9>9H=@+C&Tbbi)gb%DA!ww%)ba=RA&w43A(Q)h_M~kN_Y6+P@Wn{5& zqE81ib)>GIV^stKU&}l-KIfo-`3eF*Qy|G|!G1J)ut%Mt^;c$L>mxeWKaOQ2=)mdo zQd!P~nptS_;i4Xgi^j`Mu;-&E*aaYreYpw(vrC}@N&L6o*Z6EzV&jNbuswn^hEa=TCuNE z|M77hwlPlq47DI<7N;l+w}$#1hNMpAH*+fsRnBkj70+hp4uZ>Ioni#G@gyjN{54HW9%MV`@4`+SL+WS7F84-UQ=0JmO2g* zdy$ZHmN3gx5D#x}A|g!(GWdpk^y$YgxiX$YKkmIIk>_rO z@4gRyHqxd!oo02qCsQxdpdurO!d#LcKd_U5f`>YAT}&&-9jFqBphGzz;w# zjj5~^1gbCn7cUeZ-p)qk;+qAxn;WV$NK9vh6J9G^$b2L~X>w|W!k_VySeM6|i4zkJN$QDhPQu$IYeq~g6a*K(*U98`}F|{ri8ku~%RS?X^#TBrw$72xq zsz4)sX_F#r0Dks3BI`kz;$IfsXt@MEz!)StsjL#{&BT0HbqWjdV3SZ?j_Pr)v;1IA zF0~{)x{ip^a5m;YwKeN-kp#WZo2QT?(5%PBa4F_!(r_OyLoV%IB*;W>C`(h$T(t%k zsF~8(yc~J6rl%9uDbnC?Gj10Z5gvCSKr3iwab8yd#6KMbS{lDV;BewPy7T@F0p;t` zFK_5Q2-ik)5Y{*dXrmTdJ+^wnPZwgNyv1lvLZJ}rM}s#}$77q0d!8iV9*Q;jKctSG zmNc$*=1Aa!*g6#leHxOl3Ct0=vg84XOtvCZs!Vex>RO6&7wa=c+QgO!wG-G=Uze$^z5u|iC*(HmEQrHLKaa2Cgx1|P1k(+}&lMu{QlJxqazL3=3^wapfu41(Y}F!SoMe~b;1uR>@azX$QM zg}ry6mD6-7?!g*& zxgk_toFZ!+k%?uU1rDMSI#`jg9t!=u*eKkv=Z202-lEi~GB5jaLKi7T+b{j05v)h6 zim6LZ!^LTHQm$(a-2l_Sb$K$L*^n#U!Vg*FLS2j201APJs=c_a)1%Jh2PTGU`UjTp z9GF~-uF(`!c;z}QqE-MEU~;h#ADV#;KUP+$L2uOg1cC5;?Y;jmqHS?NQ(Jcsc%akh2_4n^C};_ zb z0is{ijbPh<4Q&6{aRu8Sow+|7&3zX?rNMFY9W0Hz0PG9Y&{}I$;FfqoaYwxbla!&k zW~N0VOb4L30(p8ZfE{b60tQX6jGwD4#acYeX_<5i!{sP38CK`lH=k!hZb@~{a~TFr zzdtuWg3|K1$qNf~6h^|s9uJuHeNs^|4ZxkMnY~OMS3zD$I=E_rk*$--j0;H1j*c2> z^5_A<7@#d)o_$k%C5S#w15sM=do{axLC#czD3{5#PKiYdEGA}$yUPutumK$!)GEf! zVh@W#uIr#~ZbF1e=Io;V49r7{f9ZEafNJ@BgcTtgCism=VBLIFg#nCoZYM%VD{I8x zE?<5|d!#b2Ve83oKRRHLqs}L=I7pfIW6S32Q~?(D+sDf~>5W_LBqhUh7WCcSW@P_; z@y55N+gJELVR#M@K}j&k+u!#diYM&%%6uPFZw6hlx)JjWt>r9AGxU@6t3YR5EX)oCQw`5XIUo7xGiS zL>yjv;WoqCWDWE*fR@j{70*eSG2BW}!(1XM-*C0C(DU*wT$)l$(3Q-bJ1w9CbshR} zDodtuwzsUb!DT$7juYy~ub5Ei5Lm{)WLL0k#9ZPmo_Hz%f3K!Q1S8t)Kc5)Qt}oLU z{)uCpDw68r?39VH2bvvBhtBZ-I3)WGFx ze654Ne0-DLoaBydZs^QQQ1H{WU}m{#wcMXaAYHA@3>-Vy#}=OWE_0~%!!0IDyyBX2 zLXVI0WIgV~K#FNJt_iP4zcu_+)8D^nbmAx*%wR9KehsxfrVucWo{7Dopg?pl=!Bhu z_18X+4dF(H?Z|@KfF`7$4P4siCyISe}KM--16?r%8 zSOb*D+h-3{c*$*kl;Q}oU<%L=aWwmKZwvh%{Apx+8_Ot~LL6F|FrcbztkvaZ7V2i9 zS+c*2Q+hIEVjAw41r-(CPnX-K{Ii=% z!wpdY=IaA`SI93H-)6k-YfzDzjK7nk?)|093t~inPv=MhXpDs&j=zy+W&0QOeJRdF z+3sUA1R>W{2f3QgkU9|0#4aDorG^<#gJsx501rfJ^RByBpiXbhrkV|0&fO?{a||7@ zkRfCbAL1ETf;_m(+CnqavEZ8xk?eT=T}S_qoU3v}pbCzg~!H%YKsfrIgo z$>sua1-SV`YlQ$Xm>qifHw)@?BwnN-u+oe*rj?itbZWT)!+xTM1K@00JIw(Kk%~ej zh-%{v=o#yFf%I9-Dz^VaawBm`s3xd(BH64!Xf|AfqMNI}ty$w(!?IQEjWtpC0w9Ff zH|v&X6kSB?@rqI*9^lzO`^35TWupim4@P&0YgF!a#s3kU^7~Dz&-HUku*pyC67QeY z2SEx7p<$2Tz>^B8rd}2#^1?zjG};_!-AXj%eHT>~p5icptCxA+6%VE?~$7}o8`+E#Jr8~NuylfQ!D%Z+#Ms5w8pOWD?&O6ilR%ClJ>nX zPx0_z401QXcLzzEPz>^3*1Uq9HFmIFe_&}j$lXQVwb)}RZHYIrJyUY%mgt~w0Dj8n zLK0M_@kM~aD%(9Q;r&)tdVabXs`*CMUcBrPIbX~3IW54rAdj+rHEhu1f@|Byk47<% z2@q%c2n1TW2E6lNJXu6HD$1!zs@|TGy2j6h&m0|jol_DiZuZBsoQ4wF{Az|MaXgo- zkE`owBo7b4%XvZOmfL*459FmIMRDY=wYvsyOwdf*mz6+AKK0{HBX328qto7YvR75E z3M{4;mDVq9jV_SNm9V6yg2D@0t`VOz3yUMUGWcXThl!t9kwClqO14^qh4$JuI)03H zN(WWzAjl&M#4Q9J%|BFZp>M2crpq)s?j*vwv^R<6*%|m3VGCvr+QL6n{V5YY%#ibh^E%Po^9+mi$K=|=jyS&Iw@F|`SrSWDon60@RfkA5>`uYGlIW^ z2;rCNy+>th7fH*SKrsn-_7m4bnc~lZCd4^XF>Z6vjaLcw^j7| z!Aqq0#MaCquQMSpc(_2-`> zcvAgC+p7;j9$H&yt)J5tq#T%Wy;nzijPo@u1p`x9N7NoFw67a(pMh$;}N^m=G59V&;zh9k{taQ&K(>^PBuYqokYfjF zzsXU7`Od8w(((r>$aMweJgjo7{i2Cmc}*VSizTWzNQA2BXp6OwDZw>`Oosc4+IqE& z)Zn-MXM+c)-gkQKFQHKY1maB#eTM7ziged!S9y6QGe67d!(9eoeJ*jpevx zlBw78=cwv%RUrc)59jJ`yc+MD#F{x7DMfMdvyrb-vUGJhblLv4xs-wjYTkl&1aeb`G7PvG`DL$^0RvP_Ky8 zudvhQsI4dR>>!NB!hcywd{_r!ry(A9ATD&`rM>pV&4m|bd=-G>+5b<+b6F~Zw{2By zpUAS^l?hM(PTgC|W9fEe28Q_9yq3Vmx>2C(W;Ioyp=!uCE32U|?~5lyCynnEZk@H_ z^C2AUtY=|}q6fGB5!CPXW*`_bIR+9en7NE$X5W!^3be|(L^b6Vb9pw!T~~eYyMS0%Sb&!%;?w;(f#w=WYthYhCut%}1>GjoZ((s4q86;|1u%U1Eul8OJ{_Ungj-Cg*@-0x1I}oI zI|U!K+_Vm%kN9?a=?_Lut>0NT0<~>m^n|lQ$z_og`wR(jK8-yq@b$5z(HLpLWdKaXqBa1QFs>e_q`%Sj7A3-!e`llF$UwZ`OM>ebK4lMp+daFEA58W zJkz9}l9bb0G8jz(_~Cf+GKRUJ`;oUq-I^9l#>!0LuA(NT&laqA5wphwee5r8$R5PI(*is;@vAa*ae>Jgtm8}q}nQsM;l%+vtvAA zUDUKWhbi=V(rXVKIZ-73oVwJQeE`*_+K; zd4Pjk@HXpW^~};tUZozt1G02CO^h3^NnToR3SV?d5&GGp_i}1*2R|V4RwpDR_RbV+ zjX&hhOD`o^7|`RR(i^Gawr*hI;(X^1AwH13(j04-)g&7X7t%i)jyo{eAJ@9*A?MPi z<>Ifq@*3u{WTV;RKX89lGaaSs#5+Q5sqkOEZxHRY^GL zuEeoB?hf5er*#H$*I?<><`VBDeja&th45R=!FrN6Zj4;Q6VkA2V=oON`fKF}3Zg)7R)VqV2I-zlXG?V0 zL7P`cg`q^v>-SyMk6)@{X162it2gZ27b0acMNWcn$K{;PFwWPx#opU#2^U!O$na#% zmu|j@ev+9tqTQXQ?Bhhu$g52Of-G7t&>-vHE1f1v32?xY%X@=*!tRJS`{2`w+AD7M z;_55x2?qriP>}1aH2;#Xn7uQn)i*n_IgvK;fefmLjeG{Pkrq7kaaoeC04*q? zhywx{QYpf$Nz*@-nn*uP&!~nTtF(g~ra#=C%JbUTO30V$@T{c1WP}dJWw*>~KvvWU zK|qvNSmXAie`71&97-My1J zFnR6L6UCw~n~K5l0APOoWv#uZ^HaGMu48$JR_o*@@~L>QBEI^$$f9u-<92--UCorM zFhzd(5-|?*eEDWKHIR-l!ZiqHsR3@S3T#k-G7)oM#k}*H8!M-gR2oum2uZc+J)@vm zKt=^9E4yU&`7H#H==22OnL*fo6J>3@rOw8P@7NGarOQM;Gmq?cL)!EmRdC}3&gAA8Lz;O zB!hy}vn0h_yR2OxL|*cNO}}LdM@QfJ zg}Z)4E#-9>vpUb{8?fMe;S|8V4gaTm`y|k(cVE*!TVL&#X}<<%b?vq7UU&q@Vqr6_ zw;!TL{Gt)z#f1aOH;v}6Oqq5PCWuoqp=-_FJT**`JMKYl4aqJ;8Gc)JqR(`QoTDQa z8Ps}l<<`L8b?uGdUz{{Q*9umlOSYhlI;mPeU^np--{Bq3B*l zODu04@vUoY&&M%&_-oTXe~GS&Ou2R+O8Zh!mos05dllcbv%*em2(dOT#TgT1)u4t1^#j|Y{skMrb_)@?lvzm$G3VM;oX-SG?Z)vjDw3&bbOCsj z4BP%LWv>8g|5t(0(ZhQ#0%#S#^w$z8n(a4t5?X?5XU0GZrXg-gA%(+3ZC=B4mbdVt z(6Lf%kxDg0?)a-fa6_`bS%gqi)%y%7cYN!LZfyrATT_Y+X7xwL$Vr&8-cK!8?j z`DHra(_>~w>m~&p2^rp`g(cUEm!696>C0$O&{eF8t6gm0V00$UrmrL9o!R(nMDSO( zKGx=z+i_H9>TYrPD}Ew$u3#1YUab*DDhd4r4l$*@ko=i`2+kD%><)XhLDAk0ht|h* z4a(KcqPgyv#|a^)=QgmEM@dTfkFQb!Efi&ZuPdrrb2_gUvdZF9ZBDrcLtAQpLHaGH6!Ftv-tfkaeSNx|njLiC)tggE}z-N?N1gZ;yfIBC8v`X}spV_o&nva)(@ZX5b_0Y%!dbJLu!n|E^{YPo^i!4FlN z+t+E2>@wDKK*>4&avgq_54I&&H7REQV!hQJw@hvv(k*4--!av zf>gNs2yd~Xnw5@}x>cwW(CV`oAcEQs%a}zIa(2^Y&c>55 z2c#UzB>Nx~lE!=@dM)JB7Ez<3gyg#duCGMzIA*kSo|TZwL{}B_H;Hs7aT)mxdL?+b z%b}I3eI~&xXvi#-U2ih)ye0@@mEKI6mZ@;lY3u>w@6IL(YW_FF2nBHz>sL{`xk_V$ z#uhoP?PwJEfH_n)G#Wlm$jMDUREZ^vMkbQ>n3NF0BYR`n{ac&s6NP zFYvNw1FmcICpPftp5tL3rz?p&1k}Q!KO&P%_&X&p+`jb+gW)l9BV09J?NhkY*dQ_s zDd@Lg)7tE_pWLol@6PNr@KTbV&SVMwdp<7~${SR=e7i1w)pUu~Li+MrWYjbx!XmA2##CC*9ZG&*SY*(QWDaYR%1-gjG)+2$-^&nb5-tBnh$$ zW4SmOV&KobOiy5ax7(XEOwZ-tF>i3}OqA9%E%K~jxM?+cRLugO2lVaHL^VxNCyhe0 zIJ9Ob3Hgv3k+MY4<)K;eRTR{sf9Td~u>xOaMStRU#ym9_&vmvgyYg^|irMbGw3y-@ zsC!}1=9kp$vK1{|Y8gZ*(K#Ycee(Iz+W?EPom~A_V=VrEX_QA`ox>UO{fqa6uJCSf`x}6KTbqOu#Pq&7um18aLUfck22H@#!rr% z$n$Swh##NM%TgG9J3Z>4$es1(dD8-%az#wg-9-?-YBKyVv^uJi>B@W3;h|Qxq=q zJm(7I!I+u#H)iru7E145Vn5;1EKaAyibiMm!R$tm8>yYO_qLZl35i0fRY+jgrn^%3 zofk6(os}LG(_Y!kxU@S3So4A`qdX+IQN<+AK zv8zTU4S7)>-sY^YG=X>zXbIgbDeaNKT&bLl+?HwoWLLpJrfscAzb&ndIT%*1d{Au?kjb0NdJ0Zrbia*pOaK> zH}Ba%S<1~~G+y6xSs=hmse$z5^)%s5(F$T&NXy<}LwdYWX;KBW`oRvwA&~Dw&Qscs zw%zd&IaLOWeqp+N+lSrNX$!p)FuXAv4sWmYRvBVl=hpl4^Y)4r;AaQSHL3g*HG89- zt+;~+*}?oXkiCuond=C95hJCfG1WQrj%&-aJI_3XWM^wT)z*obZgHhX@lr4Somk29mb;PQQ@bl)q<|N z&gfx>dE*lI8RR4WS4`O9Gn*;;1c)_MJet)yd}QDZ=eDYT8LiA}X?y!6?dAh~F59?h zAV&f>h!P22*_nRl*rN!ZS9El8a*7Vkee;!PUq&ZQ zE^+vSx>ZchnysLVnsN z=PVl|dYSaj(X0U5dDLRE%TFF}l)d>LnwgeLG?n<$#Qvz!>bK50_T{D?Bh{?$2LAqV z_b=Ff+iU*;O|utIBz!UtM>cpXR{5wUdO<|tZ!(NBLAB+5+WkC^n@Pf2pn#o74|tZG zm&2{18zSrmkIOYXu^bIq3SVZ6KWyEaE|R`}jZxUCGG{&BJiLzV?(OgL`3Y-gOKlF^ z0i{ouUG-9@scDC+nZQFt7$rjadn$AFdejPxMqL#WM$43!9`U$wj;YT-cN?*R1O^h034{aR@P)yDZyg@eQDogt<4Xe^E zqH?GXnkNjErIX?B37nhxWQ;>?9(D{ErVTc07?}t1X^n0jkq=Us9et{hyTfyE*b8D~ ze|b*&}@FZGoEIJ!{dF@!4; zHCls7*$zDTN$Dg`s(C&o^s06DIug@#WGD|Y>g4BSW3%wCgk(~Jg+L&L(~4M5QSB{5 z{h5=7nrYLc7PQQYPD}1udDh8!X-v{hFpRLJ-ix`Z>(0?~n%%Tys(M2SS8InuRnBKC z8o`GfKS<>%K!nvgo?a#Xm6^o)YQS2;$zLj=qg`Ff%e&Yd{InRO9l*YY-ZJFR{;E3W zYux>pl;x-C=%O57u6$K=wD3pps}?AYGCD!5CoYcm#Kp0qjr)F&yu{nPv%`ooVDTWB z)y0}A1d+ewp57`ck~|IE&b0~?a2l^J*?G?k`x&71Wb zkz;x6pg#BK+QSG}wQrvM99$n`=x(X>cjx@ft^XPA_J)Ro?;JfMqwnwc$Nx>!xo6(r ze_Pv?78&h}1pht6UkmxFD&(7U{B<;6DO&(6zbNwm6Gg82*lG3@e}li@=>B}356zFy zFhacwv|A-K?TLW?Pq^KW*!lIDy|V8YHGWa!JLdbA)qjEH7f61AHauS^8ezlf9>@D!rHgIbclM?-jM&D`=UC>KY$l{;=vM0)Ab};CZ1nmQS{JuOUkNf=C zvd+k5fu&5w)!ut-xT$lpE_F)Lm}-;rknRyR`OTrFl0R9S^^)>&OA9P>g)PKW>--eT`R-{y%5tEwi%cXI|E$ z_xC~m3u-UCZ~y+zgRi{r|2L=#<2TOE~)Pe$o!9QB=X#~52NnrKEq`+R2|lw76E;NV7cahNTl36EK9dx z)HMwM(`ADfe-!zTt33Zk6_fi<*B#=lIsMyud*#x9VevzpMv*LMKK(`TKOXg8?$Q5x zK3AFZGh-9_a{s@s`%h7K{V?msnZ0tq_n*)FpQGgq^wldz4>8#P`Y!*|tN)7)TTXLw zs+#OCzVKf-(7wnK)@jOsTWk_dWSidkoM)1ug-q`Gr;VZQEUKZN|KQ&TSJP*RhnHDN!z$&O-Amrl0n(_hQmA{VzcB!Z!`?9}N z@YxwO_-v4Wask;mnVws(f7fx^p&b?gY(_B3!*YY4R4Fi**4I}YO!_m`l|4bs%RJpp zhRAos5Thv^`8`Ck1Ll7#yYsz#*HJ%&7(e;%wx+T7ePT@nV(RP>6+N5m0BWAy@CqpY zu{o&1kt=gvm+OpeIK)cJE~z4KoUiH<$hMTk2myRs>u`+&3e?Zj+^+=L-N2c@yzkH-!+67VjzMWMM@kG|p|TD{XkzHyd!X3I&kdM;pN)56J(+8q`&g?eDPIqG9u=e`QxMMi9IiYIAm)8eFi zG4-t;eCZd)GcJAi5wWjZl%gdnd?fR3uBbES2hZdiP@8?5&uf~95>#i;o*UTm;04os zo9zU!WYk8YJU>a`Q%`~L$8Fn_WO}o2j)0D77G85hD{h}JLk!xi?2jSo*wh8#MnabY zChP_5Uv_O&C#{IS3p(CG(eyjTM~Ef#r6=cZciDYk&i{DgHah#4_}hwA=nuH;D5~Mk+GwiAw>eId!xKLc6WVa*+7U=>)P@To*~zfFrlxUtqIK?x3}ekR$h8fw zw_qsxJJWdq5fc0&mI(Y*4&D+gS$Qzg~C(VUNZ-w){JDcTJ|w&)>MFDVdj zOT-GsOLV%jib6~mRlsJH=T+jS#qJbh`+7i|o*9V7lx2aMi<%~JF zV=WC1=-PyH-rYsLSWBSV`h^2$0~|{mHnvH;JFvh&S|UR9oE0rDcyyqv5f0Dexz}h| zk&wE3v$Rz8g<@hC;n=JEIB`Yj&KtA7myYO`%{p6R6ydnnWUbezYVhoAYWi_+qq~EM=^xJ)>TJYDB7DG4l1v644H9kv zb2<4|$v{_^t+rh&OOSIQNqA$izK)|!%A?cM(>UA|hY*YK7G_M_d zE8#X9e9akX{{~U%aATNqnz6ORq^cuZE%a(li$!r}LT=WQ240*Y_=V1``mz7d`|Mi{2qpHe74Xgb~*cjSmk+?N%^ zC6t{`Jdyg{QeVFb=ty8}u@o4Qz)0i;@E#)u0 zI+ic<-KugKD49~+gPRuC#T?N#@M5|jSr?jiNfo(hAqmY?h{(d|5R=2N=P z`Y$sRW_V^Eh;KZv=qP9IGMy6(%&+hn-&L<&KXP!DTT;eq!<1Lhitj?{)e}8>f3ZXN zuwJnrrfAQvn(fk36c1#1V5pkSKi%qvKViEPbLuQcp-;jA548fvQa`~@rkRQ&{V-(- z`$n?snf9PtBSZ)gn4T>^jp?*OBVJMt=h5Glv=RBuL_CF?YU@>yzwa*>(7L0LVv{Oe zNVh}WOgBKG3JQcp8YE8Q*F#l<+|vF~{qfG*@0|5w9?^_K&x|0NNMTsq&na^6kh_v@`g!K03=eMAE&|PGqPCmD3!|i@y%kxD zCx*Hm>WNPFZy1{WO6C^0(5(~uVU`X%!S|ByeeTDk@cAmCNwvVWsRk6#Fs-6AhI_Jz z@YvWAoD~cJ75Q{kbq2cssl;rZkahqUEc2W3(gigv;*izir07jCYKC~Et20deLcGes%A7DH}Iz1FB7#p`>fj5`VFFrZ=!jQ5F^O-rT zh0V&nIPV-#6w1bD^Qmhan;4|vbyk|BO_p}faFDo5Uj`)?NMXQ`%$)l+f3#|)Mo!il zZzjIw;xG}JRGCHs^T3GspGP{vY&%y5=ijWA<5OUUX^_cE`2xQ$pEspTA^OZ@evha% zr@@Z&e3#ztWYYqx(NAlA8eUk1w|peobDENbwM1V>fB$|(UT^vns5P+Tcl6f^f#EYG ziB2UNKt%{yhLi zHW~Z;1A&+Klxe%Xonh`mg3ukFZ3N%fc!zldD(+I3hu_pP?`aM3X`QFd&nh`DfW>Ym zVZCfswq}(VWu>hYulqV@e$wU}Zx!c?BVT>DuS>}Q9RDi>sMEL9sA*6hXGcmjB+{R; z%6?WZrR{kc$!>Xq-YGWNUO702jTt=w^MJTX>?#vhsMzakpa1Ckd5@ykaS&j$Yka8g zN`_~jWb3@29+bA-xr4ZuLU#-Sdm6fTRTSq<#)92w-F7W=;;3Y#2DAn-)~_b{H%0&wUWpl&qUK zSgn=rz3br?VU#^sX_XAgF6f#%yXEJH$nM{sd3e@PtUGT$@_9;uW zLyTWog(gC|Z$W3S8J<-D6>|na%ywt22ZBPU@Gc!N&|T@ZoAUF`u50CFe)z!rm?CoJ z=G5D_P!6k~$3xEK^IYw1xabPe5S`xXV<>_#ezuJ7^>PvR2(h{Sp!=DZ$w^p@I7Sp` z-(Rf+HpTuTpFc3YI{gM?HT$Q~&eA^EToows=2+AhKaq%!YZIhDQp&awq!-ADe9gg% zlGU@?(4hdsDaVZ|FM4cPP~XOQHhd|k%6GOJ0;B(~Puxwg_)!<+0UYguL7Ff83lsgf zfXlz5+v0cM^2B{M-j)CGys5)k=0}EGuk#e_;KNMM`yNoGx&;kw;>P`j=(|uNR!+-# zyMLUkN)f1c=$q(3Vg5s#H=5IFpL%SP@8?Yvf;7FX{C|aghogKQWnODC?aLD3E@lf= zK8l}ReAcS~T~+A!ImglcYkeR5?0pjK@Y)T2sA7WhxiH&a`DtQn(16Cd4w7v(3y$}H z>q2&gPM^8(v9>LNH*rEy-8zi;JNk)(j9}PahAhYlny$WYKt+giC{+|{uOZ8GKRHCq zzD?JpiiiA6?ruE6LOXHW79&Iq<7ZGgX=+dIi2~ZQfmlQE3+0>3{{f z1e(+JzKyZq#MlX}99}H^1%7C|H+ZFUC^zpB{KhSvw+KYskHve*7HGw0?_P_J_)0>j z9G>>If{?6`ytamjUti&x)6RzaID9Aj5v1QeSV>Xmzwf1}-#O|0yZQ$!H4FSAe%tkR z1Ur1h4uh^Vb_z&aUN4~i#t8h<+E$Tg+|HtO0>htI!$ZR|J5hca@aF>491D4+qTbR| zWl&L7k+LU9v2kAtkH$|WGw`y-lWUW-=YWCwIQxO_%hYr|=BLBLN$MBg?{mU|j4m)P zyyL00{?4|M^hM#oTg3U*KZRuxAyh6B^baftxMo*vE7-3)4d3%p3Jd_S3_Lr zAr>q=%x?gO$#rOdTdL=*$eGL{{Xu733Tu0#*QvOYv~O;p_O@0CqkRlWNM-RFP+7la z!h~|U&8|bgc14Be4*ilLdb-<-yiqy${7{JYNdjcZwb`#O&kYYA!Kb3nYM8=ROo?7l z>(v$aM~w{{UC7v{C^f4zEs0=~&6%$?-&gc4^ixuM?~V^nTi8EDKifzmhkH{LOyQ}L z-{r1x_N=P>s`Z)-ZimS9*I#b6jIv6mqb+d;H2wt2$*)BUiK>k_^mnyem-#?BW-B2< zQ<5$Mi65{!iNNC!9dOtSw7iO^C1rJoUd+|KQ1?f+mr|EOYz5M?V1#Rzkx6hLWt`8* zw2eI@o^vvLv=lLI&*OP_W>K%waV4*8MUaGoag7@aeBwZBO2vg0<~()3N@+UlVgG%D zyQ9_|z0CPm^c1qyCAFtB-;F``lRJH?Q*Bq;`<+fK?JIoJan^&l*bBJR-^h8?C)`z; zrc~Lqiklv<{4F`a zozp8k(WBfXD4(}*0E4nV@kFdNf0!t)ODvHGp3u2_8v89;#7lXbg_1HZuda3r$X5&_ zV2p_ZQ^k5^+jGR^Cu!c{_UlwRa#G)?7 z;6`z6sb3*JrKQxQ1e^s7)HabShiItX-Ct?9;PD(KuGe(Xo>jQ4GcT_mp4klFse@{O zHNn0Z%l8R5bMA$vVgw)dQT?FPp{UR>0Xb${rDF{(WRF5tYZk>+S;L>ppwUnumSD0a zIu2M*GQT!@qQcIKrvUDQ-p(fsqV{*@qk7c`q=65s;jq;Z26h%T4Mm0hW%wvtM3=L= z84Y#bH6SERr=1iUinp{7mVlAJav1k~-Imr=$5C` zSF_9E`ei?x6lL;I0g^9zLc7Sx&&&bgEDrgv?Ef(*)0ufZrLP-pUvu#3 z?qAF9CY4eV!Kua|oNLP03HQVYf5{#O@Y1pX!<5k)rKO!XcfUR<QO@oW+IiTa-ihx&4EV|zcd`D_+r!eEJ#c*OO-8pS!%bJ9#n6aBvN~~ju3m^)gDdy zu^`nmQ1=Id*vd%LGgfa3YjkLKk?fJUOv-xUQl{Ma;>C+qxy%uxFPOwEeFq_jH?8tG zjm*1O)^B1yKmYmqaKK!BnubjWX+pC(VtBGc!Ml2bx|^1ms4aI3uA*2nJ_rT`L67fW zEz-CL5`Bbhoxa>7Cd$*Ve4vFg3Ydm_*lUGroFgZsel5AHFFv0!kNEsYRbh85wB>rR z7D_OZtPt0y#4w?FS5;HDW@nXjq>8=jK>6B(7&z_e*PEHrQl+1DZvx!N^)gJG_swY) z*)4y$*=Apf^M5y)ZaR@0!nfE|YEI#&k=FLW4SpeWU;|&zS$o7Hmwew!YvsWFr?&JG1Hn|#}0yd4~66MyPeRknytTV>O8Fi3|R%FIEQyrT4;24KnYtP zdfq=6S4sRhvWvl?e#};h4Q);^uc}{b?LkTxt69=`pf1lW5el!%?PRpXKMC?qbE){i zo5jEW8spu@4dQY>{q1Z^HGaY6HuBD@eH^^hmU$Wg05$5kib`0;3#KrS7s=P{_Z1QAe zl=*oF7I;}v%CVhZ+Rgz@_nE}xmE(E&KCAS`0$gMNPLwqjQ{LS2D;qhzzfGmR0f=~F zd?@iE;M1AtBQ^!b*M!(%+GP(qqe4=u48uII;42a41Gr<90AGAeDK^?B{f287V zODf^q4=;U)hI@r%Heb7%Zb+-;hdDeD)zm8g7TYaIQ%YHy`Vm2j{zh{m zujg%j+AvC4=vvkzRidyivF;s{ki?PbIQV2prjB@rai_+1WQ9%*`0XCMk{4A9!lhNz zi&o%Yyeiiow`(FTGXMI|wf(5xYa2MH4j_L8h4=~;Aj)dR)ewy1p5JR%lRwjqDhO3V zATeMw=l93m>Wy)Z=wzp)_mbAQzb|COVuRW@*=mL(mruGJCl(Q2o{C7a;CI;kH`} zXYvB5HpB6&S*iR4qjTf|j{ME36_41kN~iO|pBii?hC-LG(Mc=$KJD zi%?2ACRN0UI;JBb?aWMyGe#8sfKs6HN0Ke`b!!!mSr+***Rqu{1BqgLKngjVm= zlO_Vy%&FSYsQv49wy+LB0lK(oK@wZGZXYY?NkOhQV)D_GHEx5(Z=o3JjAi7OdM7z~ zRh&29``qy|jyc+yikB;@6u~rPmP&<`VK9e;&^E!_CN9?)V_VqfVN96lth zA>*#erU?7KI&?96>t|7>dTX46w5WzoZM`!ywtOvDbtpyBqs|)@ie=~ruMMuRJB|Kw zwIL`y2cyqu4$LoCRYj<1St1ywnFSe}u83XmoK5=T=r+iHgEOVEb1lX-6-Vd)(7ahvN zH|7&}C<~hgr|p&| zCspiLz}!hO1$l>$nz-9(1Tk%jNzI{b2COZ!QhzrbUW>v%^=1DX1^z$ThE+ionaCpR zn{PZK>{Wv^6Xk~Z4|en!adCAT7pI>YHLCh%PV3a{VQ93*qkF0+hR6Hn!`j+BR!J+J$MMxI}V#Li~MJQ}D8v~?Td`<8ES zL63dvEX|O7Xvqwya9lakFDfK$vLR*6)%MPB6NRzTE33hQs^;IN#S9&^kjxIJt~3N- zWzj4SQ=Zly6NbI8EsgK1YQ;Xy#1bGYJ)uwXs$ig}?GOGCi;KXxr^1Lj!1^y~LmV=V z7&{^Wk`KqH53d3(?5xJ^A8UMMw9S2vQ2Ddu>hA^sJgWIlAYzdaW;fO3JKx4sfDwLL zwH^z^Kq#v!j|uTJ%>r!j;rZ)1dt)IJuK+f4#K@vk?@&h^u~35X@VeEFP{aPQN^vy) zTPBbuaN>COIo?kFwaMTLa-|e{y-~@b!6Gb1C|asS=MlQy8$*4kegQ1DLQx_2t(iN@ZaJGS-2%;iIvwzr-f2i>5|GFR@h-u(Cu4sb|>4)SF$N z&}D)34Y5^_%-zf?6s{PWdp(?=nBEbfY&Ed^XO2q8wJXJ#=pAy5xS`ohP0z9Sv%b$2 z^_urQ#k~OQrDUJ&Aw=Y_Xc`vg$d!}pb$6Wh0}4EEww5uK^>Un|mKPvwuH`wbvGiCU z_2M|vhoh`kg#q>}4zA0hd7zy7pZx`$2 zWPW&t2UD8#xI;2cmbGpKGjQ$zM&Xq!sglbnbM;XS@{<5;-|Cu-oDKr?lrmYZ2qcdM ztp39Ocxe$o7<^AHIH38%D(fKYgNb{+B=IZ5{&n5;j*0g5>vJjjtuIbFI`3ui6^G_h1ZZPMXQ(a81-!d% zQ}t-O==XEDy7wlB@pF8BjJI{&roI8#$J_dEsd%g3?9Qi4z0;4cU3OR%I!k}TY?*Na z9_(@W;XB(>)XSaKuo^Q`eZ6PUEVf+lP(>~OaA1{=alXVmRF_ili&9Ok?Tkmr)Q?9a zJi{<~ox(s{Ds|gk(jtguGZq7&tLPySFZ)D3_Ng~&@{ThYhN3kzweh(b&Qm;k{C5q^eEdE*BI=S=QIbhPh_;iViI6@aMCR5Frf&a!>5ULpavC@N4_YQB5Sy0fby#iD8Zi1UX z-XL6&j$@GPirt3N(;L(3LeoY8LVb7jEtJ*Gh1lkP(w@4?$MXmqU~_%pfE(Dzt;O$} z$@x~+TT@LO$ycsdxX#f5 z`cA8#gyr=;rO-|;_Mt&$c8W8_$_<>8noF??0fYGsHK^cc)mO%ejaU>oZJDZeqiS1# zrFRer1YjX?KNdcUd{!|Q<#pM5$7#%1zP+hdrybZeBe|)FZiak960ZiJv zxkp{738N0QNs(qcIPE=8Gv z(le@nwW&1)(XDlgL7H6b`T#~eA<#Q}7d4Bl9}Px(`;9{3tK3({_KFob)x!2v35iD zrIK@#({(SG&E|e^Bzd990WwIPdV&jq&9V1IF7vx$9Eds7hd(E<%YbJQEY0rlhFH*- zqi)+Cuw5uO zIMb#hWSZvUZLlQZF`>EWM*g@yo2b!!DtMT?%VEXjds5N=0jK}Y-6F4^%+LWbcq_A} z{0`qnH?TA)c88T>9s|IS0l}d^^ArhV-0yPiZ8R_o(D#HHRfMhf)z=Ax_aa9T=WfC* zej)iLKNBAmj@NiEBt?&&$Pv5Hsq?O7+$TS<*G*mOz}L8x+^u%_xRM2vB2+>(!B(ZwNHzMDG#U^vs znaGhB%OQIbBvj*K`81U|@(!@!=|SpNiI0z;nkc@tr}X|`&v+fWms)Thfoy)S`z2C$ zweWkQ=ig?K*`=m!pY@`Tx>rO{g}WczdY&^|jy9i>vG?1^euU%^AKTg78B~hAFrjk^ zS;E^$3Dokm>`CEY6;q{jbE+{WPmJ1yS(l|DzLd=jU&=q;`O+y$D-TQ{@sz&ox)km2 znsp!~^K`U;VMvfQ0yBI`qquhg`Kt=RT_F;BKYUcjI?%t8ws_lAl@hPQKb%m<^abV< z{JK3Td12EkG^>^SX-c@sI^ecE*~)S&>FUtHN;fcR1I*5&Ga^`e-enaPaWV_#{>7%w8y0|}eod{6mc(iRaz4*fn29HcES>|+ zPP2xR)QUY}OmllxV9>4eNS`Z$`DtnSi`qUn{KD(fc|3x=OjHlg^k zT#}E)ENiNXsDQgZ9Z%6{7;D@346ks26GxyMls@2O2R!N0w(cX@^p4u3-R#TDGh?;8 z+E@xSYowgA72&1*@813|mKc0dMT8UI_{w2W*T z_67f-@m7d+(*uS_z%%a!p!GO80`k!;>OUpGOMii*2i)wa>T=<{PV!c#eC=W5?SaY1`q6VOvAf z^Y4h?m-xaaIvCjw`&I-w|3A+HkcFHEk+oFZv*4SGVzGT!kHxCwm~;G#`Ti}~uHR)1 ze~-UQ`nMnc3yf^p;>1fW;iObOJPtm}9-Fdlcr(bbKRU=1aRMv}vz8!B)Ozj={lEXG8wIkD*JiE&_ z(VzdI5t3$o}8vouab9$rKHrjMP7FA&tBeT2RqfC4}tp zs`c}0;d?ThO0!u+qRV?;ce-`vQ2Fl^dRrS*gqiyI+VF-jx=`7)8pG4EK8hOMF18HV zMWSUbe>^oA#XCaMobxy}7m=}FkOW0_$hX>1-NH{nFo~52M4>tJ(4m3(y0`iZ%TdQX z)Gbev$!;RTYhOgJ1$&kerpWq2EKNp1*F|DrAU2LzZiRmL0mEzT9<1WxMwJA{rE~M_ z>88wsR<}fCkz3|xgjighe|7*89TpcRkIP{d9m87B@v86rmb|ZQCklb4HnzKREY&m9 z7wMwIwvr!|{2)?a)N90s$Z-ny9dJwT;7%-afR@uE%cQmD_>Kuw7?e5O>ICL;uQ>c+6;7cGKS6jJv{S7i#iG2!PDNN}evea4V974)A5k z)8XQ`Zi#V4=?JmrSB2r&CAUf;jMT;cC+H1n`Y)KKm^9?P{r9 z#pMe05ATJ+^1@PzxdsVS_Wr5&vljW_BN|H)94ea4i!QU9)3Z=12n?4f!W&!)yD$=B zmW4&<0MZ#hdaa!>C+lSpL_+}K(ipu^Q_W{jW6{f{J{n-(z+(Q-utDCxq5!#?#u3wf zD-oM*gl3uYeh~lg-5-=}np42^qsOCnnem|fx@RATHt|t!gRB}09~xUAOAq^LJy#fz zsL({ichgp=60l6Re&cpVbt$4jy%+1F&JGC7=pLQ#O6Mvt8Pq>B5?J`fr)wq?9?6d; zINaFTc*%rMNB$h$mbJ3CA4AY9VX!F4VAss6f)a8yM%4q=`L3H+#gv16!k?XLdT_@0 z^o3ji%l|3)-x|BN^fPLF1-i!kDnfoZ-$(&0jeYzGMHCV|KjSAzg)H1qlk-emO0h0S zT?xat^D@u>wyM%yAuHb=G^fp5=MC7&>44EhXt>Yt3O{@yyNndwDq`M^jz&GjMrU$qZncGp?zr zy5fa?F5>+cE~UE57)(!h2wPczflv1)vlE40);9(r%f_bRp8A;%>a}iY$(PeE?c^hb7QZ78XsKBnq`OX|_e}dJ^{>PN&Fr?LAbp%r*Y~+KeiQK^J!x3xHG)T2ue#e##d5 z*%s4b+j*iwsPTtu-}`N!G2d~X=c5BE`sbJI$~x{5>G4Yy=ocS}n=6I7*0k`y@WSB? zw`BNvDte+gpAX0@k%)SvAwTd(2-7&Rt&*Kd+Uii4DNS<`*sV^~oF#l)OtKk)E@gKi zx6Tv*E6Rn60TD~TO0jm&B8Fh<1n=LQo)4Q*LZ`GhcjnbL$Q0h*6d? z#>#tBR!v!mjN+x|2;&o(rBs+=WWK+wvXXghAg@ovF zo$EV!@XJvigbA+_N+}A}37KvNK)``pDYWp+# zbgV<>K@)Mr*bh0?ztZwsSbK}rZZUOzqA(7h-kc$H><0I}==@yNxrgtPa9hcm_IlW} zmn-l#&EKvcM|ozmhNnKkeF+nF=GW>D2j@LRgKO(uaQi!971G86YrB7!jKLML&No-@ zDo?P%AT9e(o%GezUC(FN3p|v32@MOd`go_Yx$t?_u2q^q^c4@cQ%$8J*%f2Fev0O` z0FDyplK2lpCBcMuW2IkiukPvlBzgr2_Gg}b8ncyp|HYR2<|Sa{e`F=YHN)l2Wv?)a zlbd3-f(7D0tKgeim_P2-vhPKRiydn(kX&&boybRRn8wY+Cnhz9RI8pX*=%@uhKQ_u zEq6!@o*Mdk^+a`23v2afr97Q5IJE9nBt=V!77=kbst&y0N%(t%XX$zyE=HS==Lbm2 z`qx^gZ6@f;m6rdnxq2r`+*Svifu!vn+B_b1VcoysbmZ?ejp;>}`gr&ydqwv$Yap5( z5x{M`SOQ%t_onIiRS`}3Th2%?wL4%Rw4WjA3+iVje491XL@cP` z*I142Od!4|4kuyJAKtb{<(TWwNEacz&DEfZH{|)vfmV^F$OK}S2-8C5*C}7_!On<5MtG{ zZr*>kmr4_m!zFFdvPd2X{h7s;?a^Q4Z2@wPVuS^cj95@Om7EY3XXCP>0NeRPuBd;$ zx`OQFZZA8Cw}a}DzK5oOCEM5PHZ(ox-6{0q1Y9&6U2h@9Hi>>Yl)oha<53t*hN7@l zzZie7KvT^wNpB&KN2)l>m5I_wgEcPgpni_@Pc(jOlr{yKJvJjC8;PB@W-323$L0e@ zx;bBopr$ttlLxh<9(xRz(ZW6H%EpkFHygt5WBR)4wl8VD-9N_}c=WI_xN(_tPirXC zYQY?@nORFz>|v1hpwkz&wT{!cO1Vu^A^@5FAm z^NIoxk>W|Xtr@Y&Q!8z}r~G1ZJ@+R9vz;TD8g)zd?F*;9_QhR~r|eM67m>)Zb^?S8 zH!#oZqPa6$AyX>ka&u3)uJ2lQy|Bu|BI3#$kh>q%M^55ilC)DEPp?#Z(zS9CfL`d< z(Pc!<#7!KScaVpNr-CxB1rb7&SQ}BMYy1+FvLQ5=@zo}BI48`lH|95s6zVk3Ak#5#Wa8lXWC;<6}hx!dn4* z|8P@fmrKDST1CFzhrvV1O zxYCFdJ>gK(pjqn+nY5Se6k{k4<}~HNHRz+cl};#UY5XkA{5lsH|ail*KY7>&$RqN zF5p@)B|dD8eDTLlK8#qiAyF%&k)KedbvfjbA-WoSyBgxx3m#PY5pIifRUUz_!WMo? z>YXP!(Q1`K9ePtf-d&wgm%MIkrFII9PS6wYfUU>#RjENNYh3$Tn)foQfNb;5{J1{A zE8zo~&t`bpL;8mYt?rX7=91HdwL1CzZQwoUmYk+ss~vo<&&U!-FB#mKyGP)K3e6PpfNv#BgK0?M_*+yUymv9`E>^I{4$qLgY@zK26#) zzGJ*f=QLOYqHv)3HDGB3L8-`XDqBKdWzBU;@MaNNADNR7MsPDu!OMlFOB0$3ocswZ z8ek_Yb=&X=iiJv@b)O=7y`O8XcV3=013jxTFW%l_#y4ec)i2* zN*Fs$;Z8jfIczb`em+*+eYz#V7G!R79=C|rKK&KUgQrYwxn%amqJC(f{#lLCWUYG) z*Ocq&Gv^irvVFVHp-lz?BjT!>nl^h>N*zf3bB$32&-WS}e~2&4Y^3fFN&=9}j7Pxq z*4h=J)5Cy|I99K#%q^{kzce0-%ESu-fl*&Gm4^p{0X%=~Q5K?jFZBmmB73}N`$XxE zE8cUbL-oZ0glSKbS3oh!DNDtHk;SQ7=8etZR4b4DG9+|H)>gIr4mx zAX%ayepQI3cHp8@V5KI290frU{PL(4R0g(x~81uXPny%A@Mz7g-AqdlI^%UO|VSAoE`~HmL z^j3+7at8pDf5xeoz|>@h(CtK<^w{f;Ilk+T{;a>^aq(YzR{xeXxOF0P>d-sgYvQIa z{STcLn^?h%Uw*{g?Z^^KCY8Qinp*(XUO*7P8{|Fm3NnHbzC}h3vR0s&!aqCU8Q<4g z5Z&2C)aYe9cMJxE;0yuIAJy<$(l zEt;}THxOp7aCSk(mP!`ng!jflbLZBI&X!4}Bd@JaF9d9Bl^IVvbx}EOBc8I>PMKyr zaY{w*W5P}1U?GhOnCk0$QgfnemCqguCAfj7R9FhX)v5Hb=t=uYMtVS%n!wuygRlv~ z-FRs4dmL58E+8??iIwgVi1N@*#1J6F5g1yIX9JNADek&oFM4d>? zaYJq$DX<)?VJ&WG&^XLX6I#>x&m>(fyxdjn8Y^bcHkf{1VLg82x;;PJ+Zyw=WwAGl zU^%8!pWCFd>z^I*?1RkfAncvL9=USM6l3-4s*iezrk9{wF2lLOc2(a)J=6;VP0S6E zag8yo-Nt6l9ZmU&KDRh!WQm)7*hbp@g;B3zu@`z-%3h6btkt-7lwFrVYhhWAY}>Nh zGiElYxgFonVjk>pxCitxO)oRO+!Qe4ZR|4KSGZv3;$#(J{sz;Rcpdqgb5cYFY?kD) zwVluezzHKC8dNph2=roYMu1{bTwH+KIAS4_fxC>-s!k)zo(tE%V|_RH(jZuji+2dd@~+I{;sEcD8)WmA5*Hf`|3kIaa4<-MazZ>p<9Zu^-*iQxEfhq); z{pxYCCch5g#$2ddEt-_T)1*Tf9rOLgdCxmuJ~sHK%<)eu!r{@%-quaqlCZdRrV|ST zx?)Z{)Ve-hjyCof4KGfZfL&Ht{R}7n>Y(IX_sYqn4)}2hSNGdVb*iQMvja-3Vb~$z z0^Z5T>>NhdBXVyCA1}pYpPV+(^R4Royrq~-t19`unB1R%t+xhOUu%(=3X+T=hjBZd zU?zIb6S-Cu`by@ar`$w+1nC0I!wx9f!TCp~D`g;)vkoeNKOVWRDIP(8W*emwZ#TPk z?5@benB(JG7mAc+08!Jspo^+XtY-v_!h_MZ8_kHXp;#M}E4@X8(wFWwoq2AirBo#5 zG^1J41Oe)Liucc8sE+snvoT#C$2GKCA^K3I5tN=E-Y405^{_#j*7=T5hZ09aw?poo zyuPJL&}X|VCs^}FDMvhiS;usR+%2z-hVaqL1~?08gT|&5aIQuJu1#wg)9l$)Y%qav zLuFP>DZB;@UjDHsaO658Z8@Zc-I3v)08VWE$#RHj3mS#97J&O)VdAdfwG1#xSiRcQ z7O{e=48XTiE}>Co(w-BK&ry+_fgnl7 zw@}Mx8`H1p%xZv+ZrT%4P74x$mq$x-usoU$g2Jdm4Hq103AQ;hS^#a^gtld%tJ5Pw`Sx1g~>5 z=aa4nB8>nnc=dJedUFIJh#UqaG?9s=L?GEpa#tt9@F z$;`SAF5Npo`3^hUGGzKR-1EgnNCF{T*2r9!a+MxD{A>H;=^H|H{bPUfJ}lxV2Uf}3 z{8q-t;u*a=_@N0@696e4DAv^3u^ppITT>iuzI78>k$xbl_kmHj01VT>lGd~($VV*y z^;C5X_rx*p=L58L2*RcVB3peqn8%ZCNjmhI)S?bAFAfsrg~FUhR!k2d(Hjm$8h(Z+ z=n~6o70gWSK1_2V#+)rpP4Y&kpc=Iap5cq1UDT&@Y;$7BySkzuP43e^wUPiqEYW7 z3(rM$)U_Ka-Pe^z26H;L*TcIz-=M{i1=Ezzl^=bSuqfsW3}mpb zc~{Wy=KfA+&OlAbRK|R54FqB8;y4Adm$G0n2}-nf+0%&-PDR3|3fya6WSO)sbpfW| ziy4rO>2PrbGo7g-yLo{(8L<}4--zg>YSz<(ZUH(IdmRh`(YsM+3amw_3qgqbWxFzy zKy~vx@|Fp-RvqLU?C(JZE-6cc&WncT|803$A}B1*4p7)%`Agw>EGJBmWM`~bEypM;0jUpj# zPN!XyzO}Rc5uHQM^O7B_1kMwexgkELT9cFi;{_TH48o`HY97x6i7`ApuG68oG#9fm ze9!6`13)ajl1pxSXC&BhQ8T^u3+B6cdclhY+Mv4WAGPvM$>inQ<)YgahJfp^#!j;= zZ@&ZbFi~x#R8zS2;iXzR?_x3AoVDX2W{pzIMVQID@QWhGa^ygtfpJVU==wKO^;w1e zog<3R2tWW(bC*1f_~S>Vv@P#mh)Y->^%yA)StfE|H*)V2mWfwr4PPN5LSd?JJS|j4 z^X#YUSqi&?#&aIW+3}GF*(`~l-?lkh=&a6ZWE8InUX$LA{HzIbJccm}Jgdzdl-K^h z*!#-3s_Jdq`MnDS49QsZWQT~=Fp0Oba!`mNk5w_ zhdwC!ykDNzFZ?$9>^*DN%ztLhnmubR@4=<`aJGSLtHD$)i$Si>T{3#J40)cAI~ z0Zo(L!V(B?3;E(#)T1FH4vopI*54D}GtWuC-e9jBOu(XNr#Bliiu|G>;?v%dpcz<7 zRDhWqIFYqErfA}aq?AACbvo!XWx>| zeG}WPseuHS(-(pi3+#M60NO>fkt~jl>r>-totzwOsPXqvhN(WR5h6G9mpV$-pY@_1 zdI+G^>||d;?~0B=McvEt*}zr=SmR-yE3)A#2D6tX-(}dFc;Q@io*4MD^t6?4P9t*35RTtY%ojef4)=Vd{g z#2A+~eL{P#po#Vc)M5kGi>-HkioZkLNtQ-Wug_~RF>-U-dq+nLn2cvk4AW0#Yc%Mr zHtB{;YYa0?=ZNNCFf!7MzC||X>*|_Tp}fSvzg?W<^eeJuuXk9|wC;6tvYHhzTH#Ptx=C83-9kGfq!wQu-EwRVI3(vSQ`EZ{25guNxRn1^W1CsyKFXmXXgD;o-WwU ztu@UV^di)MoeDrRGG%^cx$yCyR*adkE)4Cq)Nwx=Op#gq3a*VBuXtwl?`>#$tB_}N zg_O3vBD#~``<4<3)oR+|FNEn1p{kW}6;ZBpd%m{$7M?FoLXwJh#at>$xm*C7LOj3? z?zzfqUeA9&P0uA4DLS3i4kH-2A2pgV$Cp_2DAqy*a}01?@6)Vl?pQp7gs!r@Vzsd} zjnGpbDZq+!WD1XG3S;$)U-QaR&8G#%wsk_u%lP~$oc=Ns-?+2%o*Iq@&|Db>jUxmi z-&c0XY`OKZ^L;KRJ}xAx9dpnQJ>rXn`Ss66*o^Id9gEhMaeBjR(@r(AYFx~XW<@Z+ zBF+Fg56^H6W0-*grChMfgrSLyL}09MfbV)mtq|8-I$JFj&U_hNrHpY6zYepuGARxn zJ9eP8>QI-+_{4hZl*h7(65V%iT%)=uUYQ*In?cJ9REmm1VfBwv6U?2Ib04rzCo3x3 z_R>ne;jUfj4e89RRhqXGdngZ2on_kFYMWo>)gp*DJN15g+>fh4`<+2Q+Gh7eQq7nK zDX}|hADb;#VH=O7cTc1$`olEkdshj%BJIc_Z=01+9O(cIRzx7K>J1lG4Q~s_LQy1JOMr z?^154;^J47$ztwQgTbKz=@ou`z-eCp$kr=crR{)**J@GP)Py=-$jlcU4=35XhC9%i zqtR$2WP;fXdl!#uKw;P=hWy9-7}zVqh#=7A9=G9&x6wKBW}bM%OVNJvWQL~Ngti;b zfr|`Xz+o9QK)or|j4OYQDqPtM=!yKyS zi+kJLzhY9I$XN+@!Va@DOS!y&tA@rUsWDz2>q_^@fB~V%(pFXny+BYdd{5&0=-Zy5 z7XGQ$Uoj`M^ol&=lQ~C8VUZSo40J!yxG?AF?!N_P<;uP?!C)FVjRu*SlDR9{QSE(XUEc?f?6wFHBl}w=P{8uo|4lE{ zuQ?BPCk5Qode&^CVVlM3;5WIHg3V|}fKX(e+iOH?ZT$3U-?U@zS%NA>aEb;~j(YRM zQti^jsTxR_<5kXeUG`5#N5T;ufTNDi*t;X)cm)yXjY#=(#g5cEN;W;y{+WPi@px-~ zAbrJbR&DFx73#;m&J!+B6EF=#^i!_`3R#4L8{sq><;)+Y#!QEzB-XZ+<0>u`Z(7kN zKw-_C+(i^3{96-`771bZ z1V%Kq$vN{%JD?#pbIzTaeHB~nZD}oo zY&+6T+jJF6;}NYVnwqJtM(+r#_!Y(r{0fttWzn9RUIV_Jnse9?+ILr1Ar=*3S~IsN zOves0JrN5x^Bq+D#>JVVRjOk`*MJENXu+0nvTJ%VmaVCTuparLFj0a3+`7k?*Wwl7 zQJB3PS4$_((Q;O;+DV$;=wPK(st}! zK#1Su3FStU8s|ek+I%5*bP>c0#ZY;709H+ zhB=w~Azlagp7&GoUuLG_3(#`Yk#AA&>^S*xl*86(K^g#widpnP(XG5pg;B?#D>exnf$OR2?34(zOlsOrzLPcrvi~R+a zim(H%bJj4qY)DVzsu zj@y=o4{Z24N-uV-q{wHiEwqLYdN)3Sg?fcNVC(DCF;7xqexcqJH)sw1zX+Iy%bfHT z4z^dgaFe9{#W%F~!$z;qpzsNJxQGFyyU;mbobXYwDwD$IjDk^y0Rr zjos@Np>BKu%L7N)0n=GA*2wd)*!(e2T%DRQk=JjldRW?7^QOD3>WvP+&D>f7+L9Fd z?S(|l9$HDybKz&3YH6RX)GS&JujXJT&2dcFo0;kkbcqDz4u|;#Mk07#6QGz1T+C7t zOYI4r)O_X;B>28H2$%<5&3^18RC(d!q(!a9qP)n=!g{k$`g|C!^tz53asusMN zP&Iy!%TpU|q*1x0K!DH?kALNxRV&PB1y%~>$Od3p@1flMuRly0p4UpfR=OB#-N|s> z%fQ5#2Rimu(SUh#Nf>`)E9xO$0$bCXx%b?euW&M}Z`y7FM;I6I4I^`ph>zPdOc^j~ zA9H$nlEIc5^S%JM#~2tOKek$jQl2|2#`BazM|jfBZLT9*6%CHdQ-lvdsQgNt%nufV zD}dSuqlugr7%mEy?GOm_p14|c)3_iIiV+T=NN0>C66v^vKnm4xw;kuYtzJiR%(vZG%Z)?B* zC=!696vwF7+r9tA3o#57{TTDt^;^)c19rJfAaQt1a@?Mg6BG1FL>iNCnQcTkpfrw^ zIQ8S&+S#1v&b?iW!dpo@P%JP=YqVPzz!*zmvZ;`V@Dk^A7C3clH^@3kpko#7;;{B? zhUp@s2!M{gqK*Rp5NO1EpkwF!;+`G1W1KT1+xut0z?+`i%@u-+FanGAf*&?Ty_YmJOF;g@1KJD z3%kp})GOm;!xk`qoh@(&0O^H}x_4VFfboDcEfz5A43Eu{@Fl=Iy3&7Pgf_kjBI!(K zSX(zjA3NBP7z$t=w=wFs>{(QlAX-(nnk!lSjyUuDs?IpZf2Gf z^tsE6-x5!IQpx9-FQ$l(jdxz#DZBP|0#1amOn&#HG!+4Pb&fdb{sy)q%*9Cuoy{i# zKmZ5{5NI~$_XlNC574{og7~b-7V|4{ED+I^M64~~g9JA%v&B^DxRx303Ap7H&CCy5 z1BLz>Amx}uvlBub><^4PYnM>;$Gl?9po88lP}J*>;RSP#XP8>marYyrQ+upDvb zxPAvkA_DNJxBF$F!Ha}DfUghVWsW=KoPhJ#w1zv_=(m3Okh7iuA%*!;ZK(VY+7FC7 z!SVnE)?68&QDSf?&^dw2r;kr)AC{g1!>PSq8u`z?-?kh4ap5ioc;}dagsl@_f}a9j z6N`SS5j{%lqMYq30Q%V5i*k?oP<3^qTfLoA&fGV#4%+}}CD=+fqyV0@#fiu3I-?0)l=`U;uwH=&+P98PEFw6s1O3U6-I<0^ zqOF%foq)7fcr~6LpV(V~So72fuyt$6Y4u0(H=#a&<03FwLki(PsHvaa2@Zq%;=rJl zmH2}axBwWZs5`38!ZO$c-U;gUgoRFY7n(1~)^8!g3FxB*yc)~{aG>uU&}4+ib6v&Q zmRN}&(h0@~B)=xiaQz^5esV`t5|o}$(k`~OgUY98n=0hhx_W%6KL_Sk+Cw1n^yyR?&#ZOfOOx(m3L=e|nO$CtJ@z|xbs%UiA`iX51$)mKE>!&J`! zF>hIpIWa+769xY)v@hp3tEtcCiyT`>1iCMu>-JtGu!j1{xqE>9iMtLQ-z7vLAu>IV z4H?;b^BytKxhUf%m}M-m`i2O&fb?B1AUFe1T>W@#$xm$JAT$-809Jx*V1T{_IR!a@ z(rnMoJHu3AO|F{dpYo zzU27$j00H8e0p^P?6KL1uJ1XK-|6zGLYi}AOZsDzg+&Gk}UxIellrJ z9I%+&9A)TKyG%gAR4(M)mYX?){w;CL{*`uK&o6Trd;yO3r5pFV>+?@W^25=z*=6#f z$x{##eB!0C&zE6n9yWM`3*iQ3mLo3|VRF^*!F3nzO2B;*1{dC@F~mJKI#4kG0miQr zs;*R9G3-1Vif)Wbr`XXSkPfy3W<^9~YIt9&4)yVmH`5@}D!sofGN}8h;stbZ&QWCR zMexndLYnu;3%_oKP|0Sap=ghVhC!hJTLFRp7;20fGD4Gf~GyZbAkEV zYPrnT^zES0!MxKCutcdMKL*WnQ~~jN0Yd+a8>VxL}lRJhKKsXia4m}2j2a691(6gFQa$5Ax*rbnR^Hdh2Msr9#(&cK)`UaS_@+ClI=v*uL?r7r<<& z^Bhx-Y`R`?KNa+#XNj8t)QLk$l|4h#A@*)J?-w2lyuPSIRNjRL!N4_C~pcw6c|CCTKU0 z&D1Zq`j_%CF#T7me*p;%QU1c_rylLEZ}|(7|35(@d=eaUihb`~|q? zyD@wD?j`u}XkRN$bXpA><(jgv_b=@K5j=n#5Uw!9$qvH|C4Ry1K(~Oo2f}A*&*QVW zJqrGIt;8-+kOJA#1On62W6wBt_y)5A>Fox{iaIpEvCZHyE)amk(PelY$v+t2gjIl8 znbkxd=FVL^EW=b#Fabd0-!7^nOD=yxXM_^S6b?>ALqBuC#M%+!K_dcapkygQgWBJK z8Yp0MR@!Eja@<+zt`Qde?`Bl~Z`#&(q~GZYN<kr6`H~tgVIs-IrO?g7o5l2W4 z=1)dI!Ap$6V*P_%{ey`E7|)ymW}cLY9f`l84k&1TyOwa5@c%lGUBUna``3Bw0rFqx zvBy4z|2mIdU-X~zKWtb5t0S$BHOH4niMgn6 z2WC;EUFRLe#-e<;?;QUER15?F0v!ac$4=eu@+M-NUu>R=*TdXy1m<|VV}tjofna9l zR5RW#x;wxATE`+j01*IsW)#VvL=pNxaOcYOZQ<9NJXC=|*%18hvZ3SlW0NS@6$qF; zvu>qAOZ^UQ=_t$qf{X$-m69lki-Ng9@-Gl|mjQzRzH^Bje8tZ*> z6YnhTTG7suHjpHIqVubeMD%VxTgP+p^2Se@EeayLsY->AxP5HIlPNh8FC1wCr}%*H z&AR4F;d}KF>^sxIw1NwcV+a8yK9)>Wa)^dWOX&nm+*)>1$93PU|P+eZmOVhU@eDF>M|C>q%DUe2I%k>Y);f>P5x)PT>You#$kQy# zZ&*v4H+{UrF$!roXf{b_mhRKfRv07$GhNN`p8fj{RGkKIS_)oq&{%$1*v|JcVNURX zO6P!z!qGN>h@wgbK(VFZ;eK>nfU`K?QU6*}kbgVA3C@D!Jn5CjWX!vL>yQi3G~l8N z7W2;U)h$R1K8JHnK}%h1rwib)ymro$7FYM9XD^0b69;FMN>_C7!TD(DKy6mHiFxC7ss> z^w*%3dH>F3a}or$O}$uxJMlnp+ot?7xr}#i9VWkoP5CWL|4F8)ZnH+jQ%n3$mc1S> zqhm#TV4zBlxgl}fZmTM4ui*CS1eTNw02-G^b&U1_^bpCxBq%HZAV?}zwt7de2U%I? zvM^aB6gTTUv1+nVprMHzoAB&3{N;k~0&TIYXeYIWKNDYcF@LCcvF>qPM0)UB(xsD8 zATm+Nn>y?(AL%jf-hC|n{+^5|guuhh<&7rj1hrHq*gFGsHOwf>ZZ)+-R>&$wrm48B zWu+{TrP6=UAjTt0Wx#C7${Lx~Aggniqd$~MRdGP0vilKxJZ)S9T%P^*+Mi>+3w;=r z34n9_&%lXB1_Zg!TG5YNF9zjdQ_U^F9wVB-x#ow;3Ko*~KKx_fgWo7z+U2dPfXWNS z%x*`G_NPR^#(-v9=hIk^_14~pNjF`_w^WMbFVNj#FPQ=u zOX70t|0Kcf0sxHXC#Jg7;x1i4M{ojAuqJ)3_Jo%?nf9}ihC-m?TaU8k*}hH3g0bO7qkOc{KI~dYE>7BY<@=fU>@ZJ`-VA2J zhez%pHn_$I+UhD#qIU{CRQbDD8}q1&kFG%h>jCRoab8n5?>oJB=uO_ja4c@*9jqO# z$KCH@Y|Ils)i1eo?~nitMpWN&v56rmB;ZvnbRh%~o)Mn_IQ&VK3^-wvLwm;NeYL|w zwt2W3+KyDH*qIOCk?-&!!yBBl5E~E1J(5Wjf+~3+%+0k=+V>dfB%E9Qov~(Jo0D{Y zuG>ZrvWA;a(Xl!=_LC_EcpYw?C*=^b+;A6S+Og7vbTm|rYj?EtjXOAJAU59WXWZV{ zTv7eWfp4du{|#2)Q}9tycGr~`zhaQYt0@?Z>`+#f56)?bjW-+BkpEdm=`4^@!qM!# zxX0rI>bVMzzTQw|qKto0U-Mfdw9@|s+&EHQcj2;BTFwCMbCG_CHE#w8;!9Ud-M0n# z<3jarmu+PEbQ^c z0OwfXWf85M%;{>=rLxw9#jiQc<`brL$Ua+Z=7HBiFW{s}7z+c>Ef$^BFyR#wI~Vs% zeV+h!xwy?13W+uIx_8+aKbFW0y}V7fj<-p-`L8f9hhLgD*YPCZGXih5B@ z_}Q~II6E-YzZmhs&Zz#r3>$MM|?Gp;{lfvfr9m4Br1bgLju&tx+ z5OfWx93*{=JD9*nmELc~+iNS1YHWAhyN78+=4GtkdJ!riUf^RWv%Ws^jvf96IXFk_ zQ~L~k50}u!U_AqN0lq*w`nUbN13Flb!lwKDc}NiKUk|s(J+1tO#g6!Ssc$gMpdX4& z&_y^E3uHZob>S?t_0hFgLOnwe2y?q1%?D$iZ4~tp>WVR-N|Pw}dJd2F#`%ovEtWY- zh%ANASh7BYx%e4W47Xk%)^}*FA~4R`XNQ^&*I`fh6gNJNGf4!a-`~mbcanJH^ye&9tfh#4F~k z?`jZ4^6j$HZ&Jb5sE+vl-=B=J&1$oAtmtY6wt3H6Yr*zrRZ8Hu)EME6hj}mF>r=I_ zoO%-!E6*@FI=0>0Rt7g>oleO^`P_!agd=vn1RDdjk*$YrVgr7FR2S6teZ04lZ%vJP z7meO)!vB$-pN~1hjVsIn?lOca?b<=>0w_~1c=i-SFkWJ_)r%EC4ZY-I1h?b-yrz}x z=n?J&GZyfGs@buJe3S_`fTIa8Cp=GM-2%x#+phKyb_7{*NXs{}?lIiGPL~%* zw>AYGx);DU3&F^6z+hS+Gx-eT-ouknYM^eod82Xg(u-FVq+6>`50T3aDWHjJbLm4O zYEU?Q2P?j=;jX^e<@v?wWY*k+9nvTQyGSvUI3x-l0yma3S-DgEj;)Ei4qD-di_>vy z+FqQu(-xFhBEU>YYO$cJ0im}yAIz&ZFhJSSiXm^_yq1boB+4O8{|9V^`v6>vK0Zv{ zG2v|)67Yd*vzKtvl!Ufz0Nxc|rjsNAYa0m%M#Q^7On5(x4!W(uf!}vC3u`dA3*_eS zd(gqQ-(R5mmk%iDe@OYSJpRh#peNb6^A{g~@v(!3gI@YCKK|n4FFs&JEPw4~uXX&D z$6t9I6!OP8W}c+TbVpn{^@*`x=iL*t=Z zb*%y%8FK+0#*uzr4Ik3H%;#1nJih0e_OAx%u}x&p7B*Qe3ihUs=d4)G=e?d@mgZaL zeB8sIJ+9q2|5&H!R^Emi&A97^HqHD{jfr_{sJ5JxeTUpgb?D^WLP_RQKi87=WJyp! zBc08vP-nzUi%a{qkm!lKP;vZrc39rk2{0UdgJ|t$PVG#D=-V9>?9BPMscc`wqyh;Z zb$j1K)-Qmbz+H&Tyq+p3D=txA0xjU)e)G@^1qmQO@&v(QI|H|4&BEzd2o3_Op19Bl z_uSlSP$D5OwwCFufnN@KYgnB0w8Y87p4#aX6wRbUz-OGS!00_2QT|W>ND*%SI*awB zZ&7dcIh&zc9BdIr+MikYW(#0##QR1eXT=T^khA@!Eal~GbXKb~I_*!+zV0cXm7k=s zyu?>oUNGEYN!^w$te89_mVLlS}jo1j4iIh)bDgBqd)KNp+=8XRhf#MdkP+xYpLJD~dq zY!B2t(lkt&z{-v7dKM;O1)|MwSWi;7_Mn4eFx}%!_*k{3&^*<>$M0|HgL7gaC z;^nW0s9P-AVR#4H(IF%21)C!Nvn^2!z$DiQ4#VYI> z@CNjBoc#)wc+f?Wo1z1G-D(koAtcslJ*(5w3RjM1kpO%E=lv|o&u{DP_!q(G z;K;|1Q+5_r90WV~3xcQlU{632{@3-T7kaAQA>TEJCHzOq`6gHK4^|0)a^S>76XJZ| zj*>7HXw^z9HkLW2E0&kjN-mB0SY|4#Y%>V~3S0+TuvU&(oj3Fm6yIp?RhflrAv9(m zGU#Zsdo>jlpc$0Hxwqm;pVX=*V<_|h!+B^nF77rQN@K~ z@H4My?u-&3Y&g!-+2@Qjw!g|jncjf{+V!dFp26uRnsvGKg!BrEowt9OGbw+twPN%# zp0E|ow_O{6=Jjv7FrB_##Wp)Mp{>|nir|7w#}5QL1<|ZMjW9cVa4p3*pyuPWXeV;i zBofOm0upjS@rW6|(-DlLVBQOE94(+Tuy}?2`Vswpz6ox~!5Mu^$hj`siLDQ12xtLD zS;`*dou!(Z@k#5=ewEK%sR(z#NrR^^MNA^qFWT-ZkwAn|k+R3|xsfR6D+)SLS8RRr z6%HYb9x-{Ye(}w2vU8i~b4~UfwciQ&JwgY|$}JoY6U!m9`v-mGjX=2h?DxxfF1x<) z0YyK+e(SRSM}=KClJfwQT z-oaoX0%5(D;~Lklxy31f-uaKyM~x}qZg@^n4X-h=B8-eu>7Ph51cx`4hdkSc#5 zGBZe3J_y3?5t3Aeg0sq{0DpjFt?Km{zpLX=Th}W z^;Xwbhjp+Bcmk=-V$h+T;vUboO~SkYUj_st4RdqTcky`>nL1ZG%UfCY_{CjrbO%BN zgy6UD7M@bTZ0zSGo`bFe@_9usOi+)+Tt1D02a#M zIGb{J*I9s#MIy4~&5Q0T*B}Vw-)aQU6SsTQqJ1 zG?3Mjw-2Z}CNwsi20Z{wpT#I&ZWnZa!ukO~|KD;Sd7=;e5{~aZFy+Cys~s~I8=bw= zgYxUWlbva8N!F{ColV(oy?EPO?hH8suEk_~%lThZ^rpm<$)`HHNAyQKF~!s4Du_y*^kK%j5ON~=eU-rlO zot{ELbQ3kboZ~|o>r=LE?tj6;fa2LWAKZW_KHBZKwl63!fP_H5@(zdYn;brf7dwi> zr(m{*HXLA4i=(Mn>A~cD-!2I6#0P8A3w*(l9V#CQ=RT9Qw+tO4AFMitq`<>I*hyBuwP-X%rNi7^j%CqIBE0ZD-w*E(+!EP`83C~ z)>CKjFY0F9CKV#Ax&U6Z|rTvy0%nkYuuJnRa*g~SBX8sY~NwyFG zTcpumthfQA;s3@WGPrq~bgS_ww%%xsT(n95HG+jo{CC{Q_ysBgMLnWAoaTcY%m+K8%H_2T5$-$)FP6_|jV8BTZvzl=-Ospv6G#d-ILp+~;XX z!$+qM_e<%36R8RYJmHJjfsxUAUy(J@ZFfq5g&j8<8H^=CFw%E)tey{8Xd>OTUtR-OKs5?d)7!0X}MKqBPd++}Xy-FJFLc zfPBX~b;TddBTvAc+sypF(fG+y#G};&fc3ee4aV_LJKm*n@F_sO*k2i7ui)7AFf28} zSYx|wroVClEYyT^JlEs8;}=5T$J+D(yy9rH?6GN7?L<90-G;tkmp`%=nsrOuxh4? z;e<(2M#IDTJw&!unAdb>bL~>7h|X90>+<%Hj^c#a{K7qktw0Yq@7IkZ$!gv`%BNy| z38*bGRb8{1?#i%u8s&>qP7-rSI!OT$5c}1sv#>s{L=*2?P&TYCsD`&^+#$UmY@Ou4 zZ&jIu_jCO{oc)L8LP3S6ZH$_^Fn|!O0mmi#of(g9Ax2lR@k;cbc9MyNbIl&Cu_jq( z4#*tG)S)jFIhfdCQ%etV%?tSk^6AR~7#K`M3#Iqf?N?adUWK)(x zx4?6_JqBP%j8PT18PFgy8RyiV*>2=k)3a`u%w6BL5Nf;iZ=1t-QC!nb(bgVh zqFtS;iDc@T>Ivtmi@Li<=Bj+bzf09nu1izrsvP^0)Zpm83X)_R;oNohR%uNeLgQ^) z*tzIx+>Ce5eNBU*KA_g*$4at3QqRP_E5^~NH!0t+`J{_*$hWFKV2tus6d&8(innW{ zITQ7JdWrsY@1%t?iPobb@?=Ua7hJn^(PtM^C^0BQ)owoVHRS*Id1lwV4Mv-=XdZ>O zc-^S^XChfwN1u;m!j$$#2w!`?@PKXwYf=dyhtdnWu}p5TybqOBuCP$P=~hORJG)d@ z=D)}?Im4ksg340Kj!5wQAGn=AA#z<8T)V#6?!E{7nx~7@R@(VKpQoI8=h>YrEJWun z?-Q5b@7&pc)pxIJvR!B}iemZJKPTW(Ks$f($n!eiKfVUH`yTXHUeI&+qKhDvyL;Y% z{`%h=sTnjc_VfT~IqwtFl*!Pihjb{rdIsePbPA4$S@}5+=^?*u^%`<%h+Y;lVrRXd zr%z|5Od>qq{BsC9;EN~(pY`CYurBTG-X3Q6)7vG6UC)6Ns6qesO60;xaKYvg+MRuh zI`A(c0YD2en2zYrNh!(#+DNcQ_+))Z|Bu=0U(oEY3F`!WVFt)F=H%}(1#Uyk8Yk^- ztwwIGN3G2I7&LY6!^Q2+D3k%nW>$W{fXuT?LEAt7^xh1`RJM@BGSi64va^$A@g*L* zZ8hX!Ueqo8#wtm&edmDgepSbaXwLs2IU*R|^gi?F#1wfpwX7c|dHv66ArOK7RXh=J zHtgTu@7tDt%9FSV*jJ3dTEp8vrUg2I^b%Q>`aZ(1D{>zg3!Op9851`pCtBA$HIrM~ z8rr@ya0!wU_KSzRnB#O@ zVQ$j~%qxf~_Pg$%!hfICP(Z<|@OO17eMJiH{5fNo9DpNt3^XXdcc7VLGe&yl#3da2 z0a4nA{|m0o2~c#%piJVWKc{8K3mD@E7P0$i$I;lK-3M0_0wci?i&qT%pVRtq5fP*O zzRwY3=r8%M8+HiqMmbe)FGpJo(8sNZ$g=@<;M{$+*WZ7sU zzC${pazDnEhcojHHFPh4Yx~27L-GGEiEL?0VEnPEFRtqSRYoVOjdbJ6>f&wQEt z>R#C#N9#OfH*`9(rrjAVk5;BDYJ-C z>DzA7hU;WoK9w#+!B&VFWg z{o`nxqJLawBO?=+*jSulWB$8pkv@OH>4vB8T8O!` zhz3V_%vN0=E%TUJaS@L$_s5M7G-5N-XES~!L!0jo8OpAD`b;4+my z)qS_c`Qmu_+7Wb?hn3%q*51jCR({i1%6X7Rm&1u?^FBgI;lu;EjbI4q_ORCS+T&qkTr{Aw{=L_K0aQPr7?P1g{pL#~LtBt8s z(wmNMMBVBeU#_UP8+M>%0A!9&p1x3qgFU3*1dU%NV>vCFll|i?R&3k`gy_DQq;f`N z3Zy&0(>u8w-G0>TW%81K#y=338QVJoy(YEEY}ExulWrfqS{er1|`z-N+eq?aSh*u9=Fn zs$Akn;EVhHArP|wP731YZtVCV_fD_ad+cPYs3;Bwv<3&)=f-2fh0ocDT%zvZrnoI7 zE*e=Q6kU+A(U1`~jB(;ZrDW6UQYne2;8_~CC<}b|kAZ26#*|oQ&Du}(|HWD9tVri? zOFmV5|?xt!inRQrP%37KC-( zEV$u)ay&x3iq`$yMsDZo!exv1i-8xZaZO@nT8(=8;c2lk3FB)&jalU4tXo4I_b^nOaSs9ukSF$BJdhoQ6* z5qogdHOr8fu~*7*`qB*=5BM13M+|d*t@j7#v3sWjJQek8vLeEimA|=LO0m^cKCcU&Ftn@8fV%E;T>%204fP5^p4urjE6xOnY4D`%vpLrbwTSVID6Fxg1wg)Y{cS)ac~?;7riD?sQLa_87+;^#Jt|jK_^okheM)zxRgrQ?q{^?@RzAq77e;twTdy!c$8X+N*`?Y%Der`HP zT*kzJX)m2MZ9&UpxtwR_5%jCtvHo;@UX7HJMWUhQq9Jo9N+JgPEkJ>uu0@sKUXO=6 zLGm6Zj*& zs`H5=eSv*1iKKBk>v9rVI-;M6qmw%%lZz1+l~UC}TBD*o@d&f6v?4gJFg2rmbTUv( z;78>S4eW2e)nkAcQh!#0w#bvC?EgJ#QrV zb|Nw=F0))LOmEQP1r&EhaH%i#a zBH!R9#^sjUrQUtqa~TqDmdH48Co}-{vzWZMhkJpY!a?@%EblNFJ)Ge z3Wo0v>j#cGnrbx&M)SLRj}Z5GN-H8$D!6B1jP%}C(s;G7<}+2Iu|Q^q@6}tTs4vUj zD|=z)6c;z!$PF_acFDMEWUa|qxsEqk=O3FbUDf=IaC5l;D@*zskYWKRP~G6V%oaf@ zY970KNt!y>`(Dygd-9@_A2Mzz4W*x9mGM%3s9t94(vrjU_lktr(vjat%zS(QyjqUd z&1cx${UvX)lIVDgk@y6?Z0;%#dk&S@*k;t@$W;Yts$N!fMKO|z@)lhoIyYdBZXYW? zZF*D8f@B4UVdC^9e0oT=$DP0pK@yAgaE_djO0_~W!s*0B^CjsN;~c`b8j}L3tvYEj zqZ*G2CLCqN65Q6Q(HfDL&$$+$vII~tX#;04$odf?@!&3hXR4@Ee~Tn3 ze`NR|IT=GeCQB#Ny$)oG=usNaMfJk|m>7h8*4QFNxn?k|FnB$&^~=;ta@@ zDXOhWFAB5W9bRIC8nvG^pVb2UWXrsz84+J`lt};#PDYy^?!NYLMJk zX%*AUJn0-|T}n(g{?FvlGbi=Cjiotj)e_m_Po~{g0$0}QM^1#8Q>PG>&W}E=4HImi z(a9c@p(&OydPFvbeY5e!GQv62Yy2WfMYx#j^3AxoRqoZbirk?=uA_aJqqhp#sg2Ik zl=NL0ZdK9H$bT`jY6r}1Mhjn;Sd%MK548qP=rMn!c1?YUibJ88qQXDY9NU=sBGH~g>?p&1>yilfNVI-gmgpLK4`*&;GSREc1s2_q zgeIcca+~551!7MI4l0bV*Za(;qBx||j7A&AU4t+4U(#mlz}mon)V?q~W0^G%f{THh z!Ta%n&KVUfK}n=kr&D(oH_a-&%$R+tw?9HyVk0#wW5J;2^kjGQZGD_0gW|)7S57$B zxkQ0S?>W6ywcnlM*T$*x;HPI=ql|P@5r8Y7J=yYz(?isshYTBTA|@urZy-WJY>B8` zqd-8KDVL`8Ys#Ih>A}P}gHX>-iqv8LFLi3b{MFUeGf1%Fp_6`jqid!Kj*mp`cb3S@ zK2sHq<e97yZVdyrPB}fsc+~ym8{7l!PomGK1Cer=vFImwX2nZ1kj#p2 zL=e5;JJJH&A;S-H#2RL*@>0Q-#Kj4T`g-FHnIkq{7)=+J z7srW{bA+r@y$C8^GM5*te`IlUxQiQEStubJ18>OPn!oKr>Pg?zCfC`7kbK*k(sEDt zPIJUk2{vZ%8>D{EYD;q{YDRL#aSOoa+Z<=t;Il4uKMarIxrD-16@T@Dka?-S#-J}S zaq=51_v0h(V8^UIel9D|`*Ksjm|o?BWcqbO(lZ)<;;V9y;s~PKSkhds&b`FXn*|(0 zhwplM>mZQTToYN6(=bqMFLc1Rs4+xWO(Q~BYZp^C3r z$kcJI#-097p;dma)A^;C#x=EX?=o*K2dMiw`q$&9rP9kK)8cii*{7sjzeEu@Iy~li zN4QsvW3C;CPoBwUJ@IYMEww9zp)REhVu|-rdpua$dNVlcH>0C4hvE-UHYX${$e&xn z#j8bK;hf5zy?KR<;aRMg<3ZJL87mN@mci*}U0Yz@=r=J85q7mq3$<@f>wKB7GHD_6 zlFNQ1`%_}Z0+AW&Ip=>g*OLcRCd}k!#)T{RrL(3d0`d6uMF$Gr&*dkGZuDH)oU7PU zb`fJ$a1MNATrzfvx@zLde_E$k;tEk!bgkVlyK1T^@JsOKc@po7Hbt6HFg7QSiO)+Gm|6E8MkB6kNcPeCb+S54aw!}Tnyj{la3cEl6s=J=xoct%D6^+o zSs&{dWoA7iOR2Z9U{WYKx8W79uZY0uaZa9|b{ z51Au*2`*f^RFJr2?4}Onf@Rx|e;-)~ySkH; zWg$1jN(Z=oyc3DI68=#?yU_#fty1aZ=Vu(7OY}C~$|rG!k~zsne$S16&w~0&@73w_ zz3AL1Io(QAe0o%z0T43Y>e&7HN7!P;BoxVB#^q!Wkb+0-H9%naxo!;k~5CU z3~E*JPPeppwKO+9%&nEz>D*D{Yte??ej0>}mRzm#>Xkg)vH$w0M`>vZRtj9|;(L@b z->#CJal7E@`%LHwRGyzsX7>ZY>>odHJ`q#*aZ73PMkUP%^k#R_7ITaIb2rOQkS?aqGd zXyfrAu{1=vijJ&CGbGHkV~Y!$T2 z@|v_h$(hXsZ_Y`Z@(rZo?xTKJYRtKq_G-Vl1qA-;7 zkT1(iy;LOU^C&J9dltf|^FW_!ndIw2MXOltsfx}?h`RYHeG5O>Mw0E!{gi{B&WwQ`&aj#6C(r zm!|Ve4P`cdLOfNy(2^(kQ^@V;fr6)eTJMTX!t9>l|HinAC;|VGysPDV$!W@m&5Rol z&a;1>R0x^#TWt|Z@er@C@bL4wAXuMj6V$w3(@@SdZWGwb7sdv;%`+!%PQeg1EHx3% zIXfp2GdE$HMJIT-tKF4;!7KH5b8F(yHjwaZJ85+;*Y!T?HZZZ3MSoy=F$Y7vmol-wm_81|DgSM>F@7>C zCN8(5I_j#t;=2KBeqEe$6|a=5&p~nsUqW4IiWm9rC!BlgZUI1bm|VIoZ(-AMJ{A?t zYR6*I3$<*@RZy_#9CUqVHYF-M;K7|6HffFbJ4qMNjT4RDK?wF`?Dt$TY-o^#y_WFR zNS4wrXrJS&`|#OfWzr=|seI{v>OG>z96VD{LzE`5(Ng>pZk6LiZMmg^lGT+0>*3c@ov#4?KN?I#it$nQc zjkYlJ@xMB8D?;cqLVkwH5cQ5MGd(k^IrzyYyyk|QI?9I~xet@jTP>cLX)aaIz;RYj z!q1UhFT%DN#h|N8rJ9KQe)U1K$;WSE)na2fBQ3AZ*^wN%(fdq+6rLNL{nOG1$s6FS z_yrCSvT9c|{a((xg@pyLes0nFEk$73dP=)DaVvnqchZN4YGYlo5;j%s% znUka#s0fhGRI3e}xOHNziAMO@nnJQb)K7_Wh@o-tGwUDr}Jb?ulHNkwnKz;{crFGkT0sQ1QCQX#@aJ!SFFG-o>y>( zh+LAWz0Q13<{R^*C;kgp^dz5N+jD~PN(M4vMVJBZIh|^L&>hm8`R;gcL^=33f344}0$! z*VMMH4{xzo6i`$^MCk~T-ieCz-a9D0g$gK`gO{)#0Cvl&?Y0nZIH|`|}09CIdQAepcgq=Zi0hU^OD$Pc191 z4efEX^n4s=(Ycl;&#WsEzjX3P#1_>_m~f6NS#I(N%&@lcyIZv;|s6R(`j{DAF7 z!?g@BT98vcQ?mwl(4{UvNmo!h;dtHDW!Y8h(_y2bhZ66iy<056SbV4o~=};~hhU4}{{M90kK%79%@E zjxx^k;h>r?pg@4p^M|JeP@K<~-{F;cLf_s$|NCD9c$on5q{XyA**_VQ0ZeDU!m7hB zO}Q+6_+W{rrjN~|th(q}PA=ol%EoxYqb_9n&L;?+PWLn|iVT=>cJ&5ZE5-0Brt!30 z^>gYw!j)r7FlBl6_NY1A{e}RC?iTlw?ZQoJ5d{NH3Z65fwF^HXH_5i1`_6f}4q&}2 zc|ekB8+UHjy!0ei?&r`bRQ!?MO44Pn(LmFW-rpT8*pbNe^ckH5wTawglvYl!{qp@) z(2r6@-H*}>vJaYepRh1mN>#OG^Z}#7%jkeFe)7vu)!pzqetz5`aXdIWPFtL8CW5}>P_CY|A zp+rOHlSVFKZVp_0qhR7`vp06Om{nu~$imAyEH>E324#u&y0y^jn>NnAM<9lMTor^1 ztx#IGjNq1!^Qsi_b9QbfX*?U7RHxgZZKKKDFeJpu+3s0>Zmm-ab3A#%8JvAw0TC^# zaPk3e{6j=7tDzR@qXcT4z3hXH9w`ukn7292z@O_)gNA;v*t{?%!$NqO6u1#%H^*YW zG!x|BWzGlyz~g~=2n=s)o9@7 z4n7F-#2;%_j(#OMc}qI;sWZMg^?S9;RFv@RRyjlrnc?aHK63r0m}*q*yu2IwY-y3l zWX8S?3dAW4t`jHu-P1H&SG{pq>6xEeC5*P&wPnln2d8AS;b(0wkjArbLNmE-6`q>SLMb1wzX2#!A}Y!NQTLq811aMa z=cJGrHdM|CKh6Fbf!*YaM9!ec?M=A2%8B zgW>lXi3Ogk%}HwZvTxBUIDw_%;`AwhGkShs^1Cp1*r#gRUkI{+aHgw9ortBTE_Q$fV7~ z6%nni#kFD2DmuuBC#&~lDZr0-%5FkCU2-HhThBMc{oT$w9iL{PGhx`JC#!zv{3*{O zUi%}?`2^Zky=oPzvUjpZOy)-s^*r?x#qtcr-o)0xpXqY;?GcW zz|+g@{Z}fUeU>23r1~%Ab zvk2LQ1x>4-cvdWbEfu}zElv^EyQfvpk~yz=S+M80#eEm?#$&%O-&IR2qSs(_#2YL%exUD!YPNq}1w8)1gD`+mUaf!*eq}G@A1xc z!{3PJSDPVg_`Y;m*%OBc=d}5zGA!iWm->C6(!_pMgwrp3{ougz*JB;1^qqQggK^>Y z%Hp&)de0@*Y>1Euc8fdWqIZX*>O=~@<@K^mUu3vCOmub)CBMabr=?AD7(#tYaBs#$ zYrkOCb+ph<%Ijlq<6=oPJ1jH8D03(%X2~;f(${w^3SZ1huQ;&80w7XPZl7u7g>ky!XxsX1uh+-+Rfz@$pPG44BEA}mw&GWL+9)!T!-(;L=69C z0E3;$ow{pWSA%8p9Fn~1>WbkgKVx`mjFf0VNE*=PPXzW&a> z{@$g(O?@`~pmw1YX*ytx&oJ!&VdR6?V^%yp--v#?-0xaHTN_#x+(K09c$68gxYH&# zX9ka^%Pjvu(^sp+(@t3{OTJd$@r)_%I2Csoc)ZeTLm4ed+eweFd_|cv+L_sp$>F*f zD|JVvSxO+IM?@szd4ES@{dKEr+r2(If z`8i?NZn3wYDI@k?>YewGlj8Rn56v@_rA7O<_FHFa7bgE0bl*~m>SOm|q?CU&L&{Ls zZsBsml|Oc$5j8FSV1Zp3^-dm-qyrLr|nG{V!&L5VE#;If~>~MJgtsKO5vXJ2oRDghUm@J?d9wZ zfTGU@>8S{2(#oVO;n=A1QoFol3U!4QyzppPINH9Z{BTH^RSrm|>nSBR5SA2j7YgVV zeF)|fYNFHJ$xh{o7D=HQ2Vf^)Q|{s!@bje#BlN^*ciwBTKUlM$zg`G`uBR~r)dJ~3 z>fm`J^_3b8j;3^}0XP0of$(JL%rhN2TD5mb&dwg~rwlg*m&}=dyU&5Xm{gU%ABTxW z9k89o*mhQBks$}htyiU=_<69BvUOwS>xXoTrQZ)eM9p_RGJZY7dYqov15ep2+6T-2TI+<|$fg%zfD+PyM9JWSlo>!ALrw8g`N)4dHf_e`Za(4aRh)}e!$ zv?7Mwml5vnsjE4|q>qB0Me^ozxlEVcvr$b#ht`?&g7s&E)(vle>i5z-Rldt=dGXEM z0z37)qg?A&Kih#e)y86=yB*)eSx%}NHp2$Lhaat4_D4fguoXy!c-h9qByHOvDDCi2 zm+=^OPg}^s(k;}Fz5M1D^LP^9f;#{yrXgVt*Dr#5!&|Zx@2oO4I`CTe0tq7=5CIQW z!}mKF)*+XQz6ND0EYxmB1$TDFD-(X`Hwc4FZ7pN6?;s~(r4N;@8xo?rrZ;F|4ib$rzNr3PoQ`;kTVKA`EO=D$ zW4f3eoo6i#`R1ZAOO>kNy5P+pXKI#9$kKGTMS)>AZHpi5@-p`wZ)m~oH{vzI5Hxpjs#}QOj+r8e)&}!v7 z)6n89!Elz5X@yy2hpGQB*gQ1vAG}3mlfu9?B(r0x;T*r!!@qTuf7$jl&!1~w#quZ! zlO_bWg&sDR9;Mb-w-pG(s`6B~x@)37u8(Lh zm8@)>gT78)^_RzG#YNO$EFq2iA~BXNGl`R9>aCbUdTCBDSQK zRM|q;VD{4u2wymzRet$dgsDvwFUlA`EBHx+t^uo?V9=OwduA~#eNMv4L-V}Ix3fz( zDi>}PFjlq9mYS6A2zNU434iK8nB5 z+Yb-3OU8ZWBpTCmYk4?^cLva|F_rE7*7Q7@YBYIq+BtiLmIlga1^m0Qcw;d^VHar47)Dr{-T&Qz|me-a~X zbp)VR`j|?^Fkrp?C))-N6=H`wCov%Tml`bOdX?ZeI zpp&b195FHoiHR%#+QIynhK?{gnau?X?G2&e?7xGv||4u^J6; zC53lS_1k8A3)A~)Q5}3Ph4nqZvPW#wSCM9nlU)VW4Hzyb*t9UoQT>#7?%JMMym5eg;rmN@{zt20&JG>im z6xMq}ixc!zlzKrZef}9)HAx7vs7f1^w{Kd)&oXn;r(n*$tG^Ej^3td7Sr-S|=1=4m zZ?``Px=wtqAsNpK5Lsy1O<&a0*6kUADA9VlXmq=CjPuu!_1mbKx%k0l*VBQAdgdSD zuE)pfORO0rZC>Z&4lg<`9byewdpDxK>)*(O75TY8cUEw2-!$C~dFE$yC`ty?so>iE zGg}5ZSt^j76^r6eXBN8~kBK_Ph)P$E6>0F_4Vy1Lk(^r8&s1y zwy;uX=q0-JVNb5PgB`7M-f+6iTk(qC;anC63CkMcLB{*ffaH&oC_PV`NdlC`pnRZ6 zYvol>tXI)TvwjEw`%@X1?VO8+>RE=8P0dv-CZgyXo-$KIf)7*3d|KRkYR8<9^4cAn zjr3tWko%EeotcZQvLLll{=(>M($IMd(?&!+N#dSdYQ&apfzl;$^PaO&L7V7uUQv6I z4(^onN43my<6#L-=Xlb6y1s@tJvy?8Lx@_AzP?swQ#bFFNCmfjmrQ z@gD*p$J3Oebgky54MB90&E5LwyR>A@1pyGcmU_Um@@$;~TCA-R7TU8h=RQz;;*ImTc6{P)K_5tGMji-K`^ox&0eHuWTr3~*tN-f zVSJ*xh)?Af4&BPjSgBm`?yU>7>4HQf?R(ZLrDX*ij4RWdbVjJs$_+=}*BG8*I`ET} zBn=}m@A_53NHx=`uabDVZ?zFoi9YSeO2U%tpm$0Z6~+f)2<_!Ds_)RlBVhZ0W;rc1=_v>p6O#`o&xe#0z*uFSyX#R27JmSL*)OCMo@o6 zp|-#A39Rt?@gekGm}n2z=V-#5j_!y`G&DYZm9b4Ce|jy$zw2u`Yn_S1i}YEG`b>c0 z(l?d(R7R#{HaD?0%T{H@Q%8`L!VBDO}nf>PqAtq6NG-fBF zBdd%hP=bd}TWAHdU$AnY!`HKfo0WDYO4(0=XXl0kZ1wZ?yEgT?_6xl0Pd{KQjTNSA z9mYo38We+DuhEq^q+1Ymt2&dp8Jvh@%@`D|1Q@?ENsP)sBrNg#>f|nP@Ow#kuBGy&z0I>QGMUmeF8~@f$ zZn<3yB!Zl3ko)0q_gKYXF-OHXGO(GtA8(3|z>SIG4qazXEoFu<)x^a}iT20bo=Cg`3+ogll z;t;~Oa0{P4_}0_(X^xywQK$?(k@A9>4-T-ioKo~9429O-qciM(92d)&iR#<3s@aS0 zK%GBg7@k-GB~{~Z5gnFX4dWlDcdopS&u6%DU%hF3-@eY6zwQ*CuxsoVjxf{Kf!a^P zYT(8<^2D_Yt1R5d3Hm?6fwDN4IIA#L`U5@k6tt1$*xG2J9+U@D?P{S`Xn?^arlOtJ z?^eRYboDF^-NZ(T9rg=o;eMqk@jSl$IEpdjxtEU$BwJJcMyta6{Ze-d%dRta1|_+U z_^<_4y-44q*lNHxkQKL3#O5?llN0~9F8;GNdGPSuyPsk^DJFKMGKivx5>|~`Lxa5h zi*{7fBF~&}wGbdC@=-%q8?Ih~1V}pqfXC?L6?VXntyiFk&5&nTh#5q7q^Aoz+WJCV zh7Kg#i_gY<|5h<5QG1vzA?cQUo?^?G>V8k14ZKi0yI%bsA}Z}eOxx@3&oR61*lx3XYjm|iW+Yg&bu{jI6rMY^uirvP|B zR5kr}5?Ao7_I;JV2-JTUcl-+e{U%WV#?==saM5%+2J}Xk%{R8i&myo--@rgvC8}$J zF!q`^#Y8;nq|vATq&nAOH9vIUST^!io7zscq?7HH%;-mH5fTv`m8Q&;UlNBhW4T0G zwhA(slhO3)!HJ1?XB7*|dLQ2@KbQQSo%XQxC)oVGH0EJ_AT1``o&n>sQ`B74@@|_+ z$wAnd@tlY{v{;hid+G48WbVA9>u9+ix2fseRh!OveME`&=5bq(GsAf) z!ka;3U3fYGk?UtgtvVMI?@nS}YPhY{>+eEQ@Fk^C(oNA{&$4jWP2^b@A+Y6p;}&5m(*egR{=mf z(FQ6@Wur!%L|bigt=|SB$83H0N|!JWQiW@OAF9U@5Zn>-z@eW)InXIE7Bz)NAeVFYtqW zv|BX@W!-us1eUvHb6U4^iYmN{h)=0g1}cpk;I#Mj#WIIDmLhV~PaCwc-Y$4xYWmJj zF8%4Zll5INg-U1=debJm&<9ac>R}cAv+dLPI3av zqE#EL3UOt+5;2w))LEjLT=U&Yv?5nPUq9ypiC>PhF~no$;~8TcH9xu9_o||~Sm#)r zcbpA%=F>Zn#_opzM-jbEIf2s2Wl&v4wUH7u3J(}3&)CJdd%?@IgwlP!?QCLzstTmi zqeVH#5`}azAXuVQc@SLc`E{wPdnCm8yp4)yhf@(yrR*3&bN~SrZLqz7sj#S7SfUv{ z#0!A?S7c=vc%Mdz>6kuy+Ii=Q)8LCL4z!9EK7V*ZED*ZmECOgsu)v;N*+FYVJ5Uxs*vOjO?CCf zfAj+IRJ{K;oQX3uDbE#h^MKfpIoTC4lV6ZuXk-ZxVxezgQQxa%2U;hbLUanr9|f}> zP~Z`5;$aSrRJLMAt;xsOof_s;n3w83UGp%tE9w>1TQ73> zt-HT?>kfkGF|=ELu$HSe$bnj$9Q}XierVhJ1Hdc+lwd zT`CnP_dY$HY~T~-$0|?IRL8R`G?|#`n4oL3t;x{Y>yr*TRCUcdAxk~6+h*)+o9;D@ zl$3mzc)gB6i6AKBW18aU+9~2%6Zc@k>BRJ-2pKz#hc917Y6r&p;RKR2c0BCXC$2AB zMeJ@eeG+bexUsXmNa~~G|48H;^-)0uK3C(kk3*@kN)Cdsw!`cznpZ@(g+_X`ToN7y z4~?9VW6KEy(56y|66Dc(NKNN8Y4N{xAW%)I=t{F&3HON?YH<6{P{Us#L{pyjiZ;<{aQ}EgJmMOlu6yJ zK7GD_x~1Mk3g%+{MS@Fe;9SgGv$Llp2kje0fwI*z%p zt0kGv?vJKG@+0!EL#_Ndt`mh79$bpMpJe}F%j#Z)t=(P#vuhd%iInL3w4QoR_k*pD z=l(UH$yx)|MT-u6+LLLu&Dxlj;$dbkGWUJTB_481#l=VUcshbYDyYkQ#ZV#Zdg zu2cxHIYBGq1Fq4{iQj)TtSPY`xbrng53Y5BLXQKmH~mn5iLXN~P+A4`g6#TaI4+Xv zXS6Cvcz5lQvj~Li;vRfoz7^$4OijyV5Lao|7gw{pe$KB72xJe3GjzTty1eU@>F{(q zbjcAsi>P^w1pN)K^PKjR@mkM1pd>2y?iE;(a=&j;3G)a8*IQEt4V|5jg*i(uVKpW- zI|~_ZRoR8JdA0XE@@E!Bsk}d86(!}UXjG%qtODYkl{M)|rNp)E^8wT6?`1bM?9TyJ zsx6VlJ1N&s=6Z0b;zIVxQ9yMKv+lTKl(hxltW=(gZQjI8W3Cc*wD2tBipadJ0zzCC zuV%(hdZpH5x!{Iq93C+$;Ezs^F{E|I(ii0HS89XGbt|JipVi1R2FJt=>FT0kavi#! z9yvX2&S7m;7wA1dy~xyhG}Fv4{Zvv9Uy0Hl*}ww>^3r9185$*26g&qU^J_RC@Xa5;oA3o*f4&RAcbF(Z0r|E(Z&o`_HnGyKAb* z-Qi;KgH?6??x<=5zDWaFQMUFObe?C&@g?gn)$}M@rDTs@p-_sfjab@i@v36a#_4ce zU%zXHX%&Wa)~h2Y$+2t=y}K*Qqv_1@l?sfNoY^C<)?-q_r&EBgNYl|=O3@@W$>G|^ z0Q-jOlD}E?uRjP`zKcAMUpMOOgI0Aeg`@^F%*RPhqW%oEk>BKEgIaS}6E)OE;p^;y|S!S8%O;ir!lds`=coa|JO;2QOK?&b@fn6UWFH@4o7qa}Kwn zZl8RbqqHG#9ej*??!B;(Lh|CbM$e5{px$EjQ{^mvDJ!s}z9ZN!M2EPfxulsuL|UCq z)xr)+R3)MA5oEKv~>VoBc)`yJVVAk8@e*6b@&}VEG{Q@`Rl8 zj}bvG!#w-Nn;8V(PVFsWCCwy>w+(Q`jO9o))!S^%s~Sb%1-jZp*XMHM3YmErBLGsp z9T!1=KHm)0=rHy9_0`ByjwKd5QiCX7O2ylnSjn9PN)WQB^p{bWX37hA4)Y(r;5g$G zd0tyv`x#5{@nj20nTa<@$|V$+pBdhm|}8H$7OSJR@&h`_@_PS*$GaRWxo+21Cxb-}gM=;px zyk%TT^_EzH{4tIGhYK2A3a&ELNeV{71g zn3~}e&wPIze4&|0_p$1FmP2EdtzETp%go4^9QbMEv*`jEI_lUgJWV$lGgm7M6P@x( zg}K+S-EfXb8A~}m8?*#|-_H4|Ux`Acx-KrnB-;Pb$6nAHY0MR0QDmo|g1quW=y(7E zfzYGw4zp<|u3ljwXxyN`d4H(h+8Ah;Bk22<{`)Pz)lXZ-k_ELCUQ*Vdmu)Qq%{RFP zGLmy-8ge6gd=CnB4`OLDU%9;2y0#r};zOq5NXzfp*oZNE;84HpgnFk9u^>xiuCoup z5T!t@PI;KwEau-*A0f_BjxpB!wfg1FFTNF0ha}ISoB$5{% z50q0mIN06n@9=pzpQsWV{|?>YkfiPAQ53n`TF~NNdc$IscfLrI1~>jzFwE>aHf}$4 zL@>--{&g>@T@-p}*YpJg$Lm0+XeB~*L1;I{45Z1io_*BzV>Lu! zjJ5s!+qhQW_WXM7*3md2p0frP9*1vuLYtmtM0?u>ovFFLkoB@Q;srgTay*A zA*ZyLyv_hMwL@Yf2XxLY9+S~7{F)?}0k;fg{p9*n*j2~_3AGUU+ehr47{k{xwe6tk zd`=PH&S5B4_Nr@PY*d?Fo`B+KjN93Z+IV5(o2GYA^e{7!!paVtPL?R;k))Y8`o=Lk zOhdLaET1opWrZa^g}zl{riiDlZK9YIpHhkx{xJ~|WuYyfwHua>CXGn$In(5b1);6rxaw=Q$&5(WSu0!igb=fd@q1EUi!&%T&i*&-J3KO{G3bzG5B5uy2Y4njg%^UASh{0@U_Q(g{;5jr3+Pr z^Y)jnr+RF}uZIb@GpG<0(D)U~gg*bT;f_THGb}fGjH@&5YU-|gwu6nFI?t~qg_tEV zUj{Q*vHBZFY4|VD%8h+JhrJo@9C<@KUT@fh?ov+3tZrS%Y@!HLx`EGCvDn23#q6-C zZ`&aOZ3sH_-Piewr|`G7>)pR2N)E8XPaX1SXm)qz;=2NB_Al~x-9cEuWROW3N6oZs zRrVpJ2PYTIGYZ@Q3t;srmq)B(VP|37x$CZjk?!7vFssUpzLc7s{5d!8fgiX>m9#Og z))#TJR$?P-<*ooC+7ta*|0K(8fA(y%4a`(T`=yO1#RCu?k?8K5X5t0&??{!6G}-2q9c2 zw>V*wtB>Z1Y80EjlV0J@LiC}Z%Sx6?Qtfei{)~v9>&h?Lvyh32#WQ}73g3oUT|8|$ zp2J+HIE_A4^h`TqFL(tgeJNL8@Xq`){K~_kqd;};sMf#^sA&xM;H}(W8YzHlm`efS z8#MCN5c15j6b zLT*9EO@*GLXhwP#7dnxq$Y5FSYhX&-dp$EPwxd9;Q*^SpWxU}4|4ah%;v@ej`4{O{ zgY!O0U*0(YCzdA7&F8LvMRq(+k$0KN5u-AZk*tn6AxEHfw)`A(?uL2n(4MdL+~{4_Rj(j)uxEW1}5pz4vN#V(EW?mhMS0>Gb^?rQ} zP*H%QJ&u`<&$u5EVrrqP1F{UvBwt46)aX+@vi8qB6x@CYz4d9b-8j9H3$7ktoT#kK zx}xBVN!s)E@$n(sqzts%C{lc1=;#MpbWCu%Ckx$o)uQ5sTAF~qw(PygC!SR}RhQo8 zDZIR5^WAMwYVvwGA*OYinsaaq7(7`-sLCXcMGg6sYYU`s;ie&_h^kIhv}MXf1-SYS zk~hU|#d#Rx7+axu@_{=luM2b3r$9k+ppT?D$(2bN8_dn-5o18X9@mRg9r9A;Tu){wrn$Mpa|MuUyWyDL&f*{Z0)*bmxe%rHqxyHX2}q+V0I;c=eKK8GC<_^$7faaA--^2cS-*CvxGpA$cw zPpncV?G72+OqWl%b)pVezeKAM9K4u44`$qyk0Z!9(ks@<1n8Q11HAK#MmZIQ#og7i zM(F1>@ame2-u^*JLf^fgTctmtIxFz`(7ERD?i6gplo3G?v8O7)YX00Gzv&Cr_Vh*Y zjkOo1ZmkM;Z#DmOu>4=NS(m-jN1!P?9Cc1slv==ZJe5t>o2VU30)^yWsn4eE)E4prMHz^zeJR5xlmg=WEA5Dx)uVP0VbYbA%TTGB z7xaD-_j*z#i=wPOBn_AMT5RkqT1_{=Hfi~Q^a z@vXsqsf*_!^kRaD?Ve+8ttBJKi(;^j!{MmZRQv4V#g`-HIa8adsW-?t?}uf}5GViQ zDpKj#m_X*!Gs@=zTHC`uhec288ZaMq*AGp9FaxIO+pgdHG3Eb*xy`kn^5dgd$N!{@y$KQR|%|;_jYEhC0F3d>)|lc zQe%C8>)^|#h?8Q$EUq<7D$ih`%uVBx{4KCPbjDZXRhGNJ(wMD#TE{C^b?60QAtRNU z-JCk9nRnKLEA~+9ejnr1qJ?7!^5b?wm4npGWuUt#xq5ds_4J5XjVmAap2N{_f&f=q z+gJ2-X-Q$dYLvg5D)8%Ig1`FkWR`+MTaBBJ<=WavY2v~N&v%j6gl(LCMbJ7*7Xptm z4juM6e~n&P=Y~+e(Qe8f!4oy=;m=!dUA(2AlTV)yArNWslf+d?J>dclJC8KQ=d9}P ziX{^FziCKXDaRRer?DX!4`Iq9=2R5akH7PW!|wV)d%LME1>+t*KbnET+W26`y=;xV z@XObKXJ`y-*#lCk4kN)7XrWUF)ot)y8_?;MDnlY2jJLEDdO=Cva+pyGoVHLCjbqwp zHj>JxI>AluISJi*@t|Y`WJGX8JT+m=*-n=~VQifvmi($^8~czH@%hI1I2U4J#iVZX zN)g$z(||V%kM-^=tXT-=HmY2{S^qxs8Yzb*YYo)#bd)X$CcbPm6|#W_A-3$bSoV;B z(g;_Z)Gs<6V0TAdO-8?NW@FH@s`8h?QW&Mfd9?riz31aV|AUSk4cOnlUmth)+sE8o}t!W!1veb%^x zJ1u7z@2tzuFt&HTmBSxBi0-Jo&NW=tVbAKmi!jH{k!hRWGky8?%tIG+y}Io#LJXfw z@085H>U&5c`hJ~i_maROz1R_3*LkmHvWd(SvSiX55N&OsG^FVle}<%rcdwha&-=NP z!at9EzduC;$8q(9Rh_DK|!7j3#uSYjN?Dnq9Oa z6A8GV#t*NrG<3h2qFkp>VcBrxvP+BWra)-zRU$8=j4NA`W#K*s-Ih+y%t;#xp?bn& z(!o#LkFAur^&dm;n8{B;=^WzA=V(KAhZw!48z_RG|`R>LOT5B-p&@P97*cLP2C z^spGt%NFf9Zx)2FflT3!M!03+ZwZo;CLgHaMU$B?8@p~4_A-_~O*C76H@};JY{_ppsNrZ*lJdPxbc~KEnp?Qc=hC!66^ql-PrfNEsfT(xeGE;Qbjf z(uE1q5T~aVD}1M(Sm4YqhxCZzK9bRpP~fD)163$^Gx&;127iD8v&&%W z7;C>hckIs(uYB)VS~~R-T>s+#E`fiZ;tM*r$LSv#i|7}%zC{chq~K<^4#oAmxI2Fe*Hb_@khaZr`@Dc(s zm?hx1TFG7??ClxP9*KNkC#Smd{t3aVffSU0`0JuIB`ZjJB?BcXhauR4UzT$~r-Ul$ zB)xP+c0FH_^^$Et_M5fksBdluSCrBi3dmtfPwt$<8lar-@N_2razryOuZoIsI7Q3+ zyA!OMZd1ryk&2yBczG*yEF#_>!W((fV^^K9!dh2?i|3mRcFo=@rTWF2_eK{{ZkZE6u}TC{{Xv()8ntE(x-_W< zI~*$Gaw7a)=5>EUerjq>ddX)NC0&L{e^=Cpg>M`y6C3iA!zvB}RPW3w!m~`Y9OWvd zf7Vs4s=y)nW{~vju$O;ou)jVw7|^OdhMXf|-|sPdUG2w@#1hy3k22kt9y^GAX=-KZ z)U6k?d4E7^S~@<7jM-U~c{bU!Gy&!T1{q##q}J^3Sj%yRx8>KMz!-WH_dMtBW36PX zjf{`gAM>rVS715S)plt#X08)HxNg&-!DzL*P>_RelK+e#W1a!-AqIhfMQh}HA0o$AzhmmRzG}(2#pUtuHy$$hmDaLxmhO|jP+Tb?_ zHP80DgN~iwFqbEc>Kknktu0h>m6}Zpq5L{q6GldZNzskM@hfb)({;TBpOvzX#lW$H z_4pzQW)ci)a9BS=vlHxTE+pVN` z^0T>>XMuY0vc6q3V+l|Eb-r@M zUOW+fsYdXY*R!sTHe?Jb6ur4+>IL-~@8i132AbCKWkDP_2Qz}X!9E-3Ag;A4xETTc!WT#5yU z`oBYyTC4>z|9Gi?;mH5%pR^R-dyiWQ&#q2^agq%sv9P(91#mX)X~L`&xGP9wpFP9F zVJ5-6nww@Sm4-`B?~$vYleiRmzG&)Ia)_@gd|bf0d=ksC!v4tkhJD2j9NjRg_TIB^ zzt)F;^aTE%muWIJ3dL^Zt$HcV>v5*T%14)7E-nXrZRSkWI>^{V=i*|e}jwez0M zoTH`^hMg$KQx~1*SHIQxe0NX0Xi7s=6L4TuNZJ*kYT#Wl%2404BZsbmK?aRGY8^`* zT!0Go)jK%wgBVyvPS?4qaOMrF{=2*Xw}N`7NMYx_UFKIj$}K&r)uWk`C>?^EE%Lw2 z6-&}Hp9@y3SSQ=F7*9M)D?$kd(f9V&WKVvbsZ4EA$}xR4-A#u!aobQI;b(9#l1o8| zY1#Io(e~n6krRUhaqE=hSLD)G$s0XTmGemMxbSbVsYUPkw8^S?y?Je2#ds5wx@Bm= z&?RJ+E1#7reE1ZB4u`uuTK{RRxw79lZFlG~{DmMy-+LjJyj!uE@nuhGYoFG*dgpWG zJ^%L7;XLy<{w6}Q2bUY4&gjG&>*yj={#v#D>s@dpiTlGRgdFwVC-HY0*lmK~_fw&r z8uGN5@eP?sWB-<2tx2Fbx)xB+HZ{19qEIyqz{WMNLDT=gVqUx>)<=Q{d%{ z=mXyJ>n@HLjFx>@z4Q2S5^RiNJ6u+Svwb6j78SlGK%9`au%tx5mLClGb1eb&Z+{-{(P|Jo0g^E=kWu*!}2 zqU{y?T=;F2M0)<#+SW9|x2SfRt6(_y4iK;X#~M{xB|x@%6$ps=)2`SEuB#9_M02I( zU`UQ(tqTOaN!7gnY|Z)&=&b@^5N0K(pv6L!04I+F{uynM#29h}T>II4T_5H4e5 z<0nH}*QuI+juRxeK^z(ku~)Q+`0Bc6B0cP?_cYtuV->|j54Y#~o=AX@HS-mGThxLw zii}+!iqxdm;G3u{wOP+bZ}yEZ?+P2YFr8JDnY}I!F=zjShaUY!V6s@gX5LS+$s0N&J@k5l?cyQnspzWin}H1 z2W>6@9~_7qa_auu=Jj_9W|~5H#J~DC%l9(gRPJ&p_^zyr?rraFZ4ZAcd++G@i`*yZ}$bd=Xh}b_A5D3ek5`$z7k6 zs*#pIc_J)rvf769vAaUBideVI@*nU1Bgf(aIKZK(!EZM(k7r+>P^;8~-T#WeyTnm{ z=jv3i`?c9|EcK;_6`xFz??zr37D(yu+M1@}Np=E|G10Bziq1xq$HHQKR?p^ZKg@#0 z3jOYTi%burE7MBCg{bNImz8z$f8+RHzU}h7X}();?~a+{D{4yLXP5;^Nq(pNVK2tM z{?{5;Twcs%7ptZ|q-)E^`)nTgMOmx~i^|knIO?W0CQ0b#c|tVW!$cSVOBaXeuo&3)u*gWiyy3`leO4mYwcS5P7J>$o)#Z{@HtHF{r3Md zL;YXtGuxNXT`cG>2i<@5eCBd-mi+?2!(?1(SRe47YZ_8G(ki=tj@ny^pBu@MC{533wUOSw+>^MF>O2K0pUjW(JGYBhM?6_V^?4928@gtsZVY=R5z zf%M(SHndS!xd*oPs&kq~hx5XTafT}sUY8*E{(UIycQ3_)yN`E-I?;Lh-12$leY87N z7gk(GHL3k};VSu6Gw${Eyu13IJ<3_Vfp_)>3&1sNi;%I(l^dV*)B7r_MP4l$C0$H@ z^i_}=$16z~{Icfk$G8x1srXp1Y2rpsr|#)_u~pr=MuV@M54BXIsA z7S$Ny1uKT=HK_uWTX)j>ExKO2wdzLQu>|3r53L(Il9K1GTMcNxT<`7tFct0g?AgOh zl09^1B9DLK@Ry#5L~)~TKHBL>ZCnK7B-S(S~3L4E~c=hKEr73 zu90z!gca{P@{qfYil02p?v=QsJJ1XZ$Jh98(A!&M)sk`P$L? z+>or_#HxTT!3HLDb=pfvvSvTu6?o{})o2A7uO*i3t;fN7VR|s8zxSH&OP=5yv`F&$ z=hXUtY=HmZe#*?iO5b`2=fD2XosnOkTbUZTiJeMl$P5!%*KOO4-zMWt9*smY{ zf09W6!md

    ?mpSAGsj`8rQaoNMsg2woVeezOg&atr8;eE8?e2cXSZk^B^(**-vzoC=@&N2g_S%6bu=C5-6&p!k# z+yQQ4f-i^t`oMqVCjVW@pS+R(zMVgx!T-wSzcTr=1pa*GPy3{q`VG^iqIA<=@PG#g zG(^$GvHw8JUuOh#_?oLQ*DwF8Pc~rG)tXL@pXTT9?@TGA1dH|JeJ=xTv=8 z?<=Ar2#BB{Z6MMh-6&U)Mo^GOM!LHhM+qfWq+6uBy95U5l8ym}?jDAk=Zpn)aP2LooDO zoqJfF=I(sv-8T8A;J&c_9wrPc{^qm*$+JT zFRQVbY zUf_JDuj=vn5Cz_=Wo9GuH9+5z#ijZNN5b@Q^oHC~k0CHMcGln$;uicOgkz`6CVT=s zdhfCPhRi_rG3j^9*FCG6YVEqStQnX7UB}3C5sOJ zcU-#wq%!Pb$JPly^Vft}KjQ8t1ZjpH?XW=hXwk3Dv($UhB)aCy*4#Kvt{zIINWPP^ z;-%b%&jxMCvo@mneR)mD0>p}_qPXCT4CRVkDO+vKAa;X=ZkDaI3=3oTF7LP&8MK*< zweGl#b>uO;+XD-HVqMjJqUV=902{%R;P@v``nsF>RI{G@bSn?deUJ{lX>Zi;!!?LF zBLJ{om_@~ZS@tA`FKhj}o)Bm`w@l^4KN*;xER$6c!_SO_!C?I-D)sB1WPxn5koeet zA>bQr03{fHAQ}_O-w>~VT~X9-U9A_oN7pk!Mg=QRg_>MT`UZV7>}%B;rL0P(x~4mab^MMukT=a(&A(Z*h< zgelmw`X8*9Wkyez9|*W@_(s;X4xlqy1TB19k#HcXPi$-JrW7y7%elOoMAF%T6rv17 zSU<)F+5+fcsn&L`E_NgH>X$C`dJfMjl$RSVlkZN3z4`-E4Vd~<0T%zMa3!K8fmN20 zs8`+Gx0yn&t44H7?agtE`enhXj%cOF5%SN@()WmHLI+>S&RmJgi0)n!q8#jU>Uv%T(YnqR10_7!Sg(`SspA)D>4|cUWg!$&%udadkOLQx^mrWjIq;jNEq%lR` z?f*bre!~t7eBHVfD4Fj=M^c%Q!3UeXA9$~LV#_XpO;EkF75lJ4#PDk!hEl7nRq#Z9 z6yNJtR;W9emX9cn9=Rejfa*HS(qQrYa5$H`L;YQ7cQA15D$bdC;@*|SuK$>`shYu# zsB}iI4mT;fa9G>!@to=(8CWuhS9MQ+91>@{e=u9CAcH~|!}RcsseBQV!RZJ4Ks&*` zk-_=3ZIcG9NtcLo+5=&-ynCLxZLW(E9S;$5PysN~3#L+H+^s~Pog)ORCvYdjzhOo* zSRVTbDCDM80&N^tGdB&@E{kZrIWe%-Wb~$2-~K;s(lHFC(~RZjlc^~NQ}~wDOZa#A z`*_T6A|F=~GIVddFGW4br0nZyMfi$nJRbv3@3)vv7x`xAsKVdWaND&kZYy57KCrZ^ zN|Bje?~#jL*4W_ee@oP{LNI~4 zyIQDrnIp$juxQ^@nse0n1IeKDgW*dX2KjlNJnGp5AB^&A%;!lHsIjO zhnnv&sTF1AI`ub7Lbd&YOzHd0gRf@+1Y1QW*0r{hl?12To1*452&q|100SL2Z)E+S z*a=}f`!L6O6KKH;o`_MYCz0y4_A2v$vOT9#DYHjrOXGO9+dIm(`NTIVYKE4UIbaRn zQ6z;ou+xv>WOrILe>iEFj!+g9w2~E=+>ot*sNAjM&!tqCMeWfTOq%$lz~ONX0X6IM zsv^{T%E#)=U)~3x)H5N>nZj3Z&?^?Y~`+(t3@FA zGUF{k0v!Ghn}ABOi~!{k7fBZ)fGNg+a6Rgp{cq(Y))5R}Ewv;?R7Rl9yS)#ODkVkf z53&&{O%-9g6Nq%dAZB$J$GN+e!O#3sE%i-J8OkjeQ;AvBHy5Pmdz}jdcd8A87*+R6 z$OnuBheA5L>AF!${giF|%VqtMp3TB6p2Vmd{;1~askU7n?W_XmW{Ag2MvBKV3JmI0 zAL8=JO`Gj3L7JMYBhvFM`-L|>oY&UgyW{Zt3nMpjQ!X7CRM>e(8Iw%h#)}oxaC~=MvA#Oq3sZYtogOew_FC(ntD-?Y7qK-W%`_rwTZ{=e{5?W` z#z}O!q^Y7q-i3Jp-PW-;^jg!j;Zetfz_SN1a%+JcWfb&9Bs>26_Ox2paGDI+F^nsO zAB6q?4VFF%hOR|djn_5#r@w2*n&fnuct!>;(wkl)#^oOB1FxX#BAXfx4aLZT`W;QG z4Vxh{bMn06w6QFSUd_p@;d?qZNY+L&s?=p?*VHlS0etvj>Y_&FQYYfV=0+#^OXnEy zpnjzg7`X-q(Wnh|3eatiC<*MSik{}iBe$z5k?eUVxH&zs{385h0j_iOa4=)B`K0Pe z4Mfx*kgd}|zd~E*-G8gmpLZH`y}iF12|Wvsl}?m0u)DXxG?+Kr$RE~&o4wj`YIo$- zt?Lx*Q2T+-c;ZNo+A%~Bi*mu5(z^I7Cd`G-LR;Y+y?a-jc_SR6W^&Pe8xNgS-6qq5s!isye_h+uz-X;XB(cZ>^LNGo$+6LV9;AQH_CStq?+V zQ-9vIm$xSO$p0|j`A}ViFiTTESWLKGqd$~0nd2FxQ8aE6E5H9(8F4>3BFM;9UXeRX zFq(RMjQpi>6iR&TATNd}3v<&PU0gQ20TrAVilnQAd#E8Vu_3bsP9`3fl$Ce|6H z7$i&4m|qIj*VUl)QUcR9QE<#do!dQgavC2oM7hLFiN)WQ@mk|B#f{}tE}|515l9c< zB-&CEyIM^djQNDNVY5pdpz!Nm19kr?PfsX1I>;Sd=kvbH^5vufAt+z*=98+|dL z-KW_+rn!d2B=~IwI}vt3w|9HdK}!C7X>j_^BfLvd1J)siYVCRZ)V{+_8cug85j3X^ z2A1W$X7R~r&n?+mxS3^CSS~6BYZT3a-6fpsu{;e21Z}MMEDk30N#7brye^&}t57l< zF3HcF)dIMWE|pr|-E84~R}!m?AgUR>CEMnCD_zk{G12!ANGuvp4oe-K$+$z^9z6G` zCJsfK;rQ6do?7t6(^uAqAuB5vhA--DJ=10>LMYS8ST%$g|1>dW&UMDn{kQD|si0h} z+@A(+K%;<<1hN5q3F@!uv^Y74KQjcy;f+Dm2&k<;0TK9-ti8#Wcg=Z(9q!bdUhMn< zRMy{< zUGjK9nd~LLbXpL-vzMzXJMh`iBYFP!QAM4voXO7Ydi9O9BlkYIS<Le> zt;lo@A_C{Ahr&vz1jkLrlIHfrv$Q4uNe$6vo7ElDa>WFMRX9Px85P~uwol0dMm9o| zUm%~<-E*ydNNUcgJwx6|&&Lp0r?bfxk5W}+@7$U<%8!%`&eRB~cNn}|KL@L{FtapY z>s>$XCr#x-H>E0Y&EMI3%N0MJ#JYeabv=dMmA#N^rJPHj3KtXW4K`18_M5xMj+mP( z1T&JGD@&)C2~5dPzpeP@PY6|F2rP~FS_ZcVd%rYr|2;~8ytraMY!fVW<3g%Smth<@k3{Yp_i;`%%CJqc&38|#RXFZ2m5!{ZG@_` zIf@=i2NL^Dk@`q$@nN5t4Tf7|=I9*()_1ODjdg%=xe>!`IX5@19cD?YPmqGwxem5| zp#`Ma@8*0i*s7bv>Y6p6akCY5&eglYl*J{s{rD;rR#$B{)$v`~CxKv(q>oz>)NpX? zgTldJm1`VQXgSLr+1M)>X2ih-NxlvwoX#(&2p;A?LZ~rHyQ0jyuc*>~Fl0RB6%g=o z9Bd6o?%c6E7xz;D&Uy*+0<&6Q@jJaq$zWzpWVM737}k25LNOGXcNWwo@>qt5PvaleP9ZX2ExW?KMzAYf~y{F2?>UN=gCGBt7q8n%U@ zBUUWqM16FE?~LZ?4&Yl(bzZ5V5|KS*alqY})xu@m(h4dKf%*Ob{qCTjwl0a_UEiVhg4p`Gl)j@Iz4 z6G!G%=uODMW_gF~R4RHlYc-mlI>>&+MHmr`KkdAN0n*1@g)Y4ETIqUNk`xpiVD)r$ zrE^YV63%PBo!wbtd*Pk+pB25*VrjNE9KaM{-+l2bke(Bmk3Wm`#ThG`vg|DnuHb|9=sR}>3rq2p=BVgibJ}dUq6O!XATeUxutfa)vV1&^vU(p z++26)wZnwn2m;Sh%K_EthZyfd4}$?9$uXoZz`ED!cv!Fwal<$`8Rf>^^7?12&8tF( zA+iFqN^N+7);C)-%Bej!WSYoEh%=g2i_1@!I7f|!(9WLJW4U!*u$oPAT=bbgD>f5b z0h65avVgIx4N(w}qNw8h%_@tutQK}AAmf%;XHp=3xbg5SS?r6X(JxvK#ZoH*a}QTQ zW+mYe1$;9M-!521v>I{(6DRw|Rjr;_D7P|Gc4Ty&Iwi$UZ?^78{02{rW#9r2V+Mjv0~o1iupHlbRH~|ak!axFD+%%nv0iC zHN%(ayipGn_OonTdK-B*<#_W3Bp%^#(z(|(ZuP!rg)7M?H7dDpN2Ii6=N@Hq(H}xo#_4aa)xQ@`3>{$`;GMfrm!=*5A0J04dM7ocGi!JzbKk&rV)9o`B>GI zKhYpDkFih}Jlz-B*z>WG$2Uyy_K;AVLj~K!E?a70fv#vODCLTX)*4WwkQs5(n>@ma zp917>8UeH!+fou0wYP;uk9~-RnnC5%B5#!>6!vXK%B6ODcA~WM=@h358TJb8 z_DGia`vi8?C0F0=kNB^JmsP?CIqOB zLSY3?O(G{6rt|nT!slG$q~UID-G=x~>IErLEhmNR*$RPP$I&8zX{N=nULH&=DJjTQ z<>Ixeuyn6Oqj>~O{fb^KQXCE$GaoueQ+Mj~W9sw0>TBcoPIFJ4s6C6%qADiDn=^T) z&2OU*mrgj$FZBH^a2n`>u}W70w9Cm+*G)h(+{5ThfV_`gurKjEMH^#khs=|z31WR@ zFnj`*Lh;+X_Z`JjCm_y_@cHnlKZ?u(cxks0<600Zb&0k!k;B`qiE_;|9^*}K$@()M@3!sz8 z<=Fp5RYIT$0MG3txg*}I^HY@|WF_YBJUlDvsM3EH`QiN~UHfsRCTNjt5V zB(uSwj|d{o4C|oBLZI#ZI`3dA;IqSlc6&L~Gvy_KZJv6d*Mhu@J*&rd-}Llti}|QG zrVcYBgFvWkGY^X6&V6-S$_&Wc&AOl#qyCH7u_j~k@lQv$cRmHdrd3kNZgyl_^hjDF zlyyWWvUK#6rBPes*`C~rx~=>AK96is^ZT09&139snk8%8nHu0(U^d)rdY(owRf7m5 zkRJ)+f;t`V;29LN^>GzT z5|a;@TG`_2O3HPBZ-hBhR|3PWviWgq^*Wqa4EzIm=;70Z1iNb`6)FV<fj0Jke_n^@y|DUPVs=0~HkxOZTd~03;E6o5=E=^8BvKRfS;J z%8`#Wy0Pn8`xu%|J)LZd4wBta?QT?Ai+v)B%_AhFyAKZbjSR4B0uE=0P*CKK*or64 zcoUz^YE^w26v8EMsauH)F1FYAd+DZf0Qvz#tIylH1r;VR6b*gr_MloZN77HKn^Kzbl^y>pTQ#pOw+gec|c zYkI4(u`m3Lvjo`Bl@np8%TxzNN8*Q{Ae_Jcq+eR~>CB#Y)}sAYKt57v@r_=2Wtu70 zBAX{J>_IZn16Ci!>*mTUN{mM)?@~ElA!b*}>9!$PDS65zwAcEM&Qt`r=~%N8WnRq0 zcqfh$<6_7#(##Cm|4+WCq6KPZEO(rWwO;*fkQP#`0hQS zNlOhxPWQ5ZV7q+Qjm*Wdti)yYQx_X)Ffa$G+dDki&K%8e{R{J^)r`3_H|BM*dnMT6 zy>fhj;`;}@?OUrAk);PI1Q!_7mzA=B(Y>y<7anHKiB~fGE|GU+8r)?8VT-wcKCDP| zt|+s}Za575AyKU2@~N}Lg&&2XOMY^T8YRY?>U8gn)&|U&<%`&(uR3Usw7p_W0NwIR zbPH*1TJ|i@r0sdF?RU&?^xn52)>^+C z<`%-T(SyKL*th%g7#8+s6AklAPnwPbCAl!o$ZiYg{431Ou~pm?T5z0GJ!hPw*RFVC znziNE;~P7zx%hOi-|AK1_B0<1==IpTyy(0#f>Sz0na`V;`Q3XAzbrft5mifp%wU=!o)RNx?h+-#B;6?{8ca-UZ@QfMc zcSb#c<^#O4CBb7Di$Ge&I6D5-5wrVOUH-@af!-ZG*a0o`m+_Ta6krEpt*j;D`(8^*7ph?Rs z=YQsK{&{B!<$yC82F##uO4d8Ke8$$iF1uGQG1*pZgbjp6b59Ltm#>xY#p_piaEw0z zf%ec7yhBctVZra-jF8LQXCWhM45aIcFp# zqJTcELZ7X3>kUt%Ws;YDaOatk-Ur8u<}*q+UXgR{N#r7dil?kFy2Hj$k=?D}Baj0? z^(zbPG@tE!AUe~V#}R#|m@ZlZ349fgd4EtXq%gcqAx$_w&LHv5BQFyi9*dplfU+b;(>*aZxhdab-40lxQIDhR*m{s%-6QvzI_cjGzfVXo^JYirjWO#q`5 zx9iI75S7+d^P>3SWUg=rJxX&IdQJO{R}^Cz{ARG-q16@9N4+5a8W6I-P|t2=j_)eD z<;73O z>%u(tBWP9Aao94s{iAow+-4(t`jl{WAc^K|uNXhyM$ZNe_UgHyH(yE}Q42xaYO z3?5XXXRImOmoR|YAbKG7Oes70n|YV24WgpOo@lq|gY8q&+~A3O^-|tX;2?e~T<+oR z#g%7dm4uYeqX##oDV?=;kD2&_163BBPf<&+;kYV#fR8U1 z&-WfyJF0rS)GL#?EojUh^&RqFVRyDTDqij z>5F7r-fcWq|5{86@#g-SQU~-wJ4<_-ZHvmA)vXfF3N#5y6Z--zu)(>kQ@F8r>0G#o z#-dI48+?__fd0ESENExo9}ohpZSX9~KVsaY38OErRf`o8=`VDy;-EWy%u{$9A(x5I zVjPI7HW7+ks&|TJ*KNga<5}!|kJG9~#HaTrD(XgGjv6g14W`#yEL6$nlN$Gg+7aU~ z?JFe3RS5$Fs1I>_W8-z+sIpGZ<=cTot!D!5?_kaHvbkS5c@n+`8+PHvv^T>$7!E(~ zTR5>YG`-_BUg83zb7Ru;2mI6o_C$1=*g08-vX(l-EpNm7gT!cTD8pRBgyoBJ9jZU5 zewqfRoMa&dj_EAcKWOf!U}++#51w|~N<~&shCYiIo3d%8;88fJR9wVn8;&-r2-e;1I|WH{YQ>%_5|b%IT-j#>?yf>@*RZV7z6P$3d?=`Q{T4 zYBr;MYYi&6I|1A$jZQ-etk!l#bt>Un@y(6J@~4(k>f7>722hj(_2?l0KGpk<)c(S1 zeG>=ZEEm$mMwi+3UJumvft{F%5x(HINS}Q-&6tB;8IJ%0$t5CvdH%5g;mx+A~hmxs|4u zWrL1NSWYy$mha_sGW@oM6#)+MbS$S7D%Gh;@*s;25r`+Mu-FfNpO~RROd2bJkI{8| z^k8>sc-M+ceak5N%gP}ys9)aN{Fq}?dwE0!odr9Faf%S=%dft#_BkL&2|FPuu1-6^ zP9O9VzG8nxcrkhpxCp3R#`ob}SViZ_gWvQQH$)O{3x*Kx=V zVtoT%3SpNG{*Dpbie^1S$FEE8#2P|WDOD1L+B5Kh#Ke>qcogkSr{_GWgp|TYoH``c zZ691EY*13kneLfNRT2YI$WegUIKBQCUDZT%D|3=y~U_pku&4OaaGO$BVA8*Jkj5o7QvUCR~-H| zjT5ts5z2Bc`#l?Q7;cC_!edX}Wv5Z2<*U86kFS)*G2HMM+9@{^>PH;lz#NwMH6w#w zG?yb)uAORLb{HS?gf;cvp#V&q#KM9@@5S zT5LXzXVacYqFm;+HU}F!SeYygWYctZUBU_6AgX5r)z@!?DqS#4i4v(hofpAN;g6b$ z<*`}&v#w>>?Q3mrpWwrvT*yD(T(rvemy&-y&E6??j;}12nT-2t+bNFGzBeyW%opR> z93~iy)*b+5fF~kCkP_#>xqrrcKWa53>O`l)WR%O=Y8088JiFD_gVm~MV4~iP5*o82 zwsIW_%My4b!ue8#IZ6pM@u4GlqjSH@*z$2>SEW4%b6%kxtm*;AdHR=8x;ry9Fe1~m z_I-Af&y0(S#t4;yC=lfKIv=c8vgb#~ZP@QhBVfWDbL0L_#w+1oS8b3|v0QV&YTE;z zJORnx$oOet{hP@FE+7~fW@T%)@(Lqa+UYH0p>57%5E4FP`|5&Vt`H6f6=x*kz@a%p z5YgE*EF=uh;N;2xrw{TGpT7ULDkD-A?g`t!^NyB>{;R+b5Peu`EL__7vwr82Z*)s< zg0PLkoacdTgZL{r&W1&7s^IQEhRJ<^z8En8Z&MoF|7zBM$!mbcBmAz61*RG>t7ou} z&6;_P8I?%Cr;Cn;?Lz`rg^>L$_3!Aq91YhFHgSzKUKFfPkDx{fF#>_MzW$3Wf~dy= zo4HiHfqCb++ao?p2ITc=Yn!?@Y#YVkhP}Zy@PQredBfhp9iVS*j*4~+3RYUFFy7Mh z7g$S&pBOYi^NSL#pHD1~l!t#2A5Re4Z^lnw4sV$XHurWhnH^g?;tW@K^>nh|Fn1z>o4aICNxds!Zh*V!yavw+r zfS-sEz;tK6-xq8VZj40P$miu|%L@rp&w@?MCR>v-Gf|^;LayA_Gg(Eu?1=MaP73GF z7zqx_XQ>x+**o=a+0T%H1Nh6A+aWHKeA4Tre?VNEZ0l-J8wd-I)5Tw9fy+w z8I4|o+})U1x#OEZ>f%3dNY$wm1;#3fYMfzbZWHK2$PJS_+?+0z)^)XXY)-A+*?I^+ zO8(iIndc^|T($YBrnA}7#Y*pJK}!S0oCp%`N7{hpyfM|HdCu3@cT&h^eg0Whu~qsi zl!2vo%GYSeyioP46n8Vd^z}p0=0+n=Km&TDcxe+);PK4M&UQ0#-{N|oop0e z`9vbO205!>oyF8bsx~aDkXd8rLodAEuQoY=HiaImZ^TxJE5nHi5yI^u?EnT*P@i z2nYKn6fq!Y_)Gf?u0;oA0v}C`bk>qt^A6RdYJ3P@T^i5zAzrpe8F3BvpH~O3%^)ZT zHV-&ufh}yuhq#tq*#zA$JZvdBk`MViTmr#9-&bj7R~_0GCYY%ee!x*QH+TDl8En6& z=cx&OeszUl>A0Tofrj&RIEwDHKn1CZ`1ZZzGmvM?3%*n?2$Z*8)V4)tjqf@!lYg7b zY&8j=wY?5w593!KNpsGA;*}YN*>dAZm~WrrTjt%5(l-vOGcpz;{hgO%fwMf1a|4|O zG3c`?Oyz*lnZG0Q?^I9l^n$F6Iph6=mI(DE@ya>iQ_v4q4u<(z<||#8c`D^OcwFZG zEZckEwdxN>QNi8yTrbqa1Xa9DH`zom&>#jR?0nsiLhxfML8k(rS;=C?_VzI1j?4aT zJ>F&tzVFus1TwD#cvjkss_D(wzplyZ)$q%|nrJTk%BOPtl_zQy*~C`CcKU3R2E=1d zJ?!_JA|RdE5xg$+GLgY_!k4 zIOs-Sf(kt!A9HPx^t`1%U6(HO^&|tlv3NZ$qm}pUWL|xPu-EdYHPnZK%I<14v_T;O zv7^ELRnSwUN2l?HosC!b5iky6_6beeO3d-L%K;bJMvadtoOP65C`j{N~qft!An! z!mxNWyk2JTx&UbM#s#Kwt_w`X*F5mZTnz}@7(}RX6EljI5{i~~RO2eDh3(RxFtOkM z{Bo_Am|Vhp=DSnRD=QEOnXaYD8S|N!=P2(U0J97qYiPxx&%`5EV~ro9#$GKSzgmz`mr<-(i_xDe`wiQIY=v5M?GSb+8&|L0#4=@4Z0g0Q%!rXXKRcNXu^eemyfVi4_tpu$O+k;Z5aMf1B&1CO2-T>@7bVP06Qz+*^7V8z6(} z4g3Dqx1HxZmR;6oCmh!91HHXfRX{HZURvhJR{{N{$@an=;n(09y{WwFcG|t3 zHw=@UA>Bu^$7VXwmBCb~Hw=@9yEcfs7me(9%6u|1GWv>56qYOi4^|ule;_{Ek^IEv z8bubFCoUBMgW29g#)o+VorL7)zlsLrJ%;f9tZ?LAmHp|B)#A4OCAXbN(!D)8@RFn^ zjqnHtVQyY`3#mVKtu4>NlP2CFuZq19jPz};o|vzGcc8@;E?%LFuUo3EDK^-UNaeJU z&F(JNjZQtp1KDrJ$oSu$Q;umoAwv!>9K{y%qg2n5rX%$sQMQjjj0nnm#WG7$iF)FB z8!3(uF=UjoZ7mO<59iMCMWc{jHNX@QwFz$8ryX>6WotEfJuKTOA>mEQ0#%;0vAPL? zCL|P1m4t(4IeW!GAW-!d=~vytvSwM?hQq{B&n6pS0z#eofRb^WxNeVa`QI->W4k7x zjn)3*D(-JLv;|%qOBd(%eZ-m;ORI6^$<^1g(IyqiUTB=etBkv;qmPh}AjO`%h9VHHRvkr!X(GwwN{W?u9vv zsoB+-tZrp2HYEEQjRWW|m}|Cs-I$Ba$#(gJAYjp=arjy_ix@izPXs~;?}AN(88t8B z5NAH0x+Hqalv?n+s6?b6=w}MrzYt3OjciEc`H=hE*S4%FlZQ@(@P7OMzg|EjF*^g! z;soA{t0yv)@7&>#Q)V5tpL;A7CXoIPuCdfQH$M+gNhO}7oZTrbE9SB_c41#O2y< zUaMWTUk#S6O=S@#Hu%7XI~pG=7H^GVV7?I$UCg4}7D+X0kcj>L8_$|=upco#RZmW< z4fCj1SyWluNd2=HWbMP4R%yFHZu$5e4|((JC^w}fG_Fq)zuoIk2EIMySTAT%!$jpD zImkUNV7^ZRO8Y0e@Uvjtx1A?5XX4Q~Hclxi& z{E5sYPQW9p3|h<|+D_Kz7`@yqKD2F_7i$S{Kc5Tn_rOtW6w~W8j~@x=hhxB$>~TL6 zlj&1<@ji~G#>QT^B{LcX{m6x0;_xj2AvXXgcLkd4w#q4Hui$1dtJtbDs+9?NoT2Ce z1og+cL1#IUs?}$d6-QbwV%#veX^z#ig^pR=P~GP44>EtK{{SW zy}Durip&t6cqJEoV3!~&1P|G`!|jsmWoo02*RwmVaUC`z%drC*#F^XuV zNmEAoCA)L*psO>$*`K2blA&IPuF-<;#RPt9`G$}ZT;Jm+`q@u?#A942jx&?GM9!^3O- z!|_lijtDSdtDxAjbCl*e1S zcA$z2pc^%(EJ<9xvj<;@`bs8=Ib>s_;#Pb})uUJ;M+MF1(|(Z@6y>uybAx^)y!490 z)=mQ3o|^u$u2U5uTp-)Zo`|Rg;DBLbc&yo;^xoQR^j0bh_xEprr~r9KQ2Jug<{_y? zXD|ro>!B##E&Op<)b^c-=BV;B?7-LizSFVqusP%a$J+J+9yS+|#anHZ-WtsDx^5ZZ ziBY!C^plmK>N_#tt- z2G9vIra~Bn<~}h>bjYH&+@kIsox54*kHfArg--~{ccqh#A6d88dB9Qa4O(dWi>YY( zEJyFhcaeL$cm3pi?ifb2Iq3-=19nS2QfEx*bim!?DL5Bc+64V5c|-WSSJM~6<3hUl z86LSz{a`a1rA4Z;hiq$|4~6D_v507b&=Qralk=m#mQ{(t9(zuQg`yQ1@>#x%(>W#& zDjb`>S(v0hA+e!TXa|yE8bQg;!Q!I_AC^$_;Qd|*j$zQ8ota}KCo)c=8UhC?DQRk& z>-1-2JT>9iPO7RER{)IPm`ONaYrmG7@Q*pj(hof4b<#UIIwFn&0nVDR2BoPEL zJg48t0j>97Y!XZ{Q1AJ+U2^*vZPy6Iemom9NgA?q`MtOLGxBTAwmLYbcM>u(w8(IU z=wB{PRtTEaCEyy3N4k^>4^(|aZ(pP`1_s!x{c9hOjw~H91K$A8w~LQYML-ImY8_i! zTOwnH#*=kEN}5XBZ|db+lRJ0$dL6k6OjW2_d{Ep*5nF}s%&cc&e8VjeRNW~uiVP$ z%$9gWU9&9N_m7eD1yY&4em&tCnjefmlV(Iq*lDvNem^MwUQ zo^@>*b7wu+AfWGxS()-2Awuv*eM3{`+e$7=@LH$ksaw8-g{D3@i;FHx)8oGEx-Bng znw09A`>j|s*1nB0etG%zTL9Csu1bH$%%2ekY_Y;kw;X}$?1~lP^UkyGiQ~1apcB1H zUMDN}oluvdoU)9!mCEPhQd1d5M)C!TjRm$_M>K{9)!`f~eBh3pt9?Z`02BvPC+z*< zr9hh49o6$zlsj6=V6~Wo$fo&iD@Bpyf3{ z;)qpWi_P~3xwe5fLM)2qK%}O|h9zaR_qLqa_`dS@iwFlR19;+w*^{v!w(Bpi{pvNsb z&Tur|56gmJ9TOA&X%pgiH^g8;v@!h9u&+a5HY~5;g3zn1APm1n6O(M0d8k^}Do}{o z_W+jZb;3JnTE@6lGM6+zkb*?04r;G^aF4IV z+DY8VR94am#3WOy^HH{v=!};6aq)YAx+7ga9{T4WoRJTcM8{+&yo2kwNSN2b)i%G2K~9w(!mre-weD73DDu-@rD4j1Ci0+x7Mt(mJNo+WFPIGch>K~6 zfZ+O4LdFm5`e*4cIRW*pylhs*hC7}gWpeS2+*p55*NN<|vzlx=9j}eX)#m%P!?sFW zUdGmVEyV#~wXZ-2WIR%IO(vS}-i8a3w6(-igHb)3KB2Abh9_7@UdXLyp8?l6P5P2XeermJ&Y>p1hC71vs1)`7V;ywzeBihO^>9I6-O8fv+|} zlR7|ur;?q(yMv(jSbm`CA9qUbI0ncY>kK7d7E3})h~OycDOkUfxsUdgYF5Ba)2yY< zV0d?MG($s@0kw_zxsfu*>P~%QV@C04z8~Gb1T8Qd$R~MZ%=PEuDH4d|BZ*<<-dJt7 zb-_^n^^P_)^oGqHaVTlC4DfLkgi(>yHNxDC2;$k~H_r$-7JDBL{}50bssT;_3jTvJ z`qT10rikbjF!U5G`*fSAYR|j7SR}36=jje!4&ReKB$kx~BlJUUmXwuLIu#WTsUB3i z#d~}p!QozXGMUeq@831Hicp<^UOiTSNz}6rX_YRxWj?V2nE_H49v+ZgT1Vo6LHNw8 zXM(Ml(n)SS)|U$5;WZ04Offd4Jcn~36WA*y=!bi8sRjmznCG*dNKw=pJ=l6@*ud!X zd3%x`Lx6mNYy?=4($L|WpLJx?M107%PeO^fgBx!-FD&r9bP{kiwUJ68Bg6MA zVqdzLL*EHw&G^w~p@l%s6~o5H1|2+QPqyR>%4Fo`vUk3O=gsCi8h63GYCN~Zc2JOM zhS^#7{1rQ2I8>8B@^u$!rbgD)(}7%5SMC9Y_wQ8cO(;->2sL>=`{ixNY0XRKjrS%S zJhk`T)sJ&Ok2-Wi8=}; zm3qBTPLd;N_KE4c^jhgYzSExX8l76;RjTj2+Q(z_$Fq0maUPFHbQ}T`>SJ7!p>eoPHBqRc{ud z;E{U+XdVB~9BO5qm}*rG;T{cX)vGD#)kbF|6%?!*5_)Y9)XqgX$7zARgEUmLq>H)l zqBJ;g4VeLZ+o{0*7qq_v?0Kr0Z2I%hQd;!q=!^Bt`W>87=HhX>Z!Cv2p1UjDmJn?u z%(=1C7*^E({Te_0U6Lh)!<^f>_5`0FJ1byY~K zv68^YHL3pAw%S9k5HY-T&5#y`9i}mluKZ%Y9Mj+i$YuD+1`mlvtg3V%C%e7 zLf2%XE-0y&Cu#BH9xm;RshFg0m%|HS{d(kIl?Yav={yX@%W>AKllB{-kP7T~(Pm1V zZOikEOF!}wZP9`-($kr;A1$WpuonX6^^S{+OXif-JO!e*26n*mszrBw1hLUZ^}^1d z!f)QJgR^_mM&Q{Y{vf#vF%g2{UCpdGG#BCZ&qU(J?aft(HHKC&Qqhk0tyt_$AE9*^Wimcs*Iem-kzhdDeOdta*(MMzT z<}*S7jBYVz&Z}}YQL-WR+T2+dptXDNe)0^=dmVDUkX( zH0b=|vB2#yj3)qB9}Gs|qid|EtX-8f@~JB}_`zr6>U}8MEv&zqq;E_EB-#VApawfM z1%FZE?|+Kv0g4Uhmx~ci*@(r+)p3^S=j@5Hfj-oN6l*<$!WWfVG}Us1Z2pp=C{hEW ziD1c*qVFH~1J*dg0=aGsG#KpCZT4d_%kUoNFNBVXE^j!L!)c|w;u#dLRYPey+DD@g zHZOMi{_s!{<ja{IIWhzD@CLz3g)RyT{8R zfu#pldMf^ugrlJAJa7cz_Vb5j3nfGFach1)zPI`LGy|=X-atK1A3z#m;hul0s}I** ztN4psOHl-5#;~yG{V(?M8{=b!PLPt4YJW^(BbXKa3%fpsJkeae4FN z#gGF&yXg^eGvw$Z|K$S#N}hmxO^PP}vgF@gDUeRE1ST#SJ=MQ?r2j&X90=G>&VsR^ zujh|{Q>K4$(Rec^mi*cyyno3Az!@Sjz&nM=Uh{=N_i|NTr-fm!GP@;V`vLybpL%9L z8woinWX`;_^RZ3{z&b>mIi^D1ZE%zx$hGCPREtUGSsY&NMwDPXAv}>wPq8bvFr_{a;+OA%I%bJk;|}es;|`NO>%geF(Q;l8d?e3cp8TY(Fxb%kF9GA;UVvh;aulB-{zXRXWc9Q@%!g;ohbcs!4c*;rkk|;!jZ0-5B_G=U(fW* zhdU6Au~8V`{iTG^S)4Yo;a#k+BL_o||8A8n^4q$TabfG$W;>3&`x?Kz8~vEwfJkO| zetzB7^Qz?1QH&2?{QU~uHMXyV2jz*Ao6CyTC&n87)Q$O*)!dH<+{v?)%Jap=M{_pp zBM0Y!`CyOU&GAA1&XR9}IBpQPy)|QYx@X-w{TGcYm+7to?_OpS)0mGzzmZDl1D$(Ay$+V|qODp39ssvG+q}lfUt_}GU@%hP7&Ow| z|KbOHbUEOFZbgBpLUf5b_u8+=$x=X^jN(_~WIqxD*ezHi8%P`OE2{b;cwvo~0uJI= z9yG66`+u%E_*?S)$@yT;Ry>1n%XYcofLLe$e-P#)*YNq@u3U?wfd6FDKQiDq-v}B< z@-&UlqB`GHg;h(rJ0}Da1i1g5EEjjgL}2Sh!~WG(PU1&kfs9D+W{RE(2#>*n!;$)O z2_|fZad7fzJ(#a>{wQq^8S09<$32@xgOPKvKx@e>Lc3lHT1-bxhgtyIDm>>-`h_AwkN9 z*h3_f0}znfcbo9vEMG$l23(Ps-_|kewJuSG+W&s*J8T5=!l=>bkZV{SW3k`)Q^FMA zJ^>=}bsjuf@A`YRef<3VR-=={zeNNv1geU00C(Aio%~xm`ff` zNXR7OVgK!~tN4d+;(o`i^Nl^zlU^=>(#Q(qzHCu(`@1Pz!Q2VC+DG{BDEvJg$Ea7s zOVRV+jg@=~RywER@!+d2_`mrO46gH4kuH1mH|^N|8>f6e*4r3mrBBEDt;hGjkvE_B z^FADCrOS&UKg4B!IFx^$4~*%oQ8L_C<@mq)7v^CN0~m>Qo}(dfFfIsK3g3=)O#qfl4l=BFKvjvd!98E#n^}Al_!HVxWD^0fGMS!~kM) z7PvaI7)4#=<6-wQjhwpal9=g6Rrcxj#J>^#RgcbZ4T!OFIPe}^;+j-AcrDJ{_4U<% zsk0iW~p_Q-`U+ z(ycts({%vlO!CWFe+!#^Ir00^RqmUc{5F=_AT&4ERn1r$FVn+W4~RFjRlCUVP%@&PY3LaY9Of2$?N0ku_Tb)aDyB^E0Tf(jt4VPs_F z^Qy+9w(V6-U{;Nu%4(Rf?X9LT9!3u$n!CL7SL}XIl8<>lWC4-cT+q*_hlVU^H->k7 z?CX1}u29NHcgxT3EZwcz-WB49OXcnYPap2K_AK?jyq+_?7>zg(*lzAMRF|JNHwK{# z4|@Mj?u}?c7bV%-u3F^eAJ$^$HH+I~tc4%XU`oB$=Ii^oFcqwwT|LvTOC7QMIpYlU zVrY*oE6Y{Z1c_+W}4NA{yslPut}+ z(q-m*dAh@Hx&Hob@6J?dZ;75Y+j3_pb;p^`*Kd02_HH&l{v*3`{bMY&zrUp(l)%-+ zXiE3et137-DQSOr#>`L;(rESM&*4N_)+SWi++7zDDzfkcwSGvwT~J6yNf??wQ&b&ZuydTDCSXTC0$BfH@r->I~4B;HS5v5NzSI3 zzaF9DI^@ggy!C!5VtrQaQe#@C)_F`pRFMk}$SNy@^j#>kH}~b9=#7LQ=br|NCWNG{ z{wqiK*6uZxs|D(GVbZx__L~exkQF7_4>-}2rhy*$iSm)4{8)o3MNd;xvvTU`J>^p< zRSft_a$7MfnukbBRBASua%*4YaWbo%8{BPW6x)4M4r#X>YGfCVE^4;r8zcvroQ(sC zV!>To!(}$LsZumU!RxId>jedR1Cb@-Hao7E8I>osMiEG*8pC}e4V~}}jb5ygP_G9F{mQ1zXyc3oD zyt(>mw;5UqfoD3kJ}3z?N|1>6Do zx{J&u^@H@;cJ-REzIAnp5=D~|@f1Ck32y|lB{L+V)6A5*Ou?{Tq(F?omQWXw!dgSt&=(V(3RjdR;Gnw;A-o+BNFjD|9EuaFN=qb=nAQ9#iD_@D zqfwZm=35%Y7f8sYtutLHx3ZTfHos=@P6ggQLWIJ^P3X67PH3gZst6DOO<;E1(~ZaXZvKnc%iDqHm= zAufQV6ZV@$$99uF(CJUS(*uqdYz;>qT+G`nAPbD$CEIltw6q_9j#b!?C;7Ybky*+< zovM)xTDm{5RA=q`4amS&$7uSQ(@0bFpj;b4*(yS00H+G;=w222z`A**?vC-tvz zY+aL<$hdrcPjRJ*Sd2Qra{LV!Xqs_OVQ&I&rJ75O*_f7|3N-3Lk9|bBadjhdoT{pQ z$$H=i56dCa4hy562Qrk-hN#-crw1EAHjU#Uw|C(l6zptcATv{-6_^2<dEQ1cl)2M|3Du2a_=`nyT!?g43J>spn?Q{CbwA@qpp5^0yKPH%`xpJqGi$uUakl_w9&a5f7OU6%< z!79l3K~W0z;m4x!=8de4gDKZRjF52PCAc3l7gP(Z)4J2}?4g`^&mGRRVPt=|ov7>T zd$GxCzY0W|n?0o)+DLfksYNYl`mob;H>OMhRPU2z&&tY=l;5I+U}~yj44n^k0A1+4!J6a4%-3 z&mZ3N&f`e9L_xVB5q`^~nS4?tv^CaLZWxWAGRAj~li8u(+d7S)nq|kkgN5^wHGu)p zKLlm|&PqFiBLFK{6dIT6soEbu(}at^u1)LWo0#?BuL06HdyI81D}T6~_0pq@O|q}V z#aejHTXL6*X&?upD2$#MmrkCqo>aNTmcH4T3pdOAKajb~$V|)E{3oT&5%3ZFA@5a4LM0&)E6^kc($B)8>GhGfM7g zIYMgqupRyOEfb>xrGHYO7&Eq?%0xNtm0hU$k9RGIGi8@T2=ReePMp>yBhw^C`v@1$ zm-YhR-~w<$M9E8-`tL8l((=TS^r0llo7SKLNk87YHIg^T*G&}UIi1-BASnG+DTSf= z<`kvb2FJjpmggXmW`kfV+QovAaZF@*9xCZaPQ_)YO#*UQ8#44}+H^tnPDm8omO&tR zS(dUF!8j~pSFF5muCjX@0JK^nKDTmZ^hTi-dA8&41n;8-tW!rah=_HtSx-!NS*%sbc5u|c7QVHuhxA^XZk?>C$joE|reav{DgR3^AWpA! z$b)3B{{cC+v~OAesWSm6qUPK5oZs2q2i}6Q>4q1cDaMtLLFHbK&d1d-PE&bIlP^%? zY~9SBUsRux;6K#Ry3#^#q}~*hz_-*_K{xGyB17DR+b#mJdM?{V5P44YUTCEqqkIgx zUcJKmeP?b4STj(BQfM|8*8HM^o8$>{3OD7nJ1Rsn)~ek%2UEgg_ZGnYRd(8J_gkP! z42Hc!mUf zO2W#T1f{wm3W|(+%b+@Mvt%HLjYzW`3JPC9B}GF(R{55h zY^>uWA1c0A1C`Lyu;XW-)^HtiN;Qybq@iM7{cfqpYF5Sg0xE+lQg5Fk%^H>^Gw3)E$Nz7v-91L`FG4C#X6X>wVLbERhZC*$pmf z+ABo3S^`Duh!GZ|+$rYS)(I+mBt;;D!hE4BDCt^_2n&Lym(FhX<}5Z(7#^l*%oGh! zu$6@QRY2A}REqU$FDpY2xrq!b_o@|dA{{`#v>eM>9!|#(>ss#2NSW#!^dGO?IcJd)giZ(jN#4=8?7Pj|m5A-N@I8ioG-nRLJ1`mVi1Jt*jD-p6*`2(2Eq_2nrltmzvptui=$;t%E`xM7*LJas5_1`} zE^>+wG_$f{qm#=3ojk~W8S;Z&xWi-#Dj2jaY{6xZPp^y=2tM!iz@m^(?9Qa6Bl);p zgWQ}sQ8wZlt9iwE3^4S99!5;}C&04UL5ngvk!VgLt!RiqAbPl^#m`|WCH9@m>Ak%2 zMYast-Mzker1yGI@xu@K^k(qPc)m*?K;HmNf2SARgW<V1h@!h68wwoA^wK0D@IE;_2qn*gClv%IcY)ft#P)pO^@QyM4{KB z0>|>g;F+u<#@C4EMS_+cw{+= z@_zIavW1G97A`&}f@pv)f0srI`UzR&usylJY==dVVydoYI)*{sJsg(=(LXJIpL}p< z&;>yzCr2aFfU1eKTwjJ=v56Rdg9au0gdh9+AJ4v_j?F3`EgTefA~rA2&jSW}J)`LM zcJX|NHp|>ZIP!(#o$2*E+?EBPwj7#{Pyy|+iU+ATcm^GEV%O)hN9o46 zv>j+c8T@8@qlhRn8aH}ZF?CWTP3-ujO%{6i^wO>C=l-nR2WjvkXxEZt0Q>BdUdO`o*nOv5Rqe!6JT z)jqv^MNgy~o%lP^>X0>R|D^{3VK8W!bzY=i!i6N&0=O{9K!f(KClYdH844)Gl81U( z2D6RTACF5Pk?+Y~7j?Nx%qZs@+=EnF?q0Xa13n72-punv=Eal5ISUaKJ6#Vz`uaKk z{l~z)lB6RR*^Qu{Z7_4Wib?YKBP3fLj;QoN9W;^YZ9RYKIIvN;TKDcMkq{}^D&>yCYlA><7$39lLwi-9VBT-2 zGgUgy%}q#I?%a=sNm!r^o4Cbq5>g6gi`kh$^G3)qrEeHvBc%|^bTgdCP%EA3cl}~F zR3`t*aM>|W=kZ`|5|U+=;3$J)o7#gfVI|)kuN&IEB6@VKj+* zP<`e`^Sa^0o9WoOjxeYliD)yV_Cw^Zt%RWhv8joj#Gm zLj(czUE#Dd(=R25(3*GdTg8r@PYSqyF8L1=f-7;j@Dno^ZP7-CPNnwUW>AP>kv%lg z5DMvcg48=fAy5Ozo2ePnFam4^>*y*MfTr)nV7^OSi{%jtXe1q(~F1WD0%k@Uz{di-;}#9SHk70}I- zE-Y))!sU6_@CL?6DxjVUZB~q~ER>;_E?I3U24}%F@8(eskngcIT{~`zwRWS-KXL1) zoaV9Y&mYWAY)?4sIDXLWH`xciwkfQ!_M@Qb!Meel?zBP)xOgc2vji#&3y1iJ!AY?W z+_vbt%*>_bJ{dMbcZumdrCwbmL|4Sxc8dV9E{hH*BZtChkv z{*-b@Vk>^$bdwCk7`+soVtXHpHEFeRe0Pa6odsU}02cphXn*Q zw-C)OK0iETy}w(M zRK(}JH>i*(T4>uimAw*8l(J!x5X*DWevR+7%;lufsZO703< zcSn6RKV4?OEPj|h&qC(tVEE`o9@U%!`c&+jH=a~yL9Qg+Rd{`LbGsN#Gw z*5Qj+;+C$F_!(gl57wT2GnPJc(98|kMpgx!+zx)SYiOt#aV>b^WXbyS?QvHQysfY1uAU!5k<>YUb$4q;qvbqDSO~Cw;ERK;HSr`wMXjuLj~tsErK? zS!A4+8G9DyM8oVSn3*ZZ(BP|>B!3#Zxscjxo_pOO_o<$Ue@aJU5^v9YDoO<-orf0Z zdROWEi2}!mE-|~fAqTeXX1whMn@YVW$k-4J^g<prSBJeN3#oS(FU9I|ORC@usY_Lv z{nk4cji}1;g=NNE?aQc<{P>V(qxrnSSlF(uALksUI@HzYAKJNK5fJP=ELzC^8DwS2 zGuZE{Vv_qU21wAZyj`x0tBF7^eIK{rUC2an;6eL2N?+cOM#Az}r{=Tt zLF=b^9+`F{H44=aW!DyVBIqrYtS2U;AILRjX!Mk!eSYMZC za-&?+U{MO?5H`4ueW#ZZ#SsEO2-kz--)?%vI-bAihbki+(3OpHaWfmbMwY;+O$3c= zAR8#Eq&Y=7lhtdQOE|K}Kbjpb**^WmeYW9MvnE2+T1j(&nJ?tBRnpxuDm%ko zf^6Xyrx-F8wR^2ob!Mjh`4@FpgHrTD7>~KQ?AE{zJ|Ne;h`3WY`d)JPb)i1S3ZHS^ ziKpqT=mG1S^Sky3krVoqXRH8pX_>;da;`aGM1_fDelTqc>lBP^VIL0jmD2o&e~~|D z9AGZHn~CI2tpN$4{ZdDn^0I zJ!hF}Z1H-Ym+LBT-_?ypqF{UD!7_V=u&x22k1RBsu?v;OYfl$-=XWEk6Vby2k-U|} z!^m<@dTBpj!Q6DTlA9sVhs3@`gK%$oC$xM-P-V@S!&GvTd-s9E*fI;&_HU4YMQST# zU>Uzy0{eR?0~*2V!{~LH7Q?zmG*jAl6qu0+H*K|G4vXRHPC?&f4*TJ~PJ=ujv4WSE zND^iBpXfa$cJ_GIx9rbl*nFx&&d2bUM>HRLX^6_8Ii_Q|SJEzkKKJqqA0e?`PKV(f z7t`KKTbbTUh)h(IgQfn4ht~9Hbl06Y2Hu#-p)P6EaLh>|7C%F$*JpgEty-~yIRT1* z)&!zebbo0PGv{G07lABsQLfy-X!k0L+wsFQ+1Ui?K*im~0R&(P9ulWgXlja9Pbomf zwh)WoDNOBN8(DY^_2s7bs5PHjM|-la$otSN^pYQ%ou+w5;0r#I+SSFL2DDnxS1J}- z8`p(YwUrEyI}T7(&F(M}+7hd)owzkFJwI=~>MwSL;Fh9Q&%I;u)G13L&*h?>rP-9c z^YwTPaTdEB4!`pE*|okTDeW;{cR}(fMzW-g8yy5l=Fv+&QqA(@7OwM!!C=eXky-A1 zT2KKqSJcqkp$qlN{r8IUF6c{>LQ>-TZ8$KnrUh+Y?1^onMbOc{%@JsE`n3$7xIb`I z;qFdOj6BKc9quwQnJ448wE>rNa)*WZ?QD0`ARAi-JVL5h8wCd4helnDb(X%NYiDux;!&Aog7f@{r}?9$IP89_Uu0E_rDo$%)Z^1Xjm zQmg9ASe+Z#`F}7<{F9Wz7j|}cnQpa{ppJ*_<*$Bhm*%~Aj`{RDx+l1Zwjq%%M%50P zE+dJ6BqTAqBz5lZnjjasq)$4JO^Xq*i=u6=1mE7Ebn58;&?uBLm7!ixczR;oX8&8Kf3!CIs2?{CL{-*h1V&nnncylXzcf4F1#FfGWdw)4mh_ZKhX z!*tE;G-RVI2~ygICZaj;j-N`b3%^Y8IR936#h&w4!*#TW6Jv*8xAOU$x(&LL&-DDo z1kR65%+TyF$9zofvSDXOrAIumcA-D*>1Rn!xQ7rq12CPUH;+mySA1l0lt<}(yM?`if^P;i{Ur|GsZM*t=KP$* z#l#3V-az7Z)Y!Knch|OvULN^$M=@AamfoH!7QeUSdqP&<-t6K`x~-v zJ~Zt9E^QDcVy_j3(7CIUZ8SSl(j9%7d4oqy6<7rEwcUQ`a_E3nw-0t-wQ>R}%ccPz zuP^Da|A278PvV{gpbMw~&s$FEb(2}#dRZa2CzjO((_$l06`s9Tjo%)ee<{9MXx$)}#IW7Ys`AkGtk)@BUP0D~mlFo>iSUU!% ze``QJP*@q;Kh)_X#4ozKeENEK^cpZ=NJY29WyTZg+*9_I)pZXK)`xa)zq$9Id1^HJ z!GkJRjlQxXcgFfQENC!u0k(3uc7y8rdp|>NHwb&8b9M5 zpKuyi>iL>(G@tv!9DXIMnk9?AJuQ;~FkAUgv)x=6CfQsZ7zpyiJ4JocSh^T!sHZ}i zi9}bVo4}O|%*I{E8(T#WJ4U^_didx|IA>`aGlRMq_`#fmjUV^N!R;+5WZ{>vbtm+Q z5dV~`*Rdbk8~%LOv`8F6G8$4Xmq~Z8Kg=7m33Y3ar(W61LL*6rk?ZDRoYr<>7Z01B z2Oh$yP8l{u@i>S$e(1ce7Rl#!cV=mc0Pi$4e?q#mo(`=%t+)OA$ReG)+e$U9(h`&1 zCmUV?-L`eVbMO!N1?bjUNpLHAH(n70Dwc40r#Xg8OwpQM?9j|aUHW=b)aHt1w*Z-e zNnZDD7nfNf{<^0bsoQ|sQ%@ATGegmZ%M%yCz0qGunF^!H^b*zLzJBvhA6Bns8#U;r zz#x!^PsS7Nk*CV+T+WF0=D2iP$%PBDYM?j7PPt2r)~rrgpQ{HJ<>B(M5e=_Je{Uyl zJw%<)6X@HI9@iFzqe}YD^>~O(yuON~9s%rxM@g-8{-SJ(P3#rRW+oN-RMBpuH&Zt1 z)OCAWs8ZyDBy0)%fJOc0b-wg5)G%-Ol^U#4UfLcUntcM*%$oC*SPT1yWjK-mnPoLI zm%3+Bf$FW~Sr{(J8^zs4FH*^xr+0o_5LolL%oj4x1{?(2<6+Z1-4BLS9yj5cvS2O~ zP>huw3m1eqnTU`u+!|RV%Hn%7-qT$X*5CS4k|tIy34vf==->Xsq{2Vi)ox}kvwTUB zLOLa2A-NQ|9NQ!;Mz$T$kxc#`3UdS z>8V)PFo|AKau9liSC_C~DS{KQZPLFH%rR>a|w`;YNx;31( zRyzfmWTV7aqm6AIqn3t*uD2_^(W`6f3%8?b7VNGGrBY2PD-Ehx>y8Y!<6tH2uos{b zj01D`sOxNrt$+?VZu(%!9GEaZ5biZBUcXdwB<@Mb(hxwo^+9H~!D;+a6a0oiw+b1n zfLUiWm89Qn?Q|@)W7?%2i^?3fg@H20g2kGU45^Ix>USr~;g{~(3e#zzA3CirzpPI{ z4Tt&YqBGZrC<2}G3|gh-i9~qQ-FOstYxyr0_v@vjtJ>_;_M>@h(i^6)c-TPw==Z z2Kj71Cy`9*R@8lXOw0KooWpWi=lb!FNF-UzZdYImgvQaP5VwAR7oAiXeBa0>cd#W| zNTT~)n|{T8=)`C=&$TIuSF766kWPEc3I5cI&7}+_S(k52j*H)^9JhKX*ImT^i`I;I z+$9{~ziSLKLU+&%ksUYbg|Ifp#X9Z13<#MCqUYF@kKi`2PKtIM4tq5D1n9SFf12_l z=e~fo4h2#2LNizPf<=1iw4@}yf|wJ;%-PKH-O$yQSYG`z2!{zt-PR>dl7X0OwrWwm zP={K<$?B_$i9upY!U%+;UhV42d4`5Sv7NU=qgAj2+2+nw`G!Uz(<^t>p6HFKd3Uvm zl5YSFb2qmiyC<@EDR8yg(DD3SI{$c#Mn_?QKjr7H6hU0Xgx5OtQZ`aEvh{Wgmz_N{ z7vx1Zn!My%BM}^wuJ7)W%+tHiYr9=uTY8-0>zkZCxi2@GqOC9EA++6@cT0FN_rb9k z$JHG0{dz)|NnPy1^PRn{nA-<)HUavcyq}_p=#W1zVQE~hnW$gYWdv%?C9%@sN z5YoUMu!Hx@*c`!i7y*3urm2@V=F8{@i#~s{Of&5Rqmvu zyMXm{(yi^zaWO*R6Ae*gH+4U{zds7wU6LBAuzFj4S$FjHo0R4lyQz{CTlfhv$C=^M zx|jJSdb{10$f2S)b?{D+l^`yOT*Byh5nTAt8g((crE`ch?^YfF3(GZ7(8MZ+-p^&h zXMJM)@pUePCKk7J2UpO3k0O_1MBSp2n%Zq))XT-6WdVhvaAfYV1)y+Jhw(vgbRoFb zBSdb}pVfYI2~u*ElNO6k1#gHy(esqe=fK9df4f$jCsp;WyZ24PYGE+TLlGo_Mfv7q zlcD?*K$rRimM`I}k87Up@MF=F>6e1Xgiu!sXOsl2N55O?FC|l)HvmF;$-bvQ^4jI9 z%P)M%oad}nBOWBV;{tug>Wq#xKUHSGNZFGNkG_Md+z-jNK*e<0iQ0Z7lXw-dpT#5} zA$*xd?XikUSp#~5%wISNv&ZyDxcf@hckNw-=uyv2{OB7>y6GCBuYk!r*&A3QBjmcV zJ|2E9xuf6sMv1)z-TeaWUIQ>2*n*o$z;FmT*5PAGr@!n&*d18BzX&JBAD*Ijh145d z*oF|*;?Fw|KQNy?9Ouoykk0-S8 zcOB5gqLhDP^ZcYLcV>2s=ZDJ945`NXr+$55Jq?tNt(T@R4hMUEn&egJ>_ki>my#{p$hr{e#x^Vm>V78-mj7P8*YC)dI090%06o!0BsJ z@rp0-U7h`~8jb5go@aG8>#We7kXVt!>Q;|!LegON=z%-5+o#-6Ep8|4V;xNRclQ)L zXSoa#Z@paT-&%_mxxkyQpfDpx9w}(+#6SHcIAzgKX*6tH{**#pgU8W}cK2k@IVmgXHHxxm;W^#-jg)ZR zxqRalHus84R~6u8aQ)bFnvxCn0EJ+Gv-R~Bx^OVOtT-?QFyjQvoaHaJ{S zR#S7@nSF2l)$X`{b37g~F96LX^_(u*hx(x<6;?POV}{C(6H_1^_Jac+)z`fmzYP}` zv)d84Zm~UBha>&iVl_{HHrLXoym2N9^pgt2Wt5Acgj12**CT^7)OqV0@>-RHaU-2k zV%vQ(*W2Y;C08p%HhipWfCh`$aeyaB(O<1R$9UCs!6)x}?(TR~cpe@CO-CaSzYsXl zg62bzX*nH)FKb2NM^8$fV+^$A>VD}17Feg*_%hVK8@%(HXKXoC3p(k3eU1?sO`lW1 zJbcye^(oZw3wY0eE@;Fkrf4RB!4(a)27hVSAG2_<59=yKfJT9VD;#zurd-Ap1Z+Rjr#LT_A1mO!8#-MMov%(li_PNt=K@B;UP%Ag zjqyia{>60cmm5jelgC%gQX^hM>M=jrC2B zR_)D6OSIU~IgIDMw1R$=v(J`iJ7rSJuAqEoP5IgEig{+_wT1Pq7WMfW&K0(>ae@B* z=-_f1`JLYI(NG29Yt$ZZ1tMo~B~imxa<4oIo{8-Ye85LpoJOEum>!xxdZXq)xZ~S; zpGo=at!V)?z#;(F_RD`#1mBEj4hfwVcuh^JsVlqlVWxUb%jrV@E0Cr~7N~^flvY^gvI758hMa@>lVkAclkEHLbD&gb zCX#-XXHM!|d!^-efiAGcZRdZN4l1AUR`!GtdmN-l~Ki&_2%FyKC{GmrQg1l zFa_Z6O&wOM{@S9&X_IiIAC3fD!N*@DMSWbR!VjECNJDJy+RB-;C^VjB_B*Alx!YO; z6caPjWM(`25p(q%KWT$6(IfNQ<3mFtPs=Vbl+E*u@jtBP1pWcOPM(q+;g%YI{%aa` z8@9T2KB|-IL_+N9uJ`vcw+u%Y*2MO8AM3TNDkxZ}P@-m3W_!2a`YjKaStzH%OVXo} zs+vPXqpxMyPlIHGMoQFsFA%N~TkU?P1x3qWL9}3jo0;E$kHM~rp72|=bhK=&pKL!`S!+M(hx0iC@BQ$_mn0q1}f}tWxq>y+I*SM18pgPJG8|M zaTeMe`{-FyW`R)cVS`i6`w8!C1<+Rmz;idK^r)idZjZ`1E}Xtz-|!=d245B4t3p?2 zMhn#qspguFm&9(FVH{E8=4tniN{=niW$pj5HCu!Bv}G4jPBD>tBzR$2%`|E{CnmM9u+PLopg0>Pj{2y5^g0k6MZ-Q?T<9x;i|%S_}feievhB zBwdf4@+MYBluwB5UJmul zYboInMlS@8z9EYt0&+0f@ijbQ;w}H!yV_crdU-Pzd&~74Mi1aXeQLQp((97xwjYeK zHu%pvns4V=#SUyl^FVfdmfvhqZS0hi)s%+D5?X+mG4QR23)`Xk1yqVih|?>eTyn2& zV=IZ|+f`y&4@g2keEO2s4d=A*=Iuf;krwTl-@hbZx-u`9$hm!Z@*9lztb@)+?Scj= zqAx9nzln`DEgW?u@;vXm7k;Sn3eknW3g(4@BEAKGw_DT|6@k@3DIF5Ik%R4z8f$bt zv+$+i_tPlV$M}bxxctU5_-cmvjq@n$+Y(;K3=ebY)unUjO{F_5AOzUeoK;jF_nyr- z=V5jKQULwcgLO%?vh%~qT5U?%Ta;<-Q|MtLcEncN@=C=%r&IhCyC%8*DTUvc~xlmbZ*iG(uJBM*Vy>ESDKQTEyaupiwU6q8Q5 zq63G>fo?Og@>JS!( z!s?6NnIwq3!#%^pE%pOVQ~&mv&!7~hM4J6nkGa(^=a;-Vv~89!M?zBfiD{oa_O0wJ z`nSx|j~C_#XyEVa=tmR7g}7b$1DHgc{Pm<h(?-H z8m_!LP8)EOR(b;RCR#er$I&R7euZ;;HlGdzVal5#?Jcwn?onbHp|JoSzG|6*8ypTre0kw|w{%vGlU{Nz8#>#93Id0*#o{vrSK z-whe@J3xn5oBqs4f%)5ah@HSrnmh{fk0$-p?nq$~a&R^R*j{D+I#_tftVkt(<5#XI*nT_ay z-6aec-z8uvP#Umk_X+!l?f&)?%nLa3m{+f0d5JNoahr@kXZrab-c<=4fiwJX)tcY3 zocJYg^6s=#mdb28DV{yi)D)y7Qxl@{N6r};!t{L^OuK%O9bm$k@LGqN*AL}JowRKvB-IhKWL?V% zlK5wgLd0`#1s2TxB3X;a;O=E43pXdKd{1%WzBr+>qC!!_);7KhP@msmw@%QTa;e6j z!dB=1aQ+i=U>yBm+P z8y;Fq`|<2J`iNH?gHP092JiUgJqDcAG@kB-6*tBZf9I9#bg1qYV)|g-(y8?$PvHnB z4({Bgfb_TetPG~@zdCyP37csi4yzx&-O>}ccia6t-!`$aB;2PaFDii#eVNc?zGRX4!mqNH4Y zxX%qnN+NYGjV=1I{MO)vzo+(Q7NP$}#ThN(>`P{pz2uuAI}%q9TL1X7Y76)E8TXWr z3E2szS{P2VJV*$v4zH;9$V}#_^bq-(S6#dwpVGei>ZNULKm2FD>3;wSq^u>%>U)lY zG2qj`P%=~ybX`s&7@Ga@`m>)c{fosoLwn?q(iu;#@Xq&w=9}F33*!FkPvU7H%wZNB z&he{s{xcS6`vYf)>U`SY&7-RLnCtplPyJAxVT+K-AjIfzOkQk;Y+C{nYQW{V?|-_7 z{~&xKn3l)ud;H792>iwwI{&(^AHNAW3X?eX@C?o`Y{`#*{rcuM1I(Pvs!x7vVE(ty ze*M#rA13|)WI%0j_vH6-;up5$+emTEK${o#J-hh7H1~V|6!_tVn_geQ-u7?r_@xPn zM*TUk7hY7#~u;SOX>rbZ$Q03`2VwobMWk_rrf+y8rqU zgaP22W`-uq|Ai^3!TC<RP!L2Whl{oS-K;gLcn~_{}|BJx@VF&!Qm~`j2|9b%=9=DH?m&mA# zoPXV}U;f4j3d9sd%pK?FtnzE)EHZ-C>`yg4|6ioT03i^%yI0Tu4PX8AKz}?(BQda= zMLxq@KbQbNImW*quU`u=-+;oa-~QFVzJD0lLSQur0j`Jt3-fJanD74=C0IQ2|20t} zZof;^_l0d>PA7zydzs%j6(+HMPc;GdFtNi(&pN6xH<$6(Rz0C9Ul-$&J+k0*J04qb(xb1|JtxbL*) zhZ@{dCYcA$F`fG`=1S)p3hTld(R)?0_qo->mz!O#BrMo`vrB-dBV#BH<(a?X^6xlQ^30 z?S>N159021#Z*_hy=WBTeY^6(w9?Kl<^E3SU1!R%^T7->k6Zt}Ncen85KM?&j`;W!F15itb*xT*B!#>hky^F1Yp~*o3>$5)La>9@$nproHNOgm1RQ@Q&cb6TU+vRRedaN&o{p?85{H3zX*nOn%`<9QM3G?N*^d<7|U5{1iTDwi?Yzv|#VIR9q;tykatn!uCwjk#e>5eg&WH4~5X8A^`LZVFp=UkALenQ>_jcx5a7J%Ih|t|i&-WUGAC%IbYQK&d%{=T`WXDi=%L=U zm^&U?gzCU-GlCg2=BIOU&S_CS$(5jeY}|h%qdgZv_J%L^wtnH&W1+XZ1Zi-;303zQYt; zt-Vlu^Rb>}bEyvFJ=*Q3d-~0nO|v3u)G}_#XyS~MCEI3%KtD4$IXg!TAP!zv*L86K zg1Sva&KLYa8wobLFRJr*j*mFws%Z_q`{vOz7dzb;349E}0;AXE&d%Q;^fAl`sW)#9 zxg15nt)(Mu+N}oJ+na%jtAfu3gC5v{9~uB2_TE=B0&UBEQ>Tlkho*K6^*1YAhQQc|toB!zrKx~vLo;w9EP;S=EP>O%U2I#OeSE*4%Jr5Ja zZ$6uB3+B@tPE&E_Le`%Jf?9vH5+}=p4WBFTWZ<}qQ;^xEU+!8Y-z5O^qUavHl5cnjc-aY<59;2@71#Zk_zJeZR> zAM;!yzkD=$7WAs&xWnWDgpd>ncN3hE>ek9Vo=4ejLVDR2pZiu^M1dML?30jb(}0-( zA=5v&{gDEub}YS1ZBkfL#~HApUu$azZp7s~i)wOpllG>029jd99)$PH0CkMIX5w?p z#Px$wLcfiY{$-S0@dDF=y(sLB#^{GrpXwW79{5qV-KruhExw0W=e)W-&NQ1zF_K>v zA2|cVxYHxO22PitK)+*Ovi}!Nx|C4Eb+Mo^gvR=V!il@?vXx2N6J86ZkhVx!A;x1pFM_nnn^`|)Nd{&5|6fx}`8kkA9HjnK* zmtY3C>Vq`<-~&*DUjd~F@ItASMKb~<9l(@rvJoLuCV}d>d%i^`BOUmyZOJfdx3nnb z4;DJZVS`;&b&#fF6v7szCGxR9V0R)TMPSQnG?Q z^jL5O)}{Ck-a>)?}XV>dB!qGx(q1BZQ+6fU9=DN)7Vy1!ub7na{gR zw~Q+t_u}(>`8dyOo_$~Od3hzC#I!Jm&+|=e(4F-YH~&l@&8%Tn__6_iZt(rFT>3Tp z|9voUn`y`;=>Y8^4x9O4nss=4@KC!>h?gzPfbz{f2l{92%vRSX#L}h&Y=KxHyW)P& z=`yGkFbexjzct5Gb#bWJKGUz~+KE_k%`fygPs?*F?m8}S)SgMLR92*&Rqg8IR|RSS zr?g=c&V+hg|AsPT-+BIpAAkd&XdhyJ-ia>24W`zu&DuJleeW4=1l^b8xRphQ6HTYH zZ`-2b{q-K-I4ht_C=Amc;ou71(CM0+7uB5cIuLv|Gv4K?u`|_J>vGsk@G;1Z=Jj0c zChlKQsrcHlVZk?#pNt!l?(44My^;Ly#*{b(2=|%QNSnU8#5k)`SZDHz;G*7@DdUr2 zPJ@*Txu&6{r6*1+4=JT^-|=~Uc+jG5u_G2}sIaxV@qlmiFBAB0Hvbn(Y!QR6q7AbI z^!(Y%=1TLNN^8h&zp~UQuNkbq5=82LFQ20YqH*WM$F~j_BIA?uuO5_NmA7|hzfoH+ z;Wj%gv{d{^&)Oq7RHaH-uB|7otSs#gr031O+1h)Y&igF|e4%YSE@4W#b zl{e{AjjNPAeuR#?*~sQCv0i)TruOrqL-io;ShJ&Bk#q;#4rhDtUolq}u~gXGrZh*?}`h%{8Z^;%YeD-O3`WmU7N{Ji>=u4XCEk-c+NAbi`vvf_zt{p z+s%6LMf;!p3Xm6?S$0iceGeIbMbKQBa%D}ra$MC)-LRJCaN&cg`BwK^7nj4v zA$#P`l0&~-3~R`3u?40ZZg+E{@2S|;k|^(;wjZ+X@$=f4`#-#hQ!Ylinv)5Yps(bIX)z7jD?7BvgJKjsYjLMZzNfmwiP^3}0TU|J1NEy*y z-LqJ}5Mn3-HhA}wt0`xdnVRb@$xnig=}*>X-tQfNi+j5wEYtqTZrbtU zkjv8Z+aBjw8RYzk+hUry?m6r}^ok~}q~rjs|>sIkq^+wW$Tsv@k8*;Lx-oaoUh z%XkbIrmC%v>`CUcv4{@ldW@cNTozNwQkmH10 zKYMY<1E64^5y8nwrd95^mN9{hc6I|%DN(MRqkw{_rQa=~(kq<@X-tgq=(4H`wp*h^ zvMsf*-YQ;LbuCbWJHE{Qz9ll8n>+E#bzlQ4DgRzHTD+~%8Pbnihoe!_BPglb>bKY}dA|9PLIi^cI zItYG?er=cR#15*Ngk_B-75il=%xfr}KqNvxj;3A!UGcHwiGrsNf+NlsZ4N$2&&GGc zfdUL?2hUA*1Zu-zpy$a=bB5(>?CyM+CcOQfUl|~`5{>TDJ0arLGhDH?<&DZxVe}ws zA17Nos$^Xk>D0xn9NPs+(i;>%xpWUp+NP&Q|FX09;^_10d$S=BV|)=Zo$;#3`Uv$3&2UHQo&h=ium~dl<56^XVGv zFa>gD*pHLWReM9jc`$$w2}o{R#Vd}*11qq@?kziv`_Na=?=hU|9GvWsMxUzm&))d& z{ipB)re+B%Ywzkk735@p33=jOq*AK?6uL;%by2|I6bo>%Jk?rnBws0Yb*%3Fn;u{n zeA)xuO0=$Dd5o*JtRL%cpc|ig@`d5G8*0`reP?}+B08^;nz6*J@bcdH%a*&ppTmCw z?r;BSjZNUr$z*GN5V0zNG55CWxUI(Sjay+X$d)u<1pINJx}a+ zHECy^)i1TItUd}5KUDqu+rdoKDPc>u!Hfi<89HJDL&i(k_Bb{wAGclHA|AD#?^qBr zybISvdt=?5bJ|xFy+U-)+^pPA0pV-)QdQ;VGpI5 zKyR+Mvag36`tN6}WbhMm(=O3S;a>^~JTkf60R*$g_DI2#(XRfqj{$#+ENx9;*UnNCRkM<0vH$QEV zPULYkJB;m*BPc)e?LWrE#uLtu^d9M8aL-}qYP84J;#o7aMFLbI{@Srr{$%FXi`y{aIa zCiJXW(PQ{c5+7)kz@37c8!(xxq%9eIeFi z3v3i~Sb_b{v}q%trYsb*Yn=D7L=j|@{BpBX>?tIzc>7Vq$^$B-h@h&Rg|FruJC(TN zgLvqK(a6Yf=vaKam7P~AzZnVtS z{HwTZ#036PhHx-i7rGE3k1X-~%-+Q&HJVghJ|-8)60?}^aE`X>>0#U!%=v`qibf** z)-ippGkDx2)uxx~A7LsA*h5-JFGTGXw9tHyBwK;r$mBUn|7j1JL@__Ny?Cq>NgJxY z9!wNG>#J&Ou~_KlZRa(>r}4!TOMF4bB4Y;pmbeG#7ddn~PvsQOO*ggK=n0%(KAyJt zFIW2gFd8KdPxgMeHoU?xb@H~A@=Z=qrNcylz>c;_g`+AscoHGwPo+5@(GK@@QAWtJ z5HFTYMOUclSmDer+^Uj@o38m>+(VhA?o#v6F0%rYN(M35u-KjLQXadY8?3Z8T4r8h z<01=xgJ2MlXE~zvlL;wPs{>FJuo9h8(}E&01j|pAS>Lu|Pd>;03mdRz-$oYt9f%N?FbHo`?t8&Wifo7^VK68U0-= zzn57uTnxCdnfZd(l6qFM#W#Y2P}BY0Q>ISdU6G!lRdsN%P8LWys@1_vy`nUI&kJD8 z!+QM61$IjFSE{2KipZ)$UAt0O-d>46{H<|S24^{Q8niCmFx8|_jn?q z;SJf&dMWVvQxru%>)(JAf_tAIx0ePka$?YV*Qz`>@k<1kRsoLpM0(|FBKQ4F zm8tmmA8i<2&QB<(t}fd_Dkj<#WtAW{%c6;6-!_{43w7TU_d)st)ZKu6m9{1*?NjkN z>6tYg)Z0j&-4sCu2WbR2Vu;7AnabEj%ckozD>B4THBF6ON#OKl>()af=;v&=NEox z=f5Acm{C^Xvl+OkW!r*|7Avb-ksh|P9iCehB1@Kxo}c>oc!%z$$o(0E?5d6IAWmEi zW;e1*gnn&U(JPwAB3e)@Vxa%plB;FB1fy;@N&U+C;}v7}m{dVi25YCzwJfCZ+t=HN zNF@KnuzY1nn7#;VYc-X0HNJwwvq!*#+_OuBrVwqyQd2=fBJa}Uu4XD99k8)}VM=7> zIms2bp)BnlF~L}QZnl7XK1|0e$H1ki5O=KD)tnkHI_=jTC(i#g42@DHvlFK*Gj8eW z+AqIRBv%&#bvhoYL0s?Olx) z^rNu+&c5`2R7)5?G1$UXUnNGUW0;Fm^xT*|boxxZ5xRIv0`GovyK$*Wy9ct}BS;!U z@Awj|B|7ceIj1c?J=D>ITB_THh})xD#aYa~J5DhDJn@Oo2#~5CYnquo#ZjV(mK&7RsHL>ibXP<$OEs6}3$+-eFdjA8+?(|7*kRj#v+Yo=&-h=X zmLq9@@EpjWfUsx&s(neo7EzdA{D)k3Fp4g5Lj$z&TaC*&6|nDKXT&q|u)ccSJ$n?bb-A0hJ3B$;VHiny><*UlmB z>U~3Sr7nG;WO>xxT{@8U+K}MMQQ+65<37~&}I^{T3GioV}REZlBLio19wiEO9GK2NTDZfB)M!rXi<0{iL0 zt}MvW=|&Yk@hG%>s?8IPZ$}T7#YCB|Pe@~Fwly8CtRPyu=F3sFzAHFDr}HL z`7Lugb~6#=QW8Qbc)a%OYz9}iow70B%HZgq*5I3a8ayVr1Ol$^3YKSZ`zZ7J62ftq zKS$Y8e|I{#DvUSMV~uT22Qm>+#Cpj+RLc0_qpjJskKPFm>nB+ebXQ83KY%|ujS0>> zm_l}}iiTYWQS#XECB`!9!okYXI1aRYxAW+;RMiY&19a5*kj)G(-haXDPDIzt_L1zP zxa2Xjqv(lBPPvl$Q~%x2f8Q(U)e_wn?Ss_H8*X24U>^JMa#TMa9H>-N1PH(Z={qYh zgIlflQsll!&K9=!FAvG9D!>^rqMlFBtUe}30FoF!yz z17f>SivBQRPE~SQPO8)Qd!0f5iSv3PL?FRt%hlQervWmzYo$-~ZC%%LA?F7n@6Q9h zE3n-O&!7t_564{bxcIH{3b(6h2(9pC|7KzAfYocADyaT){^x1FnyvMlXT^JRJ{(2A z>iFes=+C-pqXRQxGxN^&is>LN5;GO5BetyE=`)1c@(Zn;uM?Tj5?X2e924AVy_WJN zu%dQU55J{|TOO)1e$?-Bfs~P!_lkq_cpHamw%AiM$l6>WHvQO|&&{2Y(8V$=dyOT%6pzJ3M&;Ac%^?fjXvmfjTK^|*t zHFW*4su5CmVX;3@)M2Sg`m}%CbRrV#-&Y7Ym#+A&$r8MW&lNYd6K%ozzBAnGWV_vG zal|%X`rRI1050cxag0^oT$HuMZhha~xMBJpVOdL9<$|VZ@BZmq{{*Bz23K>Q1b76L zSaW*HXsFi_xD9M=1hcn}+s;YWK?igR&1+0PqK(Jg9u-{0aAR?{P3)$U(^*Ir^wD-Q zaz-dPD%-elR9ZX?SUbVNvK=V0Co zENb-HBI|ZPTMOcMR$k9#3a<+Ut+~2{_e*Peqn4U1Mn)Ftq4DD+K8kLiox`@nnLF0E zaa>Zh6SORH4*9D;x$GpVrwXX&q^}kzj>ZFA&M}qKPzvlNU~@ZP%V}{>pluj0A;D8* z(uDN{3p;cs#ep56yaIOH9wgZtOCMffAj7LYJ-C&WYcn_=esfXV`_`S@tf*|c41Wg0 z#txS)ObdFT1a(&*%iW8qB{SWw^YF>q(E@$EyL|5rNDu8ci$O3r)Pfc`aM?<2F5O^X1N3RBgD(T*qsa9Uz#kck@CxtLT%Q_G{%=Z=rPDv6FhLAr75bE6m68!JQ03#Zg0frWDRO!PT1~c0hCC;^U|U4XGtykB7aMpi$SD>?37Do8nl_OPQGQ|qu(7@7}5Z_PEZ06ovzg+mCgfG z0$6EMnDagQzmO}8_XY@FLiibTo_kN%|26z!r-| zx}z@E?~ms~d)jnxn%`?F!AFt}g@GQ5UvCaa36|PMxjtB%v5ZQ)&IdH^8YH)$@eV45 zDkNWQRe&wh@*JNqxp~w`6MhDuk%wF;5QMWJb+aka|8k+7) zM7DW(sE`E}sQzbyS;2ibp_c=Q@&iA-#A>BUKq7bLXb-NL66cbgawo||zQ~31J++c9V8~yfv^D;BH<+Jq+ES{e*NL%fywLF&X2LtcYp=n`VtH2Ie zt&y&Do$h5y;qx#965q%dsP(7o#tZ({^Tgv#AIfotYPHQLdF7epcPD20%Y*vPz|be^ zy3n8cWFlDuSn7&uuEBxAoFxUN zI&Atj4N7?%x>e-&iBUd!wpeg9DKMgbqXzT19NOX%v$?L#Xg|!jI8?Dc^3p0qu)OL? z=WoyxEZe>j@r~PqzRKaxCjlM4OOUO;8EA0>VixK@pFwzzC16tL8#TVYCmE4|eaTc1 zIIKL#>ZW5%KLciGQu6_l8@=mCs?vitY468gft8m;pg%*NYPaQ*!V>p~URhgtC8T5)+`Pu51I1(f<2NW3C7k2jn4Fz^vgXc< zmS?w25PKsWQ^1(#VyQa&Gm<1QyLZ$pmJF%^I>*rhRPw=XYsP*iq*e@t%(>vK& z+L*F4?CBjaE5yXU;f|4fvuC&wNZCP3bvxWsR`J!OBHTuYRbAIBw)>JyMbC!227Wt6 z9_B6iM(e)7Y>^H7=zAk1kvE{M3mc)em}F*!i)wj(4g1U!eutbUCXS|JRp|KC$B0eD(zM#tx*H0J%3Ol7 zjXanM##C-Koh2YFWwz*^Wjtkv-VR+#JsPvv@CF`K-ASZVIfl5)%&-K&5YGvEnmdlPr zJ6YXxW_$!vM3JHuqegQ$yWQJBIzqkQw z0MML2{fnqge)NdC-+NXe)seK~Tmlv4{SpK03vf3=v+Ba{o7)ubF>&Q@)u3GY6Y!`w znj=75kRTK?sqtp4)>_T7ptLtEy!svRZWI0k^+=-xvjEez!Cd2kT_+kh36Gd$&M5~_ zHleL`sg)OY-I7l%Xt+B`s@t^>tZ*B!p8T#nQK`lRPsUAbFYilBg;n7;!0XvDeFK2k zv?UQuiEwS$s*L{ywz|kcN?y|`Z#p_^^8SO=bB(02t15x&AoBxu3}|6is`681wo^^G zk$?|1%-Dr3eL0Ht%wEH)7Wg`xv{oIN;LxGpHCo_)In|U2uanG@`($_(_J-iegh^q* zKF=Fyz&rpEo0w}XupqRU&EuDA64uLsLgTX(q>LxtvI8DoMK0(}%Dvb|Kxt-*3f?P( zhe_@|lg15#%@Q^=cA19m;|8o7+ZYv`K7mB~5?+U0x8jS;n_Yifpqt_N>d6#7olN?( zPKy@M|sFBYQ=VGfCzT!MutyTjBGhDCY^vT1bt4~%GjHc-~pM;1{mgkffy?^Uh z*QIp!{tdlXl2j!TTnC|zC1F~FT8W07bK?yr%9Ob3YjDjhgO3`325o3_A=@|!)2LI} z)jC|Z0jK9Uxfmh&U{6EV?^_O~5v4VBnGPRNDjl1)N9iXdCqQYMC0}G!gDOlvxby

    4Mes8fxIYZNnDzUJ6+fmz&eplyd^%CK?$omA+ zCx9Ow;O)AtDJ|E$6pAJFQmXSU`3B!QGn_joaFYJQRbWF%y7bGBL;Sldd*9C^SD81@hKyKTO=rck%-U7p)9P7ipB4q#Y1qy1Mc$Ms z*Iy17=6x|%OG?zQxR2awBUtz*O;ww8+lzghK@Mha?IJ)jtJZ$IvJdP*R?+K-*};7x zk0+*Q)u|@@Bh&o08r$js6#dp*h(JP|=+cJhQc&Uv6(w9&N4X@@f*E)vy+JlsdQeh+vi0=N(&TpUm zmVw`pUQI~gO;1*%JYav;3)s26U(y{VAf_jy!F+Q+$`8C~U#=jK!uZCisIl7T!%Q^? z@GklC5-h~cXpzpEBdaF^$VzdQiZAu5OGm|MRvOC$VFZyxJA zSmz5n4agwx3~Slp!9q(pt@S2;td@4$qwk<^OIRfk<<20hNx#G8@6R_jkpRwSsWQ;p z4$09!0GtV6236;t+8$Q)HCZf)oLTBhUp z*TSfA5j_5d-S<>c(vYzIMs)|b1H=neOeHS@FeE%X>&1c3ZXgu>kb7N&avuag3dMa* z0HRzohd@A7iO{vLeX+ooW2;vq^IgbPbHwbnR!*`_^W=9rQ^fXg+& zxX@7&(;d394$C6uE!eXMYE^HV4?1?uo>3T4fBmPct$)xaR4c%er~oSI&s`Dt8e35S zvJ81Z<{c2VUm?Q%j4T;HwLJmWuCKztRE6Pyok7c8M&U z4?Qy@z9q|HU3P?pQ>Byv>_;`aJ}`~imwaOefY|WUaKA$< zrer_SEc>3-TZaU$99TVry~R`mt$FW%1l9i#u_%{<4PcLN>+&9ONKb=cr9-3laJ!rK z*wcCi7Fs2E-Z9p5-9wB_lzxG>ufFLCQ9Cp8z%p&XKih)s0SNvoc;MZBfhiHjDH&Wl z`5iTz_Np%R$)??9<&m|$gJ-IXsVo?mN#-5Y#PhzCcNxO-y}|gHU2dyXHSMSAn7{Qn zQYJr}s(&59EApn-fMVe(qR{19libC^@LSs&xn{ce@+e>ZbvQh{xAu{;^;o1p5jVja z9L#1pprX3xowmjdz6J++4zQu+Q0<>7mPcF;5U;<+buY?V*PD)&iua(>K@Q$qK&17a z{2!wD3jckq{jqfMtw{M!-qYB#RY{T$zh1L^mJL{Lx%h_kzdZDvpJE>pNbQ*yehz=z zoBN+Dy#iSGx#3@?exPgBpXUk-2i0i;6N6r^G40s@UThbjE1C>Co`=^JeiGlkcjYz> z6MWUQJPv?JTYbZ+?~wBM%-SE!B&s}KY)tMxR{QJqtgoupK@FJXg%j#Ov3KDfH~=JM zE=No&z##57R$C+Wdl%S{9}z4Nw;ESGCIpN+@cOqp46SNv0Mwu9OKBNOoYfvQ`6^79Gk}y8hnc_YBl#`% z|3>!bxB{q!_#I#H|Go>s{&uYNAzDZNjRqdG@>=e`x;a{?CVlo=gCfguQbEduyY~p1m-O!}gBze_kEh{@UO6*Y1ig zgM{8>SlRrsyNjPs$PVMxA7LvOX=u#bp{cPFHgBbd=1L890}Kw`eSyYb+h5I~4~==6 z^A{}CTC{kHHgp1c1#F&%rsll)nhO^Eyp2W_^gC?+$_2k{+IMK-sxz0hHV3ZWfA7i5 zMOzNP`=ooeL$uZ8%I*7$m#ooSyKeoq?MA=u*lB8Zz}(`X<&mSuj@zBEKk0nV#ntWn z1$VEj*RFft@bSG96dV#779R27VRTGv+@r@&Q=X-!r9Xd>k(>7_zkv9? z|Dmd`zM-+H`E$#cPDWRE5A$0ui#y008vZddIyNqznx2tJXJvEppZ$Wy^B>)UKL63P z|IsgK5H#k^pRYMz>u0|-=7s+3_{#YUHtk#Z%b_z`mjhRA-hXe=>cdZ7zWcOzi^*A$ z?v>jeOV(^Pp(7hdq2 z&LI(@Ir%}q1p~j8pQ9&>=Jug}lN~Dbkz*Qwm6^nMB4+fDZrfj!NSVMZ^vCrE1=JtwFv4Q|= z2kwdOG<%#2cc>e-Ck!cNsWNN)i;{KDht)Tt4a673p?7Om;8u#w2ZT+82UMgZb}*Ip zIKXzp_tpWOc%%+ak7t2H@B#<0>)By`&1Xj+4J3M;5yd{5DMqcZFC1N8lK)!E2di*W zYFEp=22qJ?a%?KAWzJlTu4kCZS+St<(O_iqvVOlx{Ejk_W3zILk))}bt8nyoE~{cz zgkgWPrn{@Efa}Y)922?L`tbT9n|km_QPNAEU>==|r~?v!^|(X+1BqLlEe+oda`eKS zxq5f4{Rfj{sfHajM+`VI!HFKta%9Y#3c~8Ks^%E8#eXno{5_n?V50%Zo0#5QN;Lpw8Un=~UEQL&(n zm{bp68+O8d`{UsGgkxf6J#F2MUwm(~3Lb^-d9+OsLkuoJ`^-`H|%M-bbny7(VB+tBF4TGV>IG>!`3dep& zKCaoeocxk~(Kq8-ZTHM7-!*~lurL$pXVnXEqYT7E=d6^K=larC42pCTz+cs{y6vpc zxC((g08?7Yh=a8I_yc0&1ozV9%Z4w#xwOZIi^WFKX1~7K_P#c0u!(cV!MlU(x#cVk z#x>>0e)XP6DfUf^h)Qn2u52NCHi|0kYeEs92uZHAMHTk8W-_}^{Z=wJuR%^?D6p&q zTp`=#Dker#H_7zRbk1P(!Z=5}pJH@J-LDB!3rb!ex-ziJQ2~H|hKQ6$x=PLH@u#6T zYfBNM>NzXGZ4L?q*mXu}AZpJS*){fW7Qq_}s%4!VEj_MT(~AL}H8O|B_2zLBw$8e~ z8zIhN!1?!<@x5cS<||=Q@B!gtw1(_nsSNTG_@!b4S{xY_%>vW8SOYc8+kJe$Pln#3 z;AHJls%iPxkMFl=*yD8`!~4PMswW>U2s&T`Kqgb_ z=xh_|Rq-=Uxf&L4yAJH7>WkE{7@UV#4LdGN9^_~!?L;*cg2+h?>m^Ff#3PS_XuZV_ zsw^@phPfOUC_**hEkv^q{B6xgbM!xmC~>CQOUC{6Cz!6QHM*1`swXr*Bx`D<&Tylw zT5n=n6xs;awAKra6cr(2ZLMXoVh_lMSAeF(M#McpcP8=3;4iXRu3KHU-6zZfUKd_- zOuoH(z`z*Xs8C{{; zQqMAa6ud)uC4-PK#0WaQjJ5**g-~I}oY3XLdp9FjDEElHA1hada{(=o4kDOj&WbQ+ zlo*+(cS})Npa^M@+>q^fr16;ExRTWRYUK0d%z}S`G6-JgOo*2*$=z4>G~k zvgt#z2Y)E9WJ;YngGtN7nvaXU@0vMkH4t>J$*hWRaf?M9fxqGizQ^6J;WfBP)`@+bG0bd2!EuqYJ}zX(tk^oK?1)};#Uh-!*qi5B z`J!0X!x<#S>7SNGa#!3`ZqtiXI&KNOsX0peRxjNyN}i}5fIqzB=bj#KjTKWq0p-HG z_{DypiP)}@_87UU!c*Vz>kMiMQ0K;_MMnS)%RFc33uJJ2v3K;0%B4hU!^LQ$`5`Ps z%oe7gAE|RA#PT<-#`Os0Dm|XzbJ|L!rw=bV25%^<&zHGRyQs2ofd%L=;<*QP!0J5J z8zdH+c-E)?AUU&6ZqwLYM~~wy0{jg0xLS?+%Qi>0T^Tvm4Q{pjqu!_J23kv;_-� zJ=y?)LvD`2&sPSho*`lB+>BNUd!dI;AgDXb2l4_5q`8HL<8@Hf^d<) z8g@e)s}*)^|zJ!G- z(lC%I@{Xo%9eW!_r~?hfj@gt)N>r{dh)UX>*%*=js7sTN`DP+L#{{lmqb=zZyAae{^t6}Y)kd~&4T*2Jh0y|06cVEIt>zT)=14vRe|ODvSVR379REF2H7 zIW|z@@EfD%*sC!X^!iD`G~K&VjsG`p`Y-nmS9knfU#*h^_?2KUh?o;E6;86``V1=Z zkFeO^R5^H4e#DyUTuh&jdwcwwG0d_Whp7B<`EjC`aji77aEwwZLZ8R<(ie1RckR3A z(Ym1{?~RMU^J_-==}c7lnUE*|QR5dpo@5TL^WsvFVmFY1lx4sxjSQPXM3c>O!0-j> zV_wsS6a63Z9ROy+W<|+ip?oqD?)p*fWSlzdz7cR62e!*%-A%i z*skDBp|N^_t2%xJ!&RRkTbX*Un&tbO{;(}M0!ERjhoNw}8+Skrdr6bNfX_xCD>p$f z+@w-xBsl~$;5OoZV=5DTF<{MQ0?1>?CqCyGK8`r#L8(>4TI@hA56?RXfonj_!|6da zjODF{t#Xhys9_`H#IXl+cdEn0PB^H&ks5Y?FGraV0>Pi9rnD!(5J&ko1zhLBC90Nj zD(K+$WI?)C z>aQvQ#HPSaG-uXRrH1K9t4hU#)?;iHH9`$z6v5NqqU4?OO_575EwUE8^O$T__8h3+ zsf0%8WwrFhj2f2Gsd^W#hW$DDTQm1}+gaa!>qA-QrIsP&iLtMkT}6IVic-X7hdIMa zzKu1wr}oXn?E7I=Dt?_I)%dS_({5g-gK+#@)BbQN5^?m-1WOv$0t{5am4))~kgH#! z%_>5s(>OAH7JDpDfjY#KI?JvR=JWxdETDG`{FWzC!=fjns__>}csbXT?c!6_uupcF zr>|(J?945_3=q|HgvNbFT`LuP3ru7n3#dd`7{>A_xFi_wwQ(mU>COu?P=Zj-OXn!} zP)V(!Um`7m7TM=e<$gjvLavYW$$@%n4%vg|3{o5wH~V?X-o+ypCA`e*%Yd2|HH`g? z*}_65R|l!WoryZb6jX&^M7&GtKy!PW2R*D8VQxL-W|EbT2=?kAVX_SUYMM}AdC1tK z*|hlr@{PTmK_r1EW~U;8sIL?jAxezL1jC2Lrk<0tn1;w}h#2^c8OVUkGBA}0iEI=} z81{J6ye^;bpoVD_P}Q(LHlDNR-UN?bNx@F`Ail7wM+HH#kV*cELqM;IBz-4a1XK#`!7t)<)#Ko&0l=QHa=0~{~r z(=R85!$ZRc#mo3-*qun_V>Uh;UHSI59c9o04~+ukN=N-v1xT?e?IsokbZhpRk>0T9 z?7-z9Aq+Z;r-m(?Q`IH+T5&^F%YcecYFITx_0A5QecZ2G)EbUosfKNnhhto?`2iS^ zp26&F8HakfG*d(YV+brSXjY)4O-d2oI=KL8(45F_dX>@5AFHhbcM0_*DqJnnJ`5|c zQP|3ka5)p*iX*t+M4?I0z$d~R`xnae#okS}Yr1LNcWRhl;t|EJ2K+jaPMlfEsxyk^ zh2lEJDXL-(@tWTqspH|Msiu)HQ&{6Y#(iA>+({a#nLb47R2&6qB{al))smvWNzebu z;MJXf+daOZ`J?7c02Tvm%j7kKMRZ-)U|O^rSW8#K?7cil=OS+ng({I6Nee9mId*sM z*U{1jdop}RI-;a(W>m_J+2#n9jfpI1#Gq=#W}$7s>uNJL(@*Y+k2H44)Bxzid?A}Q zgMMxf@aal5Y@nER#-?T#Bn{6>BOBl#WKJUnAC&Jr_eLQ}o}l}lTK~x*CnRBx4+1C0 z9(4B6NYAz9Ku)%bCjZtJ^r;W0hLH$vR7GuWI9sl?qIJ%`r4cD-_sK40{c=Y@NVf=h zUY*YlNQayfss__7MU6z3Jf&pU^3gky%|;{&kIzX-8Ld4NA*E&rNc{v(!0g)`c1VcF zETgE0KYTT%#~C0@gFtFfBfo&wLv`asdQ_3~;~-+1Gol=^;L*0mY#mrK_MnHu9GOFS zu=_Mc5&^iuyL2&ZcHjVO?7^6v){12cwN$`~Gin%r3xQPn8}GyXY&Gn_66PtxG>|qB z6orkXcF>2}C_7>szr+1|;jZal{^Pcv`T!3+hJ=27fIkx2!^R5Dhz=z;$!-Y`h_*_e zO;m(8+_gRQ>R^=@2~7*-P_wa!vv8tP-v_7-N&bU2lGImqFZtBfd0Jb4zu;DY@TL&I zE7UO0lFQ5CC_ubHBg^|c;bX>Fe7B#UL;%zxseQq3wLySB2fB1?ktC|%xlp%?vZv)^ zwSrk{6@QrM*ts2@L#@Gktz9P{8cOM`+BgBdM^7agHA#!S(INnWZEa$c)Enh|u8?AXQ1l zJbhNoP>v8VwT#N+`}iFZFKGc)>`A^1nMtXE{3cl=-;^Z{(YK-@xY@WIpNt`?yt=~Jk`4U z>Hb{1_4B4qOcbv#K4^7_&Cm*}=m9Cf4>sFlhRo98lT-M?m3^+&Xv=9_2UdXv8!4w7 zZ770B1p+l+y!9(et63Pm>X_;NQrU3KdFA=B-A)Ol4Ue7$UHWqJ3FT{&{wg)>x9*v~ z$o1(nUk2NAbkgl#m34iVf!4iVNVMZQs3k&2{y12%1lO zziqcON*)|f($md5f71?JR46O;M2Zk*9a`2v4ibmD(oHqOnSI4A)@w8l9n`xwgNZR2 ztJ*`>YVh}H%z-m-iw`6Y+V9>Z`cbSGFGO%f(_>Z2s?zi|@vrS(>pPy2XwQF_#w`_e zdTv@=mGwBQ&w)HK=Rw%tce}TOcfCJjt8`a^)RVpm0$&{}$VNvekk{|ufozn?#UfAR z&gvX&zI~8A=J~b$iZfFO!|Xi=nY33iTS@-WMpk}lMLnhu$=&;tOA8Y-rM;>If!8k= zK_AtXjMDE_oEEyr#4cL?Fr z--7rLO%?+yYWGG&1W{(7z{8!@0Ce~=AcbDJtH25iAgA4Xl}gEn1PY543RcY7IqtU6 znHy2Vx*w0lJMLkJDg{$2|J(NeY*d6mGZ~&_1w$d|mqJmUtE9c)QN+QA_$sUlt*S10 zMdPY4r4TYM>2GdT!zvORn-kq2mY$Ss-GNaC)^xoTw>7>>HfV|$8Px%os$}HX>n`sy zE)UCPpZ$to?ZZ5?H;3*jd*WSuc%5nS1HG5gvm5x;?vG~?JvG7!NGH+RT}1YGv9ju0 zwI{xq*=AQd8+Ef{s$7B5#y7|l>=pa*ZKB393$9F86tnfwyobpxBCXiGsZ^~mRQK2L zJxl(8?Tq>&-f^X?+&U^Jes7aqZC0R}GoP9bS^DzkA*Zsi@1EI$goY9{#ib~S*^1Ao zc5fcjdmyonqu#uVcxmCqfO#$ZHt*o@p(eMP-NaQQ1-rMHKh+mRu5hfTMIIU3>k>k9 z2v)LCbSE@rwlclx022eD$oszdru2uri5`N;VXjppAsn*-d0oq>)MBJm=Y8K}K?_e1 zmh{jVkBbWHovSM9!|+0>6uWPY=oOw_dK|9_q3rzPekhB=Y2;N}-ZGik*A=vhb{T9* zpE9;4I0{)z3dT0cQlTBhMEzib+7xV6^A|!TD(NZk5WfOd0Z*3}s~(Ohper=f1T=;9 zckd0q{1(5}d;ECkuARLe4_6Ig8ec9Wut|%0e}?r^!Pgh3eOZ2Ox=c)zvBHlr{6T_I z?w&8a1qcVlYy*pR(zWB3D_CmS)z-N+IjDkJfghVhuv;%(GgeAyBPsO)IKj!Oz)ga| zR!D49Zp;av1AJ-UoNxhJpp6>lFYNs>kaf#`-tH%a+s7BZDH;w~fl39|JUjqKh!Sa2 z0ZfL2XUfiH9**I;ReWJS%|8N1u`Plu>u&a>j!KEP@*-;5)*ILz*2he|05&sPX8Khn zk-^iWJak>38DowN4Y`0l`EGzv5t`uUEJEFTbF?u5GD*(F)dNDUZGfNfUT4r+qlZJ@ z-$#7h2mYKIIim5Xh!=2TS9NHHYo7<6!tP$zoy%J|QCiYbJ)WN0Q)}x+PNt@ltOMzI zY9B;stu--v%V?x(37_`G+OsBySy?;Fp!-uoXYm359RwxAiIA%ao)vK92p1dwaT;k{ zC1)8)W~E9wzD$0l>gd0yR>RfZ{|`OQ2{mjB5rf+chDi+k=FZxfavTsgrcBBJ*jjFT zPYrXB&q)Fltl4KACUHY>`>k8YQBKs4!4N2NX3|KljLtqQd+oVo@ff{-tl25&RiNkI z2sV>6tn#D%rU&_7LaWjWLcILeDdh~s(|Yn~&m$#`#Q*W;*|QCBHOy_=Rwl>0FtqZa zgTrHyCJ4TqJ*9ZWFVI2R@kf}TPV1$4hwq<{cA5XQVS(m`rzn_4%huN)`Hex1quEqU zw;aeIcS3N)I2AHSWHbHvcf7kQ=bZ95(>hAwfPP?1YG@Hy8;er@T<3wjhu5m^x-6oW z+<*^j4#vg_l@4MH^pT;KD6T*7vC>>@-ET-)8R3E{KPOT|BJm}uLUhOvM^9v5b6V<$ ziY#CIZGS%5;Yo>_ad5f8i9-1G){IvG)e891;Q&wHr7U$se)H$E%F$ zgt09D$?Q_)CDFo^sB3qKVGed>Iqj2-GvfIs?2Byrsz$S#j#t0`nUHgMg(p?dxZXRZ z%_QB~sI8Cz_Wn71_|?=z&}G*LP2J~;&Sn+<>XbFWvvB(5*~P2*juy6eL~>J;#a`vw z1A#?I*4kd@%=#;*-%meDm$-4k}NomYdVep4yVMP^aIvMy?L75Tmv4`|EaDy7w5WAHI7O=$%p%+_vE=1nGipjSC zKu;AA!c1>h!^m0!f=J6l=Ep5Iw&LmB74F%j%0*r^c0b{-wDUH7bQ#hrSgix4wMCv~ zyC24xs!SZKyY{?7?sn~&`XXC-p#Rj({jtY4y!dhGjOq@;(V1Qrk!Z)4VSUWkdTn^N zqfWBZvGI7S#g|vZYmeWK3HIqc<9KB+X|^Khe3li^R6uP*Kk92e`qZDix_1VpG&i3Z zX;z}?u`dK_n66DS9M!)9x$40uLE3~g9FqOm->t^Lw?h!g?NWpbX1~+Cj{$!=L0vkJ z#mp%=c*@MXX^P;G!dt1)1Fo0(wmQk`2drA-fE&rZ+OALaft9^&o6m;<(lXgDl5hIW z_)VLaAx|*^B-({Kt-9ozGU|LsN{T*ehji~y`_%em{)^kbIgu${zDP{{j<%+?EiH$$ z5NBU~Kj-6|P~`af_??*G%g2s+tc(sVw!7(MK;t>PRruDX1UQWy+s+<#vg<&KzH`b6 zaB-xqp*KZ&PI}QWOUE4U6h0uH%`ES|s)o(jaD~bf`yl~lJh*07A)Fj7{L1EAATcGE zD>)!w&MBWRI|!`AcLNJ;xANgyI9;yJqy80QqqtxK>w)yxn)Q26qJ};20Ba$Z)48^X-)-0d zhN;3O0uIqIUYV~I%wSAYt2o1@ZJa*2p`)qu^y|$j2W)phiR7aGu#<|@Z=Y~!B6?lo{vMknJb1#!oc&L~eP_u| zUK0Va9csbJM9Muq?8FYHsDLGjv-(4jZECw9LvRoV7idFB!n^k>kyfipnm+`+qCYw1ajsna z0_8R|Z=3Z3uIyXXi2k#}rNvjEiR0%Nd7F?2>U1N!6TjcG3bI~)V8$kPgX&Mh3*hjVzyrhGv)!a^ z*rzkt{2A}gOGSdvNQmr7%ceV7oYH1v#hPiw2CxMJf_XP&s|ZBpPRxEPW)s| zx>CntgLjJLqkJf9h-cPeK%o(oIu609inv=4Lij__95A;rcv7zj;SDhgIbY@D2O~4| znAj#xuy9sbT4|EQ2iJ3w(Qnd(zXFuQnTz$pIxb}jQ$5k<_@g9(k*#S+G6PXZUD1XX;SMJoakPA zGVyHmZcABiE!iI!o)|3A%A=JLZ%lsh-dWf`=-#L5$HcmkSSp<&fL{6AET1VF=_y#= z4xx7vL6TP;h#Zjj(h!cO)l=oj7o2a^Qo1UA2KAEyl|C56vK$5wu@X-mu6KROfoZ26 z5TUf`#J-ulX}u+n^}V;>y6WMK{6~)FtlM%UW6_0WxKht(x>1I=A=9Ys zrK9({gxBZNJQ5NuE3PFJjdc;t?GL{)(p%@8b+zjcq3+rXgxHRk$!S)5saosvB+840 zZCt3=4d*HbAB zzYBz8;1|5L2noUbOTu1{X0(}@5N*9_ARgWuD-C^56(>K&9m|tN)>Yo}C63+k z42yNEA$J<*c9A67+PfBCt3Fp#T~kp>@wxuto&hsqB%$H(-mD*DUFOy+-!omdv(C7s zcDnUm@U5J+l_Tc^_3+OVNNxq-MWu;);N|uQ>B@9$bZzY+F$y;`qP6V~lpE92%ad1b z89AJ*JZP%uqqvp9O(RLQfTyL@bOyuXNS(#o-@THB*3&AO6SE}Q15DRMkf-+H4!kMu zK(|@8-}&x<+_#-GiAJa1iMKUQ<`s^cGgn;rcE0zJS$E%`?+#&?memCI@O9!6hj;4c ztuFmu=(sgaK+)eyx&5QVm-0@}t<%BtjeNGFSV|_;$S=y*LBZ%_)19oViXgMV5P&&c zK)S|mm+G|0=f;9qAr6W{)hof|XX0QPu+B?KM~EP{kBe2rIE z%H3V1nKduLZ9OVCl7#JrsFqU-z(}bh8tKFgyZ?7;J^#D>{u-Du`mIj9pJjVLljZTFjS+HU~sZ z1wI~oP%cs7p1cey0g0RHoH8XF#To}n_2K4mKe4ArR zpNQ8`Ao-#)Rz&y|k=T?j*5Xk=s9{Suhq`a->w&99g2w0`q8Kr_ta@7Nqg*whZG0Ol zBC2|UZ$TtZTEp|&`P<6_NMo0}!t>p8&k|$No%X5^o;%M8N>urFhS!8kH1`=#+x z`Mb*dEZyr@wmdqt^}g?izT+d?hj-Y2abyWv`W(W;BE1Z5xIokOlhFTd(1sxu^VMO5+=iMH~6^m{@SFlPxbT2 z(Yw!vr`ghvB{$jS^{mZ!5)$g^JT}DgA^dL15xlPsY8-UQ6u9ShG97Qs&atG(6V~v; zC&RY`r{q!ti)O8Zn)fYB_xt@NmjzVuds;lPG_*j_64^R-%LPJ<6c7AKR~peDq&HT& zVXi>RjUU+Lc6Wdc@%c{(VjtZQ)2!2buPl&&YG$Xnr5s_2&DKXw^TaFd{lu2F2mIvT z-;Uimyld{69tVqQ71~1n`=RPX+vnNV9Geg8M!KF%EY>fUq9$^(5-|!|acv1z2=rKZsy0g_BckXu39DyKAHg$- zR-zi|V~tI`>23j9EVR~*Z1A!$oY>#n!s>=3>oI&S%}OO57hbiKsQl-mp!B&!9wNAR zccC3^WAMlCWQLYEUj=wp7qfnwP(%TApk5B8kt%cZB7NqRZ$1g&udqKw9*Wg zVuJBIhz0dW$mz%MwFkEuAm#NEM(3}BES3nB9OwW| z;yEmLF{D=(ioFMonxVD;w^l=fy+>H6jr-cdUNcQLb}))2X@(Hid$ z>6S70)iO7ax4(aS^Ld+OC(`R=+WLOwvp`*Cz6E|Tvjf@Jk$!-%l<&06*7eU&e`yB} za#4u%7ZUFg)g3-(&WpCWK*s^VWGd5BC2JugAm&M?Aw$#}ehXljMCCr2iDGeBHUjY0 zxfepcgt#V3=4OA;|NZG^(e;FV_dZZhFFa(GUi$4^E~Rex!W+LIbHjW5zQlxQob5B8 z_PJ!U9tVZbjVJ{cHtX__EJ@m^laNb$d!XJlvE$7de=FMNEWaIzT{qV!#Db>QOPe z*gqHe^egRJ;&thum6f9jWZa@te13f4^R2EO9=L0&7uTNp7`&Z#ytpRiE{v`#QEoaM zMm_lD*37{8xdv~iXFD3v&ROFwj~@or4)s;IOGoi3F%OH*)iWLkIA^bYmCE*S`qB!R zfu|rQ7V}uN200Y=HHyX<;H7-CpBe90UdjMJLV?%>kn0Z8MCWL87|FD4io!(KPlYCe z+Q6_}GKKrD)c{Y#>xhrwjJ^hObjQ|5Pv$93gUyB5YyRP@+78A_OJruZzr-0l>fM0Y zG3TLiUYwjc^qwC*y<2Os_w%Z`s0h>7+|46rgND~OXVk9U&e=@<{?vW4vvXv-xdu9E zO^N;4EHA_3Lj&C@g=OgrHwVC~CZW6x|!gWgnX^J z_4kv^DQm+@p$eD$lC6|5OZwY@AtnS&HrN1kM$#W4L-IQbkiZQQLmGExEWhd^(nimt zI;y97+I^S1`8MY#$-}~C#`LGPGoJ%q+gSQ^az<)ZR>IqMaUNmrgfOtyco8 zk;`z)3c~W5-<0jD4-V$?jkm>Rp1TL{{8aZ#p}_a}%q?+nQKR73#{4C~?D|(P9RoVk z%^jMq&y@8$wYFBz83W6SZ)CeZ$+hY^S~AQ>s_r{a^Lq61ucm}$H(Yl%9r4e;S-n!DQ+dW=n*ynGzH^>-1>*9OzRp0Pgr?XhUw#pgzfEkBl#3TA1PAmi8 zAjl|GfdrBfY27@^1E!oKuT_M4oG<_ZIcEqwU;5mZz$7Wnc{Nb!1xPcQ8&xGk`6R?Q z4qBBBYWS_EPvj{~{oG57MPA&1ykg%7;zv?+Mp9mpuN!*N*B@O7vNoT#q~%U{RB-rK zM!%=0<5tG^maXOXCgo=Sc4iGV%^T8oRwo30d;ePDn|5;T<0pX>oAg~y2iOj1jQ5mV z!kc&R$%3NWZ_t~OjTx>k*WaY4mgFmfD}Y0|9g5{(-E=rQbmFpNdt27LYpN_z=ktlC zABujR>+LqDD=n;!Ye!AGB^27CeZF{LUVQK$7Z#)|58mvZv9O$edeS+iyl2Xvs&e!6 zRHkKE$SD(r14zvWJbX1$n)S{UZs2x?JP5M%JpjN~ko<<(aqRFd{%^Lz+ z<_|4(E!KV1`Ry?u9BgOF+l=NSM(a8zs66MRs;9_n$Yp#>R9uafVjoD4(26rPaB6n{ zO72@VK&g*HT<9=M+&Qi&395_$Ia9a`2_)A2dZ@8y+U zU&eZh7>=fpP)N*k?Y6N}PL$qnEmI`F1mtrzK=Vd#=KSnAi_N8b=&kZ;Pa8Ln$XB(N z7siMjrlZ`aY}g*nYi1a7K*(1RoKhKDB&98(KuGy+Ts7AxsbnkKd^-6a#xA_gGdAxJ z+Tf{KI)(f!C>IrFi>D1=ac55|Dxg9pH7tPDGu(ZHfTZTkhO5tn8t z5*3o>AlZ6`56OoY1DWf1DNpaTe`lULAXEYFe4uXAw3{IBe|j_0G$dA+enD{x)G!)c zwUZO8G<@Y445g#+A=G|g50N3^ulpAlqGj^R827nLqYfw@;CYc8a_9Vnh}5}`TlabuhA@k>Fjq>OxlA{3bNK?_(v4#LgQ!^jW*3FQa<^`6)T@&;QRci%M4>Fe2!?`#ww(x-8KM% zh?G`24K?jrb@VCo9GYVsj+GsNiYy{n8}X1Mz;Ab;_0k58Rf?%}u3r85*gtQ(BDF@v z`xF1P`@H27Yx3N+i*+xrU=mtQ&NJJ}t1ikr+NaO&sueuHvgK-KeqXa)N#8pcud5S1S=1;Oo*$e@{i)aDdYd}33I_|wQ4gnHUA3iy^tjej!6u6Q{_%E89~Q!uGHludrj}DWPh; zY6U*~^@A#qiXFk1%()8|K|veC5J@zX2E1<2E#_5um4Q@LKaJ4aF{b*8M5`=Sha`Q% z2eW}n&ohz8oMJhJ=ieD}BAtOTqLp-4lB;_q?WT!yGVY-Qe>cZQ5F=t0NGa!1`aL#5 z72>78sqmr+oy!R;v;5J|Z2*!CX!WZt`BhJ(T5J%(8*aBB@E%KY2s#AG1T@`9jwAfb zgiZ#{A3kS7V6}p6dH*4j00f3v?;*w4sVu;(dTbhl&rPpW=@9w7d;d%K_*+%CI)AJ7 z$GS?$)43lcd@NPN7LpxWvk^^}$-##ME-z_eH0)d+7K$163rsr}GswEA`j3ti3wP~hlO=1j zj(f+-D_5-`ZrdHDmwLtTT%Akw`qaGw*SeZhK3975y8K3Tik86UGNn-LNHJHHASZgM z*1-51vCB1#Oqa&*MyWO>2GpLJ=mS39N!piWi@BWRR%raRSQ)KHBuGe{>dB}zOr_9o zPU1J**2YNtUZ(&Lew06HzNk~`;BDk%#?Z^1nuhY6cgZq*E=0Mz`^dxj7uydcGN6=%wwDuemD$;qg<9!XG9Pw61#0TaLEWUv(*2dn zYD09u&|Bxgl!OmlZ~emHY8ffgrZQ?hd3ZR-Sd|Q1Te~y{-WxBCc$hHID8g`SqT=TJ zi#|J@*s^tRyM5CF!1LNcyoaruHBECXlT~+_80_zo4arL%%f|68FRtwS{l`$_;FWH0 zeYcmN!0#MohSjlUgr==rEjuJ)NQ+dzlS#3_61;wxwrj1dg{Rs+eNN`7#)}7_oczp8j_GoW9cCB}PvVq%F#}My}sKLDAv%`F^`ttr=5V zezDA!6|+j`BxyI%Im5Oy+oHXl`7G#cltE^K=zxiV$D?Z}sU~{v=~Ddmli3%K`9AA< zHu@{ozOAt$b2!A`sB=W1!T_;B!8q^NEP0Qo#rVv;alKk zT5(A4bBwLZPp`^qt6@o0MPZn~^QNZ)%mPI&zv?11)!SOJ`Ef zwg!d<2b(f3es|c~*<$i8RG6Q6rT=nbY>(LbUDM#DJOTf9Aak)}@*eIISDEGs+7^0|t?D?>>)s$z&!HNtNU>FLak5a!FK zu0U3MhdyX&w-Q{@ShsWeF5&(f{|i&o-C0nqyq*3-NG6fcX*k-E2MtcHKeC3pDVbQN z8^7A~z`GIBt1$dDxHwp`F${3~TH)_jN1;L_;r`Vd8N?W}@%}H0dU=NXPJqvEhQ^+_Pjllyr%~{#Dl6g}h!GriMLTDWW7R$<>CIhEaY} z%*07Q(9?PDu#F0Sl$P&)+k{)?%04%@ALq0Pw7E~ zmNmT_?W{Qi#qNjw^F;Kx85#xOgIrLdP{Sq!*kdA9TE3T5uxD}>Rjn}VI{c$)@<*rh zHgZD;Ej@eqR{ik7Xcwm)^~;=HbjN}^T{(G7lG)d6LrjFcfCSC9F_OaM&@OZSszs?NI6K+k=l7^Z~S zKRbS$CAXRtM6ysx`Y!5r0sXo0!ESoncK=!Aex6R_P~(*RAVYBzvLl~p@0Uv zB^%#^k@~|8AOX%*_Bwb&h%iMSJpp;Ka2>H7gbyD~O_F?!%4fX0bOq1l?*7_1<=0m6uEMA7*Gh z3^q3(9Aq@wZi#j648WOld;OUg#JZ&=`JOi_&3jm>&nU1BH2@`fVlIjYX!@|sla<;f zxXoh91Vc=h65tiK1fFoIy(s;N2=x&ehgn253~MrQYe4#fZIlC5>5bRZ&vhAfm6RwC znm-KgLtJ>wIc=BpU9*}&y9xAqxyxQ)MIJ|GDiP5|_9^lb0-~%jAym^2fV-cXsx*K% zSj>c(+Eshh4;~rV@lsqvUAr%NB|FUWoY=7`b3{?Qg@2a;Sbcx=!3vr(4dE${E%J{2B0xZ#|^DTU1fad8|z7N!5Hl@ z*U02GQWblmmEb^P&58u2!Eq3nqM)1l`| z*Da6GM*CZC<62iraP-}Ihs~yUQUk|a$?cNY!>#x>N~x@7Y9kDHL((^qkKoGUq-v)*jy$(;2G z2Zy?RUcb5o)ZfBI&B)B4S2*1S>99&PLNq(-5P=cTe{Tc=(9%r9{W;Y@n;GafQQ`eU z4O=S@h{R%D7-S!Cy&Em%j;4^Or zv{=Bq6W4#wAm`VV+Q;wMPZ$T3$T2gN`v?eO{kYD^P}pGnxD z4{3^aEl)?5Ox+08XqRe3X{UwZYFJ0u|Niuw2=$`&eRj%2dlo~KSLwzE?be~7LN;5B z$c3~8+#-xaPV?_W6ZTUMFoTARzQf^Bw#xu+jhHqD6~e_UZREc+|7UB66UHVGOF7ey zK&rxc?f>cFmxAz^z{kkJ#kWupDHx=3rYyby@^w>?rI&=m#c&*itO*zOprKw0jZE%z z9eyLuL5yky)?5s0Z4Nbrd%A%ug-d0jD9nOUIXz|zBc+bTc_dL;xl#zH#n?yWwUYN_ z_$mkE5>>V|=cfCYPKU6@x|+6GoK^igw-Jr7+?nR*w&ZLjvP>tcuYeyVx=tk+CO^nvEiRxvl*qEka zzvX3rhFp7oS$wN7uI-NX(ZIJ2*PXL1XZ+4c(RI%Xn@pT;tF%< zy@p>m28%}GuC8FQa7W+%f2_TCR8!geE{-yaiVzj)B{~*VN=5}iN-_>AB5G6=1cZzQ z5fBh*0Yaikk(yBy6oiO~h?Iy@6hcC06cI5%1cbCfIypy3IFjPsem=MSKEL%{cip?z zUH+x$cAvBN+53H;=XsyE>ek<57Xu8di_^*ig?{0IJ2VejdJ_NPSX|Cp)>}2K|9zpA zwR-d~8^*rBPUannd%|u%r)_`uji!I*XqwN~FO#G9{4H0TZ}pb1e9hl^%q1*9z$zkb zg)iA1m?jdEOWDP(RVCHg!qK;fscSzLs%$^JGbO=hZ`-Pn(Drk`-radhXd0ou>S2gA zNqblmc_ST|vzjRZI7G12gEH`n1n#&#cfs>_EXahyc}`n&&>M~}9AK$GnBG0nePWyv zeIV0`u@)-R zpW!eQBUpom!zi>3;%_0aBpt(wpN9>58XnrC(&h^2=Lz5F%MGux#G zl|lukBcsE*Tb6^H!S2HmGg|Kut#z!oGAMe!6!o2|il=Y9v+ERFj@p=2WX(*@*6c-$ z3>e3_^54(1KMhI`8&Sy8!KoQ?0ock4)|gm4d-#WvPSYffbi3TOD&Y?KS|33?rjVzC~TST>5oj7#s60 zF+7JoQ57PdsCYhOJy5lPl#OLe z@lk|5N{t_4=)+T7l}%~OHzaJhNhk%N!MEYT;Rrxx<<>|RB0q80lUd5AC+PHvVge{p zc=JvBR`rUiZe$B#x~Qbv!u8{4Bv`RBH_~+OQ00Mj4sfpI!tEz_4uW5_qLxWPMH+wyt`KZ@Iy{Y>fNI4z0dES3K|NUd^dbLleK-(YsH(G6;FolKHL1b zLa}Y)X@%6&)2e@ViwN;D0ufRH?GM1}wq=mxW1ncm*<7F>iCacF3y=+K?xIo~P>Wf5 zvli6lsDMdLn@`WVWJk7Vap%QZX_Q~79arGnrlvti;$O-Ppifw3YtyPg^hvxuG&Gr& ziQi6@(2?D?zohKF7`gGhdD|)ea>3G@wWCDguWf%KF1D?`&F8l_XRY0PtZ3C=abL?1 zYv28{&h1+D&~W)|=y~vD{4uLzAt(L`IvIb5z({fn@SccXsHP6$$&9fdO8uw`M+4rs z*JF|2T8)@2PB?Q2_x{^5#e=CdY4i=?|B_z#`j?;dYHcj#sd(P`gJk!7-4Lt$6`7jz zDdN+_9n@Sq&fy`Ml5g@D&FQ3nkPp{wi9l!59S6H+f*2*D&C_qZ_Zq(LE}nWQrGzcv zShs~WO-#rG$|dvcOXAT7VX=yU(Z`~%A|}UhF-B-Nwzs46;k+_#R`}J7tzBH)?ImW> zb7bo;elM1QFfP+ZUdEap#M;JE{>eg0>4XbI8Syt#rVC;A4c_A@A7v!~mW|Hj9*bNAkk5TtO?(NwW61X>G z%(;&q*!uBU=HS~MUJlGdr}d|0xWB(CHVfXfwsv7F#RlH`HhFIGC3q2)U30ZFk@Lmu5zd4f>>=LNvv*Hb3|Ab?+Hoi1pK|B5e?ZpkXAWT!l$n#)}T`KW@@N>xmDxh@WbeirtS>A<@K<}A4s0yGI%MR zRFzsaW z+rJRE7VM6sc7<(U5qyaSa_iN8G?dI%idFPrjS-=5xolgYCE0k8SSSJ3K+Sn>LRQ=b zBuZY(AA`$9X$pgJEhF93HtgEmGj&x-ZA|^xyZSA+di08 zEZ`=L=pd;Sl@z8X=@jG${(orhM9)xb$mb!y{9k+8+AD$JpP2QrBDc(4TgkT(6jc~E z0o))NYp58)X>txDJ9$)1k_j7R%g+ZF-0NV?q}Elpze@x2Vk7|WrdiNOO| zd5_F1K@~+rwCla1vy*OFS+B0|r$NW@17{GN3mnh1fV&>nIkwvC^DW0(V(~omj7kyt zqZ+)aHD5QoeZ&X~Oq%X4A(>Un3&erJUXXn`xKzSG4#rRTHc4LbDV(J<;AE9pWluQ$ zQfW-Y;sIj4G9fXKZ|?;@y_6qHhhgs|m1LjZc}yI^h~ab&lJ27ktZp&afaNOZCMk6M zkCQ({WKq_W421+Ge>w~+^g$UlOPM%zpfBsgLX)>i>LI1CBo%-FS;_NdCkkW^{S4EL^$^iELh=yc(O)rjve0N`$yyn#%n{JE3|$GaZU1+%m#tE7o?aAvfEM zHhq$z_|)}Bd4?*E4ZSko&?f^2&2^Of8QY~q@;kCE9NgT!1Thxe&BwshcFJl;FehSg z3w9}zT`^gkuR&Og(3j!!YT007IR_8W8LHIn6)o z3FCvk`l@1a$CVamuj79Ftc6e(Qic{JfgDNzeE=`eo|f__7$%wv8H@u^^z1aTf6ny< zG|Wkoq2O11PAf*skF#K7lvzIMXI{b&UKL;ylxv_DmW(Y8zYrWAT0GDh@@()r$q=qp ztlaV^$qhcDc$-C71#TT%gr1qSiU$YE$n?51EfQTgb*|^}Ngcj~A z?IeX2g6^Ow(e2EnyZykof!z`Sr~?Lp$Gu4a>lBDjuqnP}=L?SrCDA}Znj9ud9eg9N zVK|o|__MF%wCb+iEz$U9Am}^!&3Tj~whwLjQ-@}R<7E@{aDLtEWTr@-5bOv|WFKt8 z&u-}+3~d|HtwfBC)65jQUs}M$X*E%sB8_X-A7?OA&{{0Ip70a5}*t z2&=95$TA>>^KgdvaOgZv8L4f#O@7DKQmpnTs9SJT%ITVrm^R}8b$76Sk$DR6%l$lp zQ-Fn(^U+IVZjjcaC3#X?ax1OowfrGaFQW!jC21hE@!=|w4CKia{#`MBJ@SgjYR9Lr z^z9m9|9Dx4<nZFR2qV2aQ_VWh+-AOzJ;OIyav|2DYt_)*xTO{3jo?bWH8g~VJfaecV;4U)P0Zm_P?;ScN5?tOPj%XF)=#v`C+cv0PtMVPV0 z#`wyKQ}eRY5ygkrA4(rBgJo+0V^Iwe=5-S}ipSdt1guXr*KOz-+GrfWrmYJ{_Wn?M zc8~VVeio;m*U)rYo}0{;=dctjY2O2i7DWwJU$uhcJ|XXcu;KMQ>4y?i)H7Q<`H|8b z`v{C5dUO#mI|Fk=xTy|8@hDzutMSuof(7lrDJ1}X^uGInl*wQ3iRZq0dWmD}ZosoQ zyJh&W7TRL6`{Khts3O&qkf?+wr1gTPXKl=j{=zQJP82x2iW~Jrb+wVySHa*U+-T zw!>!1-$uKcrWUYg;iG@ex2|)g2%FGDUq`$`GKgLfUQBXkSqp*-(XBTCsQ5*1Ka4x_Os$cD%b(ed#->N6t1altdma$bv8kmyR`vU|oXs=Jpu2I5hyUByJZuUysI)q>YiXjNeg8zzlg$3Ewrk(r z+xk}pnH6_0&-sLpsV*1!zKn<-C~TK`N>?h{e*2z}pN^H9JM^MVfIH+-BkK(pjMkj` zC+*sb4^HD&ls`wUkF5WG79u(s(;JcsY$qz_MaW41u?5_Y_pr|E&}-)_clOCFpC9a6 zJi6r@%QcTmlWmk*k-KTDz?$PekAcs6HS-slJ`yg#AYSM-^L#pj;wQzCHkPaEcMlj# z?g!MbeNLqO!4@6JYjIeTKQyyc*Y18=Wz9#oMwqU0qk#Y6*Qff+;tSlQE+%Quds=#m zPqd%D^u>U`;%(b&<85!;UqAiy@><67whJa9rX`%V^QA6|?$)<1rQX$twr~IF^z6g6 zR6i@o0QUIzjekd6)VvcyQ7iGXpNXLTB zS(N?s4_j9tdL)4PM3>|G5?74y)Nu@ax3jL zI87`)o#V8FeRz9>tJ4;rp@UyozcY-bYsf7ww`8Kv-9Nd;P|WlUl>7}vXy7lmTfOG+ z9h=<|^oDnEwQpzl$JqvXa#l_w&YNHAAhtRP5aFb?RRnJj9`=Cn8 zQLVh_4`$b@COUH4=q`66wXDm3l}7)JsD9FU z&)U$(-ss3}L)s6$Z4xyGRFzT8t{O;I<~a)>O)J7=)qNLF{u{-aU%upbbS|s5BaP(3&!h0#vTlGKpiA`b zKGmbx;pM-R=m@Iyq2%76s(Nn1$BCEz>yUucpPn{o?ewq98SDt%T=_?EkfEmgKi7KiILR)d3gYrz3oU-0G23B&eJ`)$4=vS+=$J z3T1cf#@!6zj&|s@!K=L=UOqp=u+lu;+F{%^XIdoiKj#m%Uj5YSY~FUR$o+VR$-0t* zWkotM(HYzKT48SD_?M6$N{W=)c7RgPd0@BGT7~WCtTWd_KCGmF0T6eFK3lPqtO?=^ z0-#RDg7F_Frn|@J;@TA4I9-Sb4yrV~tTS31bEOOoy<$*+ZIY+ZN?}Az$&)+Ewkpqy zhJSBEPj;vQw#BTONlkdUj7*z1`#xS3GyK~RB~R-@Marzj_mFH28Aq1R#N%K8@1|5J zKf?zO-`aeF?7%maB4S=PoBtlyKLY5Y$ESBIA z6cId>!UDXr&SdcO_qBAlns-e=|6()=gxn7nuO2}Rjeu6*4W zlX1jzgBxIx{@~`PIySIT;#c4;R|xUhx*@nO%HKo(%uCbcF73>%oIu>G>K{r*vZL_w z`$6;42@Bd9(n%z#VnSQn=2$_%^NE)uem3-Ek;PP#Z^?iM!LuVW*dic2cy+#~g@0d9 z{M%imWtRn4b+u8vkP`|1SKIz1QnogqFKJnO@TpN<(XEz0{al|+9xrvhn`d)7Gk$FO z^gkJAFH!O?Sr}>T6c`!5zj~)MsnkW~eh8M5(bQWK-+Ho*KkB(M<^3doYR8$J^`(DU zp0j>_x4`_A;9(FkK!8gosS09uw<}b858e~RZm`(-*8F?e%!MOaheqom(XBEeF{7Y> zd!N&l8vW(-u&GUcu6>^Tp+@he@5?<~bpuH>>qoGTfy3C@nhfKWw^BxnG8+5tWLLJG zI7t@g*}S+5iTNiJoLV`?f+IsoyRYntxqTj6`hf3}FxF+a$5wUwcGu2Oy&cX|Iy+{J zw-xK%kAAN6(Da$W&*gzM0NNznlL~YAVp{rqEOz101B}SRjN;hf_LracrcYU@8D|rF z$9E|l54JGiR(|*C%J;h+jc)!HL+t)4N^fcFO)X3nqIx3MkLfg`iqt&1zQCHfNA@SQ5LZRlCv#l$ZC~eiO5*V8 zi`pQ!GQ$3}_%;Zj8CG*w`r81XZ$p_bo8^aIjZhkkdA^AGjpW3H0v0;R-+LKz-;1sApsVp_mc5wz_vka1I>fV((Z-O z??18=DWUM!mYpRTB72{c44``zH+fzrCtgC5Joy6wH-%faJ6aAF7&S*w7TWX zo>`}a+JS_F(fw)s4|)8fZU>OJP^J@73P6B(RR|@(B;qca%xn)d$7&!fXL$1h*C;G6 zKsc5XPD=@dRe`}b9ffopM0|kj`&@J7ky_?9XynG@0AWieaz=T|@B)bRii=@d3xVq4 zI75IG_`%>NfLYH#V`pKFliKBMF(xY z0!`Wvob9`;&eTJ-ZJ5sjT%rdjUJR~bB%-5raqG_`T%9;1K(!>390fpSwSZGEs07GO zikP+v!MlFM1Xj^C-IZbkrN5Cs&@>b%;+W(wq!>;Ts>Cx*kp&4A?!H0td2W1Nv5r96 z00qaD%T&O3$5jpMR!1lY`mx(c9-L}hcW~URdQNFaKYb&p&;1_mDTVc#ier=T0Mf95 z9Az3bNnQ`mCwL{fooptrz%HMnELZ4XCMyvV7sujR7wCpW2f^2LAt8>t#%?EQl|oM? zISBG%Bp;ERJiNwdBvfD=$H0^jInA)lw8xAQS>Q40jjrmo+YoNqTXx+>?x0vHh6_EA zTL$+Sj%X5H;x{bg62|D7UIxwvH$o5VxkTJ&wjB~#W2kaWm_5588`W!6$qWMS$y)~?Wi><5v~ zX4ZQ0)dOxHnBp$u)cn1z*kZ%8pNYBru@ur40V<)6bK^-* z)yyWZn*w#(p4W9BRSL-Oa4$%NS5~b7q7*CIdVDKxJwoFH)&*nm#fcX{S=CW4b3&@6 zCuITha;ynav_A!U2w`81j2?epw%YFfEj|K;t*KAPi0u1Ca|6BRE!47Qp%iyGhZ3T@3}L zWhr*09I_ZToL1NqnNzMdSF*A>nVu8RvagzMd8RyOrm9$F3yktL5Fx0+kD0`7hkd76 zFk12!^!?b*@&&=ln&Anq(L8FPQ*s^&l)4}>ZAQ8f?G`DdgqYMgva64*3jq0sc&6${ zOOdvp2er)vw`RP%;=3L? z#j47{dJ72gbxG3?r2A!;tC>hBY|9I#GPe+YPxrIi0fS-q$a~x-uor7XZG8cX4$O)# z>s8ZJNVXKZ5;=EJ(KBN3>4(xKFJDOsQp%@lkTfAwqC%HQ7h;m+yCPuGi1yqIaXfmI?6pcQVo`KhdeP%x>6V&Tfw4aROTGt74ZwZnVi`lf zL8;142KlvGOcU7-p+hII$}*Q~(8FL_p$%&lJ}06j?L7=@+Aqvi<2C)>o4rsckODq| zT1&>tvz{>f#KU?Wo_!>3K1&bD0SSa;#Qb%DlKKcxA9=!#Ercp=2O)u`iMvEgWoq@! z)0Qfl^R*C^)W$Is@^k|sXDcBcxR9lSl9yaMxEIjK?Z>hsbx3Fk1iJ6gUVTWdn2Kg4 zGYAEaylUhEymuozSkxA811&?v(;xtc+*+izNEGQ#Qb;dzE5I6SaB1YFvbBgw z!4OHm2FAoOY@vF|PI)TX3i%d%i@YaS5Ke1(K-!DBMYaNjSQ1}kNYgnBplU>I-d_92 zi`=$_5go|&$r4Rq66h;AP=B+nuI*{MU^uu%i-N-LCOIK~&xls-EG1c}fRL((X&f@i zBeZGNaMkS0;MC(tDpa@7YNT7)%LwMxQ`5-?NYvwDqVb_VF-?1NWF_N8_Mu1WLN~x{ z)yZr=0xn2HfGvV3gjz~6LAQdeO56%FtreYSkY{W36SUf9q$E%fYa*fH)6*d%3M#;3 z|8B&gWs$_(d9!_y)hZ33sSF|Zfd#ST>$qb~E#w;}1*gVPfqzA*5ot}8L{YPcN~%V) zw`CRk3d4(E#Hs2Gbe5u8vj)veE5=L9{3w^DhSKqz+1kflc6jXVGXI>gngx!M;pqRb zI}J${?vyyN5V_QuNNHE>U{Udi!y}m`^dYJ2^ME(Qdjm|aEkfuWHjI3uo6{f{%`DJA z8qp-qJKMdy&@N~&h^psou~?BSbyPuU&T+3*4e$C?`FJM7ZzMR~1iI^;ELD?cx6YSo zyV~C#o6<;=yguBinmBq-=dj7zu;UMcYg5tQZp{QK{qU!%F4ZIKc`vQz?!%uf zN^N6-otTA`SZ;d7H1O9mW`SmA1*Z~6Mn={jdYoc$YX9-)N=sRWt%4Ap_XnKmQIuIKzPW zb|!}4I~G*jUiaP>=ZF8yZy?(u)5`5vDHkb&q&TAW^fi3v!!_I}Ad?!UsKKiwH39b~ zSTaUQk*vWoa`62ak9ZBXrHGfLP6U6Ca_E*VdeN&FlNGTI-9b)2?Xd3zATR$%ITvaW z+_zZacj{LaCMfd4Pr-`n@480=C{v5y7YE0q|7Gw9@=)yOv8lf+?%Ug==`;d@GrNUb zg*!*kaEy`ek*&CH&Gm&+`DHp^l2n6v4eNk@HlGx9#Mc-dKKxlH`|Hd1pDgFjPSPxg zR|6aL(o}_!O|?B)yN(-lC&&XG*8^OPfjUZ@=#!B#H7)ROsfuy;a-SZ&Q^h(a14b18 z@aBg;P+srajTofSRsK?HqW67!c=Wru*fS^@mqsDYBS!yt}efdKPu*OF6I~;;%TMHfkZRgtw4sPZ`&2nc@UQFw* zkcwHnGRAbWtiikcadm&zH+)p!`F$J!ugHlN&l+50QiUo7XBoI`Q4CJdq{;s0z#(IJ z)OcQK-M#n2p^edFkM^RQ4?2O0IWi?XQ)WgF>{02$)lg**I5QW(UD^Y4Yvm`o>%~*u z4l=uIyVKl!RHTCxVFu0eY4*2ZPrve32{~;sk$J0eoAO+XWCqy@2M8*PA#k!2cyDEY z=Ic}02!SNA+kpo4Ij3sd?1k4xM!kLMnO_)ctD@fO(vCZ_S@pzvX@EgPIf#e%1W1++P*h#VK?KDQysibu9jMaWc=rMPag4&!i|V~ zNnqVqXG_n3>?OgtG#1CVMGkZ*E*=S&=z=ycLpZ?>BoM$(J}B8P7~}Or=_^QhQFFMZ z;3~Bju;gorpk@^UgzA1Q>lvkG3%MUXzhCSEKIN6%W>Y!tpMr<6l;00t27kSjUn|$V zi-S5_>nZ$Pg#k{0BGSP>^fmz_GByDS3nhL5hEVhyQut1jTs(42u@>Z)4>Sy86CI!a z=MW8%;0IPR-24Ax6#66@dP}HIlCQNt1xh)ZBcW6r{`)pxG~wggod0+rb1njSm?RW( za|BGHM;j;(T0|=VjN~A{0>+?JClQd7(Y0)a1}pw7(`6g?JP6;1k4Lta2 z*tb}r#oSeQx-#k#8l_@@H7es&moKw2I`yS!J@Na`9CE|e_S3gM%*|@G{kf}b%Mn9U zSF^8QtddW8+v+EFLbIo?y7wDvT6b!u7Ou28wZpn3^zF8TC#fOQo#YS9E%L|Wr@g`X z(%+Gzh2Hn=90WwwAucg-cDQ5E&V;XxC}7_q(l&A$i$WH7WAOqv%kFnttB`$sFSBgthfx%aKh> znu$9`x_7?kM~A=C>m8N8FWp9y$)q~dq^N=$@eTJderbMSIU*TW{05b#MfH1pit5B3 z5ACFR+ihWTV<&dO8Pn^=NAQrTbrwox6WN z;KI(=@7VO@R!(l7N8csU_QUwuw38>|PsVT3IdBj+@*v3b`r8*eN9ZQa#~Vixz25*O zc>nT_u*)ans?M#bC?9con49PH?(_~&RDZW->aSVsY7pJs2(X&sp|6i9)zrb*%wp1c zc_qt-nnd2tnKyKH`)E;$RzanDpwjJ5JR)|q=BUkNloyVpHL55oUu3_&s;#~E2zEcD>XI!SEJloXdy{LPJq1a|<%n%!RxHLk}aFyjTj5@CAnR z*2^=2E`l3JTS{74h~S!{=`-75G+#n}dd4p?$_@=2oAPWuK@+nnvrT_|tbPDc9oY$( z)Al>6$4gnUYOmBOfKc_6{@7i%EZ1F51IhXzO&B~~mSdjB^Yk(nZ0Rq70$PCZ4Xw}! zUM&N-#f>P)}sk8G6AZVJXV=4*pmyuk(B{CzDafjUYD^4yjky=$_vHc-Zsb=+?DR3Qod+L z4Zg`QZdD^R=BP}^JRx3@?PA+`LZU|3Z1QgqP++XX2l_zS?LU;(D==Op9Nb9P6kfC2m>*TjD$ty^K z0{ozM3)uobm95x40tWQZ^6 zPF#FBxx4;4d7tb!%$Z4+I?CK&LaUduKZ#EW>yF6^wN>)a^ zq>cj!S};<=+f+ufk-s8qjn}!BVoRRaB*WBJvLWP{C<}yb4<%AoP&E)-v%13Y5$dTK zP#rv{Rf(=M(Xvx8rX90Xw!H!F6E&x+3W6K)N!S%gl+ZfNEb<_X1A;_4&y3ZIS|KAs z3+=2l+%lX$ATrnc>>Woa_>;%~fsXg%<=P--?p%kJwOPIfDGgXuZp~x8*X}Ii! z{28##5G8q=^vl*lna4oay$~TQ3B7(^N{#aX_~=(~it!S9-a>~Qh|h;>Ry?k+FIniHk`>}jr$mCRad3hKge zw`y)S=MY+kt=f%#g~UnmVmYNP-xlJ=m7AJXgm)XsEI!&f3QUs2-%O!rV^M$ zs*o~`PU=_efy&2j@%yKhaNENOLsNoF^+LlnM5%z##7#i2A#2Ju$r%Eq2v!BVMH%NO zT0tc6Bg&x|5NATi=)htJ)f)U+dR$&;u_C-%7s=!o&!`l46{Kr}73?fcu4XO&iDPU# zPy~QDZcFewr7$C7c@0>R$K>UoaK3~jAWJ<532=id!j3o`xG1S8Od{)&$Z%?q5~o;0 z+5j6jqfrQ|5#Xr1!=>jg4`$tjJzHzg+EtWwvNQejBC5CPM=a{g;9H&)y-IcfT(0qz zzhiI66Ue)fOhHnIC{?e$9CVj!(IV)vE>b@SOXv*f06pDJ;0V6xgnS%Q|Cr?2Lki{F zqBPI=*Ci!@v|3DL=<-X4_R4e$TbRE={mJKqI=`&*U(I=heqNqKMhn{8^5~qayNdkg8 zUb=s3kk%HcEg%j0wxjfAj`CueIY`7Yg0U3lCXb1=Qxb4WmH(OYZlQ58>E;-epTJA} zWMQWIJ0!1-n^)6su2C|e#^ZQd=wIgPw;TkVR{Sl3hGGxNsUB&7=7gwZ#sMK0rRdH# zVlzU@NM?vLkj;(V3?t|g(R}SjdLmh=KWyC2Ub>(3M}K&Eq+;!T>;__Ls~Mxp@{<I_>teEbr^U| zO148kS%W(w+YDPZ8~P9N{(@*NrKmN2i`lK1M2Z$*K8||WVpe*et!)jy!O~D@5-A<% z<)jln;Xc4eu%BK+%o%cj0pzL`XZq$q`T>&UjlD${Q$@L+Ia4z~rp3c*QYAy%{fZ~l6?8GRVvOty!{l7Squw25g6 z%{D|C74xh5#KCJP7kMh-kfUPdFf_$uwUsDr&XUd{Vi0jZErJPx;`ZP*#9~2yEz~~> znnUAL!E~6)vR`1E;fefZh&#{TyZC3o2p;Gj_yT(V`M zg{z5Z@bQVv%}AZ7$!Omvh0z&VG|XxzpokY*7PLK9d=!Pf9UQs7vxRU#3ifxLJC~sM+DzNQ#Pes>ZA-%+ZI(MtB?De zJqMWe(*hwZh=~b2kJf9>`sluYiKKXX6w+wqz7#cz02&5n@7BKs!ZmlRc_O|ob5jd? zvF)$IgV!%;M{4IJNw5zJ#v@*h5WI%HD3_4PKR%MIo;-P5goOUlV_|EnX5OnjIR0N* zMf@LlCfvN8M!rA2M65fel$pQls`{Y9`e(YRsO7$WEw>gC#?pj#b7`RA*VMy@dgDo6 zWUmI>FJ0r3=^65nC;nZxv%B|LJFZ&PLQ4WBS`0_&Mxrt3XHL?G(M^NYnVs~r1n^e= zyIN@|-B#i#e;$0rnDJ4z@4_=bH!i|T^U5<5$?_|B}KLXRW6wS2C zBSyZNQX@s@UvS4gDNbg5OL)7P9{+o{LEMv|$!og_pC264Scdf`8Le{y?a++kMNezL z#&H{Fvjb>%zY^2V?tQcK3`j8b`p$#M!;x@c;-Xyi|5YsEfntgKUkaA&HkQZ6El8Z? zPdz68_RmLuoR{)_uYPP9^NbSclJ38+d~?bN#OeB1iK~nZ+)9%g`P)uKRk-P%E=npq zTzIp@YTMrCwRbZ^j-6o2%7>r5wAR;7D{uW&Owdd&Kk9=#*byG0>Aw#fy=vF-hfi`- z>Qbx10)gF(pecuonSUlqieLaGVnZZ-l>r!dt=mDOu_hbN?+_ ziv1~B8g_Q=v!wh>vgCZKmkuaYeC)GHZ4Og2YZ7>F&N@{D%~m|g3P}FvPLz?8VNtMu zR$!{)V|&wi*nCa9pg>0&&8SQZ(MU<5THZZ>?@o`q`+?Hk-y+j1D(~Md?HkQ7S^LX9 z_a}D)j&3V@{_^FC6Ei-67s)q>EtdSoL^M#fI#oj2^nhfX|D1 zc97YVSFc}2nfb&wiuAv~v9M=poh%vCa6477>uO^Qo&nv%NrPyVT(H6!BI!0pkWSoUunO%i~Gl?7RcIl>cTQ7b)bpF^*0(IY1 z@KkZmOixt(fEI>7H6*j{KD%zozw}EgY#Au1V-Xm@>vQ8zm#t{=;tlbJv4~^;+ZKeN zijCIKy$67#28SSKWNcuKV$LJ1uD|m%@oyV4=|+A-=oGc&t?iq3lczRfCJ3%>EO>gd zTpbjQnIc7?y&2j8SQ0xbj!a1^LEhnqh}aJ)ZRPOs>79#9mELY%S30y$A;}zC_-jt; z1TuY2U~&xZV5|e-(4YU=_8^MZj!FQ{m^sjbJK%ci^GJ0#ij&Kl9&0H8{)DHCrx@bC^zc=xz z-g&aAH{D==LH4Elrk^a%xgIZ>F7$r7%3?=v{OL12B`13FcD&QvG(Ze($w9?i7#g=2 z>3q5phCz+H{^nPA@b=TjtbSK5*NK<@doMDnug%WrxUk#qlJRfv zcUo57|Kj7vX%z%?^_F^+J4PM57F{!vY;thtVel*M!u05yAZ6pmbGsPyl@7|;yvq)1 z@lE`a=V8CiZ!i60Js;ihb_68JU%!lQclLjy{_zn?1E{JXXJ%OQQYDh68|97kAD?X334kRainJgX2IYO1Zt?x*O>oPC7&bgk?x zj*aPyFGD16I3u?hc_oO~*wiNcp|tL^djQka%kxxq+=P;DpwynE{0ga#guk(Q)Mi^k zMovQUXAE<h((>H9b#|ooep}!o-tBFY4qUE;|GkJ_b9YpK`PS|kSD*08c*JcI0~LV z@P*j?$m_C|>zR9@)mUR{Q;zr_rXJ8(L~dC~GKQ(QDyoM5Ha81)kzP2%9<9qgz>zsV z4cK6G1NN1K%4t~pE`BaGgP@E>$$VZzUR_;dH^-0?h||Pc;{#RqO$|83;F6xe-sz>- z^ezA71m$fR^83o2Iq4_ag>RUi8Qh?rv%q%*SHbB9jwL3Msnl!P;M|H`14k5=c_X$h zwH^QoPEnvYpFmVw7wX~{!?Gj2H$wZbI-uh%OTNZBw>YhKF7$FQ^IxyrCEG3i6&S&< zq6y5pZvqsTaviHp6DS^(UZ!b|0(C+YfVKu5NQZkYrMdMpks!1BY-g_oCRfe{Qdvc1 zQ2;HFEw+_A9%nYx=1(rp`u%QpneY;{Yv)tc5UVDpzC4$yd>OkA35Qp82*yAjv>K#d zE6C`HL6sW{lZo3->Ul7(+2#N}N$>(lWIR{+WSQXf$02tlXK?T4L*bTgR~z-P2Ec`J zB{7(H7x(HeLCsFLjHIErr|1!ZWC5cSWVkY9)0t1z4TtC4`>W(`&DtL4psK}7#ijn* zwd~7R$7>dsm7a~wKJuw73^BXHQxX)*Z{aSe(v>|)I~(M;!E*FF1d`pVSOwQ6Ym)ZH@MF?90Fcx{h)d+E%Lo-&Rc2+x!bYqry!>%cc(Cf_$Or)0 zeD;-W!z#Z-KmoyOiqxFmVrJV`>y2eA3_Bnd{t@rxm2j8jUT^_{*v@VaK~99}2e0Rj zyJigoGPf!e^WY{!)+VK8eFD6!TUX$ z$Txu8;uPyI54PXq(Nz!^AdbniIZ#=d+d8tfw}LK?$-|$h%XQ88u(!NwcsIW4mh$jr zrAMg$i^l2y2bRq3#Y5cMQJF&p>gkd#x6l~4%)8qw@Rex6Y9Y3R4e)OC5rs9N4(tpv zQhrhg_O6TbV(*&qx@u05a2;2cSIhby)SJnyM_ZC2yto>y3kN_w_-&X_DU{gBOFyXZ z&TMSd-^W{>mLPX*esB$Tm;n_fQzJEcl9{PsXCrU_H!rTu-8JKQMf4G3g2cmOnv>Av z%J)R2IG_4g5wL&mZXLmLx8vy9BO9WLDc{3 zC7aDpn#KrD)ys3qONX?VOp^iUa8v)y&JJF~dX2gD&56Ptd-7QgTM~%wLB4te6`{^F z58ZxWyjy8nNZ-A|s)%T>%Ut)G@04XlDP!+<>&&^c--lGz)SDhXJ!oV6^b6g`mmJKd zs!UQQnZdJ5uc=3TL!E9?&UaLy-Cg4;K2oef_KNYJnG`SN)MO7xUe;6l|MZvHh%Soc z(WKkowBC^FB4z7ZB*%Ka`Ym5U3o?yeqj{fqJWCbgrrSp!P~U;_t=4CaV$B&S-HJfn zHMo7R&6~b6o2!F!!cV%gf&f0>M=R<{y#u;4{4-uo@3$}G-m73oaxTwvp&HYWA4(f- zkHrQF#3JeYhkr)c!+-rwpQ28$V$GvYf<7aWGQgJHk@tViri($2u}NH2&Ms$_SNA*K zC!I!;{mR`*zp;O1)!v&0`K0LcjH|_)ixOVf^*MA%q&a?88*#^$bfuGahgP_lEaJxX}mQ03o`yU@X& zQ0Ul4&rjOK*;sw6^6s7=Ui7Pd3r2n&Ae!#UOZ54QN3vPhA#28A?E|)bz)q} z8W5|2!BZkioQJ>YA||(guvDRMHTU&jtz-{1;Lt~e+?rVHy@r)JpaX(DToyPNvu(UFV-MpFHVZ-*2l$jRFI+~o6q`1dX=!;%$oR;f~ z)nfNv|G#c>^Z(SLx!GoZ3OKJyl;+y;SN;jDMc$zfj4mjG|Cyf&L=|rq&rCX6|2-kI z#~F-;d{_YvonNJ78m33qW>&+jm9j7#pEoddCI%VhgQX!2NJMpSdo|n-| ziO!woJN%t<*1d!Mok~kEfeHQ-rS>QNC*KYf6;bv7SJE8Q5b5zW_h>O}++$+?-&%X` zs3y~OZy3jM7{|g`5Rj4?QBe>O1*K&UyEnosCn2DhCttL8?34SyX_7uT3CZ>N^v-0HXXP2Cf3wmZLJZ@_u|*C z0q(|1*FffU^s{BR;CmfyMLFvCGkL>!Z(}6g4+K>`n_9VHyV*5Utpp}rWC_51)0@yA zl)CZvl7l6wW#3c~0D=J`AAP*y*5cnY^W@6R+`G?|O#ORi*1ox(eCq1C&lS~jUWUYW zjoJo*`VEQBuAi#ABO4lPZ!CqJjfmO(S!bG_UhMxMGOfBXt)@Bxhj{GJ29|NK@uwBX$6-pkyur+v%@<$pK}UR`k({9EmF zxl3k~{0mBDIhDpym1|DDx-_(kEH>)w$C}#t(t>$C9*uy=QABR6Wu8%9MQ&5uWaW>E ziiIZ-iIqGg2=G?09TWxj5Q!?F@xMaM5=Yf^jM-iqtktJJX?#Gz5!o}r$dN-G#} z9^N|ckvsL%A32p?_P#BhPPJc(ysJs`ge|kv@}Ivh?~uW-Fk-^$S@2JwSlr#JxJUWg z1HBTCT|&Tq6aevOs%>JhjH+u?Wz`ctjDbIr3qBP9w^^SPGvdJyRl&ncOmV#eTox?& z&iSzTB+)@dmfWc7tI~i^nJtwaibu4M=nm|&0M-+!MRe{ln+^&FKJ@a0@fCwD1#*yX#4VA zA3h0r)P-?YCe#dba3v9Ig6<0ILZWeD_gVfT1_jO@nV!US!8=B!f`{osszj(fzO<4< zJ*SAJOobJZ?u6NXQ?UstwcY0iQYbFFWbe|D9oEEmHJ;G;GFZ!AR1o<_+AG7()Bv z+^7`A!JdTCJ2UPo3?qvO!+MlP2_sCUpksys=o;C?1ee9~K#nPBy*Q9cf%@BscmJt9UP^ zfI^iuB1qq|EY?AuT=QCf>fFypG6*10!KO&e5#W;3Ykz++APVJENbmDgLKJoNk2!4bkb>RNW&siWUi%Im#ik^nly ze86a&XW)kiGQ)pJHQhpyX^ctzm4T;;C=7}SJ%dyr8@R=zqYH^KLX}_(xr;^xIu0vb zSYY-kUco+~v)oW(H|uDj{)3s6!4VSLlp7#b0hww_7zac|>EY3SXqPHX+z$A0C9u0( z^mc~bg}tKO-@GWgE?K97#&m%$qD?ziCw&db@z{T0{FX+=?=O$6?lJ6aPA4ABNTyre z${Sukb>O-3ocw5jcj2|UtMdgmZjPg8UpvRkJ8hZt4`#2ErmuT>m8G3P@QKkmAvH_m zEz&0n292t_R%mxHi^JolS3gjA?rk4F23mYOFNK&}1N^}}x%(|p+N9Y!*20osSeSGR zsv{Npnl}3giN1=y`F#`@5Y#EDI#ErI(sZ@j573iW$Ynl!~lu4+7x8qM%Yda*oU4dYM3p|Ig5dK@t97aJojf@K-TUV)$Oy6ZQ+$l*y z)ct__O2D}RZx-@`Lq)@a=87+?1wLXP{7ap_p$YW$c8KMGF!;GFU~FB*z+P2A4mV2! z%!I(-7;R4`GM4CkHEvRJRdIb?op;hZn_ZA1^olPvfk|bFKnBmJV5Jk{0h!z$bd9S2 z%f%$XTkd=Yd{04xISro>CpT(!1%DNR)8aW}D6J{p^WW_p{=dHn{il4Y|2Dt=XF|Nl z%~M1JK2>v^jw_!rlGK|F4zOI4q+K-> z(U&RAfRQsC+5^ZI`9;nI<=A(-$YqTlGlzN~_c3R-zG-k=X6x)fcO81Sp(2<3nzoN_ zm?CQU9KK_%bzgq&Yt09>&rD zBc(r?&sbdLX&7kP98Pf9r>GAsRvHi(R;w+i%B>7+FH!D~Ed_)zrL6f7xqRu3?6?0K z+?vUZr7N{p$cHozZ&?OH*wO}~DO@q$d|}c5{#@N$RnfEpojA>9k*1le^LKcGTl5Awj_# zyIfrS{o|kBw<|u_;_>0NvdiTC-Zw#l@F-mFl|M?H8iiI<9yeN}`pXZ;dBo^cxcTIM zVKq~Ge+LdgwA+&1zc3bN(`KQ=L zza*k}WVkKGDJ9R!Z|Rz`7xT%9XyqK#-`8eW>G1L!lAeEwfpTq*?ALSCjZ+Wg(2dVV z*%AY_l)4Q*t;+oZ_dB*8$!24;`>+~piu}w)__+H~w!Y4Y^Ec|sTk(c(@(*mr?I}Nk1FAvo$alMQ{3BT|0SD*mL}n9> z;pTlcOgSl9t{i7VImhHXfzMuzC|7F~-5IR>1U+9Yk1=2s2lr7fd{a3aNs(^jXei!h ziiNPOvckV;L;cDSm>ZGDC`nin*3nEQuSeC^$}RhB*NhRqldwg%Y6R<{wWhQ9r}}VM z9c=FKxUnA3L4ck+$j@m}?&-6DtpldBlO#?C!ga9Ow2JFzC(NvQ#%8&|ktUb6{1|9@ z*qYN!S(`hldGErFr76b^-VwPWC!9?N-Av}Yk@>Uq>8Bf=ABLXqlRPb9@xqXkLX=;X z3|kt+SOjDB6d#JUSl)pITbUFwPRttofN)>QXSEQk=_vJ5UddZ#67;jq6wahD7&t7f zkWtqt`oolw<;hqQ(ZHIb`o^*7BSXvRKrH`o+{DQ>|0RfaKve-?zYSrqr4J)TOEt^O zvr^-#M&qCerII~W7A1l7tk7?l6gK_>QrjZkwaymEE{; z3PM^+Hsod=K=K^}j~E3s_{e6ULQzSrQp&9-;ZcpB@8coQ7?ctW#jerUZ$X+#ZkYU+ za|Tx2p#Q0w|KHVb%PessFsYs-C?Ed1;+zqW7N)+CS3^%-SpC=cK7#Xl7;M%%;slHC zIsHz908t>?D(qX8Nui}-vkAVfkA>dS8J6}5~a%x!RVQPZzf^*0etTH zH_A|rEZuF`PK5J%`0GPX(`W|&; zbWTg9`CuvDp@=Z{gTM1$qK`FUYj#t(XH~Xe04Z^o#1B^5 zn|0uc-uwgJpdL6{gM-%JR3v_l-smqzSHQvm`^k~6^n1w4wokR}LId^MX3w5pj8cVx~O1e+-VDmP){ z6;S4kSo z#;|wEyZHt8jk|$YrvI4Tw)u~tT~`u;&!My8;naL3lo6ts{^8q7p&-UT5_8qly^FCJ8g!4g2%>uZS;qw?i#Lw+Axl?`8Vk z7|;BE+{`GEBwB)f41KhpYz*xXa|tEN-AOe?}kGgV>lmeV*t)KSAXvU zMUt9eLAeUeXsEom@L=d_aLIT??L`m3UWZ#cV|L_e?Tvb281V&vJ6rVN?Cg|7OmWWX zH=4;fVk@NH$lPh{YNs!WYElcCy1NZK6qFsVz zNAhi5Y!!A;*%Upjmm{wgE;77`1y37~kC2tv-=)F!Z>S%CChZ?#Ghp0OrFXU(j8E&2 zo+9WbE@>_|UR0nlibK6IgVOy_Ox7o{=$Hgjp`@W##oQ;5ZT()}MO3r=z@Mi_$x5x!@<~fxaKV+I zb6WlrEldr7W$^A^dx8N%-Q0iJm>7>#iGb^b5<@|Yx2qZaMOYfx7@k$nIVpA zQYI0svyr(Ivu8`K4ktKA=00bIy?bM+ybV%8j3rzC?ik%NT2$vL;V@SCCLpJZk{xwe zrIJi)x)YSaesU$Utt(%0eg}!Xez%?;c8z(#bWE_(C#kJYxl4iAd>ga#j=DIKE~|~L z%TqE!_^U_7ZL|ys-yJcO!f|XTZRECNz8?T9NZL?-*fD4OUj_DAXbakHKqV1(1nM-y zHh;D5DHuyjQkb^eLwbeh2y-9|{py=)&luZj<5u?t)BT-S5+<%wO*(v9f^Iy{++C%u zaredarTxMnzY4EC{dY>;o|`NiZ1i<>I?*wNjd6A}!|icCoqKkd)={?uX-?+jdur)! zgAs}0gs8(x-2#~wy4O@h!em*&62f~s zXaL~qehfcIk&WyC4kbxsVtr%f?hNE=<=S^L2yk0KX$b=J_^fu|u8Gy`(f5-Nmv=qovbV$-Z}C7H1oSN_+}4G8x{+#_CTqHscr7<3+b*TUDi_&4rgs9#pZOY7q< z>$%1_4lGk{{SkWb}8{6K0?_Pv&G=KC)C+E*|~2jL$&f)pwt4epS-dONFbPd z>?lZg+Fa?BBDU^+L7S=L1lkTc#jZ#)@|)@(5IOGomsd25e@}P$j~jpnQ@*lqYk-_K zs|_x`NV)%~2vq`;Z&R3(#r@J$Hd%p&NKAUBM;JSG0p4v|PGxWz^t|bmfe?ytgq;?U z7`W$;NGMfn8u-2GARSlRrr()P8Aij(*nCF#TN}A_v3Tlx)ALKJSseYH#8Y})npEDb z^cGi#((rW)y;aj>)&jS=s(|!YJcwT+>qv@5DO=)IJLn0+pJy4#}ey)_P*(N&WC;6SY@>y z>3QRpq+Q$SjbUpmD{Y2`cGk)27L|+cNr4NCroW@C1$wW`fczQPp7vrMWtyx&hr=deHz1XV`}%)ur{9X^ zessMkSld&Kh;ws(U$5s88-DikvS)Vlv~!6g;@T=b?Z$0{03@z8e&5M(QxeGW;5y`udC zO1y}UHbL)JE55{I~qL0v&Q_-GsL zR7^`xFX-rg{1S-@!Ztd|GNk#Q_bJAu@Z@IVWGE9ZA)YoKqlUeJK?MPk;Zoq$PB(&0S{0`tkE&nt>LQ=BxxG3-$A^j0vjxXU+v zQ|UXi)Nx!F=z4)dH4t2(D_O|Jl|yTVOCiy{%iOR=>U*vSf)0eR&__~` zAV`^FGyLIrIFVissv%Q$)__0}IlNy3 z?4+*Oi-@@=AQb|X3Xrw?cclmz!x9$d+@>GDsi2R&uQOE`Bfz9fL^>6;84KO3u%(0b zj3S>f?Qoc<53~4RZWllPTZ_g2Q8Fc&CS0IIokCE$ydMiaSd&fMGU*Ii7A<}gZsiW6 zF=p)IX_diNsyQp6gIqpE*n9nmp-^8Jtq6RiiaC%J_x*#fX>}Mpl0igDmlG<7Rlpm^ zoj#bp$kMgPGpsBv23FG1kqYwsbC)D(0+iuHrx&<<1~bg>a}U!as6v&?*=zl#QwE1Y z5Y$6$G^SD5G(?aWD)#jSe*n%FuKq33f>a7%F|7$<6c*f5YJ3_AVgH>%Ax+?Fb$&GcT_pQG9*2bZpPLQ~?_20)SIl)?(OlI3+P%)dFWKek zwEeFiEwo)>M;WzUy1^#~YNxZusAv0P{Ldkx=N#To9_T&q7szzq&|UkrPqV^8hrM@H?}e$kw~giy@?-*INWQ%GQY*6fcu4E`bMnl$h3q3cMl${&;+W|3c^2w-(ZE zKKpF($tGK3_?%aK_K9%k)5Ij-=_^&~D@~gl`^}&AgXY($^An}Z22bm8W>34KN&yq@ z0OhhV)RkfD#gW87RKXW4leN(m@vu(4y?pEgwY#>yf`g(cKKg;V47Oc@@a>Zaly%`H zLFWdd{T1%+4C9!PS>d|ok<^kAGJ4Mnz?XbuU>X2?LzqKbaF*q*eq(;;^J{8+{ERG~ z`RUK0Cq5*ElbEsf(ghJ^N{j(?g`ZS8riu-)+V~|Xp9!glAYc+QrOY8I$U#_%xu;G`j zdoL{>Bx`rQdh8H%zx4MudWnv)Z|j*S-*b$7oot4k4i)xsW8g`xp04&G$!A<`N-OKd z+@}LbMlu^LH9UoK=;LYbyU$1s!6zF6SGveeTK2P%ll{Od>v`Q=Y2V1g!$dU3>}JYa zz!Geee-i&Yb2VM96{Cj!~HkyQhO%4MMjs+ zaj%}Fq!Plh1>a}hxM8EWiWOj%h(A0>SN_Bh_Kli?$6`Q`d|59Y&7l~8_>KT_$Q2Ju}VZnNc zP>J9VBp8JvVeVe_guo^Qv-^EtEVtXh6h=r6q8DG*i@GU(m8hcDO;;_Y&}b84Q~$?V zkf|~UH1Emu9QBO2piGVDCuR_YWhCB>UH=!)Z?;|mwJ(ChS@4j(`7C=D3kAf)Ub}`> zIdr?bNUIq;HGX>3Uz&k0NIsZM8aaz1tiwmzhXt|%q%3I;pcRRTtS7y#d+Htj6{7IBw@MTXFqz2}~@@GjYz#^?i*YnS3i zLXr$=un~uYIZ^F<9?#D8av_8EvBL==y;xrWY+awKvYc7j;$AN7{7)l{pP*{!i-b+P z*s~1&MvKTm+UUwQ*2Dh#O~h!%PY0k`Paq}{3kwflv}b6=25(=+Ak=pQ)bGIaCG-MM z%kPk+cEBdIs_u+)A9NM&!4!4U5b(5vGO}>HcU183lr{}fcAOiu45c<;_9bKJZJ^>e z9pT%lTQ@$Bt0pY_3vB@(&o!2q#|S`9aES_1Y^m_kcGe*=8%A@rpKAIn5qmYsAo9&q zBA4bKcFk8b|5W)RrpVr0`61(T>{}(Tyk~zCKi7_U&nm_ypr2;8pD4lI3q(Rr5RTVF zPd+DXHw0Q?Z(+oeT*=kT0yx|{3NwqqC|-#B!%F2#81-1w_Y5Ih1Qn|{4GqBnI-e)X zdLl&3b6MR9iT2-AVpDVE7X$?&OdfxbI=4pID|1s)U5GLk6b`-BcUdmKfIF(GLY7rd z#Iub>ItG}*FT~wHxD$L^4=vOUi(xU(@)B)pCyc`Oz$vV|jsA`Gn0i+%o-)+yqdgQNM6p5k$s85d#kU-=sNlC(aIsWx(N&yFpyJ6Y#m*(5Q1CZ;P zXg{#!4Y`U>QbCcVG13%n6I*qDT+(<+ga$t=25i!zOR{$@G2ZdGF=6;{5i)$Ge4tmV zhrSYz8z(9&cee|0BNL|6UdR$1vBnpd(;F>0c;s=zT-+0*Gr41A`G;_1WaS#&?JJyCP0e>`YFk_CF1a(0^0Rpx^SvL znm6~AbQ-yD#Vo^pap~jnl!p=>30cvx&z>cwsLtywbm^esD(A%N??E<@=9ali)|;VX-n7dmTaS z!xy3UnJo*$t|@u`cO z#yg#^d|yg)b2|IX^K8W#%qvVmO$ajrz2@ZmjZU3M-MF-?O%~v>nHncP6KoOLDgICb zR&NdMZNMcsFY8cTBdvvU*RSlUxhZp8^qN1B78q*3?&Caf1168d%V-SDD`Q&pFFcA? z9O&$6?LNd}viy&1jTJ7%2j9Xy3L@pha$CA|)1pdjar@F|oW??}oSzSBU`n0b7Ro)M zqKa9$lanl{w#T!-;kFdmTs$UZpWdeT?yUZ>{(@?c;?JaMx5&uc;-bi$WcHK5t-wlb zjkCTQ0Y&e^fCVj07Jj714Cn}t{5gCx)9eq+-wwc+_Enurh-7cL%?><&NKc!|UbDwf zQE(`e6^h)wsjr0Ke2Y=~li2|JZayW>weIzxl7{`Ud6ghbxmGMQl^aPio(0CIwrt1E zvSkN82Tqrdxo^GJm|GC2#s?U*x+Gj`x!fSOE(_9?Aj7|Cp3;??CiD6Fw&`X&v3||PUk+S zPJ_T^dOCN^K%?0pu=dYcx{pUuT8&dB!R|V;e1(`hKFuJGcPGyVk_cr@@Rh|aY4!>@ z;^CAjlG4W|)+!zf`w&ajNnH5NPVx>+S`4X@02dM8_dUM;O~s;Sk%B1<5cyO;T%qmQ z*U?q+lqw9IcmiDdm!Wcul}b|NK?as;RP1xg`kP9TydFfts}aT$m6##j9;e})Ti_$5 z6oNz-;y=Qx^<>duTReimRx`aS){;F#e@z z$+#&+6qmgmMA^~!=TabSphGHBsLKwK=_Gm`&os||Cm?I5OK=wjV;`3x8|{B}0^^X1 z98C-~j&>0i8bdf)oe5jMv-COI)RYSTsoiK%+^kgrAn5HSAFQEm2kN*S?BEOAgM)P@ zi4B1Z-jjs8WMb3%E?=*m~l4`%=gOf;L5_%m$n~jkfKwv}9Mjg5Q zrdGE51U308*?m8nk9%YXWk&$a$Ecr+urF>*Jf{GgT8{EZ^t1uJA`+@VE<g( zNl_(G8eoOhEKM`$F0ihFz;{{}R~cA>P;owfinvTAM^KZab@_*ZJfc=M2LQGtsLMUr ze@S9IdsvAVhoFtZrjk}I!sf65{YsFXOG0(SbV&yRJm1uDPN>D}Waq}UsIybhbCyWx zgwPbr(+9_8C%%%DUdr#1n}Uo+2HYO*wA2_34mb5*KivKQ&MECReR(uiQ<^UPk`6d6 zO3jo(_89QxrUU8Fh+5;vNx*PMx+ZBs<80HVgcZE z0pXFm%QuyPZEu|zM54}kdQetw;`-4{9N?s;4J|3_>M>1;yKwuuC~~zDabO|4v~W_5 z{{?u%feZtcB%*O~Nm=8cvkEkUJKsnuh48JLnnDIeO-qS>z0K!1pV216um4b$jMJYk z9r-(q_KWEtOURAGGZ3{{SC1+Wjg2uq`vI5(zWOaW(Z5^J%QQIiLz#mL|Ha(c=mRI@}TNgJM zg!?qzc(nwfUUC1 zL*3Vv=Uf-s-CyqUd~n-hF%aSIu9}FejI)htTizLLowl!~A}`o5&tSH|e9Fq|y=Q)= zM=mS(d4JU~GZ>*d>%+HaDD8x5%*`9WsWd78@e@AKsiLWkvLb{Z%)zAEIPd~9BeHOi z$WqM&s_HFRzOKJJ1!>ZssGmFWN^&Dwe_A_in#i7bvIM);tjnRLqBa7160j@|p?p$q z;vc>n4N;Z&JIMl+PZR|l+WKklLX-PAB<1WIQa=IJQReH%R1nUPnkjDEo^LkF& zYrRTBbMR;0oUODvTW)m1YNb|1z;$UiW(7|4d;) zXb|AjRZIm)*We}nhz2DCI<*W&D+*mnahK=+Mwwzk??$nf1%r(jgh~pn>*?a)$jqYL z7*A1U3!LX_O$iJ0W^*3}m_z}#zy@BX*A%U+MaN>|G!V3B%S)tuOzio=NIhI;QXH2~ zgq#=mAn$>j$XJnxA|cR5j#kD93A&9t(Kop0<^i567BZy^?ztJWyVGfna;Pa+FmsYe z7)!82ZyZi6&S`kv0eEtBDT&8L+x@m&`JcT4be2iB0jswP0I0hGIl+x2{zfLR^{}uQ zS`z9LKoTy)rH}gqXGxH-Iq^~eZ-xM=s9f5T{?8G${?}Xw{q}Dncl|xFwMo)|$(4Zw z&}>_QXZ5c5?5>uB@l+1F!#ypQd_dra`%@Cd5v6gkx5?I$4g4usy#VqcX82O;1_R## zU^(R=x2@)r%IgamXQ8a0-ywsM8w-Lf01kKYfIQGh&1cJ9oZ)*Afp9x-`{{DeP1X&K znzP4stxu;HbAC#Q-Qg9$(?0tCOjIta%ro83bz#KB2mRBTT+b%*T(^l&lAX!?a+BJ6 zd64A9I4qR`+t#craOcp{?A1T9;_pgiDVtIr)iw%m+b?WkXJb^^cB_$Fhdjr{ll)DX0KtNmsybwN1*sYwr9Qzy_Nt2GFoW3`o9xI4cH zuKSr|^b1-SBsb-5jn45I$#z>C1-;v6LxWFv51b%;TGdeX;)*Y>_AIpKwQ{J9SphxU zp5*9C85{u4g&mxnAEkifvdD?6IGM@RU@?o;Nur3sk;i=xGTYGr|C} z$sU?PtHwk8`{W15q3Ue{ET^5U&BxxipvKp_tHtL8WVKg^B$~W=>7w&6#LxR#96aeo z_tsZKEw&GBLwa?H2Rw036Oswv@^g#r@nZ-s)SQm{nsRsY|0(ZgBORH@k)Qd%8Bcv$9 zTKgoP7v6NSAn3~M6-RKQ=U#EQ=5NpQ>)BfBjn{0f*vi(AdS;LJ8_Z+xywYPm;^ey^ z>oJ_~3X7?(9!^R~wXL=NUjrQAl4)e3H%WM_W;O~JaiK1xH$ z6@v~~+Q=%&-cm`{n92Q!F}Mxk2Z>{wy$J}C?xJYJ3%}N6wJ|lAy!D6UBAC+*(z)wI z-2nQRHt%#JPl8AKl=I7ruum6f#?J5Q%62YwqCc~6+fT8r6r6vtczdbnqj94fFQ z)05j%fJfNfobq$<*w^t4ncs$^om|&?dR2@zJnd{>{`SzOwo5x-#i$d66)~e@U!HQb zj+Sh%3+mP~(3ma0P7m8%T$DXAnMI~Fj?=qqv25|z__#KxWDo2uR4-nwP0v)|^?|Zl z?miCo(e>X{H0(9ynQQwH)%`wYorURl(004kRMZAZ0sdUg*c? zxz7wX0*dtpx02G`%uJJzp}CkCjjnEZp!Coz!|`ez-LsnS2*4F{xl6D8HaL5|awl+D zISx^`4dXykUk$~t?f|X^cS}bh>Js`}JtN7(=hLvQ#^!2U*Z_UdjoX$oIvhD7>I*U9 zaRx4I`^Y@c!Ve6S39Hhc&ve%9?Tfx>T~PVcA6^%Kdi8w51+$K6u(y-7yM$r<=J@lH zm%OiZ;1M!{fu(qev+iQGP(RB<`h<3fQC{2xs`d}G-1;c0y1(D}sw;_rO;)c*^5K2^Y>cM0n7 z$h^T~=D9z>vO!;Hr92-qW%0MX$Uog847JzyE9^t(tbLgAj(suL9h7|b~ojWg=gsI zdiN^-G9GljVlTLb-YRSuE?b_&HtQvlyUbLMf9)j(V-KISsNxzG!5f5-7F5=AE@-b zQ0ZJ9{o3M&I(&4w{|z)^*Zcem?~@+UI>9nY(&xe2!Q%IW5nh){hog2ePS!u{j$h){!1 z>pb55T4d7fK8#kOXQ|_}nR~Psl&ji`i%({C;^Ph#zc}6Ams;C09;|BXa8W0*($aRH zfBF?odbxX{P@{Qj-JVSbC)nctx!stYtdo&wn-G#x_fYN2&kg9`!C(o!47)vYKypQ( z+5zF8Q5&kEVvHma_-k&a;V9#o#*c%|1{}!mFbMg_c>mvA2pJ&0lR+&(v}wd1Kz$?| z3r@{(!h%onFR_l3HF}d*KUGwV+YgGyrZAU6F702EAPzlo*|Z%q^R^+Ym%xeN#!WJ-!_D-No5UbWuOc6;A7Qh=8g4!g-_weD8_#Ic@BT;j2e6ML>DEVvI}y}L0cz~G5nQX zt9Dy>EKH!<0;^ZOxAS8ya!Y68p=ZF8MsHoODXe|u@W;BaptjAk6UE-=X3k}5_vq@h zX*?dQG>ncojaPqkDe}YF5PGHUhLaXvd?JVq z6~;)B(DUm+p}W7%4@5pogpio=SC1(l@?oNPiW2)BsE@Es7UxIhjoIUQTmN3+ZN1%F z1?szwIPAGQ-Fe_lx$C=y+J4USr}G$m;NJSVq!&2#?hnOek|cw}lNe~U!|p<=5iJ|w zqDPAC7+wmcnvvNI8M~*1$WMTtsResK(V#s;?Jal-_PdgQ>9w^POEUdAo@9=rXaw5Q zElV=^a7SXUn*!TZ8s5JTq)4)O_57 z-DR+|u+;GU!`#j3!)5&mKZU2A&a$&VXFmJo`SaZKG0jyNV&i0!w^Oo*gfgx+nWUm2!uZX*zk^jCEE@bcu_H=te{F}e6 z2RGf7rg_cuwY$Y!xkX5OSv+E3+HNstU-M%*{YJaTS#(T*!^1@`_)E6Zo{SH4wnL!KK~L9?QW;wUZ%= zSLk3ibKJWKSZ>~@gkm+5E%?gv${F_eF_0fW$5(O?ZG=?4or1GuixetWgGV-s$F@F~ z+-ddA-Q$oOxt?Bfx;)h@^mOIk)?&WqOVXbAT1Z3sgUsCr&vL!-KN_OC)O5@ZWvOqH zWas?E)geaZopD4PAF*Gfvz(DU=r{~rPu_>3VmI2Z6$G^y$wHOgeid?eU{u5!`>kJN zjM_S97rk4;8V3PuodG)MvLGWHPDv7Bn_NPr1}qoU7{_zz>ldfNgCT=Stmo$P^(io^RMI0p&L*P`>m3Q7)WOH zv_(dgbF+W}dg6O}hxpsxRHWJ2P`4BAM9WOoPR#dJW~JZ_{up28dUc+S6nWCQUb-i% zLPia}6s(E#JnPzWuXKc=bL6C1c1E`7RdU7NQ1bg43MI_D=@2$a*P(>3EeaB-7u+;P znW0-wK`*m^hPJWquzysp3E3>)1yXqUNCawYvVkO6c0_Jl?=K-huc7bVh1|2Qk)!%! zirlw^)c8vLTb(#x=(dOX%6>_TPe1&GZHqOzZTE)@6@!F(1w@=5snWahPQ|RU+(itZ zd2eEDMJsXkg6ue6V+YEF z4=9wYod^^v{?Z8|KdyD1srnI)&39OBN+^M;?^o9#LuAO^(Z;EF_=HRl|Y*5uzG z?Y0|j?zh;}(f1)D`GHm7q1<1s2dq#p7?gmPZoef)L`q%t6U}{iGw@%lN8H=JU@!N^PhJm@7Pcr6?rP!3IQyG zIX;km7Z#(Tm&_KUscSMQva-GC&m@@Af#D}4#y%{tMPs1J=us5x2Jb&UNEKZZL-{1K z271wxC77DrdGo=D<*7^P`Gz7i<3S_(3js-7tf`r_8kY-XW627cpK_30jk^!M;?9J( zk#!Yi=wK4MeLL_tSr%YVd|Wct7>BkYea;QWlz4Ke_xDA21ABgR;LQ9vFw`3JEGc3` z$(@YiEde6|E?c(=Y_p$-v2V9s3cB~_L@GhEqQ59S!+&SRiF(DSj-7R3Bx}~(DPW#q pa!qb}wJ8A|V2v%U6H_(+<$m&?W|RLf2<%^!L;nALPVL*!{{!cjjK2T? literal 0 HcmV?d00001 diff --git a/docs/static/images/pcache-fileindex.jpg b/docs/static/images/pcache-fileindex.jpg new file mode 100644 index 0000000000000000000000000000000000000000..51f4e095ce67aa2878c248b111bee57eeee22cc9 GIT binary patch literal 54922 zcmbrl4LpX7!G1k_(B&XZO$`P+1pUBNOxT?RiD748FgG{YHW&=H7`8}l5lkIAQiJ^hy{-D^ zG1Nd!54P~<|6wp6wRQhG4piIn@AK2cU<>|v4%Fw*7jzY|pE-nw zy;v}LVac6o>0X@bHEiW#wO`cw)zwzQep#uezEVxq217tsU#Rx;^|KlDruK{af`yAT z7B5-43_79qSJ*FV>gvBNP+z$4$7R$~q3>Y}RxVuioB5taS{^4gevj6+xOlU8vEkmT zH>*8+1sivsI)7=&(lt73b@euFHrldvo2At*Ya3g;eftj_bUx&A*z2gbkMFVLenF>$ z&xC}Ag~!Ck#U~^tC8u4!ay9+h^&2_4w{GX<-?@A5VacP?$HXUPPhV8m)YjF%eAUp> z+ScCD`S#uWK5GBKAno(eFq6gpI{u9_F*zlinH7m8bJBU)54)i8{D)c4`#&uEPj*3r zp!Umx1?mele%PhDNEirifu`<-rLh0{!@*f{Rmyn?b`zEdOA9I!T|0^#vQY+#_0r&n(w)qyL>C2` zh1OHVOWVhg7YXp1tgVV97GF(j*Bhls5Uy*jn)1V6YeGMmymuxCQ>)8|4_&snkmyq_ zkC*OyD@_+FyzX@q@u+J@_oQ8-B{M69L&n3fAW z*%Z7fxDI|}_%dQC38xk+3rSZP8eRYH@LS}r@+XROQoi7fE&sc?;CSCUlZ96G*&_~e zVx8}SKjEGfX(zM7jBVBmT=bRUC>S>3x(c?W*;Enq>0KQ)xVJ_HTY=RU`o1PIOZM~W-(*Ebh9>j#+e( zea*@~AgCig*(Z%OHGE8Mj0AN$mxbLowlJNU-w~j}wC;fGGN56wk~>S&4+)kd7ddT9 zd@<@6A7Iea`mPjvXwb=cLeDvi)~Yw7uoLFDA+HoI5g_qj>wAj@#Oqeav5{j7gvT^= znHcHT{AoFxpa%7JdyEWc<==2v#|AWTT{z>x+6u$gBK^EhW?+f@l$5~KNO#h}+Dr2R zWX2fvEVyNAkfG=?HCwr3!X=TD?))F2_g_Or!Rd z0?!_1db^^ouz^;=3lRf?a>Y|(#h*-MXphJ1t}?&(x7(S#@ zS~4nqI2vHFP5|F1AStxDk_aK81z4sC{8TK>FvYWwHwZeQW`H!D->Mgxp4DW?MyDZv z!C8QPeqWqSKk*$7EbxoJ%OYGxEQ~|xB&vH=^;>Om@FKyBjZR`7fpzlJmMurL&MOlC~=&M<4pz!ifgxL`BgImP!7FwYSv5(6 zm_g<=EIL6z>5w_>+Lhckx`e6-R}=FuTgmaOrKnbeEsX-&wIyY069H_-I| z0$Xs^r@w(|PUn>6ln}TkZa@XoU`x%}OKSEoT}c^?>V!7Rs>z~_f-HP?lOv0~Yz)EE z7nHC^%g0v4wCa`evI~Gm3g;?m2=YSh68^BH=WwlXeM5Wqb=KP0xleH$;CW8FRIoSJ zHSjeflRD^X&UGhU+{ZivcpoL?i9&5GOKZ$AuR_q-nyiAoM$%K5^}Ofz3Og@3XbCl# z+SbXw*$yrO8M{RWW<8t=TuaWzteP?~)l&{AP>%yHoCkDNu(2XQyV+iGTtMoeWPp~+ z%QzEhPb*`Y8Myl0BkU;^>{6mMgN46f5<8ELG|N0|1)*zVM4Jj$7duQ!YudkHCQOe< zPb>*YJ5iV2cPvr_YLR7FUoVk=Eu$NqrItmMzm{!VK#b&9s5W4E%tH;>G z(iNuFKJYLIUny;}Uz)2&cLEiI#c>pq6G4-RLg->78W z@%u9QM?m0(#4o_}&h{miEB1|b7Ypa!P;nmuH%Y}w^1V{Z>ov#)aS@2?bECfrjXK!! zBZ(>)^V>zCCx?Bo>_I4DX;ZMUV=kb7QNa_eiBKcU!CufA&@7E8l@9t(SOWTCyLGT? za!sgJ%RS&V{<7U%Cvt@}UE-s>>k!TC(N%1ah7lDTD4%d!KL!h^@;%KO&^j_}9|E^= ziwuNjm%(38xIGL!b|Nu?iMZt8%HrY|CT_(ZsJIiWg6WX(yB(~V35r$gT9N8Fb*au6 z@(m@!2MKR-e2iTp)ndZ03nmduBdM)OEo2yD2!C7}$ws6h>oywvn%MpH^lw$o)Yne;q`C4fQjIof$-@Qfm^cCxbrD=BfOjH>i0oZgiW3cA z%e;wK&!$hy?wN5QMblS4$HxF~!&z^g67%@^Q0@7lDUAiDg)AlFP2j3N+=j#wS0N?4rf0=O1CWUgewY}b&?=FV z+6I&3$jz@Z8c6$j=Z^IV*L{h6+@$l}Hv* z1v`ir+3<^uzxiGh2T+*XiYQ_ovleOG3>9r(i15HGlzf zylLOlVf0X}SOt4h2FzShCVWQyl0&J*bmaqF*%xG$3U(GRGeuWhgTDjqNHEKcJMUA) zmf^wRbI?S7g;dgf6rSIm2MiT_h=;0R-!BtB6Y6Yr5$jd3GZ@8wh^zogy)+eUk77lH zf&CK|?A!5$Dp<_<1QqN(f+Gzy`ghZd%5?c@pf^T&mm<_^-l~F$!c{Qz`x^YZ{(MDO z@uV2KLp+xce`=ZwNZ~3N21Gx~m%}BUD%eAgd}k0{;w~jBD9l-zi-bjp|S6r=cD5CIO6s5B;g+Kg@pW)P-7s$ka&gh-Og&r3yAPD;uQ4!Pm+ccF+Kbf8Hx@D|iANM)#D4zpQj; zOnX314jPTyW<~en$k$}#W(HFQTL=Q@N?QN%{Ln)MNJ$t)TLoLLf^~#xQu2@F-XAQ# z{%E9`h*0R8f%_CH*t#|ef}w!M>M|hdfGp(7cdB44BdjMP8_BFevF`_w&PN;*YC}^Y z4hj-pLclT(kQh(w5TpNW^ALB%-U>__f%2envN66~r=`X-J z@UTcu2rPsqRLy+{D|rxMzLHXnJi-Qbty%eF$gd();49vj^*092@Kms_%LGhBJ?NPy zSHTEF?u88s9_Tg8pIZju8I70Kqd;SZ2zXfy{|34XT(@7u&oSrwZU0D_Ki}+)l%pI8 zW*GAnBoZ3M1`UNFvJR=#ff_(d!JY-Z9|Y=&<9^R#;+4ZRlD!IcK`fG|4b6iHZ&{;Q~8;@(7YN^{RgI>J2=hFKOUL5*A zg{ezns=mJibsl}luQN^%It*2V?>IA3qC7MOaUz~g#-Uhf;dwwd2SovW&xs`$hl$Ss z33~!1(72E|*BO+ZRyH|IvC3$enwCLpL|{Eg>yzLUDMa`CD%efBL+T>6s5@jWMKT!S z%`NAj18DPS=S1==DwqR&HWy6sJsQ|U1ty18uz}k{X+H6@VvthV*7cT87m3&^*kDZ7 z%Y_Z6rfCE*hoRuTOpgUqk;)_m#liQ8K_gl;8$c-`H5&(hXTMbn>8_MCq? z-*8HT%N|8%)$!(X%)U9=eqA{cott^_Vs^}@RS}QPydNvgX#Bte*~qS6pql`T*qQ7H z>*K$d&&IY-)qrGcI=Ey7mP8N171lS-PZWf;vx!uE*Q`vdBRTt3j=vGomT*HHuCiQ zFYw3zwb=46_pPG-oyq)1pF8p_A=`-{_`rB^H5usjQr0*g;xcC0Fw)o2&GtU;nkL!# zpxc#M^rci$)x=nQ85i;1`4ff1{jB*}JlTmoxfcwBb7$$C@$tst81_cf22S;WDg zr5!<5By*|M7xLHWaS-TH!Jan!`MsVlhH%?J7TDqCg9|{8MJQtfoQpaA@}T-ELmEwB zv#23RiWVpgM#N(B+f5*v1D!4_e{3>2GcVzli{zCsM->b)`v!4Q@!a_7D%v~9{h8S_ z_%7`b(m0O;15i}TotzKB3#To12JB$Szy`@=A9FsEOpYYECdKuBPA)2;1itFFQ==lT zE2?i#rcKU5533{H@IHVf`erct5plIBGo z9}>7!%1N+WBT5tH?qa(LvW!yEh)J~^5tBz=I~9!*uDaVxY-H?&E~?JM)Cy^;q+v$> zE`I_Fgd{_;+~%e-_=z8DgT6kNU*t^jo6Hyz%}~WYyyl{Kqghb`+7aLl=8+<>hBR9V zNT!?!nsJPHYQn(eK@;!moEEJvEF_^D>f}DbUBqeaz z^a<)HO)OKv0_xY3fu6w8m$ETgbdK*sIEvfzF8(`s&`cpyFnpWKt7kEH^2Nw}_g10U zNPp*BI#q`V445evf+*6_Rsotbz^A^Tr66l(onHh0X%bInzjdZ+zy2R5GE&w5cjmEs zALR!qpLbRSA*YMqa=>r`uUG^EdB!s+ElO5y1ZIR?N8_DL4!|X2D)7!~f&np}tHkM} ziaSL;>Qg(jqdN_Yl22cp#f7XPeO19oACnR$!xA#2L64D@HX?TYr_*{$7Y)K_mRuUXfUX6aO6^S#O1YlRhL3K}gn2a;c^U~ta-nT9{-&-E3bULH>HPVDZg|zjS}!`}ER>qrXHvHdT;5M$*9b0V>#< zW}avOIpaHkxGD$-w@W#453HB4{sztv8{UgENTiH$Uh~&swFT%4pp7C_>Q)@8f@wnG z45B9gvL#0A3-sp-hlDY%3Cc%WOia`|BE_4Qmiv!{`!qOTs|i@#wJOYMFZV7xIA6K< z3L;qdePZdgn3BEag@>LQ)=p8M^gWt-pTGU|?Rf(E!+o6n^_|-Tm#s|8HfSu>>*pT! zxRG{#ywJ%x%t-NZVt|i3ZWY4?R$vzkg8BIpiUE$gbCs-a7a4_A${L< zxL;IYrkD7*ZyoP{ zLn%fgP}Vi;Cnlf-oK^-s2$D{c#sbt^K@Dk)d`}|kHMq9d!F&v}R2tKo>~ovR&aICk_XAhj#_z1(|NNHGwwK}OZ;fmT^nBq~KQQrqE;TMBoVazQ?C{5T z7}K2a!G-{PdL^*|X#EYlKm{w7OQ;Iu*dvO-!$55eK5w^=I`y*WgIh9s>f1`-!=`o6 zs%gjJBL{~kL}BUA^OGV-kO(wy_OU}*^;{pc@31D6OW={5Tsa&*o3RFpr@{o9=9 z%La1M)~{>BoC~IEVhay3hDs~!vY|)0g*fF_KDt(9rm#S~ zj-m*jBO~2y+2B%FD1KNqk*E&3{}tQAEQ;x?17AO&eAByKTP=z9QP`nTj`~zj#Pj@w z%0?AT0--(Gpo!uLejrp6ApuGsYa`rT%SPNF)YVm$9|J!TYU|S~%n;xYC3WF~7dw7h z;`>ia)NnKg_**)P%0lf-Ff;qNcq6u!+$6) z`6nyKy+_DuRIo@maaK`2gVa*NGI9Z0Qv!JVisSZ011Y1BzWR=$Um+#zB^|*T$>7|g zRN=+O*)5K1n#l?CT#9eyN3nf6fO^nvcX1kCX@en3_h`qPrW!)go zfiY!=x)C*9nzOD4u2?1b>XT1c8z(ASl{@6}G*ou=wxH`^{{sxtO^9wIXij;2vp2p} zl((w)8)=h5KPa^{5d9`&Vw(DNu>Ojz5xHd5mXW1r5?^)Gi&mA%9HIR--q#@IgRJDMw)l^qdwToYEAFlo>$l>d5J~O;dDQV z-2(l1LtCxFQmw~$C?yK{MWg@b+fLodPQwkSu_c?EON`nS+mv}_$#au)D%cgv?(Hkk zSB!{V^(d#JfEA7Q#WC^#1^jCvq1u|~llq*YO$plhm)-h{beN_sy`g!yHW0hqEY)0_ zq8B$>$%`lf&XWuBM^Ddkyd6pqZvG(UF;-u0q1a6op_X6+shcUaSN+j+G`MuL-d59+ zEZRpU#0A5S)6VDn1axh|+}*t)`@;(0kRT9+)@qrzO6)DO>2GN~yd%=dZy>0|LN%N0G0>#LW!6R2M)QmmmTuTpL@P^_g)T zq!~2Yf1W~47?8ka`6`&P9j?rAK9~_Qi(Zt$p{AN*d*quHZd4Q88tKzr{m=R-Jac*2 z7Z~PuPanRdL&}fM!dq{!>t3sMd)w=;R~Nk$*c$vE8?tF{dEL$(SO3Z#EJCh!BVC-j z@0WV7F0bnL_U&IVA($}l(4wi@0}9gPdvuD~zDc~)?;cvY@N6T{HSI(O2G5;8XS@q%AHPf9y5JK3D?6r#+)Q3-<{}OYSIZ zDq+^qTa6RsLD+4z*|@;3PF%sS)Fi*JQmCA*$@$9J6hRk~43#J9a6F-6h! zc_YiqPM%?>*ryyW-#_R5)NDM}a&Tlm8F2(?>4%v4gmEw(M4uFM!RoZ!i*GvvV`e14 z|2}9|vPW~{`Kg4DSUod>o9GqCFOamRQT{u0-+p47&Dn{ZSJIC1jZqs9&H!Ts0<3t0 zMc#UgUz2Pc3~oyqI?U3`EIpBt&XR*`ox3(tPxOa%9p#bvFR0Dgftq#(Ju7W$s5REp zOx<{2YJ)GC%ymM4ZgWVh^Y2BS4 z0W}Du%5cMgB*tJh6tLSUbB*f|OXtB;e-8-<|Gyw1FjW07Ae@DwTGCrA3UqR#4+S-d zIgq>a$2`7Ql&8)t=OUbxmeih-y16QmUYs&!lF#p?J{PZ;&$RdF&_RP?{)^dYK0s4A z)q_O^tAHZA6n3@^o3Vfy@mP}!YEo$9BRp+LRv_8f&e?G8$LjV(MLdS1oJVCig960> z848)|70p5@%!P38eZwgcdVsR}shHON&0=JZ460zQJ)AQm6}rEQ4H)Q7#5I0T^TOEn zX(h5E5<~g!voV)w+hN;7g^Yk=?5CO^lRyN-CQ9Qqx{xpp#deCeVz2VicNHv^nD{MG zp&>X&F{dk53pSSktMS*)zFD|iTFy4cuS}E}Pt@uAzwyG_ToqW}+qlqi8>J!3!LgDl ze?OwEJ@5>T9@&?-X3}fe-^F50eX86fzuaR`CX4lVWg>_1V+pPJ3`)K84H6cLntnai z1Sz4@RdTz1kGjTyOolE_dphF}X|3Gfib2<6qO%pcYs!C1IkrG6{2gF*?^)Hb%K~4W zrK4wp#b~M9ZR{HK^)fw3at`*@z3`g7t=E0+>P-h98^ir#k21_0!VWLR%SN4wIq{=n0?nRo5c6}XND$9IYX>b@ZeP`j&KDA4iL^WHe{E0?Nb^E0< zw;9sOtF8yWGoSa?muGDhVcd%5!Ur99G8wNa>5%e>Jd3>Si(A@{Gal_CfggZcZM{nh zXQ@)iW*=yBIVnU*cIAxJWS}w$tFS2%%=w|partT40=C(ze^E0KmyeNpIeeb-BU8=kQ z1f#Cw45YW&jJ1gx?K;hnFvvs6!>1ZXzTXv=u=B5;t;CcnpM!SWX*(vIbfh;+LiK>c zs=#BS5sSX9Ar@Z_T@4FP5DA(v9lAb<75U@$JqW?l<&I02UWG5b$KLTd+`P=)>P*=m zrGDCzW&MSRbH+Ws-RBtbZEaIxg^3O4j!v?KMdvLwxHOXdaDp)u&GjOF~sphF_ZKjftxuTM&Ghs2Ks&y-iO$PlZmS%Q)d zy|}`T@=9**9u3|y`QfvEgPo(+i=0Bmd%YyW5?=;de8%y7RBC&8|Fd{I<2;)$ zC%n$%zHfVhw;b;=n@@@gb9?npN!$!>K1)()t&^BK;?^ZXlYOKW*N0R{I%ntDs|OIj z+KiJEJFi1Z==@z^)=LGu*y#Rxb~$haXN{6&wE6W!uIE+I{cBU=J-T`+W4DpdJJTyR zKkvn9C$^%hUk=3a;zKtmiz#8dbHbZ9)2z%^Ma)fals0|jgeQNmcrxXyZvWG&3|;Sl zEy5l>FL!Ne^|i_N?owa9q;9ACsion$6@Dd|<}S8_6`pb7J*+aHn6h2tTW!x&eV(X1 z#;&^Qd+S-1ci*@BuO5l+`Q9qHML51|NDuU|Zn{!~odS?q4l39rr$84fm`Ihn$Zc^g zj5^F^5T1Ajs`)@%$1TR#2{wY!!1u`dTs_-u}Td}{;Iucp*9 z`m**(%`m59?FanK4}GohEIl*jUFP|0gk)E{$+qjZ-`4KP#*I0?$(w>B;hz@RAWr~A z>63I)mDT;gj4lzcjzLJx1`6k9bl4p`!gA7v}~Mfgfc5r;9UE!7TD2tw)I( zN8_a>3T!?|4x)&f%!2w}NBWZ47)1dG2W1v^PMG)O2%LF9Tt2&)>--h1gLjH^j zmOguUl!B}D6w)TCnHN7V7ua%)vhO#{={HNvj46#8QYMB@vU-Gyuve)t2p5bs6(Y98*iTAC1fT9)75vz~sq z)UTVfPq2416x8qE_d7jN1#350dii-o&5Fsf{-dRVE_R7=Jt0qTyFH=Ky2r<4_{J7{ zFi=U3vHbbCRJ+JZ$!8gSe{T&*Gjtf0USqs0(s=);*sc}pocpFrRvw=m4$O~Oon3e` zVsT4*WJt(@U4ny)R&D8Bv}o(>VN{XZdr#~~@`-|ksPD8$ZhhdzsC)jweO)i=CuN9R z0K%#UdK?(M_V8Q!!D9J#T=xx|x9_n5%4eMpm5iq7oc3(k^)JQfeAd|(n-fuc4!ma^ zOM2?($X4b@C?(akqxY-hJroqYxl^aby9XoWO2nGR@TY$;-px9a;-5Vn65-Rt`{WOg z&FEwn`8&RNWqab1%Uh32U4<3m<4(p*S_<$+?l;Mq#SgRIcn`&dHPEP0d^b@Y=s8r= zsDr3wy#9{kPxr{Z>!Y%Su9BUdZsh2x2bzm;;c^Ezbl8PUylanT5y{>PQ7Uc_-%Ciy z?I}D8zLkpJg9J_`jQi%Dh?k>th^&(@RswIdKtg_isbr{hhGH_l zflHVb@gNS#sf-RVk8mGF-zJqxM(EwIA#L1JlrS$&wu&7YuD$P~QBT8DrM{om7v140 z0?bpwD5!R`W(IZVDJZ>onZ25i8TCweX@~L5*KeQ?$IOCOhP5ZtP=+58~j-(1o z*@a3Wq2XzLI~p}EYLDv$Hv;X59#B&vuQaL_g-{Ji&}!mB2@l&z|5lQC4j^=u<)Vc3 zDZA#9!-uD9M=Ea4V*a53=l?T}0i5n3&M3-1l`3o?d4HFS=ws-A4-F7(sQtalEH~=U z{2T-h;6H@`%_3!0#~xclT~cS0C&YnOu)r#YEKpnY9I*m@j7yn~YE0rrT_5fIgan%j zpExtNBF^f0Y+z9YM~UIgd=?=f5Oa*>rpC_(oUOc0d!RtN!=vmV3Y_(76U1lz$a&<{ zNF0bu==+*3RsSU9XMc(Po*Z9!hwM{_9w)IQAs?_%sj*fC6DP;>WWS8;NQT4_+T$X> zFLx`DB(99ICmDgGC>>8B5rG!Qh7@??5{5Paa z^ybifCO<9il}@MAju+jKpwaR{ntw9U@KvT`UG*3VVr-gm87@Z>G9&t*C~t2|j4TcI zt80+Ekj}$FvS--v`+;o(e#5=4J`A_~DG)>Dyb>cJs8fgy3V|bMAoh-O1l)G{D`-QQ zD2fU64dtC9zJouU@skD<{u2$ZmfN;NG#Haf`9XuPLK%1(FN#Jui@Fl%C^(T^lPP(g zq!r$kV)gKPP)**bEeD>5oT>=UAzkVcS{eEyrk@5`DAw$%*fR5{?SS5$r>ny~J!`xV z96sk2X1lp$4Hm8=M^ub=moB$DzU^4{ao^UH+-?JR!Hk3#kl?fg z>;TkH&!llFH4Tt#-jgej0*C)%>o1ipA}@x|b~3$Ca%wX{p3 zxh-Dxd3in!=j9H^w%uZEldL^mBR%&SQotuI^HWV(AlD#QFgq4}zEar=v|?^_e(XU=Im-#PrAUjrqLXi5+3Sb~loi zJaKG`uM5-obf)=Tu*k7++d1I)rTl8*;sE)!I?A?eTKFjcUfoTZqA{m${BjUPh0Sj; zYCLCZ`84L|8O)F?GvT5lVhnM^z)~Nd~bbiF9ib( zqxSbzhaaLh6r!~NARU$66{JpJhf1qJx}_vg18n3=pj6>_=M zGwxiC|NXMf^gsR>s7-x%TT#KsAP(&HY;^ZIpZ8^Oq#Rv9d|1toKgM|m)Gp&cpB$`mi?e8XOtJsCr4@cRXp2|o$)(;&+P~*`)C8q$!jYPrPu1lO z$|^>i-IdAQ%7)D&Y_S=%A{~Jqq95n#2w+f1Va^S|e*;ERoS!z#4onaC+WYnWwH}IU zj|5TNTh)n}_fGXX@pisM(yM~y+9{tkhDbuB{7*5RE&SCn+4ez*o83DDVw-Ky^CeHkqr;-1 zh>+2i6y(9wolrs*Jv%vApEeLSLqWGv0i)pan<WEo8CH$K+T_I)VV+g6dW(Rj}-Qc{6+R-lzY*h#$%;tN*=Huz%w(?)?I# zMmv(euRq%nAbKmHnv%K9f6_BkhF~=LnCSUBZ+;?O1=~D4k`gI%omfeIjO=03^(otZ z$DeaSfY{^Km~dX?#B%^KuCRNv4_eVnHtd`^H@Cw+I$%daU@C;Dex^#h z<2f#nS5%~Ewtf~;Kf`LN8gcVKsOm>X!kOV~2o4wTUZ_Or+f+Uw|KMR`ys6=-@1Ki$ zAobNgneHZbN>Y|6JV?hAot>qTW040_%~;MIN}So%K={D+Yx0CE&uW*g8*%L<{93&B zq?W=-u&9t_wkj;f{c1O--3QeDM2yS#uX9P=zCLJcL#^S*!Tc5sQvrT`;DD-NS2-He&uPzMpq0ocxGr>+bfh+xbq{v2R>619GW1 z{ac~hG%lU!dU1Y}knitiCYcj@mp}JNl^cWI<=wV)VS?nvAGX6WvtQL-;RS6Utj#_O z4IEFJ-(9hWC%%5B%RihZIUE?tebLC%^%MSJXa*2N8yg`%M(B(`ElZ!Jv4(6T1zpoh z!!adFdHkIdIr*zv_Y#NGh$y|?n!~aW1T6tqnm~otP$lVT_WBh8xA`<&CEY)PL!2}z zh#=K~G(dg+9cqXmDQACHcqlV>JFs;-)i2mZ-;l45M2F?Su})a;hIDkiohnJDM_LXK zI**KNalEaD%@t6|1ar>GK$(vXQs$2_GLnHJuW6`qtX`q*JFf8{U!qi&ma><_g`B^3 zPwaV1SL(dEiy5>*Z>s3RVfDm)rHA5O-O)3RbMU9F9-I|D%M;I-f~S>5wR0 zu^n32T|-=vV=|O#@w#Bsk?<#HTn0=7viN4sWYlmx6|i?Lfs*DUR5Z7lY&9u4JH%x? zFQw6ekm`iSgkhW`jz;}1w!qPV^>?)HkJUN*1a&OY{Rr%e-tU;dDnwg+cJcBaEKVx+Zjh04X;sA%WqH=1jrSIxrXDU zl7>b;Dd1FJbwcv-wwowS=MTn8yVQM#!Y7g2|5&* z8L`!QR$eqB<3=o*RwBgISvmf3#6zP(Lb_){M$@V3q=ail(pb* z*l5BWxLOb*p}oeZoz4=V!iu>*d|`QqBIp-6xVyIQeGbOJ9-uh8*GuB+8?s31H~Q(oI_-xYZ0&UR~b-n6Gd7b zZeW|s+V0zIpv*&t7^iMV{h~MvRhHjg1r0<7P}@3E@YzXQu}5%Juj!fO)3wUDP<}{_ zo6x@tv%GI(Rrb|Yt|obg3oo<{T*0Q4dg)wz!+P55NxZ;0O1xI$6U#RB?k;oRhPvGq zY&e$q#v`$*w!b*NVw3C2ciZ~6_}n+M^OZzBTX!k^%NM-=X8!5%38z)c3~=H5-&RzD z)P0ws^(jAx8y2uNGLsNB%TvzTKw`ZqOAS;0o_IAJ(mPkzze#KH64h^7Odk~1l0qKZ zJokDu55%PVhpl%q*u@i2eHiWyWUT~zDb_){S|A^r<6RM4;=i%Ow-*dy_sGq0{eMg? z6BA^G+ntU zs%(2h8u4(UJuB$r^jJy4^i%%(k~*?uu99uv4c)iksaHNQ=jDo;wyM1eNL$Y zEs2)$F7Efl2$pk}!kY=#L$Q!rxPC@mUHP*gbs6h%4pLwDmlT4eO*+Z>;x7eS2wIjX zcvwgLp6$Jp4c@nQp0KR9Fzm6M>7rg`J8kdUa#CQMuqx!(dIJPJJ!|=99gFrqie20@ zx1ol5iB^lZ9=%UAJ^G5c)n(+}D`eB zL%xz|B9QJyJv^n^$&-!ED;_JHQo#ZfFBq$ue%;dzmQS z$Gy+x@BtenCo^FpIn`F8TX~J_rASEUa3~gmO1_`vU*^hVj*9RlAjpeg=BI>s1qT9J#Sj{Os@f?j6p=H+>i5-q6U0@f*VFd$*46DEZ8b@BL<5V zTL3yk=CuQ$p*zb&184t6Q2w{-43GRPD962De>w)zo>7ud;tpq$qw)>9qsbd|f*$&R zMCCs@p^!(bCwSL1U({~^p&F&tqPLD!*%sC~UmYM9DguUxYr$-Fq;s#lLTN1_r1(wE z%><>BI7)zo@CewE5~~1=t!I+OFHTdYM&$Jskbt4#dhAZBBV43jU)7D=4kMeVI4LQv z!*hPdm1I-HH#KcO74VqFl|*-+4d<8u_d{DDCxR$->RoLi9qTi$91IQOs zG4?aHTXW-Ij5L}Vi|382sS1rT5?o;|@V(-o#`HhXsl#5mlkxoEr!E1FVGn%{TUS}Q z9N6@tH&(1@DbV-Z z{8k~y-)*NvA`G~nvjD258Sp@TWh_%kmhtMas8e<&dW01 zhQ2y}YE!{+^g;IevVD&$XI}W#_u9YeoYk~%OIhD#los#3L)uf2b$Cg#!at>_Ca@BD zNQ(4AHrR@K1`iNz{?S zuKlUsP)>2a6PkTvL%a060M3-XmUfVc{0AFpVBHAk{5S55=me`tA0 zT(zX?$W0!+M@@1TL7wV?(kPf3jy*%zW7etvk#tJuXEZgi0*q6r_lyHXM@sov-nt z3An98-1nuTJtO}0$q1IWT9<0*Kj>ifw5NEgPOr%>aQvl?B$* z#hva1?VqevqbQnw30EuaxzVF6F`O;ML!z&s0Lp-49_NRYUXX^%TK&Kp^}` z%(VYwOU&)#_~ulkZZ%AkpeN?st%?htF`p z2=+0ZGqI?0Gu}nH?+_4{nBF$}%cr* z<}R8!i3=K=HQ36s(~%r6O&pq(H`&+nt8eSj+(r}?-<1schKPH#>nhl3%1}aE^Tq)& z^4G+SHgq-djY$HUCNAfaef|-Ei1EcnBTbJaroV)(wp+yMdMdX zw(c6=rcKWsc}MldKpY|^RpQ(W{HSs#;0Ag$NRZa=Gt~}lo1rTD&uHLLturDXb%vw+ zLl~CKkiiI9qfbqmuv79tsxRQMIJ_QcE9JU(!llL^)g{N3_j#gM4l9HNwoyyg^C^T5 zGUB*ty9ij?lsfjZHo+_;>STeh|Ipc%2EVSexgYdd&E;MB@0;OfZT(r;gzBQ5YH1xu zi8{;4?>x$Gl=#J#?8+_E^SpeTtz$X=Cq9h@jaOAtOqufvpTPGNZLYLp*2eH+=JtfxfU;7|PGK*0C@dGWQ4Q)||M zcs{^y)mNy;*9mjyY-}*#atB9DukpyxdKH*$Pi3IZ<9Jy zdIf)BVZ=M2`GB$#L}C-ARE%~!Vl@zS-6R_yCd_5|;wv1G zJFWT|X9=x>$$Yy@sfAX$Q4ectvuX5hWlmRUirvvkN=0YE5cRU@z_vkY3e=;<0Yp`o zwvJt95F){M?Uv1?>hT*s)Ku)fc#j#<5R!?xh>aA;Sqaw^9xQZrS499gM1Yi7*K3NM zau1Mdp#JI4e=D&ykP@rKVx&75J*3ma?BcRsr5)D9}Z`8bW_sdVR|{mSp`#1u{WGBQJ4HF zg)<{`MLn8R=$eCg;ukJ%qj0?>4_SxdWh}Sg7VQwODPu9Trn1x6Rf}d%2u=gL&+vW7b%~al37k4b}%_?h=cu;-IUkq@G zO-Fqj&+r=>X%!8lP-1=Vi!6;wvgu@YK29798h^J@u*A4aBZLb*B{X+{7}0@7n}Er( zh|0Qn7Zbh1s408=EdQn~725s*)%YV4P>s8lQ8h#J)+y)&TI#B^w{VUxyWs}uxzsP3 z{?xC~t`3rzo&pr|A}o(k{FO2i#do}V0+K|+fFFuT0P^b)Xn#LQ-)W8XS;&HDx_b5< zqs;nXc(#73G+7aV^FbR$UQ0ouxb>Vl#1VLxLZnWt;#6%g6GrMhgr=`D8RaTh&wVW+`+N6qpX`J0%#(-2CwIB7f4g3%;g7V+gZXMR zhhZxdn?{R6G5bAj%Wv0gL#nu>!C+P@j|!rjF4sHeukLAfc$t{vy3QqcjqO&!)^}YQ zmwh@6SdHV(_YAi^zjo(E?7D5qt*8r){6&ih{gAEoboPyx6W*KJT@#c`iWH(=Y5JA}JVsSM_^ zNT9{^1ziFvTLrn%7t#O;5yT-y=3UGxgc5vH=(yXS?+WM8TP6|wBAJKuXPHBp1fB0T zG6@~&hpP>u)Kijist+$8Q2ZF&sR6=09G|$z({a}b3GVu}wA#y3XO8>YO37sw>_@PM z$}UqH%!U<3x4W>1WTzxnQtheip9IK;)RmL!n$Jm^s|3Vk2Nr|sDY?$ia1w}97WCGd zm|I6Md2y!RO3!5*fW15pq7lj%jajFgO!b>6gVrM7KJqGvs<})42CFN{U_0F}Cu|H= zV)@s-!*~Km!giz~Bh-mqTr4@qcWCHGlyGPB1u8A@9RlMPRJ+zC$&)-{oG{DoJRf7j z9$iOJBAY>dur(MIoAD^Bg`r|@$2X=T_>RbUSm}5k-ygjql;|pnUNUYvF*JnO9ExO( ziJLYYrP`8T1Q7*K?!{*0VOd>s1$eTiCwx%zQr$vE!H8&xv|X2VBl99L=mC z2d0m`u-ty>WkP%4dQI2Ztg!Q2e?snx&x_5`>5efiO+L_DplW|1*Z%yP%ASn#N9PUS z(>DC}ZLlINDaQBtuNjA^dH$)FW9Gi&Pv4!= zYej%!z7dB3Me{2+0VvK^V1Zb39&jfBgXIteKZN3;xUagP2L)&v6h}dlCs;UvfjMAF z&(L-N!8BqWZov`8XE2m;lX%RnWt1=mClfBp;AOa`H7w*3><$r90Z{1DF7QY9@xH%E!Y9oGIGWJ)QH>cvv-5={XOM=rlUsX@|cg#eiNtQH*bbOXFHFY zx5WjT+d5=iAB%0de)DM#);HMalY5$$%H@c1ejn5KDe9CUe+|+y9Z^vk^5|~%!PYx3 zMywN3bQKd5$L@VAJXHqM!|c~d6!RsuMrWpO9Jq8oXkvSLVs!41)8mNiKg0I>5az;Q z8i<)eil_2Xux}E9Cpw(1Mazapi7e=lnbV&Pxha(gng2932INle;4B5R(f@-Hcdj3!_9x<_%1U>`lHw?XVw;0raz-)KK_wN5`7Z z;T;~Sc>if=l(|HxY&@LZ@_1P9CG7C1As?1ttPitp8>PS)y6 zD_~IV#qSC|0dRucjD51_wf5Z%Upaotc(t^dwryGAT^VoNanT5G3 zhG`tu-ScXgTx>p=>s4psnN|4iu8WHuNiSDxUh{7NCx**CBB9dU&wZKc>6V`$xL>~2 znInVckDjsK(h!h&Q#m4kbRwT!?A0ELwM&rrOMS+~VqC%az-YbWAhNWa=2?*zDW1+I ziG%l412q=y;|O+QS&ilO6_$V~&NfTQNxJE$xYKDTieso=yBNGLqC7r#0{4|=afpl3 z3z{r?iWToC7?({iYUM65>dNS6coowV&9D+`65X0_*POUwe-~N`N`?BfKkw|ty_Zuc zK4F1LK!cJN_wWbNiH>O~VYIgRhKcLm_v{e!t-1ET0d(x$^`^ZTCcIp6($TC&Vyr!x2eZ3vWq_ML zJPo-FbRgfPV^t@?m$#9sAnoktCruv^92ja13-uh1I1=1YLVyQY&q8v!AJL{PGKivI zw`VDEI&4=>V5)oAlw{jf6bUOO_7OY3%{tvtk2UUW{y|gx)}59 zz>U(YjY?_EJa_w;)lSqKKFqhRWiM^tv(FppH1|F|cvY)f}0W+m|Eszs{qX7JqN8?pV2(n-8*5SI0T5uA(28MiFoXl9685f-zu(rgJSbWNr{u3F@-InU2^$1?{yHx~EG&UBhcV<&F_ z^qt6>6Kd2PPumU~df~6S=TnKyR>?3A?_s!hQ=Y@N2ThmionrSCVzwc@;-mFUCE$cB zM6Q7!k~pySqqrDXrq8#N<2!eu&(senSJiS7%iO^6?S+?D$=UU?Xp9RFRx8<{j@ z7c+yw`Q^KZp5b7%LE;0W8N0J*HaO8u%f9kq$HXP;x6IPCbEC;lhgNW)jgmz^yM=6l zXQn&}WT8|vPcrp9sCOk^&F(7&c~QZ8`6-D4)h5_FFWRqgoiAlkqIKZb4@KES_4vF; zwdPA+iDd|`$~umzE%Dxp>GTc|!xd=NhdIsK;9&PEVg*mL8*YgvXXfV9O4*;ZZAEy*)tXvE3xV01hL z8i#0J@gCG<%PcD*F*)zx8Y@E%`~(-p(AF>BCV|K+Fczg;yQ z=8Wv^E92_)pS!H*kv*k89lG1;g@T!>!uNjYGvYRg^1wEyD?O0ujq20U=On?b?+QQ2 z>;#9>G&f0b0vOYc*J5MVjMe2A;e%OG$DfU6z*b8-e9FPryXTZXqh1znZX!V0F~EU>WTfe1Af1<5KYQ)y<{_@jz4CtN$8yll}AOj7mrn! zlnTTxc+IUKs%W#H_8C4oH}b8_Rua#yaV{fn2*q2K^z8>(@VRp zzDIo1N3ufL8{lTRjlOA19eF-_T$%c_hWn9?zfFXo;*y zO9?dyVUc$MHn2ryS=W;w(1vViFx=p_X5`WOiXgA1XL=EGd(A6R?BeJ~APV$RcSbP0c<$O=Jsrba73!{qG)0`3jYd}w2Fk>RS!nRtnJe%>E z*mIWV2CIhgf5Y`SS==$!r2ITA6@6qYq~!^N2g7;uQw7kI@&_!!{5jMxeYUP9gyHA6 z1=_4U5_l)v-+W;rrLNfH;Y8#U&Rw29#fvnryU-O(vzm>Q?bYyV z_g?|+fxl)_-~&p3?8}$C!Q)Au3}mCz)LevOvk}qIy>MtasE%R*UXFa1{J!}bfiB=) zB(Kr4G7CPjDLl}b?(K9@6}C|rs(GN$9O;ER-1TO_n}7B=!H0OJAdnS0HunSh_cv2w zLr_3U#&@Sqy{BSNAV5i+5C8gOPWvoN zw?7*3hMa0{(Zp}^sC;!Lp(6F`Lf`*v4V7?`Tj;^lFiFrd4Mu>9Tf&2h&9%-D93a{5V39AmZxkLK z3+)%ijae+9c<$lOu%B^Da)VS>i{YdK)k68_MT&M*w;T%*_!!7_mmJ~}0rw?B6Wymf ze|ambR&ggVd*-jHssYhGq7W;lS=Cf54XzRX6Z@qFwvzrsn3_JqQ)Mt4>J)vIzbUic z$s6R1)^XoNy^$b91{vqN^bV8NCH~&Wl8^d%h^+d82ZQ8CcWm$OaPxBQ6Z-hQiZ9<~ zzqB}BgO?`$)X47rhYBF-9JHzKTQFqw)8~)&8h2Nu0?aL854i_`o zVLEtUt}oLuo$gd2Mz(37RWX|-&9HQsh%SHgW_o8kh?>5(;>75YIFVKKrkJ$>HJZ%R zPSaJkSI0ixZi6Z1h#g&KM$!Yv=6q>&Wefci6V7ie&K~2dft?i3#^VQCn(sND6VmJ= zW_bg5jT1#W@>JI2F#p#el~+!9Y@Oe~GMjZqV#C}U5uet-4&|ACV|_}6sGpC%b>&*Z zk++nI4p4k*eI9d#ZdIka;rlr*Lkt=gkK*D^F)70zfO~<3hcZ%$6JmlrV5{aK+_Qj_ z3(}e;Y)x>)Al|4{3d$P%q|T@&AvKSJ8(X};>;wMGNx+#$dKn6yh>9cz!N7&X6&%j- z6lAc74T-5L*fv;cnJ%m`5gBc4;LE-ssgXK=n()nG;Tmx9_NrwlBkP;pV?BC<+RsN$ z2C;@&ov{0@0Vtiq=bj72&prs}SH|T#Y|vCvKH*S0*U2Y zLmDP^P0~&Hm-yM!SG=v-FE<+#B?S~cKiV1L_yQviw?<~gr;8dSX9J2bSV_I$l?77d zA%Ak7qhyjPzLWuZbNv)K4x3uq6j}iTin_Yzta9?mgZJV^XoRV_Q+< z)@0egwATA$vgg})bI-VVLho&VdDZv~Nop%K(`}L-db5?;`AO5d*}ePevl?UVy%d$Y zJQ#tz7FTZ}+bK}}MUuzPlPpTlg~E~Bv3-n+eo`QlvPzQ5^drUwwbk(GJK67P@vmJ` zjMe5=1&!!{pzh&ncaCXPVq!d|%3kI0-la@pbINh%3`OyY=are28kChWFEuQ5sx)m9 z3s$;h25dVaP$+tqj_lD4Y+u5?2*yw)gS5y{WzW_mMx! z%%oslaUhJ%{T1l9H-)7i6aU#X=7_}qoSAWymVzn3Dxz;Lc`UzB%3pr}` zuQQ}unX9e9yD2g8I∓+GZs1Y2N-EHEp0k%h{T<%f0KI-eVW7f$s_i^e;&P)Ey?H znlDaFt}#k-Pn@Uap&QGGrS2C*3$hxVd%^WaqC}|4wlMkFq8xr#0i25jzliW4`Zws_ zO)Safwk)HLSi>R5AOFQPmADA9vYrJs&Isus#lnI%(enqTy@LCux*(4cSybk7EHxb* z-1>`#grT<7#R^=?L{DA=W1DQCo_bA*G-A?r?*gtaw|_nkKjb2Y%BN3r=DUIFn}8)~ zO%d!G7nYQV4Zg^Ll+E?zAaQ;%jDaOQ^m_p4FuVVhlYneH+)YN?6g@Yh(r(nMAGzdU zKxl%Hzc2I!J^t^)<^MN@3v8jU^XdjF@ryB5{=wwqa^Z5%y+*bK8r0X^YoU3vZWr-A zDbaseOn0@KB@7tXBR2^tuwj`)7jag0g5S~KEpCK%r+a@+qijKQ`@IW~)xo=eJ;M*R zz5wnhnPW~zS)J4_wyZx(-~8taSe#@y>kb6d&S`v}zEvPQty5O~9DeR@@1F)^ZJ za$+01@zqmfYVqga&tK-*7jo}@J@)S1U}_&gSx}2tAu7V)Sn5Dev!=GZ7VwaEsn8Jt ztc9T<(P|n}nJGl`TiHxJ)erYETU{XJHTtci`moKAJ3_VV-L**srzor5U_F@KO+isI z+Asf#uhYFDXL4}_>kn9#rjh&s#ZS+J$!pTnJ&)y0H(h>K z>cg%2u5cX|Z_1NxsS%CKpO%~9He)v04dHDiQ%piMSw&Dm;JaFOlQ()3wva)hJF`~T zi%$`G)zaWwFh}`tBAc$iS;BCVTpzVll~n9(5x^c{E{5e#f=u;(B2$0;{UF443_^Md z8bt6ru&V@!nK%}2umtu1xACTAP+-Nj^_!lEdfYd0_%v5ZopOjkd_ z-9Qja@nEA>LzjBE?nfmrnEFv7Q>O;&$h*|b-7G;$sd04BeU(p?jZhn6BhHI3;4hlP z4emh>B+N4@vdugy2E?S^+R^l1Lrab(v;2}B5 zW~>Y~{>WG@1e~DAh4%pC{B}xHk-tBS{bd7X8-S(}wH9n-bhu$C3Rg{OnevV5;UwK}lyz8e#{<_aMT1Scf-C;D*5(z>C-mwq;DAYJ;wlq*!3a zj>0yJ-IYpf@T0S(e0$tr9+R9eK%A}Up)j`!RN^qkeg&BDK0O5^;APN{BW&T4k3w|Z zrWx`kjHkdU+JVKBZPcfl2J;#jQH(9{{l4%6V3>zo8rCULW&82&1C_40fxKT}++;?H zA+M^yH$lg`Fza$8ajst$-jFJ5dY*9a6__Xu5tM`*kfKLmX+VJngBgh@3X4Wn9i0q6-YTzrIKCUYx#g@*4$)n4S3;Y?fO(tZdv!5lk<8_tj{GfO*GzP?tM#Y|4~`z z%7iw-K#1wnXBf@nF~1()pO|kM#BmUn>2zo;!HPN44`}tx*ux+$ck+LKG6&K^#KRU4 z09ycWcmr`y-MjeOE09O-2r z^wX4!RkG1rbBm3Hro+K)gt&`71_Na-+~($bLHX|_o1;sdGij0MIl>NoLF^8%bvLhE z&Da(7*`($;^D}-+`mHP4(fP5T4AkBitTRV_t9MnhGf&%p`RRu{(TEb?`P#GUS1M0M zBQ}nr72i#0Ae)}Q2dz1E45RWm0J)KQ6jN~I%2H^@PYq1N+S^AZ?_NY_o8~E>R$6Zs zYx;V8z8cil(mEO4d{e{N$ z-o?#Zee>0IULCmb!*>Op-h9*2ESJ)(^Ut((p6OhAk~wYb*BY4l?Q8Cdb5-Z<{XSLD z{k}q<^K7%!gC@K3)?Ug#b!^H$4=6ZI?<6FF{2lwGpH}CpwH&lGIJfulFX?}L_(SE% zvCc*F)D#V1Ed@ba<^dh>gV4~fl%p15yxFyoE{i5@0YIjEsS2JFC6m#Gmc8v)(l<`qQ-~EZkaFHYAm?M;5Sd7N+7sGv@=0+!=7Rz7PG_Vdi1( zVPf_n;OjR(AXV#Gs%AulEb4BdQ#vDq$s|9ne_aLKSdRZgNpwQ~G!;NCsSmK36U>kp zSB-@e->$7lQ%r!y|J1>fph7T`=6S|%w#kUdi z(46lIZ&;D$!1O!SL&;|kRgMa+U4fFET`GN?R^ntw5lTeT>I4&TdVmmU0I(xz?!%^6 zn`AE~Q-z1Kq~W0@U6}`Z5uSv>&*5I5B&AZB$J)CbJOo# zchB9>8Lzf5iatt;>~Al$e^yg(Y&d_|6c`n#+Ho@3%ft%Ctsq<{mN0y-DiSuzb&y|V z^zKF&lLNkD8u{9UtXR~{U5s5l(Vx?4Sw*7z;p#*f8E~~q-8kYJXq{V7O;`Hw!;yKS zYRS%dFRz&{spDqD%q;>d(b1X6pB0PltT5?Noen%t?$OH~OU#vYHMx}+*w9hjuhOsS z7^s^*XJtLZW4!5iqOiS%-xW$iWRH1Ufe)h%J2(L(f~xEaggO6C#9c#o%b%b*73d`& zhY-2IGnOK$Rl5lOK195stZ+EoY%}(6&%#$AWU_{a+yKC4oGf6Z*!e&2)U4yYDV5I{ zlRh*o;yxPz)sn-$rM7u^I*bUD`wz?SM)K9)sUSJBKGH~M1zwHv&y2L&#z+LQ$G=*Z z6)nO|pj$dR$HZ5VP%&`p@KY-P)COrk!)_NMh=AdWlbHrpYY`cYiQJ`ocDyn1YuE%y z0y(o8MqNt5{snUjVEIhse!;n#=8sbuhC~;BNSl zb3gD&PX&sk{9Pf;no<@AJuQ^zb58A}Rgkgh0v=-){^DQEt-*$ zvIc0aPK!WWTKb$%f{Ht5UMxV!U!5pX-xWgQ-cuHc_%ZQ66qx?i@dlW4OfIZ>M3Yu~ zkhRm`?WCr5SFoK5Dm}s`@V0(gq;E^4u|uix_CQy+kB8qEdEIYE1+4shQugQmgv(a$ zwX4jwrFnb^OIua&GPz9!3T9D=6-9R-?mY;Pnzn_<9c7tJTYav?Wgv3+kkX? zxCfj1K^52jBm#?IUWo9?vkKPXTA~)CSznO#|Pq+vjTgFZ`|qJ$ zELgIUe_FfId}i z`z-J0v+mf(uaQEO*1(P>*^bALW^%Oj$( zeYtPn%?bP5bFH)XPc!_;@?%z_!%B)#ZAU_{QuG@y){ow~bddSs`L7lx-7n)_TCUxi zZ=bg{NA1+Xy^rTZZub=hzWql&d~8HS#IrM14|hhVcpu)R9l!5VW%-N4dUpd!t{%l6s7e$LePRuccI180i z$Y7cu!|PpLKb`)n7X)*>l&?^HXd13j+%QT5!C@f(>u{URxkd35cBpJ5pEvAH)YY0p zhyo=)2_A|?7v(Q_T!dfe5b8`ZMV?JwH!?VB>J^Rf*in|H+&{r2#_cC{ANy3EI=J%^ z&9i6W&%LhJ2ZOWFSNg`vW4HGPTT92Y=a92WaqHH6X*8h`Z5Q&L9fDj^kL?)(VDK=$ zfw*7#X$FhRxUp~Tvn{#jeGM}sg$~FF%SD~;bHd2rYHM$nq-GS~qSo}1;zt?s^*@SX zC_ycXl-Gz7^D`O>uXTj}f2=ZEdUG+Y>Ms2#wSJ0!sA^gy`JgQC6S`&b^g9B74ZlkJ zcW@NF|r?)&xRy{4-8?w4g$MT#_ZO23on>e{1FJm$h<288km_ z_rJL8x4Oqe7C57K!ai|T4Mb`4qU3$mUmV3{66S}NN01QuD@Y8nu9}e-8iC|2&0vp0 zeTY^5&;e)|yRY{>*${I{axnnVcY{b#$O8{aAc~Kj-Qb3X^q_wKTYV0!)COd{|IM=F zzG;cMpK=@HCyXz;wd!Rn?gY$)^`T{EBH-EN4a045`A(ZsUrjdTnwR$_#H#F!%e!dP zu%Wsuzm-+BZ)Z88Gqh7Bsvx;dJHDZzcB-o>;$TP7e$Dn1-Q(vj5G{|6`y!hD;w8q~ zCqcyLb4)1f2&IC@by#5*FeV8U8JPXzZaGD{d7|Tv<+*)-ev3+wUVRtq>^(QOkOOI0 zS}Y9BH(v@7)jVP{!l6#_h2`Ut_rPPnL89}T@US&znI|9CIZZiUL#83 z3!~M8Moi>cG88MnR%)TMuBIwDb*pbloxHi{^r7vsBdrY1QSz3Lw|4j|?r=;#cZnF) zc;4o6{?@NPr*s$LUs@_6Uatyw()v5cqEF);p4BwzDM$g5DK zs7ao?AU`T@)-)DmectdA`zNhX`w>9&>LHr zi!7dqE(`1uWN!QzqRfX?XGc{Rr3z4@M83XkUwQ|DnQVBaKL{C%E3;}w#JQ_h_r0&( z0aoX_(CbEf`IMM4*NDXhw6f2366u0<=xt_Nm^C&m<9hyRsk!6!r&oQxn(-0X=V(4q>liYECWyW38^MVCLLE zkqLE_@eaz(k*^*yAqRO9pmg`*K)bo{#sd*Aqi9W4bN2olgGc_Y!FT6-QV_ROlR;GN zZAM8d`!8dAC;Kg@hd6vM%GIMFY$>$i54a}}PbgNHw3u$ZV`UraJC#_TO)#Si(X=8kItQ0X z@}fNH7e}}jRin1B(SzZPkeNW~!QUBRj_tOoUd&;Bfy(TY?7B;lUsgb{*$!Af)EU#? zHegR{du;5ITQSTHd6R%h9|!$kO=LbIOFT@92=gy0yY+YfXDAjd@dHwFO!U5>1G1kj zjo3Jh(%g)=<=F*p6%^}WG0w0(pMo8UJWh&4rVslwbc?4b#oDqLsWXXG8C~V07Wcct z-n@qU$^$bI>?*Y%C>@zR$f~LnDY^XF$;~eb4A=?g%#n+b|J(;!$Pw#D?@hJX1yU$Z?=4(g$p_{4z*$A~T2+5^=ZaXnN_~6tKY|Ij^R%7i4$(9c<;bwMehM z58#aD6v~K`xBb1z;JnUS75?^a6MJpd{$E-gk;cYUs}0C~IS2P~r*GBf*L~G`%B7v* zz3bAQKPBsyHKr76QVo_ZGVp`pA-1{ZigiPK`p$Y+k;!cKyw$ z7A53!U5yfXwq3j=n2RQ4F~-KL$TY&kf)Yv=#1#px=j+;!RZFe7fqdgkE^r|EHue(g zmis|Pigb54`^VqAuo=hSO?y5?M;Zjt7rUB#g>#WJHG?MdSsOUo2mQ?s$BP_t9&~rh zfFG7W1Jq#n$C@Bu=o=bU+!(XMD6EXebrv<=S z9t~g=Q)xTN?$7@bc)1^$0~$oSXHU`6+-r*fpe*i0-VW#?hiS@DZ;tnn@MEX~q~F4P zF@lg&y~qJutE@c2mHCebl7iB=14u{;5Gu%Y7~61!FEmdj>5y9{bFl!2)UG-@$So)0 z^up_&l&8t{QEgKcBu!02dEn2FJL)?vl&ZN(#9%Tf@9z;xb{WtQxK9^AXF4;!Z1i~_ zDBUkd;Kz_w5B}q9srY|&w)|U@=Fay}{+H@>2{0BwZa2RqA~s>%aNHd#S%;_~)u=v< z=w9PfV^xsGb?t{q0~S_Knov(KC{{|TyA-r(20$q!;j8|4 zg)6}mDay(!wpkCpEzeG~(4#NOW^XMmadNrX=_%(rbb-s^G=vra@)66L8Js6l5A5sDdl*2DEKIlyJ62VW&VZB>4!YYl zZt+i8Pj+OyXku0MZUzd<&Z=GDMtGm;t`Ny1ad!0#4747mO~n|K3d!j}Wuk}&B063Zyg_9e*MN>so^alIAq_W}n%lIU6&Sd!_to&L zo=p#0iw)Xz+5D}=!FI^lgyQ0Wr9<#A%2cYDjZh!d7`4zh`) zx7T&pM?bOdd?A_!6>GLSn+J+mzc8S9Kn%uFQ@_OkkGJo0 zWrv;SW>Qshw-Op5I-Ow-%Lw>1z1Z0<8oa!|MzZEkypqM!guM$YvDlO4{AbLoD=%KDFB0=5hW7cs- zW4o8YL3ASQ@G(a)*4U0|)Z0D1hpAQLc8}vmI#XT&btJT&z9-gf!JUseL{y%1xG>)0 zFkUuo?YoWZAJ*DWPLl;&4UQcFdLgfEDgl9e$M&JPVdrs=t0Rwj#mm;oV~*)-Od__( zJXE+Fq`!pfLz{6!HKLLJF@9%MIWhWz1a-cq7XgOoiJiKp3_=q}X^(_&3un=q5iQnS ze|bM^{JO>`g)o&JKcp!s#WY$*>wK%S)p?bC{sIwx+frM}rIub` zfA{JZ3w(27=i87cy@Y5BlYvk7mer-?W*(oOAxYFznvf`Fl=X&ThGz`^{9nq?6v? zD8qM>_igXC6RWxe1NkQQF=n;I=nn{Z8P}JgV@A|zwN9*L<6rtQ0W4L4O7iRipdYvT zE>W9cyCis){v%#6Dog1Me>|mA|52^eyPa2Nj#;n1R_ zRqe!!IuKF<#+h==c2>U=bVE9=}Qw{s4c?WH6=bmQ1?oM*J8*=aB zd0p43t@xW|Rv(svxhc4`FcUWADtmW&i}FwiR2L*idle+bfulPj;sF>c$?eI%2yjYp zI|#o~lpDhH0HXw1^JGD%Qbr^T+-0Nry>CxGj)OsMoUu8sB)D&+zf_Bo#-kyPimHbk z&*^o;Z_CRl-#RELnDqGaJrgH$VQdSZ5W=!`E289F{iVeXdh)TUsqX&a*uM z9u$apVz}{8gH}`ZorquWP0^GxuLVqoY3*qDQH?-IVDIRCejpO`6A#4l@oI$O1Y_Bc zAbcskZVRH*y?kK+Cicaaf(Vfp(UA6>EcFfN{cr!}bmxC>F62L5 zJ&1o?J#Qf6vjbcBbJ`@Xed8tFWnpcw&XFUq__B^ zdp?);hLA#{TS_?|J+QyiYS?+Flvis6{?^yGinwSYo|lr4!AWD3u^a}ptH71$cZqyl zjxQ8rsfdynK$HyF4EHRn2Hx5a(9{V|pVhDej0y39e{V+O<|j>XS-fk>ai^oTQ8hDI zpNvMvJFe(86vDVS%pI$)?s^UxR!}pj;`>}~8JniPW5$iTpK|rnJB%r3(I=-5m1n&` zLWaznFMIbE!>6V6;JQV(RBJh47{7hS-MnUo{fq>b9^vIs2gM4D!13pl+H;l}8IL@Wq4 zu{`!~Y^Zt<&&z*=qF3HS$}Nf(K@=Eg&T0(SBKHj9(Wx!lQ|MUwa%N{-zQXfm`~g)H zC4>9ly>y_nT}QFR{ZNm_hrSYDn_N)O7VaOCcs$jeDz74;tT`sg5g93^K&nAK5a-Mk zpl9<>B5&cF>k8$QU;i#s9FTRs)1GNoJ1EbI*>K6U|mS z-2`?bm!+Tz#**z^kLz+~CqgmoN9|>DBOnuyUl-&_mUm1Q%K|GOE3|E1Lww!gEB&=3 zd8oR%XK^G&%S~1T(1Q@(U7vZW9bdnbgtzgmAUeF4KcRfXhvq>YuA(o9#Y)1i;%o$! zxc64A#%cMt2su&-m#rKpqzTF*|D0KnI#}nB4C(F!{k=1=67+wtBG8aCk19K^Prai6 zJE-}*)7IA1_-u&uPp)kJzPjz(<8pfZ<-yfAQX7`YK6-_s48|o{#kr~eiJrtS0SN)& z+VfBui8(iU+R^VG1Ox?$T969G$qwYuVGnjp26n6(d21A0l`73injttn!WpSg#C=Lico5b0>m6Iu<92rK;n#!}6x>wa#N0hYR2#BI9^8kD zXc>;cXtx7&QTC#f>j4&Zmw@qI;W9dboIMP(ZvA&aBA~F#`weVnvK&LR#cwUQ6*<`< z)9S+A3Sn;gZM?|ic?jAcf5IMh?oi5O|K@PTtRNV~)DVaN$nAktCQ2zmGvYEVtpl+F z@h`D%k#Tb}&Q!^F1+mLtr^OhU{+T&1A+*iGGm5%T{eO&V`bSg+{9I3`?3NX<$>9bw zL>Ix*&XalpT{mSjNSlOqv6TdKDoeuWz&{wO^`AJZfF}HNmjQ!pYmfr30_p6pW^0xS zJ}f7~s%NigX@yWjBhoZjVb0PpeL8Uv6t2~bytHh;J)ZJlsA4z|t?jA*C&1`BN8?)7 zF$S$fUcHBaOp&sTodPjGz)qbf)zQn?X;9YYA@+Y_r-Z?smua~GJ9YCr^rkz~o%bgCiWsFsLrbanpS#9n_bn1$|OBR>8BDb!^exKBH=Oya_?bjp_57DVRVb*)O z3@o~aa=r3fqUezPw>`_6?o`oLIidS6O*gZLiHPo+-S6tHR~TKoY5bRp{{Amz-T9N zy@?Ae%5h$X8-=olxw~S(@i(CapU{|3i8{e;2eXeSr3^ceIGs+a{>RX6>@ld9v5JU2 zfc*9-Rn5cM8kB|XOH{4MK#bPA#~%76Va znu=#K2I`BW{sfNInYiPlN_v&^ePT`B*}Ly{c)o1ApSz0;@Kafx>=Hesg3AcIJJ@1w zq7!P|c3aPci@MYdID;onTv}09>N!aSAIP^hPyl{vtUmEKetP9E{8WwW2S4D0`{5YC zPj?~`2*qum;7Q8~Zfh|6tJB@iTs;&LHtb^D&35 z4*9`Rmoj43p#+AEdR6`fqB7_d0HSh}Ofe%T=i7dHJ;M$uEc5zf z*!q*znK8Ymt`dTqyVNaGWKmjAD5X8ZD86?0c2}2|;CyOqOhKq0)=W_N*})rDvWrc5 z1m7FVZbELtI0cn_#kO{z|J_|u&}`byt$PLA(>la5(>F*Nf4J zRHt7Y9!3XEOl{2>7h-7sI>K;xAy*kp^o!&zP2Eg&mp=mqHgP*f>+yUG_E2waK+z;) zJH?yw1FpgO{S>6uEzCllm3T?dVC*GM={s!I(W@f;E72IgjxPiO&8Mr-%0Bn7&nK;mi%y?CbCL| z?SRskCoc7aOJK;yJZT}l=8=$62+5roWSWaM3U&hXe=9VFJ8;rmgbQKG^+cWkf&Uh1 z?Djh5|6z>JUn72&(|w@-5@m(|@_{Y-Q!)&0#7^k}F)*Ej=!i&!X6UX+Cw`|IZse}Y z()@GqKSqx(%t_1Xr~mWrfSyy)^F4D5&~=Jt)Rf#AyLPx7D}EBN@JVT6c6B`!AVt5O zJfWC!SH1crNhlUdd2=rDGv5_D7|B_Lw%K96OO+m2L;VWTnKrnMf6Ri8% zJC{_RUi;<;iI~%%$h;gmc@ep|r@DgFW5cVdks_Zjt7dBaP9V$}kx*pBu_W{kS6rbQ zqNz%}gFabx!kjzX9!p0S-Rv`psvAjR;Kzpjy2S{9n96Wt<=OZ?P9e#a@oZp$Fz^KW zwNqpXBndYpN~tm<_pNLSQ*Hn%0pn7}BFqcE;&V3Ee(1B*ERay>n8YIAV+x%8-Mi#rBx8^h^sow6A+Pf!02;vo4rXWeyasHCO+ z>jCnze!$D|+k<&mnX|m`=iT2nu6PDb(2JI$Vu?!Xn0{uA;`}VB}AuatFKq|Lhw4Kv0j)2J^V6qHQA2eWl4dFUE^n0F!()Gr7{OD-L8D|M(;?Y2D|5s=P@S--Iw=@il2U0P*+Q&G`i|%5@(meQk)NQAa#PCTkcA6 z#Gs`IBpnvIA935tL{EH3@^akhtHza&s2SkH6y$IA!CG?EuuATgErG}`jcsYQR%>rw zF2rJ1>P2egjL{**^3>NkXKe{a*(LcUxcNEc2+3|tHX8|SbnBG$7r6}S1P!XbYo-j2 zwTK4*GX>2b9&Gz4&_l|(7m0l`f_v)xxU|e^3YFD4V;rQ9{lP9&`7Djfqp!jwWiC?6 zC#c3s=66COc>q{c)?JK9WX@FPndTC!lAmf4+E(xDbM5aO3ycjQnL^GTG#n$OLS9kL zaKBX)iABgUI3aZ&sAxv(IcUxj^nT{+UD$#M!r)eO= z*&b0*e7I(+!}cUNWDtcEm27gj+m?N~;2>}b7txXF_;%!SWlH0EfFovomw^6PX<9w1 zT&qE_9sDWcPN%pG!5Kry^#F7Fnp_9A(P*J7-OFDd=78_;hVxC@0gWqWb%Bv}ZKVSlfMueHAG`95WG zBagZG;#AQTS8IB|U#EsFctW=X3~WjN&u7<0>j z#67O!X*+ei_E#`o%tdBzj0IJw$FwFS_zPj)|ql%?hqu$`z+?kEtTu&t+F zCm#&X0C(y3Vo0YIcWtX?-^Uq&O^mVK!}5GW=j)0q1}~*~d);UUHIm7Rk4TASkD8sE z&*`)tD|0EGcLpespKh?Pq3^XrfS5{~QHRv~UWla&M&2E+iI&Q%U(&^MwFNr;%}v8A&iT_mJ>-k<=dC7PR^{lh_AZ+~ zGabpX2!3-kGVrlrM>5{)-;qxIl-=k4k-P(Sl!PuU=MF zev>V30hp2LGi!KqZ0t3onNsIi@9-(pHbyS5P z?qGj@YIKpqraa13g9+hr)^1VBPw##S!qTqYd_VCz;_4OW<$0q8w=(sJ#F~NZngU5j zAI5giqjp1dRL0>lUF`37ryYtg43c)4Rh8^Z)!675r?BV{fHDMn1M-6+(0f))K-^Xy zDS(5wB^jb1?-?-dk_70g-@Xu)ARkyFeGB&FvZe82=bdgyJKhXdU{Pe!@n+1=KRon@ zhSK8&rJ4;uI@Fi_GHp37M_0vgIooKxxTnCy-RNanCf?osd3*jWM#F=91nYoqEci7o zhw!efkchGS#6(<_e~(q~wNQ{A@1?1J(FDE|Nu?)qFm0L#vYN+F6VKN0U8rT)bNtyv z+QEn?U{QyZMA418nyy*R7J35~og339#|-;MTKxSpwluETtPgrWp9)THQ^{nj7$vq2 zhAv3daUEH&`kk_M#;r5c$(SaJb|uIO|ieVGE;jgYZW+ z`(hN>)n}a7>t54aLE{na3$O(YX^gSd)3vx(wSXKHx{j@}E{#d(wjfTA1c9(53P^FX zH4T*c`(J^w0((`=BQS1gOw2=S4J>7=7a&YYpAj8oynK?L3UkYjP3#pdY6YPli z66!}kqQ-jUsz)e&t*VU?`swfg!`SlwHqD3rTCnB6wybZ}aF^r=a185{RZ7A&>r+o; z&+3Ulwk~(CqAc@BX^igz)(_e713u@l9WCvSYFq{ zXg)qbWRfRE$;r8vH<5CgK7;upPoG=KZ;Xn1hywbWViAKk?@(^sSGMm4Oz zDwwF{gpb;K)KR+vd}K4h8;x1(tjEga0F`%zJXm0Vgn`sgM`p>TIAeif);R5jG(dekr zzA>i9{zNDKd`yF4z}SgYT+Nw{wKtdONG(Povzi?q^5HX8DnqkqWTdUZ4K@Q*qw{U% zQQo}M-)GJx^&*IC1$1S?IZIrFsQ|V(0s0K9pP&AYDF$SK4C1WU7~0=x!++#fE(qR$ zI(+!L3$U;}r z`rIf3cK%P{N6aO2|6AfE`r*M^;kxNzcm^$Xo=a;8O|tG9oAcf^i~bG0>9-jW z25rxSgX*mO(h{H4V$g@F*b$@Rs5&yj)_))O9%~mIrxjsgg|#BNi=ZjvSZY%DcTyK# z0j0u5F;4!IkG&hMXzPQgIQeW-?OMuA))aT6->x5?y6}KX6H)Z%8>`4e_Mw9>UpZ4W za?~F?`#3?%%W2#{r+e)TtaJWB?sPex+R>Y~RL`hHL0QNUiBU}NB5N{NagtH3_HH8E z^UanCF=WFam*-UN1xl5x($gX;J8+ba?w6}cukaL`x)%|{o`T3EhiX2#5x`@%CZz<{ zp05fR)v6~Q9*S&4BeeLe?8%@HDbUeW?_DLF?&C#8LsWnM|wrDWfnDKosqUMKJS<@K{{oTKY@zIBMtO=m>O zV1f0i9BVsVuTzstP+ zvus{1wsNNm#0dn(Q@gM>{_pkQVR>VFV&@aX+v5P_YTRZ*f#)8C-x=1f0_eR%gj4 zQJ&y8(ST-+N`H`BLH?dFoMuM*#M_L>weM>a4b1HhHT5=4Uk$f*_UcX{LB2;!gC_YJ zL5*3ox;hTi6;AEYIY*>?4j$B%a*XL|o&)BCwb)a_8uT#6UY2imVFI409{<2XAG?u9 zikp&=p}-W?OFdYm#9aD!ej(kBmBMYZ-Ef7s6fZf4>oyK^!>7`YVE>-!;VQT)!W;DgiHC|@ zOH@5YOfYyEjU3(B@pIuYMt;11k>ZoPv9`L;VDUynZ9n|rTUzZZ>D_7HE~}}#;a5m@ z?e>*UzKv%06N%p=zOXI9AJt{qizP=wHCdJ~=i>!6ON4wWG|XA)+Xp?h!VOMx}PNk67toIXoq~HT11i1$mnw^Nu`xiq@w8 zRUxQg0qhcXmu8Cibi(I4l)NFF;W^jA{po)NM-hK5aP!})Mu4G$EtB!BYFsr-RX)?| z$_v~}X}$v-BqKl6)5et;D|E~$JX&w0T2=$Jh@oG&7~fhOo7}a3XW*e*Z!R*xR#Fxd zKF2)Y7loLEa=UHi%V(4CcJF^XCz3yB7e;o4o`;xX1`pM6C{fYQg7%0mpDw_d2n-?yS&V9LTW{PzSi5yVKOk zw(S+=N%n3Do0x#^#7u4Q4vo$}>pJ3fIwq`@0mW>w8{xdHlK953h0jl}H;_aJq{7jF zkw8Y7jD-hq7fWp`{2ff?Hg-AJl2ZckRaK2h_6I-@|4q&~F1aiDki3gY+KTS^q(X?s z?fTPiIXHNFHHX_!UPl}E5D$s1ovu%0rf_vLIb;ffXoC*4EW?waNd#if$=Gpujs3S}f2OT_D!i(}0@)7Sa+ML? zisdFzzigYx%@$~2k2JUl=ur0>XIZQ4)ZpM~39G!Up|qM|Bk2RQp=G_o|D%#T4im3( zhoH)Pqp?pJPXNxEU_oTgReRv<-hEqppl?f$1(DZ<*iT<`jUex3)AzBL$CibYXl@Z) z9F44AAp`ref}VoPRS1N=ABr*jAL&q^_7BvOJ3*sAm+%j4sgxVKGT^tCk|9sAnT7uE zGE@-hDoKHTD?`2iy$t=beYe8@a#V%=wa87&H~%a{|8-FPkG75X3Kw6F`y*4T+6oC% zfh89C3_(t~54N{|t~aG7=zPjDvTr=);5b;0JV#mkza8K`@@AwrO${5s#}OiPq#c?k zrv-Y1&_9ZAn@5Gi$acp@HRJIP$6`HnIP3BfNLDv9z&@13TA5?)M?wL|mV!2F6#pAS z`*#GOc$uuUDJm+G?A|R~UI&X&)tn56%bF`J3~~WRcpiGIm@!&O3~Hodck)*SeBm)- zmWI{--_*MI+gSuWvmoNW5;K#^6qsRsAQ&tozu= z{t(H~fx1fCoBD?wuKp6I&?RnaaAwn?`%{~)wCN)c1w7oQQJkIW+w!)c zE7RBAy*%Ts1>GV?QYeUKg}$t(SYSj-uh3Fs<{Et%-z+2;JBU=R(L1E7OryQiHT8Vx z-7x$hBSfN<)GoAwrqB;e(p$()z$c;piA{9q3oD+a#Vwl)v6UVq4K};gA56J#Gt%C) zOXVf$QT548kDpP-B}&(HqTPItFdm(7Ggf4Hk43Hdevy`ni3+$u*ftA;2&RC?-&n+J zn}U9`fNk9+aFyjw_LtTB^9aqvT^EZCxGm(7ma>@&!rw)`ytKJvBU+T+?%i+Z$zk3X ztxEg$9(?<;9GO0I3{;?VGuBnRc0E6L|NRtCvv7^@?-1QYB^Jp(~?i|tsfWm~jYj9O(?Rz4ohE-U?N}!CUum;`sMdECYj9z&6L#%qkFFZ0w6*ExMuYNO z1l>nf0Q-zqlpcj~0PVrDWRfVoh+#40$G1zuU}e$v;(yF4BiD>$nX;yCv|K=#T{qrO zr_8&{m>XU$Aa?Z0l*q*KTI1_>CV!f)b%o zd(WF<>19zK8=fdmjBlck<{mcjYIw*>zSUR8jtRaJ`Jy6q-pYQ)=ND<9`A5II8-t(C z8oeIEYF%#IrGJ_C7L8epI-coDy5LgeP+Xwlwzf zQ`4IgNz(pxJe%>f+Evg1RJ+Pm;2vCs;8{9FnlK z-SVd}wsh@JnZ@&#&{;SZlZGCoaL#~`n+qeld*eL68Q7BMO_24t<{;>5E zZziaD! zVl}bT^^z$9Eds}FRGBI2sCYeag~{*;t*o%}>2p|TO{hxKxIj_$(X){yZbMi2#}qot z*_MN3dCv1LM>Yg+2aweJ5(!B=2)`dIOxTV)~OdLG+=hDpf>8N zZZGMKV+mBk#8;MKs?xce-?&Luz5O1GA{XbzE(ZT<73yB*x*j;-3p`FK_>W`ZQ*H6M zgYv;BLW-HOR==BT5*?)4wNBoqs$Iw+=lNB^)BuLiaKfE8UUPFgpHY7SIEJ(ALCZSf zWR2P<-`c?(=*FP{p6Sb0W`uf8z|OqujOJVZL7i61O*(hdP><}hZxz^*xQ$l%Ylvg) z(sOY)(g16rxoIjBJy~+Wjp-U5Mipa^TXgVnB%=Q8tF0en?zAa>zt59D`RvGdG}e(8 ztjYS?HyVd|J!jTHKk}_E*)|kCK1V9H!kI=_QT*FO1;i271lqI;*M*?oI-?lNKQB10 zuq_@2mmECvljlsJcf!2na-Q;ZDrwF;z9o#7d6K$HBaQ#cm;v5MN%U@8{aTLzE@sa2 zmFy`3a7$l>!6;--4w%{T{88yD$jMm0?kM9}&==n-F34NrSEV%eCwDAsCgz0nTRjI1 z5JB650}#8BaRG{uq`I;Oo+m@P0^)bqB=m`InV=n0lClb!zYua!UXGp}Qcg>reVm(} z?YWhq2x>xJj45KiJA&NurK&ntOq#Ah+EpWVSRB9&=c@3k8zGS%w<*U`kjEp%(|rQJ z!yqWD^bI%|OX}G}At$5M7$@OFm6NlnSU9{1`Qs9yiwn1UQhpBx%Qo4BTfr zc%WSU}wm!SJA$b!*f=hst!8^PwhYZw`Clk397B|MUOWId=lp~ z(y7BoSQm!+QPs~jPy+%oz4`UJ(t3^Z!esy_RHo|{JT_i1|khC2E+>`_844J;k-O}j}a`eakud*ACCx%Jo$#b0IALq98w?~=1CsPKtjcKoiaoeJc@1$D>FiEHbm^_Xu%_TOBcfxcozPA{h$BuiwYD9)(OA`8_p zko(wvJwzx)Y$IkVyajvhoD3tR)tZl05B`ZPZxG|cqs_VXA%kw)_U#y@-JKKd6STq7 zLt0A8vv7DiK}bKW4PiR`m3`EKiD&oX2X21}5oo$E6p5g6AY% zHA16>^`jA#SgR57w9$l|3ZLb>#w~@QlOd&FdF%XhqTT|&VXA+qhW~j~MIB$8xaS3V zek$hJv)hNg^@}6A%XMomzahG2sO~zK?8j&+g)c1>59vD+Dk6R@dTXuv(9#au^T5f% z6)}1FFqHWIX35l>(~%Q-ZbzNW2?iIg^##oeet*gsp6g)ae1p+;DDd=TfJoryf8?il zL9$5ka94gLy#iRuhVfUx1i8Jt{mk6G?y*rk;R`Qf(NH!0{MIBQ{FM-qcL+CQ2 zp!!(edy(6zsuR2Ppp(A1Q*K`sT&4$d(|NNKfh`RhQ^I(;E+9cx@>W{&8lf%n16Y%u zld!*G)%frx2gPRmZmBlku&C5eOiI9=kzQ5(s_+Z&+e4DB4h>zKs|BN(Nab{Ot&U&b z?)oD2qi1Sk)8^NG`FU}`I=;^GD zjcM7Fm9^!8i=>CNZSp>}@&UeK;)HT4Yy$_YRf@F{)$n&&Rq*2%nQYcBQ4**K9#Z+p zR)h+CHN~=79nP_z=sI!L#Dc`u>pI?jrHWUTD9F+__b+K2KXnWvKL6X&CApV-z9 zwc8(&)H;XkAF606KFH4d@JH75<$*e@YX>Wx2Li<5%x*E~4PZ<*8c1$Jypo} zTxH9}V+fJ6%~yrzuxa5_UrcG}{2yNx){UqVx}Ku+0INz{rW!a*szR`L$#rl<{`1d# z2EIvQhsTcGJqtp$6o1m7FPNP3bK*Us8q33kXA;=8#uds{-zMFg&|c|f0{0>H1mT?KHrB4Wc>OxspI9C$A} z@{yASynHu?1%p5&K@+q?x@n3YU)%NZq2!3vk}`1nBtlh$lQd`f#9% z2aPy7QeI7WlVI~?jRa9KG}j&lc?eI>V*{tx;b@VCJIsz*_*Z&ze}%$|FvXpe-sH_L zuJZ|kD$(#1=R~tQD~_gyYeQEel8wq>jTvHRR${2l{)1EA9OB6;b(N}`(g^5cmSe5y z4T^{3^}gF?OvlhyZjI_*(i1Ho_Frx~IeA%SFN5T`vvTv%9B(b_m`j)%jmXg6`F^?qwhb+OlcuEv*I-kWm#)0-en6W zu!p!)5exwi+AbnDY3{CK1Yps?JJKCwRSfX`x}-*v%A6c^*sgiEf~9BTlfaHvYr59W zZw~<}qFz@$fZUpp_lz+^+BERq{)2gr3H5rq?um$c&26Z@>O*6v8+XpwKS>JMIlY!f z=4&e_S)CrsZI2 zrbxc2^S5f0Q?L802-=g*_P>U;&deph^4*hR;$PZ1b=okr3uZG&F*tago@8C#y;$E8 zZMkocVeyNLIk#;{4_ksJ^Bl5D{Fa<<&Mmhd4+s{e@{LTp2718_2G2|uL3x9C_%1Q}z z7sdEa&4tEjH&#S|GmB0CPXkm$g=vJNdvE9&gM;(|{nbKH?S9bd&nKO`Lqa_`w)>4E z>DmqNy%RLri_IUiZ|LT}w>CV66#ky==x6AF8dP5>_CGxq42PcnIz~9*CK+rHza?5l zn6lxGumqeIpKxFLH@@R!TdLU&-n>v^I;Hr~4914Gh@Lj*ZtTa6zVdz9)*{wSc{MhV zQ?HXV`a*5Z-j|LHcfeO0F8%l6G8k56Q0Vwo#PR8>P&LZj|D4IX=Utec64< z0|kl^!5((-OK&S8UyfR}EPq*KR78(qs){?9b2!H~i~aIZbft2$4I*9#)(VT4EAhiy z!n}Cejobv3DG&aTrWf%Lp9qdP`^`^w|E+Thlj3PF9~qSDmAm^8hl^rluWlE_4N*V2 z?%_p+9lZdV7vGqaymj^;{5*}<*FD>vMaU4|wJNmE4iSQ(xFB^O zk&IOhX1EI^s3N`|oBNI&tBQj$04L5@WXpRAR1PV}MQ%+~$Y%795$Y?=W&V*MenamM zHxcn_-|V18S4^qz9#_QhT<~WPzs%73(L?oBu1DWqcR74K0(jbAAu=-k*BM@F3Y#xM;<~-x(GKv9 zX75W8>|^}A$@jd%x+&QyqpT~u!jvhP$O@`3LhQJDRzF-d(;(-wIGsv){K-}!v3IoU z^cIK`lB>9vbK?hx5E}Vfe~ONW?i2ihC$V#fO9eAgPrH@SnWqBUSM<=;w>m7202c&L z`i2E{VLZr@ca?YHb}*#|rwRh6{FR$o`{ZgqZuI!_lO9$f{lDq;J3DcWL?L1Y;E;NhCAcbDg9}Llh+Rw6&Oa8w6b=Gy1KU#UvomoVDl z0jb9nT6qlfdVWOpA(*`5!x8b0Osc!P2OQbM^*4&HdmAEoWvDfhtD$n_T>hP*M28Y1 zJC%bM^B&Aw+wFMZ)xp-O(cX5I=6$HIF6kfbP3S1Ic64Y+HGP`?xeNMH8YcRgEiuq@ z;K5pv4^c)8DoYI|HS6I@wfIVnsDl-QW>BHXPtux+zqeYF19Pp9wdcpg$5Og1TL`)8 zm66n}z%6sak<~$~0=MfP7w7~0-T4F7Dn)PKJ3qW~_F_-yp+5?l@Duj7 zr_w5Fqm3G>7e<+u;@rS8922dy3fm%e<2%m~*{TfO7I_a&L6%v6HAC!@Aq+=sw>Y>! zS2Q#K1NG&xMcqDEoNAne9Useb=r^2w({E5#7CLOOnpLB}__E*B_l9nb!~OJ=3zNk+ zr#=;U*`;ARtR$IsLy^N;hh+Vqm!@~L>v8jJW;kQz3*_;TopJ!GR+QCZ&qvaIq2G>a zH|l}Gn94)6D#sVow^D6wnvFGgFeu&oXPZy+a%6xLYhWnV^Yku{iJRdz4-!zY``QDX zLW*6&jk~svU>$2lQODq)`epO>GXa{jbRReW^1#rQQ%|qAcObgID(Rs z+lQ@!wF?+wR(|QL4NS{*?U$cs<@)j=4ljuv7%$)KN3RTT%ZaIXoMi2Z=sqiY@e%_D zW$7uecHup=?Y2}YoNwGxof!{$WWETH)OEI(rU^@k(y9ih`9nO^<07XVr!bGkztOe!tu#p}ONwJo(N>NYmA(>bqvq}p+~TGRDVd0LITwNA&!!7$&2P|P86 zU2V0jj#+VY9?i3d50R{ySS8XYTp#OuG&nog8}mI;J#yV9e(+?xRQFOKZY%I_4FNyY z*q;A6_h^!J75{?(aQ)2olfR5>#B*F#QR-ACwWQ{@zkVsvX)aEAcYo%KYp1QVy^zEG z&Ib0*cL|lB10S0lPtF&FSyc@ajxW_N0Q;YCt}sVRrveEbCyyl(x3wX5OLy_g;R;f> zN|AwK2`JoATfs&;#odRJ>WhN;aU;xk)nZn{4SaLha>V(m_;`0A+Pfb&=3jDzkxKLr zjEOEi;v(D!cBQ_oBA<&vcOMlw5a^veNl$>lZ*Ty0Hk4`WU&6AHqv)A<#{Y=RbX6>) zlNo{Nr?1WFWD$bt-<=}|;_Y@@83h51Ti~EI6bg)6Ok@aMNp_a}!YaMR^;>J6UBvGM zm7zxiZe?y{?ZadR*%{hxxVNcYqd#HzcpMQcH&o1c|q~)Q4<6nM})K@?7 z;7V$-g!=1Rs@#tpz%RkrYPBDJVgj-QH}J#n(T+$j@PU)g)p zTYTp$H}&EfkvKo_E!p&VEP~e>gPhVL#+zwOX`JB4x8jrmJGVhE z3wRhkF;q&m)r`JKQ&m1k+eRVXzaQb3_U2@lJP-s`zk@fo|f?AwjCzEU zRBsFz!b7`<@UBmtz(#Z@7dkxNjoI(8ClAP zd@^jNjyUew<@qw5zU(i3 zX+^ix_CZTOpZ`H|O)J5kw8-n%Vayb9SX`$wd7a+fT@l+deG5L2_n!5rS+GD;rpw=q zUie)8dIFP=MX{70ju1%TdXq@AFk;4?tuKMjh zVxy#xz=hlJfWxUb_P^^At#xM+Tbo3b&^%CDeb-SHg_3P zjd>{$P8L4Cu_w{|bA>@S-XQTf{o%`vBgJRxvt~kaxEWOqWn@{-ewW{Ubt|y#T7Jm& zwnZUxT?`?*N$k-gHp&8EjSdUr79wxZ{<%k>Q#50}c}drvvV}=l^Ucy7HMOGj0;!V> z;6YxZn~3M29+2je~Yv906!eOM*miSDftnV5+BpC{ajiAj^tR+RFN zsNKP@7$sXA8rzv@mk0C_gA2WlY#;W|YJGxjS4zhi>-sNSIl`Bb$LY^ zEiFV|Xj%C1)U6tWE*Pu9=SdGrPR_rUkK+*EgttF-0hmX=RfL&ryB{BG2E5^wflOpU z@Mr+5&)Qb+%x`KGTihf2xe4RR*c}p!i12qn!zTss;U!)}Yrze8LK($;r~k&r&Sp1h z{jVsGi}5Bokpn~ufqQ0d(i4H%Vm58`qmiSE$y?%Bhp&aDj*i8pTWnuiXYZJB&AN9^ zgt~5-0u+TAtPS-srPy2i8R{QWoqAC_Fv=?zU69yDKx|YIPp?tE4PY8!&}OsqJSDgI z$&YxpUc;q`;;B(rsz~P*@Lsy#iSBvX_X~e|g%rolV;CBQF4bPEI*|Fo?s(~|zW0NT z*7uw3P8OT=Hd~IoHS|Hjw-XJBCkh)xtW2Hvr^X30z;nZ8nnA0Q#vs*TJZo|yQ+@yJPJGWK}Lehv9f<%ei>hi{s8JCSUXm?GcLzmNh*nJQ7gW=AeN0*8umH*#f0$C%yw+wAW{G4efkr5M%tY&aYd7}=j?qaSd^rCk2kQZ z0ji`kHqV{z3j)#6F_b9N@vjP+Ibrro(Uwhck+xyU1b*|_9O{sWZ=70hUnE7!ey1Ip z*E~uS#RPB5J0eSyA03stP7=EvKOIAu`Fx_z?Yguswj zSWsv+1=GabnMd%RiSSLq?!MzHRed2@Xxo|H#SRb8+?YRderWlv3(0Cc<+AgI0yEwF zbuCsFS1*1lTF~pbcA6rzUFZ{=hNlsS=cfy*hT53)iIk{d-zePE3D$^4v)H+iI0@f_ z)!&ZcVQ<74*i0IyAiU8-vlDx2@*;AP_gNJ0cO^jCFDhtT44#B`g9ddHoQjV(>9;t> ztT&&+KM*g9QTL{73=?i4Bkhv{>`O?MwVC9^(gGEi6lWDV<4C#l$C_QqeNPh&EyCuO zeGOBEj9MuiK4QD;QYBBGl<%W{CnrR<>oQ|C>2JYi1mo`*m#L3BhHQf^*2@oXZ&J(f zRcqJJ`W$3)!qvgW@P6Bl^a9Q05H5Q3V_}Y^M=^L5xYGIUbHN>E^6pVbqBcT5vc|uwQ)B$k@wD0R*R_v$EOcsX zjABNZRaLT3OZKev6p|AIXfV!_OMoi9EXo1=mJ3+W*J}9Ppu&s@_(spPKr0YIeqLQo z2ChQIb0of&9Gaak!0C7cfxKc!pCvWIrKemuSi}q%Y2Tl}Dui}VL92Yrb?L&3U-zmIqz9%= z;p-Wbrf_k%9D0(-@dHNxBSg_Y`uZc&+PON+z?M7T#hQLjAOB=wI zZsCSttD$rXxDYZ?W{&vm3uDeX%kXIi6{Xc+YrE44L}@n567EuNA4w`Df*&bD5XZqt z26AG-GxU&HLzAl_@X)$k14+|TBiZA0sHYN+7HY+PRhYJy*99INags-Z-6=>}belD* zC{L{uq=6Be<0HN*WWJF{Z-$1if&SX_fDPb!yoF_w3_BEUP3%TUx19y=LYCy0uL`|P zTpv*sW1$V~J6(WnsM~qHPDAkWm~N<+eRqWm$UQCHh*^UCho7OzO8RZ^aE;LtYw&?p z(H{uHqb#ZTGlwZT0p`LB9FWm(Isz@fo+Yewa({=KyyZja)zGgB6tWzKeuR+r!)DhR z@-A@WUp9az_<({~zZ%@RJ}vwcj4|jQ!o^_Y8gT`qOm2b6{dDP-fVck3>m>Q~D>=9^ za{Wi=!?{m$!H8(}LRs=>|GWb9->%@8x?w}8yzRjl^uZh^cd?`pSGvJ+42Y|dO5m1X zNN#{z9s+2DD28?f+_JqK+WbYF`h$OLBe*AX6)-!%g`U;SN|(VKP#}X);0xeupbZDU zH3Vidhi?f+7eBY|D9%tW+_i=-SAlg>*qO%4{nIK3K$;Mpc`p4I)xXy z%L`+H^wd#YxJ!NylzXBB;Hmxu-q!jv)6!+hXULw&I}cZnXq!@9tmX%{YFhBU6M;|R z7+Ua*Z!D;OWOk!`KKC$xPmzqsi=K!h+>qs!Ex5YY=5U87oqFab436>fn9SC`!M^<& zybJ14rz6hoPP?Jjs8e@whyA0vFHdY-%agdf{O3ml>p%^N>m8i>egtikBM^u~mbZhYF_A7z$JG`HR*optEcrypi7 zTGP2$HoCR%JNwTzcJBYz&i;F2|JYY6^b1hZ|Ff44O!9v=0~oDq18TWi*8)w_2L>@s ze;NcsL?&ZA;{z!xrqXCi^adHiM0|D7IcFQ$QI2*S(S)|NTgZ@d&X|b!N_&hnl;y&l@ej-W-{qmw!2Tiff%SDHkaJZ+)-^0u7 zT|A~?>!33(I_T~h{2Fq23QPKqm354V@8DfDU}B4h;7L&mz1;nSr@;hQ`J)q{g!jLU zyMbH)7bn|2(jqo*jqIhqG!0PDSQc;42 z9DY*y8!Bvou`w|A0`9y9`w(DhFat41aM7Tjg6(>EHAs-mHu#aF4Yt=ok4nJ?VTurOIpo(Wq{9(A86*Yv8B8Vc<@zUVj2TB=b76Y zdT7XC1JCOr7uj-(3E8NJ*`b^HN+N^6jC?|Na4hnAKP}ORrM(XC; zygN_YnKzWdN6Y!%T3Su`(q}jrn2QGl_!8q zjHT!>EOR~Va~%_2oi^-0zuPA2>wPAhALP~+u55GnY#vDWeP5CU_rwr-({zxunwmL+ z&t(qqn>iu&LFuoaca&}(Ig4!T8Bjokng9rTf51C%|JguS#g$8KsLgY)~Z%YlSe zbcXAopSb*bhF!Y({i7tg={00ztrtc(S+YHyI}RDFQs?Bv#q)d8k1Y6WByfOWBvrkTYIwOj54+Ny zsNnMFr2E|S0fMDhq40}F;KnSd@kd9pDZUi-Lg4twlYn4Bu;wBM4Cob7PH*ObD zJ)@>!Hulqh$*A%SJh$z{ zVNuwft%=6XSB86I>B<9XpBq1Kaqvu_0D+M4>%i{ofcs!!R>$LQh;Nl^;8~A~vEN%y-TtXblQtMxDuJWHBW~GDPdt2xrg9c3fR1@k^ zvfF=i;!R>R{>c>V!b@uk4D2EWPk^PBn5AQ+4tkSTrk|WBcVs3s+8FDg4f|LaX+Op< zndiwU^=n=>|NB^DVh*Zw5Nj3_VmK)VVK9KO(eNJb+L1(8(tY#p=` zVF^zDvl3ENM~s@;EDbPFX15OdF=K;%;^9EaxgT;WjIW3;RXX%xK!QV&bd5aATlSXJ z!)%-SUkWlRj%1|&0=9~4F^7l!SSjcEyz4zVv6bDJ3CpJej2X|G zb)IeObdb0$<94hWd`Jg1Nr-E(65L`clZ32?+14ELuYgnAutP5>##sYXep_SWmO{xwA&0(tXJ(>pA z-gv~BCyP#tbBb$i4*Od20$|IU7^K3O!S#YWZ=-+d5yh=Q2VL2-nQK}ddGprZ98DFX z#@&J3HecVaatOHfOBB!GT?i&qsaMG)geifh`45> zJf;wKxRifVA#Ywp9VVXGwt7^jyCTyW-dL&Ra*3m)H>JSU<(2jVDKmT^6qq*<0tr#= zBpCu*ui0&GYGjr;d?jZW8^c({IC90FT$~mVx!~W{`0KcI(EY8=zs2>R0tl5NXb_9O zzF4-=6N?W(I%q?82k-**<)wC4L?GR)-utuY*F&I|J=ToM)5b>gvot=LqA6T_rdC@Gwc9dtP@< z6hQKeEC0S4!K_Gzc3k}36SPj&v-f5qt0kTohu}E@sVz?OtPOZT4naq&kh>McoGhgs z0GMs~eH}zPOEQE3MwOH}-{*dK9H>Q0h&Y7rbzW1KM~~A%NJs`CF}?%AKvoa5z$!a0 zsrCaGkf~6RuK5|)Apoj6+D@m?xMY71}Rf3oi5Svc@~Qo=s!jFD6D%^RCeb9%j<5!pRq7*LF+ zBY*0kXOqaDd7ni`3ZtGzso;_0-nVqn=Vtja3}&f@6GxQ7SwkzfMQ4D$v zDkp{IUKfP(vOEMwTI}U*znZ~373=xf+t=Mw*Cl+0`K)(ZiN6OgY_MPh!h_!rJvB7AgJ7E%;2tSp6zLgWi#lK2d=2ei|nj z76n8Fls-sp7kiHwafdYu$L@2dCYba@PHsuB@NV}Am6sM-?k@Z^&~_eEbel3_EbOXJ z>Y3Pn2r#`X874|2`?W=DXV7$zucx@zKQJRFz{dZ&0XCvha7`lyBC5wUa1lz04stDW z{a`4MBKK>(`&xY)au(|#SSU-6R|s*$b2SbZWi-~&R`?YaZg~=r`sHv^!bFPOD@;vR7i`0D? zH+owf1gXuARRE11E_{`faVye46AgG34pVIQyON{0qWLQF1^ zsP|?v^1zswBD(TbZrsZm%Xn(xZQjZ8QzA?{m#;?^Yxd$L2p`E~0jZsrf=?KqJR)78E zyk2?Lfp7x-KueKFhGorY&3l}en$Vvi4mS-bd5chMSd9D;A}Ni=Y&|k&CkI4Uwq~_rn%tAYYbnhha+$8x;dxxgX-q2eG!J! z5Y`9fUir)MQ3{D>mpAG^)nHq0vFLeh;V&4 zhWjJ02lHgVPny)dmZQLFYP@qqGVD!V%bdaL|Bbt($mQwUB-8VIT-Q9uBQ=()NMgM8&_vB_LCo3Rl zfkyP1h|B9V1p53gReC_^=?#)bkJmuAhp%@%o7>RPc8|P$j$X1rbE%p2VmR?r(D}?L>ev$bf!qMybR6aLKVjthG=O1a65%=0C>( zyp$nGkBpcLC`6XZK?m)Puci*`p#F4y4S{mr>puO%=?aNJU*$)M4syRB<0s#rW76#? z%yU5RsC1CGRV}bg<-A;Uz;;hScw_s6bRJ?rh+RKZHJ;8_4b0WSECQV)Vg}u$E(rl7 z6s8U_SkEUvb#ex|2C*Qcb-%?&12X$mgu(SwwzU+ul6iq&t_pkWz)f@r6qkaNtz}3I z!%PKNB|FI&DfrD!SYPGuC5ySTmiCHzoYc%PmDa|GzYs-U&aAVfg*xzF_Qko|z=!Jw z200nUS+i1}j;y)?A2qwSZM%nUq?It;JKcWG!*sLbcaO~3edSdA@~?MpZO^Zwx*vJJ zq<;79<|Ts1eBO|jHw!qU=PE$XevTJg@!+AOq+C)jOSSWKx3sJ$Y_cM#BXGrbq}<7i ztb?rPz@;@aqA-Pr_lmCWh(vg6?AH`gPI^>dQQ@bQz~BFQs#fAeg|b2@AU5xFjPhB~ z({n4GtzlNd@}+5jhreWDjU5PiQ<9RGAn9qOSL`+F_9hgAHA0k-$ZRT&RF-tE=+ zOFYt-nCWiYqcw}Uy7y*B@>1&X_Zg(gRK*NSK~G-EW4f;qt&3dfo!3(1WfShPrW6pz zKN%)W^&QlY$yI_$)TT`A28VmCV7}r_+JRYBhQyIsi1dkprV)w%Lh^Xj_>?|9zOBKZ zkuDCHbq|#`P$$U%%Z?Y5MZAH7S`c&($11R*kaXkmK{ogg$AMSF%+chWIzEp>uTmsB zp_c#;8Nuwm(vD%QVYAy(UKjsdsbzD-^^vmM5%(#7L~o5=*hROlHiJ}WrFE5&#+Q)J z`BF*jZAyLg3YkTP^K{u-l-0?at)h8m7X_>su*sd9^CCCi`s+rDg=xIcsXs-N@9l`2 zo5h~{ZhGPpJTb8=Mf=>%zVMVs0UW-^cd*Z-{PEZG+XEgS=j9fbjG%3Yx5%u!1m_sj znpCH>)wd(=u#8prBH)U-)bqu9AcnZiFy}kE=%5LbOV|NTSW+kLV(A=N_~i`e51pKe z5w|^yi@X-TSotF4!Sjs3N0l3LYK8^^{q}cCXPW((?e{K?J@-zxHvgT~oq|7Wr-cPy zc@@q-dbi6hD3xRLj^p32P)D9-Om_VM$W)rMi7&AcZuwmaj+BhQEQ8hOHjHJIM3~PS zbUClOiM)UHcu1)Ngu2Mo-BSm({#o;Y{Nf{UkFkR{kcz$0fDS#PDC0GNFcVElh_uQq zNwq+_$!bWTc~XhUS3W}r*8x2qEXN4^7ttD6*rk0+k~Xj=WieI1>L5P^`x(qS!=jbg zyncaD(08>l?GM$JmfjqDz=ttwIlrZpg?Q$CF#qSV$hX~i3gC_)lBmSco zJq|3ZT*gste-4yb+6s7KZx>NCU~%b@ueNH$6=U^|OaWGqraA%)iY+FO#))9ZC|SJ_ z5Xz58%CUU+O8~Hc>}RQLCWF=ay&@kQ@AzR3c7HKQufV*wUW=l9&tV5Ud6&GWNLkF{ zh0k=5zsi+sHuq!k!OnnvE!2I;@rdxsLa&UR$Hz&(H(0(Jn4pHR3YMYJko zQ~^?L^^6F84`!ElW>+h9@-fD0^fSX0o22TL(X`}>-651J`tDXQYvqxe<9C0T z)GRGYPtO>rKeT6|f#22-BEI}8nxPpQ7DyY=N$|Jd@Dx-w2|nQr504}D4fuazP7>}CTtd9IOTVItd<#^wz8 zYrWSf3(2tx((FEu$vrR0W#bF4%o9oURk!9WiJQ=oBp3aWo3MXD(dIP=)+Spr9$9*! zu@?)6>!ucA;>Jd!09Q4i*sg=#Rg_b6Ic2nYs@)aNYozVxwVRJSn*x6=r_DTOe*qm` z=)ea0W7T=L+=h^%ra(*Q2?soQlRsShGNZ~er?PQq(G}@C*{h;HBE#3;$#=>j3%vj=4Z(bu6&_AT*xK+4eULcfh=G<|qG}J`vI2nggT+hDlrIVorjf5J z@jHkn&o4!Wgt;hkYnOKmP@XCaL>rFY)eeal!v5qx;J{BSX#>u-5mCf+(u@etGEE@6 z+o;(``B>^6V{i!z=yD>Lo6V}rA?>KMG%Sz@g9uWV_*`MAYPkqN=v(e95Fy@j zZ)^Faapy2fg5P#a4bIk{+Au%pymyEf@B&~P26?yJPJCFW4)L&1m6mQ{w^@H><1dJ* z55$4hIP2sdGsq!e#-5mq0j5mTqiE8dp^t7wPp9#owo{+%I(LLk#axJ zJ0BWSvrlDhXVdGbAdWra8xYyN=_jL>icW?Bwo8zW5({I*F+AdTWLS_Mvhw!Y+_ezTk9FuW!4`4>=L0G#>^((Q`5Rpj$d?nJ1*!`=DAz&<*&@TtTv|A*$bl^wo5# z>L(`10U7xjeWfxLICT_PGgZJb{tUE2MO1<1R@sRGMlaGN`aFAMu_VLJ`3Qd^88dN_ z5aim%xI*@`sLjYv7{U>GeX5HvR$f;5$&9L3tMq@RTuyL_%_z3cP?dAHsxC{dyP4$_ zF=a}G?;dGwY}aI0mwM;rn7#DqgBParcF9;5(sK-*Rt*hmjKoDIPjmN~wgq#}RvkRV zzVx`Sd{@u*chg5+wB5Se+U(R&GVhn&SCeiReUbC#MsO=wMvkUk0+Om?C-FwgJd3T( zP*qgfbDGHty%be=h5eDnob-Ou!Ovn^j)NrAFV5YadbsWdjhwM<$%+;G2Td17q^9hw zwsODIa&=d32Ia2h=}1MGha}*~c0PTr7UlE{L7rwG2F&CURT*zKnnnhKS*C!Ps?6mP zN=Kn%$3^*GcP4dsGTHl>U*GZPguS95-q9glnN#Yud%Gyg!SnGkaE_!46~qy0izlDh zR-vp^%m;jfQhItaZ$U>R+MF-Z7{m)G0Jx}er9TT}h8bpF1p&SOC2PWRxFAKcKYH{A zJe6@mminhrkkLVj6!nNUC#WVfjX2=SYmWhmcBNM#?a!?)jt(5x{S14D9OUBO7UfPr zD<&V2zC_-0w5~`OfHoG^1p+2AwZt7~fy@_cIxwlO`7+`NzM{R>DJ(~^*G+H)Y_ySO zXv+gy=Xa2$0cO`aRR)Y}_97pWcck>xihUhH$naO!#*>jDjWv7N8JczVyAwkjqT>Wt zC=C{^8-3~+bE3VaaJ3<6nn1M#na)!E>bNbMG)hlMMi?UHReF#>a<9`=DvW(n7fLdW z-YVt(5xw1@#nHM;GFcQBe(fp!+z{72C!w}Kdh_$7t$UWf{rTzX!=J+fH8zKFenwi& zr=~6JJOv2DzL92>k|RhRc+5o!77)}0Ha#)sa?JT+t5g;Inl#mUNd)tNjD(pd|0oMd z2xE0pFD4tyd@r-LHW#B8cWn{aj4&PCq<_fIAE{oo#UJuG!`#5Fy3lE~GpCCA{mJ>lZT;chNHX`>s9>qNu?E{EI@?>c|6Eg+G;Y5TB z*V*JF_%whz5Z_2@DV--pi^$6Oo zc}iA;So*dc-|Wu}Ks%6IC@tv)SyZR-M$ZKi8-P5pp5+dNYxj{T&Ft<1g<~{XMsbQS-7p_-e{w~6%2)>(?gzo$(3%?-Ib)>S>LdCJqQ;t3>oYFv-GT*^Af$XZ2~*Son3E$Ew)k& zBdp#!lh@`CE$k#0A*rPK>i{)+6aH%Ow?QMVYs6vSdiDUKUCItlULDM;x$53oYL2`i z;_=v%V0ytJv@^%)gt3&Q>Q%d>KN!w>753>Htc)q14Nrr89O4%L%=k{-B z?a9Ty{kCI!hRbx&;i>yT0_%@dXM+b4jM{VEXu(;YE%&`%4>dE0HxR0|n6JPSoX3)Q zowN&-VUBQsM{GLHq$me`Yw*XZgqLY9)w(gJ%t*=|0wyX~-`hL)RUev5u5R#cISNWQ zM}9Wo#LHZjHD-8t&^e3=-9%F5;r9C{f2MU2069={HbA)QQ|ksRd{e7YIMt_&M##{xH%+8lyj zHj=pvs|oMC0;Ri1BTRnkdtRjpuJs?Xg+*+2w7lIhA9fZ{zfbUz6|J< z$n8OhGXT2pimiZcK;0ubD*W{q|Bnk`pcq7?T*Q}sv%W5MwLRYbsnx9n3}s79xFx7werZ2vP$=l?o|ZzeEO?lyjd)X=ZhoDLSGpWp0!}3ZwYs zgz6PvU(#^)9URH@??8gKr zt7xCxKt=8Up5+c9%^3sK2py!X$)B7Li`RB$Ca+h+K9c4Ql-*xA&ab;^3yUZ5@UO6< zAJ+Va!*G7%ylCKBMA%2h=e%=Vz&IbV^F zI2ZdmStmy&yX@HPv@!Y)9Wi8Z-X0scb)9eTGMd(v!<4V{HkR^-BqKoNCk<7;PE3xd zuGP7@>O`)mT);_|3@dWZsM*T;6O3v>&ZF81_*~3Jm3&|^bWH*_WX2pr=Q6@&% z^&vz%w*ywJVkVcgC-_CRof3Mg_8_7!pa&@q;EeRv$gtDNsKiLQX^C2RYduwT`)dU; zovQH@*Z5p75Sk0-2VZv!0_d@*1mhhr6PlG6&={wCZp<8Yukl1e)TyTKjMHyMxbwSj zCw@0hGGfg)DByB7vFB%82wTs&5SyiOman-@7GT|fFy9hqaY~GJNbo5bQ#B3^Q zYSL$*j)5c{L^+g8u)aPK{f?g3K|yA$#r-+>N^w-I)}sxNtrTS9)nNm8YEEqgL2foz zK40$uS_Mq14Wb(&P65gp27;&OVdm2G3;aL%y@9pj*m>A2!j{;U2Yd#ZW-aBtd@ZhZ zv8C25a33_dLz3RoL3I?QDu}pvmRLO5PY-Nc<9Z{(NBQWaw0~8 zTMal)Zl_H_I%w*s$NAS8cNn;9XC@NW=&LOXa2ommT6`}Uktm;0Ru-%_`3m#_J%l6e z%Om}Jhxk6Fn$F1c39GU4z?a-07f)!24`FJTIpvqJoEZPy53AZ zxeK-kmDf4SYlE)~uU-v0+gE*h$r(p#cWQwWos2o=35wq1si4pvbXCIu4som-5(|ff zY4mUA)qVi|O78cNro{-pu7-_mMy@y&shb&aSWF+PR@j=Nt0H)$Tg1;OyWw=&elenQ zFk>s4*MX{BYi?;?{8!5BlJJ4jd8!NJvD4ISq4A!!Wjy5V;WL#>9hX0l#!1owz&5MNf&M ziC+e3O$fe0ozh38Jl%IwnpH*DjyPU_Voq8cU2TMmjDW3w(?NYrpv)1Noy-0kO~#F$ z?!TH8KELy?P2^c#w-+Wdql0Sg?5Y4W`Zr_Vq5mIQ@)*fLc&L&$m1r_H0V>cE{;P>7 z34P@<@a(*Wb0zbA;A_KWDQ;HP-tMT&V9&K3H3cJn)tm1n(WH~v%e z`fhRfnh|Xq?}S;g!mcOO5c}?y{$;}TX7So-zzV6atQ7dYQoTeW@!b7_y_r>Yq>t?N z-j4VxvJB>Bv>e`Nb7Qz2p}ptZ^ze#-{juhoAbPpeHC z<-Z4zvl<`S!W<(Ja93YTXhUxB-G4U{w9z{fbFD41%)0w*Ze*ni{ z^?6X5dr8KBf%a&Ce}-3`%K;TRH5SZ6%5!5u%T^KcItO)rW^^A3 z2xd0%lzGmqED|@XmALGCMcw-n{yCz0^)HcXE ztfhG!tV;~zFW3f|pgN5fQ|g>6qo)JnKNvl((}B}G(k-d3G2Kg@pz^O~OsenVNB?3` zmcK?YDK+OslV5yed|?uI#hA9%GeKx(xOh*Oq{YZ3pYOQ8TXHUWadD~Yw6u=v!9GLG zil9~7!mUtv@M1*~2Yh!Lj((M=DyE*2{MjKUHQ>VKlk< z_*_RTh|c?0sjN4P(N6zDPuqc~3&-9NWDnl46AcmXNUuI?WOcS!%<+FmJ2ZoJ<&@$j zwG8Qw??KZXmfX(Uvl!3K7zegFaMQ}xQ0vy8Z;o@CVP(p0e#RC#mPx5#s!4mOidB^% zI*8rmjfPVS%dQdAb?Rp4MTW793Z$n*Z3M1Y*Qwf89ly8W!XWC zVZ9nGNsInfHani9YN??#z{hAUF5MQd$;J^i?)+l1Y(=@Z=}T3R)ExAv*)XGz%ugT7 zXEmEG{HUPKQjxd&qs9^Y^anW2+OFyp*}TrU2VK`Zt3e5vVSd0hBcftUEEoMDggP|> zd2P1hm2@O_ZTk2ccPSCy-!n3E7A_DaaZ36O!XQhK)V!oc>(~aIG zNCuZfIyVmkAAWYn^uN+#N4jd1A5?J@&EMnq(4!1z?=Wl+&M6Zme_%;%XK5&U$eSz> z^&k8!0vQLneF#Pl%q^(8qUt6{^@y^c*%jEHRC0Pgi}Jb~;(SSDcah7H7IJy%T+lF| z9DTg_f$~y)@djxZwV7LW65Dsa+Ie+EHfU=FF+drbV{kniHS{yHk;YsWhs>j#^)M*s z8TEUxDnR3h<&*V=NL7h>b%J|kKv?(w2JB#p>VY@Z2_KcTKI#6-)LIj@!4bE+3XUh%;E4}Py$X8fC!s|8Lr&6z(QMXg za0Xj}dkToBsBQp-M_^DgYr=t#b4IT_kchl=#`;JWD*!bOEX1|mpar!&eV=D@wA-1? zcZ@oa;9U|K#A>PNb`2H1ab&rbq$ww{0j~nl?<2tQS9NoE!8Ok@kbb}JWy#Mhkr{TG zJ?{{ViKwVbFut&sL-J?LlY+$aF=B#r*fX`Dt8KU2%jEuwiiF?dSLI6EMaDZf9vnM- zbg3D#_Vcel-~Bd~;T3Izzrw<$BBd{$jRkdcruW+mxTK4PsMxq0jxjS))ui$|YA7=z z&UXr5Pd4i6<(@4z?igvXbX@Ojs&e1uD)%zlhCcB6h#6a!G-hSA`&+k*;@6wKm5wM= zRaK$djnwE_s@pPa<*7OaYZA8Mm2|P7T)d(a#MVycUChcP!(A$q)M#(mxL7~friwZ% zBc{V^#eqheL7|+sOr$aIZu; ze$UyN#c-af;#fGIib_6dKkce}O()zg6)2~H*vIYHZ}AmmDf&k3s|`J^X7aBitMD`M z&d=y(Tgqh}6iAud?w34H$HcBdCkZ&M#ng@@2Wsx;L5w^Fbg7Ne=lab}5nNtd6cK7Nxabu$sDlnkTGFZn*=Ee3jEF3*V4S1tX}WZ{ zlYKphqK7S)3a*AAGq4|IXk@tRxEdIm~(gfkFtp<&Dx6YE>C*6W`nBs9V<1$uGZtRn{T@OAO zm6^?!#tf8nkz%|JKbKndqm!dOW#LsId~$|Wz;6j=XOkh78LuKi7(+N7*Z5vo14AAW zx66{(T-Q`o%w1A)C}gAbXo+lgO_$3B$bomAlq;#cW>`(wJ?M~Jan|?3W=m}bJ;|WfcPi_TI&{W48^1IKIV5SGAmj|7(y>B5{6n+_Puk6gYwR*6I@Ctu&;4`|ExgF% zbcp|}QwTrsK`7|>N^WKCV#@j?ghhVex#I3%Lqw#>qpuS`gW>H(^fr6XtdCUwkWi4} z1USL*HpH8OI8{is@}U-j%Gs@+06>J$L2tXUwSas2O2dhC@>R4k=b``>0lsQQ(lrq8 zkt3tsqYMCPS@-a45Uw_KMu!ONe?Q!Dvi-|NVuwthaGgI)pq+(C{S;#f5%2Tu8j|Y!BRJeD{L%MXwG&F>(-V8 z^MA~v!YoktFraxtl3^Z)%3ygr9L+vI%qK{a=IEps;)s~{NbV=@fREN|n5n8ZVzP%+ zySva{-Zo!BN|fCwU8QN~_6judzt-kSLF~^j0O8Y0)d|so+bA17z*r9dB1ya^095rd zo<;EO1|vq>BV!zW5)+`^>8IR?WIRaq3^yu0Wdp4_-e1tNer45l+nV`D9;9x84k!BR zkL^f+f*K1f5DFt7`;8GJu$y=$Y!d@a)(biNa@g7ItL@y}~ck^7Y*J{Q&L#;ZF!U z2|>T>y@7ll8k{|6AU#8BrHW4m3uFo*`ys}BuQw7uu#q=F)h^T$?@167R&I@6itthp z=dXe4IFM{UsN&(9({P3hXV$Tb?<{CH9;Iphr5kBkJ~F_{MVJ?ZzB1mEvceffj1>&N zb6b{3T+IM^3`2moJgre2YlQsEFg|7F_Z&F~DB(prD%h5WM5U^X^LDMrY9$4Bl=0HR z7in5I?BI>5lURD7n|ItHTg$921M&8e{I{=vA5SrS&6PoRQ=XEqWhCK7nL4P8iR|{iJSNGP&ExUUs;9N$oCA7^JM@vBX$D}|{e{S0HOr6qVnE8IiKCHzekq(xAq{Pf-Y-BoPSq|l z_xA_k_Lt~ipzXpR*4AE*{s|PdFeb0!UoDGqRHMuk~a(CD~?p`L>#OG>O$lC>zZeQ*AJ#ohRDfawD zl0FHabcOj*D>Of8P*0Ll{D@C}+I_BxgZQ%nwPJ;K1*7K5_+ar($(yz}RMrQwB880= zFp0ZP3WcB{`1q3?IEH9PZnbV#Vl#T(hN!Bg+dxev8^4|Ub{=wkMSHMFyPQ!U%X>$O z{=GZn zrbH3W#d106ruq#0D}1n<)x=S*cSc`HUh7LhGXSfc+D!y|2Ztx-Xm_E>oOhzPQE6W5 zjzuBy$qCAaNW6vHA8IECwM`L!kw#&9YJ+WR3Aq=rHkzok?C*SrW=)0zM(Bftj-1Klx%uu-h1lb z<<`dkDX-B3@IHVj@QoCSU$(*h`i6}obY)#Y;C=Z%^2jKBs8aZzoFzF~9?hg*K@Mz) zVrpH>7@4nMHcR%c4cyNdfs}!J51e$=G7M0=>hY%-Xt+tm4=oOehY|^Ad(AF4RYwvA zU^jviKTsIv^y9_&di8l_yDc7z`d9R6$rwW}ma?l7mHOACkk^s!c{hDY2qF^Fhf;ni^7o#2Af;xLWPE^7g6qcB|2JbVx@M%$f}UeZem+uBH3 z>-<3I-!D6<_a9ti(98l)urhOw337hzz9Gn&oUemQkt1mH1*ACAhyQJgGp)SXC9LZQ zI?`z~c?ego7%VLsO1Ej`rk^8>aFqNuRQ_!s!L4^=o3Wyl zo;W)u+ILan@SE=c3EaothoQMlrwU@V<6OQbojXxpqZ zS7z6bix;!Ip)F>#ffB8~COEH)EB2G{lZ`%H#oIT7V(VWxG9{jP@)1yy>yxNlv*f|H z9N#;DJ#j`J^eMz6nx0du>BH2JQ^}?e{Q)nZyi9gvWJXa(_<%eKJJ(Mr=?1g3besTk8Q>OBlInII{a+#7KB01*6c4ZtJTTu4idr$}c9t!F$t*cnF zg`gb4U=ArVabQklp3EG0dMA-gE@{L=J&3Tnw_z;N<9(&OG!JlLUGI%$L75Py*~dU0TrB3P*!>mM0LYwWc4L zTtZxp!1e6jn(T!Xf@Y%~#)90uKcGcwtvK>-hg;fU#X>NVqx^L`xBF5ZS37{b%f+1{OX`1ijS7F97JQ!52 z#!hvee-e$PJMP8alrv|L#MBRe(x*|22P$%co?m6ku*XV!_!DMWslorb%MbvEf z@+~)Wkk+Cdl3B;Zp;3nl^DY)sQoN2O4Bq1x27E|N6lzz@_pKn!+=TRhMf?WoXiw%eF6wVp}-l6wU z!~uA#YcB#Sbo_uSkQF@5cqfprj0WFT-pASWAf|dHQa^nISKl(I+j%K6Y+D82okhdg zoVUt=O_BuX^q&Rg*NE^IZK$^4F6XHFkH6;N5=U59V5Znw>voe1z{dEJk68>YkiBAW zehmmyEo0_&)Qvn3f; zI%qKyzS%PZonc4SS7^x?AFs%L=FItppl<@(=a=NFsR3U?@05&m@LzK- zlAvtMs0xzX!7BrOy=|{DevC+vvF9#-QEzwG_T8zg0TZem>}vKt zRpflmTXo|PzqC!umlZur8|1qVTSU=v_@2*yRlSw2=#1gE8E%jYtDFpXH_CaFRLNwM zi0E&Gm5g&XXS+q^%WuvLq~-m46GqMyyL4U)TSqfeu`BI=Q=%;aKEeLaYqMIn))=Pd zPvXvtYlyfy<;RHQpHutn@2C%yB=(xvwx3MclD`oRb+Sxpnb|q9YFTyvNLG=1E^h3N z-`-m5a^6T~!5sKAQswiPIyf=$10{t@9HJd}^sJ;s3jnjf15(ECTpvdgdq;*NB5jD7 z3|H`9D^MHxpP1l0!-;n+`3h_&kBzP2i5+1h^~$8oTc>$D!ISjh@kLcEaj}V27!O+c zfMeK8PWuF!6@s*=HWUxfsUS=R+T}qyXeL?(77AahrJQn*frx;a*}vMEeK*~B)@XC& z{S5QE5h@+OQ|Qr#5pNZd@bDlE%ss?N6^@MlhZ;YlRr81WN}!W5oj@0E$W0@K&&0X> zSwWNe*z!a0P<$V#zn`RgGchf*4Ky}#L?z()TZaneZ{5T)PCHm`Xi$;$vZ1s(#m7Q*@g)O%@c8NiHg1}?-`&i5pEMe~h(nZx|3uPIfxtt%1 zT%k`J?HM(+4;eOA1U2or_abeX=DP&W#aw>&w<~s(S{Fzan^$hdI<0JZ@Z#_Z?ziXZ zM7!9!;aA6QP}$3EYq+#J2M0YO^ds-?-)7IKV?_g~cQy^yi}kT;z;(Vxt`r3!2YAW; zWTunC`MgI%Uk=iU#Ev%FQZAB9C*5TD_3=+Z7ubi>IIjIfdmG1O#Q*WRDaNSGUFgZx0e_;sfsY zrdH_UCilOnH@e;Uusx*W_3@6f!}I$hA;;Qo_q=Y5!trp55j=6A%K0c5))jK@8O@T~ zT&(K`x!;`CJGBJ3&~DTt2dS$@>?#uzO^@7sN(1$|4MJ_<5c%tH{A$bF@V3#~73D(2 z?LCD~r)#RaV??4Awa;#r);w6AksYLTr2EB`OTya=k9aS=@xp$qSD=^i&S-wGnE7;m z_SWjI6JdSs_(0A5+5~)mzp|Gr?J_Q+6c~sf&)4e_ax&LY#mOrLPJJ$3M)m=$iNrlb zYi7!E>?1Z&HrA^m)96gF+KZa{cMEPMt976W+5RAU2(t<+Ouhs8mWeKTtku&>E)hb) z;dR{_d;d>-FaG7HnWN)BUF%!&mcj6Gez)~i;r27N88pA0w7HuXoq27SvyGVBu|r4f zAOGZ+?f$`$z;&u)=lguj+zsSIecWfcrupJ!b3UKMLx&OLzo!*Ozn_w<48ARWP=Qf0 z$rGRrB?{43id-LR_i!_k>?X71v>2V&o@N`BpSahD25sOjHJ&3k9)nP=NWK`cl%83?@(fRnUlcg!GEVK8zljmeNJc#+NIcIcS)Z?n`q&^((~ z6D%58gy+&#`QlXpfHDBv=uApSXfZO>V%W17+GP5bGb?eix`4WvV-=5S>hbt4>y@G9 z`kbmE4A>PclQy_>Z^fDd8f*KjO3(Cy_N@T8XZD4=P;VqUq=8QH)2 z&Jf4d^3FBdZHCM5srNfG(=xqYkG8t!%NhAy;Y-rWHfdq*><@{N09++{szOs>Yd!BM z8ruN7sa|sjc*{q4c+!Ph?ydUc09+9OxDsCM&Y@jm8PK1>)wm)O2Q@ktcGr(&K0R4b zw6%Rl=Z&JRoc0}R!sKPa=?XZq$`|98e()n%kH?HXnsK=_fyf~*hAH_aylB%U99id12tU{u|+>w$IPe=IO9c9%r;L8&Q#X6JVy8 zq*n3NHpmZw5r%3h15>ZbgI9>wEnT6@k2eR%{gIz?foxXhk^5IrhC&b6Eiwk!lxq@tz5d`AH?k}8{oU^Os!e3nrC-y%-_w}k2=Z!tLSs(1)swg)!6?>K-1Nq^ixuI-%4 zWi;=o?MrZ^=k>fX*aG}|M`D%mAwhEW@LWblnz)C8iLFhL{~5z(yVZ&PYS1+TQrC4v zP^^ix%j~1ZeB*VVV^ka#sgLB+u&z+R#u!>tug!;=JAN6d&(2#J~?G9tQb z!mQzAi0!YWuP)m-WE!j4{bk{lJjWt`i%9_sbI!}T-Nnpv-<;~}Q%XZpN}P!6Lu}Eg z ztpc}qq9(tt4t>`poiVL6c@73xGQgfU8@4L>bORojv*bGMW)q1OYtXhj+nS|*9o;$; zxd14D#s8g$a)j%E!i_3e`4bbr0+b>j&?2Zw1ns&|$A{)}jnqvu=B}5u1ZP*1c@@)r zG5E>Io|#Az4KO+oq;9F8FB!o!ksIV#59&u-Q>?LGX}Xj+tws#N%&?)&`h%)yAAUVX zO+EHItIHwa1ooCH;1$^dQ=Nf|mkwnQr)GROh(%=moVaZuWx;Q5r`2B=dQiCUWG-J( zZzrM$`we@}OB5BX`SYsjkas7lv$)`?kZk4oZoZd1Vc+-h?pAchKs~JVH)x_pJ(fc2 zI$+DdaNV=;y!2GgI{m9p;*SX zoEl?IFeYkjF1ej#FpVR=El4wWT<0|t-4fU==JY{2O(Y|`?w84cRL*>Pt;L;K%w2BP zcv*?wZfFJLX9p;3pzlHu?Nxee=HpG5;XV8qedUhCoszJ1cU`d-cVWva@?ueO-0ymZ zm%2Zh)~iq2A&ec8tUhx?>Kit%f*5Es5IJ-WA925AWgp9Op2ww%Sd1gUstuEXXJ+Wn zsu|zwGh)m2sAHW>q*>xA6|;V6Jby4CWns04?fXZR!>sFpeB0_AoWpC6d0mxD_y^Uz zo8wFei_y)2Z;AI1=<2R1)PllQC%wJO@0W@fztmC0V-AtD`1WNKKF?pQeucU_~DYk++ut?z#VYiyEz z_e?2kDlk5-{ht46aXG_gLPT9}umPORljlJXu%>5<&=)J>+|^nF+61i1G`5=Jn!3dQ zI=qHOBh>~Ar|?IAU7IGIhTSCM)N^r;wRn^mw(#U5?ZQS1o`v(Uthaw&-vn8N)h zdZyCBAkqW}bjV0g*Ed3j*wKWS63zh`KTo8d9RFgb{oxlc|8Vs~%{1Y_DZ?ST|QOq(@u7xZytfTY9sqB`H$0Qcl? zFCgx0@#xIF3EL6GoX-zoq)L18oHUjb|P@Nma#I)Io z+`VlEIh;gFGMbMVlU6G)svwu%=c6w!*ht0VRE-?AvO9v1ONH9$5$!WV@>?D}Cr4ih z-b*AteTgasn=5rVFvSoUUW?+~?IDEo!-AWxs=;;wcmT8OGTz2gGu(NYtTM2ko~Rnh zB1T^PlhVf>2)kHZQjQMvDyMI#!zAm%r(eL_U`QiRW6y2af*K$ z2US8w>Ko)|f60Gog>^beUYXlCNftm0am>C-2f0BsVN9RNmH9`G@j1*3$T1L_-#IQH z4XKOz)^K*-f_>B$u+O(ZjPWB^V?Lj>Vu7hA7c{*{)0YhH1f_NUKm6!g%7altmL@1o zGqD7ayR~7=sj#@HQ6f1&XuOn#3eyI&o*mG`F9t#>55BZ09h=Oq`^UaZCMW4Dr9U_&(NWNx$b&65Rl}X9ewx5?p>9rp0 zsMmzpmF?f{EbuSWOarXhq8^RcjPKM0N%ytd1vVNw#c?&tei8a?G+6H^4N|kvPa9vz z8d}#}osOC5Dz^3;!(!fK`b3TjNHLwQku_=B1A1y@G`}+b%aAll~pzzcUh`W1yJ2b~n4MLNpHN^+esYBdfZd+=A#{1M2Ua7RtW{v`HCNDx zf$3l)!ak3GnDd@Hv&b``#&8zQnOTaBqMyl~3K*{}?Frm7yd7qfTkS=Z7t&2jQ~m#d zSu6?zp2M2Fl0mjBi@Z5B|Rs`cQL=)?9wZTa#P$c4i%mR#OJoBsAgeMMj+&)r;{(_v95FG zAHaVdV0Yd=82K!|WoUT#h8jI${l(d&hDRu5=n&cFT}ZL-qY=t?Q`?Ty{N{7E!0$?B z$5WU`53(7={=zT$C(FiP>>2cVFj;$ii@0tLrh@Q9EBi@2D9F*}mP0%Bs6oUHjuqKL zAGoG>n9BunYUB|5idqwC882?OlrQd1JS(wJib2t%n^tGs-JCay+1}sYxxn4ltD0Yp zM)>k^UGnz&7qpV>qLmfj7wni0d*ym~o4Zq8eg6Fd+SaJy&T(-*z?+Kg2>cL&xgEH^5JeAOa#!2(ceDc3nysgO1`}1@=$dO zTr*ZzX{shv^gXlynNx4zADOe?CT&Z!Qsp)i(Ob}GnMezL-7DQvvEFjX>BczUQ?2~P z5p}yMIygQWHL)XLjP)d=fv%7CXLh@78%o)Y+z=q&u>WlJqDu+7`j&=_c*B8aq9(z)F z)Nw3Zde~1BR{89@kFHuUx%u64b;BX&R&<3er}wGYae_r$!Hbs)O^aJ-da30R*F;!}?wKi4JGQw4?fKVK zXaUy_fodf)1YFLQpGj%+RcF3gb3AThyho0vU!?t1P5QRj-H&WakchAIH$LYDA8~w& z2JIqDp>~46?7CwI0^1BI!=;)_eo&0NQ9cGTg30vnLfcYBy)_ixgBxB>_vl@U`6kV8yf0N+on~<1*1tX zHpZFJLHY*BDcfj(Yaq@5gnZE+Z(aYzPVjN^%z2yq{{WeH9<^b;?t;_nFq0wytzW^@ zu@6Xamsl7TVAJH4FKwtvr2P`xw4Ia(y6LpjaVfPR!A|}u*SD|hKqJ__C}fKJuAKo^ zFuBtdCo6gS((dyWy^|{j2XzA{MJ0dtO{D{^PtKo=d@BeNggYm~ShUA5r_Zp5x#=V| zh*ED#(5y71U#8d{gXuVquA7{~D?z1bWJkI5T8!=Fsrxfo8wHD*#+b0cfA(R7>tHKsc z?gzt`YUITpvHMwSTiX;LkF`HJ>F;|6@!!bkvDg4t)+QK|H8m`eUu{ICE?oCK$(D*? z1V8+l5MoPvxcnJNiPInj!GjZ`$0jFYuxxrvbnq7*4a1nx%*ZGb+g0@STsfeBa;Pdp z%O8VZ`sOQepH)N~9@m|?cz0ee zPDZ1%NZ>{x{monZ*Q=E&NiOnQR%7HZ!fsx&o5kaBOM*=^BHchZXm_^1_fH%%J(>H} zg%@{;3nN>8f0HSLx2Q$U#6jwQzAQYKl`Gi& z!6SBidv%|8=9K~p!!K`Brq`iOnW-m^-wi6;l(}tFb|y)6 zm?n`0tl;!R05xDo+UKmy*Pk_4wCrXpPXVNZi&Qh4rwziDzQR;9 z+40L(2+<-IQIG0C3@V7Fgc9pyEZqK?GTseF@R<=cKq$jexSElvlltz)8pRZw%Ww>( zp>d1ffPL#xJmZJ%M8_f08XvV~a%|h0d%MrRXVxu1pFGlCEr0%}lfO$^A#trXH;%Pr zXWALtJC8HsFS=vn-yX}nk4k+!o);HZSw=sJ7K}J_i-rxln%cO(rlXB^L&!0nQT%no zPU(H3;o0s~SLEru34G+SHt3^=X+iCNU=i4VP!va!MK^63;&@YxTZj)kUsZyW#V zr)b#h;rjQbe&5iX=F#RI>MgSm2p$#qjrx$fv+B`GMpAx~dq8kuME28I8<>+|Fc3mh z)gMBZKNcCtIfYtzQUj0}fh<#_riN`bzMZQ6sF9YfdWpE>zLSDI0td=j(>nzI#JH@? zY=!=;eSOww?G$RHFk>rtAfyE_xiFl|kRe(j!-LPe;|Cil1Q3r$q1_n+OfXFODc{NK zm@moTb}pq@*p^AtZk)T@0m5OX&4^M+n)Mg(5xzdW^WEh_0eIO$F77Ib=B4Zu6t+A2PnOpl1@M78$j6% zvQusB5c<`qp}1!thm>^sF-YW{3H>Z$L2Dp5!st-jpqs`Aqo*(n_-D6temVjqC8(WuHP1N>Jxc6#V}8FoUK4}&i=2B?-^VRRs7SEHJ)pYgOX2&{~Z~XY0Pp^@7Cl646Hpz9HnT1=FP+usUlA#v;*VPn z%1Ww56%ztd+z0p|;{PGyhRcM&4&#yaHJF|p6K0U+ zi%Zz%V3$aT@!f1n&S;*c!Kf~n;$cp{aZ63TvgQc4QOYM70{eDUum~<%HRc&C-7Ww;xgU%M z-_7S|E3q1W{`yLk{QbbkvUTTcKsYR-lH2o`h>%ZmAq)FPH6ln#3sTK(v1$qW)$bq( z;#SwxY6XHKAg3K$AY|20#f<3s5Y^{H%@X8~N>?50K8>1W+3cUY34oAzgT2At*m}7_c z(ixL83pCI*c&LWTml4OH8m#Z?jyL#e!#^2ExEwKvr{HxC>Jm0_H;or+*<*RA%Hsh~ z0BKQ3N>IK(chv*8PB9%T*Huw}3a)`Lj0|5%OLq>n3N5%jtz+aU+CiuEDP@g;$oQ{t! z))z?__Z3!xTkx{coX!KJLD2cC^!LFLhnQK7<@#3$CqIxpQF&h>!E4s995@ksDl~=g zDwV$V$C_$6J!HF~f0<34ud#c&l>K0JTV!&D++p*ubXa~LGq z_^Nctdlq^KygK&yu5K6mbxzTDy-yrN^QruT z;ce|Fu7EKS#;zj3mh7U{ zOo>d}C8h*?Z^b2s6dP9m6Q+}oL5b4$Q1FxlpECfP8(uHZ8{q!FL90Xx~W&7XTaK#VzzVh6l6jtc=V z?Pk#EK(2L7RG=pbQ6V#EN0{lt*)yBL1mag*fW{OiXdAZ#q#Wi6o6M_3d~-mN9`*c! z|F7hpl-4A?M$jM8;o(2Q*7o$}h{SDuOkN&zQ5ArGU++^6jHFa1dUpe6F*Bk0Z?k)D zR!J`tE^$QMk;1DBSo8Z*|YjAHxRqbw38~*q*cAneY8R z54K+em|9%)fe4w`1Umuk>~TJ@s`gnVQQ?4cck*G6iFd`Nqg5=4|4sD<`Ds;6QAs$( zOkXibjJH+OUNKgX@*I|q{LEX=x{m4`x8FiNs!qQaYj;1U!!3F+rSRbXE-UNT2~QnG z=#xME-g&Cz%i_E;Z{&xH1ryxy(GQRIaJKq&_GqK3zS>+ zKsDW0IltMrG3`72_h1k3=+6m*U8e6qCpiWbxZ%=B5?}rO3|P9 z9dLL0zW{1`sT*PH%}aL|C(`rv%oOf)ztSKQ#8#R8$F|s$5^G8dG`+M*ms92>$bQ{# zh~Vv8cT4EP&^sAFvkyb$&Jn0^oW+6`Hp3j)w7XSJ$!);S1`xl@$+U%O|HbLS4-r8LgR_#jO12;E@Je3{Qy0`&_x(0$b6=C97u1l#n50 zleBA6oKa`y-+{*n1}jJ|)4k7D!(4PO39;-ZgycHQ1e?qb?~@E`8aCeyG;$HyPrI|l zQqm2FnF3lcN*tJ&_cE~874#u74CyDv&r8APs@snLzCmHon(Qy)1?;IP z(7=+vl#8;U;78BDV8UE*oFMAW{r?bf#YsjD->-GNCVW4rq4eZVo6GFg8^?e(0dHB9yRZCxj#5c zW=u-=t&g{@@v_Oj3}8DtNd@Y3o`gb#mlp?9(hVjYE|N3y)k%%XMLij7Mv8a;&U`~P zrW=@~&A}S6N-8M$82xgl#_S*qG{jz1z{e>z0EGWJks$~#1kZ#-5BoW@wG+=wPB>I; zRaM8?_5X`|fA$v!FhhZnr7~8D0M}^hT3CHAXLy%zCyt3%&_VV8YoM!<}^)YHzQv_ zinYis$7GuvViZ;KKTC4ya)Q&XYSPIjj}6^1iUkKczTLq@hn($J+E4Y_dGo*hk(l3i z@Hn?F(+=h8dS+uLHs!(akw=38xCS{GSA;#O=4^&rQTZ2Mf3b`ii>ECm! zzrTOenZau-Ll&$()^xXQZB%*7+VGA0m(2=_!bgnj@VHTN+_*s_Cf3lEXPxAx!=CQR z+?vC?JOA~R`CKJ~wKW^q0t=`I@_HQDz8VGo-ote` z{d^|T4aShbnKCo;A_ov72tCe2vMSOPHq0+!1x#6hww_?7$M4r2MedO$ZSVIpB#a_I zk0X%0*P2|&7+%_S#>{Ct!&ZHP`Jew^jG;=B*p*BUbx&vb`2Lxiv*+qeNu%KDW9W~8 z-yyCFQqVF-Yxuvxb)bxiA_q%>9$VgW;6n_0qbZ4N&nWGC+N_W9=kqzDxDkHs6$FnV zNoxTJT|P6XLLXV@*D$RWse9xMF>@>-pxNO%XQ`fnE%B=GD0>*s0^IJX^&I`ZLD%L^ zw}b&-|M;e8@n>$=a*S2B_BfVe)lGMdoUeb}JmX~7x3PcP!o>Sv{jHTzlvV}_q@ z$*IZYqek;|MOgm*3%})YV|zdRAeU2Z4TpGxWy5g}6I@-0icxQaHf7cE-yP-B|YW&hBR$g`&$D*Uw zYHOEY%#O~0LR;mj(%QEUaBxFptnyG|Vp;Ut2tgD5nXjvZpmmhaNeLOvb_Q2*BKm!Z z?Cx)MbS_{bok+PKj%HJU^SA9Iy^c$70T-Jm0TE{mIU~}CARe#VbQ~r#rrOPs-o508 zej`_(WIEkfIh#ERN4Jj6lKvstxbjnVa-@7*BrUC(yfPb_G!&%9jweh}Ly3VyB96bp2JD0SvERu|JLUP`dftLk=-Nog6>le5I` z08*U>3ll{~JCq894W=FPwvQZ3Fu)>r7r=4<8hydF@c^K8A^4;AIR)B}{k%bfiJ)l( z-*C4ez3$~q4^_toFy{9KaDoB!6YH<-N?5t_C2HnDnlZZs zt7k)3*GwCwwyCPQQl)vowSD`V$s6^z z6|Y|?O-Zi+p1!gY6$~PVrYp`E+$@cB2d_labYw?`th-s>(_yev!9@(@Nx3G7aS~&l zMiG-kA9FMPMZQ@*+PfAl>h^dIz(fiNt(7b+#zmiY?-?rht zT@SjA)L@RuqdxB4B`&W-8(h)Z=dU>Pm6hi{0|5_{{xA&AU?+xD_lvE@S^fbuG);uH zYqK|f?Vm9r;6(~bQz~;h&{w(Xl7cBYW!6dqxQkv3AMJv3$vI_moR(Di7ak%Sq$}kb zM9^ofL_is26GS+m`+?}cszC^pmA}qfDM`;2e3`R6N(^y#faI|=uuttI_riFgv9hPu z|3?^6iL@eKnL$p2*o!QUiKI1Mpwj?k>z%gm$!ApWi<3e}tn0~YTR~@aa@*U#&GIw> zs2DHd8xQ5C+>eEPj)0xz6Zree23;xW~5^0io^ zD1|%+=25H7o2?f0;5LF@mK$+^eOM@ZO_{B#dA7a>vMb-r{*)C;4$=C0x4$(A#^PP) z-F*F1tSn^xwob#%1Y)+Wcer1TwY9=KVt2~V1rI-5T=VLLpX_eWg(LMn@8;HAZnyWWb_3vziSTRuh|22|C)B5pfuAafk>*(Z) zAkZM$e;T2xS`LOIa4YNU-WW6y0N2gBO@0MNMf`2%S!DDLVL(9T`iCIN6B34*TM_QD zlP=Hxevq>Yb&f_hE&%e;2`N~5x}KhYeeBX#ZterzdoDDEvzj=#oTtb){$Md5q*8m1 zN!2!oG9Bi?Zx&Q=ZzpYo_M6J;cMruY85UztRXmz!DYRCV66W90D{8|=@A%_emZew1 z7iTes7meXNbGxLW+Z=RHeW#9*C$&7XgKOXfIhfAfo1eH7JdlfsFZJ`(>texk8usk z%%T2d#crO-Tl|NZY-H|;YB{PIShNj%)XzH87k=R0%#80F7OQ}4$xfb~$eqB*4HJNz znQt=`V6q0U9YqY~n9=~~c;{=7SVg@&xBJvH4{OZiJOs3@3DfO}8RnDkQw7`c(1v(~ zV%3219%epl6HjdFUumi$U%KOHe^SyiR;3ymjzxm2YZ=$rC|(I7%Xv32bD<#hA&W6a zg#1bKAY!e2*NAS*;k!J4pEa%Y$o1EJwnaE^Iba)E?lYJVjEzmE-(HY;U0Sq`f64|` zZj6P4^MGy*6%F?1!Gn~FJgxFb{KFe+wxIX9%z6W(yieK+6T{Wi^%ok4!T zNBg(g!&FbdTy3V3i&SFzJZ3&y%s8#<6~7%r4E~Vl1-|-wQz?o3(chHD3wBBG-Sg2v zn415yuVeN{>}5BIOU9_#he(NljYXB2wu;)~6ne63>l*>^yr-|wGY)kpIt`J9Al*xC zk<6Uo`vW^1xwGNdE~Qx&VXUa%Dlf-cdFZrWc|GHuLn|)i^~aao{bBI_z^&)Q(|ZE% z|E$@#xg5Ln?7r4m-_zW7VS>01x%?!~Uf97ZLUWMA)ihJu#_r2WU&`P$-H0(ucUG(g+t5e zx)8GE-&Jq&MH7L&LU`6*8;5^p(W2oUoqWG|zag}8*Ie9rrHKG3Sy|nckEde+{0D6- zJ~@^imlqX};0dOz^er_0x__0)L~;Z3x^xoAS8z)e>cqW%q7>Ajz9be`GwD-0K5E-G z%+!jC>HuF*9l$JhCi5oA5gd7=8hrjuNn4})+X4rG$YWA%M;7_ zj0hz8nBNyY9e9`>)6FUY;B!moEKjLYu8(Uu*g^~HX4DyESVuOb2qj$&1QELAe378D zsV#yFT3boA{UdfqAOTsQlp`<%J!>=6$P(m5wHh!Q@mHSSK>$%4X4=Q;|Fn-`vME-4 zaDY5S22r_)`hVBzcIR8W`#6AQ5=;MLw@6H$X!;v|8^4N=2V)XJ%pfRycjg>}8(`Ky%(Kdpq)*L|9(Xkk) zx=+)oJ1_NkE*A-&zX58z_WkQpKlc1eP7)=tA}_{1>x(^-AyL6(Nqe+dRN$ZTubGQ8 zl(?AsqtayF2RbKxlVd%s-ncZj({c0kmt+c*_H>@D`S!Uh8|WJiCXiTW%41S;@j;+p zNd1&>%-L=E$B))#vt~dMo$XL6yaV9W9B~nsCcRM1l`^@)EIM3BoS5`eFz4Ftd(ztXuI|a}`6spq`1PSWu$Aq`-atb{ zEt6iPB#;#f<9C<_iD!OG|GM{%7RkXY)ix9F5%$V^+lw$=WM3Gtvqb0uY4~v?6}_Orq%O zTC<(4JR*<^;3C=g!sI&M--kybHr3HWBz@*+{uD-p(NcPtKF`Oh-`Q&2P4_fb-dY^& zTX>7A#%fJ<^)GTGq)SAimQByj9f>RY@rqeQ?r-9S2g>$J?=&4qV1B(Ng(p7b>~V%u z--)?$D;J8(#a*6$j!DMUq&MM08w}*1cbEq{=Aqe!)*WQaGXe@)#2i<_{!e$+s|Ll?iQ<^J`F(s-h-Grl~$ z_{qs8m*0NtK81F;04C?;XOufSOL=YfYs%&aq}MY-7hmc5Aiy(gz=RjacRi+>j*87z zrT}=pyK(DCPP9)&eAN&xhaTpTI@ASOIE)F5N2zyRRwsI>c-IrLLvr2{XiNIw8tAC% zQ(*~k*};p{7XnM`2JI2Z?s`}Hfv1;Pf%gh5vru;{H=u}4maSo?5R0)A#pV->k5?$z z1^I=}uR8AM-48++-cKhO9UKVq;M`iaP|?{gKWN*?Rh>ZgO3+=f-CsvMwtvrEm}Kv% zqIu(%4urNK)`laNUI2*Bbs+toQ_^mI^q}&qL`MMt-OKbFeO2On8%Ul_!WD((m;Cdt z=%btX;bEY9zRg|4fQ}BIs68ciMD(A~GV<6fw_`psPqM>A-_b!~DTluL^ z4=bm-!?}4qJ>I<25sZLR;g#xBYB97-r}b6cx+5X^cmnfg(z?F*gN>X`-A|K$N%|)7 zqT2EX6jC{2W9F$qnE#TvS$*(`p7aFTB}fdTLtn>xYzd>-bV8>|Gxy0o^ z@r@XCJ6iJPblBwEZIj(4tG?gG&;%#&V>h1o?lkZUOSfK`G_rh)9p4o(Im9gr*6z4Z zfu?rZL_WDXxoM#jkQ}tA@I!Ml!ss$#z&-LMvEDZHZj8}l$w;8|R;3s+z*e_@ZWyt!m6&4coHOV}*))YYK;Hvfikk!zjHZ+>xJ9 z7Az9A-Td~|>C!TO?w-7?s_<=xmqZtoY;#=sdU4AAk}X9mn_uk7ZURPQjW~vx!nTl?sT5u-hi6j< zYF-SGoQ6VgMtE4hGw#O1o7CLfZU>7*mkhQyW3kj_`iy#~L4-w6e9AFdVfgPQil8j7 zZs~Br=DFV&x;Y*~;8wX@IlW@bl545?_f2nr_x3tYRMh8=>eyNb#R>9DO{Pz2lEF|# zvvFTyX!1)-YcnP2RrNyDE6XGomBTFs z6U4;H#S4>`$Ah?nT8Ff$=?J&wW$6q1F7@O=ob{9z@4CCs?@p0u)x*JWcbs+(KS8Y; zF7r#hKfPr-bai&vt5d?0>5d-~Vy_Nw<{~}b1p}9(e~$2SmLT_Pq<&laSduRAG^J1? zwuw(eM;}*lQi>5b5TWV9gRl4I!t2#7*8+x|t|!_@K8eQ+9;Tm53>cz*mD@Aru%I<2 zZ@g%&)842vdy8=Pd11V}Rk-i>p3IUmlCK}XVJa97JB`d-s=J7|88P;>v0otv&3H$l zc=IGaw1>PfXGx)jbI-ie`e5KON@PVCO)?@@M-N&RgXUO$pQ=P++VQYLkBOG z=&%3Kb$-?L;~kC)Fe|1yf24Ac0)>3mWglUmm05aDcB;@NbGTrWWOK||Pg=Wub7f~r z6!w%5zck;E7h_b0P6l!@cD8Lc-98`P==`dwq|H|T(@^s0;2S;pT&k2I0Czh`Wr-)c z?el7MRly@O{(xZmg8z0qJOI}rJ?YdJ7BsiC-x?zddvxIUbxiT!3JJE`k^nONMNm5} z4sIU-V`o!iKmMN?*ucBM8m(u5D}LsHiIl_!eIKCV%eZRxMGcf_yvmhU#>=mZvI-<_ zUpT)W?X-vl(76r|n9bYs5lc-8a)CAQgns8T|NU1pHv*JE+l4#iu!SVjX=Vt<%&+{< zNxPByHJ5=_R$r~e@jmW`7ifSBFGVGS-xdAi!UH_(-)5%?E6KlrrkaMU(TlKbK#vTN zgb7KQC=lt=2JEjh2%NGLpvzUejiFPcp3q*)7NlBhIuiN+0>GupwxoL)ze9lj_U| z%Yl$1_mpXzRgVv-UBpc17zJ%eD&G4m%$Q(vJ?b{;Qf@nw zT(e%fMQ6OeLWms^#eED#?6~Txh6ogQVk#-XxqX*+k33Qii3cC|1`B?5&>$**o|AOm zl~S-aUZ#m(kozObK8p6^{eFnyEXJ-a8CKM@*@rQbbtfQkeS(m;r-U z@+k>!G4F`0aFIVEzk4!OGiP<5C@5)GWF^!esELaR{s~vQ5Ihcu^#USocr%_xJEeGM zLofyyJz1-!bmH`*R_rKU)MNmF+=Zl>_R;inh~+W~akT8<=IBui`7wh~uin0P=DVOXy@ztk{w_D88eeh~Gh4+T=Tcxt*tJX`ry4a>8KxS_qsV_ehS-`)+qR`H8zF z{`PNoC)?>l{qD;!tZuf`i9q+&Xh(mJP!{=vq(KIx>AvLIV16>Ds=aXFk{bdyfJjEH z)sTheL%DY?kAj49-s|AK-7&oR`V+hc#DdEnt08l}t*T{kmwt|J$g`1rjl3c;p!_w( zx@!$azkD#om2T0@vbOKmD9i(z7saUko{DSe$!V)^IG3NM`8vPrdOA83wH#|aeYkSN zy?w^E+$DR<5B9#=s5N-&@Ie(xb#*M;m`Wy$TgpuDwqTN(Fxh+~&D`C5_PQVI$uo%D z#rWq7fkwu9rI~y-)orsW{v0Y87E!CSOFk|^2L>m#BL1M$yXk+1IkMkXi&uaIr8A|h zylqaNdrtnKSesG%%xlfAU4yqHWS_z=KKCjs?sxsptBkz)>p!M(Jn=a5-C99a8aIN3 zs6(PT{zx8r$)`B73YngaRE^qPr)<-=sC6ymxnNp|w28XFV`-wh+To7dS2$YYY88i7 zbVeV2=(&n~J8`E<+1S4q{2Oc?9|2|rCVHqno70y;o;@+X_R_3l&wH5p?+ymkcBtJ< zw3?f4_AV0@^Seh6or+wt@9l}qvQ<5W`@VmgpMB&Q&3ji)O3wD}cA|t$BHwBjmMs#; z6m!E{cRlR%CF+!+$Yz_B`p}`!8*Tv-@yb4ECtu?m`Mv(Zfby2d zmw8PioLPD>slZ&eQ}5Xb*t|XK2Uk{GEDGD(c%mCRS)tFd_$J)tRa@BZAusQOZQrjX zoO-nVKPTfx$eSXiwaDjEPTpA=mOV{TXs6g&ZpqDD zl{dLp@_Mgq4r7R6gE1vHan`CmlLgJzD@+BP%`$i;w*xi<3E>5fO&t(I9Xeo}?za19 z&OR0L6(ZS1aJw?rWP|FePG21#EI6&Q(p`_acQ8l=S5)tN#&*xP`f)2Vr*zBx(vmwB zchmPgIUnXrbjdD%y?xi+7-`!{Ve8?a$!8||Wugbc6Ibf0bKZ^Qdj2&0Vib_;4IW8f zaBrMfZ3f6HJw`bn1?}(QtP@@`L@);1e6V)(Xsulj_s4mO&Y@MWZqn;_hNwb=Y5_Pt za(6TicDu$ug*%V2_yJx{wbA5p5!Df?v@wZqYnkg4iR8CvF3Hv@*p1GNycj=GhPX7J z3u>(zk}Z#9l+{ugNA$7v3ygc9bt<3uER32AZNOAVLQ8p#DMRVYY?B|Wt?Tm~ z6bz%&ndWkAlpguLighdCR9#x%W;WWqk+34i_NTph7oV;V3Ap|)1hpzS_@ndh33R_B z-hMl4P^o2?Cf^oM2iURp6pt~p`ZsO2-K)U9E}97bSj{XU?G^MGsQaNv{r}b1yT>JY z_woO>)~uYlYGuv?Z5^zf(%JHK)tXtkq;l2Ll$Dh!A>}y_T&AuIu_-*XMnBy`Jx_w@e9o*z-h! zBJL@B_DJ1!(8lD3yQm4LVpbmZ3GPKXrG2kfW`G%!HqSz)t7nt?u(oeS1Afa?g_-xC z7wfYdh;vo%E2AcX(WlNcceT)M@=@njm+|l*TQ~vq~PMcL6?w5go%At*G#eAn09{^`L}z-V4RGo0+yr5BTDw?r%=@ zQW+UUFPuU88>$bJLG?qW1?t6Wo8Z@b6Rta$hkhQ|~ERvg7ugeYZ1m=sJ zXQEfl?_ag2d8g}|z{^CinzPyY<(+rwI+5ZkDVK8kNM9t|NU-(e^jh*{8LCo4su$ZQmva3km~Z*b zOHS>>JcV4qir@W|;LzATgV|=<6;o%^yLKgO$+4dYJwmQMthgs@yDNKD?u(-@8EM#ZYs!1!0zIj6`d)}xCyGt2v9NxSk zz+v@W*AV3$CN5^=eNy)atHhj+j?wh>s-#|8279t;g(|0>HWiEW%E+j5q`aa;B^Azn z^hd40V{Rd$+@8d#;U9_5b2um@GeW5@5B9{GH%?n8)ie|9 zQc1()@EWS5#1fKo2rtY-!%^-ilTCUTlBh`72lGIf=tn*iWp5^aBL@yUk#ERXy*yaX z{X=mnXc=!q)>JS=PpJ9T-UNEAx{tBl0|%fY|Ce~}D+tA1D-o^%qC0P0SJvtiXe_rn zS_|!YJSz@Vx4|4fv$q?IRuzm?n_~d{t@l$i7ih7B)wSoQV#`It0ZrwCYAs)} zFwaQERq@ZuhSQUqQzCu9bfsIDQ!e~Vxb}&)D;CgyK@Z+P3e<nB=MW9&C(o-E><6CkM+3(d&y}bCbp0*F9dw(3Y8m*lo@Vr=H^)8J^h_t|?e<>Y z{9W}W5s1^NF?FrV;vq}Tx0~gsi-B2#c8<{!y;!4!3!4S}q!-pwWFCbwXN;F-;x4HL zMq#9EBJh;U!7qEYLcs8+Nh*;g5RyNuT5+SGRv{co8rLQUNk<$&DFDcVvY+X3pz+hv zqcn$L9o$|w;I>+na@)HS5kji+ww(c~vUn(^A6{Fc`a^bqli>fhWPzeO5Z`P;{MC*~ z;7>Z>JBBqACx4WZlkV1$x$fkh0zs;8*)?{i69Q2lG_RLFdNLlD^{en17v^&g0;`tk z5{-}L9K>G-ia8x;DSI5E&|ICYHUguU*8#zkegfLJN*G!iAJxsn%?A<}%p7^Mgy;0x zQh&hHNI_10biPg>iLxr95t#R#|+fZb= zP%gY5PMLcuf7YNaP^n@0bOXB}9$4ipSE+8g#ZC|a3g~NqB7rt4E72Iw;rLt2=mLb} zN?|cmmi`x>qdE+xk)^uqk#TfwplI;M16fhXs$W5hnamBxOaydj>l5pM=B0}vN}@>! z8|hKr-D-$RXLly>6Ow9ovl`Ss1|V*kc#7}f>uW_l)z9!Z#AsRk(T-|T4iv?5?6ZBP z92%9_WpJixl;?%0FWGAbU6!d)XML8H>1S>f!o7~|a3t99QMQdm5uQ)6bM*ENIU9vf ziI8eg6oUH7KkbP05tGhf&aNoli7m%W`2F?cosU}^`FF;l^h^S5#8UcVAOQQZY_?x z1}c+})NdkCe~`EwlRI>PhFq9z-M z=lak%W@s>*jdn)1PF5yIXC_2Sc#B$mGqGMzZe&^35E$Xb73vw;gepAH{5Rs?3R zH%|d$;}FnaAK2fd?xNCW8XZ?EPp3nf+Y`ik^cer(QE(h|wS+MgVVRq4gXvfVmRoJn zz%hM<^8?EqJjPFvpToTt^w5|G>dhm{o_&xCk3qJZ)2{S(quN0iJIqsiO<$=Fk!Y9c zMIWptb1(V<1qWoe@SH{ETOlezpg*m`OHn zTzvsu;l>+iYg)rRzRSk;%g8UxF_-q;6qXlX?DJp0Pd(?zy~0`i2T1wi7p0|VpOv&+ zn>Ts=wxieH4sQHj}sv9wM&bZAZx9EVE|-KcCienqe|^=rlM( zm1+sav@C~P7D+KJge$OXz&W}-H(n!?w>~)F1J2?56~6(%?Nc2;0T9Xrq@7~s;GIIu z&GMQc@w5A8l*%s!?H;lpT#Gk)^#`ks2h|{cv4J|iPX%ZD>}|R7&B%%Zx@G>dlh~{T z^$At)Q#p9!E-(Ua>Nz+vXp@7VR7yaxtiQbBx{mB0!{kmPe8fgX#-tOxgyIm zCXU8<$89~T1jqXtI}xm_hVway(U$0!vKOjYfqq|~h%CTx^eA6nuX#_Q!(qYN96h!E8zez1KVU<5`!N#TNi z*faawj1KCT52u*M#$sP)XQ}kL^kr+BujD4`Oo7$IbW5;0^(Tk35J=!18Sx5bCk_WW`=Pxw+`6*ldebhyxy>I@XfY=7G5s*P(ID ztHadyC>5RUn+Oi8K~-RFelNR%i6|K?V@X z@Q~Y(>L3Q$NlTwyP^`sRj-i&=2)VE@A3wl31#ZNEnXXr(Iqfl!?0ioofB}0f0N(+8 zxv*b26>+O->%n~pMWeHc&&}q<3RVL$o53X;R-j1X6vPeF;U=V+^1(Wh&=_<`6a3T~ z0N?l8N8}sI7^c1J+)9mqzdbd~`>nwr(+1Qf z z6qj#ao~X1>V1dJ~O;z9yGsD*@Ep2h@?6%$-S=KkeIY?*+X0~GOLu)ohCV$SgnqJ8D zxpZ?CLn3`^@wohdC@t!Dh>p!sb;^lM_tnQH#F6(VDCBPs0IQ)CAZTK7{$@CY2p>ba zr{eiVmH0h}sgcxH=61-113VaHmaeA9^F$R3uF^?!~PQjO}HJnRe3c%qlE<} z0#ID~8?thpY_fG%d+#~Y&S00cSS%pGZFobhA|P;Ko?C{PsR?rz`&aUJzx&w8XWs{Gfx&C%FE`Q?Vxh`%y&G2Rf zBW)vnG~k{3bT^KWkIUsP-W<>JVGE`TrA)Liyw)^Z&2x1qU55rjQ(%BTePCqiQmZgN zQ^aN>VO8mzA3q}@oW0#ng0pu zrh_2qjGAvC6WY!qKdQkF+=wH?!XY$rYac{S96S8~rvqICUrtwZZF%KZNANu(xIUpq zZcgtS^8$XCf>tKyN5kPU%ucs}>tYgZEzgCue5qZ~MR5#>y~HYahUqKQs*n)j$RL>| z&|MnTtrs%FZ~DZhL;`w4d#tzM3M|#RO8G!>t^Gl!*QU4`7FM@vgI;+9=OM^{3sVk! z<(F64Jo#Jt!VstxT>e!$zXJdN8mEjyVe7@EBO%`Ni|Ef$K)S9m#{y>vabH!a0`>q% zI*rbXb2jdc0b%)8D)M(V9pr2^j1Pm`BJ`IjD+=|d*rVQsr>9>L=S#D%Q|AtW<-s9~ z#=GZ&`vWDK9-+fSoI2VJM8j!-0g#_ATAA|AN%eNz#5mE$h~DaHXj& z|1V69#l}l1C%ewVL*+sac7cq=p8G${2VhsD%VJoHW1pA~lK>&b^%9kj3X-g;m)VyX z8A{z?Ri`-r3?H$(uB;sCpsvIzY!I+q%+L!`&wV{z-15>e>tO9z0fz$2BIKZV1N=v{g7?nnAn|Y*5*RF;*@37?9r`QfnA@@^ z`!096!r=Qfy$T-+g&WFq0bhu7)6MV3Pdhy(87H4PxcPUNSGLSC51#vb0~ z*twgo3S^u>$Tb%G&%CV2S+tL4jQ1enX7uYm_TMO!!3{V-GZ9MD>k6p}6To9~FhwHv zJ&t$;IB%xT{tLrWs0Di+&g#FKD%Jm@gshY64TUSn;gkoyUcfQ88$5Z-W$=@(*3J_| zs65$0dT;$iS*=gcNyq)zu`1yUQC>B5KqDBy%9TpK4>?$ul9e?^{S|?eOx8l_q4wbx z2x@qttIn)ocs3N#%a@qr^iooj5BxYz=rG?sqp7?Lew+mN(W$8mzSUHPf%)wCF@cel zXRpzR%U06v<KJf+0{GxB&`f1>O2Tui>6g}wryt$CZ@0bNe0+>fd(SKl*@PW!_L6?2 zsyuqJbbO<{tUL^8|WJx1~Am6yofc{tL_B>3F?PWRY#Dsp*@qiIy)%#~k zGkjId7f3&}{_g|aLInFOEC0cgl^?NW<-5tIOjX{ILR~fIWPnBRcGzGMyme^8@P7&3 zt^ja^#xSa(WqH+tzU76kg_d-j@H@?XwkV*y&<_FF@2&*i#=J%=HWV(MXZKT+PBrA@#EeyK0}O8 zJC8#g`8eOO-0;AFj=3g~TzE)H!9wM+$I#+gcd<9b0?tJ$K0cMQXudTdiAorEA-fjRoW{vGpwtJ?p+Nqx&g1Y;7T4Jo zp4Ew<6`}4-@XMv+@+AEtGPr{Mx=aKPr(EBDGMVdhiaVi-gu%+NyBT*v6MlNp3})I~ zFQk^b6Yd!n4t+F%JnuP>FSSxlw!Y8?67iub@m4-zSG)4)r!2Z(h+)ieT{|HaUxB|h zj&C96v1suoE&g^uzg{XHb@j=Td>*GO_2+=N!wuIJX8a`-HhPH(=$#3rQMG%Mz()TU zU+JhSpflp`tFYV#bjrkK+- zGMe}ZP0{FlQ+h^UXG!Kw{Cv{ipq6i8bD9fxRLKi}`sI82sZsN4g-DS;RylGGMbkJq zX2}1)YG_`X$|9}|&b4aar6gfpiNexS2JS)ArAQ*%?(7AW{`8{fjHwRc> zi11N6OGrxQ8WKNog;l)&=$>w*f~_%97}#la1Kn@%=i4NgOo@`URN#|!_v|xY@#>CcJ z0%)mRMh;)WJ(jhrB?W%M;YrJzD1kPJ<5pj91pxg;re=(5nlZs-!{mF zLI%zE`0z1ZO2ij*u-}0*Qa4G*8|S$`v@YakWkuu){5XYd*38<`lwrkDJFZCwoTy@& zr8*X3#e1Xyv%VdyVpE;60H*d624!{^sQrQYh0TJiU<0*KJzSN?F5A8E|CiE$#5Yv4 zavD_fLxKln5_>sdQa|uKLx7Gnpv z_S`@)3%v3Lk=J>VedU(EX?yF?${I}lPwc7udNuAf&J*?5^^lZFp|lkK**%dGhj1dLI2ovpYe(l61c=!Wh+=9H`xI2HobU zh5FP5BF<0*O5wjieEP0>Zz8062=(TsIhW|Gdno6EeoH`|nWs*X9eRE+dz0Gg_2=T~ z1i|MUt$*JUL!Fu?OkfpObI?`StKXAW><&u^+*rmUaZbEevR;X#_&OxxlDc?^Z=+V?3tHM&iWOz&q#cCcn5l8 zr|6GwtaLrMkYl@``P#P^W5xT#l@iq1LQnk@Fl1Z5m1^?T@AaFhC-rg8`&kj&as_T%-QAo%pUT2Fs<|1z3T@hfty}gAg75Xd6#Q{4tu^!F#?Gv4q9&Wa18qh(u|*nr8{sKQzrphq=p5$p&DBEZOc1FM6SMAu4YkiS(I1Fw~ zK^}pT4)dPQ1j4(gywiR*n!4#AJ=>+$9YI*={uHt5=eA zbZ?HfDr9Rj>3*^sXyAgg?_&==HG(-%9(BIAP2mF56t@PnmhjB^hu2TiJuZCJ|33Ko zc>a-Fxv`qK%u0~{<8VTA_w|%Um+$qJJyLTa(Ir$Lt?R{}7_*y|d{S4h!lj(iT}S{! z6cBs9$B^rpRbO;$t9pdqzUq~_wLdFA{D!VLwbo6BqYk}!R3(#A6`=i)zS885evQh& z(zV%Yw3e$^8eIp%r`FBJe&*+F@=*uT)Y~=2<11gmjo~YsDP_ran^k>LcL=WaE?^#q zO=);?87^sc?pbOd9@8*gc7wtGy3HfT+VO`KOkG~ZP)_K0tt=*|Rl)|((;PC)MwA_( z#tt6Geg#X4MqvS?eZKJEOO5~Y6d$@T+KVd!8rjqM)U z(iK&a(%rINua)Pk<@~w7HA9)~*2(xHMwl_wbmJ0|hdc0u=C((S!{Xhk?c$A!>ed|> z{%f!;4wQoyn3I?>BFXP1Ftg=YAlfNf4CsgeI|`@tWs9n zaYRzY_;F$mZEuoJqM5@x60iK|-{wFy)}Ri}7yB+>F-%l`K4V2|btb|NRGMeeYCw;5 z?MG1%XgvKn|FVFA(8qD$SNF+mP>U6#<6&C z^XORAu9E6k(79_)5`)vb)clE8F2kos^Jgprj_w{YpFJop%z1bQA!d+nnb!K6*ZWLr zMH8?i9B~409wycO1{yju{7x0x@ln6y*FknXAr#CX8J^NdCy}Kxd~Nkudhp;I>v4lb zrGbVd>SrMcnE-(*arQaT9uk8N(EGc9N8X=dVT6MbZ9H0Ym!PD|&o2%id~ z1fZQGz5fPysPVEAnY2WC3mDuDrVRABwb0j#j15|LIcV#V&&442fqo*+Uws+w7n`UD z`H|@!{B@Rm_oGv_*=;AV<10bfcWOWjGEYc43XB{uBoC#ntd*r&gW5?NXVn5ZI8`X_ zd?`aR8arFUCby57Q{zLp3U^=oaU8ykDuUlt2L+64);Kf&N>4$1RZx(|?m=q9*qrp* zJ%;W8kY1Y2quI~L)^t7jbrb$OGvv{vN&QHb!M2wdmWB&L5^^Q*@ia~2R zU)nY%ziPMUpA;65L?e8egPa#S`qiN-*fx91pW}f~p+9sDn3-u)%q4JSlOl<`J$Z$e zg>nJK{e#gratlwe7ro{xX7;i+)FZ3?tXX*tGowNxwY$jod7fuDj4{Ft^Ck8E?A_nI zU7~H(G*(@c{k+A!MVnSzWzGxo>7jz7=4xIU1Pq-iQ4MgYpsaFsgbU}jC$wqP#vYn^w71O!@LP{;^*pO-RZi366BSE z_AG3na%0$GeNlZBtVZp5SfO5clTuEPr0Mt3(YDc!8@}QfqYrLw&Z+Fr)8?dQXOfoh zJRUAcyf934t0|4It;8@!x(C(rcm2iL0WIMUdS_Y(AilevK;|Q_IZnhiGI)Bto6`1e zbog3O#`f&N%t#L2+#X+m31}BS$<+<@t5SQyZ{HoX*BX8=SXKKd!JQDmdH9f}8ZIj0 zIXCQu4(DYR1$}iaNq4*x?~7I}@TJmo4*jke!K+Mbt&(l;7Y8XH)}_jw>$$ zV>3l!56+Zx@@P+unM`gZPsbzE1`NaJRU8Ggym)oqVdB?m^YplTIHqWrQo=5d>$CeN zs(TUhbF)+DFE3A9qjJukS@?Kq$Yprn#BDcJ+&pFG9PMg9xXQMU`q{!~HR`P1F>;+WHAR z+wbeN4SGUJ@`Yw-%A4+}!X>Rf_P20eX*$FE+)C&ST@=xcX_g%_QVdOkWFoW&6b-@_57PwQjH>d>(~NW2$cy6q$U@+3|L7n(CQ81_2q4 zj8KspQ2R+6nYn$V8_7Fs)fegP0{oFqR^x4tyze)KY@M+=?{8RtkVRgK<$I=4wuvxaofVl~+z$1|eakfyuz% z|6$th_s67!u+Y7&*8yKbXNAx2RWB`)r}@`rCcH3p(PrY7d%yMin*q7xmuU4_2AVu^Np-j1W_;YAm>O zSeyIvyuiTo@z(=PsPhAj6bNuY;_pBp(>ld$ zHG-+Anr^8iR;&$JTbKuvIEn80XP1{*H+smtefEqs?9@H6BbxmUWQWHgi)(|6V^@o Q_!9h>RknKkw+|iv2Mcm$5dZ)H literal 0 HcmV?d00001 diff --git a/docs/static/images/pcache-readiopath.jpg b/docs/static/images/pcache-readiopath.jpg new file mode 100644 index 0000000000000000000000000000000000000000..4993f0072af5c3e56cb43b8d694931dfcb6824fe GIT binary patch literal 16381 zcmdse3s@6ZyY3(gDq;i_6eLkmQ4#S1Vj(78s;CsBrOHi+lp=%_QVWzQAwdwrMNHKS z6)I5?Q4ynZYq=yqxYSw=_duxR*2>+?R0vZtnVg~jzyJ1}efEF)oV}lYp8tf$%$Q8p z%v$T4Z@usPy<_MzD1aqL4jw)T7#RV;A?yz@D1omJFllE1z{dyJ1OUK%z{JP|Fvj+b zfH~Nk<=^+Q9~jvJ^M3yv08Sd&{bN7GXu}`Jb7{cbzaN9O`TGky3)a@``DVAqx3GV| z?_ru(wV+rE(%_B~*j0CpXVWW+IIBN?Bq+OcITuxsC8pSiQGW3S)$*Zsb` zq}%>dKM!~|cO-ei*W>U$wxI#An2$A5Z){`%%&{;swlFd@0|e~s^NfCf{r)rT&1jDC z+<7LZ^A{{M!w#ri0?aWoHl8!rc;39((-{G z!@j4D%RcM?*Y5cE^OOY(t!Q}vt9eWXMe3@DgReJ`$xzA=+^+S9}Cj|+{=J9@}Fw~`?cXU7M9Bm zGGLK0c0*VgTL5GLeJ$LEb`$2a=fXve`1#P3q&Y~k9610QfXhgbl3>Q#4%Rb^= z$w)uXORnJA!&h~OVFz$bWj@WFbTJ?0FC`k4gp3ZZ;U0N%b-KkPyz=g}mFsiBw3KPn z7P15`P>2?=-Uo3Raq?xbLmPW#2NJBrS<)A~MzMXD~3+8(z1`XKN(le%8 zude!PZAZ(7xMTBK-jH!4it9!3aBTDW?!cHz$hXx`ghPx!?v+YiRJi3dISx4s9i2XK zImk08JBj2^GzR6Z!Zq$56}>xM7H4SBz9y9<-s=xeKCG7Gnchn71%wVcw4MJk&@@G8 z4|_F-5TU|q=lY4!;X+HW@Jf<^nHrGW_CZyB_ght0#Z3?GMA@Xyu6E#p@qSwLZt>bJ zM!zC_@R&>gnSrLkwNOZjN|S*n@S2_>TX@GoUY00ABy(!wSToeH6tX%1dxLj$P5ila z-eA#TxJ6y&*v7Xa?k|D~Vwog`Wu`J;|E@WK(=U&{Os7{S8+Q=hb$+ac5T!|zO;|q8 z@+%u5nZiqi8yS~GU_SC5TmWiXsf*-|h4VYnb_0?~LcRz&+O9bbCq9l0aeMNl z4onFd6E2s=?WXTd^?lMCH<10rj4+=ZMM&kZtjviJv*&dq(P642(>)zX05-Xq2EdfP zon;PvB(#)1W{k)2tw%`9;Qs*3zs#5qa z6f7(*iwd!S)+5u6i~2t;AM+p;g=ycfkKVVlYWs_*++Pp4-m=dDmIB9&_RRbD8&g9& z+Cq+`^v|Kl%(P#_I4CF(;^we6DsXAka?OMv*ye8l()=KS8rMk3%nR8H|DrYk{Z`HF zCAD3Zl8b?ov@KJV1+;vZ2EM%kXz~gxgYP*dCb6d*rgKB=rE4#%KLqM5@x?N+!4KU{}Dn6f7J+Ycz#!RKa^`O;Z{a0@M#XgWjg8?vN?Vy8!1BdEB z(S&Ic91%dVTua&pAHHj60CLDy6U0y@&JGUp>8jhU`fTw@DZ0Q7DhPq;V5+ialss2- zhGGHF;g|)m4(W?HX9P>(&S^Eb?PieJ06gPQq=D{1uv#sE4+~eojD36+u9;xRN&-tV zh`x8|mT+DhL-{&UJvSEZm?`{@kh*215s_g4=8E-2#4r^n6J6WRTCHr1gt|AZh!s^2d%HAA z%wI=wwKT>h*%;jB4NP%l`?ag|WumBHtG$p*P8=_pJr`O zbH|n$msC!)px;hg0zWTfmS``5(s4bgQ~5O+fF)6EJN;KgvXXG|Dnf>-f>CraWL^E3 znMJV)vJd__H+BC5p^^Yed*~bwQ_`0o)lS8g3F1Z*l z!J{uxJJ=at4LM;kB2fh6)CtW2m1P`pudKLftkA0WXE;_-8*I}cG%q*-PID`Q*MD!^ zMj~<=ok@<&)MGASwu)W!h#BMy5*l9bXbrJ~HBep)$3{>`DMJWw=3_YhUE_Jb3{j*e zjanJPQ;O1rn>*_Q95XAaxKZ{3h;xy_Ud7r5Gd3H5MH6od4M5AxR?{n^a`1>m_H~p2 zFkaWr+7AW}4k?#nkrP97ht_bGM}CPMhQseN_iMj`G%b=$5&(vjM(Au0+~=9o@J7&2 zg+_2t1en@PcqSLW-oK~oH0YO2S~7y-s-8aEgL@)6E$B;aGXUk}7U%t~s3RP#+m3AM zl8o1b2?|mvxq-ZtSxuFawjg`qXcfg~bZ{+}s%*_UN`lxs(I=s`cSiq>)RG#-UqTFb z{@HyST+qf|=U*{a@ENXW2H1e1o#ci8euFb~py=PfY;CXGe;Y3RA<0`TE6XhpjeZ1Y zi={T_StDoXdJJ7;?R^0IHqU%?b@irdnN4O~aM1hR3i9s9#U;T5hvt6y?J@KGf8YN9 zd{UgD^B-sXtw8H-0Imk81iJV51HJ|z$U*%QNZRqg%N>7CC`N{^zto12QdBX0iRA@q zv{Cxo!c|Pq<7gX{a6xE-1VX&DO2#w)C_+Em0hA=i>*@Qu*^Lha<+MD}$KB&)kJ-=f z_lNkO`Ypugby-X8b1NX04BjDaqZ;8m>qfTMKMtuz3W z>)VYCz(NOAo*th!vCRNn6pY6jfcAntrpN|f7IpGGwW1BR6g`y~fPK*b;a{fFIC(h^ z8%)Rw8_YuXDrVbpv=2(S%r-&RLATm8<&M$6Zzp2m3o2f5!;R_d^IyL zVg2l-jcoH~pI?q~lD9yEc->c5ipId(3>vYRR zlZvimy$7e6SGFayI10haph0GsxX`_v--vE%uk$gNryyZxAIq5gTY5W(tEEpRO5UB+saHUig(OLO|x0wE9BroB^$eEOZ?&q-Y5t{}^B3da_*HC)|5Uc`1sR>L`J0Z?}Z z)-;p+;C4P4#$;r{zS(X1S^S??+J5yyIoyP&AvYL8E{9|Z(_J!O}e?PwNj*0u*< zzh{NVdeLoWZo_Ed%H9h85?YD@@aw!>bSG-(`Lg70gj^ajY^%*@Y(DdCwav`>+Ivxf zTLckrl8StKDs-&`$?FU%4j({x;z{BdM3Zu!E*_iJA zM~tCyNEo(;@7G@!#b|6`hL#T*03JD#Z^8xw<~$y)DNF^SDSOW*ZSTuR$T|uzx^*niuh6x|qa^FGAl~i?uD@D;owWS$blq{? zA>K6kQ@(75tha$)l1th@+c0OH{u*kt@q36e;cNir=*tbjr|yJntOSQi1CT6mHKLKj z1?jXLp`*TNF;s zmAC)oMAYEm8vr-_1{+1>wO+)wWyL|fOXN>*cF2+rp$GK^n5UXJ~l!2Bt?ncM=thJ}9i7tea!#^`3x85J)g1N$}34P;ND z1^PX;%1@lg$3@KyHOX^Kl`X%Bn#OS1A!hJ#NTaM4lVubY|AH&$vIz7eWLj4hu7!rg zf=i-B$i};@pr;USVx!Ix3D@7L(4-jvDmqmK-5S#o%5%L_Y}mWW)O#&hq;;AAk$sa3wE2(4EIR9;OhiU`t4z9a9yM&80ez z@?rUkxR*hV_+1n{lh^G07*1^1v(ma)L6AkMwNY>osBREFY0)%EGRc4_Lhqz_#uOic z^T2EGLkA1!=Tt5igci^=*8qGXvPE3sUzCjHVufG{lOQHJ!PyCp*+`Ji70zq<3^rEE zQ;T|p_N+b7+(Ab;#OMuSA!^2!@2m=3e+|0UNUoG-P6p#XgrA3lxzYwvC3+34^#chR zQ<}>HH6MTn)JN;jVtgt{ikt%nUntq-O34D4BgSuq{3fXJibRGHYcF{DHmGa0^~)q~ zM;(FeX2MhXL#v;*t^GS3;cw_kJXPYbQYdi2O0t z6o~@e#|^;kVKTmwXE`d|MO}ux-@RD@UH`F$n>C3u87+KIDnM+M^|2>vV2SSiQQL21 zUfhyrSxre)HMjY7l+-I}$B|y5-~CaKc=0OJ-mkEn|DyqT8iCws8^>0#GZN6j0Gwc? zMH-J}VHw9D{YP5)XRyiWzig>&|8G9uA0YIf_D{^Bz?>qMKTL;k%7~1|@t!U?#2a+a zW)sG$^EdZa$zI;eQ9BzS1K*a!-u;Oh5Os2E)Mo)vzAr-E-rVHSmvuHzTl*a=ZBXF% z{(lM6{xYC66(Cz@htd(_uGs(_QadxR(La%?%Z`mt3CPT*`8$km$M7se;=^_5A=oKIl_MyRFak1 ztE8Wgkw0>i1Pvbs3_$v3{bkN@7x@ea*=hjH$iL;?GDeq|Yu_^fMIY#~8x|BJ7+ZW3 zl8l2tmh=swIXkow20)Ahv6c76dEj5H*}njw{|R~fuggMzfGM~f!-BYU(IS$i{1isC z(-eGeFN#SB27oS_&y3>O!lnDcijdV1HNyZz3vy&+F0$jcusZ4En#ZED(dypwG1u?< zuh3{TF=Z2Fx31EzEH|F;nB}XuRFup*2RlF#?G}AG16$BTU>X!66>|U#tSyEwQ@Lz+ zP=3jjJ2gR2N-w!iP`sozd%6Yk!`+Vr(Qm2?UmOccObh@1QjybE)4`=ipbfILOIeal z+RFaHQ7KI&+7-hFz#p=iaM#V#mp2L5fTUu`v2l9Ps*}vX^_K@j6Y^#kd#;XYF z9)t@O5(~QAYGPf@v#yWfqk8Psyxn&iwnis@*1u!}Ohy7_tCIq1z$>x{IOGw^vo zYKFY0FPNn3qRCCvg(Od=q={V4Nki;a_Ey@x`dh5`wd`H+*?Tl}4cH@X3APdASHham zbGJr!iYge5-#fS7hL#%dN3A0!T9o7r(W1$Et0VA;5{D_;+H3Gg#AB8#h#UXqZsZxY zn6v}Cj9j!ma0z_j5&Q&wLA4>-k3Pq4PXmys;NhdHF#uAc+wMG+{&set=@3DCp&JQ* zK>Gr~M}YJ;KKLPzi=oD6c}?|;sh9Ih(&PAZu_>y2%}`A@iI)0Dg=~X)vATn-gP8hf zqtIk^MRJ}9vd72kHYzx2HI(Fg@Hw@U<`SXY`o{f8(%Kq0=Pv62Jn~5%2h0ZINiGEN zm=8yp&``f$g==BRMu(aCa;*bA1RWV@IOa>+p#R={})kPEe@%az9!I(D)WK|R}!bwKCQCl98d z4<4*n*+n~7OdOORI#8bOos7#zEOdJ~jo!pvIa-_%yZxkaA?Vkz!p+5;^&UhKW87;@ zwWU;1W~Vi;f=W`RYWw3UeNu70S0*{ijxRg7dxx04sXbmId+FvwZE;djxp|!O^#!qK zX{w}E#`oc~7~|27_7qDIG8EKGuChr*Z6Xc8GSVj5FkuB^q+1h3utc{o^E=7O)Mq6b z!UfEeLwg|HxD|NM*;kpDMcTkD(AH)*<2|eNU$gN)jf6P-zFxha=DAKdG@p0RAbw^v z)1otpB&`PpczkiY2rk$Q?c{;n#u;|~9XR`N z&+V$7xcUU|p6IvZ@kyS()v3pxb+7uisy^N6>$?Yl8DTOhLZ40k`4#ySOkCw$Ak37q zoc0htz{G9BpWqmJK3)#5!HvJCi)EJj=EB+o62c%cux)!SL_m0}l;A*blwcGi6A5 z@TpS(`?tR-|3<&bRr1T2`j#zRg?6bGXKw4d7&6t)P*`A2jSz67uXvHnL~}X#S$*!snpvXhgYb~XVr5D{jEA<-L^hvf;fLm zHdLhx08L6CBmVGhg{QLxC(@a$c=;8&LORfzkk?DH?I3#H9|2YfS7FU(F}aQ8W$`|^ zHuAISHp*fav0Z4%B0|#3c#SvgN4QK{OofZoL(SN+HLW6O8L#y>qMM4ry+@cIiQf#q zlr$a|I?`)I=|j@^TF>AQTIfa1Mx*3%7i6ig_MH9^+Yv5O`ZX1djlHDrZ6{Jx6OJD; zELj^Mb8c3hqe5fr&om2>hIO*>h&Lq4VdKA!ucwQ^oe|v~>y@RaD)l~b zyk%|5c3j=x#oXWcS`3j~+oiuo9`8rzaz@L*G9YOirmk|ZLd?HOzW)?${~y={?glYI zM_?JjSJYT0bL+PSjN9Whebw5St~kO{IG$+wK;8$do4zPq2o(?3LUyWzjJmz5`h{I8mrJD0w;|K4 zO8N3A()ynGQeH;LZcHI`!z~RSeFpPCDmb~*_Oc-MZr@0i^&WdplcZmuG;K`c1YvPn zZ_}LbGgy&dUw1y{0py^fW)n9mqdlq6+`^5$6;v*1KE%nM(wHDy+TF-Y4%OtmNd<@Y zfvs1zm3KaR$=80ty(9An66^m~JR|HtTll|55S~To3&;&T{R!c+Tey3(h~-b3{eNZx z{}*i@gR%b+@cV1v-snbIWQ}`fTy&6FxTZ7iB}Y7i-r(FBtFf*Or9T{&_2xX3y$E|S z;!$-YX{pvD_r`+&%I=%`w4bd42LoR^*EfH--=PR#O<}1mU{*};xX1p{0KlQBHRqiR zCZKPNnAOs;G7^Relw3H#0Q^{wrMedWQ;FU-VjPpy(+Nrpc5j~qIAUvd8+nq(FDIkR zIca?4#qSyOU#D5q_L?o(MNIm^PL`+gwc`ivMDi_KuI-kz+mTh)H79fK49iYDI_Yud z+TD@dZF|G+jpRpvwIra=KcLaP?B=6wK?5x-x82!ba&wlN{k3|@e+8}j=WYB+fB$Fn z;YV27N?HjgDCE}29_VtBo~I-jGb2L&R%f7&Vnv&k;H&HLW%PVrpKEQ<#L?;T!0_uY zCH<~rPA_zPr{@)w-+s$foyM7C5m62YVWk$qY%L}uNEgO=cM+p*wh|>nzfgpgVpcOI zaty#LON_YD$!Vg=O(<61NfTAM8-QI$Fy8hEwZ_UfrC5PSWsUa95#@$iI{2^AsPRvA z%hZI$n2&>T@o)97K9F~rwc{4t)+DR$cfnN43Cdz-!I8?|irrr0w#8+aOTx9Uw`*lX zT7FC2dmvj3jE^Z!ovLeHV=IgMB{iD1Bv376}fbQ_RU zu#b*1##teBV)t?ECWE&~--oT#FP$1s%1m1&17<3dN+rXF7UBt!t`rT+EjP#1tx)T6 z%`?GyO)fCyZTvZ`Sa>ngEidC@?@k5bsj2n>{ILg2)qY8)^r^LV485u1Gn9H0ILL|8 zq^SjIm{=^R})O zu~uub3Ry}PYmvU5y#O{=V3X{uz-5pZvlp^^@ymMztROI2TbPVvMfh`^M%*hC8kZBg$6)z2X(22Z>Xsk2g#i-q}|NB?v3V-%AIh%Gapy7eHMg#(`GA`#-P= zaR)!0Ie6D-lA0o7XyzdwK#Q}L0t|`IB}bB1Mr}QCn}2D+?MQ<8C4PC?+j6K}#T}oD zFG5Z{5y2&)rS$XFX~In`E4a1@7RnWuZ$fNy<3li~6n>ak0uDY<`gk|*A#7ziVq@Nn zI6{8o*Fll?vJ_dNVl3lC$q*|H0#TMA9^E7>*bDPWa3To=#E5#Cyrj;|F-Q0wD+)B> zDzFko^Bl3b8g+ngsFKXYQlSeZN@nAkVJ#s$AmQytw3=fkRi!!x+8)Y@;`8u$o}x!H z-j8_%!r;}40$$pAkyv6oB45c00N<&o+0Jv6nlC>3m zRjvPuL!+ixhb-hNc9uiuar1v-Qnh@zM6Izv7U_$Lm@~;;8FAww%N?fNrm@!sZ&J4o z?yj^hw1s_DweG-_3#QFf*|Zq|fZbz*&gq@R#Qq(MZt4os3f4BzO>l{{2su-etxo5v zH4B;e6_tVoXdk)~N|TSP!BSN}T&tR(D)TOgB3yFt3&Eib;Tn2~Rp*4OWF>6`a@f~B zd3ru+8HOK^S5U23n?MSWy$Y3(ZCTrx@`XZs}~fZ``9Kbrpt*9Ko*+}RbyLY zSkh|q0=hItTGd`zE8vpcpv^T2;@u7^+$G`;>=s)5nvUNAO1O^Qu1@x+jGd_1bpL?< z7-GrYUk7nh(M?RJqtLwSOk{t_DdxaQeIY`4KxKJ=oJ&DUv=VaJ1M)ISB%06zs!AW+ z-c=V|6e+X>ZPMyaK#XPxwhQTHxqJTj=zv6)E}6lvI)B zBo#hPT8a3BF4;m`CN}sYSzr}UFH-RBhi$oO$ca0cax-i*bF#LW6Ez%IrsrvMvmcDq zRPy-Rn>*20zixdw@GlK+#*-jql+qPbdg>2fYoF5Ond$7wNz_G1`8^H9ZN%xtQ;O@zwTZ=s@ z`iq?K35O5Clcjf=roo*tb>`v9M6*Mak}SE+_@_jwO^R?0oT|b+YB8j(v|Nk7Ymmo8)47Ka%65DeDf^m`%y2Oz$7TDn{wR6Gxv%Wb}xqATmUTP>^SA4 zs^4iJlk7p}8h|H)@gA_P4UdOYl@xQ<-e`Iwx*fg%Sv3ZdvrsbHE?R=(nRadbrMydC zm+L|m)Qh4e?2RyA8Ie+VxGV^HA9{NeFHkZ}M*L(Xo|>|BIDe@uN#kcW^;DuW@i~M;XzmXcV>Un5c7x!c@~2*YqG8 z1d1zqLK4{jMXPZ;Adut{CF7JnxR4#}Kp|X2BU#AI-iU8R(qWTl#1$@Kq@&Z?5_~ z5uqeKh&q3L9Q>V#xb^(f-GLF2wVi9WnnWOO{3iL#G7gQ0?2`}J8GvurVce~kW5yXi zp}$9l%s8Ji(9gcYm|MVXNZMw}uvA zl<;4rQCrTlo!O6Iy7svKYs6FK?C9H0+tz#{SvnPu{10cG%y) z`;&|1uN*y#(v6!@N3>I3p4aG1e;+e*b9h)^pyo`9TGL2b%A$h$-lyRL447zID09X* zmT}&a#ou=@Q@Z@XodtqmCf}!~z1S<;KjocwO7!;kKdHZLvS@>GPka(*ABZf770?5Z zW->O^ZTl%|=6Yel+|;98E|v9r?pBxYwKKD~^X{%pY$_JMxJQ#p-;GyRe0v>mrp=_K z3qr#lj`;PSFU)(;KM-XbQ(W?sI5G6Te@Ii_&V*j~T**wr5r#H{s4D!N0lcHyA_*`795?_`GD&N&uK6kmi($Lw{D@Q*T>wi}go`ZaJ21J` zwwHUG$@c7UWDfhB>oH9J0F0b$4K*&xZzJKV+R&8Yp`aT1uO?O8x>} z22s=M;*~FLT1g9_^%vBH`EcIRs$Qa}iobBoBO!zJo?yiNY38R>J~pR1W427b^gCLc zkr^%cG$x{|a65iuz#&(oU+^CK`itmQ!@$Q6ZblCk2G$~KJ0y|D}5~3*Q+-!=cvr-wfl+AO3xJ$&$?YF_Pt`pj5Ep{ z^JQ;k{v{~^YvR5Dly%!?=P##j56e*@$;NPAu<8m;74%LiSj5yc$*^@|9xs*XWs_PL zdOj}N=|pY!fs!<0FcH5Da&D~tHO|H2r8W1V+vEEq^gZ}I%$xTH^X7&56a+x)EAPUI zUe)V7nNbEjF(k(mMSu3PytC_T~tji_6&c~ZBb@I+$iSSkIG@lO=XttO5c zq}8x-6XnqXm0%_8Dc%kzH02c%p}Fi;(9X;vQko*#gLb-`81ZO*QPQWfh_ua)>5&ZL z0oJOHi8ERLLv>zWfxr;U43a4LEx5U(*s8Yz35A0o8|AC?nMdcPJY)f6+{E4jp1iU? z?M!}pHNHe8z>U^i=ANF}eevnwXl8tp9Zh?6-OIjoYp>Ibv51hmQmuw-$eAp+M>O_o zjH|@elTWp~>2b#@dMbir5A>9+5p`DP5KcVFRkitcPI=m8xWwuA{cvu=CE79TjVxgG z6?-hp&r0>i@9eaB`s>2otTXyz3|+c#3D&G=klb{>di+EgYZbQ@Z`mQ7%M`w77VhpL ztzvRdRDt{~mJ<})h$nXsykF7fGBM?pU2oDR-a06bKU?x#b^LQnyU%dR3fLGkL>*|~ z@^Kx=;35f%m+oQ?fH;;Qn_(Y}_hcXMgRL56M>BPEhS7Q7>c3$SPdsGH@=&ZGILOz1 z6bNVeSgDev%8!?Fk@ZHM7#CcpJ2%orpGj!p%whNAx5JL6 zEtIU9wp8Lg%zRcuNax$Ac`cafEZoOVNyuc>F;tX+F;z*9pR(-&Myy=4PxD&HixFQP zRw7;p`L#&6D7hM3Us6PKClSE;ruSGrdd$s_MIDA!OAsrp=9ti4G0a*3?NIW@Z>l+| zR4ca^{H4R$bnsdm-U$PjbXK7Lo5``UdWu-~`sJJ2h+@_$D299I^(eu7NUFJ0og18Z zsK$phpgZ%E@hBsmZ3!1+W#wGh7&@ZUhF}%S+t}dr>lCIfjI^1Tf=uKvNcpIsRN z1e39Um&;=|v0| zIKU&}N^&;eB3@(7+SgqdkJ;Aq^db(8&>yCa?~0eCgy6umX&$} z)SFo-ZlG59We`KbAqr*^-RNA2UQtjpx^|b{9?>+z z&0_6PF|yDd-(C}=B*$4u=#>02m9q0YHY-eVII<1=P=3+NSu*PUsQPAEJWcjzzE(gIn1F+zBePYi3 z#t@61ui?U`V>r6Lfasv}g|P=OTCTF&(Vh6sPivobb~SbIvj~gPHD-2w_v)jM?q?_% zeRS;}{s&h3v{B*RZVbD&_+NheLThU$Uia#j5qR&45|=THk+Ip|1Gz;K;Z-Id7BJY zY$$nRfjVe+Lc@$Gr_KU0dDYdpzl~bqnBkg3@dJl4!>hnI7tYa?ytLCRrsV*Z(RS%8 zNt@7~=b-l~xJ_ZCqpIY1n-?2b(k&%c7GLNuB4<)U^p0gB5|8 zh9Iy-Uy1C5=2V!|jBWtAIpH9sS+sl%wRze*S^IkDiba!k9|BRa!RCll)8i-v{oL2JeeZF@C{t=wV~x9~M{Oq%hUhB*Cz>#w5*@A6iTe z_F@~oqhg_x!l}+{;-_=WNJfXjT2*eI@?QPyr+3+FK=(E(Fha714}j%L_m-O^7nxaO z;rd+irF`n8Je5s`&}#ULYR=M+;lsD!Y^<#%@jNTKc5oSlf6ACc`^rd(V#U-HCMv~l zc*HLDA+o9xZc%EiCF1Sn8v!1raA(fsJ!WQy=pRk#!+b#W zI_6e@{Z9(rJEmOh#~KI`Yz)5uFs#__vzYSmw=1m-Ga8{AMnteZ0<|_K8ssripsa+FZ2e z`pcWAuKeO-Gu{QB9;veQe;LZT4WFR`!@f+i*h5kHp^TS_I4j?Cr7mWcHe1LdbA^u3 e`n-qC%P*P+Ey{bGk!^P|3&9@>Tk{r>}QsP3u& literal 0 HcmV?d00001 diff --git a/docs/static/images/pcache-tieredstorage.jpg b/docs/static/images/pcache-tieredstorage.jpg new file mode 100644 index 0000000000000000000000000000000000000000..c362a2d693327fd53a6b9a0673e99bfb38058095 GIT binary patch literal 78208 zcmeFZcU)8JwkR4zMJYylCreRLst8CC$g&^;0@9Hd5s@Y$O#+0(B?1BhOR7>uIspRG zq(-`m80k$22_n)Yp@a!(Z@SLD`|N$*JNKO5z3-2A-*3&j4$j7_8jD> z@l~U%5DpFq>Zx9D*F=I0)ec?>Hd)z&C-v-Gd!CBp?U= zdJciua!CID-i71zzqgO^g6#iW8*t3OE^robtY7o}`s92C{{3|aK6|zAf3)VPd&T)b z?jaw*?;-3ph}I2HzaYOrPrnDhsVSd@Xk9Wg-T!NJ@cQfi1x&OdHu0w~pdB6ts zM_kO_z;`86Y0^k1)vi|@Vhy;#(`}cG1=lTT~ z$G(tXfb;J^a7^`}z(or#SAW6d=l(b(bm@71O~+wbHA{-{?SMgUkrV2qlhj|J{RP>7 z53tAoOOX8?uz!bZ8gdyV(*M1S4G!|ZHv-t3Jp+=ZCVL9P%Ly_BKPNu~4q>rQgQWY2 z8}c7MI5$7Z_;P(V7k2N|jSW{At3J6*Jny zy{w`qjir^IjpkCPe1HD&NPn85(hWVpxc=oQ%47~E;FT-?| z6hiM58`6~|^W)q#%X1n~ZeX#q{Wzt!QsnF9a?hPVu{yodk1j|+PIEpRMjr!Y?Wv3L zXwfC~sUaNtH$bo}mzx^$O!wq|`gRf&=|R?5e^$57>En>r@b0%7s@_f)QPR=Na@|iK z!RP%bYUAJ>G)rHO4e7XuzfFjQ3Elzps3x>Tz;_P!^n6%Sbe(U=wJ{BoC&R7M708bY zK1qjuupviLRtHj90ze=ZnL9@7-T8f3H@FH2quW*Et49=nd;6xnF82nsY!zik^0n&L zVMCtO5e0roB*ih#_`F6SOS;bMe9j3W!}unVFEtM9YKP!{H0+%7cF@B;l^qH;Qa4Gs zIyI$aqds%7LWXp&iuZTPEyxXy-w!~9{_77^1a*4!oj(jd^I(0S6xztX#=A*XnG@cNpe)Xr@H(y?f1476eJ6W3#lGaD1&@M^*?bh z!wYBRIVB1cIo4AlH@L*-;ZJi90mybyj3kg{O!Y)fOy$L3^;gy3r7*p0$P{!gZ*`g^iIGMH0~sVlw2K&i>>f(95P{6N zanuLEl>X!s zI=dhPyPNN(RXR=*?Oi_HZfG2gL%qO;I5d)^A?RbsX<9zBpk5Et#gYbUsPxM~C`HCk1|(>Un8&Q+ zj5}<|<0p)J%t!Z7=F~t^9z$4?mIok6#QjTn$#xNxAkc?pTp43=F?cDBUE#1OOzq-R z4JFA=hrFnIB^NIg4^_>Sk}4v@57{;~>_bp)U__Ccw3on3(Z2O~nG?V^T?3r#1m2aH zsLu`fQmR(^+Mz(iJh*XkeDSpl)sFQPS$dM(8jL&QIAQ4ufD@Qe3A@!~KkxT<=re1J2B*qI7i?3xv z)`)ZiOvj~GKe{ebtq)&kS`|qxA%Bjg8!#N0S^DB#lnd5Wt_~t#bQ>s|bB6Xv(26L6 zT@LOUz2#6VfYl94H}G!t9&do3n0?wua1y__Uv+vLW3nLZ|?6uG^T7%le`K zPcKWFS;4qjhB`~hi27{|?;(je0?fZ7!nQH{(IYZPDSc1W{Wx8aF7$cZ43BxgI}jB9+(UC$a)R+^&b|_+9@baJdgI0WvZYD4=U%I+Rct@_KXt!;Ol6#yW-) zV`k_dLwUq^Vj6aO;L(_a8cav~A&nNoq#>!e1UI=fd!E9EJi;lqANCO>_UHE$n3G#( zL=li5?Z+7&)RVo9R6|nIswCZyaR*4D%da3L;d9CQd_!!Amy*gnt%+_jgHi! zjer|W;DnMjv#40 z(!#|e1lbVpyl8yAXb&-R&u~rlHBya?>r|P#JZIj8n3U~=%fRPeu_5(RFrju?l)z{Z zA<|AKVii$i7sZC$#l(k+u9cD2pG}RmQ3cI`6e_NJR-Gyr%Q#SKG%u|@+=JgAgp0%m z7Cpi^jQrO!78Ju=-?1Xu8&!0&N>{-hTyoFS*4ucV8EHiKH^j)DW-=o6>b7> z&BOS;Xe<~rmJM;s6LMLi0tGH#>R>Um!qoA#`FWKXe4Ssfk7T7}N1nh>hGZ4sMhlJU z&P>Nr)vf@3^M3qD*-MmG7nR{w1u_yCnK6 zjF)wUAqm(}4QX4<;```RKtIj2n(EPoKk~!E8Pfr;Q<=;lLnB$bNS}GTjU=jFud4{ zGE-GM_q?ntaDF=6~5yE)kCne|XlJNHlPcTv- ziq_|L)((^VBV-*~$JN&b(c^pOF({j0$_feE1?Tp3kpbZIc~fenhz2}4dW_-dMoA(4 zZ1EuvNPioG=${3I=?-6kOy3J6=o$yz!v_?eAeF&*(KEz4#N*a*Dz*dPP}0M+)Z^?F zg}qsPhlq#23F)l#pwX#4gf2poDFs3Hi(>>+YrA34cJ-CnQxsET&kCLcY1Qc>NKM3& zlx(|<#v%F*6nH1`FnSnXPgqqW4P2nTWa2=XT}-6?1`4h`7XU#9@u|2j!4L+y-R_+P zbS&D8!3mUf_*B1VnjN`~Zd12NL2iQnZJqHo~wdLSnmR zdwzp)dxM;uv1UMq?L&Wlh>*Z|>2Z?AA7#)Ee`z(KG?LeQ@YBS3(bba`3|r=Fb=b2! zX_gGbkZOv1+Rn}Tj1eO<%h75`g?Fq2fM|K+yCkt5W_-C5a(1WNi!w}37*EhRHunI! z!b$TTVS)5ii+|_qHgEbprIc}d6wQZH9cfqa);X9^1^8VaLCX@63Rjw_ZLA~`LC8}^ zn5r35*H0AaFLRpo)IUZc#B8@oOxf4X*j*j=tE^HaL)e4=nHGn&MA|R@=iDzxquZ^@OAB`*JO>Qw#`194ijcWrZ zo}oP+-o0&~{BFO$A+MK^YN5rI^5;Dg@`e(pJ6|7vR9X(SBYXnr#o&7 zs9XQ;9N)zK>a_Ce=RTO4s#~%rM78(MP2*!}!hwIa&y zll{!|9AWBP**gE)(qlKDAKqV{2H9^>ktE;y_@=Oc(Z18qhmL<&4}qvD-o44KAaL~k zf`3~n$Ej$Ar$vvE1(}EEk$0o-+&rFnMyT+)iR9yC!{e%kR}b8@$L>3FdLR6aLqw`Q zGo844Nw`r2eFft#vYao%;aAsv%$h?)DQn-OR|obzlKJvG*#>#rJ&zey=XnMAJ#t__ z|0#YO{RhP3QQsg!`y3!{eNtJdZo`R{<9;`$PdKdwzrWxd@UXf=YHss)=qo6Dh?S|Y zjOnmZPuj;YqZ0q9BLbK1DWSy45q6y)gIn|*?MM7d29kDh+ke)6In6&HT@7k}G!5o%mhEN}cu65Q!ON&uDbRRH z+xc|Q$up>f?!5t>ZZ-DCL6kvz@8Q=oHkO48@_y--sJC(Qn#m876e}ve*5B6$6&5t2 zfem>T)1`}?iYzU9w#{4oLjLNfwIq%AbM4rZM%CcHqNPqPfU_;*8 z;CCDQw}!rU{3lb7{KuJJVQ|5ij02#2!D2RjrWYgX+PYXWbYC{)A~ZSz|J{kog}Ba! zbape|?nN3hF6|)(USQ@v!Wi$_kgD;wfA!vv|0(Rp^4(ynH=f0G5qY@ZIVN)QXBB#sBO8*1!ka36f>Cy+{q_!_)cTn5 zxySlg71!zLrnSEf>VHENi#MMQ`INM`79R6gUlSGyv}3;$RfbIhxfpu1Y1U&HFKBt) zUVQ4$hF}HtiB;hie_OIl?oiC)HoTXp{Kvnx_g~xl|6}eAT*pte&{}Jl*q|hsDX)&o zbnD@=A2*y_0F-lAiSr~fao8fWTQ6;mR;HJ;UAL!SAhoO^S$en8M|*xldZE!ghCD0A zhS;v$nwLm(=XIBVV^LACc1-8oax}cEhCM!6PXL%)DMSGUe^H4_ECg z40v_#jE=ZF6nd4ZS4^4NdFi2^`D>(0LJ|BY79@nc$NbGc8>gN;pE5R3Iz^+eqLsif z+niD%bFK;S^1^?os}3?QVrUP(?!xcOXbz(%7xHG9C3OA%&$6HBiju8q%upo4^rrTA zfrn$pLBD0Kf7rgcF|*Q7Q1Jc4lHJVc3znXWKB7?QBtL*tc7OIbt!1N^zVY*y_`AOs z(*0V?*2!c}k^zp0P1*~HcD&IfBDeh1f7^F}cGGT^e8E;h>qK-e=kce;=Z1&6F z#UB_!TYRdT*vB%KBWc@A!(DLEaZzh z{%tnIqw=2=quee!`R^+ZZeQOS$CH?AoPSqte=S;U74z*i8s@^dodiT};j_+y(51q6 z6IfY}m0f?smhdk%gU7A=3I6I!ya9Opg)8_j+3k_xI5W5IpHxNO!rrxiSo9bjPjmqn zyi@mgeg4;~dEIK*kmOh(BAglrCyNfT?sc&t2XBBNYVP^oZ-o90Wj21S`z(B0tNAZu zAOcj5zwpI|EXF2sz!0>s!GBhcn2aUae{J!O-NYi`MJmxcVC)^%6Mn(|-fJp4 z!#{vFR$D*t+IeXzhZT<=vwDwJOjkpAHLBUyeXNa0zILT+0NS{*x~>?i?~~B)u4%Pg zZ*9FGb5rYcwtQEnrkClZLhIGFPdOD85j4BRo9Uc<2^Kf@x}~zMz8hO)ml4u_d<}i| z)eon<^OZfkg_-KRCJL(~^u@;2s8FqXK?_wO#H3Fq@^N6I*%4L5_*ZF3O2fs)G?ETx zxCm3%j8o2Gu7Ce|GOdeb-~ zdi(_Ge3<62yvu-pVr0tur9TE=CK?hcU>LVxyx$jBko?`R9UoKDRgYU-2*1YaMEq>T zm@zLLnc405uv-(S4clrU(m&EegCZgp3HYp*@rB~pwIbGi1?2w?7oP(3K{?8X_{y|9 z`bJ1)s6}iu^z@}GF>r+`LmKG(1h?#kEKNLI|9r%J;F$&JpA><<*R0)if5FPSegtD? z7O$jk?yIut@7T@iLj24v`NEQNd-ZKY=T^{Nx92+yINfu5W7_A(x2CLh9BGNN=Cix& zG6b*ApjQ%y@n&6(=m!Dld$$t}nM#eUjrLh*7$S5N+|2D=uh-#Tn5?*2Ihq6;611%! zLz$Q6l6yYPS)eBMkNbD>6EOk%UK zz5;}yE;z8zc6CXsCs`{YTkFHsYs74+yezST=WK`x=+U*}8Z!*XXHoRsmNvA5XYLf@ zJGn8I8I7OOGtXye^eYY0uG*xe(FXtlz%5NF111*%-b;0|V2V!mZ}38my(|2q>vn=ZLv% zG1t|*+As{LMs<;}sWsIZqXM&cU`UBaCW*~_>xH-PzO_YyO45R3^}EQg^n{tS6G z-Y)I(~41%x@f2{>)8C76etze{UB#xNR9@&G_UJNn76p!Tmz3D(aIVPc| z&k6Y$2;UL^aqa>cUk#{Yde8icnZO-lLpH#qF?_ji6CsHCa4{P*(FLXhOJJZ^#fyxg zX*!jd-;dNo&e0!Phj69iigeGiz_I~128(k%+gDur{RgWp79IycU=uR|IeN{!+w+k> zIV`@)I6Q)``qAKoIla){!RKF}rNPZt_P|^x*Oyr?W?HN1JdO{L>$YX718$^oAcgGf z*1>{+)I#^BumF=GQ zbj4ogk1X2$xP2CI2wpMEG2SjSPIW)TTDG1KwJj2B`h2*_=Oi{sw2}bMW z9U3a<2LzqQgPb|LXyci)O~Q>ay6)sWPHzfgX-p*9NP4EKX^wfP?jg}TX7#XNfQN-b zpJ)W{yh>Hg8yk{w`o4N@;T7^pZgr^|F4WiuG4>}b5aCKX)Lg%LsYR5{rLcHMT> z$XnNDPh<{X8L4umY;%^prRZ`62uHSWgrL#?V0WJ>DHA%bwslbZ8; zbsjFMF2`v;6LKe(XN)vfcX7eK!`cPW-KI7V@*Rw$m@NrsK3LQb(;7^zYRwpH4uo1w zFdY@xy_(T4sS2G_h==- z-0xU(V?*Y`v2$Z$iv2u^@oC~4^>KY9Su^Yt`4Mb7x!hINEL%G(`!~TIaVH+0GKe5e%Zeq!r2KPK)e)SE+`07h?`>NfI)o!A2oO&;*{%D*P%B#!z1*RH zhRVv#{cTvo^n_2l3?(taziQbMi88shR^jyZ=CpcPknJD&EzRj-2A!e#D=C863lrY{ zWs0WqHOQ~Z2+d2C^6g#x@s$!nqv59V7S9&KvT3Qe3s=|Nilc8P;4}|hD$flJ`l@5wN|rZ$x7IEgP}bs|jfeXWw2kC>kNRbqQ&R*FN!) zH$x?=T4pqdA-_h7K2ALIvEfK7MgCmRVAojciYrMzb?MAU!4i{9?jnZ=DJ6vW-?LM1 zPs2~<5OTeGj#kZ=qoGt2$-Dq;%Y`yO8V!grTxH49>Zl_YFk z${4Kl>%zm@g}#ej5uAiOjB&n(ebBM1HE%G zZ}3JKPZq(xc6_6hS&3_ZHqKMhgp;&7?-!IEGON5`5I7v4T)&E3OhilgAfPA&8NMHg zQ*f<7y4kc0`MG@bt6$nt@;!BYM7{srbETgdrSERzhyC~kF7)%nhnG@PxM>fxK8i;} zp$T2uI$D_)*dxdd>$v1G;bRX;w%IuDKkrkr-9>$5?yB(Fam7|fRV8CD6@pQU%}Zt~ zMxBnv*`p<9CPtSmlD{rsYVZ@Fj(@bhD0$T|;G7-yFO|)ug>ei17)1M_Hl7H5d%*BP zs5b=u&n>N)aNG(D!u21`fLBoBI&fSAH>r-9(pF8F%IoayV>SZ}DTXpHUEIZ2*2Jqi(Ua}z*crgB3ItnE>(j}RRVOs>?8Ladt6=v{`L2`YwM7f}anEm`MBu*CGzgWJ=r1cr@#2F z-x;b37FYaS#G6w1U^fN#Y3?nSBcbfclLwR0GP(MdVPcAAa!@m^d#3kvRfXj(qa18j zv_5q4x!jb#w7-34<<^S%VQw3Lbk^mb zjM~(%+T$AIZ7_9e#ygx0Alr>Os;ga!45M3NqV;7*Q7ZF%fxlOe)Fb%3!irNul@bU! z6-B8JDUL?EioQi{blHA!-s6@%3L_og0p9zPD&51~R>iftdWvR8qQX+kZhH@74_Kam z^?l*YNtIBm??y8|K8BHk8Mi6*HK`s~1NRcbQdKp)E(M>qDl@w;&}MT$tavQZsnK@u zYf?uZCs?$X-<&6=(o!($@E8{EA-v|~6qK^l6q;$&azR5zf4xIuAZ64aX2#^(q^Xw~_5fnhgh7FZ3%rB%XVaQR}N5ZWPd>=}}N? zSchz(IZ-_#>ym_>XMqAzFP3Z`L!IcSN|I(H$rIZ!86@|G6}st3d}C0T`;nKSr(RYd zCcPxKWF*E=7lTR6OQc(4x2vZw^pNI&49)~xRFOF}sl-rfSrISo$g@hZUp!=1PE(^* zRR}w%FWNyBO(podt>S~6O?u_9vG1w&4jRj54M%Jq#6iWM6e042qpUKEnT1ZWX7_W9 zuI6}t!P;7TC|FyJ<~oK@3i`s(*rFdHvs_g7^v0?ywLhBmh|$~e5mxrmzxQg_RqFek zY#BM8;&v~2NL=`FwOK;rSTvl|GRntFJz`(Wjo3StQqwI>g-YgbB-avW0Yr6`N zRf`>ay)j2J%KOP+J#X#cL40^{>5tfw`kurL57#BRRw94&` zZkP7jR6p;vs8|NIeq9M-nUbLMvqZ@_rHpb(-syvj3NnXBq0a=Wd)?F8zvyYYMA1pJ z0-RZ)SGbq<(2e^?pD7KTfd{v!S3QV)hd!nwiF}EISHSox7irv9Ywq zT6K?Ox$PZlqeXIbl9R@3Mm8A2+VW{$C#{5^<`@_uGE$%}Ryq3i^6k-_ay@sC+MF?q zI&EBhJlowR;Bm(2gOuLecbdFLT2EU%lQzxM@HW=`o}GUeGa25C!=cm$TlLBTh2933 zs7C+#5kRgdm!HZpf>H>@dWmMbmpoP4=9$JyN;0L8K|T_`wE9ye#g0nF83CD3(7{-7 zZ`W=lSIj5uSV??G|CxN=tD1aqukvG~C3>8I_g#|u7 zx3(rfD^KH1;n%jau3a_IRpz6g3=3X%&?ksw6RU%k;cj;Gv|y`vuTvkRFTSwGW?DS+ z(s^khKHiP+J+vxaASplI-bz@Z>uUYSjf*ORFc3$8)SX1v;z__HZYktsc zyt$C-GX?tR(7K4go1>Mhb->mnd9(g>t48G{GWb+WJl)N@CPJStLvX5a0e!mv>~6H0 zQr?&}YTvt7_iMxP`Ioe#yoEeJS_L0pGbMYPADtIjvMepX(5vQu=8o3hgzkAA{T|ES z2*2eoX@+a%t-?C>@~69=V+U?vxz7xQ8$bVM)X2GLzUl`mc-1dk($In2cLKfbPpOyx zS)qD@u6-*fU2+s|{wYY0>++^Y_5PYy;DdkEy8q9OpeviK!#%Jxx-jZ+5!AfI1Cm+mq%f(EXs`MR>L1V^!lYUfzm6gdpXTUmN4PE^AQ;pk;cq{fO-Dm!1 zawANt{p@uskX2&awNGraA(xd4RXg^xocN#T`p{(`lkubaFgB#dg>PpnbF0yBbAshX z1{1a7M*=dKTx8$x%o3NW8SgQsVs>i^82DVw8^(=<=)s`|KNXgC?MmhwxA(RTpDaul zF!m5s^U8|dkmtInAbZ(0XXwMzm@GD|LY@WfW`2K@tz1Su#z4_Y3Uqqmj&8L&K{(a z#8$*S58eC#<1|GyQQcf(0?1tK%n)h5M&*iQxE4Dj?|&at;hS}ECct90#p0Hia>7De`}TB|AIIiD2iy1>tMj&_>!nF}8tv3`K$HBdBIa1k&@W{S|f zoC6zSwI6H0$_R2~SmY{5$6){Hz3nf2ET;38&v5spx*x_zgZ;Idt%}^QdcCzM3%*eF zbLzn?0|WS1BGHJ0eXmhQfL*?K-jVvG{pM9jS<~wAdfb6}!Ws`)$eOTc7L^aysU3!c z*#wih4_hkyOK8#YMV-DFzuwxSqW5pwt7|6F^Ih7N+>~B3d&SR{QvH0-(s-SXN6xrO zA$>Sg#&EVl6l70KkD+i(rAtz6er(C=^2e;Q!)BI`L%$$z2v-xyh$OGJAj z*QVO9w(e#LS_J;+U{B*H@1XKPo34?-$H~tM4iLh7Fm-v6sJmZgOhTyh??l_hsrpT1 zl_MYKF2258j{3fT0|h1C{Cd2Gen+ipbf0&Z6ZDjkXBKFD zV1#;5!AJ+nMkcAm9?1ok)`;mI3nmQ+K#xpK16fvY-)3BCZRvZ;@N>r$^eD%>79_}Z z_|M*KZclxSkhbG_)MeLk=Ph!soU*3f&$k_yruqVfmB5mV9D>gxgT+RQgnry|*4ZN2 z20LF5nukC1_5iNg^^6t74Jeoh^DaNI$&q-_IsWNvV!Diaft0C-+~F4M9{E)F)0Z-% zg^iUAFn6kLfYM0nOzVM&gK=~KRNUXMd}EwQ#mTu-o9?W=yHA+^WLqVsw6 z{T+E6goW_6HA~iJQP;?)JEOEl5^T%J!GKg4>8Kqk<|CAia+JpE6gtJr ztI{52NJ;7yrYz=2++0c@sPvlU83~d;|8Yw48m|83R)VDIA1B%x8*#7u<`i5@DS3+C zw{0z6-g1q-kzib#D)0W{Q_=0v&=QMsVQNr~u$-JGw*}U=F!RYbByinbep@U0r;z)L zIe*Jn3#TpPvQ;NgJzq>u565Fo_;jqvO>UM2CC0j~gO)U?AbMn05Qv$FCs?2^eQC8s z^Pp^7Da`K)MSvlx@6i@KcFM<7MvsyatBV)$o;p>NKMTb1h+S_Gg&mtKveuknF{fd3&P`)wO1;T3+Hmu}?Q9sI6V~1@Yy`1SMspJ2>vol6+wBS;nz6}3^Cnu4&rqM&~(ruIo+!I9S ztb9Ec_LLe8~>vlpMdS1p~rIh?(8>;J!T|Vh2JH{}m z`r73eE_I3Q64R+Q6&f>^<1N@Rr-nMBruZrijXKY5zmv5nG!%7abF4D{TeM8JsBFR2 zRH@RB)M&(thyc?3R@rExf+*;$Pb}jmH^zJV4!&7!xNnI)VjT1%P-i&PGFW~j)$QcE zc?ZUmXGRh{P9oB3Uhb*+aIYb$q=S@qXu`+#{;4Vr{+yKg~%-P@{U8+8sV`g3%zN+MCfMpmxXTSvJTR;uWHQi zpbPNk9dFFF3Ao>+j<&NQ^7N^QZA37BZhK-Q6!&^pMW4`6R_V~TcT*cS0Up%RW#ym? znwEet6@NqdEyhC@-sP#8b2FS!#D)m%O%XCe!Df9wnTF`R$UC`#c`ms(1gF#zcep^bX0j134LMJYvo*RqZ0@;cFS)s_PShOl@HXbKbH+?bv3Q8y=Ng456i$zY{ z+W{)O(%*|J8?t)6BsE&fPuFW?~)Bc(1K;H(t}r4cB*wYhl>NC!p!=Q zbsKERh4tl@h`x>G49C4)FJJ{UHHGV&{{)MK;4O44ymA{%kizb5Zn7GjqXx>? zSk9>RQ7@lh7s_9)^k>5Mn5#!xx0bRad!(@DDx2*rY~xoA<;LxeVd{82OTj1jH&&Mm zCWJ80(puL0y{|w*+irVtk$JC$C(CA<(Y#A69~xV3oBP%1VMO!xHhn@mbZJ8_u#J0vIns3|hS!$l@zxLqv7W{iu?i?Jx=}hqBUx z;YD$gU_5|=!<}Q15Yn!GJE4sMPdRAC-^R0H~Bt_?vFTO`|$@cV?p<1 zCP~3vYAfKY`kU3e$J_3qVbU+{x(=d6F+NSb4}g+h^SCa%Q$vPQlL2 zBz|`wQ#&r! zZIGNU)SSh{wN;ji=Nm5>RbY(_2T#e?4{rIB5V7en2^9^;{wqLMsA${}jE^{BP+k>D zUI`v#_=psd+GMZS4*gs!zI=S(2z0=6ZfVY4s<{C72Z^EQ6jXj&zczj3qwG};bb;yI z6i(bY6-%1Aeib3F6<05BWVU?LmVCXv>J+k$T5atgI^gLl%dH%(BmQjJIB#rmyKu=! zJ}xcTEb*Mi>CPOjjp6@>vJ!| z!*s7xc601vbM@=5uibVQ8*30YiNMC89i&~~>4|d9H$Xf?swc5RQMSINSY!UpeR&$7_Q)aQ@ zB)k+7VKTTis%K&FeT}TaD_snf2v2dd>{YFZW5CFw5NJn{OmI)cFJ&LZ5ywv$U3$ z4?bpe<8fZb%kLxpFR~BnKarL);0_`=#GJ4d$m$$nc2`&1fX2)Ni(t=yUjFiBGl9H@ z)Pg~u*|W?n4TK~zlKcj#K%T|Un@b>vhI>PDg`46CoqHjj(<>l(J0NTOGG;}AKMIiL zj_WeZjg(?6Y5f4k3+dNIOFz0IcXQ0OX;i&m4g=@Q@K@43B6!B?W2x)LsxV5vr&FUw zIPmoFJDZL<*VvBRXKxflm8_jKZdbk0Lq6#|L<^eHvC7n0ncvVRQ^jMHT^$-Ug-wo9 zHiCY1?O%v5iS02LmFR^dlU##3RK36iQLq2r|b^Vxez4vmkYt-w^;s8ssqTb=TQJu*I`x~F%tol0C zmza*>o^-l;l9o@UxeK2nX&65#(FrOG=pcJ|9LprG@y{{MNxprNTQj2FnWvhGN48a> z27$XOho?+P3IkxxSMN@)a#Hs>B^QKd)FbEm$N!IDZ;-A;mtv_hHTKB+#H>caXb_ z((Y!iQxacknc%>qUXa-@K-EY?e$3sMD@+=LEt}-oH(TIz2QDwa30jOyO^($FTou}A zx>3TNx*Tz}n8U*QOt?rS^_W$Nn&HFB>1JqE+MVrWx*f@0x_$C$oBUhr zBD28k<^*mH~8&#m% z^KIgjjgcKL_LH098?BGEC1%C0F`tfE=3IByed8Z3-{xRV;=Pn8GkgPEbmT-=gd@~Z z&~W;$nBsYl===l53A}>#yRNs&jm2~(j4nOsB1nsyK(7kRo%zz{@qxgXbAeV#Y0vx< znPmoqmI2{p-F9y9p!8bEl`l%l$}Gv{8z^ZYzR-J5sd|iIPM-1W8E>|Gs(p<@1t!k+LiOU%3Thyvh3aee{vkGmnvpjcS3?(ATCH`e# znILLx-Igun!O;55|71Hxko4P_a;}6mHTdky`!a13{^mTE+7I>ll*53`i45+^d$Q5x z_;jVUR z%99O)imx9+sa$=za)Tu;BS8vcRh3oscyVWSos_Ga_4re=(P|F%*Mpr*O5*w!Z{9mA z_JL#*>Ui#x8o|%D)vDltIp-gpBgJK-Wv4$2J*bH<`1E?QO=?FaSMD%3cMb1EsMC7dMk3S+bZyzS!N#-zhtAzN55K?M1ZeqEbgFK@cG1o5#BIW+-TF^ z%oZ5O|0m3#i@%Jz^}IfY|47&$Nmtbaa4Q7D-sD5k^Vr7S;7z9o9a3!GDRkxw03mu`QLwl0aZtdrQ7y zG+SMMcnwoMLS1|Xy$LL}8tv4oq4GUrHv2sk6 zP1z7Wl~M2rQ&2-&xB?_^f)%@#2-OGP5u9!X&2s{dkC&LCDCf1xy49bs@ogshHdw5- zGu?Mz?*c0VV^*AT3j8ISjAb^Yz1h*FE$xx{D&`8Uk1G##%cJV7j{hGFrH&w0Yr8US zZ0cbMbY+i)SH;xO*XA2#a@Y`ke2JAw3Z~0s8$)zf!nu5WCr5+74tI+rw4YTQ7n9a{ z#}b)q#<(Cv!+m@&T!`6bVh}Iz6HFR)I?0R;$wLg66@9>yiuUm9Rq5k6_gjK=*q~lb z$0AWoeC64Z>{}R~2hMPuN_<;=LrzW?I63YUViiqCuJx5*2)Lz@p>IUW5EHR8ZsIfk zrur~5=|}Fw+~)c--tAs4v$OE1x5|0r89lR8(`KJZ;N0WiHMXui;Owo+PkR)^%XxK~ z`j#u9|2USjcZ?$=S6s&nG^^9b*6B0C_3Mn5g!2tsyYQ90>h=E_PDKAppd^BBT|9X9 zqNMSk;$~sdtAX#uxMUy0n`bmOSam>sd*;6_&iRiZ8v~mwHU=CPHox20yu0xV@`{(^ z5kwjyemtYrC8wfp3n+NgXr785ZW$+F;2=>uhw=Q+ASQ$8TGs` zQ`@T*`!H^Mat&I#^m$G_|38MGK}q*{O12lMAq@?~wQoUmZSTf2lZbxOZ@1QWx0yPb zPX9=(oHP^G;WIuo8-Ve4x(W2`wDj6#k4#7hP8aLk#^>q!8o34VV7{E!V?!34G_#m` z&vkY$LG2^NDy##BP9j1m2ncP|GBdz7OhRtgWGx1=B1Sf;o%BWju8=Zou zbr!`&Fw5n7sGgY+}U8r5A z?UbUV_7bFOR7p|$?wM9q?Ne*3NR^1Cq?SmpY!?s z&iS76TR-proa4AX_kCaY^}5#AWod#YeaIuuxkg7sQ~VbEvN*p#r(Xg09iiG)Vp==d zFl@6cKsf+-3-%nWLe%tDgF}9SR5M2dX&(w6Cg-I=G1a>0-T{Qu1+rU(*%PAONi**%~{g1-%x)!`>)VHSx_$@e)e^Z@Wif%0m+}G=i+YvbPMwM zg`!BTNi7^{Spa=LM4rcV=r_VptLuWN;8XkaO-TRgC=373Z$hi?rzqhBXVtwD5qtJ6 z1td4d-mD1tRw%ajwa!Fy?j}8kFW0f@7I}20lqB~%;SbJ8dz>mAcmRnJQQXriIZR`pMzCy0PxvJ3{-?y5Soq*tIxb zUvBrc1v%#Dhj@ zI7=LeUTsB8693y4HSnj<8#_XCmfbC|s(W5Q$B)w(juOada8Mr3Lv4Y5SB>!;WG7@p z5e(%|+iLCz^^8rBNw5hj@XsYq(?J%6Pt`=-ZeE0J-oeRp=VPIQ6T-WJp^Fc=)pEQt z03HP0gd_r=p$R)eZ%d;o$2Z7mST!(X#ij`!!jFQEzWysjB~Oi`M85X#!gU2Cw+1Js7Q~aD0fB9|(+6Uht04C5(?qZ+?0zeJ_oJ2Hs*N{EkuK- z0+9jUCs8Rp#o#o-Ps@?>_=P<=|CFV8P~v`}YV8R1!T|yK4N%6j0S_$-D%gD)%FFku zYz}U&QvBBn1C(`jgv|Lopu2GKEuAQ@0}9MPE>u#Z@esI1&gRxnf7(X;B$Hr)4{nqY zh|8rCK3`$_3B>TAEs2|u(_pAA4}-)~HZEI!GT0{(HQ2P7uJR?vBK4)^7sx3R*eluYVP zCjI+m;lL#FB!~!4^kuhw8(Kk=^LKG71o=gDQh>D_3Yghkk^A$*0sZ z`VW}`Ti>OP3ESpii4!?g^iOXKyzh)kEUpTN1z+nw7jXo6Nc!DN0@K(VhB`9%pFQ^= z|BIIRkCOOrl*Blv5_Ee;8N!KCKX@>c?3sIFOYhh_(Mk7DYobP|T($nWTxlcqHzT)U zrlJ#WxHN9WfaKKtiEt0c-X9XI9RU%h`{RHpu3RSQFwDxQJQCfXV7Ej}xk|Lg*k?RE zv2^L6wN=T-%Al(H=Cn}*qn_H+=)F&y`b18jW@>%bvt3X!i>fQMGPAf?myyXQFY)N% zF@+9u-JdW1iCOe3Di}X=O^T*M?*rmZkP>5s5iP z@%yFS@*fN3uJkgGFrH<*Bvt?(kGkE?iig|HFMp;gk04c+wr?%tn-5-#K1RK#^U+X0 z(u{rWilobU;AoMK@#@P@>5ZQO-9AP}H_G81Lo})I*ZVCO-fJmevy53grZ$ticb~TX zBgb0-+9s+$94Azf9vd7P255UkTogT}7At6;+)9oXagYkhmlI~1;RW3t`WLED#sd>B zG0a60yp0ZOynj9)Tpn{r=-W@#6u~4=Cfs82$K&~;6oIv>mV6d|{@)LXmPJK$i*$X?OcK zzN{BOW~9f!<;a-L@3(r2{snGdbdnse0)vjOYWNqxZG3t`U~Sb?PEM|ja7{H)LW&}|rSwm$6-O@INlVhenMDf}mhGK4HF3^G3aq1u}k(%2?i;A$)y z|El5IoquV9o1iB60xz^h3Zly9p8ylBTdKitrsI0FBe4p<>jA}R7bqvc&{iNOf&q`Kn|+0<6hYCcSJ{k|59M}($}H$CdMtvCP8+H`mG``oXW z-A_~c@;si-ocC26IDbII#9?njr}FjES*5_XBVN`ER$0ZFD2q4is&DEht76vMbxs?* zJn?vA;C$rl5hX4AeP+(-`DR(VuoUWqjp_KfJ}PkMJwFrV`<#ALV8Dhk2dVMxaACm! zSOe%I_SW^xSd_xiT57(ZLyqH^dXOsmb<2qQ38IfrWu^5e*4~MUkH_XS%>&CV80(S$a_Gt%E`~D^4R51ydFkST)P`~!H4dwJ9qQ=(k(knOU z6Z~3c-VeO>E?51&xHoBMe>=ldUl-3FK^vE-SNUvec-h;|SaJ2v_EpDF?M`H(tjsL8 zlC6`|K$ZCT7o)Q~LPNDY2@Xhw8-|o4kqI%=qz^zx#mJ6OQN|ep8`$673C2T6%kHX)zqW(dqL3VL#Kk|6qw3-zqLr9;ELu;=V=Rmup@9Z zBzV`l6a|lS0rnH~9ihAe#7kZ@QE)qO4u-BMKfigU%!;p7+yN%rrp?a+*Nm+H|l5EtA}>J!$N$ z2;@eIsS1-YZ^}~?D^VKxFO#2 zHON0@)(qw7>iF?4RR9~V5cabu2mT>VF3@vX1IjrJddcxi;PVphf^aSX;Ol|Yp<=Tf zZs_VqDG1eW2^i{W2dH0IpI$QmbJ72~=>K^yn(|>shfviM? z@1o8!h2wn`V%w$S`d8vByo^p)k17&9Q}sU&uaK;;4Gckr_}cltZ(cVizpiP#mcaIV z{PC;D44M%t-M*i({qd}B^npH+)WZ7}Ioih9yT($s6*re|Rrh9@1v9X(W|A>dnJ7yD zpOLR47@fh%nRG~_k3DUwakO(js=XBadi6+k=G>&x=)*-4OETP4r*}BHv^(3Z@0pXy zl1$WB9Hl=`a0thA!5@bZ_8N)tjM>OYR8QqZm~d;s4@{4)v?D!!r_e_eN%=DTILUC^~}H?GaE?If|9b+;e4IBP4~T8o=WBERuRg5v-1BH z2{@Atw1KcL1Xzyz7mYe7IOPt88Go{X@UEhEga)9Hv57S>5b*~N{*ENf&FKXhd_mX( z=t2BdIf8iDM;#eNIBPo{^YWS)bPInpd9a7 zi|X$H_-h{sfEYaePx#d>5HYaW5gMZpx(0Tg{~HWxu%9JEK^WQ*VxGv^5z+-yr!YjH z*dO2t{{o(h-K5yOkfre1g8z#BAjXPmB>ofo&w<#ly3+U;;0Dc?Sg07V*%(O5V z_)YrHh5zTm|G)jhm3~*W=gJNcYE4?RZU%fi8fE#deBOg^zu)*icIAA=-`*TZdfEi} z9G58*`qT1#L^-wn=%#4dNNwtj*IUDGM3!3cs6TP)~ThDm(v*}Xm?4{x& zVEfcmk)q;^0TR*S@WNt*Yh_f;H|DoGuyl$sa~Po?I2Cq8G-dMCnoI< zZ;}60Z`O#-i@4tl+m#g;_Ug$=q4ylCY#S@tVUreywBu_DUe-|v8TN0GNCLGZ755r9*b{*YSzS-HM-&aAE#?bj=E6N`M%@V)O!*;ZMPhKPpUaK#Dmw7!-; zOy3m+!8@6pjc&*g|{M5kNM`Tl^V z4y5B=H>>T-CBymUf@VLlB|-^d#Qyo#nm0*|d%oo?j{vDUpC#MyjTOJ@%a+8!09QIj zo5-PlYq|FXC(RH{m_{KSgpEwp77< zX$e$Au$Rl#Ron{Fki^V0HM$7U`E7@7X4j}8=9s?+bb)$p&1LIdwUPN6cW`y}pnKoj zCN0?g$xn z`@UJsfCut1e6T}mg#@h7gNv$}5W8rC9E1_EL5o&z=biyVIf91n#gCh_-hvP0DXhRJ z0<*UHSN97y;9p zi(?#@c-Bq9>Lx!hjo3iz8s%5xni0H-P53E-;tlVf6U2f%5Xq*K(bRa3Eg6r-VLE6SH5&B3kqU~1w*uE6e zjYi%Nf88;Z3kUT)X|ifZXmWyPa+6a*XM^>)u_QEXF~s>bgU+f1JWyUv(VtM|-k~8A zG5IBnETSuANn$HQ0h^zdf~x5;9G#r(cyiIl=NoGagW*?J@{K1(p!z8~dKnc`_a1ua z4g^>w%kK_1^I4QEe(v`(HsX|>fw)<19GJ+qt^`?O^n$1h*2SMj+$_IKR#3JK@IIUD zp!wWX8J>Lsj2rKvO|;ZX#0ORGcNpPaG<-L9{f<-2^T^0o{o%q-J?oOL)ju*HS9d#P z{^@Nt{dA0N+)#)P%7B%n=Zn2|MaMePI@nXkI5%m+(r>ahz&^sYrviOzasET%Xtq?= zKCJ)Yj0c65c?(W97E>zeC7Y#npAv z*iZh7VROgfhv~;=vZQp~o`}!>s;BqJS;0ilCfP&kiGN;r=F5=`%YIX>ppopd&T({b z(7XBAJO!nJ!w;z+J@%?Q8e2uxB)^;q>oJt_(i+v*>GaV&UU_M0tgUT;%Hpu7$Mse9 zuPQV&Ha0e7-NIq9xHwnG6~*kI5+zTVq&qquI#Rp)rQ3*8EF4&bEzp*4blDMV#Ezx5{YB*D#$&kz(&hlfashMPfz>s&iCIUC z0y&r;ITceB#$=1!VkrAESDTM42^2)8PWls1A^*Bw>!t&lxxOPLjIqQGFN6HXTL6ZZ z7n48edI3XEX{NzHk;*}b&Gr7KLo;EfF$yxg%838V2SVrom_~+&H0}t6A}VEKmoT{5 z1%aq*z8~rXIE-}#{2fV$uV2vQ11BGk93%8&>;$gOyOuDKKzj%qKv8Bymnp26XM@`- z`oZ1Ta)F5*A&Q`d9x7NNIq3_J!u`4jd9uxV6!X`DcXE>gLX|%!)G%y@T)NB7()XYzq z^K7J zc20qc^Ug2_@1Ls0F$oW%9H+@Cmy|tU$9B3(j~N(hZJg2T)~tWB#IlD5A33LYvgT6q zlg7A!{Ys^U9XfX%&bPat*&LCw4vQG6P^z!W%-2@89}cl3jghrw{2h(-zn+!$-7o$y zSnOWvsW@l-+|=}Yc3xL{REw8$9ct_IKK9+IEk5hqTvy>|kluZOe*D#WQ=t@escO5%=K`=+-b;qo`@X|#g0y9j1Bbisz|lE1&2+Qbz1D_iWF=8 zap#ieOtJ5t*Aky;BZ|rx=r)2Zn?Xt26(-RotEqRr-sd_k!X#a<%I} z_93PWZ#X}fOJODLzTY`(n4^K^RLNG!kFT!>L_$5xob!0K*FV+doX!mK)(xHdmU;BK z2!8-IGl_uWhOu|*)T|@v>VjbzZCJtn#!^khc9q&}>Y!cjL>G&yW;FA%DSf|wnf0El zia~D{Ms80RK1ONl=)F!>xnvncObkAe5j8^0v&nNV3dvd-%3S+sp=E7%X0xm-#qQE8 zwB7d~$+=a3gCW#FkA%(}k1IsL3p zs8NB&vc<-Avo)vvedqPMSx=L?Ss$fQyX_AaAMw!hwhDZZzb#!9LPht1Vqra(qqf|a zz2}uzTV0uLib{IfzF=o3w6$%6;VZ9xA~?5#dsVlzvAaNRV)41`5w_2)hU^lwDC~Hv zK#^HcEmMc9@p2V=ujO~h+thRKm0bFAP!f!cs!u3R@yKgCky3npS^I@EQ~GjzTld7R z>Yy_t+nIJCVe(LHhJ%$&>F}qJ5QoOGo~8cR&y2NYtlqz!X6p3zo&yJy7djlWx&%2g zsi%onx9j&*Rj90Mh>LvTvPjCYmrBX%PvK{EjN$6FUt1u;y+8!-HpGr%+lt>iUsS2H zxvX-s`nNb4@zGd>^wRlc&wWqwe$FgX!YcI`+E}<~5-qfpPh~&<*cPUexu9hGu+TaW zwYLqCeJ{@ib?i#3sj1l_x|ay`3;(T}vAI$f=~?2HtsRkyDs_Q<)saF;BwV0E zGb%s-)&A344h{~7uZT1^9Wo$VoamN1k1rA%DEf9F5VVyZ1ZK{kER+P507}_Qvhx%< z1h!Op@6kS#XLBCAM z)_|Tr39QP4;i?>gy8tlHk64ZNLH&-)AZj402%e0)&_9C=gdWY`Xx3^$LfHDM-tB*W z8DAXv)W*mK866sBN2roaI!&mD5Y`0Ep)K{pzuOKVW@wS%wd@PPT?oTmb8ScHCUSfc z(on?i2Q(52v}D1Q2smX$mZ=6sbVOaYiHX|M$u0;_RU~p&8~Ckn56w>x98bH8IYCOv^nGC zL|Hr6W$rma-&~!So0Bza*Y$rC4f)iiNEWy)249cXZ{d1}g|cqz$E1++JU?{_I|puZ zD>f)(e?d4rn&dhKA2zx;Bij>mVVL9fo4|>`CBfCGhcJoA!dBorOjg~{y+`a zeMd+}m-F4(C5Y^^?KrtFp8y+!00;_fX@ignEjI+uxN*%eXCwOoaN8aCd9tA^=I+hl zZU`KYW_<(-94NsgT+YV*{~JCxaDs4qybg0PW(e|?!Nsbjh>(QB&k(7YqX@l$+KMUI zBHr~0%Bw*4Jq9ZRwH@(Ocre@!eO>nm*4RgGcv)YEOLYA7jB~g*7j|8Vc#~_V-X`se##cAbn1zG7$|uZVdJl zo2i_A_=AM9nRQ~K1y1rK2ie~8RU-o|>bawaJkoN&OQ6UepB9y4d(MFM#md}^QSzgQ zi5(K~!%ZSo8chb;Q6R^}P9obE4d4u#DKj))v){c_FFyX2-wIay{o!rT5gCadOg^#G zZS9R&wz}=>8+lsJE-Fv;Xt{^lNT`E1EHhn#&Bau-!o9kD4HCUnPet2{sg(WF#XL~- z)Z@F3+YK>pmsK$FpjVKng?e_Ac|Oi*#NSd!BH?Yj*d_6^T1uCED_$%}Seqn8MZO+x zQ5Ot3#bM$Nyrvs_)T7Qbw%R$X9WsHPPWDfxDc^-Hw90+UQ+*lr6Z2%jnGw&}t9Tq! zBGtfjk+9>-|~?JMLN{n~o9> zUj97tF#Kp;(QzZYz9&Khp}f&E=}q*F>l?|jnh&2xbjH-LGlag zf}*jNWp2a>TKi<7(xQ59L5#~UoK*iOb``LFi+$qR?jCwK677C4PU<9P+{ngAiF=je z?6XXbZL{*W!=LAjOd#6{d)eC5K|0t>(lxQQJU`19?a))~7CjWDUIR@A_O-~vDC~i) z8+xP%>b?eAGjdW(dPOe%(saW|&q^oSWzpKtODueXPQIxF6g)>&rq8{Tm34e18-)G^ zm5|;}YR6tWvaPCMjne6?s4(*nO~n30-TpcH+FH~FF>R`JpN!3kuJ@HD`kfvlV_w;_ zBh|jR`ql=w*B{D8Z00lO;_SRyyaONRXkJT8)rNfCba^^0A$#g5=$y1dM3*?mDO`0n zrZq=Sbmf)LX1+~li#9t1kLm809WNSPMUB)UMZ+*Q8P`S(l--XzvcIPNgsnarafQN4 zcWEMA=(g&Zak=Yyp-%DiaLOH0My9-3s(j8awOg5%CLeT&^pJ;(+26G30C`L2wD3i_}kAoFjp$ybN7ABQbJh|z zO$eK~7?T=7`g`c|yAb|yoX%i>^1(L?v@P4;zk<7hXRQf-HIN_I=$e3A=*soMTc|ri z9;|MGUf{;}?jtdV=hyc^>SuXtxK}X(QVR|o>Z^qOf7SOcY=GgjZ{%@Ytu*?N1vvN~ za*5~}-gTM)c!ueDLpfUgY+fY+*O%q?L#Q9;z&kkf0++%_+c1ee(p+Gbr${KMB z?2lBPm=NTGiPs-T*A$lv|J=y14x#6s=bjP_xs#?>(@Yj-xJv`ISSbuvFKs2sQ}z}n zXErBFRsoooif7RHPoX6-@$1vc;GLei?63i6O$fqWwyT=!w;h8P-Pw^#X@0cIx;b-) zEknD_i~MIF4ic6N`1*mwhOVy+qn>n9d(9$fJDA`$$0h<4D!2vZoEL(3%ffILG$y}c@#Cye z!9i_}-vy~@)8#alhoyk=jAG2^t(HkaTMV8yMiO0ai$1>rL4M;-*?u5}{L4q*PW9QmX~wN-V0@A_Sh0s8DacQ0Vp+G8h`|cVcCEcb@%167 zly=(;mZ|B;e6n=j-IeTS(eml1HtqiEd$19icZO3Aj2%QpT{}}}Tj3$ObSAl4JUZ_V z*DUJ?3){dHwC1L-5=Z{h`<8Zu+pKAE+BWu$7WMp>WP4NzXU6T2^MTJU7zsp zhci~VZ3T)<(FD!tOFoxiVNG7eu!}5VHo}KWca6s9*QajILv^}r+Q@1Z31AK)OxFVp ztiPQ8+;6Ehqi4GaQY403J+I4_A3vFQWN2gK%rT2=J$**0Fn-3Ss$HD=4~6;EQNPL| z*16U`kpzru^86`~xl_)6+WwU9sJOB;rc+e`3=p)+eZ^CM%@kd3DE6M($A19^7ZCJl zp2jRZ-f2%liB~Qr&R{n+ZI3J4-1-g4!FnW0l*$DThT$&1$3DN|cXu=2{2~MTa5M6A zl*Q335C5SD{n^LQbj+Sw*L#A=o)4O(yF0yFPzgSJWT|}UI_N$3zEV``%K{rY?en7Z zN(K&)_irxe?8+W0Yaj8RUKgCq_q;6rOZzZt?+ha>?|i$(D66)O>Sm1H7km>fm33$4 z%xH1-zD$J-?x{(y@1l4`TQv(w!{FjN;xEaEcD1foByc$^L7#09c^r@O! zG&)TJSuyHcEX24lbG_7jkHGVC^yB9>ysHoJ4 zhso}=Li!KT?4qK_1B~Eu+qhF`DXpcMXslUS5VAJE$ZW6inRw!?ta(c3t(nBm>FOiQ zV=Wd5LEh9^8xd)HiHEgjNA8Svnv4&M=gh2Yl)F{y#(HZNs^7ihXjZ(SAqgfHfSDif zKaB8PehYZpv9BNuyC40`^!vqRnJbrM(%D1BH>3*S;y_zI0Va-HIX`x?jE=Yne@o|$FAIU zxFAQqGHz*^ay5$1$UVk`aSg({XGDLA0%lg@fg|byH8!F<_nqEWhvBXnkcEyNUBZUh z4WHJeL8aKp8;q5NP%xPyn&4Z7r48dsbAs_c=2-B=dS1EZ+ZF$S#Sv*4yX% z8SHYJSW02`E8njroiZ0~QWeE)L@q)1KF%^56xCw}umlsCDg!A+n`v#}C>v>}N#Z7# zk_?e-`GyyoY z9w=_zrBh*jR?L=cg7n=ORWDghM(}DEt#O`+CFh%wuM7p>Fz}G!vL|}GXX23U6^6%M z8TFMHs*Rre11<8}owt-foh^f%1u6O0y;b7nZzw*@-^Xn2H1?lc$dr+}LlINq^_I`s zSWUN1uf3CTPDDx0X)-cZ?{#`n7ZQt3Pv|E5e=z+tjI6<^RHH1>W0?iYCTgksG|Syk z^gKoT?)cmM=S@J_=K^Ksqvs6ItIQ3-_x#d2pIw)RS9w|`>`qO$OPxnq+MU2a_n|Exdgxux(8?vEnU$N-E{2?jpGw~R>s97=EGR}19;h`DGc>XUm zLC-1?XQ-cdQ>?WM%y0bFRJq*tziX>p2X2S#!f+J{mLE4({C|N8AJ6&X^t(n>WjhI+ zR>8!iwZldv#TOhZ32_OgfrVObM$b5MgAdNG@>)fpTpz%oibrxKN4>%ay^gk`^MmX2 z61)Wqav!2ZWi1c? zCeLdK20KYm5i$035BHssE#$H9hIWZ2nBe@5(cl=__|%3TGTq}C^7Y$G zIp?;{O3wQLpY@MyF5z6FPXlU9lx&Rh0Lq6MgIrrmUs=Wd)DmhC$~B0HNS@slD_AJq z#s0DD$D5WEDA$$Z1LqzO zXpc~65kwyzF~ZDmV@AQ+mG#B|8c&o#vtzQFf@EaPk2}O(Wr{x5QnRtkM!3iuzc)}W zT#`LW+RV3KIu0xISq_?%H*b9NYp37on%|y~VHf=a{ST-Y`1iKx(_cor=2v21l?E+? zmZGOCQmRU==95^#6_&xeMV9%}hRUQ$>*Acc^kijXX7h($>zKJ)bHj!Zs_-(ojHiP= zLxUeO5{I*K%d%o4VlypqR5iFYlUL0qLWi&^Mwq`dvabdv1I3(z!RhZTc;Uinv=q$%GExoPxV{i%R0$(1d98BQ?X=`pb?R97 zk=j>LvdppAX$=vc`Dl}NsbiQ%XYT6bCheC9u^;;maJu7&*D~zupY4mUYaNwO?J|En zY-kKqy9HWWxkkm=6<`viD!MXySuoceLp%9gy~HA^D7G-I?2>;Q72!~5R%D^*aIg&j zBt4>=U3Rh0^o^ZWKu}5H5(!!32S#`$wafu=p4O<=VvD$A)E$WED+8;0Ee1(5laxm< znaTK}VZRN8j1kr1e7m(#W$7YIs(Ez7z8>f2;}>%4XDgji^9)`fNLx zJ8c;%-zzz%6E>C-C2p49SC|`ZE&iz*v0HEdk)=poVv4TWk>h1Lm+C$h9SbOPENU-d z2U0uZuWHMF$2`$WH9ua#|F-|d(j}Ql3oI_Og4VAXom*1o;d2KAiqg;OYBTi?>7F-Y z&rw+2Tf{}m>X0A7bLeCkn6r6I_hg z)twsjuwp(N`!iTN@C$x?B@2IJbpGjF=8_vTXu7;yz0yf*)>0g1o^GA)eZJ8lWxJrq z&_V_}O-g=+8dJN;$i`NANhv0(8%U_*Ua~`B&5Y5pW<)vQaS{Dts;8J-*wEbaUCDDd zkO;iY%oF$cL#Mw|oG3lcBm87#?V>MVg5sn3dLh%@P4DBIh)s7BvD2y@)+m>utYDjQ zrAtK*n#wJEze~3}o}Tm|qq>kf_zU5%ff$t{#g=elT&Op=IPK*JnK@HO)O?=rM6!4= zIeu!?cU6;&NV+h5>QHvljdS^WOW&PJq~1t4eR{?*k~TFq$Rt|0V2N{84Sr(jwYv_n08ooE-Z|` zk#BOj7C#y@>Ko)AlfG(OiyD1HLzu>BJ8ANA$*5&NqodvRE#;qYADNHCB-y>zQnZ%s z6G4UbYW;AjNu_mSWXTGxlGii5aE;l{*y?QSHXen>8C=yQ9GX(kpP^sEpwP~uKS6s8y{B00kYf%)5{l!6J#`FNnimv@p+G+zhlk1Y|~pX z(A1!0F;rUU6pB+&HdHZfKH{cq+K8(4ps?*?jJWDWV94WJ#pIaIq7N`{Ja(MXg=M*@ zUTc0h>A2(say7fH-My<$AFWdJQ@<9nPJWcF>uOMgB+F6Y3qE=<3}r~|mIKT3Y*(=W zoj8a3cBD+BVr(95iO4;lS9#}>)u<`TUv9ifA4sa2}V zXMudpNS%ufoKcSqcRN}ZJyU_`H| zu|AjF&uSc|RrTr>y(}2s6!xDS%Oex17Toj|3K_Plr!7Xt2UIHji})3-brB3p4H~ z|ME5;&1!C$0Mh2NStLgN&{4U*RXlHw&gS7D(U9r*_uu}&FUdS`8yah|jp>B`2oYTz ziU~a1!AG3UTa9r(dQ=;}2@-1a8d)?{Rm|dy7Q{3Ac14R(23X|~woTCJgK29_P0T}X z{ACPYE^xOHIewXyAWIPDg+SKsM>?}{chwbG?PEZ&t~B`4wxs>|4{iY)?*^%ZT*HXj zlJG8|P-pgisC?B37fz4Kqj5-)FdqnP36P71_Xr?4T3|TO5AwZWKzc41GDuV%@gK?C z{7v`B7mEc#8z!r&iF05-atJx#_ukhQz^6vQL#xvB%=6@kr5Xwc=+Q58;b=Vpc4~|a z9pl80@{qIJfne9tsr5C(+8OO-+N@l%vDgfG@SxWj7j}7jYrTW3;)riuAA%piZJ$zL zI=^=^tbbOj`Fc@GG*0U{|MpyIYOAm2)5(;s3a4Fdji^RkNqqzaEM@ zMAg_Ib$i6`w4c_o7i@iX!mMdf1w#3RshllNR;C7DZ+~93i`i;$QT$zchnd2m0l&t1 z5*_A>cf?jra#Y8EOWYQTuu~pxw=H@$a4&N^VDxcE@1a;z?o~a_xfX?T>$J!V4Pj3O z$HOpIL#x=wJQ=Dg{p&7)p{n($JVSsAE_PfBusC3~h{Lt@ELW~hg!{~<4pTiE!iyiO zB*+}C;KGY?ffV|ddNz6G{ah+6DQMnF^6q7{rFY&^SBmRZTOWnJwy95^2bgEo5ktmU z`t$s!JOIseW8mKM&D-r=kFoDl=?-lZu4#T1DvlqHHQmW}H^ygN$JM`)y9MBiC321q zIUk2T#@wAtqhG)sU6doAHSN))hrT4uuMwjgW!7eR-CU?>rhRNta>q!!HCl?8juwpm zwrJWnwPiS0Ohxr6^{NwNt#i8MYF)82`C`|=Asgjk`>D6@m$FNFMi%BX1!(QZ{W-}; z$Ab~@{Zn2!PDVMEmw{;S8dopGQEKkhg``1T9)y zRid5Uq^1&F2op^$vN_C143omD>QF!GW*sBuDcjpATCN<8O&<9EuIzuTIW5`cL&w`qdKYQ4#iX_l*h8;0)288+`v z-jeTR8FXd??NmHF=&7{Uo-JJywW+Z0-qymm&{UAB#fxg==2nW_gXq3CruEZ%&A!@(gIQue(d8HKr5yGeTUGlQSaG7QCf*Kg!1=he7_;9GKjw<}O@%J= z;0k(3hmJMd@DBL|emv@wnh@@#&>hFqLb-Tz9+jcH0q;lK9=N*5=yUpKPvYxQYW2_1ueM0P!gSW- z#!Q!TqA8QLq}%B&jrlsxY3DlZvh*acU8;~)i1IvLHiA}mE49fo%laPGyU;I*wJ>=$ z@;pT^H1DPU%+_0dLx^cY zH?dZUp@*<|(y_{QQIa1{G3T-oP3$)KYc$jb6ZK67LmjA3pK1{-N z%)p|OZMio{M!SndIjA0WrO4d9YEfch6NRcA)1;f5Yuj_O(*VCvZ*Z2apZtxf{vT=* z;n;4q8m+uj0ySrwQJ6(gByETjH1tR#e*LtJN>+tcMxn_ugX0xu-dOF&$?mE|a0;Le zT1$zmm&nLOszmi(*Xa=)3L6jhGSCkMEh0n}wlb~*e`M;&n1sUxZQQRnyfRvzoqEn{ zM+BF;!W?gy4M|pM_`Y~B-ruo!90K-aOvFS^Q38g?%m>Xs4?vfjg1otM<+o;?3~E`H zaUOOmH4o)AJU#R+10J4O&O|93A$y5ef#bO`rbNvY>(w^d6GTlPUmKqn-r7cTBbA!{ zu2IO1=o2e29eVKI(60?D6tgYJuXR;0To*Nv%E%>@)LhTaf0noQ(bmlpd+ovrO`4ID zouXNJildKrKvtFss`BWL(0&Ot+i{lc;?(yeo}rzrhMlwwudJgVo^96~(Q7FT^!rBJ z;Q~wVZoO)Fs9|$uC-xol;{G0Kv3AYphiy-TP)O{cw=_2{*V|)@V$yEg^j1G9+jp*& zT4Z0G=VoTBs#2St?4i{5JTWiNU8&bpvG3TaF;qTm#_zGFpRkueyE*Y2qB_r!Dz}T` zh^*^=#SZNNnZ$>KRrAkxIQfj`o%0%o-&kTgdtH;VYvjsezFqST>LZ8cu(~P(Lr6s% zK&*vVN%@vam5pom;evDiW_e64MtXcp`B2C86y@H0?||$6fhl$>J{M3u_UgkZojh&U zy#@I)LCiDPqSWGrs*As&!Lm%y$HJK3eO2eM=yMQSu?@i4VqkR+2;hIS6d+hmAcp2b z_o?IimHzLV34&(A^fOv5>wC5&a0er z(hXtmC)4ikRse%Hp@fAk;M{PdL*Mewq`ufJkyB8pxC$$L{14>|N{xSowsYt9pkW|?t(d{)9uSP`6{S+r$f zTH|}sn5M4@V2gRZ8mpiM(10z*;Aq&{b%MQs23Oly+um4Zo_r5@rh##_yshoX;6a0e z4NwBYw`uF^I3I|}jc6wpv3J5}bZcBa$rxmO^Jnh|UV742=myyZPKJq@ zk6~er(fZejn|z0hIoID;u3}b_L&oem0#q?M%&JQLr1>3+E3~dpq%8MN9qPHUwSVxD z@IrlE%^bZneAR}<1ZfBnGFlaR?4&ZWG6iMTmC-pL##7MClJ;uTD!;x^g1SH#x3=Y{ zoga#X^_04Ff6!W;$YbZsDn8)Zl{>RnAP)pbfH<%qs!h3gQ4dTg03PAf8oR=Pc5Tk1 z3``hcN90{@6dYF5_+RY3XH?T!_cn~s{}9-{-m4;v?t$ zPWCywoW1wCF4F-;9F9?!6dwAT;PsI-v>J{GyS+(hpsq3tnbNHZNP~CR*;U}Zlv-4jb4L^e(rIS^egm& zJv~n}44QXs4ljf$T801MuEkG+-eRfdI=F5wr7?^qCjuU7RBd6kizK`#kDCij4G1ZY zXV_k12t;FDnN_)+(;ZM{moas5y`gh-eTw;&4AFDb-|)>k^7a(N{1@?O zuU|&yKP(vC@gUzEp%64@>W3`jRvBdLpFuzC#TdOlpsN;BG%5H43(p9x$L|l@D;TAC zxlVePq^8@l6blds6y=UKSNH^>jcVzE%|SH~saG_fLLEbMx&#`}Q9_Q(Ci#1X=hBd5 zBt3b~<~c*%0kcV&(U{0Woytqk*ObT_&x!wxIYurCOJ{6{IlR=G0!rl)hQ$f3bT)(>S1N;$gRT#aiJ9?(Wfak@|{ zAd>ESPPZLjZ-5CAeCfpT#jMGZ@UYY4a%UT}p+x?sLJ3e7X!T_sY*H59*D8?BJAyO3 zcs}`>J+(YwrQNYqV_Iu2F01lL)`nN7S4bzKc0JTDp@!__Td;QG2L02eH@ykQ_uIYq z1ziz3eIPt_pOCi@ufiy~B;ld1aBW7@g)`{I_u9UfHjiHAR)h(Mnb!eQQ0yZ$|E}B# z<2|f_mGd}y!gE)=Aa3E}I5eS7!c{!*A+CK1f`ARtsW$kygL7j!`#%mlt*wbGGyeJn;19cMYg|6?Dn;16K=YD`8Bk|%+TUO zFV}3Iwx52lW|Ctu>E8D0;gu`jpWgF0{Prs0Y^U^N7O)?eX0rJkcu%nE7(2b(GAy8LO7mk?ZfebkN|HQ3pjNbF4(=cG&)wQ2l;iy&L(<7h+$RJO>m+vJ-b$ zWYZMU%9h+P`J;ky4+_|QvvX$ohHa1t=+SK zQA=`*B6~%PKdgZXm31l`y`y)IxpJf@Fu$@2FJ(7cl5W=0PwgE(hw>Znj;^VtUTc5u zc5r!UoF|o%-ovJh^E^{f*R?7VnxL4AL$|s+-3-U6Mi}l>xP{xTJ#kJK+zOOt){-YJ z3sZM!0JdYzqKMw~2y70mN;zqgW*}$GyiH%K4~Jf=Fi_FZd=<2Qk@b8W|7hm~Lj7l@ zl=8RTMb~@$fA+JF=;}jQZl-h=nYam;FL!z)GP+Kdaj6LKfW^)1Iozskp*5vO2-R(pb!C(`b(?RFnsyqx$Dd%H%vU~t4>%-(Ts=g48L zLiDwuSMB=Z{{4CJ9t8%6wsulO?WLeD(%gP2|B9wHhpHi4#EJ1oEjMHsQdb5I4!vC~ zIdFQ%)#N*@8jXTp6tpB$w%>5P;G@1szVVQbMya{exg2T*ybRkI?6t6Ojy+-Z2)C0G z4H41cpG-`!gl(@8p|4Ls9@|dHXE?i!jU&fkyI2FVFd<#~++53aaF`y3RjbW-^oFmb9$3^GKDhD<56^Gkgu#$}{V-u26nD*&v@MgV7s1d*!(7 z8EXyUgCbV6-}2rLvqDtRMiu?J+m%g=r7Th9X*J2$r@XFk3pKXn(S8fUjj^ipM-j^4 z8C?bE7(_t6Ec{TA$ap!5e$bxofD3;Urql;%_R*c#ZD18yU+S8@he)|V_i;6cH8OTQHU`AsI zv_4-0F=lA?p?40?uVGKmv8d}U!|tX^o*K}weXSe*k|o$AqII}}A$_k#i9Zr&Xl2U}()?*Ldf(ehYd~PQ*F&}}k?vZtDv^H==;*Nj-GDBVtr@Q7O+>4BHWgZ39BHs z&k*rVNxF+@&R!eA!@SwiV63L~+d;&^SyGKDqP#mOfCKm1_Z zRIB>+lEiY6J-oTIs@{}EYXbYHCL}c<)oR5Inn`r<-dq)(nO0dP{OEz*&vpXOayK|^ zcP=!agIn{G#i$2XWQ4fapbTC^25BS-pO6qxgw&N5(`x+9Hyu7COngqAA#N^~m8XFk z<>yT|Ptuz0ro*PnJY;Y(TQj(o*6oCPHc8ajTaE(tArxh0gW10czq(nGGLTh0)LEu=81UjXwOcScfv;mm? zlZn~u3eWUQJ&ds3~JF>A=2wB~WPp8Hy`0UKwN~15e9jo1W^tjD$`1(X8j9y9Umr$`m`B?Uf5g|;V{VRM{8zTLEQV;v>gh7Zp={dgyzz@|E-c#EAPgWXM#0ODYBL1Qx;ZBc zILApGi<9d_rs54bRQsinEFqIwnJh3<>hVQIdALce`L@X`Q%ba{u@VQ|51}!a^@C zvmlfr&U2G-Sgn=cAoGOoefZnuYSE5g}YH;=N7O|HbP;k`{sV&NgbNj;Bhh% z@a_KoH|HzON(Y8N2Do>&(SsLUCqVP}&s@`;2ipyHN7j$mrr0?M7awr6I3%U9TMd`r z5~81LZJrsNU!;9-B?}2kH11jIX7tqBaR~n$WZ%3v+ngo#Q^lkCG7s5$Gg`I)OV@*; zW{E~loa=4`1=U3qEgyKZuBrWyN8(m-~(K8-R% zOgB^qTwFG?Bj;ZEIhGc93ED&n2R*Co`#8REL)<#hO7QNr$U?CeC5C_&Qzp)}!51{f z_1Z1#E9449w;*xwM$G_wqs11|M0sryB4R)5W6mg|B3q)IZvVDVO9t6+T0Zp{-GHOb zNQ}I#RA?|FXj;p_CuoShW|6(hL&;<&wzx^Wz8w-;yofhMHj?Ya{bz&M?)+=DA* z)tF4c_Ooo-$1TzxC0!;u@je>FhO4tr1vd-0vWd8sZLw z)~xNs%M1#+M%j&iw6X3my-#u%@iVbl(fEGEDLU#=ahO>yE7F7&9q6o8J@G8;xIaQd zfY1;j(O1MVoA3yrm@#nE>HcE=3>P8O#F`<8PCXl0ZD=^UW07pDG{EVU)pwhDCgVpx z_b|f!^hei$S8rD~_xXO@+dG|bRsG9rEY`8a`W@Ijpap@!V?$6o#uBS-8VSL#Qlh&Z?q!sf@S zym*(J>%!~l=g|I0?=!Y-T_@dw4MHk#SxN9}w{zHA*$E~+BgzG;Z(I~@9v}#!Luvai zwVdP{Gf%#D5?f+s)voz(+M9a@x6~IsL3AjbIEw>=sM;U8&{$-FJaRa8GlMq|KmS3sb`YQRUU4dm_3gzbW*e-CVSGx1e{rzS?FDLe; z@j)Af9Q$^$XU?q0rgzY>O`&Rgaw0J<;w#BCtAgQ^7d(B|C_hE?!+t(oO&>^n-O3PT z>mbEAKlaZLP3+FJw1G!?YKhO1QIR<80$@Wt!AhHGj0Hcd{MG7McjDNB_)xzcV>=jr8ttfz6skx_QNpGl}PR{pf?wj^hlJCp!Y_(p7gVxP}Bw|)s)~ZL7S?l%XO%)9( z{k0Oqu~M4@vU=ks7I(n=@|Np`#3Xc7hm^K&k(|E&iB^Sazk;sUuWO6m&O~u;v#{e! zkKnu4qD$(Yf~s*VN*Jth@Y=W>b%ng+P*3)|eAmKaXVLTaAGND}^=*}V2HV<98KrR+ zhi?5)aV&XMRC>EGzDR>S+fsG+lyZ;>tuL2EM*?D>QU zr^G9C9Wv;17)2rpGecI?@QZV!RyF}kUguW(gSFBwf2#)nZE{6?xg7c|k$_$sar1zF z%^$D!f^%Q7TD$7~iC2Oj%Wx}Y%Cb(NMxt8_H^NL96&nky60)Rr_fWe-GDywab^zn^ zl>IHFF1=uCeE5~-TR+n9RLPeCw%Mfi-Ly$qi|Xktiip_{v}T5Ua_VDyZD72AT` z-*pyQI6atkkRv**KLVHpJt;evHff=dhu+5_D9w&s%03RS!-Lq4Ph*7*vCj^n3+{bb zpZUUfep2jU6)A&T$z8;EeBs+TxW0rKM2BWW`or)r-bJ*%#Tt43>x)(5##VwuxFGw* zX7-Tr!pPb}S#}*<&_ij6U>)5b3MKV$WO)K3Ucz-xpPtk^YIU*RDqjCLfnff_%iGRu z+wpn-`WxOz1$nLeFjV{&3vLukb$*Ff-SoXWX3gVl=Wc842# zpqY!COtLW?8eN{gFjJ(x=Fq;jirfJnkyqF@BTF@_Lf#w_j6kkRo_A;QyB35AZse)VUXTGyzsccTkJYVj0pC8H!zj@pQOsGSh8zd<_l-;`xnAvIq~T z-VY#x{CK~vkWd$Svs&nKB0DfV)I%DZ);MGsurlz={{gbTZn%T2YU5Y_@VH}O+b`Nr zO7A!QE)v*z4`uoyYXk)Oo(oF>Y-11j^7>kNr)KLKC^*blAv{)zDROG-yoJG&y(Ted zb;NQUdr~bN6D*S7CAF34I=8Lhp6xq7Wbk)+9oek#P+SR|t81$(;33tULDyjRh^Dy! zKOVLzDEj-zK;i;n$)5*+Hh-t!LmfKRhqVf)z)meosLf%ENseE^T2Q}BhOEpYNBH}g zWTfX3*5mcsGAq=rcr>I*t8g%)E^!=A51<8MnJyK^7F+Kv5qOa;XO#VO*uLx$03d{MqK&EwX}Cf54e;d6tAefBx^Vb?%MI4KRub2xIiVJRiuVFKtFb$7 z^Yhu0qnxJ=7qPc5&ZQw)9zsadu^kZv{zS{32{$b;Ev*-pT4%YA_d+Zgg+&gBUmoyUvJ zg)PPllqKY60_atXp7By&aR*x8OEZjs-#{#f;&MNM>|*_FxbNMdxzl~9Cc&#}a*&~F z03Y_b6Uvn~RNNRLcm1+vPoY*IODW5t_RDCd3rMfrg|U@yL=sef?@~ZBm)7yiF0}w$ zb)_O}1NVL7khTl`@3GA5t6G&GH2g*9j3@ZtNC+n`jHH0gxRu6B%1aMhTyWX*|C*5U zxz9n)>AE1V^V9W}t}|!jPI9ZcOPuFRNIGX>Cc@+5Tpb+sh3^;20NdYFmitUou=tLk zgpOJg>)sr{cFuq7uU26-_uLXQlxK{nGv@)XgNEz!cODu9)fj3%k@C*(-zaeiIf`l^Rp{SME&())$4*N2gC-)!fisiv!+*1)x4^Sk27_WM$-~6j&*2 zkT$;sXngf9$>0_p?}xAoB-qm=%_SZrt7fV5&38|vzDe=KG#}L0JCM??_U^Z~-;0Yd zoMRG0$#M6^?gWp;$;4_7Ip90`Yj_fhcuF8bCAuF|&^ua%{9G%}QN&SBdAq)~&A)s& z;efQ1WBk$Krd>&8H!@sz=pFklCws@6l_4(X3ttTo8chQ&LvmG@_^%f)wyNPCX``cY=vh5r;Gv@o1Z#iSuiJ!2Qk)4~e>zq9|4 zIV;}wZtyw+LHCAL)xjWX&CrA4u(T=oTY4wl*BDgq+Q~v z#M_EiJVPiA{pu57#QcRdK8D)wiW28Rsbo?#F6-PX$dM1cc;eue_ExS1g<++c46J6* zRBztZ4s{OAb3d1+FhZ`OJC90CxJqmDoF+KO{f!^{BArvFmE6o*%nS7amG%+bNn+tr zgGrW}Z)8;<;<*xLek8H&Y)6HN2I-)wip@&AezAA*N%Kr+;godmSX)0zmE{#spl0df zAe;Q7{(!aX8ZU=uozU2?6D0b%qLaOg&3mwc=k0NVZRoBo>!xtiK-%Snh86E;u+P71 z@!&6z4VrI&x7S(+?YAzRGned3Nz?T+B|^1P;a@2Z0cyx@7U5&h>Kt~;4>D*CypqB* z6lbq32$I4%n@m6#@;`sPlqt5(-M)h49Bx4_C(=GgvY}$&xpNTq+t}a2Irv|}v$0SU zu$=HE-VX%U%}3QP*v?3TeBcBcSAPguSu0u!y23 zy2DllYiNKxx|FNSDUpaSh@+qW=rVtHX>P%LkNixBv%I?u__x7_*&7k} zK;tko2P}q7;rxW6f7}~DfM@V$H<|vqWaUYiv@Wp#pVEzVb`s$SWdi?o==t?! z?pEI31zp}fpj$g(<4=z)WaG~XYCg*S*K)mDB8(Ez7D(9_ytWqT#skn}F&0DN#4gQ{ zDysh6r4=k^lduL^ZVHerIQyYh3iXzNpEb?^ZfqL}Et8S(W01ih!sgayO40k4!*MeF?b_<0KXwZAz;1y z{(DL_KtyCu7#T`>Npg zg8_cVDUc1l99tg3`J}--lJ>79!e#{z{4rV(pEe8m<7@x} z`r+CazFkj5Zm(^;4W8GA{<-SLh_lOn;cA5&_WVrC)5wnF(jdaI1a23O)T*r|e&4Aq zE%Y$4g+160(>3m3E(7Y-_iy z>&{_tr%ME$d;1FMy(!M%e^lm$8@3Vin8>EvibeG7Zt{*89KQuo$hq4oGf97zDP7)t z)!b4hMgMF%+6b*4C)2WT%E;Sppp;sw#eQ%nh5p_SWa&)P%CP)DnVyM!8h46pIzTF~ z4e^XMzMvDEdh0NsUbFWt^Eu1*lGS|eE{vV9OqPn=jH#cD4tNxNQn7;8-MU}Q zSkPgwRq||qoK>*?KE1FZSaMUhlC*_yuKRDLCypP)oZIZ07CwASJ=QkEbx{tjv*CQ- zH78x#Qcp;2FZ?4`5T)G)mF`^3%6cnWUYwPco^z{OsRNZ$$s)FuJT9XLWsG1>SRjnG zY{Q79vmjCh`YcHfpD}{iFP<);ed_I3%f}=xsd@8a zVPPRn>T#J#t;O>Ne=*h2>v&l8uRs$$*8X&CSi<6`uZn2QXL&G%&fZtbK6;m{%-aTY;cFZ z+FM{lpM}rxe&|w1k|b4CXVqNZsQ=qiv2ur}i}jK%s(Af5ZTY zFMLM8pK3+G<-Q6_7rY-LCazv%PY(PteDia*W}vgxUwv`tPa9nF*I?=Y=gKP-6lm&r zz;9^U>25)J+O7^Gg#(vVk3Ct}nbzbxHvPBo<6l1+#E16afsQR{16DjDNX0S*=)8(? z)S&XXaMFatQPXamaY<02Svb_Xq>|`=;uLO4G6-$od$XFjbR=rA%b-)*w4_#eU7^|g zP~gv{m!59Z=4FlVP>F->b`~n}E4bREyECM$Ag|zj7e+5vrCOyuRd2AcaFE==b&<+d z6LZ*+a#=rfT>In&{ocgd<4MNm^8<;iurJ-bQVp=KpTv9o;KHoJ#wz~C5f}jO!au?x zcbzMofIotcht}$Y6l_h}m410S#dzfpJo#lm)|z@~wZSzn|DB=jPi~XsoObOSs=*6B z_ihn{woE7(>{J5}ar_F5EGD(pe(QBV&9tzM3AF2MVOI5aFc@OZG6&FlX31HqK6;fW z*zKXRx9b9DdkDzJHssth7C2>s$AP8%%0Y`VN&z-&@BW=2nid3&;s7A813$x3K7_0Q z*)c*i$g^4j6T{-nFD73EKjh>Wz5=H)0!JeR57?SU@t@ZK{qAuQ-dzY!xD_z1Alv)7 zDcovfUYODsK5sL?H*85sOi2O;Rq$U!(}B?fz3bs>AUOKZFN=H|4daLdi`qKl;`a)| zI%&)`gKR^NPxgZ>PS+Z^)C2y=y$bkMJ`k>#!#M3c2kbm{fs9-k$FgHkJaRcFtSHl% z75shN=YISK0kT5v1uVPI;G29%mMw(m4D4C`d;GUiAjAbz|IUM{tLN?k5wQNBVv+ffXK{S>rt$rH6+is7WonIHeR{{ zB>x;3tmMV>ZRKqG{bOSYbWiBF zaY&XDav7G`)kVDoqnn~O^R#w>0tJ&l;Y0)k>i+_Q{Nyiu z{io?JJV{NEQ^{O{FNgH}$3d8y~h%p`8CQvbOvsIg)Aw{hS|x&zmUgk)v)K0=P= z!q%>Yfq)@85$Ms3I*F4F$VdNi7((#x@ZZ!e3co0WWV;VBmfv~9#;5)qngo39j0Tc` z*8j)efTJ_-dstH*t{z;W3nDGA?2tK+Zku`+vCMzU(EG8}zAp`yg! zgVTgrt2sRM>y+z?#e5pI9-S>eOEyG5zB*zvVmqwd_gR$((aLeai2LLKV9W-Ls-d|9 z$L{5cP8|{BR6Vq4*(5GKx%QCoInhr>Cb5y|W4Xc`m~h9wr}0ZJM8h&R+{CIT;Nf5I z3=jrCplEOO+8R{R$~$a}?$J2-!aW-RW40)zGv><*^*m<`T(c&1vwDw|6#o9m8YJSR zd@+_nC;G9bb6Mj4GM%{rFO39z-hCyq$9Fvccwt{A)>X$eV6?|7VC5Z5Wj}9+UFi`6 zP^;Qy`})nhW8#U;?Dt;dR#zKjKNUp?y9b;e(GmFAOEkroU%U9Tm;BMevk{@ty$Wez z@5NMBLFdOAMDE1f6}@$R--^`es1>qRZ!99me_z0gX5<{!h}!X5{#kCI~yMS;`W3@SB6u@LC?jhYn_&yr|80FZ@fAcb*dpo$pjDT4gNSJz3^JRrjCS)!e+o)(o3Rz@U4wC|^vHYzcSv>Q`n$w~7Xb~2ajbCfWa#u+FZ*yqYi5qMvCt33~2+yNA z$F)EV`7+vM^$VYMcIDP&&AY0y#Nr^K@V`$g_}?l2E6e|T4w1_$Fg!p0(ywH1*~)y8 zVNsjC>AG0AyICmd?XAlkY>eUC8&*@p9m9ze3%&S2`u*2)*L~5ZMuv{`#|~tulT}6F zMhMEKq1PuTw%ThQ+?G}!L%Ovi8p5ys?MVZH?Q#}6pTX8SvrjBSfVGyK@G?3Y`{?Mf zB!U1+PoK@6)4W*5EwmwSdU2j8cKx<4HwawR^FAX5VD#rf->d;%(lHdWF+)^YYL(am z8tE~Pu$g)Mve_~Us4OT)G&oN}H%0xAQn>D+zC29TWSqy^*bXh+BXCoF2N>-xl8rZ> zq5j9&{%$K`zgZOg?2W!etl;G6quBpi|7KPbrwemswy zL0y@|AM)Fl_l3!I&AoAiIayikLIoc-G-CYG2&EPmfWS%R-t#$>Q`x{<>*+ljzqqQVQAanA~YBTbvHyW^Vu5;rMJXcPmGDv=;D)aD!RSo5NhheTJgCN}} zfXUY3-&S{h)hby>vvqYM6^=) z`LTx>EP?DXe+lmuKIu=jm7AfH*~V`qY&GHpcJo0$vB$>NCbMB%h{FOo#ah>YWQ$b5 zjKh3Cu5B%Jd2+A*#ZUiFhy?Z$RJ?xq(ZK@XysK^$^LA(Ja!<2fY4W4JTWcfM$ClU$ zfdFJI$e4T9ob7OcCvcWm*9$s`jv>!0u}`ku_y?qW7U8Si7KBIE0UOpAvi_~fZKX7E zLueYslyCXMH>YH_%Zgoolve|?HAEj+)QWST6X_80^m3waWOu@|gZMZYmbS@xzM*gs z>X~&%&50Np=YmSx`sAsAE8jVL9Zm1n708Ko70sWi_uljHSlaUSZS6y7JAr4bW0PVt zQy)Ixj7;a-oo13~=gnzt)9ezwpmyE!+nmp~ykk;HTk|wkR9b>J?x>0#p1!;FAD>U8 zZSwHP`EJ)%`G24$8VQ`vj<6HZxvctvZ_r{V>Ci_|I060AvKt4omz( zWa7zc9eV5wVZKSO8+8g{`ZE%(asGF=!_quM4t=&-77_nge2M@%2%SJ8(_^ict4QrP zSRNj=>|hjZavkBZHgXI7?o0hXxr?mUPr8s3t%t6MFet>K7azMK;Au0+sc;W;3N9jH zL?Sa$i@n}2HRDk;nzk{h>fSiNHL^^wQ9^A$m87XU>-$7konwy$j0Ie>A1?Ezs?FRc zcb(*%mt~P%MQuy@9R9$2InKL)6Aur`z zUVxF!TQ=X=!d7~#(SuEYf)V&7CS3p~2*Cuzw*gslE@A?fRdx9AwLv$ug-dajPi`>e z#-XsBL*Dmjtw>|fNNLp#qd=b306Ey!YIdP#@OLOa+|(gpsFRA!N=hx#`kB;o&4-vl9-l zZ>@vwuFFR|q$9j6T$ZQwD#H&z=Si3#X*2F?-r=wSmc-;EKjR%NxGe+S$W?|%H)-+p za14t+GOEWb9{z|ZXhr;3QFSFmIv$JyGE=Y1RWoZHXdXxCWSua= z{G_C&kPpRN&3o!5ePX%aW4Sihl|0-mNWM6RZh(SCO$5B8uQaF;qU0mQ!#r8OW3q@& zvt}!;qZv(VkS0RZs2bz*yn$rbA`kK<(=`BC!*7Ovc)1b(Fo%P<2n|y`Gl+LKHgD`gLP2Y4QSH#Gnd*r0Z*OnZEMt3}-c?JM@8u#F92Orc9Ao%PVL-!Jb)zD|WwnfaVy?}PO~ z(>Iw`>t&_^?r@Xgu$`<7hAz{Pi-OjPaZWQ_5(FMjQpiFsPkn?tL_HeZeQhwpY&dfLXes#(ti#@_u6;{{fJlP9N& zpJHUL1{msKr8SF^H>{dM_BBk%PQ5>QW7Y!GZVOm*%%`+$+XbDaMXOzNJ-yVACEY}c z*6j>C)}sMoI0)_t(zWcyEj_fcnkNo=_SZtqPt zKm5|tI|U@cP0EsobF{2$EJXx&sGyv;>*R^6i1}6}L=b7_LNg)B=I6bl-i9ls-zMwX2!4_0$J5?99M~5Mn$N!xBB4v*Vs$N?S^WFn*PKT+27ee zPfm$W#tAy8Z-)AbsS?_J@ym7P0AvWS>zUAXKAb)GbftXM%YZYViMzqvQcva|PsrN7 zO3Th%RsB6DiziQ7KpI9YsM&O#IfU^Kk6_c@5seB6_I^|P%UTU88#*eJTS*DL?2&Co zzRQ_Q94w}6xg6UwlvDZ}JYw^8%|b;6@cio1a@OH@?*!aqIiW9vtIeP_1$<7-AY*ZC zI1yfLlv)}WEo%>l*KN+15erI5_Nt~eiEk6MOcGQ+tWDz473Y&OIcW7Se6lnT4NT!` z!0h~I5lUS?RNZ5G72ox--4gOo!KhLu79vQiq(x$G(c~{}BrDQVUA3IzT+tfI$Q*lg z{%B&dQ-XqK&=Pf_`{oSnB(Sq>n?uI3?p%o~b=9BNmkJEJhJtIMEAM>zQQiEUJ!Hsn z3NzEr>1>#|N((G<;u&Ew(YOmAL3g8U$wq#XsJ#G0FqR1PBFS|p^jJZz$ZQ2(;>V0O-{3K6X70rz;!EeT!Hy+^89#h#d zWF~jG76yCII}!0gbwjs8!j~Z7+oiV3G>hTM&LBzcABXHUl}g=KB6K@faErBSO#v;hY0kFtocn^c@`FL*&2>p3w3kB+B%Rp3va{k`YtDW zs?B7p&1Cvh=h46~(55~w6lbRomTjadcaf@A=g5}j?7(-XZ^IuT??}EEl^mVXibPk4 z3i-ksMgvP}#pHlGuB`-<)W49MUF#7FUn#F=ZXvX3zNcje(Xzcs=y#c4eHk__sk|x# zaf>)@M^Sg65ewaHh89LY>en;(azBKOT@MY-+!V+jzHZV13g*G1gyL#9GJEZI;1JxZ zJT+tuw}9*#;{U`CJHebR4hDa%4%uzqy6U_-2O~0-ipem3?i;ociZ#OIBD^X>rDXX( z>8@{~-MhE2=sT+x>U~f5EJto8RI6$Mjace0p?HKlkgoyG9UM5Kx5IVWLT(Dfynf0L zax-O7i(? zfYPzP~vzr;+Hf~v+HfT4j*tIx0Fpgoh_VjB)M|Z$gKY2 z0Tu1#8*>CQPTya(TH(FjEy+}=A|-1_^2_<7rin{Z#$=lsfIJ?mtZhEUx&a#BDl3Bo zTaeKMTC#wv@Z2o?CqMOYm&Ph&e1sHDTpl%kDJuL3seva})xQ>gi*`u?KS*h`(R}!D ztqUbrjl(nLiuT!xJjh$(S8b5tXlLDc7bAn)VcpS&lLh>OaXv@fZx}5tQc?QAtTZ7(_+3hoAHEcNSBbEl zM?#hcL{CBm4fE_M9AlPtQ9RVL{Ba%W1uM;>8y9|LVGMar_4b;*cN^nVyZL>6vYioh#Iy+>%UQZwY|l%I~}V^5zq1*Hm~izy$zz? zIy0!HmV77K&FTZOsdevt^rOrbjpM=v)1S?_bUrmzSwy6Iaea*e#(xZHdcq6 zYr)mH*_KyXqJ3U_^)qo(HLr3^7jy{LgP6o^+?NrDSjh3#7UUzzUGxG6Y}BfUvq1E= zT(wcSjT<@TVFJeEhHp?wBXjWf(K=xmLgB)?0Y6z>(d8z5>mvm}pN<)Y`Wuxztj;^8 z%RSrFOHoQ#^YtNWLjD+}m$l8Wi51zEvYJj~L zwA&@<<%>E`9E#haiW>J)A6z4OsKX;L32ek1vzOpdUW?rry3XdCnUWj^pnU!W6&i8S z2so&Jn^$?SEUZ7ZzYOEST)LTio&7U!O!JO{W^GqEzpZ*k1`wuzKdBQ8TDEj~+Zhh|4Qh6QLQd|Ui2Tl(V zxv!9)d0^J6)+gZ)LIjQJT3pR=oyknAc6%foN$ursj%s;<3x6PShO#7B0#FVeRlB?a z@BX)wG*x5VU;epJf zg{-vJh!eyb*>U4tRMRq3+m9G`Qd*kY0}}^B7dPSfR|wL&NRQNG-L*5rXt4~nLVi&_ zVzxkx8A=-0cwU~P?$4HSg!9*5V8feljFq~J#n?qiz78rSolfN&}v*a5(krD zRs1^Op(Jte%7C1|RJUJjYplwJ8vkH&he}PgaHg-c>Uhb4 zV{iJ}=>FcUf&*^-IXYGOYawU-T*Qq9ZS?zS`n|~^QfZc^b}4t(4it`>&*?2|D2Z7V zZr3r1>#NPQjlFs&$ucfwC=n!&8)_JdgR2Mv>r$8@6Sh}Oq>%*wq=p^DO+ViCbho%S zV#+;ArRVi;*EFW~T9@-})WGtcR9%hdo)up)x@ns1+gA7mlw9gdbWP>oPQ zd=qjXLTe9QFuLa2L~Pc62Zqa|GTAKQt|cSlfz{x$N3TN(dtJ-Rm)+Gmlg}8wxr9}` z{tC}m7rjec^0_enKbyzoXO58a@!PbfJ;rh=1CatbyS5F4TgV?xA0D(FuKnoW@XL{!$Ja~BYBz42J{{*R zD7c{cP}i8IkQDn;>`j$#L9TFZCS0k%Xlcnu&Emx()!eR>sc-ibLDn6)V<>MWOKlHU z$3U@)cQ^+buk`|bA$MzT_qdktMFcK?e91}=QpCvLtcjvn)wX%y0b5aIhI?5 zcX|JVrGi4N?){GRHvqI@>Yv_BkcJEQT4Bn#)ed z=xFr1CznOyYh};d$P-3AM`<|jFK90U3e#dNtRG%lnP7aE+DC<## z3(YdMr0&iwm!tMy6FvO?a`B?)-rL%pnBPn4s!Po)vkRPAv@W69{^k?oZ{L%98?Rz4 z74sbx7+50vRIiTy(8(qSw_kZDo#{C_oY!$mWq-7=`5vVpXVz*#6r7< zD;#KUC&kkV9|RveK&t7tmFs?D97`M3q|+G{59a4j`D+@{EPEC&F_-NcogLHm1^;%@ zW5)iIhK}vQ;04c_D52VlC##s`TYZuOnRJ!tQ;j)qhR9v5~itBYM&}U89B= zzcIf3f|>rlXgTBEvC$=QZ#rwVr7WMu9m=Fu$l^rvHUPAEzK#ZFMA5qtpfH9!tb18} ze%x4S_?jDiL@PgkkV;p2P5}Hx>`ms-?##KIX_FUpoYqUL=MYjJrx-3#3p-r7g1iT6XS{|Lpce zzUa6Y&e~9>UgN6zE6Y#A6{kNCb{mUC=j#W}YeDu#$Hv!UWO}%rMU=XfJ~wkK z*Q&nLLiQ^C=o3|Rv}rnGA`b8-#8|p1ML54oJ2_8A%||jMN$@gyR5T_G#;AxlZrTk6 zn8Y;)#|4GB_@;>MjIQGK#~!bPin}%&4Pp*nmLe)$#tH6qUAG}cDYg(jd6c4|B6s*F zOoM8!+A6)Q;JWv-k4{wQl(*093{sEs$78x)8tmVZoUi*t&EMjR;r9LpBGZhS;&gMW zUjutUtR+)Q2jk+P;%1s_x^|*Jr8oX^`;%)v?dh+jO*4b-r&Hh2X-iSr=ZHC^MX1Kn z36-Ukrv)a((hEEx1|IX#P_~QQT6FdJ63r^&NFYm z&$g};jj-9=Kgf|~MZYn0^FXt9wxp57<7-Do9@e4&eCcy6rpT!zn>o>Y;`Ys_b30Z1 zHP6}R-`IU5UY09; z@O6QiN3P20A}uN0|0?di!kSLswNd8pC<7K0M5Hveti{d# z_dRm$?E|}k$x3p%Lx=T6pJ9j45zxJ^(6)QvU+NrwwI1`7Rk#>S5gkzJU2@d&d$0MF zhMKBIiH;eor9aW*A4Ei(2D+jR!~=yR-$Dx% zB%7ta=)V7o4tZz?^k>*y%?CT@2VAmoNG#55ZqKtuUotMQj+yb6NHjUdCJn?5CTRtq zKDT^5?DVV;n16I~d*9gB_LGr)eRoKtTH)iDYomS&^?Ypo+GX<>;NySs|0ZAKXZ|vvm;BRmRBC`Ec_G@Zs!x~qkOBd zq_4=~c~7iTC*m_8g%}*Cmg*4Pcg83cD zj_rN3?e2dol=yBu9bZT9o9Fl@niv4TJi1BT>&-iBMD zJ{wexFTMTvi_Aq#rV`0WhN>YvO6>|8h%4*@WFu}gG%6gPlsXWQHOxqFea>QR{Sa`p|_4v!9wRh5R#mmFPaeRAYY+cDh(HqAk~z0+u0 zi6`9y5D%dBCo6VI(&O_7wd;n70plRFv?wL_RiN@jncjHumGoAXt+V<#P(c~5c!kwNkzP`_7Rb6k zr)ZC$qN_amgzR*Xo=w3$+rp?Cv%$i(8@b>i!rkHsyGY~n@xJKwr_W*3tN=T!JjV&` zOMlQOv0JfnoSwB$Vr~=tZIfFqU>cyIPQ(*~clH&-nr04{Ae#Pqa&vdlzy(}8iFPe^ z3Azy)wfuXnBd+-s*;*xk4>f*h4>c-xyX~cUCasLB{2FXFuNMD_{qkE;mki_dbk?i* z}F;%uisr@v`kGV~6(F`<1_$wpV_ ziZ{E+42LhWMmhuwPN|{W|K51MWM45D-eUTjy@bC11Fpq&e3q=G;nCzi5TsO1_(#Pb zE@diB_o~h8Ph8WCy5rJ4l~IOz7$@GiZXr39SA7lDJ0MmMDzjX3N-TB0O;@h%H)zQC zx{SLTVsT0CQY@ehZx?YWEq(zjuJ8?m8LXOF&1ECppK$hVTc4gwxzW_Tuj;1x*Zsz!TgL7_;?<7Iz;@36AW^0N}VNCcIx*5vI3MWVIshBxOz%ssX5 zqNJ=nR&`$?sgx&tV-~br^vy@E7|pQ6UV$S;rR-tAxVrn9qHN%?Q2~@x#5dhU^Ac+;I&!) z$szw04RsEDqi+geeOJjLk2)o`$zNDjoNAk+!o2q{kef>`0^R?Aa05`*G@gzWko-zugshRfrNaQ&-PS-~<=7OD*b)NjOecFD6NKI2! z2Xs3kE{q&3&-HX}htyRBCy_-8@3zjcy|iV;5#Ss^U&wE^VJyp_xu#ifaHh3KNV(b7 z@xfx!rO6^St*>6d%L)-=1IRKr(QPCu=~6bX;7C0SHR-$CV3P=$32oo#xJ5qlbmMIATZt zQFQ5qowR26qTVdrq~)Nc)+OpH?UMg;`U8~xVJjga`xAg5g+aO&O7L4x>9XSFtyeRP z7yW8ou_`(@%CsG$y={B6s1`O(r5h3olCPgS<*k%JF`jk5=tfSc-tV-lUpfCLJAtZk z7@}mwPJ_`k1cE4JzHCWP+;XZ|@2R4l3FGE$jQFf{&zwc|%1q&NzxvgO9-){T%4ajQ zyKUUu&1{(uRfFOl`S<7zt=Gk62P!oVxRGx_Y}h$9J+ES`iM3LeHAPJ**8+pdRbsUV zTG=tN5^QuxQMj&R$fHctr6qM)F-SD|bfk$Fc~?&c3U&j{xJz^MeQ zV5|_ka0+S_qOhDYLpPi9lU~es5UOndgd?As+@L*5dsW^yF8UXA;LbIV&gZw@Jl!$+ zeBIggoH_&^Bp$Gxst{wKo3rVU0_&SuyJKLWK%2&|T`C!6kxrc#@+Uev)r+(o8K_*Jjr=7m@x1I(Ycvun^*pOUbm?VQjz)(+-If>Yjl>nn!cba zuP`HR?CR|<-<3F*2-dA_2LhLT)?9?;`%DKx!U@7PL`frl*C1CDr+G-#|`#H zd<^)vP9zLsy(Zh+ z`$_R3Bm~g0^16(pdw$&*!q|IcDI(X2NcBpv00J#|@fIGnL>}4dYQNWvo8gHtBKbmf z9j5+aPlR~nu+!Pbcx4Eg!Tl*LhpohKjtr%8Qu= zBuj{NbQL07UfNf3yq}Vm@|}fmH5LgAC|y{9QRsNRFkUE4WM2|M+L7TN8#QD*k>R(< zJ?Q`fgG(?Uf%mTi1%LkhqlMed*+BhwbKkS=IXHSVXRD=4J@zrsc{GS%^5u>h=Xl8; z5mpDLgv@RS*KbWQHUuYp|LAn19u=Wta+)+m4_CGl;TO2#?6;>%UO}D#mE!q}AEAN~ z^svK_95ZHNXKrs~2#u`8E~sQ@T$6@o!9wJUYJ)Gyknin^T5RtfG8D2>{Q#!jrmZZ^ znwHp<8QD)=ku22_uGAGtMJ3l20kUfl&duT=BtaZhe@Du)G#_^TXi#`BDU;Hj_KsE> z00DPX5irb)U7>QrfD2H&8?pz_BxN1V;7drzruD} zu>Xx~LEJ*5$qRORGD*GoL(>{srs6?G$m1KTTw7YauNb?eznG6XI#v+u7Zr6>WpVRv z6Oz&w;p(za0K1(o_J@_n$Zx3n98m9}`7@3P|mqMhV*yYaOO}i+>MCR=Y9^_O-eeRXlJT zzNi`Bp!q9IPp(}%3zG)*WJV6iGm{{%2SIPyG0e&z)C1;`;wev>gApf!8X-@}PPv~U z!B#i9+El$?SxAs{S#q@;%BVY7)kkCq{_+lM#; zKh4I|hPXHR$MSl0eUEYNXl15eB&bIHY8Z766yn3MO8M()V4itvHseO|gka;3%Um}% zjiCpbr+g*NrJnRJL5iChw;s%zSF*15*dA3?fV#+9o)>X689>zdF@Nq;O$sCD1Univ)6SmsU)F7z6c8=9Za1M9`j=k2?*K_fkI(WWjU zHwbEtg7{K*pFk}t%&GC-ww6=t+!OS<^8;}ynJCu7YI;#qi3{mi9hS>N-#U8ir43HEh+GUAvmuRd`N3&}nA4HqtLfNR8e4?)P)GPNC zUM=){GH~Z}5-3P+BFiiEl-69Ks8_nSjTBlPdi{-9pS5#8sOel=Y^j2jg{5|YM}e2| zU+59YShJjO%$H{IaSYXdW`sIjRAr{#Cf}doeQ4Z_Ef%PevXyh>X7fOE4fh<|`V~}@ z^?SQ~hLI*qOT~OkV^WCnF!Jfp+ zvxZWRXq&tOo990Ew(5QDk~R8;2c(Ky<6gE?^mtl#R!zAQL5KGENQK z_iU?7Y`|B}$ci?qe$>>WDW{(k){i{60zbywDo%bH_?o=h{<;ZueNtOQVkYS9tp=F? zlBXx{!P?rIWi({`u+d%N`ZpOO_NV(?SoxFp%o$p5jgHA5PR_45at-ZRhqkF>sn_KC zOnc>FFM=E&j8@y!OTl+@wOK!E>JK|-D`pu)k<)8X7j<5(MS9X6mXe_qBd9$PZ2~wZ zQLgIbd0(j@H@Gl6mMZc)1Z&0;RYwLGlM-m2y2xI+b1l=`h9x6mqF*d=5^hAWf%ek# z2R)j{UijN4?`1A)_b!RPUmLhtoFiCxD;zQ`-gZ_5oYSNY zOKU#j>^3HeCZDp&zW|Z%k=E$*O~MN^O>m0UG!229T(7q*ODqkU5hJHcOh!^0@CqSf z7KP)W9G5PfEdwPh@9UP)*y`HtZ~QvoxpVJwtgBy2x#7{rn2eF|o7c^&(|gP{F&Ws5 z4?gFzs!Z!ujm0G`;bWTlbP-qgO6jX86R71mXV$Zzt+}8maW7-($fa5vz0}Dvk8j~n zWY3e>a1|6hsm>LlVzcCqf8;8}PalilC^c7>izArqlXr?kOTt|=o3fKnoVNT`-`_J z1GNY#NKUt$1e2160FS~&gvM$1DL|6ccO2@~Nb0l#i)0yzY2>72$n>W;lc>zXv@q)U zVfIq*z{7T(q;tV?V*Jr5e)pD^W+zrWEw;-j+DlPFB%M}g-fIN%Hlqa;n~SBTOgA8B zYmS0ruGl3I^C2|Pj`GfE_-vVmu2Ff)wdXt zxR_uo^N;@<=e3CnE0KhXe;l(sz9p+b*1p%fMy-Idh4U{zhi-VTk?Jq@+ZLAxq|`bc z!>ah=-%VAaC9j=CJxR~!m05XUj7t5K9nw)L@ZGmik{6{sZgR9fSh}TQdi6f11%s&8 zZVEW@j(~}b?2o)6 zqh&+S@&^SSkvbdS+n)coPv7E|oh@som|yRzdVDddl`N6KX~m-S>)lz5)3 z8D5;HPtk#0CH@~>68XSMe=%^CaA=&t^27>k4H91rTqb-hKWw=8Wto>uMQruk`;PH8 zD|U^}bB_e_U}%7~)s_QL#F<70@4RL7Jo-<5*aJ)BWEsAl(@yeFfkq2Dl6QIDt*jLB z9;5YB;LSq-7~xR!%%3n9VY*Gj@k;=T2w28Pm>^@7NVtFOQ9=tLwGA1@U^V)#qkrm%RpcWEYP}qWZ=-@&NikXf$&Q%N1?Zcm8TBUaCv&{SODs349@`tz8 z!g_a1bl5R;APFVdwgk`m^hv2|TB@=34XcQ4yPf2ND+6z92p z%gCBTeQGzYp$)Gj?&AM|GsSFrqi6I`Lfq1|-s41B*u90CvBZV=$v4j6ULhy43reHf z_VB1#PbAv0ToG*@ZzEc5sjWPn^tUags>oE+ZO1VY_kvevB6W|`CowP26iM6YGcjRD z(mHqn7EELUS1^WV4OL^-(gb@65?q@W=IfN|aq8wFsKw;==4mr}*bzvut%S#)_)kD9 z(ra>%=+Fh8ea82=o6nq4Nw4!GjoylTG#i;57Bc#);fqdS5$)|CROU81xrcryb-=OT z0c~EK805GFNgZvowcogYLFMAL(v9s*V{2_tfrPHTP-=xEK|x>7sczXr+fVYMnT>9= z*he%~ex~SM`5}ZbJAR#Vl7A7R0mO|{A*R%`tn4&27_UZxMhf<2$ z4tB#&uws(ewQ4KdSeBSrogCV@+?>biTG2yJz;Yt34lAfCcM3Il_vUqVVS>(iF~s3ev$dRd$Xan3J#q{rLM922ponCo|9M zyWV`}->}+CcFqJ7s}I}mPUL0T)GK^IX-KiRS-FGsOAz^erRyZEsx4#TO-B}XXWiAo zmZp)(SMRX1q7`LnZNBlIS1r~^RlnbKDdcMK+d~yl2NHq1te~6g!FTE3#TtuE>(hipN4?Ls3oB+C0}U6?P`zg@E9ropP0d1I-aquh#rl zo%kc==hdn`5}ED~D12Rkq@(7?swTdv5)tq&As^#Tz-G^E&w~b!WPK zZ5wrVwcYvRSkuLLeLdG&P)W63Td##mqo0 zH;Zo==hsGp%CGczJ;kU*Us1`mwk1#HvE_{|^g<+rWry-0hC6NS80x-A5=l-`UJ~jH zh|va_3f-MNrFMvDT}ZTN-_0e@_ea-&k;j{Wf*RKaS3ER|d~B5l{S5`d{gqAqIt2y8 zBEa#HSl7duSKFN%1A3Ti`A;}I0@DpRQGB=x#sWzeB%S^{cL!!RFNM`5*|D* z<>-)K+YSUaS9;oWP8j5)6CX4M$iTeghe3LHEc>UfXKrLtvf9lCDC9}z%J^x(bn z-%Wv>!j@P{9hl~=FQ1h2~VYMQti_o*}=BRs5K-5P0G z8bqC>Ui6(AvXG8LY`8Fg3j940opIX0G_}zOII;jGO)B?FL^VWofaWXb!S$S|k7C`S zn=AFTH6ew(R?l688rK{1GgyigS0=3M33v$Y)w`&^#l~GiTZ#Q)bQ^6iRYp@Dt`reb z%1#cpyqp-JJL;{fRgGR(A=EC};H598XGz5;4PFHbB5!M>-K6w>I411WiXD`!L2h_8 zfa7!FBgfId=Pt%so>tnyf83&+_F})Ljnm*LloieXi@DFoD2Y?TTJH&gS*BKRS?K!a zw~(|Atu!Oo!s5l2#Ob>h)9c&MliPTOI9&K^XCND}zg&LKR>t5&e|R)hTX9t*P&T_v zI5yD{l6I}6UXqk0f45$Fst6ag_w`jpiIdBs!+zjFCxG-3@7aF~Kowt_3Q`B#T_d#@ zbymcTEcgijsvMnX1$waFu`~uQomOPCbND9+sd6Wu96e7%Bx^K3RoV$m!*Z4R7m;?X zlP4|>RkCxs-O5#Ur13d0*x?}6Az|I|J8k7+LP&pBWB*ckg+st88TsU>m4FrJdala% zUS7z@dQ^eIXLmXZRIfj_o#$Y6NdAOPwhlTBoO+|E*_lCHObQB(S*)UNcqwt;Rx%z8 zUyaE%E^!Aa3}h%C5QbGt1FN8K{tDX^#g=hrXJNQnOyuKGz9QF*x!hkSMtk@i3t6Tj zlRDkNFIPCn&@b-&!@V+PWD!=!fS*I!=Gp~1HB5;vd0*8)i1+QeSmablq~*VHKP`J2 z8Wk=nt8jf#K52CI?&Wfg-;xG;kJ{h?Gey<(N$d!v`XUxl*!EVY-yV&v8uW2 zvS3gn%CfX}8k3%V)*v4|btCf*$fU>7sSM;BTRm5`J&kDIeXJEG6Ev?&v>zax7@=)u zf>}t139i$tl@b$CFEYf9cIq)0D3~XIgSy8HrUn%64z^6k7!2l75C6_AdcrjoUI}~1 zI9TLCmu*Zofvh-xV3~T3O+Q$rUG%FjsJiu5)hPV)sW;8yx7Pyvgj`E6o34NMIPNS< zOi}}KS7sgKqQ8nkYkx*WwKUkT-s;3bP=-rojgrc+I!Mb+LC-38ZSdj(0!Un@Gnf9z9+KJAW<#s_Uz4LiX#E}wj!Ce&n5?^hkw+aQkYC3omI6oxtZpG1yw5P zysCCwgg~Y%Y!681RSP#`(radK1z26~tvjFO?p6S%zKOzo=xW?4D4h+mbm|){)CphC zyk`lrag0`YNZ-;;gB!9nsGmr9CGX%FWY#AB3^ayq#=1L5EdW%r1ZiD{JskU#EfIM_ zY5+T0A{pvd?*r!eCV6^)yl-S&#l~8)VF-ooqNFG>>SuMXejSC(le=U((zxfZY4EHl z^XB}Sp-6*RXhiFg>z601s4PBd&7p@L?2NG&W;kR?KWmqY_JQ2!?bL$Ty1t5EqbuKt zzH$szQ$ufnq<(U_=j=?Ej^5*?V_Z=VmCNEnE=fW@S_{=yU&bPsfK(IW2@tg%gkF<& zeN#p8Sp6v=sK~aaMgbP|KKC?D{y6YvX`%*+9GuC*BdOJ$!K?$mc|CaXkDcdVukFUw z@1}3t+WMaI@pz@vZWCO1YS-5^V?10h^U&Ru>BuD6>{u5`63XhZ=ZVx0Z%cJ_3(KYT z)_ZDu{gnM8v;$pe3`@!4r_g*@=gC$Tx-uyFd~Qh9t$AtTh$Q_nNT#BqIZLZrMFsnoylBLS!7%i+cVuhZ&Opqc`Bv2Xx1JsY>Z zYz%*A7W@orm&XviuJXNm0e~)*zW|`7TelfwalFf(tK~-}$b6ZwORGftc!K3SlirPwJY&uRCm`2e z&oC0`vBm<}%kSUhNx(%A*#(lou0gC%Bx&^tN z*O->a%bBl-e6+5rPq;sPe*-rh6Qazmlh>fqg|!}xD=X`c30 z-dM@#VLjIymS4ly{yvjgt9&W1SHSWQ*J{U^ac{Sf!miK`&-qV`sYja)`%$L$@K%#& zU-QO}&3-~K05B}KBW0V1=;d|qXoN*DNTu7HTR5VTv<8l=Gw(*MEZ>D^J%auF0R(!J zeY5$<#aRG5&6odX3KZ-YvcvauO`GC4-OT#{4~Kn*-0~_r92Crv=yrp47y$=)R-+K# zgWj5+GvLhbm!IGt`d@Hy1%BYhDEu=F@^3^RbH5~4hHCjA5j)mKKzn64%=vNw31o2K&rfdn_( zkq^UGt&$IXT$UV&On(28@jV`1{19H!G1JXMcp|)afhP9IaUyvih4Wh2x^O#GgD}D( zb8z#(mh<3$hwa2Aga4%*LE7HHa2RB5x?*BUb*dgFz;PEKTeG&800I4yVa~as74G#DV zCv-DdaGMtbR3m@!5}4YDeaoH$!1qs|w&H&ZXbgs}Q6Rvj+ul0^+(^=7)W+kL$jku! z-rEk*d&UGiLsPpsfK!T4Pk6nb069pVZRIw>1)Ee z3B}ie*dkalb|rK9=u->Q*BzeLI!`yIIpNppcYFhk!ZH$r+qDX9>t6Z${j4vdh>qJ1 z>--P2|5G0|xtT+O$upAP{QZ7W%Zayz%2Qdx$q{I=78@SUcRVB>3=e4)t>(*Q(5 zE$~kIMLnI)vvER3eJ^-C}HqBH=Ej7+IoB)D&Z=b~XBv7Aa~A7dg)F z-`E&On2pW%Dq*F@9#r5hI_5SRZcEQJ<;~GY@$ITD5K9V`Ry2)l5 z9}}UzKO-xk-1Uvi+1lWCtd(ut#{UVNqMc>Ji{&uCJW}Se{*B(44hjiJ3GpOP@7VPT zd=arn3_e~7M;tQ~SoqCiGiI5D30tH%Zw4!T7XS`0&X^}?7=NU^Gf;mxS*B~A--X`~ znxDrthMqa}J}#bbBx0Vxe(;B&fUrh9!KBL&q0Yg?yKDUO%e}O1uIQ6L-`=L{k2znS z`?kBiu}gN{+3w1V5YRCvQdkRdvhuzE{HZ745I?uKy-MDTnKGOif&%Y&|Gv=4n=u4z z7=8wat6wik{jE{1{HH+9fsYG(*(blf`&Y*N<~SBa8NpZ2PuqU|4R8UW@tk81cchFV zW03_%bmO~t@vxn#t}Y-*{`2TRTf1C{T#6v`>+xd(Cw+F|u(*eO?`_adqwg8Oe8xvo zK1Sjih8P2prB#0aw!Z@ZKW*>h{qq1VRq~_y@eP@)7w-wM&2JNx70YYkJn#Dh-bgDHdn8-J4umEXXZVn|MgJ zomsW|)@p59OP8@Pn8pNtY8H%-X(=Loo8-O1l0{zs=i9Bo3lRVX794L|0>~Pr&=<=J zI!SJBrozzRlc#oAoy5`eD@A^e%KvC~ln=kio9X1vTrGEbJ|HywV1)H1(f{?LQgYSH z-06*$T-PapSX+_#IWaHfV_mLC%bfltg(>9l{mbT&C*4j>SQ&{8H1=w#H(F?cYFIv1GW3YvY_4j=fAGi(d*CzRZ zvHkEtyNz9A=*Ew={g|1JM@auI`68T2N+#d^-Fv;eWdM)X0qm%nd0>G79`YcF3$(Ya@((8Ve#$fxm0;~7_M90I%eGQ=;{M=5*Ev3(0f{PpD6?D_W@Tskvtp_d3KNmyzJ2^p_^6>LU(mPM z!!(AgONJnS`E!3>2H6zrqltunmRf_~xadQPB#lu_usGQ3&j4wU*;7)wnCs6U^zF*G!hs3(i7am|t_IKWmJAHp_v$J20f4H{|>)3es&i+lD$Osu} zK7deNxGk#*ng&SKNlS$}b3iD(Gp_`MTE5Q+{yXCgz@SiC4#gq!h23zo+dpvQwHuV+ z7Jzo>Biv_}Bd56+c;F1R_#uvajG)y7-(-iiu>Qk&_3ulxAbdIlO?VBOliU}MUHD~e z86M^vy6@cpw6!OE-&y|1LXPN32_7Lf6BeTP(5PKRFASIV%ONdkeC~S70gTG-#^Z9# z;=c<1zufSY&ySX6f6FWHB^kdZXX~3Pzub>qh2wtB9Bb>{U|NY_E7jS_ug5cER9uu& z26hYA-3}WU?Diog=ReQ^>L21Xp5t9ySvl5BVKlcwd$C@<&TjWQFF_hP#=Vm1k#qC*F9ww0NJ`_VXiehu4buO6P|B6!77Q z@SoRTqWwVbNxA?#q`}h_d*2T@6CVJvKjHA*E;jJ;e~nB3E4=?dMNWR(2g;GZ^8hkz zcnW`j7Xzr|MWyen7V(AmNVkZVW<2QF!vDP tzc7&pSlLq+koza0K&_}UScP{ig$a;-lWGv0YsM?k6M#4n;{4Aq{{#3f_dfst literal 0 HcmV?d00001 diff --git a/docs/static/images/pcache-writeiopath.jpg b/docs/static/images/pcache-writeiopath.jpg new file mode 100644 index 0000000000000000000000000000000000000000..561b55181174ff5a97953571f02e946ff34b8140 GIT binary patch literal 22616 zcmeFZ2~-o=zCK!rii#K}K|qKTq9UYKhL#~wQ4ta22m(Ud0U08uZ9pP~gh2?1n4`3y za3rW8B0>}hC_^G7fmRzMQ;^US61r6;sZxp6CaJu_bGrNXf1P*lIq#nP)_rTexFi** z+Lb;0rtkZ9XW}l8k-*HqpM$pf<%Qk_ZEN~|)9<>)=>IQA z=fCu|=8G&WZCBc@TD``}*~Rtijc#6Bw{7>{;j?r9frI`5hXR9+ z9S=JZ9uXN86B~CXo_aPR?Ob{WGc)V_<=pSD6P@o?3c`pw&sQRw}Lk5kG|@UNez`-P6@-@1kV{9Did zkA9(ppfh9UOx>A!)BVz!5kK8=qnWdouA6PVd9U7)(;o4b z{jPJK*$OAQEi~P=FFpI`I+ptXsAs=-?2mp80AA=K{qMcBXe0l<7SOA;uh3=bu9X7w zby0&b(lr7I0MdhBhxDY$14M+mYAf*5V|EUEF)c>HVdB4n?@0)AOJTb(vV3wTb-9pk z1lHA*lLOd`5GVCA-|TW{mQ}?4k_>a9!nu9{shAW5}=G7NrzZOvxP zp`C(HBCFv6P|%_U79mz@7l}uhd_rGQ#|J;P(=j%l;2@7ZpBu8XukC(Iyxh4#a@;5O zAerpvelYKwOC$LlV73#fu5Y~g?klP_%>ue5=L;*X+IXH2PH;_40tecWnGidzNuL7~ zhx&v#mgSFZqo!>o)S_VA+>sYFPngUZouL?ee%15&B@fB{5%ZX+x?~HsDdUu^KSs_v zN4JA7VzYTO;42DjI(EJzf*aEU^Y9C4o8i+6PIeG0WvDZjU>GziwM%;g<^3g$+gr;p zr3UZmcJ}kv0y|!uxDOGSaTZgFA+EvQ97n)oOvnKDu;){^!0-kUp{9Pqx>$X>P(;x8 za4kW^`4TOlLlhE>f@sGdK^?3Vk6^hLFb}5D9>R0W;h@RAaP)ArYhV!}NBc)ca5$YT)rulrC7Z6oFKB@U&IXiW zy^Aa1IIwUv%ASYj3cZB#x0$^ff5HSleM26dX)e zb~(FG>W?OrR%YB@OF22o6U40zSo!q5WPEG%yKgQ%_FS1f4_Jjd@qhQ5_670;?@n)J z>+&Pn7x5-=B{bZ^z#wbk;8wma-Iium2Zuv?a%HZf!uAo&QfL7tvZ4_xy2gph|Bjb_ z=Vj#JoJTKaTGB7p_Zg8ni3=Lbt`o-Vhp@{cv;er1ZN?WgQc?%qG-cuo$Re1z2gVJv zvx9m_b1AWei-;5a2#i+-!4%axT(-wL=sQYw!fW;r2Lrc0lM~K)P%0zb@{^mQ6^UdO z-vSPjDQ5D_A%uP7q7s%o7d zY5TN5YA7uP<|abHdnNoR)?7G23!noa5SzTEJLR{az*a8UE?}F`zU!kAibp>ceFcmnZj2TeWC^{i z;9QXhJOC9A5rmlYd#Kq%X_0P}Of7KyIx$33cmrmDq(JzR;`C5Q%+PkeYDW~Z0@_|W zjJCFm-=p5E2-!cp_dy139=xo3D|ov=>d%Q$7-CMs-l{nfMKd*57!lZhvdCsR829F! z7Kk_jH%8xu6Nf&v*22p{vYf%>IZ`dfj|(_y$Q<=zq7@9&@L;t!IcYeF=x6hF??4dD zR>-X39}h={3(HGKim@J}1kmT zvFhndK}J^JuHINJpc^%4);)2m$xfki?^7+&D?vo02UjM?*_2^>nW5S@Uy+~f5h7=& z7tP8QVsLHO%w#Pxz44|1irjt%+X@{dg48aU`%p!>=s{#SGt<@Ha>~q?-bs|Oo74@v zsg9FVn=lP`{s)W_HFS)&6G8qb4E`UqNziuvey?9|DILz_D%~`>A^q27$_Uu--8

    +g^t$}EG${i*t?OxVt zf!4WNV9um^B>`Z_S80KBxv&brZ^nWvLm{zu37PFr$ES`v!tsj=JueEMejFVc$=V6x92I9 z1A$Kql6v0=wE*T>dAn^#(dAAc!Sxh){T&l*ZfG@T+hqOG zoB#VMwTTBaKJ4qEo6vAQ&g{h_Sd%GqzL@lRO}%tmDt&-h^Mbx~G$7Qn+l~i4&k8ST zH@iYEq1KK1#A#dxcAfXc>$}58t8*OkocCa~J^$hmhnEx3As#iNTJ7j@N+h8}ympdj z>lag2)hu_|sRht6w(Dv#xZVF`NpE-;eyMas`kY_+Q@`+YQso-~1|uqeV%_oja&___ zLrfDx8YJ$5?g-K#(ugJm9}=M1Ir)a2DTG3w(CE z1E9-1K-s1RZceH;prL7{dcPKUixuXwLoSeMAQPeb|z!sZH$wY&yHH4 zhJY-ihzLl|n*Yu1ZY<=R(7X5xkd?GqFOVf@vTHTWo}&(iuPHFhATBG7HxpLL#hD-Z z<60nvzX-8EM7E62jN?na6lbmu#1S%Ff?p3k@5Aq@+Jq=Gnn( z;nQ|z6=e)t{oT&=$H(cmUNOJhnfaSunn8JB`_kX+%!PmR@Zi?J+nM3E+dinB-Fxu6 zone>2!NYbF_K-2wgv`buv(*L-;6)a+LGejYxl+|esI6x;ZBTgbM;2m5Bn#SVzQh@e zdCRUYyYR6Cs|w!_hP*3}EE}Mf1*@FXf4LV!@_c;FR2L1Q=PJI;wb7Tk4x~-bwF5fW z#vF96GrgzhIxB2?u8Zr?x!%b~=Xxm?ood^BbgEOO=v1$nUP9;S(@Q9iG`)oMBc_+o z|H@`ZcyrZ6C|~HqTN}YJ^e~aLb*YVVJK;0Zm$`3BysO?0zAo3?04wAS5!v9_ZfJHr z_(^{8c4MtLOY*mwkKlDMN<+|@|D_3RNKU-+H%~b2b;SF+d8pUvo?`1pV&^+&V^F74 z(@VF6>2HWlFX%Q={yfX>B|Ok2>ukvr$18Cq)SaUnV+~y1?8@`QC%VrGtG+Z*;PMSd z+xss$os~6TA_p#>t*p>ITt#qa^Pr51RA7u6$%#OXM)ZaUt~^;WN7x1;az%6#=Q zESMhsE8_uQPI&eFDKORWTP^S_TSR(K)dDjJ;72czd2u>4}^Bc zuWLvT!Zm+wKhK}KNFcibpO5}gPrUcDLX2c^s1`8BcTC?`ZOaDB@$(qbntFJJW3hh=;54ynCF6+$6oB4tqD%`^ zvLHn={^~ynxWBT8ywz8514QKcu~z=)c5L+@JNi|8qmLg7 z7p;lHBxDKo+A&vP7$7$-|FO04G*QY88JLo0upq$A57ruvc(cHhby z2jDjf)`0sETA$Ps&4QYBPJ$p&~u2YG$YO`R8xr&QW|mcPZ%6gzMwln8(sKYe zh_H3>ZJ1eP`r1b@U-i}7eUaEj{0LThMVQ`r4HTrYtB0*sn`nOU>d1Swld7m_OFtH(N(@Ga%@Eg2XD?00)9KZKOFK7&%(Y z(3ZdiXvX^w3cXg0C5u^T3S%_d5b{IyMbG71LCveGM-0U`3)22y8jA&m z$~d%&~VU2YvG>D8HQ&WpE zxBo$5hYyF1w1BOzvK!6$u7VuoLHqc%P1s$3tFUz5Fy>I49lYhPts1XD==mjxaEK4b z4^?yoC<2%hwJJ|Ng-vOkymEcw?m-&zIc7aw{=4We|F2(lBa3m7c5nFG2eUrXDDVrb-ot?W^f}jQIv4IhDduYw`XRU*_ zmU$dbUC~%1A%(kI{Y^B@y~OM8)v`z2zFLlH=uabWm6ryC4*yVp37~I;WkXE0IRVYD zQZu6vH~5I^F#ShFPm>u2#-}W61erN>OjMC`ZBJYyapEmkMVwFjx*q!8#-*E^d8sxl zt~m*Dg|nh=RqIWbHpytum|OjK#a0#($~I1W?p>-Z5uO?iJgZt4y@O6Dx?Vx!#W64-L(7 z)s7cB!-QW6nEpnj1FJ+GUhfkXuxEDt#N+&Gx=6VO@LD$@V+nADgdJhgfRvVLtP7NHwim0CoQ~AI2}Lkai^D*rIPR zEdHrU6o3K`V6gP3GE+HLRd=}svsCnVVKhZS}74b!oYZJQ1GQ6Hi2wb9M%}=#Tqe=PR<&$G0j&p zG&gCRZPJEGQk?u?<)<8)!40B_;rLYiN}8{x2)`UQ-w&IhX#`2W|NeYG5v;cMv>PW>)BUE>{2TPrBD2*LDYJiA;KjCz*&uR+&Vk{<1N zugBR$`M`u;tBtPPDdw=)2+n^B7b-9YG%_5d^T~Ytn(8DSUDmbnZ8#<#XsUtDOJL&= zyJ0Pmtqz3r{W@dCCZRMxu)Z}qQ5pNfX%5~>N?WEWr@PW{py$NvdZ=DhWC7u8Wfq9b zKwuPm-Y-`p+;-nkG|J{v4Rzn}O=#iW4s=(fXD{rk{3|peo46)t!K)}OVkSus^@R2~ zsaJdG%6bWY!B|lEcK2Clx?#ViWfekq<;=W3&z@qO`D+w zvbF}>HyIhFr*vLBXn|Qf;YlcCAhZ^&Z{zfL%OxqdXj?R?$cmS+k}xx{0)@2aAdYlt z1F1!pMV|)^UOA=;feQn>Zgiv3N4+t(-I67oSUzMY^qeOMfk^4*kSusqQH##^U zP~|F@qm!NnH!AZrWxVC;#gZw`xdPca1sp$2$qJoLb#cC3YR_SZdCyu17AT_ZB^X@W zeH0CsYvG&7`#$eJn$_tlwvBp&rU)geP&nZyqFFgOC=rvo#y@_8>Dvjc(jA_Z-tBIy zA6%!o7|pYxrnL^5L&w>*gS^-YavL4f+jLewcBotFrKt%0l|oy01AZfKE2V!jvp-FV zTw&wb?f>-MAeFB9YQozI{SWv*X};&#m3`3EtU$v~LQ#b72Ng2U z7Q`U)G*I|sl3pzT1p#=_XQ!E_3Lu_d^8+B)Fcz8ng}OGfzN1N;;qTj0zc=#V&wU-a@r?KKut-6 z?uN(EQ^L><5_e`9G2?5j(n)h}IqP7+g`Yxwv2By^s&2osr^&xQTX6MqOU$WvH(is0 ze%>0`^-ZxQqpn?ji9egs{}cT&_VXaNCJsnCy$M^yuHiG$v@uaU6Mi@RSPPi(W>ez@ z`;cx~9g6o8&!a1ug{3|_T%AmRNm$%F+(Yv!p?4!Q@As}Gr#O{+ax$D^cZAxA4Oho-hC@hra+Yoy+>Bf$snAt#>s6`jzGtVkho>iFocjrs-rKi8u`5dCMseDhv_U+$a z&^5e|Bz09j3)!M*m6F{pCi9go@LR{%)wK^Bcn`U?d8d~T3=osfIvG}Pv+WgB>>9%E zeW7O!VOl@YzlIsZ>{*e>iarl(2AaO>Q43p@p^jkNjrbBcx3!+Wq?1xxNHL}+9=Km5 zQTg>bZLI3v-xa5CUzphFt_n^pjFcCwxU`8-hysIrE_NaiIzu@ZYS-g&7TO??FzQcK zf{n_rK_snoG+{6|46ISe#`%@B`HwV}QaDyQ^y&hyO6t=mWEV4r z>C{_izLgr^@S;iYPD@z%$`YM7q0_!GQ+*iT581aCStR+&J@ZqX#nSzXH!13E#}pf# z`8MhxXjVp{>QIn+=>ukqCFUYO63Z-FhIFt5_0?sUK(7`w2)R_sIc8Cz)?8u=-UPH$ zMv*sRH={w-ff0qv#xC|)gv^F}DRxW`pCL{dn-u_?+g!z;$(Uano?sosOM;b z%cavfdH;{p04;E6Sh?$u%=`oYoJ&I9b+AxQXWv8NM)=6RLo|2rU{M9wC-~%+^g0dv zu#XE$uFd+CQtR8X(kHf}a(QdO-e<*;^=`+jBR}-N^m6!~b+WStr4wHKo^Sv9pV0|A z?=k5-0|Z4;<&X;#)B?l#j|WI|#;_)M9Bo_IN!l@$6RZ*rE=SoI`6=2#C|^X@VE00^ zvs4G+4Z9?S2V8<>=*XFZ^!oH#V(rU7=@aMkhb^urZ>+Q*a-BF*qCN`A+Ng{OM!&)@ zzFlU3H&K7p6I;$WF3TeIyCe`>38u6NC^Y*{;7m=m zmzY1ns0$I2`lWS^L)Pv)B`(|R?R=VfX8WasR*6@WH%zhTAw6uc8L`ro4f9(_urEQr zk48p>D)aF|$ov0<+OBD8>rtjCL{o}dI#azgm`)m}z{F1!#|I-(-?JAa zi4r6?60pi=WbuUc`VT{SDbc0Bny(Aij}AVhI&5xSXvYP*U^W=6tVA#GQ(#-gM)Vih zn%;aSVhdY|_?W4I4s+#xIG1uy8Zo#YUCj$d5{Q7LIOm(k&D|q=^-rUp*KpSmjC7VX%=(+{^0U@H_Id2V0)rI~M)+(2Tw3*PCrQ z>%Z#VwMFjM`L_%oFDRaQ@W8>Xd+rvmxtqN3;Px4J7ELRq<^x}?OWCBqY5t*GUf<^! zzV^DZZT~g)kKcoC&z2bPNNCLIc$9GfICOJM;PP!p59b{?xZ;O{B`cEWu3ulVQT*dU z-{a}UdN01%So{|oGh?78R`JEeYW`+om}}}IVE!;x?MN`D1;M`1emVObL5_pN!2!91 zkLF65{8>#0p$&)+`#jx|n!i)xloTK!SR-hS9FDTRjMjmkdt3wjC%;}a7ce>&6P=`Y z{q(8FHj&V=>9r2Uw@L%jojFphlG?rYBRh*R3$YW#lp;^~#_2y6)VJU+gwBM;#gXqd z9Obiz*(!JcKALV>WKi1=lAlLh>sy~D1`8-LhI;4{YqeJ$V|2!Au4w_tguYq}e2mA0qP zLV-N*ij!fZ-27QQmq(UvkP*WrNh&Vo!c*GCCR7XdjUM>xKE0JM-sowFr zi66Pio%3%tR8QPLB&*!nXqbs(qm?4Y`5IU8H0A z^{ZRqIo3CiLL1i9Z2iS@J#=~Q_H~U*!`AHFfLnTXkM0)|{l9=|erBV@`82(51nx(S zFwB zrxD|C=>7i;0sWu(lt05)zp{t+k0KVv$y{{fwli2&G^p( z{qNKEKSXMydDU;MHW$nnu;;5$EyWz>G^@RUiyEnV8=R})WH8?E_dr=~VT1pJTcbP+ z*c*ybd8r7V3Lqa^6Ek0BOT%+E2)$n{A#}HU`gGrC&pjv-DaTj z9W)qj4_57B$PcPeJoIDW9vzgDkSX#t7L$1r|vJF<}U z042M(T>-FYez%&@nTK>^;e+&cg2wCs@V~UtnyPEVtO>rCVhZ@P0!R4+X{0oU{wY911 zc>d0YE)Uzd#SOCyV{ZGdWP4sex4wUevcu{ty?;@t^zSCL{~QMX!6%Pglzc`BVROEa zRr5uQH8&j9Vx_VN7||#hJjZUD48E$SqbHFNBR+~io@-S3BF3oZ!7rZ1e<=f#qu(R} z39tScRn%F#$Jl8`ek%=COQ32s<4JfcS}5(Kef_0*6Ad4{j)q@%^uA|urXi%cRpMOq ztZ^Co)qLuILHboE23P6+%gUtxU;Onii@*Lg9mZhUzb#IF7i2!+u5yNNDWXw~fixd^ z!Pjlt2IcPLNj`S$mpi7T$Y>B>_{w!3*!AP!@@|e=uMAFoBl3?Q#0Hb|BomHKcdDF9 zQ;#K?&ahjfGeWnA%F>$jN(Ghh8x#`~l~tpk@(TP}Wh*+)mX8MbOI-Gd<2?3$ts}=+Qr#5`(kKeH4z)&rcLhbS1}lz` z6@)otDJd(~s&GQzpZ6l}nSByzdqZ>g>&8xQ$A-O~alzCo%V>iCeO`A}m({aC`KI~& zOqnU(l4cE-zPX4v(zb%QRCDS&L7nu@H*ahK2TT|6T{g}08j3}%NHj}i>%YiM7i@&UNayI1Wc&z{t! z4h!%OHIPk;T;uK6IW-Zp`Oyn!zbyvGygQwf!gq9WlZeE`Y){v{y|DvU{l$-X-1FaJ zy#Il$Fn=$RuXj*<91}|%+4t#uP3^Vb(h+AKvEm++8RVJj;>OBGb%>%5(48N$2*LUvi+)G}}o-Ec6u z5DpHl@1Dx9ZYJ}9 ztTe(LXO?+sPmxI%ZyV=D(SpD#!+nLgp z%&Lfno-?jTt@69#a&UVO+1uZejOiQn)lSZ zE~5XR*9QGP!KPQ1ja;ThfU_$c@-H5joJ7pkq21%o)_Cc5m`uJMg{n7}o~{U_)`3gU zV-}B^2kp)#2%d#sTG8;rNtX3qc-i5v9fo@H7*r`=*Zf9EuC($2ITM)Dml34;fQc07 z!{ffebr5?z;E2i=Cbh~Iam7*x4|~YFwNDEe#Pu^yEr&M20g9dk<~b*psisB{q$jjK zGde7!W4mkpOm-G-13s2%t~1CuDXC31(;0V>ZGeIslS~^u_;kF=w63&7hV=S}n)V%o zhypQ_!F`>Lk}KOEG;7dM7Ew6X!YCg@++oinF;!lbjMiX<>X1|O_QAQuU2zAxl0DLf z0>;x&rfd=H9j;nN)%ESeucDG#`1<^C*~?FYnxH`S9trIj3bz0MSpVji*H3=ifW*CvmjFYpbf-K^N0f{MY6?BCkpyR4vC^r zgWh~W-)B-L*B+xsJx+q%k7N6bbit;&ef@`aqD;39W9Y9kv)G()1tABSOO*|aFYwlq zkZ0I>+=W`~`2_E_XpI4>y^JE&#HS%^M=qmWBL=ZT+VIQKm19k!t?NaWF@IURNyX1 z8v4|xYZw#J?h3o{v2xnd?pWDeYF8gJ&md9Wth-Hu%A8@8D73GA1nK2$BHW1jO=0+F zH~RNa?=^PB?&Waju*TmWx48Y@APon5mB~ik(rdFp$=I%Lj{t}_>{AV1k#9CdHg><` zMn^F+&pcWFgj^g&ns0k$dGHTGdoVxB27;BAaIVUi-dP5fe8A{ZbBp_lWl`+ zU>Uwm=*>?f;q+Auj-ZhSH= zpjhpLqL7S?ujo$*mUJW976{GuDXE{4Ta{!RPxuN*K2Q%Hh2s^I>Gb8^;bxVn85&UY z{h5JvApO?um(OmuAG=H1(Rb$anPMI2VwuX2-bDZc6sI1k>_;2AJtn_)}gxxS!NHSHMfofUp)`q0YPwA-M(jDmt zBAkn5FP5QI-K_O}cn6p+X&+{$@ivY&;42k4>HcAMx*8Kd9LzBqLk*@~=($|@nUTS> zOimK3R;$-&8b4yC9~8J8o+q3yV#>MiDpcM-zPOI=*9wLM6_ojKg<|m9J^J$mLh8p? zUdk#+QBbShsoi@V)~aWDTNbml=GEM0hP2}J=_Z1BKVx{RKp+o*gqQrs(4Lx zqhBjgo5NnE5p|&(yPcvp$Qj8_8oS*vJKB~ltR(EJ0dMD0j)XxAo*%dvOj;}rYB#lU zsLT8M=QNVv7Q1a>Vd$oXKZH~N1T6Otqv3z^m;W@E^rsN~-@NIcFK7H~;N+h#{`Y)h z{Kq{||25F|@44xpTlp9F2z|MDkyMtc_LSeb=jh?hiIC2;tg0!2V|)vPd)LO^;o=O( zK0SzRFiSgP`%yV^_TE^~1u*jS?)+0>ydKVp^OEOt%PtuuZyY0>#~L_W4A*rd%cy3b zUz4n(`1*?a8t7C?oCn+ZqXS`Y?-ZYTS~R=(J>g<)w4HbFlxINePX55>Gwjk)*ZKsS zza(IN@oZ6kw%YE_m|GbWbr7c1Gw2o?%ax@ob1EbJg9Z!Z?u}SBxtTMY_PboU8~K;R z&F|Rd+fZnTN)+xp^r~gy(}++zP+8UK$gUizH{WDF)lAtAKF_IuS;7R~HPn+~sEl2F z$<#z*Fu%5|Hrix+SEVh#*ExxFfS)(m?$_-!V0Cs9H#r%e+V#9^?kPvxi;J%J=>8Z- zkz$8tr$G7w#+)hHMUq}E={&NYd!Gj~1lGX{N^0$7X6Tyg-ff*-lOC=^jUA`3#^kz& zzf?E5m|LE>7H2S2+u$NzlDaS1h_+XAiyzKO`JB&;rR0!o@blG6Bp#vp`>4d7@GZ!z z#RlkU+LIUH;vdhE4`9v61%8scym*xLCC5rn2H4N)(0p%Vnm79Dux@V^H)&1#&PRQe zI8VITtET&P@4KnOiUD%cn!cU&b>t^g?$@^m`ad_o>u*gg=Y(C0zE$Uc{TCy9w`8x| zeHTEiQMR3J{`sr`{qtspeRh)WHiO#Hf*x?hAD^Ty z*9Bq^=*W52nlkAZ1BX7*MVDbjGqR;cOA#kY+1y)^GXD$y)1F86ao~KOYwz9#DNBL~d|K z3f?jAysb;%ib|hX9Jx0VkhhLVxfA5`wO+{3K@XiCdzXXiSblabFr=Oqi@A4`vWy+mt8U&ZF@(x=PxY{Q~d&@O*a*&lPGH;dEhQtBpfLQQMmXsRIJMZQs-@!uGzZmECLKO)a|7 zHF>J!a;ro!v9NL|Z!7STGY{spB5vU8>|yLoICm4oI!E&@3ntqvhKT8LIqpLI;=YfC zFNKnX+u2o-uC4_qD#}8Ht-<6m{ql*joHUVrxxuEzMQKpk(3|tLU4!~yyNI1pv>ZAg z0`pqbcE#{ytCXLWr;;nn!fwLKC$$cQgAaTCCtEC^-zpoy5n}flh?;-7+b)q^h)Y2k)2H4VG24f05F9AsMsFua z6MVaBpXn!Nzw+M8dzkwCY1iDUQzy<(dwJYps4vxHLtM-;1<`^UAn2=s!ze?ERKmEgmB)a^`KsCI%rrPJc&&93{Zxv8H&5cz$m@H46)?D|yY zm0s>I9i^h4c*CHdHlJP_Ilr?p`3&A1B4&7;mcKF=A^MO_I-1wdg>!dvB_8ey*8Hf? zbiRBd(Wm)>U-(_rn_oE{cYCxT?x(fY1EKwg#nB!oZB|?CSgX_gr~O=d{~DhBB~idS zB2w&pbqEx!jDmB~V9IpcNtP}2aN(K2jbOZ2?w;=2%qdO|R!4nW&dK5HeGKF#g^srS zbVYMnSp5}hYSqsUChP_E`M0e7rHmE7^!8j13@D!(UDBJ9p@p*GgAIHxOW&<-rt+@@Qqk8veyo4jsP2ZSKprP>YkeoCEea)=L8RD6qx zfMY-_Wlb;P2yviB9xmKLOS%X5p|d04T)>;d=HQqjJZsjc^RWXgQ$*}Ph)LUS;|Q@Q z(p1NP3L3Fl2{Ro&)}@IUwPBi!{5#@%QJwwhs%yA%y>`=L%z1t|R=NzE%HVRw%Oh1D zWArr;VPfxa9z;4vh^io(tF4(WgavkxcN$_H5o!ia3J-BPnF;Zt0YWCb+V<1AlSkp` z+pvke?_#ub1tx<(&#}otfo?pUG~PC}2~)v{VD+adWNjR2_v1kauun)@q+SLeoK`l~ z%dsqv$RSjoRiEYT7rY<(BowD5RI*cdr$aHVB*U92afAg_vap~eVSI-wpmszk)t+P?8;Lu7Zc^e71H27n=)@e-Q3 z3K#lJDVDw-eBNdq+>XoxdDj%=Bs=tZ9;X%7c84KK=ErgShL9cI|{ZH z&mu5un)B7B`pVMosg5me(ZL~_tLTQ3j$1w7^7MsVstX|N9ME8|Ndn;)uhmJL*Icjd; zXnR2wGIv0=3+dP!!M16NR_TerdI5eld<`6KX>duGllqI$#}&mu`_$WeC`~@o{+qJv zA>=yYeuos9KoaCQzVq?$Dy7!!ZA#jqJBpee%2uUji0{DVP0o=j?|y3f6gxXc9Sb@4 zO?cnJ&O{Yr3!F*PGCBXe$I>BjzkcfnTJ$59@;f*iH~y#$-B?RVYx-u;So2qt$3d?j z91G{b9HC|R0BvjMCI>RRhB1MvX!MD4R}m$>Xdx_@6VeCgyqFrGSko-wLe-8)o3AxD z5nHP9nDPS1$ts2q9#d_@UP!pj9da+EUt z=LTa_v8WEAg&@rv^E-qX9D1eR91_~sj@6}1AWO+oyBzu|m=e(YQ9nobabA%-$!z?vXO zLrevVBVD+MpFwl#8(-El-V|8c0ilZTASj(9LLY^LqDbcol}}+U5K}^1la6zGMHHSv z3_$j{=C(+_|0XE{TSGjTFK~Nztptdd)taXx9%@IpOm&$0p#}ROXL6$DHcEugnbHLZCT%KicTSUML{r48{uxOxX-npAVgSBP=rFhaB6YDWjc+a4m*3=Uf4* zKmvDIGlr%sDKi?rm$#fmJM{3fSUH9ED88My946mU+cko(SeZu%CMzB`Hl|H{56IH_ z28^Ln(tOJgqJTjQbipj8#sqMwaq;6{0w;2g*lg#;Xeo6#=> zXl|SD1*6&7s8Tf2Ff>pd+tcKQ)*yya1>!8V1<3O5YznA~N=VkbO@Gdx#UHxaU^fwn zieE*9MQRc>gUEDqhrgqkg!m~M*@jfiAvl3ueKA+=&S5t6USQWPs4NblPVUy+kT6U~ z2Um6zj&Y&w755_RZI-f>={+>t%0ZZT0C)j0AB&|{BI%n62QUcR-!3|n;4MNorMR}R zrM{2ZwN;#9LJ9$_^spErG24|SXmL$Opt}M*_0$j zVzW^k#c;&Nu*cc+Opop``q&Q=xR;SZw*r3=w~LLD&gsqkBn$ot$_$z}C~IY3aH|(| zgb#;j<9PO?^&}%_Rw~`4LDPU=!DyW&CmO;Bor5EtlFWB%GS$xID3fGk`l9c#jY+SE z>rDJ?U}BPRgd;6%YbeWcW-Wx5D>JXsd{F1ZnG3s3KShK~g)rH86WUjB5<2cy`NOl* zhQyf#rF|1R_cq_79RMfCKa@unyh%rvz~88SBu<5_`J((ZY9j?B6OV(@=ytvu zP76Udt0+)fpZPf;`&ds27i6V+c;EIbe=AAQW7cNh$A~|u@JP?#YS4h);!f;;*2Z2y z^9B2qYE1(nil~otHpp0ru49^A3bK;kjpcGg9Ok#`CFsJmQ+zs4w+}=CPxDmzI;!{M zgkSP=)F$9lfq-rTkyyb(snGIb_A&XSVe;w0<(hoP(9Osp70w7_PD_$Ft7I37WGn=CCNNK0Qd=_ml!)?sG zn%eicfSxOV1{#C}D+4uLkE9_q%`%|7^dns|WBM|xUpuuP90bbBEq!;+K%kc@}+B9A;mr zV;2{Pp?*>}LmXz4wSr}QXcYCNr<&!>Xl`$piL+6vbFb3NBL<=0DYJn5jxioS$ z$RC%ZPl^j16sb0hVyjEpLe>nmXRh*FI1A*o-lrzCE1&c%h5hcng+?v;%#Q}!rNMqA zp;^OWapLV3lYu9RfZh*`xrja*JvJdmH1}M2-6gPb*z2WwQw_X==3X92u;S@QCP3A{#w~z4+ zd#vT#|H)_6ac%5iKC{LChkE(jT;RylKJAa}2R-Zze^di^YBBz(e^?u&>-}xskGUWD z13$92iGO?f<1FyZkoM+3s>@&1Z#{pwVoSZ?E8x*sTlceNPn)%G{>SKtwnrvq-_yGE zrM_dI#K-epd(ta4*cYGvty_Q4Gv7|G;&IsINBPIiKW=OOupGE#*-osY?D1psjxE*M zw=L~+&)+IPx_?W0bE!OMMg6zq)xG79&e(4(+x;Q`;qWYoZ-uB_erTv-N z^__9jSM2mF0+`S5)2lnRfBXAhV5>>K+n#q%?Zf%EiXZ1*KO%o~I?$7?C{XzaA{g%QX(T~*jf7te);kcdM z^*Y%f+aK-kG1ZsnPrtGySN!m< z{8s%vPK>~6isjeuko}w1H<^m<+PeFqDe(M6;6@1l#(CF-n1BOf;m7WG)G7VQm$@}z zo_D^GFK`8s-|~g~|ESKY-8OSs#XR-ysLNkYM@QdiW&kh8M+_CSlXm0?LL(8!Y7wcV z9)6&PUG3l1e*`~JtJSbZ>+jNie89X246)Sn z9RGj?!^izycEG(l(z^Lat;+k~F5dW&{qSD(5XL>?A2#J$|5kU~cjep; + + + + + + + diff --git a/docs/static/images/promo-flash.svg b/docs/static/images/promo-flash.svg new file mode 100644 index 00000000000..79810c30a9d --- /dev/null +++ b/docs/static/images/promo-flash.svg @@ -0,0 +1,28 @@ + + + + + + + + + + +]> + + + + + + + + + + + diff --git a/docs/static/images/promo-operations.svg b/docs/static/images/promo-operations.svg new file mode 100644 index 00000000000..3036294ab9f --- /dev/null +++ b/docs/static/images/promo-operations.svg @@ -0,0 +1,6 @@ + + + + + + diff --git a/docs/static/images/promo-performance.svg b/docs/static/images/promo-performance.svg new file mode 100644 index 00000000000..be8a101203c --- /dev/null +++ b/docs/static/images/promo-performance.svg @@ -0,0 +1,134 @@ + + + + + + + + + + +netalloy chequered flag + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/static/images/tree_example1.png b/docs/static/images/tree_example1.png new file mode 100644 index 0000000000000000000000000000000000000000..9f725860c6ebddaaf516e8fa65193726742edf9e GIT binary patch literal 17804 zcmch;1yo&4(=Lbw3r-+72M-Y3-QC^Y_2BLyIKkaQa0%`N2ol^iaPZ*nE^~O_B;SAk zd+*GwSu?}hu=cLK+p4O&>Up{eSCp4PM#MvefPg@jk`z^jfPj7lzAuD_0{?}vs5^&% zz>&5R5mA&95dkPV0nM#!%^)B&5x#19@~R$R`yVZ8;g}Gbu3Y<4nXRN8cz>*}6fx1D zmPr(q!iYeVQlcTJsi}Kc@I}ouJYF(F)o;x5)<=J3?)rY^xLyBx`8so(cRPD?eiRqt zMk;Y<2ObF@B5qnUJ>g?@SZGlSyb+FJ0E8iIreNqgF{a(ds^ROW?MpK>z(ub_gp1a5 z!eIwT6&y_O4RiqKck_+_oM*`}2FL)j)0n~^PyzcvE~mjiExEIUQWIdYf^T>T;VO-I zr->Zy9=kAXc3;{~->I@+?Wl8kNmj!SJjSvFt2&MxRX%3R%tK0qIUrpX< z17d_tP2Uj4h`j%G&A`?9-1R16#USmC^oha45cW?@zn`qEl-xn_byvTh&(U$w;U4SV z!n>%bgUxqgRx-zd1#3#|F0w&4yfb5?M8+s#AmLoGj?>3jP8p4`-3PKKCd%=Y0Q~ha z$TuqZVXq**qy6gAi}o|2MnwL?<`?B?S)McmJP?m`x}zh;6pyaTi$Q${S4&1dzK1%B z8pY%s;!+Gv`TpJXBBC)4?eOjf+FSRq-0alk8$}MDFfG7Q1cH0vzF-PVEP=6bQRV6Q zdgczdjNqLr+DN87*@lo*9245t0f`?E)HW;YfiBTM(ca2PhWaXj0t8i9!q(iCVgyU~ zfxhFhJDMgf1l;~kEt}E3iPj7;{75i{LTCEyzb|wZk$jt};{JK* ze-~c$G=T{W;rOAmK5|;)|L!}nH6bJr5B`yX75^tT{MWors>>MU=l*WFb)xm<)$CPq zhp_;B9D{^2kaEsA>jE+Ilk_r-_~$wUeS~15yr5kHUQO!addgXxGPujo7L(>`A3^jo zNIIr2JV6jE$z{y-j&|SU%ptMC)9F1if6FZCbImD7me?n;^G0kx|MH-?hhO2(iDwEj z@8AP;*`Ei(kHi?hv{}M=;oFI11j2g}+6mo{yKz5#t-Vcm4|?_+u^P!+8xI*5;H7b; znW9;Fez>!teWkpEH*Nya=WlCh{GP!7J2?$^I5r*u2CXrS#+o2HZD~h*P*{3IDJ;YV zTh=J__n@f-?P!PVXOY+H(1OlHqjbCembns;O^DmXMj!!JYR()@ZiptmRbq+AX9`pq z|1}8os;;!o*TDgtv4QL*kaSg$I8ZEx$OnP1PhghE_elU4)>#m40jjYGS06sC!oPzx z`;3tZyDdm&32XNm`XZop719Gr|9dqP@Wt5D8)EVQ@Hp~2NGv-#vA0yk-CBv0K$H|4U#h?T(QD9g?Z zC}wduli=#kCc_5fASMvwKq`_feIdL`AXlVSmy6+%akim2<1tg@JHa$Xd(}5;qOOXV z(?plT9~AM@gUs_jTE5bbY3R}v5SAFnb-|77i?1ft6yxNltFF-WKVY{-WQWs<;)}NH zax)?}lBjn6nZs$G2_a8Z6j|HlxfR=+K*3dte#{H_D;PwQo4zI2BAtjNi;OjZ94vO{dN3ERYOkjfYLjjcv^iEHVvsNop_)!o5V_8sjh} zX^NIpxnUY$yiH9_?Ma8mKNI(yDr(Bk2i@t$ubGW z!czODmZbKr^LuA{XS#E_Eu$?8Z01OeNGutMA^K#DWXfd7WG!`Zbt!c+b>OV&tnh4P z`C~bYZIrFDEv4<7hBplu4Yal*wvh*>Zfb7cSCUr*S1bpO^F^HS1j+>F1ds%{1W5!O zT)qA8v@&gluc(R=%KDc!5afA_Sc|YMaJKzMoinewwwpMcSeq7`Fq);C7MjJIlj4el zN1zT-^=7%6lRY~_Mg?RBBz$)Zln&$%6cZv5dfjQ+X%fWo z$|AVwBhtsNV8q}G1i9AAiKY0tv+BRp>=4L@Nx4$SNs_Y5aAc za-rmn+Mv{Og^eWR@-6dmDm9BZOMAYWmL(QWm$R0=&pjyck!;quEs7|V4wgz?olc=yO^I=Fkf;jBBK*os8-4L zl`i+NOs|wLzq{nu=Y%|G8$SEC(>uL=&@98g^=|JT`<~z8=Oz-L?lfc-`~nVG`|<_O zLaIWE37!&;FAQIf%M2<6i~-hyOvQ{IChzQV3^na;W+i9e4eyjslvhVEjoj3nYGCRm zWKnX+Dyh7lvYom&Cor!t|6$&}WxUn7<+|mJosJEnv)5dIZ}%Qm3q=b-`&MU0dqm4d z=eX{uhGd=-^ak{;I;pl`zQMlBwr1JIV@ifcilISm|L>3J@GzK7@z2nyI0Z2SK3y-~&fKUER(GnX-%7O%ooEH}N+GFpa^H;RoTC)PM3)?Q&7qdv6QA@Zc-tVub z&1y_1nL}&XO_L6P93d}x`{&%Xo80@E!~AcK8O^Z&z~op_n`o}7gYhuv3e)BVPkQik9Z%=B*Gy$-frmw^3g|1AS@#6sOQOM(V z=WWyZG5w7`j%}UQ!M9@P>ZkaPg2|zcp?Nt;73C>iXUTKPNQ59gaHv_EMMn zLbt(z?4Gv98RtfToM)-EXtE+J>&{k2o%Et{ZtZpFdx&^@h&DBEfPKrSb21 z+FsrqVYX7C!s)@~pxEK!dE%Y8>}mDX-fo|cY3LvLg?pzxMjWcGReg8(VKYC6Or}kC zm{*wR3(Q>khI3BrEkMNb2*b-oLC%-)gVLHy`vMtKf>A3CX( zHZMbrEO0gAIV?9kwWT4j`Tm4{+GqA=v3aM8^MT@QXmIX;V!2>yhYb1O(|&Wu{FuG*KLSKXf&I z4Qeg&OrS*z!XljyGD!?cI8$s>+Cb7q|3UA1&)$!%A)+oJ3JWJU=h{lpu*wL}Sceio z<|M5uLNa2<#L+X{Wk6%Hg;}h|U_J+&h>RY_Fod?|-DqNSO&zwYrDP=k^zu=1%q zH(|mIlW0n`)&iS5S>h|Uov!UZd-Qw)VH5#b)+cWFCfOGBB{RNZzP2{YjHyO~MeTLp z$)bg8)`H&3-VWwf(bdk+dY#i~$BM00(Q9Yu4{@?_E5%bv3VGg*Y*W_8h$h!PKYH(} zI4STbsL3lqpKNpHnd)lEHV-QDT2)lYrg$gim=mu~%+{H=YxR%X>W`J(Zl5wUq2bP) zoX?$5w}`l7JDyLLgX+k5UYo>-zquuJrfr~^S1fhVU4@wn2bv{pUMuLAzbjo-+H@bR z(Jl$8HQ&6*Sm3dspT^3O<9VxeykO{{e+6~hHP%}a8AD&MOK-QZbg*LVCf1Ua>$+zR z1eIJ(+vrr-tQfX3Elo^lxg;!@O^$&0R?a_PuYc_V4Y+?h#=9O^G-+=1csBE&`%sG- zgOtyU=yz};>aOS$=DD*!HDLig#$&+RUhd&)>2bYZ9QQSE7RcHs+8h2Whca7E-(QC< ziH-2de#*N5cXD_<^lbhlX2H3sIrp|UhZ);P0H03U&*<^&*Z$(<^jQWFsNUPI@N{!> ze!jP-{V>8PUd+rLdHCIIEek;!Nf!x->m4>D%a!k8PK+(1ZpSX&6ZmYgY*DP%torOT95NihS`TY3yL!_Pjb-eb#;BsztLY*`7=3T{?C8iQ0 ztGnHGTqN9DZz78Z^Q7aYS_WQ_QHMDSaC9&5bWz9i67C%J$2O4PRuSL?L*S|B@o4F( zw8tb}?HKMs-QcqBaU8x-^SC*kfP%G`q?;t8#vrd(R#7a)bEskCBVLa36`#lZTvyH4 zOXR!f)N73BpS`D%ODXbtqn3!4>~?WYK^@FfcN=^44GUM@bIKK=D<=n4JJapdQ$OZL zCO_O^;EU{OrAfZwgDhlAR{hB2Zt*7-=wn&ro&jF+bV_AKsz31^ z?j6sQbieno<23G2KwD5`pUv7f0v!@{RI(`KI)TdDl31>&A5m(FmsC92A2QzeHD?b+ zf$a2f#_X5Le70{|UxmEtKyZ#I4qFHp3*VN`OgKosvnSUW7raGfb zC(5Vut9*;^D=y)r7p~qAG9Xmvhe)2MwvYIr|?+^hEE8+OkXZTl8ONjM*bB(|mL7 zzyBPQ#IOz$m%7q9zMkoz2CysAIq(;ga59V1o6^*AzX#}&Kd6a1=Hn;H9>y#X+chIM z9#Yb=i3@CbA0##Pc#PcK2H*9ZkniSKZ!mKzH@PDn49it7emrVv=7IZ!HBfhZaTRsI zXP>s7MqFEQG_tUrNSF9%|Gb&G1ZV5i>@0CUN)#nvnmLyKeCA*$^njiqeATs<=PD-E zHyp_oDX~`3eBN}3(2l8MCbx-8l|@TVt4;$~Hkvb7fl|!GIpTjObND?iq$RU9cl-Tb z0bUjIuWg`yv8`+ej>EuRjeUZ2txMzyR<`9X}@*l2W6Nl$SVV@^sw@Xa? zdM>*XU%@=lhEVNni3DLn=o+!h^u@zEJnuk%55$E?xrC^dfH#~)hm4=Pc7GOJr4NFW zt*YsPD-#|xBpYVoGpaY^>Hjdlft4yCPxh`0wN})ohj>2x4M~b2_6>S>kW!Crm#7hn z5xr5ivEOIoVl2F9>`vK7zQxF>YE@h6D-<_eD(LeXTV4DkGJS&Mz_B0KtNs{u2&`RP zT|BT>A%GC;ChY-cab|tP2kB`lbBz16n^dSY`q&e4#F8wn08hAc8*I#EZFOcF#C$41_n52fKR+E}( zQE@nq)Io_yy=YK=ePS>xDIe(+wGOmKi|9u|F$)S7Gd(OdH7+)4S1DvGIZryaLuBsK zeKS`Ijxb>2c*2AR`#4)2`+_lnO}vFRklpZUM0s#ibrqUb;@jSxn(*lyvgsbT zImNTwONss3R_%iRLhPNAUnO)Vl=lZ@k&0m4kJn=Mxuasoc~ULdmZ@WY9uNIVd7gd3(z2f-Gd1amzwT6&R?>5~ZObld zReB|y4@P&TaHfaRHL3gHpR+CFpJh?%C1wxnM7u99{yKE`)x1dsR&Ax-MV589kiG5e zp{!YM_blR1-~};rv;t}@AKA=re-6$NliBRUK`G++V*sFqG=bv*muME6$WkzHf>`Aw zNP(|{k>iA-;^-tvl>=D2e)@OCGNt374x^;5taJ_M@)k4Ug%gQ*kH0GCoW;$6JU_Qe zP2%a&{YH3=TN#GhtFdl?34@>mUl6Q;M2ZB7RFC8w#T``=^fXvG$*E1cY>%+s z=u%8ho$i$#KH>4DV~Z#L9{ch{29qJjG#er-J>xEu9+O7xXYE-n-zueAxy5O_vHhx% z>M^ZZv&#Lb);x)_l>vwGLf8YH& z#!}%8+zLOP|It(YP3dj@^SfUeEWeU5yMZMtE6W8VpNZYZDu%_zV(6ND+E2xvY}S}> zu)jGShi;9ru!Mp(MCsLstPaA;eXj|!N_YnnSg;u@p_Z4#r*#kW z-(R07pC%1CbnSP=-A#PrxbB7R%aP^HJn#Z>s&TiSXN`Hwc^La#=9(v8DeeE7Tui$C zsUV$G_*Pl~#9)j)=0AUzbaOd5IcxTC)(n(iNml>l&vC@7e68|!(Y|&BbIrm0wEx1Ri#9QR6QUMvy>$9ckho^_&SIYoM`^{iq7EHQE=p>vUT)(59hIqMxSEr&E2~;^7dE10(ac zoZ!n_`k*eMwehY%-DnPKg7^%Chpg{i&snkmA8 zi{L;5AfW!ePa9yk;FCf=aG1^zGMrM6SOgIoa&?TQzcW z*FP{V7`nEX*<`~Mi`bQ|w+RmO#>hc;5AzsC_tyo`d zOpiflR$-gtj(uF7`L6Jm z162Fri8hoLxcHQ7>coIEyn@_E!~QJ@0zOsxd)=-4qOSsbPEsARJW{NvJgb_;HeC0$ zS=LIZ;AS{BdlX0H*c<^xv6P<4dunf zOyDJ3-O2Dd;xig)R#K$W()u$WIO}UXj}ITY4vp{-&`~g}C8&QeIRJf~TPmFvW;CLw zD!JDP;R6I)5mK2}D2%8ubV`&kGzET^-&R|XpdZx0et>be*{+I@08CJg1C%jxx=OT zJ7ep>Q+(3=1KHn6JqiF%YTQSpq`#5{7)4a8+D`Ppi+}~1Am2umUx@Tyi5*mNXx;oO z{NI&>1>A8baRCvEzY=!H|Ear*s9bqDxfp*2AeaHMM`Fj$2?Z66Aju`v&+fJxl7|vE zkF56!`H*}yHhdK=tvoKsD@^PaJK8D$koAlnSs;co_Q}mLZ1~d=895LNGE^2OwBHmZ z)t~t*BUlX%XgixHyHIaK`wxP$gMu3{9Qs8a!eo0DZ@1RhgN#BDU<3s-1kG*Ck1jnA zPeTVj=m@z-^HsT3;W`b`XW5hwd@EOr)1oD~2aI3B)KLmY6P~x>i$m_^+V6;2GNuKP zZ=H#0Fr9lJKuY;NbgI4gdgRxB=)V$HlIq>RP8rNaVYeP&O=Is045$YehuUhbe>C=R znHC8l6Z;@f0zklz!{OpgSKG_BPdY?PTUziPH+%DY44)8r1$ug*$iQQ8xEx#g<*@Ja zGK>p*;ip5&-^!Q>x+KBI`uQL8ib8ZWx+?=kX#c1kA`-0A|MtmcY=RfQ_FoUs1e>?I z_&e#I{ zpG!05Lr_e37N*gODgcJ$fyQx}jnmoWQugkT&#*(RPN7CQBX!KtQG#?srgaFI@Gb<9 zN8y4N*d%*ta;YqztZd_uYvl^#(8F{ne=VYLuxS%JdHO(V_+4J!XF8JJf)YiIwr`gl zIKPp(JW@IQy7@xEg3#0y#3bQ#U0H^&-(4R<(~GE*W910p+RF zr=?rCUj)Kl#`3%&>|o*GH!+=XVO?IYBy%r*u_)cmXb05%Dx{YIqzc1 z+HfZJ`HGZI!-hj@*!+h!^p()}2AH2@TD*^rYnQ6Lo(@%y9+v)VM-{wgsp5V{lXJgJ zj>`_)+73COsVWVQ2AxBf%p(JDRD!7BIX?w>-=u8xwLx+9nzO zj$H*i$2l9W^m!sZ#Wd7lIwz>EAi zfAMhk)|O|B2eZ0-a){Na^txTKqG@ZfLB3?TYE^)ygtAO{M;F|(&J zW+yoc^(#HWWTZVK9u>K(@}2trP$(Aj8{&&`TBh#Wis}#`U+3pg$sdCp2r2E$?InSM zC2RWE1qj{%Lj$O&INg4;w)5_a#a-~M-A#D7{q)Jauo_ouefORP<1nSNcbGk`*Bok% z06Y)aAfSiLu6!u4h$o#xAxqhXJyP7OuJ6aq;rWr(r_!O|cn~!@@?NWJO%uZQo zevod?LA!TL(+%tCT5xOvuinn}MIT)MStvs=d`D4#TTVzZDrgFRy1i2exUyXI1A`rH zM-hrXg`q=wPU~&NIS>UUK5(*+r%27dH$p-7@fODaSk(!_or@nz^&GdKFodvX%=a9=#HZ`bd&-DU_Sm)^Yx z$C(y*4_FCtc-&og3(AzS9vzY|-gx>vD?hpmeAA~)ZxZ6?vm|ep@0P2M1Y2BOJDl?_ zNfxM#`85C^ZL^OC6nt9RTrQ7g-^B{Pd$YG+kfD6<0h?!H`aGd*y>nXrjlcM?#J+Z; z;z?p~@~|wHj67k$R0dV#oxJd!Cnv9SYSMWpEK~#4cbtLFTKt+zE^Z`bv4#QQK|d86sj%|l<$aunT5w?6HLisY!*toZAAhD7ZS*w(uEJqk?T zuA6e*-A-mHNW1(fNiAxOAEM~GV>u;)IS>utiudgvxKTYor!#;@kfrdarQ2qVrl9uy zrLv(C@syv?uc2{8>B}cKktQ@s)I%vl6nObzg#Tp{wW?`?W5Wen?6ISJ;=Q@y!RghQ z(yi%>C@2$?^CjNK8TigaL$SL zEU4@dw<#Zf*LR^AiI(Bg7)P5al&nI-m2J*K=Ky1o3+=qo3# zTu*J$!^*5|r$w13mn!)A7rcW4jHh`CRYPx-ik!%>`@VR{)tzE@CXTB2AYH!N?X9V zBa8=C(Co1B!hTfhu{ug5gFEN->XcS@N2gKZ>)kBJgnw(%hcnw`Jp}&|4E!)K--_d8xZgZv^{P8n??D16=m+CrDaVL z_D|73p`^j1#0}ty6^Cl78bU*&Kuz8fS8Hl-{I7%Y*>j^s!z(g)&M3i3mJ4ptLKbdJ znhdIc(~l^}fakF!$XAqTma>3aHkRwVZ?oG*XRGStdM2plgtj4zH8R_HHp~n*YNoS@ zef;n)e2`zm6;1kljzg3Jyo&L?(D7s`0pZO)6KcZ`MBak&#QqLgk@_7m*a9_eimLIa zcaH^5#UG-zndJ(j9RP-O;Nk(d_I4dOM1+^IYmyV6Rlq(a{^C>SNi9V>W;~+ijc&v) zxWA2+mIpP4%2fRw9*O`)_woFibCG-L2MIGzTSlyZD*~q}DW@o%4F8!Z?ThEWr{+Ey5 z17c-tpar=EYN<(qXRFF*f-&+#)!P++9@SR6zBU@B1lSKW6?VuF%5!WG##m^#h}JW& z-t3DwRa*QP1n&A~Lt&*r9aF2s8bBYzusT@T?w8YAxBD~UZ&o$UTACNAt|u2qU5Ke_ zLXLqD`nIv=YoYDW+d|6}H*?cX)qD3y9+aty3srlQ5R{W2QEbqe?scgwIBuKG`pfM6 zk>3Wi)+`XwXk_m)<$W}SnQN1(h2H^XF#{yiz7Fmv>f79B|ILFO#4aZc@~hPC+}%4Kofk~Wsd*dW||`%^V#5|#!yTDVG%s~K0Epq4v-(UQ)P@vhoinQ z2%|xG9Vi)|3kauhG8FYiG3~64VVWsEk-(8@d8fYW&4inUj3pihyH;LP8Z_acC;gZ! zi+pUJx!9=@Xwq!@f-{DO=ej-%7&(tENJsC73w1JJ*N`vsrFmC2MXjfxbWWaAGbut# z1i3=_4gA{0@z5WN!o{qzdgD2H$}10x!^EVXvOqqh5w5VGk;x+L$UAk zHO!`PndN}pq-YwJ?!P%%hptl@0ptCQ>S26UbrUBhkz5>pqn*PVhf-@3%XPqTrI<*2 z`8;1aS$4SKM%n!yf!-}D1Erfyhud77{k@(XH0kC?UXmEfq~nlw(5>yj#}lv5g^(qM zqzCQ1Bq}SY3$+BKTac&G<$MHQZ4L>3W>4R2l~kXDls+-tnmE+tU86{yu^B(9q0y)o#Y&9tuJZX+B18qrWA!+phfC;jhk}m2W96C~?LKa8cGaOd3=bF8_SKrI0DaX7>pX3`mBp#M5EX{ej5&Jo@hCSY1ZC)2h307}#yR}n1{ex8iwaCf z+n#OjjD7a1`R0S2n$u{<)FW!jiRKQ>Nx>G{4bSPg%r3wP7PV{Q>=DhyL53scpk+YG zS_}hb_{;$T3D2>of*Wc<=jXBwm6Z~H5&aQ|FRU)@`T|X1oi|5~BU!U&Sq(RpnK`NR z*{|urt$z~AP^WiqlVzbdVB z{I+zm!lU%utfzDjN+$_i_Q2}CjwfDvf9wW77BO$NLf+9F{#vmj4d+r2(<{jlL!5Mf z^c=!!M^8&cv^xtq6wHK8nayYpPz>B56W<=J#O_xu!AS+G6kU*nd1X))IY&bpv?#AE zuU8^BZ2>cyLVH+p4hJwZBThmI6;~wT37NVPgzYY1a$#Awj+Q2S>9bJs_EA3{6atmw zM%i%0dIlK>Co&iP$}RK^GIyK%BLW8+``A&f6Lhsx@MogE_CPqXzlqizJm=%})tzN0 z{+}TG86V7d?e*$^r2I==@PPyIZtg3CKY~De@|Wbq8!P=UuW|v-!16<}eg1MLEpZTF z4(LG566#NiHZB5Y)`+xEz*zNPmSjg99IRWaHFCim)4%k%ASyTw^s?-H`$zD9uQTH_ zm$8YM!L~(Lc>ZS(+3cpBIlLW3$Y#tc>l!`B14z@<;(lol*;!w*(x^IQ{S?eU%QKw~ zZCoF`IUMG_H9PZWhIWmpW=mVrMU~q2?=iNjR^dLg&VCp;_tuX#jC6mwSrEpNz<7wb z;p6IBnEPlz*-_*><7C>)kg$?KR_?2J1FmI3gHFQyf%*2IPYT4+p6e=^p_%vDJ|hpM zuPS7(OffSp1l}fWUr&EBuOxe^>##u4g8ik0YoQ^7`%53Ml#5yLUv9Wns#17fI_ph6 zohB#4{11tWTM50$qQl}`Ic9_Do66gS0t2zY(8aB~P+w%mt*ul$EUL!~tL3k^d%U*aZui(VW=JJ%I@=$W76o!_Fv=tLn z$+KUm6~+_$*Awd!4R0e5;VM@QXhLl*rls|le6_PanumIMRd}Z0 z0j_VY!S>`rnwG;sTA2v|X0)1y|1>~--0MPm4?vQ0*Vy-5Fcl|F@_wyqdf3#k}oc*Eu z(5fQ9lCplXzZC`Oy%dsB0HFTAUyX}1A2|ZgS{mU1P2JZYo8Yn6)<-QkvA;OKom3ri z^cn5?wXI0{7Tdtf;J+P!(89s$u)v>Wt(1Ckt}U~b5Xl$61H~44$~C;J?estVANH7@ z8E({FckNW-$s1dDpFh=Wl5f+QEoytXSuV4`c(AF3sE5iVXO`8LT>{t!V-4`erKb3F zGo$k1R+}RUo;5^dK)@6I*7YoyFjYX!MTrgvWA(Do9#!I{Nf8LR+Sn){C&0SY*YA1^vk!Ng^ zXsM)5vMo*AroL9$*k&%JC<2)_HY$GxvX>_wVeQP+KqPgb5jGfCav?2_0dRqMwX`mO ziCD;hmNKk-?cm9LPF_weaP^^W+@*UrJA@G| z^Sbd50pUHkuMh1)d>h~-94My%qgo`@4F1O#u4<=7bmtl0Jz8_;q%T3!o|7@s+IF*W zaqUFJ-UfEZhjtyfoSu(P8ci;-J+=+HM!I-9Aj}umGAZA<8Ta&}@8g`x=|*Sg)vw&g zkFBk0VwsNQfl+OeM~A-oAkiEN{<2MWE)kpNEK17b_hv_x6jst5au@a4@jQ&}S&PYy zF`!fYU=KL2UFqxs=7NKxMP-`yr#QDoUa24YM6|3pOSvDV%u;H0`z>uZBYB}BFS;eH zOz^(Wj`k-|4vjBs6Ia(`8s9*uC~`bA?BjLGx3d9)vBLECL;R)u7I{8vRJA80xrY`+ z6#oz$seP9G-bjT)}tL4sNi;Bjjst2%UdN3|{ zxBlEH#HVU4TAB!-L@99mUff>3*SI<(FD_ceGu~frY>7ZmZB}uo(LFDLqa$$;tKOm+ zSuG=}!3Mmzfj0(`@A-(HS9>4^Ht1h}9u6g@qLOGVrR8FyVr$Oe`9zn~Gq;Qs=R}dh zcT1+Q5aR(A4!5bF4L z8_{}iO2aR-&u?Sq$F~d!?3>`v0UhReu`BSKmHsNM;JJ}%c@!DgSf1Dd1*SWvf*p-T zMOnFGYS}pZK=2U9knMZMyqVsMm%#^DK}`fdb^@{TzYx7iH}y#>N~v+Pc0NM2#8iE(=NNO?-U3?FttCi$ZHiZ(zl1@X@fD2%>1)5{taBvBhWu*wLY z6tNFh?k0|ol)UMr9QH9cB?(4>G>{oSUkAWCLh4UYXoUppNRdk}iHhV9mn2FsiNLz0 z+ZqtnYUCZj2wnfnSS@h)y0nrg9`Q`h2)s@E<3~|ADphdi#_q`S6frSih`4?HRn>~h z#~Rehe6~*b@^Rv^BrgZLXCa z(lAku5AHde|5k2F6IrsVd|*z_+)`De02MgdI1qJYn6r{TqkV|i z%u?4#H+F@@h-~!De3mW$NY^Bw7`nOb5fihKSl)BM&v=^!*;@8<%fV%PTRN5VAWke* zjvF-8Zj!-v%!-On z|EGNT;4kL0d25BQ>!db^#G+(f^Wo9ByLe;d3YbcL=j!wiu-x-Ry|i-u<7Qp*W)wqg*~jwR0+I$mUVD2 z4f-ZO{*g~1FR{sV@*w!J2LTsEq&`R@T?)ya*TZVr$Kv#SIN>`_@1Se3<-J{Mb-i04W`jOGRyTkVG@ zA$vR;-Z)G1pf$Zbh)pS1eXBN`XhjIWHnsdtkq55V40P$K&Tom&C#)*CA-H!`g#mwW z&#Bj*iw`f#5<40>W5$42KWFJ1rjO9$Qz3{r&%2b+ap~y2TY8+P2nbYsH$5lAVRrF9 zF!l81|1$M=p(u%e_S$1}met!`;+KLi&K*3qP9{HdYlL=+5F^mrfk69?45z(w@xLGy z94sRm9J3k5B^n!(Q~Jjb1vj5OAADR~SikTRln_e9drB#*bL#5zzm~n6s>QD>S^8uo zvDoITA1|H%>P~FZkwysKm%UU;+J{-@-1BxrJae{hb3{9h72(doKiO%;i~IWS;dE>z z`7U~bP`bs3x#0a$J0h31i5c_!S)MIY-pm3n^#su*+nWs7FYWWkq2ex3vir z@wLZoM3(&g@Y8yLsHw4?OYg1#<5WR_;I1zI@d-5efnQd^Jfom+?ciR4*A^Dn_JSA~S_;C)kjl^tnERl!$rx`=+?)u$FRyS(0!uc4 zDX&$3Zz42(>Yw9?mP_`O5aM8cUgx+oBUtiDx3jJ@E}mz(0Y#P!NK{X9{{&;fbEm5X zYR7?m)bPhkeSLxm^`TsGJG@efJSOR4@Lzrtm zyQ5TQq>RxEylj(f)5EgiC3-w#TY)EHVnN!$@j>SVMdS+J#CIv5Jq2K4W@ z=TMiEBUwhK)V7oP>JKiFPr*n)YcIMiVxy`w(WSp!q4i9E#(Yz7iS{LbbtRDoYTe=h z60oF45_ZL!onsXI6x81(yUi<@aX^HX%9#VylbwVqAt&I0yl815UhPc%Hm1`9f-Y7T zDEP36><|-!3Fui9j@*T^Hi4ub(nU-k9)q9gnGz@O}DzoQy z^z6bX`zvpfD&CAI<)=p!#K!THb?!{g3-aEsw-|>tsJh$N+tGQ?Tv-X_%mKR~fTe@SKABy8CFzMf}KvtGjH>K5zNdzRH;erB*})E+*z zg+1v_YOl0gedS9AYy;{r@1P%>4V-4vM6NWe8s)LE*a_qkNB zeky$!yj`QJ^>Y$nwhOEf$xR13+onOQ`;_{asKzcbC=a6rS_7~}QgYnW{6^cm9Uv44~SH!@@rt& zYenoTXFB<1nDW{Go)V*eJyD-F2oLK8PbuU%u~ZSh7z4&A(@M=x+YonH@Kl*$ox-&*TQQ z67zFs<<%mEQG@n^)@Rh$)*5b10-g{hx|M^N*n$wjvEmxQsMtaY$=mcHuia5ENepN3 zp{~?hzN-Lofu~g(>^47QIzLnI++v55mwl*yH}S63Yg z7#uL=;&HYiRU|o+vAyzqd);yQR3@--+!LCA^TN8k9*6ON(W#syC>HI-=*UvTrQ$Cp z(JuHA+w`K+|BXaDaeic9MBNqPKNNrqcA))Jo3F(sOwC_H>;G30;Qt0C@!cPiS$L + + + + + + + + + + + + + + + diff --git a/docs/static/og_image.png b/docs/static/og_image.png new file mode 100644 index 0000000000000000000000000000000000000000..4e2759e61784914e2d4ce92daddc9cfb4a01013f GIT binary patch literal 17639 zcmZ^~19YX$@-G~FqKQ4p#I|kQwzFe%Voz+_wmq?J+xE@8|8ws--}&y{Ywf-Isj6R9 zS9MqO*`abWqTgV!V1R&tzKM$oDF6WhyL^>xpg_KU72*12fPi3x%moGI#03Qj?^ z=2pf)Kw{rA5?z!~7SV?`$E;}UwG#=Ak2{*c^JzN75=dZ1FF+*K7mU%XS%ts}!$P1` zev%n6=k3b+3;qW+8=Bb6nTk{L^Gh(TmXpGq)N${_RF zf8g`tX&{SvTN}tVCz@Cnk~AJ?hdzRRo9`JPgBTxv1NP<_56CXhA|6M}euFS}XUU3# z1qvdZCHA!87+D}w?4ORYia!M5%g*;2AlI!4(Dgll2I3U07orlAOrncyA(6s1HAQ3e z&B_``5~J#6Uz3VT9x^&K3TNvj2J(C{QAd%Bolgu!Tb>?Vo}TVqK&JjYypK-H>iEDk z>UiI~X4sj`MDRUXr%nCf0osEFHamYk@xZ8c z6HgKoqES(Ketu2_;aOBrnVLNH;qZKT^LRSpl3aGnvK+eGF0#*8M=mTcUP~{>DM?PD z+_8O@3jf60@xgsk*3jM8#~YU24}TW8E5>jf0qbR|Ul+*aBm9HJ0N&emdesH@N&$)K zoob0<=7bN-J&a6tHw}DaiO@au2S|L)|AH^;6-^)lxHb*FjQhd-LSE$Yv;I?mi{TD+ z0yUrBn1PcabvPmH)+k~FI;95$`8T9vHblKI9vuW)HyRtT>K3#oKOz+9GyxtJh}LiT zT0e|E|9!o0WFW+PuuJfzJvwrbSbX2vAXd7?ZQ-@RRC}0hzpsG`>Y;Ff#%(bt`w>7P z$p65~hcFj}_>E}n>J`4|eUN>u zeZ>($xB&e>P)0`dBN#&%O&FI-Ov>#_PZViNsY(z^h02G;?xi=%;mT>sfYMGy!_wzc z<5Fv7OGS_DD0x=pXt_z%%{*^@F9Ue5s8gJa^d0u0gSB~IW+`Su=0tOxsiW!SnYo#} zMHKUkIj$=9oPnvExsj>TDUJ%}O6ghWMKm*2v#e?F5hW9M`nqK0F`9$?s2~~NC?(S_ znW~A1%9sn!^@i5*R)jk-cd8Dh4ww#^4skE*ce(fdm&g~FZy^v={*L~*5bF@LK_Wpl zLD+Dsa9J27Sm0RH%oa?o04e4NIvp!5OV2(q(_V&Qre$mUaTFFOI%(Dl`g2DAah!25 zleY1f@tT8)1A$wOTYlKEz$D>J;kgV-^xkO7%Lyx{Fi$l|XB|P4DP;Ysgf=;y#FAnP7w{nJ9xH zk|8@%uOg!&Z5e+m4$5@pk=DHkrw3=9eb1TaitxPf{{7@pKqXpbN@cKi));f@XqG0c zOTYAD1p6%O^sC{|2*;=I)c(!?oSsNQa1q(MWy9-z@*MbA?Z2i?ofz zj1)~|O_(I=EIOSVn+q@65%my76W+DuY&Z5rVzWgsOu=2$Lk<#r~QuDll3Ty}kO z{p;xg%u_&0z^iu-WD|T9J%WVeCmQjFhFN{J-uAxmNq8+QXa z=-oN{LIn^-=K?AFI=0%K73eOKc*ivv> z1`vh^6DiXmW4y8S@vw1=2@)ex(=0>^XvRQob`5rL_IE-?`4}^^n$ex}aMFREofj=K z7k!(yfT$0td$GuZ(}F&?3pZvrz>VQeG&YVL9B+lWk#{1w@ecE)nPc2|?E>RHg=nqRx;Mc#Bb!pgF#;~ZfbP5;#R6u>#X-Q(mZNhfH4ua8z?&$U!_%DQN38_U-_|4uY{^F->K+xn3!={ zHd_W*a9w1&oLOoxU%a!~xbWl7^hkZMxS(EswaL=kJb(Xq)wKL(Nr|upzlZCpg|4{x zPJOx>*0@uqfk|GTQ1>PjVtj`g|Sk@@rRc2T0v(8dB!QRjo@#3IMb;X@YQ6$YEj`r*p} zwjXQdM!HMg=faz26XbWuV4GVT-j+D0!-4hoHpWMl&7loi@9eAWo1DSEl`Ls*8$5d) z4(2n)wOg{2=?lh¨{7(sI-qTHpc6Z7 z*eBXY_r7OOaA;^N`q8w`)btYwH2L-o34U7dYgq|xE2i!M1Vlsh_Y3S=!0QYI1O{oY zq~@q5EyZC7u%^*B0vH(6xLVtONdp0KxpI6Jt&JV^30$qMY#caTxe5P~;P@*4CDRfT z{3GIM$xWyxEk_^-us0@PreUI?BjkZ0ARyqfH!|T+5EA(p{OgIE(9F@%mV=ho#l?lj zg^>neZ%Rwg&dyFt$3V-#K>Z~_?ciqPsP9T`<3RL3O8&PVA!7$advjYybAS!OU%mPU z04GOoLc+g>{?F%M^|ZGAKZa}^{^iw|f3&XpwzTv#bhQ6BkfXWDzpMBELH*_YKVinM z=Kl}aU(Wx){t3qa)RF5eq8!oyBXbisA$><<9tJvkMrwL`YI+7GdIk;#W)6BrF53T+ z`L~Dvpat!X^&J8BN&tWr5C7i=Bakx#I078Z0Ja2ziYx?V()xzxHh&4^|D)$XQy!zh?{@)V+txN6Sws`*S z<9|~B1M#<&IpoY;jjhy#%&m=W{_&HMm6?n7|H}Dqq#(cwV6SMaZ)p5C{Qp4ygY@6h z|JYIgw;g)=uNL^Xo&Vtc3&}*CJ)S45Bc9o&I1ECkSqWM#KIyj z#INKEe4YX6i6ZckE->*(ih?vS01OMa4=jBDLamu(hO_C(y(ld^}v-1j%%k65>M89%{hSO*wHH-qE zpDze7IE*hyfkXm@uZ}+rC=3`16xmh*pD&Y+1=#anu7M;lA_-3;H$(^c7Yk9YJDbm! z0fPV6=s!#;KOkcRsj2L?zfW*5U}T1fKbzsc#0UmapyWsf!M?S^{tqbE4p%EEhj0DF zVpB`q?!`Gw<@0-?fh}b=A0IFRIEZ9pFvgF%@zc`*&ID#{b@LI865P!)S7;-P-B^~X#0<5$42 zyE{s?u_&gjv}NDJ$P4mWAabyN3}ddbd6FY$M?9~SqEClrI4%-fEb~kJR#AU4Ft#`u zM3N?vHm5gVIKg(O5wD1$QDu%R;Z9TLV&}t*OYU*v113Yv9`ok)4>OK`D2z=T6+nDS zT1Qr4;|USaWG>cRIozD%Xzd$O*k`Qoo28Sy7FJ>!Rzx=X+Z2Ba`LdIS3xys&7w@Kb zAH~wFLxySUKOoH{EYDL3QNu2yBohM}DzSl{6a>yErtbnbAozLy1~ezh4qbu|J{+VV zwt-#pc$EukhjGbAka%`V@#k+(*iy)@mM?V2bvg2pFLbk(_LiyQ zDyvP%->jRbZ;?T4DG3r`~fTsXMVUr@4b0t$`SFZagGR`ue0Gl_U|FoR( znV7G_FLppUMlg@GP8WffNB|OYe*RT8O#g#yes=M@(}0+y;rIN%-Ah0|`?nm)51p(C zrSLeEE|u{Yvtuw+e~99*t-oD@NLIt8cJ(d<+yNS*H11Z?<{HlHuwTt{-5P zbf2!=4?&+ga>#N7+1=hKWIrXH?PE0>ekZEykZKqWqlX3t0A9H(9&>MzUbe&&tf{E0 z*WIo|>=fv&$Hm%Rtw!Axu|FH_PGDIl8{dql&T{X-)f-pzz@lf>z7HMcl`L2M`urz< z`27SlPFGz{Mu!yyrC*e+UrUD~S9VmZf|3pRQFNCC<2F^6EoN{ywSb7^p|LG7sASYJ zr4Zi6SL)YA^qbRuTCWsmU|YqP1DM@zp`E%42nx~eFy*%7^7wvSGyJxm%Copm*?52H zR=g^7xFb-Q(J;>-#wh?qXWbZE+-jta6kIgACZ1Cyq}okl!VZyr1a0_eNT4e*Dl&W2TBkGt z@bl@Rh2V*X0u|Nu7bX`21_yx=6800A8EtmJwr$UJQ#Phpp?N#DnD6dT+)8PKI2zPP z<_Bti;8=>OBpi6qTD+ZfpI?N(=qxvqaCSrv!{EuWS_0>p)${cQ2}S{j;fyMACZ>KC zDa{_>HS5RdxCw0FmHd$}DKiC2D{0Pxz8k2rl(@r^-Q7~{1sB(o*OWhCs$N)d&!P*u zqckt8Th^x>NH?n&@{cxn-~M{f43)8kG~VMEQP}ACamy3%dJ3OqM9@Nl@4&=*!Et=w zrRGpT=uryIPI*IB*EegWXYUv9v@HUqe+lu!{2e1TV6Gr&=TPvgM9?ibVL3Sy`gsco zDps}z`wy-1yL00WOo6cO81x_?pMYp4z|uz*us-=4RC&yvPsX}lT-23wJX?Qtl+<>W zZ+-fL$jfz8atTdyO-ehPp7|#$M$I=xYib^%X_e~mXK@wYj9aUhgcXic&pLL8%C;-6 zB8Q}lLRwBupLH=y8T=aPP)iof@T6mSB5bs-tURA9mg(U(m~!Cr^Wg-CNlUUjnr;le z=n&|DkeF@2Qf&eeGnO>FVsCx5Ck7$ArN&u~Lw!8!*XW|I^_P*Blvpt&H}J)H zyXHe_yb)&Gv9Eh^(P6K+99xK7ObU+4`HfPgy_7~ML!DZzi>_^7O0ih|IyS?ZW!8gS;{SN z&B&^qO75Nt3pyd>6-ZS~t}X@@te43OMIX+V;qBuT@(i~vF^m;3Q0h!z{?28{6x(YG z{wpWTGX65EvqV47cJBruL#f0~;ihj$N;h>s3Ch9Av%9y2{CWjbO7|V#pIf&tRL6931?aJOlEaKD8mO3`Tnv?B=puoJ=Z|Hc_L)18WpMHrO z#VFY2G@DblaEm}C)H_EKw7(iS&I0ceF4=cMz-nLGp+%fUjCvp|F;gkY#nePm&8e@5 zac-a!U#X?H+fJXkFSQ|0t6iaaq{h5q5OM)?SnJYNLea%B6OVujK7g zwIEL@G542jO{AS`x;!aiWp>)=Y~-i;3YVbYulpo@b6xvZJdARC*O$r$HW0~|Kk-=R zw*tx>>YRo-MMqOwL=dH#fVRVg2VL4uI-GJL%(OOOT_AJ-Mb4#3Vp=DGlinq-Wc*0| zRDjT}(l_ts<>aK*%nj2EO_zy~%|v7MwnwsiFzYo7?tVd44SQYi?tN+X$nW&4e&@)t4l@60a$ulnoOZM3xK%^hb3;JOnWXrFK`o{v!>8gZXg;uAR7WEmib(E5|7t_hZaJgCmy!>hI{nWqBwdX+vx2 z+23i*7vIZH!!PFZmnFh+iw=0Ce?ybmPRVz_&O|To$;-ZR$pCn(TC3@Nma5!eOw3hk zRCdcR=gySIwKJ=>LxSCJNXETnO|t+VN0kaM2T#X}3XA1Zjmhw=HODTVZ`nzyk6mqt z4OJ}bicAx=#4VvkqDK8aF5(Y4k`oo{VUh})jb)UHtFF$k^Wp=KYD@iZjLc8@9g)!V zO&Zgh+$E_zZkpfRF9R_w+TYZH=xGxC$F^nsFiwqEnCGu3=ll!)q-vRXT=cd79!l5J zu|zwWhP>;hd?s|a5Z9j96{J-ZP<|C-nX7>wp{RPzs$#vn&WpZy8Db?WL(xq8e1(kfx8U@`JC4S@)X#E#K@`Apd@y|!R<}ec$99U|vVn{#2`cWViTz%1 zqwq+eZnxs$(dA7ysd$pebb6er1O$2MJg=3#?5gt&-8s^Q+i{YJB-PrTWDGAJQ_4^0-OHmzcx{Py}Od7i>;JtL+S zxr8P99iO@%e5|}V4ateOF%uUB^8U(s2^9i5rVeQEsylb;@}H0V4X1cEJ_=R5I?WHN zqbTP>N}OC!FIWs?FbR>f6zZJF$Q&C}<_`$clc$fnPn@Respbc8+1-zFaW6pM0(J0$ zYbY$v@k{4vAtVYs5sA#O& z?TEy^r^Xd#)F0Z_3l`_rH52qeI2R{=In{huqrO1*gVL9tMp zuV>s)(pqi&naNmc-M5m4*XjA8vj9;E&Qs9NtyvA7pB*4MC!)ncEPeNC#C+0KRrT=g zenyPh98f23g##SrTC}Z_&pPt_n|}+$A7>2YzMG()D>Ri;d`Ks!E3esLmcsyLgwj~gHDpk)7i=$}vR{a2o%_?DYBuAuN82P!rs<~#! zx%v#;zyOng5A_ZkiYUEU*O@;Laipw~lFKu@$xaKM-*k9#(l^qSo;wd$KAY?Jgtccz zl4HVik$1LOuKPIUMUd7bcy{(S8w zsK|hNs|{|_*?P}?npRJT)HiIwgq;q#u|5nk+iqCM6tz^d%5sF{RU2K>1%4<@4AKT5 zkrL?Xp0h!^u+16irlgH3TpIi2 z?~nShOkm{6yv6xc5AQ;GeS<|oMj|JiOWj?cz|WNYfl-MnEzNrUIBOL_0*HLGTwang zCm(C*(HXD%xToK-vC2P$=STgF&q#878qEhdD9`i|{+JX&%-@e%ov&Gun?0P>G~zUT zm{64DN@R~ie;pU}MSx6aj|mqW#-OO>=3`;;a_Az?b~V{_$w{xSPS9?_LaP&rAGhsZ zdR6tXDq@+&uT9?{eY5NCv3aFA>l3*@?I_*EETf76yQ9Z52u2P!S@(Jekt7di7}z#` zI68|e+jzOVWI0!1Ev^|?*eAdQ=A6IZcFk~zYcw?s;1+4-J#MSFuegU!K&y~Dy%ku8 z8v%vEaYms-GrO3KZgv#F*4zd1m(#*DZ@pMv)7)3fGbN>*DsdxS<*Q>_4cJp!&wC0e zk2fIT*OIt~gPj9{C1tmQ{TfdRGG4)yy6eFNax8##XCA7$WVBO1I^G%(Vv*~pJb|mX z(sr0Ui!VHB+Dxu|{+^nE^8vd=imb4DV8x3PZy(W0P*?{oYuO~fd;RIU@%8!5$D;X|mag4>!2HfPXJj=dSjy<8ka}A4k z2z(z+O44KkWEDQwCOX?p_`Te-^#&|WUHWy|} z9*i2x+?i3uQ9ObUQ_ss(lt zltHCVs9m@4=tcg8L{Gn#+~w$9x4~jQ7|)%IhQ4Wwll5~yflnbv0RSN=f{SrW{bVCFN4l66nmaYvc5N+ zXMy#XSNlyA>ho;o*bW^z6~e886jRCGOR=+qQAV=CAA4pF94O2T6%lt2Ye; zR6@4MN4`IW+ld+Ae{VDW*dU`XgB#HsNFuM{j5^2Ff?FwfM{S&R+e*W2L)KVaHG1xy z1~oCXrPh%=IBsdW+Zuf}x33kYnER|T0{Dl9vk?FM8BeR5+2p8__`1h_Ck!s(7Pkv{ zp5%o}3|3uffMwqBHecru(EErCrA3i4GSTmpdt6=X@OYfa;B-0bmZVM<;*CJX$HzCJ zD$+IWkBER9TxpC%P2NQLHqy?izMHso@$BhximEDDM_N2tk;-PhX~njFj+0wlr7%xq zT`>ZJt^0KCPeg_25^1H97L?YSRqfdKVsMC8XP<>mJHOLlXDzTCF9nO8{!$z;D-TU; zrf%oaWjMQ|R#}-ZlHrIuZB-U1N>#?-Q6$>jm)vys_WDYq^3+nApwtu=1<30(Jr12- zQ1Os2I+-u~{CK^5gX;a1OV@>VdeXL8s@CuM1+G{q^?XuRWip{W65_BbC8%UzFv&leXSG?evrJg@sl!iAMT8mRK?fbJTn{ zA~_`my7=1T_4zQ1_w7nhWUZe3G6=72h-^2xl2b66i*md{9iqOp8ZPwZn6u-x!r0zI zN$@?Y2>}@h>PpQHSq#{Ml+qy(*%$lNDQ(=YyU`XsABYv#xiB^}L+&FHRP3MQyg(Zuj~2(4)lhFvA5N zpJz?;hIV+dO4nyPS6E&)i0$z6{nBTW>lW@ZT8oqXD{_0k_b$H`y6*jnA#2C6PU;dU zj`b9lobXhsc#)Wx^+ST<7P>~nghH&|nqJA(2WAsn14pMY8+YWs4ywG!qhLP!V{WP= zqv?v6G`TOe8W9l1>XgaZ@DYQHKJMzspkf_9p5qWe@Z#u7_#52SKx?F>O}%XKXcnLO z!$CSVX8eOq5M^ZZNvNT`D30@VFKQu$BJ@hpvs+z<<>`{{ytMUBqvOf^uC7G8?cuAH zashxQt**db#;kL@!8!<tH{ z+WKRA;r~ct8`sNb;AahK(`^+LRJ1zg=3p6+^s}s)?JO#c^jW|V)Rwp zwLu=7F78iTKNu*&T9Itl<0fx*tkE7Lyv2ctA1qCCwqlWt*Xu)MK*?}Lo$X^9lR1Ow zL^>L^#-8{*D))$$tm-N70!{kiebb{S-z)3_P~jEH&+MG~fnKBXDOlO#g=r62PD^=N zX!tHcT#CLlg54>^yoNS-7jkl&$t6kej<;mgbLtSPv^@Hvxow_-fmz| zKNodiO6f0;DkJ@RFa%blg; zLox?qg5J2o^8geP;_8N;kLxa`8BQw6)4tehYo8yFT^;WC45eN-!=xd4BKo#&dgQ*D z`<<^Bu)M(<4Hg*=6YO6u+2x5uMx<`~d~zRcHPvFyr;p-$KLN8-4r$uxe{GeLWux)_ zP`xVDnT%t+7bNMdsG_hO-XQ2(26jC6S>3lwG)%}vhdaGfw;s}bYt=c2EM72gead_% z2(s~omiz>1uCWv>#d|KdNgI_tzExX=L+QO@v&jX)icDziCC0X3^eyU@7XI$|$|von zVh%V8R(0I8bvi2*ItZ=ws^R8-LSLLvZgy}Gq0~UJq0`#w07JgNSbq(ptfV`+HspR> zbBOHSy6IK`3L;*FrqXOY^Ml0Wdk;`qEsj9myI<60g%Q+7qt)S>qi*sXjKJjfc zqcDlEi;A+tToFk!S`tL!cjJ_eQZ*w@tPUoYkoY$n|##I|eiC?3jl74@>E%u4v^igib5hhe(ZZ`u&LlKrA@99dEN1;Od%qm8LIR%FR71@ALw`G+& z0qka9$X?w+PLNby!M%r#Z#R(O;1`{*HhoBimW!1wVKZ)COhTl2;$rT3{!^4j`$S8somrfVObtH~^JI?T~HN2gN5%Y`QB?5I=*?p0G0(GzzbH^Lo{ zrDs4iJvo8caDrT)JMNcFvwEgoJ&u=Zb~OPV`_!Ry(HfATXetOw&}^8y4T0F;CgEJm zY<&?4si|S?{vj~U)a_hXf`Ld5BK-77h5F?-sN@O(hcL6)?h9Y{lnIoo6DdCuSOFXD zPNymhTteND3{tC08v2(faw; zyFOVGG`$rI?wtZ;#CG3b?)zBm`jJGREd9XaNTl5_S`Kf%XSjZ)wyyhe0cpA|$b-@+ zV!SWIrswT=+hu1ss?Yt&Jp011;`X#C{yDg=3b_mr#^uM`cVYv0=yk}IyNmTU57mq& zJjZS)@F7nJcD%h!GS@6{@lL_9l3^5h^f-kJ^0# zoFy?jsy{+M&m_vvjUwBf{gk0@dx;&!rWo;8P+^`X#jWqh6rH!44S8s{wqd$n&sUpv zf1EhQ_!@_JV!5o@LB6##wM?Ul;Ci5iJ|&e^b-s+!b}>-|be%0!V3!r}`$KG(^@wPj zky7D4Zg>EmMp`xoXl~D&HnC_ZsyZI9kOV&(sWlt-RHC3xE8uWBmdyJ@kiN7~=s0Y& z(N9*X*Fo_0$1R<+i)LXx+MrBs6O9YpjUdKcR)9tXrR^K)!J=Yo z&+`4jAXWX#nPx6+b14D#Mz(C_<%oE-E-D$w)CQb=4fq(lGII}i<>}ZJ;C9=-dW~i% z9_)t!cn|u9y>p#F3e~(Fgx&e^a;o)XkaJ`#g|Q=PdY||6%>mSa=lxFcC%jL$%xa_c zwdW|RL4^6+Jv#3`h%7`l(S&a%Jr3%`K(%L4L4oknryoBi3qq7XI)so|LA{r#K$|8< z(Gie5sq#;o2wt5qOTMp%d_#%n)N-b1smh(s!xfM(nAg+Yxm@$K#uLJbOY(Gmoh4 zZV-VdiF{fVd*K8JI6gN&-*t;s=&3|q?;ao+0|8A3$9y z2i3Gj;`#jn1ps2wpNA@uylM5-Q0E41zX#~fP^V$&RdVb`FhV(9Klsc_vtEB7bj_U1 zl`xu&VE`y^z+}4>x3VL2K&2^WMV2Gvd|4@6AxA+$V<*=gR#d1xt))68@Bcs~amiUpf>7Ch@l^si5nkTHMf6 zW^t*^uP!;C3MPfyg@DgmUNN*-$jaY2AI}U(S_*jZe;1(X2sER2o$qVc@_D}x>cFXT zn~T$D$}U$1sP2(UCHsP4P{%gm6woT9fvWk!k0uA=hK*mdjL}Cs!J#nP(37QAV+3mD zCIHcN>=z5r?_+675eV@mvTl0Ln*WN$5P|*`Y~aw2d%$?fd%p2$@MjV+=!(?FuJxd4u8|NKE#feRAZi+Y`T(1_9w5!rCsmK$Ryo!fq zKveG*^AH~W2oHR!N&gC_!`Du%QhujHNQzDh^@^vxtnUa5&utIsBA4*~;r4D;_WSb= zwcYI%PqOQpBmKH`+xfSEywkiBlY}V4*Jh&2A5v zKXEmYD0u0J@Vlc5{#{4o%px?oo{s2IFzY;io;{vogFT^1LZ9-~2jYrGeMlII`m6Bo zZGb%zs-yx^8y%0#xJB7w$_(HEqc{uUeowc9OskLU?rSpzxZ)Ag4)Vh&qD)3l#;v;L zM9cX)Jv<=GzywyA(^n0nQw$rXrUMxsT}3tXST;^_lWcd>GHg%UBXVK5v5bkVG_XG| zvaVbZ^jE%<1?9&GWSx$MV3b5bedT>R4W=OHix(%?uW$fKT49;lcUf(V=_}f*^m7{K zq~b%kC?CoE;ik0f#C+}KgdeM&ob+kRlNZ*GwYc04o#BuB#>|7lfQV1EZ|)of6YRcI z;J%-r>YjzHN-1c0)FHVMG9)nrMII? zw*m7!nYY$)J&4`$4g{7nM=!(Nr7OjwEU+6he~tW1 zrMxbwm3pll?e}6tTLb%jUgd*+z^QPxXD*#npTLD5?nd*nr0#Bg_x@z*!QHZq;WLyUcu4GQE5- zkFE_PkKMAJ1s&?TLi0PYj;l_2T7;zgvJ!0<%_j~n*Y9HfIbfn{pO?e>?rxg~dOZ8_ zioYe^xOhioh|)K+otkXc+liB1BYxXlYWr+l(fn8n&RDCq@UX!RuTqCJ%5qVbL?H(eFEFUug)>|8zvf6|A4L}(Pb#XS;0a-ysw1e_X-hja5JzPAP0)`br zKydT+(>h-}>KfOBx1+)l`5Llg8Q0f)C|vFz-#D-`r@<9nVz(KOx}eU+YET2Av|M*u zxUUikky?e_AC^t8a~%QusiDMQ$3hZbH1Shi@pCux8j-oG7SZ7<>>f?lrZjH>rIxeE z_g$*6iImnA3^nuvggI7tz70oJeNYCuB&+e{ic2^Io(@(^PA7BM3-pcXR~sYr2u9s$ z)hR6=i(UTt`qq(hK#rUdbh}o8;quGH2uOsiKsh%;jOXXTk;o7pPn_;1S?D6<>3VjT zIBQ%IT7m@kr~d0BG7#HHd-r*85p;oXK)+4x82EBbY#WkcTa=exN0q3|ttl4|K~{f8 z$mG|f88IdoZFNM-^h-ZwBu+i|-5)Dat79^Tqc-0Vfk$<8=NQooooL@&lCi&D&i>kWJbf;*B|0G2|U7n4sSx8Qq&MXe>ly z1be{d&Pr>L?{zgd(c43ox@{@Sx|%|a68h!M)H2&(_+)0x`PM0D-^P0USqlHdOp2&0 zDS{n^W#l09W_9t$N@f=PD@AYSnmsIo^#$}pGJa)~PGy${z?c+}9h3y>LV=M+Ej*$A zGz6mvQoU2XtD=YE>K&_ztPi1^a~0(2X;T^6}*-pA95g^$2 zmaE8EZ|};EK^rR9asJ2%qf9R4cIo^QKnshI1m1{?MFd~E_RZZVY`$297?l!miFn~C z-#e1^sh9kb+wP=6ipnh@PzSn!BNHmPb!B>ZxZ@RF6DDRA{mWp97y?ypNTgEpdh}WT z=>}T@?e}O#bR4~JggG{&d=NdnGX!g~*6IPKF_zMTwxHw3xyq!!%xrS7HKIDj z==rx0esD!y^Gov@YI@wB==60-oMUK_!bBeL9*%tKgX5mj+DQP{m|ReErd&JprN!lO zF#%G7IyMrXu4S_D^GG`%+BT~h)3twyyunvdCG(!HAc5$im zlhY(trY*F49trgOISj3O+rxnyC^hApE8}+#Zf>dbI@D*<#wJ|-Zge21X7vi#yqgAx zEQ%qx!$FoKPgl%pZH|EJ%YF&#^fS<1-yN$oko2U04>xo1uH>YOnH|O`B2awQ(d+iK-vC$NF+z`8KBGWv?n!$R!94q86;Le z(ezjUl(`LyQNC3ZUdxPuj@+H+nqL+R37gmQU=}1@G!diJS2Xv{&FySDEv}b(l~%gj zZpM&wtgO(1`2de7O@E)fyq^t!DgoBG$@qVvF+gf`v@jm*&c()<+_8$o({WX+YBd{= z+1~c8uWLM}Sz6xX9ZTh|=nJo54z&eoW{N*M6|NpXELG)SJhfjVJQz(mIA!jyKvDZz7vr#o%jhbbA3GRb~HU*^bSe6QhrTxkHZHiEi98><_4th z*DR6!8R!uQpoYK7zTUf9aQdTqJ${>D^6m{7GR7s%P&(qV-x>~&YpGc?Tl4dVV6bPNUUh^mI?m9n;5z?f zR7;M8o?n+cKU{Q41K|opbVEc!c5$_;#Ms9PKYvniH6L25{tBwFEm1{!S`q2J`kir2 zQ=y`^wQ(L))#AdH+D#ijaz6V<>FS>gpQ3L@Khp3nSf}_gM5K43iJqsiqxL?M^RD9W zZ?C*9!?$Zd+Kkwa1MmrZ+bT)06Dn$y>3ZR@cZ8N?_vV*gCPOx^AX!B>G%J~r0wOtL zD7PYlwrStr%0HFhd=Of);erNUb!KI0DRJK=bD`)l88w__R5vx5J#xBO%krUm<1?yh zG)adds$5JuCN{X^x{g^q8eGPR<7>?`Z5Fgg&R6-_@qH*7KZ1@}IMu|#7Cx=`vRu|GP4MwV;P0(}z6 zF0c-j(SOq&Nu7PXPRCv(30My`=oYl=V&Ht|oO>$J^QmgdkxJyrEQLoeG0|ma6J?fN z*n+ILsOZY_B4&G)@sU0hFcWWjt ze3wJ&tM0hP0R}VGm^Hwui$n{&^eEkI$sVKEqc=K1!*!a~i^74aR@Dbp2rhtd{}MC}^UTwa+l$eOsaF#8pJu7t?YOFg3o@CFCzWkmwKia!iL%XlmD z&LP{Cp`jCFpE5qpy)}xb$8QFe8@2?Q^;N&SW)(Dg!5B~F#8mGOk=OBQX%Z14iXb`< zGnL+eGNaUsfcv8k8AITf$`AR7X9a!oY>#@+SIu61?cuK;x;xcljw0HnbN}q_{SyDP z`Wu7t(`0u8#%&zCTq_FgqW!^qP85+vYE&uv=i7mi2q}_IuIf#2S`-dDL+k;)NXke_ zsjB3PRhi_7nvE;EXt}zjf;8i%oCU7tcJCSjy&u%rI?buvIB3#RUiTHEDb03ts&Jq= z({>WK=hGD(0x+y#b@cYL{dvP;kj2hs3w%aQ&9P8%=u#WMPdkQcPSB2zC{3`*0+WL;IM@ z{DCXuW#G4~Z1HjU=x*)bf$tv?#17!rWCEl&qQl3F0W z!4wGpA_&4aiOOROOvme~gvTg~?1mS~2P>kGY1d=ByAIkIGRJGN!Vb>)VJ`;DXn43# z9qx44G2h3N%MyBR{fT$cP6UeXkZY6qir?ywyigEnxsqNu$xT3`#2(HyLQFd^=cYov&2R_ zP}0Y6;QLdlkxn#Oej3)b>7Zx|%0C-*mTrRXM<+eXXCO~nGKnG2nY$qnB`RI;dp2zsD(9PN-}_RCWjBi4pu0r#5F4tT!rB-x@jK7_zv-EW5CJ@h|bkn+9`pp*@8#=Altt?eu{Tj4{MNWpGhA znpK6&ri9t0_uF4$pYa27-j!}(2tJ-2a>K8+prC?M@X>(0M>bg&k)5%44$~b3ly7C3 zmB!`v%|!HMt_?>}4RC&tDe=a{_qExov&fGzWJ8Ilh={I= zny!Fp)Fd`R>{PwRw7X1>{+4>PjH^w-=^s{@!Eu4B69xMCZQrzgZXIHG&Lfey9%bNAr_V(}Qd)ZQ0{6*!782`CX zZpNp+Boxn?;2rn%oZhr8P7h_Xs@~b0{J?uiYDe(bdqto7R&jSU9$LWT{DgD%ow^_E zB)bLw&eGZ5H@VQQrHbeHzlw=J?n>HL=qoN>Qx93Izrf*O07HxAlN}vR3qvksgO +#include +#include "options/db_options.h" +#include "port/port.h" +#include "port/sys_time.h" #include "rocksdb/options.h" #include "util/arena.h" #include "util/autovector.h" @@ -19,6 +22,48 @@ namespace rocksdb { Env::~Env() { } +uint64_t Env::GetThreadID() const { + std::hash hasher; + return hasher(std::this_thread::get_id()); +} + +Status Env::ReuseWritableFile(const std::string& fname, + const std::string& old_fname, + unique_ptr* result, + const EnvOptions& options) { + Status s = RenameFile(old_fname, fname); + if (!s.ok()) { + return s; + } + return NewWritableFile(fname, result, options); +} + +Status Env::GetChildrenFileAttributes(const std::string& dir, + std::vector* result) { + assert(result != nullptr); + std::vector child_fnames; + Status s = GetChildren(dir, &child_fnames); + if (!s.ok()) { + return s; + } + result->resize(child_fnames.size()); + size_t result_size = 0; + for (size_t i = 0; i < child_fnames.size(); ++i) { + const std::string path = dir + "/" + child_fnames[i]; + if (!(s = GetFileSize(path, &(*result)[result_size].size_bytes)).ok()) { + if (FileExists(path).IsNotFound()) { + // The file may have been deleted since we listed the directory + continue; + } + return s; + } + (*result)[result_size].name = std::move(child_fnames[i]); + result_size++; + } + result->resize(result_size); + return Status::OK(); +} + SequentialFile::~SequentialFile() { } @@ -41,7 +86,7 @@ void LogFlush(Logger *info_log) { } void Log(Logger* info_log, const char* format, ...) { - if (info_log) { + if (info_log && info_log->GetInfoLogLevel() <= InfoLogLevel::INFO_LEVEL) { va_list ap; va_start(ap, format); info_log->Logv(InfoLogLevel::INFO_LEVEL, format, ap); @@ -49,18 +94,56 @@ void Log(Logger* info_log, const char* format, ...) { } } +void Logger::Logv(const InfoLogLevel log_level, const char* format, va_list ap) { + static const char* kInfoLogLevelNames[5] = { "DEBUG", "INFO", "WARN", + "ERROR", "FATAL" }; + if (log_level < log_level_) { + return; + } + + if (log_level == InfoLogLevel::INFO_LEVEL) { + // Doesn't print log level if it is INFO level. + // This is to avoid unexpected performance regression after we add + // the feature of log level. All the logs before we add the feature + // are INFO level. We don't want to add extra costs to those existing + // logging. + Logv(format, ap); + } else { + char new_format[500]; + snprintf(new_format, sizeof(new_format) - 1, "[%s] %s", + kInfoLogLevelNames[log_level], format); + Logv(new_format, ap); + } +} + + void Log(const InfoLogLevel log_level, Logger* info_log, const char* format, ...) { + if (info_log && info_log->GetInfoLogLevel() <= log_level) { + va_list ap; + va_start(ap, format); + + if (log_level == InfoLogLevel::HEADER_LEVEL) { + info_log->LogHeader(format, ap); + } else { + info_log->Logv(log_level, format, ap); + } + + va_end(ap); + } +} + +void Header(Logger* info_log, const char* format, ...) { if (info_log) { va_list ap; va_start(ap, format); - info_log->Logv(log_level, format, ap); + info_log->LogHeader(format, ap); va_end(ap); } } void Debug(Logger* info_log, const char* format, ...) { - if (info_log) { + if (info_log && info_log->GetInfoLogLevel() <= InfoLogLevel::DEBUG_LEVEL) { va_list ap; va_start(ap, format); info_log->Logv(InfoLogLevel::DEBUG_LEVEL, format, ap); @@ -69,7 +152,7 @@ void Debug(Logger* info_log, const char* format, ...) { } void Info(Logger* info_log, const char* format, ...) { - if (info_log) { + if (info_log && info_log->GetInfoLogLevel() <= InfoLogLevel::INFO_LEVEL) { va_list ap; va_start(ap, format); info_log->Logv(InfoLogLevel::INFO_LEVEL, format, ap); @@ -78,7 +161,7 @@ void Info(Logger* info_log, const char* format, ...) { } void Warn(Logger* info_log, const char* format, ...) { - if (info_log) { + if (info_log && info_log->GetInfoLogLevel() <= InfoLogLevel::WARN_LEVEL) { va_list ap; va_start(ap, format); info_log->Logv(InfoLogLevel::WARN_LEVEL, format, ap); @@ -86,7 +169,7 @@ void Warn(Logger* info_log, const char* format, ...) { } } void Error(Logger* info_log, const char* format, ...) { - if (info_log) { + if (info_log && info_log->GetInfoLogLevel() <= InfoLogLevel::ERROR_LEVEL) { va_list ap; va_start(ap, format); info_log->Logv(InfoLogLevel::ERROR_LEVEL, format, ap); @@ -94,7 +177,7 @@ void Error(Logger* info_log, const char* format, ...) { } } void Fatal(Logger* info_log, const char* format, ...) { - if (info_log) { + if (info_log && info_log->GetInfoLogLevel() <= InfoLogLevel::FATAL_LEVEL) { va_list ap; va_start(ap, format); info_log->Logv(InfoLogLevel::FATAL_LEVEL, format, ap); @@ -118,6 +201,15 @@ void Log(const InfoLogLevel log_level, const shared_ptr& info_log, } } +void Header(const shared_ptr& info_log, const char* format, ...) { + if (info_log) { + va_list ap; + va_start(ap, format); + info_log->LogHeader(format, ap); + va_end(ap); + } +} + void Debug(const shared_ptr& info_log, const char* format, ...) { if (info_log) { va_list ap; @@ -221,24 +313,61 @@ EnvWrapper::~EnvWrapper() { namespace { // anonymous namespace void AssignEnvOptions(EnvOptions* env_options, const DBOptions& options) { - env_options->use_os_buffer = options.allow_os_buffer; env_options->use_mmap_reads = options.allow_mmap_reads; env_options->use_mmap_writes = options.allow_mmap_writes; + env_options->use_direct_reads = options.use_direct_reads; env_options->set_fd_cloexec = options.is_fd_close_on_exec; env_options->bytes_per_sync = options.bytes_per_sync; + env_options->compaction_readahead_size = options.compaction_readahead_size; + env_options->random_access_max_buffer_size = + options.random_access_max_buffer_size; env_options->rate_limiter = options.rate_limiter.get(); + env_options->writable_file_max_buffer_size = + options.writable_file_max_buffer_size; + env_options->allow_fallocate = options.allow_fallocate; } } -EnvOptions Env::OptimizeForLogWrite(const EnvOptions& env_options) const { - return env_options; +EnvOptions Env::OptimizeForLogWrite(const EnvOptions& env_options, + const DBOptions& db_options) const { + EnvOptions optimized_env_options(env_options); + optimized_env_options.bytes_per_sync = db_options.wal_bytes_per_sync; + return optimized_env_options; } EnvOptions Env::OptimizeForManifestWrite(const EnvOptions& env_options) const { return env_options; } +EnvOptions Env::OptimizeForLogRead(const EnvOptions& env_options) const { + EnvOptions optimized_env_options(env_options); + optimized_env_options.use_direct_reads = false; + return optimized_env_options; +} + +EnvOptions Env::OptimizeForManifestRead(const EnvOptions& env_options) const { + EnvOptions optimized_env_options(env_options); + optimized_env_options.use_direct_reads = false; + return optimized_env_options; +} + +EnvOptions Env::OptimizeForCompactionTableWrite( + const EnvOptions& env_options, const ImmutableDBOptions& db_options) const { + EnvOptions optimized_env_options(env_options); + optimized_env_options.use_direct_writes = + db_options.use_direct_io_for_flush_and_compaction; + return optimized_env_options; +} + +EnvOptions Env::OptimizeForCompactionTableRead( + const EnvOptions& env_options, const ImmutableDBOptions& db_options) const { + EnvOptions optimized_env_options(env_options); + optimized_env_options.use_direct_reads = + db_options.use_direct_io_for_flush_and_compaction; + return optimized_env_options; +} + EnvOptions::EnvOptions(const DBOptions& options) { AssignEnvOptions(this, options); } diff --git a/env/env_basic_test.cc b/env/env_basic_test.cc new file mode 100644 index 00000000000..254c71fadc1 --- /dev/null +++ b/env/env_basic_test.cc @@ -0,0 +1,356 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include +#include +#include + +#include "env/mock_env.h" +#include "rocksdb/env.h" +#include "rocksdb/utilities/object_registry.h" +#include "util/testharness.h" + +namespace rocksdb { + +// Normalizes trivial differences across Envs such that these test cases can +// run on all Envs. +class NormalizingEnvWrapper : public EnvWrapper { + public: + explicit NormalizingEnvWrapper(Env* base) : EnvWrapper(base) {} + + // Removes . and .. from directory listing + virtual Status GetChildren(const std::string& dir, + std::vector* result) override { + Status status = EnvWrapper::GetChildren(dir, result); + if (status.ok()) { + result->erase(std::remove_if(result->begin(), result->end(), + [](const std::string& s) { + return s == "." || s == ".."; + }), + result->end()); + } + return status; + } + + // Removes . and .. from directory listing + virtual Status GetChildrenFileAttributes( + const std::string& dir, std::vector* result) override { + Status status = EnvWrapper::GetChildrenFileAttributes(dir, result); + if (status.ok()) { + result->erase(std::remove_if(result->begin(), result->end(), + [](const FileAttributes& fa) { + return fa.name == "." || fa.name == ".."; + }), + result->end()); + } + return status; + } +}; + +class EnvBasicTestWithParam : public testing::Test, + public ::testing::WithParamInterface { + public: + Env* env_; + const EnvOptions soptions_; + std::string test_dir_; + + EnvBasicTestWithParam() : env_(GetParam()) { + test_dir_ = test::TmpDir(env_) + "/env_basic_test"; + } + + void SetUp() { + env_->CreateDirIfMissing(test_dir_); + } + + void TearDown() { + std::vector files; + env_->GetChildren(test_dir_, &files); + for (const auto& file : files) { + // don't know whether it's file or directory, try both. The tests must + // only create files or empty directories, so one must succeed, else the + // directory's corrupted. + Status s = env_->DeleteFile(test_dir_ + "/" + file); + if (!s.ok()) { + ASSERT_OK(env_->DeleteDir(test_dir_ + "/" + file)); + } + } + } +}; + +class EnvMoreTestWithParam : public EnvBasicTestWithParam {}; + +static std::unique_ptr def_env(new NormalizingEnvWrapper(Env::Default())); +INSTANTIATE_TEST_CASE_P(EnvDefault, EnvBasicTestWithParam, + ::testing::Values(def_env.get())); +INSTANTIATE_TEST_CASE_P(EnvDefault, EnvMoreTestWithParam, + ::testing::Values(def_env.get())); + +static std::unique_ptr mock_env(new MockEnv(Env::Default())); +INSTANTIATE_TEST_CASE_P(MockEnv, EnvBasicTestWithParam, + ::testing::Values(mock_env.get())); +#ifndef ROCKSDB_LITE +static std::unique_ptr mem_env(NewMemEnv(Env::Default())); +INSTANTIATE_TEST_CASE_P(MemEnv, EnvBasicTestWithParam, + ::testing::Values(mem_env.get())); + +namespace { + +// Returns a vector of 0 or 1 Env*, depending whether an Env is registered for +// TEST_ENV_URI. +// +// The purpose of returning an empty vector (instead of nullptr) is that gtest +// ValuesIn() will skip running tests when given an empty collection. +std::vector GetCustomEnvs() { + static Env* custom_env; + static std::unique_ptr custom_env_guard; + static bool init = false; + if (!init) { + init = true; + const char* uri = getenv("TEST_ENV_URI"); + if (uri != nullptr) { + custom_env = NewCustomObject(uri, &custom_env_guard); + } + } + + std::vector res; + if (custom_env != nullptr) { + res.emplace_back(custom_env); + } + return res; +} + +} // anonymous namespace + +INSTANTIATE_TEST_CASE_P(CustomEnv, EnvBasicTestWithParam, + ::testing::ValuesIn(GetCustomEnvs())); + +INSTANTIATE_TEST_CASE_P(CustomEnv, EnvMoreTestWithParam, + ::testing::ValuesIn(GetCustomEnvs())); + +#endif // ROCKSDB_LITE + +TEST_P(EnvBasicTestWithParam, Basics) { + uint64_t file_size; + unique_ptr writable_file; + std::vector children; + + // Check that the directory is empty. + ASSERT_EQ(Status::NotFound(), env_->FileExists(test_dir_ + "/non_existent")); + ASSERT_TRUE(!env_->GetFileSize(test_dir_ + "/non_existent", &file_size).ok()); + ASSERT_OK(env_->GetChildren(test_dir_, &children)); + ASSERT_EQ(0U, children.size()); + + // Create a file. + ASSERT_OK(env_->NewWritableFile(test_dir_ + "/f", &writable_file, soptions_)); + ASSERT_OK(writable_file->Close()); + writable_file.reset(); + + // Check that the file exists. + ASSERT_OK(env_->FileExists(test_dir_ + "/f")); + ASSERT_OK(env_->GetFileSize(test_dir_ + "/f", &file_size)); + ASSERT_EQ(0U, file_size); + ASSERT_OK(env_->GetChildren(test_dir_, &children)); + ASSERT_EQ(1U, children.size()); + ASSERT_EQ("f", children[0]); + ASSERT_OK(env_->DeleteFile(test_dir_ + "/f")); + + // Write to the file. + ASSERT_OK( + env_->NewWritableFile(test_dir_ + "/f1", &writable_file, soptions_)); + ASSERT_OK(writable_file->Append("abc")); + ASSERT_OK(writable_file->Close()); + writable_file.reset(); + ASSERT_OK( + env_->NewWritableFile(test_dir_ + "/f2", &writable_file, soptions_)); + ASSERT_OK(writable_file->Close()); + writable_file.reset(); + + // Check for expected size. + ASSERT_OK(env_->GetFileSize(test_dir_ + "/f1", &file_size)); + ASSERT_EQ(3U, file_size); + + // Check that renaming works. + ASSERT_TRUE( + !env_->RenameFile(test_dir_ + "/non_existent", test_dir_ + "/g").ok()); + ASSERT_OK(env_->RenameFile(test_dir_ + "/f1", test_dir_ + "/g")); + ASSERT_EQ(Status::NotFound(), env_->FileExists(test_dir_ + "/f1")); + ASSERT_OK(env_->FileExists(test_dir_ + "/g")); + ASSERT_OK(env_->GetFileSize(test_dir_ + "/g", &file_size)); + ASSERT_EQ(3U, file_size); + + // Check that renaming overwriting works + ASSERT_OK(env_->RenameFile(test_dir_ + "/f2", test_dir_ + "/g")); + ASSERT_OK(env_->GetFileSize(test_dir_ + "/g", &file_size)); + ASSERT_EQ(0U, file_size); + + // Check that opening non-existent file fails. + unique_ptr seq_file; + unique_ptr rand_file; + ASSERT_TRUE(!env_->NewSequentialFile(test_dir_ + "/non_existent", &seq_file, + soptions_) + .ok()); + ASSERT_TRUE(!seq_file); + ASSERT_TRUE(!env_->NewRandomAccessFile(test_dir_ + "/non_existent", + &rand_file, soptions_) + .ok()); + ASSERT_TRUE(!rand_file); + + // Check that deleting works. + ASSERT_TRUE(!env_->DeleteFile(test_dir_ + "/non_existent").ok()); + ASSERT_OK(env_->DeleteFile(test_dir_ + "/g")); + ASSERT_EQ(Status::NotFound(), env_->FileExists(test_dir_ + "/g")); + ASSERT_OK(env_->GetChildren(test_dir_, &children)); + ASSERT_EQ(0U, children.size()); + ASSERT_TRUE( + env_->GetChildren(test_dir_ + "/non_existent", &children).IsNotFound()); +} + +TEST_P(EnvBasicTestWithParam, ReadWrite) { + unique_ptr writable_file; + unique_ptr seq_file; + unique_ptr rand_file; + Slice result; + char scratch[100]; + + ASSERT_OK(env_->NewWritableFile(test_dir_ + "/f", &writable_file, soptions_)); + ASSERT_OK(writable_file->Append("hello ")); + ASSERT_OK(writable_file->Append("world")); + ASSERT_OK(writable_file->Close()); + writable_file.reset(); + + // Read sequentially. + ASSERT_OK(env_->NewSequentialFile(test_dir_ + "/f", &seq_file, soptions_)); + ASSERT_OK(seq_file->Read(5, &result, scratch)); // Read "hello". + ASSERT_EQ(0, result.compare("hello")); + ASSERT_OK(seq_file->Skip(1)); + ASSERT_OK(seq_file->Read(1000, &result, scratch)); // Read "world". + ASSERT_EQ(0, result.compare("world")); + ASSERT_OK(seq_file->Read(1000, &result, scratch)); // Try reading past EOF. + ASSERT_EQ(0U, result.size()); + ASSERT_OK(seq_file->Skip(100)); // Try to skip past end of file. + ASSERT_OK(seq_file->Read(1000, &result, scratch)); + ASSERT_EQ(0U, result.size()); + + // Random reads. + ASSERT_OK(env_->NewRandomAccessFile(test_dir_ + "/f", &rand_file, soptions_)); + ASSERT_OK(rand_file->Read(6, 5, &result, scratch)); // Read "world". + ASSERT_EQ(0, result.compare("world")); + ASSERT_OK(rand_file->Read(0, 5, &result, scratch)); // Read "hello". + ASSERT_EQ(0, result.compare("hello")); + ASSERT_OK(rand_file->Read(10, 100, &result, scratch)); // Read "d". + ASSERT_EQ(0, result.compare("d")); + + // Too high offset. + ASSERT_TRUE(rand_file->Read(1000, 5, &result, scratch).ok()); +} + +TEST_P(EnvBasicTestWithParam, Misc) { + unique_ptr writable_file; + ASSERT_OK(env_->NewWritableFile(test_dir_ + "/b", &writable_file, soptions_)); + + // These are no-ops, but we test they return success. + ASSERT_OK(writable_file->Sync()); + ASSERT_OK(writable_file->Flush()); + ASSERT_OK(writable_file->Close()); + writable_file.reset(); +} + +TEST_P(EnvBasicTestWithParam, LargeWrite) { + const size_t kWriteSize = 300 * 1024; + char* scratch = new char[kWriteSize * 2]; + + std::string write_data; + for (size_t i = 0; i < kWriteSize; ++i) { + write_data.append(1, static_cast(i)); + } + + unique_ptr writable_file; + ASSERT_OK(env_->NewWritableFile(test_dir_ + "/f", &writable_file, soptions_)); + ASSERT_OK(writable_file->Append("foo")); + ASSERT_OK(writable_file->Append(write_data)); + ASSERT_OK(writable_file->Close()); + writable_file.reset(); + + unique_ptr seq_file; + Slice result; + ASSERT_OK(env_->NewSequentialFile(test_dir_ + "/f", &seq_file, soptions_)); + ASSERT_OK(seq_file->Read(3, &result, scratch)); // Read "foo". + ASSERT_EQ(0, result.compare("foo")); + + size_t read = 0; + std::string read_data; + while (read < kWriteSize) { + ASSERT_OK(seq_file->Read(kWriteSize - read, &result, scratch)); + read_data.append(result.data(), result.size()); + read += result.size(); + } + ASSERT_TRUE(write_data == read_data); + delete [] scratch; +} + +TEST_P(EnvMoreTestWithParam, GetModTime) { + ASSERT_OK(env_->CreateDirIfMissing(test_dir_ + "/dir1")); + uint64_t mtime1 = 0x0; + ASSERT_OK(env_->GetFileModificationTime(test_dir_ + "/dir1", &mtime1)); +} + +TEST_P(EnvMoreTestWithParam, MakeDir) { + ASSERT_OK(env_->CreateDir(test_dir_ + "/j")); + ASSERT_OK(env_->FileExists(test_dir_ + "/j")); + std::vector children; + env_->GetChildren(test_dir_, &children); + ASSERT_EQ(1U, children.size()); + // fail because file already exists + ASSERT_TRUE(!env_->CreateDir(test_dir_ + "/j").ok()); + ASSERT_OK(env_->CreateDirIfMissing(test_dir_ + "/j")); + ASSERT_OK(env_->DeleteDir(test_dir_ + "/j")); + ASSERT_EQ(Status::NotFound(), env_->FileExists(test_dir_ + "/j")); +} + +TEST_P(EnvMoreTestWithParam, GetChildren) { + // empty folder returns empty vector + std::vector children; + std::vector childAttr; + ASSERT_OK(env_->CreateDirIfMissing(test_dir_)); + ASSERT_OK(env_->GetChildren(test_dir_, &children)); + ASSERT_OK(env_->FileExists(test_dir_)); + ASSERT_OK(env_->GetChildrenFileAttributes(test_dir_, &childAttr)); + ASSERT_EQ(0U, children.size()); + ASSERT_EQ(0U, childAttr.size()); + + // folder with contents returns relative path to test dir + ASSERT_OK(env_->CreateDirIfMissing(test_dir_ + "/niu")); + ASSERT_OK(env_->CreateDirIfMissing(test_dir_ + "/you")); + ASSERT_OK(env_->CreateDirIfMissing(test_dir_ + "/guo")); + ASSERT_OK(env_->GetChildren(test_dir_, &children)); + ASSERT_OK(env_->GetChildrenFileAttributes(test_dir_, &childAttr)); + ASSERT_EQ(3U, children.size()); + ASSERT_EQ(3U, childAttr.size()); + for (auto each : children) { + env_->DeleteDir(test_dir_ + "/" + each); + } // necessary for default POSIX env + + // non-exist directory returns IOError + ASSERT_OK(env_->DeleteDir(test_dir_)); + ASSERT_TRUE(!env_->FileExists(test_dir_).ok()); + ASSERT_TRUE(!env_->GetChildren(test_dir_, &children).ok()); + ASSERT_TRUE(!env_->GetChildrenFileAttributes(test_dir_, &childAttr).ok()); + + // if dir is a file, returns IOError + ASSERT_OK(env_->CreateDir(test_dir_)); + unique_ptr writable_file; + ASSERT_OK( + env_->NewWritableFile(test_dir_ + "/file", &writable_file, soptions_)); + ASSERT_OK(writable_file->Close()); + writable_file.reset(); + ASSERT_TRUE(!env_->GetChildren(test_dir_ + "/file", &children).ok()); + ASSERT_EQ(0U, children.size()); +} + +} // namespace rocksdb +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/env/env_chroot.cc b/env/env_chroot.cc new file mode 100644 index 00000000000..6a1fda8a834 --- /dev/null +++ b/env/env_chroot.cc @@ -0,0 +1,324 @@ +// Copyright (c) 2016-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#if !defined(ROCKSDB_LITE) && !defined(OS_WIN) + +#include "env/env_chroot.h" + +#include +#include +#include +#include + +#include +#include +#include + +#include "rocksdb/status.h" + +namespace rocksdb { + +class ChrootEnv : public EnvWrapper { + public: + ChrootEnv(Env* base_env, const std::string& chroot_dir) + : EnvWrapper(base_env) { +#if defined(OS_AIX) + char resolvedName[PATH_MAX]; + char* real_chroot_dir = realpath(chroot_dir.c_str(), resolvedName); +#else + char* real_chroot_dir = realpath(chroot_dir.c_str(), nullptr); +#endif + // chroot_dir must exist so realpath() returns non-nullptr. + assert(real_chroot_dir != nullptr); + chroot_dir_ = real_chroot_dir; +#if !defined(OS_AIX) + free(real_chroot_dir); +#endif + } + + virtual Status NewSequentialFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) override { + auto status_and_enc_path = EncodePathWithNewBasename(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::NewSequentialFile(status_and_enc_path.second, result, + options); + } + + virtual Status NewRandomAccessFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + auto status_and_enc_path = EncodePathWithNewBasename(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::NewRandomAccessFile(status_and_enc_path.second, result, + options); + } + + virtual Status NewWritableFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + auto status_and_enc_path = EncodePathWithNewBasename(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::NewWritableFile(status_and_enc_path.second, result, + options); + } + + virtual Status ReuseWritableFile(const std::string& fname, + const std::string& old_fname, + unique_ptr* result, + const EnvOptions& options) override { + auto status_and_enc_path = EncodePathWithNewBasename(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + auto status_and_old_enc_path = EncodePath(old_fname); + if (!status_and_old_enc_path.first.ok()) { + return status_and_old_enc_path.first; + } + return EnvWrapper::ReuseWritableFile(status_and_old_enc_path.second, + status_and_old_enc_path.second, result, + options); + } + + virtual Status NewRandomRWFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + auto status_and_enc_path = EncodePathWithNewBasename(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::NewRandomRWFile(status_and_enc_path.second, result, + options); + } + + virtual Status NewDirectory(const std::string& dir, + unique_ptr* result) override { + auto status_and_enc_path = EncodePathWithNewBasename(dir); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::NewDirectory(status_and_enc_path.second, result); + } + + virtual Status FileExists(const std::string& fname) override { + auto status_and_enc_path = EncodePathWithNewBasename(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::FileExists(status_and_enc_path.second); + } + + virtual Status GetChildren(const std::string& dir, + std::vector* result) override { + auto status_and_enc_path = EncodePath(dir); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::GetChildren(status_and_enc_path.second, result); + } + + virtual Status GetChildrenFileAttributes( + const std::string& dir, std::vector* result) override { + auto status_and_enc_path = EncodePath(dir); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::GetChildrenFileAttributes(status_and_enc_path.second, + result); + } + + virtual Status DeleteFile(const std::string& fname) override { + auto status_and_enc_path = EncodePath(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::DeleteFile(status_and_enc_path.second); + } + + virtual Status CreateDir(const std::string& dirname) override { + auto status_and_enc_path = EncodePathWithNewBasename(dirname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::CreateDir(status_and_enc_path.second); + } + + virtual Status CreateDirIfMissing(const std::string& dirname) override { + auto status_and_enc_path = EncodePathWithNewBasename(dirname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::CreateDirIfMissing(status_and_enc_path.second); + } + + virtual Status DeleteDir(const std::string& dirname) override { + auto status_and_enc_path = EncodePath(dirname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::DeleteDir(status_and_enc_path.second); + } + + virtual Status GetFileSize(const std::string& fname, + uint64_t* file_size) override { + auto status_and_enc_path = EncodePath(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::GetFileSize(status_and_enc_path.second, file_size); + } + + virtual Status GetFileModificationTime(const std::string& fname, + uint64_t* file_mtime) override { + auto status_and_enc_path = EncodePath(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::GetFileModificationTime(status_and_enc_path.second, + file_mtime); + } + + virtual Status RenameFile(const std::string& src, + const std::string& dest) override { + auto status_and_src_enc_path = EncodePath(src); + if (!status_and_src_enc_path.first.ok()) { + return status_and_src_enc_path.first; + } + auto status_and_dest_enc_path = EncodePathWithNewBasename(dest); + if (!status_and_dest_enc_path.first.ok()) { + return status_and_dest_enc_path.first; + } + return EnvWrapper::RenameFile(status_and_src_enc_path.second, + status_and_dest_enc_path.second); + } + + virtual Status LinkFile(const std::string& src, + const std::string& dest) override { + auto status_and_src_enc_path = EncodePath(src); + if (!status_and_src_enc_path.first.ok()) { + return status_and_src_enc_path.first; + } + auto status_and_dest_enc_path = EncodePathWithNewBasename(dest); + if (!status_and_dest_enc_path.first.ok()) { + return status_and_dest_enc_path.first; + } + return EnvWrapper::LinkFile(status_and_src_enc_path.second, + status_and_dest_enc_path.second); + } + + virtual Status LockFile(const std::string& fname, FileLock** lock) override { + auto status_and_enc_path = EncodePathWithNewBasename(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + // FileLock subclasses may store path (e.g., PosixFileLock stores it). We + // can skip stripping the chroot directory from this path because callers + // shouldn't use it. + return EnvWrapper::LockFile(status_and_enc_path.second, lock); + } + + virtual Status GetTestDirectory(std::string* path) override { + // Adapted from PosixEnv's implementation since it doesn't provide a way to + // create directory in the chroot. + char buf[256]; + snprintf(buf, sizeof(buf), "/rocksdbtest-%d", static_cast(geteuid())); + *path = buf; + + // Directory may already exist, so ignore return + CreateDir(*path); + return Status::OK(); + } + + virtual Status NewLogger(const std::string& fname, + shared_ptr* result) override { + auto status_and_enc_path = EncodePathWithNewBasename(fname); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::NewLogger(status_and_enc_path.second, result); + } + + virtual Status GetAbsolutePath(const std::string& db_path, + std::string* output_path) override { + auto status_and_enc_path = EncodePath(db_path); + if (!status_and_enc_path.first.ok()) { + return status_and_enc_path.first; + } + return EnvWrapper::GetAbsolutePath(status_and_enc_path.second, output_path); + } + + private: + // Returns status and expanded absolute path including the chroot directory. + // Checks whether the provided path breaks out of the chroot. If it returns + // non-OK status, the returned path should not be used. + std::pair EncodePath(const std::string& path) { + if (path.empty() || path[0] != '/') { + return {Status::InvalidArgument(path, "Not an absolute path"), ""}; + } + std::pair res; + res.second = chroot_dir_ + path; +#if defined(OS_AIX) + char resolvedName[PATH_MAX]; + char* normalized_path = realpath(res.second.c_str(), resolvedName); +#else + char* normalized_path = realpath(res.second.c_str(), nullptr); +#endif + if (normalized_path == nullptr) { + res.first = Status::NotFound(res.second, strerror(errno)); + } else if (strlen(normalized_path) < chroot_dir_.size() || + strncmp(normalized_path, chroot_dir_.c_str(), + chroot_dir_.size()) != 0) { + res.first = Status::IOError(res.second, + "Attempted to access path outside chroot"); + } else { + res.first = Status::OK(); + } +#if !defined(OS_AIX) + free(normalized_path); +#endif + return res; + } + + // Similar to EncodePath() except assumes the basename in the path hasn't been + // created yet. + std::pair EncodePathWithNewBasename( + const std::string& path) { + if (path.empty() || path[0] != '/') { + return {Status::InvalidArgument(path, "Not an absolute path"), ""}; + } + // Basename may be followed by trailing slashes + size_t final_idx = path.find_last_not_of('/'); + if (final_idx == std::string::npos) { + // It's only slashes so no basename to extract + return EncodePath(path); + } + + // Pull off the basename temporarily since realname(3) (used by + // EncodePath()) requires a path that exists + size_t base_sep = path.rfind('/', final_idx); + auto status_and_enc_path = EncodePath(path.substr(0, base_sep + 1)); + status_and_enc_path.second.append(path.substr(base_sep + 1)); + return status_and_enc_path; + } + + std::string chroot_dir_; +}; + +Env* NewChrootEnv(Env* base_env, const std::string& chroot_dir) { + if (!base_env->FileExists(chroot_dir).ok()) { + return nullptr; + } + return new ChrootEnv(base_env, chroot_dir); +} + +} // namespace rocksdb + +#endif // !defined(ROCKSDB_LITE) && !defined(OS_WIN) diff --git a/env/env_chroot.h b/env/env_chroot.h new file mode 100644 index 00000000000..b2760bc0a3c --- /dev/null +++ b/env/env_chroot.h @@ -0,0 +1,22 @@ +// Copyright (c) 2016-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#if !defined(ROCKSDB_LITE) && !defined(OS_WIN) + +#include + +#include "rocksdb/env.h" + +namespace rocksdb { + +// Returns an Env that translates paths such that the root directory appears to +// be chroot_dir. chroot_dir should refer to an existing directory. +Env* NewChrootEnv(Env* base_env, const std::string& chroot_dir); + +} // namespace rocksdb + +#endif // !defined(ROCKSDB_LITE) && !defined(OS_WIN) diff --git a/env/env_encryption.cc b/env/env_encryption.cc new file mode 100644 index 00000000000..6b688a66020 --- /dev/null +++ b/env/env_encryption.cc @@ -0,0 +1,909 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include +#include +#include + +#include "rocksdb/env_encryption.h" +#include "util/aligned_buffer.h" +#include "util/coding.h" +#include "util/random.h" + +#endif + +namespace rocksdb { + +#ifndef ROCKSDB_LITE + +class EncryptedSequentialFile : public SequentialFile { + private: + std::unique_ptr file_; + std::unique_ptr stream_; + uint64_t offset_; + size_t prefixLength_; + + public: + // Default ctor. Given underlying sequential file is supposed to be at + // offset == prefixLength. + EncryptedSequentialFile(SequentialFile* f, BlockAccessCipherStream* s, size_t prefixLength) + : file_(f), stream_(s), offset_(prefixLength), prefixLength_(prefixLength) { + } + + // Read up to "n" bytes from the file. "scratch[0..n-1]" may be + // written by this routine. Sets "*result" to the data that was + // read (including if fewer than "n" bytes were successfully read). + // May set "*result" to point at data in "scratch[0..n-1]", so + // "scratch[0..n-1]" must be live when "*result" is used. + // If an error was encountered, returns a non-OK status. + // + // REQUIRES: External synchronization + virtual Status Read(size_t n, Slice* result, char* scratch) override { + assert(scratch); + Status status = file_->Read(n, result, scratch); + if (!status.ok()) { + return status; + } + status = stream_->Decrypt(offset_, (char*)result->data(), result->size()); + offset_ += result->size(); // We've already ready data from disk, so update offset_ even if decryption fails. + return status; + } + + // Skip "n" bytes from the file. This is guaranteed to be no + // slower that reading the same data, but may be faster. + // + // If end of file is reached, skipping will stop at the end of the + // file, and Skip will return OK. + // + // REQUIRES: External synchronization + virtual Status Skip(uint64_t n) override { + auto status = file_->Skip(n); + if (!status.ok()) { + return status; + } + offset_ += n; + return status; + } + + // Indicates the upper layers if the current SequentialFile implementation + // uses direct IO. + virtual bool use_direct_io() const override { + return file_->use_direct_io(); + } + + // Use the returned alignment value to allocate + // aligned buffer for Direct I/O + virtual size_t GetRequiredBufferAlignment() const override { + return file_->GetRequiredBufferAlignment(); + } + + // Remove any kind of caching of data from the offset to offset+length + // of this file. If the length is 0, then it refers to the end of file. + // If the system is not caching the file contents, then this is a noop. + virtual Status InvalidateCache(size_t offset, size_t length) override { + return file_->InvalidateCache(offset + prefixLength_, length); + } + + // Positioned Read for direct I/O + // If Direct I/O enabled, offset, n, and scratch should be properly aligned + virtual Status PositionedRead(uint64_t offset, size_t n, Slice* result, char* scratch) override { + assert(scratch); + offset += prefixLength_; // Skip prefix + auto status = file_->PositionedRead(offset, n, result, scratch); + if (!status.ok()) { + return status; + } + offset_ = offset + result->size(); + status = stream_->Decrypt(offset, (char*)result->data(), result->size()); + return status; + } + +}; + +// A file abstraction for randomly reading the contents of a file. +class EncryptedRandomAccessFile : public RandomAccessFile { + private: + std::unique_ptr file_; + std::unique_ptr stream_; + size_t prefixLength_; + + public: + EncryptedRandomAccessFile(RandomAccessFile* f, BlockAccessCipherStream* s, size_t prefixLength) + : file_(f), stream_(s), prefixLength_(prefixLength) { } + + // Read up to "n" bytes from the file starting at "offset". + // "scratch[0..n-1]" may be written by this routine. Sets "*result" + // to the data that was read (including if fewer than "n" bytes were + // successfully read). May set "*result" to point at data in + // "scratch[0..n-1]", so "scratch[0..n-1]" must be live when + // "*result" is used. If an error was encountered, returns a non-OK + // status. + // + // Safe for concurrent use by multiple threads. + // If Direct I/O enabled, offset, n, and scratch should be aligned properly. + virtual Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const override { + assert(scratch); + offset += prefixLength_; + auto status = file_->Read(offset, n, result, scratch); + if (!status.ok()) { + return status; + } + status = stream_->Decrypt(offset, (char*)result->data(), result->size()); + return status; + } + + // Readahead the file starting from offset by n bytes for caching. + virtual Status Prefetch(uint64_t offset, size_t n) override { + //return Status::OK(); + return file_->Prefetch(offset + prefixLength_, n); + } + + // Tries to get an unique ID for this file that will be the same each time + // the file is opened (and will stay the same while the file is open). + // Furthermore, it tries to make this ID at most "max_size" bytes. If such an + // ID can be created this function returns the length of the ID and places it + // in "id"; otherwise, this function returns 0, in which case "id" + // may not have been modified. + // + // This function guarantees, for IDs from a given environment, two unique ids + // cannot be made equal to eachother by adding arbitrary bytes to one of + // them. That is, no unique ID is the prefix of another. + // + // This function guarantees that the returned ID will not be interpretable as + // a single varint. + // + // Note: these IDs are only valid for the duration of the process. + virtual size_t GetUniqueId(char* id, size_t max_size) const override { + return file_->GetUniqueId(id, max_size); + }; + + virtual void Hint(AccessPattern pattern) override { + file_->Hint(pattern); + } + + // Indicates the upper layers if the current RandomAccessFile implementation + // uses direct IO. + virtual bool use_direct_io() const override { + return file_->use_direct_io(); + } + + // Use the returned alignment value to allocate + // aligned buffer for Direct I/O + virtual size_t GetRequiredBufferAlignment() const override { + return file_->GetRequiredBufferAlignment(); + } + + // Remove any kind of caching of data from the offset to offset+length + // of this file. If the length is 0, then it refers to the end of file. + // If the system is not caching the file contents, then this is a noop. + virtual Status InvalidateCache(size_t offset, size_t length) override { + return file_->InvalidateCache(offset + prefixLength_, length); + } +}; + +// A file abstraction for sequential writing. The implementation +// must provide buffering since callers may append small fragments +// at a time to the file. +class EncryptedWritableFile : public WritableFileWrapper { + private: + std::unique_ptr file_; + std::unique_ptr stream_; + size_t prefixLength_; + + public: + // Default ctor. Prefix is assumed to be written already. + EncryptedWritableFile(WritableFile* f, BlockAccessCipherStream* s, size_t prefixLength) + : WritableFileWrapper(f), file_(f), stream_(s), prefixLength_(prefixLength) { } + + Status Append(const Slice& data) override { + AlignedBuffer buf; + Status status; + Slice dataToAppend(data); + if (data.size() > 0) { + auto offset = file_->GetFileSize(); // size including prefix + // Encrypt in cloned buffer + buf.Alignment(GetRequiredBufferAlignment()); + buf.AllocateNewBuffer(data.size()); + memmove(buf.BufferStart(), data.data(), data.size()); + status = stream_->Encrypt(offset, buf.BufferStart(), data.size()); + if (!status.ok()) { + return status; + } + dataToAppend = Slice(buf.BufferStart(), data.size()); + } + status = file_->Append(dataToAppend); + if (!status.ok()) { + return status; + } + return status; + } + + Status PositionedAppend(const Slice& data, uint64_t offset) override { + AlignedBuffer buf; + Status status; + Slice dataToAppend(data); + offset += prefixLength_; + if (data.size() > 0) { + // Encrypt in cloned buffer + buf.Alignment(GetRequiredBufferAlignment()); + buf.AllocateNewBuffer(data.size()); + memmove(buf.BufferStart(), data.data(), data.size()); + status = stream_->Encrypt(offset, buf.BufferStart(), data.size()); + if (!status.ok()) { + return status; + } + dataToAppend = Slice(buf.BufferStart(), data.size()); + } + status = file_->PositionedAppend(dataToAppend, offset); + if (!status.ok()) { + return status; + } + return status; + } + + // Indicates the upper layers if the current WritableFile implementation + // uses direct IO. + virtual bool use_direct_io() const override { return file_->use_direct_io(); } + + // Use the returned alignment value to allocate + // aligned buffer for Direct I/O + virtual size_t GetRequiredBufferAlignment() const override { return file_->GetRequiredBufferAlignment(); } + + /* + * Get the size of valid data in the file. + */ + virtual uint64_t GetFileSize() override { + return file_->GetFileSize() - prefixLength_; + } + + // Truncate is necessary to trim the file to the correct size + // before closing. It is not always possible to keep track of the file + // size due to whole pages writes. The behavior is undefined if called + // with other writes to follow. + virtual Status Truncate(uint64_t size) override { + return file_->Truncate(size + prefixLength_); + } + + // Remove any kind of caching of data from the offset to offset+length + // of this file. If the length is 0, then it refers to the end of file. + // If the system is not caching the file contents, then this is a noop. + // This call has no effect on dirty pages in the cache. + virtual Status InvalidateCache(size_t offset, size_t length) override { + return file_->InvalidateCache(offset + prefixLength_, length); + } + + // Sync a file range with disk. + // offset is the starting byte of the file range to be synchronized. + // nbytes specifies the length of the range to be synchronized. + // This asks the OS to initiate flushing the cached data to disk, + // without waiting for completion. + // Default implementation does nothing. + virtual Status RangeSync(uint64_t offset, uint64_t nbytes) override { + return file_->RangeSync(offset + prefixLength_, nbytes); + } + + // PrepareWrite performs any necessary preparation for a write + // before the write actually occurs. This allows for pre-allocation + // of space on devices where it can result in less file + // fragmentation and/or less waste from over-zealous filesystem + // pre-allocation. + virtual void PrepareWrite(size_t offset, size_t len) override { + file_->PrepareWrite(offset + prefixLength_, len); + } + + // Pre-allocates space for a file. + virtual Status Allocate(uint64_t offset, uint64_t len) override { + return file_->Allocate(offset + prefixLength_, len); + } +}; + +// A file abstraction for random reading and writing. +class EncryptedRandomRWFile : public RandomRWFile { + private: + std::unique_ptr file_; + std::unique_ptr stream_; + size_t prefixLength_; + + public: + EncryptedRandomRWFile(RandomRWFile* f, BlockAccessCipherStream* s, size_t prefixLength) + : file_(f), stream_(s), prefixLength_(prefixLength) {} + + // Indicates if the class makes use of direct I/O + // If false you must pass aligned buffer to Write() + virtual bool use_direct_io() const override { return file_->use_direct_io(); } + + // Use the returned alignment value to allocate + // aligned buffer for Direct I/O + virtual size_t GetRequiredBufferAlignment() const override { + return file_->GetRequiredBufferAlignment(); + } + + // Write bytes in `data` at offset `offset`, Returns Status::OK() on success. + // Pass aligned buffer when use_direct_io() returns true. + virtual Status Write(uint64_t offset, const Slice& data) override { + AlignedBuffer buf; + Status status; + Slice dataToWrite(data); + offset += prefixLength_; + if (data.size() > 0) { + // Encrypt in cloned buffer + buf.Alignment(GetRequiredBufferAlignment()); + buf.AllocateNewBuffer(data.size()); + memmove(buf.BufferStart(), data.data(), data.size()); + status = stream_->Encrypt(offset, buf.BufferStart(), data.size()); + if (!status.ok()) { + return status; + } + dataToWrite = Slice(buf.BufferStart(), data.size()); + } + status = file_->Write(offset, dataToWrite); + return status; + } + + // Read up to `n` bytes starting from offset `offset` and store them in + // result, provided `scratch` size should be at least `n`. + // Returns Status::OK() on success. + virtual Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const override { + assert(scratch); + offset += prefixLength_; + auto status = file_->Read(offset, n, result, scratch); + if (!status.ok()) { + return status; + } + status = stream_->Decrypt(offset, (char*)result->data(), result->size()); + return status; + } + + virtual Status Flush() override { + return file_->Flush(); + } + + virtual Status Sync() override { + return file_->Sync(); + } + + virtual Status Fsync() override { + return file_->Fsync(); + } + + virtual Status Close() override { + return file_->Close(); + } +}; + +// EncryptedEnv implements an Env wrapper that adds encryption to files stored on disk. +class EncryptedEnv : public EnvWrapper { + public: + EncryptedEnv(Env* base_env, EncryptionProvider *provider) + : EnvWrapper(base_env) { + provider_ = provider; + } + + // NewSequentialFile opens a file for sequential reading. + virtual Status NewSequentialFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) override { + result->reset(); + if (options.use_mmap_reads) { + return Status::InvalidArgument(); + } + // Open file using underlying Env implementation + std::unique_ptr underlying; + auto status = EnvWrapper::NewSequentialFile(fname, &underlying, options); + if (!status.ok()) { + return status; + } + // Read prefix (if needed) + AlignedBuffer prefixBuf; + Slice prefixSlice; + size_t prefixLength = provider_->GetPrefixLength(); + if (prefixLength > 0) { + // Read prefix + prefixBuf.Alignment(underlying->GetRequiredBufferAlignment()); + prefixBuf.AllocateNewBuffer(prefixLength); + status = underlying->Read(prefixLength, &prefixSlice, prefixBuf.BufferStart()); + if (!status.ok()) { + return status; + } + } + // Create cipher stream + std::unique_ptr stream; + status = provider_->CreateCipherStream(fname, options, prefixSlice, &stream); + if (!status.ok()) { + return status; + } + (*result) = std::unique_ptr(new EncryptedSequentialFile(underlying.release(), stream.release(), prefixLength)); + return Status::OK(); + } + + // NewRandomAccessFile opens a file for random read access. + virtual Status NewRandomAccessFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + result->reset(); + if (options.use_mmap_reads) { + return Status::InvalidArgument(); + } + // Open file using underlying Env implementation + std::unique_ptr underlying; + auto status = EnvWrapper::NewRandomAccessFile(fname, &underlying, options); + if (!status.ok()) { + return status; + } + // Read prefix (if needed) + AlignedBuffer prefixBuf; + Slice prefixSlice; + size_t prefixLength = provider_->GetPrefixLength(); + if (prefixLength > 0) { + // Read prefix + prefixBuf.Alignment(underlying->GetRequiredBufferAlignment()); + prefixBuf.AllocateNewBuffer(prefixLength); + status = underlying->Read(0, prefixLength, &prefixSlice, prefixBuf.BufferStart()); + if (!status.ok()) { + return status; + } + } + // Create cipher stream + std::unique_ptr stream; + status = provider_->CreateCipherStream(fname, options, prefixSlice, &stream); + if (!status.ok()) { + return status; + } + (*result) = std::unique_ptr(new EncryptedRandomAccessFile(underlying.release(), stream.release(), prefixLength)); + return Status::OK(); + } + + // NewWritableFile opens a file for sequential writing. + virtual Status NewWritableFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + result->reset(); + if (options.use_mmap_writes) { + return Status::InvalidArgument(); + } + // Open file using underlying Env implementation + std::unique_ptr underlying; + Status status = EnvWrapper::NewWritableFile(fname, &underlying, options); + if (!status.ok()) { + return status; + } + // Initialize & write prefix (if needed) + AlignedBuffer prefixBuf; + Slice prefixSlice; + size_t prefixLength = provider_->GetPrefixLength(); + if (prefixLength > 0) { + // Initialize prefix + prefixBuf.Alignment(underlying->GetRequiredBufferAlignment()); + prefixBuf.AllocateNewBuffer(prefixLength); + provider_->CreateNewPrefix(fname, prefixBuf.BufferStart(), prefixLength); + prefixSlice = Slice(prefixBuf.BufferStart(), prefixLength); + // Write prefix + status = underlying->Append(prefixSlice); + if (!status.ok()) { + return status; + } + } + // Create cipher stream + std::unique_ptr stream; + status = provider_->CreateCipherStream(fname, options, prefixSlice, &stream); + if (!status.ok()) { + return status; + } + (*result) = std::unique_ptr(new EncryptedWritableFile(underlying.release(), stream.release(), prefixLength)); + return Status::OK(); + } + + // Create an object that writes to a new file with the specified + // name. Deletes any existing file with the same name and creates a + // new file. On success, stores a pointer to the new file in + // *result and returns OK. On failure stores nullptr in *result and + // returns non-OK. + // + // The returned file will only be accessed by one thread at a time. + virtual Status ReopenWritableFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + result->reset(); + if (options.use_mmap_writes) { + return Status::InvalidArgument(); + } + // Open file using underlying Env implementation + std::unique_ptr underlying; + Status status = EnvWrapper::ReopenWritableFile(fname, &underlying, options); + if (!status.ok()) { + return status; + } + // Initialize & write prefix (if needed) + AlignedBuffer prefixBuf; + Slice prefixSlice; + size_t prefixLength = provider_->GetPrefixLength(); + if (prefixLength > 0) { + // Initialize prefix + prefixBuf.Alignment(underlying->GetRequiredBufferAlignment()); + prefixBuf.AllocateNewBuffer(prefixLength); + provider_->CreateNewPrefix(fname, prefixBuf.BufferStart(), prefixLength); + prefixSlice = Slice(prefixBuf.BufferStart(), prefixLength); + // Write prefix + status = underlying->Append(prefixSlice); + if (!status.ok()) { + return status; + } + } + // Create cipher stream + std::unique_ptr stream; + status = provider_->CreateCipherStream(fname, options, prefixSlice, &stream); + if (!status.ok()) { + return status; + } + (*result) = std::unique_ptr(new EncryptedWritableFile(underlying.release(), stream.release(), prefixLength)); + return Status::OK(); + } + + // Reuse an existing file by renaming it and opening it as writable. + virtual Status ReuseWritableFile(const std::string& fname, + const std::string& old_fname, + unique_ptr* result, + const EnvOptions& options) override { + result->reset(); + if (options.use_mmap_writes) { + return Status::InvalidArgument(); + } + // Open file using underlying Env implementation + std::unique_ptr underlying; + Status status = EnvWrapper::ReuseWritableFile(fname, old_fname, &underlying, options); + if (!status.ok()) { + return status; + } + // Initialize & write prefix (if needed) + AlignedBuffer prefixBuf; + Slice prefixSlice; + size_t prefixLength = provider_->GetPrefixLength(); + if (prefixLength > 0) { + // Initialize prefix + prefixBuf.Alignment(underlying->GetRequiredBufferAlignment()); + prefixBuf.AllocateNewBuffer(prefixLength); + provider_->CreateNewPrefix(fname, prefixBuf.BufferStart(), prefixLength); + prefixSlice = Slice(prefixBuf.BufferStart(), prefixLength); + // Write prefix + status = underlying->Append(prefixSlice); + if (!status.ok()) { + return status; + } + } + // Create cipher stream + std::unique_ptr stream; + status = provider_->CreateCipherStream(fname, options, prefixSlice, &stream); + if (!status.ok()) { + return status; + } + (*result) = std::unique_ptr(new EncryptedWritableFile(underlying.release(), stream.release(), prefixLength)); + return Status::OK(); + } + + // Open `fname` for random read and write, if file dont exist the file + // will be created. On success, stores a pointer to the new file in + // *result and returns OK. On failure returns non-OK. + // + // The returned file will only be accessed by one thread at a time. + virtual Status NewRandomRWFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + result->reset(); + if (options.use_mmap_reads || options.use_mmap_writes) { + return Status::InvalidArgument(); + } + // Check file exists + bool isNewFile = !FileExists(fname).ok(); + + // Open file using underlying Env implementation + std::unique_ptr underlying; + Status status = EnvWrapper::NewRandomRWFile(fname, &underlying, options); + if (!status.ok()) { + return status; + } + // Read or Initialize & write prefix (if needed) + AlignedBuffer prefixBuf; + Slice prefixSlice; + size_t prefixLength = provider_->GetPrefixLength(); + if (prefixLength > 0) { + prefixBuf.Alignment(underlying->GetRequiredBufferAlignment()); + prefixBuf.AllocateNewBuffer(prefixLength); + if (!isNewFile) { + // File already exists, read prefix + status = underlying->Read(0, prefixLength, &prefixSlice, prefixBuf.BufferStart()); + if (!status.ok()) { + return status; + } + } else { + // File is new, initialize & write prefix + provider_->CreateNewPrefix(fname, prefixBuf.BufferStart(), prefixLength); + prefixSlice = Slice(prefixBuf.BufferStart(), prefixLength); + // Write prefix + status = underlying->Write(0, prefixSlice); + if (!status.ok()) { + return status; + } + } + } + // Create cipher stream + std::unique_ptr stream; + status = provider_->CreateCipherStream(fname, options, prefixSlice, &stream); + if (!status.ok()) { + return status; + } + (*result) = std::unique_ptr(new EncryptedRandomRWFile(underlying.release(), stream.release(), prefixLength)); + return Status::OK(); + } + + // Store in *result the attributes of the children of the specified directory. + // In case the implementation lists the directory prior to iterating the files + // and files are concurrently deleted, the deleted files will be omitted from + // result. + // The name attributes are relative to "dir". + // Original contents of *results are dropped. + // Returns OK if "dir" exists and "*result" contains its children. + // NotFound if "dir" does not exist, the calling process does not have + // permission to access "dir", or if "dir" is invalid. + // IOError if an IO Error was encountered + virtual Status GetChildrenFileAttributes(const std::string& dir, std::vector* result) override { + auto status = EnvWrapper::GetChildrenFileAttributes(dir, result); + if (!status.ok()) { + return status; + } + size_t prefixLength = provider_->GetPrefixLength(); + for (auto it = std::begin(*result); it!=std::end(*result); ++it) { + assert(it->size_bytes >= prefixLength); + it->size_bytes -= prefixLength; + } + return Status::OK(); + } + + // Store the size of fname in *file_size. + virtual Status GetFileSize(const std::string& fname, uint64_t* file_size) override { + auto status = EnvWrapper::GetFileSize(fname, file_size); + if (!status.ok()) { + return status; + } + size_t prefixLength = provider_->GetPrefixLength(); + assert(*file_size >= prefixLength); + *file_size -= prefixLength; + return Status::OK(); + } + + private: + EncryptionProvider *provider_; +}; + + +// Returns an Env that encrypts data when stored on disk and decrypts data when +// read from disk. +Env* NewEncryptedEnv(Env* base_env, EncryptionProvider* provider) { + return new EncryptedEnv(base_env, provider); +} + +// Encrypt one or more (partial) blocks of data at the file offset. +// Length of data is given in dataSize. +Status BlockAccessCipherStream::Encrypt(uint64_t fileOffset, char *data, size_t dataSize) { + // Calculate block index + auto blockSize = BlockSize(); + uint64_t blockIndex = fileOffset / blockSize; + size_t blockOffset = fileOffset % blockSize; + unique_ptr blockBuffer; + + std::string scratch; + AllocateScratch(scratch); + + // Encrypt individual blocks. + while (1) { + char *block = data; + size_t n = std::min(dataSize, blockSize - blockOffset); + if (n != blockSize) { + // We're not encrypting a full block. + // Copy data to blockBuffer + if (!blockBuffer.get()) { + // Allocate buffer + blockBuffer = unique_ptr(new char[blockSize]); + } + block = blockBuffer.get(); + // Copy plain data to block buffer + memmove(block + blockOffset, data, n); + } + auto status = EncryptBlock(blockIndex, block, (char*)scratch.data()); + if (!status.ok()) { + return status; + } + if (block != data) { + // Copy encrypted data back to `data`. + memmove(data, block + blockOffset, n); + } + dataSize -= n; + if (dataSize == 0) { + return Status::OK(); + } + data += n; + blockOffset = 0; + blockIndex++; + } +} + +// Decrypt one or more (partial) blocks of data at the file offset. +// Length of data is given in dataSize. +Status BlockAccessCipherStream::Decrypt(uint64_t fileOffset, char *data, size_t dataSize) { + // Calculate block index + auto blockSize = BlockSize(); + uint64_t blockIndex = fileOffset / blockSize; + size_t blockOffset = fileOffset % blockSize; + unique_ptr blockBuffer; + + std::string scratch; + AllocateScratch(scratch); + + // Decrypt individual blocks. + while (1) { + char *block = data; + size_t n = std::min(dataSize, blockSize - blockOffset); + if (n != blockSize) { + // We're not decrypting a full block. + // Copy data to blockBuffer + if (!blockBuffer.get()) { + // Allocate buffer + blockBuffer = unique_ptr(new char[blockSize]); + } + block = blockBuffer.get(); + // Copy encrypted data to block buffer + memmove(block + blockOffset, data, n); + } + auto status = DecryptBlock(blockIndex, block, (char*)scratch.data()); + if (!status.ok()) { + return status; + } + if (block != data) { + // Copy decrypted data back to `data`. + memmove(data, block + blockOffset, n); + } + dataSize -= n; + if (dataSize == 0) { + return Status::OK(); + } + data += n; + blockOffset = 0; + blockIndex++; + } +} + +// Encrypt a block of data. +// Length of data is equal to BlockSize(). +Status ROT13BlockCipher::Encrypt(char *data) { + for (size_t i = 0; i < blockSize_; ++i) { + data[i] += 13; + } + return Status::OK(); +} + +// Decrypt a block of data. +// Length of data is equal to BlockSize(). +Status ROT13BlockCipher::Decrypt(char *data) { + return Encrypt(data); +} + +// Allocate scratch space which is passed to EncryptBlock/DecryptBlock. +void CTRCipherStream::AllocateScratch(std::string& scratch) { + auto blockSize = cipher_.BlockSize(); + scratch.reserve(blockSize); +} + +// Encrypt a block of data at the given block index. +// Length of data is equal to BlockSize(); +Status CTRCipherStream::EncryptBlock(uint64_t blockIndex, char *data, char* scratch) { + + // Create nonce + counter + auto blockSize = cipher_.BlockSize(); + memmove(scratch, iv_.data(), blockSize); + EncodeFixed64(scratch, blockIndex + initialCounter_); + + // Encrypt nonce+counter + auto status = cipher_.Encrypt(scratch); + if (!status.ok()) { + return status; + } + + // XOR data with ciphertext. + for (size_t i = 0; i < blockSize; i++) { + data[i] = data[i] ^ scratch[i]; + } + return Status::OK(); +} + +// Decrypt a block of data at the given block index. +// Length of data is equal to BlockSize(); +Status CTRCipherStream::DecryptBlock(uint64_t blockIndex, char *data, char* scratch) { + // For CTR decryption & encryption are the same + return EncryptBlock(blockIndex, data, scratch); +} + +// GetPrefixLength returns the length of the prefix that is added to every file +// and used for storing encryption options. +// For optimal performance, the prefix length should be a multiple of +// the a page size. +size_t CTREncryptionProvider::GetPrefixLength() { + return defaultPrefixLength; +} + +// decodeCTRParameters decodes the initial counter & IV from the given +// (plain text) prefix. +static void decodeCTRParameters(const char *prefix, size_t blockSize, uint64_t &initialCounter, Slice &iv) { + // First block contains 64-bit initial counter + initialCounter = DecodeFixed64(prefix); + // Second block contains IV + iv = Slice(prefix + blockSize, blockSize); +} + +// CreateNewPrefix initialized an allocated block of prefix memory +// for a new file. +Status CTREncryptionProvider::CreateNewPrefix(const std::string& fname, char *prefix, size_t prefixLength) { + // Create & seed rnd. + Random rnd((uint32_t)Env::Default()->NowMicros()); + // Fill entire prefix block with random values. + for (size_t i = 0; i < prefixLength; i++) { + prefix[i] = rnd.Uniform(256) & 0xFF; + } + // Take random data to extract initial counter & IV + auto blockSize = cipher_.BlockSize(); + uint64_t initialCounter; + Slice prefixIV; + decodeCTRParameters(prefix, blockSize, initialCounter, prefixIV); + + // Now populate the rest of the prefix, starting from the third block. + PopulateSecretPrefixPart(prefix + (2 * blockSize), prefixLength - (2 * blockSize), blockSize); + + // Encrypt the prefix, starting from block 2 (leave block 0, 1 with initial counter & IV unencrypted) + CTRCipherStream cipherStream(cipher_, prefixIV.data(), initialCounter); + auto status = cipherStream.Encrypt(0, prefix + (2 * blockSize), prefixLength - (2 * blockSize)); + if (!status.ok()) { + return status; + } + return Status::OK(); +} + +// PopulateSecretPrefixPart initializes the data into a new prefix block +// in plain text. +// Returns the amount of space (starting from the start of the prefix) +// that has been initialized. +size_t CTREncryptionProvider::PopulateSecretPrefixPart(char *prefix, size_t prefixLength, size_t blockSize) { + // Nothing to do here, put in custom data in override when needed. + return 0; +} + +Status CTREncryptionProvider::CreateCipherStream(const std::string& fname, const EnvOptions& options, Slice &prefix, unique_ptr* result) { + // Read plain text part of prefix. + auto blockSize = cipher_.BlockSize(); + uint64_t initialCounter; + Slice iv; + decodeCTRParameters(prefix.data(), blockSize, initialCounter, iv); + + // Decrypt the encrypted part of the prefix, starting from block 2 (block 0, 1 with initial counter & IV are unencrypted) + CTRCipherStream cipherStream(cipher_, iv.data(), initialCounter); + auto status = cipherStream.Decrypt(0, (char*)prefix.data() + (2 * blockSize), prefix.size() - (2 * blockSize)); + if (!status.ok()) { + return status; + } + + // Create cipher stream + return CreateCipherStreamFromPrefix(fname, options, initialCounter, iv, prefix, result); +} + +// CreateCipherStreamFromPrefix creates a block access cipher stream for a file given +// given name and options. The given prefix is already decrypted. +Status CTREncryptionProvider::CreateCipherStreamFromPrefix(const std::string& fname, const EnvOptions& options, + uint64_t initialCounter, const Slice& iv, const Slice& prefix, unique_ptr* result) { + (*result) = unique_ptr(new CTRCipherStream(cipher_, iv.data(), initialCounter)); + return Status::OK(); +} + +#endif // ROCKSDB_LITE + +} // namespace rocksdb diff --git a/util/env_hdfs.cc b/env/env_hdfs.cc similarity index 77% rename from util/env_hdfs.cc rename to env/env_hdfs.cc index 1618e54685a..d98020c76b3 100644 --- a/util/env_hdfs.cc +++ b/env/env_hdfs.cc @@ -1,8 +1,12 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // + +#include "rocksdb/env.h" +#include "hdfs/env_hdfs.h" + #ifdef USE_HDFS #ifndef ROCKSDB_HDFS_FILE_C #define ROCKSDB_HDFS_FILE_C @@ -13,9 +17,8 @@ #include #include #include -#include "rocksdb/env.h" #include "rocksdb/status.h" -#include "hdfs/env_hdfs.h" +#include "util/string_util.h" #define HDFS_EXISTS 0 #define HDFS_DOESNT_EXIST -1 @@ -33,7 +36,9 @@ namespace { // Log error message static Status IOError(const std::string& context, int err_number) { - return Status::IOError(context, strerror(err_number)); + return (err_number == ENOSPC) ? + Status::NoSpace(context, strerror(err_number)) : + Status::IOError(context, strerror(err_number)); } // assume that there is one global logger for now. It is not thread-safe, @@ -52,19 +57,20 @@ class HdfsReadableFile : virtual public SequentialFile, public: HdfsReadableFile(hdfsFS fileSys, const std::string& fname) : fileSys_(fileSys), filename_(fname), hfile_(nullptr) { - Log(mylog, "[hdfs] HdfsReadableFile opening file %s\n", - filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsReadableFile opening file %s\n", + filename_.c_str()); hfile_ = hdfsOpenFile(fileSys_, filename_.c_str(), O_RDONLY, 0, 0, 0); - Log(mylog, "[hdfs] HdfsReadableFile opened file %s hfile_=0x%p\n", - filename_.c_str(), hfile_); + ROCKS_LOG_DEBUG(mylog, + "[hdfs] HdfsReadableFile opened file %s hfile_=0x%p\n", + filename_.c_str(), hfile_); } virtual ~HdfsReadableFile() { - Log(mylog, "[hdfs] HdfsReadableFile closing file %s\n", - filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsReadableFile closing file %s\n", + filename_.c_str()); hdfsCloseFile(fileSys_, hfile_); - Log(mylog, "[hdfs] HdfsReadableFile closed file %s\n", - filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsReadableFile closed file %s\n", + filename_.c_str()); hfile_ = nullptr; } @@ -75,8 +81,8 @@ class HdfsReadableFile : virtual public SequentialFile, // sequential access, read data at current offset in file virtual Status Read(size_t n, Slice* result, char* scratch) { Status s; - Log(mylog, "[hdfs] HdfsReadableFile reading %s %ld\n", - filename_.c_str(), n); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsReadableFile reading %s %ld\n", + filename_.c_str(), n); char* buffer = scratch; size_t total_bytes_read = 0; @@ -97,7 +103,8 @@ class HdfsReadableFile : virtual public SequentialFile, } assert(total_bytes_read <= n); - Log(mylog, "[hdfs] HdfsReadableFile read %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsReadableFile read %s\n", + filename_.c_str()); if (bytes_read < 0) { s = IOError(filename_, errno); @@ -112,10 +119,12 @@ class HdfsReadableFile : virtual public SequentialFile, virtual Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const { Status s; - Log(mylog, "[hdfs] HdfsReadableFile preading %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsReadableFile preading %s\n", + filename_.c_str()); ssize_t bytes_read = hdfsPread(fileSys_, hfile_, offset, (void*)scratch, (tSize)n); - Log(mylog, "[hdfs] HdfsReadableFile pread %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsReadableFile pread %s\n", + filename_.c_str()); *result = Slice(scratch, (bytes_read < 0) ? 0 : bytes_read); if (bytes_read < 0) { // An error: return a non-ok status @@ -125,7 +134,8 @@ class HdfsReadableFile : virtual public SequentialFile, } virtual Status Skip(uint64_t n) { - Log(mylog, "[hdfs] HdfsReadableFile skip %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsReadableFile skip %s\n", + filename_.c_str()); // get current offset from file tOffset current = hdfsTell(fileSys_, hfile_); if (current < 0) { @@ -144,7 +154,8 @@ class HdfsReadableFile : virtual public SequentialFile, // returns true if we are at the end of file, false otherwise bool feof() { - Log(mylog, "[hdfs] HdfsReadableFile feof %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsReadableFile feof %s\n", + filename_.c_str()); if (hdfsTell(fileSys_, hfile_) == fileSize()) { return true; } @@ -153,7 +164,8 @@ class HdfsReadableFile : virtual public SequentialFile, // the current size of the file tOffset fileSize() { - Log(mylog, "[hdfs] HdfsReadableFile fileSize %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsReadableFile fileSize %s\n", + filename_.c_str()); hdfsFileInfo* pFileInfo = hdfsGetPathInfo(fileSys_, filename_.c_str()); tOffset size = 0L; if (pFileInfo != nullptr) { @@ -176,16 +188,20 @@ class HdfsWritableFile: public WritableFile { public: HdfsWritableFile(hdfsFS fileSys, const std::string& fname) : fileSys_(fileSys), filename_(fname) , hfile_(nullptr) { - Log(mylog, "[hdfs] HdfsWritableFile opening %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsWritableFile opening %s\n", + filename_.c_str()); hfile_ = hdfsOpenFile(fileSys_, filename_.c_str(), O_WRONLY, 0, 0, 0); - Log(mylog, "[hdfs] HdfsWritableFile opened %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsWritableFile opened %s\n", + filename_.c_str()); assert(hfile_ != nullptr); } virtual ~HdfsWritableFile() { if (hfile_ != nullptr) { - Log(mylog, "[hdfs] HdfsWritableFile closing %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsWritableFile closing %s\n", + filename_.c_str()); hdfsCloseFile(fileSys_, hfile_); - Log(mylog, "[hdfs] HdfsWritableFile closed %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsWritableFile closed %s\n", + filename_.c_str()); hfile_ = nullptr; } } @@ -202,11 +218,13 @@ class HdfsWritableFile: public WritableFile { } virtual Status Append(const Slice& data) { - Log(mylog, "[hdfs] HdfsWritableFile Append %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsWritableFile Append %s\n", + filename_.c_str()); const char* src = data.data(); size_t left = data.size(); size_t ret = hdfsWrite(fileSys_, hfile_, src, left); - Log(mylog, "[hdfs] HdfsWritableFile Appended %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsWritableFile Appended %s\n", + filename_.c_str()); if (ret != left) { return IOError(filename_, errno); } @@ -219,14 +237,16 @@ class HdfsWritableFile: public WritableFile { virtual Status Sync() { Status s; - Log(mylog, "[hdfs] HdfsWritableFile Sync %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsWritableFile Sync %s\n", + filename_.c_str()); if (hdfsFlush(fileSys_, hfile_) == -1) { return IOError(filename_, errno); } if (hdfsHSync(fileSys_, hfile_) == -1) { return IOError(filename_, errno); } - Log(mylog, "[hdfs] HdfsWritableFile Synced %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsWritableFile Synced %s\n", + filename_.c_str()); return Status::OK(); } @@ -239,11 +259,13 @@ class HdfsWritableFile: public WritableFile { } virtual Status Close() { - Log(mylog, "[hdfs] HdfsWritableFile closing %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsWritableFile closing %s\n", + filename_.c_str()); if (hdfsCloseFile(fileSys_, hfile_) != 0) { return IOError(filename_, errno); } - Log(mylog, "[hdfs] HdfsWritableFile closed %s\n", filename_.c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsWritableFile closed %s\n", + filename_.c_str()); hfile_ = nullptr; return Status::OK(); } @@ -258,13 +280,13 @@ class HdfsLogger : public Logger { public: HdfsLogger(HdfsWritableFile* f, uint64_t (*gettid)()) : file_(f), gettid_(gettid) { - Log(mylog, "[hdfs] HdfsLogger opened %s\n", - file_->getName().c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsLogger opened %s\n", + file_->getName().c_str()); } virtual ~HdfsLogger() { - Log(mylog, "[hdfs] HdfsLogger closed %s\n", - file_->getName().c_str()); + ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsLogger closed %s\n", + file_->getName().c_str()); delete file_; if (mylog != nullptr && mylog == this) { mylog = nullptr; @@ -392,12 +414,6 @@ Status HdfsEnv::NewWritableFile(const std::string& fname, return Status::OK(); } -Status HdfsEnv::NewRandomRWFile(const std::string& fname, - unique_ptr* result, - const EnvOptions& options) { - return Status::NotSupported("NewRandomRWFile not supported on HdfsEnv"); -} - class HdfsDirectory : public Directory { public: explicit HdfsDirectory(int fd) : fd_(fd) {} @@ -417,26 +433,24 @@ Status HdfsEnv::NewDirectory(const std::string& name, result->reset(new HdfsDirectory(0)); return Status::OK(); default: // fail if the directory doesn't exist - Log(mylog, "NewDirectory hdfsExists call failed"); + ROCKS_LOG_FATAL(mylog, "NewDirectory hdfsExists call failed"); throw HdfsFatalException("hdfsExists call failed with error " + - std::to_string(value) + " on path " + name + + ToString(value) + " on path " + name + ".\n"); } } -bool HdfsEnv::FileExists(const std::string& fname) { - +Status HdfsEnv::FileExists(const std::string& fname) { int value = hdfsExists(fileSys_, fname.c_str()); switch (value) { case HDFS_EXISTS: - return true; + return Status::OK(); case HDFS_DOESNT_EXIST: - return false; + return Status::NotFound(); default: // anything else should be an error - Log(mylog, "FileExists hdfsExists call failed"); - throw HdfsFatalException("hdfsExists call failed with error " + - std::to_string(value) + " on path " + fname + - ".\n"); + ROCKS_LOG_FATAL(mylog, "FileExists hdfsExists call failed"); + return Status::IOError("hdfsExists call failed with error " + + ToString(value) + " on path " + fname + ".\n"); } } @@ -451,7 +465,7 @@ Status HdfsEnv::GetChildren(const std::string& path, if (numEntries >= 0) { for(int i = 0; i < numEntries; i++) { char* pathname = pHdfsFileInfo[i].mName; - char* filename = rindex(pathname, '/'); + char* filename = std::rindex(pathname, '/'); if (filename != nullptr) { result->push_back(filename+1); } @@ -461,18 +475,18 @@ Status HdfsEnv::GetChildren(const std::string& path, } } else { // numEntries < 0 indicates error - Log(mylog, "hdfsListDirectory call failed with error "); + ROCKS_LOG_FATAL(mylog, "hdfsListDirectory call failed with error "); throw HdfsFatalException( "hdfsListDirectory call failed negative error.\n"); } break; } case HDFS_DOESNT_EXIST: // directory does not exist, exit - break; + return Status::NotFound(); default: // anything else should be an error - Log(mylog, "GetChildren hdfsExists call failed"); + ROCKS_LOG_FATAL(mylog, "GetChildren hdfsExists call failed"); throw HdfsFatalException("hdfsExists call failed with error " + - std::to_string(value) + ".\n"); + ToString(value) + ".\n"); } return Status::OK(); } @@ -500,9 +514,9 @@ Status HdfsEnv::CreateDirIfMissing(const std::string& name) { case HDFS_DOESNT_EXIST: return CreateDir(name); default: // anything else should be an error - Log(mylog, "CreateDirIfMissing hdfsExists call failed"); + ROCKS_LOG_FATAL(mylog, "CreateDirIfMissing hdfsExists call failed"); throw HdfsFatalException("hdfsExists call failed with error " + - std::to_string(value) + ".\n"); + ToString(value) + ".\n"); } }; @@ -534,7 +548,7 @@ Status HdfsEnv::GetFileModificationTime(const std::string& fname, } // The rename is not atomic. HDFS does not allow a renaming if the -// target already exists. So, we delete the target before attemting the +// target already exists. So, we delete the target before attempting the // rename. Status HdfsEnv::RenameFile(const std::string& src, const std::string& target) { hdfsDelete(fileSys_, target.c_str(), 1); @@ -571,6 +585,11 @@ Status HdfsEnv::NewLogger(const std::string& fname, return Status::OK(); } +// The factory method for creating an HDFS Env +Status NewHdfsEnv(Env** hdfs_env, const std::string& fsname) { + *hdfs_env = new HdfsEnv(fsname); + return Status::OK(); +} } // namespace rocksdb #endif // ROCKSDB_HDFS_FILE_C @@ -578,14 +597,16 @@ Status HdfsEnv::NewLogger(const std::string& fname, #else // USE_HDFS // dummy placeholders used when HDFS is not available -#include "rocksdb/env.h" -#include "hdfs/env_hdfs.h" namespace rocksdb { Status HdfsEnv::NewSequentialFile(const std::string& fname, unique_ptr* result, const EnvOptions& options) { return Status::NotSupported("Not compiled with hdfs support"); } + + Status NewHdfsEnv(Env** hdfs_env, const std::string& fsname) { + return Status::NotSupported("Not compiled with hdfs support"); + } } #endif diff --git a/env/env_posix.cc b/env/env_posix.cc new file mode 100644 index 00000000000..5a671d72fe4 --- /dev/null +++ b/env/env_posix.cc @@ -0,0 +1,976 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors +#include +#include +#include +#if defined(OS_LINUX) +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#include +#ifdef OS_LINUX +#include +#include +#endif +#include +#include +#include +#include +// Get nano time includes +#if defined(OS_LINUX) || defined(OS_FREEBSD) +#elif defined(__MACH__) +#include +#include +#else +#include +#endif +#include +#include +#include + +#include "env/io_posix.h" +#include "env/posix_logger.h" +#include "monitoring/iostats_context_imp.h" +#include "monitoring/thread_status_updater.h" +#include "port/port.h" +#include "rocksdb/options.h" +#include "rocksdb/slice.h" +#include "util/coding.h" +#include "util/logging.h" +#include "util/random.h" +#include "util/string_util.h" +#include "util/sync_point.h" +#include "util/thread_local.h" +#include "util/threadpool_imp.h" + +#if !defined(TMPFS_MAGIC) +#define TMPFS_MAGIC 0x01021994 +#endif +#if !defined(XFS_SUPER_MAGIC) +#define XFS_SUPER_MAGIC 0x58465342 +#endif +#if !defined(EXT4_SUPER_MAGIC) +#define EXT4_SUPER_MAGIC 0xEF53 +#endif + +namespace rocksdb { + +namespace { + +ThreadStatusUpdater* CreateThreadStatusUpdater() { + return new ThreadStatusUpdater(); +} + +// list of pathnames that are locked +static std::set lockedFiles; +static port::Mutex mutex_lockedFiles; + +static int LockOrUnlock(const std::string& fname, int fd, bool lock) { + mutex_lockedFiles.Lock(); + if (lock) { + // If it already exists in the lockedFiles set, then it is already locked, + // and fail this lock attempt. Otherwise, insert it into lockedFiles. + // This check is needed because fcntl() does not detect lock conflict + // if the fcntl is issued by the same thread that earlier acquired + // this lock. + if (lockedFiles.insert(fname).second == false) { + mutex_lockedFiles.Unlock(); + errno = ENOLCK; + return -1; + } + } else { + // If we are unlocking, then verify that we had locked it earlier, + // it should already exist in lockedFiles. Remove it from lockedFiles. + if (lockedFiles.erase(fname) != 1) { + mutex_lockedFiles.Unlock(); + errno = ENOLCK; + return -1; + } + } + errno = 0; + struct flock f; + memset(&f, 0, sizeof(f)); + f.l_type = (lock ? F_WRLCK : F_UNLCK); + f.l_whence = SEEK_SET; + f.l_start = 0; + f.l_len = 0; // Lock/unlock entire file + int value = fcntl(fd, F_SETLK, &f); + if (value == -1 && lock) { + // if there is an error in locking, then remove the pathname from lockedfiles + lockedFiles.erase(fname); + } + mutex_lockedFiles.Unlock(); + return value; +} + +class PosixFileLock : public FileLock { + public: + int fd_; + std::string filename; +}; + +class PosixEnv : public Env { + public: + PosixEnv(); + + virtual ~PosixEnv() { + for (const auto tid : threads_to_join_) { + pthread_join(tid, nullptr); + } + for (int pool_id = 0; pool_id < Env::Priority::TOTAL; ++pool_id) { + thread_pools_[pool_id].JoinAllThreads(); + } + // Delete the thread_status_updater_ only when the current Env is not + // Env::Default(). This is to avoid the free-after-use error when + // Env::Default() is destructed while some other child threads are + // still trying to update thread status. + if (this != Env::Default()) { + delete thread_status_updater_; + } + } + + void SetFD_CLOEXEC(int fd, const EnvOptions* options) { + if ((options == nullptr || options->set_fd_cloexec) && fd > 0) { + fcntl(fd, F_SETFD, fcntl(fd, F_GETFD) | FD_CLOEXEC); + } + } + + virtual Status NewSequentialFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + result->reset(); + int fd = -1; + int flags = O_RDONLY; + FILE* file = nullptr; + + if (options.use_direct_reads && !options.use_mmap_reads) { +#ifdef ROCKSDB_LITE + return Status::IOError(fname, "Direct I/O not supported in RocksDB lite"); +#endif // !ROCKSDB_LITE +#if !defined(OS_MACOSX) && !defined(OS_OPENBSD) && !defined(OS_SOLARIS) + flags |= O_DIRECT; +#endif + } + + do { + IOSTATS_TIMER_GUARD(open_nanos); + fd = open(fname.c_str(), flags, 0644); + } while (fd < 0 && errno == EINTR); + if (fd < 0) { + return IOError("While opening a file for sequentially reading", fname, + errno); + } + + SetFD_CLOEXEC(fd, &options); + + if (options.use_direct_reads && !options.use_mmap_reads) { +#ifdef OS_MACOSX + if (fcntl(fd, F_NOCACHE, 1) == -1) { + close(fd); + return IOError("While fcntl NoCache", fname, errno); + } +#endif + } else { + do { + IOSTATS_TIMER_GUARD(open_nanos); + file = fdopen(fd, "r"); + } while (file == nullptr && errno == EINTR); + if (file == nullptr) { + close(fd); + return IOError("While opening file for sequentially read", fname, + errno); + } + } + result->reset(new PosixSequentialFile(fname, file, fd, options)); + return Status::OK(); + } + + virtual Status NewRandomAccessFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + result->reset(); + Status s; + int fd; + int flags = O_RDONLY; + if (options.use_direct_reads && !options.use_mmap_reads) { +#ifdef ROCKSDB_LITE + return Status::IOError(fname, "Direct I/O not supported in RocksDB lite"); +#endif // !ROCKSDB_LITE +#if !defined(OS_MACOSX) && !defined(OS_OPENBSD) && !defined(OS_SOLARIS) + flags |= O_DIRECT; + TEST_SYNC_POINT_CALLBACK("NewRandomAccessFile:O_DIRECT", &flags); +#endif + } + + do { + IOSTATS_TIMER_GUARD(open_nanos); + fd = open(fname.c_str(), flags, 0644); + } while (fd < 0 && errno == EINTR); + if (fd < 0) { + return IOError("While open a file for random read", fname, errno); + } + SetFD_CLOEXEC(fd, &options); + + if (options.use_mmap_reads && sizeof(void*) >= 8) { + // Use of mmap for random reads has been removed because it + // kills performance when storage is fast. + // Use mmap when virtual address-space is plentiful. + uint64_t size; + s = GetFileSize(fname, &size); + if (s.ok()) { + void* base = mmap(nullptr, size, PROT_READ, MAP_SHARED, fd, 0); + if (base != MAP_FAILED) { + result->reset(new PosixMmapReadableFile(fd, fname, base, + size, options)); + } else { + s = IOError("while mmap file for read", fname, errno); + } + } + close(fd); + } else { + if (options.use_direct_reads && !options.use_mmap_reads) { +#ifdef OS_MACOSX + if (fcntl(fd, F_NOCACHE, 1) == -1) { + close(fd); + return IOError("while fcntl NoCache", fname, errno); + } +#endif + } + result->reset(new PosixRandomAccessFile(fname, fd, options)); + } + return s; + } + + virtual Status OpenWritableFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options, + bool reopen = false) { + result->reset(); + Status s; + int fd = -1; + int flags = (reopen) ? (O_CREAT | O_APPEND) : (O_CREAT | O_TRUNC); + // Direct IO mode with O_DIRECT flag or F_NOCAHCE (MAC OSX) + if (options.use_direct_writes && !options.use_mmap_writes) { + // Note: we should avoid O_APPEND here due to ta the following bug: + // POSIX requires that opening a file with the O_APPEND flag should + // have no affect on the location at which pwrite() writes data. + // However, on Linux, if a file is opened with O_APPEND, pwrite() + // appends data to the end of the file, regardless of the value of + // offset. + // More info here: https://linux.die.net/man/2/pwrite +#ifdef ROCKSDB_LITE + return Status::IOError(fname, "Direct I/O not supported in RocksDB lite"); +#endif // ROCKSDB_LITE + flags |= O_WRONLY; +#if !defined(OS_MACOSX) && !defined(OS_OPENBSD) && !defined(OS_SOLARIS) + flags |= O_DIRECT; +#endif + TEST_SYNC_POINT_CALLBACK("NewWritableFile:O_DIRECT", &flags); + } else if (options.use_mmap_writes) { + // non-direct I/O + flags |= O_RDWR; + } else { + flags |= O_WRONLY; + } + + do { + IOSTATS_TIMER_GUARD(open_nanos); + fd = open(fname.c_str(), flags, 0644); + } while (fd < 0 && errno == EINTR); + + if (fd < 0) { + s = IOError("While open a file for appending", fname, errno); + return s; + } + SetFD_CLOEXEC(fd, &options); + + if (options.use_mmap_writes) { + if (!checkedDiskForMmap_) { + // this will be executed once in the program's lifetime. + // do not use mmapWrite on non ext-3/xfs/tmpfs systems. + if (!SupportsFastAllocate(fname)) { + forceMmapOff_ = true; + } + checkedDiskForMmap_ = true; + } + } + if (options.use_mmap_writes && !forceMmapOff_) { + result->reset(new PosixMmapFile(fname, fd, page_size_, options)); + } else if (options.use_direct_writes && !options.use_mmap_writes) { +#ifdef OS_MACOSX + if (fcntl(fd, F_NOCACHE, 1) == -1) { + close(fd); + s = IOError("While fcntl NoCache an opened file for appending", fname, + errno); + return s; + } +#elif defined(OS_SOLARIS) + if (directio(fd, DIRECTIO_ON) == -1) { + if (errno != ENOTTY) { // ZFS filesystems don't support DIRECTIO_ON + close(fd); + s = IOError("While calling directio()", fname, errno); + return s; + } + } +#endif + result->reset(new PosixWritableFile(fname, fd, options)); + } else { + // disable mmap writes + EnvOptions no_mmap_writes_options = options; + no_mmap_writes_options.use_mmap_writes = false; + result->reset(new PosixWritableFile(fname, fd, no_mmap_writes_options)); + } + return s; + } + + virtual Status NewWritableFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + return OpenWritableFile(fname, result, options, false); + } + + virtual Status ReopenWritableFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + return OpenWritableFile(fname, result, options, true); + } + + virtual Status ReuseWritableFile(const std::string& fname, + const std::string& old_fname, + unique_ptr* result, + const EnvOptions& options) override { + result->reset(); + Status s; + int fd = -1; + + int flags = 0; + // Direct IO mode with O_DIRECT flag or F_NOCAHCE (MAC OSX) + if (options.use_direct_writes && !options.use_mmap_writes) { +#ifdef ROCKSDB_LITE + return Status::IOError(fname, "Direct I/O not supported in RocksDB lite"); +#endif // !ROCKSDB_LITE + flags |= O_WRONLY; +#if !defined(OS_MACOSX) && !defined(OS_OPENBSD) && !defined(OS_SOLARIS) + flags |= O_DIRECT; +#endif + TEST_SYNC_POINT_CALLBACK("NewWritableFile:O_DIRECT", &flags); + } else if (options.use_mmap_writes) { + // mmap needs O_RDWR mode + flags |= O_RDWR; + } else { + flags |= O_WRONLY; + } + + do { + IOSTATS_TIMER_GUARD(open_nanos); + fd = open(old_fname.c_str(), flags, 0644); + } while (fd < 0 && errno == EINTR); + if (fd < 0) { + s = IOError("while reopen file for write", fname, errno); + return s; + } + + SetFD_CLOEXEC(fd, &options); + // rename into place + if (rename(old_fname.c_str(), fname.c_str()) != 0) { + s = IOError("while rename file to " + fname, old_fname, errno); + close(fd); + return s; + } + + if (options.use_mmap_writes) { + if (!checkedDiskForMmap_) { + // this will be executed once in the program's lifetime. + // do not use mmapWrite on non ext-3/xfs/tmpfs systems. + if (!SupportsFastAllocate(fname)) { + forceMmapOff_ = true; + } + checkedDiskForMmap_ = true; + } + } + if (options.use_mmap_writes && !forceMmapOff_) { + result->reset(new PosixMmapFile(fname, fd, page_size_, options)); + } else if (options.use_direct_writes && !options.use_mmap_writes) { +#ifdef OS_MACOSX + if (fcntl(fd, F_NOCACHE, 1) == -1) { + close(fd); + s = IOError("while fcntl NoCache for reopened file for append", fname, + errno); + return s; + } +#elif defined(OS_SOLARIS) + if (directio(fd, DIRECTIO_ON) == -1) { + if (errno != ENOTTY) { // ZFS filesystems don't support DIRECTIO_ON + close(fd); + s = IOError("while calling directio()", fname, errno); + return s; + } + } +#endif + result->reset(new PosixWritableFile(fname, fd, options)); + } else { + // disable mmap writes + EnvOptions no_mmap_writes_options = options; + no_mmap_writes_options.use_mmap_writes = false; + result->reset(new PosixWritableFile(fname, fd, no_mmap_writes_options)); + } + return s; + + return s; + } + + virtual Status NewRandomRWFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + int fd = -1; + while (fd < 0) { + IOSTATS_TIMER_GUARD(open_nanos); + fd = open(fname.c_str(), O_CREAT | O_RDWR, 0644); + if (fd < 0) { + // Error while opening the file + if (errno == EINTR) { + continue; + } + return IOError("While open file for random read/write", fname, errno); + } + } + + SetFD_CLOEXEC(fd, &options); + result->reset(new PosixRandomRWFile(fname, fd, options)); + return Status::OK(); + } + + virtual Status NewDirectory(const std::string& name, + unique_ptr* result) override { + result->reset(); + int fd; + { + IOSTATS_TIMER_GUARD(open_nanos); + fd = open(name.c_str(), 0); + } + if (fd < 0) { + return IOError("While open directory", name, errno); + } else { + result->reset(new PosixDirectory(fd)); + } + return Status::OK(); + } + + virtual Status FileExists(const std::string& fname) override { + int result = access(fname.c_str(), F_OK); + + if (result == 0) { + return Status::OK(); + } + + switch (errno) { + case EACCES: + case ELOOP: + case ENAMETOOLONG: + case ENOENT: + case ENOTDIR: + return Status::NotFound(); + default: + assert(result == EIO || result == ENOMEM); + return Status::IOError("Unexpected error(" + ToString(result) + + ") accessing file `" + fname + "' "); + } + } + + virtual Status GetChildren(const std::string& dir, + std::vector* result) override { + result->clear(); + DIR* d = opendir(dir.c_str()); + if (d == nullptr) { + switch (errno) { + case EACCES: + case ENOENT: + case ENOTDIR: + return Status::NotFound(); + default: + return IOError("While opendir", dir, errno); + } + } + struct dirent* entry; + while ((entry = readdir(d)) != nullptr) { + result->push_back(entry->d_name); + } + closedir(d); + return Status::OK(); + } + + virtual Status DeleteFile(const std::string& fname) override { + Status result; + if (unlink(fname.c_str()) != 0) { + result = IOError("while unlink() file", fname, errno); + } + return result; + }; + + virtual Status CreateDir(const std::string& name) override { + Status result; + if (mkdir(name.c_str(), 0755) != 0) { + result = IOError("While mkdir", name, errno); + } + return result; + }; + + virtual Status CreateDirIfMissing(const std::string& name) override { + Status result; + if (mkdir(name.c_str(), 0755) != 0) { + if (errno != EEXIST) { + result = IOError("While mkdir if missing", name, errno); + } else if (!DirExists(name)) { // Check that name is actually a + // directory. + // Message is taken from mkdir + result = Status::IOError("`"+name+"' exists but is not a directory"); + } + } + return result; + }; + + virtual Status DeleteDir(const std::string& name) override { + Status result; + if (rmdir(name.c_str()) != 0) { + result = IOError("file rmdir", name, errno); + } + return result; + }; + + virtual Status GetFileSize(const std::string& fname, + uint64_t* size) override { + Status s; + struct stat sbuf; + if (stat(fname.c_str(), &sbuf) != 0) { + *size = 0; + s = IOError("while stat a file for size", fname, errno); + } else { + *size = sbuf.st_size; + } + return s; + } + + virtual Status GetFileModificationTime(const std::string& fname, + uint64_t* file_mtime) override { + struct stat s; + if (stat(fname.c_str(), &s) !=0) { + return IOError("while stat a file for modification time", fname, errno); + } + *file_mtime = static_cast(s.st_mtime); + return Status::OK(); + } + virtual Status RenameFile(const std::string& src, + const std::string& target) override { + Status result; + if (rename(src.c_str(), target.c_str()) != 0) { + result = IOError("While renaming a file to " + target, src, errno); + } + return result; + } + + virtual Status LinkFile(const std::string& src, + const std::string& target) override { + Status result; + if (link(src.c_str(), target.c_str()) != 0) { + if (errno == EXDEV) { + return Status::NotSupported("No cross FS links allowed"); + } + result = IOError("while link file to " + target, src, errno); + } + return result; + } + + virtual Status LockFile(const std::string& fname, FileLock** lock) override { + *lock = nullptr; + Status result; + int fd; + { + IOSTATS_TIMER_GUARD(open_nanos); + fd = open(fname.c_str(), O_RDWR | O_CREAT, 0644); + } + if (fd < 0) { + result = IOError("while open a file for lock", fname, errno); + } else if (LockOrUnlock(fname, fd, true) == -1) { + result = IOError("While lock file", fname, errno); + close(fd); + } else { + SetFD_CLOEXEC(fd, nullptr); + PosixFileLock* my_lock = new PosixFileLock; + my_lock->fd_ = fd; + my_lock->filename = fname; + *lock = my_lock; + } + return result; + } + + virtual Status UnlockFile(FileLock* lock) override { + PosixFileLock* my_lock = reinterpret_cast(lock); + Status result; + if (LockOrUnlock(my_lock->filename, my_lock->fd_, false) == -1) { + result = IOError("unlock", my_lock->filename, errno); + } + close(my_lock->fd_); + delete my_lock; + return result; + } + + virtual void Schedule(void (*function)(void* arg1), void* arg, + Priority pri = LOW, void* tag = nullptr, + void (*unschedFunction)(void* arg) = 0) override; + + virtual int UnSchedule(void* arg, Priority pri) override; + + virtual void StartThread(void (*function)(void* arg), void* arg) override; + + virtual void WaitForJoin() override; + + virtual unsigned int GetThreadPoolQueueLen(Priority pri = LOW) const override; + + virtual Status GetTestDirectory(std::string* result) override { + const char* env = getenv("TEST_TMPDIR"); + if (env && env[0] != '\0') { + *result = env; + } else { + char buf[100]; + snprintf(buf, sizeof(buf), "/tmp/rocksdbtest-%d", int(geteuid())); + *result = buf; + } + // Directory may already exist + CreateDir(*result); + return Status::OK(); + } + + virtual Status GetThreadList( + std::vector* thread_list) override { + assert(thread_status_updater_); + return thread_status_updater_->GetThreadList(thread_list); + } + + static uint64_t gettid(pthread_t tid) { + uint64_t thread_id = 0; + memcpy(&thread_id, &tid, std::min(sizeof(thread_id), sizeof(tid))); + return thread_id; + } + + static uint64_t gettid() { + pthread_t tid = pthread_self(); + return gettid(tid); + } + + virtual uint64_t GetThreadID() const override { + return gettid(pthread_self()); + } + + virtual Status NewLogger(const std::string& fname, + shared_ptr* result) override { + FILE* f; + { + IOSTATS_TIMER_GUARD(open_nanos); + f = fopen(fname.c_str(), "w"); + } + if (f == nullptr) { + result->reset(); + return IOError("when fopen a file for new logger", fname, errno); + } else { + int fd = fileno(f); +#ifdef ROCKSDB_FALLOCATE_PRESENT + fallocate(fd, FALLOC_FL_KEEP_SIZE, 0, 4 * 1024); +#endif + SetFD_CLOEXEC(fd, nullptr); + result->reset(new PosixLogger(f, &PosixEnv::gettid, this)); + return Status::OK(); + } + } + + virtual uint64_t NowMicros() override { + struct timeval tv; + gettimeofday(&tv, nullptr); + return static_cast(tv.tv_sec) * 1000000 + tv.tv_usec; + } + + virtual uint64_t NowNanos() override { +#if defined(OS_LINUX) || defined(OS_FREEBSD) || defined(OS_AIX) + struct timespec ts; + clock_gettime(CLOCK_MONOTONIC, &ts); + return static_cast(ts.tv_sec) * 1000000000 + ts.tv_nsec; +#elif defined(OS_SOLARIS) + return gethrtime(); +#elif defined(__MACH__) + clock_serv_t cclock; + mach_timespec_t ts; + host_get_clock_service(mach_host_self(), CALENDAR_CLOCK, &cclock); + clock_get_time(cclock, &ts); + mach_port_deallocate(mach_task_self(), cclock); + return static_cast(ts.tv_sec) * 1000000000 + ts.tv_nsec; +#else + return std::chrono::duration_cast( + std::chrono::steady_clock::now().time_since_epoch()).count(); +#endif + } + + virtual void SleepForMicroseconds(int micros) override { usleep(micros); } + + virtual Status GetHostName(char* name, uint64_t len) override { + int ret = gethostname(name, static_cast(len)); + if (ret < 0) { + if (errno == EFAULT || errno == EINVAL) + return Status::InvalidArgument(strerror(errno)); + else + return IOError("GetHostName", name, errno); + } + return Status::OK(); + } + + virtual Status GetCurrentTime(int64_t* unix_time) override { + time_t ret = time(nullptr); + if (ret == (time_t) -1) { + return IOError("GetCurrentTime", "", errno); + } + *unix_time = (int64_t) ret; + return Status::OK(); + } + + virtual Status GetAbsolutePath(const std::string& db_path, + std::string* output_path) override { + if (db_path.find('/') == 0) { + *output_path = db_path; + return Status::OK(); + } + + char the_path[256]; + char* ret = getcwd(the_path, 256); + if (ret == nullptr) { + return Status::IOError(strerror(errno)); + } + + *output_path = ret; + return Status::OK(); + } + + // Allow increasing the number of worker threads. + virtual void SetBackgroundThreads(int num, Priority pri) override { + assert(pri >= Priority::BOTTOM && pri <= Priority::HIGH); + thread_pools_[pri].SetBackgroundThreads(num); + } + + virtual int GetBackgroundThreads(Priority pri) override { + assert(pri >= Priority::BOTTOM && pri <= Priority::HIGH); + return thread_pools_[pri].GetBackgroundThreads(); + } + + // Allow increasing the number of worker threads. + virtual void IncBackgroundThreadsIfNeeded(int num, Priority pri) override { + assert(pri >= Priority::BOTTOM && pri <= Priority::HIGH); + thread_pools_[pri].IncBackgroundThreadsIfNeeded(num); + } + + virtual void LowerThreadPoolIOPriority(Priority pool = LOW) override { + assert(pool >= Priority::BOTTOM && pool <= Priority::HIGH); +#ifdef OS_LINUX + thread_pools_[pool].LowerIOPriority(); +#endif + } + + virtual std::string TimeToString(uint64_t secondsSince1970) override { + const time_t seconds = (time_t)secondsSince1970; + struct tm t; + int maxsize = 64; + std::string dummy; + dummy.reserve(maxsize); + dummy.resize(maxsize); + char* p = &dummy[0]; + localtime_r(&seconds, &t); + snprintf(p, maxsize, + "%04d/%02d/%02d-%02d:%02d:%02d ", + t.tm_year + 1900, + t.tm_mon + 1, + t.tm_mday, + t.tm_hour, + t.tm_min, + t.tm_sec); + return dummy; + } + + EnvOptions OptimizeForLogWrite(const EnvOptions& env_options, + const DBOptions& db_options) const override { + EnvOptions optimized = env_options; + optimized.use_mmap_writes = false; + optimized.use_direct_writes = false; + optimized.bytes_per_sync = db_options.wal_bytes_per_sync; + // TODO(icanadi) it's faster if fallocate_with_keep_size is false, but it + // breaks TransactionLogIteratorStallAtLastRecord unit test. Fix the unit + // test and make this false + optimized.fallocate_with_keep_size = true; + return optimized; + } + + EnvOptions OptimizeForManifestWrite( + const EnvOptions& env_options) const override { + EnvOptions optimized = env_options; + optimized.use_mmap_writes = false; + optimized.use_direct_writes = false; + optimized.fallocate_with_keep_size = true; + return optimized; + } + + private: + bool checkedDiskForMmap_; + bool forceMmapOff_; // do we override Env options? + + // Returns true iff the named directory exists and is a directory. + virtual bool DirExists(const std::string& dname) { + struct stat statbuf; + if (stat(dname.c_str(), &statbuf) == 0) { + return S_ISDIR(statbuf.st_mode); + } + return false; // stat() failed return false + } + + bool SupportsFastAllocate(const std::string& path) { +#ifdef ROCKSDB_FALLOCATE_PRESENT + struct statfs s; + if (statfs(path.c_str(), &s)){ + return false; + } + switch (s.f_type) { + case EXT4_SUPER_MAGIC: + return true; + case XFS_SUPER_MAGIC: + return true; + case TMPFS_MAGIC: + return true; + default: + return false; + } +#else + return false; +#endif + } + + size_t page_size_; + + std::vector thread_pools_; + pthread_mutex_t mu_; + std::vector threads_to_join_; +}; + +PosixEnv::PosixEnv() + : checkedDiskForMmap_(false), + forceMmapOff_(false), + page_size_(getpagesize()), + thread_pools_(Priority::TOTAL) { + ThreadPoolImpl::PthreadCall("mutex_init", pthread_mutex_init(&mu_, nullptr)); + for (int pool_id = 0; pool_id < Env::Priority::TOTAL; ++pool_id) { + thread_pools_[pool_id].SetThreadPriority( + static_cast(pool_id)); + // This allows later initializing the thread-local-env of each thread. + thread_pools_[pool_id].SetHostEnv(this); + } + thread_status_updater_ = CreateThreadStatusUpdater(); +} + +void PosixEnv::Schedule(void (*function)(void* arg1), void* arg, Priority pri, + void* tag, void (*unschedFunction)(void* arg)) { + assert(pri >= Priority::BOTTOM && pri <= Priority::HIGH); + thread_pools_[pri].Schedule(function, arg, tag, unschedFunction); +} + +int PosixEnv::UnSchedule(void* arg, Priority pri) { + return thread_pools_[pri].UnSchedule(arg); +} + +unsigned int PosixEnv::GetThreadPoolQueueLen(Priority pri) const { + assert(pri >= Priority::BOTTOM && pri <= Priority::HIGH); + return thread_pools_[pri].GetQueueLen(); +} + +struct StartThreadState { + void (*user_function)(void*); + void* arg; +}; + +static void* StartThreadWrapper(void* arg) { + StartThreadState* state = reinterpret_cast(arg); + state->user_function(state->arg); + delete state; + return nullptr; +} + +void PosixEnv::StartThread(void (*function)(void* arg), void* arg) { + pthread_t t; + StartThreadState* state = new StartThreadState; + state->user_function = function; + state->arg = arg; + ThreadPoolImpl::PthreadCall( + "start thread", pthread_create(&t, nullptr, &StartThreadWrapper, state)); + ThreadPoolImpl::PthreadCall("lock", pthread_mutex_lock(&mu_)); + threads_to_join_.push_back(t); + ThreadPoolImpl::PthreadCall("unlock", pthread_mutex_unlock(&mu_)); +} + +void PosixEnv::WaitForJoin() { + for (const auto tid : threads_to_join_) { + pthread_join(tid, nullptr); + } + threads_to_join_.clear(); +} + +} // namespace + +std::string Env::GenerateUniqueId() { + std::string uuid_file = "/proc/sys/kernel/random/uuid"; + + Status s = FileExists(uuid_file); + if (s.ok()) { + std::string uuid; + s = ReadFileToString(this, uuid_file, &uuid); + if (s.ok()) { + return uuid; + } + } + // Could not read uuid_file - generate uuid using "nanos-random" + Random64 r(time(nullptr)); + uint64_t random_uuid_portion = + r.Uniform(std::numeric_limits::max()); + uint64_t nanos_uuid_portion = NowNanos(); + char uuid2[200]; + snprintf(uuid2, + 200, + "%lx-%lx", + (unsigned long)nanos_uuid_portion, + (unsigned long)random_uuid_portion); + return uuid2; +} + +// +// Default Posix Env +// +Env* Env::Default() { + // The following function call initializes the singletons of ThreadLocalPtr + // right before the static default_env. This guarantees default_env will + // always being destructed before the ThreadLocalPtr singletons get + // destructed as C++ guarantees that the destructions of static variables + // is in the reverse order of their constructions. + // + // Since static members are destructed in the reverse order + // of their construction, having this call here guarantees that + // the destructor of static PosixEnv will go first, then the + // the singletons of ThreadLocalPtr. + ThreadLocalPtr::InitSingletons(); + static PosixEnv default_env; + return &default_env; +} + +} // namespace rocksdb diff --git a/env/env_test.cc b/env/env_test.cc new file mode 100644 index 00000000000..9ec2f142ed8 --- /dev/null +++ b/env/env_test.cc @@ -0,0 +1,1498 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef OS_WIN +#include +#endif + +#ifdef ROCKSDB_MALLOC_USABLE_SIZE +#ifdef OS_FREEBSD +#include +#else +#include +#endif +#endif +#include + +#include +#include +#include +#include + +#ifdef OS_LINUX +#include +#include +#include +#include +#include +#endif + +#ifdef ROCKSDB_FALLOCATE_PRESENT +#include +#endif + +#include "env/env_chroot.h" +#include "port/port.h" +#include "rocksdb/env.h" +#include "util/coding.h" +#include "util/log_buffer.h" +#include "util/mutexlock.h" +#include "util/string_util.h" +#include "util/sync_point.h" +#include "util/testharness.h" +#include "util/testutil.h" + +#ifdef OS_LINUX +static const size_t kPageSize = sysconf(_SC_PAGESIZE); +#else +static const size_t kPageSize = 4 * 1024; +#endif + +namespace rocksdb { + +static const int kDelayMicros = 100000; + +struct Deleter { + explicit Deleter(void (*fn)(void*)) : fn_(fn) {} + + void operator()(void* ptr) { + assert(fn_); + assert(ptr); + (*fn_)(ptr); + } + + void (*fn_)(void*); +}; + +std::unique_ptr NewAligned(const size_t size, const char ch) { + char* ptr = nullptr; +#ifdef OS_WIN + if (!(ptr = reinterpret_cast(_aligned_malloc(size, kPageSize)))) { + return std::unique_ptr(nullptr, Deleter(_aligned_free)); + } + std::unique_ptr uptr(ptr, Deleter(_aligned_free)); +#else + if (posix_memalign(reinterpret_cast(&ptr), kPageSize, size) != 0) { + return std::unique_ptr(nullptr, Deleter(free)); + } + std::unique_ptr uptr(ptr, Deleter(free)); +#endif + memset(uptr.get(), ch, size); + return uptr; +} + +class EnvPosixTest : public testing::Test { + private: + port::Mutex mu_; + std::string events_; + + public: + Env* env_; + bool direct_io_; + EnvPosixTest() : env_(Env::Default()), direct_io_(false) {} +}; + +class EnvPosixTestWithParam + : public EnvPosixTest, + public ::testing::WithParamInterface> { + public: + EnvPosixTestWithParam() { + std::pair param_pair = GetParam(); + env_ = param_pair.first; + direct_io_ = param_pair.second; + } + + void WaitThreadPoolsEmpty() { + // Wait until the thread pools are empty. + while (env_->GetThreadPoolQueueLen(Env::Priority::LOW) != 0) { + Env::Default()->SleepForMicroseconds(kDelayMicros); + } + while (env_->GetThreadPoolQueueLen(Env::Priority::HIGH) != 0) { + Env::Default()->SleepForMicroseconds(kDelayMicros); + } + } + + ~EnvPosixTestWithParam() { WaitThreadPoolsEmpty(); } +}; + +static void SetBool(void* ptr) { + reinterpret_cast*>(ptr)->store(true); +} + +TEST_F(EnvPosixTest, RunImmediately) { + for (int pri = Env::BOTTOM; pri < Env::TOTAL; ++pri) { + std::atomic called(false); + env_->SetBackgroundThreads(1, static_cast(pri)); + env_->Schedule(&SetBool, &called, static_cast(pri)); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_TRUE(called.load()); + } +} + +TEST_P(EnvPosixTestWithParam, UnSchedule) { + std::atomic called(false); + env_->SetBackgroundThreads(1, Env::LOW); + + /* Block the low priority queue */ + test::SleepingBackgroundTask sleeping_task, sleeping_task1; + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task, + Env::Priority::LOW); + + /* Schedule another task */ + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &sleeping_task1, + Env::Priority::LOW, &sleeping_task1); + + /* Remove it with a different tag */ + ASSERT_EQ(0, env_->UnSchedule(&called, Env::Priority::LOW)); + + /* Remove it from the queue with the right tag */ + ASSERT_EQ(1, env_->UnSchedule(&sleeping_task1, Env::Priority::LOW)); + + // Unblock background thread + sleeping_task.WakeUp(); + + /* Schedule another task */ + env_->Schedule(&SetBool, &called); + for (int i = 0; i < kDelayMicros; i++) { + if (called.load()) { + break; + } + Env::Default()->SleepForMicroseconds(1); + } + ASSERT_TRUE(called.load()); + + ASSERT_TRUE(!sleeping_task.IsSleeping() && !sleeping_task1.IsSleeping()); + WaitThreadPoolsEmpty(); +} + +TEST_P(EnvPosixTestWithParam, RunMany) { + std::atomic last_id(0); + + struct CB { + std::atomic* last_id_ptr; // Pointer to shared slot + int id; // Order# for the execution of this callback + + CB(std::atomic* p, int i) : last_id_ptr(p), id(i) {} + + static void Run(void* v) { + CB* cb = reinterpret_cast(v); + int cur = cb->last_id_ptr->load(); + ASSERT_EQ(cb->id - 1, cur); + cb->last_id_ptr->store(cb->id); + } + }; + + // Schedule in different order than start time + CB cb1(&last_id, 1); + CB cb2(&last_id, 2); + CB cb3(&last_id, 3); + CB cb4(&last_id, 4); + env_->Schedule(&CB::Run, &cb1); + env_->Schedule(&CB::Run, &cb2); + env_->Schedule(&CB::Run, &cb3); + env_->Schedule(&CB::Run, &cb4); + + Env::Default()->SleepForMicroseconds(kDelayMicros); + int cur = last_id.load(std::memory_order_acquire); + ASSERT_EQ(4, cur); + WaitThreadPoolsEmpty(); +} + +struct State { + port::Mutex mu; + int val; + int num_running; +}; + +static void ThreadBody(void* arg) { + State* s = reinterpret_cast(arg); + s->mu.Lock(); + s->val += 1; + s->num_running -= 1; + s->mu.Unlock(); +} + +TEST_P(EnvPosixTestWithParam, StartThread) { + State state; + state.val = 0; + state.num_running = 3; + for (int i = 0; i < 3; i++) { + env_->StartThread(&ThreadBody, &state); + } + while (true) { + state.mu.Lock(); + int num = state.num_running; + state.mu.Unlock(); + if (num == 0) { + break; + } + Env::Default()->SleepForMicroseconds(kDelayMicros); + } + ASSERT_EQ(state.val, 3); + WaitThreadPoolsEmpty(); +} + +TEST_P(EnvPosixTestWithParam, TwoPools) { + // Data structures to signal tasks to run. + port::Mutex mutex; + port::CondVar cv(&mutex); + bool should_start = false; + + class CB { + public: + CB(const std::string& pool_name, int pool_size, port::Mutex* trigger_mu, + port::CondVar* trigger_cv, bool* _should_start) + : mu_(), + num_running_(0), + num_finished_(0), + pool_size_(pool_size), + pool_name_(pool_name), + trigger_mu_(trigger_mu), + trigger_cv_(trigger_cv), + should_start_(_should_start) {} + + static void Run(void* v) { + CB* cb = reinterpret_cast(v); + cb->Run(); + } + + void Run() { + { + MutexLock l(&mu_); + num_running_++; + // make sure we don't have more than pool_size_ jobs running. + ASSERT_LE(num_running_, pool_size_.load()); + } + + { + MutexLock l(trigger_mu_); + while (!(*should_start_)) { + trigger_cv_->Wait(); + } + } + + { + MutexLock l(&mu_); + num_running_--; + num_finished_++; + } + } + + int NumFinished() { + MutexLock l(&mu_); + return num_finished_; + } + + void Reset(int pool_size) { + pool_size_.store(pool_size); + num_finished_ = 0; + } + + private: + port::Mutex mu_; + int num_running_; + int num_finished_; + std::atomic pool_size_; + std::string pool_name_; + port::Mutex* trigger_mu_; + port::CondVar* trigger_cv_; + bool* should_start_; + }; + + const int kLowPoolSize = 2; + const int kHighPoolSize = 4; + const int kJobs = 8; + + CB low_pool_job("low", kLowPoolSize, &mutex, &cv, &should_start); + CB high_pool_job("high", kHighPoolSize, &mutex, &cv, &should_start); + + env_->SetBackgroundThreads(kLowPoolSize); + env_->SetBackgroundThreads(kHighPoolSize, Env::Priority::HIGH); + + ASSERT_EQ(0U, env_->GetThreadPoolQueueLen(Env::Priority::LOW)); + ASSERT_EQ(0U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + + // schedule same number of jobs in each pool + for (int i = 0; i < kJobs; i++) { + env_->Schedule(&CB::Run, &low_pool_job); + env_->Schedule(&CB::Run, &high_pool_job, Env::Priority::HIGH); + } + // Wait a short while for the jobs to be dispatched. + int sleep_count = 0; + while ((unsigned int)(kJobs - kLowPoolSize) != + env_->GetThreadPoolQueueLen(Env::Priority::LOW) || + (unsigned int)(kJobs - kHighPoolSize) != + env_->GetThreadPoolQueueLen(Env::Priority::HIGH)) { + env_->SleepForMicroseconds(kDelayMicros); + if (++sleep_count > 100) { + break; + } + } + + ASSERT_EQ((unsigned int)(kJobs - kLowPoolSize), + env_->GetThreadPoolQueueLen()); + ASSERT_EQ((unsigned int)(kJobs - kLowPoolSize), + env_->GetThreadPoolQueueLen(Env::Priority::LOW)); + ASSERT_EQ((unsigned int)(kJobs - kHighPoolSize), + env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + + // Trigger jobs to run. + { + MutexLock l(&mutex); + should_start = true; + cv.SignalAll(); + } + + // wait for all jobs to finish + while (low_pool_job.NumFinished() < kJobs || + high_pool_job.NumFinished() < kJobs) { + env_->SleepForMicroseconds(kDelayMicros); + } + + ASSERT_EQ(0U, env_->GetThreadPoolQueueLen(Env::Priority::LOW)); + ASSERT_EQ(0U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + + // Hold jobs to schedule; + should_start = false; + + // call IncBackgroundThreadsIfNeeded to two pools. One increasing and + // the other decreasing + env_->IncBackgroundThreadsIfNeeded(kLowPoolSize - 1, Env::Priority::LOW); + env_->IncBackgroundThreadsIfNeeded(kHighPoolSize + 1, Env::Priority::HIGH); + high_pool_job.Reset(kHighPoolSize + 1); + low_pool_job.Reset(kLowPoolSize); + + // schedule same number of jobs in each pool + for (int i = 0; i < kJobs; i++) { + env_->Schedule(&CB::Run, &low_pool_job); + env_->Schedule(&CB::Run, &high_pool_job, Env::Priority::HIGH); + } + // Wait a short while for the jobs to be dispatched. + sleep_count = 0; + while ((unsigned int)(kJobs - kLowPoolSize) != + env_->GetThreadPoolQueueLen(Env::Priority::LOW) || + (unsigned int)(kJobs - (kHighPoolSize + 1)) != + env_->GetThreadPoolQueueLen(Env::Priority::HIGH)) { + env_->SleepForMicroseconds(kDelayMicros); + if (++sleep_count > 100) { + break; + } + } + ASSERT_EQ((unsigned int)(kJobs - kLowPoolSize), + env_->GetThreadPoolQueueLen()); + ASSERT_EQ((unsigned int)(kJobs - kLowPoolSize), + env_->GetThreadPoolQueueLen(Env::Priority::LOW)); + ASSERT_EQ((unsigned int)(kJobs - (kHighPoolSize + 1)), + env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + + // Trigger jobs to run. + { + MutexLock l(&mutex); + should_start = true; + cv.SignalAll(); + } + + // wait for all jobs to finish + while (low_pool_job.NumFinished() < kJobs || + high_pool_job.NumFinished() < kJobs) { + env_->SleepForMicroseconds(kDelayMicros); + } + + env_->SetBackgroundThreads(kHighPoolSize, Env::Priority::HIGH); + WaitThreadPoolsEmpty(); +} + +TEST_P(EnvPosixTestWithParam, DecreaseNumBgThreads) { + std::vector tasks(10); + + // Set number of thread to 1 first. + env_->SetBackgroundThreads(1, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + + // Schedule 3 tasks. 0 running; Task 1, 2 waiting. + for (size_t i = 0; i < 3; i++) { + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &tasks[i], + Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + } + ASSERT_EQ(2U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + ASSERT_TRUE(tasks[0].IsSleeping()); + ASSERT_TRUE(!tasks[1].IsSleeping()); + ASSERT_TRUE(!tasks[2].IsSleeping()); + + // Increase to 2 threads. Task 0, 1 running; 2 waiting + env_->SetBackgroundThreads(2, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ(1U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + ASSERT_TRUE(tasks[0].IsSleeping()); + ASSERT_TRUE(tasks[1].IsSleeping()); + ASSERT_TRUE(!tasks[2].IsSleeping()); + + // Shrink back to 1 thread. Still task 0, 1 running, 2 waiting + env_->SetBackgroundThreads(1, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ(1U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + ASSERT_TRUE(tasks[0].IsSleeping()); + ASSERT_TRUE(tasks[1].IsSleeping()); + ASSERT_TRUE(!tasks[2].IsSleeping()); + + // The last task finishes. Task 0 running, 2 waiting. + tasks[1].WakeUp(); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ(1U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + ASSERT_TRUE(tasks[0].IsSleeping()); + ASSERT_TRUE(!tasks[1].IsSleeping()); + ASSERT_TRUE(!tasks[2].IsSleeping()); + + // Increase to 5 threads. Task 0 and 2 running. + env_->SetBackgroundThreads(5, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ((unsigned int)0, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + ASSERT_TRUE(tasks[0].IsSleeping()); + ASSERT_TRUE(tasks[2].IsSleeping()); + + // Change number of threads a couple of times while there is no sufficient + // tasks. + env_->SetBackgroundThreads(7, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + tasks[2].WakeUp(); + ASSERT_EQ(0U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + env_->SetBackgroundThreads(3, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ(0U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + env_->SetBackgroundThreads(4, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ(0U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + env_->SetBackgroundThreads(5, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ(0U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + env_->SetBackgroundThreads(4, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ(0U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + + Env::Default()->SleepForMicroseconds(kDelayMicros * 50); + + // Enqueue 5 more tasks. Thread pool size now is 4. + // Task 0, 3, 4, 5 running;6, 7 waiting. + for (size_t i = 3; i < 8; i++) { + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &tasks[i], + Env::Priority::HIGH); + } + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ(2U, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + ASSERT_TRUE(tasks[3].IsSleeping()); + ASSERT_TRUE(tasks[4].IsSleeping()); + ASSERT_TRUE(tasks[5].IsSleeping()); + ASSERT_TRUE(!tasks[6].IsSleeping()); + ASSERT_TRUE(!tasks[7].IsSleeping()); + + // Wake up task 0, 3 and 4. Task 5, 6, 7 running. + tasks[0].WakeUp(); + tasks[3].WakeUp(); + tasks[4].WakeUp(); + + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ((unsigned int)0, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + for (size_t i = 5; i < 8; i++) { + ASSERT_TRUE(tasks[i].IsSleeping()); + } + + // Shrink back to 1 thread. Still task 5, 6, 7 running + env_->SetBackgroundThreads(1, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_TRUE(tasks[5].IsSleeping()); + ASSERT_TRUE(tasks[6].IsSleeping()); + ASSERT_TRUE(tasks[7].IsSleeping()); + + // Wake up task 6. Task 5, 7 running + tasks[6].WakeUp(); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_TRUE(tasks[5].IsSleeping()); + ASSERT_TRUE(!tasks[6].IsSleeping()); + ASSERT_TRUE(tasks[7].IsSleeping()); + + // Wake up threads 7. Task 5 running + tasks[7].WakeUp(); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_TRUE(!tasks[7].IsSleeping()); + + // Enqueue thread 8 and 9. Task 5 running; one of 8, 9 might be running. + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &tasks[8], + Env::Priority::HIGH); + env_->Schedule(&test::SleepingBackgroundTask::DoSleepTask, &tasks[9], + Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_GT(env_->GetThreadPoolQueueLen(Env::Priority::HIGH), (unsigned int)0); + ASSERT_TRUE(!tasks[8].IsSleeping() || !tasks[9].IsSleeping()); + + // Increase to 4 threads. Task 5, 8, 9 running. + env_->SetBackgroundThreads(4, Env::Priority::HIGH); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_EQ((unsigned int)0, env_->GetThreadPoolQueueLen(Env::Priority::HIGH)); + ASSERT_TRUE(tasks[8].IsSleeping()); + ASSERT_TRUE(tasks[9].IsSleeping()); + + // Shrink to 1 thread + env_->SetBackgroundThreads(1, Env::Priority::HIGH); + + // Wake up thread 9. + tasks[9].WakeUp(); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_TRUE(!tasks[9].IsSleeping()); + ASSERT_TRUE(tasks[8].IsSleeping()); + + // Wake up thread 8 + tasks[8].WakeUp(); + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_TRUE(!tasks[8].IsSleeping()); + + // Wake up the last thread + tasks[5].WakeUp(); + + Env::Default()->SleepForMicroseconds(kDelayMicros); + ASSERT_TRUE(!tasks[5].IsSleeping()); + WaitThreadPoolsEmpty(); +} + +#if (defined OS_LINUX || defined OS_WIN) +// Travis doesn't support fallocate or getting unique ID from files for whatever +// reason. +#ifndef TRAVIS + +namespace { +bool IsSingleVarint(const std::string& s) { + Slice slice(s); + + uint64_t v; + if (!GetVarint64(&slice, &v)) { + return false; + } + + return slice.size() == 0; +} + +bool IsUniqueIDValid(const std::string& s) { + return !s.empty() && !IsSingleVarint(s); +} + +const size_t MAX_ID_SIZE = 100; +char temp_id[MAX_ID_SIZE]; + + +} // namespace + +// Determine whether we can use the FS_IOC_GETVERSION ioctl +// on a file in directory DIR. Create a temporary file therein, +// try to apply the ioctl (save that result), cleanup and +// return the result. Return true if it is supported, and +// false if anything fails. +// Note that this function "knows" that dir has just been created +// and is empty, so we create a simply-named test file: "f". +bool ioctl_support__FS_IOC_GETVERSION(const std::string& dir) { +#ifdef OS_WIN + return true; +#else + const std::string file = dir + "/f"; + int fd; + do { + fd = open(file.c_str(), O_CREAT | O_RDWR | O_TRUNC, 0644); + } while (fd < 0 && errno == EINTR); + long int version; + bool ok = (fd >= 0 && ioctl(fd, FS_IOC_GETVERSION, &version) >= 0); + + close(fd); + unlink(file.c_str()); + + return ok; +#endif +} + +// To ensure that Env::GetUniqueId-related tests work correctly, the files +// should be stored in regular storage like "hard disk" or "flash device", +// and not on a tmpfs file system (like /dev/shm and /tmp on some systems). +// Otherwise we cannot get the correct id. +// +// This function serves as the replacement for test::TmpDir(), which may be +// customized to be on a file system that doesn't work with GetUniqueId(). + +class IoctlFriendlyTmpdir { + public: + explicit IoctlFriendlyTmpdir() { + char dir_buf[100]; + + const char *fmt = "%s/rocksdb.XXXXXX"; + const char *tmp = getenv("TEST_IOCTL_FRIENDLY_TMPDIR"); + +#ifdef OS_WIN +#define rmdir _rmdir + if(tmp == nullptr) { + tmp = getenv("TMP"); + } + + snprintf(dir_buf, sizeof dir_buf, fmt, tmp); + auto result = _mktemp(dir_buf); + assert(result != nullptr); + BOOL ret = CreateDirectory(dir_buf, NULL); + assert(ret == TRUE); + dir_ = dir_buf; +#else + std::list candidate_dir_list = {"/var/tmp", "/tmp"}; + + // If $TEST_IOCTL_FRIENDLY_TMPDIR/rocksdb.XXXXXX fits, use + // $TEST_IOCTL_FRIENDLY_TMPDIR; subtract 2 for the "%s", and + // add 1 for the trailing NUL byte. + if (tmp && strlen(tmp) + strlen(fmt) - 2 + 1 <= sizeof dir_buf) { + // use $TEST_IOCTL_FRIENDLY_TMPDIR value + candidate_dir_list.push_front(tmp); + } + + for (const std::string& d : candidate_dir_list) { + snprintf(dir_buf, sizeof dir_buf, fmt, d.c_str()); + if (mkdtemp(dir_buf)) { + if (ioctl_support__FS_IOC_GETVERSION(dir_buf)) { + dir_ = dir_buf; + return; + } else { + // Diagnose ioctl-related failure only if this is the + // directory specified via that envvar. + if (tmp && tmp == d) { + fprintf(stderr, "TEST_IOCTL_FRIENDLY_TMPDIR-specified directory is " + "not suitable: %s\n", d.c_str()); + } + rmdir(dir_buf); // ignore failure + } + } else { + // mkdtemp failed: diagnose it, but don't give up. + fprintf(stderr, "mkdtemp(%s/...) failed: %s\n", d.c_str(), + strerror(errno)); + } + } + + fprintf(stderr, "failed to find an ioctl-friendly temporary directory;" + " specify one via the TEST_IOCTL_FRIENDLY_TMPDIR envvar\n"); + std::abort(); +#endif +} + + ~IoctlFriendlyTmpdir() { + rmdir(dir_.c_str()); + } + + const std::string& name() const { + return dir_; + } + + private: + std::string dir_; +}; + +#ifndef ROCKSDB_LITE +TEST_F(EnvPosixTest, PositionedAppend) { + unique_ptr writable_file; + EnvOptions options; + options.use_direct_writes = true; + options.use_mmap_writes = false; + IoctlFriendlyTmpdir ift; + ASSERT_OK(env_->NewWritableFile(ift.name() + "/f", &writable_file, options)); + const size_t kBlockSize = 4096; + const size_t kPageSize = 4096; + const size_t kDataSize = kPageSize; + // Write a page worth of 'a' + auto data_ptr = NewAligned(kDataSize, 'a'); + Slice data_a(data_ptr.get(), kDataSize); + ASSERT_OK(writable_file->PositionedAppend(data_a, 0U)); + // Write a page worth of 'b' right after the first sector + data_ptr = NewAligned(kDataSize, 'b'); + Slice data_b(data_ptr.get(), kDataSize); + ASSERT_OK(writable_file->PositionedAppend(data_b, kBlockSize)); + ASSERT_OK(writable_file->Close()); + // The file now has 1 sector worth of a followed by a page worth of b + + // Verify the above + unique_ptr seq_file; + ASSERT_OK(env_->NewSequentialFile(ift.name() + "/f", &seq_file, options)); + char scratch[kPageSize * 2]; + Slice result; + ASSERT_OK(seq_file->Read(sizeof(scratch), &result, scratch)); + ASSERT_EQ(kPageSize + kBlockSize, result.size()); + ASSERT_EQ('a', result[kBlockSize - 1]); + ASSERT_EQ('b', result[kBlockSize]); +} +#endif // !ROCKSDB_LITE + +// Only works in linux platforms +TEST_P(EnvPosixTestWithParam, RandomAccessUniqueID) { + // Create file. + if (env_ == Env::Default()) { + EnvOptions soptions; + soptions.use_direct_reads = soptions.use_direct_writes = direct_io_; + IoctlFriendlyTmpdir ift; + std::string fname = ift.name() + "/testfile"; + unique_ptr wfile; + ASSERT_OK(env_->NewWritableFile(fname, &wfile, soptions)); + + unique_ptr file; + + // Get Unique ID + ASSERT_OK(env_->NewRandomAccessFile(fname, &file, soptions)); + size_t id_size = file->GetUniqueId(temp_id, MAX_ID_SIZE); + ASSERT_TRUE(id_size > 0); + std::string unique_id1(temp_id, id_size); + ASSERT_TRUE(IsUniqueIDValid(unique_id1)); + + // Get Unique ID again + ASSERT_OK(env_->NewRandomAccessFile(fname, &file, soptions)); + id_size = file->GetUniqueId(temp_id, MAX_ID_SIZE); + ASSERT_TRUE(id_size > 0); + std::string unique_id2(temp_id, id_size); + ASSERT_TRUE(IsUniqueIDValid(unique_id2)); + + // Get Unique ID again after waiting some time. + env_->SleepForMicroseconds(1000000); + ASSERT_OK(env_->NewRandomAccessFile(fname, &file, soptions)); + id_size = file->GetUniqueId(temp_id, MAX_ID_SIZE); + ASSERT_TRUE(id_size > 0); + std::string unique_id3(temp_id, id_size); + ASSERT_TRUE(IsUniqueIDValid(unique_id3)); + + // Check IDs are the same. + ASSERT_EQ(unique_id1, unique_id2); + ASSERT_EQ(unique_id2, unique_id3); + + // Delete the file + env_->DeleteFile(fname); + } +} + +// only works in linux platforms +#ifdef ROCKSDB_FALLOCATE_PRESENT +TEST_P(EnvPosixTestWithParam, AllocateTest) { + if (env_ == Env::Default()) { + IoctlFriendlyTmpdir ift; + std::string fname = ift.name() + "/preallocate_testfile"; + + // Try fallocate in a file to see whether the target file system supports + // it. + // Skip the test if fallocate is not supported. + std::string fname_test_fallocate = ift.name() + "/preallocate_testfile_2"; + int fd = -1; + do { + fd = open(fname_test_fallocate.c_str(), O_CREAT | O_RDWR | O_TRUNC, 0644); + } while (fd < 0 && errno == EINTR); + ASSERT_GT(fd, 0); + + int alloc_status = fallocate(fd, 0, 0, 1); + + int err_number = 0; + if (alloc_status != 0) { + err_number = errno; + fprintf(stderr, "Warning: fallocate() fails, %s\n", strerror(err_number)); + } + close(fd); + ASSERT_OK(env_->DeleteFile(fname_test_fallocate)); + if (alloc_status != 0 && err_number == EOPNOTSUPP) { + // The filesystem containing the file does not support fallocate + return; + } + + EnvOptions soptions; + soptions.use_mmap_writes = false; + soptions.use_direct_reads = soptions.use_direct_writes = direct_io_; + unique_ptr wfile; + ASSERT_OK(env_->NewWritableFile(fname, &wfile, soptions)); + + // allocate 100 MB + size_t kPreallocateSize = 100 * 1024 * 1024; + size_t kBlockSize = 512; + size_t kPageSize = 4096; + size_t kDataSize = 1024 * 1024; + auto data_ptr = NewAligned(kDataSize, 'A'); + Slice data(data_ptr.get(), kDataSize); + wfile->SetPreallocationBlockSize(kPreallocateSize); + wfile->PrepareWrite(wfile->GetFileSize(), kDataSize); + ASSERT_OK(wfile->Append(data)); + ASSERT_OK(wfile->Flush()); + + struct stat f_stat; + ASSERT_EQ(stat(fname.c_str(), &f_stat), 0); + ASSERT_EQ((unsigned int)kDataSize, f_stat.st_size); + // verify that blocks are preallocated + // Note here that we don't check the exact number of blocks preallocated -- + // we only require that number of allocated blocks is at least what we + // expect. + // It looks like some FS give us more blocks that we asked for. That's fine. + // It might be worth investigating further. + ASSERT_LE((unsigned int)(kPreallocateSize / kBlockSize), f_stat.st_blocks); + + // close the file, should deallocate the blocks + wfile.reset(); + + stat(fname.c_str(), &f_stat); + ASSERT_EQ((unsigned int)kDataSize, f_stat.st_size); + // verify that preallocated blocks were deallocated on file close + // Because the FS might give us more blocks, we add a full page to the size + // and expect the number of blocks to be less or equal to that. + ASSERT_GE((f_stat.st_size + kPageSize + kBlockSize - 1) / kBlockSize, + (unsigned int)f_stat.st_blocks); + } +} +#endif // ROCKSDB_FALLOCATE_PRESENT + +// Returns true if any of the strings in ss are the prefix of another string. +bool HasPrefix(const std::unordered_set& ss) { + for (const std::string& s: ss) { + if (s.empty()) { + return true; + } + for (size_t i = 1; i < s.size(); ++i) { + if (ss.count(s.substr(0, i)) != 0) { + return true; + } + } + } + return false; +} + +// Only works in linux and WIN platforms +TEST_P(EnvPosixTestWithParam, RandomAccessUniqueIDConcurrent) { + if (env_ == Env::Default()) { + // Check whether a bunch of concurrently existing files have unique IDs. + EnvOptions soptions; + soptions.use_direct_reads = soptions.use_direct_writes = direct_io_; + + // Create the files + IoctlFriendlyTmpdir ift; + std::vector fnames; + for (int i = 0; i < 1000; ++i) { + fnames.push_back(ift.name() + "/" + "testfile" + ToString(i)); + + // Create file. + unique_ptr wfile; + ASSERT_OK(env_->NewWritableFile(fnames[i], &wfile, soptions)); + } + + // Collect and check whether the IDs are unique. + std::unordered_set ids; + for (const std::string fname : fnames) { + unique_ptr file; + std::string unique_id; + ASSERT_OK(env_->NewRandomAccessFile(fname, &file, soptions)); + size_t id_size = file->GetUniqueId(temp_id, MAX_ID_SIZE); + ASSERT_TRUE(id_size > 0); + unique_id = std::string(temp_id, id_size); + ASSERT_TRUE(IsUniqueIDValid(unique_id)); + + ASSERT_TRUE(ids.count(unique_id) == 0); + ids.insert(unique_id); + } + + // Delete the files + for (const std::string fname : fnames) { + ASSERT_OK(env_->DeleteFile(fname)); + } + + ASSERT_TRUE(!HasPrefix(ids)); + } +} + +// Only works in linux and WIN platforms +TEST_P(EnvPosixTestWithParam, RandomAccessUniqueIDDeletes) { + if (env_ == Env::Default()) { + EnvOptions soptions; + soptions.use_direct_reads = soptions.use_direct_writes = direct_io_; + + IoctlFriendlyTmpdir ift; + std::string fname = ift.name() + "/" + "testfile"; + + // Check that after file is deleted we don't get same ID again in a new + // file. + std::unordered_set ids; + for (int i = 0; i < 1000; ++i) { + // Create file. + { + unique_ptr wfile; + ASSERT_OK(env_->NewWritableFile(fname, &wfile, soptions)); + } + + // Get Unique ID + std::string unique_id; + { + unique_ptr file; + ASSERT_OK(env_->NewRandomAccessFile(fname, &file, soptions)); + size_t id_size = file->GetUniqueId(temp_id, MAX_ID_SIZE); + ASSERT_TRUE(id_size > 0); + unique_id = std::string(temp_id, id_size); + } + + ASSERT_TRUE(IsUniqueIDValid(unique_id)); + ASSERT_TRUE(ids.count(unique_id) == 0); + ids.insert(unique_id); + + // Delete the file + ASSERT_OK(env_->DeleteFile(fname)); + } + + ASSERT_TRUE(!HasPrefix(ids)); + } +} + +// Only works in linux platforms +#ifdef OS_WIN +TEST_P(EnvPosixTestWithParam, DISABLED_InvalidateCache) { +#else +TEST_P(EnvPosixTestWithParam, InvalidateCache) { +#endif + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + EnvOptions soptions; + soptions.use_direct_reads = soptions.use_direct_writes = direct_io_; + std::string fname = test::TmpDir(env_) + "/" + "testfile"; + + const size_t kSectorSize = 512; + auto data = NewAligned(kSectorSize, 0); + Slice slice(data.get(), kSectorSize); + + // Create file. + { + unique_ptr wfile; +#if !defined(OS_MACOSX) && !defined(OS_WIN) && !defined(OS_SOLARIS) && !defined(OS_AIX) + if (soptions.use_direct_writes) { + soptions.use_direct_writes = false; + } +#endif + ASSERT_OK(env_->NewWritableFile(fname, &wfile, soptions)); + ASSERT_OK(wfile->Append(slice)); + ASSERT_OK(wfile->InvalidateCache(0, 0)); + ASSERT_OK(wfile->Close()); + } + + // Random Read + { + unique_ptr file; + auto scratch = NewAligned(kSectorSize, 0); + Slice result; +#if !defined(OS_MACOSX) && !defined(OS_WIN) && !defined(OS_SOLARIS) && !defined(OS_AIX) + if (soptions.use_direct_reads) { + soptions.use_direct_reads = false; + } +#endif + ASSERT_OK(env_->NewRandomAccessFile(fname, &file, soptions)); + ASSERT_OK(file->Read(0, kSectorSize, &result, scratch.get())); + ASSERT_EQ(memcmp(scratch.get(), data.get(), kSectorSize), 0); + ASSERT_OK(file->InvalidateCache(0, 11)); + ASSERT_OK(file->InvalidateCache(0, 0)); + } + + // Sequential Read + { + unique_ptr file; + auto scratch = NewAligned(kSectorSize, 0); + Slice result; +#if !defined(OS_MACOSX) && !defined(OS_WIN) && !defined(OS_SOLARIS) && !defined(OS_AIX) + if (soptions.use_direct_reads) { + soptions.use_direct_reads = false; + } +#endif + ASSERT_OK(env_->NewSequentialFile(fname, &file, soptions)); + if (file->use_direct_io()) { + ASSERT_OK(file->PositionedRead(0, kSectorSize, &result, scratch.get())); + } else { + ASSERT_OK(file->Read(kSectorSize, &result, scratch.get())); + } + ASSERT_EQ(memcmp(scratch.get(), data.get(), kSectorSize), 0); + ASSERT_OK(file->InvalidateCache(0, 11)); + ASSERT_OK(file->InvalidateCache(0, 0)); + } + // Delete the file + ASSERT_OK(env_->DeleteFile(fname)); + rocksdb::SyncPoint::GetInstance()->ClearTrace(); +} +#endif // not TRAVIS +#endif // OS_LINUX || OS_WIN + +class TestLogger : public Logger { + public: + using Logger::Logv; + virtual void Logv(const char* format, va_list ap) override { + log_count++; + + char new_format[550]; + std::fill_n(new_format, sizeof(new_format), '2'); + { + va_list backup_ap; + va_copy(backup_ap, ap); + int n = vsnprintf(new_format, sizeof(new_format) - 1, format, backup_ap); + // 48 bytes for extra information + bytes allocated + +// When we have n == -1 there is not a terminating zero expected +#ifdef OS_WIN + if (n < 0) { + char_0_count++; + } +#endif + + if (new_format[0] == '[') { + // "[DEBUG] " + ASSERT_TRUE(n <= 56 + (512 - static_cast(sizeof(struct timeval)))); + } else { + ASSERT_TRUE(n <= 48 + (512 - static_cast(sizeof(struct timeval)))); + } + va_end(backup_ap); + } + + for (size_t i = 0; i < sizeof(new_format); i++) { + if (new_format[i] == 'x') { + char_x_count++; + } else if (new_format[i] == '\0') { + char_0_count++; + } + } + } + int log_count; + int char_x_count; + int char_0_count; +}; + +TEST_P(EnvPosixTestWithParam, LogBufferTest) { + TestLogger test_logger; + test_logger.SetInfoLogLevel(InfoLogLevel::INFO_LEVEL); + test_logger.log_count = 0; + test_logger.char_x_count = 0; + test_logger.char_0_count = 0; + LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, &test_logger); + LogBuffer log_buffer_debug(DEBUG_LEVEL, &test_logger); + + char bytes200[200]; + std::fill_n(bytes200, sizeof(bytes200), '1'); + bytes200[sizeof(bytes200) - 1] = '\0'; + char bytes600[600]; + std::fill_n(bytes600, sizeof(bytes600), '1'); + bytes600[sizeof(bytes600) - 1] = '\0'; + char bytes9000[9000]; + std::fill_n(bytes9000, sizeof(bytes9000), '1'); + bytes9000[sizeof(bytes9000) - 1] = '\0'; + + ROCKS_LOG_BUFFER(&log_buffer, "x%sx", bytes200); + ROCKS_LOG_BUFFER(&log_buffer, "x%sx", bytes600); + ROCKS_LOG_BUFFER(&log_buffer, "x%sx%sx%sx", bytes200, bytes200, bytes200); + ROCKS_LOG_BUFFER(&log_buffer, "x%sx%sx", bytes200, bytes600); + ROCKS_LOG_BUFFER(&log_buffer, "x%sx%sx", bytes600, bytes9000); + + ROCKS_LOG_BUFFER(&log_buffer_debug, "x%sx", bytes200); + test_logger.SetInfoLogLevel(DEBUG_LEVEL); + ROCKS_LOG_BUFFER(&log_buffer_debug, "x%sx%sx%sx", bytes600, bytes9000, + bytes200); + + ASSERT_EQ(0, test_logger.log_count); + log_buffer.FlushBufferToLog(); + log_buffer_debug.FlushBufferToLog(); + ASSERT_EQ(6, test_logger.log_count); + ASSERT_EQ(6, test_logger.char_0_count); + ASSERT_EQ(10, test_logger.char_x_count); +} + +class TestLogger2 : public Logger { + public: + explicit TestLogger2(size_t max_log_size) : max_log_size_(max_log_size) {} + using Logger::Logv; + virtual void Logv(const char* format, va_list ap) override { + char new_format[2000]; + std::fill_n(new_format, sizeof(new_format), '2'); + { + va_list backup_ap; + va_copy(backup_ap, ap); + int n = vsnprintf(new_format, sizeof(new_format) - 1, format, backup_ap); + // 48 bytes for extra information + bytes allocated + ASSERT_TRUE( + n <= 48 + static_cast(max_log_size_ - sizeof(struct timeval))); + ASSERT_TRUE(n > static_cast(max_log_size_ - sizeof(struct timeval))); + va_end(backup_ap); + } + } + size_t max_log_size_; +}; + +TEST_P(EnvPosixTestWithParam, LogBufferMaxSizeTest) { + char bytes9000[9000]; + std::fill_n(bytes9000, sizeof(bytes9000), '1'); + bytes9000[sizeof(bytes9000) - 1] = '\0'; + + for (size_t max_log_size = 256; max_log_size <= 1024; + max_log_size += 1024 - 256) { + TestLogger2 test_logger(max_log_size); + test_logger.SetInfoLogLevel(InfoLogLevel::INFO_LEVEL); + LogBuffer log_buffer(InfoLogLevel::INFO_LEVEL, &test_logger); + ROCKS_LOG_BUFFER_MAX_SZ(&log_buffer, max_log_size, "%s", bytes9000); + log_buffer.FlushBufferToLog(); + } +} + +TEST_P(EnvPosixTestWithParam, Preallocation) { + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + const std::string src = test::TmpDir(env_) + "/" + "testfile"; + unique_ptr srcfile; + EnvOptions soptions; + soptions.use_direct_reads = soptions.use_direct_writes = direct_io_; +#if !defined(OS_MACOSX) && !defined(OS_WIN) && !defined(OS_SOLARIS) && !defined(OS_AIX) + if (soptions.use_direct_writes) { + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "NewWritableFile:O_DIRECT", [&](void* arg) { + int* val = static_cast(arg); + *val &= ~O_DIRECT; + }); + } +#endif + ASSERT_OK(env_->NewWritableFile(src, &srcfile, soptions)); + srcfile->SetPreallocationBlockSize(1024 * 1024); + + // No writes should mean no preallocation + size_t block_size, last_allocated_block; + srcfile->GetPreallocationStatus(&block_size, &last_allocated_block); + ASSERT_EQ(last_allocated_block, 0UL); + + // Small write should preallocate one block + size_t kStrSize = 4096; + auto data = NewAligned(kStrSize, 'A'); + Slice str(data.get(), kStrSize); + srcfile->PrepareWrite(srcfile->GetFileSize(), kStrSize); + srcfile->Append(str); + srcfile->GetPreallocationStatus(&block_size, &last_allocated_block); + ASSERT_EQ(last_allocated_block, 1UL); + + // Write an entire preallocation block, make sure we increased by two. + { + auto buf_ptr = NewAligned(block_size, ' '); + Slice buf(buf_ptr.get(), block_size); + srcfile->PrepareWrite(srcfile->GetFileSize(), block_size); + srcfile->Append(buf); + srcfile->GetPreallocationStatus(&block_size, &last_allocated_block); + ASSERT_EQ(last_allocated_block, 2UL); + } + + // Write five more blocks at once, ensure we're where we need to be. + { + auto buf_ptr = NewAligned(block_size * 5, ' '); + Slice buf = Slice(buf_ptr.get(), block_size * 5); + srcfile->PrepareWrite(srcfile->GetFileSize(), buf.size()); + srcfile->Append(buf); + srcfile->GetPreallocationStatus(&block_size, &last_allocated_block); + ASSERT_EQ(last_allocated_block, 7UL); + } + rocksdb::SyncPoint::GetInstance()->ClearTrace(); +} + +// Test that the two ways to get children file attributes (in bulk or +// individually) behave consistently. +TEST_P(EnvPosixTestWithParam, ConsistentChildrenAttributes) { + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + EnvOptions soptions; + soptions.use_direct_reads = soptions.use_direct_writes = direct_io_; + const int kNumChildren = 10; + + std::string data; + for (int i = 0; i < kNumChildren; ++i) { + std::ostringstream oss; + oss << test::TmpDir(env_) << "/testfile_" << i; + const std::string path = oss.str(); + unique_ptr file; +#if !defined(OS_MACOSX) && !defined(OS_WIN) && !defined(OS_SOLARIS) && !defined(OS_AIX) + if (soptions.use_direct_writes) { + rocksdb::SyncPoint::GetInstance()->SetCallBack( + "NewWritableFile:O_DIRECT", [&](void* arg) { + int* val = static_cast(arg); + *val &= ~O_DIRECT; + }); + } +#endif + ASSERT_OK(env_->NewWritableFile(path, &file, soptions)); + auto buf_ptr = NewAligned(data.size(), 'T'); + Slice buf(buf_ptr.get(), data.size()); + file->Append(buf); + data.append(std::string(4096, 'T')); + } + + std::vector file_attrs; + ASSERT_OK(env_->GetChildrenFileAttributes(test::TmpDir(env_), &file_attrs)); + for (int i = 0; i < kNumChildren; ++i) { + std::ostringstream oss; + oss << "testfile_" << i; + const std::string name = oss.str(); + const std::string path = test::TmpDir(env_) + "/" + name; + + auto file_attrs_iter = std::find_if( + file_attrs.begin(), file_attrs.end(), + [&name](const Env::FileAttributes& fm) { return fm.name == name; }); + ASSERT_TRUE(file_attrs_iter != file_attrs.end()); + uint64_t size; + ASSERT_OK(env_->GetFileSize(path, &size)); + ASSERT_EQ(size, 4096 * i); + ASSERT_EQ(size, file_attrs_iter->size_bytes); + } + rocksdb::SyncPoint::GetInstance()->ClearTrace(); +} + +// Test that all WritableFileWrapper forwards all calls to WritableFile. +TEST_P(EnvPosixTestWithParam, WritableFileWrapper) { + class Base : public WritableFile { + public: + mutable int *step_; + + void inc(int x) const { + EXPECT_EQ(x, (*step_)++); + } + + explicit Base(int* step) : step_(step) { + inc(0); + } + + Status Append(const Slice& data) override { inc(1); return Status::OK(); } + Status Truncate(uint64_t size) override { return Status::OK(); } + Status Close() override { inc(2); return Status::OK(); } + Status Flush() override { inc(3); return Status::OK(); } + Status Sync() override { inc(4); return Status::OK(); } + Status Fsync() override { inc(5); return Status::OK(); } + void SetIOPriority(Env::IOPriority pri) override { inc(6); } + uint64_t GetFileSize() override { inc(7); return 0; } + void GetPreallocationStatus(size_t* block_size, + size_t* last_allocated_block) override { + inc(8); + } + size_t GetUniqueId(char* id, size_t max_size) const override { + inc(9); + return 0; + } + Status InvalidateCache(size_t offset, size_t length) override { + inc(10); + return Status::OK(); + } + + protected: + Status Allocate(uint64_t offset, uint64_t len) override { + inc(11); + return Status::OK(); + } + Status RangeSync(uint64_t offset, uint64_t nbytes) override { + inc(12); + return Status::OK(); + } + + public: + ~Base() { + inc(13); + } + }; + + class Wrapper : public WritableFileWrapper { + public: + explicit Wrapper(WritableFile* target) : WritableFileWrapper(target) {} + + void CallProtectedMethods() { + Allocate(0, 0); + RangeSync(0, 0); + } + }; + + int step = 0; + + { + Base b(&step); + Wrapper w(&b); + w.Append(Slice()); + w.Close(); + w.Flush(); + w.Sync(); + w.Fsync(); + w.SetIOPriority(Env::IOPriority::IO_HIGH); + w.GetFileSize(); + w.GetPreallocationStatus(nullptr, nullptr); + w.GetUniqueId(nullptr, 0); + w.InvalidateCache(0, 0); + w.CallProtectedMethods(); + } + + EXPECT_EQ(14, step); +} + +TEST_P(EnvPosixTestWithParam, PosixRandomRWFile) { + const std::string path = test::TmpDir(env_) + "/random_rw_file"; + + env_->DeleteFile(path); + + std::unique_ptr file; + ASSERT_OK(env_->NewRandomRWFile(path, &file, EnvOptions())); + + char buf[10000]; + Slice read_res; + + ASSERT_OK(file->Write(0, "ABCD")); + ASSERT_OK(file->Read(0, 10, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "ABCD"); + + ASSERT_OK(file->Write(2, "XXXX")); + ASSERT_OK(file->Read(0, 10, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "ABXXXX"); + + ASSERT_OK(file->Write(10, "ZZZ")); + ASSERT_OK(file->Read(10, 10, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "ZZZ"); + + ASSERT_OK(file->Write(11, "Y")); + ASSERT_OK(file->Read(10, 10, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "ZYZ"); + + ASSERT_OK(file->Write(200, "FFFFF")); + ASSERT_OK(file->Read(200, 10, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "FFFFF"); + + ASSERT_OK(file->Write(205, "XXXX")); + ASSERT_OK(file->Read(200, 10, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "FFFFFXXXX"); + + ASSERT_OK(file->Write(5, "QQQQ")); + ASSERT_OK(file->Read(0, 9, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "ABXXXQQQQ"); + + ASSERT_OK(file->Read(2, 4, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "XXXQ"); + + // Close file and reopen it + file->Close(); + ASSERT_OK(env_->NewRandomRWFile(path, &file, EnvOptions())); + + ASSERT_OK(file->Read(0, 9, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "ABXXXQQQQ"); + + ASSERT_OK(file->Read(10, 3, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "ZYZ"); + + ASSERT_OK(file->Read(200, 9, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "FFFFFXXXX"); + + ASSERT_OK(file->Write(4, "TTTTTTTTTTTTTTTT")); + ASSERT_OK(file->Read(0, 10, &read_res, buf)); + ASSERT_EQ(read_res.ToString(), "ABXXTTTTTT"); + + // Clean up + env_->DeleteFile(path); +} + +class RandomRWFileWithMirrorString { + public: + explicit RandomRWFileWithMirrorString(RandomRWFile* _file) : file_(_file) {} + + void Write(size_t offset, const std::string& data) { + // Write to mirror string + StringWrite(offset, data); + + // Write to file + Status s = file_->Write(offset, data); + ASSERT_OK(s) << s.ToString(); + } + + void Read(size_t offset = 0, size_t n = 1000000) { + Slice str_res(nullptr, 0); + if (offset < file_mirror_.size()) { + size_t str_res_sz = std::min(file_mirror_.size() - offset, n); + str_res = Slice(file_mirror_.data() + offset, str_res_sz); + StopSliceAtNull(&str_res); + } + + Slice file_res; + Status s = file_->Read(offset, n, &file_res, buf_); + ASSERT_OK(s) << s.ToString(); + StopSliceAtNull(&file_res); + + ASSERT_EQ(str_res.ToString(), file_res.ToString()) << offset << " " << n; + } + + void SetFile(RandomRWFile* _file) { file_ = _file; } + + private: + void StringWrite(size_t offset, const std::string& src) { + if (offset + src.size() > file_mirror_.size()) { + file_mirror_.resize(offset + src.size(), '\0'); + } + + char* pos = const_cast(file_mirror_.data() + offset); + memcpy(pos, src.data(), src.size()); + } + + void StopSliceAtNull(Slice* slc) { + for (size_t i = 0; i < slc->size(); i++) { + if ((*slc)[i] == '\0') { + *slc = Slice(slc->data(), i); + break; + } + } + } + + char buf_[10000]; + RandomRWFile* file_; + std::string file_mirror_; +}; + +TEST_P(EnvPosixTestWithParam, PosixRandomRWFileRandomized) { + const std::string path = test::TmpDir(env_) + "/random_rw_file_rand"; + env_->DeleteFile(path); + + unique_ptr file; + ASSERT_OK(env_->NewRandomRWFile(path, &file, EnvOptions())); + RandomRWFileWithMirrorString file_with_mirror(file.get()); + + Random rnd(301); + std::string buf; + for (int i = 0; i < 10000; i++) { + // Genrate random data + test::RandomString(&rnd, 10, &buf); + + // Pick random offset for write + size_t write_off = rnd.Next() % 1000; + file_with_mirror.Write(write_off, buf); + + // Pick random offset for read + size_t read_off = rnd.Next() % 1000; + size_t read_sz = rnd.Next() % 20; + file_with_mirror.Read(read_off, read_sz); + + if (i % 500 == 0) { + // Reopen the file every 500 iters + ASSERT_OK(env_->NewRandomRWFile(path, &file, EnvOptions())); + file_with_mirror.SetFile(file.get()); + } + } + + // clean up + env_->DeleteFile(path); +} + +INSTANTIATE_TEST_CASE_P(DefaultEnvWithoutDirectIO, EnvPosixTestWithParam, + ::testing::Values(std::pair(Env::Default(), + false))); +#if !defined(ROCKSDB_LITE) +INSTANTIATE_TEST_CASE_P(DefaultEnvWithDirectIO, EnvPosixTestWithParam, + ::testing::Values(std::pair(Env::Default(), + true))); +#endif // !defined(ROCKSDB_LITE) + +#if !defined(ROCKSDB_LITE) && !defined(OS_WIN) +static unique_ptr chroot_env(NewChrootEnv(Env::Default(), + test::TmpDir(Env::Default()))); +INSTANTIATE_TEST_CASE_P( + ChrootEnvWithoutDirectIO, EnvPosixTestWithParam, + ::testing::Values(std::pair(chroot_env.get(), false))); +INSTANTIATE_TEST_CASE_P( + ChrootEnvWithDirectIO, EnvPosixTestWithParam, + ::testing::Values(std::pair(chroot_env.get(), true))); +#endif // !defined(ROCKSDB_LITE) && !defined(OS_WIN) + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/env/io_posix.cc b/env/io_posix.cc new file mode 100644 index 00000000000..c5b14d3effe --- /dev/null +++ b/env/io_posix.cc @@ -0,0 +1,1028 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifdef ROCKSDB_LIB_IO_POSIX +#include "env/io_posix.h" +#include +#include +#include +#if defined(OS_LINUX) +#include +#endif +#include +#include +#include +#include +#include +#include +#include +#ifdef OS_LINUX +#include +#include +#include +#endif +#include "env/posix_logger.h" +#include "monitoring/iostats_context_imp.h" +#include "port/port.h" +#include "rocksdb/slice.h" +#include "util/coding.h" +#include "util/string_util.h" +#include "util/sync_point.h" + +namespace rocksdb { + +// A wrapper for fadvise, if the platform doesn't support fadvise, +// it will simply return 0. +int Fadvise(int fd, off_t offset, size_t len, int advice) { +#ifdef OS_LINUX + return posix_fadvise(fd, offset, len, advice); +#else + return 0; // simply do nothing. +#endif +} + +namespace { +size_t GetLogicalBufferSize(int __attribute__((__unused__)) fd) { +#ifdef OS_LINUX + struct stat buf; + int result = fstat(fd, &buf); + if (result == -1) { + return kDefaultPageSize; + } + if (major(buf.st_dev) == 0) { + // Unnamed devices (e.g. non-device mounts), reserved as null device number. + // These don't have an entry in /sys/dev/block/. Return a sensible default. + return kDefaultPageSize; + } + + // Reading queue/logical_block_size does not require special permissions. + const int kBufferSize = 100; + char path[kBufferSize]; + char real_path[PATH_MAX + 1]; + snprintf(path, kBufferSize, "/sys/dev/block/%u:%u", major(buf.st_dev), + minor(buf.st_dev)); + if (realpath(path, real_path) == nullptr) { + return kDefaultPageSize; + } + std::string device_dir(real_path); + if (!device_dir.empty() && device_dir.back() == '/') { + device_dir.pop_back(); + } + // NOTE: sda3 does not have a `queue/` subdir, only the parent sda has it. + // $ ls -al '/sys/dev/block/8:3' + // lrwxrwxrwx. 1 root root 0 Jun 26 01:38 /sys/dev/block/8:3 -> + // ../../block/sda/sda3 + size_t parent_end = device_dir.rfind('/', device_dir.length() - 1); + if (parent_end == std::string::npos) { + return kDefaultPageSize; + } + size_t parent_begin = device_dir.rfind('/', parent_end - 1); + if (parent_begin == std::string::npos) { + return kDefaultPageSize; + } + if (device_dir.substr(parent_begin + 1, parent_end - parent_begin - 1) != + "block") { + device_dir = device_dir.substr(0, parent_end); + } + std::string fname = device_dir + "/queue/logical_block_size"; + FILE* fp; + size_t size = 0; + fp = fopen(fname.c_str(), "r"); + if (fp != nullptr) { + char* line = nullptr; + size_t len = 0; + if (getline(&line, &len, fp) != -1) { + sscanf(line, "%zu", &size); + } + free(line); + fclose(fp); + } + if (size != 0 && (size & (size - 1)) == 0) { + return size; + } +#endif + return kDefaultPageSize; +} +} // namespace + +/* + * DirectIOHelper + */ +#ifndef NDEBUG +namespace { + +bool IsSectorAligned(const size_t off, size_t sector_size) { + return off % sector_size == 0; +} + +bool IsSectorAligned(const void* ptr, size_t sector_size) { + return uintptr_t(ptr) % sector_size == 0; +} + +} +#endif + +/* + * PosixSequentialFile + */ +PosixSequentialFile::PosixSequentialFile(const std::string& fname, FILE* file, + int fd, const EnvOptions& options) + : filename_(fname), + file_(file), + fd_(fd), + use_direct_io_(options.use_direct_reads), + logical_sector_size_(GetLogicalBufferSize(fd_)) { + assert(!options.use_direct_reads || !options.use_mmap_reads); +} + +PosixSequentialFile::~PosixSequentialFile() { + if (!use_direct_io()) { + assert(file_); + fclose(file_); + } else { + assert(fd_); + close(fd_); + } +} + +Status PosixSequentialFile::Read(size_t n, Slice* result, char* scratch) { + assert(result != nullptr && !use_direct_io()); + Status s; + size_t r = 0; + do { + r = fread_unlocked(scratch, 1, n, file_); + } while (r == 0 && ferror(file_) && errno == EINTR); + *result = Slice(scratch, r); + if (r < n) { + if (feof(file_)) { + // We leave status as ok if we hit the end of the file + // We also clear the error so that the reads can continue + // if a new data is written to the file + clearerr(file_); + } else { + // A partial read with an error: return a non-ok status + s = IOError("While reading file sequentially", filename_, errno); + } + } + return s; +} + +Status PosixSequentialFile::PositionedRead(uint64_t offset, size_t n, + Slice* result, char* scratch) { + if (use_direct_io()) { + assert(IsSectorAligned(offset, GetRequiredBufferAlignment())); + assert(IsSectorAligned(n, GetRequiredBufferAlignment())); + assert(IsSectorAligned(scratch, GetRequiredBufferAlignment())); + } + Status s; + ssize_t r = -1; + size_t left = n; + char* ptr = scratch; + assert(use_direct_io()); + while (left > 0) { + r = pread(fd_, ptr, left, static_cast(offset)); + if (r <= 0) { + if (r == -1 && errno == EINTR) { + continue; + } + break; + } + ptr += r; + offset += r; + left -= r; + if (r % static_cast(GetRequiredBufferAlignment()) != 0) { + // Bytes reads don't fill sectors. Should only happen at the end + // of the file. + break; + } + } + if (r < 0) { + // An error: return a non-ok status + s = IOError( + "While pread " + ToString(n) + " bytes from offset " + ToString(offset), + filename_, errno); + } + *result = Slice(scratch, (r < 0) ? 0 : n - left); + return s; +} + +Status PosixSequentialFile::Skip(uint64_t n) { + if (fseek(file_, static_cast(n), SEEK_CUR)) { + return IOError("While fseek to skip " + ToString(n) + " bytes", filename_, + errno); + } + return Status::OK(); +} + +Status PosixSequentialFile::InvalidateCache(size_t offset, size_t length) { +#ifndef OS_LINUX + return Status::OK(); +#else + if (!use_direct_io()) { + // free OS pages + int ret = Fadvise(fd_, offset, length, POSIX_FADV_DONTNEED); + if (ret != 0) { + return IOError("While fadvise NotNeeded offset " + ToString(offset) + + " len " + ToString(length), + filename_, errno); + } + } + return Status::OK(); +#endif +} + +/* + * PosixRandomAccessFile + */ +#if defined(OS_LINUX) +size_t PosixHelper::GetUniqueIdFromFile(int fd, char* id, size_t max_size) { + if (max_size < kMaxVarint64Length * 3) { + return 0; + } + + struct stat buf; + int result = fstat(fd, &buf); + assert(result != -1); + if (result == -1) { + return 0; + } + + long version = 0; + result = ioctl(fd, FS_IOC_GETVERSION, &version); + TEST_SYNC_POINT_CALLBACK("GetUniqueIdFromFile:FS_IOC_GETVERSION", &result); + if (result == -1) { + return 0; + } + uint64_t uversion = (uint64_t)version; + + char* rid = id; + rid = EncodeVarint64(rid, buf.st_dev); + rid = EncodeVarint64(rid, buf.st_ino); + rid = EncodeVarint64(rid, uversion); + assert(rid >= id); + return static_cast(rid - id); +} +#endif + +#if defined(OS_MACOSX) || defined(OS_AIX) +size_t PosixHelper::GetUniqueIdFromFile(int fd, char* id, size_t max_size) { + if (max_size < kMaxVarint64Length * 3) { + return 0; + } + + struct stat buf; + int result = fstat(fd, &buf); + if (result == -1) { + return 0; + } + + char* rid = id; + rid = EncodeVarint64(rid, buf.st_dev); + rid = EncodeVarint64(rid, buf.st_ino); + rid = EncodeVarint64(rid, buf.st_gen); + assert(rid >= id); + return static_cast(rid - id); +} +#endif +/* + * PosixRandomAccessFile + * + * pread() based random-access + */ +PosixRandomAccessFile::PosixRandomAccessFile(const std::string& fname, int fd, + const EnvOptions& options) + : filename_(fname), + fd_(fd), + use_direct_io_(options.use_direct_reads), + logical_sector_size_(GetLogicalBufferSize(fd_)) { + assert(!options.use_direct_reads || !options.use_mmap_reads); + assert(!options.use_mmap_reads || sizeof(void*) < 8); +} + +PosixRandomAccessFile::~PosixRandomAccessFile() { close(fd_); } + +Status PosixRandomAccessFile::Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + if (use_direct_io()) { + assert(IsSectorAligned(offset, GetRequiredBufferAlignment())); + assert(IsSectorAligned(n, GetRequiredBufferAlignment())); + assert(IsSectorAligned(scratch, GetRequiredBufferAlignment())); + } + Status s; + ssize_t r = -1; + size_t left = n; + char* ptr = scratch; + while (left > 0) { + r = pread(fd_, ptr, left, static_cast(offset)); + if (r <= 0) { + if (r == -1 && errno == EINTR) { + continue; + } + break; + } + ptr += r; + offset += r; + left -= r; + if (use_direct_io() && + r % static_cast(GetRequiredBufferAlignment()) != 0) { + // Bytes reads don't fill sectors. Should only happen at the end + // of the file. + break; + } + } + if (r < 0) { + // An error: return a non-ok status + s = IOError( + "While pread offset " + ToString(offset) + " len " + ToString(n), + filename_, errno); + } + *result = Slice(scratch, (r < 0) ? 0 : n - left); + return s; +} + +Status PosixRandomAccessFile::Prefetch(uint64_t offset, size_t n) { + Status s; + if (!use_direct_io()) { + ssize_t r = 0; +#ifdef OS_LINUX + r = readahead(fd_, offset, n); +#endif +#ifdef OS_MACOSX + radvisory advice; + advice.ra_offset = static_cast(offset); + advice.ra_count = static_cast(n); + r = fcntl(fd_, F_RDADVISE, &advice); +#endif + if (r == -1) { + s = IOError("While prefetching offset " + ToString(offset) + " len " + + ToString(n), + filename_, errno); + } + } + return s; +} + +#if defined(OS_LINUX) || defined(OS_MACOSX) || defined(OS_AIX) +size_t PosixRandomAccessFile::GetUniqueId(char* id, size_t max_size) const { + return PosixHelper::GetUniqueIdFromFile(fd_, id, max_size); +} +#endif + +void PosixRandomAccessFile::Hint(AccessPattern pattern) { + if (use_direct_io()) { + return; + } + switch (pattern) { + case NORMAL: + Fadvise(fd_, 0, 0, POSIX_FADV_NORMAL); + break; + case RANDOM: + Fadvise(fd_, 0, 0, POSIX_FADV_RANDOM); + break; + case SEQUENTIAL: + Fadvise(fd_, 0, 0, POSIX_FADV_SEQUENTIAL); + break; + case WILLNEED: + Fadvise(fd_, 0, 0, POSIX_FADV_WILLNEED); + break; + case DONTNEED: + Fadvise(fd_, 0, 0, POSIX_FADV_DONTNEED); + break; + default: + assert(false); + break; + } +} + +Status PosixRandomAccessFile::InvalidateCache(size_t offset, size_t length) { + if (use_direct_io()) { + return Status::OK(); + } +#ifndef OS_LINUX + return Status::OK(); +#else + // free OS pages + int ret = Fadvise(fd_, offset, length, POSIX_FADV_DONTNEED); + if (ret == 0) { + return Status::OK(); + } + return IOError("While fadvise NotNeeded offset " + ToString(offset) + + " len " + ToString(length), + filename_, errno); +#endif +} + +/* + * PosixMmapReadableFile + * + * mmap() based random-access + */ +// base[0,length-1] contains the mmapped contents of the file. +PosixMmapReadableFile::PosixMmapReadableFile(const int fd, + const std::string& fname, + void* base, size_t length, + const EnvOptions& options) + : fd_(fd), filename_(fname), mmapped_region_(base), length_(length) { + fd_ = fd_ + 0; // suppress the warning for used variables + assert(options.use_mmap_reads); + assert(!options.use_direct_reads); +} + +PosixMmapReadableFile::~PosixMmapReadableFile() { + int ret = munmap(mmapped_region_, length_); + if (ret != 0) { + fprintf(stdout, "failed to munmap %p length %" ROCKSDB_PRIszt " \n", + mmapped_region_, length_); + } +} + +Status PosixMmapReadableFile::Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + Status s; + if (offset > length_) { + *result = Slice(); + return IOError("While mmap read offset " + ToString(offset) + + " larger than file length " + ToString(length_), + filename_, EINVAL); + } else if (offset + n > length_) { + n = static_cast(length_ - offset); + } + *result = Slice(reinterpret_cast(mmapped_region_) + offset, n); + return s; +} + +Status PosixMmapReadableFile::InvalidateCache(size_t offset, size_t length) { +#ifndef OS_LINUX + return Status::OK(); +#else + // free OS pages + int ret = Fadvise(fd_, offset, length, POSIX_FADV_DONTNEED); + if (ret == 0) { + return Status::OK(); + } + return IOError("While fadvise not needed. Offset " + ToString(offset) + + " len" + ToString(length), + filename_, errno); +#endif +} + +/* + * PosixMmapFile + * + * We preallocate up to an extra megabyte and use memcpy to append new + * data to the file. This is safe since we either properly close the + * file before reading from it, or for log files, the reading code + * knows enough to skip zero suffixes. + */ +Status PosixMmapFile::UnmapCurrentRegion() { + TEST_KILL_RANDOM("PosixMmapFile::UnmapCurrentRegion:0", rocksdb_kill_odds); + if (base_ != nullptr) { + int munmap_status = munmap(base_, limit_ - base_); + if (munmap_status != 0) { + return IOError("While munmap", filename_, munmap_status); + } + file_offset_ += limit_ - base_; + base_ = nullptr; + limit_ = nullptr; + last_sync_ = nullptr; + dst_ = nullptr; + + // Increase the amount we map the next time, but capped at 1MB + if (map_size_ < (1 << 20)) { + map_size_ *= 2; + } + } + return Status::OK(); +} + +Status PosixMmapFile::MapNewRegion() { +#ifdef ROCKSDB_FALLOCATE_PRESENT + assert(base_ == nullptr); + TEST_KILL_RANDOM("PosixMmapFile::UnmapCurrentRegion:0", rocksdb_kill_odds); + // we can't fallocate with FALLOC_FL_KEEP_SIZE here + if (allow_fallocate_) { + IOSTATS_TIMER_GUARD(allocate_nanos); + int alloc_status = fallocate(fd_, 0, file_offset_, map_size_); + if (alloc_status != 0) { + // fallback to posix_fallocate + alloc_status = posix_fallocate(fd_, file_offset_, map_size_); + } + if (alloc_status != 0) { + return Status::IOError("Error allocating space to file : " + filename_ + + "Error : " + strerror(alloc_status)); + } + } + + TEST_KILL_RANDOM("PosixMmapFile::Append:1", rocksdb_kill_odds); + void* ptr = mmap(nullptr, map_size_, PROT_READ | PROT_WRITE, MAP_SHARED, fd_, + file_offset_); + if (ptr == MAP_FAILED) { + return Status::IOError("MMap failed on " + filename_); + } + TEST_KILL_RANDOM("PosixMmapFile::Append:2", rocksdb_kill_odds); + + base_ = reinterpret_cast(ptr); + limit_ = base_ + map_size_; + dst_ = base_; + last_sync_ = base_; + return Status::OK(); +#else + return Status::NotSupported("This platform doesn't support fallocate()"); +#endif +} + +Status PosixMmapFile::Msync() { + if (dst_ == last_sync_) { + return Status::OK(); + } + // Find the beginnings of the pages that contain the first and last + // bytes to be synced. + size_t p1 = TruncateToPageBoundary(last_sync_ - base_); + size_t p2 = TruncateToPageBoundary(dst_ - base_ - 1); + last_sync_ = dst_; + TEST_KILL_RANDOM("PosixMmapFile::Msync:0", rocksdb_kill_odds); + if (msync(base_ + p1, p2 - p1 + page_size_, MS_SYNC) < 0) { + return IOError("While msync", filename_, errno); + } + return Status::OK(); +} + +PosixMmapFile::PosixMmapFile(const std::string& fname, int fd, size_t page_size, + const EnvOptions& options) + : filename_(fname), + fd_(fd), + page_size_(page_size), + map_size_(Roundup(65536, page_size)), + base_(nullptr), + limit_(nullptr), + dst_(nullptr), + last_sync_(nullptr), + file_offset_(0) { +#ifdef ROCKSDB_FALLOCATE_PRESENT + allow_fallocate_ = options.allow_fallocate; + fallocate_with_keep_size_ = options.fallocate_with_keep_size; +#endif + assert((page_size & (page_size - 1)) == 0); + assert(options.use_mmap_writes); + assert(!options.use_direct_writes); +} + +PosixMmapFile::~PosixMmapFile() { + if (fd_ >= 0) { + PosixMmapFile::Close(); + } +} + +Status PosixMmapFile::Append(const Slice& data) { + const char* src = data.data(); + size_t left = data.size(); + while (left > 0) { + assert(base_ <= dst_); + assert(dst_ <= limit_); + size_t avail = limit_ - dst_; + if (avail == 0) { + Status s = UnmapCurrentRegion(); + if (!s.ok()) { + return s; + } + s = MapNewRegion(); + if (!s.ok()) { + return s; + } + TEST_KILL_RANDOM("PosixMmapFile::Append:0", rocksdb_kill_odds); + } + + size_t n = (left <= avail) ? left : avail; + assert(dst_); + memcpy(dst_, src, n); + dst_ += n; + src += n; + left -= n; + } + return Status::OK(); +} + +Status PosixMmapFile::Close() { + Status s; + size_t unused = limit_ - dst_; + + s = UnmapCurrentRegion(); + if (!s.ok()) { + s = IOError("While closing mmapped file", filename_, errno); + } else if (unused > 0) { + // Trim the extra space at the end of the file + if (ftruncate(fd_, file_offset_ - unused) < 0) { + s = IOError("While ftruncating mmaped file", filename_, errno); + } + } + + if (close(fd_) < 0) { + if (s.ok()) { + s = IOError("While closing mmapped file", filename_, errno); + } + } + + fd_ = -1; + base_ = nullptr; + limit_ = nullptr; + return s; +} + +Status PosixMmapFile::Flush() { return Status::OK(); } + +Status PosixMmapFile::Sync() { + if (fdatasync(fd_) < 0) { + return IOError("While fdatasync mmapped file", filename_, errno); + } + + return Msync(); +} + +/** + * Flush data as well as metadata to stable storage. + */ +Status PosixMmapFile::Fsync() { + if (fsync(fd_) < 0) { + return IOError("While fsync mmaped file", filename_, errno); + } + + return Msync(); +} + +/** + * Get the size of valid data in the file. This will not match the + * size that is returned from the filesystem because we use mmap + * to extend file by map_size every time. + */ +uint64_t PosixMmapFile::GetFileSize() { + size_t used = dst_ - base_; + return file_offset_ + used; +} + +Status PosixMmapFile::InvalidateCache(size_t offset, size_t length) { +#ifndef OS_LINUX + return Status::OK(); +#else + // free OS pages + int ret = Fadvise(fd_, offset, length, POSIX_FADV_DONTNEED); + if (ret == 0) { + return Status::OK(); + } + return IOError("While fadvise NotNeeded mmapped file", filename_, errno); +#endif +} + +#ifdef ROCKSDB_FALLOCATE_PRESENT +Status PosixMmapFile::Allocate(uint64_t offset, uint64_t len) { + assert(offset <= std::numeric_limits::max()); + assert(len <= std::numeric_limits::max()); + TEST_KILL_RANDOM("PosixMmapFile::Allocate:0", rocksdb_kill_odds); + int alloc_status = 0; + if (allow_fallocate_) { + alloc_status = fallocate( + fd_, fallocate_with_keep_size_ ? FALLOC_FL_KEEP_SIZE : 0, + static_cast(offset), static_cast(len)); + } + if (alloc_status == 0) { + return Status::OK(); + } else { + return IOError( + "While fallocate offset " + ToString(offset) + " len " + ToString(len), + filename_, errno); + } +} +#endif + +/* + * PosixWritableFile + * + * Use posix write to write data to a file. + */ +PosixWritableFile::PosixWritableFile(const std::string& fname, int fd, + const EnvOptions& options) + : filename_(fname), + use_direct_io_(options.use_direct_writes), + fd_(fd), + filesize_(0), + logical_sector_size_(GetLogicalBufferSize(fd_)) { +#ifdef ROCKSDB_FALLOCATE_PRESENT + allow_fallocate_ = options.allow_fallocate; + fallocate_with_keep_size_ = options.fallocate_with_keep_size; +#endif + assert(!options.use_mmap_writes); +} + +PosixWritableFile::~PosixWritableFile() { + if (fd_ >= 0) { + PosixWritableFile::Close(); + } +} + +Status PosixWritableFile::Append(const Slice& data) { + if (use_direct_io()) { + assert(IsSectorAligned(data.size(), GetRequiredBufferAlignment())); + assert(IsSectorAligned(data.data(), GetRequiredBufferAlignment())); + } + const char* src = data.data(); + size_t left = data.size(); + while (left != 0) { + ssize_t done = write(fd_, src, left); + if (done < 0) { + if (errno == EINTR) { + continue; + } + return IOError("While appending to file", filename_, errno); + } + left -= done; + src += done; + } + filesize_ += data.size(); + return Status::OK(); +} + +Status PosixWritableFile::PositionedAppend(const Slice& data, uint64_t offset) { + if (use_direct_io()) { + assert(IsSectorAligned(offset, GetRequiredBufferAlignment())); + assert(IsSectorAligned(data.size(), GetRequiredBufferAlignment())); + assert(IsSectorAligned(data.data(), GetRequiredBufferAlignment())); + } + assert(offset <= std::numeric_limits::max()); + const char* src = data.data(); + size_t left = data.size(); + while (left != 0) { + ssize_t done = pwrite(fd_, src, left, static_cast(offset)); + if (done < 0) { + if (errno == EINTR) { + continue; + } + return IOError("While pwrite to file at offset " + ToString(offset), + filename_, errno); + } + left -= done; + offset += done; + src += done; + } + filesize_ = offset; + return Status::OK(); +} + +Status PosixWritableFile::Truncate(uint64_t size) { + Status s; + int r = ftruncate(fd_, size); + if (r < 0) { + s = IOError("While ftruncate file to size " + ToString(size), filename_, + errno); + } else { + filesize_ = size; + } + return s; +} + +Status PosixWritableFile::Close() { + Status s; + + size_t block_size; + size_t last_allocated_block; + GetPreallocationStatus(&block_size, &last_allocated_block); + if (last_allocated_block > 0) { + // trim the extra space preallocated at the end of the file + // NOTE(ljin): we probably don't want to surface failure as an IOError, + // but it will be nice to log these errors. + int dummy __attribute__((unused)); + dummy = ftruncate(fd_, filesize_); +#if defined(ROCKSDB_FALLOCATE_PRESENT) && !defined(TRAVIS) + // in some file systems, ftruncate only trims trailing space if the + // new file size is smaller than the current size. Calling fallocate + // with FALLOC_FL_PUNCH_HOLE flag to explicitly release these unused + // blocks. FALLOC_FL_PUNCH_HOLE is supported on at least the following + // filesystems: + // XFS (since Linux 2.6.38) + // ext4 (since Linux 3.0) + // Btrfs (since Linux 3.7) + // tmpfs (since Linux 3.5) + // We ignore error since failure of this operation does not affect + // correctness. + // TRAVIS - this code does not work on TRAVIS filesystems. + // the FALLOC_FL_KEEP_SIZE option is expected to not change the size + // of the file, but it does. Simple strace report will show that. + // While we work with Travis-CI team to figure out if this is a + // quirk of Docker/AUFS, we will comment this out. + struct stat file_stats; + int result = fstat(fd_, &file_stats); + // After ftruncate, we check whether ftruncate has the correct behavior. + // If not, we should hack it with FALLOC_FL_PUNCH_HOLE + if (result == 0 && + (file_stats.st_size + file_stats.st_blksize - 1) / + file_stats.st_blksize != + file_stats.st_blocks / (file_stats.st_blksize / 512)) { + IOSTATS_TIMER_GUARD(allocate_nanos); + if (allow_fallocate_) { + fallocate(fd_, FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE, filesize_, + block_size * last_allocated_block - filesize_); + } + } +#endif + } + + if (close(fd_) < 0) { + s = IOError("While closing file after writing", filename_, errno); + } + fd_ = -1; + return s; +} + +// write out the cached data to the OS cache +Status PosixWritableFile::Flush() { return Status::OK(); } + +Status PosixWritableFile::Sync() { + if (fdatasync(fd_) < 0) { + return IOError("While fdatasync", filename_, errno); + } + return Status::OK(); +} + +Status PosixWritableFile::Fsync() { + if (fsync(fd_) < 0) { + return IOError("While fsync", filename_, errno); + } + return Status::OK(); +} + +bool PosixWritableFile::IsSyncThreadSafe() const { return true; } + +uint64_t PosixWritableFile::GetFileSize() { return filesize_; } + +Status PosixWritableFile::InvalidateCache(size_t offset, size_t length) { + if (use_direct_io()) { + return Status::OK(); + } +#ifndef OS_LINUX + return Status::OK(); +#else + // free OS pages + int ret = Fadvise(fd_, offset, length, POSIX_FADV_DONTNEED); + if (ret == 0) { + return Status::OK(); + } + return IOError("While fadvise NotNeeded", filename_, errno); +#endif +} + +#ifdef ROCKSDB_FALLOCATE_PRESENT +Status PosixWritableFile::Allocate(uint64_t offset, uint64_t len) { + assert(offset <= std::numeric_limits::max()); + assert(len <= std::numeric_limits::max()); + TEST_KILL_RANDOM("PosixWritableFile::Allocate:0", rocksdb_kill_odds); + IOSTATS_TIMER_GUARD(allocate_nanos); + int alloc_status = 0; + if (allow_fallocate_) { + alloc_status = fallocate( + fd_, fallocate_with_keep_size_ ? FALLOC_FL_KEEP_SIZE : 0, + static_cast(offset), static_cast(len)); + } + if (alloc_status == 0) { + return Status::OK(); + } else { + return IOError( + "While fallocate offset " + ToString(offset) + " len " + ToString(len), + filename_, errno); + } +} +#endif + +#ifdef ROCKSDB_RANGESYNC_PRESENT +Status PosixWritableFile::RangeSync(uint64_t offset, uint64_t nbytes) { + assert(offset <= std::numeric_limits::max()); + assert(nbytes <= std::numeric_limits::max()); + if (sync_file_range(fd_, static_cast(offset), + static_cast(nbytes), SYNC_FILE_RANGE_WRITE) == 0) { + return Status::OK(); + } else { + return IOError("While sync_file_range offset " + ToString(offset) + + " bytes " + ToString(nbytes), + filename_, errno); + } +} +#endif + +#ifdef OS_LINUX +size_t PosixWritableFile::GetUniqueId(char* id, size_t max_size) const { + return PosixHelper::GetUniqueIdFromFile(fd_, id, max_size); +} +#endif + +/* + * PosixRandomRWFile + */ + +PosixRandomRWFile::PosixRandomRWFile(const std::string& fname, int fd, + const EnvOptions& options) + : filename_(fname), fd_(fd) {} + +PosixRandomRWFile::~PosixRandomRWFile() { + if (fd_ >= 0) { + Close(); + } +} + +Status PosixRandomRWFile::Write(uint64_t offset, const Slice& data) { + const char* src = data.data(); + size_t left = data.size(); + while (left != 0) { + ssize_t done = pwrite(fd_, src, left, offset); + if (done < 0) { + // error while writing to file + if (errno == EINTR) { + // write was interrupted, try again. + continue; + } + return IOError( + "While write random read/write file at offset " + ToString(offset), + filename_, errno); + } + + // Wrote `done` bytes + left -= done; + offset += done; + src += done; + } + + return Status::OK(); +} + +Status PosixRandomRWFile::Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + size_t left = n; + char* ptr = scratch; + while (left > 0) { + ssize_t done = pread(fd_, ptr, left, offset); + if (done < 0) { + // error while reading from file + if (errno == EINTR) { + // read was interrupted, try again. + continue; + } + return IOError("While reading random read/write file offset " + + ToString(offset) + " len " + ToString(n), + filename_, errno); + } else if (done == 0) { + // Nothing more to read + break; + } + + // Read `done` bytes + ptr += done; + offset += done; + left -= done; + } + + *result = Slice(scratch, n - left); + return Status::OK(); +} + +Status PosixRandomRWFile::Flush() { return Status::OK(); } + +Status PosixRandomRWFile::Sync() { + if (fdatasync(fd_) < 0) { + return IOError("While fdatasync random read/write file", filename_, errno); + } + return Status::OK(); +} + +Status PosixRandomRWFile::Fsync() { + if (fsync(fd_) < 0) { + return IOError("While fsync random read/write file", filename_, errno); + } + return Status::OK(); +} + +Status PosixRandomRWFile::Close() { + if (close(fd_) < 0) { + return IOError("While close random read/write file", filename_, errno); + } + fd_ = -1; + return Status::OK(); +} + +/* + * PosixDirectory + */ + +PosixDirectory::~PosixDirectory() { close(fd_); } + +Status PosixDirectory::Fsync() { +#ifndef OS_AIX + if (fsync(fd_) == -1) { + return IOError("While fsync", "a directory", errno); + } +#endif + return Status::OK(); +} +} // namespace rocksdb +#endif diff --git a/env/io_posix.h b/env/io_posix.h new file mode 100644 index 00000000000..69c98438f27 --- /dev/null +++ b/env/io_posix.h @@ -0,0 +1,248 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#pragma once +#include +#include +#include +#include +#include "rocksdb/env.h" + +// For non linux platform, the following macros are used only as place +// holder. +#if !(defined OS_LINUX) && !(defined CYGWIN) && !(defined OS_AIX) +#define POSIX_FADV_NORMAL 0 /* [MC1] no further special treatment */ +#define POSIX_FADV_RANDOM 1 /* [MC1] expect random page refs */ +#define POSIX_FADV_SEQUENTIAL 2 /* [MC1] expect sequential page refs */ +#define POSIX_FADV_WILLNEED 3 /* [MC1] will need these pages */ +#define POSIX_FADV_DONTNEED 4 /* [MC1] dont need these pages */ +#endif + +namespace rocksdb { +static std::string IOErrorMsg(const std::string& context, + const std::string& file_name) { + if (file_name.empty()) { + return context; + } + return context + ": " + file_name; +} + +// file_name can be left empty if it is not unkown. +static Status IOError(const std::string& context, const std::string& file_name, + int err_number) { + switch (err_number) { + case ENOSPC: + return Status::NoSpace(IOErrorMsg(context, file_name), + strerror(err_number)); + case ESTALE: + return Status::IOError(Status::kStaleFile); + default: + return Status::IOError(IOErrorMsg(context, file_name), + strerror(err_number)); + } +} + +class PosixHelper { + public: + static size_t GetUniqueIdFromFile(int fd, char* id, size_t max_size); +}; + +class PosixSequentialFile : public SequentialFile { + private: + std::string filename_; + FILE* file_; + int fd_; + bool use_direct_io_; + size_t logical_sector_size_; + + public: + PosixSequentialFile(const std::string& fname, FILE* file, int fd, + const EnvOptions& options); + virtual ~PosixSequentialFile(); + + virtual Status Read(size_t n, Slice* result, char* scratch) override; + virtual Status PositionedRead(uint64_t offset, size_t n, Slice* result, + char* scratch) override; + virtual Status Skip(uint64_t n) override; + virtual Status InvalidateCache(size_t offset, size_t length) override; + virtual bool use_direct_io() const override { return use_direct_io_; } + virtual size_t GetRequiredBufferAlignment() const override { + return logical_sector_size_; + } +}; + +class PosixRandomAccessFile : public RandomAccessFile { + protected: + std::string filename_; + int fd_; + bool use_direct_io_; + size_t logical_sector_size_; + + public: + PosixRandomAccessFile(const std::string& fname, int fd, + const EnvOptions& options); + virtual ~PosixRandomAccessFile(); + + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const override; + + virtual Status Prefetch(uint64_t offset, size_t n) override; + +#if defined(OS_LINUX) || defined(OS_MACOSX) || defined(OS_AIX) + virtual size_t GetUniqueId(char* id, size_t max_size) const override; +#endif + virtual void Hint(AccessPattern pattern) override; + virtual Status InvalidateCache(size_t offset, size_t length) override; + virtual bool use_direct_io() const override { return use_direct_io_; } + virtual size_t GetRequiredBufferAlignment() const override { + return logical_sector_size_; + } +}; + +class PosixWritableFile : public WritableFile { + protected: + const std::string filename_; + const bool use_direct_io_; + int fd_; + uint64_t filesize_; + size_t logical_sector_size_; +#ifdef ROCKSDB_FALLOCATE_PRESENT + bool allow_fallocate_; + bool fallocate_with_keep_size_; +#endif + + public: + explicit PosixWritableFile(const std::string& fname, int fd, + const EnvOptions& options); + virtual ~PosixWritableFile(); + + // Need to implement this so the file is truncated correctly + // with direct I/O + virtual Status Truncate(uint64_t size) override; + virtual Status Close() override; + virtual Status Append(const Slice& data) override; + virtual Status PositionedAppend(const Slice& data, uint64_t offset) override; + virtual Status Flush() override; + virtual Status Sync() override; + virtual Status Fsync() override; + virtual bool IsSyncThreadSafe() const override; + virtual bool use_direct_io() const override { return use_direct_io_; } + virtual uint64_t GetFileSize() override; + virtual Status InvalidateCache(size_t offset, size_t length) override; + virtual size_t GetRequiredBufferAlignment() const override { + return logical_sector_size_; + } +#ifdef ROCKSDB_FALLOCATE_PRESENT + virtual Status Allocate(uint64_t offset, uint64_t len) override; +#endif +#ifdef ROCKSDB_RANGESYNC_PRESENT + virtual Status RangeSync(uint64_t offset, uint64_t nbytes) override; +#endif +#ifdef OS_LINUX + virtual size_t GetUniqueId(char* id, size_t max_size) const override; +#endif +}; + +// mmap() based random-access +class PosixMmapReadableFile : public RandomAccessFile { + private: + int fd_; + std::string filename_; + void* mmapped_region_; + size_t length_; + + public: + PosixMmapReadableFile(const int fd, const std::string& fname, void* base, + size_t length, const EnvOptions& options); + virtual ~PosixMmapReadableFile(); + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const override; + virtual Status InvalidateCache(size_t offset, size_t length) override; +}; + +class PosixMmapFile : public WritableFile { + private: + std::string filename_; + int fd_; + size_t page_size_; + size_t map_size_; // How much extra memory to map at a time + char* base_; // The mapped region + char* limit_; // Limit of the mapped region + char* dst_; // Where to write next (in range [base_,limit_]) + char* last_sync_; // Where have we synced up to + uint64_t file_offset_; // Offset of base_ in file +#ifdef ROCKSDB_FALLOCATE_PRESENT + bool allow_fallocate_; // If false, fallocate calls are bypassed + bool fallocate_with_keep_size_; +#endif + + // Roundup x to a multiple of y + static size_t Roundup(size_t x, size_t y) { return ((x + y - 1) / y) * y; } + + size_t TruncateToPageBoundary(size_t s) { + s -= (s & (page_size_ - 1)); + assert((s % page_size_) == 0); + return s; + } + + Status MapNewRegion(); + Status UnmapCurrentRegion(); + Status Msync(); + + public: + PosixMmapFile(const std::string& fname, int fd, size_t page_size, + const EnvOptions& options); + ~PosixMmapFile(); + + // Means Close() will properly take care of truncate + // and it does not need any additional information + virtual Status Truncate(uint64_t size) override { return Status::OK(); } + virtual Status Close() override; + virtual Status Append(const Slice& data) override; + virtual Status Flush() override; + virtual Status Sync() override; + virtual Status Fsync() override; + virtual uint64_t GetFileSize() override; + virtual Status InvalidateCache(size_t offset, size_t length) override; +#ifdef ROCKSDB_FALLOCATE_PRESENT + virtual Status Allocate(uint64_t offset, uint64_t len) override; +#endif +}; + +class PosixRandomRWFile : public RandomRWFile { + public: + explicit PosixRandomRWFile(const std::string& fname, int fd, + const EnvOptions& options); + virtual ~PosixRandomRWFile(); + + virtual Status Write(uint64_t offset, const Slice& data) override; + + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const override; + + virtual Status Flush() override; + virtual Status Sync() override; + virtual Status Fsync() override; + virtual Status Close() override; + + private: + const std::string filename_; + int fd_; +}; + +class PosixDirectory : public Directory { + public: + explicit PosixDirectory(int fd) : fd_(fd) {} + ~PosixDirectory(); + virtual Status Fsync() override; + + private: + int fd_; +}; + +} // namespace rocksdb diff --git a/env/mock_env.cc b/env/mock_env.cc new file mode 100644 index 00000000000..669011c4ee4 --- /dev/null +++ b/env/mock_env.cc @@ -0,0 +1,799 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "env/mock_env.h" +#include +#include +#include "port/sys_time.h" +#include "util/cast_util.h" +#include "util/murmurhash.h" +#include "util/random.h" +#include "util/rate_limiter.h" + +namespace rocksdb { + +class MemFile { + public: + explicit MemFile(Env* env, const std::string& fn, bool _is_lock_file = false) + : env_(env), + fn_(fn), + refs_(0), + is_lock_file_(_is_lock_file), + locked_(false), + size_(0), + modified_time_(Now()), + rnd_(static_cast( + MurmurHash(fn.data(), static_cast(fn.size()), 0))), + fsynced_bytes_(0) {} + + void Ref() { + MutexLock lock(&mutex_); + ++refs_; + } + + bool is_lock_file() const { return is_lock_file_; } + + bool Lock() { + assert(is_lock_file_); + MutexLock lock(&mutex_); + if (locked_) { + return false; + } else { + locked_ = true; + return true; + } + } + + void Unlock() { + assert(is_lock_file_); + MutexLock lock(&mutex_); + locked_ = false; + } + + void Unref() { + bool do_delete = false; + { + MutexLock lock(&mutex_); + --refs_; + assert(refs_ >= 0); + if (refs_ <= 0) { + do_delete = true; + } + } + + if (do_delete) { + delete this; + } + } + + uint64_t Size() const { + return size_; + } + + void Truncate(size_t size) { + MutexLock lock(&mutex_); + if (size < size_) { + data_.resize(size); + size_ = size; + } + } + + void CorruptBuffer() { + if (fsynced_bytes_ >= size_) { + return; + } + uint64_t buffered_bytes = size_ - fsynced_bytes_; + uint64_t start = + fsynced_bytes_ + rnd_.Uniform(static_cast(buffered_bytes)); + uint64_t end = std::min(start + 512, size_.load()); + MutexLock lock(&mutex_); + for (uint64_t pos = start; pos < end; ++pos) { + data_[pos] = static_cast(rnd_.Uniform(256)); + } + } + + Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const { + MutexLock lock(&mutex_); + const uint64_t available = Size() - std::min(Size(), offset); + if (n > available) { + n = available; + } + if (n == 0) { + *result = Slice(); + return Status::OK(); + } + if (scratch) { + memcpy(scratch, &(data_[offset]), n); + *result = Slice(scratch, n); + } else { + *result = Slice(&(data_[offset]), n); + } + return Status::OK(); + } + + Status Write(uint64_t offset, const Slice& data) { + MutexLock lock(&mutex_); + if (offset + data.size() > data_.size()) { + data_.resize(offset + data.size()); + } + data_.replace(offset, data.size(), data.data(), data.size()); + size_ = data_.size(); + modified_time_ = Now(); + return Status::OK(); + } + + Status Append(const Slice& data) { + MutexLock lock(&mutex_); + data_.append(data.data(), data.size()); + size_ = data_.size(); + modified_time_ = Now(); + return Status::OK(); + } + + Status Fsync() { + fsynced_bytes_ = size_.load(); + return Status::OK(); + } + + uint64_t ModifiedTime() const { + return modified_time_; + } + + private: + uint64_t Now() { + int64_t unix_time = 0; + auto s = env_->GetCurrentTime(&unix_time); + assert(s.ok()); + return static_cast(unix_time); + } + + // Private since only Unref() should be used to delete it. + ~MemFile() { + assert(refs_ == 0); + } + + // No copying allowed. + MemFile(const MemFile&); + void operator=(const MemFile&); + + Env* env_; + const std::string fn_; + mutable port::Mutex mutex_; + int refs_; + bool is_lock_file_; + bool locked_; + + // Data written into this file, all bytes before fsynced_bytes are + // persistent. + std::string data_; + std::atomic size_; + std::atomic modified_time_; + + Random rnd_; + std::atomic fsynced_bytes_; +}; + +namespace { + +class MockSequentialFile : public SequentialFile { + public: + explicit MockSequentialFile(MemFile* file) : file_(file), pos_(0) { + file_->Ref(); + } + + ~MockSequentialFile() { + file_->Unref(); + } + + virtual Status Read(size_t n, Slice* result, char* scratch) override { + Status s = file_->Read(pos_, n, result, scratch); + if (s.ok()) { + pos_ += result->size(); + } + return s; + } + + virtual Status Skip(uint64_t n) override { + if (pos_ > file_->Size()) { + return Status::IOError("pos_ > file_->Size()"); + } + const size_t available = file_->Size() - pos_; + if (n > available) { + n = available; + } + pos_ += n; + return Status::OK(); + } + + private: + MemFile* file_; + size_t pos_; +}; + +class MockRandomAccessFile : public RandomAccessFile { + public: + explicit MockRandomAccessFile(MemFile* file) : file_(file) { + file_->Ref(); + } + + ~MockRandomAccessFile() { + file_->Unref(); + } + + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const override { + return file_->Read(offset, n, result, scratch); + } + + private: + MemFile* file_; +}; + +class MockRandomRWFile : public RandomRWFile { + public: + explicit MockRandomRWFile(MemFile* file) : file_(file) { file_->Ref(); } + + ~MockRandomRWFile() { file_->Unref(); } + + virtual Status Write(uint64_t offset, const Slice& data) override { + return file_->Write(offset, data); + } + + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const override { + return file_->Read(offset, n, result, scratch); + } + + virtual Status Close() override { return file_->Fsync(); } + + virtual Status Flush() override { return Status::OK(); } + + virtual Status Sync() override { return file_->Fsync(); } + + private: + MemFile* file_; +}; + +class MockWritableFile : public WritableFile { + public: + MockWritableFile(MemFile* file, RateLimiter* rate_limiter) + : file_(file), + rate_limiter_(rate_limiter) { + file_->Ref(); + } + + ~MockWritableFile() { + file_->Unref(); + } + + virtual Status Append(const Slice& data) override { + uint64_t bytes_written = 0; + while (bytes_written < data.size()) { + auto bytes = RequestToken(data.size() - bytes_written); + Status s = file_->Append(Slice(data.data() + bytes_written, bytes)); + if (!s.ok()) { + return s; + } + bytes_written += bytes; + } + return Status::OK(); + } + virtual Status Truncate(uint64_t size) override { + file_->Truncate(size); + return Status::OK(); + } + virtual Status Close() override { return file_->Fsync(); } + + virtual Status Flush() override { return Status::OK(); } + + virtual Status Sync() override { return file_->Fsync(); } + + virtual uint64_t GetFileSize() override { return file_->Size(); } + + private: + inline size_t RequestToken(size_t bytes) { + if (rate_limiter_ && io_priority_ < Env::IO_TOTAL) { + bytes = std::min(bytes, + static_cast(rate_limiter_->GetSingleBurstBytes())); + rate_limiter_->Request(bytes, io_priority_); + } + return bytes; + } + + MemFile* file_; + RateLimiter* rate_limiter_; +}; + +class MockEnvDirectory : public Directory { + public: + virtual Status Fsync() override { return Status::OK(); } +}; + +class MockEnvFileLock : public FileLock { + public: + explicit MockEnvFileLock(const std::string& fname) + : fname_(fname) {} + + std::string FileName() const { + return fname_; + } + + private: + const std::string fname_; +}; + +class TestMemLogger : public Logger { + private: + std::unique_ptr file_; + std::atomic_size_t log_size_; + static const uint64_t flush_every_seconds_ = 5; + std::atomic_uint_fast64_t last_flush_micros_; + Env* env_; + bool flush_pending_; + + public: + TestMemLogger(std::unique_ptr f, Env* env, + const InfoLogLevel log_level = InfoLogLevel::ERROR_LEVEL) + : Logger(log_level), + file_(std::move(f)), + log_size_(0), + last_flush_micros_(0), + env_(env), + flush_pending_(false) {} + virtual ~TestMemLogger() { + } + + virtual void Flush() override { + if (flush_pending_) { + flush_pending_ = false; + } + last_flush_micros_ = env_->NowMicros(); + } + + using Logger::Logv; + virtual void Logv(const char* format, va_list ap) override { + // We try twice: the first time with a fixed-size stack allocated buffer, + // and the second time with a much larger dynamically allocated buffer. + char buffer[500]; + for (int iter = 0; iter < 2; iter++) { + char* base; + int bufsize; + if (iter == 0) { + bufsize = sizeof(buffer); + base = buffer; + } else { + bufsize = 30000; + base = new char[bufsize]; + } + char* p = base; + char* limit = base + bufsize; + + struct timeval now_tv; + gettimeofday(&now_tv, nullptr); + const time_t seconds = now_tv.tv_sec; + struct tm t; + memset(&t, 0, sizeof(t)); + auto ret __attribute__((__unused__)) = localtime_r(&seconds, &t); + assert(ret); + p += snprintf(p, limit - p, + "%04d/%02d/%02d-%02d:%02d:%02d.%06d ", + t.tm_year + 1900, + t.tm_mon + 1, + t.tm_mday, + t.tm_hour, + t.tm_min, + t.tm_sec, + static_cast(now_tv.tv_usec)); + + // Print the message + if (p < limit) { + va_list backup_ap; + va_copy(backup_ap, ap); + p += vsnprintf(p, limit - p, format, backup_ap); + va_end(backup_ap); + } + + // Truncate to available space if necessary + if (p >= limit) { + if (iter == 0) { + continue; // Try again with larger buffer + } else { + p = limit - 1; + } + } + + // Add newline if necessary + if (p == base || p[-1] != '\n') { + *p++ = '\n'; + } + + assert(p <= limit); + const size_t write_size = p - base; + + file_->Append(Slice(base, write_size)); + flush_pending_ = true; + log_size_ += write_size; + uint64_t now_micros = static_cast(now_tv.tv_sec) * 1000000 + + now_tv.tv_usec; + if (now_micros - last_flush_micros_ >= flush_every_seconds_ * 1000000) { + flush_pending_ = false; + last_flush_micros_ = now_micros; + } + if (base != buffer) { + delete[] base; + } + break; + } + } + size_t GetLogFileSize() const override { return log_size_; } +}; + +} // Anonymous namespace + +MockEnv::MockEnv(Env* base_env) : EnvWrapper(base_env), fake_sleep_micros_(0) {} + +MockEnv::~MockEnv() { + for (FileSystem::iterator i = file_map_.begin(); i != file_map_.end(); ++i) { + i->second->Unref(); + } +} + + // Partial implementation of the Env interface. +Status MockEnv::NewSequentialFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& soptions) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + if (file_map_.find(fn) == file_map_.end()) { + *result = NULL; + return Status::IOError(fn, "File not found"); + } + auto* f = file_map_[fn]; + if (f->is_lock_file()) { + return Status::InvalidArgument(fn, "Cannot open a lock file."); + } + result->reset(new MockSequentialFile(f)); + return Status::OK(); +} + +Status MockEnv::NewRandomAccessFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& soptions) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + if (file_map_.find(fn) == file_map_.end()) { + *result = NULL; + return Status::IOError(fn, "File not found"); + } + auto* f = file_map_[fn]; + if (f->is_lock_file()) { + return Status::InvalidArgument(fn, "Cannot open a lock file."); + } + result->reset(new MockRandomAccessFile(f)); + return Status::OK(); +} + +Status MockEnv::NewRandomRWFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& soptions) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + if (file_map_.find(fn) == file_map_.end()) { + *result = NULL; + return Status::IOError(fn, "File not found"); + } + auto* f = file_map_[fn]; + if (f->is_lock_file()) { + return Status::InvalidArgument(fn, "Cannot open a lock file."); + } + result->reset(new MockRandomRWFile(f)); + return Status::OK(); +} + +Status MockEnv::ReuseWritableFile(const std::string& fname, + const std::string& old_fname, + unique_ptr* result, + const EnvOptions& options) { + auto s = RenameFile(old_fname, fname); + if (!s.ok()) { + return s; + } + result->reset(); + return NewWritableFile(fname, result, options); +} + +Status MockEnv::NewWritableFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& env_options) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + if (file_map_.find(fn) != file_map_.end()) { + DeleteFileInternal(fn); + } + MemFile* file = new MemFile(this, fn, false); + file->Ref(); + file_map_[fn] = file; + + result->reset(new MockWritableFile(file, env_options.rate_limiter)); + return Status::OK(); +} + +Status MockEnv::NewDirectory(const std::string& name, + unique_ptr* result) { + result->reset(new MockEnvDirectory()); + return Status::OK(); +} + +Status MockEnv::FileExists(const std::string& fname) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + if (file_map_.find(fn) != file_map_.end()) { + // File exists + return Status::OK(); + } + // Now also check if fn exists as a dir + for (const auto& iter : file_map_) { + const std::string& filename = iter.first; + if (filename.size() >= fn.size() + 1 && + filename[fn.size()] == '/' && + Slice(filename).starts_with(Slice(fn))) { + return Status::OK(); + } + } + return Status::NotFound(); +} + +Status MockEnv::GetChildren(const std::string& dir, + std::vector* result) { + auto d = NormalizePath(dir); + bool found_dir = false; + { + MutexLock lock(&mutex_); + result->clear(); + for (const auto& iter : file_map_) { + const std::string& filename = iter.first; + + if (filename == d) { + found_dir = true; + } else if (filename.size() >= d.size() + 1 && filename[d.size()] == '/' && + Slice(filename).starts_with(Slice(d))) { + found_dir = true; + size_t next_slash = filename.find('/', d.size() + 1); + if (next_slash != std::string::npos) { + result->push_back(filename.substr( + d.size() + 1, next_slash - d.size() - 1)); + } else { + result->push_back(filename.substr(d.size() + 1)); + } + } + } + } + result->erase(std::unique(result->begin(), result->end()), result->end()); + return found_dir ? Status::OK() : Status::NotFound(); +} + +void MockEnv::DeleteFileInternal(const std::string& fname) { + assert(fname == NormalizePath(fname)); + const auto& pair = file_map_.find(fname); + if (pair != file_map_.end()) { + pair->second->Unref(); + file_map_.erase(fname); + } +} + +Status MockEnv::DeleteFile(const std::string& fname) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + if (file_map_.find(fn) == file_map_.end()) { + return Status::IOError(fn, "File not found"); + } + + DeleteFileInternal(fn); + return Status::OK(); +} + +Status MockEnv::CreateDir(const std::string& dirname) { + auto dn = NormalizePath(dirname); + if (file_map_.find(dn) == file_map_.end()) { + MemFile* file = new MemFile(this, dn, false); + file->Ref(); + file_map_[dn] = file; + } else { + return Status::IOError(); + } + return Status::OK(); +} + +Status MockEnv::CreateDirIfMissing(const std::string& dirname) { + CreateDir(dirname); + return Status::OK(); +} + +Status MockEnv::DeleteDir(const std::string& dirname) { + return DeleteFile(dirname); +} + +Status MockEnv::GetFileSize(const std::string& fname, uint64_t* file_size) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + auto iter = file_map_.find(fn); + if (iter == file_map_.end()) { + return Status::IOError(fn, "File not found"); + } + + *file_size = iter->second->Size(); + return Status::OK(); +} + +Status MockEnv::GetFileModificationTime(const std::string& fname, + uint64_t* time) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + auto iter = file_map_.find(fn); + if (iter == file_map_.end()) { + return Status::IOError(fn, "File not found"); + } + *time = iter->second->ModifiedTime(); + return Status::OK(); +} + +Status MockEnv::RenameFile(const std::string& src, const std::string& dest) { + auto s = NormalizePath(src); + auto t = NormalizePath(dest); + MutexLock lock(&mutex_); + if (file_map_.find(s) == file_map_.end()) { + return Status::IOError(s, "File not found"); + } + + DeleteFileInternal(t); + file_map_[t] = file_map_[s]; + file_map_.erase(s); + return Status::OK(); +} + +Status MockEnv::LinkFile(const std::string& src, const std::string& dest) { + auto s = NormalizePath(src); + auto t = NormalizePath(dest); + MutexLock lock(&mutex_); + if (file_map_.find(s) == file_map_.end()) { + return Status::IOError(s, "File not found"); + } + + DeleteFileInternal(t); + file_map_[t] = file_map_[s]; + file_map_[t]->Ref(); // Otherwise it might get deleted when noone uses s + return Status::OK(); +} + +Status MockEnv::NewLogger(const std::string& fname, + shared_ptr* result) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + auto iter = file_map_.find(fn); + MemFile* file = nullptr; + if (iter == file_map_.end()) { + file = new MemFile(this, fn, false); + file->Ref(); + file_map_[fn] = file; + } else { + file = iter->second; + } + std::unique_ptr f(new MockWritableFile(file, nullptr)); + result->reset(new TestMemLogger(std::move(f), this)); + return Status::OK(); +} + +Status MockEnv::LockFile(const std::string& fname, FileLock** flock) { + auto fn = NormalizePath(fname); + { + MutexLock lock(&mutex_); + if (file_map_.find(fn) != file_map_.end()) { + if (!file_map_[fn]->is_lock_file()) { + return Status::InvalidArgument(fname, "Not a lock file."); + } + if (!file_map_[fn]->Lock()) { + return Status::IOError(fn, "Lock is already held."); + } + } else { + auto* file = new MemFile(this, fn, true); + file->Ref(); + file->Lock(); + file_map_[fn] = file; + } + } + *flock = new MockEnvFileLock(fn); + return Status::OK(); +} + +Status MockEnv::UnlockFile(FileLock* flock) { + std::string fn = + static_cast_with_check(flock)->FileName(); + { + MutexLock lock(&mutex_); + if (file_map_.find(fn) != file_map_.end()) { + if (!file_map_[fn]->is_lock_file()) { + return Status::InvalidArgument(fn, "Not a lock file."); + } + file_map_[fn]->Unlock(); + } + } + delete flock; + return Status::OK(); +} + +Status MockEnv::GetTestDirectory(std::string* path) { + *path = "/test"; + return Status::OK(); +} + +Status MockEnv::GetCurrentTime(int64_t* unix_time) { + auto s = EnvWrapper::GetCurrentTime(unix_time); + if (s.ok()) { + *unix_time += fake_sleep_micros_.load() / (1000 * 1000); + } + return s; +} + +uint64_t MockEnv::NowMicros() { + return EnvWrapper::NowMicros() + fake_sleep_micros_.load(); +} + +uint64_t MockEnv::NowNanos() { + return EnvWrapper::NowNanos() + fake_sleep_micros_.load() * 1000; +} + +// Non-virtual functions, specific to MockEnv +Status MockEnv::Truncate(const std::string& fname, size_t size) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + auto iter = file_map_.find(fn); + if (iter == file_map_.end()) { + return Status::IOError(fn, "File not found"); + } + iter->second->Truncate(size); + return Status::OK(); +} + +Status MockEnv::CorruptBuffer(const std::string& fname) { + auto fn = NormalizePath(fname); + MutexLock lock(&mutex_); + auto iter = file_map_.find(fn); + if (iter == file_map_.end()) { + return Status::IOError(fn, "File not found"); + } + iter->second->CorruptBuffer(); + return Status::OK(); +} + +std::string MockEnv::NormalizePath(const std::string path) { + std::string dst; + for (auto c : path) { + if (!dst.empty() && c == '/' && dst.back() == '/') { + continue; + } + dst.push_back(c); + } + return dst; +} + +void MockEnv::FakeSleepForMicroseconds(int64_t micros) { + fake_sleep_micros_.fetch_add(micros); +} + +#ifndef ROCKSDB_LITE +// This is to maintain the behavior before swithcing from InMemoryEnv to MockEnv +Env* NewMemEnv(Env* base_env) { return new MockEnv(base_env); } + +#else // ROCKSDB_LITE + +Env* NewMemEnv(Env* base_env) { return nullptr; } + +#endif // !ROCKSDB_LITE + +} // namespace rocksdb diff --git a/env/mock_env.h b/env/mock_env.h new file mode 100644 index 00000000000..ba1e5fa31e7 --- /dev/null +++ b/env/mock_env.h @@ -0,0 +1,115 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#pragma once + +#include +#include +#include +#include +#include "rocksdb/env.h" +#include "rocksdb/status.h" +#include "port/port.h" +#include "util/mutexlock.h" + +namespace rocksdb { + +class MemFile; +class MockEnv : public EnvWrapper { + public: + explicit MockEnv(Env* base_env); + + virtual ~MockEnv(); + + // Partial implementation of the Env interface. + virtual Status NewSequentialFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& soptions) override; + + virtual Status NewRandomAccessFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& soptions) override; + + virtual Status NewRandomRWFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override; + + virtual Status ReuseWritableFile(const std::string& fname, + const std::string& old_fname, + unique_ptr* result, + const EnvOptions& options) override; + + virtual Status NewWritableFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& env_options) override; + + virtual Status NewDirectory(const std::string& name, + unique_ptr* result) override; + + virtual Status FileExists(const std::string& fname) override; + + virtual Status GetChildren(const std::string& dir, + std::vector* result) override; + + void DeleteFileInternal(const std::string& fname); + + virtual Status DeleteFile(const std::string& fname) override; + + virtual Status CreateDir(const std::string& dirname) override; + + virtual Status CreateDirIfMissing(const std::string& dirname) override; + + virtual Status DeleteDir(const std::string& dirname) override; + + virtual Status GetFileSize(const std::string& fname, + uint64_t* file_size) override; + + virtual Status GetFileModificationTime(const std::string& fname, + uint64_t* time) override; + + virtual Status RenameFile(const std::string& src, + const std::string& target) override; + + virtual Status LinkFile(const std::string& src, + const std::string& target) override; + + virtual Status NewLogger(const std::string& fname, + shared_ptr* result) override; + + virtual Status LockFile(const std::string& fname, FileLock** flock) override; + + virtual Status UnlockFile(FileLock* flock) override; + + virtual Status GetTestDirectory(std::string* path) override; + + // Results of these can be affected by FakeSleepForMicroseconds() + virtual Status GetCurrentTime(int64_t* unix_time) override; + virtual uint64_t NowMicros() override; + virtual uint64_t NowNanos() override; + + // Non-virtual functions, specific to MockEnv + Status Truncate(const std::string& fname, size_t size); + + Status CorruptBuffer(const std::string& fname); + + // Doesn't really sleep, just affects output of GetCurrentTime(), NowMicros() + // and NowNanos() + void FakeSleepForMicroseconds(int64_t micros); + + private: + std::string NormalizePath(const std::string path); + + // Map from filenames to MemFile objects, representing a simple file system. + typedef std::map FileSystem; + port::Mutex mutex_; + FileSystem file_map_; // Protected by mutex_. + + std::atomic fake_sleep_micros_; +}; + +} // namespace rocksdb diff --git a/env/mock_env_test.cc b/env/mock_env_test.cc new file mode 100644 index 00000000000..19e259ccd85 --- /dev/null +++ b/env/mock_env_test.cc @@ -0,0 +1,85 @@ +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "env/mock_env.h" + +#include +#include + +#include "rocksdb/env.h" +#include "util/testharness.h" + +namespace rocksdb { + +class MockEnvTest : public testing::Test { + public: + MockEnv* env_; + const EnvOptions soptions_; + + MockEnvTest() + : env_(new MockEnv(Env::Default())) { + } + ~MockEnvTest() { + delete env_; + } +}; + +TEST_F(MockEnvTest, Corrupt) { + const std::string kGood = "this is a good string, synced to disk"; + const std::string kCorrupted = "this part may be corrupted"; + const std::string kFileName = "/dir/f"; + unique_ptr writable_file; + ASSERT_OK(env_->NewWritableFile(kFileName, &writable_file, soptions_)); + ASSERT_OK(writable_file->Append(kGood)); + ASSERT_TRUE(writable_file->GetFileSize() == kGood.size()); + + std::string scratch; + scratch.resize(kGood.size() + kCorrupted.size() + 16); + Slice result; + unique_ptr rand_file; + ASSERT_OK(env_->NewRandomAccessFile(kFileName, &rand_file, soptions_)); + ASSERT_OK(rand_file->Read(0, kGood.size(), &result, &(scratch[0]))); + ASSERT_EQ(result.compare(kGood), 0); + + // Sync + corrupt => no change + ASSERT_OK(writable_file->Fsync()); + ASSERT_OK(dynamic_cast(env_)->CorruptBuffer(kFileName)); + result.clear(); + ASSERT_OK(rand_file->Read(0, kGood.size(), &result, &(scratch[0]))); + ASSERT_EQ(result.compare(kGood), 0); + + // Add new data and corrupt it + ASSERT_OK(writable_file->Append(kCorrupted)); + ASSERT_TRUE(writable_file->GetFileSize() == kGood.size() + kCorrupted.size()); + result.clear(); + ASSERT_OK(rand_file->Read(kGood.size(), kCorrupted.size(), + &result, &(scratch[0]))); + ASSERT_EQ(result.compare(kCorrupted), 0); + // Corrupted + ASSERT_OK(dynamic_cast(env_)->CorruptBuffer(kFileName)); + result.clear(); + ASSERT_OK(rand_file->Read(kGood.size(), kCorrupted.size(), + &result, &(scratch[0]))); + ASSERT_NE(result.compare(kCorrupted), 0); +} + +TEST_F(MockEnvTest, FakeSleeping) { + int64_t now = 0; + auto s = env_->GetCurrentTime(&now); + ASSERT_OK(s); + env_->FakeSleepForMicroseconds(3 * 1000 * 1000); + int64_t after_sleep = 0; + s = env_->GetCurrentTime(&after_sleep); + ASSERT_OK(s); + auto delta = after_sleep - now; + // this will be true unless test runs for 2 seconds + ASSERT_TRUE(delta == 3 || delta == 4); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/util/posix_logger.h b/env/posix_logger.h similarity index 78% rename from util/posix_logger.h rename to env/posix_logger.h index 6aba769f1a4..3ec6f574a33 100644 --- a/util/posix_logger.h +++ b/env/posix_logger.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -13,20 +13,23 @@ #pragma once #include #include -#include +#include "port/sys_time.h" #include #include -#include + #ifdef OS_LINUX +#ifndef FALLOC_FL_KEEP_SIZE #include #endif -#include "rocksdb/env.h" +#endif + #include +#include "monitoring/iostats_context_imp.h" +#include "rocksdb/env.h" +#include "util/sync_point.h" namespace rocksdb { -const int kDebugLogChunkSize = 128 * 1024; - class PosixLogger : public Logger { private: FILE* file_; @@ -36,7 +39,7 @@ class PosixLogger : public Logger { const static uint64_t flush_every_seconds_ = 5; std::atomic_uint_fast64_t last_flush_micros_; Env* env_; - bool flush_pending_; + std::atomic flush_pending_; public: PosixLogger(FILE* f, uint64_t (*gettid)(), Env* env, const InfoLogLevel log_level = InfoLogLevel::ERROR_LEVEL) @@ -51,14 +54,20 @@ class PosixLogger : public Logger { virtual ~PosixLogger() { fclose(file_); } - virtual void Flush() { + virtual void Flush() override { + TEST_SYNC_POINT("PosixLogger::Flush:Begin1"); + TEST_SYNC_POINT("PosixLogger::Flush:Begin2"); if (flush_pending_) { flush_pending_ = false; fflush(file_); } last_flush_micros_ = env_->NowMicros(); } - virtual void Logv(const char* format, va_list ap) { + + using Logger::Logv; + virtual void Logv(const char* format, va_list ap) override { + IOSTATS_TIMER_GUARD(logger_nanos); + const uint64_t thread_id = (*gettid_)(); // We try twice: the first time with a fixed-size stack allocated buffer, @@ -71,7 +80,7 @@ class PosixLogger : public Logger { bufsize = sizeof(buffer); base = buffer; } else { - bufsize = 30000; + bufsize = 65536; base = new char[bufsize]; } char* p = base; @@ -119,18 +128,21 @@ class PosixLogger : public Logger { const size_t write_size = p - base; #ifdef ROCKSDB_FALLOCATE_PRESENT + const int kDebugLogChunkSize = 128 * 1024; + // If this write would cross a boundary of kDebugLogChunkSize // space, pre-allocate more space to avoid overly large // allocations from filesystem allocsize options. const size_t log_size = log_size_; - const int last_allocation_chunk = + const size_t last_allocation_chunk = ((kDebugLogChunkSize - 1 + log_size) / kDebugLogChunkSize); - const int desired_allocation_chunk = + const size_t desired_allocation_chunk = ((kDebugLogChunkSize - 1 + log_size + write_size) / kDebugLogChunkSize); if (last_allocation_chunk != desired_allocation_chunk) { - fallocate(fd_, FALLOC_FL_KEEP_SIZE, 0, - desired_allocation_chunk * kDebugLogChunkSize); + fallocate( + fd_, FALLOC_FL_KEEP_SIZE, 0, + static_cast(desired_allocation_chunk * kDebugLogChunkSize)); } #endif @@ -143,9 +155,7 @@ class PosixLogger : public Logger { uint64_t now_micros = static_cast(now_tv.tv_sec) * 1000000 + now_tv.tv_usec; if (now_micros - last_flush_micros_ >= flush_every_seconds_ * 1000000) { - flush_pending_ = false; - fflush(file_); - last_flush_micros_ = now_micros; + Flush(); } if (base != buffer) { delete[] base; @@ -153,9 +163,7 @@ class PosixLogger : public Logger { break; } } - size_t GetLogFileSize() const { - return log_size_; - } + size_t GetLogFileSize() const override { return log_size_; } }; } // namespace rocksdb diff --git a/examples/.gitignore b/examples/.gitignore index d3c22099a5a..b5a05e44a27 100644 --- a/examples/.gitignore +++ b/examples/.gitignore @@ -1,2 +1,8 @@ +c_simple_example column_families_example +compact_files_example +compaction_filter_example +optimistic_transaction_example +options_file_example simple_example +transaction_example diff --git a/examples/Makefile b/examples/Makefile index 2567fdf864a..57cd1a75a1c 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -1,9 +1,50 @@ -include ../build_config.mk +include ../make_config.mk -all: simple_example column_families_example +ifndef DISABLE_JEMALLOC + ifdef JEMALLOC + PLATFORM_CXXFLAGS += -DROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE + endif + EXEC_LDFLAGS := $(JEMALLOC_LIB) $(EXEC_LDFLAGS) -lpthread + PLATFORM_CXXFLAGS += $(JEMALLOC_INCLUDE) +endif -simple_example: simple_example.cc +ifneq ($(USE_RTTI), 1) + CXXFLAGS += -fno-rtti +endif + +.PHONY: clean librocksdb + +all: simple_example column_families_example compact_files_example c_simple_example optimistic_transaction_example transaction_example compaction_filter_example options_file_example + +simple_example: librocksdb simple_example.cc + $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) + +column_families_example: librocksdb column_families_example.cc $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) -column_families_example: column_families_example.cc +compaction_filter_example: librocksdb compaction_filter_example.cc $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) + +compact_files_example: librocksdb compact_files_example.cc + $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) + +.c.o: + $(CC) $(CFLAGS) -c $< -o $@ -I../include + +c_simple_example: librocksdb c_simple_example.o + $(CXX) $@.o -o$@ ../librocksdb.a $(PLATFORM_LDFLAGS) $(EXEC_LDFLAGS) + +optimistic_transaction_example: librocksdb optimistic_transaction_example.cc + $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) + +transaction_example: librocksdb transaction_example.cc + $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) + +options_file_example: librocksdb options_file_example.cc + $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) + +clean: + rm -rf ./simple_example ./column_families_example ./compact_files_example ./compaction_filter_example ./c_simple_example c_simple_example.o ./optimistic_transaction_example ./transaction_example ./options_file_example + +librocksdb: + cd .. && $(MAKE) static_lib diff --git a/examples/README.md b/examples/README.md index b07b3903a6a..f4ba2384b8f 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1 +1,2 @@ -Compile RocksDB first by executing `make static_lib` in parent dir +1. Compile RocksDB first by executing `make static_lib` in parent dir +2. Compile all examples: `cd examples/; make all` diff --git a/examples/c_simple_example.c b/examples/c_simple_example.c new file mode 100644 index 00000000000..5564361d1e6 --- /dev/null +++ b/examples/c_simple_example.c @@ -0,0 +1,79 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include +#include +#include +#include + +#include "rocksdb/c.h" + +#include // sysconf() - get CPU count + +const char DBPath[] = "/tmp/rocksdb_simple_example"; +const char DBBackupPath[] = "/tmp/rocksdb_simple_example_backup"; + +int main(int argc, char **argv) { + rocksdb_t *db; + rocksdb_backup_engine_t *be; + rocksdb_options_t *options = rocksdb_options_create(); + // Optimize RocksDB. This is the easiest way to + // get RocksDB to perform well + long cpus = sysconf(_SC_NPROCESSORS_ONLN); // get # of online cores + rocksdb_options_increase_parallelism(options, (int)(cpus)); + rocksdb_options_optimize_level_style_compaction(options, 0); + // create the DB if it's not already present + rocksdb_options_set_create_if_missing(options, 1); + + // open DB + char *err = NULL; + db = rocksdb_open(options, DBPath, &err); + assert(!err); + + // open Backup Engine that we will use for backing up our database + be = rocksdb_backup_engine_open(options, DBBackupPath, &err); + assert(!err); + + // Put key-value + rocksdb_writeoptions_t *writeoptions = rocksdb_writeoptions_create(); + const char key[] = "key"; + const char *value = "value"; + rocksdb_put(db, writeoptions, key, strlen(key), value, strlen(value) + 1, + &err); + assert(!err); + // Get value + rocksdb_readoptions_t *readoptions = rocksdb_readoptions_create(); + size_t len; + char *returned_value = + rocksdb_get(db, readoptions, key, strlen(key), &len, &err); + assert(!err); + assert(strcmp(returned_value, "value") == 0); + free(returned_value); + + // create new backup in a directory specified by DBBackupPath + rocksdb_backup_engine_create_new_backup(be, db, &err); + assert(!err); + + rocksdb_close(db); + + // If something is wrong, you might want to restore data from last backup + rocksdb_restore_options_t *restore_options = rocksdb_restore_options_create(); + rocksdb_backup_engine_restore_db_from_latest_backup(be, DBPath, DBPath, + restore_options, &err); + assert(!err); + rocksdb_restore_options_destroy(restore_options); + + db = rocksdb_open(options, DBPath, &err); + assert(!err); + + // cleanup + rocksdb_writeoptions_destroy(writeoptions); + rocksdb_readoptions_destroy(readoptions); + rocksdb_options_destroy(options); + rocksdb_backup_engine_close(be); + rocksdb_close(db); + + return 0; +} diff --git a/examples/column_families_example.cc b/examples/column_families_example.cc index 2bdf6ec42a1..589ff8ec294 100644 --- a/examples/column_families_example.cc +++ b/examples/column_families_example.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #include #include #include @@ -33,7 +33,7 @@ int main() { // open DB with two column families std::vector column_families; - // have to open default column familiy + // have to open default column family column_families.push_back(ColumnFamilyDescriptor( kDefaultColumnFamilyName, ColumnFamilyOptions())); // open the new one, too diff --git a/examples/compact_files_example.cc b/examples/compact_files_example.cc new file mode 100644 index 00000000000..c27df8ee79d --- /dev/null +++ b/examples/compact_files_example.cc @@ -0,0 +1,171 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// An example code demonstrating how to use CompactFiles, EventListener, +// and GetColumnFamilyMetaData APIs to implement custom compaction algorithm. + +#include +#include +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/options.h" + +using namespace rocksdb; +std::string kDBPath = "/tmp/rocksdb_compact_files_example"; +struct CompactionTask; + +// This is an example interface of external-compaction algorithm. +// Compaction algorithm can be implemented outside the core-RocksDB +// code by using the pluggable compaction APIs that RocksDb provides. +class Compactor : public EventListener { + public: + // Picks and returns a compaction task given the specified DB + // and column family. It is the caller's responsibility to + // destroy the returned CompactionTask. Returns "nullptr" + // if it cannot find a proper compaction task. + virtual CompactionTask* PickCompaction( + DB* db, const std::string& cf_name) = 0; + + // Schedule and run the specified compaction task in background. + virtual void ScheduleCompaction(CompactionTask *task) = 0; +}; + +// Example structure that describes a compaction task. +struct CompactionTask { + CompactionTask( + DB* _db, Compactor* _compactor, + const std::string& _column_family_name, + const std::vector& _input_file_names, + const int _output_level, + const CompactionOptions& _compact_options, + bool _retry_on_fail) + : db(_db), + compactor(_compactor), + column_family_name(_column_family_name), + input_file_names(_input_file_names), + output_level(_output_level), + compact_options(_compact_options), + retry_on_fail(_retry_on_fail) {} + DB* db; + Compactor* compactor; + const std::string& column_family_name; + std::vector input_file_names; + int output_level; + CompactionOptions compact_options; + bool retry_on_fail; +}; + +// A simple compaction algorithm that always compacts everything +// to the highest level whenever possible. +class FullCompactor : public Compactor { + public: + explicit FullCompactor(const Options options) : options_(options) { + compact_options_.compression = options_.compression; + compact_options_.output_file_size_limit = + options_.target_file_size_base; + } + + // When flush happens, it determines whether to trigger compaction. If + // triggered_writes_stop is true, it will also set the retry flag of + // compaction-task to true. + void OnFlushCompleted( + DB* db, const FlushJobInfo& info) override { + CompactionTask* task = PickCompaction(db, info.cf_name); + if (task != nullptr) { + if (info.triggered_writes_stop) { + task->retry_on_fail = true; + } + // Schedule compaction in a different thread. + ScheduleCompaction(task); + } + } + + // Always pick a compaction which includes all files whenever possible. + CompactionTask* PickCompaction( + DB* db, const std::string& cf_name) override { + ColumnFamilyMetaData cf_meta; + db->GetColumnFamilyMetaData(&cf_meta); + + std::vector input_file_names; + for (auto level : cf_meta.levels) { + for (auto file : level.files) { + if (file.being_compacted) { + return nullptr; + } + input_file_names.push_back(file.name); + } + } + return new CompactionTask( + db, this, cf_name, input_file_names, + options_.num_levels - 1, compact_options_, false); + } + + // Schedule the specified compaction task in background. + void ScheduleCompaction(CompactionTask* task) override { + options_.env->Schedule(&FullCompactor::CompactFiles, task); + } + + static void CompactFiles(void* arg) { + std::unique_ptr task( + reinterpret_cast(arg)); + assert(task); + assert(task->db); + Status s = task->db->CompactFiles( + task->compact_options, + task->input_file_names, + task->output_level); + printf("CompactFiles() finished with status %s\n", s.ToString().c_str()); + if (!s.ok() && !s.IsIOError() && task->retry_on_fail) { + // If a compaction task with its retry_on_fail=true failed, + // try to schedule another compaction in case the reason + // is not an IO error. + CompactionTask* new_task = task->compactor->PickCompaction( + task->db, task->column_family_name); + task->compactor->ScheduleCompaction(new_task); + } + } + + private: + Options options_; + CompactionOptions compact_options_; +}; + +int main() { + Options options; + options.create_if_missing = true; + // Disable RocksDB background compaction. + options.compaction_style = kCompactionStyleNone; + // Small slowdown and stop trigger for experimental purpose. + options.level0_slowdown_writes_trigger = 3; + options.level0_stop_writes_trigger = 5; + options.IncreaseParallelism(5); + options.listeners.emplace_back(new FullCompactor(options)); + + DB* db = nullptr; + DestroyDB(kDBPath, options); + Status s = DB::Open(options, kDBPath, &db); + assert(s.ok()); + assert(db); + + // if background compaction is not working, write will stall + // because of options.level0_stop_writes_trigger + for (int i = 1000; i < 99999; ++i) { + db->Put(WriteOptions(), std::to_string(i), + std::string(500, 'a' + (i % 26))); + } + + // verify the values are still there + std::string value; + for (int i = 1000; i < 99999; ++i) { + db->Get(ReadOptions(), std::to_string(i), + &value); + assert(value == std::string(500, 'a' + (i % 26))); + } + + // close the db. + delete db; + + return 0; +} diff --git a/examples/compaction_filter_example.cc b/examples/compaction_filter_example.cc new file mode 100644 index 00000000000..226dfe7905d --- /dev/null +++ b/examples/compaction_filter_example.cc @@ -0,0 +1,87 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include +#include +#include +#include + +class MyMerge : public rocksdb::MergeOperator { + public: + virtual bool FullMergeV2(const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const override { + merge_out->new_value.clear(); + if (merge_in.existing_value != nullptr) { + merge_out->new_value.assign(merge_in.existing_value->data(), + merge_in.existing_value->size()); + } + for (const rocksdb::Slice& m : merge_in.operand_list) { + fprintf(stderr, "Merge(%s)\n", m.ToString().c_str()); + // the compaction filter filters out bad values + assert(m.ToString() != "bad"); + merge_out->new_value.assign(m.data(), m.size()); + } + return true; + } + + const char* Name() const override { return "MyMerge"; } +}; + +class MyFilter : public rocksdb::CompactionFilter { + public: + bool Filter(int level, const rocksdb::Slice& key, + const rocksdb::Slice& existing_value, std::string* new_value, + bool* value_changed) const override { + fprintf(stderr, "Filter(%s)\n", key.ToString().c_str()); + ++count_; + assert(*value_changed == false); + return false; + } + + bool FilterMergeOperand(int level, const rocksdb::Slice& key, + const rocksdb::Slice& existing_value) const override { + fprintf(stderr, "FilterMerge(%s)\n", key.ToString().c_str()); + ++merge_count_; + return existing_value == "bad"; + } + + const char* Name() const override { return "MyFilter"; } + + mutable int count_ = 0; + mutable int merge_count_ = 0; +}; + +int main() { + rocksdb::DB* raw_db; + rocksdb::Status status; + + MyFilter filter; + + int ret = system("rm -rf /tmp/rocksmergetest"); + if (ret != 0) { + fprintf(stderr, "Error deleting /tmp/rocksmergetest, code: %d\n", ret); + return ret; + } + rocksdb::Options options; + options.create_if_missing = true; + options.merge_operator.reset(new MyMerge); + options.compaction_filter = &filter; + status = rocksdb::DB::Open(options, "/tmp/rocksmergetest", &raw_db); + assert(status.ok()); + std::unique_ptr db(raw_db); + + rocksdb::WriteOptions wopts; + db->Merge(wopts, "0", "bad"); // This is filtered out + db->Merge(wopts, "1", "data1"); + db->Merge(wopts, "1", "bad"); + db->Merge(wopts, "1", "data2"); + db->Merge(wopts, "1", "bad"); + db->Merge(wopts, "3", "data3"); + db->CompactRange(rocksdb::CompactRangeOptions(), nullptr, nullptr); + fprintf(stderr, "filter.count_ = %d\n", filter.count_); + assert(filter.count_ == 0); + fprintf(stderr, "filter.merge_count_ = %d\n", filter.merge_count_); + assert(filter.merge_count_ == 6); +} diff --git a/examples/optimistic_transaction_example.cc b/examples/optimistic_transaction_example.cc new file mode 100644 index 00000000000..94444e16259 --- /dev/null +++ b/examples/optimistic_transaction_example.cc @@ -0,0 +1,142 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include "rocksdb/db.h" +#include "rocksdb/options.h" +#include "rocksdb/slice.h" +#include "rocksdb/utilities/transaction.h" +#include "rocksdb/utilities/optimistic_transaction_db.h" + +using namespace rocksdb; + +std::string kDBPath = "/tmp/rocksdb_transaction_example"; + +int main() { + // open DB + Options options; + options.create_if_missing = true; + DB* db; + OptimisticTransactionDB* txn_db; + + Status s = OptimisticTransactionDB::Open(options, kDBPath, &txn_db); + assert(s.ok()); + db = txn_db->GetBaseDB(); + + WriteOptions write_options; + ReadOptions read_options; + OptimisticTransactionOptions txn_options; + std::string value; + + //////////////////////////////////////////////////////// + // + // Simple OptimisticTransaction Example ("Read Committed") + // + //////////////////////////////////////////////////////// + + // Start a transaction + Transaction* txn = txn_db->BeginTransaction(write_options); + assert(txn); + + // Read a key in this transaction + s = txn->Get(read_options, "abc", &value); + assert(s.IsNotFound()); + + // Write a key in this transaction + txn->Put("abc", "def"); + + // Read a key OUTSIDE this transaction. Does not affect txn. + s = db->Get(read_options, "abc", &value); + + // Write a key OUTSIDE of this transaction. + // Does not affect txn since this is an unrelated key. If we wrote key 'abc' + // here, the transaction would fail to commit. + s = db->Put(write_options, "xyz", "zzz"); + + // Commit transaction + s = txn->Commit(); + assert(s.ok()); + delete txn; + + //////////////////////////////////////////////////////// + // + // "Repeatable Read" (Snapshot Isolation) Example + // -- Using a single Snapshot + // + //////////////////////////////////////////////////////// + + // Set a snapshot at start of transaction by setting set_snapshot=true + txn_options.set_snapshot = true; + txn = txn_db->BeginTransaction(write_options, txn_options); + + const Snapshot* snapshot = txn->GetSnapshot(); + + // Write a key OUTSIDE of transaction + db->Put(write_options, "abc", "xyz"); + + // Read a key using the snapshot + read_options.snapshot = snapshot; + s = txn->GetForUpdate(read_options, "abc", &value); + assert(value == "def"); + + // Attempt to commit transaction + s = txn->Commit(); + + // Transaction could not commit since the write outside of the txn conflicted + // with the read! + assert(s.IsBusy()); + + delete txn; + // Clear snapshot from read options since it is no longer valid + read_options.snapshot = nullptr; + snapshot = nullptr; + + //////////////////////////////////////////////////////// + // + // "Read Committed" (Monotonic Atomic Views) Example + // --Using multiple Snapshots + // + //////////////////////////////////////////////////////// + + // In this example, we set the snapshot multiple times. This is probably + // only necessary if you have very strict isolation requirements to + // implement. + + // Set a snapshot at start of transaction + txn_options.set_snapshot = true; + txn = txn_db->BeginTransaction(write_options, txn_options); + + // Do some reads and writes to key "x" + read_options.snapshot = db->GetSnapshot(); + s = txn->Get(read_options, "x", &value); + txn->Put("x", "x"); + + // Do a write outside of the transaction to key "y" + s = db->Put(write_options, "y", "y"); + + // Set a new snapshot in the transaction + txn->SetSnapshot(); + read_options.snapshot = db->GetSnapshot(); + + // Do some reads and writes to key "y" + s = txn->GetForUpdate(read_options, "y", &value); + txn->Put("y", "y"); + + // Commit. Since the snapshot was advanced, the write done outside of the + // transaction does not prevent this transaction from Committing. + s = txn->Commit(); + assert(s.ok()); + delete txn; + // Clear snapshot from read options since it is no longer valid + read_options.snapshot = nullptr; + + // Cleanup + delete txn_db; + DestroyDB(kDBPath, options); + return 0; +} + +#endif // ROCKSDB_LITE diff --git a/examples/options_file_example.cc b/examples/options_file_example.cc new file mode 100644 index 00000000000..5dd0a479c00 --- /dev/null +++ b/examples/options_file_example.cc @@ -0,0 +1,113 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file demonstrates how to use the utility functions defined in +// rocksdb/utilities/options_util.h to open a rocksdb database without +// remembering all the rocksdb options. +#include +#include +#include + +#include "rocksdb/cache.h" +#include "rocksdb/compaction_filter.h" +#include "rocksdb/db.h" +#include "rocksdb/options.h" +#include "rocksdb/slice.h" +#include "rocksdb/table.h" +#include "rocksdb/utilities/options_util.h" + +using namespace rocksdb; + +std::string kDBPath = "/tmp/rocksdb_options_file_example"; + +namespace { +// A dummy compaction filter +class DummyCompactionFilter : public CompactionFilter { + public: + virtual ~DummyCompactionFilter() {} + virtual bool Filter(int level, const Slice& key, const Slice& existing_value, + std::string* new_value, bool* value_changed) const { + return false; + } + virtual const char* Name() const { return "DummyCompactionFilter"; } +}; + +} // namespace + +int main() { + DBOptions db_opt; + db_opt.create_if_missing = true; + + std::vector cf_descs; + cf_descs.push_back({kDefaultColumnFamilyName, ColumnFamilyOptions()}); + cf_descs.push_back({"new_cf", ColumnFamilyOptions()}); + + // initialize BlockBasedTableOptions + auto cache = NewLRUCache(1 * 1024 * 1024 * 1024); + BlockBasedTableOptions bbt_opts; + bbt_opts.block_size = 32 * 1024; + bbt_opts.block_cache = cache; + + // initialize column families options + std::unique_ptr compaction_filter; + compaction_filter.reset(new DummyCompactionFilter()); + cf_descs[0].options.table_factory.reset(NewBlockBasedTableFactory(bbt_opts)); + cf_descs[0].options.compaction_filter = compaction_filter.get(); + cf_descs[1].options.table_factory.reset(NewBlockBasedTableFactory(bbt_opts)); + + // destroy and open DB + DB* db; + Status s = DestroyDB(kDBPath, Options(db_opt, cf_descs[0].options)); + assert(s.ok()); + s = DB::Open(Options(db_opt, cf_descs[0].options), kDBPath, &db); + assert(s.ok()); + + // Create column family, and rocksdb will persist the options. + ColumnFamilyHandle* cf; + s = db->CreateColumnFamily(ColumnFamilyOptions(), "new_cf", &cf); + assert(s.ok()); + + // close DB + delete cf; + delete db; + + // In the following code, we will reopen the rocksdb instance using + // the options file stored in the db directory. + + // Load the options file. + DBOptions loaded_db_opt; + std::vector loaded_cf_descs; + s = LoadLatestOptions(kDBPath, Env::Default(), &loaded_db_opt, + &loaded_cf_descs); + assert(s.ok()); + assert(loaded_db_opt.create_if_missing == db_opt.create_if_missing); + + // Initialize pointer options for each column family + for (size_t i = 0; i < loaded_cf_descs.size(); ++i) { + auto* loaded_bbt_opt = reinterpret_cast( + loaded_cf_descs[0].options.table_factory->GetOptions()); + // Expect the same as BlockBasedTableOptions will be loaded form file. + assert(loaded_bbt_opt->block_size == bbt_opts.block_size); + // However, block_cache needs to be manually initialized as documented + // in rocksdb/utilities/options_util.h. + loaded_bbt_opt->block_cache = cache; + } + // In addition, as pointer options are initialized with default value, + // we need to properly initialized all the pointer options if non-defalut + // values are used before calling DB::Open(). + assert(loaded_cf_descs[0].options.compaction_filter == nullptr); + loaded_cf_descs[0].options.compaction_filter = compaction_filter.get(); + + // reopen the db using the loaded options. + std::vector handles; + s = DB::Open(loaded_db_opt, kDBPath, loaded_cf_descs, &handles, &db); + assert(s.ok()); + + // close DB + for (auto* handle : handles) { + delete handle; + } + delete db; +} diff --git a/examples/rocksdb_option_file_example.ini b/examples/rocksdb_option_file_example.ini new file mode 100644 index 00000000000..8e07131b39d --- /dev/null +++ b/examples/rocksdb_option_file_example.ini @@ -0,0 +1,143 @@ +# This is a RocksDB option file. +# +# A typical RocksDB options file has four sections, which are +# Version section, DBOptions section, at least one CFOptions +# section, and one TableOptions section for each column family. +# The RocksDB options file in general follows the basic INI +# file format with the following extensions / modifications: +# +# * Escaped characters +# We escaped the following characters: +# - \n -- line feed - new line +# - \r -- carriage return +# - \\ -- backslash \ +# - \: -- colon symbol : +# - \# -- hash tag # +# * Comments +# We support # style comments. Comments can appear at the ending +# part of a line. +# * Statements +# A statement is of the form option_name = value. +# Each statement contains a '=', where extra white-spaces +# are supported. However, we don't support multi-lined statement. +# Furthermore, each line can only contain at most one statement. +# * Sections +# Sections are of the form [SecitonTitle "SectionArgument"], +# where section argument is optional. +# * List +# We use colon-separated string to represent a list. +# For instance, n1:n2:n3:n4 is a list containing four values. +# +# Below is an example of a RocksDB options file: +[Version] + rocksdb_version=4.3.0 + options_file_version=1.1 + +[DBOptions] + stats_dump_period_sec=600 + max_manifest_file_size=18446744073709551615 + bytes_per_sync=8388608 + delayed_write_rate=2097152 + WAL_ttl_seconds=0 + WAL_size_limit_MB=0 + max_subcompactions=1 + wal_dir= + wal_bytes_per_sync=0 + db_write_buffer_size=0 + keep_log_file_num=1000 + table_cache_numshardbits=4 + max_file_opening_threads=1 + writable_file_max_buffer_size=1048576 + random_access_max_buffer_size=1048576 + use_fsync=false + max_total_wal_size=0 + max_open_files=-1 + skip_stats_update_on_db_open=false + max_background_compactions=16 + manifest_preallocation_size=4194304 + max_background_flushes=7 + is_fd_close_on_exec=true + max_log_file_size=0 + advise_random_on_open=true + create_missing_column_families=false + paranoid_checks=true + delete_obsolete_files_period_micros=21600000000 + log_file_time_to_roll=0 + compaction_readahead_size=0 + create_if_missing=false + use_adaptive_mutex=false + enable_thread_tracking=false + allow_fallocate=true + error_if_exists=false + recycle_log_file_num=0 + skip_log_error_on_recovery=false + db_log_dir= + new_table_reader_for_compaction_inputs=true + allow_mmap_reads=false + allow_mmap_writes=false + use_direct_reads=false + use_direct_writes=false + + +[CFOptions "default"] + compaction_style=kCompactionStyleLevel + compaction_filter=nullptr + num_levels=6 + table_factory=BlockBasedTable + comparator=leveldb.BytewiseComparator + max_sequential_skip_in_iterations=8 + soft_rate_limit=0.000000 + max_bytes_for_level_base=1073741824 + memtable_prefix_bloom_probes=6 + memtable_prefix_bloom_bits=0 + memtable_prefix_bloom_huge_page_tlb_size=0 + max_successive_merges=0 + arena_block_size=16777216 + min_write_buffer_number_to_merge=1 + target_file_size_multiplier=1 + source_compaction_factor=1 + max_bytes_for_level_multiplier=8 + max_bytes_for_level_multiplier_additional=2:3:5 + compaction_filter_factory=nullptr + max_write_buffer_number=8 + level0_stop_writes_trigger=20 + compression=kSnappyCompression + level0_file_num_compaction_trigger=4 + purge_redundant_kvs_while_flush=true + max_write_buffer_number_to_maintain=0 + memtable_factory=SkipListFactory + max_grandparent_overlap_factor=8 + expanded_compaction_factor=25 + hard_pending_compaction_bytes_limit=137438953472 + inplace_update_num_locks=10000 + level_compaction_dynamic_level_bytes=true + level0_slowdown_writes_trigger=12 + filter_deletes=false + verify_checksums_in_compaction=true + min_partial_merge_operands=2 + paranoid_file_checks=false + target_file_size_base=134217728 + optimize_filters_for_hits=false + merge_operator=PutOperator + compression_per_level=kNoCompression:kNoCompression:kNoCompression:kSnappyCompression:kSnappyCompression:kSnappyCompression + compaction_measure_io_stats=false + prefix_extractor=nullptr + bloom_locality=0 + write_buffer_size=134217728 + disable_auto_compactions=false + inplace_update_support=false + +[TableOptions/BlockBasedTable "default"] + format_version=2 + whole_key_filtering=true + no_block_cache=false + checksum=kCRC32c + filter_policy=rocksdb.BuiltinBloomFilter + block_size_deviation=10 + block_size=8192 + block_restart_interval=16 + cache_index_and_filter_blocks=false + pin_l0_filter_and_index_blocks_in_cache=false + index_type=kBinarySearch + hash_index_allow_collision=true + flush_block_policy_factory=FlushBlockBySizePolicyFactory diff --git a/examples/simple_example.cc b/examples/simple_example.cc index 20e7faa4b05..a8f80f091e6 100644 --- a/examples/simple_example.cc +++ b/examples/simple_example.cc @@ -1,7 +1,8 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + #include #include @@ -27,14 +28,55 @@ int main() { assert(s.ok()); // Put key-value - s = db->Put(WriteOptions(), "key", "value"); + s = db->Put(WriteOptions(), "key1", "value"); assert(s.ok()); std::string value; // get value - s = db->Get(ReadOptions(), "key", &value); + s = db->Get(ReadOptions(), "key1", &value); assert(s.ok()); assert(value == "value"); + // atomically apply a set of updates + { + WriteBatch batch; + batch.Delete("key1"); + batch.Put("key2", value); + s = db->Write(WriteOptions(), &batch); + } + + s = db->Get(ReadOptions(), "key1", &value); + assert(s.IsNotFound()); + + db->Get(ReadOptions(), "key2", &value); + assert(value == "value"); + + { + PinnableSlice pinnable_val; + db->Get(ReadOptions(), db->DefaultColumnFamily(), "key2", &pinnable_val); + assert(pinnable_val == "value"); + } + + { + std::string string_val; + // If it cannot pin the value, it copies the value to its internal buffer. + // The intenral buffer could be set during construction. + PinnableSlice pinnable_val(&string_val); + db->Get(ReadOptions(), db->DefaultColumnFamily(), "key2", &pinnable_val); + assert(pinnable_val == "value"); + // If the value is not pinned, the internal buffer must have the value. + assert(pinnable_val.IsPinned() || string_val == "value"); + } + + PinnableSlice pinnable_val; + db->Get(ReadOptions(), db->DefaultColumnFamily(), "key1", &pinnable_val); + assert(s.IsNotFound()); + // Reset PinnableSlice after each use and before each reuse + pinnable_val.Reset(); + db->Get(ReadOptions(), db->DefaultColumnFamily(), "key2", &pinnable_val); + assert(pinnable_val == "value"); + pinnable_val.Reset(); + // The Slice pointed by pinnable_val is not valid after this point + delete db; return 0; diff --git a/examples/transaction_example.cc b/examples/transaction_example.cc new file mode 100644 index 00000000000..7274cf7ec07 --- /dev/null +++ b/examples/transaction_example.cc @@ -0,0 +1,144 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include "rocksdb/db.h" +#include "rocksdb/options.h" +#include "rocksdb/slice.h" +#include "rocksdb/utilities/transaction.h" +#include "rocksdb/utilities/transaction_db.h" + +using namespace rocksdb; + +std::string kDBPath = "/tmp/rocksdb_transaction_example"; + +int main() { + // open DB + Options options; + TransactionDBOptions txn_db_options; + options.create_if_missing = true; + TransactionDB* txn_db; + + Status s = TransactionDB::Open(options, txn_db_options, kDBPath, &txn_db); + assert(s.ok()); + + WriteOptions write_options; + ReadOptions read_options; + TransactionOptions txn_options; + std::string value; + + //////////////////////////////////////////////////////// + // + // Simple Transaction Example ("Read Committed") + // + //////////////////////////////////////////////////////// + + // Start a transaction + Transaction* txn = txn_db->BeginTransaction(write_options); + assert(txn); + + // Read a key in this transaction + s = txn->Get(read_options, "abc", &value); + assert(s.IsNotFound()); + + // Write a key in this transaction + s = txn->Put("abc", "def"); + assert(s.ok()); + + // Read a key OUTSIDE this transaction. Does not affect txn. + s = txn_db->Get(read_options, "abc", &value); + + // Write a key OUTSIDE of this transaction. + // Does not affect txn since this is an unrelated key. If we wrote key 'abc' + // here, the transaction would fail to commit. + s = txn_db->Put(write_options, "xyz", "zzz"); + + // Commit transaction + s = txn->Commit(); + assert(s.ok()); + delete txn; + + //////////////////////////////////////////////////////// + // + // "Repeatable Read" (Snapshot Isolation) Example + // -- Using a single Snapshot + // + //////////////////////////////////////////////////////// + + // Set a snapshot at start of transaction by setting set_snapshot=true + txn_options.set_snapshot = true; + txn = txn_db->BeginTransaction(write_options, txn_options); + + const Snapshot* snapshot = txn->GetSnapshot(); + + // Write a key OUTSIDE of transaction + s = txn_db->Put(write_options, "abc", "xyz"); + assert(s.ok()); + + // Attempt to read a key using the snapshot. This will fail since + // the previous write outside this txn conflicts with this read. + read_options.snapshot = snapshot; + s = txn->GetForUpdate(read_options, "abc", &value); + assert(s.IsBusy()); + + txn->Rollback(); + + delete txn; + // Clear snapshot from read options since it is no longer valid + read_options.snapshot = nullptr; + snapshot = nullptr; + + //////////////////////////////////////////////////////// + // + // "Read Committed" (Monotonic Atomic Views) Example + // --Using multiple Snapshots + // + //////////////////////////////////////////////////////// + + // In this example, we set the snapshot multiple times. This is probably + // only necessary if you have very strict isolation requirements to + // implement. + + // Set a snapshot at start of transaction + txn_options.set_snapshot = true; + txn = txn_db->BeginTransaction(write_options, txn_options); + + // Do some reads and writes to key "x" + read_options.snapshot = txn_db->GetSnapshot(); + s = txn->Get(read_options, "x", &value); + txn->Put("x", "x"); + + // Do a write outside of the transaction to key "y" + s = txn_db->Put(write_options, "y", "y"); + + // Set a new snapshot in the transaction + txn->SetSnapshot(); + txn->SetSavePoint(); + read_options.snapshot = txn_db->GetSnapshot(); + + // Do some reads and writes to key "y" + // Since the snapshot was advanced, the write done outside of the + // transaction does not conflict. + s = txn->GetForUpdate(read_options, "y", &value); + txn->Put("y", "y"); + + // Decide we want to revert the last write from this transaction. + txn->RollbackToSavePoint(); + + // Commit. + s = txn->Commit(); + assert(s.ok()); + delete txn; + // Clear snapshot from read options since it is no longer valid + read_options.snapshot = nullptr; + + // Cleanup + delete txn_db; + DestroyDB(kDBPath, options); + return 0; +} + +#endif // ROCKSDB_LITE diff --git a/hdfs/README b/hdfs/README index f4f1106e45f..9036511692e 100644 --- a/hdfs/README +++ b/hdfs/README @@ -10,10 +10,10 @@ The env_hdfs.h file defines the rocksdb objects that are needed to talk to an underlying filesystem. If you want to compile rocksdb with hdfs support, please set the following -enviroment variables appropriately (also defined in setup.sh for convenience) +environment variables appropriately (also defined in setup.sh for convenience) USE_HDFS=1 - JAVA_HOME=/usr/local/jdk-6u22-64 - LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/jdk-6u22-64/jre/lib/amd64/server:/usr/local/jdk-6u22-64/jre/lib/amd64/:./snappy/libs + JAVA_HOME=/usr/local/jdk-7u79-64 + LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/jdk-7u79-64/jre/lib/amd64/server:/usr/local/jdk-7u79-64/jre/lib/amd64/:./snappy/libs make clean all db_bench To run dbbench, diff --git a/hdfs/env_hdfs.h b/hdfs/env_hdfs.h index 5e7de77d307..3a62bc8cb92 100644 --- a/hdfs/env_hdfs.h +++ b/hdfs/env_hdfs.h @@ -1,15 +1,15 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #pragma once #include #include -#include #include #include +#include "port/sys_time.h" #include "rocksdb/env.h" #include "rocksdb/status.h" @@ -66,14 +66,10 @@ class HdfsEnv : public Env { std::unique_ptr* result, const EnvOptions& options); - virtual Status NewRandomRWFile(const std::string& fname, - std::unique_ptr* result, - const EnvOptions& options); - virtual Status NewDirectory(const std::string& name, std::unique_ptr* result); - virtual bool FileExists(const std::string& fname); + virtual Status FileExists(const std::string& fname); virtual Status GetChildren(const std::string& path, std::vector* result); @@ -93,6 +89,10 @@ class HdfsEnv : public Env { virtual Status RenameFile(const std::string& src, const std::string& target); + virtual Status LinkFile(const std::string& src, const std::string& target) { + return Status::NotSupported(); // not supported + } + virtual Status LockFile(const std::string& fname, FileLock** lock); virtual Status UnlockFile(FileLock* lock); @@ -101,8 +101,12 @@ class HdfsEnv : public Env { std::shared_ptr* result); virtual void Schedule(void (*function)(void* arg), void* arg, - Priority pri = LOW) { - posixEnv->Schedule(function, arg, pri); + Priority pri = LOW, void* tag = nullptr, void (*unschedFunction)(void* arg) = 0) { + posixEnv->Schedule(function, arg, pri, tag, unschedFunction); + } + + virtual int UnSchedule(void* tag, Priority pri) { + return posixEnv->UnSchedule(tag, pri); } virtual void StartThread(void (*function)(void* arg), void* arg) { @@ -145,6 +149,14 @@ class HdfsEnv : public Env { posixEnv->SetBackgroundThreads(number, pri); } + virtual int GetBackgroundThreads(Priority pri = LOW) { + return posixEnv->GetBackgroundThreads(pri); + } + + virtual void IncBackgroundThreadsIfNeeded(int number, Priority pri) override { + posixEnv->IncBackgroundThreadsIfNeeded(number, pri); + } + virtual std::string TimeToString(uint64_t number) { return posixEnv->TimeToString(number); } @@ -154,6 +166,10 @@ class HdfsEnv : public Env { return (uint64_t)pthread_self(); } + virtual uint64_t GetThreadID() const override { + return HdfsEnv::gettid(); + } + private: std::string fsname_; // string of the form "hdfs://hostname:port/" hdfsFS fileSys_; // a single FileSystem object for all files @@ -232,7 +248,7 @@ class HdfsEnv : public Env { explicit HdfsEnv(const std::string& fsname) { fprintf(stderr, "You have not build rocksdb with HDFS support\n"); fprintf(stderr, "Please see hdfs/README for details\n"); - throw std::exception(); + abort(); } virtual ~HdfsEnv() { @@ -240,87 +256,118 @@ class HdfsEnv : public Env { virtual Status NewSequentialFile(const std::string& fname, unique_ptr* result, - const EnvOptions& options); + const EnvOptions& options) override; virtual Status NewRandomAccessFile(const std::string& fname, unique_ptr* result, - const EnvOptions& options) { + const EnvOptions& options) override { return notsup; } virtual Status NewWritableFile(const std::string& fname, unique_ptr* result, - const EnvOptions& options) { + const EnvOptions& options) override { return notsup; } - virtual Status NewRandomRWFile(const std::string& fname, - unique_ptr* result, - const EnvOptions& options) { + virtual Status NewDirectory(const std::string& name, + unique_ptr* result) override { return notsup; } - virtual Status NewDirectory(const std::string& name, - unique_ptr* result) { + virtual Status FileExists(const std::string& fname) override { return notsup; } - virtual bool FileExists(const std::string& fname){return false;} - virtual Status GetChildren(const std::string& path, - std::vector* result){return notsup;} + std::vector* result) override { + return notsup; + } - virtual Status DeleteFile(const std::string& fname){return notsup;} + virtual Status DeleteFile(const std::string& fname) override { + return notsup; + } - virtual Status CreateDir(const std::string& name){return notsup;} + virtual Status CreateDir(const std::string& name) override { return notsup; } - virtual Status CreateDirIfMissing(const std::string& name){return notsup;} + virtual Status CreateDirIfMissing(const std::string& name) override { + return notsup; + } - virtual Status DeleteDir(const std::string& name){return notsup;} + virtual Status DeleteDir(const std::string& name) override { return notsup; } - virtual Status GetFileSize(const std::string& fname, uint64_t* size){return notsup;} + virtual Status GetFileSize(const std::string& fname, + uint64_t* size) override { + return notsup; + } virtual Status GetFileModificationTime(const std::string& fname, - uint64_t* time) { + uint64_t* time) override { + return notsup; + } + + virtual Status RenameFile(const std::string& src, + const std::string& target) override { return notsup; } - virtual Status RenameFile(const std::string& src, const std::string& target){return notsup;} + virtual Status LinkFile(const std::string& src, + const std::string& target) override { + return notsup; + } - virtual Status LockFile(const std::string& fname, FileLock** lock){return notsup;} + virtual Status LockFile(const std::string& fname, FileLock** lock) override { + return notsup; + } - virtual Status UnlockFile(FileLock* lock){return notsup;} + virtual Status UnlockFile(FileLock* lock) override { return notsup; } virtual Status NewLogger(const std::string& fname, - shared_ptr* result){return notsup;} + shared_ptr* result) override { + return notsup; + } virtual void Schedule(void (*function)(void* arg), void* arg, - Priority pri = LOW) {} + Priority pri = LOW, void* tag = nullptr, + void (*unschedFunction)(void* arg) = 0) override {} + + virtual int UnSchedule(void* tag, Priority pri) override { return 0; } - virtual void StartThread(void (*function)(void* arg), void* arg) {} + virtual void StartThread(void (*function)(void* arg), void* arg) override {} - virtual void WaitForJoin() {} + virtual void WaitForJoin() override {} - virtual unsigned int GetThreadPoolQueueLen(Priority pri = LOW) const { + virtual unsigned int GetThreadPoolQueueLen( + Priority pri = LOW) const override { return 0; } - virtual Status GetTestDirectory(std::string* path) {return notsup;} + virtual Status GetTestDirectory(std::string* path) override { return notsup; } - virtual uint64_t NowMicros() {return 0;} + virtual uint64_t NowMicros() override { return 0; } - virtual void SleepForMicroseconds(int micros) {} + virtual void SleepForMicroseconds(int micros) override {} - virtual Status GetHostName(char* name, uint64_t len) {return notsup;} + virtual Status GetHostName(char* name, uint64_t len) override { + return notsup; + } - virtual Status GetCurrentTime(int64_t* unix_time) {return notsup;} + virtual Status GetCurrentTime(int64_t* unix_time) override { return notsup; } virtual Status GetAbsolutePath(const std::string& db_path, - std::string* outputpath) {return notsup;} + std::string* outputpath) override { + return notsup; + } - virtual void SetBackgroundThreads(int number, Priority pri = LOW) {} + virtual void SetBackgroundThreads(int number, Priority pri = LOW) override {} + virtual int GetBackgroundThreads(Priority pri = LOW) override { return 0; } + virtual void IncBackgroundThreadsIfNeeded(int number, Priority pri) override { + } + virtual std::string TimeToString(uint64_t number) override { return ""; } - virtual std::string TimeToString(uint64_t number) { return "";} + virtual uint64_t GetThreadID() const override { + return 0; + } }; } diff --git a/helpers/memenv/memenv.cc b/helpers/memenv/memenv.cc deleted file mode 100644 index 185e7d822bb..00000000000 --- a/helpers/memenv/memenv.cc +++ /dev/null @@ -1,395 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#include "rocksdb/env.h" -#include "rocksdb/status.h" -#include "port/port.h" -#include "util/mutexlock.h" -#include -#include -#include -#include - -namespace rocksdb { - -namespace { - -class FileState { - public: - // FileStates are reference counted. The initial reference count is zero - // and the caller must call Ref() at least once. - FileState() : refs_(0), size_(0) {} - - // Increase the reference count. - void Ref() { - MutexLock lock(&refs_mutex_); - ++refs_; - } - - // Decrease the reference count. Delete if this is the last reference. - void Unref() { - bool do_delete = false; - - { - MutexLock lock(&refs_mutex_); - --refs_; - assert(refs_ >= 0); - if (refs_ <= 0) { - do_delete = true; - } - } - - if (do_delete) { - delete this; - } - } - - uint64_t Size() const { return size_; } - - Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const { - if (offset > size_) { - return Status::IOError("Offset greater than file size."); - } - const uint64_t available = size_ - offset; - if (n > available) { - n = available; - } - if (n == 0) { - *result = Slice(); - return Status::OK(); - } - - size_t block = offset / kBlockSize; - size_t block_offset = offset % kBlockSize; - - if (n <= kBlockSize - block_offset) { - // The requested bytes are all in the first block. - *result = Slice(blocks_[block] + block_offset, n); - return Status::OK(); - } - - size_t bytes_to_copy = n; - char* dst = scratch; - - while (bytes_to_copy > 0) { - size_t avail = kBlockSize - block_offset; - if (avail > bytes_to_copy) { - avail = bytes_to_copy; - } - memcpy(dst, blocks_[block] + block_offset, avail); - - bytes_to_copy -= avail; - dst += avail; - block++; - block_offset = 0; - } - - *result = Slice(scratch, n); - return Status::OK(); - } - - Status Append(const Slice& data) { - const char* src = data.data(); - size_t src_len = data.size(); - - while (src_len > 0) { - size_t avail; - size_t offset = size_ % kBlockSize; - - if (offset != 0) { - // There is some room in the last block. - avail = kBlockSize - offset; - } else { - // No room in the last block; push new one. - blocks_.push_back(new char[kBlockSize]); - avail = kBlockSize; - } - - if (avail > src_len) { - avail = src_len; - } - memcpy(blocks_.back() + offset, src, avail); - src_len -= avail; - src += avail; - size_ += avail; - } - - return Status::OK(); - } - - private: - // Private since only Unref() should be used to delete it. - ~FileState() { - for (std::vector::iterator i = blocks_.begin(); i != blocks_.end(); - ++i) { - delete [] *i; - } - } - - // No copying allowed. - FileState(const FileState&); - void operator=(const FileState&); - - port::Mutex refs_mutex_; - int refs_; // Protected by refs_mutex_; - - // The following fields are not protected by any mutex. They are only mutable - // while the file is being written, and concurrent access is not allowed - // to writable files. - std::vector blocks_; - uint64_t size_; - - enum { kBlockSize = 8 * 1024 }; -}; - -class SequentialFileImpl : public SequentialFile { - public: - explicit SequentialFileImpl(FileState* file) : file_(file), pos_(0) { - file_->Ref(); - } - - ~SequentialFileImpl() { - file_->Unref(); - } - - virtual Status Read(size_t n, Slice* result, char* scratch) { - Status s = file_->Read(pos_, n, result, scratch); - if (s.ok()) { - pos_ += result->size(); - } - return s; - } - - virtual Status Skip(uint64_t n) { - if (pos_ > file_->Size()) { - return Status::IOError("pos_ > file_->Size()"); - } - const size_t available = file_->Size() - pos_; - if (n > available) { - n = available; - } - pos_ += n; - return Status::OK(); - } - - private: - FileState* file_; - size_t pos_; -}; - -class RandomAccessFileImpl : public RandomAccessFile { - public: - explicit RandomAccessFileImpl(FileState* file) : file_(file) { - file_->Ref(); - } - - ~RandomAccessFileImpl() { - file_->Unref(); - } - - virtual Status Read(uint64_t offset, size_t n, Slice* result, - char* scratch) const { - return file_->Read(offset, n, result, scratch); - } - - private: - FileState* file_; -}; - -class WritableFileImpl : public WritableFile { - public: - WritableFileImpl(FileState* file) : file_(file) { - file_->Ref(); - } - - ~WritableFileImpl() { - file_->Unref(); - } - - virtual Status Append(const Slice& data) { - return file_->Append(data); - } - - virtual Status Close() { return Status::OK(); } - virtual Status Flush() { return Status::OK(); } - virtual Status Sync() { return Status::OK(); } - - private: - FileState* file_; -}; - -class InMemoryDirectory : public Directory { - public: - virtual Status Fsync() { return Status::OK(); } -}; - -class InMemoryEnv : public EnvWrapper { - public: - explicit InMemoryEnv(Env* base_env) : EnvWrapper(base_env) { } - - virtual ~InMemoryEnv() { - for (FileSystem::iterator i = file_map_.begin(); i != file_map_.end(); ++i){ - i->second->Unref(); - } - } - - // Partial implementation of the Env interface. - virtual Status NewSequentialFile(const std::string& fname, - unique_ptr* result, - const EnvOptions& soptions) { - MutexLock lock(&mutex_); - if (file_map_.find(fname) == file_map_.end()) { - *result = NULL; - return Status::IOError(fname, "File not found"); - } - - result->reset(new SequentialFileImpl(file_map_[fname])); - return Status::OK(); - } - - virtual Status NewRandomAccessFile(const std::string& fname, - unique_ptr* result, - const EnvOptions& soptions) { - MutexLock lock(&mutex_); - if (file_map_.find(fname) == file_map_.end()) { - *result = NULL; - return Status::IOError(fname, "File not found"); - } - - result->reset(new RandomAccessFileImpl(file_map_[fname])); - return Status::OK(); - } - - virtual Status NewWritableFile(const std::string& fname, - unique_ptr* result, - const EnvOptions& soptions) { - MutexLock lock(&mutex_); - if (file_map_.find(fname) != file_map_.end()) { - DeleteFileInternal(fname); - } - - FileState* file = new FileState(); - file->Ref(); - file_map_[fname] = file; - - result->reset(new WritableFileImpl(file)); - return Status::OK(); - } - - virtual Status NewDirectory(const std::string& name, - unique_ptr* result) { - result->reset(new InMemoryDirectory()); - return Status::OK(); - } - - virtual bool FileExists(const std::string& fname) { - MutexLock lock(&mutex_); - return file_map_.find(fname) != file_map_.end(); - } - - virtual Status GetChildren(const std::string& dir, - std::vector* result) { - MutexLock lock(&mutex_); - result->clear(); - - for (FileSystem::iterator i = file_map_.begin(); i != file_map_.end(); ++i){ - const std::string& filename = i->first; - - if (filename.size() >= dir.size() + 1 && filename[dir.size()] == '/' && - Slice(filename).starts_with(Slice(dir))) { - result->push_back(filename.substr(dir.size() + 1)); - } - } - - return Status::OK(); - } - - void DeleteFileInternal(const std::string& fname) { - if (file_map_.find(fname) == file_map_.end()) { - return; - } - - file_map_[fname]->Unref(); - file_map_.erase(fname); - } - - virtual Status DeleteFile(const std::string& fname) { - MutexLock lock(&mutex_); - if (file_map_.find(fname) == file_map_.end()) { - return Status::IOError(fname, "File not found"); - } - - DeleteFileInternal(fname); - return Status::OK(); - } - - virtual Status CreateDir(const std::string& dirname) { - return Status::OK(); - } - - virtual Status CreateDirIfMissing(const std::string& dirname) { - return Status::OK(); - } - - virtual Status DeleteDir(const std::string& dirname) { - return Status::OK(); - } - - virtual Status GetFileSize(const std::string& fname, uint64_t* file_size) { - MutexLock lock(&mutex_); - if (file_map_.find(fname) == file_map_.end()) { - return Status::IOError(fname, "File not found"); - } - - *file_size = file_map_[fname]->Size(); - return Status::OK(); - } - - virtual Status GetFileModificationTime(const std::string& fname, - uint64_t* time) { - return Status::NotSupported("getFileMTime", "Not supported in MemEnv"); - } - - virtual Status RenameFile(const std::string& src, - const std::string& target) { - MutexLock lock(&mutex_); - if (file_map_.find(src) == file_map_.end()) { - return Status::IOError(src, "File not found"); - } - - DeleteFileInternal(target); - file_map_[target] = file_map_[src]; - file_map_.erase(src); - return Status::OK(); - } - - virtual Status LockFile(const std::string& fname, FileLock** lock) { - *lock = new FileLock; - return Status::OK(); - } - - virtual Status UnlockFile(FileLock* lock) { - delete lock; - return Status::OK(); - } - - virtual Status GetTestDirectory(std::string* path) { - *path = "/test"; - return Status::OK(); - } - - private: - // Map from filenames to FileState objects, representing a simple file system. - typedef std::map FileSystem; - port::Mutex mutex_; - FileSystem file_map_; // Protected by mutex_. -}; - -} // namespace - -Env* NewMemEnv(Env* base_env) { - return new InMemoryEnv(base_env); -} - -} // namespace rocksdb diff --git a/include/rocksdb/advanced_options.h b/include/rocksdb/advanced_options.h new file mode 100644 index 00000000000..6f45134a683 --- /dev/null +++ b/include/rocksdb/advanced_options.h @@ -0,0 +1,578 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include + +#include "rocksdb/memtablerep.h" +#include "rocksdb/universal_compaction.h" + +namespace rocksdb { + +class Slice; +class SliceTransform; +enum CompressionType : unsigned char; +class TablePropertiesCollectorFactory; +class TableFactory; +struct Options; + +enum CompactionStyle : char { + // level based compaction style + kCompactionStyleLevel = 0x0, + // Universal compaction style + // Not supported in ROCKSDB_LITE. + kCompactionStyleUniversal = 0x1, + // FIFO compaction style + // Not supported in ROCKSDB_LITE + kCompactionStyleFIFO = 0x2, + // Disable background compaction. Compaction jobs are submitted + // via CompactFiles(). + // Not supported in ROCKSDB_LITE + kCompactionStyleNone = 0x3, +}; + +// In Level-based compaction, it Determines which file from a level to be +// picked to merge to the next level. We suggest people try +// kMinOverlappingRatio first when you tune your database. +enum CompactionPri : char { + // Slightly prioritize larger files by size compensated by #deletes + kByCompensatedSize = 0x0, + // First compact files whose data's latest update time is oldest. + // Try this if you only update some hot keys in small ranges. + kOldestLargestSeqFirst = 0x1, + // First compact files whose range hasn't been compacted to the next level + // for the longest. If your updates are random across the key space, + // write amplification is slightly better with this option. + kOldestSmallestSeqFirst = 0x2, + // First compact files whose ratio between overlapping size in next level + // and its size is the smallest. It in many cases can optimize write + // amplification. + kMinOverlappingRatio = 0x3, +}; + +struct CompactionOptionsFIFO { + // once the total sum of table files reaches this, we will delete the oldest + // table file + // Default: 1GB + uint64_t max_table_files_size; + + // Drop files older than TTL. TTL based deletion will take precedence over + // size based deletion if ttl > 0. + // delete if sst_file_creation_time < (current_time - ttl) + // unit: seconds. Ex: 1 day = 1 * 24 * 60 * 60 + // Default: 0 (disabled) + uint64_t ttl = 0; + + // If true, try to do compaction to compact smaller files into larger ones. + // Minimum files to compact follows options.level0_file_num_compaction_trigger + // and compaction won't trigger if average compact bytes per del file is + // larger than options.write_buffer_size. This is to protect large files + // from being compacted again. + // Default: false; + bool allow_compaction = false; + + CompactionOptionsFIFO() : max_table_files_size(1 * 1024 * 1024 * 1024) {} + CompactionOptionsFIFO(uint64_t _max_table_files_size, bool _allow_compaction, + uint64_t _ttl = 0) + : max_table_files_size(_max_table_files_size), + ttl(_ttl), + allow_compaction(_allow_compaction) {} +}; + +// Compression options for different compression algorithms like Zlib +struct CompressionOptions { + int window_bits; + int level; + int strategy; + // Maximum size of dictionary used to prime the compression library. Currently + // this dictionary will be constructed by sampling the first output file in a + // subcompaction when the target level is bottommost. This dictionary will be + // loaded into the compression library before compressing/uncompressing each + // data block of subsequent files in the subcompaction. Effectively, this + // improves compression ratios when there are repetitions across data blocks. + // A value of 0 indicates the feature is disabled. + // Default: 0. + uint32_t max_dict_bytes; + + CompressionOptions() + : window_bits(-14), level(-1), strategy(0), max_dict_bytes(0) {} + CompressionOptions(int wbits, int _lev, int _strategy, int _max_dict_bytes) + : window_bits(wbits), + level(_lev), + strategy(_strategy), + max_dict_bytes(_max_dict_bytes) {} +}; + +enum UpdateStatus { // Return status For inplace update callback + UPDATE_FAILED = 0, // Nothing to update + UPDATED_INPLACE = 1, // Value updated inplace + UPDATED = 2, // No inplace update. Merged value set +}; + + +struct AdvancedColumnFamilyOptions { + // The maximum number of write buffers that are built up in memory. + // The default and the minimum number is 2, so that when 1 write buffer + // is being flushed to storage, new writes can continue to the other + // write buffer. + // If max_write_buffer_number > 3, writing will be slowed down to + // options.delayed_write_rate if we are writing to the last write buffer + // allowed. + // + // Default: 2 + // + // Dynamically changeable through SetOptions() API + int max_write_buffer_number = 2; + + // The minimum number of write buffers that will be merged together + // before writing to storage. If set to 1, then + // all write buffers are flushed to L0 as individual files and this increases + // read amplification because a get request has to check in all of these + // files. Also, an in-memory merge may result in writing lesser + // data to storage if there are duplicate records in each of these + // individual write buffers. Default: 1 + int min_write_buffer_number_to_merge = 1; + + // The total maximum number of write buffers to maintain in memory including + // copies of buffers that have already been flushed. Unlike + // max_write_buffer_number, this parameter does not affect flushing. + // This controls the minimum amount of write history that will be available + // in memory for conflict checking when Transactions are used. + // + // When using an OptimisticTransactionDB: + // If this value is too low, some transactions may fail at commit time due + // to not being able to determine whether there were any write conflicts. + // + // When using a TransactionDB: + // If Transaction::SetSnapshot is used, TransactionDB will read either + // in-memory write buffers or SST files to do write-conflict checking. + // Increasing this value can reduce the number of reads to SST files + // done for conflict detection. + // + // Setting this value to 0 will cause write buffers to be freed immediately + // after they are flushed. + // If this value is set to -1, 'max_write_buffer_number' will be used. + // + // Default: + // If using a TransactionDB/OptimisticTransactionDB, the default value will + // be set to the value of 'max_write_buffer_number' if it is not explicitly + // set by the user. Otherwise, the default is 0. + int max_write_buffer_number_to_maintain = 0; + + // Allows thread-safe inplace updates. If this is true, there is no way to + // achieve point-in-time consistency using snapshot or iterator (assuming + // concurrent updates). Hence iterator and multi-get will return results + // which are not consistent as of any point-in-time. + // If inplace_callback function is not set, + // Put(key, new_value) will update inplace the existing_value iff + // * key exists in current memtable + // * new sizeof(new_value) <= sizeof(existing_value) + // * existing_value for that key is a put i.e. kTypeValue + // If inplace_callback function is set, check doc for inplace_callback. + // Default: false. + bool inplace_update_support = false; + + // Number of locks used for inplace update + // Default: 10000, if inplace_update_support = true, else 0. + // + // Dynamically changeable through SetOptions() API + size_t inplace_update_num_locks = 10000; + + // existing_value - pointer to previous value (from both memtable and sst). + // nullptr if key doesn't exist + // existing_value_size - pointer to size of existing_value). + // nullptr if key doesn't exist + // delta_value - Delta value to be merged with the existing_value. + // Stored in transaction logs. + // merged_value - Set when delta is applied on the previous value. + + // Applicable only when inplace_update_support is true, + // this callback function is called at the time of updating the memtable + // as part of a Put operation, lets say Put(key, delta_value). It allows the + // 'delta_value' specified as part of the Put operation to be merged with + // an 'existing_value' of the key in the database. + + // If the merged value is smaller in size that the 'existing_value', + // then this function can update the 'existing_value' buffer inplace and + // the corresponding 'existing_value'_size pointer, if it wishes to. + // The callback should return UpdateStatus::UPDATED_INPLACE. + // In this case. (In this case, the snapshot-semantics of the rocksdb + // Iterator is not atomic anymore). + + // If the merged value is larger in size than the 'existing_value' or the + // application does not wish to modify the 'existing_value' buffer inplace, + // then the merged value should be returned via *merge_value. It is set by + // merging the 'existing_value' and the Put 'delta_value'. The callback should + // return UpdateStatus::UPDATED in this case. This merged value will be added + // to the memtable. + + // If merging fails or the application does not wish to take any action, + // then the callback should return UpdateStatus::UPDATE_FAILED. + + // Please remember that the original call from the application is Put(key, + // delta_value). So the transaction log (if enabled) will still contain (key, + // delta_value). The 'merged_value' is not stored in the transaction log. + // Hence the inplace_callback function should be consistent across db reopens. + + // Default: nullptr + UpdateStatus (*inplace_callback)(char* existing_value, + uint32_t* existing_value_size, + Slice delta_value, + std::string* merged_value) = nullptr; + + // if prefix_extractor is set and memtable_prefix_bloom_size_ratio is not 0, + // create prefix bloom for memtable with the size of + // write_buffer_size * memtable_prefix_bloom_size_ratio. + // If it is larger than 0.25, it is santinized to 0.25. + // + // Default: 0 (disable) + // + // Dynamically changeable through SetOptions() API + double memtable_prefix_bloom_size_ratio = 0.0; + + // Page size for huge page for the arena used by the memtable. If <=0, it + // won't allocate from huge page but from malloc. + // Users are responsible to reserve huge pages for it to be allocated. For + // example: + // sysctl -w vm.nr_hugepages=20 + // See linux doc Documentation/vm/hugetlbpage.txt + // If there isn't enough free huge page available, it will fall back to + // malloc. + // + // Dynamically changeable through SetOptions() API + size_t memtable_huge_page_size = 0; + + // If non-nullptr, memtable will use the specified function to extract + // prefixes for keys, and for each prefix maintain a hint of insert location + // to reduce CPU usage for inserting keys with the prefix. Keys out of + // domain of the prefix extractor will be insert without using hints. + // + // Currently only the default skiplist based memtable implements the feature. + // All other memtable implementation will ignore the option. It incurs ~250 + // additional bytes of memory overhead to store a hint for each prefix. + // Also concurrent writes (when allow_concurrent_memtable_write is true) will + // ignore the option. + // + // The option is best suited for workloads where keys will likely to insert + // to a location close the last inserted key with the same prefix. + // One example could be inserting keys of the form (prefix + timestamp), + // and keys of the same prefix always comes in with time order. Another + // example would be updating the same key over and over again, in which case + // the prefix can be the key itself. + // + // Default: nullptr (disable) + std::shared_ptr + memtable_insert_with_hint_prefix_extractor = nullptr; + + // Control locality of bloom filter probes to improve cache miss rate. + // This option only applies to memtable prefix bloom and plaintable + // prefix bloom. It essentially limits every bloom checking to one cache line. + // This optimization is turned off when set to 0, and positive number to turn + // it on. + // Default: 0 + uint32_t bloom_locality = 0; + + // size of one block in arena memory allocation. + // If <= 0, a proper value is automatically calculated (usually 1/8 of + // writer_buffer_size, rounded up to a multiple of 4KB). + // + // There are two additional restriction of the specified size: + // (1) size should be in the range of [4096, 2 << 30] and + // (2) be the multiple of the CPU word (which helps with the memory + // alignment). + // + // We'll automatically check and adjust the size number to make sure it + // conforms to the restrictions. + // + // Default: 0 + // + // Dynamically changeable through SetOptions() API + size_t arena_block_size = 0; + + // Different levels can have different compression policies. There + // are cases where most lower levels would like to use quick compression + // algorithms while the higher levels (which have more data) use + // compression algorithms that have better compression but could + // be slower. This array, if non-empty, should have an entry for + // each level of the database; these override the value specified in + // the previous field 'compression'. + // + // NOTICE if level_compaction_dynamic_level_bytes=true, + // compression_per_level[0] still determines L0, but other elements + // of the array are based on base level (the level L0 files are merged + // to), and may not match the level users see from info log for metadata. + // If L0 files are merged to level-n, then, for i>0, compression_per_level[i] + // determines compaction type for level n+i-1. + // For example, if we have three 5 levels, and we determine to merge L0 + // data to L4 (which means L1..L3 will be empty), then the new files go to + // L4 uses compression type compression_per_level[1]. + // If now L0 is merged to L2. Data goes to L2 will be compressed + // according to compression_per_level[1], L3 using compression_per_level[2] + // and L4 using compression_per_level[3]. Compaction for each level can + // change when data grows. + std::vector compression_per_level; + + // Number of levels for this database + int num_levels = 7; + + // Soft limit on number of level-0 files. We start slowing down writes at this + // point. A value <0 means that no writing slow down will be triggered by + // number of files in level-0. + // + // Default: 20 + // + // Dynamically changeable through SetOptions() API + int level0_slowdown_writes_trigger = 20; + + // Maximum number of level-0 files. We stop writes at this point. + // + // Default: 36 + // + // Dynamically changeable through SetOptions() API + int level0_stop_writes_trigger = 36; + + // Target file size for compaction. + // target_file_size_base is per-file size for level-1. + // Target file size for level L can be calculated by + // target_file_size_base * (target_file_size_multiplier ^ (L-1)) + // For example, if target_file_size_base is 2MB and + // target_file_size_multiplier is 10, then each file on level-1 will + // be 2MB, and each file on level 2 will be 20MB, + // and each file on level-3 will be 200MB. + // + // Default: 64MB. + // + // Dynamically changeable through SetOptions() API + uint64_t target_file_size_base = 64 * 1048576; + + // By default target_file_size_multiplier is 1, which means + // by default files in different levels will have similar size. + // + // Dynamically changeable through SetOptions() API + int target_file_size_multiplier = 1; + + // If true, RocksDB will pick target size of each level dynamically. + // We will pick a base level b >= 1. L0 will be directly merged into level b, + // instead of always into level 1. Level 1 to b-1 need to be empty. + // We try to pick b and its target size so that + // 1. target size is in the range of + // (max_bytes_for_level_base / max_bytes_for_level_multiplier, + // max_bytes_for_level_base] + // 2. target size of the last level (level num_levels-1) equals to extra size + // of the level. + // At the same time max_bytes_for_level_multiplier and + // max_bytes_for_level_multiplier_additional are still satisfied. + // + // With this option on, from an empty DB, we make last level the base level, + // which means merging L0 data into the last level, until it exceeds + // max_bytes_for_level_base. And then we make the second last level to be + // base level, to start to merge L0 data to second last level, with its + // target size to be 1/max_bytes_for_level_multiplier of the last level's + // extra size. After the data accumulates more so that we need to move the + // base level to the third last one, and so on. + // + // For example, assume max_bytes_for_level_multiplier=10, num_levels=6, + // and max_bytes_for_level_base=10MB. + // Target sizes of level 1 to 5 starts with: + // [- - - - 10MB] + // with base level is level. Target sizes of level 1 to 4 are not applicable + // because they will not be used. + // Until the size of Level 5 grows to more than 10MB, say 11MB, we make + // base target to level 4 and now the targets looks like: + // [- - - 1.1MB 11MB] + // While data are accumulated, size targets are tuned based on actual data + // of level 5. When level 5 has 50MB of data, the target is like: + // [- - - 5MB 50MB] + // Until level 5's actual size is more than 100MB, say 101MB. Now if we keep + // level 4 to be the base level, its target size needs to be 10.1MB, which + // doesn't satisfy the target size range. So now we make level 3 the target + // size and the target sizes of the levels look like: + // [- - 1.01MB 10.1MB 101MB] + // In the same way, while level 5 further grows, all levels' targets grow, + // like + // [- - 5MB 50MB 500MB] + // Until level 5 exceeds 1000MB and becomes 1001MB, we make level 2 the + // base level and make levels' target sizes like this: + // [- 1.001MB 10.01MB 100.1MB 1001MB] + // and go on... + // + // By doing it, we give max_bytes_for_level_multiplier a priority against + // max_bytes_for_level_base, for a more predictable LSM tree shape. It is + // useful to limit worse case space amplification. + // + // max_bytes_for_level_multiplier_additional is ignored with this flag on. + // + // Turning this feature on or off for an existing DB can cause unexpected + // LSM tree structure so it's not recommended. + // + // NOTE: this option is experimental + // + // Default: false + bool level_compaction_dynamic_level_bytes = false; + + // Default: 10. + // + // Dynamically changeable through SetOptions() API + double max_bytes_for_level_multiplier = 10; + + // Different max-size multipliers for different levels. + // These are multiplied by max_bytes_for_level_multiplier to arrive + // at the max-size of each level. + // + // Default: 1 + // + // Dynamically changeable through SetOptions() API + std::vector max_bytes_for_level_multiplier_additional = + std::vector(num_levels, 1); + + // We try to limit number of bytes in one compaction to be lower than this + // threshold. But it's not guaranteed. + // Value 0 will be sanitized. + // + // Default: result.target_file_size_base * 25 + uint64_t max_compaction_bytes = 0; + + // All writes will be slowed down to at least delayed_write_rate if estimated + // bytes needed to be compaction exceed this threshold. + // + // Default: 64GB + uint64_t soft_pending_compaction_bytes_limit = 64 * 1073741824ull; + + // All writes are stopped if estimated bytes needed to be compaction exceed + // this threshold. + // + // Default: 256GB + uint64_t hard_pending_compaction_bytes_limit = 256 * 1073741824ull; + + // The compaction style. Default: kCompactionStyleLevel + CompactionStyle compaction_style = kCompactionStyleLevel; + + // If level compaction_style = kCompactionStyleLevel, for each level, + // which files are prioritized to be picked to compact. + // Default: kByCompensatedSize + CompactionPri compaction_pri = kByCompensatedSize; + + // The options needed to support Universal Style compactions + CompactionOptionsUniversal compaction_options_universal; + + // The options for FIFO compaction style + CompactionOptionsFIFO compaction_options_fifo; + + // An iteration->Next() sequentially skips over keys with the same + // user-key unless this option is set. This number specifies the number + // of keys (with the same userkey) that will be sequentially + // skipped before a reseek is issued. + // + // Default: 8 + // + // Dynamically changeable through SetOptions() API + uint64_t max_sequential_skip_in_iterations = 8; + + // This is a factory that provides MemTableRep objects. + // Default: a factory that provides a skip-list-based implementation of + // MemTableRep. + std::shared_ptr memtable_factory = + std::shared_ptr(new SkipListFactory); + + // Block-based table related options are moved to BlockBasedTableOptions. + // Related options that were originally here but now moved include: + // no_block_cache + // block_cache + // block_cache_compressed + // block_size + // block_size_deviation + // block_restart_interval + // filter_policy + // whole_key_filtering + // If you'd like to customize some of these options, you will need to + // use NewBlockBasedTableFactory() to construct a new table factory. + + // This option allows user to collect their own interested statistics of + // the tables. + // Default: empty vector -- no user-defined statistics collection will be + // performed. + typedef std::vector> + TablePropertiesCollectorFactories; + TablePropertiesCollectorFactories table_properties_collector_factories; + + // Maximum number of successive merge operations on a key in the memtable. + // + // When a merge operation is added to the memtable and the maximum number of + // successive merges is reached, the value of the key will be calculated and + // inserted into the memtable instead of the merge operation. This will + // ensure that there are never more than max_successive_merges merge + // operations in the memtable. + // + // Default: 0 (disabled) + // + // Dynamically changeable through SetOptions() API + size_t max_successive_merges = 0; + + // This flag specifies that the implementation should optimize the filters + // mainly for cases where keys are found rather than also optimize for keys + // missed. This would be used in cases where the application knows that + // there are very few misses or the performance in the case of misses is not + // important. + // + // For now, this flag allows us to not store filters for the last level i.e + // the largest level which contains data of the LSM store. For keys which + // are hits, the filters in this level are not useful because we will search + // for the data anyway. NOTE: the filters in other levels are still useful + // even for key hit because they tell us whether to look in that level or go + // to the higher level. + // + // Default: false + bool optimize_filters_for_hits = false; + + // After writing every SST file, reopen it and read all the keys. + // Default: false + bool paranoid_file_checks = false; + + // In debug mode, RocksDB run consistency checks on the LSM everytime the LSM + // change (Flush, Compaction, AddFile). These checks are disabled in release + // mode, use this option to enable them in release mode as well. + // Default: false + bool force_consistency_checks = false; + + // Measure IO stats in compactions and flushes, if true. + // Default: false + bool report_bg_io_stats = false; + + // Create ColumnFamilyOptions with default values for all fields + AdvancedColumnFamilyOptions(); + // Create ColumnFamilyOptions from Options + explicit AdvancedColumnFamilyOptions(const Options& options); + + // ---------------- OPTIONS NOT SUPPORTED ANYMORE ---------------- + + // NOT SUPPORTED ANYMORE + // This does not do anything anymore. + int max_mem_compaction_level; + + // NOT SUPPORTED ANYMORE -- this options is no longer used + // Puts are delayed to options.delayed_write_rate when any level has a + // compaction score that exceeds soft_rate_limit. This is ignored when == 0.0. + // + // Default: 0 (disabled) + // + // Dynamically changeable through SetOptions() API + double soft_rate_limit = 0.0; + + // NOT SUPPORTED ANYMORE -- this options is no longer used + double hard_rate_limit = 0.0; + + // NOT SUPPORTED ANYMORE -- this options is no longer used + unsigned int rate_limit_delay_max_milliseconds = 100; + + // NOT SUPPORTED ANYMORE + // Does not have any effect. + bool purge_redundant_kvs_while_flush = true; +}; + +} // namespace rocksdb diff --git a/include/rocksdb/c.h b/include/rocksdb/c.h index c54e6707f05..2269f7261ca 100644 --- a/include/rocksdb/c.h +++ b/include/rocksdb/c.h @@ -1,8 +1,9 @@ -/* Copyright (c) 2013, Facebook, Inc. All rights reserved. - This source code is licensed under the BSD-style license found in the - LICENSE file in the root directory of this source tree. An additional grant - of patent rights can be found in the PATENTS file in the same directory. - Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +/* Copyright (c) 2011 The LevelDB Authors. All rights reserved. Use of this source code is governed by a BSD-style license that can be found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -44,6 +45,22 @@ #ifndef STORAGE_ROCKSDB_INCLUDE_C_H_ #define STORAGE_ROCKSDB_INCLUDE_C_H_ +#pragma once + +#ifdef _WIN32 +#ifdef ROCKSDB_DLL +#ifdef ROCKSDB_LIBRARY_EXPORTS +#define ROCKSDB_LIBRARY_API __declspec(dllexport) +#else +#define ROCKSDB_LIBRARY_API __declspec(dllimport) +#endif +#else +#define ROCKSDB_LIBRARY_API +#endif +#else +#define ROCKSDB_LIBRARY_API +#endif + #ifdef __cplusplus extern "C" { #endif @@ -55,17 +72,17 @@ extern "C" { /* Exported types */ typedef struct rocksdb_t rocksdb_t; +typedef struct rocksdb_backup_engine_t rocksdb_backup_engine_t; +typedef struct rocksdb_backup_engine_info_t rocksdb_backup_engine_info_t; +typedef struct rocksdb_restore_options_t rocksdb_restore_options_t; typedef struct rocksdb_cache_t rocksdb_cache_t; typedef struct rocksdb_compactionfilter_t rocksdb_compactionfilter_t; typedef struct rocksdb_compactionfiltercontext_t rocksdb_compactionfiltercontext_t; typedef struct rocksdb_compactionfilterfactory_t rocksdb_compactionfilterfactory_t; -typedef struct rocksdb_compactionfilterv2_t - rocksdb_compactionfilterv2_t; -typedef struct rocksdb_compactionfilterfactoryv2_t - rocksdb_compactionfilterfactoryv2_t; typedef struct rocksdb_comparator_t rocksdb_comparator_t; +typedef struct rocksdb_dbpath_t rocksdb_dbpath_t; typedef struct rocksdb_env_t rocksdb_env_t; typedef struct rocksdb_fifo_compaction_options_t rocksdb_fifo_compaction_options_t; typedef struct rocksdb_filelock_t rocksdb_filelock_t; @@ -75,8 +92,11 @@ typedef struct rocksdb_iterator_t rocksdb_iterator_t; typedef struct rocksdb_logger_t rocksdb_logger_t; typedef struct rocksdb_mergeoperator_t rocksdb_mergeoperator_t; typedef struct rocksdb_options_t rocksdb_options_t; +typedef struct rocksdb_compactoptions_t rocksdb_compactoptions_t; typedef struct rocksdb_block_based_table_options_t rocksdb_block_based_table_options_t; +typedef struct rocksdb_cuckoo_table_options_t + rocksdb_cuckoo_table_options_t; typedef struct rocksdb_randomfile_t rocksdb_randomfile_t; typedef struct rocksdb_readoptions_t rocksdb_readoptions_t; typedef struct rocksdb_seqfile_t rocksdb_seqfile_t; @@ -84,465 +104,809 @@ typedef struct rocksdb_slicetransform_t rocksdb_slicetransform_t; typedef struct rocksdb_snapshot_t rocksdb_snapshot_t; typedef struct rocksdb_writablefile_t rocksdb_writablefile_t; typedef struct rocksdb_writebatch_t rocksdb_writebatch_t; +typedef struct rocksdb_writebatch_wi_t rocksdb_writebatch_wi_t; typedef struct rocksdb_writeoptions_t rocksdb_writeoptions_t; typedef struct rocksdb_universal_compaction_options_t rocksdb_universal_compaction_options_t; typedef struct rocksdb_livefiles_t rocksdb_livefiles_t; typedef struct rocksdb_column_family_handle_t rocksdb_column_family_handle_t; +typedef struct rocksdb_envoptions_t rocksdb_envoptions_t; +typedef struct rocksdb_ingestexternalfileoptions_t rocksdb_ingestexternalfileoptions_t; +typedef struct rocksdb_sstfilewriter_t rocksdb_sstfilewriter_t; +typedef struct rocksdb_ratelimiter_t rocksdb_ratelimiter_t; +typedef struct rocksdb_pinnableslice_t rocksdb_pinnableslice_t; +typedef struct rocksdb_transactiondb_options_t rocksdb_transactiondb_options_t; +typedef struct rocksdb_transactiondb_t rocksdb_transactiondb_t; +typedef struct rocksdb_transaction_options_t rocksdb_transaction_options_t; +typedef struct rocksdb_optimistictransactiondb_t rocksdb_optimistictransactiondb_t; +typedef struct rocksdb_optimistictransaction_options_t rocksdb_optimistictransaction_options_t; +typedef struct rocksdb_transaction_t rocksdb_transaction_t; +typedef struct rocksdb_checkpoint_t rocksdb_checkpoint_t; /* DB operations */ -extern rocksdb_t* rocksdb_open( - const rocksdb_options_t* options, - const char* name, - char** errptr); +extern ROCKSDB_LIBRARY_API rocksdb_t* rocksdb_open( + const rocksdb_options_t* options, const char* name, char** errptr); -extern rocksdb_t* rocksdb_open_for_read_only( - const rocksdb_options_t* options, - const char* name, - unsigned char error_if_log_file_exist, - char** errptr); +extern ROCKSDB_LIBRARY_API rocksdb_t* rocksdb_open_for_read_only( + const rocksdb_options_t* options, const char* name, + unsigned char error_if_log_file_exist, char** errptr); -extern rocksdb_t* rocksdb_open_column_families( - const rocksdb_options_t* options, - const char* name, - int num_column_families, +extern ROCKSDB_LIBRARY_API rocksdb_backup_engine_t* rocksdb_backup_engine_open( + const rocksdb_options_t* options, const char* path, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_backup_engine_create_new_backup( + rocksdb_backup_engine_t* be, rocksdb_t* db, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_backup_engine_purge_old_backups( + rocksdb_backup_engine_t* be, uint32_t num_backups_to_keep, char** errptr); + +extern ROCKSDB_LIBRARY_API rocksdb_restore_options_t* +rocksdb_restore_options_create(); +extern ROCKSDB_LIBRARY_API void rocksdb_restore_options_destroy( + rocksdb_restore_options_t* opt); +extern ROCKSDB_LIBRARY_API void rocksdb_restore_options_set_keep_log_files( + rocksdb_restore_options_t* opt, int v); + +extern ROCKSDB_LIBRARY_API void +rocksdb_backup_engine_restore_db_from_latest_backup( + rocksdb_backup_engine_t* be, const char* db_dir, const char* wal_dir, + const rocksdb_restore_options_t* restore_options, char** errptr); + +extern ROCKSDB_LIBRARY_API const rocksdb_backup_engine_info_t* +rocksdb_backup_engine_get_backup_info(rocksdb_backup_engine_t* be); + +extern ROCKSDB_LIBRARY_API int rocksdb_backup_engine_info_count( + const rocksdb_backup_engine_info_t* info); + +extern ROCKSDB_LIBRARY_API int64_t +rocksdb_backup_engine_info_timestamp(const rocksdb_backup_engine_info_t* info, + int index); + +extern ROCKSDB_LIBRARY_API uint32_t +rocksdb_backup_engine_info_backup_id(const rocksdb_backup_engine_info_t* info, + int index); + +extern ROCKSDB_LIBRARY_API uint64_t +rocksdb_backup_engine_info_size(const rocksdb_backup_engine_info_t* info, + int index); + +extern ROCKSDB_LIBRARY_API uint32_t rocksdb_backup_engine_info_number_files( + const rocksdb_backup_engine_info_t* info, int index); + +extern ROCKSDB_LIBRARY_API void rocksdb_backup_engine_info_destroy( + const rocksdb_backup_engine_info_t* info); + +extern ROCKSDB_LIBRARY_API void rocksdb_backup_engine_close( + rocksdb_backup_engine_t* be); + +extern ROCKSDB_LIBRARY_API rocksdb_checkpoint_t* +rocksdb_checkpoint_object_create(rocksdb_t* db, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_checkpoint_create( + rocksdb_checkpoint_t* checkpoint, const char* checkpoint_dir, + uint64_t log_size_for_flush, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_checkpoint_object_destroy( + rocksdb_checkpoint_t* checkpoint); + +extern ROCKSDB_LIBRARY_API rocksdb_t* rocksdb_open_column_families( + const rocksdb_options_t* options, const char* name, int num_column_families, const char** column_family_names, const rocksdb_options_t** column_family_options, - rocksdb_column_family_handle_t** column_family_handles, - char** errptr); + rocksdb_column_family_handle_t** column_family_handles, char** errptr); -extern rocksdb_t* rocksdb_open_for_read_only_column_families( - const rocksdb_options_t* options, - const char* name, - int num_column_families, +extern ROCKSDB_LIBRARY_API rocksdb_t* +rocksdb_open_for_read_only_column_families( + const rocksdb_options_t* options, const char* name, int num_column_families, const char** column_family_names, const rocksdb_options_t** column_family_options, rocksdb_column_family_handle_t** column_family_handles, - unsigned char error_if_log_file_exist, - char** errptr); + unsigned char error_if_log_file_exist, char** errptr); -char** rocksdb_list_column_families( - const rocksdb_options_t* options, - const char* name, - size_t* lencf, +extern ROCKSDB_LIBRARY_API char** rocksdb_list_column_families( + const rocksdb_options_t* options, const char* name, size_t* lencf, char** errptr); -void rocksdb_list_column_families_destroy(char** list, size_t len); -extern rocksdb_column_family_handle_t* rocksdb_create_column_family( - rocksdb_t* db, - const rocksdb_options_t* column_family_options, - const char* column_family_name, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_list_column_families_destroy( + char** list, size_t len); -extern void rocksdb_drop_column_family( - rocksdb_t* db, - rocksdb_column_family_handle_t* handle, - char** errptr); +extern ROCKSDB_LIBRARY_API rocksdb_column_family_handle_t* +rocksdb_create_column_family(rocksdb_t* db, + const rocksdb_options_t* column_family_options, + const char* column_family_name, char** errptr); -extern void rocksdb_column_family_handle_destroy(rocksdb_column_family_handle_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_drop_column_family( + rocksdb_t* db, rocksdb_column_family_handle_t* handle, char** errptr); -extern void rocksdb_close(rocksdb_t* db); +extern ROCKSDB_LIBRARY_API void rocksdb_column_family_handle_destroy( + rocksdb_column_family_handle_t*); -extern void rocksdb_put( - rocksdb_t* db, - const rocksdb_writeoptions_t* options, - const char* key, size_t keylen, - const char* val, size_t vallen, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_close(rocksdb_t* db); -extern void rocksdb_put_cf( - rocksdb_t* db, - const rocksdb_writeoptions_t* options, - rocksdb_column_family_handle_t* column_family, - const char* key, size_t keylen, - const char* val, size_t vallen, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_put( + rocksdb_t* db, const rocksdb_writeoptions_t* options, const char* key, + size_t keylen, const char* val, size_t vallen, char** errptr); -extern void rocksdb_delete( - rocksdb_t* db, - const rocksdb_writeoptions_t* options, - const char* key, size_t keylen, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_put_cf( + rocksdb_t* db, const rocksdb_writeoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, const char* val, size_t vallen, char** errptr); -void rocksdb_delete_cf( - rocksdb_t* db, - const rocksdb_writeoptions_t* options, - rocksdb_column_family_handle_t* column_family, - const char* key, size_t keylen, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_delete( + rocksdb_t* db, const rocksdb_writeoptions_t* options, const char* key, + size_t keylen, char** errptr); -extern void rocksdb_merge( - rocksdb_t* db, - const rocksdb_writeoptions_t* options, - const char* key, size_t keylen, - const char* val, size_t vallen, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_delete_cf( + rocksdb_t* db, const rocksdb_writeoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, char** errptr); -extern void rocksdb_merge_cf( - rocksdb_t* db, - const rocksdb_writeoptions_t* options, - rocksdb_column_family_handle_t* column_family, - const char* key, size_t keylen, - const char* val, size_t vallen, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_merge( + rocksdb_t* db, const rocksdb_writeoptions_t* options, const char* key, + size_t keylen, const char* val, size_t vallen, char** errptr); -extern void rocksdb_write( - rocksdb_t* db, - const rocksdb_writeoptions_t* options, - rocksdb_writebatch_t* batch, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_merge_cf( + rocksdb_t* db, const rocksdb_writeoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, const char* val, size_t vallen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_write( + rocksdb_t* db, const rocksdb_writeoptions_t* options, + rocksdb_writebatch_t* batch, char** errptr); /* Returns NULL if not found. A malloc()ed array otherwise. Stores the length of the array in *vallen. */ -extern char* rocksdb_get( - rocksdb_t* db, - const rocksdb_readoptions_t* options, - const char* key, size_t keylen, - size_t* vallen, - char** errptr); - -extern char* rocksdb_get_cf( - rocksdb_t* db, - const rocksdb_readoptions_t* options, - rocksdb_column_family_handle_t* column_family, - const char* key, size_t keylen, - size_t* vallen, - char** errptr); - -extern rocksdb_iterator_t* rocksdb_create_iterator( - rocksdb_t* db, - const rocksdb_readoptions_t* options); - -extern rocksdb_iterator_t* rocksdb_create_iterator_cf( - rocksdb_t* db, - const rocksdb_readoptions_t* options, +extern ROCKSDB_LIBRARY_API char* rocksdb_get( + rocksdb_t* db, const rocksdb_readoptions_t* options, const char* key, + size_t keylen, size_t* vallen, char** errptr); + +extern ROCKSDB_LIBRARY_API char* rocksdb_get_cf( + rocksdb_t* db, const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, size_t* vallen, char** errptr); + +// if values_list[i] == NULL and errs[i] == NULL, +// then we got status.IsNotFound(), which we will not return. +// all errors except status status.ok() and status.IsNotFound() are returned. +// +// errs, values_list and values_list_sizes must be num_keys in length, +// allocated by the caller. +// errs is a list of strings as opposed to the conventional one error, +// where errs[i] is the status for retrieval of keys_list[i]. +// each non-NULL errs entry is a malloc()ed, null terminated string. +// each non-NULL values_list entry is a malloc()ed array, with +// the length for each stored in values_list_sizes[i]. +extern ROCKSDB_LIBRARY_API void rocksdb_multi_get( + rocksdb_t* db, const rocksdb_readoptions_t* options, size_t num_keys, + const char* const* keys_list, const size_t* keys_list_sizes, + char** values_list, size_t* values_list_sizes, char** errs); + +extern ROCKSDB_LIBRARY_API void rocksdb_multi_get_cf( + rocksdb_t* db, const rocksdb_readoptions_t* options, + const rocksdb_column_family_handle_t* const* column_families, + size_t num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, char** values_list, + size_t* values_list_sizes, char** errs); + +extern ROCKSDB_LIBRARY_API rocksdb_iterator_t* rocksdb_create_iterator( + rocksdb_t* db, const rocksdb_readoptions_t* options); + +extern ROCKSDB_LIBRARY_API rocksdb_iterator_t* rocksdb_create_iterator_cf( + rocksdb_t* db, const rocksdb_readoptions_t* options, rocksdb_column_family_handle_t* column_family); -extern const rocksdb_snapshot_t* rocksdb_create_snapshot( +extern ROCKSDB_LIBRARY_API void rocksdb_create_iterators( + rocksdb_t *db, rocksdb_readoptions_t* opts, + rocksdb_column_family_handle_t** column_families, + rocksdb_iterator_t** iterators, size_t size, char** errptr); + +extern ROCKSDB_LIBRARY_API const rocksdb_snapshot_t* rocksdb_create_snapshot( rocksdb_t* db); -extern void rocksdb_release_snapshot( - rocksdb_t* db, - const rocksdb_snapshot_t* snapshot); +extern ROCKSDB_LIBRARY_API void rocksdb_release_snapshot( + rocksdb_t* db, const rocksdb_snapshot_t* snapshot); /* Returns NULL if property name is unknown. Else returns a pointer to a malloc()-ed null-terminated value. */ -extern char* rocksdb_property_value( +extern ROCKSDB_LIBRARY_API char* rocksdb_property_value(rocksdb_t* db, + const char* propname); +/* returns 0 on success, -1 otherwise */ +int rocksdb_property_int( rocksdb_t* db, - const char* propname); + const char* propname, uint64_t *out_val); -extern char* rocksdb_property_value_cf( - rocksdb_t* db, - rocksdb_column_family_handle_t* column_family, +extern ROCKSDB_LIBRARY_API char* rocksdb_property_value_cf( + rocksdb_t* db, rocksdb_column_family_handle_t* column_family, const char* propname); -extern void rocksdb_approximate_sizes( - rocksdb_t* db, - int num_ranges, - const char* const* range_start_key, const size_t* range_start_key_len, - const char* const* range_limit_key, const size_t* range_limit_key_len, - uint64_t* sizes); - -extern void rocksdb_approximate_sizes_cf( - rocksdb_t* db, - rocksdb_column_family_handle_t* column_family, - int num_ranges, - const char* const* range_start_key, const size_t* range_start_key_len, - const char* const* range_limit_key, const size_t* range_limit_key_len, - uint64_t* sizes); - -extern void rocksdb_compact_range( - rocksdb_t* db, - const char* start_key, size_t start_key_len, - const char* limit_key, size_t limit_key_len); - -extern void rocksdb_compact_range_cf( - rocksdb_t* db, - rocksdb_column_family_handle_t* column_family, - const char* start_key, size_t start_key_len, +extern ROCKSDB_LIBRARY_API void rocksdb_approximate_sizes( + rocksdb_t* db, int num_ranges, const char* const* range_start_key, + const size_t* range_start_key_len, const char* const* range_limit_key, + const size_t* range_limit_key_len, uint64_t* sizes); + +extern ROCKSDB_LIBRARY_API void rocksdb_approximate_sizes_cf( + rocksdb_t* db, rocksdb_column_family_handle_t* column_family, + int num_ranges, const char* const* range_start_key, + const size_t* range_start_key_len, const char* const* range_limit_key, + const size_t* range_limit_key_len, uint64_t* sizes); + +extern ROCKSDB_LIBRARY_API void rocksdb_compact_range(rocksdb_t* db, + const char* start_key, + size_t start_key_len, + const char* limit_key, + size_t limit_key_len); + +extern ROCKSDB_LIBRARY_API void rocksdb_compact_range_cf( + rocksdb_t* db, rocksdb_column_family_handle_t* column_family, + const char* start_key, size_t start_key_len, const char* limit_key, + size_t limit_key_len); + +extern ROCKSDB_LIBRARY_API void rocksdb_compact_range_opt( + rocksdb_t* db, rocksdb_compactoptions_t* opt, const char* start_key, + size_t start_key_len, const char* limit_key, size_t limit_key_len); + +extern ROCKSDB_LIBRARY_API void rocksdb_compact_range_cf_opt( + rocksdb_t* db, rocksdb_column_family_handle_t* column_family, + rocksdb_compactoptions_t* opt, const char* start_key, size_t start_key_len, const char* limit_key, size_t limit_key_len); -extern void rocksdb_delete_file( - rocksdb_t* db, - const char* name); +extern ROCKSDB_LIBRARY_API void rocksdb_delete_file(rocksdb_t* db, + const char* name); -extern const rocksdb_livefiles_t* rocksdb_livefiles( +extern ROCKSDB_LIBRARY_API const rocksdb_livefiles_t* rocksdb_livefiles( rocksdb_t* db); -extern void rocksdb_flush( - rocksdb_t* db, - const rocksdb_flushoptions_t* options, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_flush( + rocksdb_t* db, const rocksdb_flushoptions_t* options, char** errptr); -extern void rocksdb_disable_file_deletions( - rocksdb_t* db, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_disable_file_deletions(rocksdb_t* db, + char** errptr); -extern void rocksdb_enable_file_deletions( - rocksdb_t* db, - unsigned char force, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_enable_file_deletions( + rocksdb_t* db, unsigned char force, char** errptr); /* Management operations */ -extern void rocksdb_destroy_db( - const rocksdb_options_t* options, - const char* name, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_destroy_db( + const rocksdb_options_t* options, const char* name, char** errptr); -extern void rocksdb_repair_db( - const rocksdb_options_t* options, - const char* name, - char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_repair_db( + const rocksdb_options_t* options, const char* name, char** errptr); /* Iterator */ -extern void rocksdb_iter_destroy(rocksdb_iterator_t*); -extern unsigned char rocksdb_iter_valid(const rocksdb_iterator_t*); -extern void rocksdb_iter_seek_to_first(rocksdb_iterator_t*); -extern void rocksdb_iter_seek_to_last(rocksdb_iterator_t*); -extern void rocksdb_iter_seek(rocksdb_iterator_t*, const char* k, size_t klen); -extern void rocksdb_iter_next(rocksdb_iterator_t*); -extern void rocksdb_iter_prev(rocksdb_iterator_t*); -extern const char* rocksdb_iter_key(const rocksdb_iterator_t*, size_t* klen); -extern const char* rocksdb_iter_value(const rocksdb_iterator_t*, size_t* vlen); -extern void rocksdb_iter_get_error(const rocksdb_iterator_t*, char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_iter_destroy(rocksdb_iterator_t*); +extern ROCKSDB_LIBRARY_API unsigned char rocksdb_iter_valid( + const rocksdb_iterator_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_iter_seek_to_first(rocksdb_iterator_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_iter_seek_to_last(rocksdb_iterator_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_iter_seek(rocksdb_iterator_t*, + const char* k, size_t klen); +extern ROCKSDB_LIBRARY_API void rocksdb_iter_seek_for_prev(rocksdb_iterator_t*, + const char* k, + size_t klen); +extern ROCKSDB_LIBRARY_API void rocksdb_iter_next(rocksdb_iterator_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_iter_prev(rocksdb_iterator_t*); +extern ROCKSDB_LIBRARY_API const char* rocksdb_iter_key( + const rocksdb_iterator_t*, size_t* klen); +extern ROCKSDB_LIBRARY_API const char* rocksdb_iter_value( + const rocksdb_iterator_t*, size_t* vlen); +extern ROCKSDB_LIBRARY_API void rocksdb_iter_get_error( + const rocksdb_iterator_t*, char** errptr); /* Write batch */ -extern rocksdb_writebatch_t* rocksdb_writebatch_create(); -extern rocksdb_writebatch_t* rocksdb_writebatch_create_from(const char* rep, - size_t size); -extern void rocksdb_writebatch_destroy(rocksdb_writebatch_t*); -extern void rocksdb_writebatch_clear(rocksdb_writebatch_t*); -extern int rocksdb_writebatch_count(rocksdb_writebatch_t*); -extern void rocksdb_writebatch_put( - rocksdb_writebatch_t*, - const char* key, size_t klen, - const char* val, size_t vlen); -extern void rocksdb_writebatch_put_cf( - rocksdb_writebatch_t*, - rocksdb_column_family_handle_t* column_family, - const char* key, size_t klen, - const char* val, size_t vlen); -extern void rocksdb_writebatch_merge( - rocksdb_writebatch_t*, - const char* key, size_t klen, - const char* val, size_t vlen); -extern void rocksdb_writebatch_merge_cf( - rocksdb_writebatch_t*, - rocksdb_column_family_handle_t* column_family, - const char* key, size_t klen, - const char* val, size_t vlen); -extern void rocksdb_writebatch_delete( - rocksdb_writebatch_t*, +extern ROCKSDB_LIBRARY_API rocksdb_writebatch_t* rocksdb_writebatch_create(); +extern ROCKSDB_LIBRARY_API rocksdb_writebatch_t* rocksdb_writebatch_create_from( + const char* rep, size_t size); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_destroy( + rocksdb_writebatch_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_clear(rocksdb_writebatch_t*); +extern ROCKSDB_LIBRARY_API int rocksdb_writebatch_count(rocksdb_writebatch_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_put(rocksdb_writebatch_t*, + const char* key, + size_t klen, + const char* val, + size_t vlen); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_put_cf( + rocksdb_writebatch_t*, rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, const char* val, size_t vlen); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_putv( + rocksdb_writebatch_t* b, int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, int num_values, + const char* const* values_list, const size_t* values_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_putv_cf( + rocksdb_writebatch_t* b, rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_merge(rocksdb_writebatch_t*, + const char* key, + size_t klen, + const char* val, + size_t vlen); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_merge_cf( + rocksdb_writebatch_t*, rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, const char* val, size_t vlen); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_mergev( + rocksdb_writebatch_t* b, int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, int num_values, + const char* const* values_list, const size_t* values_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_mergev_cf( + rocksdb_writebatch_t* b, rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_delete(rocksdb_writebatch_t*, + const char* key, + size_t klen); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_delete_cf( + rocksdb_writebatch_t*, rocksdb_column_family_handle_t* column_family, const char* key, size_t klen); -extern void rocksdb_writebatch_delete_cf( - rocksdb_writebatch_t*, - rocksdb_column_family_handle_t* column_family, +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_deletev( + rocksdb_writebatch_t* b, int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_deletev_cf( + rocksdb_writebatch_t* b, rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, const size_t* keys_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_delete_range( + rocksdb_writebatch_t* b, const char* start_key, size_t start_key_len, + const char* end_key, size_t end_key_len); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_delete_range_cf( + rocksdb_writebatch_t* b, rocksdb_column_family_handle_t* column_family, + const char* start_key, size_t start_key_len, const char* end_key, + size_t end_key_len); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_delete_rangev( + rocksdb_writebatch_t* b, int num_keys, const char* const* start_keys_list, + const size_t* start_keys_list_sizes, const char* const* end_keys_list, + const size_t* end_keys_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_delete_rangev_cf( + rocksdb_writebatch_t* b, rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* start_keys_list, + const size_t* start_keys_list_sizes, const char* const* end_keys_list, + const size_t* end_keys_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_put_log_data( + rocksdb_writebatch_t*, const char* blob, size_t len); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_iterate( + rocksdb_writebatch_t*, void* state, + void (*put)(void*, const char* k, size_t klen, const char* v, size_t vlen), + void (*deleted)(void*, const char* k, size_t klen)); +extern ROCKSDB_LIBRARY_API const char* rocksdb_writebatch_data( + rocksdb_writebatch_t*, size_t* size); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_set_save_point( + rocksdb_writebatch_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_rollback_to_save_point( + rocksdb_writebatch_t*, char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_pop_save_point( + rocksdb_writebatch_t*, char** errptr); + +/* Write batch with index */ + +extern ROCKSDB_LIBRARY_API rocksdb_writebatch_wi_t* rocksdb_writebatch_wi_create( + size_t reserved_bytes, + unsigned char overwrite_keys); +extern ROCKSDB_LIBRARY_API rocksdb_writebatch_wi_t* rocksdb_writebatch_wi_create_from( + const char* rep, size_t size); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_destroy( + rocksdb_writebatch_wi_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_clear(rocksdb_writebatch_wi_t*); +extern ROCKSDB_LIBRARY_API int rocksdb_writebatch_wi_count(rocksdb_writebatch_wi_t* b); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_put(rocksdb_writebatch_wi_t*, + const char* key, + size_t klen, + const char* val, + size_t vlen); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_put_cf( + rocksdb_writebatch_wi_t*, rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, const char* val, size_t vlen); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_putv( + rocksdb_writebatch_wi_t* b, int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, int num_values, + const char* const* values_list, const size_t* values_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_putv_cf( + rocksdb_writebatch_wi_t* b, rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_merge(rocksdb_writebatch_wi_t*, + const char* key, + size_t klen, + const char* val, + size_t vlen); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_merge_cf( + rocksdb_writebatch_wi_t*, rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, const char* val, size_t vlen); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_mergev( + rocksdb_writebatch_wi_t* b, int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes, int num_values, + const char* const* values_list, const size_t* values_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_mergev_cf( + rocksdb_writebatch_wi_t* b, rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, const size_t* keys_list_sizes, + int num_values, const char* const* values_list, + const size_t* values_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_delete(rocksdb_writebatch_wi_t*, + const char* key, + size_t klen); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_delete_cf( + rocksdb_writebatch_wi_t*, rocksdb_column_family_handle_t* column_family, const char* key, size_t klen); -extern void rocksdb_writebatch_iterate( - rocksdb_writebatch_t*, +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_deletev( + rocksdb_writebatch_wi_t* b, int num_keys, const char* const* keys_list, + const size_t* keys_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_deletev_cf( + rocksdb_writebatch_wi_t* b, rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* keys_list, const size_t* keys_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_delete_range( + rocksdb_writebatch_wi_t* b, const char* start_key, size_t start_key_len, + const char* end_key, size_t end_key_len); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_delete_range_cf( + rocksdb_writebatch_wi_t* b, rocksdb_column_family_handle_t* column_family, + const char* start_key, size_t start_key_len, const char* end_key, + size_t end_key_len); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_delete_rangev( + rocksdb_writebatch_wi_t* b, int num_keys, const char* const* start_keys_list, + const size_t* start_keys_list_sizes, const char* const* end_keys_list, + const size_t* end_keys_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_delete_rangev_cf( + rocksdb_writebatch_wi_t* b, rocksdb_column_family_handle_t* column_family, + int num_keys, const char* const* start_keys_list, + const size_t* start_keys_list_sizes, const char* const* end_keys_list, + const size_t* end_keys_list_sizes); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_put_log_data( + rocksdb_writebatch_wi_t*, const char* blob, size_t len); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_iterate( + rocksdb_writebatch_wi_t* b, void* state, void (*put)(void*, const char* k, size_t klen, const char* v, size_t vlen), void (*deleted)(void*, const char* k, size_t klen)); -extern const char* rocksdb_writebatch_data(rocksdb_writebatch_t*, size_t *size); +extern ROCKSDB_LIBRARY_API const char* rocksdb_writebatch_wi_data( + rocksdb_writebatch_wi_t* b, + size_t* size); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_set_save_point( + rocksdb_writebatch_wi_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_writebatch_wi_rollback_to_save_point( + rocksdb_writebatch_wi_t*, char** errptr); +extern ROCKSDB_LIBRARY_API char* rocksdb_writebatch_wi_get_from_batch( + rocksdb_writebatch_wi_t* wbwi, + const rocksdb_options_t* options, + const char* key, size_t keylen, + size_t* vallen, + char** errptr); +extern ROCKSDB_LIBRARY_API char* rocksdb_writebatch_wi_get_from_batch_cf( + rocksdb_writebatch_wi_t* wbwi, + const rocksdb_options_t* options, + rocksdb_column_family_handle_t* column_family, + const char* key, size_t keylen, + size_t* vallen, + char** errptr); +extern ROCKSDB_LIBRARY_API char* rocksdb_writebatch_wi_get_from_batch_and_db( + rocksdb_writebatch_wi_t* wbwi, + rocksdb_t* db, + const rocksdb_readoptions_t* options, + const char* key, size_t keylen, + size_t* vallen, + char** errptr); +extern ROCKSDB_LIBRARY_API char* rocksdb_writebatch_wi_get_from_batch_and_db_cf( + rocksdb_writebatch_wi_t* wbwi, + rocksdb_t* db, + const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* column_family, + const char* key, size_t keylen, + size_t* vallen, + char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_write_writebatch_wi( + rocksdb_t* db, + const rocksdb_writeoptions_t* options, + rocksdb_writebatch_wi_t* wbwi, + char** errptr); +extern ROCKSDB_LIBRARY_API rocksdb_iterator_t* rocksdb_writebatch_wi_create_iterator_with_base( + rocksdb_writebatch_wi_t* wbwi, + rocksdb_iterator_t* base_iterator); +extern ROCKSDB_LIBRARY_API rocksdb_iterator_t* rocksdb_writebatch_wi_create_iterator_with_base_cf( + rocksdb_writebatch_wi_t* wbwi, + rocksdb_iterator_t* base_iterator, + rocksdb_column_family_handle_t* cf); + /* Block based table options */ -extern rocksdb_block_based_table_options_t* - rocksdb_block_based_options_create(); -extern void rocksdb_block_based_options_destroy( +extern ROCKSDB_LIBRARY_API rocksdb_block_based_table_options_t* +rocksdb_block_based_options_create(); +extern ROCKSDB_LIBRARY_API void rocksdb_block_based_options_destroy( rocksdb_block_based_table_options_t* options); -extern void rocksdb_block_based_options_set_block_size( +extern ROCKSDB_LIBRARY_API void rocksdb_block_based_options_set_block_size( rocksdb_block_based_table_options_t* options, size_t block_size); -extern void rocksdb_block_based_options_set_block_size_deviation( +extern ROCKSDB_LIBRARY_API void +rocksdb_block_based_options_set_block_size_deviation( rocksdb_block_based_table_options_t* options, int block_size_deviation); -extern void rocksdb_block_based_options_set_block_restart_interval( +extern ROCKSDB_LIBRARY_API void +rocksdb_block_based_options_set_block_restart_interval( rocksdb_block_based_table_options_t* options, int block_restart_interval); -extern void rocksdb_block_based_options_set_filter_policy( +extern ROCKSDB_LIBRARY_API void rocksdb_block_based_options_set_filter_policy( rocksdb_block_based_table_options_t* options, rocksdb_filterpolicy_t* filter_policy); -extern void rocksdb_block_based_options_set_no_block_cache( - rocksdb_block_based_table_options_t* options, - unsigned char no_block_cache); -extern void rocksdb_block_based_options_set_block_cache( +extern ROCKSDB_LIBRARY_API void rocksdb_block_based_options_set_no_block_cache( + rocksdb_block_based_table_options_t* options, unsigned char no_block_cache); +extern ROCKSDB_LIBRARY_API void rocksdb_block_based_options_set_block_cache( rocksdb_block_based_table_options_t* options, rocksdb_cache_t* block_cache); -extern void rocksdb_block_based_options_set_block_cache_compressed( +extern ROCKSDB_LIBRARY_API void +rocksdb_block_based_options_set_block_cache_compressed( rocksdb_block_based_table_options_t* options, rocksdb_cache_t* block_cache_compressed); -extern void rocksdb_block_based_options_set_whole_key_filtering( +extern ROCKSDB_LIBRARY_API void +rocksdb_block_based_options_set_whole_key_filtering( + rocksdb_block_based_table_options_t*, unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_block_based_options_set_format_version( + rocksdb_block_based_table_options_t*, int); +enum { + rocksdb_block_based_table_index_type_binary_search = 0, + rocksdb_block_based_table_index_type_hash_search = 1, + rocksdb_block_based_table_index_type_two_level_index_search = 2, +}; +extern ROCKSDB_LIBRARY_API void rocksdb_block_based_options_set_index_type( + rocksdb_block_based_table_options_t*, int); // uses one of the above enums +extern ROCKSDB_LIBRARY_API void +rocksdb_block_based_options_set_hash_index_allow_collision( rocksdb_block_based_table_options_t*, unsigned char); -extern void rocksdb_options_set_block_based_table_factory( - rocksdb_options_t *opt, rocksdb_block_based_table_options_t* table_options); +extern ROCKSDB_LIBRARY_API void +rocksdb_block_based_options_set_cache_index_and_filter_blocks( + rocksdb_block_based_table_options_t*, unsigned char); +extern ROCKSDB_LIBRARY_API void +rocksdb_block_based_options_set_pin_l0_filter_and_index_blocks_in_cache( + rocksdb_block_based_table_options_t*, unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_block_based_table_factory( + rocksdb_options_t* opt, rocksdb_block_based_table_options_t* table_options); + +/* Cuckoo table options */ + +extern ROCKSDB_LIBRARY_API rocksdb_cuckoo_table_options_t* +rocksdb_cuckoo_options_create(); +extern ROCKSDB_LIBRARY_API void rocksdb_cuckoo_options_destroy( + rocksdb_cuckoo_table_options_t* options); +extern ROCKSDB_LIBRARY_API void rocksdb_cuckoo_options_set_hash_ratio( + rocksdb_cuckoo_table_options_t* options, double v); +extern ROCKSDB_LIBRARY_API void rocksdb_cuckoo_options_set_max_search_depth( + rocksdb_cuckoo_table_options_t* options, uint32_t v); +extern ROCKSDB_LIBRARY_API void rocksdb_cuckoo_options_set_cuckoo_block_size( + rocksdb_cuckoo_table_options_t* options, uint32_t v); +extern ROCKSDB_LIBRARY_API void +rocksdb_cuckoo_options_set_identity_as_first_hash( + rocksdb_cuckoo_table_options_t* options, unsigned char v); +extern ROCKSDB_LIBRARY_API void rocksdb_cuckoo_options_set_use_module_hash( + rocksdb_cuckoo_table_options_t* options, unsigned char v); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_cuckoo_table_factory( + rocksdb_options_t* opt, rocksdb_cuckoo_table_options_t* table_options); /* Options */ +extern ROCKSDB_LIBRARY_API void rocksdb_set_options( + rocksdb_t* db, int count, const char* const keys[], const char* const values[], char** errptr); -extern rocksdb_options_t* rocksdb_options_create(); -extern void rocksdb_options_destroy(rocksdb_options_t*); -extern void rocksdb_options_increase_parallelism( +extern ROCKSDB_LIBRARY_API rocksdb_options_t* rocksdb_options_create(); +extern ROCKSDB_LIBRARY_API void rocksdb_options_destroy(rocksdb_options_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_increase_parallelism( rocksdb_options_t* opt, int total_threads); -extern void rocksdb_options_optimize_for_point_lookup( +extern ROCKSDB_LIBRARY_API void rocksdb_options_optimize_for_point_lookup( rocksdb_options_t* opt, uint64_t block_cache_size_mb); -extern void rocksdb_options_optimize_level_style_compaction( +extern ROCKSDB_LIBRARY_API void rocksdb_options_optimize_level_style_compaction( rocksdb_options_t* opt, uint64_t memtable_memory_budget); -extern void rocksdb_options_optimize_universal_style_compaction( +extern ROCKSDB_LIBRARY_API void +rocksdb_options_optimize_universal_style_compaction( rocksdb_options_t* opt, uint64_t memtable_memory_budget); -extern void rocksdb_options_set_compaction_filter( - rocksdb_options_t*, - rocksdb_compactionfilter_t*); -extern void rocksdb_options_set_compaction_filter_factory( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_compaction_filter( + rocksdb_options_t*, rocksdb_compactionfilter_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_compaction_filter_factory( rocksdb_options_t*, rocksdb_compactionfilterfactory_t*); -extern void rocksdb_options_set_compaction_filter_factory_v2( - rocksdb_options_t*, - rocksdb_compactionfilterfactoryv2_t*); -extern void rocksdb_options_set_comparator( - rocksdb_options_t*, - rocksdb_comparator_t*); -extern void rocksdb_options_set_merge_operator( - rocksdb_options_t*, - rocksdb_mergeoperator_t*); -extern void rocksdb_options_set_compression_per_level( - rocksdb_options_t* opt, - int* level_values, - size_t num_levels); -extern void rocksdb_options_set_create_if_missing( - rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_create_missing_column_families( +extern ROCKSDB_LIBRARY_API void rocksdb_options_compaction_readahead_size( + rocksdb_options_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_comparator( + rocksdb_options_t*, rocksdb_comparator_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_merge_operator( + rocksdb_options_t*, rocksdb_mergeoperator_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_uint64add_merge_operator( + rocksdb_options_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_compression_per_level( + rocksdb_options_t* opt, int* level_values, size_t num_levels); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_create_if_missing( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_error_if_exists( +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_create_missing_column_families(rocksdb_options_t*, + unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_error_if_exists( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_paranoid_checks( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_paranoid_checks( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_env(rocksdb_options_t*, rocksdb_env_t*); -extern void rocksdb_options_set_info_log(rocksdb_options_t*, rocksdb_logger_t*); -extern void rocksdb_options_set_info_log_level(rocksdb_options_t*, int); -extern void rocksdb_options_set_write_buffer_size(rocksdb_options_t*, size_t); -extern void rocksdb_options_set_max_open_files(rocksdb_options_t*, int); -extern void rocksdb_options_set_compression_options( - rocksdb_options_t*, int, int, int); -extern void rocksdb_options_set_prefix_extractor( - rocksdb_options_t*, rocksdb_slicetransform_t*); -extern void rocksdb_options_set_num_levels(rocksdb_options_t*, int); -extern void rocksdb_options_set_level0_file_num_compaction_trigger( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_db_paths(rocksdb_options_t*, + const rocksdb_dbpath_t** path_values, + size_t num_paths); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_env(rocksdb_options_t*, + rocksdb_env_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_info_log(rocksdb_options_t*, + rocksdb_logger_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_info_log_level( rocksdb_options_t*, int); -extern void rocksdb_options_set_level0_slowdown_writes_trigger( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_write_buffer_size( + rocksdb_options_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_db_write_buffer_size( + rocksdb_options_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_open_files( + rocksdb_options_t*, int); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_file_opening_threads( + rocksdb_options_t*, int); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_total_wal_size( + rocksdb_options_t* opt, uint64_t n); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_compression_options( + rocksdb_options_t*, int, int, int, int); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_prefix_extractor( + rocksdb_options_t*, rocksdb_slicetransform_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_num_levels( rocksdb_options_t*, int); -extern void rocksdb_options_set_level0_stop_writes_trigger( +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_level0_file_num_compaction_trigger(rocksdb_options_t*, int); +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_level0_slowdown_writes_trigger(rocksdb_options_t*, int); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_level0_stop_writes_trigger( rocksdb_options_t*, int); -extern void rocksdb_options_set_max_mem_compaction_level( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_mem_compaction_level( rocksdb_options_t*, int); -extern void rocksdb_options_set_target_file_size_base( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_target_file_size_base( rocksdb_options_t*, uint64_t); -extern void rocksdb_options_set_target_file_size_multiplier( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_target_file_size_multiplier( rocksdb_options_t*, int); -extern void rocksdb_options_set_max_bytes_for_level_base( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_bytes_for_level_base( rocksdb_options_t*, uint64_t); -extern void rocksdb_options_set_max_bytes_for_level_multiplier( +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_level_compaction_dynamic_level_bytes(rocksdb_options_t*, + unsigned char); +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_max_bytes_for_level_multiplier(rocksdb_options_t*, double); +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_max_bytes_for_level_multiplier_additional( + rocksdb_options_t*, int* level_values, size_t num_levels); +extern ROCKSDB_LIBRARY_API void rocksdb_options_enable_statistics( + rocksdb_options_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_skip_stats_update_on_db_open( + rocksdb_options_t* opt, unsigned char val); + +/* returns a pointer to a malloc()-ed, null terminated string */ +extern ROCKSDB_LIBRARY_API char* rocksdb_options_statistics_get_string( + rocksdb_options_t* opt); + +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_write_buffer_number( rocksdb_options_t*, int); -extern void rocksdb_options_set_expanded_compaction_factor( +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_min_write_buffer_number_to_merge(rocksdb_options_t*, int); +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_max_write_buffer_number_to_maintain(rocksdb_options_t*, + int); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_background_compactions( rocksdb_options_t*, int); -extern void rocksdb_options_set_max_grandparent_overlap_factor( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_base_background_compactions( rocksdb_options_t*, int); -extern void rocksdb_options_set_max_bytes_for_level_multiplier_additional( - rocksdb_options_t*, int* level_values, size_t num_levels); -extern void rocksdb_options_enable_statistics(rocksdb_options_t*); - -extern void rocksdb_options_set_max_write_buffer_number(rocksdb_options_t*, int); -extern void rocksdb_options_set_min_write_buffer_number_to_merge(rocksdb_options_t*, int); -extern void rocksdb_options_set_max_background_compactions(rocksdb_options_t*, int); -extern void rocksdb_options_set_max_background_flushes(rocksdb_options_t*, int); -extern void rocksdb_options_set_max_log_file_size(rocksdb_options_t*, size_t); -extern void rocksdb_options_set_log_file_time_to_roll(rocksdb_options_t*, size_t); -extern void rocksdb_options_set_keep_log_file_num(rocksdb_options_t*, size_t); -extern void rocksdb_options_set_soft_rate_limit(rocksdb_options_t*, double); -extern void rocksdb_options_set_hard_rate_limit(rocksdb_options_t*, double); -extern void rocksdb_options_set_rate_limit_delay_max_milliseconds( - rocksdb_options_t*, unsigned int); -extern void rocksdb_options_set_max_manifest_file_size( - rocksdb_options_t*, size_t); -extern void rocksdb_options_set_no_block_cache( - rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_table_cache_numshardbits( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_background_flushes( rocksdb_options_t*, int); -extern void rocksdb_options_set_table_cache_remove_scan_count_limit( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_log_file_size( + rocksdb_options_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_log_file_time_to_roll( + rocksdb_options_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_keep_log_file_num( + rocksdb_options_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_recycle_log_file_num( + rocksdb_options_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_soft_rate_limit( + rocksdb_options_t*, double); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_hard_rate_limit( + rocksdb_options_t*, double); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_soft_pending_compaction_bytes_limit( + rocksdb_options_t* opt, size_t v); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_hard_pending_compaction_bytes_limit( + rocksdb_options_t* opt, size_t v); +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_rate_limit_delay_max_milliseconds(rocksdb_options_t*, + unsigned int); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_manifest_file_size( + rocksdb_options_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_table_cache_numshardbits( rocksdb_options_t*, int); -extern void rocksdb_options_set_arena_block_size( +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_table_cache_remove_scan_count_limit(rocksdb_options_t*, + int); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_arena_block_size( rocksdb_options_t*, size_t); -extern void rocksdb_options_set_use_fsync( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_use_fsync( rocksdb_options_t*, int); -extern void rocksdb_options_set_db_log_dir( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_db_log_dir( rocksdb_options_t*, const char*); -extern void rocksdb_options_set_wal_dir( - rocksdb_options_t*, const char*); -extern void rocksdb_options_set_WAL_ttl_seconds( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_wal_dir(rocksdb_options_t*, + const char*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_WAL_ttl_seconds( rocksdb_options_t*, uint64_t); -extern void rocksdb_options_set_WAL_size_limit_MB( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_WAL_size_limit_MB( rocksdb_options_t*, uint64_t); -extern void rocksdb_options_set_manifest_preallocation_size( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_manifest_preallocation_size( rocksdb_options_t*, size_t); -extern void rocksdb_options_set_purge_redundant_kvs_while_flush( - rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_allow_os_buffer( +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_purge_redundant_kvs_while_flush(rocksdb_options_t*, + unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_allow_mmap_reads( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_allow_mmap_reads( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_allow_mmap_writes( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_allow_mmap_writes( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_use_direct_reads( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_is_fd_close_on_exec( +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_use_direct_io_for_flush_and_compaction(rocksdb_options_t*, + unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_is_fd_close_on_exec( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_skip_log_error_on_recovery( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_skip_log_error_on_recovery( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_stats_dump_period_sec( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_stats_dump_period_sec( rocksdb_options_t*, unsigned int); -extern void rocksdb_options_set_block_size_deviation( - rocksdb_options_t*, int); -extern void rocksdb_options_set_advise_random_on_open( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_advise_random_on_open( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_access_hint_on_compaction_start( - rocksdb_options_t*, int); -extern void rocksdb_options_set_use_adaptive_mutex( +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_access_hint_on_compaction_start(rocksdb_options_t*, int); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_use_adaptive_mutex( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_bytes_per_sync( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_bytes_per_sync( rocksdb_options_t*, uint64_t); -extern void rocksdb_options_set_verify_checksums_in_compaction( - rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_filter_deletes( - rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_max_sequential_skip_in_iterations( - rocksdb_options_t*, uint64_t); -extern void rocksdb_options_set_disable_data_sync(rocksdb_options_t*, int); -extern void rocksdb_options_set_disable_auto_compactions(rocksdb_options_t*, int); -extern void rocksdb_options_set_delete_obsolete_files_period_micros( +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_allow_concurrent_memtable_write(rocksdb_options_t*, + unsigned char); +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_enable_write_thread_adaptive_yield(rocksdb_options_t*, + unsigned char); +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_max_sequential_skip_in_iterations(rocksdb_options_t*, + uint64_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_disable_auto_compactions( + rocksdb_options_t*, int); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_optimize_filters_for_hits( + rocksdb_options_t*, int); +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_delete_obsolete_files_period_micros(rocksdb_options_t*, + uint64_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_prepare_for_bulk_load( + rocksdb_options_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_memtable_vector_rep( + rocksdb_options_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_memtable_prefix_bloom_size_ratio( + rocksdb_options_t*, double); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_compaction_bytes( rocksdb_options_t*, uint64_t); -extern void rocksdb_options_set_source_compaction_factor(rocksdb_options_t*, int); -extern void rocksdb_options_prepare_for_bulk_load(rocksdb_options_t*); -extern void rocksdb_options_set_memtable_vector_rep(rocksdb_options_t*); -extern void rocksdb_options_set_hash_skip_list_rep(rocksdb_options_t*, size_t, int32_t, int32_t); -extern void rocksdb_options_set_hash_link_list_rep(rocksdb_options_t*, size_t); -extern void rocksdb_options_set_plain_table_factory(rocksdb_options_t*, uint32_t, int, double, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_hash_skip_list_rep( + rocksdb_options_t*, size_t, int32_t, int32_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_hash_link_list_rep( + rocksdb_options_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_plain_table_factory( + rocksdb_options_t*, uint32_t, int, double, size_t); -extern void rocksdb_options_set_max_bytes_for_level_base(rocksdb_options_t* opt, uint64_t n); -extern void rocksdb_options_set_stats_dump_period_sec(rocksdb_options_t* opt, unsigned int sec); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_min_level_to_compress( + rocksdb_options_t* opt, int level); -extern void rocksdb_options_set_min_level_to_compress(rocksdb_options_t* opt, int level); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_memtable_huge_page_size( + rocksdb_options_t*, size_t); -extern void rocksdb_options_set_memtable_prefix_bloom_bits( - rocksdb_options_t*, uint32_t); -extern void rocksdb_options_set_memtable_prefix_bloom_probes( - rocksdb_options_t*, uint32_t); -extern void rocksdb_options_set_max_successive_merges( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_max_successive_merges( rocksdb_options_t*, size_t); -extern void rocksdb_options_set_min_partial_merge_operands( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_bloom_locality( rocksdb_options_t*, uint32_t); -extern void rocksdb_options_set_bloom_locality( - rocksdb_options_t*, uint32_t); -extern void rocksdb_options_set_allow_thread_local( - rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_inplace_update_support( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_inplace_update_support( rocksdb_options_t*, unsigned char); -extern void rocksdb_options_set_inplace_update_num_locks( +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_inplace_update_num_locks( rocksdb_options_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_report_bg_io_stats( + rocksdb_options_t*, int); + +enum { + rocksdb_tolerate_corrupted_tail_records_recovery = 0, + rocksdb_absolute_consistency_recovery = 1, + rocksdb_point_in_time_recovery = 2, + rocksdb_skip_any_corrupted_records_recovery = 3 +}; +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_wal_recovery_mode( + rocksdb_options_t*, int); enum { rocksdb_no_compression = 0, @@ -550,204 +914,281 @@ enum { rocksdb_zlib_compression = 2, rocksdb_bz2_compression = 3, rocksdb_lz4_compression = 4, - rocksdb_lz4hc_compression = 5 + rocksdb_lz4hc_compression = 5, + rocksdb_xpress_compression = 6, + rocksdb_zstd_compression = 7 }; -extern void rocksdb_options_set_compression(rocksdb_options_t*, int); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_compression( + rocksdb_options_t*, int); enum { rocksdb_level_compaction = 0, rocksdb_universal_compaction = 1, rocksdb_fifo_compaction = 2 }; -extern void rocksdb_options_set_compaction_style(rocksdb_options_t*, int); -extern void rocksdb_options_set_universal_compaction_options(rocksdb_options_t*, rocksdb_universal_compaction_options_t*); -extern void rocksdb_options_set_fifo_compaction_options(rocksdb_options_t* opt, - rocksdb_fifo_compaction_options_t* fifo); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_compaction_style( + rocksdb_options_t*, int); +extern ROCKSDB_LIBRARY_API void +rocksdb_options_set_universal_compaction_options( + rocksdb_options_t*, rocksdb_universal_compaction_options_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_fifo_compaction_options( + rocksdb_options_t* opt, rocksdb_fifo_compaction_options_t* fifo); +extern ROCKSDB_LIBRARY_API void rocksdb_options_set_ratelimiter( + rocksdb_options_t* opt, rocksdb_ratelimiter_t* limiter); + +/* RateLimiter */ +extern ROCKSDB_LIBRARY_API rocksdb_ratelimiter_t* rocksdb_ratelimiter_create( + int64_t rate_bytes_per_sec, int64_t refill_period_us, int32_t fairness); +extern ROCKSDB_LIBRARY_API void rocksdb_ratelimiter_destroy(rocksdb_ratelimiter_t*); /* Compaction Filter */ -extern rocksdb_compactionfilter_t* rocksdb_compactionfilter_create( - void* state, - void (*destructor)(void*), - unsigned char (*filter)( - void*, - int level, - const char* key, size_t key_length, - const char* existing_value, size_t value_length, - char** new_value, size_t *new_value_length, - unsigned char* value_changed), +extern ROCKSDB_LIBRARY_API rocksdb_compactionfilter_t* +rocksdb_compactionfilter_create( + void* state, void (*destructor)(void*), + unsigned char (*filter)(void*, int level, const char* key, + size_t key_length, const char* existing_value, + size_t value_length, char** new_value, + size_t* new_value_length, + unsigned char* value_changed), const char* (*name)(void*)); -extern void rocksdb_compactionfilter_destroy(rocksdb_compactionfilter_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_compactionfilter_set_ignore_snapshots( + rocksdb_compactionfilter_t*, unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_compactionfilter_destroy( + rocksdb_compactionfilter_t*); /* Compaction Filter Context */ -extern unsigned char rocksdb_compactionfiltercontext_is_full_compaction( +extern ROCKSDB_LIBRARY_API unsigned char +rocksdb_compactionfiltercontext_is_full_compaction( rocksdb_compactionfiltercontext_t* context); -extern unsigned char rocksdb_compactionfiltercontext_is_manual_compaction( +extern ROCKSDB_LIBRARY_API unsigned char +rocksdb_compactionfiltercontext_is_manual_compaction( rocksdb_compactionfiltercontext_t* context); /* Compaction Filter Factory */ -extern rocksdb_compactionfilterfactory_t* - rocksdb_compactionfilterfactory_create( - void* state, void (*destructor)(void*), - rocksdb_compactionfilter_t* (*create_compaction_filter)( - void*, rocksdb_compactionfiltercontext_t* context), - const char* (*name)(void*)); -extern void rocksdb_compactionfilterfactory_destroy( - rocksdb_compactionfilterfactory_t*); - -/* Compaction Filter V2 */ - -extern rocksdb_compactionfilterv2_t* rocksdb_compactionfilterv2_create( - void* state, - void (*destructor)(void*), - // num_keys specifies the number of array entries in every *list parameter. - // New values added to the new_values_list should be malloc'd and will be - // freed by the caller. Specify true in the to_delete_list to remove an - // entry during compaction; false to keep it. - void (*filter)( - void*, int level, size_t num_keys, - const char* const* keys_list, const size_t* keys_list_sizes, - const char* const* existing_values_list, const size_t* existing_values_list_sizes, - char** new_values_list, size_t* new_values_list_sizes, - unsigned char* to_delete_list), - const char* (*name)(void*)); -extern void rocksdb_compactionfilterv2_destroy(rocksdb_compactionfilterv2_t*); - -/* Compaction Filter Factory V2 */ - -extern rocksdb_compactionfilterfactoryv2_t* rocksdb_compactionfilterfactoryv2_create( - void* state, - rocksdb_slicetransform_t* prefix_extractor, - void (*destructor)(void*), - rocksdb_compactionfilterv2_t* (*create_compaction_filter_v2)( - void*, const rocksdb_compactionfiltercontext_t* context), +extern ROCKSDB_LIBRARY_API rocksdb_compactionfilterfactory_t* +rocksdb_compactionfilterfactory_create( + void* state, void (*destructor)(void*), + rocksdb_compactionfilter_t* (*create_compaction_filter)( + void*, rocksdb_compactionfiltercontext_t* context), const char* (*name)(void*)); -extern void rocksdb_compactionfilterfactoryv2_destroy(rocksdb_compactionfilterfactoryv2_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_compactionfilterfactory_destroy( + rocksdb_compactionfilterfactory_t*); /* Comparator */ -extern rocksdb_comparator_t* rocksdb_comparator_create( - void* state, - void (*destructor)(void*), - int (*compare)( - void*, - const char* a, size_t alen, - const char* b, size_t blen), +extern ROCKSDB_LIBRARY_API rocksdb_comparator_t* rocksdb_comparator_create( + void* state, void (*destructor)(void*), + int (*compare)(void*, const char* a, size_t alen, const char* b, + size_t blen), const char* (*name)(void*)); -extern void rocksdb_comparator_destroy(rocksdb_comparator_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_comparator_destroy( + rocksdb_comparator_t*); /* Filter policy */ -extern rocksdb_filterpolicy_t* rocksdb_filterpolicy_create( - void* state, - void (*destructor)(void*), - char* (*create_filter)( - void*, - const char* const* key_array, const size_t* key_length_array, - int num_keys, - size_t* filter_length), - unsigned char (*key_may_match)( - void*, - const char* key, size_t length, - const char* filter, size_t filter_length), - void (*delete_filter)( - void*, - const char* filter, size_t filter_length), +extern ROCKSDB_LIBRARY_API rocksdb_filterpolicy_t* rocksdb_filterpolicy_create( + void* state, void (*destructor)(void*), + char* (*create_filter)(void*, const char* const* key_array, + const size_t* key_length_array, int num_keys, + size_t* filter_length), + unsigned char (*key_may_match)(void*, const char* key, size_t length, + const char* filter, size_t filter_length), + void (*delete_filter)(void*, const char* filter, size_t filter_length), const char* (*name)(void*)); -extern void rocksdb_filterpolicy_destroy(rocksdb_filterpolicy_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_filterpolicy_destroy( + rocksdb_filterpolicy_t*); -extern rocksdb_filterpolicy_t* rocksdb_filterpolicy_create_bloom( - int bits_per_key); +extern ROCKSDB_LIBRARY_API rocksdb_filterpolicy_t* +rocksdb_filterpolicy_create_bloom(int bits_per_key); +extern ROCKSDB_LIBRARY_API rocksdb_filterpolicy_t* +rocksdb_filterpolicy_create_bloom_full(int bits_per_key); /* Merge Operator */ -extern rocksdb_mergeoperator_t* rocksdb_mergeoperator_create( - void* state, - void (*destructor)(void*), - char* (*full_merge)( - void*, - const char* key, size_t key_length, - const char* existing_value, size_t existing_value_length, - const char* const* operands_list, const size_t* operands_list_length, - int num_operands, - unsigned char* success, size_t* new_value_length), - char* (*partial_merge)( - void*, - const char* key, size_t key_length, - const char* const* operands_list, const size_t* operands_list_length, - int num_operands, - unsigned char* success, size_t* new_value_length), - void (*delete_value)( - void*, - const char* value, size_t value_length), +extern ROCKSDB_LIBRARY_API rocksdb_mergeoperator_t* +rocksdb_mergeoperator_create( + void* state, void (*destructor)(void*), + char* (*full_merge)(void*, const char* key, size_t key_length, + const char* existing_value, + size_t existing_value_length, + const char* const* operands_list, + const size_t* operands_list_length, int num_operands, + unsigned char* success, size_t* new_value_length), + char* (*partial_merge)(void*, const char* key, size_t key_length, + const char* const* operands_list, + const size_t* operands_list_length, int num_operands, + unsigned char* success, size_t* new_value_length), + void (*delete_value)(void*, const char* value, size_t value_length), const char* (*name)(void*)); -extern void rocksdb_mergeoperator_destroy(rocksdb_mergeoperator_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_mergeoperator_destroy( + rocksdb_mergeoperator_t*); /* Read options */ -extern rocksdb_readoptions_t* rocksdb_readoptions_create(); -extern void rocksdb_readoptions_destroy(rocksdb_readoptions_t*); -extern void rocksdb_readoptions_set_verify_checksums( - rocksdb_readoptions_t*, - unsigned char); -extern void rocksdb_readoptions_set_fill_cache( +extern ROCKSDB_LIBRARY_API rocksdb_readoptions_t* rocksdb_readoptions_create(); +extern ROCKSDB_LIBRARY_API void rocksdb_readoptions_destroy( + rocksdb_readoptions_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_readoptions_set_verify_checksums( + rocksdb_readoptions_t*, unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_readoptions_set_fill_cache( rocksdb_readoptions_t*, unsigned char); -extern void rocksdb_readoptions_set_snapshot( - rocksdb_readoptions_t*, - const rocksdb_snapshot_t*); -extern void rocksdb_readoptions_set_read_tier( +extern ROCKSDB_LIBRARY_API void rocksdb_readoptions_set_snapshot( + rocksdb_readoptions_t*, const rocksdb_snapshot_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_readoptions_set_iterate_upper_bound( + rocksdb_readoptions_t*, const char* key, size_t keylen); +extern ROCKSDB_LIBRARY_API void rocksdb_readoptions_set_read_tier( rocksdb_readoptions_t*, int); -extern void rocksdb_readoptions_set_tailing( +extern ROCKSDB_LIBRARY_API void rocksdb_readoptions_set_tailing( + rocksdb_readoptions_t*, unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_readoptions_set_readahead_size( + rocksdb_readoptions_t*, size_t); +extern ROCKSDB_LIBRARY_API void rocksdb_readoptions_set_pin_data( + rocksdb_readoptions_t*, unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_readoptions_set_total_order_seek( rocksdb_readoptions_t*, unsigned char); /* Write options */ -extern rocksdb_writeoptions_t* rocksdb_writeoptions_create(); -extern void rocksdb_writeoptions_destroy(rocksdb_writeoptions_t*); -extern void rocksdb_writeoptions_set_sync( +extern ROCKSDB_LIBRARY_API rocksdb_writeoptions_t* +rocksdb_writeoptions_create(); +extern ROCKSDB_LIBRARY_API void rocksdb_writeoptions_destroy( + rocksdb_writeoptions_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_writeoptions_set_sync( rocksdb_writeoptions_t*, unsigned char); -extern void rocksdb_writeoptions_disable_WAL(rocksdb_writeoptions_t* opt, int disable); +extern ROCKSDB_LIBRARY_API void rocksdb_writeoptions_disable_WAL( + rocksdb_writeoptions_t* opt, int disable); + +/* Compact range options */ + +extern ROCKSDB_LIBRARY_API rocksdb_compactoptions_t* +rocksdb_compactoptions_create(); +extern ROCKSDB_LIBRARY_API void rocksdb_compactoptions_destroy( + rocksdb_compactoptions_t*); +extern ROCKSDB_LIBRARY_API void +rocksdb_compactoptions_set_exclusive_manual_compaction( + rocksdb_compactoptions_t*, unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_compactoptions_set_change_level( + rocksdb_compactoptions_t*, unsigned char); +extern ROCKSDB_LIBRARY_API void rocksdb_compactoptions_set_target_level( + rocksdb_compactoptions_t*, int); /* Flush options */ -extern rocksdb_flushoptions_t* rocksdb_flushoptions_create(); -extern void rocksdb_flushoptions_destroy(rocksdb_flushoptions_t*); -extern void rocksdb_flushoptions_set_wait( +extern ROCKSDB_LIBRARY_API rocksdb_flushoptions_t* +rocksdb_flushoptions_create(); +extern ROCKSDB_LIBRARY_API void rocksdb_flushoptions_destroy( + rocksdb_flushoptions_t*); +extern ROCKSDB_LIBRARY_API void rocksdb_flushoptions_set_wait( rocksdb_flushoptions_t*, unsigned char); /* Cache */ -extern rocksdb_cache_t* rocksdb_cache_create_lru(size_t capacity); -extern void rocksdb_cache_destroy(rocksdb_cache_t* cache); +extern ROCKSDB_LIBRARY_API rocksdb_cache_t* rocksdb_cache_create_lru( + size_t capacity); +extern ROCKSDB_LIBRARY_API void rocksdb_cache_destroy(rocksdb_cache_t* cache); +extern ROCKSDB_LIBRARY_API void rocksdb_cache_set_capacity( + rocksdb_cache_t* cache, size_t capacity); +extern ROCKSDB_LIBRARY_API size_t +rocksdb_cache_get_usage(rocksdb_cache_t* cache); +extern ROCKSDB_LIBRARY_API size_t +rocksdb_cache_get_pinned_usage(rocksdb_cache_t* cache); + +/* DBPath */ + +extern ROCKSDB_LIBRARY_API rocksdb_dbpath_t* rocksdb_dbpath_create(const char* path, uint64_t target_size); +extern ROCKSDB_LIBRARY_API void rocksdb_dbpath_destroy(rocksdb_dbpath_t*); /* Env */ -extern rocksdb_env_t* rocksdb_create_default_env(); -extern void rocksdb_env_set_background_threads(rocksdb_env_t* env, int n); -extern void rocksdb_env_set_high_priority_background_threads(rocksdb_env_t* env, int n); -extern void rocksdb_env_destroy(rocksdb_env_t*); +extern ROCKSDB_LIBRARY_API rocksdb_env_t* rocksdb_create_default_env(); +extern ROCKSDB_LIBRARY_API rocksdb_env_t* rocksdb_create_mem_env(); +extern ROCKSDB_LIBRARY_API void rocksdb_env_set_background_threads( + rocksdb_env_t* env, int n); +extern ROCKSDB_LIBRARY_API void +rocksdb_env_set_high_priority_background_threads(rocksdb_env_t* env, int n); +extern ROCKSDB_LIBRARY_API void rocksdb_env_join_all_threads( + rocksdb_env_t* env); +extern ROCKSDB_LIBRARY_API void rocksdb_env_destroy(rocksdb_env_t*); + +extern ROCKSDB_LIBRARY_API rocksdb_envoptions_t* rocksdb_envoptions_create(); +extern ROCKSDB_LIBRARY_API void rocksdb_envoptions_destroy( + rocksdb_envoptions_t* opt); + +/* SstFile */ + +extern ROCKSDB_LIBRARY_API rocksdb_sstfilewriter_t* +rocksdb_sstfilewriter_create(const rocksdb_envoptions_t* env, + const rocksdb_options_t* io_options); +extern ROCKSDB_LIBRARY_API rocksdb_sstfilewriter_t* +rocksdb_sstfilewriter_create_with_comparator( + const rocksdb_envoptions_t* env, const rocksdb_options_t* io_options, + const rocksdb_comparator_t* comparator); +extern ROCKSDB_LIBRARY_API void rocksdb_sstfilewriter_open( + rocksdb_sstfilewriter_t* writer, const char* name, char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_sstfilewriter_add( + rocksdb_sstfilewriter_t* writer, const char* key, size_t keylen, + const char* val, size_t vallen, char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_sstfilewriter_put( + rocksdb_sstfilewriter_t* writer, const char* key, size_t keylen, + const char* val, size_t vallen, char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_sstfilewriter_merge( + rocksdb_sstfilewriter_t* writer, const char* key, size_t keylen, + const char* val, size_t vallen, char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_sstfilewriter_delete( + rocksdb_sstfilewriter_t* writer, const char* key, size_t keylen, + char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_sstfilewriter_finish( + rocksdb_sstfilewriter_t* writer, char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_sstfilewriter_destroy( + rocksdb_sstfilewriter_t* writer); + +extern ROCKSDB_LIBRARY_API rocksdb_ingestexternalfileoptions_t* +rocksdb_ingestexternalfileoptions_create(); +extern ROCKSDB_LIBRARY_API void +rocksdb_ingestexternalfileoptions_set_move_files( + rocksdb_ingestexternalfileoptions_t* opt, unsigned char move_files); +extern ROCKSDB_LIBRARY_API void +rocksdb_ingestexternalfileoptions_set_snapshot_consistency( + rocksdb_ingestexternalfileoptions_t* opt, + unsigned char snapshot_consistency); +extern ROCKSDB_LIBRARY_API void +rocksdb_ingestexternalfileoptions_set_allow_global_seqno( + rocksdb_ingestexternalfileoptions_t* opt, unsigned char allow_global_seqno); +extern ROCKSDB_LIBRARY_API void +rocksdb_ingestexternalfileoptions_set_allow_blocking_flush( + rocksdb_ingestexternalfileoptions_t* opt, + unsigned char allow_blocking_flush); +extern ROCKSDB_LIBRARY_API void rocksdb_ingestexternalfileoptions_destroy( + rocksdb_ingestexternalfileoptions_t* opt); + +extern ROCKSDB_LIBRARY_API void rocksdb_ingest_external_file( + rocksdb_t* db, const char* const* file_list, const size_t list_len, + const rocksdb_ingestexternalfileoptions_t* opt, char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_ingest_external_file_cf( + rocksdb_t* db, rocksdb_column_family_handle_t* handle, + const char* const* file_list, const size_t list_len, + const rocksdb_ingestexternalfileoptions_t* opt, char** errptr); /* SliceTransform */ -extern rocksdb_slicetransform_t* rocksdb_slicetransform_create( - void* state, - void (*destructor)(void*), - char* (*transform)( - void*, - const char* key, size_t length, - size_t* dst_length), - unsigned char (*in_domain)( - void*, - const char* key, size_t length), - unsigned char (*in_range)( - void*, - const char* key, size_t length), +extern ROCKSDB_LIBRARY_API rocksdb_slicetransform_t* +rocksdb_slicetransform_create( + void* state, void (*destructor)(void*), + char* (*transform)(void*, const char* key, size_t length, + size_t* dst_length), + unsigned char (*in_domain)(void*, const char* key, size_t length), + unsigned char (*in_range)(void*, const char* key, size_t length), const char* (*name)(void*)); -extern rocksdb_slicetransform_t* rocksdb_slicetransform_create_fixed_prefix(size_t); -extern void rocksdb_slicetransform_destroy(rocksdb_slicetransform_t*); +extern ROCKSDB_LIBRARY_API rocksdb_slicetransform_t* + rocksdb_slicetransform_create_fixed_prefix(size_t); +extern ROCKSDB_LIBRARY_API rocksdb_slicetransform_t* +rocksdb_slicetransform_create_noop(); +extern ROCKSDB_LIBRARY_API void rocksdb_slicetransform_destroy( + rocksdb_slicetransform_t*); /* Universal Compaction options */ @@ -756,49 +1197,276 @@ enum { rocksdb_total_size_compaction_stop_style = 1 }; -extern rocksdb_universal_compaction_options_t* rocksdb_universal_compaction_options_create() ; -extern void rocksdb_universal_compaction_options_set_size_ratio( - rocksdb_universal_compaction_options_t*, int); -extern void rocksdb_universal_compaction_options_set_min_merge_width( - rocksdb_universal_compaction_options_t*, int); -extern void rocksdb_universal_compaction_options_set_max_merge_width( - rocksdb_universal_compaction_options_t*, int); -extern void rocksdb_universal_compaction_options_set_max_size_amplification_percent( - rocksdb_universal_compaction_options_t*, int); -extern void rocksdb_universal_compaction_options_set_compression_size_percent( - rocksdb_universal_compaction_options_t*, int); -extern void rocksdb_universal_compaction_options_set_stop_style( - rocksdb_universal_compaction_options_t*, int); -extern void rocksdb_universal_compaction_options_destroy( - rocksdb_universal_compaction_options_t*); - -extern rocksdb_fifo_compaction_options_t* rocksdb_fifo_compaction_options_create(); -extern void rocksdb_fifo_compaction_options_set_max_table_files_size( +extern ROCKSDB_LIBRARY_API rocksdb_universal_compaction_options_t* +rocksdb_universal_compaction_options_create(); +extern ROCKSDB_LIBRARY_API void +rocksdb_universal_compaction_options_set_size_ratio( + rocksdb_universal_compaction_options_t*, int); +extern ROCKSDB_LIBRARY_API void +rocksdb_universal_compaction_options_set_min_merge_width( + rocksdb_universal_compaction_options_t*, int); +extern ROCKSDB_LIBRARY_API void +rocksdb_universal_compaction_options_set_max_merge_width( + rocksdb_universal_compaction_options_t*, int); +extern ROCKSDB_LIBRARY_API void +rocksdb_universal_compaction_options_set_max_size_amplification_percent( + rocksdb_universal_compaction_options_t*, int); +extern ROCKSDB_LIBRARY_API void +rocksdb_universal_compaction_options_set_compression_size_percent( + rocksdb_universal_compaction_options_t*, int); +extern ROCKSDB_LIBRARY_API void +rocksdb_universal_compaction_options_set_stop_style( + rocksdb_universal_compaction_options_t*, int); +extern ROCKSDB_LIBRARY_API void rocksdb_universal_compaction_options_destroy( + rocksdb_universal_compaction_options_t*); + +extern ROCKSDB_LIBRARY_API rocksdb_fifo_compaction_options_t* +rocksdb_fifo_compaction_options_create(); +extern ROCKSDB_LIBRARY_API void +rocksdb_fifo_compaction_options_set_max_table_files_size( rocksdb_fifo_compaction_options_t* fifo_opts, uint64_t size); -extern void rocksdb_fifo_compaction_options_destroy( +extern ROCKSDB_LIBRARY_API void rocksdb_fifo_compaction_options_destroy( rocksdb_fifo_compaction_options_t* fifo_opts); -extern int rocksdb_livefiles_count( - const rocksdb_livefiles_t*); -extern const char* rocksdb_livefiles_name( - const rocksdb_livefiles_t*, - int index); -extern int rocksdb_livefiles_level( - const rocksdb_livefiles_t*, - int index); -extern size_t rocksdb_livefiles_size( - const rocksdb_livefiles_t*, - int index); -extern const char* rocksdb_livefiles_smallestkey( - const rocksdb_livefiles_t*, - int index, - size_t* size); -extern const char* rocksdb_livefiles_largestkey( - const rocksdb_livefiles_t*, - int index, - size_t* size); -extern void rocksdb_livefiles_destroy( - const rocksdb_livefiles_t*); +extern ROCKSDB_LIBRARY_API int rocksdb_livefiles_count( + const rocksdb_livefiles_t*); +extern ROCKSDB_LIBRARY_API const char* rocksdb_livefiles_name( + const rocksdb_livefiles_t*, int index); +extern ROCKSDB_LIBRARY_API int rocksdb_livefiles_level( + const rocksdb_livefiles_t*, int index); +extern ROCKSDB_LIBRARY_API size_t +rocksdb_livefiles_size(const rocksdb_livefiles_t*, int index); +extern ROCKSDB_LIBRARY_API const char* rocksdb_livefiles_smallestkey( + const rocksdb_livefiles_t*, int index, size_t* size); +extern ROCKSDB_LIBRARY_API const char* rocksdb_livefiles_largestkey( + const rocksdb_livefiles_t*, int index, size_t* size); +extern ROCKSDB_LIBRARY_API void rocksdb_livefiles_destroy( + const rocksdb_livefiles_t*); + +/* Utility Helpers */ + +extern ROCKSDB_LIBRARY_API void rocksdb_get_options_from_string( + const rocksdb_options_t* base_options, const char* opts_str, + rocksdb_options_t* new_options, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_delete_file_in_range( + rocksdb_t* db, const char* start_key, size_t start_key_len, + const char* limit_key, size_t limit_key_len, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_delete_file_in_range_cf( + rocksdb_t* db, rocksdb_column_family_handle_t* column_family, + const char* start_key, size_t start_key_len, const char* limit_key, + size_t limit_key_len, char** errptr); + +/* Transactions */ + +extern ROCKSDB_LIBRARY_API rocksdb_column_family_handle_t* +rocksdb_transactiondb_create_column_family( + rocksdb_transactiondb_t* txn_db, + const rocksdb_options_t* column_family_options, + const char* column_family_name, char** errptr); + +extern ROCKSDB_LIBRARY_API rocksdb_transactiondb_t* rocksdb_transactiondb_open( + const rocksdb_options_t* options, + const rocksdb_transactiondb_options_t* txn_db_options, const char* name, + char** errptr); + +extern ROCKSDB_LIBRARY_API const rocksdb_snapshot_t* +rocksdb_transactiondb_create_snapshot(rocksdb_transactiondb_t* txn_db); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_release_snapshot( + rocksdb_transactiondb_t* txn_db, const rocksdb_snapshot_t* snapshot); + +extern ROCKSDB_LIBRARY_API rocksdb_transaction_t* rocksdb_transaction_begin( + rocksdb_transactiondb_t* txn_db, + const rocksdb_writeoptions_t* write_options, + const rocksdb_transaction_options_t* txn_options, + rocksdb_transaction_t* old_txn); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_commit( + rocksdb_transaction_t* txn, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_rollback( + rocksdb_transaction_t* txn, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_destroy( + rocksdb_transaction_t* txn); + +// This snapshot should be freed using rocksdb_free +extern ROCKSDB_LIBRARY_API const rocksdb_snapshot_t* +rocksdb_transaction_get_snapshot(rocksdb_transaction_t* txn); + +extern ROCKSDB_LIBRARY_API char* rocksdb_transaction_get( + rocksdb_transaction_t* txn, const rocksdb_readoptions_t* options, + const char* key, size_t klen, size_t* vlen, char** errptr); + +extern ROCKSDB_LIBRARY_API char* rocksdb_transaction_get_cf( + rocksdb_transaction_t* txn, const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, size_t klen, + size_t* vlen, char** errptr); + +extern ROCKSDB_LIBRARY_API char* rocksdb_transaction_get_for_update( + rocksdb_transaction_t* txn, const rocksdb_readoptions_t* options, + const char* key, size_t klen, size_t* vlen, unsigned char exclusive, + char** errptr); + +extern ROCKSDB_LIBRARY_API char* rocksdb_transactiondb_get( + rocksdb_transactiondb_t* txn_db, const rocksdb_readoptions_t* options, + const char* key, size_t klen, size_t* vlen, char** errptr); + +extern ROCKSDB_LIBRARY_API char* rocksdb_transactiondb_get_cf( + rocksdb_transactiondb_t* txn_db, const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, size_t* vallen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_put( + rocksdb_transaction_t* txn, const char* key, size_t klen, const char* val, + size_t vlen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_put_cf( + rocksdb_transaction_t* txn, rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, const char* val, size_t vlen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_put( + rocksdb_transactiondb_t* txn_db, const rocksdb_writeoptions_t* options, + const char* key, size_t klen, const char* val, size_t vlen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_put_cf( + rocksdb_transactiondb_t* txn_db, const rocksdb_writeoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, const char* val, size_t vallen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_write( + rocksdb_transactiondb_t* txn_db, const rocksdb_writeoptions_t* options, + rocksdb_writebatch_t *batch, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_merge( + rocksdb_transaction_t* txn, const char* key, size_t klen, const char* val, + size_t vlen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_merge( + rocksdb_transactiondb_t* txn_db, const rocksdb_writeoptions_t* options, + const char* key, size_t klen, const char* val, size_t vlen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_delete( + rocksdb_transaction_t* txn, const char* key, size_t klen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_delete_cf( + rocksdb_transaction_t* txn, rocksdb_column_family_handle_t* column_family, + const char* key, size_t klen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_delete( + rocksdb_transactiondb_t* txn_db, const rocksdb_writeoptions_t* options, + const char* key, size_t klen, char** errptr); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_delete_cf( + rocksdb_transactiondb_t* txn_db, const rocksdb_writeoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, char** errptr); + +extern ROCKSDB_LIBRARY_API rocksdb_iterator_t* +rocksdb_transaction_create_iterator(rocksdb_transaction_t* txn, + const rocksdb_readoptions_t* options); + +extern ROCKSDB_LIBRARY_API rocksdb_iterator_t* +rocksdb_transactiondb_create_iterator(rocksdb_transactiondb_t* txn_db, + const rocksdb_readoptions_t* options); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_close( + rocksdb_transactiondb_t* txn_db); + +extern ROCKSDB_LIBRARY_API rocksdb_checkpoint_t* +rocksdb_transactiondb_checkpoint_object_create(rocksdb_transactiondb_t* txn_db, + char** errptr); + +extern ROCKSDB_LIBRARY_API rocksdb_optimistictransactiondb_t* +rocksdb_optimistictransactiondb_open(const rocksdb_options_t* options, + const char* name, char** errptr); + +extern ROCKSDB_LIBRARY_API rocksdb_transaction_t* +rocksdb_optimistictransaction_begin( + rocksdb_optimistictransactiondb_t* otxn_db, + const rocksdb_writeoptions_t* write_options, + const rocksdb_optimistictransaction_options_t* otxn_options, + rocksdb_transaction_t* old_txn); + +extern ROCKSDB_LIBRARY_API void rocksdb_optimistictransactiondb_close( + rocksdb_optimistictransactiondb_t* otxn_db); + +/* Transaction Options */ + +extern ROCKSDB_LIBRARY_API rocksdb_transactiondb_options_t* +rocksdb_transactiondb_options_create(); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_options_destroy( + rocksdb_transactiondb_options_t* opt); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_options_set_max_num_locks( + rocksdb_transactiondb_options_t* opt, int64_t max_num_locks); + +extern ROCKSDB_LIBRARY_API void rocksdb_transactiondb_options_set_num_stripes( + rocksdb_transactiondb_options_t* opt, size_t num_stripes); + +extern ROCKSDB_LIBRARY_API void +rocksdb_transactiondb_options_set_transaction_lock_timeout( + rocksdb_transactiondb_options_t* opt, int64_t txn_lock_timeout); + +extern ROCKSDB_LIBRARY_API void +rocksdb_transactiondb_options_set_default_lock_timeout( + rocksdb_transactiondb_options_t* opt, int64_t default_lock_timeout); + +extern ROCKSDB_LIBRARY_API rocksdb_transaction_options_t* +rocksdb_transaction_options_create(); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_options_destroy( + rocksdb_transaction_options_t* opt); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_options_set_set_snapshot( + rocksdb_transaction_options_t* opt, unsigned char v); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_options_set_deadlock_detect( + rocksdb_transaction_options_t* opt, unsigned char v); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_options_set_lock_timeout( + rocksdb_transaction_options_t* opt, int64_t lock_timeout); + +extern ROCKSDB_LIBRARY_API void rocksdb_transaction_options_set_expiration( + rocksdb_transaction_options_t* opt, int64_t expiration); + +extern ROCKSDB_LIBRARY_API void +rocksdb_transaction_options_set_deadlock_detect_depth( + rocksdb_transaction_options_t* opt, int64_t depth); + +extern ROCKSDB_LIBRARY_API void +rocksdb_transaction_options_set_max_write_batch_size( + rocksdb_transaction_options_t* opt, size_t size); + + +extern ROCKSDB_LIBRARY_API rocksdb_optimistictransaction_options_t* +rocksdb_optimistictransaction_options_create(); + +extern ROCKSDB_LIBRARY_API void rocksdb_optimistictransaction_options_destroy( + rocksdb_optimistictransaction_options_t* opt); + +extern ROCKSDB_LIBRARY_API void +rocksdb_optimistictransaction_options_set_set_snapshot( + rocksdb_optimistictransaction_options_t* opt, unsigned char v); + +// referring to convention (3), this should be used by client +// to free memory that was malloc()ed +extern ROCKSDB_LIBRARY_API void rocksdb_free(void* ptr); + +extern ROCKSDB_LIBRARY_API rocksdb_pinnableslice_t* rocksdb_get_pinned( + rocksdb_t* db, const rocksdb_readoptions_t* options, const char* key, + size_t keylen, char** errptr); +extern ROCKSDB_LIBRARY_API rocksdb_pinnableslice_t* rocksdb_get_pinned_cf( + rocksdb_t* db, const rocksdb_readoptions_t* options, + rocksdb_column_family_handle_t* column_family, const char* key, + size_t keylen, char** errptr); +extern ROCKSDB_LIBRARY_API void rocksdb_pinnableslice_destroy( + rocksdb_pinnableslice_t* v); +extern ROCKSDB_LIBRARY_API const char* rocksdb_pinnableslice_value( + const rocksdb_pinnableslice_t* t, size_t* vlen); #ifdef __cplusplus } /* end extern "C" */ diff --git a/include/rocksdb/cache.h b/include/rocksdb/cache.h index 65d44b6cbfb..5ebd66bde88 100644 --- a/include/rocksdb/cache.h +++ b/include/rocksdb/cache.h @@ -1,7 +1,8 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -19,70 +20,109 @@ // they want something more sophisticated (like scan-resistance, a // custom eviction policy, variable cache sizing, etc.) -#ifndef STORAGE_ROCKSDB_INCLUDE_CACHE_H_ -#define STORAGE_ROCKSDB_INCLUDE_CACHE_H_ +#pragma once -#include #include +#include +#include #include "rocksdb/slice.h" +#include "rocksdb/statistics.h" +#include "rocksdb/status.h" namespace rocksdb { -using std::shared_ptr; - class Cache; // Create a new cache with a fixed size capacity. The cache is sharded -// to 2^numShardBits shards, by hash of the key. The total capacity -// is divided and evenly assigned to each shard. Inside each shard, -// the eviction is done in two passes: first try to free spaces by -// evicting entries that are among the most least used removeScanCountLimit -// entries and do not have reference other than by the cache itself, in -// the least-used order. If not enough space is freed, further free the -// entries in least used order. +// to 2^num_shard_bits shards, by hash of the key. The total capacity +// is divided and evenly assigned to each shard. If strict_capacity_limit +// is set, insert to the cache will fail when cache is full. User can also +// set percentage of the cache reserves for high priority entries via +// high_pri_pool_pct. +// num_shard_bits = -1 means it is automatically determined: every shard +// will be at least 512KB and number of shard bits will not exceed 6. +extern std::shared_ptr NewLRUCache(size_t capacity, + int num_shard_bits = -1, + bool strict_capacity_limit = false, + double high_pri_pool_ratio = 0.0); + +// Similar to NewLRUCache, but create a cache based on CLOCK algorithm with +// better concurrent performance in some cases. See util/clock_cache.cc for +// more detail. // -// The functions without parameter numShardBits and/or removeScanCountLimit -// use default values. removeScanCountLimit's default value is 0, which -// means a strict LRU order inside each shard. -extern shared_ptr NewLRUCache(size_t capacity); -extern shared_ptr NewLRUCache(size_t capacity, int numShardBits); -extern shared_ptr NewLRUCache(size_t capacity, int numShardBits, - int removeScanCountLimit); +// Return nullptr if it is not supported. +extern std::shared_ptr NewClockCache(size_t capacity, + int num_shard_bits = -1, + bool strict_capacity_limit = false); class Cache { public: - Cache() { } + // Depending on implementation, cache entries with high priority could be less + // likely to get evicted than low priority entries. + enum class Priority { HIGH, LOW }; + + Cache() {} // Destroys all existing entries by calling the "deleter" - // function that was passed to the constructor. - virtual ~Cache(); + // function that was passed via the Insert() function. + // + // @See Insert + virtual ~Cache() {} // Opaque handle to an entry stored in the cache. - struct Handle { }; + struct Handle {}; + + // The type of the Cache + virtual const char* Name() const = 0; // Insert a mapping from key->value into the cache and assign it // the specified charge against the total cache capacity. + // If strict_capacity_limit is true and cache reaches its full capacity, + // return Status::Incomplete. // - // Returns a handle that corresponds to the mapping. The caller - // must call this->Release(handle) when the returned mapping is no - // longer needed. + // If handle is not nullptr, returns a handle that corresponds to the + // mapping. The caller must call this->Release(handle) when the returned + // mapping is no longer needed. In case of error caller is responsible to + // cleanup the value (i.e. calling "deleter"). + // + // If handle is nullptr, it is as if Release is called immediately after + // insert. In case of error value will be cleanup. // // When the inserted entry is no longer needed, the key and // value will be passed to "deleter". - virtual Handle* Insert(const Slice& key, void* value, size_t charge, - void (*deleter)(const Slice& key, void* value)) = 0; + virtual Status Insert(const Slice& key, void* value, size_t charge, + void (*deleter)(const Slice& key, void* value), + Handle** handle = nullptr, + Priority priority = Priority::LOW) = 0; // If the cache has no mapping for "key", returns nullptr. // // Else return a handle that corresponds to the mapping. The caller // must call this->Release(handle) when the returned mapping is no // longer needed. - virtual Handle* Lookup(const Slice& key) = 0; + // If stats is not nullptr, relative tickers could be used inside the + // function. + virtual Handle* Lookup(const Slice& key, Statistics* stats = nullptr) = 0; - // Release a mapping returned by a previous Lookup(). + // Increments the reference count for the handle if it refers to an entry in + // the cache. Returns true if refcount was incremented; otherwise, returns + // false. + // REQUIRES: handle must have been returned by a method on *this. + virtual bool Ref(Handle* handle) = 0; + + /** + * Release a mapping returned by a previous Lookup(). A released entry might + * still remain in cache in case it is later looked up by others. If + * force_erase is set then it also erase it from the cache if there is no + * other reference to it. Erasing it should call the deleter function that + * was provided when the + * entry was inserted. + * + * Returns true if the entry was also erased. + */ // REQUIRES: handle must not have been released yet. // REQUIRES: handle must have been returned by a method on *this. - virtual void Release(Handle* handle) = 0; + virtual bool Release(Handle* handle, bool force_erase = false) = 0; // Return the value encapsulated in a handle returned by a // successful Lookup(). @@ -94,26 +134,45 @@ class Cache { // underlying entry will be kept around until all existing handles // to it have been released. virtual void Erase(const Slice& key) = 0; - // Return a new numeric id. May be used by multiple clients who are - // sharing the same cache to partition the key space. Typically the + // sharding the same cache to partition the key space. Typically the // client will allocate a new id at startup and prepend the id to // its cache keys. virtual uint64_t NewId() = 0; + // sets the maximum configured capacity of the cache. When the new + // capacity is less than the old capacity and the existing usage is + // greater than new capacity, the implementation will do its best job to + // purge the released entries from the cache in order to lower the usage + virtual void SetCapacity(size_t capacity) = 0; + + // Set whether to return error on insertion when cache reaches its full + // capacity. + virtual void SetStrictCapacityLimit(bool strict_capacity_limit) = 0; + + // Get the flag whether to return error on insertion when cache reaches its + // full capacity. + virtual bool HasStrictCapacityLimit() const = 0; + // returns the maximum configured capacity of the cache virtual size_t GetCapacity() const = 0; // returns the memory size for the entries residing in the cache. virtual size_t GetUsage() const = 0; + // returns the memory size for a specific entry in the cache. + virtual size_t GetUsage(Handle* handle) const = 0; + + // returns the memory size for the entries in use by the system + virtual size_t GetPinnedUsage() const = 0; + // Call this on shutdown if you want to speed it up. Cache will disown // any underlying data and will not free it on delete. This call will leak // memory - call this only if you're shutting down the process. // Any attempts of using cache after this call will fail terribly. // Always delete the DB object before calling this method! - virtual void DisownData() { - // default implementation is noop + virtual void DisownData(){ + // default implementation is noop }; // Apply callback to all entries in the cache @@ -122,19 +181,20 @@ class Cache { virtual void ApplyToAllCacheEntries(void (*callback)(void*, size_t), bool thread_safe) = 0; - private: - void LRU_Remove(Handle* e); - void LRU_Append(Handle* e); - void Unref(Handle* e); + // Remove all entries. + // Prerequisite: no entry is referenced. + virtual void EraseUnRefEntries() = 0; + + virtual std::string GetPrintableOptions() const { return ""; } - struct Rep; - Rep* rep_; + // Mark the last inserted object as being a raw data block. This will be used + // in tests. The default implementation does nothing. + virtual void TEST_mark_as_data_block(const Slice& key, size_t charge) {} + private: // No copying allowed Cache(const Cache&); - void operator=(const Cache&); + Cache& operator=(const Cache&); }; } // namespace rocksdb - -#endif // STORAGE_ROCKSDB_UTIL_CACHE_H_ diff --git a/include/rocksdb/cleanable.h b/include/rocksdb/cleanable.h new file mode 100644 index 00000000000..cd2e9425f12 --- /dev/null +++ b/include/rocksdb/cleanable.h @@ -0,0 +1,82 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// An iterator yields a sequence of key/value pairs from a source. +// The following class defines the interface. Multiple implementations +// are provided by this library. In particular, iterators are provided +// to access the contents of a Table or a DB. +// +// Multiple threads can invoke const methods on an Iterator without +// external synchronization, but if any of the threads may call a +// non-const method, all threads accessing the same Iterator must use +// external synchronization. + +#ifndef INCLUDE_ROCKSDB_CLEANABLE_H_ +#define INCLUDE_ROCKSDB_CLEANABLE_H_ + +namespace rocksdb { + +class Cleanable { + public: + Cleanable(); + ~Cleanable(); + + // No copy constructor and copy assignment allowed. + Cleanable(Cleanable&) = delete; + Cleanable& operator=(Cleanable&) = delete; + + // Move consturctor and move assignment is allowed. + Cleanable(Cleanable&&); + Cleanable& operator=(Cleanable&&); + + // Clients are allowed to register function/arg1/arg2 triples that + // will be invoked when this iterator is destroyed. + // + // Note that unlike all of the preceding methods, this method is + // not abstract and therefore clients should not override it. + typedef void (*CleanupFunction)(void* arg1, void* arg2); + void RegisterCleanup(CleanupFunction function, void* arg1, void* arg2); + void DelegateCleanupsTo(Cleanable* other); + // DoCleanup and also resets the pointers for reuse + inline void Reset() { + DoCleanup(); + cleanup_.function = nullptr; + cleanup_.next = nullptr; + } + + protected: + struct Cleanup { + CleanupFunction function; + void* arg1; + void* arg2; + Cleanup* next; + }; + Cleanup cleanup_; + // It also becomes the owner of c + void RegisterCleanup(Cleanup* c); + + private: + // Performs all the cleanups. It does not reset the pointers. Making it + // private + // to prevent misuse + inline void DoCleanup() { + if (cleanup_.function != nullptr) { + (*cleanup_.function)(cleanup_.arg1, cleanup_.arg2); + for (Cleanup* c = cleanup_.next; c != nullptr;) { + (*c->function)(c->arg1, c->arg2); + Cleanup* next = c->next; + delete c; + c = next; + } + } + } +}; + +} // namespace rocksdb + +#endif // INCLUDE_ROCKSDB_CLEANABLE_H_ diff --git a/include/rocksdb/compaction_filter.h b/include/rocksdb/compaction_filter.h index dce69d2d70c..64f61a35e0c 100644 --- a/include/rocksdb/compaction_filter.h +++ b/include/rocksdb/compaction_filter.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2013 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -9,6 +9,7 @@ #ifndef STORAGE_ROCKSDB_INCLUDE_COMPACTION_FILTER_H_ #define STORAGE_ROCKSDB_INCLUDE_COMPACTION_FILTER_H_ +#include #include #include #include @@ -32,6 +33,19 @@ struct CompactionFilterContext { class CompactionFilter { public: + enum ValueType { + kValue, + kMergeOperand, + kBlobIndex, // used internally by BlobDB. + }; + + enum class Decision { + kKeep, + kRemove, + kChangeValue, + kRemoveAndSkipUntil, + }; + // Context information of a compaction run struct Context { // Does this compaction run include all data files @@ -39,6 +53,8 @@ class CompactionFilter { // Is this compaction requested by the client (true), // or is it occurring as an automatic compaction process bool is_manual_compaction; + // Which column family this compaction is for. + uint32_t column_family_id; }; virtual ~CompactionFilter() {} @@ -51,10 +67,24 @@ class CompactionFilter { // output of the compaction. The application can inspect // the existing value of the key and make decision based on it. // + // Key-Values that are results of merge operation during compaction are not + // passed into this function. Currently, when you have a mix of Put()s and + // Merge()s on a same key, we only guarantee to process the merge operands + // through the compaction filters. Put()s might be processed, or might not. + // // When the value is to be preserved, the application has the option // to modify the existing_value and pass it back through new_value. // value_changed needs to be set to true in this case. // + // If you use snapshot feature of RocksDB (i.e. call GetSnapshot() API on a + // DB* object), CompactionFilter might not be very useful for you. Due to + // guarantees we need to maintain, compaction process will not call Filter() + // on any keys that were written before the latest snapshot. In other words, + // compaction will only call Filter() on keys written after your most recent + // call to GetSnapshot(). In most cases, Filter() will not be called very + // often. This is something we're fixing. See the discussion at: + // https://www.facebook.com/groups/mysqlonrocksdb/permalink/999723240091865/ + // // If multithreaded compaction is being used *and* a single CompactionFilter // instance was supplied via Options::compaction_filter, this method may be // called from different threads concurrently. The application must ensure @@ -64,44 +94,100 @@ class CompactionFilter { // be used by a single thread that is doing the compaction run, and this // call does not need to be thread-safe. However, multiple filters may be // in existence and operating concurrently. - virtual bool Filter(int level, - const Slice& key, - const Slice& existing_value, - std::string* new_value, - bool* value_changed) const = 0; - - // Returns a name that identifies this compaction filter. - // The name will be printed to LOG file on start up for diagnosis. - virtual const char* Name() const = 0; -}; + // + // The last paragraph is not true if you set max_subcompactions to more than + // 1. In that case, subcompaction from multiple threads may call a single + // CompactionFilter concurrently. + virtual bool Filter(int level, const Slice& key, const Slice& existing_value, + std::string* new_value, bool* value_changed) const { + return false; + } -// CompactionFilterV2 that buffers kv pairs sharing the same prefix and let -// application layer to make individual decisions for all the kv pairs in the -// buffer. -class CompactionFilterV2 { - public: - virtual ~CompactionFilterV2() {} + // The compaction process invokes this method on every merge operand. If this + // method returns true, the merge operand will be ignored and not written out + // in the compaction output + // + // Note: If you are using a TransactionDB, it is not recommended to implement + // FilterMergeOperand(). If a Merge operation is filtered out, TransactionDB + // may not realize there is a write conflict and may allow a Transaction to + // Commit that should have failed. Instead, it is better to implement any + // Merge filtering inside the MergeOperator. + virtual bool FilterMergeOperand(int level, const Slice& key, + const Slice& operand) const { + return false; + } - // The compaction process invokes this method for all the kv pairs - // sharing the same prefix. It is a "roll-up" version of CompactionFilter. + // An extended API. Called for both values and merge operands. + // Allows changing value and skipping ranges of keys. + // The default implementation uses Filter() and FilterMergeOperand(). + // If you're overriding this method, no need to override the other two. + // `value_type` indicates whether this key-value corresponds to a normal + // value (e.g. written with Put()) or a merge operand (written with Merge()). + // + // Possible return values: + // * kKeep - keep the key-value pair. + // * kRemove - remove the key-value pair or merge operand. + // * kChangeValue - keep the key and change the value/operand to *new_value. + // * kRemoveAndSkipUntil - remove this key-value pair, and also remove + // all key-value pairs with key in [key, *skip_until). This range + // of keys will be skipped without reading, potentially saving some + // IO operations compared to removing the keys one by one. // - // Each entry in the return vector indicates if the corresponding kv should - // be preserved in the output of this compaction run. The application can - // inspect the existing values of the keys and make decision based on it. + // *skip_until <= key is treated the same as Decision::kKeep + // (since the range [key, *skip_until) is empty). // - // When a value is to be preserved, the application has the option - // to modify the entry in existing_values and pass it back through an entry - // in new_values. A corresponding values_changed entry needs to be set to - // true in this case. Note that the new_values vector contains only changed - // values, i.e. new_values.size() <= values_changed.size(). + // Caveats: + // - The keys are skipped even if there are snapshots containing them, + // as if IgnoreSnapshots() was true; i.e. values removed + // by kRemoveAndSkipUntil can disappear from a snapshot - beware + // if you're using TransactionDB or DB::GetSnapshot(). + // - If value for a key was overwritten or merged into (multiple Put()s + // or Merge()s), and compaction filter skips this key with + // kRemoveAndSkipUntil, it's possible that it will remove only + // the new value, exposing the old value that was supposed to be + // overwritten. + // - Doesn't work with PlainTableFactory in prefix mode. + // - If you use kRemoveAndSkipUntil, consider also reducing + // compaction_readahead_size option. // - typedef std::vector SliceVector; - virtual std::vector Filter(int level, - const SliceVector& keys, - const SliceVector& existing_values, - std::vector* new_values, - std::vector* values_changed) - const = 0; + // Note: If you are using a TransactionDB, it is not recommended to filter + // out or modify merge operands (ValueType::kMergeOperand). + // If a merge operation is filtered out, TransactionDB may not realize there + // is a write conflict and may allow a Transaction to Commit that should have + // failed. Instead, it is better to implement any Merge filtering inside the + // MergeOperator. + virtual Decision FilterV2(int level, const Slice& key, ValueType value_type, + const Slice& existing_value, std::string* new_value, + std::string* skip_until) const { + switch (value_type) { + case ValueType::kValue: { + bool value_changed = false; + bool rv = Filter(level, key, existing_value, new_value, &value_changed); + if (rv) { + return Decision::kRemove; + } + return value_changed ? Decision::kChangeValue : Decision::kKeep; + } + case ValueType::kMergeOperand: { + bool rv = FilterMergeOperand(level, key, existing_value); + return rv ? Decision::kRemove : Decision::kKeep; + } + case ValueType::kBlobIndex: + return Decision::kKeep; + } + assert(false); + return Decision::kKeep; + } + + // By default, compaction will only call Filter() on keys written after the + // most recent call to GetSnapshot(). However, if the compaction filter + // overrides IgnoreSnapshots to make it return true, the compaction filter + // will be called even if the keys were written before the last snapshot. + // This behavior is to be used only when we want to delete a set of keys + // irrespective of snapshots. In particular, care should be taken + // to understand that the values of these keys will change even if we are + // using a snapshot. + virtual bool IgnoreSnapshots() const { return false; } // Returns a name that identifies this compaction filter. // The name will be printed to LOG file on start up for diagnosis. @@ -121,79 +207,6 @@ class CompactionFilterFactory { virtual const char* Name() const = 0; }; -// Default implementation of CompactionFilterFactory which does not -// return any filter -class DefaultCompactionFilterFactory : public CompactionFilterFactory { - public: - virtual std::unique_ptr CreateCompactionFilter( - const CompactionFilter::Context& context) override { - return std::unique_ptr(nullptr); - } - - virtual const char* Name() const override { - return "DefaultCompactionFilterFactory"; - } -}; - -// Each compaction will create a new CompactionFilterV2 -// -// CompactionFilterFactoryV2 enables application to specify a prefix and use -// CompactionFilterV2 to filter kv-pairs in batches. Each batch contains all -// the kv-pairs sharing the same prefix. -// -// This is useful for applications that require grouping kv-pairs in -// compaction filter to make a purge/no-purge decision. For example, if the -// key prefix is user id and the rest of key represents the type of value. -// This batching filter will come in handy if the application's compaction -// filter requires knowledge of all types of values for any user id. -// -class CompactionFilterFactoryV2 { - public: - // NOTE: CompactionFilterFactoryV2 will not delete prefix_extractor - explicit CompactionFilterFactoryV2(const SliceTransform* prefix_extractor) - : prefix_extractor_(prefix_extractor) { } - - virtual ~CompactionFilterFactoryV2() { } - - virtual std::unique_ptr CreateCompactionFilterV2( - const CompactionFilterContext& context) = 0; - - // Returns a name that identifies this compaction filter factory. - virtual const char* Name() const = 0; - - const SliceTransform* GetPrefixExtractor() const { - return prefix_extractor_; - } - - void SetPrefixExtractor(const SliceTransform* prefix_extractor) { - prefix_extractor_ = prefix_extractor; - } - - private: - // Prefix extractor for compaction filter v2 - // Keys sharing the same prefix will be buffered internally. - // Client can implement a Filter callback function to operate on the buffer - const SliceTransform* prefix_extractor_; -}; - -// Default implementation of CompactionFilterFactoryV2 which does not -// return any filter -class DefaultCompactionFilterFactoryV2 : public CompactionFilterFactoryV2 { - public: - explicit DefaultCompactionFilterFactoryV2() - : CompactionFilterFactoryV2(nullptr) { } - - virtual std::unique_ptr - CreateCompactionFilterV2( - const CompactionFilterContext& context) override { - return std::unique_ptr(nullptr); - } - - virtual const char* Name() const override { - return "DefaultCompactionFilterFactoryV2"; - } -}; - } // namespace rocksdb #endif // STORAGE_ROCKSDB_INCLUDE_COMPACTION_FILTER_H_ diff --git a/include/rocksdb/compaction_job_stats.h b/include/rocksdb/compaction_job_stats.h new file mode 100644 index 00000000000..ebb04a46bff --- /dev/null +++ b/include/rocksdb/compaction_job_stats.h @@ -0,0 +1,91 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#include +#include +#include + +namespace rocksdb { +struct CompactionJobStats { + CompactionJobStats() { Reset(); } + void Reset(); + // Aggregate the CompactionJobStats from another instance with this one + void Add(const CompactionJobStats& stats); + + // the elapsed time of this compaction in microseconds. + uint64_t elapsed_micros; + + // the number of compaction input records. + uint64_t num_input_records; + // the number of compaction input files. + size_t num_input_files; + // the number of compaction input files at the output level. + size_t num_input_files_at_output_level; + + // the number of compaction output records. + uint64_t num_output_records; + // the number of compaction output files. + size_t num_output_files; + + // true if the compaction is a manual compaction + bool is_manual_compaction; + + // the size of the compaction input in bytes. + uint64_t total_input_bytes; + // the size of the compaction output in bytes. + uint64_t total_output_bytes; + + // number of records being replaced by newer record associated with same key. + // this could be a new value or a deletion entry for that key so this field + // sums up all updated and deleted keys + uint64_t num_records_replaced; + + // the sum of the uncompressed input keys in bytes. + uint64_t total_input_raw_key_bytes; + // the sum of the uncompressed input values in bytes. + uint64_t total_input_raw_value_bytes; + + // the number of deletion entries before compaction. Deletion entries + // can disappear after compaction because they expired + uint64_t num_input_deletion_records; + // number of deletion records that were found obsolete and discarded + // because it is not possible to delete any more keys with this entry + // (i.e. all possible deletions resulting from it have been completed) + uint64_t num_expired_deletion_records; + + // number of corrupt keys (ParseInternalKey returned false when applied to + // the key) encountered and written out. + uint64_t num_corrupt_keys; + + // Following counters are only populated if + // options.report_bg_io_stats = true; + + // Time spent on file's Append() call. + uint64_t file_write_nanos; + + // Time spent on sync file range. + uint64_t file_range_sync_nanos; + + // Time spent on file fsync. + uint64_t file_fsync_nanos; + + // Time spent on preparing file write (falocate, etc) + uint64_t file_prepare_write_nanos; + + // 0-terminated strings storing the first 8 bytes of the smallest and + // largest key in the output. + static const size_t kMaxPrefixLength = 8; + + std::string smallest_output_key_prefix; + std::string largest_output_key_prefix; + + // number of single-deletes which do not meet a put + uint64_t num_single_del_fallthru; + + // number of single-deletes which meet something other than a put + uint64_t num_single_del_mismatch; +}; +} // namespace rocksdb diff --git a/include/rocksdb/comparator.h b/include/rocksdb/comparator.h index f3a8499a8f5..64db73a7244 100644 --- a/include/rocksdb/comparator.h +++ b/include/rocksdb/comparator.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -29,6 +29,15 @@ class Comparator { // > 0 iff "a" > "b" virtual int Compare(const Slice& a, const Slice& b) const = 0; + // Compares two slices for equality. The following invariant should always + // hold (and is the default implementation): + // Equal(a, b) iff Compare(a, b) == 0 + // Overwrite only if equality comparisons can be done more efficiently than + // three-way comparisons. + virtual bool Equal(const Slice& a, const Slice& b) const { + return Compare(a, b) == 0; + } + // The name of the comparator. Used to check for comparator // mismatches (i.e., a DB created with one comparator is // accessed using a different comparator. @@ -55,6 +64,10 @@ class Comparator { // Simple comparator implementations may return with *key unchanged, // i.e., an implementation of this method that does nothing is correct. virtual void FindShortSuccessor(std::string* key) const = 0; + + // if it is a wrapped comparator, may return the root one. + // return itself it is not wrapped. + virtual const Comparator* GetRootComparator() const { return this; } }; // Return a builtin comparator that uses lexicographic byte-wise @@ -62,6 +75,10 @@ class Comparator { // must not be deleted. extern const Comparator* BytewiseComparator(); +// Return a builtin comparator that uses reverse lexicographic byte-wise +// ordering. +extern const Comparator* ReverseBytewiseComparator(); + } // namespace rocksdb #endif // STORAGE_ROCKSDB_INCLUDE_COMPARATOR_H_ diff --git a/include/rocksdb/convenience.h b/include/rocksdb/convenience.h new file mode 100644 index 00000000000..4a60afb11dc --- /dev/null +++ b/include/rocksdb/convenience.h @@ -0,0 +1,339 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include +#include + +#include "rocksdb/db.h" +#include "rocksdb/options.h" +#include "rocksdb/table.h" + +namespace rocksdb { + +#ifndef ROCKSDB_LITE +// The following set of functions provide a way to construct RocksDB Options +// from a string or a string-to-string map. Here're the general rule of +// setting option values from strings by type. Some RocksDB types are also +// supported in these APIs. Please refer to the comment of the function itself +// to find more information about how to config those RocksDB types. +// +// * Strings: +// Strings will be used as values directly without any truncating or +// trimming. +// +// * Booleans: +// - "true" or "1" => true +// - "false" or "0" => false. +// [Example]: +// - {"optimize_filters_for_hits", "1"} in GetColumnFamilyOptionsFromMap, or +// - "optimize_filters_for_hits=true" in GetColumnFamilyOptionsFromString. +// +// * Integers: +// Integers are converted directly from string, in addition to the following +// units that we support: +// - 'k' or 'K' => 2^10 +// - 'm' or 'M' => 2^20 +// - 'g' or 'G' => 2^30 +// - 't' or 'T' => 2^40 // only for unsigned int with sufficient bits. +// [Example]: +// - {"arena_block_size", "19G"} in GetColumnFamilyOptionsFromMap, or +// - "arena_block_size=19G" in GetColumnFamilyOptionsFromString. +// +// * Doubles / Floating Points: +// Doubles / Floating Points are converted directly from string. Note that +// currently we do not support units. +// [Example]: +// - {"hard_rate_limit", "2.1"} in GetColumnFamilyOptionsFromMap, or +// - "hard_rate_limit=2.1" in GetColumnFamilyOptionsFromString. +// * Array / Vectors: +// An array is specified by a list of values, where ':' is used as +// the delimiter to separate each value. +// [Example]: +// - {"compression_per_level", "kNoCompression:kSnappyCompression"} +// in GetColumnFamilyOptionsFromMap, or +// - "compression_per_level=kNoCompression:kSnappyCompression" in +// GetColumnFamilyOptionsFromMapString +// * Enums: +// The valid values of each enum are identical to the names of its constants. +// [Example]: +// - CompressionType: valid values are "kNoCompression", +// "kSnappyCompression", "kZlibCompression", "kBZip2Compression", ... +// - CompactionStyle: valid values are "kCompactionStyleLevel", +// "kCompactionStyleUniversal", "kCompactionStyleFIFO", and +// "kCompactionStyleNone". +// + +// Take a default ColumnFamilyOptions "base_options" in addition to a +// map "opts_map" of option name to option value to construct the new +// ColumnFamilyOptions "new_options". +// +// Below are the instructions of how to config some non-primitive-typed +// options in ColumnFOptions: +// +// * table_factory: +// table_factory can be configured using our custom nested-option syntax. +// +// {option_a=value_a; option_b=value_b; option_c=value_c; ... } +// +// A nested option is enclosed by two curly braces, within which there are +// multiple option assignments. Each assignment is of the form +// "variable_name=value;". +// +// Currently we support the following types of TableFactory: +// - BlockBasedTableFactory: +// Use name "block_based_table_factory" to initialize table_factory with +// BlockBasedTableFactory. Its BlockBasedTableFactoryOptions can be +// configured using the nested-option syntax. +// [Example]: +// * {"block_based_table_factory", "{block_cache=1M;block_size=4k;}"} +// is equivalent to assigning table_factory with a BlockBasedTableFactory +// that has 1M LRU block-cache with block size equals to 4k: +// ColumnFamilyOptions cf_opt; +// BlockBasedTableOptions blk_opt; +// blk_opt.block_cache = NewLRUCache(1 * 1024 * 1024); +// blk_opt.block_size = 4 * 1024; +// cf_opt.table_factory.reset(NewBlockBasedTableFactory(blk_opt)); +// - PlainTableFactory: +// Use name "plain_table_factory" to initialize table_factory with +// PlainTableFactory. Its PlainTableFactoryOptions can be configured using +// the nested-option syntax. +// [Example]: +// * {"plain_table_factory", "{user_key_len=66;bloom_bits_per_key=20;}"} +// +// * memtable_factory: +// Use "memtable" to config memtable_factory. Here are the supported +// memtable factories: +// - SkipList: +// Pass "skip_list:" to config memtable to use SkipList, +// or simply "skip_list" to use the default SkipList. +// [Example]: +// * {"memtable", "skip_list:5"} is equivalent to setting +// memtable to SkipListFactory(5). +// - PrefixHash: +// Pass "prfix_hash:" to config memtable +// to use PrefixHash, or simply "prefix_hash" to use the default +// PrefixHash. +// [Example]: +// * {"memtable", "prefix_hash:1000"} is equivalent to setting +// memtable to NewHashSkipListRepFactory(hash_bucket_count). +// - HashLinkedList: +// Pass "hash_linkedlist:" to config memtable +// to use HashLinkedList, or simply "hash_linkedlist" to use the default +// HashLinkedList. +// [Example]: +// * {"memtable", "hash_linkedlist:1000"} is equivalent to +// setting memtable to NewHashLinkListRepFactory(1000). +// - VectorRepFactory: +// Pass "vector:" to config memtable to use VectorRepFactory, +// or simply "vector" to use the default Vector memtable. +// [Example]: +// * {"memtable", "vector:1024"} is equivalent to setting memtable +// to VectorRepFactory(1024). +// - HashCuckooRepFactory: +// Pass "cuckoo:" to use HashCuckooRepFactory with the +// specified write buffer size, or simply "cuckoo" to use the default +// HashCuckooRepFactory. +// [Example]: +// * {"memtable", "cuckoo:1024"} is equivalent to setting memtable +// to NewHashCuckooRepFactory(1024). +// +// * compression_opts: +// Use "compression_opts" to config compression_opts. The value format +// is of the form ":::". +// [Example]: +// * {"compression_opts", "4:5:6:7"} is equivalent to setting: +// ColumnFamilyOptions cf_opt; +// cf_opt.compression_opts.window_bits = 4; +// cf_opt.compression_opts.level = 5; +// cf_opt.compression_opts.strategy = 6; +// cf_opt.compression_opts.max_dict_bytes = 7; +// +// @param base_options the default options of the output "new_options". +// @param opts_map an option name to value map for specifying how "new_options" +// should be set. +// @param new_options the resulting options based on "base_options" with the +// change specified in "opts_map". +// @param input_strings_escaped when set to true, each escaped characters +// prefixed by '\' in the values of the opts_map will be further converted +// back to the raw string before assigning to the associated options. +// @param ignore_unknown_options when set to true, unknown options are ignored +// instead of resulting in an unknown-option error. +// @return Status::OK() on success. Otherwise, a non-ok status indicating +// error will be returned, and "new_options" will be set to "base_options". +Status GetColumnFamilyOptionsFromMap( + const ColumnFamilyOptions& base_options, + const std::unordered_map& opts_map, + ColumnFamilyOptions* new_options, bool input_strings_escaped = false, + bool ignore_unknown_options = false); + +// Take a default DBOptions "base_options" in addition to a +// map "opts_map" of option name to option value to construct the new +// DBOptions "new_options". +// +// Below are the instructions of how to config some non-primitive-typed +// options in DBOptions: +// +// * rate_limiter_bytes_per_sec: +// RateLimiter can be configured directly by specifying its bytes_per_sec. +// [Example]: +// - Passing {"rate_limiter_bytes_per_sec", "1024"} is equivalent to +// passing NewGenericRateLimiter(1024) to rate_limiter_bytes_per_sec. +// +// @param base_options the default options of the output "new_options". +// @param opts_map an option name to value map for specifying how "new_options" +// should be set. +// @param new_options the resulting options based on "base_options" with the +// change specified in "opts_map". +// @param input_strings_escaped when set to true, each escaped characters +// prefixed by '\' in the values of the opts_map will be further converted +// back to the raw string before assigning to the associated options. +// @param ignore_unknown_options when set to true, unknown options are ignored +// instead of resulting in an unknown-option error. +// @return Status::OK() on success. Otherwise, a non-ok status indicating +// error will be returned, and "new_options" will be set to "base_options". +Status GetDBOptionsFromMap( + const DBOptions& base_options, + const std::unordered_map& opts_map, + DBOptions* new_options, bool input_strings_escaped = false, + bool ignore_unknown_options = false); + +// Take a default BlockBasedTableOptions "table_options" in addition to a +// map "opts_map" of option name to option value to construct the new +// BlockBasedTableOptions "new_table_options". +// +// Below are the instructions of how to config some non-primitive-typed +// options in BlockBasedTableOptions: +// +// * filter_policy: +// We currently only support the following FilterPolicy in the convenience +// functions: +// - BloomFilter: use "bloomfilter:[bits_per_key]:[use_block_based_builder]" +// to specify BloomFilter. The above string is equivalent to calling +// NewBloomFilterPolicy(bits_per_key, use_block_based_builder). +// [Example]: +// - Pass {"filter_policy", "bloomfilter:4:true"} in +// GetBlockBasedTableOptionsFromMap to use a BloomFilter with 4-bits +// per key and use_block_based_builder enabled. +// +// * block_cache / block_cache_compressed: +// We currently only support LRU cache in the GetOptions API. The LRU +// cache can be set by directly specifying its size. +// [Example]: +// - Passing {"block_cache", "1M"} in GetBlockBasedTableOptionsFromMap is +// equivalent to setting block_cache using NewLRUCache(1024 * 1024). +// +// @param table_options the default options of the output "new_table_options". +// @param opts_map an option name to value map for specifying how +// "new_table_options" should be set. +// @param new_table_options the resulting options based on "table_options" +// with the change specified in "opts_map". +// @param input_strings_escaped when set to true, each escaped characters +// prefixed by '\' in the values of the opts_map will be further converted +// back to the raw string before assigning to the associated options. +// @param ignore_unknown_options when set to true, unknown options are ignored +// instead of resulting in an unknown-option error. +// @return Status::OK() on success. Otherwise, a non-ok status indicating +// error will be returned, and "new_table_options" will be set to +// "table_options". +Status GetBlockBasedTableOptionsFromMap( + const BlockBasedTableOptions& table_options, + const std::unordered_map& opts_map, + BlockBasedTableOptions* new_table_options, + bool input_strings_escaped = false, bool ignore_unknown_options = false); + +// Take a default PlainTableOptions "table_options" in addition to a +// map "opts_map" of option name to option value to construct the new +// PlainTableOptions "new_table_options". +// +// @param table_options the default options of the output "new_table_options". +// @param opts_map an option name to value map for specifying how +// "new_table_options" should be set. +// @param new_table_options the resulting options based on "table_options" +// with the change specified in "opts_map". +// @param input_strings_escaped when set to true, each escaped characters +// prefixed by '\' in the values of the opts_map will be further converted +// back to the raw string before assigning to the associated options. +// @param ignore_unknown_options when set to true, unknown options are ignored +// instead of resulting in an unknown-option error. +// @return Status::OK() on success. Otherwise, a non-ok status indicating +// error will be returned, and "new_table_options" will be set to +// "table_options". +Status GetPlainTableOptionsFromMap( + const PlainTableOptions& table_options, + const std::unordered_map& opts_map, + PlainTableOptions* new_table_options, bool input_strings_escaped = false, + bool ignore_unknown_options = false); + +// Take a string representation of option names and values, apply them into the +// base_options, and return the new options as a result. The string has the +// following format: +// "write_buffer_size=1024;max_write_buffer_number=2" +// Nested options config is also possible. For example, you can define +// BlockBasedTableOptions as part of the string for block-based table factory: +// "write_buffer_size=1024;block_based_table_factory={block_size=4k};" +// "max_write_buffer_num=2" +Status GetColumnFamilyOptionsFromString( + const ColumnFamilyOptions& base_options, + const std::string& opts_str, + ColumnFamilyOptions* new_options); + +Status GetDBOptionsFromString( + const DBOptions& base_options, + const std::string& opts_str, + DBOptions* new_options); + +Status GetStringFromDBOptions(std::string* opts_str, + const DBOptions& db_options, + const std::string& delimiter = "; "); + +Status GetStringFromColumnFamilyOptions(std::string* opts_str, + const ColumnFamilyOptions& cf_options, + const std::string& delimiter = "; "); + +Status GetStringFromCompressionType(std::string* compression_str, + CompressionType compression_type); + +std::vector GetSupportedCompressions(); + +Status GetBlockBasedTableOptionsFromString( + const BlockBasedTableOptions& table_options, + const std::string& opts_str, + BlockBasedTableOptions* new_table_options); + +Status GetPlainTableOptionsFromString( + const PlainTableOptions& table_options, + const std::string& opts_str, + PlainTableOptions* new_table_options); + +Status GetMemTableRepFactoryFromString( + const std::string& opts_str, + std::unique_ptr* new_mem_factory); + +Status GetOptionsFromString(const Options& base_options, + const std::string& opts_str, Options* new_options); + +Status StringToMap(const std::string& opts_str, + std::unordered_map* opts_map); + +// Request stopping background work, if wait is true wait until it's done +void CancelAllBackgroundWork(DB* db, bool wait = false); + +// Delete files which are entirely in the given range +// Could leave some keys in the range which are in files which are not +// entirely in the range. +// Snapshots before the delete might not see the data in the given range. +Status DeleteFilesInRange(DB* db, ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end); + +// Verify the checksum of file +Status VerifySstFileChecksum(const Options& options, + const EnvOptions& env_options, + const std::string& file_path); +#endif // ROCKSDB_LITE + +} // namespace rocksdb diff --git a/include/rocksdb/db.h b/include/rocksdb/db.h index d9be6b4270a..964f7b1db45 100644 --- a/include/rocksdb/db.h +++ b/include/rocksdb/db.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -11,26 +11,52 @@ #include #include +#include #include -#include #include #include -#include "rocksdb/version.h" +#include #include "rocksdb/iterator.h" +#include "rocksdb/listener.h" +#include "rocksdb/metadata.h" #include "rocksdb/options.h" -#include "rocksdb/types.h" +#include "rocksdb/snapshot.h" +#include "rocksdb/sst_file_writer.h" +#include "rocksdb/thread_status.h" #include "rocksdb/transaction_log.h" +#include "rocksdb/types.h" +#include "rocksdb/version.h" + +#ifdef _WIN32 +// Windows API macro interference +#undef DeleteFile +#endif + +#if defined(__GNUC__) || defined(__clang__) +#define ROCKSDB_DEPRECATED_FUNC __attribute__((__deprecated__)) +#elif _WIN32 +#define ROCKSDB_DEPRECATED_FUNC __declspec(deprecated) +#endif namespace rocksdb { +struct Options; +struct DBOptions; +struct ColumnFamilyOptions; +struct ReadOptions; +struct WriteOptions; +struct FlushOptions; +struct CompactionOptions; +struct CompactRangeOptions; +struct TableProperties; +struct ExternalSstFileInfo; +class WriteBatch; +class Env; +class EventListener; + using std::unique_ptr; -class ColumnFamilyHandle { - public: - virtual ~ColumnFamilyHandle() {} -}; extern const std::string kDefaultColumnFamilyName; - struct ColumnFamilyDescriptor { std::string name; ColumnFamilyOptions options; @@ -41,37 +67,28 @@ struct ColumnFamilyDescriptor { : name(_name), options(_options) {} }; -static const int kMajorVersion = __ROCKSDB_MAJOR__; -static const int kMinorVersion = __ROCKSDB_MINOR__; - -struct Options; -struct ReadOptions; -struct WriteOptions; -struct FlushOptions; -struct TableProperties; -class WriteBatch; -class Env; - -// Metadata associated with each SST file. -struct LiveFileMetaData { - std::string column_family_name; // Name of the column family - std::string db_path; - std::string name; // Name of the file - int level; // Level at which this file resides. - size_t size; // File size in bytes. - std::string smallestkey; // Smallest user defined key in the file. - std::string largestkey; // Largest user defined key in the file. - SequenceNumber smallest_seqno; // smallest seqno in file - SequenceNumber largest_seqno; // largest seqno in file +class ColumnFamilyHandle { + public: + virtual ~ColumnFamilyHandle() {} + // Returns the name of the column family associated with the current handle. + virtual const std::string& GetName() const = 0; + // Returns the ID of the column family associated with the current handle. + virtual uint32_t GetID() const = 0; + // Fills "*desc" with the up-to-date descriptor of the column family + // associated with this handle. Since it fills "*desc" with the up-to-date + // information, this call might internally lock and release DB mutex to + // access the up-to-date CF options. In addition, all the pointer-typed + // options cannot be referenced any longer than the original options exist. + // + // Note that this function is not supported in RocksDBLite. + virtual Status GetDescriptor(ColumnFamilyDescriptor* desc) = 0; + // Returns the comparator of the column family associated with the + // current handle. + virtual const Comparator* GetComparator() const = 0; }; -// Abstract handle to particular state of a DB. -// A Snapshot is an immutable object and can therefore be safely -// accessed from multiple threads without any external synchronization. -class Snapshot { - protected: - virtual ~Snapshot(); -}; +static const int kMajorVersion = __ROCKSDB_MAJOR__; +static const int kMinorVersion = __ROCKSDB_MINOR__; // A range of keys struct Range { @@ -106,6 +123,9 @@ class DB { // that modify data, like put/delete, will return error. // If the db is opened in read only mode, then no compactions // will happen. + // + // Not supported in ROCKSDB_LITE, in which case the function will + // return Status::NotSupported. static Status OpenForReadOnly(const Options& options, const std::string& name, DB** dbptr, bool error_if_log_file_exist = false); @@ -115,6 +135,9 @@ class DB { // database that should be opened. However, you always need to specify default // column family. The default column family name is 'default' and it's stored // in rocksdb::kDefaultColumnFamilyName + // + // Not supported in ROCKSDB_LITE, in which case the function will + // return Status::NotSupported. static Status OpenForReadOnly( const DBOptions& db_options, const std::string& name, const std::vector& column_families, @@ -123,7 +146,7 @@ class DB { // Open DB with column families. // db_options specify database specific options - // column_families is the vector of all column families in the databse, + // column_families is the vector of all column families in the database, // containing column family name and options. You need to open ALL column // families in the database. To get the list of column families, you can use // ListColumnFamilies(). Also, you can open only a subset of column families @@ -132,7 +155,9 @@ class DB { // in rocksdb::kDefaultColumnFamilyName. // If everything is OK, handles will on return be the same size // as column_families --- handles[i] will be a handle that you - // will use to operate on column family column_family[i] + // will use to operate on column family column_family[i]. + // Before delete DB, you have to close All column families by calling + // DestroyColumnFamilyHandle() with all the handles. static Status Open(const DBOptions& db_options, const std::string& name, const std::vector& column_families, std::vector* handles, DB** dbptr); @@ -154,11 +179,43 @@ class DB { const std::string& column_family_name, ColumnFamilyHandle** handle); + // Bulk create column families with the same column family options. + // Return the handles of the column families through the argument handles. + // In case of error, the request may succeed partially, and handles will + // contain column family handles that it managed to create, and have size + // equal to the number of created column families. + virtual Status CreateColumnFamilies( + const ColumnFamilyOptions& options, + const std::vector& column_family_names, + std::vector* handles); + + // Bulk create column families. + // Return the handles of the column families through the argument handles. + // In case of error, the request may succeed partially, and handles will + // contain column family handles that it managed to create, and have size + // equal to the number of created column families. + virtual Status CreateColumnFamilies( + const std::vector& column_families, + std::vector* handles); + // Drop a column family specified by column_family handle. This call // only records a drop record in the manifest and prevents the column // family from flushing and compacting. virtual Status DropColumnFamily(ColumnFamilyHandle* column_family); + // Bulk drop column families. This call only records drop records in the + // manifest and prevents the column families from flushing and compacting. + // In case of error, the request may succeed partially. User may call + // ListColumnFamilies to check the result. + virtual Status DropColumnFamilies( + const std::vector& column_families); + + // Close a column family specified by column_family handle and destroy + // the column family handle specified to avoid double deletion. This call + // deletes the column family handle by default. Use this method to + // close column family instead of deleting column family handle directly + virtual Status DestroyColumnFamilyHandle(ColumnFamilyHandle* column_family); + // Set the database entry for "key" to "value". // If "key" already exists, it will be overwritten. // Returns OK on success, and a non-OK status on error. @@ -182,6 +239,48 @@ class DB { return Delete(options, DefaultColumnFamily(), key); } + // Remove the database entry for "key". Requires that the key exists + // and was not overwritten. Returns OK on success, and a non-OK status + // on error. It is not an error if "key" did not exist in the database. + // + // If a key is overwritten (by calling Put() multiple times), then the result + // of calling SingleDelete() on this key is undefined. SingleDelete() only + // behaves correctly if there has been only one Put() for this key since the + // previous call to SingleDelete() for this key. + // + // This feature is currently an experimental performance optimization + // for a very specific workload. It is up to the caller to ensure that + // SingleDelete is only used for a key that is not deleted using Delete() or + // written using Merge(). Mixing SingleDelete operations with Deletes and + // Merges can result in undefined behavior. + // + // Note: consider setting options.sync = true. + virtual Status SingleDelete(const WriteOptions& options, + ColumnFamilyHandle* column_family, + const Slice& key) = 0; + virtual Status SingleDelete(const WriteOptions& options, const Slice& key) { + return SingleDelete(options, DefaultColumnFamily(), key); + } + + // Removes the database entries in the range ["begin_key", "end_key"), i.e., + // including "begin_key" and excluding "end_key". Returns OK on success, and + // a non-OK status on error. It is not an error if no keys exist in the range + // ["begin_key", "end_key"). + // + // This feature is currently an experimental performance optimization for + // deleting very large ranges of contiguous keys. Invoking it many times or on + // small ranges may severely degrade read performance; in particular, the + // resulting performance can be worse than calling Delete() for each key in + // the range. Note also the degraded read performance affects keys outside the + // deleted ranges, and affects database operations involving scans, like flush + // and compaction. + // + // Consider setting ReadOptions::ignore_range_deletions = true to speed + // up reads for key(s) that are known to be unaffected by range deletions. + virtual Status DeleteRange(const WriteOptions& options, + ColumnFamilyHandle* column_family, + const Slice& begin_key, const Slice& end_key); + // Merge the database entry for "key" with "value". Returns OK on success, // and a non-OK status on error. The semantics of this operation is // determined by the user provided merge_operator when opening DB. @@ -195,6 +294,8 @@ class DB { } // Apply the specified updates to the database. + // If `updates` contains no update, WAL will still be synced if + // options.sync=true. // Returns OK on success, non-OK on failure. // Note: consider setting options.sync = true. virtual Status Write(const WriteOptions& options, WriteBatch* updates) = 0; @@ -206,9 +307,21 @@ class DB { // a status for which Status::IsNotFound() returns true. // // May return some other Status on an error. + virtual inline Status Get(const ReadOptions& options, + ColumnFamilyHandle* column_family, const Slice& key, + std::string* value) { + assert(value != nullptr); + PinnableSlice pinnable_val(value); + assert(!pinnable_val.IsPinned()); + auto s = Get(options, column_family, key, &pinnable_val); + if (s.ok() && pinnable_val.IsPinned()) { + value->assign(pinnable_val.data(), pinnable_val.size()); + } // else value is already assigned + return s; + } virtual Status Get(const ReadOptions& options, ColumnFamilyHandle* column_family, const Slice& key, - std::string* value) = 0; + PinnableSlice* value) = 0; virtual Status Get(const ReadOptions& options, const Slice& key, std::string* value) { return Get(options, DefaultColumnFamily(), key, value); } @@ -242,9 +355,10 @@ class DB { // This check is potentially lighter-weight than invoking DB::Get(). One way // to make this lighter weight is to avoid doing any IOs. // Default implementation here returns true and sets 'value_found' to false - virtual bool KeyMayExist(const ReadOptions& options, - ColumnFamilyHandle* column_family, const Slice& key, - std::string* value, bool* value_found = nullptr) { + virtual bool KeyMayExist(const ReadOptions& /*options*/, + ColumnFamilyHandle* /*column_family*/, + const Slice& /*key*/, std::string* /*value*/, + bool* value_found = nullptr) { if (value_found != nullptr) { *value_found = false; } @@ -287,34 +401,270 @@ class DB { // use "snapshot" after this call. virtual void ReleaseSnapshot(const Snapshot* snapshot) = 0; - // DB implementations can export properties about their state - // via this method. If "property" is a valid property understood by this - // DB implementation, fills "*value" with its current value and returns - // true. Otherwise returns false. - // - // - // Valid property names include: +#ifndef ROCKSDB_LITE + // Contains all valid property arguments for GetProperty(). // - // "rocksdb.num-files-at-level" - return the number of files at level , - // where is an ASCII representation of a level number (e.g. "0"). - // "rocksdb.stats" - returns a multi-line string that describes statistics - // about the internal operation of the DB. - // "rocksdb.sstables" - returns a multi-line string that describes all - // of the sstables that make up the db contents. + // NOTE: Property names cannot end in numbers since those are interpreted as + // arguments, e.g., see kNumFilesAtLevelPrefix. + struct Properties { + // "rocksdb.num-files-at-level" - returns string containing the number + // of files at level , where is an ASCII representation of a + // level number (e.g., "0"). + static const std::string kNumFilesAtLevelPrefix; + + // "rocksdb.compression-ratio-at-level" - returns string containing the + // compression ratio of data at level , where is an ASCII + // representation of a level number (e.g., "0"). Here, compression + // ratio is defined as uncompressed data size / compressed file size. + // Returns "-1.0" if no open files at level . + static const std::string kCompressionRatioAtLevelPrefix; + + // "rocksdb.stats" - returns a multi-line string containing the data + // described by kCFStats followed by the data described by kDBStats. + static const std::string kStats; + + // "rocksdb.sstables" - returns a multi-line string summarizing current + // SST files. + static const std::string kSSTables; + + // "rocksdb.cfstats" - Both of "rocksdb.cfstats-no-file-histogram" and + // "rocksdb.cf-file-histogram" together. See below for description + // of the two. + static const std::string kCFStats; + + // "rocksdb.cfstats-no-file-histogram" - returns a multi-line string with + // general columm family stats per-level over db's lifetime ("L"), + // aggregated over db's lifetime ("Sum"), and aggregated over the + // interval since the last retrieval ("Int"). + // It could also be used to return the stats in the format of the map. + // In this case there will a pair of string to array of double for + // each level as well as for "Sum". "Int" stats will not be affected + // when this form of stats are retrieved. + static const std::string kCFStatsNoFileHistogram; + + // "rocksdb.cf-file-histogram" - print out how many file reads to every + // level, as well as the histogram of latency of single requests. + static const std::string kCFFileHistogram; + + // "rocksdb.dbstats" - returns a multi-line string with general database + // stats, both cumulative (over the db's lifetime) and interval (since + // the last retrieval of kDBStats). + static const std::string kDBStats; + + // "rocksdb.levelstats" - returns multi-line string containing the number + // of files per level and total size of each level (MB). + static const std::string kLevelStats; + + // "rocksdb.num-immutable-mem-table" - returns number of immutable + // memtables that have not yet been flushed. + static const std::string kNumImmutableMemTable; + + // "rocksdb.num-immutable-mem-table-flushed" - returns number of immutable + // memtables that have already been flushed. + static const std::string kNumImmutableMemTableFlushed; + + // "rocksdb.mem-table-flush-pending" - returns 1 if a memtable flush is + // pending; otherwise, returns 0. + static const std::string kMemTableFlushPending; + + // "rocksdb.num-running-flushes" - returns the number of currently running + // flushes. + static const std::string kNumRunningFlushes; + + // "rocksdb.compaction-pending" - returns 1 if at least one compaction is + // pending; otherwise, returns 0. + static const std::string kCompactionPending; + + // "rocksdb.num-running-compactions" - returns the number of currently + // running compactions. + static const std::string kNumRunningCompactions; + + // "rocksdb.background-errors" - returns accumulated number of background + // errors. + static const std::string kBackgroundErrors; + + // "rocksdb.cur-size-active-mem-table" - returns approximate size of active + // memtable (bytes). + static const std::string kCurSizeActiveMemTable; + + // "rocksdb.cur-size-all-mem-tables" - returns approximate size of active + // and unflushed immutable memtables (bytes). + static const std::string kCurSizeAllMemTables; + + // "rocksdb.size-all-mem-tables" - returns approximate size of active, + // unflushed immutable, and pinned immutable memtables (bytes). + static const std::string kSizeAllMemTables; + + // "rocksdb.num-entries-active-mem-table" - returns total number of entries + // in the active memtable. + static const std::string kNumEntriesActiveMemTable; + + // "rocksdb.num-entries-imm-mem-tables" - returns total number of entries + // in the unflushed immutable memtables. + static const std::string kNumEntriesImmMemTables; + + // "rocksdb.num-deletes-active-mem-table" - returns total number of delete + // entries in the active memtable. + static const std::string kNumDeletesActiveMemTable; + + // "rocksdb.num-deletes-imm-mem-tables" - returns total number of delete + // entries in the unflushed immutable memtables. + static const std::string kNumDeletesImmMemTables; + + // "rocksdb.estimate-num-keys" - returns estimated number of total keys in + // the active and unflushed immutable memtables and storage. + static const std::string kEstimateNumKeys; + + // "rocksdb.estimate-table-readers-mem" - returns estimated memory used for + // reading SST tables, excluding memory used in block cache (e.g., + // filter and index blocks). + static const std::string kEstimateTableReadersMem; + + // "rocksdb.is-file-deletions-enabled" - returns 0 if deletion of obsolete + // files is enabled; otherwise, returns a non-zero number. + static const std::string kIsFileDeletionsEnabled; + + // "rocksdb.num-snapshots" - returns number of unreleased snapshots of the + // database. + static const std::string kNumSnapshots; + + // "rocksdb.oldest-snapshot-time" - returns number representing unix + // timestamp of oldest unreleased snapshot. + static const std::string kOldestSnapshotTime; + + // "rocksdb.num-live-versions" - returns number of live versions. `Version` + // is an internal data structure. See version_set.h for details. More + // live versions often mean more SST files are held from being deleted, + // by iterators or unfinished compactions. + static const std::string kNumLiveVersions; + + // "rocksdb.current-super-version-number" - returns number of current LSM + // version. It is a uint64_t integer number, incremented after there is + // any change to the LSM tree. The number is not preserved after restarting + // the DB. After DB restart, it will start from 0 again. + static const std::string kCurrentSuperVersionNumber; + + // "rocksdb.estimate-live-data-size" - returns an estimate of the amount of + // live data in bytes. + static const std::string kEstimateLiveDataSize; + + // "rocksdb.min-log-number-to-keep" - return the minimum log number of the + // log files that should be kept. + static const std::string kMinLogNumberToKeep; + + // "rocksdb.total-sst-files-size" - returns total size (bytes) of all SST + // files. + // WARNING: may slow down online queries if there are too many files. + static const std::string kTotalSstFilesSize; + + // "rocksdb.base-level" - returns number of level to which L0 data will be + // compacted. + static const std::string kBaseLevel; + + // "rocksdb.estimate-pending-compaction-bytes" - returns estimated total + // number of bytes compaction needs to rewrite to get all levels down + // to under target size. Not valid for other compactions than level- + // based. + static const std::string kEstimatePendingCompactionBytes; + + // "rocksdb.aggregated-table-properties" - returns a string representation + // of the aggregated table properties of the target column family. + static const std::string kAggregatedTableProperties; + + // "rocksdb.aggregated-table-properties-at-level", same as the previous + // one but only returns the aggregated table properties of the + // specified level "N" at the target column family. + static const std::string kAggregatedTablePropertiesAtLevel; + + // "rocksdb.actual-delayed-write-rate" - returns the current actual delayed + // write rate. 0 means no delay. + static const std::string kActualDelayedWriteRate; + + // "rocksdb.is-write-stopped" - Return 1 if write has been stopped. + static const std::string kIsWriteStopped; + + // "rocksdb.estimate-oldest-key-time" - returns an estimation of + // oldest key timestamp in the DB. Currently only available for + // FIFO compaction with + // compaction_options_fifo.allow_compaction = false. + static const std::string kEstimateOldestKeyTime; + }; +#endif /* ROCKSDB_LITE */ + + // DB implementations can export properties about their state via this method. + // If "property" is a valid property understood by this DB implementation (see + // Properties struct above for valid options), fills "*value" with its current + // value and returns true. Otherwise, returns false. virtual bool GetProperty(ColumnFamilyHandle* column_family, const Slice& property, std::string* value) = 0; virtual bool GetProperty(const Slice& property, std::string* value) { return GetProperty(DefaultColumnFamily(), property, value); } + virtual bool GetMapProperty(ColumnFamilyHandle* column_family, + const Slice& property, + std::map* value) = 0; + virtual bool GetMapProperty(const Slice& property, + std::map* value) { + return GetMapProperty(DefaultColumnFamily(), property, value); + } // Similar to GetProperty(), but only works for a subset of properties whose - // return value is an integer. Return the value by integer. + // return value is an integer. Return the value by integer. Supported + // properties: + // "rocksdb.num-immutable-mem-table" + // "rocksdb.mem-table-flush-pending" + // "rocksdb.compaction-pending" + // "rocksdb.background-errors" + // "rocksdb.cur-size-active-mem-table" + // "rocksdb.cur-size-all-mem-tables" + // "rocksdb.size-all-mem-tables" + // "rocksdb.num-entries-active-mem-table" + // "rocksdb.num-entries-imm-mem-tables" + // "rocksdb.num-deletes-active-mem-table" + // "rocksdb.num-deletes-imm-mem-tables" + // "rocksdb.estimate-num-keys" + // "rocksdb.estimate-table-readers-mem" + // "rocksdb.is-file-deletions-enabled" + // "rocksdb.num-snapshots" + // "rocksdb.oldest-snapshot-time" + // "rocksdb.num-live-versions" + // "rocksdb.current-super-version-number" + // "rocksdb.estimate-live-data-size" + // "rocksdb.min-log-number-to-keep" + // "rocksdb.total-sst-files-size" + // "rocksdb.base-level" + // "rocksdb.estimate-pending-compaction-bytes" + // "rocksdb.num-running-compactions" + // "rocksdb.num-running-flushes" + // "rocksdb.actual-delayed-write-rate" + // "rocksdb.is-write-stopped" + // "rocksdb.estimate-oldest-key-time" virtual bool GetIntProperty(ColumnFamilyHandle* column_family, const Slice& property, uint64_t* value) = 0; virtual bool GetIntProperty(const Slice& property, uint64_t* value) { return GetIntProperty(DefaultColumnFamily(), property, value); } + // Reset internal stats for DB and all column families. + // Note this doesn't reset options.statistics as it is not owned by + // DB. + virtual Status ResetStats() { + return Status::NotSupported("Not implemented"); + } + + // Same as GetIntProperty(), but this one returns the aggregated int + // property from all column families. + virtual bool GetAggregatedIntProperty(const Slice& property, + uint64_t* value) = 0; + + // Flags for DB::GetSizeApproximation that specify whether memtable + // stats should be included, or file stats approximation or both + enum SizeApproximationFlags : uint8_t { + NONE = 0, + INCLUDE_MEMTABLES = 1, + INCLUDE_FILES = 1 << 1 + }; + // For each i in [0,n-1], store in "sizes[i]", the approximate // file system space used by keys in "[range[i].start .. range[i].limit)". // @@ -322,12 +672,52 @@ class DB { // if the user data compresses by a factor of ten, the returned // sizes will be one-tenth the size of the corresponding user data size. // - // The results may not include the sizes of recently written data. + // If include_flags defines whether the returned size should include + // the recently written data in the mem-tables (if + // the mem-table type supports it), data serialized to disk, or both. + // include_flags should be of type DB::SizeApproximationFlags virtual void GetApproximateSizes(ColumnFamilyHandle* column_family, - const Range* range, int n, - uint64_t* sizes) = 0; - virtual void GetApproximateSizes(const Range* range, int n, uint64_t* sizes) { - GetApproximateSizes(DefaultColumnFamily(), range, n, sizes); + const Range* range, int n, uint64_t* sizes, + uint8_t include_flags + = INCLUDE_FILES) = 0; + virtual void GetApproximateSizes(const Range* range, int n, uint64_t* sizes, + uint8_t include_flags + = INCLUDE_FILES) { + GetApproximateSizes(DefaultColumnFamily(), range, n, sizes, + include_flags); + } + + // The method is similar to GetApproximateSizes, except it + // returns approximate number of records in memtables. + virtual void GetApproximateMemTableStats(ColumnFamilyHandle* column_family, + const Range& range, + uint64_t* const count, + uint64_t* const size) = 0; + virtual void GetApproximateMemTableStats(const Range& range, + uint64_t* const count, + uint64_t* const size) { + GetApproximateMemTableStats(DefaultColumnFamily(), range, count, size); + } + + // Deprecated versions of GetApproximateSizes + ROCKSDB_DEPRECATED_FUNC virtual void GetApproximateSizes( + const Range* range, int n, uint64_t* sizes, + bool include_memtable) { + uint8_t include_flags = SizeApproximationFlags::INCLUDE_FILES; + if (include_memtable) { + include_flags |= SizeApproximationFlags::INCLUDE_MEMTABLES; + } + GetApproximateSizes(DefaultColumnFamily(), range, n, sizes, include_flags); + } + ROCKSDB_DEPRECATED_FUNC virtual void GetApproximateSizes( + ColumnFamilyHandle* column_family, + const Range* range, int n, uint64_t* sizes, + bool include_memtable) { + uint8_t include_flags = SizeApproximationFlags::INCLUDE_FILES; + if (include_memtable) { + include_flags |= SizeApproximationFlags::INCLUDE_MEMTABLES; + } + GetApproximateSizes(column_family, range, n, sizes, include_flags); } // Compact the underlying storage for the key range [*begin,*end]. @@ -340,26 +730,94 @@ class DB { // begin==nullptr is treated as a key before all keys in the database. // end==nullptr is treated as a key after all keys in the database. // Therefore the following call will compact the entire database: - // db->CompactRange(nullptr, nullptr); + // db->CompactRange(options, nullptr, nullptr); // Note that after the entire database is compacted, all data are pushed - // down to the last level containing any data. If the total data size - // after compaction is reduced, that level might not be appropriate for - // hosting all the files. In this case, client could set reduce_level - // to true, to move the files back to the minimum level capable of holding - // the data set or a given level (specified by non-negative target_level). - // Compaction outputs should be placed in options.db_paths[target_path_id]. - // Behavior is undefined if target_path_id is out of range. - virtual Status CompactRange(ColumnFamilyHandle* column_family, - const Slice* begin, const Slice* end, - bool reduce_level = false, int target_level = -1, - uint32_t target_path_id = 0) = 0; - virtual Status CompactRange(const Slice* begin, const Slice* end, - bool reduce_level = false, int target_level = -1, - uint32_t target_path_id = 0) { - return CompactRange(DefaultColumnFamily(), begin, end, reduce_level, - target_level, target_path_id); + // down to the last level containing any data. If the total data size after + // compaction is reduced, that level might not be appropriate for hosting all + // the files. In this case, client could set options.change_level to true, to + // move the files back to the minimum level capable of holding the data set + // or a given level (specified by non-negative options.target_level). + virtual Status CompactRange(const CompactRangeOptions& options, + ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) = 0; + virtual Status CompactRange(const CompactRangeOptions& options, + const Slice* begin, const Slice* end) { + return CompactRange(options, DefaultColumnFamily(), begin, end); } + ROCKSDB_DEPRECATED_FUNC virtual Status CompactRange( + ColumnFamilyHandle* column_family, const Slice* begin, const Slice* end, + bool change_level = false, int target_level = -1, + uint32_t target_path_id = 0) { + CompactRangeOptions options; + options.change_level = change_level; + options.target_level = target_level; + options.target_path_id = target_path_id; + return CompactRange(options, column_family, begin, end); + } + + ROCKSDB_DEPRECATED_FUNC virtual Status CompactRange( + const Slice* begin, const Slice* end, bool change_level = false, + int target_level = -1, uint32_t target_path_id = 0) { + CompactRangeOptions options; + options.change_level = change_level; + options.target_level = target_level; + options.target_path_id = target_path_id; + return CompactRange(options, DefaultColumnFamily(), begin, end); + } + + virtual Status SetOptions( + ColumnFamilyHandle* /*column_family*/, + const std::unordered_map& /*new_options*/) { + return Status::NotSupported("Not implemented"); + } + virtual Status SetOptions( + const std::unordered_map& new_options) { + return SetOptions(DefaultColumnFamily(), new_options); + } + + virtual Status SetDBOptions( + const std::unordered_map& new_options) = 0; + + // CompactFiles() inputs a list of files specified by file numbers and + // compacts them to the specified level. Note that the behavior is different + // from CompactRange() in that CompactFiles() performs the compaction job + // using the CURRENT thread. + // + // @see GetDataBaseMetaData + // @see GetColumnFamilyMetaData + virtual Status CompactFiles( + const CompactionOptions& compact_options, + ColumnFamilyHandle* column_family, + const std::vector& input_file_names, + const int output_level, const int output_path_id = -1) = 0; + + virtual Status CompactFiles( + const CompactionOptions& compact_options, + const std::vector& input_file_names, + const int output_level, const int output_path_id = -1) { + return CompactFiles(compact_options, DefaultColumnFamily(), + input_file_names, output_level, output_path_id); + } + + // This function will wait until all currently running background processes + // finish. After it returns, no background process will be run until + // UnblockBackgroundWork is called + virtual Status PauseBackgroundWork() = 0; + virtual Status ContinueBackgroundWork() = 0; + + // This function will enable automatic compactions for the given column + // families if they were previously disabled. The function will first set the + // disable_auto_compactions option for each column family to 'false', after + // which it will schedule a flush/compaction. + // + // NOTE: Setting disable_auto_compactions to 'false' through SetOptions() API + // does NOT schedule a flush/compaction afterwards, and only changes the + // parameter itself within the column family option. + // + virtual Status EnableAutoCompaction( + const std::vector& column_family_handles) = 0; + // Number of levels used for this DB. virtual int NumberLevels(ColumnFamilyHandle* column_family) = 0; virtual int NumberLevels() { return NumberLevels(DefaultColumnFamily()); } @@ -384,13 +842,17 @@ class DB { // Get Env object from the DB virtual Env* GetEnv() const = 0; - // Get DB Options that we use - virtual const Options& GetOptions(ColumnFamilyHandle* column_family) - const = 0; - virtual const Options& GetOptions() const { + // Get DB Options that we use. During the process of opening the + // column family, the options provided when calling DB::Open() or + // DB::CreateColumnFamily() will have been "sanitized" and transformed + // in an implementation-defined manner. + virtual Options GetOptions(ColumnFamilyHandle* column_family) const = 0; + virtual Options GetOptions() const { return GetOptions(DefaultColumnFamily()); } + virtual DBOptions GetDBOptions() const = 0; + // Flush all mem-table data. virtual Status Flush(const FlushOptions& options, ColumnFamilyHandle* column_family) = 0; @@ -398,6 +860,17 @@ class DB { return Flush(options, DefaultColumnFamily()); } + // Flush the WAL memory buffer to the file. If sync is true, it calls SyncWAL + // afterwards. + virtual Status FlushWAL(bool sync) { + return Status::NotSupported("FlushWAL not implemented"); + } + // Sync the wal. Note that Write() followed by SyncWAL() is not exactly the + // same as Write() with sync=true: in the latter case the changes won't be + // visible until the sync is done. + // Currently only works if allow_mmap_writes = false in Options. + virtual Status SyncWAL() = 0; + // The sequence number of the most recent transaction. virtual SequenceNumber GetLatestSequenceNumber() const = 0; @@ -421,8 +894,6 @@ class DB { // GetLiveFiles followed by GetSortedWalFiles can generate a lossless backup - // THIS METHOD IS DEPRECATED. Use the GetLiveFilesMetaData to get more - // detailed information on the live files. // Retrieve the list of all files in the database. The files are // relative to the dbname and are not absolute paths. The valid size of the // manifest file is returned in manifest_file_size. The manifest file is an @@ -457,6 +928,8 @@ class DB { const TransactionLogIterator::ReadOptions& read_options = TransactionLogIterator::ReadOptions()) = 0; +// Windows API macro interference +#undef DeleteFile // Delete the file name from the db directory and update the internal state to // reflect that. Supports deletion of sst and log files only. 'name' must be // path relative to the db directory. eg. 000001.sst, /archive/000003.log @@ -464,14 +937,162 @@ class DB { // Returns a list of all table files with their level, start key // and end key - virtual void GetLiveFilesMetaData(std::vector* metadata) {} + virtual void GetLiveFilesMetaData( + std::vector* /*metadata*/) {} + + // Obtains the meta data of the specified column family of the DB. + // Status::NotFound() will be returned if the current DB does not have + // any column family match the specified name. + // + // If cf_name is not specified, then the metadata of the default + // column family will be returned. + virtual void GetColumnFamilyMetaData(ColumnFamilyHandle* /*column_family*/, + ColumnFamilyMetaData* /*metadata*/) {} + + // Get the metadata of the default column family. + void GetColumnFamilyMetaData( + ColumnFamilyMetaData* metadata) { + GetColumnFamilyMetaData(DefaultColumnFamily(), metadata); + } + + // IngestExternalFile() will load a list of external SST files (1) into the DB + // Two primary modes are supported: + // - Duplicate keys in the new files will overwrite exiting keys (default) + // - Duplicate keys will be skipped (set ingest_behind=true) + // In the first mode we will try to find the lowest possible level that + // the file can fit in, and ingest the file into this level (2). A file that + // have a key range that overlap with the memtable key range will require us + // to Flush the memtable first before ingesting the file. + // In the second mode we will always ingest in the bottom mode level (see + // docs to IngestExternalFileOptions::ingest_behind). + // + // (1) External SST files can be created using SstFileWriter + // (2) We will try to ingest the files to the lowest possible level + // even if the file compression doesn't match the level compression + // (3) If IngestExternalFileOptions->ingest_behind is set to true, + // we always ingest at the bottommost level, which should be reserved + // for this purpose (see DBOPtions::allow_ingest_behind flag). + virtual Status IngestExternalFile( + ColumnFamilyHandle* column_family, + const std::vector& external_files, + const IngestExternalFileOptions& options) = 0; + + virtual Status IngestExternalFile( + const std::vector& external_files, + const IngestExternalFileOptions& options) { + return IngestExternalFile(DefaultColumnFamily(), external_files, options); + } + + virtual Status VerifyChecksum() = 0; + + // AddFile() is deprecated, please use IngestExternalFile() + ROCKSDB_DEPRECATED_FUNC virtual Status AddFile( + ColumnFamilyHandle* column_family, + const std::vector& file_path_list, bool move_file = false, + bool skip_snapshot_check = false) { + IngestExternalFileOptions ifo; + ifo.move_files = move_file; + ifo.snapshot_consistency = !skip_snapshot_check; + ifo.allow_global_seqno = false; + ifo.allow_blocking_flush = false; + return IngestExternalFile(column_family, file_path_list, ifo); + } + + ROCKSDB_DEPRECATED_FUNC virtual Status AddFile( + const std::vector& file_path_list, bool move_file = false, + bool skip_snapshot_check = false) { + IngestExternalFileOptions ifo; + ifo.move_files = move_file; + ifo.snapshot_consistency = !skip_snapshot_check; + ifo.allow_global_seqno = false; + ifo.allow_blocking_flush = false; + return IngestExternalFile(DefaultColumnFamily(), file_path_list, ifo); + } + + // AddFile() is deprecated, please use IngestExternalFile() + ROCKSDB_DEPRECATED_FUNC virtual Status AddFile( + ColumnFamilyHandle* column_family, const std::string& file_path, + bool move_file = false, bool skip_snapshot_check = false) { + IngestExternalFileOptions ifo; + ifo.move_files = move_file; + ifo.snapshot_consistency = !skip_snapshot_check; + ifo.allow_global_seqno = false; + ifo.allow_blocking_flush = false; + return IngestExternalFile(column_family, {file_path}, ifo); + } + + ROCKSDB_DEPRECATED_FUNC virtual Status AddFile( + const std::string& file_path, bool move_file = false, + bool skip_snapshot_check = false) { + IngestExternalFileOptions ifo; + ifo.move_files = move_file; + ifo.snapshot_consistency = !skip_snapshot_check; + ifo.allow_global_seqno = false; + ifo.allow_blocking_flush = false; + return IngestExternalFile(DefaultColumnFamily(), {file_path}, ifo); + } + + // Load table file with information "file_info" into "column_family" + ROCKSDB_DEPRECATED_FUNC virtual Status AddFile( + ColumnFamilyHandle* column_family, + const std::vector& file_info_list, + bool move_file = false, bool skip_snapshot_check = false) { + std::vector external_files; + for (const ExternalSstFileInfo& file_info : file_info_list) { + external_files.push_back(file_info.file_path); + } + IngestExternalFileOptions ifo; + ifo.move_files = move_file; + ifo.snapshot_consistency = !skip_snapshot_check; + ifo.allow_global_seqno = false; + ifo.allow_blocking_flush = false; + return IngestExternalFile(column_family, external_files, ifo); + } + + ROCKSDB_DEPRECATED_FUNC virtual Status AddFile( + const std::vector& file_info_list, + bool move_file = false, bool skip_snapshot_check = false) { + std::vector external_files; + for (const ExternalSstFileInfo& file_info : file_info_list) { + external_files.push_back(file_info.file_path); + } + IngestExternalFileOptions ifo; + ifo.move_files = move_file; + ifo.snapshot_consistency = !skip_snapshot_check; + ifo.allow_global_seqno = false; + ifo.allow_blocking_flush = false; + return IngestExternalFile(DefaultColumnFamily(), external_files, ifo); + } + + ROCKSDB_DEPRECATED_FUNC virtual Status AddFile( + ColumnFamilyHandle* column_family, const ExternalSstFileInfo* file_info, + bool move_file = false, bool skip_snapshot_check = false) { + IngestExternalFileOptions ifo; + ifo.move_files = move_file; + ifo.snapshot_consistency = !skip_snapshot_check; + ifo.allow_global_seqno = false; + ifo.allow_blocking_flush = false; + return IngestExternalFile(column_family, {file_info->file_path}, ifo); + } + + ROCKSDB_DEPRECATED_FUNC virtual Status AddFile( + const ExternalSstFileInfo* file_info, bool move_file = false, + bool skip_snapshot_check = false) { + IngestExternalFileOptions ifo; + ifo.move_files = move_file; + ifo.snapshot_consistency = !skip_snapshot_check; + ifo.allow_global_seqno = false; + ifo.allow_blocking_flush = false; + return IngestExternalFile(DefaultColumnFamily(), {file_info->file_path}, + ifo); + } #endif // ROCKSDB_LITE // Sets the globally unique ID created at database creation time by invoking // Env::GenerateUniqueId(), in identity. Returns Status::OK if identity could // be set properly - virtual Status GetDbIdentity(std::string& identity) = 0; + virtual Status GetDbIdentity(std::string& identity) const = 0; // Returns default column family handle virtual ColumnFamilyHandle* DefaultColumnFamily() const = 0; @@ -482,8 +1103,25 @@ class DB { virtual Status GetPropertiesOfAllTables(TablePropertiesCollection* props) { return GetPropertiesOfAllTables(DefaultColumnFamily(), props); } + virtual Status GetPropertiesOfTablesInRange( + ColumnFamilyHandle* column_family, const Range* range, std::size_t n, + TablePropertiesCollection* props) = 0; + + virtual Status SuggestCompactRange(ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) { + return Status::NotSupported("SuggestCompactRange() is not implemented."); + } + + virtual Status PromoteL0(ColumnFamilyHandle* column_family, + int target_level) { + return Status::NotSupported("PromoteL0() is not implemented."); + } + #endif // ROCKSDB_LITE + // Needed for StackableDB + virtual DB* GetRootDB() { return this; } + private: // No copying allowed DB(const DB&); @@ -499,7 +1137,24 @@ Status DestroyDB(const std::string& name, const Options& options); // resurrect as much of the contents of the database as possible. // Some data may be lost, so be careful when calling this function // on a database that contains important information. +// +// With this API, we will warn and skip data associated with column families not +// specified in column_families. +// +// @param column_families Descriptors for known column families +Status RepairDB(const std::string& dbname, const DBOptions& db_options, + const std::vector& column_families); + +// @param unknown_cf_opts Options for column families encountered during the +// repair that were not specified in column_families. +Status RepairDB(const std::string& dbname, const DBOptions& db_options, + const std::vector& column_families, + const ColumnFamilyOptions& unknown_cf_opts); + +// @param options These options will be used for the database and for ALL column +// families encountered during the repair Status RepairDB(const std::string& dbname, const Options& options); + #endif } // namespace rocksdb diff --git a/include/rocksdb/db_bench_tool.h b/include/rocksdb/db_bench_tool.h new file mode 100644 index 00000000000..047c4256ce6 --- /dev/null +++ b/include/rocksdb/db_bench_tool.h @@ -0,0 +1,9 @@ +// Copyright (c) 2013-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once + +namespace rocksdb { +int db_bench_tool(int argc, char** argv); +} // namespace rocksdb diff --git a/include/rocksdb/db_dump_tool.h b/include/rocksdb/db_dump_tool.h new file mode 100644 index 00000000000..cb9a265f5c8 --- /dev/null +++ b/include/rocksdb/db_dump_tool.h @@ -0,0 +1,45 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#ifndef ROCKSDB_LITE + +#include + +#include "rocksdb/db.h" + +namespace rocksdb { + +struct DumpOptions { + // Database that will be dumped + std::string db_path; + // File location that will contain dump output + std::string dump_location; + // Dont include db information header in the dump + bool anonymous = false; +}; + +class DbDumpTool { + public: + bool Run(const DumpOptions& dump_options, + rocksdb::Options options = rocksdb::Options()); +}; + +struct UndumpOptions { + // Database that we will load the dumped file into + std::string db_path; + // File location of the dumped file that will be loaded + std::string dump_location; + // Compact the db after loading the dumped file + bool compact_db = false; +}; + +class DbUndumpTool { + public: + bool Run(const UndumpOptions& undump_options, + rocksdb::Options options = rocksdb::Options()); +}; +} // namespace rocksdb +#endif // ROCKSDB_LITE diff --git a/include/rocksdb/env.h b/include/rocksdb/env.h index 70244bb3134..709d5036685 100644 --- a/include/rocksdb/env.h +++ b/include/rocksdb/env.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -17,12 +17,21 @@ #ifndef STORAGE_ROCKSDB_INCLUDE_ENV_H_ #define STORAGE_ROCKSDB_INCLUDE_ENV_H_ +#include #include -#include +#include +#include #include +#include #include -#include #include "rocksdb/status.h" +#include "rocksdb/thread_status.h" + +#ifdef _WIN32 +// Windows API macro interference +#undef DeleteFile +#undef GetCurrentTime +#endif namespace rocksdb { @@ -35,30 +44,40 @@ class WritableFile; class RandomRWFile; class Directory; struct DBOptions; +struct ImmutableDBOptions; class RateLimiter; +class ThreadStatusUpdater; +struct ThreadStatus; using std::unique_ptr; using std::shared_ptr; +const size_t kDefaultPageSize = 4 * 1024; // Options while opening a file to read/write struct EnvOptions { - // construct with default Options + // Construct with default Options EnvOptions(); - // construct from Options + // Construct from Options explicit EnvOptions(const DBOptions& options); - // If true, then allow caching of data in environment buffers - bool use_os_buffer = true; - // If true, then use mmap to read data bool use_mmap_reads = false; // If true, then use mmap to write data bool use_mmap_writes = true; + // If true, then use O_DIRECT for reading data + bool use_direct_reads = false; + + // If true, then use O_DIRECT for writing data + bool use_direct_writes = false; + + // If false, fallocate() calls are bypassed + bool allow_fallocate = true; + // If true, set the FD_CLOEXEC on open fd. bool set_fd_cloexec = true; @@ -76,13 +95,31 @@ struct EnvOptions { // WAL writes bool fallocate_with_keep_size = true; + // See DBOptions doc + size_t compaction_readahead_size; + + // See DBOptions doc + size_t random_access_max_buffer_size; + + // See DBOptions doc + size_t writable_file_max_buffer_size = 1024 * 1024; + // If not nullptr, write rate limiting is enabled for flush and compaction RateLimiter* rate_limiter = nullptr; }; class Env { public: - Env() { } + struct FileAttributes { + // File name + std::string name; + + // Size of file in bytes + uint64_t size_bytes; + }; + + Env() : thread_status_updater_(nullptr) {} + virtual ~Env(); // Return a default environment suitable for the current operating @@ -126,14 +163,35 @@ class Env { unique_ptr* result, const EnvOptions& options) = 0; - // Create an object that both reads and writes to a file on - // specified offsets (random access). If file already exists, - // does not overwrite it. On success, stores a pointer to the - // new file in *result and returns OK. On failure stores nullptr - // in *result and returns non-OK. + // Create an object that writes to a new file with the specified + // name. Deletes any existing file with the same name and creates a + // new file. On success, stores a pointer to the new file in + // *result and returns OK. On failure stores nullptr in *result and + // returns non-OK. + // + // The returned file will only be accessed by one thread at a time. + virtual Status ReopenWritableFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) { + return Status::NotSupported(); + } + + // Reuse an existing file by renaming it and opening it as writable. + virtual Status ReuseWritableFile(const std::string& fname, + const std::string& old_fname, + unique_ptr* result, + const EnvOptions& options); + + // Open `fname` for random read and write, if file doesn't exist the file + // will be created. On success, stores a pointer to the new file in + // *result and returns OK. On failure returns non-OK. + // + // The returned file will only be accessed by one thread at a time. virtual Status NewRandomRWFile(const std::string& fname, unique_ptr* result, - const EnvOptions& options) = 0; + const EnvOptions& options) { + return Status::NotSupported("RandomRWFile is not implemented in this Env"); + } // Create an object that represents a directory. Will fail if directory // doesn't exist. If the directory exists, it will open the directory @@ -145,15 +203,36 @@ class Env { virtual Status NewDirectory(const std::string& name, unique_ptr* result) = 0; - // Returns true iff the named file exists. - virtual bool FileExists(const std::string& fname) = 0; + // Returns OK if the named file exists. + // NotFound if the named file does not exist, + // the calling process does not have permission to determine + // whether this file exists, or if the path is invalid. + // IOError if an IO Error was encountered + virtual Status FileExists(const std::string& fname) = 0; // Store in *result the names of the children of the specified directory. // The names are relative to "dir". // Original contents of *results are dropped. + // Returns OK if "dir" exists and "*result" contains its children. + // NotFound if "dir" does not exist, the calling process does not have + // permission to access "dir", or if "dir" is invalid. + // IOError if an IO Error was encountered virtual Status GetChildren(const std::string& dir, std::vector* result) = 0; + // Store in *result the attributes of the children of the specified directory. + // In case the implementation lists the directory prior to iterating the files + // and files are concurrently deleted, the deleted files will be omitted from + // result. + // The name attributes are relative to "dir". + // Original contents of *results are dropped. + // Returns OK if "dir" exists and "*result" contains its children. + // NotFound if "dir" does not exist, the calling process does not have + // permission to access "dir", or if "dir" is invalid. + // IOError if an IO Error was encountered + virtual Status GetChildrenFileAttributes(const std::string& dir, + std::vector* result); + // Delete the named file. virtual Status DeleteFile(const std::string& fname) = 0; @@ -177,6 +256,11 @@ class Env { virtual Status RenameFile(const std::string& src, const std::string& target) = 0; + // Hard Link file src to target. + virtual Status LinkFile(const std::string& src, const std::string& target) { + return Status::NotSupported("LinkFile is not supported for this Env"); + } + // Lock the specified file. Used to prevent concurrent access to // the same db by multiple processes. On failure, stores nullptr in // *lock and returns non-OK. @@ -199,7 +283,7 @@ class Env { virtual Status UnlockFile(FileLock* lock) = 0; // Priority for scheduling job in thread pool - enum Priority { LOW, HIGH, TOTAL }; + enum Priority { BOTTOM, LOW, HIGH, TOTAL }; // Priority for requesting bytes in rate limiter scheduler enum IOPriority { @@ -216,10 +300,15 @@ class Env { // added to the same Env may run concurrently in different threads. // I.e., the caller may not assume that background work items are // serialized. - virtual void Schedule( - void (*function)(void* arg), - void* arg, - Priority pri = LOW) = 0; + // When the UnSchedule function is called, the unschedFunction + // registered at the time of Schedule is invoked with arg as a parameter. + virtual void Schedule(void (*function)(void* arg), void* arg, + Priority pri = LOW, void* tag = nullptr, + void (*unschedFunction)(void* arg) = 0) = 0; + + // Arrange to remove jobs for given arg from the queue_ if they are not + // already scheduled. Caller is expected to have exclusive lock on arg. + virtual int UnSchedule(void* arg, Priority pri) { return 0; } // Start a new thread, invoking "function(arg)" within the new thread. // When "function(arg)" returns, the thread will be destroyed. @@ -228,7 +317,7 @@ class Env { // Wait for all threads started by StartThread to terminate. virtual void WaitForJoin() {} - // Get thread pool queue length for specific thrad pool. + // Get thread pool queue length for specific thread pool. virtual unsigned int GetThreadPoolQueueLen(Priority pri = LOW) const { return 0; } @@ -243,13 +332,16 @@ class Env { virtual Status NewLogger(const std::string& fname, shared_ptr* result) = 0; - // Returns the number of micro-seconds since some fixed point in time. Only - // useful for computing deltas of time. + // Returns the number of micro-seconds since some fixed point in time. + // It is often used as system time such as in GenericRateLimiter + // and other places so a port needs to return system time in order to work. virtual uint64_t NowMicros() = 0; // Returns the number of nano-seconds since some fixed point in time. Only // useful for computing deltas of time in one run. - // Default implementation simply relies on NowMicros + // Default implementation simply relies on NowMicros. + // In platform-specific implementations, NowNanos() should return time points + // that are MONOTONIC. virtual uint64_t NowNanos() { return NowMicros() * 1000; } @@ -261,6 +353,7 @@ class Env { virtual Status GetHostName(char* name, uint64_t len) = 0; // Get the number of seconds since the Epoch, 1970-01-01 00:00:00 (UTC). + // Only overwrites *unix_time on success. virtual Status GetCurrentTime(int64_t* unix_time) = 0; // Get full directory name for this db. @@ -271,6 +364,12 @@ class Env { // for this environment. 'LOW' is the default pool. // default number: 1 virtual void SetBackgroundThreads(int number, Priority pri = LOW) = 0; + virtual int GetBackgroundThreads(Priority pri = LOW) = 0; + + // Enlarge number of background worker threads of a specific thread pool + // for this environment if it is smaller than specified. 'LOW' is the default + // pool. + virtual void IncBackgroundThreadsIfNeeded(int number, Priority pri) = 0; // Lower IO priority for threads from the specified pool. virtual void LowerThreadPoolIOPriority(Priority pool = LOW) {} @@ -281,15 +380,60 @@ class Env { // Generates a unique id that can be used to identify a db virtual std::string GenerateUniqueId(); + // OptimizeForLogWrite will create a new EnvOptions object that is a copy of + // the EnvOptions in the parameters, but is optimized for reading log files. + virtual EnvOptions OptimizeForLogRead(const EnvOptions& env_options) const; + + // OptimizeForManifestRead will create a new EnvOptions object that is a copy + // of the EnvOptions in the parameters, but is optimized for reading manifest + // files. + virtual EnvOptions OptimizeForManifestRead( + const EnvOptions& env_options) const; + // OptimizeForLogWrite will create a new EnvOptions object that is a copy of // the EnvOptions in the parameters, but is optimized for writing log files. // Default implementation returns the copy of the same object. - virtual EnvOptions OptimizeForLogWrite(const EnvOptions& env_options) const; + virtual EnvOptions OptimizeForLogWrite(const EnvOptions& env_options, + const DBOptions& db_options) const; // OptimizeForManifestWrite will create a new EnvOptions object that is a copy // of the EnvOptions in the parameters, but is optimized for writing manifest // files. Default implementation returns the copy of the same object. - virtual EnvOptions OptimizeForManifestWrite(const EnvOptions& env_options) - const; + virtual EnvOptions OptimizeForManifestWrite( + const EnvOptions& env_options) const; + + // OptimizeForCompactionTableWrite will create a new EnvOptions object that is + // a copy of the EnvOptions in the parameters, but is optimized for writing + // table files. + virtual EnvOptions OptimizeForCompactionTableWrite( + const EnvOptions& env_options, + const ImmutableDBOptions& db_options) const; + + // OptimizeForCompactionTableWrite will create a new EnvOptions object that + // is a copy of the EnvOptions in the parameters, but is optimized for reading + // table files. + virtual EnvOptions OptimizeForCompactionTableRead( + const EnvOptions& env_options, + const ImmutableDBOptions& db_options) const; + + // Returns the status of all threads that belong to the current Env. + virtual Status GetThreadList(std::vector* thread_list) { + return Status::NotSupported("Not supported."); + } + + // Returns the pointer to ThreadStatusUpdater. This function will be + // used in RocksDB internally to update thread status and supports + // GetThreadList(). + virtual ThreadStatusUpdater* GetThreadStatusUpdater() const { + return thread_status_updater_; + } + + // Returns the ID of the current thread. + virtual uint64_t GetThreadID() const; + + protected: + // The pointer to an internal structure that will update the + // status of each thread. + ThreadStatusUpdater* thread_status_updater_; private: // No copying allowed @@ -297,6 +441,11 @@ class Env { void operator=(const Env&); }; +// The factory function to construct a ThreadStatusUpdater. Any Env +// that supports GetThreadList() feature should call this function in its +// constructor to initialize thread_status_updater_. +ThreadStatusUpdater* CreateThreadStatusUpdater(); + // A file abstraction for reading sequentially through a file class SequentialFile { public: @@ -322,17 +471,33 @@ class SequentialFile { // REQUIRES: External synchronization virtual Status Skip(uint64_t n) = 0; + // Indicates the upper layers if the current SequentialFile implementation + // uses direct IO. + virtual bool use_direct_io() const { return false; } + + // Use the returned alignment value to allocate + // aligned buffer for Direct I/O + virtual size_t GetRequiredBufferAlignment() const { return kDefaultPageSize; } + // Remove any kind of caching of data from the offset to offset+length // of this file. If the length is 0, then it refers to the end of file. // If the system is not caching the file contents, then this is a noop. virtual Status InvalidateCache(size_t offset, size_t length) { return Status::NotSupported("InvalidateCache not supported."); } + + // Positioned Read for direct I/O + // If Direct I/O enabled, offset, n, and scratch should be properly aligned + virtual Status PositionedRead(uint64_t offset, size_t n, Slice* result, + char* scratch) { + return Status::NotSupported(); + } }; // A file abstraction for randomly reading the contents of a file. class RandomAccessFile { public: + RandomAccessFile() { } virtual ~RandomAccessFile(); @@ -345,9 +510,15 @@ class RandomAccessFile { // status. // // Safe for concurrent use by multiple threads. + // If Direct I/O enabled, offset, n, and scratch should be aligned properly. virtual Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const = 0; + // Readahead the file starting from offset by n bytes for caching. + virtual Status Prefetch(uint64_t offset, size_t n) { + return Status::OK(); + } + // Tries to get an unique ID for this file that will be the same each time // the file is opened (and will stay the same while the file is open). // Furthermore, it tries to make this ID at most "max_size" bytes. If such an @@ -356,7 +527,7 @@ class RandomAccessFile { // may not have been modified. // // This function guarantees, for IDs from a given environment, two unique ids - // cannot be made equal to eachother by adding arbitrary bytes to one of + // cannot be made equal to each other by adding arbitrary bytes to one of // them. That is, no unique ID is the prefix of another. // // This function guarantees that the returned ID will not be interpretable as @@ -368,11 +539,18 @@ class RandomAccessFile { // compatibility. }; - enum AccessPattern { NORMAL, RANDOM, SEQUENTIAL, WILLNEED, DONTNEED }; virtual void Hint(AccessPattern pattern) {} + // Indicates the upper layers if the current RandomAccessFile implementation + // uses direct IO. + virtual bool use_direct_io() const { return false; } + + // Use the returned alignment value to allocate + // aligned buffer for Direct I/O + virtual size_t GetRequiredBufferAlignment() const { return kDefaultPageSize; } + // Remove any kind of caching of data from the offset to offset+length // of this file. If the length is 0, then it refers to the end of file. // If the system is not caching the file contents, then this is a noop. @@ -393,7 +571,42 @@ class WritableFile { } virtual ~WritableFile(); + // Append data to the end of the file + // Note: A WriteabelFile object must support either Append or + // PositionedAppend, so the users cannot mix the two. virtual Status Append(const Slice& data) = 0; + + // PositionedAppend data to the specified offset. The new EOF after append + // must be larger than the previous EOF. This is to be used when writes are + // not backed by OS buffers and hence has to always start from the start of + // the sector. The implementation thus needs to also rewrite the last + // partial sector. + // Note: PositionAppend does not guarantee moving the file offset after the + // write. A WritableFile object must support either Append or + // PositionedAppend, so the users cannot mix the two. + // + // PositionedAppend() can only happen on the page/sector boundaries. For that + // reason, if the last write was an incomplete sector we still need to rewind + // back to the nearest sector/page and rewrite the portion of it with whatever + // we need to add. We need to keep where we stop writing. + // + // PositionedAppend() can only write whole sectors. For that reason we have to + // pad with zeros for the last write and trim the file when closing according + // to the position we keep in the previous step. + // + // PositionedAppend() requires aligned buffer to be passed in. The alignment + // required is queried via GetRequiredBufferAlignment() + virtual Status PositionedAppend(const Slice& /* data */, uint64_t /* offset */) { + return Status::NotSupported(); + } + + // Truncate is necessary to trim the file to the correct size + // before closing. It is not always possible to keep track of the file + // size due to whole pages writes. The behavior is undefined if called + // with other writes to follow. + virtual Status Truncate(uint64_t size) { + return Status::OK(); + } virtual Status Close() = 0; virtual Status Flush() = 0; virtual Status Sync() = 0; // sync data @@ -408,6 +621,19 @@ class WritableFile { return Sync(); } + // true if Sync() and Fsync() are safe to call concurrently with Append() + // and Flush(). + virtual bool IsSyncThreadSafe() const { + return false; + } + + // Indicates the upper layers if the current WritableFile implementation + // uses direct IO. + virtual bool use_direct_io() const { return false; } + + // Use the returned alignment value to allocate + // aligned buffer for Direct I/O + virtual size_t GetRequiredBufferAlignment() const { return kDefaultPageSize; } /* * Change the priority in rate limiter if rate limiting is enabled. * If rate limiting is not enabled, this call has no effect. @@ -416,6 +642,8 @@ class WritableFile { io_priority_ = pri; } + virtual Env::IOPriority GetIOPriority() { return io_priority_; } + /* * Get the size of valid data in the file. */ @@ -429,7 +657,7 @@ class WritableFile { * underlying storage of a file (generally via fallocate) if the Env * instance supports it. */ - void SetPreallocationBlockSize(size_t size) { + virtual void SetPreallocationBlockSize(size_t size) { preallocation_block_size_ = size; } @@ -452,18 +680,25 @@ class WritableFile { return Status::NotSupported("InvalidateCache not supported."); } - protected: + // Sync a file range with disk. + // offset is the starting byte of the file range to be synchronized. + // nbytes specifies the length of the range to be synchronized. + // This asks the OS to initiate flushing the cached data to disk, + // without waiting for completion. + // Default implementation does nothing. + virtual Status RangeSync(uint64_t offset, uint64_t nbytes) { return Status::OK(); } + // PrepareWrite performs any necessary preparation for a write // before the write actually occurs. This allows for pre-allocation // of space on devices where it can result in less file // fragmentation and/or less waste from over-zealous filesystem // pre-allocation. - void PrepareWrite(size_t offset, size_t len) { + virtual void PrepareWrite(size_t offset, size_t len) { if (preallocation_block_size_ == 0) { return; } // If this write would cross one or more preallocation blocks, - // determine what the last preallocation block necesessary to + // determine what the last preallocation block necessary to // cover this write would be and Allocate to that point. const auto block_size = preallocation_block_size_; size_t new_last_preallocated_block = @@ -477,22 +712,13 @@ class WritableFile { } } - /* - * Pre-allocate space for a file. - */ - virtual Status Allocate(off_t offset, off_t len) { + // Pre-allocates space for a file. + virtual Status Allocate(uint64_t offset, uint64_t len) { return Status::OK(); } - // Sync a file range with disk. - // offset is the starting byte of the file range to be synchronized. - // nbytes specifies the length of the range to be synchronized. - // This asks the OS to initiate flushing the cached data to disk, - // without waiting for completion. - // Default implementation does nothing. - virtual Status RangeSync(off_t offset, off_t nbytes) { - return Status::OK(); - } + protected: + size_t preallocation_block_size() { return preallocation_block_size_; } private: size_t last_preallocated_block_; @@ -502,6 +728,9 @@ class WritableFile { void operator=(const WritableFile&); protected: + friend class WritableFileWrapper; + friend class WritableFileMirror; + Env::IOPriority io_priority_; }; @@ -511,47 +740,35 @@ class RandomRWFile { RandomRWFile() {} virtual ~RandomRWFile() {} - // Write data from Slice data to file starting from offset - // Returns IOError on failure, but does not guarantee - // atomicity of a write. Returns OK status on success. - // - // Safe for concurrent use. + // Indicates if the class makes use of direct I/O + // If false you must pass aligned buffer to Write() + virtual bool use_direct_io() const { return false; } + + // Use the returned alignment value to allocate + // aligned buffer for Direct I/O + virtual size_t GetRequiredBufferAlignment() const { return kDefaultPageSize; } + + // Write bytes in `data` at offset `offset`, Returns Status::OK() on success. + // Pass aligned buffer when use_direct_io() returns true. virtual Status Write(uint64_t offset, const Slice& data) = 0; - // Read up to "n" bytes from the file starting at "offset". - // "scratch[0..n-1]" may be written by this routine. Sets "*result" - // to the data that was read (including if fewer than "n" bytes were - // successfully read). May set "*result" to point at data in - // "scratch[0..n-1]", so "scratch[0..n-1]" must be live when - // "*result" is used. If an error was encountered, returns a non-OK - // status. - // - // Safe for concurrent use by multiple threads. + + // Read up to `n` bytes starting from offset `offset` and store them in + // result, provided `scratch` size should be at least `n`. + // Returns Status::OK() on success. virtual Status Read(uint64_t offset, size_t n, Slice* result, char* scratch) const = 0; - virtual Status Close() = 0; // closes the file - virtual Status Sync() = 0; // sync data - /* - * Sync data and/or metadata as well. - * By default, sync only data. - * Override this method for environments where we need to sync - * metadata as well. - */ - virtual Status Fsync() { - return Sync(); - } + virtual Status Flush() = 0; - /* - * Pre-allocate space for a file. - */ - virtual Status Allocate(off_t offset, off_t len) { - return Status::OK(); - } + virtual Status Sync() = 0; + + virtual Status Fsync() { return Sync(); } + + virtual Status Close() = 0; - private: // No copying allowed - RandomRWFile(const RandomRWFile&); - void operator=(const RandomRWFile&); + RandomRWFile(const RandomRWFile&) = delete; + RandomRWFile& operator=(const RandomRWFile&) = delete; }; // Directory object represents collection of files and implements @@ -559,7 +776,7 @@ class RandomRWFile { class Directory { public: virtual ~Directory() {} - // Fsync directory + // Fsync directory. Can be called concurrently from multiple threads. virtual Status Fsync() = 0; }; @@ -569,17 +786,28 @@ enum InfoLogLevel : unsigned char { WARN_LEVEL, ERROR_LEVEL, FATAL_LEVEL, + HEADER_LEVEL, NUM_INFO_LOG_LEVELS, }; // An interface for writing log messages. class Logger { public: - enum { DO_NOT_SUPPORT_GET_LOG_FILE_SIZE = -1 }; + size_t kDoNotSupportGetLogFileSize = (std::numeric_limits::max)(); + explicit Logger(const InfoLogLevel log_level = InfoLogLevel::INFO_LEVEL) : log_level_(log_level) {} virtual ~Logger(); + // Write a header to the log file with the specified format + // It is recommended that you log all header information at the start of the + // application. But it is not enforced. + virtual void LogHeader(const char* format, va_list ap) { + // Default implementation does a simple INFO level log write. + // Please override as per the logger class requirement. + Logv(format, ap); + } + // Write an entry to the log file with the specified format. virtual void Logv(const char* format, va_list ap) = 0; @@ -587,30 +815,9 @@ class Logger { // and format. Any log with level under the internal log level // of *this (see @SetInfoLogLevel and @GetInfoLogLevel) will not be // printed. - void Logv(const InfoLogLevel log_level, const char* format, va_list ap) { - static const char* kInfoLogLevelNames[5] = {"DEBUG", "INFO", "WARN", - "ERROR", "FATAL"}; - if (log_level < log_level_) { - return; - } + virtual void Logv(const InfoLogLevel log_level, const char* format, va_list ap); - if (log_level == InfoLogLevel::INFO_LEVEL) { - // Doesn't print log level if it is INFO level. - // This is to avoid unexpected performance regression after we add - // the feature of log level. All the logs before we add the feature - // are INFO level. We don't want to add extra costs to those existing - // logging. - Logv(format, ap); - } else { - char new_format[500]; - snprintf(new_format, sizeof(new_format) - 1, "[%s] %s", - kInfoLogLevelNames[log_level], format); - Logv(new_format, ap); - } - } - virtual size_t GetLogFileSize() const { - return DO_NOT_SUPPORT_GET_LOG_FILE_SIZE; - } + virtual size_t GetLogFileSize() const { return kDoNotSupportGetLogFileSize; } // Flush to the OS buffers virtual void Flush() {} virtual InfoLogLevel GetInfoLogLevel() const { return log_level_; } @@ -643,6 +850,7 @@ extern void Log(const InfoLogLevel log_level, const shared_ptr& info_log, const char* format, ...); // a set of log functions with different log levels. +extern void Header(const shared_ptr& info_log, const char* format, ...); extern void Debug(const shared_ptr& info_log, const char* format, ...); extern void Info(const shared_ptr& info_log, const char* format, ...); extern void Warn(const shared_ptr& info_log, const char* format, ...); @@ -650,7 +858,7 @@ extern void Error(const shared_ptr& info_log, const char* format, ...); extern void Fatal(const shared_ptr& info_log, const char* format, ...); // Log the specified data to *info_log if info_log is non-nullptr. -// The default info log level is InfoLogLevel::ERROR. +// The default info log level is InfoLogLevel::INFO_LEVEL. extern void Log(const shared_ptr& info_log, const char* format, ...) # if defined(__GNUC__) || defined(__clang__) __attribute__((__format__ (__printf__, 2, 3))) @@ -662,7 +870,7 @@ extern void LogFlush(Logger *info_log); extern void Log(const InfoLogLevel log_level, Logger* info_log, const char* format, ...); -// The default info log level is InfoLogLevel::ERROR. +// The default info log level is InfoLogLevel::INFO_LEVEL. extern void Log(Logger* info_log, const char* format, ...) # if defined(__GNUC__) || defined(__clang__) __attribute__((__format__ (__printf__, 2, 3))) @@ -670,6 +878,7 @@ extern void Log(Logger* info_log, const char* format, ...) ; // a set of log functions with different log levels. +extern void Header(Logger* info_log, const char* format, ...); extern void Debug(Logger* info_log, const char* format, ...); extern void Info(Logger* info_log, const char* format, ...); extern void Warn(Logger* info_log, const char* format, ...); @@ -692,113 +901,237 @@ class EnvWrapper : public Env { public: // Initialize an EnvWrapper that delegates all calls to *t explicit EnvWrapper(Env* t) : target_(t) { } - virtual ~EnvWrapper(); + ~EnvWrapper() override; // Return the target to which this Env forwards all calls Env* target() const { return target_; } // The following text is boilerplate that forwards all methods to target() - Status NewSequentialFile(const std::string& f, - unique_ptr* r, - const EnvOptions& options) { + Status NewSequentialFile(const std::string& f, unique_ptr* r, + const EnvOptions& options) override { return target_->NewSequentialFile(f, r, options); } Status NewRandomAccessFile(const std::string& f, unique_ptr* r, - const EnvOptions& options) { + const EnvOptions& options) override { return target_->NewRandomAccessFile(f, r, options); } Status NewWritableFile(const std::string& f, unique_ptr* r, - const EnvOptions& options) { + const EnvOptions& options) override { return target_->NewWritableFile(f, r, options); } - Status NewRandomRWFile(const std::string& f, unique_ptr* r, - const EnvOptions& options) { - return target_->NewRandomRWFile(f, r, options); + Status ReopenWritableFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + return target_->ReopenWritableFile(fname, result, options); } - virtual Status NewDirectory(const std::string& name, - unique_ptr* result) { + Status ReuseWritableFile(const std::string& fname, + const std::string& old_fname, + unique_ptr* r, + const EnvOptions& options) override { + return target_->ReuseWritableFile(fname, old_fname, r, options); + } + Status NewRandomRWFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override { + return target_->NewRandomRWFile(fname, result, options); + } + Status NewDirectory(const std::string& name, + unique_ptr* result) override { return target_->NewDirectory(name, result); } - bool FileExists(const std::string& f) { return target_->FileExists(f); } - Status GetChildren(const std::string& dir, std::vector* r) { + Status FileExists(const std::string& f) override { + return target_->FileExists(f); + } + Status GetChildren(const std::string& dir, + std::vector* r) override { return target_->GetChildren(dir, r); } - Status DeleteFile(const std::string& f) { return target_->DeleteFile(f); } - Status CreateDir(const std::string& d) { return target_->CreateDir(d); } - Status CreateDirIfMissing(const std::string& d) { + Status GetChildrenFileAttributes( + const std::string& dir, std::vector* result) override { + return target_->GetChildrenFileAttributes(dir, result); + } + Status DeleteFile(const std::string& f) override { + return target_->DeleteFile(f); + } + Status CreateDir(const std::string& d) override { + return target_->CreateDir(d); + } + Status CreateDirIfMissing(const std::string& d) override { return target_->CreateDirIfMissing(d); } - Status DeleteDir(const std::string& d) { return target_->DeleteDir(d); } - Status GetFileSize(const std::string& f, uint64_t* s) { + Status DeleteDir(const std::string& d) override { + return target_->DeleteDir(d); + } + Status GetFileSize(const std::string& f, uint64_t* s) override { return target_->GetFileSize(f, s); } Status GetFileModificationTime(const std::string& fname, - uint64_t* file_mtime) { + uint64_t* file_mtime) override { return target_->GetFileModificationTime(fname, file_mtime); } - Status RenameFile(const std::string& s, const std::string& t) { + Status RenameFile(const std::string& s, const std::string& t) override { return target_->RenameFile(s, t); } - Status LockFile(const std::string& f, FileLock** l) { + + Status LinkFile(const std::string& s, const std::string& t) override { + return target_->LinkFile(s, t); + } + + Status LockFile(const std::string& f, FileLock** l) override { return target_->LockFile(f, l); } - Status UnlockFile(FileLock* l) { return target_->UnlockFile(l); } - void Schedule(void (*f)(void*), void* a, Priority pri) { - return target_->Schedule(f, a, pri); + + Status UnlockFile(FileLock* l) override { return target_->UnlockFile(l); } + + void Schedule(void (*f)(void* arg), void* a, Priority pri, + void* tag = nullptr, void (*u)(void* arg) = 0) override { + return target_->Schedule(f, a, pri, tag, u); + } + + int UnSchedule(void* tag, Priority pri) override { + return target_->UnSchedule(tag, pri); } - void StartThread(void (*f)(void*), void* a) { + + void StartThread(void (*f)(void*), void* a) override { return target_->StartThread(f, a); } - void WaitForJoin() { return target_->WaitForJoin(); } - virtual unsigned int GetThreadPoolQueueLen(Priority pri = LOW) const { + void WaitForJoin() override { return target_->WaitForJoin(); } + unsigned int GetThreadPoolQueueLen(Priority pri = LOW) const override { return target_->GetThreadPoolQueueLen(pri); } - virtual Status GetTestDirectory(std::string* path) { + Status GetTestDirectory(std::string* path) override { return target_->GetTestDirectory(path); } - virtual Status NewLogger(const std::string& fname, - shared_ptr* result) { + Status NewLogger(const std::string& fname, + shared_ptr* result) override { return target_->NewLogger(fname, result); } - uint64_t NowMicros() { - return target_->NowMicros(); - } - void SleepForMicroseconds(int micros) { + uint64_t NowMicros() override { return target_->NowMicros(); } + + void SleepForMicroseconds(int micros) override { target_->SleepForMicroseconds(micros); } - Status GetHostName(char* name, uint64_t len) { + Status GetHostName(char* name, uint64_t len) override { return target_->GetHostName(name, len); } - Status GetCurrentTime(int64_t* unix_time) { + Status GetCurrentTime(int64_t* unix_time) override { return target_->GetCurrentTime(unix_time); } Status GetAbsolutePath(const std::string& db_path, - std::string* output_path) { + std::string* output_path) override { return target_->GetAbsolutePath(db_path, output_path); } - void SetBackgroundThreads(int num, Priority pri) { + void SetBackgroundThreads(int num, Priority pri) override { return target_->SetBackgroundThreads(num, pri); } + int GetBackgroundThreads(Priority pri) override { + return target_->GetBackgroundThreads(pri); + } + + void IncBackgroundThreadsIfNeeded(int num, Priority pri) override { + return target_->IncBackgroundThreadsIfNeeded(num, pri); + } + void LowerThreadPoolIOPriority(Priority pool = LOW) override { target_->LowerThreadPoolIOPriority(pool); } - std::string TimeToString(uint64_t time) { + + std::string TimeToString(uint64_t time) override { return target_->TimeToString(time); } + Status GetThreadList(std::vector* thread_list) override { + return target_->GetThreadList(thread_list); + } + + ThreadStatusUpdater* GetThreadStatusUpdater() const override { + return target_->GetThreadStatusUpdater(); + } + + uint64_t GetThreadID() const override { + return target_->GetThreadID(); + } + + std::string GenerateUniqueId() override { + return target_->GenerateUniqueId(); + } + private: Env* target_; }; +// An implementation of WritableFile that forwards all calls to another +// WritableFile. May be useful to clients who wish to override just part of the +// functionality of another WritableFile. +// It's declared as friend of WritableFile to allow forwarding calls to +// protected virtual methods. +class WritableFileWrapper : public WritableFile { + public: + explicit WritableFileWrapper(WritableFile* t) : target_(t) { } + + Status Append(const Slice& data) override { return target_->Append(data); } + Status PositionedAppend(const Slice& data, uint64_t offset) override { + return target_->PositionedAppend(data, offset); + } + Status Truncate(uint64_t size) override { return target_->Truncate(size); } + Status Close() override { return target_->Close(); } + Status Flush() override { return target_->Flush(); } + Status Sync() override { return target_->Sync(); } + Status Fsync() override { return target_->Fsync(); } + bool IsSyncThreadSafe() const override { return target_->IsSyncThreadSafe(); } + void SetIOPriority(Env::IOPriority pri) override { + target_->SetIOPriority(pri); + } + Env::IOPriority GetIOPriority() override { return target_->GetIOPriority(); } + uint64_t GetFileSize() override { return target_->GetFileSize(); } + void GetPreallocationStatus(size_t* block_size, + size_t* last_allocated_block) override { + target_->GetPreallocationStatus(block_size, last_allocated_block); + } + size_t GetUniqueId(char* id, size_t max_size) const override { + return target_->GetUniqueId(id, max_size); + } + Status InvalidateCache(size_t offset, size_t length) override { + return target_->InvalidateCache(offset, length); + } + + void SetPreallocationBlockSize(size_t size) override { + target_->SetPreallocationBlockSize(size); + } + void PrepareWrite(size_t offset, size_t len) override { + target_->PrepareWrite(offset, len); + } + + protected: + Status Allocate(uint64_t offset, uint64_t len) override { + return target_->Allocate(offset, len); + } + Status RangeSync(uint64_t offset, uint64_t nbytes) override { + return target_->RangeSync(offset, nbytes); + } + + private: + WritableFile* target_; +}; + // Returns a new environment that stores its data in memory and delegates // all non-file-storage tasks to base_env. The caller must delete the result // when it is no longer needed. // *base_env must remain live while the result is in use. Env* NewMemEnv(Env* base_env); +// Returns a new environment that is used for HDFS environment. +// This is a factory method for HdfsEnv declared in hdfs/env_hdfs.h +Status NewHdfsEnv(Env** hdfs_env, const std::string& fsname); + +// Returns a new environment that measures function call times for filesystem +// operations, reporting results to variables in PerfContext. +// This is a factory method for TimedEnv defined in utilities/env_timed.cc. +Env* NewTimedEnv(Env* base_env); + } // namespace rocksdb #endif // STORAGE_ROCKSDB_INCLUDE_ENV_H_ diff --git a/include/rocksdb/env_encryption.h b/include/rocksdb/env_encryption.h new file mode 100644 index 00000000000..e4c924a4b4c --- /dev/null +++ b/include/rocksdb/env_encryption.h @@ -0,0 +1,196 @@ +// Copyright (c) 2016-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#if !defined(ROCKSDB_LITE) + +#include + +#include "env.h" + +namespace rocksdb { + +class EncryptionProvider; + +// Returns an Env that encrypts data when stored on disk and decrypts data when +// read from disk. +Env* NewEncryptedEnv(Env* base_env, EncryptionProvider* provider); + +// BlockAccessCipherStream is the base class for any cipher stream that +// supports random access at block level (without requiring data from other blocks). +// E.g. CTR (Counter operation mode) supports this requirement. +class BlockAccessCipherStream { + public: + virtual ~BlockAccessCipherStream() {}; + + // BlockSize returns the size of each block supported by this cipher stream. + virtual size_t BlockSize() = 0; + + // Encrypt one or more (partial) blocks of data at the file offset. + // Length of data is given in dataSize. + virtual Status Encrypt(uint64_t fileOffset, char *data, size_t dataSize); + + // Decrypt one or more (partial) blocks of data at the file offset. + // Length of data is given in dataSize. + virtual Status Decrypt(uint64_t fileOffset, char *data, size_t dataSize); + + protected: + // Allocate scratch space which is passed to EncryptBlock/DecryptBlock. + virtual void AllocateScratch(std::string&) = 0; + + // Encrypt a block of data at the given block index. + // Length of data is equal to BlockSize(); + virtual Status EncryptBlock(uint64_t blockIndex, char *data, char* scratch) = 0; + + // Decrypt a block of data at the given block index. + // Length of data is equal to BlockSize(); + virtual Status DecryptBlock(uint64_t blockIndex, char *data, char* scratch) = 0; +}; + +// BlockCipher +class BlockCipher { + public: + virtual ~BlockCipher() {}; + + // BlockSize returns the size of each block supported by this cipher stream. + virtual size_t BlockSize() = 0; + + // Encrypt a block of data. + // Length of data is equal to BlockSize(). + virtual Status Encrypt(char *data) = 0; + + // Decrypt a block of data. + // Length of data is equal to BlockSize(). + virtual Status Decrypt(char *data) = 0; +}; + +// Implements a BlockCipher using ROT13. +// +// Note: This is a sample implementation of BlockCipher, +// it is NOT considered safe and should NOT be used in production. +class ROT13BlockCipher : public BlockCipher { + private: + size_t blockSize_; + public: + ROT13BlockCipher(size_t blockSize) + : blockSize_(blockSize) {} + virtual ~ROT13BlockCipher() {}; + + // BlockSize returns the size of each block supported by this cipher stream. + virtual size_t BlockSize() override { return blockSize_; } + + // Encrypt a block of data. + // Length of data is equal to BlockSize(). + virtual Status Encrypt(char *data) override; + + // Decrypt a block of data. + // Length of data is equal to BlockSize(). + virtual Status Decrypt(char *data) override; +}; + +// CTRCipherStream implements BlockAccessCipherStream using an +// Counter operations mode. +// See https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation +// +// Note: This is a possible implementation of BlockAccessCipherStream, +// it is considered suitable for use. +class CTRCipherStream final : public BlockAccessCipherStream { + private: + BlockCipher& cipher_; + std::string iv_; + uint64_t initialCounter_; + public: + CTRCipherStream(BlockCipher& c, const char *iv, uint64_t initialCounter) + : cipher_(c), iv_(iv, c.BlockSize()), initialCounter_(initialCounter) {}; + virtual ~CTRCipherStream() {}; + + // BlockSize returns the size of each block supported by this cipher stream. + virtual size_t BlockSize() override { return cipher_.BlockSize(); } + + protected: + // Allocate scratch space which is passed to EncryptBlock/DecryptBlock. + virtual void AllocateScratch(std::string&) override; + + // Encrypt a block of data at the given block index. + // Length of data is equal to BlockSize(); + virtual Status EncryptBlock(uint64_t blockIndex, char *data, char *scratch) override; + + // Decrypt a block of data at the given block index. + // Length of data is equal to BlockSize(); + virtual Status DecryptBlock(uint64_t blockIndex, char *data, char *scratch) override; +}; + +// The encryption provider is used to create a cipher stream for a specific file. +// The returned cipher stream will be used for actual encryption/decryption +// actions. +class EncryptionProvider { + public: + virtual ~EncryptionProvider() {}; + + // GetPrefixLength returns the length of the prefix that is added to every file + // and used for storing encryption options. + // For optimal performance, the prefix length should be a multiple of + // the a page size. + virtual size_t GetPrefixLength() = 0; + + // CreateNewPrefix initialized an allocated block of prefix memory + // for a new file. + virtual Status CreateNewPrefix(const std::string& fname, char *prefix, size_t prefixLength) = 0; + + // CreateCipherStream creates a block access cipher stream for a file given + // given name and options. + virtual Status CreateCipherStream(const std::string& fname, const EnvOptions& options, + Slice& prefix, unique_ptr* result) = 0; +}; + +// This encryption provider uses a CTR cipher stream, with a given block cipher +// and IV. +// +// Note: This is a possible implementation of EncryptionProvider, +// it is considered suitable for use, provided a safe BlockCipher is used. +class CTREncryptionProvider : public EncryptionProvider { + private: + BlockCipher& cipher_; + protected: + const static size_t defaultPrefixLength = 4096; + + public: + CTREncryptionProvider(BlockCipher& c) + : cipher_(c) {}; + virtual ~CTREncryptionProvider() {} + + // GetPrefixLength returns the length of the prefix that is added to every file + // and used for storing encryption options. + // For optimal performance, the prefix length should be a multiple of + // the a page size. + virtual size_t GetPrefixLength() override; + + // CreateNewPrefix initialized an allocated block of prefix memory + // for a new file. + virtual Status CreateNewPrefix(const std::string& fname, char *prefix, size_t prefixLength) override; + + // CreateCipherStream creates a block access cipher stream for a file given + // given name and options. + virtual Status CreateCipherStream(const std::string& fname, const EnvOptions& options, + Slice& prefix, unique_ptr* result) override; + + protected: + // PopulateSecretPrefixPart initializes the data into a new prefix block + // that will be encrypted. This function will store the data in plain text. + // It will be encrypted later (before written to disk). + // Returns the amount of space (starting from the start of the prefix) + // that has been initialized. + virtual size_t PopulateSecretPrefixPart(char *prefix, size_t prefixLength, size_t blockSize); + + // CreateCipherStreamFromPrefix creates a block access cipher stream for a file given + // given name and options. The given prefix is already decrypted. + virtual Status CreateCipherStreamFromPrefix(const std::string& fname, const EnvOptions& options, + uint64_t initialCounter, const Slice& iv, const Slice& prefix, unique_ptr* result); +}; + +} // namespace rocksdb + +#endif // !defined(ROCKSDB_LITE) diff --git a/include/rocksdb/experimental.h b/include/rocksdb/experimental.h new file mode 100644 index 00000000000..0592fe36b17 --- /dev/null +++ b/include/rocksdb/experimental.h @@ -0,0 +1,29 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include "rocksdb/db.h" +#include "rocksdb/status.h" + +namespace rocksdb { +namespace experimental { + +// Supported only for Leveled compaction +Status SuggestCompactRange(DB* db, ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end); +Status SuggestCompactRange(DB* db, const Slice* begin, const Slice* end); + +// Move all L0 files to target_level skipping compaction. +// This operation succeeds only if the files in L0 have disjoint ranges; this +// is guaranteed to happen, for instance, if keys are inserted in sorted +// order. Furthermore, all levels between 1 and target_level must be empty. +// If any of the above condition is violated, InvalidArgument will be +// returned. +Status PromoteL0(DB* db, ColumnFamilyHandle* column_family, + int target_level = 1); + +} // namespace experimental +} // namespace rocksdb diff --git a/include/rocksdb/filter_policy.h b/include/rocksdb/filter_policy.h index fa44db45ff5..8add48e4963 100644 --- a/include/rocksdb/filter_policy.h +++ b/include/rocksdb/filter_policy.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2012 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -20,12 +20,66 @@ #ifndef STORAGE_ROCKSDB_INCLUDE_FILTER_POLICY_H_ #define STORAGE_ROCKSDB_INCLUDE_FILTER_POLICY_H_ +#include +#include +#include #include +#include namespace rocksdb { class Slice; +// A class that takes a bunch of keys, then generates filter +class FilterBitsBuilder { + public: + virtual ~FilterBitsBuilder() {} + + // Add Key to filter, you could use any way to store the key. + // Such as: storing hashes or original keys + // Keys are in sorted order and duplicated keys are possible. + virtual void AddKey(const Slice& key) = 0; + + // Generate the filter using the keys that are added + // The return value of this function would be the filter bits, + // The ownership of actual data is set to buf + virtual Slice Finish(std::unique_ptr* buf) = 0; + + // Calculate num of entries fit into a space. + virtual int CalculateNumEntry(const uint32_t space) { +#ifndef ROCKSDB_LITE + throw std::runtime_error("CalculateNumEntry not Implemented"); +#else + abort(); +#endif + return 0; + } +}; + +// A class that checks if a key can be in filter +// It should be initialized by Slice generated by BitsBuilder +class FilterBitsReader { + public: + virtual ~FilterBitsReader() {} + + // Check if the entry match the bits in filter + virtual bool MayMatch(const Slice& entry) = 0; +}; + +// We add a new format of filter block called full filter block +// This new interface gives you more space of customization +// +// For the full filter block, you can plug in your version by implement +// the FilterBitsBuilder and FilterBitsReader +// +// There are two sets of interface in FilterPolicy +// Set 1: CreateFilter, KeyMayMatch: used for blockbased filter +// Set 2: GetFilterBitsBuilder, GetFilterBitsReader, they are used for +// full filter. +// Set 1 MUST be implemented correctly, Set 2 is optional +// RocksDB would first try using functions in Set 2. if they return nullptr, +// it would use Set 1 instead. +// You can choose filter type in NewBloomFilterPolicy class FilterPolicy { public: virtual ~FilterPolicy(); @@ -51,11 +105,28 @@ class FilterPolicy { // This method may return true or false if the key was not on the // list, but it should aim to return false with a high probability. virtual bool KeyMayMatch(const Slice& key, const Slice& filter) const = 0; + + // Get the FilterBitsBuilder, which is ONLY used for full filter block + // It contains interface to take individual key, then generate filter + virtual FilterBitsBuilder* GetFilterBitsBuilder() const { + return nullptr; + } + + // Get the FilterBitsReader, which is ONLY used for full filter block + // It contains interface to tell if key can be in filter + // The input slice should NOT be deleted by FilterPolicy + virtual FilterBitsReader* GetFilterBitsReader(const Slice& contents) const { + return nullptr; + } }; // Return a new filter policy that uses a bloom filter with approximately -// the specified number of bits per key. A good value for bits_per_key +// the specified number of bits per key. +// +// bits_per_key: bits per key in bloom filter. A good value for bits_per_key // is 10, which yields a filter with ~ 1% false positive rate. +// use_block_based_builder: use block based filter rather than full filter. +// If you want to builder full filter, it needs to be set to false. // // Callers must delete the result after any database that is using the // result has been closed. @@ -67,8 +138,8 @@ class FilterPolicy { // ignores trailing spaces, it would be incorrect to use a // FilterPolicy (like NewBloomFilterPolicy) that does not ignore // trailing spaces in keys. -extern const FilterPolicy* NewBloomFilterPolicy(int bits_per_key); - +extern const FilterPolicy* NewBloomFilterPolicy(int bits_per_key, + bool use_block_based_builder = true); } #endif // STORAGE_ROCKSDB_INCLUDE_FILTER_POLICY_H_ diff --git a/include/rocksdb/flush_block_policy.h b/include/rocksdb/flush_block_policy.h index 939725cf409..5daa9676248 100644 --- a/include/rocksdb/flush_block_policy.h +++ b/include/rocksdb/flush_block_policy.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once @@ -48,13 +48,15 @@ class FlushBlockBySizePolicyFactory : public FlushBlockPolicyFactory { public: FlushBlockBySizePolicyFactory() {} - virtual const char* Name() const override { - return "FlushBlockBySizePolicyFactory"; - } + const char* Name() const override { return "FlushBlockBySizePolicyFactory"; } - virtual FlushBlockPolicy* NewFlushBlockPolicy( + FlushBlockPolicy* NewFlushBlockPolicy( const BlockBasedTableOptions& table_options, const BlockBuilder& data_block_builder) const override; + + static FlushBlockPolicy* NewFlushBlockPolicy( + const uint64_t size, const int deviation, + const BlockBuilder& data_block_builder); }; } // rocksdb diff --git a/include/rocksdb/iostats_context.h b/include/rocksdb/iostats_context.h index e06ee1773a8..77a59643a1c 100644 --- a/include/rocksdb/iostats_context.h +++ b/include/rocksdb/iostats_context.h @@ -1,22 +1,24 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -#ifndef INCLUDE_ROCKSDB_IOSTATS_CONTEXT_H_ -#define INCLUDE_ROCKSDB_IOSTATS_CONTEXT_H_ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once #include #include +#include "rocksdb/perf_level.h" + // A thread local context for gathering io-stats efficiently and transparently. +// Use SetPerfLevel(PerfLevel::kEnableTime) to enable time stats. + namespace rocksdb { struct IOStatsContext { // reset all io-stats counter to zero void Reset(); - std::string ToString() const; + std::string ToString(bool exclude_zero_counters = false) const; // the thread pool id uint64_t thread_pool_id; @@ -25,12 +27,26 @@ struct IOStatsContext { uint64_t bytes_written; // number of bytes that has been read. uint64_t bytes_read; + + // time spent in open() and fopen(). + uint64_t open_nanos; + // time spent in fallocate(). + uint64_t allocate_nanos; + // time spent in write() and pwrite(). + uint64_t write_nanos; + // time spent in read() and pread() + uint64_t read_nanos; + // time spent in sync_file_range(). + uint64_t range_sync_nanos; + // time spent in fsync + uint64_t fsync_nanos; + // time spent in preparing write (fallocate etc). + uint64_t prepare_write_nanos; + // time spent in Logger::Logv(). + uint64_t logger_nanos; }; -#ifndef IOS_CROSS_COMPILE -extern __thread IOStatsContext iostats_context; -#endif // IOS_CROSS_COMPILE +// Get Thread-local IOStatsContext object pointer +IOStatsContext* get_iostats_context(); } // namespace rocksdb - -#endif // INCLUDE_ROCKSDB_IOSTATS_CONTEXT_H_ diff --git a/include/rocksdb/iterator.h b/include/rocksdb/iterator.h index 7538e9cfb54..d4ac5281816 100644 --- a/include/rocksdb/iterator.h +++ b/include/rocksdb/iterator.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -19,15 +19,17 @@ #ifndef STORAGE_ROCKSDB_INCLUDE_ITERATOR_H_ #define STORAGE_ROCKSDB_INCLUDE_ITERATOR_H_ +#include +#include "rocksdb/cleanable.h" #include "rocksdb/slice.h" #include "rocksdb/status.h" namespace rocksdb { -class Iterator { +class Iterator : public Cleanable { public: - Iterator(); - virtual ~Iterator(); + Iterator() {} + virtual ~Iterator() {} // An iterator is either positioned at a key/value pair, or // not valid. This method returns true iff the iterator is valid. @@ -46,6 +48,11 @@ class Iterator { // an entry that comes at or past target. virtual void Seek(const Slice& target) = 0; + // Position at the last key in the source that at or before target + // The iterator is Valid() after this call iff the source contains + // an entry that comes at or before target. + virtual void SeekForPrev(const Slice& target) {} + // Moves to the next entry in the source. After this call, Valid() is // true iff the iterator was not positioned at the last entry in the source. // REQUIRES: Valid() @@ -73,23 +80,26 @@ class Iterator { // satisfied without doing some IO, then this returns Status::Incomplete(). virtual Status status() const = 0; - // Clients are allowed to register function/arg1/arg2 triples that - // will be invoked when this iterator is destroyed. - // - // Note that unlike all of the preceding methods, this method is - // not abstract and therefore clients should not override it. - typedef void (*CleanupFunction)(void* arg1, void* arg2); - void RegisterCleanup(CleanupFunction function, void* arg1, void* arg2); + // If supported, renew the iterator to represent the latest state. The + // iterator will be invalidated after the call. Not supported if + // ReadOptions.snapshot is given when creating the iterator. + virtual Status Refresh() { + return Status::NotSupported("Refresh() is not supported"); + } + + // Property "rocksdb.iterator.is-key-pinned": + // If returning "1", this means that the Slice returned by key() is valid + // as long as the iterator is not deleted. + // It is guaranteed to always return "1" if + // - Iterator created with ReadOptions::pin_data = true + // - DB tables were created with + // BlockBasedTableOptions::use_delta_encoding = false. + // Property "rocksdb.iterator.super-version-number": + // LSM version used by the iterator. The same format as DB Property + // kCurrentSuperVersionNumber. See its comment for more information. + virtual Status GetProperty(std::string prop_name, std::string* prop); private: - struct Cleanup { - CleanupFunction function; - void* arg1; - void* arg2; - Cleanup* next; - }; - Cleanup cleanup_; - // No copying allowed Iterator(const Iterator&); void operator=(const Iterator&); diff --git a/include/rocksdb/ldb_tool.h b/include/rocksdb/ldb_tool.h index 1b1c64b0677..0ec2da9fc05 100644 --- a/include/rocksdb/ldb_tool.h +++ b/include/rocksdb/ldb_tool.h @@ -1,10 +1,12 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef ROCKSDB_LITE #pragma once #include +#include +#include "rocksdb/db.h" #include "rocksdb/options.h" namespace rocksdb { @@ -24,12 +26,16 @@ struct LDBOptions { // Key formatter that converts a slice to a readable string. // Default: Slice::ToString() std::shared_ptr key_formatter; + + std::string print_help_header = "ldb - RocksDB Tool"; }; class LDBTool { public: - void Run(int argc, char** argv, Options db_options= Options(), - const LDBOptions& ldb_options = LDBOptions()); + void Run( + int argc, char** argv, Options db_options = Options(), + const LDBOptions& ldb_options = LDBOptions(), + const std::vector* column_families = nullptr); }; } // namespace rocksdb diff --git a/include/rocksdb/listener.h b/include/rocksdb/listener.h new file mode 100644 index 00000000000..e132033db2a --- /dev/null +++ b/include/rocksdb/listener.h @@ -0,0 +1,391 @@ +// Copyright (c) 2014 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include +#include +#include +#include +#include "rocksdb/compaction_job_stats.h" +#include "rocksdb/status.h" +#include "rocksdb/table_properties.h" + +namespace rocksdb { + +typedef std::unordered_map> + TablePropertiesCollection; + +class DB; +class ColumnFamilyHandle; +class Status; +struct CompactionJobStats; +enum CompressionType : unsigned char; + +enum class TableFileCreationReason { + kFlush, + kCompaction, + kRecovery, +}; + +struct TableFileCreationBriefInfo { + // the name of the database where the file was created + std::string db_name; + // the name of the column family where the file was created. + std::string cf_name; + // the path to the created file. + std::string file_path; + // the id of the job (which could be flush or compaction) that + // created the file. + int job_id; + // reason of creating the table. + TableFileCreationReason reason; +}; + +struct TableFileCreationInfo : public TableFileCreationBriefInfo { + TableFileCreationInfo() = default; + explicit TableFileCreationInfo(TableProperties&& prop) + : table_properties(prop) {} + // the size of the file. + uint64_t file_size; + // Detailed properties of the created file. + TableProperties table_properties; + // The status indicating whether the creation was successful or not. + Status status; +}; + +enum class CompactionReason { + kUnknown, + // [Level] number of L0 files > level0_file_num_compaction_trigger + kLevelL0FilesNum, + // [Level] total size of level > MaxBytesForLevel() + kLevelMaxLevelSize, + // [Universal] Compacting for size amplification + kUniversalSizeAmplification, + // [Universal] Compacting for size ratio + kUniversalSizeRatio, + // [Universal] number of sorted runs > level0_file_num_compaction_trigger + kUniversalSortedRunNum, + // [FIFO] total size > max_table_files_size + kFIFOMaxSize, + // [FIFO] reduce number of files. + kFIFOReduceNumFiles, + // [FIFO] files with creation time < (current_time - interval) + kFIFOTtl, + // Manual compaction + kManualCompaction, + // DB::SuggestCompactRange() marked files for compaction + kFilesMarkedForCompaction, +}; + +enum class BackgroundErrorReason { + kFlush, + kCompaction, + kWriteCallback, + kMemTable, +}; + +#ifndef ROCKSDB_LITE + +struct TableFileDeletionInfo { + // The name of the database where the file was deleted. + std::string db_name; + // The path to the deleted file. + std::string file_path; + // The id of the job which deleted the file. + int job_id; + // The status indicating whether the deletion was successful or not. + Status status; +}; + +struct FlushJobInfo { + // the name of the column family + std::string cf_name; + // the path to the newly created file + std::string file_path; + // the id of the thread that completed this flush job. + uint64_t thread_id; + // the job id, which is unique in the same thread. + int job_id; + // If true, then rocksdb is currently slowing-down all writes to prevent + // creating too many Level 0 files as compaction seems not able to + // catch up the write request speed. This indicates that there are + // too many files in Level 0. + bool triggered_writes_slowdown; + // If true, then rocksdb is currently blocking any writes to prevent + // creating more L0 files. This indicates that there are too many + // files in level 0. Compactions should try to compact L0 files down + // to lower levels as soon as possible. + bool triggered_writes_stop; + // The smallest sequence number in the newly created file + SequenceNumber smallest_seqno; + // The largest sequence number in the newly created file + SequenceNumber largest_seqno; + // Table properties of the table being flushed + TableProperties table_properties; +}; + +struct CompactionJobInfo { + CompactionJobInfo() = default; + explicit CompactionJobInfo(const CompactionJobStats& _stats) : + stats(_stats) {} + + // the name of the column family where the compaction happened. + std::string cf_name; + // the status indicating whether the compaction was successful or not. + Status status; + // the id of the thread that completed this compaction job. + uint64_t thread_id; + // the job id, which is unique in the same thread. + int job_id; + // the smallest input level of the compaction. + int base_input_level; + // the output level of the compaction. + int output_level; + // the names of the compaction input files. + std::vector input_files; + + // the names of the compaction output files. + std::vector output_files; + // Table properties for input and output tables. + // The map is keyed by values from input_files and output_files. + TablePropertiesCollection table_properties; + + // Reason to run the compaction + CompactionReason compaction_reason; + + // Compression algorithm used for output files + CompressionType compression; + + // If non-null, this variable stores detailed information + // about this compaction. + CompactionJobStats stats; +}; + +struct MemTableInfo { + // the name of the column family to which memtable belongs + std::string cf_name; + // Sequence number of the first element that was inserted + // into the memtable. + SequenceNumber first_seqno; + // Sequence number that is guaranteed to be smaller than or equal + // to the sequence number of any key that could be inserted into this + // memtable. It can then be assumed that any write with a larger(or equal) + // sequence number will be present in this memtable or a later memtable. + SequenceNumber earliest_seqno; + // Total number of entries in memtable + uint64_t num_entries; + // Total number of deletes in memtable + uint64_t num_deletes; + +}; + +struct ExternalFileIngestionInfo { + // the name of the column family + std::string cf_name; + // Path of the file outside the DB + std::string external_file_path; + // Path of the file inside the DB + std::string internal_file_path; + // The global sequence number assigned to keys in this file + SequenceNumber global_seqno; + // Table properties of the table being flushed + TableProperties table_properties; +}; + +// A call-back function to RocksDB which will be called when the compaction +// iterator is compacting values. It is mean to be returned from +// EventListner::GetCompactionEventListner() at the beginning of compaction +// job. +class CompactionEventListener { + public: + enum CompactionListenerValueType { + kValue, + kMergeOperand, + kDelete, + kSingleDelete, + kRangeDelete, + kBlobIndex, + kInvalid, + }; + + virtual void OnCompaction(int level, const Slice& key, + CompactionListenerValueType value_type, + const Slice& existing_value, + const SequenceNumber& sn, bool is_new) = 0; + + virtual ~CompactionEventListener() = default; +}; + +// EventListener class contains a set of call-back functions that will +// be called when specific RocksDB event happens such as flush. It can +// be used as a building block for developing custom features such as +// stats-collector or external compaction algorithm. +// +// Note that call-back functions should not run for an extended period of +// time before the function returns, otherwise RocksDB may be blocked. +// For example, it is not suggested to do DB::CompactFiles() (as it may +// run for a long while) or issue many of DB::Put() (as Put may be blocked +// in certain cases) in the same thread in the EventListener callback. +// However, doing DB::CompactFiles() and DB::Put() in another thread is +// considered safe. +// +// [Threading] All EventListener callback will be called using the +// actual thread that involves in that specific event. For example, it +// is the RocksDB background flush thread that does the actual flush to +// call EventListener::OnFlushCompleted(). +// +// [Locking] All EventListener callbacks are designed to be called without +// the current thread holding any DB mutex. This is to prevent potential +// deadlock and performance issue when using EventListener callback +// in a complex way. However, all EventListener call-back functions +// should not run for an extended period of time before the function +// returns, otherwise RocksDB may be blocked. For example, it is not +// suggested to do DB::CompactFiles() (as it may run for a long while) +// or issue many of DB::Put() (as Put may be blocked in certain cases) +// in the same thread in the EventListener callback. However, doing +// DB::CompactFiles() and DB::Put() in a thread other than the +// EventListener callback thread is considered safe. +class EventListener { + public: + // A call-back function to RocksDB which will be called whenever a + // registered RocksDB flushes a file. The default implementation is + // no-op. + // + // Note that the this function must be implemented in a way such that + // it should not run for an extended period of time before the function + // returns. Otherwise, RocksDB may be blocked. + virtual void OnFlushCompleted(DB* /*db*/, + const FlushJobInfo& /*flush_job_info*/) {} + + // A call-back function to RocksDB which will be called before a + // RocksDB starts to flush memtables. The default implementation is + // no-op. + // + // Note that the this function must be implemented in a way such that + // it should not run for an extended period of time before the function + // returns. Otherwise, RocksDB may be blocked. + virtual void OnFlushBegin(DB* /*db*/, + const FlushJobInfo& /*flush_job_info*/) {} + + // A call-back function for RocksDB which will be called whenever + // a SST file is deleted. Different from OnCompactionCompleted and + // OnFlushCompleted, this call-back is designed for external logging + // service and thus only provide string parameters instead + // of a pointer to DB. Applications that build logic basic based + // on file creations and deletions is suggested to implement + // OnFlushCompleted and OnCompactionCompleted. + // + // Note that if applications would like to use the passed reference + // outside this function call, they should make copies from the + // returned value. + virtual void OnTableFileDeleted(const TableFileDeletionInfo& /*info*/) {} + + // A call-back function for RocksDB which will be called whenever + // a registered RocksDB compacts a file. The default implementation + // is a no-op. + // + // Note that this function must be implemented in a way such that + // it should not run for an extended period of time before the function + // returns. Otherwise, RocksDB may be blocked. + // + // @param db a pointer to the rocksdb instance which just compacted + // a file. + // @param ci a reference to a CompactionJobInfo struct. 'ci' is released + // after this function is returned, and must be copied if it is needed + // outside of this function. + virtual void OnCompactionCompleted(DB* /*db*/, + const CompactionJobInfo& /*ci*/) {} + + // A call-back function for RocksDB which will be called whenever + // a SST file is created. Different from OnCompactionCompleted and + // OnFlushCompleted, this call-back is designed for external logging + // service and thus only provide string parameters instead + // of a pointer to DB. Applications that build logic basic based + // on file creations and deletions is suggested to implement + // OnFlushCompleted and OnCompactionCompleted. + // + // Historically it will only be called if the file is successfully created. + // Now it will also be called on failure case. User can check info.status + // to see if it succeeded or not. + // + // Note that if applications would like to use the passed reference + // outside this function call, they should make copies from these + // returned value. + virtual void OnTableFileCreated(const TableFileCreationInfo& /*info*/) {} + + // A call-back function for RocksDB which will be called before + // a SST file is being created. It will follow by OnTableFileCreated after + // the creation finishes. + // + // Note that if applications would like to use the passed reference + // outside this function call, they should make copies from these + // returned value. + virtual void OnTableFileCreationStarted( + const TableFileCreationBriefInfo& /*info*/) {} + + // A call-back function for RocksDB which will be called before + // a memtable is made immutable. + // + // Note that the this function must be implemented in a way such that + // it should not run for an extended period of time before the function + // returns. Otherwise, RocksDB may be blocked. + // + // Note that if applications would like to use the passed reference + // outside this function call, they should make copies from these + // returned value. + virtual void OnMemTableSealed( + const MemTableInfo& /*info*/) {} + + // A call-back function for RocksDB which will be called before + // a column family handle is deleted. + // + // Note that the this function must be implemented in a way such that + // it should not run for an extended period of time before the function + // returns. Otherwise, RocksDB may be blocked. + // @param handle is a pointer to the column family handle to be deleted + // which will become a dangling pointer after the deletion. + virtual void OnColumnFamilyHandleDeletionStarted(ColumnFamilyHandle* handle) { + } + + // A call-back function for RocksDB which will be called after an external + // file is ingested using IngestExternalFile. + // + // Note that the this function will run on the same thread as + // IngestExternalFile(), if this function is blocked, IngestExternalFile() + // will be blocked from finishing. + virtual void OnExternalFileIngested( + DB* /*db*/, const ExternalFileIngestionInfo& /*info*/) {} + + // A call-back function for RocksDB which will be called before setting the + // background error status to a non-OK value. The new background error status + // is provided in `bg_error` and can be modified by the callback. E.g., a + // callback can suppress errors by resetting it to Status::OK(), thus + // preventing the database from entering read-only mode. We do not provide any + // guarantee when failed flushes/compactions will be rescheduled if the user + // suppresses an error. + // + // Note that this function can run on the same threads as flush, compaction, + // and user writes. So, it is extremely important not to perform heavy + // computations or blocking calls in this function. + virtual void OnBackgroundError(BackgroundErrorReason /* reason */, + Status* /* bg_error */) {} + + // Factory method to return CompactionEventListener. If multiple listeners + // provides CompactionEventListner, only the first one will be used. + virtual CompactionEventListener* GetCompactionEventListener() { + return nullptr; + } + + virtual ~EventListener() {} +}; + +#else + +class EventListener { +}; + +#endif // ROCKSDB_LITE + +} // namespace rocksdb diff --git a/include/rocksdb/memtablerep.h b/include/rocksdb/memtablerep.h index b7fc39c812a..347dd3096c2 100644 --- a/include/rocksdb/memtablerep.h +++ b/include/rocksdb/memtablerep.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file contains the interface that must be implemented by any collection // to be used as the backing store for a MemTable. Such a collection must @@ -14,8 +14,8 @@ // (4) Items are never deleted. // The liberal use of assertions is encouraged to enforce (1). // -// The factory will be passed an Arena object when a new MemTableRep is -// requested. The API for this object is in rocksdb/arena.h. +// The factory will be passed an MemTableAllocator object when a new MemTableRep +// is requested. // // Users can implement their own memtable representations. We include three // types built in: @@ -36,11 +36,14 @@ #pragma once #include +#include #include +#include namespace rocksdb { class Arena; +class Allocator; class LookupKey; class Slice; class SliceTransform; @@ -65,25 +68,49 @@ class MemTableRep { virtual ~KeyComparator() { } }; - explicit MemTableRep(Arena* arena) : arena_(arena) {} + explicit MemTableRep(Allocator* allocator) : allocator_(allocator) {} - // Allocate a buf of len size for storing key. The idea is that a specific - // memtable representation knows its underlying data structure better. By - // allowing it to allocate memory, it can possibly put correlated stuff - // in consecutive memory area to make processor prefetching more efficient. + // Allocate a buf of len size for storing key. The idea is that a + // specific memtable representation knows its underlying data structure + // better. By allowing it to allocate memory, it can possibly put + // correlated stuff in consecutive memory area to make processor + // prefetching more efficient. virtual KeyHandle Allocate(const size_t len, char** buf); // Insert key into the collection. (The caller will pack key and value into a // single buffer and pass that in as the parameter to Insert). // REQUIRES: nothing that compares equal to key is currently in the - // collection. + // collection, and no concurrent modifications to the table in progress virtual void Insert(KeyHandle handle) = 0; + // Same as Insert(), but in additional pass a hint to insert location for + // the key. If hint points to nullptr, a new hint will be populated. + // otherwise the hint will be updated to reflect the last insert location. + // + // Currently only skip-list based memtable implement the interface. Other + // implementations will fallback to Insert() by default. + virtual void InsertWithHint(KeyHandle handle, void** hint) { + // Ignore the hint by default. + Insert(handle); + } + + // Like Insert(handle), but may be called concurrent with other calls + // to InsertConcurrently for other handles + virtual void InsertConcurrently(KeyHandle handle) { +#ifndef ROCKSDB_LITE + throw std::runtime_error("concurrent insert not supported"); +#else + abort(); +#endif + } + // Returns true iff an entry that compares equal to key is in the collection. virtual bool Contains(const char* key) const = 0; - // Notify this table rep that it will no longer be added to. By default, does - // nothing. + // Notify this table rep that it will no longer be added to. By default, + // does nothing. After MarkReadOnly() is called, this table rep will + // not be written to (ie No more calls to Allocate(), Insert(), + // or any writes done directly to entries accessed through the iterator.) virtual void MarkReadOnly() { } // Look up key from the mem table, since the first key in the mem table whose @@ -91,6 +118,7 @@ class MemTableRep { // callback_args directly forwarded as the first parameter, and the mem table // key as the second parameter. If the return value is false, then terminates. // Otherwise, go through the next key. + // // It's safe for Get() to terminate after having finished all the potential // key for the k.user_key(), or not. // @@ -100,8 +128,13 @@ class MemTableRep { virtual void Get(const LookupKey& k, void* callback_args, bool (*callback_func)(void* arg, const char* entry)); + virtual uint64_t ApproximateNumEntries(const Slice& start_ikey, + const Slice& end_key) { + return 0; + } + // Report an approximation of how much memory has been used other than memory - // that was allocated through the arena. + // that was allocated through the allocator. Safe to call from any thread. virtual size_t ApproximateMemoryUsage() = 0; virtual ~MemTableRep() { } @@ -132,6 +165,10 @@ class MemTableRep { // Advance to the first entry with a key >= target virtual void Seek(const Slice& internal_key, const char* memtable_key) = 0; + // retreat to the first entry with a key <= target + virtual void SeekForPrev(const Slice& internal_key, + const char* memtable_key) = 0; + // Position at the first entry in collection. // Final state of iterator is Valid() iff collection is not empty. virtual void SeekToFirst() = 0; @@ -150,7 +187,7 @@ class MemTableRep { // Return an iterator that has a special Seek semantics. The result of // a Seek might only include keys with the same prefix as the target key. - // arena: If not null, the arena needs to be used to allocate the Iterator. + // arena: If not null, the arena is used to allocate the Iterator. // When destroying the iterator, the caller will not call "delete" // but Iterator::~Iterator() directly. The destructor needs to destroy // all the states but those allocated in arena. @@ -171,7 +208,7 @@ class MemTableRep { // user key. virtual Slice UserKey(const char* key) const; - Arena* arena_; + Allocator* allocator_; }; // This is the base class for all factories that are used by RocksDB to create @@ -179,19 +216,45 @@ class MemTableRep { class MemTableRepFactory { public: virtual ~MemTableRepFactory() {} + virtual MemTableRep* CreateMemTableRep(const MemTableRep::KeyComparator&, - Arena*, const SliceTransform*, + Allocator*, const SliceTransform*, Logger* logger) = 0; + virtual MemTableRep* CreateMemTableRep( + const MemTableRep::KeyComparator& key_cmp, Allocator* allocator, + const SliceTransform* slice_transform, Logger* logger, + uint32_t /* column_family_id */) { + return CreateMemTableRep(key_cmp, allocator, slice_transform, logger); + } + virtual const char* Name() const = 0; + + // Return true if the current MemTableRep supports concurrent inserts + // Default: false + virtual bool IsInsertConcurrentlySupported() const { return false; } }; // This uses a skip list to store keys. It is the default. +// +// Parameters: +// lookahead: If non-zero, each iterator's seek operation will start the +// search from the previously visited record (doing at most 'lookahead' +// steps). This is an optimization for the access pattern including many +// seeks with consecutive keys. class SkipListFactory : public MemTableRepFactory { public: + explicit SkipListFactory(size_t lookahead = 0) : lookahead_(lookahead) {} + + using MemTableRepFactory::CreateMemTableRep; virtual MemTableRep* CreateMemTableRep(const MemTableRep::KeyComparator&, - Arena*, const SliceTransform*, + Allocator*, const SliceTransform*, Logger* logger) override; virtual const char* Name() const override { return "SkipListFactory"; } + + bool IsInsertConcurrentlySupported() const override { return true; } + + private: + const size_t lookahead_; }; #ifndef ROCKSDB_LITE @@ -208,9 +271,12 @@ class VectorRepFactory : public MemTableRepFactory { public: explicit VectorRepFactory(size_t count = 0) : count_(count) { } + + using MemTableRepFactory::CreateMemTableRep; virtual MemTableRep* CreateMemTableRep(const MemTableRep::KeyComparator&, - Arena*, const SliceTransform*, + Allocator*, const SliceTransform*, Logger* logger) override; + virtual const char* Name() const override { return "VectorRepFactory"; } diff --git a/include/rocksdb/merge_operator.h b/include/rocksdb/merge_operator.h index 2ae64c1bc27..f2947100553 100644 --- a/include/rocksdb/merge_operator.h +++ b/include/rocksdb/merge_operator.h @@ -1,14 +1,16 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef STORAGE_ROCKSDB_INCLUDE_MERGE_OPERATOR_H_ #define STORAGE_ROCKSDB_INCLUDE_MERGE_OPERATOR_H_ +#include #include #include -#include +#include + #include "rocksdb/slice.h" namespace rocksdb { @@ -31,7 +33,7 @@ class Logger; // into rocksdb); numeric addition and string concatenation are examples; // // b) MergeOperator - the generic class for all the more abstract / complex -// operations; one method (FullMerge) to merge a Put/Delete value with a +// operations; one method (FullMergeV2) to merge a Put/Delete value with a // merge operand; and another method (PartialMerge) that merges multiple // operands together. this is especially useful if your key values have // complex structures but you would still like to support client-specific @@ -54,7 +56,8 @@ class MergeOperator { // merge operation semantics // existing: (IN) null indicates that the key does not exist before this op // operand_list:(IN) the sequence of merge operations to apply, front() first. - // new_value:(OUT) Client is responsible for filling the merge result here + // new_value:(OUT) Client is responsible for filling the merge result here. + // The string that new_value is pointing to will be empty. // logger: (IN) Client could use this to log errors during merge. // // Return true on success. @@ -67,7 +70,49 @@ class MergeOperator { const Slice* existing_value, const std::deque& operand_list, std::string* new_value, - Logger* logger) const = 0; + Logger* logger) const { + // deprecated, please use FullMergeV2() + assert(false); + return false; + } + + struct MergeOperationInput { + explicit MergeOperationInput(const Slice& _key, + const Slice* _existing_value, + const std::vector& _operand_list, + Logger* _logger) + : key(_key), + existing_value(_existing_value), + operand_list(_operand_list), + logger(_logger) {} + + // The key associated with the merge operation. + const Slice& key; + // The existing value of the current key, nullptr means that the + // value dont exist. + const Slice* existing_value; + // A list of operands to apply. + const std::vector& operand_list; + // Logger could be used by client to log any errors that happen during + // the merge operation. + Logger* logger; + }; + + struct MergeOperationOutput { + explicit MergeOperationOutput(std::string& _new_value, + Slice& _existing_operand) + : new_value(_new_value), existing_operand(_existing_operand) {} + + // Client is responsible for filling the merge result here. + std::string& new_value; + // If the merge result is one of the existing operands (or existing_value), + // client can set this field to the operand (or existing_value) instead of + // using new_value. + Slice& existing_operand; + }; + + virtual bool FullMergeV2(const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const; // This function performs merge(left_op, right_op) // when both the operands are themselves merge operation types @@ -80,6 +125,8 @@ class MergeOperator { // DB::Merge(key, *new_value) would yield the same result as a call // to DB::Merge(key, left_op) followed by DB::Merge(key, right_op). // + // The string that new_value is pointing to will be empty. + // // The default implementation of PartialMergeMulti will use this function // as a helper, for backward compatibility. Any successor class of // MergeOperator should either implement PartialMerge or PartialMergeMulti, @@ -95,7 +142,7 @@ class MergeOperator { // TODO: Presently there is no way to differentiate between error/corruption // and simply "return false". For now, the client should simply return // false in any case it cannot perform partial-merge, regardless of reason. - // If there is corruption in the data, handle it in the FullMerge() function, + // If there is corruption in the data, handle it in the FullMergeV2() function // and return false there. The default implementation of PartialMerge will // always return false. virtual bool PartialMerge(const Slice& key, const Slice& left_operand, @@ -116,10 +163,10 @@ class MergeOperator { // the same result as subquential individual calls to DB::Merge(key, operand) // for each operand in operand_list from front() to back(). // - // The PartialMergeMulti function will be called only when the list of - // operands are long enough. The minimum amount of operands that will be - // passed to the function are specified by the "min_partial_merge_operands" - // option. + // The string that new_value is pointing to will be empty. + // + // The PartialMergeMulti function will be called when there are at least two + // operands. // // In the default implementation, PartialMergeMulti will invoke PartialMerge // multiple times, where each time it only merges two operands. Developers @@ -136,18 +183,26 @@ class MergeOperator { // no checking is enforced. Client is responsible for providing // consistent MergeOperator between DB opens. virtual const char* Name() const = 0; + + // Determines whether the MergeOperator can be called with just a single + // merge operand. + // Override and return true for allowing a single operand. FullMergeV2 and + // PartialMerge/PartialMergeMulti should be implemented accordingly to handle + // a single operand. + virtual bool AllowSingleOperand() const { return false; } }; // The simpler, associative merge operator. class AssociativeMergeOperator : public MergeOperator { public: - virtual ~AssociativeMergeOperator() {} + ~AssociativeMergeOperator() override {} // Gives the client a way to express the read -> modify -> write semantics // key: (IN) The key that's associated with this merge operation. // existing_value:(IN) null indicates the key does not exist before this op // value: (IN) the value to update/merge the existing_value with - // new_value: (OUT) Client is responsible for filling the merge result here + // new_value: (OUT) Client is responsible for filling the merge result + // here. The string that new_value is pointing to will be empty. // logger: (IN) Client could use this to log errors during merge. // // Return true on success. @@ -164,17 +219,12 @@ class AssociativeMergeOperator : public MergeOperator { private: // Default implementations of the MergeOperator functions - virtual bool FullMerge(const Slice& key, - const Slice* existing_value, - const std::deque& operand_list, - std::string* new_value, - Logger* logger) const override; + bool FullMergeV2(const MergeOperationInput& merge_in, + MergeOperationOutput* merge_out) const override; - virtual bool PartialMerge(const Slice& key, - const Slice& left_operand, - const Slice& right_operand, - std::string* new_value, - Logger* logger) const override; + bool PartialMerge(const Slice& key, const Slice& left_operand, + const Slice& right_operand, std::string* new_value, + Logger* logger) const override; }; } // namespace rocksdb diff --git a/include/rocksdb/metadata.h b/include/rocksdb/metadata.h new file mode 100644 index 00000000000..37e7b50b9b7 --- /dev/null +++ b/include/rocksdb/metadata.h @@ -0,0 +1,94 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include + +#include +#include +#include + +#include "rocksdb/types.h" + +namespace rocksdb { +struct ColumnFamilyMetaData; +struct LevelMetaData; +struct SstFileMetaData; + +// The metadata that describes a column family. +struct ColumnFamilyMetaData { + ColumnFamilyMetaData() : size(0), name("") {} + ColumnFamilyMetaData(const std::string& _name, uint64_t _size, + const std::vector&& _levels) : + size(_size), name(_name), levels(_levels) {} + + // The size of this column family in bytes, which is equal to the sum of + // the file size of its "levels". + uint64_t size; + // The number of files in this column family. + size_t file_count; + // The name of the column family. + std::string name; + // The metadata of all levels in this column family. + std::vector levels; +}; + +// The metadata that describes a level. +struct LevelMetaData { + LevelMetaData(int _level, uint64_t _size, + const std::vector&& _files) : + level(_level), size(_size), + files(_files) {} + + // The level which this meta data describes. + const int level; + // The size of this level in bytes, which is equal to the sum of + // the file size of its "files". + const uint64_t size; + // The metadata of all sst files in this level. + const std::vector files; +}; + +// The metadata that describes a SST file. +struct SstFileMetaData { + SstFileMetaData() {} + SstFileMetaData(const std::string& _file_name, const std::string& _path, + uint64_t _size, SequenceNumber _smallest_seqno, + SequenceNumber _largest_seqno, + const std::string& _smallestkey, + const std::string& _largestkey, uint64_t _num_reads_sampled, + bool _being_compacted) + : size(_size), + name(_file_name), + db_path(_path), + smallest_seqno(_smallest_seqno), + largest_seqno(_largest_seqno), + smallestkey(_smallestkey), + largestkey(_largestkey), + num_reads_sampled(_num_reads_sampled), + being_compacted(_being_compacted) {} + + // File size in bytes. + uint64_t size; + // The name of the file. + std::string name; + // The full path where the file locates. + std::string db_path; + + SequenceNumber smallest_seqno; // Smallest sequence number in file. + SequenceNumber largest_seqno; // Largest sequence number in file. + std::string smallestkey; // Smallest user defined key in the file. + std::string largestkey; // Largest user defined key in the file. + uint64_t num_reads_sampled; // How many times the file is read. + bool being_compacted; // true if the file is currently being compacted. +}; + +// The full set of metadata associated with each SST file. +struct LiveFileMetaData : SstFileMetaData { + std::string column_family_name; // Name of the column family + int level; // Level at which this file resides. +}; +} // namespace rocksdb diff --git a/include/rocksdb/options.h b/include/rocksdb/options.h index 11d976fb2b0..4d2f143a0f7 100644 --- a/include/rocksdb/options.h +++ b/include/rocksdb/options.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -10,93 +10,89 @@ #define STORAGE_ROCKSDB_INCLUDE_OPTIONS_H_ #include +#include #include #include #include -#include +#include +#include -#include "rocksdb/version.h" +#include "rocksdb/advanced_options.h" +#include "rocksdb/comparator.h" +#include "rocksdb/env.h" +#include "rocksdb/listener.h" #include "rocksdb/universal_compaction.h" +#include "rocksdb/version.h" +#include "rocksdb/write_buffer_manager.h" + +#ifdef max +#undef max +#endif namespace rocksdb { class Cache; class CompactionFilter; class CompactionFilterFactory; -class CompactionFilterFactoryV2; class Comparator; class Env; enum InfoLogLevel : unsigned char; +class SstFileManager; class FilterPolicy; class Logger; class MergeOperator; class Snapshot; -class TableFactory; class MemTableRepFactory; -class TablePropertiesCollectorFactory; class RateLimiter; class Slice; -class SliceTransform; class Statistics; class InternalKeyComparator; +class WalFilter; // DB contents are stored in a set of blocks, each of which holds a // sequence of key,value pairs. Each block may be compressed before // being stored in a file. The following enum describes which // compression method (if any) is used to compress a block. -enum CompressionType : char { +enum CompressionType : unsigned char { // NOTE: do not change the values of existing entries, as these are // part of the persistent format on disk. - kNoCompression = 0x0, kSnappyCompression = 0x1, kZlibCompression = 0x2, - kBZip2Compression = 0x3, kLZ4Compression = 0x4, kLZ4HCCompression = 0x5 -}; - -enum CompactionStyle : char { - kCompactionStyleLevel = 0x0, // level based compaction style - kCompactionStyleUniversal = 0x1, // Universal compaction style - kCompactionStyleFIFO = 0x2, // FIFO compaction style -}; - -struct CompactionOptionsFIFO { - // once the total sum of table files reaches this, we will delete the oldest - // table file - // Default: 1GB - uint64_t max_table_files_size; - - CompactionOptionsFIFO() : max_table_files_size(1 * 1024 * 1024 * 1024) {} -}; - -// Compression options for different compression algorithms like Zlib -struct CompressionOptions { - int window_bits; - int level; - int strategy; - CompressionOptions() : window_bits(-14), level(-1), strategy(0) {} - CompressionOptions(int wbits, int _lev, int _strategy) - : window_bits(wbits), level(_lev), strategy(_strategy) {} -}; - -enum UpdateStatus { // Return status For inplace update callback - UPDATE_FAILED = 0, // Nothing to update - UPDATED_INPLACE = 1, // Value updated inplace - UPDATED = 2, // No inplace update. Merged value set -}; - -struct DbPath { - std::string path; - uint64_t target_size; // Target size of total files under the path, in byte. - - DbPath() : target_size(0) {} - DbPath(const std::string& p, uint64_t t) : path(p), target_size(t) {} + kNoCompression = 0x0, + kSnappyCompression = 0x1, + kZlibCompression = 0x2, + kBZip2Compression = 0x3, + kLZ4Compression = 0x4, + kLZ4HCCompression = 0x5, + kXpressCompression = 0x6, + kZSTD = 0x7, + + // Only use kZSTDNotFinalCompression if you have to use ZSTD lib older than + // 0.8.0 or consider a possibility of downgrading the service or copying + // the database files to another service running with an older version of + // RocksDB that doesn't have kZSTD. Otherwise, you should use kZSTD. We will + // eventually remove the option from the public API. + kZSTDNotFinalCompression = 0x40, + + // kDisableCompressionOption is used to disable some compression options. + kDisableCompressionOption = 0xff, }; struct Options; -struct ColumnFamilyOptions { +struct ColumnFamilyOptions : public AdvancedColumnFamilyOptions { + // The function recovers options to a previous version. Only 4.6 or later + // versions are supported. + ColumnFamilyOptions* OldDefaults(int rocksdb_major_version = 4, + int rocksdb_minor_version = 6); + // Some functions that make it easier to optimize RocksDB + // Use this if your DB is very small (like under 1GB) and you don't want to + // spend lots of memory for memtables. + ColumnFamilyOptions* OptimizeForSmallDb(); // Use this if you don't need to keep the data sorted, i.e. you'll never use // an iterator, only Put() and Get() API calls + // + // Not supported in ROCKSDB_LITE ColumnFamilyOptions* OptimizeForPointLookup( uint64_t block_cache_size_mb); @@ -114,6 +110,8 @@ struct ColumnFamilyOptions { // biggest performance gains. // Note: we might use more memory than memtable_memory_budget during high // write rate period + // + // OptimizeUniversalStyleCompaction is not supported in ROCKSDB_LITE ColumnFamilyOptions* OptimizeLevelStyleCompaction( uint64_t memtable_memory_budget = 512 * 1024 * 1024); ColumnFamilyOptions* OptimizeUniversalStyleCompaction( @@ -128,7 +126,7 @@ struct ColumnFamilyOptions { // REQUIRES: The client must ensure that the comparator supplied // here has the same name and orders keys *exactly* the same as the // comparator provided to previous open calls on the same DB. - const Comparator* comparator; + const Comparator* comparator = BytewiseComparator(); // REQUIRES: The client must provide a merge operator if Merge operation // needs to be accessed. Calling Merge on a DB without a merge operator @@ -138,9 +136,9 @@ struct ColumnFamilyOptions { // the same DB. The only exception is reserved for upgrade, where a DB // previously without a merge operator is introduced to Merge operation // for the first time. It's necessary to specify a merge operator when - // openning the DB in this case. + // opening the DB in this case. // Default: nullptr - std::shared_ptr merge_operator; + std::shared_ptr merge_operator = nullptr; // A single CompactionFilter instance to call into during compaction. // Allows an application to modify/delete a key-value during background @@ -157,7 +155,7 @@ struct ColumnFamilyOptions { // thread-safe. // // Default: nullptr - const CompactionFilter* compaction_filter; + const CompactionFilter* compaction_filter = nullptr; // This is a factory that provides compaction filter objects which allow // an application to modify/delete a key-value during background compaction. @@ -166,14 +164,8 @@ struct ColumnFamilyOptions { // compaction is being used, each created CompactionFilter will only be used // from a single thread and so does not need to be thread-safe. // - // Default: a factory that doesn't provide any object - std::shared_ptr compaction_filter_factory; - - // Version TWO of the compaction_filter_factory - // It supports rolling compaction - // - // Default: a factory that doesn't provide any object - std::shared_ptr compaction_filter_factory_v2; + // Default: nullptr + std::shared_ptr compaction_filter_factory = nullptr; // ------------------- // Parameters that affect performance @@ -188,30 +180,19 @@ struct ColumnFamilyOptions { // Also, a larger write buffer will result in a longer recovery time // the next time the database is opened. // - // Default: 4MB - size_t write_buffer_size; - - // The maximum number of write buffers that are built up in memory. - // The default and the minimum number is 2, so that when 1 write buffer - // is being flushed to storage, new writes can continue to the other - // write buffer. - // Default: 2 - int max_write_buffer_number; - - // The minimum number of write buffers that will be merged together - // before writing to storage. If set to 1, then - // all write buffers are fushed to L0 as individual files and this increases - // read amplification because a get request has to check in all of these - // files. Also, an in-memory merge may result in writing lesser - // data to storage if there are duplicate records in each of these - // individual write buffers. Default: 1 - int min_write_buffer_number_to_merge; + // Note that write_buffer_size is enforced per column family. + // See db_write_buffer_size for sharing memory across column families. + // + // Default: 64MB + // + // Dynamically changeable through SetOptions() API + size_t write_buffer_size = 64 << 20; // Compress blocks using the specified compression algorithm. This // parameter can be changed dynamically. // - // Default: kSnappyCompression, which gives lightweight but fast - // compression. + // Default: kSnappyCompression, if it's supported. If snappy is not linked + // with the library, the default is kNoCompression. // // Typical speeds of kSnappyCompression on an Intel(R) Core(TM)2 2.4GHz: // ~200-500MB/s compression @@ -223,23 +204,24 @@ struct ColumnFamilyOptions { // efficiently detect that and will switch to uncompressed mode. CompressionType compression; - // Different levels can have different compression policies. There - // are cases where most lower levels would like to quick compression - // algorithm while the higher levels (which have more data) use - // compression algorithms that have better compression but could - // be slower. This array, if non nullptr, should have an entry for - // each level of the database. This array, if non nullptr, overides the - // value specified in the previous field 'compression'. The caller is - // reponsible for allocating memory and initializing the values in it - // before invoking Open(). The caller is responsible for freeing this - // array and it could be freed anytime after the return from Open(). - // This could have been a std::vector but that makes the equivalent - // java/C api hard to construct. - std::vector compression_per_level; + // Compression algorithm that will be used for the bottommost level that + // contain files. If level-compaction is used, this option will only affect + // levels after base level. + // + // Default: kDisableCompressionOption (Disabled) + CompressionType bottommost_compression = kDisableCompressionOption; // different options for compression algorithms CompressionOptions compression_opts; + // Number of files to trigger level-0 compaction. A value <0 means that + // level-0 compaction will not be triggered by number of files at all. + // + // Default: 4 + // + // Dynamically changeable through SetOptions() API + int level0_file_num_compaction_trigger = 4; + // If non-nullptr, use the specified function to determine the // prefixes for keys. These prefixes will be placed in the filter. // Depending on the workload, this can reduce the number of read-IOP @@ -254,159 +236,27 @@ struct ColumnFamilyOptions { // 4) prefix(prefix(key)) == prefix(key) // // Default: nullptr - std::shared_ptr prefix_extractor; - - // Number of levels for this database - int num_levels; - - // Number of files to trigger level-0 compaction. A value <0 means that - // level-0 compaction will not be triggered by number of files at all. - // - // Default: 4 - int level0_file_num_compaction_trigger; - - // Soft limit on number of level-0 files. We start slowing down writes at this - // point. A value <0 means that no writing slow down will be triggered by - // number of files in level-0. - int level0_slowdown_writes_trigger; - - // Maximum number of level-0 files. We stop writes at this point. - int level0_stop_writes_trigger; - - // Maximum level to which a new compacted memtable is pushed if it - // does not create overlap. We try to push to level 2 to avoid the - // relatively expensive level 0=>1 compactions and to avoid some - // expensive manifest file operations. We do not push all the way to - // the largest level since that can generate a lot of wasted disk - // space if the same key space is being repeatedly overwritten. - int max_mem_compaction_level; - - // Target file size for compaction. - // target_file_size_base is per-file size for level-1. - // Target file size for level L can be calculated by - // target_file_size_base * (target_file_size_multiplier ^ (L-1)) - // For example, if target_file_size_base is 2MB and - // target_file_size_multiplier is 10, then each file on level-1 will - // be 2MB, and each file on level 2 will be 20MB, - // and each file on level-3 will be 200MB. - - // by default target_file_size_base is 2MB. - int target_file_size_base; - // by default target_file_size_multiplier is 1, which means - // by default files in different levels will have similar size. - int target_file_size_multiplier; + std::shared_ptr prefix_extractor = nullptr; // Control maximum total data size for a level. // max_bytes_for_level_base is the max total for level-1. // Maximum number of bytes for level L can be calculated as // (max_bytes_for_level_base) * (max_bytes_for_level_multiplier ^ (L-1)) - // For example, if max_bytes_for_level_base is 20MB, and if + // For example, if max_bytes_for_level_base is 200MB, and if // max_bytes_for_level_multiplier is 10, total data size for level-1 - // will be 20MB, total file size for level-2 will be 200MB, - // and total file size for level-3 will be 2GB. - - // by default 'max_bytes_for_level_base' is 10MB. - uint64_t max_bytes_for_level_base; - // by default 'max_bytes_for_level_base' is 10. - int max_bytes_for_level_multiplier; - - // Different max-size multipliers for different levels. - // These are multiplied by max_bytes_for_level_multiplier to arrive - // at the max-size of each level. - // Default: 1 - std::vector max_bytes_for_level_multiplier_additional; - - // Maximum number of bytes in all compacted files. We avoid expanding - // the lower level file set of a compaction if it would make the - // total compaction cover more than - // (expanded_compaction_factor * targetFileSizeLevel()) many bytes. - int expanded_compaction_factor; - - // Maximum number of bytes in all source files to be compacted in a - // single compaction run. We avoid picking too many files in the - // source level so that we do not exceed the total source bytes - // for compaction to exceed - // (source_compaction_factor * targetFileSizeLevel()) many bytes. - // Default:1, i.e. pick maxfilesize amount of data as the source of - // a compaction. - int source_compaction_factor; - - // Control maximum bytes of overlaps in grandparent (i.e., level+2) before we - // stop building a single file in a level->level+1 compaction. - int max_grandparent_overlap_factor; - - // Puts are delayed 0-1 ms when any level has a compaction score that exceeds - // soft_rate_limit. This is ignored when == 0.0. - // CONSTRAINT: soft_rate_limit <= hard_rate_limit. If this constraint does not - // hold, RocksDB will set soft_rate_limit = hard_rate_limit - // Default: 0 (disabled) - double soft_rate_limit; - - // Puts are delayed 1ms at a time when any level has a compaction score that - // exceeds hard_rate_limit. This is ignored when <= 1.0. - // Default: 0 (disabled) - double hard_rate_limit; - - // Max time a put will be stalled when hard_rate_limit is enforced. If 0, then - // there is no limit. - // Default: 1000 - unsigned int rate_limit_delay_max_milliseconds; - - // size of one block in arena memory allocation. - // If <= 0, a proper value is automatically calculated (usually 1/10 of - // writer_buffer_size). + // will be 200MB, total file size for level-2 will be 2GB, + // and total file size for level-3 will be 20GB. // - // There are two additonal restriction of the The specified size: - // (1) size should be in the range of [4096, 2 << 30] and - // (2) be the multiple of the CPU word (which helps with the memory - // alignment). + // Default: 256MB. // - // We'll automatically check and adjust the size number to make sure it - // conforms to the restrictions. - // - // Default: 0 - size_t arena_block_size; + // Dynamically changeable through SetOptions() API + uint64_t max_bytes_for_level_base = 256 * 1048576; // Disable automatic compactions. Manual compactions can still // be issued on this column family - bool disable_auto_compactions; - - // Purge duplicate/deleted keys when a memtable is flushed to storage. - // Default: true - bool purge_redundant_kvs_while_flush; - - // The compaction style. Default: kCompactionStyleLevel - CompactionStyle compaction_style; - - // If true, compaction will verify checksum on every read that happens - // as part of compaction - // Default: true - bool verify_checksums_in_compaction; - - // The options needed to support Universal Style compactions - CompactionOptionsUniversal compaction_options_universal; - - // The options for FIFO compaction style - CompactionOptionsFIFO compaction_options_fifo; - - // Use KeyMayExist API to filter deletes when this is true. - // If KeyMayExist returns false, i.e. the key definitely does not exist, then - // the delete is a noop. KeyMayExist only incurs in-memory look up. - // This optimization avoids writing the delete to storage when appropriate. - // Default: false - bool filter_deletes; - - // An iteration->Next() sequentially skips over keys with the same - // user-key unless this option is set. This number specifies the number - // of keys (with the same userkey) that will be sequentially - // skipped before a reseek is issued. - // Default: 8 - uint64_t max_sequential_skip_in_iterations; - - // This is a factory that provides MemTableRep objects. - // Default: a factory that provides a skip-list-based implementation of - // MemTableRep. - std::shared_ptr memtable_factory; + // + // Dynamically changeable through SetOptions() API + bool disable_auto_compactions = false; // This is a factory that provides TableFactory objects. // Default: a block-based table factory that provides a default @@ -414,128 +264,6 @@ struct ColumnFamilyOptions { // BlockBasedTableOptions. std::shared_ptr table_factory; - // Block-based table related options are moved to BlockBasedTableOptions. - // Related options that were originally here but now moved include: - // no_block_cache - // block_cache - // block_cache_compressed - // block_size - // block_size_deviation - // block_restart_interval - // filter_policy - // whole_key_filtering - // If you'd like to customize some of these options, you will need to - // use NewBlockBasedTableFactory() to construct a new table factory. - - // This option allows user to to collect their own interested statistics of - // the tables. - // Default: empty vector -- no user-defined statistics collection will be - // performed. - typedef std::vector> - TablePropertiesCollectorFactories; - TablePropertiesCollectorFactories table_properties_collector_factories; - - // Allows thread-safe inplace updates. If this is true, there is no way to - // achieve point-in-time consistency using snapshot or iterator (assuming - // concurrent updates). - // If inplace_callback function is not set, - // Put(key, new_value) will update inplace the existing_value iff - // * key exists in current memtable - // * new sizeof(new_value) <= sizeof(existing_value) - // * existing_value for that key is a put i.e. kTypeValue - // If inplace_callback function is set, check doc for inplace_callback. - // Default: false. - bool inplace_update_support; - - // Number of locks used for inplace update - // Default: 10000, if inplace_update_support = true, else 0. - size_t inplace_update_num_locks; - - // existing_value - pointer to previous value (from both memtable and sst). - // nullptr if key doesn't exist - // existing_value_size - pointer to size of existing_value). - // nullptr if key doesn't exist - // delta_value - Delta value to be merged with the existing_value. - // Stored in transaction logs. - // merged_value - Set when delta is applied on the previous value. - - // Applicable only when inplace_update_support is true, - // this callback function is called at the time of updating the memtable - // as part of a Put operation, lets say Put(key, delta_value). It allows the - // 'delta_value' specified as part of the Put operation to be merged with - // an 'existing_value' of the key in the database. - - // If the merged value is smaller in size that the 'existing_value', - // then this function can update the 'existing_value' buffer inplace and - // the corresponding 'existing_value'_size pointer, if it wishes to. - // The callback should return UpdateStatus::UPDATED_INPLACE. - // In this case. (In this case, the snapshot-semantics of the rocksdb - // Iterator is not atomic anymore). - - // If the merged value is larger in size than the 'existing_value' or the - // application does not wish to modify the 'existing_value' buffer inplace, - // then the merged value should be returned via *merge_value. It is set by - // merging the 'existing_value' and the Put 'delta_value'. The callback should - // return UpdateStatus::UPDATED in this case. This merged value will be added - // to the memtable. - - // If merging fails or the application does not wish to take any action, - // then the callback should return UpdateStatus::UPDATE_FAILED. - - // Please remember that the original call from the application is Put(key, - // delta_value). So the transaction log (if enabled) will still contain (key, - // delta_value). The 'merged_value' is not stored in the transaction log. - // Hence the inplace_callback function should be consistent across db reopens. - - // Default: nullptr - UpdateStatus (*inplace_callback)(char* existing_value, - uint32_t* existing_value_size, - Slice delta_value, - std::string* merged_value); - - // if prefix_extractor is set and bloom_bits is not 0, create prefix bloom - // for memtable - uint32_t memtable_prefix_bloom_bits; - - // number of hash probes per key - uint32_t memtable_prefix_bloom_probes; - - // Page size for huge page TLB for bloom in memtable. If <=0, not allocate - // from huge page TLB but from malloc. - // Need to reserve huge pages for it to be allocated. For example: - // sysctl -w vm.nr_hugepages=20 - // See linux doc Documentation/vm/hugetlbpage.txt - - size_t memtable_prefix_bloom_huge_page_tlb_size; - - // Control locality of bloom filter probes to improve cache miss rate. - // This option only applies to memtable prefix bloom and plaintable - // prefix bloom. It essentially limits every bloom checking to one cache line. - // This optimization is turned off when set to 0, and positive number to turn - // it on. - // Default: 0 - uint32_t bloom_locality; - - // Maximum number of successive merge operations on a key in the memtable. - // - // When a merge operation is added to the memtable and the maximum number of - // successive merges is reached, the value of the key will be calculated and - // inserted into the memtable instead of the merge operation. This will - // ensure that there are never more than max_successive_merges merge - // operations in the memtable. - // - // Default: 0 (disabled) - size_t max_successive_merges; - - // The number of partial merge operands to accumulate before partial - // merge will be performed. Partial merge will not be called - // if the list of values to merge is less than min_partial_merge_operands. - // - // If min_partial_merge_operands < 2, then it will be treated as 2. - // - // Default: 2 - uint32_t min_partial_merge_operands; - // Create ColumnFamilyOptions with default values for all fields ColumnFamilyOptions(); // Create ColumnFamilyOptions from Options @@ -544,93 +272,149 @@ struct ColumnFamilyOptions { void Dump(Logger* log) const; }; +enum class WALRecoveryMode : char { + // Original levelDB recovery + // We tolerate incomplete record in trailing data on all logs + // Use case : This is legacy behavior (default) + kTolerateCorruptedTailRecords = 0x00, + // Recover from clean shutdown + // We don't expect to find any corruption in the WAL + // Use case : This is ideal for unit tests and rare applications that + // can require high consistency guarantee + kAbsoluteConsistency = 0x01, + // Recover to point-in-time consistency + // We stop the WAL playback on discovering WAL inconsistency + // Use case : Ideal for systems that have disk controller cache like + // hard disk, SSD without super capacitor that store related data + kPointInTimeRecovery = 0x02, + // Recovery after a disaster + // We ignore any corruption in the WAL and try to salvage as much data as + // possible + // Use case : Ideal for last ditch effort to recover data or systems that + // operate with low grade unrelated data + kSkipAnyCorruptedRecords = 0x03, +}; + +struct DbPath { + std::string path; + uint64_t target_size; // Target size of total files under the path, in byte. + + DbPath() : target_size(0) {} + DbPath(const std::string& p, uint64_t t) : path(p), target_size(t) {} +}; + + struct DBOptions { + // The function recovers options to the option as in version 4.6. + DBOptions* OldDefaults(int rocksdb_major_version = 4, + int rocksdb_minor_version = 6); + // Some functions that make it easier to optimize RocksDB + // Use this if your DB is very small (like under 1GB) and you don't want to + // spend lots of memory for memtables. + DBOptions* OptimizeForSmallDb(); + +#ifndef ROCKSDB_LITE // By default, RocksDB uses only one background thread for flush and // compaction. Calling this function will set it up such that total of // `total_threads` is used. Good value for `total_threads` is the number of // cores. You almost definitely want to call this function if your system is // bottlenecked by RocksDB. DBOptions* IncreaseParallelism(int total_threads = 16); +#endif // ROCKSDB_LITE // If true, the database will be created if it is missing. // Default: false - bool create_if_missing; + bool create_if_missing = false; // If true, missing column families will be automatically created. // Default: false - bool create_missing_column_families; + bool create_missing_column_families = false; // If true, an error is raised if the database already exists. // Default: false - bool error_if_exists; - - // If true, the implementation will do aggressive checking of the - // data it is processing and will stop early if it detects any - // errors. This may have unforeseen ramifications: for example, a - // corruption of one DB entry may cause a large number of entries to - // become unreadable or for the entire DB to become unopenable. - // If any of the writes to the database fails (Put, Delete, Merge, Write), - // the database will switch to read-only mode and fail all other + bool error_if_exists = false; + + // If true, RocksDB will aggressively check consistency of the data. + // Also, if any of the writes to the database fails (Put, Delete, Merge, + // Write), the database will switch to read-only mode and fail all other // Write operations. + // In most cases you want this to be set to true. // Default: true - bool paranoid_checks; + bool paranoid_checks = true; // Use the specified object to interact with the environment, // e.g. to read/write files, schedule background work, etc. // Default: Env::Default() - Env* env; + Env* env = Env::Default(); // Use to control write rate of flush and compaction. Flush has higher // priority than compaction. Rate limiting is disabled if nullptr. // If rate limiter is enabled, bytes_per_sync is set to 1MB by default. // Default: nullptr - std::shared_ptr rate_limiter; + std::shared_ptr rate_limiter = nullptr; + + // Use to track SST files and control their file deletion rate. + // + // Features: + // - Throttle the deletion rate of the SST files. + // - Keep track the total size of all SST files. + // - Set a maximum allowed space limit for SST files that when reached + // the DB wont do any further flushes or compactions and will set the + // background error. + // - Can be shared between multiple dbs. + // Limitations: + // - Only track and throttle deletes of SST files in + // first db_path (db_name if db_paths is empty). + // + // Default: nullptr + std::shared_ptr sst_file_manager = nullptr; // Any internal progress/error information generated by the db will // be written to info_log if it is non-nullptr, or to a file stored // in the same directory as the DB contents if info_log is nullptr. // Default: nullptr - std::shared_ptr info_log; + std::shared_ptr info_log = nullptr; - InfoLogLevel info_log_level; +#ifdef NDEBUG + InfoLogLevel info_log_level = INFO_LEVEL; +#else + InfoLogLevel info_log_level = DEBUG_LEVEL; +#endif // NDEBUG // Number of open files that can be used by the DB. You may need to // increase this if your database has a large working set. Value -1 means // files opened are always kept open. You can estimate number of files based // on target_file_size_base and target_file_size_multiplier for level-based // compaction. For universal-style compaction, you can usually set it to -1. - // Default: 5000 - int max_open_files; + // Default: -1 + int max_open_files = -1; + + // If max_open_files is -1, DB will open all files on DB::Open(). You can + // use this option to increase the number of threads used to open the files. + // Default: 16 + int max_file_opening_threads = 16; // Once write-ahead logs exceed this size, we will start forcing the flush of // column families whose memtables are backed by the oldest live WAL file // (i.e. the ones that are causing all the space amplification). If set to 0 // (default), we will dynamically choose the WAL size limit to be - // [sum of all write_buffer_size * max_write_buffer_number] * 2 + // [sum of all write_buffer_size * max_write_buffer_number] * 4 // Default: 0 - uint64_t max_total_wal_size; + uint64_t max_total_wal_size = 0; // If non-null, then we should collect metrics about database operations - // Statistics objects should not be shared between DB instances as - // it does not use any locks to prevent concurrent updates. - std::shared_ptr statistics; - - // If true, then the contents of data files are not synced - // to stable storage. Their contents remain in the OS buffers till the - // OS decides to flush them. This option is good for bulk-loading - // of data. Once the bulk-loading is complete, please issue a - // sync to the OS to flush all dirty buffesrs to stable storage. - // Default: false - bool disableDataSync; + std::shared_ptr statistics = nullptr; // If true, then every store to stable storage will issue a fsync. // If false, then every store to stable storage will issue a fdatasync. // This parameter should be set to true while storing data to // filesystem like ext3 that can lose files after a reboot. // Default: false - bool use_fsync; + // Note: on many platforms fdatasync is defined as fsync, so this parameter + // would make no difference. Refer to fdatasync definition in this code base. + bool use_fsync = false; // A list of paths where SST files can be put into, with its target size. // Newer data is placed into paths specified earlier in the vector while @@ -650,7 +434,7 @@ struct DBOptions { // If none of the paths has sufficient room to place a file, the file will // be placed to the last path anyway, despite to the target size. // - // Placing newer data to ealier paths is also best-efforts. User should + // Placing newer data to earlier paths is also best-efforts. User should // expect user files to be placed in higher levels in some extreme cases. // // If left empty, only one path will be used, which is db_name passed when @@ -663,7 +447,7 @@ struct DBOptions { // If it is non empty, the log files will be in the specified dir, // and the db data dir's absolute path will be used as the log file // name's prefix. - std::string db_log_dir; + std::string db_log_dir = ""; // This specifies the absolute dir path for write-ahead logs (WAL). // If it is empty, the log files will be in the same dir as data, @@ -671,72 +455,102 @@ struct DBOptions { // If it is non empty, the log files will be in kept the specified dir. // When destroying the db, // all log files in wal_dir and the dir itself is deleted - std::string wal_dir; + std::string wal_dir = ""; // The periodicity when obsolete files get deleted. The default // value is 6 hours. The files that get out of scope by compaction // process will still get automatically delete on every compaction, // regardless of this setting - uint64_t delete_obsolete_files_period_micros; + uint64_t delete_obsolete_files_period_micros = 6ULL * 60 * 60 * 1000000; + + // Maximum number of concurrent background jobs (compactions and flushes). + int max_background_jobs = 2; + // NOT SUPPORTED ANYMORE: RocksDB automatically decides this based on the + // value of max_background_jobs. This option is ignored. + int base_background_compactions = -1; + + // NOT SUPPORTED ANYMORE: RocksDB automatically decides this based on the + // value of max_background_jobs. For backwards compatibility we will set + // `max_background_jobs = max_background_compactions + max_background_flushes` + // in the case where user sets at least one of `max_background_compactions` or + // `max_background_flushes` (we replace -1 by 1 in case one option is unset). + // // Maximum number of concurrent background compaction jobs, submitted to // the default LOW priority thread pool. + // // If you're increasing this, also consider increasing number of threads in // LOW priority thread pool. For more information, see // Env::SetBackgroundThreads - // Default: 1 - int max_background_compactions; - - // Maximum number of concurrent background memtable flush jobs, submitted to - // the HIGH priority thread pool. + // Default: -1 + int max_background_compactions = -1; + + // This value represents the maximum number of threads that will + // concurrently perform a compaction job by breaking it into multiple, + // smaller ones that are run simultaneously. + // Default: 1 (i.e. no subcompactions) + uint32_t max_subcompactions = 1; + + // NOT SUPPORTED ANYMORE: RocksDB automatically decides this based on the + // value of max_background_jobs. For backwards compatibility we will set + // `max_background_jobs = max_background_compactions + max_background_flushes` + // in the case where user sets at least one of `max_background_compactions` or + // `max_background_flushes`. // - // By default, all background jobs (major compaction and memtable flush) go - // to the LOW priority pool. If this option is set to a positive number, - // memtable flush jobs will be submitted to the HIGH priority pool. - // It is important when the same Env is shared by multiple db instances. - // Without a separate pool, long running major compaction jobs could - // potentially block memtable flush jobs of other db instances, leading to - // unnecessary Put stalls. + // Maximum number of concurrent background memtable flush jobs, submitted by + // default to the HIGH priority thread pool. If the HIGH priority thread pool + // is configured to have zero threads, flush jobs will share the LOW priority + // thread pool with compaction jobs. + // + // It is important to use both thread pools when the same Env is shared by + // multiple db instances. Without a separate pool, long running compaction + // jobs could potentially block memtable flush jobs of other db instances, + // leading to unnecessary Put stalls. // // If you're increasing this, also consider increasing number of threads in // HIGH priority thread pool. For more information, see // Env::SetBackgroundThreads - // Default: 1 - int max_background_flushes; + // Default: -1 + int max_background_flushes = -1; // Specify the maximal size of the info log file. If the log file // is larger than `max_log_file_size`, a new info log file will // be created. // If max_log_file_size == 0, all logs will be written to one // log file. - size_t max_log_file_size; + size_t max_log_file_size = 0; // Time for the info log file to roll (in seconds). // If specified with non-zero value, log file will be rolled // if it has been active longer than `log_file_time_to_roll`. // Default: 0 (disabled) - size_t log_file_time_to_roll; + // Not supported in ROCKSDB_LITE mode! + size_t log_file_time_to_roll = 0; // Maximal info log files to be kept. // Default: 1000 - size_t keep_log_file_num; + size_t keep_log_file_num = 1000; + + // Recycle log files. + // If non-zero, we will reuse previously written log files for new + // logs, overwriting the old data. The value indicates how many + // such files we will keep around at any point in time for later + // use. This is more efficient because the blocks are already + // allocated and fdatasync does not need to update the inode after + // each write. + // Default: 0 + size_t recycle_log_file_num = 0; // manifest file is rolled over on reaching this limit. // The older manifest file be deleted. // The default value is MAX_INT so that roll-over does not take place. - uint64_t max_manifest_file_size; + uint64_t max_manifest_file_size = std::numeric_limits::max(); // Number of shards used for table cache. - int table_cache_numshardbits; + int table_cache_numshardbits = 6; - // During data eviction of table's LRU cache, it would be inefficient - // to strictly follow LRU because this piece of memory will not really - // be released unless its refcount falls to zero. Instead, make two - // passes: the first pass will release items with refcount = 1, - // and if not enough space releases after scanning the number of - // elements specified by this parameter, we will remove items in LRU - // order. - int table_cache_remove_scan_count_limit; + // NOT SUPPORTED ANYMORE + // int table_cache_remove_scan_count_limit; // The following two fields affect how archived logs will be deleted. // 1. If both set to 0, logs will be deleted asap and will not get into @@ -750,62 +564,154 @@ struct DBOptions { // are older than WAL_ttl_seconds will be deleted. // 4. If both are not 0, WAL files will be checked every 10 min and both // checks will be performed with ttl being first. - uint64_t WAL_ttl_seconds; - uint64_t WAL_size_limit_MB; + uint64_t WAL_ttl_seconds = 0; + uint64_t WAL_size_limit_MB = 0; // Number of bytes to preallocate (via fallocate) the manifest // files. Default is 4mb, which is reasonable to reduce random IO // as well as prevent overallocation for mounts that preallocate // large amounts of data (such as xfs's allocsize option). - size_t manifest_preallocation_size; - - // Data being read from file storage may be buffered in the OS - // Default: true - bool allow_os_buffer; + size_t manifest_preallocation_size = 4 * 1024 * 1024; // Allow the OS to mmap file for reading sst tables. Default: false - bool allow_mmap_reads; + bool allow_mmap_reads = false; - // Allow the OS to mmap file for writing. Default: false - bool allow_mmap_writes; + // Allow the OS to mmap file for writing. + // DB::SyncWAL() only works if this is set to false. + // Default: false + bool allow_mmap_writes = false; - // Disable child process inherit open files. Default: true - bool is_fd_close_on_exec; + // Enable direct I/O mode for read/write + // they may or may not improve performance depending on the use case + // + // Files will be opened in "direct I/O" mode + // which means that data r/w from the disk will not be cached or + // buffered. The hardware buffer of the devices may however still + // be used. Memory mapped files are not impacted by these parameters. - // Skip log corruption error on recovery (If client is ok with - // losing most recent changes) + // Use O_DIRECT for user reads // Default: false - bool skip_log_error_on_recovery; + // Not supported in ROCKSDB_LITE mode! + bool use_direct_reads = false; + + // Use O_DIRECT for both reads and writes in background flush and compactions + // When true, we also force new_table_reader_for_compaction_inputs to true. + // Default: false + // Not supported in ROCKSDB_LITE mode! + bool use_direct_io_for_flush_and_compaction = false; + + // If false, fallocate() calls are bypassed + bool allow_fallocate = true; + + // Disable child process inherit open files. Default: true + bool is_fd_close_on_exec = true; + + // NOT SUPPORTED ANYMORE -- this options is no longer used + bool skip_log_error_on_recovery = false; // if not zero, dump rocksdb.stats to LOG every stats_dump_period_sec - // Default: 3600 (1 hour) - unsigned int stats_dump_period_sec; + // Default: 600 (10 min) + unsigned int stats_dump_period_sec = 600; // If set true, will hint the underlying file system that the file // access pattern is random, when a sst file is opened. // Default: true - bool advise_random_on_open; + bool advise_random_on_open = true; + + // Amount of data to build up in memtables across all column + // families before writing to disk. + // + // This is distinct from write_buffer_size, which enforces a limit + // for a single memtable. + // + // This feature is disabled by default. Specify a non-zero value + // to enable it. + // + // Default: 0 (disabled) + size_t db_write_buffer_size = 0; + + // The memory usage of memtable will report to this object. The same object + // can be passed into multiple DBs and it will track the sum of size of all + // the DBs. If the total size of all live memtables of all the DBs exceeds + // a limit, a flush will be triggered in the next DB to which the next write + // is issued. + // + // If the object is only passed to on DB, the behavior is the same as + // db_write_buffer_size. When write_buffer_manager is set, the value set will + // override db_write_buffer_size. + // + // This feature is disabled by default. Specify a non-zero value + // to enable it. + // + // Default: null + std::shared_ptr write_buffer_manager = nullptr; // Specify the file access pattern once a compaction is started. // It will be applied to all input files of a compaction. // Default: NORMAL - enum { - NONE, - NORMAL, - SEQUENTIAL, - WILLNEED - } access_hint_on_compaction_start; + enum AccessHint { + NONE, + NORMAL, + SEQUENTIAL, + WILLNEED + }; + AccessHint access_hint_on_compaction_start = NORMAL; + + // If true, always create a new file descriptor and new table reader + // for compaction inputs. Turn this parameter on may introduce extra + // memory usage in the table reader, if it allocates extra memory + // for indexes. This will allow file descriptor prefetch options + // to be set for compaction input files and not to impact file + // descriptors for the same file used by user queries. + // Suggest to enable BlockBasedTableOptions.cache_index_and_filter_blocks + // for this mode if using block-based table. + // + // Default: false + bool new_table_reader_for_compaction_inputs = false; + + // If non-zero, we perform bigger reads when doing compaction. If you're + // running RocksDB on spinning disks, you should set this to at least 2MB. + // That way RocksDB's compaction is doing sequential instead of random reads. + // + // When non-zero, we also force new_table_reader_for_compaction_inputs to + // true. + // + // Default: 0 + size_t compaction_readahead_size = 0; + + // This is a maximum buffer size that is used by WinMmapReadableFile in + // unbuffered disk I/O mode. We need to maintain an aligned buffer for + // reads. We allow the buffer to grow until the specified value and then + // for bigger requests allocate one shot buffers. In unbuffered mode we + // always bypass read-ahead buffer at ReadaheadRandomAccessFile + // When read-ahead is required we then make use of compaction_readahead_size + // value and always try to read ahead. With read-ahead we always + // pre-allocate buffer to the size instead of growing it up to a limit. + // + // This option is currently honored only on Windows + // + // Default: 1 Mb + // + // Special value: 0 - means do not maintain per instance buffer. Allocate + // per request buffer and avoid locking. + size_t random_access_max_buffer_size = 1024 * 1024; + + // This is the maximum buffer size that is used by WritableFileWriter. + // On Windows, we need to maintain an aligned buffer for writes. + // We allow the buffer to grow until it's size hits the limit in buffered + // IO and fix the buffer size when using direct IO to ensure alignment of + // write requests if the logical sector size is unusual + // + // Default: 1024 * 1024 (1 MB) + size_t writable_file_max_buffer_size = 1024 * 1024; + // Use adaptive mutex, which spins in the user space before resorting // to kernel. This could reduce context switch when the mutex is not // heavily contended. However, if the mutex is hot, we could end up // wasting spin time. // Default: false - bool use_adaptive_mutex; - - // Allow RocksDB to use thread local storage to optimize performance. - // Default: true - bool allow_thread_local; + bool use_adaptive_mutex = false; // Create DBOptions with default values for all fields DBOptions(); @@ -815,29 +721,205 @@ struct DBOptions { void Dump(Logger* log) const; // Allows OS to incrementally sync files to disk while they are being - // written, asynchronously, in the background. + // written, asynchronously, in the background. This operation can be used + // to smooth out write I/Os over time. Users shouldn't rely on it for + // persistency guarantee. // Issue one request for every bytes_per_sync written. 0 turns it off. // Default: 0 // // You may consider using rate_limiter to regulate write rate to device. // When rate limiter is enabled, it automatically enables bytes_per_sync // to 1MB. - uint64_t bytes_per_sync; + // + // This option applies to table files + uint64_t bytes_per_sync = 0; + + // Same as bytes_per_sync, but applies to WAL files + // Default: 0, turned off + uint64_t wal_bytes_per_sync = 0; + + // A vector of EventListeners which call-back functions will be called + // when specific RocksDB event happens. + std::vector> listeners; + + // If true, then the status of the threads involved in this DB will + // be tracked and available via GetThreadList() API. + // + // Default: false + bool enable_thread_tracking = false; + + // The limited write rate to DB if soft_pending_compaction_bytes_limit or + // level0_slowdown_writes_trigger is triggered, or we are writing to the + // last mem table allowed and we allow more than 3 mem tables. It is + // calculated using size of user write requests before compression. + // RocksDB may decide to slow down more if the compaction still + // gets behind further. + // If the value is 0, we will infer a value from `rater_limiter` value + // if it is not empty, or 16MB if `rater_limiter` is empty. Note that + // if users change the rate in `rate_limiter` after DB is opened, + // `delayed_write_rate` won't be adjusted. + // + // Unit: byte per second. + // + // Default: 0 + uint64_t delayed_write_rate = 0; + + // By default, a single write thread queue is maintained. The thread gets + // to the head of the queue becomes write batch group leader and responsible + // for writing to WAL and memtable for the batch group. + // + // If enable_pipelined_write is true, separate write thread queue is + // maintained for WAL write and memtable write. A write thread first enter WAL + // writer queue and then memtable writer queue. Pending thread on the WAL + // writer queue thus only have to wait for previous writers to finish their + // WAL writing but not the memtable writing. Enabling the feature may improve + // write throughput and reduce latency of the prepare phase of two-phase + // commit. + // + // Default: false + bool enable_pipelined_write = false; + + // If true, allow multi-writers to update mem tables in parallel. + // Only some memtable_factory-s support concurrent writes; currently it + // is implemented only for SkipListFactory. Concurrent memtable writes + // are not compatible with inplace_update_support or filter_deletes. + // It is strongly recommended to set enable_write_thread_adaptive_yield + // if you are going to use this feature. + // + // Default: true + bool allow_concurrent_memtable_write = true; + + // If true, threads synchronizing with the write batch group leader will + // wait for up to write_thread_max_yield_usec before blocking on a mutex. + // This can substantially improve throughput for concurrent workloads, + // regardless of whether allow_concurrent_memtable_write is enabled. + // + // Default: true + bool enable_write_thread_adaptive_yield = true; + + // The maximum number of microseconds that a write operation will use + // a yielding spin loop to coordinate with other write threads before + // blocking on a mutex. (Assuming write_thread_slow_yield_usec is + // set properly) increasing this value is likely to increase RocksDB + // throughput at the expense of increased CPU usage. + // + // Default: 100 + uint64_t write_thread_max_yield_usec = 100; + + // The latency in microseconds after which a std::this_thread::yield + // call (sched_yield on Linux) is considered to be a signal that + // other processes or threads would like to use the current core. + // Increasing this makes writer threads more likely to take CPU + // by spinning, which will show up as an increase in the number of + // involuntary context switches. + // + // Default: 3 + uint64_t write_thread_slow_yield_usec = 3; + + // If true, then DB::Open() will not update the statistics used to optimize + // compaction decision by loading table properties from many files. + // Turning off this feature will improve DBOpen time especially in + // disk environment. + // + // Default: false + bool skip_stats_update_on_db_open = false; + + // Recovery mode to control the consistency while replaying WAL + // Default: kPointInTimeRecovery + WALRecoveryMode wal_recovery_mode = WALRecoveryMode::kPointInTimeRecovery; + + // if set to false then recovery will fail when a prepared + // transaction is encountered in the WAL + bool allow_2pc = false; + + // A global cache for table-level rows. + // Default: nullptr (disabled) + // Not supported in ROCKSDB_LITE mode! + std::shared_ptr row_cache = nullptr; + +#ifndef ROCKSDB_LITE + // A filter object supplied to be invoked while processing write-ahead-logs + // (WALs) during recovery. The filter provides a way to inspect log + // records, ignoring a particular record or skipping replay. + // The filter is invoked at startup and is invoked from a single-thread + // currently. + WalFilter* wal_filter = nullptr; +#endif // ROCKSDB_LITE + + // If true, then DB::Open / CreateColumnFamily / DropColumnFamily + // / SetOptions will fail if options file is not detected or properly + // persisted. + // + // DEFAULT: false + bool fail_if_options_file_error = false; + + // If true, then print malloc stats together with rocksdb.stats + // when printing to LOG. + // DEFAULT: false + bool dump_malloc_stats = false; + + // By default RocksDB replay WAL logs and flush them on DB open, which may + // create very small SST files. If this option is enabled, RocksDB will try + // to avoid (but not guarantee not to) flush during recovery. Also, existing + // WAL logs will be kept, so that if crash happened before flush, we still + // have logs to recover from. + // + // DEFAULT: false + bool avoid_flush_during_recovery = false; + + // By default RocksDB will flush all memtables on DB close if there are + // unpersisted data (i.e. with WAL disabled) The flush can be skip to speedup + // DB close. Unpersisted data WILL BE LOST. + // + // DEFAULT: false + // + // Dynamically changeable through SetDBOptions() API. + bool avoid_flush_during_shutdown = false; + + // Set this option to true during creation of database if you want + // to be able to ingest behind (call IngestExternalFile() skipping keys + // that already exist, rather than overwriting matching keys). + // Setting this option to true will affect 2 things: + // 1) Disable some internal optimizations around SST file compression + // 2) Reserve bottom-most level for ingested files only. + // 3) Note that num_levels should be >= 3 if this option is turned on. + // + // DEFAULT: false + // Immutable. + bool allow_ingest_behind = false; + + // If enabled it uses two queues for writes, one for the ones with + // disable_memtable and one for the ones that also write to memtable. This + // allows the memtable writes not to lag behind other writes. It can be used + // to optimize MySQL 2PC in which only the commits, which are serial, write to + // memtable. + bool concurrent_prepare = false; + + // If true WAL is not flushed automatically after each write. Instead it + // relies on manual invocation of FlushWAL to write the WAL buffer to its + // file. + bool manual_wal_flush = false; }; // Options to control the behavior of a database (passed to DB::Open) struct Options : public DBOptions, public ColumnFamilyOptions { // Create an Options object with default values for all fields. - Options() : - DBOptions(), - ColumnFamilyOptions() {} + Options() : DBOptions(), ColumnFamilyOptions() {} Options(const DBOptions& db_options, const ColumnFamilyOptions& column_family_options) : DBOptions(db_options), ColumnFamilyOptions(column_family_options) {} + // The function recovers options to the option as in version 4.6. + Options* OldDefaults(int rocksdb_major_version = 4, + int rocksdb_minor_version = 6); + void Dump(Logger* log) const; + void DumpCFOptions(Logger* log) const; + + // Some functions that make it easier to optimize RocksDB + // Set appropriate parameters for bulk loading. // The reason that this is a function that returns "this" instead of a // constructor is to enable chaining of multiple similar calls in the future. @@ -847,6 +929,10 @@ struct Options : public DBOptions, public ColumnFamilyOptions { // It's recommended to manually call CompactRange(NULL, NULL) before reading // from the database, because otherwise the read can be very slow. Options* PrepareForBulkLoad(); + + // Use this if your DB is very small (like under 1GB) and you don't want to + // spend lots of memory for memtables. + Options* OptimizeForSmallDb(); }; // @@ -857,51 +943,47 @@ struct Options : public DBOptions, public ColumnFamilyOptions { // the block cache. It will not page in data from the OS cache or data that // resides in storage. enum ReadTier { - kReadAllTier = 0x0, // data in memtable, block cache, OS cache or storage - kBlockCacheTier = 0x1 // data in memtable or block cache + kReadAllTier = 0x0, // data in memtable, block cache, OS cache or storage + kBlockCacheTier = 0x1, // data in memtable or block cache + kPersistedTier = 0x2, // persisted data. When WAL is disabled, this option + // will skip data in memtable. + // Note that this ReadTier currently only supports + // Get and MultiGet and does not support iterators. + kMemtableTier = 0x3 // data in memtable. used for memtable-only iterators. }; // Options that control read operations struct ReadOptions { - // If true, all data read from underlying storage will be - // verified against corresponding checksums. - // Default: true - bool verify_checksums; - - // Should the "data block"/"index block"/"filter block" read for this - // iteration be cached in memory? - // Callers may wish to set this field to false for bulk scans. - // Default: true - bool fill_cache; - - // If this option is set and memtable implementation allows, Seek - // might only return keys with the same prefix as the seek-key - // - // ! DEPRECATED: prefix_seek is on by default when prefix_extractor - // is configured - // bool prefix_seek; - // If "snapshot" is non-nullptr, read as of the supplied snapshot // (which must belong to the DB that is being read and which must - // not have been released). If "snapshot" is nullptr, use an impliicit + // not have been released). If "snapshot" is nullptr, use an implicit // snapshot of the state at the beginning of this read operation. // Default: nullptr const Snapshot* snapshot; - // If "prefix" is non-nullptr, and ReadOptions is being passed to - // db.NewIterator, only return results when the key begins with this - // prefix. This field is ignored by other calls (e.g., Get). - // Options.prefix_extractor must also be set, and - // prefix_extractor.InRange(prefix) must be true. The iterator - // returned by NewIterator when this option is set will behave just - // as if the underlying store did not contain any non-matching keys, - // with two exceptions. Seek() only accepts keys starting with the - // prefix, and SeekToLast() is not supported. prefix filter with this - // option will sometimes reduce the number of read IOPs. - // Default: nullptr + // "iterate_upper_bound" defines the extent upto which the forward iterator + // can returns entries. Once the bound is reached, Valid() will be false. + // "iterate_upper_bound" is exclusive ie the bound value is + // not a valid entry. If iterator_extractor is not null, the Seek target + // and iterator_upper_bound need to have the same prefix. + // This is because ordering is not guaranteed outside of prefix domain. + // There is no lower bound on the iterator. If needed, that can be easily + // implemented. // - // ! DEPRECATED - // const Slice* prefix; + // Default: nullptr + const Slice* iterate_upper_bound; + + // If non-zero, NewIterator will create a new table reader which + // performs reads of the given size. Using a large size (> 2MB) can + // improve the performance of forward iteration on spinning disks. + // Default: 0 + size_t readahead_size; + + // A threshold for the number of keys that can be skipped before failing an + // iterator seek as incomplete. The default value of 0 should be used to + // never fail a request as incomplete, even on skipping too many keys. + // Default: 0 + uint64_t max_skippable_internal_keys; // Specify if this read request should process data that ALREADY // resides on a particular cache. If the required data is not @@ -909,6 +991,17 @@ struct ReadOptions { // Default: kReadAllTier ReadTier read_tier; + // If true, all data read from underlying storage will be + // verified against corresponding checksums. + // Default: true + bool verify_checksums; + + // Should the "data block"/"index block"/"filter block" read for this + // iteration be cached in memory? + // Callers may wish to set this field to false for bulk scans. + // Default: true + bool fill_cache; + // Specify to create a tailing iterator -- a special iterator that has a // view of the complete database (i.e. it can also be used to read newly // added data) and is optimized for sequential reads. It will return records @@ -917,25 +1010,51 @@ struct ReadOptions { // Not supported in ROCKSDB_LITE mode! bool tailing; + // Specify to create a managed iterator -- a special iterator that + // uses less resources by having the ability to free its underlying + // resources on request. + // Default: false + // Not supported in ROCKSDB_LITE mode! + bool managed; + // Enable a total order seek regardless of index format (e.g. hash index) // used in the table. Some table format (e.g. plain table) may not support // this option. + // If true when calling Get(), we also skip prefix bloom when reading from + // block based table. It provides a way to read existing data after + // changing implementation of prefix extractor. bool total_order_seek; - ReadOptions() - : verify_checksums(true), - fill_cache(true), - snapshot(nullptr), - read_tier(kReadAllTier), - tailing(false), - total_order_seek(false) {} - ReadOptions(bool cksum, bool cache) - : verify_checksums(cksum), - fill_cache(cache), - snapshot(nullptr), - read_tier(kReadAllTier), - tailing(false), - total_order_seek(false) {} + // Enforce that the iterator only iterates over the same prefix as the seek. + // This option is effective only for prefix seeks, i.e. prefix_extractor is + // non-null for the column family and total_order_seek is false. Unlike + // iterate_upper_bound, prefix_same_as_start only works within a prefix + // but in both directions. + // Default: false + bool prefix_same_as_start; + + // Keep the blocks loaded by the iterator pinned in memory as long as the + // iterator is not deleted, If used when reading from tables created with + // BlockBasedTableOptions::use_delta_encoding = false, + // Iterator's property "rocksdb.iterator.is-key-pinned" is guaranteed to + // return 1. + // Default: false + bool pin_data; + + // If true, when PurgeObsoleteFile is called in CleanupIteratorState, we + // schedule a background job in the flush job queue and delete obsolete files + // in background. + // Default: false + bool background_purge_on_iterator_cleanup; + + // If true, keys deleted using the DeleteRange() API will be visible to + // readers until they are naturally deleted during compaction. This improves + // read performance in DBs with many range deletions. + // Default: false + bool ignore_range_deletions; + + ReadOptions(); + ReadOptions(bool cksum, bool cache); }; // Options that control write operations @@ -962,28 +1081,31 @@ struct WriteOptions { // and the write may got lost after a crash. bool disableWAL; - // If non-zero, then associated write waiting longer than the specified - // time MAY be aborted and returns Status::TimedOut. A write that takes - // less than the specified time is guaranteed to not fail with - // Status::TimedOut. - // - // The number of times a write call encounters a timeout is recorded in - // Statistics.WRITE_TIMEDOUT - // - // Default: 0 - uint64_t timeout_hint_us; - // If true and if user is trying to write to column families that don't exist // (they were dropped), ignore the write (don't return an error). If there // are multiple writes in a WriteBatch, other writes will succeed. // Default: false bool ignore_missing_column_families; + // If true and we need to wait or sleep for the write request, fails + // immediately with Status::Incomplete(). + bool no_slowdown; + + // If true, this write request is of lower priority if compaction is + // behind. In this case, no_slowdown = true, the request will be cancelled + // immediately with Status::Incomplete() returned. Otherwise, it will be + // slowed down. The slowdown value is determined by RocksDB to guarantee + // it introduces minimum impacts to high priority writes. + // + // Default: false + bool low_pri; + WriteOptions() : sync(false), disableWAL(false), - timeout_hint_us(0), - ignore_missing_column_families(false) {} + ignore_missing_column_families(false), + no_slowdown(false), + low_pri(false) {} }; // Options that control flush operations @@ -995,16 +1117,80 @@ struct FlushOptions { FlushOptions() : wait(true) {} }; -// Get options based on some guidelines. Now only tune parameter based on -// flush/compaction and fill default parameters for other parameters. -// total_write_buffer_limit: budget for memory spent for mem tables -// read_amplification_threshold: comfortable value of read amplification -// write_amplification_threshold: comfortable value of write amplification. -// target_db_size: estimated total DB size. -extern Options GetOptions(size_t total_write_buffer_limit, - int read_amplification_threshold = 8, - int write_amplification_threshold = 32, - uint64_t target_db_size = 68719476736 /* 64GB */); +// Create a Logger from provided DBOptions +extern Status CreateLoggerFromOptions(const std::string& dbname, + const DBOptions& options, + std::shared_ptr* logger); + +// CompactionOptions are used in CompactFiles() call. +struct CompactionOptions { + // Compaction output compression type + // Default: snappy + CompressionType compression; + // Compaction will create files of size `output_file_size_limit`. + // Default: MAX, which means that compaction will create a single file + uint64_t output_file_size_limit; + + CompactionOptions() + : compression(kSnappyCompression), + output_file_size_limit(std::numeric_limits::max()) {} +}; + +// For level based compaction, we can configure if we want to skip/force +// bottommost level compaction. +enum class BottommostLevelCompaction { + // Skip bottommost level compaction + kSkip, + // Only compact bottommost level if there is a compaction filter + // This is the default option + kIfHaveCompactionFilter, + // Always compact bottommost level + kForce, +}; + +// CompactRangeOptions is used by CompactRange() call. +struct CompactRangeOptions { + // If true, no other compaction will run at the same time as this + // manual compaction + bool exclusive_manual_compaction = true; + // If true, compacted files will be moved to the minimum level capable + // of holding the data or given level (specified non-negative target_level). + bool change_level = false; + // If change_level is true and target_level have non-negative value, compacted + // files will be moved to target_level. + int target_level = -1; + // Compaction outputs will be placed in options.db_paths[target_path_id]. + // Behavior is undefined if target_path_id is out of range. + uint32_t target_path_id = 0; + // By default level based compaction will only compact the bottommost level + // if there is a compaction filter + BottommostLevelCompaction bottommost_level_compaction = + BottommostLevelCompaction::kIfHaveCompactionFilter; +}; + +// IngestExternalFileOptions is used by IngestExternalFile() +struct IngestExternalFileOptions { + // Can be set to true to move the files instead of copying them. + bool move_files = false; + // If set to false, an ingested file keys could appear in existing snapshots + // that where created before the file was ingested. + bool snapshot_consistency = true; + // If set to false, IngestExternalFile() will fail if the file key range + // overlaps with existing keys or tombstones in the DB. + bool allow_global_seqno = true; + // If set to false and the file key range overlaps with the memtable key range + // (memtable flush required), IngestExternalFile will fail. + bool allow_blocking_flush = true; + // Set to true if you would like duplicate keys in the file being ingested + // to be skipped rather than overwriting existing data under that key. + // Usecase: back-fill of some historical data in the database without + // over-writing existing newer version of data. + // This option could only be used if the DB has been running + // with allow_ingest_behind=true since the dawn of time. + // All files will be ingested at the bottommost level with seqno=0. + bool ingest_behind = false; +}; + } // namespace rocksdb #endif // STORAGE_ROCKSDB_INCLUDE_OPTIONS_H_ diff --git a/include/rocksdb/perf_context.h b/include/rocksdb/perf_context.h index e96d09d2a61..1095d063bd6 100644 --- a/include/rocksdb/perf_context.h +++ b/include/rocksdb/perf_context.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef STORAGE_ROCKSDB_INCLUDE_PERF_CONTEXT_H #define STORAGE_ROCKSDB_INCLUDE_PERF_CONTEXT_H @@ -9,69 +9,155 @@ #include #include -namespace rocksdb { - -enum PerfLevel { - kDisable = 0, // disable perf stats - kEnableCount = 1, // enable only count stats - kEnableTime = 2 // enable time stats too -}; - -// set the perf stats level -void SetPerfLevel(PerfLevel level); +#include "rocksdb/perf_level.h" -// get current perf stats level -PerfLevel GetPerfLevel(); +namespace rocksdb { // A thread local context for gathering performance counter efficiently // and transparently. +// Use SetPerfLevel(PerfLevel::kEnableTime) to enable time stats. struct PerfContext { void Reset(); // reset all performance counters to zero - std::string ToString() const; + std::string ToString(bool exclude_zero_counters = false) const; uint64_t user_key_comparison_count; // total number of user key comparisons uint64_t block_cache_hit_count; // total number of block cache hits uint64_t block_read_count; // total number of block reads (with IO) uint64_t block_read_byte; // total number of bytes from block reads - uint64_t block_read_time; // total time spent on block reads - uint64_t block_checksum_time; // total time spent on block checksum - uint64_t block_decompress_time; // total time spent on block decompression - // total number of internal keys skipped over during iteration (overwritten or - // deleted, to be more specific, hidden by a put or delete of the same key) + uint64_t block_read_time; // total nanos spent on block reads + uint64_t block_checksum_time; // total nanos spent on block checksum + uint64_t block_decompress_time; // total nanos spent on block decompression + + uint64_t get_read_bytes; // bytes for vals returned by Get + uint64_t multiget_read_bytes; // bytes for vals returned by MultiGet + uint64_t iter_read_bytes; // bytes for keys/vals decoded by iterator + + // total number of internal keys skipped over during iteration. + // There are several reasons for it: + // 1. when calling Next(), the iterator is in the position of the previous + // key, so that we'll need to skip it. It means this counter will always + // be incremented in Next(). + // 2. when calling Next(), we need to skip internal entries for the previous + // keys that are overwritten. + // 3. when calling Next(), Seek() or SeekToFirst(), after previous key + // before calling Next(), the seek key in Seek() or the beginning for + // SeekToFirst(), there may be one or more deleted keys before the next + // valid key that the operation should place the iterator to. We need + // to skip both of the tombstone and updates hidden by the tombstones. The + // tombstones are not included in this counter, while previous updates + // hidden by the tombstones will be included here. + // 4. symmetric cases for Prev() and SeekToLast() + // internal_recent_skipped_count is not included in this counter. + // uint64_t internal_key_skipped_count; - // total number of deletes skipped over during iteration + // Total number of deletes and single deletes skipped over during iteration + // When calling Next(), Seek() or SeekToFirst(), after previous position + // before calling Next(), the seek key in Seek() or the beginning for + // SeekToFirst(), there may be one or more deleted keys before the next valid + // key. Every deleted key is counted once. We don't recount here if there are + // still older updates invalidated by the tombstones. + // uint64_t internal_delete_skipped_count; + // How many times iterators skipped over internal keys that are more recent + // than the snapshot that iterator is using. + // + uint64_t internal_recent_skipped_count; + // How many values were fed into merge operator by iterators. + // + uint64_t internal_merge_count; - uint64_t get_snapshot_time; // total time spent on getting snapshot - uint64_t get_from_memtable_time; // total time spent on querying memtables + uint64_t get_snapshot_time; // total nanos spent on getting snapshot + uint64_t get_from_memtable_time; // total nanos spent on querying memtables uint64_t get_from_memtable_count; // number of mem tables queried - // total time spent after Get() finds a key + // total nanos spent after Get() finds a key uint64_t get_post_process_time; - uint64_t get_from_output_files_time; // total time reading from output files - // total time spent on seeking child iters + uint64_t get_from_output_files_time; // total nanos reading from output files + // total nanos spent on seeking memtable + uint64_t seek_on_memtable_time; + // number of seeks issued on memtable + // (including SeekForPrev but not SeekToFirst and SeekToLast) + uint64_t seek_on_memtable_count; + // number of Next()s issued on memtable + uint64_t next_on_memtable_count; + // number of Prev()s issued on memtable + uint64_t prev_on_memtable_count; + // total nanos spent on seeking child iters uint64_t seek_child_seek_time; // number of seek issued in child iterators uint64_t seek_child_seek_count; - uint64_t seek_min_heap_time; // total time spent on the merge heap - // total time spent on seeking the internal entries + uint64_t seek_min_heap_time; // total nanos spent on the merge min heap + uint64_t seek_max_heap_time; // total nanos spent on the merge max heap + // total nanos spent on seeking the internal entries uint64_t seek_internal_seek_time; - // total time spent on iterating internal entries to find the next user entry + // total nanos spent on iterating internal entries to find the next user entry uint64_t find_next_user_entry_time; - // total time spent on pre or post processing when writing a record - uint64_t write_pre_and_post_process_time; - uint64_t write_wal_time; // total time spent on writing to WAL - // total time spent on writing to mem tables + + // total nanos spent on writing to WAL + uint64_t write_wal_time; + // total nanos spent on writing to mem tables uint64_t write_memtable_time; + // total nanos spent on delaying write + uint64_t write_delay_time; + // total nanos spent on writing a record, excluding the above three times + uint64_t write_pre_and_post_process_time; + + uint64_t db_mutex_lock_nanos; // time spent on acquiring DB mutex. + // Time spent on waiting with a condition variable created with DB mutex. + uint64_t db_condition_wait_nanos; + // Time spent on merge operator. + uint64_t merge_operator_time_nanos; + + // Time spent on reading index block from block cache or SST file + uint64_t read_index_block_nanos; + // Time spent on reading filter block from block cache or SST file + uint64_t read_filter_block_nanos; + // Time spent on creating data block iterator + uint64_t new_table_block_iter_nanos; + // Time spent on creating a iterator of an SST file. + uint64_t new_table_iterator_nanos; + // Time spent on seeking a key in data/index blocks + uint64_t block_seek_nanos; + // Time spent on finding or creating a table reader + uint64_t find_table_nanos; + // total number of mem table bloom hits + uint64_t bloom_memtable_hit_count; + // total number of mem table bloom misses + uint64_t bloom_memtable_miss_count; + // total number of SST table bloom hits + uint64_t bloom_sst_hit_count; + // total number of SST table bloom misses + uint64_t bloom_sst_miss_count; + + // Total time spent in Env filesystem operations. These are only populated + // when TimedEnv is used. + uint64_t env_new_sequential_file_nanos; + uint64_t env_new_random_access_file_nanos; + uint64_t env_new_writable_file_nanos; + uint64_t env_reuse_writable_file_nanos; + uint64_t env_new_random_rw_file_nanos; + uint64_t env_new_directory_nanos; + uint64_t env_file_exists_nanos; + uint64_t env_get_children_nanos; + uint64_t env_get_children_file_attributes_nanos; + uint64_t env_delete_file_nanos; + uint64_t env_create_dir_nanos; + uint64_t env_create_dir_if_missing_nanos; + uint64_t env_delete_dir_nanos; + uint64_t env_get_file_size_nanos; + uint64_t env_get_file_modification_time_nanos; + uint64_t env_rename_file_nanos; + uint64_t env_link_file_nanos; + uint64_t env_lock_file_nanos; + uint64_t env_unlock_file_nanos; + uint64_t env_new_logger_nanos; }; -#if defined(NPERF_CONTEXT) || defined(IOS_CROSS_COMPILE) -extern PerfContext perf_context; -#else -extern __thread PerfContext perf_context; -#endif +// Get Thread-local PerfContext object pointer +// if defined(NPERF_CONTEXT), then the pointer is not thread-local +PerfContext* get_perf_context(); } diff --git a/include/rocksdb/perf_level.h b/include/rocksdb/perf_level.h new file mode 100644 index 00000000000..84a331c355e --- /dev/null +++ b/include/rocksdb/perf_level.h @@ -0,0 +1,33 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef INCLUDE_ROCKSDB_PERF_LEVEL_H_ +#define INCLUDE_ROCKSDB_PERF_LEVEL_H_ + +#include +#include + +namespace rocksdb { + +// How much perf stats to collect. Affects perf_context and iostats_context. +enum PerfLevel : unsigned char { + kUninitialized = 0, // unknown setting + kDisable = 1, // disable perf stats + kEnableCount = 2, // enable only count stats + kEnableTimeExceptForMutex = 3, // Other than count stats, also enable time + // stats except for mutexes + kEnableTime = 4, // enable count and time stats + kOutOfBounds = 5 // N.B. Must always be the last value! +}; + +// set the perf stats level for current thread +void SetPerfLevel(PerfLevel level); + +// get current perf stats level for current thread +PerfLevel GetPerfLevel(); + +} // namespace rocksdb + +#endif // INCLUDE_ROCKSDB_PERF_LEVEL_H_ diff --git a/include/rocksdb/persistent_cache.h b/include/rocksdb/persistent_cache.h new file mode 100644 index 00000000000..05c36852a55 --- /dev/null +++ b/include/rocksdb/persistent_cache.h @@ -0,0 +1,67 @@ +// Copyright (c) 2013, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#pragma once + +#include +#include +#include + +#include "rocksdb/env.h" +#include "rocksdb/slice.h" +#include "rocksdb/statistics.h" +#include "rocksdb/status.h" + +namespace rocksdb { + +// PersistentCache +// +// Persistent cache interface for caching IO pages on a persistent medium. The +// cache interface is specifically designed for persistent read cache. +class PersistentCache { + public: + typedef std::vector> StatsType; + + virtual ~PersistentCache() {} + + // Insert to page cache + // + // page_key Identifier to identify a page uniquely across restarts + // data Page data + // size Size of the page + virtual Status Insert(const Slice& key, const char* data, + const size_t size) = 0; + + // Lookup page cache by page identifier + // + // page_key Page identifier + // buf Buffer where the data should be copied + // size Size of the page + virtual Status Lookup(const Slice& key, std::unique_ptr* data, + size_t* size) = 0; + + // Is cache storing uncompressed data ? + // + // True if the cache is configured to store uncompressed data else false + virtual bool IsCompressed() = 0; + + // Return stats as map of {string, double} per-tier + // + // Persistent cache can be initialized as a tier of caches. The stats are per + // tire top-down + virtual StatsType Stats() = 0; + + virtual std::string GetPrintableOptions() const = 0; +}; + +// Factor method to create a new persistent cache +Status NewPersistentCache(Env* const env, const std::string& path, + const uint64_t size, + const std::shared_ptr& log, + const bool optimized_for_nvm, + std::shared_ptr* cache); +} // namespace rocksdb diff --git a/include/rocksdb/rate_limiter.h b/include/rocksdb/rate_limiter.h index 027c9fa54d5..838c98a6de6 100644 --- a/include/rocksdb/rate_limiter.h +++ b/include/rocksdb/rate_limiter.h @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,17 +10,73 @@ #pragma once #include "rocksdb/env.h" +#include "rocksdb/statistics.h" namespace rocksdb { class RateLimiter { public: + enum class OpType { + // Limitation: we currently only invoke Request() with OpType::kRead for + // compactions when DBOptions::new_table_reader_for_compaction_inputs is set + kRead, + kWrite, + }; + enum class Mode { + kReadsOnly, + kWritesOnly, + kAllIo, + }; + + // For API compatibility, default to rate-limiting writes only. + explicit RateLimiter(Mode mode = Mode::kWritesOnly) : mode_(mode) {} + virtual ~RateLimiter() {} - // Request for token to write bytes. If this request can not be satisfied, - // the call is blocked. Caller is responsible to make sure - // bytes < GetSingleBurstBytes() - virtual void Request(const int64_t bytes, const Env::IOPriority pri) = 0; + // This API allows user to dynamically change rate limiter's bytes per second. + // REQUIRED: bytes_per_second > 0 + virtual void SetBytesPerSecond(int64_t bytes_per_second) = 0; + + // Deprecated. New RateLimiter derived classes should override + // Request(const int64_t, const Env::IOPriority, Statistics*) or + // Request(const int64_t, const Env::IOPriority, Statistics*, OpType) + // instead. + // + // Request for token for bytes. If this request can not be satisfied, the call + // is blocked. Caller is responsible to make sure + // bytes <= GetSingleBurstBytes() + virtual void Request(const int64_t bytes, const Env::IOPriority pri) { + assert(false); + } + + // Request for token for bytes and potentially update statistics. If this + // request can not be satisfied, the call is blocked. Caller is responsible to + // make sure bytes <= GetSingleBurstBytes(). + virtual void Request(const int64_t bytes, const Env::IOPriority pri, + Statistics* /* stats */) { + // For API compatibility, default implementation calls the older API in + // which statistics are unsupported. + Request(bytes, pri); + } + + // Requests token to read or write bytes and potentially updates statistics. + // + // If this request can not be satisfied, the call is blocked. Caller is + // responsible to make sure bytes <= GetSingleBurstBytes(). + virtual void Request(const int64_t bytes, const Env::IOPriority pri, + Statistics* stats, OpType op_type) { + if (IsRateLimited(op_type)) { + Request(bytes, pri, stats); + } + } + + // Requests token to read or write bytes and potentially updates statistics. + // Takes into account GetSingleBurstBytes() and alignment (e.g., in case of + // direct I/O) to allocate an appropriate number of bytes, which may be less + // than the number of bytes requested. + virtual size_t RequestToken(size_t bytes, size_t alignment, + Env::IOPriority io_priority, Statistics* stats, + RateLimiter::OpType op_type); // Max bytes can be granted in a single burst virtual int64_t GetSingleBurstBytes() const = 0; @@ -32,6 +88,24 @@ class RateLimiter { // Total # of requests that go though rate limiter virtual int64_t GetTotalRequests( const Env::IOPriority pri = Env::IO_TOTAL) const = 0; + + virtual int64_t GetBytesPerSecond() const = 0; + + virtual bool IsRateLimited(OpType op_type) { + if ((mode_ == RateLimiter::Mode::kWritesOnly && + op_type == RateLimiter::OpType::kRead) || + (mode_ == RateLimiter::Mode::kReadsOnly && + op_type == RateLimiter::OpType::kWrite)) { + return false; + } + return true; + } + + protected: + Mode GetMode() { return mode_; } + + private: + const Mode mode_; }; // Create a RateLimiter object, which can be shared among RocksDB instances to @@ -47,14 +121,15 @@ class RateLimiter { // The default should work for most cases. // @fairness: RateLimiter accepts high-pri requests and low-pri requests. // A low-pri request is usually blocked in favor of hi-pri request. Currently, -// RocksDB assigns low-pri to request from compaciton and high-pri to request +// RocksDB assigns low-pri to request from compaction and high-pri to request // from flush. Low-pri requests can get blocked if flush requests come in -// continuouly. This fairness parameter grants low-pri requests permission by +// continuously. This fairness parameter grants low-pri requests permission by // 1/fairness chance even though high-pri requests exist to avoid starvation. // You should be good by leaving it at default 10. +// @mode: Mode indicates which types of operations count against the limit. extern RateLimiter* NewGenericRateLimiter( - int64_t rate_bytes_per_sec, - int64_t refill_period_us = 100 * 1000, - int32_t fairness = 10); + int64_t rate_bytes_per_sec, int64_t refill_period_us = 100 * 1000, + int32_t fairness = 10, + RateLimiter::Mode mode = RateLimiter::Mode::kWritesOnly); } // namespace rocksdb diff --git a/include/rocksdb/slice.h b/include/rocksdb/slice.h index 406a8abb9ab..4f24c8a2217 100644 --- a/include/rocksdb/slice.h +++ b/include/rocksdb/slice.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -20,10 +20,13 @@ #define STORAGE_ROCKSDB_INCLUDE_SLICE_H_ #include +#include #include #include #include +#include "rocksdb/cleanable.h" + namespace rocksdb { class Slice { @@ -42,6 +45,10 @@ class Slice { /* implicit */ Slice(const char* s) : data_(s), size_(strlen(s)) { } + // Create a single slice from SliceParts using buf as storage. + // buf must exist as long as the returned Slice exists. + Slice(const struct SliceParts& parts, std::string* buf); + // Return a pointer to the beginning of the referenced data const char* data() const { return data_; } @@ -68,21 +75,22 @@ class Slice { size_ -= n; } - // Return a string that contains the copy of the referenced data. - std::string ToString(bool hex = false) const { - if (hex) { - std::string result; - char buf[10]; - for (size_t i = 0; i < size_; i++) { - snprintf(buf, 10, "%02X", (unsigned char)data_[i]); - result += buf; - } - return result; - } else { - return std::string(data_, size_); - } + void remove_suffix(size_t n) { + assert(n <= size()); + size_ -= n; } + // Return a string that contains the copy of the referenced data. + // when hex is true, returns a string of twice the length hex encoded (0-9A-F) + std::string ToString(bool hex = false) const; + + // Decodes the current slice interpreted as an hexadecimal string into result, + // if successful returns true, if this isn't a valid hex string + // (e.g not coming from Slice::ToString(true)) DecodeHex returns false. + // This slice is expected to have an even number of 0-9A-F characters + // also accepts lowercase (a-f) + bool DecodeHex(std::string* result) const; + // Three-way comparison. Returns value: // < 0 iff "*this" < "b", // == 0 iff "*this" == "b", @@ -95,6 +103,14 @@ class Slice { (memcmp(data_, x.data_, x.size_) == 0)); } + bool ends_with(const Slice& x) const { + return ((size_ >= x.size_) && + (memcmp(data_ + size_ - x.size_, x.data_, x.size_) == 0)); + } + + // Compare two slices and returns the first byte where they differ + size_t difference_offset(const Slice& b) const; + // private: make these public for rocksdbjni access const char* data_; size_t size_; @@ -102,6 +118,85 @@ class Slice { // Intentionally copyable }; +/** + * A Slice that can be pinned with some cleanup tasks, which will be run upon + * ::Reset() or object destruction, whichever is invoked first. This can be used + * to avoid memcpy by having the PinnsableSlice object referring to the data + * that is locked in the memory and release them after the data is consumed. + */ +class PinnableSlice : public Slice, public Cleanable { + public: + PinnableSlice() { buf_ = &self_space_; } + explicit PinnableSlice(std::string* buf) { buf_ = buf; } + + // No copy constructor and copy assignment allowed. + PinnableSlice(PinnableSlice&) = delete; + PinnableSlice& operator=(PinnableSlice&) = delete; + + inline void PinSlice(const Slice& s, CleanupFunction f, void* arg1, + void* arg2) { + assert(!pinned_); + pinned_ = true; + data_ = s.data(); + size_ = s.size(); + RegisterCleanup(f, arg1, arg2); + assert(pinned_); + } + + inline void PinSlice(const Slice& s, Cleanable* cleanable) { + assert(!pinned_); + pinned_ = true; + data_ = s.data(); + size_ = s.size(); + cleanable->DelegateCleanupsTo(this); + assert(pinned_); + } + + inline void PinSelf(const Slice& slice) { + assert(!pinned_); + buf_->assign(slice.data(), slice.size()); + data_ = buf_->data(); + size_ = buf_->size(); + assert(!pinned_); + } + + inline void PinSelf() { + assert(!pinned_); + data_ = buf_->data(); + size_ = buf_->size(); + assert(!pinned_); + } + + void remove_suffix(size_t n) { + assert(n <= size()); + if (pinned_) { + size_ -= n; + } else { + buf_->erase(size() - n, n); + PinSelf(); + } + } + + void remove_prefix(size_t n) { + assert(0); // Not implemented + } + + void Reset() { + Cleanable::Reset(); + pinned_ = false; + } + + inline std::string* GetSelf() { return buf_; } + + inline bool IsPinned() { return pinned_; } + + private: + friend class PinnableSlice4Test; + std::string self_space_; + std::string* buf_; + bool pinned_ = false; +}; + // A set of Slices that are virtually concatenated together. 'parts' points // to an array of Slices. The number of elements in the array is 'num_parts'. struct SliceParts { @@ -123,7 +218,8 @@ inline bool operator!=(const Slice& x, const Slice& y) { } inline int Slice::compare(const Slice& b) const { - const int min_len = (size_ < b.size_) ? size_ : b.size_; + assert(data_ != nullptr && b.data_ != nullptr); + const size_t min_len = (size_ < b.size_) ? size_ : b.size_; int r = memcmp(data_, b.data_, min_len); if (r == 0) { if (size_ < b.size_) r = -1; @@ -132,6 +228,15 @@ inline int Slice::compare(const Slice& b) const { return r; } +inline size_t Slice::difference_offset(const Slice& b) const { + size_t off = 0; + const size_t len = (size_ < b.size_) ? size_ : b.size_; + for (; off < len; off++) { + if (data_[off] != b.data_[off]) break; + } + return off; +} + } // namespace rocksdb #endif // STORAGE_ROCKSDB_INCLUDE_SLICE_H_ diff --git a/include/rocksdb/slice_transform.h b/include/rocksdb/slice_transform.h index a78455001a4..fc82bf58456 100644 --- a/include/rocksdb/slice_transform.h +++ b/include/rocksdb/slice_transform.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2012 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -21,6 +21,12 @@ namespace rocksdb { class Slice; +/* + * A SliceTranform is a generic pluggable way of transforming one string + * to another. Its primary use-case is in configuring rocksdb + * to store prefix blooms by setting prefix_extractor in + * ColumnFamilyOptions. + */ class SliceTransform { public: virtual ~SliceTransform() {}; @@ -28,18 +34,64 @@ class SliceTransform { // Return the name of this transformation. virtual const char* Name() const = 0; - // transform a src in domain to a dst in the range - virtual Slice Transform(const Slice& src) const = 0; + // Extract a prefix from a specified key. This method is called when + // a key is inserted into the db, and the returned slice is used to + // create a bloom filter. + virtual Slice Transform(const Slice& key) const = 0; - // determine whether this is a valid src upon the function applies - virtual bool InDomain(const Slice& src) const = 0; + // Determine whether the specified key is compatible with the logic + // specified in the Transform method. This method is invoked for every + // key that is inserted into the db. If this method returns true, + // then Transform is called to translate the key to its prefix and + // that returned prefix is inserted into the bloom filter. If this + // method returns false, then the call to Transform is skipped and + // no prefix is inserted into the bloom filters. + // + // For example, if the Transform method operates on a fixed length + // prefix of size 4, then an invocation to InDomain("abc") returns + // false because the specified key length(3) is shorter than the + // prefix size of 4. + // + // Wiki documentation here: + // https://github.com/facebook/rocksdb/wiki/Prefix-Seek-API-Changes + // + virtual bool InDomain(const Slice& key) const = 0; - // determine whether dst=Transform(src) for some src - virtual bool InRange(const Slice& dst) const = 0; + // This is currently not used and remains here for backward compatibility. + virtual bool InRange(const Slice& dst) const { return false; } + + // Transform(s)=Transform(`prefix`) for any s with `prefix` as a prefix. + // + // This function is not used by RocksDB, but for users. If users pass + // Options by string to RocksDB, they might not know what prefix extractor + // they are using. This function is to help users can determine: + // if they want to iterate all keys prefixing `prefix`, whether it is + // safe to use prefix bloom filter and seek to key `prefix`. + // If this function returns true, this means a user can Seek() to a prefix + // using the bloom filter. Otherwise, user needs to skip the bloom filter + // by setting ReadOptions.total_order_seek = true. + // + // Here is an example: Suppose we implement a slice transform that returns + // the first part of the string after spliting it using delimiter ",": + // 1. SameResultWhenAppended("abc,") should return true. If applying prefix + // bloom filter using it, all slices matching "abc:.*" will be extracted + // to "abc,", so any SST file or memtable containing any of those key + // will not be filtered out. + // 2. SameResultWhenAppended("abc") should return false. A user will not + // guaranteed to see all the keys matching "abc.*" if a user seek to "abc" + // against a DB with the same setting. If one SST file only contains + // "abcd,e", the file can be filtered out and the key will be invisible. + // + // i.e., an implementation always returning false is safe. + virtual bool SameResultWhenAppended(const Slice& prefix) const { + return false; + } }; extern const SliceTransform* NewFixedPrefixTransform(size_t prefix_len); +extern const SliceTransform* NewCappedPrefixTransform(size_t cap_len); + extern const SliceTransform* NewNoopTransform(); } diff --git a/include/rocksdb/snapshot.h b/include/rocksdb/snapshot.h new file mode 100644 index 00000000000..a96eb763e3e --- /dev/null +++ b/include/rocksdb/snapshot.h @@ -0,0 +1,48 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include "rocksdb/types.h" + +namespace rocksdb { + +class DB; + +// Abstract handle to particular state of a DB. +// A Snapshot is an immutable object and can therefore be safely +// accessed from multiple threads without any external synchronization. +// +// To Create a Snapshot, call DB::GetSnapshot(). +// To Destroy a Snapshot, call DB::ReleaseSnapshot(snapshot). +class Snapshot { + public: + // returns Snapshot's sequence number + virtual SequenceNumber GetSequenceNumber() const = 0; + + protected: + virtual ~Snapshot(); +}; + +// Simple RAII wrapper class for Snapshot. +// Constructing this object will create a snapshot. Destructing will +// release the snapshot. +class ManagedSnapshot { + public: + explicit ManagedSnapshot(DB* db); + + // Instead of creating a snapshot, take ownership of the input snapshot. + ManagedSnapshot(DB* db, const Snapshot* _snapshot); + + ~ManagedSnapshot(); + + const Snapshot* snapshot(); + + private: + DB* db_; + const Snapshot* snapshot_; +}; + +} // namespace rocksdb diff --git a/include/rocksdb/sst_dump_tool.h b/include/rocksdb/sst_dump_tool.h new file mode 100644 index 00000000000..021faa019cc --- /dev/null +++ b/include/rocksdb/sst_dump_tool.h @@ -0,0 +1,17 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#ifndef ROCKSDB_LITE +#pragma once + +namespace rocksdb { + +class SSTDumpTool { + public: + int Run(int argc, char** argv); +}; + +} // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/include/rocksdb/sst_file_manager.h b/include/rocksdb/sst_file_manager.h new file mode 100644 index 00000000000..692007d31a2 --- /dev/null +++ b/include/rocksdb/sst_file_manager.h @@ -0,0 +1,85 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include +#include + +#include "rocksdb/status.h" + +namespace rocksdb { + +class Env; +class Logger; + +// SstFileManager is used to track SST files in the DB and control there +// deletion rate. +// All SstFileManager public functions are thread-safe. +class SstFileManager { + public: + virtual ~SstFileManager() {} + + // Update the maximum allowed space that should be used by RocksDB, if + // the total size of the SST files exceeds max_allowed_space, writes to + // RocksDB will fail. + // + // Setting max_allowed_space to 0 will disable this feature, maximum allowed + // space will be infinite (Default value). + // + // thread-safe. + virtual void SetMaxAllowedSpaceUsage(uint64_t max_allowed_space) = 0; + + // Return true if the total size of SST files exceeded the maximum allowed + // space usage. + // + // thread-safe. + virtual bool IsMaxAllowedSpaceReached() = 0; + + // Return the total size of all tracked files. + // thread-safe + virtual uint64_t GetTotalSize() = 0; + + // Return a map containing all tracked files and there corresponding sizes. + // thread-safe + virtual std::unordered_map GetTrackedFiles() = 0; + + // Return delete rate limit in bytes per second. + // thread-safe + virtual int64_t GetDeleteRateBytesPerSecond() = 0; + + // Update the delete rate limit in bytes per second. + // zero means disable delete rate limiting and delete files immediately + // thread-safe + virtual void SetDeleteRateBytesPerSecond(int64_t delete_rate) = 0; +}; + +// Create a new SstFileManager that can be shared among multiple RocksDB +// instances to track SST file and control there deletion rate. +// +// @param env: Pointer to Env object, please see "rocksdb/env.h". +// @param info_log: If not nullptr, info_log will be used to log errors. +// +// == Deletion rate limiting specific arguments == +// @param trash_dir: Path to the directory where deleted files will be moved +// to be deleted in a background thread while applying rate limiting. If this +// directory doesn't exist, it will be created. This directory should not be +// used by any other process or any other SstFileManager, Set to "" to +// disable deletion rate limiting. +// @param rate_bytes_per_sec: How many bytes should be deleted per second, If +// this value is set to 1024 (1 Kb / sec) and we deleted a file of size 4 Kb +// in 1 second, we will wait for another 3 seconds before we delete other +// files, Set to 0 to disable deletion rate limiting. +// @param delete_existing_trash: If set to true, the newly created +// SstFileManager will delete files that already exist in trash_dir. +// @param status: If not nullptr, status will contain any errors that happened +// during creating the missing trash_dir or deleting existing files in trash. +extern SstFileManager* NewSstFileManager( + Env* env, std::shared_ptr info_log = nullptr, + std::string trash_dir = "", int64_t rate_bytes_per_sec = 0, + bool delete_existing_trash = true, Status* status = nullptr); + +} // namespace rocksdb diff --git a/include/rocksdb/sst_file_writer.h b/include/rocksdb/sst_file_writer.h new file mode 100644 index 00000000000..04d5c271a0a --- /dev/null +++ b/include/rocksdb/sst_file_writer.h @@ -0,0 +1,115 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#pragma once + +#include +#include + +#include "rocksdb/env.h" +#include "rocksdb/options.h" +#include "rocksdb/table_properties.h" +#include "rocksdb/types.h" + +#if defined(__GNUC__) || defined(__clang__) +#define ROCKSDB_DEPRECATED_FUNC __attribute__((__deprecated__)) +#elif _WIN32 +#define ROCKSDB_DEPRECATED_FUNC __declspec(deprecated) +#endif + +namespace rocksdb { + +class Comparator; + +// ExternalSstFileInfo include information about sst files created +// using SstFileWriter. +struct ExternalSstFileInfo { + ExternalSstFileInfo() {} + ExternalSstFileInfo(const std::string& _file_path, + const std::string& _smallest_key, + const std::string& _largest_key, + SequenceNumber _sequence_number, uint64_t _file_size, + int32_t _num_entries, int32_t _version) + : file_path(_file_path), + smallest_key(_smallest_key), + largest_key(_largest_key), + sequence_number(_sequence_number), + file_size(_file_size), + num_entries(_num_entries), + version(_version) {} + + std::string file_path; // external sst file path + std::string smallest_key; // smallest user key in file + std::string largest_key; // largest user key in file + SequenceNumber sequence_number; // sequence number of all keys in file + uint64_t file_size; // file size in bytes + uint64_t num_entries; // number of entries in file + int32_t version; // file version +}; + +// SstFileWriter is used to create sst files that can be added to database later +// All keys in files generated by SstFileWriter will have sequence number = 0. +class SstFileWriter { + public: + // User can pass `column_family` to specify that the generated file will + // be ingested into this column_family, note that passing nullptr means that + // the column_family is unknown. + // If invalidate_page_cache is set to true, SstFileWriter will give the OS a + // hint that this file pages is not needed everytime we write 1MB to the file. + // To use the rate limiter an io_priority smaller than IO_TOTAL can be passed. + SstFileWriter(const EnvOptions& env_options, const Options& options, + ColumnFamilyHandle* column_family = nullptr, + bool invalidate_page_cache = true, + Env::IOPriority io_priority = Env::IOPriority::IO_TOTAL) + : SstFileWriter(env_options, options, options.comparator, column_family, + invalidate_page_cache, io_priority) {} + + // Deprecated API + SstFileWriter(const EnvOptions& env_options, const Options& options, + const Comparator* user_comparator, + ColumnFamilyHandle* column_family = nullptr, + bool invalidate_page_cache = true, + Env::IOPriority io_priority = Env::IOPriority::IO_TOTAL); + + ~SstFileWriter(); + + // Prepare SstFileWriter to write into file located at "file_path". + Status Open(const std::string& file_path); + + // Add a Put key with value to currently opened file (deprecated) + // REQUIRES: key is after any previously added key according to comparator. + ROCKSDB_DEPRECATED_FUNC Status Add(const Slice& user_key, const Slice& value); + + // Add a Put key with value to currently opened file + // REQUIRES: key is after any previously added key according to comparator. + Status Put(const Slice& user_key, const Slice& value); + + // Add a Merge key with value to currently opened file + // REQUIRES: key is after any previously added key according to comparator. + Status Merge(const Slice& user_key, const Slice& value); + + // Add a deletion key to currently opened file + // REQUIRES: key is after any previously added key according to comparator. + Status Delete(const Slice& user_key); + + // Finalize writing to sst file and close file. + // + // An optional ExternalSstFileInfo pointer can be passed to the function + // which will be populated with information about the created sst file. + Status Finish(ExternalSstFileInfo* file_info = nullptr); + + // Return the current file size. + uint64_t FileSize(); + + private: + void InvalidatePageCache(bool closing); + struct Rep; + std::unique_ptr rep_; +}; +} // namespace rocksdb + +#endif // !ROCKSDB_LITE diff --git a/include/rocksdb/statistics.h b/include/rocksdb/statistics.h index 6785833b4d7..731ff780963 100644 --- a/include/rocksdb/statistics.h +++ b/include/rocksdb/statistics.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef STORAGE_ROCKSDB_INCLUDE_STATISTICS_H_ #define STORAGE_ROCKSDB_INCLUDE_STATISTICS_H_ @@ -13,12 +13,15 @@ #include #include +#include "rocksdb/status.h" + namespace rocksdb { /** * Keep adding ticker's here. * 1. Any ticker should be added before TICKER_ENUM_MAX. * 2. Add a readable string in TickersNameMap below for the newly added ticker. + * 3. Add a corresponding enum value to TickerType.java in the java API */ enum Tickers : uint32_t { // total block cache misses @@ -33,33 +36,78 @@ enum Tickers : uint32_t { BLOCK_CACHE_HIT, // # of blocks added to block cache. BLOCK_CACHE_ADD, + // # of failures when adding blocks to block cache. + BLOCK_CACHE_ADD_FAILURES, // # of times cache miss when accessing index block from block cache. BLOCK_CACHE_INDEX_MISS, // # of times cache hit when accessing index block from block cache. BLOCK_CACHE_INDEX_HIT, + // # of index blocks added to block cache. + BLOCK_CACHE_INDEX_ADD, + // # of bytes of index blocks inserted into cache + BLOCK_CACHE_INDEX_BYTES_INSERT, + // # of bytes of index block erased from cache + BLOCK_CACHE_INDEX_BYTES_EVICT, // # of times cache miss when accessing filter block from block cache. BLOCK_CACHE_FILTER_MISS, // # of times cache hit when accessing filter block from block cache. BLOCK_CACHE_FILTER_HIT, + // # of filter blocks added to block cache. + BLOCK_CACHE_FILTER_ADD, + // # of bytes of bloom filter blocks inserted into cache + BLOCK_CACHE_FILTER_BYTES_INSERT, + // # of bytes of bloom filter block erased from cache + BLOCK_CACHE_FILTER_BYTES_EVICT, // # of times cache miss when accessing data block from block cache. BLOCK_CACHE_DATA_MISS, // # of times cache hit when accessing data block from block cache. BLOCK_CACHE_DATA_HIT, + // # of data blocks added to block cache. + BLOCK_CACHE_DATA_ADD, + // # of bytes of data blocks inserted into cache + BLOCK_CACHE_DATA_BYTES_INSERT, + // # of bytes read from cache. + BLOCK_CACHE_BYTES_READ, + // # of bytes written into cache. + BLOCK_CACHE_BYTES_WRITE, + // # of times bloom filter has avoided file reads. BLOOM_FILTER_USEFUL, + // # persistent cache hit + PERSISTENT_CACHE_HIT, + // # persistent cache miss + PERSISTENT_CACHE_MISS, + + // # total simulation block cache hits + SIM_BLOCK_CACHE_HIT, + // # total simulation block cache misses + SIM_BLOCK_CACHE_MISS, + // # of memtable hits. MEMTABLE_HIT, // # of memtable misses. MEMTABLE_MISS, + // # of Get() queries served by L0 + GET_HIT_L0, + // # of Get() queries served by L1 + GET_HIT_L1, + // # of Get() queries served by L2 and up + GET_HIT_L2_AND_UP, + /** * COMPACTION_KEY_DROP_* count the reasons for key drop during compaction - * There are 3 reasons currently. + * There are 4 reasons currently. */ COMPACTION_KEY_DROP_NEWER_ENTRY, // key was written with a newer value. + // Also includes keys dropped for range del. COMPACTION_KEY_DROP_OBSOLETE, // The key is obsolete. + COMPACTION_KEY_DROP_RANGE_DEL, // key was covered by a range tombstone. COMPACTION_KEY_DROP_USER, // user compaction function has dropped the key. + COMPACTION_RANGE_DEL_DROP_OBSOLETE, // all keys in range were deleted. + // Deletions obsoleted before bottom level due to file gap optimization. + COMPACTION_OPTIMIZED_DEL_DROP_OBSOLETE, // Number of keys written to the database via the Put and Write call's NUMBER_KEYS_WRITTEN, @@ -67,18 +115,39 @@ enum Tickers : uint32_t { NUMBER_KEYS_READ, // Number keys updated, if inplace update is enabled NUMBER_KEYS_UPDATED, - // Bytes written / read + // The number of uncompressed bytes issued by DB::Put(), DB::Delete(), + // DB::Merge(), and DB::Write(). BYTES_WRITTEN, + // The number of uncompressed bytes read from DB::Get(). It could be + // either from memtables, cache, or table files. + // For the number of logical bytes read from DB::MultiGet(), + // please use NUMBER_MULTIGET_BYTES_READ. BYTES_READ, + // The number of calls to seek/next/prev + NUMBER_DB_SEEK, + NUMBER_DB_NEXT, + NUMBER_DB_PREV, + // The number of calls to seek/next/prev that returned data + NUMBER_DB_SEEK_FOUND, + NUMBER_DB_NEXT_FOUND, + NUMBER_DB_PREV_FOUND, + // The number of uncompressed bytes read from an iterator. + // Includes size of key and value. + ITER_BYTES_READ, NO_FILE_CLOSES, NO_FILE_OPENS, NO_FILE_ERRORS, - // Time system had to wait to do LO-L1 compactions + // DEPRECATED Time system had to wait to do LO-L1 compactions STALL_L0_SLOWDOWN_MICROS, - // Time system had to wait to move memtable to L1. + // DEPRECATED Time system had to wait to move memtable to L1. STALL_MEMTABLE_COMPACTION_MICROS, - // write throttle because of too many files in L0 + // DEPRECATED write throttle because of too many files in L0 STALL_L0_NUM_FILES_MICROS, + // Writer has to wait for compaction or flush to finish. + STALL_MICROS, + // The wait time for db mutex. + // Disabled by default. To enable it set stats level to kAll + DB_MUTEX_WAIT_MICROS, RATE_LIMIT_DELAY_MILLIS, NO_ITERATORS, // number of iterators currently open @@ -91,7 +160,6 @@ enum Tickers : uint32_t { // written to storage because key does not exist NUMBER_FILTERED_DELETES, NUMBER_MERGE_FAILURES, - SEQUENCE_NUMBER, // number of times bloom was checked before creating iterator on a // file, and the number of times the check was useful in avoiding @@ -108,14 +176,18 @@ enum Tickers : uint32_t { GET_UPDATES_SINCE_CALLS, BLOCK_CACHE_COMPRESSED_MISS, // miss in the compressed block cache BLOCK_CACHE_COMPRESSED_HIT, // hit in the compressed block cache - WAL_FILE_SYNCED, // Number of times WAL sync is done - WAL_FILE_BYTES, // Number of bytes written to WAL + // Number of blocks added to compressed block cache + BLOCK_CACHE_COMPRESSED_ADD, + // Number of failures when adding blocks to compressed block cache + BLOCK_CACHE_COMPRESSED_ADD_FAILURES, + WAL_FILE_SYNCED, // Number of times WAL sync is done + WAL_FILE_BYTES, // Number of bytes written to WAL // Writes can be processed by requesting thread or by the thread at the // head of the writers queue. WRITE_DONE_BY_SELF, - WRITE_DONE_BY_OTHER, - WRITE_TIMEDOUT, // Number of writes ending up with timed-out. + WRITE_DONE_BY_OTHER, // Equivalent to writes done for others + WRITE_TIMEDOUT, // Number of writes ending up with timed-out. WRITE_WITH_WAL, // Number of Write calls that request WAL COMPACT_READ_BYTES, // Bytes read during compaction COMPACT_WRITE_BYTES, // Bytes written during compaction @@ -127,7 +199,30 @@ enum Tickers : uint32_t { NUMBER_SUPERVERSION_ACQUIRES, NUMBER_SUPERVERSION_RELEASES, NUMBER_SUPERVERSION_CLEANUPS, + + // # of compressions/decompressions executed + NUMBER_BLOCK_COMPRESSED, + NUMBER_BLOCK_DECOMPRESSED, + NUMBER_BLOCK_NOT_COMPRESSED, + MERGE_OPERATION_TOTAL_TIME, + FILTER_OPERATION_TOTAL_TIME, + + // Row cache. + ROW_CACHE_HIT, + ROW_CACHE_MISS, + + // Read amplification statistics. + // Read amplification can be calculated using this formula + // (READ_AMP_TOTAL_READ_BYTES / READ_AMP_ESTIMATE_USEFUL_BYTES) + // + // REQUIRES: ReadOptions::read_amp_bytes_per_bit to be enabled + READ_AMP_ESTIMATE_USEFUL_BYTES, // Estimate of total bytes actually used. + READ_AMP_TOTAL_READ_BYTES, // Total size of loaded data blocks. + + // Number of refill intervals where rate limiter's bytes are fully consumed. + NUMBER_RATE_LIMITER_DRAINS, + TICKER_ENUM_MAX }; @@ -137,29 +232,62 @@ const std::vector> TickersNameMap = { {BLOCK_CACHE_MISS, "rocksdb.block.cache.miss"}, {BLOCK_CACHE_HIT, "rocksdb.block.cache.hit"}, {BLOCK_CACHE_ADD, "rocksdb.block.cache.add"}, + {BLOCK_CACHE_ADD_FAILURES, "rocksdb.block.cache.add.failures"}, {BLOCK_CACHE_INDEX_MISS, "rocksdb.block.cache.index.miss"}, {BLOCK_CACHE_INDEX_HIT, "rocksdb.block.cache.index.hit"}, + {BLOCK_CACHE_INDEX_ADD, "rocksdb.block.cache.index.add"}, + {BLOCK_CACHE_INDEX_BYTES_INSERT, "rocksdb.block.cache.index.bytes.insert"}, + {BLOCK_CACHE_INDEX_BYTES_EVICT, "rocksdb.block.cache.index.bytes.evict"}, {BLOCK_CACHE_FILTER_MISS, "rocksdb.block.cache.filter.miss"}, {BLOCK_CACHE_FILTER_HIT, "rocksdb.block.cache.filter.hit"}, + {BLOCK_CACHE_FILTER_ADD, "rocksdb.block.cache.filter.add"}, + {BLOCK_CACHE_FILTER_BYTES_INSERT, + "rocksdb.block.cache.filter.bytes.insert"}, + {BLOCK_CACHE_FILTER_BYTES_EVICT, "rocksdb.block.cache.filter.bytes.evict"}, {BLOCK_CACHE_DATA_MISS, "rocksdb.block.cache.data.miss"}, {BLOCK_CACHE_DATA_HIT, "rocksdb.block.cache.data.hit"}, + {BLOCK_CACHE_DATA_ADD, "rocksdb.block.cache.data.add"}, + {BLOCK_CACHE_DATA_BYTES_INSERT, "rocksdb.block.cache.data.bytes.insert"}, + {BLOCK_CACHE_BYTES_READ, "rocksdb.block.cache.bytes.read"}, + {BLOCK_CACHE_BYTES_WRITE, "rocksdb.block.cache.bytes.write"}, {BLOOM_FILTER_USEFUL, "rocksdb.bloom.filter.useful"}, + {PERSISTENT_CACHE_HIT, "rocksdb.persistent.cache.hit"}, + {PERSISTENT_CACHE_MISS, "rocksdb.persistent.cache.miss"}, + {SIM_BLOCK_CACHE_HIT, "rocksdb.sim.block.cache.hit"}, + {SIM_BLOCK_CACHE_MISS, "rocksdb.sim.block.cache.miss"}, {MEMTABLE_HIT, "rocksdb.memtable.hit"}, {MEMTABLE_MISS, "rocksdb.memtable.miss"}, + {GET_HIT_L0, "rocksdb.l0.hit"}, + {GET_HIT_L1, "rocksdb.l1.hit"}, + {GET_HIT_L2_AND_UP, "rocksdb.l2andup.hit"}, {COMPACTION_KEY_DROP_NEWER_ENTRY, "rocksdb.compaction.key.drop.new"}, {COMPACTION_KEY_DROP_OBSOLETE, "rocksdb.compaction.key.drop.obsolete"}, + {COMPACTION_KEY_DROP_RANGE_DEL, "rocksdb.compaction.key.drop.range_del"}, {COMPACTION_KEY_DROP_USER, "rocksdb.compaction.key.drop.user"}, + {COMPACTION_RANGE_DEL_DROP_OBSOLETE, + "rocksdb.compaction.range_del.drop.obsolete"}, + {COMPACTION_OPTIMIZED_DEL_DROP_OBSOLETE, + "rocksdb.compaction.optimized.del.drop.obsolete"}, {NUMBER_KEYS_WRITTEN, "rocksdb.number.keys.written"}, {NUMBER_KEYS_READ, "rocksdb.number.keys.read"}, {NUMBER_KEYS_UPDATED, "rocksdb.number.keys.updated"}, {BYTES_WRITTEN, "rocksdb.bytes.written"}, {BYTES_READ, "rocksdb.bytes.read"}, + {NUMBER_DB_SEEK, "rocksdb.number.db.seek"}, + {NUMBER_DB_NEXT, "rocksdb.number.db.next"}, + {NUMBER_DB_PREV, "rocksdb.number.db.prev"}, + {NUMBER_DB_SEEK_FOUND, "rocksdb.number.db.seek.found"}, + {NUMBER_DB_NEXT_FOUND, "rocksdb.number.db.next.found"}, + {NUMBER_DB_PREV_FOUND, "rocksdb.number.db.prev.found"}, + {ITER_BYTES_READ, "rocksdb.db.iter.bytes.read"}, {NO_FILE_CLOSES, "rocksdb.no.file.closes"}, {NO_FILE_OPENS, "rocksdb.no.file.opens"}, {NO_FILE_ERRORS, "rocksdb.no.file.errors"}, {STALL_L0_SLOWDOWN_MICROS, "rocksdb.l0.slowdown.micros"}, {STALL_MEMTABLE_COMPACTION_MICROS, "rocksdb.memtable.compaction.micros"}, {STALL_L0_NUM_FILES_MICROS, "rocksdb.l0.num.files.stall.micros"}, + {STALL_MICROS, "rocksdb.stall.micros"}, + {DB_MUTEX_WAIT_MICROS, "rocksdb.db.mutex.wait.micros"}, {RATE_LIMIT_DELAY_MILLIS, "rocksdb.rate.limit.delay.millis"}, {NO_ITERATORS, "rocksdb.num.iterators"}, {NUMBER_MULTIGET_CALLS, "rocksdb.number.multiget.get"}, @@ -167,41 +295,54 @@ const std::vector> TickersNameMap = { {NUMBER_MULTIGET_BYTES_READ, "rocksdb.number.multiget.bytes.read"}, {NUMBER_FILTERED_DELETES, "rocksdb.number.deletes.filtered"}, {NUMBER_MERGE_FAILURES, "rocksdb.number.merge.failures"}, - {SEQUENCE_NUMBER, "rocksdb.sequence.number"}, {BLOOM_FILTER_PREFIX_CHECKED, "rocksdb.bloom.filter.prefix.checked"}, {BLOOM_FILTER_PREFIX_USEFUL, "rocksdb.bloom.filter.prefix.useful"}, {NUMBER_OF_RESEEKS_IN_ITERATION, "rocksdb.number.reseeks.iteration"}, {GET_UPDATES_SINCE_CALLS, "rocksdb.getupdatessince.calls"}, {BLOCK_CACHE_COMPRESSED_MISS, "rocksdb.block.cachecompressed.miss"}, {BLOCK_CACHE_COMPRESSED_HIT, "rocksdb.block.cachecompressed.hit"}, + {BLOCK_CACHE_COMPRESSED_ADD, "rocksdb.block.cachecompressed.add"}, + {BLOCK_CACHE_COMPRESSED_ADD_FAILURES, + "rocksdb.block.cachecompressed.add.failures"}, {WAL_FILE_SYNCED, "rocksdb.wal.synced"}, {WAL_FILE_BYTES, "rocksdb.wal.bytes"}, {WRITE_DONE_BY_SELF, "rocksdb.write.self"}, {WRITE_DONE_BY_OTHER, "rocksdb.write.other"}, - {WRITE_TIMEDOUT, "rocksdb.write.timedout"}, + {WRITE_TIMEDOUT, "rocksdb.write.timeout"}, {WRITE_WITH_WAL, "rocksdb.write.wal"}, - {FLUSH_WRITE_BYTES, "rocksdb.flush.write.bytes"}, {COMPACT_READ_BYTES, "rocksdb.compact.read.bytes"}, {COMPACT_WRITE_BYTES, "rocksdb.compact.write.bytes"}, + {FLUSH_WRITE_BYTES, "rocksdb.flush.write.bytes"}, {NUMBER_DIRECT_LOAD_TABLE_PROPERTIES, "rocksdb.number.direct.load.table.properties"}, {NUMBER_SUPERVERSION_ACQUIRES, "rocksdb.number.superversion_acquires"}, {NUMBER_SUPERVERSION_RELEASES, "rocksdb.number.superversion_releases"}, {NUMBER_SUPERVERSION_CLEANUPS, "rocksdb.number.superversion_cleanups"}, + {NUMBER_BLOCK_COMPRESSED, "rocksdb.number.block.compressed"}, + {NUMBER_BLOCK_DECOMPRESSED, "rocksdb.number.block.decompressed"}, {NUMBER_BLOCK_NOT_COMPRESSED, "rocksdb.number.block.not_compressed"}, + {MERGE_OPERATION_TOTAL_TIME, "rocksdb.merge.operation.time.nanos"}, + {FILTER_OPERATION_TOTAL_TIME, "rocksdb.filter.operation.time.nanos"}, + {ROW_CACHE_HIT, "rocksdb.row.cache.hit"}, + {ROW_CACHE_MISS, "rocksdb.row.cache.miss"}, + {READ_AMP_ESTIMATE_USEFUL_BYTES, "rocksdb.read.amp.estimate.useful.bytes"}, + {READ_AMP_TOTAL_READ_BYTES, "rocksdb.read.amp.total.read.bytes"}, + {NUMBER_RATE_LIMITER_DRAINS, "rocksdb.number.rate_limiter.drains"}, }; /** * Keep adding histogram's here. - * Any histogram whould have value less than HISTOGRAM_ENUM_MAX + * Any histogram should have value less than HISTOGRAM_ENUM_MAX * Add a new Histogram by assigning it the current value of HISTOGRAM_ENUM_MAX * Add a string representation in HistogramsNameMap below * And increment HISTOGRAM_ENUM_MAX + * Add a corresponding enum value to HistogramType.java in the java API */ enum Histograms : uint32_t { DB_GET = 0, DB_WRITE, COMPACTION_TIME, + SUBCOMPACTION_SETUP_TIME, TABLE_SYNC_MICROS, COMPACTION_OUTFILE_SYNC_MICROS, WAL_FILE_SYNC_MICROS, @@ -212,7 +353,6 @@ enum Histograms : uint32_t { READ_BLOCK_COMPACTION_MICROS, READ_BLOCK_GET_MICROS, WRITE_RAW_BLOCK_MICROS, - STALL_L0_SLOWDOWN_COUNT, STALL_MEMTABLE_COMPACTION_COUNT, STALL_L0_NUM_FILES_COUNT, @@ -220,29 +360,60 @@ enum Histograms : uint32_t { SOFT_RATE_LIMIT_DELAY_COUNT, NUM_FILES_IN_SINGLE_COMPACTION, DB_SEEK, - HISTOGRAM_ENUM_MAX, + WRITE_STALL, + SST_READ_MICROS, + // The number of subcompactions actually scheduled during a compaction + NUM_SUBCOMPACTIONS_SCHEDULED, + // Value size distribution in each operation + BYTES_PER_READ, + BYTES_PER_WRITE, + BYTES_PER_MULTIGET, + + // number of bytes compressed/decompressed + // number of bytes is when uncompressed; i.e. before/after respectively + BYTES_COMPRESSED, + BYTES_DECOMPRESSED, + COMPRESSION_TIMES_NANOS, + DECOMPRESSION_TIMES_NANOS, + // Number of merge operands passed to the merge operator in user read + // requests. + READ_NUM_MERGE_OPERANDS, + + HISTOGRAM_ENUM_MAX, // TODO(ldemailly): enforce HistogramsNameMap match }; const std::vector> HistogramsNameMap = { - { DB_GET, "rocksdb.db.get.micros" }, - { DB_WRITE, "rocksdb.db.write.micros" }, - { COMPACTION_TIME, "rocksdb.compaction.times.micros" }, - { TABLE_SYNC_MICROS, "rocksdb.table.sync.micros" }, - { COMPACTION_OUTFILE_SYNC_MICROS, "rocksdb.compaction.outfile.sync.micros" }, - { WAL_FILE_SYNC_MICROS, "rocksdb.wal.file.sync.micros" }, - { MANIFEST_FILE_SYNC_MICROS, "rocksdb.manifest.file.sync.micros" }, - { TABLE_OPEN_IO_MICROS, "rocksdb.table.open.io.micros" }, - { DB_MULTIGET, "rocksdb.db.multiget.micros" }, - { READ_BLOCK_COMPACTION_MICROS, "rocksdb.read.block.compaction.micros" }, - { READ_BLOCK_GET_MICROS, "rocksdb.read.block.get.micros" }, - { WRITE_RAW_BLOCK_MICROS, "rocksdb.write.raw.block.micros" }, - { STALL_L0_SLOWDOWN_COUNT, "rocksdb.l0.slowdown.count"}, - { STALL_MEMTABLE_COMPACTION_COUNT, "rocksdb.memtable.compaction.count"}, - { STALL_L0_NUM_FILES_COUNT, "rocksdb.num.files.stall.count"}, - { HARD_RATE_LIMIT_DELAY_COUNT, "rocksdb.hard.rate.limit.delay.count"}, - { SOFT_RATE_LIMIT_DELAY_COUNT, "rocksdb.soft.rate.limit.delay.count"}, - { NUM_FILES_IN_SINGLE_COMPACTION, "rocksdb.numfiles.in.singlecompaction" }, - { DB_SEEK, "rocksdb.db.seek.micros" }, + {DB_GET, "rocksdb.db.get.micros"}, + {DB_WRITE, "rocksdb.db.write.micros"}, + {COMPACTION_TIME, "rocksdb.compaction.times.micros"}, + {SUBCOMPACTION_SETUP_TIME, "rocksdb.subcompaction.setup.times.micros"}, + {TABLE_SYNC_MICROS, "rocksdb.table.sync.micros"}, + {COMPACTION_OUTFILE_SYNC_MICROS, "rocksdb.compaction.outfile.sync.micros"}, + {WAL_FILE_SYNC_MICROS, "rocksdb.wal.file.sync.micros"}, + {MANIFEST_FILE_SYNC_MICROS, "rocksdb.manifest.file.sync.micros"}, + {TABLE_OPEN_IO_MICROS, "rocksdb.table.open.io.micros"}, + {DB_MULTIGET, "rocksdb.db.multiget.micros"}, + {READ_BLOCK_COMPACTION_MICROS, "rocksdb.read.block.compaction.micros"}, + {READ_BLOCK_GET_MICROS, "rocksdb.read.block.get.micros"}, + {WRITE_RAW_BLOCK_MICROS, "rocksdb.write.raw.block.micros"}, + {STALL_L0_SLOWDOWN_COUNT, "rocksdb.l0.slowdown.count"}, + {STALL_MEMTABLE_COMPACTION_COUNT, "rocksdb.memtable.compaction.count"}, + {STALL_L0_NUM_FILES_COUNT, "rocksdb.num.files.stall.count"}, + {HARD_RATE_LIMIT_DELAY_COUNT, "rocksdb.hard.rate.limit.delay.count"}, + {SOFT_RATE_LIMIT_DELAY_COUNT, "rocksdb.soft.rate.limit.delay.count"}, + {NUM_FILES_IN_SINGLE_COMPACTION, "rocksdb.numfiles.in.singlecompaction"}, + {DB_SEEK, "rocksdb.db.seek.micros"}, + {WRITE_STALL, "rocksdb.db.write.stall"}, + {SST_READ_MICROS, "rocksdb.sst.read.micros"}, + {NUM_SUBCOMPACTIONS_SCHEDULED, "rocksdb.num.subcompactions.scheduled"}, + {BYTES_PER_READ, "rocksdb.bytes.per.read"}, + {BYTES_PER_WRITE, "rocksdb.bytes.per.write"}, + {BYTES_PER_MULTIGET, "rocksdb.bytes.per.multiget"}, + {BYTES_COMPRESSED, "rocksdb.bytes.compressed"}, + {BYTES_DECOMPRESSED, "rocksdb.bytes.decompressed"}, + {COMPRESSION_TIMES_NANOS, "rocksdb.compression.times.nanos"}, + {DECOMPRESSION_TIMES_NANOS, "rocksdb.decompression.times.nanos"}, + {READ_NUM_MERGE_OPERANDS, "rocksdb.read.num.merge_operands"}, }; struct HistogramData { @@ -251,6 +422,22 @@ struct HistogramData { double percentile99; double average; double standard_deviation; + // zero-initialize new members since old Statistics::histogramData() + // implementations won't write them. + double max = 0.0; +}; + +enum StatsLevel { + // Collect all stats except time inside mutex lock AND time spent on + // compression. + kExceptDetailedTimers, + // Collect all stats except the counters requiring to get time inside the + // mutex lock. + kExceptTimeForMutex, + // Collect all stats, including measuring duration of mutex operations. + // If getting time is expensive on the platform to run, it can + // reduce scalability to more threads, especially for writes. + kAll, }; // Analyze the performance of a db @@ -261,11 +448,17 @@ class Statistics { virtual uint64_t getTickerCount(uint32_t tickerType) const = 0; virtual void histogramData(uint32_t type, HistogramData* const data) const = 0; - + virtual std::string getHistogramString(uint32_t type) const { return ""; } virtual void recordTick(uint32_t tickerType, uint64_t count = 0) = 0; virtual void setTickerCount(uint32_t tickerType, uint64_t count) = 0; + virtual uint64_t getAndResetTickerCount(uint32_t tickerType) = 0; virtual void measureTime(uint32_t histogramType, uint64_t time) = 0; + // Resets all ticker and histogram stats + virtual Status Reset() { + return Status::NotSupported("Not implemented"); + } + // String representation of the statistic object. virtual std::string ToString() const { // Do nothing by default @@ -276,6 +469,8 @@ class Statistics { virtual bool HistEnabledForType(uint32_t type) const { return type < HISTOGRAM_ENUM_MAX; } + + StatsLevel stats_level_ = kExceptDetailedTimers; }; // Create a concrete DBStatistics object diff --git a/include/rocksdb/status.h b/include/rocksdb/status.h index d13ff9d81f0..709f3837098 100644 --- a/include/rocksdb/status.h +++ b/include/rocksdb/status.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -25,12 +25,60 @@ namespace rocksdb { class Status { public: // Create a success status. - Status() : code_(kOk), state_(nullptr) { } + Status() : code_(kOk), subcode_(kNone), state_(nullptr) {} ~Status() { delete[] state_; } // Copy the specified status. Status(const Status& s); - void operator=(const Status& s); + Status& operator=(const Status& s); + Status(Status&& s) +#if !(defined _MSC_VER) || ((defined _MSC_VER) && (_MSC_VER >= 1900)) + noexcept +#endif + ; + Status& operator=(Status&& s) +#if !(defined _MSC_VER) || ((defined _MSC_VER) && (_MSC_VER >= 1900)) + noexcept +#endif + ; + bool operator==(const Status& rhs) const; + bool operator!=(const Status& rhs) const; + + enum Code { + kOk = 0, + kNotFound = 1, + kCorruption = 2, + kNotSupported = 3, + kInvalidArgument = 4, + kIOError = 5, + kMergeInProgress = 6, + kIncomplete = 7, + kShutdownInProgress = 8, + kTimedOut = 9, + kAborted = 10, + kBusy = 11, + kExpired = 12, + kTryAgain = 13 + }; + + Code code() const { return code_; } + + enum SubCode { + kNone = 0, + kMutexTimeout = 1, + kLockTimeout = 2, + kLockLimit = 3, + kNoSpace = 4, + kDeadlock = 5, + kStaleFile = 6, + kMemoryLimit = 7, + kMaxSubCode + }; + + SubCode subcode() const { return subcode_; } + + // Returns a C style string indicating the message of the Status + const char* getState() const { return state_; } // Return a success status. static Status OK() { return Status(); } @@ -40,38 +88,90 @@ class Status { return Status(kNotFound, msg, msg2); } // Fast path for not found without malloc; - static Status NotFound() { - return Status(kNotFound); - } + static Status NotFound(SubCode msg = kNone) { return Status(kNotFound, msg); } + static Status Corruption(const Slice& msg, const Slice& msg2 = Slice()) { return Status(kCorruption, msg, msg2); } + static Status Corruption(SubCode msg = kNone) { + return Status(kCorruption, msg); + } + static Status NotSupported(const Slice& msg, const Slice& msg2 = Slice()) { return Status(kNotSupported, msg, msg2); } + static Status NotSupported(SubCode msg = kNone) { + return Status(kNotSupported, msg); + } + static Status InvalidArgument(const Slice& msg, const Slice& msg2 = Slice()) { return Status(kInvalidArgument, msg, msg2); } + static Status InvalidArgument(SubCode msg = kNone) { + return Status(kInvalidArgument, msg); + } + static Status IOError(const Slice& msg, const Slice& msg2 = Slice()) { return Status(kIOError, msg, msg2); } + static Status IOError(SubCode msg = kNone) { return Status(kIOError, msg); } + static Status MergeInProgress(const Slice& msg, const Slice& msg2 = Slice()) { return Status(kMergeInProgress, msg, msg2); } + static Status MergeInProgress(SubCode msg = kNone) { + return Status(kMergeInProgress, msg); + } + static Status Incomplete(const Slice& msg, const Slice& msg2 = Slice()) { return Status(kIncomplete, msg, msg2); } + static Status Incomplete(SubCode msg = kNone) { + return Status(kIncomplete, msg); + } + + static Status ShutdownInProgress(SubCode msg = kNone) { + return Status(kShutdownInProgress, msg); + } static Status ShutdownInProgress(const Slice& msg, const Slice& msg2 = Slice()) { return Status(kShutdownInProgress, msg, msg2); } - static Status TimedOut() { - return Status(kTimedOut); + static Status Aborted(SubCode msg = kNone) { return Status(kAborted, msg); } + static Status Aborted(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kAborted, msg, msg2); + } + + static Status Busy(SubCode msg = kNone) { return Status(kBusy, msg); } + static Status Busy(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kBusy, msg, msg2); } + + static Status TimedOut(SubCode msg = kNone) { return Status(kTimedOut, msg); } static Status TimedOut(const Slice& msg, const Slice& msg2 = Slice()) { return Status(kTimedOut, msg, msg2); } + static Status Expired(SubCode msg = kNone) { return Status(kExpired, msg); } + static Status Expired(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kExpired, msg, msg2); + } + + static Status TryAgain(SubCode msg = kNone) { return Status(kTryAgain, msg); } + static Status TryAgain(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kTryAgain, msg, msg2); + } + + static Status NoSpace() { return Status(kIOError, kNoSpace); } + static Status NoSpace(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kIOError, kNoSpace, msg, msg2); + } + + static Status MemoryLimit() { return Status(kAborted, kMemoryLimit); } + static Status MemoryLimit(const Slice& msg, const Slice& msg2 = Slice()) { + return Status(kAborted, kMemoryLimit, msg, msg2); + } + // Returns true iff the status indicates success. bool ok() const { return code() == kOk; } @@ -101,26 +201,46 @@ class Status { bool IsTimedOut() const { return code() == kTimedOut; } + bool IsAborted() const { return code() == kAborted; } + + bool IsLockLimit() const { + return code() == kAborted && subcode() == kLockLimit; + } + + // Returns true iff the status indicates that a resource is Busy and + // temporarily could not be acquired. + bool IsBusy() const { return code() == kBusy; } + + bool IsDeadlock() const { return code() == kBusy && subcode() == kDeadlock; } + + // Returns true iff the status indicated that the operation has Expired. + bool IsExpired() const { return code() == kExpired; } + + // Returns true iff the status indicates a TryAgain error. + // This usually means that the operation failed, but may succeed if + // re-attempted. + bool IsTryAgain() const { return code() == kTryAgain; } + + // Returns true iff the status indicates a NoSpace error + // This is caused by an I/O error returning the specific "out of space" + // error condition. Stricto sensu, an NoSpace error is an I/O error + // with a specific subcode, enabling users to take the appropriate action + // if needed + bool IsNoSpace() const { + return (code() == kIOError) && (subcode() == kNoSpace); + } + + // Returns true iff the status indicates a memory limit error. There may be + // cases where we limit the memory used in certain operations (eg. the size + // of a write batch) in order to avoid out of memory exceptions. + bool IsMemoryLimit() const { + return (code() == kAborted) && (subcode() == kMemoryLimit); + } + // Return a string representation of this status suitable for printing. // Returns the string "OK" for success. std::string ToString() const; - enum Code { - kOk = 0, - kNotFound = 1, - kCorruption = 2, - kNotSupported = 3, - kInvalidArgument = 4, - kIOError = 5, - kMergeInProgress = 6, - kIncomplete = 7, - kShutdownInProgress = 8, - kTimedOut = 9 - }; - - Code code() const { - return code_; - } private: // A nullptr state_ (which is always the case for OK) means the message // is empty. @@ -128,25 +248,67 @@ class Status { // state_[0..3] == length of message // state_[4..] == message Code code_; + SubCode subcode_; const char* state_; - explicit Status(Code code) : code_(code), state_(nullptr) { } - Status(Code code, const Slice& msg, const Slice& msg2); + static const char* msgs[static_cast(kMaxSubCode)]; + + explicit Status(Code _code, SubCode _subcode = kNone) + : code_(_code), subcode_(_subcode), state_(nullptr) {} + + Status(Code _code, SubCode _subcode, const Slice& msg, const Slice& msg2); + Status(Code _code, const Slice& msg, const Slice& msg2) + : Status(_code, kNone, msg, msg2) {} + static const char* CopyState(const char* s); }; -inline Status::Status(const Status& s) { - code_ = s.code_; +inline Status::Status(const Status& s) : code_(s.code_), subcode_(s.subcode_) { state_ = (s.state_ == nullptr) ? nullptr : CopyState(s.state_); } -inline void Status::operator=(const Status& s) { +inline Status& Status::operator=(const Status& s) { // The following condition catches both aliasing (when this == &s), // and the common case where both s and *this are ok. - code_ = s.code_; - if (state_ != s.state_) { + if (this != &s) { + code_ = s.code_; + subcode_ = s.subcode_; delete[] state_; state_ = (s.state_ == nullptr) ? nullptr : CopyState(s.state_); } + return *this; +} + +inline Status::Status(Status&& s) +#if !(defined _MSC_VER) || ((defined _MSC_VER) && (_MSC_VER >= 1900)) + noexcept +#endif + : Status() { + *this = std::move(s); +} + +inline Status& Status::operator=(Status&& s) +#if !(defined _MSC_VER) || ((defined _MSC_VER) && (_MSC_VER >= 1900)) + noexcept +#endif +{ + if (this != &s) { + code_ = std::move(s.code_); + s.code_ = kOk; + subcode_ = std::move(s.subcode_); + s.subcode_ = kNone; + delete[] state_; + state_ = nullptr; + std::swap(state_, s.state_); + } + return *this; +} + +inline bool Status::operator==(const Status& rhs) const { + return (code_ == rhs.code_); +} + +inline bool Status::operator!=(const Status& rhs) const { + return !(*this == rhs); } } // namespace rocksdb diff --git a/include/rocksdb/table.h b/include/rocksdb/table.h index 0f8b41074d7..1b4c0ced90d 100644 --- a/include/rocksdb/table.h +++ b/include/rocksdb/table.h @@ -20,6 +20,7 @@ #include #include +#include "rocksdb/cache.h" #include "rocksdb/env.h" #include "rocksdb/iterator.h" #include "rocksdb/options.h" @@ -29,17 +30,20 @@ namespace rocksdb { // -- Block-based Table class FlushBlockPolicyFactory; +class PersistentCache; class RandomAccessFile; +struct TableReaderOptions; +struct TableBuilderOptions; class TableBuilder; class TableReader; -class WritableFile; +class WritableFileWriter; struct EnvOptions; struct Options; using std::unique_ptr; enum ChecksumType : char { - kNoChecksum = 0x0, // not yet supported. Will fail + kNoChecksum = 0x0, kCRC32c = 0x1, kxxHash = 0x2, }; @@ -61,6 +65,18 @@ struct BlockBasedTableOptions { // block during table initialization. bool cache_index_and_filter_blocks = false; + // If cache_index_and_filter_blocks is enabled, cache index and filter + // blocks with high priority. If set to true, depending on implementation of + // block cache, index and filter blocks may be less likely to be evicted + // than data blocks. + bool cache_index_and_filter_blocks_with_high_priority = false; + + // if cache_index_and_filter_blocks is true and the below is true, then + // filter and index blocks are stored in the cache, but a reference is + // held in the "table reader" object so the blocks are pinned and only + // evicted from cache when the table reader is freed. + bool pin_l0_filter_and_index_blocks_in_cache = false; + // The index type that will be used for this table. enum IndexType : char { // A space efficient index block that is optimized for @@ -70,14 +86,18 @@ struct BlockBasedTableOptions { // The hash index, if enabled, will do the hash lookup when // `Options.prefix_extractor` is provided. kHashSearch, + + // TODO(myabandeh): this feature is in experimental phase and shall not be + // used in production; either remove the feature or remove this comment if + // it is ready to be used in production. + // A two-level index implementation. Both levels are binary search indexes. + kTwoLevelIndexSearch, }; IndexType index_type = kBinarySearch; - // Influence the behavior when kHashSearch is used. - // if false, stores a precise prefix to block range mapping - // if true, does not store prefix and allows prefix hash collision - // (less memory consumption) + // This option is now deprecated. No matter what value it is set to, + // it will behave as if hash_index_allow_collision=true. bool hash_index_allow_collision = true; // Use the specified checksum type. Newly created table files will be @@ -94,6 +114,10 @@ struct BlockBasedTableOptions { // If NULL, rocksdb will automatically create and use an 8MB internal cache. std::shared_ptr block_cache = nullptr; + // If non-NULL use the specified cache for pages read from device + // IF NULL, no page cache is used + std::shared_ptr persistent_cache = nullptr; + // If non-NULL use the specified cache for compressed blocks. // If NULL, rocksdb will not use a compressed block cache. std::shared_ptr block_cache_compressed = nullptr; @@ -113,9 +137,39 @@ struct BlockBasedTableOptions { // Number of keys between restart points for delta encoding of keys. // This parameter can be changed dynamically. Most clients should - // leave this parameter alone. + // leave this parameter alone. The minimum value allowed is 1. Any smaller + // value will be silently overwritten with 1. int block_restart_interval = 16; + // Same as block_restart_interval but used for the index block. + int index_block_restart_interval = 1; + + // Block size for partitioned metadata. Currently applied to indexes when + // kTwoLevelIndexSearch is used and to filters when partition_filters is used. + // Note: Since in the current implementation the filters and index partitions + // are aligned, an index/filter block is created when either index or filter + // block size reaches the specified limit. + // Note: this limit is currently applied to only index blocks; a filter + // partition is cut right after an index block is cut + // TODO(myabandeh): remove the note above when filter partitions are cut + // separately + uint64_t metadata_block_size = 4096; + + // Note: currently this option requires kTwoLevelIndexSearch to be set as + // well. + // TODO(myabandeh): remove the note above once the limitation is lifted + // TODO(myabandeh): this feature is in experimental phase and shall not be + // used in production; either remove the feature or remove this comment if + // it is ready to be used in production. + // Use partitioned full filters for each SST file + bool partition_filters = false; + + // Use delta encoding to compress keys in blocks. + // ReadOptions::pin_data requires this option to be disabled. + // + // Default: true + bool use_delta_encoding = true; + // If non-nullptr, use the specified filter policy to reduce disk reads. // Many applications will benefit from passing the result of // NewBloomFilterPolicy() here. @@ -124,12 +178,60 @@ struct BlockBasedTableOptions { // If true, place whole keys in the filter (not just prefixes). // This must generally be true for gets to be efficient. bool whole_key_filtering = true; + + // Verify that decompressing the compressed block gives back the input. This + // is a verification mode that we use to detect bugs in compression + // algorithms. + bool verify_compression = false; + + // If used, For every data block we load into memory, we will create a bitmap + // of size ((block_size / `read_amp_bytes_per_bit`) / 8) bytes. This bitmap + // will be used to figure out the percentage we actually read of the blocks. + // + // When this feature is used Tickers::READ_AMP_ESTIMATE_USEFUL_BYTES and + // Tickers::READ_AMP_TOTAL_READ_BYTES can be used to calculate the + // read amplification using this formula + // (READ_AMP_TOTAL_READ_BYTES / READ_AMP_ESTIMATE_USEFUL_BYTES) + // + // value => memory usage (percentage of loaded blocks memory) + // 1 => 12.50 % + // 2 => 06.25 % + // 4 => 03.12 % + // 8 => 01.56 % + // 16 => 00.78 % + // + // Note: This number must be a power of 2, if not it will be sanitized + // to be the next lowest power of 2, for example a value of 7 will be + // treated as 4, a value of 19 will be treated as 16. + // + // Default: 0 (disabled) + uint32_t read_amp_bytes_per_bit = 0; + + // We currently have three versions: + // 0 -- This version is currently written out by all RocksDB's versions by + // default. Can be read by really old RocksDB's. Doesn't support changing + // checksum (default is CRC32). + // 1 -- Can be read by RocksDB's versions since 3.0. Supports non-default + // checksum, like xxHash. It is written by RocksDB when + // BlockBasedTableOptions::checksum is something other than kCRC32c. (version + // 0 is silently upconverted) + // 2 -- Can be read by RocksDB's versions since 3.10. Changes the way we + // encode compressed blocks with LZ4, BZip2 and Zlib compression. If you + // don't plan to run RocksDB before version 3.10, you should probably use + // this. + // This option only affects newly written tables. When reading exising tables, + // the information about version is read from the footer. + uint32_t format_version = 2; }; // Table Properties that are specific to block-based table properties. struct BlockBasedTablePropertyNames { // value of this propertis is a fixed int32 number. static const std::string kIndexType; + // value is "1" for true and "0" for false. + static const std::string kWholeKeyFiltering; + // value is "1" for true and "0" for false. + static const std::string kPrefixFiltering; }; // Create default block based table factory. @@ -157,7 +259,6 @@ enum EncodingType : char { // Table Properties that are specific to plain table properties. struct PlainTablePropertyNames { - static const std::string kPrefixExtractorName; static const std::string kEncodingType; static const std::string kBloomVersion; static const std::string kNumBloomBlocks; @@ -250,26 +351,51 @@ struct CuckooTablePropertyNames { // Denotes if the key sorted in the file is Internal Key (if false) // or User Key only (if true). static const std::string kIsLastLevel; + // Indicate if using identity function for the first hash function. + static const std::string kIdentityAsFirstHash; + // Indicate if using module or bit and to calculate hash value + static const std::string kUseModuleHash; + // Fixed user key length + static const std::string kUserKeyLength; +}; + +struct CuckooTableOptions { + // Determines the utilization of hash tables. Smaller values + // result in larger hash tables with fewer collisions. + double hash_table_ratio = 0.9; + // A property used by builder to determine the depth to go to + // to search for a path to displace elements in case of + // collision. See Builder.MakeSpaceForKey method. Higher + // values result in more efficient hash tables with fewer + // lookups but take more time to build. + uint32_t max_search_depth = 100; + // In case of collision while inserting, the builder + // attempts to insert in the next cuckoo_block_size + // locations before skipping over to the next Cuckoo hash + // function. This makes lookups more cache friendly in case + // of collisions. + uint32_t cuckoo_block_size = 5; + // If this option is enabled, user key is treated as uint64_t and its value + // is used as hash value directly. This option changes builder's behavior. + // Reader ignore this option and behave according to what specified in table + // property. + bool identity_as_first_hash = false; + // If this option is set to true, module is used during hash calculation. + // This often yields better space efficiency at the cost of performance. + // If this optino is set to false, # of entries in table is constrained to be + // power of two, and bit and is used to calculate hash, which is faster in + // general. + bool use_module_hash = true; }; // Cuckoo Table Factory for SST table format using Cache Friendly Cuckoo Hashing -// @hash_table_ratio: Determines the utilization of hash tables. Smaller values -// result in larger hash tables with fewer collisions. -// @max_search_depth: A property used by builder to determine the depth to go to -// to search for a path to displace elements in case of -// collision. See Builder.MakeSpaceForKey method. Higher -// values result in more efficient hash tables with fewer -// lookups but take more time to build. -// @cuckoo_block_size: In case of collision while inserting, the builder -// attempts to insert in the next cuckoo_block_size -// locations before skipping over to the next Cuckoo hash -// function. This makes lookups more cache friendly in case -// of collisions. -extern TableFactory* NewCuckooTableFactory(double hash_table_ratio = 0.9, - uint32_t max_search_depth = 100, uint32_t cuckoo_block_size = 5); +extern TableFactory* NewCuckooTableFactory( + const CuckooTableOptions& table_options = CuckooTableOptions()); #endif // ROCKSDB_LITE +class RandomAccessFileReader; + // A base class for table factories. class TableFactory { public: @@ -288,22 +414,24 @@ class TableFactory { // in parameter file. It's the caller's responsibility to make sure // file is in the correct format. // - // NewTableReader() is called in two places: + // NewTableReader() is called in three places: // (1) TableCache::FindTable() calls the function when table cache miss // and cache the table object returned. - // (1) SstFileReader (for SST Dump) opens the table and dump the table - // contents using the interator of the table. - // options and soptions are options. options is the general options. - // Multiple configured can be accessed from there, including and not - // limited to block cache and key comparators. - // file is a file handler to handle the file for the table - // file_size is the physical file size of the file - // table_reader is the output table reader + // (2) SstFileReader (for SST Dump) opens the table and dump the table + // contents using the iterator of the table. + // (3) DBImpl::AddFile() calls this function to read the contents of + // the sst file it's attempting to add + // + // table_reader_options is a TableReaderOptions which contain all the + // needed parameters and configuration to open the table. + // file is a file handler to handle the file for the table. + // file_size is the physical file size of the file. + // table_reader is the output table reader. virtual Status NewTableReader( - const Options& options, const EnvOptions& soptions, - const InternalKeyComparator& internal_comparator, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table_reader) const = 0; + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + unique_ptr* table_reader, + bool prefetch_index_and_filter_in_cache = true) const = 0; // Return a table builder to write to a file for this table type. // @@ -318,34 +446,62 @@ class TableFactory { // (4) When running Repairer, it creates a table builder to convert logs to // SST files (In Repairer::ConvertLogToTable() by calling BuildTable()) // - // options is the general options. Multiple configured can be acceseed from - // there, including and not limited to compression options. - // file is a handle of a writable file. It is the caller's responsibility to - // keep the file open and close the file after closing the table builder. - // compression_type is the compression type to use in this table. + // Multiple configured can be accessed from there, including and not limited + // to compression options. file is a handle of a writable file. + // It is the caller's responsibility to keep the file open and close the file + // after closing the table builder. compression_type is the compression type + // to use in this table. virtual TableBuilder* NewTableBuilder( - const Options& options, const InternalKeyComparator& internal_comparator, - WritableFile* file, CompressionType compression_type) const = 0; + const TableBuilderOptions& table_builder_options, + uint32_t column_family_id, WritableFileWriter* file) const = 0; - // Sanitizes the specified DB Options. + // Sanitizes the specified DB Options and ColumnFamilyOptions. // // If the function cannot find a way to sanitize the input DB Options, // a non-ok Status will be returned. - virtual Status SanitizeDBOptions(const DBOptions* db_opts) const = 0; + virtual Status SanitizeOptions( + const DBOptions& db_opts, + const ColumnFamilyOptions& cf_opts) const = 0; // Return a string that contains printable format of table configurations. // RocksDB prints configurations at DB Open(). virtual std::string GetPrintableTableOptions() const = 0; + + virtual Status GetOptionString(std::string* opt_string, + const std::string& delimiter) const { + return Status::NotSupported( + "The table factory doesn't implement GetOptionString()."); + } + + // Returns the raw pointer of the table options that is used by this + // TableFactory, or nullptr if this function is not supported. + // Since the return value is a raw pointer, the TableFactory owns the + // pointer and the caller should not delete the pointer. + // + // In certain case, it is desirable to alter the underlying options when the + // TableFactory is not used by any open DB by casting the returned pointer + // to the right class. For instance, if BlockBasedTableFactory is used, + // then the pointer can be casted to BlockBasedTableOptions. + // + // Note that changing the underlying TableFactory options while the + // TableFactory is currently used by any open DB is undefined behavior. + // Developers should use DB::SetOption() instead to dynamically change + // options while the DB is open. + virtual void* GetOptions() { return nullptr; } + + // Return is delete range supported + virtual bool IsDeleteRangeSupported() const { return false; } }; #ifndef ROCKSDB_LITE -// Create a special table factory that can open both of block based table format -// and plain table, based on setting inside the SST files. It should be used to +// Create a special table factory that can open either of the supported +// table formats, based on setting inside the SST files. It should be used to // convert a DB from one table format to another. // @table_factory_to_write: the table factory used when writing to new files. // @block_based_table_factory: block based table factory to use. If NULL, use // a default one. // @plain_table_factory: plain table factory to use. If NULL, use a default one. +// @cuckoo_table_factory: cuckoo table factory to use. If NULL, use a default one. extern TableFactory* NewAdaptiveTableFactory( std::shared_ptr table_factory_to_write = nullptr, std::shared_ptr block_based_table_factory = nullptr, diff --git a/include/rocksdb/table_properties.h b/include/rocksdb/table_properties.h index d6b3f4d7b5d..2605fadd257 100644 --- a/include/rocksdb/table_properties.h +++ b/include/rocksdb/table_properties.h @@ -3,9 +3,11 @@ // found in the LICENSE file. See the AUTHORS file for names of contributors. #pragma once -#include +#include #include +#include #include "rocksdb/status.h" +#include "rocksdb/types.h" namespace rocksdb { @@ -23,48 +25,14 @@ namespace rocksdb { // ++pos) { // ... // } -typedef std::map UserCollectedProperties; - -// TableProperties contains a bunch of read-only properties of its associated -// table. -struct TableProperties { - public: - // the total size of all data blocks. - uint64_t data_size = 0; - // the size of index block. - uint64_t index_size = 0; - // the size of filter block. - uint64_t filter_size = 0; - // total raw key size - uint64_t raw_key_size = 0; - // total raw value size - uint64_t raw_value_size = 0; - // the number of blocks in this table - uint64_t num_data_blocks = 0; - // the number of entries in this table - uint64_t num_entries = 0; - // format version, reserved for backward compatibility - uint64_t format_version = 0; - // If 0, key is variable length. Otherwise number of bytes for each key. - uint64_t fixed_key_len = 0; - - // The name of the filter policy used in this table. - // If no filter policy is used, `filter_policy_name` will be an empty string. - std::string filter_policy_name; - - // user collected properties - UserCollectedProperties user_collected_properties; - - // convert this object to a human readable form - // @prop_delim: delimiter for each property. - std::string ToString(const std::string& prop_delim = "; ", - const std::string& kv_delim = "=") const; -}; +typedef std::map UserCollectedProperties; // table properties' human-readable names in the property block. struct TablePropertiesNames { static const std::string kDataSize; static const std::string kIndexSize; + static const std::string kIndexPartitions; + static const std::string kTopLevelIndexSize; static const std::string kFilterSize; static const std::string kRawKeySize; static const std::string kRawValueSize; @@ -73,24 +41,59 @@ struct TablePropertiesNames { static const std::string kFormatVersion; static const std::string kFixedKeyLen; static const std::string kFilterPolicy; + static const std::string kColumnFamilyName; + static const std::string kColumnFamilyId; + static const std::string kComparator; + static const std::string kMergeOperator; + static const std::string kPrefixExtractorName; + static const std::string kPropertyCollectors; + static const std::string kCompression; + static const std::string kCreationTime; + static const std::string kOldestKeyTime; }; extern const std::string kPropertiesBlock; +extern const std::string kCompressionDictBlock; +extern const std::string kRangeDelBlock; + +enum EntryType { + kEntryPut, + kEntryDelete, + kEntrySingleDelete, + kEntryMerge, + kEntryOther, +}; // `TablePropertiesCollector` provides the mechanism for users to collect -// their own interested properties. This class is essentially a collection -// of callback functions that will be invoked during table building. -// It is construced with TablePropertiesCollectorFactory. The methods don't -// need to be thread-safe, as we will create exactly one +// their own properties that they are interested in. This class is essentially +// a collection of callback functions that will be invoked during table +// building. It is construced with TablePropertiesCollectorFactory. The methods +// don't need to be thread-safe, as we will create exactly one // TablePropertiesCollector object per table and then call it sequentially class TablePropertiesCollector { public: virtual ~TablePropertiesCollector() {} + // DEPRECATE User defined collector should implement AddUserKey(), though + // this old function still works for backward compatible reason. // Add() will be called when a new key/value pair is inserted into the table. - // @params key the original key that is inserted into the table. - // @params value the original value that is inserted into the table. - virtual Status Add(const Slice& key, const Slice& value) = 0; + // @params key the user key that is inserted into the table. + // @params value the value that is inserted into the table. + virtual Status Add(const Slice& /*key*/, const Slice& /*value*/) { + return Status::InvalidArgument( + "TablePropertiesCollector::Add() deprecated."); + } + + // AddUserKey() will be called when a new key/value pair is inserted into the + // table. + // @params key the user key that is inserted into the table. + // @params value the value that is inserted into the table. + virtual Status AddUserKey(const Slice& key, const Slice& value, + EntryType /*type*/, SequenceNumber /*seq*/, + uint64_t /*file_size*/) { + // For backwards-compatibility. + return Add(key, value); + } // Finish() will be called when a table has already been built and is ready // for writing the properties block. @@ -104,24 +107,115 @@ class TablePropertiesCollector { // The name of the properties collector can be used for debugging purpose. virtual const char* Name() const = 0; + + // EXPERIMENTAL Return whether the output file should be further compacted + virtual bool NeedCompact() const { return false; } }; // Constructs TablePropertiesCollector. Internals create a new // TablePropertiesCollector for each new table class TablePropertiesCollectorFactory { public: + struct Context { + uint32_t column_family_id; + static const uint32_t kUnknownColumnFamily; + }; + virtual ~TablePropertiesCollectorFactory() {} // has to be thread-safe - virtual TablePropertiesCollector* CreateTablePropertiesCollector() = 0; + virtual TablePropertiesCollector* CreateTablePropertiesCollector( + TablePropertiesCollectorFactory::Context context) = 0; // The name of the properties collector can be used for debugging purpose. virtual const char* Name() const = 0; }; +// TableProperties contains a bunch of read-only properties of its associated +// table. +struct TableProperties { + public: + // the total size of all data blocks. + uint64_t data_size = 0; + // the size of index block. + uint64_t index_size = 0; + // Total number of index partitions if kTwoLevelIndexSearch is used + uint64_t index_partitions = 0; + // Size of the top-level index if kTwoLevelIndexSearch is used + uint64_t top_level_index_size = 0; + // the size of filter block. + uint64_t filter_size = 0; + // total raw key size + uint64_t raw_key_size = 0; + // total raw value size + uint64_t raw_value_size = 0; + // the number of blocks in this table + uint64_t num_data_blocks = 0; + // the number of entries in this table + uint64_t num_entries = 0; + // format version, reserved for backward compatibility + uint64_t format_version = 0; + // If 0, key is variable length. Otherwise number of bytes for each key. + uint64_t fixed_key_len = 0; + // ID of column family for this SST file, corresponding to the CF identified + // by column_family_name. + uint64_t column_family_id = + rocksdb::TablePropertiesCollectorFactory::Context::kUnknownColumnFamily; + // The time when the SST file was created. + // Since SST files are immutable, this is equivalent to last modified time. + uint64_t creation_time = 0; + // Timestamp of the earliest key. 0 means unknown. + uint64_t oldest_key_time = 0; + + // Name of the column family with which this SST file is associated. + // If column family is unknown, `column_family_name` will be an empty string. + std::string column_family_name; + + // The name of the filter policy used in this table. + // If no filter policy is used, `filter_policy_name` will be an empty string. + std::string filter_policy_name; + + // The name of the comparator used in this table. + std::string comparator_name; + + // The name of the merge operator used in this table. + // If no merge operator is used, `merge_operator_name` will be "nullptr". + std::string merge_operator_name; + + // The name of the prefix extractor used in this table + // If no prefix extractor is used, `prefix_extractor_name` will be "nullptr". + std::string prefix_extractor_name; + + // The names of the property collectors factories used in this table + // separated by commas + // {collector_name[1]},{collector_name[2]},{collector_name[3]} .. + std::string property_collectors_names; + + // The compression algo used to compress the SST files. + std::string compression_name; + + // user collected properties + UserCollectedProperties user_collected_properties; + UserCollectedProperties readable_properties; + + // The offset of the value of each property in the file. + std::map properties_offsets; + + // convert this object to a human readable form + // @prop_delim: delimiter for each property. + std::string ToString(const std::string& prop_delim = "; ", + const std::string& kv_delim = "=") const; + + // Aggregate the numerical member variables of the specified + // TableProperties. + void Add(const TableProperties& tp); +}; + // Extra properties // Below is a list of non-basic properties that are collected by database // itself. Especially some properties regarding to the internal keys (which // is unknown to `table`). extern uint64_t GetDeletedKeys(const UserCollectedProperties& props); +extern uint64_t GetMergeOperands(const UserCollectedProperties& props, + bool* property_present); } // namespace rocksdb diff --git a/include/rocksdb/thread_status.h b/include/rocksdb/thread_status.h new file mode 100644 index 00000000000..55c32ed6d2f --- /dev/null +++ b/include/rocksdb/thread_status.h @@ -0,0 +1,193 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file defines the structures for exposing run-time status of any +// rocksdb-related thread. Such run-time status can be obtained via +// GetThreadList() API. +// +// Note that all thread-status features are still under-development, and +// thus APIs and class definitions might subject to change at this point. +// Will remove this comment once the APIs have been finalized. + +#pragma once + +#include +#include +#include +#include +#include +#include + +#if !defined(ROCKSDB_LITE) && \ + !defined(NROCKSDB_THREAD_STATUS) && \ + defined(ROCKSDB_SUPPORT_THREAD_LOCAL) +#define ROCKSDB_USING_THREAD_STATUS +#endif + +namespace rocksdb { + +// TODO(yhchiang): remove this function once c++14 is available +// as std::max will be able to cover this. +// Current MS compiler does not support constexpr +template +struct constexpr_max { + static const int result = (A > B) ? A : B; +}; + +// A structure that describes the current status of a thread. +// The status of active threads can be fetched using +// rocksdb::GetThreadList(). +struct ThreadStatus { + // The type of a thread. + enum ThreadType : int { + HIGH_PRIORITY = 0, // RocksDB BG thread in high-pri thread pool + LOW_PRIORITY, // RocksDB BG thread in low-pri thread pool + USER, // User thread (Non-RocksDB BG thread) + NUM_THREAD_TYPES + }; + + // The type used to refer to a thread operation. + // A thread operation describes high-level action of a thread. + // Examples include compaction and flush. + enum OperationType : int { + OP_UNKNOWN = 0, + OP_COMPACTION, + OP_FLUSH, + NUM_OP_TYPES + }; + + enum OperationStage : int { + STAGE_UNKNOWN = 0, + STAGE_FLUSH_RUN, + STAGE_FLUSH_WRITE_L0, + STAGE_COMPACTION_PREPARE, + STAGE_COMPACTION_RUN, + STAGE_COMPACTION_PROCESS_KV, + STAGE_COMPACTION_INSTALL, + STAGE_COMPACTION_SYNC_FILE, + STAGE_PICK_MEMTABLES_TO_FLUSH, + STAGE_MEMTABLE_ROLLBACK, + STAGE_MEMTABLE_INSTALL_FLUSH_RESULTS, + NUM_OP_STAGES + }; + + enum CompactionPropertyType : int { + COMPACTION_JOB_ID = 0, + COMPACTION_INPUT_OUTPUT_LEVEL, + COMPACTION_PROP_FLAGS, + COMPACTION_TOTAL_INPUT_BYTES, + COMPACTION_BYTES_READ, + COMPACTION_BYTES_WRITTEN, + NUM_COMPACTION_PROPERTIES + }; + + enum FlushPropertyType : int { + FLUSH_JOB_ID = 0, + FLUSH_BYTES_MEMTABLES, + FLUSH_BYTES_WRITTEN, + NUM_FLUSH_PROPERTIES + }; + + // The maximum number of properties of an operation. + // This number should be set to the biggest NUM_XXX_PROPERTIES. + static const int kNumOperationProperties = + constexpr_max::result; + + // The type used to refer to a thread state. + // A state describes lower-level action of a thread + // such as reading / writing a file or waiting for a mutex. + enum StateType : int { + STATE_UNKNOWN = 0, + STATE_MUTEX_WAIT = 1, + NUM_STATE_TYPES + }; + + ThreadStatus(const uint64_t _id, + const ThreadType _thread_type, + const std::string& _db_name, + const std::string& _cf_name, + const OperationType _operation_type, + const uint64_t _op_elapsed_micros, + const OperationStage _operation_stage, + const uint64_t _op_props[], + const StateType _state_type) : + thread_id(_id), thread_type(_thread_type), + db_name(_db_name), + cf_name(_cf_name), + operation_type(_operation_type), + op_elapsed_micros(_op_elapsed_micros), + operation_stage(_operation_stage), + state_type(_state_type) { + for (int i = 0; i < kNumOperationProperties; ++i) { + op_properties[i] = _op_props[i]; + } + } + + // An unique ID for the thread. + const uint64_t thread_id; + + // The type of the thread, it could be HIGH_PRIORITY, + // LOW_PRIORITY, and USER + const ThreadType thread_type; + + // The name of the DB instance where the thread is currently + // involved with. It would be set to empty string if the thread + // does not involve in any DB operation. + const std::string db_name; + + // The name of the column family where the thread is currently + // It would be set to empty string if the thread does not involve + // in any column family. + const std::string cf_name; + + // The operation (high-level action) that the current thread is involved. + const OperationType operation_type; + + // The elapsed time of the current thread operation in microseconds. + const uint64_t op_elapsed_micros; + + // An integer showing the current stage where the thread is involved + // in the current operation. + const OperationStage operation_stage; + + // A list of properties that describe some details about the current + // operation. Same field in op_properties[] might have different + // meanings for different operations. + uint64_t op_properties[kNumOperationProperties]; + + // The state (lower-level action) that the current thread is involved. + const StateType state_type; + + // The followings are a set of utility functions for interpreting + // the information of ThreadStatus + + static const std::string& GetThreadTypeName(ThreadType thread_type); + + // Obtain the name of an operation given its type. + static const std::string& GetOperationName(OperationType op_type); + + static const std::string MicrosToString(uint64_t op_elapsed_time); + + // Obtain a human-readable string describing the specified operation stage. + static const std::string& GetOperationStageName( + OperationStage stage); + + // Obtain the name of the "i"th operation property of the + // specified operation. + static const std::string& GetOperationPropertyName( + OperationType op_type, int i); + + // Translate the "i"th property of the specified operation given + // a property value. + static std::map + InterpretOperationProperties( + OperationType op_type, const uint64_t* op_properties); + + // Obtain the name of a state given its type. + static const std::string& GetStateName(StateType state_type); +}; + + +} // namespace rocksdb diff --git a/include/rocksdb/threadpool.h b/include/rocksdb/threadpool.h new file mode 100644 index 00000000000..e871ee18c7a --- /dev/null +++ b/include/rocksdb/threadpool.h @@ -0,0 +1,57 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#pragma once + +#include + +namespace rocksdb { + +/* + * ThreadPool is a component that will spawn N background threads that will + * be used to execute scheduled work, The number of background threads could + * be modified by calling SetBackgroundThreads(). + * */ +class ThreadPool { + public: + virtual ~ThreadPool() {} + + // Wait for all threads to finish. + // Discard those threads that did not start + // executing + virtual void JoinAllThreads() = 0; + + // Set the number of background threads that will be executing the + // scheduled jobs. + virtual void SetBackgroundThreads(int num) = 0; + virtual int GetBackgroundThreads() = 0; + + // Get the number of jobs scheduled in the ThreadPool queue. + virtual unsigned int GetQueueLen() const = 0; + + // Waits for all jobs to complete those + // that already started running and those that did not + // start yet. This ensures that everything that was thrown + // on the TP runs even though + // we may not have specified enough threads for the amount + // of jobs + virtual void WaitForJobsAndJoinAllThreads() = 0; + + // Submit a fire and forget jobs + // This allows to submit the same job multiple times + virtual void SubmitJob(const std::function&) = 0; + // This moves the function in for efficiency + virtual void SubmitJob(std::function&&) = 0; + +}; + +// NewThreadPool() is a function that could be used to create a ThreadPool +// with `num_threads` background threads. +extern ThreadPool* NewThreadPool(int num_threads); + +} // namespace rocksdb diff --git a/include/rocksdb/transaction_log.h b/include/rocksdb/transaction_log.h index 30443bba557..7fc46ae2645 100644 --- a/include/rocksdb/transaction_log.h +++ b/include/rocksdb/transaction_log.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef STORAGE_ROCKSDB_INCLUDE_TRANSACTION_LOG_ITERATOR_H_ #define STORAGE_ROCKSDB_INCLUDE_TRANSACTION_LOG_ITERATOR_H_ @@ -58,6 +58,27 @@ class LogFile { struct BatchResult { SequenceNumber sequence = 0; std::unique_ptr writeBatchPtr; + + // Add empty __ctor and __dtor for the rule of five + // However, preserve the original semantics and prohibit copying + // as the unique_ptr member does not copy. + BatchResult() {} + + ~BatchResult() {} + + BatchResult(const BatchResult&) = delete; + + BatchResult& operator=(const BatchResult&) = delete; + + BatchResult(BatchResult&& bResult) + : sequence(std::move(bResult.sequence)), + writeBatchPtr(std::move(bResult.writeBatchPtr)) {} + + BatchResult& operator=(BatchResult&& bResult) { + sequence = std::move(bResult.sequence); + writeBatchPtr = std::move(bResult.writeBatchPtr); + return *this; + } }; // A TransactionLogIterator is used to iterate over the transactions in a db. diff --git a/include/rocksdb/types.h b/include/rocksdb/types.h index f20bf8277fe..106ac2f76bf 100644 --- a/include/rocksdb/types.h +++ b/include/rocksdb/types.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef STORAGE_ROCKSDB_INCLUDE_TYPES_H_ #define STORAGE_ROCKSDB_INCLUDE_TYPES_H_ diff --git a/include/rocksdb/universal_compaction.h b/include/rocksdb/universal_compaction.h index 229e50b25c1..ed2220873cd 100644 --- a/include/rocksdb/universal_compaction.h +++ b/include/rocksdb/universal_compaction.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef STORAGE_ROCKSDB_UNIVERSAL_COMPACTION_OPTIONS_H #define STORAGE_ROCKSDB_UNIVERSAL_COMPACTION_OPTIONS_H @@ -24,7 +24,7 @@ enum CompactionStopStyle { class CompactionOptionsUniversal { public: - // Percentage flexibilty while comparing file size. If the candidate file(s) + // Percentage flexibility while comparing file size. If the candidate file(s) // size is 1% smaller than the next file's size, then include next file into // this candidate set. // Default: 1 unsigned int size_ratio; @@ -69,6 +69,11 @@ class CompactionOptionsUniversal { // Default: kCompactionStopStyleTotalSize CompactionStopStyle stop_style; + // Option to optimize the universal multi level compaction by enabling + // trivial move for non overlapping files. + // Default: false + bool allow_trivial_move; + // Default set of parameters CompactionOptionsUniversal() : size_ratio(1), @@ -76,7 +81,8 @@ class CompactionOptionsUniversal { max_merge_width(UINT_MAX), max_size_amplification_percent(200), compression_size_percent(-1), - stop_style(kCompactionStopStyleTotalSize) {} + stop_style(kCompactionStopStyleTotalSize), + allow_trivial_move(false) {} }; } // namespace rocksdb diff --git a/include/rocksdb/utilities/backupable_db.h b/include/rocksdb/utilities/backupable_db.h index 78365769d2e..fc2b6ba43f7 100644 --- a/include/rocksdb/utilities/backupable_db.h +++ b/include/rocksdb/utilities/backupable_db.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,11 +10,15 @@ #pragma once #ifndef ROCKSDB_LITE +#ifndef __STDC_FORMAT_MACROS #define __STDC_FORMAT_MACROS +#endif + #include #include #include #include +#include #include "rocksdb/utilities/stackable_db.h" @@ -71,11 +75,21 @@ struct BackupableDBOptions { // Default: 0 uint64_t backup_rate_limit; + // Backup rate limiter. Used to control transfer speed for backup. If this is + // not null, backup_rate_limit is ignored. + // Default: nullptr + std::shared_ptr backup_rate_limiter{nullptr}; + // Max bytes that can be transferred in a second during restore. // If 0, go as fast as you can // Default: 0 uint64_t restore_rate_limit; + // Restore rate limiter. Used to control transfer speed during restore. If + // this is not null, restore_rate_limit is ignored. + // Default: nullptr + std::shared_ptr restore_rate_limiter{nullptr}; + // Only used if share_table_files is set to true. If true, will consider that // backups can come from different databases, hence a sst is not uniquely // identifed by its name, but by the triple (file name, crc32, file length) @@ -84,16 +98,31 @@ struct BackupableDBOptions { // *turn it on only if you know what you're doing* bool share_files_with_checksum; + // Up to this many background threads will copy files for CreateNewBackup() + // and RestoreDBFromBackup() + // Default: 1 + int max_background_operations; + + // During backup user can get callback every time next + // callback_trigger_interval_size bytes being copied. + // Default: 4194304 + uint64_t callback_trigger_interval_size; + + // When Open() is called, it will open at most this many of the latest + // non-corrupted backups. If 0, it will open all available backups. + // Default: 0 + int max_valid_backups_to_open; + void Dump(Logger* logger) const; - explicit BackupableDBOptions(const std::string& _backup_dir, - Env* _backup_env = nullptr, - bool _share_table_files = true, - Logger* _info_log = nullptr, bool _sync = true, - bool _destroy_old_data = false, - bool _backup_log_files = true, - uint64_t _backup_rate_limit = 0, - uint64_t _restore_rate_limit = 0) + explicit BackupableDBOptions( + const std::string& _backup_dir, Env* _backup_env = nullptr, + bool _share_table_files = true, Logger* _info_log = nullptr, + bool _sync = true, bool _destroy_old_data = false, + bool _backup_log_files = true, uint64_t _backup_rate_limit = 0, + uint64_t _restore_rate_limit = 0, int _max_background_operations = 1, + uint64_t _callback_trigger_interval_size = 4 * 1024 * 1024, + int _max_valid_backups_to_open = 0) : backup_dir(_backup_dir), backup_env(_backup_env), share_table_files(_share_table_files), @@ -103,7 +132,10 @@ struct BackupableDBOptions { backup_log_files(_backup_log_files), backup_rate_limit(_backup_rate_limit), restore_rate_limit(_restore_rate_limit), - share_files_with_checksum(false) { + share_files_with_checksum(false), + max_background_operations(_max_background_operations), + callback_trigger_interval_size(_callback_trigger_interval_size), + max_valid_backups_to_open(_max_valid_backups_to_open) { assert(share_table_files || !share_files_with_checksum); } }; @@ -127,76 +159,124 @@ struct BackupInfo { int64_t timestamp; uint64_t size; + uint32_t number_files; + std::string app_metadata; + BackupInfo() {} - BackupInfo(BackupID _backup_id, int64_t _timestamp, uint64_t _size) - : backup_id(_backup_id), timestamp(_timestamp), size(_size) {} + + BackupInfo(BackupID _backup_id, int64_t _timestamp, uint64_t _size, + uint32_t _number_files, const std::string& _app_metadata) + : backup_id(_backup_id), + timestamp(_timestamp), + size(_size), + number_files(_number_files), + app_metadata(_app_metadata) {} +}; + +class BackupStatistics { + public: + BackupStatistics() { + number_success_backup = 0; + number_fail_backup = 0; + } + + BackupStatistics(uint32_t _number_success_backup, + uint32_t _number_fail_backup) + : number_success_backup(_number_success_backup), + number_fail_backup(_number_fail_backup) {} + + ~BackupStatistics() {} + + void IncrementNumberSuccessBackup(); + void IncrementNumberFailBackup(); + + uint32_t GetNumberSuccessBackup() const; + uint32_t GetNumberFailBackup() const; + + std::string ToString() const; + + private: + uint32_t number_success_backup; + uint32_t number_fail_backup; }; +// A backup engine for accessing information about backups and restoring from +// them. class BackupEngineReadOnly { public: virtual ~BackupEngineReadOnly() {} - static BackupEngineReadOnly* NewReadOnlyBackupEngine( - Env* db_env, const BackupableDBOptions& options); + static Status Open(Env* db_env, const BackupableDBOptions& options, + BackupEngineReadOnly** backup_engine_ptr); + // Returns info about backups in backup_info // You can GetBackupInfo safely, even with other BackupEngine performing // backups on the same directory virtual void GetBackupInfo(std::vector* backup_info) = 0; + // Returns info about corrupt backups in corrupt_backups + virtual void GetCorruptedBackups( + std::vector* corrupt_backup_ids) = 0; + // Restoring DB from backup is NOT safe when there is another BackupEngine // running that might call DeleteBackup() or PurgeOldBackups(). It is caller's // responsibility to synchronize the operation, i.e. don't delete the backup // when you're restoring from it + // See also the corresponding doc in BackupEngine virtual Status RestoreDBFromBackup( BackupID backup_id, const std::string& db_dir, const std::string& wal_dir, const RestoreOptions& restore_options = RestoreOptions()) = 0; + + // See the corresponding doc in BackupEngine virtual Status RestoreDBFromLatestBackup( const std::string& db_dir, const std::string& wal_dir, const RestoreOptions& restore_options = RestoreOptions()) = 0; + + // checks that each file exists and that the size of the file matches our + // expectations. it does not check file checksum. + // + // If this BackupEngine created the backup, it compares the files' current + // sizes against the number of bytes written to them during creation. + // Otherwise, it compares the files' current sizes against their sizes when + // the BackupEngine was opened. + // + // Returns Status::OK() if all checks are good + virtual Status VerifyBackup(BackupID backup_id) = 0; }; -// Please see the documentation in BackupableDB and RestoreBackupableDB +// A backup engine for creating new backups. class BackupEngine { public: virtual ~BackupEngine() {} - static BackupEngine* NewBackupEngine(Env* db_env, - const BackupableDBOptions& options); - - virtual Status CreateNewBackup(DB* db, bool flush_before_backup = false) = 0; - virtual Status PurgeOldBackups(uint32_t num_backups_to_keep) = 0; - virtual Status DeleteBackup(BackupID backup_id) = 0; - virtual void StopBackup() = 0; - - virtual void GetBackupInfo(std::vector* backup_info) = 0; - virtual Status RestoreDBFromBackup( - BackupID backup_id, const std::string& db_dir, const std::string& wal_dir, - const RestoreOptions& restore_options = RestoreOptions()) = 0; - virtual Status RestoreDBFromLatestBackup( - const std::string& db_dir, const std::string& wal_dir, - const RestoreOptions& restore_options = RestoreOptions()) = 0; -}; + // BackupableDBOptions have to be the same as the ones used in previous + // BackupEngines for the same backup directory. + static Status Open(Env* db_env, + const BackupableDBOptions& options, + BackupEngine** backup_engine_ptr); -// Stack your DB with BackupableDB to be able to backup the DB -class BackupableDB : public StackableDB { - public: - // BackupableDBOptions have to be the same as the ones used in a previous - // incarnation of the DB - // - // BackupableDB ownes the pointer `DB* db` now. You should not delete it or - // use it after the invocation of BackupableDB - BackupableDB(DB* db, const BackupableDBOptions& options); - virtual ~BackupableDB(); + // same as CreateNewBackup, but stores extra application metadata + // Flush will always trigger if 2PC is enabled. + virtual Status CreateNewBackupWithMetadata( + DB* db, const std::string& app_metadata, bool flush_before_backup = false, + std::function progress_callback = []() {}) = 0; // Captures the state of the database in the latest backup // NOT a thread safe call - Status CreateNewBackup(bool flush_before_backup = false); - // Returns info about backups in backup_info - void GetBackupInfo(std::vector* backup_info); + // Flush will always trigger if 2PC is enabled. + virtual Status CreateNewBackup(DB* db, bool flush_before_backup = false, + std::function progress_callback = + []() {}) { + return CreateNewBackupWithMetadata(db, "", flush_before_backup, + progress_callback); + } + // deletes old backups, keeping latest num_backups_to_keep alive - Status PurgeOldBackups(uint32_t num_backups_to_keep); + virtual Status PurgeOldBackups(uint32_t num_backups_to_keep) = 0; + // deletes a specific backup - Status DeleteBackup(BackupID backup_id); + virtual Status DeleteBackup(BackupID backup_id) = 0; + // Call this from another thread if you want to stop the backup // that is currently happening. It will return immediatelly, will // not wait for the backup to stop. @@ -204,48 +284,44 @@ class BackupableDB : public StackableDB { // return Status::Incomplete(). It will not clean up after itself, but // the state will remain consistent. The state will be cleaned up // next time you create BackupableDB or RestoreBackupableDB. - void StopBackup(); - - private: - BackupEngine* backup_engine_; -}; - -// Use this class to access information about backups and restore from them -class RestoreBackupableDB { - public: - RestoreBackupableDB(Env* db_env, const BackupableDBOptions& options); - ~RestoreBackupableDB(); + virtual void StopBackup() = 0; // Returns info about backups in backup_info - void GetBackupInfo(std::vector* backup_info); + virtual void GetBackupInfo(std::vector* backup_info) = 0; + + // Returns info about corrupt backups in corrupt_backups + virtual void GetCorruptedBackups( + std::vector* corrupt_backup_ids) = 0; // restore from backup with backup_id - // IMPORTANT -- if options_.share_table_files == true and you restore DB - // from some backup that is not the latest, and you start creating new - // backups from the new DB, they will probably fail + // IMPORTANT -- if options_.share_table_files == true, + // options_.share_files_with_checksum == false, you restore DB from some + // backup that is not the latest, and you start creating new backups from the + // new DB, they will probably fail. // // Example: Let's say you have backups 1, 2, 3, 4, 5 and you restore 3. // If you add new data to the DB and try creating a new backup now, the // database will diverge from backups 4 and 5 and the new backup will fail. // If you want to create new backup, you will first have to delete backups 4 // and 5. - Status RestoreDBFromBackup(BackupID backup_id, const std::string& db_dir, - const std::string& wal_dir, - const RestoreOptions& restore_options = - RestoreOptions()); + virtual Status RestoreDBFromBackup( + BackupID backup_id, const std::string& db_dir, const std::string& wal_dir, + const RestoreOptions& restore_options = RestoreOptions()) = 0; // restore from the latest backup - Status RestoreDBFromLatestBackup(const std::string& db_dir, - const std::string& wal_dir, - const RestoreOptions& restore_options = - RestoreOptions()); - // deletes old backups, keeping latest num_backups_to_keep alive - Status PurgeOldBackups(uint32_t num_backups_to_keep); - // deletes a specific backup - Status DeleteBackup(BackupID backup_id); + virtual Status RestoreDBFromLatestBackup( + const std::string& db_dir, const std::string& wal_dir, + const RestoreOptions& restore_options = RestoreOptions()) = 0; - private: - BackupEngine* backup_engine_; + // checks that each file exists and that the size of the file matches our + // expectations. it does not check file checksum. + // Returns Status::OK() if all checks are good + virtual Status VerifyBackup(BackupID backup_id) = 0; + + // Will delete all the files we don't need anymore + // It will do the full scan of the files/ directory and delete all the + // files that are not referenced. + virtual Status GarbageCollect() = 0; }; } // namespace rocksdb diff --git a/include/rocksdb/utilities/checkpoint.h b/include/rocksdb/utilities/checkpoint.h new file mode 100644 index 00000000000..aa0a394d4d0 --- /dev/null +++ b/include/rocksdb/utilities/checkpoint.h @@ -0,0 +1,43 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// A checkpoint is an openable snapshot of a database at a point in time. + +#pragma once +#ifndef ROCKSDB_LITE + +#include +#include "rocksdb/status.h" + +namespace rocksdb { + +class DB; + +class Checkpoint { + public: + // Creates a Checkpoint object to be used for creating openable snapshots + static Status Create(DB* db, Checkpoint** checkpoint_ptr); + + // Builds an openable snapshot of RocksDB on the same disk, which + // accepts an output directory on the same disk, and under the directory + // (1) hard-linked SST files pointing to existing live SST files + // SST files will be copied if output directory is on a different filesystem + // (2) a copied manifest files and other files + // The directory should not already exist and will be created by this API. + // The directory will be an absolute path + // log_size_for_flush: if the total log file size is equal or larger than + // this value, then a flush is triggered for all the column families. The + // default value is 0, which means flush is always triggered. If you move + // away from the default, the checkpoint may not contain up-to-date data + // if WAL writing is not always enabled. + // Flush will always trigger if it is 2PC. + virtual Status CreateCheckpoint(const std::string& checkpoint_dir, + uint64_t log_size_for_flush = 0); + + virtual ~Checkpoint() {} +}; + +} // namespace rocksdb +#endif // !ROCKSDB_LITE diff --git a/include/rocksdb/utilities/convenience.h b/include/rocksdb/utilities/convenience.h new file mode 100644 index 00000000000..f61afd69ef8 --- /dev/null +++ b/include/rocksdb/utilities/convenience.h @@ -0,0 +1,10 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +// This file was moved to rocksdb/convenience.h" + +#include "rocksdb/convenience.h" diff --git a/include/rocksdb/utilities/date_tiered_db.h b/include/rocksdb/utilities/date_tiered_db.h new file mode 100644 index 00000000000..f259b05a8ae --- /dev/null +++ b/include/rocksdb/utilities/date_tiered_db.h @@ -0,0 +1,108 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#ifndef ROCKSDB_LITE + +#include +#include +#include + +#include "rocksdb/db.h" + +namespace rocksdb { + +// Date tiered database is a wrapper of DB that implements +// a simplified DateTieredCompactionStrategy by using multiple column famillies +// as time windows. +// +// DateTieredDB provides an interface similar to DB, but it assumes that user +// provides keys with last 8 bytes encoded as timestamp in seconds. DateTieredDB +// is assigned with a TTL to declare when data should be deleted. +// +// DateTieredDB hides column families layer from standard RocksDB instance. It +// uses multiple column families to manage time series data, each containing a +// specific range of time. Column families are named by its maximum possible +// timestamp. A column family is created automatically when data newer than +// latest timestamp of all existing column families. The time range of a column +// family is configurable by `column_family_interval`. By doing this, we +// guarantee that compaction will only happen in a column family. +// +// DateTieredDB is assigned with a TTL. When all data in a column family are +// expired (CF_Timestamp <= CUR_Timestamp - TTL), we directly drop the whole +// column family. +// +// TODO(jhli): This is only a simplified version of DTCS. In a complete DTCS, +// time windows can be merged over time, so that older time windows will have +// larger time range. Also, compaction are executed only for adjacent SST files +// to guarantee there is no time overlap between SST files. + +class DateTieredDB { + public: + // Open a DateTieredDB whose name is `dbname`. + // Similar to DB::Open(), created database object is stored in dbptr. + // + // Two parameters can be configured: `ttl` to specify the length of time that + // keys should exist in the database, and `column_family_interval` to specify + // the time range of a column family interval. + // + // Open a read only database if read only is set as true. + // TODO(jhli): Should use an option object that includes ttl and + // column_family_interval. + static Status Open(const Options& options, const std::string& dbname, + DateTieredDB** dbptr, int64_t ttl, + int64_t column_family_interval, bool read_only = false); + + explicit DateTieredDB() {} + + virtual ~DateTieredDB() {} + + // Wrapper for Put method. Similar to DB::Put(), but column family to be + // inserted is decided by the timestamp in keys, i.e. the last 8 bytes of user + // key. If key is already obsolete, it will not be inserted. + // + // When client put a key value pair in DateTieredDB, it assumes last 8 bytes + // of keys are encoded as timestamp. Timestamp is a 64-bit signed integer + // encoded as the number of seconds since 1970-01-01 00:00:00 (UTC) (Same as + // Env::GetCurrentTime()). Timestamp should be encoded in big endian. + virtual Status Put(const WriteOptions& options, const Slice& key, + const Slice& val) = 0; + + // Wrapper for Get method. Similar to DB::Get() but column family is decided + // by timestamp in keys. If key is already obsolete, it will not be found. + virtual Status Get(const ReadOptions& options, const Slice& key, + std::string* value) = 0; + + // Wrapper for Delete method. Similar to DB::Delete() but column family is + // decided by timestamp in keys. If key is already obsolete, return NotFound + // status. + virtual Status Delete(const WriteOptions& options, const Slice& key) = 0; + + // Wrapper for KeyMayExist method. Similar to DB::KeyMayExist() but column + // family is decided by timestamp in keys. Return false when key is already + // obsolete. + virtual bool KeyMayExist(const ReadOptions& options, const Slice& key, + std::string* value, bool* value_found = nullptr) = 0; + + // Wrapper for Merge method. Similar to DB::Merge() but column family is + // decided by timestamp in keys. + virtual Status Merge(const WriteOptions& options, const Slice& key, + const Slice& value) = 0; + + // Create an iterator that hides low level details. This iterator internally + // merge results from all active time series column families. Note that + // column families are not deleted until all data are obsolete, so this + // iterator can possibly access obsolete key value pairs. + virtual Iterator* NewIterator(const ReadOptions& opts) = 0; + + // Explicitly drop column families in which all keys are obsolete. This + // process is also inplicitly done in Put() operation. + virtual Status DropObsoleteColumnFamilies() = 0; + + static const uint64_t kTSLength = sizeof(int64_t); // size of timestamp +}; + +} // namespace rocksdb +#endif // ROCKSDB_LITE diff --git a/include/rocksdb/utilities/db_ttl.h b/include/rocksdb/utilities/db_ttl.h index 4534e1ff716..7c9c0cc55a6 100644 --- a/include/rocksdb/utilities/db_ttl.h +++ b/include/rocksdb/utilities/db_ttl.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #ifndef ROCKSDB_LITE diff --git a/include/rocksdb/utilities/debug.h b/include/rocksdb/utilities/debug.h new file mode 100644 index 00000000000..bc5b9bf03d2 --- /dev/null +++ b/include/rocksdb/utilities/debug.h @@ -0,0 +1,41 @@ +// Copyright (c) 2017-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#ifndef ROCKSDB_LITE + +#include "rocksdb/db.h" +#include "rocksdb/types.h" + +namespace rocksdb { + +// Data associated with a particular version of a key. A database may internally +// store multiple versions of a same user key due to snapshots, compaction not +// happening yet, etc. +struct KeyVersion { + KeyVersion() : user_key(""), value(""), sequence(0), type(0) {} + + KeyVersion(const std::string& _user_key, const std::string& _value, + SequenceNumber _sequence, int _type) + : user_key(_user_key), value(_value), sequence(_sequence), type(_type) {} + + std::string user_key; + std::string value; + SequenceNumber sequence; + // TODO(ajkr): we should provide a helper function that converts the int to a + // string describing the type for easier debugging. + int type; +}; + +// Returns listing of all versions of keys in the provided user key range. +// The range is inclusive-inclusive, i.e., [`begin_key`, `end_key`]. +// The result is inserted into the provided vector, `key_versions`. +Status GetAllKeyVersions(DB* db, Slice begin_key, Slice end_key, + std::vector* key_versions); + +} // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/include/rocksdb/utilities/document_db.h b/include/rocksdb/utilities/document_db.h index 7fde5ec9f16..3668a50b9d2 100644 --- a/include/rocksdb/utilities/document_db.h +++ b/include/rocksdb/utilities/document_db.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #ifndef ROCKSDB_LITE diff --git a/include/rocksdb/utilities/env_librados.h b/include/rocksdb/utilities/env_librados.h new file mode 100644 index 00000000000..272365f0c64 --- /dev/null +++ b/include/rocksdb/utilities/env_librados.h @@ -0,0 +1,176 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#ifndef ROCKSDB_UTILITIES_ENV_LIBRADOS_H +#define ROCKSDB_UTILITIES_ENV_LIBRADOS_H + +#include +#include + +#include "rocksdb/status.h" +#include "rocksdb/utilities/env_mirror.h" + +#include + +namespace rocksdb { +class LibradosWritableFile; + +class EnvLibrados : public EnvWrapper { + public: + // Create a brand new sequentially-readable file with the specified name. + // On success, stores a pointer to the new file in *result and returns OK. + // On failure stores nullptr in *result and returns non-OK. If the file does + // not exist, returns a non-OK status. + // + // The returned file will only be accessed by one thread at a time. + Status NewSequentialFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) override; + + // Create a brand new random access read-only file with the + // specified name. On success, stores a pointer to the new file in + // *result and returns OK. On failure stores nullptr in *result and + // returns non-OK. If the file does not exist, returns a non-OK + // status. + // + // The returned file may be concurrently accessed by multiple threads. + Status NewRandomAccessFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) override; + + // Create an object that writes to a new file with the specified + // name. Deletes any existing file with the same name and creates a + // new file. On success, stores a pointer to the new file in + // *result and returns OK. On failure stores nullptr in *result and + // returns non-OK. + // + // The returned file will only be accessed by one thread at a time. + Status NewWritableFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) override; + + // Reuse an existing file by renaming it and opening it as writable. + Status ReuseWritableFile(const std::string& fname, + const std::string& old_fname, + std::unique_ptr* result, + const EnvOptions& options) override; + + // Create an object that represents a directory. Will fail if directory + // doesn't exist. If the directory exists, it will open the directory + // and create a new Directory object. + // + // On success, stores a pointer to the new Directory in + // *result and returns OK. On failure stores nullptr in *result and + // returns non-OK. + Status NewDirectory(const std::string& name, + std::unique_ptr* result) override; + + // Returns OK if the named file exists. + // NotFound if the named file does not exist, + // the calling process does not have permission to determine + // whether this file exists, or if the path is invalid. + // IOError if an IO Error was encountered + Status FileExists(const std::string& fname) override; + + // Store in *result the names of the children of the specified directory. + // The names are relative to "dir". + // Original contents of *results are dropped. + Status GetChildren(const std::string& dir, std::vector* result); + + // Delete the named file. + Status DeleteFile(const std::string& fname) override; + + // Create the specified directory. Returns error if directory exists. + Status CreateDir(const std::string& dirname) override; + + // Creates directory if missing. Return Ok if it exists, or successful in + // Creating. + Status CreateDirIfMissing(const std::string& dirname) override; + + // Delete the specified directory. + Status DeleteDir(const std::string& dirname) override; + + // Store the size of fname in *file_size. + Status GetFileSize(const std::string& fname, uint64_t* file_size) override; + + // Store the last modification time of fname in *file_mtime. + Status GetFileModificationTime(const std::string& fname, + uint64_t* file_mtime) override; + // Rename file src to target. + Status RenameFile(const std::string& src, const std::string& target) override; + // Hard Link file src to target. + Status LinkFile(const std::string& src, const std::string& target) override; + + // Lock the specified file. Used to prevent concurrent access to + // the same db by multiple processes. On failure, stores nullptr in + // *lock and returns non-OK. + // + // On success, stores a pointer to the object that represents the + // acquired lock in *lock and returns OK. The caller should call + // UnlockFile(*lock) to release the lock. If the process exits, + // the lock will be automatically released. + // + // If somebody else already holds the lock, finishes immediately + // with a failure. I.e., this call does not wait for existing locks + // to go away. + // + // May create the named file if it does not already exist. + Status LockFile(const std::string& fname, FileLock** lock); + + // Release the lock acquired by a previous successful call to LockFile. + // REQUIRES: lock was returned by a successful LockFile() call + // REQUIRES: lock has not already been unlocked. + Status UnlockFile(FileLock* lock); + + // Get full directory name for this db. + Status GetAbsolutePath(const std::string& db_path, std::string* output_path); + + // Generate unique id + std::string GenerateUniqueId(); + + // Get default EnvLibrados + static EnvLibrados* Default(); + + explicit EnvLibrados(const std::string& db_name, + const std::string& config_path, + const std::string& db_pool); + + explicit EnvLibrados( + const std::string& client_name, // first 3 parameters are + // for RADOS client init + const std::string& cluster_name, const uint64_t flags, + const std::string& db_name, const std::string& config_path, + const std::string& db_pool, const std::string& wal_dir, + const std::string& wal_pool, const uint64_t write_buffer_size); + ~EnvLibrados() { _rados.shutdown(); } + + private: + std::string _client_name; + std::string _cluster_name; + uint64_t _flags; + std::string _db_name; // get from user, readable string; Also used as db_id + // for db metadata + std::string _config_path; + librados::Rados _rados; // RADOS client + std::string _db_pool_name; + librados::IoCtx _db_pool_ioctx; // IoCtx for connecting db_pool + std::string _wal_dir; // WAL dir path + std::string _wal_pool_name; + librados::IoCtx _wal_pool_ioctx; // IoCtx for connecting wal_pool + uint64_t _write_buffer_size; // WritableFile buffer max size + + /* private function to communicate with rados */ + std::string _CreateFid(); + Status _GetFid(const std::string& fname, std::string& fid); + Status _GetFid(const std::string& fname, std::string& fid, int fid_len); + Status _RenameFid(const std::string& old_fname, const std::string& new_fname); + Status _AddFid(const std::string& fname, const std::string& fid); + Status _DelFid(const std::string& fname); + Status _GetSubFnames(const std::string& dirname, + std::vector* result); + librados::IoCtx* _GetIoctx(const std::string& prefix); + friend class LibradosWritableFile; +}; +} +#endif diff --git a/include/rocksdb/utilities/env_mirror.h b/include/rocksdb/utilities/env_mirror.h new file mode 100644 index 00000000000..ffd175ae5e6 --- /dev/null +++ b/include/rocksdb/utilities/env_mirror.h @@ -0,0 +1,176 @@ +// Copyright (c) 2015, Red Hat, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// MirrorEnv is an Env implementation that mirrors all file-related +// operations to two backing Env's (provided at construction time). +// Writes are mirrored. For read operations, we do the read from both +// backends and assert that the results match. +// +// This is useful when implementing a new Env and ensuring that the +// semantics and behavior are correct (in that they match that of an +// existing, stable Env, like the default POSIX one). + +#pragma once + +#ifndef ROCKSDB_LITE + +#include +#include +#include +#include "rocksdb/env.h" + +namespace rocksdb { + +class SequentialFileMirror; +class RandomAccessFileMirror; +class WritableFileMirror; + +class EnvMirror : public EnvWrapper { + Env* a_, *b_; + bool free_a_, free_b_; + + public: + EnvMirror(Env* a, Env* b, bool free_a=false, bool free_b=false) + : EnvWrapper(a), + a_(a), + b_(b), + free_a_(free_a), + free_b_(free_b) {} + ~EnvMirror() { + if (free_a_) + delete a_; + if (free_b_) + delete b_; + } + + Status NewSequentialFile(const std::string& f, unique_ptr* r, + const EnvOptions& options) override; + Status NewRandomAccessFile(const std::string& f, + unique_ptr* r, + const EnvOptions& options) override; + Status NewWritableFile(const std::string& f, unique_ptr* r, + const EnvOptions& options) override; + Status ReuseWritableFile(const std::string& fname, + const std::string& old_fname, + unique_ptr* r, + const EnvOptions& options) override; + virtual Status NewDirectory(const std::string& name, + unique_ptr* result) override { + unique_ptr br; + Status as = a_->NewDirectory(name, result); + Status bs = b_->NewDirectory(name, &br); + assert(as == bs); + return as; + } + Status FileExists(const std::string& f) override { + Status as = a_->FileExists(f); + Status bs = b_->FileExists(f); + assert(as == bs); + return as; + } + Status GetChildren(const std::string& dir, + std::vector* r) override { + std::vector ar, br; + Status as = a_->GetChildren(dir, &ar); + Status bs = b_->GetChildren(dir, &br); + assert(as == bs); + std::sort(ar.begin(), ar.end()); + std::sort(br.begin(), br.end()); + if (!as.ok() || ar != br) { + assert(0 == "getchildren results don't match"); + } + *r = ar; + return as; + } + Status DeleteFile(const std::string& f) override { + Status as = a_->DeleteFile(f); + Status bs = b_->DeleteFile(f); + assert(as == bs); + return as; + } + Status CreateDir(const std::string& d) override { + Status as = a_->CreateDir(d); + Status bs = b_->CreateDir(d); + assert(as == bs); + return as; + } + Status CreateDirIfMissing(const std::string& d) override { + Status as = a_->CreateDirIfMissing(d); + Status bs = b_->CreateDirIfMissing(d); + assert(as == bs); + return as; + } + Status DeleteDir(const std::string& d) override { + Status as = a_->DeleteDir(d); + Status bs = b_->DeleteDir(d); + assert(as == bs); + return as; + } + Status GetFileSize(const std::string& f, uint64_t* s) override { + uint64_t asize, bsize; + Status as = a_->GetFileSize(f, &asize); + Status bs = b_->GetFileSize(f, &bsize); + assert(as == bs); + assert(!as.ok() || asize == bsize); + *s = asize; + return as; + } + + Status GetFileModificationTime(const std::string& fname, + uint64_t* file_mtime) override { + uint64_t amtime, bmtime; + Status as = a_->GetFileModificationTime(fname, &amtime); + Status bs = b_->GetFileModificationTime(fname, &bmtime); + assert(as == bs); + assert(!as.ok() || amtime - bmtime < 10000 || bmtime - amtime < 10000); + *file_mtime = amtime; + return as; + } + + Status RenameFile(const std::string& s, const std::string& t) override { + Status as = a_->RenameFile(s, t); + Status bs = b_->RenameFile(s, t); + assert(as == bs); + return as; + } + + Status LinkFile(const std::string& s, const std::string& t) override { + Status as = a_->LinkFile(s, t); + Status bs = b_->LinkFile(s, t); + assert(as == bs); + return as; + } + + class FileLockMirror : public FileLock { + public: + FileLock* a_, *b_; + FileLockMirror(FileLock* a, FileLock* b) : a_(a), b_(b) {} + }; + + Status LockFile(const std::string& f, FileLock** l) override { + FileLock* al, *bl; + Status as = a_->LockFile(f, &al); + Status bs = b_->LockFile(f, &bl); + assert(as == bs); + if (as.ok()) *l = new FileLockMirror(al, bl); + return as; + } + + Status UnlockFile(FileLock* l) override { + FileLockMirror* ml = static_cast(l); + Status as = a_->UnlockFile(ml->a_); + Status bs = b_->UnlockFile(ml->b_); + assert(as == bs); + delete ml; + return as; + } +}; + +} // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/include/rocksdb/utilities/geo_db.h b/include/rocksdb/utilities/geo_db.h index 41c0f140819..408774c5990 100644 --- a/include/rocksdb/utilities/geo_db.h +++ b/include/rocksdb/utilities/geo_db.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #ifndef ROCKSDB_LITE @@ -59,6 +59,16 @@ class GeoObject { } }; +class GeoIterator { + public: + GeoIterator() = default; + virtual ~GeoIterator() {} + virtual void Next() = 0; + virtual bool Valid() const = 0; + virtual const GeoObject& geo_object() = 0; + virtual Status status() const = 0; +}; + // // Stack your DB with GeoDB to be able to get geo-spatial support // @@ -91,14 +101,13 @@ class GeoDB : public StackableDB { // Delete the specified object virtual Status Remove(const Slice& id) = 0; - // Returns a list of all items within a circular radius from the + // Returns an iterator for the items within a circular radius from the // specified gps location. If 'number_of_values' is specified, - // then this call returns at most that many number of objects. + // then the iterator is capped to that number of objects. // The radius is specified in 'meters'. - virtual Status SearchRadial(const GeoPosition& pos, - double radius, - std::vector* values, - int number_of_values = INT_MAX) = 0; + virtual GeoIterator* SearchRadial(const GeoPosition& pos, + double radius, + int number_of_values = INT_MAX) = 0; }; } // namespace rocksdb diff --git a/include/rocksdb/utilities/info_log_finder.h b/include/rocksdb/utilities/info_log_finder.h new file mode 100644 index 00000000000..6df056ffae8 --- /dev/null +++ b/include/rocksdb/utilities/info_log_finder.h @@ -0,0 +1,19 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include + +#include "rocksdb/db.h" +#include "rocksdb/options.h" + +namespace rocksdb { + +// This function can be used to list the Information logs, +// given the db pointer. +Status GetInfoLogList(DB* db, std::vector* info_log_list); +} // namespace rocksdb diff --git a/include/rocksdb/utilities/json_document.h b/include/rocksdb/utilities/json_document.h index ceb058cf95f..5d841f95155 100644 --- a/include/rocksdb/utilities/json_document.h +++ b/include/rocksdb/utilities/json_document.h @@ -1,19 +1,31 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #ifndef ROCKSDB_LITE -#include +#include #include +#include +#include #include +#include #include #include "rocksdb/slice.h" // We use JSONDocument for DocumentDB API -// Implementation inspired by folly::dynamic and rapidjson +// Implementation inspired by folly::dynamic, rapidjson and fbson + +namespace fbson { + class FbsonValue; + class ObjectVal; + template + class FbsonWriterT; + class FbsonOutStream; + typedef FbsonWriterT FbsonWriter; +} // namespace fbson namespace rocksdb { @@ -33,52 +45,38 @@ class JSONDocument { kString, }; - JSONDocument(); // null + /* implicit */ JSONDocument(); // null /* implicit */ JSONDocument(bool b); /* implicit */ JSONDocument(double d); + /* implicit */ JSONDocument(int8_t i); + /* implicit */ JSONDocument(int16_t i); + /* implicit */ JSONDocument(int32_t i); /* implicit */ JSONDocument(int64_t i); /* implicit */ JSONDocument(const std::string& s); /* implicit */ JSONDocument(const char* s); // constructs JSONDocument of specific type with default value - explicit JSONDocument(Type type); + explicit JSONDocument(Type _type); - // copy constructor JSONDocument(const JSONDocument& json_document); - ~JSONDocument(); + JSONDocument(JSONDocument&& json_document); Type type() const; // REQUIRES: IsObject() bool Contains(const std::string& key) const; - // Returns nullptr if !Contains() - // don't delete the returned pointer - // REQUIRES: IsObject() - const JSONDocument* Get(const std::string& key) const; - // REQUIRES: IsObject() - JSONDocument& operator[](const std::string& key); // REQUIRES: IsObject() - const JSONDocument& operator[](const std::string& key) const; - // returns `this`, so you can chain operations. - // Copies value - // REQUIRES: IsObject() - JSONDocument* Set(const std::string& key, const JSONDocument& value); + // Returns non-owner object + JSONDocument operator[](const std::string& key) const; // REQUIRES: IsArray() == true || IsObject() == true size_t Count() const; // REQUIRES: IsArray() - const JSONDocument* GetFromArray(size_t i) const; - // REQUIRES: IsArray() - JSONDocument& operator[](size_t i); - // REQUIRES: IsArray() - const JSONDocument& operator[](size_t i) const; - // returns `this`, so you can chain operations. - // Copies the value - // REQUIRES: IsArray() && i < Count() - JSONDocument* SetInArray(size_t i, const JSONDocument& value); - // REQUIRES: IsArray() - JSONDocument* PushBack(const JSONDocument& value); + // Returns non-owner object + JSONDocument operator[](size_t i) const; + + JSONDocument& operator=(JSONDocument jsonDocument); bool IsNull() const; bool IsArray() const; @@ -95,10 +93,16 @@ class JSONDocument { // REQUIRES: IsInt64() == true int64_t GetInt64() const; // REQUIRES: IsString() == true - const std::string& GetString() const; + std::string GetString() const; bool operator==(const JSONDocument& rhs) const; + bool operator!=(const JSONDocument& rhs) const; + + JSONDocument Copy() const; + + bool IsOwner() const; + std::string DebugString() const; private: @@ -114,53 +118,42 @@ class JSONDocument { static JSONDocument* Deserialize(const Slice& src); private: - void SerializeInternal(std::string* dst, bool type_prefix) const; - // returns false if Slice doesn't represent valid serialized JSONDocument. - // Otherwise, true - bool DeserializeInternal(Slice* input); + friend class JSONDocumentBuilder; - typedef std::vector Array; - typedef std::unordered_map Object; + JSONDocument(fbson::FbsonValue* val, bool makeCopy); + + void InitFromValue(const fbson::FbsonValue* val); // iteration on objects class const_item_iterator { + private: + class Impl; public: - typedef Object::const_iterator It; - typedef Object::value_type value_type; - /* implicit */ const_item_iterator(It it) : it_(it) {} - It& operator++() { return ++it_; } - bool operator!=(const const_item_iterator& other) { - return it_ != other.it_; - } - value_type operator*() { return *it_; } - + typedef std::pair value_type; + explicit const_item_iterator(Impl* impl); + const_item_iterator(const_item_iterator&&); + const_item_iterator& operator++(); + bool operator!=(const const_item_iterator& other); + value_type operator*(); + ~const_item_iterator(); private: - It it_; + friend class ItemsIteratorGenerator; + std::unique_ptr it_; }; + class ItemsIteratorGenerator { public: - /* implicit */ ItemsIteratorGenerator(const Object& object) - : object_(object) {} - const_item_iterator begin() { return object_.begin(); } - const_item_iterator end() { return object_.end(); } + explicit ItemsIteratorGenerator(const fbson::ObjectVal& object); + const_item_iterator begin() const; + + const_item_iterator end() const; private: - const Object& object_; + const fbson::ObjectVal& object_; }; - union Data { - Data() : n(nullptr) {} - ~Data() {} - - void* n; - Array a; - bool b; - double d; - int64_t i; - std::string s; - Object o; - } data_; - const Type type_; + std::unique_ptr data_; + mutable fbson::FbsonValue* value_; // Our serialization format's first byte specifies the encoding version. That // way, we can easily change our format while providing backwards @@ -169,6 +162,34 @@ class JSONDocument { static const char kSerializationFormatVersion; }; +class JSONDocumentBuilder { + public: + JSONDocumentBuilder(); + + explicit JSONDocumentBuilder(fbson::FbsonOutStream* out); + + void Reset(); + + bool WriteStartArray(); + + bool WriteEndArray(); + + bool WriteStartObject(); + + bool WriteEndObject(); + + bool WriteKeyValue(const std::string& key, const JSONDocument& value); + + bool WriteJSONDocument(const JSONDocument& value); + + JSONDocument GetJSONDocument(); + + ~JSONDocumentBuilder(); + + private: + std::unique_ptr writer_; +}; + } // namespace rocksdb #endif // ROCKSDB_LITE diff --git a/include/rocksdb/utilities/ldb_cmd.h b/include/rocksdb/utilities/ldb_cmd.h new file mode 100644 index 00000000000..b9eb1035fb2 --- /dev/null +++ b/include/rocksdb/utilities/ldb_cmd.h @@ -0,0 +1,260 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#pragma once + +#ifndef ROCKSDB_LITE + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rocksdb/env.h" +#include "rocksdb/iterator.h" +#include "rocksdb/ldb_tool.h" +#include "rocksdb/options.h" +#include "rocksdb/slice.h" +#include "rocksdb/utilities/db_ttl.h" +#include "rocksdb/utilities/ldb_cmd_execute_result.h" + +namespace rocksdb { + +class LDBCommand { + public: + // Command-line arguments + static const std::string ARG_DB; + static const std::string ARG_PATH; + static const std::string ARG_HEX; + static const std::string ARG_KEY_HEX; + static const std::string ARG_VALUE_HEX; + static const std::string ARG_CF_NAME; + static const std::string ARG_TTL; + static const std::string ARG_TTL_START; + static const std::string ARG_TTL_END; + static const std::string ARG_TIMESTAMP; + static const std::string ARG_TRY_LOAD_OPTIONS; + static const std::string ARG_IGNORE_UNKNOWN_OPTIONS; + static const std::string ARG_FROM; + static const std::string ARG_TO; + static const std::string ARG_MAX_KEYS; + static const std::string ARG_BLOOM_BITS; + static const std::string ARG_FIX_PREFIX_LEN; + static const std::string ARG_COMPRESSION_TYPE; + static const std::string ARG_COMPRESSION_MAX_DICT_BYTES; + static const std::string ARG_BLOCK_SIZE; + static const std::string ARG_AUTO_COMPACTION; + static const std::string ARG_DB_WRITE_BUFFER_SIZE; + static const std::string ARG_WRITE_BUFFER_SIZE; + static const std::string ARG_FILE_SIZE; + static const std::string ARG_CREATE_IF_MISSING; + static const std::string ARG_NO_VALUE; + + struct ParsedParams { + std::string cmd; + std::vector cmd_params; + std::map option_map; + std::vector flags; + }; + + static LDBCommand* SelectCommand(const ParsedParams& parsed_parms); + + static LDBCommand* InitFromCmdLineArgs( + const std::vector& args, const Options& options, + const LDBOptions& ldb_options, + const std::vector* column_families, + const std::function& selector = + SelectCommand); + + static LDBCommand* InitFromCmdLineArgs( + int argc, char** argv, const Options& options, + const LDBOptions& ldb_options, + const std::vector* column_families); + + bool ValidateCmdLineOptions(); + + virtual Options PrepareOptionsForOpenDB(); + + virtual void SetDBOptions(Options options) { options_ = options; } + + virtual void SetColumnFamilies( + const std::vector* column_families) { + if (column_families != nullptr) { + column_families_ = *column_families; + } else { + column_families_.clear(); + } + } + + void SetLDBOptions(const LDBOptions& ldb_options) { + ldb_options_ = ldb_options; + } + + virtual bool NoDBOpen() { return false; } + + virtual ~LDBCommand() { CloseDB(); } + + /* Run the command, and return the execute result. */ + void Run(); + + virtual void DoCommand() = 0; + + LDBCommandExecuteResult GetExecuteState() { return exec_state_; } + + void ClearPreviousRunState() { exec_state_.Reset(); } + + // Consider using Slice::DecodeHex directly instead if you don't need the + // 0x prefix + static std::string HexToString(const std::string& str); + + // Consider using Slice::ToString(true) directly instead if + // you don't need the 0x prefix + static std::string StringToHex(const std::string& str); + + static const char* DELIM; + + protected: + LDBCommandExecuteResult exec_state_; + std::string db_path_; + std::string column_family_name_; + DB* db_; + DBWithTTL* db_ttl_; + std::map cf_handles_; + + /** + * true implies that this command can work if the db is opened in read-only + * mode. + */ + bool is_read_only_; + + /** If true, the key is input/output as hex in get/put/scan/delete etc. */ + bool is_key_hex_; + + /** If true, the value is input/output as hex in get/put/scan/delete etc. */ + bool is_value_hex_; + + /** If true, the value is treated as timestamp suffixed */ + bool is_db_ttl_; + + // If true, the kvs are output with their insert/modify timestamp in a ttl db + bool timestamp_; + + // If true, try to construct options from DB's option files. + bool try_load_options_; + + bool ignore_unknown_options_; + + bool create_if_missing_; + + /** + * Map of options passed on the command-line. + */ + const std::map option_map_; + + /** + * Flags passed on the command-line. + */ + const std::vector flags_; + + /** List of command-line options valid for this command */ + const std::vector valid_cmd_line_options_; + + bool ParseKeyValue(const std::string& line, std::string* key, + std::string* value, bool is_key_hex, bool is_value_hex); + + LDBCommand(const std::map& options, + const std::vector& flags, bool is_read_only, + const std::vector& valid_cmd_line_options); + + void OpenDB(); + + void CloseDB(); + + ColumnFamilyHandle* GetCfHandle(); + + static std::string PrintKeyValue(const std::string& key, + const std::string& value, bool is_key_hex, + bool is_value_hex); + + static std::string PrintKeyValue(const std::string& key, + const std::string& value, bool is_hex); + + /** + * Return true if the specified flag is present in the specified flags vector + */ + static bool IsFlagPresent(const std::vector& flags, + const std::string& flag) { + return (std::find(flags.begin(), flags.end(), flag) != flags.end()); + } + + static std::string HelpRangeCmdArgs(); + + /** + * A helper function that returns a list of command line options + * used by this command. It includes the common options and the ones + * passed in. + */ + static std::vector BuildCmdLineOptions( + std::vector options); + + bool ParseIntOption(const std::map& options, + const std::string& option, int& value, + LDBCommandExecuteResult& exec_state); + + bool ParseStringOption(const std::map& options, + const std::string& option, std::string* value); + + Options options_; + std::vector column_families_; + LDBOptions ldb_options_; + + private: + /** + * Interpret command line options and flags to determine if the key + * should be input/output in hex. + */ + bool IsKeyHex(const std::map& options, + const std::vector& flags); + + /** + * Interpret command line options and flags to determine if the value + * should be input/output in hex. + */ + bool IsValueHex(const std::map& options, + const std::vector& flags); + + /** + * Returns the value of the specified option as a boolean. + * default_val is used if the option is not found in options. + * Throws an exception if the value of the option is not + * "true" or "false" (case insensitive). + */ + bool ParseBooleanOption(const std::map& options, + const std::string& option, bool default_val); + + /** + * Converts val to a boolean. + * val must be either true or false (case insensitive). + * Otherwise an exception is thrown. + */ + bool StringToBool(std::string val); +}; + +class LDBCommandRunner { + public: + static void PrintHelp(const LDBOptions& ldb_options, const char* exec_name); + + static void RunCommand( + int argc, char** argv, Options options, const LDBOptions& ldb_options, + const std::vector* column_families); +}; + +} // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/util/ldb_cmd_execute_result.h b/include/rocksdb/utilities/ldb_cmd_execute_result.h similarity index 63% rename from util/ldb_cmd_execute_result.h rename to include/rocksdb/utilities/ldb_cmd_execute_result.h index b9121b2b0a4..5ddc6feb696 100644 --- a/util/ldb_cmd_execute_result.h +++ b/include/rocksdb/utilities/ldb_cmd_execute_result.h @@ -1,10 +1,14 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #pragma once +#ifdef FAILED +#undef FAILED +#endif + namespace rocksdb { class LDBCommandExecuteResult { @@ -13,15 +17,10 @@ class LDBCommandExecuteResult { EXEC_NOT_STARTED = 0, EXEC_SUCCEED = 1, EXEC_FAILED = 2, }; - LDBCommandExecuteResult() { - state_ = EXEC_NOT_STARTED; - message_ = ""; - } + LDBCommandExecuteResult() : state_(EXEC_NOT_STARTED), message_("") {} - LDBCommandExecuteResult(State state, std::string& msg) { - state_ = state; - message_ = msg; - } + LDBCommandExecuteResult(State state, std::string& msg) : + state_(state), message_(msg) {} std::string ToString() { std::string ret; @@ -57,11 +56,11 @@ class LDBCommandExecuteResult { return state_ == EXEC_FAILED; } - static LDBCommandExecuteResult SUCCEED(std::string msg) { + static LDBCommandExecuteResult Succeed(std::string msg) { return LDBCommandExecuteResult(EXEC_SUCCEED, msg); } - static LDBCommandExecuteResult FAILED(std::string msg) { + static LDBCommandExecuteResult Failed(std::string msg) { return LDBCommandExecuteResult(EXEC_FAILED, msg); } diff --git a/include/rocksdb/utilities/leveldb_options.h b/include/rocksdb/utilities/leveldb_options.h new file mode 100644 index 00000000000..fb5a440bbc6 --- /dev/null +++ b/include/rocksdb/utilities/leveldb_options.h @@ -0,0 +1,144 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include + +namespace rocksdb { + +class Cache; +class Comparator; +class Env; +class FilterPolicy; +class Logger; +struct Options; +class Snapshot; + +enum CompressionType : unsigned char; + +// Options to control the behavior of a database (passed to +// DB::Open). A LevelDBOptions object can be initialized as though +// it were a LevelDB Options object, and then it can be converted into +// a RocksDB Options object. +struct LevelDBOptions { + // ------------------- + // Parameters that affect behavior + + // Comparator used to define the order of keys in the table. + // Default: a comparator that uses lexicographic byte-wise ordering + // + // REQUIRES: The client must ensure that the comparator supplied + // here has the same name and orders keys *exactly* the same as the + // comparator provided to previous open calls on the same DB. + const Comparator* comparator; + + // If true, the database will be created if it is missing. + // Default: false + bool create_if_missing; + + // If true, an error is raised if the database already exists. + // Default: false + bool error_if_exists; + + // If true, the implementation will do aggressive checking of the + // data it is processing and will stop early if it detects any + // errors. This may have unforeseen ramifications: for example, a + // corruption of one DB entry may cause a large number of entries to + // become unreadable or for the entire DB to become unopenable. + // Default: false + bool paranoid_checks; + + // Use the specified object to interact with the environment, + // e.g. to read/write files, schedule background work, etc. + // Default: Env::Default() + Env* env; + + // Any internal progress/error information generated by the db will + // be written to info_log if it is non-NULL, or to a file stored + // in the same directory as the DB contents if info_log is NULL. + // Default: NULL + Logger* info_log; + + // ------------------- + // Parameters that affect performance + + // Amount of data to build up in memory (backed by an unsorted log + // on disk) before converting to a sorted on-disk file. + // + // Larger values increase performance, especially during bulk loads. + // Up to two write buffers may be held in memory at the same time, + // so you may wish to adjust this parameter to control memory usage. + // Also, a larger write buffer will result in a longer recovery time + // the next time the database is opened. + // + // Default: 4MB + size_t write_buffer_size; + + // Number of open files that can be used by the DB. You may need to + // increase this if your database has a large working set (budget + // one open file per 2MB of working set). + // + // Default: 1000 + int max_open_files; + + // Control over blocks (user data is stored in a set of blocks, and + // a block is the unit of reading from disk). + + // If non-NULL, use the specified cache for blocks. + // If NULL, leveldb will automatically create and use an 8MB internal cache. + // Default: NULL + Cache* block_cache; + + // Approximate size of user data packed per block. Note that the + // block size specified here corresponds to uncompressed data. The + // actual size of the unit read from disk may be smaller if + // compression is enabled. This parameter can be changed dynamically. + // + // Default: 4K + size_t block_size; + + // Number of keys between restart points for delta encoding of keys. + // This parameter can be changed dynamically. Most clients should + // leave this parameter alone. + // + // Default: 16 + int block_restart_interval; + + // Compress blocks using the specified compression algorithm. This + // parameter can be changed dynamically. + // + // Default: kSnappyCompression, which gives lightweight but fast + // compression. + // + // Typical speeds of kSnappyCompression on an Intel(R) Core(TM)2 2.4GHz: + // ~200-500MB/s compression + // ~400-800MB/s decompression + // Note that these speeds are significantly faster than most + // persistent storage speeds, and therefore it is typically never + // worth switching to kNoCompression. Even if the input data is + // incompressible, the kSnappyCompression implementation will + // efficiently detect that and will switch to uncompressed mode. + CompressionType compression; + + // If non-NULL, use the specified filter policy to reduce disk reads. + // Many applications will benefit from passing the result of + // NewBloomFilterPolicy() here. + // + // Default: NULL + const FilterPolicy* filter_policy; + + // Create a LevelDBOptions object with default values for all fields. + LevelDBOptions(); +}; + +// Converts a LevelDBOptions object into a RocksDB Options object. +Options ConvertOptions(const LevelDBOptions& leveldb_options); + +} // namespace rocksdb diff --git a/include/rocksdb/utilities/lua/rocks_lua_compaction_filter.h b/include/rocksdb/utilities/lua/rocks_lua_compaction_filter.h new file mode 100644 index 00000000000..a7af592d8c5 --- /dev/null +++ b/include/rocksdb/utilities/lua/rocks_lua_compaction_filter.h @@ -0,0 +1,189 @@ +// Copyright (c) 2016, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#if defined(LUA) && !defined(ROCKSDB_LITE) +// lua headers +extern "C" { +#include +#include +#include +} + +#include +#include +#include + +#include "rocksdb/compaction_filter.h" +#include "rocksdb/env.h" +#include "rocksdb/slice.h" +#include "rocksdb/utilities/lua/rocks_lua_custom_library.h" +#include "rocksdb/utilities/lua/rocks_lua_util.h" + +namespace rocksdb { +namespace lua { + +struct RocksLuaCompactionFilterOptions { + // The lua script in string that implements all necessary CompactionFilter + // virtual functions. The specified lua_script must implement the following + // functions, which are Name and Filter, as described below. + // + // 0. The Name function simply returns a string representing the name of + // the lua script. If there's any erorr in the Name function, an + // empty string will be used. + // --- Example + // function Name() + // return "DefaultLuaCompactionFilter" + // end + // + // + // 1. The script must contains a function called Filter, which implements + // CompactionFilter::Filter() , takes three input arguments, and returns + // three values as the following API: + // + // function Filter(level, key, existing_value) + // ... + // return is_filtered, is_changed, new_value + // end + // + // Note that if ignore_value is set to true, then Filter should implement + // the following API: + // + // function Filter(level, key) + // ... + // return is_filtered + // end + // + // If there're any error in the Filter() function, then it will keep + // the input key / value pair. + // + // -- Input + // The function must take three arguments (integer, string, string), + // which map to "level", "key", and "existing_value" passed from + // RocksDB. + // + // -- Output + // The function must return three values (boolean, boolean, string). + // - is_filtered: if the first return value is true, then it indicates + // the input key / value pair should be filtered. + // - is_changed: if the second return value is true, then it indicates + // the existing_value needs to be changed, and the resulting value + // is stored in the third return value. + // - new_value: if the second return value is true, then this third + // return value stores the new value of the input key / value pair. + // + // -- Examples + // -- a filter that keeps all key-value pairs + // function Filter(level, key, existing_value) + // return false, false, "" + // end + // + // -- a filter that keeps all keys and change their values to "Rocks" + // function Filter(level, key, existing_value) + // return false, true, "Rocks" + // end + + std::string lua_script; + + // If set to true, then existing_value will not be passed to the Filter + // function, and the Filter function only needs to return a single boolean + // flag indicating whether to filter out this key or not. + // + // function Filter(level, key) + // ... + // return is_filtered + // end + bool ignore_value = false; + + // A boolean flag to determine whether to ignore snapshots. + bool ignore_snapshots = false; + + // When specified a non-null pointer, the first "error_limit_per_filter" + // errors of each CompactionFilter that is lua related will be included + // in this log. + std::shared_ptr error_log; + + // The number of errors per CompactionFilter will be printed + // to error_log. + int error_limit_per_filter = 1; + + // A string to luaL_reg array map that allows the Lua CompactionFilter + // to use custom C library. The string will be used as the library + // name in Lua. + std::vector> libraries; + + /////////////////////////////////////////////////////////////////////////// + // NOT YET SUPPORTED + // The name of the Lua function in "lua_script" that implements + // CompactionFilter::FilterMergeOperand(). The function must take + // three input arguments (integer, string, string), which map to "level", + // "key", and "operand" passed from the RocksDB. In addition, the + // function must return a single boolean value, indicating whether + // to filter the input key / operand. + // + // DEFAULT: the default implementation always returns false. + // @see CompactionFilter::FilterMergeOperand +}; + +class RocksLuaCompactionFilterFactory : public CompactionFilterFactory { + public: + explicit RocksLuaCompactionFilterFactory( + const RocksLuaCompactionFilterOptions opt); + + virtual ~RocksLuaCompactionFilterFactory() {} + + std::unique_ptr CreateCompactionFilter( + const CompactionFilter::Context& context) override; + + // Change the Lua script so that the next compaction after this + // function call will use the new Lua script. + void SetScript(const std::string& new_script); + + // Obtain the current Lua script + std::string GetScript(); + + const char* Name() const override; + + private: + RocksLuaCompactionFilterOptions opt_; + std::string name_; + // A lock to protect "opt_" to make it dynamically changeable. + std::mutex opt_mutex_; +}; + +// A wrapper class that invokes Lua script to perform CompactionFilter +// functions. +class RocksLuaCompactionFilter : public rocksdb::CompactionFilter { + public: + explicit RocksLuaCompactionFilter(const RocksLuaCompactionFilterOptions& opt) + : options_(opt), + lua_state_wrapper_(opt.lua_script, opt.libraries), + error_count_(0), + name_("") {} + + virtual bool Filter(int level, const Slice& key, const Slice& existing_value, + std::string* new_value, + bool* value_changed) const override; + // Not yet supported + virtual bool FilterMergeOperand(int level, const Slice& key, + const Slice& operand) const override { + return false; + } + virtual bool IgnoreSnapshots() const override; + virtual const char* Name() const override; + + protected: + void LogLuaError(const char* format, ...) const; + + RocksLuaCompactionFilterOptions options_; + LuaStateWrapper lua_state_wrapper_; + mutable int error_count_; + mutable std::string name_; +}; + +} // namespace lua +} // namespace rocksdb +#endif // defined(LUA) && !defined(ROCKSDB_LITE) diff --git a/include/rocksdb/utilities/lua/rocks_lua_custom_library.h b/include/rocksdb/utilities/lua/rocks_lua_custom_library.h new file mode 100644 index 00000000000..3ca8b32f3e2 --- /dev/null +++ b/include/rocksdb/utilities/lua/rocks_lua_custom_library.h @@ -0,0 +1,43 @@ +// Copyright (c) 2016, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#ifdef LUA + +// lua headers +extern "C" { +#include +#include +#include +} + +namespace rocksdb { +namespace lua { +// A class that used to define custom C Library that is callable +// from Lua script +class RocksLuaCustomLibrary { + public: + virtual ~RocksLuaCustomLibrary() {} + // The name of the C library. This name will also be used as the table + // (namespace) in Lua that contains the C library. + virtual const char* Name() const = 0; + + // Returns a "static const struct luaL_Reg[]", which includes a list of + // C functions. Note that the last entry of this static array must be + // {nullptr, nullptr} as required by Lua. + // + // More details about how to implement Lua C libraries can be found + // in the official Lua document http://www.lua.org/pil/26.2.html + virtual const struct luaL_Reg* Lib() const = 0; + + // A function that will be called right after the library has been created + // and pushed on the top of the lua_State. This custom setup function + // allows developers to put additional table or constant values inside + // the same table / namespace. + virtual void CustomSetup(lua_State* L) const {} +}; +} // namespace lua +} // namespace rocksdb +#endif // LUA diff --git a/include/rocksdb/utilities/lua/rocks_lua_util.h b/include/rocksdb/utilities/lua/rocks_lua_util.h new file mode 100644 index 00000000000..36b007cc73c --- /dev/null +++ b/include/rocksdb/utilities/lua/rocks_lua_util.h @@ -0,0 +1,55 @@ +// Copyright (c) 2016, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +// lua headers +extern "C" { +#include +#include +#include +} + +#ifdef LUA +#include +#include + +#include "rocksdb/utilities/lua/rocks_lua_custom_library.h" + +namespace rocksdb { +namespace lua { +class LuaStateWrapper { + public: + explicit LuaStateWrapper(const std::string& lua_script) { + lua_state_ = luaL_newstate(); + Init(lua_script, {}); + } + LuaStateWrapper( + const std::string& lua_script, + const std::vector>& libraries) { + lua_state_ = luaL_newstate(); + Init(lua_script, libraries); + } + lua_State* GetLuaState() const { return lua_state_; } + ~LuaStateWrapper() { lua_close(lua_state_); } + + private: + void Init( + const std::string& lua_script, + const std::vector>& libraries) { + if (lua_state_) { + luaL_openlibs(lua_state_); + for (const auto& library : libraries) { + luaL_openlib(lua_state_, library->Name(), library->Lib(), 0); + library->CustomSetup(lua_state_); + } + luaL_dostring(lua_state_, lua_script.c_str()); + } + } + + lua_State* lua_state_; +}; +} // namespace lua +} // namespace rocksdb +#endif // LUA diff --git a/include/rocksdb/utilities/memory_util.h b/include/rocksdb/utilities/memory_util.h new file mode 100644 index 00000000000..c6128909e90 --- /dev/null +++ b/include/rocksdb/utilities/memory_util.h @@ -0,0 +1,50 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#pragma once + +#include +#include +#include +#include + +#include "rocksdb/cache.h" +#include "rocksdb/db.h" + +namespace rocksdb { + +// Returns the current memory usage of the specified DB instances. +class MemoryUtil { + public: + enum UsageType : int { + // Memory usage of all the mem-tables. + kMemTableTotal = 0, + // Memory usage of those un-flushed mem-tables. + kMemTableUnFlushed = 1, + // Memory usage of all the table readers. + kTableReadersTotal = 2, + // Memory usage by Cache. + kCacheTotal = 3, + kNumUsageTypes = 4 + }; + + // Returns the approximate memory usage of different types in the input + // list of DBs and Cache set. For instance, in the output map + // usage_by_type, usage_by_type[kMemTableTotal] will store the memory + // usage of all the mem-tables from all the input rocksdb instances. + // + // Note that for memory usage inside Cache class, we will + // only report the usage of the input "cache_set" without + // including those Cache usage inside the input list "dbs" + // of DBs. + static Status GetApproximateMemoryUsageByType( + const std::vector& dbs, + const std::unordered_set cache_set, + std::map* usage_by_type); +}; +} // namespace rocksdb +#endif // !ROCKSDB_LITE diff --git a/include/rocksdb/utilities/object_registry.h b/include/rocksdb/utilities/object_registry.h new file mode 100644 index 00000000000..b046ba7c1f5 --- /dev/null +++ b/include/rocksdb/utilities/object_registry.h @@ -0,0 +1,90 @@ +// Copyright (c) 2016-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#ifndef ROCKSDB_LITE + +#include +#include +#include +#include +#include + +#include "rocksdb/env.h" + +namespace rocksdb { + +// Creates a new T using the factory function that was registered with a pattern +// that matches the provided "target" string according to std::regex_match. +// +// If no registered functions match, returns nullptr. If multiple functions +// match, the factory function used is unspecified. +// +// Populates res_guard with result pointer if caller is granted ownership. +template +T* NewCustomObject(const std::string& target, std::unique_ptr* res_guard); + +// Returns a new T when called with a string. Populates the unique_ptr argument +// if granting ownership to caller. +template +using FactoryFunc = std::function*)>; + +// To register a factory function for a type T, initialize a Registrar object +// with static storage duration. For example: +// +// static Registrar hdfs_reg("hdfs://.*", &CreateHdfsEnv); +// +// Then, calling NewCustomObject("hdfs://some_path", ...) will match the +// regex provided above, so it returns the result of invoking CreateHdfsEnv. +template +class Registrar { + public: + explicit Registrar(std::string pattern, FactoryFunc factory); +}; + +// Implementation details follow. + +namespace internal { + +template +struct RegistryEntry { + std::regex pattern; + FactoryFunc factory; +}; + +template +struct Registry { + static Registry* Get() { + static Registry instance; + return &instance; + } + std::vector> entries; + + private: + Registry() = default; +}; + +} // namespace internal + +template +T* NewCustomObject(const std::string& target, std::unique_ptr* res_guard) { + res_guard->reset(); + for (const auto& entry : internal::Registry::Get()->entries) { + if (std::regex_match(target, entry.pattern)) { + return entry.factory(target, res_guard); + } + } + return nullptr; +} + +template +Registrar::Registrar(std::string pattern, FactoryFunc factory) { + internal::Registry::Get()->entries.emplace_back(internal::RegistryEntry{ + std::regex(std::move(pattern)), std::move(factory)}); +} + +} // namespace rocksdb +#endif // ROCKSDB_LITE diff --git a/include/rocksdb/utilities/optimistic_transaction_db.h b/include/rocksdb/utilities/optimistic_transaction_db.h new file mode 100644 index 00000000000..02917ff5830 --- /dev/null +++ b/include/rocksdb/utilities/optimistic_transaction_db.h @@ -0,0 +1,76 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#ifndef ROCKSDB_LITE + +#include +#include + +#include "rocksdb/comparator.h" +#include "rocksdb/db.h" + +namespace rocksdb { + +class Transaction; + +// Database with Transaction support. +// +// See optimistic_transaction.h and examples/transaction_example.cc + +// Options to use when starting an Optimistic Transaction +struct OptimisticTransactionOptions { + // Setting set_snapshot=true is the same as calling SetSnapshot(). + bool set_snapshot = false; + + // Should be set if the DB has a non-default comparator. + // See comment in WriteBatchWithIndex constructor. + const Comparator* cmp = BytewiseComparator(); +}; + +class OptimisticTransactionDB { + public: + // Open an OptimisticTransactionDB similar to DB::Open(). + static Status Open(const Options& options, const std::string& dbname, + OptimisticTransactionDB** dbptr); + + static Status Open(const DBOptions& db_options, const std::string& dbname, + const std::vector& column_families, + std::vector* handles, + OptimisticTransactionDB** dbptr); + + virtual ~OptimisticTransactionDB() {} + + // Starts a new Transaction. + // + // Caller is responsible for deleting the returned transaction when no + // longer needed. + // + // If old_txn is not null, BeginTransaction will reuse this Transaction + // handle instead of allocating a new one. This is an optimization to avoid + // extra allocations when repeatedly creating transactions. + virtual Transaction* BeginTransaction( + const WriteOptions& write_options, + const OptimisticTransactionOptions& txn_options = + OptimisticTransactionOptions(), + Transaction* old_txn = nullptr) = 0; + + // Return the underlying Database that was opened + virtual DB* GetBaseDB() = 0; + + protected: + // To Create an OptimisticTransactionDB, call Open() + explicit OptimisticTransactionDB(DB* db) {} + OptimisticTransactionDB() {} + + private: + // No copying allowed + OptimisticTransactionDB(const OptimisticTransactionDB&); + void operator=(const OptimisticTransactionDB&); +}; + +} // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/include/rocksdb/utilities/option_change_migration.h b/include/rocksdb/utilities/option_change_migration.h new file mode 100644 index 00000000000..81f674c9737 --- /dev/null +++ b/include/rocksdb/utilities/option_change_migration.h @@ -0,0 +1,19 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include "rocksdb/options.h" +#include "rocksdb/status.h" + +namespace rocksdb { +// Try to migrate DB created with old_opts to be use new_opts. +// Multiple column families is not supported. +// It is best-effort. No guarantee to succeed. +// A full compaction may be executed. +Status OptionChangeMigration(std::string dbname, const Options& old_opts, + const Options& new_opts); +} // namespace rocksdb diff --git a/include/rocksdb/utilities/options_util.h b/include/rocksdb/utilities/options_util.h new file mode 100644 index 00000000000..d02c574104a --- /dev/null +++ b/include/rocksdb/utilities/options_util.h @@ -0,0 +1,93 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +// This file contains utility functions for RocksDB Options. +#pragma once + +#ifndef ROCKSDB_LITE + +#include +#include + +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/options.h" +#include "rocksdb/status.h" + +namespace rocksdb { +// Constructs the DBOptions and ColumnFamilyDescriptors by loading the +// latest RocksDB options file stored in the specified rocksdb database. +// +// Note that the all the pointer options (except table_factory, which will +// be described in more details below) will be initialized with the default +// values. Developers can further initialize them after this function call. +// Below is an example list of pointer options which will be initialized +// +// * env +// * memtable_factory +// * compaction_filter_factory +// * prefix_extractor +// * comparator +// * merge_operator +// * compaction_filter +// +// For table_factory, this function further supports deserializing +// BlockBasedTableFactory and its BlockBasedTableOptions except the +// pointer options of BlockBasedTableOptions (flush_block_policy_factory, +// block_cache, and block_cache_compressed), which will be initialized with +// default values. Developers can further specify these three options by +// casting the return value of TableFactoroy::GetOptions() to +// BlockBasedTableOptions and making necessary changes. +// +// ignore_unknown_options can be set to true if you want to ignore options +// that are from a newer version of the db, esentially for forward +// compatibility. +// +// examples/options_file_example.cc demonstrates how to use this function +// to open a RocksDB instance. +// +// @return the function returns an OK status when it went successfully. If +// the specified "dbpath" does not contain any option file, then a +// Status::NotFound will be returned. A return value other than +// Status::OK or Status::NotFound indicates there're some error related +// to the options file itself. +// +// @see LoadOptionsFromFile +Status LoadLatestOptions(const std::string& dbpath, Env* env, + DBOptions* db_options, + std::vector* cf_descs, + bool ignore_unknown_options = false); + +// Similar to LoadLatestOptions, this function constructs the DBOptions +// and ColumnFamilyDescriptors based on the specified RocksDB Options file. +// +// @see LoadLatestOptions +Status LoadOptionsFromFile(const std::string& options_file_name, Env* env, + DBOptions* db_options, + std::vector* cf_descs, + bool ignore_unknown_options = false); + +// Returns the latest options file name under the specified db path. +Status GetLatestOptionsFileName(const std::string& dbpath, Env* env, + std::string* options_file_name); + +// Returns Status::OK if the input DBOptions and ColumnFamilyDescriptors +// are compatible with the latest options stored in the specified DB path. +// +// If the return status is non-ok, it means the specified RocksDB instance +// might not be correctly opened with the input set of options. Currently, +// changing one of the following options will fail the compatibility check: +// +// * comparator +// * prefix_extractor +// * table_factory +// * merge_operator +Status CheckOptionsCompatibility( + const std::string& dbpath, Env* env, const DBOptions& db_options, + const std::vector& cf_descs, + bool ignore_unknown_options = false); + +} // namespace rocksdb +#endif // !ROCKSDB_LITE diff --git a/include/rocksdb/utilities/sim_cache.h b/include/rocksdb/utilities/sim_cache.h new file mode 100644 index 00000000000..f29fd5e8f68 --- /dev/null +++ b/include/rocksdb/utilities/sim_cache.h @@ -0,0 +1,89 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include +#include +#include "rocksdb/cache.h" +#include "rocksdb/env.h" +#include "rocksdb/slice.h" +#include "rocksdb/statistics.h" +#include "rocksdb/status.h" + +namespace rocksdb { + +class SimCache; + +// For instrumentation purpose, use NewSimCache instead of NewLRUCache API +// NewSimCache is a wrapper function returning a SimCache instance that can +// have additional interface provided in Simcache class besides Cache interface +// to predict block cache hit rate without actually allocating the memory. It +// can help users tune their current block cache size, and determine how +// efficient they are using the memory. +// +// Since GetSimCapacity() returns the capacity for simulutation, it differs from +// actual memory usage, which can be estimated as: +// sim_capacity * entry_size / (entry_size + block_size), +// where 76 <= entry_size <= 104, +// BlockBasedTableOptions.block_size = 4096 by default but is configurable, +// Therefore, generally the actual memory overhead of SimCache is Less than +// sim_capacity * 2% +extern std::shared_ptr NewSimCache(std::shared_ptr cache, + size_t sim_capacity, + int num_shard_bits); + +class SimCache : public Cache { + public: + SimCache() {} + + ~SimCache() override {} + + const char* Name() const override { return "SimCache"; } + + // returns the maximum configured capacity of the simcache for simulation + virtual size_t GetSimCapacity() const = 0; + + // simcache doesn't provide internal handler reference to user, so always + // PinnedUsage = 0 and the behavior will be not exactly consistent the + // with real cache. + // returns the memory size for the entries residing in the simcache. + virtual size_t GetSimUsage() const = 0; + + // sets the maximum configured capacity of the simcache. When the new + // capacity is less than the old capacity and the existing usage is + // greater than new capacity, the implementation will purge old entries + // to fit new capapicty. + virtual void SetSimCapacity(size_t capacity) = 0; + + // returns the lookup times of simcache + virtual uint64_t get_miss_counter() const = 0; + // returns the hit times of simcache + virtual uint64_t get_hit_counter() const = 0; + // reset the lookup and hit counters + virtual void reset_counter() = 0; + // String representation of the statistics of the simcache + virtual std::string ToString() const = 0; + + // Start storing logs of the cache activity (Add/Lookup) into + // a file located at activity_log_file, max_logging_size option can be used to + // stop logging to the file automatically after reaching a specific size in + // bytes, a values of 0 disable this feature + virtual Status StartActivityLogging(const std::string& activity_log_file, + Env* env, uint64_t max_logging_size = 0) = 0; + + // Stop cache activity logging if any + virtual void StopActivityLogging() = 0; + + // Status of cache logging happening in background + virtual Status GetActivityLoggingStatus() = 0; + + private: + SimCache(const SimCache&); + SimCache& operator=(const SimCache&); +}; + +} // namespace rocksdb diff --git a/include/rocksdb/utilities/spatial_db.h b/include/rocksdb/utilities/spatial_db.h index cba93cd5fa3..477b77cf626 100644 --- a/include/rocksdb/utilities/spatial_db.h +++ b/include/rocksdb/utilities/spatial_db.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #ifndef ROCKSDB_LITE @@ -57,34 +57,57 @@ struct Variant { new (&data_.s) std::string(s); } - Variant(const Variant& v); + Variant(const Variant& v) : type_(v.type_) { Init(v, data_); } - ~Variant() { - if (type_ == kString) { - using std::string; - (&data_.s)->~string(); - } - } + Variant& operator=(const Variant& v); + + Variant(Variant&& rhs) : type_(kNull) { *this = std::move(rhs); } + + Variant& operator=(Variant&& v); + + ~Variant() { Destroy(type_, data_); } Type type() const { return type_; } bool get_bool() const { return data_.b; } uint64_t get_int() const { return data_.i; } double get_double() const { return data_.d; } - const std::string& get_string() const { return data_.s; } + const std::string& get_string() const { return *GetStringPtr(data_); } - bool operator==(const Variant& other); - bool operator!=(const Variant& other); + bool operator==(const Variant& other) const; + bool operator!=(const Variant& other) const { return !(*this == other); } private: Type type_; + union Data { - Data() {} - ~Data() {} bool b; uint64_t i; double d; - std::string s; + // Current version of MS compiler not C++11 compliant so can not put + // std::string + // however, even then we still need the rest of the maintenance. + char s[sizeof(std::string)]; } data_; + + // Avoid type_punned aliasing problem + static std::string* GetStringPtr(Data& d) { + void* p = d.s; + return reinterpret_cast(p); + } + + static const std::string* GetStringPtr(const Data& d) { + const void* p = d.s; + return reinterpret_cast(p); + } + + static void Init(const Variant&, Data&); + + static void Destroy(Type t, Data& d) { + if (t == kString) { + using std::string; + GetStringPtr(d)->~string(); + } + } }; // FeatureSet is a map of key-value pairs. One feature set is associated with @@ -222,7 +245,9 @@ class SpatialDB : public StackableDB { // Calling Compact() after inserting a bunch of elements should speed up // reading. This is especially useful if you use SpatialDBOptions::bulk_load - virtual Status Compact() = 0; + // Num threads determines how many threads we'll use for compactions. Setting + // this to bigger number will use more IO and CPU, but finish faster + virtual Status Compact(int num_threads = 1) = 0; // Query the specified spatial_index. Query will return all elements that // intersect bbox, but it may also return some extra elements. diff --git a/include/rocksdb/utilities/stackable_db.h b/include/rocksdb/utilities/stackable_db.h index 417378f5dd9..991de90aab0 100644 --- a/include/rocksdb/utilities/stackable_db.h +++ b/include/rocksdb/utilities/stackable_db.h @@ -3,8 +3,16 @@ // found in the LICENSE file. See the AUTHORS file for names of contributors. #pragma once +#include +#include #include "rocksdb/db.h" +#ifdef _WIN32 +// Windows API macro interference +#undef DeleteFile +#endif + + namespace rocksdb { // This class contains APIs to stack rocksdb wrappers.Eg. Stack TTL over base d @@ -21,16 +29,41 @@ class StackableDB : public DB { return db_; } + virtual DB* GetRootDB() override { return db_->GetRootDB(); } + virtual Status CreateColumnFamily(const ColumnFamilyOptions& options, const std::string& column_family_name, - ColumnFamilyHandle** handle) { + ColumnFamilyHandle** handle) override { return db_->CreateColumnFamily(options, column_family_name, handle); } - virtual Status DropColumnFamily(ColumnFamilyHandle* column_family) { + virtual Status CreateColumnFamilies( + const ColumnFamilyOptions& options, + const std::vector& column_family_names, + std::vector* handles) override { + return db_->CreateColumnFamilies(options, column_family_names, handles); + } + + virtual Status CreateColumnFamilies( + const std::vector& column_families, + std::vector* handles) override { + return db_->CreateColumnFamilies(column_families, handles); + } + + virtual Status DropColumnFamily(ColumnFamilyHandle* column_family) override { return db_->DropColumnFamily(column_family); } + virtual Status DropColumnFamilies( + const std::vector& column_families) override { + return db_->DropColumnFamilies(column_families); + } + + virtual Status DestroyColumnFamilyHandle( + ColumnFamilyHandle* column_family) override { + return db_->DestroyColumnFamilyHandle(column_family); + } + using DB::Put; virtual Status Put(const WriteOptions& options, ColumnFamilyHandle* column_family, const Slice& key, @@ -41,7 +74,7 @@ class StackableDB : public DB { using DB::Get; virtual Status Get(const ReadOptions& options, ColumnFamilyHandle* column_family, const Slice& key, - std::string* value) override { + PinnableSlice* value) override { return db_->Get(options, column_family, key, value); } @@ -54,6 +87,16 @@ class StackableDB : public DB { return db_->MultiGet(options, column_family, keys, values); } + using DB::IngestExternalFile; + virtual Status IngestExternalFile( + ColumnFamilyHandle* column_family, + const std::vector& external_files, + const IngestExternalFileOptions& options) override { + return db_->IngestExternalFile(column_family, external_files, options); + } + + virtual Status VerifyChecksum() override { return db_->VerifyChecksum(); } + using DB::KeyMayExist; virtual bool KeyMayExist(const ReadOptions& options, ColumnFamilyHandle* column_family, const Slice& key, @@ -69,6 +112,13 @@ class StackableDB : public DB { return db_->Delete(wopts, column_family, key); } + using DB::SingleDelete; + virtual Status SingleDelete(const WriteOptions& wopts, + ColumnFamilyHandle* column_family, + const Slice& key) override { + return db_->SingleDelete(wopts, column_family, key); + } + using DB::Merge; virtual Status Merge(const WriteOptions& options, ColumnFamilyHandle* column_family, const Slice& key, @@ -91,7 +141,7 @@ class StackableDB : public DB { virtual Status NewIterators( const ReadOptions& options, const std::vector& column_families, - std::vector* iterators) { + std::vector* iterators) override { return db_->NewIterators(options, column_families, iterators); } @@ -104,11 +154,17 @@ class StackableDB : public DB { return db_->ReleaseSnapshot(snapshot); } + using DB::GetMapProperty; using DB::GetProperty; virtual bool GetProperty(ColumnFamilyHandle* column_family, const Slice& property, std::string* value) override { return db_->GetProperty(column_family, property, value); } + virtual bool GetMapProperty(ColumnFamilyHandle* column_family, + const Slice& property, + std::map* value) override { + return db_->GetMapProperty(column_family, property, value); + } using DB::GetIntProperty; virtual bool GetIntProperty(ColumnFamilyHandle* column_family, @@ -116,20 +172,57 @@ class StackableDB : public DB { return db_->GetIntProperty(column_family, property, value); } + using DB::GetAggregatedIntProperty; + virtual bool GetAggregatedIntProperty(const Slice& property, + uint64_t* value) override { + return db_->GetAggregatedIntProperty(property, value); + } + using DB::GetApproximateSizes; virtual void GetApproximateSizes(ColumnFamilyHandle* column_family, - const Range* r, int n, - uint64_t* sizes) override { - return db_->GetApproximateSizes(column_family, r, n, sizes); + const Range* r, int n, uint64_t* sizes, + uint8_t include_flags + = INCLUDE_FILES) override { + return db_->GetApproximateSizes(column_family, r, n, sizes, + include_flags); + } + + using DB::GetApproximateMemTableStats; + virtual void GetApproximateMemTableStats(ColumnFamilyHandle* column_family, + const Range& range, + uint64_t* const count, + uint64_t* const size) override { + return db_->GetApproximateMemTableStats(column_family, range, count, size); } using DB::CompactRange; - virtual Status CompactRange(ColumnFamilyHandle* column_family, - const Slice* begin, const Slice* end, - bool reduce_level = false, int target_level = -1, - uint32_t target_path_id = 0) override { - return db_->CompactRange(column_family, begin, end, reduce_level, - target_level, target_path_id); + virtual Status CompactRange(const CompactRangeOptions& options, + ColumnFamilyHandle* column_family, + const Slice* begin, const Slice* end) override { + return db_->CompactRange(options, column_family, begin, end); + } + + using DB::CompactFiles; + virtual Status CompactFiles( + const CompactionOptions& compact_options, + ColumnFamilyHandle* column_family, + const std::vector& input_file_names, + const int output_level, const int output_path_id = -1) override { + return db_->CompactFiles( + compact_options, column_family, input_file_names, + output_level, output_path_id); + } + + virtual Status PauseBackgroundWork() override { + return db_->PauseBackgroundWork(); + } + virtual Status ContinueBackgroundWork() override { + return db_->ContinueBackgroundWork(); + } + + virtual Status EnableAutoCompaction( + const std::vector& column_family_handles) override { + return db_->EnableAutoCompaction(column_family_handles); } using DB::NumberLevels; @@ -158,17 +251,29 @@ class StackableDB : public DB { } using DB::GetOptions; - virtual const Options& GetOptions(ColumnFamilyHandle* column_family) const - override { + virtual Options GetOptions(ColumnFamilyHandle* column_family) const override { return db_->GetOptions(column_family); } + using DB::GetDBOptions; + virtual DBOptions GetDBOptions() const override { + return db_->GetDBOptions(); + } + using DB::Flush; virtual Status Flush(const FlushOptions& fopts, ColumnFamilyHandle* column_family) override { return db_->Flush(fopts, column_family); } + virtual Status SyncWAL() override { + return db_->SyncWAL(); + } + + virtual Status FlushWAL(bool sync) override { return db_->FlushWAL(sync); } + +#ifndef ROCKSDB_LITE + virtual Status DisableFileDeletions() override { return db_->DisableFileDeletions(); } @@ -182,6 +287,14 @@ class StackableDB : public DB { db_->GetLiveFilesMetaData(metadata); } + virtual void GetColumnFamilyMetaData( + ColumnFamilyHandle *column_family, + ColumnFamilyMetaData* cf_meta) override { + db_->GetColumnFamilyMetaData(column_family, cf_meta); + } + +#endif // ROCKSDB_LITE + virtual Status GetLiveFiles(std::vector& vec, uint64_t* mfs, bool flush_memtable = true) override { return db_->GetLiveFiles(vec, mfs, flush_memtable); @@ -199,22 +312,57 @@ class StackableDB : public DB { return db_->DeleteFile(name); } - virtual Status GetDbIdentity(std::string& identity) { + virtual Status GetDbIdentity(std::string& identity) const override { return db_->GetDbIdentity(identity); } + using DB::SetOptions; + virtual Status SetOptions(ColumnFamilyHandle* column_family_handle, + const std::unordered_map& + new_options) override { + return db_->SetOptions(column_family_handle, new_options); + } + + virtual Status SetDBOptions( + const std::unordered_map& new_options) + override { + return db_->SetDBOptions(new_options); + } + + using DB::ResetStats; + virtual Status ResetStats() override { return db_->ResetStats(); } + using DB::GetPropertiesOfAllTables; - virtual Status GetPropertiesOfAllTables(ColumnFamilyHandle* column_family, - TablePropertiesCollection* props) { + virtual Status GetPropertiesOfAllTables( + ColumnFamilyHandle* column_family, + TablePropertiesCollection* props) override { return db_->GetPropertiesOfAllTables(column_family, props); } + using DB::GetPropertiesOfTablesInRange; + virtual Status GetPropertiesOfTablesInRange( + ColumnFamilyHandle* column_family, const Range* range, std::size_t n, + TablePropertiesCollection* props) override { + return db_->GetPropertiesOfTablesInRange(column_family, range, n, props); + } + virtual Status GetUpdatesSince( SequenceNumber seq_number, unique_ptr* iter, const TransactionLogIterator::ReadOptions& read_options) override { return db_->GetUpdatesSince(seq_number, iter, read_options); } + virtual Status SuggestCompactRange(ColumnFamilyHandle* column_family, + const Slice* begin, + const Slice* end) override { + return db_->SuggestCompactRange(column_family, begin, end); + } + + virtual Status PromoteL0(ColumnFamilyHandle* column_family, + int target_level) override { + return db_->PromoteL0(column_family, target_level); + } + virtual ColumnFamilyHandle* DefaultColumnFamily() const override { return db_->DefaultColumnFamily(); } diff --git a/include/rocksdb/utilities/table_properties_collectors.h b/include/rocksdb/utilities/table_properties_collectors.h new file mode 100644 index 00000000000..0f8827037b2 --- /dev/null +++ b/include/rocksdb/utilities/table_properties_collectors.h @@ -0,0 +1,29 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#ifndef ROCKSDB_LITE +#include + +#include "rocksdb/table_properties.h" + +namespace rocksdb { + +// Creates a factory of a table property collector that marks a SST +// file as need-compaction when it observe at least "D" deletion +// entries in any "N" consecutive entires. +// +// @param sliding_window_size "N". Note that this number will be +// round up to the smallest multiple of 128 that is no less +// than the specified size. +// @param deletion_trigger "D". Note that even when "N" is changed, +// the specified number for "D" will not be changed. +extern std::shared_ptr + NewCompactOnDeletionCollectorFactory( + size_t sliding_window_size, + size_t deletion_trigger); +} // namespace rocksdb + +#endif // !ROCKSDB_LITE diff --git a/include/rocksdb/utilities/transaction.h b/include/rocksdb/utilities/transaction.h new file mode 100644 index 00000000000..a3519739c21 --- /dev/null +++ b/include/rocksdb/utilities/transaction.h @@ -0,0 +1,479 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#ifndef ROCKSDB_LITE + +#include +#include + +#include "rocksdb/comparator.h" +#include "rocksdb/db.h" +#include "rocksdb/status.h" + +namespace rocksdb { + +class Iterator; +class TransactionDB; +class WriteBatchWithIndex; + +using TransactionName = std::string; + +using TransactionID = uint64_t; + +// Provides notification to the caller of SetSnapshotOnNextOperation when +// the actual snapshot gets created +class TransactionNotifier { + public: + virtual ~TransactionNotifier() {} + + // Implement this method to receive notification when a snapshot is + // requested via SetSnapshotOnNextOperation. + virtual void SnapshotCreated(const Snapshot* newSnapshot) = 0; +}; + +// Provides BEGIN/COMMIT/ROLLBACK transactions. +// +// To use transactions, you must first create either an OptimisticTransactionDB +// or a TransactionDB. See examples/[optimistic_]transaction_example.cc for +// more information. +// +// To create a transaction, use [Optimistic]TransactionDB::BeginTransaction(). +// +// It is up to the caller to synchronize access to this object. +// +// See examples/transaction_example.cc for some simple examples. +// +// TODO(agiardullo): Not yet implemented +// -PerfContext statistics +// -Support for using Transactions with DBWithTTL +class Transaction { + public: + virtual ~Transaction() {} + + // If a transaction has a snapshot set, the transaction will ensure that + // any keys successfully written(or fetched via GetForUpdate()) have not + // been modified outside of this transaction since the time the snapshot was + // set. + // If a snapshot has not been set, the transaction guarantees that keys have + // not been modified since the time each key was first written (or fetched via + // GetForUpdate()). + // + // Using SetSnapshot() will provide stricter isolation guarantees at the + // expense of potentially more transaction failures due to conflicts with + // other writes. + // + // Calling SetSnapshot() has no effect on keys written before this function + // has been called. + // + // SetSnapshot() may be called multiple times if you would like to change + // the snapshot used for different operations in this transaction. + // + // Calling SetSnapshot will not affect the version of Data returned by Get() + // methods. See Transaction::Get() for more details. + virtual void SetSnapshot() = 0; + + // Similar to SetSnapshot(), but will not change the current snapshot + // until Put/Merge/Delete/GetForUpdate/MultigetForUpdate is called. + // By calling this function, the transaction will essentially call + // SetSnapshot() for you right before performing the next write/GetForUpdate. + // + // Calling SetSnapshotOnNextOperation() will not affect what snapshot is + // returned by GetSnapshot() until the next write/GetForUpdate is executed. + // + // When the snapshot is created the notifier's SnapshotCreated method will + // be called so that the caller can get access to the snapshot. + // + // This is an optimization to reduce the likelihood of conflicts that + // could occur in between the time SetSnapshot() is called and the first + // write/GetForUpdate operation. Eg, this prevents the following + // race-condition: + // + // txn1->SetSnapshot(); + // txn2->Put("A", ...); + // txn2->Commit(); + // txn1->GetForUpdate(opts, "A", ...); // FAIL! + virtual void SetSnapshotOnNextOperation( + std::shared_ptr notifier = nullptr) = 0; + + // Returns the Snapshot created by the last call to SetSnapshot(). + // + // REQUIRED: The returned Snapshot is only valid up until the next time + // SetSnapshot()/SetSnapshotOnNextSavePoint() is called, ClearSnapshot() + // is called, or the Transaction is deleted. + virtual const Snapshot* GetSnapshot() const = 0; + + // Clears the current snapshot (i.e. no snapshot will be 'set') + // + // This removes any snapshot that currently exists or is set to be created + // on the next update operation (SetSnapshotOnNextOperation). + // + // Calling ClearSnapshot() has no effect on keys written before this function + // has been called. + // + // If a reference to a snapshot was retrieved via GetSnapshot(), it will no + // longer be valid and should be discarded after a call to ClearSnapshot(). + virtual void ClearSnapshot() = 0; + + // Prepare the current transation for 2PC + virtual Status Prepare() = 0; + + // Write all batched keys to the db atomically. + // + // Returns OK on success. + // + // May return any error status that could be returned by DB:Write(). + // + // If this transaction was created by an OptimisticTransactionDB(), + // Status::Busy() may be returned if the transaction could not guarantee + // that there are no write conflicts. Status::TryAgain() may be returned + // if the memtable history size is not large enough + // (See max_write_buffer_number_to_maintain). + // + // If this transaction was created by a TransactionDB(), Status::Expired() + // may be returned if this transaction has lived for longer than + // TransactionOptions.expiration. + virtual Status Commit() = 0; + + // Discard all batched writes in this transaction. + virtual Status Rollback() = 0; + + // Records the state of the transaction for future calls to + // RollbackToSavePoint(). May be called multiple times to set multiple save + // points. + virtual void SetSavePoint() = 0; + + // Undo all operations in this transaction (Put, Merge, Delete, PutLogData) + // since the most recent call to SetSavePoint() and removes the most recent + // SetSavePoint(). + // If there is no previous call to SetSavePoint(), returns Status::NotFound() + virtual Status RollbackToSavePoint() = 0; + + // This function is similar to DB::Get() except it will also read pending + // changes in this transaction. Currently, this function will return + // Status::MergeInProgress if the most recent write to the queried key in + // this batch is a Merge. + // + // If read_options.snapshot is not set, the current version of the key will + // be read. Calling SetSnapshot() does not affect the version of the data + // returned. + // + // Note that setting read_options.snapshot will affect what is read from the + // DB but will NOT change which keys are read from this transaction (the keys + // in this transaction do not yet belong to any snapshot and will be fetched + // regardless). + virtual Status Get(const ReadOptions& options, + ColumnFamilyHandle* column_family, const Slice& key, + std::string* value) = 0; + + // An overload of the the above method that receives a PinnableSlice + // For backward compatiblity a default implementation is provided + virtual Status Get(const ReadOptions& options, + ColumnFamilyHandle* column_family, const Slice& key, + PinnableSlice* pinnable_val) { + assert(pinnable_val != nullptr); + auto s = Get(options, column_family, key, pinnable_val->GetSelf()); + pinnable_val->PinSelf(); + return s; + } + + virtual Status Get(const ReadOptions& options, const Slice& key, + std::string* value) = 0; + virtual Status Get(const ReadOptions& options, const Slice& key, + PinnableSlice* pinnable_val) { + assert(pinnable_val != nullptr); + auto s = Get(options, key, pinnable_val->GetSelf()); + pinnable_val->PinSelf(); + return s; + } + + virtual std::vector MultiGet( + const ReadOptions& options, + const std::vector& column_family, + const std::vector& keys, std::vector* values) = 0; + + virtual std::vector MultiGet(const ReadOptions& options, + const std::vector& keys, + std::vector* values) = 0; + + // Read this key and ensure that this transaction will only + // be able to be committed if this key is not written outside this + // transaction after it has first been read (or after the snapshot if a + // snapshot is set in this transaction). The transaction behavior is the + // same regardless of whether the key exists or not. + // + // Note: Currently, this function will return Status::MergeInProgress + // if the most recent write to the queried key in this batch is a Merge. + // + // The values returned by this function are similar to Transaction::Get(). + // If value==nullptr, then this function will not read any data, but will + // still ensure that this key cannot be written to by outside of this + // transaction. + // + // If this transaction was created by an OptimisticTransaction, GetForUpdate() + // could cause commit() to fail. Otherwise, it could return any error + // that could be returned by DB::Get(). + // + // If this transaction was created by a TransactionDB, it can return + // Status::OK() on success, + // Status::Busy() if there is a write conflict, + // Status::TimedOut() if a lock could not be acquired, + // Status::TryAgain() if the memtable history size is not large enough + // (See max_write_buffer_number_to_maintain) + // Status::MergeInProgress() if merge operations cannot be resolved. + // or other errors if this key could not be read. + virtual Status GetForUpdate(const ReadOptions& options, + ColumnFamilyHandle* column_family, + const Slice& key, std::string* value, + bool exclusive = true) = 0; + + // An overload of the the above method that receives a PinnableSlice + // For backward compatiblity a default implementation is provided + virtual Status GetForUpdate(const ReadOptions& options, + ColumnFamilyHandle* column_family, + const Slice& key, PinnableSlice* pinnable_val, + bool exclusive = true) { + if (pinnable_val == nullptr) { + std::string* null_str = nullptr; + return GetForUpdate(options, key, null_str); + } else { + auto s = GetForUpdate(options, key, pinnable_val->GetSelf()); + pinnable_val->PinSelf(); + return s; + } + } + + virtual Status GetForUpdate(const ReadOptions& options, const Slice& key, + std::string* value, bool exclusive = true) = 0; + + virtual std::vector MultiGetForUpdate( + const ReadOptions& options, + const std::vector& column_family, + const std::vector& keys, std::vector* values) = 0; + + virtual std::vector MultiGetForUpdate( + const ReadOptions& options, const std::vector& keys, + std::vector* values) = 0; + + // Returns an iterator that will iterate on all keys in the default + // column family including both keys in the DB and uncommitted keys in this + // transaction. + // + // Setting read_options.snapshot will affect what is read from the + // DB but will NOT change which keys are read from this transaction (the keys + // in this transaction do not yet belong to any snapshot and will be fetched + // regardless). + // + // Caller is responsible for deleting the returned Iterator. + // + // The returned iterator is only valid until Commit(), Rollback(), or + // RollbackToSavePoint() is called. + virtual Iterator* GetIterator(const ReadOptions& read_options) = 0; + + virtual Iterator* GetIterator(const ReadOptions& read_options, + ColumnFamilyHandle* column_family) = 0; + + // Put, Merge, Delete, and SingleDelete behave similarly to the corresponding + // functions in WriteBatch, but will also do conflict checking on the + // keys being written. + // + // If this Transaction was created on an OptimisticTransactionDB, these + // functions should always return Status::OK(). + // + // If this Transaction was created on a TransactionDB, the status returned + // can be: + // Status::OK() on success, + // Status::Busy() if there is a write conflict, + // Status::TimedOut() if a lock could not be acquired, + // Status::TryAgain() if the memtable history size is not large enough + // (See max_write_buffer_number_to_maintain) + // or other errors on unexpected failures. + virtual Status Put(ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) = 0; + virtual Status Put(const Slice& key, const Slice& value) = 0; + virtual Status Put(ColumnFamilyHandle* column_family, const SliceParts& key, + const SliceParts& value) = 0; + virtual Status Put(const SliceParts& key, const SliceParts& value) = 0; + + virtual Status Merge(ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) = 0; + virtual Status Merge(const Slice& key, const Slice& value) = 0; + + virtual Status Delete(ColumnFamilyHandle* column_family, + const Slice& key) = 0; + virtual Status Delete(const Slice& key) = 0; + virtual Status Delete(ColumnFamilyHandle* column_family, + const SliceParts& key) = 0; + virtual Status Delete(const SliceParts& key) = 0; + + virtual Status SingleDelete(ColumnFamilyHandle* column_family, + const Slice& key) = 0; + virtual Status SingleDelete(const Slice& key) = 0; + virtual Status SingleDelete(ColumnFamilyHandle* column_family, + const SliceParts& key) = 0; + virtual Status SingleDelete(const SliceParts& key) = 0; + + // PutUntracked() will write a Put to the batch of operations to be committed + // in this transaction. This write will only happen if this transaction + // gets committed successfully. But unlike Transaction::Put(), + // no conflict checking will be done for this key. + // + // If this Transaction was created on a TransactionDB, this function will + // still acquire locks necessary to make sure this write doesn't cause + // conflicts in other transactions and may return Status::Busy(). + virtual Status PutUntracked(ColumnFamilyHandle* column_family, + const Slice& key, const Slice& value) = 0; + virtual Status PutUntracked(const Slice& key, const Slice& value) = 0; + virtual Status PutUntracked(ColumnFamilyHandle* column_family, + const SliceParts& key, + const SliceParts& value) = 0; + virtual Status PutUntracked(const SliceParts& key, + const SliceParts& value) = 0; + + virtual Status MergeUntracked(ColumnFamilyHandle* column_family, + const Slice& key, const Slice& value) = 0; + virtual Status MergeUntracked(const Slice& key, const Slice& value) = 0; + + virtual Status DeleteUntracked(ColumnFamilyHandle* column_family, + const Slice& key) = 0; + + virtual Status DeleteUntracked(const Slice& key) = 0; + virtual Status DeleteUntracked(ColumnFamilyHandle* column_family, + const SliceParts& key) = 0; + virtual Status DeleteUntracked(const SliceParts& key) = 0; + + // Similar to WriteBatch::PutLogData + virtual void PutLogData(const Slice& blob) = 0; + + // By default, all Put/Merge/Delete operations will be indexed in the + // transaction so that Get/GetForUpdate/GetIterator can search for these + // keys. + // + // If the caller does not want to fetch the keys about to be written, + // they may want to avoid indexing as a performance optimization. + // Calling DisableIndexing() will turn off indexing for all future + // Put/Merge/Delete operations until EnableIndexing() is called. + // + // If a key is Put/Merge/Deleted after DisableIndexing is called and then + // is fetched via Get/GetForUpdate/GetIterator, the result of the fetch is + // undefined. + virtual void DisableIndexing() = 0; + virtual void EnableIndexing() = 0; + + // Returns the number of distinct Keys being tracked by this transaction. + // If this transaction was created by a TransactinDB, this is the number of + // keys that are currently locked by this transaction. + // If this transaction was created by an OptimisticTransactionDB, this is the + // number of keys that need to be checked for conflicts at commit time. + virtual uint64_t GetNumKeys() const = 0; + + // Returns the number of Puts/Deletes/Merges that have been applied to this + // transaction so far. + virtual uint64_t GetNumPuts() const = 0; + virtual uint64_t GetNumDeletes() const = 0; + virtual uint64_t GetNumMerges() const = 0; + + // Returns the elapsed time in milliseconds since this Transaction began. + virtual uint64_t GetElapsedTime() const = 0; + + // Fetch the underlying write batch that contains all pending changes to be + // committed. + // + // Note: You should not write or delete anything from the batch directly and + // should only use the functions in the Transaction class to + // write to this transaction. + virtual WriteBatchWithIndex* GetWriteBatch() = 0; + + // Change the value of TransactionOptions.lock_timeout (in milliseconds) for + // this transaction. + // Has no effect on OptimisticTransactions. + virtual void SetLockTimeout(int64_t timeout) = 0; + + // Return the WriteOptions that will be used during Commit() + virtual WriteOptions* GetWriteOptions() = 0; + + // Reset the WriteOptions that will be used during Commit(). + virtual void SetWriteOptions(const WriteOptions& write_options) = 0; + + // If this key was previously fetched in this transaction using + // GetForUpdate/MultigetForUpdate(), calling UndoGetForUpdate will tell + // the transaction that it no longer needs to do any conflict checking + // for this key. + // + // If a key has been fetched N times via GetForUpdate/MultigetForUpdate(), + // then UndoGetForUpdate will only have an effect if it is also called N + // times. If this key has been written to in this transaction, + // UndoGetForUpdate() will have no effect. + // + // If SetSavePoint() has been called after the GetForUpdate(), + // UndoGetForUpdate() will not have any effect. + // + // If this Transaction was created by an OptimisticTransactionDB, + // calling UndoGetForUpdate can affect whether this key is conflict checked + // at commit time. + // If this Transaction was created by a TransactionDB, + // calling UndoGetForUpdate may release any held locks for this key. + virtual void UndoGetForUpdate(ColumnFamilyHandle* column_family, + const Slice& key) = 0; + virtual void UndoGetForUpdate(const Slice& key) = 0; + + virtual Status RebuildFromWriteBatch(WriteBatch* src_batch) = 0; + + virtual WriteBatch* GetCommitTimeWriteBatch() = 0; + + virtual void SetLogNumber(uint64_t log) { log_number_ = log; } + + virtual uint64_t GetLogNumber() const { return log_number_; } + + virtual Status SetName(const TransactionName& name) = 0; + + virtual TransactionName GetName() const { return name_; } + + virtual TransactionID GetID() const { return 0; } + + virtual bool IsDeadlockDetect() const { return false; } + + virtual std::vector GetWaitingTxns(uint32_t* column_family_id, + std::string* key) const { + assert(false); + return std::vector(); + } + + enum TransactionState { + STARTED = 0, + AWAITING_PREPARE = 1, + PREPARED = 2, + AWAITING_COMMIT = 3, + COMMITED = 4, + AWAITING_ROLLBACK = 5, + ROLLEDBACK = 6, + LOCKS_STOLEN = 7, + }; + + TransactionState GetState() const { return txn_state_; } + void SetState(TransactionState state) { txn_state_ = state; } + + protected: + explicit Transaction(const TransactionDB* db) {} + Transaction() {} + + // the log in which the prepared section for this txn resides + // (for two phase commit) + uint64_t log_number_; + TransactionName name_; + + // Execution status of the transaction. + std::atomic txn_state_; + + private: + // No copying allowed + Transaction(const Transaction&); + void operator=(const Transaction&); +}; + +} // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/include/rocksdb/utilities/transaction_db.h b/include/rocksdb/utilities/transaction_db.h new file mode 100644 index 00000000000..77043897a70 --- /dev/null +++ b/include/rocksdb/utilities/transaction_db.h @@ -0,0 +1,226 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#ifndef ROCKSDB_LITE + +#include +#include +#include + +#include "rocksdb/comparator.h" +#include "rocksdb/db.h" +#include "rocksdb/utilities/stackable_db.h" +#include "rocksdb/utilities/transaction.h" + +// Database with Transaction support. +// +// See transaction.h and examples/transaction_example.cc + +namespace rocksdb { + +class TransactionDBMutexFactory; + +enum TxnDBWritePolicy { + WRITE_COMMITTED = 0, // write only the committed data + // TODO(myabandeh): Not implemented yet + WRITE_PREPARED, // write data after the prepare phase of 2pc + // TODO(myabandeh): Not implemented yet + WRITE_UNPREPARED // write data before the prepare phase of 2pc +}; + +const uint32_t kInitialMaxDeadlocks = 5; + +struct TransactionDBOptions { + // Specifies the maximum number of keys that can be locked at the same time + // per column family. + // If the number of locked keys is greater than max_num_locks, transaction + // writes (or GetForUpdate) will return an error. + // If this value is not positive, no limit will be enforced. + int64_t max_num_locks = -1; + + // Stores the number of latest deadlocks to track + uint32_t max_num_deadlocks = kInitialMaxDeadlocks; + + // Increasing this value will increase the concurrency by dividing the lock + // table (per column family) into more sub-tables, each with their own + // separate + // mutex. + size_t num_stripes = 16; + + // If positive, specifies the default wait timeout in milliseconds when + // a transaction attempts to lock a key if not specified by + // TransactionOptions::lock_timeout. + // + // If 0, no waiting is done if a lock cannot instantly be acquired. + // If negative, there is no timeout. Not using a timeout is not recommended + // as it can lead to deadlocks. Currently, there is no deadlock-detection to + // recover + // from a deadlock. + int64_t transaction_lock_timeout = 1000; // 1 second + + // If positive, specifies the wait timeout in milliseconds when writing a key + // OUTSIDE of a transaction (ie by calling DB::Put(),Merge(),Delete(),Write() + // directly). + // If 0, no waiting is done if a lock cannot instantly be acquired. + // If negative, there is no timeout and will block indefinitely when acquiring + // a lock. + // + // Not using a timeout can lead to deadlocks. Currently, there + // is no deadlock-detection to recover from a deadlock. While DB writes + // cannot deadlock with other DB writes, they can deadlock with a transaction. + // A negative timeout should only be used if all transactions have a small + // expiration set. + int64_t default_lock_timeout = 1000; // 1 second + + // If set, the TransactionDB will use this implemenation of a mutex and + // condition variable for all transaction locking instead of the default + // mutex/condvar implementation. + std::shared_ptr custom_mutex_factory; + + // The policy for when to write the data into the DB. The default policy is to + // write only the committed data (WRITE_COMMITTED). The data could be written + // before the commit phase. The DB then needs to provide the mechanisms to + // tell apart committed from uncommitted data. + TxnDBWritePolicy write_policy = TxnDBWritePolicy::WRITE_COMMITTED; +}; + +struct TransactionOptions { + // Setting set_snapshot=true is the same as calling + // Transaction::SetSnapshot(). + bool set_snapshot = false; + + // Setting to true means that before acquiring locks, this transaction will + // check if doing so will cause a deadlock. If so, it will return with + // Status::Busy. The user should retry their transaction. + bool deadlock_detect = false; + + // TODO(agiardullo): TransactionDB does not yet support comparators that allow + // two non-equal keys to be equivalent. Ie, cmp->Compare(a,b) should only + // return 0 if + // a.compare(b) returns 0. + + + // If positive, specifies the wait timeout in milliseconds when + // a transaction attempts to lock a key. + // + // If 0, no waiting is done if a lock cannot instantly be acquired. + // If negative, TransactionDBOptions::transaction_lock_timeout will be used. + int64_t lock_timeout = -1; + + // Expiration duration in milliseconds. If non-negative, transactions that + // last longer than this many milliseconds will fail to commit. If not set, + // a forgotten transaction that is never committed, rolled back, or deleted + // will never relinquish any locks it holds. This could prevent keys from + // being written by other writers. + int64_t expiration = -1; + + // The number of traversals to make during deadlock detection. + int64_t deadlock_detect_depth = 50; + + // The maximum number of bytes used for the write batch. 0 means no limit. + size_t max_write_batch_size = 0; +}; + +struct KeyLockInfo { + std::string key; + std::vector ids; + bool exclusive; +}; + +struct DeadlockInfo { + TransactionID m_txn_id; + uint32_t m_cf_id; + std::string m_waiting_key; + bool m_exclusive; +}; + +struct DeadlockPath { + std::vector path; + bool limit_exceeded; + + explicit DeadlockPath(std::vector path_entry) + : path(path_entry), limit_exceeded(false) {} + + // empty path, limit exceeded constructor and default constructor + explicit DeadlockPath(bool limit = false) : path(0), limit_exceeded(limit) {} + + bool empty() { return path.empty() && !limit_exceeded; } +}; + +class TransactionDB : public StackableDB { + public: + // Open a TransactionDB similar to DB::Open(). + // Internally call PrepareWrap() and WrapDB() + static Status Open(const Options& options, + const TransactionDBOptions& txn_db_options, + const std::string& dbname, TransactionDB** dbptr); + + static Status Open(const DBOptions& db_options, + const TransactionDBOptions& txn_db_options, + const std::string& dbname, + const std::vector& column_families, + std::vector* handles, + TransactionDB** dbptr); + // The following functions are used to open a TransactionDB internally using + // an opened DB or StackableDB. + // 1. Call prepareWrap(), passing an empty std::vector to + // compaction_enabled_cf_indices. + // 2. Open DB or Stackable DB with db_options and column_families passed to + // prepareWrap() + // Note: PrepareWrap() may change parameters, make copies before the + // invocation if needed. + // 3. Call Wrap*DB() with compaction_enabled_cf_indices in step 1 and handles + // of the opened DB/StackableDB in step 2 + static void PrepareWrap(DBOptions* db_options, + std::vector* column_families, + std::vector* compaction_enabled_cf_indices); + static Status WrapDB(DB* db, const TransactionDBOptions& txn_db_options, + const std::vector& compaction_enabled_cf_indices, + const std::vector& handles, + TransactionDB** dbptr); + static Status WrapStackableDB( + StackableDB* db, const TransactionDBOptions& txn_db_options, + const std::vector& compaction_enabled_cf_indices, + const std::vector& handles, TransactionDB** dbptr); + ~TransactionDB() override {} + + // Starts a new Transaction. + // + // Caller is responsible for deleting the returned transaction when no + // longer needed. + // + // If old_txn is not null, BeginTransaction will reuse this Transaction + // handle instead of allocating a new one. This is an optimization to avoid + // extra allocations when repeatedly creating transactions. + virtual Transaction* BeginTransaction( + const WriteOptions& write_options, + const TransactionOptions& txn_options = TransactionOptions(), + Transaction* old_txn = nullptr) = 0; + + virtual Transaction* GetTransactionByName(const TransactionName& name) = 0; + virtual void GetAllPreparedTransactions(std::vector* trans) = 0; + + // Returns set of all locks held. + // + // The mapping is column family id -> KeyLockInfo + virtual std::unordered_multimap + GetLockStatusData() = 0; + virtual std::vector GetDeadlockInfoBuffer() = 0; + virtual void SetDeadlockInfoBufferSize(uint32_t target_size) = 0; + + protected: + // To Create an TransactionDB, call Open() + explicit TransactionDB(DB* db) : StackableDB(db) {} + + private: + // No copying allowed + TransactionDB(const TransactionDB&); + void operator=(const TransactionDB&); +}; + +} // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/include/rocksdb/utilities/transaction_db_mutex.h b/include/rocksdb/utilities/transaction_db_mutex.h new file mode 100644 index 00000000000..df59e7a9e5b --- /dev/null +++ b/include/rocksdb/utilities/transaction_db_mutex.h @@ -0,0 +1,92 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#ifndef ROCKSDB_LITE + +#include + +#include "rocksdb/status.h" + +namespace rocksdb { + +// TransactionDBMutex and TransactionDBCondVar APIs allows applications to +// implement custom mutexes and condition variables to be used by a +// TransactionDB when locking keys. +// +// To open a TransactionDB with a custom TransactionDBMutexFactory, set +// TransactionDBOptions.custom_mutex_factory. + +class TransactionDBMutex { + public: + virtual ~TransactionDBMutex() {} + + // Attempt to acquire lock. Return OK on success, or other Status on failure. + // If returned status is OK, TransactionDB will eventually call UnLock(). + virtual Status Lock() = 0; + + // Attempt to acquire lock. If timeout is non-negative, operation may be + // failed after this many microseconds. + // Returns OK on success, + // TimedOut if timed out, + // or other Status on failure. + // If returned status is OK, TransactionDB will eventually call UnLock(). + virtual Status TryLockFor(int64_t timeout_time) = 0; + + // Unlock Mutex that was successfully locked by Lock() or TryLockUntil() + virtual void UnLock() = 0; +}; + +class TransactionDBCondVar { + public: + virtual ~TransactionDBCondVar() {} + + // Block current thread until condition variable is notified by a call to + // Notify() or NotifyAll(). Wait() will be called with mutex locked. + // Returns OK if notified. + // Returns non-OK if TransactionDB should stop waiting and fail the operation. + // May return OK spuriously even if not notified. + virtual Status Wait(std::shared_ptr mutex) = 0; + + // Block current thread until condition variable is notified by a call to + // Notify() or NotifyAll(), or if the timeout is reached. + // Wait() will be called with mutex locked. + // + // If timeout is non-negative, operation should be failed after this many + // microseconds. + // If implementing a custom version of this class, the implementation may + // choose to ignore the timeout. + // + // Returns OK if notified. + // Returns TimedOut if timeout is reached. + // Returns other status if TransactionDB should otherwis stop waiting and + // fail the operation. + // May return OK spuriously even if not notified. + virtual Status WaitFor(std::shared_ptr mutex, + int64_t timeout_time) = 0; + + // If any threads are waiting on *this, unblock at least one of the + // waiting threads. + virtual void Notify() = 0; + + // Unblocks all threads waiting on *this. + virtual void NotifyAll() = 0; +}; + +// Factory class that can allocate mutexes and condition variables. +class TransactionDBMutexFactory { + public: + // Create a TransactionDBMutex object. + virtual std::shared_ptr AllocateMutex() = 0; + + // Create a TransactionDBCondVar object. + virtual std::shared_ptr AllocateCondVar() = 0; + + virtual ~TransactionDBMutexFactory() {} +}; + +} // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/include/rocksdb/utilities/utility_db.h b/include/rocksdb/utilities/utility_db.h index f4db6653237..a34a638980f 100644 --- a/include/rocksdb/utilities/utility_db.h +++ b/include/rocksdb/utilities/utility_db.h @@ -19,7 +19,12 @@ class UtilityDB { // This function is here only for backwards compatibility. Please use the // functions defined in DBWithTTl (rocksdb/utilities/db_ttl.h) // (deprecated) - __attribute__((deprecated)) static Status OpenTtlDB(const Options& options, +#if defined(__GNUC__) || defined(__clang__) + __attribute__((deprecated)) +#elif _WIN32 + __declspec(deprecated) +#endif + static Status OpenTtlDB(const Options& options, const std::string& name, StackableDB** dbptr, int32_t ttl = 0, diff --git a/include/rocksdb/utilities/write_batch_with_index.h b/include/rocksdb/utilities/write_batch_with_index.h index c09f53d112a..24d8f30aa51 100644 --- a/include/rocksdb/utilities/write_batch_with_index.h +++ b/include/rocksdb/utilities/write_batch_with_index.h @@ -1,30 +1,47 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. // // A WriteBatchWithIndex with a binary searchable index built for all the keys // inserted. - #pragma once -#include "rocksdb/status.h" +#ifndef ROCKSDB_LITE + +#include +#include + +#include "rocksdb/comparator.h" +#include "rocksdb/iterator.h" #include "rocksdb/slice.h" +#include "rocksdb/status.h" #include "rocksdb/write_batch.h" +#include "rocksdb/write_batch_base.h" namespace rocksdb { class ColumnFamilyHandle; -struct SliceParts; class Comparator; +class DB; +struct ReadOptions; +struct DBOptions; + +enum WriteType { + kPutRecord, + kMergeRecord, + kDeleteRecord, + kSingleDeleteRecord, + kDeleteRangeRecord, + kLogDataRecord, + kXIDRecord, +}; -enum WriteType { kPutRecord, kMergeRecord, kDeleteRecord, kLogDataRecord }; - -// an entry for Put, Merge or Delete entry for write batches. Used in -// WBWIIterator. +// an entry for Put, Merge, Delete, or SingleDelete entry for write batches. +// Used in WBWIIterator. struct WriteEntry { WriteType type; Slice key; @@ -38,65 +55,181 @@ class WBWIIterator { virtual bool Valid() const = 0; + virtual void SeekToFirst() = 0; + + virtual void SeekToLast() = 0; + virtual void Seek(const Slice& key) = 0; + virtual void SeekForPrev(const Slice& key) = 0; + virtual void Next() = 0; - virtual const WriteEntry& Entry() const = 0; + virtual void Prev() = 0; + + // the return WriteEntry is only valid until the next mutation of + // WriteBatchWithIndex + virtual WriteEntry Entry() const = 0; virtual Status status() const = 0; }; // A WriteBatchWithIndex with a binary searchable index built for all the keys // inserted. -// In Put(), Merge() or Delete(), the same function of the wrapped will be -// called. At the same time, indexes will be built. +// In Put(), Merge() Delete(), or SingleDelete(), the same function of the +// wrapped will be called. At the same time, indexes will be built. // By calling GetWriteBatch(), a user will get the WriteBatch for the data // they inserted, which can be used for DB::Write(). // A user can call NewIterator() to create an iterator. -class WriteBatchWithIndex { +class WriteBatchWithIndex : public WriteBatchBase { public: - // index_comparator indicates the order when iterating data in the write - // batch. Technically, it doesn't have to be the same as the one used in - // the DB. + // backup_index_comparator: the backup comparator used to compare keys + // within the same column family, if column family is not given in the + // interface, or we can't find a column family from the column family handle + // passed in, backup_index_comparator will be used for the column family. // reserved_bytes: reserved bytes in underlying WriteBatch - explicit WriteBatchWithIndex(const Comparator* index_comparator, - size_t reserved_bytes = 0); - virtual ~WriteBatchWithIndex(); + // max_bytes: maximum size of underlying WriteBatch in bytes + // overwrite_key: if true, overwrite the key in the index when inserting + // the same key as previously, so iterator will never + // show two entries with the same key. + explicit WriteBatchWithIndex( + const Comparator* backup_index_comparator = BytewiseComparator(), + size_t reserved_bytes = 0, bool overwrite_key = false, + size_t max_bytes = 0); - WriteBatch* GetWriteBatch(); + ~WriteBatchWithIndex() override; - virtual void Put(ColumnFamilyHandle* column_family, const Slice& key, - const Slice& value); + using WriteBatchBase::Put; + Status Put(ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) override; - virtual void Put(const Slice& key, const Slice& value); + Status Put(const Slice& key, const Slice& value) override; - virtual void Merge(ColumnFamilyHandle* column_family, const Slice& key, - const Slice& value); + using WriteBatchBase::Merge; + Status Merge(ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) override; - virtual void Merge(const Slice& key, const Slice& value); + Status Merge(const Slice& key, const Slice& value) override; - virtual void PutLogData(const Slice& blob); + using WriteBatchBase::Delete; + Status Delete(ColumnFamilyHandle* column_family, const Slice& key) override; + Status Delete(const Slice& key) override; - virtual void Delete(ColumnFamilyHandle* column_family, const Slice& key); - virtual void Delete(const Slice& key); + using WriteBatchBase::SingleDelete; + Status SingleDelete(ColumnFamilyHandle* column_family, + const Slice& key) override; + Status SingleDelete(const Slice& key) override; - virtual void Delete(ColumnFamilyHandle* column_family, const SliceParts& key); + using WriteBatchBase::DeleteRange; + Status DeleteRange(ColumnFamilyHandle* column_family, const Slice& begin_key, + const Slice& end_key) override; + Status DeleteRange(const Slice& begin_key, const Slice& end_key) override; - virtual void Delete(const SliceParts& key); + using WriteBatchBase::PutLogData; + Status PutLogData(const Slice& blob) override; + + using WriteBatchBase::Clear; + void Clear() override; + + using WriteBatchBase::GetWriteBatch; + WriteBatch* GetWriteBatch() override; // Create an iterator of a column family. User can call iterator.Seek() to // search to the next entry of or after a key. Keys will be iterated in the // order given by index_comparator. For multiple updates on the same key, // each update will be returned as a separate entry, in the order of update // time. - virtual WBWIIterator* NewIterator(ColumnFamilyHandle* column_family); + // + // The returned iterator should be deleted by the caller. + WBWIIterator* NewIterator(ColumnFamilyHandle* column_family); // Create an iterator of the default column family. - virtual WBWIIterator* NewIterator(); + WBWIIterator* NewIterator(); + + // Will create a new Iterator that will use WBWIIterator as a delta and + // base_iterator as base. + // + // This function is only supported if the WriteBatchWithIndex was + // constructed with overwrite_key=true. + // + // The returned iterator should be deleted by the caller. + // The base_iterator is now 'owned' by the returned iterator. Deleting the + // returned iterator will also delete the base_iterator. + Iterator* NewIteratorWithBase(ColumnFamilyHandle* column_family, + Iterator* base_iterator); + // default column family + Iterator* NewIteratorWithBase(Iterator* base_iterator); + + // Similar to DB::Get() but will only read the key from this batch. + // If the batch does not have enough data to resolve Merge operations, + // MergeInProgress status may be returned. + Status GetFromBatch(ColumnFamilyHandle* column_family, + const DBOptions& options, const Slice& key, + std::string* value); + + // Similar to previous function but does not require a column_family. + // Note: An InvalidArgument status will be returned if there are any Merge + // operators for this key. Use previous method instead. + Status GetFromBatch(const DBOptions& options, const Slice& key, + std::string* value) { + return GetFromBatch(nullptr, options, key, value); + } + + // Similar to DB::Get() but will also read writes from this batch. + // + // This function will query both this batch and the DB and then merge + // the results using the DB's merge operator (if the batch contains any + // merge requests). + // + // Setting read_options.snapshot will affect what is read from the DB + // but will NOT change which keys are read from the batch (the keys in + // this batch do not yet belong to any snapshot and will be fetched + // regardless). + Status GetFromBatchAndDB(DB* db, const ReadOptions& read_options, + const Slice& key, std::string* value); + + // An overload of the the above method that receives a PinnableSlice + Status GetFromBatchAndDB(DB* db, const ReadOptions& read_options, + const Slice& key, PinnableSlice* value); + + Status GetFromBatchAndDB(DB* db, const ReadOptions& read_options, + ColumnFamilyHandle* column_family, const Slice& key, + std::string* value); + + // An overload of the the above method that receives a PinnableSlice + Status GetFromBatchAndDB(DB* db, const ReadOptions& read_options, + ColumnFamilyHandle* column_family, const Slice& key, + PinnableSlice* value); + + // Records the state of the batch for future calls to RollbackToSavePoint(). + // May be called multiple times to set multiple save points. + void SetSavePoint() override; + + // Remove all entries in this batch (Put, Merge, Delete, SingleDelete, + // PutLogData) since the most recent call to SetSavePoint() and removes the + // most recent save point. + // If there is no previous call to SetSavePoint(), behaves the same as + // Clear(). + // + // Calling RollbackToSavePoint invalidates any open iterators on this batch. + // + // Returns Status::OK() on success, + // Status::NotFound() if no previous call to SetSavePoint(), + // or other Status on corruption. + Status RollbackToSavePoint() override; + + // Pop the most recent save point. + // If there is no previous call to SetSavePoint(), Status::NotFound() + // will be returned. + // Otherwise returns Status::OK(). + Status PopSavePoint() override; + + void SetMaxBytes(size_t max_bytes) override; private: struct Rep; - Rep* rep; + std::unique_ptr rep; }; } // namespace rocksdb + +#endif // !ROCKSDB_LITE diff --git a/include/rocksdb/version.h b/include/rocksdb/version.h index 41098684311..d592e6dae39 100644 --- a/include/rocksdb/version.h +++ b/include/rocksdb/version.h @@ -1,13 +1,12 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once -// Also update Makefile if you change these -#define ROCKSDB_MAJOR 3 -#define ROCKSDB_MINOR 5 -#define ROCKSDB_PATCH 1 +#define ROCKSDB_MAJOR 5 +#define ROCKSDB_MINOR 8 +#define ROCKSDB_PATCH 7 // Do not use these. We made the mistake of declaring macros starting with // double underscore. Now we have to live with our choice. We'll deprecate these diff --git a/include/rocksdb/wal_filter.h b/include/rocksdb/wal_filter.h new file mode 100644 index 00000000000..686fa499893 --- /dev/null +++ b/include/rocksdb/wal_filter.h @@ -0,0 +1,101 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#include +#include + +namespace rocksdb { + +class WriteBatch; + +// WALFilter allows an application to inspect write-ahead-log (WAL) +// records or modify their processing on recovery. +// Please see the details below. +class WalFilter { + public: + enum class WalProcessingOption { + // Continue processing as usual + kContinueProcessing = 0, + // Ignore the current record but continue processing of log(s) + kIgnoreCurrentRecord = 1, + // Stop replay of logs and discard logs + // Logs won't be replayed on subsequent recovery + kStopReplay = 2, + // Corrupted record detected by filter + kCorruptedRecord = 3, + // Marker for enum count + kWalProcessingOptionMax = 4 + }; + + virtual ~WalFilter() {} + + // Provide ColumnFamily->LogNumber map to filter + // so that filter can determine whether a log number applies to a given + // column family (i.e. that log hasn't been flushed to SST already for the + // column family). + // We also pass in name->id map as only name is known during + // recovery (as handles are opened post-recovery). + // while write batch callbacks happen in terms of column family id. + // + // @params cf_lognumber_map column_family_id to lognumber map + // @params cf_name_id_map column_family_name to column_family_id map + + virtual void ColumnFamilyLogNumberMap( + const std::map& cf_lognumber_map, + const std::map& cf_name_id_map) {} + + // LogRecord is invoked for each log record encountered for all the logs + // during replay on logs on recovery. This method can be used to: + // * inspect the record (using the batch parameter) + // * ignoring current record + // (by returning WalProcessingOption::kIgnoreCurrentRecord) + // * reporting corrupted record + // (by returning WalProcessingOption::kCorruptedRecord) + // * stop log replay + // (by returning kStop replay) - please note that this implies + // discarding the logs from current record onwards. + // + // @params log_number log_number of the current log. + // Filter might use this to determine if the log + // record is applicable to a certain column family. + // @params log_file_name log file name - only for informational purposes + // @params batch batch encountered in the log during recovery + // @params new_batch new_batch to populate if filter wants to change + // the batch (for example to filter some records out, + // or alter some records). + // Please note that the new batch MUST NOT contain + // more records than original, else recovery would + // be failed. + // @params batch_changed Whether batch was changed by the filter. + // It must be set to true if new_batch was populated, + // else new_batch has no effect. + // @returns Processing option for the current record. + // Please see WalProcessingOption enum above for + // details. + virtual WalProcessingOption LogRecordFound(unsigned long long log_number, + const std::string& log_file_name, + const WriteBatch& batch, + WriteBatch* new_batch, + bool* batch_changed) { + // Default implementation falls back to older function for compatibility + return LogRecord(batch, new_batch, batch_changed); + } + + // Please see the comments for LogRecord above. This function is for + // compatibility only and contains a subset of parameters. + // New code should use the function above. + virtual WalProcessingOption LogRecord(const WriteBatch& batch, + WriteBatch* new_batch, + bool* batch_changed) const { + return WalProcessingOption::kContinueProcessing; + } + + // Returns a name that identifies this WAL filter. + // The name will be printed to LOG file on start up for diagnosis. + virtual const char* Name() const = 0; +}; + +} // namespace rocksdb diff --git a/include/rocksdb/write_batch.h b/include/rocksdb/write_batch.h index db440be023f..336391ead56 100644 --- a/include/rocksdb/write_batch.h +++ b/include/rocksdb/write_batch.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. @@ -25,71 +25,161 @@ #ifndef STORAGE_ROCKSDB_INCLUDE_WRITE_BATCH_H_ #define STORAGE_ROCKSDB_INCLUDE_WRITE_BATCH_H_ +#include +#include #include +#include #include "rocksdb/status.h" +#include "rocksdb/write_batch_base.h" namespace rocksdb { class Slice; class ColumnFamilyHandle; +struct SavePoints; struct SliceParts; -class WriteBatch { +struct SavePoint { + size_t size; // size of rep_ + int count; // count of elements in rep_ + uint32_t content_flags; + + SavePoint() : size(0), count(0), content_flags(0) {} + + SavePoint(size_t _size, int _count, uint32_t _flags) + : size(_size), count(_count), content_flags(_flags) {} + + void clear() { + size = 0; + count = 0; + content_flags = 0; + } + + bool is_cleared() const { return (size | count | content_flags) == 0; } +}; + +class WriteBatch : public WriteBatchBase { public: - explicit WriteBatch(size_t reserved_bytes = 0); - ~WriteBatch(); + explicit WriteBatch(size_t reserved_bytes = 0, size_t max_bytes = 0); + ~WriteBatch() override; + using WriteBatchBase::Put; // Store the mapping "key->value" in the database. - void Put(ColumnFamilyHandle* column_family, const Slice& key, - const Slice& value); - void Put(const Slice& key, const Slice& value) { - Put(nullptr, key, value); + Status Put(ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) override; + Status Put(const Slice& key, const Slice& value) override { + return Put(nullptr, key, value); } // Variant of Put() that gathers output like writev(2). The key and value - // that will be written to the database are concatentations of arrays of + // that will be written to the database are concatenations of arrays of // slices. - void Put(ColumnFamilyHandle* column_family, const SliceParts& key, - const SliceParts& value); - void Put(const SliceParts& key, const SliceParts& value) { - Put(nullptr, key, value); + Status Put(ColumnFamilyHandle* column_family, const SliceParts& key, + const SliceParts& value) override; + Status Put(const SliceParts& key, const SliceParts& value) override { + return Put(nullptr, key, value); + } + + using WriteBatchBase::Delete; + // If the database contains a mapping for "key", erase it. Else do nothing. + Status Delete(ColumnFamilyHandle* column_family, const Slice& key) override; + Status Delete(const Slice& key) override { return Delete(nullptr, key); } + + // variant that takes SliceParts + Status Delete(ColumnFamilyHandle* column_family, + const SliceParts& key) override; + Status Delete(const SliceParts& key) override { return Delete(nullptr, key); } + + using WriteBatchBase::SingleDelete; + // WriteBatch implementation of DB::SingleDelete(). See db.h. + Status SingleDelete(ColumnFamilyHandle* column_family, + const Slice& key) override; + Status SingleDelete(const Slice& key) override { + return SingleDelete(nullptr, key); + } + + // variant that takes SliceParts + Status SingleDelete(ColumnFamilyHandle* column_family, + const SliceParts& key) override; + Status SingleDelete(const SliceParts& key) override { + return SingleDelete(nullptr, key); + } + + using WriteBatchBase::DeleteRange; + // WriteBatch implementation of DB::DeleteRange(). See db.h. + Status DeleteRange(ColumnFamilyHandle* column_family, const Slice& begin_key, + const Slice& end_key) override; + Status DeleteRange(const Slice& begin_key, const Slice& end_key) override { + return DeleteRange(nullptr, begin_key, end_key); + } + + // variant that takes SliceParts + Status DeleteRange(ColumnFamilyHandle* column_family, + const SliceParts& begin_key, + const SliceParts& end_key) override; + Status DeleteRange(const SliceParts& begin_key, + const SliceParts& end_key) override { + return DeleteRange(nullptr, begin_key, end_key); } + using WriteBatchBase::Merge; // Merge "value" with the existing value of "key" in the database. // "key->merge(existing, value)" - void Merge(ColumnFamilyHandle* column_family, const Slice& key, - const Slice& value); - void Merge(const Slice& key, const Slice& value) { - Merge(nullptr, key, value); + Status Merge(ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) override; + Status Merge(const Slice& key, const Slice& value) override { + return Merge(nullptr, key, value); } - // If the database contains a mapping for "key", erase it. Else do nothing. - void Delete(ColumnFamilyHandle* column_family, const Slice& key); - void Delete(const Slice& key) { Delete(nullptr, key); } - // variant that takes SliceParts - void Delete(ColumnFamilyHandle* column_family, const SliceParts& key); - void Delete(const SliceParts& key) { Delete(nullptr, key); } + Status Merge(ColumnFamilyHandle* column_family, const SliceParts& key, + const SliceParts& value) override; + Status Merge(const SliceParts& key, const SliceParts& value) override { + return Merge(nullptr, key, value); + } + using WriteBatchBase::PutLogData; // Append a blob of arbitrary size to the records in this batch. The blob will // be stored in the transaction log but not in any other file. In particular, // it will not be persisted to the SST files. When iterating over this // WriteBatch, WriteBatch::Handler::LogData will be called with the contents // of the blob as it is encountered. Blobs, puts, deletes, and merges will be - // encountered in the same order in thich they were inserted. The blob will + // encountered in the same order in which they were inserted. The blob will // NOT consume sequence number(s) and will NOT increase the count of the batch // // Example application: add timestamps to the transaction log for use in // replication. - void PutLogData(const Slice& blob); + Status PutLogData(const Slice& blob) override; + using WriteBatchBase::Clear; // Clear all updates buffered in this batch. - void Clear(); + void Clear() override; + + // Records the state of the batch for future calls to RollbackToSavePoint(). + // May be called multiple times to set multiple save points. + void SetSavePoint() override; + + // Remove all entries in this batch (Put, Merge, Delete, PutLogData) since the + // most recent call to SetSavePoint() and removes the most recent save point. + // If there is no previous call to SetSavePoint(), Status::NotFound() + // will be returned. + // Otherwise returns Status::OK(). + Status RollbackToSavePoint() override; + + // Pop the most recent save point. + // If there is no previous call to SetSavePoint(), Status::NotFound() + // will be returned. + // Otherwise returns Status::OK(). + Status PopSavePoint() override; // Support for iterating over the contents of a batch. class Handler { public: virtual ~Handler(); + // All handler functions in this class provide default implementations so + // we won't break existing clients of Handler on a source code level when + // adding a new member function. + // default implementation will just call Put without column family for // backwards compatibility. If the column family is not default, // the function is noop @@ -105,10 +195,33 @@ class WriteBatch { return Status::InvalidArgument( "non-default column family and PutCF not implemented"); } - virtual void Put(const Slice& key, const Slice& value); - // Merge and LogData are not pure virtual. Otherwise, we would break - // existing clients of Handler on a source code level. The default - // implementation of Merge simply throws a runtime exception. + virtual void Put(const Slice& /*key*/, const Slice& /*value*/) {} + + virtual Status DeleteCF(uint32_t column_family_id, const Slice& key) { + if (column_family_id == 0) { + Delete(key); + return Status::OK(); + } + return Status::InvalidArgument( + "non-default column family and DeleteCF not implemented"); + } + virtual void Delete(const Slice& /*key*/) {} + + virtual Status SingleDeleteCF(uint32_t column_family_id, const Slice& key) { + if (column_family_id == 0) { + SingleDelete(key); + return Status::OK(); + } + return Status::InvalidArgument( + "non-default column family and SingleDeleteCF not implemented"); + } + virtual void SingleDelete(const Slice& /*key*/) {} + + virtual Status DeleteRangeCF(uint32_t column_family_id, + const Slice& begin_key, const Slice& end_key) { + return Status::InvalidArgument("DeleteRangeCF not implemented"); + } + virtual Status MergeCF(uint32_t column_family_id, const Slice& key, const Slice& value) { if (column_family_id == 0) { @@ -118,18 +231,34 @@ class WriteBatch { return Status::InvalidArgument( "non-default column family and MergeCF not implemented"); } - virtual void Merge(const Slice& key, const Slice& value); + virtual void Merge(const Slice& /*key*/, const Slice& /*value*/) {} + + virtual Status PutBlobIndexCF(uint32_t /*column_family_id*/, + const Slice& /*key*/, + const Slice& /*value*/) { + return Status::InvalidArgument("PutBlobIndexCF not implemented"); + } + // The default implementation of LogData does nothing. virtual void LogData(const Slice& blob); - virtual Status DeleteCF(uint32_t column_family_id, const Slice& key) { - if (column_family_id == 0) { - Delete(key); - return Status::OK(); - } + + virtual Status MarkBeginPrepare() { + return Status::InvalidArgument("MarkBeginPrepare() handler not defined."); + } + + virtual Status MarkEndPrepare(const Slice& xid) { + return Status::InvalidArgument("MarkEndPrepare() handler not defined."); + } + + virtual Status MarkRollback(const Slice& xid) { return Status::InvalidArgument( - "non-default column family and DeleteCF not implemented"); + "MarkRollbackPrepare() handler not defined."); } - virtual void Delete(const Slice& key); + + virtual Status MarkCommit(const Slice& xid) { + return Status::InvalidArgument("MarkCommit() handler not defined."); + } + // Continue is called by WriteBatch::Iterate. If it returns false, // iteration is halted. Otherwise, it continues iterating. The default // implementation always returns true. @@ -146,11 +275,69 @@ class WriteBatch { // Returns the number of updates in the batch int Count() const; + // Returns true if PutCF will be called during Iterate + bool HasPut() const; + + // Returns true if DeleteCF will be called during Iterate + bool HasDelete() const; + + // Returns true if SingleDeleteCF will be called during Iterate + bool HasSingleDelete() const; + + // Returns true if DeleteRangeCF will be called during Iterate + bool HasDeleteRange() const; + + // Returns true if MergeCF will be called during Iterate + bool HasMerge() const; + + // Returns true if MarkBeginPrepare will be called during Iterate + bool HasBeginPrepare() const; + + // Returns true if MarkEndPrepare will be called during Iterate + bool HasEndPrepare() const; + + // Returns trie if MarkCommit will be called during Iterate + bool HasCommit() const; + + // Returns trie if MarkRollback will be called during Iterate + bool HasRollback() const; + + using WriteBatchBase::GetWriteBatch; + WriteBatch* GetWriteBatch() override { return this; } + // Constructor with a serialized string object - explicit WriteBatch(std::string rep): rep_(rep) {} + explicit WriteBatch(const std::string& rep); + + WriteBatch(const WriteBatch& src); + WriteBatch(WriteBatch&& src); + WriteBatch& operator=(const WriteBatch& src); + WriteBatch& operator=(WriteBatch&& src); + + // marks this point in the WriteBatch as the last record to + // be inserted into the WAL, provided the WAL is enabled + void MarkWalTerminationPoint(); + const SavePoint& GetWalTerminationPoint() const { return wal_term_point_; } + + void SetMaxBytes(size_t max_bytes) override { max_bytes_ = max_bytes; } private: friend class WriteBatchInternal; + friend class LocalSavePoint; + SavePoints* save_points_; + + // When sending a WriteBatch through WriteImpl we might want to + // specify that only the first x records of the batch be written to + // the WAL. + SavePoint wal_term_point_; + + // For HasXYZ. Mutable to allow lazy computation of results + mutable std::atomic content_flags_; + + // Performs deferred computation of content_flags if necessary + uint32_t ComputeContentFlags() const; + + // Maximum size of rep_. + size_t max_bytes_; protected: std::string rep_; // See comment in write_batch.cc for the format of rep_ diff --git a/include/rocksdb/write_batch_base.h b/include/rocksdb/write_batch_base.h new file mode 100644 index 00000000000..3e6d011bd59 --- /dev/null +++ b/include/rocksdb/write_batch_base.h @@ -0,0 +1,125 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include + +namespace rocksdb { + +class Slice; +class Status; +class ColumnFamilyHandle; +class WriteBatch; +struct SliceParts; + +// Abstract base class that defines the basic interface for a write batch. +// See WriteBatch for a basic implementation and WrithBatchWithIndex for an +// indexed implemenation. +class WriteBatchBase { + public: + virtual ~WriteBatchBase() {} + + // Store the mapping "key->value" in the database. + virtual Status Put(ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) = 0; + virtual Status Put(const Slice& key, const Slice& value) = 0; + + // Variant of Put() that gathers output like writev(2). The key and value + // that will be written to the database are concatenations of arrays of + // slices. + virtual Status Put(ColumnFamilyHandle* column_family, const SliceParts& key, + const SliceParts& value); + virtual Status Put(const SliceParts& key, const SliceParts& value); + + // Merge "value" with the existing value of "key" in the database. + // "key->merge(existing, value)" + virtual Status Merge(ColumnFamilyHandle* column_family, const Slice& key, + const Slice& value) = 0; + virtual Status Merge(const Slice& key, const Slice& value) = 0; + + // variant that takes SliceParts + virtual Status Merge(ColumnFamilyHandle* column_family, const SliceParts& key, + const SliceParts& value); + virtual Status Merge(const SliceParts& key, const SliceParts& value); + + // If the database contains a mapping for "key", erase it. Else do nothing. + virtual Status Delete(ColumnFamilyHandle* column_family, + const Slice& key) = 0; + virtual Status Delete(const Slice& key) = 0; + + // variant that takes SliceParts + virtual Status Delete(ColumnFamilyHandle* column_family, + const SliceParts& key); + virtual Status Delete(const SliceParts& key); + + // If the database contains a mapping for "key", erase it. Expects that the + // key was not overwritten. Else do nothing. + virtual Status SingleDelete(ColumnFamilyHandle* column_family, + const Slice& key) = 0; + virtual Status SingleDelete(const Slice& key) = 0; + + // variant that takes SliceParts + virtual Status SingleDelete(ColumnFamilyHandle* column_family, + const SliceParts& key); + virtual Status SingleDelete(const SliceParts& key); + + // If the database contains mappings in the range ["begin_key", "end_key"], + // erase them. Else do nothing. + virtual Status DeleteRange(ColumnFamilyHandle* column_family, + const Slice& begin_key, const Slice& end_key) = 0; + virtual Status DeleteRange(const Slice& begin_key, const Slice& end_key) = 0; + + // variant that takes SliceParts + virtual Status DeleteRange(ColumnFamilyHandle* column_family, + const SliceParts& begin_key, + const SliceParts& end_key); + virtual Status DeleteRange(const SliceParts& begin_key, + const SliceParts& end_key); + + // Append a blob of arbitrary size to the records in this batch. The blob will + // be stored in the transaction log but not in any other file. In particular, + // it will not be persisted to the SST files. When iterating over this + // WriteBatch, WriteBatch::Handler::LogData will be called with the contents + // of the blob as it is encountered. Blobs, puts, deletes, and merges will be + // encountered in the same order in which they were inserted. The blob will + // NOT consume sequence number(s) and will NOT increase the count of the batch + // + // Example application: add timestamps to the transaction log for use in + // replication. + virtual Status PutLogData(const Slice& blob) = 0; + + // Clear all updates buffered in this batch. + virtual void Clear() = 0; + + // Covert this batch into a WriteBatch. This is an abstracted way of + // converting any WriteBatchBase(eg WriteBatchWithIndex) into a basic + // WriteBatch. + virtual WriteBatch* GetWriteBatch() = 0; + + // Records the state of the batch for future calls to RollbackToSavePoint(). + // May be called multiple times to set multiple save points. + virtual void SetSavePoint() = 0; + + // Remove all entries in this batch (Put, Merge, Delete, PutLogData) since the + // most recent call to SetSavePoint() and removes the most recent save point. + // If there is no previous call to SetSavePoint(), behaves the same as + // Clear(). + virtual Status RollbackToSavePoint() = 0; + + // Pop the most recent save point. + // If there is no previous call to SetSavePoint(), Status::NotFound() + // will be returned. + // Otherwise returns Status::OK(). + virtual Status PopSavePoint() = 0; + + // Sets the maximum size of the write batch in bytes. 0 means no limit. + virtual void SetMaxBytes(size_t max_bytes) = 0; +}; + +} // namespace rocksdb diff --git a/include/rocksdb/write_buffer_manager.h b/include/rocksdb/write_buffer_manager.h new file mode 100644 index 00000000000..856cf4b2463 --- /dev/null +++ b/include/rocksdb/write_buffer_manager.h @@ -0,0 +1,100 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// WriteBufferManager is for managing memory allocation for one or more +// MemTables. + +#pragma once + +#include +#include +#include "rocksdb/cache.h" + +namespace rocksdb { + +class WriteBufferManager { + public: + // _buffer_size = 0 indicates no limit. Memory won't be capped. + // memory_usage() won't be valid and ShouldFlush() will always return true. + // if `cache` is provided, we'll put dummy entries in the cache and cost + // the memory allocated to the cache. It can be used even if _buffer_size = 0. + explicit WriteBufferManager(size_t _buffer_size, + std::shared_ptr cache = {}); + ~WriteBufferManager(); + + bool enabled() const { return buffer_size_ != 0; } + + // Only valid if enabled() + size_t memory_usage() const { + return memory_used_.load(std::memory_order_relaxed); + } + size_t mutable_memtable_memory_usage() const { + return memory_active_.load(std::memory_order_relaxed); + } + size_t buffer_size() const { return buffer_size_; } + + // Should only be called from write thread + bool ShouldFlush() const { + if (enabled()) { + if (mutable_memtable_memory_usage() > mutable_limit_) { + return true; + } + if (memory_usage() >= buffer_size_ && + mutable_memtable_memory_usage() >= buffer_size_ / 2) { + // If the memory exceeds the buffer size, we trigger more aggressive + // flush. But if already more than half memory is being flushed, + // triggering more flush may not help. We will hold it instead. + return true; + } + } + return false; + } + + void ReserveMem(size_t mem) { + if (cache_rep_ != nullptr) { + ReserveMemWithCache(mem); + } else if (enabled()) { + memory_used_.fetch_add(mem, std::memory_order_relaxed); + } + if (enabled()) { + memory_active_.fetch_add(mem, std::memory_order_relaxed); + } + } + // We are in the process of freeing `mem` bytes, so it is not considered + // when checking the soft limit. + void ScheduleFreeMem(size_t mem) { + if (enabled()) { + memory_active_.fetch_sub(mem, std::memory_order_relaxed); + } + } + void FreeMem(size_t mem) { + if (cache_rep_ != nullptr) { + FreeMemWithCache(mem); + } else if (enabled()) { + memory_used_.fetch_sub(mem, std::memory_order_relaxed); + } + } + + private: + const size_t buffer_size_; + const size_t mutable_limit_; + std::atomic memory_used_; + // Memory that hasn't been scheduled to free. + std::atomic memory_active_; + struct CacheRep; + std::unique_ptr cache_rep_; + + void ReserveMemWithCache(size_t mem); + void FreeMemWithCache(size_t mem); + + // No copying allowed + WriteBufferManager(const WriteBufferManager&) = delete; + WriteBufferManager& operator=(const WriteBufferManager&) = delete; +}; +} // namespace rocksdb diff --git a/include/utilities/backupable_db.h b/include/utilities/backupable_db.h deleted file mode 100644 index 43d5a5cece6..00000000000 --- a/include/utilities/backupable_db.h +++ /dev/null @@ -1,12 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#pragma once -#warning This file was moved to rocksdb/utilities/backupable_db.h -#include "rocksdb/utilities/backupable_db.h" diff --git a/include/utilities/db_ttl.h b/include/utilities/db_ttl.h deleted file mode 100644 index c3d5c2bcfe2..00000000000 --- a/include/utilities/db_ttl.h +++ /dev/null @@ -1,8 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -#pragma once -#warning This file was moved to rocksdb/utilities/db_ttl.h -#include "rocksdb/utilities/db_ttl.h" diff --git a/include/utilities/document_db.h b/include/utilities/document_db.h deleted file mode 100644 index 1d1330bcae8..00000000000 --- a/include/utilities/document_db.h +++ /dev/null @@ -1,8 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -#pragma once -#warning This file was moved to rocksdb/utilities/document_db.h -#include "rocksdb/utilities/document_db.h" diff --git a/include/utilities/geo_db.h b/include/utilities/geo_db.h deleted file mode 100644 index 48957d4075b..00000000000 --- a/include/utilities/geo_db.h +++ /dev/null @@ -1,8 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -#pragma once -#warning This file was moved to rocksdb/utilities/geo_db.h -#include "rocksdb/utilities/geo_db.h" diff --git a/include/utilities/json_document.h b/include/utilities/json_document.h deleted file mode 100644 index f3f93969d12..00000000000 --- a/include/utilities/json_document.h +++ /dev/null @@ -1,7 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -#pragma once -#warning This file was moved to rocksdb/utilities/json_document.h -#include "rocksdb/utilities/json_document.h" diff --git a/include/utilities/stackable_db.h b/include/utilities/stackable_db.h deleted file mode 100644 index 435818d2b69..00000000000 --- a/include/utilities/stackable_db.h +++ /dev/null @@ -1,7 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#pragma once -#warning This file was moved to rocksdb/utilities/stackable_db.h -#include "rocksdb/utilities/stackable_db.h" diff --git a/include/utilities/utility_db.h b/include/utilities/utility_db.h deleted file mode 100644 index 4a8bbaec314..00000000000 --- a/include/utilities/utility_db.h +++ /dev/null @@ -1,7 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#pragma once -#warning This file was moved to rocksdb/utilities/utility_db.h -#include "rocksdb/utilities/utility_db.h" diff --git a/java/CMakeLists.txt b/java/CMakeLists.txt new file mode 100644 index 00000000000..d67896c2cd8 --- /dev/null +++ b/java/CMakeLists.txt @@ -0,0 +1,207 @@ +cmake_minimum_required(VERSION 2.6) + +set(JNI_NATIVE_SOURCES + rocksjni/backupablejni.cc + rocksjni/backupenginejni.cc + rocksjni/checkpoint.cc + rocksjni/clock_cache.cc + rocksjni/columnfamilyhandle.cc + rocksjni/compaction_filter.cc + rocksjni/compaction_options_fifo.cc + rocksjni/compaction_options_universal.cc + rocksjni/comparator.cc + rocksjni/comparatorjnicallback.cc + rocksjni/compression_options.cc + rocksjni/env.cc + rocksjni/env_options.cc + rocksjni/filter.cc + rocksjni/ingest_external_file_options.cc + rocksjni/iterator.cc + rocksjni/loggerjnicallback.cc + rocksjni/lru_cache.cc + rocksjni/memtablejni.cc + rocksjni/merge_operator.cc + rocksjni/options.cc + rocksjni/ratelimiterjni.cc + rocksjni/remove_emptyvalue_compactionfilterjni.cc + rocksjni/cassandra_compactionfilterjni.cc + rocksjni/restorejni.cc + rocksjni/rocksdb_exception_test.cc + rocksjni/rocksjni.cc + rocksjni/slice.cc + rocksjni/snapshot.cc + rocksjni/sst_file_writerjni.cc + rocksjni/statistics.cc + rocksjni/statisticsjni.cc + rocksjni/table.cc + rocksjni/transaction_log.cc + rocksjni/ttl.cc + rocksjni/write_batch.cc + rocksjni/write_batch_test.cc + rocksjni/write_batch_with_index.cc + rocksjni/writebatchhandlerjnicallback.cc +) + +set(NATIVE_JAVA_CLASSES + org.rocksdb.AbstractCompactionFilter + org.rocksdb.AbstractComparator + org.rocksdb.AbstractImmutableNativeReference + org.rocksdb.AbstractNativeReference + org.rocksdb.AbstractRocksIterator + org.rocksdb.AbstractSlice + org.rocksdb.AbstractWriteBatch + org.rocksdb.BackupableDBOptions + org.rocksdb.BackupEngine + org.rocksdb.BackupEngineTest + org.rocksdb.BlockBasedTableConfig + org.rocksdb.BloomFilter + org.rocksdb.Cache + org.rocksdb.CassandraCompactionFilter + org.rocksdb.CassandraValueMergeOperator + org.rocksdb.Checkpoint + org.rocksdb.ClockCache + org.rocksdb.ColumnFamilyHandle + org.rocksdb.ColumnFamilyOptions + org.rocksdb.CompactionOptionsFIFO + org.rocksdb.CompactionOptionsUniversal + org.rocksdb.Comparator + org.rocksdb.ComparatorOptions + org.rocksdb.CompressionOptions + org.rocksdb.DBOptions + org.rocksdb.DirectComparator + org.rocksdb.DirectSlice + org.rocksdb.Env + org.rocksdb.EnvOptions + org.rocksdb.ExternalSstFileInfo + org.rocksdb.Filter + org.rocksdb.FlushOptions + org.rocksdb.HashLinkedListMemTableConfig + org.rocksdb.HashSkipListMemTableConfig + org.rocksdb.IngestExternalFileOptions + org.rocksdb.Logger + org.rocksdb.LRUCache + org.rocksdb.MemTableConfig + org.rocksdb.MergeOperator + org.rocksdb.NativeLibraryLoader + org.rocksdb.Options + org.rocksdb.PlainTableConfig + org.rocksdb.RateLimiter + org.rocksdb.ReadOptions + org.rocksdb.RemoveEmptyValueCompactionFilter + org.rocksdb.RestoreOptions + org.rocksdb.RocksDB + org.rocksdb.RocksDBExceptionTest + org.rocksdb.RocksEnv + org.rocksdb.RocksIterator + org.rocksdb.RocksIteratorInterface + org.rocksdb.RocksMemEnv + org.rocksdb.RocksMutableObject + org.rocksdb.RocksObject + org.rocksdb.SkipListMemTableConfig + org.rocksdb.Slice + org.rocksdb.Snapshot + org.rocksdb.SnapshotTest + org.rocksdb.SstFileWriter + org.rocksdb.Statistics + org.rocksdb.StringAppendOperator + org.rocksdb.TableFormatConfig + org.rocksdb.TransactionLogIterator + org.rocksdb.TtlDB + org.rocksdb.VectorMemTableConfig + org.rocksdb.WBWIRocksIterator + org.rocksdb.WriteBatch + org.rocksdb.WriteBatch.Handler + org.rocksdb.WriteBatchTest + org.rocksdb.WriteBatchTestInternalHelper + org.rocksdb.WriteBatchWithIndex + org.rocksdb.WriteOptions +) + +include_directories($ENV{JAVA_HOME}/include) +include_directories($ENV{JAVA_HOME}/include/win32) +include_directories(${PROJECT_SOURCE_DIR}/java) + +set(JAVA_TEST_LIBDIR ${PROJECT_SOURCE_DIR}/java/test-libs) +set(JAVA_TMP_JAR ${JAVA_TEST_LIBDIR}/tmp.jar) +set(JAVA_JUNIT_JAR ${JAVA_TEST_LIBDIR}/junit-4.12.jar) +set(JAVA_HAMCR_JAR ${JAVA_TEST_LIBDIR}/hamcrest-core-1.3.jar) +set(JAVA_MOCKITO_JAR ${JAVA_TEST_LIBDIR}/mockito-all-1.10.19.jar) +set(JAVA_CGLIB_JAR ${JAVA_TEST_LIBDIR}/cglib-2.2.2.jar) +set(JAVA_ASSERTJ_JAR ${JAVA_TEST_LIBDIR}/assertj-core-1.7.1.jar) +set(JAVA_TESTCLASSPATH "${JAVA_JUNIT_JAR}\;${JAVA_HAMCR_JAR}\;${JAVA_MOCKITO_JAR}\;${JAVA_CGLIB_JAR}\;${JAVA_ASSERTJ_JAR}") + +if(NOT EXISTS ${PROJECT_SOURCE_DIR}/java/classes) + file(MAKE_DIRECTORY ${PROJECT_SOURCE_DIR}/java/classes) +endif() + +if(NOT EXISTS ${JAVA_TEST_LIBDIR}) + file(MAKE_DIRECTORY mkdir ${JAVA_TEST_LIBDIR}) +endif() + +if (DEFINED CUSTOM_REPO_URL) + set(SEARCH_REPO_URL ${CUSTOM_REPO_URL}/) + set(CENTRAL_REPO_URL ${CUSTOM_REPO_URL}/) +else () + set(SEARCH_REPO_URL "http://search.maven.org/remotecontent?filepath=") + set(CENTRAL_REPO_URL "http://central.maven.org/maven2/") +endif() + +if(NOT EXISTS ${JAVA_JUNIT_JAR}) + message("Downloading ${JAVA_JUNIT_JAR}") + file(DOWNLOAD ${SEARCH_REPO_URL}junit/junit/4.12/junit-4.12.jar ${JAVA_TMP_JAR} STATUS downloadStatus) + list(GET downloadStatus 0 error_code) + if(NOT error_code EQUAL 0) + message(FATAL_ERROR "Failed downloading ${JAVA_JUNIT_JAR}") + endif() + file(RENAME ${JAVA_TMP_JAR} ${JAVA_JUNIT_JAR}) +endif() +if(NOT EXISTS ${JAVA_HAMCR_JAR}) + message("Downloading ${JAVA_HAMCR_JAR}") + file(DOWNLOAD ${SEARCH_REPO_URL}org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar ${JAVA_TMP_JAR} STATUS downloadStatus) + list(GET downloadStatus 0 error_code) + if(NOT error_code EQUAL 0) + message(FATAL_ERROR "Failed downloading ${JAVA_HAMCR_JAR}") + endif() + file(RENAME ${JAVA_TMP_JAR} ${JAVA_HAMCR_JAR}) +endif() +if(NOT EXISTS ${JAVA_MOCKITO_JAR}) + message("Downloading ${JAVA_MOCKITO_JAR}") + file(DOWNLOAD ${SEARCH_REPO_URL}org/mockito/mockito-all/1.10.19/mockito-all-1.10.19.jar ${JAVA_TMP_JAR} STATUS downloadStatus) + list(GET downloadStatus 0 error_code) + if(NOT error_code EQUAL 0) + message(FATAL_ERROR "Failed downloading ${JAVA_MOCKITO_JAR}") + endif() + file(RENAME ${JAVA_TMP_JAR} ${JAVA_MOCKITO_JAR}) +endif() +if(NOT EXISTS ${JAVA_CGLIB_JAR}) + message("Downloading ${JAVA_CGLIB_JAR}") + file(DOWNLOAD ${SEARCH_REPO_URL}cglib/cglib/2.2.2/cglib-2.2.2.jar ${JAVA_TMP_JAR} STATUS downloadStatus) + list(GET downloadStatus 0 error_code) + if(NOT error_code EQUAL 0) + message(FATAL_ERROR "Failed downloading ${JAVA_CGLIB_JAR}") + endif() + file(RENAME ${JAVA_TMP_JAR} ${JAVA_CGLIB_JAR}) +endif() +if(NOT EXISTS ${JAVA_ASSERTJ_JAR}) + message("Downloading ${JAVA_ASSERTJ_JAR}") + file(DOWNLOAD ${CENTRAL_REPO_URL}org/assertj/assertj-core/1.7.1/assertj-core-1.7.1.jar ${JAVA_TMP_JAR} STATUS downloadStatus) + list(GET downloadStatus 0 error_code) + if(NOT error_code EQUAL 0) + message(FATAL_ERROR "Failed downloading ${JAVA_ASSERTJ_JAR}") + endif() + file(RENAME ${JAVA_TMP_JAR} ${JAVA_ASSERTJ_JAR}) +endif() + +if(WIN32) + set(JAVAC cmd /c javac) + set(JAVAH cmd /c javah) +else() + set(JAVAC javac) + set(JAVAH javah) +endif() + +execute_process(COMMAND ${JAVAC} ${JAVAC_ARGS} -cp ${JAVA_TESTCLASSPATH} -d ${PROJECT_SOURCE_DIR}/java/classes ${PROJECT_SOURCE_DIR}/java/src/main/java/org/rocksdb/util/*.java ${PROJECT_SOURCE_DIR}/java/src/main/java/org/rocksdb/*.java ${PROJECT_SOURCE_DIR}/java/src/test/java/org/rocksdb/*.java) +execute_process(COMMAND ${JAVAH} -cp ${PROJECT_SOURCE_DIR}/java/classes -d ${PROJECT_SOURCE_DIR}/java/include -jni ${NATIVE_JAVA_CLASSES}) +add_library(rocksdbjni${ARTIFACT_SUFFIX} SHARED ${JNI_NATIVE_SOURCES}) +set_target_properties(rocksdbjni${ARTIFACT_SUFFIX} PROPERTIES COMPILE_FLAGS "/Fd${CMAKE_CFG_INTDIR}/rocksdbjni${ARTIFACT_SUFFIX}.pdb") +target_link_libraries(rocksdbjni${ARTIFACT_SUFFIX} rocksdb${ARTIFACT_SUFFIX} ${LIBS}) diff --git a/java/HISTORY-JAVA.md b/java/HISTORY-JAVA.md index 4cf0f7d184f..731886a610e 100644 --- a/java/HISTORY-JAVA.md +++ b/java/HISTORY-JAVA.md @@ -1,5 +1,37 @@ # RocksJava Change Log +## 3.13 (8/4/2015) +### New Features +* Exposed BackupEngine API. +* Added CappedPrefixExtractor support. To use such extractor, simply call useCappedPrefixExtractor in either Options or ColumnFamilyOptions. +* Added RemoveEmptyValueCompactionFilter. + +## 3.10.0 (3/24/2015) +### New Features +* Added compression per level API. +* MemEnv is now available in RocksJava via RocksMemEnv class. +* lz4 compression is now included in rocksjava static library when running `make rocksdbjavastatic`. + +### Public API Changes +* Overflowing a size_t when setting rocksdb options now throws an IllegalArgumentException, which removes the necessity for a developer to catch these Exceptions explicitly. +* The set and get functions for tableCacheRemoveScanCountLimit are deprecated. + + +## By 01/31/2015 +### New Features +* WriteBatchWithIndex support. +* Iterator support for WriteBatch and WriteBatchWithIndex +* GetUpdatesSince support. +* Snapshots carry now information about the related sequence number. +* TTL DB support. + +## By 11/14/2014 +### New Features +* Full support for Column Family. +* Slice and Comparator support. +* Default merge operator support. +* RateLimiter support. + ## By 06/15/2014 ### New Features * Added basic Java binding for rocksdb::Env such that multiple RocksDB can share the same thread pool and environment. diff --git a/java/Makefile b/java/Makefile index 47b2afb9e2a..b29447bd8a7 100644 --- a/java/Makefile +++ b/java/Makefile @@ -1,34 +1,231 @@ -NATIVE_JAVA_CLASSES = org.rocksdb.RocksDB org.rocksdb.Options org.rocksdb.WriteBatch org.rocksdb.WriteBatchInternal org.rocksdb.WriteBatchTest org.rocksdb.WriteOptions org.rocksdb.BackupableDB org.rocksdb.BackupableDBOptions org.rocksdb.Statistics org.rocksdb.RocksIterator org.rocksdb.VectorMemTableConfig org.rocksdb.SkipListMemTableConfig org.rocksdb.HashLinkedListMemTableConfig org.rocksdb.HashSkipListMemTableConfig org.rocksdb.PlainTableConfig org.rocksdb.BlockBasedTableConfig org.rocksdb.ReadOptions org.rocksdb.Filter org.rocksdb.BloomFilter org.rocksdb.RestoreOptions org.rocksdb.RestoreBackupableDB org.rocksdb.RocksEnv +NATIVE_JAVA_CLASSES = org.rocksdb.AbstractCompactionFilter\ + org.rocksdb.AbstractComparator\ + org.rocksdb.AbstractSlice\ + org.rocksdb.BackupEngine\ + org.rocksdb.BackupableDBOptions\ + org.rocksdb.BlockBasedTableConfig\ + org.rocksdb.BloomFilter\ + org.rocksdb.Checkpoint\ + org.rocksdb.ClockCache\ + org.rocksdb.CassandraCompactionFilter\ + org.rocksdb.CassandraValueMergeOperator\ + org.rocksdb.ColumnFamilyHandle\ + org.rocksdb.ColumnFamilyOptions\ + org.rocksdb.CompactionOptionsFIFO\ + org.rocksdb.CompactionOptionsUniversal\ + org.rocksdb.Comparator\ + org.rocksdb.ComparatorOptions\ + org.rocksdb.CompressionOptions\ + org.rocksdb.DBOptions\ + org.rocksdb.DirectComparator\ + org.rocksdb.DirectSlice\ + org.rocksdb.Env\ + org.rocksdb.EnvOptions\ + org.rocksdb.FlushOptions\ + org.rocksdb.Filter\ + org.rocksdb.IngestExternalFileOptions\ + org.rocksdb.HashLinkedListMemTableConfig\ + org.rocksdb.HashSkipListMemTableConfig\ + org.rocksdb.Logger\ + org.rocksdb.LRUCache\ + org.rocksdb.MergeOperator\ + org.rocksdb.Options\ + org.rocksdb.PlainTableConfig\ + org.rocksdb.RateLimiter\ + org.rocksdb.ReadOptions\ + org.rocksdb.RemoveEmptyValueCompactionFilter\ + org.rocksdb.RestoreOptions\ + org.rocksdb.RocksDB\ + org.rocksdb.RocksEnv\ + org.rocksdb.RocksIterator\ + org.rocksdb.RocksMemEnv\ + org.rocksdb.SkipListMemTableConfig\ + org.rocksdb.Slice\ + org.rocksdb.SstFileWriter\ + org.rocksdb.Statistics\ + org.rocksdb.TransactionLogIterator\ + org.rocksdb.TtlDB\ + org.rocksdb.VectorMemTableConfig\ + org.rocksdb.Snapshot\ + org.rocksdb.StringAppendOperator\ + org.rocksdb.WriteBatch\ + org.rocksdb.WriteBatch.Handler\ + org.rocksdb.WriteOptions\ + org.rocksdb.WriteBatchWithIndex\ + org.rocksdb.WBWIRocksIterator + +NATIVE_JAVA_TEST_CLASSES = org.rocksdb.RocksDBExceptionTest\ + org.rocksdb.WriteBatchTest\ + org.rocksdb.WriteBatchTestInternalHelper + +ROCKSDB_MAJOR = $(shell egrep "ROCKSDB_MAJOR.[0-9]" ../include/rocksdb/version.h | cut -d ' ' -f 3) +ROCKSDB_MINOR = $(shell egrep "ROCKSDB_MINOR.[0-9]" ../include/rocksdb/version.h | cut -d ' ' -f 3) +ROCKSDB_PATCH = $(shell egrep "ROCKSDB_PATCH.[0-9]" ../include/rocksdb/version.h | cut -d ' ' -f 3) NATIVE_INCLUDE = ./include -ROCKSDB_JAR = rocksdbjni.jar +ARCH := $(shell getconf LONG_BIT) +ROCKSDB_JAR = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-linux$(ARCH).jar +ifeq ($(PLATFORM), OS_MACOSX) +ROCKSDB_JAR = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-osx.jar +endif + +JAVA_TESTS = org.rocksdb.BackupableDBOptionsTest\ + org.rocksdb.BackupEngineTest\ + org.rocksdb.BlockBasedTableConfigTest\ + org.rocksdb.util.BytewiseComparatorTest\ + org.rocksdb.CheckPointTest\ + org.rocksdb.ClockCacheTest\ + org.rocksdb.ColumnFamilyOptionsTest\ + org.rocksdb.ColumnFamilyTest\ + org.rocksdb.CompactionOptionsFIFOTest\ + org.rocksdb.CompactionOptionsUniversalTest\ + org.rocksdb.CompactionPriorityTest\ + org.rocksdb.CompactionStopStyleTest\ + org.rocksdb.ComparatorOptionsTest\ + org.rocksdb.ComparatorTest\ + org.rocksdb.CompressionOptionsTest\ + org.rocksdb.CompressionTypesTest\ + org.rocksdb.DBOptionsTest\ + org.rocksdb.DirectComparatorTest\ + org.rocksdb.DirectSliceTest\ + org.rocksdb.EnvOptionsTest\ + org.rocksdb.IngestExternalFileOptionsTest\ + org.rocksdb.util.EnvironmentTest\ + org.rocksdb.FilterTest\ + org.rocksdb.FlushTest\ + org.rocksdb.InfoLogLevelTest\ + org.rocksdb.KeyMayExistTest\ + org.rocksdb.LoggerTest\ + org.rocksdb.LRUCacheTest\ + org.rocksdb.MemTableTest\ + org.rocksdb.MergeTest\ + org.rocksdb.MixedOptionsTest\ + org.rocksdb.MutableColumnFamilyOptionsTest\ + org.rocksdb.NativeLibraryLoaderTest\ + org.rocksdb.OptionsTest\ + org.rocksdb.PlainTableConfigTest\ + org.rocksdb.RateLimiterTest\ + org.rocksdb.ReadOnlyTest\ + org.rocksdb.ReadOptionsTest\ + org.rocksdb.RocksDBTest\ + org.rocksdb.RocksDBExceptionTest\ + org.rocksdb.RocksEnvTest\ + org.rocksdb.RocksIteratorTest\ + org.rocksdb.RocksMemEnvTest\ + org.rocksdb.util.SizeUnitTest\ + org.rocksdb.SliceTest\ + org.rocksdb.SnapshotTest\ + org.rocksdb.SstFileWriterTest\ + org.rocksdb.TransactionLogIteratorTest\ + org.rocksdb.TtlDBTest\ + org.rocksdb.StatisticsTest\ + org.rocksdb.StatisticsCollectorTest\ + org.rocksdb.WALRecoveryModeTest\ + org.rocksdb.WriteBatchHandlerTest\ + org.rocksdb.WriteBatchTest\ + org.rocksdb.WriteBatchThreadedTest\ + org.rocksdb.WriteOptionsTest\ + org.rocksdb.WriteBatchWithIndexTest + +MAIN_SRC = src/main/java +TEST_SRC = src/test/java +OUTPUT = target +MAIN_CLASSES = $(OUTPUT)/classes +TEST_CLASSES = $(OUTPUT)/test-classes +JAVADOC = $(OUTPUT)/apidocs + +BENCHMARK_MAIN_SRC = benchmark/src/main/java +BENCHMARK_OUTPUT = benchmark/target +BENCHMARK_MAIN_CLASSES = $(BENCHMARK_OUTPUT)/classes + +SAMPLES_MAIN_SRC = samples/src/main/java +SAMPLES_OUTPUT = samples/target +SAMPLES_MAIN_CLASSES = $(SAMPLES_OUTPUT)/classes + +JAVA_TEST_LIBDIR = test-libs +JAVA_JUNIT_JAR = $(JAVA_TEST_LIBDIR)/junit-4.12.jar +JAVA_HAMCR_JAR = $(JAVA_TEST_LIBDIR)/hamcrest-core-1.3.jar +JAVA_MOCKITO_JAR = $(JAVA_TEST_LIBDIR)/mockito-all-1.10.19.jar +JAVA_CGLIB_JAR = $(JAVA_TEST_LIBDIR)/cglib-2.2.2.jar +JAVA_ASSERTJ_JAR = $(JAVA_TEST_LIBDIR)/assertj-core-1.7.1.jar +JAVA_TESTCLASSPATH = $(JAVA_JUNIT_JAR):$(JAVA_HAMCR_JAR):$(JAVA_MOCKITO_JAR):$(JAVA_CGLIB_JAR):$(JAVA_ASSERTJ_JAR) + +MVN_LOCAL = ~/.m2/repository + +# Set the default JAVA_ARGS to "" for DEBUG_LEVEL=0 +JAVA_ARGS? = + +JAVAC_ARGS? = + +# When debugging add -Xcheck:jni to the java args +ifneq ($(DEBUG_LEVEL),0) + JAVA_ARGS = -ea -Xcheck:jni + JAVAC_ARGS = -Xlint:deprecation -Xlint:unchecked +endif + +SEARCH_REPO_URL?=http://search.maven.org/remotecontent?filepath= +CENTRAL_REPO_URL?=http://central.maven.org/maven2/ clean: - -find . -name "*.class" -exec rm {} \; - -find . -name "hs*.log" -exec rm {} \; - rm -f $(ROCKSDB_JAR) + $(AM_V_at)rm -rf include/* + $(AM_V_at)rm -rf test-libs/ + $(AM_V_at)rm -rf $(OUTPUT) + $(AM_V_at)rm -rf $(BENCHMARK_OUTPUT) + $(AM_V_at)rm -rf $(SAMPLES_OUTPUT) + + +javadocs: java + $(AM_V_GEN)mkdir -p $(JAVADOC) + $(AM_V_at)javadoc -d $(JAVADOC) -sourcepath $(MAIN_SRC) -subpackages org + +javalib: java java_test javadocs java: - javac org/rocksdb/util/*.java org/rocksdb/*.java - @cp ../HISTORY.md ./HISTORY-CPP.md - @rm -f ./HISTORY-CPP.md - javah -d $(NATIVE_INCLUDE) -jni $(NATIVE_JAVA_CLASSES) + $(AM_V_GEN)mkdir -p $(MAIN_CLASSES) + $(AM_V_at)javac $(JAVAC_ARGS) -d $(MAIN_CLASSES)\ + $(MAIN_SRC)/org/rocksdb/util/*.java\ + $(MAIN_SRC)/org/rocksdb/*.java + $(AM_V_at)@cp ../HISTORY.md ./HISTORY-CPP.md + $(AM_V_at)@rm -f ./HISTORY-CPP.md + $(AM_V_at)javah -cp $(MAIN_CLASSES) -d $(NATIVE_INCLUDE) -jni $(NATIVE_JAVA_CLASSES) sample: java - javac -cp $(ROCKSDB_JAR) RocksDBSample.java - @rm -rf /tmp/rocksdbjni - @rm -rf /tmp/rocksdbjni_not_found - java -ea -Djava.library.path=.:../ -cp ".:./*" -Xcheck:jni RocksDBSample /tmp/rocksdbjni - @rm -rf /tmp/rocksdbjni - @rm -rf /tmp/rocksdbjni_not_found - -test: java - javac org/rocksdb/test/*.java - java -ea -Djava.library.path=.:../ -cp "$(ROCKSDB_JAR):.:./*" org.rocksdb.WriteBatchTest - java -ea -Djava.library.path=.:../ -cp "$(ROCKSDB_JAR):.:./*" org.rocksdb.test.BackupableDBTest - java -ea -Djava.library.path=.:../ -cp "$(ROCKSDB_JAR):.:./*" org.rocksdb.test.OptionsTest - java -ea -Djava.library.path=.:../ -cp "$(ROCKSDB_JAR):.:./*" org.rocksdb.test.ReadOptionsTest - java -ea -Djava.library.path=.:../ -cp "$(ROCKSDB_JAR):.:./*" org.rocksdb.test.StatisticsCollectorTest + $(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES) + $(AM_V_at)javac $(JAVAC_ARGS) -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/RocksDBSample.java + $(AM_V_at)@rm -rf /tmp/rocksdbjni + $(AM_V_at)@rm -rf /tmp/rocksdbjni_not_found + java $(JAVA_ARGS) -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) RocksDBSample /tmp/rocksdbjni + $(AM_V_at)@rm -rf /tmp/rocksdbjni + $(AM_V_at)@rm -rf /tmp/rocksdbjni_not_found + +column_family_sample: java + $(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES) + $(AM_V_at)javac $(JAVAC_ARGS) -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/RocksDBColumnFamilySample.java + $(AM_V_at)@rm -rf /tmp/rocksdbjni + java $(JAVA_ARGS) -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) RocksDBColumnFamilySample /tmp/rocksdbjni + $(AM_V_at)@rm -rf /tmp/rocksdbjni + +resolve_test_deps: + test -d "$(JAVA_TEST_LIBDIR)" || mkdir -p "$(JAVA_TEST_LIBDIR)" + test -s "$(JAVA_JUNIT_JAR)" || cp $(MVN_LOCAL)/junit/junit/4.12/junit-4.12.jar $(JAVA_TEST_LIBDIR) || curl -k -L -o $(JAVA_JUNIT_JAR) $(SEARCH_REPO_URL)junit/junit/4.12/junit-4.12.jar + test -s "$(JAVA_HAMCR_JAR)" || cp $(MVN_LOCAL)/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar $(JAVA_TEST_LIBDIR) || curl -k -L -o $(JAVA_HAMCR_JAR) $(SEARCH_REPO_URL)org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar + test -s "$(JAVA_MOCKITO_JAR)" || cp $(MVN_LOCAL)/org/mockito/mockito-all/1.10.19/mockito-all-1.10.19.jar $(JAVA_TEST_LIBDIR) || curl -k -L -o "$(JAVA_MOCKITO_JAR)" $(SEARCH_REPO_URL)org/mockito/mockito-all/1.10.19/mockito-all-1.10.19.jar + test -s "$(JAVA_CGLIB_JAR)" || cp $(MVN_LOCAL)/cglib/cglib/2.2.2/cglib-2.2.2.jar $(JAVA_TEST_LIBDIR) || curl -k -L -o "$(JAVA_CGLIB_JAR)" $(SEARCH_REPO_URL)cglib/cglib/2.2.2/cglib-2.2.2.jar + test -s "$(JAVA_ASSERTJ_JAR)" || cp $(MVN_LOCAL)/org/assertj/assertj-core/1.7.1/assertj-core-1.7.1.jar $(JAVA_TEST_LIBDIR) || curl -k -L -o "$(JAVA_ASSERTJ_JAR)" $(CENTRAL_REPO_URL)org/assertj/assertj-core/1.7.1/assertj-core-1.7.1.jar + +java_test: java resolve_test_deps + $(AM_V_GEN)mkdir -p $(TEST_CLASSES) + $(AM_V_at)javac $(JAVAC_ARGS) -cp $(MAIN_CLASSES):$(JAVA_TESTCLASSPATH) -d $(TEST_CLASSES)\ + $(TEST_SRC)/org/rocksdb/test/*.java\ + $(TEST_SRC)/org/rocksdb/util/*.java\ + $(TEST_SRC)/org/rocksdb/*.java + $(AM_V_at)javah -cp $(MAIN_CLASSES):$(TEST_CLASSES) -d $(NATIVE_INCLUDE) -jni $(NATIVE_JAVA_TEST_CLASSES) + +test: java java_test run_test + +run_test: + java $(JAVA_ARGS) -Djava.library.path=target -cp "$(MAIN_CLASSES):$(TEST_CLASSES):$(JAVA_TESTCLASSPATH):target/*" org.rocksdb.test.RocksJunitRunner $(JAVA_TESTS) db_bench: java - javac org/rocksdb/benchmark/*.java + $(AM_V_GEN)mkdir -p $(BENCHMARK_MAIN_CLASSES) + $(AM_V_at)javac $(JAVAC_ARGS) -cp $(MAIN_CLASSES) -d $(BENCHMARK_MAIN_CLASSES) $(BENCHMARK_MAIN_SRC)/org/rocksdb/benchmark/*.java diff --git a/java/RELEASE.md b/java/RELEASE.md new file mode 100644 index 00000000000..cb9aaf987b4 --- /dev/null +++ b/java/RELEASE.md @@ -0,0 +1,54 @@ +## Cross-building + +RocksDB can be built as a single self contained cross-platform JAR. The cross-platform jar can be usd on any 64-bit OSX system, 32-bit Linux system, or 64-bit Linux system. + +Building a cross-platform JAR requires: + + * [Vagrant](https://www.vagrantup.com/) + * [Virtualbox](https://www.virtualbox.org/) + * A Mac OSX machine that can compile RocksDB. + * Java 7 set as JAVA_HOME. + +Once you have these items, run this make command from RocksDB's root source directory: + + make jclean clean rocksdbjavastaticrelease + +This command will build RocksDB natively on OSX, and will then spin up two Vagrant Virtualbox Ubuntu images to build RocksDB for both 32-bit and 64-bit Linux. + +You can find all native binaries and JARs in the java/target directory upon completion: + + librocksdbjni-linux32.so + librocksdbjni-linux64.so + librocksdbjni-osx.jnilib + rocksdbjni-3.5.0-javadoc.jar + rocksdbjni-3.5.0-linux32.jar + rocksdbjni-3.5.0-linux64.jar + rocksdbjni-3.5.0-osx.jar + rocksdbjni-3.5.0-sources.jar + rocksdbjni-3.5.0.jar + +## Maven publication + +Set ~/.m2/settings.xml to contain: + + + + + sonatype-nexus-staging + your-sonatype-jira-username + your-sonatype-jira-password + + + + +From RocksDB's root directory, first build the Java static JARs: + + make jclean clean rocksdbjavastaticpublish + +This command will [stage the JAR artifacts on the Sonatype staging repository](http://central.sonatype.org/pages/manual-staging-bundle-creation-and-deployment.html). To release the staged artifacts. + +1. Go to [https://oss.sonatype.org/#stagingRepositories](https://oss.sonatype.org/#stagingRepositories) and search for "rocksdb" in the upper right hand search box. +2. Select the rocksdb staging repository, and inspect its contents. +3. If all is well, follow [these steps](https://oss.sonatype.org/#stagingRepositories) to close the repository and release it. + +After the release has occurred, the artifacts will be synced to Maven central within 24-48 hours. diff --git a/java/RocksDBSample.java b/java/RocksDBSample.java deleted file mode 100644 index 72da4b5e862..00000000000 --- a/java/RocksDBSample.java +++ /dev/null @@ -1,259 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -import java.util.Arrays; -import java.util.List; -import java.util.Map; -import java.util.ArrayList; -import org.rocksdb.*; -import org.rocksdb.util.SizeUnit; -import java.io.IOException; - -public class RocksDBSample { - static { - RocksDB.loadLibrary(); - } - - public static void main(String[] args) { - if (args.length < 1) { - System.out.println("usage: RocksDBSample db_path"); - return; - } - String db_path = args[0]; - String db_path_not_found = db_path + "_not_found"; - - System.out.println("RocksDBSample"); - RocksDB db = null; - Options options = new Options(); - try { - db = RocksDB.open(options, db_path_not_found); - assert(false); - } catch (RocksDBException e) { - System.out.format("caught the expceted exception -- %s\n", e); - assert(db == null); - } - - options.setCreateIfMissing(true) - .createStatistics() - .setWriteBufferSize(8 * SizeUnit.KB) - .setMaxWriteBufferNumber(3) - .setMaxBackgroundCompactions(10) - .setCompressionType(CompressionType.SNAPPY_COMPRESSION) - .setCompactionStyle(CompactionStyle.UNIVERSAL); - Statistics stats = options.statisticsPtr(); - - assert(options.createIfMissing() == true); - assert(options.writeBufferSize() == 8 * SizeUnit.KB); - assert(options.maxWriteBufferNumber() == 3); - assert(options.maxBackgroundCompactions() == 10); - assert(options.compressionType() == CompressionType.SNAPPY_COMPRESSION); - assert(options.compactionStyle() == CompactionStyle.UNIVERSAL); - - assert(options.memTableFactoryName().equals("SkipListFactory")); - options.setMemTableConfig( - new HashSkipListMemTableConfig() - .setHeight(4) - .setBranchingFactor(4) - .setBucketCount(2000000)); - assert(options.memTableFactoryName().equals("HashSkipListRepFactory")); - - options.setMemTableConfig( - new HashLinkedListMemTableConfig() - .setBucketCount(100000)); - assert(options.memTableFactoryName().equals("HashLinkedListRepFactory")); - - options.setMemTableConfig( - new VectorMemTableConfig().setReservedSize(10000)); - assert(options.memTableFactoryName().equals("VectorRepFactory")); - - options.setMemTableConfig(new SkipListMemTableConfig()); - assert(options.memTableFactoryName().equals("SkipListFactory")); - - options.setTableFormatConfig(new PlainTableConfig()); - assert(options.tableFactoryName().equals("PlainTable")); - - BlockBasedTableConfig table_options = new BlockBasedTableConfig(); - table_options.setBlockCacheSize(64 * SizeUnit.KB) - .setFilterBitsPerKey(10) - .setCacheNumShardBits(6); - assert(table_options.blockCacheSize() == 64 * SizeUnit.KB); - assert(table_options.cacheNumShardBits() == 6); - options.setTableFormatConfig(table_options); - assert(options.tableFactoryName().equals("BlockBasedTable")); - - try { - db = RocksDB.open(options, db_path_not_found); - db.put("hello".getBytes(), "world".getBytes()); - byte[] value = db.get("hello".getBytes()); - assert("world".equals(new String(value))); - } catch (RocksDBException e) { - System.out.format("[ERROR] caught the unexpceted exception -- %s\n", e); - assert(db == null); - assert(false); - } - // be sure to release the c++ pointer - db.close(); - - ReadOptions readOptions = new ReadOptions(); - readOptions.setFillCache(false); - - try { - db = RocksDB.open(options, db_path); - db.put("hello".getBytes(), "world".getBytes()); - byte[] value = db.get("hello".getBytes()); - System.out.format("Get('hello') = %s\n", - new String(value)); - - for (int i = 1; i <= 9; ++i) { - for (int j = 1; j <= 9; ++j) { - db.put(String.format("%dx%d", i, j).getBytes(), - String.format("%d", i * j).getBytes()); - } - } - - for (int i = 1; i <= 9; ++i) { - for (int j = 1; j <= 9; ++j) { - System.out.format("%s ", new String(db.get( - String.format("%dx%d", i, j).getBytes()))); - } - System.out.println(""); - } - - value = db.get("1x1".getBytes()); - assert(value != null); - value = db.get("world".getBytes()); - assert(value == null); - value = db.get(readOptions, "world".getBytes()); - assert(value == null); - - byte[] testKey = "asdf".getBytes(); - byte[] testValue = - "asdfghjkl;'?> insufficientArray.length); - len = db.get("asdfjkl;".getBytes(), enoughArray); - assert(len == RocksDB.NOT_FOUND); - len = db.get(testKey, enoughArray); - assert(len == testValue.length); - - len = db.get(readOptions, testKey, insufficientArray); - assert(len > insufficientArray.length); - len = db.get(readOptions, "asdfjkl;".getBytes(), enoughArray); - assert(len == RocksDB.NOT_FOUND); - len = db.get(readOptions, testKey, enoughArray); - assert(len == testValue.length); - - db.remove(testKey); - len = db.get(testKey, enoughArray); - assert(len == RocksDB.NOT_FOUND); - - // repeat the test with WriteOptions - WriteOptions writeOpts = new WriteOptions(); - writeOpts.setSync(true); - writeOpts.setDisableWAL(true); - db.put(writeOpts, testKey, testValue); - len = db.get(testKey, enoughArray); - assert(len == testValue.length); - assert(new String(testValue).equals( - new String(enoughArray, 0, len))); - writeOpts.dispose(); - - try { - for (TickerType statsType : TickerType.values()) { - stats.getTickerCount(statsType); - } - System.out.println("getTickerCount() passed."); - } catch (Exception e) { - System.out.println("Failed in call to getTickerCount()"); - assert(false); //Should never reach here. - } - - try { - for (HistogramType histogramType : HistogramType.values()) { - HistogramData data = stats.geHistogramData(histogramType); - } - System.out.println("geHistogramData() passed."); - } catch (Exception e) { - System.out.println("Failed in call to geHistogramData()"); - assert(false); //Should never reach here. - } - - RocksIterator iterator = db.newIterator(); - - boolean seekToFirstPassed = false; - for (iterator.seekToFirst(); iterator.isValid(); iterator.next()) { - iterator.status(); - assert(iterator.key() != null); - assert(iterator.value() != null); - seekToFirstPassed = true; - } - if(seekToFirstPassed) { - System.out.println("iterator seekToFirst tests passed."); - } - - boolean seekToLastPassed = false; - for (iterator.seekToLast(); iterator.isValid(); iterator.prev()) { - iterator.status(); - assert(iterator.key() != null); - assert(iterator.value() != null); - seekToLastPassed = true; - } - - if(seekToLastPassed) { - System.out.println("iterator seekToLastPassed tests passed."); - } - - iterator.seekToFirst(); - iterator.seek(iterator.key()); - assert(iterator.key() != null); - assert(iterator.value() != null); - - System.out.println("iterator seek test passed."); - - iterator.dispose(); - System.out.println("iterator tests passed."); - - iterator = db.newIterator(); - List keys = new ArrayList(); - for (iterator.seekToLast(); iterator.isValid(); iterator.prev()) { - keys.add(iterator.key()); - } - iterator.dispose(); - - Map values = db.multiGet(keys); - assert(values.size() == keys.size()); - for(byte[] value1 : values.values()) { - assert(value1 != null); - } - - values = db.multiGet(new ReadOptions(), keys); - assert(values.size() == keys.size()); - for(byte[] value1 : values.values()) { - assert(value1 != null); - } - } catch (RocksDBException e) { - System.err.println(e); - } - if (db != null) { - db.close(); - } - // be sure to dispose c++ pointers - options.dispose(); - readOptions.dispose(); - } -} diff --git a/java/org/rocksdb/benchmark/DbBenchmark.java b/java/benchmark/src/main/java/org/rocksdb/benchmark/DbBenchmark.java similarity index 86% rename from java/org/rocksdb/benchmark/DbBenchmark.java rename to java/benchmark/src/main/java/org/rocksdb/benchmark/DbBenchmark.java index b715f9af153..8af6d2edfb8 100644 --- a/java/org/rocksdb/benchmark/DbBenchmark.java +++ b/java/benchmark/src/main/java/org/rocksdb/benchmark/DbBenchmark.java @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). /** * Copyright (C) 2011 the original author or authors. * See the notice.md file distributed with this work for additional @@ -21,10 +21,14 @@ */ package org.rocksdb.benchmark; +import java.io.IOException; import java.lang.Runnable; import java.lang.Math; import java.io.File; +import java.lang.reflect.Constructor; +import java.lang.reflect.InvocationTargetException; import java.nio.ByteBuffer; +import java.nio.file.Files; import java.util.Collection; import java.util.Date; import java.util.EnumMap; @@ -40,6 +44,7 @@ import java.util.concurrent.Future; import java.util.concurrent.TimeUnit; import org.rocksdb.*; +import org.rocksdb.RocksMemEnv; import org.rocksdb.util.SizeUnit; class Stats { @@ -93,7 +98,7 @@ void merge(final Stats other) { void stop() { finish_ = System.nanoTime(); - seconds_ = (double) (finish_ - start_) / 1000000; + seconds_ = (double) (finish_ - start_) * 1e-9; } void addMessage(String msg) { @@ -139,15 +144,15 @@ void report(String name) { if (bytes_ > 0) { // Rate is computed on actual elapsed time, not the sum of per-thread // elapsed times. - double elapsed = (finish_ - start_) * 1e-6; + double elapsed = (finish_ - start_) * 1e-9; extra.append(String.format("%6.1f MB/s", (bytes_ / 1048576.0) / elapsed)); } extra.append(message_.toString()); - double elapsed = (finish_ - start_) * 1e-6; - double throughput = (double) done_ / elapsed; + double elapsed = (finish_ - start_); + double throughput = (double) done_ / (elapsed * 1e-9); System.out.format("%-12s : %11.3f micros/op %d ops/sec;%s%s\n", - name, elapsed * 1e6 / done_, + name, (elapsed * 1e-6) / done_, (long) throughput, (extra.length() == 0 ? "" : " "), extra.toString()); } } @@ -163,15 +168,6 @@ enum DBState { EXISTING } - enum CompressionType { - NONE, - SNAPPY, - ZLIB, - BZIP2, - LZ4, - LZ4HC - } - static { RocksDB.loadLibrary(); } @@ -255,7 +251,7 @@ public WriteTask( for (long j = 0; j < entriesPerBatch_; j++) { getKey(key, i + j, keyRange_); DbBenchmark.this.gen_.generate(value); - db_.put(writeOpt_, key, value); + batch.put(key, value); stats_.finishedSingleOp(keySize_ + valueSize_); } db_.write(writeOpt_, batch); @@ -427,9 +423,11 @@ public ReadSequentialTask( stats_.found_++; stats_.finishedSingleOp(iter.key().length + iter.value().length); if (isFinished()) { + iter.dispose(); return; } } + iter.dispose(); } } @@ -452,23 +450,22 @@ public DbBenchmark(Map flags) throws Exception { keysPerPrefix_ = (Integer) flags.get(Flag.keys_per_prefix); hashBucketCount_ = (Long) flags.get(Flag.hash_bucket_count); usePlainTable_ = (Boolean) flags.get(Flag.use_plain_table); + useMemenv_ = (Boolean) flags.get(Flag.use_mem_env); flags_ = flags; finishLock_ = new Object(); // options.setPrefixSize((Integer)flags_.get(Flag.prefix_size)); // options.setKeysPerPrefix((Long)flags_.get(Flag.keys_per_prefix)); compressionType_ = (String) flags.get(Flag.compression_type); - compression_ = CompressionType.NONE; + compression_ = CompressionType.NO_COMPRESSION; try { - if (compressionType_.equals("snappy")) { - System.loadLibrary("snappy"); - } else if (compressionType_.equals("zlib")) { - System.loadLibrary("z"); - } else if (compressionType_.equals("bzip2")) { - System.loadLibrary("bzip2"); - } else if (compressionType_.equals("lz4")) { - System.loadLibrary("lz4"); - } else if (compressionType_.equals("lz4hc")) { - System.loadLibrary("lz4hc"); + if (compressionType_!=null) { + final CompressionType compressionType = + CompressionType.getCompressionType(compressionType_); + if (compressionType != null && + compressionType != CompressionType.NO_COMPRESSION) { + System.loadLibrary(compressionType.getLibraryName()); + } + } } catch (UnsatisfiedLinkError e) { System.err.format("Unable to load %s library:%s%n" + @@ -489,32 +486,41 @@ private void prepareWriteOptions(WriteOptions options) { options.setDisableWAL((Boolean)flags_.get(Flag.disable_wal)); } - private void prepareOptions(Options options) { + private void prepareOptions(Options options) throws RocksDBException { if (!useExisting_) { options.setCreateIfMissing(true); } else { options.setCreateIfMissing(false); } - if (memtable_.equals("skip_list")) { - options.setMemTableConfig(new SkipListMemTableConfig()); - } else if (memtable_.equals("vector")) { - options.setMemTableConfig(new VectorMemTableConfig()); - } else if (memtable_.equals("hash_linkedlist")) { - options.setMemTableConfig( - new HashLinkedListMemTableConfig() - .setBucketCount(hashBucketCount_)); - options.useFixedLengthPrefixExtractor(prefixSize_); - } else if (memtable_.equals("hash_skiplist") || - memtable_.equals("prefix_hash")) { - options.setMemTableConfig( - new HashSkipListMemTableConfig() - .setBucketCount(hashBucketCount_)); - options.useFixedLengthPrefixExtractor(prefixSize_); - } else { - System.err.format( - "unable to detect the specified memtable, " + - "use the default memtable factory %s%n", - options.memTableFactoryName()); + if (useMemenv_) { + options.setEnv(new RocksMemEnv()); + } + switch (memtable_) { + case "skip_list": + options.setMemTableConfig(new SkipListMemTableConfig()); + break; + case "vector": + options.setMemTableConfig(new VectorMemTableConfig()); + break; + case "hash_linkedlist": + options.setMemTableConfig( + new HashLinkedListMemTableConfig() + .setBucketCount(hashBucketCount_)); + options.useFixedLengthPrefixExtractor(prefixSize_); + break; + case "hash_skiplist": + case "prefix_hash": + options.setMemTableConfig( + new HashSkipListMemTableConfig() + .setBucketCount(hashBucketCount_)); + options.useFixedLengthPrefixExtractor(prefixSize_); + break; + default: + System.err.format( + "unable to detect the specified memtable, " + + "use the default memtable factory %s%n", + options.memTableFactoryName()); + break; } if (usePlainTable_) { options.setTableFormatConfig( @@ -523,8 +529,8 @@ private void prepareOptions(Options options) { BlockBasedTableConfig table_options = new BlockBasedTableConfig(); table_options.setBlockSize((Long)flags_.get(Flag.block_size)) .setBlockCacheSize((Long)flags_.get(Flag.cache_size)) - .setFilterBitsPerKey((Integer)flags_.get(Flag.bloom_bits)) - .setCacheNumShardBits((Integer)flags_.get(Flag.cache_numshardbits)); + .setCacheNumShardBits( + (Integer)flags_.get(Flag.cache_numshardbits)); options.setTableFormatConfig(table_options); } options.setWriteBufferSize( @@ -539,10 +545,6 @@ private void prepareOptions(Options options) { (Integer)flags_.get(Flag.max_background_flushes)); options.setMaxOpenFiles( (Integer)flags_.get(Flag.open_files)); - options.setTableCacheRemoveScanCountLimit( - (Integer)flags_.get(Flag.cache_remove_scan_count_limit)); - options.setDisableDataSync( - (Boolean)flags_.get(Flag.disable_data_sync)); options.setUseFsync( (Boolean)flags_.get(Flag.use_fsync)); options.setWalDir( @@ -565,42 +567,34 @@ private void prepareOptions(Options options) { (Integer)flags_.get(Flag.bloom_locality)); options.setMinWriteBufferNumberToMerge( (Integer)flags_.get(Flag.min_write_buffer_number_to_merge)); - options.setMemtablePrefixBloomBits( - (Integer)flags_.get(Flag.memtable_bloom_bits)); + options.setMemtablePrefixBloomSizeRatio((Double) flags_.get(Flag.memtable_bloom_size_ratio)); options.setNumLevels( (Integer)flags_.get(Flag.num_levels)); options.setTargetFileSizeBase( (Integer)flags_.get(Flag.target_file_size_base)); - options.setTargetFileSizeMultiplier( - (Integer)flags_.get(Flag.target_file_size_multiplier)); + options.setTargetFileSizeMultiplier((Integer)flags_.get(Flag.target_file_size_multiplier)); options.setMaxBytesForLevelBase( (Integer)flags_.get(Flag.max_bytes_for_level_base)); - options.setMaxBytesForLevelMultiplier( - (Integer)flags_.get(Flag.max_bytes_for_level_multiplier)); + options.setMaxBytesForLevelMultiplier((Double) flags_.get(Flag.max_bytes_for_level_multiplier)); options.setLevelZeroStopWritesTrigger( (Integer)flags_.get(Flag.level0_stop_writes_trigger)); options.setLevelZeroSlowdownWritesTrigger( (Integer)flags_.get(Flag.level0_slowdown_writes_trigger)); options.setLevelZeroFileNumCompactionTrigger( (Integer)flags_.get(Flag.level0_file_num_compaction_trigger)); - options.setSoftRateLimit( - (Double)flags_.get(Flag.soft_rate_limit)); - options.setHardRateLimit( - (Double)flags_.get(Flag.hard_rate_limit)); - options.setRateLimitDelayMaxMilliseconds( - (Integer)flags_.get(Flag.rate_limit_delay_max_milliseconds)); - options.setMaxGrandparentOverlapFactor( - (Integer)flags_.get(Flag.max_grandparent_overlap_factor)); + options.setMaxCompactionBytes( + (Long) flags_.get(Flag.max_compaction_bytes)); options.setDisableAutoCompactions( (Boolean)flags_.get(Flag.disable_auto_compactions)); - options.setSourceCompactionFactor( - (Integer)flags_.get(Flag.source_compaction_factor)); - options.setFilterDeletes( - (Boolean)flags_.get(Flag.filter_deletes)); options.setMaxSuccessiveMerges( (Integer)flags_.get(Flag.max_successive_merges)); options.setWalTtlSeconds((Long)flags_.get(Flag.wal_ttl_seconds)); options.setWalSizeLimitMB((Long)flags_.get(Flag.wal_size_limit_MB)); + if(flags_.get(Flag.java_comparator) != null) { + options.setComparator( + (AbstractComparator)flags_.get(Flag.java_comparator)); + } + /* TODO(yhchiang): enable the following parameters options.setCompressionType((String)flags_.get(Flag.compression_type)); options.setCompressionLevel((Integer)flags_.get(Flag.compression_level)); @@ -645,53 +639,65 @@ private void run() throws RocksDBException { int currentTaskId = 0; boolean known = true; - if (benchmark.equals("fillseq")) { - tasks.add(new WriteSequentialTask( - currentTaskId++, randSeed_, num_, num_, writeOpt, 1)); - } else if (benchmark.equals("fillbatch")) { - tasks.add(new WriteRandomTask( - currentTaskId++, randSeed_, num_ / 1000, num_, writeOpt, 1000)); - } else if (benchmark.equals("fillrandom")) { - tasks.add(new WriteRandomTask( - currentTaskId++, randSeed_, num_, num_, writeOpt, 1)); - } else if (benchmark.equals("filluniquerandom")) { - tasks.add(new WriteUniqueRandomTask( - currentTaskId++, randSeed_, num_, num_, writeOpt, 1)); - } else if (benchmark.equals("fillsync")) { - writeOpt.setSync(true); - tasks.add(new WriteRandomTask( - currentTaskId++, randSeed_, num_ / 1000, num_ / 1000, - writeOpt, 1)); - } else if (benchmark.equals("readseq")) { - for (int t = 0; t < threadNum_; ++t) { - tasks.add(new ReadSequentialTask( - currentTaskId++, randSeed_, reads_ / threadNum_, num_)); - } - } else if (benchmark.equals("readrandom")) { - for (int t = 0; t < threadNum_; ++t) { - tasks.add(new ReadRandomTask( - currentTaskId++, randSeed_, reads_ / threadNum_, num_)); - } - } else if (benchmark.equals("readwhilewriting")) { - WriteTask writeTask = new WriteRandomTask( - -1, randSeed_, Long.MAX_VALUE, num_, writeOpt, 1, writesPerSeconds_); - writeTask.stats_.setExcludeFromMerge(); - bgTasks.add(writeTask); - for (int t = 0; t < threadNum_; ++t) { - tasks.add(new ReadRandomTask( - currentTaskId++, randSeed_, reads_ / threadNum_, num_)); - } - } else if (benchmark.equals("readhot")) { - for (int t = 0; t < threadNum_; ++t) { - tasks.add(new ReadRandomTask( - currentTaskId++, randSeed_, reads_ / threadNum_, num_ / 100)); - } - } else if (benchmark.equals("delete")) { - destroyDb(); - open(options); - } else { - known = false; - System.err.println("Unknown benchmark: " + benchmark); + switch (benchmark) { + case "fillseq": + tasks.add(new WriteSequentialTask( + currentTaskId++, randSeed_, num_, num_, writeOpt, 1)); + break; + case "fillbatch": + tasks.add(new WriteRandomTask( + currentTaskId++, randSeed_, num_ / 1000, num_, writeOpt, 1000)); + break; + case "fillrandom": + tasks.add(new WriteRandomTask( + currentTaskId++, randSeed_, num_, num_, writeOpt, 1)); + break; + case "filluniquerandom": + tasks.add(new WriteUniqueRandomTask( + currentTaskId++, randSeed_, num_, num_, writeOpt, 1)); + break; + case "fillsync": + writeOpt.setSync(true); + tasks.add(new WriteRandomTask( + currentTaskId++, randSeed_, num_ / 1000, num_ / 1000, + writeOpt, 1)); + break; + case "readseq": + for (int t = 0; t < threadNum_; ++t) { + tasks.add(new ReadSequentialTask( + currentTaskId++, randSeed_, reads_ / threadNum_, num_)); + } + break; + case "readrandom": + for (int t = 0; t < threadNum_; ++t) { + tasks.add(new ReadRandomTask( + currentTaskId++, randSeed_, reads_ / threadNum_, num_)); + } + break; + case "readwhilewriting": + WriteTask writeTask = new WriteRandomTask( + -1, randSeed_, Long.MAX_VALUE, num_, writeOpt, 1, writesPerSeconds_); + writeTask.stats_.setExcludeFromMerge(); + bgTasks.add(writeTask); + for (int t = 0; t < threadNum_; ++t) { + tasks.add(new ReadRandomTask( + currentTaskId++, randSeed_, reads_ / threadNum_, num_)); + } + break; + case "readhot": + for (int t = 0; t < threadNum_; ++t) { + tasks.add(new ReadRandomTask( + currentTaskId++, randSeed_, reads_ / threadNum_, num_ / 100)); + } + break; + case "delete": + destroyDb(); + open(options); + break; + default: + known = false; + System.err.println("Unknown benchmark: " + benchmark); + break; } if (known) { ExecutorService executor = Executors.newCachedThreadPool(); @@ -762,6 +768,7 @@ void printWarnings() { } private void open(Options options) throws RocksDBException { + System.out.println("Using database directory: " + databaseDir_); db_ = RocksDB.open(options, databaseDir_); } @@ -800,7 +807,7 @@ private void stop( System.out.printf( "%-16s : %11.5f micros/op; %6.1f MB/s;%s %d / %d task(s) finished.\n", - benchmark, (double) elapsedSeconds / stats.done_ * 1e6, + benchmark, elapsedSeconds / stats.done_ * 1e6, (stats.bytes_ / 1048576.0) / elapsedSeconds, extra, taskFinishedCount, concurrentThreads); } @@ -963,7 +970,7 @@ private enum Flag { return Integer.parseInt(value); } }, - write_buffer_size(4 * SizeUnit.MB, + write_buffer_size(4L * SizeUnit.MB, "Number of bytes to buffer in memtable before compacting\n" + "\t(initialized to default value by 'main'.)") { @Override public Object parseValue(String value) { @@ -1041,7 +1048,7 @@ private enum Flag { return Integer.parseInt(value); } }, - numdistinct(1000, + numdistinct(1000L, "Number of distinct keys to use. Used in RandomWithVerify to\n" + "\tread/write on fewer keys so that gets are more likely to find the\n" + "\tkey and puts are more likely to update the same key.") { @@ -1049,7 +1056,7 @@ private enum Flag { return Long.parseLong(value); } }, - merge_keys(-1, + merge_keys(-1L, "Number of distinct keys to use for MergeRandom and\n" + "\tReadRandomMergeRandom.\n" + "\tIf negative, there will be FLAGS_num keys.") { @@ -1154,7 +1161,7 @@ private enum Flag { return Long.parseLong(value); } }, - compressed_cache_size(-1, + compressed_cache_size(-1L, "Number of bytes to use as a cache of compressed data.") { @Override public Object parseValue(String value) { return Long.parseLong(value); @@ -1173,10 +1180,10 @@ private enum Flag { return Integer.parseInt(value); } }, - memtable_bloom_bits(0,"Bloom filter bits per key for memtable.\n" + - "\tNegative means no bloom filter.") { + memtable_bloom_size_ratio(0.0d, "Ratio of memtable used by the bloom filter.\n" + + "\t0 means no bloom filter.") { @Override public Object parseValue(String value) { - return Integer.parseInt(value); + return Double.parseDouble(value); } }, cache_numshardbits(-1,"Number of shards for the block cache\n" + @@ -1186,11 +1193,6 @@ private enum Flag { return Integer.parseInt(value); } }, - cache_remove_scan_count_limit(32,"") { - @Override public Object parseValue(String value) { - return Integer.parseInt(value); - } - }, verify_checksum(false,"Verify checksum for every block read\n" + "\tfrom storage.") { @Override public Object parseValue(String value) { @@ -1202,7 +1204,7 @@ private enum Flag { return parseBoolean(value); } }, - writes(-1,"Number of write operations to do. If negative, do\n" + + writes(-1L, "Number of write operations to do. If negative, do\n" + "\t--num reads.") { @Override public Object parseValue(String value) { return Long.parseLong(value); @@ -1213,12 +1215,6 @@ private enum Flag { return parseBoolean(value); } }, - disable_data_sync(false,"If true, do not wait until data is\n" + - "\tsynced to disk.") { - @Override public Object parseValue(String value) { - return parseBoolean(value); - } - }, use_fsync(false,"If true, issue fsync instead of fdatasync.") { @Override public Object parseValue(String value) { return parseBoolean(value); @@ -1251,10 +1247,10 @@ private enum Flag { return Integer.parseInt(value); } }, - max_bytes_for_level_multiplier(10, + max_bytes_for_level_multiplier(10.0d, "A multiplier to compute max bytes for level-N (N >= 2)") { @Override public Object parseValue(String value) { - return Integer.parseInt(value); + return Double.parseDouble(value); } }, level0_stop_writes_trigger(12,"Number of files in level-0\n" + @@ -1333,7 +1329,7 @@ private enum Flag { return Integer.parseInt(value); } }, - stats_interval(0,"Stats are reported every N operations when\n" + + stats_interval(0L, "Stats are reported every N operations when\n" + "\tthis is greater than zero. When 0 the interval grows over time.") { @Override public Object parseValue(String value) { return Long.parseLong(value); @@ -1350,12 +1346,12 @@ private enum Flag { return Integer.parseInt(value); } }, - soft_rate_limit(0.0,"") { + soft_rate_limit(0.0d,"") { @Override public Object parseValue(String value) { return Double.parseDouble(value); } }, - hard_rate_limit(0.0,"When not equal to 0 this make threads\n" + + hard_rate_limit(0.0d,"When not equal to 0 this make threads\n" + "\tsleep at each stats reporting interval until the compaction\n" + "\tscore for all levels is less than or equal to this value.") { @Override public Object parseValue(String value) { @@ -1369,11 +1365,10 @@ private enum Flag { return Integer.parseInt(value); } }, - max_grandparent_overlap_factor(10,"Control maximum bytes of\n" + - "\toverlaps in grandparent (i.e., level+2) before we stop building a\n" + - "\tsingle file in a level->level+1 compaction.") { + max_compaction_bytes(0L, "Limit number of bytes in one compaction to be lower than this\n" + + "\threshold. But it's not guaranteed.") { @Override public Object parseValue(String value) { - return Integer.parseInt(value); + return Long.parseLong(value); } }, readonly(false,"Run read only benchmarks.") { @@ -1386,13 +1381,6 @@ private enum Flag { return parseBoolean(value); } }, - source_compaction_factor(1,"Cap the size of data in level-K for\n" + - "\ta compaction run that compacts Level-K with Level-(K+1) (for\n" + - "\tK >= 1)") { - @Override public Object parseValue(String value) { - return Integer.parseInt(value); - } - }, wal_ttl_seconds(0L,"Set the TTL for the WAL Files in seconds.") { @Override public Object parseValue(String value) { return Long.parseLong(value); @@ -1405,12 +1393,18 @@ private enum Flag { } }, /* TODO(yhchiang): enable the following - bufferedio(rocksdb::EnvOptions().use_os_buffer, - "Allow buffered io using OS buffers.") { + direct_reads(rocksdb::EnvOptions().use_direct_reads, + "Allow direct I/O reads.") { @Override public Object parseValue(String value) { return parseBoolean(value); } - }, + }, + direct_writes(rocksdb::EnvOptions().use_direct_reads, + "Allow direct I/O reads.") { + @Override public Object parseValue(String value) { + return parseBoolean(value); + } + }, */ mmap_read(false, "Allow reads to occur via mmap-ing files.") { @@ -1468,11 +1462,42 @@ private enum Flag { return Integer.parseInt(value); } }, - db("/tmp/rocksdbjni-bench", + db(getTempDir("rocksdb-jni"), "Use the db with the following name.") { @Override public Object parseValue(String value) { return value; } + }, + use_mem_env(false, "Use RocksMemEnv instead of default filesystem based\n" + + "environment.") { + @Override public Object parseValue(String value) { + return parseBoolean(value); + } + }, + java_comparator(null, "Class name of a Java Comparator to use instead\n" + + "\tof the default C++ ByteWiseComparatorImpl. Must be available on\n" + + "\tthe classpath") { + @Override + protected Object parseValue(final String value) { + try { + final ComparatorOptions copt = new ComparatorOptions(); + final Class clsComparator = + (Class)Class.forName(value); + final Constructor cstr = + clsComparator.getConstructor(ComparatorOptions.class); + return cstr.newInstance(copt); + } catch(final ClassNotFoundException cnfe) { + throw new IllegalArgumentException("Java Comparator '" + value + "'" + + " not found on the classpath", cnfe); + } catch(final NoSuchMethodException nsme) { + throw new IllegalArgumentException("Java Comparator '" + value + "'" + + " does not have a public ComparatorOptions constructor", nsme); + } catch(final IllegalAccessException | InstantiationException + | InvocationTargetException ie) { + throw new IllegalArgumentException("Unable to construct Java" + + " Comparator '" + value + "'", ie); + } + } }; private Flag(Object defaultValue, String desc) { @@ -1503,6 +1528,18 @@ public boolean parseBoolean(String value) { private final String desc_; } + private final static String DEFAULT_TEMP_DIR = "/tmp"; + + private static String getTempDir(final String dirName) { + try { + return Files.createTempDirectory(dirName).toAbsolutePath().toString(); + } catch(final IOException ioe) { + System.err.println("Unable to create temp directory, defaulting to: " + + DEFAULT_TEMP_DIR); + return DEFAULT_TEMP_DIR + File.pathSeparator + dirName; + } + } + private static class RandomGenerator { private final byte[] data_; private int dataLength_; @@ -1585,6 +1622,9 @@ void setFinished(boolean flag) { RandomGenerator gen_; long startTime_; + // env + boolean useMemenv_; + // memtable related final int maxWriteBufferNumber_; final int prefixSize_; diff --git a/java/crossbuild/Vagrantfile b/java/crossbuild/Vagrantfile new file mode 100644 index 00000000000..4a321774888 --- /dev/null +++ b/java/crossbuild/Vagrantfile @@ -0,0 +1,27 @@ +# -*- mode: ruby -*- +# vi: set ft=ruby : + +# Vagrantfile API/syntax version. Don't touch unless you know what you're doing! +VAGRANTFILE_API_VERSION = "2" + +Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| + + config.vm.define "linux32" do |linux32| + linux32.vm.box = "hansode/centos-6.7-i386" + end + + config.vm.define "linux64" do |linux64| + linux64.vm.box = "hansode/centos-6.7-x86_64" + end + + config.vm.provider "virtualbox" do |v| + v.memory = 2048 + v.cpus = 4 + v.customize ["modifyvm", :id, "--nictype1", "virtio" ] + end + + config.vm.provision :shell, path: "build-linux-centos.sh" + config.vm.synced_folder "../target", "/rocksdb-build" + config.vm.synced_folder "../..", "/rocksdb", type: "rsync" + config.vm.boot_timeout = 1200 +end diff --git a/java/crossbuild/build-linux-centos.sh b/java/crossbuild/build-linux-centos.sh new file mode 100755 index 00000000000..2832eed8b8f --- /dev/null +++ b/java/crossbuild/build-linux-centos.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash + +set -e + +# remove fixed relesever variable present in the hanscode boxes +sudo rm -f /etc/yum/vars/releasever + +# enable EPEL +sudo yum -y install epel-release + +# install all required packages for rocksdb that are available through yum +sudo yum -y install openssl java-1.7.0-openjdk-devel zlib-devel bzip2-devel lz4-devel snappy-devel libzstd-devel + +# install gcc/g++ 4.8.2 from tru/devtools-2 +sudo wget -O /etc/yum.repos.d/devtools-2.repo https://people.centos.org/tru/devtools-2/devtools-2.repo +sudo yum -y install devtoolset-2-binutils devtoolset-2-gcc devtoolset-2-gcc-c++ + +# install gflags +wget https://github.com/gflags/gflags/archive/v2.0.tar.gz -O gflags-2.0.tar.gz +tar xvfz gflags-2.0.tar.gz; cd gflags-2.0; scl enable devtoolset-2 ./configure; scl enable devtoolset-2 make; sudo make install +export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib + +# set java home so we can build rocksdb jars +export JAVA_HOME=/usr/lib/jvm/java-1.7.0 + +# build rocksdb +cd /rocksdb +scl enable devtoolset-2 'make jclean clean' +scl enable devtoolset-2 'PORTABLE=1 make rocksdbjavastatic' +cp /rocksdb/java/target/librocksdbjni-* /rocksdb-build +cp /rocksdb/java/target/rocksdbjni-* /rocksdb-build + diff --git a/java/crossbuild/build-linux.sh b/java/crossbuild/build-linux.sh new file mode 100755 index 00000000000..48d1c28d922 --- /dev/null +++ b/java/crossbuild/build-linux.sh @@ -0,0 +1,14 @@ +#!/usr/bin/env bash +# install all required packages for rocksdb +sudo apt-get update +sudo apt-get -y install git make gcc g++ libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev default-jdk + +# set java home so we can build rocksdb jars +export JAVA_HOME=$(echo /usr/lib/jvm/java-7-openjdk*) +cd /rocksdb +make jclean clean +make -j 4 rocksdbjavastatic +cp /rocksdb/java/target/librocksdbjni-* /rocksdb-build +cp /rocksdb/java/target/rocksdbjni-* /rocksdb-build +sudo shutdown -h now + diff --git a/java/crossbuild/docker-build-linux-centos.sh b/java/crossbuild/docker-build-linux-centos.sh new file mode 100755 index 00000000000..44a8bfe06d3 --- /dev/null +++ b/java/crossbuild/docker-build-linux-centos.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash + +set -e + +rm -rf /rocksdb-local +cp -r /rocksdb-host /rocksdb-local +cd /rocksdb-local +scl enable devtoolset-2 'make jclean clean' +scl enable devtoolset-2 'PORTABLE=1 make rocksdbjavastatic' +cp java/target/librocksdbjni-linux*.so java/target/rocksdbjni-*-linux*.jar /rocksdb-host/java/target + diff --git a/java/jdb_bench.sh b/java/jdb_bench.sh index dba7dbd319b..9665de785ed 100755 --- a/java/jdb_bench.sh +++ b/java/jdb_bench.sh @@ -1 +1,10 @@ -java -server -d64 -XX:NewSize=4m -XX:+AggressiveOpts -Djava.library.path=.:../ -cp "rocksdbjni.jar:.:./*" org.rocksdb.benchmark.DbBenchmark $@ +PLATFORM=64 +if [ `getconf LONG_BIT` != "64" ] +then + PLATFORM=32 +fi + +ROCKS_JAR=`find target -name rocksdbjni*.jar` + +echo "Running benchmark in $PLATFORM-Bit mode." +java -server -d$PLATFORM -XX:NewSize=4m -XX:+AggressiveOpts -Djava.library.path=target -cp "${ROCKS_JAR}:benchmark/target/classes" org.rocksdb.benchmark.DbBenchmark $@ diff --git a/java/org/rocksdb/BackupableDB.java b/java/org/rocksdb/BackupableDB.java deleted file mode 100644 index 108c4deb547..00000000000 --- a/java/org/rocksdb/BackupableDB.java +++ /dev/null @@ -1,90 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * A subclass of RocksDB which supports backup-related operations. - * - * @see BackupableDBOptions - */ -public class BackupableDB extends RocksDB { - /** - * Open a BackupableDB under the specified path. - * Note that the backup path should be set properly in the - * input BackupableDBOptions. - * - * @param opt options for db. - * @param bopt backup related options. - * @param the db path for storing data. The path for storing - * backup should be specified in the BackupableDBOptions. - * @return reference to the opened BackupableDB. - */ - public static BackupableDB open( - Options opt, BackupableDBOptions bopt, String db_path) - throws RocksDBException { - - RocksDB db = RocksDB.open(opt, db_path); - BackupableDB bdb = new BackupableDB(); - bdb.open(db.nativeHandle_, bopt.nativeHandle_); - - // Prevent the RocksDB object from attempting to delete - // the underly C++ DB object. - db.disOwnNativeHandle(); - - return bdb; - } - - /** - * Captures the state of the database in the latest backup. - * Note that this function is not thread-safe. - * - * @param flushBeforeBackup if true, then all data will be flushed - * before creating backup. - */ - public void createNewBackup(boolean flushBeforeBackup) { - createNewBackup(nativeHandle_, flushBeforeBackup); - } - - /** - * Deletes old backups, keeping latest numBackupsToKeep alive. - * - * @param numBackupsToKeep Number of latest backups to keep. - */ - public void purgeOldBackups(int numBackupsToKeep) { - purgeOldBackups(nativeHandle_, numBackupsToKeep); - } - - - /** - * Close the BackupableDB instance and release resource. - * - * Internally, BackupableDB owns the rocksdb::DB pointer to its - * associated RocksDB. The release of that RocksDB pointer is - * handled in the destructor of the c++ rocksdb::BackupableDB and - * should be transparent to Java developers. - */ - @Override public synchronized void close() { - if (isInitialized()) { - super.close(); - } - } - - /** - * A protected construction that will be used in the static factory - * method BackupableDB.open(). - */ - protected BackupableDB() { - super(); - } - - @Override protected void finalize() { - close(); - } - - protected native void open(long rocksDBHandle, long backupDBOptionsHandle); - protected native void createNewBackup(long handle, boolean flag); - protected native void purgeOldBackups(long handle, int numBackupsToKeep); -} diff --git a/java/org/rocksdb/BackupableDBOptions.java b/java/org/rocksdb/BackupableDBOptions.java deleted file mode 100644 index 2c5047f773f..00000000000 --- a/java/org/rocksdb/BackupableDBOptions.java +++ /dev/null @@ -1,73 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * BackupableDBOptions to control the behavior of a backupable database. - * It will be used during the creation of a BackupableDB. - * - * Note that dispose() must be called before an Options instance - * become out-of-scope to release the allocated memory in c++. - * - * @param path Where to keep the backup files. Has to be different than dbname. - Best to set this to dbname_ + "/backups" - * @param shareTableFiles If share_table_files == true, backup will assume that - * table files with same name have the same contents. This enables - * incremental backups and avoids unnecessary data copies. If - * share_table_files == false, each backup will be on its own and will not - * share any data with other backups. default: true - * @param sync If sync == true, we can guarantee you'll get consistent backup - * even on a machine crash/reboot. Backup process is slower with sync - * enabled. If sync == false, we don't guarantee anything on machine reboot. - * However, chances are some of the backups are consistent. Default: true - * @param destroyOldData If true, it will delete whatever backups there are - * already. Default: false - * @param backupLogFiles If false, we won't backup log files. This option can be - * useful for backing up in-memory databases where log file are persisted, - * but table files are in memory. Default: true - * @param backupRateLimit Max bytes that can be transferred in a second during - * backup. If 0 or negative, then go as fast as you can. Default: 0 - * @param restoreRateLimit Max bytes that can be transferred in a second during - * restore. If 0 or negative, then go as fast as you can. Default: 0 - */ -public class BackupableDBOptions extends RocksObject { - public BackupableDBOptions(String path, boolean shareTableFiles, boolean sync, - boolean destroyOldData, boolean backupLogFiles, long backupRateLimit, - long restoreRateLimit) { - super(); - - backupRateLimit = (backupRateLimit <= 0) ? 0 : backupRateLimit; - restoreRateLimit = (restoreRateLimit <= 0) ? 0 : restoreRateLimit; - - newBackupableDBOptions(path, shareTableFiles, sync, destroyOldData, - backupLogFiles, backupRateLimit, restoreRateLimit); - } - - /** - * Returns the path to the BackupableDB directory. - * - * @return the path to the BackupableDB directory. - */ - public String backupDir() { - assert(isInitialized()); - return backupDir(nativeHandle_); - } - - /** - * Release the memory allocated for the current instance - * in the c++ side. - */ - @Override protected void disposeInternal() { - assert(isInitialized()); - disposeInternal(nativeHandle_); - } - - private native void newBackupableDBOptions(String path, - boolean shareTableFiles, boolean sync, boolean destroyOldData, - boolean backupLogFiles, long backupRateLimit, long restoreRateLimit); - private native String backupDir(long handle); - private native void disposeInternal(long handle); -} diff --git a/java/org/rocksdb/BlockBasedTableConfig.java b/java/org/rocksdb/BlockBasedTableConfig.java deleted file mode 100644 index 523a5769118..00000000000 --- a/java/org/rocksdb/BlockBasedTableConfig.java +++ /dev/null @@ -1,210 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -package org.rocksdb; - -/** - * The config for plain table sst format. - * - * BlockBasedTable is a RocksDB's default SST file format. - */ -public class BlockBasedTableConfig extends TableFormatConfig { - - public BlockBasedTableConfig() { - noBlockCache_ = false; - blockCacheSize_ = 8 * 1024 * 1024; - blockSize_ = 4 * 1024; - blockSizeDeviation_ =10; - blockRestartInterval_ =16; - wholeKeyFiltering_ = true; - bitsPerKey_ = 0; - } - - /** - * Disable block cache. If this is set to true, - * then no block cache should be used, and the block_cache should - * point to a nullptr object. - * Default: false - * - * @param noBlockCache if use block cache - * @return the reference to the current config. - */ - public BlockBasedTableConfig setNoBlockCache(boolean noBlockCache) { - noBlockCache_ = noBlockCache; - return this; - } - - /** - * @return if block cache is disabled - */ - public boolean noBlockCache() { - return noBlockCache_; - } - - /** - * Set the amount of cache in bytes that will be used by RocksDB. - * If cacheSize is non-positive, then cache will not be used. - * DEFAULT: 8M - * - * @param blockCacheSize block cache size in bytes - * @return the reference to the current config. - */ - public BlockBasedTableConfig setBlockCacheSize(long blockCacheSize) { - blockCacheSize_ = blockCacheSize; - return this; - } - - /** - * @return block cache size in bytes - */ - public long blockCacheSize() { - return blockCacheSize_; - } - - /** - * Controls the number of shards for the block cache. - * This is applied only if cacheSize is set to non-negative. - * - * @param numShardBits the number of shard bits. The resulting - * number of shards would be 2 ^ numShardBits. Any negative - * number means use default settings." - * @return the reference to the current option. - */ - public BlockBasedTableConfig setCacheNumShardBits(int numShardBits) { - numShardBits_ = numShardBits; - return this; - } - - /** - * Returns the number of shard bits used in the block cache. - * The resulting number of shards would be 2 ^ (returned value). - * Any negative number means use default settings. - * - * @return the number of shard bits used in the block cache. - */ - public int cacheNumShardBits() { - return numShardBits_; - } - - /** - * Approximate size of user data packed per block. Note that the - * block size specified here corresponds to uncompressed data. The - * actual size of the unit read from disk may be smaller if - * compression is enabled. This parameter can be changed dynamically. - * Default: 4K - * - * @param blockSize block size in bytes - * @return the reference to the current config. - */ - public BlockBasedTableConfig setBlockSize(long blockSize) { - blockSize_ = blockSize; - return this; - } - - /** - * @return block size in bytes - */ - public long blockSize() { - return blockSize_; - } - - /** - * This is used to close a block before it reaches the configured - * 'block_size'. If the percentage of free space in the current block is less - * than this specified number and adding a new record to the block will - * exceed the configured block size, then this block will be closed and the - * new record will be written to the next block. - * Default is 10. - * - * @param blockSizeDeviation the deviation to block size allowed - * @return the reference to the current config. - */ - public BlockBasedTableConfig setBlockSizeDeviation(int blockSizeDeviation) { - blockSizeDeviation_ = blockSizeDeviation; - return this; - } - - /** - * @return the hash table ratio. - */ - public int blockSizeDeviation() { - return blockSizeDeviation_; - } - - /** - * Set block restart interval - * - * @param restartInterval block restart interval. - * @return the reference to the current config. - */ - public BlockBasedTableConfig setBlockRestartInterval(int restartInterval) { - blockRestartInterval_ = restartInterval; - return this; - } - - /** - * @return block restart interval - */ - public int blockRestartInterval() { - return blockRestartInterval_; - } - - /** - * If true, place whole keys in the filter (not just prefixes). - * This must generally be true for gets to be efficient. - * Default: true - * - * @param wholeKeyFiltering if enable whole key filtering - * @return the reference to the current config. - */ - public BlockBasedTableConfig setWholeKeyFiltering(boolean wholeKeyFiltering) { - wholeKeyFiltering_ = wholeKeyFiltering; - return this; - } - - /** - * @return if whole key filtering is enabled - */ - public boolean wholeKeyFiltering() { - return wholeKeyFiltering_; - } - - /** - * Use the specified filter policy to reduce disk reads. - * - * Filter should not be disposed before options instances using this filter is - * disposed. If dispose() function is not called, then filter object will be - * GC'd automatically. - * - * Filter instance can be re-used in multiple options instances. - * - * @param Filter policy java instance. - * @return the reference to the current config. - */ - public BlockBasedTableConfig setFilterBitsPerKey(int bitsPerKey) { - bitsPerKey_ = bitsPerKey; - return this; - } - - @Override protected long newTableFactoryHandle() { - return newTableFactoryHandle(noBlockCache_, blockCacheSize_, numShardBits_, - blockSize_, blockSizeDeviation_, blockRestartInterval_, - wholeKeyFiltering_, bitsPerKey_); - } - - private native long newTableFactoryHandle( - boolean noBlockCache, long blockCacheSize, int numShardbits, - long blockSize, int blockSizeDeviation, int blockRestartInterval, - boolean wholeKeyFiltering, int bitsPerKey); - - private boolean noBlockCache_; - private long blockCacheSize_; - private int numShardBits_; - private long shard; - private long blockSize_; - private int blockSizeDeviation_; - private int blockRestartInterval_; - private boolean wholeKeyFiltering_; - private int bitsPerKey_; -} diff --git a/java/org/rocksdb/BloomFilter.java b/java/org/rocksdb/BloomFilter.java deleted file mode 100644 index 9c4913a8c68..00000000000 --- a/java/org/rocksdb/BloomFilter.java +++ /dev/null @@ -1,37 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * This class creates a new filter policy that uses a bloom filter - * with approximately the specified number of bits per key. - * A good value for bitsPerKey is 10, which yields a filter - * with ~ 1% false positive rate. - * - * Default value of bits per key is 10. - */ -public class BloomFilter extends Filter { - private static final int DEFAULT_BITS_PER_KEY = 10; - private final int bitsPerKey_; - - public BloomFilter() { - this(DEFAULT_BITS_PER_KEY); - } - - public BloomFilter(int bitsPerKey) { - super(); - bitsPerKey_ = bitsPerKey; - - createNewFilter(); - } - - @Override - protected void createNewFilter() { - createNewFilter0(bitsPerKey_); - } - - private native void createNewFilter0(int bitsKeyKey); -} diff --git a/java/org/rocksdb/CompactionStyle.java b/java/org/rocksdb/CompactionStyle.java deleted file mode 100644 index 5c41dfdd233..00000000000 --- a/java/org/rocksdb/CompactionStyle.java +++ /dev/null @@ -1,22 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -public enum CompactionStyle { - LEVEL((byte) 0), - UNIVERSAL((byte) 1), - FIFO((byte) 2); - - private final byte value_; - - private CompactionStyle(byte value) { - value_ = value; - } - - public byte getValue() { - return value_; - } -} diff --git a/java/org/rocksdb/CompressionType.java b/java/org/rocksdb/CompressionType.java deleted file mode 100644 index c5d6253a913..00000000000 --- a/java/org/rocksdb/CompressionType.java +++ /dev/null @@ -1,25 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -public enum CompressionType { - NO_COMPRESSION((byte) 0), - SNAPPY_COMPRESSION((byte) 1), - ZLIB_COMPRESSION((byte) 2), - BZLIB2_COMPRESSION((byte) 3), - LZ4_COMPRESSION((byte) 4), - LZ4HC_COMPRESSION((byte) 5); - - private final byte value_; - - private CompressionType(byte value) { - value_ = value; - } - - public byte getValue() { - return value_; - } -} diff --git a/java/org/rocksdb/HashLinkedListMemTableConfig.java b/java/org/rocksdb/HashLinkedListMemTableConfig.java deleted file mode 100644 index 24fcd8b52be..00000000000 --- a/java/org/rocksdb/HashLinkedListMemTableConfig.java +++ /dev/null @@ -1,52 +0,0 @@ -package org.rocksdb; - -/** - * The config for hash linked list memtable representation - * Such memtable contains a fix-sized array of buckets, where - * each bucket points to a sorted singly-linked - * list (or null if the bucket is empty). - * - * Note that since this mem-table representation relies on the - * key prefix, it is required to invoke one of the usePrefixExtractor - * functions to specify how to extract key prefix given a key. - * If proper prefix-extractor is not set, then RocksDB will - * use the default memtable representation (SkipList) instead - * and post a warning in the LOG. - */ -public class HashLinkedListMemTableConfig extends MemTableConfig { - public static final long DEFAULT_BUCKET_COUNT = 50000; - - public HashLinkedListMemTableConfig() { - bucketCount_ = DEFAULT_BUCKET_COUNT; - } - - /** - * Set the number of buckets in the fixed-size array used - * in the hash linked-list mem-table. - * - * @param count the number of hash buckets. - * @return the reference to the current HashLinkedListMemTableConfig. - */ - public HashLinkedListMemTableConfig setBucketCount(long count) { - bucketCount_ = count; - return this; - } - - /** - * Returns the number of buckets that will be used in the memtable - * created based on this config. - * - * @return the number of buckets - */ - public long bucketCount() { - return bucketCount_; - } - - @Override protected long newMemTableFactoryHandle() { - return newMemTableFactoryHandle(bucketCount_); - } - - private native long newMemTableFactoryHandle(long bucketCount); - - private long bucketCount_; -} diff --git a/java/org/rocksdb/HistogramType.java b/java/org/rocksdb/HistogramType.java deleted file mode 100644 index 751c03a1111..00000000000 --- a/java/org/rocksdb/HistogramType.java +++ /dev/null @@ -1,39 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -public enum HistogramType { - DB_GET(0), - DB_WRITE(1), - COMPACTION_TIME(2), - TABLE_SYNC_MICROS(3), - COMPACTION_OUTFILE_SYNC_MICROS(4), - WAL_FILE_SYNC_MICROS(5), - MANIFEST_FILE_SYNC_MICROS(6), - // TIME SPENT IN IO DURING TABLE OPEN - TABLE_OPEN_IO_MICROS(7), - DB_MULTIGET(8), - READ_BLOCK_COMPACTION_MICROS(9), - READ_BLOCK_GET_MICROS(10), - WRITE_RAW_BLOCK_MICROS(11), - - STALL_L0_SLOWDOWN_COUNT(12), - STALL_MEMTABLE_COMPACTION_COUNT(13), - STALL_L0_NUM_FILES_COUNT(14), - HARD_RATE_LIMIT_DELAY_COUNT(15), - SOFT_RATE_LIMIT_DELAY_COUNT(16), - NUM_FILES_IN_SINGLE_COMPACTION(17); - - private final int value_; - - private HistogramType(int value) { - value_ = value; - } - - public int getValue() { - return value_; - } -} diff --git a/java/org/rocksdb/Options.java b/java/org/rocksdb/Options.java deleted file mode 100644 index 125f06afdfc..00000000000 --- a/java/org/rocksdb/Options.java +++ /dev/null @@ -1,2240 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * Options to control the behavior of a database. It will be used - * during the creation of a RocksDB (i.e., RocksDB.open()). - * - * If dispose() function is not called, then it will be GC'd automatically and - * native resources will be released as part of the process. - */ -public class Options extends RocksObject { - static final long DEFAULT_CACHE_SIZE = 8 << 20; - static final int DEFAULT_NUM_SHARD_BITS = -1; - /** - * Construct options for opening a RocksDB. - * - * This constructor will create (by allocating a block of memory) - * an rocksdb::Options in the c++ side. - */ - public Options() { - super(); - cacheSize_ = DEFAULT_CACHE_SIZE; - numShardBits_ = DEFAULT_NUM_SHARD_BITS; - newOptions(); - env_ = RocksEnv.getDefault(); - } - - /** - * If this value is set to true, then the database will be created - * if it is missing during RocksDB.open(). - * Default: false - * - * @param flag a flag indicating whether to create a database the - * specified database in RocksDB.open() operation is missing. - * @return the instance of the current Options. - * @see RocksDB.open() - */ - public Options setCreateIfMissing(boolean flag) { - assert(isInitialized()); - setCreateIfMissing(nativeHandle_, flag); - return this; - } - - /** - * Use the specified object to interact with the environment, - * e.g. to read/write files, schedule background work, etc. - * Default: RocksEnv.getDefault() - */ - public Options setEnv(RocksEnv env) { - assert(isInitialized()); - setEnv(nativeHandle_, env.nativeHandle_); - env_ = env; - return this; - } - private native void setEnv(long optHandle, long envHandle); - - public RocksEnv getEnv() { - return env_; - } - private native long getEnvHandle(long handle); - - /** - * Return true if the create_if_missing flag is set to true. - * If true, the database will be created if it is missing. - * - * @return true if the createIfMissing option is set to true. - * @see setCreateIfMissing() - */ - public boolean createIfMissing() { - assert(isInitialized()); - return createIfMissing(nativeHandle_); - } - - /** - * Amount of data to build up in memory (backed by an unsorted log - * on disk) before converting to a sorted on-disk file. - * - * Larger values increase performance, especially during bulk loads. - * Up to max_write_buffer_number write buffers may be held in memory - * at the same time, so you may wish to adjust this parameter - * to control memory usage. - * - * Also, a larger write buffer will result in a longer recovery time - * the next time the database is opened. - * - * Default: 4MB - * @param writeBufferSize the size of write buffer. - * @return the instance of the current Options. - * @see RocksDB.open() - */ - public Options setWriteBufferSize(long writeBufferSize) { - assert(isInitialized()); - setWriteBufferSize(nativeHandle_, writeBufferSize); - return this; - } - - /** - * Return size of write buffer size. - * - * @return size of write buffer. - * @see setWriteBufferSize() - */ - public long writeBufferSize() { - assert(isInitialized()); - return writeBufferSize(nativeHandle_); - } - - /** - * The maximum number of write buffers that are built up in memory. - * The default is 2, so that when 1 write buffer is being flushed to - * storage, new writes can continue to the other write buffer. - * Default: 2 - * - * @param maxWriteBufferNumber maximum number of write buffers. - * @return the instance of the current Options. - * @see RocksDB.open() - */ - public Options setMaxWriteBufferNumber(int maxWriteBufferNumber) { - assert(isInitialized()); - setMaxWriteBufferNumber(nativeHandle_, maxWriteBufferNumber); - return this; - } - - /** - * Returns maximum number of write buffers. - * - * @return maximum number of write buffers. - * @see setMaxWriteBufferNumber() - */ - public int maxWriteBufferNumber() { - assert(isInitialized()); - return maxWriteBufferNumber(nativeHandle_); - } - - /** - * If true, an error will be thrown during RocksDB.open() if the - * database already exists. - * - * @return if true, an error is raised when the specified database - * already exists before open. - */ - public boolean errorIfExists() { - assert(isInitialized()); - return errorIfExists(nativeHandle_); - } - private native boolean errorIfExists(long handle); - - /** - * If true, an error will be thrown during RocksDB.open() if the - * database already exists. - * Default: false - * - * @param errorIfExists if true, an exception will be thrown - * during RocksDB.open() if the database already exists. - * @return the reference to the current option. - * @see RocksDB.open() - */ - public Options setErrorIfExists(boolean errorIfExists) { - assert(isInitialized()); - setErrorIfExists(nativeHandle_, errorIfExists); - return this; - } - private native void setErrorIfExists(long handle, boolean errorIfExists); - - /** - * If true, the implementation will do aggressive checking of the - * data it is processing and will stop early if it detects any - * errors. This may have unforeseen ramifications: for example, a - * corruption of one DB entry may cause a large number of entries to - * become unreadable or for the entire DB to become unopenable. - * If any of the writes to the database fails (Put, Delete, Merge, Write), - * the database will switch to read-only mode and fail all other - * Write operations. - * - * @return a boolean indicating whether paranoid-check is on. - */ - public boolean paranoidChecks() { - assert(isInitialized()); - return paranoidChecks(nativeHandle_); - } - private native boolean paranoidChecks(long handle); - - /** - * If true, the implementation will do aggressive checking of the - * data it is processing and will stop early if it detects any - * errors. This may have unforeseen ramifications: for example, a - * corruption of one DB entry may cause a large number of entries to - * become unreadable or for the entire DB to become unopenable. - * If any of the writes to the database fails (Put, Delete, Merge, Write), - * the database will switch to read-only mode and fail all other - * Write operations. - * Default: true - * - * @param paranoidChecks a flag to indicate whether paranoid-check - * is on. - * @return the reference to the current option. - */ - public Options setParanoidChecks(boolean paranoidChecks) { - assert(isInitialized()); - setParanoidChecks(nativeHandle_, paranoidChecks); - return this; - } - private native void setParanoidChecks( - long handle, boolean paranoidChecks); - - /** - * Number of open files that can be used by the DB. You may need to - * increase this if your database has a large working set. Value -1 means - * files opened are always kept open. You can estimate number of files based - * on target_file_size_base and target_file_size_multiplier for level-based - * compaction. For universal-style compaction, you can usually set it to -1. - * - * @return the maximum number of open files. - */ - public int maxOpenFiles() { - assert(isInitialized()); - return maxOpenFiles(nativeHandle_); - } - private native int maxOpenFiles(long handle); - - /** - * Number of open files that can be used by the DB. You may need to - * increase this if your database has a large working set. Value -1 means - * files opened are always kept open. You can estimate number of files based - * on target_file_size_base and target_file_size_multiplier for level-based - * compaction. For universal-style compaction, you can usually set it to -1. - * Default: 5000 - * - * @param maxOpenFiles the maximum number of open files. - * @return the reference to the current option. - */ - public Options setMaxOpenFiles(int maxOpenFiles) { - assert(isInitialized()); - setMaxOpenFiles(nativeHandle_, maxOpenFiles); - return this; - } - private native void setMaxOpenFiles(long handle, int maxOpenFiles); - - /** - * If true, then the contents of data files are not synced - * to stable storage. Their contents remain in the OS buffers till the - * OS decides to flush them. This option is good for bulk-loading - * of data. Once the bulk-loading is complete, please issue a - * sync to the OS to flush all dirty buffesrs to stable storage. - * - * @return if true, then data-sync is disabled. - */ - public boolean disableDataSync() { - assert(isInitialized()); - return disableDataSync(nativeHandle_); - } - private native boolean disableDataSync(long handle); - - /** - * If true, then the contents of data files are not synced - * to stable storage. Their contents remain in the OS buffers till the - * OS decides to flush them. This option is good for bulk-loading - * of data. Once the bulk-loading is complete, please issue a - * sync to the OS to flush all dirty buffesrs to stable storage. - * Default: false - * - * @param disableDataSync a boolean flag to specify whether to - * disable data sync. - * @return the reference to the current option. - */ - public Options setDisableDataSync(boolean disableDataSync) { - assert(isInitialized()); - setDisableDataSync(nativeHandle_, disableDataSync); - return this; - } - private native void setDisableDataSync(long handle, boolean disableDataSync); - - /** - * If true, then every store to stable storage will issue a fsync. - * If false, then every store to stable storage will issue a fdatasync. - * This parameter should be set to true while storing data to - * filesystem like ext3 that can lose files after a reboot. - * - * @return true if fsync is used. - */ - public boolean useFsync() { - assert(isInitialized()); - return useFsync(nativeHandle_); - } - private native boolean useFsync(long handle); - - /** - * If true, then every store to stable storage will issue a fsync. - * If false, then every store to stable storage will issue a fdatasync. - * This parameter should be set to true while storing data to - * filesystem like ext3 that can lose files after a reboot. - * Default: false - * - * @param useFsync a boolean flag to specify whether to use fsync - * @return the reference to the current option. - */ - public Options setUseFsync(boolean useFsync) { - assert(isInitialized()); - setUseFsync(nativeHandle_, useFsync); - return this; - } - private native void setUseFsync(long handle, boolean useFsync); - - /** - * The time interval in seconds between each two consecutive stats logs. - * This number controls how often a new scribe log about - * db deploy stats is written out. - * -1 indicates no logging at all. - * - * @return the time interval in seconds between each two consecutive - * stats logs. - */ - public int dbStatsLogInterval() { - assert(isInitialized()); - return dbStatsLogInterval(nativeHandle_); - } - private native int dbStatsLogInterval(long handle); - - /** - * The time interval in seconds between each two consecutive stats logs. - * This number controls how often a new scribe log about - * db deploy stats is written out. - * -1 indicates no logging at all. - * Default value is 1800 (half an hour). - * - * @param dbStatsLogInterval the time interval in seconds between each - * two consecutive stats logs. - * @return the reference to the current option. - */ - public Options setDbStatsLogInterval(int dbStatsLogInterval) { - assert(isInitialized()); - setDbStatsLogInterval(nativeHandle_, dbStatsLogInterval); - return this; - } - private native void setDbStatsLogInterval( - long handle, int dbStatsLogInterval); - - /** - * Returns the directory of info log. - * - * If it is empty, the log files will be in the same dir as data. - * If it is non empty, the log files will be in the specified dir, - * and the db data dir's absolute path will be used as the log file - * name's prefix. - * - * @return the path to the info log directory - */ - public String dbLogDir() { - assert(isInitialized()); - return dbLogDir(nativeHandle_); - } - private native String dbLogDir(long handle); - - /** - * This specifies the info LOG dir. - * If it is empty, the log files will be in the same dir as data. - * If it is non empty, the log files will be in the specified dir, - * and the db data dir's absolute path will be used as the log file - * name's prefix. - * - * @param dbLogDir the path to the info log directory - * @return the reference to the current option. - */ - public Options setDbLogDir(String dbLogDir) { - assert(isInitialized()); - setDbLogDir(nativeHandle_, dbLogDir); - return this; - } - private native void setDbLogDir(long handle, String dbLogDir); - - /** - * Returns the path to the write-ahead-logs (WAL) directory. - * - * If it is empty, the log files will be in the same dir as data, - * dbname is used as the data dir by default - * If it is non empty, the log files will be in kept the specified dir. - * When destroying the db, - * all log files in wal_dir and the dir itself is deleted - * - * @return the path to the write-ahead-logs (WAL) directory. - */ - public String walDir() { - assert(isInitialized()); - return walDir(nativeHandle_); - } - private native String walDir(long handle); - - /** - * This specifies the absolute dir path for write-ahead logs (WAL). - * If it is empty, the log files will be in the same dir as data, - * dbname is used as the data dir by default - * If it is non empty, the log files will be in kept the specified dir. - * When destroying the db, - * all log files in wal_dir and the dir itself is deleted - * - * @param walDir the path to the write-ahead-log directory. - * @return the reference to the current option. - */ - public Options setWalDir(String walDir) { - assert(isInitialized()); - setWalDir(nativeHandle_, walDir); - return this; - } - private native void setWalDir(long handle, String walDir); - - /** - * The periodicity when obsolete files get deleted. The default - * value is 6 hours. The files that get out of scope by compaction - * process will still get automatically delete on every compaction, - * regardless of this setting - * - * @return the time interval in micros when obsolete files will be deleted. - */ - public long deleteObsoleteFilesPeriodMicros() { - assert(isInitialized()); - return deleteObsoleteFilesPeriodMicros(nativeHandle_); - } - private native long deleteObsoleteFilesPeriodMicros(long handle); - - /** - * The periodicity when obsolete files get deleted. The default - * value is 6 hours. The files that get out of scope by compaction - * process will still get automatically delete on every compaction, - * regardless of this setting - * - * @param micros the time interval in micros - * @return the reference to the current option. - */ - public Options setDeleteObsoleteFilesPeriodMicros(long micros) { - assert(isInitialized()); - setDeleteObsoleteFilesPeriodMicros(nativeHandle_, micros); - return this; - } - private native void setDeleteObsoleteFilesPeriodMicros( - long handle, long micros); - - /** - * Returns the maximum number of concurrent background compaction jobs, - * submitted to the default LOW priority thread pool. - * When increasing this number, we may also want to consider increasing - * number of threads in LOW priority thread pool. - * Default: 1 - * - * @return the maximum number of concurrent background compaction jobs. - * @see Env.setBackgroundThreads() - */ - public int maxBackgroundCompactions() { - assert(isInitialized()); - return maxBackgroundCompactions(nativeHandle_); - } - - /** - * Creates statistics object which collects metrics about database operations. - Statistics objects should not be shared between DB instances as - it does not use any locks to prevent concurrent updates. - * - * @return the instance of the current Options. - * @see RocksDB.open() - */ - public Options createStatistics() { - assert(isInitialized()); - createStatistics(nativeHandle_); - return this; - } - - /** - * Returns statistics object. Calls createStatistics() if - * C++ returns NULL pointer for statistics. - * - * @return the instance of the statistics object. - * @see createStatistics() - */ - public Statistics statisticsPtr() { - assert(isInitialized()); - - long statsPtr = statisticsPtr(nativeHandle_); - if(statsPtr == 0) { - createStatistics(); - statsPtr = statisticsPtr(nativeHandle_); - } - - return new Statistics(statsPtr); - } - - /** - * Specifies the maximum number of concurrent background compaction jobs, - * submitted to the default LOW priority thread pool. - * If you're increasing this, also consider increasing number of threads in - * LOW priority thread pool. For more information, see - * Default: 1 - * - * @param maxBackgroundCompactions the maximum number of background - * compaction jobs. - * @return the reference to the current option. - * - * @see Env.setBackgroundThreads() - * @see maxBackgroundFlushes() - */ - public Options setMaxBackgroundCompactions(int maxBackgroundCompactions) { - assert(isInitialized()); - setMaxBackgroundCompactions(nativeHandle_, maxBackgroundCompactions); - return this; - } - - /** - * Returns the maximum number of concurrent background flush jobs. - * If you're increasing this, also consider increasing number of threads in - * HIGH priority thread pool. For more information, see - * Default: 1 - * - * @return the maximum number of concurrent background flush jobs. - * @see Env.setBackgroundThreads() - */ - public int maxBackgroundFlushes() { - assert(isInitialized()); - return maxBackgroundFlushes(nativeHandle_); - } - private native int maxBackgroundFlushes(long handle); - - /** - * Specifies the maximum number of concurrent background flush jobs. - * If you're increasing this, also consider increasing number of threads in - * HIGH priority thread pool. For more information, see - * Default: 1 - * - * @param maxBackgroundFlushes - * @return the reference to the current option. - * - * @see Env.setBackgroundThreads() - * @see maxBackgroundCompactions() - */ - public Options setMaxBackgroundFlushes(int maxBackgroundFlushes) { - assert(isInitialized()); - setMaxBackgroundFlushes(nativeHandle_, maxBackgroundFlushes); - return this; - } - private native void setMaxBackgroundFlushes( - long handle, int maxBackgroundFlushes); - - /** - * Returns the maximum size of a info log file. If the current log file - * is larger than this size, a new info log file will be created. - * If 0, all logs will be written to one log file. - * - * @return the maximum size of the info log file. - */ - public long maxLogFileSize() { - assert(isInitialized()); - return maxLogFileSize(nativeHandle_); - } - private native long maxLogFileSize(long handle); - - /** - * Specifies the maximum size of a info log file. If the current log file - * is larger than `max_log_file_size`, a new info log file will - * be created. - * If 0, all logs will be written to one log file. - * - * @param maxLogFileSize the maximum size of a info log file. - * @return the reference to the current option. - */ - public Options setMaxLogFileSize(long maxLogFileSize) { - assert(isInitialized()); - setMaxLogFileSize(nativeHandle_, maxLogFileSize); - return this; - } - private native void setMaxLogFileSize(long handle, long maxLogFileSize); - - /** - * Returns the time interval for the info log file to roll (in seconds). - * If specified with non-zero value, log file will be rolled - * if it has been active longer than `log_file_time_to_roll`. - * Default: 0 (disabled) - * - * @return the time interval in seconds. - */ - public long logFileTimeToRoll() { - assert(isInitialized()); - return logFileTimeToRoll(nativeHandle_); - } - private native long logFileTimeToRoll(long handle); - - /** - * Specifies the time interval for the info log file to roll (in seconds). - * If specified with non-zero value, log file will be rolled - * if it has been active longer than `log_file_time_to_roll`. - * Default: 0 (disabled) - * - * @param logFileTimeToRoll the time interval in seconds. - * @return the reference to the current option. - */ - public Options setLogFileTimeToRoll(long logFileTimeToRoll) { - assert(isInitialized()); - setLogFileTimeToRoll(nativeHandle_, logFileTimeToRoll); - return this; - } - private native void setLogFileTimeToRoll( - long handle, long logFileTimeToRoll); - - /** - * Returns the maximum number of info log files to be kept. - * Default: 1000 - * - * @return the maximum number of info log files to be kept. - */ - public long keepLogFileNum() { - assert(isInitialized()); - return keepLogFileNum(nativeHandle_); - } - private native long keepLogFileNum(long handle); - - /** - * Specifies the maximum number of info log files to be kept. - * Default: 1000 - * - * @param keepLogFileNum the maximum number of info log files to be kept. - * @return the reference to the current option. - */ - public Options setKeepLogFileNum(long keepLogFileNum) { - assert(isInitialized()); - setKeepLogFileNum(nativeHandle_, keepLogFileNum); - return this; - } - private native void setKeepLogFileNum(long handle, long keepLogFileNum); - - /** - * Manifest file is rolled over on reaching this limit. - * The older manifest file be deleted. - * The default value is MAX_INT so that roll-over does not take place. - * - * @return the size limit of a manifest file. - */ - public long maxManifestFileSize() { - assert(isInitialized()); - return maxManifestFileSize(nativeHandle_); - } - private native long maxManifestFileSize(long handle); - - /** - * Manifest file is rolled over on reaching this limit. - * The older manifest file be deleted. - * The default value is MAX_INT so that roll-over does not take place. - * - * @param maxManifestFileSize the size limit of a manifest file. - * @return the reference to the current option. - */ - public Options setMaxManifestFileSize(long maxManifestFileSize) { - assert(isInitialized()); - setMaxManifestFileSize(nativeHandle_, maxManifestFileSize); - return this; - } - private native void setMaxManifestFileSize( - long handle, long maxManifestFileSize); - - /** - * Number of shards used for table cache. - * - * @return the number of shards used for table cache. - */ - public int tableCacheNumshardbits() { - assert(isInitialized()); - return tableCacheNumshardbits(nativeHandle_); - } - private native int tableCacheNumshardbits(long handle); - - /** - * Number of shards used for table cache. - * - * @param tableCacheNumshardbits the number of chards - * @return the reference to the current option. - */ - public Options setTableCacheNumshardbits(int tableCacheNumshardbits) { - assert(isInitialized()); - setTableCacheNumshardbits(nativeHandle_, tableCacheNumshardbits); - return this; - } - private native void setTableCacheNumshardbits( - long handle, int tableCacheNumshardbits); - - /** - * During data eviction of table's LRU cache, it would be inefficient - * to strictly follow LRU because this piece of memory will not really - * be released unless its refcount falls to zero. Instead, make two - * passes: the first pass will release items with refcount = 1, - * and if not enough space releases after scanning the number of - * elements specified by this parameter, we will remove items in LRU - * order. - * - * @return scan count limit - */ - public int tableCacheRemoveScanCountLimit() { - assert(isInitialized()); - return tableCacheRemoveScanCountLimit(nativeHandle_); - } - private native int tableCacheRemoveScanCountLimit(long handle); - - /** - * During data eviction of table's LRU cache, it would be inefficient - * to strictly follow LRU because this piece of memory will not really - * be released unless its refcount falls to zero. Instead, make two - * passes: the first pass will release items with refcount = 1, - * and if not enough space releases after scanning the number of - * elements specified by this parameter, we will remove items in LRU - * order. - * - * @param limit scan count limit - * @return the reference to the current option. - */ - public Options setTableCacheRemoveScanCountLimit(int limit) { - assert(isInitialized()); - setTableCacheRemoveScanCountLimit(nativeHandle_, limit); - return this; - } - private native void setTableCacheRemoveScanCountLimit( - long handle, int limit); - - /** - * WalTtlSeconds() and walSizeLimitMB() affect how archived logs - * will be deleted. - * 1. If both set to 0, logs will be deleted asap and will not get into - * the archive. - * 2. If WAL_ttl_seconds is 0 and WAL_size_limit_MB is not 0, - * WAL files will be checked every 10 min and if total size is greater - * then WAL_size_limit_MB, they will be deleted starting with the - * earliest until size_limit is met. All empty files will be deleted. - * 3. If WAL_ttl_seconds is not 0 and WAL_size_limit_MB is 0, then - * WAL files will be checked every WAL_ttl_secondsi / 2 and those that - * are older than WAL_ttl_seconds will be deleted. - * 4. If both are not 0, WAL files will be checked every 10 min and both - * checks will be performed with ttl being first. - * - * @return the wal-ttl seconds - * @see walSizeLimitMB() - */ - public long walTtlSeconds() { - assert(isInitialized()); - return walTtlSeconds(nativeHandle_); - } - private native long walTtlSeconds(long handle); - - /** - * WalTtlSeconds() and walSizeLimitMB() affect how archived logs - * will be deleted. - * 1. If both set to 0, logs will be deleted asap and will not get into - * the archive. - * 2. If WAL_ttl_seconds is 0 and WAL_size_limit_MB is not 0, - * WAL files will be checked every 10 min and if total size is greater - * then WAL_size_limit_MB, they will be deleted starting with the - * earliest until size_limit is met. All empty files will be deleted. - * 3. If WAL_ttl_seconds is not 0 and WAL_size_limit_MB is 0, then - * WAL files will be checked every WAL_ttl_secondsi / 2 and those that - * are older than WAL_ttl_seconds will be deleted. - * 4. If both are not 0, WAL files will be checked every 10 min and both - * checks will be performed with ttl being first. - * - * @param walTtlSeconds the ttl seconds - * @return the reference to the current option. - * @see setWalSizeLimitMB() - */ - public Options setWalTtlSeconds(long walTtlSeconds) { - assert(isInitialized()); - setWalTtlSeconds(nativeHandle_, walTtlSeconds); - return this; - } - private native void setWalTtlSeconds(long handle, long walTtlSeconds); - - /** - * WalTtlSeconds() and walSizeLimitMB() affect how archived logs - * will be deleted. - * 1. If both set to 0, logs will be deleted asap and will not get into - * the archive. - * 2. If WAL_ttl_seconds is 0 and WAL_size_limit_MB is not 0, - * WAL files will be checked every 10 min and if total size is greater - * then WAL_size_limit_MB, they will be deleted starting with the - * earliest until size_limit is met. All empty files will be deleted. - * 3. If WAL_ttl_seconds is not 0 and WAL_size_limit_MB is 0, then - * WAL files will be checked every WAL_ttl_secondsi / 2 and those that - * are older than WAL_ttl_seconds will be deleted. - * 4. If both are not 0, WAL files will be checked every 10 min and both - * checks will be performed with ttl being first. - * - * @return size limit in mega-bytes. - * @see walSizeLimitMB() - */ - public long walSizeLimitMB() { - assert(isInitialized()); - return walSizeLimitMB(nativeHandle_); - } - private native long walSizeLimitMB(long handle); - - /** - * WalTtlSeconds() and walSizeLimitMB() affect how archived logs - * will be deleted. - * 1. If both set to 0, logs will be deleted asap and will not get into - * the archive. - * 2. If WAL_ttl_seconds is 0 and WAL_size_limit_MB is not 0, - * WAL files will be checked every 10 min and if total size is greater - * then WAL_size_limit_MB, they will be deleted starting with the - * earliest until size_limit is met. All empty files will be deleted. - * 3. If WAL_ttl_seconds is not 0 and WAL_size_limit_MB is 0, then - * WAL files will be checked every WAL_ttl_secondsi / 2 and those that - * are older than WAL_ttl_seconds will be deleted. - * 4. If both are not 0, WAL files will be checked every 10 min and both - * checks will be performed with ttl being first. - * - * @param sizeLimitMB size limit in mega-bytes. - * @return the reference to the current option. - * @see setWalSizeLimitMB() - */ - public Options setWalSizeLimitMB(long sizeLimitMB) { - assert(isInitialized()); - setWalSizeLimitMB(nativeHandle_, sizeLimitMB); - return this; - } - private native void setWalSizeLimitMB(long handle, long sizeLimitMB); - - /** - * Number of bytes to preallocate (via fallocate) the manifest - * files. Default is 4mb, which is reasonable to reduce random IO - * as well as prevent overallocation for mounts that preallocate - * large amounts of data (such as xfs's allocsize option). - * - * @return size in bytes. - */ - public long manifestPreallocationSize() { - assert(isInitialized()); - return manifestPreallocationSize(nativeHandle_); - } - private native long manifestPreallocationSize(long handle); - - /** - * Number of bytes to preallocate (via fallocate) the manifest - * files. Default is 4mb, which is reasonable to reduce random IO - * as well as prevent overallocation for mounts that preallocate - * large amounts of data (such as xfs's allocsize option). - * - * @param size the size in byte - * @return the reference to the current option. - */ - public Options setManifestPreallocationSize(long size) { - assert(isInitialized()); - setManifestPreallocationSize(nativeHandle_, size); - return this; - } - private native void setManifestPreallocationSize( - long handle, long size); - - /** - * Data being read from file storage may be buffered in the OS - * Default: true - * - * @return if true, then OS buffering is allowed. - */ - public boolean allowOsBuffer() { - assert(isInitialized()); - return allowOsBuffer(nativeHandle_); - } - private native boolean allowOsBuffer(long handle); - - /** - * Data being read from file storage may be buffered in the OS - * Default: true - * - * @param allowOsBufferif true, then OS buffering is allowed. - * @return the reference to the current option. - */ - public Options setAllowOsBuffer(boolean allowOsBuffer) { - assert(isInitialized()); - setAllowOsBuffer(nativeHandle_, allowOsBuffer); - return this; - } - private native void setAllowOsBuffer( - long handle, boolean allowOsBuffer); - - /** - * Allow the OS to mmap file for reading sst tables. - * Default: false - * - * @return true if mmap reads are allowed. - */ - public boolean allowMmapReads() { - assert(isInitialized()); - return allowMmapReads(nativeHandle_); - } - private native boolean allowMmapReads(long handle); - - /** - * Allow the OS to mmap file for reading sst tables. - * Default: false - * - * @param allowMmapReads true if mmap reads are allowed. - * @return the reference to the current option. - */ - public Options setAllowMmapReads(boolean allowMmapReads) { - assert(isInitialized()); - setAllowMmapReads(nativeHandle_, allowMmapReads); - return this; - } - private native void setAllowMmapReads( - long handle, boolean allowMmapReads); - - /** - * Allow the OS to mmap file for writing. Default: false - * - * @return true if mmap writes are allowed. - */ - public boolean allowMmapWrites() { - assert(isInitialized()); - return allowMmapWrites(nativeHandle_); - } - private native boolean allowMmapWrites(long handle); - - /** - * Allow the OS to mmap file for writing. Default: false - * - * @param allowMmapWrites true if mmap writes are allowd. - * @return the reference to the current option. - */ - public Options setAllowMmapWrites(boolean allowMmapWrites) { - assert(isInitialized()); - setAllowMmapWrites(nativeHandle_, allowMmapWrites); - return this; - } - private native void setAllowMmapWrites( - long handle, boolean allowMmapWrites); - - /** - * Disable child process inherit open files. Default: true - * - * @return true if child process inheriting open files is disabled. - */ - public boolean isFdCloseOnExec() { - assert(isInitialized()); - return isFdCloseOnExec(nativeHandle_); - } - private native boolean isFdCloseOnExec(long handle); - - /** - * Disable child process inherit open files. Default: true - * - * @param isFdCloseOnExec true if child process inheriting open - * files is disabled. - * @return the reference to the current option. - */ - public Options setIsFdCloseOnExec(boolean isFdCloseOnExec) { - assert(isInitialized()); - setIsFdCloseOnExec(nativeHandle_, isFdCloseOnExec); - return this; - } - private native void setIsFdCloseOnExec( - long handle, boolean isFdCloseOnExec); - - /** - * Skip log corruption error on recovery (If client is ok with - * losing most recent changes) - * Default: false - * - * @return true if log corruption errors are skipped during recovery. - */ - public boolean skipLogErrorOnRecovery() { - assert(isInitialized()); - return skipLogErrorOnRecovery(nativeHandle_); - } - private native boolean skipLogErrorOnRecovery(long handle); - - /** - * Skip log corruption error on recovery (If client is ok with - * losing most recent changes) - * Default: false - * - * @param skip true if log corruption errors are skipped during recovery. - * @return the reference to the current option. - */ - public Options setSkipLogErrorOnRecovery(boolean skip) { - assert(isInitialized()); - setSkipLogErrorOnRecovery(nativeHandle_, skip); - return this; - } - private native void setSkipLogErrorOnRecovery( - long handle, boolean skip); - - /** - * If not zero, dump rocksdb.stats to LOG every stats_dump_period_sec - * Default: 3600 (1 hour) - * - * @return time interval in seconds. - */ - public int statsDumpPeriodSec() { - assert(isInitialized()); - return statsDumpPeriodSec(nativeHandle_); - } - private native int statsDumpPeriodSec(long handle); - - /** - * if not zero, dump rocksdb.stats to LOG every stats_dump_period_sec - * Default: 3600 (1 hour) - * - * @param statsDumpPeriodSec time interval in seconds. - * @return the reference to the current option. - */ - public Options setStatsDumpPeriodSec(int statsDumpPeriodSec) { - assert(isInitialized()); - setStatsDumpPeriodSec(nativeHandle_, statsDumpPeriodSec); - return this; - } - private native void setStatsDumpPeriodSec( - long handle, int statsDumpPeriodSec); - - /** - * If set true, will hint the underlying file system that the file - * access pattern is random, when a sst file is opened. - * Default: true - * - * @return true if hinting random access is on. - */ - public boolean adviseRandomOnOpen() { - return adviseRandomOnOpen(nativeHandle_); - } - private native boolean adviseRandomOnOpen(long handle); - - /** - * If set true, will hint the underlying file system that the file - * access pattern is random, when a sst file is opened. - * Default: true - * - * @param adviseRandomOnOpen true if hinting random access is on. - * @return the reference to the current option. - */ - public Options setAdviseRandomOnOpen(boolean adviseRandomOnOpen) { - assert(isInitialized()); - setAdviseRandomOnOpen(nativeHandle_, adviseRandomOnOpen); - return this; - } - private native void setAdviseRandomOnOpen( - long handle, boolean adviseRandomOnOpen); - - /** - * Use adaptive mutex, which spins in the user space before resorting - * to kernel. This could reduce context switch when the mutex is not - * heavily contended. However, if the mutex is hot, we could end up - * wasting spin time. - * Default: false - * - * @return true if adaptive mutex is used. - */ - public boolean useAdaptiveMutex() { - assert(isInitialized()); - return useAdaptiveMutex(nativeHandle_); - } - private native boolean useAdaptiveMutex(long handle); - - /** - * Use adaptive mutex, which spins in the user space before resorting - * to kernel. This could reduce context switch when the mutex is not - * heavily contended. However, if the mutex is hot, we could end up - * wasting spin time. - * Default: false - * - * @param useAdaptiveMutex true if adaptive mutex is used. - * @return the reference to the current option. - */ - public Options setUseAdaptiveMutex(boolean useAdaptiveMutex) { - assert(isInitialized()); - setUseAdaptiveMutex(nativeHandle_, useAdaptiveMutex); - return this; - } - private native void setUseAdaptiveMutex( - long handle, boolean useAdaptiveMutex); - - /** - * Allows OS to incrementally sync files to disk while they are being - * written, asynchronously, in the background. - * Issue one request for every bytes_per_sync written. 0 turns it off. - * Default: 0 - * - * @return size in bytes - */ - public long bytesPerSync() { - return bytesPerSync(nativeHandle_); - } - private native long bytesPerSync(long handle); - - /** - * Allows OS to incrementally sync files to disk while they are being - * written, asynchronously, in the background. - * Issue one request for every bytes_per_sync written. 0 turns it off. - * Default: 0 - * - * @param bytesPerSync size in bytes - * @return the reference to the current option. - */ - public Options setBytesPerSync(long bytesPerSync) { - assert(isInitialized()); - setBytesPerSync(nativeHandle_, bytesPerSync); - return this; - } - private native void setBytesPerSync( - long handle, long bytesPerSync); - - /** - * Allow RocksDB to use thread local storage to optimize performance. - * Default: true - * - * @return true if thread-local storage is allowed - */ - public boolean allowThreadLocal() { - assert(isInitialized()); - return allowThreadLocal(nativeHandle_); - } - private native boolean allowThreadLocal(long handle); - - /** - * Allow RocksDB to use thread local storage to optimize performance. - * Default: true - * - * @param allowThreadLocal true if thread-local storage is allowed. - * @return the reference to the current option. - */ - public Options setAllowThreadLocal(boolean allowThreadLocal) { - assert(isInitialized()); - setAllowThreadLocal(nativeHandle_, allowThreadLocal); - return this; - } - private native void setAllowThreadLocal( - long handle, boolean allowThreadLocal); - - /** - * Set the config for mem-table. - * - * @param config the mem-table config. - * @return the instance of the current Options. - */ - public Options setMemTableConfig(MemTableConfig config) { - setMemTableFactory(nativeHandle_, config.newMemTableFactoryHandle()); - return this; - } - - /** - * Returns the name of the current mem table representation. - * Memtable format can be set using setTableFormatConfig. - * - * @return the name of the currently-used memtable factory. - * @see setTableFormatConfig() - */ - public String memTableFactoryName() { - assert(isInitialized()); - return memTableFactoryName(nativeHandle_); - } - - /** - * Set the config for table format. - * - * @param config the table format config. - * @return the reference of the current Options. - */ - public Options setTableFormatConfig(TableFormatConfig config) { - setTableFactory(nativeHandle_, config.newTableFactoryHandle()); - return this; - } - - /** - * @return the name of the currently used table factory. - */ - public String tableFactoryName() { - assert(isInitialized()); - return tableFactoryName(nativeHandle_); - } - - /** - * This prefix-extractor uses the first n bytes of a key as its prefix. - * - * In some hash-based memtable representation such as HashLinkedList - * and HashSkipList, prefixes are used to partition the keys into - * several buckets. Prefix extractor is used to specify how to - * extract the prefix given a key. - * - * @param n use the first n bytes of a key as its prefix. - */ - public Options useFixedLengthPrefixExtractor(int n) { - assert(isInitialized()); - useFixedLengthPrefixExtractor(nativeHandle_, n); - return this; - } - -/////////////////////////////////////////////////////////////////////// - /** - * Number of keys between restart points for delta encoding of keys. - * This parameter can be changed dynamically. Most clients should - * leave this parameter alone. - * Default: 16 - * - * @return the number of keys between restart points. - */ - public int blockRestartInterval() { - return blockRestartInterval(nativeHandle_); - } - private native int blockRestartInterval(long handle); - - /** - * Number of keys between restart points for delta encoding of keys. - * This parameter can be changed dynamically. Most clients should - * leave this parameter alone. - * Default: 16 - * - * @param blockRestartInterval the number of keys between restart points. - * @return the reference to the current option. - */ - public Options setBlockRestartInterval(int blockRestartInterval) { - setBlockRestartInterval(nativeHandle_, blockRestartInterval); - return this; - } - private native void setBlockRestartInterval( - long handle, int blockRestartInterval); - - /** - * Compress blocks using the specified compression algorithm. This - parameter can be changed dynamically. - * - * Default: SNAPPY_COMPRESSION, which gives lightweight but fast compression. - * - * @return Compression type. - */ - public CompressionType compressionType() { - return CompressionType.values()[compressionType(nativeHandle_)]; - } - private native byte compressionType(long handle); - - /** - * Compress blocks using the specified compression algorithm. This - parameter can be changed dynamically. - * - * Default: SNAPPY_COMPRESSION, which gives lightweight but fast compression. - * - * @param compressionType Compression Type. - * @return the reference to the current option. - */ - public Options setCompressionType(CompressionType compressionType) { - setCompressionType(nativeHandle_, compressionType.getValue()); - return this; - } - private native void setCompressionType(long handle, byte compressionType); - - /** - * Compaction style for DB. - * - * @return Compaction style. - */ - public CompactionStyle compactionStyle() { - return CompactionStyle.values()[compactionStyle(nativeHandle_)]; - } - private native byte compactionStyle(long handle); - - /** - * Set compaction style for DB. - * - * Default: LEVEL. - * - * @param compactionStyle Compaction style. - * @return the reference to the current option. - */ - public Options setCompactionStyle(CompactionStyle compactionStyle) { - setCompactionStyle(nativeHandle_, compactionStyle.getValue()); - return this; - } - private native void setCompactionStyle(long handle, byte compactionStyle); - - /** - * If level-styled compaction is used, then this number determines - * the total number of levels. - * - * @return the number of levels. - */ - public int numLevels() { - return numLevels(nativeHandle_); - } - private native int numLevels(long handle); - - /** - * Set the number of levels for this database - * If level-styled compaction is used, then this number determines - * the total number of levels. - * - * @param numLevels the number of levels. - * @return the reference to the current option. - */ - public Options setNumLevels(int numLevels) { - setNumLevels(nativeHandle_, numLevels); - return this; - } - private native void setNumLevels( - long handle, int numLevels); - - /** - * The number of files in leve 0 to trigger compaction from level-0 to - * level-1. A value < 0 means that level-0 compaction will not be - * triggered by number of files at all. - * Default: 4 - * - * @return the number of files in level 0 to trigger compaction. - */ - public int levelZeroFileNumCompactionTrigger() { - return levelZeroFileNumCompactionTrigger(nativeHandle_); - } - private native int levelZeroFileNumCompactionTrigger(long handle); - - /** - * Number of files to trigger level-0 compaction. A value <0 means that - * level-0 compaction will not be triggered by number of files at all. - * Default: 4 - * - * @param numFiles the number of files in level-0 to trigger compaction. - * @return the reference to the current option. - */ - public Options setLevelZeroFileNumCompactionTrigger( - int numFiles) { - setLevelZeroFileNumCompactionTrigger( - nativeHandle_, numFiles); - return this; - } - private native void setLevelZeroFileNumCompactionTrigger( - long handle, int numFiles); - - /** - * Soft limit on the number of level-0 files. We start slowing down writes - * at this point. A value < 0 means that no writing slow down will be - * triggered by number of files in level-0. - * - * @return the soft limit on the number of level-0 files. - */ - public int levelZeroSlowdownWritesTrigger() { - return levelZeroSlowdownWritesTrigger(nativeHandle_); - } - private native int levelZeroSlowdownWritesTrigger(long handle); - - /** - * Soft limit on number of level-0 files. We start slowing down writes at this - * point. A value <0 means that no writing slow down will be triggered by - * number of files in level-0. - * - * @param numFiles soft limit on number of level-0 files. - * @return the reference to the current option. - */ - public Options setLevelZeroSlowdownWritesTrigger( - int numFiles) { - setLevelZeroSlowdownWritesTrigger(nativeHandle_, numFiles); - return this; - } - private native void setLevelZeroSlowdownWritesTrigger( - long handle, int numFiles); - - /** - * Maximum number of level-0 files. We stop writes at this point. - * - * @return the hard limit of the number of level-0 file. - */ - public int levelZeroStopWritesTrigger() { - return levelZeroStopWritesTrigger(nativeHandle_); - } - private native int levelZeroStopWritesTrigger(long handle); - - /** - * Maximum number of level-0 files. We stop writes at this point. - * - * @param numFiles the hard limit of the number of level-0 files. - * @return the reference to the current option. - */ - public Options setLevelZeroStopWritesTrigger(int numFiles) { - setLevelZeroStopWritesTrigger(nativeHandle_, numFiles); - return this; - } - private native void setLevelZeroStopWritesTrigger( - long handle, int numFiles); - - /** - * The highest level to which a new compacted memtable is pushed if it - * does not create overlap. We try to push to level 2 to avoid the - * relatively expensive level 0=>1 compactions and to avoid some - * expensive manifest file operations. We do not push all the way to - * the largest level since that can generate a lot of wasted disk - * space if the same key space is being repeatedly overwritten. - * - * @return the highest level where a new compacted memtable will be pushed. - */ - public int maxMemCompactionLevel() { - return maxMemCompactionLevel(nativeHandle_); - } - private native int maxMemCompactionLevel(long handle); - - /** - * The highest level to which a new compacted memtable is pushed if it - * does not create overlap. We try to push to level 2 to avoid the - * relatively expensive level 0=>1 compactions and to avoid some - * expensive manifest file operations. We do not push all the way to - * the largest level since that can generate a lot of wasted disk - * space if the same key space is being repeatedly overwritten. - * - * @param maxMemCompactionLevel the highest level to which a new compacted - * mem-table will be pushed. - * @return the reference to the current option. - */ - public Options setMaxMemCompactionLevel(int maxMemCompactionLevel) { - setMaxMemCompactionLevel(nativeHandle_, maxMemCompactionLevel); - return this; - } - private native void setMaxMemCompactionLevel( - long handle, int maxMemCompactionLevel); - - /** - * The target file size for compaction. - * This targetFileSizeBase determines a level-1 file size. - * Target file size for level L can be calculated by - * targetFileSizeBase * (targetFileSizeMultiplier ^ (L-1)) - * For example, if targetFileSizeBase is 2MB and - * target_file_size_multiplier is 10, then each file on level-1 will - * be 2MB, and each file on level 2 will be 20MB, - * and each file on level-3 will be 200MB. - * by default targetFileSizeBase is 2MB. - * - * @return the target size of a level-0 file. - * - * @see targetFileSizeMultiplier() - */ - public int targetFileSizeBase() { - return targetFileSizeBase(nativeHandle_); - } - private native int targetFileSizeBase(long handle); - - /** - * The target file size for compaction. - * This targetFileSizeBase determines a level-1 file size. - * Target file size for level L can be calculated by - * targetFileSizeBase * (targetFileSizeMultiplier ^ (L-1)) - * For example, if targetFileSizeBase is 2MB and - * target_file_size_multiplier is 10, then each file on level-1 will - * be 2MB, and each file on level 2 will be 20MB, - * and each file on level-3 will be 200MB. - * by default targetFileSizeBase is 2MB. - * - * @param targetFileSizeBase the target size of a level-0 file. - * @return the reference to the current option. - * - * @see setTargetFileSizeMultiplier() - */ - public Options setTargetFileSizeBase(int targetFileSizeBase) { - setTargetFileSizeBase(nativeHandle_, targetFileSizeBase); - return this; - } - private native void setTargetFileSizeBase( - long handle, int targetFileSizeBase); - - /** - * targetFileSizeMultiplier defines the size ratio between a - * level-(L+1) file and level-L file. - * By default targetFileSizeMultiplier is 1, meaning - * files in different levels have the same target. - * - * @return the size ratio between a level-(L+1) file and level-L file. - */ - public int targetFileSizeMultiplier() { - return targetFileSizeMultiplier(nativeHandle_); - } - private native int targetFileSizeMultiplier(long handle); - - /** - * targetFileSizeMultiplier defines the size ratio between a - * level-L file and level-(L+1) file. - * By default target_file_size_multiplier is 1, meaning - * files in different levels have the same target. - * - * @param multiplier the size ratio between a level-(L+1) file - * and level-L file. - * @return the reference to the current option. - */ - public Options setTargetFileSizeMultiplier(int multiplier) { - setTargetFileSizeMultiplier(nativeHandle_, multiplier); - return this; - } - private native void setTargetFileSizeMultiplier( - long handle, int multiplier); - - /** - * The upper-bound of the total size of level-1 files in bytes. - * Maximum number of bytes for level L can be calculated as - * (maxBytesForLevelBase) * (maxBytesForLevelMultiplier ^ (L-1)) - * For example, if maxBytesForLevelBase is 20MB, and if - * max_bytes_for_level_multiplier is 10, total data size for level-1 - * will be 20MB, total file size for level-2 will be 200MB, - * and total file size for level-3 will be 2GB. - * by default 'maxBytesForLevelBase' is 10MB. - * - * @return the upper-bound of the total size of leve-1 files in bytes. - * @see maxBytesForLevelMultiplier() - */ - public long maxBytesForLevelBase() { - return maxBytesForLevelBase(nativeHandle_); - } - private native long maxBytesForLevelBase(long handle); - - /** - * The upper-bound of the total size of level-1 files in bytes. - * Maximum number of bytes for level L can be calculated as - * (maxBytesForLevelBase) * (maxBytesForLevelMultiplier ^ (L-1)) - * For example, if maxBytesForLevelBase is 20MB, and if - * max_bytes_for_level_multiplier is 10, total data size for level-1 - * will be 20MB, total file size for level-2 will be 200MB, - * and total file size for level-3 will be 2GB. - * by default 'maxBytesForLevelBase' is 10MB. - * - * @return maxBytesForLevelBase the upper-bound of the total size of - * leve-1 files in bytes. - * @return the reference to the current option. - * @see setMaxBytesForLevelMultiplier() - */ - public Options setMaxBytesForLevelBase(long maxBytesForLevelBase) { - setMaxBytesForLevelBase(nativeHandle_, maxBytesForLevelBase); - return this; - } - private native void setMaxBytesForLevelBase( - long handle, long maxBytesForLevelBase); - - /** - * The ratio between the total size of level-(L+1) files and the total - * size of level-L files for all L. - * DEFAULT: 10 - * - * @return the ratio between the total size of level-(L+1) files and - * the total size of level-L files for all L. - * @see maxBytesForLevelBase() - */ - public int maxBytesForLevelMultiplier() { - return maxBytesForLevelMultiplier(nativeHandle_); - } - private native int maxBytesForLevelMultiplier(long handle); - - /** - * The ratio between the total size of level-(L+1) files and the total - * size of level-L files for all L. - * DEFAULT: 10 - * - * @param multiplier the ratio between the total size of level-(L+1) - * files and the total size of level-L files for all L. - * @return the reference to the current option. - * @see setMaxBytesForLevelBase() - */ - public Options setMaxBytesForLevelMultiplier(int multiplier) { - setMaxBytesForLevelMultiplier(nativeHandle_, multiplier); - return this; - } - private native void setMaxBytesForLevelMultiplier( - long handle, int multiplier); - - /** - * Maximum number of bytes in all compacted files. We avoid expanding - * the lower level file set of a compaction if it would make the - * total compaction cover more than - * (expanded_compaction_factor * targetFileSizeLevel()) many bytes. - * - * @return the maximum number of bytes in all compacted files. - * @see sourceCompactionFactor() - */ - public int expandedCompactionFactor() { - return expandedCompactionFactor(nativeHandle_); - } - private native int expandedCompactionFactor(long handle); - - /** - * Maximum number of bytes in all compacted files. We avoid expanding - * the lower level file set of a compaction if it would make the - * total compaction cover more than - * (expanded_compaction_factor * targetFileSizeLevel()) many bytes. - * - * @param expandedCompactionFactor the maximum number of bytes in all - * compacted files. - * @return the reference to the current option. - * @see setSourceCompactionFactor() - */ - public Options setExpandedCompactionFactor(int expandedCompactionFactor) { - setExpandedCompactionFactor(nativeHandle_, expandedCompactionFactor); - return this; - } - private native void setExpandedCompactionFactor( - long handle, int expandedCompactionFactor); - - /** - * Maximum number of bytes in all source files to be compacted in a - * single compaction run. We avoid picking too many files in the - * source level so that we do not exceed the total source bytes - * for compaction to exceed - * (source_compaction_factor * targetFileSizeLevel()) many bytes. - * Default:1, i.e. pick maxfilesize amount of data as the source of - * a compaction. - * - * @return the maximum number of bytes in all source files to be compactedo. - * @see expendedCompactionFactor() - */ - public int sourceCompactionFactor() { - return sourceCompactionFactor(nativeHandle_); - } - private native int sourceCompactionFactor(long handle); - - /** - * Maximum number of bytes in all source files to be compacted in a - * single compaction run. We avoid picking too many files in the - * source level so that we do not exceed the total source bytes - * for compaction to exceed - * (source_compaction_factor * targetFileSizeLevel()) many bytes. - * Default:1, i.e. pick maxfilesize amount of data as the source of - * a compaction. - * - * @param sourceCompactionFactor the maximum number of bytes in all - * source files to be compacted in a single compaction run. - * @return the reference to the current option. - * @see setExpendedCompactionFactor() - */ - public Options setSourceCompactionFactor(int sourceCompactionFactor) { - setSourceCompactionFactor(nativeHandle_, sourceCompactionFactor); - return this; - } - private native void setSourceCompactionFactor( - long handle, int sourceCompactionFactor); - - /** - * Control maximum bytes of overlaps in grandparent (i.e., level+2) before we - * stop building a single file in a level->level+1 compaction. - * - * @return maximum bytes of overlaps in "grandparent" level. - */ - public int maxGrandparentOverlapFactor() { - return maxGrandparentOverlapFactor(nativeHandle_); - } - private native int maxGrandparentOverlapFactor(long handle); - - /** - * Control maximum bytes of overlaps in grandparent (i.e., level+2) before we - * stop building a single file in a level->level+1 compaction. - * - * @param maxGrandparentOverlapFactor maximum bytes of overlaps in - * "grandparent" level. - * @return the reference to the current option. - */ - public Options setMaxGrandparentOverlapFactor( - int maxGrandparentOverlapFactor) { - setMaxGrandparentOverlapFactor(nativeHandle_, maxGrandparentOverlapFactor); - return this; - } - private native void setMaxGrandparentOverlapFactor( - long handle, int maxGrandparentOverlapFactor); - - /** - * Puts are delayed 0-1 ms when any level has a compaction score that exceeds - * soft_rate_limit. This is ignored when == 0.0. - * CONSTRAINT: soft_rate_limit <= hard_rate_limit. If this constraint does not - * hold, RocksDB will set soft_rate_limit = hard_rate_limit - * Default: 0 (disabled) - * - * @return soft-rate-limit for put delay. - */ - public double softRateLimit() { - return softRateLimit(nativeHandle_); - } - private native double softRateLimit(long handle); - - /** - * Puts are delayed 0-1 ms when any level has a compaction score that exceeds - * soft_rate_limit. This is ignored when == 0.0. - * CONSTRAINT: soft_rate_limit <= hard_rate_limit. If this constraint does not - * hold, RocksDB will set soft_rate_limit = hard_rate_limit - * Default: 0 (disabled) - * - * @param softRateLimit the soft-rate-limit of a compaction score - * for put delay. - * @return the reference to the current option. - */ - public Options setSoftRateLimit(double softRateLimit) { - setSoftRateLimit(nativeHandle_, softRateLimit); - return this; - } - private native void setSoftRateLimit( - long handle, double softRateLimit); - - /** - * Puts are delayed 1ms at a time when any level has a compaction score that - * exceeds hard_rate_limit. This is ignored when <= 1.0. - * Default: 0 (disabled) - * - * @return the hard-rate-limit of a compaction score for put delay. - */ - public double hardRateLimit() { - return hardRateLimit(nativeHandle_); - } - private native double hardRateLimit(long handle); - - /** - * Puts are delayed 1ms at a time when any level has a compaction score that - * exceeds hard_rate_limit. This is ignored when <= 1.0. - * Default: 0 (disabled) - * - * @param hardRateLimit the hard-rate-limit of a compaction score for put - * delay. - * @return the reference to the current option. - */ - public Options setHardRateLimit(double hardRateLimit) { - setHardRateLimit(nativeHandle_, hardRateLimit); - return this; - } - private native void setHardRateLimit( - long handle, double hardRateLimit); - - /** - * The maximum time interval a put will be stalled when hard_rate_limit - * is enforced. If 0, then there is no limit. - * Default: 1000 - * - * @return the maximum time interval a put will be stalled when - * hard_rate_limit is enforced. - */ - public int rateLimitDelayMaxMilliseconds() { - return rateLimitDelayMaxMilliseconds(nativeHandle_); - } - private native int rateLimitDelayMaxMilliseconds(long handle); - - /** - * The maximum time interval a put will be stalled when hard_rate_limit - * is enforced. If 0, then there is no limit. - * Default: 1000 - * - * @param rateLimitDelayMaxMilliseconds the maximum time interval a put - * will be stalled. - * @return the reference to the current option. - */ - public Options setRateLimitDelayMaxMilliseconds( - int rateLimitDelayMaxMilliseconds) { - setRateLimitDelayMaxMilliseconds( - nativeHandle_, rateLimitDelayMaxMilliseconds); - return this; - } - private native void setRateLimitDelayMaxMilliseconds( - long handle, int rateLimitDelayMaxMilliseconds); - - /** - * The size of one block in arena memory allocation. - * If <= 0, a proper value is automatically calculated (usually 1/10 of - * writer_buffer_size). - * - * There are two additonal restriction of the The specified size: - * (1) size should be in the range of [4096, 2 << 30] and - * (2) be the multiple of the CPU word (which helps with the memory - * alignment). - * - * We'll automatically check and adjust the size number to make sure it - * conforms to the restrictions. - * Default: 0 - * - * @return the size of an arena block - */ - public long arenaBlockSize() { - return arenaBlockSize(nativeHandle_); - } - private native long arenaBlockSize(long handle); - - /** - * The size of one block in arena memory allocation. - * If <= 0, a proper value is automatically calculated (usually 1/10 of - * writer_buffer_size). - * - * There are two additonal restriction of the The specified size: - * (1) size should be in the range of [4096, 2 << 30] and - * (2) be the multiple of the CPU word (which helps with the memory - * alignment). - * - * We'll automatically check and adjust the size number to make sure it - * conforms to the restrictions. - * Default: 0 - * - * @param arenaBlockSize the size of an arena block - * @return the reference to the current option. - */ - public Options setArenaBlockSize(long arenaBlockSize) { - setArenaBlockSize(nativeHandle_, arenaBlockSize); - return this; - } - private native void setArenaBlockSize( - long handle, long arenaBlockSize); - - /** - * Disable automatic compactions. Manual compactions can still - * be issued on this column family - * - * @return true if auto-compactions are disabled. - */ - public boolean disableAutoCompactions() { - return disableAutoCompactions(nativeHandle_); - } - private native boolean disableAutoCompactions(long handle); - - /** - * Disable automatic compactions. Manual compactions can still - * be issued on this column family - * - * @param disableAutoCompactions true if auto-compactions are disabled. - * @return the reference to the current option. - */ - public Options setDisableAutoCompactions(boolean disableAutoCompactions) { - setDisableAutoCompactions(nativeHandle_, disableAutoCompactions); - return this; - } - private native void setDisableAutoCompactions( - long handle, boolean disableAutoCompactions); - - /** - * Purge duplicate/deleted keys when a memtable is flushed to storage. - * Default: true - * - * @return true if purging keys is disabled. - */ - public boolean purgeRedundantKvsWhileFlush() { - return purgeRedundantKvsWhileFlush(nativeHandle_); - } - private native boolean purgeRedundantKvsWhileFlush(long handle); - - /** - * Purge duplicate/deleted keys when a memtable is flushed to storage. - * Default: true - * - * @param purgeRedundantKvsWhileFlush true if purging keys is disabled. - * @return the reference to the current option. - */ - public Options setPurgeRedundantKvsWhileFlush( - boolean purgeRedundantKvsWhileFlush) { - setPurgeRedundantKvsWhileFlush( - nativeHandle_, purgeRedundantKvsWhileFlush); - return this; - } - private native void setPurgeRedundantKvsWhileFlush( - long handle, boolean purgeRedundantKvsWhileFlush); - - /** - * If true, compaction will verify checksum on every read that happens - * as part of compaction - * Default: true - * - * @return true if compaction verifies checksum on every read. - */ - public boolean verifyChecksumsInCompaction() { - return verifyChecksumsInCompaction(nativeHandle_); - } - private native boolean verifyChecksumsInCompaction(long handle); - - /** - * If true, compaction will verify checksum on every read that happens - * as part of compaction - * Default: true - * - * @param verifyChecksumsInCompaction true if compaction verifies - * checksum on every read. - * @return the reference to the current option. - */ - public Options setVerifyChecksumsInCompaction( - boolean verifyChecksumsInCompaction) { - setVerifyChecksumsInCompaction( - nativeHandle_, verifyChecksumsInCompaction); - return this; - } - private native void setVerifyChecksumsInCompaction( - long handle, boolean verifyChecksumsInCompaction); - - /** - * Use KeyMayExist API to filter deletes when this is true. - * If KeyMayExist returns false, i.e. the key definitely does not exist, then - * the delete is a noop. KeyMayExist only incurs in-memory look up. - * This optimization avoids writing the delete to storage when appropriate. - * Default: false - * - * @return true if filter-deletes behavior is on. - */ - public boolean filterDeletes() { - return filterDeletes(nativeHandle_); - } - private native boolean filterDeletes(long handle); - - /** - * Use KeyMayExist API to filter deletes when this is true. - * If KeyMayExist returns false, i.e. the key definitely does not exist, then - * the delete is a noop. KeyMayExist only incurs in-memory look up. - * This optimization avoids writing the delete to storage when appropriate. - * Default: false - * - * @param filterDeletes true if filter-deletes behavior is on. - * @return the reference to the current option. - */ - public Options setFilterDeletes(boolean filterDeletes) { - setFilterDeletes(nativeHandle_, filterDeletes); - return this; - } - private native void setFilterDeletes( - long handle, boolean filterDeletes); - - /** - * An iteration->Next() sequentially skips over keys with the same - * user-key unless this option is set. This number specifies the number - * of keys (with the same userkey) that will be sequentially - * skipped before a reseek is issued. - * Default: 8 - * - * @return the number of keys could be skipped in a iteration. - */ - public long maxSequentialSkipInIterations() { - return maxSequentialSkipInIterations(nativeHandle_); - } - private native long maxSequentialSkipInIterations(long handle); - - /** - * An iteration->Next() sequentially skips over keys with the same - * user-key unless this option is set. This number specifies the number - * of keys (with the same userkey) that will be sequentially - * skipped before a reseek is issued. - * Default: 8 - * - * @param maxSequentialSkipInIterations the number of keys could - * be skipped in a iteration. - * @return the reference to the current option. - */ - public Options setMaxSequentialSkipInIterations(long maxSequentialSkipInIterations) { - setMaxSequentialSkipInIterations(nativeHandle_, maxSequentialSkipInIterations); - return this; - } - private native void setMaxSequentialSkipInIterations( - long handle, long maxSequentialSkipInIterations); - - /** - * Allows thread-safe inplace updates. - * If inplace_callback function is not set, - * Put(key, new_value) will update inplace the existing_value iff - * * key exists in current memtable - * * new sizeof(new_value) <= sizeof(existing_value) - * * existing_value for that key is a put i.e. kTypeValue - * If inplace_callback function is set, check doc for inplace_callback. - * Default: false. - * - * @return true if thread-safe inplace updates are allowed. - */ - public boolean inplaceUpdateSupport() { - return inplaceUpdateSupport(nativeHandle_); - } - private native boolean inplaceUpdateSupport(long handle); - - /** - * Allows thread-safe inplace updates. - * If inplace_callback function is not set, - * Put(key, new_value) will update inplace the existing_value iff - * * key exists in current memtable - * * new sizeof(new_value) <= sizeof(existing_value) - * * existing_value for that key is a put i.e. kTypeValue - * If inplace_callback function is set, check doc for inplace_callback. - * Default: false. - * - * @param inplaceUpdateSupport true if thread-safe inplace updates - * are allowed. - * @return the reference to the current option. - */ - public Options setInplaceUpdateSupport(boolean inplaceUpdateSupport) { - setInplaceUpdateSupport(nativeHandle_, inplaceUpdateSupport); - return this; - } - private native void setInplaceUpdateSupport( - long handle, boolean inplaceUpdateSupport); - - /** - * Number of locks used for inplace update - * Default: 10000, if inplace_update_support = true, else 0. - * - * @return the number of locks used for inplace update. - */ - public long inplaceUpdateNumLocks() { - return inplaceUpdateNumLocks(nativeHandle_); - } - private native long inplaceUpdateNumLocks(long handle); - - /** - * Number of locks used for inplace update - * Default: 10000, if inplace_update_support = true, else 0. - * - * @param inplaceUpdateNumLocks the number of locks used for - * inplace updates. - * @return the reference to the current option. - */ - public Options setInplaceUpdateNumLocks(long inplaceUpdateNumLocks) { - setInplaceUpdateNumLocks(nativeHandle_, inplaceUpdateNumLocks); - return this; - } - private native void setInplaceUpdateNumLocks( - long handle, long inplaceUpdateNumLocks); - - /** - * Returns the number of bits used in the prefix bloom filter. - * - * This value will be used only when a prefix-extractor is specified. - * - * @return the number of bloom-bits. - * @see useFixedLengthPrefixExtractor() - */ - public int memtablePrefixBloomBits() { - return memtablePrefixBloomBits(nativeHandle_); - } - private native int memtablePrefixBloomBits(long handle); - - /** - * Sets the number of bits used in the prefix bloom filter. - * - * This value will be used only when a prefix-extractor is specified. - * - * @param memtablePrefixBloomBits the number of bits used in the - * prefix bloom filter. - * @return the reference to the current option. - */ - public Options setMemtablePrefixBloomBits(int memtablePrefixBloomBits) { - setMemtablePrefixBloomBits(nativeHandle_, memtablePrefixBloomBits); - return this; - } - private native void setMemtablePrefixBloomBits( - long handle, int memtablePrefixBloomBits); - - /** - * The number of hash probes per key used in the mem-table. - * - * @return the number of hash probes per key. - */ - public int memtablePrefixBloomProbes() { - return memtablePrefixBloomProbes(nativeHandle_); - } - private native int memtablePrefixBloomProbes(long handle); - - /** - * The number of hash probes per key used in the mem-table. - * - * @param memtablePrefixBloomProbes the number of hash probes per key. - * @return the reference to the current option. - */ - public Options setMemtablePrefixBloomProbes(int memtablePrefixBloomProbes) { - setMemtablePrefixBloomProbes(nativeHandle_, memtablePrefixBloomProbes); - return this; - } - private native void setMemtablePrefixBloomProbes( - long handle, int memtablePrefixBloomProbes); - - /** - * Control locality of bloom filter probes to improve cache miss rate. - * This option only applies to memtable prefix bloom and plaintable - * prefix bloom. It essentially limits the max number of cache lines each - * bloom filter check can touch. - * This optimization is turned off when set to 0. The number should never - * be greater than number of probes. This option can boost performance - * for in-memory workload but should use with care since it can cause - * higher false positive rate. - * Default: 0 - * - * @return the level of locality of bloom-filter probes. - * @see setMemTablePrefixBloomProbes - */ - public int bloomLocality() { - return bloomLocality(nativeHandle_); - } - private native int bloomLocality(long handle); - - /** - * Control locality of bloom filter probes to improve cache miss rate. - * This option only applies to memtable prefix bloom and plaintable - * prefix bloom. It essentially limits the max number of cache lines each - * bloom filter check can touch. - * This optimization is turned off when set to 0. The number should never - * be greater than number of probes. This option can boost performance - * for in-memory workload but should use with care since it can cause - * higher false positive rate. - * Default: 0 - * - * @param bloomLocality the level of locality of bloom-filter probes. - * @return the reference to the current option. - */ - public Options setBloomLocality(int bloomLocality) { - setBloomLocality(nativeHandle_, bloomLocality); - return this; - } - private native void setBloomLocality( - long handle, int bloomLocality); - - /** - * Maximum number of successive merge operations on a key in the memtable. - * - * When a merge operation is added to the memtable and the maximum number of - * successive merges is reached, the value of the key will be calculated and - * inserted into the memtable instead of the merge operation. This will - * ensure that there are never more than max_successive_merges merge - * operations in the memtable. - * - * Default: 0 (disabled) - * - * @return the maximum number of successive merges. - */ - public long maxSuccessiveMerges() { - return maxSuccessiveMerges(nativeHandle_); - } - private native long maxSuccessiveMerges(long handle); - - /** - * Maximum number of successive merge operations on a key in the memtable. - * - * When a merge operation is added to the memtable and the maximum number of - * successive merges is reached, the value of the key will be calculated and - * inserted into the memtable instead of the merge operation. This will - * ensure that there are never more than max_successive_merges merge - * operations in the memtable. - * - * Default: 0 (disabled) - * - * @param maxSuccessiveMerges the maximum number of successive merges. - * @return the reference to the current option. - */ - public Options setMaxSuccessiveMerges(long maxSuccessiveMerges) { - setMaxSuccessiveMerges(nativeHandle_, maxSuccessiveMerges); - return this; - } - private native void setMaxSuccessiveMerges( - long handle, long maxSuccessiveMerges); - - /** - * The minimum number of write buffers that will be merged together - * before writing to storage. If set to 1, then - * all write buffers are fushed to L0 as individual files and this increases - * read amplification because a get request has to check in all of these - * files. Also, an in-memory merge may result in writing lesser - * data to storage if there are duplicate records in each of these - * individual write buffers. Default: 1 - * - * @return the minimum number of write buffers that will be merged together. - */ - public int minWriteBufferNumberToMerge() { - return minWriteBufferNumberToMerge(nativeHandle_); - } - private native int minWriteBufferNumberToMerge(long handle); - - /** - * The minimum number of write buffers that will be merged together - * before writing to storage. If set to 1, then - * all write buffers are fushed to L0 as individual files and this increases - * read amplification because a get request has to check in all of these - * files. Also, an in-memory merge may result in writing lesser - * data to storage if there are duplicate records in each of these - * individual write buffers. Default: 1 - * - * @param minWriteBufferNumberToMerge the minimum number of write buffers - * that will be merged together. - * @return the reference to the current option. - */ - public Options setMinWriteBufferNumberToMerge(int minWriteBufferNumberToMerge) { - setMinWriteBufferNumberToMerge(nativeHandle_, minWriteBufferNumberToMerge); - return this; - } - private native void setMinWriteBufferNumberToMerge( - long handle, int minWriteBufferNumberToMerge); - - /** - * The number of partial merge operands to accumulate before partial - * merge will be performed. Partial merge will not be called - * if the list of values to merge is less than min_partial_merge_operands. - * - * If min_partial_merge_operands < 2, then it will be treated as 2. - * - * Default: 2 - * - * @return - */ - public int minPartialMergeOperands() { - return minPartialMergeOperands(nativeHandle_); - } - private native int minPartialMergeOperands(long handle); - - /** - * The number of partial merge operands to accumulate before partial - * merge will be performed. Partial merge will not be called - * if the list of values to merge is less than min_partial_merge_operands. - * - * If min_partial_merge_operands < 2, then it will be treated as 2. - * - * Default: 2 - * - * @param minPartialMergeOperands - * @return the reference to the current option. - */ - public Options setMinPartialMergeOperands(int minPartialMergeOperands) { - setMinPartialMergeOperands(nativeHandle_, minPartialMergeOperands); - return this; - } - private native void setMinPartialMergeOperands( - long handle, int minPartialMergeOperands); - - /** - * Release the memory allocated for the current instance - * in the c++ side. - */ - @Override protected void disposeInternal() { - assert(isInitialized()); - disposeInternal(nativeHandle_); - } - - static final int DEFAULT_PLAIN_TABLE_BLOOM_BITS_PER_KEY = 10; - static final double DEFAULT_PLAIN_TABLE_HASH_TABLE_RATIO = 0.75; - static final int DEFAULT_PLAIN_TABLE_INDEX_SPARSENESS = 16; - - private native void newOptions(); - private native void disposeInternal(long handle); - private native void setCreateIfMissing(long handle, boolean flag); - private native boolean createIfMissing(long handle); - private native void setWriteBufferSize(long handle, long writeBufferSize); - private native long writeBufferSize(long handle); - private native void setMaxWriteBufferNumber( - long handle, int maxWriteBufferNumber); - private native int maxWriteBufferNumber(long handle); - private native void setMaxBackgroundCompactions( - long handle, int maxBackgroundCompactions); - private native int maxBackgroundCompactions(long handle); - private native void createStatistics(long optHandle); - private native long statisticsPtr(long optHandle); - - private native void setMemTableFactory(long handle, long factoryHandle); - private native String memTableFactoryName(long handle); - - private native void setTableFactory(long handle, long factoryHandle); - private native String tableFactoryName(long handle); - - private native void useFixedLengthPrefixExtractor( - long handle, int prefixLength); - - long cacheSize_; - int numShardBits_; - RocksEnv env_; -} diff --git a/java/org/rocksdb/PlainTableConfig.java b/java/org/rocksdb/PlainTableConfig.java deleted file mode 100644 index 554ce3840dc..00000000000 --- a/java/org/rocksdb/PlainTableConfig.java +++ /dev/null @@ -1,123 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -package org.rocksdb; - -/** - * The config for plain table sst format. - * - * PlainTable is a RocksDB's SST file format optimized for low query latency - * on pure-memory or really low-latency media. It also support prefix - * hash feature. - */ -public class PlainTableConfig extends TableFormatConfig { - public static final int VARIABLE_LENGTH = 0; - public static final int DEFAULT_BLOOM_BITS_PER_KEY = 10; - public static final double DEFAULT_HASH_TABLE_RATIO = 0.75; - public static final int DEFAULT_INDEX_SPARSENESS = 16; - - public PlainTableConfig() { - keySize_ = VARIABLE_LENGTH; - bloomBitsPerKey_ = DEFAULT_BLOOM_BITS_PER_KEY; - hashTableRatio_ = DEFAULT_HASH_TABLE_RATIO; - indexSparseness_ = DEFAULT_INDEX_SPARSENESS; - } - - /** - * Set the length of the user key. If it is set to be VARIABLE_LENGTH, - * then it indicates the user keys are variable-lengthed. Otherwise, - * all the keys need to have the same length in byte. - * DEFAULT: VARIABLE_LENGTH - * - * @param keySize the length of the user key. - * @return the reference to the current config. - */ - public PlainTableConfig setKeySize(int keySize) { - keySize_ = keySize; - return this; - } - - /** - * @return the specified size of the user key. If VARIABLE_LENGTH, - * then it indicates variable-length key. - */ - public int keySize() { - return keySize_; - } - - /** - * Set the number of bits per key used by the internal bloom filter - * in the plain table sst format. - * - * @param bitsPerKey the number of bits per key for bloom filer. - * @return the reference to the current config. - */ - public PlainTableConfig setBloomBitsPerKey(int bitsPerKey) { - bloomBitsPerKey_ = bitsPerKey; - return this; - } - - /** - * @return the number of bits per key used for the bloom filter. - */ - public int bloomBitsPerKey() { - return bloomBitsPerKey_; - } - - /** - * hashTableRatio is the desired utilization of the hash table used - * for prefix hashing. The ideal ratio would be the number of - * prefixes / the number of hash buckets. If this value is set to - * zero, then hash table will not be used. - * - * @param ratio the hash table ratio. - * @return the reference to the current config. - */ - public PlainTableConfig setHashTableRatio(double ratio) { - hashTableRatio_ = ratio; - return this; - } - - /** - * @return the hash table ratio. - */ - public double hashTableRatio() { - return hashTableRatio_; - } - - /** - * Index sparseness determines the index interval for keys inside the - * same prefix. This number is equal to the maximum number of linear - * search required after hash and binary search. If it's set to 0, - * then each key will be indexed. - * - * @param sparseness the index sparseness. - * @return the reference to the current config. - */ - public PlainTableConfig setIndexSparseness(int sparseness) { - indexSparseness_ = sparseness; - return this; - } - - /** - * @return the index sparseness. - */ - public int indexSparseness() { - return indexSparseness_; - } - - @Override protected long newTableFactoryHandle() { - return newTableFactoryHandle(keySize_, bloomBitsPerKey_, - hashTableRatio_, indexSparseness_); - } - - private native long newTableFactoryHandle( - int keySize, int bloomBitsPerKey, - double hashTableRatio, int indexSparseness); - - private int keySize_; - private int bloomBitsPerKey_; - private double hashTableRatio_; - private int indexSparseness_; -} diff --git a/java/org/rocksdb/ReadOptions.java b/java/org/rocksdb/ReadOptions.java deleted file mode 100644 index 97c47c7d625..00000000000 --- a/java/org/rocksdb/ReadOptions.java +++ /dev/null @@ -1,125 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * The class that controls the get behavior. - * - * Note that dispose() must be called before an Options instance - * become out-of-scope to release the allocated memory in c++. - */ -public class ReadOptions extends RocksObject { - public ReadOptions() { - super(); - newReadOptions(); - } - private native void newReadOptions(); - - /** - * If true, all data read from underlying storage will be - * verified against corresponding checksums. - * Default: true - * - * @return true if checksum verification is on. - */ - public boolean verifyChecksums() { - assert(isInitialized()); - return verifyChecksums(nativeHandle_); - } - private native boolean verifyChecksums(long handle); - - /** - * If true, all data read from underlying storage will be - * verified against corresponding checksums. - * Default: true - * - * @param verifyChecksums if true, then checksum verification - * will be performed on every read. - * @return the reference to the current ReadOptions. - */ - public ReadOptions setVerifyChecksums(boolean verifyChecksums) { - assert(isInitialized()); - setVerifyChecksums(nativeHandle_, verifyChecksums); - return this; - } - private native void setVerifyChecksums( - long handle, boolean verifyChecksums); - - // TODO(yhchiang): this option seems to be block-based table only. - // move this to a better place? - /** - * Fill the cache when loading the block-based sst formated db. - * Callers may wish to set this field to false for bulk scans. - * Default: true - * - * @return true if the fill-cache behavior is on. - */ - public boolean fillCache() { - assert(isInitialized()); - return fillCache(nativeHandle_); - } - private native boolean fillCache(long handle); - - /** - * Fill the cache when loading the block-based sst formated db. - * Callers may wish to set this field to false for bulk scans. - * Default: true - * - * @param fillCache if true, then fill-cache behavior will be - * performed. - * @return the reference to the current ReadOptions. - */ - public ReadOptions setFillCache(boolean fillCache) { - assert(isInitialized()); - setFillCache(nativeHandle_, fillCache); - return this; - } - private native void setFillCache( - long handle, boolean fillCache); - - /** - * Specify to create a tailing iterator -- a special iterator that has a - * view of the complete database (i.e. it can also be used to read newly - * added data) and is optimized for sequential reads. It will return records - * that were inserted into the database after the creation of the iterator. - * Default: false - * Not supported in ROCKSDB_LITE mode! - * - * @return true if tailing iterator is enabled. - */ - public boolean tailing() { - assert(isInitialized()); - return tailing(nativeHandle_); - } - private native boolean tailing(long handle); - - /** - * Specify to create a tailing iterator -- a special iterator that has a - * view of the complete database (i.e. it can also be used to read newly - * added data) and is optimized for sequential reads. It will return records - * that were inserted into the database after the creation of the iterator. - * Default: false - * Not supported in ROCKSDB_LITE mode! - * - * @param tailing if true, then tailing iterator will be enabled. - * @return the reference to the current ReadOptions. - */ - public ReadOptions setTailing(boolean tailing) { - assert(isInitialized()); - setTailing(nativeHandle_, tailing); - return this; - } - private native void setTailing( - long handle, boolean tailing); - - - @Override protected void disposeInternal() { - assert(isInitialized()); - disposeInternal(nativeHandle_); - } - private native void disposeInternal(long handle); - -} diff --git a/java/org/rocksdb/RestoreBackupableDB.java b/java/org/rocksdb/RestoreBackupableDB.java deleted file mode 100644 index dbde447a0f8..00000000000 --- a/java/org/rocksdb/RestoreBackupableDB.java +++ /dev/null @@ -1,84 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * This class is used to access information about backups and restore from them. - * - * Note that dispose() must be called before this instance become out-of-scope - * to release the allocated memory in c++. - * - * @param options Instance of BackupableDBOptions. - */ -public class RestoreBackupableDB extends RocksObject { - public RestoreBackupableDB(BackupableDBOptions options) { - super(); - nativeHandle_ = newRestoreBackupableDB(options.nativeHandle_); - } - - /** - * Restore from backup with backup_id - * IMPORTANT -- if options_.share_table_files == true and you restore DB - * from some backup that is not the latest, and you start creating new - * backups from the new DB, they will probably fail. - * - * Example: Let's say you have backups 1, 2, 3, 4, 5 and you restore 3. - * If you add new data to the DB and try creating a new backup now, the - * database will diverge from backups 4 and 5 and the new backup will fail. - * If you want to create new backup, you will first have to delete backups 4 - * and 5. - */ - public void restoreDBFromBackup(long backupId, String dbDir, String walDir, - RestoreOptions restoreOptions) throws RocksDBException { - restoreDBFromBackup0(nativeHandle_, backupId, dbDir, walDir, - restoreOptions.nativeHandle_); - } - - /** - * Restore from the latest backup. - */ - public void restoreDBFromLatestBackup(String dbDir, String walDir, - RestoreOptions restoreOptions) throws RocksDBException { - restoreDBFromLatestBackup0(nativeHandle_, dbDir, walDir, - restoreOptions.nativeHandle_); - } - - /** - * Deletes old backups, keeping latest numBackupsToKeep alive. - * - * @param Number of latest backups to keep - */ - public void purgeOldBackups(int numBackupsToKeep) throws RocksDBException { - purgeOldBackups0(nativeHandle_, numBackupsToKeep); - } - - /** - * Deletes a specific backup. - * - * @param ID of backup to delete. - */ - public void deleteBackup(long backupId) throws RocksDBException { - deleteBackup0(nativeHandle_, backupId); - } - - /** - * Release the memory allocated for the current instance - * in the c++ side. - */ - @Override public synchronized void disposeInternal() { - assert(isInitialized()); - dispose(nativeHandle_); - } - - private native long newRestoreBackupableDB(long options); - private native void restoreDBFromBackup0(long nativeHandle, long backupId, - String dbDir, String walDir, long restoreOptions) throws RocksDBException; - private native void restoreDBFromLatestBackup0(long nativeHandle, - String dbDir, String walDir, long restoreOptions) throws RocksDBException; - private native void purgeOldBackups0(long nativeHandle, int numBackupsToKeep); - private native void deleteBackup0(long nativeHandle, long backupId); - private native void dispose(long nativeHandle); -} diff --git a/java/org/rocksdb/RestoreOptions.java b/java/org/rocksdb/RestoreOptions.java deleted file mode 100644 index 77a2b99bc29..00000000000 --- a/java/org/rocksdb/RestoreOptions.java +++ /dev/null @@ -1,37 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * RestoreOptions to control the behavior of restore. - * - * Note that dispose() must be called before this instance become out-of-scope - * to release the allocated memory in c++. - * - * @param If true, restore won't overwrite the existing log files in wal_dir. It - * will also move all log files from archive directory to wal_dir. Use this - * option in combination with BackupableDBOptions::backup_log_files = false - * for persisting in-memory databases. - * Default: false - */ -public class RestoreOptions extends RocksObject { - public RestoreOptions(boolean keepLogFiles) { - super(); - nativeHandle_ = newRestoreOptions(keepLogFiles); - } - - /** - * Release the memory allocated for the current instance - * in the c++ side. - */ - @Override public synchronized void disposeInternal() { - assert(isInitialized()); - dispose(nativeHandle_); - } - - private native long newRestoreOptions(boolean keepLogFiles); - private native void dispose(long handle); -} diff --git a/java/org/rocksdb/RocksDB.java b/java/org/rocksdb/RocksDB.java deleted file mode 100644 index f8968d14d15..00000000000 --- a/java/org/rocksdb/RocksDB.java +++ /dev/null @@ -1,370 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -import java.util.List; -import java.util.Map; -import java.util.HashMap; -import java.io.Closeable; -import java.io.IOException; -import org.rocksdb.util.Environment; - -/** - * A RocksDB is a persistent ordered map from keys to values. It is safe for - * concurrent access from multiple threads without any external synchronization. - * All methods of this class could potentially throw RocksDBException, which - * indicates sth wrong at the rocksdb library side and the call failed. - */ -public class RocksDB extends RocksObject { - public static final int NOT_FOUND = -1; - private static final String[] compressionLibs_ = { - "snappy", "z", "bzip2", "lz4", "lz4hc"}; - - /** - * Loads the necessary library files. - * Calling this method twice will have no effect. - */ - public static synchronized void loadLibrary() { - // loading possibly necessary libraries. - for (String lib : compressionLibs_) { - try { - System.loadLibrary(lib); - } catch (UnsatisfiedLinkError e) { - // since it may be optional, we ignore its loading failure here. - } - } - // However, if any of them is required. We will see error here. - System.loadLibrary("rocksdbjni"); - } - - /** - * Tries to load the necessary library files from the given list of - * directories. - * - * @param paths a list of strings where each describes a directory - * of a library. - */ - public static synchronized void loadLibrary(List paths) { - for (String lib : compressionLibs_) { - for (String path : paths) { - try { - System.load(path + "/" + Environment.getSharedLibraryName(lib)); - break; - } catch (UnsatisfiedLinkError e) { - // since they are optional, we ignore loading fails. - } - } - } - boolean success = false; - UnsatisfiedLinkError err = null; - for (String path : paths) { - try { - System.load(path + "/" + Environment.getJniLibraryName("rocksdbjni")); - success = true; - break; - } catch (UnsatisfiedLinkError e) { - err = e; - } - } - if (success == false) { - throw err; - } - } - - /** - * The factory constructor of RocksDB that opens a RocksDB instance given - * the path to the database using the default options w/ createIfMissing - * set to true. - * - * @param path the path to the rocksdb. - * @param status an out value indicating the status of the Open(). - * @return a rocksdb instance on success, null if the specified rocksdb can - * not be opened. - * - * @see Options.setCreateIfMissing() - * @see Options.createIfMissing() - */ - public static RocksDB open(String path) throws RocksDBException { - RocksDB db = new RocksDB(); - - // This allows to use the rocksjni default Options instead of - // the c++ one. - Options options = new Options(); - return open(options, path); - } - - /** - * The factory constructor of RocksDB that opens a RocksDB instance given - * the path to the database using the specified options and db path. - * - * Options instance *should* not be disposed before all DBs using this options - * instance have been closed. If user doesn't call options dispose explicitly, - * then this options instance will be GC'd automatically. - * - * Options instance can be re-used to open multiple DBs if DB statistics is - * not used. If DB statistics are required, then its recommended to open DB - * with new Options instance as underlying native statistics instance does not - * use any locks to prevent concurrent updates. - */ - public static RocksDB open(Options options, String path) - throws RocksDBException { - // when non-default Options is used, keeping an Options reference - // in RocksDB can prevent Java to GC during the life-time of - // the currently-created RocksDB. - RocksDB db = new RocksDB(); - db.open(options.nativeHandle_, path); - - db.storeOptionsInstance(options); - return db; - } - - private void storeOptionsInstance(Options options) { - options_ = options; - } - - @Override protected void disposeInternal() { - assert(isInitialized()); - disposeInternal(nativeHandle_); - } - - /** - * Close the RocksDB instance. - * This function is equivalent to dispose(). - */ - public void close() { - dispose(); - } - - /** - * Set the database entry for "key" to "value". - * - * @param key the specified key to be inserted. - * @param value the value associated with the specified key. - */ - public void put(byte[] key, byte[] value) throws RocksDBException { - put(nativeHandle_, key, key.length, value, value.length); - } - - /** - * Set the database entry for "key" to "value". - * - * @param key the specified key to be inserted. - * @param value the value associated with the specified key. - */ - public void put(WriteOptions writeOpts, byte[] key, byte[] value) - throws RocksDBException { - put(nativeHandle_, writeOpts.nativeHandle_, - key, key.length, value, value.length); - } - - /** - * Apply the specified updates to the database. - */ - public void write(WriteOptions writeOpts, WriteBatch updates) - throws RocksDBException { - write(writeOpts.nativeHandle_, updates.nativeHandle_); - } - - /** - * Get the value associated with the specified key. - * - * @param key the key to retrieve the value. - * @param value the out-value to receive the retrieved value. - * @return The size of the actual value that matches the specified - * {@code key} in byte. If the return value is greater than the - * length of {@code value}, then it indicates that the size of the - * input buffer {@code value} is insufficient and partial result will - * be returned. RocksDB.NOT_FOUND will be returned if the value not - * found. - */ - public int get(byte[] key, byte[] value) throws RocksDBException { - return get(nativeHandle_, key, key.length, value, value.length); - } - - /** - * Get the value associated with the specified key. - * - * @param key the key to retrieve the value. - * @param value the out-value to receive the retrieved value. - * @return The size of the actual value that matches the specified - * {@code key} in byte. If the return value is greater than the - * length of {@code value}, then it indicates that the size of the - * input buffer {@code value} is insufficient and partial result will - * be returned. RocksDB.NOT_FOUND will be returned if the value not - * found. - */ - public int get(ReadOptions opt, byte[] key, byte[] value) - throws RocksDBException { - return get(nativeHandle_, opt.nativeHandle_, - key, key.length, value, value.length); - } - - /** - * The simplified version of get which returns a new byte array storing - * the value associated with the specified input key if any. null will be - * returned if the specified key is not found. - * - * @param key the key retrieve the value. - * @return a byte array storing the value associated with the input key if - * any. null if it does not find the specified key. - * - * @see RocksDBException - */ - public byte[] get(byte[] key) throws RocksDBException { - return get(nativeHandle_, key, key.length); - } - - /** - * The simplified version of get which returns a new byte array storing - * the value associated with the specified input key if any. null will be - * returned if the specified key is not found. - * - * @param key the key retrieve the value. - * @param opt Read options. - * @return a byte array storing the value associated with the input key if - * any. null if it does not find the specified key. - * - * @see RocksDBException - */ - public byte[] get(ReadOptions opt, byte[] key) throws RocksDBException { - return get(nativeHandle_, opt.nativeHandle_, key, key.length); - } - - /** - * Returns a map of keys for which values were found in DB. - * - * @param keys List of keys for which values need to be retrieved. - * @return Map where key of map is the key passed by user and value for map - * entry is the corresponding value in DB. - * - * @see RocksDBException - */ - public Map multiGet(List keys) - throws RocksDBException { - assert(keys.size() != 0); - - List values = multiGet( - nativeHandle_, keys, keys.size()); - - Map keyValueMap = new HashMap(); - for(int i = 0; i < values.size(); i++) { - if(values.get(i) == null) { - continue; - } - - keyValueMap.put(keys.get(i), values.get(i)); - } - - return keyValueMap; - } - - - /** - * Returns a map of keys for which values were found in DB. - * - * @param List of keys for which values need to be retrieved. - * @param opt Read options. - * @return Map where key of map is the key passed by user and value for map - * entry is the corresponding value in DB. - * - * @see RocksDBException - */ - public Map multiGet(ReadOptions opt, List keys) - throws RocksDBException { - assert(keys.size() != 0); - - List values = multiGet( - nativeHandle_, opt.nativeHandle_, keys, keys.size()); - - Map keyValueMap = new HashMap(); - for(int i = 0; i < values.size(); i++) { - if(values.get(i) == null) { - continue; - } - - keyValueMap.put(keys.get(i), values.get(i)); - } - - return keyValueMap; - } - - /** - * Remove the database entry (if any) for "key". Returns OK on - * success, and a non-OK status on error. It is not an error if "key" - * did not exist in the database. - */ - public void remove(byte[] key) throws RocksDBException { - remove(nativeHandle_, key, key.length); - } - - /** - * Remove the database entry (if any) for "key". Returns OK on - * success, and a non-OK status on error. It is not an error if "key" - * did not exist in the database. - */ - public void remove(WriteOptions writeOpt, byte[] key) - throws RocksDBException { - remove(nativeHandle_, writeOpt.nativeHandle_, key, key.length); - } - - /** - * Return a heap-allocated iterator over the contents of the database. - * The result of newIterator() is initially invalid (caller must - * call one of the Seek methods on the iterator before using it). - * - * Caller should close the iterator when it is no longer needed. - * The returned iterator should be closed before this db is closed. - * - * @return instance of iterator object. - */ - public RocksIterator newIterator() { - return new RocksIterator(iterator0(nativeHandle_)); - } - - /** - * Private constructor. - */ - protected RocksDB() { - super(); - } - - // native methods - protected native void open( - long optionsHandle, String path) throws RocksDBException; - protected native void put( - long handle, byte[] key, int keyLen, - byte[] value, int valueLen) throws RocksDBException; - protected native void put( - long handle, long writeOptHandle, - byte[] key, int keyLen, - byte[] value, int valueLen) throws RocksDBException; - protected native void write( - long writeOptHandle, long batchHandle) throws RocksDBException; - protected native int get( - long handle, byte[] key, int keyLen, - byte[] value, int valueLen) throws RocksDBException; - protected native int get( - long handle, long readOptHandle, byte[] key, int keyLen, - byte[] value, int valueLen) throws RocksDBException; - protected native List multiGet( - long dbHandle, List keys, int keysCount); - protected native List multiGet( - long dbHandle, long rOptHandle, List keys, int keysCount); - protected native byte[] get( - long handle, byte[] key, int keyLen) throws RocksDBException; - protected native byte[] get( - long handle, long readOptHandle, - byte[] key, int keyLen) throws RocksDBException; - protected native void remove( - long handle, byte[] key, int keyLen) throws RocksDBException; - protected native void remove( - long handle, long writeOptHandle, - byte[] key, int keyLen) throws RocksDBException; - protected native long iterator0(long optHandle); - private native void disposeInternal(long handle); - - protected Options options_; -} diff --git a/java/org/rocksdb/RocksDBException.java b/java/org/rocksdb/RocksDBException.java deleted file mode 100644 index acc93669eec..00000000000 --- a/java/org/rocksdb/RocksDBException.java +++ /dev/null @@ -1,23 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -import java.util.*; - -/** - * A RocksDBException encapsulates the error of an operation. This exception - * type is used to describe an internal error from the c++ rocksdb library. - */ -public class RocksDBException extends Exception { - /** - * The private construct used by a set of public static factory method. - * - * @param msg the specified error message. - */ - public RocksDBException(String msg) { - super(msg); - } -} diff --git a/java/org/rocksdb/RocksEnv.java b/java/org/rocksdb/RocksEnv.java deleted file mode 100644 index ce73ba6541e..00000000000 --- a/java/org/rocksdb/RocksEnv.java +++ /dev/null @@ -1,102 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * A RocksEnv is an interface used by the rocksdb implementation to access - * operating system functionality like the filesystem etc. - * - * All Env implementations are safe for concurrent access from - * multiple threads without any external synchronization. - */ -public class RocksEnv extends RocksObject { - public static final int FLUSH_POOL = 0; - public static final int COMPACTION_POOL = 1; - - static { - default_env_ = new RocksEnv(getDefaultEnvInternal()); - } - private static native long getDefaultEnvInternal(); - - /** - * Returns the default environment suitable for the current operating - * system. - * - * The result of getDefault() is a singleton whose ownership belongs - * to rocksdb c++. As a result, the returned RocksEnv will not - * have the ownership of its c++ resource, and calling its dispose() - * will be no-op. - */ - public static RocksEnv getDefault() { - return default_env_; - } - - /** - * Sets the number of background worker threads of the flush pool - * for this environment. - * default number: 1 - */ - public RocksEnv setBackgroundThreads(int num) { - return setBackgroundThreads(num, FLUSH_POOL); - } - - /** - * Sets the number of background worker threads of the specified thread - * pool for this environment. - * - * @param num the number of threads - * @param poolID the id to specified a thread pool. Should be either - * FLUSH_POOL or COMPACTION_POOL. - * Default number: 1 - */ - public RocksEnv setBackgroundThreads(int num, int poolID) { - setBackgroundThreads(nativeHandle_, num, poolID); - return this; - } - private native void setBackgroundThreads( - long handle, int num, int priority); - - /** - * Returns the length of the queue associated with the specified - * thread pool. - * - * @param poolID the id to specified a thread pool. Should be either - * FLUSH_POOL or COMPACTION_POOL. - */ - public int getThreadPoolQueueLen(int poolID) { - return getThreadPoolQueueLen(nativeHandle_, poolID); - } - private native int getThreadPoolQueueLen(long handle, int poolID); - - /** - * Package-private constructor that uses the specified native handle - * to construct a RocksEnv. Note that the ownership of the input handle - * belongs to the caller, and the newly created RocksEnv will not take - * the ownership of the input handle. As a result, calling dispose() - * of the created RocksEnv will be no-op. - */ - RocksEnv(long handle) { - super(); - nativeHandle_ = handle; - disOwnNativeHandle(); - } - - /** - * The helper function of dispose() which all subclasses of RocksObject - * must implement to release their associated C++ resource. - */ - protected void disposeInternal() { - disposeInternal(nativeHandle_); - } - private native void disposeInternal(long handle); - - /** - * The static default RocksEnv. The ownership of its native handle - * belongs to rocksdb c++ and is not able to be released on the Java - * side. - */ - static RocksEnv default_env_; -} diff --git a/java/org/rocksdb/RocksIterator.java b/java/org/rocksdb/RocksIterator.java deleted file mode 100644 index 9ef2e8c24ac..00000000000 --- a/java/org/rocksdb/RocksIterator.java +++ /dev/null @@ -1,136 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * An iterator yields a sequence of key/value pairs from a source. - * The following class defines the interface. Multiple implementations - * are provided by this library. In particular, iterators are provided - * to access the contents of a Table or a DB. - * - * Multiple threads can invoke const methods on an RocksIterator without - * external synchronization, but if any of the threads may call a - * non-const method, all threads accessing the same RocksIterator must use - * external synchronization. - */ -public class RocksIterator extends RocksObject { - public RocksIterator(long nativeHandle) { - super(); - nativeHandle_ = nativeHandle; - } - - /** - * An iterator is either positioned at a key/value pair, or - * not valid. This method returns true iff the iterator is valid. - * @return true if iterator is valid. - */ - public boolean isValid() { - assert(isInitialized()); - return isValid0(nativeHandle_); - } - - /** - * Position at the first key in the source. The iterator is Valid() - * after this call iff the source is not empty. - */ - public void seekToFirst() { - assert(isInitialized()); - seekToFirst0(nativeHandle_); - } - - /** - * Position at the last key in the source. The iterator is - * Valid() after this call iff the source is not empty. - */ - public void seekToLast() { - assert(isInitialized()); - seekToLast0(nativeHandle_); - } - - /** - * Moves to the next entry in the source. After this call, Valid() is - * true iff the iterator was not positioned at the last entry in the source. - * REQUIRES: Valid() - */ - public void next() { - assert(isInitialized()); - next0(nativeHandle_); - } - - /** - * Moves to the previous entry in the source. After this call, Valid() is - * true iff the iterator was not positioned at the first entry in source. - * REQUIRES: Valid() - */ - public void prev() { - assert(isInitialized()); - prev0(nativeHandle_); - } - - /** - * Return the key for the current entry. The underlying storage for - * the returned slice is valid only until the next modification of - * the iterator. - * REQUIRES: Valid() - * @return key for the current entry. - */ - public byte[] key() { - assert(isInitialized()); - return key0(nativeHandle_); - } - - /** - * Return the value for the current entry. The underlying storage for - * the returned slice is valid only until the next modification of - * the iterator. - * REQUIRES: !AtEnd() && !AtStart() - * @return value for the current entry. - */ - public byte[] value() { - assert(isInitialized()); - return value0(nativeHandle_); - } - - /** - * Position at the first key in the source that at or past target - * The iterator is Valid() after this call iff the source contains - * an entry that comes at or past target. - */ - public void seek(byte[] target) { - assert(isInitialized()); - seek0(nativeHandle_, target, target.length); - } - - /** - * If an error has occurred, return it. Else return an ok status. - * If non-blocking IO is requested and this operation cannot be - * satisfied without doing some IO, then this returns Status::Incomplete(). - * - */ - public void status() throws RocksDBException { - assert(isInitialized()); - status0(nativeHandle_); - } - - /** - * Deletes underlying C++ iterator pointer. - */ - @Override protected void disposeInternal() { - assert(isInitialized()); - disposeInternal(nativeHandle_); - } - - private native boolean isValid0(long handle); - private native void disposeInternal(long handle); - private native void seekToFirst0(long handle); - private native void seekToLast0(long handle); - private native void next0(long handle); - private native void prev0(long handle); - private native byte[] key0(long handle); - private native byte[] value0(long handle); - private native void seek0(long handle, byte[] target, int targetLen); - private native void status0(long handle); -} diff --git a/java/org/rocksdb/RocksObject.java b/java/org/rocksdb/RocksObject.java deleted file mode 100644 index 353918d2e2d..00000000000 --- a/java/org/rocksdb/RocksObject.java +++ /dev/null @@ -1,117 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * RocksObject is the base-class of all RocksDB classes that has a pointer to - * some c++ rocksdb object. - * - * RocksObject has dispose() function, which releases its associated c++ resource. - * This function can be either called manually, or being called automatically - * during the regular Java GC process. However, since Java may wrongly assume a - * RocksObject only contains a long member variable and think it is small in size, - * Java may give RocksObject low priority in the GC process. For this, it is - * suggested to call dispose() manually. However, it is safe to let RocksObject go - * out-of-scope without manually calling dispose() as dispose() will be called - * in the finalizer during the regular GC process. - */ -public abstract class RocksObject { - protected RocksObject() { - nativeHandle_ = 0; - owningHandle_ = true; - } - - /** - * Release the c++ object manually pointed by the native handle. - * - * Note that dispose() will also be called during the GC process - * if it was not called before its RocksObject went out-of-scope. - * However, since Java may wrongly wrongly assume those objects are - * small in that they seems to only hold a long variable. As a result, - * they might have low priority in the GC process. To prevent this, - * it is suggested to call dispose() manually. - * - * Note that once an instance of RocksObject has been disposed, - * calling its function will lead undefined behavior. - */ - public final synchronized void dispose() { - if (isOwningNativeHandle() && isInitialized()) { - disposeInternal(); - } - nativeHandle_ = 0; - disOwnNativeHandle(); - } - - /** - * The helper function of dispose() which all subclasses of RocksObject - * must implement to release their associated C++ resource. - */ - protected abstract void disposeInternal(); - - /** - * Revoke ownership of the native object. - * - * This will prevent the object from attempting to delete the underlying - * native object in its finalizer. This must be used when another object - * takes over ownership of the native object or both will attempt to delete - * the underlying object when garbage collected. - * - * When disOwnNativeHandle() is called, dispose() will simply set nativeHandle_ - * to 0 without releasing its associated C++ resource. As a result, - * incorrectly use this function may cause memory leak, and this function call - * will not affect the return value of isInitialized(). - * - * @see dispose() - * @see isInitialized() - */ - protected void disOwnNativeHandle() { - owningHandle_ = false; - } - - /** - * Returns true if the current RocksObject is responsable to release its - * native handle. - * - * @return true if the current RocksObject is responsible to release its - * native handle. - * - * @see disOwnNativeHandle() - * @see dispose() - */ - protected boolean isOwningNativeHandle() { - return owningHandle_; - } - - /** - * Returns true if the associated native handle has been initialized. - * - * @return true if the associated native handle has been initialized. - * - * @see dispose() - */ - protected boolean isInitialized() { - return (nativeHandle_ != 0); - } - - /** - * Simply calls dispose() and release its c++ resource if it has not - * yet released. - */ - @Override protected void finalize() { - dispose(); - } - - /** - * A long variable holding c++ pointer pointing to some RocksDB C++ object. - */ - protected long nativeHandle_; - - /** - * A flag indicating whether the current RocksObject is responsible to - * release the c++ object stored in its nativeHandle_. - */ - private boolean owningHandle_; -} diff --git a/java/org/rocksdb/SkipListMemTableConfig.java b/java/org/rocksdb/SkipListMemTableConfig.java deleted file mode 100644 index 7f9f5cb5f43..00000000000 --- a/java/org/rocksdb/SkipListMemTableConfig.java +++ /dev/null @@ -1,15 +0,0 @@ -package org.rocksdb; - -/** - * The config for skip-list memtable representation. - */ -public class SkipListMemTableConfig extends MemTableConfig { - public SkipListMemTableConfig() { - } - - @Override protected long newMemTableFactoryHandle() { - return newMemTableFactoryHandle0(); - } - - private native long newMemTableFactoryHandle0(); -} diff --git a/java/org/rocksdb/Statistics.java b/java/org/rocksdb/Statistics.java deleted file mode 100644 index bed2b881089..00000000000 --- a/java/org/rocksdb/Statistics.java +++ /dev/null @@ -1,38 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * Statistics to analyze the performance of a db. Pointer for statistics object - * is managed by Options class. - */ -public class Statistics { - - private final long statsHandle_; - - public Statistics(long statsHandle) { - statsHandle_ = statsHandle; - } - - public long getTickerCount(TickerType tickerType) { - assert(isInitialized()); - return getTickerCount0(tickerType.getValue(), statsHandle_); - } - - public HistogramData geHistogramData(HistogramType histogramType) { - assert(isInitialized()); - HistogramData hist = geHistogramData0( - histogramType.getValue(), statsHandle_); - return hist; - } - - private boolean isInitialized() { - return (statsHandle_ != 0); - } - - private native long getTickerCount0(int tickerType, long handle); - private native HistogramData geHistogramData0(int histogramType, long handle); -} diff --git a/java/org/rocksdb/TickerType.java b/java/org/rocksdb/TickerType.java deleted file mode 100644 index 5ad714d3095..00000000000 --- a/java/org/rocksdb/TickerType.java +++ /dev/null @@ -1,123 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -public enum TickerType { - // total block cache misses - // REQUIRES: BLOCK_CACHE_MISS == BLOCK_CACHE_INDEX_MISS + - // BLOCK_CACHE_FILTER_MISS + - // BLOCK_CACHE_DATA_MISS; - BLOCK_CACHE_MISS(0), - // total block cache hit - // REQUIRES: BLOCK_CACHE_HIT == BLOCK_CACHE_INDEX_HIT + - // BLOCK_CACHE_FILTER_HIT + - // BLOCK_CACHE_DATA_HIT; - BLOCK_CACHE_HIT(1), - // # of blocks added to block cache. - BLOCK_CACHE_ADD(2), - // # of times cache miss when accessing index block from block cache. - BLOCK_CACHE_INDEX_MISS(3), - // # of times cache hit when accessing index block from block cache. - BLOCK_CACHE_INDEX_HIT(4), - // # of times cache miss when accessing filter block from block cache. - BLOCK_CACHE_FILTER_MISS(5), - // # of times cache hit when accessing filter block from block cache. - BLOCK_CACHE_FILTER_HIT(6), - // # of times cache miss when accessing data block from block cache. - BLOCK_CACHE_DATA_MISS(7), - // # of times cache hit when accessing data block from block cache. - BLOCK_CACHE_DATA_HIT(8), - // # of times bloom filter has avoided file reads. - BLOOM_FILTER_USEFUL(9), - - // # of memtable hits. - MEMTABLE_HIT(10), - // # of memtable misses. - MEMTABLE_MISS(11), - - /** - * COMPACTION_KEY_DROP_* count the reasons for key drop during compaction - * There are 3 reasons currently. - */ - COMPACTION_KEY_DROP_NEWER_ENTRY(12), // key was written with a newer value. - COMPACTION_KEY_DROP_OBSOLETE(13), // The key is obsolete. - COMPACTION_KEY_DROP_USER(14), // user compaction function has dropped the key. - - // Number of keys written to the database via the Put and Write call's - NUMBER_KEYS_WRITTEN(15), - // Number of Keys read, - NUMBER_KEYS_READ(16), - // Number keys updated, if inplace update is enabled - NUMBER_KEYS_UPDATED(17), - // Bytes written / read - BYTES_WRITTEN(18), - BYTES_READ(19), - NO_FILE_CLOSES(20), - NO_FILE_OPENS(21), - NO_FILE_ERRORS(22), - // Time system had to wait to do LO-L1 compactions - STALL_L0_SLOWDOWN_MICROS(23), - // Time system had to wait to move memtable to L1. - STALL_MEMTABLE_COMPACTION_MICROS(24), - // write throttle because of too many files in L0 - STALL_L0_NUM_FILES_MICROS(25), - RATE_LIMIT_DELAY_MILLIS(26), - NO_ITERATORS(27), // number of iterators currently open - - // Number of MultiGet calls, keys read, and bytes read - NUMBER_MULTIGET_CALLS(28), - NUMBER_MULTIGET_KEYS_READ(29), - NUMBER_MULTIGET_BYTES_READ(30), - - // Number of deletes records that were not required to be - // written to storage because key does not exist - NUMBER_FILTERED_DELETES(31), - NUMBER_MERGE_FAILURES(32), - SEQUENCE_NUMBER(33), - - // number of times bloom was checked before creating iterator on a - // file, and the number of times the check was useful in avoiding - // iterator creation (and thus likely IOPs). - BLOOM_FILTER_PREFIX_CHECKED(34), - BLOOM_FILTER_PREFIX_USEFUL(35), - - // Number of times we had to reseek inside an iteration to skip - // over large number of keys with same userkey. - NUMBER_OF_RESEEKS_IN_ITERATION(36), - - // Record the number of calls to GetUpadtesSince. Useful to keep track of - // transaction log iterator refreshes - GET_UPDATES_SINCE_CALLS(37), - BLOCK_CACHE_COMPRESSED_MISS(38), // miss in the compressed block cache - BLOCK_CACHE_COMPRESSED_HIT(39), // hit in the compressed block cache - WAL_FILE_SYNCED(40), // Number of times WAL sync is done - WAL_FILE_BYTES(41), // Number of bytes written to WAL - - // Writes can be processed by requesting thread or by the thread at the - // head of the writers queue. - WRITE_DONE_BY_SELF(42), - WRITE_DONE_BY_OTHER(43), - WRITE_WITH_WAL(44), // Number of Write calls that request WAL - COMPACT_READ_BYTES(45), // Bytes read during compaction - COMPACT_WRITE_BYTES(46), // Bytes written during compaction - - // Number of table's properties loaded directly from file, without creating - // table reader object. - NUMBER_DIRECT_LOAD_TABLE_PROPERTIES(47), - NUMBER_SUPERVERSION_ACQUIRES(48), - NUMBER_SUPERVERSION_RELEASES(49), - NUMBER_SUPERVERSION_CLEANUPS(50); - - private final int value_; - - private TickerType(int value) { - value_ = value; - } - - public int getValue() { - return value_; - } -} diff --git a/java/org/rocksdb/WriteBatch.java b/java/org/rocksdb/WriteBatch.java deleted file mode 100644 index f538dc1a0b8..00000000000 --- a/java/org/rocksdb/WriteBatch.java +++ /dev/null @@ -1,112 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -import java.util.*; - -/** - * WriteBatch holds a collection of updates to apply atomically to a DB. - * - * The updates are applied in the order in which they are added - * to the WriteBatch. For example, the value of "key" will be "v3" - * after the following batch is written: - * - * batch.put("key", "v1"); - * batch.remove("key"); - * batch.put("key", "v2"); - * batch.put("key", "v3"); - * - * Multiple threads can invoke const methods on a WriteBatch without - * external synchronization, but if any of the threads may call a - * non-const method, all threads accessing the same WriteBatch must use - * external synchronization. - */ -public class WriteBatch extends RocksObject { - public WriteBatch() { - super(); - newWriteBatch(0); - } - - public WriteBatch(int reserved_bytes) { - nativeHandle_ = 0; - newWriteBatch(reserved_bytes); - } - - /** - * Returns the number of updates in the batch. - */ - public native int count(); - - /** - * Store the mapping "key->value" in the database. - */ - public void put(byte[] key, byte[] value) { - put(key, key.length, value, value.length); - } - - /** - * Merge "value" with the existing value of "key" in the database. - * "key->merge(existing, value)" - */ - public void merge(byte[] key, byte[] value) { - merge(key, key.length, value, value.length); - } - - /** - * If the database contains a mapping for "key", erase it. Else do nothing. - */ - public void remove(byte[] key) { - remove(key, key.length); - } - - /** - * Append a blob of arbitrary size to the records in this batch. The blob will - * be stored in the transaction log but not in any other file. In particular, - * it will not be persisted to the SST files. When iterating over this - * WriteBatch, WriteBatch::Handler::LogData will be called with the contents - * of the blob as it is encountered. Blobs, puts, deletes, and merges will be - * encountered in the same order in thich they were inserted. The blob will - * NOT consume sequence number(s) and will NOT increase the count of the batch - * - * Example application: add timestamps to the transaction log for use in - * replication. - */ - public void putLogData(byte[] blob) { - putLogData(blob, blob.length); - } - - /** - * Clear all updates buffered in this batch - */ - public native void clear(); - - /** - * Delete the c++ side pointer. - */ - @Override protected void disposeInternal() { - assert(isInitialized()); - disposeInternal(nativeHandle_); - } - - private native void newWriteBatch(int reserved_bytes); - private native void put(byte[] key, int keyLen, - byte[] value, int valueLen); - private native void merge(byte[] key, int keyLen, - byte[] value, int valueLen); - private native void remove(byte[] key, int keyLen); - private native void putLogData(byte[] blob, int blobLen); - private native void disposeInternal(long handle); -} - -/** - * Package-private class which provides java api to access - * c++ WriteBatchInternal. - */ -class WriteBatchInternal { - static native void setSequence(WriteBatch batch, long sn); - static native long sequence(WriteBatch batch); - static native void append(WriteBatch b1, WriteBatch b2); -} diff --git a/java/org/rocksdb/WriteBatchTest.java b/java/org/rocksdb/WriteBatchTest.java deleted file mode 100644 index 03a86631335..00000000000 --- a/java/org/rocksdb/WriteBatchTest.java +++ /dev/null @@ -1,124 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. -package org.rocksdb; - -import java.util.*; -import java.io.UnsupportedEncodingException; - -/** - * This class mimics the db/write_batch_test.cc in the c++ rocksdb library. - */ -public class WriteBatchTest { - static { - RocksDB.loadLibrary(); - } - - public static void main(String args[]) { - System.out.println("Testing WriteBatchTest.Empty ==="); - Empty(); - - System.out.println("Testing WriteBatchTest.Multiple ==="); - Multiple(); - - System.out.println("Testing WriteBatchTest.Append ==="); - Append(); - - System.out.println("Testing WriteBatchTest.Blob ==="); - Blob(); - - // The following tests have not yet ported. - // Continue(); - // PutGatherSlices(); - - System.out.println("Passed all WriteBatchTest!"); - } - - static void Empty() { - WriteBatch batch = new WriteBatch(); - assert(batch.count() == 0); - } - - static void Multiple() { - try { - WriteBatch batch = new WriteBatch(); - batch.put("foo".getBytes("US-ASCII"), "bar".getBytes("US-ASCII")); - batch.remove("box".getBytes("US-ASCII")); - batch.put("baz".getBytes("US-ASCII"), "boo".getBytes("US-ASCII")); - WriteBatchInternal.setSequence(batch, 100); - assert(100 == WriteBatchInternal.sequence(batch)); - assert(3 == batch.count()); - assert(new String("Put(baz, boo)@102" + - "Delete(box)@101" + - "Put(foo, bar)@100") - .equals(new String(getContents(batch), "US-ASCII"))); - } catch (UnsupportedEncodingException e) { - System.err.println(e); - assert(false); - } - } - - static void Append() { - WriteBatch b1 = new WriteBatch(); - WriteBatch b2 = new WriteBatch(); - WriteBatchInternal.setSequence(b1, 200); - WriteBatchInternal.setSequence(b2, 300); - WriteBatchInternal.append(b1, b2); - assert(getContents(b1).length == 0); - assert(b1.count() == 0); - try { - b2.put("a".getBytes("US-ASCII"), "va".getBytes("US-ASCII")); - WriteBatchInternal.append(b1, b2); - assert("Put(a, va)@200".equals(new String(getContents(b1), "US-ASCII"))); - assert(1 == b1.count()); - b2.clear(); - b2.put("b".getBytes("US-ASCII"), "vb".getBytes("US-ASCII")); - WriteBatchInternal.append(b1, b2); - assert(new String("Put(a, va)@200" + - "Put(b, vb)@201") - .equals(new String(getContents(b1), "US-ASCII"))); - assert(2 == b1.count()); - b2.remove("foo".getBytes("US-ASCII")); - WriteBatchInternal.append(b1, b2); - assert(new String("Put(a, va)@200" + - "Put(b, vb)@202" + - "Put(b, vb)@201" + - "Delete(foo)@203") - .equals(new String(getContents(b1), "US-ASCII"))); - assert(4 == b1.count()); - } catch (UnsupportedEncodingException e) { - System.err.println(e); - assert(false); - } - } - - static void Blob() { - WriteBatch batch = new WriteBatch(); - try { - batch.put("k1".getBytes("US-ASCII"), "v1".getBytes("US-ASCII")); - batch.put("k2".getBytes("US-ASCII"), "v2".getBytes("US-ASCII")); - batch.put("k3".getBytes("US-ASCII"), "v3".getBytes("US-ASCII")); - batch.putLogData("blob1".getBytes("US-ASCII")); - batch.remove("k2".getBytes("US-ASCII")); - batch.putLogData("blob2".getBytes("US-ASCII")); - batch.merge("foo".getBytes("US-ASCII"), "bar".getBytes("US-ASCII")); - assert(5 == batch.count()); - assert(new String("Merge(foo, bar)@4" + - "Put(k1, v1)@0" + - "Delete(k2)@3" + - "Put(k2, v2)@1" + - "Put(k3, v3)@2") - .equals(new String(getContents(batch), "US-ASCII"))); - } catch (UnsupportedEncodingException e) { - System.err.println(e); - assert(false); - } - } - - static native byte[] getContents(WriteBatch batch); -} diff --git a/java/org/rocksdb/WriteOptions.java b/java/org/rocksdb/WriteOptions.java deleted file mode 100644 index d26dbb918c3..00000000000 --- a/java/org/rocksdb/WriteOptions.java +++ /dev/null @@ -1,99 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb; - -/** - * Options that control write operations. - * - * Note that developers should call WriteOptions.dispose() to release the - * c++ side memory before a WriteOptions instance runs out of scope. - */ -public class WriteOptions extends RocksObject { - public WriteOptions() { - super(); - newWriteOptions(); - } - - @Override protected void disposeInternal() { - assert(isInitialized()); - disposeInternal(nativeHandle_); - } - - /** - * If true, the write will be flushed from the operating system - * buffer cache (by calling WritableFile::Sync()) before the write - * is considered complete. If this flag is true, writes will be - * slower. - * - * If this flag is false, and the machine crashes, some recent - * writes may be lost. Note that if it is just the process that - * crashes (i.e., the machine does not reboot), no writes will be - * lost even if sync==false. - * - * In other words, a DB write with sync==false has similar - * crash semantics as the "write()" system call. A DB write - * with sync==true has similar crash semantics to a "write()" - * system call followed by "fdatasync()". - * - * Default: false - * - * @param flag a boolean flag to indicate whether a write - * should be synchronized. - * @return the instance of the current WriteOptions. - */ - public WriteOptions setSync(boolean flag) { - setSync(nativeHandle_, flag); - return this; - } - - /** - * If true, the write will be flushed from the operating system - * buffer cache (by calling WritableFile::Sync()) before the write - * is considered complete. If this flag is true, writes will be - * slower. - * - * If this flag is false, and the machine crashes, some recent - * writes may be lost. Note that if it is just the process that - * crashes (i.e., the machine does not reboot), no writes will be - * lost even if sync==false. - * - * In other words, a DB write with sync==false has similar - * crash semantics as the "write()" system call. A DB write - * with sync==true has similar crash semantics to a "write()" - * system call followed by "fdatasync()". - */ - public boolean sync() { - return sync(nativeHandle_); - } - - /** - * If true, writes will not first go to the write ahead log, - * and the write may got lost after a crash. - * - * @param flag a boolean flag to specify whether to disable - * write-ahead-log on writes. - * @return the instance of the current WriteOptions. - */ - public WriteOptions setDisableWAL(boolean flag) { - setDisableWAL(nativeHandle_, flag); - return this; - } - - /** - * If true, writes will not first go to the write ahead log, - * and the write may got lost after a crash. - */ - public boolean disableWAL() { - return disableWAL(nativeHandle_); - } - - private native void newWriteOptions(); - private native void setSync(long handle, boolean flag); - private native boolean sync(long handle); - private native void setDisableWAL(long handle, boolean flag); - private native boolean disableWAL(long handle); - private native void disposeInternal(long handle); -} diff --git a/java/org/rocksdb/test/BackupableDBTest.java b/java/org/rocksdb/test/BackupableDBTest.java deleted file mode 100644 index 9d3a64c067e..00000000000 --- a/java/org/rocksdb/test/BackupableDBTest.java +++ /dev/null @@ -1,63 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb.test; - -import org.rocksdb.*; - -public class BackupableDBTest { - static final String db_path = "/tmp/backupablejni_db"; - static final String backup_path = "/tmp/backupablejni_db_backup"; - static { - RocksDB.loadLibrary(); - } - public static void main(String[] args) { - - Options opt = new Options(); - opt.setCreateIfMissing(true); - - BackupableDBOptions bopt = new BackupableDBOptions(backup_path, false, - true, false, true, 0, 0); - BackupableDB bdb = null; - - try { - bdb = BackupableDB.open(opt, bopt, db_path); - - bdb.put("abc".getBytes(), "def".getBytes()); - bdb.put("ghi".getBytes(), "jkl".getBytes()); - bdb.createNewBackup(true); - - // delete record after backup - bdb.remove("abc".getBytes()); - byte[] value = bdb.get("abc".getBytes()); - assert(value == null); - bdb.close(); - - // restore from backup - RestoreOptions ropt = new RestoreOptions(false); - RestoreBackupableDB rdb = new RestoreBackupableDB(bopt); - rdb.restoreDBFromLatestBackup(db_path, db_path, - ropt); - rdb.dispose(); - ropt.dispose(); - - // verify that backed up data contains deleted record - bdb = BackupableDB.open(opt, bopt, db_path); - value = bdb.get("abc".getBytes()); - assert(new String(value).equals("def")); - - System.out.println("Backup and restore test passed"); - } catch (RocksDBException e) { - System.err.format("[ERROR]: %s%n", e); - e.printStackTrace(); - } finally { - opt.dispose(); - bopt.dispose(); - if (bdb != null) { - bdb.close(); - } - } - } -} diff --git a/java/org/rocksdb/test/OptionsTest.java b/java/org/rocksdb/test/OptionsTest.java deleted file mode 100644 index b065c9023d4..00000000000 --- a/java/org/rocksdb/test/OptionsTest.java +++ /dev/null @@ -1,388 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb.test; - -import java.util.Random; -import org.rocksdb.RocksDB; -import org.rocksdb.Options; - -public class OptionsTest { - static { - RocksDB.loadLibrary(); - } - public static void main(String[] args) { - Options opt = new Options(); - Random rand = new Random(); - { // CreateIfMissing test - boolean boolValue = rand.nextBoolean(); - opt.setCreateIfMissing(boolValue); - assert(opt.createIfMissing() == boolValue); - } - - { // ErrorIfExists test - boolean boolValue = rand.nextBoolean(); - opt.setErrorIfExists(boolValue); - assert(opt.errorIfExists() == boolValue); - } - - { // ParanoidChecks test - boolean boolValue = rand.nextBoolean(); - opt.setParanoidChecks(boolValue); - assert(opt.paranoidChecks() == boolValue); - } - - { // MaxOpenFiles test - int intValue = rand.nextInt(); - opt.setMaxOpenFiles(intValue); - assert(opt.maxOpenFiles() == intValue); - } - - { // DisableDataSync test - boolean boolValue = rand.nextBoolean(); - opt.setDisableDataSync(boolValue); - assert(opt.disableDataSync() == boolValue); - } - - { // UseFsync test - boolean boolValue = rand.nextBoolean(); - opt.setUseFsync(boolValue); - assert(opt.useFsync() == boolValue); - } - - { // DbStatsLogInterval test - int intValue = rand.nextInt(); - opt.setDbStatsLogInterval(intValue); - assert(opt.dbStatsLogInterval() == intValue); - } - - { // DbLogDir test - String str = "path/to/DbLogDir"; - opt.setDbLogDir(str); - assert(opt.dbLogDir().equals(str)); - } - - { // WalDir test - String str = "path/to/WalDir"; - opt.setWalDir(str); - assert(opt.walDir().equals(str)); - } - - { // DeleteObsoleteFilesPeriodMicros test - long longValue = rand.nextLong(); - opt.setDeleteObsoleteFilesPeriodMicros(longValue); - assert(opt.deleteObsoleteFilesPeriodMicros() == longValue); - } - - { // MaxBackgroundCompactions test - int intValue = rand.nextInt(); - opt.setMaxBackgroundCompactions(intValue); - assert(opt.maxBackgroundCompactions() == intValue); - } - - { // MaxBackgroundFlushes test - int intValue = rand.nextInt(); - opt.setMaxBackgroundFlushes(intValue); - assert(opt.maxBackgroundFlushes() == intValue); - } - - { // MaxLogFileSize test - long longValue = rand.nextLong(); - opt.setMaxLogFileSize(longValue); - assert(opt.maxLogFileSize() == longValue); - } - - { // LogFileTimeToRoll test - long longValue = rand.nextLong(); - opt.setLogFileTimeToRoll(longValue); - assert(opt.logFileTimeToRoll() == longValue); - } - - { // KeepLogFileNum test - long longValue = rand.nextLong(); - opt.setKeepLogFileNum(longValue); - assert(opt.keepLogFileNum() == longValue); - } - - { // MaxManifestFileSize test - long longValue = rand.nextLong(); - opt.setMaxManifestFileSize(longValue); - assert(opt.maxManifestFileSize() == longValue); - } - - { // TableCacheNumshardbits test - int intValue = rand.nextInt(); - opt.setTableCacheNumshardbits(intValue); - assert(opt.tableCacheNumshardbits() == intValue); - } - - { // TableCacheRemoveScanCountLimit test - int intValue = rand.nextInt(); - opt.setTableCacheRemoveScanCountLimit(intValue); - assert(opt.tableCacheRemoveScanCountLimit() == intValue); - } - - { // WalTtlSeconds test - long longValue = rand.nextLong(); - opt.setWalTtlSeconds(longValue); - assert(opt.walTtlSeconds() == longValue); - } - - { // ManifestPreallocationSize test - long longValue = rand.nextLong(); - opt.setManifestPreallocationSize(longValue); - assert(opt.manifestPreallocationSize() == longValue); - } - - { // AllowOsBuffer test - boolean boolValue = rand.nextBoolean(); - opt.setAllowOsBuffer(boolValue); - assert(opt.allowOsBuffer() == boolValue); - } - - { // AllowMmapReads test - boolean boolValue = rand.nextBoolean(); - opt.setAllowMmapReads(boolValue); - assert(opt.allowMmapReads() == boolValue); - } - - { // AllowMmapWrites test - boolean boolValue = rand.nextBoolean(); - opt.setAllowMmapWrites(boolValue); - assert(opt.allowMmapWrites() == boolValue); - } - - { // IsFdCloseOnExec test - boolean boolValue = rand.nextBoolean(); - opt.setIsFdCloseOnExec(boolValue); - assert(opt.isFdCloseOnExec() == boolValue); - } - - { // SkipLogErrorOnRecovery test - boolean boolValue = rand.nextBoolean(); - opt.setSkipLogErrorOnRecovery(boolValue); - assert(opt.skipLogErrorOnRecovery() == boolValue); - } - - { // StatsDumpPeriodSec test - int intValue = rand.nextInt(); - opt.setStatsDumpPeriodSec(intValue); - assert(opt.statsDumpPeriodSec() == intValue); - } - - { // AdviseRandomOnOpen test - boolean boolValue = rand.nextBoolean(); - opt.setAdviseRandomOnOpen(boolValue); - assert(opt.adviseRandomOnOpen() == boolValue); - } - - { // UseAdaptiveMutex test - boolean boolValue = rand.nextBoolean(); - opt.setUseAdaptiveMutex(boolValue); - assert(opt.useAdaptiveMutex() == boolValue); - } - - { // BytesPerSync test - long longValue = rand.nextLong(); - opt.setBytesPerSync(longValue); - assert(opt.bytesPerSync() == longValue); - } - - { // AllowThreadLocal test - boolean boolValue = rand.nextBoolean(); - opt.setAllowThreadLocal(boolValue); - assert(opt.allowThreadLocal() == boolValue); - } - - { // WriteBufferSize test - long longValue = rand.nextLong(); - opt.setWriteBufferSize(longValue); - assert(opt.writeBufferSize() == longValue); - } - - { // MaxWriteBufferNumber test - int intValue = rand.nextInt(); - opt.setMaxWriteBufferNumber(intValue); - assert(opt.maxWriteBufferNumber() == intValue); - } - - { // MinWriteBufferNumberToMerge test - int intValue = rand.nextInt(); - opt.setMinWriteBufferNumberToMerge(intValue); - assert(opt.minWriteBufferNumberToMerge() == intValue); - } - - { // NumLevels test - int intValue = rand.nextInt(); - opt.setNumLevels(intValue); - assert(opt.numLevels() == intValue); - } - - { // LevelFileNumCompactionTrigger test - int intValue = rand.nextInt(); - opt.setLevelZeroFileNumCompactionTrigger(intValue); - assert(opt.levelZeroFileNumCompactionTrigger() == intValue); - } - - { // LevelSlowdownWritesTrigger test - int intValue = rand.nextInt(); - opt.setLevelZeroSlowdownWritesTrigger(intValue); - assert(opt.levelZeroSlowdownWritesTrigger() == intValue); - } - - { // LevelStopWritesTrigger test - int intValue = rand.nextInt(); - opt.setLevelZeroStopWritesTrigger(intValue); - assert(opt.levelZeroStopWritesTrigger() == intValue); - } - - { // MaxMemCompactionLevel test - int intValue = rand.nextInt(); - opt.setMaxMemCompactionLevel(intValue); - assert(opt.maxMemCompactionLevel() == intValue); - } - - { // TargetFileSizeBase test - int intValue = rand.nextInt(); - opt.setTargetFileSizeBase(intValue); - assert(opt.targetFileSizeBase() == intValue); - } - - { // TargetFileSizeMultiplier test - int intValue = rand.nextInt(); - opt.setTargetFileSizeMultiplier(intValue); - assert(opt.targetFileSizeMultiplier() == intValue); - } - - { // MaxBytesForLevelBase test - long longValue = rand.nextLong(); - opt.setMaxBytesForLevelBase(longValue); - assert(opt.maxBytesForLevelBase() == longValue); - } - - { // MaxBytesForLevelMultiplier test - int intValue = rand.nextInt(); - opt.setMaxBytesForLevelMultiplier(intValue); - assert(opt.maxBytesForLevelMultiplier() == intValue); - } - - { // ExpandedCompactionFactor test - int intValue = rand.nextInt(); - opt.setExpandedCompactionFactor(intValue); - assert(opt.expandedCompactionFactor() == intValue); - } - - { // SourceCompactionFactor test - int intValue = rand.nextInt(); - opt.setSourceCompactionFactor(intValue); - assert(opt.sourceCompactionFactor() == intValue); - } - - { // MaxGrandparentOverlapFactor test - int intValue = rand.nextInt(); - opt.setMaxGrandparentOverlapFactor(intValue); - assert(opt.maxGrandparentOverlapFactor() == intValue); - } - - { // SoftRateLimit test - double doubleValue = rand.nextDouble(); - opt.setSoftRateLimit(doubleValue); - assert(opt.softRateLimit() == doubleValue); - } - - { // HardRateLimit test - double doubleValue = rand.nextDouble(); - opt.setHardRateLimit(doubleValue); - assert(opt.hardRateLimit() == doubleValue); - } - - { // RateLimitDelayMaxMilliseconds test - int intValue = rand.nextInt(); - opt.setRateLimitDelayMaxMilliseconds(intValue); - assert(opt.rateLimitDelayMaxMilliseconds() == intValue); - } - - { // ArenaBlockSize test - long longValue = rand.nextLong(); - opt.setArenaBlockSize(longValue); - assert(opt.arenaBlockSize() == longValue); - } - - { // DisableAutoCompactions test - boolean boolValue = rand.nextBoolean(); - opt.setDisableAutoCompactions(boolValue); - assert(opt.disableAutoCompactions() == boolValue); - } - - { // PurgeRedundantKvsWhileFlush test - boolean boolValue = rand.nextBoolean(); - opt.setPurgeRedundantKvsWhileFlush(boolValue); - assert(opt.purgeRedundantKvsWhileFlush() == boolValue); - } - - { // VerifyChecksumsInCompaction test - boolean boolValue = rand.nextBoolean(); - opt.setVerifyChecksumsInCompaction(boolValue); - assert(opt.verifyChecksumsInCompaction() == boolValue); - } - - { // FilterDeletes test - boolean boolValue = rand.nextBoolean(); - opt.setFilterDeletes(boolValue); - assert(opt.filterDeletes() == boolValue); - } - - { // MaxSequentialSkipInIterations test - long longValue = rand.nextLong(); - opt.setMaxSequentialSkipInIterations(longValue); - assert(opt.maxSequentialSkipInIterations() == longValue); - } - - { // InplaceUpdateSupport test - boolean boolValue = rand.nextBoolean(); - opt.setInplaceUpdateSupport(boolValue); - assert(opt.inplaceUpdateSupport() == boolValue); - } - - { // InplaceUpdateNumLocks test - long longValue = rand.nextLong(); - opt.setInplaceUpdateNumLocks(longValue); - assert(opt.inplaceUpdateNumLocks() == longValue); - } - - { // MemtablePrefixBloomBits test - int intValue = rand.nextInt(); - opt.setMemtablePrefixBloomBits(intValue); - assert(opt.memtablePrefixBloomBits() == intValue); - } - - { // MemtablePrefixBloomProbes test - int intValue = rand.nextInt(); - opt.setMemtablePrefixBloomProbes(intValue); - assert(opt.memtablePrefixBloomProbes() == intValue); - } - - { // BloomLocality test - int intValue = rand.nextInt(); - opt.setBloomLocality(intValue); - assert(opt.bloomLocality() == intValue); - } - - { // MaxSuccessiveMerges test - long longValue = rand.nextLong(); - opt.setMaxSuccessiveMerges(longValue); - assert(opt.maxSuccessiveMerges() == longValue); - } - - { // MinPartialMergeOperands test - int intValue = rand.nextInt(); - opt.setMinPartialMergeOperands(intValue); - assert(opt.minPartialMergeOperands() == intValue); - } - - opt.dispose(); - System.out.println("Passed OptionsTest"); - } -} diff --git a/java/org/rocksdb/test/ReadOptionsTest.java b/java/org/rocksdb/test/ReadOptionsTest.java deleted file mode 100644 index b3b5b2690c1..00000000000 --- a/java/org/rocksdb/test/ReadOptionsTest.java +++ /dev/null @@ -1,40 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb.test; - -import java.util.Random; -import org.rocksdb.RocksDB; -import org.rocksdb.ReadOptions; - -public class ReadOptionsTest { - static { - RocksDB.loadLibrary(); - } - public static void main(String[] args) { - ReadOptions opt = new ReadOptions(); - Random rand = new Random(); - { // VerifyChecksums test - boolean boolValue = rand.nextBoolean(); - opt.setVerifyChecksums(boolValue); - assert(opt.verifyChecksums() == boolValue); - } - - { // FillCache test - boolean boolValue = rand.nextBoolean(); - opt.setFillCache(boolValue); - assert(opt.fillCache() == boolValue); - } - - { // Tailing test - boolean boolValue = rand.nextBoolean(); - opt.setTailing(boolValue); - assert(opt.tailing() == boolValue); - } - - opt.dispose(); - System.out.println("Passed ReadOptionsTest"); - } -} diff --git a/java/org/rocksdb/test/StatisticsCollectorTest.java b/java/org/rocksdb/test/StatisticsCollectorTest.java deleted file mode 100644 index e497d14dfcf..00000000000 --- a/java/org/rocksdb/test/StatisticsCollectorTest.java +++ /dev/null @@ -1,43 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb.test; - -import java.util.Collections; -import org.rocksdb.*; - -public class StatisticsCollectorTest { - static final String db_path = "/tmp/backupablejni_db"; - static { - RocksDB.loadLibrary(); - } - - public static void main(String[] args) - throws InterruptedException, RocksDBException { - Options opt = new Options().createStatistics().setCreateIfMissing(true); - Statistics stats = opt.statisticsPtr(); - - RocksDB db = RocksDB.open(db_path); - - StatsCallbackMock callback = new StatsCallbackMock(); - StatsCollectorInput statsInput = new StatsCollectorInput(stats, callback); - - StatisticsCollector statsCollector = new StatisticsCollector( - Collections.singletonList(statsInput), 100); - statsCollector.start(); - - Thread.sleep(1000); - - assert(callback.tickerCallbackCount > 0); - assert(callback.histCallbackCount > 0); - - statsCollector.shutDown(1000); - - db.close(); - opt.dispose(); - - System.out.println("Stats collector test passed.!"); - } -} diff --git a/java/org/rocksdb/util/Environment.java b/java/org/rocksdb/util/Environment.java deleted file mode 100644 index c2e3bc088e6..00000000000 --- a/java/org/rocksdb/util/Environment.java +++ /dev/null @@ -1,37 +0,0 @@ -package org.rocksdb.util; - -public class Environment { - private static String OS = System.getProperty("os.name").toLowerCase(); - - public static boolean isWindows() { - return (OS.indexOf("win") >= 0); - } - - public static boolean isMac() { - return (OS.indexOf("mac") >= 0); - } - - public static boolean isUnix() { - return (OS.indexOf("nix") >= 0 || - OS.indexOf("nux") >= 0 || - OS.indexOf("aix") >= 0); - } - - public static String getSharedLibraryName(String name) { - if (isUnix()) { - return String.format("lib%s.so", name); - } else if (isMac()) { - return String.format("lib%s.dylib", name); - } - throw new UnsupportedOperationException(); - } - - public static String getJniLibraryName(String name) { - if (isUnix()) { - return String.format("lib%s.so", name); - } else if (isMac()) { - return String.format("lib%s.jnilib", name); - } - throw new UnsupportedOperationException(); - } -} diff --git a/java/org/rocksdb/util/SizeUnit.java b/java/org/rocksdb/util/SizeUnit.java deleted file mode 100644 index 8d50cd10e6a..00000000000 --- a/java/org/rocksdb/util/SizeUnit.java +++ /dev/null @@ -1,16 +0,0 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -package org.rocksdb.util; - -public class SizeUnit { - public static final long KB = 1024L; - public static final long MB = KB * KB; - public static final long GB = KB * MB; - public static final long TB = KB * GB; - public static final long PB = KB * TB; - - private SizeUnit() {} -} diff --git a/java/rocksjni.pom b/java/rocksjni.pom new file mode 100644 index 00000000000..94f07551c36 --- /dev/null +++ b/java/rocksjni.pom @@ -0,0 +1,150 @@ + + + 4.0.0 + RocksDB JNI + http://rocksdb.org/ + org.rocksdb + rocksdbjni + + - + RocksDB fat jar that contains .so files for linux32 and linux64, jnilib files + for Mac OSX, and a .dll for Windows x64. + + + + Apache License 2.0 + http://www.apache.org/licenses/LICENSE-2.0.html + repo + + + GNU General Public License, version 2 + http://www.gnu.org/licenses/gpl-2.0.html + repo + + + + scm:git:git://github.com/dropwizard/metrics.git + scm:git:git@github.com:dropwizard/metrics.git + http://github.com/dropwizard/metrics/ + HEAD + + + + Facebook + help@facebook.com + America/New_York + + architect + + + + + + 1.7 + 1.7 + UTF-8 + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.2 + + ${project.build.source} + ${project.build.target} + ${project.build.sourceEncoding} + + + + org.apache.maven.plugins + maven-surefire-plugin + 2.18.1 + + ${argLine} -ea -Xcheck:jni -Djava.library.path=${project.build.directory} + false + false + + ${project.build.directory}/* + + + + + org.jacoco + jacoco-maven-plugin + 0.7.2.201409121644 + + + + prepare-agent + + + + report + prepare-package + + report + + + + + + org.codehaus.gmaven + groovy-maven-plugin + 2.0 + + + process-classes + + execute + + + + Xenu + + + String fileContents = new File(project.basedir.absolutePath + '/../include/rocksdb/version.h').getText('UTF-8') + matcher = (fileContents =~ /(?s).*ROCKSDB_MAJOR ([0-9]+).*?/) + String major_version = matcher.getAt(0).getAt(1) + matcher = (fileContents =~ /(?s).*ROCKSDB_MINOR ([0-9]+).*?/) + String minor_version = matcher.getAt(0).getAt(1) + matcher = (fileContents =~ /(?s).*ROCKSDB_PATCH ([0-9]+).*?/) + String patch_version = matcher.getAt(0).getAt(1) + String version = String.format('%s.%s.%s', major_version, minor_version, patch_version) + // Set version to be used in pom.properties + project.version = version + // Set version to be set as jar name + project.build.finalName = project.artifactId + "-" + version + + + + + + + + + + + junit + junit + 4.12 + test + + + org.assertj + assertj-core + 1.7.1 + test + + + org.mockito + mockito-all + 1.10.19 + test + + + diff --git a/java/rocksjni/backupablejni.cc b/java/rocksjni/backupablejni.cc index 2aa1d0b1d75..28db2b0210a 100644 --- a/java/rocksjni/backupablejni.cc +++ b/java/rocksjni/backupablejni.cc @@ -1,102 +1,303 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ and enables -// calling c++ rocksdb::BackupableDB and rocksdb::BackupableDBOptions methods +// calling c++ rocksdb::BackupEnginge and rocksdb::BackupableDBOptions methods // from Java side. #include #include #include #include +#include -#include "include/org_rocksdb_BackupableDB.h" #include "include/org_rocksdb_BackupableDBOptions.h" #include "rocksjni/portal.h" #include "rocksdb/utilities/backupable_db.h" +/////////////////////////////////////////////////////////////////////////// +// BackupDBOptions + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: newBackupableDBOptions + * Signature: (Ljava/lang/String;)J + */ +jlong Java_org_rocksdb_BackupableDBOptions_newBackupableDBOptions( + JNIEnv* env, jclass jcls, jstring jpath) { + const char* cpath = env->GetStringUTFChars(jpath, nullptr); + if(cpath == nullptr) { + // exception thrown: OutOfMemoryError + return 0; + } + auto* bopt = new rocksdb::BackupableDBOptions(cpath); + env->ReleaseStringUTFChars(jpath, cpath); + return reinterpret_cast(bopt); +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: backupDir + * Signature: (J)Ljava/lang/String; + */ +jstring Java_org_rocksdb_BackupableDBOptions_backupDir( + JNIEnv* env, jobject jopt, jlong jhandle) { + auto* bopt = reinterpret_cast(jhandle); + return env->NewStringUTF(bopt->backup_dir.c_str()); +} + /* - * Class: org_rocksdb_BackupableDB - * Method: open + * Class: org_rocksdb_BackupableDBOptions + * Method: setBackupEnv * Signature: (JJ)V */ -void Java_org_rocksdb_BackupableDB_open( - JNIEnv* env, jobject jbdb, jlong jdb_handle, jlong jopt_handle) { - auto db = reinterpret_cast(jdb_handle); - auto opt = reinterpret_cast(jopt_handle); - auto bdb = new rocksdb::BackupableDB(db, *opt); +void Java_org_rocksdb_BackupableDBOptions_setBackupEnv( + JNIEnv* env, jobject jopt, jlong jhandle, jlong jrocks_env_handle) { + auto* bopt = reinterpret_cast(jhandle); + auto* rocks_env = reinterpret_cast(jrocks_env_handle); + bopt->backup_env = rocks_env; +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: setShareTableFiles + * Signature: (JZ)V + */ +void Java_org_rocksdb_BackupableDBOptions_setShareTableFiles( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean flag) { + auto* bopt = reinterpret_cast(jhandle); + bopt->share_table_files = flag; +} - // as BackupableDB extends RocksDB on the java side, we can reuse - // the RocksDB portal here. - rocksdb::RocksDBJni::setHandle(env, jbdb, bdb); +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: shareTableFiles + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_BackupableDBOptions_shareTableFiles( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* bopt = reinterpret_cast(jhandle); + return bopt->share_table_files; } /* - * Class: org_rocksdb_BackupableDB - * Method: createNewBackup + * Class: org_rocksdb_BackupableDBOptions + * Method: setInfoLog + * Signature: (JJ)V + */ +void Java_org_rocksdb_BackupableDBOptions_setInfoLog( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jlogger_handle) { + auto* bopt = reinterpret_cast(jhandle); + auto* sptr_logger = + reinterpret_cast *>(jhandle); + bopt->info_log = sptr_logger->get(); +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: setSync * Signature: (JZ)V */ -void Java_org_rocksdb_BackupableDB_createNewBackup( - JNIEnv* env, jobject jbdb, jlong jhandle, jboolean jflag) { - rocksdb::Status s = - reinterpret_cast(jhandle)->CreateNewBackup(jflag); - if (!s.ok()) { - rocksdb::RocksDBExceptionJni::ThrowNew(env, s); - } +void Java_org_rocksdb_BackupableDBOptions_setSync( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean flag) { + auto* bopt = reinterpret_cast(jhandle); + bopt->sync = flag; } /* - * Class: org_rocksdb_BackupableDB - * Method: purgeOldBackups - * Signature: (JI)V + * Class: org_rocksdb_BackupableDBOptions + * Method: sync + * Signature: (J)Z */ -void Java_org_rocksdb_BackupableDB_purgeOldBackups( - JNIEnv* env, jobject jbdb, jlong jhandle, jboolean jnumBackupsToKeep) { - rocksdb::Status s = - reinterpret_cast(jhandle)-> - PurgeOldBackups(jnumBackupsToKeep); - if (!s.ok()) { - rocksdb::RocksDBExceptionJni::ThrowNew(env, s); - } +jboolean Java_org_rocksdb_BackupableDBOptions_sync( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* bopt = reinterpret_cast(jhandle); + return bopt->sync; } -/////////////////////////////////////////////////////////////////////////// -// BackupDBOptions +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: setDestroyOldData + * Signature: (JZ)V + */ +void Java_org_rocksdb_BackupableDBOptions_setDestroyOldData( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean flag) { + auto* bopt = reinterpret_cast(jhandle); + bopt->destroy_old_data = flag; +} /* * Class: org_rocksdb_BackupableDBOptions - * Method: newBackupableDBOptions - * Signature: (Ljava/lang/String;)V + * Method: destroyOldData + * Signature: (J)Z */ -void Java_org_rocksdb_BackupableDBOptions_newBackupableDBOptions( - JNIEnv* env, jobject jobj, jstring jpath, jboolean jshare_table_files, - jboolean jsync, jboolean jdestroy_old_data, jboolean jbackup_log_files, - jlong jbackup_rate_limit, jlong jrestore_rate_limit) { - jbackup_rate_limit = (jbackup_rate_limit <= 0) ? 0 : jbackup_rate_limit; - jrestore_rate_limit = (jrestore_rate_limit <= 0) ? 0 : jrestore_rate_limit; +jboolean Java_org_rocksdb_BackupableDBOptions_destroyOldData( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* bopt = reinterpret_cast(jhandle); + return bopt->destroy_old_data; +} - const char* cpath = env->GetStringUTFChars(jpath, 0); +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: setBackupLogFiles + * Signature: (JZ)V + */ +void Java_org_rocksdb_BackupableDBOptions_setBackupLogFiles( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean flag) { + auto* bopt = reinterpret_cast(jhandle); + bopt->backup_log_files = flag; +} - auto bopt = new rocksdb::BackupableDBOptions(cpath, nullptr, - jshare_table_files, nullptr, jsync, jdestroy_old_data, jbackup_log_files, - jbackup_rate_limit, jrestore_rate_limit); +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: backupLogFiles + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_BackupableDBOptions_backupLogFiles( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* bopt = reinterpret_cast(jhandle); + return bopt->backup_log_files; +} - env->ReleaseStringUTFChars(jpath, cpath); +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: setBackupRateLimit + * Signature: (JJ)V + */ +void Java_org_rocksdb_BackupableDBOptions_setBackupRateLimit( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jbackup_rate_limit) { + auto* bopt = reinterpret_cast(jhandle); + bopt->backup_rate_limit = jbackup_rate_limit; +} - rocksdb::BackupableDBOptionsJni::setHandle(env, jobj, bopt); +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: backupRateLimit + * Signature: (J)J + */ +jlong Java_org_rocksdb_BackupableDBOptions_backupRateLimit( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* bopt = reinterpret_cast(jhandle); + return bopt->backup_rate_limit; } /* * Class: org_rocksdb_BackupableDBOptions - * Method: backupDir - * Signature: (J)Ljava/lang/String; + * Method: setBackupRateLimiter + * Signature: (JJ)V */ -jstring Java_org_rocksdb_BackupableDBOptions_backupDir( - JNIEnv* env, jobject jopt, jlong jhandle, jstring jpath) { - auto bopt = reinterpret_cast(jhandle); - return env->NewStringUTF(bopt->backup_dir.c_str()); +void Java_org_rocksdb_BackupableDBOptions_setBackupRateLimiter( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jrate_limiter_handle) { + auto* bopt = reinterpret_cast(jhandle); + auto* sptr_rate_limiter = + reinterpret_cast *>(jrate_limiter_handle); + bopt->backup_rate_limiter = *sptr_rate_limiter; +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: setRestoreRateLimit + * Signature: (JJ)V + */ +void Java_org_rocksdb_BackupableDBOptions_setRestoreRateLimit( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jrestore_rate_limit) { + auto* bopt = reinterpret_cast(jhandle); + bopt->restore_rate_limit = jrestore_rate_limit; +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: restoreRateLimit + * Signature: (J)J + */ +jlong Java_org_rocksdb_BackupableDBOptions_restoreRateLimit( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* bopt = reinterpret_cast(jhandle); + return bopt->restore_rate_limit; +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: setRestoreRateLimiter + * Signature: (JJ)V + */ +void Java_org_rocksdb_BackupableDBOptions_setRestoreRateLimiter( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jrate_limiter_handle) { + auto* bopt = reinterpret_cast(jhandle); + auto* sptr_rate_limiter = + reinterpret_cast *>(jrate_limiter_handle); + bopt->restore_rate_limiter = *sptr_rate_limiter; +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: setShareFilesWithChecksum + * Signature: (JZ)V + */ +void Java_org_rocksdb_BackupableDBOptions_setShareFilesWithChecksum( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean flag) { + auto* bopt = reinterpret_cast(jhandle); + bopt->share_files_with_checksum = flag; +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: shareFilesWithChecksum + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_BackupableDBOptions_shareFilesWithChecksum( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* bopt = reinterpret_cast(jhandle); + return bopt->share_files_with_checksum; +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: setMaxBackgroundOperations + * Signature: (JI)V + */ +void Java_org_rocksdb_BackupableDBOptions_setMaxBackgroundOperations( + JNIEnv* env, jobject jobj, jlong jhandle, jint max_background_operations) { + auto* bopt = reinterpret_cast(jhandle); + bopt->max_background_operations = + static_cast(max_background_operations); +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: maxBackgroundOperations + * Signature: (J)I + */ +jint Java_org_rocksdb_BackupableDBOptions_maxBackgroundOperations( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* bopt = reinterpret_cast(jhandle); + return static_cast(bopt->max_background_operations); +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: setCallbackTriggerIntervalSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_BackupableDBOptions_setCallbackTriggerIntervalSize( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jcallback_trigger_interval_size) { + auto* bopt = reinterpret_cast(jhandle); + bopt->callback_trigger_interval_size = + static_cast(jcallback_trigger_interval_size); +} + +/* + * Class: org_rocksdb_BackupableDBOptions + * Method: callbackTriggerIntervalSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_BackupableDBOptions_callbackTriggerIntervalSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* bopt = reinterpret_cast(jhandle); + return static_cast(bopt->callback_trigger_interval_size); } /* @@ -106,9 +307,7 @@ jstring Java_org_rocksdb_BackupableDBOptions_backupDir( */ void Java_org_rocksdb_BackupableDBOptions_disposeInternal( JNIEnv* env, jobject jopt, jlong jhandle) { - auto bopt = reinterpret_cast(jhandle); - assert(bopt); + auto* bopt = reinterpret_cast(jhandle); + assert(bopt != nullptr); delete bopt; - - rocksdb::BackupableDBOptionsJni::setHandle(env, jopt, nullptr); } diff --git a/java/rocksjni/backupenginejni.cc b/java/rocksjni/backupenginejni.cc new file mode 100644 index 00000000000..004de976cbe --- /dev/null +++ b/java/rocksjni/backupenginejni.cc @@ -0,0 +1,236 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ and enables +// calling C++ rocksdb::BackupEngine methods from the Java side. + +#include +#include + +#include "include/org_rocksdb_BackupEngine.h" +#include "rocksdb/utilities/backupable_db.h" +#include "rocksjni/portal.h" + +/* + * Class: org_rocksdb_BackupEngine + * Method: open + * Signature: (JJ)J + */ +jlong Java_org_rocksdb_BackupEngine_open( + JNIEnv* env, jclass jcls, jlong env_handle, + jlong backupable_db_options_handle) { + auto* rocks_env = reinterpret_cast(env_handle); + auto* backupable_db_options = + reinterpret_cast( + backupable_db_options_handle); + rocksdb::BackupEngine* backup_engine; + auto status = rocksdb::BackupEngine::Open(rocks_env, + *backupable_db_options, &backup_engine); + + if (status.ok()) { + return reinterpret_cast(backup_engine); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, status); + return 0; + } +} + +/* + * Class: org_rocksdb_BackupEngine + * Method: createNewBackup + * Signature: (JJZ)V + */ +void Java_org_rocksdb_BackupEngine_createNewBackup( + JNIEnv* env, jobject jbe, jlong jbe_handle, jlong db_handle, + jboolean jflush_before_backup) { + auto* db = reinterpret_cast(db_handle); + auto* backup_engine = reinterpret_cast(jbe_handle); + auto status = backup_engine->CreateNewBackup(db, + static_cast(jflush_before_backup)); + + if (status.ok()) { + return; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, status); +} + +/* + * Class: org_rocksdb_BackupEngine + * Method: getBackupInfo + * Signature: (J)Ljava/util/List; + */ +jobject Java_org_rocksdb_BackupEngine_getBackupInfo( + JNIEnv* env, jobject jbe, jlong jbe_handle) { + auto* backup_engine = reinterpret_cast(jbe_handle); + std::vector backup_infos; + backup_engine->GetBackupInfo(&backup_infos); + return rocksdb::BackupInfoListJni::getBackupInfo(env, backup_infos); +} + +/* + * Class: org_rocksdb_BackupEngine + * Method: getCorruptedBackups + * Signature: (J)[I + */ +jintArray Java_org_rocksdb_BackupEngine_getCorruptedBackups( + JNIEnv* env, jobject jbe, jlong jbe_handle) { + auto* backup_engine = reinterpret_cast(jbe_handle); + std::vector backup_ids; + backup_engine->GetCorruptedBackups(&backup_ids); + // store backupids in int array + std::vector int_backup_ids(backup_ids.begin(), backup_ids.end()); + + // Store ints in java array + // Its ok to loose precision here (64->32) + jsize ret_backup_ids_size = static_cast(backup_ids.size()); + jintArray ret_backup_ids = env->NewIntArray(ret_backup_ids_size); + if(ret_backup_ids == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + env->SetIntArrayRegion(ret_backup_ids, 0, ret_backup_ids_size, + int_backup_ids.data()); + return ret_backup_ids; +} + +/* + * Class: org_rocksdb_BackupEngine + * Method: garbageCollect + * Signature: (J)V + */ +void Java_org_rocksdb_BackupEngine_garbageCollect( + JNIEnv* env, jobject jbe, jlong jbe_handle) { + auto* backup_engine = reinterpret_cast(jbe_handle); + auto status = backup_engine->GarbageCollect(); + + if (status.ok()) { + return; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, status); +} + +/* + * Class: org_rocksdb_BackupEngine + * Method: purgeOldBackups + * Signature: (JI)V + */ +void Java_org_rocksdb_BackupEngine_purgeOldBackups( + JNIEnv* env, jobject jbe, jlong jbe_handle, jint jnum_backups_to_keep) { + auto* backup_engine = reinterpret_cast(jbe_handle); + auto status = + backup_engine-> + PurgeOldBackups(static_cast(jnum_backups_to_keep)); + + if (status.ok()) { + return; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, status); +} + +/* + * Class: org_rocksdb_BackupEngine + * Method: deleteBackup + * Signature: (JI)V + */ +void Java_org_rocksdb_BackupEngine_deleteBackup( + JNIEnv* env, jobject jbe, jlong jbe_handle, jint jbackup_id) { + auto* backup_engine = reinterpret_cast(jbe_handle); + auto status = + backup_engine->DeleteBackup(static_cast(jbackup_id)); + + if (status.ok()) { + return; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, status); +} + +/* + * Class: org_rocksdb_BackupEngine + * Method: restoreDbFromBackup + * Signature: (JILjava/lang/String;Ljava/lang/String;J)V + */ +void Java_org_rocksdb_BackupEngine_restoreDbFromBackup( + JNIEnv* env, jobject jbe, jlong jbe_handle, jint jbackup_id, + jstring jdb_dir, jstring jwal_dir, jlong jrestore_options_handle) { + auto* backup_engine = reinterpret_cast(jbe_handle); + const char* db_dir = env->GetStringUTFChars(jdb_dir, nullptr); + if(db_dir == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + const char* wal_dir = env->GetStringUTFChars(jwal_dir, nullptr); + if(wal_dir == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseStringUTFChars(jdb_dir, db_dir); + return; + } + auto* restore_options = + reinterpret_cast(jrestore_options_handle); + auto status = + backup_engine->RestoreDBFromBackup( + static_cast(jbackup_id), db_dir, wal_dir, + *restore_options); + + env->ReleaseStringUTFChars(jwal_dir, wal_dir); + env->ReleaseStringUTFChars(jdb_dir, db_dir); + + if (status.ok()) { + return; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, status); +} + +/* + * Class: org_rocksdb_BackupEngine + * Method: restoreDbFromLatestBackup + * Signature: (JLjava/lang/String;Ljava/lang/String;J)V + */ +void Java_org_rocksdb_BackupEngine_restoreDbFromLatestBackup( + JNIEnv* env, jobject jbe, jlong jbe_handle, jstring jdb_dir, + jstring jwal_dir, jlong jrestore_options_handle) { + auto* backup_engine = reinterpret_cast(jbe_handle); + const char* db_dir = env->GetStringUTFChars(jdb_dir, nullptr); + if(db_dir == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + const char* wal_dir = env->GetStringUTFChars(jwal_dir, nullptr); + if(wal_dir == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseStringUTFChars(jdb_dir, db_dir); + return; + } + auto* restore_options = + reinterpret_cast(jrestore_options_handle); + auto status = + backup_engine->RestoreDBFromLatestBackup(db_dir, wal_dir, + *restore_options); + + env->ReleaseStringUTFChars(jwal_dir, wal_dir); + env->ReleaseStringUTFChars(jdb_dir, db_dir); + + if (status.ok()) { + return; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, status); +} + +/* + * Class: org_rocksdb_BackupEngine + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_BackupEngine_disposeInternal( + JNIEnv* env, jobject jbe, jlong jbe_handle) { + auto* be = reinterpret_cast(jbe_handle); + assert(be != nullptr); + delete be; +} diff --git a/java/rocksjni/cassandra_compactionfilterjni.cc b/java/rocksjni/cassandra_compactionfilterjni.cc new file mode 100644 index 00000000000..9d77559ab5d --- /dev/null +++ b/java/rocksjni/cassandra_compactionfilterjni.cc @@ -0,0 +1,22 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include + +#include "include/org_rocksdb_CassandraCompactionFilter.h" +#include "utilities/cassandra/cassandra_compaction_filter.h" + +/* + * Class: org_rocksdb_CassandraCompactionFilter + * Method: createNewCassandraCompactionFilter0 + * Signature: ()J + */ +jlong Java_org_rocksdb_CassandraCompactionFilter_createNewCassandraCompactionFilter0( + JNIEnv* env, jclass jcls, jboolean purge_ttl_on_expiration) { + auto* compaction_filter = + new rocksdb::cassandra::CassandraCompactionFilter(purge_ttl_on_expiration); + // set the native handle to our native compaction filter + return reinterpret_cast(compaction_filter); +} diff --git a/java/rocksjni/cassandra_value_operator.cc b/java/rocksjni/cassandra_value_operator.cc new file mode 100644 index 00000000000..aa58eccc24a --- /dev/null +++ b/java/rocksjni/cassandra_value_operator.cc @@ -0,0 +1,45 @@ +// Copyright (c) 2017-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include +#include +#include +#include +#include + +#include "include/org_rocksdb_CassandraValueMergeOperator.h" +#include "rocksjni/portal.h" +#include "rocksdb/db.h" +#include "rocksdb/options.h" +#include "rocksdb/statistics.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/table.h" +#include "rocksdb/slice_transform.h" +#include "rocksdb/merge_operator.h" +#include "utilities/cassandra/merge_operator.h" + +/* + * Class: org_rocksdb_CassandraValueMergeOperator + * Method: newSharedCassandraValueMergeOperator + * Signature: ()J + */ +jlong Java_org_rocksdb_CassandraValueMergeOperator_newSharedCassandraValueMergeOperator +(JNIEnv* env, jclass jclazz) { + auto* sptr_string_append_op = new std::shared_ptr( + rocksdb::CassandraValueMergeOperator::CreateSharedInstance()); + return reinterpret_cast(sptr_string_append_op); +} + +/* + * Class: org_rocksdb_CassandraValueMergeOperator + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_CassandraValueMergeOperator_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* sptr_string_append_op = + reinterpret_cast* >(jhandle); + delete sptr_string_append_op; // delete std::shared_ptr +} diff --git a/java/rocksjni/checkpoint.cc b/java/rocksjni/checkpoint.cc new file mode 100644 index 00000000000..426f5d029e2 --- /dev/null +++ b/java/rocksjni/checkpoint.cc @@ -0,0 +1,68 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ and enables +// calling c++ rocksdb::Checkpoint methods from Java side. + +#include +#include +#include +#include + +#include "include/org_rocksdb_Checkpoint.h" +#include "rocksjni/portal.h" +#include "rocksdb/db.h" +#include "rocksdb/utilities/checkpoint.h" +/* + * Class: org_rocksdb_Checkpoint + * Method: newCheckpoint + * Signature: (J)J + */ +jlong Java_org_rocksdb_Checkpoint_newCheckpoint(JNIEnv* env, + jclass jclazz, jlong jdb_handle) { + auto* db = reinterpret_cast(jdb_handle); + rocksdb::Checkpoint* checkpoint; + rocksdb::Checkpoint::Create(db, &checkpoint); + return reinterpret_cast(checkpoint); +} + +/* + * Class: org_rocksdb_Checkpoint + * Method: dispose + * Signature: (J)V + */ +void Java_org_rocksdb_Checkpoint_disposeInternal(JNIEnv* env, jobject jobj, + jlong jhandle) { + auto* checkpoint = reinterpret_cast(jhandle); + assert(checkpoint != nullptr); + delete checkpoint; +} + +/* + * Class: org_rocksdb_Checkpoint + * Method: createCheckpoint + * Signature: (JLjava/lang/String;)V + */ +void Java_org_rocksdb_Checkpoint_createCheckpoint( + JNIEnv* env, jobject jobj, jlong jcheckpoint_handle, + jstring jcheckpoint_path) { + const char* checkpoint_path = env->GetStringUTFChars( + jcheckpoint_path, 0); + if(checkpoint_path == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + auto* checkpoint = reinterpret_cast( + jcheckpoint_handle); + rocksdb::Status s = checkpoint->CreateCheckpoint( + checkpoint_path); + + env->ReleaseStringUTFChars(jcheckpoint_path, checkpoint_path); + + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} diff --git a/java/rocksjni/clock_cache.cc b/java/rocksjni/clock_cache.cc new file mode 100644 index 00000000000..0a4d7b28d65 --- /dev/null +++ b/java/rocksjni/clock_cache.cc @@ -0,0 +1,40 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ for +// rocksdb::ClockCache. + +#include + +#include "cache/clock_cache.h" +#include "include/org_rocksdb_ClockCache.h" + +/* + * Class: org_rocksdb_ClockCache + * Method: newClockCache + * Signature: (JIZ)J + */ +jlong Java_org_rocksdb_ClockCache_newClockCache( + JNIEnv* env, jclass jcls, jlong jcapacity, jint jnum_shard_bits, + jboolean jstrict_capacity_limit) { + auto* sptr_clock_cache = + new std::shared_ptr(rocksdb::NewClockCache( + static_cast(jcapacity), + static_cast(jnum_shard_bits), + static_cast(jstrict_capacity_limit))); + return reinterpret_cast(sptr_clock_cache); +} + +/* + * Class: org_rocksdb_ClockCache + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_ClockCache_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* sptr_clock_cache = + reinterpret_cast *>(jhandle); + delete sptr_clock_cache; // delete std::shared_ptr +} diff --git a/java/rocksjni/columnfamilyhandle.cc b/java/rocksjni/columnfamilyhandle.cc new file mode 100644 index 00000000000..6e40a7e010b --- /dev/null +++ b/java/rocksjni/columnfamilyhandle.cc @@ -0,0 +1,26 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ and enables +// calling c++ rocksdb::Iterator methods from Java side. + +#include +#include +#include + +#include "include/org_rocksdb_ColumnFamilyHandle.h" +#include "rocksjni/portal.h" + +/* + * Class: org_rocksdb_ColumnFamilyHandle + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_ColumnFamilyHandle_disposeInternal( + JNIEnv* env, jobject jobj, jlong handle) { + auto* cfh = reinterpret_cast(handle); + assert(cfh != nullptr); + delete cfh; +} diff --git a/java/rocksjni/compaction_filter.cc b/java/rocksjni/compaction_filter.cc new file mode 100644 index 00000000000..72de46b3fba --- /dev/null +++ b/java/rocksjni/compaction_filter.cc @@ -0,0 +1,27 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ for +// rocksdb::CompactionFilter. + +#include + +#include "include/org_rocksdb_AbstractCompactionFilter.h" +#include "rocksdb/compaction_filter.h" + +// + +/* + * Class: org_rocksdb_AbstractCompactionFilter + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_AbstractCompactionFilter_disposeInternal( + JNIEnv* env, jobject jobj, jlong handle) { + auto* cf = reinterpret_cast(handle); + assert(cf != nullptr); + delete cf; +} +// diff --git a/java/rocksjni/compaction_options_fifo.cc b/java/rocksjni/compaction_options_fifo.cc new file mode 100644 index 00000000000..ef04d81c648 --- /dev/null +++ b/java/rocksjni/compaction_options_fifo.cc @@ -0,0 +1,55 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ for +// rocksdb::CompactionOptionsFIFO. + +#include + +#include "include/org_rocksdb_CompactionOptionsFIFO.h" +#include "rocksdb/advanced_options.h" + +/* + * Class: org_rocksdb_CompactionOptionsFIFO + * Method: newCompactionOptionsFIFO + * Signature: ()J + */ +jlong Java_org_rocksdb_CompactionOptionsFIFO_newCompactionOptionsFIFO( + JNIEnv* env, jclass jcls) { + const auto* opt = new rocksdb::CompactionOptionsFIFO(); + return reinterpret_cast(opt); +} + +/* + * Class: org_rocksdb_CompactionOptionsFIFO + * Method: setMaxTableFilesSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_CompactionOptionsFIFO_setMaxTableFilesSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jmax_table_files_size) { + auto* opt = reinterpret_cast(jhandle); + opt->max_table_files_size = static_cast(jmax_table_files_size); +} + +/* + * Class: org_rocksdb_CompactionOptionsFIFO + * Method: maxTableFilesSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_CompactionOptionsFIFO_maxTableFilesSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->max_table_files_size); +} + +/* + * Class: org_rocksdb_CompactionOptionsFIFO + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_CompactionOptionsFIFO_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + delete reinterpret_cast(jhandle); +} diff --git a/java/rocksjni/compaction_options_universal.cc b/java/rocksjni/compaction_options_universal.cc new file mode 100644 index 00000000000..d397db8e438 --- /dev/null +++ b/java/rocksjni/compaction_options_universal.cc @@ -0,0 +1,194 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ for +// rocksdb::CompactionOptionsUniversal. + +#include + +#include "include/org_rocksdb_CompactionOptionsUniversal.h" +#include "rocksdb/advanced_options.h" +#include "rocksjni/portal.h" + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: newCompactionOptionsUniversal + * Signature: ()J + */ +jlong Java_org_rocksdb_CompactionOptionsUniversal_newCompactionOptionsUniversal( + JNIEnv* env, jclass jcls) { + const auto* opt = new rocksdb::CompactionOptionsUniversal(); + return reinterpret_cast(opt); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: setSizeRatio + * Signature: (JI)V + */ +void Java_org_rocksdb_CompactionOptionsUniversal_setSizeRatio( + JNIEnv* env, jobject jobj, jlong jhandle, jint jsize_ratio) { + auto* opt = reinterpret_cast(jhandle); + opt->size_ratio = static_cast(jsize_ratio); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: sizeRatio + * Signature: (J)I + */ +jint Java_org_rocksdb_CompactionOptionsUniversal_sizeRatio( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->size_ratio); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: setMinMergeWidth + * Signature: (JI)V + */ +void Java_org_rocksdb_CompactionOptionsUniversal_setMinMergeWidth( + JNIEnv* env, jobject jobj, jlong jhandle, jint jmin_merge_width) { + auto* opt = reinterpret_cast(jhandle); + opt->min_merge_width = static_cast(jmin_merge_width); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: minMergeWidth + * Signature: (J)I + */ +jint Java_org_rocksdb_CompactionOptionsUniversal_minMergeWidth( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->min_merge_width); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: setMaxMergeWidth + * Signature: (JI)V + */ +void Java_org_rocksdb_CompactionOptionsUniversal_setMaxMergeWidth( + JNIEnv* env, jobject jobj, jlong jhandle, jint jmax_merge_width) { + auto* opt = reinterpret_cast(jhandle); + opt->max_merge_width = static_cast(jmax_merge_width); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: maxMergeWidth + * Signature: (J)I + */ +jint Java_org_rocksdb_CompactionOptionsUniversal_maxMergeWidth( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->max_merge_width); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: setMaxSizeAmplificationPercent + * Signature: (JI)V + */ +void Java_org_rocksdb_CompactionOptionsUniversal_setMaxSizeAmplificationPercent( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jmax_size_amplification_percent) { + auto* opt = reinterpret_cast(jhandle); + opt->max_size_amplification_percent = + static_cast(jmax_size_amplification_percent); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: maxSizeAmplificationPercent + * Signature: (J)I + */ +jint Java_org_rocksdb_CompactionOptionsUniversal_maxSizeAmplificationPercent( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->max_size_amplification_percent); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: setCompressionSizePercent + * Signature: (JI)V + */ +void Java_org_rocksdb_CompactionOptionsUniversal_setCompressionSizePercent( + JNIEnv* env, jobject jobj, jlong jhandle, jint jcompression_size_percent) { + auto* opt = reinterpret_cast(jhandle); + opt->compression_size_percent = + static_cast(jcompression_size_percent); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: compressionSizePercent + * Signature: (J)I + */ +jint Java_org_rocksdb_CompactionOptionsUniversal_compressionSizePercent( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->compression_size_percent); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: setStopStyle + * Signature: (JB)V + */ +void Java_org_rocksdb_CompactionOptionsUniversal_setStopStyle( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jstop_style_value) { + auto* opt = reinterpret_cast(jhandle); + opt->stop_style = + rocksdb::CompactionStopStyleJni::toCppCompactionStopStyle( + jstop_style_value); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: stopStyle + * Signature: (J)B + */ +jbyte Java_org_rocksdb_CompactionOptionsUniversal_stopStyle( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return rocksdb::CompactionStopStyleJni::toJavaCompactionStopStyle( + opt->stop_style); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: setAllowTrivialMove + * Signature: (JZ)V + */ +void Java_org_rocksdb_CompactionOptionsUniversal_setAllowTrivialMove( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jallow_trivial_move) { + auto* opt = reinterpret_cast(jhandle); + opt->allow_trivial_move = static_cast(jallow_trivial_move); +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: allowTrivialMove + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_CompactionOptionsUniversal_allowTrivialMove( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return opt->allow_trivial_move; +} + +/* + * Class: org_rocksdb_CompactionOptionsUniversal + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_CompactionOptionsUniversal_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + delete reinterpret_cast(jhandle); +} diff --git a/java/rocksjni/comparator.cc b/java/rocksjni/comparator.cc new file mode 100644 index 00000000000..5955d0bf75e --- /dev/null +++ b/java/rocksjni/comparator.cc @@ -0,0 +1,68 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ for +// rocksdb::Comparator. + +#include +#include +#include +#include +#include + +#include "include/org_rocksdb_AbstractComparator.h" +#include "include/org_rocksdb_Comparator.h" +#include "include/org_rocksdb_DirectComparator.h" +#include "rocksjni/comparatorjnicallback.h" +#include "rocksjni/portal.h" + +// + +/* + * Class: org_rocksdb_Comparator + * Method: createNewComparator0 + * Signature: ()J + */ +jlong Java_org_rocksdb_Comparator_createNewComparator0( + JNIEnv* env, jobject jobj, jlong copt_handle) { + const rocksdb::ComparatorJniCallbackOptions* copt = + reinterpret_cast(copt_handle); + const rocksdb::ComparatorJniCallback* c = + new rocksdb::ComparatorJniCallback(env, jobj, copt); + return reinterpret_cast(c); +} +// + +// use_adaptive_mutex)), + mtx_findShortestSeparator(new port::Mutex(copt->use_adaptive_mutex)) { + // Note: Comparator methods may be accessed by multiple threads, + // so we ref the jvm not the env + const jint rs = env->GetJavaVM(&m_jvm); + if(rs != JNI_OK) { + // exception thrown + return; + } + + // Note: we want to access the Java Comparator instance + // across multiple method calls, so we create a global ref + assert(jComparator != nullptr); + m_jComparator = env->NewGlobalRef(jComparator); + if(m_jComparator == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + // Note: The name of a Comparator will not change during it's lifetime, + // so we cache it in a global var + jmethodID jNameMethodId = AbstractComparatorJni::getNameMethodId(env); + if(jNameMethodId == nullptr) { + // exception thrown: NoSuchMethodException or OutOfMemoryError + return; + } + jstring jsName = (jstring)env->CallObjectMethod(m_jComparator, jNameMethodId); + if(env->ExceptionCheck()) { + // exception thrown + return; + } + jboolean has_exception = JNI_FALSE; + m_name = JniUtil::copyString(env, jsName, + &has_exception); // also releases jsName + if (has_exception == JNI_TRUE) { + // exception thrown + return; + } + + m_jCompareMethodId = AbstractComparatorJni::getCompareMethodId(env); + if(m_jCompareMethodId == nullptr) { + // exception thrown: NoSuchMethodException or OutOfMemoryError + return; + } + + m_jFindShortestSeparatorMethodId = + AbstractComparatorJni::getFindShortestSeparatorMethodId(env); + if(m_jFindShortestSeparatorMethodId == nullptr) { + // exception thrown: NoSuchMethodException or OutOfMemoryError + return; + } + + m_jFindShortSuccessorMethodId = + AbstractComparatorJni::getFindShortSuccessorMethodId(env); + if(m_jFindShortSuccessorMethodId == nullptr) { + // exception thrown: NoSuchMethodException or OutOfMemoryError + return; + } +} + +const char* BaseComparatorJniCallback::Name() const { + return m_name.c_str(); +} + +int BaseComparatorJniCallback::Compare(const Slice& a, const Slice& b) const { + jboolean attached_thread = JNI_FALSE; + JNIEnv* env = JniUtil::getJniEnv(m_jvm, &attached_thread); + assert(env != nullptr); + + // TODO(adamretter): slice objects can potentially be cached using thread + // local variables to avoid locking. Could make this configurable depending on + // performance. + mtx_compare->Lock(); + + bool pending_exception = + AbstractSliceJni::setHandle(env, m_jSliceA, &a, JNI_FALSE); + if(pending_exception) { + if(env->ExceptionCheck()) { + // exception thrown from setHandle or descendant + env->ExceptionDescribe(); // print out exception to stderr + } + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return 0; + } + + pending_exception = + AbstractSliceJni::setHandle(env, m_jSliceB, &b, JNI_FALSE); + if(pending_exception) { + if(env->ExceptionCheck()) { + // exception thrown from setHandle or descendant + env->ExceptionDescribe(); // print out exception to stderr + } + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return 0; + } + + jint result = + env->CallIntMethod(m_jComparator, m_jCompareMethodId, m_jSliceA, + m_jSliceB); + + mtx_compare->Unlock(); + + if(env->ExceptionCheck()) { + // exception thrown from CallIntMethod + env->ExceptionDescribe(); // print out exception to stderr + result = 0; // we could not get a result from java callback so use 0 + } + + JniUtil::releaseJniEnv(m_jvm, attached_thread); + + return result; +} + +void BaseComparatorJniCallback::FindShortestSeparator( + std::string* start, const Slice& limit) const { + if (start == nullptr) { + return; + } + + jboolean attached_thread = JNI_FALSE; + JNIEnv* env = JniUtil::getJniEnv(m_jvm, &attached_thread); + assert(env != nullptr); + + const char* startUtf = start->c_str(); + jstring jsStart = env->NewStringUTF(startUtf); + if(jsStart == nullptr) { + // unable to construct string + if(env->ExceptionCheck()) { + env->ExceptionDescribe(); // print out exception to stderr + } + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + if(env->ExceptionCheck()) { + // exception thrown: OutOfMemoryError + env->ExceptionDescribe(); // print out exception to stderr + env->DeleteLocalRef(jsStart); + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + + // TODO(adamretter): slice object can potentially be cached using thread local + // variable to avoid locking. Could make this configurable depending on + // performance. + mtx_findShortestSeparator->Lock(); + + bool pending_exception = + AbstractSliceJni::setHandle(env, m_jSliceLimit, &limit, JNI_FALSE); + if(pending_exception) { + if(env->ExceptionCheck()) { + // exception thrown from setHandle or descendant + env->ExceptionDescribe(); // print out exception to stderr + } + if(jsStart != nullptr) { + env->DeleteLocalRef(jsStart); + } + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + + jstring jsResultStart = + (jstring)env->CallObjectMethod(m_jComparator, + m_jFindShortestSeparatorMethodId, jsStart, m_jSliceLimit); + + mtx_findShortestSeparator->Unlock(); + + if(env->ExceptionCheck()) { + // exception thrown from CallObjectMethod + env->ExceptionDescribe(); // print out exception to stderr + env->DeleteLocalRef(jsStart); + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + + env->DeleteLocalRef(jsStart); + + if (jsResultStart != nullptr) { + // update start with result + jboolean has_exception = JNI_FALSE; + std::string result = JniUtil::copyString(env, jsResultStart, + &has_exception); // also releases jsResultStart + if (has_exception == JNI_TRUE) { + if (env->ExceptionCheck()) { + env->ExceptionDescribe(); // print out exception to stderr + } + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + + *start = result; + } + + JniUtil::releaseJniEnv(m_jvm, attached_thread); +} + +void BaseComparatorJniCallback::FindShortSuccessor(std::string* key) const { + if (key == nullptr) { + return; + } + + jboolean attached_thread = JNI_FALSE; + JNIEnv* env = JniUtil::getJniEnv(m_jvm, &attached_thread); + assert(env != nullptr); + + const char* keyUtf = key->c_str(); + jstring jsKey = env->NewStringUTF(keyUtf); + if(jsKey == nullptr) { + // unable to construct string + if(env->ExceptionCheck()) { + env->ExceptionDescribe(); // print out exception to stderr + } + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } else if(env->ExceptionCheck()) { + // exception thrown: OutOfMemoryError + env->ExceptionDescribe(); // print out exception to stderr + env->DeleteLocalRef(jsKey); + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + + jstring jsResultKey = + (jstring)env->CallObjectMethod(m_jComparator, + m_jFindShortSuccessorMethodId, jsKey); + + if(env->ExceptionCheck()) { + // exception thrown from CallObjectMethod + env->ExceptionDescribe(); // print out exception to stderr + env->DeleteLocalRef(jsKey); + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + + env->DeleteLocalRef(jsKey); + + if (jsResultKey != nullptr) { + // updates key with result, also releases jsResultKey. + jboolean has_exception = JNI_FALSE; + std::string result = JniUtil::copyString(env, jsResultKey, &has_exception); + if (has_exception == JNI_TRUE) { + if (env->ExceptionCheck()) { + env->ExceptionDescribe(); // print out exception to stderr + } + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + + *key = result; + } + + JniUtil::releaseJniEnv(m_jvm, attached_thread); +} + +BaseComparatorJniCallback::~BaseComparatorJniCallback() { + jboolean attached_thread = JNI_FALSE; + JNIEnv* env = JniUtil::getJniEnv(m_jvm, &attached_thread); + assert(env != nullptr); + + if(m_jComparator != nullptr) { + env->DeleteGlobalRef(m_jComparator); + } + + JniUtil::releaseJniEnv(m_jvm, attached_thread); +} + +ComparatorJniCallback::ComparatorJniCallback( + JNIEnv* env, jobject jComparator, + const ComparatorJniCallbackOptions* copt) : + BaseComparatorJniCallback(env, jComparator, copt) { + m_jSliceA = env->NewGlobalRef(SliceJni::construct0(env)); + if(m_jSliceA == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + m_jSliceB = env->NewGlobalRef(SliceJni::construct0(env)); + if(m_jSliceB == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + m_jSliceLimit = env->NewGlobalRef(SliceJni::construct0(env)); + if(m_jSliceLimit == nullptr) { + // exception thrown: OutOfMemoryError + return; + } +} + +ComparatorJniCallback::~ComparatorJniCallback() { + jboolean attached_thread = JNI_FALSE; + JNIEnv* env = JniUtil::getJniEnv(m_jvm, &attached_thread); + assert(env != nullptr); + + if(m_jSliceA != nullptr) { + env->DeleteGlobalRef(m_jSliceA); + } + + if(m_jSliceB != nullptr) { + env->DeleteGlobalRef(m_jSliceB); + } + + if(m_jSliceLimit != nullptr) { + env->DeleteGlobalRef(m_jSliceLimit); + } + + JniUtil::releaseJniEnv(m_jvm, attached_thread); +} + +DirectComparatorJniCallback::DirectComparatorJniCallback( + JNIEnv* env, jobject jComparator, + const ComparatorJniCallbackOptions* copt) : + BaseComparatorJniCallback(env, jComparator, copt) { + m_jSliceA = env->NewGlobalRef(DirectSliceJni::construct0(env)); + if(m_jSliceA == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + m_jSliceB = env->NewGlobalRef(DirectSliceJni::construct0(env)); + if(m_jSliceB == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + m_jSliceLimit = env->NewGlobalRef(DirectSliceJni::construct0(env)); + if(m_jSliceLimit == nullptr) { + // exception thrown: OutOfMemoryError + return; + } +} + +DirectComparatorJniCallback::~DirectComparatorJniCallback() { + jboolean attached_thread = JNI_FALSE; + JNIEnv* env = JniUtil::getJniEnv(m_jvm, &attached_thread); + assert(env != nullptr); + + if(m_jSliceA != nullptr) { + env->DeleteGlobalRef(m_jSliceA); + } + + if(m_jSliceB != nullptr) { + env->DeleteGlobalRef(m_jSliceB); + } + + if(m_jSliceLimit != nullptr) { + env->DeleteGlobalRef(m_jSliceLimit); + } + + JniUtil::releaseJniEnv(m_jvm, attached_thread); +} +} // namespace rocksdb diff --git a/java/rocksjni/comparatorjnicallback.h b/java/rocksjni/comparatorjnicallback.h new file mode 100644 index 00000000000..a753008b338 --- /dev/null +++ b/java/rocksjni/comparatorjnicallback.h @@ -0,0 +1,94 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the callback "bridge" between Java and C++ for +// rocksdb::Comparator and rocksdb::DirectComparator. + +#ifndef JAVA_ROCKSJNI_COMPARATORJNICALLBACK_H_ +#define JAVA_ROCKSJNI_COMPARATORJNICALLBACK_H_ + +#include +#include +#include "rocksdb/comparator.h" +#include "rocksdb/slice.h" +#include "port/port.h" + +namespace rocksdb { + +struct ComparatorJniCallbackOptions { + // Use adaptive mutex, which spins in the user space before resorting + // to kernel. This could reduce context switch when the mutex is not + // heavily contended. However, if the mutex is hot, we could end up + // wasting spin time. + // Default: false + bool use_adaptive_mutex; + + ComparatorJniCallbackOptions() : use_adaptive_mutex(false) { + } +}; + +/** + * This class acts as a bridge between C++ + * and Java. The methods in this class will be + * called back from the RocksDB storage engine (C++) + * we then callback to the appropriate Java method + * this enables Comparators to be implemented in Java. + * + * The design of this Comparator caches the Java Slice + * objects that are used in the compare and findShortestSeparator + * method callbacks. Instead of creating new objects for each callback + * of those functions, by reuse via setHandle we are a lot + * faster; Unfortunately this means that we have to + * introduce independent locking in regions of each of those methods + * via the mutexs mtx_compare and mtx_findShortestSeparator respectively + */ +class BaseComparatorJniCallback : public Comparator { + public: + BaseComparatorJniCallback( + JNIEnv* env, jobject jComparator, + const ComparatorJniCallbackOptions* copt); + virtual ~BaseComparatorJniCallback(); + virtual const char* Name() const; + virtual int Compare(const Slice& a, const Slice& b) const; + virtual void FindShortestSeparator( + std::string* start, const Slice& limit) const; + virtual void FindShortSuccessor(std::string* key) const; + + private: + // used for synchronisation in compare method + port::Mutex* mtx_compare; + // used for synchronisation in findShortestSeparator method + port::Mutex* mtx_findShortestSeparator; + jobject m_jComparator; + std::string m_name; + jmethodID m_jCompareMethodId; + jmethodID m_jFindShortestSeparatorMethodId; + jmethodID m_jFindShortSuccessorMethodId; + + protected: + JavaVM* m_jvm; + jobject m_jSliceA; + jobject m_jSliceB; + jobject m_jSliceLimit; +}; + +class ComparatorJniCallback : public BaseComparatorJniCallback { + public: + ComparatorJniCallback( + JNIEnv* env, jobject jComparator, + const ComparatorJniCallbackOptions* copt); + ~ComparatorJniCallback(); +}; + +class DirectComparatorJniCallback : public BaseComparatorJniCallback { + public: + DirectComparatorJniCallback( + JNIEnv* env, jobject jComparator, + const ComparatorJniCallbackOptions* copt); + ~DirectComparatorJniCallback(); +}; +} // namespace rocksdb + +#endif // JAVA_ROCKSJNI_COMPARATORJNICALLBACK_H_ diff --git a/java/rocksjni/compression_options.cc b/java/rocksjni/compression_options.cc new file mode 100644 index 00000000000..7d5af645ae8 --- /dev/null +++ b/java/rocksjni/compression_options.cc @@ -0,0 +1,121 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ for +// rocksdb::CompressionOptions. + +#include + +#include "include/org_rocksdb_CompressionOptions.h" +#include "rocksdb/advanced_options.h" + +/* + * Class: org_rocksdb_CompressionOptions + * Method: newCompressionOptions + * Signature: ()J + */ +jlong Java_org_rocksdb_CompressionOptions_newCompressionOptions( + JNIEnv* env, jclass jcls) { + const auto* opt = new rocksdb::CompressionOptions(); + return reinterpret_cast(opt); +} + +/* + * Class: org_rocksdb_CompressionOptions + * Method: setWindowBits + * Signature: (JI)V + */ +void Java_org_rocksdb_CompressionOptions_setWindowBits( + JNIEnv* env, jobject jobj, jlong jhandle, jint jwindow_bits) { + auto* opt = reinterpret_cast(jhandle); + opt->window_bits = static_cast(jwindow_bits); +} + +/* + * Class: org_rocksdb_CompressionOptions + * Method: windowBits + * Signature: (J)I + */ +jint Java_org_rocksdb_CompressionOptions_windowBits( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->window_bits); +} + +/* + * Class: org_rocksdb_CompressionOptions + * Method: setLevel + * Signature: (JI)V + */ +void Java_org_rocksdb_CompressionOptions_setLevel( + JNIEnv* env, jobject jobj, jlong jhandle, jint jlevel) { + auto* opt = reinterpret_cast(jhandle); + opt->level = static_cast(jlevel); +} + +/* + * Class: org_rocksdb_CompressionOptions + * Method: level + * Signature: (J)I + */ +jint Java_org_rocksdb_CompressionOptions_level( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->level); +} + +/* + * Class: org_rocksdb_CompressionOptions + * Method: setStrategy + * Signature: (JI)V + */ +void Java_org_rocksdb_CompressionOptions_setStrategy( + JNIEnv* env, jobject jobj, jlong jhandle, jint jstrategy) { + auto* opt = reinterpret_cast(jhandle); + opt->strategy = static_cast(jstrategy); +} + +/* + * Class: org_rocksdb_CompressionOptions + * Method: strategy + * Signature: (J)I + */ +jint Java_org_rocksdb_CompressionOptions_strategy( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->strategy); +} + +/* + * Class: org_rocksdb_CompressionOptions + * Method: setMaxDictBytes + * Signature: (JI)V + */ +void Java_org_rocksdb_CompressionOptions_setMaxDictBytes( + JNIEnv* env, jobject jobj, jlong jhandle, jint jmax_dict_bytes) { + auto* opt = reinterpret_cast(jhandle); + opt->max_dict_bytes = static_cast(jmax_dict_bytes); +} + +/* + * Class: org_rocksdb_CompressionOptions + * Method: maxDictBytes + * Signature: (J)I + */ +jint Java_org_rocksdb_CompressionOptions_maxDictBytes( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->max_dict_bytes); +} + +/* + * Class: org_rocksdb_CompressionOptions + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_CompressionOptions_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + delete reinterpret_cast(jhandle); +} diff --git a/java/rocksjni/env.cc b/java/rocksjni/env.cc index 3aed9f5a082..dc949a07fa0 100644 --- a/java/rocksjni/env.cc +++ b/java/rocksjni/env.cc @@ -1,49 +1,51 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ and enables // calling c++ rocksdb::Env methods from Java side. +#include "include/org_rocksdb_Env.h" #include "include/org_rocksdb_RocksEnv.h" +#include "include/org_rocksdb_RocksMemEnv.h" #include "rocksdb/env.h" /* - * Class: org_rocksdb_RocksEnv + * Class: org_rocksdb_Env * Method: getDefaultEnvInternal * Signature: ()J */ -jlong Java_org_rocksdb_RocksEnv_getDefaultEnvInternal( - JNIEnv* env, jclass jclass) { +jlong Java_org_rocksdb_Env_getDefaultEnvInternal( + JNIEnv* env, jclass jclazz) { return reinterpret_cast(rocksdb::Env::Default()); } /* - * Class: org_rocksdb_RocksEnv + * Class: org_rocksdb_Env * Method: setBackgroundThreads * Signature: (JII)V */ -void Java_org_rocksdb_RocksEnv_setBackgroundThreads( +void Java_org_rocksdb_Env_setBackgroundThreads( JNIEnv* env, jobject jobj, jlong jhandle, jint num, jint priority) { auto* rocks_env = reinterpret_cast(jhandle); switch (priority) { - case org_rocksdb_RocksEnv_FLUSH_POOL: + case org_rocksdb_Env_FLUSH_POOL: rocks_env->SetBackgroundThreads(num, rocksdb::Env::Priority::LOW); break; - case org_rocksdb_RocksEnv_COMPACTION_POOL: + case org_rocksdb_Env_COMPACTION_POOL: rocks_env->SetBackgroundThreads(num, rocksdb::Env::Priority::HIGH); break; } } /* - * Class: org_rocksdb_RocksEnv + * Class: org_rocksdb_sEnv * Method: getThreadPoolQueueLen * Signature: (JI)I */ -jint Java_org_rocksdb_RocksEnv_getThreadPoolQueueLen( +jint Java_org_rocksdb_Env_getThreadPoolQueueLen( JNIEnv* env, jobject jobj, jlong jhandle, jint pool_id) { auto* rocks_env = reinterpret_cast(jhandle); switch (pool_id) { @@ -56,11 +58,24 @@ jint Java_org_rocksdb_RocksEnv_getThreadPoolQueueLen( } /* - * Class: org_rocksdb_RocksEnv + * Class: org_rocksdb_RocksMemEnv + * Method: createMemEnv + * Signature: ()J + */ +jlong Java_org_rocksdb_RocksMemEnv_createMemEnv( + JNIEnv* env, jclass jclazz) { + return reinterpret_cast(rocksdb::NewMemEnv( + rocksdb::Env::Default())); +} + +/* + * Class: org_rocksdb_RocksMemEnv * Method: disposeInternal * Signature: (J)V */ -void Java_org_rocksdb_RocksEnv_disposeInternal( +void Java_org_rocksdb_RocksMemEnv_disposeInternal( JNIEnv* env, jobject jobj, jlong jhandle) { - delete reinterpret_cast(jhandle); + auto* e = reinterpret_cast(jhandle); + assert(e != nullptr); + delete e; } diff --git a/java/rocksjni/env_options.cc b/java/rocksjni/env_options.cc new file mode 100644 index 00000000000..538b0b69f74 --- /dev/null +++ b/java/rocksjni/env_options.cc @@ -0,0 +1,297 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ and enables +// calling C++ rocksdb::EnvOptions methods +// from Java side. + +#include + +#include "include/org_rocksdb_EnvOptions.h" +#include "rocksdb/env.h" + +#define ENV_OPTIONS_SET_BOOL(_jhandle, _opt) \ + reinterpret_cast(_jhandle)->_opt = \ + static_cast(_opt) + +#define ENV_OPTIONS_SET_SIZE_T(_jhandle, _opt) \ + reinterpret_cast(_jhandle)->_opt = \ + static_cast(_opt) + +#define ENV_OPTIONS_SET_UINT64_T(_jhandle, _opt) \ + reinterpret_cast(_jhandle)->_opt = \ + static_cast(_opt) + +#define ENV_OPTIONS_GET(_jhandle, _opt) \ + reinterpret_cast(_jhandle)->_opt + +/* + * Class: org_rocksdb_EnvOptions + * Method: newEnvOptions + * Signature: ()J + */ +jlong Java_org_rocksdb_EnvOptions_newEnvOptions(JNIEnv *env, jclass jcls) { + auto *env_opt = new rocksdb::EnvOptions(); + return reinterpret_cast(env_opt); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_EnvOptions_disposeInternal(JNIEnv *env, jobject jobj, + jlong jhandle) { + auto* eo = reinterpret_cast(jhandle); + assert(eo != nullptr); + delete eo; +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setUseDirectReads + * Signature: (JZ)V + */ +void Java_org_rocksdb_EnvOptions_setUseDirectReads(JNIEnv *env, jobject jobj, + jlong jhandle, + jboolean use_direct_reads) { + ENV_OPTIONS_SET_BOOL(jhandle, use_direct_reads); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: useDirectReads + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_EnvOptions_useDirectReads(JNIEnv *env, jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, use_direct_reads); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setUseDirectWrites + * Signature: (JZ)V + */ +void Java_org_rocksdb_EnvOptions_setUseDirectWrites( + JNIEnv *env, jobject jobj, jlong jhandle, jboolean use_direct_writes) { + ENV_OPTIONS_SET_BOOL(jhandle, use_direct_writes); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: useDirectWrites + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_EnvOptions_useDirectWrites(JNIEnv *env, jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, use_direct_writes); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setUseMmapReads + * Signature: (JZ)V + */ +void Java_org_rocksdb_EnvOptions_setUseMmapReads(JNIEnv *env, jobject jobj, + jlong jhandle, + jboolean use_mmap_reads) { + ENV_OPTIONS_SET_BOOL(jhandle, use_mmap_reads); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: useMmapReads + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_EnvOptions_useMmapReads(JNIEnv *env, jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, use_mmap_reads); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setUseMmapWrites + * Signature: (JZ)V + */ +void Java_org_rocksdb_EnvOptions_setUseMmapWrites(JNIEnv *env, jobject jobj, + jlong jhandle, + jboolean use_mmap_writes) { + ENV_OPTIONS_SET_BOOL(jhandle, use_mmap_writes); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: useMmapWrites + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_EnvOptions_useMmapWrites(JNIEnv *env, jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, use_mmap_writes); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setAllowFallocate + * Signature: (JZ)V + */ +void Java_org_rocksdb_EnvOptions_setAllowFallocate(JNIEnv *env, jobject jobj, + jlong jhandle, + jboolean allow_fallocate) { + ENV_OPTIONS_SET_BOOL(jhandle, allow_fallocate); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: allowFallocate + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_EnvOptions_allowFallocate(JNIEnv *env, jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, allow_fallocate); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setSetFdCloexec + * Signature: (JZ)V + */ +void Java_org_rocksdb_EnvOptions_setSetFdCloexec(JNIEnv *env, jobject jobj, + jlong jhandle, + jboolean set_fd_cloexec) { + ENV_OPTIONS_SET_BOOL(jhandle, set_fd_cloexec); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setFdCloexec + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_EnvOptions_setFdCloexec(JNIEnv *env, jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, set_fd_cloexec); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setBytesPerSync + * Signature: (JJ)V + */ +void Java_org_rocksdb_EnvOptions_setBytesPerSync(JNIEnv *env, jobject jobj, + jlong jhandle, + jlong bytes_per_sync) { + ENV_OPTIONS_SET_UINT64_T(jhandle, bytes_per_sync); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: bytesPerSync + * Signature: (J)J + */ +jlong Java_org_rocksdb_EnvOptions_bytesPerSync(JNIEnv *env, jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, bytes_per_sync); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setFallocateWithKeepSize + * Signature: (JZ)V + */ +void Java_org_rocksdb_EnvOptions_setFallocateWithKeepSize( + JNIEnv *env, jobject jobj, jlong jhandle, + jboolean fallocate_with_keep_size) { + ENV_OPTIONS_SET_BOOL(jhandle, fallocate_with_keep_size); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: fallocateWithKeepSize + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_EnvOptions_fallocateWithKeepSize(JNIEnv *env, + jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, fallocate_with_keep_size); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setCompactionReadaheadSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_EnvOptions_setCompactionReadaheadSize( + JNIEnv *env, jobject jobj, jlong jhandle, jlong compaction_readahead_size) { + ENV_OPTIONS_SET_SIZE_T(jhandle, compaction_readahead_size); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: compactionReadaheadSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_EnvOptions_compactionReadaheadSize(JNIEnv *env, + jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, compaction_readahead_size); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setRandomAccessMaxBufferSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_EnvOptions_setRandomAccessMaxBufferSize( + JNIEnv *env, jobject jobj, jlong jhandle, + jlong random_access_max_buffer_size) { + ENV_OPTIONS_SET_SIZE_T(jhandle, random_access_max_buffer_size); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: randomAccessMaxBufferSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_EnvOptions_randomAccessMaxBufferSize(JNIEnv *env, + jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, random_access_max_buffer_size); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setWritableFileMaxBufferSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_EnvOptions_setWritableFileMaxBufferSize( + JNIEnv *env, jobject jobj, jlong jhandle, + jlong writable_file_max_buffer_size) { + ENV_OPTIONS_SET_SIZE_T(jhandle, writable_file_max_buffer_size); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: writableFileMaxBufferSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_EnvOptions_writableFileMaxBufferSize(JNIEnv *env, + jobject jobj, + jlong jhandle) { + return ENV_OPTIONS_GET(jhandle, writable_file_max_buffer_size); +} + +/* + * Class: org_rocksdb_EnvOptions + * Method: setRateLimiter + * Signature: (JJ)V + */ +void Java_org_rocksdb_EnvOptions_setRateLimiter(JNIEnv *env, jobject jobj, + jlong jhandle, + jlong rl_handle) { + auto* sptr_rate_limiter = + reinterpret_cast *>(rl_handle); + auto* env_opt = reinterpret_cast(jhandle); + env_opt->rate_limiter = sptr_rate_limiter->get(); +} diff --git a/java/rocksjni/filter.cc b/java/rocksjni/filter.cc index 572b4a66d46..7b186b8943c 100644 --- a/java/rocksjni/filter.cc +++ b/java/rocksjni/filter.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ for // rocksdb::FilterPolicy. @@ -18,13 +18,16 @@ /* * Class: org_rocksdb_BloomFilter - * Method: createNewFilter0 - * Signature: (I)V + * Method: createBloomFilter + * Signature: (IZ)J */ -void Java_org_rocksdb_BloomFilter_createNewFilter0( - JNIEnv* env, jobject jobj, jint bits_per_key) { - const rocksdb::FilterPolicy* fp = rocksdb::NewBloomFilterPolicy(bits_per_key); - rocksdb::FilterJni::setHandle(env, jobj, fp); +jlong Java_org_rocksdb_BloomFilter_createNewBloomFilter( + JNIEnv* env, jclass jcls, jint bits_per_key, + jboolean use_block_base_builder) { + auto* sptr_filter = + new std::shared_ptr( + rocksdb::NewBloomFilterPolicy(bits_per_key, use_block_base_builder)); + return reinterpret_cast(sptr_filter); } /* @@ -33,6 +36,8 @@ void Java_org_rocksdb_BloomFilter_createNewFilter0( * Signature: (J)V */ void Java_org_rocksdb_Filter_disposeInternal( - JNIEnv* env, jobject jobj, jlong handle) { - delete reinterpret_cast(handle); + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* handle = + reinterpret_cast *>(jhandle); + delete handle; // delete std::shared_ptr } diff --git a/java/rocksjni/ingest_external_file_options.cc b/java/rocksjni/ingest_external_file_options.cc new file mode 100644 index 00000000000..251a6e3c627 --- /dev/null +++ b/java/rocksjni/ingest_external_file_options.cc @@ -0,0 +1,149 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ for +// rocksdb::FilterPolicy. + +#include + +#include "include/org_rocksdb_IngestExternalFileOptions.h" +#include "rocksdb/options.h" + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: newIngestExternalFileOptions + * Signature: ()J + */ +jlong Java_org_rocksdb_IngestExternalFileOptions_newIngestExternalFileOptions__( + JNIEnv* env, jclass jclazz) { + auto* options = new rocksdb::IngestExternalFileOptions(); + return reinterpret_cast(options); +} + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: newIngestExternalFileOptions + * Signature: (ZZZZ)J + */ +jlong Java_org_rocksdb_IngestExternalFileOptions_newIngestExternalFileOptions__ZZZZ( + JNIEnv* env, jclass jcls, jboolean jmove_files, + jboolean jsnapshot_consistency, jboolean jallow_global_seqno, + jboolean jallow_blocking_flush) { + auto* options = new rocksdb::IngestExternalFileOptions(); + options->move_files = static_cast(jmove_files); + options->snapshot_consistency = static_cast(jsnapshot_consistency); + options->allow_global_seqno = static_cast(jallow_global_seqno); + options->allow_blocking_flush = static_cast(jallow_blocking_flush); + return reinterpret_cast(options); +} + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: moveFiles + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_IngestExternalFileOptions_moveFiles( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* options = + reinterpret_cast(jhandle); + return static_cast(options->move_files); +} + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: setMoveFiles + * Signature: (JZ)V + */ +void Java_org_rocksdb_IngestExternalFileOptions_setMoveFiles( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jmove_files) { + auto* options = + reinterpret_cast(jhandle); + options->move_files = static_cast(jmove_files); +} + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: snapshotConsistency + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_IngestExternalFileOptions_snapshotConsistency( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* options = + reinterpret_cast(jhandle); + return static_cast(options->snapshot_consistency); +} + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: setSnapshotConsistency + * Signature: (JZ)V + */ +void Java_org_rocksdb_IngestExternalFileOptions_setSnapshotConsistency( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jsnapshot_consistency) { + auto* options = + reinterpret_cast(jhandle); + options->snapshot_consistency = static_cast(jsnapshot_consistency); +} + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: allowGlobalSeqNo + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_IngestExternalFileOptions_allowGlobalSeqNo( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* options = + reinterpret_cast(jhandle); + return static_cast(options->allow_global_seqno); +} + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: setAllowGlobalSeqNo + * Signature: (JZ)V + */ +void Java_org_rocksdb_IngestExternalFileOptions_setAllowGlobalSeqNo( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jallow_global_seqno) { + auto* options = + reinterpret_cast(jhandle); + options->allow_global_seqno = static_cast(jallow_global_seqno); +} + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: allowBlockingFlush + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_IngestExternalFileOptions_allowBlockingFlush( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* options = + reinterpret_cast(jhandle); + return static_cast(options->allow_blocking_flush); +} + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: setAllowBlockingFlush + * Signature: (JZ)V + */ +void Java_org_rocksdb_IngestExternalFileOptions_setAllowBlockingFlush( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jallow_blocking_flush) { + auto* options = + reinterpret_cast(jhandle); + options->allow_blocking_flush = static_cast(jallow_blocking_flush); +} + +/* + * Class: org_rocksdb_IngestExternalFileOptions + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_IngestExternalFileOptions_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* options = + reinterpret_cast(jhandle); + delete options; +} \ No newline at end of file diff --git a/java/rocksjni/iterator.cc b/java/rocksjni/iterator.cc index 84b0b313327..3ac9d5033f2 100644 --- a/java/rocksjni/iterator.cc +++ b/java/rocksjni/iterator.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ and enables // calling c++ rocksdb::Iterator methods from Java side. @@ -14,6 +14,18 @@ #include "rocksjni/portal.h" #include "rocksdb/iterator.h" +/* + * Class: org_rocksdb_RocksIterator + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_RocksIterator_disposeInternal( + JNIEnv* env, jobject jobj, jlong handle) { + auto* it = reinterpret_cast(handle); + assert(it != nullptr); + delete it; +} + /* * Class: org_rocksdb_RocksIterator * Method: isValid0 @@ -36,7 +48,7 @@ void Java_org_rocksdb_RocksIterator_seekToFirst0( /* * Class: org_rocksdb_RocksIterator - * Method: seekToFirst0 + * Method: seekToLast0 * Signature: (J)V */ void Java_org_rocksdb_RocksIterator_seekToLast0( @@ -46,7 +58,7 @@ void Java_org_rocksdb_RocksIterator_seekToLast0( /* * Class: org_rocksdb_RocksIterator - * Method: seekToLast0 + * Method: next0 * Signature: (J)V */ void Java_org_rocksdb_RocksIterator_next0( @@ -56,7 +68,7 @@ void Java_org_rocksdb_RocksIterator_next0( /* * Class: org_rocksdb_RocksIterator - * Method: next0 + * Method: prev0 * Signature: (J)V */ void Java_org_rocksdb_RocksIterator_prev0( @@ -66,51 +78,22 @@ void Java_org_rocksdb_RocksIterator_prev0( /* * Class: org_rocksdb_RocksIterator - * Method: prev0 - * Signature: (J)V - */ -jbyteArray Java_org_rocksdb_RocksIterator_key0( - JNIEnv* env, jobject jobj, jlong handle) { - auto it = reinterpret_cast(handle); - rocksdb::Slice key_slice = it->key(); - - jbyteArray jkey = env->NewByteArray(key_slice.size()); - env->SetByteArrayRegion( - jkey, 0, key_slice.size(), - reinterpret_cast(key_slice.data())); - return jkey; -} - -/* - * Class: org_rocksdb_RocksIterator - * Method: key0 - * Signature: (J)[B - */ -jbyteArray Java_org_rocksdb_RocksIterator_value0( - JNIEnv* env, jobject jobj, jlong handle) { - auto it = reinterpret_cast(handle); - rocksdb::Slice value_slice = it->value(); - - jbyteArray jvalue = env->NewByteArray(value_slice.size()); - env->SetByteArrayRegion( - jvalue, 0, value_slice.size(), - reinterpret_cast(value_slice.data())); - return jvalue; -} - -/* - * Class: org_rocksdb_RocksIterator - * Method: value0 - * Signature: (J)[B + * Method: seek0 + * Signature: (J[BI)V */ void Java_org_rocksdb_RocksIterator_seek0( JNIEnv* env, jobject jobj, jlong handle, jbyteArray jtarget, jint jtarget_len) { - auto it = reinterpret_cast(handle); - jbyte* target = env->GetByteArrayElements(jtarget, 0); + jbyte* target = env->GetByteArrayElements(jtarget, nullptr); + if(target == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + rocksdb::Slice target_slice( reinterpret_cast(target), jtarget_len); + auto* it = reinterpret_cast(handle); it->Seek(target_slice); env->ReleaseByteArrayElements(jtarget, target, JNI_ABORT); @@ -118,12 +101,12 @@ void Java_org_rocksdb_RocksIterator_seek0( /* * Class: org_rocksdb_RocksIterator - * Method: seek0 - * Signature: (J[BI)V + * Method: status0 + * Signature: (J)V */ void Java_org_rocksdb_RocksIterator_status0( JNIEnv* env, jobject jobj, jlong handle) { - auto it = reinterpret_cast(handle); + auto* it = reinterpret_cast(handle); rocksdb::Status s = it->status(); if (s.ok()) { @@ -135,11 +118,41 @@ void Java_org_rocksdb_RocksIterator_status0( /* * Class: org_rocksdb_RocksIterator - * Method: disposeInternal - * Signature: (J)V + * Method: key0 + * Signature: (J)[B */ -void Java_org_rocksdb_RocksIterator_disposeInternal( +jbyteArray Java_org_rocksdb_RocksIterator_key0( JNIEnv* env, jobject jobj, jlong handle) { - auto it = reinterpret_cast(handle); - delete it; + auto* it = reinterpret_cast(handle); + rocksdb::Slice key_slice = it->key(); + + jbyteArray jkey = env->NewByteArray(static_cast(key_slice.size())); + if(jkey == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + env->SetByteArrayRegion(jkey, 0, static_cast(key_slice.size()), + const_cast(reinterpret_cast(key_slice.data()))); + return jkey; +} + +/* + * Class: org_rocksdb_RocksIterator + * Method: value0 + * Signature: (J)[B + */ +jbyteArray Java_org_rocksdb_RocksIterator_value0( + JNIEnv* env, jobject jobj, jlong handle) { + auto* it = reinterpret_cast(handle); + rocksdb::Slice value_slice = it->value(); + + jbyteArray jkeyValue = + env->NewByteArray(static_cast(value_slice.size())); + if(jkeyValue == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + env->SetByteArrayRegion(jkeyValue, 0, static_cast(value_slice.size()), + const_cast(reinterpret_cast(value_slice.data()))); + return jkeyValue; } diff --git a/java/rocksjni/loggerjnicallback.cc b/java/rocksjni/loggerjnicallback.cc new file mode 100644 index 00000000000..09140ed709a --- /dev/null +++ b/java/rocksjni/loggerjnicallback.cc @@ -0,0 +1,313 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the callback "bridge" between Java and C++ for +// rocksdb::Logger. + +#include "include/org_rocksdb_Logger.h" + +#include "rocksjni/loggerjnicallback.h" +#include "rocksjni/portal.h" +#include +#include + +namespace rocksdb { + +LoggerJniCallback::LoggerJniCallback( + JNIEnv* env, jobject jlogger) { + // Note: Logger methods may be accessed by multiple threads, + // so we ref the jvm not the env + const jint rs = env->GetJavaVM(&m_jvm); + if(rs != JNI_OK) { + // exception thrown + return; + } + + // Note: we want to access the Java Logger instance + // across multiple method calls, so we create a global ref + assert(jlogger != nullptr); + m_jLogger = env->NewGlobalRef(jlogger); + if(m_jLogger == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + m_jLogMethodId = LoggerJni::getLogMethodId(env); + if(m_jLogMethodId == nullptr) { + // exception thrown: NoSuchMethodException or OutOfMemoryError + return; + } + + jobject jdebug_level = InfoLogLevelJni::DEBUG_LEVEL(env); + if(jdebug_level == nullptr) { + // exception thrown: NoSuchFieldError, ExceptionInInitializerError + // or OutOfMemoryError + return; + } + m_jdebug_level = env->NewGlobalRef(jdebug_level); + if(m_jdebug_level == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + jobject jinfo_level = InfoLogLevelJni::INFO_LEVEL(env); + if(jinfo_level == nullptr) { + // exception thrown: NoSuchFieldError, ExceptionInInitializerError + // or OutOfMemoryError + return; + } + m_jinfo_level = env->NewGlobalRef(jinfo_level); + if(m_jinfo_level == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + jobject jwarn_level = InfoLogLevelJni::WARN_LEVEL(env); + if(jwarn_level == nullptr) { + // exception thrown: NoSuchFieldError, ExceptionInInitializerError + // or OutOfMemoryError + return; + } + m_jwarn_level = env->NewGlobalRef(jwarn_level); + if(m_jwarn_level == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + jobject jerror_level = InfoLogLevelJni::ERROR_LEVEL(env); + if(jerror_level == nullptr) { + // exception thrown: NoSuchFieldError, ExceptionInInitializerError + // or OutOfMemoryError + return; + } + m_jerror_level = env->NewGlobalRef(jerror_level); + if(m_jerror_level == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + jobject jfatal_level = InfoLogLevelJni::FATAL_LEVEL(env); + if(jfatal_level == nullptr) { + // exception thrown: NoSuchFieldError, ExceptionInInitializerError + // or OutOfMemoryError + return; + } + m_jfatal_level = env->NewGlobalRef(jfatal_level); + if(m_jfatal_level == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + jobject jheader_level = InfoLogLevelJni::HEADER_LEVEL(env); + if(jheader_level == nullptr) { + // exception thrown: NoSuchFieldError, ExceptionInInitializerError + // or OutOfMemoryError + return; + } + m_jheader_level = env->NewGlobalRef(jheader_level); + if(m_jheader_level == nullptr) { + // exception thrown: OutOfMemoryError + return; + } +} + +void LoggerJniCallback::Logv(const char* format, va_list ap) { + // We implement this method because it is virtual but we don't + // use it because we need to know about the log level. +} + +void LoggerJniCallback::Logv(const InfoLogLevel log_level, + const char* format, va_list ap) { + if (GetInfoLogLevel() <= log_level) { + + // determine InfoLogLevel java enum instance + jobject jlog_level; + switch (log_level) { + case rocksdb::InfoLogLevel::DEBUG_LEVEL: + jlog_level = m_jdebug_level; + break; + case rocksdb::InfoLogLevel::INFO_LEVEL: + jlog_level = m_jinfo_level; + break; + case rocksdb::InfoLogLevel::WARN_LEVEL: + jlog_level = m_jwarn_level; + break; + case rocksdb::InfoLogLevel::ERROR_LEVEL: + jlog_level = m_jerror_level; + break; + case rocksdb::InfoLogLevel::FATAL_LEVEL: + jlog_level = m_jfatal_level; + break; + case rocksdb::InfoLogLevel::HEADER_LEVEL: + jlog_level = m_jheader_level; + break; + default: + jlog_level = m_jfatal_level; + break; + } + + assert(format != nullptr); + assert(ap != nullptr); + const std::unique_ptr msg = format_str(format, ap); + + // pass msg to java callback handler + jboolean attached_thread = JNI_FALSE; + JNIEnv* env = JniUtil::getJniEnv(m_jvm, &attached_thread); + assert(env != nullptr); + + jstring jmsg = env->NewStringUTF(msg.get()); + if(jmsg == nullptr) { + // unable to construct string + if(env->ExceptionCheck()) { + env->ExceptionDescribe(); // print out exception to stderr + } + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + if(env->ExceptionCheck()) { + // exception thrown: OutOfMemoryError + env->ExceptionDescribe(); // print out exception to stderr + env->DeleteLocalRef(jmsg); + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + + env->CallVoidMethod(m_jLogger, m_jLogMethodId, jlog_level, jmsg); + if(env->ExceptionCheck()) { + // exception thrown + env->ExceptionDescribe(); // print out exception to stderr + env->DeleteLocalRef(jmsg); + JniUtil::releaseJniEnv(m_jvm, attached_thread); + return; + } + + env->DeleteLocalRef(jmsg); + JniUtil::releaseJniEnv(m_jvm, attached_thread); + } +} + +std::unique_ptr LoggerJniCallback::format_str(const char* format, va_list ap) const { + va_list ap_copy; + + va_copy(ap_copy, ap); + const size_t required = vsnprintf(nullptr, 0, format, ap_copy) + 1; // Extra space for '\0' + va_end(ap_copy); + + std::unique_ptr buf(new char[required]); + + va_copy(ap_copy, ap); + vsnprintf(buf.get(), required, format, ap_copy); + va_end(ap_copy); + + return buf; +} + +LoggerJniCallback::~LoggerJniCallback() { + jboolean attached_thread = JNI_FALSE; + JNIEnv* env = JniUtil::getJniEnv(m_jvm, &attached_thread); + assert(env != nullptr); + + if(m_jLogger != nullptr) { + env->DeleteGlobalRef(m_jLogger); + } + + if(m_jdebug_level != nullptr) { + env->DeleteGlobalRef(m_jdebug_level); + } + + if(m_jinfo_level != nullptr) { + env->DeleteGlobalRef(m_jinfo_level); + } + + if(m_jwarn_level != nullptr) { + env->DeleteGlobalRef(m_jwarn_level); + } + + if(m_jerror_level != nullptr) { + env->DeleteGlobalRef(m_jerror_level); + } + + if(m_jfatal_level != nullptr) { + env->DeleteGlobalRef(m_jfatal_level); + } + + if(m_jheader_level != nullptr) { + env->DeleteGlobalRef(m_jheader_level); + } + + JniUtil::releaseJniEnv(m_jvm, attached_thread); +} + +} // namespace rocksdb + +/* + * Class: org_rocksdb_Logger + * Method: createNewLoggerOptions + * Signature: (J)J + */ +jlong Java_org_rocksdb_Logger_createNewLoggerOptions( + JNIEnv* env, jobject jobj, jlong joptions) { + auto* sptr_logger = new std::shared_ptr( + new rocksdb::LoggerJniCallback(env, jobj)); + + // set log level + auto* options = reinterpret_cast(joptions); + sptr_logger->get()->SetInfoLogLevel(options->info_log_level); + + return reinterpret_cast(sptr_logger); +} + +/* + * Class: org_rocksdb_Logger + * Method: createNewLoggerDbOptions + * Signature: (J)J + */ +jlong Java_org_rocksdb_Logger_createNewLoggerDbOptions( + JNIEnv* env, jobject jobj, jlong jdb_options) { + auto* sptr_logger = new std::shared_ptr( + new rocksdb::LoggerJniCallback(env, jobj)); + + // set log level + auto* db_options = reinterpret_cast(jdb_options); + sptr_logger->get()->SetInfoLogLevel(db_options->info_log_level); + + return reinterpret_cast(sptr_logger); +} + +/* + * Class: org_rocksdb_Logger + * Method: setInfoLogLevel + * Signature: (JB)V + */ +void Java_org_rocksdb_Logger_setInfoLogLevel( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jlog_level) { + auto* handle = + reinterpret_cast *>(jhandle); + handle->get()-> + SetInfoLogLevel(static_cast(jlog_level)); +} + +/* + * Class: org_rocksdb_Logger + * Method: infoLogLevel + * Signature: (J)B + */ +jbyte Java_org_rocksdb_Logger_infoLogLevel( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* handle = + reinterpret_cast *>(jhandle); + return static_cast(handle->get()->GetInfoLogLevel()); +} + +/* + * Class: org_rocksdb_Logger + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_Logger_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* handle = + reinterpret_cast *>(jhandle); + delete handle; // delete std::shared_ptr +} diff --git a/java/rocksjni/loggerjnicallback.h b/java/rocksjni/loggerjnicallback.h new file mode 100644 index 00000000000..2db85975d66 --- /dev/null +++ b/java/rocksjni/loggerjnicallback.h @@ -0,0 +1,50 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the callback "bridge" between Java and C++ for +// rocksdb::Logger + +#ifndef JAVA_ROCKSJNI_LOGGERJNICALLBACK_H_ +#define JAVA_ROCKSJNI_LOGGERJNICALLBACK_H_ + +#include +#include +#include +#include "port/port.h" +#include "rocksdb/env.h" + +namespace rocksdb { + + class LoggerJniCallback : public Logger { + public: + LoggerJniCallback(JNIEnv* env, jobject jLogger); + virtual ~LoggerJniCallback(); + + using Logger::SetInfoLogLevel; + using Logger::GetInfoLogLevel; + // Write an entry to the log file with the specified format. + virtual void Logv(const char* format, va_list ap); + // Write an entry to the log file with the specified log level + // and format. Any log with level under the internal log level + // of *this (see @SetInfoLogLevel and @GetInfoLogLevel) will not be + // printed. + virtual void Logv(const InfoLogLevel log_level, + const char* format, va_list ap); + + private: + JavaVM* m_jvm; + jobject m_jLogger; + jmethodID m_jLogMethodId; + jobject m_jdebug_level; + jobject m_jinfo_level; + jobject m_jwarn_level; + jobject m_jerror_level; + jobject m_jfatal_level; + jobject m_jheader_level; + std::unique_ptr format_str(const char* format, va_list ap) const; + }; +} // namespace rocksdb + +#endif // JAVA_ROCKSJNI_LOGGERJNICALLBACK_H_ diff --git a/java/rocksjni/lru_cache.cc b/java/rocksjni/lru_cache.cc new file mode 100644 index 00000000000..16582689e79 --- /dev/null +++ b/java/rocksjni/lru_cache.cc @@ -0,0 +1,41 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ for +// rocksdb::LRUCache. + +#include + +#include "cache/lru_cache.h" +#include "include/org_rocksdb_LRUCache.h" + +/* + * Class: org_rocksdb_LRUCache + * Method: newLRUCache + * Signature: (JIZD)J + */ +jlong Java_org_rocksdb_LRUCache_newLRUCache( + JNIEnv* env, jclass jcls, jlong jcapacity, jint jnum_shard_bits, + jboolean jstrict_capacity_limit, jdouble jhigh_pri_pool_ratio) { + auto* sptr_lru_cache = + new std::shared_ptr(rocksdb::NewLRUCache( + static_cast(jcapacity), + static_cast(jnum_shard_bits), + static_cast(jstrict_capacity_limit), + static_cast(jhigh_pri_pool_ratio))); + return reinterpret_cast(sptr_lru_cache); +} + +/* + * Class: org_rocksdb_LRUCache + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_LRUCache_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* sptr_lru_cache = + reinterpret_cast *>(jhandle); + delete sptr_lru_cache; // delete std::shared_ptr +} diff --git a/java/rocksjni/memtablejni.cc b/java/rocksjni/memtablejni.cc index a0d50f5f5e0..56a04f9f814 100644 --- a/java/rocksjni/memtablejni.cc +++ b/java/rocksjni/memtablejni.cc @@ -1,10 +1,11 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ for MemTables. +#include "rocksjni/portal.h" #include "include/org_rocksdb_HashSkipListMemTableConfig.h" #include "include/org_rocksdb_HashLinkedListMemTableConfig.h" #include "include/org_rocksdb_VectorMemTableConfig.h" @@ -19,21 +20,41 @@ jlong Java_org_rocksdb_HashSkipListMemTableConfig_newMemTableFactoryHandle( JNIEnv* env, jobject jobj, jlong jbucket_count, jint jheight, jint jbranching_factor) { - return reinterpret_cast(rocksdb::NewHashSkipListRepFactory( - static_cast(jbucket_count), - static_cast(jheight), - static_cast(jbranching_factor))); + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(jbucket_count); + if (s.ok()) { + return reinterpret_cast(rocksdb::NewHashSkipListRepFactory( + static_cast(jbucket_count), + static_cast(jheight), + static_cast(jbranching_factor))); + } + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + return 0; } /* * Class: org_rocksdb_HashLinkedListMemTableConfig * Method: newMemTableFactoryHandle - * Signature: (J)J + * Signature: (JJIZI)J */ jlong Java_org_rocksdb_HashLinkedListMemTableConfig_newMemTableFactoryHandle( - JNIEnv* env, jobject jobj, jlong jbucket_count) { - return reinterpret_cast(rocksdb::NewHashLinkListRepFactory( - static_cast(jbucket_count))); + JNIEnv* env, jobject jobj, jlong jbucket_count, jlong jhuge_page_tlb_size, + jint jbucket_entries_logging_threshold, + jboolean jif_log_bucket_dist_when_flash, jint jthreshold_use_skiplist) { + rocksdb::Status statusBucketCount = + rocksdb::check_if_jlong_fits_size_t(jbucket_count); + rocksdb::Status statusHugePageTlb = + rocksdb::check_if_jlong_fits_size_t(jhuge_page_tlb_size); + if (statusBucketCount.ok() && statusHugePageTlb.ok()) { + return reinterpret_cast(rocksdb::NewHashLinkListRepFactory( + static_cast(jbucket_count), + static_cast(jhuge_page_tlb_size), + static_cast(jbucket_entries_logging_threshold), + static_cast(jif_log_bucket_dist_when_flash), + static_cast(jthreshold_use_skiplist))); + } + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, + !statusBucketCount.ok()?statusBucketCount:statusHugePageTlb); + return 0; } /* @@ -43,16 +64,27 @@ jlong Java_org_rocksdb_HashLinkedListMemTableConfig_newMemTableFactoryHandle( */ jlong Java_org_rocksdb_VectorMemTableConfig_newMemTableFactoryHandle( JNIEnv* env, jobject jobj, jlong jreserved_size) { - return reinterpret_cast(new rocksdb::VectorRepFactory( - static_cast(jreserved_size))); + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(jreserved_size); + if (s.ok()) { + return reinterpret_cast(new rocksdb::VectorRepFactory( + static_cast(jreserved_size))); + } + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + return 0; } /* * Class: org_rocksdb_SkipListMemTableConfig * Method: newMemTableFactoryHandle0 - * Signature: ()J + * Signature: (J)J */ jlong Java_org_rocksdb_SkipListMemTableConfig_newMemTableFactoryHandle0( - JNIEnv* env, jobject jobj) { - return reinterpret_cast(new rocksdb::SkipListFactory()); + JNIEnv* env, jobject jobj, jlong jlookahead) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(jlookahead); + if (s.ok()) { + return reinterpret_cast(new rocksdb::SkipListFactory( + static_cast(jlookahead))); + } + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + return 0; } diff --git a/java/rocksjni/merge_operator.cc b/java/rocksjni/merge_operator.cc new file mode 100644 index 00000000000..1b94382ef04 --- /dev/null +++ b/java/rocksjni/merge_operator.cc @@ -0,0 +1,48 @@ +// Copyright (c) 2014, Vlad Balan (vlad.gm@gmail.com). All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ +// for rocksdb::MergeOperator. + +#include +#include +#include +#include +#include + +#include "include/org_rocksdb_StringAppendOperator.h" +#include "rocksjni/portal.h" +#include "rocksdb/db.h" +#include "rocksdb/options.h" +#include "rocksdb/statistics.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/table.h" +#include "rocksdb/slice_transform.h" +#include "rocksdb/merge_operator.h" +#include "utilities/merge_operators.h" + +/* + * Class: org_rocksdb_StringAppendOperator + * Method: newSharedStringAppendOperator + * Signature: ()J + */ +jlong Java_org_rocksdb_StringAppendOperator_newSharedStringAppendOperator +(JNIEnv* env, jclass jclazz) { + auto* sptr_string_append_op = new std::shared_ptr( + rocksdb::MergeOperators::CreateFromStringId("stringappend")); + return reinterpret_cast(sptr_string_append_op); +} + +/* + * Class: org_rocksdb_StringAppendOperator + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_StringAppendOperator_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* sptr_string_append_op = + reinterpret_cast* >(jhandle); + delete sptr_string_append_op; // delete std::shared_ptr +} diff --git a/java/rocksjni/options.cc b/java/rocksjni/options.cc index da420c78f25..8194abaf6b6 100644 --- a/java/rocksjni/options.cc +++ b/java/rocksjni/options.cc @@ -1,35 +1,62 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ for rocksdb::Options. #include #include #include -#include #include +#include #include "include/org_rocksdb_Options.h" +#include "include/org_rocksdb_DBOptions.h" +#include "include/org_rocksdb_ColumnFamilyOptions.h" #include "include/org_rocksdb_WriteOptions.h" #include "include/org_rocksdb_ReadOptions.h" +#include "include/org_rocksdb_ComparatorOptions.h" +#include "include/org_rocksdb_FlushOptions.h" + +#include "rocksjni/comparatorjnicallback.h" #include "rocksjni/portal.h" +#include "rocksjni/statisticsjni.h" + #include "rocksdb/db.h" #include "rocksdb/options.h" #include "rocksdb/statistics.h" #include "rocksdb/memtablerep.h" #include "rocksdb/table.h" #include "rocksdb/slice_transform.h" +#include "rocksdb/rate_limiter.h" +#include "rocksdb/comparator.h" +#include "rocksdb/convenience.h" +#include "rocksdb/merge_operator.h" +#include "utilities/merge_operators.h" /* * Class: org_rocksdb_Options * Method: newOptions - * Signature: ()V + * Signature: ()J + */ +jlong Java_org_rocksdb_Options_newOptions__(JNIEnv* env, jclass jcls) { + auto* op = new rocksdb::Options(); + return reinterpret_cast(op); +} + +/* + * Class: org_rocksdb_Options + * Method: newOptions + * Signature: (JJ)J */ -void Java_org_rocksdb_Options_newOptions(JNIEnv* env, jobject jobj) { - rocksdb::Options* op = new rocksdb::Options(); - rocksdb::OptionsJni::setHandle(env, jobj, op); +jlong Java_org_rocksdb_Options_newOptions__JJ(JNIEnv* env, jclass jcls, + jlong jdboptions, jlong jcfoptions) { + auto* dbOpt = reinterpret_cast(jdboptions); + auto* cfOpt = reinterpret_cast( + jcfoptions); + auto* op = new rocksdb::Options(*dbOpt, *cfOpt); + return reinterpret_cast(op); } /* @@ -39,7 +66,20 @@ void Java_org_rocksdb_Options_newOptions(JNIEnv* env, jobject jobj) { */ void Java_org_rocksdb_Options_disposeInternal( JNIEnv* env, jobject jobj, jlong handle) { - delete reinterpret_cast(handle); + auto* op = reinterpret_cast(handle); + assert(op != nullptr); + delete op; +} + +/* + * Class: org_rocksdb_Options + * Method: setIncreaseParallelism + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setIncreaseParallelism( + JNIEnv * env, jobject jobj, jlong jhandle, jint totalThreads) { + reinterpret_cast + (jhandle)->IncreaseParallelism(static_cast(totalThreads)); } /* @@ -62,6 +102,90 @@ jboolean Java_org_rocksdb_Options_createIfMissing( return reinterpret_cast(jhandle)->create_if_missing; } +/* + * Class: org_rocksdb_Options + * Method: setCreateMissingColumnFamilies + * Signature: (JZ)V + */ +void Java_org_rocksdb_Options_setCreateMissingColumnFamilies( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean flag) { + reinterpret_cast + (jhandle)->create_missing_column_families = flag; +} + +/* + * Class: org_rocksdb_Options + * Method: createMissingColumnFamilies + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_Options_createMissingColumnFamilies( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast + (jhandle)->create_missing_column_families; +} + +/* + * Class: org_rocksdb_Options + * Method: setComparatorHandle + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setComparatorHandle__JI( + JNIEnv* env, jobject jobj, jlong jhandle, jint builtinComparator) { + switch (builtinComparator) { + case 1: + reinterpret_cast(jhandle)->comparator = + rocksdb::ReverseBytewiseComparator(); + break; + default: + reinterpret_cast(jhandle)->comparator = + rocksdb::BytewiseComparator(); + break; + } +} + +/* + * Class: org_rocksdb_Options + * Method: setComparatorHandle + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setComparatorHandle__JJ( + JNIEnv* env, jobject jobj, jlong jopt_handle, jlong jcomparator_handle) { + reinterpret_cast(jopt_handle)->comparator = + reinterpret_cast(jcomparator_handle); +} + +/* + * Class: org_rocksdb_Options + * Method: setMergeOperatorName + * Signature: (JJjava/lang/String)V + */ +void Java_org_rocksdb_Options_setMergeOperatorName( + JNIEnv* env, jobject jobj, jlong jhandle, jstring jop_name) { + const char* op_name = env->GetStringUTFChars(jop_name, nullptr); + if(op_name == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + auto* options = reinterpret_cast(jhandle); + options->merge_operator = rocksdb::MergeOperators::CreateFromStringId( + op_name); + + env->ReleaseStringUTFChars(jop_name, op_name); +} + +/* + * Class: org_rocksdb_Options + * Method: setMergeOperator + * Signature: (JJjava/lang/String)V + */ +void Java_org_rocksdb_Options_setMergeOperator( + JNIEnv* env, jobject jobj, jlong jhandle, jlong mergeOperatorHandle) { + reinterpret_cast(jhandle)->merge_operator = + *(reinterpret_cast*> + (mergeOperatorHandle)); +} + /* * Class: org_rocksdb_Options * Method: setWriteBufferSize @@ -69,11 +193,15 @@ jboolean Java_org_rocksdb_Options_createIfMissing( */ void Java_org_rocksdb_Options_setWriteBufferSize( JNIEnv* env, jobject jobj, jlong jhandle, jlong jwrite_buffer_size) { - reinterpret_cast(jhandle)->write_buffer_size = - static_cast(jwrite_buffer_size); + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(jwrite_buffer_size); + if (s.ok()) { + reinterpret_cast(jhandle)->write_buffer_size = + jwrite_buffer_size; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } } - /* * Class: org_rocksdb_Options * Method: writeBufferSize @@ -97,24 +225,34 @@ void Java_org_rocksdb_Options_setMaxWriteBufferNumber( /* * Class: org_rocksdb_Options - * Method: createStatistics - * Signature: (J)V + * Method: setStatistics + * Signature: (JJ)V */ -void Java_org_rocksdb_Options_createStatistics( - JNIEnv* env, jobject jobj, jlong jOptHandle) { - reinterpret_cast(jOptHandle)->statistics = - rocksdb::CreateDBStatistics(); +void Java_org_rocksdb_Options_setStatistics( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jstatistics_handle) { + auto* opt = reinterpret_cast(jhandle); + auto* pSptr = + reinterpret_cast*>( + jstatistics_handle); + opt->statistics = *pSptr; } /* * Class: org_rocksdb_Options - * Method: statisticsPtr + * Method: statistics * Signature: (J)J */ -jlong Java_org_rocksdb_Options_statisticsPtr( - JNIEnv* env, jobject jobj, jlong jOptHandle) { - auto st = reinterpret_cast(jOptHandle)->statistics.get(); - return reinterpret_cast(st); +jlong Java_org_rocksdb_Options_statistics( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + std::shared_ptr sptr = opt->statistics; + if (sptr == nullptr) { + return 0; + } else { + std::shared_ptr* pSptr = + new std::shared_ptr(sptr); + return reinterpret_cast(pSptr); + } } /* @@ -169,6 +307,40 @@ void Java_org_rocksdb_Options_setParanoidChecks( static_cast(paranoid_checks); } +/* + * Class: org_rocksdb_Options + * Method: setEnv + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setEnv( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jenv) { + reinterpret_cast(jhandle)->env = + reinterpret_cast(jenv); +} + +/* + * Class: org_rocksdb_Options + * Method: setMaxTotalWalSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setMaxTotalWalSize( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jmax_total_wal_size) { + reinterpret_cast(jhandle)->max_total_wal_size = + static_cast(jmax_total_wal_size); +} + +/* + * Class: org_rocksdb_Options + * Method: maxTotalWalSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_Options_maxTotalWalSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + max_total_wal_size; +} + /* * Class: org_rocksdb_Options * Method: maxOpenFiles @@ -192,23 +364,24 @@ void Java_org_rocksdb_Options_setMaxOpenFiles( /* * Class: org_rocksdb_Options - * Method: disableDataSync - * Signature: (J)Z + * Method: setMaxFileOpeningThreads + * Signature: (JI)V */ -jboolean Java_org_rocksdb_Options_disableDataSync( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->disableDataSync; +void Java_org_rocksdb_Options_setMaxFileOpeningThreads( + JNIEnv* env, jobject jobj, jlong jhandle, jint jmax_file_opening_threads) { + reinterpret_cast(jhandle)->max_file_opening_threads = + static_cast(jmax_file_opening_threads); } /* * Class: org_rocksdb_Options - * Method: setDisableDataSync - * Signature: (JZ)V + * Method: maxFileOpeningThreads + * Signature: (J)I */ -void Java_org_rocksdb_Options_setDisableDataSync( - JNIEnv* env, jobject jobj, jlong jhandle, jboolean disableDataSync) { - reinterpret_cast(jhandle)->disableDataSync = - static_cast(disableDataSync); +jint Java_org_rocksdb_Options_maxFileOpeningThreads( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->max_file_opening_threads); } /* @@ -232,6 +405,106 @@ void Java_org_rocksdb_Options_setUseFsync( static_cast(use_fsync); } +/* + * Class: org_rocksdb_Options + * Method: setDbPaths + * Signature: (J[Ljava/lang/String;[J)V + */ +void Java_org_rocksdb_Options_setDbPaths( + JNIEnv* env, jobject jobj, jlong jhandle, jobjectArray jpaths, + jlongArray jtarget_sizes) { + std::vector db_paths; + jlong* ptr_jtarget_size = env->GetLongArrayElements(jtarget_sizes, nullptr); + if(ptr_jtarget_size == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + jboolean has_exception = JNI_FALSE; + const jsize len = env->GetArrayLength(jpaths); + for(jsize i = 0; i < len; i++) { + jobject jpath = reinterpret_cast(env-> + GetObjectArrayElement(jpaths, i)); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->ReleaseLongArrayElements( + jtarget_sizes, ptr_jtarget_size, JNI_ABORT); + return; + } + std::string path = rocksdb::JniUtil::copyString( + env, static_cast(jpath), &has_exception); + env->DeleteLocalRef(jpath); + + if(has_exception == JNI_TRUE) { + env->ReleaseLongArrayElements( + jtarget_sizes, ptr_jtarget_size, JNI_ABORT); + return; + } + + jlong jtarget_size = ptr_jtarget_size[i]; + + db_paths.push_back( + rocksdb::DbPath(path, static_cast(jtarget_size))); + } + + env->ReleaseLongArrayElements(jtarget_sizes, ptr_jtarget_size, JNI_ABORT); + + auto* opt = reinterpret_cast(jhandle); + opt->db_paths = db_paths; +} + +/* + * Class: org_rocksdb_Options + * Method: dbPathsLen + * Signature: (J)J + */ +jlong Java_org_rocksdb_Options_dbPathsLen( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->db_paths.size()); +} + +/* + * Class: org_rocksdb_Options + * Method: dbPaths + * Signature: (J[Ljava/lang/String;[J)V + */ +void Java_org_rocksdb_Options_dbPaths( + JNIEnv* env, jobject jobj, jlong jhandle, jobjectArray jpaths, + jlongArray jtarget_sizes) { + jlong* ptr_jtarget_size = env->GetLongArrayElements(jtarget_sizes, nullptr); + if(ptr_jtarget_size == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + auto* opt = reinterpret_cast(jhandle); + const jsize len = env->GetArrayLength(jpaths); + for(jsize i = 0; i < len; i++) { + rocksdb::DbPath db_path = opt->db_paths[i]; + + jstring jpath = env->NewStringUTF(db_path.path.c_str()); + if(jpath == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseLongArrayElements( + jtarget_sizes, ptr_jtarget_size, JNI_ABORT); + return; + } + env->SetObjectArrayElement(jpaths, i, jpath); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jpath); + env->ReleaseLongArrayElements( + jtarget_sizes, ptr_jtarget_size, JNI_ABORT); + return; + } + + ptr_jtarget_size[i] = static_cast(db_path.target_size); + } + + env->ReleaseLongArrayElements(jtarget_sizes, ptr_jtarget_size, JNI_COMMIT); +} + /* * Class: org_rocksdb_Options * Method: dbLogDir @@ -250,7 +523,11 @@ jstring Java_org_rocksdb_Options_dbLogDir( */ void Java_org_rocksdb_Options_setDbLogDir( JNIEnv* env, jobject jobj, jlong jhandle, jstring jdb_log_dir) { - const char* log_dir = env->GetStringUTFChars(jdb_log_dir, 0); + const char* log_dir = env->GetStringUTFChars(jdb_log_dir, nullptr); + if(log_dir == nullptr) { + // exception thrown: OutOfMemoryError + return; + } reinterpret_cast(jhandle)->db_log_dir.assign(log_dir); env->ReleaseStringUTFChars(jdb_log_dir, log_dir); } @@ -273,7 +550,11 @@ jstring Java_org_rocksdb_Options_walDir( */ void Java_org_rocksdb_Options_setWalDir( JNIEnv* env, jobject jobj, jlong jhandle, jstring jwal_dir) { - const char* wal_dir = env->GetStringUTFChars(jwal_dir, 0); + const char* wal_dir = env->GetStringUTFChars(jwal_dir, nullptr); + if(wal_dir == nullptr) { + // exception thrown: OutOfMemoryError + return; + } reinterpret_cast(jhandle)->wal_dir.assign(wal_dir); env->ReleaseStringUTFChars(jwal_dir, wal_dir); } @@ -301,6 +582,28 @@ void Java_org_rocksdb_Options_setDeleteObsoleteFilesPeriodMicros( static_cast(micros); } +/* + * Class: org_rocksdb_Options + * Method: setBaseBackgroundCompactions + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setBaseBackgroundCompactions( + JNIEnv* env, jobject jobj, jlong jhandle, jint max) { + reinterpret_cast(jhandle) + ->base_background_compactions = static_cast(max); +} + +/* + * Class: org_rocksdb_Options + * Method: baseBackgroundCompactions + * Signature: (J)I + */ +jint Java_org_rocksdb_Options_baseBackgroundCompactions( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle) + ->base_background_compactions; +} + /* * Class: org_rocksdb_Options * Method: maxBackgroundCompactions @@ -323,6 +626,27 @@ void Java_org_rocksdb_Options_setMaxBackgroundCompactions( ->max_background_compactions = static_cast(max); } +/* + * Class: org_rocksdb_Options + * Method: setMaxSubcompactions + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setMaxSubcompactions( + JNIEnv* env, jobject jobj, jlong jhandle, jint max) { + reinterpret_cast(jhandle) + ->max_subcompactions = static_cast(max); +} + +/* + * Class: org_rocksdb_Options + * Method: maxSubcompactions + * Signature: (J)I + */ +jint Java_org_rocksdb_Options_maxSubcompactions( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->max_subcompactions; +} + /* * Class: org_rocksdb_Options * Method: maxBackgroundFlushes @@ -361,8 +685,13 @@ jlong Java_org_rocksdb_Options_maxLogFileSize( */ void Java_org_rocksdb_Options_setMaxLogFileSize( JNIEnv* env, jobject jobj, jlong jhandle, jlong max_log_file_size) { - reinterpret_cast(jhandle)->max_log_file_size = - static_cast(max_log_file_size); + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(max_log_file_size); + if (s.ok()) { + reinterpret_cast(jhandle)->max_log_file_size = + max_log_file_size; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } } /* @@ -382,8 +711,14 @@ jlong Java_org_rocksdb_Options_logFileTimeToRoll( */ void Java_org_rocksdb_Options_setLogFileTimeToRoll( JNIEnv* env, jobject jobj, jlong jhandle, jlong log_file_time_to_roll) { - reinterpret_cast(jhandle)->log_file_time_to_roll = - static_cast(log_file_time_to_roll); + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t( + log_file_time_to_roll); + if (s.ok()) { + reinterpret_cast(jhandle)->log_file_time_to_roll = + log_file_time_to_roll; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } } /* @@ -403,8 +738,40 @@ jlong Java_org_rocksdb_Options_keepLogFileNum( */ void Java_org_rocksdb_Options_setKeepLogFileNum( JNIEnv* env, jobject jobj, jlong jhandle, jlong keep_log_file_num) { - reinterpret_cast(jhandle)->keep_log_file_num = - static_cast(keep_log_file_num); + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(keep_log_file_num); + if (s.ok()) { + reinterpret_cast(jhandle)->keep_log_file_num = + keep_log_file_num; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_Options + * Method: recycleLogFileNum + * Signature: (J)J + */ +jlong Java_org_rocksdb_Options_recycleLogFileNum( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->recycle_log_file_num; +} + +/* + * Class: org_rocksdb_Options + * Method: setRecycleLogFileNum + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setRecycleLogFileNum( + JNIEnv* env, jobject jobj, jlong jhandle, jlong recycle_log_file_num) { + rocksdb::Status s = + rocksdb::check_if_jlong_fits_size_t(recycle_log_file_num); + if (s.ok()) { + reinterpret_cast(jhandle)->recycle_log_file_num = + recycle_log_file_num; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } } /* @@ -423,7 +790,7 @@ jlong Java_org_rocksdb_Options_maxManifestFileSize( */ jstring Java_org_rocksdb_Options_memTableFactoryName( JNIEnv* env, jobject jobj, jlong jhandle) { - auto opt = reinterpret_cast(jhandle); + auto* opt = reinterpret_cast(jhandle); rocksdb::MemTableRepFactory* tf = opt->memtable_factory.get(); // Should never be nullptr. @@ -459,6 +826,55 @@ void Java_org_rocksdb_Options_setMemTableFactory( reinterpret_cast(jfactory_handle)); } +/* + * Class: org_rocksdb_Options + * Method: setRateLimiter + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setRateLimiter( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jrate_limiter_handle) { + std::shared_ptr *pRateLimiter = + reinterpret_cast *>( + jrate_limiter_handle); + reinterpret_cast(jhandle)-> + rate_limiter = *pRateLimiter; +} + +/* + * Class: org_rocksdb_Options + * Method: setLogger + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setLogger( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jlogger_handle) { +std::shared_ptr *pLogger = + reinterpret_cast *>( + jlogger_handle); + reinterpret_cast(jhandle)->info_log = *pLogger; +} + +/* + * Class: org_rocksdb_Options + * Method: setInfoLogLevel + * Signature: (JB)V + */ +void Java_org_rocksdb_Options_setInfoLogLevel( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jlog_level) { + reinterpret_cast(jhandle)->info_log_level = + static_cast(jlog_level); +} + +/* + * Class: org_rocksdb_Options + * Method: infoLogLevel + * Signature: (J)B + */ +jbyte Java_org_rocksdb_Options_infoLogLevel( + JNIEnv* env, jobject jobj, jlong jhandle) { + return static_cast( + reinterpret_cast(jhandle)->info_log_level); +} + /* * Class: org_rocksdb_Options * Method: tableCacheNumshardbits @@ -481,35 +897,25 @@ void Java_org_rocksdb_Options_setTableCacheNumshardbits( } /* - * Class: org_rocksdb_Options - * Method: tableCacheRemoveScanCountLimit - * Signature: (J)I - */ -jint Java_org_rocksdb_Options_tableCacheRemoveScanCountLimit( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->table_cache_remove_scan_count_limit; -} - -/* - * Class: org_rocksdb_Options - * Method: setTableCacheRemoveScanCountLimit + * Method: useFixedLengthPrefixExtractor * Signature: (JI)V */ -void Java_org_rocksdb_Options_setTableCacheRemoveScanCountLimit( - JNIEnv* env, jobject jobj, jlong jhandle, jint limit) { - reinterpret_cast( - jhandle)->table_cache_remove_scan_count_limit = static_cast(limit); +void Java_org_rocksdb_Options_useFixedLengthPrefixExtractor( + JNIEnv* env, jobject jobj, jlong jhandle, jint jprefix_length) { + reinterpret_cast(jhandle)->prefix_extractor.reset( + rocksdb::NewFixedPrefixTransform( + static_cast(jprefix_length))); } /* - * Method: useFixedLengthPrefixExtractor + * Method: useCappedPrefixExtractor * Signature: (JI)V */ -void Java_org_rocksdb_Options_useFixedLengthPrefixExtractor( +void Java_org_rocksdb_Options_useCappedPrefixExtractor( JNIEnv* env, jobject jobj, jlong jhandle, jint jprefix_length) { reinterpret_cast(jhandle)->prefix_extractor.reset( - rocksdb::NewFixedPrefixTransform(static_cast(jprefix_length))); + rocksdb::NewCappedPrefixTransform( + static_cast(jprefix_length))); } /* @@ -572,29 +978,13 @@ jlong Java_org_rocksdb_Options_manifestPreallocationSize( */ void Java_org_rocksdb_Options_setManifestPreallocationSize( JNIEnv* env, jobject jobj, jlong jhandle, jlong preallocation_size) { - reinterpret_cast(jhandle)->manifest_preallocation_size = - static_cast(preallocation_size); -} - -/* - * Class: org_rocksdb_Options - * Method: allowOsBuffer - * Signature: (J)Z - */ -jboolean Java_org_rocksdb_Options_allowOsBuffer( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->allow_os_buffer; -} - -/* - * Class: org_rocksdb_Options - * Method: setAllowOsBuffer - * Signature: (JZ)V - */ -void Java_org_rocksdb_Options_setAllowOsBuffer( - JNIEnv* env, jobject jobj, jlong jhandle, jboolean allow_os_buffer) { - reinterpret_cast(jhandle)->allow_os_buffer = - static_cast(allow_os_buffer); + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(preallocation_size); + if (s.ok()) { + reinterpret_cast(jhandle)->manifest_preallocation_size = + preallocation_size; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } } /* @@ -651,897 +1041,4582 @@ void Java_org_rocksdb_Options_setAllowMmapWrites( /* * Class: org_rocksdb_Options - * Method: isFdCloseOnExec + * Method: useDirectReads * Signature: (J)Z */ -jboolean Java_org_rocksdb_Options_isFdCloseOnExec( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->is_fd_close_on_exec; +jboolean Java_org_rocksdb_Options_useDirectReads(JNIEnv* env, jobject jobj, + jlong jhandle) { + return reinterpret_cast(jhandle)->use_direct_reads; } /* * Class: org_rocksdb_Options - * Method: setIsFdCloseOnExec + * Method: setUseDirectReads * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setIsFdCloseOnExec( - JNIEnv* env, jobject jobj, jlong jhandle, jboolean is_fd_close_on_exec) { - reinterpret_cast(jhandle)->is_fd_close_on_exec = - static_cast(is_fd_close_on_exec); +void Java_org_rocksdb_Options_setUseDirectReads(JNIEnv* env, jobject jobj, + jlong jhandle, + jboolean use_direct_reads) { + reinterpret_cast(jhandle)->use_direct_reads = + static_cast(use_direct_reads); } /* * Class: org_rocksdb_Options - * Method: skipLogErrorOnRecovery + * Method: useDirectIoForFlushAndCompaction * Signature: (J)Z */ -jboolean Java_org_rocksdb_Options_skipLogErrorOnRecovery( +jboolean Java_org_rocksdb_Options_useDirectIoForFlushAndCompaction( JNIEnv* env, jobject jobj, jlong jhandle) { return reinterpret_cast(jhandle) - ->skip_log_error_on_recovery; + ->use_direct_io_for_flush_and_compaction; } /* * Class: org_rocksdb_Options - * Method: setSkipLogErrorOnRecovery + * Method: setUseDirectIoForFlushAndCompaction * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setSkipLogErrorOnRecovery( - JNIEnv* env, jobject jobj, jlong jhandle, jboolean skip) { - reinterpret_cast(jhandle)->skip_log_error_on_recovery = - static_cast(skip); +void Java_org_rocksdb_Options_setUseDirectIoForFlushAndCompaction( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean use_direct_io_for_flush_and_compaction) { + reinterpret_cast(jhandle) + ->use_direct_io_for_flush_and_compaction = + static_cast(use_direct_io_for_flush_and_compaction); } /* * Class: org_rocksdb_Options - * Method: statsDumpPeriodSec - * Signature: (J)I + * Method: setAllowFAllocate + * Signature: (JZ)V */ -jint Java_org_rocksdb_Options_statsDumpPeriodSec( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->stats_dump_period_sec; +void Java_org_rocksdb_Options_setAllowFAllocate( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jallow_fallocate) { + reinterpret_cast(jhandle)->allow_fallocate = + static_cast(jallow_fallocate); } /* * Class: org_rocksdb_Options - * Method: setStatsDumpPeriodSec - * Signature: (JI)V + * Method: allowFAllocate + * Signature: (J)Z */ -void Java_org_rocksdb_Options_setStatsDumpPeriodSec( - JNIEnv* env, jobject jobj, jlong jhandle, jint stats_dump_period_sec) { - reinterpret_cast(jhandle)->stats_dump_period_sec = - static_cast(stats_dump_period_sec); +jboolean Java_org_rocksdb_Options_allowFAllocate( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->allow_fallocate); } /* * Class: org_rocksdb_Options - * Method: adviseRandomOnOpen + * Method: isFdCloseOnExec * Signature: (J)Z */ -jboolean Java_org_rocksdb_Options_adviseRandomOnOpen( +jboolean Java_org_rocksdb_Options_isFdCloseOnExec( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->advise_random_on_open; + return reinterpret_cast(jhandle)->is_fd_close_on_exec; } /* * Class: org_rocksdb_Options - * Method: setAdviseRandomOnOpen + * Method: setIsFdCloseOnExec * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setAdviseRandomOnOpen( - JNIEnv* env, jobject jobj, jlong jhandle, jboolean advise_random_on_open) { - reinterpret_cast(jhandle)->advise_random_on_open = - static_cast(advise_random_on_open); +void Java_org_rocksdb_Options_setIsFdCloseOnExec( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean is_fd_close_on_exec) { + reinterpret_cast(jhandle)->is_fd_close_on_exec = + static_cast(is_fd_close_on_exec); } /* * Class: org_rocksdb_Options - * Method: useAdaptiveMutex - * Signature: (J)Z + * Method: statsDumpPeriodSec + * Signature: (J)I */ -jboolean Java_org_rocksdb_Options_useAdaptiveMutex( +jint Java_org_rocksdb_Options_statsDumpPeriodSec( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->use_adaptive_mutex; + return reinterpret_cast(jhandle)->stats_dump_period_sec; } /* * Class: org_rocksdb_Options - * Method: setUseAdaptiveMutex - * Signature: (JZ)V + * Method: setStatsDumpPeriodSec + * Signature: (JI)V */ -void Java_org_rocksdb_Options_setUseAdaptiveMutex( - JNIEnv* env, jobject jobj, jlong jhandle, jboolean use_adaptive_mutex) { - reinterpret_cast(jhandle)->use_adaptive_mutex = - static_cast(use_adaptive_mutex); +void Java_org_rocksdb_Options_setStatsDumpPeriodSec( + JNIEnv* env, jobject jobj, jlong jhandle, jint stats_dump_period_sec) { + reinterpret_cast(jhandle)->stats_dump_period_sec = + static_cast(stats_dump_period_sec); } /* * Class: org_rocksdb_Options - * Method: bytesPerSync - * Signature: (J)J + * Method: adviseRandomOnOpen + * Signature: (J)Z */ -jlong Java_org_rocksdb_Options_bytesPerSync( +jboolean Java_org_rocksdb_Options_adviseRandomOnOpen( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->bytes_per_sync; + return reinterpret_cast(jhandle)->advise_random_on_open; } /* * Class: org_rocksdb_Options - * Method: setBytesPerSync - * Signature: (JJ)V + * Method: setAdviseRandomOnOpen + * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setBytesPerSync( - JNIEnv* env, jobject jobj, jlong jhandle, jlong bytes_per_sync) { - reinterpret_cast(jhandle)->bytes_per_sync = - static_cast(bytes_per_sync); +void Java_org_rocksdb_Options_setAdviseRandomOnOpen( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean advise_random_on_open) { + reinterpret_cast(jhandle)->advise_random_on_open = + static_cast(advise_random_on_open); } /* * Class: org_rocksdb_Options - * Method: allowThreadLocal - * Signature: (J)Z + * Method: setDbWriteBufferSize + * Signature: (JJ)V */ -jboolean Java_org_rocksdb_Options_allowThreadLocal( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->allow_thread_local; +void Java_org_rocksdb_Options_setDbWriteBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jdb_write_buffer_size) { + auto* opt = reinterpret_cast(jhandle); + opt->db_write_buffer_size = static_cast(jdb_write_buffer_size); } /* * Class: org_rocksdb_Options - * Method: setAllowThreadLocal - * Signature: (JZ)V + * Method: dbWriteBufferSize + * Signature: (J)J */ -void Java_org_rocksdb_Options_setAllowThreadLocal( - JNIEnv* env, jobject jobj, jlong jhandle, jboolean allow_thread_local) { - reinterpret_cast(jhandle)->allow_thread_local = - static_cast(allow_thread_local); +jlong Java_org_rocksdb_Options_dbWriteBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->db_write_buffer_size); } /* - * Method: tableFactoryName - * Signature: (J)Ljava/lang/String + * Class: org_rocksdb_Options + * Method: setAccessHintOnCompactionStart + * Signature: (JB)V */ -jstring Java_org_rocksdb_Options_tableFactoryName( - JNIEnv* env, jobject jobj, jlong jhandle) { - auto opt = reinterpret_cast(jhandle); - rocksdb::TableFactory* tf = opt->table_factory.get(); - - // Should never be nullptr. - // Default memtable factory is SkipListFactory - assert(tf); - - return env->NewStringUTF(tf->Name()); +void Java_org_rocksdb_Options_setAccessHintOnCompactionStart( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jaccess_hint_value) { + auto* opt = reinterpret_cast(jhandle); + opt->access_hint_on_compaction_start = + rocksdb::AccessHintJni::toCppAccessHint(jaccess_hint_value); } - /* * Class: org_rocksdb_Options - * Method: minWriteBufferNumberToMerge - * Signature: (J)I + * Method: accessHintOnCompactionStart + * Signature: (J)B */ -jint Java_org_rocksdb_Options_minWriteBufferNumberToMerge( +jbyte Java_org_rocksdb_Options_accessHintOnCompactionStart( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->min_write_buffer_number_to_merge; + auto* opt = reinterpret_cast(jhandle); + return rocksdb::AccessHintJni::toJavaAccessHint( + opt->access_hint_on_compaction_start); } /* * Class: org_rocksdb_Options - * Method: setMinWriteBufferNumberToMerge - * Signature: (JI)V + * Method: setNewTableReaderForCompactionInputs + * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setMinWriteBufferNumberToMerge( +void Java_org_rocksdb_Options_setNewTableReaderForCompactionInputs( JNIEnv* env, jobject jobj, jlong jhandle, - jint jmin_write_buffer_number_to_merge) { - reinterpret_cast( - jhandle)->min_write_buffer_number_to_merge = - static_cast(jmin_write_buffer_number_to_merge); + jboolean jnew_table_reader_for_compaction_inputs) { + auto* opt = reinterpret_cast(jhandle); + opt->new_table_reader_for_compaction_inputs = + static_cast(jnew_table_reader_for_compaction_inputs); } /* * Class: org_rocksdb_Options - * Method: setCompressionType - * Signature: (JB)V + * Method: newTableReaderForCompactionInputs + * Signature: (J)Z */ -void Java_org_rocksdb_Options_setCompressionType( - JNIEnv* env, jobject jobj, jlong jhandle, jbyte compression) { - reinterpret_cast(jhandle)->compression = - static_cast(compression); +jboolean Java_org_rocksdb_Options_newTableReaderForCompactionInputs( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->new_table_reader_for_compaction_inputs); } /* * Class: org_rocksdb_Options - * Method: compressionType - * Signature: (J)B + * Method: setCompactionReadaheadSize + * Signature: (JJ)V */ -jbyte Java_org_rocksdb_Options_compressionType( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->compression; +void Java_org_rocksdb_Options_setCompactionReadaheadSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jcompaction_readahead_size) { + auto* opt = reinterpret_cast(jhandle); + opt->compaction_readahead_size = + static_cast(jcompaction_readahead_size); } /* * Class: org_rocksdb_Options - * Method: setCompactionStyle - * Signature: (JB)V + * Method: compactionReadaheadSize + * Signature: (J)J */ -void Java_org_rocksdb_Options_setCompactionStyle( - JNIEnv* env, jobject jobj, jlong jhandle, jbyte compaction_style) { - reinterpret_cast(jhandle)->compaction_style = - static_cast(compaction_style); +jlong Java_org_rocksdb_Options_compactionReadaheadSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->compaction_readahead_size); } /* * Class: org_rocksdb_Options - * Method: compactionStyle - * Signature: (J)B + * Method: setRandomAccessMaxBufferSize + * Signature: (JJ)V */ -jbyte Java_org_rocksdb_Options_compactionStyle( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->compaction_style; +void Java_org_rocksdb_Options_setRandomAccessMaxBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jrandom_access_max_buffer_size) { + auto* opt = reinterpret_cast(jhandle); + opt->random_access_max_buffer_size = + static_cast(jrandom_access_max_buffer_size); } /* * Class: org_rocksdb_Options - * Method: numLevels - * Signature: (J)I + * Method: randomAccessMaxBufferSize + * Signature: (J)J */ -jint Java_org_rocksdb_Options_numLevels( +jlong Java_org_rocksdb_Options_randomAccessMaxBufferSize( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->num_levels; + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->random_access_max_buffer_size); } /* * Class: org_rocksdb_Options - * Method: setNumLevels - * Signature: (JI)V + * Method: setWritableFileMaxBufferSize + * Signature: (JJ)V */ -void Java_org_rocksdb_Options_setNumLevels( - JNIEnv* env, jobject jobj, jlong jhandle, jint jnum_levels) { - reinterpret_cast(jhandle)->num_levels = - static_cast(jnum_levels); +void Java_org_rocksdb_Options_setWritableFileMaxBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jwritable_file_max_buffer_size) { + auto* opt = reinterpret_cast(jhandle); + opt->writable_file_max_buffer_size = + static_cast(jwritable_file_max_buffer_size); } /* * Class: org_rocksdb_Options - * Method: levelZeroFileNumCompactionTrigger - * Signature: (J)I + * Method: writableFileMaxBufferSize + * Signature: (J)J */ -jint Java_org_rocksdb_Options_levelZeroFileNumCompactionTrigger( +jlong Java_org_rocksdb_Options_writableFileMaxBufferSize( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->level0_file_num_compaction_trigger; + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->writable_file_max_buffer_size); } /* * Class: org_rocksdb_Options - * Method: setLevelZeroFileNumCompactionTrigger - * Signature: (JI)V + * Method: useAdaptiveMutex + * Signature: (J)Z */ -void Java_org_rocksdb_Options_setLevelZeroFileNumCompactionTrigger( - JNIEnv* env, jobject jobj, jlong jhandle, - jint jlevel0_file_num_compaction_trigger) { - reinterpret_cast( - jhandle)->level0_file_num_compaction_trigger = - static_cast(jlevel0_file_num_compaction_trigger); +jboolean Java_org_rocksdb_Options_useAdaptiveMutex( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->use_adaptive_mutex; } /* * Class: org_rocksdb_Options - * Method: levelZeroSlowdownWritesTrigger - * Signature: (J)I + * Method: setUseAdaptiveMutex + * Signature: (JZ)V */ -jint Java_org_rocksdb_Options_levelZeroSlowdownWritesTrigger( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->level0_slowdown_writes_trigger; +void Java_org_rocksdb_Options_setUseAdaptiveMutex( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean use_adaptive_mutex) { + reinterpret_cast(jhandle)->use_adaptive_mutex = + static_cast(use_adaptive_mutex); } /* * Class: org_rocksdb_Options - * Method: setLevelSlowdownWritesTrigger - * Signature: (JI)V + * Method: bytesPerSync + * Signature: (J)J */ -void Java_org_rocksdb_Options_setLevelZeroSlowdownWritesTrigger( - JNIEnv* env, jobject jobj, jlong jhandle, - jint jlevel0_slowdown_writes_trigger) { - reinterpret_cast( - jhandle)->level0_slowdown_writes_trigger = - static_cast(jlevel0_slowdown_writes_trigger); +jlong Java_org_rocksdb_Options_bytesPerSync( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->bytes_per_sync; } /* * Class: org_rocksdb_Options - * Method: levelZeroStopWritesTrigger - * Signature: (J)I + * Method: setBytesPerSync + * Signature: (JJ)V */ -jint Java_org_rocksdb_Options_levelZeroStopWritesTrigger( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->level0_stop_writes_trigger; +void Java_org_rocksdb_Options_setBytesPerSync( + JNIEnv* env, jobject jobj, jlong jhandle, jlong bytes_per_sync) { + reinterpret_cast(jhandle)->bytes_per_sync = + static_cast(bytes_per_sync); } /* * Class: org_rocksdb_Options - * Method: setLevelStopWritesTrigger - * Signature: (JI)V + * Method: setWalBytesPerSync + * Signature: (JJ)V */ -void Java_org_rocksdb_Options_setLevelZeroStopWritesTrigger( - JNIEnv* env, jobject jobj, jlong jhandle, - jint jlevel0_stop_writes_trigger) { - reinterpret_cast(jhandle)->level0_stop_writes_trigger = - static_cast(jlevel0_stop_writes_trigger); +void Java_org_rocksdb_Options_setWalBytesPerSync( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jwal_bytes_per_sync) { + reinterpret_cast(jhandle)->wal_bytes_per_sync = + static_cast(jwal_bytes_per_sync); } /* * Class: org_rocksdb_Options - * Method: maxMemCompactionLevel - * Signature: (J)I + * Method: walBytesPerSync + * Signature: (J)J */ -jint Java_org_rocksdb_Options_maxMemCompactionLevel( +jlong Java_org_rocksdb_Options_walBytesPerSync( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->max_mem_compaction_level; + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->wal_bytes_per_sync); } /* * Class: org_rocksdb_Options - * Method: setMaxMemCompactionLevel - * Signature: (JI)V + * Method: setEnableThreadTracking + * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setMaxMemCompactionLevel( +void Java_org_rocksdb_Options_setEnableThreadTracking( JNIEnv* env, jobject jobj, jlong jhandle, - jint jmax_mem_compaction_level) { - reinterpret_cast(jhandle)->max_mem_compaction_level = - static_cast(jmax_mem_compaction_level); + jboolean jenable_thread_tracking) { + auto* opt = reinterpret_cast(jhandle); + opt->enable_thread_tracking = static_cast(jenable_thread_tracking); } /* * Class: org_rocksdb_Options - * Method: targetFileSizeBase - * Signature: (J)I + * Method: enableThreadTracking + * Signature: (J)Z */ -jint Java_org_rocksdb_Options_targetFileSizeBase( +jboolean Java_org_rocksdb_Options_enableThreadTracking( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->target_file_size_base; + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->enable_thread_tracking); } /* * Class: org_rocksdb_Options - * Method: setTargetFileSizeBase - * Signature: (JI)V + * Method: setDelayedWriteRate + * Signature: (JJ)V */ -void Java_org_rocksdb_Options_setTargetFileSizeBase( - JNIEnv* env, jobject jobj, jlong jhandle, - jint jtarget_file_size_base) { - reinterpret_cast(jhandle)->target_file_size_base = - static_cast(jtarget_file_size_base); +void Java_org_rocksdb_Options_setDelayedWriteRate( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jdelayed_write_rate) { + auto* opt = reinterpret_cast(jhandle); + opt->delayed_write_rate = static_cast(jdelayed_write_rate); } /* * Class: org_rocksdb_Options - * Method: targetFileSizeMultiplier - * Signature: (J)I + * Method: delayedWriteRate + * Signature: (J)J */ -jint Java_org_rocksdb_Options_targetFileSizeMultiplier( +jlong Java_org_rocksdb_Options_delayedWriteRate( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->target_file_size_multiplier; + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->delayed_write_rate); } /* * Class: org_rocksdb_Options - * Method: setTargetFileSizeMultiplier - * Signature: (JI)V + * Method: setAllowConcurrentMemtableWrite + * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setTargetFileSizeMultiplier( - JNIEnv* env, jobject jobj, jlong jhandle, - jint jtarget_file_size_multiplier) { - reinterpret_cast( - jhandle)->target_file_size_multiplier = - static_cast(jtarget_file_size_multiplier); +void Java_org_rocksdb_Options_setAllowConcurrentMemtableWrite( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean allow) { + reinterpret_cast(jhandle)-> + allow_concurrent_memtable_write = static_cast(allow); } /* * Class: org_rocksdb_Options - * Method: maxBytesForLevelBase - * Signature: (J)J + * Method: allowConcurrentMemtableWrite + * Signature: (J)Z */ -jlong Java_org_rocksdb_Options_maxBytesForLevelBase( +jboolean Java_org_rocksdb_Options_allowConcurrentMemtableWrite( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->max_bytes_for_level_base; + return reinterpret_cast(jhandle)-> + allow_concurrent_memtable_write; } /* * Class: org_rocksdb_Options - * Method: setMaxBytesForLevelBase - * Signature: (JJ)V + * Method: setEnableWriteThreadAdaptiveYield + * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setMaxBytesForLevelBase( - JNIEnv* env, jobject jobj, jlong jhandle, - jlong jmax_bytes_for_level_base) { - reinterpret_cast( - jhandle)->max_bytes_for_level_base = - static_cast(jmax_bytes_for_level_base); +void Java_org_rocksdb_Options_setEnableWriteThreadAdaptiveYield( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean yield) { + reinterpret_cast(jhandle)-> + enable_write_thread_adaptive_yield = static_cast(yield); } /* * Class: org_rocksdb_Options - * Method: maxBytesForLevelMultiplier - * Signature: (J)I + * Method: enableWriteThreadAdaptiveYield + * Signature: (J)Z */ -jint Java_org_rocksdb_Options_maxBytesForLevelMultiplier( +jboolean Java_org_rocksdb_Options_enableWriteThreadAdaptiveYield( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->max_bytes_for_level_multiplier; + return reinterpret_cast(jhandle)-> + enable_write_thread_adaptive_yield; } /* * Class: org_rocksdb_Options - * Method: setMaxBytesForLevelMultiplier - * Signature: (JI)V + * Method: setWriteThreadMaxYieldUsec + * Signature: (JJ)V */ -void Java_org_rocksdb_Options_setMaxBytesForLevelMultiplier( - JNIEnv* env, jobject jobj, jlong jhandle, - jint jmax_bytes_for_level_multiplier) { - reinterpret_cast( - jhandle)->max_bytes_for_level_multiplier = - static_cast(jmax_bytes_for_level_multiplier); +void Java_org_rocksdb_Options_setWriteThreadMaxYieldUsec( + JNIEnv* env, jobject jobj, jlong jhandle, jlong max) { + reinterpret_cast(jhandle)-> + write_thread_max_yield_usec = static_cast(max); } /* * Class: org_rocksdb_Options - * Method: expandedCompactionFactor - * Signature: (J)I + * Method: writeThreadMaxYieldUsec + * Signature: (J)J */ -jint Java_org_rocksdb_Options_expandedCompactionFactor( +jlong Java_org_rocksdb_Options_writeThreadMaxYieldUsec( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->expanded_compaction_factor; + return reinterpret_cast(jhandle)-> + write_thread_max_yield_usec; } /* * Class: org_rocksdb_Options - * Method: setExpandedCompactionFactor - * Signature: (JI)V + * Method: setWriteThreadSlowYieldUsec + * Signature: (JJ)V */ -void Java_org_rocksdb_Options_setExpandedCompactionFactor( - JNIEnv* env, jobject jobj, jlong jhandle, - jint jexpanded_compaction_factor) { - reinterpret_cast( - jhandle)->expanded_compaction_factor = - static_cast(jexpanded_compaction_factor); +void Java_org_rocksdb_Options_setWriteThreadSlowYieldUsec( + JNIEnv* env, jobject jobj, jlong jhandle, jlong slow) { + reinterpret_cast(jhandle)-> + write_thread_slow_yield_usec = static_cast(slow); } /* * Class: org_rocksdb_Options - * Method: sourceCompactionFactor - * Signature: (J)I + * Method: writeThreadSlowYieldUsec + * Signature: (J)J */ -jint Java_org_rocksdb_Options_sourceCompactionFactor( +jlong Java_org_rocksdb_Options_writeThreadSlowYieldUsec( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->source_compaction_factor; + return reinterpret_cast(jhandle)-> + write_thread_slow_yield_usec; } /* * Class: org_rocksdb_Options - * Method: setSourceCompactionFactor - * Signature: (JI)V + * Method: setSkipStatsUpdateOnDbOpen + * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setSourceCompactionFactor( +void Java_org_rocksdb_Options_setSkipStatsUpdateOnDbOpen( JNIEnv* env, jobject jobj, jlong jhandle, - jint jsource_compaction_factor) { - reinterpret_cast( - jhandle)->source_compaction_factor = - static_cast(jsource_compaction_factor); + jboolean jskip_stats_update_on_db_open) { + auto* opt = reinterpret_cast(jhandle); + opt->skip_stats_update_on_db_open = + static_cast(jskip_stats_update_on_db_open); } /* * Class: org_rocksdb_Options - * Method: maxGrandparentOverlapFactor - * Signature: (J)I + * Method: skipStatsUpdateOnDbOpen + * Signature: (J)Z */ -jint Java_org_rocksdb_Options_maxGrandparentOverlapFactor( +jboolean Java_org_rocksdb_Options_skipStatsUpdateOnDbOpen( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->max_grandparent_overlap_factor; + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->skip_stats_update_on_db_open); } /* * Class: org_rocksdb_Options - * Method: setMaxGrandparentOverlapFactor - * Signature: (JI)V + * Method: setWalRecoveryMode + * Signature: (JB)V */ -void Java_org_rocksdb_Options_setMaxGrandparentOverlapFactor( - JNIEnv* env, jobject jobj, jlong jhandle, - jint jmax_grandparent_overlap_factor) { - reinterpret_cast( - jhandle)->max_grandparent_overlap_factor = - static_cast(jmax_grandparent_overlap_factor); +void Java_org_rocksdb_Options_setWalRecoveryMode( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jwal_recovery_mode_value) { + auto* opt = reinterpret_cast(jhandle); + opt->wal_recovery_mode = + rocksdb::WALRecoveryModeJni::toCppWALRecoveryMode( + jwal_recovery_mode_value); } /* * Class: org_rocksdb_Options - * Method: softRateLimit - * Signature: (J)D + * Method: walRecoveryMode + * Signature: (J)B */ -jdouble Java_org_rocksdb_Options_softRateLimit( +jbyte Java_org_rocksdb_Options_walRecoveryMode( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->soft_rate_limit; + auto* opt = reinterpret_cast(jhandle); + return rocksdb::WALRecoveryModeJni::toJavaWALRecoveryMode( + opt->wal_recovery_mode); } /* * Class: org_rocksdb_Options - * Method: setSoftRateLimit - * Signature: (JD)V + * Method: setAllow2pc + * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setSoftRateLimit( - JNIEnv* env, jobject jobj, jlong jhandle, jdouble jsoft_rate_limit) { - reinterpret_cast(jhandle)->soft_rate_limit = - static_cast(jsoft_rate_limit); +void Java_org_rocksdb_Options_setAllow2pc( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jallow_2pc) { + auto* opt = reinterpret_cast(jhandle); + opt->allow_2pc = static_cast(jallow_2pc); } /* * Class: org_rocksdb_Options - * Method: hardRateLimit - * Signature: (J)D + * Method: allow2pc + * Signature: (J)Z */ -jdouble Java_org_rocksdb_Options_hardRateLimit( - JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->hard_rate_limit; +jboolean Java_org_rocksdb_Options_allow2pc(JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->allow_2pc); } /* * Class: org_rocksdb_Options - * Method: setHardRateLimit - * Signature: (JD)V + * Method: setRowCache + * Signature: (JJ)V */ -void Java_org_rocksdb_Options_setHardRateLimit( - JNIEnv* env, jobject jobj, jlong jhandle, jdouble jhard_rate_limit) { - reinterpret_cast(jhandle)->hard_rate_limit = - static_cast(jhard_rate_limit); +void Java_org_rocksdb_Options_setRowCache( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jrow_cache_handle) { + auto* opt = reinterpret_cast(jhandle); + auto* row_cache = reinterpret_cast*>(jrow_cache_handle); + opt->row_cache = *row_cache; } /* * Class: org_rocksdb_Options - * Method: rateLimitDelayMaxMilliseconds - * Signature: (J)I + * Method: setFailIfOptionsFileError + * Signature: (JZ)V + */ +void Java_org_rocksdb_Options_setFailIfOptionsFileError( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jfail_if_options_file_error) { + auto* opt = reinterpret_cast(jhandle); + opt->fail_if_options_file_error = + static_cast(jfail_if_options_file_error); +} + +/* + * Class: org_rocksdb_Options + * Method: failIfOptionsFileError + * Signature: (J)Z */ -jint Java_org_rocksdb_Options_rateLimitDelayMaxMilliseconds( +jboolean Java_org_rocksdb_Options_failIfOptionsFileError( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->rate_limit_delay_max_milliseconds; + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->fail_if_options_file_error); } /* * Class: org_rocksdb_Options - * Method: setRateLimitDelayMaxMilliseconds - * Signature: (JI)V + * Method: setDumpMallocStats + * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setRateLimitDelayMaxMilliseconds( - JNIEnv* env, jobject jobj, jlong jhandle, - jint jrate_limit_delay_max_milliseconds) { - reinterpret_cast( - jhandle)->rate_limit_delay_max_milliseconds = - static_cast(jrate_limit_delay_max_milliseconds); +void Java_org_rocksdb_Options_setDumpMallocStats( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jdump_malloc_stats) { + auto* opt = reinterpret_cast(jhandle); + opt->dump_malloc_stats = static_cast(jdump_malloc_stats); } /* * Class: org_rocksdb_Options - * Method: arenaBlockSize - * Signature: (J)J + * Method: dumpMallocStats + * Signature: (J)Z */ -jlong Java_org_rocksdb_Options_arenaBlockSize( +jboolean Java_org_rocksdb_Options_dumpMallocStats( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->arena_block_size; + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->dump_malloc_stats); } /* * Class: org_rocksdb_Options - * Method: setArenaBlockSize - * Signature: (JJ)V + * Method: setAvoidFlushDuringRecovery + * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setArenaBlockSize( - JNIEnv* env, jobject jobj, jlong jhandle, jlong jarena_block_size) { - reinterpret_cast(jhandle)->arena_block_size = - static_cast(jarena_block_size); +void Java_org_rocksdb_Options_setAvoidFlushDuringRecovery( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean javoid_flush_during_recovery) { + auto* opt = reinterpret_cast(jhandle); + opt->avoid_flush_during_recovery = static_cast(javoid_flush_during_recovery); } /* * Class: org_rocksdb_Options - * Method: disableAutoCompactions + * Method: avoidFlushDuringRecovery * Signature: (J)Z */ -jboolean Java_org_rocksdb_Options_disableAutoCompactions( +jboolean Java_org_rocksdb_Options_avoidFlushDuringRecovery( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->disable_auto_compactions; + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->avoid_flush_during_recovery); } /* * Class: org_rocksdb_Options - * Method: setDisableAutoCompactions + * Method: setAvoidFlushDuringShutdown * Signature: (JZ)V */ -void Java_org_rocksdb_Options_setDisableAutoCompactions( +void Java_org_rocksdb_Options_setAvoidFlushDuringShutdown( JNIEnv* env, jobject jobj, jlong jhandle, - jboolean jdisable_auto_compactions) { - reinterpret_cast( - jhandle)->disable_auto_compactions = - static_cast(jdisable_auto_compactions); + jboolean javoid_flush_during_shutdown) { + auto* opt = reinterpret_cast(jhandle); + opt->avoid_flush_during_shutdown = static_cast(javoid_flush_during_shutdown); } /* * Class: org_rocksdb_Options - * Method: purgeRedundantKvsWhileFlush + * Method: avoidFlushDuringShutdown * Signature: (J)Z */ -jboolean Java_org_rocksdb_Options_purgeRedundantKvsWhileFlush( +jboolean Java_org_rocksdb_Options_avoidFlushDuringShutdown( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->avoid_flush_during_shutdown); +} + +/* + * Method: tableFactoryName + * Signature: (J)Ljava/lang/String + */ +jstring Java_org_rocksdb_Options_tableFactoryName( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + rocksdb::TableFactory* tf = opt->table_factory.get(); + + // Should never be nullptr. + // Default memtable factory is SkipListFactory + assert(tf); + + return env->NewStringUTF(tf->Name()); +} + + +/* + * Class: org_rocksdb_Options + * Method: minWriteBufferNumberToMerge + * Signature: (J)I + */ +jint Java_org_rocksdb_Options_minWriteBufferNumberToMerge( JNIEnv* env, jobject jobj, jlong jhandle) { return reinterpret_cast( - jhandle)->purge_redundant_kvs_while_flush; + jhandle)->min_write_buffer_number_to_merge; } /* * Class: org_rocksdb_Options - * Method: setPurgeRedundantKvsWhileFlush - * Signature: (JZ)V + * Method: setMinWriteBufferNumberToMerge + * Signature: (JI)V */ -void Java_org_rocksdb_Options_setPurgeRedundantKvsWhileFlush( +void Java_org_rocksdb_Options_setMinWriteBufferNumberToMerge( JNIEnv* env, jobject jobj, jlong jhandle, - jboolean jpurge_redundant_kvs_while_flush) { + jint jmin_write_buffer_number_to_merge) { reinterpret_cast( - jhandle)->purge_redundant_kvs_while_flush = - static_cast(jpurge_redundant_kvs_while_flush); + jhandle)->min_write_buffer_number_to_merge = + static_cast(jmin_write_buffer_number_to_merge); +} +/* + * Class: org_rocksdb_Options + * Method: maxWriteBufferNumberToMaintain + * Signature: (J)I + */ +jint Java_org_rocksdb_Options_maxWriteBufferNumberToMaintain(JNIEnv* env, + jobject jobj, + jlong jhandle) { + return reinterpret_cast(jhandle) + ->max_write_buffer_number_to_maintain; } /* * Class: org_rocksdb_Options - * Method: verifyChecksumsInCompaction - * Signature: (J)Z + * Method: setMaxWriteBufferNumberToMaintain + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setMaxWriteBufferNumberToMaintain( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jmax_write_buffer_number_to_maintain) { + reinterpret_cast(jhandle) + ->max_write_buffer_number_to_maintain = + static_cast(jmax_write_buffer_number_to_maintain); +} + +/* + * Class: org_rocksdb_Options + * Method: setCompressionType + * Signature: (JB)V + */ +void Java_org_rocksdb_Options_setCompressionType( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jcompression_type_value) { + auto* opts = reinterpret_cast(jhandle); + opts->compression = rocksdb::CompressionTypeJni::toCppCompressionType( + jcompression_type_value); +} + +/* + * Class: org_rocksdb_Options + * Method: compressionType + * Signature: (J)B */ -jboolean Java_org_rocksdb_Options_verifyChecksumsInCompaction( +jbyte Java_org_rocksdb_Options_compressionType( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->verify_checksums_in_compaction; + auto* opts = reinterpret_cast(jhandle); + return rocksdb::CompressionTypeJni::toJavaCompressionType( + opts->compression); +} + +/** + * Helper method to convert a Java byte array of compression levels + * to a C++ vector of rocksdb::CompressionType + * + * @param env A pointer to the Java environment + * @param jcompression_levels A reference to a java byte array + * where each byte indicates a compression level + * + * @return A unique_ptr to the vector, or unique_ptr(nullptr) if a JNI exception occurs + */ +std::unique_ptr> rocksdb_compression_vector_helper( + JNIEnv* env, jbyteArray jcompression_levels) { + jsize len = env->GetArrayLength(jcompression_levels); + jbyte* jcompression_level = + env->GetByteArrayElements(jcompression_levels, nullptr); + if(jcompression_level == nullptr) { + // exception thrown: OutOfMemoryError + return std::unique_ptr>(); + } + + auto* compression_levels = new std::vector(); + std::unique_ptr> uptr_compression_levels(compression_levels); + + for(jsize i = 0; i < len; i++) { + jbyte jcl = jcompression_level[i]; + compression_levels->push_back(static_cast(jcl)); + } + + env->ReleaseByteArrayElements(jcompression_levels, jcompression_level, + JNI_ABORT); + + return uptr_compression_levels; +} + +/** + * Helper method to convert a C++ vector of rocksdb::CompressionType + * to a Java byte array of compression levels + * + * @param env A pointer to the Java environment + * @param jcompression_levels A reference to a java byte array + * where each byte indicates a compression level + * + * @return A jbytearray or nullptr if an exception occurs + */ +jbyteArray rocksdb_compression_list_helper(JNIEnv* env, + std::vector compression_levels) { + const size_t len = compression_levels.size(); + jbyte* jbuf = new jbyte[len]; + + for (size_t i = 0; i < len; i++) { + jbuf[i] = compression_levels[i]; + } + + // insert in java array + jbyteArray jcompression_levels = env->NewByteArray(static_cast(len)); + if(jcompression_levels == nullptr) { + // exception thrown: OutOfMemoryError + delete [] jbuf; + return nullptr; + } + env->SetByteArrayRegion(jcompression_levels, 0, static_cast(len), + jbuf); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jcompression_levels); + delete [] jbuf; + return nullptr; + } + + delete [] jbuf; + + return jcompression_levels; } /* * Class: org_rocksdb_Options - * Method: setVerifyChecksumsInCompaction - * Signature: (JZ)V + * Method: setCompressionPerLevel + * Signature: (J[B)V */ -void Java_org_rocksdb_Options_setVerifyChecksumsInCompaction( +void Java_org_rocksdb_Options_setCompressionPerLevel( JNIEnv* env, jobject jobj, jlong jhandle, - jboolean jverify_checksums_in_compaction) { - reinterpret_cast( - jhandle)->verify_checksums_in_compaction = - static_cast(jverify_checksums_in_compaction); + jbyteArray jcompressionLevels) { + auto uptr_compression_levels = + rocksdb_compression_vector_helper(env, jcompressionLevels); + if(!uptr_compression_levels) { + // exception occurred + return; + } + auto* options = reinterpret_cast(jhandle); + options->compression_per_level = *(uptr_compression_levels.get()); } /* * Class: org_rocksdb_Options - * Method: filterDeletes - * Signature: (J)Z + * Method: compressionPerLevel + * Signature: (J)[B */ -jboolean Java_org_rocksdb_Options_filterDeletes( +jbyteArray Java_org_rocksdb_Options_compressionPerLevel( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->filter_deletes; + auto* options = reinterpret_cast(jhandle); + return rocksdb_compression_list_helper(env, + options->compression_per_level); } /* * Class: org_rocksdb_Options - * Method: setFilterDeletes - * Signature: (JZ)V + * Method: setBottommostCompressionType + * Signature: (JB)V */ -void Java_org_rocksdb_Options_setFilterDeletes( - JNIEnv* env, jobject jobj, jlong jhandle, jboolean jfilter_deletes) { - reinterpret_cast(jhandle)->filter_deletes = - static_cast(jfilter_deletes); +void Java_org_rocksdb_Options_setBottommostCompressionType( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jcompression_type_value) { + auto* options = reinterpret_cast(jhandle); + options->bottommost_compression = + rocksdb::CompressionTypeJni::toCppCompressionType( + jcompression_type_value); } /* * Class: org_rocksdb_Options - * Method: maxSequentialSkipInIterations - * Signature: (J)J + * Method: bottommostCompressionType + * Signature: (J)B */ -jlong Java_org_rocksdb_Options_maxSequentialSkipInIterations( +jbyte Java_org_rocksdb_Options_bottommostCompressionType( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->max_sequential_skip_in_iterations; + auto* options = reinterpret_cast(jhandle); + return rocksdb::CompressionTypeJni::toJavaCompressionType( + options->bottommost_compression); } /* * Class: org_rocksdb_Options - * Method: setMaxSequentialSkipInIterations + * Method: setCompressionOptions * Signature: (JJ)V */ -void Java_org_rocksdb_Options_setMaxSequentialSkipInIterations( +void Java_org_rocksdb_Options_setCompressionOptions( JNIEnv* env, jobject jobj, jlong jhandle, - jlong jmax_sequential_skip_in_iterations) { - reinterpret_cast( - jhandle)->max_sequential_skip_in_iterations = - static_cast(jmax_sequential_skip_in_iterations); + jlong jcompression_options_handle) { + auto* options = reinterpret_cast(jhandle); + auto* compression_options = + reinterpret_cast(jcompression_options_handle); + options->compression_opts = *compression_options; } /* * Class: org_rocksdb_Options - * Method: inplaceUpdateSupport - * Signature: (J)Z + * Method: setCompactionStyle + * Signature: (JB)V */ -jboolean Java_org_rocksdb_Options_inplaceUpdateSupport( +void Java_org_rocksdb_Options_setCompactionStyle( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte compaction_style) { + reinterpret_cast(jhandle)->compaction_style = + static_cast(compaction_style); +} + +/* + * Class: org_rocksdb_Options + * Method: compactionStyle + * Signature: (J)B + */ +jbyte Java_org_rocksdb_Options_compactionStyle( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->inplace_update_support; + return reinterpret_cast(jhandle)->compaction_style; } /* * Class: org_rocksdb_Options - * Method: setInplaceUpdateSupport - * Signature: (JZ)V + * Method: setMaxTableFilesSizeFIFO + * Signature: (JJ)V */ -void Java_org_rocksdb_Options_setInplaceUpdateSupport( - JNIEnv* env, jobject jobj, jlong jhandle, - jboolean jinplace_update_support) { - reinterpret_cast( - jhandle)->inplace_update_support = - static_cast(jinplace_update_support); +void Java_org_rocksdb_Options_setMaxTableFilesSizeFIFO( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jmax_table_files_size) { + reinterpret_cast(jhandle)->compaction_options_fifo.max_table_files_size = + static_cast(jmax_table_files_size); } /* * Class: org_rocksdb_Options - * Method: inplaceUpdateNumLocks + * Method: maxTableFilesSizeFIFO * Signature: (J)J */ -jlong Java_org_rocksdb_Options_inplaceUpdateNumLocks( +jlong Java_org_rocksdb_Options_maxTableFilesSizeFIFO( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast( - jhandle)->inplace_update_num_locks; + return reinterpret_cast(jhandle)->compaction_options_fifo.max_table_files_size; } /* * Class: org_rocksdb_Options - * Method: setInplaceUpdateNumLocks - * Signature: (JJ)V + * Method: numLevels + * Signature: (J)I */ -void Java_org_rocksdb_Options_setInplaceUpdateNumLocks( - JNIEnv* env, jobject jobj, jlong jhandle, - jlong jinplace_update_num_locks) { - reinterpret_cast( - jhandle)->inplace_update_num_locks = - static_cast(jinplace_update_num_locks); +jint Java_org_rocksdb_Options_numLevels( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->num_levels; +} + +/* + * Class: org_rocksdb_Options + * Method: setNumLevels + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setNumLevels( + JNIEnv* env, jobject jobj, jlong jhandle, jint jnum_levels) { + reinterpret_cast(jhandle)->num_levels = + static_cast(jnum_levels); } /* * Class: org_rocksdb_Options - * Method: memtablePrefixBloomBits + * Method: levelZeroFileNumCompactionTrigger * Signature: (J)I */ -jint Java_org_rocksdb_Options_memtablePrefixBloomBits( +jint Java_org_rocksdb_Options_levelZeroFileNumCompactionTrigger( JNIEnv* env, jobject jobj, jlong jhandle) { return reinterpret_cast( - jhandle)->memtable_prefix_bloom_bits; + jhandle)->level0_file_num_compaction_trigger; } /* * Class: org_rocksdb_Options - * Method: setMemtablePrefixBloomBits + * Method: setLevelZeroFileNumCompactionTrigger * Signature: (JI)V */ -void Java_org_rocksdb_Options_setMemtablePrefixBloomBits( +void Java_org_rocksdb_Options_setLevelZeroFileNumCompactionTrigger( JNIEnv* env, jobject jobj, jlong jhandle, - jint jmemtable_prefix_bloom_bits) { + jint jlevel0_file_num_compaction_trigger) { reinterpret_cast( - jhandle)->memtable_prefix_bloom_bits = - static_cast(jmemtable_prefix_bloom_bits); + jhandle)->level0_file_num_compaction_trigger = + static_cast(jlevel0_file_num_compaction_trigger); } /* * Class: org_rocksdb_Options - * Method: memtablePrefixBloomProbes + * Method: levelZeroSlowdownWritesTrigger * Signature: (J)I */ -jint Java_org_rocksdb_Options_memtablePrefixBloomProbes( +jint Java_org_rocksdb_Options_levelZeroSlowdownWritesTrigger( JNIEnv* env, jobject jobj, jlong jhandle) { return reinterpret_cast( - jhandle)->memtable_prefix_bloom_probes; + jhandle)->level0_slowdown_writes_trigger; } /* * Class: org_rocksdb_Options - * Method: setMemtablePrefixBloomProbes + * Method: setLevelSlowdownWritesTrigger * Signature: (JI)V */ -void Java_org_rocksdb_Options_setMemtablePrefixBloomProbes( +void Java_org_rocksdb_Options_setLevelZeroSlowdownWritesTrigger( JNIEnv* env, jobject jobj, jlong jhandle, - jint jmemtable_prefix_bloom_probes) { + jint jlevel0_slowdown_writes_trigger) { reinterpret_cast( - jhandle)->memtable_prefix_bloom_probes = - static_cast(jmemtable_prefix_bloom_probes); + jhandle)->level0_slowdown_writes_trigger = + static_cast(jlevel0_slowdown_writes_trigger); } /* * Class: org_rocksdb_Options - * Method: bloomLocality + * Method: levelZeroStopWritesTrigger * Signature: (J)I */ -jint Java_org_rocksdb_Options_bloomLocality( +jint Java_org_rocksdb_Options_levelZeroStopWritesTrigger( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->bloom_locality; + return reinterpret_cast( + jhandle)->level0_stop_writes_trigger; } /* * Class: org_rocksdb_Options - * Method: setBloomLocality + * Method: setLevelStopWritesTrigger * Signature: (JI)V */ -void Java_org_rocksdb_Options_setBloomLocality( - JNIEnv* env, jobject jobj, jlong jhandle, jint jbloom_locality) { - reinterpret_cast(jhandle)->bloom_locality = - static_cast(jbloom_locality); +void Java_org_rocksdb_Options_setLevelZeroStopWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jlevel0_stop_writes_trigger) { + reinterpret_cast(jhandle)->level0_stop_writes_trigger = + static_cast(jlevel0_stop_writes_trigger); } /* * Class: org_rocksdb_Options - * Method: maxSuccessiveMerges + * Method: targetFileSizeBase * Signature: (J)J */ -jlong Java_org_rocksdb_Options_maxSuccessiveMerges( +jlong Java_org_rocksdb_Options_targetFileSizeBase( JNIEnv* env, jobject jobj, jlong jhandle) { - return reinterpret_cast(jhandle)->max_successive_merges; + return reinterpret_cast(jhandle)->target_file_size_base; } /* * Class: org_rocksdb_Options - * Method: setMaxSuccessiveMerges + * Method: setTargetFileSizeBase * Signature: (JJ)V */ -void Java_org_rocksdb_Options_setMaxSuccessiveMerges( +void Java_org_rocksdb_Options_setTargetFileSizeBase( JNIEnv* env, jobject jobj, jlong jhandle, - jlong jmax_successive_merges) { - reinterpret_cast(jhandle)->max_successive_merges = - static_cast(jmax_successive_merges); + jlong jtarget_file_size_base) { + reinterpret_cast(jhandle)->target_file_size_base = + static_cast(jtarget_file_size_base); } /* * Class: org_rocksdb_Options - * Method: minPartialMergeOperands + * Method: targetFileSizeMultiplier * Signature: (J)I */ -jint Java_org_rocksdb_Options_minPartialMergeOperands( +jint Java_org_rocksdb_Options_targetFileSizeMultiplier( JNIEnv* env, jobject jobj, jlong jhandle) { return reinterpret_cast( - jhandle)->min_partial_merge_operands; + jhandle)->target_file_size_multiplier; } /* * Class: org_rocksdb_Options - * Method: setMinPartialMergeOperands + * Method: setTargetFileSizeMultiplier * Signature: (JI)V */ -void Java_org_rocksdb_Options_setMinPartialMergeOperands( +void Java_org_rocksdb_Options_setTargetFileSizeMultiplier( JNIEnv* env, jobject jobj, jlong jhandle, - jint jmin_partial_merge_operands) { + jint jtarget_file_size_multiplier) { reinterpret_cast( - jhandle)->min_partial_merge_operands = - static_cast(jmin_partial_merge_operands); + jhandle)->target_file_size_multiplier = + static_cast(jtarget_file_size_multiplier); } -////////////////////////////////////////////////////////////////////////////// -// WriteOptions - /* - * Class: org_rocksdb_WriteOptions - * Method: newWriteOptions - * Signature: ()V + * Class: org_rocksdb_Options + * Method: maxBytesForLevelBase + * Signature: (J)J */ -void Java_org_rocksdb_WriteOptions_newWriteOptions( - JNIEnv* env, jobject jwrite_options) { - rocksdb::WriteOptions* op = new rocksdb::WriteOptions(); - rocksdb::WriteOptionsJni::setHandle(env, jwrite_options, op); -} - +jlong Java_org_rocksdb_Options_maxBytesForLevelBase( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->max_bytes_for_level_base; +} + +/* + * Class: org_rocksdb_Options + * Method: setMaxBytesForLevelBase + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setMaxBytesForLevelBase( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jmax_bytes_for_level_base) { + reinterpret_cast( + jhandle)->max_bytes_for_level_base = + static_cast(jmax_bytes_for_level_base); +} + +/* + * Class: org_rocksdb_Options + * Method: levelCompactionDynamicLevelBytes + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_Options_levelCompactionDynamicLevelBytes( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level_compaction_dynamic_level_bytes; +} + +/* + * Class: org_rocksdb_Options + * Method: setLevelCompactionDynamicLevelBytes + * Signature: (JZ)V + */ +void Java_org_rocksdb_Options_setLevelCompactionDynamicLevelBytes( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jenable_dynamic_level_bytes) { + reinterpret_cast( + jhandle)->level_compaction_dynamic_level_bytes = + (jenable_dynamic_level_bytes); +} + +/* + * Class: org_rocksdb_Options + * Method: maxBytesForLevelMultiplier + * Signature: (J)D + */ +jdouble Java_org_rocksdb_Options_maxBytesForLevelMultiplier(JNIEnv* env, + jobject jobj, + jlong jhandle) { + return reinterpret_cast( + jhandle)->max_bytes_for_level_multiplier; +} + +/* + * Class: org_rocksdb_Options + * Method: setMaxBytesForLevelMultiplier + * Signature: (JD)V + */ +void Java_org_rocksdb_Options_setMaxBytesForLevelMultiplier( + JNIEnv* env, jobject jobj, jlong jhandle, + jdouble jmax_bytes_for_level_multiplier) { + reinterpret_cast(jhandle)->max_bytes_for_level_multiplier = + static_cast(jmax_bytes_for_level_multiplier); +} + +/* + * Class: org_rocksdb_Options + * Method: maxCompactionBytes + * Signature: (J)I + */ +jlong Java_org_rocksdb_Options_maxCompactionBytes(JNIEnv* env, jobject jobj, + jlong jhandle) { + return static_cast( + reinterpret_cast(jhandle)->max_compaction_bytes); +} + +/* + * Class: org_rocksdb_Options + * Method: setMaxCompactionBytes + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setMaxCompactionBytes( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jmax_compaction_bytes) { + reinterpret_cast(jhandle)->max_compaction_bytes = + static_cast(jmax_compaction_bytes); +} + +/* + * Class: org_rocksdb_Options + * Method: arenaBlockSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_Options_arenaBlockSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->arena_block_size; +} + +/* + * Class: org_rocksdb_Options + * Method: setArenaBlockSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setArenaBlockSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jarena_block_size) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(jarena_block_size); + if (s.ok()) { + reinterpret_cast(jhandle)->arena_block_size = + jarena_block_size; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_Options + * Method: disableAutoCompactions + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_Options_disableAutoCompactions( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->disable_auto_compactions; +} + +/* + * Class: org_rocksdb_Options + * Method: setDisableAutoCompactions + * Signature: (JZ)V + */ +void Java_org_rocksdb_Options_setDisableAutoCompactions( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jdisable_auto_compactions) { + reinterpret_cast( + jhandle)->disable_auto_compactions = + static_cast(jdisable_auto_compactions); +} + +/* + * Class: org_rocksdb_Options + * Method: maxSequentialSkipInIterations + * Signature: (J)J + */ +jlong Java_org_rocksdb_Options_maxSequentialSkipInIterations( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->max_sequential_skip_in_iterations; +} + +/* + * Class: org_rocksdb_Options + * Method: setMaxSequentialSkipInIterations + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setMaxSequentialSkipInIterations( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jmax_sequential_skip_in_iterations) { + reinterpret_cast( + jhandle)->max_sequential_skip_in_iterations = + static_cast(jmax_sequential_skip_in_iterations); +} + +/* + * Class: org_rocksdb_Options + * Method: inplaceUpdateSupport + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_Options_inplaceUpdateSupport( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->inplace_update_support; +} + +/* + * Class: org_rocksdb_Options + * Method: setInplaceUpdateSupport + * Signature: (JZ)V + */ +void Java_org_rocksdb_Options_setInplaceUpdateSupport( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jinplace_update_support) { + reinterpret_cast( + jhandle)->inplace_update_support = + static_cast(jinplace_update_support); +} + +/* + * Class: org_rocksdb_Options + * Method: inplaceUpdateNumLocks + * Signature: (J)J + */ +jlong Java_org_rocksdb_Options_inplaceUpdateNumLocks( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->inplace_update_num_locks; +} + +/* + * Class: org_rocksdb_Options + * Method: setInplaceUpdateNumLocks + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setInplaceUpdateNumLocks( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jinplace_update_num_locks) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t( + jinplace_update_num_locks); + if (s.ok()) { + reinterpret_cast(jhandle)->inplace_update_num_locks = + jinplace_update_num_locks; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_Options + * Method: memtablePrefixBloomSizeRatio + * Signature: (J)I + */ +jdouble Java_org_rocksdb_Options_memtablePrefixBloomSizeRatio(JNIEnv* env, + jobject jobj, + jlong jhandle) { + return reinterpret_cast(jhandle) + ->memtable_prefix_bloom_size_ratio; +} + +/* + * Class: org_rocksdb_Options + * Method: setMemtablePrefixBloomSizeRatio + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setMemtablePrefixBloomSizeRatio( + JNIEnv* env, jobject jobj, jlong jhandle, + jdouble jmemtable_prefix_bloom_size_ratio) { + reinterpret_cast(jhandle) + ->memtable_prefix_bloom_size_ratio = + static_cast(jmemtable_prefix_bloom_size_ratio); +} + +/* + * Class: org_rocksdb_Options + * Method: bloomLocality + * Signature: (J)I + */ +jint Java_org_rocksdb_Options_bloomLocality( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->bloom_locality; +} + +/* + * Class: org_rocksdb_Options + * Method: setBloomLocality + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setBloomLocality( + JNIEnv* env, jobject jobj, jlong jhandle, jint jbloom_locality) { + reinterpret_cast(jhandle)->bloom_locality = + static_cast(jbloom_locality); +} + +/* + * Class: org_rocksdb_Options + * Method: maxSuccessiveMerges + * Signature: (J)J + */ +jlong Java_org_rocksdb_Options_maxSuccessiveMerges( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->max_successive_merges; +} + +/* + * Class: org_rocksdb_Options + * Method: setMaxSuccessiveMerges + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setMaxSuccessiveMerges( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jmax_successive_merges) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t( + jmax_successive_merges); + if (s.ok()) { + reinterpret_cast(jhandle)->max_successive_merges = + jmax_successive_merges; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_Options + * Method: optimizeFiltersForHits + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_Options_optimizeFiltersForHits( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->optimize_filters_for_hits; +} + +/* + * Class: org_rocksdb_Options + * Method: setOptimizeFiltersForHits + * Signature: (JZ)V + */ +void Java_org_rocksdb_Options_setOptimizeFiltersForHits( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean joptimize_filters_for_hits) { + reinterpret_cast( + jhandle)->optimize_filters_for_hits = + static_cast(joptimize_filters_for_hits); +} + +/* + * Class: org_rocksdb_Options + * Method: optimizeForSmallDb + * Signature: (J)V + */ +void Java_org_rocksdb_Options_optimizeForSmallDb( + JNIEnv* env, jobject jobj, jlong jhandle) { + reinterpret_cast(jhandle)->OptimizeForSmallDb(); +} + +/* + * Class: org_rocksdb_Options + * Method: optimizeForPointLookup + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_optimizeForPointLookup( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong block_cache_size_mb) { + reinterpret_cast(jhandle)-> + OptimizeForPointLookup(block_cache_size_mb); +} + +/* + * Class: org_rocksdb_Options + * Method: optimizeLevelStyleCompaction + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_optimizeLevelStyleCompaction( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong memtable_memory_budget) { + reinterpret_cast(jhandle)-> + OptimizeLevelStyleCompaction(memtable_memory_budget); +} + +/* + * Class: org_rocksdb_Options + * Method: optimizeUniversalStyleCompaction + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_optimizeUniversalStyleCompaction( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong memtable_memory_budget) { + reinterpret_cast(jhandle)-> + OptimizeUniversalStyleCompaction(memtable_memory_budget); +} + +/* + * Class: org_rocksdb_Options + * Method: prepareForBulkLoad + * Signature: (J)V + */ +void Java_org_rocksdb_Options_prepareForBulkLoad( + JNIEnv* env, jobject jobj, jlong jhandle) { + reinterpret_cast(jhandle)-> + PrepareForBulkLoad(); +} + +/* + * Class: org_rocksdb_Options + * Method: memtableHugePageSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_Options_memtableHugePageSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->memtable_huge_page_size; +} + +/* + * Class: org_rocksdb_Options + * Method: setMemtableHugePageSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setMemtableHugePageSize( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jmemtable_huge_page_size) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t( + jmemtable_huge_page_size); + if (s.ok()) { + reinterpret_cast( + jhandle)->memtable_huge_page_size = + jmemtable_huge_page_size; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_Options + * Method: softPendingCompactionBytesLimit + * Signature: (J)J + */ +jlong Java_org_rocksdb_Options_softPendingCompactionBytesLimit( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->soft_pending_compaction_bytes_limit; +} + +/* + * Class: org_rocksdb_Options + * Method: setSoftPendingCompactionBytesLimit + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setSoftPendingCompactionBytesLimit( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jsoft_pending_compaction_bytes_limit) { + reinterpret_cast( + jhandle)->soft_pending_compaction_bytes_limit = + static_cast(jsoft_pending_compaction_bytes_limit); +} + +/* + * Class: org_rocksdb_Options + * Method: softHardCompactionBytesLimit + * Signature: (J)J + */ +jlong Java_org_rocksdb_Options_hardPendingCompactionBytesLimit( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->hard_pending_compaction_bytes_limit; +} + +/* + * Class: org_rocksdb_Options + * Method: setHardPendingCompactionBytesLimit + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setHardPendingCompactionBytesLimit( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jhard_pending_compaction_bytes_limit) { + reinterpret_cast( + jhandle)->hard_pending_compaction_bytes_limit = + static_cast(jhard_pending_compaction_bytes_limit); +} + +/* + * Class: org_rocksdb_Options + * Method: level0FileNumCompactionTrigger + * Signature: (J)I + */ +jint Java_org_rocksdb_Options_level0FileNumCompactionTrigger( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level0_file_num_compaction_trigger; +} + +/* + * Class: org_rocksdb_Options + * Method: setLevel0FileNumCompactionTrigger + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setLevel0FileNumCompactionTrigger( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jlevel0_file_num_compaction_trigger) { + reinterpret_cast( + jhandle)->level0_file_num_compaction_trigger = + static_cast(jlevel0_file_num_compaction_trigger); +} + +/* + * Class: org_rocksdb_Options + * Method: level0SlowdownWritesTrigger + * Signature: (J)I + */ +jint Java_org_rocksdb_Options_level0SlowdownWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level0_slowdown_writes_trigger; +} + +/* + * Class: org_rocksdb_Options + * Method: setLevel0SlowdownWritesTrigger + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setLevel0SlowdownWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jlevel0_slowdown_writes_trigger) { + reinterpret_cast( + jhandle)->level0_slowdown_writes_trigger = + static_cast(jlevel0_slowdown_writes_trigger); +} + +/* + * Class: org_rocksdb_Options + * Method: level0StopWritesTrigger + * Signature: (J)I + */ +jint Java_org_rocksdb_Options_level0StopWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level0_stop_writes_trigger; +} + +/* + * Class: org_rocksdb_Options + * Method: setLevel0StopWritesTrigger + * Signature: (JI)V + */ +void Java_org_rocksdb_Options_setLevel0StopWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jlevel0_stop_writes_trigger) { + reinterpret_cast( + jhandle)->level0_stop_writes_trigger = + static_cast(jlevel0_stop_writes_trigger); +} + +/* + * Class: org_rocksdb_Options + * Method: maxBytesForLevelMultiplierAdditional + * Signature: (J)[I + */ +jintArray Java_org_rocksdb_Options_maxBytesForLevelMultiplierAdditional( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto mbflma = + reinterpret_cast(jhandle)-> + max_bytes_for_level_multiplier_additional; + + const size_t size = mbflma.size(); + + jint* additionals = new jint[size]; + for (size_t i = 0; i < size; i++) { + additionals[i] = static_cast(mbflma[i]); + } + + jsize jlen = static_cast(size); + jintArray result = env->NewIntArray(jlen); + if(result == nullptr) { + // exception thrown: OutOfMemoryError + delete [] additionals; + return nullptr; + } + + env->SetIntArrayRegion(result, 0, jlen, additionals); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(result); + delete [] additionals; + return nullptr; + } + + delete [] additionals; + + return result; +} + +/* + * Class: org_rocksdb_Options + * Method: setMaxBytesForLevelMultiplierAdditional + * Signature: (J[I)V + */ +void Java_org_rocksdb_Options_setMaxBytesForLevelMultiplierAdditional( + JNIEnv* env, jobject jobj, jlong jhandle, + jintArray jmax_bytes_for_level_multiplier_additional) { + jsize len = env->GetArrayLength(jmax_bytes_for_level_multiplier_additional); + jint *additionals = + env->GetIntArrayElements(jmax_bytes_for_level_multiplier_additional, nullptr); + if(additionals == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + auto* opt = reinterpret_cast(jhandle); + opt->max_bytes_for_level_multiplier_additional.clear(); + for (jsize i = 0; i < len; i++) { + opt->max_bytes_for_level_multiplier_additional.push_back(static_cast(additionals[i])); + } + + env->ReleaseIntArrayElements(jmax_bytes_for_level_multiplier_additional, + additionals, JNI_ABORT); +} + +/* + * Class: org_rocksdb_Options + * Method: paranoidFileChecks + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_Options_paranoidFileChecks( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->paranoid_file_checks; +} + +/* + * Class: org_rocksdb_Options + * Method: setParanoidFileChecks + * Signature: (JZ)V + */ +void Java_org_rocksdb_Options_setParanoidFileChecks( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jparanoid_file_checks) { + reinterpret_cast( + jhandle)->paranoid_file_checks = + static_cast(jparanoid_file_checks); +} + +/* + * Class: org_rocksdb_Options + * Method: setCompactionPriority + * Signature: (JB)V + */ +void Java_org_rocksdb_Options_setCompactionPriority( + JNIEnv* env, jobject jobj, jlong jhandle, + jbyte jcompaction_priority_value) { + auto* opts = reinterpret_cast(jhandle); + opts->compaction_pri = + rocksdb::CompactionPriorityJni::toCppCompactionPriority(jcompaction_priority_value); +} + +/* + * Class: org_rocksdb_Options + * Method: compactionPriority + * Signature: (J)B + */ +jbyte Java_org_rocksdb_Options_compactionPriority( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opts = reinterpret_cast(jhandle); + return rocksdb::CompactionPriorityJni::toJavaCompactionPriority( + opts->compaction_pri); +} + +/* + * Class: org_rocksdb_Options + * Method: setReportBgIoStats + * Signature: (JZ)V + */ +void Java_org_rocksdb_Options_setReportBgIoStats( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jreport_bg_io_stats) { + auto* opts = reinterpret_cast(jhandle); + opts->report_bg_io_stats = static_cast(jreport_bg_io_stats); +} + +/* + * Class: org_rocksdb_Options + * Method: reportBgIoStats + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_Options_reportBgIoStats( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opts = reinterpret_cast(jhandle); + return static_cast(opts->report_bg_io_stats); +} + +/* + * Class: org_rocksdb_Options + * Method: setCompactionOptionsUniversal + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setCompactionOptionsUniversal( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jcompaction_options_universal_handle) { + auto* opts = reinterpret_cast(jhandle); + auto* opts_uni = + reinterpret_cast( + jcompaction_options_universal_handle); + opts->compaction_options_universal = *opts_uni; +} + +/* + * Class: org_rocksdb_Options + * Method: setCompactionOptionsFIFO + * Signature: (JJ)V + */ +void Java_org_rocksdb_Options_setCompactionOptionsFIFO( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jcompaction_options_fifo_handle) { + auto* opts = reinterpret_cast(jhandle); + auto* opts_fifo = + reinterpret_cast( + jcompaction_options_fifo_handle); + opts->compaction_options_fifo = *opts_fifo; +} + +/* + * Class: org_rocksdb_Options + * Method: setForceConsistencyChecks + * Signature: (JZ)V + */ +void Java_org_rocksdb_Options_setForceConsistencyChecks( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jforce_consistency_checks) { + auto* opts = reinterpret_cast(jhandle); + opts->force_consistency_checks = static_cast(jforce_consistency_checks); +} + +/* + * Class: org_rocksdb_Options + * Method: forceConsistencyChecks + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_Options_forceConsistencyChecks( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opts = reinterpret_cast(jhandle); + return static_cast(opts->force_consistency_checks); +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::ColumnFamilyOptions + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: newColumnFamilyOptions + * Signature: ()J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_newColumnFamilyOptions( + JNIEnv* env, jclass jcls) { + auto* op = new rocksdb::ColumnFamilyOptions(); + return reinterpret_cast(op); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: getColumnFamilyOptionsFromProps + * Signature: (Ljava/util/String;)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_getColumnFamilyOptionsFromProps( + JNIEnv* env, jclass jclazz, jstring jopt_string) { + const char* opt_string = env->GetStringUTFChars(jopt_string, nullptr); + if(opt_string == nullptr) { + // exception thrown: OutOfMemoryError + return 0; + } + + auto* cf_options = new rocksdb::ColumnFamilyOptions(); + rocksdb::Status status = rocksdb::GetColumnFamilyOptionsFromString( + rocksdb::ColumnFamilyOptions(), opt_string, cf_options); + + env->ReleaseStringUTFChars(jopt_string, opt_string); + + // Check if ColumnFamilyOptions creation was possible. + jlong ret_value = 0; + if (status.ok()) { + ret_value = reinterpret_cast(cf_options); + } else { + // if operation failed the ColumnFamilyOptions need to be deleted + // again to prevent a memory leak. + delete cf_options; + } + return ret_value; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_disposeInternal( + JNIEnv* env, jobject jobj, jlong handle) { + auto* cfo = reinterpret_cast(handle); + assert(cfo != nullptr); + delete cfo; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: optimizeForSmallDb + * Signature: (J)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_optimizeForSmallDb( + JNIEnv* env, jobject jobj, jlong jhandle) { + reinterpret_cast(jhandle)-> + OptimizeForSmallDb(); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: optimizeForPointLookup + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_optimizeForPointLookup( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong block_cache_size_mb) { + reinterpret_cast(jhandle)-> + OptimizeForPointLookup(block_cache_size_mb); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: optimizeLevelStyleCompaction + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_optimizeLevelStyleCompaction( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong memtable_memory_budget) { + reinterpret_cast(jhandle)-> + OptimizeLevelStyleCompaction(memtable_memory_budget); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: optimizeUniversalStyleCompaction + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_optimizeUniversalStyleCompaction( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong memtable_memory_budget) { + reinterpret_cast(jhandle)-> + OptimizeUniversalStyleCompaction(memtable_memory_budget); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setComparatorHandle + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setComparatorHandle__JI( + JNIEnv* env, jobject jobj, jlong jhandle, jint builtinComparator) { + switch (builtinComparator) { + case 1: + reinterpret_cast(jhandle)->comparator = + rocksdb::ReverseBytewiseComparator(); + break; + default: + reinterpret_cast(jhandle)->comparator = + rocksdb::BytewiseComparator(); + break; + } +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setComparatorHandle + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setComparatorHandle__JJ( + JNIEnv* env, jobject jobj, jlong jopt_handle, jlong jcomparator_handle) { + reinterpret_cast(jopt_handle)->comparator = + reinterpret_cast(jcomparator_handle); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMergeOperatorName + * Signature: (JJjava/lang/String)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMergeOperatorName( + JNIEnv* env, jobject jobj, jlong jhandle, jstring jop_name) { + auto* options = reinterpret_cast(jhandle); + const char* op_name = env->GetStringUTFChars(jop_name, nullptr); + if(op_name == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + options->merge_operator = + rocksdb::MergeOperators::CreateFromStringId(op_name); + env->ReleaseStringUTFChars(jop_name, op_name); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMergeOperator + * Signature: (JJjava/lang/String)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMergeOperator( + JNIEnv* env, jobject jobj, jlong jhandle, jlong mergeOperatorHandle) { + reinterpret_cast(jhandle)->merge_operator = + *(reinterpret_cast*> + (mergeOperatorHandle)); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setCompactionFilterHandle + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setCompactionFilterHandle( + JNIEnv* env, jobject jobj, jlong jopt_handle, + jlong jcompactionfilter_handle) { + reinterpret_cast(jopt_handle)-> + compaction_filter = reinterpret_cast + (jcompactionfilter_handle); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setWriteBufferSize + * Signature: (JJ)I + */ +void Java_org_rocksdb_ColumnFamilyOptions_setWriteBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jwrite_buffer_size) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(jwrite_buffer_size); + if (s.ok()) { + reinterpret_cast(jhandle)-> + write_buffer_size = jwrite_buffer_size; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: writeBufferSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_writeBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + write_buffer_size; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMaxWriteBufferNumber + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMaxWriteBufferNumber( + JNIEnv* env, jobject jobj, jlong jhandle, jint jmax_write_buffer_number) { + reinterpret_cast(jhandle)-> + max_write_buffer_number = jmax_write_buffer_number; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: maxWriteBufferNumber + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_maxWriteBufferNumber( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + max_write_buffer_number; +} + +/* + * Method: setMemTableFactory + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMemTableFactory( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jfactory_handle) { + reinterpret_cast(jhandle)-> + memtable_factory.reset( + reinterpret_cast(jfactory_handle)); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: memTableFactoryName + * Signature: (J)Ljava/lang/String + */ +jstring Java_org_rocksdb_ColumnFamilyOptions_memTableFactoryName( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + rocksdb::MemTableRepFactory* tf = opt->memtable_factory.get(); + + // Should never be nullptr. + // Default memtable factory is SkipListFactory + assert(tf); + + // temporarly fix for the historical typo + if (strcmp(tf->Name(), "HashLinkListRepFactory") == 0) { + return env->NewStringUTF("HashLinkedListRepFactory"); + } + + return env->NewStringUTF(tf->Name()); +} + +/* + * Method: useFixedLengthPrefixExtractor + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_useFixedLengthPrefixExtractor( + JNIEnv* env, jobject jobj, jlong jhandle, jint jprefix_length) { + reinterpret_cast(jhandle)-> + prefix_extractor.reset(rocksdb::NewFixedPrefixTransform( + static_cast(jprefix_length))); +} + +/* + * Method: useCappedPrefixExtractor + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_useCappedPrefixExtractor( + JNIEnv* env, jobject jobj, jlong jhandle, jint jprefix_length) { + reinterpret_cast(jhandle)-> + prefix_extractor.reset(rocksdb::NewCappedPrefixTransform( + static_cast(jprefix_length))); +} + +/* + * Method: setTableFactory + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setTableFactory( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jfactory_handle) { + reinterpret_cast(jhandle)-> + table_factory.reset(reinterpret_cast( + jfactory_handle)); +} + +/* + * Method: tableFactoryName + * Signature: (J)Ljava/lang/String + */ +jstring Java_org_rocksdb_ColumnFamilyOptions_tableFactoryName( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + rocksdb::TableFactory* tf = opt->table_factory.get(); + + // Should never be nullptr. + // Default memtable factory is SkipListFactory + assert(tf); + + return env->NewStringUTF(tf->Name()); +} + + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: minWriteBufferNumberToMerge + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_minWriteBufferNumberToMerge( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->min_write_buffer_number_to_merge; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMinWriteBufferNumberToMerge + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMinWriteBufferNumberToMerge( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jmin_write_buffer_number_to_merge) { + reinterpret_cast( + jhandle)->min_write_buffer_number_to_merge = + static_cast(jmin_write_buffer_number_to_merge); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: maxWriteBufferNumberToMaintain + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_maxWriteBufferNumberToMaintain( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle) + ->max_write_buffer_number_to_maintain; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMaxWriteBufferNumberToMaintain + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMaxWriteBufferNumberToMaintain( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jmax_write_buffer_number_to_maintain) { + reinterpret_cast(jhandle) + ->max_write_buffer_number_to_maintain = + static_cast(jmax_write_buffer_number_to_maintain); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setCompressionType + * Signature: (JB)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setCompressionType( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jcompression_type_value) { + auto* cf_opts = reinterpret_cast(jhandle); + cf_opts->compression = rocksdb::CompressionTypeJni::toCppCompressionType( + jcompression_type_value); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: compressionType + * Signature: (J)B + */ +jbyte Java_org_rocksdb_ColumnFamilyOptions_compressionType( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* cf_opts = reinterpret_cast(jhandle); + return rocksdb::CompressionTypeJni::toJavaCompressionType( + cf_opts->compression); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setCompressionPerLevel + * Signature: (J[B)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setCompressionPerLevel( + JNIEnv* env, jobject jobj, jlong jhandle, + jbyteArray jcompressionLevels) { + auto* options = reinterpret_cast(jhandle); + auto uptr_compression_levels = + rocksdb_compression_vector_helper(env, jcompressionLevels); + if(!uptr_compression_levels) { + // exception occurred + return; + } + options->compression_per_level = *(uptr_compression_levels.get()); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: compressionPerLevel + * Signature: (J)[B + */ +jbyteArray Java_org_rocksdb_ColumnFamilyOptions_compressionPerLevel( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* cf_options = reinterpret_cast(jhandle); + return rocksdb_compression_list_helper(env, + cf_options->compression_per_level); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setBottommostCompressionType + * Signature: (JB)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setBottommostCompressionType( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jcompression_type_value) { + auto* cf_options = reinterpret_cast(jhandle); + cf_options->bottommost_compression = + rocksdb::CompressionTypeJni::toCppCompressionType( + jcompression_type_value); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: bottommostCompressionType + * Signature: (J)B + */ +jbyte Java_org_rocksdb_ColumnFamilyOptions_bottommostCompressionType( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* cf_options = reinterpret_cast(jhandle); + return rocksdb::CompressionTypeJni::toJavaCompressionType( + cf_options->bottommost_compression); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setCompressionOptions + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setCompressionOptions( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jcompression_options_handle) { + auto* cf_options = reinterpret_cast(jhandle); + auto* compression_options = + reinterpret_cast(jcompression_options_handle); + cf_options->compression_opts = *compression_options; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setCompactionStyle + * Signature: (JB)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setCompactionStyle( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte compaction_style) { + reinterpret_cast(jhandle)->compaction_style = + static_cast(compaction_style); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: compactionStyle + * Signature: (J)B + */ +jbyte Java_org_rocksdb_ColumnFamilyOptions_compactionStyle( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast + (jhandle)->compaction_style; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMaxTableFilesSizeFIFO + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMaxTableFilesSizeFIFO( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jmax_table_files_size) { + reinterpret_cast(jhandle)->compaction_options_fifo.max_table_files_size = + static_cast(jmax_table_files_size); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: maxTableFilesSizeFIFO + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_maxTableFilesSizeFIFO( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->compaction_options_fifo.max_table_files_size; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: numLevels + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_numLevels( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->num_levels; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setNumLevels + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setNumLevels( + JNIEnv* env, jobject jobj, jlong jhandle, jint jnum_levels) { + reinterpret_cast(jhandle)->num_levels = + static_cast(jnum_levels); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: levelZeroFileNumCompactionTrigger + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_levelZeroFileNumCompactionTrigger( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level0_file_num_compaction_trigger; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setLevelZeroFileNumCompactionTrigger + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setLevelZeroFileNumCompactionTrigger( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jlevel0_file_num_compaction_trigger) { + reinterpret_cast( + jhandle)->level0_file_num_compaction_trigger = + static_cast(jlevel0_file_num_compaction_trigger); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: levelZeroSlowdownWritesTrigger + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_levelZeroSlowdownWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level0_slowdown_writes_trigger; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setLevelSlowdownWritesTrigger + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setLevelZeroSlowdownWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jlevel0_slowdown_writes_trigger) { + reinterpret_cast( + jhandle)->level0_slowdown_writes_trigger = + static_cast(jlevel0_slowdown_writes_trigger); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: levelZeroStopWritesTrigger + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_levelZeroStopWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level0_stop_writes_trigger; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setLevelStopWritesTrigger + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setLevelZeroStopWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jlevel0_stop_writes_trigger) { + reinterpret_cast(jhandle)-> + level0_stop_writes_trigger = static_cast( + jlevel0_stop_writes_trigger); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: targetFileSizeBase + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_targetFileSizeBase( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + target_file_size_base; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setTargetFileSizeBase + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setTargetFileSizeBase( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jtarget_file_size_base) { + reinterpret_cast(jhandle)-> + target_file_size_base = static_cast(jtarget_file_size_base); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: targetFileSizeMultiplier + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_targetFileSizeMultiplier( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->target_file_size_multiplier; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setTargetFileSizeMultiplier + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setTargetFileSizeMultiplier( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jtarget_file_size_multiplier) { + reinterpret_cast( + jhandle)->target_file_size_multiplier = + static_cast(jtarget_file_size_multiplier); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: maxBytesForLevelBase + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_maxBytesForLevelBase( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->max_bytes_for_level_base; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMaxBytesForLevelBase + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMaxBytesForLevelBase( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jmax_bytes_for_level_base) { + reinterpret_cast( + jhandle)->max_bytes_for_level_base = + static_cast(jmax_bytes_for_level_base); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: levelCompactionDynamicLevelBytes + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ColumnFamilyOptions_levelCompactionDynamicLevelBytes( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level_compaction_dynamic_level_bytes; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setLevelCompactionDynamicLevelBytes + * Signature: (JZ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setLevelCompactionDynamicLevelBytes( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jenable_dynamic_level_bytes) { + reinterpret_cast( + jhandle)->level_compaction_dynamic_level_bytes = + (jenable_dynamic_level_bytes); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: maxBytesForLevelMultiplier + * Signature: (J)D + */ +jdouble Java_org_rocksdb_ColumnFamilyOptions_maxBytesForLevelMultiplier( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->max_bytes_for_level_multiplier; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMaxBytesForLevelMultiplier + * Signature: (JD)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMaxBytesForLevelMultiplier( + JNIEnv* env, jobject jobj, jlong jhandle, + jdouble jmax_bytes_for_level_multiplier) { + reinterpret_cast(jhandle) + ->max_bytes_for_level_multiplier = + static_cast(jmax_bytes_for_level_multiplier); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: maxCompactionBytes + * Signature: (J)I + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_maxCompactionBytes(JNIEnv* env, + jobject jobj, + jlong jhandle) { + return static_cast( + reinterpret_cast(jhandle) + ->max_compaction_bytes); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMaxCompactionBytes + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMaxCompactionBytes( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jmax_compaction_bytes) { + reinterpret_cast(jhandle) + ->max_compaction_bytes = static_cast(jmax_compaction_bytes); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: arenaBlockSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_arenaBlockSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + arena_block_size; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setArenaBlockSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setArenaBlockSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jarena_block_size) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(jarena_block_size); + if (s.ok()) { + reinterpret_cast(jhandle)-> + arena_block_size = jarena_block_size; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: disableAutoCompactions + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ColumnFamilyOptions_disableAutoCompactions( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->disable_auto_compactions; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setDisableAutoCompactions + * Signature: (JZ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setDisableAutoCompactions( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jdisable_auto_compactions) { + reinterpret_cast( + jhandle)->disable_auto_compactions = + static_cast(jdisable_auto_compactions); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: maxSequentialSkipInIterations + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_maxSequentialSkipInIterations( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->max_sequential_skip_in_iterations; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMaxSequentialSkipInIterations + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMaxSequentialSkipInIterations( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jmax_sequential_skip_in_iterations) { + reinterpret_cast( + jhandle)->max_sequential_skip_in_iterations = + static_cast(jmax_sequential_skip_in_iterations); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: inplaceUpdateSupport + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ColumnFamilyOptions_inplaceUpdateSupport( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->inplace_update_support; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setInplaceUpdateSupport + * Signature: (JZ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setInplaceUpdateSupport( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jinplace_update_support) { + reinterpret_cast( + jhandle)->inplace_update_support = + static_cast(jinplace_update_support); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: inplaceUpdateNumLocks + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_inplaceUpdateNumLocks( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->inplace_update_num_locks; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setInplaceUpdateNumLocks + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setInplaceUpdateNumLocks( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jinplace_update_num_locks) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t( + jinplace_update_num_locks); + if (s.ok()) { + reinterpret_cast(jhandle)-> + inplace_update_num_locks = jinplace_update_num_locks; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: memtablePrefixBloomSizeRatio + * Signature: (J)I + */ +jdouble Java_org_rocksdb_ColumnFamilyOptions_memtablePrefixBloomSizeRatio( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle) + ->memtable_prefix_bloom_size_ratio; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMemtablePrefixBloomSizeRatio + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMemtablePrefixBloomSizeRatio( + JNIEnv* env, jobject jobj, jlong jhandle, + jdouble jmemtable_prefix_bloom_size_ratio) { + reinterpret_cast(jhandle) + ->memtable_prefix_bloom_size_ratio = + static_cast(jmemtable_prefix_bloom_size_ratio); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: bloomLocality + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_bloomLocality( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + bloom_locality; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setBloomLocality + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setBloomLocality( + JNIEnv* env, jobject jobj, jlong jhandle, jint jbloom_locality) { + reinterpret_cast(jhandle)->bloom_locality = + static_cast(jbloom_locality); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: maxSuccessiveMerges + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_maxSuccessiveMerges( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + max_successive_merges; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMaxSuccessiveMerges + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMaxSuccessiveMerges( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jmax_successive_merges) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t( + jmax_successive_merges); + if (s.ok()) { + reinterpret_cast(jhandle)-> + max_successive_merges = jmax_successive_merges; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: optimizeFiltersForHits + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ColumnFamilyOptions_optimizeFiltersForHits( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->optimize_filters_for_hits; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setOptimizeFiltersForHits + * Signature: (JZ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setOptimizeFiltersForHits( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean joptimize_filters_for_hits) { + reinterpret_cast( + jhandle)->optimize_filters_for_hits = + static_cast(joptimize_filters_for_hits); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: memtableHugePageSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_memtableHugePageSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->memtable_huge_page_size; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMemtableHugePageSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMemtableHugePageSize( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jmemtable_huge_page_size) { + + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t( + jmemtable_huge_page_size); + if (s.ok()) { + reinterpret_cast( + jhandle)->memtable_huge_page_size = + jmemtable_huge_page_size; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: softPendingCompactionBytesLimit + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_softPendingCompactionBytesLimit( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->soft_pending_compaction_bytes_limit; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setSoftPendingCompactionBytesLimit + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setSoftPendingCompactionBytesLimit( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jsoft_pending_compaction_bytes_limit) { + reinterpret_cast( + jhandle)->soft_pending_compaction_bytes_limit = + static_cast(jsoft_pending_compaction_bytes_limit); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: softHardCompactionBytesLimit + * Signature: (J)J + */ +jlong Java_org_rocksdb_ColumnFamilyOptions_hardPendingCompactionBytesLimit( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->hard_pending_compaction_bytes_limit; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setHardPendingCompactionBytesLimit + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setHardPendingCompactionBytesLimit( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jhard_pending_compaction_bytes_limit) { + reinterpret_cast( + jhandle)->hard_pending_compaction_bytes_limit = + static_cast(jhard_pending_compaction_bytes_limit); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: level0FileNumCompactionTrigger + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_level0FileNumCompactionTrigger( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level0_file_num_compaction_trigger; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setLevel0FileNumCompactionTrigger + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setLevel0FileNumCompactionTrigger( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jlevel0_file_num_compaction_trigger) { + reinterpret_cast( + jhandle)->level0_file_num_compaction_trigger = + static_cast(jlevel0_file_num_compaction_trigger); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: level0SlowdownWritesTrigger + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_level0SlowdownWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level0_slowdown_writes_trigger; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setLevel0SlowdownWritesTrigger + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setLevel0SlowdownWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jlevel0_slowdown_writes_trigger) { + reinterpret_cast( + jhandle)->level0_slowdown_writes_trigger = + static_cast(jlevel0_slowdown_writes_trigger); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: level0StopWritesTrigger + * Signature: (J)I + */ +jint Java_org_rocksdb_ColumnFamilyOptions_level0StopWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->level0_stop_writes_trigger; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setLevel0StopWritesTrigger + * Signature: (JI)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setLevel0StopWritesTrigger( + JNIEnv* env, jobject jobj, jlong jhandle, + jint jlevel0_stop_writes_trigger) { + reinterpret_cast( + jhandle)->level0_stop_writes_trigger = + static_cast(jlevel0_stop_writes_trigger); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: maxBytesForLevelMultiplierAdditional + * Signature: (J)[I + */ +jintArray Java_org_rocksdb_ColumnFamilyOptions_maxBytesForLevelMultiplierAdditional( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto mbflma = reinterpret_cast( + jhandle)->max_bytes_for_level_multiplier_additional; + + const size_t size = mbflma.size(); + + jint* additionals = new jint[size]; + for (size_t i = 0; i < size; i++) { + additionals[i] = static_cast(mbflma[i]); + } + + jsize jlen = static_cast(size); + jintArray result = env->NewIntArray(jlen); + if(result == nullptr) { + // exception thrown: OutOfMemoryError + delete [] additionals; + return nullptr; + } + env->SetIntArrayRegion(result, 0, jlen, additionals); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(result); + delete [] additionals; + return nullptr; + } + + delete [] additionals; + + return result; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setMaxBytesForLevelMultiplierAdditional + * Signature: (J[I)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setMaxBytesForLevelMultiplierAdditional( + JNIEnv* env, jobject jobj, jlong jhandle, + jintArray jmax_bytes_for_level_multiplier_additional) { + jsize len = env->GetArrayLength(jmax_bytes_for_level_multiplier_additional); + jint *additionals = + env->GetIntArrayElements(jmax_bytes_for_level_multiplier_additional, 0); + if(additionals == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + auto* cf_opt = reinterpret_cast(jhandle); + cf_opt->max_bytes_for_level_multiplier_additional.clear(); + for (jsize i = 0; i < len; i++) { + cf_opt->max_bytes_for_level_multiplier_additional.push_back(static_cast(additionals[i])); + } + + env->ReleaseIntArrayElements(jmax_bytes_for_level_multiplier_additional, + additionals, JNI_ABORT); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: paranoidFileChecks + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ColumnFamilyOptions_paranoidFileChecks( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->paranoid_file_checks; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setParanoidFileChecks + * Signature: (JZ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setParanoidFileChecks( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jparanoid_file_checks) { + reinterpret_cast( + jhandle)->paranoid_file_checks = + static_cast(jparanoid_file_checks); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setCompactionPriority + * Signature: (JB)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setCompactionPriority( + JNIEnv* env, jobject jobj, jlong jhandle, + jbyte jcompaction_priority_value) { + auto* cf_opts = reinterpret_cast(jhandle); + cf_opts->compaction_pri = + rocksdb::CompactionPriorityJni::toCppCompactionPriority(jcompaction_priority_value); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: compactionPriority + * Signature: (J)B + */ +jbyte Java_org_rocksdb_ColumnFamilyOptions_compactionPriority( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* cf_opts = reinterpret_cast(jhandle); + return rocksdb::CompactionPriorityJni::toJavaCompactionPriority( + cf_opts->compaction_pri); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setReportBgIoStats + * Signature: (JZ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setReportBgIoStats( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jreport_bg_io_stats) { + auto* cf_opts = reinterpret_cast(jhandle); + cf_opts->report_bg_io_stats = static_cast(jreport_bg_io_stats); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: reportBgIoStats + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ColumnFamilyOptions_reportBgIoStats( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* cf_opts = reinterpret_cast(jhandle); + return static_cast(cf_opts->report_bg_io_stats); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setCompactionOptionsUniversal + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setCompactionOptionsUniversal( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jcompaction_options_universal_handle) { + auto* cf_opts = reinterpret_cast(jhandle); + auto* opts_uni = + reinterpret_cast( + jcompaction_options_universal_handle); + cf_opts->compaction_options_universal = *opts_uni; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setCompactionOptionsFIFO + * Signature: (JJ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setCompactionOptionsFIFO( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jcompaction_options_fifo_handle) { + auto* cf_opts = reinterpret_cast(jhandle); + auto* opts_fifo = + reinterpret_cast( + jcompaction_options_fifo_handle); + cf_opts->compaction_options_fifo = *opts_fifo; +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: setForceConsistencyChecks + * Signature: (JZ)V + */ +void Java_org_rocksdb_ColumnFamilyOptions_setForceConsistencyChecks( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jforce_consistency_checks) { + auto* cf_opts = reinterpret_cast(jhandle); + cf_opts->force_consistency_checks = static_cast(jforce_consistency_checks); +} + +/* + * Class: org_rocksdb_ColumnFamilyOptions + * Method: forceConsistencyChecks + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ColumnFamilyOptions_forceConsistencyChecks( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* cf_opts = reinterpret_cast(jhandle); + return static_cast(cf_opts->force_consistency_checks); +} + +///////////////////////////////////////////////////////////////////// +// rocksdb::DBOptions + +/* + * Class: org_rocksdb_DBOptions + * Method: newDBOptions + * Signature: ()J + */ +jlong Java_org_rocksdb_DBOptions_newDBOptions(JNIEnv* env, + jclass jcls) { + auto* dbop = new rocksdb::DBOptions(); + return reinterpret_cast(dbop); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: getDBOptionsFromProps + * Signature: (Ljava/util/String;)J + */ +jlong Java_org_rocksdb_DBOptions_getDBOptionsFromProps( + JNIEnv* env, jclass jclazz, jstring jopt_string) { + const char* opt_string = env->GetStringUTFChars(jopt_string, nullptr); + if(opt_string == nullptr) { + // exception thrown: OutOfMemoryError + return 0; + } + + auto* db_options = new rocksdb::DBOptions(); + rocksdb::Status status = rocksdb::GetDBOptionsFromString( + rocksdb::DBOptions(), opt_string, db_options); + + env->ReleaseStringUTFChars(jopt_string, opt_string); + + // Check if DBOptions creation was possible. + jlong ret_value = 0; + if (status.ok()) { + ret_value = reinterpret_cast(db_options); + } else { + // if operation failed the DBOptions need to be deleted + // again to prevent a memory leak. + delete db_options; + } + return ret_value; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_DBOptions_disposeInternal( + JNIEnv* env, jobject jobj, jlong handle) { + auto* dbo = reinterpret_cast(handle); + assert(dbo != nullptr); + delete dbo; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: optimizeForSmallDb + * Signature: (J)V + */ +void Java_org_rocksdb_DBOptions_optimizeForSmallDb( + JNIEnv* env, jobject jobj, jlong jhandle) { + reinterpret_cast(jhandle)->OptimizeForSmallDb(); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setEnv + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setEnv( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jenv_handle) { + reinterpret_cast(jhandle)->env = + reinterpret_cast(jenv_handle); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setIncreaseParallelism + * Signature: (JI)V + */ +void Java_org_rocksdb_DBOptions_setIncreaseParallelism( + JNIEnv * env, jobject jobj, jlong jhandle, jint totalThreads) { + reinterpret_cast + (jhandle)->IncreaseParallelism(static_cast(totalThreads)); +} + + +/* + * Class: org_rocksdb_DBOptions + * Method: setCreateIfMissing + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setCreateIfMissing( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean flag) { + reinterpret_cast(jhandle)-> + create_if_missing = flag; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: createIfMissing + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_createIfMissing( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->create_if_missing; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setCreateMissingColumnFamilies + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setCreateMissingColumnFamilies( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean flag) { + reinterpret_cast + (jhandle)->create_missing_column_families = flag; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: createMissingColumnFamilies + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_createMissingColumnFamilies( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast + (jhandle)->create_missing_column_families; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setErrorIfExists + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setErrorIfExists( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean error_if_exists) { + reinterpret_cast(jhandle)->error_if_exists = + static_cast(error_if_exists); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: errorIfExists + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_errorIfExists( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->error_if_exists; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setParanoidChecks + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setParanoidChecks( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean paranoid_checks) { + reinterpret_cast(jhandle)->paranoid_checks = + static_cast(paranoid_checks); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: paranoidChecks + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_paranoidChecks( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->paranoid_checks; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setRateLimiter + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setRateLimiter( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jrate_limiter_handle) { + std::shared_ptr *pRateLimiter = + reinterpret_cast *>( + jrate_limiter_handle); + reinterpret_cast(jhandle)->rate_limiter = *pRateLimiter; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setLogger + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setLogger( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jlogger_handle) { + std::shared_ptr *pLogger = + reinterpret_cast *>( + jlogger_handle); + reinterpret_cast(jhandle)->info_log = *pLogger; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setInfoLogLevel + * Signature: (JB)V + */ +void Java_org_rocksdb_DBOptions_setInfoLogLevel( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jlog_level) { + reinterpret_cast(jhandle)->info_log_level = + static_cast(jlog_level); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: infoLogLevel + * Signature: (J)B + */ +jbyte Java_org_rocksdb_DBOptions_infoLogLevel( + JNIEnv* env, jobject jobj, jlong jhandle) { + return static_cast( + reinterpret_cast(jhandle)->info_log_level); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setMaxTotalWalSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setMaxTotalWalSize( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jmax_total_wal_size) { + reinterpret_cast(jhandle)->max_total_wal_size = + static_cast(jmax_total_wal_size); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: maxTotalWalSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_maxTotalWalSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + max_total_wal_size; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setMaxOpenFiles + * Signature: (JI)V + */ +void Java_org_rocksdb_DBOptions_setMaxOpenFiles( + JNIEnv* env, jobject jobj, jlong jhandle, jint max_open_files) { + reinterpret_cast(jhandle)->max_open_files = + static_cast(max_open_files); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: maxOpenFiles + * Signature: (J)I + */ +jint Java_org_rocksdb_DBOptions_maxOpenFiles( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->max_open_files; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setMaxFileOpeningThreads + * Signature: (JI)V + */ +void Java_org_rocksdb_DBOptions_setMaxFileOpeningThreads( + JNIEnv* env, jobject jobj, jlong jhandle, jint jmax_file_opening_threads) { + reinterpret_cast(jhandle)->max_file_opening_threads = + static_cast(jmax_file_opening_threads); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: maxFileOpeningThreads + * Signature: (J)I + */ +jint Java_org_rocksdb_DBOptions_maxFileOpeningThreads( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->max_file_opening_threads); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setStatistics + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setStatistics( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jstatistics_handle) { + auto* opt = reinterpret_cast(jhandle); + auto* pSptr = + reinterpret_cast*>( + jstatistics_handle); + opt->statistics = *pSptr; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: statistics + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_statistics( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + std::shared_ptr sptr = opt->statistics; + if (sptr == nullptr) { + return 0; + } else { + std::shared_ptr* pSptr = + new std::shared_ptr(sptr); + return reinterpret_cast(pSptr); + } +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setUseFsync + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setUseFsync( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean use_fsync) { + reinterpret_cast(jhandle)->use_fsync = + static_cast(use_fsync); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: useFsync + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_useFsync( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->use_fsync; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setDbPaths + * Signature: (J[Ljava/lang/String;[J)V + */ +void Java_org_rocksdb_DBOptions_setDbPaths( + JNIEnv* env, jobject jobj, jlong jhandle, jobjectArray jpaths, + jlongArray jtarget_sizes) { + std::vector db_paths; + jlong* ptr_jtarget_size = env->GetLongArrayElements(jtarget_sizes, nullptr); + if(ptr_jtarget_size == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + jboolean has_exception = JNI_FALSE; + const jsize len = env->GetArrayLength(jpaths); + for(jsize i = 0; i < len; i++) { + jobject jpath = reinterpret_cast(env-> + GetObjectArrayElement(jpaths, i)); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->ReleaseLongArrayElements( + jtarget_sizes, ptr_jtarget_size, JNI_ABORT); + return; + } + std::string path = rocksdb::JniUtil::copyString( + env, static_cast(jpath), &has_exception); + env->DeleteLocalRef(jpath); + + if(has_exception == JNI_TRUE) { + env->ReleaseLongArrayElements( + jtarget_sizes, ptr_jtarget_size, JNI_ABORT); + return; + } + + jlong jtarget_size = ptr_jtarget_size[i]; + + db_paths.push_back( + rocksdb::DbPath(path, static_cast(jtarget_size))); + } + + env->ReleaseLongArrayElements(jtarget_sizes, ptr_jtarget_size, JNI_ABORT); + + auto* opt = reinterpret_cast(jhandle); + opt->db_paths = db_paths; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: dbPathsLen + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_dbPathsLen( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->db_paths.size()); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: dbPaths + * Signature: (J[Ljava/lang/String;[J)V + */ +void Java_org_rocksdb_DBOptions_dbPaths( + JNIEnv* env, jobject jobj, jlong jhandle, jobjectArray jpaths, + jlongArray jtarget_sizes) { + jlong* ptr_jtarget_size = env->GetLongArrayElements(jtarget_sizes, nullptr); + if(ptr_jtarget_size == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + auto* opt = reinterpret_cast(jhandle); + const jsize len = env->GetArrayLength(jpaths); + for(jsize i = 0; i < len; i++) { + rocksdb::DbPath db_path = opt->db_paths[i]; + + jstring jpath = env->NewStringUTF(db_path.path.c_str()); + if(jpath == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseLongArrayElements( + jtarget_sizes, ptr_jtarget_size, JNI_ABORT); + return; + } + env->SetObjectArrayElement(jpaths, i, jpath); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jpath); + env->ReleaseLongArrayElements( + jtarget_sizes, ptr_jtarget_size, JNI_ABORT); + return; + } + + ptr_jtarget_size[i] = static_cast(db_path.target_size); + } + + env->ReleaseLongArrayElements(jtarget_sizes, ptr_jtarget_size, JNI_COMMIT); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setDbLogDir + * Signature: (JLjava/lang/String)V + */ +void Java_org_rocksdb_DBOptions_setDbLogDir( + JNIEnv* env, jobject jobj, jlong jhandle, jstring jdb_log_dir) { + const char* log_dir = env->GetStringUTFChars(jdb_log_dir, nullptr); + if(log_dir == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + reinterpret_cast(jhandle)->db_log_dir.assign(log_dir); + env->ReleaseStringUTFChars(jdb_log_dir, log_dir); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: dbLogDir + * Signature: (J)Ljava/lang/String + */ +jstring Java_org_rocksdb_DBOptions_dbLogDir( + JNIEnv* env, jobject jobj, jlong jhandle) { + return env->NewStringUTF( + reinterpret_cast(jhandle)->db_log_dir.c_str()); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setWalDir + * Signature: (JLjava/lang/String)V + */ +void Java_org_rocksdb_DBOptions_setWalDir( + JNIEnv* env, jobject jobj, jlong jhandle, jstring jwal_dir) { + const char* wal_dir = env->GetStringUTFChars(jwal_dir, 0); + reinterpret_cast(jhandle)->wal_dir.assign(wal_dir); + env->ReleaseStringUTFChars(jwal_dir, wal_dir); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: walDir + * Signature: (J)Ljava/lang/String + */ +jstring Java_org_rocksdb_DBOptions_walDir( + JNIEnv* env, jobject jobj, jlong jhandle) { + return env->NewStringUTF( + reinterpret_cast(jhandle)->wal_dir.c_str()); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setDeleteObsoleteFilesPeriodMicros + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setDeleteObsoleteFilesPeriodMicros( + JNIEnv* env, jobject jobj, jlong jhandle, jlong micros) { + reinterpret_cast(jhandle) + ->delete_obsolete_files_period_micros = + static_cast(micros); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: deleteObsoleteFilesPeriodMicros + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_deleteObsoleteFilesPeriodMicros( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle) + ->delete_obsolete_files_period_micros; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setBaseBackgroundCompactions + * Signature: (JI)V + */ +void Java_org_rocksdb_DBOptions_setBaseBackgroundCompactions( + JNIEnv* env, jobject jobj, jlong jhandle, jint max) { + reinterpret_cast(jhandle) + ->base_background_compactions = static_cast(max); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: baseBackgroundCompactions + * Signature: (J)I + */ +jint Java_org_rocksdb_DBOptions_baseBackgroundCompactions( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle) + ->base_background_compactions; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setMaxBackgroundCompactions + * Signature: (JI)V + */ +void Java_org_rocksdb_DBOptions_setMaxBackgroundCompactions( + JNIEnv* env, jobject jobj, jlong jhandle, jint max) { + reinterpret_cast(jhandle) + ->max_background_compactions = static_cast(max); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: maxBackgroundCompactions + * Signature: (J)I + */ +jint Java_org_rocksdb_DBOptions_maxBackgroundCompactions( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast( + jhandle)->max_background_compactions; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setMaxSubcompactions + * Signature: (JI)V + */ +void Java_org_rocksdb_DBOptions_setMaxSubcompactions( + JNIEnv* env, jobject jobj, jlong jhandle, jint max) { + reinterpret_cast(jhandle) + ->max_subcompactions = static_cast(max); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: maxSubcompactions + * Signature: (J)I + */ +jint Java_org_rocksdb_DBOptions_maxSubcompactions( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle) + ->max_subcompactions; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setMaxBackgroundFlushes + * Signature: (JI)V + */ +void Java_org_rocksdb_DBOptions_setMaxBackgroundFlushes( + JNIEnv* env, jobject jobj, jlong jhandle, jint max_background_flushes) { + reinterpret_cast(jhandle)->max_background_flushes = + static_cast(max_background_flushes); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: maxBackgroundFlushes + * Signature: (J)I + */ +jint Java_org_rocksdb_DBOptions_maxBackgroundFlushes( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + max_background_flushes; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setMaxLogFileSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setMaxLogFileSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong max_log_file_size) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(max_log_file_size); + if (s.ok()) { + reinterpret_cast(jhandle)->max_log_file_size = + max_log_file_size; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_DBOptions + * Method: maxLogFileSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_maxLogFileSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->max_log_file_size; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setLogFileTimeToRoll + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setLogFileTimeToRoll( + JNIEnv* env, jobject jobj, jlong jhandle, jlong log_file_time_to_roll) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t( + log_file_time_to_roll); + if (s.ok()) { + reinterpret_cast(jhandle)->log_file_time_to_roll = + log_file_time_to_roll; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_DBOptions + * Method: logFileTimeToRoll + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_logFileTimeToRoll( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->log_file_time_to_roll; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setKeepLogFileNum + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setKeepLogFileNum( + JNIEnv* env, jobject jobj, jlong jhandle, jlong keep_log_file_num) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(keep_log_file_num); + if (s.ok()) { + reinterpret_cast(jhandle)->keep_log_file_num = + keep_log_file_num; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_DBOptions + * Method: keepLogFileNum + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_keepLogFileNum( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->keep_log_file_num; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setRecycleLogFileNum + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setRecycleLogFileNum( + JNIEnv* env, jobject jobj, jlong jhandle, jlong recycle_log_file_num) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(recycle_log_file_num); + if (s.ok()) { + reinterpret_cast(jhandle)->recycle_log_file_num = + recycle_log_file_num; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_DBOptions + * Method: recycleLogFileNum + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_recycleLogFileNum(JNIEnv* env, jobject jobj, + jlong jhandle) { + return reinterpret_cast(jhandle)->recycle_log_file_num; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setMaxManifestFileSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setMaxManifestFileSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong max_manifest_file_size) { + reinterpret_cast(jhandle)->max_manifest_file_size = + static_cast(max_manifest_file_size); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: maxManifestFileSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_maxManifestFileSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + max_manifest_file_size; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setTableCacheNumshardbits + * Signature: (JI)V + */ +void Java_org_rocksdb_DBOptions_setTableCacheNumshardbits( + JNIEnv* env, jobject jobj, jlong jhandle, jint table_cache_numshardbits) { + reinterpret_cast(jhandle)->table_cache_numshardbits = + static_cast(table_cache_numshardbits); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: tableCacheNumshardbits + * Signature: (J)I + */ +jint Java_org_rocksdb_DBOptions_tableCacheNumshardbits( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + table_cache_numshardbits; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setWalTtlSeconds + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setWalTtlSeconds( + JNIEnv* env, jobject jobj, jlong jhandle, jlong WAL_ttl_seconds) { + reinterpret_cast(jhandle)->WAL_ttl_seconds = + static_cast(WAL_ttl_seconds); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: walTtlSeconds + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_walTtlSeconds( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->WAL_ttl_seconds; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setWalSizeLimitMB + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setWalSizeLimitMB( + JNIEnv* env, jobject jobj, jlong jhandle, jlong WAL_size_limit_MB) { + reinterpret_cast(jhandle)->WAL_size_limit_MB = + static_cast(WAL_size_limit_MB); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: walTtlSeconds + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_walSizeLimitMB( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->WAL_size_limit_MB; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setManifestPreallocationSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setManifestPreallocationSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong preallocation_size) { + rocksdb::Status s = rocksdb::check_if_jlong_fits_size_t(preallocation_size); + if (s.ok()) { + reinterpret_cast(jhandle)-> + manifest_preallocation_size = preallocation_size; + } else { + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_DBOptions + * Method: manifestPreallocationSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_manifestPreallocationSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle) + ->manifest_preallocation_size; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: useDirectReads + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_useDirectReads(JNIEnv* env, jobject jobj, + jlong jhandle) { + return reinterpret_cast(jhandle)->use_direct_reads; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setUseDirectReads + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setUseDirectReads(JNIEnv* env, jobject jobj, + jlong jhandle, + jboolean use_direct_reads) { + reinterpret_cast(jhandle)->use_direct_reads = + static_cast(use_direct_reads); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: useDirectIoForFlushAndCompaction + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_useDirectIoForFlushAndCompaction( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle) + ->use_direct_io_for_flush_and_compaction; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setUseDirectReads + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setUseDirectIoForFlushAndCompaction( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean use_direct_io_for_flush_and_compaction) { + reinterpret_cast(jhandle) + ->use_direct_io_for_flush_and_compaction = + static_cast(use_direct_io_for_flush_and_compaction); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setAllowFAllocate + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setAllowFAllocate( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jallow_fallocate) { + reinterpret_cast(jhandle)->allow_fallocate = + static_cast(jallow_fallocate); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: allowFAllocate + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_allowFAllocate( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->allow_fallocate); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setAllowMmapReads + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setAllowMmapReads( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean allow_mmap_reads) { + reinterpret_cast(jhandle)->allow_mmap_reads = + static_cast(allow_mmap_reads); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: allowMmapReads + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_allowMmapReads( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->allow_mmap_reads; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setAllowMmapWrites + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setAllowMmapWrites( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean allow_mmap_writes) { + reinterpret_cast(jhandle)->allow_mmap_writes = + static_cast(allow_mmap_writes); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: allowMmapWrites + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_allowMmapWrites( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->allow_mmap_writes; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setIsFdCloseOnExec + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setIsFdCloseOnExec( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean is_fd_close_on_exec) { + reinterpret_cast(jhandle)->is_fd_close_on_exec = + static_cast(is_fd_close_on_exec); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: isFdCloseOnExec + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_isFdCloseOnExec( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->is_fd_close_on_exec; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setStatsDumpPeriodSec + * Signature: (JI)V + */ +void Java_org_rocksdb_DBOptions_setStatsDumpPeriodSec( + JNIEnv* env, jobject jobj, jlong jhandle, jint stats_dump_period_sec) { + reinterpret_cast(jhandle)->stats_dump_period_sec = + static_cast(stats_dump_period_sec); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: statsDumpPeriodSec + * Signature: (J)I + */ +jint Java_org_rocksdb_DBOptions_statsDumpPeriodSec( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->stats_dump_period_sec; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setAdviseRandomOnOpen + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setAdviseRandomOnOpen( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean advise_random_on_open) { + reinterpret_cast(jhandle)->advise_random_on_open = + static_cast(advise_random_on_open); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: adviseRandomOnOpen + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_adviseRandomOnOpen( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->advise_random_on_open; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setDbWriteBufferSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setDbWriteBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jdb_write_buffer_size) { + auto* opt = reinterpret_cast(jhandle); + opt->db_write_buffer_size = static_cast(jdb_write_buffer_size); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: dbWriteBufferSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_dbWriteBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->db_write_buffer_size); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setAccessHintOnCompactionStart + * Signature: (JB)V + */ +void Java_org_rocksdb_DBOptions_setAccessHintOnCompactionStart( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jaccess_hint_value) { + auto* opt = reinterpret_cast(jhandle); + opt->access_hint_on_compaction_start = + rocksdb::AccessHintJni::toCppAccessHint(jaccess_hint_value); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: accessHintOnCompactionStart + * Signature: (J)B + */ +jbyte Java_org_rocksdb_DBOptions_accessHintOnCompactionStart( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return rocksdb::AccessHintJni::toJavaAccessHint( + opt->access_hint_on_compaction_start); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setNewTableReaderForCompactionInputs + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setNewTableReaderForCompactionInputs( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jnew_table_reader_for_compaction_inputs) { + auto* opt = reinterpret_cast(jhandle); + opt->new_table_reader_for_compaction_inputs = + static_cast(jnew_table_reader_for_compaction_inputs); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: newTableReaderForCompactionInputs + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_newTableReaderForCompactionInputs( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->new_table_reader_for_compaction_inputs); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setCompactionReadaheadSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setCompactionReadaheadSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jcompaction_readahead_size) { + auto* opt = reinterpret_cast(jhandle); + opt->compaction_readahead_size = + static_cast(jcompaction_readahead_size); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: compactionReadaheadSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_compactionReadaheadSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->compaction_readahead_size); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setRandomAccessMaxBufferSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setRandomAccessMaxBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jrandom_access_max_buffer_size) { + auto* opt = reinterpret_cast(jhandle); + opt->random_access_max_buffer_size = + static_cast(jrandom_access_max_buffer_size); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: randomAccessMaxBufferSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_randomAccessMaxBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->random_access_max_buffer_size); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setWritableFileMaxBufferSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setWritableFileMaxBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle, + jlong jwritable_file_max_buffer_size) { + auto* opt = reinterpret_cast(jhandle); + opt->writable_file_max_buffer_size = + static_cast(jwritable_file_max_buffer_size); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: writableFileMaxBufferSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_writableFileMaxBufferSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->writable_file_max_buffer_size); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setUseAdaptiveMutex + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setUseAdaptiveMutex( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean use_adaptive_mutex) { + reinterpret_cast(jhandle)->use_adaptive_mutex = + static_cast(use_adaptive_mutex); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: useAdaptiveMutex + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_useAdaptiveMutex( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->use_adaptive_mutex; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setBytesPerSync + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setBytesPerSync( + JNIEnv* env, jobject jobj, jlong jhandle, jlong bytes_per_sync) { + reinterpret_cast(jhandle)->bytes_per_sync = + static_cast(bytes_per_sync); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: bytesPerSync + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_bytesPerSync( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->bytes_per_sync; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setWalBytesPerSync + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setWalBytesPerSync( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jwal_bytes_per_sync) { + reinterpret_cast(jhandle)->wal_bytes_per_sync = + static_cast(jwal_bytes_per_sync); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: walBytesPerSync + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_walBytesPerSync( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->wal_bytes_per_sync); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setEnableThreadTracking + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setEnableThreadTracking( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jenable_thread_tracking) { + auto* opt = reinterpret_cast(jhandle); + opt->enable_thread_tracking = static_cast(jenable_thread_tracking); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: enableThreadTracking + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_enableThreadTracking( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->enable_thread_tracking); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setDelayedWriteRate + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setDelayedWriteRate( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jdelayed_write_rate) { + auto* opt = reinterpret_cast(jhandle); + opt->delayed_write_rate = static_cast(jdelayed_write_rate); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: delayedWriteRate + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_delayedWriteRate( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->delayed_write_rate); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setAllowConcurrentMemtableWrite + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setAllowConcurrentMemtableWrite( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean allow) { + reinterpret_cast(jhandle)-> + allow_concurrent_memtable_write = static_cast(allow); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: allowConcurrentMemtableWrite + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_allowConcurrentMemtableWrite( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + allow_concurrent_memtable_write; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setEnableWriteThreadAdaptiveYield + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setEnableWriteThreadAdaptiveYield( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean yield) { + reinterpret_cast(jhandle)-> + enable_write_thread_adaptive_yield = static_cast(yield); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: enableWriteThreadAdaptiveYield + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_enableWriteThreadAdaptiveYield( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + enable_write_thread_adaptive_yield; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setWriteThreadMaxYieldUsec + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setWriteThreadMaxYieldUsec( + JNIEnv* env, jobject jobj, jlong jhandle, jlong max) { + reinterpret_cast(jhandle)-> + write_thread_max_yield_usec = static_cast(max); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: writeThreadMaxYieldUsec + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_writeThreadMaxYieldUsec( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + write_thread_max_yield_usec; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setWriteThreadSlowYieldUsec + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setWriteThreadSlowYieldUsec( + JNIEnv* env, jobject jobj, jlong jhandle, jlong slow) { + reinterpret_cast(jhandle)-> + write_thread_slow_yield_usec = static_cast(slow); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: writeThreadSlowYieldUsec + * Signature: (J)J + */ +jlong Java_org_rocksdb_DBOptions_writeThreadSlowYieldUsec( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)-> + write_thread_slow_yield_usec; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setSkipStatsUpdateOnDbOpen + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setSkipStatsUpdateOnDbOpen( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jskip_stats_update_on_db_open) { + auto* opt = reinterpret_cast(jhandle); + opt->skip_stats_update_on_db_open = + static_cast(jskip_stats_update_on_db_open); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: skipStatsUpdateOnDbOpen + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_skipStatsUpdateOnDbOpen( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->skip_stats_update_on_db_open); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setWalRecoveryMode + * Signature: (JB)V + */ +void Java_org_rocksdb_DBOptions_setWalRecoveryMode( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jwal_recovery_mode_value) { + auto* opt = reinterpret_cast(jhandle); + opt->wal_recovery_mode = + rocksdb::WALRecoveryModeJni::toCppWALRecoveryMode( + jwal_recovery_mode_value); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: walRecoveryMode + * Signature: (J)B + */ +jbyte Java_org_rocksdb_DBOptions_walRecoveryMode( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return rocksdb::WALRecoveryModeJni::toJavaWALRecoveryMode( + opt->wal_recovery_mode); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setAllow2pc + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setAllow2pc( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jallow_2pc) { + auto* opt = reinterpret_cast(jhandle); + opt->allow_2pc = static_cast(jallow_2pc); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: allow2pc + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_allow2pc(JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->allow_2pc); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setRowCache + * Signature: (JJ)V + */ +void Java_org_rocksdb_DBOptions_setRowCache( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jrow_cache_handle) { + auto* opt = reinterpret_cast(jhandle); + auto* row_cache = reinterpret_cast*>(jrow_cache_handle); + opt->row_cache = *row_cache; +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setFailIfOptionsFileError + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setFailIfOptionsFileError( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jfail_if_options_file_error) { + auto* opt = reinterpret_cast(jhandle); + opt->fail_if_options_file_error = + static_cast(jfail_if_options_file_error); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: failIfOptionsFileError + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_failIfOptionsFileError( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->fail_if_options_file_error); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setDumpMallocStats + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setDumpMallocStats( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jdump_malloc_stats) { + auto* opt = reinterpret_cast(jhandle); + opt->dump_malloc_stats = static_cast(jdump_malloc_stats); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: dumpMallocStats + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_dumpMallocStats( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->dump_malloc_stats); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setAvoidFlushDuringRecovery + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setAvoidFlushDuringRecovery( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean javoid_flush_during_recovery) { + auto* opt = reinterpret_cast(jhandle); + opt->avoid_flush_during_recovery = static_cast(javoid_flush_during_recovery); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: avoidFlushDuringRecovery + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_avoidFlushDuringRecovery( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->avoid_flush_during_recovery); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: setAvoidFlushDuringShutdown + * Signature: (JZ)V + */ +void Java_org_rocksdb_DBOptions_setAvoidFlushDuringShutdown( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean javoid_flush_during_shutdown) { + auto* opt = reinterpret_cast(jhandle); + opt->avoid_flush_during_shutdown = static_cast(javoid_flush_during_shutdown); +} + +/* + * Class: org_rocksdb_DBOptions + * Method: avoidFlushDuringShutdown + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_DBOptions_avoidFlushDuringShutdown( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->avoid_flush_during_shutdown); +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::WriteOptions + +/* + * Class: org_rocksdb_WriteOptions + * Method: newWriteOptions + * Signature: ()J + */ +jlong Java_org_rocksdb_WriteOptions_newWriteOptions( + JNIEnv* env, jclass jcls) { + auto* op = new rocksdb::WriteOptions(); + return reinterpret_cast(op); +} + /* * Class: org_rocksdb_WriteOptions * Method: disposeInternal @@ -1549,10 +5624,9 @@ void Java_org_rocksdb_WriteOptions_newWriteOptions( */ void Java_org_rocksdb_WriteOptions_disposeInternal( JNIEnv* env, jobject jwrite_options, jlong jhandle) { - auto write_options = reinterpret_cast(jhandle); + auto* write_options = reinterpret_cast(jhandle); + assert(write_options != nullptr); delete write_options; - - rocksdb::WriteOptionsJni::setHandle(env, jwrite_options, nullptr); } /* @@ -1595,18 +5669,63 @@ jboolean Java_org_rocksdb_WriteOptions_disableWAL( return reinterpret_cast(jhandle)->disableWAL; } +/* + * Class: org_rocksdb_WriteOptions + * Method: setIgnoreMissingColumnFamilies + * Signature: (JZ)V + */ +void Java_org_rocksdb_WriteOptions_setIgnoreMissingColumnFamilies( + JNIEnv* env, jobject jwrite_options, jlong jhandle, + jboolean jignore_missing_column_families) { + reinterpret_cast(jhandle)-> + ignore_missing_column_families = + static_cast(jignore_missing_column_families); +} + +/* + * Class: org_rocksdb_WriteOptions + * Method: ignoreMissingColumnFamilies + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_WriteOptions_ignoreMissingColumnFamilies( + JNIEnv* env, jobject jwrite_options, jlong jhandle) { + return reinterpret_cast(jhandle)-> + ignore_missing_column_families; +} + +/* + * Class: org_rocksdb_WriteOptions + * Method: setNoSlowdown + * Signature: (JZ)V + */ +void Java_org_rocksdb_WriteOptions_setNoSlowdown( + JNIEnv* env, jobject jwrite_options, jlong jhandle, jboolean jno_slowdown) { + reinterpret_cast(jhandle)->no_slowdown = + static_cast(jno_slowdown); +} + +/* + * Class: org_rocksdb_WriteOptions + * Method: noSlowdown + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_WriteOptions_noSlowdown( + JNIEnv* env, jobject jwrite_options, jlong jhandle) { + return reinterpret_cast(jhandle)->no_slowdown; +} + ///////////////////////////////////////////////////////////////////// // rocksdb::ReadOptions /* * Class: org_rocksdb_ReadOptions * Method: newReadOptions - * Signature: ()V + * Signature: ()J */ -void Java_org_rocksdb_ReadOptions_newReadOptions( - JNIEnv* env, jobject jobj) { - auto read_opt = new rocksdb::ReadOptions(); - rocksdb::ReadOptionsJni::setHandle(env, jobj, read_opt); +jlong Java_org_rocksdb_ReadOptions_newReadOptions( + JNIEnv* env, jclass jcls) { + auto* read_options = new rocksdb::ReadOptions(); + return reinterpret_cast(read_options); } /* @@ -1616,8 +5735,21 @@ void Java_org_rocksdb_ReadOptions_newReadOptions( */ void Java_org_rocksdb_ReadOptions_disposeInternal( JNIEnv* env, jobject jobj, jlong jhandle) { - delete reinterpret_cast(jhandle); - rocksdb::ReadOptionsJni::setHandle(env, jobj, nullptr); + auto* read_options = reinterpret_cast(jhandle); + assert(read_options != nullptr); + delete read_options; +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: setVerifyChecksums + * Signature: (JZ)V + */ +void Java_org_rocksdb_ReadOptions_setVerifyChecksums( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jverify_checksums) { + reinterpret_cast(jhandle)->verify_checksums = + static_cast(jverify_checksums); } /* @@ -1633,14 +5765,13 @@ jboolean Java_org_rocksdb_ReadOptions_verifyChecksums( /* * Class: org_rocksdb_ReadOptions - * Method: setVerifyChecksums + * Method: setFillCache * Signature: (JZ)V */ -void Java_org_rocksdb_ReadOptions_setVerifyChecksums( - JNIEnv* env, jobject jobj, jlong jhandle, - jboolean jverify_checksums) { - reinterpret_cast(jhandle)->verify_checksums = - static_cast(jverify_checksums); +void Java_org_rocksdb_ReadOptions_setFillCache( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jfill_cache) { + reinterpret_cast(jhandle)->fill_cache = + static_cast(jfill_cache); } /* @@ -1655,13 +5786,13 @@ jboolean Java_org_rocksdb_ReadOptions_fillCache( /* * Class: org_rocksdb_ReadOptions - * Method: setFillCache + * Method: setTailing * Signature: (JZ)V */ -void Java_org_rocksdb_ReadOptions_setFillCache( - JNIEnv* env, jobject jobj, jlong jhandle, jboolean jfill_cache) { - reinterpret_cast(jhandle)->fill_cache = - static_cast(jfill_cache); +void Java_org_rocksdb_ReadOptions_setTailing( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jtailing) { + reinterpret_cast(jhandle)->tailing = + static_cast(jtailing); } /* @@ -1676,11 +5807,295 @@ jboolean Java_org_rocksdb_ReadOptions_tailing( /* * Class: org_rocksdb_ReadOptions - * Method: setTailing + * Method: managed + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ReadOptions_managed( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->managed; +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: setManaged * Signature: (JZ)V */ -void Java_org_rocksdb_ReadOptions_setTailing( - JNIEnv* env, jobject jobj, jlong jhandle, jboolean jtailing) { - reinterpret_cast(jhandle)->tailing = - static_cast(jtailing); +void Java_org_rocksdb_ReadOptions_setManaged( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jmanaged) { + reinterpret_cast(jhandle)->managed = + static_cast(jmanaged); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: totalOrderSeek + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ReadOptions_totalOrderSeek( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->total_order_seek; +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: setTotalOrderSeek + * Signature: (JZ)V + */ +void Java_org_rocksdb_ReadOptions_setTotalOrderSeek( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jtotal_order_seek) { + reinterpret_cast(jhandle)->total_order_seek = + static_cast(jtotal_order_seek); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: prefixSameAsStart + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ReadOptions_prefixSameAsStart( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->prefix_same_as_start; +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: setPrefixSameAsStart + * Signature: (JZ)V + */ +void Java_org_rocksdb_ReadOptions_setPrefixSameAsStart( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jprefix_same_as_start) { + reinterpret_cast(jhandle)->prefix_same_as_start = + static_cast(jprefix_same_as_start); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: pinData + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ReadOptions_pinData( + JNIEnv* env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle)->pin_data; +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: setPinData + * Signature: (JZ)V + */ +void Java_org_rocksdb_ReadOptions_setPinData( + JNIEnv* env, jobject jobj, jlong jhandle, jboolean jpin_data) { + reinterpret_cast(jhandle)->pin_data = + static_cast(jpin_data); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: backgroundPurgeOnIteratorCleanup + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ReadOptions_backgroundPurgeOnIteratorCleanup( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->background_purge_on_iterator_cleanup); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: setBackgroundPurgeOnIteratorCleanup + * Signature: (JZ)V + */ +void Java_org_rocksdb_ReadOptions_setBackgroundPurgeOnIteratorCleanup( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jbackground_purge_on_iterator_cleanup) { + auto* opt = reinterpret_cast(jhandle); + opt->background_purge_on_iterator_cleanup = + static_cast(jbackground_purge_on_iterator_cleanup); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: readaheadSize + * Signature: (J)J + */ +jlong Java_org_rocksdb_ReadOptions_readaheadSize( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->readahead_size); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: setReadaheadSize + * Signature: (JJ)V + */ +void Java_org_rocksdb_ReadOptions_setReadaheadSize( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jreadahead_size) { + auto* opt = reinterpret_cast(jhandle); + opt->readahead_size = static_cast(jreadahead_size); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: ignoreRangeDeletions + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ReadOptions_ignoreRangeDeletions( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* opt = reinterpret_cast(jhandle); + return static_cast(opt->ignore_range_deletions); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: setIgnoreRangeDeletions + * Signature: (JZ)V + */ +void Java_org_rocksdb_ReadOptions_setIgnoreRangeDeletions( + JNIEnv* env, jobject jobj, jlong jhandle, + jboolean jignore_range_deletions) { + auto* opt = reinterpret_cast(jhandle); + opt->ignore_range_deletions = static_cast(jignore_range_deletions); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: setSnapshot + * Signature: (JJ)V + */ +void Java_org_rocksdb_ReadOptions_setSnapshot( + JNIEnv* env, jobject jobj, jlong jhandle, jlong jsnapshot) { + reinterpret_cast(jhandle)->snapshot = + reinterpret_cast(jsnapshot); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: snapshot + * Signature: (J)J + */ +jlong Java_org_rocksdb_ReadOptions_snapshot( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto& snapshot = + reinterpret_cast(jhandle)->snapshot; + return reinterpret_cast(snapshot); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: readTier + * Signature: (J)B + */ +jbyte Java_org_rocksdb_ReadOptions_readTier( + JNIEnv* env, jobject jobj, jlong jhandle) { + return static_cast( + reinterpret_cast(jhandle)->read_tier); +} + +/* + * Class: org_rocksdb_ReadOptions + * Method: setReadTier + * Signature: (JB)V + */ +void Java_org_rocksdb_ReadOptions_setReadTier( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jread_tier) { + reinterpret_cast(jhandle)->read_tier = + static_cast(jread_tier); +} + +///////////////////////////////////////////////////////////////////// +// rocksdb::ComparatorOptions + +/* + * Class: org_rocksdb_ComparatorOptions + * Method: newComparatorOptions + * Signature: ()J + */ +jlong Java_org_rocksdb_ComparatorOptions_newComparatorOptions( + JNIEnv* env, jclass jcls) { + auto* comparator_opt = new rocksdb::ComparatorJniCallbackOptions(); + return reinterpret_cast(comparator_opt); +} + +/* + * Class: org_rocksdb_ComparatorOptions + * Method: useAdaptiveMutex + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_ComparatorOptions_useAdaptiveMutex( + JNIEnv * env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle) + ->use_adaptive_mutex; +} + +/* + * Class: org_rocksdb_ComparatorOptions + * Method: setUseAdaptiveMutex + * Signature: (JZ)V + */ +void Java_org_rocksdb_ComparatorOptions_setUseAdaptiveMutex( + JNIEnv * env, jobject jobj, jlong jhandle, jboolean juse_adaptive_mutex) { + reinterpret_cast(jhandle) + ->use_adaptive_mutex = static_cast(juse_adaptive_mutex); +} + +/* + * Class: org_rocksdb_ComparatorOptions + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_ComparatorOptions_disposeInternal( + JNIEnv * env, jobject jobj, jlong jhandle) { + auto* comparator_opt = + reinterpret_cast(jhandle); + assert(comparator_opt != nullptr); + delete comparator_opt; +} + +///////////////////////////////////////////////////////////////////// +// rocksdb::FlushOptions + +/* + * Class: org_rocksdb_FlushOptions + * Method: newFlushOptions + * Signature: ()J + */ +jlong Java_org_rocksdb_FlushOptions_newFlushOptions( + JNIEnv* env, jclass jcls) { + auto* flush_opt = new rocksdb::FlushOptions(); + return reinterpret_cast(flush_opt); +} + +/* + * Class: org_rocksdb_FlushOptions + * Method: setWaitForFlush + * Signature: (JZ)V + */ +void Java_org_rocksdb_FlushOptions_setWaitForFlush( + JNIEnv * env, jobject jobj, jlong jhandle, jboolean jwait) { + reinterpret_cast(jhandle) + ->wait = static_cast(jwait); +} + +/* + * Class: org_rocksdb_FlushOptions + * Method: waitForFlush + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_FlushOptions_waitForFlush( + JNIEnv * env, jobject jobj, jlong jhandle) { + return reinterpret_cast(jhandle) + ->wait; +} + +/* + * Class: org_rocksdb_FlushOptions + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_FlushOptions_disposeInternal( + JNIEnv * env, jobject jobj, jlong jhandle) { + auto* flush_opt = reinterpret_cast(jhandle); + assert(flush_opt != nullptr); + delete flush_opt; } diff --git a/java/rocksjni/portal.h b/java/rocksjni/portal.h index 28fe754f0f6..ed671ce6e95 100644 --- a/java/rocksjni/portal.h +++ b/java/rocksjni/portal.h @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // This file is designed for caching those frequently used IDs and provide // efficient portal (i.e, a set of static functions) to access java code @@ -11,373 +11,3331 @@ #define JAVA_ROCKSJNI_PORTAL_H_ #include +#include +#include +#include +#include +#include + #include "rocksdb/db.h" #include "rocksdb/filter_policy.h" +#include "rocksdb/status.h" #include "rocksdb/utilities/backupable_db.h" +#include "rocksdb/utilities/write_batch_with_index.h" +#include "rocksjni/comparatorjnicallback.h" +#include "rocksjni/loggerjnicallback.h" +#include "rocksjni/writebatchhandlerjnicallback.h" + +// Remove macro on windows +#ifdef DELETE +#undef DELETE +#endif namespace rocksdb { -// The portal class for org.rocksdb.RocksDB -class RocksDBJni { +// Detect if jlong overflows size_t +inline Status check_if_jlong_fits_size_t(const jlong& jvalue) { + Status s = Status::OK(); + if (static_cast(jvalue) > std::numeric_limits::max()) { + s = Status::InvalidArgument(Slice("jlong overflows 32 bit value.")); + } + return s; +} + +class JavaClass { public: - // Get the java class id of org.rocksdb.RocksDB. - static jclass getJClass(JNIEnv* env) { - jclass jclazz = env->FindClass("org/rocksdb/RocksDB"); + /** + * Gets and initializes a Java Class + * + * @param env A pointer to the Java environment + * @param jclazz_name The fully qualified JNI name of the Java Class + * e.g. "java/lang/String" + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env, const char* jclazz_name) { + jclass jclazz = env->FindClass(jclazz_name); assert(jclazz != nullptr); return jclazz; } +}; - // Get the field id of the member variable of org.rocksdb.RocksDB - // that stores the pointer to rocksdb::DB. - static jfieldID getHandleFieldID(JNIEnv* env) { - static jfieldID fid = env->GetFieldID( - getJClass(env), "nativeHandle_", "J"); - assert(fid != nullptr); - return fid; +// Native class template +template class RocksDBNativeClass : public JavaClass { +}; + +// Native class template for sub-classes of RocksMutableObject +template class NativeRocksMutableObject + : public RocksDBNativeClass { + public: + + /** + * Gets the Java Method ID for the + * RocksMutableObject#setNativeHandle(long, boolean) method + * + * @param env A pointer to the Java environment + * @return The Java Method ID or nullptr the RocksMutableObject class cannot + * be accessed, or if one of the NoSuchMethodError, + * ExceptionInInitializerError or OutOfMemoryError exceptions is thrown + */ + static jmethodID getSetNativeHandleMethod(JNIEnv* env) { + static jclass jclazz = DERIVED::getJClass(env); + if(jclazz == nullptr) { + return nullptr; + } + + static jmethodID mid = env->GetMethodID( + jclazz, "setNativeHandle", "(JZ)V"); + assert(mid != nullptr); + return mid; } - // Get the pointer to rocksdb::DB of the specified org.rocksdb.RocksDB. - static rocksdb::DB* getHandle(JNIEnv* env, jobject jdb) { - return reinterpret_cast( - env->GetLongField(jdb, getHandleFieldID(env))); + /** + * Sets the C++ object pointer handle in the Java object + * + * @param env A pointer to the Java environment + * @param jobj The Java object on which to set the pointer handle + * @param ptr The C++ object pointer + * @param java_owns_handle JNI_TRUE if ownership of the C++ object is + * managed by the Java object + * + * @return true if a Java exception is pending, false otherwise + */ + static bool setHandle(JNIEnv* env, jobject jobj, PTR ptr, + jboolean java_owns_handle) { + assert(jobj != nullptr); + static jmethodID mid = getSetNativeHandleMethod(env); + if(mid == nullptr) { + return true; // signal exception + } + + env->CallVoidMethod(jobj, mid, reinterpret_cast(ptr), java_owns_handle); + if(env->ExceptionCheck()) { + return true; // signal exception + } + + return false; } +}; + +// Java Exception template +template class JavaException : public JavaClass { + public: + /** + * Create and throw a java exception with the provided message + * + * @param env A pointer to the Java environment + * @param msg The message for the exception + * + * @return true if an exception was thrown, false otherwise + */ + static bool ThrowNew(JNIEnv* env, const std::string& msg) { + jclass jclazz = DERIVED::getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + std::cerr << "JavaException::ThrowNew - Error: unexpected exception!" << std::endl; + return env->ExceptionCheck(); + } + + const jint rs = env->ThrowNew(jclazz, msg.c_str()); + if(rs != JNI_OK) { + // exception could not be thrown + std::cerr << "JavaException::ThrowNew - Fatal: could not throw exception!" << std::endl; + return env->ExceptionCheck(); + } - // Pass the rocksdb::DB pointer to the java side. - static void setHandle(JNIEnv* env, jobject jdb, rocksdb::DB* db) { - env->SetLongField( - jdb, getHandleFieldID(env), - reinterpret_cast(db)); + return true; } }; -// The portal class for org.rocksdb.RocksDBException -class RocksDBExceptionJni { +// The portal class for org.rocksdb.RocksDB +class RocksDBJni : public RocksDBNativeClass { public: - // Get the jclass of org.rocksdb.RocksDBException + /** + * Get the Java Class org.rocksdb.RocksDB + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ static jclass getJClass(JNIEnv* env) { - jclass jclazz = env->FindClass("org/rocksdb/RocksDBException"); - assert(jclazz != nullptr); - return jclazz; + return RocksDBNativeClass::getJClass(env, "org/rocksdb/RocksDB"); } +}; - // Create and throw a java exception by converting the input - // Status to an RocksDBException. - // - // In case s.ok() is true, then this function will not throw any - // exception. - static void ThrowNew(JNIEnv* env, Status s) { - if (s.ok()) { - return; +// The portal class for org.rocksdb.Status +class StatusJni : public RocksDBNativeClass { + public: + /** + * Get the Java Class org.rocksdb.Status + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/Status"); + } + + /** + * Create a new Java org.rocksdb.Status object with the same properties as + * the provided C++ rocksdb::Status object + * + * @param env A pointer to the Java environment + * @param status The rocksdb::Status object + * + * @return A reference to a Java org.rocksdb.Status object, or nullptr + * if an an exception occurs + */ + static jobject construct(JNIEnv* env, const Status& status) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + jmethodID mid = + env->GetMethodID(jclazz, "", "(BBLjava/lang/String;)V"); + if(mid == nullptr) { + // exception thrown: NoSuchMethodException or OutOfMemoryError + return nullptr; + } + + // convert the Status state for Java + jstring jstate = nullptr; + if (status.getState() != nullptr) { + const char* const state = status.getState(); + jstate = env->NewStringUTF(state); + if(env->ExceptionCheck()) { + if(jstate != nullptr) { + env->DeleteLocalRef(jstate); + } + return nullptr; + } + } + + jobject jstatus = + env->NewObject(jclazz, mid, toJavaStatusCode(status.code()), + toJavaStatusSubCode(status.subcode()), jstate); + if(env->ExceptionCheck()) { + // exception occurred + if(jstate != nullptr) { + env->DeleteLocalRef(jstate); + } + return nullptr; + } + + if(jstate != nullptr) { + env->DeleteLocalRef(jstate); } - jstring msg = env->NewStringUTF(s.ToString().c_str()); - // get the constructor id of org.rocksdb.RocksDBException - static jmethodID mid = env->GetMethodID( - getJClass(env), "", "(Ljava/lang/String;)V"); - assert(mid != nullptr); - env->Throw((jthrowable)env->NewObject(getJClass(env), mid, msg)); + return jstatus; + } + + // Returns the equivalent org.rocksdb.Status.Code for the provided + // C++ rocksdb::Status::Code enum + static jbyte toJavaStatusCode(const rocksdb::Status::Code& code) { + switch (code) { + case rocksdb::Status::Code::kOk: + return 0x0; + case rocksdb::Status::Code::kNotFound: + return 0x1; + case rocksdb::Status::Code::kCorruption: + return 0x2; + case rocksdb::Status::Code::kNotSupported: + return 0x3; + case rocksdb::Status::Code::kInvalidArgument: + return 0x4; + case rocksdb::Status::Code::kIOError: + return 0x5; + case rocksdb::Status::Code::kMergeInProgress: + return 0x6; + case rocksdb::Status::Code::kIncomplete: + return 0x7; + case rocksdb::Status::Code::kShutdownInProgress: + return 0x8; + case rocksdb::Status::Code::kTimedOut: + return 0x9; + case rocksdb::Status::Code::kAborted: + return 0xA; + case rocksdb::Status::Code::kBusy: + return 0xB; + case rocksdb::Status::Code::kExpired: + return 0xC; + case rocksdb::Status::Code::kTryAgain: + return 0xD; + default: + return 0x7F; // undefined + } + } + + // Returns the equivalent org.rocksdb.Status.SubCode for the provided + // C++ rocksdb::Status::SubCode enum + static jbyte toJavaStatusSubCode(const rocksdb::Status::SubCode& subCode) { + switch (subCode) { + case rocksdb::Status::SubCode::kNone: + return 0x0; + case rocksdb::Status::SubCode::kMutexTimeout: + return 0x1; + case rocksdb::Status::SubCode::kLockTimeout: + return 0x2; + case rocksdb::Status::SubCode::kLockLimit: + return 0x3; + case rocksdb::Status::SubCode::kMaxSubCode: + return 0x7E; + default: + return 0x7F; // undefined + } } }; -class OptionsJni { +// The portal class for org.rocksdb.RocksDBException +class RocksDBExceptionJni : + public JavaException { public: - // Get the java class id of org.rocksdb.Options. + /** + * Get the Java Class org.rocksdb.RocksDBException + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ static jclass getJClass(JNIEnv* env) { - jclass jclazz = env->FindClass("org/rocksdb/Options"); - assert(jclazz != nullptr); - return jclazz; + return JavaException::getJClass(env, "org/rocksdb/RocksDBException"); } - // Get the field id of the member variable of org.rocksdb.Options - // that stores the pointer to rocksdb::Options - static jfieldID getHandleFieldID(JNIEnv* env) { - static jfieldID fid = env->GetFieldID( - getJClass(env), "nativeHandle_", "J"); - assert(fid != nullptr); - return fid; + /** + * Create and throw a Java RocksDBException with the provided message + * + * @param env A pointer to the Java environment + * @param msg The message for the exception + * + * @return true if an exception was thrown, false otherwise + */ + static bool ThrowNew(JNIEnv* env, const std::string& msg) { + return JavaException::ThrowNew(env, msg); } - // Get the pointer to rocksdb::Options - static rocksdb::Options* getHandle(JNIEnv* env, jobject jobj) { - return reinterpret_cast( - env->GetLongField(jobj, getHandleFieldID(env))); + /** + * Create and throw a Java RocksDBException with the provided status + * + * If s.ok() == true, then this function will not throw any exception. + * + * @param env A pointer to the Java environment + * @param s The status for the exception + * + * @return true if an exception was thrown, false otherwise + */ + static bool ThrowNew(JNIEnv* env, const Status& s) { + assert(!s.ok()); + if (s.ok()) { + return false; + } + + // get the RocksDBException class + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + std::cerr << "RocksDBExceptionJni::ThrowNew/class - Error: unexpected exception!" << std::endl; + return env->ExceptionCheck(); + } + + // get the constructor of org.rocksdb.RocksDBException + jmethodID mid = + env->GetMethodID(jclazz, "", "(Lorg/rocksdb/Status;)V"); + if(mid == nullptr) { + // exception thrown: NoSuchMethodException or OutOfMemoryError + std::cerr << "RocksDBExceptionJni::ThrowNew/cstr - Error: unexpected exception!" << std::endl; + return env->ExceptionCheck(); + } + + // get the Java status object + jobject jstatus = StatusJni::construct(env, s); + if(jstatus == nullptr) { + // exception occcurred + std::cerr << "RocksDBExceptionJni::ThrowNew/StatusJni - Error: unexpected exception!" << std::endl; + return env->ExceptionCheck(); + } + + // construct the RocksDBException + jthrowable rocksdb_exception = reinterpret_cast(env->NewObject(jclazz, mid, jstatus)); + if(env->ExceptionCheck()) { + if(jstatus != nullptr) { + env->DeleteLocalRef(jstatus); + } + if(rocksdb_exception != nullptr) { + env->DeleteLocalRef(rocksdb_exception); + } + std::cerr << "RocksDBExceptionJni::ThrowNew/NewObject - Error: unexpected exception!" << std::endl; + return true; + } + + // throw the RocksDBException + const jint rs = env->Throw(rocksdb_exception); + if(rs != JNI_OK) { + // exception could not be thrown + std::cerr << "RocksDBExceptionJni::ThrowNew - Fatal: could not throw exception!" << std::endl; + if(jstatus != nullptr) { + env->DeleteLocalRef(jstatus); + } + if(rocksdb_exception != nullptr) { + env->DeleteLocalRef(rocksdb_exception); + } + return env->ExceptionCheck(); + } + + if(jstatus != nullptr) { + env->DeleteLocalRef(jstatus); + } + if(rocksdb_exception != nullptr) { + env->DeleteLocalRef(rocksdb_exception); + } + + return true; } - // Pass the rocksdb::Options pointer to the java side. - static void setHandle(JNIEnv* env, jobject jobj, rocksdb::Options* op) { - env->SetLongField( - jobj, getHandleFieldID(env), - reinterpret_cast(op)); + /** + * Create and throw a Java RocksDBException with the provided message + * and status + * + * If s.ok() == true, then this function will not throw any exception. + * + * @param env A pointer to the Java environment + * @param msg The message for the exception + * @param s The status for the exception + * + * @return true if an exception was thrown, false otherwise + */ + static bool ThrowNew(JNIEnv* env, const std::string& msg, const Status& s) { + assert(!s.ok()); + if (s.ok()) { + return false; + } + + // get the RocksDBException class + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + std::cerr << "RocksDBExceptionJni::ThrowNew/class - Error: unexpected exception!" << std::endl; + return env->ExceptionCheck(); + } + + // get the constructor of org.rocksdb.RocksDBException + jmethodID mid = + env->GetMethodID(jclazz, "", "(Ljava/lang/String;Lorg/rocksdb/Status;)V"); + if(mid == nullptr) { + // exception thrown: NoSuchMethodException or OutOfMemoryError + std::cerr << "RocksDBExceptionJni::ThrowNew/cstr - Error: unexpected exception!" << std::endl; + return env->ExceptionCheck(); + } + + jstring jmsg = env->NewStringUTF(msg.c_str()); + if(jmsg == nullptr) { + // exception thrown: OutOfMemoryError + std::cerr << "RocksDBExceptionJni::ThrowNew/msg - Error: unexpected exception!" << std::endl; + return env->ExceptionCheck(); + } + + // get the Java status object + jobject jstatus = StatusJni::construct(env, s); + if(jstatus == nullptr) { + // exception occcurred + std::cerr << "RocksDBExceptionJni::ThrowNew/StatusJni - Error: unexpected exception!" << std::endl; + if(jmsg != nullptr) { + env->DeleteLocalRef(jmsg); + } + return env->ExceptionCheck(); + } + + // construct the RocksDBException + jthrowable rocksdb_exception = reinterpret_cast(env->NewObject(jclazz, mid, jmsg, jstatus)); + if(env->ExceptionCheck()) { + if(jstatus != nullptr) { + env->DeleteLocalRef(jstatus); + } + if(jmsg != nullptr) { + env->DeleteLocalRef(jmsg); + } + if(rocksdb_exception != nullptr) { + env->DeleteLocalRef(rocksdb_exception); + } + std::cerr << "RocksDBExceptionJni::ThrowNew/NewObject - Error: unexpected exception!" << std::endl; + return true; + } + + // throw the RocksDBException + const jint rs = env->Throw(rocksdb_exception); + if(rs != JNI_OK) { + // exception could not be thrown + std::cerr << "RocksDBExceptionJni::ThrowNew - Fatal: could not throw exception!" << std::endl; + if(jstatus != nullptr) { + env->DeleteLocalRef(jstatus); + } + if(jmsg != nullptr) { + env->DeleteLocalRef(jmsg); + } + if(rocksdb_exception != nullptr) { + env->DeleteLocalRef(rocksdb_exception); + } + return env->ExceptionCheck(); + } + + if(jstatus != nullptr) { + env->DeleteLocalRef(jstatus); + } + if(jmsg != nullptr) { + env->DeleteLocalRef(jmsg); + } + if(rocksdb_exception != nullptr) { + env->DeleteLocalRef(rocksdb_exception); + } + + return true; } }; -class WriteOptionsJni { +// The portal class for java.lang.IllegalArgumentException +class IllegalArgumentExceptionJni : + public JavaException { public: - // Get the java class id of org.rocksdb.WriteOptions. + /** + * Get the Java Class java.lang.IllegalArgumentException + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ static jclass getJClass(JNIEnv* env) { - jclass jclazz = env->FindClass("org/rocksdb/WriteOptions"); - assert(jclazz != nullptr); - return jclazz; + return JavaException::getJClass(env, "java/lang/IllegalArgumentException"); } - // Get the field id of the member variable of org.rocksdb.WriteOptions - // that stores the pointer to rocksdb::WriteOptions - static jfieldID getHandleFieldID(JNIEnv* env) { - static jfieldID fid = env->GetFieldID( - getJClass(env), "nativeHandle_", "J"); - assert(fid != nullptr); - return fid; - } + /** + * Create and throw a Java IllegalArgumentException with the provided status + * + * If s.ok() == true, then this function will not throw any exception. + * + * @param env A pointer to the Java environment + * @param s The status for the exception + * + * @return true if an exception was thrown, false otherwise + */ + static bool ThrowNew(JNIEnv* env, const Status& s) { + assert(!s.ok()); + if (s.ok()) { + return false; + } - // Get the pointer to rocksdb::WriteOptions - static rocksdb::WriteOptions* getHandle(JNIEnv* env, jobject jobj) { - return reinterpret_cast( - env->GetLongField(jobj, getHandleFieldID(env))); - } + // get the IllegalArgumentException class + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + std::cerr << "IllegalArgumentExceptionJni::ThrowNew/class - Error: unexpected exception!" << std::endl; + return env->ExceptionCheck(); + } - // Pass the rocksdb::WriteOptions pointer to the java side. - static void setHandle(JNIEnv* env, jobject jobj, rocksdb::WriteOptions* op) { - env->SetLongField( - jobj, getHandleFieldID(env), - reinterpret_cast(op)); + return JavaException::ThrowNew(env, s.ToString()); } }; -class ReadOptionsJni { +// The portal class for org.rocksdb.Options +class OptionsJni : public RocksDBNativeClass< + rocksdb::Options*, OptionsJni> { public: - // Get the java class id of org.rocksdb.ReadOptions. + /** + * Get the Java Class org.rocksdb.Options + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ static jclass getJClass(JNIEnv* env) { - jclass jclazz = env->FindClass("org/rocksdb/ReadOptions"); - assert(jclazz != nullptr); - return jclazz; + return RocksDBNativeClass::getJClass(env, "org/rocksdb/Options"); } +}; - // Get the field id of the member variable of org.rocksdb.ReadOptions - // that stores the pointer to rocksdb::ReadOptions - static jfieldID getHandleFieldID(JNIEnv* env) { - static jfieldID fid = env->GetFieldID( - getJClass(env), "nativeHandle_", "J"); - assert(fid != nullptr); - return fid; +// The portal class for org.rocksdb.DBOptions +class DBOptionsJni : public RocksDBNativeClass< + rocksdb::DBOptions*, DBOptionsJni> { + public: + /** + * Get the Java Class org.rocksdb.DBOptions + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/DBOptions"); + } +}; + +class ColumnFamilyDescriptorJni : public JavaClass { + public: + /** + * Get the Java Class org.rocksdb.ColumnFamilyDescriptor + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return JavaClass::getJClass(env, "org/rocksdb/ColumnFamilyDescriptor"); } - // Get the pointer to rocksdb::ReadOptions - static rocksdb::ReadOptions* getHandle(JNIEnv* env, jobject jobj) { - return reinterpret_cast( - env->GetLongField(jobj, getHandleFieldID(env))); + /** + * Get the Java Method: ColumnFamilyDescriptor#columnFamilyName + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getColumnFamilyNameMethod(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jclazz, "columnFamilyName", "()[B"); + assert(mid != nullptr); + return mid; } - // Pass the rocksdb::ReadOptions pointer to the java side. - static void setHandle(JNIEnv* env, jobject jobj, - rocksdb::ReadOptions* op) { - env->SetLongField( - jobj, getHandleFieldID(env), - reinterpret_cast(op)); + /** + * Get the Java Method: ColumnFamilyDescriptor#columnFamilyOptions + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getColumnFamilyOptionsMethod(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jclazz, "columnFamilyOptions", + "()Lorg/rocksdb/ColumnFamilyOptions;"); + assert(mid != nullptr); + return mid; } }; - -class WriteBatchJni { +// The portal class for org.rocksdb.ColumnFamilyOptions +class ColumnFamilyOptionsJni : public RocksDBNativeClass< + rocksdb::ColumnFamilyOptions*, ColumnFamilyOptionsJni> { public: + /** + * Get the Java Class org.rocksdb.ColumnFamilyOptions + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ static jclass getJClass(JNIEnv* env) { - jclass jclazz = env->FindClass("org/rocksdb/WriteBatch"); - assert(jclazz != nullptr); - return jclazz; + return RocksDBNativeClass::getJClass(env, + "org/rocksdb/ColumnFamilyOptions"); } +}; - static jfieldID getHandleFieldID(JNIEnv* env) { - static jfieldID fid = env->GetFieldID( - getJClass(env), "nativeHandle_", "J"); - assert(fid != nullptr); - return fid; +// The portal class for org.rocksdb.WriteOptions +class WriteOptionsJni : public RocksDBNativeClass< + rocksdb::WriteOptions*, WriteOptionsJni> { + public: + /** + * Get the Java Class org.rocksdb.WriteOptions + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/WriteOptions"); } +}; - // Get the pointer to rocksdb::WriteBatch of the specified - // org.rocksdb.WriteBatch. - static rocksdb::WriteBatch* getHandle(JNIEnv* env, jobject jwb) { - return reinterpret_cast( - env->GetLongField(jwb, getHandleFieldID(env))); +// The portal class for org.rocksdb.ReadOptions +class ReadOptionsJni : public RocksDBNativeClass< + rocksdb::ReadOptions*, ReadOptionsJni> { + public: + /** + * Get the Java Class org.rocksdb.ReadOptions + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/ReadOptions"); } +}; - // Pass the rocksdb::WriteBatch pointer to the java side. - static void setHandle(JNIEnv* env, jobject jwb, rocksdb::WriteBatch* wb) { - env->SetLongField( - jwb, getHandleFieldID(env), - reinterpret_cast(wb)); +// The portal class for org.rocksdb.WriteBatch +class WriteBatchJni : public RocksDBNativeClass< + rocksdb::WriteBatch*, WriteBatchJni> { + public: + /** + * Get the Java Class org.rocksdb.WriteBatch + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/WriteBatch"); } }; -class HistogramDataJni { +// The portal class for org.rocksdb.WriteBatch.Handler +class WriteBatchHandlerJni : public RocksDBNativeClass< + const rocksdb::WriteBatchHandlerJniCallback*, + WriteBatchHandlerJni> { public: - static jmethodID getConstructorMethodId(JNIEnv* env, jclass jclazz) { - static jmethodID mid = env->GetMethodID(jclazz, "", "(DDDDD)V"); + /** + * Get the Java Class org.rocksdb.WriteBatch.Handler + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, + "org/rocksdb/WriteBatch$Handler"); + } + + /** + * Get the Java Method: WriteBatch.Handler#put + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getPutMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jclazz, "put", "([B[B)V"); assert(mid != nullptr); return mid; } -}; -class BackupableDBOptionsJni { - public: - // Get the java class id of org.rocksdb.BackupableDBOptions. - static jclass getJClass(JNIEnv* env) { - jclass jclazz = env->FindClass("org/rocksdb/BackupableDBOptions"); - assert(jclazz != nullptr); - return jclazz; + /** + * Get the Java Method: WriteBatch.Handler#merge + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getMergeMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jclazz, "merge", "([B[B)V"); + assert(mid != nullptr); + return mid; } - // Get the field id of the member variable of org.rocksdb.BackupableDBOptions - // that stores the pointer to rocksdb::BackupableDBOptions - static jfieldID getHandleFieldID(JNIEnv* env) { - static jfieldID fid = env->GetFieldID( - getJClass(env), "nativeHandle_", "J"); - assert(fid != nullptr); - return fid; + /** + * Get the Java Method: WriteBatch.Handler#delete + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getDeleteMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jclazz, "delete", "([B)V"); + assert(mid != nullptr); + return mid; + } + + /** + * Get the Java Method: WriteBatch.Handler#deleteRange + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getDeleteRangeMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if (jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jclazz, "deleteRange", "([B[B)V"); + assert(mid != nullptr); + return mid; } - // Get the pointer to rocksdb::BackupableDBOptions - static rocksdb::BackupableDBOptions* getHandle(JNIEnv* env, jobject jobj) { - return reinterpret_cast( - env->GetLongField(jobj, getHandleFieldID(env))); + /** + * Get the Java Method: WriteBatch.Handler#logData + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getLogDataMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jclazz, "logData", "([B)V"); + assert(mid != nullptr); + return mid; } - // Pass the rocksdb::BackupableDBOptions pointer to the java side. - static void setHandle( - JNIEnv* env, jobject jobj, rocksdb::BackupableDBOptions* op) { - env->SetLongField( - jobj, getHandleFieldID(env), - reinterpret_cast(op)); + /** + * Get the Java Method: WriteBatch.Handler#shouldContinue + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getContinueMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jclazz, "shouldContinue", "()Z"); + assert(mid != nullptr); + return mid; } }; -class IteratorJni { +// The portal class for org.rocksdb.WriteBatchWithIndex +class WriteBatchWithIndexJni : public RocksDBNativeClass< + rocksdb::WriteBatchWithIndex*, WriteBatchWithIndexJni> { public: - // Get the java class id of org.rocksdb.Iteartor. + /** + * Get the Java Class org.rocksdb.WriteBatchWithIndex + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ static jclass getJClass(JNIEnv* env) { - jclass jclazz = env->FindClass("org/rocksdb/RocksIterator"); - assert(jclazz != nullptr); - return jclazz; + return RocksDBNativeClass::getJClass(env, + "org/rocksdb/WriteBatchWithIndex"); } +}; - // Get the field id of the member variable of org.rocksdb.Iterator - // that stores the pointer to rocksdb::Iterator. - static jfieldID getHandleFieldID(JNIEnv* env) { - static jfieldID fid = env->GetFieldID( - getJClass(env), "nativeHandle_", "J"); - assert(fid != nullptr); - return fid; +// The portal class for org.rocksdb.HistogramData +class HistogramDataJni : public JavaClass { + public: + /** + * Get the Java Class org.rocksdb.HistogramData + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return JavaClass::getJClass(env, "org/rocksdb/HistogramData"); } - // Get the pointer to rocksdb::Iterator. - static rocksdb::Iterator* getHandle(JNIEnv* env, jobject jobj) { - return reinterpret_cast( - env->GetLongField(jobj, getHandleFieldID(env))); + /** + * Get the Java Method: HistogramData constructor + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getConstructorMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jclazz, "", "(DDDDD)V"); + assert(mid != nullptr); + return mid; } +}; - // Pass the rocksdb::Iterator pointer to the java side. - static void setHandle( - JNIEnv* env, jobject jobj, rocksdb::Iterator* op) { - env->SetLongField( - jobj, getHandleFieldID(env), - reinterpret_cast(op)); +// The portal class for org.rocksdb.BackupableDBOptions +class BackupableDBOptionsJni : public RocksDBNativeClass< + rocksdb::BackupableDBOptions*, BackupableDBOptionsJni> { + public: + /** + * Get the Java Class org.rocksdb.BackupableDBOptions + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, + "org/rocksdb/BackupableDBOptions"); } }; -class FilterJni { +// The portal class for org.rocksdb.BackupEngine +class BackupEngineJni : public RocksDBNativeClass< + rocksdb::BackupEngine*, BackupEngineJni> { public: - // Get the java class id of org.rocksdb.FilterPolicy. + /** + * Get the Java Class org.rocksdb.BackupableEngine + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ static jclass getJClass(JNIEnv* env) { - jclass jclazz = env->FindClass("org/rocksdb/Filter"); - assert(jclazz != nullptr); - return jclazz; + return RocksDBNativeClass::getJClass(env, "org/rocksdb/BackupEngine"); } +}; - // Get the field id of the member variable of org.rocksdb.Filter - // that stores the pointer to rocksdb::FilterPolicy. - static jfieldID getHandleFieldID(JNIEnv* env) { - static jfieldID fid = env->GetFieldID( - getJClass(env), "nativeHandle_", "J"); - assert(fid != nullptr); - return fid; +// The portal class for org.rocksdb.RocksIterator +class IteratorJni : public RocksDBNativeClass< + rocksdb::Iterator*, IteratorJni> { + public: + /** + * Get the Java Class org.rocksdb.RocksIterator + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/RocksIterator"); } +}; - // Get the pointer to rocksdb::FilterPolicy. - static rocksdb::FilterPolicy* getHandle(JNIEnv* env, jobject jobj) { - return reinterpret_cast( - env->GetLongField(jobj, getHandleFieldID(env))); +// The portal class for org.rocksdb.Filter +class FilterJni : public RocksDBNativeClass< + std::shared_ptr*, FilterJni> { + public: + /** + * Get the Java Class org.rocksdb.Filter + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/Filter"); } +}; - // Pass the rocksdb::FilterPolicy pointer to the java side. - static void setHandle( - JNIEnv* env, jobject jobj, const rocksdb::FilterPolicy* op) { - env->SetLongField( - jobj, getHandleFieldID(env), - reinterpret_cast(op)); +// The portal class for org.rocksdb.ColumnFamilyHandle +class ColumnFamilyHandleJni : public RocksDBNativeClass< + rocksdb::ColumnFamilyHandle*, ColumnFamilyHandleJni> { + public: + /** + * Get the Java Class org.rocksdb.ColumnFamilyHandle + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, + "org/rocksdb/ColumnFamilyHandle"); } }; -class ListJni { +// The portal class for org.rocksdb.FlushOptions +class FlushOptionsJni : public RocksDBNativeClass< + rocksdb::FlushOptions*, FlushOptionsJni> { public: - // Get the java class id of java.util.List. - static jclass getListClass(JNIEnv* env) { - jclass jclazz = env->FindClass("java/util/List"); - assert(jclazz != nullptr); - return jclazz; + /** + * Get the Java Class org.rocksdb.FlushOptions + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/FlushOptions"); } +}; - // Get the java class id of java.util.ArrayList. - static jclass getArrayListClass(JNIEnv* env) { - jclass jclazz = env->FindClass("java/util/ArrayList"); - assert(jclazz != nullptr); - return jclazz; +// The portal class for org.rocksdb.ComparatorOptions +class ComparatorOptionsJni : public RocksDBNativeClass< + rocksdb::ComparatorJniCallbackOptions*, ComparatorOptionsJni> { + public: + /** + * Get the Java Class org.rocksdb.ComparatorOptions + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/ComparatorOptions"); } +}; - // Get the java class id of java.util.Iterator. - static jclass getIteratorClass(JNIEnv* env) { - jclass jclazz = env->FindClass("java/util/Iterator"); - assert(jclazz != nullptr); - return jclazz; +// The portal class for org.rocksdb.AbstractComparator +class AbstractComparatorJni : public RocksDBNativeClass< + const rocksdb::BaseComparatorJniCallback*, + AbstractComparatorJni> { + public: + /** + * Get the Java Class org.rocksdb.AbstractComparator + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, + "org/rocksdb/AbstractComparator"); } - // Get the java method id of java.util.List.iterator(). - static jmethodID getIteratorMethod(JNIEnv* env) { - static jmethodID mid = env->GetMethodID( - getListClass(env), "iterator", "()Ljava/util/Iterator;"); + /** + * Get the Java Method: Comparator#name + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getNameMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jclazz, "name", "()Ljava/lang/String;"); assert(mid != nullptr); return mid; } - // Get the java method id of java.util.Iterator.hasNext(). - static jmethodID getHasNextMethod(JNIEnv* env) { - static jmethodID mid = env->GetMethodID( - getIteratorClass(env), "hasNext", "()Z"); + /** + * Get the Java Method: Comparator#compare + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getCompareMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jclazz, "compare", + "(Lorg/rocksdb/AbstractSlice;Lorg/rocksdb/AbstractSlice;)I"); assert(mid != nullptr); return mid; } - // Get the java method id of java.util.Iterator.next(). - static jmethodID getNextMethod(JNIEnv* env) { - static jmethodID mid = env->GetMethodID( - getIteratorClass(env), "next", "()Ljava/lang/Object;"); + /** + * Get the Java Method: Comparator#findShortestSeparator + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getFindShortestSeparatorMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jclazz, "findShortestSeparator", + "(Ljava/lang/String;Lorg/rocksdb/AbstractSlice;)Ljava/lang/String;"); assert(mid != nullptr); return mid; } - // Get the java method id of arrayList constructor. - static jmethodID getArrayListConstructorMethodId(JNIEnv* env, jclass jclazz) { - static jmethodID mid = env->GetMethodID( - jclazz, "", "(I)V"); + /** + * Get the Java Method: Comparator#findShortSuccessor + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getFindShortSuccessorMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jclazz, "findShortSuccessor", + "(Ljava/lang/String;)Ljava/lang/String;"); assert(mid != nullptr); return mid; } +}; - // Get the java method id of java.util.List.add(). - static jmethodID getListAddMethodId(JNIEnv* env) { - static jmethodID mid = env->GetMethodID( - getListClass(env), "add", "(Ljava/lang/Object;)Z"); - assert(mid != nullptr); - return mid; +// The portal class for org.rocksdb.AbstractSlice +class AbstractSliceJni : public NativeRocksMutableObject< + const rocksdb::Slice*, AbstractSliceJni> { + public: + /** + * Get the Java Class org.rocksdb.AbstractSlice + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/AbstractSlice"); + } +}; + +// The portal class for org.rocksdb.Slice +class SliceJni : public NativeRocksMutableObject< + const rocksdb::Slice*, AbstractSliceJni> { + public: + /** + * Get the Java Class org.rocksdb.Slice + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/Slice"); + } + + /** + * Constructs a Slice object + * + * @param env A pointer to the Java environment + * + * @return A reference to a Java Slice object, or a nullptr if an + * exception occurs + */ + static jobject construct0(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jclazz, "", "()V"); + if(mid == nullptr) { + // exception occurred accessing method + return nullptr; + } + + jobject jslice = env->NewObject(jclazz, mid); + if(env->ExceptionCheck()) { + return nullptr; + } + + return jslice; } }; + +// The portal class for org.rocksdb.DirectSlice +class DirectSliceJni : public NativeRocksMutableObject< + const rocksdb::Slice*, AbstractSliceJni> { + public: + /** + * Get the Java Class org.rocksdb.DirectSlice + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/DirectSlice"); + } + + /** + * Constructs a DirectSlice object + * + * @param env A pointer to the Java environment + * + * @return A reference to a Java DirectSlice object, or a nullptr if an + * exception occurs + */ + static jobject construct0(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jclazz, "", "()V"); + if(mid == nullptr) { + // exception occurred accessing method + return nullptr; + } + + jobject jdirect_slice = env->NewObject(jclazz, mid); + if(env->ExceptionCheck()) { + return nullptr; + } + + return jdirect_slice; + } +}; + +// The portal class for java.util.List +class ListJni : public JavaClass { + public: + /** + * Get the Java Class java.util.List + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getListClass(JNIEnv* env) { + return JavaClass::getJClass(env, "java/util/List"); + } + + /** + * Get the Java Class java.util.ArrayList + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getArrayListClass(JNIEnv* env) { + return JavaClass::getJClass(env, "java/util/ArrayList"); + } + + /** + * Get the Java Class java.util.Iterator + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getIteratorClass(JNIEnv* env) { + return JavaClass::getJClass(env, "java/util/Iterator"); + } + + /** + * Get the Java Method: List#iterator + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getIteratorMethod(JNIEnv* env) { + jclass jlist_clazz = getListClass(env); + if(jlist_clazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jlist_clazz, "iterator", "()Ljava/util/Iterator;"); + assert(mid != nullptr); + return mid; + } + + /** + * Get the Java Method: Iterator#hasNext + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getHasNextMethod(JNIEnv* env) { + jclass jiterator_clazz = getIteratorClass(env); + if(jiterator_clazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jiterator_clazz, "hasNext", "()Z"); + assert(mid != nullptr); + return mid; + } + + /** + * Get the Java Method: Iterator#next + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getNextMethod(JNIEnv* env) { + jclass jiterator_clazz = getIteratorClass(env); + if(jiterator_clazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jiterator_clazz, "next", "()Ljava/lang/Object;"); + assert(mid != nullptr); + return mid; + } + + /** + * Get the Java Method: ArrayList constructor + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getArrayListConstructorMethodId(JNIEnv* env) { + jclass jarray_list_clazz = getArrayListClass(env); + if(jarray_list_clazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + static jmethodID mid = + env->GetMethodID(jarray_list_clazz, "", "(I)V"); + assert(mid != nullptr); + return mid; + } + + /** + * Get the Java Method: List#add + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getListAddMethodId(JNIEnv* env) { + jclass jlist_clazz = getListClass(env); + if(jlist_clazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jlist_clazz, "add", "(Ljava/lang/Object;)Z"); + assert(mid != nullptr); + return mid; + } +}; + +// The portal class for java.lang.Byte +class ByteJni : public JavaClass { + public: + /** + * Get the Java Class java.lang.Byte + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return JavaClass::getJClass(env, "java/lang/Byte"); + } + + /** + * Get the Java Class byte[] + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getArrayJClass(JNIEnv* env) { + return JavaClass::getJClass(env, "[B"); + } + + /** + * Creates a new 2-dimensional Java Byte Array byte[][] + * + * @param env A pointer to the Java environment + * @param len The size of the first dimension + * + * @return A reference to the Java byte[][] or nullptr if an exception occurs + */ + static jobjectArray new2dByteArray(JNIEnv* env, const jsize len) { + jclass clazz = getArrayJClass(env); + if(clazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + return env->NewObjectArray(len, clazz, nullptr); + } + + /** + * Get the Java Method: Byte#byteValue + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getByteValueMethod(JNIEnv* env) { + jclass clazz = getJClass(env); + if(clazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(clazz, "byteValue", "()B"); + assert(mid != nullptr); + return mid; + } +}; + +// The portal class for java.lang.StringBuilder +class StringBuilderJni : public JavaClass { + public: + /** + * Get the Java Class java.lang.StringBuilder + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return JavaClass::getJClass(env, "java/lang/StringBuilder"); + } + + /** + * Get the Java Method: StringBuilder#append + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getListAddMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jclazz, "append", + "(Ljava/lang/String;)Ljava/lang/StringBuilder;"); + assert(mid != nullptr); + return mid; + } + + /** + * Appends a C-style string to a StringBuilder + * + * @param env A pointer to the Java environment + * @param jstring_builder Reference to a java.lang.StringBuilder + * @param c_str A C-style string to append to the StringBuilder + * + * @return A reference to the updated StringBuilder, or a nullptr if + * an exception occurs + */ + static jobject append(JNIEnv* env, jobject jstring_builder, + const char* c_str) { + jmethodID mid = getListAddMethodId(env); + if(mid == nullptr) { + // exception occurred accessing class or method + return nullptr; + } + + jstring new_value_str = env->NewStringUTF(c_str); + if(new_value_str == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + jobject jresult_string_builder = + env->CallObjectMethod(jstring_builder, mid, new_value_str); + if(env->ExceptionCheck()) { + // exception occurred + env->DeleteLocalRef(new_value_str); + return nullptr; + } + + return jresult_string_builder; + } +}; + +// The portal class for org.rocksdb.BackupInfo +class BackupInfoJni : public JavaClass { + public: + /** + * Get the Java Class org.rocksdb.BackupInfo + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return JavaClass::getJClass(env, "org/rocksdb/BackupInfo"); + } + + /** + * Constructs a BackupInfo object + * + * @param env A pointer to the Java environment + * @param backup_id id of the backup + * @param timestamp timestamp of the backup + * @param size size of the backup + * @param number_files number of files related to the backup + * + * @return A reference to a Java BackupInfo object, or a nullptr if an + * exception occurs + */ + static jobject construct0(JNIEnv* env, uint32_t backup_id, int64_t timestamp, + uint64_t size, uint32_t number_files) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = env->GetMethodID(jclazz, "", "(IJJI)V"); + if(mid == nullptr) { + // exception occurred accessing method + return nullptr; + } + + jobject jbackup_info = + env->NewObject(jclazz, mid, backup_id, timestamp, size, number_files); + if(env->ExceptionCheck()) { + return nullptr; + } + + return jbackup_info; + } +}; + +class BackupInfoListJni { + public: + /** + * Converts a C++ std::vector object to + * a Java ArrayList object + * + * @param env A pointer to the Java environment + * @param backup_infos A vector of BackupInfo + * + * @return Either a reference to a Java ArrayList object, or a nullptr + * if an exception occurs + */ + static jobject getBackupInfo(JNIEnv* env, + std::vector backup_infos) { + jclass jarray_list_clazz = rocksdb::ListJni::getArrayListClass(env); + if(jarray_list_clazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + jmethodID cstr_mid = rocksdb::ListJni::getArrayListConstructorMethodId(env); + if(cstr_mid == nullptr) { + // exception occurred accessing method + return nullptr; + } + + jmethodID add_mid = rocksdb::ListJni::getListAddMethodId(env); + if(add_mid == nullptr) { + // exception occurred accessing method + return nullptr; + } + + // create java list + jobject jbackup_info_handle_list = + env->NewObject(jarray_list_clazz, cstr_mid, backup_infos.size()); + if(env->ExceptionCheck()) { + // exception occurred constructing object + return nullptr; + } + + // insert in java list + auto end = backup_infos.end(); + for (auto it = backup_infos.begin(); it != end; ++it) { + auto backup_info = *it; + + jobject obj = rocksdb::BackupInfoJni::construct0(env, + backup_info.backup_id, + backup_info.timestamp, + backup_info.size, + backup_info.number_files); + if(env->ExceptionCheck()) { + // exception occurred constructing object + if(obj != nullptr) { + env->DeleteLocalRef(obj); + } + if(jbackup_info_handle_list != nullptr) { + env->DeleteLocalRef(jbackup_info_handle_list); + } + return nullptr; + } + + jboolean rs = + env->CallBooleanMethod(jbackup_info_handle_list, add_mid, obj); + if(env->ExceptionCheck() || rs == JNI_FALSE) { + // exception occurred calling method, or could not add + if(obj != nullptr) { + env->DeleteLocalRef(obj); + } + if(jbackup_info_handle_list != nullptr) { + env->DeleteLocalRef(jbackup_info_handle_list); + } + return nullptr; + } + } + + return jbackup_info_handle_list; + } +}; + +// The portal class for org.rocksdb.WBWIRocksIterator +class WBWIRocksIteratorJni : public JavaClass { + public: + /** + * Get the Java Class org.rocksdb.WBWIRocksIterator + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return JavaClass::getJClass(env, "org/rocksdb/WBWIRocksIterator"); + } + + /** + * Get the Java Field: WBWIRocksIterator#entry + * + * @param env A pointer to the Java environment + * + * @return The Java Field ID or nullptr if the class or field id could not + * be retieved + */ + static jfieldID getWriteEntryField(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jfieldID fid = + env->GetFieldID(jclazz, "entry", + "Lorg/rocksdb/WBWIRocksIterator$WriteEntry;"); + assert(fid != nullptr); + return fid; + } + + /** + * Gets the value of the WBWIRocksIterator#entry + * + * @param env A pointer to the Java environment + * @param jwbwi_rocks_iterator A reference to a WBWIIterator + * + * @return A reference to a Java WBWIRocksIterator.WriteEntry object, or + * a nullptr if an exception occurs + */ + static jobject getWriteEntry(JNIEnv* env, jobject jwbwi_rocks_iterator) { + assert(jwbwi_rocks_iterator != nullptr); + + jfieldID jwrite_entry_field = getWriteEntryField(env); + if(jwrite_entry_field == nullptr) { + // exception occurred accessing the field + return nullptr; + } + + jobject jwe = env->GetObjectField(jwbwi_rocks_iterator, jwrite_entry_field); + assert(jwe != nullptr); + return jwe; + } +}; + +// The portal class for org.rocksdb.WBWIRocksIterator.WriteType +class WriteTypeJni : public JavaClass { + public: + /** + * Get the PUT enum field value of WBWIRocksIterator.WriteType + * + * @param env A pointer to the Java environment + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject PUT(JNIEnv* env) { + return getEnum(env, "PUT"); + } + + /** + * Get the MERGE enum field value of WBWIRocksIterator.WriteType + * + * @param env A pointer to the Java environment + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject MERGE(JNIEnv* env) { + return getEnum(env, "MERGE"); + } + + /** + * Get the DELETE enum field value of WBWIRocksIterator.WriteType + * + * @param env A pointer to the Java environment + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject DELETE(JNIEnv* env) { + return getEnum(env, "DELETE"); + } + + /** + * Get the LOG enum field value of WBWIRocksIterator.WriteType + * + * @param env A pointer to the Java environment + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject LOG(JNIEnv* env) { + return getEnum(env, "LOG"); + } + + private: + /** + * Get the Java Class org.rocksdb.WBWIRocksIterator.WriteType + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return JavaClass::getJClass(env, "org/rocksdb/WBWIRocksIterator$WriteType"); + } + + /** + * Get an enum field of org.rocksdb.WBWIRocksIterator.WriteType + * + * @param env A pointer to the Java environment + * @param name The name of the enum field + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject getEnum(JNIEnv* env, const char name[]) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + jfieldID jfid = + env->GetStaticFieldID(jclazz, name, + "Lorg/rocksdb/WBWIRocksIterator$WriteType;"); + if(env->ExceptionCheck()) { + // exception occurred while getting field + return nullptr; + } else if(jfid == nullptr) { + return nullptr; + } + + jobject jwrite_type = env->GetStaticObjectField(jclazz, jfid); + assert(jwrite_type != nullptr); + return jwrite_type; + } +}; + +// The portal class for org.rocksdb.WBWIRocksIterator.WriteEntry +class WriteEntryJni : public JavaClass { + public: + /** + * Get the Java Class org.rocksdb.WBWIRocksIterator.WriteEntry + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return JavaClass::getJClass(env, "org/rocksdb/WBWIRocksIterator$WriteEntry"); + } +}; + +// The portal class for org.rocksdb.InfoLogLevel +class InfoLogLevelJni : public JavaClass { + public: + /** + * Get the DEBUG_LEVEL enum field value of InfoLogLevel + * + * @param env A pointer to the Java environment + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject DEBUG_LEVEL(JNIEnv* env) { + return getEnum(env, "DEBUG_LEVEL"); + } + + /** + * Get the INFO_LEVEL enum field value of InfoLogLevel + * + * @param env A pointer to the Java environment + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject INFO_LEVEL(JNIEnv* env) { + return getEnum(env, "INFO_LEVEL"); + } + + /** + * Get the WARN_LEVEL enum field value of InfoLogLevel + * + * @param env A pointer to the Java environment + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject WARN_LEVEL(JNIEnv* env) { + return getEnum(env, "WARN_LEVEL"); + } + + /** + * Get the ERROR_LEVEL enum field value of InfoLogLevel + * + * @param env A pointer to the Java environment + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject ERROR_LEVEL(JNIEnv* env) { + return getEnum(env, "ERROR_LEVEL"); + } + + /** + * Get the FATAL_LEVEL enum field value of InfoLogLevel + * + * @param env A pointer to the Java environment + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject FATAL_LEVEL(JNIEnv* env) { + return getEnum(env, "FATAL_LEVEL"); + } + + /** + * Get the HEADER_LEVEL enum field value of InfoLogLevel + * + * @param env A pointer to the Java environment + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject HEADER_LEVEL(JNIEnv* env) { + return getEnum(env, "HEADER_LEVEL"); + } + + private: + /** + * Get the Java Class org.rocksdb.InfoLogLevel + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return JavaClass::getJClass(env, "org/rocksdb/InfoLogLevel"); + } + + /** + * Get an enum field of org.rocksdb.InfoLogLevel + * + * @param env A pointer to the Java environment + * @param name The name of the enum field + * + * @return A reference to the enum field value or a nullptr if + * the enum field value could not be retrieved + */ + static jobject getEnum(JNIEnv* env, const char name[]) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + jfieldID jfid = + env->GetStaticFieldID(jclazz, name, "Lorg/rocksdb/InfoLogLevel;"); + if(env->ExceptionCheck()) { + // exception occurred while getting field + return nullptr; + } else if(jfid == nullptr) { + return nullptr; + } + + jobject jinfo_log_level = env->GetStaticObjectField(jclazz, jfid); + assert(jinfo_log_level != nullptr); + return jinfo_log_level; + } +}; + +// The portal class for org.rocksdb.Logger +class LoggerJni : public RocksDBNativeClass< + std::shared_ptr*, LoggerJni> { + public: + /** + * Get the Java Class org/rocksdb/Logger + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return RocksDBNativeClass::getJClass(env, "org/rocksdb/Logger"); + } + + /** + * Get the Java Method: Logger#log + * + * @param env A pointer to the Java environment + * + * @return The Java Method ID or nullptr if the class or method id could not + * be retieved + */ + static jmethodID getLogMethodId(JNIEnv* env) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + static jmethodID mid = + env->GetMethodID(jclazz, "log", + "(Lorg/rocksdb/InfoLogLevel;Ljava/lang/String;)V"); + assert(mid != nullptr); + return mid; + } +}; + +// The portal class for org.rocksdb.TransactionLogIterator.BatchResult +class BatchResultJni : public JavaClass { + public: + /** + * Get the Java Class org.rocksdb.TransactionLogIterator.BatchResult + * + * @param env A pointer to the Java environment + * + * @return The Java Class or nullptr if one of the + * ClassFormatError, ClassCircularityError, NoClassDefFoundError, + * OutOfMemoryError or ExceptionInInitializerError exceptions is thrown + */ + static jclass getJClass(JNIEnv* env) { + return JavaClass::getJClass(env, + "org/rocksdb/TransactionLogIterator$BatchResult"); + } + + /** + * Create a new Java org.rocksdb.TransactionLogIterator.BatchResult object + * with the same properties as the provided C++ rocksdb::BatchResult object + * + * @param env A pointer to the Java environment + * @param batch_result The rocksdb::BatchResult object + * + * @return A reference to a Java + * org.rocksdb.TransactionLogIterator.BatchResult object, + * or nullptr if an an exception occurs + */ + static jobject construct(JNIEnv* env, + rocksdb::BatchResult& batch_result) { + jclass jclazz = getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } + + jmethodID mid = env->GetMethodID( + jclazz, "", "(JJ)V"); + if(mid == nullptr) { + // exception thrown: NoSuchMethodException or OutOfMemoryError + return nullptr; + } + + jobject jbatch_result = env->NewObject(jclazz, mid, + batch_result.sequence, batch_result.writeBatchPtr.get()); + if(jbatch_result == nullptr) { + // exception thrown: InstantiationException or OutOfMemoryError + return nullptr; + } + + batch_result.writeBatchPtr.release(); + return jbatch_result; + } +}; + +// The portal class for org.rocksdb.CompactionStopStyle +class CompactionStopStyleJni { + public: + // Returns the equivalent org.rocksdb.CompactionStopStyle for the provided + // C++ rocksdb::CompactionStopStyle enum + static jbyte toJavaCompactionStopStyle( + const rocksdb::CompactionStopStyle& compaction_stop_style) { + switch(compaction_stop_style) { + case rocksdb::CompactionStopStyle::kCompactionStopStyleSimilarSize: + return 0x0; + case rocksdb::CompactionStopStyle::kCompactionStopStyleTotalSize: + return 0x1; + default: + return 0x7F; // undefined + } + } + + // Returns the equivalent C++ rocksdb::CompactionStopStyle enum for the + // provided Java org.rocksdb.CompactionStopStyle + static rocksdb::CompactionStopStyle toCppCompactionStopStyle( + jbyte jcompaction_stop_style) { + switch(jcompaction_stop_style) { + case 0x0: + return rocksdb::CompactionStopStyle::kCompactionStopStyleSimilarSize; + case 0x1: + return rocksdb::CompactionStopStyle::kCompactionStopStyleTotalSize; + default: + // undefined/default + return rocksdb::CompactionStopStyle::kCompactionStopStyleSimilarSize; + } + } +}; + +// The portal class for org.rocksdb.CompressionType +class CompressionTypeJni { + public: + // Returns the equivalent org.rocksdb.CompressionType for the provided + // C++ rocksdb::CompressionType enum + static jbyte toJavaCompressionType( + const rocksdb::CompressionType& compression_type) { + switch(compression_type) { + case rocksdb::CompressionType::kNoCompression: + return 0x0; + case rocksdb::CompressionType::kSnappyCompression: + return 0x1; + case rocksdb::CompressionType::kZlibCompression: + return 0x2; + case rocksdb::CompressionType::kBZip2Compression: + return 0x3; + case rocksdb::CompressionType::kLZ4Compression: + return 0x4; + case rocksdb::CompressionType::kLZ4HCCompression: + return 0x5; + case rocksdb::CompressionType::kXpressCompression: + return 0x6; + case rocksdb::CompressionType::kZSTD: + return 0x7; + case rocksdb::CompressionType::kDisableCompressionOption: + default: + return 0x7F; + } + } + + // Returns the equivalent C++ rocksdb::CompressionType enum for the + // provided Java org.rocksdb.CompressionType + static rocksdb::CompressionType toCppCompressionType( + jbyte jcompression_type) { + switch(jcompression_type) { + case 0x0: + return rocksdb::CompressionType::kNoCompression; + case 0x1: + return rocksdb::CompressionType::kSnappyCompression; + case 0x2: + return rocksdb::CompressionType::kZlibCompression; + case 0x3: + return rocksdb::CompressionType::kBZip2Compression; + case 0x4: + return rocksdb::CompressionType::kLZ4Compression; + case 0x5: + return rocksdb::CompressionType::kLZ4HCCompression; + case 0x6: + return rocksdb::CompressionType::kXpressCompression; + case 0x7: + return rocksdb::CompressionType::kZSTD; + case 0x7F: + default: + return rocksdb::CompressionType::kDisableCompressionOption; + } + } +}; + +// The portal class for org.rocksdb.CompactionPriority +class CompactionPriorityJni { + public: + // Returns the equivalent org.rocksdb.CompactionPriority for the provided + // C++ rocksdb::CompactionPri enum + static jbyte toJavaCompactionPriority( + const rocksdb::CompactionPri& compaction_priority) { + switch(compaction_priority) { + case rocksdb::CompactionPri::kByCompensatedSize: + return 0x0; + case rocksdb::CompactionPri::kOldestLargestSeqFirst: + return 0x1; + case rocksdb::CompactionPri::kOldestSmallestSeqFirst: + return 0x2; + case rocksdb::CompactionPri::kMinOverlappingRatio: + return 0x3; + default: + return 0x0; // undefined + } + } + + // Returns the equivalent C++ rocksdb::CompactionPri enum for the + // provided Java org.rocksdb.CompactionPriority + static rocksdb::CompactionPri toCppCompactionPriority( + jbyte jcompaction_priority) { + switch(jcompaction_priority) { + case 0x0: + return rocksdb::CompactionPri::kByCompensatedSize; + case 0x1: + return rocksdb::CompactionPri::kOldestLargestSeqFirst; + case 0x2: + return rocksdb::CompactionPri::kOldestSmallestSeqFirst; + case 0x3: + return rocksdb::CompactionPri::kMinOverlappingRatio; + default: + // undefined/default + return rocksdb::CompactionPri::kByCompensatedSize; + } + } +}; + +// The portal class for org.rocksdb.AccessHint +class AccessHintJni { + public: + // Returns the equivalent org.rocksdb.AccessHint for the provided + // C++ rocksdb::DBOptions::AccessHint enum + static jbyte toJavaAccessHint( + const rocksdb::DBOptions::AccessHint& access_hint) { + switch(access_hint) { + case rocksdb::DBOptions::AccessHint::NONE: + return 0x0; + case rocksdb::DBOptions::AccessHint::NORMAL: + return 0x1; + case rocksdb::DBOptions::AccessHint::SEQUENTIAL: + return 0x2; + case rocksdb::DBOptions::AccessHint::WILLNEED: + return 0x3; + default: + // undefined/default + return 0x1; + } + } + + // Returns the equivalent C++ rocksdb::DBOptions::AccessHint enum for the + // provided Java org.rocksdb.AccessHint + static rocksdb::DBOptions::AccessHint toCppAccessHint(jbyte jaccess_hint) { + switch(jaccess_hint) { + case 0x0: + return rocksdb::DBOptions::AccessHint::NONE; + case 0x1: + return rocksdb::DBOptions::AccessHint::NORMAL; + case 0x2: + return rocksdb::DBOptions::AccessHint::SEQUENTIAL; + case 0x3: + return rocksdb::DBOptions::AccessHint::WILLNEED; + default: + // undefined/default + return rocksdb::DBOptions::AccessHint::NORMAL; + } + } +}; + +// The portal class for org.rocksdb.WALRecoveryMode +class WALRecoveryModeJni { + public: + // Returns the equivalent org.rocksdb.WALRecoveryMode for the provided + // C++ rocksdb::WALRecoveryMode enum + static jbyte toJavaWALRecoveryMode( + const rocksdb::WALRecoveryMode& wal_recovery_mode) { + switch(wal_recovery_mode) { + case rocksdb::WALRecoveryMode::kTolerateCorruptedTailRecords: + return 0x0; + case rocksdb::WALRecoveryMode::kAbsoluteConsistency: + return 0x1; + case rocksdb::WALRecoveryMode::kPointInTimeRecovery: + return 0x2; + case rocksdb::WALRecoveryMode::kSkipAnyCorruptedRecords: + return 0x3; + default: + // undefined/default + return 0x2; + } + } + + // Returns the equivalent C++ rocksdb::WALRecoveryMode enum for the + // provided Java org.rocksdb.WALRecoveryMode + static rocksdb::WALRecoveryMode toCppWALRecoveryMode(jbyte jwal_recovery_mode) { + switch(jwal_recovery_mode) { + case 0x0: + return rocksdb::WALRecoveryMode::kTolerateCorruptedTailRecords; + case 0x1: + return rocksdb::WALRecoveryMode::kAbsoluteConsistency; + case 0x2: + return rocksdb::WALRecoveryMode::kPointInTimeRecovery; + case 0x3: + return rocksdb::WALRecoveryMode::kSkipAnyCorruptedRecords; + default: + // undefined/default + return rocksdb::WALRecoveryMode::kPointInTimeRecovery; + } + } +}; + +// The portal class for org.rocksdb.TickerType +class TickerTypeJni { + public: + // Returns the equivalent org.rocksdb.TickerType for the provided + // C++ rocksdb::Tickers enum + static jbyte toJavaTickerType( + const rocksdb::Tickers& tickers) { + switch(tickers) { + case rocksdb::Tickers::BLOCK_CACHE_MISS: + return 0x0; + case rocksdb::Tickers::BLOCK_CACHE_HIT: + return 0x1; + case rocksdb::Tickers::BLOCK_CACHE_ADD: + return 0x2; + case rocksdb::Tickers::BLOCK_CACHE_ADD_FAILURES: + return 0x3; + case rocksdb::Tickers::BLOCK_CACHE_INDEX_MISS: + return 0x4; + case rocksdb::Tickers::BLOCK_CACHE_INDEX_HIT: + return 0x5; + case rocksdb::Tickers::BLOCK_CACHE_INDEX_ADD: + return 0x6; + case rocksdb::Tickers::BLOCK_CACHE_INDEX_BYTES_INSERT: + return 0x7; + case rocksdb::Tickers::BLOCK_CACHE_INDEX_BYTES_EVICT: + return 0x8; + case rocksdb::Tickers::BLOCK_CACHE_FILTER_MISS: + return 0x9; + case rocksdb::Tickers::BLOCK_CACHE_FILTER_HIT: + return 0xA; + case rocksdb::Tickers::BLOCK_CACHE_FILTER_ADD: + return 0xB; + case rocksdb::Tickers::BLOCK_CACHE_FILTER_BYTES_INSERT: + return 0xC; + case rocksdb::Tickers::BLOCK_CACHE_FILTER_BYTES_EVICT: + return 0xD; + case rocksdb::Tickers::BLOCK_CACHE_DATA_MISS: + return 0xE; + case rocksdb::Tickers::BLOCK_CACHE_DATA_HIT: + return 0xF; + case rocksdb::Tickers::BLOCK_CACHE_DATA_ADD: + return 0x10; + case rocksdb::Tickers::BLOCK_CACHE_DATA_BYTES_INSERT: + return 0x11; + case rocksdb::Tickers::BLOCK_CACHE_BYTES_READ: + return 0x12; + case rocksdb::Tickers::BLOCK_CACHE_BYTES_WRITE: + return 0x13; + case rocksdb::Tickers::BLOOM_FILTER_USEFUL: + return 0x14; + case rocksdb::Tickers::PERSISTENT_CACHE_HIT: + return 0x15; + case rocksdb::Tickers::PERSISTENT_CACHE_MISS: + return 0x16; + case rocksdb::Tickers::SIM_BLOCK_CACHE_HIT: + return 0x17; + case rocksdb::Tickers::SIM_BLOCK_CACHE_MISS: + return 0x18; + case rocksdb::Tickers::MEMTABLE_HIT: + return 0x19; + case rocksdb::Tickers::MEMTABLE_MISS: + return 0x1A; + case rocksdb::Tickers::GET_HIT_L0: + return 0x1B; + case rocksdb::Tickers::GET_HIT_L1: + return 0x1C; + case rocksdb::Tickers::GET_HIT_L2_AND_UP: + return 0x1D; + case rocksdb::Tickers::COMPACTION_KEY_DROP_NEWER_ENTRY: + return 0x1E; + case rocksdb::Tickers::COMPACTION_KEY_DROP_OBSOLETE: + return 0x1F; + case rocksdb::Tickers::COMPACTION_KEY_DROP_RANGE_DEL: + return 0x20; + case rocksdb::Tickers::COMPACTION_KEY_DROP_USER: + return 0x21; + case rocksdb::Tickers::COMPACTION_RANGE_DEL_DROP_OBSOLETE: + return 0x22; + case rocksdb::Tickers::NUMBER_KEYS_WRITTEN: + return 0x23; + case rocksdb::Tickers::NUMBER_KEYS_READ: + return 0x24; + case rocksdb::Tickers::NUMBER_KEYS_UPDATED: + return 0x25; + case rocksdb::Tickers::BYTES_WRITTEN: + return 0x26; + case rocksdb::Tickers::BYTES_READ: + return 0x27; + case rocksdb::Tickers::NUMBER_DB_SEEK: + return 0x28; + case rocksdb::Tickers::NUMBER_DB_NEXT: + return 0x29; + case rocksdb::Tickers::NUMBER_DB_PREV: + return 0x2A; + case rocksdb::Tickers::NUMBER_DB_SEEK_FOUND: + return 0x2B; + case rocksdb::Tickers::NUMBER_DB_NEXT_FOUND: + return 0x2C; + case rocksdb::Tickers::NUMBER_DB_PREV_FOUND: + return 0x2D; + case rocksdb::Tickers::ITER_BYTES_READ: + return 0x2E; + case rocksdb::Tickers::NO_FILE_CLOSES: + return 0x2F; + case rocksdb::Tickers::NO_FILE_OPENS: + return 0x30; + case rocksdb::Tickers::NO_FILE_ERRORS: + return 0x31; + case rocksdb::Tickers::STALL_L0_SLOWDOWN_MICROS: + return 0x32; + case rocksdb::Tickers::STALL_MEMTABLE_COMPACTION_MICROS: + return 0x33; + case rocksdb::Tickers::STALL_L0_NUM_FILES_MICROS: + return 0x34; + case rocksdb::Tickers::STALL_MICROS: + return 0x35; + case rocksdb::Tickers::DB_MUTEX_WAIT_MICROS: + return 0x36; + case rocksdb::Tickers::RATE_LIMIT_DELAY_MILLIS: + return 0x37; + case rocksdb::Tickers::NO_ITERATORS: + return 0x38; + case rocksdb::Tickers::NUMBER_MULTIGET_CALLS: + return 0x39; + case rocksdb::Tickers::NUMBER_MULTIGET_KEYS_READ: + return 0x3A; + case rocksdb::Tickers::NUMBER_MULTIGET_BYTES_READ: + return 0x3B; + case rocksdb::Tickers::NUMBER_FILTERED_DELETES: + return 0x3C; + case rocksdb::Tickers::NUMBER_MERGE_FAILURES: + return 0x3D; + case rocksdb::Tickers::BLOOM_FILTER_PREFIX_CHECKED: + return 0x3E; + case rocksdb::Tickers::BLOOM_FILTER_PREFIX_USEFUL: + return 0x3F; + case rocksdb::Tickers::NUMBER_OF_RESEEKS_IN_ITERATION: + return 0x40; + case rocksdb::Tickers::GET_UPDATES_SINCE_CALLS: + return 0x41; + case rocksdb::Tickers::BLOCK_CACHE_COMPRESSED_MISS: + return 0x42; + case rocksdb::Tickers::BLOCK_CACHE_COMPRESSED_HIT: + return 0x43; + case rocksdb::Tickers::BLOCK_CACHE_COMPRESSED_ADD: + return 0x44; + case rocksdb::Tickers::BLOCK_CACHE_COMPRESSED_ADD_FAILURES: + return 0x45; + case rocksdb::Tickers::WAL_FILE_SYNCED: + return 0x46; + case rocksdb::Tickers::WAL_FILE_BYTES: + return 0x47; + case rocksdb::Tickers::WRITE_DONE_BY_SELF: + return 0x48; + case rocksdb::Tickers::WRITE_DONE_BY_OTHER: + return 0x49; + case rocksdb::Tickers::WRITE_TIMEDOUT: + return 0x4A; + case rocksdb::Tickers::WRITE_WITH_WAL: + return 0x4B; + case rocksdb::Tickers::COMPACT_READ_BYTES: + return 0x4C; + case rocksdb::Tickers::COMPACT_WRITE_BYTES: + return 0x4D; + case rocksdb::Tickers::FLUSH_WRITE_BYTES: + return 0x4E; + case rocksdb::Tickers::NUMBER_DIRECT_LOAD_TABLE_PROPERTIES: + return 0x4F; + case rocksdb::Tickers::NUMBER_SUPERVERSION_ACQUIRES: + return 0x50; + case rocksdb::Tickers::NUMBER_SUPERVERSION_RELEASES: + return 0x51; + case rocksdb::Tickers::NUMBER_SUPERVERSION_CLEANUPS: + return 0x52; + case rocksdb::Tickers::NUMBER_BLOCK_COMPRESSED: + return 0x53; + case rocksdb::Tickers::NUMBER_BLOCK_DECOMPRESSED: + return 0x54; + case rocksdb::Tickers::NUMBER_BLOCK_NOT_COMPRESSED: + return 0x55; + case rocksdb::Tickers::MERGE_OPERATION_TOTAL_TIME: + return 0x56; + case rocksdb::Tickers::FILTER_OPERATION_TOTAL_TIME: + return 0x57; + case rocksdb::Tickers::ROW_CACHE_HIT: + return 0x58; + case rocksdb::Tickers::ROW_CACHE_MISS: + return 0x59; + case rocksdb::Tickers::READ_AMP_ESTIMATE_USEFUL_BYTES: + return 0x5A; + case rocksdb::Tickers::READ_AMP_TOTAL_READ_BYTES: + return 0x5B; + case rocksdb::Tickers::NUMBER_RATE_LIMITER_DRAINS: + return 0x5C; + case rocksdb::Tickers::TICKER_ENUM_MAX: + return 0x5D; + + default: + // undefined/default + return 0x0; + } + } + + // Returns the equivalent C++ rocksdb::Tickers enum for the + // provided Java org.rocksdb.TickerType + static rocksdb::Tickers toCppTickers(jbyte jticker_type) { + switch(jticker_type) { + case 0x0: + return rocksdb::Tickers::BLOCK_CACHE_MISS; + case 0x1: + return rocksdb::Tickers::BLOCK_CACHE_HIT; + case 0x2: + return rocksdb::Tickers::BLOCK_CACHE_ADD; + case 0x3: + return rocksdb::Tickers::BLOCK_CACHE_ADD_FAILURES; + case 0x4: + return rocksdb::Tickers::BLOCK_CACHE_INDEX_MISS; + case 0x5: + return rocksdb::Tickers::BLOCK_CACHE_INDEX_HIT; + case 0x6: + return rocksdb::Tickers::BLOCK_CACHE_INDEX_ADD; + case 0x7: + return rocksdb::Tickers::BLOCK_CACHE_INDEX_BYTES_INSERT; + case 0x8: + return rocksdb::Tickers::BLOCK_CACHE_INDEX_BYTES_EVICT; + case 0x9: + return rocksdb::Tickers::BLOCK_CACHE_FILTER_MISS; + case 0xA: + return rocksdb::Tickers::BLOCK_CACHE_FILTER_HIT; + case 0xB: + return rocksdb::Tickers::BLOCK_CACHE_FILTER_ADD; + case 0xC: + return rocksdb::Tickers::BLOCK_CACHE_FILTER_BYTES_INSERT; + case 0xD: + return rocksdb::Tickers::BLOCK_CACHE_FILTER_BYTES_EVICT; + case 0xE: + return rocksdb::Tickers::BLOCK_CACHE_DATA_MISS; + case 0xF: + return rocksdb::Tickers::BLOCK_CACHE_DATA_HIT; + case 0x10: + return rocksdb::Tickers::BLOCK_CACHE_DATA_ADD; + case 0x11: + return rocksdb::Tickers::BLOCK_CACHE_DATA_BYTES_INSERT; + case 0x12: + return rocksdb::Tickers::BLOCK_CACHE_BYTES_READ; + case 0x13: + return rocksdb::Tickers::BLOCK_CACHE_BYTES_WRITE; + case 0x14: + return rocksdb::Tickers::BLOOM_FILTER_USEFUL; + case 0x15: + return rocksdb::Tickers::PERSISTENT_CACHE_HIT; + case 0x16: + return rocksdb::Tickers::PERSISTENT_CACHE_MISS; + case 0x17: + return rocksdb::Tickers::SIM_BLOCK_CACHE_HIT; + case 0x18: + return rocksdb::Tickers::SIM_BLOCK_CACHE_MISS; + case 0x19: + return rocksdb::Tickers::MEMTABLE_HIT; + case 0x1A: + return rocksdb::Tickers::MEMTABLE_MISS; + case 0x1B: + return rocksdb::Tickers::GET_HIT_L0; + case 0x1C: + return rocksdb::Tickers::GET_HIT_L1; + case 0x1D: + return rocksdb::Tickers::GET_HIT_L2_AND_UP; + case 0x1E: + return rocksdb::Tickers::COMPACTION_KEY_DROP_NEWER_ENTRY; + case 0x1F: + return rocksdb::Tickers::COMPACTION_KEY_DROP_OBSOLETE; + case 0x20: + return rocksdb::Tickers::COMPACTION_KEY_DROP_RANGE_DEL; + case 0x21: + return rocksdb::Tickers::COMPACTION_KEY_DROP_USER; + case 0x22: + return rocksdb::Tickers::COMPACTION_RANGE_DEL_DROP_OBSOLETE; + case 0x23: + return rocksdb::Tickers::NUMBER_KEYS_WRITTEN; + case 0x24: + return rocksdb::Tickers::NUMBER_KEYS_READ; + case 0x25: + return rocksdb::Tickers::NUMBER_KEYS_UPDATED; + case 0x26: + return rocksdb::Tickers::BYTES_WRITTEN; + case 0x27: + return rocksdb::Tickers::BYTES_READ; + case 0x28: + return rocksdb::Tickers::NUMBER_DB_SEEK; + case 0x29: + return rocksdb::Tickers::NUMBER_DB_NEXT; + case 0x2A: + return rocksdb::Tickers::NUMBER_DB_PREV; + case 0x2B: + return rocksdb::Tickers::NUMBER_DB_SEEK_FOUND; + case 0x2C: + return rocksdb::Tickers::NUMBER_DB_NEXT_FOUND; + case 0x2D: + return rocksdb::Tickers::NUMBER_DB_PREV_FOUND; + case 0x2E: + return rocksdb::Tickers::ITER_BYTES_READ; + case 0x2F: + return rocksdb::Tickers::NO_FILE_CLOSES; + case 0x30: + return rocksdb::Tickers::NO_FILE_OPENS; + case 0x31: + return rocksdb::Tickers::NO_FILE_ERRORS; + case 0x32: + return rocksdb::Tickers::STALL_L0_SLOWDOWN_MICROS; + case 0x33: + return rocksdb::Tickers::STALL_MEMTABLE_COMPACTION_MICROS; + case 0x34: + return rocksdb::Tickers::STALL_L0_NUM_FILES_MICROS; + case 0x35: + return rocksdb::Tickers::STALL_MICROS; + case 0x36: + return rocksdb::Tickers::DB_MUTEX_WAIT_MICROS; + case 0x37: + return rocksdb::Tickers::RATE_LIMIT_DELAY_MILLIS; + case 0x38: + return rocksdb::Tickers::NO_ITERATORS; + case 0x39: + return rocksdb::Tickers::NUMBER_MULTIGET_CALLS; + case 0x3A: + return rocksdb::Tickers::NUMBER_MULTIGET_KEYS_READ; + case 0x3B: + return rocksdb::Tickers::NUMBER_MULTIGET_BYTES_READ; + case 0x3C: + return rocksdb::Tickers::NUMBER_FILTERED_DELETES; + case 0x3D: + return rocksdb::Tickers::NUMBER_MERGE_FAILURES; + case 0x3E: + return rocksdb::Tickers::BLOOM_FILTER_PREFIX_CHECKED; + case 0x3F: + return rocksdb::Tickers::BLOOM_FILTER_PREFIX_USEFUL; + case 0x40: + return rocksdb::Tickers::NUMBER_OF_RESEEKS_IN_ITERATION; + case 0x41: + return rocksdb::Tickers::GET_UPDATES_SINCE_CALLS; + case 0x42: + return rocksdb::Tickers::BLOCK_CACHE_COMPRESSED_MISS; + case 0x43: + return rocksdb::Tickers::BLOCK_CACHE_COMPRESSED_HIT; + case 0x44: + return rocksdb::Tickers::BLOCK_CACHE_COMPRESSED_ADD; + case 0x45: + return rocksdb::Tickers::BLOCK_CACHE_COMPRESSED_ADD_FAILURES; + case 0x46: + return rocksdb::Tickers::WAL_FILE_SYNCED; + case 0x47: + return rocksdb::Tickers::WAL_FILE_BYTES; + case 0x48: + return rocksdb::Tickers::WRITE_DONE_BY_SELF; + case 0x49: + return rocksdb::Tickers::WRITE_DONE_BY_OTHER; + case 0x4A: + return rocksdb::Tickers::WRITE_TIMEDOUT; + case 0x4B: + return rocksdb::Tickers::WRITE_WITH_WAL; + case 0x4C: + return rocksdb::Tickers::COMPACT_READ_BYTES; + case 0x4D: + return rocksdb::Tickers::COMPACT_WRITE_BYTES; + case 0x4E: + return rocksdb::Tickers::FLUSH_WRITE_BYTES; + case 0x4F: + return rocksdb::Tickers::NUMBER_DIRECT_LOAD_TABLE_PROPERTIES; + case 0x50: + return rocksdb::Tickers::NUMBER_SUPERVERSION_ACQUIRES; + case 0x51: + return rocksdb::Tickers::NUMBER_SUPERVERSION_RELEASES; + case 0x52: + return rocksdb::Tickers::NUMBER_SUPERVERSION_CLEANUPS; + case 0x53: + return rocksdb::Tickers::NUMBER_BLOCK_COMPRESSED; + case 0x54: + return rocksdb::Tickers::NUMBER_BLOCK_DECOMPRESSED; + case 0x55: + return rocksdb::Tickers::NUMBER_BLOCK_NOT_COMPRESSED; + case 0x56: + return rocksdb::Tickers::MERGE_OPERATION_TOTAL_TIME; + case 0x57: + return rocksdb::Tickers::FILTER_OPERATION_TOTAL_TIME; + case 0x58: + return rocksdb::Tickers::ROW_CACHE_HIT; + case 0x59: + return rocksdb::Tickers::ROW_CACHE_MISS; + case 0x5A: + return rocksdb::Tickers::READ_AMP_ESTIMATE_USEFUL_BYTES; + case 0x5B: + return rocksdb::Tickers::READ_AMP_TOTAL_READ_BYTES; + case 0x5C: + return rocksdb::Tickers::NUMBER_RATE_LIMITER_DRAINS; + case 0x5D: + return rocksdb::Tickers::TICKER_ENUM_MAX; + + default: + // undefined/default + return rocksdb::Tickers::BLOCK_CACHE_MISS; + } + } +}; + +// The portal class for org.rocksdb.HistogramType +class HistogramTypeJni { + public: + // Returns the equivalent org.rocksdb.HistogramType for the provided + // C++ rocksdb::Histograms enum + static jbyte toJavaHistogramsType( + const rocksdb::Histograms& histograms) { + switch(histograms) { + case rocksdb::Histograms::DB_GET: + return 0x0; + case rocksdb::Histograms::DB_WRITE: + return 0x1; + case rocksdb::Histograms::COMPACTION_TIME: + return 0x2; + case rocksdb::Histograms::SUBCOMPACTION_SETUP_TIME: + return 0x3; + case rocksdb::Histograms::TABLE_SYNC_MICROS: + return 0x4; + case rocksdb::Histograms::COMPACTION_OUTFILE_SYNC_MICROS: + return 0x5; + case rocksdb::Histograms::WAL_FILE_SYNC_MICROS: + return 0x6; + case rocksdb::Histograms::MANIFEST_FILE_SYNC_MICROS: + return 0x7; + case rocksdb::Histograms::TABLE_OPEN_IO_MICROS: + return 0x8; + case rocksdb::Histograms::DB_MULTIGET: + return 0x9; + case rocksdb::Histograms::READ_BLOCK_COMPACTION_MICROS: + return 0xA; + case rocksdb::Histograms::READ_BLOCK_GET_MICROS: + return 0xB; + case rocksdb::Histograms::WRITE_RAW_BLOCK_MICROS: + return 0xC; + case rocksdb::Histograms::STALL_L0_SLOWDOWN_COUNT: + return 0xD; + case rocksdb::Histograms::STALL_MEMTABLE_COMPACTION_COUNT: + return 0xE; + case rocksdb::Histograms::STALL_L0_NUM_FILES_COUNT: + return 0xF; + case rocksdb::Histograms::HARD_RATE_LIMIT_DELAY_COUNT: + return 0x10; + case rocksdb::Histograms::SOFT_RATE_LIMIT_DELAY_COUNT: + return 0x11; + case rocksdb::Histograms::NUM_FILES_IN_SINGLE_COMPACTION: + return 0x12; + case rocksdb::Histograms::DB_SEEK: + return 0x13; + case rocksdb::Histograms::WRITE_STALL: + return 0x14; + case rocksdb::Histograms::SST_READ_MICROS: + return 0x15; + case rocksdb::Histograms::NUM_SUBCOMPACTIONS_SCHEDULED: + return 0x16; + case rocksdb::Histograms::BYTES_PER_READ: + return 0x17; + case rocksdb::Histograms::BYTES_PER_WRITE: + return 0x18; + case rocksdb::Histograms::BYTES_PER_MULTIGET: + return 0x19; + case rocksdb::Histograms::BYTES_COMPRESSED: + return 0x1A; + case rocksdb::Histograms::BYTES_DECOMPRESSED: + return 0x1B; + case rocksdb::Histograms::COMPRESSION_TIMES_NANOS: + return 0x1C; + case rocksdb::Histograms::DECOMPRESSION_TIMES_NANOS: + return 0x1D; + case rocksdb::Histograms::READ_NUM_MERGE_OPERANDS: + return 0x1E; + case rocksdb::Histograms::HISTOGRAM_ENUM_MAX: + return 0x1F; + + default: + // undefined/default + return 0x0; + } + } + + // Returns the equivalent C++ rocksdb::Histograms enum for the + // provided Java org.rocksdb.HistogramsType + static rocksdb::Histograms toCppHistograms(jbyte jhistograms_type) { + switch(jhistograms_type) { + case 0x0: + return rocksdb::Histograms::DB_GET; + case 0x1: + return rocksdb::Histograms::DB_WRITE; + case 0x2: + return rocksdb::Histograms::COMPACTION_TIME; + case 0x3: + return rocksdb::Histograms::SUBCOMPACTION_SETUP_TIME; + case 0x4: + return rocksdb::Histograms::TABLE_SYNC_MICROS; + case 0x5: + return rocksdb::Histograms::COMPACTION_OUTFILE_SYNC_MICROS; + case 0x6: + return rocksdb::Histograms::WAL_FILE_SYNC_MICROS; + case 0x7: + return rocksdb::Histograms::MANIFEST_FILE_SYNC_MICROS; + case 0x8: + return rocksdb::Histograms::TABLE_OPEN_IO_MICROS; + case 0x9: + return rocksdb::Histograms::DB_MULTIGET; + case 0xA: + return rocksdb::Histograms::READ_BLOCK_COMPACTION_MICROS; + case 0xB: + return rocksdb::Histograms::READ_BLOCK_GET_MICROS; + case 0xC: + return rocksdb::Histograms::WRITE_RAW_BLOCK_MICROS; + case 0xD: + return rocksdb::Histograms::STALL_L0_SLOWDOWN_COUNT; + case 0xE: + return rocksdb::Histograms::STALL_MEMTABLE_COMPACTION_COUNT; + case 0xF: + return rocksdb::Histograms::STALL_L0_NUM_FILES_COUNT; + case 0x10: + return rocksdb::Histograms::HARD_RATE_LIMIT_DELAY_COUNT; + case 0x11: + return rocksdb::Histograms::SOFT_RATE_LIMIT_DELAY_COUNT; + case 0x12: + return rocksdb::Histograms::NUM_FILES_IN_SINGLE_COMPACTION; + case 0x13: + return rocksdb::Histograms::DB_SEEK; + case 0x14: + return rocksdb::Histograms::WRITE_STALL; + case 0x15: + return rocksdb::Histograms::SST_READ_MICROS; + case 0x16: + return rocksdb::Histograms::NUM_SUBCOMPACTIONS_SCHEDULED; + case 0x17: + return rocksdb::Histograms::BYTES_PER_READ; + case 0x18: + return rocksdb::Histograms::BYTES_PER_WRITE; + case 0x19: + return rocksdb::Histograms::BYTES_PER_MULTIGET; + case 0x1A: + return rocksdb::Histograms::BYTES_COMPRESSED; + case 0x1B: + return rocksdb::Histograms::BYTES_DECOMPRESSED; + case 0x1C: + return rocksdb::Histograms::COMPRESSION_TIMES_NANOS; + case 0x1D: + return rocksdb::Histograms::DECOMPRESSION_TIMES_NANOS; + case 0x1E: + return rocksdb::Histograms::READ_NUM_MERGE_OPERANDS; + case 0x1F: + return rocksdb::Histograms::HISTOGRAM_ENUM_MAX; + + default: + // undefined/default + return rocksdb::Histograms::DB_GET; + } + } +}; + +// The portal class for org.rocksdb.StatsLevel +class StatsLevelJni { + public: + // Returns the equivalent org.rocksdb.StatsLevel for the provided + // C++ rocksdb::StatsLevel enum + static jbyte toJavaStatsLevel( + const rocksdb::StatsLevel& stats_level) { + switch(stats_level) { + case rocksdb::StatsLevel::kExceptDetailedTimers: + return 0x0; + case rocksdb::StatsLevel::kExceptTimeForMutex: + return 0x1; + case rocksdb::StatsLevel::kAll: + return 0x2; + + default: + // undefined/default + return 0x0; + } + } + + // Returns the equivalent C++ rocksdb::StatsLevel enum for the + // provided Java org.rocksdb.StatsLevel + static rocksdb::StatsLevel toCppStatsLevel(jbyte jstats_level) { + switch(jstats_level) { + case 0x0: + return rocksdb::StatsLevel::kExceptDetailedTimers; + case 0x1: + return rocksdb::StatsLevel::kExceptTimeForMutex; + case 0x2: + return rocksdb::StatsLevel::kAll; + + default: + // undefined/default + return rocksdb::StatsLevel::kExceptDetailedTimers; + } + } +}; + +// various utility functions for working with RocksDB and JNI +class JniUtil { + public: + /** + * Obtains a reference to the JNIEnv from + * the JVM + * + * If the current thread is not attached to the JavaVM + * then it will be attached so as to retrieve the JNIEnv + * + * If a thread is attached, it must later be manually + * released by calling JavaVM::DetachCurrentThread. + * This can be handled by always matching calls to this + * function with calls to {@link JniUtil::releaseJniEnv(JavaVM*, jboolean)} + * + * @param jvm (IN) A pointer to the JavaVM instance + * @param attached (OUT) A pointer to a boolean which + * will be set to JNI_TRUE if we had to attach the thread + * + * @return A pointer to the JNIEnv or nullptr if a fatal error + * occurs and the JNIEnv cannot be retrieved + */ + static JNIEnv* getJniEnv(JavaVM* jvm, jboolean* attached) { + assert(jvm != nullptr); + + JNIEnv *env; + const jint env_rs = jvm->GetEnv(reinterpret_cast(&env), + JNI_VERSION_1_2); + + if(env_rs == JNI_OK) { + // current thread is already attached, return the JNIEnv + *attached = JNI_FALSE; + return env; + } else if(env_rs == JNI_EDETACHED) { + // current thread is not attached, attempt to attach + const jint rs_attach = jvm->AttachCurrentThread(reinterpret_cast(&env), NULL); + if(rs_attach == JNI_OK) { + *attached = JNI_TRUE; + return env; + } else { + // error, could not attach the thread + std::cerr << "JniUtil::getJinEnv - Fatal: could not attach current thread to JVM!" << std::endl; + return nullptr; + } + } else if(env_rs == JNI_EVERSION) { + // error, JDK does not support JNI_VERSION_1_2+ + std::cerr << "JniUtil::getJinEnv - Fatal: JDK does not support JNI_VERSION_1_2" << std::endl; + return nullptr; + } else { + std::cerr << "JniUtil::getJinEnv - Fatal: Unknown error: env_rs=" << env_rs << std::endl; + return nullptr; + } + } + + /** + * Counterpart to {@link JniUtil::getJniEnv(JavaVM*, jboolean*)} + * + * Detachess the current thread from the JVM if it was previously + * attached + * + * @param jvm (IN) A pointer to the JavaVM instance + * @param attached (IN) JNI_TRUE if we previously had to attach the thread + * to the JavaVM to get the JNIEnv + */ + static void releaseJniEnv(JavaVM* jvm, jboolean& attached) { + assert(jvm != nullptr); + if(attached == JNI_TRUE) { + const jint rs_detach = jvm->DetachCurrentThread(); + assert(rs_detach == JNI_OK); + if(rs_detach != JNI_OK) { + std::cerr << "JniUtil::getJinEnv - Warn: Unable to detach current thread from JVM!" << std::endl; + } + } + } + + /** + * Copies a Java String[] to a C++ std::vector + * + * @param env (IN) A pointer to the java environment + * @param jss (IN) The Java String array to copy + * @param has_exception (OUT) will be set to JNI_TRUE + * if an OutOfMemoryError or ArrayIndexOutOfBoundsException + * exception occurs + * + * @return A std::vector containing copies of the Java strings + */ + static std::vector copyStrings(JNIEnv* env, + jobjectArray jss, jboolean* has_exception) { + return rocksdb::JniUtil::copyStrings(env, jss, + env->GetArrayLength(jss), has_exception); + } + + /** + * Copies a Java String[] to a C++ std::vector + * + * @param env (IN) A pointer to the java environment + * @param jss (IN) The Java String array to copy + * @param jss_len (IN) The length of the Java String array to copy + * @param has_exception (OUT) will be set to JNI_TRUE + * if an OutOfMemoryError or ArrayIndexOutOfBoundsException + * exception occurs + * + * @return A std::vector containing copies of the Java strings + */ + static std::vector copyStrings(JNIEnv* env, + jobjectArray jss, const jsize jss_len, jboolean* has_exception) { + std::vector strs; + for (jsize i = 0; i < jss_len; i++) { + jobject js = env->GetObjectArrayElement(jss, i); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + *has_exception = JNI_TRUE; + return strs; + } + + jstring jstr = static_cast(js); + const char* str = env->GetStringUTFChars(jstr, nullptr); + if(str == nullptr) { + // exception thrown: OutOfMemoryError + env->DeleteLocalRef(js); + *has_exception = JNI_TRUE; + return strs; + } + + strs.push_back(std::string(str)); + + env->ReleaseStringUTFChars(jstr, str); + env->DeleteLocalRef(js); + } + + *has_exception = JNI_FALSE; + return strs; + } + + /** + * Copies a jstring to a std::string + * and releases the original jstring + * + * If an exception occurs, then JNIEnv::ExceptionCheck() + * will have been called + * + * @param env (IN) A pointer to the java environment + * @param js (IN) The java string to copy + * @param has_exception (OUT) will be set to JNI_TRUE + * if an OutOfMemoryError exception occurs + * + * @return A std:string copy of the jstring, or an + * empty std::string if has_exception == JNI_TRUE + */ + static std::string copyString(JNIEnv* env, jstring js, + jboolean* has_exception) { + const char *utf = env->GetStringUTFChars(js, nullptr); + if(utf == nullptr) { + // exception thrown: OutOfMemoryError + env->ExceptionCheck(); + *has_exception = JNI_TRUE; + return std::string(); + } else if(env->ExceptionCheck()) { + // exception thrown + env->ReleaseStringUTFChars(js, utf); + *has_exception = JNI_TRUE; + return std::string(); + } + + std::string name(utf); + env->ReleaseStringUTFChars(js, utf); + *has_exception = JNI_FALSE; + return name; + } + + /** + * Copies bytes from a std::string to a jByteArray + * + * @param env A pointer to the java environment + * @param bytes The bytes to copy + * + * @return the Java byte[] or nullptr if an exception occurs + */ + static jbyteArray copyBytes(JNIEnv* env, std::string bytes) { + const jsize jlen = static_cast(bytes.size()); + + jbyteArray jbytes = env->NewByteArray(jlen); + if(jbytes == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + env->SetByteArrayRegion(jbytes, 0, jlen, + const_cast(reinterpret_cast(bytes.c_str()))); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jbytes); + return nullptr; + } + + return jbytes; + } + + /** + * Given a Java byte[][] which is an array of java.lang.Strings + * where each String is a byte[], the passed function `string_fn` + * will be called on each String, the result is the collected by + * calling the passed function `collector_fn` + * + * @param env (IN) A pointer to the java environment + * @param jbyte_strings (IN) A Java array of Strings expressed as bytes + * @param string_fn (IN) A transform function to call for each String + * @param collector_fn (IN) A collector which is called for the result + * of each `string_fn` + * @param has_exception (OUT) will be set to JNI_TRUE + * if an ArrayIndexOutOfBoundsException or OutOfMemoryError + * exception occurs + */ + template static void byteStrings(JNIEnv* env, + jobjectArray jbyte_strings, + std::function string_fn, + std::function collector_fn, + jboolean *has_exception) { + const jsize jlen = env->GetArrayLength(jbyte_strings); + + for(jsize i = 0; i < jlen; i++) { + jobject jbyte_string_obj = env->GetObjectArrayElement(jbyte_strings, i); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + *has_exception = JNI_TRUE; // signal error + return; + } + + jbyteArray jbyte_string_ary = + reinterpret_cast(jbyte_string_obj); + T result = byteString(env, jbyte_string_ary, string_fn, has_exception); + + env->DeleteLocalRef(jbyte_string_obj); + + if(*has_exception == JNI_TRUE) { + // exception thrown: OutOfMemoryError + return; + } + + collector_fn(i, result); + } + + *has_exception = JNI_FALSE; + } + + /** + * Given a Java String which is expressed as a Java Byte Array byte[], + * the passed function `string_fn` will be called on the String + * and the result returned + * + * @param env (IN) A pointer to the java environment + * @param jbyte_string_ary (IN) A Java String expressed in bytes + * @param string_fn (IN) A transform function to call on the String + * @param has_exception (OUT) will be set to JNI_TRUE + * if an OutOfMemoryError exception occurs + */ + template static T byteString(JNIEnv* env, + jbyteArray jbyte_string_ary, + std::function string_fn, + jboolean* has_exception) { + const jsize jbyte_string_len = env->GetArrayLength(jbyte_string_ary); + jbyte* jbyte_string = + env->GetByteArrayElements(jbyte_string_ary, nullptr); + if(jbyte_string == nullptr) { + // exception thrown: OutOfMemoryError + *has_exception = JNI_TRUE; + return nullptr; // signal error + } + + T result = + string_fn(reinterpret_cast(jbyte_string), jbyte_string_len); + + env->ReleaseByteArrayElements(jbyte_string_ary, jbyte_string, JNI_ABORT); + + *has_exception = JNI_FALSE; + return result; + } + + /** + * Converts a std::vector to a Java byte[][] where each Java String + * is expressed as a Java Byte Array byte[]. + * + * @param env A pointer to the java environment + * @param strings A vector of Strings + * + * @return A Java array of Strings expressed as bytes + */ + static jobjectArray stringsBytes(JNIEnv* env, std::vector strings) { + jclass jcls_ba = ByteJni::getArrayJClass(env); + if(jcls_ba == nullptr) { + // exception occurred + return nullptr; + } + + const jsize len = static_cast(strings.size()); + + jobjectArray jbyte_strings = env->NewObjectArray(len, jcls_ba, nullptr); + if(jbyte_strings == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + for (jsize i = 0; i < len; i++) { + std::string *str = &strings[i]; + const jsize str_len = static_cast(str->size()); + + jbyteArray jbyte_string_ary = env->NewByteArray(str_len); + if(jbyte_string_ary == nullptr) { + // exception thrown: OutOfMemoryError + env->DeleteLocalRef(jbyte_strings); + return nullptr; + } + + env->SetByteArrayRegion( + jbyte_string_ary, 0, str_len, + const_cast(reinterpret_cast(str->c_str()))); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jbyte_string_ary); + env->DeleteLocalRef(jbyte_strings); + return nullptr; + } + + env->SetObjectArrayElement(jbyte_strings, i, jbyte_string_ary); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + // or ArrayStoreException + env->DeleteLocalRef(jbyte_string_ary); + env->DeleteLocalRef(jbyte_strings); + return nullptr; + } + + env->DeleteLocalRef(jbyte_string_ary); + } + + return jbyte_strings; + } + + /* + * Helper for operations on a key and value + * for example WriteBatch->Put + * + * TODO(AR) could be extended to cover returning rocksdb::Status + * from `op` and used for RocksDB->Put etc. + */ + static void kv_op( + std::function op, + JNIEnv* env, jobject jobj, + jbyteArray jkey, jint jkey_len, + jbyteArray jentry_value, jint jentry_value_len) { + jbyte* key = env->GetByteArrayElements(jkey, nullptr); + if(env->ExceptionCheck()) { + // exception thrown: OutOfMemoryError + return; + } + + jbyte* value = env->GetByteArrayElements(jentry_value, nullptr); + if(env->ExceptionCheck()) { + // exception thrown: OutOfMemoryError + if(key != nullptr) { + env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + } + return; + } + + rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); + rocksdb::Slice value_slice(reinterpret_cast(value), + jentry_value_len); + + op(key_slice, value_slice); + + if(value != nullptr) { + env->ReleaseByteArrayElements(jentry_value, value, JNI_ABORT); + } + if(key != nullptr) { + env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + } + } + + /* + * Helper for operations on a key + * for example WriteBatch->Delete + * + * TODO(AR) could be extended to cover returning rocksdb::Status + * from `op` and used for RocksDB->Delete etc. + */ + static void k_op( + std::function op, + JNIEnv* env, jobject jobj, + jbyteArray jkey, jint jkey_len) { + jbyte* key = env->GetByteArrayElements(jkey, nullptr); + if(env->ExceptionCheck()) { + // exception thrown: OutOfMemoryError + return; + } + + rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); + + op(key_slice); + + if(key != nullptr) { + env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + } + } + + /* + * Helper for operations on a value + * for example WriteBatchWithIndex->GetFromBatch + */ + static jbyteArray v_op( + std::function op, + JNIEnv* env, jbyteArray jkey, jint jkey_len) { + jbyte* key = env->GetByteArrayElements(jkey, nullptr); + if(env->ExceptionCheck()) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); + + std::string value; + rocksdb::Status s = op(key_slice, &value); + + if(key != nullptr) { + env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + } + + if (s.IsNotFound()) { + return nullptr; + } + + if (s.ok()) { + jbyteArray jret_value = + env->NewByteArray(static_cast(value.size())); + if(jret_value == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + env->SetByteArrayRegion(jret_value, 0, static_cast(value.size()), + const_cast(reinterpret_cast(value.c_str()))); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + if(jret_value != nullptr) { + env->DeleteLocalRef(jret_value); + } + return nullptr; + } + + return jret_value; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return nullptr; + } +}; + } // namespace rocksdb #endif // JAVA_ROCKSJNI_PORTAL_H_ diff --git a/java/rocksjni/ratelimiterjni.cc b/java/rocksjni/ratelimiterjni.cc new file mode 100644 index 00000000000..b4174ff102e --- /dev/null +++ b/java/rocksjni/ratelimiterjni.cc @@ -0,0 +1,96 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ for RateLimiter. + +#include "rocksjni/portal.h" +#include "include/org_rocksdb_RateLimiter.h" +#include "rocksdb/rate_limiter.h" + +/* + * Class: org_rocksdb_RateLimiter + * Method: newRateLimiterHandle + * Signature: (JJI)J + */ +jlong Java_org_rocksdb_RateLimiter_newRateLimiterHandle( + JNIEnv* env, jclass jclazz, jlong jrate_bytes_per_second, + jlong jrefill_period_micros, jint jfairness) { + auto * sptr_rate_limiter = + new std::shared_ptr(rocksdb::NewGenericRateLimiter( + static_cast(jrate_bytes_per_second), + static_cast(jrefill_period_micros), + static_cast(jfairness))); + + return reinterpret_cast(sptr_rate_limiter); +} + +/* + * Class: org_rocksdb_RateLimiter + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_RateLimiter_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* handle = + reinterpret_cast *>(jhandle); + delete handle; // delete std::shared_ptr +} + +/* + * Class: org_rocksdb_RateLimiter + * Method: setBytesPerSecond + * Signature: (JJ)V + */ +void Java_org_rocksdb_RateLimiter_setBytesPerSecond( + JNIEnv* env, jobject jobj, jlong handle, + jlong jbytes_per_second) { + reinterpret_cast *>(handle)->get()-> + SetBytesPerSecond(jbytes_per_second); +} + +/* + * Class: org_rocksdb_RateLimiter + * Method: request + * Signature: (JJ)V + */ +void Java_org_rocksdb_RateLimiter_request( + JNIEnv* env, jobject jobj, jlong handle, + jlong jbytes) { + reinterpret_cast *>(handle)->get()-> + Request(jbytes, rocksdb::Env::IO_TOTAL); +} + +/* + * Class: org_rocksdb_RateLimiter + * Method: getSingleBurstBytes + * Signature: (J)J + */ +jlong Java_org_rocksdb_RateLimiter_getSingleBurstBytes( + JNIEnv* env, jobject jobj, jlong handle) { + return reinterpret_cast *>(handle)-> + get()->GetSingleBurstBytes(); +} + +/* + * Class: org_rocksdb_RateLimiter + * Method: getTotalBytesThrough + * Signature: (J)J + */ +jlong Java_org_rocksdb_RateLimiter_getTotalBytesThrough( + JNIEnv* env, jobject jobj, jlong handle) { + return reinterpret_cast *>(handle)-> + get()->GetTotalBytesThrough(); +} + +/* + * Class: org_rocksdb_RateLimiter + * Method: getTotalRequests + * Signature: (J)J + */ +jlong Java_org_rocksdb_RateLimiter_getTotalRequests( + JNIEnv* env, jobject jobj, jlong handle) { + return reinterpret_cast *>(handle)-> + get()->GetTotalRequests(); +} diff --git a/java/rocksjni/remove_emptyvalue_compactionfilterjni.cc b/java/rocksjni/remove_emptyvalue_compactionfilterjni.cc new file mode 100644 index 00000000000..8c54a46b864 --- /dev/null +++ b/java/rocksjni/remove_emptyvalue_compactionfilterjni.cc @@ -0,0 +1,24 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include + +#include "include/org_rocksdb_RemoveEmptyValueCompactionFilter.h" +#include "utilities/compaction_filters/remove_emptyvalue_compactionfilter.h" + + +/* + * Class: org_rocksdb_RemoveEmptyValueCompactionFilter + * Method: createNewRemoveEmptyValueCompactionFilter0 + * Signature: ()J + */ +jlong Java_org_rocksdb_RemoveEmptyValueCompactionFilter_createNewRemoveEmptyValueCompactionFilter0( + JNIEnv* env, jclass jcls) { + auto* compaction_filter = + new rocksdb::RemoveEmptyValueCompactionFilter(); + + // set the native handle to our native compaction filter + return reinterpret_cast(compaction_filter); +} diff --git a/java/rocksjni/restorejni.cc b/java/rocksjni/restorejni.cc index 942e707e68a..eb8e65b4a1b 100644 --- a/java/rocksjni/restorejni.cc +++ b/java/rocksjni/restorejni.cc @@ -1,20 +1,18 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ and enables -// calling c++ rocksdb::RestoreBackupableDB and rocksdb::RestoreOptions methods +// calling C++ rocksdb::RestoreOptions methods // from Java side. #include #include #include -#include #include #include "include/org_rocksdb_RestoreOptions.h" -#include "include/org_rocksdb_RestoreBackupableDB.h" #include "rocksjni/portal.h" #include "rocksdb/utilities/backupable_db.h" /* @@ -23,123 +21,19 @@ * Signature: (Z)J */ jlong Java_org_rocksdb_RestoreOptions_newRestoreOptions(JNIEnv* env, - jobject jobj, jboolean keep_log_files) { - auto ropt = new rocksdb::RestoreOptions(keep_log_files); + jclass jcls, jboolean keep_log_files) { + auto* ropt = new rocksdb::RestoreOptions(keep_log_files); return reinterpret_cast(ropt); } /* * Class: org_rocksdb_RestoreOptions - * Method: dispose + * Method: disposeInternal * Signature: (J)V */ -void Java_org_rocksdb_RestoreOptions_dispose(JNIEnv* env, jobject jobj, +void Java_org_rocksdb_RestoreOptions_disposeInternal(JNIEnv* env, jobject jobj, jlong jhandle) { - auto ropt = reinterpret_cast(jhandle); - assert(ropt); - delete ropt; -} - -/* - * Class: org_rocksdb_RestoreBackupableDB - * Method: newRestoreBackupableDB - * Signature: (J)J - */ -jlong Java_org_rocksdb_RestoreBackupableDB_newRestoreBackupableDB(JNIEnv* env, - jobject jobj, jlong jopt_handle) { - auto opt = reinterpret_cast(jopt_handle); - auto rdb = new rocksdb::RestoreBackupableDB(rocksdb::Env::Default(), *opt); - return reinterpret_cast(rdb); -} - -/* - * Class: org_rocksdb_RestoreBackupableDB - * Method: restoreDBFromBackup0 - * Signature: (JJLjava/lang/String;Ljava/lang/String;J)V - */ -void Java_org_rocksdb_RestoreBackupableDB_restoreDBFromBackup0(JNIEnv* env, - jobject jobj, jlong jhandle, jlong jbackup_id, jstring jdb_dir, - jstring jwal_dir, jlong jopt_handle) { - auto opt = reinterpret_cast(jopt_handle); - - const char* cdb_dir = env->GetStringUTFChars(jdb_dir, 0); - const char* cwal_dir = env->GetStringUTFChars(jwal_dir, 0); - - auto rdb = reinterpret_cast(jhandle); - rocksdb::Status s = - rdb->RestoreDBFromBackup(jbackup_id, cdb_dir, cwal_dir, *opt); - - env->ReleaseStringUTFChars(jdb_dir, cdb_dir); - env->ReleaseStringUTFChars(jwal_dir, cwal_dir); - - if(!s.ok()) { - rocksdb::RocksDBExceptionJni::ThrowNew(env, s); - } -} - -/* - * Class: org_rocksdb_RestoreBackupableDB - * Method: restoreDBFromLatestBackup0 - * Signature: (JLjava/lang/String;Ljava/lang/String;J)V - */ -void Java_org_rocksdb_RestoreBackupableDB_restoreDBFromLatestBackup0( - JNIEnv* env, jobject jobj, jlong jhandle, jstring jdb_dir, jstring jwal_dir, - jlong jopt_handle) { - auto opt = reinterpret_cast(jopt_handle); - - const char* cdb_dir = env->GetStringUTFChars(jdb_dir, 0); - const char* cwal_dir = env->GetStringUTFChars(jwal_dir, 0); - - auto rdb = reinterpret_cast(jhandle); - rocksdb::Status s = - rdb->RestoreDBFromLatestBackup(cdb_dir, cwal_dir, *opt); - - env->ReleaseStringUTFChars(jdb_dir, cdb_dir); - env->ReleaseStringUTFChars(jwal_dir, cwal_dir); - - if(!s.ok()) { - rocksdb::RocksDBExceptionJni::ThrowNew(env, s); - } -} - -/* - * Class: org_rocksdb_RestoreBackupableDB - * Method: purgeOldBackups0 - * Signature: (JI)V - */ -void Java_org_rocksdb_RestoreBackupableDB_purgeOldBackups0(JNIEnv* env, - jobject jobj, jlong jhandle, jint jnum_backups_to_keep) { - auto rdb = reinterpret_cast(jhandle); - rocksdb::Status s = rdb->PurgeOldBackups(jnum_backups_to_keep); - - if(!s.ok()) { - rocksdb::RocksDBExceptionJni::ThrowNew(env, s); - } -} - -/* - * Class: org_rocksdb_RestoreBackupableDB - * Method: deleteBackup0 - * Signature: (JJ)V - */ -void Java_org_rocksdb_RestoreBackupableDB_deleteBackup0(JNIEnv* env, - jobject jobj, jlong jhandle, jlong jbackup_id) { - auto rdb = reinterpret_cast(jhandle); - rocksdb::Status s = rdb->DeleteBackup(jbackup_id); - - if(!s.ok()) { - rocksdb::RocksDBExceptionJni::ThrowNew(env, s); - } -} - -/* - * Class: org_rocksdb_RestoreBackupableDB - * Method: dispose - * Signature: (J)V - */ -void Java_org_rocksdb_RestoreBackupableDB_dispose(JNIEnv* env, jobject jobj, - jlong jhandle) { - auto ropt = reinterpret_cast(jhandle); + auto* ropt = reinterpret_cast(jhandle); assert(ropt); delete ropt; } diff --git a/java/rocksjni/rocksdb_exception_test.cc b/java/rocksjni/rocksdb_exception_test.cc new file mode 100644 index 00000000000..339d4c5eda3 --- /dev/null +++ b/java/rocksjni/rocksdb_exception_test.cc @@ -0,0 +1,78 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include + +#include "include/org_rocksdb_RocksDBExceptionTest.h" + +#include "rocksdb/slice.h" +#include "rocksdb/status.h" +#include "rocksjni/portal.h" + +/* + * Class: org_rocksdb_RocksDBExceptionTest + * Method: raiseException + * Signature: ()V + */ +void Java_org_rocksdb_RocksDBExceptionTest_raiseException(JNIEnv* env, + jobject jobj) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, std::string("test message")); +} + +/* + * Class: org_rocksdb_RocksDBExceptionTest + * Method: raiseExceptionWithStatusCode + * Signature: ()V + */ +void Java_org_rocksdb_RocksDBExceptionTest_raiseExceptionWithStatusCode( + JNIEnv* env, jobject jobj) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, "test message", + rocksdb::Status::NotSupported()); +} + +/* + * Class: org_rocksdb_RocksDBExceptionTest + * Method: raiseExceptionNoMsgWithStatusCode + * Signature: ()V + */ +void Java_org_rocksdb_RocksDBExceptionTest_raiseExceptionNoMsgWithStatusCode( + JNIEnv* env, jobject jobj) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, rocksdb::Status::NotSupported()); +} + +/* + * Class: org_rocksdb_RocksDBExceptionTest + * Method: raiseExceptionWithStatusCodeSubCode + * Signature: ()V + */ +void Java_org_rocksdb_RocksDBExceptionTest_raiseExceptionWithStatusCodeSubCode( + JNIEnv* env, jobject jobj) { + rocksdb::RocksDBExceptionJni::ThrowNew( + env, "test message", + rocksdb::Status::TimedOut(rocksdb::Status::SubCode::kLockTimeout)); +} + +/* + * Class: org_rocksdb_RocksDBExceptionTest + * Method: raiseExceptionNoMsgWithStatusCodeSubCode + * Signature: ()V + */ +void Java_org_rocksdb_RocksDBExceptionTest_raiseExceptionNoMsgWithStatusCodeSubCode( + JNIEnv* env, jobject jobj) { + rocksdb::RocksDBExceptionJni::ThrowNew( + env, rocksdb::Status::TimedOut(rocksdb::Status::SubCode::kLockTimeout)); +} + +/* + * Class: org_rocksdb_RocksDBExceptionTest + * Method: raiseExceptionWithStatusCodeState + * Signature: ()V + */ +void Java_org_rocksdb_RocksDBExceptionTest_raiseExceptionWithStatusCodeState( + JNIEnv* env, jobject jobj) { + rocksdb::Slice state("test state"); + rocksdb::RocksDBExceptionJni::ThrowNew(env, "test message", + rocksdb::Status::NotSupported(state)); +} diff --git a/java/rocksjni/rocksjni.cc b/java/rocksjni/rocksjni.cc index f55290f6493..a08a4597142 100644 --- a/java/rocksjni/rocksjni.cc +++ b/java/rocksjni/rocksjni.cc @@ -1,220 +1,691 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ and enables // calling c++ rocksdb::DB methods from Java side. +#include #include #include -#include +#include +#include +#include #include +#include #include #include "include/org_rocksdb_RocksDB.h" -#include "rocksjni/portal.h" -#include "rocksdb/db.h" #include "rocksdb/cache.h" +#include "rocksdb/db.h" +#include "rocksdb/options.h" +#include "rocksdb/types.h" +#include "rocksjni/portal.h" + +#ifdef min +#undef min +#endif ////////////////////////////////////////////////////////////////////////////// // rocksdb::DB::Open +jlong rocksdb_open_helper(JNIEnv* env, jlong jopt_handle, jstring jdb_path, + std::function open_fn + ) { + const char* db_path = env->GetStringUTFChars(jdb_path, nullptr); + if(db_path == nullptr) { + // exception thrown: OutOfMemoryError + return 0; + } + + auto* opt = reinterpret_cast(jopt_handle); + rocksdb::DB* db = nullptr; + rocksdb::Status s = open_fn(*opt, db_path, &db); + + env->ReleaseStringUTFChars(jdb_path, db_path); + + if (s.ok()) { + return reinterpret_cast(db); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return 0; + } +} /* * Class: org_rocksdb_RocksDB * Method: open - * Signature: (JLjava/lang/String;)V + * Signature: (JLjava/lang/String;)J + */ +jlong Java_org_rocksdb_RocksDB_open__JLjava_lang_String_2( + JNIEnv* env, jclass jcls, jlong jopt_handle, jstring jdb_path) { + return rocksdb_open_helper(env, jopt_handle, jdb_path, + (rocksdb::Status(*) + (const rocksdb::Options&, const std::string&, rocksdb::DB**) + )&rocksdb::DB::Open + ); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: openROnly + * Signature: (JLjava/lang/String;)J */ -void Java_org_rocksdb_RocksDB_open( - JNIEnv* env, jobject jdb, jlong jopt_handle, jstring jdb_path) { - auto opt = reinterpret_cast(jopt_handle); +jlong Java_org_rocksdb_RocksDB_openROnly__JLjava_lang_String_2( + JNIEnv* env, jclass jcls, jlong jopt_handle, jstring jdb_path) { + return rocksdb_open_helper(env, jopt_handle, jdb_path, []( + const rocksdb::Options& options, + const std::string& db_path, rocksdb::DB** db) { + return rocksdb::DB::OpenForReadOnly(options, db_path, db); + }); +} + +jlongArray rocksdb_open_helper(JNIEnv* env, jlong jopt_handle, + jstring jdb_path, jobjectArray jcolumn_names, jlongArray jcolumn_options, + std::function&, + std::vector*, + rocksdb::DB**)> open_fn + ) { + const char* db_path = env->GetStringUTFChars(jdb_path, nullptr); + if(db_path == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + const jsize len_cols = env->GetArrayLength(jcolumn_names); + jlong* jco = env->GetLongArrayElements(jcolumn_options, nullptr); + if(jco == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseStringUTFChars(jdb_path, db_path); + return nullptr; + } + + std::vector column_families; + jboolean has_exception = JNI_FALSE; + rocksdb::JniUtil::byteStrings( + env, + jcolumn_names, + [](const char* str_data, const size_t str_len) { + return std::string(str_data, str_len); + }, + [&jco, &column_families](size_t idx, std::string cf_name) { + rocksdb::ColumnFamilyOptions* cf_options = + reinterpret_cast(jco[idx]); + column_families.push_back( + rocksdb::ColumnFamilyDescriptor(cf_name, *cf_options)); + }, + &has_exception); + + env->ReleaseLongArrayElements(jcolumn_options, jco, JNI_ABORT); + + if(has_exception == JNI_TRUE) { + // exception occurred + env->ReleaseStringUTFChars(jdb_path, db_path); + return nullptr; + } + + auto* opt = reinterpret_cast(jopt_handle); + std::vector handles; rocksdb::DB* db = nullptr; - const char* db_path = env->GetStringUTFChars(jdb_path, 0); - rocksdb::Status s = rocksdb::DB::Open(*opt, db_path, &db); + rocksdb::Status s = open_fn(*opt, db_path, column_families, + &handles, &db); + + // we have now finished with db_path env->ReleaseStringUTFChars(jdb_path, db_path); + // check if open operation was successful if (s.ok()) { - rocksdb::RocksDBJni::setHandle(env, jdb, db); - return; + const jsize resultsLen = 1 + len_cols; //db handle + column family handles + std::unique_ptr results = + std::unique_ptr(new jlong[resultsLen]); + results[0] = reinterpret_cast(db); + for(int i = 1; i <= len_cols; i++) { + results[i] = reinterpret_cast(handles[i - 1]); + } + + jlongArray jresults = env->NewLongArray(resultsLen); + if(jresults == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + env->SetLongArrayRegion(jresults, 0, resultsLen, results.get()); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jresults); + return nullptr; + } + + return jresults; + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return nullptr; } - rocksdb::RocksDBExceptionJni::ThrowNew(env, s); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: openROnly + * Signature: (JLjava/lang/String;[[B[J)[J + */ +jlongArray Java_org_rocksdb_RocksDB_openROnly__JLjava_lang_String_2_3_3B_3J( + JNIEnv* env, jclass jcls, jlong jopt_handle, jstring jdb_path, + jobjectArray jcolumn_names, jlongArray jcolumn_options) { + return rocksdb_open_helper(env, jopt_handle, jdb_path, jcolumn_names, + jcolumn_options, []( + const rocksdb::DBOptions& options, const std::string& db_path, + const std::vector& column_families, + std::vector* handles, rocksdb::DB** db) { + return rocksdb::DB::OpenForReadOnly(options, db_path, column_families, + handles, db); + }); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: open + * Signature: (JLjava/lang/String;[[B[J)[J + */ +jlongArray Java_org_rocksdb_RocksDB_open__JLjava_lang_String_2_3_3B_3J( + JNIEnv* env, jclass jcls, jlong jopt_handle, jstring jdb_path, + jobjectArray jcolumn_names, jlongArray jcolumn_options) { + return rocksdb_open_helper(env, jopt_handle, jdb_path, jcolumn_names, + jcolumn_options, (rocksdb::Status(*) + (const rocksdb::DBOptions&, const std::string&, + const std::vector&, + std::vector*, rocksdb::DB**) + )&rocksdb::DB::Open + ); +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::ListColumnFamilies + +/* + * Class: org_rocksdb_RocksDB + * Method: listColumnFamilies + * Signature: (JLjava/lang/String;)[[B + */ +jobjectArray Java_org_rocksdb_RocksDB_listColumnFamilies( + JNIEnv* env, jclass jclazz, jlong jopt_handle, jstring jdb_path) { + std::vector column_family_names; + const char* db_path = env->GetStringUTFChars(jdb_path, nullptr); + if(db_path == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + auto* opt = reinterpret_cast(jopt_handle); + rocksdb::Status s = rocksdb::DB::ListColumnFamilies(*opt, db_path, + &column_family_names); + + env->ReleaseStringUTFChars(jdb_path, db_path); + + jobjectArray jcolumn_family_names = + rocksdb::JniUtil::stringsBytes(env, column_family_names); + + return jcolumn_family_names; } ////////////////////////////////////////////////////////////////////////////// // rocksdb::DB::Put -void rocksdb_put_helper( - JNIEnv* env, rocksdb::DB* db, const rocksdb::WriteOptions& write_options, - jbyteArray jkey, jint jkey_len, - jbyteArray jvalue, jint jvalue_len) { +/** + * @return true if the put succeeded, false if a Java Exception was thrown + */ +bool rocksdb_put_helper(JNIEnv* env, rocksdb::DB* db, + const rocksdb::WriteOptions& write_options, + rocksdb::ColumnFamilyHandle* cf_handle, jbyteArray jkey, + jint jkey_off, jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len) { + jbyte* key = new jbyte[jkey_len]; + env->GetByteArrayRegion(jkey, jkey_off, jkey_len, key); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + delete [] key; + return false; + } + + jbyte* value = new jbyte[jval_len]; + env->GetByteArrayRegion(jval, jval_off, jval_len, value); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + delete [] value; + delete [] key; + return false; + } - jbyte* key = env->GetByteArrayElements(jkey, 0); - jbyte* value = env->GetByteArrayElements(jvalue, 0); rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); - rocksdb::Slice value_slice(reinterpret_cast(value), jvalue_len); + rocksdb::Slice value_slice(reinterpret_cast(value), jval_len); - rocksdb::Status s = db->Put(write_options, key_slice, value_slice); + rocksdb::Status s; + if (cf_handle != nullptr) { + s = db->Put(write_options, cf_handle, key_slice, value_slice); + } else { + // backwards compatibility + s = db->Put(write_options, key_slice, value_slice); + } - // trigger java unref on key and value. - // by passing JNI_ABORT, it will simply release the reference without - // copying the result back to the java byte array. - env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); - env->ReleaseByteArrayElements(jvalue, value, JNI_ABORT); + // cleanup + delete [] value; + delete [] key; if (s.ok()) { - return; + return true; + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return false; } - rocksdb::RocksDBExceptionJni::ThrowNew(env, s); } /* * Class: org_rocksdb_RocksDB * Method: put - * Signature: (J[BI[BI)V + * Signature: (J[BII[BII)V */ -void Java_org_rocksdb_RocksDB_put__J_3BI_3BI( - JNIEnv* env, jobject jdb, jlong jdb_handle, - jbyteArray jkey, jint jkey_len, - jbyteArray jvalue, jint jvalue_len) { - auto db = reinterpret_cast(jdb_handle); +void Java_org_rocksdb_RocksDB_put__J_3BII_3BII(JNIEnv* env, jobject jdb, + jlong jdb_handle, + jbyteArray jkey, jint jkey_off, + jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len) { + auto* db = reinterpret_cast(jdb_handle); static const rocksdb::WriteOptions default_write_options = rocksdb::WriteOptions(); - rocksdb_put_helper(env, db, default_write_options, - jkey, jkey_len, - jvalue, jvalue_len); + rocksdb_put_helper(env, db, default_write_options, nullptr, jkey, jkey_off, + jkey_len, jval, jval_off, jval_len); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: put + * Signature: (J[BII[BIIJ)V + */ +void Java_org_rocksdb_RocksDB_put__J_3BII_3BIIJ(JNIEnv* env, jobject jdb, + jlong jdb_handle, + jbyteArray jkey, jint jkey_off, + jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len, + jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + static const rocksdb::WriteOptions default_write_options = + rocksdb::WriteOptions(); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + rocksdb_put_helper(env, db, default_write_options, cf_handle, jkey, + jkey_off, jkey_len, jval, jval_off, jval_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + } } /* * Class: org_rocksdb_RocksDB * Method: put - * Signature: (JJ[BI[BI)V - */ -void Java_org_rocksdb_RocksDB_put__JJ_3BI_3BI( - JNIEnv* env, jobject jdb, - jlong jdb_handle, jlong jwrite_options_handle, - jbyteArray jkey, jint jkey_len, - jbyteArray jvalue, jint jvalue_len) { - auto db = reinterpret_cast(jdb_handle); - auto write_options = reinterpret_cast( + * Signature: (JJ[BII[BII)V + */ +void Java_org_rocksdb_RocksDB_put__JJ_3BII_3BII(JNIEnv* env, jobject jdb, + jlong jdb_handle, + jlong jwrite_options_handle, + jbyteArray jkey, jint jkey_off, + jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = reinterpret_cast( jwrite_options_handle); - rocksdb_put_helper(env, db, *write_options, - jkey, jkey_len, - jvalue, jvalue_len); + rocksdb_put_helper(env, db, *write_options, nullptr, jkey, jkey_off, jkey_len, + jval, jval_off, jval_len); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: put + * Signature: (JJ[BII[BIIJ)V + */ +void Java_org_rocksdb_RocksDB_put__JJ_3BII_3BIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jwrite_options_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len, jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = reinterpret_cast( + jwrite_options_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + rocksdb_put_helper(env, db, *write_options, cf_handle, jkey, jkey_off, + jkey_len, jval, jval_off, jval_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + } } ////////////////////////////////////////////////////////////////////////////// // rocksdb::DB::Write /* * Class: org_rocksdb_RocksDB - * Method: write - * Signature: (JJ)V + * Method: write0 + * Signature: (JJJ)V + */ +void Java_org_rocksdb_RocksDB_write0( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jlong jwrite_options_handle, jlong jwb_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = reinterpret_cast( + jwrite_options_handle); + auto* wb = reinterpret_cast(jwb_handle); + + rocksdb::Status s = db->Write(*write_options, wb); + + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_RocksDB + * Method: write1 + * Signature: (JJJ)V */ -void Java_org_rocksdb_RocksDB_write( - JNIEnv* env, jobject jdb, - jlong jwrite_options_handle, jlong jbatch_handle) { - rocksdb::DB* db = rocksdb::RocksDBJni::getHandle(env, jdb); - auto write_options = reinterpret_cast( +void Java_org_rocksdb_RocksDB_write1( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jlong jwrite_options_handle, jlong jwbwi_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = reinterpret_cast( jwrite_options_handle); - auto batch = reinterpret_cast(jbatch_handle); + auto* wbwi = reinterpret_cast(jwbwi_handle); + auto* wb = wbwi->GetWriteBatch(); - rocksdb::Status s = db->Write(*write_options, batch); + rocksdb::Status s = db->Write(*write_options, wb); if (!s.ok()) { rocksdb::RocksDBExceptionJni::ThrowNew(env, s); } } +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::KeyMayExist +jboolean key_may_exist_helper(JNIEnv* env, rocksdb::DB* db, + const rocksdb::ReadOptions& read_opt, + rocksdb::ColumnFamilyHandle* cf_handle, jbyteArray jkey, jint jkey_off, + jint jkey_len, jobject jstring_builder, bool* has_exception) { + + jbyte* key = new jbyte[jkey_len]; + env->GetByteArrayRegion(jkey, jkey_off, jkey_len, key); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + delete [] key; + *has_exception = true; + return false; + } + + rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); + + std::string value; + bool value_found = false; + bool keyMayExist; + if (cf_handle != nullptr) { + keyMayExist = db->KeyMayExist(read_opt, cf_handle, key_slice, + &value, &value_found); + } else { + keyMayExist = db->KeyMayExist(read_opt, key_slice, + &value, &value_found); + } + + // cleanup + delete [] key; + + // extract the value + if (value_found && !value.empty()) { + jobject jresult_string_builder = + rocksdb::StringBuilderJni::append(env, jstring_builder, + value.c_str()); + if(jresult_string_builder == nullptr) { + *has_exception = true; + return false; + } + } + + *has_exception = false; + return static_cast(keyMayExist); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: keyMayExist + * Signature: (J[BIILjava/lang/StringBuilder;)Z + */ +jboolean Java_org_rocksdb_RocksDB_keyMayExist__J_3BIILjava_lang_StringBuilder_2( + JNIEnv* env, jobject jdb, jlong jdb_handle, jbyteArray jkey, jint jkey_off, + jint jkey_len, jobject jstring_builder) { + auto* db = reinterpret_cast(jdb_handle); + bool has_exception = false; + return key_may_exist_helper(env, db, rocksdb::ReadOptions(), + nullptr, jkey, jkey_off, jkey_len, jstring_builder, &has_exception); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: keyMayExist + * Signature: (J[BIIJLjava/lang/StringBuilder;)Z + */ +jboolean Java_org_rocksdb_RocksDB_keyMayExist__J_3BIIJLjava_lang_StringBuilder_2( + JNIEnv* env, jobject jdb, jlong jdb_handle, jbyteArray jkey, jint jkey_off, + jint jkey_len, jlong jcf_handle, jobject jstring_builder) { + auto* db = reinterpret_cast(jdb_handle); + auto* cf_handle = reinterpret_cast( + jcf_handle); + if (cf_handle != nullptr) { + bool has_exception = false; + return key_may_exist_helper(env, db, rocksdb::ReadOptions(), + cf_handle, jkey, jkey_off, jkey_len, jstring_builder, &has_exception); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + return true; + } +} + +/* + * Class: org_rocksdb_RocksDB + * Method: keyMayExist + * Signature: (JJ[BIILjava/lang/StringBuilder;)Z + */ +jboolean Java_org_rocksdb_RocksDB_keyMayExist__JJ_3BIILjava_lang_StringBuilder_2( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jread_options_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, jobject jstring_builder) { + auto* db = reinterpret_cast(jdb_handle); + auto& read_options = *reinterpret_cast( + jread_options_handle); + bool has_exception = false; + return key_may_exist_helper(env, db, read_options, + nullptr, jkey, jkey_off, jkey_len, jstring_builder, &has_exception); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: keyMayExist + * Signature: (JJ[BIIJLjava/lang/StringBuilder;)Z + */ +jboolean Java_org_rocksdb_RocksDB_keyMayExist__JJ_3BIIJLjava_lang_StringBuilder_2( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jread_options_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, jlong jcf_handle, + jobject jstring_builder) { + auto* db = reinterpret_cast(jdb_handle); + auto& read_options = *reinterpret_cast( + jread_options_handle); + auto* cf_handle = reinterpret_cast( + jcf_handle); + if (cf_handle != nullptr) { + bool has_exception = false; + return key_may_exist_helper(env, db, read_options, cf_handle, + jkey, jkey_off, jkey_len, jstring_builder, &has_exception); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + return true; + } +} + ////////////////////////////////////////////////////////////////////////////// // rocksdb::DB::Get jbyteArray rocksdb_get_helper( JNIEnv* env, rocksdb::DB* db, const rocksdb::ReadOptions& read_opt, - jbyteArray jkey, jint jkey_len) { - jboolean isCopy; - jbyte* key = env->GetByteArrayElements(jkey, &isCopy); + rocksdb::ColumnFamilyHandle* column_family_handle, jbyteArray jkey, + jint jkey_off, jint jkey_len) { + + jbyte* key = new jbyte[jkey_len]; + env->GetByteArrayRegion(jkey, jkey_off, jkey_len, key); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + delete [] key; + return nullptr; + } + rocksdb::Slice key_slice( reinterpret_cast(key), jkey_len); std::string value; - rocksdb::Status s = db->Get( - read_opt, key_slice, &value); + rocksdb::Status s; + if (column_family_handle != nullptr) { + s = db->Get(read_opt, column_family_handle, key_slice, &value); + } else { + // backwards compatibility + s = db->Get(read_opt, key_slice, &value); + } - // trigger java unref on key. - // by passing JNI_ABORT, it will simply release the reference without - // copying the result back to the java byte array. - env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + // cleanup + delete [] key; if (s.IsNotFound()) { return nullptr; } if (s.ok()) { - jbyteArray jvalue = env->NewByteArray(value.size()); - env->SetByteArrayRegion( - jvalue, 0, value.size(), - reinterpret_cast(value.c_str())); - return jvalue; + jbyteArray jret_value = rocksdb::JniUtil::copyBytes(env, value); + if(jret_value == nullptr) { + // exception occurred + return nullptr; + } + return jret_value; } - rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); return nullptr; } /* * Class: org_rocksdb_RocksDB * Method: get - * Signature: (J[BI)[B + * Signature: (J[BII)[B */ -jbyteArray Java_org_rocksdb_RocksDB_get__J_3BI( +jbyteArray Java_org_rocksdb_RocksDB_get__J_3BII( JNIEnv* env, jobject jdb, jlong jdb_handle, - jbyteArray jkey, jint jkey_len) { + jbyteArray jkey, jint jkey_off, jint jkey_len) { return rocksdb_get_helper(env, reinterpret_cast(jdb_handle), - rocksdb::ReadOptions(), - jkey, jkey_len); + rocksdb::ReadOptions(), nullptr, + jkey, jkey_off, jkey_len); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: get + * Signature: (J[BIIJ)[B + */ +jbyteArray Java_org_rocksdb_RocksDB_get__J_3BIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, jlong jcf_handle) { + auto db_handle = reinterpret_cast(jdb_handle); + auto cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + return rocksdb_get_helper(env, db_handle, rocksdb::ReadOptions(), + cf_handle, jkey, jkey_off, jkey_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + return nullptr; + } } /* * Class: org_rocksdb_RocksDB * Method: get - * Signature: (JJ[BI)[B + * Signature: (JJ[BII)[B */ -jbyteArray Java_org_rocksdb_RocksDB_get__JJ_3BI( +jbyteArray Java_org_rocksdb_RocksDB_get__JJ_3BII( JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jropt_handle, - jbyteArray jkey, jint jkey_len) { + jbyteArray jkey, jint jkey_off, jint jkey_len) { return rocksdb_get_helper(env, reinterpret_cast(jdb_handle), - *reinterpret_cast(jropt_handle), - jkey, jkey_len); + *reinterpret_cast(jropt_handle), nullptr, + jkey, jkey_off, jkey_len); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: get + * Signature: (JJ[BIIJ)[B + */ +jbyteArray Java_org_rocksdb_RocksDB_get__JJ_3BIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jropt_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, jlong jcf_handle) { + auto* db_handle = reinterpret_cast(jdb_handle); + auto& ro_opt = *reinterpret_cast(jropt_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + return rocksdb_get_helper(env, db_handle, ro_opt, cf_handle, + jkey, jkey_off, jkey_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + return nullptr; + } } -jint rocksdb_get_helper( - JNIEnv* env, rocksdb::DB* db, const rocksdb::ReadOptions& read_options, - jbyteArray jkey, jint jkey_len, - jbyteArray jvalue, jint jvalue_len) { +jint rocksdb_get_helper(JNIEnv* env, rocksdb::DB* db, + const rocksdb::ReadOptions& read_options, + rocksdb::ColumnFamilyHandle* column_family_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, + jbyteArray jval, jint jval_off, jint jval_len, + bool* has_exception) { static const int kNotFound = -1; static const int kStatusError = -2; - jbyte* key = env->GetByteArrayElements(jkey, 0); - rocksdb::Slice key_slice( - reinterpret_cast(key), jkey_len); + jbyte* key = new jbyte[jkey_len]; + env->GetByteArrayRegion(jkey, jkey_off, jkey_len, key); + if(env->ExceptionCheck()) { + // exception thrown: OutOfMemoryError + delete [] key; + *has_exception = true; + return kStatusError; + } + rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); // TODO(yhchiang): we might save one memory allocation here by adding // a DB::Get() function which takes preallocated jbyte* as input. std::string cvalue; - rocksdb::Status s = db->Get( - read_options, key_slice, &cvalue); + rocksdb::Status s; + if (column_family_handle != nullptr) { + s = db->Get(read_options, column_family_handle, key_slice, &cvalue); + } else { + // backwards compatibility + s = db->Get(read_options, key_slice, &cvalue); + } - // trigger java unref on key. - // by passing JNI_ABORT, it will simply release the reference without - // copying the result back to the java byte array. - env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + // cleanup + delete [] key; if (s.IsNotFound()) { + *has_exception = false; return kNotFound; } else if (!s.ok()) { + *has_exception = true; // Here since we are throwing a Java exception from c++ side. // As a result, c++ does not know calling this function will in fact // throwing an exception. As a result, the execution flow will @@ -227,178 +698,787 @@ jint rocksdb_get_helper( return kStatusError; } - int cvalue_len = static_cast(cvalue.size()); - int length = std::min(jvalue_len, cvalue_len); + const jint cvalue_len = static_cast(cvalue.size()); + const jint length = std::min(jval_len, cvalue_len); + + env->SetByteArrayRegion(jval, jval_off, length, + const_cast(reinterpret_cast(cvalue.c_str()))); + if(env->ExceptionCheck()) { + // exception thrown: OutOfMemoryError + *has_exception = true; + return kStatusError; + } - env->SetByteArrayRegion( - jvalue, 0, length, - reinterpret_cast(cvalue.c_str())); + *has_exception = false; return cvalue_len; } -jobject multi_get_helper(JNIEnv* env, jobject jdb, rocksdb::DB* db, - const rocksdb::ReadOptions& rOpt, jobject jkey_list, jint jkeys_count) { +inline void multi_get_helper_release_keys(JNIEnv* env, + std::vector> &keys_to_free) { + auto end = keys_to_free.end(); + for (auto it = keys_to_free.begin(); it != end; ++it) { + delete [] it->first; + env->DeleteLocalRef(it->second); + } + keys_to_free.clear(); +} + +/** + * cf multi get + * + * @return byte[][] of values or nullptr if an exception occurs + */ +jobjectArray multi_get_helper(JNIEnv* env, jobject jdb, rocksdb::DB* db, + const rocksdb::ReadOptions& rOpt, jobjectArray jkeys, + jintArray jkey_offs, jintArray jkey_lens, + jlongArray jcolumn_family_handles) { + std::vector cf_handles; + if (jcolumn_family_handles != nullptr) { + const jsize len_cols = env->GetArrayLength(jcolumn_family_handles); + + jlong* jcfh = env->GetLongArrayElements(jcolumn_family_handles, nullptr); + if(jcfh == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + for (jsize i = 0; i < len_cols; i++) { + auto* cf_handle = + reinterpret_cast(jcfh[i]); + cf_handles.push_back(cf_handle); + } + env->ReleaseLongArrayElements(jcolumn_family_handles, jcfh, JNI_ABORT); + } + + const jsize len_keys = env->GetArrayLength(jkeys); + if (env->EnsureLocalCapacity(len_keys) != 0) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + jint* jkey_off = env->GetIntArrayElements(jkey_offs, nullptr); + if(jkey_off == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + jint* jkey_len = env->GetIntArrayElements(jkey_lens, nullptr); + if(jkey_len == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseIntArrayElements(jkey_offs, jkey_off, JNI_ABORT); + return nullptr; + } + std::vector keys; - std::vector keys_to_free; - - // get iterator - jobject iteratorObj = env->CallObjectMethod( - jkey_list, rocksdb::ListJni::getIteratorMethod(env)); - - // iterate over keys and convert java byte array to slice - while(env->CallBooleanMethod( - iteratorObj, rocksdb::ListJni::getHasNextMethod(env)) == JNI_TRUE) { - jbyteArray jkey = (jbyteArray) env->CallObjectMethod( - iteratorObj, rocksdb::ListJni::getNextMethod(env)); - jint key_length = env->GetArrayLength(jkey); - - jbyte* key = new jbyte[key_length]; - env->GetByteArrayRegion(jkey, 0, key_length, key); - // store allocated jbyte to free it after multiGet call - keys_to_free.push_back(key); - - rocksdb::Slice key_slice( - reinterpret_cast(key), key_length); + std::vector> keys_to_free; + for (jsize i = 0; i < len_keys; i++) { + jobject jkey = env->GetObjectArrayElement(jkeys, i); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->ReleaseIntArrayElements(jkey_lens, jkey_len, JNI_ABORT); + env->ReleaseIntArrayElements(jkey_offs, jkey_off, JNI_ABORT); + multi_get_helper_release_keys(env, keys_to_free); + return nullptr; + } + + jbyteArray jkey_ba = reinterpret_cast(jkey); + + const jint len_key = jkey_len[i]; + jbyte* key = new jbyte[len_key]; + env->GetByteArrayRegion(jkey_ba, jkey_off[i], len_key, key); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + delete [] key; + env->DeleteLocalRef(jkey); + env->ReleaseIntArrayElements(jkey_lens, jkey_len, JNI_ABORT); + env->ReleaseIntArrayElements(jkey_offs, jkey_off, JNI_ABORT); + multi_get_helper_release_keys(env, keys_to_free); + return nullptr; + } + + rocksdb::Slice key_slice(reinterpret_cast(key), len_key); keys.push_back(key_slice); + + keys_to_free.push_back(std::pair(key, jkey)); } + // cleanup jkey_off and jken_len + env->ReleaseIntArrayElements(jkey_lens, jkey_len, JNI_ABORT); + env->ReleaseIntArrayElements(jkey_offs, jkey_off, JNI_ABORT); + std::vector values; - std::vector s = db->MultiGet(rOpt, keys, &values); - - // Don't reuse class pointer - jclass jclazz = env->FindClass("java/util/ArrayList"); - jmethodID mid = rocksdb::ListJni::getArrayListConstructorMethodId( - env, jclazz); - jobject jvalue_list = env->NewObject(jclazz, mid, jkeys_count); - - // insert in java list - for(std::vector::size_type i = 0; i != s.size(); i++) { - if(s[i].ok()) { - jbyteArray jvalue = env->NewByteArray(values[i].size()); - env->SetByteArrayRegion( - jvalue, 0, values[i].size(), - reinterpret_cast(values[i].c_str())); - env->CallBooleanMethod( - jvalue_list, rocksdb::ListJni::getListAddMethodId(env), jvalue); - } - else { - env->CallBooleanMethod( - jvalue_list, rocksdb::ListJni::getListAddMethodId(env), nullptr); - } + std::vector s; + if (cf_handles.size() == 0) { + s = db->MultiGet(rOpt, keys, &values); + } else { + s = db->MultiGet(rOpt, cf_handles, keys, &values); } // free up allocated byte arrays - for(std::vector::size_type i = 0; i != keys_to_free.size(); i++) { - delete[] keys_to_free[i]; + multi_get_helper_release_keys(env, keys_to_free); + + // prepare the results + jobjectArray jresults = + rocksdb::ByteJni::new2dByteArray(env, static_cast(s.size())); + if(jresults == nullptr) { + // exception occurred + return nullptr; + } + + // TODO(AR) it is not clear to me why EnsureLocalCapacity is needed for the + // loop as we cleanup references with env->DeleteLocalRef(jentry_value); + if (env->EnsureLocalCapacity(static_cast(s.size())) != 0) { + // exception thrown: OutOfMemoryError + return nullptr; + } + // add to the jresults + for (std::vector::size_type i = 0; i != s.size(); i++) { + if (s[i].ok()) { + std::string* value = &values[i]; + const jsize jvalue_len = static_cast(value->size()); + jbyteArray jentry_value = env->NewByteArray(jvalue_len); + if(jentry_value == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + env->SetByteArrayRegion(jentry_value, 0, static_cast(jvalue_len), + const_cast(reinterpret_cast(value->c_str()))); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jentry_value); + return nullptr; + } + + env->SetObjectArrayElement(jresults, static_cast(i), jentry_value); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jentry_value); + return nullptr; + } + + env->DeleteLocalRef(jentry_value); + } } - keys_to_free.clear(); - return jvalue_list; + return jresults; } /* * Class: org_rocksdb_RocksDB * Method: multiGet - * Signature: (JLjava/util/List;I)Ljava/util/List; + * Signature: (J[[B[I[I)[[B */ -jobject Java_org_rocksdb_RocksDB_multiGet__JLjava_util_List_2I( - JNIEnv* env, jobject jdb, jlong jdb_handle, - jobject jkey_list, jint jkeys_count) { +jobjectArray Java_org_rocksdb_RocksDB_multiGet__J_3_3B_3I_3I( + JNIEnv* env, jobject jdb, jlong jdb_handle, jobjectArray jkeys, + jintArray jkey_offs, jintArray jkey_lens) { return multi_get_helper(env, jdb, reinterpret_cast(jdb_handle), - rocksdb::ReadOptions(), jkey_list, jkeys_count); + rocksdb::ReadOptions(), jkeys, jkey_offs, jkey_lens, nullptr); } /* * Class: org_rocksdb_RocksDB * Method: multiGet - * Signature: (JJLjava/util/List;I)Ljava/util/List; + * Signature: (J[[B[I[I[J)[[B */ -jobject Java_org_rocksdb_RocksDB_multiGet__JJLjava_util_List_2I( - JNIEnv* env, jobject jdb, jlong jdb_handle, - jlong jropt_handle, jobject jkey_list, jint jkeys_count) { +jobjectArray Java_org_rocksdb_RocksDB_multiGet__J_3_3B_3I_3I_3J( + JNIEnv* env, jobject jdb, jlong jdb_handle, jobjectArray jkeys, + jintArray jkey_offs, jintArray jkey_lens, + jlongArray jcolumn_family_handles) { return multi_get_helper(env, jdb, reinterpret_cast(jdb_handle), - *reinterpret_cast(jropt_handle), jkey_list, - jkeys_count); + rocksdb::ReadOptions(), jkeys, jkey_offs, jkey_lens, + jcolumn_family_handles); } /* * Class: org_rocksdb_RocksDB - * Method: get - * Signature: (J[BI[BI)I + * Method: multiGet + * Signature: (JJ[[B[I[I)[[B */ -jint Java_org_rocksdb_RocksDB_get__J_3BI_3BI( - JNIEnv* env, jobject jdb, jlong jdb_handle, - jbyteArray jkey, jint jkey_len, - jbyteArray jvalue, jint jvalue_len) { - return rocksdb_get_helper(env, - reinterpret_cast(jdb_handle), - rocksdb::ReadOptions(), - jkey, jkey_len, jvalue, jvalue_len); +jobjectArray Java_org_rocksdb_RocksDB_multiGet__JJ_3_3B_3I_3I( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jropt_handle, + jobjectArray jkeys, jintArray jkey_offs, jintArray jkey_lens) { + return multi_get_helper(env, jdb, reinterpret_cast(jdb_handle), + *reinterpret_cast(jropt_handle), jkeys, jkey_offs, + jkey_lens, nullptr); } /* * Class: org_rocksdb_RocksDB - * Method: get - * Signature: (JJ[BI[BI)I + * Method: multiGet + * Signature: (JJ[[B[I[I[J)[[B */ -jint Java_org_rocksdb_RocksDB_get__JJ_3BI_3BI( +jobjectArray Java_org_rocksdb_RocksDB_multiGet__JJ_3_3B_3I_3I_3J( JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jropt_handle, - jbyteArray jkey, jint jkey_len, - jbyteArray jvalue, jint jvalue_len) { - return rocksdb_get_helper(env, - reinterpret_cast(jdb_handle), - *reinterpret_cast(jropt_handle), - jkey, jkey_len, jvalue, jvalue_len); + jobjectArray jkeys, jintArray jkey_offs, jintArray jkey_lens, + jlongArray jcolumn_family_handles) { + return multi_get_helper(env, jdb, reinterpret_cast(jdb_handle), + *reinterpret_cast(jropt_handle), jkeys, jkey_offs, + jkey_lens, jcolumn_family_handles); } -////////////////////////////////////////////////////////////////////////////// -// rocksdb::DB::Delete() -void rocksdb_remove_helper( +/* + * Class: org_rocksdb_RocksDB + * Method: get + * Signature: (J[BII[BII)I + */ +jint Java_org_rocksdb_RocksDB_get__J_3BII_3BII(JNIEnv* env, jobject jdb, + jlong jdb_handle, + jbyteArray jkey, jint jkey_off, + jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len) { + bool has_exception = false; + return rocksdb_get_helper(env, reinterpret_cast(jdb_handle), + rocksdb::ReadOptions(), nullptr, jkey, jkey_off, + jkey_len, jval, jval_off, jval_len, + &has_exception); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: get + * Signature: (J[BII[BIIJ)I + */ +jint Java_org_rocksdb_RocksDB_get__J_3BII_3BIIJ(JNIEnv* env, jobject jdb, + jlong jdb_handle, + jbyteArray jkey, jint jkey_off, + jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len, + jlong jcf_handle) { + auto* db_handle = reinterpret_cast(jdb_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + bool has_exception = false; + return rocksdb_get_helper(env, db_handle, rocksdb::ReadOptions(), cf_handle, + jkey, jkey_off, jkey_len, jval, jval_off, + jval_len, &has_exception); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + // will never be evaluated + return 0; + } +} + +/* + * Class: org_rocksdb_RocksDB + * Method: get + * Signature: (JJ[BII[BII)I + */ +jint Java_org_rocksdb_RocksDB_get__JJ_3BII_3BII(JNIEnv* env, jobject jdb, + jlong jdb_handle, + jlong jropt_handle, + jbyteArray jkey, jint jkey_off, + jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len) { + bool has_exception = false; + return rocksdb_get_helper( + env, reinterpret_cast(jdb_handle), + *reinterpret_cast(jropt_handle), nullptr, jkey, + jkey_off, jkey_len, jval, jval_off, jval_len, &has_exception); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: get + * Signature: (JJ[BII[BIIJ)I + */ +jint Java_org_rocksdb_RocksDB_get__JJ_3BII_3BIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jropt_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len, jlong jcf_handle) { + auto* db_handle = reinterpret_cast(jdb_handle); + auto& ro_opt = *reinterpret_cast(jropt_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + bool has_exception = false; + return rocksdb_get_helper(env, db_handle, ro_opt, cf_handle, jkey, jkey_off, + jkey_len, jval, jval_off, jval_len, + &has_exception); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + // will never be evaluated + return 0; + } +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::Delete() + +/** + * @return true if the delete succeeded, false if a Java Exception was thrown + */ +bool rocksdb_delete_helper( JNIEnv* env, rocksdb::DB* db, const rocksdb::WriteOptions& write_options, - jbyteArray jkey, jint jkey_len) { - jbyte* key = env->GetByteArrayElements(jkey, 0); + rocksdb::ColumnFamilyHandle* cf_handle, jbyteArray jkey, jint jkey_off, + jint jkey_len) { + jbyte* key = new jbyte[jkey_len]; + env->GetByteArrayRegion(jkey, jkey_off, jkey_len, key); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + delete [] key; + return false; + } rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); - rocksdb::Status s = db->Delete(write_options, key_slice); + rocksdb::Status s; + if (cf_handle != nullptr) { + s = db->Delete(write_options, cf_handle, key_slice); + } else { + // backwards compatibility + s = db->Delete(write_options, key_slice); + } + + // cleanup + delete [] key; + + if (s.ok()) { + return true; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return false; +} + +/* + * Class: org_rocksdb_RocksDB + * Method: delete + * Signature: (J[BII)V + */ +void Java_org_rocksdb_RocksDB_delete__J_3BII( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len) { + auto* db = reinterpret_cast(jdb_handle); + static const rocksdb::WriteOptions default_write_options = + rocksdb::WriteOptions(); + rocksdb_delete_helper(env, db, default_write_options, nullptr, + jkey, jkey_off, jkey_len); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: delete + * Signature: (J[BIIJ)V + */ +void Java_org_rocksdb_RocksDB_delete__J_3BIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + static const rocksdb::WriteOptions default_write_options = + rocksdb::WriteOptions(); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + rocksdb_delete_helper(env, db, default_write_options, cf_handle, + jkey, jkey_off, jkey_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + } +} + +/* + * Class: org_rocksdb_RocksDB + * Method: delete + * Signature: (JJ[BII)V + */ +void Java_org_rocksdb_RocksDB_delete__JJ_3BII( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jlong jwrite_options, jbyteArray jkey, jint jkey_off, jint jkey_len) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = + reinterpret_cast(jwrite_options); + rocksdb_delete_helper(env, db, *write_options, nullptr, jkey, jkey_off, + jkey_len); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: delete + * Signature: (JJ[BIIJ)V + */ +void Java_org_rocksdb_RocksDB_delete__JJ_3BIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jlong jwrite_options, jbyteArray jkey, jint jkey_off, jint jkey_len, + jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = + reinterpret_cast(jwrite_options); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + rocksdb_delete_helper(env, db, *write_options, cf_handle, jkey, jkey_off, + jkey_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + } +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::SingleDelete() +/** + * @return true if the single delete succeeded, false if a Java Exception + * was thrown + */ +bool rocksdb_single_delete_helper( + JNIEnv* env, rocksdb::DB* db, const rocksdb::WriteOptions& write_options, + rocksdb::ColumnFamilyHandle* cf_handle, jbyteArray jkey, jint jkey_len) { + jbyte* key = env->GetByteArrayElements(jkey, nullptr); + if(key == nullptr) { + // exception thrown: OutOfMemoryError + return false; + } + rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); + + rocksdb::Status s; + if (cf_handle != nullptr) { + s = db->SingleDelete(write_options, cf_handle, key_slice); + } else { + // backwards compatibility + s = db->SingleDelete(write_options, key_slice); + } // trigger java unref on key and value. // by passing JNI_ABORT, it will simply release the reference without // copying the result back to the java byte array. env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); - if (!s.ok()) { - rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + if (s.ok()) { + return true; } - return; + + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return false; } /* * Class: org_rocksdb_RocksDB - * Method: remove + * Method: singleDelete * Signature: (J[BI)V */ -void Java_org_rocksdb_RocksDB_remove__J_3BI( +void Java_org_rocksdb_RocksDB_singleDelete__J_3BI( JNIEnv* env, jobject jdb, jlong jdb_handle, jbyteArray jkey, jint jkey_len) { - auto db = reinterpret_cast(jdb_handle); + auto* db = reinterpret_cast(jdb_handle); static const rocksdb::WriteOptions default_write_options = rocksdb::WriteOptions(); + rocksdb_single_delete_helper(env, db, default_write_options, nullptr, + jkey, jkey_len); +} - rocksdb_remove_helper(env, db, default_write_options, jkey, jkey_len); +/* + * Class: org_rocksdb_RocksDB + * Method: singleDelete + * Signature: (J[BIJ)V + */ +void Java_org_rocksdb_RocksDB_singleDelete__J_3BIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jbyteArray jkey, jint jkey_len, jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + static const rocksdb::WriteOptions default_write_options = + rocksdb::WriteOptions(); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + rocksdb_single_delete_helper(env, db, default_write_options, cf_handle, + jkey, jkey_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + } } /* * Class: org_rocksdb_RocksDB - * Method: remove - * Signature: (JJ[BI)V + * Method: singleDelete + * Signature: (JJ[BIJ)V */ -void Java_org_rocksdb_RocksDB_remove__JJ_3BI( +void Java_org_rocksdb_RocksDB_singleDelete__JJ_3BI( JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jwrite_options, jbyteArray jkey, jint jkey_len) { - auto db = reinterpret_cast(jdb_handle); - auto write_options = reinterpret_cast(jwrite_options); + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = + reinterpret_cast(jwrite_options); + rocksdb_single_delete_helper(env, db, *write_options, nullptr, jkey, + jkey_len); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: singleDelete + * Signature: (JJ[BIJ)V + */ +void Java_org_rocksdb_RocksDB_singleDelete__JJ_3BIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jlong jwrite_options, jbyteArray jkey, jint jkey_len, + jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = + reinterpret_cast(jwrite_options); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + rocksdb_single_delete_helper(env, db, *write_options, cf_handle, jkey, + jkey_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + } +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::DeleteRange() +/** + * @return true if the delete range succeeded, false if a Java Exception + * was thrown + */ +bool rocksdb_delete_range_helper(JNIEnv* env, rocksdb::DB* db, + const rocksdb::WriteOptions& write_options, + rocksdb::ColumnFamilyHandle* cf_handle, + jbyteArray jbegin_key, jint jbegin_key_off, + jint jbegin_key_len, jbyteArray jend_key, + jint jend_key_off, jint jend_key_len) { + jbyte* begin_key = new jbyte[jbegin_key_len]; + env->GetByteArrayRegion(jbegin_key, jbegin_key_off, jbegin_key_len, + begin_key); + if (env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + delete[] begin_key; + return false; + } + rocksdb::Slice begin_key_slice(reinterpret_cast(begin_key), + jbegin_key_len); + + jbyte* end_key = new jbyte[jend_key_len]; + env->GetByteArrayRegion(jend_key, jend_key_off, jend_key_len, end_key); + if (env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + delete[] begin_key; + delete[] end_key; + return false; + } + rocksdb::Slice end_key_slice(reinterpret_cast(end_key), jend_key_len); + + rocksdb::Status s = + db->DeleteRange(write_options, cf_handle, begin_key_slice, end_key_slice); + + // cleanup + delete[] begin_key; + delete[] end_key; + + if (s.ok()) { + return true; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return false; +} + +/* + * Class: org_rocksdb_RocksDB + * Method: deleteRange + * Signature: (J[BII[BII)V + */ +void Java_org_rocksdb_RocksDB_deleteRange__J_3BII_3BII( + JNIEnv* env, jobject jdb, jlong jdb_handle, jbyteArray jbegin_key, + jint jbegin_key_off, jint jbegin_key_len, jbyteArray jend_key, + jint jend_key_off, jint jend_key_len) { + auto* db = reinterpret_cast(jdb_handle); + static const rocksdb::WriteOptions default_write_options = + rocksdb::WriteOptions(); + rocksdb_delete_range_helper(env, db, default_write_options, nullptr, + jbegin_key, jbegin_key_off, jbegin_key_len, + jend_key, jend_key_off, jend_key_len); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: deleteRange + * Signature: (J[BII[BIIJ)V + */ +void Java_org_rocksdb_RocksDB_deleteRange__J_3BII_3BIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, jbyteArray jbegin_key, + jint jbegin_key_off, jint jbegin_key_len, jbyteArray jend_key, + jint jend_key_off, jint jend_key_len, jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + static const rocksdb::WriteOptions default_write_options = + rocksdb::WriteOptions(); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + rocksdb_delete_range_helper(env, db, default_write_options, cf_handle, + jbegin_key, jbegin_key_off, jbegin_key_len, + jend_key, jend_key_off, jend_key_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew( + env, rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + } +} + +/* + * Class: org_rocksdb_RocksDB + * Method: deleteRange + * Signature: (JJ[BII[BII)V + */ +void Java_org_rocksdb_RocksDB_deleteRange__JJ_3BII_3BII( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jwrite_options, + jbyteArray jbegin_key, jint jbegin_key_off, jint jbegin_key_len, + jbyteArray jend_key, jint jend_key_off, jint jend_key_len) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = + reinterpret_cast(jwrite_options); + rocksdb_delete_range_helper(env, db, *write_options, nullptr, jbegin_key, + jbegin_key_off, jbegin_key_len, jend_key, + jend_key_off, jend_key_len); +} - rocksdb_remove_helper(env, db, *write_options, jkey, jkey_len); +/* + * Class: org_rocksdb_RocksDB + * Method: deleteRange + * Signature: (JJ[BII[BIIJ)V + */ +void Java_org_rocksdb_RocksDB_deleteRange__JJ_3BII_3BIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jwrite_options, + jbyteArray jbegin_key, jint jbegin_key_off, jint jbegin_key_len, + jbyteArray jend_key, jint jend_key_off, jint jend_key_len, + jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = + reinterpret_cast(jwrite_options); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + rocksdb_delete_range_helper(env, db, *write_options, cf_handle, jbegin_key, + jbegin_key_off, jbegin_key_len, jend_key, + jend_key_off, jend_key_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew( + env, rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + } +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::Merge + +/** + * @return true if the merge succeeded, false if a Java Exception was thrown + */ +bool rocksdb_merge_helper(JNIEnv* env, rocksdb::DB* db, + const rocksdb::WriteOptions& write_options, + rocksdb::ColumnFamilyHandle* cf_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, + jbyteArray jval, jint jval_off, jint jval_len) { + jbyte* key = new jbyte[jkey_len]; + env->GetByteArrayRegion(jkey, jkey_off, jkey_len, key); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + delete [] key; + return false; + } + rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); + + jbyte* value = new jbyte[jval_len]; + env->GetByteArrayRegion(jval, jval_off, jval_len, value); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + delete [] value; + delete [] key; + return false; + } + rocksdb::Slice value_slice(reinterpret_cast(value), jval_len); + + rocksdb::Status s; + if (cf_handle != nullptr) { + s = db->Merge(write_options, cf_handle, key_slice, value_slice); + } else { + s = db->Merge(write_options, key_slice, value_slice); + } + + // cleanup + delete [] value; + delete [] key; + + if (s.ok()) { + return true; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return false; +} + +/* + * Class: org_rocksdb_RocksDB + * Method: merge + * Signature: (J[BII[BII)V + */ +void Java_org_rocksdb_RocksDB_merge__J_3BII_3BII(JNIEnv* env, jobject jdb, + jlong jdb_handle, + jbyteArray jkey, jint jkey_off, + jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len) { + auto* db = reinterpret_cast(jdb_handle); + static const rocksdb::WriteOptions default_write_options = + rocksdb::WriteOptions(); + + rocksdb_merge_helper(env, db, default_write_options, nullptr, jkey, jkey_off, + jkey_len, jval, jval_off, jval_len); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: merge + * Signature: (J[BII[BIIJ)V + */ +void Java_org_rocksdb_RocksDB_merge__J_3BII_3BIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, jbyteArray jkey, jint jkey_off, + jint jkey_len, jbyteArray jval, jint jval_off, jint jval_len, + jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + static const rocksdb::WriteOptions default_write_options = + rocksdb::WriteOptions(); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + rocksdb_merge_helper(env, db, default_write_options, cf_handle, jkey, + jkey_off, jkey_len, jval, jval_off, jval_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + } +} + +/* + * Class: org_rocksdb_RocksDB + * Method: merge + * Signature: (JJ[BII[BII)V + */ +void Java_org_rocksdb_RocksDB_merge__JJ_3BII_3BII( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jwrite_options_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = + reinterpret_cast(jwrite_options_handle); + + rocksdb_merge_helper(env, db, *write_options, nullptr, jkey, jkey_off, + jkey_len, jval, jval_off, jval_len); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: merge + * Signature: (JJ[BII[BIIJ)V + */ +void Java_org_rocksdb_RocksDB_merge__JJ_3BII_3BIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jwrite_options_handle, + jbyteArray jkey, jint jkey_off, jint jkey_len, jbyteArray jval, + jint jval_off, jint jval_len, jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto* write_options = + reinterpret_cast(jwrite_options_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + if (cf_handle != nullptr) { + rocksdb_merge_helper(env, db, *write_options, cf_handle, jkey, jkey_off, + jkey_len, jval, jval_off, jval_len); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument("Invalid ColumnFamilyHandle.")); + } } ////////////////////////////////////////////////////////////////////////////// @@ -411,17 +1491,708 @@ void Java_org_rocksdb_RocksDB_remove__JJ_3BI( */ void Java_org_rocksdb_RocksDB_disposeInternal( JNIEnv* env, jobject java_db, jlong jhandle) { - delete reinterpret_cast(jhandle); + auto* db = reinterpret_cast(jhandle); + assert(db != nullptr); + delete db; +} + +jlong rocksdb_iterator_helper( + rocksdb::DB* db, rocksdb::ReadOptions read_options, + rocksdb::ColumnFamilyHandle* cf_handle) { + rocksdb::Iterator* iterator = nullptr; + if (cf_handle != nullptr) { + iterator = db->NewIterator(read_options, cf_handle); + } else { + iterator = db->NewIterator(read_options); + } + return reinterpret_cast(iterator); } /* * Class: org_rocksdb_RocksDB - * Method: iterator0 + * Method: iterator * Signature: (J)J */ -jlong Java_org_rocksdb_RocksDB_iterator0( +jlong Java_org_rocksdb_RocksDB_iterator__J( JNIEnv* env, jobject jdb, jlong db_handle) { - auto db = reinterpret_cast(db_handle); - rocksdb::Iterator* iterator = db->NewIterator(rocksdb::ReadOptions()); - return reinterpret_cast(iterator); + auto* db = reinterpret_cast(db_handle); + return rocksdb_iterator_helper(db, rocksdb::ReadOptions(), + nullptr); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: iterator + * Signature: (JJ)J + */ +jlong Java_org_rocksdb_RocksDB_iterator__JJ( + JNIEnv* env, jobject jdb, jlong db_handle, + jlong jread_options_handle) { + auto* db = reinterpret_cast(db_handle); + auto& read_options = *reinterpret_cast( + jread_options_handle); + return rocksdb_iterator_helper(db, read_options, + nullptr); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: iteratorCF + * Signature: (JJ)J + */ +jlong Java_org_rocksdb_RocksDB_iteratorCF__JJ( + JNIEnv* env, jobject jdb, jlong db_handle, jlong jcf_handle) { + auto* db = reinterpret_cast(db_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + return rocksdb_iterator_helper(db, rocksdb::ReadOptions(), + cf_handle); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: iteratorCF + * Signature: (JJJ)J + */ +jlong Java_org_rocksdb_RocksDB_iteratorCF__JJJ( + JNIEnv* env, jobject jdb, jlong db_handle, jlong jcf_handle, + jlong jread_options_handle) { + auto* db = reinterpret_cast(db_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + auto& read_options = *reinterpret_cast( + jread_options_handle); + return rocksdb_iterator_helper(db, read_options, + cf_handle); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: iterators + * Signature: (J[JJ)[J + */ +jlongArray Java_org_rocksdb_RocksDB_iterators( + JNIEnv* env, jobject jdb, jlong db_handle, + jlongArray jcolumn_family_handles, jlong jread_options_handle) { + auto* db = reinterpret_cast(db_handle); + auto& read_options = *reinterpret_cast( + jread_options_handle); + + std::vector cf_handles; + if (jcolumn_family_handles != nullptr) { + const jsize len_cols = env->GetArrayLength(jcolumn_family_handles); + jlong* jcfh = env->GetLongArrayElements(jcolumn_family_handles, nullptr); + if(jcfh == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + for (jsize i = 0; i < len_cols; i++) { + auto* cf_handle = + reinterpret_cast(jcfh[i]); + cf_handles.push_back(cf_handle); + } + + env->ReleaseLongArrayElements(jcolumn_family_handles, jcfh, JNI_ABORT); + } + + std::vector iterators; + rocksdb::Status s = db->NewIterators(read_options, + cf_handles, &iterators); + if (s.ok()) { + jlongArray jLongArray = + env->NewLongArray(static_cast(iterators.size())); + if(jLongArray == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + for (std::vector::size_type i = 0; + i < iterators.size(); i++) { + env->SetLongArrayRegion(jLongArray, static_cast(i), 1, + const_cast(reinterpret_cast(&iterators[i]))); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jLongArray); + return nullptr; + } + } + + return jLongArray; + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return nullptr; + } +} + +/* + * Class: org_rocksdb_RocksDB + * Method: getDefaultColumnFamily + * Signature: (J)J + */ +jlong Java_org_rocksdb_RocksDB_getDefaultColumnFamily( + JNIEnv* env, jobject jobj, jlong jdb_handle) { + auto* db_handle = reinterpret_cast(jdb_handle); + auto* cf_handle = db_handle->DefaultColumnFamily(); + return reinterpret_cast(cf_handle); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: createColumnFamily + * Signature: (J[BJ)J + */ +jlong Java_org_rocksdb_RocksDB_createColumnFamily( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jbyteArray jcolumn_name, jlong jcolumn_options) { + rocksdb::ColumnFamilyHandle* handle; + jboolean has_exception = JNI_FALSE; + std::string column_name = rocksdb::JniUtil::byteString(env, + jcolumn_name, + [](const char* str, const size_t len) { return std::string(str, len); }, + &has_exception); + if(has_exception == JNI_TRUE) { + // exception occurred + return 0; + } + + auto* db_handle = reinterpret_cast(jdb_handle); + auto* cfOptions = + reinterpret_cast(jcolumn_options); + + rocksdb::Status s = db_handle->CreateColumnFamily( + *cfOptions, column_name, &handle); + + if (s.ok()) { + return reinterpret_cast(handle); + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return 0; +} + +/* + * Class: org_rocksdb_RocksDB + * Method: dropColumnFamily + * Signature: (JJ)V; + */ +void Java_org_rocksdb_RocksDB_dropColumnFamily( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jcf_handle) { + auto* cf_handle = reinterpret_cast(jcf_handle); + auto* db_handle = reinterpret_cast(jdb_handle); + rocksdb::Status s = db_handle->DropColumnFamily(cf_handle); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Method: getSnapshot + * Signature: (J)J + */ +jlong Java_org_rocksdb_RocksDB_getSnapshot( + JNIEnv* env, jobject jdb, jlong db_handle) { + auto* db = reinterpret_cast(db_handle); + const rocksdb::Snapshot* snapshot = db->GetSnapshot(); + return reinterpret_cast(snapshot); +} + +/* + * Method: releaseSnapshot + * Signature: (JJ)V + */ +void Java_org_rocksdb_RocksDB_releaseSnapshot( + JNIEnv* env, jobject jdb, jlong db_handle, jlong snapshot_handle) { + auto* db = reinterpret_cast(db_handle); + auto* snapshot = reinterpret_cast(snapshot_handle); + db->ReleaseSnapshot(snapshot); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: getProperty0 + * Signature: (JLjava/lang/String;I)Ljava/lang/String; + */ +jstring Java_org_rocksdb_RocksDB_getProperty0__JLjava_lang_String_2I( + JNIEnv* env, jobject jdb, jlong db_handle, jstring jproperty, + jint jproperty_len) { + const char* property = env->GetStringUTFChars(jproperty, nullptr); + if(property == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + rocksdb::Slice property_slice(property, jproperty_len); + + auto *db = reinterpret_cast(db_handle); + std::string property_value; + bool retCode = db->GetProperty(property_slice, &property_value); + env->ReleaseStringUTFChars(jproperty, property); + + if (retCode) { + return env->NewStringUTF(property_value.c_str()); + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, rocksdb::Status::NotFound()); + return nullptr; +} + +/* + * Class: org_rocksdb_RocksDB + * Method: getProperty0 + * Signature: (JJLjava/lang/String;I)Ljava/lang/String; + */ +jstring Java_org_rocksdb_RocksDB_getProperty0__JJLjava_lang_String_2I( + JNIEnv* env, jobject jdb, jlong db_handle, jlong jcf_handle, + jstring jproperty, jint jproperty_len) { + const char* property = env->GetStringUTFChars(jproperty, nullptr); + if(property == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + rocksdb::Slice property_slice(property, jproperty_len); + + auto* db = reinterpret_cast(db_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + std::string property_value; + bool retCode = db->GetProperty(cf_handle, property_slice, &property_value); + env->ReleaseStringUTFChars(jproperty, property); + + if (retCode) { + return env->NewStringUTF(property_value.c_str()); + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, rocksdb::Status::NotFound()); + return nullptr; +} + +/* + * Class: org_rocksdb_RocksDB + * Method: getLongProperty + * Signature: (JLjava/lang/String;I)L; + */ +jlong Java_org_rocksdb_RocksDB_getLongProperty__JLjava_lang_String_2I( + JNIEnv* env, jobject jdb, jlong db_handle, jstring jproperty, + jint jproperty_len) { + const char* property = env->GetStringUTFChars(jproperty, nullptr); + if(property == nullptr) { + // exception thrown: OutOfMemoryError + return 0; + } + rocksdb::Slice property_slice(property, jproperty_len); + + auto* db = reinterpret_cast(db_handle); + uint64_t property_value = 0; + bool retCode = db->GetIntProperty(property_slice, &property_value); + env->ReleaseStringUTFChars(jproperty, property); + + if (retCode) { + return property_value; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, rocksdb::Status::NotFound()); + return 0; +} + +/* + * Class: org_rocksdb_RocksDB + * Method: getLongProperty + * Signature: (JJLjava/lang/String;I)L; + */ +jlong Java_org_rocksdb_RocksDB_getLongProperty__JJLjava_lang_String_2I( + JNIEnv* env, jobject jdb, jlong db_handle, jlong jcf_handle, + jstring jproperty, jint jproperty_len) { + const char* property = env->GetStringUTFChars(jproperty, nullptr); + if(property == nullptr) { + // exception thrown: OutOfMemoryError + return 0; + } + rocksdb::Slice property_slice(property, jproperty_len); + + auto* db = reinterpret_cast(db_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + uint64_t property_value; + bool retCode = db->GetIntProperty(cf_handle, property_slice, &property_value); + env->ReleaseStringUTFChars(jproperty, property); + + if (retCode) { + return property_value; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, rocksdb::Status::NotFound()); + return 0; +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::Flush + +void rocksdb_flush_helper( + JNIEnv* env, rocksdb::DB* db, const rocksdb::FlushOptions& flush_options, + rocksdb::ColumnFamilyHandle* column_family_handle) { + rocksdb::Status s; + if (column_family_handle != nullptr) { + s = db->Flush(flush_options, column_family_handle); + } else { + s = db->Flush(flush_options); + } + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_RocksDB + * Method: flush + * Signature: (JJ)V + */ +void Java_org_rocksdb_RocksDB_flush__JJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jlong jflush_options) { + auto* db = reinterpret_cast(jdb_handle); + auto* flush_options = + reinterpret_cast(jflush_options); + rocksdb_flush_helper(env, db, *flush_options, nullptr); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: flush + * Signature: (JJJ)V + */ +void Java_org_rocksdb_RocksDB_flush__JJJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jlong jflush_options, jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto* flush_options = + reinterpret_cast(jflush_options); + auto* cf_handle = reinterpret_cast(jcf_handle); + rocksdb_flush_helper(env, db, *flush_options, cf_handle); +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::CompactRange - Full + +void rocksdb_compactrange_helper(JNIEnv* env, rocksdb::DB* db, + rocksdb::ColumnFamilyHandle* cf_handle, jboolean jreduce_level, + jint jtarget_level, jint jtarget_path_id) { + + rocksdb::Status s; + rocksdb::CompactRangeOptions compact_options; + compact_options.change_level = jreduce_level; + compact_options.target_level = jtarget_level; + compact_options.target_path_id = static_cast(jtarget_path_id); + if (cf_handle != nullptr) { + s = db->CompactRange(compact_options, cf_handle, nullptr, nullptr); + } else { + // backwards compatibility + s = db->CompactRange(compact_options, nullptr, nullptr); + } + + if (s.ok()) { + return; + } + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: compactRange0 + * Signature: (JZII)V + */ +void Java_org_rocksdb_RocksDB_compactRange0__JZII(JNIEnv* env, + jobject jdb, jlong jdb_handle, jboolean jreduce_level, + jint jtarget_level, jint jtarget_path_id) { + auto* db = reinterpret_cast(jdb_handle); + rocksdb_compactrange_helper(env, db, nullptr, jreduce_level, + jtarget_level, jtarget_path_id); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: compactRange + * Signature: (JZIIJ)V + */ +void Java_org_rocksdb_RocksDB_compactRange__JZIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, + jboolean jreduce_level, jint jtarget_level, + jint jtarget_path_id, jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + rocksdb_compactrange_helper(env, db, cf_handle, jreduce_level, + jtarget_level, jtarget_path_id); +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::CompactRange - Range + +/** + * @return true if the compact range succeeded, false if a Java Exception + * was thrown + */ +bool rocksdb_compactrange_helper(JNIEnv* env, rocksdb::DB* db, + rocksdb::ColumnFamilyHandle* cf_handle, jbyteArray jbegin, jint jbegin_len, + jbyteArray jend, jint jend_len, jboolean jreduce_level, jint jtarget_level, + jint jtarget_path_id) { + + jbyte* begin = env->GetByteArrayElements(jbegin, nullptr); + if(begin == nullptr) { + // exception thrown: OutOfMemoryError + return false; + } + + jbyte* end = env->GetByteArrayElements(jend, nullptr); + if(end == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseByteArrayElements(jbegin, begin, JNI_ABORT); + return false; + } + + const rocksdb::Slice begin_slice(reinterpret_cast(begin), jbegin_len); + const rocksdb::Slice end_slice(reinterpret_cast(end), jend_len); + + rocksdb::Status s; + rocksdb::CompactRangeOptions compact_options; + compact_options.change_level = jreduce_level; + compact_options.target_level = jtarget_level; + compact_options.target_path_id = static_cast(jtarget_path_id); + if (cf_handle != nullptr) { + s = db->CompactRange(compact_options, cf_handle, &begin_slice, &end_slice); + } else { + // backwards compatibility + s = db->CompactRange(compact_options, &begin_slice, &end_slice); + } + + env->ReleaseByteArrayElements(jend, end, JNI_ABORT); + env->ReleaseByteArrayElements(jbegin, begin, JNI_ABORT); + + if (s.ok()) { + return true; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return false; +} + +/* + * Class: org_rocksdb_RocksDB + * Method: compactRange0 + * Signature: (J[BI[BIZII)V + */ +void Java_org_rocksdb_RocksDB_compactRange0__J_3BI_3BIZII(JNIEnv* env, + jobject jdb, jlong jdb_handle, jbyteArray jbegin, jint jbegin_len, + jbyteArray jend, jint jend_len, jboolean jreduce_level, + jint jtarget_level, jint jtarget_path_id) { + auto* db = reinterpret_cast(jdb_handle); + rocksdb_compactrange_helper(env, db, nullptr, jbegin, jbegin_len, + jend, jend_len, jreduce_level, jtarget_level, jtarget_path_id); +} + +/* + * Class: org_rocksdb_RocksDB + * Method: compactRange + * Signature: (JJ[BI[BIZII)V + */ +void Java_org_rocksdb_RocksDB_compactRange__J_3BI_3BIZIIJ( + JNIEnv* env, jobject jdb, jlong jdb_handle, jbyteArray jbegin, + jint jbegin_len, jbyteArray jend, jint jend_len, + jboolean jreduce_level, jint jtarget_level, + jint jtarget_path_id, jlong jcf_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + rocksdb_compactrange_helper(env, db, cf_handle, jbegin, jbegin_len, + jend, jend_len, jreduce_level, jtarget_level, jtarget_path_id); +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::PauseBackgroundWork + +/* + * Class: org_rocksdb_RocksDB + * Method: pauseBackgroundWork + * Signature: (J)V + */ +void Java_org_rocksdb_RocksDB_pauseBackgroundWork( + JNIEnv* env, jobject jobj, jlong jdb_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto s = db->PauseBackgroundWork(); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::ContinueBackgroundWork + +/* + * Class: org_rocksdb_RocksDB + * Method: continueBackgroundWork + * Signature: (J)V + */ +void Java_org_rocksdb_RocksDB_continueBackgroundWork( + JNIEnv* env, jobject jobj, jlong jdb_handle) { + auto* db = reinterpret_cast(jdb_handle); + auto s = db->ContinueBackgroundWork(); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::GetLatestSequenceNumber + +/* + * Class: org_rocksdb_RocksDB + * Method: getLatestSequenceNumber + * Signature: (J)V + */ +jlong Java_org_rocksdb_RocksDB_getLatestSequenceNumber(JNIEnv* env, + jobject jdb, jlong jdb_handle) { + auto* db = reinterpret_cast(jdb_handle); + return db->GetLatestSequenceNumber(); +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB enable/disable file deletions + +/* + * Class: org_rocksdb_RocksDB + * Method: enableFileDeletions + * Signature: (J)V + */ +void Java_org_rocksdb_RocksDB_disableFileDeletions(JNIEnv* env, + jobject jdb, jlong jdb_handle) { + auto* db = reinterpret_cast(jdb_handle); + rocksdb::Status s = db->DisableFileDeletions(); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_RocksDB + * Method: enableFileDeletions + * Signature: (JZ)V + */ +void Java_org_rocksdb_RocksDB_enableFileDeletions(JNIEnv* env, + jobject jdb, jlong jdb_handle, jboolean jforce) { + auto* db = reinterpret_cast(jdb_handle); + rocksdb::Status s = db->EnableFileDeletions(jforce); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::GetUpdatesSince + +/* + * Class: org_rocksdb_RocksDB + * Method: getUpdatesSince + * Signature: (JJ)J + */ +jlong Java_org_rocksdb_RocksDB_getUpdatesSince(JNIEnv* env, + jobject jdb, jlong jdb_handle, jlong jsequence_number) { + auto* db = reinterpret_cast(jdb_handle); + rocksdb::SequenceNumber sequence_number = + static_cast(jsequence_number); + std::unique_ptr iter; + rocksdb::Status s = db->GetUpdatesSince(sequence_number, &iter); + if (s.ok()) { + return reinterpret_cast(iter.release()); + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return 0; +} + +/* + * Class: org_rocksdb_RocksDB + * Method: setOptions + * Signature: (JJ[Ljava/lang/String;[Ljava/lang/String;)V + */ +void Java_org_rocksdb_RocksDB_setOptions(JNIEnv* env, jobject jdb, + jlong jdb_handle, jlong jcf_handle, jobjectArray jkeys, + jobjectArray jvalues) { + const jsize len = env->GetArrayLength(jkeys); + assert(len == env->GetArrayLength(jvalues)); + + std::unordered_map options_map; + for (jsize i = 0; i < len; i++) { + jobject jobj_key = env->GetObjectArrayElement(jkeys, i); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + return; + } + + jobject jobj_value = env->GetObjectArrayElement(jvalues, i); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jobj_key); + return; + } + + jstring jkey = reinterpret_cast(jobj_key); + jstring jval = reinterpret_cast(jobj_value); + + const char* key = env->GetStringUTFChars(jkey, nullptr); + if(key == nullptr) { + // exception thrown: OutOfMemoryError + env->DeleteLocalRef(jobj_value); + env->DeleteLocalRef(jobj_key); + return; + } + + const char* value = env->GetStringUTFChars(jval, nullptr); + if(value == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseStringUTFChars(jkey, key); + env->DeleteLocalRef(jobj_value); + env->DeleteLocalRef(jobj_key); + return; + } + + std::string s_key(key); + std::string s_value(value); + options_map[s_key] = s_value; + + env->ReleaseStringUTFChars(jkey, key); + env->ReleaseStringUTFChars(jval, value); + env->DeleteLocalRef(jobj_key); + env->DeleteLocalRef(jobj_value); + } + + auto* db = reinterpret_cast(jdb_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + db->SetOptions(cf_handle, options_map); +} + +////////////////////////////////////////////////////////////////////////////// +// rocksdb::DB::IngestExternalFile + +/* + * Class: org_rocksdb_RocksDB + * Method: ingestExternalFile + * Signature: (JJ[Ljava/lang/String;IJ)V + */ +void Java_org_rocksdb_RocksDB_ingestExternalFile( + JNIEnv* env, jobject jdb, jlong jdb_handle, jlong jcf_handle, + jobjectArray jfile_path_list, jint jfile_path_list_len, + jlong jingest_external_file_options_handle) { + jboolean has_exception = JNI_FALSE; + std::vector file_path_list = + rocksdb::JniUtil::copyStrings(env, jfile_path_list, jfile_path_list_len, + &has_exception); + if(has_exception == JNI_TRUE) { + // exception occurred + return; + } + + auto* db = reinterpret_cast(jdb_handle); + auto* column_family = + reinterpret_cast(jcf_handle); + auto* ifo = + reinterpret_cast( + jingest_external_file_options_handle); + rocksdb::Status s = + db->IngestExternalFile(column_family, file_path_list, *ifo); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } } diff --git a/java/rocksjni/slice.cc b/java/rocksjni/slice.cc new file mode 100644 index 00000000000..ef0e384f1a4 --- /dev/null +++ b/java/rocksjni/slice.cc @@ -0,0 +1,349 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ for +// rocksdb::Slice. + +#include +#include +#include +#include + +#include "include/org_rocksdb_AbstractSlice.h" +#include "include/org_rocksdb_Slice.h" +#include "include/org_rocksdb_DirectSlice.h" +#include "rocksdb/slice.h" +#include "rocksjni/portal.h" + +// + +/* + * Class: org_rocksdb_Slice + * Method: createNewSlice0 + * Signature: ([BI)J + */ +jlong Java_org_rocksdb_Slice_createNewSlice0( + JNIEnv * env, jclass jcls, jbyteArray data, jint offset) { + const jsize dataSize = env->GetArrayLength(data); + const int len = dataSize - offset; + + // NOTE: buf will be deleted in the Java_org_rocksdb_Slice_disposeInternalBuf method + jbyte* buf = new jbyte[len]; + env->GetByteArrayRegion(data, offset, len, buf); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + return 0; + } + + const auto* slice = new rocksdb::Slice((const char*)buf, len); + return reinterpret_cast(slice); +} + +/* + * Class: org_rocksdb_Slice + * Method: createNewSlice1 + * Signature: ([B)J + */ +jlong Java_org_rocksdb_Slice_createNewSlice1( + JNIEnv * env, jclass jcls, jbyteArray data) { + jbyte* ptrData = env->GetByteArrayElements(data, nullptr); + if(ptrData == nullptr) { + // exception thrown: OutOfMemoryError + return 0; + } + const int len = env->GetArrayLength(data) + 1; + + // NOTE: buf will be deleted in the Java_org_rocksdb_Slice_disposeInternalBuf method + char* buf = new char[len]; + memcpy(buf, ptrData, len - 1); + buf[len-1] = '\0'; + + const auto* slice = + new rocksdb::Slice(buf, len - 1); + + env->ReleaseByteArrayElements(data, ptrData, JNI_ABORT); + + return reinterpret_cast(slice); +} + +/* + * Class: org_rocksdb_Slice + * Method: data0 + * Signature: (J)[B + */ +jbyteArray Java_org_rocksdb_Slice_data0( + JNIEnv* env, jobject jobj, jlong handle) { + const auto* slice = reinterpret_cast(handle); + const jsize len = static_cast(slice->size()); + const jbyteArray data = env->NewByteArray(len); + if(data == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + env->SetByteArrayRegion(data, 0, len, + const_cast(reinterpret_cast(slice->data()))); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(data); + return nullptr; + } + + return data; +} + +/* + * Class: org_rocksdb_Slice + * Method: clear0 + * Signature: (JZJ)V + */ +void Java_org_rocksdb_Slice_clear0( + JNIEnv * env, jobject jobj, jlong handle, jboolean shouldRelease, + jlong internalBufferOffset) { + auto* slice = reinterpret_cast(handle); + if(shouldRelease == JNI_TRUE) { + const char* buf = slice->data_ - internalBufferOffset; + delete [] buf; + } + slice->clear(); +} + +/* + * Class: org_rocksdb_Slice + * Method: removePrefix0 + * Signature: (JI)V + */ +void Java_org_rocksdb_Slice_removePrefix0( + JNIEnv * env, jobject jobj, jlong handle, jint length) { + auto* slice = reinterpret_cast(handle); + slice->remove_prefix(length); +} + +/* + * Class: org_rocksdb_Slice + * Method: disposeInternalBuf + * Signature: (JJ)V + */ +void Java_org_rocksdb_Slice_disposeInternalBuf( + JNIEnv * env, jobject jobj, jlong handle, jlong internalBufferOffset) { + const auto* slice = reinterpret_cast(handle); + const char* buf = slice->data_ - internalBufferOffset; + delete [] buf; +} + +// + +// (data_addr); + const auto* slice = new rocksdb::Slice(ptrData, length); + return reinterpret_cast(slice); +} + +/* + * Class: org_rocksdb_DirectSlice + * Method: createNewDirectSlice1 + * Signature: (Ljava/nio/ByteBuffer;)J + */ +jlong Java_org_rocksdb_DirectSlice_createNewDirectSlice1( + JNIEnv* env, jclass jcls, jobject data) { + void* data_addr = env->GetDirectBufferAddress(data); + if(data_addr == nullptr) { + // error: memory region is undefined, given object is not a direct + // java.nio.Buffer, or JNI access to direct buffers is not supported by JVM + rocksdb::IllegalArgumentExceptionJni::ThrowNew(env, + rocksdb::Status::InvalidArgument( + "Could not access DirectBuffer")); + return 0; + } + + const auto* ptrData = reinterpret_cast(data_addr); + const auto* slice = new rocksdb::Slice(ptrData); + return reinterpret_cast(slice); +} + +/* + * Class: org_rocksdb_DirectSlice + * Method: data0 + * Signature: (J)Ljava/lang/Object; + */ +jobject Java_org_rocksdb_DirectSlice_data0( + JNIEnv* env, jobject jobj, jlong handle) { + const auto* slice = reinterpret_cast(handle); + return env->NewDirectByteBuffer(const_cast(slice->data()), + slice->size()); +} + +/* + * Class: org_rocksdb_DirectSlice + * Method: get0 + * Signature: (JI)B + */ +jbyte Java_org_rocksdb_DirectSlice_get0( + JNIEnv* env, jobject jobj, jlong handle, jint offset) { + const auto* slice = reinterpret_cast(handle); + return (*slice)[offset]; +} + +/* + * Class: org_rocksdb_DirectSlice + * Method: clear0 + * Signature: (JZJ)V + */ +void Java_org_rocksdb_DirectSlice_clear0( + JNIEnv* env, jobject jobj, jlong handle, + jboolean shouldRelease, jlong internalBufferOffset) { + auto* slice = reinterpret_cast(handle); + if(shouldRelease == JNI_TRUE) { + const char* buf = slice->data_ - internalBufferOffset; + delete [] buf; + } + slice->clear(); +} + +/* + * Class: org_rocksdb_DirectSlice + * Method: removePrefix0 + * Signature: (JI)V + */ +void Java_org_rocksdb_DirectSlice_removePrefix0( + JNIEnv* env, jobject jobj, jlong handle, jint length) { + auto* slice = reinterpret_cast(handle); + slice->remove_prefix(length); +} + +/* + * Class: org_rocksdb_DirectSlice + * Method: disposeInternalBuf + * Signature: (JJ)V + */ +void Java_org_rocksdb_DirectSlice_disposeInternalBuf( + JNIEnv* env, jobject jobj, jlong handle, jlong internalBufferOffset) { + const auto* slice = reinterpret_cast(handle); + const char* buf = slice->data_ - internalBufferOffset; + delete [] buf; +} + +// diff --git a/java/rocksjni/snapshot.cc b/java/rocksjni/snapshot.cc new file mode 100644 index 00000000000..04a0ebfbafa --- /dev/null +++ b/java/rocksjni/snapshot.cc @@ -0,0 +1,26 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++. + +#include +#include +#include + +#include "include/org_rocksdb_Snapshot.h" +#include "rocksdb/db.h" +#include "rocksjni/portal.h" + +/* + * Class: org_rocksdb_Snapshot + * Method: getSequenceNumber + * Signature: (J)J + */ +jlong Java_org_rocksdb_Snapshot_getSequenceNumber(JNIEnv* env, + jobject jobj, jlong jsnapshot_handle) { + auto* snapshot = reinterpret_cast( + jsnapshot_handle); + return snapshot->GetSequenceNumber(); +} diff --git a/java/rocksjni/sst_file_writerjni.cc b/java/rocksjni/sst_file_writerjni.cc new file mode 100644 index 00000000000..ceb93384acc --- /dev/null +++ b/java/rocksjni/sst_file_writerjni.cc @@ -0,0 +1,248 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ and enables +// calling C++ rocksdb::SstFileWriter methods +// from Java side. + +#include +#include + +#include "include/org_rocksdb_SstFileWriter.h" +#include "rocksdb/comparator.h" +#include "rocksdb/env.h" +#include "rocksdb/options.h" +#include "rocksdb/sst_file_writer.h" +#include "rocksjni/portal.h" + +/* + * Class: org_rocksdb_SstFileWriter + * Method: newSstFileWriter + * Signature: (JJJ)J + */ +jlong Java_org_rocksdb_SstFileWriter_newSstFileWriter__JJJ(JNIEnv *env, jclass jcls, + jlong jenvoptions, + jlong joptions, + jlong jcomparator) { + auto *env_options = + reinterpret_cast(jenvoptions); + auto *options = reinterpret_cast(joptions); + auto *comparator = reinterpret_cast(jcomparator); + rocksdb::SstFileWriter *sst_file_writer = + new rocksdb::SstFileWriter(*env_options, *options, comparator); + return reinterpret_cast(sst_file_writer); +} + +/* + * Class: org_rocksdb_SstFileWriter + * Method: newSstFileWriter + * Signature: (JJ)J + */ +jlong Java_org_rocksdb_SstFileWriter_newSstFileWriter__JJ(JNIEnv *env, jclass jcls, + jlong jenvoptions, + jlong joptions) { + auto *env_options = + reinterpret_cast(jenvoptions); + auto *options = reinterpret_cast(joptions); + rocksdb::SstFileWriter *sst_file_writer = + new rocksdb::SstFileWriter(*env_options, *options); + return reinterpret_cast(sst_file_writer); +} + +/* + * Class: org_rocksdb_SstFileWriter + * Method: open + * Signature: (JLjava/lang/String;)V + */ +void Java_org_rocksdb_SstFileWriter_open(JNIEnv *env, jobject jobj, + jlong jhandle, jstring jfile_path) { + const char *file_path = env->GetStringUTFChars(jfile_path, nullptr); + if(file_path == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + rocksdb::Status s = + reinterpret_cast(jhandle)->Open(file_path); + env->ReleaseStringUTFChars(jfile_path, file_path); + + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_SstFileWriter + * Method: put + * Signature: (JJJ)V + */ +void Java_org_rocksdb_SstFileWriter_put__JJJ(JNIEnv *env, jobject jobj, + jlong jhandle, jlong jkey_handle, + jlong jvalue_handle) { + auto *key_slice = reinterpret_cast(jkey_handle); + auto *value_slice = reinterpret_cast(jvalue_handle); + rocksdb::Status s = + reinterpret_cast(jhandle)->Put(*key_slice, + *value_slice); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_SstFileWriter + * Method: put + * Signature: (JJJ)V + */ + void Java_org_rocksdb_SstFileWriter_put__J_3B_3B(JNIEnv *env, jobject jobj, + jlong jhandle, jbyteArray jkey, + jbyteArray jval) { + jbyte* key = env->GetByteArrayElements(jkey, nullptr); + if(key == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + rocksdb::Slice key_slice( + reinterpret_cast(key), env->GetArrayLength(jkey)); + + jbyte* value = env->GetByteArrayElements(jval, nullptr); + if(value == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + return; + } + rocksdb::Slice value_slice( + reinterpret_cast(value), env->GetArrayLength(jval)); + + rocksdb::Status s = + reinterpret_cast(jhandle)->Put(key_slice, + value_slice); + + env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + env->ReleaseByteArrayElements(jval, value, JNI_ABORT); + + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_SstFileWriter + * Method: merge + * Signature: (JJJ)V + */ +void Java_org_rocksdb_SstFileWriter_merge__JJJ(JNIEnv *env, jobject jobj, + jlong jhandle, jlong jkey_handle, + jlong jvalue_handle) { + auto *key_slice = reinterpret_cast(jkey_handle); + auto *value_slice = reinterpret_cast(jvalue_handle); + rocksdb::Status s = + reinterpret_cast(jhandle)->Merge(*key_slice, + *value_slice); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_SstFileWriter + * Method: merge + * Signature: (J[B[B)V + */ +void Java_org_rocksdb_SstFileWriter_merge__J_3B_3B(JNIEnv *env, jobject jobj, + jlong jhandle, jbyteArray jkey, + jbyteArray jval) { + + jbyte* key = env->GetByteArrayElements(jkey, nullptr); + if(key == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + rocksdb::Slice key_slice( + reinterpret_cast(key), env->GetArrayLength(jkey)); + + jbyte* value = env->GetByteArrayElements(jval, nullptr); + if(value == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + return; + } + rocksdb::Slice value_slice( + reinterpret_cast(value), env->GetArrayLength(jval)); + + rocksdb::Status s = + reinterpret_cast(jhandle)->Merge(key_slice, + value_slice); + + env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + env->ReleaseByteArrayElements(jval, value, JNI_ABORT); + + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_SstFileWriter + * Method: delete + * Signature: (JJJ)V + */ +void Java_org_rocksdb_SstFileWriter_delete__J_3B(JNIEnv *env, jobject jobj, + jlong jhandle, jbyteArray jkey) { + jbyte* key = env->GetByteArrayElements(jkey, nullptr); + if(key == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + rocksdb::Slice key_slice( + reinterpret_cast(key), env->GetArrayLength(jkey)); + + rocksdb::Status s = + reinterpret_cast(jhandle)->Delete(key_slice); + + env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_SstFileWriter + * Method: delete + * Signature: (JJJ)V + */ + void Java_org_rocksdb_SstFileWriter_delete__JJ(JNIEnv *env, jobject jobj, + jlong jhandle, jlong jkey_handle) { + auto *key_slice = reinterpret_cast(jkey_handle); + rocksdb::Status s = + reinterpret_cast(jhandle)->Delete(*key_slice); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_SstFileWriter + * Method: finish + * Signature: (J)V + */ +void Java_org_rocksdb_SstFileWriter_finish(JNIEnv *env, jobject jobj, + jlong jhandle) { + rocksdb::Status s = + reinterpret_cast(jhandle)->Finish(); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_SstFileWriter + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_SstFileWriter_disposeInternal(JNIEnv *env, jobject jobj, + jlong jhandle) { + delete reinterpret_cast(jhandle); +} diff --git a/java/rocksjni/statistics.cc b/java/rocksjni/statistics.cc index bf170c6de46..7b657ada7b6 100644 --- a/java/rocksjni/statistics.cc +++ b/java/rocksjni/statistics.cc @@ -1,50 +1,244 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ and enables // calling c++ rocksdb::Statistics methods from Java side. -#include -#include #include +#include +#include #include "include/org_rocksdb_Statistics.h" #include "rocksjni/portal.h" +#include "rocksjni/statisticsjni.h" #include "rocksdb/statistics.h" /* * Class: org_rocksdb_Statistics - * Method: getTickerCount0 - * Signature: (IJ)J + * Method: newStatistics + * Signature: ()J */ -jlong Java_org_rocksdb_Statistics_getTickerCount0( - JNIEnv* env, jobject jobj, int tickerType, jlong handle) { - auto st = reinterpret_cast(handle); - assert(st != nullptr); +jlong Java_org_rocksdb_Statistics_newStatistics__(JNIEnv* env, jclass jcls) { + return Java_org_rocksdb_Statistics_newStatistics___3BJ( + env, jcls, nullptr, 0); +} + +/* + * Class: org_rocksdb_Statistics + * Method: newStatistics + * Signature: (J)J + */ +jlong Java_org_rocksdb_Statistics_newStatistics__J( + JNIEnv* env, jclass jcls, jlong jother_statistics_handle) { + return Java_org_rocksdb_Statistics_newStatistics___3BJ( + env, jcls, nullptr, jother_statistics_handle); +} + +/* + * Class: org_rocksdb_Statistics + * Method: newStatistics + * Signature: ([B)J + */ +jlong Java_org_rocksdb_Statistics_newStatistics___3B( + JNIEnv* env, jclass jcls, jbyteArray jhistograms) { + return Java_org_rocksdb_Statistics_newStatistics___3BJ( + env, jcls, jhistograms, 0); +} + +/* + * Class: org_rocksdb_Statistics + * Method: newStatistics + * Signature: ([BJ)J + */ +jlong Java_org_rocksdb_Statistics_newStatistics___3BJ( + JNIEnv* env, jclass jcls, jbyteArray jhistograms, + jlong jother_statistics_handle) { + + std::shared_ptr* pSptr_other_statistics = nullptr; + if (jother_statistics_handle > 0) { + pSptr_other_statistics = + reinterpret_cast*>( + jother_statistics_handle); + } - return st->getTickerCount(static_cast(tickerType)); + std::set histograms; + if (jhistograms != nullptr) { + const jsize len = env->GetArrayLength(jhistograms); + if (len > 0) { + jbyte* jhistogram = env->GetByteArrayElements(jhistograms, nullptr); + if (jhistogram == nullptr ) { + // exception thrown: OutOfMemoryError + return 0; + } + + for (jsize i = 0; i < len; i++) { + const rocksdb::Histograms histogram = + rocksdb::HistogramTypeJni::toCppHistograms(jhistogram[i]); + histograms.emplace(histogram); + } + + env->ReleaseByteArrayElements(jhistograms, jhistogram, JNI_ABORT); + } + } + + std::shared_ptr sptr_other_statistics = nullptr; + if (pSptr_other_statistics != nullptr) { + sptr_other_statistics = *pSptr_other_statistics; + } + + auto* pSptr_statistics = new std::shared_ptr( + new rocksdb::StatisticsJni(sptr_other_statistics, histograms)); + + return reinterpret_cast(pSptr_statistics); +} + +/* + * Class: org_rocksdb_Statistics + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_Statistics_disposeInternal( + JNIEnv* env, jobject jobj, jlong jhandle) { + if(jhandle > 0) { + auto* pSptr_statistics = + reinterpret_cast*>(jhandle); + delete pSptr_statistics; + } } /* * Class: org_rocksdb_Statistics - * Method: geHistogramData0 - * Signature: (IJ)Lorg/rocksdb/HistogramData; + * Method: statsLevel + * Signature: (J)B */ -jobject Java_org_rocksdb_Statistics_geHistogramData0( - JNIEnv* env, jobject jobj, int histogramType, jlong handle) { - auto st = reinterpret_cast(handle); - assert(st != nullptr); +jbyte Java_org_rocksdb_Statistics_statsLevel( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* pSptr_statistics = + reinterpret_cast*>(jhandle); + assert(pSptr_statistics != nullptr); + return rocksdb::StatsLevelJni::toJavaStatsLevel(pSptr_statistics->get()->stats_level_); +} - rocksdb::HistogramData data; - st->histogramData(static_cast(histogramType), - &data); +/* + * Class: org_rocksdb_Statistics + * Method: setStatsLevel + * Signature: (JB)V + */ +void Java_org_rocksdb_Statistics_setStatsLevel( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jstats_level) { + auto* pSptr_statistics = + reinterpret_cast*>(jhandle); + assert(pSptr_statistics != nullptr); + auto stats_level = rocksdb::StatsLevelJni::toCppStatsLevel(jstats_level); + pSptr_statistics->get()->stats_level_ = stats_level; +} + +/* + * Class: org_rocksdb_Statistics + * Method: getTickerCount + * Signature: (JB)J + */ +jlong Java_org_rocksdb_Statistics_getTickerCount( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jticker_type) { + auto* pSptr_statistics = + reinterpret_cast*>(jhandle); + assert(pSptr_statistics != nullptr); + auto ticker = rocksdb::TickerTypeJni::toCppTickers(jticker_type); + return pSptr_statistics->get()->getTickerCount(ticker); +} + +/* + * Class: org_rocksdb_Statistics + * Method: getAndResetTickerCount + * Signature: (JB)J + */ +jlong Java_org_rocksdb_Statistics_getAndResetTickerCount( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jticker_type) { + auto* pSptr_statistics = + reinterpret_cast*>(jhandle); + assert(pSptr_statistics != nullptr); + auto ticker = rocksdb::TickerTypeJni::toCppTickers(jticker_type); + return pSptr_statistics->get()->getAndResetTickerCount(ticker); +} + +/* + * Class: org_rocksdb_Statistics + * Method: getHistogramData + * Signature: (JB)Lorg/rocksdb/HistogramData; + */ +jobject Java_org_rocksdb_Statistics_getHistogramData( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jhistogram_type) { + auto* pSptr_statistics = + reinterpret_cast*>(jhandle); + assert(pSptr_statistics != nullptr); + + rocksdb::HistogramData data; // TODO(AR) perhaps better to construct a Java Object Wrapper that uses ptr to C++ `new HistogramData` + auto histogram = rocksdb::HistogramTypeJni::toCppHistograms(jhistogram_type); + pSptr_statistics->get()->histogramData( + static_cast(histogram), &data); + + jclass jclazz = rocksdb::HistogramDataJni::getJClass(env); + if(jclazz == nullptr) { + // exception occurred accessing class + return nullptr; + } - // Don't reuse class pointer - jclass jclazz = env->FindClass("org/rocksdb/HistogramData"); jmethodID mid = rocksdb::HistogramDataJni::getConstructorMethodId( - env, jclazz); - return env->NewObject(jclazz, mid, data.median, data.percentile95, - data.percentile99, data.average, data.standard_deviation); + env); + if(mid == nullptr) { + // exception occurred accessing method + return nullptr; + } + + return env->NewObject( + jclazz, + mid, data.median, data.percentile95,data.percentile99, data.average, + data.standard_deviation); +} + +/* + * Class: org_rocksdb_Statistics + * Method: getHistogramString + * Signature: (JB)Ljava/lang/String; + */ +jstring Java_org_rocksdb_Statistics_getHistogramString( + JNIEnv* env, jobject jobj, jlong jhandle, jbyte jhistogram_type) { + auto* pSptr_statistics = + reinterpret_cast*>(jhandle); + assert(pSptr_statistics != nullptr); + auto histogram = rocksdb::HistogramTypeJni::toCppHistograms(jhistogram_type); + auto str = pSptr_statistics->get()->getHistogramString(histogram); + return env->NewStringUTF(str.c_str()); +} + +/* + * Class: org_rocksdb_Statistics + * Method: reset + * Signature: (J)V + */ +void Java_org_rocksdb_Statistics_reset( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* pSptr_statistics = + reinterpret_cast*>(jhandle); + assert(pSptr_statistics != nullptr); + rocksdb::Status s = pSptr_statistics->get()->Reset(); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_Statistics + * Method: toString + * Signature: (J)Ljava/lang/String; + */ +jstring Java_org_rocksdb_Statistics_toString( + JNIEnv* env, jobject jobj, jlong jhandle) { + auto* pSptr_statistics = + reinterpret_cast*>(jhandle); + assert(pSptr_statistics != nullptr); + auto str = pSptr_statistics->get()->ToString(); + return env->NewStringUTF(str.c_str()); } diff --git a/java/rocksjni/statisticsjni.cc b/java/rocksjni/statisticsjni.cc new file mode 100644 index 00000000000..584ab5aa610 --- /dev/null +++ b/java/rocksjni/statisticsjni.cc @@ -0,0 +1,33 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the callback "bridge" between Java and C++ for +// rocksdb::Statistics + +#include "rocksjni/statisticsjni.h" + +namespace rocksdb { + + StatisticsJni::StatisticsJni(std::shared_ptr stats) + : StatisticsImpl(stats, false), m_ignore_histograms() { + } + + StatisticsJni::StatisticsJni(std::shared_ptr stats, + const std::set ignore_histograms) : StatisticsImpl(stats, false), + m_ignore_histograms(ignore_histograms) { + } + + bool StatisticsJni::HistEnabledForType(uint32_t type) const { + if (type >= HISTOGRAM_ENUM_MAX) { + return false; + } + + if (m_ignore_histograms.count(type) > 0) { + return false; + } + + return true; + } +}; \ No newline at end of file diff --git a/java/rocksjni/statisticsjni.h b/java/rocksjni/statisticsjni.h new file mode 100644 index 00000000000..600d9a67632 --- /dev/null +++ b/java/rocksjni/statisticsjni.h @@ -0,0 +1,33 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the callback "bridge" between Java and C++ for +// rocksdb::Statistics + +#ifndef JAVA_ROCKSJNI_STATISTICSJNI_H_ +#define JAVA_ROCKSJNI_STATISTICSJNI_H_ + +#include +#include +#include +#include "rocksdb/statistics.h" +#include "monitoring/statistics.h" + +namespace rocksdb { + + class StatisticsJni : public StatisticsImpl { + public: + StatisticsJni(std::shared_ptr stats); + StatisticsJni(std::shared_ptr stats, + const std::set ignore_histograms); + virtual bool HistEnabledForType(uint32_t type) const override; + + private: + const std::set m_ignore_histograms; + }; + +} // namespace rocksdb + +#endif // JAVA_ROCKSJNI_STATISTICSJNI_H_ \ No newline at end of file diff --git a/java/rocksjni/table.cc b/java/rocksjni/table.cc index ffda1a2ba43..5f0a4735fed 100644 --- a/java/rocksjni/table.cc +++ b/java/rocksjni/table.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ for rocksdb::Options. @@ -15,36 +15,47 @@ /* * Class: org_rocksdb_PlainTableConfig * Method: newTableFactoryHandle - * Signature: (IIDI)J + * Signature: (IIDIIBZZ)J */ jlong Java_org_rocksdb_PlainTableConfig_newTableFactoryHandle( JNIEnv* env, jobject jobj, jint jkey_size, jint jbloom_bits_per_key, - jdouble jhash_table_ratio, jint jindex_sparseness) { + jdouble jhash_table_ratio, jint jindex_sparseness, + jint jhuge_page_tlb_size, jbyte jencoding_type, + jboolean jfull_scan_mode, jboolean jstore_index_in_file) { rocksdb::PlainTableOptions options = rocksdb::PlainTableOptions(); options.user_key_len = jkey_size; options.bloom_bits_per_key = jbloom_bits_per_key; options.hash_table_ratio = jhash_table_ratio; options.index_sparseness = jindex_sparseness; + options.huge_page_tlb_size = jhuge_page_tlb_size; + options.encoding_type = static_cast( + jencoding_type); + options.full_scan_mode = jfull_scan_mode; + options.store_index_in_file = jstore_index_in_file; return reinterpret_cast(rocksdb::NewPlainTableFactory(options)); } /* * Class: org_rocksdb_BlockBasedTableConfig * Method: newTableFactoryHandle - * Signature: (ZJIJIIZI)J + * Signature: (ZJIJIIZIZZZJIBBI)J */ jlong Java_org_rocksdb_BlockBasedTableConfig_newTableFactoryHandle( JNIEnv* env, jobject jobj, jboolean no_block_cache, jlong block_cache_size, - jint num_shardbits, jlong block_size, jint block_size_deviation, + jint block_cache_num_shardbits, jlong block_size, jint block_size_deviation, jint block_restart_interval, jboolean whole_key_filtering, - jint bits_per_key) { + jlong jfilterPolicy, jboolean cache_index_and_filter_blocks, + jboolean pin_l0_filter_and_index_blocks_in_cache, + jboolean hash_index_allow_collision, jlong block_cache_compressed_size, + jint block_cache_compressd_num_shard_bits, jbyte jchecksum_type, + jbyte jindex_type, jint jformat_version) { rocksdb::BlockBasedTableOptions options; options.no_block_cache = no_block_cache; if (!no_block_cache && block_cache_size > 0) { - if (num_shardbits > 0) { + if (block_cache_num_shardbits > 0) { options.block_cache = - rocksdb::NewLRUCache(block_cache_size, num_shardbits); + rocksdb::NewLRUCache(block_cache_size, block_cache_num_shardbits); } else { options.block_cache = rocksdb::NewLRUCache(block_cache_size); } @@ -53,8 +64,29 @@ jlong Java_org_rocksdb_BlockBasedTableConfig_newTableFactoryHandle( options.block_size_deviation = block_size_deviation; options.block_restart_interval = block_restart_interval; options.whole_key_filtering = whole_key_filtering; - if (bits_per_key > 0) { - options.filter_policy.reset(rocksdb::NewBloomFilterPolicy(bits_per_key)); + if (jfilterPolicy > 0) { + std::shared_ptr *pFilterPolicy = + reinterpret_cast *>( + jfilterPolicy); + options.filter_policy = *pFilterPolicy; } + options.cache_index_and_filter_blocks = cache_index_and_filter_blocks; + options.pin_l0_filter_and_index_blocks_in_cache = + pin_l0_filter_and_index_blocks_in_cache; + options.hash_index_allow_collision = hash_index_allow_collision; + if (block_cache_compressed_size > 0) { + if (block_cache_compressd_num_shard_bits > 0) { + options.block_cache = + rocksdb::NewLRUCache(block_cache_compressed_size, + block_cache_compressd_num_shard_bits); + } else { + options.block_cache = rocksdb::NewLRUCache(block_cache_compressed_size); + } + } + options.checksum = static_cast(jchecksum_type); + options.index_type = static_cast< + rocksdb::BlockBasedTableOptions::IndexType>(jindex_type); + options.format_version = jformat_version; + return reinterpret_cast(rocksdb::NewBlockBasedTableFactory(options)); } diff --git a/java/rocksjni/transaction_log.cc b/java/rocksjni/transaction_log.cc new file mode 100644 index 00000000000..a5049e3b26a --- /dev/null +++ b/java/rocksjni/transaction_log.cc @@ -0,0 +1,71 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ and enables +// calling c++ rocksdb::Iterator methods from Java side. + +#include +#include +#include + +#include "include/org_rocksdb_TransactionLogIterator.h" +#include "rocksdb/transaction_log.h" +#include "rocksjni/portal.h" + +/* + * Class: org_rocksdb_TransactionLogIterator + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_TransactionLogIterator_disposeInternal( + JNIEnv* env, jobject jobj, jlong handle) { + delete reinterpret_cast(handle); +} + +/* + * Class: org_rocksdb_TransactionLogIterator + * Method: isValid + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_TransactionLogIterator_isValid( + JNIEnv* env, jobject jobj, jlong handle) { + return reinterpret_cast(handle)->Valid(); +} + +/* + * Class: org_rocksdb_TransactionLogIterator + * Method: next + * Signature: (J)V + */ +void Java_org_rocksdb_TransactionLogIterator_next( + JNIEnv* env, jobject jobj, jlong handle) { + reinterpret_cast(handle)->Next(); +} + +/* + * Class: org_rocksdb_TransactionLogIterator + * Method: status + * Signature: (J)V + */ +void Java_org_rocksdb_TransactionLogIterator_status( + JNIEnv* env, jobject jobj, jlong handle) { + rocksdb::Status s = reinterpret_cast< + rocksdb::TransactionLogIterator*>(handle)->status(); + if (!s.ok()) { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + } +} + +/* + * Class: org_rocksdb_TransactionLogIterator + * Method: getBatch + * Signature: (J)Lorg/rocksdb/TransactionLogIterator$BatchResult + */ +jobject Java_org_rocksdb_TransactionLogIterator_getBatch( + JNIEnv* env, jobject jobj, jlong handle) { + rocksdb::BatchResult batch_result = + reinterpret_cast(handle)->GetBatch(); + return rocksdb::BatchResultJni::construct(env, batch_result); +} diff --git a/java/rocksjni/ttl.cc b/java/rocksjni/ttl.cc new file mode 100644 index 00000000000..a66ad86d626 --- /dev/null +++ b/java/rocksjni/ttl.cc @@ -0,0 +1,183 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ and enables +// calling c++ rocksdb::TtlDB methods. +// from Java side. + +#include +#include +#include +#include +#include +#include + +#include "include/org_rocksdb_TtlDB.h" +#include "rocksdb/utilities/db_ttl.h" +#include "rocksjni/portal.h" + +/* + * Class: org_rocksdb_TtlDB + * Method: open + * Signature: (JLjava/lang/String;IZ)J + */ +jlong Java_org_rocksdb_TtlDB_open(JNIEnv* env, + jclass jcls, jlong joptions_handle, jstring jdb_path, + jint jttl, jboolean jread_only) { + const char* db_path = env->GetStringUTFChars(jdb_path, nullptr); + if(db_path == nullptr) { + // exception thrown: OutOfMemoryError + return 0; + } + + auto* opt = reinterpret_cast(joptions_handle); + rocksdb::DBWithTTL* db = nullptr; + rocksdb::Status s = rocksdb::DBWithTTL::Open(*opt, db_path, &db, + jttl, jread_only); + env->ReleaseStringUTFChars(jdb_path, db_path); + + // as TTLDB extends RocksDB on the java side, we can reuse + // the RocksDB portal here. + if (s.ok()) { + return reinterpret_cast(db); + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return 0; + } +} + +/* + * Class: org_rocksdb_TtlDB + * Method: openCF + * Signature: (JLjava/lang/String;[[B[J[IZ)[J + */ +jlongArray + Java_org_rocksdb_TtlDB_openCF( + JNIEnv* env, jclass jcls, jlong jopt_handle, jstring jdb_path, + jobjectArray jcolumn_names, jlongArray jcolumn_options, + jintArray jttls, jboolean jread_only) { + const char* db_path = env->GetStringUTFChars(jdb_path, nullptr); + if(db_path == nullptr) { + // exception thrown: OutOfMemoryError + return 0; + } + + const jsize len_cols = env->GetArrayLength(jcolumn_names); + jlong* jco = env->GetLongArrayElements(jcolumn_options, nullptr); + if(jco == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseStringUTFChars(jdb_path, db_path); + return nullptr; + } + + std::vector column_families; + jboolean has_exception = JNI_FALSE; + rocksdb::JniUtil::byteStrings( + env, + jcolumn_names, + [](const char* str_data, const size_t str_len) { + return std::string(str_data, str_len); + }, + [&jco, &column_families](size_t idx, std::string cf_name) { + rocksdb::ColumnFamilyOptions* cf_options = + reinterpret_cast(jco[idx]); + column_families.push_back( + rocksdb::ColumnFamilyDescriptor(cf_name, *cf_options)); + }, + &has_exception); + + env->ReleaseLongArrayElements(jcolumn_options, jco, JNI_ABORT); + + if(has_exception == JNI_TRUE) { + // exception occurred + env->ReleaseStringUTFChars(jdb_path, db_path); + return nullptr; + } + + std::vector ttl_values; + jint* jttlv = env->GetIntArrayElements(jttls, nullptr); + if(jttlv == nullptr) { + // exception thrown: OutOfMemoryError + env->ReleaseStringUTFChars(jdb_path, db_path); + return nullptr; + } + const jsize len_ttls = env->GetArrayLength(jttls); + for(jsize i = 0; i < len_ttls; i++) { + ttl_values.push_back(jttlv[i]); + } + env->ReleaseIntArrayElements(jttls, jttlv, JNI_ABORT); + + auto* opt = reinterpret_cast(jopt_handle); + std::vector handles; + rocksdb::DBWithTTL* db = nullptr; + rocksdb::Status s = rocksdb::DBWithTTL::Open(*opt, db_path, column_families, + &handles, &db, ttl_values, jread_only); + + // we have now finished with db_path + env->ReleaseStringUTFChars(jdb_path, db_path); + + // check if open operation was successful + if (s.ok()) { + const jsize resultsLen = 1 + len_cols; //db handle + column family handles + std::unique_ptr results = + std::unique_ptr(new jlong[resultsLen]); + results[0] = reinterpret_cast(db); + for(int i = 1; i <= len_cols; i++) { + results[i] = reinterpret_cast(handles[i - 1]); + } + + jlongArray jresults = env->NewLongArray(resultsLen); + if(jresults == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + env->SetLongArrayRegion(jresults, 0, resultsLen, results.get()); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jresults); + return nullptr; + } + + return jresults; + } else { + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return NULL; + } +} + +/* + * Class: org_rocksdb_TtlDB + * Method: createColumnFamilyWithTtl + * Signature: (JLorg/rocksdb/ColumnFamilyDescriptor;[BJI)J; + */ +jlong Java_org_rocksdb_TtlDB_createColumnFamilyWithTtl( + JNIEnv* env, jobject jobj, jlong jdb_handle, + jbyteArray jcolumn_name, jlong jcolumn_options, jint jttl) { + + jbyte* cfname = env->GetByteArrayElements(jcolumn_name, nullptr); + if(cfname == nullptr) { + // exception thrown: OutOfMemoryError + return 0; + } + const jsize len = env->GetArrayLength(jcolumn_name); + + auto* cfOptions = + reinterpret_cast(jcolumn_options); + + auto* db_handle = reinterpret_cast(jdb_handle); + rocksdb::ColumnFamilyHandle* handle; + rocksdb::Status s = db_handle->CreateColumnFamilyWithTtl( + *cfOptions, std::string(reinterpret_cast(cfname), + len), &handle, jttl); + + env->ReleaseByteArrayElements(jcolumn_name, cfname, 0); + + if (s.ok()) { + return reinterpret_cast(handle); + } + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); + return 0; +} diff --git a/java/rocksjni/write_batch.cc b/java/rocksjni/write_batch.cc index e8b2456eeed..e84f6ed7d18 100644 --- a/java/rocksjni/write_batch.cc +++ b/java/rocksjni/write_batch.cc @@ -1,45 +1,46 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // This file implements the "bridge" between Java and C++ and enables // calling c++ rocksdb::WriteBatch methods from Java side. #include -#include "include/org_rocksdb_WriteBatch.h" -#include "include/org_rocksdb_WriteBatchInternal.h" -#include "include/org_rocksdb_WriteBatchTest.h" -#include "rocksjni/portal.h" -#include "rocksdb/db.h" #include "db/memtable.h" -#include "rocksdb/write_batch.h" #include "db/write_batch_internal.h" +#include "include/org_rocksdb_WriteBatch.h" +#include "include/org_rocksdb_WriteBatch_Handler.h" +#include "rocksdb/db.h" #include "rocksdb/env.h" #include "rocksdb/memtablerep.h" +#include "rocksdb/status.h" +#include "rocksdb/write_batch.h" +#include "rocksdb/write_buffer_manager.h" +#include "rocksjni/portal.h" +#include "rocksjni/writebatchhandlerjnicallback.h" +#include "table/scoped_arena_iterator.h" #include "util/logging.h" -#include "util/testharness.h" /* * Class: org_rocksdb_WriteBatch * Method: newWriteBatch - * Signature: (I)V + * Signature: (I)J */ -void Java_org_rocksdb_WriteBatch_newWriteBatch( - JNIEnv* env, jobject jobj, jint jreserved_bytes) { - rocksdb::WriteBatch* wb = new rocksdb::WriteBatch( - static_cast(jreserved_bytes)); - - rocksdb::WriteBatchJni::setHandle(env, jobj, wb); +jlong Java_org_rocksdb_WriteBatch_newWriteBatch( + JNIEnv* env, jclass jcls, jint jreserved_bytes) { + auto* wb = new rocksdb::WriteBatch(static_cast(jreserved_bytes)); + return reinterpret_cast(wb); } /* * Class: org_rocksdb_WriteBatch - * Method: count - * Signature: ()I + * Method: count0 + * Signature: (J)I */ -jint Java_org_rocksdb_WriteBatch_count(JNIEnv* env, jobject jobj) { - rocksdb::WriteBatch* wb = rocksdb::WriteBatchJni::getHandle(env, jobj); +jint Java_org_rocksdb_WriteBatch_count0(JNIEnv* env, jobject jobj, + jlong jwb_handle) { + auto* wb = reinterpret_cast(jwb_handle); assert(wb != nullptr); return static_cast(wb->Count()); @@ -47,215 +48,266 @@ jint Java_org_rocksdb_WriteBatch_count(JNIEnv* env, jobject jobj) { /* * Class: org_rocksdb_WriteBatch - * Method: clear - * Signature: ()V + * Method: clear0 + * Signature: (J)V */ -void Java_org_rocksdb_WriteBatch_clear(JNIEnv* env, jobject jobj) { - rocksdb::WriteBatch* wb = rocksdb::WriteBatchJni::getHandle(env, jobj); +void Java_org_rocksdb_WriteBatch_clear0(JNIEnv* env, jobject jobj, + jlong jwb_handle) { + auto* wb = reinterpret_cast(jwb_handle); assert(wb != nullptr); wb->Clear(); } +/* + * Class: org_rocksdb_WriteBatch + * Method: setSavePoint0 + * Signature: (J)V + */ +void Java_org_rocksdb_WriteBatch_setSavePoint0( + JNIEnv* env, jobject jobj, jlong jwb_handle) { + auto* wb = reinterpret_cast(jwb_handle); + assert(wb != nullptr); + + wb->SetSavePoint(); +} + +/* + * Class: org_rocksdb_WriteBatch + * Method: rollbackToSavePoint0 + * Signature: (J)V + */ +void Java_org_rocksdb_WriteBatch_rollbackToSavePoint0( + JNIEnv* env, jobject jobj, jlong jwb_handle) { + auto* wb = reinterpret_cast(jwb_handle); + assert(wb != nullptr); + + auto s = wb->RollbackToSavePoint(); + + if (s.ok()) { + return; + } + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); +} + /* * Class: org_rocksdb_WriteBatch * Method: put - * Signature: ([BI[BI)V + * Signature: (J[BI[BI)V */ -void Java_org_rocksdb_WriteBatch_put( - JNIEnv* env, jobject jobj, +void Java_org_rocksdb_WriteBatch_put__J_3BI_3BI( + JNIEnv* env, jobject jobj, jlong jwb_handle, jbyteArray jkey, jint jkey_len, - jbyteArray jvalue, jint jvalue_len) { - rocksdb::WriteBatch* wb = rocksdb::WriteBatchJni::getHandle(env, jobj); + jbyteArray jentry_value, jint jentry_value_len) { + auto* wb = reinterpret_cast(jwb_handle); assert(wb != nullptr); + auto put = [&wb] (rocksdb::Slice key, rocksdb::Slice value) { + wb->Put(key, value); + }; + rocksdb::JniUtil::kv_op(put, env, jobj, jkey, jkey_len, jentry_value, + jentry_value_len); +} - jbyte* key = env->GetByteArrayElements(jkey, nullptr); - jbyte* value = env->GetByteArrayElements(jvalue, nullptr); - rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); - rocksdb::Slice value_slice(reinterpret_cast(value), jvalue_len); - wb->Put(key_slice, value_slice); - env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); - env->ReleaseByteArrayElements(jvalue, value, JNI_ABORT); +/* + * Class: org_rocksdb_WriteBatch + * Method: put + * Signature: (J[BI[BIJ)V + */ +void Java_org_rocksdb_WriteBatch_put__J_3BI_3BIJ( + JNIEnv* env, jobject jobj, jlong jwb_handle, + jbyteArray jkey, jint jkey_len, + jbyteArray jentry_value, jint jentry_value_len, jlong jcf_handle) { + auto* wb = reinterpret_cast(jwb_handle); + assert(wb != nullptr); + auto* cf_handle = reinterpret_cast(jcf_handle); + assert(cf_handle != nullptr); + auto put = [&wb, &cf_handle] (rocksdb::Slice key, rocksdb::Slice value) { + wb->Put(cf_handle, key, value); + }; + rocksdb::JniUtil::kv_op(put, env, jobj, jkey, jkey_len, jentry_value, + jentry_value_len); } /* * Class: org_rocksdb_WriteBatch * Method: merge - * Signature: ([BI[BI)V + * Signature: (J[BI[BI)V */ -JNIEXPORT void JNICALL Java_org_rocksdb_WriteBatch_merge( - JNIEnv* env, jobject jobj, +void Java_org_rocksdb_WriteBatch_merge__J_3BI_3BI( + JNIEnv* env, jobject jobj, jlong jwb_handle, jbyteArray jkey, jint jkey_len, - jbyteArray jvalue, jint jvalue_len) { - rocksdb::WriteBatch* wb = rocksdb::WriteBatchJni::getHandle(env, jobj); + jbyteArray jentry_value, jint jentry_value_len) { + auto* wb = reinterpret_cast(jwb_handle); assert(wb != nullptr); + auto merge = [&wb] (rocksdb::Slice key, rocksdb::Slice value) { + wb->Merge(key, value); + }; + rocksdb::JniUtil::kv_op(merge, env, jobj, jkey, jkey_len, jentry_value, + jentry_value_len); +} - jbyte* key = env->GetByteArrayElements(jkey, nullptr); - jbyte* value = env->GetByteArrayElements(jvalue, nullptr); - rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); - rocksdb::Slice value_slice(reinterpret_cast(value), jvalue_len); - wb->Merge(key_slice, value_slice); - env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); - env->ReleaseByteArrayElements(jvalue, value, JNI_ABORT); +/* + * Class: org_rocksdb_WriteBatch + * Method: merge + * Signature: (J[BI[BIJ)V + */ +void Java_org_rocksdb_WriteBatch_merge__J_3BI_3BIJ( + JNIEnv* env, jobject jobj, jlong jwb_handle, + jbyteArray jkey, jint jkey_len, + jbyteArray jentry_value, jint jentry_value_len, jlong jcf_handle) { + auto* wb = reinterpret_cast(jwb_handle); + assert(wb != nullptr); + auto* cf_handle = reinterpret_cast(jcf_handle); + assert(cf_handle != nullptr); + auto merge = [&wb, &cf_handle] (rocksdb::Slice key, rocksdb::Slice value) { + wb->Merge(cf_handle, key, value); + }; + rocksdb::JniUtil::kv_op(merge, env, jobj, jkey, jkey_len, jentry_value, + jentry_value_len); } /* * Class: org_rocksdb_WriteBatch * Method: remove - * Signature: ([BI)V + * Signature: (J[BI)V */ -JNIEXPORT void JNICALL Java_org_rocksdb_WriteBatch_remove( - JNIEnv* env, jobject jobj, +void Java_org_rocksdb_WriteBatch_remove__J_3BI( + JNIEnv* env, jobject jobj, jlong jwb_handle, jbyteArray jkey, jint jkey_len) { - rocksdb::WriteBatch* wb = rocksdb::WriteBatchJni::getHandle(env, jobj); + auto* wb = reinterpret_cast(jwb_handle); assert(wb != nullptr); - - jbyte* key = env->GetByteArrayElements(jkey, nullptr); - rocksdb::Slice key_slice(reinterpret_cast(key), jkey_len); - wb->Delete(key_slice); - env->ReleaseByteArrayElements(jkey, key, JNI_ABORT); + auto remove = [&wb] (rocksdb::Slice key) { + wb->Delete(key); + }; + rocksdb::JniUtil::k_op(remove, env, jobj, jkey, jkey_len); } /* * Class: org_rocksdb_WriteBatch - * Method: putLogData - * Signature: ([BI)V + * Method: remove + * Signature: (J[BIJ)V */ -void Java_org_rocksdb_WriteBatch_putLogData( - JNIEnv* env, jobject jobj, jbyteArray jblob, jint jblob_len) { - rocksdb::WriteBatch* wb = rocksdb::WriteBatchJni::getHandle(env, jobj); +void Java_org_rocksdb_WriteBatch_remove__J_3BIJ( + JNIEnv* env, jobject jobj, jlong jwb_handle, + jbyteArray jkey, jint jkey_len, jlong jcf_handle) { + auto* wb = reinterpret_cast(jwb_handle); assert(wb != nullptr); - - jbyte* blob = env->GetByteArrayElements(jblob, nullptr); - rocksdb::Slice blob_slice(reinterpret_cast(blob), jblob_len); - wb->PutLogData(blob_slice); - env->ReleaseByteArrayElements(jblob, blob, JNI_ABORT); + auto* cf_handle = reinterpret_cast(jcf_handle); + assert(cf_handle != nullptr); + auto remove = [&wb, &cf_handle] (rocksdb::Slice key) { + wb->Delete(cf_handle, key); + }; + rocksdb::JniUtil::k_op(remove, env, jobj, jkey, jkey_len); } /* * Class: org_rocksdb_WriteBatch - * Method: disposeInternal - * Signature: (J)V + * Method: deleteRange + * Signature: (J[BI[BI)V */ -void Java_org_rocksdb_WriteBatch_disposeInternal( - JNIEnv* env, jobject jobj, jlong handle) { - delete reinterpret_cast(handle); +JNIEXPORT void JNICALL Java_org_rocksdb_WriteBatch_deleteRange__J_3BI_3BI( + JNIEnv*, jobject, jlong, jbyteArray, jint, jbyteArray, jint); + +void Java_org_rocksdb_WriteBatch_deleteRange__J_3BI_3BI( + JNIEnv* env, jobject jobj, jlong jwb_handle, jbyteArray jbegin_key, + jint jbegin_key_len, jbyteArray jend_key, jint jend_key_len) { + auto* wb = reinterpret_cast(jwb_handle); + assert(wb != nullptr); + auto deleteRange = [&wb](rocksdb::Slice beginKey, rocksdb::Slice endKey) { + wb->DeleteRange(beginKey, endKey); + }; + rocksdb::JniUtil::kv_op(deleteRange, env, jobj, jbegin_key, jbegin_key_len, + jend_key, jend_key_len); } /* - * Class: org_rocksdb_WriteBatchInternal - * Method: setSequence - * Signature: (Lorg/rocksdb/WriteBatch;J)V + * Class: org_rocksdb_WriteBatch + * Method: deleteRange + * Signature: (J[BI[BIJ)V */ -void Java_org_rocksdb_WriteBatchInternal_setSequence( - JNIEnv* env, jclass jclazz, jobject jobj, jlong jsn) { - rocksdb::WriteBatch* wb = rocksdb::WriteBatchJni::getHandle(env, jobj); +void Java_org_rocksdb_WriteBatch_deleteRange__J_3BI_3BIJ( + JNIEnv* env, jobject jobj, jlong jwb_handle, jbyteArray jbegin_key, + jint jbegin_key_len, jbyteArray jend_key, jint jend_key_len, + jlong jcf_handle) { + auto* wb = reinterpret_cast(jwb_handle); assert(wb != nullptr); - - rocksdb::WriteBatchInternal::SetSequence( - wb, static_cast(jsn)); + auto* cf_handle = reinterpret_cast(jcf_handle); + assert(cf_handle != nullptr); + auto deleteRange = [&wb, &cf_handle](rocksdb::Slice beginKey, + rocksdb::Slice endKey) { + wb->DeleteRange(cf_handle, beginKey, endKey); + }; + rocksdb::JniUtil::kv_op(deleteRange, env, jobj, jbegin_key, jbegin_key_len, + jend_key, jend_key_len); } /* - * Class: org_rocksdb_WriteBatchInternal - * Method: sequence - * Signature: (Lorg/rocksdb/WriteBatch;)J + * Class: org_rocksdb_WriteBatch + * Method: putLogData + * Signature: (J[BI)V */ -jlong Java_org_rocksdb_WriteBatchInternal_sequence( - JNIEnv* env, jclass jclazz, jobject jobj) { - rocksdb::WriteBatch* wb = rocksdb::WriteBatchJni::getHandle(env, jobj); +void Java_org_rocksdb_WriteBatch_putLogData( + JNIEnv* env, jobject jobj, jlong jwb_handle, jbyteArray jblob, + jint jblob_len) { + auto* wb = reinterpret_cast(jwb_handle); assert(wb != nullptr); - - return static_cast(rocksdb::WriteBatchInternal::Sequence(wb)); + auto putLogData = [&wb] (rocksdb::Slice blob) { + wb->PutLogData(blob); + }; + rocksdb::JniUtil::k_op(putLogData, env, jobj, jblob, jblob_len); } /* - * Class: org_rocksdb_WriteBatchInternal - * Method: append - * Signature: (Lorg/rocksdb/WriteBatch;Lorg/rocksdb/WriteBatch;)V + * Class: org_rocksdb_WriteBatch + * Method: iterate + * Signature: (JJ)V */ -void Java_org_rocksdb_WriteBatchInternal_append( - JNIEnv* env, jclass jclazz, jobject jwb1, jobject jwb2) { - rocksdb::WriteBatch* wb1 = rocksdb::WriteBatchJni::getHandle(env, jwb1); - assert(wb1 != nullptr); - rocksdb::WriteBatch* wb2 = rocksdb::WriteBatchJni::getHandle(env, jwb2); - assert(wb2 != nullptr); +void Java_org_rocksdb_WriteBatch_iterate( + JNIEnv* env , jobject jobj, jlong jwb_handle, jlong handlerHandle) { + auto* wb = reinterpret_cast(jwb_handle); + assert(wb != nullptr); + + rocksdb::Status s = wb->Iterate( + reinterpret_cast(handlerHandle)); - rocksdb::WriteBatchInternal::Append(wb1, wb2); + if (s.ok()) { + return; + } + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); } /* - * Class: org_rocksdb_WriteBatchTest - * Method: getContents - * Signature: (Lorg/rocksdb/WriteBatch;)[B + * Class: org_rocksdb_WriteBatch + * Method: disposeInternal + * Signature: (J)V */ -jbyteArray Java_org_rocksdb_WriteBatchTest_getContents( - JNIEnv* env, jclass jclazz, jobject jobj) { - rocksdb::WriteBatch* b = rocksdb::WriteBatchJni::getHandle(env, jobj); - assert(b != nullptr); - - // todo: Currently the following code is directly copied from - // db/write_bench_test.cc. It could be implemented in java once - // all the necessary components can be accessed via jni api. - - rocksdb::InternalKeyComparator cmp(rocksdb::BytewiseComparator()); - auto factory = std::make_shared(); - rocksdb::Options options; - options.memtable_factory = factory; - rocksdb::MemTable* mem = new rocksdb::MemTable(cmp, options); - mem->Ref(); - std::string state; - rocksdb::ColumnFamilyMemTablesDefault cf_mems_default(mem, &options); - rocksdb::Status s = - rocksdb::WriteBatchInternal::InsertInto(b, &cf_mems_default); - int count = 0; - rocksdb::Iterator* iter = mem->NewIterator(rocksdb::ReadOptions()); - for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { - rocksdb::ParsedInternalKey ikey; - memset(reinterpret_cast(&ikey), 0, sizeof(ikey)); - ASSERT_TRUE(rocksdb::ParseInternalKey(iter->key(), &ikey)); - switch (ikey.type) { - case rocksdb::kTypeValue: - state.append("Put("); - state.append(ikey.user_key.ToString()); - state.append(", "); - state.append(iter->value().ToString()); - state.append(")"); - count++; - break; - case rocksdb::kTypeMerge: - state.append("Merge("); - state.append(ikey.user_key.ToString()); - state.append(", "); - state.append(iter->value().ToString()); - state.append(")"); - count++; - break; - case rocksdb::kTypeDeletion: - state.append("Delete("); - state.append(ikey.user_key.ToString()); - state.append(")"); - count++; - break; - default: - assert(false); - break; - } - state.append("@"); - state.append(rocksdb::NumberToString(ikey.sequence)); - } - delete iter; - if (!s.ok()) { - state.append(s.ToString()); - } else if (count != rocksdb::WriteBatchInternal::Count(b)) { - state.append("CountMismatch()"); - } - delete mem->Unref(); +void Java_org_rocksdb_WriteBatch_disposeInternal( + JNIEnv* env, jobject jobj, jlong handle) { + auto* wb = reinterpret_cast(handle); + assert(wb != nullptr); + delete wb; +} - jbyteArray jstate = env->NewByteArray(state.size()); - env->SetByteArrayRegion( - jstate, 0, state.size(), - reinterpret_cast(state.c_str())); +/* + * Class: org_rocksdb_WriteBatch_Handler + * Method: createNewHandler0 + * Signature: ()J + */ +jlong Java_org_rocksdb_WriteBatch_00024Handler_createNewHandler0( + JNIEnv* env, jobject jobj) { + auto* wbjnic = new rocksdb::WriteBatchHandlerJniCallback(env, jobj); + return reinterpret_cast(wbjnic); +} - return jstate; +/* + * Class: org_rocksdb_WriteBatch_Handler + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_WriteBatch_00024Handler_disposeInternal( + JNIEnv* env, jobject jobj, jlong handle) { + auto* wbjnic = + reinterpret_cast(handle); + assert(wbjnic != nullptr); + delete wbjnic; } diff --git a/java/rocksjni/write_batch_test.cc b/java/rocksjni/write_batch_test.cc new file mode 100644 index 00000000000..199ad239d79 --- /dev/null +++ b/java/rocksjni/write_batch_test.cc @@ -0,0 +1,161 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ and enables +// calling c++ rocksdb::WriteBatch methods testing from Java side. +#include + +#include "db/memtable.h" +#include "db/write_batch_internal.h" +#include "include/org_rocksdb_WriteBatch.h" +#include "include/org_rocksdb_WriteBatchTest.h" +#include "include/org_rocksdb_WriteBatchTestInternalHelper.h" +#include "include/org_rocksdb_WriteBatch_Handler.h" +#include "options/cf_options.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/status.h" +#include "rocksdb/write_batch.h" +#include "rocksdb/write_buffer_manager.h" +#include "rocksjni/portal.h" +#include "table/scoped_arena_iterator.h" +#include "util/string_util.h" +#include "util/testharness.h" + +/* + * Class: org_rocksdb_WriteBatchTest + * Method: getContents + * Signature: (J)[B + */ +jbyteArray Java_org_rocksdb_WriteBatchTest_getContents( + JNIEnv* env, jclass jclazz, jlong jwb_handle) { + auto* b = reinterpret_cast(jwb_handle); + assert(b != nullptr); + + // todo: Currently the following code is directly copied from + // db/write_bench_test.cc. It could be implemented in java once + // all the necessary components can be accessed via jni api. + + rocksdb::InternalKeyComparator cmp(rocksdb::BytewiseComparator()); + auto factory = std::make_shared(); + rocksdb::Options options; + rocksdb::WriteBufferManager wb(options.db_write_buffer_size); + options.memtable_factory = factory; + rocksdb::MemTable* mem = new rocksdb::MemTable( + cmp, rocksdb::ImmutableCFOptions(options), + rocksdb::MutableCFOptions(options), &wb, rocksdb::kMaxSequenceNumber, + 0 /* column_family_id */); + mem->Ref(); + std::string state; + rocksdb::ColumnFamilyMemTablesDefault cf_mems_default(mem); + rocksdb::Status s = + rocksdb::WriteBatchInternal::InsertInto(b, &cf_mems_default, nullptr); + int count = 0; + rocksdb::Arena arena; + rocksdb::ScopedArenaIterator iter(mem->NewIterator( + rocksdb::ReadOptions(), &arena)); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + rocksdb::ParsedInternalKey ikey; + ikey.clear(); + bool parsed = rocksdb::ParseInternalKey(iter->key(), &ikey); + if (!parsed) { + assert(parsed); + } + switch (ikey.type) { + case rocksdb::kTypeValue: + state.append("Put("); + state.append(ikey.user_key.ToString()); + state.append(", "); + state.append(iter->value().ToString()); + state.append(")"); + count++; + break; + case rocksdb::kTypeMerge: + state.append("Merge("); + state.append(ikey.user_key.ToString()); + state.append(", "); + state.append(iter->value().ToString()); + state.append(")"); + count++; + break; + case rocksdb::kTypeDeletion: + state.append("Delete("); + state.append(ikey.user_key.ToString()); + state.append(")"); + count++; + break; + default: + assert(false); + break; + } + state.append("@"); + state.append(rocksdb::NumberToString(ikey.sequence)); + } + if (!s.ok()) { + state.append(s.ToString()); + } else if (count != rocksdb::WriteBatchInternal::Count(b)) { + state.append("CountMismatch()"); + } + delete mem->Unref(); + + jbyteArray jstate = env->NewByteArray(static_cast(state.size())); + if(jstate == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + env->SetByteArrayRegion(jstate, 0, static_cast(state.size()), + const_cast(reinterpret_cast(state.c_str()))); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jstate); + return nullptr; + } + + return jstate; +} + +/* + * Class: org_rocksdb_WriteBatchTestInternalHelper + * Method: setSequence + * Signature: (JJ)V + */ +void Java_org_rocksdb_WriteBatchTestInternalHelper_setSequence( + JNIEnv* env, jclass jclazz, jlong jwb_handle, jlong jsn) { + auto* wb = reinterpret_cast(jwb_handle); + assert(wb != nullptr); + + rocksdb::WriteBatchInternal::SetSequence( + wb, static_cast(jsn)); +} + +/* + * Class: org_rocksdb_WriteBatchTestInternalHelper + * Method: sequence + * Signature: (J)J + */ +jlong Java_org_rocksdb_WriteBatchTestInternalHelper_sequence( + JNIEnv* env, jclass jclazz, jlong jwb_handle) { + auto* wb = reinterpret_cast(jwb_handle); + assert(wb != nullptr); + + return static_cast(rocksdb::WriteBatchInternal::Sequence(wb)); +} + +/* + * Class: org_rocksdb_WriteBatchTestInternalHelper + * Method: append + * Signature: (JJ)V + */ +void Java_org_rocksdb_WriteBatchTestInternalHelper_append( + JNIEnv* env, jclass jclazz, jlong jwb_handle_1, jlong jwb_handle_2) { + auto* wb1 = reinterpret_cast(jwb_handle_1); + assert(wb1 != nullptr); + auto* wb2 = reinterpret_cast(jwb_handle_2); + assert(wb2 != nullptr); + + rocksdb::WriteBatchInternal::Append(wb1, wb2); +} diff --git a/java/rocksjni/write_batch_with_index.cc b/java/rocksjni/write_batch_with_index.cc new file mode 100644 index 00000000000..53f2a11d121 --- /dev/null +++ b/java/rocksjni/write_batch_with_index.cc @@ -0,0 +1,582 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the "bridge" between Java and C++ and enables +// calling c++ rocksdb::WriteBatchWithIndex methods from Java side. + +#include "include/org_rocksdb_WBWIRocksIterator.h" +#include "include/org_rocksdb_WriteBatchWithIndex.h" +#include "rocksdb/comparator.h" +#include "rocksdb/utilities/write_batch_with_index.h" +#include "rocksjni/portal.h" + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: newWriteBatchWithIndex + * Signature: ()J + */ +jlong Java_org_rocksdb_WriteBatchWithIndex_newWriteBatchWithIndex__( + JNIEnv* env, jclass jcls) { + auto* wbwi = new rocksdb::WriteBatchWithIndex(); + return reinterpret_cast(wbwi); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: newWriteBatchWithIndex + * Signature: (Z)J + */ +jlong Java_org_rocksdb_WriteBatchWithIndex_newWriteBatchWithIndex__Z( + JNIEnv* env, jclass jcls, jboolean joverwrite_key) { + auto* wbwi = + new rocksdb::WriteBatchWithIndex(rocksdb::BytewiseComparator(), 0, + static_cast(joverwrite_key)); + return reinterpret_cast(wbwi); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: newWriteBatchWithIndex + * Signature: (JIZ)J + */ +jlong Java_org_rocksdb_WriteBatchWithIndex_newWriteBatchWithIndex__JIZ( + JNIEnv* env, jclass jcls, jlong jfallback_index_comparator_handle, + jint jreserved_bytes, jboolean joverwrite_key) { + auto* wbwi = + new rocksdb::WriteBatchWithIndex( + reinterpret_cast(jfallback_index_comparator_handle), + static_cast(jreserved_bytes), static_cast(joverwrite_key)); + return reinterpret_cast(wbwi); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: count0 + * Signature: (J)I + */ +jint Java_org_rocksdb_WriteBatchWithIndex_count0( + JNIEnv* env, jobject jobj, jlong jwbwi_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + + return static_cast(wbwi->GetWriteBatch()->Count()); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: put + * Signature: (J[BI[BI)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_put__J_3BI_3BI( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jkey, + jint jkey_len, jbyteArray jentry_value, jint jentry_value_len) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + auto put = [&wbwi] (rocksdb::Slice key, rocksdb::Slice value) { + wbwi->Put(key, value); + }; + rocksdb::JniUtil::kv_op(put, env, jobj, jkey, jkey_len, jentry_value, + jentry_value_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: put + * Signature: (J[BI[BIJ)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_put__J_3BI_3BIJ( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jkey, + jint jkey_len, jbyteArray jentry_value, jint jentry_value_len, + jlong jcf_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + auto* cf_handle = reinterpret_cast(jcf_handle); + assert(cf_handle != nullptr); + auto put = [&wbwi, &cf_handle] (rocksdb::Slice key, rocksdb::Slice value) { + wbwi->Put(cf_handle, key, value); + }; + rocksdb::JniUtil::kv_op(put, env, jobj, jkey, jkey_len, jentry_value, + jentry_value_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: merge + * Signature: (J[BI[BI)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_merge__J_3BI_3BI( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jkey, + jint jkey_len, jbyteArray jentry_value, jint jentry_value_len) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + auto merge = [&wbwi] (rocksdb::Slice key, rocksdb::Slice value) { + wbwi->Merge(key, value); + }; + rocksdb::JniUtil::kv_op(merge, env, jobj, jkey, jkey_len, jentry_value, + jentry_value_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: merge + * Signature: (J[BI[BIJ)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_merge__J_3BI_3BIJ( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jkey, + jint jkey_len, jbyteArray jentry_value, jint jentry_value_len, + jlong jcf_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + auto* cf_handle = reinterpret_cast(jcf_handle); + assert(cf_handle != nullptr); + auto merge = [&wbwi, &cf_handle] (rocksdb::Slice key, rocksdb::Slice value) { + wbwi->Merge(cf_handle, key, value); + }; + rocksdb::JniUtil::kv_op(merge, env, jobj, jkey, jkey_len, jentry_value, + jentry_value_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: remove + * Signature: (J[BI)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_remove__J_3BI( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jkey, + jint jkey_len) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + auto remove = [&wbwi] (rocksdb::Slice key) { + wbwi->Delete(key); + }; + rocksdb::JniUtil::k_op(remove, env, jobj, jkey, jkey_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: remove + * Signature: (J[BIJ)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_remove__J_3BIJ( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jkey, + jint jkey_len, jlong jcf_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + auto* cf_handle = reinterpret_cast(jcf_handle); + assert(cf_handle != nullptr); + auto remove = [&wbwi, &cf_handle] (rocksdb::Slice key) { + wbwi->Delete(cf_handle, key); + }; + rocksdb::JniUtil::k_op(remove, env, jobj, jkey, jkey_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: deleteRange + * Signature: (J[BI[BI)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_deleteRange__J_3BI_3BI( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jbegin_key, + jint jbegin_key_len, jbyteArray jend_key, jint jend_key_len) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + auto deleteRange = [&wbwi](rocksdb::Slice beginKey, rocksdb::Slice endKey) { + wbwi->DeleteRange(beginKey, endKey); + }; + rocksdb::JniUtil::kv_op(deleteRange, env, jobj, jbegin_key, jbegin_key_len, + jend_key, jend_key_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: deleteRange + * Signature: (J[BI[BIJ)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_deleteRange__J_3BI_3BIJ( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jbegin_key, + jint jbegin_key_len, jbyteArray jend_key, jint jend_key_len, + jlong jcf_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + auto* cf_handle = reinterpret_cast(jcf_handle); + assert(cf_handle != nullptr); + auto deleteRange = [&wbwi, &cf_handle](rocksdb::Slice beginKey, + rocksdb::Slice endKey) { + wbwi->DeleteRange(cf_handle, beginKey, endKey); + }; + rocksdb::JniUtil::kv_op(deleteRange, env, jobj, jbegin_key, jbegin_key_len, + jend_key, jend_key_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: putLogData + * Signature: (J[BI)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_putLogData( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jblob, + jint jblob_len) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + auto putLogData = [&wbwi] (rocksdb::Slice blob) { + wbwi->PutLogData(blob); + }; + rocksdb::JniUtil::k_op(putLogData, env, jobj, jblob, jblob_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: clear + * Signature: (J)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_clear0( + JNIEnv* env, jobject jobj, jlong jwbwi_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + + wbwi->Clear(); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: setSavePoint0 + * Signature: (J)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_setSavePoint0( + JNIEnv* env, jobject jobj, jlong jwbwi_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + + wbwi->SetSavePoint(); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: rollbackToSavePoint0 + * Signature: (J)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_rollbackToSavePoint0( + JNIEnv* env, jobject jobj, jlong jwbwi_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + assert(wbwi != nullptr); + + auto s = wbwi->RollbackToSavePoint(); + + if (s.ok()) { + return; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: iterator0 + * Signature: (J)J + */ +jlong Java_org_rocksdb_WriteBatchWithIndex_iterator0( + JNIEnv* env, jobject jobj, jlong jwbwi_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + auto* wbwi_iterator = wbwi->NewIterator(); + return reinterpret_cast(wbwi_iterator); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: iterator1 + * Signature: (JJ)J + */ +jlong Java_org_rocksdb_WriteBatchWithIndex_iterator1( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jlong jcf_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + auto* wbwi_iterator = wbwi->NewIterator(cf_handle); + return reinterpret_cast(wbwi_iterator); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: iteratorWithBase + * Signature: (JJJ)J + */ +jlong Java_org_rocksdb_WriteBatchWithIndex_iteratorWithBase( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jlong jcf_handle, + jlong jbi_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + auto* base_iterator = reinterpret_cast(jbi_handle); + auto* iterator = wbwi->NewIteratorWithBase(cf_handle, base_iterator); + return reinterpret_cast(iterator); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: getFromBatch + * Signature: (JJ[BI)[B + */ +jbyteArray JNICALL Java_org_rocksdb_WriteBatchWithIndex_getFromBatch__JJ_3BI( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jlong jdbopt_handle, + jbyteArray jkey, jint jkey_len) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + auto* dbopt = reinterpret_cast(jdbopt_handle); + + auto getter = [&wbwi, &dbopt](const rocksdb::Slice& key, std::string* value) { + return wbwi->GetFromBatch(*dbopt, key, value); + }; + + return rocksdb::JniUtil::v_op(getter, env, jkey, jkey_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: getFromBatch + * Signature: (JJ[BIJ)[B + */ +jbyteArray Java_org_rocksdb_WriteBatchWithIndex_getFromBatch__JJ_3BIJ( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jlong jdbopt_handle, + jbyteArray jkey, jint jkey_len, jlong jcf_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + auto* dbopt = reinterpret_cast(jdbopt_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + + auto getter = + [&wbwi, &cf_handle, &dbopt](const rocksdb::Slice& key, + std::string* value) { + return wbwi->GetFromBatch(cf_handle, *dbopt, key, value); + }; + + return rocksdb::JniUtil::v_op(getter, env, jkey, jkey_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: getFromBatchAndDB + * Signature: (JJJ[BI)[B + */ +jbyteArray Java_org_rocksdb_WriteBatchWithIndex_getFromBatchAndDB__JJJ_3BI( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jlong jdb_handle, + jlong jreadopt_handle, jbyteArray jkey, jint jkey_len) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + auto* db = reinterpret_cast(jdb_handle); + auto* readopt = reinterpret_cast(jreadopt_handle); + + auto getter = + [&wbwi, &db, &readopt](const rocksdb::Slice& key, std::string* value) { + return wbwi->GetFromBatchAndDB(db, *readopt, key, value); + }; + + return rocksdb::JniUtil::v_op(getter, env, jkey, jkey_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: getFromBatchAndDB + * Signature: (JJJ[BIJ)[B + */ +jbyteArray Java_org_rocksdb_WriteBatchWithIndex_getFromBatchAndDB__JJJ_3BIJ( + JNIEnv* env, jobject jobj, jlong jwbwi_handle, jlong jdb_handle, + jlong jreadopt_handle, jbyteArray jkey, jint jkey_len, jlong jcf_handle) { + auto* wbwi = reinterpret_cast(jwbwi_handle); + auto* db = reinterpret_cast(jdb_handle); + auto* readopt = reinterpret_cast(jreadopt_handle); + auto* cf_handle = reinterpret_cast(jcf_handle); + + auto getter = + [&wbwi, &db, &cf_handle, &readopt](const rocksdb::Slice& key, + std::string* value) { + return wbwi->GetFromBatchAndDB(db, *readopt, cf_handle, key, value); + }; + + return rocksdb::JniUtil::v_op(getter, env, jkey, jkey_len); +} + +/* + * Class: org_rocksdb_WriteBatchWithIndex + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_WriteBatchWithIndex_disposeInternal( + JNIEnv* env, jobject jobj, jlong handle) { + auto* wbwi = reinterpret_cast(handle); + assert(wbwi != nullptr); + delete wbwi; +} + +/* WBWIRocksIterator below */ + +/* + * Class: org_rocksdb_WBWIRocksIterator + * Method: disposeInternal + * Signature: (J)V + */ +void Java_org_rocksdb_WBWIRocksIterator_disposeInternal( + JNIEnv* env, jobject jobj, jlong handle) { + auto* it = reinterpret_cast(handle); + assert(it != nullptr); + delete it; +} + +/* + * Class: org_rocksdb_WBWIRocksIterator + * Method: isValid0 + * Signature: (J)Z + */ +jboolean Java_org_rocksdb_WBWIRocksIterator_isValid0( + JNIEnv* env, jobject jobj, jlong handle) { + return reinterpret_cast(handle)->Valid(); +} + +/* + * Class: org_rocksdb_WBWIRocksIterator + * Method: seekToFirst0 + * Signature: (J)V + */ +void Java_org_rocksdb_WBWIRocksIterator_seekToFirst0( + JNIEnv* env, jobject jobj, jlong handle) { + reinterpret_cast(handle)->SeekToFirst(); +} + +/* + * Class: org_rocksdb_WBWIRocksIterator + * Method: seekToLast0 + * Signature: (J)V + */ +void Java_org_rocksdb_WBWIRocksIterator_seekToLast0( + JNIEnv* env, jobject jobj, jlong handle) { + reinterpret_cast(handle)->SeekToLast(); +} + +/* + * Class: org_rocksdb_WBWIRocksIterator + * Method: next0 + * Signature: (J)V + */ +void Java_org_rocksdb_WBWIRocksIterator_next0( + JNIEnv* env, jobject jobj, jlong handle) { + reinterpret_cast(handle)->Next(); +} + +/* + * Class: org_rocksdb_WBWIRocksIterator + * Method: prev0 + * Signature: (J)V + */ +void Java_org_rocksdb_WBWIRocksIterator_prev0( + JNIEnv* env, jobject jobj, jlong handle) { + reinterpret_cast(handle)->Prev(); +} + +/* + * Class: org_rocksdb_WBWIRocksIterator + * Method: seek0 + * Signature: (J[BI)V + */ +void Java_org_rocksdb_WBWIRocksIterator_seek0( + JNIEnv* env, jobject jobj, jlong handle, jbyteArray jtarget, + jint jtarget_len) { + auto* it = reinterpret_cast(handle); + jbyte* target = env->GetByteArrayElements(jtarget, nullptr); + if(target == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + rocksdb::Slice target_slice( + reinterpret_cast(target), jtarget_len); + + it->Seek(target_slice); + + env->ReleaseByteArrayElements(jtarget, target, JNI_ABORT); +} + +/* + * Class: org_rocksdb_WBWIRocksIterator + * Method: status0 + * Signature: (J)V + */ +void Java_org_rocksdb_WBWIRocksIterator_status0( + JNIEnv* env, jobject jobj, jlong handle) { + auto* it = reinterpret_cast(handle); + rocksdb::Status s = it->status(); + + if (s.ok()) { + return; + } + + rocksdb::RocksDBExceptionJni::ThrowNew(env, s); +} + +/* + * Class: org_rocksdb_WBWIRocksIterator + * Method: entry1 + * Signature: (J)[J + */ +jlongArray Java_org_rocksdb_WBWIRocksIterator_entry1( + JNIEnv* env, jobject jobj, jlong handle) { + auto* it = reinterpret_cast(handle); + const rocksdb::WriteEntry& we = it->Entry(); + + jlong results[3]; + + //set the type of the write entry + switch (we.type) { + case rocksdb::kPutRecord: + results[0] = 0x1; + break; + + case rocksdb::kMergeRecord: + results[0] = 0x2; + break; + + case rocksdb::kDeleteRecord: + results[0] = 0x4; + break; + + case rocksdb::kLogDataRecord: + results[0] = 0x8; + break; + + default: + results[0] = 0x0; + } + + // key_slice and value_slice will be freed by org.rocksdb.DirectSlice#close + + auto* key_slice = new rocksdb::Slice(we.key.data(), we.key.size()); + results[1] = reinterpret_cast(key_slice); + if (we.type == rocksdb::kDeleteRecord + || we.type == rocksdb::kLogDataRecord) { + // set native handle of value slice to null if no value available + results[2] = 0; + } else { + auto* value_slice = new rocksdb::Slice(we.value.data(), we.value.size()); + results[2] = reinterpret_cast(value_slice); + } + + jlongArray jresults = env->NewLongArray(3); + if(jresults == nullptr) { + // exception thrown: OutOfMemoryError + if(results[2] != 0) { + auto* value_slice = reinterpret_cast(results[2]); + delete value_slice; + } + delete key_slice; + return nullptr; + } + + env->SetLongArrayRegion(jresults, 0, 3, results); + if(env->ExceptionCheck()) { + // exception thrown: ArrayIndexOutOfBoundsException + env->DeleteLocalRef(jresults); + if(results[2] != 0) { + auto* value_slice = reinterpret_cast(results[2]); + delete value_slice; + } + delete key_slice; + return nullptr; + } + + return jresults; +} diff --git a/java/rocksjni/writebatchhandlerjnicallback.cc b/java/rocksjni/writebatchhandlerjnicallback.cc new file mode 100644 index 00000000000..0f00766c532 --- /dev/null +++ b/java/rocksjni/writebatchhandlerjnicallback.cc @@ -0,0 +1,306 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the callback "bridge" between Java and C++ for +// rocksdb::Comparator. + +#include "rocksjni/writebatchhandlerjnicallback.h" +#include "rocksjni/portal.h" + +namespace rocksdb { +WriteBatchHandlerJniCallback::WriteBatchHandlerJniCallback( + JNIEnv* env, jobject jWriteBatchHandler) + : m_env(env) { + + // Note: we want to access the Java WriteBatchHandler instance + // across multiple method calls, so we create a global ref + assert(jWriteBatchHandler != nullptr); + m_jWriteBatchHandler = env->NewGlobalRef(jWriteBatchHandler); + if(m_jWriteBatchHandler == nullptr) { + // exception thrown: OutOfMemoryError + return; + } + + m_jPutMethodId = WriteBatchHandlerJni::getPutMethodId(env); + if(m_jPutMethodId == nullptr) { + // exception thrown + return; + } + + m_jMergeMethodId = WriteBatchHandlerJni::getMergeMethodId(env); + if(m_jMergeMethodId == nullptr) { + // exception thrown + return; + } + + m_jDeleteMethodId = WriteBatchHandlerJni::getDeleteMethodId(env); + if(m_jDeleteMethodId == nullptr) { + // exception thrown + return; + } + + m_jDeleteRangeMethodId = WriteBatchHandlerJni::getDeleteRangeMethodId(env); + if (m_jDeleteRangeMethodId == nullptr) { + // exception thrown + return; + } + + m_jLogDataMethodId = WriteBatchHandlerJni::getLogDataMethodId(env); + if(m_jLogDataMethodId == nullptr) { + // exception thrown + return; + } + + m_jContinueMethodId = WriteBatchHandlerJni::getContinueMethodId(env); + if(m_jContinueMethodId == nullptr) { + // exception thrown + return; + } +} + +void WriteBatchHandlerJniCallback::Put(const Slice& key, const Slice& value) { + const jbyteArray j_key = sliceToJArray(key); + if(j_key == nullptr) { + // exception thrown + if(m_env->ExceptionCheck()) { + m_env->ExceptionDescribe(); + } + return; + } + + const jbyteArray j_value = sliceToJArray(value); + if(j_value == nullptr) { + // exception thrown + if(m_env->ExceptionCheck()) { + m_env->ExceptionDescribe(); + } + if(j_key != nullptr) { + m_env->DeleteLocalRef(j_key); + } + return; + } + + m_env->CallVoidMethod( + m_jWriteBatchHandler, + m_jPutMethodId, + j_key, + j_value); + if(m_env->ExceptionCheck()) { + // exception thrown + m_env->ExceptionDescribe(); + if(j_value != nullptr) { + m_env->DeleteLocalRef(j_value); + } + if(j_key != nullptr) { + m_env->DeleteLocalRef(j_key); + } + return; + } + + if(j_value != nullptr) { + m_env->DeleteLocalRef(j_value); + } + if(j_key != nullptr) { + m_env->DeleteLocalRef(j_key); + } +} + +void WriteBatchHandlerJniCallback::Merge(const Slice& key, const Slice& value) { + const jbyteArray j_key = sliceToJArray(key); + if(j_key == nullptr) { + // exception thrown + if(m_env->ExceptionCheck()) { + m_env->ExceptionDescribe(); + } + return; + } + + const jbyteArray j_value = sliceToJArray(value); + if(j_value == nullptr) { + // exception thrown + if(m_env->ExceptionCheck()) { + m_env->ExceptionDescribe(); + } + if(j_key != nullptr) { + m_env->DeleteLocalRef(j_key); + } + return; + } + + m_env->CallVoidMethod( + m_jWriteBatchHandler, + m_jMergeMethodId, + j_key, + j_value); + if(m_env->ExceptionCheck()) { + // exception thrown + m_env->ExceptionDescribe(); + if(j_value != nullptr) { + m_env->DeleteLocalRef(j_value); + } + if(j_key != nullptr) { + m_env->DeleteLocalRef(j_key); + } + return; + } + + if(j_value != nullptr) { + m_env->DeleteLocalRef(j_value); + } + if(j_key != nullptr) { + m_env->DeleteLocalRef(j_key); + } +} + +void WriteBatchHandlerJniCallback::Delete(const Slice& key) { + const jbyteArray j_key = sliceToJArray(key); + if(j_key == nullptr) { + // exception thrown + if(m_env->ExceptionCheck()) { + m_env->ExceptionDescribe(); + } + return; + } + + m_env->CallVoidMethod( + m_jWriteBatchHandler, + m_jDeleteMethodId, + j_key); + if(m_env->ExceptionCheck()) { + // exception thrown + m_env->ExceptionDescribe(); + if(j_key != nullptr) { + m_env->DeleteLocalRef(j_key); + } + return; + } + + if(j_key != nullptr) { + m_env->DeleteLocalRef(j_key); + } +} + +void WriteBatchHandlerJniCallback::DeleteRange(const Slice& beginKey, + const Slice& endKey) { + const jbyteArray j_beginKey = sliceToJArray(beginKey); + if (j_beginKey == nullptr) { + // exception thrown + if (m_env->ExceptionCheck()) { + m_env->ExceptionDescribe(); + } + return; + } + + const jbyteArray j_endKey = sliceToJArray(beginKey); + if (j_endKey == nullptr) { + // exception thrown + if (m_env->ExceptionCheck()) { + m_env->ExceptionDescribe(); + } + return; + } + + m_env->CallVoidMethod(m_jWriteBatchHandler, m_jDeleteRangeMethodId, + j_beginKey, j_endKey); + if (m_env->ExceptionCheck()) { + // exception thrown + m_env->ExceptionDescribe(); + if (j_beginKey != nullptr) { + m_env->DeleteLocalRef(j_beginKey); + } + if (j_endKey != nullptr) { + m_env->DeleteLocalRef(j_endKey); + } + return; + } + + if (j_beginKey != nullptr) { + m_env->DeleteLocalRef(j_beginKey); + } + + if (j_endKey != nullptr) { + m_env->DeleteLocalRef(j_endKey); + } +} + +void WriteBatchHandlerJniCallback::LogData(const Slice& blob) { + const jbyteArray j_blob = sliceToJArray(blob); + if(j_blob == nullptr) { + // exception thrown + if(m_env->ExceptionCheck()) { + m_env->ExceptionDescribe(); + } + return; + } + + m_env->CallVoidMethod( + m_jWriteBatchHandler, + m_jLogDataMethodId, + j_blob); + if(m_env->ExceptionCheck()) { + // exception thrown + m_env->ExceptionDescribe(); + if(j_blob != nullptr) { + m_env->DeleteLocalRef(j_blob); + } + return; + } + + if(j_blob != nullptr) { + m_env->DeleteLocalRef(j_blob); + } +} + +bool WriteBatchHandlerJniCallback::Continue() { + jboolean jContinue = m_env->CallBooleanMethod( + m_jWriteBatchHandler, + m_jContinueMethodId); + if(m_env->ExceptionCheck()) { + // exception thrown + m_env->ExceptionDescribe(); + } + + return static_cast(jContinue == JNI_TRUE); +} + +/* + * Creates a Java Byte Array from the data in a Slice + * + * When calling this function + * you must remember to call env->DeleteLocalRef + * on the result after you have finished with it + * + * @param s A Slice to convery to a Java byte array + * + * @return A reference to a Java byte array, or a nullptr if an + * exception occurs + */ +jbyteArray WriteBatchHandlerJniCallback::sliceToJArray(const Slice& s) { + jbyteArray ja = m_env->NewByteArray(static_cast(s.size())); + if(ja == nullptr) { + // exception thrown: OutOfMemoryError + return nullptr; + } + + m_env->SetByteArrayRegion( + ja, 0, static_cast(s.size()), + const_cast(reinterpret_cast(s.data()))); + if(m_env->ExceptionCheck()) { + if(ja != nullptr) { + m_env->DeleteLocalRef(ja); + } + // exception thrown: ArrayIndexOutOfBoundsException + return nullptr; + } + + return ja; +} + +WriteBatchHandlerJniCallback::~WriteBatchHandlerJniCallback() { + if(m_jWriteBatchHandler != nullptr) { + m_env->DeleteGlobalRef(m_jWriteBatchHandler); + } +} +} // namespace rocksdb diff --git a/java/rocksjni/writebatchhandlerjnicallback.h b/java/rocksjni/writebatchhandlerjnicallback.h new file mode 100644 index 00000000000..5d3dee3b1a8 --- /dev/null +++ b/java/rocksjni/writebatchhandlerjnicallback.h @@ -0,0 +1,48 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// This file implements the callback "bridge" between Java and C++ for +// rocksdb::WriteBatch::Handler. + +#ifndef JAVA_ROCKSJNI_WRITEBATCHHANDLERJNICALLBACK_H_ +#define JAVA_ROCKSJNI_WRITEBATCHHANDLERJNICALLBACK_H_ + +#include +#include "rocksdb/write_batch.h" + +namespace rocksdb { +/** + * This class acts as a bridge between C++ + * and Java. The methods in this class will be + * called back from the RocksDB storage engine (C++) + * which calls the appropriate Java method. + * This enables Write Batch Handlers to be implemented in Java. + */ +class WriteBatchHandlerJniCallback : public WriteBatch::Handler { + public: + WriteBatchHandlerJniCallback( + JNIEnv* env, jobject jWriteBackHandler); + ~WriteBatchHandlerJniCallback(); + void Put(const Slice& key, const Slice& value); + void Merge(const Slice& key, const Slice& value); + void Delete(const Slice& key); + void DeleteRange(const Slice& beginKey, const Slice& endKey); + void LogData(const Slice& blob); + bool Continue(); + + private: + JNIEnv* m_env; + jobject m_jWriteBatchHandler; + jbyteArray sliceToJArray(const Slice& s); + jmethodID m_jPutMethodId; + jmethodID m_jMergeMethodId; + jmethodID m_jDeleteMethodId; + jmethodID m_jDeleteRangeMethodId; + jmethodID m_jLogDataMethodId; + jmethodID m_jContinueMethodId; +}; +} // namespace rocksdb + +#endif // JAVA_ROCKSJNI_WRITEBATCHHANDLERJNICALLBACK_H_ diff --git a/java/samples/src/main/java/RocksDBColumnFamilySample.java b/java/samples/src/main/java/RocksDBColumnFamilySample.java new file mode 100644 index 00000000000..650b1b2f600 --- /dev/null +++ b/java/samples/src/main/java/RocksDBColumnFamilySample.java @@ -0,0 +1,78 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +import org.rocksdb.*; + +import java.util.ArrayList; +import java.util.List; + +public class RocksDBColumnFamilySample { + static { + RocksDB.loadLibrary(); + } + + public static void main(final String[] args) throws RocksDBException { + if (args.length < 1) { + System.out.println( + "usage: RocksDBColumnFamilySample db_path"); + System.exit(-1); + } + + final String db_path = args[0]; + + System.out.println("RocksDBColumnFamilySample"); + try(final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, db_path)) { + + assert(db != null); + + // create column family + try(final ColumnFamilyHandle columnFamilyHandle = db.createColumnFamily( + new ColumnFamilyDescriptor("new_cf".getBytes(), + new ColumnFamilyOptions()))) { + assert (columnFamilyHandle != null); + } + } + + // open DB with two column families + final List columnFamilyDescriptors = + new ArrayList<>(); + // have to open default column family + columnFamilyDescriptors.add(new ColumnFamilyDescriptor( + RocksDB.DEFAULT_COLUMN_FAMILY, new ColumnFamilyOptions())); + // open the new one, too + columnFamilyDescriptors.add(new ColumnFamilyDescriptor( + "new_cf".getBytes(), new ColumnFamilyOptions())); + final List columnFamilyHandles = new ArrayList<>(); + try(final DBOptions options = new DBOptions(); + final RocksDB db = RocksDB.open(options, db_path, + columnFamilyDescriptors, columnFamilyHandles)) { + assert(db != null); + + try { + // put and get from non-default column family + db.put(columnFamilyHandles.get(0), new WriteOptions(), + "key".getBytes(), "value".getBytes()); + + // atomic write + try (final WriteBatch wb = new WriteBatch()) { + wb.put(columnFamilyHandles.get(0), "key2".getBytes(), + "value2".getBytes()); + wb.put(columnFamilyHandles.get(1), "key3".getBytes(), + "value3".getBytes()); + wb.remove(columnFamilyHandles.get(0), "key".getBytes()); + db.write(new WriteOptions(), wb); + } + + // drop column family + db.dropColumnFamily(columnFamilyHandles.get(1)); + } finally { + for (final ColumnFamilyHandle handle : columnFamilyHandles) { + handle.close(); + } + } + } + } +} diff --git a/java/samples/src/main/java/RocksDBSample.java b/java/samples/src/main/java/RocksDBSample.java new file mode 100644 index 00000000000..f61995ed98f --- /dev/null +++ b/java/samples/src/main/java/RocksDBSample.java @@ -0,0 +1,303 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +import java.lang.IllegalArgumentException; +import java.util.Arrays; +import java.util.List; +import java.util.Map; +import java.util.ArrayList; + +import org.rocksdb.*; +import org.rocksdb.util.SizeUnit; + +public class RocksDBSample { + static { + RocksDB.loadLibrary(); + } + + public static void main(final String[] args) { + if (args.length < 1) { + System.out.println("usage: RocksDBSample db_path"); + System.exit(-1); + } + + final String db_path = args[0]; + final String db_path_not_found = db_path + "_not_found"; + + System.out.println("RocksDBSample"); + try (final Options options = new Options(); + final Filter bloomFilter = new BloomFilter(10); + final ReadOptions readOptions = new ReadOptions() + .setFillCache(false); + final Statistics stats = new Statistics(); + final RateLimiter rateLimiter = new RateLimiter(10000000,10000, 10)) { + + try (final RocksDB db = RocksDB.open(options, db_path_not_found)) { + assert (false); + } catch (final RocksDBException e) { + System.out.format("Caught the expected exception -- %s\n", e); + } + + try { + options.setCreateIfMissing(true) + .setStatistics(stats) + .setWriteBufferSize(8 * SizeUnit.KB) + .setMaxWriteBufferNumber(3) + .setMaxBackgroundCompactions(10) + .setCompressionType(CompressionType.SNAPPY_COMPRESSION) + .setCompactionStyle(CompactionStyle.UNIVERSAL); + } catch (final IllegalArgumentException e) { + assert (false); + } + + assert (options.createIfMissing() == true); + assert (options.writeBufferSize() == 8 * SizeUnit.KB); + assert (options.maxWriteBufferNumber() == 3); + assert (options.maxBackgroundCompactions() == 10); + assert (options.compressionType() == CompressionType.SNAPPY_COMPRESSION); + assert (options.compactionStyle() == CompactionStyle.UNIVERSAL); + + assert (options.memTableFactoryName().equals("SkipListFactory")); + options.setMemTableConfig( + new HashSkipListMemTableConfig() + .setHeight(4) + .setBranchingFactor(4) + .setBucketCount(2000000)); + assert (options.memTableFactoryName().equals("HashSkipListRepFactory")); + + options.setMemTableConfig( + new HashLinkedListMemTableConfig() + .setBucketCount(100000)); + assert (options.memTableFactoryName().equals("HashLinkedListRepFactory")); + + options.setMemTableConfig( + new VectorMemTableConfig().setReservedSize(10000)); + assert (options.memTableFactoryName().equals("VectorRepFactory")); + + options.setMemTableConfig(new SkipListMemTableConfig()); + assert (options.memTableFactoryName().equals("SkipListFactory")); + + options.setTableFormatConfig(new PlainTableConfig()); + // Plain-Table requires mmap read + options.setAllowMmapReads(true); + assert (options.tableFactoryName().equals("PlainTable")); + + options.setRateLimiter(rateLimiter); + + final BlockBasedTableConfig table_options = new BlockBasedTableConfig(); + table_options.setBlockCacheSize(64 * SizeUnit.KB) + .setFilter(bloomFilter) + .setCacheNumShardBits(6) + .setBlockSizeDeviation(5) + .setBlockRestartInterval(10) + .setCacheIndexAndFilterBlocks(true) + .setHashIndexAllowCollision(false) + .setBlockCacheCompressedSize(64 * SizeUnit.KB) + .setBlockCacheCompressedNumShardBits(10); + + assert (table_options.blockCacheSize() == 64 * SizeUnit.KB); + assert (table_options.cacheNumShardBits() == 6); + assert (table_options.blockSizeDeviation() == 5); + assert (table_options.blockRestartInterval() == 10); + assert (table_options.cacheIndexAndFilterBlocks() == true); + assert (table_options.hashIndexAllowCollision() == false); + assert (table_options.blockCacheCompressedSize() == 64 * SizeUnit.KB); + assert (table_options.blockCacheCompressedNumShardBits() == 10); + + options.setTableFormatConfig(table_options); + assert (options.tableFactoryName().equals("BlockBasedTable")); + + try (final RocksDB db = RocksDB.open(options, db_path)) { + db.put("hello".getBytes(), "world".getBytes()); + + final byte[] value = db.get("hello".getBytes()); + assert ("world".equals(new String(value))); + + final String str = db.getProperty("rocksdb.stats"); + assert (str != null && !str.equals("")); + } catch (final RocksDBException e) { + System.out.format("[ERROR] caught the unexpected exception -- %s\n", e); + assert (false); + } + + try (final RocksDB db = RocksDB.open(options, db_path)) { + db.put("hello".getBytes(), "world".getBytes()); + byte[] value = db.get("hello".getBytes()); + System.out.format("Get('hello') = %s\n", + new String(value)); + + for (int i = 1; i <= 9; ++i) { + for (int j = 1; j <= 9; ++j) { + db.put(String.format("%dx%d", i, j).getBytes(), + String.format("%d", i * j).getBytes()); + } + } + + for (int i = 1; i <= 9; ++i) { + for (int j = 1; j <= 9; ++j) { + System.out.format("%s ", new String(db.get( + String.format("%dx%d", i, j).getBytes()))); + } + System.out.println(""); + } + + // write batch test + try (final WriteOptions writeOpt = new WriteOptions()) { + for (int i = 10; i <= 19; ++i) { + try (final WriteBatch batch = new WriteBatch()) { + for (int j = 10; j <= 19; ++j) { + batch.put(String.format("%dx%d", i, j).getBytes(), + String.format("%d", i * j).getBytes()); + } + db.write(writeOpt, batch); + } + } + } + for (int i = 10; i <= 19; ++i) { + for (int j = 10; j <= 19; ++j) { + assert (new String( + db.get(String.format("%dx%d", i, j).getBytes())).equals( + String.format("%d", i * j))); + System.out.format("%s ", new String(db.get( + String.format("%dx%d", i, j).getBytes()))); + } + System.out.println(""); + } + + value = db.get("1x1".getBytes()); + assert (value != null); + value = db.get("world".getBytes()); + assert (value == null); + value = db.get(readOptions, "world".getBytes()); + assert (value == null); + + final byte[] testKey = "asdf".getBytes(); + final byte[] testValue = + "asdfghjkl;'?> insufficientArray.length); + len = db.get("asdfjkl;".getBytes(), enoughArray); + assert (len == RocksDB.NOT_FOUND); + len = db.get(testKey, enoughArray); + assert (len == testValue.length); + + len = db.get(readOptions, testKey, insufficientArray); + assert (len > insufficientArray.length); + len = db.get(readOptions, "asdfjkl;".getBytes(), enoughArray); + assert (len == RocksDB.NOT_FOUND); + len = db.get(readOptions, testKey, enoughArray); + assert (len == testValue.length); + + db.remove(testKey); + len = db.get(testKey, enoughArray); + assert (len == RocksDB.NOT_FOUND); + + // repeat the test with WriteOptions + try (final WriteOptions writeOpts = new WriteOptions()) { + writeOpts.setSync(true); + writeOpts.setDisableWAL(true); + db.put(writeOpts, testKey, testValue); + len = db.get(testKey, enoughArray); + assert (len == testValue.length); + assert (new String(testValue).equals( + new String(enoughArray, 0, len))); + } + + try { + for (final TickerType statsType : TickerType.values()) { + if (statsType != TickerType.TICKER_ENUM_MAX) { + stats.getTickerCount(statsType); + } + } + System.out.println("getTickerCount() passed."); + } catch (final Exception e) { + System.out.println("Failed in call to getTickerCount()"); + assert (false); //Should never reach here. + } + + try { + for (final HistogramType histogramType : HistogramType.values()) { + if (histogramType != HistogramType.HISTOGRAM_ENUM_MAX) { + HistogramData data = stats.getHistogramData(histogramType); + } + } + System.out.println("getHistogramData() passed."); + } catch (final Exception e) { + System.out.println("Failed in call to getHistogramData()"); + assert (false); //Should never reach here. + } + + try (final RocksIterator iterator = db.newIterator()) { + + boolean seekToFirstPassed = false; + for (iterator.seekToFirst(); iterator.isValid(); iterator.next()) { + iterator.status(); + assert (iterator.key() != null); + assert (iterator.value() != null); + seekToFirstPassed = true; + } + if (seekToFirstPassed) { + System.out.println("iterator seekToFirst tests passed."); + } + + boolean seekToLastPassed = false; + for (iterator.seekToLast(); iterator.isValid(); iterator.prev()) { + iterator.status(); + assert (iterator.key() != null); + assert (iterator.value() != null); + seekToLastPassed = true; + } + + if (seekToLastPassed) { + System.out.println("iterator seekToLastPassed tests passed."); + } + + iterator.seekToFirst(); + iterator.seek(iterator.key()); + assert (iterator.key() != null); + assert (iterator.value() != null); + + System.out.println("iterator seek test passed."); + + } + System.out.println("iterator tests passed."); + + final List keys = new ArrayList<>(); + try (final RocksIterator iterator = db.newIterator()) { + for (iterator.seekToLast(); iterator.isValid(); iterator.prev()) { + keys.add(iterator.key()); + } + } + + Map values = db.multiGet(keys); + assert (values.size() == keys.size()); + for (final byte[] value1 : values.values()) { + assert (value1 != null); + } + + values = db.multiGet(new ReadOptions(), keys); + assert (values.size() == keys.size()); + for (final byte[] value1 : values.values()) { + assert (value1 != null); + } + } catch (final RocksDBException e) { + System.err.println(e); + } + } + } +} diff --git a/java/src/main/java/org/rocksdb/AbstractCompactionFilter.java b/java/src/main/java/org/rocksdb/AbstractCompactionFilter.java new file mode 100644 index 00000000000..976401fba08 --- /dev/null +++ b/java/src/main/java/org/rocksdb/AbstractCompactionFilter.java @@ -0,0 +1,30 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +/** + * A CompactionFilter allows an application to modify/delete a key-value at + * the time of compaction. + * + * At present we just permit an overriding Java class to wrap a C++ + * implementation + */ +public abstract class AbstractCompactionFilter> + extends RocksObject { + + protected AbstractCompactionFilter(final long nativeHandle) { + super(nativeHandle); + } + + /** + * Deletes underlying C++ compaction pointer. + * + * Note that this function should be called only after all + * RocksDB instances referencing the compaction filter are closed. + * Otherwise an undefined behavior will occur. + */ + @Override + protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/AbstractComparator.java b/java/src/main/java/org/rocksdb/AbstractComparator.java new file mode 100644 index 00000000000..0fc4a19dfbd --- /dev/null +++ b/java/src/main/java/org/rocksdb/AbstractComparator.java @@ -0,0 +1,106 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Comparators are used by RocksDB to determine + * the ordering of keys. + * + * This class is package private, implementers + * should extend either of the public abstract classes: + * @see org.rocksdb.Comparator + * @see org.rocksdb.DirectComparator + */ +public abstract class AbstractComparator> + extends AbstractImmutableNativeReference { + + protected AbstractComparator() { + super(true); + } + + /** + * The name of the comparator. Used to check for comparator + * mismatches (i.e., a DB created with one comparator is + * accessed using a different comparator). + * + * A new name should be used whenever + * the comparator implementation changes in a way that will cause + * the relative ordering of any two keys to change. + * + * Names starting with "rocksdb." are reserved and should not be used. + * + * @return The name of this comparator implementation + */ + public abstract String name(); + + /** + * Three-way key comparison + * + * @param a Slice access to first key + * @param b Slice access to second key + * + * @return Should return either: + * 1) < 0 if "a" < "b" + * 2) == 0 if "a" == "b" + * 3) > 0 if "a" > "b" + */ + public abstract int compare(final T a, final T b); + + /** + *

    Used to reduce the space requirements + * for internal data structures like index blocks.

    + * + *

    If start < limit, you may return a new start which is a + * shorter string in [start, limit).

    + * + *

    Simple comparator implementations may return null if they + * wish to use start unchanged. i.e., an implementation of + * this method that does nothing is correct.

    + * + * @param start String + * @param limit of type T + * + * @return a shorter start, or null + */ + public String findShortestSeparator(final String start, final T limit) { + return null; + } + + /** + *

    Used to reduce the space requirements + * for internal data structures like index blocks.

    + * + *

    You may return a new short key (key1) where + * key1 ≥ key.

    + * + *

    Simple comparator implementations may return null if they + * wish to leave the key unchanged. i.e., an implementation of + * this method that does nothing is correct.

    + * + * @param key String + * + * @return a shorter key, or null + */ + public String findShortSuccessor(final String key) { + return null; + } + + /** + * Deletes underlying C++ comparator pointer. + * + * Note that this function should be called only after all + * RocksDB instances referencing the comparator are closed. + * Otherwise an undefined behavior will occur. + */ + @Override + protected void disposeInternal() { + disposeInternal(getNativeHandle()); + } + + protected abstract long getNativeHandle(); + + private native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/AbstractImmutableNativeReference.java b/java/src/main/java/org/rocksdb/AbstractImmutableNativeReference.java new file mode 100644 index 00000000000..b1dc1ef3795 --- /dev/null +++ b/java/src/main/java/org/rocksdb/AbstractImmutableNativeReference.java @@ -0,0 +1,66 @@ +// Copyright (c) 2016, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.concurrent.atomic.AtomicBoolean; + +/** + * Offers functionality for implementations of + * {@link AbstractNativeReference} which have an immutable reference to the + * underlying native C++ object + */ +public abstract class AbstractImmutableNativeReference + extends AbstractNativeReference { + + /** + * A flag indicating whether the current {@code AbstractNativeReference} is + * responsible to free the underlying C++ object + */ + private final AtomicBoolean owningHandle_; + + protected AbstractImmutableNativeReference(final boolean owningHandle) { + this.owningHandle_ = new AtomicBoolean(owningHandle); + } + + @Override + public boolean isOwningHandle() { + return owningHandle_.get(); + } + + /** + * Releases this {@code AbstractNativeReference} from the responsibility of + * freeing the underlying native C++ object + *

    + * This will prevent the object from attempting to delete the underlying + * native object in its finalizer. This must be used when another object + * takes over ownership of the native object or both will attempt to delete + * the underlying object when garbage collected. + *

    + * When {@code disOwnNativeHandle()} is called, {@code dispose()} will + * subsequently take no action. As a result, incorrect use of this function + * may cause a memory leak. + *

    + * + * @see #dispose() + */ + protected final void disOwnNativeHandle() { + owningHandle_.set(false); + } + + @Override + public void close() { + if (owningHandle_.compareAndSet(true, false)) { + disposeInternal(); + } + } + + /** + * The helper function of {@link AbstractImmutableNativeReference#dispose()} + * which all subclasses of {@code AbstractImmutableNativeReference} must + * implement to release their underlying native C++ objects. + */ + protected abstract void disposeInternal(); +} diff --git a/java/src/main/java/org/rocksdb/AbstractNativeReference.java b/java/src/main/java/org/rocksdb/AbstractNativeReference.java new file mode 100644 index 00000000000..ffb0776e4a6 --- /dev/null +++ b/java/src/main/java/org/rocksdb/AbstractNativeReference.java @@ -0,0 +1,76 @@ +// Copyright (c) 2016, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * AbstractNativeReference is the base-class of all RocksDB classes that have + * a pointer to a native C++ {@code rocksdb} object. + *

    + * AbstractNativeReference has the {@link AbstractNativeReference#dispose()} + * method, which frees its associated C++ object.

    + *

    + * This function should be called manually, however, if required it will be + * called automatically during the regular Java GC process via + * {@link AbstractNativeReference#finalize()}.

    + *

    + * Note - Java can only see the long member variable (which is the C++ pointer + * value to the native object), as such it does not know the real size of the + * object and therefore may assign a low GC priority for it; So it is strongly + * suggested that you manually dispose of objects when you are finished with + * them.

    + */ +public abstract class AbstractNativeReference implements AutoCloseable { + + /** + * Returns true if we are responsible for freeing the underlying C++ object + * + * @return true if we are responsible to free the C++ object + * @see #dispose() + */ + protected abstract boolean isOwningHandle(); + + /** + * Frees the underlying C++ object + *

    + * It is strong recommended that the developer calls this after they + * have finished using the object.

    + *

    + * Note, that once an instance of {@link AbstractNativeReference} has been + * disposed, calling any of its functions will lead to undefined + * behavior.

    + */ + @Override + public abstract void close(); + + /** + * @deprecated Instead use {@link AbstractNativeReference#close()} + */ + @Deprecated + public final void dispose() { + close(); + } + + /** + * Simply calls {@link AbstractNativeReference#dispose()} to free + * any underlying C++ object reference which has not yet been manually + * released. + * + * @deprecated You should not rely on GC of Rocks objects, and instead should + * either call {@link AbstractNativeReference#close()} manually or make + * use of some sort of ARM (Automatic Resource Management) such as + * Java 7's     try-with-resources + * statement + */ + @Override + @Deprecated + protected void finalize() throws Throwable { + if(isOwningHandle()) { + //TODO(AR) log a warning message... developer should have called close() + } + dispose(); + super.finalize(); + } +} diff --git a/java/src/main/java/org/rocksdb/AbstractRocksIterator.java b/java/src/main/java/org/rocksdb/AbstractRocksIterator.java new file mode 100644 index 00000000000..52bd00f47ce --- /dev/null +++ b/java/src/main/java/org/rocksdb/AbstractRocksIterator.java @@ -0,0 +1,101 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Base class implementation for Rocks Iterators + * in the Java API + * + *

    Multiple threads can invoke const methods on an RocksIterator without + * external synchronization, but if any of the threads may call a + * non-const method, all threads accessing the same RocksIterator must use + * external synchronization.

    + * + * @param

    The type of the Parent Object from which the Rocks Iterator was + * created. This is used by disposeInternal to avoid double-free + * issues with the underlying C++ object. + * @see org.rocksdb.RocksObject + */ +public abstract class AbstractRocksIterator

    + extends RocksObject implements RocksIteratorInterface { + final P parent_; + + protected AbstractRocksIterator(final P parent, + final long nativeHandle) { + super(nativeHandle); + // parent must point to a valid RocksDB instance. + assert (parent != null); + // RocksIterator must hold a reference to the related parent instance + // to guarantee that while a GC cycle starts RocksIterator instances + // are freed prior to parent instances. + parent_ = parent; + } + + @Override + public boolean isValid() { + assert (isOwningHandle()); + return isValid0(nativeHandle_); + } + + @Override + public void seekToFirst() { + assert (isOwningHandle()); + seekToFirst0(nativeHandle_); + } + + @Override + public void seekToLast() { + assert (isOwningHandle()); + seekToLast0(nativeHandle_); + } + + @Override + public void seek(byte[] target) { + assert (isOwningHandle()); + seek0(nativeHandle_, target, target.length); + } + + @Override + public void next() { + assert (isOwningHandle()); + next0(nativeHandle_); + } + + @Override + public void prev() { + assert (isOwningHandle()); + prev0(nativeHandle_); + } + + @Override + public void status() throws RocksDBException { + assert (isOwningHandle()); + status0(nativeHandle_); + } + + /** + *

    Deletes underlying C++ iterator pointer.

    + * + *

    Note: the underlying handle can only be safely deleted if the parent + * instance related to a certain RocksIterator is still valid and initialized. + * Therefore {@code disposeInternal()} checks if the parent is initialized + * before freeing the native handle.

    + */ + @Override + protected void disposeInternal() { + if (parent_.isOwningHandle()) { + disposeInternal(nativeHandle_); + } + } + + abstract boolean isValid0(long handle); + abstract void seekToFirst0(long handle); + abstract void seekToLast0(long handle); + abstract void next0(long handle); + abstract void prev0(long handle); + abstract void seek0(long handle, byte[] target, int targetLen); + abstract void status0(long handle) throws RocksDBException; +} diff --git a/java/src/main/java/org/rocksdb/AbstractSlice.java b/java/src/main/java/org/rocksdb/AbstractSlice.java new file mode 100644 index 00000000000..5a22e29562e --- /dev/null +++ b/java/src/main/java/org/rocksdb/AbstractSlice.java @@ -0,0 +1,191 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Slices are used by RocksDB to provide + * efficient access to keys and values. + * + * This class is package private, implementers + * should extend either of the public abstract classes: + * @see org.rocksdb.Slice + * @see org.rocksdb.DirectSlice + * + * Regards the lifecycle of Java Slices in RocksDB: + * At present when you configure a Comparator from Java, it creates an + * instance of a C++ BaseComparatorJniCallback subclass and + * passes that to RocksDB as the comparator. That subclass of + * BaseComparatorJniCallback creates the Java + * @see org.rocksdb.AbstractSlice subclass Objects. When you dispose + * the Java @see org.rocksdb.AbstractComparator subclass, it disposes the + * C++ BaseComparatorJniCallback subclass, which in turn destroys the + * Java @see org.rocksdb.AbstractSlice subclass Objects. + */ +public abstract class AbstractSlice extends RocksMutableObject { + + protected AbstractSlice() { + super(); + } + + protected AbstractSlice(final long nativeHandle) { + super(nativeHandle); + } + + /** + * Returns the data of the slice. + * + * @return The slice data. Note, the type of access is + * determined by the subclass + * @see org.rocksdb.AbstractSlice#data0(long) + */ + public T data() { + return data0(getNativeHandle()); + } + + /** + * Access to the data is provided by the + * subtype as it needs to handle the + * generic typing. + * + * @param handle The address of the underlying + * native object. + * + * @return Java typed access to the data. + */ + protected abstract T data0(long handle); + + /** + * Drops the specified {@code n} + * number of bytes from the start + * of the backing slice + * + * @param n The number of bytes to drop + */ + public abstract void removePrefix(final int n); + + /** + * Clears the backing slice + */ + public abstract void clear(); + + /** + * Return the length (in bytes) of the data. + * + * @return The length in bytes. + */ + public int size() { + return size0(getNativeHandle()); + } + + /** + * Return true if the length of the + * data is zero. + * + * @return true if there is no data, false otherwise. + */ + public boolean empty() { + return empty0(getNativeHandle()); + } + + /** + * Creates a string representation of the data + * + * @param hex When true, the representation + * will be encoded in hexadecimal. + * + * @return The string representation of the data. + */ + public String toString(final boolean hex) { + return toString0(getNativeHandle(), hex); + } + + @Override + public String toString() { + return toString(false); + } + + /** + * Three-way key comparison + * + * @param other A slice to compare against + * + * @return Should return either: + * 1) < 0 if this < other + * 2) == 0 if this == other + * 3) > 0 if this > other + */ + public int compare(final AbstractSlice other) { + assert (other != null); + if(!isOwningHandle()) { + return other.isOwningHandle() ? -1 : 0; + } else { + if(!other.isOwningHandle()) { + return 1; + } else { + return compare0(getNativeHandle(), other.getNativeHandle()); + } + } + } + + @Override + public int hashCode() { + return toString().hashCode(); + } + + /** + * If other is a slice object, then + * we defer to {@link #compare(AbstractSlice) compare} + * to check equality, otherwise we return false. + * + * @param other Object to test for equality + * + * @return true when {@code this.compare(other) == 0}, + * false otherwise. + */ + @Override + public boolean equals(final Object other) { + if (other != null && other instanceof AbstractSlice) { + return compare((AbstractSlice)other) == 0; + } else { + return false; + } + } + + /** + * Determines whether this slice starts with + * another slice + * + * @param prefix Another slice which may of may not + * be a prefix of this slice. + * + * @return true when this slice starts with the + * {@code prefix} slice + */ + public boolean startsWith(final AbstractSlice prefix) { + if (prefix != null) { + return startsWith0(getNativeHandle(), prefix.getNativeHandle()); + } else { + return false; + } + } + + protected native static long createNewSliceFromString(final String str); + private native int size0(long handle); + private native boolean empty0(long handle); + private native String toString0(long handle, boolean hex); + private native int compare0(long handle, long otherHandle); + private native boolean startsWith0(long handle, long otherHandle); + + /** + * Deletes underlying C++ slice pointer. + * Note that this function should be called only after all + * RocksDB instances referencing the slice are closed. + * Otherwise an undefined behavior will occur. + */ + @Override + protected final native void disposeInternal(final long handle); + +} diff --git a/java/src/main/java/org/rocksdb/AbstractWriteBatch.java b/java/src/main/java/org/rocksdb/AbstractWriteBatch.java new file mode 100644 index 00000000000..b2e5571809a --- /dev/null +++ b/java/src/main/java/org/rocksdb/AbstractWriteBatch.java @@ -0,0 +1,119 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +public abstract class AbstractWriteBatch extends RocksObject + implements WriteBatchInterface { + + protected AbstractWriteBatch(final long nativeHandle) { + super(nativeHandle); + } + + @Override + public int count() { + return count0(nativeHandle_); + } + + @Override + public void put(byte[] key, byte[] value) { + put(nativeHandle_, key, key.length, value, value.length); + } + + @Override + public void put(ColumnFamilyHandle columnFamilyHandle, byte[] key, + byte[] value) { + put(nativeHandle_, key, key.length, value, value.length, + columnFamilyHandle.nativeHandle_); + } + + @Override + public void merge(byte[] key, byte[] value) { + merge(nativeHandle_, key, key.length, value, value.length); + } + + @Override + public void merge(ColumnFamilyHandle columnFamilyHandle, byte[] key, + byte[] value) { + merge(nativeHandle_, key, key.length, value, value.length, + columnFamilyHandle.nativeHandle_); + } + + @Override + public void remove(byte[] key) { + remove(nativeHandle_, key, key.length); + } + + @Override + public void remove(ColumnFamilyHandle columnFamilyHandle, byte[] key) { + remove(nativeHandle_, key, key.length, columnFamilyHandle.nativeHandle_); + } + + @Override + public void deleteRange(byte[] beginKey, byte[] endKey) { + deleteRange(nativeHandle_, beginKey, beginKey.length, endKey, endKey.length); + } + + @Override + public void deleteRange(ColumnFamilyHandle columnFamilyHandle, byte[] beginKey, byte[] endKey) { + deleteRange(nativeHandle_, beginKey, beginKey.length, endKey, endKey.length, + columnFamilyHandle.nativeHandle_); + } + + @Override + public void putLogData(byte[] blob) { + putLogData(nativeHandle_, blob, blob.length); + } + + @Override + public void clear() { + clear0(nativeHandle_); + } + + @Override + public void setSavePoint() { + setSavePoint0(nativeHandle_); + } + + @Override + public void rollbackToSavePoint() throws RocksDBException { + rollbackToSavePoint0(nativeHandle_); + } + + abstract int count0(final long handle); + + abstract void put(final long handle, final byte[] key, final int keyLen, + final byte[] value, final int valueLen); + + abstract void put(final long handle, final byte[] key, final int keyLen, + final byte[] value, final int valueLen, final long cfHandle); + + abstract void merge(final long handle, final byte[] key, final int keyLen, + final byte[] value, final int valueLen); + + abstract void merge(final long handle, final byte[] key, final int keyLen, + final byte[] value, final int valueLen, final long cfHandle); + + abstract void remove(final long handle, final byte[] key, + final int keyLen); + + abstract void remove(final long handle, final byte[] key, + final int keyLen, final long cfHandle); + + abstract void deleteRange(final long handle, final byte[] beginKey, final int beginKeyLen, + final byte[] endKey, final int endKeyLen); + + abstract void deleteRange(final long handle, final byte[] beginKey, final int beginKeyLen, + final byte[] endKey, final int endKeyLen, final long cfHandle); + + abstract void putLogData(final long handle, final byte[] blob, + final int blobLen); + + abstract void clear0(final long handle); + + abstract void setSavePoint0(final long handle); + + abstract void rollbackToSavePoint0(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/AccessHint.java b/java/src/main/java/org/rocksdb/AccessHint.java new file mode 100644 index 00000000000..877c4ab39ae --- /dev/null +++ b/java/src/main/java/org/rocksdb/AccessHint.java @@ -0,0 +1,53 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * File access pattern once a compaction has started + */ +public enum AccessHint { + NONE((byte)0x0), + NORMAL((byte)0x1), + SEQUENTIAL((byte)0x2), + WILLNEED((byte)0x3); + + private final byte value; + + AccessHint(final byte value) { + this.value = value; + } + + /** + *

    Returns the byte value of the enumerations value.

    + * + * @return byte representation + */ + public byte getValue() { + return value; + } + + /** + *

    Get the AccessHint enumeration value by + * passing the byte identifier to this method.

    + * + * @param byteIdentifier of AccessHint. + * + * @return AccessHint instance. + * + * @throws IllegalArgumentException if the access hint for the byteIdentifier + * cannot be found + */ + public static AccessHint getAccessHint(final byte byteIdentifier) { + for (final AccessHint accessHint : AccessHint.values()) { + if (accessHint.getValue() == byteIdentifier) { + return accessHint; + } + } + + throw new IllegalArgumentException( + "Illegal value provided for AccessHint."); + } +} diff --git a/java/src/main/java/org/rocksdb/AdvancedColumnFamilyOptionsInterface.java b/java/src/main/java/org/rocksdb/AdvancedColumnFamilyOptionsInterface.java new file mode 100644 index 00000000000..d3908d1a379 --- /dev/null +++ b/java/src/main/java/org/rocksdb/AdvancedColumnFamilyOptionsInterface.java @@ -0,0 +1,465 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.List; + +/** + * Advanced Column Family Options which are not + * mutable (i.e. present in {@link AdvancedMutableColumnFamilyOptionsInterface} + * + * Taken from include/rocksdb/advanced_options.h + */ +public interface AdvancedColumnFamilyOptionsInterface + { + + /** + * The minimum number of write buffers that will be merged together + * before writing to storage. If set to 1, then + * all write buffers are flushed to L0 as individual files and this increases + * read amplification because a get request has to check in all of these + * files. Also, an in-memory merge may result in writing lesser + * data to storage if there are duplicate records in each of these + * individual write buffers. Default: 1 + * + * @param minWriteBufferNumberToMerge the minimum number of write buffers + * that will be merged together. + * @return the reference to the current options. + */ + T setMinWriteBufferNumberToMerge( + int minWriteBufferNumberToMerge); + + /** + * The minimum number of write buffers that will be merged together + * before writing to storage. If set to 1, then + * all write buffers are flushed to L0 as individual files and this increases + * read amplification because a get request has to check in all of these + * files. Also, an in-memory merge may result in writing lesser + * data to storage if there are duplicate records in each of these + * individual write buffers. Default: 1 + * + * @return the minimum number of write buffers that will be merged together. + */ + int minWriteBufferNumberToMerge(); + + /** + * The total maximum number of write buffers to maintain in memory including + * copies of buffers that have already been flushed. Unlike + * {@link AdvancedMutableColumnFamilyOptionsInterface#maxWriteBufferNumber()}, + * this parameter does not affect flushing. + * This controls the minimum amount of write history that will be available + * in memory for conflict checking when Transactions are used. + * + * When using an OptimisticTransactionDB: + * If this value is too low, some transactions may fail at commit time due + * to not being able to determine whether there were any write conflicts. + * + * When using a TransactionDB: + * If Transaction::SetSnapshot is used, TransactionDB will read either + * in-memory write buffers or SST files to do write-conflict checking. + * Increasing this value can reduce the number of reads to SST files + * done for conflict detection. + * + * Setting this value to 0 will cause write buffers to be freed immediately + * after they are flushed. + * If this value is set to -1, + * {@link AdvancedMutableColumnFamilyOptionsInterface#maxWriteBufferNumber()} + * will be used. + * + * Default: + * If using a TransactionDB/OptimisticTransactionDB, the default value will + * be set to the value of + * {@link AdvancedMutableColumnFamilyOptionsInterface#maxWriteBufferNumber()} + * if it is not explicitly set by the user. Otherwise, the default is 0. + * + * @param maxWriteBufferNumberToMaintain The maximum number of write + * buffers to maintain + * + * @return the reference to the current options. + */ + T setMaxWriteBufferNumberToMaintain( + int maxWriteBufferNumberToMaintain); + + /** + * The total maximum number of write buffers to maintain in memory including + * copies of buffers that have already been flushed. + * + * @return maxWriteBufferNumberToMaintain The maximum number of write buffers + * to maintain + */ + int maxWriteBufferNumberToMaintain(); + + /** + * Allows thread-safe inplace updates. + * If inplace_callback function is not set, + * Put(key, new_value) will update inplace the existing_value iff + * * key exists in current memtable + * * new sizeof(new_value) ≤ sizeof(existing_value) + * * existing_value for that key is a put i.e. kTypeValue + * If inplace_callback function is set, check doc for inplace_callback. + * Default: false. + * + * @param inplaceUpdateSupport true if thread-safe inplace updates + * are allowed. + * @return the reference to the current options. + */ + T setInplaceUpdateSupport( + boolean inplaceUpdateSupport); + + /** + * Allows thread-safe inplace updates. + * If inplace_callback function is not set, + * Put(key, new_value) will update inplace the existing_value iff + * * key exists in current memtable + * * new sizeof(new_value) ≤ sizeof(existing_value) + * * existing_value for that key is a put i.e. kTypeValue + * If inplace_callback function is set, check doc for inplace_callback. + * Default: false. + * + * @return true if thread-safe inplace updates are allowed. + */ + boolean inplaceUpdateSupport(); + + /** + * Control locality of bloom filter probes to improve cache miss rate. + * This option only applies to memtable prefix bloom and plaintable + * prefix bloom. It essentially limits the max number of cache lines each + * bloom filter check can touch. + * This optimization is turned off when set to 0. The number should never + * be greater than number of probes. This option can boost performance + * for in-memory workload but should use with care since it can cause + * higher false positive rate. + * Default: 0 + * + * @param bloomLocality the level of locality of bloom-filter probes. + * @return the reference to the current options. + */ + T setBloomLocality(int bloomLocality); + + /** + * Control locality of bloom filter probes to improve cache miss rate. + * This option only applies to memtable prefix bloom and plaintable + * prefix bloom. It essentially limits the max number of cache lines each + * bloom filter check can touch. + * This optimization is turned off when set to 0. The number should never + * be greater than number of probes. This option can boost performance + * for in-memory workload but should use with care since it can cause + * higher false positive rate. + * Default: 0 + * + * @return the level of locality of bloom-filter probes. + * @see #setBloomLocality(int) + */ + int bloomLocality(); + + /** + *

    Different levels can have different compression + * policies. There are cases where most lower levels + * would like to use quick compression algorithms while + * the higher levels (which have more data) use + * compression algorithms that have better compression + * but could be slower. This array, if non-empty, should + * have an entry for each level of the database; + * these override the value specified in the previous + * field 'compression'.

    + * + * NOTICE + *

    If {@code level_compaction_dynamic_level_bytes=true}, + * {@code compression_per_level[0]} still determines {@code L0}, + * but other elements of the array are based on base level + * (the level {@code L0} files are merged to), and may not + * match the level users see from info log for metadata. + *

    + *

    If {@code L0} files are merged to {@code level - n}, + * then, for {@code i>0}, {@code compression_per_level[i]} + * determines compaction type for level {@code n+i-1}.

    + * + * Example + *

    For example, if we have 5 levels, and we determine to + * merge {@code L0} data to {@code L4} (which means {@code L1..L3} + * will be empty), then the new files go to {@code L4} uses + * compression type {@code compression_per_level[1]}.

    + * + *

    If now {@code L0} is merged to {@code L2}. Data goes to + * {@code L2} will be compressed according to + * {@code compression_per_level[1]}, {@code L3} using + * {@code compression_per_level[2]}and {@code L4} using + * {@code compression_per_level[3]}. Compaction for each + * level can change when data grows.

    + * + *

    Default: empty

    + * + * @param compressionLevels list of + * {@link org.rocksdb.CompressionType} instances. + * + * @return the reference to the current options. + */ + T setCompressionPerLevel( + List compressionLevels); + + /** + *

    Return the currently set {@link org.rocksdb.CompressionType} + * per instances.

    + * + *

    See: {@link #setCompressionPerLevel(java.util.List)}

    + * + * @return list of {@link org.rocksdb.CompressionType} + * instances. + */ + List compressionPerLevel(); + + /** + * Set the number of levels for this database + * If level-styled compaction is used, then this number determines + * the total number of levels. + * + * @param numLevels the number of levels. + * @return the reference to the current options. + */ + T setNumLevels(int numLevels); + + /** + * If level-styled compaction is used, then this number determines + * the total number of levels. + * + * @return the number of levels. + */ + int numLevels(); + + /** + *

    If {@code true}, RocksDB will pick target size of each level + * dynamically. We will pick a base level b >= 1. L0 will be + * directly merged into level b, instead of always into level 1. + * Level 1 to b-1 need to be empty. We try to pick b and its target + * size so that

    + * + *
      + *
    1. target size is in the range of + * (max_bytes_for_level_base / max_bytes_for_level_multiplier, + * max_bytes_for_level_base]
      2. + *
    2. target size of the last level (level num_levels-1) equals to extra size + * of the level.
      3. + *
    + * + *

    At the same time max_bytes_for_level_multiplier and + * max_bytes_for_level_multiplier_additional are still satisfied.

    + * + *

    With this option on, from an empty DB, we make last level the base + * level, which means merging L0 data into the last level, until it exceeds + * max_bytes_for_level_base. And then we make the second last level to be + * base level, to start to merge L0 data to second last level, with its + * target size to be {@code 1/max_bytes_for_level_multiplier} of the last + * levels extra size. After the data accumulates more so that we need to + * move the base level to the third last one, and so on.

    + * + *

    Example

    + *

    For example, assume {@code max_bytes_for_level_multiplier=10}, + * {@code num_levels=6}, and {@code max_bytes_for_level_base=10MB}.

    + * + *

    Target sizes of level 1 to 5 starts with:

    + * {@code [- - - - 10MB]} + *

    with base level is level. Target sizes of level 1 to 4 are not applicable + * because they will not be used. + * Until the size of Level 5 grows to more than 10MB, say 11MB, we make + * base target to level 4 and now the targets looks like:

    + * {@code [- - - 1.1MB 11MB]} + *

    While data are accumulated, size targets are tuned based on actual data + * of level 5. When level 5 has 50MB of data, the target is like:

    + * {@code [- - - 5MB 50MB]} + *

    Until level 5's actual size is more than 100MB, say 101MB. Now if we + * keep level 4 to be the base level, its target size needs to be 10.1MB, + * which doesn't satisfy the target size range. So now we make level 3 + * the target size and the target sizes of the levels look like:

    + * {@code [- - 1.01MB 10.1MB 101MB]} + *

    In the same way, while level 5 further grows, all levels' targets grow, + * like

    + * {@code [- - 5MB 50MB 500MB]} + *

    Until level 5 exceeds 1000MB and becomes 1001MB, we make level 2 the + * base level and make levels' target sizes like this:

    + * {@code [- 1.001MB 10.01MB 100.1MB 1001MB]} + *

    and go on...

    + * + *

    By doing it, we give {@code max_bytes_for_level_multiplier} a priority + * against {@code max_bytes_for_level_base}, for a more predictable LSM tree + * shape. It is useful to limit worse case space amplification.

    + * + *

    {@code max_bytes_for_level_multiplier_additional} is ignored with + * this flag on.

    + * + *

    Turning this feature on or off for an existing DB can cause unexpected + * LSM tree structure so it's not recommended.

    + * + *

    Caution: this option is experimental

    + * + *

    Default: false

    + * + * @param enableLevelCompactionDynamicLevelBytes boolean value indicating + * if {@code LevelCompactionDynamicLevelBytes} shall be enabled. + * @return the reference to the current options. + */ + @Experimental("Turning this feature on or off for an existing DB can cause" + + "unexpected LSM tree structure so it's not recommended") + T setLevelCompactionDynamicLevelBytes( + boolean enableLevelCompactionDynamicLevelBytes); + + /** + *

    Return if {@code LevelCompactionDynamicLevelBytes} is enabled. + *

    + * + *

    For further information see + * {@link #setLevelCompactionDynamicLevelBytes(boolean)}

    + * + * @return boolean value indicating if + * {@code levelCompactionDynamicLevelBytes} is enabled. + */ + @Experimental("Caution: this option is experimental") + boolean levelCompactionDynamicLevelBytes(); + + /** + * Maximum size of each compaction (not guarantee) + * + * @param maxCompactionBytes the compaction size limit + * @return the reference to the current options. + */ + T setMaxCompactionBytes( + long maxCompactionBytes); + + /** + * Control maximum size of each compaction (not guaranteed) + * + * @return compaction size threshold + */ + long maxCompactionBytes(); + + /** + * Set compaction style for DB. + * + * Default: LEVEL. + * + * @param compactionStyle Compaction style. + * @return the reference to the current options. + */ + ColumnFamilyOptionsInterface setCompactionStyle( + CompactionStyle compactionStyle); + + /** + * Compaction style for DB. + * + * @return Compaction style. + */ + CompactionStyle compactionStyle(); + + /** + * If level {@link #compactionStyle()} == {@link CompactionStyle#LEVEL}, + * for each level, which files are prioritized to be picked to compact. + * + * Default: {@link CompactionPriority#ByCompensatedSize} + * + * @param compactionPriority The compaction priority + * + * @return the reference to the current options. + */ + T setCompactionPriority( + CompactionPriority compactionPriority); + + /** + * Get the Compaction priority if level compaction + * is used for all levels + * + * @return The compaction priority + */ + CompactionPriority compactionPriority(); + + /** + * Set the options needed to support Universal Style compactions + * + * @param compactionOptionsUniversal The Universal Style compaction options + * + * @return the reference to the current options. + */ + T setCompactionOptionsUniversal( + CompactionOptionsUniversal compactionOptionsUniversal); + + /** + * The options needed to support Universal Style compactions + * + * @return The Universal Style compaction options + */ + CompactionOptionsUniversal compactionOptionsUniversal(); + + /** + * The options for FIFO compaction style + * + * @param compactionOptionsFIFO The FIFO compaction options + * + * @return the reference to the current options. + */ + T setCompactionOptionsFIFO( + CompactionOptionsFIFO compactionOptionsFIFO); + + /** + * The options for FIFO compaction style + * + * @return The FIFO compaction options + */ + CompactionOptionsFIFO compactionOptionsFIFO(); + + /** + *

    This flag specifies that the implementation should optimize the filters + * mainly for cases where keys are found rather than also optimize for keys + * missed. This would be used in cases where the application knows that + * there are very few misses or the performance in the case of misses is not + * important.

    + * + *

    For now, this flag allows us to not store filters for the last level i.e + * the largest level which contains data of the LSM store. For keys which + * are hits, the filters in this level are not useful because we will search + * for the data anyway.

    + * + *

    NOTE: the filters in other levels are still useful + * even for key hit because they tell us whether to look in that level or go + * to the higher level.

    + * + *

    Default: false

    + * + * @param optimizeFiltersForHits boolean value indicating if this flag is set. + * @return the reference to the current options. + */ + T setOptimizeFiltersForHits( + boolean optimizeFiltersForHits); + + /** + *

    Returns the current state of the {@code optimize_filters_for_hits} + * setting.

    + * + * @return boolean value indicating if the flag + * {@code optimize_filters_for_hits} was set. + */ + boolean optimizeFiltersForHits(); + + /** + * In debug mode, RocksDB run consistency checks on the LSM everytime the LSM + * change (Flush, Compaction, AddFile). These checks are disabled in release + * mode, use this option to enable them in release mode as well. + * + * Default: false + * + * @param forceConsistencyChecks true to force consistency checks + * + * @return the reference to the current options. + */ + T setForceConsistencyChecks( + boolean forceConsistencyChecks); + + /** + * In debug mode, RocksDB run consistency checks on the LSM everytime the LSM + * change (Flush, Compaction, AddFile). These checks are disabled in release + * mode. + * + * @return true if consistency checks are enforced + */ + boolean forceConsistencyChecks(); +} diff --git a/java/src/main/java/org/rocksdb/AdvancedMutableColumnFamilyOptionsInterface.java b/java/src/main/java/org/rocksdb/AdvancedMutableColumnFamilyOptionsInterface.java new file mode 100644 index 00000000000..092fe378435 --- /dev/null +++ b/java/src/main/java/org/rocksdb/AdvancedMutableColumnFamilyOptionsInterface.java @@ -0,0 +1,437 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Advanced Column Family Options which are mutable + * + * Taken from include/rocksdb/advanced_options.h + * and MutableCFOptions in util/cf_options.h + */ +public interface AdvancedMutableColumnFamilyOptionsInterface + { + + /** + * The maximum number of write buffers that are built up in memory. + * The default is 2, so that when 1 write buffer is being flushed to + * storage, new writes can continue to the other write buffer. + * Default: 2 + * + * @param maxWriteBufferNumber maximum number of write buffers. + * @return the instance of the current options. + */ + T setMaxWriteBufferNumber( + int maxWriteBufferNumber); + + /** + * Returns maximum number of write buffers. + * + * @return maximum number of write buffers. + * @see #setMaxWriteBufferNumber(int) + */ + int maxWriteBufferNumber(); + + /** + * Number of locks used for inplace update + * Default: 10000, if inplace_update_support = true, else 0. + * + * @param inplaceUpdateNumLocks the number of locks used for + * inplace updates. + * @return the reference to the current options. + * @throws java.lang.IllegalArgumentException thrown on 32-Bit platforms + * while overflowing the underlying platform specific value. + */ + T setInplaceUpdateNumLocks( + long inplaceUpdateNumLocks); + + /** + * Number of locks used for inplace update + * Default: 10000, if inplace_update_support = true, else 0. + * + * @return the number of locks used for inplace update. + */ + long inplaceUpdateNumLocks(); + + /** + * if prefix_extractor is set and memtable_prefix_bloom_size_ratio is not 0, + * create prefix bloom for memtable with the size of + * write_buffer_size * memtable_prefix_bloom_size_ratio. + * If it is larger than 0.25, it is santinized to 0.25. + * + * Default: 0 (disable) + * + * @param memtablePrefixBloomSizeRatio The ratio + * @return the reference to the current options. + */ + T setMemtablePrefixBloomSizeRatio( + double memtablePrefixBloomSizeRatio); + + /** + * if prefix_extractor is set and memtable_prefix_bloom_size_ratio is not 0, + * create prefix bloom for memtable with the size of + * write_buffer_size * memtable_prefix_bloom_size_ratio. + * If it is larger than 0.25, it is santinized to 0.25. + * + * Default: 0 (disable) + * + * @return the ratio + */ + double memtablePrefixBloomSizeRatio(); + + /** + * Page size for huge page TLB for bloom in memtable. If ≤ 0, not allocate + * from huge page TLB but from malloc. + * Need to reserve huge pages for it to be allocated. For example: + * sysctl -w vm.nr_hugepages=20 + * See linux doc Documentation/vm/hugetlbpage.txt + * + * @param memtableHugePageSize The page size of the huge + * page tlb + * @return the reference to the current options. + */ + T setMemtableHugePageSize( + long memtableHugePageSize); + + /** + * Page size for huge page TLB for bloom in memtable. If ≤ 0, not allocate + * from huge page TLB but from malloc. + * Need to reserve huge pages for it to be allocated. For example: + * sysctl -w vm.nr_hugepages=20 + * See linux doc Documentation/vm/hugetlbpage.txt + * + * @return The page size of the huge page tlb + */ + long memtableHugePageSize(); + + /** + * The size of one block in arena memory allocation. + * If ≤ 0, a proper value is automatically calculated (usually 1/10 of + * writer_buffer_size). + * + * There are two additional restriction of the specified size: + * (1) size should be in the range of [4096, 2 << 30] and + * (2) be the multiple of the CPU word (which helps with the memory + * alignment). + * + * We'll automatically check and adjust the size number to make sure it + * conforms to the restrictions. + * Default: 0 + * + * @param arenaBlockSize the size of an arena block + * @return the reference to the current options. + * @throws java.lang.IllegalArgumentException thrown on 32-Bit platforms + * while overflowing the underlying platform specific value. + */ + T setArenaBlockSize(long arenaBlockSize); + + /** + * The size of one block in arena memory allocation. + * If ≤ 0, a proper value is automatically calculated (usually 1/10 of + * writer_buffer_size). + * + * There are two additional restriction of the specified size: + * (1) size should be in the range of [4096, 2 << 30] and + * (2) be the multiple of the CPU word (which helps with the memory + * alignment). + * + * We'll automatically check and adjust the size number to make sure it + * conforms to the restrictions. + * Default: 0 + * + * @return the size of an arena block + */ + long arenaBlockSize(); + + /** + * Soft limit on number of level-0 files. We start slowing down writes at this + * point. A value < 0 means that no writing slow down will be triggered by + * number of files in level-0. + * + * @param level0SlowdownWritesTrigger The soft limit on the number of + * level-0 files + * @return the reference to the current options. + */ + T setLevel0SlowdownWritesTrigger( + int level0SlowdownWritesTrigger); + + /** + * Soft limit on number of level-0 files. We start slowing down writes at this + * point. A value < 0 means that no writing slow down will be triggered by + * number of files in level-0. + * + * @return The soft limit on the number of + * level-0 files + */ + int level0SlowdownWritesTrigger(); + + /** + * Maximum number of level-0 files. We stop writes at this point. + * + * @param level0StopWritesTrigger The maximum number of level-0 files + * @return the reference to the current options. + */ + T setLevel0StopWritesTrigger( + int level0StopWritesTrigger); + + /** + * Maximum number of level-0 files. We stop writes at this point. + * + * @return The maximum number of level-0 files + */ + int level0StopWritesTrigger(); + + /** + * The target file size for compaction. + * This targetFileSizeBase determines a level-1 file size. + * Target file size for level L can be calculated by + * targetFileSizeBase * (targetFileSizeMultiplier ^ (L-1)) + * For example, if targetFileSizeBase is 2MB and + * target_file_size_multiplier is 10, then each file on level-1 will + * be 2MB, and each file on level 2 will be 20MB, + * and each file on level-3 will be 200MB. + * by default targetFileSizeBase is 2MB. + * + * @param targetFileSizeBase the target size of a level-0 file. + * @return the reference to the current options. + * + * @see #setTargetFileSizeMultiplier(int) + */ + T setTargetFileSizeBase( + long targetFileSizeBase); + + /** + * The target file size for compaction. + * This targetFileSizeBase determines a level-1 file size. + * Target file size for level L can be calculated by + * targetFileSizeBase * (targetFileSizeMultiplier ^ (L-1)) + * For example, if targetFileSizeBase is 2MB and + * target_file_size_multiplier is 10, then each file on level-1 will + * be 2MB, and each file on level 2 will be 20MB, + * and each file on level-3 will be 200MB. + * by default targetFileSizeBase is 2MB. + * + * @return the target size of a level-0 file. + * + * @see #targetFileSizeMultiplier() + */ + long targetFileSizeBase(); + + /** + * targetFileSizeMultiplier defines the size ratio between a + * level-L file and level-(L+1) file. + * By default target_file_size_multiplier is 1, meaning + * files in different levels have the same target. + * + * @param multiplier the size ratio between a level-(L+1) file + * and level-L file. + * @return the reference to the current options. + */ + T setTargetFileSizeMultiplier( + int multiplier); + + /** + * targetFileSizeMultiplier defines the size ratio between a + * level-(L+1) file and level-L file. + * By default targetFileSizeMultiplier is 1, meaning + * files in different levels have the same target. + * + * @return the size ratio between a level-(L+1) file and level-L file. + */ + int targetFileSizeMultiplier(); + + /** + * The ratio between the total size of level-(L+1) files and the total + * size of level-L files for all L. + * DEFAULT: 10 + * + * @param multiplier the ratio between the total size of level-(L+1) + * files and the total size of level-L files for all L. + * @return the reference to the current options. + * + * See {@link MutableColumnFamilyOptionsInterface#setMaxBytesForLevelBase(long)} + */ + T setMaxBytesForLevelMultiplier(double multiplier); + + /** + * The ratio between the total size of level-(L+1) files and the total + * size of level-L files for all L. + * DEFAULT: 10 + * + * @return the ratio between the total size of level-(L+1) files and + * the total size of level-L files for all L. + * + * See {@link MutableColumnFamilyOptionsInterface#maxBytesForLevelBase()} + */ + double maxBytesForLevelMultiplier(); + + /** + * Different max-size multipliers for different levels. + * These are multiplied by max_bytes_for_level_multiplier to arrive + * at the max-size of each level. + * + * Default: 1 + * + * @param maxBytesForLevelMultiplierAdditional The max-size multipliers + * for each level + * @return the reference to the current options. + */ + T setMaxBytesForLevelMultiplierAdditional( + int[] maxBytesForLevelMultiplierAdditional); + + /** + * Different max-size multipliers for different levels. + * These are multiplied by max_bytes_for_level_multiplier to arrive + * at the max-size of each level. + * + * Default: 1 + * + * @return The max-size multipliers for each level + */ + int[] maxBytesForLevelMultiplierAdditional(); + + /** + * All writes will be slowed down to at least delayed_write_rate if estimated + * bytes needed to be compaction exceed this threshold. + * + * Default: 64GB + * + * @param softPendingCompactionBytesLimit The soft limit to impose on + * compaction + * @return the reference to the current options. + */ + T setSoftPendingCompactionBytesLimit( + long softPendingCompactionBytesLimit); + + /** + * All writes will be slowed down to at least delayed_write_rate if estimated + * bytes needed to be compaction exceed this threshold. + * + * Default: 64GB + * + * @return The soft limit to impose on compaction + */ + long softPendingCompactionBytesLimit(); + + /** + * All writes are stopped if estimated bytes needed to be compaction exceed + * this threshold. + * + * Default: 256GB + * + * @param hardPendingCompactionBytesLimit The hard limit to impose on + * compaction + * @return the reference to the current options. + */ + T setHardPendingCompactionBytesLimit( + long hardPendingCompactionBytesLimit); + + /** + * All writes are stopped if estimated bytes needed to be compaction exceed + * this threshold. + * + * Default: 256GB + * + * @return The hard limit to impose on compaction + */ + long hardPendingCompactionBytesLimit(); + + /** + * An iteration->Next() sequentially skips over keys with the same + * user-key unless this option is set. This number specifies the number + * of keys (with the same userkey) that will be sequentially + * skipped before a reseek is issued. + * Default: 8 + * + * @param maxSequentialSkipInIterations the number of keys could + * be skipped in a iteration. + * @return the reference to the current options. + */ + T setMaxSequentialSkipInIterations( + long maxSequentialSkipInIterations); + + /** + * An iteration->Next() sequentially skips over keys with the same + * user-key unless this option is set. This number specifies the number + * of keys (with the same userkey) that will be sequentially + * skipped before a reseek is issued. + * Default: 8 + * + * @return the number of keys could be skipped in a iteration. + */ + long maxSequentialSkipInIterations(); + + /** + * Maximum number of successive merge operations on a key in the memtable. + * + * When a merge operation is added to the memtable and the maximum number of + * successive merges is reached, the value of the key will be calculated and + * inserted into the memtable instead of the merge operation. This will + * ensure that there are never more than max_successive_merges merge + * operations in the memtable. + * + * Default: 0 (disabled) + * + * @param maxSuccessiveMerges the maximum number of successive merges. + * @return the reference to the current options. + * @throws java.lang.IllegalArgumentException thrown on 32-Bit platforms + * while overflowing the underlying platform specific value. + */ + T setMaxSuccessiveMerges( + long maxSuccessiveMerges); + + /** + * Maximum number of successive merge operations on a key in the memtable. + * + * When a merge operation is added to the memtable and the maximum number of + * successive merges is reached, the value of the key will be calculated and + * inserted into the memtable instead of the merge operation. This will + * ensure that there are never more than max_successive_merges merge + * operations in the memtable. + * + * Default: 0 (disabled) + * + * @return the maximum number of successive merges. + */ + long maxSuccessiveMerges(); + + /** + * After writing every SST file, reopen it and read all the keys. + * + * Default: false + * + * @param paranoidFileChecks true to enable paranoid file checks + * @return the reference to the current options. + */ + T setParanoidFileChecks( + boolean paranoidFileChecks); + + /** + * After writing every SST file, reopen it and read all the keys. + * + * Default: false + * + * @return true if paranoid file checks are enabled + */ + boolean paranoidFileChecks(); + + /** + * Measure IO stats in compactions and flushes, if true. + * + * Default: false + * + * @param reportBgIoStats true to enable reporting + * @return the reference to the current options. + */ + T setReportBgIoStats( + boolean reportBgIoStats); + + /** + * Determine whether IO stats in compactions and flushes are being measured + * + * @return true if reporting is enabled + */ + boolean reportBgIoStats(); +} diff --git a/java/src/main/java/org/rocksdb/BackupEngine.java b/java/src/main/java/org/rocksdb/BackupEngine.java new file mode 100644 index 00000000000..763994575ce --- /dev/null +++ b/java/src/main/java/org/rocksdb/BackupEngine.java @@ -0,0 +1,221 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import java.util.List; + +/** + * BackupEngine allows you to backup + * and restore the database + * + * Be aware, that `new BackupEngine` takes time proportional to the amount + * of backups. So if you have a slow filesystem to backup (like HDFS) + * and you have a lot of backups then restoring can take some time. + * That's why we recommend to limit the number of backups. + * Also we recommend to keep BackupEngine alive and not to recreate it every + * time you need to do a backup. + */ +public class BackupEngine extends RocksObject implements AutoCloseable { + + protected BackupEngine(final long nativeHandle) { + super(nativeHandle); + } + + /** + * Opens a new Backup Engine + * + * @param env The environment that the backup engine should operate within + * @param options Any options for the backup engine + * + * @return A new BackupEngine instance + * @throws RocksDBException thrown if the backup engine could not be opened + */ + public static BackupEngine open(final Env env, + final BackupableDBOptions options) throws RocksDBException { + return new BackupEngine(open(env.nativeHandle_, options.nativeHandle_)); + } + + /** + * Captures the state of the database in the latest backup + * + * Just a convenience for {@link #createNewBackup(RocksDB, boolean)} with + * the flushBeforeBackup parameter set to false + * + * @param db The database to backup + * + * Note - This method is not thread safe + * + * @throws RocksDBException thrown if a new backup could not be created + */ + public void createNewBackup(final RocksDB db) throws RocksDBException { + createNewBackup(db, false); + } + + /** + * Captures the state of the database in the latest backup + * + * @param db The database to backup + * @param flushBeforeBackup When true, the Backup Engine will first issue a + * memtable flush and only then copy the DB files to + * the backup directory. Doing so will prevent log + * files from being copied to the backup directory + * (since flush will delete them). + * When false, the Backup Engine will not issue a + * flush before starting the backup. In that case, + * the backup will also include log files + * corresponding to live memtables. The backup will + * always be consistent with the current state of the + * database regardless of the flushBeforeBackup + * parameter. + * + * Note - This method is not thread safe + * + * @throws RocksDBException thrown if a new backup could not be created + */ + public void createNewBackup( + final RocksDB db, final boolean flushBeforeBackup) + throws RocksDBException { + assert (isOwningHandle()); + createNewBackup(nativeHandle_, db.nativeHandle_, flushBeforeBackup); + } + + /** + * Gets information about the available + * backups + * + * @return A list of information about each available backup + */ + public List getBackupInfo() { + assert (isOwningHandle()); + return getBackupInfo(nativeHandle_); + } + + /** + *

    Returns a list of corrupted backup ids. If there + * is no corrupted backup the method will return an + * empty list.

    + * + * @return array of backup ids as int ids. + */ + public int[] getCorruptedBackups() { + assert(isOwningHandle()); + return getCorruptedBackups(nativeHandle_); + } + + /** + *

    Will delete all the files we don't need anymore. It will + * do the full scan of the files/ directory and delete all the + * files that are not referenced.

    + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void garbageCollect() throws RocksDBException { + assert(isOwningHandle()); + garbageCollect(nativeHandle_); + } + + /** + * Deletes old backups, keeping just the latest numBackupsToKeep + * + * @param numBackupsToKeep The latest n backups to keep + * + * @throws RocksDBException thrown if the old backups could not be deleted + */ + public void purgeOldBackups( + final int numBackupsToKeep) throws RocksDBException { + assert (isOwningHandle()); + purgeOldBackups(nativeHandle_, numBackupsToKeep); + } + + /** + * Deletes a backup + * + * @param backupId The id of the backup to delete + * + * @throws RocksDBException thrown if the backup could not be deleted + */ + public void deleteBackup(final int backupId) throws RocksDBException { + assert (isOwningHandle()); + deleteBackup(nativeHandle_, backupId); + } + + /** + * Restore the database from a backup + * + * IMPORTANT: if options.share_table_files == true and you restore the DB + * from some backup that is not the latest, and you start creating new + * backups from the new DB, they will probably fail! + * + * Example: Let's say you have backups 1, 2, 3, 4, 5 and you restore 3. + * If you add new data to the DB and try creating a new backup now, the + * database will diverge from backups 4 and 5 and the new backup will fail. + * If you want to create new backup, you will first have to delete backups 4 + * and 5. + * + * @param backupId The id of the backup to restore + * @param dbDir The directory to restore the backup to, i.e. where your + * database is + * @param walDir The location of the log files for your database, + * often the same as dbDir + * @param restoreOptions Options for controlling the restore + * + * @throws RocksDBException thrown if the database could not be restored + */ + public void restoreDbFromBackup( + final int backupId, final String dbDir, final String walDir, + final RestoreOptions restoreOptions) throws RocksDBException { + assert (isOwningHandle()); + restoreDbFromBackup(nativeHandle_, backupId, dbDir, walDir, + restoreOptions.nativeHandle_); + } + + /** + * Restore the database from the latest backup + * + * @param dbDir The directory to restore the backup to, i.e. where your + * database is + * @param walDir The location of the log files for your database, often the + * same as dbDir + * @param restoreOptions Options for controlling the restore + * + * @throws RocksDBException thrown if the database could not be restored + */ + public void restoreDbFromLatestBackup( + final String dbDir, final String walDir, + final RestoreOptions restoreOptions) throws RocksDBException { + assert (isOwningHandle()); + restoreDbFromLatestBackup(nativeHandle_, dbDir, walDir, + restoreOptions.nativeHandle_); + } + + private native static long open(final long env, + final long backupableDbOptions) throws RocksDBException; + + private native void createNewBackup(final long handle, final long dbHandle, + final boolean flushBeforeBackup) throws RocksDBException; + + private native List getBackupInfo(final long handle); + + private native int[] getCorruptedBackups(final long handle); + + private native void garbageCollect(final long handle) throws RocksDBException; + + private native void purgeOldBackups(final long handle, + final int numBackupsToKeep) throws RocksDBException; + + private native void deleteBackup(final long handle, final int backupId) + throws RocksDBException; + + private native void restoreDbFromBackup(final long handle, final int backupId, + final String dbDir, final String walDir, final long restoreOptionsHandle) + throws RocksDBException; + + private native void restoreDbFromLatestBackup(final long handle, + final String dbDir, final String walDir, final long restoreOptionsHandle) + throws RocksDBException; + + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/BackupInfo.java b/java/src/main/java/org/rocksdb/BackupInfo.java new file mode 100644 index 00000000000..10f418629a9 --- /dev/null +++ b/java/src/main/java/org/rocksdb/BackupInfo.java @@ -0,0 +1,66 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +/** + * Instances of this class describe a Backup made by + * {@link org.rocksdb.BackupEngine}. + */ +public class BackupInfo { + + /** + * Package private constructor used to create instances + * of BackupInfo by {@link org.rocksdb.BackupEngine} + * + * @param backupId id of backup + * @param timestamp timestamp of backup + * @param size size of backup + * @param numberFiles number of files related to this backup. + */ + BackupInfo(final int backupId, final long timestamp, final long size, + final int numberFiles) { + backupId_ = backupId; + timestamp_ = timestamp; + size_ = size; + numberFiles_ = numberFiles; + } + + /** + * + * @return the backup id. + */ + public int backupId() { + return backupId_; + } + + /** + * + * @return the timestamp of the backup. + */ + public long timestamp() { + return timestamp_; + } + + /** + * + * @return the size of the backup + */ + public long size() { + return size_; + } + + /** + * + * @return the number of files of this backup. + */ + public int numberFiles() { + return numberFiles_; + } + + private int backupId_; + private long timestamp_; + private long size_; + private int numberFiles_; +} diff --git a/java/src/main/java/org/rocksdb/BackupableDBOptions.java b/java/src/main/java/org/rocksdb/BackupableDBOptions.java new file mode 100644 index 00000000000..8bb41433f21 --- /dev/null +++ b/java/src/main/java/org/rocksdb/BackupableDBOptions.java @@ -0,0 +1,465 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.io.File; + +/** + *

    BackupableDBOptions to control the behavior of a backupable database. + * It will be used during the creation of a {@link org.rocksdb.BackupEngine}. + *

    + *

    Note that dispose() must be called before an Options instance + * become out-of-scope to release the allocated memory in c++.

    + * + * @see org.rocksdb.BackupEngine + */ +public class BackupableDBOptions extends RocksObject { + + private Env backupEnv = null; + private Logger infoLog = null; + private RateLimiter backupRateLimiter = null; + private RateLimiter restoreRateLimiter = null; + + /** + *

    BackupableDBOptions constructor.

    + * + * @param path Where to keep the backup files. Has to be different than db + * name. Best to set this to {@code db name_ + "/backups"} + * @throws java.lang.IllegalArgumentException if illegal path is used. + */ + public BackupableDBOptions(final String path) { + super(newBackupableDBOptions(ensureWritableFile(path))); + } + + private static String ensureWritableFile(final String path) { + final File backupPath = path == null ? null : new File(path); + if (backupPath == null || !backupPath.isDirectory() || + !backupPath.canWrite()) { + throw new IllegalArgumentException("Illegal path provided."); + } else { + return path; + } + } + + /** + *

    Returns the path to the BackupableDB directory.

    + * + * @return the path to the BackupableDB directory. + */ + public String backupDir() { + assert(isOwningHandle()); + return backupDir(nativeHandle_); + } + + /** + * Backup Env object. It will be used for backup file I/O. If it's + * null, backups will be written out using DBs Env. Otherwise + * backup's I/O will be performed using this object. + * + * If you want to have backups on HDFS, use HDFS Env here! + * + * Default: null + * + * @param env The environment to use + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setBackupEnv(final Env env) { + assert(isOwningHandle()); + setBackupEnv(nativeHandle_, env.nativeHandle_); + this.backupEnv = env; + return this; + } + + /** + * Backup Env object. It will be used for backup file I/O. If it's + * null, backups will be written out using DBs Env. Otherwise + * backup's I/O will be performed using this object. + * + * If you want to have backups on HDFS, use HDFS Env here! + * + * Default: null + * + * @return The environment in use + */ + public Env backupEnv() { + return this.backupEnv; + } + + /** + *

    Share table files between backups.

    + * + * @param shareTableFiles If {@code share_table_files == true}, backup will + * assume that table files with same name have the same contents. This + * enables incremental backups and avoids unnecessary data copies. If + * {@code share_table_files == false}, each backup will be on its own and + * will not share any data with other backups. + * + *

    Default: true

    + * + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setShareTableFiles(final boolean shareTableFiles) { + assert(isOwningHandle()); + setShareTableFiles(nativeHandle_, shareTableFiles); + return this; + } + + /** + *

    Share table files between backups.

    + * + * @return boolean value indicating if SST files will be shared between + * backups. + */ + public boolean shareTableFiles() { + assert(isOwningHandle()); + return shareTableFiles(nativeHandle_); + } + + /** + * Set the logger to use for Backup info and error messages + * + * @param logger The logger to use for the backup + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setInfoLog(final Logger logger) { + assert(isOwningHandle()); + setInfoLog(nativeHandle_, logger.nativeHandle_); + this.infoLog = logger; + return this; + } + + /** + * Set the logger to use for Backup info and error messages + * + * Default: null + * + * @return The logger in use for the backup + */ + public Logger infoLog() { + return this.infoLog; + } + + /** + *

    Set synchronous backups.

    + * + * @param sync If {@code sync == true}, we can guarantee you'll get consistent + * backup even on a machine crash/reboot. Backup process is slower with sync + * enabled. If {@code sync == false}, we don't guarantee anything on machine + * reboot. However, chances are some of the backups are consistent. + * + *

    Default: true

    + * + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setSync(final boolean sync) { + assert(isOwningHandle()); + setSync(nativeHandle_, sync); + return this; + } + + /** + *

    Are synchronous backups activated.

    + * + * @return boolean value if synchronous backups are configured. + */ + public boolean sync() { + assert(isOwningHandle()); + return sync(nativeHandle_); + } + + /** + *

    Set if old data will be destroyed.

    + * + * @param destroyOldData If true, it will delete whatever backups there are + * already. + * + *

    Default: false

    + * + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setDestroyOldData(final boolean destroyOldData) { + assert(isOwningHandle()); + setDestroyOldData(nativeHandle_, destroyOldData); + return this; + } + + /** + *

    Returns if old data will be destroyed will performing new backups.

    + * + * @return boolean value indicating if old data will be destroyed. + */ + public boolean destroyOldData() { + assert(isOwningHandle()); + return destroyOldData(nativeHandle_); + } + + /** + *

    Set if log files shall be persisted.

    + * + * @param backupLogFiles If false, we won't backup log files. This option can + * be useful for backing up in-memory databases where log file are + * persisted, but table files are in memory. + * + *

    Default: true

    + * + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setBackupLogFiles(final boolean backupLogFiles) { + assert(isOwningHandle()); + setBackupLogFiles(nativeHandle_, backupLogFiles); + return this; + } + + /** + *

    Return information if log files shall be persisted.

    + * + * @return boolean value indicating if log files will be persisted. + */ + public boolean backupLogFiles() { + assert(isOwningHandle()); + return backupLogFiles(nativeHandle_); + } + + /** + *

    Set backup rate limit.

    + * + * @param backupRateLimit Max bytes that can be transferred in a second during + * backup. If 0 or negative, then go as fast as you can. + * + *

    Default: 0

    + * + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setBackupRateLimit(long backupRateLimit) { + assert(isOwningHandle()); + backupRateLimit = (backupRateLimit <= 0) ? 0 : backupRateLimit; + setBackupRateLimit(nativeHandle_, backupRateLimit); + return this; + } + + /** + *

    Return backup rate limit which described the max bytes that can be + * transferred in a second during backup.

    + * + * @return numerical value describing the backup transfer limit in bytes per + * second. + */ + public long backupRateLimit() { + assert(isOwningHandle()); + return backupRateLimit(nativeHandle_); + } + + /** + * Backup rate limiter. Used to control transfer speed for backup. If this is + * not null, {@link #backupRateLimit()} is ignored. + * + * Default: null + * + * @param backupRateLimiter The rate limiter to use for the backup + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setBackupRateLimiter(final RateLimiter backupRateLimiter) { + assert(isOwningHandle()); + setBackupRateLimiter(nativeHandle_, backupRateLimiter.nativeHandle_); + this.backupRateLimiter = backupRateLimiter; + return this; + } + + /** + * Backup rate limiter. Used to control transfer speed for backup. If this is + * not null, {@link #backupRateLimit()} is ignored. + * + * Default: null + * + * @return The rate limiter in use for the backup + */ + public RateLimiter backupRateLimiter() { + assert(isOwningHandle()); + return this.backupRateLimiter; + } + + /** + *

    Set restore rate limit.

    + * + * @param restoreRateLimit Max bytes that can be transferred in a second + * during restore. If 0 or negative, then go as fast as you can. + * + *

    Default: 0

    + * + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setRestoreRateLimit(long restoreRateLimit) { + assert(isOwningHandle()); + restoreRateLimit = (restoreRateLimit <= 0) ? 0 : restoreRateLimit; + setRestoreRateLimit(nativeHandle_, restoreRateLimit); + return this; + } + + /** + *

    Return restore rate limit which described the max bytes that can be + * transferred in a second during restore.

    + * + * @return numerical value describing the restore transfer limit in bytes per + * second. + */ + public long restoreRateLimit() { + assert(isOwningHandle()); + return restoreRateLimit(nativeHandle_); + } + + /** + * Restore rate limiter. Used to control transfer speed during restore. If + * this is not null, {@link #restoreRateLimit()} is ignored. + * + * Default: null + * + * @param restoreRateLimiter The rate limiter to use during restore + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setRestoreRateLimiter(final RateLimiter restoreRateLimiter) { + assert(isOwningHandle()); + setRestoreRateLimiter(nativeHandle_, restoreRateLimiter.nativeHandle_); + this.restoreRateLimiter = restoreRateLimiter; + return this; + } + + /** + * Restore rate limiter. Used to control transfer speed during restore. If + * this is not null, {@link #restoreRateLimit()} is ignored. + * + * Default: null + * + * @return The rate limiter in use during restore + */ + public RateLimiter restoreRateLimiter() { + assert(isOwningHandle()); + return this.restoreRateLimiter; + } + + /** + *

    Only used if share_table_files is set to true. If true, will consider + * that backups can come from different databases, hence a sst is not uniquely + * identified by its name, but by the triple (file name, crc32, file length) + *

    + * + * @param shareFilesWithChecksum boolean value indicating if SST files are + * stored using the triple (file name, crc32, file length) and not its name. + * + *

    Note: this is an experimental option, and you'll need to set it manually + * turn it on only if you know what you're doing*

    + * + *

    Default: false

    + * + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setShareFilesWithChecksum( + final boolean shareFilesWithChecksum) { + assert(isOwningHandle()); + setShareFilesWithChecksum(nativeHandle_, shareFilesWithChecksum); + return this; + } + + /** + *

    Return of share files with checksum is active.

    + * + * @return boolean value indicating if share files with checksum + * is active. + */ + public boolean shareFilesWithChecksum() { + assert(isOwningHandle()); + return shareFilesWithChecksum(nativeHandle_); + } + + /** + * Up to this many background threads will copy files for + * {@link BackupEngine#createNewBackup(RocksDB, boolean)} and + * {@link BackupEngine#restoreDbFromBackup(int, String, String, RestoreOptions)} + * + * Default: 1 + * + * @param maxBackgroundOperations The maximum number of background threads + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setMaxBackgroundOperations( + final int maxBackgroundOperations) { + assert(isOwningHandle()); + setMaxBackgroundOperations(nativeHandle_, maxBackgroundOperations); + return this; + } + + /** + * Up to this many background threads will copy files for + * {@link BackupEngine#createNewBackup(RocksDB, boolean)} and + * {@link BackupEngine#restoreDbFromBackup(int, String, String, RestoreOptions)} + * + * Default: 1 + * + * @return The maximum number of background threads + */ + public int maxBackgroundOperations() { + assert(isOwningHandle()); + return maxBackgroundOperations(nativeHandle_); + } + + /** + * During backup user can get callback every time next + * {@link #callbackTriggerIntervalSize()} bytes being copied. + * + * Default: 4194304 + * + * @param callbackTriggerIntervalSize The interval size for the + * callback trigger + * @return instance of current BackupableDBOptions. + */ + public BackupableDBOptions setCallbackTriggerIntervalSize( + final long callbackTriggerIntervalSize) { + assert(isOwningHandle()); + setCallbackTriggerIntervalSize(nativeHandle_, callbackTriggerIntervalSize); + return this; + } + + /** + * During backup user can get callback every time next + * {@link #callbackTriggerIntervalSize()} bytes being copied. + * + * Default: 4194304 + * + * @return The interval size for the callback trigger + */ + public long callbackTriggerIntervalSize() { + assert(isOwningHandle()); + return callbackTriggerIntervalSize(nativeHandle_); + } + + private native static long newBackupableDBOptions(final String path); + private native String backupDir(long handle); + private native void setBackupEnv(final long handle, final long envHandle); + private native void setShareTableFiles(long handle, boolean flag); + private native boolean shareTableFiles(long handle); + private native void setInfoLog(final long handle, final long infoLogHandle); + private native void setSync(long handle, boolean flag); + private native boolean sync(long handle); + private native void setDestroyOldData(long handle, boolean flag); + private native boolean destroyOldData(long handle); + private native void setBackupLogFiles(long handle, boolean flag); + private native boolean backupLogFiles(long handle); + private native void setBackupRateLimit(long handle, long rateLimit); + private native long backupRateLimit(long handle); + private native void setBackupRateLimiter(long handle, long rateLimiterHandle); + private native void setRestoreRateLimit(long handle, long rateLimit); + private native long restoreRateLimit(long handle); + private native void setRestoreRateLimiter(final long handle, + final long rateLimiterHandle); + private native void setShareFilesWithChecksum(long handle, boolean flag); + private native boolean shareFilesWithChecksum(long handle); + private native void setMaxBackgroundOperations(final long handle, + final int maxBackgroundOperations); + private native int maxBackgroundOperations(final long handle); + private native void setCallbackTriggerIntervalSize(final long handle, + long callbackTriggerIntervalSize); + private native long callbackTriggerIntervalSize(final long handle); + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/BlockBasedTableConfig.java b/java/src/main/java/org/rocksdb/BlockBasedTableConfig.java new file mode 100644 index 00000000000..2d847de29d3 --- /dev/null +++ b/java/src/main/java/org/rocksdb/BlockBasedTableConfig.java @@ -0,0 +1,452 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +/** + * The config for plain table sst format. + * + * BlockBasedTable is a RocksDB's default SST file format. + */ +public class BlockBasedTableConfig extends TableFormatConfig { + + public BlockBasedTableConfig() { + noBlockCache_ = false; + blockCacheSize_ = 8 * 1024 * 1024; + blockCacheNumShardBits_ = 0; + blockSize_ = 4 * 1024; + blockSizeDeviation_ = 10; + blockRestartInterval_ = 16; + wholeKeyFiltering_ = true; + filter_ = null; + cacheIndexAndFilterBlocks_ = false; + pinL0FilterAndIndexBlocksInCache_ = false; + hashIndexAllowCollision_ = true; + blockCacheCompressedSize_ = 0; + blockCacheCompressedNumShardBits_ = 0; + checksumType_ = ChecksumType.kCRC32c; + indexType_ = IndexType.kBinarySearch; + formatVersion_ = 0; + } + + /** + * Disable block cache. If this is set to true, + * then no block cache should be used, and the block_cache should + * point to a {@code nullptr} object. + * Default: false + * + * @param noBlockCache if use block cache + * @return the reference to the current config. + */ + public BlockBasedTableConfig setNoBlockCache(final boolean noBlockCache) { + noBlockCache_ = noBlockCache; + return this; + } + + /** + * @return if block cache is disabled + */ + public boolean noBlockCache() { + return noBlockCache_; + } + + /** + * Set the amount of cache in bytes that will be used by RocksDB. + * If cacheSize is non-positive, then cache will not be used. + * DEFAULT: 8M + * + * @param blockCacheSize block cache size in bytes + * @return the reference to the current config. + */ + public BlockBasedTableConfig setBlockCacheSize(final long blockCacheSize) { + blockCacheSize_ = blockCacheSize; + return this; + } + + /** + * @return block cache size in bytes + */ + public long blockCacheSize() { + return blockCacheSize_; + } + + /** + * Controls the number of shards for the block cache. + * This is applied only if cacheSize is set to non-negative. + * + * @param blockCacheNumShardBits the number of shard bits. The resulting + * number of shards would be 2 ^ numShardBits. Any negative + * number means use default settings." + * @return the reference to the current option. + */ + public BlockBasedTableConfig setCacheNumShardBits( + final int blockCacheNumShardBits) { + blockCacheNumShardBits_ = blockCacheNumShardBits; + return this; + } + + /** + * Returns the number of shard bits used in the block cache. + * The resulting number of shards would be 2 ^ (returned value). + * Any negative number means use default settings. + * + * @return the number of shard bits used in the block cache. + */ + public int cacheNumShardBits() { + return blockCacheNumShardBits_; + } + + /** + * Approximate size of user data packed per block. Note that the + * block size specified here corresponds to uncompressed data. The + * actual size of the unit read from disk may be smaller if + * compression is enabled. This parameter can be changed dynamically. + * Default: 4K + * + * @param blockSize block size in bytes + * @return the reference to the current config. + */ + public BlockBasedTableConfig setBlockSize(final long blockSize) { + blockSize_ = blockSize; + return this; + } + + /** + * @return block size in bytes + */ + public long blockSize() { + return blockSize_; + } + + /** + * This is used to close a block before it reaches the configured + * 'block_size'. If the percentage of free space in the current block is less + * than this specified number and adding a new record to the block will + * exceed the configured block size, then this block will be closed and the + * new record will be written to the next block. + * Default is 10. + * + * @param blockSizeDeviation the deviation to block size allowed + * @return the reference to the current config. + */ + public BlockBasedTableConfig setBlockSizeDeviation( + final int blockSizeDeviation) { + blockSizeDeviation_ = blockSizeDeviation; + return this; + } + + /** + * @return the hash table ratio. + */ + public int blockSizeDeviation() { + return blockSizeDeviation_; + } + + /** + * Set block restart interval + * + * @param restartInterval block restart interval. + * @return the reference to the current config. + */ + public BlockBasedTableConfig setBlockRestartInterval( + final int restartInterval) { + blockRestartInterval_ = restartInterval; + return this; + } + + /** + * @return block restart interval + */ + public int blockRestartInterval() { + return blockRestartInterval_; + } + + /** + * If true, place whole keys in the filter (not just prefixes). + * This must generally be true for gets to be efficient. + * Default: true + * + * @param wholeKeyFiltering if enable whole key filtering + * @return the reference to the current config. + */ + public BlockBasedTableConfig setWholeKeyFiltering( + final boolean wholeKeyFiltering) { + wholeKeyFiltering_ = wholeKeyFiltering; + return this; + } + + /** + * @return if whole key filtering is enabled + */ + public boolean wholeKeyFiltering() { + return wholeKeyFiltering_; + } + + /** + * Use the specified filter policy to reduce disk reads. + * + * {@link org.rocksdb.Filter} should not be disposed before options instances + * using this filter is disposed. If {@link Filter#dispose()} function is not + * called, then filter object will be GC'd automatically. + * + * {@link org.rocksdb.Filter} instance can be re-used in multiple options + * instances. + * + * @param filter {@link org.rocksdb.Filter} Filter Policy java instance. + * @return the reference to the current config. + */ + public BlockBasedTableConfig setFilter( + final Filter filter) { + filter_ = filter; + return this; + } + + /** + * Indicating if we'd put index/filter blocks to the block cache. + If not specified, each "table reader" object will pre-load index/filter + block during table initialization. + * + * @return if index and filter blocks should be put in block cache. + */ + public boolean cacheIndexAndFilterBlocks() { + return cacheIndexAndFilterBlocks_; + } + + /** + * Indicating if we'd put index/filter blocks to the block cache. + If not specified, each "table reader" object will pre-load index/filter + block during table initialization. + * + * @param cacheIndexAndFilterBlocks and filter blocks should be put in block cache. + * @return the reference to the current config. + */ + public BlockBasedTableConfig setCacheIndexAndFilterBlocks( + final boolean cacheIndexAndFilterBlocks) { + cacheIndexAndFilterBlocks_ = cacheIndexAndFilterBlocks; + return this; + } + + /** + * Indicating if we'd like to pin L0 index/filter blocks to the block cache. + If not specified, defaults to false. + * + * @return if L0 index and filter blocks should be pinned to the block cache. + */ + public boolean pinL0FilterAndIndexBlocksInCache() { + return pinL0FilterAndIndexBlocksInCache_; + } + + /** + * Indicating if we'd like to pin L0 index/filter blocks to the block cache. + If not specified, defaults to false. + * + * @param pinL0FilterAndIndexBlocksInCache pin blocks in block cache + * @return the reference to the current config. + */ + public BlockBasedTableConfig setPinL0FilterAndIndexBlocksInCache( + final boolean pinL0FilterAndIndexBlocksInCache) { + pinL0FilterAndIndexBlocksInCache_ = pinL0FilterAndIndexBlocksInCache; + return this; + } + + /** + * Influence the behavior when kHashSearch is used. + if false, stores a precise prefix to block range mapping + if true, does not store prefix and allows prefix hash collision + (less memory consumption) + * + * @return if hash collisions should be allowed. + */ + public boolean hashIndexAllowCollision() { + return hashIndexAllowCollision_; + } + + /** + * Influence the behavior when kHashSearch is used. + if false, stores a precise prefix to block range mapping + if true, does not store prefix and allows prefix hash collision + (less memory consumption) + * + * @param hashIndexAllowCollision points out if hash collisions should be allowed. + * @return the reference to the current config. + */ + public BlockBasedTableConfig setHashIndexAllowCollision( + final boolean hashIndexAllowCollision) { + hashIndexAllowCollision_ = hashIndexAllowCollision; + return this; + } + + /** + * Size of compressed block cache. If 0, then block_cache_compressed is set + * to null. + * + * @return size of compressed block cache. + */ + public long blockCacheCompressedSize() { + return blockCacheCompressedSize_; + } + + /** + * Size of compressed block cache. If 0, then block_cache_compressed is set + * to null. + * + * @param blockCacheCompressedSize of compressed block cache. + * @return the reference to the current config. + */ + public BlockBasedTableConfig setBlockCacheCompressedSize( + final long blockCacheCompressedSize) { + blockCacheCompressedSize_ = blockCacheCompressedSize; + return this; + } + + /** + * Controls the number of shards for the block compressed cache. + * This is applied only if blockCompressedCacheSize is set to non-negative. + * + * @return numShardBits the number of shard bits. The resulting + * number of shards would be 2 ^ numShardBits. Any negative + * number means use default settings. + */ + public int blockCacheCompressedNumShardBits() { + return blockCacheCompressedNumShardBits_; + } + + /** + * Controls the number of shards for the block compressed cache. + * This is applied only if blockCompressedCacheSize is set to non-negative. + * + * @param blockCacheCompressedNumShardBits the number of shard bits. The resulting + * number of shards would be 2 ^ numShardBits. Any negative + * number means use default settings." + * @return the reference to the current option. + */ + public BlockBasedTableConfig setBlockCacheCompressedNumShardBits( + final int blockCacheCompressedNumShardBits) { + blockCacheCompressedNumShardBits_ = blockCacheCompressedNumShardBits; + return this; + } + + /** + * Sets the checksum type to be used with this table. + * + * @param checksumType {@link org.rocksdb.ChecksumType} value. + * @return the reference to the current option. + */ + public BlockBasedTableConfig setChecksumType( + final ChecksumType checksumType) { + checksumType_ = checksumType; + return this; + } + + /** + * + * @return the currently set checksum type + */ + public ChecksumType checksumType() { + return checksumType_; + } + + /** + * Sets the index type to used with this table. + * + * @param indexType {@link org.rocksdb.IndexType} value + * @return the reference to the current option. + */ + public BlockBasedTableConfig setIndexType( + final IndexType indexType) { + indexType_ = indexType; + return this; + } + + /** + * + * @return the currently set index type + */ + public IndexType indexType() { + return indexType_; + } + + /** + *

    We currently have three versions:

    + * + *
      + *
    • 0 - This version is currently written + * out by all RocksDB's versions by default. Can be read by really old + * RocksDB's. Doesn't support changing checksum (default is CRC32).
      • + *
    • 1 - Can be read by RocksDB's versions since 3.0. + * Supports non-default checksum, like xxHash. It is written by RocksDB when + * BlockBasedTableOptions::checksum is something other than kCRC32c. (version + * 0 is silently upconverted)
      • + *
    • 2 - Can be read by RocksDB's versions since 3.10. + * Changes the way we encode compressed blocks with LZ4, BZip2 and Zlib + * compression. If you don't plan to run RocksDB before version 3.10, + * you should probably use this.
      • + *
    + *

    This option only affects newly written tables. When reading existing + * tables, the information about version is read from the footer.

    + * + * @param formatVersion integer representing the version to be used. + * @return the reference to the current option. + */ + public BlockBasedTableConfig setFormatVersion( + final int formatVersion) { + assert(formatVersion >= 0 && formatVersion <= 2); + formatVersion_ = formatVersion; + return this; + } + + /** + * + * @return the currently configured format version. + * See also: {@link #setFormatVersion(int)}. + */ + public int formatVersion() { + return formatVersion_; + } + + + + @Override protected long newTableFactoryHandle() { + long filterHandle = 0; + if (filter_ != null) { + filterHandle = filter_.nativeHandle_; + } + + return newTableFactoryHandle(noBlockCache_, blockCacheSize_, + blockCacheNumShardBits_, blockSize_, blockSizeDeviation_, + blockRestartInterval_, wholeKeyFiltering_, + filterHandle, cacheIndexAndFilterBlocks_, + pinL0FilterAndIndexBlocksInCache_, + hashIndexAllowCollision_, blockCacheCompressedSize_, + blockCacheCompressedNumShardBits_, + checksumType_.getValue(), indexType_.getValue(), + formatVersion_); + } + + private native long newTableFactoryHandle( + boolean noBlockCache, long blockCacheSize, int blockCacheNumShardBits, + long blockSize, int blockSizeDeviation, int blockRestartInterval, + boolean wholeKeyFiltering, long filterPolicyHandle, + boolean cacheIndexAndFilterBlocks, boolean pinL0FilterAndIndexBlocksInCache, + boolean hashIndexAllowCollision, long blockCacheCompressedSize, + int blockCacheCompressedNumShardBits, byte checkSumType, + byte indexType, int formatVersion); + + private boolean cacheIndexAndFilterBlocks_; + private boolean pinL0FilterAndIndexBlocksInCache_; + private IndexType indexType_; + private boolean hashIndexAllowCollision_; + private ChecksumType checksumType_; + private boolean noBlockCache_; + private long blockSize_; + private long blockCacheSize_; + private int blockCacheNumShardBits_; + private long blockCacheCompressedSize_; + private int blockCacheCompressedNumShardBits_; + private int blockSizeDeviation_; + private int blockRestartInterval_; + private Filter filter_; + private boolean wholeKeyFiltering_; + private int formatVersion_; +} diff --git a/java/src/main/java/org/rocksdb/BloomFilter.java b/java/src/main/java/org/rocksdb/BloomFilter.java new file mode 100644 index 00000000000..316c3ad838b --- /dev/null +++ b/java/src/main/java/org/rocksdb/BloomFilter.java @@ -0,0 +1,79 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Bloom filter policy that uses a bloom filter with approximately + * the specified number of bits per key. + * + *

    + * Note: if you are using a custom comparator that ignores some parts + * of the keys being compared, you must not use this {@code BloomFilter} + * and must provide your own FilterPolicy that also ignores the + * corresponding parts of the keys. For example, if the comparator + * ignores trailing spaces, it would be incorrect to use a + * FilterPolicy (like {@code BloomFilter}) that does not ignore + * trailing spaces in keys.

    + */ +public class BloomFilter extends Filter { + + private static final int DEFAULT_BITS_PER_KEY = 10; + private static final boolean DEFAULT_MODE = true; + + /** + * BloomFilter constructor + * + *

    + * Callers must delete the result after any database that is using the + * result has been closed.

    + */ + public BloomFilter() { + this(DEFAULT_BITS_PER_KEY, DEFAULT_MODE); + } + + /** + * BloomFilter constructor + * + *

    + * bits_per_key: bits per key in bloom filter. A good value for bits_per_key + * is 10, which yields a filter with ~ 1% false positive rate. + *

    + *

    + * Callers must delete the result after any database that is using the + * result has been closed.

    + * + * @param bitsPerKey number of bits to use + */ + public BloomFilter(final int bitsPerKey) { + this(bitsPerKey, DEFAULT_MODE); + } + + /** + * BloomFilter constructor + * + *

    + * bits_per_key: bits per key in bloom filter. A good value for bits_per_key + * is 10, which yields a filter with ~ 1% false positive rate. + *

    default bits_per_key: 10

    + * + *

    use_block_based_builder: use block based filter rather than full filter. + * If you want to builder full filter, it needs to be set to false. + *

    + *

    default mode: block based filter

    + *

    + * Callers must delete the result after any database that is using the + * result has been closed.

    + * + * @param bitsPerKey number of bits to use + * @param useBlockBasedMode use block based mode or full filter mode + */ + public BloomFilter(final int bitsPerKey, final boolean useBlockBasedMode) { + super(createNewBloomFilter(bitsPerKey, useBlockBasedMode)); + } + + private native static long createNewBloomFilter(final int bitsKeyKey, + final boolean useBlockBasedMode); +} diff --git a/java/src/main/java/org/rocksdb/BuiltinComparator.java b/java/src/main/java/org/rocksdb/BuiltinComparator.java new file mode 100644 index 00000000000..2c89bf218d1 --- /dev/null +++ b/java/src/main/java/org/rocksdb/BuiltinComparator.java @@ -0,0 +1,20 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Builtin RocksDB comparators + * + *
      + *
    1. BYTEWISE_COMPARATOR - Sorts all keys in ascending bytewise + * order.
      2. + *
    2. REVERSE_BYTEWISE_COMPARATOR - Sorts all keys in descending bytewise + * order
      3. + *
    + */ +public enum BuiltinComparator { + BYTEWISE_COMPARATOR, REVERSE_BYTEWISE_COMPARATOR +} diff --git a/java/src/main/java/org/rocksdb/Cache.java b/java/src/main/java/org/rocksdb/Cache.java new file mode 100644 index 00000000000..3952e1d109c --- /dev/null +++ b/java/src/main/java/org/rocksdb/Cache.java @@ -0,0 +1,13 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + + +public abstract class Cache extends RocksObject { + protected Cache(final long nativeHandle) { + super(nativeHandle); + } +} diff --git a/java/src/main/java/org/rocksdb/CassandraCompactionFilter.java b/java/src/main/java/org/rocksdb/CassandraCompactionFilter.java new file mode 100644 index 00000000000..26bf3588355 --- /dev/null +++ b/java/src/main/java/org/rocksdb/CassandraCompactionFilter.java @@ -0,0 +1,18 @@ +// Copyright (c) 2017-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Just a Java wrapper around CassandraCompactionFilter implemented in C++ + */ +public class CassandraCompactionFilter + extends AbstractCompactionFilter { + public CassandraCompactionFilter(boolean purgeTtlOnExpiration) { + super(createNewCassandraCompactionFilter0(purgeTtlOnExpiration)); + } + + private native static long createNewCassandraCompactionFilter0(boolean purgeTtlOnExpiration); +} diff --git a/java/src/main/java/org/rocksdb/CassandraValueMergeOperator.java b/java/src/main/java/org/rocksdb/CassandraValueMergeOperator.java new file mode 100644 index 00000000000..a09556a2b8d --- /dev/null +++ b/java/src/main/java/org/rocksdb/CassandraValueMergeOperator.java @@ -0,0 +1,20 @@ +// Copyright (c) 2017-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * CassandraValueMergeOperator is a merge operator that merges two cassandra wide column + * values. + */ +public class CassandraValueMergeOperator extends MergeOperator { + public CassandraValueMergeOperator() { + super(newSharedCassandraValueMergeOperator()); + } + + private native static long newSharedCassandraValueMergeOperator(); + + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/Checkpoint.java b/java/src/main/java/org/rocksdb/Checkpoint.java new file mode 100644 index 00000000000..0009699325c --- /dev/null +++ b/java/src/main/java/org/rocksdb/Checkpoint.java @@ -0,0 +1,66 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Provides Checkpoint functionality. Checkpoints + * provide persistent snapshots of RocksDB databases. + */ +public class Checkpoint extends RocksObject { + + /** + * Creates a Checkpoint object to be used for creating open-able + * snapshots. + * + * @param db {@link RocksDB} instance. + * @return a Checkpoint instance. + * + * @throws java.lang.IllegalArgumentException if {@link RocksDB} + * instance is null. + * @throws java.lang.IllegalStateException if {@link RocksDB} + * instance is not initialized. + */ + public static Checkpoint create(final RocksDB db) { + if (db == null) { + throw new IllegalArgumentException( + "RocksDB instance shall not be null."); + } else if (!db.isOwningHandle()) { + throw new IllegalStateException( + "RocksDB instance must be initialized."); + } + Checkpoint checkpoint = new Checkpoint(db); + return checkpoint; + } + + /** + *

    Builds an open-able snapshot of RocksDB on the same disk, which + * accepts an output directory on the same disk, and under the directory + * (1) hard-linked SST files pointing to existing live SST files + * (2) a copied manifest files and other files

    + * + * @param checkpointPath path to the folder where the snapshot is going + * to be stored. + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void createCheckpoint(final String checkpointPath) + throws RocksDBException { + createCheckpoint(nativeHandle_, checkpointPath); + } + + private Checkpoint(final RocksDB db) { + super(newCheckpoint(db.nativeHandle_)); + this.db_ = db; + } + + private final RocksDB db_; + + private static native long newCheckpoint(long dbHandle); + @Override protected final native void disposeInternal(final long handle); + + private native void createCheckpoint(long handle, String checkpointPath) + throws RocksDBException; +} diff --git a/java/src/main/java/org/rocksdb/ChecksumType.java b/java/src/main/java/org/rocksdb/ChecksumType.java new file mode 100644 index 00000000000..def9f2e9f49 --- /dev/null +++ b/java/src/main/java/org/rocksdb/ChecksumType.java @@ -0,0 +1,39 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Checksum types used in conjunction with BlockBasedTable. + */ +public enum ChecksumType { + /** + * Not implemented yet. + */ + kNoChecksum((byte) 0), + /** + * CRC32 Checksum + */ + kCRC32c((byte) 1), + /** + * XX Hash + */ + kxxHash((byte) 2); + + /** + * Returns the byte value of the enumerations value + * + * @return byte representation + */ + public byte getValue() { + return value_; + } + + private ChecksumType(byte value) { + value_ = value; + } + + private final byte value_; +} diff --git a/java/src/main/java/org/rocksdb/ClockCache.java b/java/src/main/java/org/rocksdb/ClockCache.java new file mode 100644 index 00000000000..a66dc0e8a72 --- /dev/null +++ b/java/src/main/java/org/rocksdb/ClockCache.java @@ -0,0 +1,59 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Similar to {@link LRUCache}, but based on the CLOCK algorithm with + * better concurrent performance in some cases + */ +public class ClockCache extends Cache { + + /** + * Create a new cache with a fixed size capacity. + * + * @param capacity The fixed size capacity of the cache + */ + public ClockCache(final long capacity) { + super(newClockCache(capacity, -1, false)); + } + + /** + * Create a new cache with a fixed size capacity. The cache is sharded + * to 2^numShardBits shards, by hash of the key. The total capacity + * is divided and evenly assigned to each shard. + * numShardBits = -1 means it is automatically determined: every shard + * will be at least 512KB and number of shard bits will not exceed 6. + * + * @param capacity The fixed size capacity of the cache + * @param numShardBits The cache is sharded to 2^numShardBits shards, + * by hash of the key + */ + public ClockCache(final long capacity, final int numShardBits) { + super(newClockCache(capacity, numShardBits, false)); + } + + /** + * Create a new cache with a fixed size capacity. The cache is sharded + * to 2^numShardBits shards, by hash of the key. The total capacity + * is divided and evenly assigned to each shard. If strictCapacityLimit + * is set, insert to the cache will fail when cache is full. + * numShardBits = -1 means it is automatically determined: every shard + * will be at least 512KB and number of shard bits will not exceed 6. + * + * @param capacity The fixed size capacity of the cache + * @param numShardBits The cache is sharded to 2^numShardBits shards, + * by hash of the key + * @param strictCapacityLimit insert to the cache will fail when cache is full + */ + public ClockCache(final long capacity, final int numShardBits, + final boolean strictCapacityLimit) { + super(newClockCache(capacity, numShardBits, strictCapacityLimit)); + } + + private native static long newClockCache(final long capacity, + final int numShardBits, final boolean strictCapacityLimit); + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/ColumnFamilyDescriptor.java b/java/src/main/java/org/rocksdb/ColumnFamilyDescriptor.java new file mode 100644 index 00000000000..d932fd9a927 --- /dev/null +++ b/java/src/main/java/org/rocksdb/ColumnFamilyDescriptor.java @@ -0,0 +1,61 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + *

    Describes a column family with a + * name and respective Options.

    + */ +public class ColumnFamilyDescriptor { + + /** + *

    Creates a new Column Family using a name and default + * options,

    + * + * @param columnFamilyName name of column family. + * @since 3.10.0 + */ + public ColumnFamilyDescriptor(final byte[] columnFamilyName) { + this(columnFamilyName, new ColumnFamilyOptions()); + } + + /** + *

    Creates a new Column Family using a name and custom + * options.

    + * + * @param columnFamilyName name of column family. + * @param columnFamilyOptions options to be used with + * column family. + * @since 3.10.0 + */ + public ColumnFamilyDescriptor(final byte[] columnFamilyName, + final ColumnFamilyOptions columnFamilyOptions) { + columnFamilyName_ = columnFamilyName; + columnFamilyOptions_ = columnFamilyOptions; + } + + /** + * Retrieve name of column family. + * + * @return column family name. + * @since 3.10.0 + */ + public byte[] columnFamilyName() { + return columnFamilyName_; + } + + /** + * Retrieve assigned options instance. + * + * @return Options instance assigned to this instance. + */ + public ColumnFamilyOptions columnFamilyOptions() { + return columnFamilyOptions_; + } + + private final byte[] columnFamilyName_; + private final ColumnFamilyOptions columnFamilyOptions_; +} diff --git a/java/src/main/java/org/rocksdb/ColumnFamilyHandle.java b/java/src/main/java/org/rocksdb/ColumnFamilyHandle.java new file mode 100644 index 00000000000..7726cc62d79 --- /dev/null +++ b/java/src/main/java/org/rocksdb/ColumnFamilyHandle.java @@ -0,0 +1,42 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * ColumnFamilyHandle class to hold handles to underlying rocksdb + * ColumnFamily Pointers. + */ +public class ColumnFamilyHandle extends RocksObject { + ColumnFamilyHandle(final RocksDB rocksDB, + final long nativeHandle) { + super(nativeHandle); + // rocksDB must point to a valid RocksDB instance; + assert(rocksDB != null); + // ColumnFamilyHandle must hold a reference to the related RocksDB instance + // to guarantee that while a GC cycle starts ColumnFamilyHandle instances + // are freed prior to RocksDB instances. + this.rocksDB_ = rocksDB; + } + + /** + *

    Deletes underlying C++ iterator pointer.

    + * + *

    Note: the underlying handle can only be safely deleted if the RocksDB + * instance related to a certain ColumnFamilyHandle is still valid and + * initialized. Therefore {@code disposeInternal()} checks if the RocksDB is + * initialized before freeing the native handle.

    + */ + @Override + protected void disposeInternal() { + if(rocksDB_.isOwningHandle()) { + disposeInternal(nativeHandle_); + } + } + + @Override protected final native void disposeInternal(final long handle); + + private final RocksDB rocksDB_; +} diff --git a/java/src/main/java/org/rocksdb/ColumnFamilyOptions.java b/java/src/main/java/org/rocksdb/ColumnFamilyOptions.java new file mode 100644 index 00000000000..647b92e16cb --- /dev/null +++ b/java/src/main/java/org/rocksdb/ColumnFamilyOptions.java @@ -0,0 +1,909 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.ArrayList; +import java.util.List; +import java.util.Properties; + +/** + * ColumnFamilyOptions to control the behavior of a database. It will be used + * during the creation of a {@link org.rocksdb.RocksDB} (i.e., RocksDB.open()). + * + * If {@link #dispose()} function is not called, then it will be GC'd + * automatically and native resources will be released as part of the process. + */ +public class ColumnFamilyOptions extends RocksObject + implements ColumnFamilyOptionsInterface, + MutableColumnFamilyOptionsInterface { + static { + RocksDB.loadLibrary(); + } + + /** + * Construct ColumnFamilyOptions. + * + * This constructor will create (by allocating a block of memory) + * an {@code rocksdb::DBOptions} in the c++ side. + */ + public ColumnFamilyOptions() { + super(newColumnFamilyOptions()); + } + + /** + *

    Method to get a options instance by using pre-configured + * property values. If one or many values are undefined in + * the context of RocksDB the method will return a null + * value.

    + * + *

    Note: Property keys can be derived from + * getter methods within the options class. Example: the method + * {@code writeBufferSize()} has a property key: + * {@code write_buffer_size}.

    + * + * @param properties {@link java.util.Properties} instance. + * + * @return {@link org.rocksdb.ColumnFamilyOptions instance} + * or null. + * + * @throws java.lang.IllegalArgumentException if null or empty + * {@link Properties} instance is passed to the method call. + */ + public static ColumnFamilyOptions getColumnFamilyOptionsFromProps( + final Properties properties) { + if (properties == null || properties.size() == 0) { + throw new IllegalArgumentException( + "Properties value must contain at least one value."); + } + ColumnFamilyOptions columnFamilyOptions = null; + StringBuilder stringBuilder = new StringBuilder(); + for (final String name : properties.stringPropertyNames()){ + stringBuilder.append(name); + stringBuilder.append("="); + stringBuilder.append(properties.getProperty(name)); + stringBuilder.append(";"); + } + long handle = getColumnFamilyOptionsFromProps( + stringBuilder.toString()); + if (handle != 0){ + columnFamilyOptions = new ColumnFamilyOptions(handle); + } + return columnFamilyOptions; + } + + @Override + public ColumnFamilyOptions optimizeForSmallDb() { + optimizeForSmallDb(nativeHandle_); + return this; + } + + @Override + public ColumnFamilyOptions optimizeForPointLookup( + final long blockCacheSizeMb) { + optimizeForPointLookup(nativeHandle_, + blockCacheSizeMb); + return this; + } + + @Override + public ColumnFamilyOptions optimizeLevelStyleCompaction() { + optimizeLevelStyleCompaction(nativeHandle_, + DEFAULT_COMPACTION_MEMTABLE_MEMORY_BUDGET); + return this; + } + + @Override + public ColumnFamilyOptions optimizeLevelStyleCompaction( + final long memtableMemoryBudget) { + optimizeLevelStyleCompaction(nativeHandle_, + memtableMemoryBudget); + return this; + } + + @Override + public ColumnFamilyOptions optimizeUniversalStyleCompaction() { + optimizeUniversalStyleCompaction(nativeHandle_, + DEFAULT_COMPACTION_MEMTABLE_MEMORY_BUDGET); + return this; + } + + @Override + public ColumnFamilyOptions optimizeUniversalStyleCompaction( + final long memtableMemoryBudget) { + optimizeUniversalStyleCompaction(nativeHandle_, + memtableMemoryBudget); + return this; + } + + @Override + public ColumnFamilyOptions setComparator( + final BuiltinComparator builtinComparator) { + assert(isOwningHandle()); + setComparatorHandle(nativeHandle_, builtinComparator.ordinal()); + return this; + } + + @Override + public ColumnFamilyOptions setComparator( + final AbstractComparator> comparator) { + assert (isOwningHandle()); + setComparatorHandle(nativeHandle_, comparator.getNativeHandle()); + comparator_ = comparator; + return this; + } + + @Override + public ColumnFamilyOptions setMergeOperatorName(final String name) { + assert (isOwningHandle()); + if (name == null) { + throw new IllegalArgumentException( + "Merge operator name must not be null."); + } + setMergeOperatorName(nativeHandle_, name); + return this; + } + + @Override + public ColumnFamilyOptions setMergeOperator( + final MergeOperator mergeOperator) { + setMergeOperator(nativeHandle_, mergeOperator.nativeHandle_); + return this; + } + + public ColumnFamilyOptions setCompactionFilter( + final AbstractCompactionFilter> + compactionFilter) { + setCompactionFilterHandle(nativeHandle_, compactionFilter.nativeHandle_); + compactionFilter_ = compactionFilter; + return this; + } + + @Override + public ColumnFamilyOptions setWriteBufferSize(final long writeBufferSize) { + assert(isOwningHandle()); + setWriteBufferSize(nativeHandle_, writeBufferSize); + return this; + } + + @Override + public long writeBufferSize() { + assert(isOwningHandle()); + return writeBufferSize(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setMaxWriteBufferNumber( + final int maxWriteBufferNumber) { + assert(isOwningHandle()); + setMaxWriteBufferNumber(nativeHandle_, maxWriteBufferNumber); + return this; + } + + @Override + public int maxWriteBufferNumber() { + assert(isOwningHandle()); + return maxWriteBufferNumber(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setMinWriteBufferNumberToMerge( + final int minWriteBufferNumberToMerge) { + setMinWriteBufferNumberToMerge(nativeHandle_, minWriteBufferNumberToMerge); + return this; + } + + @Override + public int minWriteBufferNumberToMerge() { + return minWriteBufferNumberToMerge(nativeHandle_); + } + + @Override + public ColumnFamilyOptions useFixedLengthPrefixExtractor(final int n) { + assert(isOwningHandle()); + useFixedLengthPrefixExtractor(nativeHandle_, n); + return this; + } + + @Override + public ColumnFamilyOptions useCappedPrefixExtractor(final int n) { + assert(isOwningHandle()); + useCappedPrefixExtractor(nativeHandle_, n); + return this; + } + + @Override + public ColumnFamilyOptions setCompressionType( + final CompressionType compressionType) { + setCompressionType(nativeHandle_, compressionType.getValue()); + return this; + } + + @Override + public CompressionType compressionType() { + return CompressionType.getCompressionType(compressionType(nativeHandle_)); + } + + @Override + public ColumnFamilyOptions setCompressionPerLevel( + final List compressionLevels) { + final byte[] byteCompressionTypes = new byte[ + compressionLevels.size()]; + for (int i = 0; i < compressionLevels.size(); i++) { + byteCompressionTypes[i] = compressionLevels.get(i).getValue(); + } + setCompressionPerLevel(nativeHandle_, byteCompressionTypes); + return this; + } + + @Override + public List compressionPerLevel() { + final byte[] byteCompressionTypes = + compressionPerLevel(nativeHandle_); + final List compressionLevels = new ArrayList<>(); + for (final Byte byteCompressionType : byteCompressionTypes) { + compressionLevels.add(CompressionType.getCompressionType( + byteCompressionType)); + } + return compressionLevels; + } + + @Override + public ColumnFamilyOptions setBottommostCompressionType( + final CompressionType bottommostCompressionType) { + setBottommostCompressionType(nativeHandle_, + bottommostCompressionType.getValue()); + return this; + } + + @Override + public CompressionType bottommostCompressionType() { + return CompressionType.getCompressionType( + bottommostCompressionType(nativeHandle_)); + } + + @Override + public ColumnFamilyOptions setCompressionOptions( + final CompressionOptions compressionOptions) { + setCompressionOptions(nativeHandle_, compressionOptions.nativeHandle_); + this.compressionOptions_ = compressionOptions; + return this; + } + + @Override + public CompressionOptions compressionOptions() { + return this.compressionOptions_; + } + + @Override + public ColumnFamilyOptions setNumLevels(final int numLevels) { + setNumLevels(nativeHandle_, numLevels); + return this; + } + + @Override + public int numLevels() { + return numLevels(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setLevelZeroFileNumCompactionTrigger( + final int numFiles) { + setLevelZeroFileNumCompactionTrigger( + nativeHandle_, numFiles); + return this; + } + + @Override + public int levelZeroFileNumCompactionTrigger() { + return levelZeroFileNumCompactionTrigger(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setLevelZeroSlowdownWritesTrigger( + final int numFiles) { + setLevelZeroSlowdownWritesTrigger(nativeHandle_, numFiles); + return this; + } + + @Override + public int levelZeroSlowdownWritesTrigger() { + return levelZeroSlowdownWritesTrigger(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setLevelZeroStopWritesTrigger(final int numFiles) { + setLevelZeroStopWritesTrigger(nativeHandle_, numFiles); + return this; + } + + @Override + public int levelZeroStopWritesTrigger() { + return levelZeroStopWritesTrigger(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setTargetFileSizeBase( + final long targetFileSizeBase) { + setTargetFileSizeBase(nativeHandle_, targetFileSizeBase); + return this; + } + + @Override + public long targetFileSizeBase() { + return targetFileSizeBase(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setTargetFileSizeMultiplier( + final int multiplier) { + setTargetFileSizeMultiplier(nativeHandle_, multiplier); + return this; + } + + @Override + public int targetFileSizeMultiplier() { + return targetFileSizeMultiplier(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setMaxBytesForLevelBase( + final long maxBytesForLevelBase) { + setMaxBytesForLevelBase(nativeHandle_, maxBytesForLevelBase); + return this; + } + + @Override + public long maxBytesForLevelBase() { + return maxBytesForLevelBase(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setLevelCompactionDynamicLevelBytes( + final boolean enableLevelCompactionDynamicLevelBytes) { + setLevelCompactionDynamicLevelBytes(nativeHandle_, + enableLevelCompactionDynamicLevelBytes); + return this; + } + + @Override + public boolean levelCompactionDynamicLevelBytes() { + return levelCompactionDynamicLevelBytes(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setMaxBytesForLevelMultiplier(final double multiplier) { + setMaxBytesForLevelMultiplier(nativeHandle_, multiplier); + return this; + } + + @Override + public double maxBytesForLevelMultiplier() { + return maxBytesForLevelMultiplier(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setMaxCompactionBytes(final long maxCompactionBytes) { + setMaxCompactionBytes(nativeHandle_, maxCompactionBytes); + return this; + } + + @Override + public long maxCompactionBytes() { + return maxCompactionBytes(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setArenaBlockSize( + final long arenaBlockSize) { + setArenaBlockSize(nativeHandle_, arenaBlockSize); + return this; + } + + @Override + public long arenaBlockSize() { + return arenaBlockSize(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setDisableAutoCompactions( + final boolean disableAutoCompactions) { + setDisableAutoCompactions(nativeHandle_, disableAutoCompactions); + return this; + } + + @Override + public boolean disableAutoCompactions() { + return disableAutoCompactions(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setCompactionStyle( + final CompactionStyle compactionStyle) { + setCompactionStyle(nativeHandle_, compactionStyle.getValue()); + return this; + } + + @Override + public CompactionStyle compactionStyle() { + return CompactionStyle.values()[compactionStyle(nativeHandle_)]; + } + + @Override + public ColumnFamilyOptions setMaxTableFilesSizeFIFO( + final long maxTableFilesSize) { + assert(maxTableFilesSize > 0); // unsigned native type + assert(isOwningHandle()); + setMaxTableFilesSizeFIFO(nativeHandle_, maxTableFilesSize); + return this; + } + + @Override + public long maxTableFilesSizeFIFO() { + return maxTableFilesSizeFIFO(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setMaxSequentialSkipInIterations( + final long maxSequentialSkipInIterations) { + setMaxSequentialSkipInIterations(nativeHandle_, + maxSequentialSkipInIterations); + return this; + } + + @Override + public long maxSequentialSkipInIterations() { + return maxSequentialSkipInIterations(nativeHandle_); + } + + @Override + public MemTableConfig memTableConfig() { + return this.memTableConfig_; + } + + @Override + public ColumnFamilyOptions setMemTableConfig( + final MemTableConfig memTableConfig) { + setMemTableFactory( + nativeHandle_, memTableConfig.newMemTableFactoryHandle()); + this.memTableConfig_ = memTableConfig; + return this; + } + + @Override + public String memTableFactoryName() { + assert(isOwningHandle()); + return memTableFactoryName(nativeHandle_); + } + + @Override + public TableFormatConfig tableFormatConfig() { + return this.tableFormatConfig_; + } + + @Override + public ColumnFamilyOptions setTableFormatConfig( + final TableFormatConfig tableFormatConfig) { + setTableFactory(nativeHandle_, tableFormatConfig.newTableFactoryHandle()); + this.tableFormatConfig_ = tableFormatConfig; + return this; + } + + @Override + public String tableFactoryName() { + assert(isOwningHandle()); + return tableFactoryName(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setInplaceUpdateSupport( + final boolean inplaceUpdateSupport) { + setInplaceUpdateSupport(nativeHandle_, inplaceUpdateSupport); + return this; + } + + @Override + public boolean inplaceUpdateSupport() { + return inplaceUpdateSupport(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setInplaceUpdateNumLocks( + final long inplaceUpdateNumLocks) { + setInplaceUpdateNumLocks(nativeHandle_, inplaceUpdateNumLocks); + return this; + } + + @Override + public long inplaceUpdateNumLocks() { + return inplaceUpdateNumLocks(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setMemtablePrefixBloomSizeRatio( + final double memtablePrefixBloomSizeRatio) { + setMemtablePrefixBloomSizeRatio(nativeHandle_, memtablePrefixBloomSizeRatio); + return this; + } + + @Override + public double memtablePrefixBloomSizeRatio() { + return memtablePrefixBloomSizeRatio(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setBloomLocality(int bloomLocality) { + setBloomLocality(nativeHandle_, bloomLocality); + return this; + } + + @Override + public int bloomLocality() { + return bloomLocality(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setMaxSuccessiveMerges( + final long maxSuccessiveMerges) { + setMaxSuccessiveMerges(nativeHandle_, maxSuccessiveMerges); + return this; + } + + @Override + public long maxSuccessiveMerges() { + return maxSuccessiveMerges(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setOptimizeFiltersForHits( + final boolean optimizeFiltersForHits) { + setOptimizeFiltersForHits(nativeHandle_, optimizeFiltersForHits); + return this; + } + + @Override + public boolean optimizeFiltersForHits() { + return optimizeFiltersForHits(nativeHandle_); + } + + @Override + public ColumnFamilyOptions + setMemtableHugePageSize( + long memtableHugePageSize) { + setMemtableHugePageSize(nativeHandle_, + memtableHugePageSize); + return this; + } + + @Override + public long memtableHugePageSize() { + return memtableHugePageSize(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setSoftPendingCompactionBytesLimit(long softPendingCompactionBytesLimit) { + setSoftPendingCompactionBytesLimit(nativeHandle_, + softPendingCompactionBytesLimit); + return this; + } + + @Override + public long softPendingCompactionBytesLimit() { + return softPendingCompactionBytesLimit(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setHardPendingCompactionBytesLimit(long hardPendingCompactionBytesLimit) { + setHardPendingCompactionBytesLimit(nativeHandle_, hardPendingCompactionBytesLimit); + return this; + } + + @Override + public long hardPendingCompactionBytesLimit() { + return hardPendingCompactionBytesLimit(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setLevel0FileNumCompactionTrigger(int level0FileNumCompactionTrigger) { + setLevel0FileNumCompactionTrigger(nativeHandle_, level0FileNumCompactionTrigger); + return this; + } + + @Override + public int level0FileNumCompactionTrigger() { + return level0FileNumCompactionTrigger(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setLevel0SlowdownWritesTrigger(int level0SlowdownWritesTrigger) { + setLevel0SlowdownWritesTrigger(nativeHandle_, level0SlowdownWritesTrigger); + return this; + } + + @Override + public int level0SlowdownWritesTrigger() { + return level0SlowdownWritesTrigger(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setLevel0StopWritesTrigger(int level0StopWritesTrigger) { + setLevel0StopWritesTrigger(nativeHandle_, level0StopWritesTrigger); + return this; + } + + @Override + public int level0StopWritesTrigger() { + return level0StopWritesTrigger(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setMaxBytesForLevelMultiplierAdditional(int[] maxBytesForLevelMultiplierAdditional) { + setMaxBytesForLevelMultiplierAdditional(nativeHandle_, maxBytesForLevelMultiplierAdditional); + return this; + } + + @Override + public int[] maxBytesForLevelMultiplierAdditional() { + return maxBytesForLevelMultiplierAdditional(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setParanoidFileChecks(boolean paranoidFileChecks) { + setParanoidFileChecks(nativeHandle_, paranoidFileChecks); + return this; + } + + @Override + public boolean paranoidFileChecks() { + return paranoidFileChecks(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setMaxWriteBufferNumberToMaintain( + final int maxWriteBufferNumberToMaintain) { + setMaxWriteBufferNumberToMaintain( + nativeHandle_, maxWriteBufferNumberToMaintain); + return this; + } + + @Override + public int maxWriteBufferNumberToMaintain() { + return maxWriteBufferNumberToMaintain(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setCompactionPriority( + final CompactionPriority compactionPriority) { + setCompactionPriority(nativeHandle_, compactionPriority.getValue()); + return this; + } + + @Override + public CompactionPriority compactionPriority() { + return CompactionPriority.getCompactionPriority( + compactionPriority(nativeHandle_)); + } + + @Override + public ColumnFamilyOptions setReportBgIoStats(final boolean reportBgIoStats) { + setReportBgIoStats(nativeHandle_, reportBgIoStats); + return this; + } + + @Override + public boolean reportBgIoStats() { + return reportBgIoStats(nativeHandle_); + } + + @Override + public ColumnFamilyOptions setCompactionOptionsUniversal( + final CompactionOptionsUniversal compactionOptionsUniversal) { + setCompactionOptionsUniversal(nativeHandle_, + compactionOptionsUniversal.nativeHandle_); + this.compactionOptionsUniversal_ = compactionOptionsUniversal; + return this; + } + + @Override + public CompactionOptionsUniversal compactionOptionsUniversal() { + return this.compactionOptionsUniversal_; + } + + @Override + public ColumnFamilyOptions setCompactionOptionsFIFO(final CompactionOptionsFIFO compactionOptionsFIFO) { + setCompactionOptionsFIFO(nativeHandle_, + compactionOptionsFIFO.nativeHandle_); + this.compactionOptionsFIFO_ = compactionOptionsFIFO; + return this; + } + + @Override + public CompactionOptionsFIFO compactionOptionsFIFO() { + return this.compactionOptionsFIFO_; + } + + @Override + public ColumnFamilyOptions setForceConsistencyChecks(final boolean forceConsistencyChecks) { + setForceConsistencyChecks(nativeHandle_, forceConsistencyChecks); + return this; + } + + @Override + public boolean forceConsistencyChecks() { + return forceConsistencyChecks(nativeHandle_); + } + + /** + *

    Private constructor to be used by + * {@link #getColumnFamilyOptionsFromProps(java.util.Properties)}

    + * + * @param handle native handle to ColumnFamilyOptions instance. + */ + private ColumnFamilyOptions(final long handle) { + super(handle); + } + + private static native long getColumnFamilyOptionsFromProps( + String optString); + + private static native long newColumnFamilyOptions(); + @Override protected final native void disposeInternal(final long handle); + + private native void optimizeForSmallDb(final long handle); + private native void optimizeForPointLookup(long handle, + long blockCacheSizeMb); + private native void optimizeLevelStyleCompaction(long handle, + long memtableMemoryBudget); + private native void optimizeUniversalStyleCompaction(long handle, + long memtableMemoryBudget); + private native void setComparatorHandle(long handle, int builtinComparator); + private native void setComparatorHandle(long optHandle, + long comparatorHandle); + private native void setMergeOperatorName(long handle, String name); + private native void setMergeOperator(long handle, long mergeOperatorHandle); + private native void setCompactionFilterHandle(long handle, + long compactionFilterHandle); + private native void setWriteBufferSize(long handle, long writeBufferSize) + throws IllegalArgumentException; + private native long writeBufferSize(long handle); + private native void setMaxWriteBufferNumber( + long handle, int maxWriteBufferNumber); + private native int maxWriteBufferNumber(long handle); + private native void setMinWriteBufferNumberToMerge( + long handle, int minWriteBufferNumberToMerge); + private native int minWriteBufferNumberToMerge(long handle); + private native void setCompressionType(long handle, byte compressionType); + private native byte compressionType(long handle); + private native void setCompressionPerLevel(long handle, + byte[] compressionLevels); + private native byte[] compressionPerLevel(long handle); + private native void setBottommostCompressionType(long handle, + byte bottommostCompressionType); + private native byte bottommostCompressionType(long handle); + private native void setCompressionOptions(long handle, + long compressionOptionsHandle); + private native void useFixedLengthPrefixExtractor( + long handle, int prefixLength); + private native void useCappedPrefixExtractor( + long handle, int prefixLength); + private native void setNumLevels( + long handle, int numLevels); + private native int numLevels(long handle); + private native void setLevelZeroFileNumCompactionTrigger( + long handle, int numFiles); + private native int levelZeroFileNumCompactionTrigger(long handle); + private native void setLevelZeroSlowdownWritesTrigger( + long handle, int numFiles); + private native int levelZeroSlowdownWritesTrigger(long handle); + private native void setLevelZeroStopWritesTrigger( + long handle, int numFiles); + private native int levelZeroStopWritesTrigger(long handle); + private native void setTargetFileSizeBase( + long handle, long targetFileSizeBase); + private native long targetFileSizeBase(long handle); + private native void setTargetFileSizeMultiplier( + long handle, int multiplier); + private native int targetFileSizeMultiplier(long handle); + private native void setMaxBytesForLevelBase( + long handle, long maxBytesForLevelBase); + private native long maxBytesForLevelBase(long handle); + private native void setLevelCompactionDynamicLevelBytes( + long handle, boolean enableLevelCompactionDynamicLevelBytes); + private native boolean levelCompactionDynamicLevelBytes( + long handle); + private native void setMaxBytesForLevelMultiplier(long handle, double multiplier); + private native double maxBytesForLevelMultiplier(long handle); + private native void setMaxCompactionBytes(long handle, long maxCompactionBytes); + private native long maxCompactionBytes(long handle); + private native void setArenaBlockSize( + long handle, long arenaBlockSize) + throws IllegalArgumentException; + private native long arenaBlockSize(long handle); + private native void setDisableAutoCompactions( + long handle, boolean disableAutoCompactions); + private native boolean disableAutoCompactions(long handle); + private native void setCompactionStyle(long handle, byte compactionStyle); + private native byte compactionStyle(long handle); + private native void setMaxTableFilesSizeFIFO( + long handle, long max_table_files_size); + private native long maxTableFilesSizeFIFO(long handle); + private native void setMaxSequentialSkipInIterations( + long handle, long maxSequentialSkipInIterations); + private native long maxSequentialSkipInIterations(long handle); + private native void setMemTableFactory(long handle, long factoryHandle); + private native String memTableFactoryName(long handle); + private native void setTableFactory(long handle, long factoryHandle); + private native String tableFactoryName(long handle); + private native void setInplaceUpdateSupport( + long handle, boolean inplaceUpdateSupport); + private native boolean inplaceUpdateSupport(long handle); + private native void setInplaceUpdateNumLocks( + long handle, long inplaceUpdateNumLocks) + throws IllegalArgumentException; + private native long inplaceUpdateNumLocks(long handle); + private native void setMemtablePrefixBloomSizeRatio( + long handle, double memtablePrefixBloomSizeRatio); + private native double memtablePrefixBloomSizeRatio(long handle); + private native void setBloomLocality( + long handle, int bloomLocality); + private native int bloomLocality(long handle); + private native void setMaxSuccessiveMerges( + long handle, long maxSuccessiveMerges) + throws IllegalArgumentException; + private native long maxSuccessiveMerges(long handle); + private native void setOptimizeFiltersForHits(long handle, + boolean optimizeFiltersForHits); + private native boolean optimizeFiltersForHits(long handle); + private native void setMemtableHugePageSize(long handle, + long memtableHugePageSize); + private native long memtableHugePageSize(long handle); + private native void setSoftPendingCompactionBytesLimit(long handle, + long softPendingCompactionBytesLimit); + private native long softPendingCompactionBytesLimit(long handle); + private native void setHardPendingCompactionBytesLimit(long handle, + long hardPendingCompactionBytesLimit); + private native long hardPendingCompactionBytesLimit(long handle); + private native void setLevel0FileNumCompactionTrigger(long handle, + int level0FileNumCompactionTrigger); + private native int level0FileNumCompactionTrigger(long handle); + private native void setLevel0SlowdownWritesTrigger(long handle, + int level0SlowdownWritesTrigger); + private native int level0SlowdownWritesTrigger(long handle); + private native void setLevel0StopWritesTrigger(long handle, + int level0StopWritesTrigger); + private native int level0StopWritesTrigger(long handle); + private native void setMaxBytesForLevelMultiplierAdditional(long handle, + int[] maxBytesForLevelMultiplierAdditional); + private native int[] maxBytesForLevelMultiplierAdditional(long handle); + private native void setParanoidFileChecks(long handle, + boolean paranoidFileChecks); + private native boolean paranoidFileChecks(long handle); + private native void setMaxWriteBufferNumberToMaintain(final long handle, + final int maxWriteBufferNumberToMaintain); + private native int maxWriteBufferNumberToMaintain(final long handle); + private native void setCompactionPriority(final long handle, + final byte compactionPriority); + private native byte compactionPriority(final long handle); + private native void setReportBgIoStats(final long handle, + final boolean reportBgIoStats); + private native boolean reportBgIoStats(final long handle); + private native void setCompactionOptionsUniversal(final long handle, + final long compactionOptionsUniversalHandle); + private native void setCompactionOptionsFIFO(final long handle, + final long compactionOptionsFIFOHandle); + private native void setForceConsistencyChecks(final long handle, + final boolean forceConsistencyChecks); + private native boolean forceConsistencyChecks(final long handle); + + // instance variables + private MemTableConfig memTableConfig_; + private TableFormatConfig tableFormatConfig_; + private AbstractComparator> comparator_; + private AbstractCompactionFilter> compactionFilter_; + private CompactionOptionsUniversal compactionOptionsUniversal_; + private CompactionOptionsFIFO compactionOptionsFIFO_; + private CompressionOptions compressionOptions_; + +} diff --git a/java/src/main/java/org/rocksdb/ColumnFamilyOptionsInterface.java b/java/src/main/java/org/rocksdb/ColumnFamilyOptionsInterface.java new file mode 100644 index 00000000000..5cb68b46148 --- /dev/null +++ b/java/src/main/java/org/rocksdb/ColumnFamilyOptionsInterface.java @@ -0,0 +1,375 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +public interface ColumnFamilyOptionsInterface + + extends AdvancedColumnFamilyOptionsInterface { + + /** + * Use this if your DB is very small (like under 1GB) and you don't want to + * spend lots of memory for memtables. + * + * @return the instance of the current object. + */ + T optimizeForSmallDb(); + + /** + * Use this if you don't need to keep the data sorted, i.e. you'll never use + * an iterator, only Put() and Get() API calls + * + * @param blockCacheSizeMb Block cache size in MB + * @return the instance of the current object. + */ + T optimizeForPointLookup(long blockCacheSizeMb); + + /** + *

    Default values for some parameters in ColumnFamilyOptions are not + * optimized for heavy workloads and big datasets, which means you might + * observe write stalls under some conditions. As a starting point for tuning + * RocksDB options, use the following for level style compaction.

    + * + *

    Make sure to also call IncreaseParallelism(), which will provide the + * biggest performance gains.

    + *

    Note: we might use more memory than memtable_memory_budget during high + * write rate period

    + * + * @return the instance of the current object. + */ + T optimizeLevelStyleCompaction(); + + /** + *

    Default values for some parameters in ColumnFamilyOptions are not + * optimized for heavy workloads and big datasets, which means you might + * observe write stalls under some conditions. As a starting point for tuning + * RocksDB options, use the following for level style compaction.

    + * + *

    Make sure to also call IncreaseParallelism(), which will provide the + * biggest performance gains.

    + *

    Note: we might use more memory than memtable_memory_budget during high + * write rate period

    + * + * @param memtableMemoryBudget memory budget in bytes + * @return the instance of the current object. + */ + T optimizeLevelStyleCompaction( + long memtableMemoryBudget); + + /** + *

    Default values for some parameters in ColumnFamilyOptions are not + * optimized for heavy workloads and big datasets, which means you might + * observe write stalls under some conditions. As a starting point for tuning + * RocksDB options, use the following for universal style compaction.

    + * + *

    Universal style compaction is focused on reducing Write Amplification + * Factor for big data sets, but increases Space Amplification.

    + * + *

    Make sure to also call IncreaseParallelism(), which will provide the + * biggest performance gains.

    + * + *

    Note: we might use more memory than memtable_memory_budget during high + * write rate period

    + * + * @return the instance of the current object. + */ + T optimizeUniversalStyleCompaction(); + + /** + *

    Default values for some parameters in ColumnFamilyOptions are not + * optimized for heavy workloads and big datasets, which means you might + * observe write stalls under some conditions. As a starting point for tuning + * RocksDB options, use the following for universal style compaction.

    + * + *

    Universal style compaction is focused on reducing Write Amplification + * Factor for big data sets, but increases Space Amplification.

    + * + *

    Make sure to also call IncreaseParallelism(), which will provide the + * biggest performance gains.

    + * + *

    Note: we might use more memory than memtable_memory_budget during high + * write rate period

    + * + * @param memtableMemoryBudget memory budget in bytes + * @return the instance of the current object. + */ + T optimizeUniversalStyleCompaction( + long memtableMemoryBudget); + + /** + * Set {@link BuiltinComparator} to be used with RocksDB. + * + * Note: Comparator can be set once upon database creation. + * + * Default: BytewiseComparator. + * @param builtinComparator a {@link BuiltinComparator} type. + * @return the instance of the current object. + */ + T setComparator( + BuiltinComparator builtinComparator); + + /** + * Use the specified comparator for key ordering. + * + * Comparator should not be disposed before options instances using this comparator is + * disposed. If dispose() function is not called, then comparator object will be + * GC'd automatically. + * + * Comparator instance can be re-used in multiple options instances. + * + * @param comparator java instance. + * @return the instance of the current object. + */ + T setComparator( + AbstractComparator> comparator); + + /** + *

    Set the merge operator to be used for merging two merge operands + * of the same key. The merge function is invoked during + * compaction and at lookup time, if multiple key/value pairs belonging + * to the same key are found in the database.

    + * + * @param name the name of the merge function, as defined by + * the MergeOperators factory (see utilities/MergeOperators.h) + * The merge function is specified by name and must be one of the + * standard merge operators provided by RocksDB. The available + * operators are "put", "uint64add", "stringappend" and "stringappendtest". + * @return the instance of the current object. + */ + T setMergeOperatorName(String name); + + /** + *

    Set the merge operator to be used for merging two different key/value + * pairs that share the same key. The merge function is invoked during + * compaction and at lookup time, if multiple key/value pairs belonging + * to the same key are found in the database.

    + * + * @param mergeOperator {@link MergeOperator} instance. + * @return the instance of the current object. + */ + T setMergeOperator(MergeOperator mergeOperator); + + /** + * This prefix-extractor uses the first n bytes of a key as its prefix. + * + * In some hash-based memtable representation such as HashLinkedList + * and HashSkipList, prefixes are used to partition the keys into + * several buckets. Prefix extractor is used to specify how to + * extract the prefix given a key. + * + * @param n use the first n bytes of a key as its prefix. + * @return the reference to the current option. + */ + T useFixedLengthPrefixExtractor(int n); + + /** + * Same as fixed length prefix extractor, except that when slice is + * shorter than the fixed length, it will use the full key. + * + * @param n use the first n bytes of a key as its prefix. + * @return the reference to the current option. + */ + T useCappedPrefixExtractor(int n); + + /** + * Number of files to trigger level-0 compaction. A value < 0 means that + * level-0 compaction will not be triggered by number of files at all. + * Default: 4 + * + * @param numFiles the number of files in level-0 to trigger compaction. + * @return the reference to the current option. + */ + T setLevelZeroFileNumCompactionTrigger( + int numFiles); + + /** + * The number of files in level 0 to trigger compaction from level-0 to + * level-1. A value < 0 means that level-0 compaction will not be + * triggered by number of files at all. + * Default: 4 + * + * @return the number of files in level 0 to trigger compaction. + */ + int levelZeroFileNumCompactionTrigger(); + + /** + * Soft limit on number of level-0 files. We start slowing down writes at this + * point. A value < 0 means that no writing slow down will be triggered by + * number of files in level-0. + * + * @param numFiles soft limit on number of level-0 files. + * @return the reference to the current option. + */ + T setLevelZeroSlowdownWritesTrigger( + int numFiles); + + /** + * Soft limit on the number of level-0 files. We start slowing down writes + * at this point. A value < 0 means that no writing slow down will be + * triggered by number of files in level-0. + * + * @return the soft limit on the number of level-0 files. + */ + int levelZeroSlowdownWritesTrigger(); + + /** + * Maximum number of level-0 files. We stop writes at this point. + * + * @param numFiles the hard limit of the number of level-0 files. + * @return the reference to the current option. + */ + T setLevelZeroStopWritesTrigger(int numFiles); + + /** + * Maximum number of level-0 files. We stop writes at this point. + * + * @return the hard limit of the number of level-0 file. + */ + int levelZeroStopWritesTrigger(); + + /** + * The ratio between the total size of level-(L+1) files and the total + * size of level-L files for all L. + * DEFAULT: 10 + * + * @param multiplier the ratio between the total size of level-(L+1) + * files and the total size of level-L files for all L. + * @return the reference to the current option. + */ + T setMaxBytesForLevelMultiplier( + double multiplier); + + /** + * The ratio between the total size of level-(L+1) files and the total + * size of level-L files for all L. + * DEFAULT: 10 + * + * @return the ratio between the total size of level-(L+1) files and + * the total size of level-L files for all L. + */ + double maxBytesForLevelMultiplier(); + + /** + * FIFO compaction option. + * The oldest table file will be deleted + * once the sum of table files reaches this size. + * The default value is 1GB (1 * 1024 * 1024 * 1024). + * + * @param maxTableFilesSize the size limit of the total sum of table files. + * @return the instance of the current object. + */ + T setMaxTableFilesSizeFIFO( + long maxTableFilesSize); + + /** + * FIFO compaction option. + * The oldest table file will be deleted + * once the sum of table files reaches this size. + * The default value is 1GB (1 * 1024 * 1024 * 1024). + * + * @return the size limit of the total sum of table files. + */ + long maxTableFilesSizeFIFO(); + + /** + * Get the config for mem-table. + * + * @return the mem-table config. + */ + MemTableConfig memTableConfig(); + + /** + * Set the config for mem-table. + * + * @param memTableConfig the mem-table config. + * @return the instance of the current object. + * @throws java.lang.IllegalArgumentException thrown on 32-Bit platforms + * while overflowing the underlying platform specific value. + */ + T setMemTableConfig(MemTableConfig memTableConfig); + + /** + * Returns the name of the current mem table representation. + * Memtable format can be set using setTableFormatConfig. + * + * @return the name of the currently-used memtable factory. + * @see #setTableFormatConfig(org.rocksdb.TableFormatConfig) + */ + String memTableFactoryName(); + + /** + * Get the config for table format. + * + * @return the table format config. + */ + TableFormatConfig tableFormatConfig(); + + /** + * Set the config for table format. + * + * @param config the table format config. + * @return the reference of the current options. + */ + T setTableFormatConfig(TableFormatConfig config); + + /** + * @return the name of the currently used table factory. + */ + String tableFactoryName(); + + /** + * Compression algorithm that will be used for the bottommost level that + * contain files. If level-compaction is used, this option will only affect + * levels after base level. + * + * Default: {@link CompressionType#DISABLE_COMPRESSION_OPTION} + * + * @param bottommostCompressionType The compression type to use for the + * bottommost level + * + * @return the reference of the current options. + */ + T setBottommostCompressionType( + final CompressionType bottommostCompressionType); + + /** + * Compression algorithm that will be used for the bottommost level that + * contain files. If level-compaction is used, this option will only affect + * levels after base level. + * + * Default: {@link CompressionType#DISABLE_COMPRESSION_OPTION} + * + * @return The compression type used for the bottommost level + */ + CompressionType bottommostCompressionType(); + + + /** + * Set the different options for compression algorithms + * + * @param compressionOptions The compression options + * + * @return the reference of the current options. + */ + T setCompressionOptions( + CompressionOptions compressionOptions); + + /** + * Get the different options for compression algorithms + * + * @return The compression options + */ + CompressionOptions compressionOptions(); + + /** + * Default memtable memory budget used with the following methods: + * + *
      + *
    1. {@link #optimizeLevelStyleCompaction()}
      2. + *
    2. {@link #optimizeUniversalStyleCompaction()}
      3. + *
    + */ + long DEFAULT_COMPACTION_MEMTABLE_MEMORY_BUDGET = 512 * 1024 * 1024; +} diff --git a/java/src/main/java/org/rocksdb/CompactionOptionsFIFO.java b/java/src/main/java/org/rocksdb/CompactionOptionsFIFO.java new file mode 100644 index 00000000000..f795807804d --- /dev/null +++ b/java/src/main/java/org/rocksdb/CompactionOptionsFIFO.java @@ -0,0 +1,50 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Options for FIFO Compaction + */ +public class CompactionOptionsFIFO extends RocksObject { + + public CompactionOptionsFIFO() { + super(newCompactionOptionsFIFO()); + } + + /** + * Once the total sum of table files reaches this, we will delete the oldest + * table file + * + * Default: 1GB + * + * @param maxTableFilesSize The maximum size of the table files + * + * @return the reference to the current options. + */ + public CompactionOptionsFIFO setMaxTableFilesSize( + final long maxTableFilesSize) { + setMaxTableFilesSize(nativeHandle_, maxTableFilesSize); + return this; + } + + /** + * Once the total sum of table files reaches this, we will delete the oldest + * table file + * + * Default: 1GB + * + * @return max table file size in bytes + */ + public long maxTableFilesSize() { + return maxTableFilesSize(nativeHandle_); + } + + private native void setMaxTableFilesSize(long handle, long maxTableFilesSize); + private native long maxTableFilesSize(long handle); + + private native static long newCompactionOptionsFIFO(); + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/CompactionOptionsUniversal.java b/java/src/main/java/org/rocksdb/CompactionOptionsUniversal.java new file mode 100644 index 00000000000..d2dfa4eef1a --- /dev/null +++ b/java/src/main/java/org/rocksdb/CompactionOptionsUniversal.java @@ -0,0 +1,273 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Options for Universal Compaction + */ +public class CompactionOptionsUniversal extends RocksObject { + + public CompactionOptionsUniversal() { + super(newCompactionOptionsUniversal()); + } + + /** + * Percentage flexibility while comparing file size. If the candidate file(s) + * size is 1% smaller than the next file's size, then include next file into + * this candidate set. + * + * Default: 1 + * + * @param sizeRatio The size ratio to use + * + * @return the reference to the current options. + */ + public CompactionOptionsUniversal setSizeRatio(final int sizeRatio) { + setSizeRatio(nativeHandle_, sizeRatio); + return this; + } + + /** + * Percentage flexibility while comparing file size. If the candidate file(s) + * size is 1% smaller than the next file's size, then include next file into + * this candidate set. + * + * Default: 1 + * + * @return The size ratio in use + */ + public int sizeRatio() { + return sizeRatio(nativeHandle_); + } + + /** + * The minimum number of files in a single compaction run. + * + * Default: 2 + * + * @param minMergeWidth minimum number of files in a single compaction run + * + * @return the reference to the current options. + */ + public CompactionOptionsUniversal setMinMergeWidth(final int minMergeWidth) { + setMinMergeWidth(nativeHandle_, minMergeWidth); + return this; + } + + /** + * The minimum number of files in a single compaction run. + * + * Default: 2 + * + * @return minimum number of files in a single compaction run + */ + public int minMergeWidth() { + return minMergeWidth(nativeHandle_); + } + + /** + * The maximum number of files in a single compaction run. + * + * Default: {@link Long#MAX_VALUE} + * + * @param maxMergeWidth maximum number of files in a single compaction run + * + * @return the reference to the current options. + */ + public CompactionOptionsUniversal setMaxMergeWidth(final int maxMergeWidth) { + setMaxMergeWidth(nativeHandle_, maxMergeWidth); + return this; + } + + /** + * The maximum number of files in a single compaction run. + * + * Default: {@link Long#MAX_VALUE} + * + * @return maximum number of files in a single compaction run + */ + public int maxMergeWidth() { + return maxMergeWidth(nativeHandle_); + } + + /** + * The size amplification is defined as the amount (in percentage) of + * additional storage needed to store a single byte of data in the database. + * For example, a size amplification of 2% means that a database that + * contains 100 bytes of user-data may occupy upto 102 bytes of + * physical storage. By this definition, a fully compacted database has + * a size amplification of 0%. Rocksdb uses the following heuristic + * to calculate size amplification: it assumes that all files excluding + * the earliest file contribute to the size amplification. + * + * Default: 200, which means that a 100 byte database could require upto + * 300 bytes of storage. + * + * @param maxSizeAmplificationPercent the amount of additional storage needed + * (as a percentage) to store a single byte in the database + * + * @return the reference to the current options. + */ + public CompactionOptionsUniversal setMaxSizeAmplificationPercent( + final int maxSizeAmplificationPercent) { + setMaxSizeAmplificationPercent(nativeHandle_, maxSizeAmplificationPercent); + return this; + } + + /** + * The size amplification is defined as the amount (in percentage) of + * additional storage needed to store a single byte of data in the database. + * For example, a size amplification of 2% means that a database that + * contains 100 bytes of user-data may occupy upto 102 bytes of + * physical storage. By this definition, a fully compacted database has + * a size amplification of 0%. Rocksdb uses the following heuristic + * to calculate size amplification: it assumes that all files excluding + * the earliest file contribute to the size amplification. + * + * Default: 200, which means that a 100 byte database could require upto + * 300 bytes of storage. + * + * @return the amount of additional storage needed (as a percentage) to store + * a single byte in the database + */ + public int maxSizeAmplificationPercent() { + return maxSizeAmplificationPercent(nativeHandle_); + } + + /** + * If this option is set to be -1 (the default value), all the output files + * will follow compression type specified. + * + * If this option is not negative, we will try to make sure compressed + * size is just above this value. In normal cases, at least this percentage + * of data will be compressed. + * + * When we are compacting to a new file, here is the criteria whether + * it needs to be compressed: assuming here are the list of files sorted + * by generation time: + * A1...An B1...Bm C1...Ct + * where A1 is the newest and Ct is the oldest, and we are going to compact + * B1...Bm, we calculate the total size of all the files as total_size, as + * well as the total size of C1...Ct as total_C, the compaction output file + * will be compressed iff + * total_C / total_size < this percentage + * + * Default: -1 + * + * @param compressionSizePercent percentage of size for compression + * + * @return the reference to the current options. + */ + public CompactionOptionsUniversal setCompressionSizePercent( + final int compressionSizePercent) { + setCompressionSizePercent(nativeHandle_, compressionSizePercent); + return this; + } + + /** + * If this option is set to be -1 (the default value), all the output files + * will follow compression type specified. + * + * If this option is not negative, we will try to make sure compressed + * size is just above this value. In normal cases, at least this percentage + * of data will be compressed. + * + * When we are compacting to a new file, here is the criteria whether + * it needs to be compressed: assuming here are the list of files sorted + * by generation time: + * A1...An B1...Bm C1...Ct + * where A1 is the newest and Ct is the oldest, and we are going to compact + * B1...Bm, we calculate the total size of all the files as total_size, as + * well as the total size of C1...Ct as total_C, the compaction output file + * will be compressed iff + * total_C / total_size < this percentage + * + * Default: -1 + * + * @return percentage of size for compression + */ + public int compressionSizePercent() { + return compressionSizePercent(nativeHandle_); + } + + /** + * The algorithm used to stop picking files into a single compaction run + * + * Default: {@link CompactionStopStyle#CompactionStopStyleTotalSize} + * + * @param compactionStopStyle The compaction algorithm + * + * @return the reference to the current options. + */ + public CompactionOptionsUniversal setStopStyle( + final CompactionStopStyle compactionStopStyle) { + setStopStyle(nativeHandle_, compactionStopStyle.getValue()); + return this; + } + + /** + * The algorithm used to stop picking files into a single compaction run + * + * Default: {@link CompactionStopStyle#CompactionStopStyleTotalSize} + * + * @return The compaction algorithm + */ + public CompactionStopStyle stopStyle() { + return CompactionStopStyle.getCompactionStopStyle(stopStyle(nativeHandle_)); + } + + /** + * Option to optimize the universal multi level compaction by enabling + * trivial move for non overlapping files. + * + * Default: false + * + * @param allowTrivialMove true if trivial move is allowed + * + * @return the reference to the current options. + */ + public CompactionOptionsUniversal setAllowTrivialMove( + final boolean allowTrivialMove) { + setAllowTrivialMove(nativeHandle_, allowTrivialMove); + return this; + } + + /** + * Option to optimize the universal multi level compaction by enabling + * trivial move for non overlapping files. + * + * Default: false + * + * @return true if trivial move is allowed + */ + public boolean allowTrivialMove() { + return allowTrivialMove(nativeHandle_); + } + + private native static long newCompactionOptionsUniversal(); + @Override protected final native void disposeInternal(final long handle); + + private native void setSizeRatio(final long handle, final int sizeRatio); + private native int sizeRatio(final long handle); + private native void setMinMergeWidth( + final long handle, final int minMergeWidth); + private native int minMergeWidth(final long handle); + private native void setMaxMergeWidth( + final long handle, final int maxMergeWidth); + private native int maxMergeWidth(final long handle); + private native void setMaxSizeAmplificationPercent( + final long handle, final int maxSizeAmplificationPercent); + private native int maxSizeAmplificationPercent(final long handle); + private native void setCompressionSizePercent( + final long handle, final int compressionSizePercent); + private native int compressionSizePercent(final long handle); + private native void setStopStyle( + final long handle, final byte stopStyle); + private native byte stopStyle(final long handle); + private native void setAllowTrivialMove( + final long handle, final boolean allowTrivialMove); + private native boolean allowTrivialMove(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/CompactionPriority.java b/java/src/main/java/org/rocksdb/CompactionPriority.java new file mode 100644 index 00000000000..a4f53cd64c8 --- /dev/null +++ b/java/src/main/java/org/rocksdb/CompactionPriority.java @@ -0,0 +1,73 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Compaction Priorities + */ +public enum CompactionPriority { + + /** + * Slightly Prioritize larger files by size compensated by #deletes + */ + ByCompensatedSize((byte)0x0), + + /** + * First compact files whose data's latest update time is oldest. + * Try this if you only update some hot keys in small ranges. + */ + OldestLargestSeqFirst((byte)0x1), + + /** + * First compact files whose range hasn't been compacted to the next level + * for the longest. If your updates are random across the key space, + * write amplification is slightly better with this option. + */ + OldestSmallestSeqFirst((byte)0x2), + + /** + * First compact files whose ratio between overlapping size in next level + * and its size is the smallest. It in many cases can optimize write + * amplification. + */ + MinOverlappingRatio((byte)0x3); + + + private final byte value; + + CompactionPriority(final byte value) { + this.value = value; + } + + /** + * Returns the byte value of the enumerations value + * + * @return byte representation + */ + public byte getValue() { + return value; + } + + /** + * Get CompactionPriority by byte value. + * + * @param value byte representation of CompactionPriority. + * + * @return {@link org.rocksdb.CompactionPriority} instance or null. + * @throws java.lang.IllegalArgumentException if an invalid + * value is provided. + */ + public static CompactionPriority getCompactionPriority(final byte value) { + for (final CompactionPriority compactionPriority : + CompactionPriority.values()) { + if (compactionPriority.getValue() == value){ + return compactionPriority; + } + } + throw new IllegalArgumentException( + "Illegal value provided for CompactionPriority."); + } +} diff --git a/java/src/main/java/org/rocksdb/CompactionStopStyle.java b/java/src/main/java/org/rocksdb/CompactionStopStyle.java new file mode 100644 index 00000000000..13cc873c360 --- /dev/null +++ b/java/src/main/java/org/rocksdb/CompactionStopStyle.java @@ -0,0 +1,54 @@ +package org.rocksdb; + +/** + * Algorithm used to make a compaction request stop picking new files + * into a single compaction run + */ +public enum CompactionStopStyle { + + /** + * Pick files of similar size + */ + CompactionStopStyleSimilarSize((byte)0x0), + + /** + * Total size of picked files > next file + */ + CompactionStopStyleTotalSize((byte)0x1); + + + private final byte value; + + CompactionStopStyle(final byte value) { + this.value = value; + } + + /** + * Returns the byte value of the enumerations value + * + * @return byte representation + */ + public byte getValue() { + return value; + } + + /** + * Get CompactionStopStyle by byte value. + * + * @param value byte representation of CompactionStopStyle. + * + * @return {@link org.rocksdb.CompactionStopStyle} instance or null. + * @throws java.lang.IllegalArgumentException if an invalid + * value is provided. + */ + public static CompactionStopStyle getCompactionStopStyle(final byte value) { + for (final CompactionStopStyle compactionStopStyle : + CompactionStopStyle.values()) { + if (compactionStopStyle.getValue() == value){ + return compactionStopStyle; + } + } + throw new IllegalArgumentException( + "Illegal value provided for CompactionStopStyle."); + } +} diff --git a/java/src/main/java/org/rocksdb/CompactionStyle.java b/java/src/main/java/org/rocksdb/CompactionStyle.java new file mode 100644 index 00000000000..5e13363c44c --- /dev/null +++ b/java/src/main/java/org/rocksdb/CompactionStyle.java @@ -0,0 +1,52 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Enum CompactionStyle + * + * RocksDB supports different styles of compaction. Available + * compaction styles can be chosen using this enumeration. + * + *
      + *
    1. LEVEL - Level based Compaction style
      2. + *
    2. UNIVERSAL - Universal Compaction Style is a + * compaction style, targeting the use cases requiring lower write + * amplification, trading off read amplification and space + * amplification.
      3. + *
    3. FIFO - FIFO compaction style is the simplest + * compaction strategy. It is suited for keeping event log data with + * very low overhead (query log for example). It periodically deletes + * the old data, so it's basically a TTL compaction style.
      4. + *
    + * + * @see + * Universal Compaction + * @see + * FIFO Compaction + */ +public enum CompactionStyle { + LEVEL((byte) 0), + UNIVERSAL((byte) 1), + FIFO((byte) 2); + + private final byte value_; + + private CompactionStyle(byte value) { + value_ = value; + } + + /** + * Returns the byte value of the enumerations value + * + * @return byte representation + */ + public byte getValue() { + return value_; + } +} diff --git a/java/src/main/java/org/rocksdb/Comparator.java b/java/src/main/java/org/rocksdb/Comparator.java new file mode 100644 index 00000000000..817e00fd274 --- /dev/null +++ b/java/src/main/java/org/rocksdb/Comparator.java @@ -0,0 +1,32 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Base class for comparators which will receive + * byte[] based access via org.rocksdb.Slice in their + * compare method implementation. + * + * byte[] based slices perform better when small keys + * are involved. When using larger keys consider + * using @see org.rocksdb.DirectComparator + */ +public abstract class Comparator extends AbstractComparator { + + private final long nativeHandle_; + + public Comparator(final ComparatorOptions copt) { + super(); + this.nativeHandle_ = createNewComparator0(copt.nativeHandle_); + } + + @Override + protected final long getNativeHandle() { + return nativeHandle_; + } + + private native long createNewComparator0(final long comparatorOptionsHandle); +} diff --git a/java/src/main/java/org/rocksdb/ComparatorOptions.java b/java/src/main/java/org/rocksdb/ComparatorOptions.java new file mode 100644 index 00000000000..3a05befa448 --- /dev/null +++ b/java/src/main/java/org/rocksdb/ComparatorOptions.java @@ -0,0 +1,51 @@ +package org.rocksdb; + +/** + * This class controls the behaviour + * of Java implementations of + * AbstractComparator + * + * Note that dispose() must be called before a ComparatorOptions + * instance becomes out-of-scope to release the allocated memory in C++. + */ +public class ComparatorOptions extends RocksObject { + public ComparatorOptions() { + super(newComparatorOptions()); + } + + /** + * Use adaptive mutex, which spins in the user space before resorting + * to kernel. This could reduce context switch when the mutex is not + * heavily contended. However, if the mutex is hot, we could end up + * wasting spin time. + * Default: false + * + * @return true if adaptive mutex is used. + */ + public boolean useAdaptiveMutex() { + assert(isOwningHandle()); + return useAdaptiveMutex(nativeHandle_); + } + + /** + * Use adaptive mutex, which spins in the user space before resorting + * to kernel. This could reduce context switch when the mutex is not + * heavily contended. However, if the mutex is hot, we could end up + * wasting spin time. + * Default: false + * + * @param useAdaptiveMutex true if adaptive mutex is used. + * @return the reference to the current comparator options. + */ + public ComparatorOptions setUseAdaptiveMutex(final boolean useAdaptiveMutex) { + assert (isOwningHandle()); + setUseAdaptiveMutex(nativeHandle_, useAdaptiveMutex); + return this; + } + + private native static long newComparatorOptions(); + private native boolean useAdaptiveMutex(final long handle); + private native void setUseAdaptiveMutex(final long handle, + final boolean useAdaptiveMutex); + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/CompressionOptions.java b/java/src/main/java/org/rocksdb/CompressionOptions.java new file mode 100644 index 00000000000..4927770e523 --- /dev/null +++ b/java/src/main/java/org/rocksdb/CompressionOptions.java @@ -0,0 +1,85 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Options for Compression + */ +public class CompressionOptions extends RocksObject { + + public CompressionOptions() { + super(newCompressionOptions()); + } + + public CompressionOptions setWindowBits(final int windowBits) { + setWindowBits(nativeHandle_, windowBits); + return this; + } + + public int windowBits() { + return windowBits(nativeHandle_); + } + + public CompressionOptions setLevel(final int level) { + setLevel(nativeHandle_, level); + return this; + } + + public int level() { + return level(nativeHandle_); + } + + public CompressionOptions setStrategy(final int strategy) { + setStrategy(nativeHandle_, strategy); + return this; + } + + public int strategy() { + return strategy(nativeHandle_); + } + + /** + * Maximum size of dictionary used to prime the compression library. Currently + * this dictionary will be constructed by sampling the first output file in a + * subcompaction when the target level is bottommost. This dictionary will be + * loaded into the compression library before compressing/uncompressing each + * data block of subsequent files in the subcompaction. Effectively, this + * improves compression ratios when there are repetitions across data blocks. + * + * A value of 0 indicates the feature is disabled. + * + * Default: 0. + * + * @param maxDictBytes Maximum bytes to use for the dictionary + * + * @return the reference to the current options + */ + public CompressionOptions setMaxDictBytes(final int maxDictBytes) { + setMaxDictBytes(nativeHandle_, maxDictBytes); + return this; + } + + /** + * Maximum size of dictionary used to prime the compression library. + * + * @return The maximum bytes to use for the dictionary + */ + public int maxDictBytes() { + return maxDictBytes(nativeHandle_); + } + + private native static long newCompressionOptions(); + @Override protected final native void disposeInternal(final long handle); + + private native void setWindowBits(final long handle, final int windowBits); + private native int windowBits(final long handle); + private native void setLevel(final long handle, final int level); + private native int level(final long handle); + private native void setStrategy(final long handle, final int strategy); + private native int strategy(final long handle); + private native void setMaxDictBytes(final long handle, final int maxDictBytes); + private native int maxDictBytes(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/CompressionType.java b/java/src/main/java/org/rocksdb/CompressionType.java new file mode 100644 index 00000000000..2781537c88f --- /dev/null +++ b/java/src/main/java/org/rocksdb/CompressionType.java @@ -0,0 +1,99 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Enum CompressionType + * + *

    DB contents are stored in a set of blocks, each of which holds a + * sequence of key,value pairs. Each block may be compressed before + * being stored in a file. The following enum describes which + * compression method (if any) is used to compress a block.

    + */ +public enum CompressionType { + + NO_COMPRESSION((byte) 0x0, null), + SNAPPY_COMPRESSION((byte) 0x1, "snappy"), + ZLIB_COMPRESSION((byte) 0x2, "z"), + BZLIB2_COMPRESSION((byte) 0x3, "bzip2"), + LZ4_COMPRESSION((byte) 0x4, "lz4"), + LZ4HC_COMPRESSION((byte) 0x5, "lz4hc"), + XPRESS_COMPRESSION((byte) 0x6, "xpress"), + ZSTD_COMPRESSION((byte)0x7, "zstd"), + DISABLE_COMPRESSION_OPTION((byte)0x7F, null); + + /** + *

    Get the CompressionType enumeration value by + * passing the library name to this method.

    + * + *

    If library cannot be found the enumeration + * value {@code NO_COMPRESSION} will be returned.

    + * + * @param libraryName compression library name. + * + * @return CompressionType instance. + */ + public static CompressionType getCompressionType(String libraryName) { + if (libraryName != null) { + for (CompressionType compressionType : CompressionType.values()) { + if (compressionType.getLibraryName() != null && + compressionType.getLibraryName().equals(libraryName)) { + return compressionType; + } + } + } + return CompressionType.NO_COMPRESSION; + } + + /** + *

    Get the CompressionType enumeration value by + * passing the byte identifier to this method.

    + * + * @param byteIdentifier of CompressionType. + * + * @return CompressionType instance. + * + * @throws IllegalArgumentException If CompressionType cannot be found for the + * provided byteIdentifier + */ + public static CompressionType getCompressionType(byte byteIdentifier) { + for (final CompressionType compressionType : CompressionType.values()) { + if (compressionType.getValue() == byteIdentifier) { + return compressionType; + } + } + + throw new IllegalArgumentException( + "Illegal value provided for CompressionType."); + } + + /** + *

    Returns the byte value of the enumerations value.

    + * + * @return byte representation + */ + public byte getValue() { + return value_; + } + + /** + *

    Returns the library name of the compression type + * identified by the enumeration value.

    + * + * @return library name + */ + public String getLibraryName() { + return libraryName_; + } + + CompressionType(final byte value, final String libraryName) { + value_ = value; + libraryName_ = libraryName; + } + + private final byte value_; + private final String libraryName_; +} diff --git a/java/src/main/java/org/rocksdb/DBOptions.java b/java/src/main/java/org/rocksdb/DBOptions.java new file mode 100644 index 00000000000..14f0c6c7c9a --- /dev/null +++ b/java/src/main/java/org/rocksdb/DBOptions.java @@ -0,0 +1,1120 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.nio.file.Paths; +import java.util.*; + +/** + * DBOptions to control the behavior of a database. It will be used + * during the creation of a {@link org.rocksdb.RocksDB} (i.e., RocksDB.open()). + * + * If {@link #dispose()} function is not called, then it will be GC'd + * automatically and native resources will be released as part of the process. + */ +public class DBOptions + extends RocksObject implements DBOptionsInterface { + static { + RocksDB.loadLibrary(); + } + + /** + * Construct DBOptions. + * + * This constructor will create (by allocating a block of memory) + * an {@code rocksdb::DBOptions} in the c++ side. + */ + public DBOptions() { + super(newDBOptions()); + numShardBits_ = DEFAULT_NUM_SHARD_BITS; + } + + /** + *

    Method to get a options instance by using pre-configured + * property values. If one or many values are undefined in + * the context of RocksDB the method will return a null + * value.

    + * + *

    Note: Property keys can be derived from + * getter methods within the options class. Example: the method + * {@code allowMmapReads()} has a property key: + * {@code allow_mmap_reads}.

    + * + * @param properties {@link java.util.Properties} instance. + * + * @return {@link org.rocksdb.DBOptions instance} + * or null. + * + * @throws java.lang.IllegalArgumentException if null or empty + * {@link java.util.Properties} instance is passed to the method call. + */ + public static DBOptions getDBOptionsFromProps( + final Properties properties) { + if (properties == null || properties.size() == 0) { + throw new IllegalArgumentException( + "Properties value must contain at least one value."); + } + DBOptions dbOptions = null; + StringBuilder stringBuilder = new StringBuilder(); + for (final String name : properties.stringPropertyNames()){ + stringBuilder.append(name); + stringBuilder.append("="); + stringBuilder.append(properties.getProperty(name)); + stringBuilder.append(";"); + } + long handle = getDBOptionsFromProps( + stringBuilder.toString()); + if (handle != 0){ + dbOptions = new DBOptions(handle); + } + return dbOptions; + } + + @Override + public DBOptions optimizeForSmallDb() { + optimizeForSmallDb(nativeHandle_); + return this; + } + + @Override + public DBOptions setIncreaseParallelism( + final int totalThreads) { + assert(isOwningHandle()); + setIncreaseParallelism(nativeHandle_, totalThreads); + return this; + } + + @Override + public DBOptions setCreateIfMissing(final boolean flag) { + assert(isOwningHandle()); + setCreateIfMissing(nativeHandle_, flag); + return this; + } + + @Override + public boolean createIfMissing() { + assert(isOwningHandle()); + return createIfMissing(nativeHandle_); + } + + @Override + public DBOptions setCreateMissingColumnFamilies( + final boolean flag) { + assert(isOwningHandle()); + setCreateMissingColumnFamilies(nativeHandle_, flag); + return this; + } + + @Override + public boolean createMissingColumnFamilies() { + assert(isOwningHandle()); + return createMissingColumnFamilies(nativeHandle_); + } + + @Override + public DBOptions setEnv(final Env env) { + setEnv(nativeHandle_, env.nativeHandle_); + this.env_ = env; + return this; + } + + @Override + public Env getEnv() { + return env_; + } + + @Override + public DBOptions setErrorIfExists( + final boolean errorIfExists) { + assert(isOwningHandle()); + setErrorIfExists(nativeHandle_, errorIfExists); + return this; + } + + @Override + public boolean errorIfExists() { + assert(isOwningHandle()); + return errorIfExists(nativeHandle_); + } + + @Override + public DBOptions setParanoidChecks( + final boolean paranoidChecks) { + assert(isOwningHandle()); + setParanoidChecks(nativeHandle_, paranoidChecks); + return this; + } + + @Override + public boolean paranoidChecks() { + assert(isOwningHandle()); + return paranoidChecks(nativeHandle_); + } + + @Override + public DBOptions setRateLimiter(final RateLimiter rateLimiter) { + assert(isOwningHandle()); + rateLimiter_ = rateLimiter; + setRateLimiter(nativeHandle_, rateLimiter.nativeHandle_); + return this; + } + + @Override + public DBOptions setLogger(final Logger logger) { + assert(isOwningHandle()); + setLogger(nativeHandle_, logger.nativeHandle_); + return this; + } + + @Override + public DBOptions setInfoLogLevel( + final InfoLogLevel infoLogLevel) { + assert(isOwningHandle()); + setInfoLogLevel(nativeHandle_, infoLogLevel.getValue()); + return this; + } + + @Override + public InfoLogLevel infoLogLevel() { + assert(isOwningHandle()); + return InfoLogLevel.getInfoLogLevel( + infoLogLevel(nativeHandle_)); + } + + @Override + public DBOptions setMaxOpenFiles( + final int maxOpenFiles) { + assert(isOwningHandle()); + setMaxOpenFiles(nativeHandle_, maxOpenFiles); + return this; + } + + @Override + public int maxOpenFiles() { + assert(isOwningHandle()); + return maxOpenFiles(nativeHandle_); + } + + @Override + public DBOptions setMaxFileOpeningThreads(final int maxFileOpeningThreads) { + assert(isOwningHandle()); + setMaxFileOpeningThreads(nativeHandle_, maxFileOpeningThreads); + return this; + } + + @Override + public int maxFileOpeningThreads() { + assert(isOwningHandle()); + return maxFileOpeningThreads(nativeHandle_); + } + + @Override + public DBOptions setMaxTotalWalSize( + final long maxTotalWalSize) { + assert(isOwningHandle()); + setMaxTotalWalSize(nativeHandle_, maxTotalWalSize); + return this; + } + + @Override + public long maxTotalWalSize() { + assert(isOwningHandle()); + return maxTotalWalSize(nativeHandle_); + } + + @Override + public DBOptions setStatistics(final Statistics statistics) { + assert(isOwningHandle()); + setStatistics(nativeHandle_, statistics.nativeHandle_); + return this; + } + + @Override + public Statistics statistics() { + assert(isOwningHandle()); + final long statisticsNativeHandle = statistics(nativeHandle_); + if(statisticsNativeHandle == 0) { + return null; + } else { + return new Statistics(statisticsNativeHandle); + } + } + + @Override + public DBOptions setUseFsync( + final boolean useFsync) { + assert(isOwningHandle()); + setUseFsync(nativeHandle_, useFsync); + return this; + } + + @Override + public boolean useFsync() { + assert(isOwningHandle()); + return useFsync(nativeHandle_); + } + + @Override + public DBOptions setDbPaths(final Collection dbPaths) { + assert(isOwningHandle()); + + final int len = dbPaths.size(); + final String paths[] = new String[len]; + final long targetSizes[] = new long[len]; + + int i = 0; + for(final DbPath dbPath : dbPaths) { + paths[i] = dbPath.path.toString(); + targetSizes[i] = dbPath.targetSize; + i++; + } + setDbPaths(nativeHandle_, paths, targetSizes); + return this; + } + + @Override + public List dbPaths() { + final int len = (int)dbPathsLen(nativeHandle_); + if(len == 0) { + return Collections.emptyList(); + } else { + final String paths[] = new String[len]; + final long targetSizes[] = new long[len]; + + dbPaths(nativeHandle_, paths, targetSizes); + + final List dbPaths = new ArrayList<>(); + for(int i = 0; i < len; i++) { + dbPaths.add(new DbPath(Paths.get(paths[i]), targetSizes[i])); + } + return dbPaths; + } + } + + @Override + public DBOptions setDbLogDir( + final String dbLogDir) { + assert(isOwningHandle()); + setDbLogDir(nativeHandle_, dbLogDir); + return this; + } + + @Override + public String dbLogDir() { + assert(isOwningHandle()); + return dbLogDir(nativeHandle_); + } + + @Override + public DBOptions setWalDir( + final String walDir) { + assert(isOwningHandle()); + setWalDir(nativeHandle_, walDir); + return this; + } + + @Override + public String walDir() { + assert(isOwningHandle()); + return walDir(nativeHandle_); + } + + @Override + public DBOptions setDeleteObsoleteFilesPeriodMicros( + final long micros) { + assert(isOwningHandle()); + setDeleteObsoleteFilesPeriodMicros(nativeHandle_, micros); + return this; + } + + @Override + public long deleteObsoleteFilesPeriodMicros() { + assert(isOwningHandle()); + return deleteObsoleteFilesPeriodMicros(nativeHandle_); + } + + @Override + public void setBaseBackgroundCompactions( + final int baseBackgroundCompactions) { + assert(isOwningHandle()); + setBaseBackgroundCompactions(nativeHandle_, baseBackgroundCompactions); + } + + @Override + public int baseBackgroundCompactions() { + assert(isOwningHandle()); + return baseBackgroundCompactions(nativeHandle_); + } + + @Override + public DBOptions setMaxBackgroundCompactions( + final int maxBackgroundCompactions) { + assert(isOwningHandle()); + setMaxBackgroundCompactions(nativeHandle_, maxBackgroundCompactions); + return this; + } + + @Override + public int maxBackgroundCompactions() { + assert(isOwningHandle()); + return maxBackgroundCompactions(nativeHandle_); + } + + @Override + public void setMaxSubcompactions(final int maxSubcompactions) { + assert(isOwningHandle()); + setMaxSubcompactions(nativeHandle_, maxSubcompactions); + } + + @Override + public int maxSubcompactions() { + assert(isOwningHandle()); + return maxSubcompactions(nativeHandle_); + } + + @Override + public DBOptions setMaxBackgroundFlushes( + final int maxBackgroundFlushes) { + assert(isOwningHandle()); + setMaxBackgroundFlushes(nativeHandle_, maxBackgroundFlushes); + return this; + } + + @Override + public int maxBackgroundFlushes() { + assert(isOwningHandle()); + return maxBackgroundFlushes(nativeHandle_); + } + + @Override + public DBOptions setMaxLogFileSize( + final long maxLogFileSize) { + assert(isOwningHandle()); + setMaxLogFileSize(nativeHandle_, maxLogFileSize); + return this; + } + + @Override + public long maxLogFileSize() { + assert(isOwningHandle()); + return maxLogFileSize(nativeHandle_); + } + + @Override + public DBOptions setLogFileTimeToRoll( + final long logFileTimeToRoll) { + assert(isOwningHandle()); + setLogFileTimeToRoll(nativeHandle_, logFileTimeToRoll); + return this; + } + + @Override + public long logFileTimeToRoll() { + assert(isOwningHandle()); + return logFileTimeToRoll(nativeHandle_); + } + + @Override + public DBOptions setKeepLogFileNum( + final long keepLogFileNum) { + assert(isOwningHandle()); + setKeepLogFileNum(nativeHandle_, keepLogFileNum); + return this; + } + + @Override + public long keepLogFileNum() { + assert(isOwningHandle()); + return keepLogFileNum(nativeHandle_); + } + + @Override + public DBOptions setRecycleLogFileNum(final long recycleLogFileNum) { + assert(isOwningHandle()); + setRecycleLogFileNum(nativeHandle_, recycleLogFileNum); + return this; + } + + @Override + public long recycleLogFileNum() { + assert(isOwningHandle()); + return recycleLogFileNum(nativeHandle_); + } + + @Override + public DBOptions setMaxManifestFileSize( + final long maxManifestFileSize) { + assert(isOwningHandle()); + setMaxManifestFileSize(nativeHandle_, maxManifestFileSize); + return this; + } + + @Override + public long maxManifestFileSize() { + assert(isOwningHandle()); + return maxManifestFileSize(nativeHandle_); + } + + @Override + public DBOptions setTableCacheNumshardbits( + final int tableCacheNumshardbits) { + assert(isOwningHandle()); + setTableCacheNumshardbits(nativeHandle_, tableCacheNumshardbits); + return this; + } + + @Override + public int tableCacheNumshardbits() { + assert(isOwningHandle()); + return tableCacheNumshardbits(nativeHandle_); + } + + @Override + public DBOptions setWalTtlSeconds( + final long walTtlSeconds) { + assert(isOwningHandle()); + setWalTtlSeconds(nativeHandle_, walTtlSeconds); + return this; + } + + @Override + public long walTtlSeconds() { + assert(isOwningHandle()); + return walTtlSeconds(nativeHandle_); + } + + @Override + public DBOptions setWalSizeLimitMB( + final long sizeLimitMB) { + assert(isOwningHandle()); + setWalSizeLimitMB(nativeHandle_, sizeLimitMB); + return this; + } + + @Override + public long walSizeLimitMB() { + assert(isOwningHandle()); + return walSizeLimitMB(nativeHandle_); + } + + @Override + public DBOptions setManifestPreallocationSize( + final long size) { + assert(isOwningHandle()); + setManifestPreallocationSize(nativeHandle_, size); + return this; + } + + @Override + public long manifestPreallocationSize() { + assert(isOwningHandle()); + return manifestPreallocationSize(nativeHandle_); + } + + @Override + public DBOptions setUseDirectReads( + final boolean useDirectReads) { + assert(isOwningHandle()); + setUseDirectReads(nativeHandle_, useDirectReads); + return this; + } + + @Override + public boolean useDirectReads() { + assert(isOwningHandle()); + return useDirectReads(nativeHandle_); + } + + @Override + public DBOptions setUseDirectIoForFlushAndCompaction( + final boolean useDirectIoForFlushAndCompaction) { + assert(isOwningHandle()); + setUseDirectIoForFlushAndCompaction(nativeHandle_, + useDirectIoForFlushAndCompaction); + return this; + } + + @Override + public boolean useDirectIoForFlushAndCompaction() { + assert(isOwningHandle()); + return useDirectIoForFlushAndCompaction(nativeHandle_); + } + + @Override + public DBOptions setAllowFAllocate(final boolean allowFAllocate) { + assert(isOwningHandle()); + setAllowFAllocate(nativeHandle_, allowFAllocate); + return this; + } + + @Override + public boolean allowFAllocate() { + assert(isOwningHandle()); + return allowFAllocate(nativeHandle_); + } + + @Override + public DBOptions setAllowMmapReads( + final boolean allowMmapReads) { + assert(isOwningHandle()); + setAllowMmapReads(nativeHandle_, allowMmapReads); + return this; + } + + @Override + public boolean allowMmapReads() { + assert(isOwningHandle()); + return allowMmapReads(nativeHandle_); + } + + @Override + public DBOptions setAllowMmapWrites( + final boolean allowMmapWrites) { + assert(isOwningHandle()); + setAllowMmapWrites(nativeHandle_, allowMmapWrites); + return this; + } + + @Override + public boolean allowMmapWrites() { + assert(isOwningHandle()); + return allowMmapWrites(nativeHandle_); + } + + @Override + public DBOptions setIsFdCloseOnExec( + final boolean isFdCloseOnExec) { + assert(isOwningHandle()); + setIsFdCloseOnExec(nativeHandle_, isFdCloseOnExec); + return this; + } + + @Override + public boolean isFdCloseOnExec() { + assert(isOwningHandle()); + return isFdCloseOnExec(nativeHandle_); + } + + @Override + public DBOptions setStatsDumpPeriodSec( + final int statsDumpPeriodSec) { + assert(isOwningHandle()); + setStatsDumpPeriodSec(nativeHandle_, statsDumpPeriodSec); + return this; + } + + @Override + public int statsDumpPeriodSec() { + assert(isOwningHandle()); + return statsDumpPeriodSec(nativeHandle_); + } + + @Override + public DBOptions setAdviseRandomOnOpen( + final boolean adviseRandomOnOpen) { + assert(isOwningHandle()); + setAdviseRandomOnOpen(nativeHandle_, adviseRandomOnOpen); + return this; + } + + @Override + public boolean adviseRandomOnOpen() { + return adviseRandomOnOpen(nativeHandle_); + } + + @Override + public DBOptions setDbWriteBufferSize(final long dbWriteBufferSize) { + assert(isOwningHandle()); + setDbWriteBufferSize(nativeHandle_, dbWriteBufferSize); + return this; + } + + @Override + public long dbWriteBufferSize() { + assert(isOwningHandle()); + return dbWriteBufferSize(nativeHandle_); + } + + @Override + public DBOptions setAccessHintOnCompactionStart(final AccessHint accessHint) { + assert(isOwningHandle()); + setAccessHintOnCompactionStart(nativeHandle_, accessHint.getValue()); + return this; + } + + @Override + public AccessHint accessHintOnCompactionStart() { + assert(isOwningHandle()); + return AccessHint.getAccessHint(accessHintOnCompactionStart(nativeHandle_)); + } + + @Override + public DBOptions setNewTableReaderForCompactionInputs( + final boolean newTableReaderForCompactionInputs) { + assert(isOwningHandle()); + setNewTableReaderForCompactionInputs(nativeHandle_, + newTableReaderForCompactionInputs); + return this; + } + + @Override + public boolean newTableReaderForCompactionInputs() { + assert(isOwningHandle()); + return newTableReaderForCompactionInputs(nativeHandle_); + } + + @Override + public DBOptions setCompactionReadaheadSize(final long compactionReadaheadSize) { + assert(isOwningHandle()); + setCompactionReadaheadSize(nativeHandle_, compactionReadaheadSize); + return this; + } + + @Override + public long compactionReadaheadSize() { + assert(isOwningHandle()); + return compactionReadaheadSize(nativeHandle_); + } + + @Override + public DBOptions setRandomAccessMaxBufferSize(final long randomAccessMaxBufferSize) { + assert(isOwningHandle()); + setRandomAccessMaxBufferSize(nativeHandle_, randomAccessMaxBufferSize); + return this; + } + + @Override + public long randomAccessMaxBufferSize() { + assert(isOwningHandle()); + return randomAccessMaxBufferSize(nativeHandle_); + } + + @Override + public DBOptions setWritableFileMaxBufferSize(final long writableFileMaxBufferSize) { + assert(isOwningHandle()); + setWritableFileMaxBufferSize(nativeHandle_, writableFileMaxBufferSize); + return this; + } + + @Override + public long writableFileMaxBufferSize() { + assert(isOwningHandle()); + return writableFileMaxBufferSize(nativeHandle_); + } + + @Override + public DBOptions setUseAdaptiveMutex( + final boolean useAdaptiveMutex) { + assert(isOwningHandle()); + setUseAdaptiveMutex(nativeHandle_, useAdaptiveMutex); + return this; + } + + @Override + public boolean useAdaptiveMutex() { + assert(isOwningHandle()); + return useAdaptiveMutex(nativeHandle_); + } + + @Override + public DBOptions setBytesPerSync( + final long bytesPerSync) { + assert(isOwningHandle()); + setBytesPerSync(nativeHandle_, bytesPerSync); + return this; + } + + @Override + public long bytesPerSync() { + return bytesPerSync(nativeHandle_); + } + + @Override + public DBOptions setWalBytesPerSync(final long walBytesPerSync) { + assert(isOwningHandle()); + setWalBytesPerSync(nativeHandle_, walBytesPerSync); + return this; + } + + @Override + public long walBytesPerSync() { + assert(isOwningHandle()); + return walBytesPerSync(nativeHandle_); + } + + @Override + public DBOptions setEnableThreadTracking(final boolean enableThreadTracking) { + assert(isOwningHandle()); + setEnableThreadTracking(nativeHandle_, enableThreadTracking); + return this; + } + + @Override + public boolean enableThreadTracking() { + assert(isOwningHandle()); + return enableThreadTracking(nativeHandle_); + } + + @Override + public DBOptions setDelayedWriteRate(final long delayedWriteRate) { + assert(isOwningHandle()); + setDelayedWriteRate(nativeHandle_, delayedWriteRate); + return this; + } + + @Override + public long delayedWriteRate(){ + return delayedWriteRate(nativeHandle_); + } + + @Override + public DBOptions setAllowConcurrentMemtableWrite( + final boolean allowConcurrentMemtableWrite) { + setAllowConcurrentMemtableWrite(nativeHandle_, + allowConcurrentMemtableWrite); + return this; + } + + @Override + public boolean allowConcurrentMemtableWrite() { + return allowConcurrentMemtableWrite(nativeHandle_); + } + + @Override + public DBOptions setEnableWriteThreadAdaptiveYield( + final boolean enableWriteThreadAdaptiveYield) { + setEnableWriteThreadAdaptiveYield(nativeHandle_, + enableWriteThreadAdaptiveYield); + return this; + } + + @Override + public boolean enableWriteThreadAdaptiveYield() { + return enableWriteThreadAdaptiveYield(nativeHandle_); + } + + @Override + public DBOptions setWriteThreadMaxYieldUsec(final long writeThreadMaxYieldUsec) { + setWriteThreadMaxYieldUsec(nativeHandle_, writeThreadMaxYieldUsec); + return this; + } + + @Override + public long writeThreadMaxYieldUsec() { + return writeThreadMaxYieldUsec(nativeHandle_); + } + + @Override + public DBOptions setWriteThreadSlowYieldUsec(final long writeThreadSlowYieldUsec) { + setWriteThreadSlowYieldUsec(nativeHandle_, writeThreadSlowYieldUsec); + return this; + } + + @Override + public long writeThreadSlowYieldUsec() { + return writeThreadSlowYieldUsec(nativeHandle_); + } + + @Override + public DBOptions setSkipStatsUpdateOnDbOpen(final boolean skipStatsUpdateOnDbOpen) { + assert(isOwningHandle()); + setSkipStatsUpdateOnDbOpen(nativeHandle_, skipStatsUpdateOnDbOpen); + return this; + } + + @Override + public boolean skipStatsUpdateOnDbOpen() { + assert(isOwningHandle()); + return skipStatsUpdateOnDbOpen(nativeHandle_); + } + + @Override + public DBOptions setWalRecoveryMode(final WALRecoveryMode walRecoveryMode) { + assert(isOwningHandle()); + setWalRecoveryMode(nativeHandle_, walRecoveryMode.getValue()); + return this; + } + + @Override + public WALRecoveryMode walRecoveryMode() { + assert(isOwningHandle()); + return WALRecoveryMode.getWALRecoveryMode(walRecoveryMode(nativeHandle_)); + } + + @Override + public DBOptions setAllow2pc(final boolean allow2pc) { + assert(isOwningHandle()); + setAllow2pc(nativeHandle_, allow2pc); + return this; + } + + @Override + public boolean allow2pc() { + assert(isOwningHandle()); + return allow2pc(nativeHandle_); + } + + @Override + public DBOptions setRowCache(final Cache rowCache) { + assert(isOwningHandle()); + setRowCache(nativeHandle_, rowCache.nativeHandle_); + this.rowCache_ = rowCache; + return this; + } + + @Override + public Cache rowCache() { + assert(isOwningHandle()); + return this.rowCache_; + } + + @Override + public DBOptions setFailIfOptionsFileError(final boolean failIfOptionsFileError) { + assert(isOwningHandle()); + setFailIfOptionsFileError(nativeHandle_, failIfOptionsFileError); + return this; + } + + @Override + public boolean failIfOptionsFileError() { + assert(isOwningHandle()); + return failIfOptionsFileError(nativeHandle_); + } + + @Override + public DBOptions setDumpMallocStats(final boolean dumpMallocStats) { + assert(isOwningHandle()); + setDumpMallocStats(nativeHandle_, dumpMallocStats); + return this; + } + + @Override + public boolean dumpMallocStats() { + assert(isOwningHandle()); + return dumpMallocStats(nativeHandle_); + } + + @Override + public DBOptions setAvoidFlushDuringRecovery(final boolean avoidFlushDuringRecovery) { + assert(isOwningHandle()); + setAvoidFlushDuringRecovery(nativeHandle_, avoidFlushDuringRecovery); + return this; + } + + @Override + public boolean avoidFlushDuringRecovery() { + assert(isOwningHandle()); + return avoidFlushDuringRecovery(nativeHandle_); + } + + @Override + public DBOptions setAvoidFlushDuringShutdown(final boolean avoidFlushDuringShutdown) { + assert(isOwningHandle()); + setAvoidFlushDuringShutdown(nativeHandle_, avoidFlushDuringShutdown); + return this; + } + + @Override + public boolean avoidFlushDuringShutdown() { + assert(isOwningHandle()); + return avoidFlushDuringShutdown(nativeHandle_); + } + + static final int DEFAULT_NUM_SHARD_BITS = -1; + + + + + /** + *

    Private constructor to be used by + * {@link #getDBOptionsFromProps(java.util.Properties)}

    + * + * @param nativeHandle native handle to DBOptions instance. + */ + private DBOptions(final long nativeHandle) { + super(nativeHandle); + } + + private static native long getDBOptionsFromProps( + String optString); + + private native static long newDBOptions(); + @Override protected final native void disposeInternal(final long handle); + + private native void optimizeForSmallDb(final long handle); + private native void setIncreaseParallelism(long handle, int totalThreads); + private native void setCreateIfMissing(long handle, boolean flag); + private native boolean createIfMissing(long handle); + private native void setCreateMissingColumnFamilies( + long handle, boolean flag); + private native boolean createMissingColumnFamilies(long handle); + private native void setEnv(long handle, long envHandle); + private native void setErrorIfExists(long handle, boolean errorIfExists); + private native boolean errorIfExists(long handle); + private native void setParanoidChecks( + long handle, boolean paranoidChecks); + private native boolean paranoidChecks(long handle); + private native void setRateLimiter(long handle, + long rateLimiterHandle); + private native void setLogger(long handle, + long loggerHandle); + private native void setInfoLogLevel(long handle, byte logLevel); + private native byte infoLogLevel(long handle); + private native void setMaxOpenFiles(long handle, int maxOpenFiles); + private native int maxOpenFiles(long handle); + private native void setMaxFileOpeningThreads(final long handle, + final int maxFileOpeningThreads); + private native int maxFileOpeningThreads(final long handle); + private native void setMaxTotalWalSize(long handle, + long maxTotalWalSize); + private native long maxTotalWalSize(long handle); + private native void setStatistics(final long handle, final long statisticsHandle); + private native long statistics(final long handle); + private native boolean useFsync(long handle); + private native void setUseFsync(long handle, boolean useFsync); + private native void setDbPaths(final long handle, final String[] paths, + final long[] targetSizes); + private native long dbPathsLen(final long handle); + private native void dbPaths(final long handle, final String[] paths, + final long[] targetSizes); + private native void setDbLogDir(long handle, String dbLogDir); + private native String dbLogDir(long handle); + private native void setWalDir(long handle, String walDir); + private native String walDir(long handle); + private native void setDeleteObsoleteFilesPeriodMicros( + long handle, long micros); + private native long deleteObsoleteFilesPeriodMicros(long handle); + private native void setBaseBackgroundCompactions(long handle, + int baseBackgroundCompactions); + private native int baseBackgroundCompactions(long handle); + private native void setMaxBackgroundCompactions( + long handle, int maxBackgroundCompactions); + private native int maxBackgroundCompactions(long handle); + private native void setMaxSubcompactions(long handle, int maxSubcompactions); + private native int maxSubcompactions(long handle); + private native void setMaxBackgroundFlushes( + long handle, int maxBackgroundFlushes); + private native int maxBackgroundFlushes(long handle); + private native void setMaxLogFileSize(long handle, long maxLogFileSize) + throws IllegalArgumentException; + private native long maxLogFileSize(long handle); + private native void setLogFileTimeToRoll( + long handle, long logFileTimeToRoll) throws IllegalArgumentException; + private native long logFileTimeToRoll(long handle); + private native void setKeepLogFileNum(long handle, long keepLogFileNum) + throws IllegalArgumentException; + private native long keepLogFileNum(long handle); + private native void setRecycleLogFileNum(long handle, long recycleLogFileNum); + private native long recycleLogFileNum(long handle); + private native void setMaxManifestFileSize( + long handle, long maxManifestFileSize); + private native long maxManifestFileSize(long handle); + private native void setTableCacheNumshardbits( + long handle, int tableCacheNumshardbits); + private native int tableCacheNumshardbits(long handle); + private native void setWalTtlSeconds(long handle, long walTtlSeconds); + private native long walTtlSeconds(long handle); + private native void setWalSizeLimitMB(long handle, long sizeLimitMB); + private native long walSizeLimitMB(long handle); + private native void setManifestPreallocationSize( + long handle, long size) throws IllegalArgumentException; + private native long manifestPreallocationSize(long handle); + private native void setUseDirectReads(long handle, boolean useDirectReads); + private native boolean useDirectReads(long handle); + private native void setUseDirectIoForFlushAndCompaction( + long handle, boolean useDirectIoForFlushAndCompaction); + private native boolean useDirectIoForFlushAndCompaction(long handle); + private native void setAllowFAllocate(final long handle, + final boolean allowFAllocate); + private native boolean allowFAllocate(final long handle); + private native void setAllowMmapReads( + long handle, boolean allowMmapReads); + private native boolean allowMmapReads(long handle); + private native void setAllowMmapWrites( + long handle, boolean allowMmapWrites); + private native boolean allowMmapWrites(long handle); + private native void setIsFdCloseOnExec( + long handle, boolean isFdCloseOnExec); + private native boolean isFdCloseOnExec(long handle); + private native void setStatsDumpPeriodSec( + long handle, int statsDumpPeriodSec); + private native int statsDumpPeriodSec(long handle); + private native void setAdviseRandomOnOpen( + long handle, boolean adviseRandomOnOpen); + private native boolean adviseRandomOnOpen(long handle); + private native void setDbWriteBufferSize(final long handle, + final long dbWriteBufferSize); + private native long dbWriteBufferSize(final long handle); + private native void setAccessHintOnCompactionStart(final long handle, + final byte accessHintOnCompactionStart); + private native byte accessHintOnCompactionStart(final long handle); + private native void setNewTableReaderForCompactionInputs(final long handle, + final boolean newTableReaderForCompactionInputs); + private native boolean newTableReaderForCompactionInputs(final long handle); + private native void setCompactionReadaheadSize(final long handle, + final long compactionReadaheadSize); + private native long compactionReadaheadSize(final long handle); + private native void setRandomAccessMaxBufferSize(final long handle, + final long randomAccessMaxBufferSize); + private native long randomAccessMaxBufferSize(final long handle); + private native void setWritableFileMaxBufferSize(final long handle, + final long writableFileMaxBufferSize); + private native long writableFileMaxBufferSize(final long handle); + private native void setUseAdaptiveMutex( + long handle, boolean useAdaptiveMutex); + private native boolean useAdaptiveMutex(long handle); + private native void setBytesPerSync( + long handle, long bytesPerSync); + private native long bytesPerSync(long handle); + private native void setWalBytesPerSync(long handle, long walBytesPerSync); + private native long walBytesPerSync(long handle); + private native void setEnableThreadTracking(long handle, + boolean enableThreadTracking); + private native boolean enableThreadTracking(long handle); + private native void setDelayedWriteRate(long handle, long delayedWriteRate); + private native long delayedWriteRate(long handle); + private native void setAllowConcurrentMemtableWrite(long handle, + boolean allowConcurrentMemtableWrite); + private native boolean allowConcurrentMemtableWrite(long handle); + private native void setEnableWriteThreadAdaptiveYield(long handle, + boolean enableWriteThreadAdaptiveYield); + private native boolean enableWriteThreadAdaptiveYield(long handle); + private native void setWriteThreadMaxYieldUsec(long handle, + long writeThreadMaxYieldUsec); + private native long writeThreadMaxYieldUsec(long handle); + private native void setWriteThreadSlowYieldUsec(long handle, + long writeThreadSlowYieldUsec); + private native long writeThreadSlowYieldUsec(long handle); + private native void setSkipStatsUpdateOnDbOpen(final long handle, + final boolean skipStatsUpdateOnDbOpen); + private native boolean skipStatsUpdateOnDbOpen(final long handle); + private native void setWalRecoveryMode(final long handle, + final byte walRecoveryMode); + private native byte walRecoveryMode(final long handle); + private native void setAllow2pc(final long handle, + final boolean allow2pc); + private native boolean allow2pc(final long handle); + private native void setRowCache(final long handle, + final long row_cache_handle); + private native void setFailIfOptionsFileError(final long handle, + final boolean failIfOptionsFileError); + private native boolean failIfOptionsFileError(final long handle); + private native void setDumpMallocStats(final long handle, + final boolean dumpMallocStats); + private native boolean dumpMallocStats(final long handle); + private native void setAvoidFlushDuringRecovery(final long handle, + final boolean avoidFlushDuringRecovery); + private native boolean avoidFlushDuringRecovery(final long handle); + private native void setAvoidFlushDuringShutdown(final long handle, + final boolean avoidFlushDuringShutdown); + private native boolean avoidFlushDuringShutdown(final long handle); + + // instance variables + private Env env_; + private int numShardBits_; + private RateLimiter rateLimiter_; + private Cache rowCache_; +} diff --git a/java/src/main/java/org/rocksdb/DBOptionsInterface.java b/java/src/main/java/org/rocksdb/DBOptionsInterface.java new file mode 100644 index 00000000000..50ca083d37a --- /dev/null +++ b/java/src/main/java/org/rocksdb/DBOptionsInterface.java @@ -0,0 +1,1549 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.Collection; +import java.util.List; + +public interface DBOptionsInterface { + + /** + * Use this if your DB is very small (like under 1GB) and you don't want to + * spend lots of memory for memtables. + * + * @return the instance of the current object. + */ + T optimizeForSmallDb(); + + /** + * Use the specified object to interact with the environment, + * e.g. to read/write files, schedule background work, etc. + * Default: {@link Env#getDefault()} + * + * @param env {@link Env} instance. + * @return the instance of the current Options. + */ + T setEnv(final Env env); + + /** + * Returns the set RocksEnv instance. + * + * @return {@link RocksEnv} instance set in the options. + */ + Env getEnv(); + + /** + *

    By default, RocksDB uses only one background thread for flush and + * compaction. Calling this function will set it up such that total of + * `total_threads` is used.

    + * + *

    You almost definitely want to call this function if your system is + * bottlenecked by RocksDB.

    + * + * @param totalThreads The total number of threads to be used by RocksDB. + * A good value is the number of cores. + * + * @return the instance of the current Options + */ + T setIncreaseParallelism(int totalThreads); + + /** + * If this value is set to true, then the database will be created + * if it is missing during {@code RocksDB.open()}. + * Default: false + * + * @param flag a flag indicating whether to create a database the + * specified database in {@link RocksDB#open(org.rocksdb.Options, String)} operation + * is missing. + * @return the instance of the current Options + * @see RocksDB#open(org.rocksdb.Options, String) + */ + T setCreateIfMissing(boolean flag); + + /** + * Return true if the create_if_missing flag is set to true. + * If true, the database will be created if it is missing. + * + * @return true if the createIfMissing option is set to true. + * @see #setCreateIfMissing(boolean) + */ + boolean createIfMissing(); + + /** + *

    If true, missing column families will be automatically created

    + * + *

    Default: false

    + * + * @param flag a flag indicating if missing column families shall be + * created automatically. + * @return true if missing column families shall be created automatically + * on open. + */ + T setCreateMissingColumnFamilies(boolean flag); + + /** + * Return true if the create_missing_column_families flag is set + * to true. If true column families be created if missing. + * + * @return true if the createMissingColumnFamilies is set to + * true. + * @see #setCreateMissingColumnFamilies(boolean) + */ + boolean createMissingColumnFamilies(); + + /** + * If true, an error will be thrown during RocksDB.open() if the + * database already exists. + * Default: false + * + * @param errorIfExists if true, an exception will be thrown + * during {@code RocksDB.open()} if the database already exists. + * @return the reference to the current option. + * @see RocksDB#open(org.rocksdb.Options, String) + */ + T setErrorIfExists(boolean errorIfExists); + + /** + * If true, an error will be thrown during RocksDB.open() if the + * database already exists. + * + * @return if true, an error is raised when the specified database + * already exists before open. + */ + boolean errorIfExists(); + + /** + * If true, the implementation will do aggressive checking of the + * data it is processing and will stop early if it detects any + * errors. This may have unforeseen ramifications: for example, a + * corruption of one DB entry may cause a large number of entries to + * become unreadable or for the entire DB to become unopenable. + * If any of the writes to the database fails (Put, Delete, Merge, Write), + * the database will switch to read-only mode and fail all other + * Write operations. + * Default: true + * + * @param paranoidChecks a flag to indicate whether paranoid-check + * is on. + * @return the reference to the current option. + */ + T setParanoidChecks(boolean paranoidChecks); + + /** + * If true, the implementation will do aggressive checking of the + * data it is processing and will stop early if it detects any + * errors. This may have unforeseen ramifications: for example, a + * corruption of one DB entry may cause a large number of entries to + * become unreadable or for the entire DB to become unopenable. + * If any of the writes to the database fails (Put, Delete, Merge, Write), + * the database will switch to read-only mode and fail all other + * Write operations. + * + * @return a boolean indicating whether paranoid-check is on. + */ + boolean paranoidChecks(); + + /** + * Use to control write rate of flush and compaction. Flush has higher + * priority than compaction. Rate limiting is disabled if nullptr. + * Default: nullptr + * + * @param rateLimiter {@link org.rocksdb.RateLimiter} instance. + * @return the instance of the current object. + * + * @since 3.10.0 + */ + T setRateLimiter(RateLimiter rateLimiter); + + /** + *

    Any internal progress/error information generated by + * the db will be written to the Logger if it is non-nullptr, + * or to a file stored in the same directory as the DB + * contents if info_log is nullptr.

    + * + *

    Default: nullptr

    + * + * @param logger {@link Logger} instance. + * @return the instance of the current object. + */ + T setLogger(Logger logger); + + /** + *

    Sets the RocksDB log level. Default level is INFO

    + * + * @param infoLogLevel log level to set. + * @return the instance of the current object. + */ + T setInfoLogLevel(InfoLogLevel infoLogLevel); + + /** + *

    Returns currently set log level.

    + * @return {@link org.rocksdb.InfoLogLevel} instance. + */ + InfoLogLevel infoLogLevel(); + + /** + * Number of open files that can be used by the DB. You may need to + * increase this if your database has a large working set. Value -1 means + * files opened are always kept open. You can estimate number of files based + * on {@code target_file_size_base} and {@code target_file_size_multiplier} + * for level-based compaction. For universal-style compaction, you can usually + * set it to -1. + * Default: 5000 + * + * @param maxOpenFiles the maximum number of open files. + * @return the instance of the current object. + */ + T setMaxOpenFiles(int maxOpenFiles); + + /** + * Number of open files that can be used by the DB. You may need to + * increase this if your database has a large working set. Value -1 means + * files opened are always kept open. You can estimate number of files based + * on {@code target_file_size_base} and {@code target_file_size_multiplier} + * for level-based compaction. For universal-style compaction, you can usually + * set it to -1. + * + * @return the maximum number of open files. + */ + int maxOpenFiles(); + + /** + * If {@link #maxOpenFiles()} is -1, DB will open all files on DB::Open(). You + * can use this option to increase the number of threads used to open the + * files. + * + * Default: 16 + * + * @param maxFileOpeningThreads the maximum number of threads to use to + * open files + * + * @return the reference to the current options. + */ + T setMaxFileOpeningThreads(int maxFileOpeningThreads); + + /** + * If {@link #maxOpenFiles()} is -1, DB will open all files on DB::Open(). You + * can use this option to increase the number of threads used to open the + * files. + * + * Default: 16 + * + * @return the maximum number of threads to use to open files + */ + int maxFileOpeningThreads(); + + /** + *

    Once write-ahead logs exceed this size, we will start forcing the + * flush of column families whose memtables are backed by the oldest live + * WAL file (i.e. the ones that are causing all the space amplification). + *

    + *

    If set to 0 (default), we will dynamically choose the WAL size limit to + * be [sum of all write_buffer_size * max_write_buffer_number] * 2

    + *

    Default: 0

    + * + * @param maxTotalWalSize max total wal size. + * @return the instance of the current object. + */ + T setMaxTotalWalSize(long maxTotalWalSize); + + /** + *

    Returns the max total wal size. Once write-ahead logs exceed this size, + * we will start forcing the flush of column families whose memtables are + * backed by the oldest live WAL file (i.e. the ones that are causing all + * the space amplification).

    + * + *

    If set to 0 (default), we will dynamically choose the WAL size limit + * to be [sum of all write_buffer_size * max_write_buffer_number] * 2 + *

    + * + * @return max total wal size + */ + long maxTotalWalSize(); + + /** + *

    Sets the statistics object which collects metrics about database operations. + * Statistics objects should not be shared between DB instances as + * it does not use any locks to prevent concurrent updates.

    + * + * @return the instance of the current object. + * @see RocksDB#open(org.rocksdb.Options, String) + */ + T setStatistics(final Statistics statistics); + + /** + *

    Returns statistics object.

    + * + * @return the instance of the statistics object or null if there is no statistics object. + * @see #setStatistics(Statistics) + */ + Statistics statistics(); + + /** + *

    If true, then every store to stable storage will issue a fsync.

    + *

    If false, then every store to stable storage will issue a fdatasync. + * This parameter should be set to true while storing data to + * filesystem like ext3 that can lose files after a reboot.

    + *

    Default: false

    + * + * @param useFsync a boolean flag to specify whether to use fsync + * @return the instance of the current object. + */ + T setUseFsync(boolean useFsync); + + /** + *

    If true, then every store to stable storage will issue a fsync.

    + *

    If false, then every store to stable storage will issue a fdatasync. + * This parameter should be set to true while storing data to + * filesystem like ext3 that can lose files after a reboot.

    + * + * @return boolean value indicating if fsync is used. + */ + boolean useFsync(); + + /** + * A list of paths where SST files can be put into, with its target size. + * Newer data is placed into paths specified earlier in the vector while + * older data gradually moves to paths specified later in the vector. + * + * For example, you have a flash device with 10GB allocated for the DB, + * as well as a hard drive of 2TB, you should config it to be: + * [{"/flash_path", 10GB}, {"/hard_drive", 2TB}] + * + * The system will try to guarantee data under each path is close to but + * not larger than the target size. But current and future file sizes used + * by determining where to place a file are based on best-effort estimation, + * which means there is a chance that the actual size under the directory + * is slightly more than target size under some workloads. User should give + * some buffer room for those cases. + * + * If none of the paths has sufficient room to place a file, the file will + * be placed to the last path anyway, despite to the target size. + * + * Placing newer data to earlier paths is also best-efforts. User should + * expect user files to be placed in higher levels in some extreme cases. + * + * If left empty, only one path will be used, which is db_name passed when + * opening the DB. + * + * Default: empty + * + * @param dbPaths the paths and target sizes + * + * @return the reference to the current options + */ + T setDbPaths(final Collection dbPaths); + + /** + * A list of paths where SST files can be put into, with its target size. + * Newer data is placed into paths specified earlier in the vector while + * older data gradually moves to paths specified later in the vector. + * + * For example, you have a flash device with 10GB allocated for the DB, + * as well as a hard drive of 2TB, you should config it to be: + * [{"/flash_path", 10GB}, {"/hard_drive", 2TB}] + * + * The system will try to guarantee data under each path is close to but + * not larger than the target size. But current and future file sizes used + * by determining where to place a file are based on best-effort estimation, + * which means there is a chance that the actual size under the directory + * is slightly more than target size under some workloads. User should give + * some buffer room for those cases. + * + * If none of the paths has sufficient room to place a file, the file will + * be placed to the last path anyway, despite to the target size. + * + * Placing newer data to earlier paths is also best-efforts. User should + * expect user files to be placed in higher levels in some extreme cases. + * + * If left empty, only one path will be used, which is db_name passed when + * opening the DB. + * + * Default: {@link java.util.Collections#emptyList()} + * + * @return dbPaths the paths and target sizes + */ + List dbPaths(); + + /** + * This specifies the info LOG dir. + * If it is empty, the log files will be in the same dir as data. + * If it is non empty, the log files will be in the specified dir, + * and the db data dir's absolute path will be used as the log file + * name's prefix. + * + * @param dbLogDir the path to the info log directory + * @return the instance of the current object. + */ + T setDbLogDir(String dbLogDir); + + /** + * Returns the directory of info log. + * + * If it is empty, the log files will be in the same dir as data. + * If it is non empty, the log files will be in the specified dir, + * and the db data dir's absolute path will be used as the log file + * name's prefix. + * + * @return the path to the info log directory + */ + String dbLogDir(); + + /** + * This specifies the absolute dir path for write-ahead logs (WAL). + * If it is empty, the log files will be in the same dir as data, + * dbname is used as the data dir by default + * If it is non empty, the log files will be in kept the specified dir. + * When destroying the db, + * all log files in wal_dir and the dir itself is deleted + * + * @param walDir the path to the write-ahead-log directory. + * @return the instance of the current object. + */ + T setWalDir(String walDir); + + /** + * Returns the path to the write-ahead-logs (WAL) directory. + * + * If it is empty, the log files will be in the same dir as data, + * dbname is used as the data dir by default + * If it is non empty, the log files will be in kept the specified dir. + * When destroying the db, + * all log files in wal_dir and the dir itself is deleted + * + * @return the path to the write-ahead-logs (WAL) directory. + */ + String walDir(); + + /** + * The periodicity when obsolete files get deleted. The default + * value is 6 hours. The files that get out of scope by compaction + * process will still get automatically delete on every compaction, + * regardless of this setting + * + * @param micros the time interval in micros + * @return the instance of the current object. + */ + T setDeleteObsoleteFilesPeriodMicros(long micros); + + /** + * The periodicity when obsolete files get deleted. The default + * value is 6 hours. The files that get out of scope by compaction + * process will still get automatically delete on every compaction, + * regardless of this setting + * + * @return the time interval in micros when obsolete files will be deleted. + */ + long deleteObsoleteFilesPeriodMicros(); + + /** + * Suggested number of concurrent background compaction jobs, submitted to + * the default LOW priority thread pool. + * Default: 1 + * + * @param baseBackgroundCompactions Suggested number of background compaction + * jobs + */ + void setBaseBackgroundCompactions(int baseBackgroundCompactions); + + /** + * Suggested number of concurrent background compaction jobs, submitted to + * the default LOW priority thread pool. + * Default: 1 + * + * @return Suggested number of background compaction jobs + */ + int baseBackgroundCompactions(); + + /** + * Specifies the maximum number of concurrent background compaction jobs, + * submitted to the default LOW priority thread pool. + * If you're increasing this, also consider increasing number of threads in + * LOW priority thread pool. For more information, see + * Default: 1 + * + * @param maxBackgroundCompactions the maximum number of background + * compaction jobs. + * @return the instance of the current object. + * + * @see RocksEnv#setBackgroundThreads(int) + * @see RocksEnv#setBackgroundThreads(int, int) + * @see #maxBackgroundFlushes() + */ + T setMaxBackgroundCompactions(int maxBackgroundCompactions); + + /** + * Returns the maximum number of concurrent background compaction jobs, + * submitted to the default LOW priority thread pool. + * When increasing this number, we may also want to consider increasing + * number of threads in LOW priority thread pool. + * Default: 1 + * + * @return the maximum number of concurrent background compaction jobs. + * @see RocksEnv#setBackgroundThreads(int) + * @see RocksEnv#setBackgroundThreads(int, int) + */ + int maxBackgroundCompactions(); + + /** + * This value represents the maximum number of threads that will + * concurrently perform a compaction job by breaking it into multiple, + * smaller ones that are run simultaneously. + * Default: 1 (i.e. no subcompactions) + * + * @param maxSubcompactions The maximum number of threads that will + * concurrently perform a compaction job + */ + void setMaxSubcompactions(int maxSubcompactions); + + /** + * This value represents the maximum number of threads that will + * concurrently perform a compaction job by breaking it into multiple, + * smaller ones that are run simultaneously. + * Default: 1 (i.e. no subcompactions) + * + * @return The maximum number of threads that will concurrently perform a + * compaction job + */ + int maxSubcompactions(); + + /** + * Specifies the maximum number of concurrent background flush jobs. + * If you're increasing this, also consider increasing number of threads in + * HIGH priority thread pool. For more information, see + * Default: 1 + * + * @param maxBackgroundFlushes number of max concurrent flush jobs + * @return the instance of the current object. + * + * @see RocksEnv#setBackgroundThreads(int) + * @see RocksEnv#setBackgroundThreads(int, int) + * @see #maxBackgroundCompactions() + */ + T setMaxBackgroundFlushes(int maxBackgroundFlushes); + + /** + * Returns the maximum number of concurrent background flush jobs. + * If you're increasing this, also consider increasing number of threads in + * HIGH priority thread pool. For more information, see + * Default: 1 + * + * @return the maximum number of concurrent background flush jobs. + * @see RocksEnv#setBackgroundThreads(int) + * @see RocksEnv#setBackgroundThreads(int, int) + */ + int maxBackgroundFlushes(); + + /** + * Specifies the maximum size of a info log file. If the current log file + * is larger than `max_log_file_size`, a new info log file will + * be created. + * If 0, all logs will be written to one log file. + * + * @param maxLogFileSize the maximum size of a info log file. + * @return the instance of the current object. + * @throws java.lang.IllegalArgumentException thrown on 32-Bit platforms + * while overflowing the underlying platform specific value. + */ + T setMaxLogFileSize(long maxLogFileSize); + + /** + * Returns the maximum size of a info log file. If the current log file + * is larger than this size, a new info log file will be created. + * If 0, all logs will be written to one log file. + * + * @return the maximum size of the info log file. + */ + long maxLogFileSize(); + + /** + * Specifies the time interval for the info log file to roll (in seconds). + * If specified with non-zero value, log file will be rolled + * if it has been active longer than `log_file_time_to_roll`. + * Default: 0 (disabled) + * + * @param logFileTimeToRoll the time interval in seconds. + * @return the instance of the current object. + * @throws java.lang.IllegalArgumentException thrown on 32-Bit platforms + * while overflowing the underlying platform specific value. + */ + T setLogFileTimeToRoll(long logFileTimeToRoll); + + /** + * Returns the time interval for the info log file to roll (in seconds). + * If specified with non-zero value, log file will be rolled + * if it has been active longer than `log_file_time_to_roll`. + * Default: 0 (disabled) + * + * @return the time interval in seconds. + */ + long logFileTimeToRoll(); + + /** + * Specifies the maximum number of info log files to be kept. + * Default: 1000 + * + * @param keepLogFileNum the maximum number of info log files to be kept. + * @return the instance of the current object. + * @throws java.lang.IllegalArgumentException thrown on 32-Bit platforms + * while overflowing the underlying platform specific value. + */ + T setKeepLogFileNum(long keepLogFileNum); + + /** + * Returns the maximum number of info log files to be kept. + * Default: 1000 + * + * @return the maximum number of info log files to be kept. + */ + long keepLogFileNum(); + + /** + * Recycle log files. + * + * If non-zero, we will reuse previously written log files for new + * logs, overwriting the old data. The value indicates how many + * such files we will keep around at any point in time for later + * use. + * + * This is more efficient because the blocks are already + * allocated and fdatasync does not need to update the inode after + * each write. + * + * Default: 0 + * + * @param recycleLogFileNum the number of log files to keep for recycling + * + * @return the reference to the current options + */ + T setRecycleLogFileNum(long recycleLogFileNum); + + /** + * Recycle log files. + * + * If non-zero, we will reuse previously written log files for new + * logs, overwriting the old data. The value indicates how many + * such files we will keep around at any point in time for later + * use. + * + * This is more efficient because the blocks are already + * allocated and fdatasync does not need to update the inode after + * each write. + * + * Default: 0 + * + * @return the number of log files kept for recycling + */ + long recycleLogFileNum(); + + /** + * Manifest file is rolled over on reaching this limit. + * The older manifest file be deleted. + * The default value is MAX_INT so that roll-over does not take place. + * + * @param maxManifestFileSize the size limit of a manifest file. + * @return the instance of the current object. + */ + T setMaxManifestFileSize(long maxManifestFileSize); + + /** + * Manifest file is rolled over on reaching this limit. + * The older manifest file be deleted. + * The default value is MAX_INT so that roll-over does not take place. + * + * @return the size limit of a manifest file. + */ + long maxManifestFileSize(); + + /** + * Number of shards used for table cache. + * + * @param tableCacheNumshardbits the number of chards + * @return the instance of the current object. + */ + T setTableCacheNumshardbits(int tableCacheNumshardbits); + + /** + * Number of shards used for table cache. + * + * @return the number of shards used for table cache. + */ + int tableCacheNumshardbits(); + + /** + * {@link #walTtlSeconds()} and {@link #walSizeLimitMB()} affect how archived logs + * will be deleted. + *
      + *
    1. If both set to 0, logs will be deleted asap and will not get into + * the archive.
      2. + *
    2. If WAL_ttl_seconds is 0 and WAL_size_limit_MB is not 0, + * WAL files will be checked every 10 min and if total size is greater + * then WAL_size_limit_MB, they will be deleted starting with the + * earliest until size_limit is met. All empty files will be deleted.
      3. + *
    3. If WAL_ttl_seconds is not 0 and WAL_size_limit_MB is 0, then + * WAL files will be checked every WAL_ttl_secondsi / 2 and those that + * are older than WAL_ttl_seconds will be deleted.
      4. + *
    4. If both are not 0, WAL files will be checked every 10 min and both + * checks will be performed with ttl being first.
      5. + *
    + * + * @param walTtlSeconds the ttl seconds + * @return the instance of the current object. + * @see #setWalSizeLimitMB(long) + */ + T setWalTtlSeconds(long walTtlSeconds); + + /** + * WalTtlSeconds() and walSizeLimitMB() affect how archived logs + * will be deleted. + *
      + *
    1. If both set to 0, logs will be deleted asap and will not get into + * the archive.
      2. + *
    2. If WAL_ttl_seconds is 0 and WAL_size_limit_MB is not 0, + * WAL files will be checked every 10 min and if total size is greater + * then WAL_size_limit_MB, they will be deleted starting with the + * earliest until size_limit is met. All empty files will be deleted.
      3. + *
    3. If WAL_ttl_seconds is not 0 and WAL_size_limit_MB is 0, then + * WAL files will be checked every WAL_ttl_secondsi / 2 and those that + * are older than WAL_ttl_seconds will be deleted.
      4. + *
    4. If both are not 0, WAL files will be checked every 10 min and both + * checks will be performed with ttl being first.
      5. + *
    + * + * @return the wal-ttl seconds + * @see #walSizeLimitMB() + */ + long walTtlSeconds(); + + /** + * WalTtlSeconds() and walSizeLimitMB() affect how archived logs + * will be deleted. + *
      + *
    1. If both set to 0, logs will be deleted asap and will not get into + * the archive.
      2. + *
    2. If WAL_ttl_seconds is 0 and WAL_size_limit_MB is not 0, + * WAL files will be checked every 10 min and if total size is greater + * then WAL_size_limit_MB, they will be deleted starting with the + * earliest until size_limit is met. All empty files will be deleted.
      3. + *
    3. If WAL_ttl_seconds is not 0 and WAL_size_limit_MB is 0, then + * WAL files will be checked every WAL_ttl_secondsi / 2 and those that + * are older than WAL_ttl_seconds will be deleted.
      4. + *
    4. If both are not 0, WAL files will be checked every 10 min and both + * checks will be performed with ttl being first.
      5. + *
    + * + * @param sizeLimitMB size limit in mega-bytes. + * @return the instance of the current object. + * @see #setWalSizeLimitMB(long) + */ + T setWalSizeLimitMB(long sizeLimitMB); + + /** + * {@link #walTtlSeconds()} and {@code #walSizeLimitMB()} affect how archived logs + * will be deleted. + *
      + *
    1. If both set to 0, logs will be deleted asap and will not get into + * the archive.
      2. + *
    2. If WAL_ttl_seconds is 0 and WAL_size_limit_MB is not 0, + * WAL files will be checked every 10 min and if total size is greater + * then WAL_size_limit_MB, they will be deleted starting with the + * earliest until size_limit is met. All empty files will be deleted.
      3. + *
    3. If WAL_ttl_seconds is not 0 and WAL_size_limit_MB is 0, then + * WAL files will be checked every WAL_ttl_seconds i / 2 and those that + * are older than WAL_ttl_seconds will be deleted.
      4. + *
    4. If both are not 0, WAL files will be checked every 10 min and both + * checks will be performed with ttl being first.
      5. + *
    + * @return size limit in mega-bytes. + * @see #walSizeLimitMB() + */ + long walSizeLimitMB(); + + /** + * Number of bytes to preallocate (via fallocate) the manifest + * files. Default is 4mb, which is reasonable to reduce random IO + * as well as prevent overallocation for mounts that preallocate + * large amounts of data (such as xfs's allocsize option). + * + * @param size the size in byte + * @return the instance of the current object. + * @throws java.lang.IllegalArgumentException thrown on 32-Bit platforms + * while overflowing the underlying platform specific value. + */ + T setManifestPreallocationSize(long size); + + /** + * Number of bytes to preallocate (via fallocate) the manifest + * files. Default is 4mb, which is reasonable to reduce random IO + * as well as prevent overallocation for mounts that preallocate + * large amounts of data (such as xfs's allocsize option). + * + * @return size in bytes. + */ + long manifestPreallocationSize(); + + /** + * Enable the OS to use direct I/O for reading sst tables. + * Default: false + * + * @param useDirectReads if true, then direct read is enabled + * @return the instance of the current object. + */ + T setUseDirectReads(boolean useDirectReads); + + /** + * Enable the OS to use direct I/O for reading sst tables. + * Default: false + * + * @return if true, then direct reads are enabled + */ + boolean useDirectReads(); + + /** + * Enable the OS to use direct reads and writes in flush and + * compaction + * Default: false + * + * @param useDirectIoForFlushAndCompaction if true, then direct + * I/O will be enabled for background flush and compactions + * @return the instance of the current object. + */ + T setUseDirectIoForFlushAndCompaction(boolean useDirectIoForFlushAndCompaction); + + /** + * Enable the OS to use direct reads and writes in flush and + * compaction + * + * @return if true, then direct I/O is enabled for flush and + * compaction + */ + boolean useDirectIoForFlushAndCompaction(); + + /** + * Whether fallocate calls are allowed + * + * @param allowFAllocate false if fallocate() calls are bypassed + * + * @return the reference to the current options. + */ + T setAllowFAllocate(boolean allowFAllocate); + + /** + * Whether fallocate calls are allowed + * + * @return false if fallocate() calls are bypassed + */ + boolean allowFAllocate(); + + /** + * Allow the OS to mmap file for reading sst tables. + * Default: false + * + * @param allowMmapReads true if mmap reads are allowed. + * @return the instance of the current object. + */ + T setAllowMmapReads(boolean allowMmapReads); + + /** + * Allow the OS to mmap file for reading sst tables. + * Default: false + * + * @return true if mmap reads are allowed. + */ + boolean allowMmapReads(); + + /** + * Allow the OS to mmap file for writing. Default: false + * + * @param allowMmapWrites true if mmap writes are allowd. + * @return the instance of the current object. + */ + T setAllowMmapWrites(boolean allowMmapWrites); + + /** + * Allow the OS to mmap file for writing. Default: false + * + * @return true if mmap writes are allowed. + */ + boolean allowMmapWrites(); + + /** + * Disable child process inherit open files. Default: true + * + * @param isFdCloseOnExec true if child process inheriting open + * files is disabled. + * @return the instance of the current object. + */ + T setIsFdCloseOnExec(boolean isFdCloseOnExec); + + /** + * Disable child process inherit open files. Default: true + * + * @return true if child process inheriting open files is disabled. + */ + boolean isFdCloseOnExec(); + + /** + * if not zero, dump rocksdb.stats to LOG every stats_dump_period_sec + * Default: 600 (10 minutes) + * + * @param statsDumpPeriodSec time interval in seconds. + * @return the instance of the current object. + */ + T setStatsDumpPeriodSec(int statsDumpPeriodSec); + + /** + * If not zero, dump rocksdb.stats to LOG every stats_dump_period_sec + * Default: 600 (10 minutes) + * + * @return time interval in seconds. + */ + int statsDumpPeriodSec(); + + /** + * If set true, will hint the underlying file system that the file + * access pattern is random, when a sst file is opened. + * Default: true + * + * @param adviseRandomOnOpen true if hinting random access is on. + * @return the instance of the current object. + */ + T setAdviseRandomOnOpen(boolean adviseRandomOnOpen); + + /** + * If set true, will hint the underlying file system that the file + * access pattern is random, when a sst file is opened. + * Default: true + * + * @return true if hinting random access is on. + */ + boolean adviseRandomOnOpen(); + + /** + * Amount of data to build up in memtables across all column + * families before writing to disk. + * + * This is distinct from {@link ColumnFamilyOptions#writeBufferSize()}, + * which enforces a limit for a single memtable. + * + * This feature is disabled by default. Specify a non-zero value + * to enable it. + * + * Default: 0 (disabled) + * + * @param dbWriteBufferSize the size of the write buffer + * + * @return the reference to the current options. + */ + T setDbWriteBufferSize(long dbWriteBufferSize); + + /** + * Amount of data to build up in memtables across all column + * families before writing to disk. + * + * This is distinct from {@link ColumnFamilyOptions#writeBufferSize()}, + * which enforces a limit for a single memtable. + * + * This feature is disabled by default. Specify a non-zero value + * to enable it. + * + * Default: 0 (disabled) + * + * @return the size of the write buffer + */ + long dbWriteBufferSize(); + + /** + * Specify the file access pattern once a compaction is started. + * It will be applied to all input files of a compaction. + * + * Default: {@link AccessHint#NORMAL} + * + * @param accessHint The access hint + * + * @return the reference to the current options. + */ + T setAccessHintOnCompactionStart(final AccessHint accessHint); + + /** + * Specify the file access pattern once a compaction is started. + * It will be applied to all input files of a compaction. + * + * Default: {@link AccessHint#NORMAL} + * + * @return The access hint + */ + AccessHint accessHintOnCompactionStart(); + + /** + * If true, always create a new file descriptor and new table reader + * for compaction inputs. Turn this parameter on may introduce extra + * memory usage in the table reader, if it allocates extra memory + * for indexes. This will allow file descriptor prefetch options + * to be set for compaction input files and not to impact file + * descriptors for the same file used by user queries. + * Suggest to enable {@link BlockBasedTableConfig#cacheIndexAndFilterBlocks()} + * for this mode if using block-based table. + * + * Default: false + * + * @param newTableReaderForCompactionInputs true if a new file descriptor and + * table reader should be created for compaction inputs + * + * @return the reference to the current options. + */ + T setNewTableReaderForCompactionInputs( + boolean newTableReaderForCompactionInputs); + + /** + * If true, always create a new file descriptor and new table reader + * for compaction inputs. Turn this parameter on may introduce extra + * memory usage in the table reader, if it allocates extra memory + * for indexes. This will allow file descriptor prefetch options + * to be set for compaction input files and not to impact file + * descriptors for the same file used by user queries. + * Suggest to enable {@link BlockBasedTableConfig#cacheIndexAndFilterBlocks()} + * for this mode if using block-based table. + * + * Default: false + * + * @return true if a new file descriptor and table reader are created for + * compaction inputs + */ + boolean newTableReaderForCompactionInputs(); + + /** + * If non-zero, we perform bigger reads when doing compaction. If you're + * running RocksDB on spinning disks, you should set this to at least 2MB. + * + * That way RocksDB's compaction is doing sequential instead of random reads. + * When non-zero, we also force {@link #newTableReaderForCompactionInputs()} + * to true. + * + * Default: 0 + * + * @param compactionReadaheadSize The compaction read-ahead size + * + * @return the reference to the current options. + */ + T setCompactionReadaheadSize(final long compactionReadaheadSize); + + /** + * If non-zero, we perform bigger reads when doing compaction. If you're + * running RocksDB on spinning disks, you should set this to at least 2MB. + * + * That way RocksDB's compaction is doing sequential instead of random reads. + * When non-zero, we also force {@link #newTableReaderForCompactionInputs()} + * to true. + * + * Default: 0 + * + * @return The compaction read-ahead size + */ + long compactionReadaheadSize(); + + /** + * This is a maximum buffer size that is used by WinMmapReadableFile in + * unbuffered disk I/O mode. We need to maintain an aligned buffer for + * reads. We allow the buffer to grow until the specified value and then + * for bigger requests allocate one shot buffers. In unbuffered mode we + * always bypass read-ahead buffer at ReadaheadRandomAccessFile + * When read-ahead is required we then make use of + * {@link #compactionReadaheadSize()} value and always try to read ahead. + * With read-ahead we always pre-allocate buffer to the size instead of + * growing it up to a limit. + * + * This option is currently honored only on Windows + * + * Default: 1 Mb + * + * Special value: 0 - means do not maintain per instance buffer. Allocate + * per request buffer and avoid locking. + * + * @param randomAccessMaxBufferSize the maximum size of the random access + * buffer + * + * @return the reference to the current options. + */ + T setRandomAccessMaxBufferSize(long randomAccessMaxBufferSize); + + /** + * This is a maximum buffer size that is used by WinMmapReadableFile in + * unbuffered disk I/O mode. We need to maintain an aligned buffer for + * reads. We allow the buffer to grow until the specified value and then + * for bigger requests allocate one shot buffers. In unbuffered mode we + * always bypass read-ahead buffer at ReadaheadRandomAccessFile + * When read-ahead is required we then make use of + * {@link #compactionReadaheadSize()} value and always try to read ahead. + * With read-ahead we always pre-allocate buffer to the size instead of + * growing it up to a limit. + * + * This option is currently honored only on Windows + * + * Default: 1 Mb + * + * Special value: 0 - means do not maintain per instance buffer. Allocate + * per request buffer and avoid locking. + * + * @return the maximum size of the random access buffer + */ + long randomAccessMaxBufferSize(); + + /** + * This is the maximum buffer size that is used by WritableFileWriter. + * On Windows, we need to maintain an aligned buffer for writes. + * We allow the buffer to grow until it's size hits the limit. + * + * Default: 1024 * 1024 (1 MB) + * + * @param writableFileMaxBufferSize the maximum buffer size + * + * @return the reference to the current options. + */ + T setWritableFileMaxBufferSize(long writableFileMaxBufferSize); + + /** + * This is the maximum buffer size that is used by WritableFileWriter. + * On Windows, we need to maintain an aligned buffer for writes. + * We allow the buffer to grow until it's size hits the limit. + * + * Default: 1024 * 1024 (1 MB) + * + * @return the maximum buffer size + */ + long writableFileMaxBufferSize(); + + /** + * Use adaptive mutex, which spins in the user space before resorting + * to kernel. This could reduce context switch when the mutex is not + * heavily contended. However, if the mutex is hot, we could end up + * wasting spin time. + * Default: false + * + * @param useAdaptiveMutex true if adaptive mutex is used. + * @return the instance of the current object. + */ + T setUseAdaptiveMutex(boolean useAdaptiveMutex); + + /** + * Use adaptive mutex, which spins in the user space before resorting + * to kernel. This could reduce context switch when the mutex is not + * heavily contended. However, if the mutex is hot, we could end up + * wasting spin time. + * Default: false + * + * @return true if adaptive mutex is used. + */ + boolean useAdaptiveMutex(); + + /** + * Allows OS to incrementally sync files to disk while they are being + * written, asynchronously, in the background. + * Issue one request for every bytes_per_sync written. 0 turns it off. + * Default: 0 + * + * @param bytesPerSync size in bytes + * @return the instance of the current object. + */ + T setBytesPerSync(long bytesPerSync); + + /** + * Allows OS to incrementally sync files to disk while they are being + * written, asynchronously, in the background. + * Issue one request for every bytes_per_sync written. 0 turns it off. + * Default: 0 + * + * @return size in bytes + */ + long bytesPerSync(); + + /** + * Same as {@link #setBytesPerSync(long)} , but applies to WAL files + * + * Default: 0, turned off + * + * @param walBytesPerSync size in bytes + * @return the instance of the current object. + */ + T setWalBytesPerSync(long walBytesPerSync); + + /** + * Same as {@link #bytesPerSync()} , but applies to WAL files + * + * Default: 0, turned off + * + * @return size in bytes + */ + long walBytesPerSync(); + + /** + * If true, then the status of the threads involved in this DB will + * be tracked and available via GetThreadList() API. + * + * Default: false + * + * @param enableThreadTracking true to enable tracking + * + * @return the reference to the current options. + */ + T setEnableThreadTracking(boolean enableThreadTracking); + + /** + * If true, then the status of the threads involved in this DB will + * be tracked and available via GetThreadList() API. + * + * Default: false + * + * @return true if tracking is enabled + */ + boolean enableThreadTracking(); + + /** + * The limited write rate to DB if + * {@link ColumnFamilyOptions#softPendingCompactionBytesLimit()} or + * {@link ColumnFamilyOptions#level0SlowdownWritesTrigger()} is triggered, + * or we are writing to the last mem table allowed and we allow more than 3 + * mem tables. It is calculated using size of user write requests before + * compression. RocksDB may decide to slow down more if the compaction still + * gets behind further. + * + * Unit: bytes per second. + * + * Default: 16MB/s + * + * @param delayedWriteRate the rate in bytes per second + * + * @return the reference to the current options. + */ + T setDelayedWriteRate(long delayedWriteRate); + + /** + * The limited write rate to DB if + * {@link ColumnFamilyOptions#softPendingCompactionBytesLimit()} or + * {@link ColumnFamilyOptions#level0SlowdownWritesTrigger()} is triggered, + * or we are writing to the last mem table allowed and we allow more than 3 + * mem tables. It is calculated using size of user write requests before + * compression. RocksDB may decide to slow down more if the compaction still + * gets behind further. + * + * Unit: bytes per second. + * + * Default: 16MB/s + * + * @return the rate in bytes per second + */ + long delayedWriteRate(); + + /** + * If true, allow multi-writers to update mem tables in parallel. + * Only some memtable factorys support concurrent writes; currently it + * is implemented only for SkipListFactory. Concurrent memtable writes + * are not compatible with inplace_update_support or filter_deletes. + * It is strongly recommended to set + * {@link #setEnableWriteThreadAdaptiveYield(boolean)} if you are going to use + * this feature. + * Default: false + * + * @param allowConcurrentMemtableWrite true to enable concurrent writes + * for the memtable + * + * @return the reference to the current options. + */ + T setAllowConcurrentMemtableWrite(boolean allowConcurrentMemtableWrite); + + /** + * If true, allow multi-writers to update mem tables in parallel. + * Only some memtable factorys support concurrent writes; currently it + * is implemented only for SkipListFactory. Concurrent memtable writes + * are not compatible with inplace_update_support or filter_deletes. + * It is strongly recommended to set + * {@link #setEnableWriteThreadAdaptiveYield(boolean)} if you are going to use + * this feature. + * Default: false + * + * @return true if concurrent writes are enabled for the memtable + */ + boolean allowConcurrentMemtableWrite(); + + /** + * If true, threads synchronizing with the write batch group leader will + * wait for up to {@link #writeThreadMaxYieldUsec()} before blocking on a + * mutex. This can substantially improve throughput for concurrent workloads, + * regardless of whether {@link #allowConcurrentMemtableWrite()} is enabled. + * Default: false + * + * @param enableWriteThreadAdaptiveYield true to enable adaptive yield for the + * write threads + * + * @return the reference to the current options. + */ + T setEnableWriteThreadAdaptiveYield( + boolean enableWriteThreadAdaptiveYield); + + /** + * If true, threads synchronizing with the write batch group leader will + * wait for up to {@link #writeThreadMaxYieldUsec()} before blocking on a + * mutex. This can substantially improve throughput for concurrent workloads, + * regardless of whether {@link #allowConcurrentMemtableWrite()} is enabled. + * Default: false + * + * @return true if adaptive yield is enabled + * for the writing threads + */ + boolean enableWriteThreadAdaptiveYield(); + + /** + * The maximum number of microseconds that a write operation will use + * a yielding spin loop to coordinate with other write threads before + * blocking on a mutex. (Assuming {@link #writeThreadSlowYieldUsec()} is + * set properly) increasing this value is likely to increase RocksDB + * throughput at the expense of increased CPU usage. + * Default: 100 + * + * @param writeThreadMaxYieldUsec maximum number of microseconds + * + * @return the reference to the current options. + */ + T setWriteThreadMaxYieldUsec(long writeThreadMaxYieldUsec); + + /** + * The maximum number of microseconds that a write operation will use + * a yielding spin loop to coordinate with other write threads before + * blocking on a mutex. (Assuming {@link #writeThreadSlowYieldUsec()} is + * set properly) increasing this value is likely to increase RocksDB + * throughput at the expense of increased CPU usage. + * Default: 100 + * + * @return the maximum number of microseconds + */ + long writeThreadMaxYieldUsec(); + + /** + * The latency in microseconds after which a std::this_thread::yield + * call (sched_yield on Linux) is considered to be a signal that + * other processes or threads would like to use the current core. + * Increasing this makes writer threads more likely to take CPU + * by spinning, which will show up as an increase in the number of + * involuntary context switches. + * Default: 3 + * + * @param writeThreadSlowYieldUsec the latency in microseconds + * + * @return the reference to the current options. + */ + T setWriteThreadSlowYieldUsec(long writeThreadSlowYieldUsec); + + /** + * The latency in microseconds after which a std::this_thread::yield + * call (sched_yield on Linux) is considered to be a signal that + * other processes or threads would like to use the current core. + * Increasing this makes writer threads more likely to take CPU + * by spinning, which will show up as an increase in the number of + * involuntary context switches. + * Default: 3 + * + * @return writeThreadSlowYieldUsec the latency in microseconds + */ + long writeThreadSlowYieldUsec(); + + /** + * If true, then DB::Open() will not update the statistics used to optimize + * compaction decision by loading table properties from many files. + * Turning off this feature will improve DBOpen time especially in + * disk environment. + * + * Default: false + * + * @param skipStatsUpdateOnDbOpen true if updating stats will be skipped + * + * @return the reference to the current options. + */ + T setSkipStatsUpdateOnDbOpen(boolean skipStatsUpdateOnDbOpen); + + /** + * If true, then DB::Open() will not update the statistics used to optimize + * compaction decision by loading table properties from many files. + * Turning off this feature will improve DBOpen time especially in + * disk environment. + * + * Default: false + * + * @return true if updating stats will be skipped + */ + boolean skipStatsUpdateOnDbOpen(); + + /** + * Recovery mode to control the consistency while replaying WAL + * + * Default: {@link WALRecoveryMode#PointInTimeRecovery} + * + * @param walRecoveryMode The WAL recover mode + * + * @return the reference to the current options. + */ + T setWalRecoveryMode(WALRecoveryMode walRecoveryMode); + + /** + * Recovery mode to control the consistency while replaying WAL + * + * Default: {@link WALRecoveryMode#PointInTimeRecovery} + * + * @return The WAL recover mode + */ + WALRecoveryMode walRecoveryMode(); + + /** + * if set to false then recovery will fail when a prepared + * transaction is encountered in the WAL + * + * Default: false + * + * @param allow2pc true if two-phase-commit is enabled + * + * @return the reference to the current options. + */ + T setAllow2pc(boolean allow2pc); + + /** + * if set to false then recovery will fail when a prepared + * transaction is encountered in the WAL + * + * Default: false + * + * @return true if two-phase-commit is enabled + */ + boolean allow2pc(); + + /** + * A global cache for table-level rows. + * + * Default: null (disabled) + * + * @param rowCache The global row cache + * + * @return the reference to the current options. + */ + T setRowCache(final Cache rowCache); + + /** + * A global cache for table-level rows. + * + * Default: null (disabled) + * + * @return The global row cache + */ + Cache rowCache(); + + /** + * If true, then DB::Open / CreateColumnFamily / DropColumnFamily + * / SetOptions will fail if options file is not detected or properly + * persisted. + * + * DEFAULT: false + * + * @param failIfOptionsFileError true if we should fail if there is an error + * in the options file + * + * @return the reference to the current options. + */ + T setFailIfOptionsFileError(boolean failIfOptionsFileError); + + /** + * If true, then DB::Open / CreateColumnFamily / DropColumnFamily + * / SetOptions will fail if options file is not detected or properly + * persisted. + * + * DEFAULT: false + * + * @return true if we should fail if there is an error in the options file + */ + boolean failIfOptionsFileError(); + + /** + * If true, then print malloc stats together with rocksdb.stats + * when printing to LOG. + * + * DEFAULT: false + * + * @param dumpMallocStats true if malloc stats should be printed to LOG + * + * @return the reference to the current options. + */ + T setDumpMallocStats(boolean dumpMallocStats); + + /** + * If true, then print malloc stats together with rocksdb.stats + * when printing to LOG. + * + * DEFAULT: false + * + * @return true if malloc stats should be printed to LOG + */ + boolean dumpMallocStats(); + + /** + * By default RocksDB replay WAL logs and flush them on DB open, which may + * create very small SST files. If this option is enabled, RocksDB will try + * to avoid (but not guarantee not to) flush during recovery. Also, existing + * WAL logs will be kept, so that if crash happened before flush, we still + * have logs to recover from. + * + * DEFAULT: false + * + * @param avoidFlushDuringRecovery true to try to avoid (but not guarantee + * not to) flush during recovery + * + * @return the reference to the current options. + */ + T setAvoidFlushDuringRecovery(boolean avoidFlushDuringRecovery); + + /** + * By default RocksDB replay WAL logs and flush them on DB open, which may + * create very small SST files. If this option is enabled, RocksDB will try + * to avoid (but not guarantee not to) flush during recovery. Also, existing + * WAL logs will be kept, so that if crash happened before flush, we still + * have logs to recover from. + * + * DEFAULT: false + * + * @return true to try to avoid (but not guarantee not to) flush during + * recovery + */ + boolean avoidFlushDuringRecovery(); + + /** + * By default RocksDB will flush all memtables on DB close if there are + * unpersisted data (i.e. with WAL disabled) The flush can be skip to speedup + * DB close. Unpersisted data WILL BE LOST. + * + * DEFAULT: false + * + * Dynamically changeable through + * {@link RocksDB#setOptions(ColumnFamilyHandle, MutableColumnFamilyOptions)} + * API. + * + * @param avoidFlushDuringShutdown true if we should avoid flush during + * shutdown + * + * @return the reference to the current options. + */ + T setAvoidFlushDuringShutdown(boolean avoidFlushDuringShutdown); + + /** + * By default RocksDB will flush all memtables on DB close if there are + * unpersisted data (i.e. with WAL disabled) The flush can be skip to speedup + * DB close. Unpersisted data WILL BE LOST. + * + * DEFAULT: false + * + * Dynamically changeable through + * {@link RocksDB#setOptions(ColumnFamilyHandle, MutableColumnFamilyOptions)} + * API. + * + * @return true if we should avoid flush during shutdown + */ + boolean avoidFlushDuringShutdown(); +} diff --git a/java/src/main/java/org/rocksdb/DbPath.java b/java/src/main/java/org/rocksdb/DbPath.java new file mode 100644 index 00000000000..3f0b67557c5 --- /dev/null +++ b/java/src/main/java/org/rocksdb/DbPath.java @@ -0,0 +1,47 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.nio.file.Path; + +/** + * Tuple of database path and target size + */ +public class DbPath { + final Path path; + final long targetSize; + + public DbPath(final Path path, final long targetSize) { + this.path = path; + this.targetSize = targetSize; + } + + @Override + public boolean equals(final Object o) { + if (this == o) { + return true; + } + + if (o == null || getClass() != o.getClass()) { + return false; + } + + final DbPath dbPath = (DbPath) o; + + if (targetSize != dbPath.targetSize) { + return false; + } + + return path != null ? path.equals(dbPath.path) : dbPath.path == null; + } + + @Override + public int hashCode() { + int result = path != null ? path.hashCode() : 0; + result = 31 * result + (int) (targetSize ^ (targetSize >>> 32)); + return result; + } +} diff --git a/java/src/main/java/org/rocksdb/DirectComparator.java b/java/src/main/java/org/rocksdb/DirectComparator.java new file mode 100644 index 00000000000..4c37dfd56bb --- /dev/null +++ b/java/src/main/java/org/rocksdb/DirectComparator.java @@ -0,0 +1,33 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Base class for comparators which will receive + * ByteBuffer based access via org.rocksdb.DirectSlice + * in their compare method implementation. + * + * ByteBuffer based slices perform better when large keys + * are involved. When using smaller keys consider + * using @see org.rocksdb.Comparator + */ +public abstract class DirectComparator extends AbstractComparator { + + private final long nativeHandle_; + + public DirectComparator(final ComparatorOptions copt) { + super(); + this.nativeHandle_ = createNewDirectComparator0(copt.nativeHandle_); + } + + @Override + protected final long getNativeHandle() { + return nativeHandle_; + } + + private native long createNewDirectComparator0( + final long comparatorOptionsHandle); +} diff --git a/java/src/main/java/org/rocksdb/DirectSlice.java b/java/src/main/java/org/rocksdb/DirectSlice.java new file mode 100644 index 00000000000..b0d35c3cc5a --- /dev/null +++ b/java/src/main/java/org/rocksdb/DirectSlice.java @@ -0,0 +1,132 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.nio.ByteBuffer; + +/** + * Base class for slices which will receive direct + * ByteBuffer based access to the underlying data. + * + * ByteBuffer backed slices typically perform better with + * larger keys and values. When using smaller keys and + * values consider using @see org.rocksdb.Slice + */ +public class DirectSlice extends AbstractSlice { + public final static DirectSlice NONE = new DirectSlice(); + + /** + * Indicates whether we have to free the memory pointed to by the Slice + */ + private final boolean internalBuffer; + private volatile boolean cleared = false; + private volatile long internalBufferOffset = 0; + + /** + * Called from JNI to construct a new Java DirectSlice + * without an underlying C++ object set + * at creation time. + * + * Note: You should be aware that it is intentionally marked as + * package-private. This is so that developers cannot construct their own + * default DirectSlice objects (at present). As developers cannot construct + * their own DirectSlice objects through this, they are not creating + * underlying C++ DirectSlice objects, and so there is nothing to free + * (dispose) from Java. + */ + DirectSlice() { + super(); + this.internalBuffer = false; + } + + /** + * Constructs a slice + * where the data is taken from + * a String. + * + * @param str The string + */ + public DirectSlice(final String str) { + super(createNewSliceFromString(str)); + this.internalBuffer = true; + } + + /** + * Constructs a slice where the data is + * read from the provided + * ByteBuffer up to a certain length + * + * @param data The buffer containing the data + * @param length The length of the data to use for the slice + */ + public DirectSlice(final ByteBuffer data, final int length) { + super(createNewDirectSlice0(ensureDirect(data), length)); + this.internalBuffer = false; + } + + /** + * Constructs a slice where the data is + * read from the provided + * ByteBuffer + * + * @param data The bugger containing the data + */ + public DirectSlice(final ByteBuffer data) { + super(createNewDirectSlice1(ensureDirect(data))); + this.internalBuffer = false; + } + + private static ByteBuffer ensureDirect(final ByteBuffer data) { + if(!data.isDirect()) { + throw new IllegalArgumentException("The ByteBuffer must be direct"); + } + return data; + } + + /** + * Retrieves the byte at a specific offset + * from the underlying data + * + * @param offset The (zero-based) offset of the byte to retrieve + * + * @return the requested byte + */ + public byte get(final int offset) { + return get0(getNativeHandle(), offset); + } + + @Override + public void clear() { + clear0(getNativeHandle(), !cleared && internalBuffer, internalBufferOffset); + cleared = true; + } + + @Override + public void removePrefix(final int n) { + removePrefix0(getNativeHandle(), n); + this.internalBufferOffset += n; + } + + @Override + protected void disposeInternal() { + final long nativeHandle = getNativeHandle(); + if(!cleared && internalBuffer) { + disposeInternalBuf(nativeHandle, internalBufferOffset); + } + disposeInternal(nativeHandle); + } + + private native static long createNewDirectSlice0(final ByteBuffer data, + final int length); + private native static long createNewDirectSlice1(final ByteBuffer data); + @Override protected final native ByteBuffer data0(long handle); + private native byte get0(long handle, int offset); + private native void clear0(long handle, boolean internalBuffer, + long internalBufferOffset); + private native void removePrefix0(long handle, int length); + private native void disposeInternalBuf(final long handle, + long internalBufferOffset); +} diff --git a/java/src/main/java/org/rocksdb/EncodingType.java b/java/src/main/java/org/rocksdb/EncodingType.java new file mode 100644 index 00000000000..5ceeb54c826 --- /dev/null +++ b/java/src/main/java/org/rocksdb/EncodingType.java @@ -0,0 +1,55 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * EncodingType + * + *

    The value will determine how to encode keys + * when writing to a new SST file.

    + * + *

    This value will be stored + * inside the SST file which will be used when reading from + * the file, which makes it possible for users to choose + * different encoding type when reopening a DB. Files with + * different encoding types can co-exist in the same DB and + * can be read.

    + */ +public enum EncodingType { + /** + * Always write full keys without any special encoding. + */ + kPlain((byte) 0), + /** + *

    Find opportunity to write the same prefix once for multiple rows. + * In some cases, when a key follows a previous key with the same prefix, + * instead of writing out the full key, it just writes out the size of the + * shared prefix, as well as other bytes, to save some bytes.

    + * + *

    When using this option, the user is required to use the same prefix + * extractor to make sure the same prefix will be extracted from the same key. + * The Name() value of the prefix extractor will be stored in the file. When + * reopening the file, the name of the options.prefix_extractor given will be + * bitwise compared to the prefix extractors stored in the file. An error + * will be returned if the two don't match.

    + */ + kPrefix((byte) 1); + + /** + * Returns the byte value of the enumerations value + * + * @return byte representation + */ + public byte getValue() { + return value_; + } + + private EncodingType(byte value) { + value_ = value; + } + + private final byte value_; +} diff --git a/java/src/main/java/org/rocksdb/Env.java b/java/src/main/java/org/rocksdb/Env.java new file mode 100644 index 00000000000..a46f06178dd --- /dev/null +++ b/java/src/main/java/org/rocksdb/Env.java @@ -0,0 +1,92 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Base class for all Env implementations in RocksDB. + */ +public abstract class Env extends RocksObject { + public static final int FLUSH_POOL = 0; + public static final int COMPACTION_POOL = 1; + + /** + *

    Returns the default environment suitable for the current operating + * system.

    + * + *

    The result of {@code getDefault()} is a singleton whose ownership + * belongs to rocksdb c++. As a result, the returned RocksEnv will not + * have the ownership of its c++ resource, and calling its dispose() + * will be no-op.

    + * + * @return the default {@link org.rocksdb.RocksEnv} instance. + */ + public static Env getDefault() { + return default_env_; + } + + /** + *

    Sets the number of background worker threads of the flush pool + * for this environment.

    + *

    Default number: 1

    + * + * @param num the number of threads + * + * @return current {@link RocksEnv} instance. + */ + public Env setBackgroundThreads(final int num) { + return setBackgroundThreads(num, FLUSH_POOL); + } + + /** + *

    Sets the number of background worker threads of the specified thread + * pool for this environment.

    + * + * @param num the number of threads + * @param poolID the id to specified a thread pool. Should be either + * FLUSH_POOL or COMPACTION_POOL. + * + *

    Default number: 1

    + * @return current {@link RocksEnv} instance. + */ + public Env setBackgroundThreads(final int num, final int poolID) { + setBackgroundThreads(nativeHandle_, num, poolID); + return this; + } + + /** + *

    Returns the length of the queue associated with the specified + * thread pool.

    + * + * @param poolID the id to specified a thread pool. Should be either + * FLUSH_POOL or COMPACTION_POOL. + * + * @return the thread pool queue length. + */ + public int getThreadPoolQueueLen(final int poolID) { + return getThreadPoolQueueLen(nativeHandle_, poolID); + } + + + protected Env(final long nativeHandle) { + super(nativeHandle); + } + + static { + default_env_ = new RocksEnv(getDefaultEnvInternal()); + } + + /** + *

    The static default Env. The ownership of its native handle + * belongs to rocksdb c++ and is not able to be released on the Java + * side.

    + */ + static Env default_env_; + + private static native long getDefaultEnvInternal(); + private native void setBackgroundThreads( + long handle, int num, int priority); + private native int getThreadPoolQueueLen(long handle, int poolID); +} diff --git a/java/src/main/java/org/rocksdb/EnvOptions.java b/java/src/main/java/org/rocksdb/EnvOptions.java new file mode 100644 index 00000000000..2bca0355e43 --- /dev/null +++ b/java/src/main/java/org/rocksdb/EnvOptions.java @@ -0,0 +1,207 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +public class EnvOptions extends RocksObject { + static { + RocksDB.loadLibrary(); + } + + public EnvOptions() { + super(newEnvOptions()); + } + + public EnvOptions setUseOsBuffer(final boolean useOsBuffer) { + setUseOsBuffer(nativeHandle_, useOsBuffer); + return this; + } + + public boolean useOsBuffer() { + assert(isOwningHandle()); + return useOsBuffer(nativeHandle_); + } + + public EnvOptions setUseMmapReads(final boolean useMmapReads) { + setUseMmapReads(nativeHandle_, useMmapReads); + return this; + } + + public boolean useMmapReads() { + assert(isOwningHandle()); + return useMmapReads(nativeHandle_); + } + + public EnvOptions setUseMmapWrites(final boolean useMmapWrites) { + setUseMmapWrites(nativeHandle_, useMmapWrites); + return this; + } + + public boolean useMmapWrites() { + assert(isOwningHandle()); + return useMmapWrites(nativeHandle_); + } + + public EnvOptions setUseDirectReads(final boolean useDirectReads) { + setUseDirectReads(nativeHandle_, useDirectReads); + return this; + } + + public boolean useDirectReads() { + assert(isOwningHandle()); + return useDirectReads(nativeHandle_); + } + + public EnvOptions setUseDirectWrites(final boolean useDirectWrites) { + setUseDirectWrites(nativeHandle_, useDirectWrites); + return this; + } + + public boolean useDirectWrites() { + assert(isOwningHandle()); + return useDirectWrites(nativeHandle_); + } + + public EnvOptions setAllowFallocate(final boolean allowFallocate) { + setAllowFallocate(nativeHandle_, allowFallocate); + return this; + } + + public boolean allowFallocate() { + assert(isOwningHandle()); + return allowFallocate(nativeHandle_); + } + + public EnvOptions setSetFdCloexec(final boolean setFdCloexec) { + setSetFdCloexec(nativeHandle_, setFdCloexec); + return this; + } + + public boolean setFdCloexec() { + assert(isOwningHandle()); + return setFdCloexec(nativeHandle_); + } + + public EnvOptions setBytesPerSync(final long bytesPerSync) { + setBytesPerSync(nativeHandle_, bytesPerSync); + return this; + } + + public long bytesPerSync() { + assert(isOwningHandle()); + return bytesPerSync(nativeHandle_); + } + + public EnvOptions setFallocateWithKeepSize(final boolean fallocateWithKeepSize) { + setFallocateWithKeepSize(nativeHandle_, fallocateWithKeepSize); + return this; + } + + public boolean fallocateWithKeepSize() { + assert(isOwningHandle()); + return fallocateWithKeepSize(nativeHandle_); + } + + public EnvOptions setCompactionReadaheadSize(final long compactionReadaheadSize) { + setCompactionReadaheadSize(nativeHandle_, compactionReadaheadSize); + return this; + } + + public long compactionReadaheadSize() { + assert(isOwningHandle()); + return compactionReadaheadSize(nativeHandle_); + } + + public EnvOptions setRandomAccessMaxBufferSize(final long randomAccessMaxBufferSize) { + setRandomAccessMaxBufferSize(nativeHandle_, randomAccessMaxBufferSize); + return this; + } + + public long randomAccessMaxBufferSize() { + assert(isOwningHandle()); + return randomAccessMaxBufferSize(nativeHandle_); + } + + public EnvOptions setWritableFileMaxBufferSize(final long writableFileMaxBufferSize) { + setWritableFileMaxBufferSize(nativeHandle_, writableFileMaxBufferSize); + return this; + } + + public long writableFileMaxBufferSize() { + assert(isOwningHandle()); + return writableFileMaxBufferSize(nativeHandle_); + } + + public EnvOptions setRateLimiter(final RateLimiter rateLimiter) { + this.rateLimiter = rateLimiter; + setRateLimiter(nativeHandle_, rateLimiter.nativeHandle_); + return this; + } + + public RateLimiter rateLimiter() { + assert(isOwningHandle()); + return rateLimiter; + } + + private native static long newEnvOptions(); + + @Override protected final native void disposeInternal(final long handle); + + private native void setUseOsBuffer(final long handle, final boolean useOsBuffer); + + private native boolean useOsBuffer(final long handle); + + private native void setUseMmapReads(final long handle, final boolean useMmapReads); + + private native boolean useMmapReads(final long handle); + + private native void setUseMmapWrites(final long handle, final boolean useMmapWrites); + + private native boolean useMmapWrites(final long handle); + + private native void setUseDirectReads(final long handle, final boolean useDirectReads); + + private native boolean useDirectReads(final long handle); + + private native void setUseDirectWrites(final long handle, final boolean useDirectWrites); + + private native boolean useDirectWrites(final long handle); + + private native void setAllowFallocate(final long handle, final boolean allowFallocate); + + private native boolean allowFallocate(final long handle); + + private native void setSetFdCloexec(final long handle, final boolean setFdCloexec); + + private native boolean setFdCloexec(final long handle); + + private native void setBytesPerSync(final long handle, final long bytesPerSync); + + private native long bytesPerSync(final long handle); + + private native void setFallocateWithKeepSize( + final long handle, final boolean fallocateWithKeepSize); + + private native boolean fallocateWithKeepSize(final long handle); + + private native void setCompactionReadaheadSize( + final long handle, final long compactionReadaheadSize); + + private native long compactionReadaheadSize(final long handle); + + private native void setRandomAccessMaxBufferSize( + final long handle, final long randomAccessMaxBufferSize); + + private native long randomAccessMaxBufferSize(final long handle); + + private native void setWritableFileMaxBufferSize( + final long handle, final long writableFileMaxBufferSize); + + private native long writableFileMaxBufferSize(final long handle); + + private native void setRateLimiter(final long handle, final long rateLimiterHandle); + + private RateLimiter rateLimiter; +} diff --git a/java/src/main/java/org/rocksdb/Experimental.java b/java/src/main/java/org/rocksdb/Experimental.java new file mode 100644 index 00000000000..64b404d6f19 --- /dev/null +++ b/java/src/main/java/org/rocksdb/Experimental.java @@ -0,0 +1,23 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Marks a feature as experimental, meaning that it is likely + * to change or even be removed/re-engineered in the future + */ +@Documented +@Retention(RetentionPolicy.SOURCE) +@Target({ElementType.TYPE, ElementType.METHOD}) +public @interface Experimental { + String value(); +} diff --git a/java/org/rocksdb/Filter.java b/java/src/main/java/org/rocksdb/Filter.java similarity index 57% rename from java/org/rocksdb/Filter.java rename to java/src/main/java/org/rocksdb/Filter.java index ce5c41f26d7..011be208561 100644 --- a/java/org/rocksdb/Filter.java +++ b/java/src/main/java/org/rocksdb/Filter.java @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). package org.rocksdb; @@ -13,7 +13,10 @@ * DB::Get() call. */ public abstract class Filter extends RocksObject { - protected abstract void createNewFilter(); + + protected Filter(final long nativeHandle) { + super(nativeHandle); + } /** * Deletes underlying C++ filter pointer. @@ -22,10 +25,11 @@ public abstract class Filter extends RocksObject { * RocksDB instances referencing the filter are closed. * Otherwise an undefined behavior will occur. */ - @Override protected void disposeInternal() { - assert(isInitialized()); + @Override + protected void disposeInternal() { disposeInternal(nativeHandle_); } - private native void disposeInternal(long handle); + @Override + protected final native void disposeInternal(final long handle); } diff --git a/java/src/main/java/org/rocksdb/FlushOptions.java b/java/src/main/java/org/rocksdb/FlushOptions.java new file mode 100644 index 00000000000..ce54a528bfe --- /dev/null +++ b/java/src/main/java/org/rocksdb/FlushOptions.java @@ -0,0 +1,49 @@ +package org.rocksdb; + +/** + * FlushOptions to be passed to flush operations of + * {@link org.rocksdb.RocksDB}. + */ +public class FlushOptions extends RocksObject { + static { + RocksDB.loadLibrary(); + } + + /** + * Construct a new instance of FlushOptions. + */ + public FlushOptions(){ + super(newFlushOptions()); + } + + /** + * Set if the flush operation shall block until it terminates. + * + * @param waitForFlush boolean value indicating if the flush + * operations waits for termination of the flush process. + * + * @return instance of current FlushOptions. + */ + public FlushOptions setWaitForFlush(final boolean waitForFlush) { + assert(isOwningHandle()); + setWaitForFlush(nativeHandle_, waitForFlush); + return this; + } + + /** + * Wait for flush to finished. + * + * @return boolean value indicating if the flush operation + * waits for termination of the flush process. + */ + public boolean waitForFlush() { + assert(isOwningHandle()); + return waitForFlush(nativeHandle_); + } + + private native static long newFlushOptions(); + @Override protected final native void disposeInternal(final long handle); + private native void setWaitForFlush(long handle, + boolean wait); + private native boolean waitForFlush(long handle); +} diff --git a/java/src/main/java/org/rocksdb/HashLinkedListMemTableConfig.java b/java/src/main/java/org/rocksdb/HashLinkedListMemTableConfig.java new file mode 100644 index 00000000000..d56c46c290c --- /dev/null +++ b/java/src/main/java/org/rocksdb/HashLinkedListMemTableConfig.java @@ -0,0 +1,173 @@ +package org.rocksdb; + +/** + * The config for hash linked list memtable representation + * Such memtable contains a fix-sized array of buckets, where + * each bucket points to a sorted singly-linked + * list (or null if the bucket is empty). + * + * Note that since this mem-table representation relies on the + * key prefix, it is required to invoke one of the usePrefixExtractor + * functions to specify how to extract key prefix given a key. + * If proper prefix-extractor is not set, then RocksDB will + * use the default memtable representation (SkipList) instead + * and post a warning in the LOG. + */ +public class HashLinkedListMemTableConfig extends MemTableConfig { + public static final long DEFAULT_BUCKET_COUNT = 50000; + public static final long DEFAULT_HUGE_PAGE_TLB_SIZE = 0; + public static final int DEFAULT_BUCKET_ENTRIES_LOG_THRES = 4096; + public static final boolean + DEFAULT_IF_LOG_BUCKET_DIST_WHEN_FLUSH = true; + public static final int DEFAUL_THRESHOLD_USE_SKIPLIST = 256; + + /** + * HashLinkedListMemTableConfig constructor + */ + public HashLinkedListMemTableConfig() { + bucketCount_ = DEFAULT_BUCKET_COUNT; + hugePageTlbSize_ = DEFAULT_HUGE_PAGE_TLB_SIZE; + bucketEntriesLoggingThreshold_ = DEFAULT_BUCKET_ENTRIES_LOG_THRES; + ifLogBucketDistWhenFlush_ = DEFAULT_IF_LOG_BUCKET_DIST_WHEN_FLUSH; + thresholdUseSkiplist_ = DEFAUL_THRESHOLD_USE_SKIPLIST; + } + + /** + * Set the number of buckets in the fixed-size array used + * in the hash linked-list mem-table. + * + * @param count the number of hash buckets. + * @return the reference to the current HashLinkedListMemTableConfig. + */ + public HashLinkedListMemTableConfig setBucketCount( + final long count) { + bucketCount_ = count; + return this; + } + + /** + * Returns the number of buckets that will be used in the memtable + * created based on this config. + * + * @return the number of buckets + */ + public long bucketCount() { + return bucketCount_; + } + + /** + *

    Set the size of huge tlb or allocate the hashtable bytes from + * malloc if {@code size <= 0}.

    + * + *

    The user needs to reserve huge pages for it to be allocated, + * like: {@code sysctl -w vm.nr_hugepages=20}

    + * + *

    See linux documentation/vm/hugetlbpage.txt

    + * + * @param size if set to {@code <= 0} hashtable bytes from malloc + * @return the reference to the current HashLinkedListMemTableConfig. + */ + public HashLinkedListMemTableConfig setHugePageTlbSize( + final long size) { + hugePageTlbSize_ = size; + return this; + } + + /** + * Returns the size value of hugePageTlbSize. + * + * @return the hugePageTlbSize. + */ + public long hugePageTlbSize() { + return hugePageTlbSize_; + } + + /** + * If number of entries in one bucket exceeds that setting, log + * about it. + * + * @param threshold - number of entries in a single bucket before + * logging starts. + * @return the reference to the current HashLinkedListMemTableConfig. + */ + public HashLinkedListMemTableConfig + setBucketEntriesLoggingThreshold(final int threshold) { + bucketEntriesLoggingThreshold_ = threshold; + return this; + } + + /** + * Returns the maximum number of entries in one bucket before + * logging starts. + * + * @return maximum number of entries in one bucket before logging + * starts. + */ + public int bucketEntriesLoggingThreshold() { + return bucketEntriesLoggingThreshold_; + } + + /** + * If true the distrubition of number of entries will be logged. + * + * @param logDistribution - boolean parameter indicating if number + * of entry distribution shall be logged. + * @return the reference to the current HashLinkedListMemTableConfig. + */ + public HashLinkedListMemTableConfig + setIfLogBucketDistWhenFlush(final boolean logDistribution) { + ifLogBucketDistWhenFlush_ = logDistribution; + return this; + } + + /** + * Returns information about logging the distribution of + * number of entries on flush. + * + * @return if distrubtion of number of entries shall be logged. + */ + public boolean ifLogBucketDistWhenFlush() { + return ifLogBucketDistWhenFlush_; + } + + /** + * Set maximum number of entries in one bucket. Exceeding this val + * leads to a switch from LinkedList to SkipList. + * + * @param threshold maximum number of entries before SkipList is + * used. + * @return the reference to the current HashLinkedListMemTableConfig. + */ + public HashLinkedListMemTableConfig + setThresholdUseSkiplist(final int threshold) { + thresholdUseSkiplist_ = threshold; + return this; + } + + /** + * Returns entries per bucket threshold before LinkedList is + * replaced by SkipList usage for that bucket. + * + * @return entries per bucket threshold before SkipList is used. + */ + public int thresholdUseSkiplist() { + return thresholdUseSkiplist_; + } + + @Override protected long newMemTableFactoryHandle() { + return newMemTableFactoryHandle(bucketCount_, hugePageTlbSize_, + bucketEntriesLoggingThreshold_, ifLogBucketDistWhenFlush_, + thresholdUseSkiplist_); + } + + private native long newMemTableFactoryHandle(long bucketCount, + long hugePageTlbSize, int bucketEntriesLoggingThreshold, + boolean ifLogBucketDistWhenFlush, int thresholdUseSkiplist) + throws IllegalArgumentException; + + private long bucketCount_; + private long hugePageTlbSize_; + private int bucketEntriesLoggingThreshold_; + private boolean ifLogBucketDistWhenFlush_; + private int thresholdUseSkiplist_; +} diff --git a/java/org/rocksdb/HashSkipListMemTableConfig.java b/java/src/main/java/org/rocksdb/HashSkipListMemTableConfig.java similarity index 86% rename from java/org/rocksdb/HashSkipListMemTableConfig.java rename to java/src/main/java/org/rocksdb/HashSkipListMemTableConfig.java index 74fb0dba2d5..fe1779b1cf5 100644 --- a/java/org/rocksdb/HashSkipListMemTableConfig.java +++ b/java/src/main/java/org/rocksdb/HashSkipListMemTableConfig.java @@ -18,6 +18,9 @@ public class HashSkipListMemTableConfig extends MemTableConfig { public static final int DEFAULT_BRANCHING_FACTOR = 4; public static final int DEFAULT_HEIGHT = 4; + /** + * HashSkipListMemTableConfig constructor + */ public HashSkipListMemTableConfig() { bucketCount_ = DEFAULT_BUCKET_COUNT; branchingFactor_ = DEFAULT_BRANCHING_FACTOR; @@ -32,7 +35,8 @@ public HashSkipListMemTableConfig() { * skiplist memtable. * @return the reference to the current HashSkipListMemTableConfig. */ - public HashSkipListMemTableConfig setBucketCount(long count) { + public HashSkipListMemTableConfig setBucketCount( + final long count) { bucketCount_ = count; return this; } @@ -47,9 +51,11 @@ public long bucketCount() { /** * Set the height of the skip list. Default = 4. * + * @param height height to set. + * * @return the reference to the current HashSkipListMemTableConfig. */ - public HashSkipListMemTableConfig setHeight(int height) { + public HashSkipListMemTableConfig setHeight(final int height) { height_ = height; return this; } @@ -70,7 +76,8 @@ public int height() { * lists in the skip list. * @return the reference to the current HashSkipListMemTableConfig. */ - public HashSkipListMemTableConfig setBranchingFactor(int bf) { + public HashSkipListMemTableConfig setBranchingFactor( + final int bf) { branchingFactor_ = bf; return this; } @@ -89,7 +96,8 @@ public int branchingFactor() { } private native long newMemTableFactoryHandle( - long bucketCount, int height, int branchingFactor); + long bucketCount, int height, int branchingFactor) + throws IllegalArgumentException; private long bucketCount_; private int branchingFactor_; diff --git a/java/org/rocksdb/HistogramData.java b/java/src/main/java/org/rocksdb/HistogramData.java similarity index 62% rename from java/org/rocksdb/HistogramData.java rename to java/src/main/java/org/rocksdb/HistogramData.java index 3b2e2959977..11798eb59f2 100644 --- a/java/org/rocksdb/HistogramData.java +++ b/java/src/main/java/org/rocksdb/HistogramData.java @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). package org.rocksdb; @@ -12,8 +12,9 @@ public class HistogramData { private final double average_; private final double standardDeviation_; - public HistogramData(double median, double percentile95, - double percentile99, double average, double standardDeviation) { + public HistogramData(final double median, final double percentile95, + final double percentile99, final double average, + final double standardDeviation) { median_ = median; percentile95_ = percentile95; percentile99_ = percentile99; diff --git a/java/src/main/java/org/rocksdb/HistogramType.java b/java/src/main/java/org/rocksdb/HistogramType.java new file mode 100644 index 00000000000..2d95f5149f5 --- /dev/null +++ b/java/src/main/java/org/rocksdb/HistogramType.java @@ -0,0 +1,98 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +public enum HistogramType { + + DB_GET((byte) 0x0), + + DB_WRITE((byte) 0x1), + + COMPACTION_TIME((byte) 0x2), + + SUBCOMPACTION_SETUP_TIME((byte) 0x3), + + TABLE_SYNC_MICROS((byte) 0x4), + + COMPACTION_OUTFILE_SYNC_MICROS((byte) 0x5), + + WAL_FILE_SYNC_MICROS((byte) 0x6), + + MANIFEST_FILE_SYNC_MICROS((byte) 0x7), + + /** + * TIME SPENT IN IO DURING TABLE OPEN. + */ + TABLE_OPEN_IO_MICROS((byte) 0x8), + + DB_MULTIGET((byte) 0x9), + + READ_BLOCK_COMPACTION_MICROS((byte) 0xA), + + READ_BLOCK_GET_MICROS((byte) 0xB), + + WRITE_RAW_BLOCK_MICROS((byte) 0xC), + + STALL_L0_SLOWDOWN_COUNT((byte) 0xD), + + STALL_MEMTABLE_COMPACTION_COUNT((byte) 0xE), + + STALL_L0_NUM_FILES_COUNT((byte) 0xF), + + HARD_RATE_LIMIT_DELAY_COUNT((byte) 0x10), + + SOFT_RATE_LIMIT_DELAY_COUNT((byte) 0x11), + + NUM_FILES_IN_SINGLE_COMPACTION((byte) 0x12), + + DB_SEEK((byte) 0x13), + + WRITE_STALL((byte) 0x14), + + SST_READ_MICROS((byte) 0x15), + + /** + * The number of subcompactions actually scheduled during a compaction. + */ + NUM_SUBCOMPACTIONS_SCHEDULED((byte) 0x16), + + /** + * Value size distribution in each operation. + */ + BYTES_PER_READ((byte) 0x17), + BYTES_PER_WRITE((byte) 0x18), + BYTES_PER_MULTIGET((byte) 0x19), + + /** + * number of bytes compressed. + */ + BYTES_COMPRESSED((byte) 0x1A), + + /** + * number of bytes decompressed. + * + * number of bytes is when uncompressed; i.e. before/after respectively + */ + BYTES_DECOMPRESSED((byte) 0x1B), + + COMPRESSION_TIMES_NANOS((byte) 0x1C), + + DECOMPRESSION_TIMES_NANOS((byte) 0x1D), + + READ_NUM_MERGE_OPERANDS((byte) 0x1E), + + HISTOGRAM_ENUM_MAX((byte) 0x1F); + + private final byte value; + + HistogramType(final byte value) { + this.value = value; + } + + public byte getValue() { + return value; + } +} diff --git a/java/src/main/java/org/rocksdb/IndexType.java b/java/src/main/java/org/rocksdb/IndexType.java new file mode 100644 index 00000000000..e0c113d39ac --- /dev/null +++ b/java/src/main/java/org/rocksdb/IndexType.java @@ -0,0 +1,41 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * IndexType used in conjunction with BlockBasedTable. + */ +public enum IndexType { + /** + * A space efficient index block that is optimized for + * binary-search-based index. + */ + kBinarySearch((byte) 0), + /** + * The hash index, if enabled, will do the hash lookup when + * {@code Options.prefix_extractor} is provided. + */ + kHashSearch((byte) 1), + /** + * A two-level index implementation. Both levels are binary search indexes. + */ + kTwoLevelIndexSearch((byte) 2); + + /** + * Returns the byte value of the enumerations value + * + * @return byte representation + */ + public byte getValue() { + return value_; + } + + private IndexType(byte value) { + value_ = value; + } + + private final byte value_; +} diff --git a/java/src/main/java/org/rocksdb/InfoLogLevel.java b/java/src/main/java/org/rocksdb/InfoLogLevel.java new file mode 100644 index 00000000000..2c97991b5b4 --- /dev/null +++ b/java/src/main/java/org/rocksdb/InfoLogLevel.java @@ -0,0 +1,48 @@ +package org.rocksdb; + +/** + * RocksDB log levels. + */ +public enum InfoLogLevel { + DEBUG_LEVEL((byte)0), + INFO_LEVEL((byte)1), + WARN_LEVEL((byte)2), + ERROR_LEVEL((byte)3), + FATAL_LEVEL((byte)4), + HEADER_LEVEL((byte)5), + NUM_INFO_LOG_LEVELS((byte)6); + + private final byte value_; + + private InfoLogLevel(final byte value) { + value_ = value; + } + + /** + * Returns the byte value of the enumerations value + * + * @return byte representation + */ + public byte getValue() { + return value_; + } + + /** + * Get InfoLogLevel by byte value. + * + * @param value byte representation of InfoLogLevel. + * + * @return {@link org.rocksdb.InfoLogLevel} instance. + * @throws java.lang.IllegalArgumentException if an invalid + * value is provided. + */ + public static InfoLogLevel getInfoLogLevel(final byte value) { + for (final InfoLogLevel infoLogLevel : InfoLogLevel.values()) { + if (infoLogLevel.getValue() == value){ + return infoLogLevel; + } + } + throw new IllegalArgumentException( + "Illegal value provided for InfoLogLevel."); + } +} diff --git a/java/src/main/java/org/rocksdb/IngestExternalFileOptions.java b/java/src/main/java/org/rocksdb/IngestExternalFileOptions.java new file mode 100644 index 00000000000..7343691817b --- /dev/null +++ b/java/src/main/java/org/rocksdb/IngestExternalFileOptions.java @@ -0,0 +1,125 @@ +package org.rocksdb; +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +import java.util.List; + +/** + * IngestExternalFileOptions is used by {@link RocksDB#ingestExternalFile(ColumnFamilyHandle, List, IngestExternalFileOptions)} + */ +public class IngestExternalFileOptions extends RocksObject { + + public IngestExternalFileOptions() { + super(newIngestExternalFileOptions()); + } + + /** + * @param moveFiles {@link #setMoveFiles(boolean)} + * @param snapshotConsistency {@link #setSnapshotConsistency(boolean)} + * @param allowGlobalSeqNo {@link #setAllowGlobalSeqNo(boolean)} + * @param allowBlockingFlush {@link #setAllowBlockingFlush(boolean)} + */ + public IngestExternalFileOptions(final boolean moveFiles, + final boolean snapshotConsistency, final boolean allowGlobalSeqNo, + final boolean allowBlockingFlush) { + super(newIngestExternalFileOptions(moveFiles, snapshotConsistency, + allowGlobalSeqNo, allowBlockingFlush)); + } + + /** + * Can be set to true to move the files instead of copying them. + * + * @return true if files will be moved + */ + public boolean moveFiles() { + return moveFiles(nativeHandle_); + } + + /** + * Can be set to true to move the files instead of copying them. + * + * @param moveFiles true if files should be moved instead of copied + */ + public void setMoveFiles(final boolean moveFiles) { + setMoveFiles(nativeHandle_, moveFiles); + } + + /** + * If set to false, an ingested file keys could appear in existing snapshots + * that where created before the file was ingested. + * + * @return true if snapshot consistency is assured + */ + public boolean snapshotConsistency() { + return snapshotConsistency(nativeHandle_); + } + + /** + * If set to false, an ingested file keys could appear in existing snapshots + * that where created before the file was ingested. + * + * @param snapshotConsistency true if snapshot consistency is required + */ + public void setSnapshotConsistency(final boolean snapshotConsistency) { + setSnapshotConsistency(nativeHandle_, snapshotConsistency); + } + + /** + * If set to false, {@link RocksDB#ingestExternalFile(ColumnFamilyHandle, List, IngestExternalFileOptions)} + * will fail if the file key range overlaps with existing keys or tombstones in the DB. + * + * @return true if global seq numbers are assured + */ + public boolean allowGlobalSeqNo() { + return allowGlobalSeqNo(nativeHandle_); + } + + /** + * If set to false, {@link RocksDB#ingestExternalFile(ColumnFamilyHandle, List, IngestExternalFileOptions)} + * will fail if the file key range overlaps with existing keys or tombstones in the DB. + * + * @param allowGlobalSeqNo true if global seq numbers are required + */ + public void setAllowGlobalSeqNo(final boolean allowGlobalSeqNo) { + setAllowGlobalSeqNo(nativeHandle_, allowGlobalSeqNo); + } + + /** + * If set to false and the file key range overlaps with the memtable key range + * (memtable flush required), IngestExternalFile will fail. + * + * @return true if blocking flushes may occur + */ + public boolean allowBlockingFlush() { + return allowBlockingFlush(nativeHandle_); + } + + /** + * If set to false and the file key range overlaps with the memtable key range + * (memtable flush required), IngestExternalFile will fail. + * + * @param allowBlockingFlush true if blocking flushes are allowed + */ + public void setAllowBlockingFlush(final boolean allowBlockingFlush) { + setAllowBlockingFlush(nativeHandle_, allowBlockingFlush); + } + + private native static long newIngestExternalFileOptions(); + private native static long newIngestExternalFileOptions( + final boolean moveFiles, final boolean snapshotConsistency, + final boolean allowGlobalSeqNo, final boolean allowBlockingFlush); + private native boolean moveFiles(final long handle); + private native void setMoveFiles(final long handle, final boolean move_files); + private native boolean snapshotConsistency(final long handle); + private native void setSnapshotConsistency(final long handle, + final boolean snapshotConsistency); + private native boolean allowGlobalSeqNo(final long handle); + private native void setAllowGlobalSeqNo(final long handle, + final boolean allowGloablSeqNo); + private native boolean allowBlockingFlush(final long handle); + private native void setAllowBlockingFlush(final long handle, + final boolean allowBlockingFlush); + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/LRUCache.java b/java/src/main/java/org/rocksdb/LRUCache.java new file mode 100644 index 00000000000..5e5bdeea277 --- /dev/null +++ b/java/src/main/java/org/rocksdb/LRUCache.java @@ -0,0 +1,82 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Least Recently Used Cache + */ +public class LRUCache extends Cache { + + /** + * Create a new cache with a fixed size capacity + * + * @param capacity The fixed size capacity of the cache + */ + public LRUCache(final long capacity) { + this(capacity, -1, false, 0.0); + } + + /** + * Create a new cache with a fixed size capacity. The cache is sharded + * to 2^numShardBits shards, by hash of the key. The total capacity + * is divided and evenly assigned to each shard. + * numShardBits = -1 means it is automatically determined: every shard + * will be at least 512KB and number of shard bits will not exceed 6. + * + * @param capacity The fixed size capacity of the cache + * @param numShardBits The cache is sharded to 2^numShardBits shards, + * by hash of the key + */ + public LRUCache(final long capacity, final int numShardBits) { + super(newLRUCache(capacity, numShardBits, false,0.0)); + } + + /** + * Create a new cache with a fixed size capacity. The cache is sharded + * to 2^numShardBits shards, by hash of the key. The total capacity + * is divided and evenly assigned to each shard. If strictCapacityLimit + * is set, insert to the cache will fail when cache is full. + * numShardBits = -1 means it is automatically determined: every shard + * will be at least 512KB and number of shard bits will not exceed 6. + * + * @param capacity The fixed size capacity of the cache + * @param numShardBits The cache is sharded to 2^numShardBits shards, + * by hash of the key + * @param strictCapacityLimit insert to the cache will fail when cache is full + */ + public LRUCache(final long capacity, final int numShardBits, + final boolean strictCapacityLimit) { + super(newLRUCache(capacity, numShardBits, strictCapacityLimit,0.0)); + } + + /** + * Create a new cache with a fixed size capacity. The cache is sharded + * to 2^numShardBits shards, by hash of the key. The total capacity + * is divided and evenly assigned to each shard. If strictCapacityLimit + * is set, insert to the cache will fail when cache is full. User can also + * set percentage of the cache reserves for high priority entries via + * highPriPoolRatio. + * numShardBits = -1 means it is automatically determined: every shard + * will be at least 512KB and number of shard bits will not exceed 6. + * + * @param capacity The fixed size capacity of the cache + * @param numShardBits The cache is sharded to 2^numShardBits shards, + * by hash of the key + * @param strictCapacityLimit insert to the cache will fail when cache is full + * @param highPriPoolRatio percentage of the cache reserves for high priority + * entries + */ + public LRUCache(final long capacity, final int numShardBits, + final boolean strictCapacityLimit, final double highPriPoolRatio) { + super(newLRUCache(capacity, numShardBits, strictCapacityLimit, + highPriPoolRatio)); + } + + private native static long newLRUCache(final long capacity, + final int numShardBits, final boolean strictCapacityLimit, + final double highPriPoolRatio); + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/Logger.java b/java/src/main/java/org/rocksdb/Logger.java new file mode 100644 index 00000000000..90212592908 --- /dev/null +++ b/java/src/main/java/org/rocksdb/Logger.java @@ -0,0 +1,111 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + *

    This class provides a custom logger functionality + * in Java which wraps {@code RocksDB} logging facilities. + *

    + * + *

    Using this class RocksDB can log with common + * Java logging APIs like Log4j or Slf4j without keeping + * database logs in the filesystem.

    + * + * Performance + *

    There are certain performance penalties using a Java + * {@code Logger} implementation within production code. + *

    + * + *

    + * A log level can be set using {@link org.rocksdb.Options} or + * {@link Logger#setInfoLogLevel(InfoLogLevel)}. The set log level + * influences the underlying native code. Each log message is + * checked against the set log level and if the log level is more + * verbose as the set log level, native allocations will be made + * and data structures are allocated. + *

    + * + *

    Every log message which will be emitted by native code will + * trigger expensive native to Java transitions. So the preferred + * setting for production use is either + * {@link org.rocksdb.InfoLogLevel#ERROR_LEVEL} or + * {@link org.rocksdb.InfoLogLevel#FATAL_LEVEL}. + *

    + */ +public abstract class Logger extends AbstractImmutableNativeReference { + + final long nativeHandle_; + + /** + *

    AbstractLogger constructor.

    + * + *

    Important: the log level set within + * the {@link org.rocksdb.Options} instance will be used as + * maximum log level of RocksDB.

    + * + * @param options {@link org.rocksdb.Options} instance. + */ + public Logger(final Options options) { + super(true); + this.nativeHandle_ = createNewLoggerOptions(options.nativeHandle_); + } + + /** + *

    AbstractLogger constructor.

    + * + *

    Important: the log level set within + * the {@link org.rocksdb.DBOptions} instance will be used + * as maximum log level of RocksDB.

    + * + * @param dboptions {@link org.rocksdb.DBOptions} instance. + */ + public Logger(final DBOptions dboptions) { + super(true); + this.nativeHandle_ = createNewLoggerDbOptions(dboptions.nativeHandle_); + } + + /** + * Set {@link org.rocksdb.InfoLogLevel} to AbstractLogger. + * + * @param infoLogLevel {@link org.rocksdb.InfoLogLevel} instance. + */ + public void setInfoLogLevel(final InfoLogLevel infoLogLevel) { + setInfoLogLevel(nativeHandle_, infoLogLevel.getValue()); + } + + /** + * Return the loggers log level. + * + * @return {@link org.rocksdb.InfoLogLevel} instance. + */ + public InfoLogLevel infoLogLevel() { + return InfoLogLevel.getInfoLogLevel( + infoLogLevel(nativeHandle_)); + } + + protected abstract void log(InfoLogLevel infoLogLevel, + String logMsg); + + /** + * Deletes underlying C++ slice pointer. + * Note that this function should be called only after all + * RocksDB instances referencing the slice are closed. + * Otherwise an undefined behavior will occur. + */ + @Override + protected void disposeInternal() { + disposeInternal(nativeHandle_); + } + + protected native long createNewLoggerOptions( + long options); + protected native long createNewLoggerDbOptions( + long dbOptions); + protected native void setInfoLogLevel(long handle, + byte infoLogLevel); + protected native byte infoLogLevel(long handle); + private native void disposeInternal(final long handle); +} diff --git a/java/org/rocksdb/MemTableConfig.java b/java/src/main/java/org/rocksdb/MemTableConfig.java similarity index 65% rename from java/org/rocksdb/MemTableConfig.java rename to java/src/main/java/org/rocksdb/MemTableConfig.java index a473c258568..83cee974a75 100644 --- a/java/org/rocksdb/MemTableConfig.java +++ b/java/src/main/java/org/rocksdb/MemTableConfig.java @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). package org.rocksdb; /** @@ -21,7 +21,9 @@ public abstract class MemTableConfig { * which will create a c++ shared-pointer to the c++ MemTableRepFactory * that associated with the Java MemTableConfig. * - * @see Options.setMemTableFactory() + * @see Options#setMemTableConfig(MemTableConfig) + * + * @return native handle address to native memory table instance. */ abstract protected long newMemTableFactoryHandle(); } diff --git a/java/src/main/java/org/rocksdb/MergeOperator.java b/java/src/main/java/org/rocksdb/MergeOperator.java new file mode 100644 index 00000000000..296527f53f9 --- /dev/null +++ b/java/src/main/java/org/rocksdb/MergeOperator.java @@ -0,0 +1,17 @@ +// Copyright (c) 2014, Vlad Balan (vlad.gm@gmail.com). All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * MergeOperator holds an operator to be applied when compacting + * two merge operands held under the same key in order to obtain a single + * value. + */ +public abstract class MergeOperator extends RocksObject { + protected MergeOperator(final long nativeHandle) { + super(nativeHandle); + } +} diff --git a/java/src/main/java/org/rocksdb/MutableColumnFamilyOptions.java b/java/src/main/java/org/rocksdb/MutableColumnFamilyOptions.java new file mode 100644 index 00000000000..3585318dbda --- /dev/null +++ b/java/src/main/java/org/rocksdb/MutableColumnFamilyOptions.java @@ -0,0 +1,997 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.*; + +public class MutableColumnFamilyOptions { + private final static String KEY_VALUE_PAIR_SEPARATOR = ";"; + private final static char KEY_VALUE_SEPARATOR = '='; + private final static String INT_ARRAY_INT_SEPARATOR = ","; + + private final String[] keys; + private final String[] values; + + // user must use builder pattern, or parser + private MutableColumnFamilyOptions(final String keys[], + final String values[]) { + this.keys = keys; + this.values = values; + } + + String[] getKeys() { + return keys; + } + + String[] getValues() { + return values; + } + + /** + * Creates a builder which allows you + * to set MutableColumnFamilyOptions in a fluent + * manner + * + * @return A builder for MutableColumnFamilyOptions + */ + public static MutableColumnFamilyOptionsBuilder builder() { + return new MutableColumnFamilyOptionsBuilder(); + } + + /** + * Parses a String representation of MutableColumnFamilyOptions + * + * The format is: key1=value1;key2=value2;key3=value3 etc + * + * For int[] values, each int should be separated by a comma, e.g. + * + * key1=value1;intArrayKey1=1,2,3 + * + * @param str The string representation of the mutable column family options + * + * @return A builder for the mutable column family options + */ + public static MutableColumnFamilyOptionsBuilder parse(final String str) { + Objects.requireNonNull(str); + + final MutableColumnFamilyOptionsBuilder builder = + new MutableColumnFamilyOptionsBuilder(); + + final String options[] = str.trim().split(KEY_VALUE_PAIR_SEPARATOR); + for(final String option : options) { + final int equalsOffset = option.indexOf(KEY_VALUE_SEPARATOR); + if(equalsOffset <= 0) { + throw new IllegalArgumentException( + "options string has an invalid key=value pair"); + } + + final String key = option.substring(0, equalsOffset); + if(key == null || key.isEmpty()) { + throw new IllegalArgumentException("options string is invalid"); + } + + final String value = option.substring(equalsOffset + 1); + if(value == null || value.isEmpty()) { + throw new IllegalArgumentException("options string is invalid"); + } + + builder.fromString(key, value); + } + + return builder; + } + + /** + * Returns a string representation + * of MutableColumnFamilyOptions which is + * suitable for consumption by {@link #parse(String)} + * + * @return String representation of MutableColumnFamilyOptions + */ + @Override + public String toString() { + final StringBuilder buffer = new StringBuilder(); + for(int i = 0; i < keys.length; i++) { + buffer + .append(keys[i]) + .append(KEY_VALUE_SEPARATOR) + .append(values[i]); + + if(i + 1 < keys.length) { + buffer.append(KEY_VALUE_PAIR_SEPARATOR); + } + } + return buffer.toString(); + } + + public enum ValueType { + DOUBLE, + LONG, + INT, + BOOLEAN, + INT_ARRAY, + ENUM + } + + public enum MemtableOption implements MutableColumnFamilyOptionKey { + write_buffer_size(ValueType.LONG), + arena_block_size(ValueType.LONG), + memtable_prefix_bloom_size_ratio(ValueType.DOUBLE), + @Deprecated memtable_prefix_bloom_bits(ValueType.INT), + @Deprecated memtable_prefix_bloom_probes(ValueType.INT), + memtable_huge_page_size(ValueType.LONG), + max_successive_merges(ValueType.LONG), + @Deprecated filter_deletes(ValueType.BOOLEAN), + max_write_buffer_number(ValueType.INT), + inplace_update_num_locks(ValueType.LONG); + + private final ValueType valueType; + MemtableOption(final ValueType valueType) { + this.valueType = valueType; + } + + @Override + public ValueType getValueType() { + return valueType; + } + } + + public enum CompactionOption implements MutableColumnFamilyOptionKey { + disable_auto_compactions(ValueType.BOOLEAN), + @Deprecated soft_rate_limit(ValueType.DOUBLE), + soft_pending_compaction_bytes_limit(ValueType.LONG), + @Deprecated hard_rate_limit(ValueType.DOUBLE), + hard_pending_compaction_bytes_limit(ValueType.LONG), + level0_file_num_compaction_trigger(ValueType.INT), + level0_slowdown_writes_trigger(ValueType.INT), + level0_stop_writes_trigger(ValueType.INT), + max_compaction_bytes(ValueType.LONG), + target_file_size_base(ValueType.LONG), + target_file_size_multiplier(ValueType.INT), + max_bytes_for_level_base(ValueType.LONG), + max_bytes_for_level_multiplier(ValueType.INT), + max_bytes_for_level_multiplier_additional(ValueType.INT_ARRAY); + + private final ValueType valueType; + CompactionOption(final ValueType valueType) { + this.valueType = valueType; + } + + @Override + public ValueType getValueType() { + return valueType; + } + } + + public enum MiscOption implements MutableColumnFamilyOptionKey { + max_sequential_skip_in_iterations(ValueType.LONG), + paranoid_file_checks(ValueType.BOOLEAN), + report_bg_io_stats(ValueType.BOOLEAN), + compression_type(ValueType.ENUM); + + private final ValueType valueType; + MiscOption(final ValueType valueType) { + this.valueType = valueType; + } + + @Override + public ValueType getValueType() { + return valueType; + } + } + + private interface MutableColumnFamilyOptionKey { + String name(); + ValueType getValueType(); + } + + private static abstract class MutableColumnFamilyOptionValue { + protected final T value; + + MutableColumnFamilyOptionValue(final T value) { + this.value = value; + } + + abstract double asDouble() throws NumberFormatException; + abstract long asLong() throws NumberFormatException; + abstract int asInt() throws NumberFormatException; + abstract boolean asBoolean() throws IllegalStateException; + abstract int[] asIntArray() throws IllegalStateException; + abstract String asString(); + abstract T asObject(); + } + + private static class MutableColumnFamilyOptionStringValue + extends MutableColumnFamilyOptionValue { + MutableColumnFamilyOptionStringValue(final String value) { + super(value); + } + + @Override + double asDouble() throws NumberFormatException { + return Double.parseDouble(value); + } + + @Override + long asLong() throws NumberFormatException { + return Long.parseLong(value); + } + + @Override + int asInt() throws NumberFormatException { + return Integer.parseInt(value); + } + + @Override + boolean asBoolean() throws IllegalStateException { + return Boolean.parseBoolean(value); + } + + @Override + int[] asIntArray() throws IllegalStateException { + throw new IllegalStateException("String is not applicable as int[]"); + } + + @Override + String asString() { + return value; + } + + @Override + String asObject() { + return value; + } + } + + private static class MutableColumnFamilyOptionDoubleValue + extends MutableColumnFamilyOptionValue { + MutableColumnFamilyOptionDoubleValue(final double value) { + super(value); + } + + @Override + double asDouble() { + return value; + } + + @Override + long asLong() throws NumberFormatException { + return value.longValue(); + } + + @Override + int asInt() throws NumberFormatException { + if(value > Integer.MAX_VALUE || value < Integer.MIN_VALUE) { + throw new NumberFormatException( + "double value lies outside the bounds of int"); + } + return value.intValue(); + } + + @Override + boolean asBoolean() throws IllegalStateException { + throw new IllegalStateException( + "double is not applicable as boolean"); + } + + @Override + int[] asIntArray() throws IllegalStateException { + if(value > Integer.MAX_VALUE || value < Integer.MIN_VALUE) { + throw new NumberFormatException( + "double value lies outside the bounds of int"); + } + return new int[] { value.intValue() }; + } + + @Override + String asString() { + return Double.toString(value); + } + + @Override + Double asObject() { + return value; + } + } + + private static class MutableColumnFamilyOptionLongValue + extends MutableColumnFamilyOptionValue { + MutableColumnFamilyOptionLongValue(final long value) { + super(value); + } + + @Override + double asDouble() { + if(value > Double.MAX_VALUE || value < Double.MIN_VALUE) { + throw new NumberFormatException( + "long value lies outside the bounds of int"); + } + return value.doubleValue(); + } + + @Override + long asLong() throws NumberFormatException { + return value; + } + + @Override + int asInt() throws NumberFormatException { + if(value > Integer.MAX_VALUE || value < Integer.MIN_VALUE) { + throw new NumberFormatException( + "long value lies outside the bounds of int"); + } + return value.intValue(); + } + + @Override + boolean asBoolean() throws IllegalStateException { + throw new IllegalStateException( + "long is not applicable as boolean"); + } + + @Override + int[] asIntArray() throws IllegalStateException { + if(value > Integer.MAX_VALUE || value < Integer.MIN_VALUE) { + throw new NumberFormatException( + "long value lies outside the bounds of int"); + } + return new int[] { value.intValue() }; + } + + @Override + String asString() { + return Long.toString(value); + } + + @Override + Long asObject() { + return value; + } + } + + private static class MutableColumnFamilyOptionIntValue + extends MutableColumnFamilyOptionValue { + MutableColumnFamilyOptionIntValue(final int value) { + super(value); + } + + @Override + double asDouble() { + if(value > Double.MAX_VALUE || value < Double.MIN_VALUE) { + throw new NumberFormatException("int value lies outside the bounds of int"); + } + return value.doubleValue(); + } + + @Override + long asLong() throws NumberFormatException { + return value; + } + + @Override + int asInt() throws NumberFormatException { + return value; + } + + @Override + boolean asBoolean() throws IllegalStateException { + throw new IllegalStateException("int is not applicable as boolean"); + } + + @Override + int[] asIntArray() throws IllegalStateException { + return new int[] { value }; + } + + @Override + String asString() { + return Integer.toString(value); + } + + @Override + Integer asObject() { + return value; + } + } + + private static class MutableColumnFamilyOptionBooleanValue + extends MutableColumnFamilyOptionValue { + MutableColumnFamilyOptionBooleanValue(final boolean value) { + super(value); + } + + @Override + double asDouble() { + throw new NumberFormatException("boolean is not applicable as double"); + } + + @Override + long asLong() throws NumberFormatException { + throw new NumberFormatException("boolean is not applicable as Long"); + } + + @Override + int asInt() throws NumberFormatException { + throw new NumberFormatException("boolean is not applicable as int"); + } + + @Override + boolean asBoolean() { + return value; + } + + @Override + int[] asIntArray() throws IllegalStateException { + throw new IllegalStateException("boolean is not applicable as int[]"); + } + + @Override + String asString() { + return Boolean.toString(value); + } + + @Override + Boolean asObject() { + return value; + } + } + + private static class MutableColumnFamilyOptionIntArrayValue + extends MutableColumnFamilyOptionValue { + MutableColumnFamilyOptionIntArrayValue(final int[] value) { + super(value); + } + + @Override + double asDouble() { + throw new NumberFormatException("int[] is not applicable as double"); + } + + @Override + long asLong() throws NumberFormatException { + throw new NumberFormatException("int[] is not applicable as Long"); + } + + @Override + int asInt() throws NumberFormatException { + throw new NumberFormatException("int[] is not applicable as int"); + } + + @Override + boolean asBoolean() { + throw new NumberFormatException("int[] is not applicable as boolean"); + } + + @Override + int[] asIntArray() throws IllegalStateException { + return value; + } + + @Override + String asString() { + final StringBuilder builder = new StringBuilder(); + for(int i = 0; i < value.length; i++) { + builder.append(Integer.toString(i)); + if(i + 1 < value.length) { + builder.append(INT_ARRAY_INT_SEPARATOR); + } + } + return builder.toString(); + } + + @Override + int[] asObject() { + return value; + } + } + + private static class MutableColumnFamilyOptionEnumValue> + extends MutableColumnFamilyOptionValue { + + MutableColumnFamilyOptionEnumValue(final T value) { + super(value); + } + + @Override + double asDouble() throws NumberFormatException { + throw new NumberFormatException("Enum is not applicable as double"); + } + + @Override + long asLong() throws NumberFormatException { + throw new NumberFormatException("Enum is not applicable as long"); + } + + @Override + int asInt() throws NumberFormatException { + throw new NumberFormatException("Enum is not applicable as int"); + } + + @Override + boolean asBoolean() throws IllegalStateException { + throw new NumberFormatException("Enum is not applicable as boolean"); + } + + @Override + int[] asIntArray() throws IllegalStateException { + throw new NumberFormatException("Enum is not applicable as int[]"); + } + + @Override + String asString() { + return value.name(); + } + + @Override + T asObject() { + return value; + } + } + + public static class MutableColumnFamilyOptionsBuilder + implements MutableColumnFamilyOptionsInterface { + + private final static Map ALL_KEYS_LOOKUP = new HashMap<>(); + static { + for(final MutableColumnFamilyOptionKey key : MemtableOption.values()) { + ALL_KEYS_LOOKUP.put(key.name(), key); + } + + for(final MutableColumnFamilyOptionKey key : CompactionOption.values()) { + ALL_KEYS_LOOKUP.put(key.name(), key); + } + + for(final MutableColumnFamilyOptionKey key : MiscOption.values()) { + ALL_KEYS_LOOKUP.put(key.name(), key); + } + } + + private final Map> options = new LinkedHashMap<>(); + + public MutableColumnFamilyOptions build() { + final String keys[] = new String[options.size()]; + final String values[] = new String[options.size()]; + + int i = 0; + for(final Map.Entry> option : options.entrySet()) { + keys[i] = option.getKey().name(); + values[i] = option.getValue().asString(); + i++; + } + + return new MutableColumnFamilyOptions(keys, values); + } + + private MutableColumnFamilyOptionsBuilder setDouble( + final MutableColumnFamilyOptionKey key, final double value) { + if(key.getValueType() != ValueType.DOUBLE) { + throw new IllegalArgumentException( + key + " does not accept a double value"); + } + options.put(key, new MutableColumnFamilyOptionDoubleValue(value)); + return this; + } + + private double getDouble(final MutableColumnFamilyOptionKey key) + throws NoSuchElementException, NumberFormatException { + final MutableColumnFamilyOptionValue value = options.get(key); + if(value == null) { + throw new NoSuchElementException(key.name() + " has not been set"); + } + return value.asDouble(); + } + + private MutableColumnFamilyOptionsBuilder setLong( + final MutableColumnFamilyOptionKey key, final long value) { + if(key.getValueType() != ValueType.LONG) { + throw new IllegalArgumentException( + key + " does not accept a long value"); + } + options.put(key, new MutableColumnFamilyOptionLongValue(value)); + return this; + } + + private long getLong(final MutableColumnFamilyOptionKey key) + throws NoSuchElementException, NumberFormatException { + final MutableColumnFamilyOptionValue value = options.get(key); + if(value == null) { + throw new NoSuchElementException(key.name() + " has not been set"); + } + return value.asLong(); + } + + private MutableColumnFamilyOptionsBuilder setInt( + final MutableColumnFamilyOptionKey key, final int value) { + if(key.getValueType() != ValueType.INT) { + throw new IllegalArgumentException( + key + " does not accept an integer value"); + } + options.put(key, new MutableColumnFamilyOptionIntValue(value)); + return this; + } + + private int getInt(final MutableColumnFamilyOptionKey key) + throws NoSuchElementException, NumberFormatException { + final MutableColumnFamilyOptionValue value = options.get(key); + if(value == null) { + throw new NoSuchElementException(key.name() + " has not been set"); + } + return value.asInt(); + } + + private MutableColumnFamilyOptionsBuilder setBoolean( + final MutableColumnFamilyOptionKey key, final boolean value) { + if(key.getValueType() != ValueType.BOOLEAN) { + throw new IllegalArgumentException( + key + " does not accept a boolean value"); + } + options.put(key, new MutableColumnFamilyOptionBooleanValue(value)); + return this; + } + + private boolean getBoolean(final MutableColumnFamilyOptionKey key) + throws NoSuchElementException, NumberFormatException { + final MutableColumnFamilyOptionValue value = options.get(key); + if(value == null) { + throw new NoSuchElementException(key.name() + " has not been set"); + } + return value.asBoolean(); + } + + private MutableColumnFamilyOptionsBuilder setIntArray( + final MutableColumnFamilyOptionKey key, final int[] value) { + if(key.getValueType() != ValueType.INT_ARRAY) { + throw new IllegalArgumentException( + key + " does not accept an int array value"); + } + options.put(key, new MutableColumnFamilyOptionIntArrayValue(value)); + return this; + } + + private int[] getIntArray(final MutableColumnFamilyOptionKey key) + throws NoSuchElementException, NumberFormatException { + final MutableColumnFamilyOptionValue value = options.get(key); + if(value == null) { + throw new NoSuchElementException(key.name() + " has not been set"); + } + return value.asIntArray(); + } + + private > MutableColumnFamilyOptionsBuilder setEnum( + final MutableColumnFamilyOptionKey key, final T value) { + if(key.getValueType() != ValueType.ENUM) { + throw new IllegalArgumentException( + key + " does not accept a Enum value"); + } + options.put(key, new MutableColumnFamilyOptionEnumValue(value)); + return this; + + } + + private > T getEnum(final MutableColumnFamilyOptionKey key) + throws NoSuchElementException, NumberFormatException { + final MutableColumnFamilyOptionValue value = options.get(key); + if(value == null) { + throw new NoSuchElementException(key.name() + " has not been set"); + } + + if(!(value instanceof MutableColumnFamilyOptionEnumValue)) { + throw new NoSuchElementException(key.name() + " is not of Enum type"); + } + + return ((MutableColumnFamilyOptionEnumValue)value).asObject(); + } + + public MutableColumnFamilyOptionsBuilder fromString(final String keyStr, + final String valueStr) throws IllegalArgumentException { + Objects.requireNonNull(keyStr); + Objects.requireNonNull(valueStr); + + final MutableColumnFamilyOptionKey key = ALL_KEYS_LOOKUP.get(keyStr); + switch(key.getValueType()) { + case DOUBLE: + return setDouble(key, Double.parseDouble(valueStr)); + + case LONG: + return setLong(key, Long.parseLong(valueStr)); + + case INT: + return setInt(key, Integer.parseInt(valueStr)); + + case BOOLEAN: + return setBoolean(key, Boolean.parseBoolean(valueStr)); + + case INT_ARRAY: + final String[] strInts = valueStr + .trim().split(INT_ARRAY_INT_SEPARATOR); + if(strInts == null || strInts.length == 0) { + throw new IllegalArgumentException( + "int array value is not correctly formatted"); + } + + final int value[] = new int[strInts.length]; + int i = 0; + for(final String strInt : strInts) { + value[i++] = Integer.parseInt(strInt); + } + return setIntArray(key, value); + } + + throw new IllegalStateException( + key + " has unknown value type: " + key.getValueType()); + } + + @Override + public MutableColumnFamilyOptionsBuilder setWriteBufferSize( + final long writeBufferSize) { + return setLong(MemtableOption.write_buffer_size, writeBufferSize); + } + + @Override + public long writeBufferSize() { + return getLong(MemtableOption.write_buffer_size); + } + + @Override + public MutableColumnFamilyOptionsBuilder setArenaBlockSize( + final long arenaBlockSize) { + return setLong(MemtableOption.arena_block_size, arenaBlockSize); + } + + @Override + public long arenaBlockSize() { + return getLong(MemtableOption.arena_block_size); + } + + @Override + public MutableColumnFamilyOptionsBuilder setMemtablePrefixBloomSizeRatio( + final double memtablePrefixBloomSizeRatio) { + return setDouble(MemtableOption.memtable_prefix_bloom_size_ratio, + memtablePrefixBloomSizeRatio); + } + + @Override + public double memtablePrefixBloomSizeRatio() { + return getDouble(MemtableOption.memtable_prefix_bloom_size_ratio); + } + + @Override + public MutableColumnFamilyOptionsBuilder setMemtableHugePageSize( + final long memtableHugePageSize) { + return setLong(MemtableOption.memtable_huge_page_size, + memtableHugePageSize); + } + + @Override + public long memtableHugePageSize() { + return getLong(MemtableOption.memtable_huge_page_size); + } + + @Override + public MutableColumnFamilyOptionsBuilder setMaxSuccessiveMerges( + final long maxSuccessiveMerges) { + return setLong(MemtableOption.max_successive_merges, maxSuccessiveMerges); + } + + @Override + public long maxSuccessiveMerges() { + return getLong(MemtableOption.max_successive_merges); + } + + @Override + public MutableColumnFamilyOptionsBuilder setMaxWriteBufferNumber( + final int maxWriteBufferNumber) { + return setInt(MemtableOption.max_write_buffer_number, + maxWriteBufferNumber); + } + + @Override + public int maxWriteBufferNumber() { + return getInt(MemtableOption.max_write_buffer_number); + } + + @Override + public MutableColumnFamilyOptionsBuilder setInplaceUpdateNumLocks( + final long inplaceUpdateNumLocks) { + return setLong(MemtableOption.inplace_update_num_locks, + inplaceUpdateNumLocks); + } + + @Override + public long inplaceUpdateNumLocks() { + return getLong(MemtableOption.inplace_update_num_locks); + } + + @Override + public MutableColumnFamilyOptionsBuilder setDisableAutoCompactions( + final boolean disableAutoCompactions) { + return setBoolean(CompactionOption.disable_auto_compactions, + disableAutoCompactions); + } + + @Override + public boolean disableAutoCompactions() { + return getBoolean(CompactionOption.disable_auto_compactions); + } + + @Override + public MutableColumnFamilyOptionsBuilder setSoftPendingCompactionBytesLimit( + final long softPendingCompactionBytesLimit) { + return setLong(CompactionOption.soft_pending_compaction_bytes_limit, + softPendingCompactionBytesLimit); + } + + @Override + public long softPendingCompactionBytesLimit() { + return getLong(CompactionOption.soft_pending_compaction_bytes_limit); + } + + @Override + public MutableColumnFamilyOptionsBuilder setHardPendingCompactionBytesLimit( + final long hardPendingCompactionBytesLimit) { + return setLong(CompactionOption.hard_pending_compaction_bytes_limit, + hardPendingCompactionBytesLimit); + } + + @Override + public long hardPendingCompactionBytesLimit() { + return getLong(CompactionOption.hard_pending_compaction_bytes_limit); + } + + @Override + public MutableColumnFamilyOptionsBuilder setLevel0FileNumCompactionTrigger( + final int level0FileNumCompactionTrigger) { + return setInt(CompactionOption.level0_file_num_compaction_trigger, + level0FileNumCompactionTrigger); + } + + @Override + public int level0FileNumCompactionTrigger() { + return getInt(CompactionOption.level0_file_num_compaction_trigger); + } + + @Override + public MutableColumnFamilyOptionsBuilder setLevel0SlowdownWritesTrigger( + final int level0SlowdownWritesTrigger) { + return setInt(CompactionOption.level0_slowdown_writes_trigger, + level0SlowdownWritesTrigger); + } + + @Override + public int level0SlowdownWritesTrigger() { + return getInt(CompactionOption.level0_slowdown_writes_trigger); + } + + @Override + public MutableColumnFamilyOptionsBuilder setLevel0StopWritesTrigger( + final int level0StopWritesTrigger) { + return setInt(CompactionOption.level0_stop_writes_trigger, + level0StopWritesTrigger); + } + + @Override + public int level0StopWritesTrigger() { + return getInt(CompactionOption.level0_stop_writes_trigger); + } + + @Override + public MutableColumnFamilyOptionsBuilder setMaxCompactionBytes(final long maxCompactionBytes) { + return setLong(CompactionOption.max_compaction_bytes, maxCompactionBytes); + } + + @Override + public long maxCompactionBytes() { + return getLong(CompactionOption.max_compaction_bytes); + } + + + @Override + public MutableColumnFamilyOptionsBuilder setTargetFileSizeBase( + final long targetFileSizeBase) { + return setLong(CompactionOption.target_file_size_base, + targetFileSizeBase); + } + + @Override + public long targetFileSizeBase() { + return getLong(CompactionOption.target_file_size_base); + } + + @Override + public MutableColumnFamilyOptionsBuilder setTargetFileSizeMultiplier( + final int targetFileSizeMultiplier) { + return setInt(CompactionOption.target_file_size_multiplier, + targetFileSizeMultiplier); + } + + @Override + public int targetFileSizeMultiplier() { + return getInt(CompactionOption.target_file_size_multiplier); + } + + @Override + public MutableColumnFamilyOptionsBuilder setMaxBytesForLevelBase( + final long maxBytesForLevelBase) { + return setLong(CompactionOption.max_bytes_for_level_base, + maxBytesForLevelBase); + } + + @Override + public long maxBytesForLevelBase() { + return getLong(CompactionOption.max_bytes_for_level_base); + } + + @Override + public MutableColumnFamilyOptionsBuilder setMaxBytesForLevelMultiplier( + final double maxBytesForLevelMultiplier) { + return setDouble(CompactionOption.max_bytes_for_level_multiplier, maxBytesForLevelMultiplier); + } + + @Override + public double maxBytesForLevelMultiplier() { + return getDouble(CompactionOption.max_bytes_for_level_multiplier); + } + + @Override + public MutableColumnFamilyOptionsBuilder setMaxBytesForLevelMultiplierAdditional( + final int[] maxBytesForLevelMultiplierAdditional) { + return setIntArray( + CompactionOption.max_bytes_for_level_multiplier_additional, + maxBytesForLevelMultiplierAdditional); + } + + @Override + public int[] maxBytesForLevelMultiplierAdditional() { + return getIntArray( + CompactionOption.max_bytes_for_level_multiplier_additional); + } + + @Override + public MutableColumnFamilyOptionsBuilder setMaxSequentialSkipInIterations( + final long maxSequentialSkipInIterations) { + return setLong(MiscOption.max_sequential_skip_in_iterations, + maxSequentialSkipInIterations); + } + + @Override + public long maxSequentialSkipInIterations() { + return getLong(MiscOption.max_sequential_skip_in_iterations); + } + + @Override + public MutableColumnFamilyOptionsBuilder setParanoidFileChecks( + final boolean paranoidFileChecks) { + return setBoolean(MiscOption.paranoid_file_checks, paranoidFileChecks); + } + + @Override + public boolean paranoidFileChecks() { + return getBoolean(MiscOption.paranoid_file_checks); + } + + @Override + public MutableColumnFamilyOptionsBuilder setCompressionType( + final CompressionType compressionType) { + return setEnum(MiscOption.compression_type, compressionType); + } + + @Override + public CompressionType compressionType() { + return (CompressionType)getEnum(MiscOption.compression_type); + } + + @Override + public MutableColumnFamilyOptionsBuilder setReportBgIoStats( + final boolean reportBgIoStats) { + return setBoolean(MiscOption.report_bg_io_stats, reportBgIoStats); + } + + @Override + public boolean reportBgIoStats() { + return getBoolean(MiscOption.report_bg_io_stats); + } + } +} diff --git a/java/src/main/java/org/rocksdb/MutableColumnFamilyOptionsInterface.java b/java/src/main/java/org/rocksdb/MutableColumnFamilyOptionsInterface.java new file mode 100644 index 00000000000..c2efcc54b6b --- /dev/null +++ b/java/src/main/java/org/rocksdb/MutableColumnFamilyOptionsInterface.java @@ -0,0 +1,159 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +public interface MutableColumnFamilyOptionsInterface + + extends AdvancedMutableColumnFamilyOptionsInterface { + + /** + * Amount of data to build up in memory (backed by an unsorted log + * on disk) before converting to a sorted on-disk file. + * + * Larger values increase performance, especially during bulk loads. + * Up to {@code max_write_buffer_number} write buffers may be held in memory + * at the same time, so you may wish to adjust this parameter + * to control memory usage. + * + * Also, a larger write buffer will result in a longer recovery time + * the next time the database is opened. + * + * Default: 4MB + * @param writeBufferSize the size of write buffer. + * @return the instance of the current object. + * @throws java.lang.IllegalArgumentException thrown on 32-Bit platforms + * while overflowing the underlying platform specific value. + */ + MutableColumnFamilyOptionsInterface setWriteBufferSize(long writeBufferSize); + + /** + * Return size of write buffer size. + * + * @return size of write buffer. + * @see #setWriteBufferSize(long) + */ + long writeBufferSize(); + + /** + * Disable automatic compactions. Manual compactions can still + * be issued on this column family + * + * @param disableAutoCompactions true if auto-compactions are disabled. + * @return the reference to the current option. + */ + MutableColumnFamilyOptionsInterface setDisableAutoCompactions( + boolean disableAutoCompactions); + + /** + * Disable automatic compactions. Manual compactions can still + * be issued on this column family + * + * @return true if auto-compactions are disabled. + */ + boolean disableAutoCompactions(); + + /** + * Number of files to trigger level-0 compaction. A value < 0 means that + * level-0 compaction will not be triggered by number of files at all. + * + * Default: 4 + * + * @param level0FileNumCompactionTrigger The number of files to trigger + * level-0 compaction + * @return the reference to the current option. + */ + MutableColumnFamilyOptionsInterface setLevel0FileNumCompactionTrigger( + int level0FileNumCompactionTrigger); + + /** + * Number of files to trigger level-0 compaction. A value < 0 means that + * level-0 compaction will not be triggered by number of files at all. + * + * Default: 4 + * + * @return The number of files to trigger + */ + int level0FileNumCompactionTrigger(); + + /** + * We try to limit number of bytes in one compaction to be lower than this + * threshold. But it's not guaranteed. + * Value 0 will be sanitized. + * + * @param maxCompactionBytes max bytes in a compaction + * @return the reference to the current option. + * @see #maxCompactionBytes() + */ + MutableColumnFamilyOptionsInterface setMaxCompactionBytes(final long maxCompactionBytes); + + /** + * We try to limit number of bytes in one compaction to be lower than this + * threshold. But it's not guaranteed. + * Value 0 will be sanitized. + * + * @return the maximum number of bytes in for a compaction. + * @see #setMaxCompactionBytes(long) + */ + long maxCompactionBytes(); + + /** + * The upper-bound of the total size of level-1 files in bytes. + * Maximum number of bytes for level L can be calculated as + * (maxBytesForLevelBase) * (maxBytesForLevelMultiplier ^ (L-1)) + * For example, if maxBytesForLevelBase is 20MB, and if + * max_bytes_for_level_multiplier is 10, total data size for level-1 + * will be 20MB, total file size for level-2 will be 200MB, + * and total file size for level-3 will be 2GB. + * by default 'maxBytesForLevelBase' is 10MB. + * + * @param maxBytesForLevelBase maximum bytes for level base. + * + * @return the reference to the current option. + * + * See {@link AdvancedMutableColumnFamilyOptionsInterface#setMaxBytesForLevelMultiplier(double)} + */ + T setMaxBytesForLevelBase( + long maxBytesForLevelBase); + + /** + * The upper-bound of the total size of level-1 files in bytes. + * Maximum number of bytes for level L can be calculated as + * (maxBytesForLevelBase) * (maxBytesForLevelMultiplier ^ (L-1)) + * For example, if maxBytesForLevelBase is 20MB, and if + * max_bytes_for_level_multiplier is 10, total data size for level-1 + * will be 20MB, total file size for level-2 will be 200MB, + * and total file size for level-3 will be 2GB. + * by default 'maxBytesForLevelBase' is 10MB. + * + * @return the upper-bound of the total size of level-1 files + * in bytes. + * + * See {@link AdvancedMutableColumnFamilyOptionsInterface#maxBytesForLevelMultiplier()} + */ + long maxBytesForLevelBase(); + + /** + * Compress blocks using the specified compression algorithm. This + * parameter can be changed dynamically. + * + * Default: SNAPPY_COMPRESSION, which gives lightweight but fast compression. + * + * @param compressionType Compression Type. + * @return the reference to the current option. + */ + T setCompressionType( + CompressionType compressionType); + + /** + * Compress blocks using the specified compression algorithm. This + * parameter can be changed dynamically. + * + * Default: SNAPPY_COMPRESSION, which gives lightweight but fast compression. + * + * @return Compression type. + */ + CompressionType compressionType(); +} diff --git a/java/src/main/java/org/rocksdb/NativeLibraryLoader.java b/java/src/main/java/org/rocksdb/NativeLibraryLoader.java new file mode 100644 index 00000000000..96d364cd1c6 --- /dev/null +++ b/java/src/main/java/org/rocksdb/NativeLibraryLoader.java @@ -0,0 +1,124 @@ +package org.rocksdb; + +import java.io.*; +import java.nio.file.Files; +import java.nio.file.StandardCopyOption; + +import org.rocksdb.util.Environment; + +/** + * This class is used to load the RocksDB shared library from within the jar. + * The shared library is extracted to a temp folder and loaded from there. + */ +public class NativeLibraryLoader { + //singleton + private static final NativeLibraryLoader instance = new NativeLibraryLoader(); + private static boolean initialized = false; + + private static final String sharedLibraryName = Environment.getSharedLibraryName("rocksdb"); + private static final String jniLibraryName = Environment.getJniLibraryName("rocksdb"); + private static final String jniLibraryFileName = Environment.getJniLibraryFileName("rocksdb"); + private static final String tempFilePrefix = "librocksdbjni"; + private static final String tempFileSuffix = Environment.getJniLibraryExtension(); + + /** + * Get a reference to the NativeLibraryLoader + * + * @return The NativeLibraryLoader + */ + public static NativeLibraryLoader getInstance() { + return instance; + } + + /** + * Firstly attempts to load the library from java.library.path, + * if that fails then it falls back to extracting + * the library from the classpath + * {@link org.rocksdb.NativeLibraryLoader#loadLibraryFromJar(java.lang.String)} + * + * @param tmpDir A temporary directory to use + * to copy the native library to when loading from the classpath. + * If null, or the empty string, we rely on Java's + * {@link java.io.File#createTempFile(String, String)} + * function to provide a temporary location. + * The temporary file will be registered for deletion + * on exit. + * + * @throws java.io.IOException if a filesystem operation fails. + */ + public synchronized void loadLibrary(final String tmpDir) throws IOException { + try { + System.loadLibrary(sharedLibraryName); + } catch(final UnsatisfiedLinkError ule1) { + try { + System.loadLibrary(jniLibraryName); + } catch(final UnsatisfiedLinkError ule2) { + loadLibraryFromJar(tmpDir); + } + } + } + + /** + * Attempts to extract the native RocksDB library + * from the classpath and load it + * + * @param tmpDir A temporary directory to use + * to copy the native library to. If null, + * or the empty string, we rely on Java's + * {@link java.io.File#createTempFile(String, String)} + * function to provide a temporary location. + * The temporary file will be registered for deletion + * on exit. + * + * @throws java.io.IOException if a filesystem operation fails. + */ + void loadLibraryFromJar(final String tmpDir) + throws IOException { + if (!initialized) { + System.load(loadLibraryFromJarToTemp(tmpDir).getAbsolutePath()); + initialized = true; + } + } + + File loadLibraryFromJarToTemp(final String tmpDir) + throws IOException { + final File temp; + if (tmpDir == null || tmpDir.isEmpty()) { + temp = File.createTempFile(tempFilePrefix, tempFileSuffix); + } else { + temp = new File(tmpDir, jniLibraryFileName); + if (temp.exists() && !temp.delete()) { + throw new RuntimeException("File: " + temp.getAbsolutePath() + + " already exists and cannot be removed."); + } + if (!temp.createNewFile()) { + throw new RuntimeException("File: " + temp.getAbsolutePath() + + " could not be created."); + } + } + + if (!temp.exists()) { + throw new RuntimeException("File " + temp.getAbsolutePath() + " does not exist."); + } else { + temp.deleteOnExit(); + } + + // attempt to copy the library from the Jar file to the temp destination + try (final InputStream is = getClass().getClassLoader(). + getResourceAsStream(jniLibraryFileName)) { + if (is == null) { + throw new RuntimeException(jniLibraryFileName + " was not found inside JAR."); + } else { + Files.copy(is, temp.toPath(), StandardCopyOption.REPLACE_EXISTING); + } + } + + return temp; + } + + /** + * Private constructor to disallow instantiation + */ + private NativeLibraryLoader() { + } +} diff --git a/java/src/main/java/org/rocksdb/Options.java b/java/src/main/java/org/rocksdb/Options.java new file mode 100644 index 00000000000..dcd1138a8ab --- /dev/null +++ b/java/src/main/java/org/rocksdb/Options.java @@ -0,0 +1,1864 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.nio.file.Paths; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.List; + +/** + * Options to control the behavior of a database. It will be used + * during the creation of a {@link org.rocksdb.RocksDB} (i.e., RocksDB.open()). + * + * If {@link #dispose()} function is not called, then it will be GC'd + * automaticallyand native resources will be released as part of the process. + */ +public class Options extends RocksObject + implements DBOptionsInterface, ColumnFamilyOptionsInterface, + MutableColumnFamilyOptionsInterface { + static { + RocksDB.loadLibrary(); + } + + /** + * Construct options for opening a RocksDB. + * + * This constructor will create (by allocating a block of memory) + * an {@code rocksdb::Options} in the c++ side. + */ + public Options() { + super(newOptions()); + env_ = Env.getDefault(); + } + + /** + * Construct options for opening a RocksDB. Reusing database options + * and column family options. + * + * @param dbOptions {@link org.rocksdb.DBOptions} instance + * @param columnFamilyOptions {@link org.rocksdb.ColumnFamilyOptions} + * instance + */ + public Options(final DBOptions dbOptions, + final ColumnFamilyOptions columnFamilyOptions) { + super(newOptions(dbOptions.nativeHandle_, + columnFamilyOptions.nativeHandle_)); + env_ = Env.getDefault(); + } + + @Override + public Options setIncreaseParallelism(final int totalThreads) { + assert(isOwningHandle()); + setIncreaseParallelism(nativeHandle_, totalThreads); + return this; + } + + @Override + public Options setCreateIfMissing(final boolean flag) { + assert(isOwningHandle()); + setCreateIfMissing(nativeHandle_, flag); + return this; + } + + @Override + public Options setCreateMissingColumnFamilies(final boolean flag) { + assert(isOwningHandle()); + setCreateMissingColumnFamilies(nativeHandle_, flag); + return this; + } + + @Override + public Options setEnv(final Env env) { + assert(isOwningHandle()); + setEnv(nativeHandle_, env.nativeHandle_); + env_ = env; + return this; + } + + @Override + public Env getEnv() { + return env_; + } + + /** + *

    Set appropriate parameters for bulk loading. + * The reason that this is a function that returns "this" instead of a + * constructor is to enable chaining of multiple similar calls in the future. + *

    + * + *

    All data will be in level 0 without any automatic compaction. + * It's recommended to manually call CompactRange(NULL, NULL) before reading + * from the database, because otherwise the read can be very slow.

    + * + * @return the instance of the current Options. + */ + public Options prepareForBulkLoad() { + prepareForBulkLoad(nativeHandle_); + return this; + } + + @Override + public boolean createIfMissing() { + assert(isOwningHandle()); + return createIfMissing(nativeHandle_); + } + + @Override + public boolean createMissingColumnFamilies() { + assert(isOwningHandle()); + return createMissingColumnFamilies(nativeHandle_); + } + + @Override + public Options optimizeForSmallDb() { + optimizeForSmallDb(nativeHandle_); + return this; + } + + @Override + public Options optimizeForPointLookup( + long blockCacheSizeMb) { + optimizeForPointLookup(nativeHandle_, + blockCacheSizeMb); + return this; + } + + @Override + public Options optimizeLevelStyleCompaction() { + optimizeLevelStyleCompaction(nativeHandle_, + DEFAULT_COMPACTION_MEMTABLE_MEMORY_BUDGET); + return this; + } + + @Override + public Options optimizeLevelStyleCompaction( + long memtableMemoryBudget) { + optimizeLevelStyleCompaction(nativeHandle_, + memtableMemoryBudget); + return this; + } + + @Override + public Options optimizeUniversalStyleCompaction() { + optimizeUniversalStyleCompaction(nativeHandle_, + DEFAULT_COMPACTION_MEMTABLE_MEMORY_BUDGET); + return this; + } + + @Override + public Options optimizeUniversalStyleCompaction( + final long memtableMemoryBudget) { + optimizeUniversalStyleCompaction(nativeHandle_, + memtableMemoryBudget); + return this; + } + + @Override + public Options setComparator(final BuiltinComparator builtinComparator) { + assert(isOwningHandle()); + setComparatorHandle(nativeHandle_, builtinComparator.ordinal()); + return this; + } + + @Override + public Options setComparator( + final AbstractComparator> comparator) { + assert(isOwningHandle()); + setComparatorHandle(nativeHandle_, comparator.getNativeHandle()); + comparator_ = comparator; + return this; + } + + @Override + public Options setMergeOperatorName(final String name) { + assert(isOwningHandle()); + if (name == null) { + throw new IllegalArgumentException( + "Merge operator name must not be null."); + } + setMergeOperatorName(nativeHandle_, name); + return this; + } + + @Override + public Options setMergeOperator(final MergeOperator mergeOperator) { + setMergeOperator(nativeHandle_, mergeOperator.nativeHandle_); + return this; + } + + @Override + public Options setWriteBufferSize(final long writeBufferSize) { + assert(isOwningHandle()); + setWriteBufferSize(nativeHandle_, writeBufferSize); + return this; + } + + @Override + public long writeBufferSize() { + assert(isOwningHandle()); + return writeBufferSize(nativeHandle_); + } + + @Override + public Options setMaxWriteBufferNumber(final int maxWriteBufferNumber) { + assert(isOwningHandle()); + setMaxWriteBufferNumber(nativeHandle_, maxWriteBufferNumber); + return this; + } + + @Override + public int maxWriteBufferNumber() { + assert(isOwningHandle()); + return maxWriteBufferNumber(nativeHandle_); + } + + @Override + public boolean errorIfExists() { + assert(isOwningHandle()); + return errorIfExists(nativeHandle_); + } + + @Override + public Options setErrorIfExists(final boolean errorIfExists) { + assert(isOwningHandle()); + setErrorIfExists(nativeHandle_, errorIfExists); + return this; + } + + @Override + public boolean paranoidChecks() { + assert(isOwningHandle()); + return paranoidChecks(nativeHandle_); + } + + @Override + public Options setParanoidChecks(final boolean paranoidChecks) { + assert(isOwningHandle()); + setParanoidChecks(nativeHandle_, paranoidChecks); + return this; + } + + @Override + public int maxOpenFiles() { + assert(isOwningHandle()); + return maxOpenFiles(nativeHandle_); + } + + @Override + public Options setMaxFileOpeningThreads(final int maxFileOpeningThreads) { + assert(isOwningHandle()); + setMaxFileOpeningThreads(nativeHandle_, maxFileOpeningThreads); + return this; + } + + @Override + public int maxFileOpeningThreads() { + assert(isOwningHandle()); + return maxFileOpeningThreads(nativeHandle_); + } + + @Override + public Options setMaxTotalWalSize(final long maxTotalWalSize) { + assert(isOwningHandle()); + setMaxTotalWalSize(nativeHandle_, maxTotalWalSize); + return this; + } + + @Override + public long maxTotalWalSize() { + assert(isOwningHandle()); + return maxTotalWalSize(nativeHandle_); + } + + @Override + public Options setMaxOpenFiles(final int maxOpenFiles) { + assert(isOwningHandle()); + setMaxOpenFiles(nativeHandle_, maxOpenFiles); + return this; + } + + @Override + public boolean useFsync() { + assert(isOwningHandle()); + return useFsync(nativeHandle_); + } + + @Override + public Options setUseFsync(final boolean useFsync) { + assert(isOwningHandle()); + setUseFsync(nativeHandle_, useFsync); + return this; + } + + @Override + public Options setDbPaths(final Collection dbPaths) { + assert(isOwningHandle()); + + final int len = dbPaths.size(); + final String paths[] = new String[len]; + final long targetSizes[] = new long[len]; + + int i = 0; + for(final DbPath dbPath : dbPaths) { + paths[i] = dbPath.path.toString(); + targetSizes[i] = dbPath.targetSize; + i++; + } + setDbPaths(nativeHandle_, paths, targetSizes); + return this; + } + + @Override + public List dbPaths() { + final int len = (int)dbPathsLen(nativeHandle_); + if(len == 0) { + return Collections.emptyList(); + } else { + final String paths[] = new String[len]; + final long targetSizes[] = new long[len]; + + dbPaths(nativeHandle_, paths, targetSizes); + + final List dbPaths = new ArrayList<>(); + for(int i = 0; i < len; i++) { + dbPaths.add(new DbPath(Paths.get(paths[i]), targetSizes[i])); + } + return dbPaths; + } + } + + @Override + public String dbLogDir() { + assert(isOwningHandle()); + return dbLogDir(nativeHandle_); + } + + @Override + public Options setDbLogDir(final String dbLogDir) { + assert(isOwningHandle()); + setDbLogDir(nativeHandle_, dbLogDir); + return this; + } + + @Override + public String walDir() { + assert(isOwningHandle()); + return walDir(nativeHandle_); + } + + @Override + public Options setWalDir(final String walDir) { + assert(isOwningHandle()); + setWalDir(nativeHandle_, walDir); + return this; + } + + @Override + public long deleteObsoleteFilesPeriodMicros() { + assert(isOwningHandle()); + return deleteObsoleteFilesPeriodMicros(nativeHandle_); + } + + @Override + public Options setDeleteObsoleteFilesPeriodMicros( + final long micros) { + assert(isOwningHandle()); + setDeleteObsoleteFilesPeriodMicros(nativeHandle_, micros); + return this; + } + + @Override + public int maxBackgroundCompactions() { + assert(isOwningHandle()); + return maxBackgroundCompactions(nativeHandle_); + } + + @Override + public Options setStatistics(final Statistics statistics) { + assert(isOwningHandle()); + setStatistics(nativeHandle_, statistics.nativeHandle_); + return this; + } + + @Override + public Statistics statistics() { + assert(isOwningHandle()); + final long statisticsNativeHandle = statistics(nativeHandle_); + if(statisticsNativeHandle == 0) { + return null; + } else { + return new Statistics(statisticsNativeHandle); + } + } + + @Override + public void setBaseBackgroundCompactions( + final int baseBackgroundCompactions) { + assert(isOwningHandle()); + setBaseBackgroundCompactions(nativeHandle_, baseBackgroundCompactions); + } + + @Override + public int baseBackgroundCompactions() { + assert(isOwningHandle()); + return baseBackgroundCompactions(nativeHandle_); + } + + @Override + public Options setMaxBackgroundCompactions( + final int maxBackgroundCompactions) { + assert(isOwningHandle()); + setMaxBackgroundCompactions(nativeHandle_, maxBackgroundCompactions); + return this; + } + + @Override + public void setMaxSubcompactions(final int maxSubcompactions) { + assert(isOwningHandle()); + setMaxSubcompactions(nativeHandle_, maxSubcompactions); + } + + @Override + public int maxSubcompactions() { + assert(isOwningHandle()); + return maxSubcompactions(nativeHandle_); + } + + @Override + public int maxBackgroundFlushes() { + assert(isOwningHandle()); + return maxBackgroundFlushes(nativeHandle_); + } + + @Override + public Options setMaxBackgroundFlushes( + final int maxBackgroundFlushes) { + assert(isOwningHandle()); + setMaxBackgroundFlushes(nativeHandle_, maxBackgroundFlushes); + return this; + } + + @Override + public long maxLogFileSize() { + assert(isOwningHandle()); + return maxLogFileSize(nativeHandle_); + } + + @Override + public Options setMaxLogFileSize(final long maxLogFileSize) { + assert(isOwningHandle()); + setMaxLogFileSize(nativeHandle_, maxLogFileSize); + return this; + } + + @Override + public long logFileTimeToRoll() { + assert(isOwningHandle()); + return logFileTimeToRoll(nativeHandle_); + } + + @Override + public Options setLogFileTimeToRoll(final long logFileTimeToRoll) { + assert(isOwningHandle()); + setLogFileTimeToRoll(nativeHandle_, logFileTimeToRoll); + return this; + } + + @Override + public long keepLogFileNum() { + assert(isOwningHandle()); + return keepLogFileNum(nativeHandle_); + } + + @Override + public Options setKeepLogFileNum(final long keepLogFileNum) { + assert(isOwningHandle()); + setKeepLogFileNum(nativeHandle_, keepLogFileNum); + return this; + } + + + @Override + public Options setRecycleLogFileNum(final long recycleLogFileNum) { + assert(isOwningHandle()); + setRecycleLogFileNum(nativeHandle_, recycleLogFileNum); + return this; + } + + @Override + public long recycleLogFileNum() { + assert(isOwningHandle()); + return recycleLogFileNum(nativeHandle_); + } + + @Override + public long maxManifestFileSize() { + assert(isOwningHandle()); + return maxManifestFileSize(nativeHandle_); + } + + @Override + public Options setMaxManifestFileSize( + final long maxManifestFileSize) { + assert(isOwningHandle()); + setMaxManifestFileSize(nativeHandle_, maxManifestFileSize); + return this; + } + + @Override + public Options setMaxTableFilesSizeFIFO( + final long maxTableFilesSize) { + assert(maxTableFilesSize > 0); // unsigned native type + assert(isOwningHandle()); + setMaxTableFilesSizeFIFO(nativeHandle_, maxTableFilesSize); + return this; + } + + @Override + public long maxTableFilesSizeFIFO() { + return maxTableFilesSizeFIFO(nativeHandle_); + } + + @Override + public int tableCacheNumshardbits() { + assert(isOwningHandle()); + return tableCacheNumshardbits(nativeHandle_); + } + + @Override + public Options setTableCacheNumshardbits( + final int tableCacheNumshardbits) { + assert(isOwningHandle()); + setTableCacheNumshardbits(nativeHandle_, tableCacheNumshardbits); + return this; + } + + @Override + public long walTtlSeconds() { + assert(isOwningHandle()); + return walTtlSeconds(nativeHandle_); + } + + @Override + public Options setWalTtlSeconds(final long walTtlSeconds) { + assert(isOwningHandle()); + setWalTtlSeconds(nativeHandle_, walTtlSeconds); + return this; + } + + @Override + public long walSizeLimitMB() { + assert(isOwningHandle()); + return walSizeLimitMB(nativeHandle_); + } + + @Override + public Options setWalSizeLimitMB(final long sizeLimitMB) { + assert(isOwningHandle()); + setWalSizeLimitMB(nativeHandle_, sizeLimitMB); + return this; + } + + @Override + public long manifestPreallocationSize() { + assert(isOwningHandle()); + return manifestPreallocationSize(nativeHandle_); + } + + @Override + public Options setManifestPreallocationSize(final long size) { + assert(isOwningHandle()); + setManifestPreallocationSize(nativeHandle_, size); + return this; + } + + @Override + public Options setUseDirectReads(final boolean useDirectReads) { + assert(isOwningHandle()); + setUseDirectReads(nativeHandle_, useDirectReads); + return this; + } + + @Override + public boolean useDirectReads() { + assert(isOwningHandle()); + return useDirectReads(nativeHandle_); + } + + @Override + public Options setUseDirectIoForFlushAndCompaction( + final boolean useDirectIoForFlushAndCompaction) { + assert(isOwningHandle()); + setUseDirectIoForFlushAndCompaction(nativeHandle_, useDirectIoForFlushAndCompaction); + return this; + } + + @Override + public boolean useDirectIoForFlushAndCompaction() { + assert(isOwningHandle()); + return useDirectIoForFlushAndCompaction(nativeHandle_); + } + + @Override + public Options setAllowFAllocate(final boolean allowFAllocate) { + assert(isOwningHandle()); + setAllowFAllocate(nativeHandle_, allowFAllocate); + return this; + } + + @Override + public boolean allowFAllocate() { + assert(isOwningHandle()); + return allowFAllocate(nativeHandle_); + } + + @Override + public boolean allowMmapReads() { + assert(isOwningHandle()); + return allowMmapReads(nativeHandle_); + } + + @Override + public Options setAllowMmapReads(final boolean allowMmapReads) { + assert(isOwningHandle()); + setAllowMmapReads(nativeHandle_, allowMmapReads); + return this; + } + + @Override + public boolean allowMmapWrites() { + assert(isOwningHandle()); + return allowMmapWrites(nativeHandle_); + } + + @Override + public Options setAllowMmapWrites(final boolean allowMmapWrites) { + assert(isOwningHandle()); + setAllowMmapWrites(nativeHandle_, allowMmapWrites); + return this; + } + + @Override + public boolean isFdCloseOnExec() { + assert(isOwningHandle()); + return isFdCloseOnExec(nativeHandle_); + } + + @Override + public Options setIsFdCloseOnExec(final boolean isFdCloseOnExec) { + assert(isOwningHandle()); + setIsFdCloseOnExec(nativeHandle_, isFdCloseOnExec); + return this; + } + + @Override + public int statsDumpPeriodSec() { + assert(isOwningHandle()); + return statsDumpPeriodSec(nativeHandle_); + } + + @Override + public Options setStatsDumpPeriodSec(final int statsDumpPeriodSec) { + assert(isOwningHandle()); + setStatsDumpPeriodSec(nativeHandle_, statsDumpPeriodSec); + return this; + } + + @Override + public boolean adviseRandomOnOpen() { + return adviseRandomOnOpen(nativeHandle_); + } + + @Override + public Options setAdviseRandomOnOpen(final boolean adviseRandomOnOpen) { + assert(isOwningHandle()); + setAdviseRandomOnOpen(nativeHandle_, adviseRandomOnOpen); + return this; + } + + @Override + public Options setDbWriteBufferSize(final long dbWriteBufferSize) { + assert(isOwningHandle()); + setDbWriteBufferSize(nativeHandle_, dbWriteBufferSize); + return this; + } + + @Override + public long dbWriteBufferSize() { + assert(isOwningHandle()); + return dbWriteBufferSize(nativeHandle_); + } + + @Override + public Options setAccessHintOnCompactionStart(final AccessHint accessHint) { + assert(isOwningHandle()); + setAccessHintOnCompactionStart(nativeHandle_, accessHint.getValue()); + return this; + } + + @Override + public AccessHint accessHintOnCompactionStart() { + assert(isOwningHandle()); + return AccessHint.getAccessHint(accessHintOnCompactionStart(nativeHandle_)); + } + + @Override + public Options setNewTableReaderForCompactionInputs( + final boolean newTableReaderForCompactionInputs) { + assert(isOwningHandle()); + setNewTableReaderForCompactionInputs(nativeHandle_, + newTableReaderForCompactionInputs); + return this; + } + + @Override + public boolean newTableReaderForCompactionInputs() { + assert(isOwningHandle()); + return newTableReaderForCompactionInputs(nativeHandle_); + } + + @Override + public Options setCompactionReadaheadSize(final long compactionReadaheadSize) { + assert(isOwningHandle()); + setCompactionReadaheadSize(nativeHandle_, compactionReadaheadSize); + return this; + } + + @Override + public long compactionReadaheadSize() { + assert(isOwningHandle()); + return compactionReadaheadSize(nativeHandle_); + } + + @Override + public Options setRandomAccessMaxBufferSize(final long randomAccessMaxBufferSize) { + assert(isOwningHandle()); + setRandomAccessMaxBufferSize(nativeHandle_, randomAccessMaxBufferSize); + return this; + } + + @Override + public long randomAccessMaxBufferSize() { + assert(isOwningHandle()); + return randomAccessMaxBufferSize(nativeHandle_); + } + + @Override + public Options setWritableFileMaxBufferSize(final long writableFileMaxBufferSize) { + assert(isOwningHandle()); + setWritableFileMaxBufferSize(nativeHandle_, writableFileMaxBufferSize); + return this; + } + + @Override + public long writableFileMaxBufferSize() { + assert(isOwningHandle()); + return writableFileMaxBufferSize(nativeHandle_); + } + + @Override + public boolean useAdaptiveMutex() { + assert(isOwningHandle()); + return useAdaptiveMutex(nativeHandle_); + } + + @Override + public Options setUseAdaptiveMutex(final boolean useAdaptiveMutex) { + assert(isOwningHandle()); + setUseAdaptiveMutex(nativeHandle_, useAdaptiveMutex); + return this; + } + + @Override + public long bytesPerSync() { + return bytesPerSync(nativeHandle_); + } + + @Override + public Options setBytesPerSync(final long bytesPerSync) { + assert(isOwningHandle()); + setBytesPerSync(nativeHandle_, bytesPerSync); + return this; + } + + @Override + public Options setWalBytesPerSync(final long walBytesPerSync) { + assert(isOwningHandle()); + setWalBytesPerSync(nativeHandle_, walBytesPerSync); + return this; + } + + @Override + public long walBytesPerSync() { + assert(isOwningHandle()); + return walBytesPerSync(nativeHandle_); + } + + @Override + public Options setEnableThreadTracking(final boolean enableThreadTracking) { + assert(isOwningHandle()); + setEnableThreadTracking(nativeHandle_, enableThreadTracking); + return this; + } + + @Override + public boolean enableThreadTracking() { + assert(isOwningHandle()); + return enableThreadTracking(nativeHandle_); + } + + @Override + public Options setDelayedWriteRate(final long delayedWriteRate) { + assert(isOwningHandle()); + setDelayedWriteRate(nativeHandle_, delayedWriteRate); + return this; + } + + @Override + public long delayedWriteRate(){ + return delayedWriteRate(nativeHandle_); + } + + @Override + public Options setAllowConcurrentMemtableWrite( + final boolean allowConcurrentMemtableWrite) { + setAllowConcurrentMemtableWrite(nativeHandle_, + allowConcurrentMemtableWrite); + return this; + } + + @Override + public boolean allowConcurrentMemtableWrite() { + return allowConcurrentMemtableWrite(nativeHandle_); + } + + @Override + public Options setEnableWriteThreadAdaptiveYield( + final boolean enableWriteThreadAdaptiveYield) { + setEnableWriteThreadAdaptiveYield(nativeHandle_, + enableWriteThreadAdaptiveYield); + return this; + } + + @Override + public boolean enableWriteThreadAdaptiveYield() { + return enableWriteThreadAdaptiveYield(nativeHandle_); + } + + @Override + public Options setWriteThreadMaxYieldUsec(final long writeThreadMaxYieldUsec) { + setWriteThreadMaxYieldUsec(nativeHandle_, writeThreadMaxYieldUsec); + return this; + } + + @Override + public long writeThreadMaxYieldUsec() { + return writeThreadMaxYieldUsec(nativeHandle_); + } + + @Override + public Options setWriteThreadSlowYieldUsec(final long writeThreadSlowYieldUsec) { + setWriteThreadSlowYieldUsec(nativeHandle_, writeThreadSlowYieldUsec); + return this; + } + + @Override + public long writeThreadSlowYieldUsec() { + return writeThreadSlowYieldUsec(nativeHandle_); + } + + @Override + public Options setSkipStatsUpdateOnDbOpen(final boolean skipStatsUpdateOnDbOpen) { + assert(isOwningHandle()); + setSkipStatsUpdateOnDbOpen(nativeHandle_, skipStatsUpdateOnDbOpen); + return this; + } + + @Override + public boolean skipStatsUpdateOnDbOpen() { + assert(isOwningHandle()); + return skipStatsUpdateOnDbOpen(nativeHandle_); + } + + @Override + public Options setWalRecoveryMode(final WALRecoveryMode walRecoveryMode) { + assert(isOwningHandle()); + setWalRecoveryMode(nativeHandle_, walRecoveryMode.getValue()); + return this; + } + + @Override + public WALRecoveryMode walRecoveryMode() { + assert(isOwningHandle()); + return WALRecoveryMode.getWALRecoveryMode(walRecoveryMode(nativeHandle_)); + } + + @Override + public Options setAllow2pc(final boolean allow2pc) { + assert(isOwningHandle()); + setAllow2pc(nativeHandle_, allow2pc); + return this; + } + + @Override + public boolean allow2pc() { + assert(isOwningHandle()); + return allow2pc(nativeHandle_); + } + + @Override + public Options setRowCache(final Cache rowCache) { + assert(isOwningHandle()); + setRowCache(nativeHandle_, rowCache.nativeHandle_); + this.rowCache_ = rowCache; + return this; + } + + @Override + public Cache rowCache() { + assert(isOwningHandle()); + return this.rowCache_; + } + + @Override + public Options setFailIfOptionsFileError(final boolean failIfOptionsFileError) { + assert(isOwningHandle()); + setFailIfOptionsFileError(nativeHandle_, failIfOptionsFileError); + return this; + } + + @Override + public boolean failIfOptionsFileError() { + assert(isOwningHandle()); + return failIfOptionsFileError(nativeHandle_); + } + + @Override + public Options setDumpMallocStats(final boolean dumpMallocStats) { + assert(isOwningHandle()); + setDumpMallocStats(nativeHandle_, dumpMallocStats); + return this; + } + + @Override + public boolean dumpMallocStats() { + assert(isOwningHandle()); + return dumpMallocStats(nativeHandle_); + } + + @Override + public Options setAvoidFlushDuringRecovery(final boolean avoidFlushDuringRecovery) { + assert(isOwningHandle()); + setAvoidFlushDuringRecovery(nativeHandle_, avoidFlushDuringRecovery); + return this; + } + + @Override + public boolean avoidFlushDuringRecovery() { + assert(isOwningHandle()); + return avoidFlushDuringRecovery(nativeHandle_); + } + + @Override + public Options setAvoidFlushDuringShutdown(final boolean avoidFlushDuringShutdown) { + assert(isOwningHandle()); + setAvoidFlushDuringShutdown(nativeHandle_, avoidFlushDuringShutdown); + return this; + } + + @Override + public boolean avoidFlushDuringShutdown() { + assert(isOwningHandle()); + return avoidFlushDuringShutdown(nativeHandle_); + } + + @Override + public MemTableConfig memTableConfig() { + return this.memTableConfig_; + } + + @Override + public Options setMemTableConfig(final MemTableConfig config) { + memTableConfig_ = config; + setMemTableFactory(nativeHandle_, config.newMemTableFactoryHandle()); + return this; + } + + @Override + public Options setRateLimiter(final RateLimiter rateLimiter) { + assert(isOwningHandle()); + rateLimiter_ = rateLimiter; + setRateLimiter(nativeHandle_, rateLimiter.nativeHandle_); + return this; + } + + @Override + public Options setLogger(final Logger logger) { + assert(isOwningHandle()); + setLogger(nativeHandle_, logger.nativeHandle_); + return this; + } + + @Override + public Options setInfoLogLevel(final InfoLogLevel infoLogLevel) { + assert(isOwningHandle()); + setInfoLogLevel(nativeHandle_, infoLogLevel.getValue()); + return this; + } + + @Override + public InfoLogLevel infoLogLevel() { + assert(isOwningHandle()); + return InfoLogLevel.getInfoLogLevel( + infoLogLevel(nativeHandle_)); + } + + @Override + public String memTableFactoryName() { + assert(isOwningHandle()); + return memTableFactoryName(nativeHandle_); + } + + @Override + public TableFormatConfig tableFormatConfig() { + return this.tableFormatConfig_; + } + + @Override + public Options setTableFormatConfig(final TableFormatConfig config) { + tableFormatConfig_ = config; + setTableFactory(nativeHandle_, config.newTableFactoryHandle()); + return this; + } + + @Override + public String tableFactoryName() { + assert(isOwningHandle()); + return tableFactoryName(nativeHandle_); + } + + @Override + public Options useFixedLengthPrefixExtractor(final int n) { + assert(isOwningHandle()); + useFixedLengthPrefixExtractor(nativeHandle_, n); + return this; + } + + @Override + public Options useCappedPrefixExtractor(final int n) { + assert(isOwningHandle()); + useCappedPrefixExtractor(nativeHandle_, n); + return this; + } + + @Override + public CompressionType compressionType() { + return CompressionType.getCompressionType(compressionType(nativeHandle_)); + } + + @Override + public Options setCompressionPerLevel( + final List compressionLevels) { + final byte[] byteCompressionTypes = new byte[ + compressionLevels.size()]; + for (int i = 0; i < compressionLevels.size(); i++) { + byteCompressionTypes[i] = compressionLevels.get(i).getValue(); + } + setCompressionPerLevel(nativeHandle_, byteCompressionTypes); + return this; + } + + @Override + public List compressionPerLevel() { + final byte[] byteCompressionTypes = + compressionPerLevel(nativeHandle_); + final List compressionLevels = new ArrayList<>(); + for (final Byte byteCompressionType : byteCompressionTypes) { + compressionLevels.add(CompressionType.getCompressionType( + byteCompressionType)); + } + return compressionLevels; + } + + @Override + public Options setCompressionType(CompressionType compressionType) { + setCompressionType(nativeHandle_, compressionType.getValue()); + return this; + } + + + @Override + public Options setBottommostCompressionType( + final CompressionType bottommostCompressionType) { + setBottommostCompressionType(nativeHandle_, + bottommostCompressionType.getValue()); + return this; + } + + @Override + public CompressionType bottommostCompressionType() { + return CompressionType.getCompressionType( + bottommostCompressionType(nativeHandle_)); + } + + @Override + public Options setCompressionOptions( + final CompressionOptions compressionOptions) { + setCompressionOptions(nativeHandle_, compressionOptions.nativeHandle_); + this.compressionOptions_ = compressionOptions; + return this; + } + + @Override + public CompressionOptions compressionOptions() { + return this.compressionOptions_; + } + + @Override + public CompactionStyle compactionStyle() { + return CompactionStyle.values()[compactionStyle(nativeHandle_)]; + } + + @Override + public Options setCompactionStyle( + final CompactionStyle compactionStyle) { + setCompactionStyle(nativeHandle_, compactionStyle.getValue()); + return this; + } + + @Override + public int numLevels() { + return numLevels(nativeHandle_); + } + + @Override + public Options setNumLevels(int numLevels) { + setNumLevels(nativeHandle_, numLevels); + return this; + } + + @Override + public int levelZeroFileNumCompactionTrigger() { + return levelZeroFileNumCompactionTrigger(nativeHandle_); + } + + @Override + public Options setLevelZeroFileNumCompactionTrigger( + final int numFiles) { + setLevelZeroFileNumCompactionTrigger( + nativeHandle_, numFiles); + return this; + } + + @Override + public int levelZeroSlowdownWritesTrigger() { + return levelZeroSlowdownWritesTrigger(nativeHandle_); + } + + @Override + public Options setLevelZeroSlowdownWritesTrigger( + final int numFiles) { + setLevelZeroSlowdownWritesTrigger(nativeHandle_, numFiles); + return this; + } + + @Override + public int levelZeroStopWritesTrigger() { + return levelZeroStopWritesTrigger(nativeHandle_); + } + + @Override + public Options setLevelZeroStopWritesTrigger( + final int numFiles) { + setLevelZeroStopWritesTrigger(nativeHandle_, numFiles); + return this; + } + + @Override + public long targetFileSizeBase() { + return targetFileSizeBase(nativeHandle_); + } + + @Override + public Options setTargetFileSizeBase(long targetFileSizeBase) { + setTargetFileSizeBase(nativeHandle_, targetFileSizeBase); + return this; + } + + @Override + public int targetFileSizeMultiplier() { + return targetFileSizeMultiplier(nativeHandle_); + } + + @Override + public Options setTargetFileSizeMultiplier(int multiplier) { + setTargetFileSizeMultiplier(nativeHandle_, multiplier); + return this; + } + + @Override + public Options setMaxBytesForLevelBase(final long maxBytesForLevelBase) { + setMaxBytesForLevelBase(nativeHandle_, maxBytesForLevelBase); + return this; + } + + @Override + public long maxBytesForLevelBase() { + return maxBytesForLevelBase(nativeHandle_); + } + + @Override + public Options setLevelCompactionDynamicLevelBytes( + final boolean enableLevelCompactionDynamicLevelBytes) { + setLevelCompactionDynamicLevelBytes(nativeHandle_, + enableLevelCompactionDynamicLevelBytes); + return this; + } + + @Override + public boolean levelCompactionDynamicLevelBytes() { + return levelCompactionDynamicLevelBytes(nativeHandle_); + } + + @Override + public double maxBytesForLevelMultiplier() { + return maxBytesForLevelMultiplier(nativeHandle_); + } + + @Override + public Options setMaxBytesForLevelMultiplier(final double multiplier) { + setMaxBytesForLevelMultiplier(nativeHandle_, multiplier); + return this; + } + + @Override + public long maxCompactionBytes() { + return maxCompactionBytes(nativeHandle_); + } + + @Override + public Options setMaxCompactionBytes(final long maxCompactionBytes) { + setMaxCompactionBytes(nativeHandle_, maxCompactionBytes); + return this; + } + + @Override + public long arenaBlockSize() { + return arenaBlockSize(nativeHandle_); + } + + @Override + public Options setArenaBlockSize(final long arenaBlockSize) { + setArenaBlockSize(nativeHandle_, arenaBlockSize); + return this; + } + + @Override + public boolean disableAutoCompactions() { + return disableAutoCompactions(nativeHandle_); + } + + @Override + public Options setDisableAutoCompactions( + final boolean disableAutoCompactions) { + setDisableAutoCompactions(nativeHandle_, disableAutoCompactions); + return this; + } + + @Override + public long maxSequentialSkipInIterations() { + return maxSequentialSkipInIterations(nativeHandle_); + } + + @Override + public Options setMaxSequentialSkipInIterations( + final long maxSequentialSkipInIterations) { + setMaxSequentialSkipInIterations(nativeHandle_, + maxSequentialSkipInIterations); + return this; + } + + @Override + public boolean inplaceUpdateSupport() { + return inplaceUpdateSupport(nativeHandle_); + } + + @Override + public Options setInplaceUpdateSupport( + final boolean inplaceUpdateSupport) { + setInplaceUpdateSupport(nativeHandle_, inplaceUpdateSupport); + return this; + } + + @Override + public long inplaceUpdateNumLocks() { + return inplaceUpdateNumLocks(nativeHandle_); + } + + @Override + public Options setInplaceUpdateNumLocks( + final long inplaceUpdateNumLocks) { + setInplaceUpdateNumLocks(nativeHandle_, inplaceUpdateNumLocks); + return this; + } + + @Override + public double memtablePrefixBloomSizeRatio() { + return memtablePrefixBloomSizeRatio(nativeHandle_); + } + + @Override + public Options setMemtablePrefixBloomSizeRatio(final double memtablePrefixBloomSizeRatio) { + setMemtablePrefixBloomSizeRatio(nativeHandle_, memtablePrefixBloomSizeRatio); + return this; + } + + @Override + public int bloomLocality() { + return bloomLocality(nativeHandle_); + } + + @Override + public Options setBloomLocality(final int bloomLocality) { + setBloomLocality(nativeHandle_, bloomLocality); + return this; + } + + @Override + public long maxSuccessiveMerges() { + return maxSuccessiveMerges(nativeHandle_); + } + + @Override + public Options setMaxSuccessiveMerges(long maxSuccessiveMerges) { + setMaxSuccessiveMerges(nativeHandle_, maxSuccessiveMerges); + return this; + } + + @Override + public int minWriteBufferNumberToMerge() { + return minWriteBufferNumberToMerge(nativeHandle_); + } + + @Override + public Options setMinWriteBufferNumberToMerge( + final int minWriteBufferNumberToMerge) { + setMinWriteBufferNumberToMerge(nativeHandle_, minWriteBufferNumberToMerge); + return this; + } + + @Override + public Options setOptimizeFiltersForHits( + final boolean optimizeFiltersForHits) { + setOptimizeFiltersForHits(nativeHandle_, optimizeFiltersForHits); + return this; + } + + @Override + public boolean optimizeFiltersForHits() { + return optimizeFiltersForHits(nativeHandle_); + } + + @Override + public Options + setMemtableHugePageSize( + long memtableHugePageSize) { + setMemtableHugePageSize(nativeHandle_, + memtableHugePageSize); + return this; + } + + @Override + public long memtableHugePageSize() { + return memtableHugePageSize(nativeHandle_); + } + + @Override + public Options setSoftPendingCompactionBytesLimit(long softPendingCompactionBytesLimit) { + setSoftPendingCompactionBytesLimit(nativeHandle_, + softPendingCompactionBytesLimit); + return this; + } + + @Override + public long softPendingCompactionBytesLimit() { + return softPendingCompactionBytesLimit(nativeHandle_); + } + + @Override + public Options setHardPendingCompactionBytesLimit(long hardPendingCompactionBytesLimit) { + setHardPendingCompactionBytesLimit(nativeHandle_, hardPendingCompactionBytesLimit); + return this; + } + + @Override + public long hardPendingCompactionBytesLimit() { + return hardPendingCompactionBytesLimit(nativeHandle_); + } + + @Override + public Options setLevel0FileNumCompactionTrigger(int level0FileNumCompactionTrigger) { + setLevel0FileNumCompactionTrigger(nativeHandle_, level0FileNumCompactionTrigger); + return this; + } + + @Override + public int level0FileNumCompactionTrigger() { + return level0FileNumCompactionTrigger(nativeHandle_); + } + + @Override + public Options setLevel0SlowdownWritesTrigger(int level0SlowdownWritesTrigger) { + setLevel0SlowdownWritesTrigger(nativeHandle_, level0SlowdownWritesTrigger); + return this; + } + + @Override + public int level0SlowdownWritesTrigger() { + return level0SlowdownWritesTrigger(nativeHandle_); + } + + @Override + public Options setLevel0StopWritesTrigger(int level0StopWritesTrigger) { + setLevel0StopWritesTrigger(nativeHandle_, level0StopWritesTrigger); + return this; + } + + @Override + public int level0StopWritesTrigger() { + return level0StopWritesTrigger(nativeHandle_); + } + + @Override + public Options setMaxBytesForLevelMultiplierAdditional(int[] maxBytesForLevelMultiplierAdditional) { + setMaxBytesForLevelMultiplierAdditional(nativeHandle_, maxBytesForLevelMultiplierAdditional); + return this; + } + + @Override + public int[] maxBytesForLevelMultiplierAdditional() { + return maxBytesForLevelMultiplierAdditional(nativeHandle_); + } + + @Override + public Options setParanoidFileChecks(boolean paranoidFileChecks) { + setParanoidFileChecks(nativeHandle_, paranoidFileChecks); + return this; + } + + @Override + public boolean paranoidFileChecks() { + return paranoidFileChecks(nativeHandle_); + } + + @Override + public Options setMaxWriteBufferNumberToMaintain( + final int maxWriteBufferNumberToMaintain) { + setMaxWriteBufferNumberToMaintain( + nativeHandle_, maxWriteBufferNumberToMaintain); + return this; + } + + @Override + public int maxWriteBufferNumberToMaintain() { + return maxWriteBufferNumberToMaintain(nativeHandle_); + } + + @Override + public Options setCompactionPriority( + final CompactionPriority compactionPriority) { + setCompactionPriority(nativeHandle_, compactionPriority.getValue()); + return this; + } + + @Override + public CompactionPriority compactionPriority() { + return CompactionPriority.getCompactionPriority( + compactionPriority(nativeHandle_)); + } + + @Override + public Options setReportBgIoStats(final boolean reportBgIoStats) { + setReportBgIoStats(nativeHandle_, reportBgIoStats); + return this; + } + + @Override + public boolean reportBgIoStats() { + return reportBgIoStats(nativeHandle_); + } + + @Override + public Options setCompactionOptionsUniversal( + final CompactionOptionsUniversal compactionOptionsUniversal) { + setCompactionOptionsUniversal(nativeHandle_, + compactionOptionsUniversal.nativeHandle_); + this.compactionOptionsUniversal_ = compactionOptionsUniversal; + return this; + } + + @Override + public CompactionOptionsUniversal compactionOptionsUniversal() { + return this.compactionOptionsUniversal_; + } + + @Override + public Options setCompactionOptionsFIFO(final CompactionOptionsFIFO compactionOptionsFIFO) { + setCompactionOptionsFIFO(nativeHandle_, + compactionOptionsFIFO.nativeHandle_); + this.compactionOptionsFIFO_ = compactionOptionsFIFO; + return this; + } + + @Override + public CompactionOptionsFIFO compactionOptionsFIFO() { + return this.compactionOptionsFIFO_; + } + + @Override + public Options setForceConsistencyChecks(final boolean forceConsistencyChecks) { + setForceConsistencyChecks(nativeHandle_, forceConsistencyChecks); + return this; + } + + @Override + public boolean forceConsistencyChecks() { + return forceConsistencyChecks(nativeHandle_); + } + + private native static long newOptions(); + private native static long newOptions(long dbOptHandle, + long cfOptHandle); + @Override protected final native void disposeInternal(final long handle); + private native void setEnv(long optHandle, long envHandle); + private native void prepareForBulkLoad(long handle); + + // DB native handles + private native void setIncreaseParallelism(long handle, int totalThreads); + private native void setCreateIfMissing(long handle, boolean flag); + private native boolean createIfMissing(long handle); + private native void setCreateMissingColumnFamilies( + long handle, boolean flag); + private native boolean createMissingColumnFamilies(long handle); + private native void setErrorIfExists(long handle, boolean errorIfExists); + private native boolean errorIfExists(long handle); + private native void setParanoidChecks( + long handle, boolean paranoidChecks); + private native boolean paranoidChecks(long handle); + private native void setRateLimiter(long handle, + long rateLimiterHandle); + private native void setLogger(long handle, + long loggerHandle); + private native void setInfoLogLevel(long handle, byte logLevel); + private native byte infoLogLevel(long handle); + private native void setMaxOpenFiles(long handle, int maxOpenFiles); + private native int maxOpenFiles(long handle); + private native void setMaxTotalWalSize(long handle, + long maxTotalWalSize); + private native void setMaxFileOpeningThreads(final long handle, + final int maxFileOpeningThreads); + private native int maxFileOpeningThreads(final long handle); + private native long maxTotalWalSize(long handle); + private native void setStatistics(final long handle, final long statisticsHandle); + private native long statistics(final long handle); + private native boolean useFsync(long handle); + private native void setUseFsync(long handle, boolean useFsync); + private native void setDbPaths(final long handle, final String[] paths, + final long[] targetSizes); + private native long dbPathsLen(final long handle); + private native void dbPaths(final long handle, final String[] paths, + final long[] targetSizes); + private native void setDbLogDir(long handle, String dbLogDir); + private native String dbLogDir(long handle); + private native void setWalDir(long handle, String walDir); + private native String walDir(long handle); + private native void setDeleteObsoleteFilesPeriodMicros( + long handle, long micros); + private native long deleteObsoleteFilesPeriodMicros(long handle); + private native void setBaseBackgroundCompactions(long handle, + int baseBackgroundCompactions); + private native int baseBackgroundCompactions(long handle); + private native void setMaxBackgroundCompactions( + long handle, int maxBackgroundCompactions); + private native int maxBackgroundCompactions(long handle); + private native void setMaxSubcompactions(long handle, int maxSubcompactions); + private native int maxSubcompactions(long handle); + private native void setMaxBackgroundFlushes( + long handle, int maxBackgroundFlushes); + private native int maxBackgroundFlushes(long handle); + private native void setMaxLogFileSize(long handle, long maxLogFileSize) + throws IllegalArgumentException; + private native long maxLogFileSize(long handle); + private native void setLogFileTimeToRoll( + long handle, long logFileTimeToRoll) throws IllegalArgumentException; + private native long logFileTimeToRoll(long handle); + private native void setKeepLogFileNum(long handle, long keepLogFileNum) + throws IllegalArgumentException; + private native long keepLogFileNum(long handle); + private native void setRecycleLogFileNum(long handle, long recycleLogFileNum); + private native long recycleLogFileNum(long handle); + private native void setMaxManifestFileSize( + long handle, long maxManifestFileSize); + private native long maxManifestFileSize(long handle); + private native void setMaxTableFilesSizeFIFO( + long handle, long maxTableFilesSize); + private native long maxTableFilesSizeFIFO(long handle); + private native void setTableCacheNumshardbits( + long handle, int tableCacheNumshardbits); + private native int tableCacheNumshardbits(long handle); + private native void setWalTtlSeconds(long handle, long walTtlSeconds); + private native long walTtlSeconds(long handle); + private native void setWalSizeLimitMB(long handle, long sizeLimitMB); + private native long walSizeLimitMB(long handle); + private native void setManifestPreallocationSize( + long handle, long size) throws IllegalArgumentException; + private native long manifestPreallocationSize(long handle); + private native void setUseDirectReads(long handle, boolean useDirectReads); + private native boolean useDirectReads(long handle); + private native void setUseDirectIoForFlushAndCompaction( + long handle, boolean useDirectIoForFlushAndCompaction); + private native boolean useDirectIoForFlushAndCompaction(long handle); + private native void setAllowFAllocate(final long handle, + final boolean allowFAllocate); + private native boolean allowFAllocate(final long handle); + private native void setAllowMmapReads( + long handle, boolean allowMmapReads); + private native boolean allowMmapReads(long handle); + private native void setAllowMmapWrites( + long handle, boolean allowMmapWrites); + private native boolean allowMmapWrites(long handle); + private native void setIsFdCloseOnExec( + long handle, boolean isFdCloseOnExec); + private native boolean isFdCloseOnExec(long handle); + private native void setStatsDumpPeriodSec( + long handle, int statsDumpPeriodSec); + private native int statsDumpPeriodSec(long handle); + private native void setAdviseRandomOnOpen( + long handle, boolean adviseRandomOnOpen); + private native boolean adviseRandomOnOpen(long handle); + private native void setDbWriteBufferSize(final long handle, + final long dbWriteBufferSize); + private native long dbWriteBufferSize(final long handle); + private native void setAccessHintOnCompactionStart(final long handle, + final byte accessHintOnCompactionStart); + private native byte accessHintOnCompactionStart(final long handle); + private native void setNewTableReaderForCompactionInputs(final long handle, + final boolean newTableReaderForCompactionInputs); + private native boolean newTableReaderForCompactionInputs(final long handle); + private native void setCompactionReadaheadSize(final long handle, + final long compactionReadaheadSize); + private native long compactionReadaheadSize(final long handle); + private native void setRandomAccessMaxBufferSize(final long handle, + final long randomAccessMaxBufferSize); + private native long randomAccessMaxBufferSize(final long handle); + private native void setWritableFileMaxBufferSize(final long handle, + final long writableFileMaxBufferSize); + private native long writableFileMaxBufferSize(final long handle); + private native void setUseAdaptiveMutex( + long handle, boolean useAdaptiveMutex); + private native boolean useAdaptiveMutex(long handle); + private native void setBytesPerSync( + long handle, long bytesPerSync); + private native long bytesPerSync(long handle); + private native void setWalBytesPerSync(long handle, long walBytesPerSync); + private native long walBytesPerSync(long handle); + private native void setEnableThreadTracking(long handle, + boolean enableThreadTracking); + private native boolean enableThreadTracking(long handle); + private native void setDelayedWriteRate(long handle, long delayedWriteRate); + private native long delayedWriteRate(long handle); + private native void setAllowConcurrentMemtableWrite(long handle, + boolean allowConcurrentMemtableWrite); + private native boolean allowConcurrentMemtableWrite(long handle); + private native void setEnableWriteThreadAdaptiveYield(long handle, + boolean enableWriteThreadAdaptiveYield); + private native boolean enableWriteThreadAdaptiveYield(long handle); + private native void setWriteThreadMaxYieldUsec(long handle, + long writeThreadMaxYieldUsec); + private native long writeThreadMaxYieldUsec(long handle); + private native void setWriteThreadSlowYieldUsec(long handle, + long writeThreadSlowYieldUsec); + private native long writeThreadSlowYieldUsec(long handle); + private native void setSkipStatsUpdateOnDbOpen(final long handle, + final boolean skipStatsUpdateOnDbOpen); + private native boolean skipStatsUpdateOnDbOpen(final long handle); + private native void setWalRecoveryMode(final long handle, + final byte walRecoveryMode); + private native byte walRecoveryMode(final long handle); + private native void setAllow2pc(final long handle, + final boolean allow2pc); + private native boolean allow2pc(final long handle); + private native void setRowCache(final long handle, + final long row_cache_handle); + private native void setFailIfOptionsFileError(final long handle, + final boolean failIfOptionsFileError); + private native boolean failIfOptionsFileError(final long handle); + private native void setDumpMallocStats(final long handle, + final boolean dumpMallocStats); + private native boolean dumpMallocStats(final long handle); + private native void setAvoidFlushDuringRecovery(final long handle, + final boolean avoidFlushDuringRecovery); + private native boolean avoidFlushDuringRecovery(final long handle); + private native void setAvoidFlushDuringShutdown(final long handle, + final boolean avoidFlushDuringShutdown); + private native boolean avoidFlushDuringShutdown(final long handle); + + // CF native handles + private native void optimizeForSmallDb(final long handle); + private native void optimizeForPointLookup(long handle, + long blockCacheSizeMb); + private native void optimizeLevelStyleCompaction(long handle, + long memtableMemoryBudget); + private native void optimizeUniversalStyleCompaction(long handle, + long memtableMemoryBudget); + private native void setComparatorHandle(long handle, int builtinComparator); + private native void setComparatorHandle(long optHandle, + long comparatorHandle); + private native void setMergeOperatorName( + long handle, String name); + private native void setMergeOperator( + long handle, long mergeOperatorHandle); + private native void setWriteBufferSize(long handle, long writeBufferSize) + throws IllegalArgumentException; + private native long writeBufferSize(long handle); + private native void setMaxWriteBufferNumber( + long handle, int maxWriteBufferNumber); + private native int maxWriteBufferNumber(long handle); + private native void setMinWriteBufferNumberToMerge( + long handle, int minWriteBufferNumberToMerge); + private native int minWriteBufferNumberToMerge(long handle); + private native void setCompressionType(long handle, byte compressionType); + private native byte compressionType(long handle); + private native void setCompressionPerLevel(long handle, + byte[] compressionLevels); + private native byte[] compressionPerLevel(long handle); + private native void setBottommostCompressionType(long handle, + byte bottommostCompressionType); + private native byte bottommostCompressionType(long handle); + private native void setCompressionOptions(long handle, + long compressionOptionsHandle); + private native void useFixedLengthPrefixExtractor( + long handle, int prefixLength); + private native void useCappedPrefixExtractor( + long handle, int prefixLength); + private native void setNumLevels( + long handle, int numLevels); + private native int numLevels(long handle); + private native void setLevelZeroFileNumCompactionTrigger( + long handle, int numFiles); + private native int levelZeroFileNumCompactionTrigger(long handle); + private native void setLevelZeroSlowdownWritesTrigger( + long handle, int numFiles); + private native int levelZeroSlowdownWritesTrigger(long handle); + private native void setLevelZeroStopWritesTrigger( + long handle, int numFiles); + private native int levelZeroStopWritesTrigger(long handle); + private native void setTargetFileSizeBase( + long handle, long targetFileSizeBase); + private native long targetFileSizeBase(long handle); + private native void setTargetFileSizeMultiplier( + long handle, int multiplier); + private native int targetFileSizeMultiplier(long handle); + private native void setMaxBytesForLevelBase( + long handle, long maxBytesForLevelBase); + private native long maxBytesForLevelBase(long handle); + private native void setLevelCompactionDynamicLevelBytes( + long handle, boolean enableLevelCompactionDynamicLevelBytes); + private native boolean levelCompactionDynamicLevelBytes( + long handle); + private native void setMaxBytesForLevelMultiplier(long handle, double multiplier); + private native double maxBytesForLevelMultiplier(long handle); + private native void setMaxCompactionBytes(long handle, long maxCompactionBytes); + private native long maxCompactionBytes(long handle); + private native void setArenaBlockSize( + long handle, long arenaBlockSize) throws IllegalArgumentException; + private native long arenaBlockSize(long handle); + private native void setDisableAutoCompactions( + long handle, boolean disableAutoCompactions); + private native boolean disableAutoCompactions(long handle); + private native void setCompactionStyle(long handle, byte compactionStyle); + private native byte compactionStyle(long handle); + private native void setMaxSequentialSkipInIterations( + long handle, long maxSequentialSkipInIterations); + private native long maxSequentialSkipInIterations(long handle); + private native void setMemTableFactory(long handle, long factoryHandle); + private native String memTableFactoryName(long handle); + private native void setTableFactory(long handle, long factoryHandle); + private native String tableFactoryName(long handle); + private native void setInplaceUpdateSupport( + long handle, boolean inplaceUpdateSupport); + private native boolean inplaceUpdateSupport(long handle); + private native void setInplaceUpdateNumLocks( + long handle, long inplaceUpdateNumLocks) + throws IllegalArgumentException; + private native long inplaceUpdateNumLocks(long handle); + private native void setMemtablePrefixBloomSizeRatio( + long handle, double memtablePrefixBloomSizeRatio); + private native double memtablePrefixBloomSizeRatio(long handle); + private native void setBloomLocality( + long handle, int bloomLocality); + private native int bloomLocality(long handle); + private native void setMaxSuccessiveMerges( + long handle, long maxSuccessiveMerges) + throws IllegalArgumentException; + private native long maxSuccessiveMerges(long handle); + private native void setOptimizeFiltersForHits(long handle, + boolean optimizeFiltersForHits); + private native boolean optimizeFiltersForHits(long handle); + private native void setMemtableHugePageSize(long handle, + long memtableHugePageSize); + private native long memtableHugePageSize(long handle); + private native void setSoftPendingCompactionBytesLimit(long handle, + long softPendingCompactionBytesLimit); + private native long softPendingCompactionBytesLimit(long handle); + private native void setHardPendingCompactionBytesLimit(long handle, + long hardPendingCompactionBytesLimit); + private native long hardPendingCompactionBytesLimit(long handle); + private native void setLevel0FileNumCompactionTrigger(long handle, + int level0FileNumCompactionTrigger); + private native int level0FileNumCompactionTrigger(long handle); + private native void setLevel0SlowdownWritesTrigger(long handle, + int level0SlowdownWritesTrigger); + private native int level0SlowdownWritesTrigger(long handle); + private native void setLevel0StopWritesTrigger(long handle, + int level0StopWritesTrigger); + private native int level0StopWritesTrigger(long handle); + private native void setMaxBytesForLevelMultiplierAdditional(long handle, + int[] maxBytesForLevelMultiplierAdditional); + private native int[] maxBytesForLevelMultiplierAdditional(long handle); + private native void setParanoidFileChecks(long handle, + boolean paranoidFileChecks); + private native boolean paranoidFileChecks(long handle); + private native void setMaxWriteBufferNumberToMaintain(final long handle, + final int maxWriteBufferNumberToMaintain); + private native int maxWriteBufferNumberToMaintain(final long handle); + private native void setCompactionPriority(final long handle, + final byte compactionPriority); + private native byte compactionPriority(final long handle); + private native void setReportBgIoStats(final long handle, + final boolean reportBgIoStats); + private native boolean reportBgIoStats(final long handle); + private native void setCompactionOptionsUniversal(final long handle, + final long compactionOptionsUniversalHandle); + private native void setCompactionOptionsFIFO(final long handle, + final long compactionOptionsFIFOHandle); + private native void setForceConsistencyChecks(final long handle, + final boolean forceConsistencyChecks); + private native boolean forceConsistencyChecks(final long handle); + + // instance variables + private Env env_; + private MemTableConfig memTableConfig_; + private TableFormatConfig tableFormatConfig_; + private RateLimiter rateLimiter_; + private AbstractComparator> comparator_; + private CompactionOptionsUniversal compactionOptionsUniversal_; + private CompactionOptionsFIFO compactionOptionsFIFO_; + private CompressionOptions compressionOptions_; + private Cache rowCache_; +} diff --git a/java/src/main/java/org/rocksdb/PlainTableConfig.java b/java/src/main/java/org/rocksdb/PlainTableConfig.java new file mode 100644 index 00000000000..c099981678b --- /dev/null +++ b/java/src/main/java/org/rocksdb/PlainTableConfig.java @@ -0,0 +1,251 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +/** + * The config for plain table sst format. + * + *

    PlainTable is a RocksDB's SST file format optimized for low query + * latency on pure-memory or really low-latency media.

    + * + *

    It also support prefix hash feature.

    + */ +public class PlainTableConfig extends TableFormatConfig { + public static final int VARIABLE_LENGTH = 0; + public static final int DEFAULT_BLOOM_BITS_PER_KEY = 10; + public static final double DEFAULT_HASH_TABLE_RATIO = 0.75; + public static final int DEFAULT_INDEX_SPARSENESS = 16; + public static final int DEFAULT_HUGE_TLB_SIZE = 0; + public static final EncodingType DEFAULT_ENCODING_TYPE = + EncodingType.kPlain; + public static final boolean DEFAULT_FULL_SCAN_MODE = false; + public static final boolean DEFAULT_STORE_INDEX_IN_FILE + = false; + + public PlainTableConfig() { + keySize_ = VARIABLE_LENGTH; + bloomBitsPerKey_ = DEFAULT_BLOOM_BITS_PER_KEY; + hashTableRatio_ = DEFAULT_HASH_TABLE_RATIO; + indexSparseness_ = DEFAULT_INDEX_SPARSENESS; + hugePageTlbSize_ = DEFAULT_HUGE_TLB_SIZE; + encodingType_ = DEFAULT_ENCODING_TYPE; + fullScanMode_ = DEFAULT_FULL_SCAN_MODE; + storeIndexInFile_ = DEFAULT_STORE_INDEX_IN_FILE; + } + + /** + *

    Set the length of the user key. If it is set to be + * VARIABLE_LENGTH, then it indicates the user keys are + * of variable length.

    + * + *

    Otherwise,all the keys need to have the same length + * in byte.

    + * + *

    DEFAULT: VARIABLE_LENGTH

    + * + * @param keySize the length of the user key. + * @return the reference to the current config. + */ + public PlainTableConfig setKeySize(int keySize) { + keySize_ = keySize; + return this; + } + + /** + * @return the specified size of the user key. If VARIABLE_LENGTH, + * then it indicates variable-length key. + */ + public int keySize() { + return keySize_; + } + + /** + * Set the number of bits per key used by the internal bloom filter + * in the plain table sst format. + * + * @param bitsPerKey the number of bits per key for bloom filer. + * @return the reference to the current config. + */ + public PlainTableConfig setBloomBitsPerKey(int bitsPerKey) { + bloomBitsPerKey_ = bitsPerKey; + return this; + } + + /** + * @return the number of bits per key used for the bloom filter. + */ + public int bloomBitsPerKey() { + return bloomBitsPerKey_; + } + + /** + * hashTableRatio is the desired utilization of the hash table used + * for prefix hashing. The ideal ratio would be the number of + * prefixes / the number of hash buckets. If this value is set to + * zero, then hash table will not be used. + * + * @param ratio the hash table ratio. + * @return the reference to the current config. + */ + public PlainTableConfig setHashTableRatio(double ratio) { + hashTableRatio_ = ratio; + return this; + } + + /** + * @return the hash table ratio. + */ + public double hashTableRatio() { + return hashTableRatio_; + } + + /** + * Index sparseness determines the index interval for keys inside the + * same prefix. This number is equal to the maximum number of linear + * search required after hash and binary search. If it's set to 0, + * then each key will be indexed. + * + * @param sparseness the index sparseness. + * @return the reference to the current config. + */ + public PlainTableConfig setIndexSparseness(int sparseness) { + indexSparseness_ = sparseness; + return this; + } + + /** + * @return the index sparseness. + */ + public long indexSparseness() { + return indexSparseness_; + } + + /** + *

    huge_page_tlb_size: if ≤0, allocate hash indexes and blooms + * from malloc otherwise from huge page TLB.

    + * + *

    The user needs to reserve huge pages for it to be allocated, + * like: {@code sysctl -w vm.nr_hugepages=20}

    + * + *

    See linux doc Documentation/vm/hugetlbpage.txt

    + * + * @param hugePageTlbSize huge page tlb size + * @return the reference to the current config. + */ + public PlainTableConfig setHugePageTlbSize(int hugePageTlbSize) { + this.hugePageTlbSize_ = hugePageTlbSize; + return this; + } + + /** + * Returns the value for huge page tlb size + * + * @return hugePageTlbSize + */ + public int hugePageTlbSize() { + return hugePageTlbSize_; + } + + /** + * Sets the encoding type. + * + *

    This setting determines how to encode + * the keys. See enum {@link EncodingType} for + * the choices.

    + * + *

    The value will determine how to encode keys + * when writing to a new SST file. This value will be stored + * inside the SST file which will be used when reading from + * the file, which makes it possible for users to choose + * different encoding type when reopening a DB. Files with + * different encoding types can co-exist in the same DB and + * can be read.

    + * + * @param encodingType {@link org.rocksdb.EncodingType} value. + * @return the reference to the current config. + */ + public PlainTableConfig setEncodingType(EncodingType encodingType) { + this.encodingType_ = encodingType; + return this; + } + + /** + * Returns the active EncodingType + * + * @return currently set encoding type + */ + public EncodingType encodingType() { + return encodingType_; + } + + /** + * Set full scan mode, if true the whole file will be read + * one record by one without using the index. + * + * @param fullScanMode boolean value indicating if full + * scan mode shall be enabled. + * @return the reference to the current config. + */ + public PlainTableConfig setFullScanMode(boolean fullScanMode) { + this.fullScanMode_ = fullScanMode; + return this; + } + + /** + * Return if full scan mode is active + * @return boolean value indicating if the full scan mode is + * enabled. + */ + public boolean fullScanMode() { + return fullScanMode_; + } + + /** + *

    If set to true: compute plain table index and bloom + * filter during file building and store it in file. + * When reading file, index will be mmaped instead + * of doing recomputation.

    + * + * @param storeIndexInFile value indicating if index shall + * be stored in a file + * @return the reference to the current config. + */ + public PlainTableConfig setStoreIndexInFile(boolean storeIndexInFile) { + this.storeIndexInFile_ = storeIndexInFile; + return this; + } + + /** + * Return a boolean value indicating if index shall be stored + * in a file. + * + * @return currently set value for store index in file. + */ + public boolean storeIndexInFile() { + return storeIndexInFile_; + } + + @Override protected long newTableFactoryHandle() { + return newTableFactoryHandle(keySize_, bloomBitsPerKey_, + hashTableRatio_, indexSparseness_, hugePageTlbSize_, + encodingType_.getValue(), fullScanMode_, + storeIndexInFile_); + } + + private native long newTableFactoryHandle( + int keySize, int bloomBitsPerKey, + double hashTableRatio, int indexSparseness, + int hugePageTlbSize, byte encodingType, + boolean fullScanMode, boolean storeIndexInFile); + + private int keySize_; + private int bloomBitsPerKey_; + private double hashTableRatio_; + private int indexSparseness_; + private int hugePageTlbSize_; + private EncodingType encodingType_; + private boolean fullScanMode_; + private boolean storeIndexInFile_; +} diff --git a/java/src/main/java/org/rocksdb/RateLimiter.java b/java/src/main/java/org/rocksdb/RateLimiter.java new file mode 100644 index 00000000000..fc2388777e9 --- /dev/null +++ b/java/src/main/java/org/rocksdb/RateLimiter.java @@ -0,0 +1,119 @@ +// Copyright (c) 2015, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * RateLimiter, which is used to control write rate of flush and + * compaction. + * + * @since 3.10.0 + */ +public class RateLimiter extends RocksObject { + private static final long DEFAULT_REFILL_PERIOD_MICROS = (100 * 1000); + private static final int DEFAULT_FAIRNESS = 10; + + /** + * RateLimiter constructor + * + * @param rateBytesPerSecond this is the only parameter you want to set + * most of the time. It controls the total write rate of compaction + * and flush in bytes per second. Currently, RocksDB does not enforce + * rate limit for anything other than flush and compaction, e.g. write to WAL. + * @param refillPeriodMicros this controls how often tokens are refilled. For example, + * when rate_bytes_per_sec is set to 10MB/s and refill_period_us is set to + * 100ms, then 1MB is refilled every 100ms internally. Larger value can lead to + * burstier writes while smaller value introduces more CPU overhead. + * The default should work for most cases. + * @param fairness RateLimiter accepts high-pri requests and low-pri requests. + * A low-pri request is usually blocked in favor of hi-pri request. Currently, + * RocksDB assigns low-pri to request from compaction and high-pri to request + * from flush. Low-pri requests can get blocked if flush requests come in + * continuously. This fairness parameter grants low-pri requests permission by + * fairness chance even though high-pri requests exist to avoid starvation. + * You should be good by leaving it at default 10. + */ + public RateLimiter(final long rateBytesPerSecond, + final long refillPeriodMicros, final int fairness) { + super(newRateLimiterHandle(rateBytesPerSecond, + refillPeriodMicros, fairness)); + } + + /** + * RateLimiter constructor + * + * @param rateBytesPerSecond this is the only parameter you want to set + * most of the time. It controls the total write rate of compaction + * and flush in bytes per second. Currently, RocksDB does not enforce + * rate limit for anything other than flush and compaction, e.g. write to WAL. + */ + public RateLimiter(final long rateBytesPerSecond) { + this(rateBytesPerSecond, DEFAULT_REFILL_PERIOD_MICROS, DEFAULT_FAIRNESS); + } + + /** + *

    This API allows user to dynamically change rate limiter's bytes per second. + * REQUIRED: bytes_per_second > 0

    + * + * @param bytesPerSecond bytes per second. + */ + public void setBytesPerSecond(final long bytesPerSecond) { + assert(isOwningHandle()); + setBytesPerSecond(nativeHandle_, bytesPerSecond); + } + + /** + *

    Request for token to write bytes. If this request can not be satisfied, + * the call is blocked. Caller is responsible to make sure + * {@code bytes < GetSingleBurstBytes()}.

    + * + * @param bytes requested bytes. + */ + public void request(final long bytes) { + assert(isOwningHandle()); + request(nativeHandle_, bytes); + } + + /** + *

    Max bytes can be granted in a single burst.

    + * + * @return max bytes can be granted in a single burst. + */ + public long getSingleBurstBytes() { + assert(isOwningHandle()); + return getSingleBurstBytes(nativeHandle_); + } + + /** + *

    Total bytes that go though rate limiter.

    + * + * @return total bytes that go though rate limiter. + */ + public long getTotalBytesThrough() { + assert(isOwningHandle()); + return getTotalBytesThrough(nativeHandle_); + } + + /** + *

    Total # of requests that go though rate limiter.

    + * + * @return total # of requests that go though rate limiter. + */ + public long getTotalRequests() { + assert(isOwningHandle()); + return getTotalRequests(nativeHandle_); + } + + private static native long newRateLimiterHandle(final long rateBytesPerSecond, + final long refillPeriodMicros, final int fairness); + @Override protected final native void disposeInternal(final long handle); + + private native void setBytesPerSecond(final long handle, + final long bytesPerSecond); + private native void request(final long handle, final long bytes); + private native long getSingleBurstBytes(final long handle); + private native long getTotalBytesThrough(final long handle); + private native long getTotalRequests(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/ReadOptions.java b/java/src/main/java/org/rocksdb/ReadOptions.java new file mode 100644 index 00000000000..9d7b999561b --- /dev/null +++ b/java/src/main/java/org/rocksdb/ReadOptions.java @@ -0,0 +1,397 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * The class that controls the get behavior. + * + * Note that dispose() must be called before an Options instance + * become out-of-scope to release the allocated memory in c++. + */ +public class ReadOptions extends RocksObject { + public ReadOptions() { + super(newReadOptions()); + } + + /** + * If true, all data read from underlying storage will be + * verified against corresponding checksums. + * Default: true + * + * @return true if checksum verification is on. + */ + public boolean verifyChecksums() { + assert(isOwningHandle()); + return verifyChecksums(nativeHandle_); + } + + /** + * If true, all data read from underlying storage will be + * verified against corresponding checksums. + * Default: true + * + * @param verifyChecksums if true, then checksum verification + * will be performed on every read. + * @return the reference to the current ReadOptions. + */ + public ReadOptions setVerifyChecksums( + final boolean verifyChecksums) { + assert(isOwningHandle()); + setVerifyChecksums(nativeHandle_, verifyChecksums); + return this; + } + + // TODO(yhchiang): this option seems to be block-based table only. + // move this to a better place? + /** + * Fill the cache when loading the block-based sst formated db. + * Callers may wish to set this field to false for bulk scans. + * Default: true + * + * @return true if the fill-cache behavior is on. + */ + public boolean fillCache() { + assert(isOwningHandle()); + return fillCache(nativeHandle_); + } + + /** + * Fill the cache when loading the block-based sst formatted db. + * Callers may wish to set this field to false for bulk scans. + * Default: true + * + * @param fillCache if true, then fill-cache behavior will be + * performed. + * @return the reference to the current ReadOptions. + */ + public ReadOptions setFillCache(final boolean fillCache) { + assert(isOwningHandle()); + setFillCache(nativeHandle_, fillCache); + return this; + } + + /** + * Returns the currently assigned Snapshot instance. + * + * @return the Snapshot assigned to this instance. If no Snapshot + * is assigned null. + */ + public Snapshot snapshot() { + assert(isOwningHandle()); + long snapshotHandle = snapshot(nativeHandle_); + if (snapshotHandle != 0) { + return new Snapshot(snapshotHandle); + } + return null; + } + + /** + *

    If "snapshot" is non-nullptr, read as of the supplied snapshot + * (which must belong to the DB that is being read and which must + * not have been released). If "snapshot" is nullptr, use an implicit + * snapshot of the state at the beginning of this read operation.

    + *

    Default: null

    + * + * @param snapshot {@link Snapshot} instance + * @return the reference to the current ReadOptions. + */ + public ReadOptions setSnapshot(final Snapshot snapshot) { + assert(isOwningHandle()); + if (snapshot != null) { + setSnapshot(nativeHandle_, snapshot.nativeHandle_); + } else { + setSnapshot(nativeHandle_, 0l); + } + return this; + } + + /** + * Returns the current read tier. + * + * @return the read tier in use, by default {@link ReadTier#READ_ALL_TIER} + */ + public ReadTier readTier() { + assert(isOwningHandle()); + return ReadTier.getReadTier(readTier(nativeHandle_)); + } + + /** + * Specify if this read request should process data that ALREADY + * resides on a particular cache. If the required data is not + * found at the specified cache, then {@link RocksDBException} is thrown. + * + * @param readTier {@link ReadTier} instance + * @return the reference to the current ReadOptions. + */ + public ReadOptions setReadTier(final ReadTier readTier) { + assert(isOwningHandle()); + setReadTier(nativeHandle_, readTier.getValue()); + return this; + } + + /** + * Specify to create a tailing iterator -- a special iterator that has a + * view of the complete database (i.e. it can also be used to read newly + * added data) and is optimized for sequential reads. It will return records + * that were inserted into the database after the creation of the iterator. + * Default: false + * + * Not supported in {@code ROCKSDB_LITE} mode! + * + * @return true if tailing iterator is enabled. + */ + public boolean tailing() { + assert(isOwningHandle()); + return tailing(nativeHandle_); + } + + /** + * Specify to create a tailing iterator -- a special iterator that has a + * view of the complete database (i.e. it can also be used to read newly + * added data) and is optimized for sequential reads. It will return records + * that were inserted into the database after the creation of the iterator. + * Default: false + * Not supported in ROCKSDB_LITE mode! + * + * @param tailing if true, then tailing iterator will be enabled. + * @return the reference to the current ReadOptions. + */ + public ReadOptions setTailing(final boolean tailing) { + assert(isOwningHandle()); + setTailing(nativeHandle_, tailing); + return this; + } + + /** + * Returns whether managed iterators will be used. + * + * @return the setting of whether managed iterators will be used, by default false + */ + public boolean managed() { + assert(isOwningHandle()); + return managed(nativeHandle_); + } + + /** + * Specify to create a managed iterator -- a special iterator that + * uses less resources by having the ability to free its underlying + * resources on request. + * + * @param managed if true, then managed iterators will be enabled. + * @return the reference to the current ReadOptions. + */ + public ReadOptions setManaged(final boolean managed) { + assert(isOwningHandle()); + setManaged(nativeHandle_, managed); + return this; + } + + /** + * Returns whether a total seek order will be used + * + * @return the setting of whether a total seek order will be used + */ + public boolean totalOrderSeek() { + assert(isOwningHandle()); + return totalOrderSeek(nativeHandle_); + } + + /** + * Enable a total order seek regardless of index format (e.g. hash index) + * used in the table. Some table format (e.g. plain table) may not support + * this option. + * + * @param totalOrderSeek if true, then total order seek will be enabled. + * @return the reference to the current ReadOptions. + */ + public ReadOptions setTotalOrderSeek(final boolean totalOrderSeek) { + assert(isOwningHandle()); + setTotalOrderSeek(nativeHandle_, totalOrderSeek); + return this; + } + + /** + * Returns whether the iterator only iterates over the same prefix as the seek + * + * @return the setting of whether the iterator only iterates over the same + * prefix as the seek, default is false + */ + public boolean prefixSameAsStart() { + assert(isOwningHandle()); + return prefixSameAsStart(nativeHandle_); + } + + + /** + * Enforce that the iterator only iterates over the same prefix as the seek. + * This option is effective only for prefix seeks, i.e. prefix_extractor is + * non-null for the column family and {@link #totalOrderSeek()} is false. + * Unlike iterate_upper_bound, {@link #setPrefixSameAsStart(boolean)} only + * works within a prefix but in both directions. + * + * @param prefixSameAsStart if true, then the iterator only iterates over the + * same prefix as the seek + * @return the reference to the current ReadOptions. + */ + public ReadOptions setPrefixSameAsStart(final boolean prefixSameAsStart) { + assert(isOwningHandle()); + setPrefixSameAsStart(nativeHandle_, prefixSameAsStart); + return this; + } + + /** + * Returns whether the blocks loaded by the iterator will be pinned in memory + * + * @return the setting of whether the blocks loaded by the iterator will be + * pinned in memory + */ + public boolean pinData() { + assert(isOwningHandle()); + return pinData(nativeHandle_); + } + + /** + * Keep the blocks loaded by the iterator pinned in memory as long as the + * iterator is not deleted, If used when reading from tables created with + * BlockBasedTableOptions::use_delta_encoding = false, + * Iterator's property "rocksdb.iterator.is-key-pinned" is guaranteed to + * return 1. + * + * @param pinData if true, the blocks loaded by the iterator will be pinned + * @return the reference to the current ReadOptions. + */ + public ReadOptions setPinData(final boolean pinData) { + assert(isOwningHandle()); + setPinData(nativeHandle_, pinData); + return this; + } + + /** + * If true, when PurgeObsoleteFile is called in CleanupIteratorState, we + * schedule a background job in the flush job queue and delete obsolete files + * in background. + * + * Default: false + * + * @return true when PurgeObsoleteFile is called in CleanupIteratorState + */ + public boolean backgroundPurgeOnIteratorCleanup() { + assert(isOwningHandle()); + return backgroundPurgeOnIteratorCleanup(nativeHandle_); + } + + /** + * If true, when PurgeObsoleteFile is called in CleanupIteratorState, we + * schedule a background job in the flush job queue and delete obsolete files + * in background. + * + * Default: false + * + * @param backgroundPurgeOnIteratorCleanup true when PurgeObsoleteFile is + * called in CleanupIteratorState + * @return the reference to the current ReadOptions. + */ + public ReadOptions setBackgroundPurgeOnIteratorCleanup( + final boolean backgroundPurgeOnIteratorCleanup) { + assert(isOwningHandle()); + setBackgroundPurgeOnIteratorCleanup(nativeHandle_, + backgroundPurgeOnIteratorCleanup); + return this; + } + + /** + * If non-zero, NewIterator will create a new table reader which + * performs reads of the given size. Using a large size (> 2MB) can + * improve the performance of forward iteration on spinning disks. + * + * Default: 0 + * + * @return The readahead size is bytes + */ + public long readaheadSize() { + assert(isOwningHandle()); + return readaheadSize(nativeHandle_); + } + + /** + * If non-zero, NewIterator will create a new table reader which + * performs reads of the given size. Using a large size (> 2MB) can + * improve the performance of forward iteration on spinning disks. + * + * Default: 0 + * + * @param readaheadSize The readahead size is bytes + * @return the reference to the current ReadOptions. + */ + public ReadOptions setReadaheadSize(final long readaheadSize) { + assert(isOwningHandle()); + setReadaheadSize(nativeHandle_, readaheadSize); + return this; + } + + /** + * If true, keys deleted using the DeleteRange() API will be visible to + * readers until they are naturally deleted during compaction. This improves + * read performance in DBs with many range deletions. + * + * Default: false + * + * @return true if keys deleted using the DeleteRange() API will be visible + */ + public boolean ignoreRangeDeletions() { + assert(isOwningHandle()); + return ignoreRangeDeletions(nativeHandle_); + } + + /** + * If true, keys deleted using the DeleteRange() API will be visible to + * readers until they are naturally deleted during compaction. This improves + * read performance in DBs with many range deletions. + * + * Default: false + * + * @param ignoreRangeDeletions true if keys deleted using the DeleteRange() + * API should be visible + * @return the reference to the current ReadOptions. + */ + public ReadOptions setIgnoreRangeDeletions(final boolean ignoreRangeDeletions) { + assert(isOwningHandle()); + setIgnoreRangeDeletions(nativeHandle_, ignoreRangeDeletions); + return this; + } + + private native static long newReadOptions(); + private native boolean verifyChecksums(long handle); + private native void setVerifyChecksums(long handle, boolean verifyChecksums); + private native boolean fillCache(long handle); + private native void setFillCache(long handle, boolean fillCache); + private native long snapshot(long handle); + private native void setSnapshot(long handle, long snapshotHandle); + private native byte readTier(long handle); + private native void setReadTier(long handle, byte readTierValue); + private native boolean tailing(long handle); + private native void setTailing(long handle, boolean tailing); + private native boolean managed(long handle); + private native void setManaged(long handle, boolean managed); + private native boolean totalOrderSeek(long handle); + private native void setTotalOrderSeek(long handle, boolean totalOrderSeek); + private native boolean prefixSameAsStart(long handle); + private native void setPrefixSameAsStart(long handle, boolean prefixSameAsStart); + private native boolean pinData(long handle); + private native void setPinData(long handle, boolean pinData); + private native boolean backgroundPurgeOnIteratorCleanup(final long handle); + private native void setBackgroundPurgeOnIteratorCleanup(final long handle, + final boolean backgroundPurgeOnIteratorCleanup); + private native long readaheadSize(final long handle); + private native void setReadaheadSize(final long handle, + final long readaheadSize); + private native boolean ignoreRangeDeletions(final long handle); + private native void setIgnoreRangeDeletions(final long handle, + final boolean ignoreRangeDeletions); + + @Override protected final native void disposeInternal(final long handle); + +} diff --git a/java/src/main/java/org/rocksdb/ReadTier.java b/java/src/main/java/org/rocksdb/ReadTier.java new file mode 100644 index 00000000000..6dc76c52e58 --- /dev/null +++ b/java/src/main/java/org/rocksdb/ReadTier.java @@ -0,0 +1,48 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * RocksDB {@link ReadOptions} read tiers. + */ +public enum ReadTier { + READ_ALL_TIER((byte)0), + BLOCK_CACHE_TIER((byte)1), + PERSISTED_TIER((byte)2); + + private final byte value; + + ReadTier(final byte value) { + this.value = value; + } + + /** + * Returns the byte value of the enumerations value + * + * @return byte representation + */ + public byte getValue() { + return value; + } + + /** + * Get ReadTier by byte value. + * + * @param value byte representation of ReadTier. + * + * @return {@link org.rocksdb.ReadTier} instance or null. + * @throws java.lang.IllegalArgumentException if an invalid + * value is provided. + */ + public static ReadTier getReadTier(final byte value) { + for (final ReadTier readTier : ReadTier.values()) { + if (readTier.getValue() == value){ + return readTier; + } + } + throw new IllegalArgumentException("Illegal value provided for ReadTier."); + } +} diff --git a/java/src/main/java/org/rocksdb/RemoveEmptyValueCompactionFilter.java b/java/src/main/java/org/rocksdb/RemoveEmptyValueCompactionFilter.java new file mode 100644 index 00000000000..6ee81d858c8 --- /dev/null +++ b/java/src/main/java/org/rocksdb/RemoveEmptyValueCompactionFilter.java @@ -0,0 +1,18 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Just a Java wrapper around EmptyValueCompactionFilter implemented in C++ + */ +public class RemoveEmptyValueCompactionFilter + extends AbstractCompactionFilter { + public RemoveEmptyValueCompactionFilter() { + super(createNewRemoveEmptyValueCompactionFilter0()); + } + + private native static long createNewRemoveEmptyValueCompactionFilter0(); +} diff --git a/java/src/main/java/org/rocksdb/RestoreOptions.java b/java/src/main/java/org/rocksdb/RestoreOptions.java new file mode 100644 index 00000000000..94d93fc719c --- /dev/null +++ b/java/src/main/java/org/rocksdb/RestoreOptions.java @@ -0,0 +1,32 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * RestoreOptions to control the behavior of restore. + * + * Note that dispose() must be called before this instance become out-of-scope + * to release the allocated memory in c++. + * + */ +public class RestoreOptions extends RocksObject { + /** + * Constructor + * + * @param keepLogFiles If true, restore won't overwrite the existing log files + * in wal_dir. It will also move all log files from archive directory to + * wal_dir. Use this option in combination with + * BackupableDBOptions::backup_log_files = false for persisting in-memory + * databases. + * Default: false + */ + public RestoreOptions(final boolean keepLogFiles) { + super(newRestoreOptions(keepLogFiles)); + } + + private native static long newRestoreOptions(boolean keepLogFiles); + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/RocksDB.java b/java/src/main/java/org/rocksdb/RocksDB.java new file mode 100644 index 00000000000..eda0950990a --- /dev/null +++ b/java/src/main/java/org/rocksdb/RocksDB.java @@ -0,0 +1,2384 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.*; +import java.io.IOException; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicReference; + +import org.rocksdb.util.Environment; + +/** + * A RocksDB is a persistent ordered map from keys to values. It is safe for + * concurrent access from multiple threads without any external synchronization. + * All methods of this class could potentially throw RocksDBException, which + * indicates sth wrong at the RocksDB library side and the call failed. + */ +public class RocksDB extends RocksObject { + public static final byte[] DEFAULT_COLUMN_FAMILY = "default".getBytes(); + public static final int NOT_FOUND = -1; + + private enum LibraryState { + NOT_LOADED, + LOADING, + LOADED + } + + private static AtomicReference libraryLoaded + = new AtomicReference<>(LibraryState.NOT_LOADED); + + static { + RocksDB.loadLibrary(); + } + + /** + * Loads the necessary library files. + * Calling this method twice will have no effect. + * By default the method extracts the shared library for loading at + * java.io.tmpdir, however, you can override this temporary location by + * setting the environment variable ROCKSDB_SHAREDLIB_DIR. + */ + public static void loadLibrary() { + if (libraryLoaded.get() == LibraryState.LOADED) { + return; + } + + if (libraryLoaded.compareAndSet(LibraryState.NOT_LOADED, + LibraryState.LOADING)) { + final String tmpDir = System.getenv("ROCKSDB_SHAREDLIB_DIR"); + // loading possibly necessary libraries. + for (final CompressionType compressionType : CompressionType.values()) { + try { + if (compressionType.getLibraryName() != null) { + System.loadLibrary(compressionType.getLibraryName()); + } + } catch (UnsatisfiedLinkError e) { + // since it may be optional, we ignore its loading failure here. + } + } + try { + NativeLibraryLoader.getInstance().loadLibrary(tmpDir); + } catch (IOException e) { + libraryLoaded.set(LibraryState.NOT_LOADED); + throw new RuntimeException("Unable to load the RocksDB shared library" + + e); + } + + libraryLoaded.set(LibraryState.LOADED); + return; + } + + while (libraryLoaded.get() == LibraryState.LOADING) { + try { + Thread.sleep(10); + } catch(final InterruptedException e) { + //ignore + } + } + } + + /** + * Tries to load the necessary library files from the given list of + * directories. + * + * @param paths a list of strings where each describes a directory + * of a library. + */ + public static void loadLibrary(final List paths) { + if (libraryLoaded.get() == LibraryState.LOADED) { + return; + } + + if (libraryLoaded.compareAndSet(LibraryState.NOT_LOADED, + LibraryState.LOADING)) { + for (final CompressionType compressionType : CompressionType.values()) { + if (compressionType.equals(CompressionType.NO_COMPRESSION)) { + continue; + } + for (final String path : paths) { + try { + System.load(path + "/" + Environment.getSharedLibraryFileName( + compressionType.getLibraryName())); + break; + } catch (UnsatisfiedLinkError e) { + // since they are optional, we ignore loading fails. + } + } + } + boolean success = false; + UnsatisfiedLinkError err = null; + for (final String path : paths) { + try { + System.load(path + "/" + + Environment.getJniLibraryFileName("rocksdbjni")); + success = true; + break; + } catch (UnsatisfiedLinkError e) { + err = e; + } + } + if (!success) { + libraryLoaded.set(LibraryState.NOT_LOADED); + throw err; + } + + libraryLoaded.set(LibraryState.LOADED); + return; + } + + while (libraryLoaded.get() == LibraryState.LOADING) { + try { + Thread.sleep(10); + } catch(final InterruptedException e) { + //ignore + } + } + } + + /** + * The factory constructor of RocksDB that opens a RocksDB instance given + * the path to the database using the default options w/ createIfMissing + * set to true. + * + * @param path the path to the rocksdb. + * @return a {@link RocksDB} instance on success, null if the specified + * {@link RocksDB} can not be opened. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * @see Options#setCreateIfMissing(boolean) + */ + public static RocksDB open(final String path) throws RocksDBException { + // This allows to use the rocksjni default Options instead of + // the c++ one. + Options options = new Options(); + options.setCreateIfMissing(true); + return open(options, path); + } + + /** + * The factory constructor of RocksDB that opens a RocksDB instance given + * the path to the database using the specified options and db path and a list + * of column family names. + *

    + * If opened in read write mode every existing column family name must be + * passed within the list to this method.

    + *

    + * If opened in read-only mode only a subset of existing column families must + * be passed to this method.

    + *

    + * Options instance *should* not be disposed before all DBs using this options + * instance have been closed. If user doesn't call options dispose explicitly, + * then this options instance will be GC'd automatically

    + *

    + * ColumnFamily handles are disposed when the RocksDB instance is disposed. + *

    + * + * @param path the path to the rocksdb. + * @param columnFamilyDescriptors list of column family descriptors + * @param columnFamilyHandles will be filled with ColumnFamilyHandle instances + * on open. + * @return a {@link RocksDB} instance on success, null if the specified + * {@link RocksDB} can not be opened. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * @see DBOptions#setCreateIfMissing(boolean) + */ + public static RocksDB open(final String path, + final List columnFamilyDescriptors, + final List columnFamilyHandles) + throws RocksDBException { + // This allows to use the rocksjni default Options instead of + // the c++ one. + DBOptions options = new DBOptions(); + return open(options, path, columnFamilyDescriptors, columnFamilyHandles); + } + + /** + * The factory constructor of RocksDB that opens a RocksDB instance given + * the path to the database using the specified options and db path. + * + *

    + * Options instance *should* not be disposed before all DBs using this options + * instance have been closed. If user doesn't call options dispose explicitly, + * then this options instance will be GC'd automatically.

    + *

    + * Options instance can be re-used to open multiple DBs if DB statistics is + * not used. If DB statistics are required, then its recommended to open DB + * with new Options instance as underlying native statistics instance does not + * use any locks to prevent concurrent updates.

    + * + * @param options {@link org.rocksdb.Options} instance. + * @param path the path to the rocksdb. + * @return a {@link RocksDB} instance on success, null if the specified + * {@link RocksDB} can not be opened. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * + * @see Options#setCreateIfMissing(boolean) + */ + public static RocksDB open(final Options options, final String path) + throws RocksDBException { + // when non-default Options is used, keeping an Options reference + // in RocksDB can prevent Java to GC during the life-time of + // the currently-created RocksDB. + final RocksDB db = new RocksDB(open(options.nativeHandle_, path)); + db.storeOptionsInstance(options); + return db; + } + + /** + * The factory constructor of RocksDB that opens a RocksDB instance given + * the path to the database using the specified options and db path and a list + * of column family names. + *

    + * If opened in read write mode every existing column family name must be + * passed within the list to this method.

    + *

    + * If opened in read-only mode only a subset of existing column families must + * be passed to this method.

    + *

    + * Options instance *should* not be disposed before all DBs using this options + * instance have been closed. If user doesn't call options dispose explicitly, + * then this options instance will be GC'd automatically.

    + *

    + * Options instance can be re-used to open multiple DBs if DB statistics is + * not used. If DB statistics are required, then its recommended to open DB + * with new Options instance as underlying native statistics instance does not + * use any locks to prevent concurrent updates.

    + *

    + * ColumnFamily handles are disposed when the RocksDB instance is disposed. + *

    + * + * @param options {@link org.rocksdb.DBOptions} instance. + * @param path the path to the rocksdb. + * @param columnFamilyDescriptors list of column family descriptors + * @param columnFamilyHandles will be filled with ColumnFamilyHandle instances + * on open. + * @return a {@link RocksDB} instance on success, null if the specified + * {@link RocksDB} can not be opened. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * + * @see DBOptions#setCreateIfMissing(boolean) + */ + public static RocksDB open(final DBOptions options, final String path, + final List columnFamilyDescriptors, + final List columnFamilyHandles) + throws RocksDBException { + + final byte[][] cfNames = new byte[columnFamilyDescriptors.size()][]; + final long[] cfOptionHandles = new long[columnFamilyDescriptors.size()]; + for (int i = 0; i < columnFamilyDescriptors.size(); i++) { + final ColumnFamilyDescriptor cfDescriptor = columnFamilyDescriptors + .get(i); + cfNames[i] = cfDescriptor.columnFamilyName(); + cfOptionHandles[i] = cfDescriptor.columnFamilyOptions().nativeHandle_; + } + + final long[] handles = open(options.nativeHandle_, path, cfNames, + cfOptionHandles); + final RocksDB db = new RocksDB(handles[0]); + db.storeOptionsInstance(options); + + for (int i = 1; i < handles.length; i++) { + columnFamilyHandles.add(new ColumnFamilyHandle(db, handles[i])); + } + + return db; + } + + /** + * The factory constructor of RocksDB that opens a RocksDB instance in + * Read-Only mode given the path to the database using the default + * options. + * + * @param path the path to the RocksDB. + * @return a {@link RocksDB} instance on success, null if the specified + * {@link RocksDB} can not be opened. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public static RocksDB openReadOnly(final String path) + throws RocksDBException { + // This allows to use the rocksjni default Options instead of + // the c++ one. + Options options = new Options(); + return openReadOnly(options, path); + } + + /** + * The factory constructor of RocksDB that opens a RocksDB instance in + * Read-Only mode given the path to the database using the default + * options. + * + * @param path the path to the RocksDB. + * @param columnFamilyDescriptors list of column family descriptors + * @param columnFamilyHandles will be filled with ColumnFamilyHandle instances + * on open. + * @return a {@link RocksDB} instance on success, null if the specified + * {@link RocksDB} can not be opened. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public static RocksDB openReadOnly(final String path, + final List columnFamilyDescriptors, + final List columnFamilyHandles) + throws RocksDBException { + // This allows to use the rocksjni default Options instead of + // the c++ one. + final DBOptions options = new DBOptions(); + return openReadOnly(options, path, columnFamilyDescriptors, + columnFamilyHandles); + } + + /** + * The factory constructor of RocksDB that opens a RocksDB instance in + * Read-Only mode given the path to the database using the specified + * options and db path. + * + * Options instance *should* not be disposed before all DBs using this options + * instance have been closed. If user doesn't call options dispose explicitly, + * then this options instance will be GC'd automatically. + * + * @param options {@link Options} instance. + * @param path the path to the RocksDB. + * @return a {@link RocksDB} instance on success, null if the specified + * {@link RocksDB} can not be opened. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public static RocksDB openReadOnly(final Options options, final String path) + throws RocksDBException { + // when non-default Options is used, keeping an Options reference + // in RocksDB can prevent Java to GC during the life-time of + // the currently-created RocksDB. + final RocksDB db = new RocksDB(openROnly(options.nativeHandle_, path)); + db.storeOptionsInstance(options); + return db; + } + + /** + * The factory constructor of RocksDB that opens a RocksDB instance in + * Read-Only mode given the path to the database using the specified + * options and db path. + * + *

    This open method allows to open RocksDB using a subset of available + * column families

    + *

    Options instance *should* not be disposed before all DBs using this + * options instance have been closed. If user doesn't call options dispose + * explicitly,then this options instance will be GC'd automatically.

    + * + * @param options {@link DBOptions} instance. + * @param path the path to the RocksDB. + * @param columnFamilyDescriptors list of column family descriptors + * @param columnFamilyHandles will be filled with ColumnFamilyHandle instances + * on open. + * @return a {@link RocksDB} instance on success, null if the specified + * {@link RocksDB} can not be opened. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public static RocksDB openReadOnly(final DBOptions options, final String path, + final List columnFamilyDescriptors, + final List columnFamilyHandles) + throws RocksDBException { + // when non-default Options is used, keeping an Options reference + // in RocksDB can prevent Java to GC during the life-time of + // the currently-created RocksDB. + + final byte[][] cfNames = new byte[columnFamilyDescriptors.size()][]; + final long[] cfOptionHandles = new long[columnFamilyDescriptors.size()]; + for (int i = 0; i < columnFamilyDescriptors.size(); i++) { + final ColumnFamilyDescriptor cfDescriptor = columnFamilyDescriptors + .get(i); + cfNames[i] = cfDescriptor.columnFamilyName(); + cfOptionHandles[i] = cfDescriptor.columnFamilyOptions().nativeHandle_; + } + + final long[] handles = openROnly(options.nativeHandle_, path, cfNames, + cfOptionHandles); + final RocksDB db = new RocksDB(handles[0]); + db.storeOptionsInstance(options); + + for (int i = 1; i < handles.length; i++) { + columnFamilyHandles.add(new ColumnFamilyHandle(db, handles[i])); + } + + return db; + } + /** + * Static method to determine all available column families for a + * rocksdb database identified by path + * + * @param options Options for opening the database + * @param path Absolute path to rocksdb database + * @return List<byte[]> List containing the column family names + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public static List listColumnFamilies(final Options options, + final String path) throws RocksDBException { + return Arrays.asList(RocksDB.listColumnFamilies(options.nativeHandle_, + path)); + } + + private void storeOptionsInstance(DBOptionsInterface options) { + options_ = options; + } + + /** + * Set the database entry for "key" to "value". + * + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void put(final byte[] key, final byte[] value) + throws RocksDBException { + put(nativeHandle_, key, 0, key.length, value, 0, value.length); + } + + /** + * Set the database entry for "key" to "value" in the specified + * column family. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + * + * throws IllegalArgumentException if column family is not present + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void put(final ColumnFamilyHandle columnFamilyHandle, + final byte[] key, final byte[] value) throws RocksDBException { + put(nativeHandle_, key, 0, key.length, value, 0, value.length, + columnFamilyHandle.nativeHandle_); + } + + /** + * Set the database entry for "key" to "value". + * + * @param writeOpts {@link org.rocksdb.WriteOptions} instance. + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void put(final WriteOptions writeOpts, final byte[] key, + final byte[] value) throws RocksDBException { + put(nativeHandle_, writeOpts.nativeHandle_, + key, 0, key.length, value, 0, value.length); + } + + /** + * Set the database entry for "key" to "value" for the specified + * column family. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param writeOpts {@link org.rocksdb.WriteOptions} instance. + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + * + * throws IllegalArgumentException if column family is not present + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * @see IllegalArgumentException + */ + public void put(final ColumnFamilyHandle columnFamilyHandle, + final WriteOptions writeOpts, final byte[] key, + final byte[] value) throws RocksDBException { + put(nativeHandle_, writeOpts.nativeHandle_, key, 0, key.length, value, + 0, value.length, columnFamilyHandle.nativeHandle_); + } + + /** + * If the key definitely does not exist in the database, then this method + * returns false, else true. + * + * This check is potentially lighter-weight than invoking DB::Get(). One way + * to make this lighter weight is to avoid doing any IOs. + * + * @param key byte array of a key to search for + * @param value StringBuilder instance which is a out parameter if a value is + * found in block-cache. + * @return boolean value indicating if key does not exist or might exist. + */ + public boolean keyMayExist(final byte[] key, final StringBuilder value) { + return keyMayExist(nativeHandle_, key, 0, key.length, value); + } + + /** + * If the key definitely does not exist in the database, then this method + * returns false, else true. + * + * This check is potentially lighter-weight than invoking DB::Get(). One way + * to make this lighter weight is to avoid doing any IOs. + * + * @param columnFamilyHandle {@link ColumnFamilyHandle} instance + * @param key byte array of a key to search for + * @param value StringBuilder instance which is a out parameter if a value is + * found in block-cache. + * @return boolean value indicating if key does not exist or might exist. + */ + public boolean keyMayExist(final ColumnFamilyHandle columnFamilyHandle, + final byte[] key, final StringBuilder value) { + return keyMayExist(nativeHandle_, key, 0, key.length, + columnFamilyHandle.nativeHandle_, value); + } + + /** + * If the key definitely does not exist in the database, then this method + * returns false, else true. + * + * This check is potentially lighter-weight than invoking DB::Get(). One way + * to make this lighter weight is to avoid doing any IOs. + * + * @param readOptions {@link ReadOptions} instance + * @param key byte array of a key to search for + * @param value StringBuilder instance which is a out parameter if a value is + * found in block-cache. + * @return boolean value indicating if key does not exist or might exist. + */ + public boolean keyMayExist(final ReadOptions readOptions, + final byte[] key, final StringBuilder value) { + return keyMayExist(nativeHandle_, readOptions.nativeHandle_, + key, 0, key.length, value); + } + + /** + * If the key definitely does not exist in the database, then this method + * returns false, else true. + * + * This check is potentially lighter-weight than invoking DB::Get(). One way + * to make this lighter weight is to avoid doing any IOs. + * + * @param readOptions {@link ReadOptions} instance + * @param columnFamilyHandle {@link ColumnFamilyHandle} instance + * @param key byte array of a key to search for + * @param value StringBuilder instance which is a out parameter if a value is + * found in block-cache. + * @return boolean value indicating if key does not exist or might exist. + */ + public boolean keyMayExist(final ReadOptions readOptions, + final ColumnFamilyHandle columnFamilyHandle, final byte[] key, + final StringBuilder value) { + return keyMayExist(nativeHandle_, readOptions.nativeHandle_, + key, 0, key.length, columnFamilyHandle.nativeHandle_, + value); + } + + /** + * Apply the specified updates to the database. + * + * @param writeOpts WriteOptions instance + * @param updates WriteBatch instance + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void write(final WriteOptions writeOpts, final WriteBatch updates) + throws RocksDBException { + write0(nativeHandle_, writeOpts.nativeHandle_, updates.nativeHandle_); + } + + /** + * Apply the specified updates to the database. + * + * @param writeOpts WriteOptions instance + * @param updates WriteBatchWithIndex instance + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void write(final WriteOptions writeOpts, + final WriteBatchWithIndex updates) throws RocksDBException { + write1(nativeHandle_, writeOpts.nativeHandle_, updates.nativeHandle_); + } + + /** + * Add merge operand for key/value pair. + * + * @param key the specified key to be merged. + * @param value the value to be merged with the current value for + * the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void merge(final byte[] key, final byte[] value) + throws RocksDBException { + merge(nativeHandle_, key, 0, key.length, value, 0, value.length); + } + + /** + * Add merge operand for key/value pair in a ColumnFamily. + * + * @param columnFamilyHandle {@link ColumnFamilyHandle} instance + * @param key the specified key to be merged. + * @param value the value to be merged with the current value for + * the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void merge(final ColumnFamilyHandle columnFamilyHandle, + final byte[] key, final byte[] value) throws RocksDBException { + merge(nativeHandle_, key, 0, key.length, value, 0, value.length, + columnFamilyHandle.nativeHandle_); + } + + /** + * Add merge operand for key/value pair. + * + * @param writeOpts {@link WriteOptions} for this write. + * @param key the specified key to be merged. + * @param value the value to be merged with the current value for + * the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void merge(final WriteOptions writeOpts, final byte[] key, + final byte[] value) throws RocksDBException { + merge(nativeHandle_, writeOpts.nativeHandle_, + key, 0, key.length, value, 0, value.length); + } + + /** + * Add merge operand for key/value pair. + * + * @param columnFamilyHandle {@link ColumnFamilyHandle} instance + * @param writeOpts {@link WriteOptions} for this write. + * @param key the specified key to be merged. + * @param value the value to be merged with the current value for + * the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void merge(final ColumnFamilyHandle columnFamilyHandle, + final WriteOptions writeOpts, final byte[] key, + final byte[] value) throws RocksDBException { + merge(nativeHandle_, writeOpts.nativeHandle_, + key, 0, key.length, value, 0, value.length, + columnFamilyHandle.nativeHandle_); + } + + // TODO(AR) we should improve the #get() API, returning -1 (RocksDB.NOT_FOUND) is not very nice + // when we could communicate better status into, also the C++ code show that -2 could be returned + + /** + * Get the value associated with the specified key within column family* + * @param key the key to retrieve the value. + * @param value the out-value to receive the retrieved value. + * @return The size of the actual value that matches the specified + * {@code key} in byte. If the return value is greater than the + * length of {@code value}, then it indicates that the size of the + * input buffer {@code value} is insufficient and partial result will + * be returned. RocksDB.NOT_FOUND will be returned if the value not + * found. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public int get(final byte[] key, final byte[] value) throws RocksDBException { + return get(nativeHandle_, key, 0, key.length, value, 0, value.length); + } + + /** + * Get the value associated with the specified key within column family. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param key the key to retrieve the value. + * @param value the out-value to receive the retrieved value. + * @return The size of the actual value that matches the specified + * {@code key} in byte. If the return value is greater than the + * length of {@code value}, then it indicates that the size of the + * input buffer {@code value} is insufficient and partial result will + * be returned. RocksDB.NOT_FOUND will be returned if the value not + * found. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public int get(final ColumnFamilyHandle columnFamilyHandle, final byte[] key, + final byte[] value) throws RocksDBException, IllegalArgumentException { + return get(nativeHandle_, key, 0, key.length, value, 0, value.length, + columnFamilyHandle.nativeHandle_); + } + + /** + * Get the value associated with the specified key. + * + * @param opt {@link org.rocksdb.ReadOptions} instance. + * @param key the key to retrieve the value. + * @param value the out-value to receive the retrieved value. + * @return The size of the actual value that matches the specified + * {@code key} in byte. If the return value is greater than the + * length of {@code value}, then it indicates that the size of the + * input buffer {@code value} is insufficient and partial result will + * be returned. RocksDB.NOT_FOUND will be returned if the value not + * found. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public int get(final ReadOptions opt, final byte[] key, + final byte[] value) throws RocksDBException { + return get(nativeHandle_, opt.nativeHandle_, + key, 0, key.length, value, 0, value.length); + } + /** + * Get the value associated with the specified key within column family. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param opt {@link org.rocksdb.ReadOptions} instance. + * @param key the key to retrieve the value. + * @param value the out-value to receive the retrieved value. + * @return The size of the actual value that matches the specified + * {@code key} in byte. If the return value is greater than the + * length of {@code value}, then it indicates that the size of the + * input buffer {@code value} is insufficient and partial result will + * be returned. RocksDB.NOT_FOUND will be returned if the value not + * found. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public int get(final ColumnFamilyHandle columnFamilyHandle, + final ReadOptions opt, final byte[] key, final byte[] value) + throws RocksDBException { + return get(nativeHandle_, opt.nativeHandle_, key, 0, key.length, value, + 0, value.length, columnFamilyHandle.nativeHandle_); + } + + /** + * The simplified version of get which returns a new byte array storing + * the value associated with the specified input key if any. null will be + * returned if the specified key is not found. + * + * @param key the key retrieve the value. + * @return a byte array storing the value associated with the input key if + * any. null if it does not find the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public byte[] get(final byte[] key) throws RocksDBException { + return get(nativeHandle_, key, 0, key.length); + } + + /** + * The simplified version of get which returns a new byte array storing + * the value associated with the specified input key if any. null will be + * returned if the specified key is not found. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param key the key retrieve the value. + * @return a byte array storing the value associated with the input key if + * any. null if it does not find the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public byte[] get(final ColumnFamilyHandle columnFamilyHandle, + final byte[] key) throws RocksDBException { + return get(nativeHandle_, key, 0, key.length, + columnFamilyHandle.nativeHandle_); + } + + /** + * The simplified version of get which returns a new byte array storing + * the value associated with the specified input key if any. null will be + * returned if the specified key is not found. + * + * @param key the key retrieve the value. + * @param opt Read options. + * @return a byte array storing the value associated with the input key if + * any. null if it does not find the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public byte[] get(final ReadOptions opt, final byte[] key) + throws RocksDBException { + return get(nativeHandle_, opt.nativeHandle_, key, 0, key.length); + } + + /** + * The simplified version of get which returns a new byte array storing + * the value associated with the specified input key if any. null will be + * returned if the specified key is not found. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param key the key retrieve the value. + * @param opt Read options. + * @return a byte array storing the value associated with the input key if + * any. null if it does not find the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public byte[] get(final ColumnFamilyHandle columnFamilyHandle, + final ReadOptions opt, final byte[] key) throws RocksDBException { + return get(nativeHandle_, opt.nativeHandle_, key, 0, key.length, + columnFamilyHandle.nativeHandle_); + } + + /** + * Returns a map of keys for which values were found in DB. + * + * @param keys List of keys for which values need to be retrieved. + * @return Map where key of map is the key passed by user and value for map + * entry is the corresponding value in DB. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public Map multiGet(final List keys) + throws RocksDBException { + assert(keys.size() != 0); + + final byte[][] keysArray = keys.toArray(new byte[keys.size()][]); + final int keyOffsets[] = new int[keysArray.length]; + final int keyLengths[] = new int[keysArray.length]; + for(int i = 0; i < keyLengths.length; i++) { + keyLengths[i] = keysArray[i].length; + } + + final byte[][] values = multiGet(nativeHandle_, keysArray, keyOffsets, + keyLengths); + + final Map keyValueMap = + new HashMap<>(computeCapacityHint(values.length)); + for(int i = 0; i < values.length; i++) { + if(values[i] == null) { + continue; + } + + keyValueMap.put(keys.get(i), values[i]); + } + + return keyValueMap; + } + + private static int computeCapacityHint(final int estimatedNumberOfItems) { + // Default load factor for HashMap is 0.75, so N * 1.5 will be at the load + // limit. We add +1 for a buffer. + return (int)Math.ceil(estimatedNumberOfItems * 1.5 + 1.0); + } + + /** + * Returns a map of keys for which values were found in DB. + *

    + * Note: Every key needs to have a related column family name in + * {@code columnFamilyHandleList}. + *

    + * + * @param columnFamilyHandleList {@link java.util.List} containing + * {@link org.rocksdb.ColumnFamilyHandle} instances. + * @param keys List of keys for which values need to be retrieved. + * @return Map where key of map is the key passed by user and value for map + * entry is the corresponding value in DB. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * @throws IllegalArgumentException thrown if the size of passed keys is not + * equal to the amount of passed column family handles. + */ + public Map multiGet( + final List columnFamilyHandleList, + final List keys) throws RocksDBException, + IllegalArgumentException { + assert(keys.size() != 0); + // Check if key size equals cfList size. If not a exception must be + // thrown. If not a Segmentation fault happens. + if (keys.size() != columnFamilyHandleList.size()) { + throw new IllegalArgumentException( + "For each key there must be a ColumnFamilyHandle."); + } + final long[] cfHandles = new long[columnFamilyHandleList.size()]; + for (int i = 0; i < columnFamilyHandleList.size(); i++) { + cfHandles[i] = columnFamilyHandleList.get(i).nativeHandle_; + } + + final byte[][] keysArray = keys.toArray(new byte[keys.size()][]); + final int keyOffsets[] = new int[keysArray.length]; + final int keyLengths[] = new int[keysArray.length]; + for(int i = 0; i < keyLengths.length; i++) { + keyLengths[i] = keysArray[i].length; + } + + final byte[][] values = multiGet(nativeHandle_, keysArray, keyOffsets, + keyLengths, cfHandles); + + final Map keyValueMap = + new HashMap<>(computeCapacityHint(values.length)); + for(int i = 0; i < values.length; i++) { + if (values[i] == null) { + continue; + } + keyValueMap.put(keys.get(i), values[i]); + } + return keyValueMap; + } + + /** + * Returns a map of keys for which values were found in DB. + * + * @param opt Read options. + * @param keys of keys for which values need to be retrieved. + * @return Map where key of map is the key passed by user and value for map + * entry is the corresponding value in DB. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public Map multiGet(final ReadOptions opt, + final List keys) throws RocksDBException { + assert(keys.size() != 0); + + final byte[][] keysArray = keys.toArray(new byte[keys.size()][]); + final int keyOffsets[] = new int[keysArray.length]; + final int keyLengths[] = new int[keysArray.length]; + for(int i = 0; i < keyLengths.length; i++) { + keyLengths[i] = keysArray[i].length; + } + + final byte[][] values = multiGet(nativeHandle_, opt.nativeHandle_, + keysArray, keyOffsets, keyLengths); + + final Map keyValueMap = + new HashMap<>(computeCapacityHint(values.length)); + for(int i = 0; i < values.length; i++) { + if(values[i] == null) { + continue; + } + + keyValueMap.put(keys.get(i), values[i]); + } + + return keyValueMap; + } + + /** + * Returns a map of keys for which values were found in DB. + *

    + * Note: Every key needs to have a related column family name in + * {@code columnFamilyHandleList}. + *

    + * + * @param opt Read options. + * @param columnFamilyHandleList {@link java.util.List} containing + * {@link org.rocksdb.ColumnFamilyHandle} instances. + * @param keys of keys for which values need to be retrieved. + * @return Map where key of map is the key passed by user and value for map + * entry is the corresponding value in DB. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * @throws IllegalArgumentException thrown if the size of passed keys is not + * equal to the amount of passed column family handles. + */ + public Map multiGet(final ReadOptions opt, + final List columnFamilyHandleList, + final List keys) throws RocksDBException { + assert(keys.size() != 0); + // Check if key size equals cfList size. If not a exception must be + // thrown. If not a Segmentation fault happens. + if (keys.size()!=columnFamilyHandleList.size()){ + throw new IllegalArgumentException( + "For each key there must be a ColumnFamilyHandle."); + } + final long[] cfHandles = new long[columnFamilyHandleList.size()]; + for (int i = 0; i < columnFamilyHandleList.size(); i++) { + cfHandles[i] = columnFamilyHandleList.get(i).nativeHandle_; + } + + final byte[][] keysArray = keys.toArray(new byte[keys.size()][]); + final int keyOffsets[] = new int[keysArray.length]; + final int keyLengths[] = new int[keysArray.length]; + for(int i = 0; i < keyLengths.length; i++) { + keyLengths[i] = keysArray[i].length; + } + + final byte[][] values = multiGet(nativeHandle_, opt.nativeHandle_, + keysArray, keyOffsets, keyLengths, cfHandles); + + final Map keyValueMap + = new HashMap<>(computeCapacityHint(values.length)); + for(int i = 0; i < values.length; i++) { + if(values[i] == null) { + continue; + } + keyValueMap.put(keys.get(i), values[i]); + } + + return keyValueMap; + } + + /** + * Remove the database entry (if any) for "key". Returns OK on + * success, and a non-OK status on error. It is not an error if "key" + * did not exist in the database. + * + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * + * @deprecated Use {@link #delete(byte[])} + */ + @Deprecated + public void remove(final byte[] key) throws RocksDBException { + delete(key); + } + + /** + * Delete the database entry (if any) for "key". Returns OK on + * success, and a non-OK status on error. It is not an error if "key" + * did not exist in the database. + * + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void delete(final byte[] key) throws RocksDBException { + delete(nativeHandle_, key, 0, key.length); + } + + /** + * Remove the database entry (if any) for "key". Returns OK on + * success, and a non-OK status on error. It is not an error if "key" + * did not exist in the database. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * + * @deprecated Use {@link #delete(ColumnFamilyHandle, byte[])} + */ + @Deprecated + public void remove(final ColumnFamilyHandle columnFamilyHandle, + final byte[] key) throws RocksDBException { + delete(columnFamilyHandle, key); + } + + /** + * Delete the database entry (if any) for "key". Returns OK on + * success, and a non-OK status on error. It is not an error if "key" + * did not exist in the database. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void delete(final ColumnFamilyHandle columnFamilyHandle, + final byte[] key) throws RocksDBException { + delete(nativeHandle_, key, 0, key.length, columnFamilyHandle.nativeHandle_); + } + + /** + * Remove the database entry (if any) for "key". Returns OK on + * success, and a non-OK status on error. It is not an error if "key" + * did not exist in the database. + * + * @param writeOpt WriteOptions to be used with delete operation + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * + * @deprecated Use {@link #delete(WriteOptions, byte[])} + */ + @Deprecated + public void remove(final WriteOptions writeOpt, final byte[] key) + throws RocksDBException { + delete(writeOpt, key); + } + + /** + * Delete the database entry (if any) for "key". Returns OK on + * success, and a non-OK status on error. It is not an error if "key" + * did not exist in the database. + * + * @param writeOpt WriteOptions to be used with delete operation + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void delete(final WriteOptions writeOpt, final byte[] key) + throws RocksDBException { + delete(nativeHandle_, writeOpt.nativeHandle_, key, 0, key.length); + } + + /** + * Remove the database entry (if any) for "key". Returns OK on + * success, and a non-OK status on error. It is not an error if "key" + * did not exist in the database. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param writeOpt WriteOptions to be used with delete operation + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * + * @deprecated Use {@link #delete(ColumnFamilyHandle, WriteOptions, byte[])} + */ + @Deprecated + public void remove(final ColumnFamilyHandle columnFamilyHandle, + final WriteOptions writeOpt, final byte[] key) + throws RocksDBException { + delete(columnFamilyHandle, writeOpt, key); + } + + /** + * Delete the database entry (if any) for "key". Returns OK on + * success, and a non-OK status on error. It is not an error if "key" + * did not exist in the database. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param writeOpt WriteOptions to be used with delete operation + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void delete(final ColumnFamilyHandle columnFamilyHandle, + final WriteOptions writeOpt, final byte[] key) + throws RocksDBException { + delete(nativeHandle_, writeOpt.nativeHandle_, key, 0, key.length, + columnFamilyHandle.nativeHandle_); + } + + /** + * Remove the database entry for {@code key}. Requires that the key exists + * and was not overwritten. It is not an error if the key did not exist + * in the database. + * + * If a key is overwritten (by calling {@link #put(byte[], byte[])} multiple + * times), then the result of calling SingleDelete() on this key is undefined. + * SingleDelete() only behaves correctly if there has been only one Put() + * for this key since the previous call to SingleDelete() for this key. + * + * This feature is currently an experimental performance optimization + * for a very specific workload. It is up to the caller to ensure that + * SingleDelete is only used for a key that is not deleted using Delete() or + * written using Merge(). Mixing SingleDelete operations with Deletes and + * Merges can result in undefined behavior. + * + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + @Experimental("Performance optimization for a very specific workload") + public void singleDelete(final byte[] key) throws RocksDBException { + singleDelete(nativeHandle_, key, key.length); + } + + /** + * Remove the database entry for {@code key}. Requires that the key exists + * and was not overwritten. It is not an error if the key did not exist + * in the database. + * + * If a key is overwritten (by calling {@link #put(byte[], byte[])} multiple + * times), then the result of calling SingleDelete() on this key is undefined. + * SingleDelete() only behaves correctly if there has been only one Put() + * for this key since the previous call to SingleDelete() for this key. + * + * This feature is currently an experimental performance optimization + * for a very specific workload. It is up to the caller to ensure that + * SingleDelete is only used for a key that is not deleted using Delete() or + * written using Merge(). Mixing SingleDelete operations with Deletes and + * Merges can result in undefined behavior. + * + * @param columnFamilyHandle The column family to delete the key from + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + @Experimental("Performance optimization for a very specific workload") + public void singleDelete(final ColumnFamilyHandle columnFamilyHandle, + final byte[] key) throws RocksDBException { + singleDelete(nativeHandle_, key, key.length, + columnFamilyHandle.nativeHandle_); + } + + /** + * Remove the database entry for {@code key}. Requires that the key exists + * and was not overwritten. It is not an error if the key did not exist + * in the database. + * + * If a key is overwritten (by calling {@link #put(byte[], byte[])} multiple + * times), then the result of calling SingleDelete() on this key is undefined. + * SingleDelete() only behaves correctly if there has been only one Put() + * for this key since the previous call to SingleDelete() for this key. + * + * This feature is currently an experimental performance optimization + * for a very specific workload. It is up to the caller to ensure that + * SingleDelete is only used for a key that is not deleted using Delete() or + * written using Merge(). Mixing SingleDelete operations with Deletes and + * Merges can result in undefined behavior. + * + * Note: consider setting {@link WriteOptions#setSync(boolean)} true. + * + * @param writeOpt Write options for the delete + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + @Experimental("Performance optimization for a very specific workload") + public void singleDelete(final WriteOptions writeOpt, final byte[] key) + throws RocksDBException { + singleDelete(nativeHandle_, writeOpt.nativeHandle_, key, key.length); + } + + /** + * Remove the database entry for {@code key}. Requires that the key exists + * and was not overwritten. It is not an error if the key did not exist + * in the database. + * + * If a key is overwritten (by calling {@link #put(byte[], byte[])} multiple + * times), then the result of calling SingleDelete() on this key is undefined. + * SingleDelete() only behaves correctly if there has been only one Put() + * for this key since the previous call to SingleDelete() for this key. + * + * This feature is currently an experimental performance optimization + * for a very specific workload. It is up to the caller to ensure that + * SingleDelete is only used for a key that is not deleted using Delete() or + * written using Merge(). Mixing SingleDelete operations with Deletes and + * Merges can result in undefined behavior. + * + * Note: consider setting {@link WriteOptions#setSync(boolean)} true. + * + * @param columnFamilyHandle The column family to delete the key from + * @param writeOpt Write options for the delete + * @param key Key to delete within database + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + @Experimental("Performance optimization for a very specific workload") + public void singleDelete(final ColumnFamilyHandle columnFamilyHandle, + final WriteOptions writeOpt, final byte[] key) throws RocksDBException { + singleDelete(nativeHandle_, writeOpt.nativeHandle_, key, key.length, + columnFamilyHandle.nativeHandle_); + } + + /** + * DB implements can export properties about their state + * via this method on a per column family level. + * + *

    If {@code property} is a valid property understood by this DB + * implementation, fills {@code value} with its current value and + * returns true. Otherwise returns false.

    + * + *

    Valid property names include: + *

      + *
    • "rocksdb.num-files-at-level<N>" - return the number of files at + * level <N>, where <N> is an ASCII representation of a level + * number (e.g. "0").
      • + *
    • "rocksdb.stats" - returns a multi-line string that describes statistics + * about the internal operation of the DB.
      • + *
    • "rocksdb.sstables" - returns a multi-line string that describes all + * of the sstables that make up the db contents.
      • + *
    + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param property to be fetched. See above for examples + * @return property value + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public String getProperty(final ColumnFamilyHandle columnFamilyHandle, + final String property) throws RocksDBException { + return getProperty0(nativeHandle_, columnFamilyHandle.nativeHandle_, + property, property.length()); + } + + /** + * Removes the database entries in the range ["beginKey", "endKey"), i.e., + * including "beginKey" and excluding "endKey". a non-OK status on error. It + * is not an error if no keys exist in the range ["beginKey", "endKey"). + * + * Delete the database entry (if any) for "key". Returns OK on success, and a + * non-OK status on error. It is not an error if "key" did not exist in the + * database. + * + * @param beginKey + * First key to delete within database (included) + * @param endKey + * Last key to delete within database (excluded) + * + * @throws RocksDBException + * thrown if error happens in underlying native library. + */ + public void deleteRange(final byte[] beginKey, final byte[] endKey) throws RocksDBException { + deleteRange(nativeHandle_, beginKey, 0, beginKey.length, endKey, 0, endKey.length); + } + + /** + * Removes the database entries in the range ["beginKey", "endKey"), i.e., + * including "beginKey" and excluding "endKey". a non-OK status on error. It + * is not an error if no keys exist in the range ["beginKey", "endKey"). + * + * Delete the database entry (if any) for "key". Returns OK on success, and a + * non-OK status on error. It is not an error if "key" did not exist in the + * database. + * + * @param columnFamilyHandle + * {@link org.rocksdb.ColumnFamilyHandle} instance + * @param beginKey + * First key to delete within database (included) + * @param endKey + * Last key to delete within database (excluded) + * + * @throws RocksDBException + * thrown if error happens in underlying native library. + */ + public void deleteRange(final ColumnFamilyHandle columnFamilyHandle, final byte[] beginKey, + final byte[] endKey) throws RocksDBException { + deleteRange(nativeHandle_, beginKey, 0, beginKey.length, endKey, 0, endKey.length, + columnFamilyHandle.nativeHandle_); + } + + /** + * Removes the database entries in the range ["beginKey", "endKey"), i.e., + * including "beginKey" and excluding "endKey". a non-OK status on error. It + * is not an error if no keys exist in the range ["beginKey", "endKey"). + * + * Delete the database entry (if any) for "key". Returns OK on success, and a + * non-OK status on error. It is not an error if "key" did not exist in the + * database. + * + * @param writeOpt + * WriteOptions to be used with delete operation + * @param beginKey + * First key to delete within database (included) + * @param endKey + * Last key to delete within database (excluded) + * + * @throws RocksDBException + * thrown if error happens in underlying native library. + */ + public void deleteRange(final WriteOptions writeOpt, final byte[] beginKey, final byte[] endKey) + throws RocksDBException { + deleteRange(nativeHandle_, writeOpt.nativeHandle_, beginKey, 0, beginKey.length, endKey, 0, + endKey.length); + } + + /** + * Removes the database entries in the range ["beginKey", "endKey"), i.e., + * including "beginKey" and excluding "endKey". a non-OK status on error. It + * is not an error if no keys exist in the range ["beginKey", "endKey"). + * + * Delete the database entry (if any) for "key". Returns OK on success, and a + * non-OK status on error. It is not an error if "key" did not exist in the + * database. + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param writeOpt + * WriteOptions to be used with delete operation + * @param beginKey + * First key to delete within database (included) + * @param endKey + * Last key to delete within database (excluded) + * + * @throws RocksDBException + * thrown if error happens in underlying native library. + */ + public void deleteRange(final ColumnFamilyHandle columnFamilyHandle, final WriteOptions writeOpt, + final byte[] beginKey, final byte[] endKey) throws RocksDBException { + deleteRange(nativeHandle_, writeOpt.nativeHandle_, beginKey, 0, beginKey.length, endKey, 0, + endKey.length, columnFamilyHandle.nativeHandle_); + } + + /** + * DB implementations can export properties about their state + * via this method. If "property" is a valid property understood by this + * DB implementation, fills "*value" with its current value and returns + * true. Otherwise returns false. + * + *

    Valid property names include: + *

      + *
    • "rocksdb.num-files-at-level<N>" - return the number of files at + * level <N>, where <N> is an ASCII representation of a level + * number (e.g. "0").
      • + *
    • "rocksdb.stats" - returns a multi-line string that describes statistics + * about the internal operation of the DB.
      • + *
    • "rocksdb.sstables" - returns a multi-line string that describes all + * of the sstables that make up the db contents.
      • + *
    + * + * @param property to be fetched. See above for examples + * @return property value + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public String getProperty(final String property) throws RocksDBException { + return getProperty0(nativeHandle_, property, property.length()); + } + + /** + *

    Similar to GetProperty(), but only works for a subset of properties + * whose return value is a numerical value. Return the value as long.

    + * + *

    Note: As the returned property is of type + * {@code uint64_t} on C++ side the returning value can be negative + * because Java supports in Java 7 only signed long values.

    + * + *

    Java 7: To mitigate the problem of the non + * existent unsigned long tpye, values should be encapsulated using + * {@link java.math.BigInteger} to reflect the correct value. The correct + * behavior is guaranteed if {@code 2^64} is added to negative values.

    + * + *

    Java 8: In Java 8 the value should be treated as + * unsigned long using provided methods of type {@link Long}.

    + * + * @param property to be fetched. + * + * @return numerical property value. + * + * @throws RocksDBException if an error happens in the underlying native code. + */ + public long getLongProperty(final String property) throws RocksDBException { + return getLongProperty(nativeHandle_, property, property.length()); + } + + /** + *

    Similar to GetProperty(), but only works for a subset of properties + * whose return value is a numerical value. Return the value as long.

    + * + *

    Note: As the returned property is of type + * {@code uint64_t} on C++ side the returning value can be negative + * because Java supports in Java 7 only signed long values.

    + * + *

    Java 7: To mitigate the problem of the non + * existent unsigned long tpye, values should be encapsulated using + * {@link java.math.BigInteger} to reflect the correct value. The correct + * behavior is guaranteed if {@code 2^64} is added to negative values.

    + * + *

    Java 8: In Java 8 the value should be treated as + * unsigned long using provided methods of type {@link Long}.

    + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param property to be fetched. + * + * @return numerical property value + * + * @throws RocksDBException if an error happens in the underlying native code. + */ + public long getLongProperty(final ColumnFamilyHandle columnFamilyHandle, + final String property) throws RocksDBException { + return getLongProperty(nativeHandle_, columnFamilyHandle.nativeHandle_, + property, property.length()); + } + + /** + *

    Return a heap-allocated iterator over the contents of the + * database. The result of newIterator() is initially invalid + * (caller must call one of the Seek methods on the iterator + * before using it).

    + * + *

    Caller should close the iterator when it is no longer needed. + * The returned iterator should be closed before this db is closed. + *

    + * + * @return instance of iterator object. + */ + public RocksIterator newIterator() { + return new RocksIterator(this, iterator(nativeHandle_)); + } + + /** + *

    Return a heap-allocated iterator over the contents of the + * database. The result of newIterator() is initially invalid + * (caller must call one of the Seek methods on the iterator + * before using it).

    + * + *

    Caller should close the iterator when it is no longer needed. + * The returned iterator should be closed before this db is closed. + *

    + * + * @param readOptions {@link ReadOptions} instance. + * @return instance of iterator object. + */ + public RocksIterator newIterator(final ReadOptions readOptions) { + return new RocksIterator(this, iterator(nativeHandle_, + readOptions.nativeHandle_)); + } + + /** + *

    Return a handle to the current DB state. Iterators created with + * this handle will all observe a stable snapshot of the current DB + * state. The caller must call ReleaseSnapshot(result) when the + * snapshot is no longer needed.

    + * + *

    nullptr will be returned if the DB fails to take a snapshot or does + * not support snapshot.

    + * + * @return Snapshot {@link Snapshot} instance + */ + public Snapshot getSnapshot() { + long snapshotHandle = getSnapshot(nativeHandle_); + if (snapshotHandle != 0) { + return new Snapshot(snapshotHandle); + } + return null; + } + + /** + * Release a previously acquired snapshot. The caller must not + * use "snapshot" after this call. + * + * @param snapshot {@link Snapshot} instance + */ + public void releaseSnapshot(final Snapshot snapshot) { + if (snapshot != null) { + releaseSnapshot(nativeHandle_, snapshot.nativeHandle_); + } + } + + /** + *

    Return a heap-allocated iterator over the contents of the + * database. The result of newIterator() is initially invalid + * (caller must call one of the Seek methods on the iterator + * before using it).

    + * + *

    Caller should close the iterator when it is no longer needed. + * The returned iterator should be closed before this db is closed. + *

    + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @return instance of iterator object. + */ + public RocksIterator newIterator( + final ColumnFamilyHandle columnFamilyHandle) { + return new RocksIterator(this, iteratorCF(nativeHandle_, + columnFamilyHandle.nativeHandle_)); + } + + /** + *

    Return a heap-allocated iterator over the contents of the + * database. The result of newIterator() is initially invalid + * (caller must call one of the Seek methods on the iterator + * before using it).

    + * + *

    Caller should close the iterator when it is no longer needed. + * The returned iterator should be closed before this db is closed. + *

    + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param readOptions {@link ReadOptions} instance. + * @return instance of iterator object. + */ + public RocksIterator newIterator(final ColumnFamilyHandle columnFamilyHandle, + final ReadOptions readOptions) { + return new RocksIterator(this, iteratorCF(nativeHandle_, + columnFamilyHandle.nativeHandle_, readOptions.nativeHandle_)); + } + + /** + * Returns iterators from a consistent database state across multiple + * column families. Iterators are heap allocated and need to be deleted + * before the db is deleted + * + * @param columnFamilyHandleList {@link java.util.List} containing + * {@link org.rocksdb.ColumnFamilyHandle} instances. + * @return {@link java.util.List} containing {@link org.rocksdb.RocksIterator} + * instances + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public List newIterators( + final List columnFamilyHandleList) + throws RocksDBException { + return newIterators(columnFamilyHandleList, new ReadOptions()); + } + + /** + * Returns iterators from a consistent database state across multiple + * column families. Iterators are heap allocated and need to be deleted + * before the db is deleted + * + * @param columnFamilyHandleList {@link java.util.List} containing + * {@link org.rocksdb.ColumnFamilyHandle} instances. + * @param readOptions {@link ReadOptions} instance. + * @return {@link java.util.List} containing {@link org.rocksdb.RocksIterator} + * instances + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public List newIterators( + final List columnFamilyHandleList, + final ReadOptions readOptions) throws RocksDBException { + + final long[] columnFamilyHandles = new long[columnFamilyHandleList.size()]; + for (int i = 0; i < columnFamilyHandleList.size(); i++) { + columnFamilyHandles[i] = columnFamilyHandleList.get(i).nativeHandle_; + } + + final long[] iteratorRefs = iterators(nativeHandle_, columnFamilyHandles, + readOptions.nativeHandle_); + + final List iterators = new ArrayList<>( + columnFamilyHandleList.size()); + for (int i=0; iFlush all memory table data.

    + * + *

    Note: it must be ensured that the FlushOptions instance + * is not GC'ed before this method finishes. If the wait parameter is + * set to false, flush processing is asynchronous.

    + * + * @param flushOptions {@link org.rocksdb.FlushOptions} instance. + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void flush(final FlushOptions flushOptions) + throws RocksDBException { + flush(nativeHandle_, flushOptions.nativeHandle_); + } + + /** + *

    Flush all memory table data.

    + * + *

    Note: it must be ensured that the FlushOptions instance + * is not GC'ed before this method finishes. If the wait parameter is + * set to false, flush processing is asynchronous.

    + * + * @param flushOptions {@link org.rocksdb.FlushOptions} instance. + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} instance. + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void flush(final FlushOptions flushOptions, + final ColumnFamilyHandle columnFamilyHandle) throws RocksDBException { + flush(nativeHandle_, flushOptions.nativeHandle_, + columnFamilyHandle.nativeHandle_); + } + + /** + *

    Range compaction of database.

    + *

    Note: After the database has been compacted, + * all data will have been pushed down to the last level containing + * any data.

    + * + *

    See also

    + *
      + *
    • {@link #compactRange(boolean, int, int)}
      • + *
    • {@link #compactRange(byte[], byte[])}
      • + *
    • {@link #compactRange(byte[], byte[], boolean, int, int)}
      • + *
    + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void compactRange() throws RocksDBException { + compactRange0(nativeHandle_, false, -1, 0); + } + + /** + *

    Range compaction of database.

    + *

    Note: After the database has been compacted, + * all data will have been pushed down to the last level containing + * any data.

    + * + *

    See also

    + *
      + *
    • {@link #compactRange()}
      • + *
    • {@link #compactRange(boolean, int, int)}
      • + *
    • {@link #compactRange(byte[], byte[], boolean, int, int)}
      • + *
    + * + * @param begin start of key range (included in range) + * @param end end of key range (excluded from range) + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void compactRange(final byte[] begin, final byte[] end) + throws RocksDBException { + compactRange0(nativeHandle_, begin, begin.length, end, + end.length, false, -1, 0); + } + + /** + *

    Range compaction of database.

    + *

    Note: After the database has been compacted, + * all data will have been pushed down to the last level containing + * any data.

    + * + *

    Compaction outputs should be placed in options.db_paths + * [target_path_id]. Behavior is undefined if target_path_id is + * out of range.

    + * + *

    See also

    + *
      + *
    • {@link #compactRange()}
      • + *
    • {@link #compactRange(byte[], byte[])}
      • + *
    • {@link #compactRange(byte[], byte[], boolean, int, int)}
      • + *
    + * + * @param reduce_level reduce level after compaction + * @param target_level target level to compact to + * @param target_path_id the target path id of output path + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void compactRange(final boolean reduce_level, + final int target_level, final int target_path_id) + throws RocksDBException { + compactRange0(nativeHandle_, reduce_level, + target_level, target_path_id); + } + + + /** + *

    Range compaction of database.

    + *

    Note: After the database has been compacted, + * all data will have been pushed down to the last level containing + * any data.

    + * + *

    Compaction outputs should be placed in options.db_paths + * [target_path_id]. Behavior is undefined if target_path_id is + * out of range.

    + * + *

    See also

    + *
      + *
    • {@link #compactRange()}
      • + *
    • {@link #compactRange(boolean, int, int)}
      • + *
    • {@link #compactRange(byte[], byte[])}
      • + *
    + * + * @param begin start of key range (included in range) + * @param end end of key range (excluded from range) + * @param reduce_level reduce level after compaction + * @param target_level target level to compact to + * @param target_path_id the target path id of output path + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void compactRange(final byte[] begin, final byte[] end, + final boolean reduce_level, final int target_level, + final int target_path_id) throws RocksDBException { + compactRange0(nativeHandle_, begin, begin.length, end, end.length, + reduce_level, target_level, target_path_id); + } + + /** + *

    Range compaction of column family.

    + *

    Note: After the database has been compacted, + * all data will have been pushed down to the last level containing + * any data.

    + * + *

    See also

    + *
      + *
    • + * {@link #compactRange(ColumnFamilyHandle, boolean, int, int)} + *
      • + *
    • + * {@link #compactRange(ColumnFamilyHandle, byte[], byte[])} + *
      • + *
    • + * {@link #compactRange(ColumnFamilyHandle, byte[], byte[], + * boolean, int, int)} + *
      • + *
    + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance. + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void compactRange(final ColumnFamilyHandle columnFamilyHandle) + throws RocksDBException { + compactRange(nativeHandle_, false, -1, 0, + columnFamilyHandle.nativeHandle_); + } + + /** + *

    Range compaction of column family.

    + *

    Note: After the database has been compacted, + * all data will have been pushed down to the last level containing + * any data.

    + * + *

    See also

    + *
      + *
    • {@link #compactRange(ColumnFamilyHandle)}
      • + *
    • + * {@link #compactRange(ColumnFamilyHandle, boolean, int, int)} + *
      • + *
    • + * {@link #compactRange(ColumnFamilyHandle, byte[], byte[], + * boolean, int, int)} + *
      • + *
    + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance. + * @param begin start of key range (included in range) + * @param end end of key range (excluded from range) + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void compactRange(final ColumnFamilyHandle columnFamilyHandle, + final byte[] begin, final byte[] end) throws RocksDBException { + compactRange(nativeHandle_, begin, begin.length, end, end.length, + false, -1, 0, columnFamilyHandle.nativeHandle_); + } + + /** + *

    Range compaction of column family.

    + *

    Note: After the database has been compacted, + * all data will have been pushed down to the last level containing + * any data.

    + * + *

    Compaction outputs should be placed in options.db_paths + * [target_path_id]. Behavior is undefined if target_path_id is + * out of range.

    + * + *

    See also

    + *
      + *
    • {@link #compactRange(ColumnFamilyHandle)}
      • + *
    • + * {@link #compactRange(ColumnFamilyHandle, byte[], byte[])} + *
      • + *
    • + * {@link #compactRange(ColumnFamilyHandle, byte[], byte[], + * boolean, int, int)} + *
      • + *
    + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance. + * @param reduce_level reduce level after compaction + * @param target_level target level to compact to + * @param target_path_id the target path id of output path + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void compactRange(final ColumnFamilyHandle columnFamilyHandle, + final boolean reduce_level, final int target_level, + final int target_path_id) throws RocksDBException { + compactRange(nativeHandle_, reduce_level, target_level, + target_path_id, columnFamilyHandle.nativeHandle_); + } + + /** + *

    Range compaction of column family.

    + *

    Note: After the database has been compacted, + * all data will have been pushed down to the last level containing + * any data.

    + * + *

    Compaction outputs should be placed in options.db_paths + * [target_path_id]. Behavior is undefined if target_path_id is + * out of range.

    + * + *

    See also

    + *
      + *
    • {@link #compactRange(ColumnFamilyHandle)}
      • + *
    • + * {@link #compactRange(ColumnFamilyHandle, boolean, int, int)} + *
      • + *
    • + * {@link #compactRange(ColumnFamilyHandle, byte[], byte[])} + *
      • + *
    + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance. + * @param begin start of key range (included in range) + * @param end end of key range (excluded from range) + * @param reduce_level reduce level after compaction + * @param target_level target level to compact to + * @param target_path_id the target path id of output path + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public void compactRange(final ColumnFamilyHandle columnFamilyHandle, + final byte[] begin, final byte[] end, final boolean reduce_level, + final int target_level, final int target_path_id) + throws RocksDBException { + compactRange(nativeHandle_, begin, begin.length, end, end.length, + reduce_level, target_level, target_path_id, + columnFamilyHandle.nativeHandle_); + } + + /** + * This function will wait until all currently running background processes + * finish. After it returns, no background process will be run until + * {@link #continueBackgroundWork()} is called + * + * @throws RocksDBException If an error occurs when pausing background work + */ + public void pauseBackgroundWork() throws RocksDBException { + pauseBackgroundWork(nativeHandle_); + } + + /** + * Resumes backround work which was suspended by + * previously calling {@link #pauseBackgroundWork()} + * + * @throws RocksDBException If an error occurs when resuming background work + */ + public void continueBackgroundWork() throws RocksDBException { + continueBackgroundWork(nativeHandle_); + } + + /** + *

    The sequence number of the most recent transaction.

    + * + * @return sequence number of the most + * recent transaction. + */ + public long getLatestSequenceNumber() { + return getLatestSequenceNumber(nativeHandle_); + } + + /** + *

    Prevent file deletions. Compactions will continue to occur, + * but no obsolete files will be deleted. Calling this multiple + * times have the same effect as calling it once.

    + * + * @throws RocksDBException thrown if operation was not performed + * successfully. + */ + public void disableFileDeletions() throws RocksDBException { + disableFileDeletions(nativeHandle_); + } + + /** + *

    Allow compactions to delete obsolete files. + * If force == true, the call to EnableFileDeletions() + * will guarantee that file deletions are enabled after + * the call, even if DisableFileDeletions() was called + * multiple times before.

    + * + *

    If force == false, EnableFileDeletions will only + * enable file deletion after it's been called at least + * as many times as DisableFileDeletions(), enabling + * the two methods to be called by two threads + * concurrently without synchronization + * -- i.e., file deletions will be enabled only after both + * threads call EnableFileDeletions()

    + * + * @param force boolean value described above. + * + * @throws RocksDBException thrown if operation was not performed + * successfully. + */ + public void enableFileDeletions(final boolean force) + throws RocksDBException { + enableFileDeletions(nativeHandle_, force); + } + + /** + *

    Returns an iterator that is positioned at a write-batch containing + * seq_number. If the sequence number is non existent, it returns an iterator + * at the first available seq_no after the requested seq_no.

    + * + *

    Must set WAL_ttl_seconds or WAL_size_limit_MB to large values to + * use this api, else the WAL files will get + * cleared aggressively and the iterator might keep getting invalid before + * an update is read.

    + * + * @param sequenceNumber sequence number offset + * + * @return {@link org.rocksdb.TransactionLogIterator} instance. + * + * @throws org.rocksdb.RocksDBException if iterator cannot be retrieved + * from native-side. + */ + public TransactionLogIterator getUpdatesSince(final long sequenceNumber) + throws RocksDBException { + return new TransactionLogIterator( + getUpdatesSince(nativeHandle_, sequenceNumber)); + } + + public void setOptions(final ColumnFamilyHandle columnFamilyHandle, + final MutableColumnFamilyOptions mutableColumnFamilyOptions) + throws RocksDBException { + setOptions(nativeHandle_, columnFamilyHandle.nativeHandle_, + mutableColumnFamilyOptions.getKeys(), + mutableColumnFamilyOptions.getValues()); + } + + private long[] toNativeHandleList(final List objectList) { + final int len = objectList.size(); + final long[] handleList = new long[len]; + for (int i = 0; i < len; i++) { + handleList[i] = objectList.get(i).nativeHandle_; + } + return handleList; + } + + /** + * ingestExternalFile will load a list of external SST files (1) into the DB + * We will try to find the lowest possible level that the file can fit in, and + * ingest the file into this level (2). A file that have a key range that + * overlap with the memtable key range will require us to Flush the memtable + * first before ingesting the file. + * + * (1) External SST files can be created using {@link SstFileWriter} + * (2) We will try to ingest the files to the lowest possible level + * even if the file compression doesn't match the level compression + * + * @param filePathList The list of files to ingest + * @param ingestExternalFileOptions the options for the ingestion + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void ingestExternalFile(final List filePathList, + final IngestExternalFileOptions ingestExternalFileOptions) + throws RocksDBException { + ingestExternalFile(nativeHandle_, getDefaultColumnFamily().nativeHandle_, + filePathList.toArray(new String[filePathList.size()]), + filePathList.size(), ingestExternalFileOptions.nativeHandle_); + } + + /** + * ingestExternalFile will load a list of external SST files (1) into the DB + * We will try to find the lowest possible level that the file can fit in, and + * ingest the file into this level (2). A file that have a key range that + * overlap with the memtable key range will require us to Flush the memtable + * first before ingesting the file. + * + * (1) External SST files can be created using {@link SstFileWriter} + * (2) We will try to ingest the files to the lowest possible level + * even if the file compression doesn't match the level compression + * + * @param columnFamilyHandle The column family for the ingested files + * @param filePathList The list of files to ingest + * @param ingestExternalFileOptions the options for the ingestion + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void ingestExternalFile(final ColumnFamilyHandle columnFamilyHandle, + final List filePathList, + final IngestExternalFileOptions ingestExternalFileOptions) + throws RocksDBException { + ingestExternalFile(nativeHandle_, columnFamilyHandle.nativeHandle_, + filePathList.toArray(new String[filePathList.size()]), + filePathList.size(), ingestExternalFileOptions.nativeHandle_); + } + + /** + * Private constructor. + * + * @param nativeHandle The native handle of the C++ RocksDB object + */ + protected RocksDB(final long nativeHandle) { + super(nativeHandle); + } + + // native methods + protected native static long open(final long optionsHandle, + final String path) throws RocksDBException; + + /** + * @param optionsHandle Native handle pointing to an Options object + * @param path The directory path for the database files + * @param columnFamilyNames An array of column family names + * @param columnFamilyOptions An array of native handles pointing to + * ColumnFamilyOptions objects + * + * @return An array of native handles, [0] is the handle of the RocksDB object + * [1..1+n] are handles of the ColumnFamilyReferences + * + * @throws RocksDBException thrown if the database could not be opened + */ + protected native static long[] open(final long optionsHandle, + final String path, final byte[][] columnFamilyNames, + final long[] columnFamilyOptions) throws RocksDBException; + + protected native static long openROnly(final long optionsHandle, + final String path) throws RocksDBException; + + /** + * @param optionsHandle Native handle pointing to an Options object + * @param path The directory path for the database files + * @param columnFamilyNames An array of column family names + * @param columnFamilyOptions An array of native handles pointing to + * ColumnFamilyOptions objects + * + * @return An array of native handles, [0] is the handle of the RocksDB object + * [1..1+n] are handles of the ColumnFamilyReferences + * + * @throws RocksDBException thrown if the database could not be opened + */ + protected native static long[] openROnly(final long optionsHandle, + final String path, final byte[][] columnFamilyNames, + final long[] columnFamilyOptions + ) throws RocksDBException; + + protected native static byte[][] listColumnFamilies(long optionsHandle, + String path) throws RocksDBException; + protected native void put(long handle, byte[] key, int keyOffset, + int keyLength, byte[] value, int valueOffset, int valueLength) + throws RocksDBException; + protected native void put(long handle, byte[] key, int keyOffset, + int keyLength, byte[] value, int valueOffset, int valueLength, + long cfHandle) throws RocksDBException; + protected native void put(long handle, long writeOptHandle, byte[] key, + int keyOffset, int keyLength, byte[] value, int valueOffset, + int valueLength) throws RocksDBException; + protected native void put(long handle, long writeOptHandle, byte[] key, + int keyOffset, int keyLength, byte[] value, int valueOffset, + int valueLength, long cfHandle) throws RocksDBException; + protected native void write0(final long handle, long writeOptHandle, + long wbHandle) throws RocksDBException; + protected native void write1(final long handle, long writeOptHandle, + long wbwiHandle) throws RocksDBException; + protected native boolean keyMayExist(final long handle, final byte[] key, + final int keyOffset, final int keyLength, + final StringBuilder stringBuilder); + protected native boolean keyMayExist(final long handle, final byte[] key, + final int keyOffset, final int keyLength, final long cfHandle, + final StringBuilder stringBuilder); + protected native boolean keyMayExist(final long handle, + final long optionsHandle, final byte[] key, final int keyOffset, + final int keyLength, final StringBuilder stringBuilder); + protected native boolean keyMayExist(final long handle, + final long optionsHandle, final byte[] key, final int keyOffset, + final int keyLength, final long cfHandle, + final StringBuilder stringBuilder); + protected native void merge(long handle, byte[] key, int keyOffset, + int keyLength, byte[] value, int valueOffset, int valueLength) + throws RocksDBException; + protected native void merge(long handle, byte[] key, int keyOffset, + int keyLength, byte[] value, int valueOffset, int valueLength, + long cfHandle) throws RocksDBException; + protected native void merge(long handle, long writeOptHandle, byte[] key, + int keyOffset, int keyLength, byte[] value, int valueOffset, + int valueLength) throws RocksDBException; + protected native void merge(long handle, long writeOptHandle, byte[] key, + int keyOffset, int keyLength, byte[] value, int valueOffset, + int valueLength, long cfHandle) throws RocksDBException; + protected native int get(long handle, byte[] key, int keyOffset, + int keyLength, byte[] value, int valueOffset, int valueLength) + throws RocksDBException; + protected native int get(long handle, byte[] key, int keyOffset, + int keyLength, byte[] value, int valueOffset, int valueLength, + long cfHandle) throws RocksDBException; + protected native int get(long handle, long readOptHandle, byte[] key, + int keyOffset, int keyLength, byte[] value, int valueOffset, + int valueLength) throws RocksDBException; + protected native int get(long handle, long readOptHandle, byte[] key, + int keyOffset, int keyLength, byte[] value, int valueOffset, + int valueLength, long cfHandle) throws RocksDBException; + protected native byte[][] multiGet(final long dbHandle, final byte[][] keys, + final int[] keyOffsets, final int[] keyLengths); + protected native byte[][] multiGet(final long dbHandle, final byte[][] keys, + final int[] keyOffsets, final int[] keyLengths, + final long[] columnFamilyHandles); + protected native byte[][] multiGet(final long dbHandle, final long rOptHandle, + final byte[][] keys, final int[] keyOffsets, final int[] keyLengths); + protected native byte[][] multiGet(final long dbHandle, final long rOptHandle, + final byte[][] keys, final int[] keyOffsets, final int[] keyLengths, + final long[] columnFamilyHandles); + protected native byte[] get(long handle, byte[] key, int keyOffset, + int keyLength) throws RocksDBException; + protected native byte[] get(long handle, byte[] key, int keyOffset, + int keyLength, long cfHandle) throws RocksDBException; + protected native byte[] get(long handle, long readOptHandle, + byte[] key, int keyOffset, int keyLength) throws RocksDBException; + protected native byte[] get(long handle, long readOptHandle, byte[] key, + int keyOffset, int keyLength, long cfHandle) throws RocksDBException; + protected native void delete(long handle, byte[] key, int keyOffset, + int keyLength) throws RocksDBException; + protected native void delete(long handle, byte[] key, int keyOffset, + int keyLength, long cfHandle) throws RocksDBException; + protected native void delete(long handle, long writeOptHandle, byte[] key, + int keyOffset, int keyLength) throws RocksDBException; + protected native void delete(long handle, long writeOptHandle, byte[] key, + int keyOffset, int keyLength, long cfHandle) throws RocksDBException; + protected native void singleDelete( + long handle, byte[] key, int keyLen) throws RocksDBException; + protected native void singleDelete( + long handle, byte[] key, int keyLen, long cfHandle) + throws RocksDBException; + protected native void singleDelete( + long handle, long writeOptHandle, + byte[] key, int keyLen) throws RocksDBException; + protected native void singleDelete( + long handle, long writeOptHandle, + byte[] key, int keyLen, long cfHandle) throws RocksDBException; + protected native void deleteRange(long handle, byte[] beginKey, int beginKeyOffset, + int beginKeyLength, byte[] endKey, int endKeyOffset, int endKeyLength) + throws RocksDBException; + protected native void deleteRange(long handle, byte[] beginKey, int beginKeyOffset, + int beginKeyLength, byte[] endKey, int endKeyOffset, int endKeyLength, long cfHandle) + throws RocksDBException; + protected native void deleteRange(long handle, long writeOptHandle, byte[] beginKey, + int beginKeyOffset, int beginKeyLength, byte[] endKey, int endKeyOffset, int endKeyLength) + throws RocksDBException; + protected native void deleteRange(long handle, long writeOptHandle, byte[] beginKey, + int beginKeyOffset, int beginKeyLength, byte[] endKey, int endKeyOffset, int endKeyLength, + long cfHandle) throws RocksDBException; + protected native String getProperty0(long nativeHandle, + String property, int propertyLength) throws RocksDBException; + protected native String getProperty0(long nativeHandle, long cfHandle, + String property, int propertyLength) throws RocksDBException; + protected native long getLongProperty(long nativeHandle, String property, + int propertyLength) throws RocksDBException; + protected native long getLongProperty(long nativeHandle, long cfHandle, + String property, int propertyLength) throws RocksDBException; + protected native long iterator(long handle); + protected native long iterator(long handle, long readOptHandle); + protected native long iteratorCF(long handle, long cfHandle); + protected native long iteratorCF(long handle, long cfHandle, + long readOptHandle); + protected native long[] iterators(final long handle, + final long[] columnFamilyHandles, final long readOptHandle) + throws RocksDBException; + protected native long getSnapshot(long nativeHandle); + protected native void releaseSnapshot(long nativeHandle, long snapshotHandle); + @Override protected final native void disposeInternal(final long handle); + private native long getDefaultColumnFamily(long handle); + private native long createColumnFamily(final long handle, + final byte[] columnFamilyName, final long columnFamilyOptions) + throws RocksDBException; + private native void dropColumnFamily(long handle, long cfHandle) + throws RocksDBException; + private native void flush(long handle, long flushOptHandle) + throws RocksDBException; + private native void flush(long handle, long flushOptHandle, long cfHandle) + throws RocksDBException; + private native void compactRange0(long handle, boolean reduce_level, + int target_level, int target_path_id) throws RocksDBException; + private native void compactRange0(long handle, byte[] begin, int beginLen, + byte[] end, int endLen, boolean reduce_level, int target_level, + int target_path_id) throws RocksDBException; + private native void compactRange(long handle, boolean reduce_level, + int target_level, int target_path_id, long cfHandle) + throws RocksDBException; + private native void compactRange(long handle, byte[] begin, int beginLen, + byte[] end, int endLen, boolean reduce_level, int target_level, + int target_path_id, long cfHandle) throws RocksDBException; + private native void pauseBackgroundWork(long handle) throws RocksDBException; + private native void continueBackgroundWork(long handle) throws RocksDBException; + private native long getLatestSequenceNumber(long handle); + private native void disableFileDeletions(long handle) throws RocksDBException; + private native void enableFileDeletions(long handle, boolean force) + throws RocksDBException; + private native long getUpdatesSince(long handle, long sequenceNumber) + throws RocksDBException; + private native void setOptions(long handle, long cfHandle, String[] keys, + String[] values) throws RocksDBException; + private native void ingestExternalFile(long handle, long cfHandle, + String[] filePathList, int filePathListLen, + long ingest_external_file_options_handle) throws RocksDBException; + protected DBOptionsInterface options_; +} diff --git a/java/src/main/java/org/rocksdb/RocksDBException.java b/java/src/main/java/org/rocksdb/RocksDBException.java new file mode 100644 index 00000000000..8b035f458f3 --- /dev/null +++ b/java/src/main/java/org/rocksdb/RocksDBException.java @@ -0,0 +1,44 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * A RocksDBException encapsulates the error of an operation. This exception + * type is used to describe an internal error from the c++ rocksdb library. + */ +public class RocksDBException extends Exception { + + /* @Nullable */ private final Status status; + + /** + * The private construct used by a set of public static factory method. + * + * @param msg the specified error message. + */ + public RocksDBException(final String msg) { + this(msg, null); + } + + public RocksDBException(final String msg, final Status status) { + super(msg); + this.status = status; + } + + public RocksDBException(final Status status) { + super(status.getState() != null ? status.getState() + : status.getCodeString()); + this.status = status; + } + + /** + * Get the status returned from RocksDB + * + * @return The status reported by RocksDB, or null if no status is available + */ + public Status getStatus() { + return status; + } +} diff --git a/java/src/main/java/org/rocksdb/RocksEnv.java b/java/src/main/java/org/rocksdb/RocksEnv.java new file mode 100644 index 00000000000..8fe61fd451f --- /dev/null +++ b/java/src/main/java/org/rocksdb/RocksEnv.java @@ -0,0 +1,43 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + *

    A RocksEnv is an interface used by the rocksdb implementation to access + * operating system functionality like the filesystem etc.

    + * + *

    All Env implementations are safe for concurrent access from + * multiple threads without any external synchronization.

    + */ +public class RocksEnv extends Env { + + /** + *

    Package-private constructor that uses the specified native handle + * to construct a RocksEnv.

    + * + *

    Note that the ownership of the input handle + * belongs to the caller, and the newly created RocksEnv will not take + * the ownership of the input handle. As a result, calling + * {@code dispose()} of the created RocksEnv will be no-op.

    + */ + RocksEnv(final long handle) { + super(handle); + disOwnNativeHandle(); + } + + /** + *

    The helper function of {@link #dispose()} which all subclasses of + * {@link RocksObject} must implement to release their associated C++ + * resource.

    + * + *

    Note: this class is used to use the default + * RocksEnv with RocksJava. The default env allocation is managed + * by C++.

    + */ + @Override + protected final void disposeInternal(final long handle) { + } +} diff --git a/java/src/main/java/org/rocksdb/RocksIterator.java b/java/src/main/java/org/rocksdb/RocksIterator.java new file mode 100644 index 00000000000..9e9c6480928 --- /dev/null +++ b/java/src/main/java/org/rocksdb/RocksIterator.java @@ -0,0 +1,64 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + *

    An iterator that yields a sequence of key/value pairs from a source. + * Multiple implementations are provided by this library. + * In particular, iterators are provided + * to access the contents of a Table or a DB.

    + * + *

    Multiple threads can invoke const methods on an RocksIterator without + * external synchronization, but if any of the threads may call a + * non-const method, all threads accessing the same RocksIterator must use + * external synchronization.

    + * + * @see org.rocksdb.RocksObject + */ +public class RocksIterator extends AbstractRocksIterator { + protected RocksIterator(RocksDB rocksDB, long nativeHandle) { + super(rocksDB, nativeHandle); + } + + /** + *

    Return the key for the current entry. The underlying storage for + * the returned slice is valid only until the next modification of + * the iterator.

    + * + *

    REQUIRES: {@link #isValid()}

    + * + * @return key for the current entry. + */ + public byte[] key() { + assert(isOwningHandle()); + return key0(nativeHandle_); + } + + /** + *

    Return the value for the current entry. The underlying storage for + * the returned slice is valid only until the next modification of + * the iterator.

    + * + *

    REQUIRES: !AtEnd() && !AtStart()

    + * @return value for the current entry. + */ + public byte[] value() { + assert(isOwningHandle()); + return value0(nativeHandle_); + } + + @Override protected final native void disposeInternal(final long handle); + @Override final native boolean isValid0(long handle); + @Override final native void seekToFirst0(long handle); + @Override final native void seekToLast0(long handle); + @Override final native void next0(long handle); + @Override final native void prev0(long handle); + @Override final native void seek0(long handle, byte[] target, int targetLen); + @Override final native void status0(long handle) throws RocksDBException; + + private native byte[] key0(long handle); + private native byte[] value0(long handle); +} diff --git a/java/src/main/java/org/rocksdb/RocksIteratorInterface.java b/java/src/main/java/org/rocksdb/RocksIteratorInterface.java new file mode 100644 index 00000000000..12fdbb19731 --- /dev/null +++ b/java/src/main/java/org/rocksdb/RocksIteratorInterface.java @@ -0,0 +1,80 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + *

    Defines the interface for an Iterator which provides + * access to data one entry at a time. Multiple implementations + * are provided by this library. In particular, iterators are provided + * to access the contents of a DB and Write Batch.

    + * + *

    Multiple threads can invoke const methods on an RocksIterator without + * external synchronization, but if any of the threads may call a + * non-const method, all threads accessing the same RocksIterator must use + * external synchronization.

    + * + * @see org.rocksdb.RocksObject + */ +public interface RocksIteratorInterface { + + /** + *

    An iterator is either positioned at an entry, or + * not valid. This method returns true if the iterator is valid.

    + * + * @return true if iterator is valid. + */ + boolean isValid(); + + /** + *

    Position at the first entry in the source. The iterator is Valid() + * after this call if the source is not empty.

    + */ + void seekToFirst(); + + /** + *

    Position at the last entry in the source. The iterator is + * valid after this call if the source is not empty.

    + */ + void seekToLast(); + + /** + *

    Position at the first entry in the source whose key is that or + * past target.

    + * + *

    The iterator is valid after this call if the source contains + * a key that comes at or past target.

    + * + * @param target byte array describing a key or a + * key prefix to seek for. + */ + void seek(byte[] target); + + /** + *

    Moves to the next entry in the source. After this call, Valid() is + * true if the iterator was not positioned at the last entry in the source.

    + * + *

    REQUIRES: {@link #isValid()}

    + */ + void next(); + + /** + *

    Moves to the previous entry in the source. After this call, Valid() is + * true if the iterator was not positioned at the first entry in source.

    + * + *

    REQUIRES: {@link #isValid()}

    + */ + void prev(); + + /** + *

    If an error has occurred, return it. Else return an ok status. + * If non-blocking IO is requested and this operation cannot be + * satisfied without doing some IO, then this returns Status::Incomplete().

    + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + void status() throws RocksDBException; +} diff --git a/java/src/main/java/org/rocksdb/RocksMemEnv.java b/java/src/main/java/org/rocksdb/RocksMemEnv.java new file mode 100644 index 00000000000..d18d0ceb977 --- /dev/null +++ b/java/src/main/java/org/rocksdb/RocksMemEnv.java @@ -0,0 +1,27 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * RocksDB memory environment. + */ +public class RocksMemEnv extends Env { + + /** + *

    Creates a new RocksDB environment that stores its data + * in memory and delegates all non-file-storage tasks to + * base_env. The caller must delete the result when it is + * no longer needed.

    + * + *

    {@code *base_env} must remain live while the result is in use.

    + */ + public RocksMemEnv() { + super(createMemEnv()); + } + + private static native long createMemEnv(); + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/RocksMutableObject.java b/java/src/main/java/org/rocksdb/RocksMutableObject.java new file mode 100644 index 00000000000..e92289dc0c5 --- /dev/null +++ b/java/src/main/java/org/rocksdb/RocksMutableObject.java @@ -0,0 +1,87 @@ +// Copyright (c) 2016, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * RocksMutableObject is an implementation of {@link AbstractNativeReference} + * whose reference to the underlying native C++ object can change. + * + *

    The use of {@code RocksMutableObject} should be kept to a minimum, as it + * has synchronization overheads and introduces complexity. Instead it is + * recommended to use {@link RocksObject} where possible.

    + */ +public abstract class RocksMutableObject extends AbstractNativeReference { + + /** + * An mutable reference to the value of the C++ pointer pointing to some + * underlying native RocksDB C++ object. + */ + private long nativeHandle_; + private boolean owningHandle_; + + protected RocksMutableObject() { + } + + protected RocksMutableObject(final long nativeHandle) { + this.nativeHandle_ = nativeHandle; + this.owningHandle_ = true; + } + + /** + * Closes the existing handle, and changes the handle to the new handle + * + * @param newNativeHandle The C++ pointer to the new native object + * @param owningNativeHandle true if we own the new native object + */ + public synchronized void resetNativeHandle(final long newNativeHandle, + final boolean owningNativeHandle) { + close(); + setNativeHandle(newNativeHandle, owningNativeHandle); + } + + /** + * Sets the handle (C++ pointer) of the underlying C++ native object + * + * @param nativeHandle The C++ pointer to the native object + * @param owningNativeHandle true if we own the native object + */ + public synchronized void setNativeHandle(final long nativeHandle, + final boolean owningNativeHandle) { + this.nativeHandle_ = nativeHandle; + this.owningHandle_ = owningNativeHandle; + } + + @Override + protected synchronized boolean isOwningHandle() { + return this.owningHandle_; + } + + /** + * Gets the value of the C++ pointer pointing to the underlying + * native C++ object + * + * @return the pointer value for the native object + */ + protected synchronized long getNativeHandle() { + assert (this.nativeHandle_ != 0); + return this.nativeHandle_; + } + + @Override + public synchronized final void close() { + if (isOwningHandle()) { + disposeInternal(); + this.owningHandle_ = false; + this.nativeHandle_ = 0; + } + } + + protected void disposeInternal() { + disposeInternal(nativeHandle_); + } + + protected abstract void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/RocksObject.java b/java/src/main/java/org/rocksdb/RocksObject.java new file mode 100644 index 00000000000..545dd896a06 --- /dev/null +++ b/java/src/main/java/org/rocksdb/RocksObject.java @@ -0,0 +1,41 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * RocksObject is an implementation of {@link AbstractNativeReference} which + * has an immutable and therefore thread-safe reference to the underlying + * native C++ RocksDB object. + *

    + * RocksObject is the base-class of almost all RocksDB classes that have a + * pointer to some underlying native C++ {@code rocksdb} object.

    + *

    + * The use of {@code RocksObject} should always be preferred over + * {@link RocksMutableObject}.

    + */ +public abstract class RocksObject extends AbstractImmutableNativeReference { + + /** + * An immutable reference to the value of the C++ pointer pointing to some + * underlying native RocksDB C++ object. + */ + protected final long nativeHandle_; + + protected RocksObject(final long nativeHandle) { + super(true); + this.nativeHandle_ = nativeHandle; + } + + /** + * Deletes underlying C++ object pointer. + */ + @Override + protected void disposeInternal() { + disposeInternal(nativeHandle_); + } + + protected abstract void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/SkipListMemTableConfig.java b/java/src/main/java/org/rocksdb/SkipListMemTableConfig.java new file mode 100644 index 00000000000..e31e1991613 --- /dev/null +++ b/java/src/main/java/org/rocksdb/SkipListMemTableConfig.java @@ -0,0 +1,50 @@ +package org.rocksdb; + +/** + * The config for skip-list memtable representation. + */ +public class SkipListMemTableConfig extends MemTableConfig { + + public static final long DEFAULT_LOOKAHEAD = 0; + + /** + * SkipListMemTableConfig constructor + */ + public SkipListMemTableConfig() { + lookahead_ = DEFAULT_LOOKAHEAD; + } + + /** + * Sets lookahead for SkipList + * + * @param lookahead If non-zero, each iterator's seek operation + * will start the search from the previously visited record + * (doing at most 'lookahead' steps). This is an + * optimization for the access pattern including many + * seeks with consecutive keys. + * @return the current instance of SkipListMemTableConfig + */ + public SkipListMemTableConfig setLookahead(final long lookahead) { + lookahead_ = lookahead; + return this; + } + + /** + * Returns the currently set lookahead value. + * + * @return lookahead value + */ + public long lookahead() { + return lookahead_; + } + + + @Override protected long newMemTableFactoryHandle() { + return newMemTableFactoryHandle0(lookahead_); + } + + private native long newMemTableFactoryHandle0(long lookahead) + throws IllegalArgumentException; + + private long lookahead_; +} diff --git a/java/src/main/java/org/rocksdb/Slice.java b/java/src/main/java/org/rocksdb/Slice.java new file mode 100644 index 00000000000..a122c3769d8 --- /dev/null +++ b/java/src/main/java/org/rocksdb/Slice.java @@ -0,0 +1,112 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + *

    Base class for slices which will receive + * byte[] based access to the underlying data.

    + * + *

    byte[] backed slices typically perform better with + * small keys and values. When using larger keys and + * values consider using {@link org.rocksdb.DirectSlice}

    + */ +public class Slice extends AbstractSlice { + + /** + * Indicates whether we have to free the memory pointed to by the Slice + */ + private volatile boolean cleared; + private volatile long internalBufferOffset = 0; + + /** + *

    Called from JNI to construct a new Java Slice + * without an underlying C++ object set + * at creation time.

    + * + *

    Note: You should be aware that + * {@see org.rocksdb.RocksObject#disOwnNativeHandle()} is intentionally + * called from the default Slice constructor, and that it is marked as + * private. This is so that developers cannot construct their own default + * Slice objects (at present). As developers cannot construct their own + * Slice objects through this, they are not creating underlying C++ Slice + * objects, and so there is nothing to free (dispose) from Java.

    + */ + @SuppressWarnings("unused") + private Slice() { + super(); + } + + /** + *

    Constructs a slice where the data is taken from + * a String.

    + * + * @param str String value. + */ + public Slice(final String str) { + super(createNewSliceFromString(str)); + } + + /** + *

    Constructs a slice where the data is a copy of + * the byte array from a specific offset.

    + * + * @param data byte array. + * @param offset offset within the byte array. + */ + public Slice(final byte[] data, final int offset) { + super(createNewSlice0(data, offset)); + } + + /** + *

    Constructs a slice where the data is a copy of + * the byte array.

    + * + * @param data byte array. + */ + public Slice(final byte[] data) { + super(createNewSlice1(data)); + } + + @Override + public void clear() { + clear0(getNativeHandle(), !cleared, internalBufferOffset); + cleared = true; + } + + @Override + public void removePrefix(final int n) { + removePrefix0(getNativeHandle(), n); + this.internalBufferOffset += n; + } + + /** + *

    Deletes underlying C++ slice pointer + * and any buffered data.

    + * + *

    + * Note that this function should be called only after all + * RocksDB instances referencing the slice are closed. + * Otherwise an undefined behavior will occur.

    + */ + @Override + protected void disposeInternal() { + final long nativeHandle = getNativeHandle(); + if(!cleared) { + disposeInternalBuf(nativeHandle, internalBufferOffset); + } + super.disposeInternal(nativeHandle); + } + + @Override protected final native byte[] data0(long handle); + private native static long createNewSlice0(final byte[] data, + final int length); + private native static long createNewSlice1(final byte[] data); + private native void clear0(long handle, boolean internalBuffer, + long internalBufferOffset); + private native void removePrefix0(long handle, int length); + private native void disposeInternalBuf(final long handle, + long internalBufferOffset); +} diff --git a/java/src/main/java/org/rocksdb/Snapshot.java b/java/src/main/java/org/rocksdb/Snapshot.java new file mode 100644 index 00000000000..a6b53f495f2 --- /dev/null +++ b/java/src/main/java/org/rocksdb/Snapshot.java @@ -0,0 +1,37 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Snapshot of database + */ +public class Snapshot extends RocksObject { + Snapshot(final long nativeHandle) { + super(nativeHandle); + } + + /** + * Return the associated sequence number; + * + * @return the associated sequence number of + * this snapshot. + */ + public long getSequenceNumber() { + assert(isOwningHandle()); + return getSequenceNumber(nativeHandle_); + } + + /** + * Dont release C++ Snapshot pointer. The pointer + * to the snapshot is released by the database + * instance. + */ + @Override + protected final void disposeInternal(final long handle) { + } + + private native long getSequenceNumber(long handle); +} diff --git a/java/src/main/java/org/rocksdb/SstFileWriter.java b/java/src/main/java/org/rocksdb/SstFileWriter.java new file mode 100644 index 00000000000..5f35f0f61db --- /dev/null +++ b/java/src/main/java/org/rocksdb/SstFileWriter.java @@ -0,0 +1,256 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * SstFileWriter is used to create sst files that can be added to the + * database later. All keys in files generated by SstFileWriter will have + * sequence number = 0. + */ +public class SstFileWriter extends RocksObject { + static { + RocksDB.loadLibrary(); + } + + /** + * SstFileWriter Constructor. + * + * @param envOptions {@link org.rocksdb.EnvOptions} instance. + * @param options {@link org.rocksdb.Options} instance. + * @param comparator the comparator to specify the ordering of keys. + * + * @deprecated Use {@link #SstFileWriter(EnvOptions, Options)}. + * Passing an explicit comparator is deprecated in lieu of passing the + * comparator as part of options. Use the other constructor instead. + */ + @Deprecated + public SstFileWriter(final EnvOptions envOptions, final Options options, + final AbstractComparator> comparator) { + super(newSstFileWriter( + envOptions.nativeHandle_, options.nativeHandle_, comparator.getNativeHandle())); + } + + /** + * SstFileWriter Constructor. + * + * @param envOptions {@link org.rocksdb.EnvOptions} instance. + * @param options {@link org.rocksdb.Options} instance. + */ + public SstFileWriter(final EnvOptions envOptions, final Options options) { + super(newSstFileWriter( + envOptions.nativeHandle_, options.nativeHandle_)); + } + + /** + * Prepare SstFileWriter to write to a file. + * + * @param filePath the location of file + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void open(final String filePath) throws RocksDBException { + open(nativeHandle_, filePath); + } + + /** + * Add a Put key with value to currently opened file. + * + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * + * @deprecated Use {@link #put(Slice, Slice)} + */ + @Deprecated + public void add(final Slice key, final Slice value) + throws RocksDBException { + put(nativeHandle_, key.getNativeHandle(), value.getNativeHandle()); + } + + /** + * Add a Put key with value to currently opened file. + * + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + * + * @deprecated Use {@link #put(DirectSlice, DirectSlice)} + */ + @Deprecated + public void add(final DirectSlice key, final DirectSlice value) + throws RocksDBException { + put(nativeHandle_, key.getNativeHandle(), value.getNativeHandle()); + } + + /** + * Add a Put key with value to currently opened file. + * + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void put(final Slice key, final Slice value) throws RocksDBException { + put(nativeHandle_, key.getNativeHandle(), value.getNativeHandle()); + } + + /** + * Add a Put key with value to currently opened file. + * + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void put(final DirectSlice key, final DirectSlice value) + throws RocksDBException { + put(nativeHandle_, key.getNativeHandle(), value.getNativeHandle()); + } + + /** + * Add a Put key with value to currently opened file. + * + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ +public void put(final byte[] key, final byte[] value) + throws RocksDBException { + put(nativeHandle_, key, value); +} + + /** + * Add a Merge key with value to currently opened file. + * + * @param key the specified key to be merged. + * @param value the value to be merged with the current value for + * the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void merge(final Slice key, final Slice value) + throws RocksDBException { + merge(nativeHandle_, key.getNativeHandle(), value.getNativeHandle()); + } + + /** + * Add a Merge key with value to currently opened file. + * + * @param key the specified key to be merged. + * @param value the value to be merged with the current value for + * the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void merge(final byte[] key, final byte[] value) + throws RocksDBException { + merge(nativeHandle_, key, value); + } + + /** + * Add a Merge key with value to currently opened file. + * + * @param key the specified key to be merged. + * @param value the value to be merged with the current value for + * the specified key. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void merge(final DirectSlice key, final DirectSlice value) + throws RocksDBException { + merge(nativeHandle_, key.getNativeHandle(), value.getNativeHandle()); + } + + /** + * Add a deletion key to currently opened file. + * + * @param key the specified key to be deleted. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void delete(final Slice key) throws RocksDBException { + delete(nativeHandle_, key.getNativeHandle()); + } + + /** + * Add a deletion key to currently opened file. + * + * @param key the specified key to be deleted. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void delete(final DirectSlice key) throws RocksDBException { + delete(nativeHandle_, key.getNativeHandle()); + } + + /** + * Add a deletion key to currently opened file. + * + * @param key the specified key to be deleted. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void delete(final byte[] key) throws RocksDBException { + delete(nativeHandle_, key); + } + + /** + * Finish the process and close the sst file. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public void finish() throws RocksDBException { + finish(nativeHandle_); + } + + private native static long newSstFileWriter( + final long envOptionsHandle, final long optionsHandle, + final long userComparatorHandle); + + private native static long newSstFileWriter(final long envOptionsHandle, + final long optionsHandle); + + private native void open(final long handle, final String filePath) + throws RocksDBException; + + private native void put(final long handle, final long keyHandle, + final long valueHandle) throws RocksDBException; + + private native void put(final long handle, final byte[] key, + final byte[] value) throws RocksDBException; + + private native void merge(final long handle, final long keyHandle, + final long valueHandle) throws RocksDBException; + + private native void merge(final long handle, final byte[] key, + final byte[] value) throws RocksDBException; + + private native void delete(final long handle, final long keyHandle) + throws RocksDBException; + + private native void delete(final long handle, final byte[] key) + throws RocksDBException; + + private native void finish(final long handle) throws RocksDBException; + + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/Statistics.java b/java/src/main/java/org/rocksdb/Statistics.java new file mode 100644 index 00000000000..10c072c897e --- /dev/null +++ b/java/src/main/java/org/rocksdb/Statistics.java @@ -0,0 +1,149 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.EnumSet; + +/** + * Statistics to analyze the performance of a db. Pointer for statistics object + * is managed by Options class. + */ +public class Statistics extends RocksObject { + + public Statistics() { + super(newStatistics()); + } + + public Statistics(final Statistics otherStatistics) { + super(newStatistics(otherStatistics.nativeHandle_)); + } + + public Statistics(final EnumSet ignoreHistograms) { + super(newStatistics(toArrayValues(ignoreHistograms))); + } + + public Statistics(final EnumSet ignoreHistograms, final Statistics otherStatistics) { + super(newStatistics(toArrayValues(ignoreHistograms), otherStatistics.nativeHandle_)); + } + + /** + * Intentionally package-private. + * + * Used from {@link DBOptions#statistics()} + * + * @param existingStatisticsHandle The C++ pointer to an existing statistics object + */ + Statistics(final long existingStatisticsHandle) { + super(existingStatisticsHandle); + } + + private static byte[] toArrayValues(final EnumSet histogramTypes) { + final byte[] values = new byte[histogramTypes.size()]; + int i = 0; + for(final HistogramType histogramType : histogramTypes) { + values[i++] = histogramType.getValue(); + } + return values; + } + + /** + * Gets the current stats level. + * + * @return The stats level. + */ + public StatsLevel statsLevel() { + return StatsLevel.getStatsLevel(statsLevel(nativeHandle_)); + } + + /** + * Sets the stats level. + * + * @param statsLevel The stats level to set. + */ + public void setStatsLevel(final StatsLevel statsLevel) { + setStatsLevel(nativeHandle_, statsLevel.getValue()); + } + + /** + * Get the count for a ticker. + * + * @param tickerType The ticker to get the count for + * + * @return The count for the ticker + */ + public long getTickerCount(final TickerType tickerType) { + assert(isOwningHandle()); + return getTickerCount(nativeHandle_, tickerType.getValue()); + } + + /** + * Get the count for a ticker and reset the tickers count. + * + * @param tickerType The ticker to get the count for + * + * @return The count for the ticker + */ + public long getAndResetTickerCount(final TickerType tickerType) { + assert(isOwningHandle()); + return getAndResetTickerCount(nativeHandle_, tickerType.getValue()); + } + + /** + * Gets the histogram data for a particular histogram. + * + * @param histogramType The histogram to retrieve the data for + * + * @return The histogram data + */ + public HistogramData getHistogramData(final HistogramType histogramType) { + assert(isOwningHandle()); + return getHistogramData(nativeHandle_, histogramType.getValue()); + } + + /** + * Gets a string representation of a particular histogram. + * + * @param histogramType The histogram to retrieve the data for + * + * @return A string representation of the histogram data + */ + public String getHistogramString(final HistogramType histogramType) { + assert(isOwningHandle()); + return getHistogramString(nativeHandle_, histogramType.getValue()); + } + + /** + * Resets all ticker and histogram stats. + */ + public void reset() throws RocksDBException { + assert(isOwningHandle()); + reset(nativeHandle_); + } + + /** + * String representation of the statistic object. + */ + public String toString() { + assert(isOwningHandle()); + return toString(nativeHandle_); + } + + private native static long newStatistics(); + private native static long newStatistics(final long otherStatisticsHandle); + private native static long newStatistics(final byte[] ignoreHistograms); + private native static long newStatistics(final byte[] ignoreHistograms, final long otherStatisticsHandle); + + @Override protected final native void disposeInternal(final long handle); + + private native byte statsLevel(final long handle); + private native void setStatsLevel(final long handle, final byte statsLevel); + private native long getTickerCount(final long handle, final byte tickerType); + private native long getAndResetTickerCount(final long handle, final byte tickerType); + private native HistogramData getHistogramData(final long handle, final byte histogramType); + private native String getHistogramString(final long handle, final byte histogramType); + private native void reset(final long nativeHandle) throws RocksDBException; + private native String toString(final long nativeHandle); +} diff --git a/java/org/rocksdb/StatisticsCollector.java b/java/src/main/java/org/rocksdb/StatisticsCollector.java similarity index 57% rename from java/org/rocksdb/StatisticsCollector.java rename to java/src/main/java/org/rocksdb/StatisticsCollector.java index 29815c46d61..48cf8af88e6 100644 --- a/java/org/rocksdb/StatisticsCollector.java +++ b/java/src/main/java/org/rocksdb/StatisticsCollector.java @@ -1,25 +1,23 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). package org.rocksdb; import java.util.List; -import java.util.concurrent.ArrayBlockingQueue; import java.util.concurrent.Executors; import java.util.concurrent.ExecutorService; import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicBoolean; /** - * Helper class to collect DB statistics periodically at a period specified in + *

    Helper class to collect DB statistics periodically at a period specified in * constructor. Callback function (provided in constructor) is called with - * every statistics collection. + * every statistics collection.

    * - * Caller should call start() to start statistics collection. Shutdown() should + *

    Caller should call start() to start statistics collection. Shutdown() should * be called to stop stats collection and should be called before statistics ( - * provided in constructor) reference has been disposed. + * provided in constructor) reference has been disposed.

    */ public class StatisticsCollector { private final List _statsCollectorInputList; @@ -29,13 +27,14 @@ public class StatisticsCollector { /** * Constructor for statistics collector. - * + * * @param statsCollectorInputList List of statistics collector input. - * @param statsCollectionIntervalInMilliSeconds Statistics collection time + * @param statsCollectionIntervalInMilliSeconds Statistics collection time * period (specified in milliseconds). */ - public StatisticsCollector(List statsCollectorInputList, - int statsCollectionIntervalInMilliSeconds) { + public StatisticsCollector( + final List statsCollectorInputList, + final int statsCollectionIntervalInMilliSeconds) { _statsCollectorInputList = statsCollectorInputList; _statsCollectionInterval = statsCollectionIntervalInMilliSeconds; @@ -48,11 +47,12 @@ public void start() { /** * Shuts down statistics collector. - * + * * @param shutdownTimeout Time in milli-seconds to wait for shutdown before * killing the collection process. + * @throws java.lang.InterruptedException thrown if Threads are interrupted. */ - public void shutDown(int shutdownTimeout) throws InterruptedException { + public void shutDown(final int shutdownTimeout) throws InterruptedException { _isRunning = false; _executorService.shutdownNow(); @@ -70,34 +70,38 @@ public void run() { try { if(Thread.currentThread().isInterrupted()) { break; - } - for(StatsCollectorInput statsCollectorInput : + } + for(final StatsCollectorInput statsCollectorInput : _statsCollectorInputList) { Statistics statistics = statsCollectorInput.getStatistics(); StatisticsCollectorCallback statsCallback = statsCollectorInput.getCallback(); - - // Collect ticker data - for(TickerType ticker : TickerType.values()) { - long tickerValue = statistics.getTickerCount(ticker); - statsCallback.tickerCallback(ticker, tickerValue); + + // Collect ticker data + for(final TickerType ticker : TickerType.values()) { + if(ticker != TickerType.TICKER_ENUM_MAX) { + final long tickerValue = statistics.getTickerCount(ticker); + statsCallback.tickerCallback(ticker, tickerValue); + } } // Collect histogram data - for(HistogramType histogramType : HistogramType.values()) { - HistogramData histogramData = - statistics.geHistogramData(histogramType); - statsCallback.histogramCallback(histogramType, histogramData); + for(final HistogramType histogramType : HistogramType.values()) { + if(histogramType != HistogramType.HISTOGRAM_ENUM_MAX) { + final HistogramData histogramData = + statistics.getHistogramData(histogramType); + statsCallback.histogramCallback(histogramType, histogramData); + } } Thread.sleep(_statsCollectionInterval); } } - catch (InterruptedException e) { + catch (final InterruptedException e) { Thread.currentThread().interrupt(); break; } - catch (Exception e) { + catch (final Exception e) { throw new RuntimeException("Error while calculating statistics", e); } } diff --git a/java/org/rocksdb/StatisticsCollectorCallback.java b/java/src/main/java/org/rocksdb/StatisticsCollectorCallback.java similarity index 70% rename from java/org/rocksdb/StatisticsCollectorCallback.java rename to java/src/main/java/org/rocksdb/StatisticsCollectorCallback.java index a955ec216cc..f3785b15f6c 100644 --- a/java/org/rocksdb/StatisticsCollectorCallback.java +++ b/java/src/main/java/org/rocksdb/StatisticsCollectorCallback.java @@ -1,22 +1,20 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). package org.rocksdb; /** * Callback interface provided to StatisticsCollector. - * + * * Thread safety: - * StatisticsCollector doesn't make any guarantees about thread safety. + * StatisticsCollector doesn't make any guarantees about thread safety. * If the same reference of StatisticsCollectorCallback is passed to multiple - * StatisticsCollector references, then its the responsibility of the + * StatisticsCollector references, then its the responsibility of the * user to make StatisticsCollectorCallback's implementation thread-safe. - * - * @param tickerType - * @param tickerCount -*/ + * + */ public interface StatisticsCollectorCallback { /** * Callback function to get ticker values. diff --git a/java/org/rocksdb/StatsCollectorInput.java b/java/src/main/java/org/rocksdb/StatsCollectorInput.java similarity index 61% rename from java/org/rocksdb/StatsCollectorInput.java rename to java/src/main/java/org/rocksdb/StatsCollectorInput.java index a1aa928d335..5bf43ade5a6 100644 --- a/java/org/rocksdb/StatsCollectorInput.java +++ b/java/src/main/java/org/rocksdb/StatsCollectorInput.java @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). package org.rocksdb; @@ -12,23 +12,23 @@ public class StatsCollectorInput { private final Statistics _statistics; private final StatisticsCollectorCallback _statsCallback; - + /** * Constructor for StatsCollectorInput. - * + * * @param statistics Reference of DB statistics. * @param statsCallback Reference of statistics callback interface. */ - public StatsCollectorInput(Statistics statistics, - StatisticsCollectorCallback statsCallback) { + public StatsCollectorInput(final Statistics statistics, + final StatisticsCollectorCallback statsCallback) { _statistics = statistics; _statsCallback = statsCallback; } - + public Statistics getStatistics() { return _statistics; } - + public StatisticsCollectorCallback getCallback() { return _statsCallback; } diff --git a/java/src/main/java/org/rocksdb/StatsLevel.java b/java/src/main/java/org/rocksdb/StatsLevel.java new file mode 100644 index 00000000000..cc2a87c6a21 --- /dev/null +++ b/java/src/main/java/org/rocksdb/StatsLevel.java @@ -0,0 +1,65 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * The level of Statistics to report. + */ +public enum StatsLevel { + /** + * Collect all stats except time inside mutex lock AND time spent on + * compression. + */ + EXCEPT_DETAILED_TIMERS((byte) 0x0), + + /** + * Collect all stats except the counters requiring to get time inside the + * mutex lock. + */ + EXCEPT_TIME_FOR_MUTEX((byte) 0x1), + + /** + * Collect all stats, including measuring duration of mutex operations. + * + * If getting time is expensive on the platform to run, it can + * reduce scalability to more threads, especially for writes. + */ + ALL((byte) 0x2); + + private final byte value; + + StatsLevel(final byte value) { + this.value = value; + } + + /** + *

    Returns the byte value of the enumerations value.

    + * + * @return byte representation + */ + public byte getValue() { + return value; + } + + /** + * Get StatsLevel by byte value. + * + * @param value byte representation of StatsLevel. + * + * @return {@link org.rocksdb.StatsLevel} instance. + * @throws java.lang.IllegalArgumentException if an invalid + * value is provided. + */ + public static StatsLevel getStatsLevel(final byte value) { + for (final StatsLevel statsLevel : StatsLevel.values()) { + if (statsLevel.getValue() == value){ + return statsLevel; + } + } + throw new IllegalArgumentException( + "Illegal value provided for InfoLogLevel."); + } +} diff --git a/java/src/main/java/org/rocksdb/Status.java b/java/src/main/java/org/rocksdb/Status.java new file mode 100644 index 00000000000..d34b72c6913 --- /dev/null +++ b/java/src/main/java/org/rocksdb/Status.java @@ -0,0 +1,113 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Represents the status returned by a function call in RocksDB. + * + * Currently only used with {@link RocksDBException} when the + * status is not {@link Code#Ok} + */ +public class Status { + private final Code code; + /* @Nullable */ private final SubCode subCode; + /* @Nullable */ private final String state; + + public Status(final Code code, final SubCode subCode, final String state) { + this.code = code; + this.subCode = subCode; + this.state = state; + } + + /** + * Intentionally private as this will be called from JNI + */ + private Status(final byte code, final byte subCode, final String state) { + this.code = Code.getCode(code); + this.subCode = SubCode.getSubCode(subCode); + this.state = state; + } + + public Code getCode() { + return code; + } + + public SubCode getSubCode() { + return subCode; + } + + public String getState() { + return state; + } + + public String getCodeString() { + final StringBuilder builder = new StringBuilder() + .append(code.name()); + if(subCode != null && subCode != SubCode.None) { + builder.append("(") + .append(subCode.name()) + .append(")"); + } + return builder.toString(); + } + + public enum Code { + Ok( (byte)0x0), + NotFound( (byte)0x1), + Corruption( (byte)0x2), + NotSupported( (byte)0x3), + InvalidArgument( (byte)0x4), + IOError( (byte)0x5), + MergeInProgress( (byte)0x6), + Incomplete( (byte)0x7), + ShutdownInProgress( (byte)0x8), + TimedOut( (byte)0x9), + Aborted( (byte)0xA), + Busy( (byte)0xB), + Expired( (byte)0xC), + TryAgain( (byte)0xD); + + private final byte value; + + Code(final byte value) { + this.value = value; + } + + public static Code getCode(final byte value) { + for (final Code code : Code.values()) { + if (code.value == value){ + return code; + } + } + throw new IllegalArgumentException( + "Illegal value provided for Code."); + } + } + + public enum SubCode { + None( (byte)0x0), + MutexTimeout( (byte)0x1), + LockTimeout( (byte)0x2), + LockLimit( (byte)0x3), + MaxSubCode( (byte)0x7E); + + private final byte value; + + SubCode(final byte value) { + this.value = value; + } + + public static SubCode getSubCode(final byte value) { + for (final SubCode subCode : SubCode.values()) { + if (subCode.value == value){ + return subCode; + } + } + throw new IllegalArgumentException( + "Illegal value provided for SubCode."); + } + } +} diff --git a/java/src/main/java/org/rocksdb/StringAppendOperator.java b/java/src/main/java/org/rocksdb/StringAppendOperator.java new file mode 100644 index 00000000000..85c36adc7c1 --- /dev/null +++ b/java/src/main/java/org/rocksdb/StringAppendOperator.java @@ -0,0 +1,19 @@ +// Copyright (c) 2014, Vlad Balan (vlad.gm@gmail.com). All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * StringAppendOperator is a merge operator that concatenates + * two strings. + */ +public class StringAppendOperator extends MergeOperator { + public StringAppendOperator() { + super(newSharedStringAppendOperator()); + } + + private native static long newSharedStringAppendOperator(); + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/org/rocksdb/TableFormatConfig.java b/java/src/main/java/org/rocksdb/TableFormatConfig.java similarity index 51% rename from java/org/rocksdb/TableFormatConfig.java rename to java/src/main/java/org/rocksdb/TableFormatConfig.java index e5c63411fd5..dbe524c4226 100644 --- a/java/org/rocksdb/TableFormatConfig.java +++ b/java/src/main/java/org/rocksdb/TableFormatConfig.java @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). package org.rocksdb; /** @@ -12,9 +12,11 @@ */ public abstract class TableFormatConfig { /** - * This function should only be called by Options.setTableFormatConfig(), + *

    This function should only be called by Options.setTableFormatConfig(), * which will create a c++ shared-pointer to the c++ TableFactory - * that associated with the Java TableFormatConfig. + * that associated with the Java TableFormatConfig.

    + * + * @return native handle address to native table instance. */ abstract protected long newTableFactoryHandle(); } diff --git a/java/src/main/java/org/rocksdb/TickerType.java b/java/src/main/java/org/rocksdb/TickerType.java new file mode 100644 index 00000000000..948079c75a9 --- /dev/null +++ b/java/src/main/java/org/rocksdb/TickerType.java @@ -0,0 +1,480 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +public enum TickerType { + + /** + * total block cache misses + * + * REQUIRES: BLOCK_CACHE_MISS == BLOCK_CACHE_INDEX_MISS + + * BLOCK_CACHE_FILTER_MISS + + * BLOCK_CACHE_DATA_MISS; + */ + BLOCK_CACHE_MISS((byte) 0x0), + + /** + * total block cache hit + * + * REQUIRES: BLOCK_CACHE_HIT == BLOCK_CACHE_INDEX_HIT + + * BLOCK_CACHE_FILTER_HIT + + * BLOCK_CACHE_DATA_HIT; + */ + BLOCK_CACHE_HIT((byte) 0x1), + + BLOCK_CACHE_ADD((byte) 0x2), + + /** + * # of failures when adding blocks to block cache. + */ + BLOCK_CACHE_ADD_FAILURES((byte) 0x3), + + /** + * # of times cache miss when accessing index block from block cache. + */ + BLOCK_CACHE_INDEX_MISS((byte) 0x4), + + /** + * # of times cache hit when accessing index block from block cache. + */ + BLOCK_CACHE_INDEX_HIT((byte) 0x5), + + /** + * # of index blocks added to block cache. + */ + BLOCK_CACHE_INDEX_ADD((byte) 0x6), + + /** + * # of bytes of index blocks inserted into cache + */ + BLOCK_CACHE_INDEX_BYTES_INSERT((byte) 0x7), + + /** + * # of bytes of index block erased from cache + */ + BLOCK_CACHE_INDEX_BYTES_EVICT((byte) 0x8), + + /** + * # of times cache miss when accessing filter block from block cache. + */ + BLOCK_CACHE_FILTER_MISS((byte) 0x9), + + /** + * # of times cache hit when accessing filter block from block cache. + */ + BLOCK_CACHE_FILTER_HIT((byte) 0xA), + + /** + * # of filter blocks added to block cache. + */ + BLOCK_CACHE_FILTER_ADD((byte) 0xB), + + /** + * # of bytes of bloom filter blocks inserted into cache + */ + BLOCK_CACHE_FILTER_BYTES_INSERT((byte) 0xC), + + /** + * # of bytes of bloom filter block erased from cache + */ + BLOCK_CACHE_FILTER_BYTES_EVICT((byte) 0xD), + + /** + * # of times cache miss when accessing data block from block cache. + */ + BLOCK_CACHE_DATA_MISS((byte) 0xE), + + /** + * # of times cache hit when accessing data block from block cache. + */ + BLOCK_CACHE_DATA_HIT((byte) 0xF), + + /** + * # of data blocks added to block cache. + */ + BLOCK_CACHE_DATA_ADD((byte) 0x10), + + /** + * # of bytes of data blocks inserted into cache + */ + BLOCK_CACHE_DATA_BYTES_INSERT((byte) 0x11), + + /** + * # of bytes read from cache. + */ + BLOCK_CACHE_BYTES_READ((byte) 0x12), + + /** + * # of bytes written into cache. + */ + BLOCK_CACHE_BYTES_WRITE((byte) 0x13), + + /** + * # of times bloom filter has avoided file reads. + */ + BLOOM_FILTER_USEFUL((byte) 0x14), + + /** + * # persistent cache hit + */ + PERSISTENT_CACHE_HIT((byte) 0x15), + + /** + * # persistent cache miss + */ + PERSISTENT_CACHE_MISS((byte) 0x16), + + /** + * # total simulation block cache hits + */ + SIM_BLOCK_CACHE_HIT((byte) 0x17), + + /** + * # total simulation block cache misses + */ + SIM_BLOCK_CACHE_MISS((byte) 0x18), + + /** + * # of memtable hits. + */ + MEMTABLE_HIT((byte) 0x19), + + /** + * # of memtable misses. + */ + MEMTABLE_MISS((byte) 0x1A), + + /** + * # of Get() queries served by L0 + */ + GET_HIT_L0((byte) 0x1B), + + /** + * # of Get() queries served by L1 + */ + GET_HIT_L1((byte) 0x1C), + + /** + * # of Get() queries served by L2 and up + */ + GET_HIT_L2_AND_UP((byte) 0x1D), + + /** + * COMPACTION_KEY_DROP_* count the reasons for key drop during compaction + * There are 4 reasons currently. + */ + + /** + * key was written with a newer value. + */ + COMPACTION_KEY_DROP_NEWER_ENTRY((byte) 0x1E), + + /** + * Also includes keys dropped for range del. + * The key is obsolete. + */ + COMPACTION_KEY_DROP_OBSOLETE((byte) 0x1F), + + /** + * key was covered by a range tombstone. + */ + COMPACTION_KEY_DROP_RANGE_DEL((byte) 0x20), + + /** + * User compaction function has dropped the key. + */ + COMPACTION_KEY_DROP_USER((byte) 0x21), + + /** + * all keys in range were deleted. + */ + COMPACTION_RANGE_DEL_DROP_OBSOLETE((byte) 0x22), + + /** + * Number of keys written to the database via the Put and Write call's. + */ + NUMBER_KEYS_WRITTEN((byte) 0x23), + + /** + * Number of Keys read. + */ + NUMBER_KEYS_READ((byte) 0x24), + + /** + * Number keys updated, if inplace update is enabled + */ + NUMBER_KEYS_UPDATED((byte) 0x25), + + /** + * The number of uncompressed bytes issued by DB::Put(), DB::Delete(),\ + * DB::Merge(), and DB::Write(). + */ + BYTES_WRITTEN((byte) 0x26), + + /** + * The number of uncompressed bytes read from DB::Get(). It could be + * either from memtables, cache, or table files. + * + * For the number of logical bytes read from DB::MultiGet(), + * please use {@link #NUMBER_MULTIGET_BYTES_READ}. + */ + BYTES_READ((byte) 0x27), + + /** + * The number of calls to seek. + */ + NUMBER_DB_SEEK((byte) 0x28), + + /** + * The number of calls to next. + */ + NUMBER_DB_NEXT((byte) 0x29), + + /** + * The number of calls to prev. + */ + NUMBER_DB_PREV((byte) 0x2A), + + /** + * The number of calls to seek that returned data. + */ + NUMBER_DB_SEEK_FOUND((byte) 0x2B), + + /** + * The number of calls to next that returned data. + */ + NUMBER_DB_NEXT_FOUND((byte) 0x2C), + + /** + * The number of calls to prev that returned data. + */ + NUMBER_DB_PREV_FOUND((byte) 0x2D), + + /** + * The number of uncompressed bytes read from an iterator. + * Includes size of key and value. + */ + ITER_BYTES_READ((byte) 0x2E), + + NO_FILE_CLOSES((byte) 0x2F), + + NO_FILE_OPENS((byte) 0x30), + + NO_FILE_ERRORS((byte) 0x31), + + /** + * Time system had to wait to do LO-L1 compactions. + * + * @deprecated + */ + @Deprecated + STALL_L0_SLOWDOWN_MICROS((byte) 0x32), + + /** + * Time system had to wait to move memtable to L1. + * + * @deprecated + */ + @Deprecated + STALL_MEMTABLE_COMPACTION_MICROS((byte) 0x33), + + /** + * write throttle because of too many files in L0. + * + * @deprecated + */ + @Deprecated + STALL_L0_NUM_FILES_MICROS((byte) 0x34), + + /** + * Writer has to wait for compaction or flush to finish. + */ + STALL_MICROS((byte) 0x35), + + /** + * The wait time for db mutex. + * + * Disabled by default. To enable it set stats level to {@link StatsLevel#ALL} + */ + DB_MUTEX_WAIT_MICROS((byte) 0x36), + + RATE_LIMIT_DELAY_MILLIS((byte) 0x37), + + /** + * Number of iterators currently open. + */ + NO_ITERATORS((byte) 0x38), + + /** + * Number of MultiGet calls. + */ + NUMBER_MULTIGET_CALLS((byte) 0x39), + + /** + * Number of MultiGet keys read. + */ + NUMBER_MULTIGET_KEYS_READ((byte) 0x3A), + + /** + * Number of MultiGet bytes read. + */ + NUMBER_MULTIGET_BYTES_READ((byte) 0x3B), + + /** + * Number of deletes records that were not required to be + * written to storage because key does not exist. + */ + NUMBER_FILTERED_DELETES((byte) 0x3C), + NUMBER_MERGE_FAILURES((byte) 0x3D), + + /** + * Number of times bloom was checked before creating iterator on a + * file, and the number of times the check was useful in avoiding + * iterator creation (and thus likely IOPs). + */ + BLOOM_FILTER_PREFIX_CHECKED((byte) 0x3E), + BLOOM_FILTER_PREFIX_USEFUL((byte) 0x3F), + + /** + * Number of times we had to reseek inside an iteration to skip + * over large number of keys with same userkey. + */ + NUMBER_OF_RESEEKS_IN_ITERATION((byte) 0x40), + + /** + * Record the number of calls to {@link RocksDB#getUpdatesSince(long)}. Useful to keep track of + * transaction log iterator refreshes. + */ + GET_UPDATES_SINCE_CALLS((byte) 0x41), + + /** + * Miss in the compressed block cache. + */ + BLOCK_CACHE_COMPRESSED_MISS((byte) 0x42), + + /** + * Hit in the compressed block cache. + */ + BLOCK_CACHE_COMPRESSED_HIT((byte) 0x43), + + /** + * Number of blocks added to compressed block cache. + */ + BLOCK_CACHE_COMPRESSED_ADD((byte) 0x44), + + /** + * Number of failures when adding blocks to compressed block cache. + */ + BLOCK_CACHE_COMPRESSED_ADD_FAILURES((byte) 0x45), + + /** + * Number of times WAL sync is done. + */ + WAL_FILE_SYNCED((byte) 0x46), + + /** + * Number of bytes written to WAL. + */ + WAL_FILE_BYTES((byte) 0x47), + + /** + * Writes can be processed by requesting thread or by the thread at the + * head of the writers queue. + */ + WRITE_DONE_BY_SELF((byte) 0x48), + + /** + * Equivalent to writes done for others. + */ + WRITE_DONE_BY_OTHER((byte) 0x49), + + /** + * Number of writes ending up with timed-out. + */ + WRITE_TIMEDOUT((byte) 0x4A), + + /** + * Number of Write calls that request WAL. + */ + WRITE_WITH_WAL((byte) 0x4B), + + /** + * Bytes read during compaction. + */ + COMPACT_READ_BYTES((byte) 0x4C), + + /** + * Bytes written during compaction. + */ + COMPACT_WRITE_BYTES((byte) 0x4D), + + /** + * Bytes written during flush. + */ + FLUSH_WRITE_BYTES((byte) 0x4E), + + /** + * Number of table's properties loaded directly from file, without creating + * table reader object. + */ + NUMBER_DIRECT_LOAD_TABLE_PROPERTIES((byte) 0x4F), + NUMBER_SUPERVERSION_ACQUIRES((byte) 0x50), + NUMBER_SUPERVERSION_RELEASES((byte) 0x51), + NUMBER_SUPERVERSION_CLEANUPS((byte) 0x52), + + /** + * # of compressions/decompressions executed + */ + NUMBER_BLOCK_COMPRESSED((byte) 0x53), + NUMBER_BLOCK_DECOMPRESSED((byte) 0x54), + + NUMBER_BLOCK_NOT_COMPRESSED((byte) 0x55), + MERGE_OPERATION_TOTAL_TIME((byte) 0x56), + FILTER_OPERATION_TOTAL_TIME((byte) 0x57), + + /** + * Row cache. + */ + ROW_CACHE_HIT((byte) 0x58), + ROW_CACHE_MISS((byte) 0x59), + + /** + * Read amplification statistics. + * + * Read amplification can be calculated using this formula + * (READ_AMP_TOTAL_READ_BYTES / READ_AMP_ESTIMATE_USEFUL_BYTES) + * + * REQUIRES: ReadOptions::read_amp_bytes_per_bit to be enabled + */ + + /** + * Estimate of total bytes actually used. + */ + READ_AMP_ESTIMATE_USEFUL_BYTES((byte) 0x5A), + + /** + * Total size of loaded data blocks. + */ + READ_AMP_TOTAL_READ_BYTES((byte) 0x5B), + + /** + * Number of refill intervals where rate limiter's bytes are fully consumed. + */ + NUMBER_RATE_LIMITER_DRAINS((byte) 0x5C), + + TICKER_ENUM_MAX((byte) 0x5D); + + + private final byte value; + + TickerType(final byte value) { + this.value = value; + } + + public byte getValue() { + return value; + } +} diff --git a/java/src/main/java/org/rocksdb/TransactionLogIterator.java b/java/src/main/java/org/rocksdb/TransactionLogIterator.java new file mode 100644 index 00000000000..b6bfc495bcd --- /dev/null +++ b/java/src/main/java/org/rocksdb/TransactionLogIterator.java @@ -0,0 +1,111 @@ +package org.rocksdb; + +/** + *

    A TransactionLogIterator is used to iterate over the transactions in a db. + * One run of the iterator is continuous, i.e. the iterator will stop at the + * beginning of any gap in sequences.

    + */ +public class TransactionLogIterator extends RocksObject { + + /** + *

    An iterator is either positioned at a WriteBatch + * or not valid. This method returns true if the iterator + * is valid. Can read data from a valid iterator.

    + * + * @return true if iterator position is valid. + */ + public boolean isValid() { + return isValid(nativeHandle_); + } + + /** + *

    Moves the iterator to the next WriteBatch. + * REQUIRES: Valid() to be true.

    + */ + public void next() { + next(nativeHandle_); + } + + /** + *

    Throws RocksDBException if something went wrong.

    + * + * @throws org.rocksdb.RocksDBException if something went + * wrong in the underlying C++ code. + */ + public void status() throws RocksDBException { + status(nativeHandle_); + } + + /** + *

    If iterator position is valid, return the current + * write_batch and the sequence number of the earliest + * transaction contained in the batch.

    + * + *

    ONLY use if Valid() is true and status() is OK.

    + * + * @return {@link org.rocksdb.TransactionLogIterator.BatchResult} + * instance. + */ + public BatchResult getBatch() { + assert(isValid()); + return getBatch(nativeHandle_); + } + + /** + *

    TransactionLogIterator constructor.

    + * + * @param nativeHandle address to native address. + */ + TransactionLogIterator(final long nativeHandle) { + super(nativeHandle); + } + + /** + *

    BatchResult represents a data structure returned + * by a TransactionLogIterator containing a sequence + * number and a {@link WriteBatch} instance.

    + */ + public static final class BatchResult { + /** + *

    Constructor of BatchResult class.

    + * + * @param sequenceNumber related to this BatchResult instance. + * @param nativeHandle to {@link org.rocksdb.WriteBatch} + * native instance. + */ + public BatchResult(final long sequenceNumber, + final long nativeHandle) { + sequenceNumber_ = sequenceNumber; + writeBatch_ = new WriteBatch(nativeHandle, true); + } + + /** + *

    Return sequence number related to this BatchResult.

    + * + * @return Sequence number. + */ + public long sequenceNumber() { + return sequenceNumber_; + } + + /** + *

    Return contained {@link org.rocksdb.WriteBatch} + * instance

    + * + * @return {@link org.rocksdb.WriteBatch} instance. + */ + public WriteBatch writeBatch() { + return writeBatch_; + } + + private final long sequenceNumber_; + private final WriteBatch writeBatch_; + } + + @Override protected final native void disposeInternal(final long handle); + private native boolean isValid(long handle); + private native void next(long handle); + private native void status(long handle) + throws RocksDBException; + private native BatchResult getBatch(long handle); +} diff --git a/java/src/main/java/org/rocksdb/TtlDB.java b/java/src/main/java/org/rocksdb/TtlDB.java new file mode 100644 index 00000000000..740f51268ef --- /dev/null +++ b/java/src/main/java/org/rocksdb/TtlDB.java @@ -0,0 +1,211 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.List; + +/** + * Database with TTL support. + * + *

    Use case

    + *

    This API should be used to open the db when key-values inserted are + * meant to be removed from the db in a non-strict 'ttl' amount of time + * Therefore, this guarantees that key-values inserted will remain in the + * db for >= ttl amount of time and the db will make efforts to remove the + * key-values as soon as possible after ttl seconds of their insertion. + *

    + * + *

    Behaviour

    + *

    TTL is accepted in seconds + * (int32_t)Timestamp(creation) is suffixed to values in Put internally + * Expired TTL values deleted in compaction only:(Timestamp+ttl<time_now) + * Get/Iterator may return expired entries(compaction not run on them yet) + * Different TTL may be used during different Opens + *

    + * + *

    Example

    + *
      + *
    • Open1 at t=0 with ttl=4 and insert k1,k2, close at t=2
      • + *
    • Open2 at t=3 with ttl=5. Now k1,k2 should be deleted at t>=5
      • + *
    + * + *

    + * read_only=true opens in the usual read-only mode. Compactions will not be + * triggered(neither manual nor automatic), so no expired entries removed + *

    + * + *

    Constraints

    + *

    Not specifying/passing or non-positive TTL behaves + * like TTL = infinity

    + * + *

    !!!WARNING!!!

    + *

    Calling DB::Open directly to re-open a db created by this API will get + * corrupt values(timestamp suffixed) and no ttl effect will be there + * during the second Open, so use this API consistently to open the db + * Be careful when passing ttl with a small positive value because the + * whole database may be deleted in a small amount of time.

    + */ +public class TtlDB extends RocksDB { + + /** + *

    Opens a TtlDB.

    + * + *

    Database is opened in read-write mode without default TTL.

    + * + * @param options {@link org.rocksdb.Options} instance. + * @param db_path path to database. + * + * @return TtlDB instance. + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public static TtlDB open(final Options options, final String db_path) + throws RocksDBException { + return open(options, db_path, 0, false); + } + + /** + *

    Opens a TtlDB.

    + * + * @param options {@link org.rocksdb.Options} instance. + * @param db_path path to database. + * @param ttl time to live for new entries. + * @param readOnly boolean value indicating if database if db is + * opened read-only. + * + * @return TtlDB instance. + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + public static TtlDB open(final Options options, final String db_path, + final int ttl, final boolean readOnly) throws RocksDBException { + return new TtlDB(open(options.nativeHandle_, db_path, ttl, readOnly)); + } + + /** + *

    Opens a TtlDB.

    + * + * @param options {@link org.rocksdb.Options} instance. + * @param db_path path to database. + * @param columnFamilyDescriptors list of column family descriptors + * @param columnFamilyHandles will be filled with ColumnFamilyHandle instances + * on open. + * @param ttlValues time to live values per column family handle + * @param readOnly boolean value indicating if database if db is + * opened read-only. + * + * @return TtlDB instance. + * + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + * @throws java.lang.IllegalArgumentException when there is not a ttl value + * per given column family handle. + */ + public static TtlDB open(final DBOptions options, final String db_path, + final List columnFamilyDescriptors, + final List columnFamilyHandles, + final List ttlValues, final boolean readOnly) + throws RocksDBException { + if (columnFamilyDescriptors.size() != ttlValues.size()) { + throw new IllegalArgumentException("There must be a ttl value per column" + + "family handle."); + } + + final byte[][] cfNames = new byte[columnFamilyDescriptors.size()][]; + final long[] cfOptionHandles = new long[columnFamilyDescriptors.size()]; + for (int i = 0; i < columnFamilyDescriptors.size(); i++) { + final ColumnFamilyDescriptor cfDescriptor = + columnFamilyDescriptors.get(i); + cfNames[i] = cfDescriptor.columnFamilyName(); + cfOptionHandles[i] = cfDescriptor.columnFamilyOptions().nativeHandle_; + } + + final int ttlVals[] = new int[ttlValues.size()]; + for(int i = 0; i < ttlValues.size(); i++) { + ttlVals[i] = ttlValues.get(i); + } + final long[] handles = openCF(options.nativeHandle_, db_path, + cfNames, cfOptionHandles, ttlVals, readOnly); + + final TtlDB ttlDB = new TtlDB(handles[0]); + for (int i = 1; i < handles.length; i++) { + columnFamilyHandles.add(new ColumnFamilyHandle(ttlDB, handles[i])); + } + return ttlDB; + } + + /** + *

    Creates a new ttl based column family with a name defined + * in given ColumnFamilyDescriptor and allocates a + * ColumnFamilyHandle within an internal structure.

    + * + *

    The ColumnFamilyHandle is automatically disposed with DB + * disposal.

    + * + * @param columnFamilyDescriptor column family to be created. + * @param ttl TTL to set for this column family. + * + * @return {@link org.rocksdb.ColumnFamilyHandle} instance. + * + * @throws RocksDBException thrown if error happens in underlying + * native library. + */ + public ColumnFamilyHandle createColumnFamilyWithTtl( + final ColumnFamilyDescriptor columnFamilyDescriptor, + final int ttl) throws RocksDBException { + return new ColumnFamilyHandle(this, + createColumnFamilyWithTtl(nativeHandle_, + columnFamilyDescriptor.columnFamilyName(), + columnFamilyDescriptor.columnFamilyOptions().nativeHandle_, ttl)); + } + + /** + *

    Close the TtlDB instance and release resource.

    + * + *

    Internally, TtlDB owns the {@code rocksdb::DB} pointer + * to its associated {@link org.rocksdb.RocksDB}. The release + * of that RocksDB pointer is handled in the destructor of the + * c++ {@code rocksdb::TtlDB} and should be transparent to + * Java developers.

    + */ + @Override + public void close() { + super.close(); + } + + /** + *

    A protected constructor that will be used in the static + * factory method + * {@link #open(Options, String, int, boolean)} + * and + * {@link #open(DBOptions, String, java.util.List, java.util.List, + * java.util.List, boolean)}. + *

    + * + * @param nativeHandle The native handle of the C++ TtlDB object + */ + protected TtlDB(final long nativeHandle) { + super(nativeHandle); + } + + @Override protected void finalize() throws Throwable { + close(); //TODO(AR) revisit here when implementing AutoCloseable + super.finalize(); + } + + private native static long open(final long optionsHandle, + final String db_path, final int ttl, final boolean readOnly) + throws RocksDBException; + private native static long[] openCF(final long optionsHandle, + final String db_path, final byte[][] columnFamilyNames, + final long[] columnFamilyOptions, final int[] ttlValues, + final boolean readOnly) throws RocksDBException; + private native long createColumnFamilyWithTtl(final long handle, + final byte[] columnFamilyName, final long columnFamilyOptions, int ttl) + throws RocksDBException; +} diff --git a/java/org/rocksdb/VectorMemTableConfig.java b/java/src/main/java/org/rocksdb/VectorMemTableConfig.java similarity index 86% rename from java/org/rocksdb/VectorMemTableConfig.java rename to java/src/main/java/org/rocksdb/VectorMemTableConfig.java index b7a413f195c..378340248f5 100644 --- a/java/org/rocksdb/VectorMemTableConfig.java +++ b/java/src/main/java/org/rocksdb/VectorMemTableConfig.java @@ -5,6 +5,10 @@ */ public class VectorMemTableConfig extends MemTableConfig { public static final int DEFAULT_RESERVED_SIZE = 0; + + /** + * VectorMemTableConfig constructor + */ public VectorMemTableConfig() { reservedSize_ = DEFAULT_RESERVED_SIZE; } @@ -16,7 +20,7 @@ public VectorMemTableConfig() { * @param size the initial size of the vector. * @return the reference to the current config. */ - public VectorMemTableConfig setReservedSize(int size) { + public VectorMemTableConfig setReservedSize(final int size) { reservedSize_ = size; return this; } @@ -35,6 +39,7 @@ public int reservedSize() { return newMemTableFactoryHandle(reservedSize_); } - private native long newMemTableFactoryHandle(long reservedSize); + private native long newMemTableFactoryHandle(long reservedSize) + throws IllegalArgumentException; private int reservedSize_; } diff --git a/java/src/main/java/org/rocksdb/WALRecoveryMode.java b/java/src/main/java/org/rocksdb/WALRecoveryMode.java new file mode 100644 index 00000000000..d3fc47b631f --- /dev/null +++ b/java/src/main/java/org/rocksdb/WALRecoveryMode.java @@ -0,0 +1,83 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * The WAL Recover Mode + */ +public enum WALRecoveryMode { + + /** + * Original levelDB recovery + * + * We tolerate incomplete record in trailing data on all logs + * Use case : This is legacy behavior (default) + */ + TolerateCorruptedTailRecords((byte)0x00), + + /** + * Recover from clean shutdown + * + * We don't expect to find any corruption in the WAL + * Use case : This is ideal for unit tests and rare applications that + * can require high consistency guarantee + */ + AbsoluteConsistency((byte)0x01), + + /** + * Recover to point-in-time consistency + * We stop the WAL playback on discovering WAL inconsistency + * Use case : Ideal for systems that have disk controller cache like + * hard disk, SSD without super capacitor that store related data + */ + PointInTimeRecovery((byte)0x02), + + /** + * Recovery after a disaster + * We ignore any corruption in the WAL and try to salvage as much data as + * possible + * Use case : Ideal for last ditch effort to recover data or systems that + * operate with low grade unrelated data + */ + SkipAnyCorruptedRecords((byte)0x03); + + private byte value; + + WALRecoveryMode(final byte value) { + this.value = value; + } + + /** + *

    Returns the byte value of the enumerations value.

    + * + * @return byte representation + */ + public byte getValue() { + return value; + } + + /** + *

    Get the WALRecoveryMode enumeration value by + * passing the byte identifier to this method.

    + * + * @param byteIdentifier of WALRecoveryMode. + * + * @return CompressionType instance. + * + * @throws IllegalArgumentException If WALRecoveryMode cannot be found for the + * provided byteIdentifier + */ + public static WALRecoveryMode getWALRecoveryMode(final byte byteIdentifier) { + for (final WALRecoveryMode walRecoveryMode : WALRecoveryMode.values()) { + if (walRecoveryMode.getValue() == byteIdentifier) { + return walRecoveryMode; + } + } + + throw new IllegalArgumentException( + "Illegal value provided for WALRecoveryMode."); + } +} diff --git a/java/src/main/java/org/rocksdb/WBWIRocksIterator.java b/java/src/main/java/org/rocksdb/WBWIRocksIterator.java new file mode 100644 index 00000000000..d45da2b3a1f --- /dev/null +++ b/java/src/main/java/org/rocksdb/WBWIRocksIterator.java @@ -0,0 +1,184 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +public class WBWIRocksIterator + extends AbstractRocksIterator { + private final WriteEntry entry = new WriteEntry(); + + protected WBWIRocksIterator(final WriteBatchWithIndex wbwi, + final long nativeHandle) { + super(wbwi, nativeHandle); + } + + /** + * Get the current entry + * + * The WriteEntry is only valid + * until the iterator is repositioned. + * If you want to keep the WriteEntry across iterator + * movements, you must make a copy of its data! + * + * Note - This method is not thread-safe with respect to the WriteEntry + * as it performs a non-atomic update across the fields of the WriteEntry + * + * @return The WriteEntry of the current entry + */ + public WriteEntry entry() { + assert(isOwningHandle()); + final long ptrs[] = entry1(nativeHandle_); + + entry.type = WriteType.fromId((byte)ptrs[0]); + entry.key.resetNativeHandle(ptrs[1], ptrs[1] != 0); + entry.value.resetNativeHandle(ptrs[2], ptrs[2] != 0); + + return entry; + } + + @Override protected final native void disposeInternal(final long handle); + @Override final native boolean isValid0(long handle); + @Override final native void seekToFirst0(long handle); + @Override final native void seekToLast0(long handle); + @Override final native void next0(long handle); + @Override final native void prev0(long handle); + @Override final native void seek0(long handle, byte[] target, int targetLen); + @Override final native void status0(long handle) throws RocksDBException; + + private native long[] entry1(final long handle); + + /** + * Enumeration of the Write operation + * that created the record in the Write Batch + */ + public enum WriteType { + PUT((byte)0x1), + MERGE((byte)0x2), + DELETE((byte)0x4), + LOG((byte)0x8); + + final byte id; + WriteType(final byte id) { + this.id = id; + } + + public static WriteType fromId(final byte id) { + for(final WriteType wt : WriteType.values()) { + if(id == wt.id) { + return wt; + } + } + throw new IllegalArgumentException("No WriteType with id=" + id); + } + } + + @Override + public void close() { + entry.close(); + super.close(); + } + + /** + * Represents an entry returned by + * {@link org.rocksdb.WBWIRocksIterator#entry()} + * + * It is worth noting that a WriteEntry with + * the type {@link org.rocksdb.WBWIRocksIterator.WriteType#DELETE} + * or {@link org.rocksdb.WBWIRocksIterator.WriteType#LOG} + * will not have a value. + */ + public static class WriteEntry implements AutoCloseable { + WriteType type = null; + final DirectSlice key; + final DirectSlice value; + + /** + * Intentionally private as this + * should only be instantiated in + * this manner by the outer WBWIRocksIterator + * class; The class members are then modified + * by calling {@link org.rocksdb.WBWIRocksIterator#entry()} + */ + private WriteEntry() { + key = new DirectSlice(); + value = new DirectSlice(); + } + + public WriteEntry(final WriteType type, final DirectSlice key, + final DirectSlice value) { + this.type = type; + this.key = key; + this.value = value; + } + + /** + * Returns the type of the Write Entry + * + * @return the WriteType of the WriteEntry + */ + public WriteType getType() { + return type; + } + + /** + * Returns the key of the Write Entry + * + * @return The slice containing the key + * of the WriteEntry + */ + public DirectSlice getKey() { + return key; + } + + /** + * Returns the value of the Write Entry + * + * @return The slice containing the value of + * the WriteEntry or null if the WriteEntry has + * no value + */ + public DirectSlice getValue() { + if(!value.isOwningHandle()) { + return null; //TODO(AR) migrate to JDK8 java.util.Optional#empty() + } else { + return value; + } + } + + /** + * Generates a hash code for the Write Entry. NOTE: The hash code is based + * on the string representation of the key, so it may not work correctly + * with exotic custom comparators. + * + * @return The hash code for the Write Entry + */ + @Override + public int hashCode() { + return (key == null) ? 0 : key.hashCode(); + } + + @Override + public boolean equals(final Object other) { + if(other == null) { + return false; + } else if (this == other) { + return true; + } else if(other instanceof WriteEntry) { + final WriteEntry otherWriteEntry = (WriteEntry)other; + return type.equals(otherWriteEntry.type) + && key.equals(otherWriteEntry.key) + && value.equals(otherWriteEntry.value); + } else { + return false; + } + } + + @Override + public void close() { + value.close(); + key.close(); + } + } +} diff --git a/java/src/main/java/org/rocksdb/WriteBatch.java b/java/src/main/java/org/rocksdb/WriteBatch.java new file mode 100644 index 00000000000..272e9b4cdf0 --- /dev/null +++ b/java/src/main/java/org/rocksdb/WriteBatch.java @@ -0,0 +1,153 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * WriteBatch holds a collection of updates to apply atomically to a DB. + * + * The updates are applied in the order in which they are added + * to the WriteBatch. For example, the value of "key" will be "v3" + * after the following batch is written: + * + * batch.put("key", "v1"); + * batch.remove("key"); + * batch.put("key", "v2"); + * batch.put("key", "v3"); + * + * Multiple threads can invoke const methods on a WriteBatch without + * external synchronization, but if any of the threads may call a + * non-const method, all threads accessing the same WriteBatch must use + * external synchronization. + */ +public class WriteBatch extends AbstractWriteBatch { + /** + * Constructs a WriteBatch instance. + */ + public WriteBatch() { + this(0); + } + + /** + * Constructs a WriteBatch instance with a given size. + * + * @param reserved_bytes reserved size for WriteBatch + */ + public WriteBatch(final int reserved_bytes) { + super(newWriteBatch(reserved_bytes)); + } + + /** + * Support for iterating over the contents of a batch. + * + * @param handler A handler that is called back for each + * update present in the batch + * + * @throws RocksDBException If we cannot iterate over the batch + */ + public void iterate(final Handler handler) throws RocksDBException { + iterate(nativeHandle_, handler.nativeHandle_); + } + + /** + *

    Private WriteBatch constructor which is used to construct + * WriteBatch instances from C++ side. As the reference to this + * object is also managed from C++ side the handle will be disowned.

    + * + * @param nativeHandle address of native instance. + */ + WriteBatch(final long nativeHandle) { + this(nativeHandle, false); + } + + /** + *

    Private WriteBatch constructor which is used to construct + * WriteBatch instances.

    + * + * @param nativeHandle address of native instance. + * @param owningNativeHandle whether to own this reference from the C++ side or not + */ + WriteBatch(final long nativeHandle, final boolean owningNativeHandle) { + super(nativeHandle); + if(!owningNativeHandle) + disOwnNativeHandle(); + } + + @Override protected final native void disposeInternal(final long handle); + @Override final native int count0(final long handle); + @Override final native void put(final long handle, final byte[] key, + final int keyLen, final byte[] value, final int valueLen); + @Override final native void put(final long handle, final byte[] key, + final int keyLen, final byte[] value, final int valueLen, + final long cfHandle); + @Override final native void merge(final long handle, final byte[] key, + final int keyLen, final byte[] value, final int valueLen); + @Override final native void merge(final long handle, final byte[] key, + final int keyLen, final byte[] value, final int valueLen, + final long cfHandle); + @Override final native void remove(final long handle, final byte[] key, + final int keyLen); + @Override final native void remove(final long handle, final byte[] key, + final int keyLen, final long cfHandle); + @Override + final native void deleteRange(final long handle, final byte[] beginKey, final int beginKeyLen, + final byte[] endKey, final int endKeyLen); + @Override + final native void deleteRange(final long handle, final byte[] beginKey, final int beginKeyLen, + final byte[] endKey, final int endKeyLen, final long cfHandle); + @Override final native void putLogData(final long handle, + final byte[] blob, final int blobLen); + @Override final native void clear0(final long handle); + @Override final native void setSavePoint0(final long handle); + @Override final native void rollbackToSavePoint0(final long handle); + + private native static long newWriteBatch(final int reserved_bytes); + private native void iterate(final long handle, final long handlerHandle) + throws RocksDBException; + + + /** + * Handler callback for iterating over the contents of a batch. + */ + public static abstract class Handler + extends AbstractImmutableNativeReference { + private final long nativeHandle_; + public Handler() { + super(true); + this.nativeHandle_ = createNewHandler0(); + } + + public abstract void put(byte[] key, byte[] value); + public abstract void merge(byte[] key, byte[] value); + public abstract void delete(byte[] key); + public abstract void deleteRange(byte[] beginKey, byte[] endKey); + public abstract void logData(byte[] blob); + + /** + * shouldContinue is called by the underlying iterator + * WriteBatch::Iterate. If it returns false, + * iteration is halted. Otherwise, it continues + * iterating. The default implementation always + * returns true. + * + * @return boolean value indicating if the + * iteration is halted. + */ + public boolean shouldContinue() { + return true; + } + + /** + * Deletes underlying C++ handler pointer. + */ + @Override + protected void disposeInternal() { + disposeInternal(nativeHandle_); + } + + private native long createNewHandler0(); + private native void disposeInternal(final long handle); + } +} diff --git a/java/src/main/java/org/rocksdb/WriteBatchInterface.java b/java/src/main/java/org/rocksdb/WriteBatchInterface.java new file mode 100644 index 00000000000..cd024ad58d4 --- /dev/null +++ b/java/src/main/java/org/rocksdb/WriteBatchInterface.java @@ -0,0 +1,146 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + *

    Defines the interface for a Write Batch which + * holds a collection of updates to apply atomically to a DB.

    + */ +public interface WriteBatchInterface { + + /** + * Returns the number of updates in the batch. + * + * @return number of items in WriteBatch + */ + int count(); + + /** + *

    Store the mapping "key->value" in the database.

    + * + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + */ + void put(byte[] key, byte[] value); + + /** + *

    Store the mapping "key->value" within given column + * family.

    + * + * @param columnFamilyHandle {@link org.rocksdb.ColumnFamilyHandle} + * instance + * @param key the specified key to be inserted. + * @param value the value associated with the specified key. + */ + void put(ColumnFamilyHandle columnFamilyHandle, + byte[] key, byte[] value); + + /** + *

    Merge "value" with the existing value of "key" in the database. + * "key->merge(existing, value)"

    + * + * @param key the specified key to be merged. + * @param value the value to be merged with the current value for + * the specified key. + */ + void merge(byte[] key, byte[] value); + + /** + *

    Merge "value" with the existing value of "key" in given column family. + * "key->merge(existing, value)"

    + * + * @param columnFamilyHandle {@link ColumnFamilyHandle} instance + * @param key the specified key to be merged. + * @param value the value to be merged with the current value for + * the specified key. + */ + void merge(ColumnFamilyHandle columnFamilyHandle, + byte[] key, byte[] value); + + /** + *

    If the database contains a mapping for "key", erase it. Else do nothing.

    + * + * @param key Key to delete within database + */ + void remove(byte[] key); + + /** + *

    If column family contains a mapping for "key", erase it. Else do nothing.

    + * + * @param columnFamilyHandle {@link ColumnFamilyHandle} instance + * @param key Key to delete within database + */ + void remove(ColumnFamilyHandle columnFamilyHandle, byte[] key); + + /** + * Removes the database entries in the range ["beginKey", "endKey"), i.e., + * including "beginKey" and excluding "endKey". a non-OK status on error. It + * is not an error if no keys exist in the range ["beginKey", "endKey"). + * + * Delete the database entry (if any) for "key". Returns OK on success, and a + * non-OK status on error. It is not an error if "key" did not exist in the + * database. + * + * @param beginKey + * First key to delete within database (included) + * @param endKey + * Last key to delete within database (excluded) + */ + void deleteRange(byte[] beginKey, byte[] endKey); + + /** + * Removes the database entries in the range ["beginKey", "endKey"), i.e., + * including "beginKey" and excluding "endKey". a non-OK status on error. It + * is not an error if no keys exist in the range ["beginKey", "endKey"). + * + * Delete the database entry (if any) for "key". Returns OK on success, and a + * non-OK status on error. It is not an error if "key" did not exist in the + * database. + * + * @param columnFamilyHandle {@link ColumnFamilyHandle} instance + * @param beginKey + * First key to delete within database (included) + * @param endKey + * Last key to delete within database (excluded) + */ + void deleteRange(ColumnFamilyHandle columnFamilyHandle, byte[] beginKey, byte[] endKey); + + /** + * Append a blob of arbitrary size to the records in this batch. The blob will + * be stored in the transaction log but not in any other file. In particular, + * it will not be persisted to the SST files. When iterating over this + * WriteBatch, WriteBatch::Handler::LogData will be called with the contents + * of the blob as it is encountered. Blobs, puts, deletes, and merges will be + * encountered in the same order in thich they were inserted. The blob will + * NOT consume sequence number(s) and will NOT increase the count of the batch + * + * Example application: add timestamps to the transaction log for use in + * replication. + * + * @param blob binary object to be inserted + */ + void putLogData(byte[] blob); + + /** + * Clear all updates buffered in this batch + */ + void clear(); + + /** + * Records the state of the batch for future calls to RollbackToSavePoint(). + * May be called multiple times to set multiple save points. + */ + void setSavePoint(); + + /** + * Remove all entries in this batch (Put, Merge, Delete, PutLogData) since + * the most recent call to SetSavePoint() and removes the most recent save + * point. + * + * @throws RocksDBException if there is no previous call to SetSavePoint() + */ + void rollbackToSavePoint() throws RocksDBException; +} diff --git a/java/src/main/java/org/rocksdb/WriteBatchWithIndex.java b/java/src/main/java/org/rocksdb/WriteBatchWithIndex.java new file mode 100644 index 00000000000..fdf89b2798c --- /dev/null +++ b/java/src/main/java/org/rocksdb/WriteBatchWithIndex.java @@ -0,0 +1,282 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Similar to {@link org.rocksdb.WriteBatch} but with a binary searchable + * index built for all the keys inserted. + * + * Calling put, merge, remove or putLogData calls the same function + * as with {@link org.rocksdb.WriteBatch} whilst also building an index. + * + * A user can call {@link org.rocksdb.WriteBatchWithIndex#newIterator()} to + * create an iterator over the write batch or + * {@link org.rocksdb.WriteBatchWithIndex#newIteratorWithBase(org.rocksdb.RocksIterator)} + * to get an iterator for the database with Read-Your-Own-Writes like capability + */ +public class WriteBatchWithIndex extends AbstractWriteBatch { + /** + * Creates a WriteBatchWithIndex where no bytes + * are reserved up-front, bytewise comparison is + * used for fallback key comparisons, + * and duplicate keys operations are retained + */ + public WriteBatchWithIndex() { + super(newWriteBatchWithIndex()); + } + + + /** + * Creates a WriteBatchWithIndex where no bytes + * are reserved up-front, bytewise comparison is + * used for fallback key comparisons, and duplicate key + * assignment is determined by the constructor argument + * + * @param overwriteKey if true, overwrite the key in the index when + * inserting a duplicate key, in this way an iterator will never + * show two entries with the same key. + */ + public WriteBatchWithIndex(final boolean overwriteKey) { + super(newWriteBatchWithIndex(overwriteKey)); + } + + /** + * Creates a WriteBatchWithIndex + * + * @param fallbackIndexComparator We fallback to this comparator + * to compare keys within a column family if we cannot determine + * the column family and so look up it's comparator. + * + * @param reservedBytes reserved bytes in underlying WriteBatch + * + * @param overwriteKey if true, overwrite the key in the index when + * inserting a duplicate key, in this way an iterator will never + * show two entries with the same key. + */ + public WriteBatchWithIndex( + final AbstractComparator> + fallbackIndexComparator, final int reservedBytes, + final boolean overwriteKey) { + super(newWriteBatchWithIndex(fallbackIndexComparator.getNativeHandle(), + reservedBytes, overwriteKey)); + } + + /** + * Create an iterator of a column family. User can call + * {@link org.rocksdb.RocksIteratorInterface#seek(byte[])} to + * search to the next entry of or after a key. Keys will be iterated in the + * order given by index_comparator. For multiple updates on the same key, + * each update will be returned as a separate entry, in the order of update + * time. + * + * @param columnFamilyHandle The column family to iterate over + * @return An iterator for the Write Batch contents, restricted to the column + * family + */ + public WBWIRocksIterator newIterator( + final ColumnFamilyHandle columnFamilyHandle) { + return new WBWIRocksIterator(this, iterator1(nativeHandle_, + columnFamilyHandle.nativeHandle_)); + } + + /** + * Create an iterator of the default column family. User can call + * {@link org.rocksdb.RocksIteratorInterface#seek(byte[])} to + * search to the next entry of or after a key. Keys will be iterated in the + * order given by index_comparator. For multiple updates on the same key, + * each update will be returned as a separate entry, in the order of update + * time. + * + * @return An iterator for the Write Batch contents + */ + public WBWIRocksIterator newIterator() { + return new WBWIRocksIterator(this, iterator0(nativeHandle_)); + } + + /** + * Provides Read-Your-Own-Writes like functionality by + * creating a new Iterator that will use {@link org.rocksdb.WBWIRocksIterator} + * as a delta and baseIterator as a base + * + * @param columnFamilyHandle The column family to iterate over + * @param baseIterator The base iterator, + * e.g. {@link org.rocksdb.RocksDB#newIterator()} + * @return An iterator which shows a view comprised of both the database + * point-in-time from baseIterator and modifications made in this write batch. + */ + public RocksIterator newIteratorWithBase( + final ColumnFamilyHandle columnFamilyHandle, + final RocksIterator baseIterator) { + RocksIterator iterator = new RocksIterator( + baseIterator.parent_, + iteratorWithBase(nativeHandle_, + columnFamilyHandle.nativeHandle_, + baseIterator.nativeHandle_)); + //when the iterator is deleted it will also delete the baseIterator + baseIterator.disOwnNativeHandle(); + return iterator; + } + + /** + * Provides Read-Your-Own-Writes like functionality by + * creating a new Iterator that will use {@link org.rocksdb.WBWIRocksIterator} + * as a delta and baseIterator as a base. Operates on the default column + * family. + * + * @param baseIterator The base iterator, + * e.g. {@link org.rocksdb.RocksDB#newIterator()} + * @return An iterator which shows a view comprised of both the database + * point-in-timefrom baseIterator and modifications made in this write batch. + */ + public RocksIterator newIteratorWithBase(final RocksIterator baseIterator) { + return newIteratorWithBase(baseIterator.parent_.getDefaultColumnFamily(), + baseIterator); + } + + /** + * Similar to {@link RocksDB#get(ColumnFamilyHandle, byte[])} but will only + * read the key from this batch. + * + * @param columnFamilyHandle The column family to retrieve the value from + * @param options The database options to use + * @param key The key to read the value for + * + * @return a byte array storing the value associated with the input key if + * any. null if it does not find the specified key. + * + * @throws RocksDBException if the batch does not have enough data to resolve + * Merge operations, MergeInProgress status may be returned. + */ + public byte[] getFromBatch(final ColumnFamilyHandle columnFamilyHandle, + final DBOptions options, final byte[] key) throws RocksDBException { + return getFromBatch(nativeHandle_, options.nativeHandle_, + key, key.length, columnFamilyHandle.nativeHandle_); + } + + /** + * Similar to {@link RocksDB#get(byte[])} but will only + * read the key from this batch. + * + * @param options The database options to use + * @param key The key to read the value for + * + * @return a byte array storing the value associated with the input key if + * any. null if it does not find the specified key. + * + * @throws RocksDBException if the batch does not have enough data to resolve + * Merge operations, MergeInProgress status may be returned. + */ + public byte[] getFromBatch(final DBOptions options, final byte[] key) + throws RocksDBException { + return getFromBatch(nativeHandle_, options.nativeHandle_, key, key.length); + } + + /** + * Similar to {@link RocksDB#get(ColumnFamilyHandle, byte[])} but will also + * read writes from this batch. + * + * This function will query both this batch and the DB and then merge + * the results using the DB's merge operator (if the batch contains any + * merge requests). + * + * Setting {@link ReadOptions#setSnapshot(long, long)} will affect what is + * read from the DB but will NOT change which keys are read from the batch + * (the keys in this batch do not yet belong to any snapshot and will be + * fetched regardless). + * + * @param db The Rocks database + * @param columnFamilyHandle The column family to retrieve the value from + * @param options The read options to use + * @param key The key to read the value for + * + * @return a byte array storing the value associated with the input key if + * any. null if it does not find the specified key. + * + * @throws RocksDBException if the value for the key cannot be read + */ + public byte[] getFromBatchAndDB(final RocksDB db, final ColumnFamilyHandle columnFamilyHandle, + final ReadOptions options, final byte[] key) throws RocksDBException { + return getFromBatchAndDB(nativeHandle_, db.nativeHandle_, + options.nativeHandle_, key, key.length, + columnFamilyHandle.nativeHandle_); + } + + /** + * Similar to {@link RocksDB#get(byte[])} but will also + * read writes from this batch. + * + * This function will query both this batch and the DB and then merge + * the results using the DB's merge operator (if the batch contains any + * merge requests). + * + * Setting {@link ReadOptions#setSnapshot(long, long)} will affect what is + * read from the DB but will NOT change which keys are read from the batch + * (the keys in this batch do not yet belong to any snapshot and will be + * fetched regardless). + * + * @param db The Rocks database + * @param options The read options to use + * @param key The key to read the value for + * + * @return a byte array storing the value associated with the input key if + * any. null if it does not find the specified key. + * + * @throws RocksDBException if the value for the key cannot be read + */ + public byte[] getFromBatchAndDB(final RocksDB db, final ReadOptions options, + final byte[] key) throws RocksDBException { + return getFromBatchAndDB(nativeHandle_, db.nativeHandle_, + options.nativeHandle_, key, key.length); + } + + @Override protected final native void disposeInternal(final long handle); + @Override final native int count0(final long handle); + @Override final native void put(final long handle, final byte[] key, + final int keyLen, final byte[] value, final int valueLen); + @Override final native void put(final long handle, final byte[] key, + final int keyLen, final byte[] value, final int valueLen, + final long cfHandle); + @Override final native void merge(final long handle, final byte[] key, + final int keyLen, final byte[] value, final int valueLen); + @Override final native void merge(final long handle, final byte[] key, + final int keyLen, final byte[] value, final int valueLen, + final long cfHandle); + @Override final native void remove(final long handle, final byte[] key, + final int keyLen); + @Override final native void remove(final long handle, final byte[] key, + final int keyLen, final long cfHandle); + @Override + final native void deleteRange(final long handle, final byte[] beginKey, final int beginKeyLen, + final byte[] endKey, final int endKeyLen); + @Override + final native void deleteRange(final long handle, final byte[] beginKey, final int beginKeyLen, + final byte[] endKey, final int endKeyLen, final long cfHandle); + @Override final native void putLogData(final long handle, final byte[] blob, + final int blobLen); + @Override final native void clear0(final long handle); + @Override final native void setSavePoint0(final long handle); + @Override final native void rollbackToSavePoint0(final long handle); + + private native static long newWriteBatchWithIndex(); + private native static long newWriteBatchWithIndex(final boolean overwriteKey); + private native static long newWriteBatchWithIndex( + final long fallbackIndexComparatorHandle, final int reservedBytes, + final boolean overwriteKey); + private native long iterator0(final long handle); + private native long iterator1(final long handle, final long cfHandle); + private native long iteratorWithBase(final long handle, + final long baseIteratorHandle, final long cfHandle); + private native byte[] getFromBatch(final long handle, final long optHandle, + final byte[] key, final int keyLen); + private native byte[] getFromBatch(final long handle, final long optHandle, + final byte[] key, final int keyLen, final long cfHandle); + private native byte[] getFromBatchAndDB(final long handle, + final long dbHandle, final long readOptHandle, final byte[] key, + final int keyLen); + private native byte[] getFromBatchAndDB(final long handle, + final long dbHandle, final long readOptHandle, final byte[] key, + final int keyLen, final long cfHandle); +} diff --git a/java/src/main/java/org/rocksdb/WriteOptions.java b/java/src/main/java/org/rocksdb/WriteOptions.java new file mode 100644 index 00000000000..b9e8ad81c23 --- /dev/null +++ b/java/src/main/java/org/rocksdb/WriteOptions.java @@ -0,0 +1,159 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Options that control write operations. + * + * Note that developers should call WriteOptions.dispose() to release the + * c++ side memory before a WriteOptions instance runs out of scope. + */ +public class WriteOptions extends RocksObject { + /** + * Construct WriteOptions instance. + */ + public WriteOptions() { + super(newWriteOptions()); + + } + + /** + * If true, the write will be flushed from the operating system + * buffer cache (by calling WritableFile::Sync()) before the write + * is considered complete. If this flag is true, writes will be + * slower. + * + * If this flag is false, and the machine crashes, some recent + * writes may be lost. Note that if it is just the process that + * crashes (i.e., the machine does not reboot), no writes will be + * lost even if sync==false. + * + * In other words, a DB write with sync==false has similar + * crash semantics as the "write()" system call. A DB write + * with sync==true has similar crash semantics to a "write()" + * system call followed by "fdatasync()". + * + * Default: false + * + * @param flag a boolean flag to indicate whether a write + * should be synchronized. + * @return the instance of the current WriteOptions. + */ + public WriteOptions setSync(final boolean flag) { + setSync(nativeHandle_, flag); + return this; + } + + /** + * If true, the write will be flushed from the operating system + * buffer cache (by calling WritableFile::Sync()) before the write + * is considered complete. If this flag is true, writes will be + * slower. + * + * If this flag is false, and the machine crashes, some recent + * writes may be lost. Note that if it is just the process that + * crashes (i.e., the machine does not reboot), no writes will be + * lost even if sync==false. + * + * In other words, a DB write with sync==false has similar + * crash semantics as the "write()" system call. A DB write + * with sync==true has similar crash semantics to a "write()" + * system call followed by "fdatasync()". + * + * @return boolean value indicating if sync is active. + */ + public boolean sync() { + return sync(nativeHandle_); + } + + /** + * If true, writes will not first go to the write ahead log, + * and the write may got lost after a crash. + * + * @param flag a boolean flag to specify whether to disable + * write-ahead-log on writes. + * @return the instance of the current WriteOptions. + */ + public WriteOptions setDisableWAL(final boolean flag) { + setDisableWAL(nativeHandle_, flag); + return this; + } + + /** + * If true, writes will not first go to the write ahead log, + * and the write may got lost after a crash. + * + * @return boolean value indicating if WAL is disabled. + */ + public boolean disableWAL() { + return disableWAL(nativeHandle_); + } + + /** + * If true and if user is trying to write to column families that don't exist + * (they were dropped), ignore the write (don't return an error). If there + * are multiple writes in a WriteBatch, other writes will succeed. + * + * Default: false + * + * @param ignoreMissingColumnFamilies true to ignore writes to column families + * which don't exist + * @return the instance of the current WriteOptions. + */ + public WriteOptions setIgnoreMissingColumnFamilies( + final boolean ignoreMissingColumnFamilies) { + setIgnoreMissingColumnFamilies(nativeHandle_, ignoreMissingColumnFamilies); + return this; + } + + /** + * If true and if user is trying to write to column families that don't exist + * (they were dropped), ignore the write (don't return an error). If there + * are multiple writes in a WriteBatch, other writes will succeed. + * + * Default: false + * + * @return true if writes to column families which don't exist are ignored + */ + public boolean ignoreMissingColumnFamilies() { + return ignoreMissingColumnFamilies(nativeHandle_); + } + + /** + * If true and we need to wait or sleep for the write request, fails + * immediately with {@link Status.Code#Incomplete}. + * + * @param noSlowdown true to fail write requests if we need to wait or sleep + * @return the instance of the current WriteOptions. + */ + public WriteOptions setNoSlowdown(final boolean noSlowdown) { + setNoSlowdown(nativeHandle_, noSlowdown); + return this; + } + + /** + * If true and we need to wait or sleep for the write request, fails + * immediately with {@link Status.Code#Incomplete}. + * + * @return true when write requests are failed if we need to wait or sleep + */ + public boolean noSlowdown() { + return noSlowdown(nativeHandle_); + } + + private native static long newWriteOptions(); + private native void setSync(long handle, boolean flag); + private native boolean sync(long handle); + private native void setDisableWAL(long handle, boolean flag); + private native boolean disableWAL(long handle); + private native void setIgnoreMissingColumnFamilies(final long handle, + final boolean ignoreMissingColumnFamilies); + private native boolean ignoreMissingColumnFamilies(final long handle); + private native void setNoSlowdown(final long handle, + final boolean noSlowdown); + private native boolean noSlowdown(final long handle); + @Override protected final native void disposeInternal(final long handle); +} diff --git a/java/src/main/java/org/rocksdb/util/BytewiseComparator.java b/java/src/main/java/org/rocksdb/util/BytewiseComparator.java new file mode 100644 index 00000000000..18f73919da8 --- /dev/null +++ b/java/src/main/java/org/rocksdb/util/BytewiseComparator.java @@ -0,0 +1,91 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb.util; + +import org.rocksdb.*; + +import java.nio.ByteBuffer; + +/** + * This is a Java Native implementation of the C++ + * equivalent BytewiseComparatorImpl using {@link Slice} + * + * The performance of Comparators implemented in Java is always + * less than their C++ counterparts due to the bridging overhead, + * as such you likely don't want to use this apart from benchmarking + * and you most likely instead wanted + * {@link org.rocksdb.BuiltinComparator#BYTEWISE_COMPARATOR} + */ +public class BytewiseComparator extends Comparator { + + public BytewiseComparator(final ComparatorOptions copt) { + super(copt); + } + + @Override + public String name() { + return "rocksdb.java.BytewiseComparator"; + } + + @Override + public int compare(final Slice a, final Slice b) { + return compare(a.data(), b.data()); + } + + @Override + public String findShortestSeparator(final String start, + final Slice limit) { + final byte[] startBytes = start.getBytes(); + final byte[] limitBytes = limit.data(); + + // Find length of common prefix + final int min_length = Math.min(startBytes.length, limit.size()); + int diff_index = 0; + while ((diff_index < min_length) && + (startBytes[diff_index] == limitBytes[diff_index])) { + diff_index++; + } + + if (diff_index >= min_length) { + // Do not shorten if one string is a prefix of the other + } else { + final byte diff_byte = startBytes[diff_index]; + if(diff_byte < 0xff && diff_byte + 1 < limitBytes[diff_index]) { + final byte shortest[] = new byte[diff_index + 1]; + System.arraycopy(startBytes, 0, shortest, 0, diff_index + 1); + shortest[diff_index]++; + assert(compare(shortest, limitBytes) < 0); + return new String(shortest); + } + } + + return null; + } + + private static int compare(final byte[] a, final byte[] b) { + return ByteBuffer.wrap(a).compareTo(ByteBuffer.wrap(b)); + } + + @Override + public String findShortSuccessor(final String key) { + final byte[] keyBytes = key.getBytes(); + + // Find first character that can be incremented + final int n = keyBytes.length; + for (int i = 0; i < n; i++) { + final byte byt = keyBytes[i]; + if (byt != 0xff) { + final byte shortSuccessor[] = new byte[i + 1]; + System.arraycopy(keyBytes, 0, shortSuccessor, 0, i + 1); + shortSuccessor[i]++; + return new String(shortSuccessor); + } + } + // *key is a run of 0xffs. Leave it alone. + + return null; + } +} diff --git a/java/src/main/java/org/rocksdb/util/DirectBytewiseComparator.java b/java/src/main/java/org/rocksdb/util/DirectBytewiseComparator.java new file mode 100644 index 00000000000..9417544f7a1 --- /dev/null +++ b/java/src/main/java/org/rocksdb/util/DirectBytewiseComparator.java @@ -0,0 +1,88 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb.util; + +import org.rocksdb.ComparatorOptions; +import org.rocksdb.DirectComparator; +import org.rocksdb.DirectSlice; + +import java.nio.ByteBuffer; + +/** + * This is a Java Native implementation of the C++ + * equivalent BytewiseComparatorImpl using {@link DirectSlice} + * + * The performance of Comparators implemented in Java is always + * less than their C++ counterparts due to the bridging overhead, + * as such you likely don't want to use this apart from benchmarking + * and you most likely instead wanted + * {@link org.rocksdb.BuiltinComparator#BYTEWISE_COMPARATOR} + */ +public class DirectBytewiseComparator extends DirectComparator { + + public DirectBytewiseComparator(final ComparatorOptions copt) { + super(copt); + } + + @Override + public String name() { + return "rocksdb.java.DirectBytewiseComparator"; + } + + @Override + public int compare(final DirectSlice a, final DirectSlice b) { + return a.data().compareTo(b.data()); + } + + @Override + public String findShortestSeparator(final String start, + final DirectSlice limit) { + final byte[] startBytes = start.getBytes(); + + // Find length of common prefix + final int min_length = Math.min(startBytes.length, limit.size()); + int diff_index = 0; + while ((diff_index < min_length) && + (startBytes[diff_index] == limit.get(diff_index))) { + diff_index++; + } + + if (diff_index >= min_length) { + // Do not shorten if one string is a prefix of the other + } else { + final byte diff_byte = startBytes[diff_index]; + if(diff_byte < 0xff && diff_byte + 1 < limit.get(diff_index)) { + final byte shortest[] = new byte[diff_index + 1]; + System.arraycopy(startBytes, 0, shortest, 0, diff_index + 1); + shortest[diff_index]++; + assert(ByteBuffer.wrap(shortest).compareTo(limit.data()) < 0); + return new String(shortest); + } + } + + return null; + } + + @Override + public String findShortSuccessor(final String key) { + final byte[] keyBytes = key.getBytes(); + + // Find first character that can be incremented + final int n = keyBytes.length; + for (int i = 0; i < n; i++) { + final byte byt = keyBytes[i]; + if (byt != 0xff) { + final byte shortSuccessor[] = new byte[i + 1]; + System.arraycopy(keyBytes, 0, shortSuccessor, 0, i + 1); + shortSuccessor[i]++; + return new String(shortSuccessor); + } + } + // *key is a run of 0xffs. Leave it alone. + + return null; + } +} diff --git a/java/src/main/java/org/rocksdb/util/Environment.java b/java/src/main/java/org/rocksdb/util/Environment.java new file mode 100644 index 00000000000..f84e14bc197 --- /dev/null +++ b/java/src/main/java/org/rocksdb/util/Environment.java @@ -0,0 +1,90 @@ +package org.rocksdb.util; + +public class Environment { + private static String OS = System.getProperty("os.name").toLowerCase(); + private static String ARCH = System.getProperty("os.arch").toLowerCase(); + + public static boolean isPowerPC() { + return ARCH.contains("ppc"); + } + + public static boolean isWindows() { + return (OS.contains("win")); + } + + public static boolean isMac() { + return (OS.contains("mac")); + } + + public static boolean isAix() { + return OS.contains("aix"); + } + + public static boolean isUnix() { + return OS.contains("nix") || + OS.contains("nux"); + } + + public static boolean isSolaris() { + return OS.contains("sunos"); + } + + public static boolean is64Bit() { + if (ARCH.indexOf("sparcv9") >= 0) { + return true; + } + return (ARCH.indexOf("64") > 0); + } + + public static String getSharedLibraryName(final String name) { + return name + "jni"; + } + + public static String getSharedLibraryFileName(final String name) { + return appendLibOsSuffix("lib" + getSharedLibraryName(name), true); + } + + public static String getJniLibraryName(final String name) { + if (isUnix()) { + final String arch = is64Bit() ? "64" : "32"; + if(isPowerPC()) { + return String.format("%sjni-linux-%s", name, ARCH); + } else { + return String.format("%sjni-linux%s", name, arch); + } + } else if (isMac()) { + return String.format("%sjni-osx", name); + } else if (isAix() && is64Bit()) { + return String.format("%sjni-aix64", name); + } else if (isSolaris()) { + final String arch = is64Bit() ? "64" : "32"; + return String.format("%sjni-solaris%s", name, arch); + } else if (isWindows() && is64Bit()) { + return String.format("%sjni-win64", name); + } + + throw new UnsupportedOperationException(String.format("Cannot determine JNI library name for ARCH='%s' OS='%s' name='%s'", ARCH, OS, name)); + } + + public static String getJniLibraryFileName(final String name) { + return appendLibOsSuffix("lib" + getJniLibraryName(name), false); + } + + private static String appendLibOsSuffix(final String libraryFileName, final boolean shared) { + if (isUnix() || isAix() || isSolaris()) { + return libraryFileName + ".so"; + } else if (isMac()) { + return libraryFileName + (shared ? ".dylib" : ".jnilib"); + } else if (isWindows()) { + return libraryFileName + ".dll"; + } + throw new UnsupportedOperationException(); + } + + public static String getJniLibraryExtension() { + if (isWindows()) { + return ".dll"; + } + return (isMac()) ? ".jnilib" : ".so"; + } +} diff --git a/java/src/main/java/org/rocksdb/util/ReverseBytewiseComparator.java b/java/src/main/java/org/rocksdb/util/ReverseBytewiseComparator.java new file mode 100644 index 00000000000..7fbac2fd6b9 --- /dev/null +++ b/java/src/main/java/org/rocksdb/util/ReverseBytewiseComparator.java @@ -0,0 +1,37 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb.util; + +import org.rocksdb.BuiltinComparator; +import org.rocksdb.ComparatorOptions; +import org.rocksdb.Slice; + +/** + * This is a Java Native implementation of the C++ + * equivalent ReverseBytewiseComparatorImpl using {@link Slice} + * + * The performance of Comparators implemented in Java is always + * less than their C++ counterparts due to the bridging overhead, + * as such you likely don't want to use this apart from benchmarking + * and you most likely instead wanted + * {@link BuiltinComparator#REVERSE_BYTEWISE_COMPARATOR} + */ +public class ReverseBytewiseComparator extends BytewiseComparator { + + public ReverseBytewiseComparator(final ComparatorOptions copt) { + super(copt); + } + + @Override + public String name() { + return "rocksdb.java.ReverseBytewiseComparator"; + } + + @Override + public int compare(final Slice a, final Slice b) { + return -super.compare(a, b); + } +} diff --git a/java/src/main/java/org/rocksdb/util/SizeUnit.java b/java/src/main/java/org/rocksdb/util/SizeUnit.java new file mode 100644 index 00000000000..0f717e8d454 --- /dev/null +++ b/java/src/main/java/org/rocksdb/util/SizeUnit.java @@ -0,0 +1,16 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb.util; + +public class SizeUnit { + public static final long KB = 1024L; + public static final long MB = KB * KB; + public static final long GB = KB * MB; + public static final long TB = KB * GB; + public static final long PB = KB * TB; + + private SizeUnit() {} +} diff --git a/java/src/test/java/org/rocksdb/AbstractComparatorTest.java b/java/src/test/java/org/rocksdb/AbstractComparatorTest.java new file mode 100644 index 00000000000..91a1e99942d --- /dev/null +++ b/java/src/test/java/org/rocksdb/AbstractComparatorTest.java @@ -0,0 +1,199 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.io.IOException; +import java.nio.file.*; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.Random; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.rocksdb.Types.byteToInt; +import static org.rocksdb.Types.intToByte; + +/** + * Abstract tests for both Comparator and DirectComparator + */ +public abstract class AbstractComparatorTest { + + /** + * Get a comparator which will expect Integer keys + * and determine an ascending order + * + * @return An integer ascending order key comparator + */ + public abstract AbstractComparator getAscendingIntKeyComparator(); + + /** + * Test which stores random keys into the database + * using an @see getAscendingIntKeyComparator + * it then checks that these keys are read back in + * ascending order + * + * @param db_path A path where we can store database + * files temporarily + * + * @throws java.io.IOException if IO error happens. + */ + public void testRoundtrip(final Path db_path) throws IOException, + RocksDBException { + try (final AbstractComparator comparator = getAscendingIntKeyComparator(); + final Options opt = new Options() + .setCreateIfMissing(true) + .setComparator(comparator)) { + + // store 10,000 random integer keys + final int ITERATIONS = 10000; + try (final RocksDB db = RocksDB.open(opt, db_path.toString())) { + final Random random = new Random(); + for (int i = 0; i < ITERATIONS; i++) { + final byte key[] = intToByte(random.nextInt()); + // does key already exist (avoid duplicates) + if (i > 0 && db.get(key) != null) { + i--; // generate a different key + } else { + db.put(key, "value".getBytes()); + } + } + } + + // re-open db and read from start to end + // integer keys should be in ascending + // order as defined by SimpleIntComparator + try (final RocksDB db = RocksDB.open(opt, db_path.toString()); + final RocksIterator it = db.newIterator()) { + it.seekToFirst(); + int lastKey = Integer.MIN_VALUE; + int count = 0; + for (it.seekToFirst(); it.isValid(); it.next()) { + final int thisKey = byteToInt(it.key()); + assertThat(thisKey).isGreaterThan(lastKey); + lastKey = thisKey; + count++; + } + assertThat(count).isEqualTo(ITERATIONS); + } + } + } + + /** + * Test which stores random keys into a column family + * in the database + * using an @see getAscendingIntKeyComparator + * it then checks that these keys are read back in + * ascending order + * + * @param db_path A path where we can store database + * files temporarily + * + * @throws java.io.IOException if IO error happens. + */ + public void testRoundtripCf(final Path db_path) throws IOException, + RocksDBException { + + try(final AbstractComparator comparator = getAscendingIntKeyComparator()) { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes(), + new ColumnFamilyOptions().setComparator(comparator)) + ); + + final List cfHandles = new ArrayList<>(); + + try (final DBOptions opt = new DBOptions(). + setCreateIfMissing(true). + setCreateMissingColumnFamilies(true)) { + + // store 10,000 random integer keys + final int ITERATIONS = 10000; + + try (final RocksDB db = RocksDB.open(opt, db_path.toString(), + cfDescriptors, cfHandles)) { + try { + assertThat(cfDescriptors.size()).isEqualTo(2); + assertThat(cfHandles.size()).isEqualTo(2); + + final Random random = new Random(); + for (int i = 0; i < ITERATIONS; i++) { + final byte key[] = intToByte(random.nextInt()); + if (i > 0 && db.get(cfHandles.get(1), key) != null) { + // does key already exist (avoid duplicates) + i--; // generate a different key + } else { + db.put(cfHandles.get(1), key, "value".getBytes()); + } + } + } finally { + for (final ColumnFamilyHandle handle : cfHandles) { + handle.close(); + } + } + cfHandles.clear(); + } + + // re-open db and read from start to end + // integer keys should be in ascending + // order as defined by SimpleIntComparator + try (final RocksDB db = RocksDB.open(opt, db_path.toString(), + cfDescriptors, cfHandles); + final RocksIterator it = db.newIterator(cfHandles.get(1))) { + try { + assertThat(cfDescriptors.size()).isEqualTo(2); + assertThat(cfHandles.size()).isEqualTo(2); + + it.seekToFirst(); + int lastKey = Integer.MIN_VALUE; + int count = 0; + for (it.seekToFirst(); it.isValid(); it.next()) { + final int thisKey = byteToInt(it.key()); + assertThat(thisKey).isGreaterThan(lastKey); + lastKey = thisKey; + count++; + } + + assertThat(count).isEqualTo(ITERATIONS); + + } finally { + for (final ColumnFamilyHandle handle : cfHandles) { + handle.close(); + } + } + cfHandles.clear(); + } + } + } + } + + /** + * Compares integer keys + * so that they are in ascending order + * + * @param a 4-bytes representing an integer key + * @param b 4-bytes representing an integer key + * + * @return negative if a < b, 0 if a == b, positive otherwise + */ + protected final int compareIntKeys(final byte[] a, final byte[] b) { + + final int iA = byteToInt(a); + final int iB = byteToInt(b); + + // protect against int key calculation overflow + final double diff = (double)iA - iB; + final int result; + if (diff < Integer.MIN_VALUE) { + result = Integer.MIN_VALUE; + } else if(diff > Integer.MAX_VALUE) { + result = Integer.MAX_VALUE; + } else { + result = (int)diff; + } + + return result; + } +} diff --git a/java/src/test/java/org/rocksdb/BackupEngineTest.java b/java/src/test/java/org/rocksdb/BackupEngineTest.java new file mode 100644 index 00000000000..1caae5098ed --- /dev/null +++ b/java/src/test/java/org/rocksdb/BackupEngineTest.java @@ -0,0 +1,240 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; + +public class BackupEngineTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Rule + public TemporaryFolder backupFolder = new TemporaryFolder(); + + @Test + public void backupDb() throws RocksDBException { + // Open empty database. + try(final Options opt = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + + // Fill database with some test values + prepareDatabase(db); + + // Create two backups + try(final BackupableDBOptions bopt = new BackupableDBOptions( + backupFolder.getRoot().getAbsolutePath()); + final BackupEngine be = BackupEngine.open(opt.getEnv(), bopt)) { + be.createNewBackup(db, false); + be.createNewBackup(db, true); + verifyNumberOfValidBackups(be, 2); + } + } + } + + @Test + public void deleteBackup() throws RocksDBException { + // Open empty database. + try(final Options opt = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + // Fill database with some test values + prepareDatabase(db); + // Create two backups + try(final BackupableDBOptions bopt = new BackupableDBOptions( + backupFolder.getRoot().getAbsolutePath()); + final BackupEngine be = BackupEngine.open(opt.getEnv(), bopt)) { + be.createNewBackup(db, false); + be.createNewBackup(db, true); + final List backupInfo = + verifyNumberOfValidBackups(be, 2); + // Delete the first backup + be.deleteBackup(backupInfo.get(0).backupId()); + final List newBackupInfo = + verifyNumberOfValidBackups(be, 1); + + // The second backup must remain. + assertThat(newBackupInfo.get(0).backupId()). + isEqualTo(backupInfo.get(1).backupId()); + } + } + } + + @Test + public void purgeOldBackups() throws RocksDBException { + // Open empty database. + try(final Options opt = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + // Fill database with some test values + prepareDatabase(db); + // Create four backups + try(final BackupableDBOptions bopt = new BackupableDBOptions( + backupFolder.getRoot().getAbsolutePath()); + final BackupEngine be = BackupEngine.open(opt.getEnv(), bopt)) { + be.createNewBackup(db, false); + be.createNewBackup(db, true); + be.createNewBackup(db, true); + be.createNewBackup(db, true); + final List backupInfo = + verifyNumberOfValidBackups(be, 4); + // Delete everything except the latest backup + be.purgeOldBackups(1); + final List newBackupInfo = + verifyNumberOfValidBackups(be, 1); + // The latest backup must remain. + assertThat(newBackupInfo.get(0).backupId()). + isEqualTo(backupInfo.get(3).backupId()); + } + } + } + + @Test + public void restoreLatestBackup() throws RocksDBException { + try(final Options opt = new Options().setCreateIfMissing(true)) { + // Open empty database. + RocksDB db = null; + try { + db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath()); + // Fill database with some test values + prepareDatabase(db); + + try (final BackupableDBOptions bopt = new BackupableDBOptions( + backupFolder.getRoot().getAbsolutePath()); + final BackupEngine be = BackupEngine.open(opt.getEnv(), bopt)) { + be.createNewBackup(db, true); + verifyNumberOfValidBackups(be, 1); + db.put("key1".getBytes(), "valueV2".getBytes()); + db.put("key2".getBytes(), "valueV2".getBytes()); + be.createNewBackup(db, true); + verifyNumberOfValidBackups(be, 2); + db.put("key1".getBytes(), "valueV3".getBytes()); + db.put("key2".getBytes(), "valueV3".getBytes()); + assertThat(new String(db.get("key1".getBytes()))).endsWith("V3"); + assertThat(new String(db.get("key2".getBytes()))).endsWith("V3"); + + db.close(); + db = null; + + verifyNumberOfValidBackups(be, 2); + // restore db from latest backup + try(final RestoreOptions ropts = new RestoreOptions(false)) { + be.restoreDbFromLatestBackup(dbFolder.getRoot().getAbsolutePath(), + dbFolder.getRoot().getAbsolutePath(), ropts); + } + + // Open database again. + db = RocksDB.open(opt, dbFolder.getRoot().getAbsolutePath()); + + // Values must have suffix V2 because of restoring latest backup. + assertThat(new String(db.get("key1".getBytes()))).endsWith("V2"); + assertThat(new String(db.get("key2".getBytes()))).endsWith("V2"); + } + } finally { + if(db != null) { + db.close(); + } + } + } + } + + @Test + public void restoreFromBackup() + throws RocksDBException { + try(final Options opt = new Options().setCreateIfMissing(true)) { + RocksDB db = null; + try { + // Open empty database. + db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath()); + // Fill database with some test values + prepareDatabase(db); + try (final BackupableDBOptions bopt = new BackupableDBOptions( + backupFolder.getRoot().getAbsolutePath()); + final BackupEngine be = BackupEngine.open(opt.getEnv(), bopt)) { + be.createNewBackup(db, true); + verifyNumberOfValidBackups(be, 1); + db.put("key1".getBytes(), "valueV2".getBytes()); + db.put("key2".getBytes(), "valueV2".getBytes()); + be.createNewBackup(db, true); + verifyNumberOfValidBackups(be, 2); + db.put("key1".getBytes(), "valueV3".getBytes()); + db.put("key2".getBytes(), "valueV3".getBytes()); + assertThat(new String(db.get("key1".getBytes()))).endsWith("V3"); + assertThat(new String(db.get("key2".getBytes()))).endsWith("V3"); + + //close the database + db.close(); + db = null; + + //restore the backup + final List backupInfo = verifyNumberOfValidBackups(be, 2); + // restore db from first backup + be.restoreDbFromBackup(backupInfo.get(0).backupId(), + dbFolder.getRoot().getAbsolutePath(), + dbFolder.getRoot().getAbsolutePath(), + new RestoreOptions(false)); + // Open database again. + db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath()); + // Values must have suffix V2 because of restoring latest backup. + assertThat(new String(db.get("key1".getBytes()))).endsWith("V1"); + assertThat(new String(db.get("key2".getBytes()))).endsWith("V1"); + } + } finally { + if(db != null) { + db.close(); + } + } + } + } + + /** + * Verify backups. + * + * @param be {@link BackupEngine} instance. + * @param expectedNumberOfBackups numerical value + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + private List verifyNumberOfValidBackups(final BackupEngine be, + final int expectedNumberOfBackups) throws RocksDBException { + // Verify that backups exist + assertThat(be.getCorruptedBackups().length). + isEqualTo(0); + be.garbageCollect(); + final List backupInfo = be.getBackupInfo(); + assertThat(backupInfo.size()). + isEqualTo(expectedNumberOfBackups); + return backupInfo; + } + + /** + * Fill database with some test values. + * + * @param db {@link RocksDB} instance. + * @throws RocksDBException thrown if an error occurs within the native + * part of the library. + */ + private void prepareDatabase(final RocksDB db) + throws RocksDBException { + db.put("key1".getBytes(), "valueV1".getBytes()); + db.put("key2".getBytes(), "valueV1".getBytes()); + } +} diff --git a/java/src/test/java/org/rocksdb/BackupableDBOptionsTest.java b/java/src/test/java/org/rocksdb/BackupableDBOptionsTest.java new file mode 100644 index 00000000000..c223014fd28 --- /dev/null +++ b/java/src/test/java/org/rocksdb/BackupableDBOptionsTest.java @@ -0,0 +1,351 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.util.Random; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; + +public class BackupableDBOptionsTest { + + private final static String ARBITRARY_PATH = + System.getProperty("java.io.tmpdir"); + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public ExpectedException exception = ExpectedException.none(); + + public static final Random rand = PlatformRandomHelper. + getPlatformSpecificRandomFactory(); + + @Test + public void backupDir() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + assertThat(backupableDBOptions.backupDir()). + isEqualTo(ARBITRARY_PATH); + } + } + + @Test + public void env() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + assertThat(backupableDBOptions.backupEnv()). + isNull(); + + try(final Env env = new RocksMemEnv()) { + backupableDBOptions.setBackupEnv(env); + assertThat(backupableDBOptions.backupEnv()) + .isEqualTo(env); + } + } + } + + @Test + public void shareTableFiles() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + final boolean value = rand.nextBoolean(); + backupableDBOptions.setShareTableFiles(value); + assertThat(backupableDBOptions.shareTableFiles()). + isEqualTo(value); + } + } + + @Test + public void infoLog() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + assertThat(backupableDBOptions.infoLog()). + isNull(); + + try(final Options options = new Options(); + final Logger logger = new Logger(options){ + @Override + protected void log(InfoLogLevel infoLogLevel, String logMsg) { + + } + }) { + backupableDBOptions.setInfoLog(logger); + assertThat(backupableDBOptions.infoLog()) + .isEqualTo(logger); + } + } + } + + @Test + public void sync() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + final boolean value = rand.nextBoolean(); + backupableDBOptions.setSync(value); + assertThat(backupableDBOptions.sync()).isEqualTo(value); + } + } + + @Test + public void destroyOldData() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH);) { + final boolean value = rand.nextBoolean(); + backupableDBOptions.setDestroyOldData(value); + assertThat(backupableDBOptions.destroyOldData()). + isEqualTo(value); + } + } + + @Test + public void backupLogFiles() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + final boolean value = rand.nextBoolean(); + backupableDBOptions.setBackupLogFiles(value); + assertThat(backupableDBOptions.backupLogFiles()). + isEqualTo(value); + } + } + + @Test + public void backupRateLimit() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + final long value = Math.abs(rand.nextLong()); + backupableDBOptions.setBackupRateLimit(value); + assertThat(backupableDBOptions.backupRateLimit()). + isEqualTo(value); + // negative will be mapped to 0 + backupableDBOptions.setBackupRateLimit(-1); + assertThat(backupableDBOptions.backupRateLimit()). + isEqualTo(0); + } + } + + @Test + public void backupRateLimiter() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + assertThat(backupableDBOptions.backupEnv()). + isNull(); + + try(final RateLimiter backupRateLimiter = + new RateLimiter(999)) { + backupableDBOptions.setBackupRateLimiter(backupRateLimiter); + assertThat(backupableDBOptions.backupRateLimiter()) + .isEqualTo(backupRateLimiter); + } + } + } + + @Test + public void restoreRateLimit() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + final long value = Math.abs(rand.nextLong()); + backupableDBOptions.setRestoreRateLimit(value); + assertThat(backupableDBOptions.restoreRateLimit()). + isEqualTo(value); + // negative will be mapped to 0 + backupableDBOptions.setRestoreRateLimit(-1); + assertThat(backupableDBOptions.restoreRateLimit()). + isEqualTo(0); + } + } + + @Test + public void restoreRateLimiter() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + assertThat(backupableDBOptions.backupEnv()). + isNull(); + + try(final RateLimiter restoreRateLimiter = + new RateLimiter(911)) { + backupableDBOptions.setRestoreRateLimiter(restoreRateLimiter); + assertThat(backupableDBOptions.restoreRateLimiter()) + .isEqualTo(restoreRateLimiter); + } + } + } + + @Test + public void shareFilesWithChecksum() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + boolean value = rand.nextBoolean(); + backupableDBOptions.setShareFilesWithChecksum(value); + assertThat(backupableDBOptions.shareFilesWithChecksum()). + isEqualTo(value); + } + } + + @Test + public void maxBackgroundOperations() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + final int value = rand.nextInt(); + backupableDBOptions.setMaxBackgroundOperations(value); + assertThat(backupableDBOptions.maxBackgroundOperations()). + isEqualTo(value); + } + } + + @Test + public void callbackTriggerIntervalSize() { + try (final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH)) { + final long value = rand.nextLong(); + backupableDBOptions.setCallbackTriggerIntervalSize(value); + assertThat(backupableDBOptions.callbackTriggerIntervalSize()). + isEqualTo(value); + } + } + + @Test + public void failBackupDirIsNull() { + exception.expect(IllegalArgumentException.class); + try (final BackupableDBOptions opts = new BackupableDBOptions(null)) { + //no-op + } + } + + @Test + public void failBackupDirIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.backupDir(); + } + } + + @Test + public void failSetShareTableFilesIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.setShareTableFiles(true); + } + } + + @Test + public void failShareTableFilesIfDisposed() { + try (BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.shareTableFiles(); + } + } + + @Test + public void failSetSyncIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.setSync(true); + } + } + + @Test + public void failSyncIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.sync(); + } + } + + @Test + public void failSetDestroyOldDataIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.setDestroyOldData(true); + } + } + + @Test + public void failDestroyOldDataIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.destroyOldData(); + } + } + + @Test + public void failSetBackupLogFilesIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.setBackupLogFiles(true); + } + } + + @Test + public void failBackupLogFilesIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.backupLogFiles(); + } + } + + @Test + public void failSetBackupRateLimitIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.setBackupRateLimit(1); + } + } + + @Test + public void failBackupRateLimitIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.backupRateLimit(); + } + } + + @Test + public void failSetRestoreRateLimitIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.setRestoreRateLimit(1); + } + } + + @Test + public void failRestoreRateLimitIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.restoreRateLimit(); + } + } + + @Test + public void failSetShareFilesWithChecksumIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.setShareFilesWithChecksum(true); + } + } + + @Test + public void failShareFilesWithChecksumIfDisposed() { + try (final BackupableDBOptions options = + setupUninitializedBackupableDBOptions(exception)) { + options.shareFilesWithChecksum(); + } + } + + private BackupableDBOptions setupUninitializedBackupableDBOptions( + ExpectedException exception) { + final BackupableDBOptions backupableDBOptions = + new BackupableDBOptions(ARBITRARY_PATH); + backupableDBOptions.close(); + exception.expect(AssertionError.class); + return backupableDBOptions; + } +} diff --git a/java/src/test/java/org/rocksdb/BlockBasedTableConfigTest.java b/java/src/test/java/org/rocksdb/BlockBasedTableConfigTest.java new file mode 100644 index 00000000000..8edc8b89fd6 --- /dev/null +++ b/java/src/test/java/org/rocksdb/BlockBasedTableConfigTest.java @@ -0,0 +1,171 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class BlockBasedTableConfigTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void noBlockCache() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setNoBlockCache(true); + assertThat(blockBasedTableConfig.noBlockCache()).isTrue(); + } + + @Test + public void blockCacheSize() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setBlockCacheSize(8 * 1024); + assertThat(blockBasedTableConfig.blockCacheSize()). + isEqualTo(8 * 1024); + } + + @Test + public void blockSizeDeviation() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setBlockSizeDeviation(12); + assertThat(blockBasedTableConfig.blockSizeDeviation()). + isEqualTo(12); + } + + @Test + public void blockRestartInterval() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setBlockRestartInterval(15); + assertThat(blockBasedTableConfig.blockRestartInterval()). + isEqualTo(15); + } + + @Test + public void wholeKeyFiltering() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setWholeKeyFiltering(false); + assertThat(blockBasedTableConfig.wholeKeyFiltering()). + isFalse(); + } + + @Test + public void cacheIndexAndFilterBlocks() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setCacheIndexAndFilterBlocks(true); + assertThat(blockBasedTableConfig.cacheIndexAndFilterBlocks()). + isTrue(); + + } + + @Test + public void hashIndexAllowCollision() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setHashIndexAllowCollision(false); + assertThat(blockBasedTableConfig.hashIndexAllowCollision()). + isFalse(); + } + + @Test + public void blockCacheCompressedSize() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setBlockCacheCompressedSize(40); + assertThat(blockBasedTableConfig.blockCacheCompressedSize()). + isEqualTo(40); + } + + @Test + public void checksumType() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + assertThat(ChecksumType.values().length).isEqualTo(3); + assertThat(ChecksumType.valueOf("kxxHash")). + isEqualTo(ChecksumType.kxxHash); + blockBasedTableConfig.setChecksumType(ChecksumType.kNoChecksum); + blockBasedTableConfig.setChecksumType(ChecksumType.kxxHash); + assertThat(blockBasedTableConfig.checksumType().equals( + ChecksumType.kxxHash)); + } + + @Test + public void indexType() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + assertThat(IndexType.values().length).isEqualTo(3); + blockBasedTableConfig.setIndexType(IndexType.kHashSearch); + assertThat(blockBasedTableConfig.indexType().equals( + IndexType.kHashSearch)); + assertThat(IndexType.valueOf("kBinarySearch")).isNotNull(); + blockBasedTableConfig.setIndexType(IndexType.valueOf("kBinarySearch")); + assertThat(blockBasedTableConfig.indexType().equals( + IndexType.kBinarySearch)); + } + + @Test + public void blockCacheCompressedNumShardBits() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setBlockCacheCompressedNumShardBits(4); + assertThat(blockBasedTableConfig.blockCacheCompressedNumShardBits()). + isEqualTo(4); + } + + @Test + public void cacheNumShardBits() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setCacheNumShardBits(5); + assertThat(blockBasedTableConfig.cacheNumShardBits()). + isEqualTo(5); + } + + @Test + public void blockSize() { + BlockBasedTableConfig blockBasedTableConfig = new BlockBasedTableConfig(); + blockBasedTableConfig.setBlockSize(10); + assertThat(blockBasedTableConfig.blockSize()).isEqualTo(10); + } + + + @Test + public void blockBasedTableWithFilter() { + try(final Options options = new Options() + .setTableFormatConfig(new BlockBasedTableConfig() + .setFilter(new BloomFilter(10)))) { + assertThat(options.tableFactoryName()). + isEqualTo("BlockBasedTable"); + } + } + + @Test + public void blockBasedTableWithoutFilter() { + try(final Options options = new Options().setTableFormatConfig( + new BlockBasedTableConfig().setFilter(null))) { + assertThat(options.tableFactoryName()). + isEqualTo("BlockBasedTable"); + } + } + + @Test + public void blockBasedTableFormatVersion() { + BlockBasedTableConfig config = new BlockBasedTableConfig(); + for (int version=0; version<=2; version++) { + config.setFormatVersion(version); + assertThat(config.formatVersion()).isEqualTo(version); + } + } + + @Test(expected = AssertionError.class) + public void blockBasedTableFormatVersionFailNegative() { + BlockBasedTableConfig config = new BlockBasedTableConfig(); + config.setFormatVersion(-1); + } + + @Test(expected = AssertionError.class) + public void blockBasedTableFormatVersionFailIllegalVersion() { + BlockBasedTableConfig config = new BlockBasedTableConfig(); + config.setFormatVersion(3); + } +} diff --git a/java/src/test/java/org/rocksdb/CheckPointTest.java b/java/src/test/java/org/rocksdb/CheckPointTest.java new file mode 100644 index 00000000000..e79569fb8bd --- /dev/null +++ b/java/src/test/java/org/rocksdb/CheckPointTest.java @@ -0,0 +1,82 @@ +package org.rocksdb; + + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import static org.assertj.core.api.Assertions.assertThat; + +public class CheckPointTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Rule + public TemporaryFolder checkpointFolder = new TemporaryFolder(); + + @Test + public void checkPoint() throws RocksDBException { + try (final Options options = new Options(). + setCreateIfMissing(true)) { + + try (final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + db.put("key".getBytes(), "value".getBytes()); + try (final Checkpoint checkpoint = Checkpoint.create(db)) { + checkpoint.createCheckpoint(checkpointFolder. + getRoot().getAbsolutePath() + "/snapshot1"); + db.put("key2".getBytes(), "value2".getBytes()); + checkpoint.createCheckpoint(checkpointFolder. + getRoot().getAbsolutePath() + "/snapshot2"); + } + } + + try (final RocksDB db = RocksDB.open(options, + checkpointFolder.getRoot().getAbsolutePath() + + "/snapshot1")) { + assertThat(new String(db.get("key".getBytes()))). + isEqualTo("value"); + assertThat(db.get("key2".getBytes())).isNull(); + } + + try (final RocksDB db = RocksDB.open(options, + checkpointFolder.getRoot().getAbsolutePath() + + "/snapshot2")) { + assertThat(new String(db.get("key".getBytes()))). + isEqualTo("value"); + assertThat(new String(db.get("key2".getBytes()))). + isEqualTo("value2"); + } + } + } + + @Test(expected = IllegalArgumentException.class) + public void failIfDbIsNull() { + try (final Checkpoint checkpoint = Checkpoint.create(null)) { + + } + } + + @Test(expected = IllegalStateException.class) + public void failIfDbNotInitialized() throws RocksDBException { + try (final RocksDB db = RocksDB.open( + dbFolder.getRoot().getAbsolutePath())) { + db.close(); + Checkpoint.create(db); + } + } + + @Test(expected = RocksDBException.class) + public void failWithIllegalPath() throws RocksDBException { + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath()); + final Checkpoint checkpoint = Checkpoint.create(db)) { + checkpoint.createCheckpoint("/Z:///:\\C:\\TZ/-"); + } + } +} diff --git a/java/src/test/java/org/rocksdb/ClockCacheTest.java b/java/src/test/java/org/rocksdb/ClockCacheTest.java new file mode 100644 index 00000000000..d1241ac75b8 --- /dev/null +++ b/java/src/test/java/org/rocksdb/ClockCacheTest.java @@ -0,0 +1,26 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.Test; + +public class ClockCacheTest { + + static { + RocksDB.loadLibrary(); + } + + @Test + public void newClockCache() { + final long capacity = 1000; + final int numShardBits = 16; + final boolean strictCapacityLimit = true; + try(final Cache clockCache = new ClockCache(capacity, + numShardBits, strictCapacityLimit)) { + //no op + } + } +} diff --git a/java/src/test/java/org/rocksdb/ColumnFamilyOptionsTest.java b/java/src/test/java/org/rocksdb/ColumnFamilyOptionsTest.java new file mode 100644 index 00000000000..75749437b81 --- /dev/null +++ b/java/src/test/java/org/rocksdb/ColumnFamilyOptionsTest.java @@ -0,0 +1,567 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import java.util.ArrayList; +import java.util.List; +import java.util.Properties; +import java.util.Random; + +import static org.assertj.core.api.Assertions.assertThat; + +public class ColumnFamilyOptionsTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + public static final Random rand = PlatformRandomHelper. + getPlatformSpecificRandomFactory(); + + @Test + public void getColumnFamilyOptionsFromProps() { + Properties properties = new Properties(); + properties.put("write_buffer_size", "112"); + properties.put("max_write_buffer_number", "13"); + + try (final ColumnFamilyOptions opt = ColumnFamilyOptions. + getColumnFamilyOptionsFromProps(properties)) { + // setup sample properties + assertThat(opt).isNotNull(); + assertThat(String.valueOf(opt.writeBufferSize())). + isEqualTo(properties.get("write_buffer_size")); + assertThat(String.valueOf(opt.maxWriteBufferNumber())). + isEqualTo(properties.get("max_write_buffer_number")); + } + } + + @Test + public void failColumnFamilyOptionsFromPropsWithIllegalValue() { + // setup sample properties + final Properties properties = new Properties(); + properties.put("tomato", "1024"); + properties.put("burger", "2"); + + try (final ColumnFamilyOptions opt = + ColumnFamilyOptions.getColumnFamilyOptionsFromProps(properties)) { + assertThat(opt).isNull(); + } + } + + @Test(expected = IllegalArgumentException.class) + public void failColumnFamilyOptionsFromPropsWithNullValue() { + try (final ColumnFamilyOptions opt = + ColumnFamilyOptions.getColumnFamilyOptionsFromProps(null)) { + } + } + + @Test(expected = IllegalArgumentException.class) + public void failColumnFamilyOptionsFromPropsWithEmptyProps() { + try (final ColumnFamilyOptions opt = + ColumnFamilyOptions.getColumnFamilyOptionsFromProps( + new Properties())) { + } + } + + @Test + public void writeBufferSize() throws RocksDBException { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setWriteBufferSize(longValue); + assertThat(opt.writeBufferSize()).isEqualTo(longValue); + } + } + + @Test + public void maxWriteBufferNumber() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setMaxWriteBufferNumber(intValue); + assertThat(opt.maxWriteBufferNumber()).isEqualTo(intValue); + } + } + + @Test + public void minWriteBufferNumberToMerge() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setMinWriteBufferNumberToMerge(intValue); + assertThat(opt.minWriteBufferNumberToMerge()).isEqualTo(intValue); + } + } + + @Test + public void numLevels() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setNumLevels(intValue); + assertThat(opt.numLevels()).isEqualTo(intValue); + } + } + + @Test + public void levelZeroFileNumCompactionTrigger() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setLevelZeroFileNumCompactionTrigger(intValue); + assertThat(opt.levelZeroFileNumCompactionTrigger()).isEqualTo(intValue); + } + } + + @Test + public void levelZeroSlowdownWritesTrigger() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setLevelZeroSlowdownWritesTrigger(intValue); + assertThat(opt.levelZeroSlowdownWritesTrigger()).isEqualTo(intValue); + } + } + + @Test + public void levelZeroStopWritesTrigger() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setLevelZeroStopWritesTrigger(intValue); + assertThat(opt.levelZeroStopWritesTrigger()).isEqualTo(intValue); + } + } + + @Test + public void targetFileSizeBase() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setTargetFileSizeBase(longValue); + assertThat(opt.targetFileSizeBase()).isEqualTo(longValue); + } + } + + @Test + public void targetFileSizeMultiplier() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setTargetFileSizeMultiplier(intValue); + assertThat(opt.targetFileSizeMultiplier()).isEqualTo(intValue); + } + } + + @Test + public void maxBytesForLevelBase() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setMaxBytesForLevelBase(longValue); + assertThat(opt.maxBytesForLevelBase()).isEqualTo(longValue); + } + } + + @Test + public void levelCompactionDynamicLevelBytes() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setLevelCompactionDynamicLevelBytes(boolValue); + assertThat(opt.levelCompactionDynamicLevelBytes()) + .isEqualTo(boolValue); + } + } + + @Test + public void maxBytesForLevelMultiplier() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final double doubleValue = rand.nextDouble(); + opt.setMaxBytesForLevelMultiplier(doubleValue); + assertThat(opt.maxBytesForLevelMultiplier()).isEqualTo(doubleValue); + } + } + + @Test + public void maxBytesForLevelMultiplierAdditional() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue1 = rand.nextInt(); + final int intValue2 = rand.nextInt(); + final int[] ints = new int[]{intValue1, intValue2}; + opt.setMaxBytesForLevelMultiplierAdditional(ints); + assertThat(opt.maxBytesForLevelMultiplierAdditional()).isEqualTo(ints); + } + } + + @Test + public void maxCompactionBytes() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setMaxCompactionBytes(longValue); + assertThat(opt.maxCompactionBytes()).isEqualTo(longValue); + } + } + + @Test + public void softPendingCompactionBytesLimit() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setSoftPendingCompactionBytesLimit(longValue); + assertThat(opt.softPendingCompactionBytesLimit()).isEqualTo(longValue); + } + } + + @Test + public void hardPendingCompactionBytesLimit() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setHardPendingCompactionBytesLimit(longValue); + assertThat(opt.hardPendingCompactionBytesLimit()).isEqualTo(longValue); + } + } + + @Test + public void level0FileNumCompactionTrigger() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setLevel0FileNumCompactionTrigger(intValue); + assertThat(opt.level0FileNumCompactionTrigger()).isEqualTo(intValue); + } + } + + @Test + public void level0SlowdownWritesTrigger() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setLevel0SlowdownWritesTrigger(intValue); + assertThat(opt.level0SlowdownWritesTrigger()).isEqualTo(intValue); + } + } + + @Test + public void level0StopWritesTrigger() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setLevel0StopWritesTrigger(intValue); + assertThat(opt.level0StopWritesTrigger()).isEqualTo(intValue); + } + } + + @Test + public void arenaBlockSize() throws RocksDBException { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setArenaBlockSize(longValue); + assertThat(opt.arenaBlockSize()).isEqualTo(longValue); + } + } + + @Test + public void disableAutoCompactions() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setDisableAutoCompactions(boolValue); + assertThat(opt.disableAutoCompactions()).isEqualTo(boolValue); + } + } + + @Test + public void maxSequentialSkipInIterations() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setMaxSequentialSkipInIterations(longValue); + assertThat(opt.maxSequentialSkipInIterations()).isEqualTo(longValue); + } + } + + @Test + public void inplaceUpdateSupport() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setInplaceUpdateSupport(boolValue); + assertThat(opt.inplaceUpdateSupport()).isEqualTo(boolValue); + } + } + + @Test + public void inplaceUpdateNumLocks() throws RocksDBException { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setInplaceUpdateNumLocks(longValue); + assertThat(opt.inplaceUpdateNumLocks()).isEqualTo(longValue); + } + } + + @Test + public void memtablePrefixBloomSizeRatio() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final double doubleValue = rand.nextDouble(); + opt.setMemtablePrefixBloomSizeRatio(doubleValue); + assertThat(opt.memtablePrefixBloomSizeRatio()).isEqualTo(doubleValue); + } + } + + @Test + public void memtableHugePageSize() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setMemtableHugePageSize(longValue); + assertThat(opt.memtableHugePageSize()).isEqualTo(longValue); + } + } + + @Test + public void bloomLocality() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final int intValue = rand.nextInt(); + opt.setBloomLocality(intValue); + assertThat(opt.bloomLocality()).isEqualTo(intValue); + } + } + + @Test + public void maxSuccessiveMerges() throws RocksDBException { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final long longValue = rand.nextLong(); + opt.setMaxSuccessiveMerges(longValue); + assertThat(opt.maxSuccessiveMerges()).isEqualTo(longValue); + } + } + + @Test + public void optimizeFiltersForHits() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final boolean aBoolean = rand.nextBoolean(); + opt.setOptimizeFiltersForHits(aBoolean); + assertThat(opt.optimizeFiltersForHits()).isEqualTo(aBoolean); + } + } + + @Test + public void memTable() throws RocksDBException { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + opt.setMemTableConfig(new HashLinkedListMemTableConfig()); + assertThat(opt.memTableFactoryName()). + isEqualTo("HashLinkedListRepFactory"); + } + } + + @Test + public void comparator() throws RocksDBException { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + opt.setComparator(BuiltinComparator.BYTEWISE_COMPARATOR); + } + } + + @Test + public void linkageOfPrepMethods() { + try (final ColumnFamilyOptions options = new ColumnFamilyOptions()) { + options.optimizeUniversalStyleCompaction(); + options.optimizeUniversalStyleCompaction(4000); + options.optimizeLevelStyleCompaction(); + options.optimizeLevelStyleCompaction(3000); + options.optimizeForPointLookup(10); + options.optimizeForSmallDb(); + } + } + + @Test + public void shouldSetTestPrefixExtractor() { + try (final ColumnFamilyOptions options = new ColumnFamilyOptions()) { + options.useFixedLengthPrefixExtractor(100); + options.useFixedLengthPrefixExtractor(10); + } + } + + @Test + public void shouldSetTestCappedPrefixExtractor() { + try (final ColumnFamilyOptions options = new ColumnFamilyOptions()) { + options.useCappedPrefixExtractor(100); + options.useCappedPrefixExtractor(10); + } + } + + @Test + public void compressionTypes() { + try (final ColumnFamilyOptions columnFamilyOptions + = new ColumnFamilyOptions()) { + for (final CompressionType compressionType : + CompressionType.values()) { + columnFamilyOptions.setCompressionType(compressionType); + assertThat(columnFamilyOptions.compressionType()). + isEqualTo(compressionType); + assertThat(CompressionType.valueOf("NO_COMPRESSION")). + isEqualTo(CompressionType.NO_COMPRESSION); + } + } + } + + @Test + public void compressionPerLevel() { + try (final ColumnFamilyOptions columnFamilyOptions + = new ColumnFamilyOptions()) { + assertThat(columnFamilyOptions.compressionPerLevel()).isEmpty(); + List compressionTypeList = new ArrayList<>(); + for (int i = 0; i < columnFamilyOptions.numLevels(); i++) { + compressionTypeList.add(CompressionType.NO_COMPRESSION); + } + columnFamilyOptions.setCompressionPerLevel(compressionTypeList); + compressionTypeList = columnFamilyOptions.compressionPerLevel(); + for (CompressionType compressionType : compressionTypeList) { + assertThat(compressionType).isEqualTo( + CompressionType.NO_COMPRESSION); + } + } + } + + @Test + public void differentCompressionsPerLevel() { + try (final ColumnFamilyOptions columnFamilyOptions + = new ColumnFamilyOptions()) { + columnFamilyOptions.setNumLevels(3); + + assertThat(columnFamilyOptions.compressionPerLevel()).isEmpty(); + List compressionTypeList = new ArrayList<>(); + + compressionTypeList.add(CompressionType.BZLIB2_COMPRESSION); + compressionTypeList.add(CompressionType.SNAPPY_COMPRESSION); + compressionTypeList.add(CompressionType.LZ4_COMPRESSION); + + columnFamilyOptions.setCompressionPerLevel(compressionTypeList); + compressionTypeList = columnFamilyOptions.compressionPerLevel(); + + assertThat(compressionTypeList.size()).isEqualTo(3); + assertThat(compressionTypeList). + containsExactly( + CompressionType.BZLIB2_COMPRESSION, + CompressionType.SNAPPY_COMPRESSION, + CompressionType.LZ4_COMPRESSION); + + } + } + + @Test + public void bottommostCompressionType() { + try (final ColumnFamilyOptions columnFamilyOptions + = new ColumnFamilyOptions()) { + assertThat(columnFamilyOptions.bottommostCompressionType()) + .isEqualTo(CompressionType.DISABLE_COMPRESSION_OPTION); + + for (final CompressionType compressionType : CompressionType.values()) { + columnFamilyOptions.setBottommostCompressionType(compressionType); + assertThat(columnFamilyOptions.bottommostCompressionType()) + .isEqualTo(compressionType); + } + } + } + + @Test + public void compressionOptions() { + try (final ColumnFamilyOptions columnFamilyOptions + = new ColumnFamilyOptions(); + final CompressionOptions compressionOptions = new CompressionOptions() + .setMaxDictBytes(123)) { + + columnFamilyOptions.setCompressionOptions(compressionOptions); + assertThat(columnFamilyOptions.compressionOptions()) + .isEqualTo(compressionOptions); + assertThat(columnFamilyOptions.compressionOptions().maxDictBytes()) + .isEqualTo(123); + } + } + + @Test + public void compactionStyles() { + try (final ColumnFamilyOptions columnFamilyOptions + = new ColumnFamilyOptions()) { + for (final CompactionStyle compactionStyle : + CompactionStyle.values()) { + columnFamilyOptions.setCompactionStyle(compactionStyle); + assertThat(columnFamilyOptions.compactionStyle()). + isEqualTo(compactionStyle); + assertThat(CompactionStyle.valueOf("FIFO")). + isEqualTo(CompactionStyle.FIFO); + } + } + } + + @Test + public void maxTableFilesSizeFIFO() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + long longValue = rand.nextLong(); + // Size has to be positive + longValue = (longValue < 0) ? -longValue : longValue; + longValue = (longValue == 0) ? longValue + 1 : longValue; + opt.setMaxTableFilesSizeFIFO(longValue); + assertThat(opt.maxTableFilesSizeFIFO()). + isEqualTo(longValue); + } + } + + @Test + public void maxWriteBufferNumberToMaintain() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + int intValue = rand.nextInt(); + // Size has to be positive + intValue = (intValue < 0) ? -intValue : intValue; + intValue = (intValue == 0) ? intValue + 1 : intValue; + opt.setMaxWriteBufferNumberToMaintain(intValue); + assertThat(opt.maxWriteBufferNumberToMaintain()). + isEqualTo(intValue); + } + } + + @Test + public void compactionPriorities() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + for (final CompactionPriority compactionPriority : + CompactionPriority.values()) { + opt.setCompactionPriority(compactionPriority); + assertThat(opt.compactionPriority()). + isEqualTo(compactionPriority); + } + } + } + + @Test + public void reportBgIoStats() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final boolean booleanValue = true; + opt.setReportBgIoStats(booleanValue); + assertThat(opt.reportBgIoStats()). + isEqualTo(booleanValue); + } + } + + @Test + public void compactionOptionsUniversal() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions(); + final CompactionOptionsUniversal optUni = new CompactionOptionsUniversal() + .setCompressionSizePercent(7)) { + opt.setCompactionOptionsUniversal(optUni); + assertThat(opt.compactionOptionsUniversal()). + isEqualTo(optUni); + assertThat(opt.compactionOptionsUniversal().compressionSizePercent()) + .isEqualTo(7); + } + } + + @Test + public void compactionOptionsFIFO() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions(); + final CompactionOptionsFIFO optFifo = new CompactionOptionsFIFO() + .setMaxTableFilesSize(2000)) { + opt.setCompactionOptionsFIFO(optFifo); + assertThat(opt.compactionOptionsFIFO()). + isEqualTo(optFifo); + assertThat(opt.compactionOptionsFIFO().maxTableFilesSize()) + .isEqualTo(2000); + } + } + + @Test + public void forceConsistencyChecks() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + final boolean booleanValue = true; + opt.setForceConsistencyChecks(booleanValue); + assertThat(opt.forceConsistencyChecks()). + isEqualTo(booleanValue); + } + } +} diff --git a/java/src/test/java/org/rocksdb/ColumnFamilyTest.java b/java/src/test/java/org/rocksdb/ColumnFamilyTest.java new file mode 100644 index 00000000000..19fe332df97 --- /dev/null +++ b/java/src/test/java/org/rocksdb/ColumnFamilyTest.java @@ -0,0 +1,606 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.*; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import static org.assertj.core.api.Assertions.assertThat; + +public class ColumnFamilyTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void listColumnFamilies() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + // Test listColumnFamilies + final List columnFamilyNames = RocksDB.listColumnFamilies(options, + dbFolder.getRoot().getAbsolutePath()); + assertThat(columnFamilyNames).isNotNull(); + assertThat(columnFamilyNames.size()).isGreaterThan(0); + assertThat(columnFamilyNames.size()).isEqualTo(1); + assertThat(new String(columnFamilyNames.get(0))).isEqualTo("default"); + } + } + + @Test + public void defaultColumnFamily() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + final ColumnFamilyHandle cfh = db.getDefaultColumnFamily(); + try { + assertThat(cfh).isNotNull(); + + final byte[] key = "key".getBytes(); + final byte[] value = "value".getBytes(); + + db.put(cfh, key, value); + + final byte[] actualValue = db.get(cfh, key); + + assertThat(cfh).isNotNull(); + assertThat(actualValue).isEqualTo(value); + } finally { + cfh.close(); + } + } + } + + @Test + public void createColumnFamily() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + final ColumnFamilyHandle columnFamilyHandle = db.createColumnFamily( + new ColumnFamilyDescriptor("new_cf".getBytes(), + new ColumnFamilyOptions())); + try { + final List columnFamilyNames = RocksDB.listColumnFamilies( + options, dbFolder.getRoot().getAbsolutePath()); + assertThat(columnFamilyNames).isNotNull(); + assertThat(columnFamilyNames.size()).isGreaterThan(0); + assertThat(columnFamilyNames.size()).isEqualTo(2); + assertThat(new String(columnFamilyNames.get(0))).isEqualTo("default"); + assertThat(new String(columnFamilyNames.get(1))).isEqualTo("new_cf"); + } finally { + columnFamilyHandle.close(); + } + } + } + + @Test + public void openWithColumnFamilies() throws RocksDBException { + final List cfNames = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes()) + ); + + final List columnFamilyHandleList = + new ArrayList<>(); + + // Test open database with column family names + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), cfNames, + columnFamilyHandleList)) { + + try { + assertThat(columnFamilyHandleList.size()).isEqualTo(2); + db.put("dfkey1".getBytes(), "dfvalue".getBytes()); + db.put(columnFamilyHandleList.get(0), "dfkey2".getBytes(), + "dfvalue".getBytes()); + db.put(columnFamilyHandleList.get(1), "newcfkey1".getBytes(), + "newcfvalue".getBytes()); + + String retVal = new String(db.get(columnFamilyHandleList.get(1), + "newcfkey1".getBytes())); + assertThat(retVal).isEqualTo("newcfvalue"); + assertThat((db.get(columnFamilyHandleList.get(1), + "dfkey1".getBytes()))).isNull(); + db.remove(columnFamilyHandleList.get(1), "newcfkey1".getBytes()); + assertThat((db.get(columnFamilyHandleList.get(1), + "newcfkey1".getBytes()))).isNull(); + db.remove(columnFamilyHandleList.get(0), new WriteOptions(), + "dfkey2".getBytes()); + assertThat(db.get(columnFamilyHandleList.get(0), new ReadOptions(), + "dfkey2".getBytes())).isNull(); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test + public void getWithOutValueAndCf() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY)); + final List columnFamilyHandleList = new ArrayList<>(); + + // Test open database with column family names + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + columnFamilyHandleList)) { + try { + db.put(columnFamilyHandleList.get(0), new WriteOptions(), + "key1".getBytes(), "value".getBytes()); + db.put("key2".getBytes(), "12345678".getBytes()); + final byte[] outValue = new byte[5]; + // not found value + int getResult = db.get("keyNotFound".getBytes(), outValue); + assertThat(getResult).isEqualTo(RocksDB.NOT_FOUND); + // found value which fits in outValue + getResult = db.get(columnFamilyHandleList.get(0), "key1".getBytes(), + outValue); + assertThat(getResult).isNotEqualTo(RocksDB.NOT_FOUND); + assertThat(outValue).isEqualTo("value".getBytes()); + // found value which fits partially + getResult = db.get(columnFamilyHandleList.get(0), new ReadOptions(), + "key2".getBytes(), outValue); + assertThat(getResult).isNotEqualTo(RocksDB.NOT_FOUND); + assertThat(outValue).isEqualTo("12345".getBytes()); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test + public void createWriteDropColumnFamily() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes())); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + columnFamilyHandleList)) { + ColumnFamilyHandle tmpColumnFamilyHandle = null; + try { + tmpColumnFamilyHandle = db.createColumnFamily( + new ColumnFamilyDescriptor("tmpCF".getBytes(), + new ColumnFamilyOptions())); + db.put(tmpColumnFamilyHandle, "key".getBytes(), "value".getBytes()); + db.dropColumnFamily(tmpColumnFamilyHandle); + } finally { + if (tmpColumnFamilyHandle != null) { + tmpColumnFamilyHandle.close(); + } + for (ColumnFamilyHandle columnFamilyHandle : columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test + public void writeBatch() throws RocksDBException { + try (final StringAppendOperator stringAppendOperator = new StringAppendOperator(); + final ColumnFamilyOptions defaultCfOptions = new ColumnFamilyOptions() + .setMergeOperator(stringAppendOperator)) { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, + defaultCfOptions), + new ColumnFamilyDescriptor("new_cf".getBytes())); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), + cfDescriptors, columnFamilyHandleList); + final WriteBatch writeBatch = new WriteBatch(); + final WriteOptions writeOpt = new WriteOptions()) { + try { + writeBatch.put("key".getBytes(), "value".getBytes()); + writeBatch.put(db.getDefaultColumnFamily(), + "mergeKey".getBytes(), "merge".getBytes()); + writeBatch.merge(db.getDefaultColumnFamily(), "mergeKey".getBytes(), + "merge".getBytes()); + writeBatch.put(columnFamilyHandleList.get(1), "newcfkey".getBytes(), + "value".getBytes()); + writeBatch.put(columnFamilyHandleList.get(1), "newcfkey2".getBytes(), + "value2".getBytes()); + writeBatch.remove("xyz".getBytes()); + writeBatch.remove(columnFamilyHandleList.get(1), "xyz".getBytes()); + db.write(writeOpt, writeBatch); + + assertThat(db.get(columnFamilyHandleList.get(1), + "xyz".getBytes()) == null); + assertThat(new String(db.get(columnFamilyHandleList.get(1), + "newcfkey".getBytes()))).isEqualTo("value"); + assertThat(new String(db.get(columnFamilyHandleList.get(1), + "newcfkey2".getBytes()))).isEqualTo("value2"); + assertThat(new String(db.get("key".getBytes()))).isEqualTo("value"); + // check if key is merged + assertThat(new String(db.get(db.getDefaultColumnFamily(), + "mergeKey".getBytes()))).isEqualTo("merge,merge"); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + } + + @Test + public void iteratorOnColumnFamily() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes())); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), + cfDescriptors, columnFamilyHandleList)) { + try { + + db.put(columnFamilyHandleList.get(1), "newcfkey".getBytes(), + "value".getBytes()); + db.put(columnFamilyHandleList.get(1), "newcfkey2".getBytes(), + "value2".getBytes()); + try (final RocksIterator rocksIterator = + db.newIterator(columnFamilyHandleList.get(1))) { + rocksIterator.seekToFirst(); + Map refMap = new HashMap<>(); + refMap.put("newcfkey", "value"); + refMap.put("newcfkey2", "value2"); + int i = 0; + while (rocksIterator.isValid()) { + i++; + assertThat(refMap.get(new String(rocksIterator.key()))). + isEqualTo(new String(rocksIterator.value())); + rocksIterator.next(); + } + assertThat(i).isEqualTo(2); + } + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test + public void multiGet() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes())); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), + cfDescriptors, columnFamilyHandleList)) { + try { + db.put(columnFamilyHandleList.get(0), "key".getBytes(), + "value".getBytes()); + db.put(columnFamilyHandleList.get(1), "newcfkey".getBytes(), + "value".getBytes()); + + final List keys = Arrays.asList(new byte[][]{ + "key".getBytes(), "newcfkey".getBytes() + }); + Map retValues = db.multiGet(columnFamilyHandleList, + keys); + assertThat(retValues.size()).isEqualTo(2); + assertThat(new String(retValues.get(keys.get(0)))) + .isEqualTo("value"); + assertThat(new String(retValues.get(keys.get(1)))) + .isEqualTo("value"); + retValues = db.multiGet(new ReadOptions(), columnFamilyHandleList, + keys); + assertThat(retValues.size()).isEqualTo(2); + assertThat(new String(retValues.get(keys.get(0)))) + .isEqualTo("value"); + assertThat(new String(retValues.get(keys.get(1)))) + .isEqualTo("value"); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test + public void properties() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes())); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), + cfDescriptors, columnFamilyHandleList)) { + try { + assertThat(db.getProperty("rocksdb.estimate-num-keys")). + isNotNull(); + assertThat(db.getLongProperty(columnFamilyHandleList.get(0), + "rocksdb.estimate-num-keys")).isGreaterThanOrEqualTo(0); + assertThat(db.getProperty("rocksdb.stats")).isNotNull(); + assertThat(db.getProperty(columnFamilyHandleList.get(0), + "rocksdb.sstables")).isNotNull(); + assertThat(db.getProperty(columnFamilyHandleList.get(1), + "rocksdb.estimate-num-keys")).isNotNull(); + assertThat(db.getProperty(columnFamilyHandleList.get(1), + "rocksdb.stats")).isNotNull(); + assertThat(db.getProperty(columnFamilyHandleList.get(1), + "rocksdb.sstables")).isNotNull(); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + + @Test + public void iterators() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes())); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + columnFamilyHandleList)) { + List iterators = null; + try { + iterators = db.newIterators(columnFamilyHandleList); + assertThat(iterators.size()).isEqualTo(2); + RocksIterator iter = iterators.get(0); + iter.seekToFirst(); + final Map defRefMap = new HashMap<>(); + defRefMap.put("dfkey1", "dfvalue"); + defRefMap.put("key", "value"); + while (iter.isValid()) { + assertThat(defRefMap.get(new String(iter.key()))). + isEqualTo(new String(iter.value())); + iter.next(); + } + // iterate over new_cf key/value pairs + final Map cfRefMap = new HashMap<>(); + cfRefMap.put("newcfkey", "value"); + cfRefMap.put("newcfkey2", "value2"); + iter = iterators.get(1); + iter.seekToFirst(); + while (iter.isValid()) { + assertThat(cfRefMap.get(new String(iter.key()))). + isEqualTo(new String(iter.value())); + iter.next(); + } + } finally { + if (iterators != null) { + for (final RocksIterator rocksIterator : iterators) { + rocksIterator.close(); + } + } + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test(expected = RocksDBException.class) + public void failPutDisposedCF() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes())); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), + cfDescriptors, columnFamilyHandleList)) { + try { + db.dropColumnFamily(columnFamilyHandleList.get(1)); + db.put(columnFamilyHandleList.get(1), "key".getBytes(), + "value".getBytes()); + } finally { + for (ColumnFamilyHandle columnFamilyHandle : columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test(expected = RocksDBException.class) + public void failRemoveDisposedCF() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes())); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), + cfDescriptors, columnFamilyHandleList)) { + try { + db.dropColumnFamily(columnFamilyHandleList.get(1)); + db.remove(columnFamilyHandleList.get(1), "key".getBytes()); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test(expected = RocksDBException.class) + public void failGetDisposedCF() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes())); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + columnFamilyHandleList)) { + try { + db.dropColumnFamily(columnFamilyHandleList.get(1)); + db.get(columnFamilyHandleList.get(1), "key".getBytes()); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test(expected = RocksDBException.class) + public void failMultiGetWithoutCorrectNumberOfCF() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes())); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + columnFamilyHandleList)) { + try { + final List keys = new ArrayList<>(); + keys.add("key".getBytes()); + keys.add("newcfkey".getBytes()); + final List cfCustomList = new ArrayList<>(); + db.multiGet(cfCustomList, keys); + + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test + public void testByteCreateFolumnFamily() throws RocksDBException { + + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath()) + ) { + final byte[] b0 = new byte[]{(byte) 0x00}; + final byte[] b1 = new byte[]{(byte) 0x01}; + final byte[] b2 = new byte[]{(byte) 0x02}; + ColumnFamilyHandle cf1 = null, cf2 = null, cf3 = null; + try { + cf1 = db.createColumnFamily(new ColumnFamilyDescriptor(b0)); + cf2 = db.createColumnFamily(new ColumnFamilyDescriptor(b1)); + final List families = RocksDB.listColumnFamilies(options, + dbFolder.getRoot().getAbsolutePath()); + assertThat(families).contains("default".getBytes(), b0, b1); + cf3 = db.createColumnFamily(new ColumnFamilyDescriptor(b2)); + } finally { + if (cf1 != null) { + cf1.close(); + } + if (cf2 != null) { + cf2.close(); + } + if (cf3 != null) { + cf3.close(); + } + } + } + } + + @Test + public void testCFNamesWithZeroBytes() throws RocksDBException { + ColumnFamilyHandle cf1 = null, cf2 = null; + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath()); + ) { + try { + final byte[] b0 = new byte[]{0, 0}; + final byte[] b1 = new byte[]{0, 1}; + cf1 = db.createColumnFamily(new ColumnFamilyDescriptor(b0)); + cf2 = db.createColumnFamily(new ColumnFamilyDescriptor(b1)); + final List families = RocksDB.listColumnFamilies(options, + dbFolder.getRoot().getAbsolutePath()); + assertThat(families).contains("default".getBytes(), b0, b1); + } finally { + if (cf1 != null) { + cf1.close(); + } + if (cf2 != null) { + cf2.close(); + } + } + } + } + + @Test + public void testCFNameSimplifiedChinese() throws RocksDBException { + ColumnFamilyHandle columnFamilyHandle = null; + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath()); + ) { + try { + final String simplifiedChinese = "\u7b80\u4f53\u5b57"; + columnFamilyHandle = db.createColumnFamily( + new ColumnFamilyDescriptor(simplifiedChinese.getBytes())); + + final List families = RocksDB.listColumnFamilies(options, + dbFolder.getRoot().getAbsolutePath()); + assertThat(families).contains("default".getBytes(), + simplifiedChinese.getBytes()); + } finally { + if (columnFamilyHandle != null) { + columnFamilyHandle.close(); + } + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/CompactionOptionsFIFOTest.java b/java/src/test/java/org/rocksdb/CompactionOptionsFIFOTest.java new file mode 100644 index 00000000000..370a28e8196 --- /dev/null +++ b/java/src/test/java/org/rocksdb/CompactionOptionsFIFOTest.java @@ -0,0 +1,26 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class CompactionOptionsFIFOTest { + + static { + RocksDB.loadLibrary(); + } + + @Test + public void maxTableFilesSize() { + final long size = 500 * 1024 * 1026; + try(final CompactionOptionsFIFO opt = new CompactionOptionsFIFO()) { + opt.setMaxTableFilesSize(size); + assertThat(opt.maxTableFilesSize()).isEqualTo(size); + } + } +} diff --git a/java/src/test/java/org/rocksdb/CompactionOptionsUniversalTest.java b/java/src/test/java/org/rocksdb/CompactionOptionsUniversalTest.java new file mode 100644 index 00000000000..5e2d195b6e5 --- /dev/null +++ b/java/src/test/java/org/rocksdb/CompactionOptionsUniversalTest.java @@ -0,0 +1,80 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class CompactionOptionsUniversalTest { + + static { + RocksDB.loadLibrary(); + } + + @Test + public void sizeRatio() { + final int sizeRatio = 4; + try(final CompactionOptionsUniversal opt = new CompactionOptionsUniversal()) { + opt.setSizeRatio(sizeRatio); + assertThat(opt.sizeRatio()).isEqualTo(sizeRatio); + } + } + + @Test + public void minMergeWidth() { + final int minMergeWidth = 3; + try(final CompactionOptionsUniversal opt = new CompactionOptionsUniversal()) { + opt.setMinMergeWidth(minMergeWidth); + assertThat(opt.minMergeWidth()).isEqualTo(minMergeWidth); + } + } + + @Test + public void maxMergeWidth() { + final int maxMergeWidth = Integer.MAX_VALUE - 1234; + try(final CompactionOptionsUniversal opt = new CompactionOptionsUniversal()) { + opt.setMaxMergeWidth(maxMergeWidth); + assertThat(opt.maxMergeWidth()).isEqualTo(maxMergeWidth); + } + } + + @Test + public void maxSizeAmplificationPercent() { + final int maxSizeAmplificationPercent = 150; + try(final CompactionOptionsUniversal opt = new CompactionOptionsUniversal()) { + opt.setMaxSizeAmplificationPercent(maxSizeAmplificationPercent); + assertThat(opt.maxSizeAmplificationPercent()).isEqualTo(maxSizeAmplificationPercent); + } + } + + @Test + public void compressionSizePercent() { + final int compressionSizePercent = 500; + try(final CompactionOptionsUniversal opt = new CompactionOptionsUniversal()) { + opt.setCompressionSizePercent(compressionSizePercent); + assertThat(opt.compressionSizePercent()).isEqualTo(compressionSizePercent); + } + } + + @Test + public void stopStyle() { + final CompactionStopStyle stopStyle = CompactionStopStyle.CompactionStopStyleSimilarSize; + try(final CompactionOptionsUniversal opt = new CompactionOptionsUniversal()) { + opt.setStopStyle(stopStyle); + assertThat(opt.stopStyle()).isEqualTo(stopStyle); + } + } + + @Test + public void allowTrivialMove() { + final boolean allowTrivialMove = true; + try(final CompactionOptionsUniversal opt = new CompactionOptionsUniversal()) { + opt.setAllowTrivialMove(allowTrivialMove); + assertThat(opt.allowTrivialMove()).isEqualTo(allowTrivialMove); + } + } +} diff --git a/java/src/test/java/org/rocksdb/CompactionPriorityTest.java b/java/src/test/java/org/rocksdb/CompactionPriorityTest.java new file mode 100644 index 00000000000..b078e132f96 --- /dev/null +++ b/java/src/test/java/org/rocksdb/CompactionPriorityTest.java @@ -0,0 +1,31 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class CompactionPriorityTest { + + @Test(expected = IllegalArgumentException.class) + public void failIfIllegalByteValueProvided() { + CompactionPriority.getCompactionPriority((byte) -1); + } + + @Test + public void getCompactionPriority() { + assertThat(CompactionPriority.getCompactionPriority( + CompactionPriority.OldestLargestSeqFirst.getValue())) + .isEqualTo(CompactionPriority.OldestLargestSeqFirst); + } + + @Test + public void valueOf() { + assertThat(CompactionPriority.valueOf("OldestSmallestSeqFirst")). + isEqualTo(CompactionPriority.OldestSmallestSeqFirst); + } +} diff --git a/java/src/test/java/org/rocksdb/CompactionStopStyleTest.java b/java/src/test/java/org/rocksdb/CompactionStopStyleTest.java new file mode 100644 index 00000000000..4c8a20950c8 --- /dev/null +++ b/java/src/test/java/org/rocksdb/CompactionStopStyleTest.java @@ -0,0 +1,31 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class CompactionStopStyleTest { + + @Test(expected = IllegalArgumentException.class) + public void failIfIllegalByteValueProvided() { + CompactionStopStyle.getCompactionStopStyle((byte) -1); + } + + @Test + public void getCompactionStopStyle() { + assertThat(CompactionStopStyle.getCompactionStopStyle( + CompactionStopStyle.CompactionStopStyleTotalSize.getValue())) + .isEqualTo(CompactionStopStyle.CompactionStopStyleTotalSize); + } + + @Test + public void valueOf() { + assertThat(CompactionStopStyle.valueOf("CompactionStopStyleSimilarSize")). + isEqualTo(CompactionStopStyle.CompactionStopStyleSimilarSize); + } +} diff --git a/java/src/test/java/org/rocksdb/ComparatorOptionsTest.java b/java/src/test/java/org/rocksdb/ComparatorOptionsTest.java new file mode 100644 index 00000000000..a45c7173c27 --- /dev/null +++ b/java/src/test/java/org/rocksdb/ComparatorOptionsTest.java @@ -0,0 +1,32 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class ComparatorOptionsTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void comparatorOptions() { + try(final ComparatorOptions copt = new ComparatorOptions()) { + + assertThat(copt).isNotNull(); + // UseAdaptiveMutex test + copt.setUseAdaptiveMutex(true); + assertThat(copt.useAdaptiveMutex()).isTrue(); + + copt.setUseAdaptiveMutex(false); + assertThat(copt.useAdaptiveMutex()).isFalse(); + } + } +} diff --git a/java/src/test/java/org/rocksdb/ComparatorTest.java b/java/src/test/java/org/rocksdb/ComparatorTest.java new file mode 100644 index 00000000000..63dee7257d3 --- /dev/null +++ b/java/src/test/java/org/rocksdb/ComparatorTest.java @@ -0,0 +1,200 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.io.IOException; +import java.nio.file.FileSystems; + +import static org.assertj.core.api.Assertions.assertThat; + +public class ComparatorTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void javaComparator() throws IOException, RocksDBException { + + final AbstractComparatorTest comparatorTest = new AbstractComparatorTest() { + @Override + public AbstractComparator getAscendingIntKeyComparator() { + return new Comparator(new ComparatorOptions()) { + + @Override + public String name() { + return "test.AscendingIntKeyComparator"; + } + + @Override + public int compare(final Slice a, final Slice b) { + return compareIntKeys(a.data(), b.data()); + } + }; + } + }; + + // test the round-tripability of keys written and read with the Comparator + comparatorTest.testRoundtrip(FileSystems.getDefault().getPath( + dbFolder.getRoot().getAbsolutePath())); + } + + @Test + public void javaComparatorCf() throws IOException, RocksDBException { + + final AbstractComparatorTest comparatorTest = new AbstractComparatorTest() { + @Override + public AbstractComparator getAscendingIntKeyComparator() { + return new Comparator(new ComparatorOptions()) { + + @Override + public String name() { + return "test.AscendingIntKeyComparator"; + } + + @Override + public int compare(final Slice a, final Slice b) { + return compareIntKeys(a.data(), b.data()); + } + }; + } + }; + + // test the round-tripability of keys written and read with the Comparator + comparatorTest.testRoundtripCf(FileSystems.getDefault().getPath( + dbFolder.getRoot().getAbsolutePath())); + } + + @Test + public void builtinForwardComparator() + throws RocksDBException { + try (final Options options = new Options() + .setCreateIfMissing(true) + .setComparator(BuiltinComparator.BYTEWISE_COMPARATOR); + final RocksDB rocksDb = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath()) + ) { + rocksDb.put("abc1".getBytes(), "abc1".getBytes()); + rocksDb.put("abc2".getBytes(), "abc2".getBytes()); + rocksDb.put("abc3".getBytes(), "abc3".getBytes()); + + try(final RocksIterator rocksIterator = rocksDb.newIterator()) { + // Iterate over keys using a iterator + rocksIterator.seekToFirst(); + assertThat(rocksIterator.isValid()).isTrue(); + assertThat(rocksIterator.key()).isEqualTo( + "abc1".getBytes()); + assertThat(rocksIterator.value()).isEqualTo( + "abc1".getBytes()); + rocksIterator.next(); + assertThat(rocksIterator.isValid()).isTrue(); + assertThat(rocksIterator.key()).isEqualTo( + "abc2".getBytes()); + assertThat(rocksIterator.value()).isEqualTo( + "abc2".getBytes()); + rocksIterator.next(); + assertThat(rocksIterator.isValid()).isTrue(); + assertThat(rocksIterator.key()).isEqualTo( + "abc3".getBytes()); + assertThat(rocksIterator.value()).isEqualTo( + "abc3".getBytes()); + rocksIterator.next(); + assertThat(rocksIterator.isValid()).isFalse(); + // Get last one + rocksIterator.seekToLast(); + assertThat(rocksIterator.isValid()).isTrue(); + assertThat(rocksIterator.key()).isEqualTo( + "abc3".getBytes()); + assertThat(rocksIterator.value()).isEqualTo( + "abc3".getBytes()); + // Seek for abc + rocksIterator.seek("abc".getBytes()); + assertThat(rocksIterator.isValid()).isTrue(); + assertThat(rocksIterator.key()).isEqualTo( + "abc1".getBytes()); + assertThat(rocksIterator.value()).isEqualTo( + "abc1".getBytes()); + } + } + } + + @Test + public void builtinReverseComparator() + throws RocksDBException { + try (final Options options = new Options() + .setCreateIfMissing(true) + .setComparator(BuiltinComparator.REVERSE_BYTEWISE_COMPARATOR); + final RocksDB rocksDb = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath()) + ) { + + rocksDb.put("abc1".getBytes(), "abc1".getBytes()); + rocksDb.put("abc2".getBytes(), "abc2".getBytes()); + rocksDb.put("abc3".getBytes(), "abc3".getBytes()); + + try (final RocksIterator rocksIterator = rocksDb.newIterator()) { + // Iterate over keys using a iterator + rocksIterator.seekToFirst(); + assertThat(rocksIterator.isValid()).isTrue(); + assertThat(rocksIterator.key()).isEqualTo( + "abc3".getBytes()); + assertThat(rocksIterator.value()).isEqualTo( + "abc3".getBytes()); + rocksIterator.next(); + assertThat(rocksIterator.isValid()).isTrue(); + assertThat(rocksIterator.key()).isEqualTo( + "abc2".getBytes()); + assertThat(rocksIterator.value()).isEqualTo( + "abc2".getBytes()); + rocksIterator.next(); + assertThat(rocksIterator.isValid()).isTrue(); + assertThat(rocksIterator.key()).isEqualTo( + "abc1".getBytes()); + assertThat(rocksIterator.value()).isEqualTo( + "abc1".getBytes()); + rocksIterator.next(); + assertThat(rocksIterator.isValid()).isFalse(); + // Get last one + rocksIterator.seekToLast(); + assertThat(rocksIterator.isValid()).isTrue(); + assertThat(rocksIterator.key()).isEqualTo( + "abc1".getBytes()); + assertThat(rocksIterator.value()).isEqualTo( + "abc1".getBytes()); + // Will be invalid because abc is after abc1 + rocksIterator.seek("abc".getBytes()); + assertThat(rocksIterator.isValid()).isFalse(); + // Will be abc3 because the next one after abc999 + // is abc3 + rocksIterator.seek("abc999".getBytes()); + assertThat(rocksIterator.key()).isEqualTo( + "abc3".getBytes()); + assertThat(rocksIterator.value()).isEqualTo( + "abc3".getBytes()); + } + } + } + + @Test + public void builtinComparatorEnum(){ + assertThat(BuiltinComparator.BYTEWISE_COMPARATOR.ordinal()) + .isEqualTo(0); + assertThat( + BuiltinComparator.REVERSE_BYTEWISE_COMPARATOR.ordinal()) + .isEqualTo(1); + assertThat(BuiltinComparator.values().length).isEqualTo(2); + assertThat(BuiltinComparator.valueOf("BYTEWISE_COMPARATOR")). + isEqualTo(BuiltinComparator.BYTEWISE_COMPARATOR); + } +} diff --git a/java/src/test/java/org/rocksdb/CompressionOptionsTest.java b/java/src/test/java/org/rocksdb/CompressionOptionsTest.java new file mode 100644 index 00000000000..c49224ca36b --- /dev/null +++ b/java/src/test/java/org/rocksdb/CompressionOptionsTest.java @@ -0,0 +1,53 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class CompressionOptionsTest { + + static { + RocksDB.loadLibrary(); + } + + @Test + public void windowBits() { + final int windowBits = 7; + try(final CompressionOptions opt = new CompressionOptions()) { + opt.setWindowBits(windowBits); + assertThat(opt.windowBits()).isEqualTo(windowBits); + } + } + + @Test + public void level() { + final int level = 6; + try(final CompressionOptions opt = new CompressionOptions()) { + opt.setLevel(level); + assertThat(opt.level()).isEqualTo(level); + } + } + + @Test + public void strategy() { + final int strategy = 2; + try(final CompressionOptions opt = new CompressionOptions()) { + opt.setStrategy(strategy); + assertThat(opt.strategy()).isEqualTo(strategy); + } + } + + @Test + public void maxDictBytes() { + final int maxDictBytes = 999; + try(final CompressionOptions opt = new CompressionOptions()) { + opt.setMaxDictBytes(maxDictBytes); + assertThat(opt.maxDictBytes()).isEqualTo(maxDictBytes); + } + } +} diff --git a/java/src/test/java/org/rocksdb/CompressionTypesTest.java b/java/src/test/java/org/rocksdb/CompressionTypesTest.java new file mode 100644 index 00000000000..e26cc0aca0f --- /dev/null +++ b/java/src/test/java/org/rocksdb/CompressionTypesTest.java @@ -0,0 +1,20 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.Test; + + +public class CompressionTypesTest { + @Test + public void getCompressionType() { + for (final CompressionType compressionType : CompressionType.values()) { + String libraryName = compressionType.getLibraryName(); + compressionType.equals(CompressionType.getCompressionType( + libraryName)); + } + } +} diff --git a/java/src/test/java/org/rocksdb/DBOptionsTest.java b/java/src/test/java/org/rocksdb/DBOptionsTest.java new file mode 100644 index 00000000000..11b7435d8a9 --- /dev/null +++ b/java/src/test/java/org/rocksdb/DBOptionsTest.java @@ -0,0 +1,637 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import java.nio.file.Paths; +import java.util.*; + +import static org.assertj.core.api.Assertions.assertThat; + +public class DBOptionsTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + public static final Random rand = PlatformRandomHelper. + getPlatformSpecificRandomFactory(); + + @Test + public void getDBOptionsFromProps() { + // setup sample properties + final Properties properties = new Properties(); + properties.put("allow_mmap_reads", "true"); + properties.put("bytes_per_sync", "13"); + try(final DBOptions opt = DBOptions.getDBOptionsFromProps(properties)) { + assertThat(opt).isNotNull(); + assertThat(String.valueOf(opt.allowMmapReads())). + isEqualTo(properties.get("allow_mmap_reads")); + assertThat(String.valueOf(opt.bytesPerSync())). + isEqualTo(properties.get("bytes_per_sync")); + } + } + + @Test + public void failDBOptionsFromPropsWithIllegalValue() { + // setup sample properties + final Properties properties = new Properties(); + properties.put("tomato", "1024"); + properties.put("burger", "2"); + try(final DBOptions opt = DBOptions.getDBOptionsFromProps(properties)) { + assertThat(opt).isNull(); + } + } + + @Test(expected = IllegalArgumentException.class) + public void failDBOptionsFromPropsWithNullValue() { + try(final DBOptions opt = DBOptions.getDBOptionsFromProps(null)) { + //no-op + } + } + + @Test(expected = IllegalArgumentException.class) + public void failDBOptionsFromPropsWithEmptyProps() { + try(final DBOptions opt = DBOptions.getDBOptionsFromProps( + new Properties())) { + //no-op + } + } + + @Test + public void linkageOfPrepMethods() { + try (final DBOptions opt = new DBOptions()) { + opt.optimizeForSmallDb(); + } + } + + @Test + public void env() { + try (final DBOptions opt = new DBOptions(); + final Env env = Env.getDefault()) { + opt.setEnv(env); + assertThat(opt.getEnv()).isSameAs(env); + } + } + + @Test + public void setIncreaseParallelism() { + try(final DBOptions opt = new DBOptions()) { + final int threads = Runtime.getRuntime().availableProcessors() * 2; + opt.setIncreaseParallelism(threads); + } + } + + @Test + public void createIfMissing() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setCreateIfMissing(boolValue); + assertThat(opt.createIfMissing()).isEqualTo(boolValue); + } + } + + @Test + public void createMissingColumnFamilies() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setCreateMissingColumnFamilies(boolValue); + assertThat(opt.createMissingColumnFamilies()).isEqualTo(boolValue); + } + } + + @Test + public void errorIfExists() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setErrorIfExists(boolValue); + assertThat(opt.errorIfExists()).isEqualTo(boolValue); + } + } + + @Test + public void paranoidChecks() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setParanoidChecks(boolValue); + assertThat(opt.paranoidChecks()).isEqualTo(boolValue); + } + } + + @Test + public void maxTotalWalSize() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setMaxTotalWalSize(longValue); + assertThat(opt.maxTotalWalSize()).isEqualTo(longValue); + } + } + + @Test + public void maxOpenFiles() { + try(final DBOptions opt = new DBOptions()) { + final int intValue = rand.nextInt(); + opt.setMaxOpenFiles(intValue); + assertThat(opt.maxOpenFiles()).isEqualTo(intValue); + } + } + + @Test + public void maxFileOpeningThreads() { + try(final DBOptions opt = new DBOptions()) { + final int intValue = rand.nextInt(); + opt.setMaxFileOpeningThreads(intValue); + assertThat(opt.maxFileOpeningThreads()).isEqualTo(intValue); + } + } + + @Test + public void useFsync() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setUseFsync(boolValue); + assertThat(opt.useFsync()).isEqualTo(boolValue); + } + } + + @Test + public void dbPaths() { + final List dbPaths = new ArrayList<>(); + dbPaths.add(new DbPath(Paths.get("/a"), 10)); + dbPaths.add(new DbPath(Paths.get("/b"), 100)); + dbPaths.add(new DbPath(Paths.get("/c"), 1000)); + + try(final DBOptions opt = new DBOptions()) { + assertThat(opt.dbPaths()).isEqualTo(Collections.emptyList()); + + opt.setDbPaths(dbPaths); + + assertThat(opt.dbPaths()).isEqualTo(dbPaths); + } + } + + @Test + public void dbLogDir() { + try(final DBOptions opt = new DBOptions()) { + final String str = "path/to/DbLogDir"; + opt.setDbLogDir(str); + assertThat(opt.dbLogDir()).isEqualTo(str); + } + } + + @Test + public void walDir() { + try(final DBOptions opt = new DBOptions()) { + final String str = "path/to/WalDir"; + opt.setWalDir(str); + assertThat(opt.walDir()).isEqualTo(str); + } + } + + @Test + public void deleteObsoleteFilesPeriodMicros() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setDeleteObsoleteFilesPeriodMicros(longValue); + assertThat(opt.deleteObsoleteFilesPeriodMicros()).isEqualTo(longValue); + } + } + + @Test + public void baseBackgroundCompactions() { + try (final DBOptions opt = new DBOptions()) { + final int intValue = rand.nextInt(); + opt.setBaseBackgroundCompactions(intValue); + assertThat(opt.baseBackgroundCompactions()). + isEqualTo(intValue); + } + } + + @Test + public void maxBackgroundCompactions() { + try(final DBOptions opt = new DBOptions()) { + final int intValue = rand.nextInt(); + opt.setMaxBackgroundCompactions(intValue); + assertThat(opt.maxBackgroundCompactions()).isEqualTo(intValue); + } + } + + @Test + public void maxSubcompactions() { + try (final DBOptions opt = new DBOptions()) { + final int intValue = rand.nextInt(); + opt.setMaxSubcompactions(intValue); + assertThat(opt.maxSubcompactions()). + isEqualTo(intValue); + } + } + + @Test + public void maxBackgroundFlushes() { + try(final DBOptions opt = new DBOptions()) { + final int intValue = rand.nextInt(); + opt.setMaxBackgroundFlushes(intValue); + assertThat(opt.maxBackgroundFlushes()).isEqualTo(intValue); + } + } + + @Test + public void maxLogFileSize() throws RocksDBException { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setMaxLogFileSize(longValue); + assertThat(opt.maxLogFileSize()).isEqualTo(longValue); + } + } + + @Test + public void logFileTimeToRoll() throws RocksDBException { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setLogFileTimeToRoll(longValue); + assertThat(opt.logFileTimeToRoll()).isEqualTo(longValue); + } + } + + @Test + public void keepLogFileNum() throws RocksDBException { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setKeepLogFileNum(longValue); + assertThat(opt.keepLogFileNum()).isEqualTo(longValue); + } + } + + @Test + public void recycleLogFileNum() throws RocksDBException { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setRecycleLogFileNum(longValue); + assertThat(opt.recycleLogFileNum()).isEqualTo(longValue); + } + } + + @Test + public void maxManifestFileSize() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setMaxManifestFileSize(longValue); + assertThat(opt.maxManifestFileSize()).isEqualTo(longValue); + } + } + + @Test + public void tableCacheNumshardbits() { + try(final DBOptions opt = new DBOptions()) { + final int intValue = rand.nextInt(); + opt.setTableCacheNumshardbits(intValue); + assertThat(opt.tableCacheNumshardbits()).isEqualTo(intValue); + } + } + + @Test + public void walSizeLimitMB() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setWalSizeLimitMB(longValue); + assertThat(opt.walSizeLimitMB()).isEqualTo(longValue); + } + } + + @Test + public void walTtlSeconds() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setWalTtlSeconds(longValue); + assertThat(opt.walTtlSeconds()).isEqualTo(longValue); + } + } + + @Test + public void manifestPreallocationSize() throws RocksDBException { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setManifestPreallocationSize(longValue); + assertThat(opt.manifestPreallocationSize()).isEqualTo(longValue); + } + } + + @Test + public void useDirectReads() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setUseDirectReads(boolValue); + assertThat(opt.useDirectReads()).isEqualTo(boolValue); + } + } + + @Test + public void useDirectIoForFlushAndCompaction() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setUseDirectIoForFlushAndCompaction(boolValue); + assertThat(opt.useDirectIoForFlushAndCompaction()).isEqualTo(boolValue); + } + } + + @Test + public void allowFAllocate() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAllowFAllocate(boolValue); + assertThat(opt.allowFAllocate()).isEqualTo(boolValue); + } + } + + @Test + public void allowMmapReads() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAllowMmapReads(boolValue); + assertThat(opt.allowMmapReads()).isEqualTo(boolValue); + } + } + + @Test + public void allowMmapWrites() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAllowMmapWrites(boolValue); + assertThat(opt.allowMmapWrites()).isEqualTo(boolValue); + } + } + + @Test + public void isFdCloseOnExec() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setIsFdCloseOnExec(boolValue); + assertThat(opt.isFdCloseOnExec()).isEqualTo(boolValue); + } + } + + @Test + public void statsDumpPeriodSec() { + try(final DBOptions opt = new DBOptions()) { + final int intValue = rand.nextInt(); + opt.setStatsDumpPeriodSec(intValue); + assertThat(opt.statsDumpPeriodSec()).isEqualTo(intValue); + } + } + + @Test + public void adviseRandomOnOpen() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAdviseRandomOnOpen(boolValue); + assertThat(opt.adviseRandomOnOpen()).isEqualTo(boolValue); + } + } + + @Test + public void dbWriteBufferSize() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setDbWriteBufferSize(longValue); + assertThat(opt.dbWriteBufferSize()).isEqualTo(longValue); + } + } + + @Test + public void accessHintOnCompactionStart() { + try(final DBOptions opt = new DBOptions()) { + final AccessHint accessHint = AccessHint.SEQUENTIAL; + opt.setAccessHintOnCompactionStart(accessHint); + assertThat(opt.accessHintOnCompactionStart()).isEqualTo(accessHint); + } + } + + @Test + public void newTableReaderForCompactionInputs() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setNewTableReaderForCompactionInputs(boolValue); + assertThat(opt.newTableReaderForCompactionInputs()).isEqualTo(boolValue); + } + } + + @Test + public void compactionReadaheadSize() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setCompactionReadaheadSize(longValue); + assertThat(opt.compactionReadaheadSize()).isEqualTo(longValue); + } + } + + @Test + public void randomAccessMaxBufferSize() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setRandomAccessMaxBufferSize(longValue); + assertThat(opt.randomAccessMaxBufferSize()).isEqualTo(longValue); + } + } + + @Test + public void writableFileMaxBufferSize() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setWritableFileMaxBufferSize(longValue); + assertThat(opt.writableFileMaxBufferSize()).isEqualTo(longValue); + } + } + + @Test + public void useAdaptiveMutex() { + try(final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setUseAdaptiveMutex(boolValue); + assertThat(opt.useAdaptiveMutex()).isEqualTo(boolValue); + } + } + + @Test + public void bytesPerSync() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setBytesPerSync(longValue); + assertThat(opt.bytesPerSync()).isEqualTo(longValue); + } + } + + @Test + public void walBytesPerSync() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setWalBytesPerSync(longValue); + assertThat(opt.walBytesPerSync()).isEqualTo(longValue); + } + } + + @Test + public void enableThreadTracking() { + try (final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setEnableThreadTracking(boolValue); + assertThat(opt.enableThreadTracking()).isEqualTo(boolValue); + } + } + + @Test + public void delayedWriteRate() { + try(final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setDelayedWriteRate(longValue); + assertThat(opt.delayedWriteRate()).isEqualTo(longValue); + } + } + + @Test + public void allowConcurrentMemtableWrite() { + try (final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAllowConcurrentMemtableWrite(boolValue); + assertThat(opt.allowConcurrentMemtableWrite()).isEqualTo(boolValue); + } + } + + @Test + public void enableWriteThreadAdaptiveYield() { + try (final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setEnableWriteThreadAdaptiveYield(boolValue); + assertThat(opt.enableWriteThreadAdaptiveYield()).isEqualTo(boolValue); + } + } + + @Test + public void writeThreadMaxYieldUsec() { + try (final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setWriteThreadMaxYieldUsec(longValue); + assertThat(opt.writeThreadMaxYieldUsec()).isEqualTo(longValue); + } + } + + @Test + public void writeThreadSlowYieldUsec() { + try (final DBOptions opt = new DBOptions()) { + final long longValue = rand.nextLong(); + opt.setWriteThreadSlowYieldUsec(longValue); + assertThat(opt.writeThreadSlowYieldUsec()).isEqualTo(longValue); + } + } + + @Test + public void skipStatsUpdateOnDbOpen() { + try (final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setSkipStatsUpdateOnDbOpen(boolValue); + assertThat(opt.skipStatsUpdateOnDbOpen()).isEqualTo(boolValue); + } + } + + @Test + public void walRecoveryMode() { + try (final DBOptions opt = new DBOptions()) { + for (final WALRecoveryMode walRecoveryMode : WALRecoveryMode.values()) { + opt.setWalRecoveryMode(walRecoveryMode); + assertThat(opt.walRecoveryMode()).isEqualTo(walRecoveryMode); + } + } + } + + @Test + public void allow2pc() { + try (final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAllow2pc(boolValue); + assertThat(opt.allow2pc()).isEqualTo(boolValue); + } + } + + @Test + public void rowCache() { + try (final DBOptions opt = new DBOptions()) { + assertThat(opt.rowCache()).isNull(); + + try(final Cache lruCache = new LRUCache(1000)) { + opt.setRowCache(lruCache); + assertThat(opt.rowCache()).isEqualTo(lruCache); + } + + try(final Cache clockCache = new ClockCache(1000)) { + opt.setRowCache(clockCache); + assertThat(opt.rowCache()).isEqualTo(clockCache); + } + } + } + + @Test + public void failIfOptionsFileError() { + try (final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setFailIfOptionsFileError(boolValue); + assertThat(opt.failIfOptionsFileError()).isEqualTo(boolValue); + } + } + + @Test + public void dumpMallocStats() { + try (final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setDumpMallocStats(boolValue); + assertThat(opt.dumpMallocStats()).isEqualTo(boolValue); + } + } + + @Test + public void avoidFlushDuringRecovery() { + try (final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAvoidFlushDuringRecovery(boolValue); + assertThat(opt.avoidFlushDuringRecovery()).isEqualTo(boolValue); + } + } + + @Test + public void avoidFlushDuringShutdown() { + try (final DBOptions opt = new DBOptions()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAvoidFlushDuringShutdown(boolValue); + assertThat(opt.avoidFlushDuringShutdown()).isEqualTo(boolValue); + } + } + + @Test + public void rateLimiter() { + try(final DBOptions options = new DBOptions(); + final DBOptions anotherOptions = new DBOptions(); + final RateLimiter rateLimiter = new RateLimiter(1000, 100 * 1000, 1)) { + options.setRateLimiter(rateLimiter); + // Test with parameter initialization + anotherOptions.setRateLimiter( + new RateLimiter(1000)); + } + } + + @Test + public void statistics() { + try(final DBOptions options = new DBOptions()) { + final Statistics statistics = options.statistics(); + assertThat(statistics).isNull(); + } + + try(final Statistics statistics = new Statistics(); + final DBOptions options = new DBOptions().setStatistics(statistics); + final Statistics stats = options.statistics()) { + assertThat(stats).isNotNull(); + } + } +} diff --git a/java/src/test/java/org/rocksdb/DirectComparatorTest.java b/java/src/test/java/org/rocksdb/DirectComparatorTest.java new file mode 100644 index 00000000000..9b593d05651 --- /dev/null +++ b/java/src/test/java/org/rocksdb/DirectComparatorTest.java @@ -0,0 +1,52 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.io.IOException; +import java.nio.file.FileSystems; + +public class DirectComparatorTest { + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void directComparator() throws IOException, RocksDBException { + + final AbstractComparatorTest comparatorTest = new AbstractComparatorTest() { + @Override + public AbstractComparator getAscendingIntKeyComparator() { + return new DirectComparator(new ComparatorOptions()) { + + @Override + public String name() { + return "test.AscendingIntKeyDirectComparator"; + } + + @Override + public int compare(final DirectSlice a, final DirectSlice b) { + final byte ax[] = new byte[4], bx[] = new byte[4]; + a.data().get(ax); + b.data().get(bx); + return compareIntKeys(ax, bx); + } + }; + } + }; + + // test the round-tripability of keys written and read with the DirectComparator + comparatorTest.testRoundtrip(FileSystems.getDefault().getPath( + dbFolder.getRoot().getAbsolutePath())); + } +} diff --git a/java/src/test/java/org/rocksdb/DirectSliceTest.java b/java/src/test/java/org/rocksdb/DirectSliceTest.java new file mode 100644 index 00000000000..48ae52afd66 --- /dev/null +++ b/java/src/test/java/org/rocksdb/DirectSliceTest.java @@ -0,0 +1,93 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import java.nio.ByteBuffer; + +import static org.assertj.core.api.Assertions.assertThat; + +public class DirectSliceTest { + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void directSlice() { + try(final DirectSlice directSlice = new DirectSlice("abc"); + final DirectSlice otherSlice = new DirectSlice("abc")) { + assertThat(directSlice.toString()).isEqualTo("abc"); + // clear first slice + directSlice.clear(); + assertThat(directSlice.toString()).isEmpty(); + // get first char in otherslice + assertThat(otherSlice.get(0)).isEqualTo("a".getBytes()[0]); + // remove prefix + otherSlice.removePrefix(1); + assertThat(otherSlice.toString()).isEqualTo("bc"); + } + } + + @Test + public void directSliceWithByteBuffer() { + final byte[] data = "Some text".getBytes(); + final ByteBuffer buffer = ByteBuffer.allocateDirect(data.length + 1); + buffer.put(data); + buffer.put(data.length, (byte)0); + + try(final DirectSlice directSlice = new DirectSlice(buffer)) { + assertThat(directSlice.toString()).isEqualTo("Some text"); + } + } + + @Test + public void directSliceWithByteBufferAndLength() { + final byte[] data = "Some text".getBytes(); + final ByteBuffer buffer = ByteBuffer.allocateDirect(data.length); + buffer.put(data); + try(final DirectSlice directSlice = new DirectSlice(buffer, 4)) { + assertThat(directSlice.toString()).isEqualTo("Some"); + } + } + + @Test(expected = IllegalArgumentException.class) + public void directSliceInitWithoutDirectAllocation() { + final byte[] data = "Some text".getBytes(); + final ByteBuffer buffer = ByteBuffer.wrap(data); + try(final DirectSlice directSlice = new DirectSlice(buffer)) { + //no-op + } + } + + @Test(expected = IllegalArgumentException.class) + public void directSlicePrefixInitWithoutDirectAllocation() { + final byte[] data = "Some text".getBytes(); + final ByteBuffer buffer = ByteBuffer.wrap(data); + try(final DirectSlice directSlice = new DirectSlice(buffer, 4)) { + //no-op + } + } + + @Test + public void directSliceClear() { + try(final DirectSlice directSlice = new DirectSlice("abc")) { + assertThat(directSlice.toString()).isEqualTo("abc"); + directSlice.clear(); + assertThat(directSlice.toString()).isEmpty(); + directSlice.clear(); // make sure we don't double-free + } + } + + @Test + public void directSliceRemovePrefix() { + try(final DirectSlice directSlice = new DirectSlice("abc")) { + assertThat(directSlice.toString()).isEqualTo("abc"); + directSlice.removePrefix(1); + assertThat(directSlice.toString()).isEqualTo("bc"); + } + } +} diff --git a/java/src/test/java/org/rocksdb/EnvOptionsTest.java b/java/src/test/java/org/rocksdb/EnvOptionsTest.java new file mode 100644 index 00000000000..9933b1e1dbe --- /dev/null +++ b/java/src/test/java/org/rocksdb/EnvOptionsTest.java @@ -0,0 +1,133 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import java.util.Random; + +import static org.assertj.core.api.Assertions.assertThat; + +public class EnvOptionsTest { + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = new RocksMemoryResource(); + + public static final Random rand = PlatformRandomHelper.getPlatformSpecificRandomFactory(); + + @Test + public void useMmapReads() { + try (final EnvOptions envOptions = new EnvOptions()) { + final boolean boolValue = rand.nextBoolean(); + envOptions.setUseMmapReads(boolValue); + assertThat(envOptions.useMmapReads()).isEqualTo(boolValue); + } + } + + @Test + public void useMmapWrites() { + try (final EnvOptions envOptions = new EnvOptions()) { + final boolean boolValue = rand.nextBoolean(); + envOptions.setUseMmapWrites(boolValue); + assertThat(envOptions.useMmapWrites()).isEqualTo(boolValue); + } + } + + @Test + public void useDirectReads() { + try (final EnvOptions envOptions = new EnvOptions()) { + final boolean boolValue = rand.nextBoolean(); + envOptions.setUseDirectReads(boolValue); + assertThat(envOptions.useDirectReads()).isEqualTo(boolValue); + } + } + + @Test + public void useDirectWrites() { + try (final EnvOptions envOptions = new EnvOptions()) { + final boolean boolValue = rand.nextBoolean(); + envOptions.setUseDirectWrites(boolValue); + assertThat(envOptions.useDirectWrites()).isEqualTo(boolValue); + } + } + + @Test + public void allowFallocate() { + try (final EnvOptions envOptions = new EnvOptions()) { + final boolean boolValue = rand.nextBoolean(); + envOptions.setAllowFallocate(boolValue); + assertThat(envOptions.allowFallocate()).isEqualTo(boolValue); + } + } + + @Test + public void setFdCloexecs() { + try (final EnvOptions envOptions = new EnvOptions()) { + final boolean boolValue = rand.nextBoolean(); + envOptions.setSetFdCloexec(boolValue); + assertThat(envOptions.setFdCloexec()).isEqualTo(boolValue); + } + } + + @Test + public void bytesPerSync() { + try (final EnvOptions envOptions = new EnvOptions()) { + final long longValue = rand.nextLong(); + envOptions.setBytesPerSync(longValue); + assertThat(envOptions.bytesPerSync()).isEqualTo(longValue); + } + } + + @Test + public void fallocateWithKeepSize() { + try (final EnvOptions envOptions = new EnvOptions()) { + final boolean boolValue = rand.nextBoolean(); + envOptions.setFallocateWithKeepSize(boolValue); + assertThat(envOptions.fallocateWithKeepSize()).isEqualTo(boolValue); + } + } + + @Test + public void compactionReadaheadSize() { + try (final EnvOptions envOptions = new EnvOptions()) { + final int intValue = rand.nextInt(); + envOptions.setCompactionReadaheadSize(intValue); + assertThat(envOptions.compactionReadaheadSize()).isEqualTo(intValue); + } + } + + @Test + public void randomAccessMaxBufferSize() { + try (final EnvOptions envOptions = new EnvOptions()) { + final int intValue = rand.nextInt(); + envOptions.setRandomAccessMaxBufferSize(intValue); + assertThat(envOptions.randomAccessMaxBufferSize()).isEqualTo(intValue); + } + } + + @Test + public void writableFileMaxBufferSize() { + try (final EnvOptions envOptions = new EnvOptions()) { + final int intValue = rand.nextInt(); + envOptions.setWritableFileMaxBufferSize(intValue); + assertThat(envOptions.writableFileMaxBufferSize()).isEqualTo(intValue); + } + } + + @Test + public void rateLimiter() { + try (final EnvOptions envOptions = new EnvOptions(); + final RateLimiter rateLimiter1 = new RateLimiter(1000, 100 * 1000, 1)) { + envOptions.setRateLimiter(rateLimiter1); + assertThat(envOptions.rateLimiter()).isEqualTo(rateLimiter1); + + try(final RateLimiter rateLimiter2 = new RateLimiter(1000)) { + envOptions.setRateLimiter(rateLimiter2); + assertThat(envOptions.rateLimiter()).isEqualTo(rateLimiter2); + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/FilterTest.java b/java/src/test/java/org/rocksdb/FilterTest.java new file mode 100644 index 00000000000..c6109639e32 --- /dev/null +++ b/java/src/test/java/org/rocksdb/FilterTest.java @@ -0,0 +1,39 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +public class FilterTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void filter() { + // new Bloom filter + final BlockBasedTableConfig blockConfig = new BlockBasedTableConfig(); + try(final Options options = new Options()) { + + try(final Filter bloomFilter = new BloomFilter()) { + blockConfig.setFilter(bloomFilter); + options.setTableFormatConfig(blockConfig); + } + + try(final Filter bloomFilter = new BloomFilter(10)) { + blockConfig.setFilter(bloomFilter); + options.setTableFormatConfig(blockConfig); + } + + try(final Filter bloomFilter = new BloomFilter(10, false)) { + blockConfig.setFilter(bloomFilter); + options.setTableFormatConfig(blockConfig); + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/FlushTest.java b/java/src/test/java/org/rocksdb/FlushTest.java new file mode 100644 index 00000000000..46a5cdc6802 --- /dev/null +++ b/java/src/test/java/org/rocksdb/FlushTest.java @@ -0,0 +1,49 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import static org.assertj.core.api.Assertions.assertThat; + +public class FlushTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void flush() throws RocksDBException { + try(final Options options = new Options() + .setCreateIfMissing(true) + .setMaxWriteBufferNumber(10) + .setMinWriteBufferNumberToMerge(10); + final WriteOptions wOpt = new WriteOptions() + .setDisableWAL(true); + final FlushOptions flushOptions = new FlushOptions() + .setWaitForFlush(true)) { + assertThat(flushOptions.waitForFlush()).isTrue(); + + try(final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + db.put(wOpt, "key1".getBytes(), "value1".getBytes()); + db.put(wOpt, "key2".getBytes(), "value2".getBytes()); + db.put(wOpt, "key3".getBytes(), "value3".getBytes()); + db.put(wOpt, "key4".getBytes(), "value4".getBytes()); + assertThat(db.getProperty("rocksdb.num-entries-active-mem-table")) + .isEqualTo("4"); + db.flush(flushOptions); + assertThat(db.getProperty("rocksdb.num-entries-active-mem-table")) + .isEqualTo("0"); + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/InfoLogLevelTest.java b/java/src/test/java/org/rocksdb/InfoLogLevelTest.java new file mode 100644 index 00000000000..48ecfa16a97 --- /dev/null +++ b/java/src/test/java/org/rocksdb/InfoLogLevelTest.java @@ -0,0 +1,107 @@ +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.rocksdb.util.Environment; + +import java.io.IOException; + +import static java.nio.file.Files.readAllBytes; +import static java.nio.file.Paths.get; +import static org.assertj.core.api.Assertions.assertThat; + +public class InfoLogLevelTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void testInfoLogLevel() throws RocksDBException, + IOException { + try (final RocksDB db = + RocksDB.open(dbFolder.getRoot().getAbsolutePath())) { + db.put("key".getBytes(), "value".getBytes()); + assertThat(getLogContentsWithoutHeader()).isNotEmpty(); + } + } + + @Test + public void testFatalLogLevel() throws RocksDBException, + IOException { + try (final Options options = new Options(). + setCreateIfMissing(true). + setInfoLogLevel(InfoLogLevel.FATAL_LEVEL); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + assertThat(options.infoLogLevel()). + isEqualTo(InfoLogLevel.FATAL_LEVEL); + db.put("key".getBytes(), "value".getBytes()); + // As InfoLogLevel is set to FATAL_LEVEL, here we expect the log + // content to be empty. + assertThat(getLogContentsWithoutHeader()).isEmpty(); + } + } + + @Test + public void testFatalLogLevelWithDBOptions() + throws RocksDBException, IOException { + try (final DBOptions dbOptions = new DBOptions(). + setInfoLogLevel(InfoLogLevel.FATAL_LEVEL); + final Options options = new Options(dbOptions, + new ColumnFamilyOptions()). + setCreateIfMissing(true); + final RocksDB db = + RocksDB.open(options, dbFolder.getRoot().getAbsolutePath())) { + assertThat(dbOptions.infoLogLevel()). + isEqualTo(InfoLogLevel.FATAL_LEVEL); + assertThat(options.infoLogLevel()). + isEqualTo(InfoLogLevel.FATAL_LEVEL); + db.put("key".getBytes(), "value".getBytes()); + assertThat(getLogContentsWithoutHeader()).isEmpty(); + } + } + + @Test(expected = IllegalArgumentException.class) + public void failIfIllegalByteValueProvided() { + InfoLogLevel.getInfoLogLevel((byte) -1); + } + + @Test + public void valueOf() { + assertThat(InfoLogLevel.valueOf("DEBUG_LEVEL")). + isEqualTo(InfoLogLevel.DEBUG_LEVEL); + } + + /** + * Read LOG file contents into String. + * + * @return LOG file contents as String. + * @throws IOException if file is not found. + */ + private String getLogContentsWithoutHeader() throws IOException { + final String separator = Environment.isWindows() ? + "\n" : System.getProperty("line.separator"); + final String[] lines = new String(readAllBytes(get( + dbFolder.getRoot().getAbsolutePath() + "/LOG"))).split(separator); + + int first_non_header = lines.length; + // Identify the last line of the header + for (int i = lines.length - 1; i >= 0; --i) { + if (lines[i].indexOf("Options.") >= 0 && lines[i].indexOf(':') >= 0) { + first_non_header = i + 1; + break; + } + } + StringBuilder builder = new StringBuilder(); + for (int i = first_non_header; i < lines.length; ++i) { + builder.append(lines[i]).append(separator); + } + return builder.toString(); + } +} diff --git a/java/src/test/java/org/rocksdb/IngestExternalFileOptionsTest.java b/java/src/test/java/org/rocksdb/IngestExternalFileOptionsTest.java new file mode 100644 index 00000000000..83e0dd17af4 --- /dev/null +++ b/java/src/test/java/org/rocksdb/IngestExternalFileOptionsTest.java @@ -0,0 +1,87 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import java.util.Random; + +import static org.assertj.core.api.Assertions.assertThat; + +public class IngestExternalFileOptionsTest { + @ClassRule + public static final RocksMemoryResource rocksMemoryResource + = new RocksMemoryResource(); + + public static final Random rand = + PlatformRandomHelper.getPlatformSpecificRandomFactory(); + + @Test + public void createExternalSstFileInfoWithoutParameters() { + try (final IngestExternalFileOptions options = + new IngestExternalFileOptions()) { + assertThat(options).isNotNull(); + } + } + + @Test + public void createExternalSstFileInfoWithParameters() { + final boolean moveFiles = rand.nextBoolean(); + final boolean snapshotConsistency = rand.nextBoolean(); + final boolean allowGlobalSeqNo = rand.nextBoolean(); + final boolean allowBlockingFlush = rand.nextBoolean(); + try (final IngestExternalFileOptions options = + new IngestExternalFileOptions(moveFiles, snapshotConsistency, + allowGlobalSeqNo, allowBlockingFlush)) { + assertThat(options).isNotNull(); + assertThat(options.moveFiles()).isEqualTo(moveFiles); + assertThat(options.snapshotConsistency()).isEqualTo(snapshotConsistency); + assertThat(options.allowGlobalSeqNo()).isEqualTo(allowGlobalSeqNo); + assertThat(options.allowBlockingFlush()).isEqualTo(allowBlockingFlush); + } + } + + @Test + public void moveFiles() { + try (final IngestExternalFileOptions options = + new IngestExternalFileOptions()) { + final boolean moveFiles = rand.nextBoolean(); + options.setMoveFiles(moveFiles); + assertThat(options.moveFiles()).isEqualTo(moveFiles); + } + } + + @Test + public void snapshotConsistency() { + try (final IngestExternalFileOptions options = + new IngestExternalFileOptions()) { + final boolean snapshotConsistency = rand.nextBoolean(); + options.setSnapshotConsistency(snapshotConsistency); + assertThat(options.snapshotConsistency()).isEqualTo(snapshotConsistency); + } + } + + @Test + public void allowGlobalSeqNo() { + try (final IngestExternalFileOptions options = + new IngestExternalFileOptions()) { + final boolean allowGlobalSeqNo = rand.nextBoolean(); + options.setAllowGlobalSeqNo(allowGlobalSeqNo); + assertThat(options.allowGlobalSeqNo()).isEqualTo(allowGlobalSeqNo); + } + } + + @Test + public void allowBlockingFlush() { + try (final IngestExternalFileOptions options = + new IngestExternalFileOptions()) { + final boolean allowBlockingFlush = rand.nextBoolean(); + options.setAllowBlockingFlush(allowBlockingFlush); + assertThat(options.allowBlockingFlush()).isEqualTo(allowBlockingFlush); + } + } +} diff --git a/java/src/test/java/org/rocksdb/KeyMayExistTest.java b/java/src/test/java/org/rocksdb/KeyMayExistTest.java new file mode 100644 index 00000000000..8092270eb2d --- /dev/null +++ b/java/src/test/java/org/rocksdb/KeyMayExistTest.java @@ -0,0 +1,87 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; + +public class KeyMayExistTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void keyMayExist() throws RocksDBException { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes()) + ); + + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), + cfDescriptors, columnFamilyHandleList)) { + try { + assertThat(columnFamilyHandleList.size()). + isEqualTo(2); + db.put("key".getBytes(), "value".getBytes()); + // Test without column family + StringBuilder retValue = new StringBuilder(); + boolean exists = db.keyMayExist("key".getBytes(), retValue); + assertThat(exists).isTrue(); + assertThat(retValue.toString()).isEqualTo("value"); + + // Test without column family but with readOptions + try (final ReadOptions readOptions = new ReadOptions()) { + retValue = new StringBuilder(); + exists = db.keyMayExist(readOptions, "key".getBytes(), retValue); + assertThat(exists).isTrue(); + assertThat(retValue.toString()).isEqualTo("value"); + } + + // Test with column family + retValue = new StringBuilder(); + exists = db.keyMayExist(columnFamilyHandleList.get(0), "key".getBytes(), + retValue); + assertThat(exists).isTrue(); + assertThat(retValue.toString()).isEqualTo("value"); + + // Test with column family and readOptions + try (final ReadOptions readOptions = new ReadOptions()) { + retValue = new StringBuilder(); + exists = db.keyMayExist(readOptions, + columnFamilyHandleList.get(0), "key".getBytes(), + retValue); + assertThat(exists).isTrue(); + assertThat(retValue.toString()).isEqualTo("value"); + } + + // KeyMayExist in CF1 must return false + assertThat(db.keyMayExist(columnFamilyHandleList.get(1), + "key".getBytes(), retValue)).isFalse(); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/LRUCacheTest.java b/java/src/test/java/org/rocksdb/LRUCacheTest.java new file mode 100644 index 00000000000..d2cd15b7e97 --- /dev/null +++ b/java/src/test/java/org/rocksdb/LRUCacheTest.java @@ -0,0 +1,27 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.Test; + +public class LRUCacheTest { + + static { + RocksDB.loadLibrary(); + } + + @Test + public void newLRUCache() { + final long capacity = 1000; + final int numShardBits = 16; + final boolean strictCapacityLimit = true; + final double highPriPoolRatio = 5; + try(final Cache lruCache = new LRUCache(capacity, + numShardBits, strictCapacityLimit, highPriPoolRatio)) { + //no op + } + } +} diff --git a/java/src/test/java/org/rocksdb/LoggerTest.java b/java/src/test/java/org/rocksdb/LoggerTest.java new file mode 100644 index 00000000000..f83cff3b7fe --- /dev/null +++ b/java/src/test/java/org/rocksdb/LoggerTest.java @@ -0,0 +1,238 @@ +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.concurrent.atomic.AtomicInteger; + +import static org.assertj.core.api.Assertions.assertThat; + +public class LoggerTest { + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void customLogger() throws RocksDBException { + final AtomicInteger logMessageCounter = new AtomicInteger(); + try (final Options options = new Options(). + setInfoLogLevel(InfoLogLevel.DEBUG_LEVEL). + setCreateIfMissing(true); + final Logger logger = new Logger(options) { + // Create new logger with max log level passed by options + @Override + protected void log(InfoLogLevel infoLogLevel, String logMsg) { + assertThat(logMsg).isNotNull(); + assertThat(logMsg.length()).isGreaterThan(0); + logMessageCounter.incrementAndGet(); + } + } + ) { + // Set custom logger to options + options.setLogger(logger); + + try (final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + // there should be more than zero received log messages in + // debug level. + assertThat(logMessageCounter.get()).isGreaterThan(0); + } + } + } + + @Test + public void warnLogger() throws RocksDBException { + final AtomicInteger logMessageCounter = new AtomicInteger(); + try (final Options options = new Options(). + setInfoLogLevel(InfoLogLevel.WARN_LEVEL). + setCreateIfMissing(true); + + final Logger logger = new Logger(options) { + // Create new logger with max log level passed by options + @Override + protected void log(InfoLogLevel infoLogLevel, String logMsg) { + assertThat(logMsg).isNotNull(); + assertThat(logMsg.length()).isGreaterThan(0); + logMessageCounter.incrementAndGet(); + } + } + ) { + + // Set custom logger to options + options.setLogger(logger); + + try (final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + // there should be zero messages + // using warn level as log level. + assertThat(logMessageCounter.get()).isEqualTo(0); + } + } + } + + + @Test + public void fatalLogger() throws RocksDBException { + final AtomicInteger logMessageCounter = new AtomicInteger(); + try (final Options options = new Options(). + setInfoLogLevel(InfoLogLevel.FATAL_LEVEL). + setCreateIfMissing(true); + + final Logger logger = new Logger(options) { + // Create new logger with max log level passed by options + @Override + protected void log(InfoLogLevel infoLogLevel, String logMsg) { + assertThat(logMsg).isNotNull(); + assertThat(logMsg.length()).isGreaterThan(0); + logMessageCounter.incrementAndGet(); + } + } + ) { + + // Set custom logger to options + options.setLogger(logger); + + try (final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + // there should be zero messages + // using fatal level as log level. + assertThat(logMessageCounter.get()).isEqualTo(0); + } + } + } + + @Test + public void dbOptionsLogger() throws RocksDBException { + final AtomicInteger logMessageCounter = new AtomicInteger(); + try (final DBOptions options = new DBOptions(). + setInfoLogLevel(InfoLogLevel.FATAL_LEVEL). + setCreateIfMissing(true); + final Logger logger = new Logger(options) { + // Create new logger with max log level passed by options + @Override + protected void log(InfoLogLevel infoLogLevel, String logMsg) { + assertThat(logMsg).isNotNull(); + assertThat(logMsg.length()).isGreaterThan(0); + logMessageCounter.incrementAndGet(); + } + } + ) { + // Set custom logger to options + options.setLogger(logger); + + final List cfDescriptors = + Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY)); + final List cfHandles = new ArrayList<>(); + + try (final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), + cfDescriptors, cfHandles)) { + try { + // there should be zero messages + // using fatal level as log level. + assertThat(logMessageCounter.get()).isEqualTo(0); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : cfHandles) { + columnFamilyHandle.close(); + } + } + } + } + } + + @Test + public void setWarnLogLevel() { + final AtomicInteger logMessageCounter = new AtomicInteger(); + try (final Options options = new Options(). + setInfoLogLevel(InfoLogLevel.FATAL_LEVEL). + setCreateIfMissing(true); + final Logger logger = new Logger(options) { + // Create new logger with max log level passed by options + @Override + protected void log(InfoLogLevel infoLogLevel, String logMsg) { + assertThat(logMsg).isNotNull(); + assertThat(logMsg.length()).isGreaterThan(0); + logMessageCounter.incrementAndGet(); + } + } + ) { + assertThat(logger.infoLogLevel()). + isEqualTo(InfoLogLevel.FATAL_LEVEL); + logger.setInfoLogLevel(InfoLogLevel.WARN_LEVEL); + assertThat(logger.infoLogLevel()). + isEqualTo(InfoLogLevel.WARN_LEVEL); + } + } + + @Test + public void setInfoLogLevel() { + final AtomicInteger logMessageCounter = new AtomicInteger(); + try (final Options options = new Options(). + setInfoLogLevel(InfoLogLevel.FATAL_LEVEL). + setCreateIfMissing(true); + final Logger logger = new Logger(options) { + // Create new logger with max log level passed by options + @Override + protected void log(InfoLogLevel infoLogLevel, String logMsg) { + assertThat(logMsg).isNotNull(); + assertThat(logMsg.length()).isGreaterThan(0); + logMessageCounter.incrementAndGet(); + } + } + ) { + assertThat(logger.infoLogLevel()). + isEqualTo(InfoLogLevel.FATAL_LEVEL); + logger.setInfoLogLevel(InfoLogLevel.DEBUG_LEVEL); + assertThat(logger.infoLogLevel()). + isEqualTo(InfoLogLevel.DEBUG_LEVEL); + } + } + + @Test + public void changeLogLevelAtRuntime() throws RocksDBException { + final AtomicInteger logMessageCounter = new AtomicInteger(); + try (final Options options = new Options(). + setInfoLogLevel(InfoLogLevel.FATAL_LEVEL). + setCreateIfMissing(true); + + // Create new logger with max log level passed by options + final Logger logger = new Logger(options) { + @Override + protected void log(InfoLogLevel infoLogLevel, String logMsg) { + assertThat(logMsg).isNotNull(); + assertThat(logMsg.length()).isGreaterThan(0); + logMessageCounter.incrementAndGet(); + } + } + ) { + // Set custom logger to options + options.setLogger(logger); + + try (final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + + // there should be zero messages + // using fatal level as log level. + assertThat(logMessageCounter.get()).isEqualTo(0); + + // change log level to debug level + logger.setInfoLogLevel(InfoLogLevel.DEBUG_LEVEL); + + db.put("key".getBytes(), "value".getBytes()); + db.flush(new FlushOptions().setWaitForFlush(true)); + + // messages shall be received due to previous actions. + assertThat(logMessageCounter.get()).isNotEqualTo(0); + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/MemTableTest.java b/java/src/test/java/org/rocksdb/MemTableTest.java new file mode 100644 index 00000000000..59503d48181 --- /dev/null +++ b/java/src/test/java/org/rocksdb/MemTableTest.java @@ -0,0 +1,111 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class MemTableTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void hashSkipListMemTable() throws RocksDBException { + try(final Options options = new Options()) { + // Test HashSkipListMemTableConfig + HashSkipListMemTableConfig memTableConfig = + new HashSkipListMemTableConfig(); + assertThat(memTableConfig.bucketCount()). + isEqualTo(1000000); + memTableConfig.setBucketCount(2000000); + assertThat(memTableConfig.bucketCount()). + isEqualTo(2000000); + assertThat(memTableConfig.height()). + isEqualTo(4); + memTableConfig.setHeight(5); + assertThat(memTableConfig.height()). + isEqualTo(5); + assertThat(memTableConfig.branchingFactor()). + isEqualTo(4); + memTableConfig.setBranchingFactor(6); + assertThat(memTableConfig.branchingFactor()). + isEqualTo(6); + options.setMemTableConfig(memTableConfig); + } + } + + @Test + public void skipListMemTable() throws RocksDBException { + try(final Options options = new Options()) { + SkipListMemTableConfig skipMemTableConfig = + new SkipListMemTableConfig(); + assertThat(skipMemTableConfig.lookahead()). + isEqualTo(0); + skipMemTableConfig.setLookahead(20); + assertThat(skipMemTableConfig.lookahead()). + isEqualTo(20); + options.setMemTableConfig(skipMemTableConfig); + } + } + + @Test + public void hashLinkedListMemTable() throws RocksDBException { + try(final Options options = new Options()) { + HashLinkedListMemTableConfig hashLinkedListMemTableConfig = + new HashLinkedListMemTableConfig(); + assertThat(hashLinkedListMemTableConfig.bucketCount()). + isEqualTo(50000); + hashLinkedListMemTableConfig.setBucketCount(100000); + assertThat(hashLinkedListMemTableConfig.bucketCount()). + isEqualTo(100000); + assertThat(hashLinkedListMemTableConfig.hugePageTlbSize()). + isEqualTo(0); + hashLinkedListMemTableConfig.setHugePageTlbSize(1); + assertThat(hashLinkedListMemTableConfig.hugePageTlbSize()). + isEqualTo(1); + assertThat(hashLinkedListMemTableConfig. + bucketEntriesLoggingThreshold()). + isEqualTo(4096); + hashLinkedListMemTableConfig. + setBucketEntriesLoggingThreshold(200); + assertThat(hashLinkedListMemTableConfig. + bucketEntriesLoggingThreshold()). + isEqualTo(200); + assertThat(hashLinkedListMemTableConfig. + ifLogBucketDistWhenFlush()).isTrue(); + hashLinkedListMemTableConfig. + setIfLogBucketDistWhenFlush(false); + assertThat(hashLinkedListMemTableConfig. + ifLogBucketDistWhenFlush()).isFalse(); + assertThat(hashLinkedListMemTableConfig. + thresholdUseSkiplist()). + isEqualTo(256); + hashLinkedListMemTableConfig.setThresholdUseSkiplist(29); + assertThat(hashLinkedListMemTableConfig. + thresholdUseSkiplist()). + isEqualTo(29); + options.setMemTableConfig(hashLinkedListMemTableConfig); + } + } + + @Test + public void vectorMemTable() throws RocksDBException { + try(final Options options = new Options()) { + VectorMemTableConfig vectorMemTableConfig = + new VectorMemTableConfig(); + assertThat(vectorMemTableConfig.reservedSize()). + isEqualTo(0); + vectorMemTableConfig.setReservedSize(123); + assertThat(vectorMemTableConfig.reservedSize()). + isEqualTo(123); + options.setMemTableConfig(vectorMemTableConfig); + } + } +} diff --git a/java/src/test/java/org/rocksdb/MergeTest.java b/java/src/test/java/org/rocksdb/MergeTest.java new file mode 100644 index 00000000000..73b90869cf1 --- /dev/null +++ b/java/src/test/java/org/rocksdb/MergeTest.java @@ -0,0 +1,240 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.Arrays; +import java.util.List; +import java.util.ArrayList; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import static org.assertj.core.api.Assertions.assertThat; + +public class MergeTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void stringOption() + throws InterruptedException, RocksDBException { + try (final Options opt = new Options() + .setCreateIfMissing(true) + .setMergeOperatorName("stringappend"); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + // writing aa under key + db.put("key".getBytes(), "aa".getBytes()); + // merge bb under key + db.merge("key".getBytes(), "bb".getBytes()); + + final byte[] value = db.get("key".getBytes()); + final String strValue = new String(value); + assertThat(strValue).isEqualTo("aa,bb"); + } + } + + @Test + public void cFStringOption() + throws InterruptedException, RocksDBException { + + try (final ColumnFamilyOptions cfOpt1 = new ColumnFamilyOptions() + .setMergeOperatorName("stringappend"); + final ColumnFamilyOptions cfOpt2 = new ColumnFamilyOptions() + .setMergeOperatorName("stringappend") + ) { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, cfOpt1), + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, cfOpt2) + ); + + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions opt = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + columnFamilyHandleList)) { + try { + // writing aa under key + db.put(columnFamilyHandleList.get(1), + "cfkey".getBytes(), "aa".getBytes()); + // merge bb under key + db.merge(columnFamilyHandleList.get(1), + "cfkey".getBytes(), "bb".getBytes()); + + byte[] value = db.get(columnFamilyHandleList.get(1), + "cfkey".getBytes()); + String strValue = new String(value); + assertThat(strValue).isEqualTo("aa,bb"); + } finally { + for (final ColumnFamilyHandle handle : columnFamilyHandleList) { + handle.close(); + } + } + } + } + } + + @Test + public void operatorOption() + throws InterruptedException, RocksDBException { + try (final StringAppendOperator stringAppendOperator = new StringAppendOperator(); + final Options opt = new Options() + .setCreateIfMissing(true) + .setMergeOperator(stringAppendOperator); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + // Writing aa under key + db.put("key".getBytes(), "aa".getBytes()); + + // Writing bb under key + db.merge("key".getBytes(), "bb".getBytes()); + + final byte[] value = db.get("key".getBytes()); + final String strValue = new String(value); + + assertThat(strValue).isEqualTo("aa,bb"); + } + } + + @Test + public void cFOperatorOption() + throws InterruptedException, RocksDBException { + try (final StringAppendOperator stringAppendOperator = new StringAppendOperator(); + final ColumnFamilyOptions cfOpt1 = new ColumnFamilyOptions() + .setMergeOperator(stringAppendOperator); + final ColumnFamilyOptions cfOpt2 = new ColumnFamilyOptions() + .setMergeOperator(stringAppendOperator) + ) { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, cfOpt1), + new ColumnFamilyDescriptor("new_cf".getBytes(), cfOpt2) + ); + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions opt = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + columnFamilyHandleList) + ) { + try { + // writing aa under key + db.put(columnFamilyHandleList.get(1), + "cfkey".getBytes(), "aa".getBytes()); + // merge bb under key + db.merge(columnFamilyHandleList.get(1), + "cfkey".getBytes(), "bb".getBytes()); + byte[] value = db.get(columnFamilyHandleList.get(1), + "cfkey".getBytes()); + String strValue = new String(value); + + // Test also with createColumnFamily + try (final ColumnFamilyOptions cfHandleOpts = + new ColumnFamilyOptions() + .setMergeOperator(stringAppendOperator); + final ColumnFamilyHandle cfHandle = + db.createColumnFamily( + new ColumnFamilyDescriptor("new_cf2".getBytes(), + cfHandleOpts)) + ) { + // writing xx under cfkey2 + db.put(cfHandle, "cfkey2".getBytes(), "xx".getBytes()); + // merge yy under cfkey2 + db.merge(cfHandle, new WriteOptions(), "cfkey2".getBytes(), + "yy".getBytes()); + value = db.get(cfHandle, "cfkey2".getBytes()); + String strValueTmpCf = new String(value); + + assertThat(strValue).isEqualTo("aa,bb"); + assertThat(strValueTmpCf).isEqualTo("xx,yy"); + } + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + } + + @Test + public void operatorGcBehaviour() + throws RocksDBException { + try (final StringAppendOperator stringAppendOperator = new StringAppendOperator()) { + try (final Options opt = new Options() + .setCreateIfMissing(true) + .setMergeOperator(stringAppendOperator); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + //no-op + } + + + // test reuse + try (final Options opt = new Options() + .setMergeOperator(stringAppendOperator); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + //no-op + } + + // test param init + try (final StringAppendOperator stringAppendOperator2 = new StringAppendOperator(); + final Options opt = new Options() + .setMergeOperator(stringAppendOperator2); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + //no-op + } + + // test replace one with another merge operator instance + try (final Options opt = new Options() + .setMergeOperator(stringAppendOperator); + final StringAppendOperator newStringAppendOperator = new StringAppendOperator()) { + opt.setMergeOperator(newStringAppendOperator); + try (final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + //no-op + } + } + } + } + + @Test + public void emptyStringInSetMergeOperatorByName() { + try (final Options opt = new Options() + .setMergeOperatorName(""); + final ColumnFamilyOptions cOpt = new ColumnFamilyOptions() + .setMergeOperatorName("")) { + //no-op + } + } + + @Test(expected = IllegalArgumentException.class) + public void nullStringInSetMergeOperatorByNameOptions() { + try (final Options opt = new Options()) { + opt.setMergeOperatorName(null); + } + } + + @Test(expected = IllegalArgumentException.class) + public void + nullStringInSetMergeOperatorByNameColumnFamilyOptions() { + try (final ColumnFamilyOptions opt = new ColumnFamilyOptions()) { + opt.setMergeOperatorName(null); + } + } +} diff --git a/java/src/test/java/org/rocksdb/MixedOptionsTest.java b/java/src/test/java/org/rocksdb/MixedOptionsTest.java new file mode 100644 index 00000000000..ff68b1b00e3 --- /dev/null +++ b/java/src/test/java/org/rocksdb/MixedOptionsTest.java @@ -0,0 +1,55 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class MixedOptionsTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void mixedOptionsTest(){ + // Set a table factory and check the names + try(final Filter bloomFilter = new BloomFilter(); + final ColumnFamilyOptions cfOptions = new ColumnFamilyOptions() + .setTableFormatConfig( + new BlockBasedTableConfig().setFilter(bloomFilter)) + ) { + assertThat(cfOptions.tableFactoryName()).isEqualTo( + "BlockBasedTable"); + cfOptions.setTableFormatConfig(new PlainTableConfig()); + assertThat(cfOptions.tableFactoryName()).isEqualTo("PlainTable"); + // Initialize a dbOptions object from cf options and + // db options + try (final DBOptions dbOptions = new DBOptions(); + final Options options = new Options(dbOptions, cfOptions)) { + assertThat(options.tableFactoryName()).isEqualTo("PlainTable"); + // Free instances + } + } + + // Test Optimize for statements + try(final ColumnFamilyOptions cfOptions = new ColumnFamilyOptions()) { + cfOptions.optimizeUniversalStyleCompaction(); + cfOptions.optimizeLevelStyleCompaction(); + cfOptions.optimizeForPointLookup(1024); + try(final Options options = new Options()) { + options.optimizeLevelStyleCompaction(); + options.optimizeLevelStyleCompaction(400); + options.optimizeUniversalStyleCompaction(); + options.optimizeUniversalStyleCompaction(400); + options.optimizeForPointLookup(1024); + options.prepareForBulkLoad(); + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/MutableColumnFamilyOptionsTest.java b/java/src/test/java/org/rocksdb/MutableColumnFamilyOptionsTest.java new file mode 100644 index 00000000000..f631905e19f --- /dev/null +++ b/java/src/test/java/org/rocksdb/MutableColumnFamilyOptionsTest.java @@ -0,0 +1,88 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.Test; +import org.rocksdb.MutableColumnFamilyOptions.MutableColumnFamilyOptionsBuilder; + +import java.util.NoSuchElementException; + +import static org.assertj.core.api.Assertions.assertThat; + +public class MutableColumnFamilyOptionsTest { + + @Test + public void builder() { + final MutableColumnFamilyOptionsBuilder builder = + MutableColumnFamilyOptions.builder(); + builder + .setWriteBufferSize(10) + .setInplaceUpdateNumLocks(5) + .setDisableAutoCompactions(true) + .setParanoidFileChecks(true); + + assertThat(builder.writeBufferSize()).isEqualTo(10); + assertThat(builder.inplaceUpdateNumLocks()).isEqualTo(5); + assertThat(builder.disableAutoCompactions()).isEqualTo(true); + assertThat(builder.paranoidFileChecks()).isEqualTo(true); + } + + @Test(expected = NoSuchElementException.class) + public void builder_getWhenNotSet() { + final MutableColumnFamilyOptionsBuilder builder = + MutableColumnFamilyOptions.builder(); + + builder.writeBufferSize(); + } + + @Test + public void builder_build() { + final MutableColumnFamilyOptions options = MutableColumnFamilyOptions + .builder() + .setWriteBufferSize(10) + .setParanoidFileChecks(true) + .build(); + + assertThat(options.getKeys().length).isEqualTo(2); + assertThat(options.getValues().length).isEqualTo(2); + assertThat(options.getKeys()[0]) + .isEqualTo( + MutableColumnFamilyOptions.MemtableOption.write_buffer_size.name()); + assertThat(options.getValues()[0]).isEqualTo("10"); + assertThat(options.getKeys()[1]) + .isEqualTo( + MutableColumnFamilyOptions.MiscOption.paranoid_file_checks.name()); + assertThat(options.getValues()[1]).isEqualTo("true"); + } + + @Test + public void mutableColumnFamilyOptions_toString() { + final String str = MutableColumnFamilyOptions + .builder() + .setWriteBufferSize(10) + .setInplaceUpdateNumLocks(5) + .setDisableAutoCompactions(true) + .setParanoidFileChecks(true) + .build() + .toString(); + + assertThat(str).isEqualTo("write_buffer_size=10;inplace_update_num_locks=5;" + + "disable_auto_compactions=true;paranoid_file_checks=true"); + } + + @Test + public void mutableColumnFamilyOptions_parse() { + final String str = "write_buffer_size=10;inplace_update_num_locks=5;" + + "disable_auto_compactions=true;paranoid_file_checks=true"; + + final MutableColumnFamilyOptionsBuilder builder = + MutableColumnFamilyOptions.parse(str); + + assertThat(builder.writeBufferSize()).isEqualTo(10); + assertThat(builder.inplaceUpdateNumLocks()).isEqualTo(5); + assertThat(builder.disableAutoCompactions()).isEqualTo(true); + assertThat(builder.paranoidFileChecks()).isEqualTo(true); + } +} diff --git a/java/src/test/java/org/rocksdb/NativeLibraryLoaderTest.java b/java/src/test/java/org/rocksdb/NativeLibraryLoaderTest.java new file mode 100644 index 00000000000..ab60081a076 --- /dev/null +++ b/java/src/test/java/org/rocksdb/NativeLibraryLoaderTest.java @@ -0,0 +1,41 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.rocksdb.util.Environment; + +import java.io.File; +import java.io.IOException; +import java.nio.file.*; + +import static org.assertj.core.api.Assertions.assertThat; + +public class NativeLibraryLoaderTest { + + @Rule + public TemporaryFolder temporaryFolder = new TemporaryFolder(); + + @Test + public void tempFolder() throws IOException { + NativeLibraryLoader.getInstance().loadLibraryFromJarToTemp( + temporaryFolder.getRoot().getAbsolutePath()); + final Path path = Paths.get(temporaryFolder.getRoot().getAbsolutePath(), + Environment.getJniLibraryFileName("rocksdb")); + assertThat(Files.exists(path)).isTrue(); + assertThat(Files.isReadable(path)).isTrue(); + } + + @Test + public void overridesExistingLibrary() throws IOException { + File first = NativeLibraryLoader.getInstance().loadLibraryFromJarToTemp( + temporaryFolder.getRoot().getAbsolutePath()); + NativeLibraryLoader.getInstance().loadLibraryFromJarToTemp( + temporaryFolder.getRoot().getAbsolutePath()); + assertThat(first.exists()).isTrue(); + } +} diff --git a/java/src/test/java/org/rocksdb/OptionsTest.java b/java/src/test/java/org/rocksdb/OptionsTest.java new file mode 100644 index 00000000000..6afcab3300a --- /dev/null +++ b/java/src/test/java/org/rocksdb/OptionsTest.java @@ -0,0 +1,1095 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.nio.file.Paths; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Random; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + + +public class OptionsTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + public static final Random rand = PlatformRandomHelper. + getPlatformSpecificRandomFactory(); + + @Test + public void setIncreaseParallelism() { + try (final Options opt = new Options()) { + final int threads = Runtime.getRuntime().availableProcessors() * 2; + opt.setIncreaseParallelism(threads); + } + } + + @Test + public void writeBufferSize() throws RocksDBException { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setWriteBufferSize(longValue); + assertThat(opt.writeBufferSize()).isEqualTo(longValue); + } + } + + @Test + public void maxWriteBufferNumber() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setMaxWriteBufferNumber(intValue); + assertThat(opt.maxWriteBufferNumber()).isEqualTo(intValue); + } + } + + @Test + public void minWriteBufferNumberToMerge() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setMinWriteBufferNumberToMerge(intValue); + assertThat(opt.minWriteBufferNumberToMerge()).isEqualTo(intValue); + } + } + + @Test + public void numLevels() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setNumLevels(intValue); + assertThat(opt.numLevels()).isEqualTo(intValue); + } + } + + @Test + public void levelZeroFileNumCompactionTrigger() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setLevelZeroFileNumCompactionTrigger(intValue); + assertThat(opt.levelZeroFileNumCompactionTrigger()).isEqualTo(intValue); + } + } + + @Test + public void levelZeroSlowdownWritesTrigger() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setLevelZeroSlowdownWritesTrigger(intValue); + assertThat(opt.levelZeroSlowdownWritesTrigger()).isEqualTo(intValue); + } + } + + @Test + public void levelZeroStopWritesTrigger() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setLevelZeroStopWritesTrigger(intValue); + assertThat(opt.levelZeroStopWritesTrigger()).isEqualTo(intValue); + } + } + + @Test + public void targetFileSizeBase() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setTargetFileSizeBase(longValue); + assertThat(opt.targetFileSizeBase()).isEqualTo(longValue); + } + } + + @Test + public void targetFileSizeMultiplier() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setTargetFileSizeMultiplier(intValue); + assertThat(opt.targetFileSizeMultiplier()).isEqualTo(intValue); + } + } + + @Test + public void maxBytesForLevelBase() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setMaxBytesForLevelBase(longValue); + assertThat(opt.maxBytesForLevelBase()).isEqualTo(longValue); + } + } + + @Test + public void levelCompactionDynamicLevelBytes() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setLevelCompactionDynamicLevelBytes(boolValue); + assertThat(opt.levelCompactionDynamicLevelBytes()) + .isEqualTo(boolValue); + } + } + + @Test + public void maxBytesForLevelMultiplier() { + try (final Options opt = new Options()) { + final double doubleValue = rand.nextDouble(); + opt.setMaxBytesForLevelMultiplier(doubleValue); + assertThat(opt.maxBytesForLevelMultiplier()).isEqualTo(doubleValue); + } + } + + @Test + public void maxBytesForLevelMultiplierAdditional() { + try (final Options opt = new Options()) { + final int intValue1 = rand.nextInt(); + final int intValue2 = rand.nextInt(); + final int[] ints = new int[]{intValue1, intValue2}; + opt.setMaxBytesForLevelMultiplierAdditional(ints); + assertThat(opt.maxBytesForLevelMultiplierAdditional()).isEqualTo(ints); + } + } + + @Test + public void maxCompactionBytes() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setMaxCompactionBytes(longValue); + assertThat(opt.maxCompactionBytes()).isEqualTo(longValue); + } + } + + @Test + public void softPendingCompactionBytesLimit() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setSoftPendingCompactionBytesLimit(longValue); + assertThat(opt.softPendingCompactionBytesLimit()).isEqualTo(longValue); + } + } + + @Test + public void hardPendingCompactionBytesLimit() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setHardPendingCompactionBytesLimit(longValue); + assertThat(opt.hardPendingCompactionBytesLimit()).isEqualTo(longValue); + } + } + + @Test + public void level0FileNumCompactionTrigger() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setLevel0FileNumCompactionTrigger(intValue); + assertThat(opt.level0FileNumCompactionTrigger()).isEqualTo(intValue); + } + } + + @Test + public void level0SlowdownWritesTrigger() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setLevel0SlowdownWritesTrigger(intValue); + assertThat(opt.level0SlowdownWritesTrigger()).isEqualTo(intValue); + } + } + + @Test + public void level0StopWritesTrigger() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setLevel0StopWritesTrigger(intValue); + assertThat(opt.level0StopWritesTrigger()).isEqualTo(intValue); + } + } + + @Test + public void arenaBlockSize() throws RocksDBException { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setArenaBlockSize(longValue); + assertThat(opt.arenaBlockSize()).isEqualTo(longValue); + } + } + + @Test + public void disableAutoCompactions() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setDisableAutoCompactions(boolValue); + assertThat(opt.disableAutoCompactions()).isEqualTo(boolValue); + } + } + + @Test + public void maxSequentialSkipInIterations() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setMaxSequentialSkipInIterations(longValue); + assertThat(opt.maxSequentialSkipInIterations()).isEqualTo(longValue); + } + } + + @Test + public void inplaceUpdateSupport() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setInplaceUpdateSupport(boolValue); + assertThat(opt.inplaceUpdateSupport()).isEqualTo(boolValue); + } + } + + @Test + public void inplaceUpdateNumLocks() throws RocksDBException { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setInplaceUpdateNumLocks(longValue); + assertThat(opt.inplaceUpdateNumLocks()).isEqualTo(longValue); + } + } + + @Test + public void memtablePrefixBloomSizeRatio() { + try (final Options opt = new Options()) { + final double doubleValue = rand.nextDouble(); + opt.setMemtablePrefixBloomSizeRatio(doubleValue); + assertThat(opt.memtablePrefixBloomSizeRatio()).isEqualTo(doubleValue); + } + } + + @Test + public void memtableHugePageSize() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setMemtableHugePageSize(longValue); + assertThat(opt.memtableHugePageSize()).isEqualTo(longValue); + } + } + + @Test + public void bloomLocality() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setBloomLocality(intValue); + assertThat(opt.bloomLocality()).isEqualTo(intValue); + } + } + + @Test + public void maxSuccessiveMerges() throws RocksDBException { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setMaxSuccessiveMerges(longValue); + assertThat(opt.maxSuccessiveMerges()).isEqualTo(longValue); + } + } + + @Test + public void optimizeFiltersForHits() { + try (final Options opt = new Options()) { + final boolean aBoolean = rand.nextBoolean(); + opt.setOptimizeFiltersForHits(aBoolean); + assertThat(opt.optimizeFiltersForHits()).isEqualTo(aBoolean); + } + } + + @Test + public void createIfMissing() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setCreateIfMissing(boolValue); + assertThat(opt.createIfMissing()). + isEqualTo(boolValue); + } + } + + @Test + public void createMissingColumnFamilies() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setCreateMissingColumnFamilies(boolValue); + assertThat(opt.createMissingColumnFamilies()). + isEqualTo(boolValue); + } + } + + @Test + public void errorIfExists() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setErrorIfExists(boolValue); + assertThat(opt.errorIfExists()).isEqualTo(boolValue); + } + } + + @Test + public void paranoidChecks() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setParanoidChecks(boolValue); + assertThat(opt.paranoidChecks()). + isEqualTo(boolValue); + } + } + + @Test + public void maxTotalWalSize() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setMaxTotalWalSize(longValue); + assertThat(opt.maxTotalWalSize()). + isEqualTo(longValue); + } + } + + @Test + public void maxOpenFiles() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setMaxOpenFiles(intValue); + assertThat(opt.maxOpenFiles()).isEqualTo(intValue); + } + } + + @Test + public void maxFileOpeningThreads() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setMaxFileOpeningThreads(intValue); + assertThat(opt.maxFileOpeningThreads()).isEqualTo(intValue); + } + } + + @Test + public void useFsync() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setUseFsync(boolValue); + assertThat(opt.useFsync()).isEqualTo(boolValue); + } + } + + @Test + public void dbPaths() { + final List dbPaths = new ArrayList<>(); + dbPaths.add(new DbPath(Paths.get("/a"), 10)); + dbPaths.add(new DbPath(Paths.get("/b"), 100)); + dbPaths.add(new DbPath(Paths.get("/c"), 1000)); + + try (final Options opt = new Options()) { + assertThat(opt.dbPaths()).isEqualTo(Collections.emptyList()); + + opt.setDbPaths(dbPaths); + + assertThat(opt.dbPaths()).isEqualTo(dbPaths); + } + } + + @Test + public void dbLogDir() { + try (final Options opt = new Options()) { + final String str = "path/to/DbLogDir"; + opt.setDbLogDir(str); + assertThat(opt.dbLogDir()).isEqualTo(str); + } + } + + @Test + public void walDir() { + try (final Options opt = new Options()) { + final String str = "path/to/WalDir"; + opt.setWalDir(str); + assertThat(opt.walDir()).isEqualTo(str); + } + } + + @Test + public void deleteObsoleteFilesPeriodMicros() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setDeleteObsoleteFilesPeriodMicros(longValue); + assertThat(opt.deleteObsoleteFilesPeriodMicros()). + isEqualTo(longValue); + } + } + + @Test + public void baseBackgroundCompactions() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setBaseBackgroundCompactions(intValue); + assertThat(opt.baseBackgroundCompactions()). + isEqualTo(intValue); + } + } + + @Test + public void maxBackgroundCompactions() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setMaxBackgroundCompactions(intValue); + assertThat(opt.maxBackgroundCompactions()). + isEqualTo(intValue); + } + } + + @Test + public void maxSubcompactions() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setMaxSubcompactions(intValue); + assertThat(opt.maxSubcompactions()). + isEqualTo(intValue); + } + } + + @Test + public void maxBackgroundFlushes() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setMaxBackgroundFlushes(intValue); + assertThat(opt.maxBackgroundFlushes()). + isEqualTo(intValue); + } + } + + @Test + public void maxLogFileSize() throws RocksDBException { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setMaxLogFileSize(longValue); + assertThat(opt.maxLogFileSize()).isEqualTo(longValue); + } + } + + @Test + public void logFileTimeToRoll() throws RocksDBException { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setLogFileTimeToRoll(longValue); + assertThat(opt.logFileTimeToRoll()). + isEqualTo(longValue); + } + } + + @Test + public void keepLogFileNum() throws RocksDBException { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setKeepLogFileNum(longValue); + assertThat(opt.keepLogFileNum()).isEqualTo(longValue); + } + } + + @Test + public void recycleLogFileNum() throws RocksDBException { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setRecycleLogFileNum(longValue); + assertThat(opt.recycleLogFileNum()).isEqualTo(longValue); + } + } + + @Test + public void maxManifestFileSize() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setMaxManifestFileSize(longValue); + assertThat(opt.maxManifestFileSize()). + isEqualTo(longValue); + } + } + + @Test + public void tableCacheNumshardbits() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setTableCacheNumshardbits(intValue); + assertThat(opt.tableCacheNumshardbits()). + isEqualTo(intValue); + } + } + + @Test + public void walSizeLimitMB() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setWalSizeLimitMB(longValue); + assertThat(opt.walSizeLimitMB()).isEqualTo(longValue); + } + } + + @Test + public void walTtlSeconds() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setWalTtlSeconds(longValue); + assertThat(opt.walTtlSeconds()).isEqualTo(longValue); + } + } + + @Test + public void manifestPreallocationSize() throws RocksDBException { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setManifestPreallocationSize(longValue); + assertThat(opt.manifestPreallocationSize()). + isEqualTo(longValue); + } + } + + @Test + public void useDirectReads() { + try(final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setUseDirectReads(boolValue); + assertThat(opt.useDirectReads()).isEqualTo(boolValue); + } + } + + @Test + public void useDirectIoForFlushAndCompaction() { + try(final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setUseDirectIoForFlushAndCompaction(boolValue); + assertThat(opt.useDirectIoForFlushAndCompaction()).isEqualTo(boolValue); + } + } + + @Test + public void allowFAllocate() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAllowFAllocate(boolValue); + assertThat(opt.allowFAllocate()).isEqualTo(boolValue); + } + } + + @Test + public void allowMmapReads() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAllowMmapReads(boolValue); + assertThat(opt.allowMmapReads()).isEqualTo(boolValue); + } + } + + @Test + public void allowMmapWrites() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAllowMmapWrites(boolValue); + assertThat(opt.allowMmapWrites()).isEqualTo(boolValue); + } + } + + @Test + public void isFdCloseOnExec() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setIsFdCloseOnExec(boolValue); + assertThat(opt.isFdCloseOnExec()).isEqualTo(boolValue); + } + } + + @Test + public void statsDumpPeriodSec() { + try (final Options opt = new Options()) { + final int intValue = rand.nextInt(); + opt.setStatsDumpPeriodSec(intValue); + assertThat(opt.statsDumpPeriodSec()).isEqualTo(intValue); + } + } + + @Test + public void adviseRandomOnOpen() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAdviseRandomOnOpen(boolValue); + assertThat(opt.adviseRandomOnOpen()).isEqualTo(boolValue); + } + } + + @Test + public void dbWriteBufferSize() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setDbWriteBufferSize(longValue); + assertThat(opt.dbWriteBufferSize()).isEqualTo(longValue); + } + } + + @Test + public void accessHintOnCompactionStart() { + try (final Options opt = new Options()) { + final AccessHint accessHint = AccessHint.SEQUENTIAL; + opt.setAccessHintOnCompactionStart(accessHint); + assertThat(opt.accessHintOnCompactionStart()).isEqualTo(accessHint); + } + } + + @Test + public void newTableReaderForCompactionInputs() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setNewTableReaderForCompactionInputs(boolValue); + assertThat(opt.newTableReaderForCompactionInputs()).isEqualTo(boolValue); + } + } + + @Test + public void compactionReadaheadSize() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setCompactionReadaheadSize(longValue); + assertThat(opt.compactionReadaheadSize()).isEqualTo(longValue); + } + } + + @Test + public void randomAccessMaxBufferSize() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setRandomAccessMaxBufferSize(longValue); + assertThat(opt.randomAccessMaxBufferSize()).isEqualTo(longValue); + } + } + + @Test + public void writableFileMaxBufferSize() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setWritableFileMaxBufferSize(longValue); + assertThat(opt.writableFileMaxBufferSize()).isEqualTo(longValue); + } + } + + @Test + public void useAdaptiveMutex() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setUseAdaptiveMutex(boolValue); + assertThat(opt.useAdaptiveMutex()).isEqualTo(boolValue); + } + } + + @Test + public void bytesPerSync() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setBytesPerSync(longValue); + assertThat(opt.bytesPerSync()).isEqualTo(longValue); + } + } + + @Test + public void walBytesPerSync() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setWalBytesPerSync(longValue); + assertThat(opt.walBytesPerSync()).isEqualTo(longValue); + } + } + + @Test + public void enableThreadTracking() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setEnableThreadTracking(boolValue); + assertThat(opt.enableThreadTracking()).isEqualTo(boolValue); + } + } + + @Test + public void delayedWriteRate() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setDelayedWriteRate(longValue); + assertThat(opt.delayedWriteRate()).isEqualTo(longValue); + } + } + + @Test + public void allowConcurrentMemtableWrite() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAllowConcurrentMemtableWrite(boolValue); + assertThat(opt.allowConcurrentMemtableWrite()).isEqualTo(boolValue); + } + } + + @Test + public void enableWriteThreadAdaptiveYield() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setEnableWriteThreadAdaptiveYield(boolValue); + assertThat(opt.enableWriteThreadAdaptiveYield()).isEqualTo(boolValue); + } + } + + @Test + public void writeThreadMaxYieldUsec() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setWriteThreadMaxYieldUsec(longValue); + assertThat(opt.writeThreadMaxYieldUsec()).isEqualTo(longValue); + } + } + + @Test + public void writeThreadSlowYieldUsec() { + try (final Options opt = new Options()) { + final long longValue = rand.nextLong(); + opt.setWriteThreadSlowYieldUsec(longValue); + assertThat(opt.writeThreadSlowYieldUsec()).isEqualTo(longValue); + } + } + + @Test + public void skipStatsUpdateOnDbOpen() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setSkipStatsUpdateOnDbOpen(boolValue); + assertThat(opt.skipStatsUpdateOnDbOpen()).isEqualTo(boolValue); + } + } + + @Test + public void walRecoveryMode() { + try (final Options opt = new Options()) { + for (final WALRecoveryMode walRecoveryMode : WALRecoveryMode.values()) { + opt.setWalRecoveryMode(walRecoveryMode); + assertThat(opt.walRecoveryMode()).isEqualTo(walRecoveryMode); + } + } + } + + @Test + public void allow2pc() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAllow2pc(boolValue); + assertThat(opt.allow2pc()).isEqualTo(boolValue); + } + } + + @Test + public void rowCache() { + try (final Options opt = new Options()) { + assertThat(opt.rowCache()).isNull(); + + try(final Cache lruCache = new LRUCache(1000)) { + opt.setRowCache(lruCache); + assertThat(opt.rowCache()).isEqualTo(lruCache); + } + + try(final Cache clockCache = new ClockCache(1000)) { + opt.setRowCache(clockCache); + assertThat(opt.rowCache()).isEqualTo(clockCache); + } + } + } + + @Test + public void failIfOptionsFileError() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setFailIfOptionsFileError(boolValue); + assertThat(opt.failIfOptionsFileError()).isEqualTo(boolValue); + } + } + + @Test + public void dumpMallocStats() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setDumpMallocStats(boolValue); + assertThat(opt.dumpMallocStats()).isEqualTo(boolValue); + } + } + + @Test + public void avoidFlushDuringRecovery() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAvoidFlushDuringRecovery(boolValue); + assertThat(opt.avoidFlushDuringRecovery()).isEqualTo(boolValue); + } + } + + @Test + public void avoidFlushDuringShutdown() { + try (final Options opt = new Options()) { + final boolean boolValue = rand.nextBoolean(); + opt.setAvoidFlushDuringShutdown(boolValue); + assertThat(opt.avoidFlushDuringShutdown()).isEqualTo(boolValue); + } + } + + @Test + public void env() { + try (final Options options = new Options(); + final Env env = Env.getDefault()) { + options.setEnv(env); + assertThat(options.getEnv()).isSameAs(env); + } + } + + @Test + public void linkageOfPrepMethods() { + try (final Options options = new Options()) { + options.optimizeUniversalStyleCompaction(); + options.optimizeUniversalStyleCompaction(4000); + options.optimizeLevelStyleCompaction(); + options.optimizeLevelStyleCompaction(3000); + options.optimizeForPointLookup(10); + options.optimizeForSmallDb(); + options.prepareForBulkLoad(); + } + } + + @Test + public void compressionTypes() { + try (final Options options = new Options()) { + for (final CompressionType compressionType : + CompressionType.values()) { + options.setCompressionType(compressionType); + assertThat(options.compressionType()). + isEqualTo(compressionType); + assertThat(CompressionType.valueOf("NO_COMPRESSION")). + isEqualTo(CompressionType.NO_COMPRESSION); + } + } + } + + @Test + public void compressionPerLevel() { + try (final Options options = new Options()) { + assertThat(options.compressionPerLevel()).isEmpty(); + List compressionTypeList = + new ArrayList<>(); + for (int i = 0; i < options.numLevels(); i++) { + compressionTypeList.add(CompressionType.NO_COMPRESSION); + } + options.setCompressionPerLevel(compressionTypeList); + compressionTypeList = options.compressionPerLevel(); + for (final CompressionType compressionType : compressionTypeList) { + assertThat(compressionType).isEqualTo( + CompressionType.NO_COMPRESSION); + } + } + } + + @Test + public void differentCompressionsPerLevel() { + try (final Options options = new Options()) { + options.setNumLevels(3); + + assertThat(options.compressionPerLevel()).isEmpty(); + List compressionTypeList = new ArrayList<>(); + + compressionTypeList.add(CompressionType.BZLIB2_COMPRESSION); + compressionTypeList.add(CompressionType.SNAPPY_COMPRESSION); + compressionTypeList.add(CompressionType.LZ4_COMPRESSION); + + options.setCompressionPerLevel(compressionTypeList); + compressionTypeList = options.compressionPerLevel(); + + assertThat(compressionTypeList.size()).isEqualTo(3); + assertThat(compressionTypeList). + containsExactly( + CompressionType.BZLIB2_COMPRESSION, + CompressionType.SNAPPY_COMPRESSION, + CompressionType.LZ4_COMPRESSION); + + } + } + + @Test + public void bottommostCompressionType() { + try (final Options options = new Options()) { + assertThat(options.bottommostCompressionType()) + .isEqualTo(CompressionType.DISABLE_COMPRESSION_OPTION); + + for (final CompressionType compressionType : CompressionType.values()) { + options.setBottommostCompressionType(compressionType); + assertThat(options.bottommostCompressionType()) + .isEqualTo(compressionType); + } + } + } + + @Test + public void compressionOptions() { + try (final Options options = new Options(); + final CompressionOptions compressionOptions = new CompressionOptions() + .setMaxDictBytes(123)) { + + options.setCompressionOptions(compressionOptions); + assertThat(options.compressionOptions()) + .isEqualTo(compressionOptions); + assertThat(options.compressionOptions().maxDictBytes()) + .isEqualTo(123); + } + } + + @Test + public void compactionStyles() { + try (final Options options = new Options()) { + for (final CompactionStyle compactionStyle : + CompactionStyle.values()) { + options.setCompactionStyle(compactionStyle); + assertThat(options.compactionStyle()). + isEqualTo(compactionStyle); + assertThat(CompactionStyle.valueOf("FIFO")). + isEqualTo(CompactionStyle.FIFO); + } + } + } + + @Test + public void maxTableFilesSizeFIFO() { + try (final Options opt = new Options()) { + long longValue = rand.nextLong(); + // Size has to be positive + longValue = (longValue < 0) ? -longValue : longValue; + longValue = (longValue == 0) ? longValue + 1 : longValue; + opt.setMaxTableFilesSizeFIFO(longValue); + assertThat(opt.maxTableFilesSizeFIFO()). + isEqualTo(longValue); + } + } + + @Test + public void rateLimiter() { + try (final Options options = new Options(); + final Options anotherOptions = new Options(); + final RateLimiter rateLimiter = + new RateLimiter(1000, 100 * 1000, 1)) { + options.setRateLimiter(rateLimiter); + // Test with parameter initialization + anotherOptions.setRateLimiter( + new RateLimiter(1000)); + } + } + + @Test + public void shouldSetTestPrefixExtractor() { + try (final Options options = new Options()) { + options.useFixedLengthPrefixExtractor(100); + options.useFixedLengthPrefixExtractor(10); + } + } + + @Test + public void shouldSetTestCappedPrefixExtractor() { + try (final Options options = new Options()) { + options.useCappedPrefixExtractor(100); + options.useCappedPrefixExtractor(10); + } + } + + @Test + public void shouldTestMemTableFactoryName() + throws RocksDBException { + try (final Options options = new Options()) { + options.setMemTableConfig(new VectorMemTableConfig()); + assertThat(options.memTableFactoryName()). + isEqualTo("VectorRepFactory"); + options.setMemTableConfig( + new HashLinkedListMemTableConfig()); + assertThat(options.memTableFactoryName()). + isEqualTo("HashLinkedListRepFactory"); + } + } + + @Test + public void statistics() { + try(final Options options = new Options()) { + final Statistics statistics = options.statistics(); + assertThat(statistics).isNull(); + } + + try(final Statistics statistics = new Statistics(); + final Options options = new Options().setStatistics(statistics); + final Statistics stats = options.statistics()) { + assertThat(stats).isNotNull(); + } + } + + @Test + public void maxWriteBufferNumberToMaintain() { + try (final Options options = new Options()) { + int intValue = rand.nextInt(); + // Size has to be positive + intValue = (intValue < 0) ? -intValue : intValue; + intValue = (intValue == 0) ? intValue + 1 : intValue; + options.setMaxWriteBufferNumberToMaintain(intValue); + assertThat(options.maxWriteBufferNumberToMaintain()). + isEqualTo(intValue); + } + } + + @Test + public void compactionPriorities() { + try (final Options options = new Options()) { + for (final CompactionPriority compactionPriority : + CompactionPriority.values()) { + options.setCompactionPriority(compactionPriority); + assertThat(options.compactionPriority()). + isEqualTo(compactionPriority); + } + } + } + + @Test + public void reportBgIoStats() { + try (final Options options = new Options()) { + final boolean booleanValue = true; + options.setReportBgIoStats(booleanValue); + assertThat(options.reportBgIoStats()). + isEqualTo(booleanValue); + } + } + + @Test + public void compactionOptionsUniversal() { + try (final Options options = new Options(); + final CompactionOptionsUniversal optUni = new CompactionOptionsUniversal() + .setCompressionSizePercent(7)) { + options.setCompactionOptionsUniversal(optUni); + assertThat(options.compactionOptionsUniversal()). + isEqualTo(optUni); + assertThat(options.compactionOptionsUniversal().compressionSizePercent()) + .isEqualTo(7); + } + } + + @Test + public void compactionOptionsFIFO() { + try (final Options options = new Options(); + final CompactionOptionsFIFO optFifo = new CompactionOptionsFIFO() + .setMaxTableFilesSize(2000)) { + options.setCompactionOptionsFIFO(optFifo); + assertThat(options.compactionOptionsFIFO()). + isEqualTo(optFifo); + assertThat(options.compactionOptionsFIFO().maxTableFilesSize()) + .isEqualTo(2000); + } + } + + @Test + public void forceConsistencyChecks() { + try (final Options options = new Options()) { + final boolean booleanValue = true; + options.setForceConsistencyChecks(booleanValue); + assertThat(options.forceConsistencyChecks()). + isEqualTo(booleanValue); + } + } +} diff --git a/java/src/test/java/org/rocksdb/PlainTableConfigTest.java b/java/src/test/java/org/rocksdb/PlainTableConfigTest.java new file mode 100644 index 00000000000..dcb6cc39f89 --- /dev/null +++ b/java/src/test/java/org/rocksdb/PlainTableConfigTest.java @@ -0,0 +1,89 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class PlainTableConfigTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void keySize() { + PlainTableConfig plainTableConfig = new PlainTableConfig(); + plainTableConfig.setKeySize(5); + assertThat(plainTableConfig.keySize()). + isEqualTo(5); + } + + @Test + public void bloomBitsPerKey() { + PlainTableConfig plainTableConfig = new PlainTableConfig(); + plainTableConfig.setBloomBitsPerKey(11); + assertThat(plainTableConfig.bloomBitsPerKey()). + isEqualTo(11); + } + + @Test + public void hashTableRatio() { + PlainTableConfig plainTableConfig = new PlainTableConfig(); + plainTableConfig.setHashTableRatio(0.95); + assertThat(plainTableConfig.hashTableRatio()). + isEqualTo(0.95); + } + + @Test + public void indexSparseness() { + PlainTableConfig plainTableConfig = new PlainTableConfig(); + plainTableConfig.setIndexSparseness(18); + assertThat(plainTableConfig.indexSparseness()). + isEqualTo(18); + } + + @Test + public void hugePageTlbSize() { + PlainTableConfig plainTableConfig = new PlainTableConfig(); + plainTableConfig.setHugePageTlbSize(1); + assertThat(plainTableConfig.hugePageTlbSize()). + isEqualTo(1); + } + + @Test + public void encodingType() { + PlainTableConfig plainTableConfig = new PlainTableConfig(); + plainTableConfig.setEncodingType(EncodingType.kPrefix); + assertThat(plainTableConfig.encodingType()).isEqualTo( + EncodingType.kPrefix); + } + + @Test + public void fullScanMode() { + PlainTableConfig plainTableConfig = new PlainTableConfig(); + plainTableConfig.setFullScanMode(true); + assertThat(plainTableConfig.fullScanMode()).isTrue(); } + + @Test + public void storeIndexInFile() { + PlainTableConfig plainTableConfig = new PlainTableConfig(); + plainTableConfig.setStoreIndexInFile(true); + assertThat(plainTableConfig.storeIndexInFile()). + isTrue(); + } + + @Test + public void plainTableConfig() { + try(final Options opt = new Options()) { + final PlainTableConfig plainTableConfig = new PlainTableConfig(); + opt.setTableFormatConfig(plainTableConfig); + assertThat(opt.tableFactoryName()).isEqualTo("PlainTable"); + } + } +} diff --git a/java/src/test/java/org/rocksdb/PlatformRandomHelper.java b/java/src/test/java/org/rocksdb/PlatformRandomHelper.java new file mode 100644 index 00000000000..80ea4d197f7 --- /dev/null +++ b/java/src/test/java/org/rocksdb/PlatformRandomHelper.java @@ -0,0 +1,58 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.Random; + +/** + * Helper class to get the appropriate Random class instance dependent + * on the current platform architecture (32bit vs 64bit) + */ +public class PlatformRandomHelper { + /** + * Determine if OS is 32-Bit/64-Bit + * + * @return boolean value indicating if operating system is 64 Bit. + */ + public static boolean isOs64Bit(){ + final boolean is64Bit; + if (System.getProperty("os.name").contains("Windows")) { + is64Bit = (System.getenv("ProgramFiles(x86)") != null); + } else { + is64Bit = (System.getProperty("os.arch").contains("64")); + } + return is64Bit; + } + + /** + * Factory to get a platform specific Random instance + * + * @return {@link java.util.Random} instance. + */ + public static Random getPlatformSpecificRandomFactory(){ + if (isOs64Bit()) { + return new Random(); + } + return new Random32Bit(); + } + + /** + * Random32Bit is a class which overrides {@code nextLong} to + * provide random numbers which fit in size_t. This workaround + * is necessary because there is no unsigned_int < Java 8 + */ + private static class Random32Bit extends Random { + @Override + public long nextLong(){ + return this.nextInt(Integer.MAX_VALUE); + } + } + + /** + * Utility class constructor + */ + private PlatformRandomHelper() { } +} diff --git a/java/src/test/java/org/rocksdb/RateLimiterTest.java b/java/src/test/java/org/rocksdb/RateLimiterTest.java new file mode 100644 index 00000000000..27567e89d1d --- /dev/null +++ b/java/src/test/java/org/rocksdb/RateLimiterTest.java @@ -0,0 +1,49 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class RateLimiterTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void setBytesPerSecond() { + try(final RateLimiter rateLimiter = + new RateLimiter(1000, 100 * 1000, 1)) { + rateLimiter.setBytesPerSecond(2000); + } + } + + @Test + public void getSingleBurstBytes() { + try(final RateLimiter rateLimiter = + new RateLimiter(1000, 100 * 1000, 1)) { + assertThat(rateLimiter.getSingleBurstBytes()).isEqualTo(100); + } + } + + @Test + public void getTotalBytesThrough() { + try(final RateLimiter rateLimiter = + new RateLimiter(1000, 100 * 1000, 1)) { + assertThat(rateLimiter.getTotalBytesThrough()).isEqualTo(0); + } + } + + @Test + public void getTotalRequests() { + try(final RateLimiter rateLimiter = + new RateLimiter(1000, 100 * 1000, 1)) { + assertThat(rateLimiter.getTotalRequests()).isEqualTo(0); + } + } +} diff --git a/java/src/test/java/org/rocksdb/ReadOnlyTest.java b/java/src/test/java/org/rocksdb/ReadOnlyTest.java new file mode 100644 index 00000000000..6b4c7b25961 --- /dev/null +++ b/java/src/test/java/org/rocksdb/ReadOnlyTest.java @@ -0,0 +1,305 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; + +public class ReadOnlyTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void readOnlyOpen() throws RocksDBException { + try (final Options options = new Options() + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + db.put("key".getBytes(), "value".getBytes()); + try (final RocksDB db2 = RocksDB.openReadOnly( + dbFolder.getRoot().getAbsolutePath())) { + assertThat("value"). + isEqualTo(new String(db2.get("key".getBytes()))); + } + } + + try (final ColumnFamilyOptions cfOpts = new ColumnFamilyOptions()) { + final List cfDescriptors = new ArrayList<>(); + cfDescriptors.add(new ColumnFamilyDescriptor( + RocksDB.DEFAULT_COLUMN_FAMILY, cfOpts)); + + final List columnFamilyHandleList = new ArrayList<>(); + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath(), + cfDescriptors, columnFamilyHandleList)) { + try (final ColumnFamilyOptions newCfOpts = new ColumnFamilyOptions(); + final ColumnFamilyOptions newCf2Opts = new ColumnFamilyOptions() + ) { + columnFamilyHandleList.add(db.createColumnFamily( + new ColumnFamilyDescriptor("new_cf".getBytes(), newCfOpts))); + columnFamilyHandleList.add(db.createColumnFamily( + new ColumnFamilyDescriptor("new_cf2".getBytes(), newCf2Opts))); + db.put(columnFamilyHandleList.get(2), "key2".getBytes(), + "value2".getBytes()); + + final List readOnlyColumnFamilyHandleList = + new ArrayList<>(); + try (final RocksDB db2 = RocksDB.openReadOnly( + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + readOnlyColumnFamilyHandleList)) { + try (final ColumnFamilyOptions newCfOpts2 = + new ColumnFamilyOptions(); + final ColumnFamilyOptions newCf2Opts2 = + new ColumnFamilyOptions() + ) { + assertThat(db2.get("key2".getBytes())).isNull(); + assertThat(db2.get(readOnlyColumnFamilyHandleList.get(0), + "key2".getBytes())). + isNull(); + cfDescriptors.clear(); + cfDescriptors.add( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, + newCfOpts2)); + cfDescriptors.add(new ColumnFamilyDescriptor("new_cf2".getBytes(), + newCf2Opts2)); + + final List readOnlyColumnFamilyHandleList2 + = new ArrayList<>(); + try (final RocksDB db3 = RocksDB.openReadOnly( + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + readOnlyColumnFamilyHandleList2)) { + try { + assertThat(new String(db3.get( + readOnlyColumnFamilyHandleList2.get(1), + "key2".getBytes()))).isEqualTo("value2"); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + readOnlyColumnFamilyHandleList2) { + columnFamilyHandle.close(); + } + } + } + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + readOnlyColumnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + } + + @Test(expected = RocksDBException.class) + public void failToWriteInReadOnly() throws RocksDBException { + try (final Options options = new Options() + .setCreateIfMissing(true)) { + + try (final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + //no-op + } + } + + try (final ColumnFamilyOptions cfOpts = new ColumnFamilyOptions()) { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, cfOpts) + ); + + final List readOnlyColumnFamilyHandleList = + new ArrayList<>(); + try (final RocksDB rDb = RocksDB.openReadOnly( + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + readOnlyColumnFamilyHandleList)) { + try { + // test that put fails in readonly mode + rDb.put("key".getBytes(), "value".getBytes()); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + readOnlyColumnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + } + + @Test(expected = RocksDBException.class) + public void failToCFWriteInReadOnly() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + //no-op + } + + try (final ColumnFamilyOptions cfOpts = new ColumnFamilyOptions()) { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, cfOpts) + ); + final List readOnlyColumnFamilyHandleList = + new ArrayList<>(); + try (final RocksDB rDb = RocksDB.openReadOnly( + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + readOnlyColumnFamilyHandleList)) { + try { + rDb.put(readOnlyColumnFamilyHandleList.get(0), + "key".getBytes(), "value".getBytes()); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + readOnlyColumnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + } + + @Test(expected = RocksDBException.class) + public void failToRemoveInReadOnly() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + //no-op + } + + try (final ColumnFamilyOptions cfOpts = new ColumnFamilyOptions()) { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, cfOpts) + ); + + final List readOnlyColumnFamilyHandleList = + new ArrayList<>(); + + try (final RocksDB rDb = RocksDB.openReadOnly( + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + readOnlyColumnFamilyHandleList)) { + try { + rDb.remove("key".getBytes()); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + readOnlyColumnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + } + + @Test(expected = RocksDBException.class) + public void failToCFRemoveInReadOnly() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + //no-op + } + + try (final ColumnFamilyOptions cfOpts = new ColumnFamilyOptions()) { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, cfOpts) + ); + + final List readOnlyColumnFamilyHandleList = + new ArrayList<>(); + try (final RocksDB rDb = RocksDB.openReadOnly( + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + readOnlyColumnFamilyHandleList)) { + try { + rDb.remove(readOnlyColumnFamilyHandleList.get(0), + "key".getBytes()); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + readOnlyColumnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + } + + @Test(expected = RocksDBException.class) + public void failToWriteBatchReadOnly() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + //no-op + } + + try (final ColumnFamilyOptions cfOpts = new ColumnFamilyOptions()) { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, cfOpts) + ); + + final List readOnlyColumnFamilyHandleList = + new ArrayList<>(); + try (final RocksDB rDb = RocksDB.openReadOnly( + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + readOnlyColumnFamilyHandleList); + final WriteBatch wb = new WriteBatch(); + final WriteOptions wOpts = new WriteOptions()) { + try { + wb.put("key".getBytes(), "value".getBytes()); + rDb.write(wOpts, wb); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + readOnlyColumnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + } + + @Test(expected = RocksDBException.class) + public void failToCFWriteBatchReadOnly() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + //no-op + } + + try (final ColumnFamilyOptions cfOpts = new ColumnFamilyOptions()) { + final List cfDescriptors = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY, cfOpts) + ); + + final List readOnlyColumnFamilyHandleList = + new ArrayList<>(); + try (final RocksDB rDb = RocksDB.openReadOnly( + dbFolder.getRoot().getAbsolutePath(), cfDescriptors, + readOnlyColumnFamilyHandleList); + final WriteBatch wb = new WriteBatch(); + final WriteOptions wOpts = new WriteOptions()) { + try { + wb.put(readOnlyColumnFamilyHandleList.get(0), "key".getBytes(), + "value".getBytes()); + rDb.write(wOpts, wb); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + readOnlyColumnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/ReadOptionsTest.java b/java/src/test/java/org/rocksdb/ReadOptionsTest.java new file mode 100644 index 00000000000..da048c4431e --- /dev/null +++ b/java/src/test/java/org/rocksdb/ReadOptionsTest.java @@ -0,0 +1,201 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.Random; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.ExpectedException; + +import static org.assertj.core.api.Assertions.assertThat; + +public class ReadOptionsTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public ExpectedException exception = ExpectedException.none(); + + @Test + public void verifyChecksum() { + try (final ReadOptions opt = new ReadOptions()) { + final Random rand = new Random(); + final boolean boolValue = rand.nextBoolean(); + opt.setVerifyChecksums(boolValue); + assertThat(opt.verifyChecksums()).isEqualTo(boolValue); + } + } + + @Test + public void fillCache() { + try (final ReadOptions opt = new ReadOptions()) { + final Random rand = new Random(); + final boolean boolValue = rand.nextBoolean(); + opt.setFillCache(boolValue); + assertThat(opt.fillCache()).isEqualTo(boolValue); + } + } + + @Test + public void tailing() { + try (final ReadOptions opt = new ReadOptions()) { + final Random rand = new Random(); + final boolean boolValue = rand.nextBoolean(); + opt.setTailing(boolValue); + assertThat(opt.tailing()).isEqualTo(boolValue); + } + } + + @Test + public void snapshot() { + try (final ReadOptions opt = new ReadOptions()) { + opt.setSnapshot(null); + assertThat(opt.snapshot()).isNull(); + } + } + + @Test + public void readTier() { + try (final ReadOptions opt = new ReadOptions()) { + opt.setReadTier(ReadTier.BLOCK_CACHE_TIER); + assertThat(opt.readTier()).isEqualTo(ReadTier.BLOCK_CACHE_TIER); + } + } + + @Test + public void managed() { + try (final ReadOptions opt = new ReadOptions()) { + opt.setManaged(true); + assertThat(opt.managed()).isTrue(); + } + } + + @Test + public void totalOrderSeek() { + try (final ReadOptions opt = new ReadOptions()) { + opt.setTotalOrderSeek(true); + assertThat(opt.totalOrderSeek()).isTrue(); + } + } + + @Test + public void prefixSameAsStart() { + try (final ReadOptions opt = new ReadOptions()) { + opt.setPrefixSameAsStart(true); + assertThat(opt.prefixSameAsStart()).isTrue(); + } + } + + @Test + public void pinData() { + try (final ReadOptions opt = new ReadOptions()) { + opt.setPinData(true); + assertThat(opt.pinData()).isTrue(); + } + } + + @Test + public void backgroundPurgeOnIteratorCleanup() { + try (final ReadOptions opt = new ReadOptions()) { + opt.setBackgroundPurgeOnIteratorCleanup(true); + assertThat(opt.backgroundPurgeOnIteratorCleanup()).isTrue(); + } + } + + @Test + public void readaheadSize() { + try (final ReadOptions opt = new ReadOptions()) { + final Random rand = new Random(); + final long longValue = rand.nextLong(); + opt.setReadaheadSize(longValue); + assertThat(opt.readaheadSize()).isEqualTo(longValue); + } + } + + @Test + public void ignoreRangeDeletions() { + try (final ReadOptions opt = new ReadOptions()) { + opt.setIgnoreRangeDeletions(true); + assertThat(opt.ignoreRangeDeletions()).isTrue(); + } + } + + @Test + public void failSetVerifyChecksumUninitialized() { + try (final ReadOptions readOptions = + setupUninitializedReadOptions(exception)) { + readOptions.setVerifyChecksums(true); + } + } + + @Test + public void failVerifyChecksumUninitialized() { + try (final ReadOptions readOptions = + setupUninitializedReadOptions(exception)) { + readOptions.verifyChecksums(); + } + } + + @Test + public void failSetFillCacheUninitialized() { + try (final ReadOptions readOptions = + setupUninitializedReadOptions(exception)) { + readOptions.setFillCache(true); + } + } + + @Test + public void failFillCacheUninitialized() { + try (final ReadOptions readOptions = + setupUninitializedReadOptions(exception)) { + readOptions.fillCache(); + } + } + + @Test + public void failSetTailingUninitialized() { + try (final ReadOptions readOptions = + setupUninitializedReadOptions(exception)) { + readOptions.setTailing(true); + } + } + + @Test + public void failTailingUninitialized() { + try (final ReadOptions readOptions = + setupUninitializedReadOptions(exception)) { + readOptions.tailing(); + } + } + + @Test + public void failSetSnapshotUninitialized() { + try (final ReadOptions readOptions = + setupUninitializedReadOptions(exception)) { + readOptions.setSnapshot(null); + } + } + + @Test + public void failSnapshotUninitialized() { + try (final ReadOptions readOptions = + setupUninitializedReadOptions(exception)) { + readOptions.snapshot(); + } + } + + private ReadOptions setupUninitializedReadOptions( + ExpectedException exception) { + final ReadOptions readOptions = new ReadOptions(); + readOptions.close(); + exception.expect(AssertionError.class); + return readOptions; + } +} diff --git a/java/src/test/java/org/rocksdb/RocksDBExceptionTest.java b/java/src/test/java/org/rocksdb/RocksDBExceptionTest.java new file mode 100644 index 00000000000..d3bd4ece7f1 --- /dev/null +++ b/java/src/test/java/org/rocksdb/RocksDBExceptionTest.java @@ -0,0 +1,115 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.Test; + +import org.rocksdb.Status.Code; +import org.rocksdb.Status.SubCode; +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.Assert.fail; + +public class RocksDBExceptionTest { + + @Test + public void exception() { + try { + raiseException(); + } catch(final RocksDBException e) { + assertThat(e.getStatus()).isNull(); + assertThat(e.getMessage()).isEqualTo("test message"); + return; + } + fail(); + } + + @Test + public void exceptionWithStatusCode() { + try { + raiseExceptionWithStatusCode(); + } catch(final RocksDBException e) { + assertThat(e.getStatus()).isNotNull(); + assertThat(e.getStatus().getCode()).isEqualTo(Code.NotSupported); + assertThat(e.getStatus().getSubCode()).isEqualTo(SubCode.None); + assertThat(e.getStatus().getState()).isNull(); + assertThat(e.getMessage()).isEqualTo("test message"); + return; + } + fail(); + } + + @Test + public void exceptionNoMsgWithStatusCode() { + try { + raiseExceptionNoMsgWithStatusCode(); + } catch(final RocksDBException e) { + assertThat(e.getStatus()).isNotNull(); + assertThat(e.getStatus().getCode()).isEqualTo(Code.NotSupported); + assertThat(e.getStatus().getSubCode()).isEqualTo(SubCode.None); + assertThat(e.getStatus().getState()).isNull(); + assertThat(e.getMessage()).isEqualTo(Code.NotSupported.name()); + return; + } + fail(); + } + + @Test + public void exceptionWithStatusCodeSubCode() { + try { + raiseExceptionWithStatusCodeSubCode(); + } catch(final RocksDBException e) { + assertThat(e.getStatus()).isNotNull(); + assertThat(e.getStatus().getCode()).isEqualTo(Code.TimedOut); + assertThat(e.getStatus().getSubCode()) + .isEqualTo(Status.SubCode.LockTimeout); + assertThat(e.getStatus().getState()).isNull(); + assertThat(e.getMessage()).isEqualTo("test message"); + return; + } + fail(); + } + + @Test + public void exceptionNoMsgWithStatusCodeSubCode() { + try { + raiseExceptionNoMsgWithStatusCodeSubCode(); + } catch(final RocksDBException e) { + assertThat(e.getStatus()).isNotNull(); + assertThat(e.getStatus().getCode()).isEqualTo(Code.TimedOut); + assertThat(e.getStatus().getSubCode()).isEqualTo(SubCode.LockTimeout); + assertThat(e.getStatus().getState()).isNull(); + assertThat(e.getMessage()).isEqualTo(Code.TimedOut.name() + + "(" + SubCode.LockTimeout.name() + ")"); + return; + } + fail(); + } + + @Test + public void exceptionWithStatusCodeState() { + try { + raiseExceptionWithStatusCodeState(); + } catch(final RocksDBException e) { + assertThat(e.getStatus()).isNotNull(); + assertThat(e.getStatus().getCode()).isEqualTo(Code.NotSupported); + assertThat(e.getStatus().getSubCode()).isEqualTo(SubCode.None); + assertThat(e.getStatus().getState()).isNotNull(); + assertThat(e.getMessage()).isEqualTo("test message"); + return; + } + fail(); + } + + private native void raiseException() throws RocksDBException; + private native void raiseExceptionWithStatusCode() throws RocksDBException; + private native void raiseExceptionNoMsgWithStatusCode() throws RocksDBException; + private native void raiseExceptionWithStatusCodeSubCode() + throws RocksDBException; + private native void raiseExceptionNoMsgWithStatusCodeSubCode() + throws RocksDBException; + private native void raiseExceptionWithStatusCodeState() + throws RocksDBException; +} diff --git a/java/src/test/java/org/rocksdb/RocksDBTest.java b/java/src/test/java/org/rocksdb/RocksDBTest.java new file mode 100644 index 00000000000..89894746d2c --- /dev/null +++ b/java/src/test/java/org/rocksdb/RocksDBTest.java @@ -0,0 +1,766 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.util.*; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.Assert.fail; + +public class RocksDBTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + public static final Random rand = PlatformRandomHelper. + getPlatformSpecificRandomFactory(); + + @Test + public void open() throws RocksDBException { + try (final RocksDB db = + RocksDB.open(dbFolder.getRoot().getAbsolutePath())) { + assertThat(db).isNotNull(); + } + } + + @Test + public void open_opt() throws RocksDBException { + try (final Options opt = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + assertThat(db).isNotNull(); + } + } + + @Test + public void openWhenOpen() throws RocksDBException { + final String dbPath = dbFolder.getRoot().getAbsolutePath(); + + try (final RocksDB db1 = RocksDB.open(dbPath)) { + try (final RocksDB db2 = RocksDB.open(dbPath)) { + fail("Should have thrown an exception when opening the same db twice"); + } catch (final RocksDBException e) { + assertThat(e.getStatus().getCode()).isEqualTo(Status.Code.IOError); + assertThat(e.getStatus().getSubCode()).isEqualTo(Status.SubCode.None); + assertThat(e.getStatus().getState()).contains("lock "); + } + } + } + + @Test + public void put() throws RocksDBException { + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath()); + final WriteOptions opt = new WriteOptions()) { + db.put("key1".getBytes(), "value".getBytes()); + db.put(opt, "key2".getBytes(), "12345678".getBytes()); + assertThat(db.get("key1".getBytes())).isEqualTo( + "value".getBytes()); + assertThat(db.get("key2".getBytes())).isEqualTo( + "12345678".getBytes()); + } + } + + @Test + public void write() throws RocksDBException { + try (final StringAppendOperator stringAppendOperator = new StringAppendOperator(); + final Options options = new Options() + .setMergeOperator(stringAppendOperator) + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath()); + final WriteOptions opts = new WriteOptions()) { + + try (final WriteBatch wb1 = new WriteBatch()) { + wb1.put("key1".getBytes(), "aa".getBytes()); + wb1.merge("key1".getBytes(), "bb".getBytes()); + + try (final WriteBatch wb2 = new WriteBatch()) { + wb2.put("key2".getBytes(), "xx".getBytes()); + wb2.merge("key2".getBytes(), "yy".getBytes()); + db.write(opts, wb1); + db.write(opts, wb2); + } + } + + assertThat(db.get("key1".getBytes())).isEqualTo( + "aa,bb".getBytes()); + assertThat(db.get("key2".getBytes())).isEqualTo( + "xx,yy".getBytes()); + } + } + + @Test + public void getWithOutValue() throws RocksDBException { + try (final RocksDB db = + RocksDB.open(dbFolder.getRoot().getAbsolutePath())) { + db.put("key1".getBytes(), "value".getBytes()); + db.put("key2".getBytes(), "12345678".getBytes()); + byte[] outValue = new byte[5]; + // not found value + int getResult = db.get("keyNotFound".getBytes(), outValue); + assertThat(getResult).isEqualTo(RocksDB.NOT_FOUND); + // found value which fits in outValue + getResult = db.get("key1".getBytes(), outValue); + assertThat(getResult).isNotEqualTo(RocksDB.NOT_FOUND); + assertThat(outValue).isEqualTo("value".getBytes()); + // found value which fits partially + getResult = db.get("key2".getBytes(), outValue); + assertThat(getResult).isNotEqualTo(RocksDB.NOT_FOUND); + assertThat(outValue).isEqualTo("12345".getBytes()); + } + } + + @Test + public void getWithOutValueReadOptions() throws RocksDBException { + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath()); + final ReadOptions rOpt = new ReadOptions()) { + db.put("key1".getBytes(), "value".getBytes()); + db.put("key2".getBytes(), "12345678".getBytes()); + byte[] outValue = new byte[5]; + // not found value + int getResult = db.get(rOpt, "keyNotFound".getBytes(), + outValue); + assertThat(getResult).isEqualTo(RocksDB.NOT_FOUND); + // found value which fits in outValue + getResult = db.get(rOpt, "key1".getBytes(), outValue); + assertThat(getResult).isNotEqualTo(RocksDB.NOT_FOUND); + assertThat(outValue).isEqualTo("value".getBytes()); + // found value which fits partially + getResult = db.get(rOpt, "key2".getBytes(), outValue); + assertThat(getResult).isNotEqualTo(RocksDB.NOT_FOUND); + assertThat(outValue).isEqualTo("12345".getBytes()); + } + } + + @Test + public void multiGet() throws RocksDBException, InterruptedException { + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath()); + final ReadOptions rOpt = new ReadOptions()) { + db.put("key1".getBytes(), "value".getBytes()); + db.put("key2".getBytes(), "12345678".getBytes()); + List lookupKeys = new ArrayList<>(); + lookupKeys.add("key1".getBytes()); + lookupKeys.add("key2".getBytes()); + Map results = db.multiGet(lookupKeys); + assertThat(results).isNotNull(); + assertThat(results.values()).isNotNull(); + assertThat(results.values()). + contains("value".getBytes(), "12345678".getBytes()); + // test same method with ReadOptions + results = db.multiGet(rOpt, lookupKeys); + assertThat(results).isNotNull(); + assertThat(results.values()).isNotNull(); + assertThat(results.values()). + contains("value".getBytes(), "12345678".getBytes()); + + // remove existing key + lookupKeys.remove("key2".getBytes()); + // add non existing key + lookupKeys.add("key3".getBytes()); + results = db.multiGet(lookupKeys); + assertThat(results).isNotNull(); + assertThat(results.values()).isNotNull(); + assertThat(results.values()). + contains("value".getBytes()); + // test same call with readOptions + results = db.multiGet(rOpt, lookupKeys); + assertThat(results).isNotNull(); + assertThat(results.values()).isNotNull(); + assertThat(results.values()). + contains("value".getBytes()); + } + } + + @Test + public void merge() throws RocksDBException { + try (final StringAppendOperator stringAppendOperator = new StringAppendOperator(); + final Options opt = new Options() + .setCreateIfMissing(true) + .setMergeOperator(stringAppendOperator); + final WriteOptions wOpt = new WriteOptions(); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath()) + ) { + db.put("key1".getBytes(), "value".getBytes()); + assertThat(db.get("key1".getBytes())).isEqualTo( + "value".getBytes()); + // merge key1 with another value portion + db.merge("key1".getBytes(), "value2".getBytes()); + assertThat(db.get("key1".getBytes())).isEqualTo( + "value,value2".getBytes()); + // merge key1 with another value portion + db.merge(wOpt, "key1".getBytes(), "value3".getBytes()); + assertThat(db.get("key1".getBytes())).isEqualTo( + "value,value2,value3".getBytes()); + // merge on non existent key shall insert the value + db.merge(wOpt, "key2".getBytes(), "xxxx".getBytes()); + assertThat(db.get("key2".getBytes())).isEqualTo( + "xxxx".getBytes()); + } + } + + @Test + public void delete() throws RocksDBException { + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath()); + final WriteOptions wOpt = new WriteOptions()) { + db.put("key1".getBytes(), "value".getBytes()); + db.put("key2".getBytes(), "12345678".getBytes()); + assertThat(db.get("key1".getBytes())).isEqualTo( + "value".getBytes()); + assertThat(db.get("key2".getBytes())).isEqualTo( + "12345678".getBytes()); + db.delete("key1".getBytes()); + db.delete(wOpt, "key2".getBytes()); + assertThat(db.get("key1".getBytes())).isNull(); + assertThat(db.get("key2".getBytes())).isNull(); + } + } + + @Test + public void singleDelete() throws RocksDBException { + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath()); + final WriteOptions wOpt = new WriteOptions()) { + db.put("key1".getBytes(), "value".getBytes()); + db.put("key2".getBytes(), "12345678".getBytes()); + assertThat(db.get("key1".getBytes())).isEqualTo( + "value".getBytes()); + assertThat(db.get("key2".getBytes())).isEqualTo( + "12345678".getBytes()); + db.singleDelete("key1".getBytes()); + db.singleDelete(wOpt, "key2".getBytes()); + assertThat(db.get("key1".getBytes())).isNull(); + assertThat(db.get("key2".getBytes())).isNull(); + } + } + + @Test + public void singleDelete_nonExisting() throws RocksDBException { + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath()); + final WriteOptions wOpt = new WriteOptions()) { + db.singleDelete("key1".getBytes()); + db.singleDelete(wOpt, "key2".getBytes()); + assertThat(db.get("key1".getBytes())).isNull(); + assertThat(db.get("key2".getBytes())).isNull(); + } + } + + @Test + public void deleteRange() throws RocksDBException { + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath()); + final WriteOptions wOpt = new WriteOptions()) { + db.put("key1".getBytes(), "value".getBytes()); + db.put("key2".getBytes(), "12345678".getBytes()); + db.put("key3".getBytes(), "abcdefg".getBytes()); + db.put("key4".getBytes(), "xyz".getBytes()); + assertThat(db.get("key1".getBytes())).isEqualTo("value".getBytes()); + assertThat(db.get("key2".getBytes())).isEqualTo("12345678".getBytes()); + assertThat(db.get("key3".getBytes())).isEqualTo("abcdefg".getBytes()); + assertThat(db.get("key4".getBytes())).isEqualTo("xyz".getBytes()); + db.deleteRange("key2".getBytes(), "key4".getBytes()); + assertThat(db.get("key1".getBytes())).isEqualTo("value".getBytes()); + assertThat(db.get("key2".getBytes())).isNull(); + assertThat(db.get("key3".getBytes())).isNull(); + assertThat(db.get("key4".getBytes())).isEqualTo("xyz".getBytes()); + } + } + + @Test + public void getIntProperty() throws RocksDBException { + try ( + final Options options = new Options() + .setCreateIfMissing(true) + .setMaxWriteBufferNumber(10) + .setMinWriteBufferNumberToMerge(10); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath()); + final WriteOptions wOpt = new WriteOptions().setDisableWAL(true) + ) { + db.put(wOpt, "key1".getBytes(), "value1".getBytes()); + db.put(wOpt, "key2".getBytes(), "value2".getBytes()); + db.put(wOpt, "key3".getBytes(), "value3".getBytes()); + db.put(wOpt, "key4".getBytes(), "value4".getBytes()); + assertThat(db.getLongProperty("rocksdb.num-entries-active-mem-table")) + .isGreaterThan(0); + assertThat(db.getLongProperty("rocksdb.cur-size-active-mem-table")) + .isGreaterThan(0); + } + } + + @Test + public void fullCompactRange() throws RocksDBException { + try (final Options opt = new Options(). + setCreateIfMissing(true). + setDisableAutoCompactions(true). + setCompactionStyle(CompactionStyle.LEVEL). + setNumLevels(4). + setWriteBufferSize(100 << 10). + setLevelZeroFileNumCompactionTrigger(3). + setTargetFileSizeBase(200 << 10). + setTargetFileSizeMultiplier(1). + setMaxBytesForLevelBase(500 << 10). + setMaxBytesForLevelMultiplier(1). + setDisableAutoCompactions(false); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + // fill database with key/value pairs + byte[] b = new byte[10000]; + for (int i = 0; i < 200; i++) { + rand.nextBytes(b); + db.put((String.valueOf(i)).getBytes(), b); + } + db.compactRange(); + } + } + + @Test + public void fullCompactRangeColumnFamily() + throws RocksDBException { + try ( + final DBOptions opt = new DBOptions(). + setCreateIfMissing(true). + setCreateMissingColumnFamilies(true); + final ColumnFamilyOptions new_cf_opts = new ColumnFamilyOptions(). + setDisableAutoCompactions(true). + setCompactionStyle(CompactionStyle.LEVEL). + setNumLevels(4). + setWriteBufferSize(100 << 10). + setLevelZeroFileNumCompactionTrigger(3). + setTargetFileSizeBase(200 << 10). + setTargetFileSizeMultiplier(1). + setMaxBytesForLevelBase(500 << 10). + setMaxBytesForLevelMultiplier(1). + setDisableAutoCompactions(false) + ) { + final List columnFamilyDescriptors = + Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes(), new_cf_opts)); + + // open database + final List columnFamilyHandles = new ArrayList<>(); + try (final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath(), + columnFamilyDescriptors, + columnFamilyHandles)) { + try { + // fill database with key/value pairs + byte[] b = new byte[10000]; + for (int i = 0; i < 200; i++) { + rand.nextBytes(b); + db.put(columnFamilyHandles.get(1), + String.valueOf(i).getBytes(), b); + } + db.compactRange(columnFamilyHandles.get(1)); + } finally { + for (final ColumnFamilyHandle handle : columnFamilyHandles) { + handle.close(); + } + } + } + } + } + + @Test + public void compactRangeWithKeys() + throws RocksDBException { + try (final Options opt = new Options(). + setCreateIfMissing(true). + setDisableAutoCompactions(true). + setCompactionStyle(CompactionStyle.LEVEL). + setNumLevels(4). + setWriteBufferSize(100 << 10). + setLevelZeroFileNumCompactionTrigger(3). + setTargetFileSizeBase(200 << 10). + setTargetFileSizeMultiplier(1). + setMaxBytesForLevelBase(500 << 10). + setMaxBytesForLevelMultiplier(1). + setDisableAutoCompactions(false); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + // fill database with key/value pairs + byte[] b = new byte[10000]; + for (int i = 0; i < 200; i++) { + rand.nextBytes(b); + db.put((String.valueOf(i)).getBytes(), b); + } + db.compactRange("0".getBytes(), "201".getBytes()); + } + } + + @Test + public void compactRangeWithKeysReduce() + throws RocksDBException { + try ( + final Options opt = new Options(). + setCreateIfMissing(true). + setDisableAutoCompactions(true). + setCompactionStyle(CompactionStyle.LEVEL). + setNumLevels(4). + setWriteBufferSize(100 << 10). + setLevelZeroFileNumCompactionTrigger(3). + setTargetFileSizeBase(200 << 10). + setTargetFileSizeMultiplier(1). + setMaxBytesForLevelBase(500 << 10). + setMaxBytesForLevelMultiplier(1). + setDisableAutoCompactions(false); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + // fill database with key/value pairs + byte[] b = new byte[10000]; + for (int i = 0; i < 200; i++) { + rand.nextBytes(b); + db.put((String.valueOf(i)).getBytes(), b); + } + db.flush(new FlushOptions().setWaitForFlush(true)); + db.compactRange("0".getBytes(), "201".getBytes(), + true, -1, 0); + } + } + + @Test + public void compactRangeWithKeysColumnFamily() + throws RocksDBException { + try (final DBOptions opt = new DBOptions(). + setCreateIfMissing(true). + setCreateMissingColumnFamilies(true); + final ColumnFamilyOptions new_cf_opts = new ColumnFamilyOptions(). + setDisableAutoCompactions(true). + setCompactionStyle(CompactionStyle.LEVEL). + setNumLevels(4). + setWriteBufferSize(100 << 10). + setLevelZeroFileNumCompactionTrigger(3). + setTargetFileSizeBase(200 << 10). + setTargetFileSizeMultiplier(1). + setMaxBytesForLevelBase(500 << 10). + setMaxBytesForLevelMultiplier(1). + setDisableAutoCompactions(false) + ) { + final List columnFamilyDescriptors = + Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes(), new_cf_opts) + ); + + // open database + final List columnFamilyHandles = + new ArrayList<>(); + try (final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath(), + columnFamilyDescriptors, + columnFamilyHandles)) { + try { + // fill database with key/value pairs + byte[] b = new byte[10000]; + for (int i = 0; i < 200; i++) { + rand.nextBytes(b); + db.put(columnFamilyHandles.get(1), + String.valueOf(i).getBytes(), b); + } + db.compactRange(columnFamilyHandles.get(1), + "0".getBytes(), "201".getBytes()); + } finally { + for (final ColumnFamilyHandle handle : columnFamilyHandles) { + handle.close(); + } + } + } + } + } + + @Test + public void compactRangeWithKeysReduceColumnFamily() + throws RocksDBException { + try (final DBOptions opt = new DBOptions(). + setCreateIfMissing(true). + setCreateMissingColumnFamilies(true); + final ColumnFamilyOptions new_cf_opts = new ColumnFamilyOptions(). + setDisableAutoCompactions(true). + setCompactionStyle(CompactionStyle.LEVEL). + setNumLevels(4). + setWriteBufferSize(100 << 10). + setLevelZeroFileNumCompactionTrigger(3). + setTargetFileSizeBase(200 << 10). + setTargetFileSizeMultiplier(1). + setMaxBytesForLevelBase(500 << 10). + setMaxBytesForLevelMultiplier(1). + setDisableAutoCompactions(false) + ) { + final List columnFamilyDescriptors = + Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes(), new_cf_opts) + ); + + final List columnFamilyHandles = new ArrayList<>(); + // open database + try (final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath(), + columnFamilyDescriptors, + columnFamilyHandles)) { + try { + // fill database with key/value pairs + byte[] b = new byte[10000]; + for (int i = 0; i < 200; i++) { + rand.nextBytes(b); + db.put(columnFamilyHandles.get(1), + String.valueOf(i).getBytes(), b); + } + db.compactRange(columnFamilyHandles.get(1), "0".getBytes(), + "201".getBytes(), true, -1, 0); + } finally { + for (final ColumnFamilyHandle handle : columnFamilyHandles) { + handle.close(); + } + } + } + } + } + + @Test + public void compactRangeToLevel() + throws RocksDBException, InterruptedException { + final int NUM_KEYS_PER_L0_FILE = 100; + final int KEY_SIZE = 20; + final int VALUE_SIZE = 300; + final int L0_FILE_SIZE = + NUM_KEYS_PER_L0_FILE * (KEY_SIZE + VALUE_SIZE); + final int NUM_L0_FILES = 10; + final int TEST_SCALE = 5; + final int KEY_INTERVAL = 100; + try (final Options opt = new Options(). + setCreateIfMissing(true). + setCompactionStyle(CompactionStyle.LEVEL). + setNumLevels(5). + // a slightly bigger write buffer than L0 file + // so that we can ensure manual flush always + // go before background flush happens. + setWriteBufferSize(L0_FILE_SIZE * 2). + // Disable auto L0 -> L1 compaction + setLevelZeroFileNumCompactionTrigger(20). + setTargetFileSizeBase(L0_FILE_SIZE * 100). + setTargetFileSizeMultiplier(1). + // To disable auto compaction + setMaxBytesForLevelBase(NUM_L0_FILES * L0_FILE_SIZE * 100). + setMaxBytesForLevelMultiplier(2). + setDisableAutoCompactions(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath()) + ) { + // fill database with key/value pairs + byte[] value = new byte[VALUE_SIZE]; + int int_key = 0; + for (int round = 0; round < 5; ++round) { + int initial_key = int_key; + for (int f = 1; f <= NUM_L0_FILES; ++f) { + for (int i = 0; i < NUM_KEYS_PER_L0_FILE; ++i) { + int_key += KEY_INTERVAL; + rand.nextBytes(value); + + db.put(String.format("%020d", int_key).getBytes(), + value); + } + db.flush(new FlushOptions().setWaitForFlush(true)); + // Make sure we do create one more L0 files. + assertThat( + db.getProperty("rocksdb.num-files-at-level0")). + isEqualTo("" + f); + } + + // Compact all L0 files we just created + db.compactRange( + String.format("%020d", initial_key).getBytes(), + String.format("%020d", int_key - 1).getBytes()); + // Making sure there isn't any L0 files. + assertThat( + db.getProperty("rocksdb.num-files-at-level0")). + isEqualTo("0"); + // Making sure there are some L1 files. + // Here we only use != 0 instead of a specific number + // as we don't want the test make any assumption on + // how compaction works. + assertThat( + db.getProperty("rocksdb.num-files-at-level1")). + isNotEqualTo("0"); + // Because we only compacted those keys we issued + // in this round, there shouldn't be any L1 -> L2 + // compaction. So we expect zero L2 files here. + assertThat( + db.getProperty("rocksdb.num-files-at-level2")). + isEqualTo("0"); + } + } + } + + @Test + public void compactRangeToLevelColumnFamily() + throws RocksDBException { + final int NUM_KEYS_PER_L0_FILE = 100; + final int KEY_SIZE = 20; + final int VALUE_SIZE = 300; + final int L0_FILE_SIZE = + NUM_KEYS_PER_L0_FILE * (KEY_SIZE + VALUE_SIZE); + final int NUM_L0_FILES = 10; + final int TEST_SCALE = 5; + final int KEY_INTERVAL = 100; + + try (final DBOptions opt = new DBOptions(). + setCreateIfMissing(true). + setCreateMissingColumnFamilies(true); + final ColumnFamilyOptions new_cf_opts = new ColumnFamilyOptions(). + setCompactionStyle(CompactionStyle.LEVEL). + setNumLevels(5). + // a slightly bigger write buffer than L0 file + // so that we can ensure manual flush always + // go before background flush happens. + setWriteBufferSize(L0_FILE_SIZE * 2). + // Disable auto L0 -> L1 compaction + setLevelZeroFileNumCompactionTrigger(20). + setTargetFileSizeBase(L0_FILE_SIZE * 100). + setTargetFileSizeMultiplier(1). + // To disable auto compaction + setMaxBytesForLevelBase(NUM_L0_FILES * L0_FILE_SIZE * 100). + setMaxBytesForLevelMultiplier(2). + setDisableAutoCompactions(true) + ) { + final List columnFamilyDescriptors = + Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes(), new_cf_opts) + ); + + final List columnFamilyHandles = new ArrayList<>(); + // open database + try (final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath(), + columnFamilyDescriptors, + columnFamilyHandles)) { + try { + // fill database with key/value pairs + byte[] value = new byte[VALUE_SIZE]; + int int_key = 0; + for (int round = 0; round < 5; ++round) { + int initial_key = int_key; + for (int f = 1; f <= NUM_L0_FILES; ++f) { + for (int i = 0; i < NUM_KEYS_PER_L0_FILE; ++i) { + int_key += KEY_INTERVAL; + rand.nextBytes(value); + + db.put(columnFamilyHandles.get(1), + String.format("%020d", int_key).getBytes(), + value); + } + db.flush(new FlushOptions().setWaitForFlush(true), + columnFamilyHandles.get(1)); + // Make sure we do create one more L0 files. + assertThat( + db.getProperty(columnFamilyHandles.get(1), + "rocksdb.num-files-at-level0")). + isEqualTo("" + f); + } + + // Compact all L0 files we just created + db.compactRange( + columnFamilyHandles.get(1), + String.format("%020d", initial_key).getBytes(), + String.format("%020d", int_key - 1).getBytes()); + // Making sure there isn't any L0 files. + assertThat( + db.getProperty(columnFamilyHandles.get(1), + "rocksdb.num-files-at-level0")). + isEqualTo("0"); + // Making sure there are some L1 files. + // Here we only use != 0 instead of a specific number + // as we don't want the test make any assumption on + // how compaction works. + assertThat( + db.getProperty(columnFamilyHandles.get(1), + "rocksdb.num-files-at-level1")). + isNotEqualTo("0"); + // Because we only compacted those keys we issued + // in this round, there shouldn't be any L1 -> L2 + // compaction. So we expect zero L2 files here. + assertThat( + db.getProperty(columnFamilyHandles.get(1), + "rocksdb.num-files-at-level2")). + isEqualTo("0"); + } + } finally { + for (final ColumnFamilyHandle handle : columnFamilyHandles) { + handle.close(); + } + } + } + } + } + + @Test + public void pauseContinueBackgroundWork() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath()) + ) { + db.pauseBackgroundWork(); + db.continueBackgroundWork(); + db.pauseBackgroundWork(); + db.continueBackgroundWork(); + } + } + + @Test + public void enableDisableFileDeletions() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath()) + ) { + db.disableFileDeletions(); + db.enableFileDeletions(false); + db.disableFileDeletions(); + db.enableFileDeletions(true); + } + } + + @Test + public void setOptions() throws RocksDBException { + try (final DBOptions options = new DBOptions() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final ColumnFamilyOptions new_cf_opts = new ColumnFamilyOptions() + .setWriteBufferSize(4096)) { + + final List columnFamilyDescriptors = + Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes(), new_cf_opts)); + + // open database + final List columnFamilyHandles = new ArrayList<>(); + try (final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath(), columnFamilyDescriptors, columnFamilyHandles)) { + try { + final MutableColumnFamilyOptions mutableOptions = + MutableColumnFamilyOptions.builder() + .setWriteBufferSize(2048) + .build(); + + db.setOptions(columnFamilyHandles.get(1), mutableOptions); + + } finally { + for (final ColumnFamilyHandle handle : columnFamilyHandles) { + handle.close(); + } + } + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/RocksEnvTest.java b/java/src/test/java/org/rocksdb/RocksEnvTest.java new file mode 100644 index 00000000000..dfb79610738 --- /dev/null +++ b/java/src/test/java/org/rocksdb/RocksEnvTest.java @@ -0,0 +1,39 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class RocksEnvTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void rocksEnv() { + try (final Env rocksEnv = RocksEnv.getDefault()) { + rocksEnv.setBackgroundThreads(5); + // default rocksenv will always return zero for flush pool + // no matter what was set via setBackgroundThreads + assertThat(rocksEnv.getThreadPoolQueueLen(RocksEnv.FLUSH_POOL)). + isEqualTo(0); + rocksEnv.setBackgroundThreads(5, RocksEnv.FLUSH_POOL); + // default rocksenv will always return zero for flush pool + // no matter what was set via setBackgroundThreads + assertThat(rocksEnv.getThreadPoolQueueLen(RocksEnv.FLUSH_POOL)). + isEqualTo(0); + rocksEnv.setBackgroundThreads(5, RocksEnv.COMPACTION_POOL); + // default rocksenv will always return zero for compaction pool + // no matter what was set via setBackgroundThreads + assertThat(rocksEnv.getThreadPoolQueueLen(RocksEnv.COMPACTION_POOL)). + isEqualTo(0); + } + } +} diff --git a/java/src/test/java/org/rocksdb/RocksIteratorTest.java b/java/src/test/java/org/rocksdb/RocksIteratorTest.java new file mode 100644 index 00000000000..982dab4fc8f --- /dev/null +++ b/java/src/test/java/org/rocksdb/RocksIteratorTest.java @@ -0,0 +1,58 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import static org.assertj.core.api.Assertions.assertThat; + +public class RocksIteratorTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void rocksIterator() throws RocksDBException { + try (final Options options = new Options() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + db.put("key1".getBytes(), "value1".getBytes()); + db.put("key2".getBytes(), "value2".getBytes()); + + try (final RocksIterator iterator = db.newIterator()) { + iterator.seekToFirst(); + assertThat(iterator.isValid()).isTrue(); + assertThat(iterator.key()).isEqualTo("key1".getBytes()); + assertThat(iterator.value()).isEqualTo("value1".getBytes()); + iterator.next(); + assertThat(iterator.isValid()).isTrue(); + assertThat(iterator.key()).isEqualTo("key2".getBytes()); + assertThat(iterator.value()).isEqualTo("value2".getBytes()); + iterator.next(); + assertThat(iterator.isValid()).isFalse(); + iterator.seekToLast(); + iterator.prev(); + assertThat(iterator.isValid()).isTrue(); + assertThat(iterator.key()).isEqualTo("key1".getBytes()); + assertThat(iterator.value()).isEqualTo("value1".getBytes()); + iterator.seekToFirst(); + iterator.seekToLast(); + assertThat(iterator.isValid()).isTrue(); + assertThat(iterator.key()).isEqualTo("key2".getBytes()); + assertThat(iterator.value()).isEqualTo("value2".getBytes()); + iterator.status(); + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/RocksMemEnvTest.java b/java/src/test/java/org/rocksdb/RocksMemEnvTest.java new file mode 100644 index 00000000000..04fae2e95de --- /dev/null +++ b/java/src/test/java/org/rocksdb/RocksMemEnvTest.java @@ -0,0 +1,148 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import static org.assertj.core.api.Assertions.assertThat; + +public class RocksMemEnvTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void memEnvFillAndReopen() throws RocksDBException { + + final byte[][] keys = { + "aaa".getBytes(), + "bbb".getBytes(), + "ccc".getBytes() + }; + + final byte[][] values = { + "foo".getBytes(), + "bar".getBytes(), + "baz".getBytes() + }; + + try (final Env env = new RocksMemEnv(); + final Options options = new Options() + .setCreateIfMissing(true) + .setEnv(env); + final FlushOptions flushOptions = new FlushOptions() + .setWaitForFlush(true); + ) { + try (final RocksDB db = RocksDB.open(options, "dir/db")) { + // write key/value pairs using MemEnv + for (int i = 0; i < keys.length; i++) { + db.put(keys[i], values[i]); + } + + // read key/value pairs using MemEnv + for (int i = 0; i < keys.length; i++) { + assertThat(db.get(keys[i])).isEqualTo(values[i]); + } + + // Check iterator access + try (final RocksIterator iterator = db.newIterator()) { + iterator.seekToFirst(); + for (int i = 0; i < keys.length; i++) { + assertThat(iterator.isValid()).isTrue(); + assertThat(iterator.key()).isEqualTo(keys[i]); + assertThat(iterator.value()).isEqualTo(values[i]); + iterator.next(); + } + // reached end of database + assertThat(iterator.isValid()).isFalse(); + } + + // flush + db.flush(flushOptions); + + // read key/value pairs after flush using MemEnv + for (int i = 0; i < keys.length; i++) { + assertThat(db.get(keys[i])).isEqualTo(values[i]); + } + } + + options.setCreateIfMissing(false); + + // After reopen the values shall still be in the mem env. + // as long as the env is not freed. + try (final RocksDB db = RocksDB.open(options, "dir/db")) { + // read key/value pairs using MemEnv + for (int i = 0; i < keys.length; i++) { + assertThat(db.get(keys[i])).isEqualTo(values[i]); + } + } + } + } + + @Test + public void multipleDatabaseInstances() throws RocksDBException { + // db - keys + final byte[][] keys = { + "aaa".getBytes(), + "bbb".getBytes(), + "ccc".getBytes() + }; + // otherDb - keys + final byte[][] otherKeys = { + "111".getBytes(), + "222".getBytes(), + "333".getBytes() + }; + // values + final byte[][] values = { + "foo".getBytes(), + "bar".getBytes(), + "baz".getBytes() + }; + + try (final Env env = new RocksMemEnv(); + final Options options = new Options() + .setCreateIfMissing(true) + .setEnv(env); + final RocksDB db = RocksDB.open(options, "dir/db"); + final RocksDB otherDb = RocksDB.open(options, "dir/otherDb") + ) { + // write key/value pairs using MemEnv + // to db and to otherDb. + for (int i = 0; i < keys.length; i++) { + db.put(keys[i], values[i]); + otherDb.put(otherKeys[i], values[i]); + } + + // verify key/value pairs after flush using MemEnv + for (int i = 0; i < keys.length; i++) { + // verify db + assertThat(db.get(otherKeys[i])).isNull(); + assertThat(db.get(keys[i])).isEqualTo(values[i]); + + // verify otherDb + assertThat(otherDb.get(keys[i])).isNull(); + assertThat(otherDb.get(otherKeys[i])).isEqualTo(values[i]); + } + } + } + + @Test(expected = RocksDBException.class) + public void createIfMissingFalse() throws RocksDBException { + try (final Env env = new RocksMemEnv(); + final Options options = new Options() + .setCreateIfMissing(false) + .setEnv(env); + final RocksDB db = RocksDB.open(options, "db/dir")) { + // shall throw an exception because db dir does not + // exist. + } + } +} diff --git a/java/src/test/java/org/rocksdb/RocksMemoryResource.java b/java/src/test/java/org/rocksdb/RocksMemoryResource.java new file mode 100644 index 00000000000..6fd1c7e6676 --- /dev/null +++ b/java/src/test/java/org/rocksdb/RocksMemoryResource.java @@ -0,0 +1,24 @@ +package org.rocksdb; + +import org.junit.rules.ExternalResource; + +/** + * Resource to trigger garbage collection after each test + * run. + * + * @deprecated Will be removed with the implementation of + * {@link RocksObject#finalize()} + */ +@Deprecated +public class RocksMemoryResource extends ExternalResource { + + static { + RocksDB.loadLibrary(); + } + + @Override + protected void after() { + System.gc(); + System.runFinalization(); + } +} diff --git a/java/src/test/java/org/rocksdb/SliceTest.java b/java/src/test/java/org/rocksdb/SliceTest.java new file mode 100644 index 00000000000..7ee656cd280 --- /dev/null +++ b/java/src/test/java/org/rocksdb/SliceTest.java @@ -0,0 +1,80 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class SliceTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void slice() { + try (final Slice slice = new Slice("testSlice")) { + assertThat(slice.empty()).isFalse(); + assertThat(slice.size()).isEqualTo(9); + assertThat(slice.data()).isEqualTo("testSlice".getBytes()); + } + + try (final Slice otherSlice = new Slice("otherSlice".getBytes())) { + assertThat(otherSlice.data()).isEqualTo("otherSlice".getBytes()); + } + + try (final Slice thirdSlice = new Slice("otherSlice".getBytes(), 5)) { + assertThat(thirdSlice.data()).isEqualTo("Slice".getBytes()); + } + } + + @Test + public void sliceClear() { + try (final Slice slice = new Slice("abc")) { + assertThat(slice.toString()).isEqualTo("abc"); + slice.clear(); + assertThat(slice.toString()).isEmpty(); + slice.clear(); // make sure we don't double-free + } + } + + @Test + public void sliceRemovePrefix() { + try (final Slice slice = new Slice("abc")) { + assertThat(slice.toString()).isEqualTo("abc"); + slice.removePrefix(1); + assertThat(slice.toString()).isEqualTo("bc"); + } + } + + @Test + public void sliceEquals() { + try (final Slice slice = new Slice("abc"); + final Slice slice2 = new Slice("abc")) { + assertThat(slice.equals(slice2)).isTrue(); + assertThat(slice.hashCode() == slice2.hashCode()).isTrue(); + } + } + + @Test + public void sliceStartWith() { + try (final Slice slice = new Slice("matchpoint"); + final Slice match = new Slice("mat"); + final Slice noMatch = new Slice("nomatch")) { + assertThat(slice.startsWith(match)).isTrue(); + assertThat(slice.startsWith(noMatch)).isFalse(); + } + } + + @Test + public void sliceToString() { + try (final Slice slice = new Slice("stringTest")) { + assertThat(slice.toString()).isEqualTo("stringTest"); + assertThat(slice.toString(true)).isNotEqualTo(""); + } + } +} diff --git a/java/src/test/java/org/rocksdb/SnapshotTest.java b/java/src/test/java/org/rocksdb/SnapshotTest.java new file mode 100644 index 00000000000..de48c898bd4 --- /dev/null +++ b/java/src/test/java/org/rocksdb/SnapshotTest.java @@ -0,0 +1,169 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import static org.assertj.core.api.Assertions.assertThat; + +public class SnapshotTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void snapshots() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + db.put("key".getBytes(), "value".getBytes()); + // Get new Snapshot of database + try (final Snapshot snapshot = db.getSnapshot()) { + assertThat(snapshot.getSequenceNumber()).isGreaterThan(0); + assertThat(snapshot.getSequenceNumber()).isEqualTo(1); + try (final ReadOptions readOptions = new ReadOptions()) { + // set snapshot in ReadOptions + readOptions.setSnapshot(snapshot); + + // retrieve key value pair + assertThat(new String(db.get("key".getBytes()))). + isEqualTo("value"); + // retrieve key value pair created before + // the snapshot was made + assertThat(new String(db.get(readOptions, + "key".getBytes()))).isEqualTo("value"); + // add new key/value pair + db.put("newkey".getBytes(), "newvalue".getBytes()); + // using no snapshot the latest db entries + // will be taken into account + assertThat(new String(db.get("newkey".getBytes()))). + isEqualTo("newvalue"); + // snapshopot was created before newkey + assertThat(db.get(readOptions, "newkey".getBytes())). + isNull(); + // Retrieve snapshot from read options + try (final Snapshot sameSnapshot = readOptions.snapshot()) { + readOptions.setSnapshot(sameSnapshot); + // results must be the same with new Snapshot + // instance using the same native pointer + assertThat(new String(db.get(readOptions, + "key".getBytes()))).isEqualTo("value"); + // update key value pair to newvalue + db.put("key".getBytes(), "newvalue".getBytes()); + // read with previously created snapshot will + // read previous version of key value pair + assertThat(new String(db.get(readOptions, + "key".getBytes()))).isEqualTo("value"); + // read for newkey using the snapshot must be + // null + assertThat(db.get(readOptions, "newkey".getBytes())). + isNull(); + // setting null to snapshot in ReadOptions leads + // to no Snapshot being used. + readOptions.setSnapshot(null); + assertThat(new String(db.get(readOptions, + "newkey".getBytes()))).isEqualTo("newvalue"); + // release Snapshot + db.releaseSnapshot(snapshot); + } + } + } + } + } + + @Test + public void iteratorWithSnapshot() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + db.put("key".getBytes(), "value".getBytes()); + + // Get new Snapshot of database + // set snapshot in ReadOptions + try (final Snapshot snapshot = db.getSnapshot(); + final ReadOptions readOptions = + new ReadOptions().setSnapshot(snapshot)) { + db.put("key2".getBytes(), "value2".getBytes()); + + // iterate over current state of db + try (final RocksIterator iterator = db.newIterator()) { + iterator.seekToFirst(); + assertThat(iterator.isValid()).isTrue(); + assertThat(iterator.key()).isEqualTo("key".getBytes()); + iterator.next(); + assertThat(iterator.isValid()).isTrue(); + assertThat(iterator.key()).isEqualTo("key2".getBytes()); + iterator.next(); + assertThat(iterator.isValid()).isFalse(); + } + + // iterate using a snapshot + try (final RocksIterator snapshotIterator = + db.newIterator(readOptions)) { + snapshotIterator.seekToFirst(); + assertThat(snapshotIterator.isValid()).isTrue(); + assertThat(snapshotIterator.key()).isEqualTo("key".getBytes()); + snapshotIterator.next(); + assertThat(snapshotIterator.isValid()).isFalse(); + } + + // release Snapshot + db.releaseSnapshot(snapshot); + } + } + } + + @Test + public void iteratorWithSnapshotOnColumnFamily() throws RocksDBException { + try (final Options options = new Options() + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + + db.put("key".getBytes(), "value".getBytes()); + + // Get new Snapshot of database + // set snapshot in ReadOptions + try (final Snapshot snapshot = db.getSnapshot(); + final ReadOptions readOptions = new ReadOptions() + .setSnapshot(snapshot)) { + db.put("key2".getBytes(), "value2".getBytes()); + + // iterate over current state of column family + try (final RocksIterator iterator = db.newIterator( + db.getDefaultColumnFamily())) { + iterator.seekToFirst(); + assertThat(iterator.isValid()).isTrue(); + assertThat(iterator.key()).isEqualTo("key".getBytes()); + iterator.next(); + assertThat(iterator.isValid()).isTrue(); + assertThat(iterator.key()).isEqualTo("key2".getBytes()); + iterator.next(); + assertThat(iterator.isValid()).isFalse(); + } + + // iterate using a snapshot on default column family + try (final RocksIterator snapshotIterator = db.newIterator( + db.getDefaultColumnFamily(), readOptions)) { + snapshotIterator.seekToFirst(); + assertThat(snapshotIterator.isValid()).isTrue(); + assertThat(snapshotIterator.key()).isEqualTo("key".getBytes()); + snapshotIterator.next(); + assertThat(snapshotIterator.isValid()).isFalse(); + + // release Snapshot + db.releaseSnapshot(snapshot); + } + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/SstFileWriterTest.java b/java/src/test/java/org/rocksdb/SstFileWriterTest.java new file mode 100644 index 00000000000..6261210b129 --- /dev/null +++ b/java/src/test/java/org/rocksdb/SstFileWriterTest.java @@ -0,0 +1,226 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.rocksdb.util.BytewiseComparator; + +import java.io.File; +import java.io.IOException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.junit.Assert.fail; + +public class SstFileWriterTest { + private static final String SST_FILE_NAME = "test.sst"; + private static final String DB_DIRECTORY_NAME = "test_db"; + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource + = new RocksMemoryResource(); + + @Rule public TemporaryFolder parentFolder = new TemporaryFolder(); + + enum OpType { PUT, PUT_BYTES, MERGE, MERGE_BYTES, DELETE, DELETE_BYTES} + + class KeyValueWithOp { + KeyValueWithOp(String key, String value, OpType opType) { + this.key = key; + this.value = value; + this.opType = opType; + } + + String getKey() { + return key; + } + + String getValue() { + return value; + } + + OpType getOpType() { + return opType; + } + + private String key; + private String value; + private OpType opType; + }; + + private File newSstFile(final List keyValues, + boolean useJavaBytewiseComparator) throws IOException, RocksDBException { + final EnvOptions envOptions = new EnvOptions(); + final StringAppendOperator stringAppendOperator = new StringAppendOperator(); + final Options options = new Options().setMergeOperator(stringAppendOperator); + SstFileWriter sstFileWriter = null; + ComparatorOptions comparatorOptions = null; + BytewiseComparator comparator = null; + if (useJavaBytewiseComparator) { + comparatorOptions = new ComparatorOptions(); + comparator = new BytewiseComparator(comparatorOptions); + options.setComparator(comparator); + sstFileWriter = new SstFileWriter(envOptions, options, comparator); + } else { + sstFileWriter = new SstFileWriter(envOptions, options); + } + + final File sstFile = parentFolder.newFile(SST_FILE_NAME); + try { + sstFileWriter.open(sstFile.getAbsolutePath()); + for (KeyValueWithOp keyValue : keyValues) { + Slice keySlice = new Slice(keyValue.getKey()); + Slice valueSlice = new Slice(keyValue.getValue()); + byte[] keyBytes = keyValue.getKey().getBytes(); + byte[] valueBytes = keyValue.getValue().getBytes(); + switch (keyValue.getOpType()) { + case PUT: + sstFileWriter.put(keySlice, valueSlice); + break; + case PUT_BYTES: + sstFileWriter.put(keyBytes, valueBytes); + break; + case MERGE: + sstFileWriter.merge(keySlice, valueSlice); + break; + case MERGE_BYTES: + sstFileWriter.merge(keyBytes, valueBytes); + break; + case DELETE: + sstFileWriter.delete(keySlice); + break; + case DELETE_BYTES: + sstFileWriter.delete(keyBytes); + break; + default: + fail("Unsupported op type"); + } + keySlice.close(); + valueSlice.close(); + } + sstFileWriter.finish(); + } finally { + assertThat(sstFileWriter).isNotNull(); + sstFileWriter.close(); + options.close(); + envOptions.close(); + if (comparatorOptions != null) { + comparatorOptions.close(); + } + if (comparator != null) { + comparator.close(); + } + } + return sstFile; + } + + @Test + public void generateSstFileWithJavaComparator() + throws RocksDBException, IOException { + final List keyValues = new ArrayList<>(); + keyValues.add(new KeyValueWithOp("key1", "value1", OpType.PUT)); + keyValues.add(new KeyValueWithOp("key2", "value2", OpType.PUT)); + keyValues.add(new KeyValueWithOp("key3", "value3", OpType.MERGE)); + keyValues.add(new KeyValueWithOp("key4", "value4", OpType.MERGE)); + keyValues.add(new KeyValueWithOp("key5", "", OpType.DELETE)); + + newSstFile(keyValues, true); + } + + @Test + public void generateSstFileWithNativeComparator() + throws RocksDBException, IOException { + final List keyValues = new ArrayList<>(); + keyValues.add(new KeyValueWithOp("key1", "value1", OpType.PUT)); + keyValues.add(new KeyValueWithOp("key2", "value2", OpType.PUT)); + keyValues.add(new KeyValueWithOp("key3", "value3", OpType.MERGE)); + keyValues.add(new KeyValueWithOp("key4", "value4", OpType.MERGE)); + keyValues.add(new KeyValueWithOp("key5", "", OpType.DELETE)); + + newSstFile(keyValues, false); + } + + @Test + public void ingestSstFile() throws RocksDBException, IOException { + final List keyValues = new ArrayList<>(); + keyValues.add(new KeyValueWithOp("key1", "value1", OpType.PUT)); + keyValues.add(new KeyValueWithOp("key2", "value2", OpType.PUT)); + keyValues.add(new KeyValueWithOp("key3", "value3", OpType.PUT_BYTES)); + keyValues.add(new KeyValueWithOp("key4", "value4", OpType.MERGE)); + keyValues.add(new KeyValueWithOp("key5", "value5", OpType.MERGE_BYTES)); + keyValues.add(new KeyValueWithOp("key6", "", OpType.DELETE)); + keyValues.add(new KeyValueWithOp("key7", "", OpType.DELETE)); + + + final File sstFile = newSstFile(keyValues, false); + final File dbFolder = parentFolder.newFolder(DB_DIRECTORY_NAME); + try(final StringAppendOperator stringAppendOperator = + new StringAppendOperator(); + final Options options = new Options() + .setCreateIfMissing(true) + .setMergeOperator(stringAppendOperator); + final RocksDB db = RocksDB.open(options, dbFolder.getAbsolutePath()); + final IngestExternalFileOptions ingestExternalFileOptions = + new IngestExternalFileOptions()) { + db.ingestExternalFile(Arrays.asList(sstFile.getAbsolutePath()), + ingestExternalFileOptions); + + assertThat(db.get("key1".getBytes())).isEqualTo("value1".getBytes()); + assertThat(db.get("key2".getBytes())).isEqualTo("value2".getBytes()); + assertThat(db.get("key3".getBytes())).isEqualTo("value3".getBytes()); + assertThat(db.get("key4".getBytes())).isEqualTo("value4".getBytes()); + assertThat(db.get("key5".getBytes())).isEqualTo("value5".getBytes()); + assertThat(db.get("key6".getBytes())).isEqualTo(null); + assertThat(db.get("key7".getBytes())).isEqualTo(null); + } + } + + @Test + public void ingestSstFile_cf() throws RocksDBException, IOException { + final List keyValues = new ArrayList<>(); + keyValues.add(new KeyValueWithOp("key1", "value1", OpType.PUT)); + keyValues.add(new KeyValueWithOp("key2", "value2", OpType.PUT)); + keyValues.add(new KeyValueWithOp("key3", "value3", OpType.MERGE)); + keyValues.add(new KeyValueWithOp("key4", "", OpType.DELETE)); + + final File sstFile = newSstFile(keyValues, false); + final File dbFolder = parentFolder.newFolder(DB_DIRECTORY_NAME); + try(final StringAppendOperator stringAppendOperator = + new StringAppendOperator(); + final Options options = new Options() + .setCreateIfMissing(true) + .setCreateMissingColumnFamilies(true) + .setMergeOperator(stringAppendOperator); + final RocksDB db = RocksDB.open(options, dbFolder.getAbsolutePath()); + final IngestExternalFileOptions ingestExternalFileOptions = + new IngestExternalFileOptions()) { + + try(final ColumnFamilyOptions cf_opts = new ColumnFamilyOptions() + .setMergeOperator(stringAppendOperator); + final ColumnFamilyHandle cf_handle = db.createColumnFamily( + new ColumnFamilyDescriptor("new_cf".getBytes(), cf_opts))) { + + db.ingestExternalFile(cf_handle, + Arrays.asList(sstFile.getAbsolutePath()), + ingestExternalFileOptions); + + assertThat(db.get(cf_handle, + "key1".getBytes())).isEqualTo("value1".getBytes()); + assertThat(db.get(cf_handle, + "key2".getBytes())).isEqualTo("value2".getBytes()); + assertThat(db.get(cf_handle, + "key3".getBytes())).isEqualTo("value3".getBytes()); + assertThat(db.get(cf_handle, + "key4".getBytes())).isEqualTo(null); + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/StatisticsCollectorTest.java b/java/src/test/java/org/rocksdb/StatisticsCollectorTest.java new file mode 100644 index 00000000000..8dd0cd49308 --- /dev/null +++ b/java/src/test/java/org/rocksdb/StatisticsCollectorTest.java @@ -0,0 +1,55 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.util.Collections; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import static org.assertj.core.api.Assertions.assertThat; + +public class StatisticsCollectorTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void statisticsCollector() + throws InterruptedException, RocksDBException { + try (final Statistics statistics = new Statistics(); + final Options opt = new Options() + .setStatistics(statistics) + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + + try(final Statistics stats = opt.statistics()) { + + final StatsCallbackMock callback = new StatsCallbackMock(); + final StatsCollectorInput statsInput = + new StatsCollectorInput(stats, callback); + + final StatisticsCollector statsCollector = new StatisticsCollector( + Collections.singletonList(statsInput), 100); + statsCollector.start(); + + Thread.sleep(1000); + + assertThat(callback.tickerCallbackCount).isGreaterThan(0); + assertThat(callback.histCallbackCount).isGreaterThan(0); + + statsCollector.shutDown(1000); + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/StatisticsTest.java b/java/src/test/java/org/rocksdb/StatisticsTest.java new file mode 100644 index 00000000000..2103c2fc787 --- /dev/null +++ b/java/src/test/java/org/rocksdb/StatisticsTest.java @@ -0,0 +1,160 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.nio.charset.StandardCharsets; + +import static org.assertj.core.api.Assertions.assertThat; + +public class StatisticsTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void statsLevel() throws RocksDBException { + final Statistics statistics = new Statistics(); + statistics.setStatsLevel(StatsLevel.ALL); + assertThat(statistics.statsLevel()).isEqualTo(StatsLevel.ALL); + } + + @Test + public void getTickerCount() throws RocksDBException { + try (final Statistics statistics = new Statistics(); + final Options opt = new Options() + .setStatistics(statistics) + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + + final byte[] key = "some-key".getBytes(StandardCharsets.UTF_8); + final byte[] value = "some-value".getBytes(StandardCharsets.UTF_8); + + db.put(key, value); + for(int i = 0; i < 10; i++) { + db.get(key); + } + + assertThat(statistics.getTickerCount(TickerType.BYTES_READ)).isGreaterThan(0); + } + } + + @Test + public void getAndResetTickerCount() throws RocksDBException { + try (final Statistics statistics = new Statistics(); + final Options opt = new Options() + .setStatistics(statistics) + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + + final byte[] key = "some-key".getBytes(StandardCharsets.UTF_8); + final byte[] value = "some-value".getBytes(StandardCharsets.UTF_8); + + db.put(key, value); + for(int i = 0; i < 10; i++) { + db.get(key); + } + + final long read = statistics.getAndResetTickerCount(TickerType.BYTES_READ); + assertThat(read).isGreaterThan(0); + + final long readAfterReset = statistics.getTickerCount(TickerType.BYTES_READ); + assertThat(readAfterReset).isLessThan(read); + } + } + + @Test + public void getHistogramData() throws RocksDBException { + try (final Statistics statistics = new Statistics(); + final Options opt = new Options() + .setStatistics(statistics) + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + + final byte[] key = "some-key".getBytes(StandardCharsets.UTF_8); + final byte[] value = "some-value".getBytes(StandardCharsets.UTF_8); + + db.put(key, value); + for(int i = 0; i < 10; i++) { + db.get(key); + } + + final HistogramData histogramData = statistics.getHistogramData(HistogramType.BYTES_PER_READ); + assertThat(histogramData).isNotNull(); + assertThat(histogramData.getAverage()).isGreaterThan(0); + } + } + + @Test + public void getHistogramString() throws RocksDBException { + try (final Statistics statistics = new Statistics(); + final Options opt = new Options() + .setStatistics(statistics) + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + + final byte[] key = "some-key".getBytes(StandardCharsets.UTF_8); + final byte[] value = "some-value".getBytes(StandardCharsets.UTF_8); + + for(int i = 0; i < 10; i++) { + db.put(key, value); + } + + assertThat(statistics.getHistogramString(HistogramType.BYTES_PER_WRITE)).isNotNull(); + } + } + + @Test + public void reset() throws RocksDBException { + try (final Statistics statistics = new Statistics(); + final Options opt = new Options() + .setStatistics(statistics) + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + + final byte[] key = "some-key".getBytes(StandardCharsets.UTF_8); + final byte[] value = "some-value".getBytes(StandardCharsets.UTF_8); + + db.put(key, value); + for(int i = 0; i < 10; i++) { + db.get(key); + } + + final long read = statistics.getTickerCount(TickerType.BYTES_READ); + assertThat(read).isGreaterThan(0); + + statistics.reset(); + + final long readAfterReset = statistics.getTickerCount(TickerType.BYTES_READ); + assertThat(readAfterReset).isLessThan(read); + } + } + + @Test + public void ToString() throws RocksDBException { + try (final Statistics statistics = new Statistics(); + final Options opt = new Options() + .setStatistics(statistics) + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(opt, + dbFolder.getRoot().getAbsolutePath())) { + assertThat(statistics.toString()).isNotNull(); + } + } +} diff --git a/java/org/rocksdb/test/StatsCallbackMock.java b/java/src/test/java/org/rocksdb/StatsCallbackMock.java similarity index 52% rename from java/org/rocksdb/test/StatsCallbackMock.java rename to java/src/test/java/org/rocksdb/StatsCallbackMock.java index 4ad2fb7b796..af8db0caabd 100644 --- a/java/org/rocksdb/test/StatsCallbackMock.java +++ b/java/src/test/java/org/rocksdb/StatsCallbackMock.java @@ -1,11 +1,9 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). -package org.rocksdb.test; - -import org.rocksdb.*; +package org.rocksdb; public class StatsCallbackMock implements StatisticsCollectorCallback { public int tickerCallbackCount = 0; diff --git a/java/src/test/java/org/rocksdb/TransactionLogIteratorTest.java b/java/src/test/java/org/rocksdb/TransactionLogIteratorTest.java new file mode 100644 index 00000000000..b619258ecc3 --- /dev/null +++ b/java/src/test/java/org/rocksdb/TransactionLogIteratorTest.java @@ -0,0 +1,138 @@ +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import static org.assertj.core.api.Assertions.assertThat; + +public class TransactionLogIteratorTest { + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void transactionLogIterator() throws RocksDBException { + try (final Options options = new Options() + .setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath()); + final TransactionLogIterator transactionLogIterator = + db.getUpdatesSince(0)) { + //no-op + } + } + + @Test + public void getBatch() throws RocksDBException { + final int numberOfPuts = 5; + try (final Options options = new Options() + .setCreateIfMissing(true) + .setWalTtlSeconds(1000) + .setWalSizeLimitMB(10); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + + for (int i = 0; i < numberOfPuts; i++) { + db.put(String.valueOf(i).getBytes(), + String.valueOf(i).getBytes()); + } + db.flush(new FlushOptions().setWaitForFlush(true)); + + // the latest sequence number is 5 because 5 puts + // were written beforehand + assertThat(db.getLatestSequenceNumber()). + isEqualTo(numberOfPuts); + + // insert 5 writes into a cf + try (final ColumnFamilyHandle cfHandle = db.createColumnFamily( + new ColumnFamilyDescriptor("new_cf".getBytes()))) { + for (int i = 0; i < numberOfPuts; i++) { + db.put(cfHandle, String.valueOf(i).getBytes(), + String.valueOf(i).getBytes()); + } + // the latest sequence number is 10 because + // (5 + 5) puts were written beforehand + assertThat(db.getLatestSequenceNumber()). + isEqualTo(numberOfPuts + numberOfPuts); + + // Get updates since the beginning + try (final TransactionLogIterator transactionLogIterator = + db.getUpdatesSince(0)) { + assertThat(transactionLogIterator.isValid()).isTrue(); + transactionLogIterator.status(); + + // The first sequence number is 1 + final TransactionLogIterator.BatchResult batchResult = + transactionLogIterator.getBatch(); + assertThat(batchResult.sequenceNumber()).isEqualTo(1); + } + } + } + } + + @Test + public void transactionLogIteratorStallAtLastRecord() + throws RocksDBException { + try (final Options options = new Options() + .setCreateIfMissing(true) + .setWalTtlSeconds(1000) + .setWalSizeLimitMB(10); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + + db.put("key1".getBytes(), "value1".getBytes()); + // Get updates since the beginning + try (final TransactionLogIterator transactionLogIterator = + db.getUpdatesSince(0)) { + transactionLogIterator.status(); + assertThat(transactionLogIterator.isValid()).isTrue(); + transactionLogIterator.next(); + assertThat(transactionLogIterator.isValid()).isFalse(); + transactionLogIterator.status(); + db.put("key2".getBytes(), "value2".getBytes()); + transactionLogIterator.next(); + transactionLogIterator.status(); + assertThat(transactionLogIterator.isValid()).isTrue(); + } + } + } + + @Test + public void transactionLogIteratorCheckAfterRestart() + throws RocksDBException { + final int numberOfKeys = 2; + try (final Options options = new Options() + .setCreateIfMissing(true) + .setWalTtlSeconds(1000) + .setWalSizeLimitMB(10)) { + + try (final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + db.put("key1".getBytes(), "value1".getBytes()); + db.put("key2".getBytes(), "value2".getBytes()); + db.flush(new FlushOptions().setWaitForFlush(true)); + + } + + // reopen + try (final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + assertThat(db.getLatestSequenceNumber()).isEqualTo(numberOfKeys); + + try (final TransactionLogIterator transactionLogIterator = + db.getUpdatesSince(0)) { + for (int i = 0; i < numberOfKeys; i++) { + transactionLogIterator.status(); + assertThat(transactionLogIterator.isValid()).isTrue(); + transactionLogIterator.next(); + } + } + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/TtlDBTest.java b/java/src/test/java/org/rocksdb/TtlDBTest.java new file mode 100644 index 00000000000..cd72634a237 --- /dev/null +++ b/java/src/test/java/org/rocksdb/TtlDBTest.java @@ -0,0 +1,112 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.concurrent.TimeUnit; + +import static org.assertj.core.api.Assertions.assertThat; + +public class TtlDBTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void ttlDBOpen() throws RocksDBException, InterruptedException { + try (final Options options = new Options().setCreateIfMissing(true).setMaxCompactionBytes(0); + final TtlDB ttlDB = TtlDB.open(options, dbFolder.getRoot().getAbsolutePath())) { + ttlDB.put("key".getBytes(), "value".getBytes()); + assertThat(ttlDB.get("key".getBytes())). + isEqualTo("value".getBytes()); + assertThat(ttlDB.get("key".getBytes())).isNotNull(); + } + } + + @Test + public void ttlDBOpenWithTtl() throws RocksDBException, InterruptedException { + try (final Options options = new Options().setCreateIfMissing(true).setMaxCompactionBytes(0); + final TtlDB ttlDB = TtlDB.open(options, dbFolder.getRoot().getAbsolutePath(), 1, false);) { + ttlDB.put("key".getBytes(), "value".getBytes()); + assertThat(ttlDB.get("key".getBytes())). + isEqualTo("value".getBytes()); + TimeUnit.SECONDS.sleep(2); + ttlDB.compactRange(); + assertThat(ttlDB.get("key".getBytes())).isNull(); + } + } + + @Test + public void ttlDbOpenWithColumnFamilies() throws RocksDBException, + InterruptedException { + final List cfNames = Arrays.asList( + new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY), + new ColumnFamilyDescriptor("new_cf".getBytes()) + ); + final List ttlValues = Arrays.asList(0, 1); + + final List columnFamilyHandleList = new ArrayList<>(); + try (final DBOptions dbOptions = new DBOptions() + .setCreateMissingColumnFamilies(true) + .setCreateIfMissing(true); + final TtlDB ttlDB = TtlDB.open(dbOptions, + dbFolder.getRoot().getAbsolutePath(), cfNames, + columnFamilyHandleList, ttlValues, false)) { + try { + ttlDB.put("key".getBytes(), "value".getBytes()); + assertThat(ttlDB.get("key".getBytes())). + isEqualTo("value".getBytes()); + ttlDB.put(columnFamilyHandleList.get(1), "key".getBytes(), + "value".getBytes()); + assertThat(ttlDB.get(columnFamilyHandleList.get(1), + "key".getBytes())).isEqualTo("value".getBytes()); + TimeUnit.SECONDS.sleep(2); + + ttlDB.compactRange(); + ttlDB.compactRange(columnFamilyHandleList.get(1)); + + assertThat(ttlDB.get("key".getBytes())).isNotNull(); + assertThat(ttlDB.get(columnFamilyHandleList.get(1), + "key".getBytes())).isNull(); + } finally { + for (final ColumnFamilyHandle columnFamilyHandle : + columnFamilyHandleList) { + columnFamilyHandle.close(); + } + } + } + } + + @Test + public void createTtlColumnFamily() throws RocksDBException, + InterruptedException { + try (final Options options = new Options().setCreateIfMissing(true); + final TtlDB ttlDB = TtlDB.open(options, + dbFolder.getRoot().getAbsolutePath()); + final ColumnFamilyHandle columnFamilyHandle = + ttlDB.createColumnFamilyWithTtl( + new ColumnFamilyDescriptor("new_cf".getBytes()), 1)) { + ttlDB.put(columnFamilyHandle, "key".getBytes(), + "value".getBytes()); + assertThat(ttlDB.get(columnFamilyHandle, "key".getBytes())). + isEqualTo("value".getBytes()); + TimeUnit.SECONDS.sleep(2); + ttlDB.compactRange(columnFamilyHandle); + assertThat(ttlDB.get(columnFamilyHandle, "key".getBytes())).isNull(); + } + } +} diff --git a/java/src/test/java/org/rocksdb/Types.java b/java/src/test/java/org/rocksdb/Types.java new file mode 100644 index 00000000000..c3c1de833a5 --- /dev/null +++ b/java/src/test/java/org/rocksdb/Types.java @@ -0,0 +1,43 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +/** + * Simple type conversion methods + * for use in tests + */ +public class Types { + + /** + * Convert first 4 bytes of a byte array to an int + * + * @param data The byte array + * + * @return An integer + */ + public static int byteToInt(final byte data[]) { + return (data[0] & 0xff) | + ((data[1] & 0xff) << 8) | + ((data[2] & 0xff) << 16) | + ((data[3] & 0xff) << 24); + } + + /** + * Convert an int to 4 bytes + * + * @param v The int + * + * @return A byte array containing 4 bytes + */ + public static byte[] intToByte(final int v) { + return new byte[] { + (byte)((v >>> 0) & 0xff), + (byte)((v >>> 8) & 0xff), + (byte)((v >>> 16) & 0xff), + (byte)((v >>> 24) & 0xff) + }; + } +} diff --git a/java/src/test/java/org/rocksdb/WALRecoveryModeTest.java b/java/src/test/java/org/rocksdb/WALRecoveryModeTest.java new file mode 100644 index 00000000000..2a0133f6b8c --- /dev/null +++ b/java/src/test/java/org/rocksdb/WALRecoveryModeTest.java @@ -0,0 +1,22 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + + +public class WALRecoveryModeTest { + + @Test + public void getWALRecoveryMode() { + for (final WALRecoveryMode walRecoveryMode : WALRecoveryMode.values()) { + assertThat(WALRecoveryMode.getWALRecoveryMode(walRecoveryMode.getValue())) + .isEqualTo(walRecoveryMode); + } + } +} diff --git a/java/src/test/java/org/rocksdb/WriteBatchHandlerTest.java b/java/src/test/java/org/rocksdb/WriteBatchHandlerTest.java new file mode 100644 index 00000000000..646a31ce7c7 --- /dev/null +++ b/java/src/test/java/org/rocksdb/WriteBatchHandlerTest.java @@ -0,0 +1,169 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + + +public class WriteBatchHandlerTest { + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void writeBatchHandler() throws IOException, RocksDBException { + // setup test data + final List>> testEvents = Arrays.asList( + new Tuple<>(Action.DELETE, + new Tuple("k0".getBytes(), null)), + new Tuple<>(Action.PUT, + new Tuple<>("k1".getBytes(), "v1".getBytes())), + new Tuple<>(Action.PUT, + new Tuple<>("k2".getBytes(), "v2".getBytes())), + new Tuple<>(Action.PUT, + new Tuple<>("k3".getBytes(), "v3".getBytes())), + new Tuple<>(Action.LOG, + new Tuple(null, "log1".getBytes())), + new Tuple<>(Action.MERGE, + new Tuple<>("k2".getBytes(), "v22".getBytes())), + new Tuple<>(Action.DELETE, + new Tuple("k3".getBytes(), null)) + ); + + // load test data to the write batch + try (final WriteBatch batch = new WriteBatch()) { + for (final Tuple> testEvent : testEvents) { + final Tuple data = testEvent.value; + switch (testEvent.key) { + + case PUT: + batch.put(data.key, data.value); + break; + + case MERGE: + batch.merge(data.key, data.value); + break; + + case DELETE: + batch.remove(data.key); + break; + + case LOG: + batch.putLogData(data.value); + break; + } + } + + // attempt to read test data back from the WriteBatch by iterating + // with a handler + try (final CapturingWriteBatchHandler handler = + new CapturingWriteBatchHandler()) { + batch.iterate(handler); + + // compare the results to the test data + final List>> actualEvents = + handler.getEvents(); + assertThat(testEvents.size()).isSameAs(actualEvents.size()); + + for (int i = 0; i < testEvents.size(); i++) { + assertThat(equals(testEvents.get(i), actualEvents.get(i))).isTrue(); + } + } + } + } + + private static boolean equals( + final Tuple> expected, + final Tuple> actual) { + if (!expected.key.equals(actual.key)) { + return false; + } + + final Tuple expectedData = expected.value; + final Tuple actualData = actual.value; + + return equals(expectedData.key, actualData.key) + && equals(expectedData.value, actualData.value); + } + + private static boolean equals(byte[] expected, byte[] actual) { + if (expected != null) { + return Arrays.equals(expected, actual); + } else { + return actual == null; + } + } + + private static class Tuple { + public final K key; + public final V value; + + public Tuple(final K key, final V value) { + this.key = key; + this.value = value; + } + } + + /** + * Enumeration of Write Batch + * event actions + */ + private enum Action { PUT, MERGE, DELETE, DELETE_RANGE, LOG } + + /** + * A simple WriteBatch Handler which adds a record + * of each event that it receives to a list + */ + private static class CapturingWriteBatchHandler extends WriteBatch.Handler { + + private final List>> events + = new ArrayList<>(); + + /** + * Returns a copy of the current events list + * + * @return a list of the events which have happened upto now + */ + public List>> getEvents() { + return new ArrayList<>(events); + } + + @Override + public void put(final byte[] key, final byte[] value) { + events.add(new Tuple<>(Action.PUT, new Tuple<>(key, value))); + } + + @Override + public void merge(final byte[] key, final byte[] value) { + events.add(new Tuple<>(Action.MERGE, new Tuple<>(key, value))); + } + + @Override + public void delete(final byte[] key) { + events.add(new Tuple<>(Action.DELETE, + new Tuple(key, null))); + } + + @Override + public void deleteRange(final byte[] beginKey, final byte[] endKey) { + events.add(new Tuple<>(Action.DELETE_RANGE, new Tuple(beginKey, endKey))); + } + + @Override + public void logData(final byte[] blob) { + events.add(new Tuple<>(Action.LOG, + new Tuple(null, blob))); + } + } +} diff --git a/java/src/test/java/org/rocksdb/WriteBatchTest.java b/java/src/test/java/org/rocksdb/WriteBatchTest.java new file mode 100644 index 00000000000..83f90c8eb45 --- /dev/null +++ b/java/src/test/java/org/rocksdb/WriteBatchTest.java @@ -0,0 +1,296 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.io.UnsupportedEncodingException; +import java.util.Arrays; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * This class mimics the db/write_batch_test.cc + * in the c++ rocksdb library. + *

    + * Not ported yet: + *

    + * Continue(); + * PutGatherSlices(); + */ +public class WriteBatchTest { + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void emptyWriteBatch() { + try (final WriteBatch batch = new WriteBatch()) { + assertThat(batch.count()).isEqualTo(0); + } + } + + @Test + public void multipleBatchOperations() + throws UnsupportedEncodingException { + try (WriteBatch batch = new WriteBatch()) { + batch.put("foo".getBytes("US-ASCII"), "bar".getBytes("US-ASCII")); + batch.remove("box".getBytes("US-ASCII")); + batch.put("baz".getBytes("US-ASCII"), "boo".getBytes("US-ASCII")); + + WriteBatchTestInternalHelper.setSequence(batch, 100); + assertThat(WriteBatchTestInternalHelper.sequence(batch)). + isNotNull(). + isEqualTo(100); + assertThat(batch.count()).isEqualTo(3); + assertThat(new String(getContents(batch), "US-ASCII")). + isEqualTo("Put(baz, boo)@102" + + "Delete(box)@101" + + "Put(foo, bar)@100"); + } + } + + @Test + public void testAppendOperation() + throws UnsupportedEncodingException { + try (final WriteBatch b1 = new WriteBatch(); + final WriteBatch b2 = new WriteBatch()) { + WriteBatchTestInternalHelper.setSequence(b1, 200); + WriteBatchTestInternalHelper.setSequence(b2, 300); + WriteBatchTestInternalHelper.append(b1, b2); + assertThat(getContents(b1).length).isEqualTo(0); + assertThat(b1.count()).isEqualTo(0); + b2.put("a".getBytes("US-ASCII"), "va".getBytes("US-ASCII")); + WriteBatchTestInternalHelper.append(b1, b2); + assertThat("Put(a, va)@200".equals(new String(getContents(b1), + "US-ASCII"))); + assertThat(b1.count()).isEqualTo(1); + b2.clear(); + b2.put("b".getBytes("US-ASCII"), "vb".getBytes("US-ASCII")); + WriteBatchTestInternalHelper.append(b1, b2); + assertThat(("Put(a, va)@200" + + "Put(b, vb)@201") + .equals(new String(getContents(b1), "US-ASCII"))); + assertThat(b1.count()).isEqualTo(2); + b2.remove("foo".getBytes("US-ASCII")); + WriteBatchTestInternalHelper.append(b1, b2); + assertThat(("Put(a, va)@200" + + "Put(b, vb)@202" + + "Put(b, vb)@201" + + "Delete(foo)@203") + .equals(new String(getContents(b1), "US-ASCII"))); + assertThat(b1.count()).isEqualTo(4); + } + } + + @Test + public void blobOperation() + throws UnsupportedEncodingException { + try (final WriteBatch batch = new WriteBatch()) { + batch.put("k1".getBytes("US-ASCII"), "v1".getBytes("US-ASCII")); + batch.put("k2".getBytes("US-ASCII"), "v2".getBytes("US-ASCII")); + batch.put("k3".getBytes("US-ASCII"), "v3".getBytes("US-ASCII")); + batch.putLogData("blob1".getBytes("US-ASCII")); + batch.remove("k2".getBytes("US-ASCII")); + batch.putLogData("blob2".getBytes("US-ASCII")); + batch.merge("foo".getBytes("US-ASCII"), "bar".getBytes("US-ASCII")); + assertThat(batch.count()).isEqualTo(5); + assertThat(("Merge(foo, bar)@4" + + "Put(k1, v1)@0" + + "Delete(k2)@3" + + "Put(k2, v2)@1" + + "Put(k3, v3)@2") + .equals(new String(getContents(batch), "US-ASCII"))); + } + } + + @Test + public void savePoints() + throws UnsupportedEncodingException, RocksDBException { + try (final WriteBatch batch = new WriteBatch()) { + batch.put("k1".getBytes("US-ASCII"), "v1".getBytes("US-ASCII")); + batch.put("k2".getBytes("US-ASCII"), "v2".getBytes("US-ASCII")); + batch.put("k3".getBytes("US-ASCII"), "v3".getBytes("US-ASCII")); + + assertThat(getFromWriteBatch(batch, "k1")).isEqualTo("v1"); + assertThat(getFromWriteBatch(batch, "k2")).isEqualTo("v2"); + assertThat(getFromWriteBatch(batch, "k3")).isEqualTo("v3"); + + + batch.setSavePoint(); + + batch.remove("k2".getBytes("US-ASCII")); + batch.put("k3".getBytes("US-ASCII"), "v3-2".getBytes("US-ASCII")); + + assertThat(getFromWriteBatch(batch, "k2")).isNull(); + assertThat(getFromWriteBatch(batch, "k3")).isEqualTo("v3-2"); + + + batch.setSavePoint(); + + batch.put("k3".getBytes("US-ASCII"), "v3-3".getBytes("US-ASCII")); + batch.put("k4".getBytes("US-ASCII"), "v4".getBytes("US-ASCII")); + + assertThat(getFromWriteBatch(batch, "k3")).isEqualTo("v3-3"); + assertThat(getFromWriteBatch(batch, "k4")).isEqualTo("v4"); + + + batch.rollbackToSavePoint(); + + assertThat(getFromWriteBatch(batch, "k2")).isNull(); + assertThat(getFromWriteBatch(batch, "k3")).isEqualTo("v3-2"); + assertThat(getFromWriteBatch(batch, "k4")).isNull(); + + + batch.rollbackToSavePoint(); + + assertThat(getFromWriteBatch(batch, "k1")).isEqualTo("v1"); + assertThat(getFromWriteBatch(batch, "k2")).isEqualTo("v2"); + assertThat(getFromWriteBatch(batch, "k3")).isEqualTo("v3"); + assertThat(getFromWriteBatch(batch, "k4")).isNull(); + } + } + + @Test + public void deleteRange() throws RocksDBException { + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath()); + final WriteOptions wOpt = new WriteOptions()) { + db.put("key1".getBytes(), "value".getBytes()); + db.put("key2".getBytes(), "12345678".getBytes()); + db.put("key3".getBytes(), "abcdefg".getBytes()); + db.put("key4".getBytes(), "xyz".getBytes()); + assertThat(db.get("key1".getBytes())).isEqualTo("value".getBytes()); + assertThat(db.get("key2".getBytes())).isEqualTo("12345678".getBytes()); + assertThat(db.get("key3".getBytes())).isEqualTo("abcdefg".getBytes()); + assertThat(db.get("key4".getBytes())).isEqualTo("xyz".getBytes()); + + WriteBatch batch = new WriteBatch(); + batch.deleteRange("key2".getBytes(), "key4".getBytes()); + db.write(new WriteOptions(), batch); + + assertThat(db.get("key1".getBytes())).isEqualTo("value".getBytes()); + assertThat(db.get("key2".getBytes())).isNull(); + assertThat(db.get("key3".getBytes())).isNull(); + assertThat(db.get("key4".getBytes())).isEqualTo("xyz".getBytes()); + } + } + + @Test(expected = RocksDBException.class) + public void restorePoints_withoutSavePoints() throws RocksDBException { + try (final WriteBatch batch = new WriteBatch()) { + batch.rollbackToSavePoint(); + } + } + + @Test(expected = RocksDBException.class) + public void restorePoints_withoutSavePoints_nested() throws RocksDBException { + try (final WriteBatch batch = new WriteBatch()) { + + batch.setSavePoint(); + batch.rollbackToSavePoint(); + + // without previous corresponding setSavePoint + batch.rollbackToSavePoint(); + } + } + + static byte[] getContents(final WriteBatch wb) { + return getContents(wb.nativeHandle_); + } + + static String getFromWriteBatch(final WriteBatch wb, final String key) + throws RocksDBException, UnsupportedEncodingException { + final WriteBatchGetter getter = + new WriteBatchGetter(key.getBytes("US-ASCII")); + wb.iterate(getter); + if(getter.getValue() != null) { + return new String(getter.getValue(), "US-ASCII"); + } else { + return null; + } + } + + private static native byte[] getContents(final long writeBatchHandle); + + private static class WriteBatchGetter extends WriteBatch.Handler { + + private final byte[] key; + private byte[] value; + + public WriteBatchGetter(final byte[] key) { + this.key = key; + } + + public byte[] getValue() { + return value; + } + + @Override + public void put(final byte[] key, final byte[] value) { + if(Arrays.equals(this.key, key)) { + this.value = value; + } + } + + @Override + public void merge(final byte[] key, final byte[] value) { + if(Arrays.equals(this.key, key)) { + throw new UnsupportedOperationException(); + } + } + + @Override + public void delete(final byte[] key) { + if(Arrays.equals(this.key, key)) { + this.value = null; + } + } + + @Override + public void deleteRange(final byte[] beginKey, final byte[] endKey) { + throw new UnsupportedOperationException(); + } + + @Override + public void logData(final byte[] blob) { + } + } +} + +/** + * Package-private class which provides java api to access + * c++ WriteBatchInternal. + */ +class WriteBatchTestInternalHelper { + static void setSequence(final WriteBatch wb, final long sn) { + setSequence(wb.nativeHandle_, sn); + } + + static long sequence(final WriteBatch wb) { + return sequence(wb.nativeHandle_); + } + + static void append(final WriteBatch wb1, final WriteBatch wb2) { + append(wb1.nativeHandle_, wb2.nativeHandle_); + } + + private static native void setSequence(final long writeBatchHandle, + final long sn); + + private static native long sequence(final long writeBatchHandle); + + private static native void append(final long writeBatchHandle1, + final long writeBatchHandle2); +} diff --git a/java/src/test/java/org/rocksdb/WriteBatchThreadedTest.java b/java/src/test/java/org/rocksdb/WriteBatchThreadedTest.java new file mode 100644 index 00000000000..c5090dbceba --- /dev/null +++ b/java/src/test/java/org/rocksdb/WriteBatchThreadedTest.java @@ -0,0 +1,104 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb; + +import org.junit.After; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; +import org.junit.runners.Parameterized.Parameter; +import org.junit.runners.Parameterized.Parameters; + +import java.nio.ByteBuffer; +import java.util.*; +import java.util.concurrent.*; + +@RunWith(Parameterized.class) +public class WriteBatchThreadedTest { + + @Parameters(name = "WriteBatchThreadedTest(threadCount={0})") + public static Iterable data() { + return Arrays.asList(new Integer[]{1, 10, 50, 100}); + } + + @Parameter + public int threadCount; + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + RocksDB db; + + @Before + public void setUp() throws Exception { + RocksDB.loadLibrary(); + final Options options = new Options() + .setCreateIfMissing(true) + .setIncreaseParallelism(32); + db = RocksDB.open(options, dbFolder.getRoot().getAbsolutePath()); + assert (db != null); + } + + @After + public void tearDown() throws Exception { + if (db != null) { + db.close(); + } + } + + @Test + public void threadedWrites() throws InterruptedException, ExecutionException { + final List> callables = new ArrayList<>(); + for (int i = 0; i < 100; i++) { + final int offset = i * 100; + callables.add(new Callable() { + @Override + public Void call() throws RocksDBException { + try (final WriteBatch wb = new WriteBatch(); + final WriteOptions w_opt = new WriteOptions()) { + for (int i = offset; i < offset + 100; i++) { + wb.put(ByteBuffer.allocate(4).putInt(i).array(), "parallel rocks test".getBytes()); + } + db.write(w_opt, wb); + } + return null; + } + }); + } + + //submit the callables + final ExecutorService executorService = + Executors.newFixedThreadPool(threadCount); + try { + final ExecutorCompletionService completionService = + new ExecutorCompletionService<>(executorService); + final Set> futures = new HashSet<>(); + for (final Callable callable : callables) { + futures.add(completionService.submit(callable)); + } + + while (futures.size() > 0) { + final Future future = completionService.take(); + futures.remove(future); + + try { + future.get(); + } catch (final ExecutionException e) { + for (final Future f : futures) { + f.cancel(true); + } + + throw e; + } + } + } finally { + executorService.shutdown(); + executorService.awaitTermination(10, TimeUnit.SECONDS); + } + } +} diff --git a/java/src/test/java/org/rocksdb/WriteBatchWithIndexTest.java b/java/src/test/java/org/rocksdb/WriteBatchWithIndexTest.java new file mode 100644 index 00000000000..1c5e34234e8 --- /dev/null +++ b/java/src/test/java/org/rocksdb/WriteBatchWithIndexTest.java @@ -0,0 +1,410 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.io.UnsupportedEncodingException; +import java.nio.ByteBuffer; +import java.util.Arrays; + +import static org.assertj.core.api.Assertions.assertThat; + + +public class WriteBatchWithIndexTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Rule + public TemporaryFolder dbFolder = new TemporaryFolder(); + + @Test + public void readYourOwnWrites() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + + final byte[] k1 = "key1".getBytes(); + final byte[] v1 = "value1".getBytes(); + final byte[] k2 = "key2".getBytes(); + final byte[] v2 = "value2".getBytes(); + + db.put(k1, v1); + db.put(k2, v2); + + try (final WriteBatchWithIndex wbwi = new WriteBatchWithIndex(true); + final RocksIterator base = db.newIterator(); + final RocksIterator it = wbwi.newIteratorWithBase(base)) { + + it.seek(k1); + assertThat(it.isValid()).isTrue(); + assertThat(it.key()).isEqualTo(k1); + assertThat(it.value()).isEqualTo(v1); + + it.seek(k2); + assertThat(it.isValid()).isTrue(); + assertThat(it.key()).isEqualTo(k2); + assertThat(it.value()).isEqualTo(v2); + + //put data to the write batch and make sure we can read it. + final byte[] k3 = "key3".getBytes(); + final byte[] v3 = "value3".getBytes(); + wbwi.put(k3, v3); + it.seek(k3); + assertThat(it.isValid()).isTrue(); + assertThat(it.key()).isEqualTo(k3); + assertThat(it.value()).isEqualTo(v3); + + //update k2 in the write batch and check the value + final byte[] v2Other = "otherValue2".getBytes(); + wbwi.put(k2, v2Other); + it.seek(k2); + assertThat(it.isValid()).isTrue(); + assertThat(it.key()).isEqualTo(k2); + assertThat(it.value()).isEqualTo(v2Other); + + //remove k1 and make sure we can read back the write + wbwi.remove(k1); + it.seek(k1); + assertThat(it.key()).isNotEqualTo(k1); + + //reinsert k1 and make sure we see the new value + final byte[] v1Other = "otherValue1".getBytes(); + wbwi.put(k1, v1Other); + it.seek(k1); + assertThat(it.isValid()).isTrue(); + assertThat(it.key()).isEqualTo(k1); + assertThat(it.value()).isEqualTo(v1Other); + } + } + } + + @Test + public void write_writeBatchWithIndex() throws RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + + final byte[] k1 = "key1".getBytes(); + final byte[] v1 = "value1".getBytes(); + final byte[] k2 = "key2".getBytes(); + final byte[] v2 = "value2".getBytes(); + + try (final WriteBatchWithIndex wbwi = new WriteBatchWithIndex()) { + wbwi.put(k1, v1); + wbwi.put(k2, v2); + + db.write(new WriteOptions(), wbwi); + } + + assertThat(db.get(k1)).isEqualTo(v1); + assertThat(db.get(k2)).isEqualTo(v2); + } + } + + @Test + public void iterator() throws RocksDBException { + try (final WriteBatchWithIndex wbwi = new WriteBatchWithIndex(true)) { + + final String k1 = "key1"; + final String v1 = "value1"; + final String k2 = "key2"; + final String v2 = "value2"; + final String k3 = "key3"; + final String v3 = "value3"; + final byte[] k1b = k1.getBytes(); + final byte[] v1b = v1.getBytes(); + final byte[] k2b = k2.getBytes(); + final byte[] v2b = v2.getBytes(); + final byte[] k3b = k3.getBytes(); + final byte[] v3b = v3.getBytes(); + + //add put records + wbwi.put(k1b, v1b); + wbwi.put(k2b, v2b); + wbwi.put(k3b, v3b); + + //add a deletion record + final String k4 = "key4"; + final byte[] k4b = k4.getBytes(); + wbwi.remove(k4b); + + final WBWIRocksIterator.WriteEntry[] expected = { + new WBWIRocksIterator.WriteEntry(WBWIRocksIterator.WriteType.PUT, + new DirectSlice(k1), new DirectSlice(v1)), + new WBWIRocksIterator.WriteEntry(WBWIRocksIterator.WriteType.PUT, + new DirectSlice(k2), new DirectSlice(v2)), + new WBWIRocksIterator.WriteEntry(WBWIRocksIterator.WriteType.PUT, + new DirectSlice(k3), new DirectSlice(v3)), + new WBWIRocksIterator.WriteEntry(WBWIRocksIterator.WriteType.DELETE, + new DirectSlice(k4), DirectSlice.NONE) + }; + + try (final WBWIRocksIterator it = wbwi.newIterator()) { + //direct access - seek to key offsets + final int[] testOffsets = {2, 0, 1, 3}; + + for (int i = 0; i < testOffsets.length; i++) { + final int testOffset = testOffsets[i]; + final byte[] key = toArray(expected[testOffset].getKey().data()); + + it.seek(key); + assertThat(it.isValid()).isTrue(); + + final WBWIRocksIterator.WriteEntry entry = it.entry(); + assertThat(entry.equals(expected[testOffset])).isTrue(); + } + + //forward iterative access + int i = 0; + for (it.seekToFirst(); it.isValid(); it.next()) { + assertThat(it.entry().equals(expected[i++])).isTrue(); + } + + //reverse iterative access + i = expected.length - 1; + for (it.seekToLast(); it.isValid(); it.prev()) { + assertThat(it.entry().equals(expected[i--])).isTrue(); + } + } + } + } + + @Test + public void zeroByteTests() { + try (final WriteBatchWithIndex wbwi = new WriteBatchWithIndex(true)) { + final byte[] zeroByteValue = new byte[]{0, 0}; + //add zero byte value + wbwi.put(zeroByteValue, zeroByteValue); + + final ByteBuffer buffer = ByteBuffer.allocateDirect(zeroByteValue.length); + buffer.put(zeroByteValue); + + final WBWIRocksIterator.WriteEntry expected = + new WBWIRocksIterator.WriteEntry(WBWIRocksIterator.WriteType.PUT, + new DirectSlice(buffer, zeroByteValue.length), + new DirectSlice(buffer, zeroByteValue.length)); + + try (final WBWIRocksIterator it = wbwi.newIterator()) { + it.seekToFirst(); + final WBWIRocksIterator.WriteEntry actual = it.entry(); + assertThat(actual.equals(expected)).isTrue(); + assertThat(it.entry().hashCode() == expected.hashCode()).isTrue(); + } + } + } + + @Test + public void savePoints() + throws UnsupportedEncodingException, RocksDBException { + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + try (final WriteBatchWithIndex wbwi = new WriteBatchWithIndex(true); + final ReadOptions readOptions = new ReadOptions()) { + wbwi.put("k1".getBytes(), "v1".getBytes()); + wbwi.put("k2".getBytes(), "v2".getBytes()); + wbwi.put("k3".getBytes(), "v3".getBytes()); + + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k1")) + .isEqualTo("v1"); + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k2")) + .isEqualTo("v2"); + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k3")) + .isEqualTo("v3"); + + + wbwi.setSavePoint(); + + wbwi.remove("k2".getBytes()); + wbwi.put("k3".getBytes(), "v3-2".getBytes()); + + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k2")) + .isNull(); + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k3")) + .isEqualTo("v3-2"); + + + wbwi.setSavePoint(); + + wbwi.put("k3".getBytes(), "v3-3".getBytes()); + wbwi.put("k4".getBytes(), "v4".getBytes()); + + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k3")) + .isEqualTo("v3-3"); + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k4")) + .isEqualTo("v4"); + + + wbwi.rollbackToSavePoint(); + + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k2")) + .isNull(); + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k3")) + .isEqualTo("v3-2"); + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k4")) + .isNull(); + + + wbwi.rollbackToSavePoint(); + + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k1")) + .isEqualTo("v1"); + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k2")) + .isEqualTo("v2"); + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k3")) + .isEqualTo("v3"); + assertThat(getFromWriteBatchWithIndex(db, readOptions, wbwi, "k4")) + .isNull(); + } + } + } + + @Test(expected = RocksDBException.class) + public void restorePoints_withoutSavePoints() throws RocksDBException { + try (final WriteBatchWithIndex wbwi = new WriteBatchWithIndex()) { + wbwi.rollbackToSavePoint(); + } + } + + @Test(expected = RocksDBException.class) + public void restorePoints_withoutSavePoints_nested() throws RocksDBException { + try (final WriteBatchWithIndex wbwi = new WriteBatchWithIndex()) { + + wbwi.setSavePoint(); + wbwi.rollbackToSavePoint(); + + // without previous corresponding setSavePoint + wbwi.rollbackToSavePoint(); + } + } + + private static String getFromWriteBatchWithIndex(final RocksDB db, + final ReadOptions readOptions, final WriteBatchWithIndex wbwi, + final String skey) { + final byte[] key = skey.getBytes(); + try(final RocksIterator baseIterator = db.newIterator(readOptions); + final RocksIterator iterator = wbwi.newIteratorWithBase(baseIterator)) { + iterator.seek(key); + + // Arrays.equals(key, iterator.key()) ensures an exact match in Rocks, + // instead of a nearest match + return iterator.isValid() && + Arrays.equals(key, iterator.key()) ? + new String(iterator.value()) : null; + } + } + + @Test + public void getFromBatch() throws RocksDBException { + final byte[] k1 = "k1".getBytes(); + final byte[] k2 = "k2".getBytes(); + final byte[] k3 = "k3".getBytes(); + final byte[] k4 = "k4".getBytes(); + + final byte[] v1 = "v1".getBytes(); + final byte[] v2 = "v2".getBytes(); + final byte[] v3 = "v3".getBytes(); + + try (final WriteBatchWithIndex wbwi = new WriteBatchWithIndex(true); + final DBOptions dbOptions = new DBOptions()) { + wbwi.put(k1, v1); + wbwi.put(k2, v2); + wbwi.put(k3, v3); + + assertThat(wbwi.getFromBatch(dbOptions, k1)).isEqualTo(v1); + assertThat(wbwi.getFromBatch(dbOptions, k2)).isEqualTo(v2); + assertThat(wbwi.getFromBatch(dbOptions, k3)).isEqualTo(v3); + assertThat(wbwi.getFromBatch(dbOptions, k4)).isNull(); + + wbwi.remove(k2); + + assertThat(wbwi.getFromBatch(dbOptions, k2)).isNull(); + } + } + + @Test + public void getFromBatchAndDB() throws RocksDBException { + final byte[] k1 = "k1".getBytes(); + final byte[] k2 = "k2".getBytes(); + final byte[] k3 = "k3".getBytes(); + final byte[] k4 = "k4".getBytes(); + + final byte[] v1 = "v1".getBytes(); + final byte[] v2 = "v2".getBytes(); + final byte[] v3 = "v3".getBytes(); + final byte[] v4 = "v4".getBytes(); + + try (final Options options = new Options().setCreateIfMissing(true); + final RocksDB db = RocksDB.open(options, + dbFolder.getRoot().getAbsolutePath())) { + + db.put(k1, v1); + db.put(k2, v2); + db.put(k4, v4); + + try (final WriteBatchWithIndex wbwi = new WriteBatchWithIndex(true); + final DBOptions dbOptions = new DBOptions(); + final ReadOptions readOptions = new ReadOptions()) { + + assertThat(wbwi.getFromBatch(dbOptions, k1)).isNull(); + assertThat(wbwi.getFromBatch(dbOptions, k2)).isNull(); + assertThat(wbwi.getFromBatch(dbOptions, k4)).isNull(); + + wbwi.put(k3, v3); + + assertThat(wbwi.getFromBatch(dbOptions, k3)).isEqualTo(v3); + + assertThat(wbwi.getFromBatchAndDB(db, readOptions, k1)).isEqualTo(v1); + assertThat(wbwi.getFromBatchAndDB(db, readOptions, k2)).isEqualTo(v2); + assertThat(wbwi.getFromBatchAndDB(db, readOptions, k3)).isEqualTo(v3); + assertThat(wbwi.getFromBatchAndDB(db, readOptions, k4)).isEqualTo(v4); + + wbwi.remove(k4); + + assertThat(wbwi.getFromBatchAndDB(db, readOptions, k4)).isNull(); + } + } + } + private byte[] toArray(final ByteBuffer buf) { + final byte[] ary = new byte[buf.remaining()]; + buf.get(ary); + return ary; + } + + @Test + public void deleteRange() throws RocksDBException { + try (final RocksDB db = RocksDB.open(dbFolder.getRoot().getAbsolutePath()); + final WriteOptions wOpt = new WriteOptions()) { + db.put("key1".getBytes(), "value".getBytes()); + db.put("key2".getBytes(), "12345678".getBytes()); + db.put("key3".getBytes(), "abcdefg".getBytes()); + db.put("key4".getBytes(), "xyz".getBytes()); + assertThat(db.get("key1".getBytes())).isEqualTo("value".getBytes()); + assertThat(db.get("key2".getBytes())).isEqualTo("12345678".getBytes()); + assertThat(db.get("key3".getBytes())).isEqualTo("abcdefg".getBytes()); + assertThat(db.get("key4".getBytes())).isEqualTo("xyz".getBytes()); + + WriteBatch batch = new WriteBatch(); + batch.deleteRange("key2".getBytes(), "key4".getBytes()); + db.write(new WriteOptions(), batch); + + assertThat(db.get("key1".getBytes())).isEqualTo("value".getBytes()); + assertThat(db.get("key2".getBytes())).isNull(); + assertThat(db.get("key3".getBytes())).isNull(); + assertThat(db.get("key4".getBytes())).isEqualTo("xyz".getBytes()); + } + } +} diff --git a/java/src/test/java/org/rocksdb/WriteOptionsTest.java b/java/src/test/java/org/rocksdb/WriteOptionsTest.java new file mode 100644 index 00000000000..72a06878664 --- /dev/null +++ b/java/src/test/java/org/rocksdb/WriteOptionsTest.java @@ -0,0 +1,45 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb; + +import org.junit.ClassRule; +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class WriteOptionsTest { + + @ClassRule + public static final RocksMemoryResource rocksMemoryResource = + new RocksMemoryResource(); + + @Test + public void writeOptions() { + try (final WriteOptions writeOptions = new WriteOptions()) { + + writeOptions.setSync(true); + assertThat(writeOptions.sync()).isTrue(); + writeOptions.setSync(false); + assertThat(writeOptions.sync()).isFalse(); + + writeOptions.setDisableWAL(true); + assertThat(writeOptions.disableWAL()).isTrue(); + writeOptions.setDisableWAL(false); + assertThat(writeOptions.disableWAL()).isFalse(); + + + writeOptions.setIgnoreMissingColumnFamilies(true); + assertThat(writeOptions.ignoreMissingColumnFamilies()).isTrue(); + writeOptions.setIgnoreMissingColumnFamilies(false); + assertThat(writeOptions.ignoreMissingColumnFamilies()).isFalse(); + + writeOptions.setNoSlowdown(true); + assertThat(writeOptions.noSlowdown()).isTrue(); + writeOptions.setNoSlowdown(false); + assertThat(writeOptions.noSlowdown()).isFalse(); + } + } +} diff --git a/java/src/test/java/org/rocksdb/test/RocksJunitRunner.java b/java/src/test/java/org/rocksdb/test/RocksJunitRunner.java new file mode 100644 index 00000000000..02ad0380ee9 --- /dev/null +++ b/java/src/test/java/org/rocksdb/test/RocksJunitRunner.java @@ -0,0 +1,70 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb.test; + +import org.junit.internal.JUnitSystem; +import org.junit.internal.RealSystem; +import org.junit.internal.TextListener; +import org.junit.runner.Description; +import org.junit.runner.JUnitCore; +import org.junit.runner.Result; + +import java.util.ArrayList; +import java.util.List; + +/** + * Custom Junit Runner to print also Test classes + * and executed methods to command prompt. + */ +public class RocksJunitRunner { + + /** + * Listener which overrides default functionality + * to print class and method to system out. + */ + static class RocksJunitListener extends TextListener { + + /** + * RocksJunitListener constructor + * + * @param system JUnitSystem + */ + public RocksJunitListener(final JUnitSystem system) { + super(system); + } + + @Override + public void testStarted(final Description description) { + System.out.format("Run: %s testing now -> %s \n", + description.getClassName(), + description.getMethodName()); + } + } + + /** + * Main method to execute tests + * + * @param args Test classes as String names + */ + public static void main(final String[] args){ + final JUnitCore runner = new JUnitCore(); + final JUnitSystem system = new RealSystem(); + runner.addListener(new RocksJunitListener(system)); + try { + final List> classes = new ArrayList<>(); + for (final String arg : args) { + classes.add(Class.forName(arg)); + } + final Class[] clazzes = classes.toArray(new Class[classes.size()]); + final Result result = runner.run(clazzes); + if(!result.wasSuccessful()) { + System.exit(-1); + } + } catch (final ClassNotFoundException e) { + e.printStackTrace(); + System.exit(-2); + } + } +} diff --git a/java/src/test/java/org/rocksdb/util/BytewiseComparatorTest.java b/java/src/test/java/org/rocksdb/util/BytewiseComparatorTest.java new file mode 100644 index 00000000000..42508bc118e --- /dev/null +++ b/java/src/test/java/org/rocksdb/util/BytewiseComparatorTest.java @@ -0,0 +1,480 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +package org.rocksdb.util; + +import org.junit.Test; +import org.rocksdb.*; +import org.rocksdb.Comparator; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.FileVisitResult; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.SimpleFileVisitor; +import java.nio.file.attribute.BasicFileAttributes; +import java.util.*; + +import static org.junit.Assert.*; + +/** + * This is a direct port of various C++ + * tests from db/comparator_db_test.cc + * and some code to adapt it to RocksJava + */ +public class BytewiseComparatorTest { + + /** + * Open the database using the C++ BytewiseComparatorImpl + * and test the results against our Java BytewiseComparator + */ + @Test + public void java_vs_cpp_bytewiseComparator() + throws IOException, RocksDBException { + for(int rand_seed = 301; rand_seed < 306; rand_seed++) { + final Path dbDir = Files.createTempDirectory("comparator_db_test"); + try(final RocksDB db = openDatabase(dbDir, + BuiltinComparator.BYTEWISE_COMPARATOR)) { + final Random rnd = new Random(rand_seed); + doRandomIterationTest( + db, + toJavaComparator(new BytewiseComparator(new ComparatorOptions())), + Arrays.asList("a", "b", "c", "d", "e", "f", "g", "h", "i"), + rnd, + 8, 100, 3 + ); + } finally { + removeData(dbDir); + } + } + } + + /** + * Open the database using the Java BytewiseComparator + * and test the results against another Java BytewiseComparator + */ + @Test + public void java_vs_java_bytewiseComparator() + throws IOException, RocksDBException { + for(int rand_seed = 301; rand_seed < 306; rand_seed++) { + final Path dbDir = Files.createTempDirectory("comparator_db_test"); + try(final RocksDB db = openDatabase(dbDir, new BytewiseComparator( + new ComparatorOptions()))) { + final Random rnd = new Random(rand_seed); + doRandomIterationTest( + db, + toJavaComparator(new BytewiseComparator(new ComparatorOptions())), + Arrays.asList("a", "b", "c", "d", "e", "f", "g", "h", "i"), + rnd, + 8, 100, 3 + ); + } finally { + removeData(dbDir); + } + } + } + + /** + * Open the database using the C++ BytewiseComparatorImpl + * and test the results against our Java DirectBytewiseComparator + */ + @Test + public void java_vs_cpp_directBytewiseComparator() + throws IOException, RocksDBException { + for(int rand_seed = 301; rand_seed < 306; rand_seed++) { + final Path dbDir = Files.createTempDirectory("comparator_db_test"); + try(final RocksDB db = openDatabase(dbDir, + BuiltinComparator.BYTEWISE_COMPARATOR)) { + final Random rnd = new Random(rand_seed); + doRandomIterationTest( + db, + toJavaComparator(new DirectBytewiseComparator( + new ComparatorOptions()) + ), + Arrays.asList("a", "b", "c", "d", "e", "f", "g", "h", "i"), + rnd, + 8, 100, 3 + ); + } finally { + removeData(dbDir); + } + } + } + + /** + * Open the database using the Java DirectBytewiseComparator + * and test the results against another Java DirectBytewiseComparator + */ + @Test + public void java_vs_java_directBytewiseComparator() + throws IOException, RocksDBException { + for(int rand_seed = 301; rand_seed < 306; rand_seed++) { + final Path dbDir = Files.createTempDirectory("comparator_db_test"); + try(final RocksDB db = openDatabase(dbDir, new DirectBytewiseComparator( + new ComparatorOptions()))) { + final Random rnd = new Random(rand_seed); + doRandomIterationTest( + db, + toJavaComparator(new DirectBytewiseComparator( + new ComparatorOptions()) + ), + Arrays.asList("a", "b", "c", "d", "e", "f", "g", "h", "i"), + rnd, + 8, 100, 3 + ); + } finally { + removeData(dbDir); + } + } + } + + /** + * Open the database using the C++ ReverseBytewiseComparatorImpl + * and test the results against our Java ReverseBytewiseComparator + */ + @Test + public void java_vs_cpp_reverseBytewiseComparator() + throws IOException, RocksDBException { + for(int rand_seed = 301; rand_seed < 306; rand_seed++) { + final Path dbDir = Files.createTempDirectory("comparator_db_test"); + try(final RocksDB db = openDatabase(dbDir, + BuiltinComparator.REVERSE_BYTEWISE_COMPARATOR)) { + final Random rnd = new Random(rand_seed); + doRandomIterationTest( + db, + toJavaComparator( + new ReverseBytewiseComparator(new ComparatorOptions()) + ), + Arrays.asList("a", "b", "c", "d", "e", "f", "g", "h", "i"), + rnd, + 8, 100, 3 + ); + } finally { + removeData(dbDir); + } + } + } + + /** + * Open the database using the Java ReverseBytewiseComparator + * and test the results against another Java ReverseBytewiseComparator + */ + @Test + public void java_vs_java_reverseBytewiseComparator() + throws IOException, RocksDBException { + + for(int rand_seed = 301; rand_seed < 306; rand_seed++) { + final Path dbDir = Files.createTempDirectory("comparator_db_test"); + try(final RocksDB db = openDatabase(dbDir, new ReverseBytewiseComparator( + new ComparatorOptions()))) { + final Random rnd = new Random(rand_seed); + doRandomIterationTest( + db, + toJavaComparator( + new ReverseBytewiseComparator(new ComparatorOptions()) + ), + Arrays.asList("a", "b", "c", "d", "e", "f", "g", "h", "i"), + rnd, + 8, 100, 3 + ); + } finally { + removeData(dbDir); + } + } + } + + private void doRandomIterationTest( + final RocksDB db, final java.util.Comparator javaComparator, + final List source_strings, final Random rnd, + final int num_writes, final int num_iter_ops, + final int num_trigger_flush) throws RocksDBException { + + final TreeMap map = new TreeMap<>(javaComparator); + + for (int i = 0; i < num_writes; i++) { + if (num_trigger_flush > 0 && i != 0 && i % num_trigger_flush == 0) { + db.flush(new FlushOptions()); + } + + final int type = rnd.nextInt(2); + final int index = rnd.nextInt(source_strings.size()); + final String key = source_strings.get(index); + switch (type) { + case 0: + // put + map.put(key, key); + db.put(new WriteOptions(), bytes(key), bytes(key)); + break; + case 1: + // delete + if (map.containsKey(key)) { + map.remove(key); + } + db.remove(new WriteOptions(), bytes(key)); + break; + + default: + fail("Should not be able to generate random outside range 1..2"); + } + } + + try(final RocksIterator iter = db.newIterator(new ReadOptions())) { + final KVIter result_iter = new KVIter(map); + + boolean is_valid = false; + for (int i = 0; i < num_iter_ops; i++) { + // Random walk and make sure iter and result_iter returns the + // same key and value + final int type = rnd.nextInt(6); + iter.status(); + switch (type) { + case 0: + // Seek to First + iter.seekToFirst(); + result_iter.seekToFirst(); + break; + case 1: + // Seek to last + iter.seekToLast(); + result_iter.seekToLast(); + break; + case 2: { + // Seek to random key + final int key_idx = rnd.nextInt(source_strings.size()); + final String key = source_strings.get(key_idx); + iter.seek(bytes(key)); + result_iter.seek(bytes(key)); + break; + } + case 3: + // Next + if (is_valid) { + iter.next(); + result_iter.next(); + } else { + continue; + } + break; + case 4: + // Prev + if (is_valid) { + iter.prev(); + result_iter.prev(); + } else { + continue; + } + break; + default: { + assert (type == 5); + final int key_idx = rnd.nextInt(source_strings.size()); + final String key = source_strings.get(key_idx); + final byte[] result = db.get(new ReadOptions(), bytes(key)); + if (!map.containsKey(key)) { + assertNull(result); + } else { + assertArrayEquals(bytes(map.get(key)), result); + } + break; + } + } + + assertEquals(result_iter.isValid(), iter.isValid()); + + is_valid = iter.isValid(); + + if (is_valid) { + assertArrayEquals(bytes(result_iter.key()), iter.key()); + + //note that calling value on a non-valid iterator from the Java API + //results in a SIGSEGV + assertArrayEquals(bytes(result_iter.value()), iter.value()); + } + } + } + } + + /** + * Open the database using a C++ Comparator + */ + private RocksDB openDatabase( + final Path dbDir, final BuiltinComparator cppComparator) + throws IOException, RocksDBException { + final Options options = new Options() + .setCreateIfMissing(true) + .setComparator(cppComparator); + return RocksDB.open(options, dbDir.toAbsolutePath().toString()); + } + + /** + * Open the database using a Java Comparator + */ + private RocksDB openDatabase( + final Path dbDir, + final AbstractComparator> javaComparator) + throws IOException, RocksDBException { + final Options options = new Options() + .setCreateIfMissing(true) + .setComparator(javaComparator); + return RocksDB.open(options, dbDir.toAbsolutePath().toString()); + } + + private void closeDatabase(final RocksDB db) { + db.close(); + } + + private void removeData(final Path dbDir) throws IOException { + Files.walkFileTree(dbDir, new SimpleFileVisitor() { + @Override + public FileVisitResult visitFile( + final Path file, final BasicFileAttributes attrs) + throws IOException { + Files.delete(file); + return FileVisitResult.CONTINUE; + } + + @Override + public FileVisitResult postVisitDirectory( + final Path dir, final IOException exc) throws IOException { + Files.delete(dir); + return FileVisitResult.CONTINUE; + } + }); + } + + private byte[] bytes(final String s) { + return s.getBytes(StandardCharsets.UTF_8); + } + + private java.util.Comparator toJavaComparator( + final Comparator rocksComparator) { + return new java.util.Comparator() { + @Override + public int compare(final String s1, final String s2) { + return rocksComparator.compare(new Slice(s1), new Slice(s2)); + } + }; + } + + private java.util.Comparator toJavaComparator( + final DirectComparator rocksComparator) { + return new java.util.Comparator() { + @Override + public int compare(final String s1, final String s2) { + return rocksComparator.compare(new DirectSlice(s1), + new DirectSlice(s2)); + } + }; + } + + private class KVIter implements RocksIteratorInterface { + + private final List> entries; + private final java.util.Comparator comparator; + private int offset = -1; + + private int lastPrefixMatchIdx = -1; + private int lastPrefixMatch = 0; + + public KVIter(final TreeMap map) { + this.entries = new ArrayList<>(); + final Iterator> iterator = map.entrySet().iterator(); + while(iterator.hasNext()) { + entries.add(iterator.next()); + } + this.comparator = map.comparator(); + } + + + @Override + public boolean isValid() { + return offset > -1 && offset < entries.size(); + } + + @Override + public void seekToFirst() { + offset = 0; + } + + @Override + public void seekToLast() { + offset = entries.size() - 1; + } + + @Override + public void seek(final byte[] target) { + for(offset = 0; offset < entries.size(); offset++) { + if(comparator.compare(entries.get(offset).getKey(), + (K)new String(target, StandardCharsets.UTF_8)) >= 0) { + return; + } + } + } + + /** + * Is `a` a prefix of `b` + * + * @return The length of the matching prefix, or 0 if it is not a prefix + */ + private int isPrefix(final byte[] a, final byte[] b) { + if(b.length >= a.length) { + for(int i = 0; i < a.length; i++) { + if(a[i] != b[i]) { + return i; + } + } + return a.length; + } else { + return 0; + } + } + + @Override + public void next() { + if(offset < entries.size()) { + offset++; + } + } + + @Override + public void prev() { + if(offset >= 0) { + offset--; + } + } + + @Override + public void status() throws RocksDBException { + if(offset < 0 || offset >= entries.size()) { + throw new RocksDBException("Index out of bounds. Size is: " + + entries.size() + ", offset is: " + offset); + } + } + + public K key() { + if(!isValid()) { + if(entries.isEmpty()) { + return (K)""; + } else if(offset == -1){ + return entries.get(0).getKey(); + } else if(offset == entries.size()) { + return entries.get(offset - 1).getKey(); + } else { + return (K)""; + } + } else { + return entries.get(offset).getKey(); + } + } + + public V value() { + if(!isValid()) { + return (V)""; + } else { + return entries.get(offset).getValue(); + } + } + } +} diff --git a/java/src/test/java/org/rocksdb/util/EnvironmentTest.java b/java/src/test/java/org/rocksdb/util/EnvironmentTest.java new file mode 100644 index 00000000000..28ee04768e9 --- /dev/null +++ b/java/src/test/java/org/rocksdb/util/EnvironmentTest.java @@ -0,0 +1,172 @@ +// Copyright (c) 2014, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb.util; + +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.Test; + +import java.lang.reflect.Field; +import java.lang.reflect.Modifier; + +import static org.assertj.core.api.Assertions.assertThat; + +public class EnvironmentTest { + private final static String ARCH_FIELD_NAME = "ARCH"; + private final static String OS_FIELD_NAME = "OS"; + + private static String INITIAL_OS; + private static String INITIAL_ARCH; + + @BeforeClass + public static void saveState() { + INITIAL_ARCH = getEnvironmentClassField(ARCH_FIELD_NAME); + INITIAL_OS = getEnvironmentClassField(OS_FIELD_NAME); + } + + @Test + public void mac32() { + setEnvironmentClassFields("mac", "32"); + assertThat(Environment.isWindows()).isFalse(); + assertThat(Environment.getJniLibraryExtension()). + isEqualTo(".jnilib"); + assertThat(Environment.getJniLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni-osx.jnilib"); + assertThat(Environment.getSharedLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni.dylib"); + } + + @Test + public void mac64() { + setEnvironmentClassFields("mac", "64"); + assertThat(Environment.isWindows()).isFalse(); + assertThat(Environment.getJniLibraryExtension()). + isEqualTo(".jnilib"); + assertThat(Environment.getJniLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni-osx.jnilib"); + assertThat(Environment.getSharedLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni.dylib"); + } + + @Test + public void nix32() { + // Linux + setEnvironmentClassFields("Linux", "32"); + assertThat(Environment.isWindows()).isFalse(); + assertThat(Environment.getJniLibraryExtension()). + isEqualTo(".so"); + assertThat(Environment.getJniLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni-linux32.so"); + assertThat(Environment.getSharedLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni.so"); + // UNIX + setEnvironmentClassFields("Unix", "32"); + assertThat(Environment.isWindows()).isFalse(); + assertThat(Environment.getJniLibraryExtension()). + isEqualTo(".so"); + assertThat(Environment.getJniLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni-linux32.so"); + assertThat(Environment.getSharedLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni.so"); + } + + @Test(expected = UnsupportedOperationException.class) + public void aix32() { + // AIX + setEnvironmentClassFields("aix", "32"); + assertThat(Environment.isWindows()).isFalse(); + assertThat(Environment.getJniLibraryExtension()). + isEqualTo(".so"); + Environment.getJniLibraryFileName("rocksdb"); + } + + @Test + public void nix64() { + setEnvironmentClassFields("Linux", "x64"); + assertThat(Environment.isWindows()).isFalse(); + assertThat(Environment.getJniLibraryExtension()). + isEqualTo(".so"); + assertThat(Environment.getJniLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni-linux64.so"); + assertThat(Environment.getSharedLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni.so"); + // UNIX + setEnvironmentClassFields("Unix", "x64"); + assertThat(Environment.isWindows()).isFalse(); + assertThat(Environment.getJniLibraryExtension()). + isEqualTo(".so"); + assertThat(Environment.getJniLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni-linux64.so"); + assertThat(Environment.getSharedLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni.so"); + // AIX + setEnvironmentClassFields("aix", "x64"); + assertThat(Environment.isWindows()).isFalse(); + assertThat(Environment.getJniLibraryExtension()). + isEqualTo(".so"); + assertThat(Environment.getJniLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni-aix64.so"); + assertThat(Environment.getSharedLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni.so"); + } + + @Test + public void detectWindows(){ + setEnvironmentClassFields("win", "x64"); + assertThat(Environment.isWindows()).isTrue(); + } + + @Test + public void win64() { + setEnvironmentClassFields("win", "x64"); + assertThat(Environment.isWindows()).isTrue(); + assertThat(Environment.getJniLibraryExtension()). + isEqualTo(".dll"); + assertThat(Environment.getJniLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni-win64.dll"); + assertThat(Environment.getSharedLibraryFileName("rocksdb")). + isEqualTo("librocksdbjni.dll"); + } + + private void setEnvironmentClassFields(String osName, + String osArch) { + setEnvironmentClassField(OS_FIELD_NAME, osName); + setEnvironmentClassField(ARCH_FIELD_NAME, osArch); + } + + @AfterClass + public static void restoreState() { + setEnvironmentClassField(OS_FIELD_NAME, INITIAL_OS); + setEnvironmentClassField(ARCH_FIELD_NAME, INITIAL_ARCH); + } + + private static String getEnvironmentClassField(String fieldName) { + final Field field; + try { + field = Environment.class.getDeclaredField(fieldName); + field.setAccessible(true); + final Field modifiersField = Field.class.getDeclaredField("modifiers"); + modifiersField.setAccessible(true); + modifiersField.setInt(field, field.getModifiers() & ~Modifier.FINAL); + return (String)field.get(null); + } catch (NoSuchFieldException | IllegalAccessException e) { + throw new RuntimeException(e); + } + } + + private static void setEnvironmentClassField(String fieldName, String value) { + final Field field; + try { + field = Environment.class.getDeclaredField(fieldName); + field.setAccessible(true); + final Field modifiersField = Field.class.getDeclaredField("modifiers"); + modifiersField.setAccessible(true); + modifiersField.setInt(field, field.getModifiers() & ~Modifier.FINAL); + field.set(null, value); + } catch (NoSuchFieldException | IllegalAccessException e) { + throw new RuntimeException(e); + } + } +} diff --git a/java/src/test/java/org/rocksdb/util/SizeUnitTest.java b/java/src/test/java/org/rocksdb/util/SizeUnitTest.java new file mode 100644 index 00000000000..990aa5f47a4 --- /dev/null +++ b/java/src/test/java/org/rocksdb/util/SizeUnitTest.java @@ -0,0 +1,27 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +package org.rocksdb.util; + +import org.junit.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +public class SizeUnitTest { + + public static final long COMPUTATION_UNIT = 1024L; + + @Test + public void sizeUnit() { + assertThat(SizeUnit.KB).isEqualTo(COMPUTATION_UNIT); + assertThat(SizeUnit.MB).isEqualTo( + SizeUnit.KB * COMPUTATION_UNIT); + assertThat(SizeUnit.GB).isEqualTo( + SizeUnit.MB * COMPUTATION_UNIT); + assertThat(SizeUnit.TB).isEqualTo( + SizeUnit.GB * COMPUTATION_UNIT); + assertThat(SizeUnit.PB).isEqualTo( + SizeUnit.TB * COMPUTATION_UNIT); + } +} diff --git a/linters/__phutil_library_init__.php b/linters/__phutil_library_init__.php deleted file mode 100644 index 4b8d3d1316c..00000000000 --- a/linters/__phutil_library_init__.php +++ /dev/null @@ -1,3 +0,0 @@ - 2, - 'class' => - array( - 'FacebookFbcodeLintEngine' => 'lint_engine/FacebookFbcodeLintEngine.php', - 'FbcodeCppLinter' => 'cpp_linter/FbcodeCppLinter.php', - 'PfffCppLinter' => 'cpp_linter/PfffCppLinter.php', - 'ArcanistCpplintLinter' => 'cpp_linter/ArcanistCpplintLinter.php', - ), - 'function' => - array( - ), - 'xmap' => - array( - 'FacebookFbcodeLintEngine' => 'ArcanistLintEngine', - 'FbcodeCppLinter' => 'ArcanistLinter', - 'PfffCppLinter' => 'ArcanistLinter', - ), -)); diff --git a/linters/cpp_linter/ArcanistCpplintLinter.php b/linters/cpp_linter/ArcanistCpplintLinter.php deleted file mode 100644 index b9c41375551..00000000000 --- a/linters/cpp_linter/ArcanistCpplintLinter.php +++ /dev/null @@ -1,88 +0,0 @@ -linterDir(), $bin); - if (!$err) { - return $this->linterDir().'/'.$bin; - } - - // Look for globally installed cpplint.py - list($err) = exec_manual('which %s', $bin); - if ($err) { - throw new ArcanistUsageException( - "cpplint.py does not appear to be installed on this system. Install ". - "it (e.g., with 'wget \"http://google-styleguide.googlecode.com/". - "svn/trunk/cpplint/cpplint.py\"') ". - "in your .arcconfig to point to the directory where it resides. ". - "Also don't forget to chmod a+x cpplint.py!"); - } - - return $bin; - } - - public function lintPath($path) { - $bin = $this->getLintPath(); - $path = $this->rocksdbDir().'/'.$path; - - $f = new ExecFuture("%C $path", $bin); - - list($err, $stdout, $stderr) = $f->resolve(); - - if ($err === 2) { - throw new Exception("cpplint failed to run correctly:\n".$stderr); - } - - $lines = explode("\n", $stderr); - $messages = array(); - foreach ($lines as $line) { - $line = trim($line); - $matches = null; - $regex = '/^[^:]+:(\d+):\s*(.*)\s*\[(.*)\] \[(\d+)\]$/'; - if (!preg_match($regex, $line, $matches)) { - continue; - } - foreach ($matches as $key => $match) { - $matches[$key] = trim($match); - } - $message = new ArcanistLintMessage(); - $message->setPath($path); - $message->setLine($matches[1]); - $message->setCode($matches[3]); - $message->setName($matches[3]); - $message->setDescription($matches[2]); - $message->setSeverity(ArcanistLintSeverity::SEVERITY_WARNING); - $this->addLintMessage($message); - } - } - - // The path of this linter - private function linterDir() { - return dirname(__FILE__); - } - - // TODO(kaili) a quick and dirty way to figure out rocksdb's root dir. - private function rocksdbDir() { - return $this->linterDir()."/../.."; - } -} diff --git a/linters/cpp_linter/FbcodeCppLinter.php b/linters/cpp_linter/FbcodeCppLinter.php deleted file mode 100644 index e62d3bbe1b8..00000000000 --- a/linters/cpp_linter/FbcodeCppLinter.php +++ /dev/null @@ -1,99 +0,0 @@ -getEngine()->getFilePathOnDisk($p); - $lpath_file = file($lpath); - if (preg_match('/\.(c)$/', $lpath) || - preg_match('/-\*-.*Mode: C[; ].*-\*-/', $lpath_file[0]) || - preg_match('/vim(:.*)*:\s*(set\s+)?filetype=c\s*:/', $lpath_file[0]) - ) { - $futures[$p] = new ExecFuture("%s %s %s 2>&1", - $CPP_LINT, self::C_FLAG, - $this->getEngine()->getFilePathOnDisk($p)); - } else { - $futures[$p] = new ExecFuture("%s %s 2>&1", - self::CPPLINT, $this->getEngine()->getFilePathOnDisk($p)); - } - } - - foreach (Futures($futures)->limit(8) as $p => $f) { - $this->rawLintOutput[$p] = $f->resolvex(); - } - } - return; - } - - public function getLinterName() { - return "FBCPP"; - } - - public function lintPath($path) { - $msgs = $this->getCppLintOutput($path); - foreach ($msgs as $m) { - $this->raiseLintAtLine($m['line'], 0, $m['severity'], $m['msg']); - } - } - - public function getLintSeverityMap() { - return array( - self::LINT_WARNING => ArcanistLintSeverity::SEVERITY_WARNING, - self::LINT_ERROR => ArcanistLintSeverity::SEVERITY_ERROR - ); - } - - public function getLintNameMap() { - return array( - self::LINT_WARNING => "CppLint Warning", - self::LINT_ERROR => "CppLint Error" - ); - } - - private function getCppLintOutput($path) { - list($output) = $this->rawLintOutput[$path]; - - $msgs = array(); - $current = null; - foreach (explode("\n", $output) as $line) { - if (preg_match('/[^:]*\((\d+)\):(.*)$/', $line, $matches)) { - if ($current) { - $msgs[] = $current; - } - $line = $matches[1]; - $text = $matches[2]; - $sev = preg_match('/.*Warning.*/', $text) - ? self::LINT_WARNING - : self::LINT_ERROR; - $current = array('line' => $line, - 'msg' => $text, - 'severity' => $sev); - } else if ($current) { - $current['msg'] .= ' ' . $line; - } - } - if ($current) { - $msgs[] = $current; - } - - return $msgs; - } -} - diff --git a/linters/cpp_linter/PfffCppLinter.php b/linters/cpp_linter/PfffCppLinter.php deleted file mode 100644 index 67366143cec..00000000000 --- a/linters/cpp_linter/PfffCppLinter.php +++ /dev/null @@ -1,68 +0,0 @@ -&1", - $program, $this->getEngine()->getFilePathOnDisk($p)); - } - foreach (Futures($futures)->limit(8) as $p => $f) { - - list($stdout, $stderr) = $f->resolvex(); - $raw = json_decode($stdout, true); - if (!is_array($raw)) { - throw new Exception( - "checkCpp returned invalid JSON!". - "Stdout: {$stdout} Stderr: {$stderr}" - ); - } - foreach($raw as $err) { - $this->addLintMessage( - ArcanistLintMessage::newFromDictionary( - array( - 'path' => $err['file'], - 'line' => $err['line'], - 'char' => 0, - 'name' => $err['name'], - 'description' => $err['info'], - 'code' => $this->getLinterName(), - 'severity' => ArcanistLintSeverity::SEVERITY_WARNING, - ) - ) - ); - } - } - } - return; - } - - public function lintPath($path) { - return; - } -} diff --git a/linters/cpp_linter/cpplint.py b/linters/cpp_linter/cpplint.py deleted file mode 100755 index d264b00da03..00000000000 --- a/linters/cpp_linter/cpplint.py +++ /dev/null @@ -1,4767 +0,0 @@ -#!/usr/bin/python -# Copyright (c) 2013, Facebook, Inc. All rights reserved. -# This source code is licensed under the BSD-style license found in the -# LICENSE file in the root directory of this source tree. An additional grant -# of patent rights can be found in the PATENTS file in the same directory. -# Copyright (c) 2011 The LevelDB Authors. All rights reserved. -# Use of this source code is governed by a BSD-style license that can be -# found in the LICENSE file. See the AUTHORS file for names of contributors. -# -# Copyright (c) 2009 Google Inc. All rights reserved. -# -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are -# met: -# * Redistributions of source code must retain the above copyright -# notice, this list of conditions and the following disclaimer. -# * Redistributions in binary form must reproduce the above -# copyright notice, this list of conditions and the following disclaimer -# in the documentation and/or other materials provided with the -# distribution. -# * Neither the name of Google Inc. nor the names of its -# contributors may be used to endorse or promote products derived from -# this software without specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -"""Does google-lint on c++ files. - -The goal of this script is to identify places in the code that *may* -be in non-compliance with google style. It does not attempt to fix -up these problems -- the point is to educate. It does also not -attempt to find all problems, or to ensure that everything it does -find is legitimately a problem. - -In particular, we can get very confused by /* and // inside strings! -We do a small hack, which is to ignore //'s with "'s after them on the -same line, but it is far from perfect (in either direction). -""" - -import codecs -import copy -import getopt -import math # for log -import os -import re -import sre_compile -import string -import sys -import unicodedata - - -_USAGE = """ -Syntax: cpplint.py [--verbose=#] [--output=vs7] [--filter=-x,+y,...] - [--counting=total|toplevel|detailed] [--root=subdir] - [--linelength=digits] - [file] ... - - The style guidelines this tries to follow are those in - http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml - - Every problem is given a confidence score from 1-5, with 5 meaning we are - certain of the problem, and 1 meaning it could be a legitimate construct. - This will miss some errors, and is not a substitute for a code review. - - To suppress false-positive errors of a certain category, add a - 'NOLINT(category)' comment to the line. NOLINT or NOLINT(*) - suppresses errors of all categories on that line. - - The files passed in will be linted; at least one file must be provided. - Default linted extensions are .cc, .cpp, .cu, .cuh and .h. Change the - extensions with the --extensions flag. - - Flags: - - output=vs7 - By default, the output is formatted to ease emacs parsing. Visual Studio - compatible output (vs7) may also be used. Other formats are unsupported. - - verbose=# - Specify a number 0-5 to restrict errors to certain verbosity levels. - - filter=-x,+y,... - Specify a comma-separated list of category-filters to apply: only - error messages whose category names pass the filters will be printed. - (Category names are printed with the message and look like - "[whitespace/indent]".) Filters are evaluated left to right. - "-FOO" and "FOO" means "do not print categories that start with FOO". - "+FOO" means "do print categories that start with FOO". - - Examples: --filter=-whitespace,+whitespace/braces - --filter=whitespace,runtime/printf,+runtime/printf_format - --filter=-,+build/include_what_you_use - - To see a list of all the categories used in cpplint, pass no arg: - --filter= - - counting=total|toplevel|detailed - The total number of errors found is always printed. If - 'toplevel' is provided, then the count of errors in each of - the top-level categories like 'build' and 'whitespace' will - also be printed. If 'detailed' is provided, then a count - is provided for each category like 'build/class'. - - root=subdir - The root directory used for deriving header guard CPP variable. - By default, the header guard CPP variable is calculated as the relative - path to the directory that contains .git, .hg, or .svn. When this flag - is specified, the relative path is calculated from the specified - directory. If the specified directory does not exist, this flag is - ignored. - - Examples: - Assuing that src/.git exists, the header guard CPP variables for - src/chrome/browser/ui/browser.h are: - - No flag => CHROME_BROWSER_UI_BROWSER_H_ - --root=chrome => BROWSER_UI_BROWSER_H_ - --root=chrome/browser => UI_BROWSER_H_ - - linelength=digits - This is the allowed line length for the project. The default value is - 80 characters. - - Examples: - --linelength=120 - - extensions=extension,extension,... - The allowed file extensions that cpplint will check - - Examples: - --extensions=hpp,cpp -""" - -# We categorize each error message we print. Here are the categories. -# We want an explicit list so we can list them all in cpplint --filter=. -# If you add a new error message with a new category, add it to the list -# here! cpplint_unittest.py should tell you if you forget to do this. -_ERROR_CATEGORIES = [ - 'build/class', - 'build/deprecated', - 'build/endif_comment', - 'build/explicit_make_pair', - 'build/forward_decl', - 'build/header_guard', - 'build/include', - 'build/include_alpha', - 'build/include_order', - 'build/include_what_you_use', - 'build/namespaces', - 'build/printf_format', - 'build/storage_class', - 'legal/copyright', - 'readability/alt_tokens', - 'readability/braces', - 'readability/casting', - 'readability/check', - 'readability/constructors', - 'readability/fn_size', - 'readability/function', - 'readability/multiline_comment', - 'readability/multiline_string', - 'readability/namespace', - 'readability/nolint', - 'readability/nul', - 'readability/streams', - 'readability/todo', - 'readability/utf8', - 'runtime/arrays', - 'runtime/casting', - 'runtime/explicit', - 'runtime/int', - 'runtime/init', - 'runtime/invalid_increment', - 'runtime/member_string_references', - 'runtime/memset', - 'runtime/operator', - 'runtime/printf', - 'runtime/printf_format', - 'runtime/references', - 'runtime/string', - 'runtime/threadsafe_fn', - 'runtime/vlog', - 'whitespace/blank_line', - 'whitespace/braces', - 'whitespace/comma', - 'whitespace/comments', - 'whitespace/empty_conditional_body', - 'whitespace/empty_loop_body', - 'whitespace/end_of_line', - 'whitespace/ending_newline', - 'whitespace/forcolon', - 'whitespace/indent', - 'whitespace/line_length', - 'whitespace/newline', - 'whitespace/operators', - 'whitespace/parens', - 'whitespace/semicolon', - 'whitespace/tab', - 'whitespace/todo' - ] - -# The default state of the category filter. This is overrided by the --filter= -# flag. By default all errors are on, so only add here categories that should be -# off by default (i.e., categories that must be enabled by the --filter= flags). -# All entries here should start with a '-' or '+', as in the --filter= flag. -_DEFAULT_FILTERS = ['-build/include_alpha'] - -# We used to check for high-bit characters, but after much discussion we -# decided those were OK, as long as they were in UTF-8 and didn't represent -# hard-coded international strings, which belong in a separate i18n file. - - -# C++ headers -_CPP_HEADERS = frozenset([ - # Legacy - 'algobase.h', - 'algo.h', - 'alloc.h', - 'builtinbuf.h', - 'bvector.h', - 'complex.h', - 'defalloc.h', - 'deque.h', - 'editbuf.h', - 'fstream.h', - 'function.h', - 'hash_map', - 'hash_map.h', - 'hash_set', - 'hash_set.h', - 'hashtable.h', - 'heap.h', - 'indstream.h', - 'iomanip.h', - 'iostream.h', - 'istream.h', - 'iterator.h', - 'list.h', - 'map.h', - 'multimap.h', - 'multiset.h', - 'ostream.h', - 'pair.h', - 'parsestream.h', - 'pfstream.h', - 'procbuf.h', - 'pthread_alloc', - 'pthread_alloc.h', - 'rope', - 'rope.h', - 'ropeimpl.h', - 'set.h', - 'slist', - 'slist.h', - 'stack.h', - 'stdiostream.h', - 'stl_alloc.h', - 'stl_relops.h', - 'streambuf.h', - 'stream.h', - 'strfile.h', - 'strstream.h', - 'tempbuf.h', - 'tree.h', - 'type_traits.h', - 'vector.h', - # 17.6.1.2 C++ library headers - 'algorithm', - 'array', - 'atomic', - 'bitset', - 'chrono', - 'codecvt', - 'complex', - 'condition_variable', - 'deque', - 'exception', - 'forward_list', - 'fstream', - 'functional', - 'future', - 'initializer_list', - 'iomanip', - 'ios', - 'iosfwd', - 'iostream', - 'istream', - 'iterator', - 'limits', - 'list', - 'locale', - 'map', - 'memory', - 'mutex', - 'new', - 'numeric', - 'ostream', - 'queue', - 'random', - 'ratio', - 'regex', - 'set', - 'sstream', - 'stack', - 'stdexcept', - 'streambuf', - 'string', - 'strstream', - 'system_error', - 'thread', - 'tuple', - 'typeindex', - 'typeinfo', - 'type_traits', - 'unordered_map', - 'unordered_set', - 'utility', - 'valarray', - 'vector', - # 17.6.1.2 C++ headers for C library facilities - 'cassert', - 'ccomplex', - 'cctype', - 'cerrno', - 'cfenv', - 'cfloat', - 'cinttypes', - 'ciso646', - 'climits', - 'clocale', - 'cmath', - 'csetjmp', - 'csignal', - 'cstdalign', - 'cstdarg', - 'cstdbool', - 'cstddef', - 'cstdint', - 'cstdio', - 'cstdlib', - 'cstring', - 'ctgmath', - 'ctime', - 'cuchar', - 'cwchar', - 'cwctype', - ]) - -# Assertion macros. These are defined in base/logging.h and -# testing/base/gunit.h. Note that the _M versions need to come first -# for substring matching to work. -_CHECK_MACROS = [ - 'DCHECK', 'CHECK', - 'EXPECT_TRUE_M', 'EXPECT_TRUE', - 'ASSERT_TRUE_M', 'ASSERT_TRUE', - 'EXPECT_FALSE_M', 'EXPECT_FALSE', - 'ASSERT_FALSE_M', 'ASSERT_FALSE', - ] - -# Replacement macros for CHECK/DCHECK/EXPECT_TRUE/EXPECT_FALSE -_CHECK_REPLACEMENT = dict([(m, {}) for m in _CHECK_MACROS]) - -for op, replacement in [('==', 'EQ'), ('!=', 'NE'), - ('>=', 'GE'), ('>', 'GT'), - ('<=', 'LE'), ('<', 'LT')]: - _CHECK_REPLACEMENT['DCHECK'][op] = 'DCHECK_%s' % replacement - _CHECK_REPLACEMENT['CHECK'][op] = 'CHECK_%s' % replacement - _CHECK_REPLACEMENT['EXPECT_TRUE'][op] = 'EXPECT_%s' % replacement - _CHECK_REPLACEMENT['ASSERT_TRUE'][op] = 'ASSERT_%s' % replacement - _CHECK_REPLACEMENT['EXPECT_TRUE_M'][op] = 'EXPECT_%s_M' % replacement - _CHECK_REPLACEMENT['ASSERT_TRUE_M'][op] = 'ASSERT_%s_M' % replacement - -for op, inv_replacement in [('==', 'NE'), ('!=', 'EQ'), - ('>=', 'LT'), ('>', 'LE'), - ('<=', 'GT'), ('<', 'GE')]: - _CHECK_REPLACEMENT['EXPECT_FALSE'][op] = 'EXPECT_%s' % inv_replacement - _CHECK_REPLACEMENT['ASSERT_FALSE'][op] = 'ASSERT_%s' % inv_replacement - _CHECK_REPLACEMENT['EXPECT_FALSE_M'][op] = 'EXPECT_%s_M' % inv_replacement - _CHECK_REPLACEMENT['ASSERT_FALSE_M'][op] = 'ASSERT_%s_M' % inv_replacement - -# Alternative tokens and their replacements. For full list, see section 2.5 -# Alternative tokens [lex.digraph] in the C++ standard. -# -# Digraphs (such as '%:') are not included here since it's a mess to -# match those on a word boundary. -_ALT_TOKEN_REPLACEMENT = { - 'and': '&&', - 'bitor': '|', - 'or': '||', - 'xor': '^', - 'compl': '~', - 'bitand': '&', - 'and_eq': '&=', - 'or_eq': '|=', - 'xor_eq': '^=', - 'not': '!', - 'not_eq': '!=' - } - -# Compile regular expression that matches all the above keywords. The "[ =()]" -# bit is meant to avoid matching these keywords outside of boolean expressions. -# -# False positives include C-style multi-line comments and multi-line strings -# but those have always been troublesome for cpplint. -_ALT_TOKEN_REPLACEMENT_PATTERN = re.compile( - r'[ =()](' + ('|'.join(_ALT_TOKEN_REPLACEMENT.keys())) + r')(?=[ (]|$)') - - -# These constants define types of headers for use with -# _IncludeState.CheckNextIncludeOrder(). -_C_SYS_HEADER = 1 -_CPP_SYS_HEADER = 2 -_LIKELY_MY_HEADER = 3 -_POSSIBLE_MY_HEADER = 4 -_OTHER_HEADER = 5 - -# These constants define the current inline assembly state -_NO_ASM = 0 # Outside of inline assembly block -_INSIDE_ASM = 1 # Inside inline assembly block -_END_ASM = 2 # Last line of inline assembly block -_BLOCK_ASM = 3 # The whole block is an inline assembly block - -# Match start of assembly blocks -_MATCH_ASM = re.compile(r'^\s*(?:asm|_asm|__asm|__asm__)' - r'(?:\s+(volatile|__volatile__))?' - r'\s*[{(]') - - -_regexp_compile_cache = {} - -# Finds occurrences of NOLINT or NOLINT(...). -_RE_SUPPRESSION = re.compile(r'\bNOLINT\b(\([^)]*\))?') - -# {str, set(int)}: a map from error categories to sets of linenumbers -# on which those errors are expected and should be suppressed. -_error_suppressions = {} - -# The root directory used for deriving header guard CPP variable. -# This is set by --root flag. -_root = None - -# The allowed line length of files. -# This is set by --linelength flag. -_line_length = 80 - -# The allowed extensions for file names -# This is set by --extensions flag. -_valid_extensions = set(['cc', 'h', 'cpp', 'cu', 'cuh']) - -def ParseNolintSuppressions(filename, raw_line, linenum, error): - """Updates the global list of error-suppressions. - - Parses any NOLINT comments on the current line, updating the global - error_suppressions store. Reports an error if the NOLINT comment - was malformed. - - Args: - filename: str, the name of the input file. - raw_line: str, the line of input text, with comments. - linenum: int, the number of the current line. - error: function, an error handler. - """ - # FIXME(adonovan): "NOLINT(" is misparsed as NOLINT(*). - matched = _RE_SUPPRESSION.search(raw_line) - if matched: - category = matched.group(1) - if category in (None, '(*)'): # => "suppress all" - _error_suppressions.setdefault(None, set()).add(linenum) - else: - if category.startswith('(') and category.endswith(')'): - category = category[1:-1] - if category in _ERROR_CATEGORIES: - _error_suppressions.setdefault(category, set()).add(linenum) - else: - error(filename, linenum, 'readability/nolint', 5, - 'Unknown NOLINT error category: %s' % category) - - -def ResetNolintSuppressions(): - "Resets the set of NOLINT suppressions to empty." - _error_suppressions.clear() - - -def IsErrorSuppressedByNolint(category, linenum): - """Returns true if the specified error category is suppressed on this line. - - Consults the global error_suppressions map populated by - ParseNolintSuppressions/ResetNolintSuppressions. - - Args: - category: str, the category of the error. - linenum: int, the current line number. - Returns: - bool, True iff the error should be suppressed due to a NOLINT comment. - """ - return (linenum in _error_suppressions.get(category, set()) or - linenum in _error_suppressions.get(None, set())) - -def Match(pattern, s): - """Matches the string with the pattern, caching the compiled regexp.""" - # The regexp compilation caching is inlined in both Match and Search for - # performance reasons; factoring it out into a separate function turns out - # to be noticeably expensive. - if pattern not in _regexp_compile_cache: - _regexp_compile_cache[pattern] = sre_compile.compile(pattern) - return _regexp_compile_cache[pattern].match(s) - - -def ReplaceAll(pattern, rep, s): - """Replaces instances of pattern in a string with a replacement. - - The compiled regex is kept in a cache shared by Match and Search. - - Args: - pattern: regex pattern - rep: replacement text - s: search string - - Returns: - string with replacements made (or original string if no replacements) - """ - if pattern not in _regexp_compile_cache: - _regexp_compile_cache[pattern] = sre_compile.compile(pattern) - return _regexp_compile_cache[pattern].sub(rep, s) - - -def Search(pattern, s): - """Searches the string for the pattern, caching the compiled regexp.""" - if pattern not in _regexp_compile_cache: - _regexp_compile_cache[pattern] = sre_compile.compile(pattern) - return _regexp_compile_cache[pattern].search(s) - - -class _IncludeState(dict): - """Tracks line numbers for includes, and the order in which includes appear. - - As a dict, an _IncludeState object serves as a mapping between include - filename and line number on which that file was included. - - Call CheckNextIncludeOrder() once for each header in the file, passing - in the type constants defined above. Calls in an illegal order will - raise an _IncludeError with an appropriate error message. - - """ - # self._section will move monotonically through this set. If it ever - # needs to move backwards, CheckNextIncludeOrder will raise an error. - _INITIAL_SECTION = 0 - _MY_H_SECTION = 1 - _C_SECTION = 2 - _CPP_SECTION = 3 - _OTHER_H_SECTION = 4 - - _TYPE_NAMES = { - _C_SYS_HEADER: 'C system header', - _CPP_SYS_HEADER: 'C++ system header', - _LIKELY_MY_HEADER: 'header this file implements', - _POSSIBLE_MY_HEADER: 'header this file may implement', - _OTHER_HEADER: 'other header', - } - _SECTION_NAMES = { - _INITIAL_SECTION: "... nothing. (This can't be an error.)", - _MY_H_SECTION: 'a header this file implements', - _C_SECTION: 'C system header', - _CPP_SECTION: 'C++ system header', - _OTHER_H_SECTION: 'other header', - } - - def __init__(self): - dict.__init__(self) - self.ResetSection() - - def ResetSection(self): - # The name of the current section. - self._section = self._INITIAL_SECTION - # The path of last found header. - self._last_header = '' - - def SetLastHeader(self, header_path): - self._last_header = header_path - - def CanonicalizeAlphabeticalOrder(self, header_path): - """Returns a path canonicalized for alphabetical comparison. - - - replaces "-" with "_" so they both cmp the same. - - removes '-inl' since we don't require them to be after the main header. - - lowercase everything, just in case. - - Args: - header_path: Path to be canonicalized. - - Returns: - Canonicalized path. - """ - return header_path.replace('-inl.h', '.h').replace('-', '_').lower() - - def IsInAlphabeticalOrder(self, clean_lines, linenum, header_path): - """Check if a header is in alphabetical order with the previous header. - - Args: - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - header_path: Canonicalized header to be checked. - - Returns: - Returns true if the header is in alphabetical order. - """ - # If previous section is different from current section, _last_header will - # be reset to empty string, so it's always less than current header. - # - # If previous line was a blank line, assume that the headers are - # intentionally sorted the way they are. - if (self._last_header > header_path and - not Match(r'^\s*$', clean_lines.elided[linenum - 1])): - return False - return True - - def CheckNextIncludeOrder(self, header_type): - """Returns a non-empty error message if the next header is out of order. - - This function also updates the internal state to be ready to check - the next include. - - Args: - header_type: One of the _XXX_HEADER constants defined above. - - Returns: - The empty string if the header is in the right order, or an - error message describing what's wrong. - - """ - error_message = ('Found %s after %s' % - (self._TYPE_NAMES[header_type], - self._SECTION_NAMES[self._section])) - - last_section = self._section - - if header_type == _C_SYS_HEADER: - if self._section <= self._C_SECTION: - self._section = self._C_SECTION - else: - self._last_header = '' - return error_message - elif header_type == _CPP_SYS_HEADER: - if self._section <= self._CPP_SECTION: - self._section = self._CPP_SECTION - else: - self._last_header = '' - return error_message - elif header_type == _LIKELY_MY_HEADER: - if self._section <= self._MY_H_SECTION: - self._section = self._MY_H_SECTION - else: - self._section = self._OTHER_H_SECTION - elif header_type == _POSSIBLE_MY_HEADER: - if self._section <= self._MY_H_SECTION: - self._section = self._MY_H_SECTION - else: - # This will always be the fallback because we're not sure - # enough that the header is associated with this file. - self._section = self._OTHER_H_SECTION - else: - assert header_type == _OTHER_HEADER - self._section = self._OTHER_H_SECTION - - if last_section != self._section: - self._last_header = '' - - return '' - - -class _CppLintState(object): - """Maintains module-wide state..""" - - def __init__(self): - self.verbose_level = 1 # global setting. - self.error_count = 0 # global count of reported errors - # filters to apply when emitting error messages - self.filters = _DEFAULT_FILTERS[:] - self.counting = 'total' # In what way are we counting errors? - self.errors_by_category = {} # string to int dict storing error counts - - # output format: - # "emacs" - format that emacs can parse (default) - # "vs7" - format that Microsoft Visual Studio 7 can parse - self.output_format = 'emacs' - - def SetOutputFormat(self, output_format): - """Sets the output format for errors.""" - self.output_format = output_format - - def SetVerboseLevel(self, level): - """Sets the module's verbosity, and returns the previous setting.""" - last_verbose_level = self.verbose_level - self.verbose_level = level - return last_verbose_level - - def SetCountingStyle(self, counting_style): - """Sets the module's counting options.""" - self.counting = counting_style - - def SetFilters(self, filters): - """Sets the error-message filters. - - These filters are applied when deciding whether to emit a given - error message. - - Args: - filters: A string of comma-separated filters (eg "+whitespace/indent"). - Each filter should start with + or -; else we die. - - Raises: - ValueError: The comma-separated filters did not all start with '+' or '-'. - E.g. "-,+whitespace,-whitespace/indent,whitespace/badfilter" - """ - # Default filters always have less priority than the flag ones. - self.filters = _DEFAULT_FILTERS[:] - for filt in filters.split(','): - clean_filt = filt.strip() - if clean_filt: - self.filters.append(clean_filt) - for filt in self.filters: - if not (filt.startswith('+') or filt.startswith('-')): - raise ValueError('Every filter in --filters must start with + or -' - ' (%s does not)' % filt) - - def ResetErrorCounts(self): - """Sets the module's error statistic back to zero.""" - self.error_count = 0 - self.errors_by_category = {} - - def IncrementErrorCount(self, category): - """Bumps the module's error statistic.""" - self.error_count += 1 - if self.counting in ('toplevel', 'detailed'): - if self.counting != 'detailed': - category = category.split('/')[0] - if category not in self.errors_by_category: - self.errors_by_category[category] = 0 - self.errors_by_category[category] += 1 - - def PrintErrorCounts(self): - """Print a summary of errors by category, and the total.""" - for category, count in self.errors_by_category.iteritems(): - sys.stderr.write('Category \'%s\' errors found: %d\n' % - (category, count)) - sys.stderr.write('Total errors found: %d\n' % self.error_count) - -_cpplint_state = _CppLintState() - - -def _OutputFormat(): - """Gets the module's output format.""" - return _cpplint_state.output_format - - -def _SetOutputFormat(output_format): - """Sets the module's output format.""" - _cpplint_state.SetOutputFormat(output_format) - - -def _VerboseLevel(): - """Returns the module's verbosity setting.""" - return _cpplint_state.verbose_level - - -def _SetVerboseLevel(level): - """Sets the module's verbosity, and returns the previous setting.""" - return _cpplint_state.SetVerboseLevel(level) - - -def _SetCountingStyle(level): - """Sets the module's counting options.""" - _cpplint_state.SetCountingStyle(level) - - -def _Filters(): - """Returns the module's list of output filters, as a list.""" - return _cpplint_state.filters - - -def _SetFilters(filters): - """Sets the module's error-message filters. - - These filters are applied when deciding whether to emit a given - error message. - - Args: - filters: A string of comma-separated filters (eg "whitespace/indent"). - Each filter should start with + or -; else we die. - """ - _cpplint_state.SetFilters(filters) - - -class _FunctionState(object): - """Tracks current function name and the number of lines in its body.""" - - _NORMAL_TRIGGER = 250 # for --v=0, 500 for --v=1, etc. - _TEST_TRIGGER = 400 # about 50% more than _NORMAL_TRIGGER. - - def __init__(self): - self.in_a_function = False - self.lines_in_function = 0 - self.current_function = '' - - def Begin(self, function_name): - """Start analyzing function body. - - Args: - function_name: The name of the function being tracked. - """ - self.in_a_function = True - self.lines_in_function = 0 - self.current_function = function_name - - def Count(self): - """Count line in current function body.""" - if self.in_a_function: - self.lines_in_function += 1 - - def Check(self, error, filename, linenum): - """Report if too many lines in function body. - - Args: - error: The function to call with any errors found. - filename: The name of the current file. - linenum: The number of the line to check. - """ - if Match(r'T(EST|est)', self.current_function): - base_trigger = self._TEST_TRIGGER - else: - base_trigger = self._NORMAL_TRIGGER - trigger = base_trigger * 2**_VerboseLevel() - - if self.lines_in_function > trigger: - error_level = int(math.log(self.lines_in_function / base_trigger, 2)) - # 50 => 0, 100 => 1, 200 => 2, 400 => 3, 800 => 4, 1600 => 5, ... - if error_level > 5: - error_level = 5 - error(filename, linenum, 'readability/fn_size', error_level, - 'Small and focused functions are preferred:' - ' %s has %d non-comment lines' - ' (error triggered by exceeding %d lines).' % ( - self.current_function, self.lines_in_function, trigger)) - - def End(self): - """Stop analyzing function body.""" - self.in_a_function = False - - -class _IncludeError(Exception): - """Indicates a problem with the include order in a file.""" - pass - - -class FileInfo: - """Provides utility functions for filenames. - - FileInfo provides easy access to the components of a file's path - relative to the project root. - """ - - def __init__(self, filename): - self._filename = filename - - def FullName(self): - """Make Windows paths like Unix.""" - return os.path.abspath(self._filename).replace('\\', '/') - - def RepositoryName(self): - """FullName after removing the local path to the repository. - - If we have a real absolute path name here we can try to do something smart: - detecting the root of the checkout and truncating /path/to/checkout from - the name so that we get header guards that don't include things like - "C:\Documents and Settings\..." or "/home/username/..." in them and thus - people on different computers who have checked the source out to different - locations won't see bogus errors. - """ - fullname = self.FullName() - - if os.path.exists(fullname): - project_dir = os.path.dirname(fullname) - - if os.path.exists(os.path.join(project_dir, ".svn")): - # If there's a .svn file in the current directory, we recursively look - # up the directory tree for the top of the SVN checkout - root_dir = project_dir - one_up_dir = os.path.dirname(root_dir) - while os.path.exists(os.path.join(one_up_dir, ".svn")): - root_dir = os.path.dirname(root_dir) - one_up_dir = os.path.dirname(one_up_dir) - - prefix = os.path.commonprefix([root_dir, project_dir]) - return fullname[len(prefix) + 1:] - - # Not SVN <= 1.6? Try to find a git, hg, or svn top level directory by - # searching up from the current path. - root_dir = os.path.dirname(fullname) - while (root_dir != os.path.dirname(root_dir) and - not os.path.exists(os.path.join(root_dir, ".git")) and - not os.path.exists(os.path.join(root_dir, ".hg")) and - not os.path.exists(os.path.join(root_dir, ".svn"))): - root_dir = os.path.dirname(root_dir) - - if (os.path.exists(os.path.join(root_dir, ".git")) or - os.path.exists(os.path.join(root_dir, ".hg")) or - os.path.exists(os.path.join(root_dir, ".svn"))): - prefix = os.path.commonprefix([root_dir, project_dir]) - return fullname[len(prefix) + 1:] - - # Don't know what to do; header guard warnings may be wrong... - return fullname - - def Split(self): - """Splits the file into the directory, basename, and extension. - - For 'chrome/browser/browser.cc', Split() would - return ('chrome/browser', 'browser', '.cc') - - Returns: - A tuple of (directory, basename, extension). - """ - - googlename = self.RepositoryName() - project, rest = os.path.split(googlename) - return (project,) + os.path.splitext(rest) - - def BaseName(self): - """File base name - text after the final slash, before the final period.""" - return self.Split()[1] - - def Extension(self): - """File extension - text following the final period.""" - return self.Split()[2] - - def NoExtension(self): - """File has no source file extension.""" - return '/'.join(self.Split()[0:2]) - - def IsSource(self): - """File has a source file extension.""" - return self.Extension()[1:] in ('c', 'cc', 'cpp', 'cxx') - - -def _ShouldPrintError(category, confidence, linenum): - """If confidence >= verbose, category passes filter and is not suppressed.""" - - # There are three ways we might decide not to print an error message: - # a "NOLINT(category)" comment appears in the source, - # the verbosity level isn't high enough, or the filters filter it out. - if IsErrorSuppressedByNolint(category, linenum): - return False - if confidence < _cpplint_state.verbose_level: - return False - - is_filtered = False - for one_filter in _Filters(): - if one_filter.startswith('-'): - if category.startswith(one_filter[1:]): - is_filtered = True - elif one_filter.startswith('+'): - if category.startswith(one_filter[1:]): - is_filtered = False - else: - assert False # should have been checked for in SetFilter. - if is_filtered: - return False - - return True - - -def Error(filename, linenum, category, confidence, message): - """Logs the fact we've found a lint error. - - We log where the error was found, and also our confidence in the error, - that is, how certain we are this is a legitimate style regression, and - not a misidentification or a use that's sometimes justified. - - False positives can be suppressed by the use of - "cpplint(category)" comments on the offending line. These are - parsed into _error_suppressions. - - Args: - filename: The name of the file containing the error. - linenum: The number of the line containing the error. - category: A string used to describe the "category" this bug - falls under: "whitespace", say, or "runtime". Categories - may have a hierarchy separated by slashes: "whitespace/indent". - confidence: A number from 1-5 representing a confidence score for - the error, with 5 meaning that we are certain of the problem, - and 1 meaning that it could be a legitimate construct. - message: The error message. - """ - if _ShouldPrintError(category, confidence, linenum): - _cpplint_state.IncrementErrorCount(category) - if _cpplint_state.output_format == 'vs7': - sys.stderr.write('%s(%s): %s [%s] [%d]\n' % ( - filename, linenum, message, category, confidence)) - elif _cpplint_state.output_format == 'eclipse': - sys.stderr.write('%s:%s: warning: %s [%s] [%d]\n' % ( - filename, linenum, message, category, confidence)) - else: - sys.stderr.write('%s:%s: %s [%s] [%d]\n' % ( - filename, linenum, message, category, confidence)) - - -# Matches standard C++ escape sequences per 2.13.2.3 of the C++ standard. -_RE_PATTERN_CLEANSE_LINE_ESCAPES = re.compile( - r'\\([abfnrtv?"\\\']|\d+|x[0-9a-fA-F]+)') -# Matches strings. Escape codes should already be removed by ESCAPES. -_RE_PATTERN_CLEANSE_LINE_DOUBLE_QUOTES = re.compile(r'"[^"]*"') -# Matches characters. Escape codes should already be removed by ESCAPES. -_RE_PATTERN_CLEANSE_LINE_SINGLE_QUOTES = re.compile(r"'.'") -# Matches multi-line C++ comments. -# This RE is a little bit more complicated than one might expect, because we -# have to take care of space removals tools so we can handle comments inside -# statements better. -# The current rule is: We only clear spaces from both sides when we're at the -# end of the line. Otherwise, we try to remove spaces from the right side, -# if this doesn't work we try on left side but only if there's a non-character -# on the right. -_RE_PATTERN_CLEANSE_LINE_C_COMMENTS = re.compile( - r"""(\s*/\*.*\*/\s*$| - /\*.*\*/\s+| - \s+/\*.*\*/(?=\W)| - /\*.*\*/)""", re.VERBOSE) - - -def IsCppString(line): - """Does line terminate so, that the next symbol is in string constant. - - This function does not consider single-line nor multi-line comments. - - Args: - line: is a partial line of code starting from the 0..n. - - Returns: - True, if next character appended to 'line' is inside a - string constant. - """ - - line = line.replace(r'\\', 'XX') # after this, \\" does not match to \" - return ((line.count('"') - line.count(r'\"') - line.count("'\"'")) & 1) == 1 - - -def CleanseRawStrings(raw_lines): - """Removes C++11 raw strings from lines. - - Before: - static const char kData[] = R"( - multi-line string - )"; - - After: - static const char kData[] = "" - (replaced by blank line) - ""; - - Args: - raw_lines: list of raw lines. - - Returns: - list of lines with C++11 raw strings replaced by empty strings. - """ - - delimiter = None - lines_without_raw_strings = [] - for line in raw_lines: - if delimiter: - # Inside a raw string, look for the end - end = line.find(delimiter) - if end >= 0: - # Found the end of the string, match leading space for this - # line and resume copying the original lines, and also insert - # a "" on the last line. - leading_space = Match(r'^(\s*)\S', line) - line = leading_space.group(1) + '""' + line[end + len(delimiter):] - delimiter = None - else: - # Haven't found the end yet, append a blank line. - line = '' - - else: - # Look for beginning of a raw string. - # See 2.14.15 [lex.string] for syntax. - matched = Match(r'^(.*)\b(?:R|u8R|uR|UR|LR)"([^\s\\()]*)\((.*)$', line) - if matched: - delimiter = ')' + matched.group(2) + '"' - - end = matched.group(3).find(delimiter) - if end >= 0: - # Raw string ended on same line - line = (matched.group(1) + '""' + - matched.group(3)[end + len(delimiter):]) - delimiter = None - else: - # Start of a multi-line raw string - line = matched.group(1) + '""' - - lines_without_raw_strings.append(line) - - # TODO(unknown): if delimiter is not None here, we might want to - # emit a warning for unterminated string. - return lines_without_raw_strings - - -def FindNextMultiLineCommentStart(lines, lineix): - """Find the beginning marker for a multiline comment.""" - while lineix < len(lines): - if lines[lineix].strip().startswith('/*'): - # Only return this marker if the comment goes beyond this line - if lines[lineix].strip().find('*/', 2) < 0: - return lineix - lineix += 1 - return len(lines) - - -def FindNextMultiLineCommentEnd(lines, lineix): - """We are inside a comment, find the end marker.""" - while lineix < len(lines): - if lines[lineix].strip().endswith('*/'): - return lineix - lineix += 1 - return len(lines) - - -def RemoveMultiLineCommentsFromRange(lines, begin, end): - """Clears a range of lines for multi-line comments.""" - # Having // dummy comments makes the lines non-empty, so we will not get - # unnecessary blank line warnings later in the code. - for i in range(begin, end): - lines[i] = '// dummy' - - -def RemoveMultiLineComments(filename, lines, error): - """Removes multiline (c-style) comments from lines.""" - lineix = 0 - while lineix < len(lines): - lineix_begin = FindNextMultiLineCommentStart(lines, lineix) - if lineix_begin >= len(lines): - return - lineix_end = FindNextMultiLineCommentEnd(lines, lineix_begin) - if lineix_end >= len(lines): - error(filename, lineix_begin + 1, 'readability/multiline_comment', 5, - 'Could not find end of multi-line comment') - return - RemoveMultiLineCommentsFromRange(lines, lineix_begin, lineix_end + 1) - lineix = lineix_end + 1 - - -def CleanseComments(line): - """Removes //-comments and single-line C-style /* */ comments. - - Args: - line: A line of C++ source. - - Returns: - The line with single-line comments removed. - """ - commentpos = line.find('//') - if commentpos != -1 and not IsCppString(line[:commentpos]): - line = line[:commentpos].rstrip() - # get rid of /* ... */ - return _RE_PATTERN_CLEANSE_LINE_C_COMMENTS.sub('', line) - - -class CleansedLines(object): - """Holds 3 copies of all lines with different preprocessing applied to them. - - 1) elided member contains lines without strings and comments, - 2) lines member contains lines without comments, and - 3) raw_lines member contains all the lines without processing. - All these three members are of , and of the same length. - """ - - def __init__(self, lines): - self.elided = [] - self.lines = [] - self.raw_lines = lines - self.num_lines = len(lines) - self.lines_without_raw_strings = CleanseRawStrings(lines) - for linenum in range(len(self.lines_without_raw_strings)): - self.lines.append(CleanseComments( - self.lines_without_raw_strings[linenum])) - elided = self._CollapseStrings(self.lines_without_raw_strings[linenum]) - self.elided.append(CleanseComments(elided)) - - def NumLines(self): - """Returns the number of lines represented.""" - return self.num_lines - - @staticmethod - def _CollapseStrings(elided): - """Collapses strings and chars on a line to simple "" or '' blocks. - - We nix strings first so we're not fooled by text like '"http://"' - - Args: - elided: The line being processed. - - Returns: - The line with collapsed strings. - """ - if not _RE_PATTERN_INCLUDE.match(elided): - # Remove escaped characters first to make quote/single quote collapsing - # basic. Things that look like escaped characters shouldn't occur - # outside of strings and chars. - elided = _RE_PATTERN_CLEANSE_LINE_ESCAPES.sub('', elided) - elided = _RE_PATTERN_CLEANSE_LINE_SINGLE_QUOTES.sub("''", elided) - elided = _RE_PATTERN_CLEANSE_LINE_DOUBLE_QUOTES.sub('""', elided) - return elided - - -def FindEndOfExpressionInLine(line, startpos, depth, startchar, endchar): - """Find the position just after the matching endchar. - - Args: - line: a CleansedLines line. - startpos: start searching at this position. - depth: nesting level at startpos. - startchar: expression opening character. - endchar: expression closing character. - - Returns: - On finding matching endchar: (index just after matching endchar, 0) - Otherwise: (-1, new depth at end of this line) - """ - for i in xrange(startpos, len(line)): - if line[i] == startchar: - depth += 1 - elif line[i] == endchar: - depth -= 1 - if depth == 0: - return (i + 1, 0) - return (-1, depth) - - -def CloseExpression(clean_lines, linenum, pos): - """If input points to ( or { or [ or <, finds the position that closes it. - - If lines[linenum][pos] points to a '(' or '{' or '[' or '<', finds the - linenum/pos that correspond to the closing of the expression. - - Args: - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - pos: A position on the line. - - Returns: - A tuple (line, linenum, pos) pointer *past* the closing brace, or - (line, len(lines), -1) if we never find a close. Note we ignore - strings and comments when matching; and the line we return is the - 'cleansed' line at linenum. - """ - - line = clean_lines.elided[linenum] - startchar = line[pos] - if startchar not in '({[<': - return (line, clean_lines.NumLines(), -1) - if startchar == '(': endchar = ')' - if startchar == '[': endchar = ']' - if startchar == '{': endchar = '}' - if startchar == '<': endchar = '>' - - # Check first line - (end_pos, num_open) = FindEndOfExpressionInLine( - line, pos, 0, startchar, endchar) - if end_pos > -1: - return (line, linenum, end_pos) - - # Continue scanning forward - while linenum < clean_lines.NumLines() - 1: - linenum += 1 - line = clean_lines.elided[linenum] - (end_pos, num_open) = FindEndOfExpressionInLine( - line, 0, num_open, startchar, endchar) - if end_pos > -1: - return (line, linenum, end_pos) - - # Did not find endchar before end of file, give up - return (line, clean_lines.NumLines(), -1) - - -def FindStartOfExpressionInLine(line, endpos, depth, startchar, endchar): - """Find position at the matching startchar. - - This is almost the reverse of FindEndOfExpressionInLine, but note - that the input position and returned position differs by 1. - - Args: - line: a CleansedLines line. - endpos: start searching at this position. - depth: nesting level at endpos. - startchar: expression opening character. - endchar: expression closing character. - - Returns: - On finding matching startchar: (index at matching startchar, 0) - Otherwise: (-1, new depth at beginning of this line) - """ - for i in xrange(endpos, -1, -1): - if line[i] == endchar: - depth += 1 - elif line[i] == startchar: - depth -= 1 - if depth == 0: - return (i, 0) - return (-1, depth) - - -def ReverseCloseExpression(clean_lines, linenum, pos): - """If input points to ) or } or ] or >, finds the position that opens it. - - If lines[linenum][pos] points to a ')' or '}' or ']' or '>', finds the - linenum/pos that correspond to the opening of the expression. - - Args: - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - pos: A position on the line. - - Returns: - A tuple (line, linenum, pos) pointer *at* the opening brace, or - (line, 0, -1) if we never find the matching opening brace. Note - we ignore strings and comments when matching; and the line we - return is the 'cleansed' line at linenum. - """ - line = clean_lines.elided[linenum] - endchar = line[pos] - if endchar not in ')}]>': - return (line, 0, -1) - if endchar == ')': startchar = '(' - if endchar == ']': startchar = '[' - if endchar == '}': startchar = '{' - if endchar == '>': startchar = '<' - - # Check last line - (start_pos, num_open) = FindStartOfExpressionInLine( - line, pos, 0, startchar, endchar) - if start_pos > -1: - return (line, linenum, start_pos) - - # Continue scanning backward - while linenum > 0: - linenum -= 1 - line = clean_lines.elided[linenum] - (start_pos, num_open) = FindStartOfExpressionInLine( - line, len(line) - 1, num_open, startchar, endchar) - if start_pos > -1: - return (line, linenum, start_pos) - - # Did not find startchar before beginning of file, give up - return (line, 0, -1) - - -def CheckForCopyright(filename, lines, error): - """Logs an error if no Copyright message appears at the top of the file.""" - - # We'll say it should occur by line 10. Don't forget there's a - # dummy line at the front. - for line in xrange(1, min(len(lines), 11)): - if re.search(r'Copyright', lines[line], re.I): break - else: # means no copyright line was found - error(filename, 0, 'legal/copyright', 5, - 'No copyright message found. ' - 'You should have a line: "Copyright [year] "') - - -def GetHeaderGuardCPPVariable(filename): - """Returns the CPP variable that should be used as a header guard. - - Args: - filename: The name of a C++ header file. - - Returns: - The CPP variable that should be used as a header guard in the - named file. - - """ - - # Restores original filename in case that cpplint is invoked from Emacs's - # flymake. - filename = re.sub(r'_flymake\.h$', '.h', filename) - filename = re.sub(r'/\.flymake/([^/]*)$', r'/\1', filename) - - fileinfo = FileInfo(filename) - file_path_from_root = fileinfo.RepositoryName() - if _root: - file_path_from_root = re.sub('^' + _root + os.sep, '', file_path_from_root) - return re.sub(r'[-./\s]', '_', file_path_from_root).upper() + '_' - - -def CheckForHeaderGuard(filename, lines, error): - """Checks that the file contains a header guard. - - Logs an error if no #ifndef header guard is present. For other - headers, checks that the full pathname is used. - - Args: - filename: The name of the C++ header file. - lines: An array of strings, each representing a line of the file. - error: The function to call with any errors found. - """ - - cppvar = GetHeaderGuardCPPVariable(filename) - - ifndef = None - ifndef_linenum = 0 - define = None - endif = None - endif_linenum = 0 - for linenum, line in enumerate(lines): - # Already been well guarded, no need for further checking. - if line.strip() == "#pragma once": - return - linesplit = line.split() - if len(linesplit) >= 2: - # find the first occurrence of #ifndef and #define, save arg - if not ifndef and linesplit[0] == '#ifndef': - # set ifndef to the header guard presented on the #ifndef line. - ifndef = linesplit[1] - ifndef_linenum = linenum - if not define and linesplit[0] == '#define': - define = linesplit[1] - # find the last occurrence of #endif, save entire line - if line.startswith('#endif'): - endif = line - endif_linenum = linenum - - if not ifndef: - error(filename, 0, 'build/header_guard', 5, - 'No #ifndef header guard found, suggested CPP variable is: %s' % - cppvar) - return - - if not define: - error(filename, 0, 'build/header_guard', 5, - 'No #define header guard found, suggested CPP variable is: %s' % - cppvar) - return - - # The guard should be PATH_FILE_H_, but we also allow PATH_FILE_H__ - # for backward compatibility. - if ifndef != cppvar: - error_level = 0 - if ifndef != cppvar + '_': - error_level = 5 - - ParseNolintSuppressions(filename, lines[ifndef_linenum], ifndef_linenum, - error) - error(filename, ifndef_linenum, 'build/header_guard', error_level, - '#ifndef header guard has wrong style, please use: %s' % cppvar) - - if define != ifndef: - error(filename, 0, 'build/header_guard', 5, - '#ifndef and #define don\'t match, suggested CPP variable is: %s' % - cppvar) - return - - if endif != ('#endif // %s' % cppvar): - error_level = 0 - if endif != ('#endif // %s' % (cppvar + '_')): - error_level = 5 - - ParseNolintSuppressions(filename, lines[endif_linenum], endif_linenum, - error) - error(filename, endif_linenum, 'build/header_guard', error_level, - '#endif line should be "#endif // %s"' % cppvar) - - -def CheckForBadCharacters(filename, lines, error): - """Logs an error for each line containing bad characters. - - Two kinds of bad characters: - - 1. Unicode replacement characters: These indicate that either the file - contained invalid UTF-8 (likely) or Unicode replacement characters (which - it shouldn't). Note that it's possible for this to throw off line - numbering if the invalid UTF-8 occurred adjacent to a newline. - - 2. NUL bytes. These are problematic for some tools. - - Args: - filename: The name of the current file. - lines: An array of strings, each representing a line of the file. - error: The function to call with any errors found. - """ - for linenum, line in enumerate(lines): - if u'\ufffd' in line: - error(filename, linenum, 'readability/utf8', 5, - 'Line contains invalid UTF-8 (or Unicode replacement character).') - if '\0' in line: - error(filename, linenum, 'readability/nul', 5, 'Line contains NUL byte.') - - -def CheckForNewlineAtEOF(filename, lines, error): - """Logs an error if there is no newline char at the end of the file. - - Args: - filename: The name of the current file. - lines: An array of strings, each representing a line of the file. - error: The function to call with any errors found. - """ - - # The array lines() was created by adding two newlines to the - # original file (go figure), then splitting on \n. - # To verify that the file ends in \n, we just have to make sure the - # last-but-two element of lines() exists and is empty. - if len(lines) < 3 or lines[-2]: - error(filename, len(lines) - 2, 'whitespace/ending_newline', 5, - 'Could not find a newline character at the end of the file.') - - -def CheckForMultilineCommentsAndStrings(filename, clean_lines, linenum, error): - """Logs an error if we see /* ... */ or "..." that extend past one line. - - /* ... */ comments are legit inside macros, for one line. - Otherwise, we prefer // comments, so it's ok to warn about the - other. Likewise, it's ok for strings to extend across multiple - lines, as long as a line continuation character (backslash) - terminates each line. Although not currently prohibited by the C++ - style guide, it's ugly and unnecessary. We don't do well with either - in this lint program, so we warn about both. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - line = clean_lines.elided[linenum] - - # Remove all \\ (escaped backslashes) from the line. They are OK, and the - # second (escaped) slash may trigger later \" detection erroneously. - line = line.replace('\\\\', '') - - if line.count('/*') > line.count('*/'): - error(filename, linenum, 'readability/multiline_comment', 5, - 'Complex multi-line /*...*/-style comment found. ' - 'Lint may give bogus warnings. ' - 'Consider replacing these with //-style comments, ' - 'with #if 0...#endif, ' - 'or with more clearly structured multi-line comments.') - - if (line.count('"') - line.count('\\"')) % 2: - error(filename, linenum, 'readability/multiline_string', 5, - 'Multi-line string ("...") found. This lint script doesn\'t ' - 'do well with such strings, and may give bogus warnings. ' - 'Use C++11 raw strings or concatenation instead.') - - -threading_list = ( - ('asctime(', 'asctime_r('), - ('ctime(', 'ctime_r('), - ('getgrgid(', 'getgrgid_r('), - ('getgrnam(', 'getgrnam_r('), - ('getlogin(', 'getlogin_r('), - ('getpwnam(', 'getpwnam_r('), - ('getpwuid(', 'getpwuid_r('), - ('gmtime(', 'gmtime_r('), - ('localtime(', 'localtime_r('), - ('rand(', 'rand_r('), - ('strtok(', 'strtok_r('), - ('ttyname(', 'ttyname_r('), - ) - - -def CheckPosixThreading(filename, clean_lines, linenum, error): - """Checks for calls to thread-unsafe functions. - - Much code has been originally written without consideration of - multi-threading. Also, engineers are relying on their old experience; - they have learned posix before threading extensions were added. These - tests guide the engineers to use thread-safe functions (when using - posix directly). - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - line = clean_lines.elided[linenum] - for single_thread_function, multithread_safe_function in threading_list: - ix = line.find(single_thread_function) - # Comparisons made explicit for clarity -- pylint: disable=g-explicit-bool-comparison - if ix >= 0 and (ix == 0 or (not line[ix - 1].isalnum() and - line[ix - 1] not in ('_', '.', '>'))): - error(filename, linenum, 'runtime/threadsafe_fn', 2, - 'Consider using ' + multithread_safe_function + - '...) instead of ' + single_thread_function + - '...) for improved thread safety.') - - -def CheckVlogArguments(filename, clean_lines, linenum, error): - """Checks that VLOG() is only used for defining a logging level. - - For example, VLOG(2) is correct. VLOG(INFO), VLOG(WARNING), VLOG(ERROR), and - VLOG(FATAL) are not. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - line = clean_lines.elided[linenum] - if Search(r'\bVLOG\((INFO|ERROR|WARNING|DFATAL|FATAL)\)', line): - error(filename, linenum, 'runtime/vlog', 5, - 'VLOG() should be used with numeric verbosity level. ' - 'Use LOG() if you want symbolic severity levels.') - - -# Matches invalid increment: *count++, which moves pointer instead of -# incrementing a value. -_RE_PATTERN_INVALID_INCREMENT = re.compile( - r'^\s*\*\w+(\+\+|--);') - - -def CheckInvalidIncrement(filename, clean_lines, linenum, error): - """Checks for invalid increment *count++. - - For example following function: - void increment_counter(int* count) { - *count++; - } - is invalid, because it effectively does count++, moving pointer, and should - be replaced with ++*count, (*count)++ or *count += 1. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - line = clean_lines.elided[linenum] - if _RE_PATTERN_INVALID_INCREMENT.match(line): - error(filename, linenum, 'runtime/invalid_increment', 5, - 'Changing pointer instead of value (or unused value of operator*).') - - -class _BlockInfo(object): - """Stores information about a generic block of code.""" - - def __init__(self, seen_open_brace): - self.seen_open_brace = seen_open_brace - self.open_parentheses = 0 - self.inline_asm = _NO_ASM - - def CheckBegin(self, filename, clean_lines, linenum, error): - """Run checks that applies to text up to the opening brace. - - This is mostly for checking the text after the class identifier - and the "{", usually where the base class is specified. For other - blocks, there isn't much to check, so we always pass. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - pass - - def CheckEnd(self, filename, clean_lines, linenum, error): - """Run checks that applies to text after the closing brace. - - This is mostly used for checking end of namespace comments. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - pass - - -class _ClassInfo(_BlockInfo): - """Stores information about a class.""" - - def __init__(self, name, class_or_struct, clean_lines, linenum): - _BlockInfo.__init__(self, False) - self.name = name - self.starting_linenum = linenum - self.is_derived = False - if class_or_struct == 'struct': - self.access = 'public' - self.is_struct = True - else: - self.access = 'private' - self.is_struct = False - - # Remember initial indentation level for this class. Using raw_lines here - # instead of elided to account for leading comments. - initial_indent = Match(r'^( *)\S', clean_lines.raw_lines[linenum]) - if initial_indent: - self.class_indent = len(initial_indent.group(1)) - else: - self.class_indent = 0 - - # Try to find the end of the class. This will be confused by things like: - # class A { - # } *x = { ... - # - # But it's still good enough for CheckSectionSpacing. - self.last_line = 0 - depth = 0 - for i in range(linenum, clean_lines.NumLines()): - line = clean_lines.elided[i] - depth += line.count('{') - line.count('}') - if not depth: - self.last_line = i - break - - def CheckBegin(self, filename, clean_lines, linenum, error): - # Look for a bare ':' - if Search('(^|[^:]):($|[^:])', clean_lines.elided[linenum]): - self.is_derived = True - - def CheckEnd(self, filename, clean_lines, linenum, error): - # Check that closing brace is aligned with beginning of the class. - # Only do this if the closing brace is indented by only whitespaces. - # This means we will not check single-line class definitions. - indent = Match(r'^( *)\}', clean_lines.elided[linenum]) - if indent and len(indent.group(1)) != self.class_indent: - if self.is_struct: - parent = 'struct ' + self.name - else: - parent = 'class ' + self.name - error(filename, linenum, 'whitespace/indent', 3, - 'Closing brace should be aligned with beginning of %s' % parent) - - -class _NamespaceInfo(_BlockInfo): - """Stores information about a namespace.""" - - def __init__(self, name, linenum): - _BlockInfo.__init__(self, False) - self.name = name or '' - self.starting_linenum = linenum - - def CheckEnd(self, filename, clean_lines, linenum, error): - """Check end of namespace comments.""" - line = clean_lines.raw_lines[linenum] - - # Check how many lines is enclosed in this namespace. Don't issue - # warning for missing namespace comments if there aren't enough - # lines. However, do apply checks if there is already an end of - # namespace comment and it's incorrect. - # - # TODO(unknown): We always want to check end of namespace comments - # if a namespace is large, but sometimes we also want to apply the - # check if a short namespace contained nontrivial things (something - # other than forward declarations). There is currently no logic on - # deciding what these nontrivial things are, so this check is - # triggered by namespace size only, which works most of the time. - if (linenum - self.starting_linenum < 10 - and not Match(r'};*\s*(//|/\*).*\bnamespace\b', line)): - return - - # Look for matching comment at end of namespace. - # - # Note that we accept C style "/* */" comments for terminating - # namespaces, so that code that terminate namespaces inside - # preprocessor macros can be cpplint clean. - # - # We also accept stuff like "// end of namespace ." with the - # period at the end. - # - # Besides these, we don't accept anything else, otherwise we might - # get false negatives when existing comment is a substring of the - # expected namespace. - if self.name: - # Named namespace - if not Match((r'};*\s*(//|/\*).*\bnamespace\s+' + re.escape(self.name) + - r'[\*/\.\\\s]*$'), - line): - error(filename, linenum, 'readability/namespace', 5, - 'Namespace should be terminated with "// namespace %s"' % - self.name) - else: - # Anonymous namespace - if not Match(r'};*\s*(//|/\*).*\bnamespace[\*/\.\\\s]*$', line): - error(filename, linenum, 'readability/namespace', 5, - 'Namespace should be terminated with "// namespace"') - - -class _PreprocessorInfo(object): - """Stores checkpoints of nesting stacks when #if/#else is seen.""" - - def __init__(self, stack_before_if): - # The entire nesting stack before #if - self.stack_before_if = stack_before_if - - # The entire nesting stack up to #else - self.stack_before_else = [] - - # Whether we have already seen #else or #elif - self.seen_else = False - - -class _NestingState(object): - """Holds states related to parsing braces.""" - - def __init__(self): - # Stack for tracking all braces. An object is pushed whenever we - # see a "{", and popped when we see a "}". Only 3 types of - # objects are possible: - # - _ClassInfo: a class or struct. - # - _NamespaceInfo: a namespace. - # - _BlockInfo: some other type of block. - self.stack = [] - - # Stack of _PreprocessorInfo objects. - self.pp_stack = [] - - def SeenOpenBrace(self): - """Check if we have seen the opening brace for the innermost block. - - Returns: - True if we have seen the opening brace, False if the innermost - block is still expecting an opening brace. - """ - return (not self.stack) or self.stack[-1].seen_open_brace - - def InNamespaceBody(self): - """Check if we are currently one level inside a namespace body. - - Returns: - True if top of the stack is a namespace block, False otherwise. - """ - return self.stack and isinstance(self.stack[-1], _NamespaceInfo) - - def UpdatePreprocessor(self, line): - """Update preprocessor stack. - - We need to handle preprocessors due to classes like this: - #ifdef SWIG - struct ResultDetailsPageElementExtensionPoint { - #else - struct ResultDetailsPageElementExtensionPoint : public Extension { - #endif - - We make the following assumptions (good enough for most files): - - Preprocessor condition evaluates to true from #if up to first - #else/#elif/#endif. - - - Preprocessor condition evaluates to false from #else/#elif up - to #endif. We still perform lint checks on these lines, but - these do not affect nesting stack. - - Args: - line: current line to check. - """ - if Match(r'^\s*#\s*(if|ifdef|ifndef)\b', line): - # Beginning of #if block, save the nesting stack here. The saved - # stack will allow us to restore the parsing state in the #else case. - self.pp_stack.append(_PreprocessorInfo(copy.deepcopy(self.stack))) - elif Match(r'^\s*#\s*(else|elif)\b', line): - # Beginning of #else block - if self.pp_stack: - if not self.pp_stack[-1].seen_else: - # This is the first #else or #elif block. Remember the - # whole nesting stack up to this point. This is what we - # keep after the #endif. - self.pp_stack[-1].seen_else = True - self.pp_stack[-1].stack_before_else = copy.deepcopy(self.stack) - - # Restore the stack to how it was before the #if - self.stack = copy.deepcopy(self.pp_stack[-1].stack_before_if) - else: - # TODO(unknown): unexpected #else, issue warning? - pass - elif Match(r'^\s*#\s*endif\b', line): - # End of #if or #else blocks. - if self.pp_stack: - # If we saw an #else, we will need to restore the nesting - # stack to its former state before the #else, otherwise we - # will just continue from where we left off. - if self.pp_stack[-1].seen_else: - # Here we can just use a shallow copy since we are the last - # reference to it. - self.stack = self.pp_stack[-1].stack_before_else - # Drop the corresponding #if - self.pp_stack.pop() - else: - # TODO(unknown): unexpected #endif, issue warning? - pass - - def Update(self, filename, clean_lines, linenum, error): - """Update nesting state with current line. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - line = clean_lines.elided[linenum] - - # Update pp_stack first - self.UpdatePreprocessor(line) - - # Count parentheses. This is to avoid adding struct arguments to - # the nesting stack. - if self.stack: - inner_block = self.stack[-1] - depth_change = line.count('(') - line.count(')') - inner_block.open_parentheses += depth_change - - # Also check if we are starting or ending an inline assembly block. - if inner_block.inline_asm in (_NO_ASM, _END_ASM): - if (depth_change != 0 and - inner_block.open_parentheses == 1 and - _MATCH_ASM.match(line)): - # Enter assembly block - inner_block.inline_asm = _INSIDE_ASM - else: - # Not entering assembly block. If previous line was _END_ASM, - # we will now shift to _NO_ASM state. - inner_block.inline_asm = _NO_ASM - elif (inner_block.inline_asm == _INSIDE_ASM and - inner_block.open_parentheses == 0): - # Exit assembly block - inner_block.inline_asm = _END_ASM - - # Consume namespace declaration at the beginning of the line. Do - # this in a loop so that we catch same line declarations like this: - # namespace proto2 { namespace bridge { class MessageSet; } } - while True: - # Match start of namespace. The "\b\s*" below catches namespace - # declarations even if it weren't followed by a whitespace, this - # is so that we don't confuse our namespace checker. The - # missing spaces will be flagged by CheckSpacing. - namespace_decl_match = Match(r'^\s*namespace\b\s*([:\w]+)?(.*)$', line) - if not namespace_decl_match: - break - - new_namespace = _NamespaceInfo(namespace_decl_match.group(1), linenum) - self.stack.append(new_namespace) - - line = namespace_decl_match.group(2) - if line.find('{') != -1: - new_namespace.seen_open_brace = True - line = line[line.find('{') + 1:] - - # Look for a class declaration in whatever is left of the line - # after parsing namespaces. The regexp accounts for decorated classes - # such as in: - # class LOCKABLE API Object { - # }; - # - # Templates with class arguments may confuse the parser, for example: - # template , - # class Vector = vector > - # class HeapQueue { - # - # Because this parser has no nesting state about templates, by the - # time it saw "class Comparator", it may think that it's a new class. - # Nested templates have a similar problem: - # template < - # typename ExportedType, - # typename TupleType, - # template class ImplTemplate> - # - # To avoid these cases, we ignore classes that are followed by '=' or '>' - class_decl_match = Match( - r'\s*(template\s*<[\w\s<>,:]*>\s*)?' - r'(class|struct)\s+([A-Z_]+\s+)*(\w+(?:::\w+)*)' - r'(([^=>]|<[^<>]*>|<[^<>]*<[^<>]*>\s*>)*)$', line) - if (class_decl_match and - (not self.stack or self.stack[-1].open_parentheses == 0)): - self.stack.append(_ClassInfo( - class_decl_match.group(4), class_decl_match.group(2), - clean_lines, linenum)) - line = class_decl_match.group(5) - - # If we have not yet seen the opening brace for the innermost block, - # run checks here. - if not self.SeenOpenBrace(): - self.stack[-1].CheckBegin(filename, clean_lines, linenum, error) - - # Update access control if we are inside a class/struct - if self.stack and isinstance(self.stack[-1], _ClassInfo): - classinfo = self.stack[-1] - access_match = Match( - r'^(.*)\b(public|private|protected|signals)(\s+(?:slots\s*)?)?' - r':(?:[^:]|$)', - line) - if access_match: - classinfo.access = access_match.group(2) - - # Check that access keywords are indented +1 space. Skip this - # check if the keywords are not preceded by whitespaces. - indent = access_match.group(1) - if (len(indent) != classinfo.class_indent + 1 and - Match(r'^\s*$', indent)): - if classinfo.is_struct: - parent = 'struct ' + classinfo.name - else: - parent = 'class ' + classinfo.name - slots = '' - if access_match.group(3): - slots = access_match.group(3) - error(filename, linenum, 'whitespace/indent', 3, - '%s%s: should be indented +1 space inside %s' % ( - access_match.group(2), slots, parent)) - - # Consume braces or semicolons from what's left of the line - while True: - # Match first brace, semicolon, or closed parenthesis. - matched = Match(r'^[^{;)}]*([{;)}])(.*)$', line) - if not matched: - break - - token = matched.group(1) - if token == '{': - # If namespace or class hasn't seen a opening brace yet, mark - # namespace/class head as complete. Push a new block onto the - # stack otherwise. - if not self.SeenOpenBrace(): - self.stack[-1].seen_open_brace = True - else: - self.stack.append(_BlockInfo(True)) - if _MATCH_ASM.match(line): - self.stack[-1].inline_asm = _BLOCK_ASM - elif token == ';' or token == ')': - # If we haven't seen an opening brace yet, but we already saw - # a semicolon, this is probably a forward declaration. Pop - # the stack for these. - # - # Similarly, if we haven't seen an opening brace yet, but we - # already saw a closing parenthesis, then these are probably - # function arguments with extra "class" or "struct" keywords. - # Also pop these stack for these. - if not self.SeenOpenBrace(): - self.stack.pop() - else: # token == '}' - # Perform end of block checks and pop the stack. - if self.stack: - self.stack[-1].CheckEnd(filename, clean_lines, linenum, error) - self.stack.pop() - line = matched.group(2) - - def InnermostClass(self): - """Get class info on the top of the stack. - - Returns: - A _ClassInfo object if we are inside a class, or None otherwise. - """ - for i in range(len(self.stack), 0, -1): - classinfo = self.stack[i - 1] - if isinstance(classinfo, _ClassInfo): - return classinfo - return None - - def CheckCompletedBlocks(self, filename, error): - """Checks that all classes and namespaces have been completely parsed. - - Call this when all lines in a file have been processed. - Args: - filename: The name of the current file. - error: The function to call with any errors found. - """ - # Note: This test can result in false positives if #ifdef constructs - # get in the way of brace matching. See the testBuildClass test in - # cpplint_unittest.py for an example of this. - for obj in self.stack: - if isinstance(obj, _ClassInfo): - error(filename, obj.starting_linenum, 'build/class', 5, - 'Failed to find complete declaration of class %s' % - obj.name) - elif isinstance(obj, _NamespaceInfo): - error(filename, obj.starting_linenum, 'build/namespaces', 5, - 'Failed to find complete declaration of namespace %s' % - obj.name) - - -def CheckForNonStandardConstructs(filename, clean_lines, linenum, - nesting_state, error): - r"""Logs an error if we see certain non-ANSI constructs ignored by gcc-2. - - Complain about several constructs which gcc-2 accepts, but which are - not standard C++. Warning about these in lint is one way to ease the - transition to new compilers. - - put storage class first (e.g. "static const" instead of "const static"). - - "%lld" instead of %qd" in printf-type functions. - - "%1$d" is non-standard in printf-type functions. - - "\%" is an undefined character escape sequence. - - text after #endif is not allowed. - - invalid inner-style forward declaration. - - >? and ?= and )\?=?\s*(\w+|[+-]?\d+)(\.\d*)?', - line): - error(filename, linenum, 'build/deprecated', 3, - '>? and ))?' - # r'\s*const\s*' + type_name + '\s*&\s*\w+\s*;' - error(filename, linenum, 'runtime/member_string_references', 2, - 'const string& members are dangerous. It is much better to use ' - 'alternatives, such as pointers or simple constants.') - - # Everything else in this function operates on class declarations. - # Return early if the top of the nesting stack is not a class, or if - # the class head is not completed yet. - classinfo = nesting_state.InnermostClass() - if not classinfo or not classinfo.seen_open_brace: - return - - # The class may have been declared with namespace or classname qualifiers. - # The constructor and destructor will not have those qualifiers. - base_classname = classinfo.name.split('::')[-1] - - # Look for single-argument constructors that aren't marked explicit. - # Technically a valid construct, but against style. - args = Match(r'\s+(?:inline\s+)?%s\s*\(([^,()]+)\)' - % re.escape(base_classname), - line) - if (args and - args.group(1) != 'void' and - not Match(r'(const\s+)?%s(\s+const)?\s*(?:<\w+>\s*)?&' - % re.escape(base_classname), args.group(1).strip())): - error(filename, linenum, 'runtime/explicit', 5, - 'Single-argument constructors should be marked explicit.') - - -def CheckSpacingForFunctionCall(filename, line, linenum, error): - """Checks for the correctness of various spacing around function calls. - - Args: - filename: The name of the current file. - line: The text of the line to check. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - - # Since function calls often occur inside if/for/while/switch - # expressions - which have their own, more liberal conventions - we - # first see if we should be looking inside such an expression for a - # function call, to which we can apply more strict standards. - fncall = line # if there's no control flow construct, look at whole line - for pattern in (r'\bif\s*\((.*)\)\s*{', - r'\bfor\s*\((.*)\)\s*{', - r'\bwhile\s*\((.*)\)\s*[{;]', - r'\bswitch\s*\((.*)\)\s*{'): - match = Search(pattern, line) - if match: - fncall = match.group(1) # look inside the parens for function calls - break - - # Except in if/for/while/switch, there should never be space - # immediately inside parens (eg "f( 3, 4 )"). We make an exception - # for nested parens ( (a+b) + c ). Likewise, there should never be - # a space before a ( when it's a function argument. I assume it's a - # function argument when the char before the whitespace is legal in - # a function name (alnum + _) and we're not starting a macro. Also ignore - # pointers and references to arrays and functions coz they're too tricky: - # we use a very simple way to recognize these: - # " (something)(maybe-something)" or - # " (something)(maybe-something," or - # " (something)[something]" - # Note that we assume the contents of [] to be short enough that - # they'll never need to wrap. - if ( # Ignore control structures. - not Search(r'\b(if|for|while|switch|return|new|delete|catch|sizeof)\b', - fncall) and - # Ignore pointers/references to functions. - not Search(r' \([^)]+\)\([^)]*(\)|,$)', fncall) and - # Ignore pointers/references to arrays. - not Search(r' \([^)]+\)\[[^\]]+\]', fncall)): - if Search(r'\w\s*\(\s(?!\s*\\$)', fncall): # a ( used for a fn call - error(filename, linenum, 'whitespace/parens', 4, - 'Extra space after ( in function call') - elif Search(r'\(\s+(?!(\s*\\)|\()', fncall): - error(filename, linenum, 'whitespace/parens', 2, - 'Extra space after (') - if (Search(r'\w\s+\(', fncall) and - not Search(r'#\s*define|typedef', fncall) and - not Search(r'\w\s+\((\w+::)*\*\w+\)\(', fncall)): - error(filename, linenum, 'whitespace/parens', 4, - 'Extra space before ( in function call') - # If the ) is followed only by a newline or a { + newline, assume it's - # part of a control statement (if/while/etc), and don't complain - if Search(r'[^)]\s+\)\s*[^{\s]', fncall): - # If the closing parenthesis is preceded by only whitespaces, - # try to give a more descriptive error message. - if Search(r'^\s+\)', fncall): - error(filename, linenum, 'whitespace/parens', 2, - 'Closing ) should be moved to the previous line') - else: - error(filename, linenum, 'whitespace/parens', 2, - 'Extra space before )') - - -def IsBlankLine(line): - """Returns true if the given line is blank. - - We consider a line to be blank if the line is empty or consists of - only white spaces. - - Args: - line: A line of a string. - - Returns: - True, if the given line is blank. - """ - return not line or line.isspace() - - -def CheckForFunctionLengths(filename, clean_lines, linenum, - function_state, error): - """Reports for long function bodies. - - For an overview why this is done, see: - http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml#Write_Short_Functions - - Uses a simplistic algorithm assuming other style guidelines - (especially spacing) are followed. - Only checks unindented functions, so class members are unchecked. - Trivial bodies are unchecked, so constructors with huge initializer lists - may be missed. - Blank/comment lines are not counted so as to avoid encouraging the removal - of vertical space and comments just to get through a lint check. - NOLINT *on the last line of a function* disables this check. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - function_state: Current function name and lines in body so far. - error: The function to call with any errors found. - """ - lines = clean_lines.lines - line = lines[linenum] - raw = clean_lines.raw_lines - raw_line = raw[linenum] - joined_line = '' - - starting_func = False - regexp = r'(\w(\w|::|\*|\&|\s)*)\(' # decls * & space::name( ... - match_result = Match(regexp, line) - if match_result: - # If the name is all caps and underscores, figure it's a macro and - # ignore it, unless it's TEST or TEST_F. - function_name = match_result.group(1).split()[-1] - if function_name == 'TEST' or function_name == 'TEST_F' or ( - not Match(r'[A-Z_]+$', function_name)): - starting_func = True - - if starting_func: - body_found = False - for start_linenum in xrange(linenum, clean_lines.NumLines()): - start_line = lines[start_linenum] - joined_line += ' ' + start_line.lstrip() - if Search(r'(;|})', start_line): # Declarations and trivial functions - body_found = True - break # ... ignore - elif Search(r'{', start_line): - body_found = True - function = Search(r'((\w|:)*)\(', line).group(1) - if Match(r'TEST', function): # Handle TEST... macros - parameter_regexp = Search(r'(\(.*\))', joined_line) - if parameter_regexp: # Ignore bad syntax - function += parameter_regexp.group(1) - else: - function += '()' - function_state.Begin(function) - break - if not body_found: - # No body for the function (or evidence of a non-function) was found. - error(filename, linenum, 'readability/fn_size', 5, - 'Lint failed to find start of function body.') - elif Match(r'^\}\s*$', line): # function end - function_state.Check(error, filename, linenum) - function_state.End() - elif not Match(r'^\s*$', line): - function_state.Count() # Count non-blank/non-comment lines. - - -_RE_PATTERN_TODO = re.compile(r'^//(\s*)TODO(\(.+?\))?:?(\s|$)?') - - -def CheckComment(comment, filename, linenum, error): - """Checks for common mistakes in TODO comments. - - Args: - comment: The text of the comment from the line in question. - filename: The name of the current file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - match = _RE_PATTERN_TODO.match(comment) - if match: - # One whitespace is correct; zero whitespace is handled elsewhere. - leading_whitespace = match.group(1) - if len(leading_whitespace) > 1: - error(filename, linenum, 'whitespace/todo', 2, - 'Too many spaces before TODO') - - username = match.group(2) - if not username: - error(filename, linenum, 'readability/todo', 2, - 'Missing username in TODO; it should look like ' - '"// TODO(my_username): Stuff."') - - middle_whitespace = match.group(3) - # Comparisons made explicit for correctness -- pylint: disable=g-explicit-bool-comparison - if middle_whitespace != ' ' and middle_whitespace != '': - error(filename, linenum, 'whitespace/todo', 2, - 'TODO(my_username) should be followed by a space') - -def CheckAccess(filename, clean_lines, linenum, nesting_state, error): - """Checks for improper use of DISALLOW* macros. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - nesting_state: A _NestingState instance which maintains information about - the current stack of nested blocks being parsed. - error: The function to call with any errors found. - """ - line = clean_lines.elided[linenum] # get rid of comments and strings - - matched = Match((r'\s*(DISALLOW_COPY_AND_ASSIGN|' - r'DISALLOW_EVIL_CONSTRUCTORS|' - r'DISALLOW_IMPLICIT_CONSTRUCTORS)'), line) - if not matched: - return - if nesting_state.stack and isinstance(nesting_state.stack[-1], _ClassInfo): - if nesting_state.stack[-1].access != 'private': - error(filename, linenum, 'readability/constructors', 3, - '%s must be in the private: section' % matched.group(1)) - - else: - # Found DISALLOW* macro outside a class declaration, or perhaps it - # was used inside a function when it should have been part of the - # class declaration. We could issue a warning here, but it - # probably resulted in a compiler error already. - pass - - -def FindNextMatchingAngleBracket(clean_lines, linenum, init_suffix): - """Find the corresponding > to close a template. - - Args: - clean_lines: A CleansedLines instance containing the file. - linenum: Current line number. - init_suffix: Remainder of the current line after the initial <. - - Returns: - True if a matching bracket exists. - """ - line = init_suffix - nesting_stack = ['<'] - while True: - # Find the next operator that can tell us whether < is used as an - # opening bracket or as a less-than operator. We only want to - # warn on the latter case. - # - # We could also check all other operators and terminate the search - # early, e.g. if we got something like this "a(),;\[\]]*([<>(),;\[\]])(.*)$', line) - if match: - # Found an operator, update nesting stack - operator = match.group(1) - line = match.group(2) - - if nesting_stack[-1] == '<': - # Expecting closing angle bracket - if operator in ('<', '(', '['): - nesting_stack.append(operator) - elif operator == '>': - nesting_stack.pop() - if not nesting_stack: - # Found matching angle bracket - return True - elif operator == ',': - # Got a comma after a bracket, this is most likely a template - # argument. We have not seen a closing angle bracket yet, but - # it's probably a few lines later if we look for it, so just - # return early here. - return True - else: - # Got some other operator. - return False - - else: - # Expecting closing parenthesis or closing bracket - if operator in ('<', '(', '['): - nesting_stack.append(operator) - elif operator in (')', ']'): - # We don't bother checking for matching () or []. If we got - # something like (] or [), it would have been a syntax error. - nesting_stack.pop() - - else: - # Scan the next line - linenum += 1 - if linenum >= len(clean_lines.elided): - break - line = clean_lines.elided[linenum] - - # Exhausted all remaining lines and still no matching angle bracket. - # Most likely the input was incomplete, otherwise we should have - # seen a semicolon and returned early. - return True - - -def FindPreviousMatchingAngleBracket(clean_lines, linenum, init_prefix): - """Find the corresponding < that started a template. - - Args: - clean_lines: A CleansedLines instance containing the file. - linenum: Current line number. - init_prefix: Part of the current line before the initial >. - - Returns: - True if a matching bracket exists. - """ - line = init_prefix - nesting_stack = ['>'] - while True: - # Find the previous operator - match = Search(r'^(.*)([<>(),;\[\]])[^<>(),;\[\]]*$', line) - if match: - # Found an operator, update nesting stack - operator = match.group(2) - line = match.group(1) - - if nesting_stack[-1] == '>': - # Expecting opening angle bracket - if operator in ('>', ')', ']'): - nesting_stack.append(operator) - elif operator == '<': - nesting_stack.pop() - if not nesting_stack: - # Found matching angle bracket - return True - elif operator == ',': - # Got a comma before a bracket, this is most likely a - # template argument. The opening angle bracket is probably - # there if we look for it, so just return early here. - return True - else: - # Got some other operator. - return False - - else: - # Expecting opening parenthesis or opening bracket - if operator in ('>', ')', ']'): - nesting_stack.append(operator) - elif operator in ('(', '['): - nesting_stack.pop() - - else: - # Scan the previous line - linenum -= 1 - if linenum < 0: - break - line = clean_lines.elided[linenum] - - # Exhausted all earlier lines and still no matching angle bracket. - return False - - -def CheckSpacing(filename, clean_lines, linenum, nesting_state, error): - """Checks for the correctness of various spacing issues in the code. - - Things we check for: spaces around operators, spaces after - if/for/while/switch, no spaces around parens in function calls, two - spaces between code and comment, don't start a block with a blank - line, don't end a function with a blank line, don't add a blank line - after public/protected/private, don't have too many blank lines in a row. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - nesting_state: A _NestingState instance which maintains information about - the current stack of nested blocks being parsed. - error: The function to call with any errors found. - """ - - # Don't use "elided" lines here, otherwise we can't check commented lines. - # Don't want to use "raw" either, because we don't want to check inside C++11 - # raw strings, - raw = clean_lines.lines_without_raw_strings - line = raw[linenum] - - # Before nixing comments, check if the line is blank for no good - # reason. This includes the first line after a block is opened, and - # blank lines at the end of a function (ie, right before a line like '}' - # - # Skip all the blank line checks if we are immediately inside a - # namespace body. In other words, don't issue blank line warnings - # for this block: - # namespace { - # - # } - # - # A warning about missing end of namespace comments will be issued instead. - if IsBlankLine(line) and not nesting_state.InNamespaceBody(): - elided = clean_lines.elided - prev_line = elided[linenum - 1] - prevbrace = prev_line.rfind('{') - # TODO(unknown): Don't complain if line before blank line, and line after, - # both start with alnums and are indented the same amount. - # This ignores whitespace at the start of a namespace block - # because those are not usually indented. - if prevbrace != -1 and prev_line[prevbrace:].find('}') == -1: - # OK, we have a blank line at the start of a code block. Before we - # complain, we check if it is an exception to the rule: The previous - # non-empty line has the parameters of a function header that are indented - # 4 spaces (because they did not fit in a 80 column line when placed on - # the same line as the function name). We also check for the case where - # the previous line is indented 6 spaces, which may happen when the - # initializers of a constructor do not fit into a 80 column line. - exception = False - if Match(r' {6}\w', prev_line): # Initializer list? - # We are looking for the opening column of initializer list, which - # should be indented 4 spaces to cause 6 space indentation afterwards. - search_position = linenum-2 - while (search_position >= 0 - and Match(r' {6}\w', elided[search_position])): - search_position -= 1 - exception = (search_position >= 0 - and elided[search_position][:5] == ' :') - else: - # Search for the function arguments or an initializer list. We use a - # simple heuristic here: If the line is indented 4 spaces; and we have a - # closing paren, without the opening paren, followed by an opening brace - # or colon (for initializer lists) we assume that it is the last line of - # a function header. If we have a colon indented 4 spaces, it is an - # initializer list. - exception = (Match(r' {4}\w[^\(]*\)\s*(const\s*)?(\{\s*$|:)', - prev_line) - or Match(r' {4}:', prev_line)) - - if not exception: - error(filename, linenum, 'whitespace/blank_line', 2, - 'Redundant blank line at the start of a code block ' - 'should be deleted.') - # Ignore blank lines at the end of a block in a long if-else - # chain, like this: - # if (condition1) { - # // Something followed by a blank line - # - # } else if (condition2) { - # // Something else - # } - if linenum + 1 < clean_lines.NumLines(): - next_line = raw[linenum + 1] - if (next_line - and Match(r'\s*}', next_line) - and next_line.find('} else ') == -1): - error(filename, linenum, 'whitespace/blank_line', 3, - 'Redundant blank line at the end of a code block ' - 'should be deleted.') - - matched = Match(r'\s*(public|protected|private):', prev_line) - if matched: - error(filename, linenum, 'whitespace/blank_line', 3, - 'Do not leave a blank line after "%s:"' % matched.group(1)) - - # Next, we complain if there's a comment too near the text - commentpos = line.find('//') - if commentpos != -1: - # Check if the // may be in quotes. If so, ignore it - # Comparisons made explicit for clarity -- pylint: disable=g-explicit-bool-comparison - if (line.count('"', 0, commentpos) - - line.count('\\"', 0, commentpos)) % 2 == 0: # not in quotes - # Allow one space for new scopes, two spaces otherwise: - if (not Match(r'^\s*{ //', line) and - ((commentpos >= 1 and - line[commentpos-1] not in string.whitespace) or - (commentpos >= 2 and - line[commentpos-2] not in string.whitespace))): - error(filename, linenum, 'whitespace/comments', 2, - 'At least two spaces is best between code and comments') - # There should always be a space between the // and the comment - commentend = commentpos + 2 - if commentend < len(line) and not line[commentend] == ' ': - # but some lines are exceptions -- e.g. if they're big - # comment delimiters like: - # //---------------------------------------------------------- - # or are an empty C++ style Doxygen comment, like: - # /// - # or C++ style Doxygen comments placed after the variable: - # ///< Header comment - # //!< Header comment - # or they begin with multiple slashes followed by a space: - # //////// Header comment - match = (Search(r'[=/-]{4,}\s*$', line[commentend:]) or - Search(r'^/$', line[commentend:]) or - Search(r'^!< ', line[commentend:]) or - Search(r'^/< ', line[commentend:]) or - Search(r'^/+ ', line[commentend:])) - if not match: - error(filename, linenum, 'whitespace/comments', 4, - 'Should have a space between // and comment') - CheckComment(line[commentpos:], filename, linenum, error) - - line = clean_lines.elided[linenum] # get rid of comments and strings - - # Don't try to do spacing checks for operator methods - line = re.sub(r'operator(==|!=|<|<<|<=|>=|>>|>)\(', 'operator\(', line) - - # We allow no-spaces around = within an if: "if ( (a=Foo()) == 0 )". - # Otherwise not. Note we only check for non-spaces on *both* sides; - # sometimes people put non-spaces on one side when aligning ='s among - # many lines (not that this is behavior that I approve of...) - if Search(r'[\w.]=[\w.]', line) and not Search(r'\b(if|while) ', line): - error(filename, linenum, 'whitespace/operators', 4, - 'Missing spaces around =') - - # It's ok not to have spaces around binary operators like + - * /, but if - # there's too little whitespace, we get concerned. It's hard to tell, - # though, so we punt on this one for now. TODO. - - # You should always have whitespace around binary operators. - # - # Check <= and >= first to avoid false positives with < and >, then - # check non-include lines for spacing around < and >. - match = Search(r'[^<>=!\s](==|!=|<=|>=)[^<>=!\s]', line) - if match: - error(filename, linenum, 'whitespace/operators', 3, - 'Missing spaces around %s' % match.group(1)) - # We allow no-spaces around << when used like this: 10<<20, but - # not otherwise (particularly, not when used as streams) - # Also ignore using ns::operator<<; - match = Search(r'(operator|\S)(?:L|UL|ULL|l|ul|ull)?<<(\S)', line) - if (match and - not (match.group(1).isdigit() and match.group(2).isdigit()) and - not (match.group(1) == 'operator' and match.group(2) == ';')): - error(filename, linenum, 'whitespace/operators', 3, - 'Missing spaces around <<') - elif not Match(r'#.*include', line): - # Avoid false positives on -> - reduced_line = line.replace('->', '') - - # Look for < that is not surrounded by spaces. This is only - # triggered if both sides are missing spaces, even though - # technically should should flag if at least one side is missing a - # space. This is done to avoid some false positives with shifts. - match = Search(r'[^\s<]<([^\s=<].*)', reduced_line) - if (match and - not FindNextMatchingAngleBracket(clean_lines, linenum, match.group(1))): - error(filename, linenum, 'whitespace/operators', 3, - 'Missing spaces around <') - - # Look for > that is not surrounded by spaces. Similar to the - # above, we only trigger if both sides are missing spaces to avoid - # false positives with shifts. - match = Search(r'^(.*[^\s>])>[^\s=>]', reduced_line) - if (match and - not FindPreviousMatchingAngleBracket(clean_lines, linenum, - match.group(1))): - error(filename, linenum, 'whitespace/operators', 3, - 'Missing spaces around >') - - # We allow no-spaces around >> for almost anything. This is because - # C++11 allows ">>" to close nested templates, which accounts for - # most cases when ">>" is not followed by a space. - # - # We still warn on ">>" followed by alpha character, because that is - # likely due to ">>" being used for right shifts, e.g.: - # value >> alpha - # - # When ">>" is used to close templates, the alphanumeric letter that - # follows would be part of an identifier, and there should still be - # a space separating the template type and the identifier. - # type> alpha - match = Search(r'>>[a-zA-Z_]', line) - if match: - error(filename, linenum, 'whitespace/operators', 3, - 'Missing spaces around >>') - - # There shouldn't be space around unary operators - match = Search(r'(!\s|~\s|[\s]--[\s;]|[\s]\+\+[\s;])', line) - if match: - error(filename, linenum, 'whitespace/operators', 4, - 'Extra space for operator %s' % match.group(1)) - - # A pet peeve of mine: no spaces after an if, while, switch, or for - match = Search(r' (if\(|for\(|while\(|switch\()', line) - if match: - error(filename, linenum, 'whitespace/parens', 5, - 'Missing space before ( in %s' % match.group(1)) - - # For if/for/while/switch, the left and right parens should be - # consistent about how many spaces are inside the parens, and - # there should either be zero or one spaces inside the parens. - # We don't want: "if ( foo)" or "if ( foo )". - # Exception: "for ( ; foo; bar)" and "for (foo; bar; )" are allowed. - match = Search(r'\b(if|for|while|switch)\s*' - r'\(([ ]*)(.).*[^ ]+([ ]*)\)\s*{\s*$', - line) - if match: - if len(match.group(2)) != len(match.group(4)): - if not (match.group(3) == ';' and - len(match.group(2)) == 1 + len(match.group(4)) or - not match.group(2) and Search(r'\bfor\s*\(.*; \)', line)): - error(filename, linenum, 'whitespace/parens', 5, - 'Mismatching spaces inside () in %s' % match.group(1)) - if len(match.group(2)) not in [0, 1]: - error(filename, linenum, 'whitespace/parens', 5, - 'Should have zero or one spaces inside ( and ) in %s' % - match.group(1)) - - # You should always have a space after a comma (either as fn arg or operator) - # - # This does not apply when the non-space character following the - # comma is another comma, since the only time when that happens is - # for empty macro arguments. - # - # We run this check in two passes: first pass on elided lines to - # verify that lines contain missing whitespaces, second pass on raw - # lines to confirm that those missing whitespaces are not due to - # elided comments. - if Search(r',[^,\s]', line) and Search(r',[^,\s]', raw[linenum]): - error(filename, linenum, 'whitespace/comma', 3, - 'Missing space after ,') - - # You should always have a space after a semicolon - # except for few corner cases - # TODO(unknown): clarify if 'if (1) { return 1;}' is requires one more - # space after ; - if Search(r';[^\s};\\)/]', line): - error(filename, linenum, 'whitespace/semicolon', 3, - 'Missing space after ;') - - # Next we will look for issues with function calls. - CheckSpacingForFunctionCall(filename, line, linenum, error) - - # Except after an opening paren, or after another opening brace (in case of - # an initializer list, for instance), you should have spaces before your - # braces. And since you should never have braces at the beginning of a line, - # this is an easy test. - match = Match(r'^(.*[^ ({]){', line) - if match: - # Try a bit harder to check for brace initialization. This - # happens in one of the following forms: - # Constructor() : initializer_list_{} { ... } - # Constructor{}.MemberFunction() - # Type variable{}; - # FunctionCall(type{}, ...); - # LastArgument(..., type{}); - # LOG(INFO) << type{} << " ..."; - # map_of_type[{...}] = ...; - # - # We check for the character following the closing brace, and - # silence the warning if it's one of those listed above, i.e. - # "{.;,)<]". - # - # To account for nested initializer list, we allow any number of - # closing braces up to "{;,)<". We can't simply silence the - # warning on first sight of closing brace, because that would - # cause false negatives for things that are not initializer lists. - # Silence this: But not this: - # Outer{ if (...) { - # Inner{...} if (...){ // Missing space before { - # }; } - # - # There is a false negative with this approach if people inserted - # spurious semicolons, e.g. "if (cond){};", but we will catch the - # spurious semicolon with a separate check. - (endline, endlinenum, endpos) = CloseExpression( - clean_lines, linenum, len(match.group(1))) - trailing_text = '' - if endpos > -1: - trailing_text = endline[endpos:] - for offset in xrange(endlinenum + 1, - min(endlinenum + 3, clean_lines.NumLines() - 1)): - trailing_text += clean_lines.elided[offset] - if not Match(r'^[\s}]*[{.;,)<\]]', trailing_text): - error(filename, linenum, 'whitespace/braces', 5, - 'Missing space before {') - - # Make sure '} else {' has spaces. - if Search(r'}else', line): - error(filename, linenum, 'whitespace/braces', 5, - 'Missing space before else') - - # You shouldn't have spaces before your brackets, except maybe after - # 'delete []' or 'new char * []'. - if Search(r'\w\s+\[', line) and not Search(r'delete\s+\[', line): - error(filename, linenum, 'whitespace/braces', 5, - 'Extra space before [') - - # You shouldn't have a space before a semicolon at the end of the line. - # There's a special case for "for" since the style guide allows space before - # the semicolon there. - if Search(r':\s*;\s*$', line): - error(filename, linenum, 'whitespace/semicolon', 5, - 'Semicolon defining empty statement. Use {} instead.') - elif Search(r'^\s*;\s*$', line): - error(filename, linenum, 'whitespace/semicolon', 5, - 'Line contains only semicolon. If this should be an empty statement, ' - 'use {} instead.') - elif (Search(r'\s+;\s*$', line) and - not Search(r'\bfor\b', line)): - error(filename, linenum, 'whitespace/semicolon', 5, - 'Extra space before last semicolon. If this should be an empty ' - 'statement, use {} instead.') - - # In range-based for, we wanted spaces before and after the colon, but - # not around "::" tokens that might appear. - if (Search('for *\(.*[^:]:[^: ]', line) or - Search('for *\(.*[^: ]:[^:]', line)): - error(filename, linenum, 'whitespace/forcolon', 2, - 'Missing space around colon in range-based for loop') - - -def CheckSectionSpacing(filename, clean_lines, class_info, linenum, error): - """Checks for additional blank line issues related to sections. - - Currently the only thing checked here is blank line before protected/private. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - class_info: A _ClassInfo objects. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - # Skip checks if the class is small, where small means 25 lines or less. - # 25 lines seems like a good cutoff since that's the usual height of - # terminals, and any class that can't fit in one screen can't really - # be considered "small". - # - # Also skip checks if we are on the first line. This accounts for - # classes that look like - # class Foo { public: ... }; - # - # If we didn't find the end of the class, last_line would be zero, - # and the check will be skipped by the first condition. - if (class_info.last_line - class_info.starting_linenum <= 24 or - linenum <= class_info.starting_linenum): - return - - matched = Match(r'\s*(public|protected|private):', clean_lines.lines[linenum]) - if matched: - # Issue warning if the line before public/protected/private was - # not a blank line, but don't do this if the previous line contains - # "class" or "struct". This can happen two ways: - # - We are at the beginning of the class. - # - We are forward-declaring an inner class that is semantically - # private, but needed to be public for implementation reasons. - # Also ignores cases where the previous line ends with a backslash as can be - # common when defining classes in C macros. - prev_line = clean_lines.lines[linenum - 1] - if (not IsBlankLine(prev_line) and - not Search(r'\b(class|struct)\b', prev_line) and - not Search(r'\\$', prev_line)): - # Try a bit harder to find the beginning of the class. This is to - # account for multi-line base-specifier lists, e.g.: - # class Derived - # : public Base { - end_class_head = class_info.starting_linenum - for i in range(class_info.starting_linenum, linenum): - if Search(r'\{\s*$', clean_lines.lines[i]): - end_class_head = i - break - if end_class_head < linenum - 1: - error(filename, linenum, 'whitespace/blank_line', 3, - '"%s:" should be preceded by a blank line' % matched.group(1)) - - -def GetPreviousNonBlankLine(clean_lines, linenum): - """Return the most recent non-blank line and its line number. - - Args: - clean_lines: A CleansedLines instance containing the file contents. - linenum: The number of the line to check. - - Returns: - A tuple with two elements. The first element is the contents of the last - non-blank line before the current line, or the empty string if this is the - first non-blank line. The second is the line number of that line, or -1 - if this is the first non-blank line. - """ - - prevlinenum = linenum - 1 - while prevlinenum >= 0: - prevline = clean_lines.elided[prevlinenum] - if not IsBlankLine(prevline): # if not a blank line... - return (prevline, prevlinenum) - prevlinenum -= 1 - return ('', -1) - - -def CheckBraces(filename, clean_lines, linenum, error): - """Looks for misplaced braces (e.g. at the end of line). - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - - line = clean_lines.elided[linenum] # get rid of comments and strings - - if Match(r'\s*{\s*$', line): - # We allow an open brace to start a line in the case where someone is using - # braces in a block to explicitly create a new scope, which is commonly used - # to control the lifetime of stack-allocated variables. Braces are also - # used for brace initializers inside function calls. We don't detect this - # perfectly: we just don't complain if the last non-whitespace character on - # the previous non-blank line is ',', ';', ':', '(', '{', or '}', or if the - # previous line starts a preprocessor block. - prevline = GetPreviousNonBlankLine(clean_lines, linenum)[0] - if (not Search(r'[,;:}{(]\s*$', prevline) and - not Match(r'\s*#', prevline)): - error(filename, linenum, 'whitespace/braces', 4, - '{ should almost always be at the end of the previous line') - - # An else clause should be on the same line as the preceding closing brace. - if Match(r'\s*else\s*', line): - prevline = GetPreviousNonBlankLine(clean_lines, linenum)[0] - if Match(r'\s*}\s*$', prevline): - error(filename, linenum, 'whitespace/newline', 4, - 'An else should appear on the same line as the preceding }') - - # If braces come on one side of an else, they should be on both. - # However, we have to worry about "else if" that spans multiple lines! - if Search(r'}\s*else[^{]*$', line) or Match(r'[^}]*else\s*{', line): - if Search(r'}\s*else if([^{]*)$', line): # could be multi-line if - # find the ( after the if - pos = line.find('else if') - pos = line.find('(', pos) - if pos > 0: - (endline, _, endpos) = CloseExpression(clean_lines, linenum, pos) - if endline[endpos:].find('{') == -1: # must be brace after if - error(filename, linenum, 'readability/braces', 5, - 'If an else has a brace on one side, it should have it on both') - else: # common case: else not followed by a multi-line if - error(filename, linenum, 'readability/braces', 5, - 'If an else has a brace on one side, it should have it on both') - - # Likewise, an else should never have the else clause on the same line - if Search(r'\belse [^\s{]', line) and not Search(r'\belse if\b', line): - error(filename, linenum, 'whitespace/newline', 4, - 'Else clause should never be on same line as else (use 2 lines)') - - # In the same way, a do/while should never be on one line - if Match(r'\s*do [^\s{]', line): - error(filename, linenum, 'whitespace/newline', 4, - 'do/while clauses should not be on a single line') - - # Block bodies should not be followed by a semicolon. Due to C++11 - # brace initialization, there are more places where semicolons are - # required than not, so we use a whitelist approach to check these - # rather than a blacklist. These are the places where "};" should - # be replaced by just "}": - # 1. Some flavor of block following closing parenthesis: - # for (;;) {}; - # while (...) {}; - # switch (...) {}; - # Function(...) {}; - # if (...) {}; - # if (...) else if (...) {}; - # - # 2. else block: - # if (...) else {}; - # - # 3. const member function: - # Function(...) const {}; - # - # 4. Block following some statement: - # x = 42; - # {}; - # - # 5. Block at the beginning of a function: - # Function(...) { - # {}; - # } - # - # Note that naively checking for the preceding "{" will also match - # braces inside multi-dimensional arrays, but this is fine since - # that expression will not contain semicolons. - # - # 6. Block following another block: - # while (true) {} - # {}; - # - # 7. End of namespaces: - # namespace {}; - # - # These semicolons seems far more common than other kinds of - # redundant semicolons, possibly due to people converting classes - # to namespaces. For now we do not warn for this case. - # - # Try matching case 1 first. - match = Match(r'^(.*\)\s*)\{', line) - if match: - # Matched closing parenthesis (case 1). Check the token before the - # matching opening parenthesis, and don't warn if it looks like a - # macro. This avoids these false positives: - # - macro that defines a base class - # - multi-line macro that defines a base class - # - macro that defines the whole class-head - # - # But we still issue warnings for macros that we know are safe to - # warn, specifically: - # - TEST, TEST_F, TEST_P, MATCHER, MATCHER_P - # - TYPED_TEST - # - INTERFACE_DEF - # - EXCLUSIVE_LOCKS_REQUIRED, SHARED_LOCKS_REQUIRED, LOCKS_EXCLUDED: - # - # We implement a whitelist of safe macros instead of a blacklist of - # unsafe macros, even though the latter appears less frequently in - # google code and would have been easier to implement. This is because - # the downside for getting the whitelist wrong means some extra - # semicolons, while the downside for getting the blacklist wrong - # would result in compile errors. - # - # In addition to macros, we also don't want to warn on compound - # literals. - closing_brace_pos = match.group(1).rfind(')') - opening_parenthesis = ReverseCloseExpression( - clean_lines, linenum, closing_brace_pos) - if opening_parenthesis[2] > -1: - line_prefix = opening_parenthesis[0][0:opening_parenthesis[2]] - macro = Search(r'\b([A-Z_]+)\s*$', line_prefix) - if ((macro and - macro.group(1) not in ( - 'TEST', 'TEST_F', 'MATCHER', 'MATCHER_P', 'TYPED_TEST', - 'EXCLUSIVE_LOCKS_REQUIRED', 'SHARED_LOCKS_REQUIRED', - 'LOCKS_EXCLUDED', 'INTERFACE_DEF')) or - Search(r'\s+=\s*$', line_prefix)): - match = None - # Whitelist lambda function definition which also requires a ";" after - # closing brace - if match: - if Match(r'^.*\[.*\]\s*(.*\)\s*)\{', line): - match = None - - else: - # Try matching cases 2-3. - match = Match(r'^(.*(?:else|\)\s*const)\s*)\{', line) - if not match: - # Try matching cases 4-6. These are always matched on separate lines. - # - # Note that we can't simply concatenate the previous line to the - # current line and do a single match, otherwise we may output - # duplicate warnings for the blank line case: - # if (cond) { - # // blank line - # } - prevline = GetPreviousNonBlankLine(clean_lines, linenum)[0] - if prevline and Search(r'[;{}]\s*$', prevline): - match = Match(r'^(\s*)\{', line) - - # Check matching closing brace - if match: - (endline, endlinenum, endpos) = CloseExpression( - clean_lines, linenum, len(match.group(1))) - if endpos > -1 and Match(r'^\s*;', endline[endpos:]): - # Current {} pair is eligible for semicolon check, and we have found - # the redundant semicolon, output warning here. - # - # Note: because we are scanning forward for opening braces, and - # outputting warnings for the matching closing brace, if there are - # nested blocks with trailing semicolons, we will get the error - # messages in reversed order. - error(filename, endlinenum, 'readability/braces', 4, - "You don't need a ; after a }") - - -def CheckEmptyBlockBody(filename, clean_lines, linenum, error): - """Look for empty loop/conditional body with only a single semicolon. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - - # Search for loop keywords at the beginning of the line. Because only - # whitespaces are allowed before the keywords, this will also ignore most - # do-while-loops, since those lines should start with closing brace. - # - # We also check "if" blocks here, since an empty conditional block - # is likely an error. - line = clean_lines.elided[linenum] - matched = Match(r'\s*(for|while|if)\s*\(', line) - if matched: - # Find the end of the conditional expression - (end_line, end_linenum, end_pos) = CloseExpression( - clean_lines, linenum, line.find('(')) - - # Output warning if what follows the condition expression is a semicolon. - # No warning for all other cases, including whitespace or newline, since we - # have a separate check for semicolons preceded by whitespace. - if end_pos >= 0 and Match(r';', end_line[end_pos:]): - if matched.group(1) == 'if': - error(filename, end_linenum, 'whitespace/empty_conditional_body', 5, - 'Empty conditional bodies should use {}') - else: - error(filename, end_linenum, 'whitespace/empty_loop_body', 5, - 'Empty loop bodies should use {} or continue') - - -def CheckCheck(filename, clean_lines, linenum, error): - """Checks the use of CHECK and EXPECT macros. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - - # Decide the set of replacement macros that should be suggested - lines = clean_lines.elided - check_macro = None - start_pos = -1 - for macro in _CHECK_MACROS: - i = lines[linenum].find(macro) - if i >= 0: - check_macro = macro - - # Find opening parenthesis. Do a regular expression match here - # to make sure that we are matching the expected CHECK macro, as - # opposed to some other macro that happens to contain the CHECK - # substring. - matched = Match(r'^(.*\b' + check_macro + r'\s*)\(', lines[linenum]) - if not matched: - continue - start_pos = len(matched.group(1)) - break - if not check_macro or start_pos < 0: - # Don't waste time here if line doesn't contain 'CHECK' or 'EXPECT' - return - - # Find end of the boolean expression by matching parentheses - (last_line, end_line, end_pos) = CloseExpression( - clean_lines, linenum, start_pos) - if end_pos < 0: - return - if linenum == end_line: - expression = lines[linenum][start_pos + 1:end_pos - 1] - else: - expression = lines[linenum][start_pos + 1:] - for i in xrange(linenum + 1, end_line): - expression += lines[i] - expression += last_line[0:end_pos - 1] - - # Parse expression so that we can take parentheses into account. - # This avoids false positives for inputs like "CHECK((a < 4) == b)", - # which is not replaceable by CHECK_LE. - lhs = '' - rhs = '' - operator = None - while expression: - matched = Match(r'^\s*(<<|<<=|>>|>>=|->\*|->|&&|\|\||' - r'==|!=|>=|>|<=|<|\()(.*)$', expression) - if matched: - token = matched.group(1) - if token == '(': - # Parenthesized operand - expression = matched.group(2) - (end, _) = FindEndOfExpressionInLine(expression, 0, 1, '(', ')') - if end < 0: - return # Unmatched parenthesis - lhs += '(' + expression[0:end] - expression = expression[end:] - elif token in ('&&', '||'): - # Logical and/or operators. This means the expression - # contains more than one term, for example: - # CHECK(42 < a && a < b); - # - # These are not replaceable with CHECK_LE, so bail out early. - return - elif token in ('<<', '<<=', '>>', '>>=', '->*', '->'): - # Non-relational operator - lhs += token - expression = matched.group(2) - else: - # Relational operator - operator = token - rhs = matched.group(2) - break - else: - # Unparenthesized operand. Instead of appending to lhs one character - # at a time, we do another regular expression match to consume several - # characters at once if possible. Trivial benchmark shows that this - # is more efficient when the operands are longer than a single - # character, which is generally the case. - matched = Match(r'^([^-=!<>()&|]+)(.*)$', expression) - if not matched: - matched = Match(r'^(\s*\S)(.*)$', expression) - if not matched: - break - lhs += matched.group(1) - expression = matched.group(2) - - # Only apply checks if we got all parts of the boolean expression - if not (lhs and operator and rhs): - return - - # Check that rhs do not contain logical operators. We already know - # that lhs is fine since the loop above parses out && and ||. - if rhs.find('&&') > -1 or rhs.find('||') > -1: - return - - # At least one of the operands must be a constant literal. This is - # to avoid suggesting replacements for unprintable things like - # CHECK(variable != iterator) - # - # The following pattern matches decimal, hex integers, strings, and - # characters (in that order). - lhs = lhs.strip() - rhs = rhs.strip() - match_constant = r'^([-+]?(\d+|0[xX][0-9a-fA-F]+)[lLuU]{0,3}|".*"|\'.*\')$' - if Match(match_constant, lhs) or Match(match_constant, rhs): - # Note: since we know both lhs and rhs, we can provide a more - # descriptive error message like: - # Consider using CHECK_EQ(x, 42) instead of CHECK(x == 42) - # Instead of: - # Consider using CHECK_EQ instead of CHECK(a == b) - # - # We are still keeping the less descriptive message because if lhs - # or rhs gets long, the error message might become unreadable. - error(filename, linenum, 'readability/check', 2, - 'Consider using %s instead of %s(a %s b)' % ( - _CHECK_REPLACEMENT[check_macro][operator], - check_macro, operator)) - - -def CheckAltTokens(filename, clean_lines, linenum, error): - """Check alternative keywords being used in boolean expressions. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - line = clean_lines.elided[linenum] - - # Avoid preprocessor lines - if Match(r'^\s*#', line): - return - - # Last ditch effort to avoid multi-line comments. This will not help - # if the comment started before the current line or ended after the - # current line, but it catches most of the false positives. At least, - # it provides a way to workaround this warning for people who use - # multi-line comments in preprocessor macros. - # - # TODO(unknown): remove this once cpplint has better support for - # multi-line comments. - if line.find('/*') >= 0 or line.find('*/') >= 0: - return - - for match in _ALT_TOKEN_REPLACEMENT_PATTERN.finditer(line): - error(filename, linenum, 'readability/alt_tokens', 2, - 'Use operator %s instead of %s' % ( - _ALT_TOKEN_REPLACEMENT[match.group(1)], match.group(1))) - - -def GetLineWidth(line): - """Determines the width of the line in column positions. - - Args: - line: A string, which may be a Unicode string. - - Returns: - The width of the line in column positions, accounting for Unicode - combining characters and wide characters. - """ - if isinstance(line, unicode): - width = 0 - for uc in unicodedata.normalize('NFC', line): - if unicodedata.east_asian_width(uc) in ('W', 'F'): - width += 2 - elif not unicodedata.combining(uc): - width += 1 - return width - else: - return len(line) - - -def CheckStyle(filename, clean_lines, linenum, file_extension, nesting_state, - error): - """Checks rules from the 'C++ style rules' section of cppguide.html. - - Most of these rules are hard to test (naming, comment style), but we - do what we can. In particular we check for 2-space indents, line lengths, - tab usage, spaces inside code, etc. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - file_extension: The extension (without the dot) of the filename. - nesting_state: A _NestingState instance which maintains information about - the current stack of nested blocks being parsed. - error: The function to call with any errors found. - """ - - # Don't use "elided" lines here, otherwise we can't check commented lines. - # Don't want to use "raw" either, because we don't want to check inside C++11 - # raw strings, - raw_lines = clean_lines.lines_without_raw_strings - line = raw_lines[linenum] - - if line.find('\t') != -1: - error(filename, linenum, 'whitespace/tab', 1, - 'Tab found; better to use spaces') - - # One or three blank spaces at the beginning of the line is weird; it's - # hard to reconcile that with 2-space indents. - # NOTE: here are the conditions rob pike used for his tests. Mine aren't - # as sophisticated, but it may be worth becoming so: RLENGTH==initial_spaces - # if(RLENGTH > 20) complain = 0; - # if(match($0, " +(error|private|public|protected):")) complain = 0; - # if(match(prev, "&& *$")) complain = 0; - # if(match(prev, "\\|\\| *$")) complain = 0; - # if(match(prev, "[\",=><] *$")) complain = 0; - # if(match($0, " <<")) complain = 0; - # if(match(prev, " +for \\(")) complain = 0; - # if(prevodd && match(prevprev, " +for \\(")) complain = 0; - initial_spaces = 0 - cleansed_line = clean_lines.elided[linenum] - while initial_spaces < len(line) and line[initial_spaces] == ' ': - initial_spaces += 1 - if line and line[-1].isspace(): - error(filename, linenum, 'whitespace/end_of_line', 4, - 'Line ends in whitespace. Consider deleting these extra spaces.') - # There are certain situations we allow one space, notably for section labels - elif ((initial_spaces == 1 or initial_spaces == 3) and - not Match(r'\s*\w+\s*:\s*$', cleansed_line)): - error(filename, linenum, 'whitespace/indent', 3, - 'Weird number of spaces at line-start. ' - 'Are you using a 2-space indent?') - - # Check if the line is a header guard. - is_header_guard = False - if file_extension == 'h': - cppvar = GetHeaderGuardCPPVariable(filename) - if (line.startswith('#ifndef %s' % cppvar) or - line.startswith('#define %s' % cppvar) or - line.startswith('#endif // %s' % cppvar)): - is_header_guard = True - # #include lines and header guards can be long, since there's no clean way to - # split them. - # - # URLs can be long too. It's possible to split these, but it makes them - # harder to cut&paste. - # - # The "$Id:...$" comment may also get very long without it being the - # developers fault. - if (not line.startswith('#include') and not is_header_guard and - not Match(r'^\s*//.*http(s?)://\S*$', line) and - not Match(r'^// \$Id:.*#[0-9]+ \$$', line)): - line_width = GetLineWidth(line) - extended_length = int((_line_length * 1.25)) - if line_width > extended_length: - error(filename, linenum, 'whitespace/line_length', 4, - 'Lines should very rarely be longer than %i characters' % - extended_length) - elif line_width > _line_length: - error(filename, linenum, 'whitespace/line_length', 2, - 'Lines should be <= %i characters long' % _line_length) - - if (cleansed_line.count(';') > 1 and - # for loops are allowed two ;'s (and may run over two lines). - cleansed_line.find('for') == -1 and - (GetPreviousNonBlankLine(clean_lines, linenum)[0].find('for') == -1 or - GetPreviousNonBlankLine(clean_lines, linenum)[0].find(';') != -1) and - # It's ok to have many commands in a switch case that fits in 1 line - not ((cleansed_line.find('case ') != -1 or - cleansed_line.find('default:') != -1) and - cleansed_line.find('break;') != -1)): - error(filename, linenum, 'whitespace/newline', 0, - 'More than one command on the same line') - - # Some more style checks - CheckBraces(filename, clean_lines, linenum, error) - CheckEmptyBlockBody(filename, clean_lines, linenum, error) - CheckAccess(filename, clean_lines, linenum, nesting_state, error) - CheckSpacing(filename, clean_lines, linenum, nesting_state, error) - CheckCheck(filename, clean_lines, linenum, error) - CheckAltTokens(filename, clean_lines, linenum, error) - classinfo = nesting_state.InnermostClass() - if classinfo: - CheckSectionSpacing(filename, clean_lines, classinfo, linenum, error) - - -_RE_PATTERN_INCLUDE_NEW_STYLE = re.compile(r'#include +"[^/]+\.h"') -_RE_PATTERN_INCLUDE = re.compile(r'^\s*#\s*include\s*([<"])([^>"]*)[>"].*$') -# Matches the first component of a filename delimited by -s and _s. That is: -# _RE_FIRST_COMPONENT.match('foo').group(0) == 'foo' -# _RE_FIRST_COMPONENT.match('foo.cc').group(0) == 'foo' -# _RE_FIRST_COMPONENT.match('foo-bar_baz.cc').group(0) == 'foo' -# _RE_FIRST_COMPONENT.match('foo_bar-baz.cc').group(0) == 'foo' -_RE_FIRST_COMPONENT = re.compile(r'^[^-_.]+') - - -def _DropCommonSuffixes(filename): - """Drops common suffixes like _test.cc or -inl.h from filename. - - For example: - >>> _DropCommonSuffixes('foo/foo-inl.h') - 'foo/foo' - >>> _DropCommonSuffixes('foo/bar/foo.cc') - 'foo/bar/foo' - >>> _DropCommonSuffixes('foo/foo_internal.h') - 'foo/foo' - >>> _DropCommonSuffixes('foo/foo_unusualinternal.h') - 'foo/foo_unusualinternal' - - Args: - filename: The input filename. - - Returns: - The filename with the common suffix removed. - """ - for suffix in ('test.cc', 'regtest.cc', 'unittest.cc', - 'inl.h', 'impl.h', 'internal.h'): - if (filename.endswith(suffix) and len(filename) > len(suffix) and - filename[-len(suffix) - 1] in ('-', '_')): - return filename[:-len(suffix) - 1] - return os.path.splitext(filename)[0] - - -def _IsTestFilename(filename): - """Determines if the given filename has a suffix that identifies it as a test. - - Args: - filename: The input filename. - - Returns: - True if 'filename' looks like a test, False otherwise. - """ - if (filename.endswith('_test.cc') or - filename.endswith('_unittest.cc') or - filename.endswith('_regtest.cc')): - return True - else: - return False - - -def _ClassifyInclude(fileinfo, include, is_system): - """Figures out what kind of header 'include' is. - - Args: - fileinfo: The current file cpplint is running over. A FileInfo instance. - include: The path to a #included file. - is_system: True if the #include used <> rather than "". - - Returns: - One of the _XXX_HEADER constants. - - For example: - >>> _ClassifyInclude(FileInfo('foo/foo.cc'), 'stdio.h', True) - _C_SYS_HEADER - >>> _ClassifyInclude(FileInfo('foo/foo.cc'), 'string', True) - _CPP_SYS_HEADER - >>> _ClassifyInclude(FileInfo('foo/foo.cc'), 'foo/foo.h', False) - _LIKELY_MY_HEADER - >>> _ClassifyInclude(FileInfo('foo/foo_unknown_extension.cc'), - ... 'bar/foo_other_ext.h', False) - _POSSIBLE_MY_HEADER - >>> _ClassifyInclude(FileInfo('foo/foo.cc'), 'foo/bar.h', False) - _OTHER_HEADER - """ - # This is a list of all standard c++ header files, except - # those already checked for above. - is_cpp_h = include in _CPP_HEADERS - - if is_system: - if is_cpp_h: - return _CPP_SYS_HEADER - else: - return _C_SYS_HEADER - - # If the target file and the include we're checking share a - # basename when we drop common extensions, and the include - # lives in . , then it's likely to be owned by the target file. - target_dir, target_base = ( - os.path.split(_DropCommonSuffixes(fileinfo.RepositoryName()))) - include_dir, include_base = os.path.split(_DropCommonSuffixes(include)) - if target_base == include_base and ( - include_dir == target_dir or - include_dir == os.path.normpath(target_dir + '/../public')): - return _LIKELY_MY_HEADER - - # If the target and include share some initial basename - # component, it's possible the target is implementing the - # include, so it's allowed to be first, but we'll never - # complain if it's not there. - target_first_component = _RE_FIRST_COMPONENT.match(target_base) - include_first_component = _RE_FIRST_COMPONENT.match(include_base) - if (target_first_component and include_first_component and - target_first_component.group(0) == - include_first_component.group(0)): - return _POSSIBLE_MY_HEADER - - return _OTHER_HEADER - - - -def CheckIncludeLine(filename, clean_lines, linenum, include_state, error): - """Check rules that are applicable to #include lines. - - Strings on #include lines are NOT removed from elided line, to make - certain tasks easier. However, to prevent false positives, checks - applicable to #include lines in CheckLanguage must be put here. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - include_state: An _IncludeState instance in which the headers are inserted. - error: The function to call with any errors found. - """ - fileinfo = FileInfo(filename) - - line = clean_lines.lines[linenum] - - # "include" should use the new style "foo/bar.h" instead of just "bar.h" - if _RE_PATTERN_INCLUDE_NEW_STYLE.search(line): - error(filename, linenum, 'build/include', 4, - 'Include the directory when naming .h files') - - # we shouldn't include a file more than once. actually, there are a - # handful of instances where doing so is okay, but in general it's - # not. - match = _RE_PATTERN_INCLUDE.search(line) - if match: - include = match.group(2) - is_system = (match.group(1) == '<') - if include in include_state: - error(filename, linenum, 'build/include', 4, - '"%s" already included at %s:%s' % - (include, filename, include_state[include])) - else: - include_state[include] = linenum - - # We want to ensure that headers appear in the right order: - # 1) for foo.cc, foo.h (preferred location) - # 2) c system files - # 3) cpp system files - # 4) for foo.cc, foo.h (deprecated location) - # 5) other google headers - # - # We classify each include statement as one of those 5 types - # using a number of techniques. The include_state object keeps - # track of the highest type seen, and complains if we see a - # lower type after that. - error_message = include_state.CheckNextIncludeOrder( - _ClassifyInclude(fileinfo, include, is_system)) - if error_message: - error(filename, linenum, 'build/include_order', 4, - '%s. Should be: %s.h, c system, c++ system, other.' % - (error_message, fileinfo.BaseName())) - canonical_include = include_state.CanonicalizeAlphabeticalOrder(include) - if not include_state.IsInAlphabeticalOrder( - clean_lines, linenum, canonical_include): - error(filename, linenum, 'build/include_alpha', 4, - 'Include "%s" not in alphabetical order' % include) - include_state.SetLastHeader(canonical_include) - - # Look for any of the stream classes that are part of standard C++. - match = _RE_PATTERN_INCLUDE.match(line) - if match: - include = match.group(2) - if Match(r'(f|ind|io|i|o|parse|pf|stdio|str|)?stream$', include): - # Many unit tests use cout, so we exempt them. - if not _IsTestFilename(filename): - error(filename, linenum, 'readability/streams', 3, - 'Streams are highly discouraged.') - - -def _GetTextInside(text, start_pattern): - r"""Retrieves all the text between matching open and close parentheses. - - Given a string of lines and a regular expression string, retrieve all the text - following the expression and between opening punctuation symbols like - (, [, or {, and the matching close-punctuation symbol. This properly nested - occurrences of the punctuations, so for the text like - printf(a(), b(c())); - a call to _GetTextInside(text, r'printf\(') will return 'a(), b(c())'. - start_pattern must match string having an open punctuation symbol at the end. - - Args: - text: The lines to extract text. Its comments and strings must be elided. - It can be single line and can span multiple lines. - start_pattern: The regexp string indicating where to start extracting - the text. - Returns: - The extracted text. - None if either the opening string or ending punctuation could not be found. - """ - # TODO(sugawarayu): Audit cpplint.py to see what places could be profitably - # rewritten to use _GetTextInside (and use inferior regexp matching today). - - # Give opening punctuations to get the matching close-punctuations. - matching_punctuation = {'(': ')', '{': '}', '[': ']'} - closing_punctuation = set(matching_punctuation.itervalues()) - - # Find the position to start extracting text. - match = re.search(start_pattern, text, re.M) - if not match: # start_pattern not found in text. - return None - start_position = match.end(0) - - assert start_position > 0, ( - 'start_pattern must ends with an opening punctuation.') - assert text[start_position - 1] in matching_punctuation, ( - 'start_pattern must ends with an opening punctuation.') - # Stack of closing punctuations we expect to have in text after position. - punctuation_stack = [matching_punctuation[text[start_position - 1]]] - position = start_position - while punctuation_stack and position < len(text): - if text[position] == punctuation_stack[-1]: - punctuation_stack.pop() - elif text[position] in closing_punctuation: - # A closing punctuation without matching opening punctuations. - return None - elif text[position] in matching_punctuation: - punctuation_stack.append(matching_punctuation[text[position]]) - position += 1 - if punctuation_stack: - # Opening punctuations left without matching close-punctuations. - return None - # punctuations match. - return text[start_position:position - 1] - - -# Patterns for matching call-by-reference parameters. -# -# Supports nested templates up to 2 levels deep using this messy pattern: -# < (?: < (?: < [^<>]* -# > -# | [^<>] )* -# > -# | [^<>] )* -# > -_RE_PATTERN_IDENT = r'[_a-zA-Z]\w*' # =~ [[:alpha:]][[:alnum:]]* -_RE_PATTERN_TYPE = ( - r'(?:const\s+)?(?:typename\s+|class\s+|struct\s+|union\s+|enum\s+)?' - r'(?:\w|' - r'\s*<(?:<(?:<[^<>]*>|[^<>])*>|[^<>])*>|' - r'::)+') -# A call-by-reference parameter ends with '& identifier'. -_RE_PATTERN_REF_PARAM = re.compile( - r'(' + _RE_PATTERN_TYPE + r'(?:\s*(?:\bconst\b|[*]))*\s*' - r'&\s*' + _RE_PATTERN_IDENT + r')\s*(?:=[^,()]+)?[,)]') -# A call-by-const-reference parameter either ends with 'const& identifier' -# or looks like 'const type& identifier' when 'type' is atomic. -_RE_PATTERN_CONST_REF_PARAM = ( - r'(?:.*\s*\bconst\s*&\s*' + _RE_PATTERN_IDENT + - r'|const\s+' + _RE_PATTERN_TYPE + r'\s*&\s*' + _RE_PATTERN_IDENT + r')') - - -def CheckLanguage(filename, clean_lines, linenum, file_extension, - include_state, nesting_state, error): - """Checks rules from the 'C++ language rules' section of cppguide.html. - - Some of these rules are hard to test (function overloading, using - uint32 inappropriately), but we do the best we can. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - file_extension: The extension (without the dot) of the filename. - include_state: An _IncludeState instance in which the headers are inserted. - nesting_state: A _NestingState instance which maintains information about - the current stack of nested blocks being parsed. - error: The function to call with any errors found. - """ - # If the line is empty or consists of entirely a comment, no need to - # check it. - line = clean_lines.elided[linenum] - if not line: - return - - match = _RE_PATTERN_INCLUDE.search(line) - if match: - CheckIncludeLine(filename, clean_lines, linenum, include_state, error) - return - - # Reset include state across preprocessor directives. This is meant - # to silence warnings for conditional includes. - if Match(r'^\s*#\s*(?:ifdef|elif|else|endif)\b', line): - include_state.ResetSection() - - # Make Windows paths like Unix. - fullname = os.path.abspath(filename).replace('\\', '/') - - # TODO(unknown): figure out if they're using default arguments in fn proto. - - # Check to see if they're using an conversion function cast. - # I just try to capture the most common basic types, though there are more. - # Parameterless conversion functions, such as bool(), are allowed as they are - # probably a member operator declaration or default constructor. - match = Search( - r'(\bnew\s+)?\b' # Grab 'new' operator, if it's there - r'(int|float|double|bool|char|int32|uint32|int64|uint64)' - r'(\([^)].*)', line) - if match: - matched_new = match.group(1) - matched_type = match.group(2) - matched_funcptr = match.group(3) - - # gMock methods are defined using some variant of MOCK_METHODx(name, type) - # where type may be float(), int(string), etc. Without context they are - # virtually indistinguishable from int(x) casts. Likewise, gMock's - # MockCallback takes a template parameter of the form return_type(arg_type), - # which looks much like the cast we're trying to detect. - # - # std::function<> wrapper has a similar problem. - # - # Return types for function pointers also look like casts if they - # don't have an extra space. - if (matched_new is None and # If new operator, then this isn't a cast - not (Match(r'^\s*MOCK_(CONST_)?METHOD\d+(_T)?\(', line) or - Search(r'\bMockCallback<.*>', line) or - Search(r'\bstd::function<.*>', line)) and - not (matched_funcptr and - Match(r'\((?:[^() ]+::\s*\*\s*)?[^() ]+\)\s*\(', - matched_funcptr))): - # Try a bit harder to catch gmock lines: the only place where - # something looks like an old-style cast is where we declare the - # return type of the mocked method, and the only time when we - # are missing context is if MOCK_METHOD was split across - # multiple lines. The missing MOCK_METHOD is usually one or two - # lines back, so scan back one or two lines. - # - # It's not possible for gmock macros to appear in the first 2 - # lines, since the class head + section name takes up 2 lines. - if (linenum < 2 or - not (Match(r'^\s*MOCK_(?:CONST_)?METHOD\d+(?:_T)?\((?:\S+,)?\s*$', - clean_lines.elided[linenum - 1]) or - Match(r'^\s*MOCK_(?:CONST_)?METHOD\d+(?:_T)?\(\s*$', - clean_lines.elided[linenum - 2]))): - error(filename, linenum, 'readability/casting', 4, - 'Using deprecated casting style. ' - 'Use static_cast<%s>(...) instead' % - matched_type) - - CheckCStyleCast(filename, linenum, line, clean_lines.raw_lines[linenum], - 'static_cast', - r'\((int|float|double|bool|char|u?int(16|32|64))\)', error) - - # This doesn't catch all cases. Consider (const char * const)"hello". - # - # (char *) "foo" should always be a const_cast (reinterpret_cast won't - # compile). - if CheckCStyleCast(filename, linenum, line, clean_lines.raw_lines[linenum], - 'const_cast', r'\((char\s?\*+\s?)\)\s*"', error): - pass - else: - # Check pointer casts for other than string constants - CheckCStyleCast(filename, linenum, line, clean_lines.raw_lines[linenum], - 'reinterpret_cast', r'\((\w+\s?\*+\s?)\)', error) - - # In addition, we look for people taking the address of a cast. This - # is dangerous -- casts can assign to temporaries, so the pointer doesn't - # point where you think. - match = Search( - r'(?:&\(([^)]+)\)[\w(])|' - r'(?:&(static|dynamic|down|reinterpret)_cast\b)', line) - if match and match.group(1) != '*': - error(filename, linenum, 'runtime/casting', 4, - ('Are you taking an address of a cast? ' - 'This is dangerous: could be a temp var. ' - 'Take the address before doing the cast, rather than after')) - - # Create an extended_line, which is the concatenation of the current and - # next lines, for more effective checking of code that may span more than one - # line. - if linenum + 1 < clean_lines.NumLines(): - extended_line = line + clean_lines.elided[linenum + 1] - else: - extended_line = line - - # Check for people declaring static/global STL strings at the top level. - # This is dangerous because the C++ language does not guarantee that - # globals with constructors are initialized before the first access. - match = Match( - r'((?:|static +)(?:|const +))string +([a-zA-Z0-9_:]+)\b(.*)', - line) - # Make sure it's not a function. - # Function template specialization looks like: "string foo(...". - # Class template definitions look like: "string Foo::Method(...". - # - # Also ignore things that look like operators. These are matched separately - # because operator names cross non-word boundaries. If we change the pattern - # above, we would decrease the accuracy of matching identifiers. - if (match and - not Search(r'\boperator\W', line) and - not Match(r'\s*(<.*>)?(::[a-zA-Z0-9_]+)?\s*\(([^"]|$)', match.group(3))): - error(filename, linenum, 'runtime/string', 4, - 'For a static/global string constant, use a C style string instead: ' - '"%schar %s[]".' % - (match.group(1), match.group(2))) - - if Search(r'\b([A-Za-z0-9_]*_)\(\1\)', line): - error(filename, linenum, 'runtime/init', 4, - 'You seem to be initializing a member variable with itself.') - - if file_extension == 'h': - # TODO(unknown): check that 1-arg constructors are explicit. - # How to tell it's a constructor? - # (handled in CheckForNonStandardConstructs for now) - # TODO(unknown): check that classes have DISALLOW_EVIL_CONSTRUCTORS - # (level 1 error) - pass - - # Check if people are using the verboten C basic types. The only exception - # we regularly allow is "unsigned short port" for port. - if Search(r'\bshort port\b', line): - if not Search(r'\bunsigned short port\b', line): - error(filename, linenum, 'runtime/int', 4, - 'Use "unsigned short" for ports, not "short"') - else: - match = Search(r'\b(short|long(?! +double)|long long)\b', line) - if match: - error(filename, linenum, 'runtime/int', 4, - 'Use int16/int64/etc, rather than the C type %s' % match.group(1)) - - # When snprintf is used, the second argument shouldn't be a literal. - match = Search(r'snprintf\s*\(([^,]*),\s*([0-9]*)\s*,', line) - if match and match.group(2) != '0': - # If 2nd arg is zero, snprintf is used to calculate size. - error(filename, linenum, 'runtime/printf', 3, - 'If you can, use sizeof(%s) instead of %s as the 2nd arg ' - 'to snprintf.' % (match.group(1), match.group(2))) - - # Check if some verboten C functions are being used. - if Search(r'\bsprintf\b', line): - error(filename, linenum, 'runtime/printf', 5, - 'Never use sprintf. Use snprintf instead.') - match = Search(r'\b(strcpy|strcat)\b', line) - if match: - error(filename, linenum, 'runtime/printf', 4, - 'Almost always, snprintf is better than %s' % match.group(1)) - - # Check if some verboten operator overloading is going on - # TODO(unknown): catch out-of-line unary operator&: - # class X {}; - # int operator&(const X& x) { return 42; } // unary operator& - # The trick is it's hard to tell apart from binary operator&: - # class Y { int operator&(const Y& x) { return 23; } }; // binary operator& - if Search(r'\boperator\s*&\s*\(\s*\)', line): - error(filename, linenum, 'runtime/operator', 4, - 'Unary operator& is dangerous. Do not use it.') - - # Check for suspicious usage of "if" like - # } if (a == b) { - if Search(r'\}\s*if\s*\(', line): - error(filename, linenum, 'readability/braces', 4, - 'Did you mean "else if"? If not, start a new line for "if".') - - # Check for potential format string bugs like printf(foo). - # We constrain the pattern not to pick things like DocidForPrintf(foo). - # Not perfect but it can catch printf(foo.c_str()) and printf(foo->c_str()) - # TODO(sugawarayu): Catch the following case. Need to change the calling - # convention of the whole function to process multiple line to handle it. - # printf( - # boy_this_is_a_really_long_variable_that_cannot_fit_on_the_prev_line); - printf_args = _GetTextInside(line, r'(?i)\b(string)?printf\s*\(') - if printf_args: - match = Match(r'([\w.\->()]+)$', printf_args) - if match and match.group(1) != '__VA_ARGS__': - function_name = re.search(r'\b((?:string)?printf)\s*\(', - line, re.I).group(1) - error(filename, linenum, 'runtime/printf', 4, - 'Potential format string bug. Do %s("%%s", %s) instead.' - % (function_name, match.group(1))) - - # Check for potential memset bugs like memset(buf, sizeof(buf), 0). - match = Search(r'memset\s*\(([^,]*),\s*([^,]*),\s*0\s*\)', line) - if match and not Match(r"^''|-?[0-9]+|0x[0-9A-Fa-f]$", match.group(2)): - error(filename, linenum, 'runtime/memset', 4, - 'Did you mean "memset(%s, 0, %s)"?' - % (match.group(1), match.group(2))) - - if Search(r'\busing namespace\b', line): - error(filename, linenum, 'build/namespaces', 5, - 'Do not use namespace using-directives. ' - 'Use using-declarations instead.') - - # Detect variable-length arrays. - match = Match(r'\s*(.+::)?(\w+) [a-z]\w*\[(.+)];', line) - if (match and match.group(2) != 'return' and match.group(2) != 'delete' and - match.group(3).find(']') == -1): - # Split the size using space and arithmetic operators as delimiters. - # If any of the resulting tokens are not compile time constants then - # report the error. - tokens = re.split(r'\s|\+|\-|\*|\/|<<|>>]', match.group(3)) - is_const = True - skip_next = False - for tok in tokens: - if skip_next: - skip_next = False - continue - - if Search(r'sizeof\(.+\)', tok): continue - if Search(r'arraysize\(\w+\)', tok): continue - - tok = tok.lstrip('(') - tok = tok.rstrip(')') - if not tok: continue - if Match(r'\d+', tok): continue - if Match(r'0[xX][0-9a-fA-F]+', tok): continue - if Match(r'k[A-Z0-9]\w*', tok): continue - if Match(r'(.+::)?k[A-Z0-9]\w*', tok): continue - if Match(r'(.+::)?[A-Z][A-Z0-9_]*', tok): continue - # A catch all for tricky sizeof cases, including 'sizeof expression', - # 'sizeof(*type)', 'sizeof(const type)', 'sizeof(struct StructName)' - # requires skipping the next token because we split on ' ' and '*'. - if tok.startswith('sizeof'): - skip_next = True - continue - is_const = False - break - if not is_const: - error(filename, linenum, 'runtime/arrays', 1, - 'Do not use variable-length arrays. Use an appropriately named ' - "('k' followed by CamelCase) compile-time constant for the size.") - - # If DISALLOW_EVIL_CONSTRUCTORS, DISALLOW_COPY_AND_ASSIGN, or - # DISALLOW_IMPLICIT_CONSTRUCTORS is present, then it should be the last thing - # in the class declaration. - match = Match( - (r'\s*' - r'(DISALLOW_(EVIL_CONSTRUCTORS|COPY_AND_ASSIGN|IMPLICIT_CONSTRUCTORS))' - r'\(.*\);$'), - line) - if match and linenum + 1 < clean_lines.NumLines(): - next_line = clean_lines.elided[linenum + 1] - # We allow some, but not all, declarations of variables to be present - # in the statement that defines the class. The [\w\*,\s]* fragment of - # the regular expression below allows users to declare instances of - # the class or pointers to instances, but not less common types such - # as function pointers or arrays. It's a tradeoff between allowing - # reasonable code and avoiding trying to parse more C++ using regexps. - if not Search(r'^\s*}[\w\*,\s]*;', next_line): - error(filename, linenum, 'readability/constructors', 3, - match.group(1) + ' should be the last thing in the class') - - # Check for use of unnamed namespaces in header files. Registration - # macros are typically OK, so we allow use of "namespace {" on lines - # that end with backslashes. - if (file_extension == 'h' - and Search(r'\bnamespace\s*{', line) - and line[-1] != '\\'): - error(filename, linenum, 'build/namespaces', 4, - 'Do not use unnamed namespaces in header files. See ' - 'http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml#Namespaces' - ' for more information.') - -def CheckForNonConstReference(filename, clean_lines, linenum, - nesting_state, error): - """Check for non-const references. - - Separate from CheckLanguage since it scans backwards from current - line, instead of scanning forward. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - nesting_state: A _NestingState instance which maintains information about - the current stack of nested blocks being parsed. - error: The function to call with any errors found. - """ - # Do nothing if there is no '&' on current line. - line = clean_lines.elided[linenum] - if '&' not in line: - return - - # Long type names may be broken across multiple lines, usually in one - # of these forms: - # LongType - # ::LongTypeContinued &identifier - # LongType:: - # LongTypeContinued &identifier - # LongType< - # ...>::LongTypeContinued &identifier - # - # If we detected a type split across two lines, join the previous - # line to current line so that we can match const references - # accordingly. - # - # Note that this only scans back one line, since scanning back - # arbitrary number of lines would be expensive. If you have a type - # that spans more than 2 lines, please use a typedef. - if linenum > 1: - previous = None - if Match(r'\s*::(?:[\w<>]|::)+\s*&\s*\S', line): - # previous_line\n + ::current_line - previous = Search(r'\b((?:const\s*)?(?:[\w<>]|::)+[\w<>])\s*$', - clean_lines.elided[linenum - 1]) - elif Match(r'\s*[a-zA-Z_]([\w<>]|::)+\s*&\s*\S', line): - # previous_line::\n + current_line - previous = Search(r'\b((?:const\s*)?(?:[\w<>]|::)+::)\s*$', - clean_lines.elided[linenum - 1]) - if previous: - line = previous.group(1) + line.lstrip() - else: - # Check for templated parameter that is split across multiple lines - endpos = line.rfind('>') - if endpos > -1: - (_, startline, startpos) = ReverseCloseExpression( - clean_lines, linenum, endpos) - if startpos > -1 and startline < linenum: - # Found the matching < on an earlier line, collect all - # pieces up to current line. - line = '' - for i in xrange(startline, linenum + 1): - line += clean_lines.elided[i].strip() - - # Check for non-const references in function parameters. A single '&' may - # found in the following places: - # inside expression: binary & for bitwise AND - # inside expression: unary & for taking the address of something - # inside declarators: reference parameter - # We will exclude the first two cases by checking that we are not inside a - # function body, including one that was just introduced by a trailing '{'. - # TODO(unknwon): Doesn't account for preprocessor directives. - # TODO(unknown): Doesn't account for 'catch(Exception& e)' [rare]. - check_params = False - if not nesting_state.stack: - check_params = True # top level - elif (isinstance(nesting_state.stack[-1], _ClassInfo) or - isinstance(nesting_state.stack[-1], _NamespaceInfo)): - check_params = True # within class or namespace - elif Match(r'.*{\s*$', line): - if (len(nesting_state.stack) == 1 or - isinstance(nesting_state.stack[-2], _ClassInfo) or - isinstance(nesting_state.stack[-2], _NamespaceInfo)): - check_params = True # just opened global/class/namespace block - # We allow non-const references in a few standard places, like functions - # called "swap()" or iostream operators like "<<" or ">>". Do not check - # those function parameters. - # - # We also accept & in static_assert, which looks like a function but - # it's actually a declaration expression. - whitelisted_functions = (r'(?:[sS]wap(?:<\w:+>)?|' - r'operator\s*[<>][<>]|' - r'static_assert|COMPILE_ASSERT' - r')\s*\(') - if Search(whitelisted_functions, line): - check_params = False - elif not Search(r'\S+\([^)]*$', line): - # Don't see a whitelisted function on this line. Actually we - # didn't see any function name on this line, so this is likely a - # multi-line parameter list. Try a bit harder to catch this case. - for i in xrange(2): - if (linenum > i and - Search(whitelisted_functions, clean_lines.elided[linenum - i - 1])): - check_params = False - break - - if check_params: - decls = ReplaceAll(r'{[^}]*}', ' ', line) # exclude function body - for parameter in re.findall(_RE_PATTERN_REF_PARAM, decls): - if not Match(_RE_PATTERN_CONST_REF_PARAM, parameter): - error(filename, linenum, 'runtime/references', 2, - 'Is this a non-const reference? ' - 'If so, make const or use a pointer: ' + - ReplaceAll(' *<', '<', parameter)) - - -def CheckCStyleCast(filename, linenum, line, raw_line, cast_type, pattern, - error): - """Checks for a C-style cast by looking for the pattern. - - Args: - filename: The name of the current file. - linenum: The number of the line to check. - line: The line of code to check. - raw_line: The raw line of code to check, with comments. - cast_type: The string for the C++ cast to recommend. This is either - reinterpret_cast, static_cast, or const_cast, depending. - pattern: The regular expression used to find C-style casts. - error: The function to call with any errors found. - - Returns: - True if an error was emitted. - False otherwise. - """ - match = Search(pattern, line) - if not match: - return False - - # Exclude lines with sizeof, since sizeof looks like a cast. - sizeof_match = Match(r'.*sizeof\s*$', line[0:match.start(1) - 1]) - if sizeof_match: - return False - - # operator++(int) and operator--(int) - if (line[0:match.start(1) - 1].endswith(' operator++') or - line[0:match.start(1) - 1].endswith(' operator--')): - return False - - # A single unnamed argument for a function tends to look like old - # style cast. If we see those, don't issue warnings for deprecated - # casts, instead issue warnings for unnamed arguments where - # appropriate. - # - # These are things that we want warnings for, since the style guide - # explicitly require all parameters to be named: - # Function(int); - # Function(int) { - # ConstMember(int) const; - # ConstMember(int) const { - # ExceptionMember(int) throw (...); - # ExceptionMember(int) throw (...) { - # PureVirtual(int) = 0; - # - # These are functions of some sort, where the compiler would be fine - # if they had named parameters, but people often omit those - # identifiers to reduce clutter: - # (FunctionPointer)(int); - # (FunctionPointer)(int) = value; - # Function((function_pointer_arg)(int)) - # ; - # <(FunctionPointerTemplateArgument)(int)>; - remainder = line[match.end(0):] - if Match(r'^\s*(?:;|const\b|throw\b|=|>|\{|\))', remainder): - # Looks like an unnamed parameter. - - # Don't warn on any kind of template arguments. - if Match(r'^\s*>', remainder): - return False - - # Don't warn on assignments to function pointers, but keep warnings for - # unnamed parameters to pure virtual functions. Note that this pattern - # will also pass on assignments of "0" to function pointers, but the - # preferred values for those would be "nullptr" or "NULL". - matched_zero = Match(r'^\s=\s*(\S+)\s*;', remainder) - if matched_zero and matched_zero.group(1) != '0': - return False - - # Don't warn on function pointer declarations. For this we need - # to check what came before the "(type)" string. - if Match(r'.*\)\s*$', line[0:match.start(0)]): - return False - - # Don't warn if the parameter is named with block comments, e.g.: - # Function(int /*unused_param*/); - if '/*' in raw_line: - return False - - # Passed all filters, issue warning here. - error(filename, linenum, 'readability/function', 3, - 'All parameters should be named in a function') - return True - - # At this point, all that should be left is actual casts. - error(filename, linenum, 'readability/casting', 4, - 'Using C-style cast. Use %s<%s>(...) instead' % - (cast_type, match.group(1))) - - return True - - -_HEADERS_CONTAINING_TEMPLATES = ( - ('', ('deque',)), - ('', ('unary_function', 'binary_function', - 'plus', 'minus', 'multiplies', 'divides', 'modulus', - 'negate', - 'equal_to', 'not_equal_to', 'greater', 'less', - 'greater_equal', 'less_equal', - 'logical_and', 'logical_or', 'logical_not', - 'unary_negate', 'not1', 'binary_negate', 'not2', - 'bind1st', 'bind2nd', - 'pointer_to_unary_function', - 'pointer_to_binary_function', - 'ptr_fun', - 'mem_fun_t', 'mem_fun', 'mem_fun1_t', 'mem_fun1_ref_t', - 'mem_fun_ref_t', - 'const_mem_fun_t', 'const_mem_fun1_t', - 'const_mem_fun_ref_t', 'const_mem_fun1_ref_t', - 'mem_fun_ref', - )), - ('', ('numeric_limits',)), - ('', ('list',)), - ('', ('map', 'multimap',)), - ('', ('allocator',)), - ('', ('queue', 'priority_queue',)), - ('', ('set', 'multiset',)), - ('', ('stack',)), - ('', ('char_traits', 'basic_string',)), - ('', ('pair',)), - ('', ('vector',)), - - # gcc extensions. - # Note: std::hash is their hash, ::hash is our hash - ('', ('hash_map', 'hash_multimap',)), - ('', ('hash_set', 'hash_multiset',)), - ('', ('slist',)), - ) - -_RE_PATTERN_STRING = re.compile(r'\bstring\b') - -_re_pattern_algorithm_header = [] -for _template in ('copy', 'max', 'min', 'min_element', 'sort', 'swap', - 'transform'): - # Match max(..., ...), max(..., ...), but not foo->max, foo.max or - # type::max(). - _re_pattern_algorithm_header.append( - (re.compile(r'[^>.]\b' + _template + r'(<.*?>)?\([^\)]'), - _template, - '')) - -_re_pattern_templates = [] -for _header, _templates in _HEADERS_CONTAINING_TEMPLATES: - for _template in _templates: - _re_pattern_templates.append( - (re.compile(r'(\<|\b)' + _template + r'\s*\<'), - _template + '<>', - _header)) - - -def FilesBelongToSameModule(filename_cc, filename_h): - """Check if these two filenames belong to the same module. - - The concept of a 'module' here is a as follows: - foo.h, foo-inl.h, foo.cc, foo_test.cc and foo_unittest.cc belong to the - same 'module' if they are in the same directory. - some/path/public/xyzzy and some/path/internal/xyzzy are also considered - to belong to the same module here. - - If the filename_cc contains a longer path than the filename_h, for example, - '/absolute/path/to/base/sysinfo.cc', and this file would include - 'base/sysinfo.h', this function also produces the prefix needed to open the - header. This is used by the caller of this function to more robustly open the - header file. We don't have access to the real include paths in this context, - so we need this guesswork here. - - Known bugs: tools/base/bar.cc and base/bar.h belong to the same module - according to this implementation. Because of this, this function gives - some false positives. This should be sufficiently rare in practice. - - Args: - filename_cc: is the path for the .cc file - filename_h: is the path for the header path - - Returns: - Tuple with a bool and a string: - bool: True if filename_cc and filename_h belong to the same module. - string: the additional prefix needed to open the header file. - """ - - if not filename_cc.endswith('.cc'): - return (False, '') - filename_cc = filename_cc[:-len('.cc')] - if filename_cc.endswith('_unittest'): - filename_cc = filename_cc[:-len('_unittest')] - elif filename_cc.endswith('_test'): - filename_cc = filename_cc[:-len('_test')] - filename_cc = filename_cc.replace('/public/', '/') - filename_cc = filename_cc.replace('/internal/', '/') - - if not filename_h.endswith('.h'): - return (False, '') - filename_h = filename_h[:-len('.h')] - if filename_h.endswith('-inl'): - filename_h = filename_h[:-len('-inl')] - filename_h = filename_h.replace('/public/', '/') - filename_h = filename_h.replace('/internal/', '/') - - files_belong_to_same_module = filename_cc.endswith(filename_h) - common_path = '' - if files_belong_to_same_module: - common_path = filename_cc[:-len(filename_h)] - return files_belong_to_same_module, common_path - - -def UpdateIncludeState(filename, include_state, io=codecs): - """Fill up the include_state with new includes found from the file. - - Args: - filename: the name of the header to read. - include_state: an _IncludeState instance in which the headers are inserted. - io: The io factory to use to read the file. Provided for testability. - - Returns: - True if a header was successfully added. False otherwise. - """ - headerfile = None - try: - headerfile = io.open(filename, 'r', 'utf8', 'replace') - except IOError: - return False - linenum = 0 - for line in headerfile: - linenum += 1 - clean_line = CleanseComments(line) - match = _RE_PATTERN_INCLUDE.search(clean_line) - if match: - include = match.group(2) - # The value formatting is cute, but not really used right now. - # What matters here is that the key is in include_state. - include_state.setdefault(include, '%s:%d' % (filename, linenum)) - return True - - -def CheckForIncludeWhatYouUse(filename, clean_lines, include_state, error, - io=codecs): - """Reports for missing stl includes. - - This function will output warnings to make sure you are including the headers - necessary for the stl containers and functions that you use. We only give one - reason to include a header. For example, if you use both equal_to<> and - less<> in a .h file, only one (the latter in the file) of these will be - reported as a reason to include the . - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - include_state: An _IncludeState instance. - error: The function to call with any errors found. - io: The IO factory to use to read the header file. Provided for unittest - injection. - """ - required = {} # A map of header name to linenumber and the template entity. - # Example of required: { '': (1219, 'less<>') } - - for linenum in xrange(clean_lines.NumLines()): - line = clean_lines.elided[linenum] - if not line or line[0] == '#': - continue - - # String is special -- it is a non-templatized type in STL. - matched = _RE_PATTERN_STRING.search(line) - if matched: - # Don't warn about strings in non-STL namespaces: - # (We check only the first match per line; good enough.) - prefix = line[:matched.start()] - if prefix.endswith('std::') or not prefix.endswith('::'): - required[''] = (linenum, 'string') - - for pattern, template, header in _re_pattern_algorithm_header: - if pattern.search(line): - required[header] = (linenum, template) - - # The following function is just a speed up, no semantics are changed. - if not '<' in line: # Reduces the cpu time usage by skipping lines. - continue - - for pattern, template, header in _re_pattern_templates: - if pattern.search(line): - required[header] = (linenum, template) - - # The policy is that if you #include something in foo.h you don't need to - # include it again in foo.cc. Here, we will look at possible includes. - # Let's copy the include_state so it is only messed up within this function. - include_state = include_state.copy() - - # Did we find the header for this file (if any) and successfully load it? - header_found = False - - # Use the absolute path so that matching works properly. - abs_filename = FileInfo(filename).FullName() - - # For Emacs's flymake. - # If cpplint is invoked from Emacs's flymake, a temporary file is generated - # by flymake and that file name might end with '_flymake.cc'. In that case, - # restore original file name here so that the corresponding header file can be - # found. - # e.g. If the file name is 'foo_flymake.cc', we should search for 'foo.h' - # instead of 'foo_flymake.h' - abs_filename = re.sub(r'_flymake\.cc$', '.cc', abs_filename) - - # include_state is modified during iteration, so we iterate over a copy of - # the keys. - header_keys = include_state.keys() - for header in header_keys: - (same_module, common_path) = FilesBelongToSameModule(abs_filename, header) - fullpath = common_path + header - if same_module and UpdateIncludeState(fullpath, include_state, io): - header_found = True - - # If we can't find the header file for a .cc, assume it's because we don't - # know where to look. In that case we'll give up as we're not sure they - # didn't include it in the .h file. - # TODO(unknown): Do a better job of finding .h files so we are confident that - # not having the .h file means there isn't one. - if filename.endswith('.cc') and not header_found: - return - - # All the lines have been processed, report the errors found. - for required_header_unstripped in required: - template = required[required_header_unstripped][1] - if required_header_unstripped.strip('<>"') not in include_state: - error(filename, required[required_header_unstripped][0], - 'build/include_what_you_use', 4, - 'Add #include ' + required_header_unstripped + ' for ' + template) - - -_RE_PATTERN_EXPLICIT_MAKEPAIR = re.compile(r'\bmake_pair\s*<') - - -def CheckMakePairUsesDeduction(filename, clean_lines, linenum, error): - """Check that make_pair's template arguments are deduced. - - G++ 4.6 in C++0x mode fails badly if make_pair's template arguments are - specified explicitly, and such use isn't intended in any case. - - Args: - filename: The name of the current file. - clean_lines: A CleansedLines instance containing the file. - linenum: The number of the line to check. - error: The function to call with any errors found. - """ - line = clean_lines.elided[linenum] - match = _RE_PATTERN_EXPLICIT_MAKEPAIR.search(line) - if match: - error(filename, linenum, 'build/explicit_make_pair', - 4, # 4 = high confidence - 'For C++11-compatibility, omit template arguments from make_pair' - ' OR use pair directly OR if appropriate, construct a pair directly') - - -def ProcessLine(filename, file_extension, clean_lines, line, - include_state, function_state, nesting_state, error, - extra_check_functions=[]): - """Processes a single line in the file. - - Args: - filename: Filename of the file that is being processed. - file_extension: The extension (dot not included) of the file. - clean_lines: An array of strings, each representing a line of the file, - with comments stripped. - line: Number of line being processed. - include_state: An _IncludeState instance in which the headers are inserted. - function_state: A _FunctionState instance which counts function lines, etc. - nesting_state: A _NestingState instance which maintains information about - the current stack of nested blocks being parsed. - error: A callable to which errors are reported, which takes 4 arguments: - filename, line number, error level, and message - extra_check_functions: An array of additional check functions that will be - run on each source line. Each function takes 4 - arguments: filename, clean_lines, line, error - """ - raw_lines = clean_lines.raw_lines - ParseNolintSuppressions(filename, raw_lines[line], line, error) - nesting_state.Update(filename, clean_lines, line, error) - if nesting_state.stack and nesting_state.stack[-1].inline_asm != _NO_ASM: - return - CheckForFunctionLengths(filename, clean_lines, line, function_state, error) - CheckForMultilineCommentsAndStrings(filename, clean_lines, line, error) - CheckStyle(filename, clean_lines, line, file_extension, nesting_state, error) - CheckLanguage(filename, clean_lines, line, file_extension, include_state, - nesting_state, error) - CheckForNonConstReference(filename, clean_lines, line, nesting_state, error) - CheckForNonStandardConstructs(filename, clean_lines, line, - nesting_state, error) - CheckVlogArguments(filename, clean_lines, line, error) - CheckPosixThreading(filename, clean_lines, line, error) - CheckInvalidIncrement(filename, clean_lines, line, error) - CheckMakePairUsesDeduction(filename, clean_lines, line, error) - for check_fn in extra_check_functions: - check_fn(filename, clean_lines, line, error) - -def ProcessFileData(filename, file_extension, lines, error, - extra_check_functions=[]): - """Performs lint checks and reports any errors to the given error function. - - Args: - filename: Filename of the file that is being processed. - file_extension: The extension (dot not included) of the file. - lines: An array of strings, each representing a line of the file, with the - last element being empty if the file is terminated with a newline. - error: A callable to which errors are reported, which takes 4 arguments: - filename, line number, error level, and message - extra_check_functions: An array of additional check functions that will be - run on each source line. Each function takes 4 - arguments: filename, clean_lines, line, error - """ - lines = (['// marker so line numbers and indices both start at 1'] + lines + - ['// marker so line numbers end in a known way']) - - include_state = _IncludeState() - function_state = _FunctionState() - nesting_state = _NestingState() - - ResetNolintSuppressions() - - CheckForCopyright(filename, lines, error) - - if file_extension == 'h': - CheckForHeaderGuard(filename, lines, error) - - RemoveMultiLineComments(filename, lines, error) - clean_lines = CleansedLines(lines) - for line in xrange(clean_lines.NumLines()): - ProcessLine(filename, file_extension, clean_lines, line, - include_state, function_state, nesting_state, error, - extra_check_functions) - nesting_state.CheckCompletedBlocks(filename, error) - - CheckForIncludeWhatYouUse(filename, clean_lines, include_state, error) - - # We check here rather than inside ProcessLine so that we see raw - # lines rather than "cleaned" lines. - CheckForBadCharacters(filename, lines, error) - - CheckForNewlineAtEOF(filename, lines, error) - -def ProcessFile(filename, vlevel, extra_check_functions=[]): - """Does google-lint on a single file. - - Args: - filename: The name of the file to parse. - - vlevel: The level of errors to report. Every error of confidence - >= verbose_level will be reported. 0 is a good default. - - extra_check_functions: An array of additional check functions that will be - run on each source line. Each function takes 4 - arguments: filename, clean_lines, line, error - """ - - _SetVerboseLevel(vlevel) - - try: - # Support the UNIX convention of using "-" for stdin. Note that - # we are not opening the file with universal newline support - # (which codecs doesn't support anyway), so the resulting lines do - # contain trailing '\r' characters if we are reading a file that - # has CRLF endings. - # If after the split a trailing '\r' is present, it is removed - # below. If it is not expected to be present (i.e. os.linesep != - # '\r\n' as in Windows), a warning is issued below if this file - # is processed. - - if filename == '-': - lines = codecs.StreamReaderWriter(sys.stdin, - codecs.getreader('utf8'), - codecs.getwriter('utf8'), - 'replace').read().split('\n') - else: - lines = codecs.open(filename, 'r', 'utf8', 'replace').read().split('\n') - - carriage_return_found = False - # Remove trailing '\r'. - for linenum in range(len(lines)): - if lines[linenum].endswith('\r'): - lines[linenum] = lines[linenum].rstrip('\r') - carriage_return_found = True - - except IOError: - sys.stderr.write( - "Skipping input '%s': Can't open for reading\n" % filename) - return - - # Note, if no dot is found, this will give the entire filename as the ext. - file_extension = filename[filename.rfind('.') + 1:] - - # When reading from stdin, the extension is unknown, so no cpplint tests - # should rely on the extension. - if filename != '-' and file_extension not in _valid_extensions: - sys.stderr.write('Ignoring %s; not a valid file name ' - '(%s)\n' % (filename, ', '.join(_valid_extensions))) - else: - ProcessFileData(filename, file_extension, lines, Error, - extra_check_functions) - if carriage_return_found and os.linesep != '\r\n': - # Use 0 for linenum since outputting only one error for potentially - # several lines. - Error(filename, 0, 'whitespace/newline', 1, - 'One or more unexpected \\r (^M) found;' - 'better to use only a \\n') - - sys.stderr.write('Done processing %s\n' % filename) - - -def PrintUsage(message): - """Prints a brief usage string and exits, optionally with an error message. - - Args: - message: The optional error message. - """ - sys.stderr.write(_USAGE) - if message: - sys.exit('\nFATAL ERROR: ' + message) - else: - sys.exit(1) - - -def PrintCategories(): - """Prints a list of all the error-categories used by error messages. - - These are the categories used to filter messages via --filter. - """ - sys.stderr.write(''.join(' %s\n' % cat for cat in _ERROR_CATEGORIES)) - sys.exit(0) - - -def ParseArguments(args): - """Parses the command line arguments. - - This may set the output format and verbosity level as side-effects. - - Args: - args: The command line arguments: - - Returns: - The list of filenames to lint. - """ - try: - (opts, filenames) = getopt.getopt(args, '', ['help', 'output=', 'verbose=', - 'counting=', - 'filter=', - 'root=', - 'linelength=', - 'extensions=']) - except getopt.GetoptError: - PrintUsage('Invalid arguments.') - - verbosity = _VerboseLevel() - output_format = _OutputFormat() - filters = '' - counting_style = '' - - for (opt, val) in opts: - if opt == '--help': - PrintUsage(None) - elif opt == '--output': - if val not in ('emacs', 'vs7', 'eclipse'): - PrintUsage('The only allowed output formats are emacs, vs7 and eclipse.') - output_format = val - elif opt == '--verbose': - verbosity = int(val) - elif opt == '--filter': - filters = val - if not filters: - PrintCategories() - elif opt == '--counting': - if val not in ('total', 'toplevel', 'detailed'): - PrintUsage('Valid counting options are total, toplevel, and detailed') - counting_style = val - elif opt == '--root': - global _root - _root = val - elif opt == '--linelength': - global _line_length - try: - _line_length = int(val) - except ValueError: - PrintUsage('Line length must be digits.') - elif opt == '--extensions': - global _valid_extensions - try: - _valid_extensions = set(val.split(',')) - except ValueError: - PrintUsage('Extensions must be comma separated list.') - - if not filenames: - PrintUsage('No files were specified.') - - _SetOutputFormat(output_format) - _SetVerboseLevel(verbosity) - _SetFilters(filters) - _SetCountingStyle(counting_style) - - return filenames - - -def main(): - filenames = ParseArguments(sys.argv[1:]) - - # Change stderr to write with replacement characters so we don't die - # if we try to print something containing non-ASCII characters. - sys.stderr = codecs.StreamReaderWriter(sys.stderr, - codecs.getreader('utf8'), - codecs.getwriter('utf8'), - 'replace') - - _cpplint_state.ResetErrorCounts() - for filename in filenames: - ProcessFile(filename, _cpplint_state.verbose_level) - _cpplint_state.PrintErrorCounts() - - sys.exit(_cpplint_state.error_count > 0) - - -if __name__ == '__main__': - main() diff --git a/linters/lint_engine/FacebookFbcodeLintEngine.php b/linters/lint_engine/FacebookFbcodeLintEngine.php deleted file mode 100644 index cb9cf9bdba4..00000000000 --- a/linters/lint_engine/FacebookFbcodeLintEngine.php +++ /dev/null @@ -1,147 +0,0 @@ -getPaths(); - - // Remove all deleted files, which are not checked by the - // following linters. - foreach ($paths as $key => $path) { - if (!Filesystem::pathExists($this->getFilePathOnDisk($path))) { - unset($paths[$key]); - } - } - - $generated_linter = new ArcanistGeneratedLinter(); - $linters[] = $generated_linter; - - $nolint_linter = new ArcanistNoLintLinter(); - $linters[] = $nolint_linter; - - $text_linter = new ArcanistTextLinter(); - $text_linter->setCustomSeverityMap(array( - ArcanistTextLinter::LINT_LINE_WRAP - => ArcanistLintSeverity::SEVERITY_ADVICE, - )); - $linters[] = $text_linter; - - $java_text_linter = new ArcanistTextLinter(); - $java_text_linter->setMaxLineLength(100); - $java_text_linter->setCustomSeverityMap(array( - ArcanistTextLinter::LINT_LINE_WRAP - => ArcanistLintSeverity::SEVERITY_ADVICE, - )); - $linters[] = $java_text_linter; - - $pep8_options = $this->getPEP8WithTextOptions().',E302'; - - $python_linter = new ArcanistPEP8Linter(); - $python_linter->setConfig(array('options' => $pep8_options)); - $linters[] = $python_linter; - - $python_2space_linter = new ArcanistPEP8Linter(); - $python_2space_linter->setConfig(array('options' => $pep8_options.',E111')); - $linters[] = $python_2space_linter; - - // Currently we can't run cpplint in commit hook mode, because it - // depends on having access to the working directory. - if (!$this->getCommitHookMode()) { - $cpp_linters = array(); - $google_linter = new ArcanistCpplintLinter(); - $google_linter->setConfig(array( - 'lint.cpplint.prefix' => '', - 'lint.cpplint.bin' => 'cpplint', - )); - $cpp_linters[] = $linters[] = $google_linter; - $cpp_linters[] = $linters[] = new FbcodeCppLinter(); - $cpp_linters[] = $linters[] = new PfffCppLinter(); - } - - $spelling_linter = new ArcanistSpellingLinter(); - $linters[] = $spelling_linter; - - foreach ($paths as $path) { - $is_text = false; - - $text_extensions = ( - '/\.('. - 'cpp|cxx|c|cc|h|hpp|hxx|tcc|'. - 'py|rb|hs|pl|pm|tw|'. - 'php|phpt|css|js|'. - 'java|'. - 'thrift|'. - 'lua|'. - 'siv|'. - 'txt'. - ')$/' - ); - if (preg_match($text_extensions, $path)) { - $is_text = true; - } - if ($is_text) { - $nolint_linter->addPath($path); - - $generated_linter->addPath($path); - $generated_linter->addData($path, $this->loadData($path)); - - if (preg_match('/\.java$/', $path)) { - $java_text_linter->addPath($path); - $java_text_linter->addData($path, $this->loadData($path)); - } else { - $text_linter->addPath($path); - $text_linter->addData($path, $this->loadData($path)); - } - - $spelling_linter->addPath($path); - $spelling_linter->addData($path, $this->loadData($path)); - } - if (preg_match('/\.(cpp|c|cc|cxx|h|hh|hpp|hxx|tcc)$/', $path)) { - foreach ($cpp_linters as &$linter) { - $linter->addPath($path); - $linter->addData($path, $this->loadData($path)); - } - } - - // Match *.py and contbuild config files - if (preg_match('/(\.(py|tw|smcprops)|^contbuild\/configs\/[^\/]*)$/', - $path)) { - $space_count = 4; - $real_path = $this->getFilePathOnDisk($path); - $dir = dirname($real_path); - do { - if (file_exists($dir.'/.python2space')) { - $space_count = 2; - break; - } - $dir = dirname($dir); - } while ($dir != '/' && $dir != '.'); - - if ($space_count == 4) { - $cur_path_linter = $python_linter; - } else { - $cur_path_linter = $python_2space_linter; - } - $cur_path_linter->addPath($path); - $cur_path_linter->addData($path, $this->loadData($path)); - - if (preg_match('/\.tw$/', $path)) { - $cur_path_linter->setCustomSeverityMap(array( - 'E251' => ArcanistLintSeverity::SEVERITY_DISABLED, - )); - } - } - } - - $name_linter = new ArcanistFilenameLinter(); - $linters[] = $name_linter; - foreach ($paths as $path) { - $name_linter->addPath($path); - } - - return $linters; - } - -} diff --git a/memtable/alloc_tracker.cc b/memtable/alloc_tracker.cc new file mode 100644 index 00000000000..9889cc4230c --- /dev/null +++ b/memtable/alloc_tracker.cc @@ -0,0 +1,59 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include +#include "rocksdb/write_buffer_manager.h" +#include "util/allocator.h" +#include "util/arena.h" + +namespace rocksdb { + +AllocTracker::AllocTracker(WriteBufferManager* write_buffer_manager) + : write_buffer_manager_(write_buffer_manager), + bytes_allocated_(0), + done_allocating_(false), + freed_(false) {} + +AllocTracker::~AllocTracker() { FreeMem(); } + +void AllocTracker::Allocate(size_t bytes) { + assert(write_buffer_manager_ != nullptr); + if (write_buffer_manager_->enabled()) { + bytes_allocated_.fetch_add(bytes, std::memory_order_relaxed); + write_buffer_manager_->ReserveMem(bytes); + } +} + +void AllocTracker::DoneAllocating() { + if (write_buffer_manager_ != nullptr && !done_allocating_) { + if (write_buffer_manager_->enabled()) { + write_buffer_manager_->ScheduleFreeMem( + bytes_allocated_.load(std::memory_order_relaxed)); + } else { + assert(bytes_allocated_.load(std::memory_order_relaxed) == 0); + } + done_allocating_ = true; + } +} + +void AllocTracker::FreeMem() { + if (!done_allocating_) { + DoneAllocating(); + } + if (write_buffer_manager_ != nullptr && !freed_) { + if (write_buffer_manager_->enabled()) { + write_buffer_manager_->FreeMem( + bytes_allocated_.load(std::memory_order_relaxed)); + } else { + assert(bytes_allocated_.load(std::memory_order_relaxed) == 0); + } + freed_ = true; + } +} +} // namespace rocksdb diff --git a/util/hash_cuckoo_rep.cc b/memtable/hash_cuckoo_rep.cc similarity index 88% rename from util/hash_cuckoo_rep.cc rename to memtable/hash_cuckoo_rep.cc index a9a79a27428..034bf5858b6 100644 --- a/util/hash_cuckoo_rep.cc +++ b/memtable/hash_cuckoo_rep.cc @@ -1,25 +1,26 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #ifndef ROCKSDB_LITE -#include "util/hash_cuckoo_rep.h" +#include "memtable/hash_cuckoo_rep.h" #include #include #include +#include #include #include -#include #include +#include "db/memtable.h" +#include "memtable/skiplist.h" +#include "memtable/stl_wrappers.h" +#include "port/port.h" #include "rocksdb/memtablerep.h" #include "util/murmurhash.h" -#include "db/memtable.h" -#include "db/skiplist.h" -#include "util/stl_wrappers.h" namespace rocksdb { namespace { @@ -39,8 +40,14 @@ struct CuckooStep { CuckooStep() : bucket_id_(-1), prev_step_id_(kNullStep), depth_(1) {} - CuckooStep(CuckooStep&&) = default; - CuckooStep& operator=(CuckooStep&&) = default; + CuckooStep(CuckooStep&& o) = default; + + CuckooStep& operator=(CuckooStep&& rhs) { + bucket_id_ = std::move(rhs.bucket_id_); + prev_step_id_ = std::move(rhs.prev_step_id_); + depth_ = std::move(rhs.depth_); + return *this; + } CuckooStep(const CuckooStep&) = delete; CuckooStep& operator=(const CuckooStep&) = delete; @@ -52,25 +59,27 @@ struct CuckooStep { class HashCuckooRep : public MemTableRep { public: explicit HashCuckooRep(const MemTableRep::KeyComparator& compare, - Arena* arena, const size_t bucket_count, - const unsigned int hash_func_count) - : MemTableRep(arena), + Allocator* allocator, const size_t bucket_count, + const unsigned int hash_func_count, + const size_t approximate_entry_size) + : MemTableRep(allocator), compare_(compare), - arena_(arena), + allocator_(allocator), bucket_count_(bucket_count), + approximate_entry_size_(approximate_entry_size), cuckoo_path_max_depth_(kDefaultCuckooPathMaxDepth), occupied_count_(0), hash_function_count_(hash_func_count), backup_table_(nullptr) { char* mem = reinterpret_cast( - arena_->Allocate(sizeof(std::atomic) * bucket_count_)); - cuckoo_array_ = new (mem) std::atomic[bucket_count_]; + allocator_->Allocate(sizeof(std::atomic) * bucket_count_)); + cuckoo_array_ = new (mem) std::atomic[bucket_count_]; for (unsigned int bid = 0; bid < bucket_count_; ++bid) { cuckoo_array_[bid].store(nullptr, std::memory_order_relaxed); } cuckoo_path_ = reinterpret_cast( - arena_->Allocate(sizeof(int*) * (cuckoo_path_max_depth_ + 1))); + allocator_->Allocate(sizeof(int) * (cuckoo_path_max_depth_ + 1))); is_nearly_full_ = false; } @@ -90,15 +99,15 @@ class HashCuckooRep : public MemTableRep { // the current mem-table already contains the specified key. virtual void Insert(KeyHandle handle) override; - // This function returns std::numeric_limits::max() in the following - // three cases to disallow further write operations: + // This function returns bucket_count_ * approximate_entry_size_ when any + // of the followings happen to disallow further write operations: // 1. when the fullness reaches kMaxFullnes. // 2. when the backup_table_ is used. // // otherwise, this function will always return 0. virtual size_t ApproximateMemoryUsage() override { if (is_nearly_full_) { - return std::numeric_limits::max(); + return bucket_count_ * approximate_entry_size_; } return 0; } @@ -109,7 +118,7 @@ class HashCuckooRep : public MemTableRep { class Iterator : public MemTableRep::Iterator { std::shared_ptr> bucket_; - typename std::vector::const_iterator mutable cit_; + std::vector::const_iterator mutable cit_; const KeyComparator& compare_; std::string tmp_; // For passing to EncodeKey bool mutable sorted_; @@ -142,6 +151,10 @@ class HashCuckooRep : public MemTableRep { // Advance to the first entry with a key >= target virtual void Seek(const Slice& user_key, const char* memtable_key) override; + // Retreat to the last entry with a key <= target + virtual void SeekForPrev(const Slice& user_key, + const char* memtable_key) override; + // Position at the first entry in collection. // Final state of iterator is Valid() iff collection is not empty. virtual void SeekToFirst() override; @@ -181,10 +194,12 @@ class HashCuckooRep : public MemTableRep { private: const MemTableRep::KeyComparator& compare_; - // the pointer to Arena to allocate memory, immutable after construction. - Arena* const arena_; + // the pointer to Allocator to allocate memory, immutable after construction. + Allocator* const allocator_; // the number of hash bucket in the hash table. const size_t bucket_count_; + // approximate size of each entry + const size_t approximate_entry_size_; // the maxinum depth of the cuckoo path. const unsigned int cuckoo_path_max_depth_; // the current number of entries in cuckoo_array_ which has been occupied. @@ -195,7 +210,7 @@ class HashCuckooRep : public MemTableRep { // a vacant bucket for inserting the key of a put request. std::shared_ptr backup_table_; // the array to store pointers, pointing to the actual data. - std::atomic* cuckoo_array_; + std::atomic* cuckoo_array_; // a buffer to store cuckoo path int* cuckoo_path_; // a boolean flag indicating whether the fullness of bucket array @@ -213,9 +228,10 @@ class HashCuckooRep : public MemTableRep { static const int kMurmurHashSeeds[HashCuckooRepFactory::kMaxHashCount] = { 545609244, 1769731426, 763324157, 13099088, 592422103, 1899789565, 248369300, 1984183468, 1613664382, 1491157517}; - return MurmurHash(slice.data(), slice.size(), - kMurmurHashSeeds[hash_func_id]) % - bucket_count_; + return static_cast( + MurmurHash(slice.data(), static_cast(slice.size()), + kMurmurHashSeeds[hash_func_id]) % + bucket_count_); } // A cuckoo path is a sequence of bucket ids, where each id points to a @@ -285,8 +301,8 @@ void HashCuckooRep::Get(const LookupKey& key, void* callback_args, const char* bucket = cuckoo_array_[GetHash(user_key, hid)].load(std::memory_order_acquire); if (bucket != nullptr) { - auto bucket_user_key = UserKey(bucket); - if (user_key.compare(bucket_user_key) == 0) { + Slice bucket_user_key = UserKey(bucket); + if (user_key == bucket_user_key) { callback_func(callback_args, bucket); break; } @@ -304,7 +320,7 @@ void HashCuckooRep::Get(const LookupKey& key, void* callback_args, } void HashCuckooRep::Insert(KeyHandle handle) { - static const float kMaxFullness = 0.90; + static const float kMaxFullness = 0.90f; auto* key = static_cast(handle); int initial_hash_id = 0; @@ -320,7 +336,7 @@ void HashCuckooRep::Insert(KeyHandle handle) { if (backup_table_.get() == nullptr) { VectorRepFactory factory(10); backup_table_.reset( - factory.CreateMemTableRep(compare_, arena_, nullptr, nullptr)); + factory.CreateMemTableRep(compare_, allocator_, nullptr, nullptr)); is_nearly_full_ = true; } backup_table_->Insert(key); @@ -398,8 +414,8 @@ bool HashCuckooRep::QuickInsert(const char* internal_key, const Slice& user_key, } if (cuckoo_bucket_id != -1) { - cuckoo_array_[cuckoo_bucket_id] - .store(internal_key, std::memory_order_release); + cuckoo_array_[cuckoo_bucket_id].store(const_cast(internal_key), + std::memory_order_release); return true; } @@ -452,10 +468,10 @@ bool HashCuckooRep::FindCuckooPath(const char* internal_key, } // again, we can perform no barrier load safely here as the current // thread is the only writer. - auto bucket_user_key = + Slice bucket_user_key = UserKey(cuckoo_array_[step.bucket_id_].load(std::memory_order_relaxed)); if (step.prev_step_id_ != CuckooStep::kNullStep) { - if (bucket_user_key.compare(user_key) == 0) { + if (bucket_user_key == user_key) { // then there is a loop in the current path, stop discovering this path. continue; } @@ -580,6 +596,12 @@ void HashCuckooRep::Iterator::Seek(const Slice& user_key, }).first; } +// Retreat to the last entry with a key <= target +void HashCuckooRep::Iterator::SeekForPrev(const Slice& user_key, + const char* memtable_key) { + assert(false); +} + // Position at the first entry in collection. // Final state of iterator is Valid() iff collection is not empty. void HashCuckooRep::Iterator::SeekToFirst() { @@ -600,18 +622,19 @@ void HashCuckooRep::Iterator::SeekToLast() { } // anom namespace MemTableRep* HashCuckooRepFactory::CreateMemTableRep( - const MemTableRep::KeyComparator& compare, Arena* arena, + const MemTableRep::KeyComparator& compare, Allocator* allocator, const SliceTransform* transform, Logger* logger) { // The estimated average fullness. The write performance of any close hash // degrades as the fullness of the mem-table increases. Setting kFullness // to a value around 0.7 can better avoid write performance degradation while // keeping efficient memory usage. - static const float kFullness = 0.7; + static const float kFullness = 0.7f; size_t pointer_size = sizeof(std::atomic); assert(write_buffer_size_ >= (average_data_size_ + pointer_size)); size_t bucket_count = + static_cast( (write_buffer_size_ / (average_data_size_ + pointer_size)) / kFullness + - 1; + 1); unsigned int hash_function_count = hash_function_count_; if (hash_function_count < 2) { hash_function_count = 2; @@ -619,7 +642,11 @@ MemTableRep* HashCuckooRepFactory::CreateMemTableRep( if (hash_function_count > kMaxHashCount) { hash_function_count = kMaxHashCount; } - return new HashCuckooRep(compare, arena, bucket_count, hash_function_count); + return new HashCuckooRep(compare, allocator, bucket_count, + hash_function_count, + static_cast( + (average_data_size_ + pointer_size) / kFullness) + ); } MemTableRepFactory* NewHashCuckooRepFactory(size_t write_buffer_size, diff --git a/util/hash_cuckoo_rep.h b/memtable/hash_cuckoo_rep.h similarity index 75% rename from util/hash_cuckoo_rep.h rename to memtable/hash_cuckoo_rep.h index 669b6b7d42d..800696e931a 100644 --- a/util/hash_cuckoo_rep.h +++ b/memtable/hash_cuckoo_rep.h @@ -1,13 +1,14 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. -#ifndef ROCKSDB_LITE #pragma once +#ifndef ROCKSDB_LITE +#include "port/port.h" #include "rocksdb/slice_transform.h" #include "rocksdb/memtablerep.h" @@ -27,8 +28,9 @@ class HashCuckooRepFactory : public MemTableRepFactory { virtual ~HashCuckooRepFactory() {} + using MemTableRepFactory::CreateMemTableRep; virtual MemTableRep* CreateMemTableRep( - const MemTableRep::KeyComparator& compare, Arena* arena, + const MemTableRep::KeyComparator& compare, Allocator* allocator, const SliceTransform* transform, Logger* logger) override; virtual const char* Name() const override { return "HashCuckooRepFactory"; } diff --git a/util/hash_linklist_rep.cc b/memtable/hash_linklist_rep.cc similarity index 74% rename from util/hash_linklist_rep.cc rename to memtable/hash_linklist_rep.cc index 8e3dc58268f..932b62a3460 100644 --- a/util/hash_linklist_rep.cc +++ b/memtable/hash_linklist_rep.cc @@ -1,40 +1,53 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #ifndef ROCKSDB_LITE -#include "util/hash_linklist_rep.h" +#include "memtable/hash_linklist_rep.h" #include +#include +#include "db/memtable.h" +#include "memtable/skiplist.h" +#include "monitoring/histogram.h" +#include "port/port.h" #include "rocksdb/memtablerep.h" -#include "util/arena.h" #include "rocksdb/slice.h" #include "rocksdb/slice_transform.h" -#include "port/port.h" -#include "port/atomic_pointer.h" -#include "util/histogram.h" +#include "util/arena.h" #include "util/murmurhash.h" -#include "db/memtable.h" -#include "db/skiplist.h" namespace rocksdb { namespace { typedef const char* Key; typedef SkipList MemtableSkipList; -typedef port::AtomicPointer Pointer; +typedef std::atomic Pointer; // A data structure used as the header of a link list of a hash bucket. struct BucketHeader { Pointer next; - uint32_t num_entries; + std::atomic num_entries; explicit BucketHeader(void* n, uint32_t count) : next(n), num_entries(count) {} - bool IsSkipListBucket() { return next.NoBarrier_Load() == this; } + bool IsSkipListBucket() { + return next.load(std::memory_order_relaxed) == this; + } + + uint32_t GetNumEntries() const { + return num_entries.load(std::memory_order_relaxed); + } + + // REQUIRES: called from single-threaded Insert() + void IncNumEntries() { + // Only one thread can do write at one time. No need to do atomic + // incremental. Update it with relaxed load and store. + num_entries.store(GetNumEntries() + 1, std::memory_order_relaxed); + } }; // A data structure used as the header of a skip list of a hash bucket. @@ -43,10 +56,10 @@ struct SkipListBucketHeader { MemtableSkipList skip_list; explicit SkipListBucketHeader(const MemTableRep::KeyComparator& cmp, - Arena* arena, uint32_t count) + Allocator* allocator, uint32_t count) : Counting_header(this, // Pointing to itself to indicate header type. count), - skip_list(cmp, arena) {} + skip_list(cmp, allocator) {} }; struct Node { @@ -55,26 +68,32 @@ struct Node { Node* Next() { // Use an 'acquire load' so that we observe a fully initialized // version of the returned Node. - return reinterpret_cast(next_.Acquire_Load()); + return next_.load(std::memory_order_acquire); } void SetNext(Node* x) { // Use a 'release store' so that anybody who reads through this // pointer observes a fully initialized version of the inserted node. - next_.Release_Store(x); + next_.store(x, std::memory_order_release); } // No-barrier variants that can be safely used in a few locations. Node* NoBarrier_Next() { - return reinterpret_cast(next_.NoBarrier_Load()); + return next_.load(std::memory_order_relaxed); } - void NoBarrier_SetNext(Node* x) { - next_.NoBarrier_Store(x); - } + void NoBarrier_SetNext(Node* x) { next_.store(x, std::memory_order_relaxed); } + + // Needed for placement new below which is fine + Node() {} private: - port::AtomicPointer next_; + std::atomic next_; + + // Prohibit copying due to the below + Node(const Node&) = delete; + Node& operator=(const Node&) = delete; + public: - char key[0]; + char key[1]; }; // Memory structure of the mem table: @@ -142,10 +161,11 @@ struct Node { // which can be significant decrease of memory utilization. class HashLinkListRep : public MemTableRep { public: - HashLinkListRep(const MemTableRep::KeyComparator& compare, Arena* arena, - const SliceTransform* transform, size_t bucket_size, - uint32_t threshold_use_skiplist, size_t huge_page_tlb_size, - Logger* logger, int bucket_entries_logging_threshold, + HashLinkListRep(const MemTableRep::KeyComparator& compare, + Allocator* allocator, const SliceTransform* transform, + size_t bucket_size, uint32_t threshold_use_skiplist, + size_t huge_page_tlb_size, Logger* logger, + int bucket_entries_logging_threshold, bool if_log_bucket_dist_when_flash); virtual KeyHandle Allocate(const size_t len, char** buf) override; @@ -165,7 +185,7 @@ class HashLinkListRep : public MemTableRep { virtual MemTableRep::Iterator* GetIterator(Arena* arena = nullptr) override; virtual MemTableRep::Iterator* GetDynamicPrefixIterator( - Arena* arena = nullptr) override; + Arena* arena = nullptr) override; private: friend class DynamicIterator; @@ -174,7 +194,7 @@ class HashLinkListRep : public MemTableRep { // Maps slices (which are transformed user keys) to buckets of keys sharing // the same transform. - port::AtomicPointer* buckets_; + Pointer* buckets_; const uint32_t threshold_use_skiplist_; @@ -199,11 +219,12 @@ class HashLinkListRep : public MemTableRep { } size_t GetHash(const Slice& slice) const { - return MurmurHash(slice.data(), slice.size(), 0) % bucket_size_; + return MurmurHash(slice.data(), static_cast(slice.size()), 0) % + bucket_size_; } Pointer* GetBucket(size_t i) const { - return static_cast(buckets_[i].Acquire_Load()); + return static_cast(buckets_[i].load(std::memory_order_acquire)); } Pointer* GetBucket(const Slice& slice) const { @@ -226,67 +247,81 @@ class HashLinkListRep : public MemTableRep { return (n != nullptr) && (compare_(n->key, key) < 0); } + bool KeyIsAfterOrAtNode(const Slice& internal_key, const Node* n) const { + // nullptr n is considered infinite + return (n != nullptr) && (compare_(n->key, internal_key) <= 0); + } + + bool KeyIsAfterOrAtNode(const Key& key, const Node* n) const { + // nullptr n is considered infinite + return (n != nullptr) && (compare_(n->key, key) <= 0); + } Node* FindGreaterOrEqualInBucket(Node* head, const Slice& key) const; + Node* FindLessOrEqualInBucket(Node* head, const Slice& key) const; class FullListIterator : public MemTableRep::Iterator { public: - explicit FullListIterator(MemtableSkipList* list, Arena* arena) - : iter_(list), full_list_(list), arena_(arena) {} + explicit FullListIterator(MemtableSkipList* list, Allocator* allocator) + : iter_(list), full_list_(list), allocator_(allocator) {} virtual ~FullListIterator() { } // Returns true iff the iterator is positioned at a valid node. - virtual bool Valid() const { - return iter_.Valid(); - } + virtual bool Valid() const override { return iter_.Valid(); } // Returns the key at the current position. // REQUIRES: Valid() - virtual const char* key() const { + virtual const char* key() const override { assert(Valid()); return iter_.key(); } // Advances to the next position. // REQUIRES: Valid() - virtual void Next() { + virtual void Next() override { assert(Valid()); iter_.Next(); } // Advances to the previous position. // REQUIRES: Valid() - virtual void Prev() { + virtual void Prev() override { assert(Valid()); iter_.Prev(); } // Advance to the first entry with a key >= target - virtual void Seek(const Slice& internal_key, const char* memtable_key) { + virtual void Seek(const Slice& internal_key, + const char* memtable_key) override { const char* encoded_key = (memtable_key != nullptr) ? memtable_key : EncodeKey(&tmp_, internal_key); iter_.Seek(encoded_key); } + // Retreat to the last entry with a key <= target + virtual void SeekForPrev(const Slice& internal_key, + const char* memtable_key) override { + const char* encoded_key = (memtable_key != nullptr) + ? memtable_key + : EncodeKey(&tmp_, internal_key); + iter_.SeekForPrev(encoded_key); + } + // Position at the first entry in collection. // Final state of iterator is Valid() iff collection is not empty. - virtual void SeekToFirst() { - iter_.SeekToFirst(); - } + virtual void SeekToFirst() override { iter_.SeekToFirst(); } // Position at the last entry in collection. // Final state of iterator is Valid() iff collection is not empty. - virtual void SeekToLast() { - iter_.SeekToLast(); - } + virtual void SeekToLast() override { iter_.SeekToLast(); } private: MemtableSkipList::Iterator iter_; // To destruct with the iterator. std::unique_ptr full_list_; - std::unique_ptr arena_; + std::unique_ptr allocator_; std::string tmp_; // For passing to EncodeKey }; @@ -301,41 +336,48 @@ class HashLinkListRep : public MemTableRep { virtual ~LinkListIterator() {} // Returns true iff the iterator is positioned at a valid node. - virtual bool Valid() const { - return node_ != nullptr; - } + virtual bool Valid() const override { return node_ != nullptr; } // Returns the key at the current position. // REQUIRES: Valid() - virtual const char* key() const { + virtual const char* key() const override { assert(Valid()); return node_->key; } // Advances to the next position. // REQUIRES: Valid() - virtual void Next() { + virtual void Next() override { assert(Valid()); node_ = node_->Next(); } // Advances to the previous position. // REQUIRES: Valid() - virtual void Prev() { + virtual void Prev() override { // Prefix iterator does not support total order. // We simply set the iterator to invalid state Reset(nullptr); } // Advance to the first entry with a key >= target - virtual void Seek(const Slice& internal_key, const char* memtable_key) { + virtual void Seek(const Slice& internal_key, + const char* memtable_key) override { node_ = hash_link_list_rep_->FindGreaterOrEqualInBucket(head_, internal_key); } + // Retreat to the last entry with a key <= target + virtual void SeekForPrev(const Slice& internal_key, + const char* memtable_key) override { + // Since we do not support Prev() + // We simply do not support SeekForPrev + Reset(nullptr); + } + // Position at the first entry in collection. // Final state of iterator is Valid() iff collection is not empty. - virtual void SeekToFirst() { + virtual void SeekToFirst() override { // Prefix iterator does not support total order. // We simply set the iterator to invalid state Reset(nullptr); @@ -343,7 +385,7 @@ class HashLinkListRep : public MemTableRep { // Position at the last entry in collection. // Final state of iterator is Valid() iff collection is not empty. - virtual void SeekToLast() { + virtual void SeekToLast() override { // Prefix iterator does not support total order. // We simply set the iterator to invalid state Reset(nullptr); @@ -372,7 +414,7 @@ class HashLinkListRep : public MemTableRep { memtable_rep_(memtable_rep) {} // Advance to the first entry with a key >= target - virtual void Seek(const Slice& k, const char* memtable_key) { + virtual void Seek(const Slice& k, const char* memtable_key) override { auto transformed = memtable_rep_.GetPrefix(k); auto* bucket = memtable_rep_.GetBucket(transformed); @@ -391,7 +433,7 @@ class HashLinkListRep : public MemTableRep { } else { IterKey encoded_key; encoded_key.EncodeLengthPrefixedKey(k); - skip_list_iter_->Seek(encoded_key.GetKey().data()); + skip_list_iter_->Seek(encoded_key.GetUserKey().data()); } } else { // The bucket is organized as a linked list @@ -401,21 +443,21 @@ class HashLinkListRep : public MemTableRep { } } - virtual bool Valid() const { + virtual bool Valid() const override { if (skip_list_iter_) { return skip_list_iter_->Valid(); } return HashLinkListRep::LinkListIterator::Valid(); } - virtual const char* key() const { + virtual const char* key() const override { if (skip_list_iter_) { return skip_list_iter_->key(); } return HashLinkListRep::LinkListIterator::key(); } - virtual void Next() { + virtual void Next() override { if (skip_list_iter_) { skip_list_iter_->Next(); } else { @@ -434,30 +476,30 @@ class HashLinkListRep : public MemTableRep { // instantiating an empty bucket over which to iterate. public: EmptyIterator() { } - virtual bool Valid() const { - return false; - } - virtual const char* key() const { + virtual bool Valid() const override { return false; } + virtual const char* key() const override { assert(false); return nullptr; } - virtual void Next() { } - virtual void Prev() { } - virtual void Seek(const Slice& user_key, const char* memtable_key) { } - virtual void SeekToFirst() { } - virtual void SeekToLast() { } + virtual void Next() override {} + virtual void Prev() override {} + virtual void Seek(const Slice& user_key, + const char* memtable_key) override {} + virtual void SeekForPrev(const Slice& user_key, + const char* memtable_key) override {} + virtual void SeekToFirst() override {} + virtual void SeekToLast() override {} + private: }; }; -HashLinkListRep::HashLinkListRep(const MemTableRep::KeyComparator& compare, - Arena* arena, const SliceTransform* transform, - size_t bucket_size, - uint32_t threshold_use_skiplist, - size_t huge_page_tlb_size, Logger* logger, - int bucket_entries_logging_threshold, - bool if_log_bucket_dist_when_flash) - : MemTableRep(arena), +HashLinkListRep::HashLinkListRep( + const MemTableRep::KeyComparator& compare, Allocator* allocator, + const SliceTransform* transform, size_t bucket_size, + uint32_t threshold_use_skiplist, size_t huge_page_tlb_size, Logger* logger, + int bucket_entries_logging_threshold, bool if_log_bucket_dist_when_flash) + : MemTableRep(allocator), bucket_size_(bucket_size), // Threshold to use skip list doesn't make sense if less than 3, so we // force it to be minimum of 3 to simplify implementation. @@ -467,13 +509,13 @@ HashLinkListRep::HashLinkListRep(const MemTableRep::KeyComparator& compare, logger_(logger), bucket_entries_logging_threshold_(bucket_entries_logging_threshold), if_log_bucket_dist_when_flash_(if_log_bucket_dist_when_flash) { - char* mem = arena_->AllocateAligned(sizeof(port::AtomicPointer) * bucket_size, + char* mem = allocator_->AllocateAligned(sizeof(Pointer) * bucket_size, huge_page_tlb_size, logger); - buckets_ = new (mem) port::AtomicPointer[bucket_size]; + buckets_ = new (mem) Pointer[bucket_size]; for (size_t i = 0; i < bucket_size_; ++i) { - buckets_[i].NoBarrier_Store(nullptr); + buckets_[i].store(nullptr, std::memory_order_relaxed); } } @@ -481,7 +523,7 @@ HashLinkListRep::~HashLinkListRep() { } KeyHandle HashLinkListRep::Allocate(const size_t len, char** buf) { - char* mem = arena_->AllocateAligned(sizeof(Node) + len); + char* mem = allocator_->AllocateAligned(sizeof(Node) + len); Node* x = new (mem) Node(); *buf = x->key; return static_cast(x); @@ -492,21 +534,21 @@ SkipListBucketHeader* HashLinkListRep::GetSkipListBucketHeader( if (first_next_pointer == nullptr) { return nullptr; } - if (first_next_pointer->NoBarrier_Load() == nullptr) { + if (first_next_pointer->load(std::memory_order_relaxed) == nullptr) { // Single entry bucket return nullptr; } // Counting header BucketHeader* header = reinterpret_cast(first_next_pointer); if (header->IsSkipListBucket()) { - assert(header->num_entries > threshold_use_skiplist_); + assert(header->GetNumEntries() > threshold_use_skiplist_); auto* skip_list_bucket_header = reinterpret_cast(header); - assert(skip_list_bucket_header->Counting_header.next.NoBarrier_Load() == - header); + assert(skip_list_bucket_header->Counting_header.next.load( + std::memory_order_relaxed) == header); return skip_list_bucket_header; } - assert(header->num_entries <= threshold_use_skiplist_); + assert(header->GetNumEntries() <= threshold_use_skiplist_); return nullptr; } @@ -514,17 +556,18 @@ Node* HashLinkListRep::GetLinkListFirstNode(Pointer* first_next_pointer) const { if (first_next_pointer == nullptr) { return nullptr; } - if (first_next_pointer->NoBarrier_Load() == nullptr) { + if (first_next_pointer->load(std::memory_order_relaxed) == nullptr) { // Single entry bucket return reinterpret_cast(first_next_pointer); } // Counting header BucketHeader* header = reinterpret_cast(first_next_pointer); if (!header->IsSkipListBucket()) { - assert(header->num_entries <= threshold_use_skiplist_); - return reinterpret_cast(header->next.NoBarrier_Load()); + assert(header->GetNumEntries() <= threshold_use_skiplist_); + return reinterpret_cast( + header->next.load(std::memory_order_acquire)); } - assert(header->num_entries > threshold_use_skiplist_); + assert(header->GetNumEntries() > threshold_use_skiplist_); return nullptr; } @@ -534,19 +577,20 @@ void HashLinkListRep::Insert(KeyHandle handle) { Slice internal_key = GetLengthPrefixedSlice(x->key); auto transformed = GetPrefix(internal_key); auto& bucket = buckets_[GetHash(transformed)]; - Pointer* first_next_pointer = static_cast(bucket.NoBarrier_Load()); + Pointer* first_next_pointer = + static_cast(bucket.load(std::memory_order_relaxed)); if (first_next_pointer == nullptr) { // Case 1. empty bucket // NoBarrier_SetNext() suffices since we will add a barrier when // we publish a pointer to "x" in prev[i]. x->NoBarrier_SetNext(nullptr); - bucket.Release_Store(x); + bucket.store(x, std::memory_order_release); return; } BucketHeader* header = nullptr; - if (first_next_pointer->NoBarrier_Load() == nullptr) { + if (first_next_pointer->load(std::memory_order_relaxed) == nullptr) { // Case 2. only one entry in the bucket // Need to convert to a Counting bucket and turn to case 4. Node* first = reinterpret_cast(first_next_pointer); @@ -555,40 +599,43 @@ void HashLinkListRep::Insert(KeyHandle handle) { // the new node. Otherwise, we might need to change next pointer of first. // In that case, a reader might sees the next pointer is NULL and wrongly // think the node is a bucket header. - auto* mem = arena_->AllocateAligned(sizeof(BucketHeader)); + auto* mem = allocator_->AllocateAligned(sizeof(BucketHeader)); header = new (mem) BucketHeader(first, 1); - bucket.Release_Store(header); + bucket.store(header, std::memory_order_release); } else { header = reinterpret_cast(first_next_pointer); if (header->IsSkipListBucket()) { // Case 4. Bucket is already a skip list - assert(header->num_entries > threshold_use_skiplist_); + assert(header->GetNumEntries() > threshold_use_skiplist_); auto* skip_list_bucket_header = reinterpret_cast(header); - skip_list_bucket_header->Counting_header.num_entries++; + // Only one thread can execute Insert() at one time. No need to do atomic + // incremental. + skip_list_bucket_header->Counting_header.IncNumEntries(); skip_list_bucket_header->skip_list.Insert(x->key); return; } } if (bucket_entries_logging_threshold_ > 0 && - header->num_entries == + header->GetNumEntries() == static_cast(bucket_entries_logging_threshold_)) { - Info(logger_, - "HashLinkedList bucket %zu has more than %d " - "entries. Key to insert: %s", - GetHash(transformed), header->num_entries, + Info(logger_, "HashLinkedList bucket %" ROCKSDB_PRIszt + " has more than %d " + "entries. Key to insert: %s", + GetHash(transformed), header->GetNumEntries(), GetLengthPrefixedSlice(x->key).ToString(true).c_str()); } - if (header->num_entries == threshold_use_skiplist_) { + if (header->GetNumEntries() == threshold_use_skiplist_) { // Case 3. number of entries reaches the threshold so need to convert to // skip list. LinkListIterator bucket_iter( - this, reinterpret_cast(first_next_pointer->NoBarrier_Load())); - auto mem = arena_->AllocateAligned(sizeof(SkipListBucketHeader)); + this, reinterpret_cast( + first_next_pointer->load(std::memory_order_relaxed))); + auto mem = allocator_->AllocateAligned(sizeof(SkipListBucketHeader)); SkipListBucketHeader* new_skip_list_header = new (mem) - SkipListBucketHeader(compare_, arena_, header->num_entries + 1); + SkipListBucketHeader(compare_, allocator_, header->GetNumEntries() + 1); auto& skip_list = new_skip_list_header->skip_list; // Add all current entries to the skip list @@ -599,16 +646,17 @@ void HashLinkListRep::Insert(KeyHandle handle) { // insert the new entry skip_list.Insert(x->key); // Set the bucket - bucket.Release_Store(new_skip_list_header); + bucket.store(new_skip_list_header, std::memory_order_release); } else { // Case 5. Need to insert to the sorted linked list without changing the // header. - Node* first = reinterpret_cast(header->next.NoBarrier_Load()); + Node* first = + reinterpret_cast(header->next.load(std::memory_order_relaxed)); assert(first != nullptr); // Advance counter unless the bucket needs to be advanced to skip list. // In that case, we need to make sure the previous count never exceeds // threshold_use_skiplist_ to avoid readers to cast to wrong format. - header->num_entries++; + header->IncNumEntries(); Node* cur = first; Node* prev = nullptr; @@ -640,7 +688,7 @@ void HashLinkListRep::Insert(KeyHandle handle) { if (prev) { prev->SetNext(x); } else { - header->next.Release_Store(static_cast(x)); + header->next.store(static_cast(x), std::memory_order_release); } } } @@ -663,7 +711,7 @@ bool HashLinkListRep::Contains(const char* key) const { } size_t HashLinkListRep::ApproximateMemoryUsage() { - // Memory is always allocated from the arena. + // Memory is always allocated from the allocator. return 0; } @@ -694,7 +742,7 @@ void HashLinkListRep::Get(const LookupKey& k, void* callback_args, MemTableRep::Iterator* HashLinkListRep::GetIterator(Arena* alloc_arena) { // allocate a new arena of similar size to the one currently in use - Arena* new_arena = new Arena(arena_->BlockSize()); + Arena* new_arena = new Arena(allocator_->BlockSize()); auto list = new MemtableSkipList(compare_, new_arena); HistogramImpl keys_per_bucket_hist; @@ -778,9 +826,9 @@ Node* HashLinkListRep::FindGreaterOrEqualInBucket(Node* head, } // anon namespace MemTableRep* HashLinkListRepFactory::CreateMemTableRep( - const MemTableRep::KeyComparator& compare, Arena* arena, + const MemTableRep::KeyComparator& compare, Allocator* allocator, const SliceTransform* transform, Logger* logger) { - return new HashLinkListRep(compare, arena, transform, bucket_count_, + return new HashLinkListRep(compare, allocator, transform, bucket_count_, threshold_use_skiplist_, huge_page_tlb_size_, logger, bucket_entries_logging_threshold_, if_log_bucket_dist_when_flash_); diff --git a/util/hash_linklist_rep.h b/memtable/hash_linklist_rep.h similarity index 79% rename from util/hash_linklist_rep.h rename to memtable/hash_linklist_rep.h index 0df35b54500..a6da3eedd51 100644 --- a/util/hash_linklist_rep.h +++ b/memtable/hash_linklist_rep.h @@ -1,13 +1,13 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. -#ifndef ROCKSDB_LITE #pragma once +#ifndef ROCKSDB_LITE #include "rocksdb/slice_transform.h" #include "rocksdb/memtablerep.h" @@ -28,8 +28,9 @@ class HashLinkListRepFactory : public MemTableRepFactory { virtual ~HashLinkListRepFactory() {} + using MemTableRepFactory::CreateMemTableRep; virtual MemTableRep* CreateMemTableRep( - const MemTableRep::KeyComparator& compare, Arena* arena, + const MemTableRep::KeyComparator& compare, Allocator* allocator, const SliceTransform* transform, Logger* logger) override; virtual const char* Name() const override { diff --git a/util/hash_skiplist_rep.cc b/memtable/hash_skiplist_rep.cc similarity index 76% rename from util/hash_skiplist_rep.cc rename to memtable/hash_skiplist_rep.cc index 1c7a459bd4d..e34743eb2c7 100644 --- a/util/hash_skiplist_rep.cc +++ b/memtable/hash_skiplist_rep.cc @@ -1,30 +1,32 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #ifndef ROCKSDB_LITE -#include "util/hash_skiplist_rep.h" +#include "memtable/hash_skiplist_rep.h" + +#include #include "rocksdb/memtablerep.h" #include "util/arena.h" #include "rocksdb/slice.h" #include "rocksdb/slice_transform.h" #include "port/port.h" -#include "port/atomic_pointer.h" #include "util/murmurhash.h" #include "db/memtable.h" -#include "db/skiplist.h" +#include "memtable/skiplist.h" namespace rocksdb { namespace { class HashSkipListRep : public MemTableRep { public: - HashSkipListRep(const MemTableRep::KeyComparator& compare, Arena* arena, - const SliceTransform* transform, size_t bucket_size, - int32_t skiplist_height, int32_t skiplist_branching_factor); + HashSkipListRep(const MemTableRep::KeyComparator& compare, + Allocator* allocator, const SliceTransform* transform, + size_t bucket_size, int32_t skiplist_height, + int32_t skiplist_branching_factor); virtual void Insert(KeyHandle handle) override; @@ -54,20 +56,21 @@ class HashSkipListRep : public MemTableRep { // Maps slices (which are transformed user keys) to buckets of keys sharing // the same transform. - port::AtomicPointer* buckets_; + std::atomic* buckets_; // The user-supplied transform whose domain is the user keys. const SliceTransform* transform_; const MemTableRep::KeyComparator& compare_; // immutable after construction - Arena* const arena_; + Allocator* const allocator_; inline size_t GetHash(const Slice& slice) const { - return MurmurHash(slice.data(), slice.size(), 0) % bucket_size_; + return MurmurHash(slice.data(), static_cast(slice.size()), 0) % + bucket_size_; } inline Bucket* GetBucket(size_t i) const { - return static_cast(buckets_[i].Acquire_Load()); + return buckets_[i].load(std::memory_order_acquire); } inline Bucket* GetBucket(const Slice& slice) const { return GetBucket(GetHash(slice)); @@ -91,33 +94,34 @@ class HashSkipListRep : public MemTableRep { } // Returns true iff the iterator is positioned at a valid node. - virtual bool Valid() const { + virtual bool Valid() const override { return list_ != nullptr && iter_.Valid(); } // Returns the key at the current position. // REQUIRES: Valid() - virtual const char* key() const { + virtual const char* key() const override { assert(Valid()); return iter_.key(); } // Advances to the next position. // REQUIRES: Valid() - virtual void Next() { + virtual void Next() override { assert(Valid()); iter_.Next(); } // Advances to the previous position. // REQUIRES: Valid() - virtual void Prev() { + virtual void Prev() override { assert(Valid()); iter_.Prev(); } // Advance to the first entry with a key >= target - virtual void Seek(const Slice& internal_key, const char* memtable_key) { + virtual void Seek(const Slice& internal_key, + const char* memtable_key) override { if (list_ != nullptr) { const char* encoded_key = (memtable_key != nullptr) ? @@ -126,9 +130,16 @@ class HashSkipListRep : public MemTableRep { } } + // Retreat to the last entry with a key <= target + virtual void SeekForPrev(const Slice& internal_key, + const char* memtable_key) override { + // not supported + assert(false); + } + // Position at the first entry in collection. // Final state of iterator is Valid() iff collection is not empty. - virtual void SeekToFirst() { + virtual void SeekToFirst() override { if (list_ != nullptr) { iter_.SeekToFirst(); } @@ -136,7 +147,7 @@ class HashSkipListRep : public MemTableRep { // Position at the last entry in collection. // Final state of iterator is Valid() iff collection is not empty. - virtual void SeekToLast() { + virtual void SeekToLast() override { if (list_ != nullptr) { iter_.SeekToLast(); } @@ -170,7 +181,7 @@ class HashSkipListRep : public MemTableRep { memtable_rep_(memtable_rep) {} // Advance to the first entry with a key >= target - virtual void Seek(const Slice& k, const char* memtable_key) { + virtual void Seek(const Slice& k, const char* memtable_key) override { auto transformed = memtable_rep_.transform_->Transform(ExtractUserKey(k)); Reset(memtable_rep_.GetBucket(transformed)); HashSkipListRep::Iterator::Seek(k, memtable_key); @@ -178,7 +189,7 @@ class HashSkipListRep : public MemTableRep { // Position at the first entry in collection. // Final state of iterator is Valid() iff collection is not empty. - virtual void SeekToFirst() { + virtual void SeekToFirst() override { // Prefix iterator does not support total order. // We simply set the iterator to invalid state Reset(nullptr); @@ -186,7 +197,7 @@ class HashSkipListRep : public MemTableRep { // Position at the last entry in collection. // Final state of iterator is Valid() iff collection is not empty. - virtual void SeekToLast() { + virtual void SeekToLast() override { // Prefix iterator does not support total order. // We simply set the iterator to invalid state Reset(nullptr); @@ -201,40 +212,42 @@ class HashSkipListRep : public MemTableRep { // instantiating an empty bucket over which to iterate. public: EmptyIterator() { } - virtual bool Valid() const { - return false; - } - virtual const char* key() const { + virtual bool Valid() const override { return false; } + virtual const char* key() const override { assert(false); return nullptr; } - virtual void Next() { } - virtual void Prev() { } + virtual void Next() override {} + virtual void Prev() override {} virtual void Seek(const Slice& internal_key, - const char* memtable_key) { } - virtual void SeekToFirst() { } - virtual void SeekToLast() { } + const char* memtable_key) override {} + virtual void SeekForPrev(const Slice& internal_key, + const char* memtable_key) override {} + virtual void SeekToFirst() override {} + virtual void SeekToLast() override {} + private: }; }; HashSkipListRep::HashSkipListRep(const MemTableRep::KeyComparator& compare, - Arena* arena, const SliceTransform* transform, + Allocator* allocator, + const SliceTransform* transform, size_t bucket_size, int32_t skiplist_height, int32_t skiplist_branching_factor) - : MemTableRep(arena), + : MemTableRep(allocator), bucket_size_(bucket_size), skiplist_height_(skiplist_height), skiplist_branching_factor_(skiplist_branching_factor), transform_(transform), compare_(compare), - arena_(arena) { - auto mem = - arena->AllocateAligned(sizeof(port::AtomicPointer) * bucket_size); - buckets_ = new (mem) port::AtomicPointer[bucket_size]; + allocator_(allocator) { + auto mem = allocator->AllocateAligned( + sizeof(std::atomic) * bucket_size); + buckets_ = new (mem) std::atomic[bucket_size]; for (size_t i = 0; i < bucket_size_; ++i) { - buckets_[i].NoBarrier_Store(nullptr); + buckets_[i].store(nullptr, std::memory_order_relaxed); } } @@ -246,10 +259,10 @@ HashSkipListRep::Bucket* HashSkipListRep::GetInitializedBucket( size_t hash = GetHash(transformed); auto bucket = GetBucket(hash); if (bucket == nullptr) { - auto addr = arena_->AllocateAligned(sizeof(Bucket)); - bucket = new (addr) Bucket(compare_, arena_, skiplist_height_, + auto addr = allocator_->AllocateAligned(sizeof(Bucket)); + bucket = new (addr) Bucket(compare_, allocator_, skiplist_height_, skiplist_branching_factor_); - buckets_[hash].Release_Store(static_cast(bucket)); + buckets_[hash].store(bucket, std::memory_order_release); } return bucket; } @@ -290,7 +303,7 @@ void HashSkipListRep::Get(const LookupKey& k, void* callback_args, MemTableRep::Iterator* HashSkipListRep::GetIterator(Arena* arena) { // allocate a new arena of similar size to the one currently in use - Arena* new_arena = new Arena(arena_->BlockSize()); + Arena* new_arena = new Arena(allocator_->BlockSize()); auto list = new Bucket(compare_, new_arena); for (size_t i = 0; i < bucket_size_; ++i) { auto bucket = GetBucket(i); @@ -321,9 +334,9 @@ MemTableRep::Iterator* HashSkipListRep::GetDynamicPrefixIterator(Arena* arena) { } // anon namespace MemTableRep* HashSkipListRepFactory::CreateMemTableRep( - const MemTableRep::KeyComparator& compare, Arena* arena, + const MemTableRep::KeyComparator& compare, Allocator* allocator, const SliceTransform* transform, Logger* logger) { - return new HashSkipListRep(compare, arena, transform, bucket_count_, + return new HashSkipListRep(compare, allocator, transform, bucket_count_, skiplist_height_, skiplist_branching_factor_); } diff --git a/util/hash_skiplist_rep.h b/memtable/hash_skiplist_rep.h similarity index 73% rename from util/hash_skiplist_rep.h rename to memtable/hash_skiplist_rep.h index 6fec60a47a4..5d1e04f34df 100644 --- a/util/hash_skiplist_rep.h +++ b/memtable/hash_skiplist_rep.h @@ -1,13 +1,13 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. -#ifndef ROCKSDB_LITE #pragma once +#ifndef ROCKSDB_LITE #include "rocksdb/slice_transform.h" #include "rocksdb/memtablerep.h" @@ -25,8 +25,9 @@ class HashSkipListRepFactory : public MemTableRepFactory { virtual ~HashSkipListRepFactory() {} + using MemTableRepFactory::CreateMemTableRep; virtual MemTableRep* CreateMemTableRep( - const MemTableRep::KeyComparator& compare, Arena* arena, + const MemTableRep::KeyComparator& compare, Allocator* allocator, const SliceTransform* transform, Logger* logger) override; virtual const char* Name() const override { diff --git a/memtable/inlineskiplist.h b/memtable/inlineskiplist.h new file mode 100644 index 00000000000..5cf6c57d573 --- /dev/null +++ b/memtable/inlineskiplist.h @@ -0,0 +1,899 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. Use of +// this source code is governed by a BSD-style license that can be found +// in the LICENSE file. See the AUTHORS file for names of contributors. +// +// InlineSkipList is derived from SkipList (skiplist.h), but it optimizes +// the memory layout by requiring that the key storage be allocated through +// the skip list instance. For the common case of SkipList this saves 1 pointer per skip list node and gives better cache +// locality, at the expense of wasted padding from using AllocateAligned +// instead of Allocate for the keys. The unused padding will be from +// 0 to sizeof(void*)-1 bytes, and the space savings are sizeof(void*) +// bytes, so despite the padding the space used is always less than +// SkipList. +// +// Thread safety ------------- +// +// Writes via Insert require external synchronization, most likely a mutex. +// InsertConcurrently can be safely called concurrently with reads and +// with other concurrent inserts. Reads require a guarantee that the +// InlineSkipList will not be destroyed while the read is in progress. +// Apart from that, reads progress without any internal locking or +// synchronization. +// +// Invariants: +// +// (1) Allocated nodes are never deleted until the InlineSkipList is +// destroyed. This is trivially guaranteed by the code since we never +// delete any skip list nodes. +// +// (2) The contents of a Node except for the next/prev pointers are +// immutable after the Node has been linked into the InlineSkipList. +// Only Insert() modifies the list, and it is careful to initialize a +// node and use release-stores to publish the nodes in one or more lists. +// +// ... prev vs. next pointer ordering ... +// + +#pragma once +#include +#include +#include +#include +#include "port/port.h" +#include "util/allocator.h" +#include "util/random.h" + +namespace rocksdb { + +template +class InlineSkipList { + private: + struct Node; + struct Splice; + + public: + static const uint16_t kMaxPossibleHeight = 32; + + // Create a new InlineSkipList object that will use "cmp" for comparing + // keys, and will allocate memory using "*allocator". Objects allocated + // in the allocator must remain allocated for the lifetime of the + // skiplist object. + explicit InlineSkipList(Comparator cmp, Allocator* allocator, + int32_t max_height = 12, + int32_t branching_factor = 4); + + // Allocates a key and a skip-list node, returning a pointer to the key + // portion of the node. This method is thread-safe if the allocator + // is thread-safe. + char* AllocateKey(size_t key_size); + + // Allocate a splice using allocator. + Splice* AllocateSplice(); + + // Inserts a key allocated by AllocateKey, after the actual key value + // has been filled in. + // + // REQUIRES: nothing that compares equal to key is currently in the list. + // REQUIRES: no concurrent calls to any of inserts. + void Insert(const char* key); + + // Inserts a key allocated by AllocateKey with a hint of last insert + // position in the skip-list. If hint points to nullptr, a new hint will be + // populated, which can be used in subsequent calls. + // + // It can be used to optimize the workload where there are multiple groups + // of keys, and each key is likely to insert to a location close to the last + // inserted key in the same group. One example is sequential inserts. + // + // REQUIRES: nothing that compares equal to key is currently in the list. + // REQUIRES: no concurrent calls to any of inserts. + void InsertWithHint(const char* key, void** hint); + + // Like Insert, but external synchronization is not required. + void InsertConcurrently(const char* key); + + // Inserts a node into the skip list. key must have been allocated by + // AllocateKey and then filled in by the caller. If UseCAS is true, + // then external synchronization is not required, otherwise this method + // may not be called concurrently with any other insertions. + // + // Regardless of whether UseCAS is true, the splice must be owned + // exclusively by the current thread. If allow_partial_splice_fix is + // true, then the cost of insertion is amortized O(log D), where D is + // the distance from the splice to the inserted key (measured as the + // number of intervening nodes). Note that this bound is very good for + // sequential insertions! If allow_partial_splice_fix is false then + // the existing splice will be ignored unless the current key is being + // inserted immediately after the splice. allow_partial_splice_fix == + // false has worse running time for the non-sequential case O(log N), + // but a better constant factor. + template + void Insert(const char* key, Splice* splice, bool allow_partial_splice_fix); + + // Returns true iff an entry that compares equal to key is in the list. + bool Contains(const char* key) const; + + // Return estimated number of entries smaller than `key`. + uint64_t EstimateCount(const char* key) const; + + // Validate correctness of the skip-list. + void TEST_Validate() const; + + // Iteration over the contents of a skip list + class Iterator { + public: + // Initialize an iterator over the specified list. + // The returned iterator is not valid. + explicit Iterator(const InlineSkipList* list); + + // Change the underlying skiplist used for this iterator + // This enables us not changing the iterator without deallocating + // an old one and then allocating a new one + void SetList(const InlineSkipList* list); + + // Returns true iff the iterator is positioned at a valid node. + bool Valid() const; + + // Returns the key at the current position. + // REQUIRES: Valid() + const char* key() const; + + // Advances to the next position. + // REQUIRES: Valid() + void Next(); + + // Advances to the previous position. + // REQUIRES: Valid() + void Prev(); + + // Advance to the first entry with a key >= target + void Seek(const char* target); + + // Retreat to the last entry with a key <= target + void SeekForPrev(const char* target); + + // Position at the first entry in list. + // Final state of iterator is Valid() iff list is not empty. + void SeekToFirst(); + + // Position at the last entry in list. + // Final state of iterator is Valid() iff list is not empty. + void SeekToLast(); + + private: + const InlineSkipList* list_; + Node* node_; + // Intentionally copyable + }; + + private: + const uint16_t kMaxHeight_; + const uint16_t kBranching_; + const uint32_t kScaledInverseBranching_; + + // Immutable after construction + Comparator const compare_; + Allocator* const allocator_; // Allocator used for allocations of nodes + + Node* const head_; + + // Modified only by Insert(). Read racily by readers, but stale + // values are ok. + std::atomic max_height_; // Height of the entire list + + // seq_splice_ is a Splice used for insertions in the non-concurrent + // case. It caches the prev and next found during the most recent + // non-concurrent insertion. + Splice* seq_splice_; + + inline int GetMaxHeight() const { + return max_height_.load(std::memory_order_relaxed); + } + + int RandomHeight(); + + Node* AllocateNode(size_t key_size, int height); + + bool Equal(const char* a, const char* b) const { + return (compare_(a, b) == 0); + } + + bool LessThan(const char* a, const char* b) const { + return (compare_(a, b) < 0); + } + + // Return true if key is greater than the data stored in "n". Null n + // is considered infinite. n should not be head_. + bool KeyIsAfterNode(const char* key, Node* n) const; + + // Returns the earliest node with a key >= key. + // Return nullptr if there is no such node. + Node* FindGreaterOrEqual(const char* key) const; + + // Return the latest node with a key < key. + // Return head_ if there is no such node. + // Fills prev[level] with pointer to previous node at "level" for every + // level in [0..max_height_-1], if prev is non-null. + Node* FindLessThan(const char* key, Node** prev = nullptr) const; + + // Return the latest node with a key < key on bottom_level. Start searching + // from root node on the level below top_level. + // Fills prev[level] with pointer to previous node at "level" for every + // level in [bottom_level..top_level-1], if prev is non-null. + Node* FindLessThan(const char* key, Node** prev, Node* root, int top_level, + int bottom_level) const; + + // Return the last node in the list. + // Return head_ if list is empty. + Node* FindLast() const; + + // Traverses a single level of the list, setting *out_prev to the last + // node before the key and *out_next to the first node after. Assumes + // that the key is not present in the skip list. On entry, before should + // point to a node that is before the key, and after should point to + // a node that is after the key. after should be nullptr if a good after + // node isn't conveniently available. + void FindSpliceForLevel(const char* key, Node* before, Node* after, int level, + Node** out_prev, Node** out_next); + + // Recomputes Splice levels from highest_level (inclusive) down to + // lowest_level (inclusive). + void RecomputeSpliceLevels(const char* key, Splice* splice, + int recompute_level); + + // No copying allowed + InlineSkipList(const InlineSkipList&); + InlineSkipList& operator=(const InlineSkipList&); +}; + +// Implementation details follow + +template +struct InlineSkipList::Splice { + // The invariant of a Splice is that prev_[i+1].key <= prev_[i].key < + // next_[i].key <= next_[i+1].key for all i. That means that if a + // key is bracketed by prev_[i] and next_[i] then it is bracketed by + // all higher levels. It is _not_ required that prev_[i]->Next(i) == + // next_[i] (it probably did at some point in the past, but intervening + // or concurrent operations might have inserted nodes in between). + int height_ = 0; + Node** prev_; + Node** next_; +}; + +// The Node data type is more of a pointer into custom-managed memory than +// a traditional C++ struct. The key is stored in the bytes immediately +// after the struct, and the next_ pointers for nodes with height > 1 are +// stored immediately _before_ the struct. This avoids the need to include +// any pointer or sizing data, which reduces per-node memory overheads. +template +struct InlineSkipList::Node { + // Stores the height of the node in the memory location normally used for + // next_[0]. This is used for passing data from AllocateKey to Insert. + void StashHeight(const int height) { + assert(sizeof(int) <= sizeof(next_[0])); + memcpy(&next_[0], &height, sizeof(int)); + } + + // Retrieves the value passed to StashHeight. Undefined after a call + // to SetNext or NoBarrier_SetNext. + int UnstashHeight() const { + int rv; + memcpy(&rv, &next_[0], sizeof(int)); + return rv; + } + + const char* Key() const { return reinterpret_cast(&next_[1]); } + + // Accessors/mutators for links. Wrapped in methods so we can add + // the appropriate barriers as necessary, and perform the necessary + // addressing trickery for storing links below the Node in memory. + Node* Next(int n) { + assert(n >= 0); + // Use an 'acquire load' so that we observe a fully initialized + // version of the returned Node. + return (next_[-n].load(std::memory_order_acquire)); + } + + void SetNext(int n, Node* x) { + assert(n >= 0); + // Use a 'release store' so that anybody who reads through this + // pointer observes a fully initialized version of the inserted node. + next_[-n].store(x, std::memory_order_release); + } + + bool CASNext(int n, Node* expected, Node* x) { + assert(n >= 0); + return next_[-n].compare_exchange_strong(expected, x); + } + + // No-barrier variants that can be safely used in a few locations. + Node* NoBarrier_Next(int n) { + assert(n >= 0); + return next_[-n].load(std::memory_order_relaxed); + } + + void NoBarrier_SetNext(int n, Node* x) { + assert(n >= 0); + next_[-n].store(x, std::memory_order_relaxed); + } + + // Insert node after prev on specific level. + void InsertAfter(Node* prev, int level) { + // NoBarrier_SetNext() suffices since we will add a barrier when + // we publish a pointer to "this" in prev. + NoBarrier_SetNext(level, prev->NoBarrier_Next(level)); + prev->SetNext(level, this); + } + + private: + // next_[0] is the lowest level link (level 0). Higher levels are + // stored _earlier_, so level 1 is at next_[-1]. + std::atomic next_[1]; +}; + +template +inline InlineSkipList::Iterator::Iterator( + const InlineSkipList* list) { + SetList(list); +} + +template +inline void InlineSkipList::Iterator::SetList( + const InlineSkipList* list) { + list_ = list; + node_ = nullptr; +} + +template +inline bool InlineSkipList::Iterator::Valid() const { + return node_ != nullptr; +} + +template +inline const char* InlineSkipList::Iterator::key() const { + assert(Valid()); + return node_->Key(); +} + +template +inline void InlineSkipList::Iterator::Next() { + assert(Valid()); + node_ = node_->Next(0); +} + +template +inline void InlineSkipList::Iterator::Prev() { + // Instead of using explicit "prev" links, we just search for the + // last node that falls before key. + assert(Valid()); + node_ = list_->FindLessThan(node_->Key()); + if (node_ == list_->head_) { + node_ = nullptr; + } +} + +template +inline void InlineSkipList::Iterator::Seek(const char* target) { + node_ = list_->FindGreaterOrEqual(target); +} + +template +inline void InlineSkipList::Iterator::SeekForPrev( + const char* target) { + Seek(target); + if (!Valid()) { + SeekToLast(); + } + while (Valid() && list_->LessThan(target, key())) { + Prev(); + } +} + +template +inline void InlineSkipList::Iterator::SeekToFirst() { + node_ = list_->head_->Next(0); +} + +template +inline void InlineSkipList::Iterator::SeekToLast() { + node_ = list_->FindLast(); + if (node_ == list_->head_) { + node_ = nullptr; + } +} + +template +int InlineSkipList::RandomHeight() { + auto rnd = Random::GetTLSInstance(); + + // Increase height with probability 1 in kBranching + int height = 1; + while (height < kMaxHeight_ && height < kMaxPossibleHeight && + rnd->Next() < kScaledInverseBranching_) { + height++; + } + assert(height > 0); + assert(height <= kMaxHeight_); + assert(height <= kMaxPossibleHeight); + return height; +} + +template +bool InlineSkipList::KeyIsAfterNode(const char* key, + Node* n) const { + // nullptr n is considered infinite + assert(n != head_); + return (n != nullptr) && (compare_(n->Key(), key) < 0); +} + +template +typename InlineSkipList::Node* +InlineSkipList::FindGreaterOrEqual(const char* key) const { + // Note: It looks like we could reduce duplication by implementing + // this function as FindLessThan(key)->Next(0), but we wouldn't be able + // to exit early on equality and the result wouldn't even be correct. + // A concurrent insert might occur after FindLessThan(key) but before + // we get a chance to call Next(0). + Node* x = head_; + int level = GetMaxHeight() - 1; + Node* last_bigger = nullptr; + while (true) { + Node* next = x->Next(level); + // Make sure the lists are sorted + assert(x == head_ || next == nullptr || KeyIsAfterNode(next->Key(), x)); + // Make sure we haven't overshot during our search + assert(x == head_ || KeyIsAfterNode(key, x)); + int cmp = (next == nullptr || next == last_bigger) + ? 1 + : compare_(next->Key(), key); + if (cmp == 0 || (cmp > 0 && level == 0)) { + return next; + } else if (cmp < 0) { + // Keep searching in this list + x = next; + } else { + // Switch to next list, reuse compare_() result + last_bigger = next; + level--; + } + } +} + +template +typename InlineSkipList::Node* +InlineSkipList::FindLessThan(const char* key, Node** prev) const { + return FindLessThan(key, prev, head_, GetMaxHeight(), 0); +} + +template +typename InlineSkipList::Node* +InlineSkipList::FindLessThan(const char* key, Node** prev, + Node* root, int top_level, + int bottom_level) const { + assert(top_level > bottom_level); + int level = top_level - 1; + Node* x = root; + // KeyIsAfter(key, last_not_after) is definitely false + Node* last_not_after = nullptr; + while (true) { + Node* next = x->Next(level); + assert(x == head_ || next == nullptr || KeyIsAfterNode(next->Key(), x)); + assert(x == head_ || KeyIsAfterNode(key, x)); + if (next != last_not_after && KeyIsAfterNode(key, next)) { + // Keep searching in this list + x = next; + } else { + if (prev != nullptr) { + prev[level] = x; + } + if (level == bottom_level) { + return x; + } else { + // Switch to next list, reuse KeyIsAfterNode() result + last_not_after = next; + level--; + } + } + } +} + +template +typename InlineSkipList::Node* +InlineSkipList::FindLast() const { + Node* x = head_; + int level = GetMaxHeight() - 1; + while (true) { + Node* next = x->Next(level); + if (next == nullptr) { + if (level == 0) { + return x; + } else { + // Switch to next list + level--; + } + } else { + x = next; + } + } +} + +template +uint64_t InlineSkipList::EstimateCount(const char* key) const { + uint64_t count = 0; + + Node* x = head_; + int level = GetMaxHeight() - 1; + while (true) { + assert(x == head_ || compare_(x->Key(), key) < 0); + Node* next = x->Next(level); + if (next == nullptr || compare_(next->Key(), key) >= 0) { + if (level == 0) { + return count; + } else { + // Switch to next list + count *= kBranching_; + level--; + } + } else { + x = next; + count++; + } + } +} + +template +InlineSkipList::InlineSkipList(const Comparator cmp, + Allocator* allocator, + int32_t max_height, + int32_t branching_factor) + : kMaxHeight_(max_height), + kBranching_(branching_factor), + kScaledInverseBranching_((Random::kMaxNext + 1) / kBranching_), + compare_(cmp), + allocator_(allocator), + head_(AllocateNode(0, max_height)), + max_height_(1), + seq_splice_(AllocateSplice()) { + assert(max_height > 0 && kMaxHeight_ == static_cast(max_height)); + assert(branching_factor > 1 && + kBranching_ == static_cast(branching_factor)); + assert(kScaledInverseBranching_ > 0); + + for (int i = 0; i < kMaxHeight_; ++i) { + head_->SetNext(i, nullptr); + } +} + +template +char* InlineSkipList::AllocateKey(size_t key_size) { + return const_cast(AllocateNode(key_size, RandomHeight())->Key()); +} + +template +typename InlineSkipList::Node* +InlineSkipList::AllocateNode(size_t key_size, int height) { + auto prefix = sizeof(std::atomic) * (height - 1); + + // prefix is space for the height - 1 pointers that we store before + // the Node instance (next_[-(height - 1) .. -1]). Node starts at + // raw + prefix, and holds the bottom-mode (level 0) skip list pointer + // next_[0]. key_size is the bytes for the key, which comes just after + // the Node. + char* raw = allocator_->AllocateAligned(prefix + sizeof(Node) + key_size); + Node* x = reinterpret_cast(raw + prefix); + + // Once we've linked the node into the skip list we don't actually need + // to know its height, because we can implicitly use the fact that we + // traversed into a node at level h to known that h is a valid level + // for that node. We need to convey the height to the Insert step, + // however, so that it can perform the proper links. Since we're not + // using the pointers at the moment, StashHeight temporarily borrow + // storage from next_[0] for that purpose. + x->StashHeight(height); + return x; +} + +template +typename InlineSkipList::Splice* +InlineSkipList::AllocateSplice() { + // size of prev_ and next_ + size_t array_size = sizeof(Node*) * (kMaxHeight_ + 1); + char* raw = allocator_->AllocateAligned(sizeof(Splice) + array_size * 2); + Splice* splice = reinterpret_cast(raw); + splice->height_ = 0; + splice->prev_ = reinterpret_cast(raw + sizeof(Splice)); + splice->next_ = reinterpret_cast(raw + sizeof(Splice) + array_size); + return splice; +} + +template +void InlineSkipList::Insert(const char* key) { + Insert(key, seq_splice_, false); +} + +template +void InlineSkipList::InsertConcurrently(const char* key) { + Node* prev[kMaxPossibleHeight]; + Node* next[kMaxPossibleHeight]; + Splice splice; + splice.prev_ = prev; + splice.next_ = next; + Insert(key, &splice, false); +} + +template +void InlineSkipList::InsertWithHint(const char* key, void** hint) { + assert(hint != nullptr); + Splice* splice = reinterpret_cast(*hint); + if (splice == nullptr) { + splice = AllocateSplice(); + *hint = reinterpret_cast(splice); + } + Insert(key, splice, true); +} + +template +void InlineSkipList::FindSpliceForLevel(const char* key, + Node* before, Node* after, + int level, Node** out_prev, + Node** out_next) { + while (true) { + Node* next = before->Next(level); + assert(before == head_ || next == nullptr || + KeyIsAfterNode(next->Key(), before)); + assert(before == head_ || KeyIsAfterNode(key, before)); + if (next == after || !KeyIsAfterNode(key, next)) { + // found it + *out_prev = before; + *out_next = next; + return; + } + before = next; + } +} + +template +void InlineSkipList::RecomputeSpliceLevels(const char* key, + Splice* splice, + int recompute_level) { + assert(recompute_level > 0); + assert(recompute_level <= splice->height_); + for (int i = recompute_level - 1; i >= 0; --i) { + FindSpliceForLevel(key, splice->prev_[i + 1], splice->next_[i + 1], i, + &splice->prev_[i], &splice->next_[i]); + } +} + +template +template +void InlineSkipList::Insert(const char* key, Splice* splice, + bool allow_partial_splice_fix) { + Node* x = reinterpret_cast(const_cast(key)) - 1; + int height = x->UnstashHeight(); + assert(height >= 1 && height <= kMaxHeight_); + + int max_height = max_height_.load(std::memory_order_relaxed); + while (height > max_height) { + if (max_height_.compare_exchange_weak(max_height, height)) { + // successfully updated it + max_height = height; + break; + } + // else retry, possibly exiting the loop because somebody else + // increased it + } + assert(max_height <= kMaxPossibleHeight); + + int recompute_height = 0; + if (splice->height_ < max_height) { + // Either splice has never been used or max_height has grown since + // last use. We could potentially fix it in the latter case, but + // that is tricky. + splice->prev_[max_height] = head_; + splice->next_[max_height] = nullptr; + splice->height_ = max_height; + recompute_height = max_height; + } else { + // Splice is a valid proper-height splice that brackets some + // key, but does it bracket this one? We need to validate it and + // recompute a portion of the splice (levels 0..recompute_height-1) + // that is a superset of all levels that don't bracket the new key. + // Several choices are reasonable, because we have to balance the work + // saved against the extra comparisons required to validate the Splice. + // + // One strategy is just to recompute all of orig_splice_height if the + // bottom level isn't bracketing. This pessimistically assumes that + // we will either get a perfect Splice hit (increasing sequential + // inserts) or have no locality. + // + // Another strategy is to walk up the Splice's levels until we find + // a level that brackets the key. This strategy lets the Splice + // hint help for other cases: it turns insertion from O(log N) into + // O(log D), where D is the number of nodes in between the key that + // produced the Splice and the current insert (insertion is aided + // whether the new key is before or after the splice). If you have + // a way of using a prefix of the key to map directly to the closest + // Splice out of O(sqrt(N)) Splices and we make it so that splices + // can also be used as hints during read, then we end up with Oshman's + // and Shavit's SkipTrie, which has O(log log N) lookup and insertion + // (compare to O(log N) for skip list). + // + // We control the pessimistic strategy with allow_partial_splice_fix. + // A good strategy is probably to be pessimistic for seq_splice_, + // optimistic if the caller actually went to the work of providing + // a Splice. + while (recompute_height < max_height) { + if (splice->prev_[recompute_height]->Next(recompute_height) != + splice->next_[recompute_height]) { + // splice isn't tight at this level, there must have been some inserts + // to this + // location that didn't update the splice. We might only be a little + // stale, but if + // the splice is very stale it would be O(N) to fix it. We haven't used + // up any of + // our budget of comparisons, so always move up even if we are + // pessimistic about + // our chances of success. + ++recompute_height; + } else if (splice->prev_[recompute_height] != head_ && + !KeyIsAfterNode(key, splice->prev_[recompute_height])) { + // key is from before splice + if (allow_partial_splice_fix) { + // skip all levels with the same node without more comparisons + Node* bad = splice->prev_[recompute_height]; + while (splice->prev_[recompute_height] == bad) { + ++recompute_height; + } + } else { + // we're pessimistic, recompute everything + recompute_height = max_height; + } + } else if (KeyIsAfterNode(key, splice->next_[recompute_height])) { + // key is from after splice + if (allow_partial_splice_fix) { + Node* bad = splice->next_[recompute_height]; + while (splice->next_[recompute_height] == bad) { + ++recompute_height; + } + } else { + recompute_height = max_height; + } + } else { + // this level brackets the key, we won! + break; + } + } + } + assert(recompute_height <= max_height); + if (recompute_height > 0) { + RecomputeSpliceLevels(key, splice, recompute_height); + } + + bool splice_is_valid = true; + if (UseCAS) { + for (int i = 0; i < height; ++i) { + while (true) { + assert(splice->next_[i] == nullptr || + compare_(x->Key(), splice->next_[i]->Key()) < 0); + assert(splice->prev_[i] == head_ || + compare_(splice->prev_[i]->Key(), x->Key()) < 0); + x->NoBarrier_SetNext(i, splice->next_[i]); + if (splice->prev_[i]->CASNext(i, splice->next_[i], x)) { + // success + break; + } + // CAS failed, we need to recompute prev and next. It is unlikely + // to be helpful to try to use a different level as we redo the + // search, because it should be unlikely that lots of nodes have + // been inserted between prev[i] and next[i]. No point in using + // next[i] as the after hint, because we know it is stale. + FindSpliceForLevel(key, splice->prev_[i], nullptr, i, &splice->prev_[i], + &splice->next_[i]); + + // Since we've narrowed the bracket for level i, we might have + // violated the Splice constraint between i and i-1. Make sure + // we recompute the whole thing next time. + if (i > 0) { + splice_is_valid = false; + } + } + } + } else { + for (int i = 0; i < height; ++i) { + if (i >= recompute_height && + splice->prev_[i]->Next(i) != splice->next_[i]) { + FindSpliceForLevel(key, splice->prev_[i], nullptr, i, &splice->prev_[i], + &splice->next_[i]); + } + assert(splice->next_[i] == nullptr || + compare_(x->Key(), splice->next_[i]->Key()) < 0); + assert(splice->prev_[i] == head_ || + compare_(splice->prev_[i]->Key(), x->Key()) < 0); + assert(splice->prev_[i]->Next(i) == splice->next_[i]); + x->NoBarrier_SetNext(i, splice->next_[i]); + splice->prev_[i]->SetNext(i, x); + } + } + if (splice_is_valid) { + for (int i = 0; i < height; ++i) { + splice->prev_[i] = x; + } + assert(splice->prev_[splice->height_] == head_); + assert(splice->next_[splice->height_] == nullptr); + for (int i = 0; i < splice->height_; ++i) { + assert(splice->next_[i] == nullptr || + compare_(key, splice->next_[i]->Key()) < 0); + assert(splice->prev_[i] == head_ || + compare_(splice->prev_[i]->Key(), key) <= 0); + assert(splice->prev_[i + 1] == splice->prev_[i] || + splice->prev_[i + 1] == head_ || + compare_(splice->prev_[i + 1]->Key(), splice->prev_[i]->Key()) < + 0); + assert(splice->next_[i + 1] == splice->next_[i] || + splice->next_[i + 1] == nullptr || + compare_(splice->next_[i]->Key(), splice->next_[i + 1]->Key()) < + 0); + } + } else { + splice->height_ = 0; + } +} + +template +bool InlineSkipList::Contains(const char* key) const { + Node* x = FindGreaterOrEqual(key); + if (x != nullptr && Equal(key, x->Key())) { + return true; + } else { + return false; + } +} + +template +void InlineSkipList::TEST_Validate() const { + // Interate over all levels at the same time, and verify nodes appear in + // the right order, and nodes appear in upper level also appear in lower + // levels. + Node* nodes[kMaxPossibleHeight]; + int max_height = GetMaxHeight(); + for (int i = 0; i < max_height; i++) { + nodes[i] = head_; + } + while (nodes[0] != nullptr) { + Node* l0_next = nodes[0]->Next(0); + if (l0_next == nullptr) { + break; + } + assert(nodes[0] == head_ || compare_(nodes[0]->Key(), l0_next->Key()) < 0); + nodes[0] = l0_next; + + int i = 1; + while (i < max_height) { + Node* next = nodes[i]->Next(i); + if (next == nullptr) { + break; + } + auto cmp = compare_(nodes[0]->Key(), next->Key()); + assert(cmp <= 0); + if (cmp == 0) { + assert(next == nodes[0]); + nodes[i] = next; + } else { + break; + } + i++; + } + } + for (int i = 1; i < max_height; i++) { + assert(nodes[i]->Next(i) == nullptr); + } +} + +} // namespace rocksdb diff --git a/memtable/inlineskiplist_test.cc b/memtable/inlineskiplist_test.cc new file mode 100644 index 00000000000..5803e5b0f55 --- /dev/null +++ b/memtable/inlineskiplist_test.cc @@ -0,0 +1,626 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "memtable/inlineskiplist.h" +#include +#include +#include "rocksdb/env.h" +#include "util/concurrent_arena.h" +#include "util/hash.h" +#include "util/random.h" +#include "util/testharness.h" + +namespace rocksdb { + +// Our test skip list stores 8-byte unsigned integers +typedef uint64_t Key; + +static const char* Encode(const uint64_t* key) { + return reinterpret_cast(key); +} + +static Key Decode(const char* key) { + Key rv; + memcpy(&rv, key, sizeof(Key)); + return rv; +} + +struct TestComparator { + int operator()(const char* a, const char* b) const { + if (Decode(a) < Decode(b)) { + return -1; + } else if (Decode(a) > Decode(b)) { + return +1; + } else { + return 0; + } + } +}; + +typedef InlineSkipList TestInlineSkipList; + +class InlineSkipTest : public testing::Test { + public: + void Insert(TestInlineSkipList* list, Key key) { + char* buf = list->AllocateKey(sizeof(Key)); + memcpy(buf, &key, sizeof(Key)); + list->Insert(buf); + keys_.insert(key); + } + + void InsertWithHint(TestInlineSkipList* list, Key key, void** hint) { + char* buf = list->AllocateKey(sizeof(Key)); + memcpy(buf, &key, sizeof(Key)); + list->InsertWithHint(buf, hint); + keys_.insert(key); + } + + void Validate(TestInlineSkipList* list) { + // Check keys exist. + for (Key key : keys_) { + ASSERT_TRUE(list->Contains(Encode(&key))); + } + // Iterate over the list, make sure keys appears in order and no extra + // keys exist. + TestInlineSkipList::Iterator iter(list); + ASSERT_FALSE(iter.Valid()); + Key zero = 0; + iter.Seek(Encode(&zero)); + for (Key key : keys_) { + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(key, Decode(iter.key())); + iter.Next(); + } + ASSERT_FALSE(iter.Valid()); + // Validate the list is well-formed. + list->TEST_Validate(); + } + + private: + std::set keys_; +}; + +TEST_F(InlineSkipTest, Empty) { + Arena arena; + TestComparator cmp; + InlineSkipList list(cmp, &arena); + Key key = 10; + ASSERT_TRUE(!list.Contains(Encode(&key))); + + InlineSkipList::Iterator iter(&list); + ASSERT_TRUE(!iter.Valid()); + iter.SeekToFirst(); + ASSERT_TRUE(!iter.Valid()); + key = 100; + iter.Seek(Encode(&key)); + ASSERT_TRUE(!iter.Valid()); + iter.SeekForPrev(Encode(&key)); + ASSERT_TRUE(!iter.Valid()); + iter.SeekToLast(); + ASSERT_TRUE(!iter.Valid()); +} + +TEST_F(InlineSkipTest, InsertAndLookup) { + const int N = 2000; + const int R = 5000; + Random rnd(1000); + std::set keys; + ConcurrentArena arena; + TestComparator cmp; + InlineSkipList list(cmp, &arena); + for (int i = 0; i < N; i++) { + Key key = rnd.Next() % R; + if (keys.insert(key).second) { + char* buf = list.AllocateKey(sizeof(Key)); + memcpy(buf, &key, sizeof(Key)); + list.Insert(buf); + } + } + + for (Key i = 0; i < R; i++) { + if (list.Contains(Encode(&i))) { + ASSERT_EQ(keys.count(i), 1U); + } else { + ASSERT_EQ(keys.count(i), 0U); + } + } + + // Simple iterator tests + { + InlineSkipList::Iterator iter(&list); + ASSERT_TRUE(!iter.Valid()); + + uint64_t zero = 0; + iter.Seek(Encode(&zero)); + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*(keys.begin()), Decode(iter.key())); + + uint64_t max_key = R - 1; + iter.SeekForPrev(Encode(&max_key)); + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*(keys.rbegin()), Decode(iter.key())); + + iter.SeekToFirst(); + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*(keys.begin()), Decode(iter.key())); + + iter.SeekToLast(); + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*(keys.rbegin()), Decode(iter.key())); + } + + // Forward iteration test + for (Key i = 0; i < R; i++) { + InlineSkipList::Iterator iter(&list); + iter.Seek(Encode(&i)); + + // Compare against model iterator + std::set::iterator model_iter = keys.lower_bound(i); + for (int j = 0; j < 3; j++) { + if (model_iter == keys.end()) { + ASSERT_TRUE(!iter.Valid()); + break; + } else { + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*model_iter, Decode(iter.key())); + ++model_iter; + iter.Next(); + } + } + } + + // Backward iteration test + for (Key i = 0; i < R; i++) { + InlineSkipList::Iterator iter(&list); + iter.SeekForPrev(Encode(&i)); + + // Compare against model iterator + std::set::iterator model_iter = keys.upper_bound(i); + for (int j = 0; j < 3; j++) { + if (model_iter == keys.begin()) { + ASSERT_TRUE(!iter.Valid()); + break; + } else { + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*--model_iter, Decode(iter.key())); + iter.Prev(); + } + } + } +} + +TEST_F(InlineSkipTest, InsertWithHint_Sequential) { + const int N = 100000; + Arena arena; + TestComparator cmp; + TestInlineSkipList list(cmp, &arena); + void* hint = nullptr; + for (int i = 0; i < N; i++) { + Key key = i; + InsertWithHint(&list, key, &hint); + } + Validate(&list); +} + +TEST_F(InlineSkipTest, InsertWithHint_MultipleHints) { + const int N = 100000; + const int S = 100; + Random rnd(534); + Arena arena; + TestComparator cmp; + TestInlineSkipList list(cmp, &arena); + void* hints[S]; + Key last_key[S]; + for (int i = 0; i < S; i++) { + hints[i] = nullptr; + last_key[i] = 0; + } + for (int i = 0; i < N; i++) { + Key s = rnd.Uniform(S); + Key key = (s << 32) + (++last_key[s]); + InsertWithHint(&list, key, &hints[s]); + } + Validate(&list); +} + +TEST_F(InlineSkipTest, InsertWithHint_MultipleHintsRandom) { + const int N = 100000; + const int S = 100; + Random rnd(534); + Arena arena; + TestComparator cmp; + TestInlineSkipList list(cmp, &arena); + void* hints[S]; + for (int i = 0; i < S; i++) { + hints[i] = nullptr; + } + for (int i = 0; i < N; i++) { + Key s = rnd.Uniform(S); + Key key = (s << 32) + rnd.Next(); + InsertWithHint(&list, key, &hints[s]); + } + Validate(&list); +} + +TEST_F(InlineSkipTest, InsertWithHint_CompatibleWithInsertWithoutHint) { + const int N = 100000; + const int S1 = 100; + const int S2 = 100; + Random rnd(534); + Arena arena; + TestComparator cmp; + TestInlineSkipList list(cmp, &arena); + std::unordered_set used; + Key with_hint[S1]; + Key without_hint[S2]; + void* hints[S1]; + for (int i = 0; i < S1; i++) { + hints[i] = nullptr; + while (true) { + Key s = rnd.Next(); + if (used.insert(s).second) { + with_hint[i] = s; + break; + } + } + } + for (int i = 0; i < S2; i++) { + while (true) { + Key s = rnd.Next(); + if (used.insert(s).second) { + without_hint[i] = s; + break; + } + } + } + for (int i = 0; i < N; i++) { + Key s = rnd.Uniform(S1 + S2); + if (s < S1) { + Key key = (with_hint[s] << 32) + rnd.Next(); + InsertWithHint(&list, key, &hints[s]); + } else { + Key key = (without_hint[s - S1] << 32) + rnd.Next(); + Insert(&list, key); + } + } + Validate(&list); +} + +// We want to make sure that with a single writer and multiple +// concurrent readers (with no synchronization other than when a +// reader's iterator is created), the reader always observes all the +// data that was present in the skip list when the iterator was +// constructor. Because insertions are happening concurrently, we may +// also observe new values that were inserted since the iterator was +// constructed, but we should never miss any values that were present +// at iterator construction time. +// +// We generate multi-part keys: +// +// where: +// key is in range [0..K-1] +// gen is a generation number for key +// hash is hash(key,gen) +// +// The insertion code picks a random key, sets gen to be 1 + the last +// generation number inserted for that key, and sets hash to Hash(key,gen). +// +// At the beginning of a read, we snapshot the last inserted +// generation number for each key. We then iterate, including random +// calls to Next() and Seek(). For every key we encounter, we +// check that it is either expected given the initial snapshot or has +// been concurrently added since the iterator started. +class ConcurrentTest { + public: + static const uint32_t K = 8; + + private: + static uint64_t key(Key key) { return (key >> 40); } + static uint64_t gen(Key key) { return (key >> 8) & 0xffffffffu; } + static uint64_t hash(Key key) { return key & 0xff; } + + static uint64_t HashNumbers(uint64_t k, uint64_t g) { + uint64_t data[2] = {k, g}; + return Hash(reinterpret_cast(data), sizeof(data), 0); + } + + static Key MakeKey(uint64_t k, uint64_t g) { + assert(sizeof(Key) == sizeof(uint64_t)); + assert(k <= K); // We sometimes pass K to seek to the end of the skiplist + assert(g <= 0xffffffffu); + return ((k << 40) | (g << 8) | (HashNumbers(k, g) & 0xff)); + } + + static bool IsValidKey(Key k) { + return hash(k) == (HashNumbers(key(k), gen(k)) & 0xff); + } + + static Key RandomTarget(Random* rnd) { + switch (rnd->Next() % 10) { + case 0: + // Seek to beginning + return MakeKey(0, 0); + case 1: + // Seek to end + return MakeKey(K, 0); + default: + // Seek to middle + return MakeKey(rnd->Next() % K, 0); + } + } + + // Per-key generation + struct State { + std::atomic generation[K]; + void Set(int k, int v) { + generation[k].store(v, std::memory_order_release); + } + int Get(int k) { return generation[k].load(std::memory_order_acquire); } + + State() { + for (unsigned int k = 0; k < K; k++) { + Set(k, 0); + } + } + }; + + // Current state of the test + State current_; + + ConcurrentArena arena_; + + // InlineSkipList is not protected by mu_. We just use a single writer + // thread to modify it. + InlineSkipList list_; + + public: + ConcurrentTest() : list_(TestComparator(), &arena_) {} + + // REQUIRES: No concurrent calls to WriteStep or ConcurrentWriteStep + void WriteStep(Random* rnd) { + const uint32_t k = rnd->Next() % K; + const int g = current_.Get(k) + 1; + const Key new_key = MakeKey(k, g); + char* buf = list_.AllocateKey(sizeof(Key)); + memcpy(buf, &new_key, sizeof(Key)); + list_.Insert(buf); + current_.Set(k, g); + } + + // REQUIRES: No concurrent calls for the same k + void ConcurrentWriteStep(uint32_t k) { + const int g = current_.Get(k) + 1; + const Key new_key = MakeKey(k, g); + char* buf = list_.AllocateKey(sizeof(Key)); + memcpy(buf, &new_key, sizeof(Key)); + list_.InsertConcurrently(buf); + ASSERT_EQ(g, current_.Get(k) + 1); + current_.Set(k, g); + } + + void ReadStep(Random* rnd) { + // Remember the initial committed state of the skiplist. + State initial_state; + for (unsigned int k = 0; k < K; k++) { + initial_state.Set(k, current_.Get(k)); + } + + Key pos = RandomTarget(rnd); + InlineSkipList::Iterator iter(&list_); + iter.Seek(Encode(&pos)); + while (true) { + Key current; + if (!iter.Valid()) { + current = MakeKey(K, 0); + } else { + current = Decode(iter.key()); + ASSERT_TRUE(IsValidKey(current)) << current; + } + ASSERT_LE(pos, current) << "should not go backwards"; + + // Verify that everything in [pos,current) was not present in + // initial_state. + while (pos < current) { + ASSERT_LT(key(pos), K) << pos; + + // Note that generation 0 is never inserted, so it is ok if + // <*,0,*> is missing. + ASSERT_TRUE((gen(pos) == 0U) || + (gen(pos) > static_cast(initial_state.Get( + static_cast(key(pos)))))) + << "key: " << key(pos) << "; gen: " << gen(pos) + << "; initgen: " << initial_state.Get(static_cast(key(pos))); + + // Advance to next key in the valid key space + if (key(pos) < key(current)) { + pos = MakeKey(key(pos) + 1, 0); + } else { + pos = MakeKey(key(pos), gen(pos) + 1); + } + } + + if (!iter.Valid()) { + break; + } + + if (rnd->Next() % 2) { + iter.Next(); + pos = MakeKey(key(pos), gen(pos) + 1); + } else { + Key new_target = RandomTarget(rnd); + if (new_target > pos) { + pos = new_target; + iter.Seek(Encode(&new_target)); + } + } + } + } +}; +const uint32_t ConcurrentTest::K; + +// Simple test that does single-threaded testing of the ConcurrentTest +// scaffolding. +TEST_F(InlineSkipTest, ConcurrentReadWithoutThreads) { + ConcurrentTest test; + Random rnd(test::RandomSeed()); + for (int i = 0; i < 10000; i++) { + test.ReadStep(&rnd); + test.WriteStep(&rnd); + } +} + +TEST_F(InlineSkipTest, ConcurrentInsertWithoutThreads) { + ConcurrentTest test; + Random rnd(test::RandomSeed()); + for (int i = 0; i < 10000; i++) { + test.ReadStep(&rnd); + uint32_t base = rnd.Next(); + for (int j = 0; j < 4; ++j) { + test.ConcurrentWriteStep((base + j) % ConcurrentTest::K); + } + } +} + +class TestState { + public: + ConcurrentTest t_; + int seed_; + std::atomic quit_flag_; + std::atomic next_writer_; + + enum ReaderState { STARTING, RUNNING, DONE }; + + explicit TestState(int s) + : seed_(s), + quit_flag_(false), + state_(STARTING), + pending_writers_(0), + state_cv_(&mu_) {} + + void Wait(ReaderState s) { + mu_.Lock(); + while (state_ != s) { + state_cv_.Wait(); + } + mu_.Unlock(); + } + + void Change(ReaderState s) { + mu_.Lock(); + state_ = s; + state_cv_.Signal(); + mu_.Unlock(); + } + + void AdjustPendingWriters(int delta) { + mu_.Lock(); + pending_writers_ += delta; + if (pending_writers_ == 0) { + state_cv_.Signal(); + } + mu_.Unlock(); + } + + void WaitForPendingWriters() { + mu_.Lock(); + while (pending_writers_ != 0) { + state_cv_.Wait(); + } + mu_.Unlock(); + } + + private: + port::Mutex mu_; + ReaderState state_; + int pending_writers_; + port::CondVar state_cv_; +}; + +static void ConcurrentReader(void* arg) { + TestState* state = reinterpret_cast(arg); + Random rnd(state->seed_); + int64_t reads = 0; + state->Change(TestState::RUNNING); + while (!state->quit_flag_.load(std::memory_order_acquire)) { + state->t_.ReadStep(&rnd); + ++reads; + } + state->Change(TestState::DONE); +} + +static void ConcurrentWriter(void* arg) { + TestState* state = reinterpret_cast(arg); + uint32_t k = state->next_writer_++ % ConcurrentTest::K; + state->t_.ConcurrentWriteStep(k); + state->AdjustPendingWriters(-1); +} + +static void RunConcurrentRead(int run) { + const int seed = test::RandomSeed() + (run * 100); + Random rnd(seed); + const int N = 1000; + const int kSize = 1000; + for (int i = 0; i < N; i++) { + if ((i % 100) == 0) { + fprintf(stderr, "Run %d of %d\n", i, N); + } + TestState state(seed + 1); + Env::Default()->SetBackgroundThreads(1); + Env::Default()->Schedule(ConcurrentReader, &state); + state.Wait(TestState::RUNNING); + for (int k = 0; k < kSize; ++k) { + state.t_.WriteStep(&rnd); + } + state.quit_flag_.store(true, std::memory_order_release); + state.Wait(TestState::DONE); + } +} + +static void RunConcurrentInsert(int run, int write_parallelism = 4) { + Env::Default()->SetBackgroundThreads(1 + write_parallelism, + Env::Priority::LOW); + const int seed = test::RandomSeed() + (run * 100); + Random rnd(seed); + const int N = 1000; + const int kSize = 1000; + for (int i = 0; i < N; i++) { + if ((i % 100) == 0) { + fprintf(stderr, "Run %d of %d\n", i, N); + } + TestState state(seed + 1); + Env::Default()->Schedule(ConcurrentReader, &state); + state.Wait(TestState::RUNNING); + for (int k = 0; k < kSize; k += write_parallelism) { + state.next_writer_ = rnd.Next(); + state.AdjustPendingWriters(write_parallelism); + for (int p = 0; p < write_parallelism; ++p) { + Env::Default()->Schedule(ConcurrentWriter, &state); + } + state.WaitForPendingWriters(); + } + state.quit_flag_.store(true, std::memory_order_release); + state.Wait(TestState::DONE); + } +} + +TEST_F(InlineSkipTest, ConcurrentRead1) { RunConcurrentRead(1); } +TEST_F(InlineSkipTest, ConcurrentRead2) { RunConcurrentRead(2); } +TEST_F(InlineSkipTest, ConcurrentRead3) { RunConcurrentRead(3); } +TEST_F(InlineSkipTest, ConcurrentRead4) { RunConcurrentRead(4); } +TEST_F(InlineSkipTest, ConcurrentRead5) { RunConcurrentRead(5); } +TEST_F(InlineSkipTest, ConcurrentInsert1) { RunConcurrentInsert(1); } +TEST_F(InlineSkipTest, ConcurrentInsert2) { RunConcurrentInsert(2); } +TEST_F(InlineSkipTest, ConcurrentInsert3) { RunConcurrentInsert(3); } + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/memtable/memtablerep_bench.cc b/memtable/memtablerep_bench.cc new file mode 100644 index 00000000000..63a0201ce82 --- /dev/null +++ b/memtable/memtablerep_bench.cc @@ -0,0 +1,698 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#ifndef GFLAGS +#include +int main() { + fprintf(stderr, "Please install gflags to run rocksdb tools\n"); + return 1; +} +#else + +#include + +#include +#include +#include +#include +#include +#include + +#include "db/dbformat.h" +#include "db/memtable.h" +#include "port/port.h" +#include "port/stack_trace.h" +#include "rocksdb/comparator.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/options.h" +#include "rocksdb/slice_transform.h" +#include "rocksdb/write_buffer_manager.h" +#include "util/arena.h" +#include "util/mutexlock.h" +#include "util/stop_watch.h" +#include "util/testutil.h" + +using GFLAGS::ParseCommandLineFlags; +using GFLAGS::RegisterFlagValidator; +using GFLAGS::SetUsageMessage; + +DEFINE_string(benchmarks, "fillrandom", + "Comma-separated list of benchmarks to run. Options:\n" + "\tfillrandom -- write N random values\n" + "\tfillseq -- write N values in sequential order\n" + "\treadrandom -- read N values in random order\n" + "\treadseq -- scan the DB\n" + "\treadwrite -- 1 thread writes while N - 1 threads " + "do random\n" + "\t reads\n" + "\tseqreadwrite -- 1 thread writes while N - 1 threads " + "do scans\n"); + +DEFINE_string(memtablerep, "skiplist", + "Which implementation of memtablerep to use. See " + "include/memtablerep.h for\n" + " more details. Options:\n" + "\tskiplist -- backed by a skiplist\n" + "\tvector -- backed by an std::vector\n" + "\thashskiplist -- backed by a hash skip list\n" + "\thashlinklist -- backed by a hash linked list\n" + "\tcuckoo -- backed by a cuckoo hash table"); + +DEFINE_int64(bucket_count, 1000000, + "bucket_count parameter to pass into NewHashSkiplistRepFactory or " + "NewHashLinkListRepFactory"); + +DEFINE_int32( + hashskiplist_height, 4, + "skiplist_height parameter to pass into NewHashSkiplistRepFactory"); + +DEFINE_int32( + hashskiplist_branching_factor, 4, + "branching_factor parameter to pass into NewHashSkiplistRepFactory"); + +DEFINE_int32( + huge_page_tlb_size, 0, + "huge_page_tlb_size parameter to pass into NewHashLinkListRepFactory"); + +DEFINE_int32(bucket_entries_logging_threshold, 4096, + "bucket_entries_logging_threshold parameter to pass into " + "NewHashLinkListRepFactory"); + +DEFINE_bool(if_log_bucket_dist_when_flash, true, + "if_log_bucket_dist_when_flash parameter to pass into " + "NewHashLinkListRepFactory"); + +DEFINE_int32( + threshold_use_skiplist, 256, + "threshold_use_skiplist parameter to pass into NewHashLinkListRepFactory"); + +DEFINE_int64( + write_buffer_size, 256, + "write_buffer_size parameter to pass into NewHashCuckooRepFactory"); + +DEFINE_int64( + average_data_size, 64, + "average_data_size parameter to pass into NewHashCuckooRepFactory"); + +DEFINE_int64( + hash_function_count, 4, + "hash_function_count parameter to pass into NewHashCuckooRepFactory"); + +DEFINE_int32( + num_threads, 1, + "Number of concurrent threads to run. If the benchmark includes writes,\n" + "then at most one thread will be a writer"); + +DEFINE_int32(num_operations, 1000000, + "Number of operations to do for write and random read benchmarks"); + +DEFINE_int32(num_scans, 10, + "Number of times for each thread to scan the memtablerep for " + "sequential read " + "benchmarks"); + +DEFINE_int32(item_size, 100, "Number of bytes each item should be"); + +DEFINE_int32(prefix_length, 8, + "Prefix length to pass into NewFixedPrefixTransform"); + +/* VectorRep settings */ +DEFINE_int64(vectorrep_count, 0, + "Number of entries to reserve on VectorRep initialization"); + +DEFINE_int64(seed, 0, + "Seed base for random number generators. " + "When 0 it is deterministic."); + +namespace rocksdb { + +namespace { +struct CallbackVerifyArgs { + bool found; + LookupKey* key; + MemTableRep* table; + InternalKeyComparator* comparator; +}; +} // namespace + +// Helper for quickly generating random data. +class RandomGenerator { + private: + std::string data_; + unsigned int pos_; + + public: + RandomGenerator() { + Random rnd(301); + auto size = (unsigned)std::max(1048576, FLAGS_item_size); + test::RandomString(&rnd, size, &data_); + pos_ = 0; + } + + Slice Generate(unsigned int len) { + assert(len <= data_.size()); + if (pos_ + len > data_.size()) { + pos_ = 0; + } + pos_ += len; + return Slice(data_.data() + pos_ - len, len); + } +}; + +enum WriteMode { SEQUENTIAL, RANDOM, UNIQUE_RANDOM }; + +class KeyGenerator { + public: + KeyGenerator(Random64* rand, WriteMode mode, uint64_t num) + : rand_(rand), mode_(mode), num_(num), next_(0) { + if (mode_ == UNIQUE_RANDOM) { + // NOTE: if memory consumption of this approach becomes a concern, + // we can either break it into pieces and only random shuffle a section + // each time. Alternatively, use a bit map implementation + // (https://reviews.facebook.net/differential/diff/54627/) + values_.resize(num_); + for (uint64_t i = 0; i < num_; ++i) { + values_[i] = i; + } + std::shuffle( + values_.begin(), values_.end(), + std::default_random_engine(static_cast(FLAGS_seed))); + } + } + + uint64_t Next() { + switch (mode_) { + case SEQUENTIAL: + return next_++; + case RANDOM: + return rand_->Next() % num_; + case UNIQUE_RANDOM: + return values_[next_++]; + } + assert(false); + return std::numeric_limits::max(); + } + + private: + Random64* rand_; + WriteMode mode_; + const uint64_t num_; + uint64_t next_; + std::vector values_; +}; + +class BenchmarkThread { + public: + explicit BenchmarkThread(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* bytes_written, uint64_t* bytes_read, + uint64_t* sequence, uint64_t num_ops, + uint64_t* read_hits) + : table_(table), + key_gen_(key_gen), + bytes_written_(bytes_written), + bytes_read_(bytes_read), + sequence_(sequence), + num_ops_(num_ops), + read_hits_(read_hits) {} + + virtual void operator()() = 0; + virtual ~BenchmarkThread() {} + + protected: + MemTableRep* table_; + KeyGenerator* key_gen_; + uint64_t* bytes_written_; + uint64_t* bytes_read_; + uint64_t* sequence_; + uint64_t num_ops_; + uint64_t* read_hits_; + RandomGenerator generator_; +}; + +class FillBenchmarkThread : public BenchmarkThread { + public: + FillBenchmarkThread(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* bytes_written, uint64_t* bytes_read, + uint64_t* sequence, uint64_t num_ops, uint64_t* read_hits) + : BenchmarkThread(table, key_gen, bytes_written, bytes_read, sequence, + num_ops, read_hits) {} + + void FillOne() { + char* buf = nullptr; + auto internal_key_size = 16; + auto encoded_len = + FLAGS_item_size + VarintLength(internal_key_size) + internal_key_size; + KeyHandle handle = table_->Allocate(encoded_len, &buf); + assert(buf != nullptr); + char* p = EncodeVarint32(buf, internal_key_size); + auto key = key_gen_->Next(); + EncodeFixed64(p, key); + p += 8; + EncodeFixed64(p, ++(*sequence_)); + p += 8; + Slice bytes = generator_.Generate(FLAGS_item_size); + memcpy(p, bytes.data(), FLAGS_item_size); + p += FLAGS_item_size; + assert(p == buf + encoded_len); + table_->Insert(handle); + *bytes_written_ += encoded_len; + } + + void operator()() override { + for (unsigned int i = 0; i < num_ops_; ++i) { + FillOne(); + } + } +}; + +class ConcurrentFillBenchmarkThread : public FillBenchmarkThread { + public: + ConcurrentFillBenchmarkThread(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* bytes_written, uint64_t* bytes_read, + uint64_t* sequence, uint64_t num_ops, + uint64_t* read_hits, + std::atomic_int* threads_done) + : FillBenchmarkThread(table, key_gen, bytes_written, bytes_read, sequence, + num_ops, read_hits) { + threads_done_ = threads_done; + } + + void operator()() override { + // # of read threads will be total threads - write threads (always 1). Loop + // while all reads complete. + while ((*threads_done_).load() < (FLAGS_num_threads - 1)) { + FillOne(); + } + } + + private: + std::atomic_int* threads_done_; +}; + +class ReadBenchmarkThread : public BenchmarkThread { + public: + ReadBenchmarkThread(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* bytes_written, uint64_t* bytes_read, + uint64_t* sequence, uint64_t num_ops, uint64_t* read_hits) + : BenchmarkThread(table, key_gen, bytes_written, bytes_read, sequence, + num_ops, read_hits) {} + + static bool callback(void* arg, const char* entry) { + CallbackVerifyArgs* callback_args = static_cast(arg); + assert(callback_args != nullptr); + uint32_t key_length; + const char* key_ptr = GetVarint32Ptr(entry, entry + 5, &key_length); + if ((callback_args->comparator) + ->user_comparator() + ->Equal(Slice(key_ptr, key_length - 8), + callback_args->key->user_key())) { + callback_args->found = true; + } + return false; + } + + void ReadOne() { + std::string user_key; + auto key = key_gen_->Next(); + PutFixed64(&user_key, key); + LookupKey lookup_key(user_key, *sequence_); + InternalKeyComparator internal_key_comp(BytewiseComparator()); + CallbackVerifyArgs verify_args; + verify_args.found = false; + verify_args.key = &lookup_key; + verify_args.table = table_; + verify_args.comparator = &internal_key_comp; + table_->Get(lookup_key, &verify_args, callback); + if (verify_args.found) { + *bytes_read_ += VarintLength(16) + 16 + FLAGS_item_size; + ++*read_hits_; + } + } + void operator()() override { + for (unsigned int i = 0; i < num_ops_; ++i) { + ReadOne(); + } + } +}; + +class SeqReadBenchmarkThread : public BenchmarkThread { + public: + SeqReadBenchmarkThread(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* bytes_written, uint64_t* bytes_read, + uint64_t* sequence, uint64_t num_ops, + uint64_t* read_hits) + : BenchmarkThread(table, key_gen, bytes_written, bytes_read, sequence, + num_ops, read_hits) {} + + void ReadOneSeq() { + std::unique_ptr iter(table_->GetIterator()); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + // pretend to read the value + *bytes_read_ += VarintLength(16) + 16 + FLAGS_item_size; + } + ++*read_hits_; + } + + void operator()() override { + for (unsigned int i = 0; i < num_ops_; ++i) { + { ReadOneSeq(); } + } + } +}; + +class ConcurrentReadBenchmarkThread : public ReadBenchmarkThread { + public: + ConcurrentReadBenchmarkThread(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* bytes_written, uint64_t* bytes_read, + uint64_t* sequence, uint64_t num_ops, + uint64_t* read_hits, + std::atomic_int* threads_done) + : ReadBenchmarkThread(table, key_gen, bytes_written, bytes_read, sequence, + num_ops, read_hits) { + threads_done_ = threads_done; + } + + void operator()() override { + for (unsigned int i = 0; i < num_ops_; ++i) { + ReadOne(); + } + ++*threads_done_; + } + + private: + std::atomic_int* threads_done_; +}; + +class SeqConcurrentReadBenchmarkThread : public SeqReadBenchmarkThread { + public: + SeqConcurrentReadBenchmarkThread(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* bytes_written, + uint64_t* bytes_read, uint64_t* sequence, + uint64_t num_ops, uint64_t* read_hits, + std::atomic_int* threads_done) + : SeqReadBenchmarkThread(table, key_gen, bytes_written, bytes_read, + sequence, num_ops, read_hits) { + threads_done_ = threads_done; + } + + void operator()() override { + for (unsigned int i = 0; i < num_ops_; ++i) { + ReadOneSeq(); + } + ++*threads_done_; + } + + private: + std::atomic_int* threads_done_; +}; + +class Benchmark { + public: + explicit Benchmark(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* sequence, uint32_t num_threads) + : table_(table), + key_gen_(key_gen), + sequence_(sequence), + num_threads_(num_threads) {} + + virtual ~Benchmark() {} + virtual void Run() { + std::cout << "Number of threads: " << num_threads_ << std::endl; + std::vector threads; + uint64_t bytes_written = 0; + uint64_t bytes_read = 0; + uint64_t read_hits = 0; + StopWatchNano timer(Env::Default(), true); + RunThreads(&threads, &bytes_written, &bytes_read, true, &read_hits); + auto elapsed_time = static_cast(timer.ElapsedNanos() / 1000); + std::cout << "Elapsed time: " << static_cast(elapsed_time) << " us" + << std::endl; + + if (bytes_written > 0) { + auto MiB_written = static_cast(bytes_written) / (1 << 20); + auto write_throughput = MiB_written / (elapsed_time / 1000000); + std::cout << "Total bytes written: " << MiB_written << " MiB" + << std::endl; + std::cout << "Write throughput: " << write_throughput << " MiB/s" + << std::endl; + auto us_per_op = elapsed_time / num_write_ops_per_thread_; + std::cout << "write us/op: " << us_per_op << std::endl; + } + if (bytes_read > 0) { + auto MiB_read = static_cast(bytes_read) / (1 << 20); + auto read_throughput = MiB_read / (elapsed_time / 1000000); + std::cout << "Total bytes read: " << MiB_read << " MiB" << std::endl; + std::cout << "Read throughput: " << read_throughput << " MiB/s" + << std::endl; + auto us_per_op = elapsed_time / num_read_ops_per_thread_; + std::cout << "read us/op: " << us_per_op << std::endl; + } + } + + virtual void RunThreads(std::vector* threads, + uint64_t* bytes_written, uint64_t* bytes_read, + bool write, uint64_t* read_hits) = 0; + + protected: + MemTableRep* table_; + KeyGenerator* key_gen_; + uint64_t* sequence_; + uint64_t num_write_ops_per_thread_; + uint64_t num_read_ops_per_thread_; + const uint32_t num_threads_; +}; + +class FillBenchmark : public Benchmark { + public: + explicit FillBenchmark(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* sequence) + : Benchmark(table, key_gen, sequence, 1) { + num_write_ops_per_thread_ = FLAGS_num_operations; + } + + void RunThreads(std::vector* threads, uint64_t* bytes_written, + uint64_t* bytes_read, bool write, + uint64_t* read_hits) override { + FillBenchmarkThread(table_, key_gen_, bytes_written, bytes_read, sequence_, + num_write_ops_per_thread_, read_hits)(); + } +}; + +class ReadBenchmark : public Benchmark { + public: + explicit ReadBenchmark(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* sequence) + : Benchmark(table, key_gen, sequence, FLAGS_num_threads) { + num_read_ops_per_thread_ = FLAGS_num_operations / FLAGS_num_threads; + } + + void RunThreads(std::vector* threads, uint64_t* bytes_written, + uint64_t* bytes_read, bool write, + uint64_t* read_hits) override { + for (int i = 0; i < FLAGS_num_threads; ++i) { + threads->emplace_back( + ReadBenchmarkThread(table_, key_gen_, bytes_written, bytes_read, + sequence_, num_read_ops_per_thread_, read_hits)); + } + for (auto& thread : *threads) { + thread.join(); + } + std::cout << "read hit%: " + << (static_cast(*read_hits) / FLAGS_num_operations) * 100 + << std::endl; + } +}; + +class SeqReadBenchmark : public Benchmark { + public: + explicit SeqReadBenchmark(MemTableRep* table, uint64_t* sequence) + : Benchmark(table, nullptr, sequence, FLAGS_num_threads) { + num_read_ops_per_thread_ = FLAGS_num_scans; + } + + void RunThreads(std::vector* threads, uint64_t* bytes_written, + uint64_t* bytes_read, bool write, + uint64_t* read_hits) override { + for (int i = 0; i < FLAGS_num_threads; ++i) { + threads->emplace_back(SeqReadBenchmarkThread( + table_, key_gen_, bytes_written, bytes_read, sequence_, + num_read_ops_per_thread_, read_hits)); + } + for (auto& thread : *threads) { + thread.join(); + } + } +}; + +template +class ReadWriteBenchmark : public Benchmark { + public: + explicit ReadWriteBenchmark(MemTableRep* table, KeyGenerator* key_gen, + uint64_t* sequence) + : Benchmark(table, key_gen, sequence, FLAGS_num_threads) { + num_read_ops_per_thread_ = + FLAGS_num_threads <= 1 + ? 0 + : (FLAGS_num_operations / (FLAGS_num_threads - 1)); + num_write_ops_per_thread_ = FLAGS_num_operations; + } + + void RunThreads(std::vector* threads, uint64_t* bytes_written, + uint64_t* bytes_read, bool write, + uint64_t* read_hits) override { + std::atomic_int threads_done; + threads_done.store(0); + threads->emplace_back(ConcurrentFillBenchmarkThread( + table_, key_gen_, bytes_written, bytes_read, sequence_, + num_write_ops_per_thread_, read_hits, &threads_done)); + for (int i = 1; i < FLAGS_num_threads; ++i) { + threads->emplace_back( + ReadThreadType(table_, key_gen_, bytes_written, bytes_read, sequence_, + num_read_ops_per_thread_, read_hits, &threads_done)); + } + for (auto& thread : *threads) { + thread.join(); + } + } +}; + +} // namespace rocksdb + +void PrintWarnings() { +#if defined(__GNUC__) && !defined(__OPTIMIZE__) + fprintf(stdout, + "WARNING: Optimization is disabled: benchmarks unnecessarily slow\n"); +#endif +#ifndef NDEBUG + fprintf(stdout, + "WARNING: Assertions are enabled; benchmarks unnecessarily slow\n"); +#endif +} + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + SetUsageMessage(std::string("\nUSAGE:\n") + std::string(argv[0]) + + " [OPTIONS]..."); + ParseCommandLineFlags(&argc, &argv, true); + + PrintWarnings(); + + rocksdb::Options options; + + std::unique_ptr factory; + if (FLAGS_memtablerep == "skiplist") { + factory.reset(new rocksdb::SkipListFactory); +#ifndef ROCKSDB_LITE + } else if (FLAGS_memtablerep == "vector") { + factory.reset(new rocksdb::VectorRepFactory); + } else if (FLAGS_memtablerep == "hashskiplist") { + factory.reset(rocksdb::NewHashSkipListRepFactory( + FLAGS_bucket_count, FLAGS_hashskiplist_height, + FLAGS_hashskiplist_branching_factor)); + options.prefix_extractor.reset( + rocksdb::NewFixedPrefixTransform(FLAGS_prefix_length)); + } else if (FLAGS_memtablerep == "hashlinklist") { + factory.reset(rocksdb::NewHashLinkListRepFactory( + FLAGS_bucket_count, FLAGS_huge_page_tlb_size, + FLAGS_bucket_entries_logging_threshold, + FLAGS_if_log_bucket_dist_when_flash, FLAGS_threshold_use_skiplist)); + options.prefix_extractor.reset( + rocksdb::NewFixedPrefixTransform(FLAGS_prefix_length)); + } else if (FLAGS_memtablerep == "cuckoo") { + factory.reset(rocksdb::NewHashCuckooRepFactory( + FLAGS_write_buffer_size, FLAGS_average_data_size, + static_cast(FLAGS_hash_function_count))); + options.prefix_extractor.reset( + rocksdb::NewFixedPrefixTransform(FLAGS_prefix_length)); +#endif // ROCKSDB_LITE + } else { + fprintf(stdout, "Unknown memtablerep: %s\n", FLAGS_memtablerep.c_str()); + exit(1); + } + + rocksdb::InternalKeyComparator internal_key_comp( + rocksdb::BytewiseComparator()); + rocksdb::MemTable::KeyComparator key_comp(internal_key_comp); + rocksdb::Arena arena; + rocksdb::WriteBufferManager wb(FLAGS_write_buffer_size); + uint64_t sequence; + auto createMemtableRep = [&] { + sequence = 0; + return factory->CreateMemTableRep(key_comp, &arena, + options.prefix_extractor.get(), + options.info_log.get()); + }; + std::unique_ptr memtablerep; + rocksdb::Random64 rng(FLAGS_seed); + const char* benchmarks = FLAGS_benchmarks.c_str(); + while (benchmarks != nullptr) { + std::unique_ptr key_gen; + const char* sep = strchr(benchmarks, ','); + rocksdb::Slice name; + if (sep == nullptr) { + name = benchmarks; + benchmarks = nullptr; + } else { + name = rocksdb::Slice(benchmarks, sep - benchmarks); + benchmarks = sep + 1; + } + std::unique_ptr benchmark; + if (name == rocksdb::Slice("fillseq")) { + memtablerep.reset(createMemtableRep()); + key_gen.reset(new rocksdb::KeyGenerator(&rng, rocksdb::SEQUENTIAL, + FLAGS_num_operations)); + benchmark.reset(new rocksdb::FillBenchmark(memtablerep.get(), + key_gen.get(), &sequence)); + } else if (name == rocksdb::Slice("fillrandom")) { + memtablerep.reset(createMemtableRep()); + key_gen.reset(new rocksdb::KeyGenerator(&rng, rocksdb::UNIQUE_RANDOM, + FLAGS_num_operations)); + benchmark.reset(new rocksdb::FillBenchmark(memtablerep.get(), + key_gen.get(), &sequence)); + } else if (name == rocksdb::Slice("readrandom")) { + key_gen.reset(new rocksdb::KeyGenerator(&rng, rocksdb::RANDOM, + FLAGS_num_operations)); + benchmark.reset(new rocksdb::ReadBenchmark(memtablerep.get(), + key_gen.get(), &sequence)); + } else if (name == rocksdb::Slice("readseq")) { + key_gen.reset(new rocksdb::KeyGenerator(&rng, rocksdb::SEQUENTIAL, + FLAGS_num_operations)); + benchmark.reset( + new rocksdb::SeqReadBenchmark(memtablerep.get(), &sequence)); + } else if (name == rocksdb::Slice("readwrite")) { + memtablerep.reset(createMemtableRep()); + key_gen.reset(new rocksdb::KeyGenerator(&rng, rocksdb::RANDOM, + FLAGS_num_operations)); + benchmark.reset(new rocksdb::ReadWriteBenchmark< + rocksdb::ConcurrentReadBenchmarkThread>(memtablerep.get(), + key_gen.get(), &sequence)); + } else if (name == rocksdb::Slice("seqreadwrite")) { + memtablerep.reset(createMemtableRep()); + key_gen.reset(new rocksdb::KeyGenerator(&rng, rocksdb::RANDOM, + FLAGS_num_operations)); + benchmark.reset(new rocksdb::ReadWriteBenchmark< + rocksdb::SeqConcurrentReadBenchmarkThread>(memtablerep.get(), + key_gen.get(), &sequence)); + } else { + std::cout << "WARNING: skipping unknown benchmark '" << name.ToString() + << std::endl; + continue; + } + std::cout << "Running " << name.ToString() << std::endl; + benchmark->Run(); + } + + return 0; +} + +#endif // GFLAGS diff --git a/db/skiplist.h b/memtable/skiplist.h similarity index 61% rename from db/skiplist.h rename to memtable/skiplist.h index 751f7c3ec95..0162dccb78a 100644 --- a/db/skiplist.h +++ b/memtable/skiplist.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -32,10 +32,10 @@ #pragma once #include +#include #include -#include "util/arena.h" #include "port/port.h" -#include "util/arena.h" +#include "util/allocator.h" #include "util/random.h" namespace rocksdb { @@ -47,9 +47,9 @@ class SkipList { public: // Create a new SkipList object that will use "cmp" for comparing keys, - // and will allocate memory using "*arena". Objects allocated in the arena - // must remain allocated for the lifetime of the skiplist object. - explicit SkipList(Comparator cmp, Arena* arena, + // and will allocate memory using "*allocator". Objects allocated in the + // allocator must remain allocated for the lifetime of the skiplist object. + explicit SkipList(Comparator cmp, Allocator* allocator, int32_t max_height = 12, int32_t branching_factor = 4); // Insert key into the list. @@ -59,6 +59,9 @@ class SkipList { // Returns true iff an entry that compares equal to key is in the list. bool Contains(const Key& key) const; + // Return estimated number of entries smaller than `key`. + uint64_t EstimateCount(const Key& key) const; + // Iteration over the contents of a skip list class Iterator { public: @@ -89,6 +92,9 @@ class SkipList { // Advance to the first entry with a key >= target void Seek(const Key& target); + // Retreat to the last entry with a key <= target + void SeekForPrev(const Key& target); + // Position at the first entry in list. // Final state of iterator is Valid() iff list is not empty. void SeekToFirst(); @@ -104,48 +110,50 @@ class SkipList { }; private: - const int32_t kMaxHeight_; - const int32_t kBranching_; + const uint16_t kMaxHeight_; + const uint16_t kBranching_; + const uint32_t kScaledInverseBranching_; // Immutable after construction Comparator const compare_; - Arena* const arena_; // Arena used for allocations of nodes + Allocator* const allocator_; // Allocator used for allocations of nodes Node* const head_; // Modified only by Insert(). Read racily by readers, but stale // values are ok. - port::AtomicPointer max_height_; // Height of the entire list + std::atomic max_height_; // Height of the entire list - // Used for optimizing sequential insert patterns + // Used for optimizing sequential insert patterns. Tricky. prev_[i] for + // i up to max_height_ is the predecessor of prev_[0] and prev_height_ + // is the height of prev_[0]. prev_[0] can only be equal to head before + // insertion, in which case max_height_ and prev_height_ are 1. Node** prev_; int32_t prev_height_; inline int GetMaxHeight() const { - return static_cast( - reinterpret_cast(max_height_.NoBarrier_Load())); + return max_height_.load(std::memory_order_relaxed); } - // Read/written only by Insert(). - Random rnd_; - Node* NewNode(const Key& key, int height); int RandomHeight(); bool Equal(const Key& a, const Key& b) const { return (compare_(a, b) == 0); } + bool LessThan(const Key& a, const Key& b) const { + return (compare_(a, b) < 0); + } // Return true if key is greater than the data stored in "n" bool KeyIsAfterNode(const Key& key, Node* n) const; - // Return the earliest node that comes at or after key. + // Returns the earliest node with a key >= key. // Return nullptr if there is no such node. - // - // If prev is non-nullptr, fills prev[level] with pointer to previous - // node at "level" for every level in [0..max_height_-1]. - Node* FindGreaterOrEqual(const Key& key, Node** prev) const; + Node* FindGreaterOrEqual(const Key& key) const; // Return the latest node with a key < key. // Return head_ if there is no such node. - Node* FindLessThan(const Key& key) const; + // Fills prev[level] with pointer to previous node at "level" for every + // level in [0..max_height_-1], if prev is non-null. + Node* FindLessThan(const Key& key, Node** prev = nullptr) const; // Return the last node in the list. // Return head_ if list is empty. @@ -169,35 +177,35 @@ struct SkipList::Node { assert(n >= 0); // Use an 'acquire load' so that we observe a fully initialized // version of the returned Node. - return reinterpret_cast(next_[n].Acquire_Load()); + return (next_[n].load(std::memory_order_acquire)); } void SetNext(int n, Node* x) { assert(n >= 0); // Use a 'release store' so that anybody who reads through this // pointer observes a fully initialized version of the inserted node. - next_[n].Release_Store(x); + next_[n].store(x, std::memory_order_release); } // No-barrier variants that can be safely used in a few locations. Node* NoBarrier_Next(int n) { assert(n >= 0); - return reinterpret_cast(next_[n].NoBarrier_Load()); + return next_[n].load(std::memory_order_relaxed); } void NoBarrier_SetNext(int n, Node* x) { assert(n >= 0); - next_[n].NoBarrier_Store(x); + next_[n].store(x, std::memory_order_relaxed); } private: // Array of length equal to the node height. next_[0] is lowest level link. - port::AtomicPointer next_[1]; + std::atomic next_[1]; }; template typename SkipList::Node* SkipList::NewNode(const Key& key, int height) { - char* mem = arena_->AllocateAligned( - sizeof(Node) + sizeof(port::AtomicPointer) * (height - 1)); + char* mem = allocator_->AllocateAligned( + sizeof(Node) + sizeof(std::atomic) * (height - 1)); return new (mem) Node(key); } @@ -242,10 +250,22 @@ inline void SkipList::Iterator::Prev() { template inline void SkipList::Iterator::Seek(const Key& target) { - node_ = list_->FindGreaterOrEqual(target, nullptr); + node_ = list_->FindGreaterOrEqual(target); } -template +template +inline void SkipList::Iterator::SeekForPrev( + const Key& target) { + Seek(target); + if (!Valid()) { + SeekToLast(); + } + while (Valid() && list_->LessThan(target, key())) { + Prev(); + } +} + +template inline void SkipList::Iterator::SeekToFirst() { node_ = list_->head_->Next(0); } @@ -260,9 +280,11 @@ inline void SkipList::Iterator::SeekToLast() { template int SkipList::RandomHeight() { + auto rnd = Random::GetTLSInstance(); + // Increase height with probability 1 in kBranching int height = 1; - while (height < kMaxHeight_ && ((rnd_.Next() % kBranching_) == 0)) { + while (height < kMaxHeight_ && rnd->Next() < kScaledInverseBranching_) { height++; } assert(height > 0); @@ -278,36 +300,59 @@ bool SkipList::KeyIsAfterNode(const Key& key, Node* n) const { template typename SkipList::Node* SkipList:: - FindGreaterOrEqual(const Key& key, Node** prev) const { - // Use prev as an optimization hint and fallback to slow path - if (prev && !KeyIsAfterNode(key, prev[0]->Next(0))) { - Node* x = prev[0]; - Node* next = x->Next(0); - if ((x == head_) || KeyIsAfterNode(key, x)) { - // Adjust all relevant insertion points to the previous entry - for (int i = 1; i < prev_height_; i++) { - prev[i] = x; - } + FindGreaterOrEqual(const Key& key) const { + // Note: It looks like we could reduce duplication by implementing + // this function as FindLessThan(key)->Next(0), but we wouldn't be able + // to exit early on equality and the result wouldn't even be correct. + // A concurrent insert might occur after FindLessThan(key) but before + // we get a chance to call Next(0). + Node* x = head_; + int level = GetMaxHeight() - 1; + Node* last_bigger = nullptr; + while (true) { + Node* next = x->Next(level); + // Make sure the lists are sorted + assert(x == head_ || next == nullptr || KeyIsAfterNode(next->key, x)); + // Make sure we haven't overshot during our search + assert(x == head_ || KeyIsAfterNode(key, x)); + int cmp = (next == nullptr || next == last_bigger) + ? 1 : compare_(next->key, key); + if (cmp == 0 || (cmp > 0 && level == 0)) { return next; + } else if (cmp < 0) { + // Keep searching in this list + x = next; + } else { + // Switch to next list, reuse compare_() result + last_bigger = next; + level--; } } - // Normal lookup +} + +template +typename SkipList::Node* +SkipList::FindLessThan(const Key& key, Node** prev) const { Node* x = head_; int level = GetMaxHeight() - 1; + // KeyIsAfter(key, last_not_after) is definitely false + Node* last_not_after = nullptr; while (true) { Node* next = x->Next(level); - // Make sure the lists are sorted. - // If x points to head_ or next points nullptr, it is trivially satisfied. - assert((x == head_) || (next == nullptr) || KeyIsAfterNode(next->key, x)); - if (KeyIsAfterNode(key, next)) { + assert(x == head_ || next == nullptr || KeyIsAfterNode(next->key, x)); + assert(x == head_ || KeyIsAfterNode(key, x)); + if (next != last_not_after && KeyIsAfterNode(key, next)) { // Keep searching in this list x = next; } else { - if (prev != nullptr) prev[level] = x; + if (prev != nullptr) { + prev[level] = x; + } if (level == 0) { - return next; + return x; } else { - // Switch to next list + // Switch to next list, reuse KeyIUsAfterNode() result + last_not_after = next; level--; } } @@ -315,14 +360,13 @@ typename SkipList::Node* SkipList:: } template -typename SkipList::Node* -SkipList::FindLessThan(const Key& key) const { +typename SkipList::Node* SkipList::FindLast() + const { Node* x = head_; int level = GetMaxHeight() - 1; while (true) { - assert(x == head_ || compare_(x->key, key) < 0); Node* next = x->Next(level); - if (next == nullptr || compare_(next->key, key) >= 0) { + if (next == nullptr) { if (level == 0) { return x; } else { @@ -335,44 +379,51 @@ SkipList::FindLessThan(const Key& key) const { } } -template -typename SkipList::Node* SkipList::FindLast() - const { +template +uint64_t SkipList::EstimateCount(const Key& key) const { + uint64_t count = 0; + Node* x = head_; int level = GetMaxHeight() - 1; while (true) { + assert(x == head_ || compare_(x->key, key) < 0); Node* next = x->Next(level); - if (next == nullptr) { + if (next == nullptr || compare_(next->key, key) >= 0) { if (level == 0) { - return x; + return count; } else { // Switch to next list + count *= kBranching_; level--; } } else { x = next; + count++; } } } -template -SkipList::SkipList(const Comparator cmp, Arena* arena, - int32_t max_height, - int32_t branching_factor) +template +SkipList::SkipList(const Comparator cmp, Allocator* allocator, + int32_t max_height, + int32_t branching_factor) : kMaxHeight_(max_height), kBranching_(branching_factor), + kScaledInverseBranching_((Random::kMaxNext + 1) / kBranching_), compare_(cmp), - arena_(arena), + allocator_(allocator), head_(NewNode(0 /* any key will do */, max_height)), - max_height_(reinterpret_cast(1)), - prev_height_(1), - rnd_(0xdeadbeef) { - assert(kMaxHeight_ > 0); - assert(kBranching_ > 0); - // Allocate the prev_ Node* array, directly from the passed-in arena. + max_height_(1), + prev_height_(1) { + assert(max_height > 0 && kMaxHeight_ == static_cast(max_height)); + assert(branching_factor > 0 && + kBranching_ == static_cast(branching_factor)); + assert(kScaledInverseBranching_ > 0); + // Allocate the prev_ Node* array, directly from the passed-in allocator. // prev_ does not need to be freed, as its life cycle is tied up with - // the arena as a whole. - prev_ = (Node**) arena_->AllocateAligned(sizeof(Node*) * kMaxHeight_); + // the allocator as a whole. + prev_ = reinterpret_cast( + allocator_->AllocateAligned(sizeof(Node*) * kMaxHeight_)); for (int i = 0; i < kMaxHeight_; i++) { head_->SetNext(i, nullptr); prev_[i] = head_; @@ -381,12 +432,27 @@ SkipList::SkipList(const Comparator cmp, Arena* arena, template void SkipList::Insert(const Key& key) { - // TODO(opt): We can use a barrier-free variant of FindGreaterOrEqual() - // here since Insert() is externally synchronized. - Node* x = FindGreaterOrEqual(key, prev_); + // fast path for sequential insertion + if (!KeyIsAfterNode(key, prev_[0]->NoBarrier_Next(0)) && + (prev_[0] == head_ || KeyIsAfterNode(key, prev_[0]))) { + assert(prev_[0] != head_ || (prev_height_ == 1 && GetMaxHeight() == 1)); + + // Outside of this method prev_[1..max_height_] is the predecessor + // of prev_[0], and prev_height_ refers to prev_[0]. Inside Insert + // prev_[0..max_height - 1] is the predecessor of key. Switch from + // the external state to the internal + for (int i = 1; i < prev_height_; i++) { + prev_[i] = prev_[0]; + } + } else { + // TODO(opt): we could use a NoBarrier predecessor search as an + // optimization for architectures where memory_order_acquire needs + // a synchronization instruction. Doesn't matter on x86 + FindLessThan(key, prev_); + } // Our data structure does not allow duplicate insertion - assert(x == nullptr || !Equal(key, x->key)); + assert(prev_[0]->Next(0) == nullptr || !Equal(key, prev_[0]->Next(0)->key)); int height = RandomHeight(); if (height > GetMaxHeight()) { @@ -402,10 +468,10 @@ void SkipList::Insert(const Key& key) { // the loop below. In the former case the reader will // immediately drop to the next level since nullptr sorts after all // keys. In the latter case the reader will use the new node. - max_height_.NoBarrier_Store(reinterpret_cast(height)); + max_height_.store(height, std::memory_order_relaxed); } - x = NewNode(key, height); + Node* x = NewNode(key, height); for (int i = 0; i < height; i++) { // NoBarrier_SetNext() suffices since we will add a barrier when // we publish a pointer to "x" in prev[i]. @@ -418,7 +484,7 @@ void SkipList::Insert(const Key& key) { template bool SkipList::Contains(const Key& key) const { - Node* x = FindGreaterOrEqual(key, nullptr); + Node* x = FindGreaterOrEqual(key); if (x != nullptr && Equal(key, x->key)) { return true; } else { diff --git a/db/skiplist_test.cc b/memtable/skiplist_test.cc similarity index 79% rename from db/skiplist_test.cc rename to memtable/skiplist_test.cc index b87ddcbb032..50c3588bb86 100644 --- a/db/skiplist_test.cc +++ b/memtable/skiplist_test.cc @@ -1,13 +1,13 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. -#include "db/skiplist.h" +#include "memtable/skiplist.h" #include #include "rocksdb/env.h" #include "util/arena.h" @@ -31,9 +31,9 @@ struct TestComparator { } }; -class SkipTest { }; +class SkipTest : public testing::Test {}; -TEST(SkipTest, Empty) { +TEST_F(SkipTest, Empty) { Arena arena; TestComparator cmp; SkipList list(cmp, &arena); @@ -45,11 +45,13 @@ TEST(SkipTest, Empty) { ASSERT_TRUE(!iter.Valid()); iter.Seek(100); ASSERT_TRUE(!iter.Valid()); + iter.SeekForPrev(100); + ASSERT_TRUE(!iter.Valid()); iter.SeekToLast(); ASSERT_TRUE(!iter.Valid()); } -TEST(SkipTest, InsertAndLookup) { +TEST_F(SkipTest, InsertAndLookup) { const int N = 2000; const int R = 5000; Random rnd(1000); @@ -81,6 +83,10 @@ TEST(SkipTest, InsertAndLookup) { ASSERT_TRUE(iter.Valid()); ASSERT_EQ(*(keys.begin()), iter.key()); + iter.SeekForPrev(R - 1); + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*(keys.rbegin()), iter.key()); + iter.SeekToFirst(); ASSERT_TRUE(iter.Valid()); ASSERT_EQ(*(keys.begin()), iter.key()); @@ -111,19 +117,22 @@ TEST(SkipTest, InsertAndLookup) { } // Backward iteration test - { + for (int i = 0; i < R; i++) { SkipList::Iterator iter(&list); - iter.SeekToLast(); + iter.SeekForPrev(i); // Compare against model iterator - for (std::set::reverse_iterator model_iter = keys.rbegin(); - model_iter != keys.rend(); - ++model_iter) { - ASSERT_TRUE(iter.Valid()); - ASSERT_EQ(*model_iter, iter.key()); - iter.Prev(); + std::set::iterator model_iter = keys.upper_bound(i); + for (int j = 0; j < 3; j++) { + if (model_iter == keys.begin()) { + ASSERT_TRUE(!iter.Valid()); + break; + } else { + ASSERT_TRUE(iter.Valid()); + ASSERT_EQ(*--model_iter, iter.key()); + iter.Prev(); + } } - ASSERT_TRUE(!iter.Valid()); } } @@ -191,13 +200,11 @@ class ConcurrentTest { // Per-key generation struct State { - port::AtomicPointer generation[K]; - void Set(int k, intptr_t v) { - generation[k].Release_Store(reinterpret_cast(v)); - } - intptr_t Get(int k) { - return reinterpret_cast(generation[k].Acquire_Load()); + std::atomic generation[K]; + void Set(int k, int v) { + generation[k].store(v, std::memory_order_release); } + int Get(int k) { return generation[k].load(std::memory_order_acquire); } State() { for (unsigned int k = 0; k < K; k++) { @@ -221,9 +228,9 @@ class ConcurrentTest { // REQUIRES: External synchronization void WriteStep(Random* rnd) { const uint32_t k = rnd->Next() % K; - const intptr_t g = current_.Get(k) + 1; - const Key key = MakeKey(k, g); - list_.Insert(key); + const int g = current_.Get(k) + 1; + const Key new_key = MakeKey(k, g); + list_.Insert(new_key); current_.Set(k, g); } @@ -255,11 +262,10 @@ class ConcurrentTest { // Note that generation 0 is never inserted, so it is ok if // <*,0,*> is missing. ASSERT_TRUE((gen(pos) == 0U) || - (gen(pos) > (uint64_t)initial_state.Get(key(pos))) - ) << "key: " << key(pos) - << "; gen: " << gen(pos) - << "; initgen: " - << initial_state.Get(key(pos)); + (gen(pos) > static_cast(initial_state.Get( + static_cast(key(pos)))))) + << "key: " << key(pos) << "; gen: " << gen(pos) + << "; initgen: " << initial_state.Get(static_cast(key(pos))); // Advance to next key in the valid key space if (key(pos) < key(current)) { @@ -290,7 +296,7 @@ const uint32_t ConcurrentTest::K; // Simple test that does single-threaded testing of the ConcurrentTest // scaffolding. -TEST(SkipTest, ConcurrentWithoutThreads) { +TEST_F(SkipTest, ConcurrentWithoutThreads) { ConcurrentTest test; Random rnd(test::RandomSeed()); for (int i = 0; i < 10000; i++) { @@ -303,7 +309,7 @@ class TestState { public: ConcurrentTest t_; int seed_; - port::AtomicPointer quit_flag_; + std::atomic quit_flag_; enum ReaderState { STARTING, @@ -312,10 +318,7 @@ class TestState { }; explicit TestState(int s) - : seed_(s), - quit_flag_(nullptr), - state_(STARTING), - state_cv_(&mu_) {} + : seed_(s), quit_flag_(false), state_(STARTING), state_cv_(&mu_) {} void Wait(ReaderState s) { mu_.Lock(); @@ -343,7 +346,7 @@ static void ConcurrentReader(void* arg) { Random rnd(state->seed_); int64_t reads = 0; state->Change(TestState::RUNNING); - while (!state->quit_flag_.Acquire_Load()) { + while (!state->quit_flag_.load(std::memory_order_acquire)) { state->t_.ReadStep(&rnd); ++reads; } @@ -360,24 +363,26 @@ static void RunConcurrent(int run) { fprintf(stderr, "Run %d of %d\n", i, N); } TestState state(seed + 1); + Env::Default()->SetBackgroundThreads(1); Env::Default()->Schedule(ConcurrentReader, &state); state.Wait(TestState::RUNNING); - for (int i = 0; i < kSize; i++) { + for (int k = 0; k < kSize; k++) { state.t_.WriteStep(&rnd); } - state.quit_flag_.Release_Store(&state); // Any non-nullptr arg will do + state.quit_flag_.store(true, std::memory_order_release); state.Wait(TestState::DONE); } } -TEST(SkipTest, Concurrent1) { RunConcurrent(1); } -TEST(SkipTest, Concurrent2) { RunConcurrent(2); } -TEST(SkipTest, Concurrent3) { RunConcurrent(3); } -TEST(SkipTest, Concurrent4) { RunConcurrent(4); } -TEST(SkipTest, Concurrent5) { RunConcurrent(5); } +TEST_F(SkipTest, Concurrent1) { RunConcurrent(1); } +TEST_F(SkipTest, Concurrent2) { RunConcurrent(2); } +TEST_F(SkipTest, Concurrent3) { RunConcurrent(3); } +TEST_F(SkipTest, Concurrent4) { RunConcurrent(4); } +TEST_F(SkipTest, Concurrent5) { RunConcurrent(5); } } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/memtable/skiplistrep.cc b/memtable/skiplistrep.cc new file mode 100644 index 00000000000..f56be5dcb62 --- /dev/null +++ b/memtable/skiplistrep.cc @@ -0,0 +1,277 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#include "memtable/inlineskiplist.h" +#include "db/memtable.h" +#include "rocksdb/memtablerep.h" +#include "util/arena.h" + +namespace rocksdb { +namespace { +class SkipListRep : public MemTableRep { + InlineSkipList skip_list_; + const MemTableRep::KeyComparator& cmp_; + const SliceTransform* transform_; + const size_t lookahead_; + + friend class LookaheadIterator; +public: + explicit SkipListRep(const MemTableRep::KeyComparator& compare, + Allocator* allocator, const SliceTransform* transform, + const size_t lookahead) + : MemTableRep(allocator), + skip_list_(compare, allocator), + cmp_(compare), + transform_(transform), + lookahead_(lookahead) {} + + virtual KeyHandle Allocate(const size_t len, char** buf) override { + *buf = skip_list_.AllocateKey(len); + return static_cast(*buf); + } + + // Insert key into the list. + // REQUIRES: nothing that compares equal to key is currently in the list. + virtual void Insert(KeyHandle handle) override { + skip_list_.Insert(static_cast(handle)); + } + + virtual void InsertWithHint(KeyHandle handle, void** hint) override { + skip_list_.InsertWithHint(static_cast(handle), hint); + } + + virtual void InsertConcurrently(KeyHandle handle) override { + skip_list_.InsertConcurrently(static_cast(handle)); + } + + // Returns true iff an entry that compares equal to key is in the list. + virtual bool Contains(const char* key) const override { + return skip_list_.Contains(key); + } + + virtual size_t ApproximateMemoryUsage() override { + // All memory is allocated through allocator; nothing to report here + return 0; + } + + virtual void Get(const LookupKey& k, void* callback_args, + bool (*callback_func)(void* arg, + const char* entry)) override { + SkipListRep::Iterator iter(&skip_list_); + Slice dummy_slice; + for (iter.Seek(dummy_slice, k.memtable_key().data()); + iter.Valid() && callback_func(callback_args, iter.key()); + iter.Next()) { + } + } + + uint64_t ApproximateNumEntries(const Slice& start_ikey, + const Slice& end_ikey) override { + std::string tmp; + uint64_t start_count = + skip_list_.EstimateCount(EncodeKey(&tmp, start_ikey)); + uint64_t end_count = skip_list_.EstimateCount(EncodeKey(&tmp, end_ikey)); + return (end_count >= start_count) ? (end_count - start_count) : 0; + } + + virtual ~SkipListRep() override { } + + // Iteration over the contents of a skip list + class Iterator : public MemTableRep::Iterator { + InlineSkipList::Iterator iter_; + + public: + // Initialize an iterator over the specified list. + // The returned iterator is not valid. + explicit Iterator( + const InlineSkipList* list) + : iter_(list) {} + + virtual ~Iterator() override { } + + // Returns true iff the iterator is positioned at a valid node. + virtual bool Valid() const override { + return iter_.Valid(); + } + + // Returns the key at the current position. + // REQUIRES: Valid() + virtual const char* key() const override { + return iter_.key(); + } + + // Advances to the next position. + // REQUIRES: Valid() + virtual void Next() override { + iter_.Next(); + } + + // Advances to the previous position. + // REQUIRES: Valid() + virtual void Prev() override { + iter_.Prev(); + } + + // Advance to the first entry with a key >= target + virtual void Seek(const Slice& user_key, const char* memtable_key) + override { + if (memtable_key != nullptr) { + iter_.Seek(memtable_key); + } else { + iter_.Seek(EncodeKey(&tmp_, user_key)); + } + } + + // Retreat to the last entry with a key <= target + virtual void SeekForPrev(const Slice& user_key, + const char* memtable_key) override { + if (memtable_key != nullptr) { + iter_.SeekForPrev(memtable_key); + } else { + iter_.SeekForPrev(EncodeKey(&tmp_, user_key)); + } + } + + // Position at the first entry in list. + // Final state of iterator is Valid() iff list is not empty. + virtual void SeekToFirst() override { + iter_.SeekToFirst(); + } + + // Position at the last entry in list. + // Final state of iterator is Valid() iff list is not empty. + virtual void SeekToLast() override { + iter_.SeekToLast(); + } + protected: + std::string tmp_; // For passing to EncodeKey + }; + + // Iterator over the contents of a skip list which also keeps track of the + // previously visited node. In Seek(), it examines a few nodes after it + // first, falling back to O(log n) search from the head of the list only if + // the target key hasn't been found. + class LookaheadIterator : public MemTableRep::Iterator { + public: + explicit LookaheadIterator(const SkipListRep& rep) : + rep_(rep), iter_(&rep_.skip_list_), prev_(iter_) {} + + virtual ~LookaheadIterator() override {} + + virtual bool Valid() const override { + return iter_.Valid(); + } + + virtual const char *key() const override { + assert(Valid()); + return iter_.key(); + } + + virtual void Next() override { + assert(Valid()); + + bool advance_prev = true; + if (prev_.Valid()) { + auto k1 = rep_.UserKey(prev_.key()); + auto k2 = rep_.UserKey(iter_.key()); + + if (k1.compare(k2) == 0) { + // same user key, don't move prev_ + advance_prev = false; + } else if (rep_.transform_) { + // only advance prev_ if it has the same prefix as iter_ + auto t1 = rep_.transform_->Transform(k1); + auto t2 = rep_.transform_->Transform(k2); + advance_prev = t1.compare(t2) == 0; + } + } + + if (advance_prev) { + prev_ = iter_; + } + iter_.Next(); + } + + virtual void Prev() override { + assert(Valid()); + iter_.Prev(); + prev_ = iter_; + } + + virtual void Seek(const Slice& internal_key, const char *memtable_key) + override { + const char *encoded_key = + (memtable_key != nullptr) ? + memtable_key : EncodeKey(&tmp_, internal_key); + + if (prev_.Valid() && rep_.cmp_(encoded_key, prev_.key()) >= 0) { + // prev_.key() is smaller or equal to our target key; do a quick + // linear search (at most lookahead_ steps) starting from prev_ + iter_ = prev_; + + size_t cur = 0; + while (cur++ <= rep_.lookahead_ && iter_.Valid()) { + if (rep_.cmp_(encoded_key, iter_.key()) <= 0) { + return; + } + Next(); + } + } + + iter_.Seek(encoded_key); + prev_ = iter_; + } + + virtual void SeekForPrev(const Slice& internal_key, + const char* memtable_key) override { + const char* encoded_key = (memtable_key != nullptr) + ? memtable_key + : EncodeKey(&tmp_, internal_key); + iter_.SeekForPrev(encoded_key); + prev_ = iter_; + } + + virtual void SeekToFirst() override { + iter_.SeekToFirst(); + prev_ = iter_; + } + + virtual void SeekToLast() override { + iter_.SeekToLast(); + prev_ = iter_; + } + + protected: + std::string tmp_; // For passing to EncodeKey + + private: + const SkipListRep& rep_; + InlineSkipList::Iterator iter_; + InlineSkipList::Iterator prev_; + }; + + virtual MemTableRep::Iterator* GetIterator(Arena* arena = nullptr) override { + if (lookahead_ > 0) { + void *mem = + arena ? arena->AllocateAligned(sizeof(SkipListRep::LookaheadIterator)) + : operator new(sizeof(SkipListRep::LookaheadIterator)); + return new (mem) SkipListRep::LookaheadIterator(*this); + } else { + void *mem = + arena ? arena->AllocateAligned(sizeof(SkipListRep::Iterator)) + : operator new(sizeof(SkipListRep::Iterator)); + return new (mem) SkipListRep::Iterator(&skip_list_); + } + } +}; +} + +MemTableRep* SkipListFactory::CreateMemTableRep( + const MemTableRep::KeyComparator& compare, Allocator* allocator, + const SliceTransform* transform, Logger* logger) { + return new SkipListRep(compare, allocator, transform, lookahead_); +} + +} // namespace rocksdb diff --git a/memtable/stl_wrappers.h b/memtable/stl_wrappers.h new file mode 100644 index 00000000000..19fa1514881 --- /dev/null +++ b/memtable/stl_wrappers.h @@ -0,0 +1,34 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once + +#include +#include + +#include "rocksdb/comparator.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/slice.h" +#include "util/coding.h" +#include "util/murmurhash.h" + +namespace rocksdb { +namespace stl_wrappers { + +class Base { + protected: + const MemTableRep::KeyComparator& compare_; + explicit Base(const MemTableRep::KeyComparator& compare) + : compare_(compare) {} +}; + +struct Compare : private Base { + explicit Compare(const MemTableRep::KeyComparator& compare) : Base(compare) {} + inline bool operator()(const char* a, const char* b) const { + return compare_(a, b) < 0; + } +}; + +} +} diff --git a/util/vectorrep.cc b/memtable/vectorrep.cc similarity index 86% rename from util/vectorrep.cc rename to memtable/vectorrep.cc index e61b8ad085d..e54025c2d3d 100644 --- a/util/vectorrep.cc +++ b/memtable/vectorrep.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #ifndef ROCKSDB_LITE #include "rocksdb/memtablerep.h" @@ -14,9 +14,9 @@ #include "util/arena.h" #include "db/memtable.h" +#include "memtable/stl_wrappers.h" #include "port/port.h" #include "util/mutexlock.h" -#include "util/stl_wrappers.h" namespace rocksdb { namespace { @@ -25,7 +25,7 @@ using namespace stl_wrappers; class VectorRep : public MemTableRep { public: - VectorRep(const KeyComparator& compare, Arena* arena, size_t count); + VectorRep(const KeyComparator& compare, Allocator* allocator, size_t count); // Insert key into the collection. (The caller will pack key and value into a // single buffer and pass that in as the parameter to Insert) @@ -49,7 +49,7 @@ class VectorRep : public MemTableRep { class Iterator : public MemTableRep::Iterator { class VectorRep* vrep_; std::shared_ptr> bucket_; - typename std::vector::const_iterator mutable cit_; + std::vector::const_iterator mutable cit_; const KeyComparator& compare_; std::string tmp_; // For passing to EncodeKey bool mutable sorted_; @@ -82,6 +82,10 @@ class VectorRep : public MemTableRep { // Advance to the first entry with a key >= target virtual void Seek(const Slice& user_key, const char* memtable_key) override; + // Advance to the first entry with a key <= target + virtual void SeekForPrev(const Slice& user_key, + const char* memtable_key) override; + // Position at the first entry in collection. // Final state of iterator is Valid() iff collection is not empty. virtual void SeekToFirst() override; @@ -131,12 +135,15 @@ size_t VectorRep::ApproximateMemoryUsage() { ); } -VectorRep::VectorRep(const KeyComparator& compare, Arena* arena, size_t count) - : MemTableRep(arena), - bucket_(new Bucket()), - immutable_(false), - sorted_(false), - compare_(compare) { bucket_.get()->reserve(count); } +VectorRep::VectorRep(const KeyComparator& compare, Allocator* allocator, + size_t count) + : MemTableRep(allocator), + bucket_(new Bucket()), + immutable_(false), + sorted_(false), + compare_(compare) { + bucket_.get()->reserve(count); +} VectorRep::Iterator::Iterator(class VectorRep* vrep, std::shared_ptr> bucket, @@ -176,14 +183,14 @@ bool VectorRep::Iterator::Valid() const { // Returns the key at the current position. // REQUIRES: Valid() const char* VectorRep::Iterator::key() const { - assert(Valid()); + assert(sorted_); return *cit_; } // Advances to the next position. // REQUIRES: Valid() void VectorRep::Iterator::Next() { - assert(Valid()); + assert(sorted_); if (cit_ == bucket_->end()) { return; } @@ -193,7 +200,7 @@ void VectorRep::Iterator::Next() { // Advances to the previous position. // REQUIRES: Valid() void VectorRep::Iterator::Prev() { - assert(Valid()); + assert(sorted_); if (cit_ == bucket_->begin()) { // If you try to go back from the first element, the iterator should be // invalidated. So we set it to past-the-end. This means that you can @@ -219,6 +226,12 @@ void VectorRep::Iterator::Seek(const Slice& user_key, }).first; } +// Advance to the first entry with a key <= target +void VectorRep::Iterator::SeekForPrev(const Slice& user_key, + const char* memtable_key) { + assert(false); +} + // Position at the first entry in collection. // Final state of iterator is Valid() iff collection is not empty. void VectorRep::Iterator::SeekToFirst() { @@ -282,9 +295,9 @@ MemTableRep::Iterator* VectorRep::GetIterator(Arena* arena) { } // anon namespace MemTableRep* VectorRepFactory::CreateMemTableRep( - const MemTableRep::KeyComparator& compare, Arena* arena, + const MemTableRep::KeyComparator& compare, Allocator* allocator, const SliceTransform*, Logger* logger) { - return new VectorRep(compare, arena, count_); + return new VectorRep(compare, allocator, count_); } } // namespace rocksdb #endif // ROCKSDB_LITE diff --git a/memtable/write_buffer_manager.cc b/memtable/write_buffer_manager.cc new file mode 100644 index 00000000000..bac0fdd8fbd --- /dev/null +++ b/memtable/write_buffer_manager.cc @@ -0,0 +1,124 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "rocksdb/write_buffer_manager.h" +#include +#include "util/coding.h" + +namespace rocksdb { +#ifndef ROCKSDB_LITE +namespace { +const size_t kSizeDummyEntry = 1024 * 1024; +// The key will be longer than keys for blocks in SST files so they won't +// conflict. +const size_t kCacheKeyPrefix = kMaxVarint64Length * 4 + 1; +} // namespace + +struct WriteBufferManager::CacheRep { + std::shared_ptr cache_; + std::mutex cache_mutex_; + std::atomic cache_allocated_size_; + // The non-prefix part will be updated according to the ID to use. + char cache_key_[kCacheKeyPrefix + kMaxVarint64Length]; + uint64_t next_cache_key_id_ = 0; + std::vector dummy_handles_; + + explicit CacheRep(std::shared_ptr cache) + : cache_(cache), cache_allocated_size_(0) { + memset(cache_key_, 0, kCacheKeyPrefix); + size_t pointer_size = sizeof(const void*); + assert(pointer_size <= kCacheKeyPrefix); + memcpy(cache_key_, static_cast(this), pointer_size); + } + + Slice GetNextCacheKey() { + memset(cache_key_ + kCacheKeyPrefix, 0, kMaxVarint64Length); + char* end = + EncodeVarint64(cache_key_ + kCacheKeyPrefix, next_cache_key_id_++); + return Slice(cache_key_, static_cast(end - cache_key_)); + } +}; +#else +struct WriteBufferManager::CacheRep {}; +#endif // ROCKSDB_LITE + +WriteBufferManager::WriteBufferManager(size_t _buffer_size, + std::shared_ptr cache) + : buffer_size_(_buffer_size), + mutable_limit_(buffer_size_ * 7 / 8), + memory_used_(0), + memory_active_(0), + cache_rep_(nullptr) { +#ifndef ROCKSDB_LITE + if (cache) { + // Construct the cache key using the pointer to this. + cache_rep_.reset(new CacheRep(cache)); + } +#endif // ROCKSDB_LITE +} + +WriteBufferManager::~WriteBufferManager() { +#ifndef ROCKSDB_LITE + if (cache_rep_) { + for (auto* handle : cache_rep_->dummy_handles_) { + cache_rep_->cache_->Release(handle, true); + } + } +#endif // ROCKSDB_LITE +} + +// Should only be called from write thread +void WriteBufferManager::ReserveMemWithCache(size_t mem) { +#ifndef ROCKSDB_LITE + assert(cache_rep_ != nullptr); + // Use a mutex to protect various data structures. Can be optimzied to a + // lock-free solution if it ends up with a performance bottleneck. + std::lock_guard lock(cache_rep_->cache_mutex_); + + size_t new_mem_used = memory_used_.load(std::memory_order_relaxed) + mem; + memory_used_.store(new_mem_used, std::memory_order_relaxed); + while (new_mem_used > cache_rep_->cache_allocated_size_) { + // Expand size by at least 1MB. + // Add a dummy record to the cache + Cache::Handle* handle; + cache_rep_->cache_->Insert(cache_rep_->GetNextCacheKey(), nullptr, + kSizeDummyEntry, nullptr, &handle); + cache_rep_->dummy_handles_.push_back(handle); + cache_rep_->cache_allocated_size_ += kSizeDummyEntry; + } +#endif // ROCKSDB_LITE +} + +void WriteBufferManager::FreeMemWithCache(size_t mem) { +#ifndef ROCKSDB_LITE + assert(cache_rep_ != nullptr); + // Use a mutex to protect various data structures. Can be optimzied to a + // lock-free solution if it ends up with a performance bottleneck. + std::lock_guard lock(cache_rep_->cache_mutex_); + size_t new_mem_used = memory_used_.load(std::memory_order_relaxed) - mem; + memory_used_.store(new_mem_used, std::memory_order_relaxed); + // Gradually shrink memory costed in the block cache if the actual + // usage is less than 3/4 of what we reserve from the block cache. + // We do this becausse: + // 1. we don't pay the cost of the block cache immediately a memtable is + // freed, as block cache insert is expensive; + // 2. eventually, if we walk away from a temporary memtable size increase, + // we make sure shrink the memory costed in block cache over time. + // In this way, we only shrink costed memory showly even there is enough + // margin. + if (new_mem_used < cache_rep_->cache_allocated_size_ / 4 * 3 && + cache_rep_->cache_allocated_size_ - kSizeDummyEntry > new_mem_used) { + assert(!cache_rep_->dummy_handles_.empty()); + cache_rep_->cache_->Release(cache_rep_->dummy_handles_.back(), true); + cache_rep_->dummy_handles_.pop_back(); + cache_rep_->cache_allocated_size_ -= kSizeDummyEntry; + } +#endif // ROCKSDB_LITE +} +} // namespace rocksdb diff --git a/memtable/write_buffer_manager_test.cc b/memtable/write_buffer_manager_test.cc new file mode 100644 index 00000000000..0fc9fd06c7c --- /dev/null +++ b/memtable/write_buffer_manager_test.cc @@ -0,0 +1,151 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "rocksdb/write_buffer_manager.h" +#include "util/testharness.h" + +namespace rocksdb { + +class WriteBufferManagerTest : public testing::Test {}; + +#ifndef ROCKSDB_LITE +TEST_F(WriteBufferManagerTest, ShouldFlush) { + // A write buffer manager of size 10MB + std::unique_ptr wbf( + new WriteBufferManager(10 * 1024 * 1024)); + + wbf->ReserveMem(8 * 1024 * 1024); + ASSERT_FALSE(wbf->ShouldFlush()); + // 90% of the hard limit will hit the condition + wbf->ReserveMem(1 * 1024 * 1024); + ASSERT_TRUE(wbf->ShouldFlush()); + // Scheduling for freeing will release the condition + wbf->ScheduleFreeMem(1 * 1024 * 1024); + ASSERT_FALSE(wbf->ShouldFlush()); + + wbf->ReserveMem(2 * 1024 * 1024); + ASSERT_TRUE(wbf->ShouldFlush()); + + wbf->ScheduleFreeMem(4 * 1024 * 1024); + // 11MB total, 6MB mutable. hard limit still hit + ASSERT_TRUE(wbf->ShouldFlush()); + + wbf->ScheduleFreeMem(2 * 1024 * 1024); + // 11MB total, 4MB mutable. hard limit stills but won't flush because more + // than half data is already being flushed. + ASSERT_FALSE(wbf->ShouldFlush()); + + wbf->ReserveMem(4 * 1024 * 1024); + // 15 MB total, 8MB mutable. + ASSERT_TRUE(wbf->ShouldFlush()); + + wbf->FreeMem(7 * 1024 * 1024); + // 9MB total, 8MB mutable. + ASSERT_FALSE(wbf->ShouldFlush()); +} + +TEST_F(WriteBufferManagerTest, CacheCost) { + // 1GB cache + std::shared_ptr cache = NewLRUCache(1024 * 1024 * 1024, 4); + // A write buffer manager of size 50MB + std::unique_ptr wbf( + new WriteBufferManager(50 * 1024 * 1024, cache)); + + // Allocate 1.5MB will allocate 2MB + wbf->ReserveMem(1536 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 2 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 2 * 1024 * 1024 + 10000); + + // Allocate another 2MB + wbf->ReserveMem(2 * 1024 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 4 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 4 * 1024 * 1024 + 10000); + + // Allocate another 20MB + wbf->ReserveMem(20 * 1024 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 24 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 24 * 1024 * 1024 + 10000); + + // Free 2MB will not cause any change in cache cost + wbf->FreeMem(2 * 1024 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 24 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 24 * 1024 * 1024 + 10000); + + ASSERT_FALSE(wbf->ShouldFlush()); + + // Allocate another 30MB + wbf->ReserveMem(30 * 1024 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 52 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 52 * 1024 * 1024 + 10000); + ASSERT_TRUE(wbf->ShouldFlush()); + + ASSERT_TRUE(wbf->ShouldFlush()); + + wbf->ScheduleFreeMem(20 * 1024 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 52 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 52 * 1024 * 1024 + 10000); + + // Still need flush as the hard limit hits + ASSERT_TRUE(wbf->ShouldFlush()); + + // Free 20MB will releae 1MB from cache + wbf->FreeMem(20 * 1024 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 51 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 51 * 1024 * 1024 + 10000); + + ASSERT_FALSE(wbf->ShouldFlush()); + + // Every free will release 1MB if still not hit 3/4 + wbf->FreeMem(16 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 50 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 50 * 1024 * 1024 + 10000); + + wbf->FreeMem(16 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 49 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 49 * 1024 * 1024 + 10000); + + // Free 2MB will not cause any change in cache cost + wbf->ReserveMem(2 * 1024 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 49 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 49 * 1024 * 1024 + 10000); + + wbf->FreeMem(16 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 48 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 48 * 1024 * 1024 + 10000); + + // Destory write buffer manger should free everything + wbf.reset(); + ASSERT_LT(cache->GetPinnedUsage(), 1024 * 1024); +} + +TEST_F(WriteBufferManagerTest, NoCapCacheCost) { + // 1GB cache + std::shared_ptr cache = NewLRUCache(1024 * 1024 * 1024, 4); + // A write buffer manager of size 256MB + std::unique_ptr wbf(new WriteBufferManager(0, cache)); + // Allocate 1.5MB will allocate 2MB + wbf->ReserveMem(10 * 1024 * 1024); + ASSERT_GE(cache->GetPinnedUsage(), 10 * 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 10 * 1024 * 1024 + 10000); + ASSERT_FALSE(wbf->ShouldFlush()); + + wbf->FreeMem(9 * 1024 * 1024); + for (int i = 0; i < 10; i++) { + wbf->FreeMem(16 * 1024); + } + ASSERT_GE(cache->GetPinnedUsage(), 1024 * 1024); + ASSERT_LT(cache->GetPinnedUsage(), 1024 * 1024 + 10000); +} +#endif // ROCKSDB_LITE +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/monitoring/file_read_sample.h b/monitoring/file_read_sample.h new file mode 100644 index 00000000000..9ad7d2f56ea --- /dev/null +++ b/monitoring/file_read_sample.h @@ -0,0 +1,23 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#pragma once +#include "db/version_edit.h" +#include "util/random.h" + +namespace rocksdb { +static const uint32_t kFileReadSampleRate = 1024; +extern bool should_sample_file_read(); +extern void sample_file_read_inc(FileMetaData*); + +inline bool should_sample_file_read() { + return (Random::GetTLSInstance()->Next() % kFileReadSampleRate == 307); +} + +inline void sample_file_read_inc(FileMetaData* meta) { + meta->stats.num_reads_sampled.fetch_add(kFileReadSampleRate, + std::memory_order_relaxed); +} +} diff --git a/monitoring/histogram.cc b/monitoring/histogram.cc new file mode 100644 index 00000000000..b3c01a78e08 --- /dev/null +++ b/monitoring/histogram.cc @@ -0,0 +1,287 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include "monitoring/histogram.h" + +#include +#include +#include +#include + +#include "port/port.h" +#include "util/cast_util.h" + +namespace rocksdb { + +HistogramBucketMapper::HistogramBucketMapper() { + // If you change this, you also need to change + // size of array buckets_ in HistogramImpl + bucketValues_ = {1, 2}; + valueIndexMap_ = {{1, 0}, {2, 1}}; + double bucket_val = static_cast(bucketValues_.back()); + while ((bucket_val = 1.5 * bucket_val) <= static_cast(port::kMaxUint64)) { + bucketValues_.push_back(static_cast(bucket_val)); + // Extracts two most significant digits to make histogram buckets more + // human-readable. E.g., 172 becomes 170. + uint64_t pow_of_ten = 1; + while (bucketValues_.back() / 10 > 10) { + bucketValues_.back() /= 10; + pow_of_ten *= 10; + } + bucketValues_.back() *= pow_of_ten; + valueIndexMap_[bucketValues_.back()] = bucketValues_.size() - 1; + } + maxBucketValue_ = bucketValues_.back(); + minBucketValue_ = bucketValues_.front(); +} + +size_t HistogramBucketMapper::IndexForValue(const uint64_t value) const { + if (value >= maxBucketValue_) { + return bucketValues_.size() - 1; + } else if ( value >= minBucketValue_ ) { + std::map::const_iterator lowerBound = + valueIndexMap_.lower_bound(value); + if (lowerBound != valueIndexMap_.end()) { + return static_cast(lowerBound->second); + } else { + return 0; + } + } else { + return 0; + } +} + +namespace { + const HistogramBucketMapper bucketMapper; +} + +HistogramStat::HistogramStat() + : num_buckets_(bucketMapper.BucketCount()) { + assert(num_buckets_ == sizeof(buckets_) / sizeof(*buckets_)); + Clear(); +} + +void HistogramStat::Clear() { + min_.store(bucketMapper.LastValue(), std::memory_order_relaxed); + max_.store(0, std::memory_order_relaxed); + num_.store(0, std::memory_order_relaxed); + sum_.store(0, std::memory_order_relaxed); + sum_squares_.store(0, std::memory_order_relaxed); + for (unsigned int b = 0; b < num_buckets_; b++) { + buckets_[b].store(0, std::memory_order_relaxed); + } +}; + +bool HistogramStat::Empty() const { return num() == 0; } + +void HistogramStat::Add(uint64_t value) { + // This function is designed to be lock free, as it's in the critical path + // of any operation. Each individual value is atomic and the order of updates + // by concurrent threads is tolerable. + const size_t index = bucketMapper.IndexForValue(value); + assert(index < num_buckets_); + buckets_[index].store(buckets_[index].load(std::memory_order_relaxed) + 1, + std::memory_order_relaxed); + + uint64_t old_min = min(); + if (value < old_min) { + min_.store(value, std::memory_order_relaxed); + } + + uint64_t old_max = max(); + if (value > old_max) { + max_.store(value, std::memory_order_relaxed); + } + + num_.store(num_.load(std::memory_order_relaxed) + 1, + std::memory_order_relaxed); + sum_.store(sum_.load(std::memory_order_relaxed) + value, + std::memory_order_relaxed); + sum_squares_.store( + sum_squares_.load(std::memory_order_relaxed) + value * value, + std::memory_order_relaxed); +} + +void HistogramStat::Merge(const HistogramStat& other) { + // This function needs to be performned with the outer lock acquired + // However, atomic operation on every member is still need, since Add() + // requires no lock and value update can still happen concurrently + uint64_t old_min = min(); + uint64_t other_min = other.min(); + while (other_min < old_min && + !min_.compare_exchange_weak(old_min, other_min)) {} + + uint64_t old_max = max(); + uint64_t other_max = other.max(); + while (other_max > old_max && + !max_.compare_exchange_weak(old_max, other_max)) {} + + num_.fetch_add(other.num(), std::memory_order_relaxed); + sum_.fetch_add(other.sum(), std::memory_order_relaxed); + sum_squares_.fetch_add(other.sum_squares(), std::memory_order_relaxed); + for (unsigned int b = 0; b < num_buckets_; b++) { + buckets_[b].fetch_add(other.bucket_at(b), std::memory_order_relaxed); + } +} + +double HistogramStat::Median() const { + return Percentile(50.0); +} + +double HistogramStat::Percentile(double p) const { + double threshold = num() * (p / 100.0); + uint64_t cumulative_sum = 0; + for (unsigned int b = 0; b < num_buckets_; b++) { + uint64_t bucket_value = bucket_at(b); + cumulative_sum += bucket_value; + if (cumulative_sum >= threshold) { + // Scale linearly within this bucket + uint64_t left_point = (b == 0) ? 0 : bucketMapper.BucketLimit(b-1); + uint64_t right_point = bucketMapper.BucketLimit(b); + uint64_t left_sum = cumulative_sum - bucket_value; + uint64_t right_sum = cumulative_sum; + double pos = 0; + uint64_t right_left_diff = right_sum - left_sum; + if (right_left_diff != 0) { + pos = (threshold - left_sum) / right_left_diff; + } + double r = left_point + (right_point - left_point) * pos; + uint64_t cur_min = min(); + uint64_t cur_max = max(); + if (r < cur_min) r = static_cast(cur_min); + if (r > cur_max) r = static_cast(cur_max); + return r; + } + } + return static_cast(max()); +} + +double HistogramStat::Average() const { + uint64_t cur_num = num(); + uint64_t cur_sum = sum(); + if (cur_num == 0) return 0; + return static_cast(cur_sum) / static_cast(cur_num); +} + +double HistogramStat::StandardDeviation() const { + uint64_t cur_num = num(); + uint64_t cur_sum = sum(); + uint64_t cur_sum_squares = sum_squares(); + if (cur_num == 0) return 0; + double variance = + static_cast(cur_sum_squares * cur_num - cur_sum * cur_sum) / + static_cast(cur_num * cur_num); + return sqrt(variance); +} +std::string HistogramStat::ToString() const { + uint64_t cur_num = num(); + std::string r; + char buf[1650]; + snprintf(buf, sizeof(buf), + "Count: %" PRIu64 " Average: %.4f StdDev: %.2f\n", + cur_num, Average(), StandardDeviation()); + r.append(buf); + snprintf(buf, sizeof(buf), + "Min: %" PRIu64 " Median: %.4f Max: %" PRIu64 "\n", + (cur_num == 0 ? 0 : min()), Median(), (cur_num == 0 ? 0 : max())); + r.append(buf); + snprintf(buf, sizeof(buf), + "Percentiles: " + "P50: %.2f P75: %.2f P99: %.2f P99.9: %.2f P99.99: %.2f\n", + Percentile(50), Percentile(75), Percentile(99), Percentile(99.9), + Percentile(99.99)); + r.append(buf); + r.append("------------------------------------------------------\n"); + const double mult = 100.0 / cur_num; + uint64_t cumulative_sum = 0; + for (unsigned int b = 0; b < num_buckets_; b++) { + uint64_t bucket_value = bucket_at(b); + if (bucket_value <= 0.0) continue; + cumulative_sum += bucket_value; + snprintf(buf, sizeof(buf), + "[ %7" PRIu64 ", %7" PRIu64 " ) %8" PRIu64 " %7.3f%% %7.3f%% ", + (b == 0) ? 0 : bucketMapper.BucketLimit(b-1), // left + bucketMapper.BucketLimit(b), // right + bucket_value, // count + (mult * bucket_value), // percentage + (mult * cumulative_sum)); // cumulative percentage + r.append(buf); + + // Add hash marks based on percentage; 20 marks for 100%. + size_t marks = static_cast(mult * bucket_value / 5 + 0.5); + r.append(marks, '#'); + r.push_back('\n'); + } + return r; +} + +void HistogramStat::Data(HistogramData * const data) const { + assert(data); + data->median = Median(); + data->percentile95 = Percentile(95); + data->percentile99 = Percentile(99); + data->max = static_cast(max()); + data->average = Average(); + data->standard_deviation = StandardDeviation(); +} + +void HistogramImpl::Clear() { + std::lock_guard lock(mutex_); + stats_.Clear(); +} + +bool HistogramImpl::Empty() const { + return stats_.Empty(); +} + +void HistogramImpl::Add(uint64_t value) { + stats_.Add(value); +} + +void HistogramImpl::Merge(const Histogram& other) { + if (strcmp(Name(), other.Name()) == 0) { + Merge( + *static_cast_with_check(&other)); + } +} + +void HistogramImpl::Merge(const HistogramImpl& other) { + std::lock_guard lock(mutex_); + stats_.Merge(other.stats_); +} + +double HistogramImpl::Median() const { + return stats_.Median(); +} + +double HistogramImpl::Percentile(double p) const { + return stats_.Percentile(p); +} + +double HistogramImpl::Average() const { + return stats_.Average(); +} + +double HistogramImpl::StandardDeviation() const { + return stats_.StandardDeviation(); +} + +std::string HistogramImpl::ToString() const { + return stats_.ToString(); +} + +void HistogramImpl::Data(HistogramData * const data) const { + stats_.Data(data); +} + +} // namespace levedb diff --git a/monitoring/histogram.h b/monitoring/histogram.h new file mode 100644 index 00000000000..6bf2e9e93f7 --- /dev/null +++ b/monitoring/histogram.h @@ -0,0 +1,149 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once +#include "rocksdb/statistics.h" + +#include +#include +#include +#include +#include + +namespace rocksdb { + +class HistogramBucketMapper { + public: + + HistogramBucketMapper(); + + // converts a value to the bucket index. + size_t IndexForValue(uint64_t value) const; + // number of buckets required. + + size_t BucketCount() const { + return bucketValues_.size(); + } + + uint64_t LastValue() const { + return maxBucketValue_; + } + + uint64_t FirstValue() const { + return minBucketValue_; + } + + uint64_t BucketLimit(const size_t bucketNumber) const { + assert(bucketNumber < BucketCount()); + return bucketValues_[bucketNumber]; + } + + private: + std::vector bucketValues_; + uint64_t maxBucketValue_; + uint64_t minBucketValue_; + std::map valueIndexMap_; +}; + +struct HistogramStat { + HistogramStat(); + ~HistogramStat() {} + + HistogramStat(const HistogramStat&) = delete; + HistogramStat& operator=(const HistogramStat&) = delete; + + void Clear(); + bool Empty() const; + void Add(uint64_t value); + void Merge(const HistogramStat& other); + + inline uint64_t min() const { return min_.load(std::memory_order_relaxed); } + inline uint64_t max() const { return max_.load(std::memory_order_relaxed); } + inline uint64_t num() const { return num_.load(std::memory_order_relaxed); } + inline uint64_t sum() const { return sum_.load(std::memory_order_relaxed); } + inline uint64_t sum_squares() const { + return sum_squares_.load(std::memory_order_relaxed); + } + inline uint64_t bucket_at(size_t b) const { + return buckets_[b].load(std::memory_order_relaxed); + } + + double Median() const; + double Percentile(double p) const; + double Average() const; + double StandardDeviation() const; + void Data(HistogramData* const data) const; + std::string ToString() const; + + // To be able to use HistogramStat as thread local variable, it + // cannot have dynamic allocated member. That's why we're + // using manually values from BucketMapper + std::atomic_uint_fast64_t min_; + std::atomic_uint_fast64_t max_; + std::atomic_uint_fast64_t num_; + std::atomic_uint_fast64_t sum_; + std::atomic_uint_fast64_t sum_squares_; + std::atomic_uint_fast64_t buckets_[109]; // 109==BucketMapper::BucketCount() + const uint64_t num_buckets_; +}; + +class Histogram { +public: + Histogram() {} + virtual ~Histogram() {}; + + virtual void Clear() = 0; + virtual bool Empty() const = 0; + virtual void Add(uint64_t value) = 0; + virtual void Merge(const Histogram&) = 0; + + virtual std::string ToString() const = 0; + virtual const char* Name() const = 0; + virtual uint64_t min() const = 0; + virtual uint64_t max() const = 0; + virtual uint64_t num() const = 0; + virtual double Median() const = 0; + virtual double Percentile(double p) const = 0; + virtual double Average() const = 0; + virtual double StandardDeviation() const = 0; + virtual void Data(HistogramData* const data) const = 0; +}; + +class HistogramImpl : public Histogram { + public: + HistogramImpl() { Clear(); } + + HistogramImpl(const HistogramImpl&) = delete; + HistogramImpl& operator=(const HistogramImpl&) = delete; + + virtual void Clear() override; + virtual bool Empty() const override; + virtual void Add(uint64_t value) override; + virtual void Merge(const Histogram& other) override; + void Merge(const HistogramImpl& other); + + virtual std::string ToString() const override; + virtual const char* Name() const override { return "HistogramImpl"; } + virtual uint64_t min() const override { return stats_.min(); } + virtual uint64_t max() const override { return stats_.max(); } + virtual uint64_t num() const override { return stats_.num(); } + virtual double Median() const override; + virtual double Percentile(double p) const override; + virtual double Average() const override; + virtual double StandardDeviation() const override; + virtual void Data(HistogramData* const data) const override; + + virtual ~HistogramImpl() {} + + private: + HistogramStat stats_; + std::mutex mutex_; +}; + +} // namespace rocksdb diff --git a/monitoring/histogram_test.cc b/monitoring/histogram_test.cc new file mode 100644 index 00000000000..b4e3c981c8e --- /dev/null +++ b/monitoring/histogram_test.cc @@ -0,0 +1,208 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#include + +#include "monitoring/histogram.h" +#include "monitoring/histogram_windowing.h" +#include "util/testharness.h" + +namespace rocksdb { + +class HistogramTest : public testing::Test {}; + +namespace { + const double kIota = 0.1; + const HistogramBucketMapper bucketMapper; + Env* env = Env::Default(); +} + +void PopulateHistogram(Histogram& histogram, + uint64_t low, uint64_t high, uint64_t loop = 1) { + for (; loop > 0; loop--) { + for (uint64_t i = low; i <= high; i++) { + histogram.Add(i); + } + } +} + +void BasicOperation(Histogram& histogram) { + PopulateHistogram(histogram, 1, 110, 10); // fill up to bucket [70, 110) + + HistogramData data; + histogram.Data(&data); + + ASSERT_LE(fabs(histogram.Percentile(100.0) - 110.0), kIota); + ASSERT_LE(fabs(data.percentile99 - 108.9), kIota); // 99 * 110 / 100 + ASSERT_LE(fabs(data.percentile95 - 104.5), kIota); // 95 * 110 / 100 + ASSERT_LE(fabs(data.median - 55.0), kIota); // 50 * 110 / 100 + ASSERT_EQ(data.average, 55.5); // (1 + 110) / 2 +} + +void MergeHistogram(Histogram& histogram, Histogram& other) { + PopulateHistogram(histogram, 1, 100); + PopulateHistogram(other, 101, 250); + histogram.Merge(other); + + HistogramData data; + histogram.Data(&data); + + ASSERT_LE(fabs(histogram.Percentile(100.0) - 250.0), kIota); + ASSERT_LE(fabs(data.percentile99 - 247.5), kIota); // 99 * 250 / 100 + ASSERT_LE(fabs(data.percentile95 - 237.5), kIota); // 95 * 250 / 100 + ASSERT_LE(fabs(data.median - 125.0), kIota); // 50 * 250 / 100 + ASSERT_EQ(data.average, 125.5); // (1 + 250) / 2 +} + +void EmptyHistogram(Histogram& histogram) { + ASSERT_EQ(histogram.min(), bucketMapper.LastValue()); + ASSERT_EQ(histogram.max(), 0); + ASSERT_EQ(histogram.num(), 0); + ASSERT_EQ(histogram.Median(), 0.0); + ASSERT_EQ(histogram.Percentile(85.0), 0.0); + ASSERT_EQ(histogram.Average(), 0.0); + ASSERT_EQ(histogram.StandardDeviation(), 0.0); +} + +void ClearHistogram(Histogram& histogram) { + for (uint64_t i = 1; i <= 100; i++) { + histogram.Add(i); + } + histogram.Clear(); + ASSERT_TRUE(histogram.Empty()); + ASSERT_EQ(histogram.Median(), 0); + ASSERT_EQ(histogram.Percentile(85.0), 0); + ASSERT_EQ(histogram.Average(), 0); +} + +TEST_F(HistogramTest, BasicOperation) { + HistogramImpl histogram; + BasicOperation(histogram); + + HistogramWindowingImpl histogramWindowing; + BasicOperation(histogramWindowing); +} + +TEST_F(HistogramTest, MergeHistogram) { + HistogramImpl histogram; + HistogramImpl other; + MergeHistogram(histogram, other); + + HistogramWindowingImpl histogramWindowing; + HistogramWindowingImpl otherWindowing; + MergeHistogram(histogramWindowing, otherWindowing); +} + +TEST_F(HistogramTest, EmptyHistogram) { + HistogramImpl histogram; + EmptyHistogram(histogram); + + HistogramWindowingImpl histogramWindowing; + EmptyHistogram(histogramWindowing); +} + +TEST_F(HistogramTest, ClearHistogram) { + HistogramImpl histogram; + ClearHistogram(histogram); + + HistogramWindowingImpl histogramWindowing; + ClearHistogram(histogramWindowing); +} + +TEST_F(HistogramTest, HistogramWindowingExpire) { + uint64_t num_windows = 3; + int micros_per_window = 1000000; + uint64_t min_num_per_window = 0; + + HistogramWindowingImpl + histogramWindowing(num_windows, micros_per_window, min_num_per_window); + + PopulateHistogram(histogramWindowing, 1, 1, 100); + env->SleepForMicroseconds(micros_per_window); + ASSERT_EQ(histogramWindowing.num(), 100); + ASSERT_EQ(histogramWindowing.min(), 1); + ASSERT_EQ(histogramWindowing.max(), 1); + ASSERT_EQ(histogramWindowing.Average(), 1); + + PopulateHistogram(histogramWindowing, 2, 2, 100); + env->SleepForMicroseconds(micros_per_window); + ASSERT_EQ(histogramWindowing.num(), 200); + ASSERT_EQ(histogramWindowing.min(), 1); + ASSERT_EQ(histogramWindowing.max(), 2); + ASSERT_EQ(histogramWindowing.Average(), 1.5); + + PopulateHistogram(histogramWindowing, 3, 3, 100); + env->SleepForMicroseconds(micros_per_window); + ASSERT_EQ(histogramWindowing.num(), 300); + ASSERT_EQ(histogramWindowing.min(), 1); + ASSERT_EQ(histogramWindowing.max(), 3); + ASSERT_EQ(histogramWindowing.Average(), 2.0); + + // dropping oldest window with value 1, remaining 2 ~ 4 + PopulateHistogram(histogramWindowing, 4, 4, 100); + env->SleepForMicroseconds(micros_per_window); + ASSERT_EQ(histogramWindowing.num(), 300); + ASSERT_EQ(histogramWindowing.min(), 2); + ASSERT_EQ(histogramWindowing.max(), 4); + ASSERT_EQ(histogramWindowing.Average(), 3.0); + + // dropping oldest window with value 2, remaining 3 ~ 5 + PopulateHistogram(histogramWindowing, 5, 5, 100); + env->SleepForMicroseconds(micros_per_window); + ASSERT_EQ(histogramWindowing.num(), 300); + ASSERT_EQ(histogramWindowing.min(), 3); + ASSERT_EQ(histogramWindowing.max(), 5); + ASSERT_EQ(histogramWindowing.Average(), 4.0); +} + +TEST_F(HistogramTest, HistogramWindowingMerge) { + uint64_t num_windows = 3; + int micros_per_window = 1000000; + uint64_t min_num_per_window = 0; + + HistogramWindowingImpl + histogramWindowing(num_windows, micros_per_window, min_num_per_window); + HistogramWindowingImpl + otherWindowing(num_windows, micros_per_window, min_num_per_window); + + PopulateHistogram(histogramWindowing, 1, 1, 100); + PopulateHistogram(otherWindowing, 1, 1, 100); + env->SleepForMicroseconds(micros_per_window); + + PopulateHistogram(histogramWindowing, 2, 2, 100); + PopulateHistogram(otherWindowing, 2, 2, 100); + env->SleepForMicroseconds(micros_per_window); + + PopulateHistogram(histogramWindowing, 3, 3, 100); + PopulateHistogram(otherWindowing, 3, 3, 100); + env->SleepForMicroseconds(micros_per_window); + + histogramWindowing.Merge(otherWindowing); + ASSERT_EQ(histogramWindowing.num(), 600); + ASSERT_EQ(histogramWindowing.min(), 1); + ASSERT_EQ(histogramWindowing.max(), 3); + ASSERT_EQ(histogramWindowing.Average(), 2.0); + + // dropping oldest window with value 1, remaining 2 ~ 4 + PopulateHistogram(histogramWindowing, 4, 4, 100); + env->SleepForMicroseconds(micros_per_window); + ASSERT_EQ(histogramWindowing.num(), 500); + ASSERT_EQ(histogramWindowing.min(), 2); + ASSERT_EQ(histogramWindowing.max(), 4); + + // dropping oldest window with value 2, remaining 3 ~ 5 + PopulateHistogram(histogramWindowing, 5, 5, 100); + env->SleepForMicroseconds(micros_per_window); + ASSERT_EQ(histogramWindowing.num(), 400); + ASSERT_EQ(histogramWindowing.min(), 3); + ASSERT_EQ(histogramWindowing.max(), 5); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/monitoring/histogram_windowing.cc b/monitoring/histogram_windowing.cc new file mode 100644 index 00000000000..28d8265f263 --- /dev/null +++ b/monitoring/histogram_windowing.cc @@ -0,0 +1,197 @@ +// Copyright (c) 2013, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "monitoring/histogram_windowing.h" +#include "monitoring/histogram.h" +#include "util/cast_util.h" + +#include + +namespace rocksdb { + +HistogramWindowingImpl::HistogramWindowingImpl() { + env_ = Env::Default(); + window_stats_.reset(new HistogramStat[num_windows_]); + Clear(); +} + +HistogramWindowingImpl::HistogramWindowingImpl( + uint64_t num_windows, + uint64_t micros_per_window, + uint64_t min_num_per_window) : + num_windows_(num_windows), + micros_per_window_(micros_per_window), + min_num_per_window_(min_num_per_window) { + env_ = Env::Default(); + window_stats_.reset(new HistogramStat[num_windows_]); + Clear(); +} + +HistogramWindowingImpl::~HistogramWindowingImpl() { +} + +void HistogramWindowingImpl::Clear() { + std::lock_guard lock(mutex_); + + stats_.Clear(); + for (size_t i = 0; i < num_windows_; i++) { + window_stats_[i].Clear(); + } + current_window_.store(0, std::memory_order_relaxed); + last_swap_time_.store(env_->NowMicros(), std::memory_order_relaxed); +} + +bool HistogramWindowingImpl::Empty() const { return stats_.Empty(); } + +// This function is designed to be lock free, as it's in the critical path +// of any operation. +// Each individual value is atomic, it is just that some samples can go +// in the older bucket which is tolerable. +void HistogramWindowingImpl::Add(uint64_t value){ + TimerTick(); + + // Parent (global) member update + stats_.Add(value); + + // Current window update + window_stats_[current_window()].Add(value); +} + +void HistogramWindowingImpl::Merge(const Histogram& other) { + if (strcmp(Name(), other.Name()) == 0) { + Merge( + *static_cast_with_check( + &other)); + } +} + +void HistogramWindowingImpl::Merge(const HistogramWindowingImpl& other) { + std::lock_guard lock(mutex_); + stats_.Merge(other.stats_); + + if (stats_.num_buckets_ != other.stats_.num_buckets_ || + micros_per_window_ != other.micros_per_window_) { + return; + } + + uint64_t cur_window = current_window(); + uint64_t other_cur_window = other.current_window(); + // going backwards for alignment + for (unsigned int i = 0; + i < std::min(num_windows_, other.num_windows_); i++) { + uint64_t window_index = + (cur_window + num_windows_ - i) % num_windows_; + uint64_t other_window_index = + (other_cur_window + other.num_windows_ - i) % other.num_windows_; + + window_stats_[window_index].Merge(other.window_stats_[other_window_index]); + } +} + +std::string HistogramWindowingImpl::ToString() const { + return stats_.ToString(); +} + +double HistogramWindowingImpl::Median() const { + return Percentile(50.0); +} + +double HistogramWindowingImpl::Percentile(double p) const { + // Retry 3 times in total + for (int retry = 0; retry < 3; retry++) { + uint64_t start_num = stats_.num(); + double result = stats_.Percentile(p); + // Detect if swap buckets or Clear() was called during calculation + if (stats_.num() >= start_num) { + return result; + } + } + return 0.0; +} + +double HistogramWindowingImpl::Average() const { + return stats_.Average(); +} + +double HistogramWindowingImpl::StandardDeviation() const { + return stats_.StandardDeviation(); +} + +void HistogramWindowingImpl::Data(HistogramData * const data) const { + stats_.Data(data); +} + +void HistogramWindowingImpl::TimerTick() { + uint64_t curr_time = env_->NowMicros(); + if (curr_time - last_swap_time() > micros_per_window_ && + window_stats_[current_window()].num() >= min_num_per_window_) { + SwapHistoryBucket(); + } +} + +void HistogramWindowingImpl::SwapHistoryBucket() { + // Threads executing Add() would be competing for this mutex, the first one + // who got the metex would take care of the bucket swap, other threads + // can skip this. + // If mutex is held by Merge() or Clear(), next Add() will take care of the + // swap, if needed. + if (mutex_.try_lock()) { + last_swap_time_.store(env_->NowMicros(), std::memory_order_relaxed); + + uint64_t curr_window = current_window(); + uint64_t next_window = (curr_window == num_windows_ - 1) ? + 0 : curr_window + 1; + + // subtract next buckets from totals and swap to next buckets + HistogramStat& stats_to_drop = window_stats_[next_window]; + + if (!stats_to_drop.Empty()) { + for (size_t b = 0; b < stats_.num_buckets_; b++){ + stats_.buckets_[b].fetch_sub( + stats_to_drop.bucket_at(b), std::memory_order_relaxed); + } + + if (stats_.min() == stats_to_drop.min()) { + uint64_t new_min = std::numeric_limits::max(); + for (unsigned int i = 0; i < num_windows_; i++) { + if (i != next_window) { + uint64_t m = window_stats_[i].min(); + if (m < new_min) new_min = m; + } + } + stats_.min_.store(new_min, std::memory_order_relaxed); + } + + if (stats_.max() == stats_to_drop.max()) { + uint64_t new_max = 0; + for (unsigned int i = 0; i < num_windows_; i++) { + if (i != next_window) { + uint64_t m = window_stats_[i].max(); + if (m > new_max) new_max = m; + } + } + stats_.max_.store(new_max, std::memory_order_relaxed); + } + + stats_.num_.fetch_sub(stats_to_drop.num(), std::memory_order_relaxed); + stats_.sum_.fetch_sub(stats_to_drop.sum(), std::memory_order_relaxed); + stats_.sum_squares_.fetch_sub( + stats_to_drop.sum_squares(), std::memory_order_relaxed); + + stats_to_drop.Clear(); + } + + // advance to next window bucket + current_window_.store(next_window, std::memory_order_relaxed); + + mutex_.unlock(); + } +} + +} // namespace rocksdb diff --git a/monitoring/histogram_windowing.h b/monitoring/histogram_windowing.h new file mode 100644 index 00000000000..2a6d0dd1587 --- /dev/null +++ b/monitoring/histogram_windowing.h @@ -0,0 +1,80 @@ +// Copyright (c) 2013, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include "monitoring/histogram.h" +#include "rocksdb/env.h" + +namespace rocksdb { + +class HistogramWindowingImpl : public Histogram +{ +public: + HistogramWindowingImpl(); + HistogramWindowingImpl(uint64_t num_windows, + uint64_t micros_per_window, + uint64_t min_num_per_window); + + HistogramWindowingImpl(const HistogramImpl&) = delete; + HistogramWindowingImpl& operator=(const HistogramImpl&) = delete; + + ~HistogramWindowingImpl(); + + virtual void Clear() override; + virtual bool Empty() const override; + virtual void Add(uint64_t value) override; + virtual void Merge(const Histogram& other) override; + void Merge(const HistogramWindowingImpl& other); + + virtual std::string ToString() const override; + virtual const char* Name() const override { return "HistogramWindowingImpl"; } + virtual uint64_t min() const override { return stats_.min(); } + virtual uint64_t max() const override { return stats_.max(); } + virtual uint64_t num() const override { return stats_.num(); } + virtual double Median() const override; + virtual double Percentile(double p) const override; + virtual double Average() const override; + virtual double StandardDeviation() const override; + virtual void Data(HistogramData* const data) const override; + +private: + void TimerTick(); + void SwapHistoryBucket(); + inline uint64_t current_window() const { + return current_window_.load(std::memory_order_relaxed); + } + inline uint64_t last_swap_time() const{ + return last_swap_time_.load(std::memory_order_relaxed); + } + + Env* env_; + std::mutex mutex_; + + // Aggregated stats over windows_stats_, all the computation is done + // upon aggregated values + HistogramStat stats_; + + // This is a circular array representing the latest N time-windows. + // Each entry stores a time-window of data. Expiration is done + // on window-based. + std::unique_ptr window_stats_; + + std::atomic_uint_fast64_t current_window_; + std::atomic_uint_fast64_t last_swap_time_; + + // Following parameters are configuable + uint64_t num_windows_ = 5; + uint64_t micros_per_window_ = 60000000; + // By default, don't care about the number of values in current window + // when decide whether to swap windows or not. + uint64_t min_num_per_window_ = 0; +}; + +} // namespace rocksdb \ No newline at end of file diff --git a/monitoring/instrumented_mutex.cc b/monitoring/instrumented_mutex.cc new file mode 100644 index 00000000000..c07a5a17a8a --- /dev/null +++ b/monitoring/instrumented_mutex.cc @@ -0,0 +1,91 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "monitoring/instrumented_mutex.h" +#include "monitoring/perf_context_imp.h" +#include "monitoring/thread_status_util.h" +#include "util/sync_point.h" + +namespace rocksdb { +namespace { +bool ShouldReportToStats(Env* env, Statistics* stats) { + return env != nullptr && stats != nullptr && + stats->stats_level_ > kExceptTimeForMutex; +} +} // namespace + +void InstrumentedMutex::Lock() { + PERF_CONDITIONAL_TIMER_FOR_MUTEX_GUARD(db_mutex_lock_nanos, + stats_code_ == DB_MUTEX_WAIT_MICROS); + uint64_t wait_time_micros = 0; + if (ShouldReportToStats(env_, stats_)) { + { + StopWatch sw(env_, nullptr, 0, &wait_time_micros); + LockInternal(); + } + RecordTick(stats_, stats_code_, wait_time_micros); + } else { + LockInternal(); + } +} + +void InstrumentedMutex::LockInternal() { +#ifndef NDEBUG + ThreadStatusUtil::TEST_StateDelay(ThreadStatus::STATE_MUTEX_WAIT); +#endif + mutex_.Lock(); +} + +void InstrumentedCondVar::Wait() { + PERF_CONDITIONAL_TIMER_FOR_MUTEX_GUARD(db_condition_wait_nanos, + stats_code_ == DB_MUTEX_WAIT_MICROS); + uint64_t wait_time_micros = 0; + if (ShouldReportToStats(env_, stats_)) { + { + StopWatch sw(env_, nullptr, 0, &wait_time_micros); + WaitInternal(); + } + RecordTick(stats_, stats_code_, wait_time_micros); + } else { + WaitInternal(); + } +} + +void InstrumentedCondVar::WaitInternal() { +#ifndef NDEBUG + ThreadStatusUtil::TEST_StateDelay(ThreadStatus::STATE_MUTEX_WAIT); +#endif + cond_.Wait(); +} + +bool InstrumentedCondVar::TimedWait(uint64_t abs_time_us) { + PERF_CONDITIONAL_TIMER_FOR_MUTEX_GUARD(db_condition_wait_nanos, + stats_code_ == DB_MUTEX_WAIT_MICROS); + uint64_t wait_time_micros = 0; + bool result = false; + if (ShouldReportToStats(env_, stats_)) { + { + StopWatch sw(env_, nullptr, 0, &wait_time_micros); + result = TimedWaitInternal(abs_time_us); + } + RecordTick(stats_, stats_code_, wait_time_micros); + } else { + result = TimedWaitInternal(abs_time_us); + } + return result; +} + +bool InstrumentedCondVar::TimedWaitInternal(uint64_t abs_time_us) { +#ifndef NDEBUG + ThreadStatusUtil::TEST_StateDelay(ThreadStatus::STATE_MUTEX_WAIT); +#endif + + TEST_SYNC_POINT_CALLBACK("InstrumentedCondVar::TimedWaitInternal", + &abs_time_us); + + return cond_.TimedWait(abs_time_us); +} + +} // namespace rocksdb diff --git a/monitoring/instrumented_mutex.h b/monitoring/instrumented_mutex.h new file mode 100644 index 00000000000..83d7523ef31 --- /dev/null +++ b/monitoring/instrumented_mutex.h @@ -0,0 +1,98 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include "monitoring/statistics.h" +#include "port/port.h" +#include "rocksdb/env.h" +#include "rocksdb/statistics.h" +#include "rocksdb/thread_status.h" +#include "util/stop_watch.h" + +namespace rocksdb { +class InstrumentedCondVar; + +// A wrapper class for port::Mutex that provides additional layer +// for collecting stats and instrumentation. +class InstrumentedMutex { + public: + explicit InstrumentedMutex(bool adaptive = false) + : mutex_(adaptive), stats_(nullptr), env_(nullptr), + stats_code_(0) {} + + InstrumentedMutex( + Statistics* stats, Env* env, + int stats_code, bool adaptive = false) + : mutex_(adaptive), stats_(stats), env_(env), + stats_code_(stats_code) {} + + void Lock(); + + void Unlock() { + mutex_.Unlock(); + } + + void AssertHeld() { + mutex_.AssertHeld(); + } + + private: + void LockInternal(); + friend class InstrumentedCondVar; + port::Mutex mutex_; + Statistics* stats_; + Env* env_; + int stats_code_; +}; + +// A wrapper class for port::Mutex that provides additional layer +// for collecting stats and instrumentation. +class InstrumentedMutexLock { + public: + explicit InstrumentedMutexLock(InstrumentedMutex* mutex) : mutex_(mutex) { + mutex_->Lock(); + } + + ~InstrumentedMutexLock() { + mutex_->Unlock(); + } + + private: + InstrumentedMutex* const mutex_; + InstrumentedMutexLock(const InstrumentedMutexLock&) = delete; + void operator=(const InstrumentedMutexLock&) = delete; +}; + +class InstrumentedCondVar { + public: + explicit InstrumentedCondVar(InstrumentedMutex* instrumented_mutex) + : cond_(&(instrumented_mutex->mutex_)), + stats_(instrumented_mutex->stats_), + env_(instrumented_mutex->env_), + stats_code_(instrumented_mutex->stats_code_) {} + + void Wait(); + + bool TimedWait(uint64_t abs_time_us); + + void Signal() { + cond_.Signal(); + } + + void SignalAll() { + cond_.SignalAll(); + } + + private: + void WaitInternal(); + bool TimedWaitInternal(uint64_t abs_time_us); + port::CondVar cond_; + Statistics* stats_; + Env* env_; + int stats_code_; +}; + +} // namespace rocksdb diff --git a/monitoring/iostats_context.cc b/monitoring/iostats_context.cc new file mode 100644 index 00000000000..8aa131a7042 --- /dev/null +++ b/monitoring/iostats_context.cc @@ -0,0 +1,61 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include +#include "monitoring/iostats_context_imp.h" +#include "rocksdb/env.h" +#include "util/thread_local.h" + +namespace rocksdb { + +#ifdef ROCKSDB_SUPPORT_THREAD_LOCAL +__thread IOStatsContext iostats_context; +#endif + +IOStatsContext* get_iostats_context() { +#ifdef ROCKSDB_SUPPORT_THREAD_LOCAL + return &iostats_context; +#else + return nullptr; +#endif +} + +void IOStatsContext::Reset() { + thread_pool_id = Env::Priority::TOTAL; + bytes_read = 0; + bytes_written = 0; + open_nanos = 0; + allocate_nanos = 0; + write_nanos = 0; + read_nanos = 0; + range_sync_nanos = 0; + prepare_write_nanos = 0; + fsync_nanos = 0; + logger_nanos = 0; +} + +#define IOSTATS_CONTEXT_OUTPUT(counter) \ + if (!exclude_zero_counters || counter > 0) { \ + ss << #counter << " = " << counter << ", "; \ + } + +std::string IOStatsContext::ToString(bool exclude_zero_counters) const { + std::ostringstream ss; + IOSTATS_CONTEXT_OUTPUT(thread_pool_id); + IOSTATS_CONTEXT_OUTPUT(bytes_read); + IOSTATS_CONTEXT_OUTPUT(bytes_written); + IOSTATS_CONTEXT_OUTPUT(open_nanos); + IOSTATS_CONTEXT_OUTPUT(allocate_nanos); + IOSTATS_CONTEXT_OUTPUT(write_nanos); + IOSTATS_CONTEXT_OUTPUT(read_nanos); + IOSTATS_CONTEXT_OUTPUT(range_sync_nanos); + IOSTATS_CONTEXT_OUTPUT(fsync_nanos); + IOSTATS_CONTEXT_OUTPUT(prepare_write_nanos); + IOSTATS_CONTEXT_OUTPUT(logger_nanos); + + return ss.str(); +} + +} // namespace rocksdb diff --git a/monitoring/iostats_context_imp.h b/monitoring/iostats_context_imp.h new file mode 100644 index 00000000000..88538297a60 --- /dev/null +++ b/monitoring/iostats_context_imp.h @@ -0,0 +1,54 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#pragma once +#include "monitoring/perf_step_timer.h" +#include "rocksdb/iostats_context.h" + +#ifdef ROCKSDB_SUPPORT_THREAD_LOCAL + +// increment a specific counter by the specified value +#define IOSTATS_ADD(metric, value) \ + (get_iostats_context()->metric += value) + +// Increase metric value only when it is positive +#define IOSTATS_ADD_IF_POSITIVE(metric, value) \ + if (value > 0) { IOSTATS_ADD(metric, value); } + +// reset a specific counter to zero +#define IOSTATS_RESET(metric) \ + (get_iostats_context()->metric = 0) + +// reset all counters to zero +#define IOSTATS_RESET_ALL() \ + (get_iostats_context()->Reset()) + +#define IOSTATS_SET_THREAD_POOL_ID(value) \ + (get_iostats_context()->thread_pool_id = value) + +#define IOSTATS_THREAD_POOL_ID() \ + (get_iostats_context()->thread_pool_id) + +#define IOSTATS(metric) \ + (get_iostats_context()->metric) + +// Declare and set start time of the timer +#define IOSTATS_TIMER_GUARD(metric) \ + PerfStepTimer iostats_step_timer_##metric(&(get_iostats_context()->metric)); \ + iostats_step_timer_##metric.Start(); + +#else // ROCKSDB_SUPPORT_THREAD_LOCAL + +#define IOSTATS_ADD(metric, value) +#define IOSTATS_ADD_IF_POSITIVE(metric, value) +#define IOSTATS_RESET(metric) +#define IOSTATS_RESET_ALL() +#define IOSTATS_SET_THREAD_POOL_ID(value) +#define IOSTATS_THREAD_POOL_ID() +#define IOSTATS(metric) 0 + +#define IOSTATS_TIMER_GUARD(metric) + +#endif // ROCKSDB_SUPPORT_THREAD_LOCAL diff --git a/monitoring/iostats_context_test.cc b/monitoring/iostats_context_test.cc new file mode 100644 index 00000000000..74d3e43291d --- /dev/null +++ b/monitoring/iostats_context_test.cc @@ -0,0 +1,29 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "rocksdb/iostats_context.h" +#include "util/testharness.h" + +namespace rocksdb { + +TEST(IOStatsContextTest, ToString) { + get_iostats_context()->Reset(); + get_iostats_context()->bytes_read = 12345; + + std::string zero_included = get_iostats_context()->ToString(); + ASSERT_NE(std::string::npos, zero_included.find("= 0")); + ASSERT_NE(std::string::npos, zero_included.find("= 12345")); + + std::string zero_excluded = get_iostats_context()->ToString(true); + ASSERT_EQ(std::string::npos, zero_excluded.find("= 0")); + ASSERT_NE(std::string::npos, zero_excluded.find("= 12345")); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/monitoring/perf_context.cc b/monitoring/perf_context.cc new file mode 100644 index 00000000000..791f4bdbe4e --- /dev/null +++ b/monitoring/perf_context.cc @@ -0,0 +1,185 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// + +#include +#include "monitoring/perf_context_imp.h" + +namespace rocksdb { + +#if defined(NPERF_CONTEXT) || !defined(ROCKSDB_SUPPORT_THREAD_LOCAL) +PerfContext perf_context; +#else +#if defined(OS_SOLARIS) +__thread PerfContext perf_context_; +#else +__thread PerfContext perf_context; +#endif +#endif + +PerfContext* get_perf_context() { +#if defined(NPERF_CONTEXT) || !defined(ROCKSDB_SUPPORT_THREAD_LOCAL) + return &perf_context; +#else +#if defined(OS_SOLARIS) + return &perf_context_; +#else + return &perf_context; +#endif +#endif +} + +void PerfContext::Reset() { +#ifndef NPERF_CONTEXT + user_key_comparison_count = 0; + block_cache_hit_count = 0; + block_read_count = 0; + block_read_byte = 0; + block_read_time = 0; + block_checksum_time = 0; + block_decompress_time = 0; + get_read_bytes = 0; + multiget_read_bytes = 0; + iter_read_bytes = 0; + internal_key_skipped_count = 0; + internal_delete_skipped_count = 0; + internal_recent_skipped_count = 0; + internal_merge_count = 0; + write_wal_time = 0; + + get_snapshot_time = 0; + get_from_memtable_time = 0; + get_from_memtable_count = 0; + get_post_process_time = 0; + get_from_output_files_time = 0; + seek_on_memtable_time = 0; + seek_on_memtable_count = 0; + next_on_memtable_count = 0; + prev_on_memtable_count = 0; + seek_child_seek_time = 0; + seek_child_seek_count = 0; + seek_min_heap_time = 0; + seek_internal_seek_time = 0; + find_next_user_entry_time = 0; + write_pre_and_post_process_time = 0; + write_memtable_time = 0; + write_delay_time = 0; + db_mutex_lock_nanos = 0; + db_condition_wait_nanos = 0; + merge_operator_time_nanos = 0; + read_index_block_nanos = 0; + read_filter_block_nanos = 0; + new_table_block_iter_nanos = 0; + new_table_iterator_nanos = 0; + block_seek_nanos = 0; + find_table_nanos = 0; + bloom_memtable_hit_count = 0; + bloom_memtable_miss_count = 0; + bloom_sst_hit_count = 0; + bloom_sst_miss_count = 0; + + env_new_sequential_file_nanos = 0; + env_new_random_access_file_nanos = 0; + env_new_writable_file_nanos = 0; + env_reuse_writable_file_nanos = 0; + env_new_random_rw_file_nanos = 0; + env_new_directory_nanos = 0; + env_file_exists_nanos = 0; + env_get_children_nanos = 0; + env_get_children_file_attributes_nanos = 0; + env_delete_file_nanos = 0; + env_create_dir_nanos = 0; + env_create_dir_if_missing_nanos = 0; + env_delete_dir_nanos = 0; + env_get_file_size_nanos = 0; + env_get_file_modification_time_nanos = 0; + env_rename_file_nanos = 0; + env_link_file_nanos = 0; + env_lock_file_nanos = 0; + env_unlock_file_nanos = 0; + env_new_logger_nanos = 0; +#endif +} + +#define PERF_CONTEXT_OUTPUT(counter) \ + if (!exclude_zero_counters || (counter > 0)) { \ + ss << #counter << " = " << counter << ", "; \ + } + +std::string PerfContext::ToString(bool exclude_zero_counters) const { +#ifdef NPERF_CONTEXT + return ""; +#else + std::ostringstream ss; + PERF_CONTEXT_OUTPUT(user_key_comparison_count); + PERF_CONTEXT_OUTPUT(block_cache_hit_count); + PERF_CONTEXT_OUTPUT(block_read_count); + PERF_CONTEXT_OUTPUT(block_read_byte); + PERF_CONTEXT_OUTPUT(block_read_time); + PERF_CONTEXT_OUTPUT(block_checksum_time); + PERF_CONTEXT_OUTPUT(block_decompress_time); + PERF_CONTEXT_OUTPUT(get_read_bytes); + PERF_CONTEXT_OUTPUT(multiget_read_bytes); + PERF_CONTEXT_OUTPUT(iter_read_bytes); + PERF_CONTEXT_OUTPUT(internal_key_skipped_count); + PERF_CONTEXT_OUTPUT(internal_delete_skipped_count); + PERF_CONTEXT_OUTPUT(internal_recent_skipped_count); + PERF_CONTEXT_OUTPUT(internal_merge_count); + PERF_CONTEXT_OUTPUT(write_wal_time); + PERF_CONTEXT_OUTPUT(get_snapshot_time); + PERF_CONTEXT_OUTPUT(get_from_memtable_time); + PERF_CONTEXT_OUTPUT(get_from_memtable_count); + PERF_CONTEXT_OUTPUT(get_post_process_time); + PERF_CONTEXT_OUTPUT(get_from_output_files_time); + PERF_CONTEXT_OUTPUT(seek_on_memtable_time); + PERF_CONTEXT_OUTPUT(seek_on_memtable_count); + PERF_CONTEXT_OUTPUT(next_on_memtable_count); + PERF_CONTEXT_OUTPUT(prev_on_memtable_count); + PERF_CONTEXT_OUTPUT(seek_child_seek_time); + PERF_CONTEXT_OUTPUT(seek_child_seek_count); + PERF_CONTEXT_OUTPUT(seek_min_heap_time); + PERF_CONTEXT_OUTPUT(seek_internal_seek_time); + PERF_CONTEXT_OUTPUT(find_next_user_entry_time); + PERF_CONTEXT_OUTPUT(write_pre_and_post_process_time); + PERF_CONTEXT_OUTPUT(write_memtable_time); + PERF_CONTEXT_OUTPUT(db_mutex_lock_nanos); + PERF_CONTEXT_OUTPUT(db_condition_wait_nanos); + PERF_CONTEXT_OUTPUT(merge_operator_time_nanos); + PERF_CONTEXT_OUTPUT(write_delay_time); + PERF_CONTEXT_OUTPUT(read_index_block_nanos); + PERF_CONTEXT_OUTPUT(read_filter_block_nanos); + PERF_CONTEXT_OUTPUT(new_table_block_iter_nanos); + PERF_CONTEXT_OUTPUT(new_table_iterator_nanos); + PERF_CONTEXT_OUTPUT(block_seek_nanos); + PERF_CONTEXT_OUTPUT(find_table_nanos); + PERF_CONTEXT_OUTPUT(bloom_memtable_hit_count); + PERF_CONTEXT_OUTPUT(bloom_memtable_miss_count); + PERF_CONTEXT_OUTPUT(bloom_sst_hit_count); + PERF_CONTEXT_OUTPUT(bloom_sst_miss_count); + PERF_CONTEXT_OUTPUT(env_new_sequential_file_nanos); + PERF_CONTEXT_OUTPUT(env_new_random_access_file_nanos); + PERF_CONTEXT_OUTPUT(env_new_writable_file_nanos); + PERF_CONTEXT_OUTPUT(env_reuse_writable_file_nanos); + PERF_CONTEXT_OUTPUT(env_new_random_rw_file_nanos); + PERF_CONTEXT_OUTPUT(env_new_directory_nanos); + PERF_CONTEXT_OUTPUT(env_file_exists_nanos); + PERF_CONTEXT_OUTPUT(env_get_children_nanos); + PERF_CONTEXT_OUTPUT(env_get_children_file_attributes_nanos); + PERF_CONTEXT_OUTPUT(env_delete_file_nanos); + PERF_CONTEXT_OUTPUT(env_create_dir_nanos); + PERF_CONTEXT_OUTPUT(env_create_dir_if_missing_nanos); + PERF_CONTEXT_OUTPUT(env_delete_dir_nanos); + PERF_CONTEXT_OUTPUT(env_get_file_size_nanos); + PERF_CONTEXT_OUTPUT(env_get_file_modification_time_nanos); + PERF_CONTEXT_OUTPUT(env_rename_file_nanos); + PERF_CONTEXT_OUTPUT(env_link_file_nanos); + PERF_CONTEXT_OUTPUT(env_lock_file_nanos); + PERF_CONTEXT_OUTPUT(env_unlock_file_nanos); + PERF_CONTEXT_OUTPUT(env_new_logger_nanos); + return ss.str(); +#endif +} + +} diff --git a/monitoring/perf_context_imp.h b/monitoring/perf_context_imp.h new file mode 100644 index 00000000000..421a8cea15c --- /dev/null +++ b/monitoring/perf_context_imp.h @@ -0,0 +1,52 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#pragma once +#include "monitoring/perf_step_timer.h" +#include "rocksdb/perf_context.h" +#include "util/stop_watch.h" + +namespace rocksdb { + +#if defined(NPERF_CONTEXT) + +#define PERF_TIMER_GUARD(metric) +#define PERF_CONDITIONAL_TIMER_FOR_MUTEX_GUARD(metric, condition) +#define PERF_TIMER_MEASURE(metric) +#define PERF_TIMER_STOP(metric) +#define PERF_TIMER_START(metric) +#define PERF_COUNTER_ADD(metric, value) + +#else + +// Stop the timer and update the metric +#define PERF_TIMER_STOP(metric) perf_step_timer_##metric.Stop(); + +#define PERF_TIMER_START(metric) perf_step_timer_##metric.Start(); + +// Declare and set start time of the timer +#define PERF_TIMER_GUARD(metric) \ + PerfStepTimer perf_step_timer_##metric(&(get_perf_context()->metric)); \ + perf_step_timer_##metric.Start(); + +#define PERF_CONDITIONAL_TIMER_FOR_MUTEX_GUARD(metric, condition) \ + PerfStepTimer perf_step_timer_##metric(&(get_perf_context()->metric), true); \ + if ((condition)) { \ + perf_step_timer_##metric.Start(); \ + } + +// Update metric with time elapsed since last START. start time is reset +// to current timestamp. +#define PERF_TIMER_MEASURE(metric) perf_step_timer_##metric.Measure(); + +// Increase metric value +#define PERF_COUNTER_ADD(metric, value) \ + if (perf_level >= PerfLevel::kEnableCount) { \ + get_perf_context()->metric += value; \ + } + +#endif + +} diff --git a/monitoring/perf_level.cc b/monitoring/perf_level.cc new file mode 100644 index 00000000000..79c718cce76 --- /dev/null +++ b/monitoring/perf_level.cc @@ -0,0 +1,28 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// + +#include +#include "monitoring/perf_level_imp.h" + +namespace rocksdb { + +#ifdef ROCKSDB_SUPPORT_THREAD_LOCAL +__thread PerfLevel perf_level = kEnableCount; +#else +PerfLevel perf_level = kEnableCount; +#endif + +void SetPerfLevel(PerfLevel level) { + assert(level > kUninitialized); + assert(level < kOutOfBounds); + perf_level = level; +} + +PerfLevel GetPerfLevel() { + return perf_level; +} + +} // namespace rocksdb diff --git a/monitoring/perf_level_imp.h b/monitoring/perf_level_imp.h new file mode 100644 index 00000000000..2a3add19cee --- /dev/null +++ b/monitoring/perf_level_imp.h @@ -0,0 +1,18 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#pragma once +#include "rocksdb/perf_level.h" +#include "port/port.h" + +namespace rocksdb { + +#ifdef ROCKSDB_SUPPORT_THREAD_LOCAL +extern __thread PerfLevel perf_level; +#else +extern PerfLevel perf_level; +#endif + +} // namespace rocksdb diff --git a/monitoring/perf_step_timer.h b/monitoring/perf_step_timer.h new file mode 100644 index 00000000000..4cb48b12562 --- /dev/null +++ b/monitoring/perf_step_timer.h @@ -0,0 +1,54 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#pragma once +#include "monitoring/perf_level_imp.h" +#include "rocksdb/env.h" +#include "util/stop_watch.h" + +namespace rocksdb { + +class PerfStepTimer { + public: + explicit PerfStepTimer(uint64_t* metric, bool for_mutex = false) + : enabled_(perf_level >= PerfLevel::kEnableTime || + (!for_mutex && perf_level >= kEnableTimeExceptForMutex)), + env_(enabled_ ? Env::Default() : nullptr), + start_(0), + metric_(metric) {} + + ~PerfStepTimer() { + Stop(); + } + + void Start() { + if (enabled_) { + start_ = env_->NowNanos(); + } + } + + void Measure() { + if (start_) { + uint64_t now = env_->NowNanos(); + *metric_ += now - start_; + start_ = now; + } + } + + void Stop() { + if (start_) { + *metric_ += env_->NowNanos() - start_; + start_ = 0; + } + } + + private: + const bool enabled_; + Env* const env_; + uint64_t start_; + uint64_t* metric_; +}; + +} // namespace rocksdb diff --git a/monitoring/statistics.cc b/monitoring/statistics.cc new file mode 100644 index 00000000000..9387043127d --- /dev/null +++ b/monitoring/statistics.cc @@ -0,0 +1,191 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#include "monitoring/statistics.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include "rocksdb/statistics.h" +#include "port/likely.h" +#include +#include + +namespace rocksdb { + +std::shared_ptr CreateDBStatistics() { + return std::make_shared(nullptr, false); +} + +StatisticsImpl::StatisticsImpl(std::shared_ptr stats, + bool enable_internal_stats) + : stats_(std::move(stats)), enable_internal_stats_(enable_internal_stats) {} + +StatisticsImpl::~StatisticsImpl() {} + +uint64_t StatisticsImpl::getTickerCount(uint32_t tickerType) const { + MutexLock lock(&aggregate_lock_); + return getTickerCountLocked(tickerType); +} + +uint64_t StatisticsImpl::getTickerCountLocked(uint32_t tickerType) const { + assert( + enable_internal_stats_ ? + tickerType < INTERNAL_TICKER_ENUM_MAX : + tickerType < TICKER_ENUM_MAX); + uint64_t res = 0; + for (size_t core_idx = 0; core_idx < per_core_stats_.Size(); ++core_idx) { + res += per_core_stats_.AccessAtCore(core_idx)->tickers_[tickerType]; + } + return res; +} + +void StatisticsImpl::histogramData(uint32_t histogramType, + HistogramData* const data) const { + MutexLock lock(&aggregate_lock_); + getHistogramImplLocked(histogramType)->Data(data); +} + +std::unique_ptr StatisticsImpl::getHistogramImplLocked( + uint32_t histogramType) const { + assert( + enable_internal_stats_ ? + histogramType < INTERNAL_HISTOGRAM_ENUM_MAX : + histogramType < HISTOGRAM_ENUM_MAX); + std::unique_ptr res_hist(new HistogramImpl()); + for (size_t core_idx = 0; core_idx < per_core_stats_.Size(); ++core_idx) { + res_hist->Merge( + per_core_stats_.AccessAtCore(core_idx)->histograms_[histogramType]); + } + return res_hist; +} + +std::string StatisticsImpl::getHistogramString(uint32_t histogramType) const { + MutexLock lock(&aggregate_lock_); + return getHistogramImplLocked(histogramType)->ToString(); +} + +void StatisticsImpl::setTickerCount(uint32_t tickerType, uint64_t count) { + { + MutexLock lock(&aggregate_lock_); + setTickerCountLocked(tickerType, count); + } + if (stats_ && tickerType < TICKER_ENUM_MAX) { + stats_->setTickerCount(tickerType, count); + } +} + +void StatisticsImpl::setTickerCountLocked(uint32_t tickerType, uint64_t count) { + assert(enable_internal_stats_ ? tickerType < INTERNAL_TICKER_ENUM_MAX + : tickerType < TICKER_ENUM_MAX); + for (size_t core_idx = 0; core_idx < per_core_stats_.Size(); ++core_idx) { + if (core_idx == 0) { + per_core_stats_.AccessAtCore(core_idx)->tickers_[tickerType] = count; + } else { + per_core_stats_.AccessAtCore(core_idx)->tickers_[tickerType] = 0; + } + } +} + +uint64_t StatisticsImpl::getAndResetTickerCount(uint32_t tickerType) { + uint64_t sum = 0; + { + MutexLock lock(&aggregate_lock_); + assert(enable_internal_stats_ ? tickerType < INTERNAL_TICKER_ENUM_MAX + : tickerType < TICKER_ENUM_MAX); + for (size_t core_idx = 0; core_idx < per_core_stats_.Size(); ++core_idx) { + sum += + per_core_stats_.AccessAtCore(core_idx)->tickers_[tickerType].exchange( + 0, std::memory_order_relaxed); + } + } + if (stats_ && tickerType < TICKER_ENUM_MAX) { + stats_->setTickerCount(tickerType, 0); + } + return sum; +} + +void StatisticsImpl::recordTick(uint32_t tickerType, uint64_t count) { + assert( + enable_internal_stats_ ? + tickerType < INTERNAL_TICKER_ENUM_MAX : + tickerType < TICKER_ENUM_MAX); + per_core_stats_.Access()->tickers_[tickerType].fetch_add( + count, std::memory_order_relaxed); + if (stats_ && tickerType < TICKER_ENUM_MAX) { + stats_->recordTick(tickerType, count); + } +} + +void StatisticsImpl::measureTime(uint32_t histogramType, uint64_t value) { + assert( + enable_internal_stats_ ? + histogramType < INTERNAL_HISTOGRAM_ENUM_MAX : + histogramType < HISTOGRAM_ENUM_MAX); + per_core_stats_.Access()->histograms_[histogramType].Add(value); + if (stats_ && histogramType < HISTOGRAM_ENUM_MAX) { + stats_->measureTime(histogramType, value); + } +} + +Status StatisticsImpl::Reset() { + MutexLock lock(&aggregate_lock_); + for (uint32_t i = 0; i < TICKER_ENUM_MAX; ++i) { + setTickerCountLocked(i, 0); + } + for (uint32_t i = 0; i < HISTOGRAM_ENUM_MAX; ++i) { + for (size_t core_idx = 0; core_idx < per_core_stats_.Size(); ++core_idx) { + per_core_stats_.AccessAtCore(core_idx)->histograms_[i].Clear(); + } + } + return Status::OK(); +} + +namespace { + +// a buffer size used for temp string buffers +const int kTmpStrBufferSize = 200; + +} // namespace + +std::string StatisticsImpl::ToString() const { + MutexLock lock(&aggregate_lock_); + std::string res; + res.reserve(20000); + for (const auto& t : TickersNameMap) { + if (t.first < TICKER_ENUM_MAX || enable_internal_stats_) { + char buffer[kTmpStrBufferSize]; + snprintf(buffer, kTmpStrBufferSize, "%s COUNT : %" PRIu64 "\n", + t.second.c_str(), getTickerCountLocked(t.first)); + res.append(buffer); + } + } + for (const auto& h : HistogramsNameMap) { + if (h.first < HISTOGRAM_ENUM_MAX || enable_internal_stats_) { + char buffer[kTmpStrBufferSize]; + HistogramData hData; + getHistogramImplLocked(h.first)->Data(&hData); + snprintf( + buffer, kTmpStrBufferSize, + "%s statistics Percentiles :=> 50 : %f 95 : %f 99 : %f 100 : %f\n", + h.second.c_str(), hData.median, hData.percentile95, + hData.percentile99, hData.max); + res.append(buffer); + } + } + res.shrink_to_fit(); + return res; +} + +bool StatisticsImpl::HistEnabledForType(uint32_t type) const { + if (LIKELY(!enable_internal_stats_)) { + return type < HISTOGRAM_ENUM_MAX; + } + return true; +} + +} // namespace rocksdb diff --git a/monitoring/statistics.h b/monitoring/statistics.h new file mode 100644 index 00000000000..6e915215deb --- /dev/null +++ b/monitoring/statistics.h @@ -0,0 +1,115 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +#pragma once +#include "rocksdb/statistics.h" + +#include +#include +#include + +#include "monitoring/histogram.h" +#include "port/likely.h" +#include "port/port.h" +#include "util/core_local.h" +#include "util/mutexlock.h" + +#ifdef __clang__ +#define ROCKSDB_FIELD_UNUSED __attribute__((__unused__)) +#else +#define ROCKSDB_FIELD_UNUSED +#endif // __clang__ + +namespace rocksdb { + +enum TickersInternal : uint32_t { + INTERNAL_TICKER_ENUM_START = TICKER_ENUM_MAX, + INTERNAL_TICKER_ENUM_MAX +}; + +enum HistogramsInternal : uint32_t { + INTERNAL_HISTOGRAM_START = HISTOGRAM_ENUM_MAX, + INTERNAL_HISTOGRAM_ENUM_MAX +}; + + +class StatisticsImpl : public Statistics { + public: + StatisticsImpl(std::shared_ptr stats, + bool enable_internal_stats); + virtual ~StatisticsImpl(); + + virtual uint64_t getTickerCount(uint32_t ticker_type) const override; + virtual void histogramData(uint32_t histogram_type, + HistogramData* const data) const override; + std::string getHistogramString(uint32_t histogram_type) const override; + + virtual void setTickerCount(uint32_t ticker_type, uint64_t count) override; + virtual uint64_t getAndResetTickerCount(uint32_t ticker_type) override; + virtual void recordTick(uint32_t ticker_type, uint64_t count) override; + virtual void measureTime(uint32_t histogram_type, uint64_t value) override; + + virtual Status Reset() override; + virtual std::string ToString() const override; + virtual bool HistEnabledForType(uint32_t type) const override; + + private: + // If non-nullptr, forwards updates to the object pointed to by `stats_`. + std::shared_ptr stats_; + // TODO(ajkr): clean this up since there are no internal stats anymore + bool enable_internal_stats_; + // Synchronizes anything that operates across other cores' local data, + // such that operations like Reset() can be performed atomically. + mutable port::Mutex aggregate_lock_; + + // The ticker/histogram data are stored in this structure, which we will store + // per-core. It is cache-aligned, so tickers/histograms belonging to different + // cores can never share the same cache line. + // + // Alignment attributes expand to nothing depending on the platform + struct StatisticsData { + std::atomic_uint_fast64_t tickers_[INTERNAL_TICKER_ENUM_MAX] = {{0}}; + HistogramImpl histograms_[INTERNAL_HISTOGRAM_ENUM_MAX]; + char + padding[(CACHE_LINE_SIZE - + (INTERNAL_TICKER_ENUM_MAX * sizeof(std::atomic_uint_fast64_t) + + INTERNAL_HISTOGRAM_ENUM_MAX * sizeof(HistogramImpl)) % + CACHE_LINE_SIZE) % + CACHE_LINE_SIZE] ROCKSDB_FIELD_UNUSED; + }; + + static_assert(sizeof(StatisticsData) % 64 == 0, "Expected 64-byte aligned"); + + CoreLocalArray per_core_stats_; + + uint64_t getTickerCountLocked(uint32_t ticker_type) const; + std::unique_ptr getHistogramImplLocked( + uint32_t histogram_type) const; + void setTickerCountLocked(uint32_t ticker_type, uint64_t count); +}; + +// Utility functions +inline void MeasureTime(Statistics* statistics, uint32_t histogram_type, + uint64_t value) { + if (statistics) { + statistics->measureTime(histogram_type, value); + } +} + +inline void RecordTick(Statistics* statistics, uint32_t ticker_type, + uint64_t count = 1) { + if (statistics) { + statistics->recordTick(ticker_type, count); + } +} + +inline void SetTickerCount(Statistics* statistics, uint32_t ticker_type, + uint64_t count) { + if (statistics) { + statistics->setTickerCount(ticker_type, count); + } +} + +} diff --git a/monitoring/statistics_test.cc b/monitoring/statistics_test.cc new file mode 100644 index 00000000000..43aacde9c1b --- /dev/null +++ b/monitoring/statistics_test.cc @@ -0,0 +1,35 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// + +#include "port/stack_trace.h" +#include "util/testharness.h" +#include "util/testutil.h" + +#include "rocksdb/statistics.h" + +namespace rocksdb { + +class StatisticsTest : public testing::Test {}; + +// Sanity check to make sure that contents and order of TickersNameMap +// match Tickers enum +TEST_F(StatisticsTest, Sanity) { + EXPECT_EQ(static_cast(Tickers::TICKER_ENUM_MAX), + TickersNameMap.size()); + + for (uint32_t t = 0; t < Tickers::TICKER_ENUM_MAX; t++) { + auto pair = TickersNameMap[static_cast(t)]; + ASSERT_EQ(pair.first, t) << "Miss match at " << pair.second; + } +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/monitoring/thread_status_impl.cc b/monitoring/thread_status_impl.cc new file mode 100644 index 00000000000..e263ce661ef --- /dev/null +++ b/monitoring/thread_status_impl.cc @@ -0,0 +1,167 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// + +#include + +#include "rocksdb/env.h" +#include "rocksdb/thread_status.h" +#include "util/string_util.h" +#include "util/thread_operation.h" + +namespace rocksdb { + +#ifdef ROCKSDB_USING_THREAD_STATUS +const std::string& ThreadStatus::GetThreadTypeName( + ThreadStatus::ThreadType thread_type) { + static std::string thread_type_names[NUM_THREAD_TYPES + 1] = { + "High Pri", "Low Pri", "User", "Unknown"}; + if (thread_type < 0 || thread_type >= NUM_THREAD_TYPES) { + return thread_type_names[NUM_THREAD_TYPES]; // "Unknown" + } + return thread_type_names[thread_type]; +} + +const std::string& ThreadStatus::GetOperationName( + ThreadStatus::OperationType op_type) { + if (op_type < 0 || op_type >= NUM_OP_TYPES) { + return global_operation_table[OP_UNKNOWN].name; + } + return global_operation_table[op_type].name; +} + +const std::string& ThreadStatus::GetOperationStageName( + ThreadStatus::OperationStage stage) { + if (stage < 0 || stage >= NUM_OP_STAGES) { + return global_op_stage_table[STAGE_UNKNOWN].name; + } + return global_op_stage_table[stage].name; +} + +const std::string& ThreadStatus::GetStateName( + ThreadStatus::StateType state_type) { + if (state_type < 0 || state_type >= NUM_STATE_TYPES) { + return global_state_table[STATE_UNKNOWN].name; + } + return global_state_table[state_type].name; +} + +const std::string ThreadStatus::MicrosToString(uint64_t micros) { + if (micros == 0) { + return ""; + } + const int kBufferLen = 100; + char buffer[kBufferLen]; + AppendHumanMicros(micros, buffer, kBufferLen, false); + return std::string(buffer); +} + +const std::string& ThreadStatus::GetOperationPropertyName( + ThreadStatus::OperationType op_type, int i) { + static const std::string empty_str = ""; + switch (op_type) { + case ThreadStatus::OP_COMPACTION: + if (i >= NUM_COMPACTION_PROPERTIES) { + return empty_str; + } + return compaction_operation_properties[i].name; + case ThreadStatus::OP_FLUSH: + if (i >= NUM_FLUSH_PROPERTIES) { + return empty_str; + } + return flush_operation_properties[i].name; + default: + return empty_str; + } +} + +std::map + ThreadStatus::InterpretOperationProperties( + ThreadStatus::OperationType op_type, + const uint64_t* op_properties) { + int num_properties; + switch (op_type) { + case OP_COMPACTION: + num_properties = NUM_COMPACTION_PROPERTIES; + break; + case OP_FLUSH: + num_properties = NUM_FLUSH_PROPERTIES; + break; + default: + num_properties = 0; + } + + std::map property_map; + for (int i = 0; i < num_properties; ++i) { + if (op_type == OP_COMPACTION && + i == COMPACTION_INPUT_OUTPUT_LEVEL) { + property_map.insert( + {"BaseInputLevel", op_properties[i] >> 32}); + property_map.insert( + {"OutputLevel", op_properties[i] % (uint64_t(1) << 32U)}); + } else if (op_type == OP_COMPACTION && + i == COMPACTION_PROP_FLAGS) { + property_map.insert( + {"IsManual", ((op_properties[i] & 2) >> 1)}); + property_map.insert( + {"IsDeletion", ((op_properties[i] & 4) >> 2)}); + property_map.insert( + {"IsTrivialMove", ((op_properties[i] & 8) >> 3)}); + } else { + property_map.insert( + {GetOperationPropertyName(op_type, i), op_properties[i]}); + } + } + return property_map; +} + + +#else + +const std::string& ThreadStatus::GetThreadTypeName( + ThreadStatus::ThreadType thread_type) { + static std::string dummy_str = ""; + return dummy_str; +} + +const std::string& ThreadStatus::GetOperationName( + ThreadStatus::OperationType op_type) { + static std::string dummy_str = ""; + return dummy_str; +} + +const std::string& ThreadStatus::GetOperationStageName( + ThreadStatus::OperationStage stage) { + static std::string dummy_str = ""; + return dummy_str; +} + +const std::string& ThreadStatus::GetStateName( + ThreadStatus::StateType state_type) { + static std::string dummy_str = ""; + return dummy_str; +} + +const std::string ThreadStatus::MicrosToString( + uint64_t op_elapsed_time) { + static std::string dummy_str = ""; + return dummy_str; +} + +const std::string& ThreadStatus::GetOperationPropertyName( + ThreadStatus::OperationType op_type, int i) { + static std::string dummy_str = ""; + return dummy_str; +} + +std::map + ThreadStatus::InterpretOperationProperties( + ThreadStatus::OperationType op_type, + const uint64_t* op_properties) { + return std::map(); +} + +#endif // ROCKSDB_USING_THREAD_STATUS +} // namespace rocksdb diff --git a/monitoring/thread_status_updater.cc b/monitoring/thread_status_updater.cc new file mode 100644 index 00000000000..7441c35f8bc --- /dev/null +++ b/monitoring/thread_status_updater.cc @@ -0,0 +1,349 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "monitoring/thread_status_updater.h" +#include +#include "port/likely.h" +#include "rocksdb/env.h" +#include "util/mutexlock.h" + +namespace rocksdb { + +#ifdef ROCKSDB_USING_THREAD_STATUS + +__thread ThreadStatusData* ThreadStatusUpdater::thread_status_data_ = nullptr; + +void ThreadStatusUpdater::RegisterThread( + ThreadStatus::ThreadType ttype, uint64_t thread_id) { + if (UNLIKELY(thread_status_data_ == nullptr)) { + thread_status_data_ = new ThreadStatusData(); + thread_status_data_->thread_type = ttype; + thread_status_data_->thread_id = thread_id; + std::lock_guard lck(thread_list_mutex_); + thread_data_set_.insert(thread_status_data_); + } + + ClearThreadOperationProperties(); +} + +void ThreadStatusUpdater::UnregisterThread() { + if (thread_status_data_ != nullptr) { + std::lock_guard lck(thread_list_mutex_); + thread_data_set_.erase(thread_status_data_); + delete thread_status_data_; + thread_status_data_ = nullptr; + } +} + +void ThreadStatusUpdater::ResetThreadStatus() { + ClearThreadState(); + ClearThreadOperation(); + SetColumnFamilyInfoKey(nullptr); +} + +void ThreadStatusUpdater::SetColumnFamilyInfoKey( + const void* cf_key) { + auto* data = Get(); + if (data == nullptr) { + return; + } + // set the tracking flag based on whether cf_key is non-null or not. + // If enable_thread_tracking is set to false, the input cf_key + // would be nullptr. + data->enable_tracking = (cf_key != nullptr); + data->cf_key.store(const_cast(cf_key), std::memory_order_relaxed); +} + +const void* ThreadStatusUpdater::GetColumnFamilyInfoKey() { + auto* data = GetLocalThreadStatus(); + if (data == nullptr) { + return nullptr; + } + return data->cf_key.load(std::memory_order_relaxed); +} + +void ThreadStatusUpdater::SetThreadOperation( + const ThreadStatus::OperationType type) { + auto* data = GetLocalThreadStatus(); + if (data == nullptr) { + return; + } + // NOTE: Our practice here is to set all the thread operation properties + // and stage before we set thread operation, and thread operation + // will be set in std::memory_order_release. This is to ensure + // whenever a thread operation is not OP_UNKNOWN, we will always + // have a consistent information on its properties. + data->operation_type.store(type, std::memory_order_release); + if (type == ThreadStatus::OP_UNKNOWN) { + data->operation_stage.store(ThreadStatus::STAGE_UNKNOWN, + std::memory_order_relaxed); + ClearThreadOperationProperties(); + } +} + +void ThreadStatusUpdater::SetThreadOperationProperty( + int i, uint64_t value) { + auto* data = GetLocalThreadStatus(); + if (data == nullptr) { + return; + } + data->op_properties[i].store(value, std::memory_order_relaxed); +} + +void ThreadStatusUpdater::IncreaseThreadOperationProperty( + int i, uint64_t delta) { + auto* data = GetLocalThreadStatus(); + if (data == nullptr) { + return; + } + data->op_properties[i].fetch_add(delta, std::memory_order_relaxed); +} + +void ThreadStatusUpdater::SetOperationStartTime(const uint64_t start_time) { + auto* data = GetLocalThreadStatus(); + if (data == nullptr) { + return; + } + data->op_start_time.store(start_time, std::memory_order_relaxed); +} + +void ThreadStatusUpdater::ClearThreadOperation() { + auto* data = GetLocalThreadStatus(); + if (data == nullptr) { + return; + } + data->operation_stage.store(ThreadStatus::STAGE_UNKNOWN, + std::memory_order_relaxed); + data->operation_type.store( + ThreadStatus::OP_UNKNOWN, std::memory_order_relaxed); + ClearThreadOperationProperties(); +} + +void ThreadStatusUpdater::ClearThreadOperationProperties() { + auto* data = GetLocalThreadStatus(); + if (data == nullptr) { + return; + } + for (int i = 0; i < ThreadStatus::kNumOperationProperties; ++i) { + data->op_properties[i].store(0, std::memory_order_relaxed); + } +} + +ThreadStatus::OperationStage ThreadStatusUpdater::SetThreadOperationStage( + ThreadStatus::OperationStage stage) { + auto* data = GetLocalThreadStatus(); + if (data == nullptr) { + return ThreadStatus::STAGE_UNKNOWN; + } + return data->operation_stage.exchange( + stage, std::memory_order_relaxed); +} + +void ThreadStatusUpdater::SetThreadState( + const ThreadStatus::StateType type) { + auto* data = GetLocalThreadStatus(); + if (data == nullptr) { + return; + } + data->state_type.store(type, std::memory_order_relaxed); +} + +void ThreadStatusUpdater::ClearThreadState() { + auto* data = GetLocalThreadStatus(); + if (data == nullptr) { + return; + } + data->state_type.store( + ThreadStatus::STATE_UNKNOWN, std::memory_order_relaxed); +} + +Status ThreadStatusUpdater::GetThreadList( + std::vector* thread_list) { + thread_list->clear(); + std::vector> valid_list; + uint64_t now_micros = Env::Default()->NowMicros(); + + std::lock_guard lck(thread_list_mutex_); + for (auto* thread_data : thread_data_set_) { + assert(thread_data); + auto thread_id = thread_data->thread_id.load( + std::memory_order_relaxed); + auto thread_type = thread_data->thread_type.load( + std::memory_order_relaxed); + // Since any change to cf_info_map requires thread_list_mutex, + // which is currently held by GetThreadList(), here we can safely + // use "memory_order_relaxed" to load the cf_key. + auto cf_key = thread_data->cf_key.load( + std::memory_order_relaxed); + auto iter = cf_info_map_.find(cf_key); + auto* cf_info = iter != cf_info_map_.end() ? + iter->second.get() : nullptr; + const std::string* db_name = nullptr; + const std::string* cf_name = nullptr; + ThreadStatus::OperationType op_type = ThreadStatus::OP_UNKNOWN; + ThreadStatus::OperationStage op_stage = ThreadStatus::STAGE_UNKNOWN; + ThreadStatus::StateType state_type = ThreadStatus::STATE_UNKNOWN; + uint64_t op_elapsed_micros = 0; + uint64_t op_props[ThreadStatus::kNumOperationProperties] = {0}; + if (cf_info != nullptr) { + db_name = &cf_info->db_name; + cf_name = &cf_info->cf_name; + op_type = thread_data->operation_type.load( + std::memory_order_acquire); + // display lower-level info only when higher-level info is available. + if (op_type != ThreadStatus::OP_UNKNOWN) { + op_elapsed_micros = now_micros - thread_data->op_start_time.load( + std::memory_order_relaxed); + op_stage = thread_data->operation_stage.load( + std::memory_order_relaxed); + state_type = thread_data->state_type.load( + std::memory_order_relaxed); + for (int i = 0; i < ThreadStatus::kNumOperationProperties; ++i) { + op_props[i] = thread_data->op_properties[i].load( + std::memory_order_relaxed); + } + } + } + thread_list->emplace_back( + thread_id, thread_type, + db_name ? *db_name : "", + cf_name ? *cf_name : "", + op_type, op_elapsed_micros, op_stage, op_props, + state_type); + } + + return Status::OK(); +} + +ThreadStatusData* ThreadStatusUpdater::GetLocalThreadStatus() { + if (thread_status_data_ == nullptr) { + return nullptr; + } + if (!thread_status_data_->enable_tracking) { + assert(thread_status_data_->cf_key.load( + std::memory_order_relaxed) == nullptr); + return nullptr; + } + return thread_status_data_; +} + +void ThreadStatusUpdater::NewColumnFamilyInfo( + const void* db_key, const std::string& db_name, + const void* cf_key, const std::string& cf_name) { + // Acquiring same lock as GetThreadList() to guarantee + // a consistent view of global column family table (cf_info_map). + std::lock_guard lck(thread_list_mutex_); + + cf_info_map_[cf_key].reset( + new ConstantColumnFamilyInfo(db_key, db_name, cf_name)); + db_key_map_[db_key].insert(cf_key); +} + +void ThreadStatusUpdater::EraseColumnFamilyInfo(const void* cf_key) { + // Acquiring same lock as GetThreadList() to guarantee + // a consistent view of global column family table (cf_info_map). + std::lock_guard lck(thread_list_mutex_); + auto cf_pair = cf_info_map_.find(cf_key); + if (cf_pair == cf_info_map_.end()) { + return; + } + + auto* cf_info = cf_pair->second.get(); + assert(cf_info); + + // Remove its entry from db_key_map_ by the following steps: + // 1. Obtain the entry in db_key_map_ whose set contains cf_key + // 2. Remove it from the set. + auto db_pair = db_key_map_.find(cf_info->db_key); + assert(db_pair != db_key_map_.end()); + size_t result __attribute__((unused)) = db_pair->second.erase(cf_key); + assert(result); + + cf_pair->second.reset(); + result = cf_info_map_.erase(cf_key); + assert(result); +} + +void ThreadStatusUpdater::EraseDatabaseInfo(const void* db_key) { + // Acquiring same lock as GetThreadList() to guarantee + // a consistent view of global column family table (cf_info_map). + std::lock_guard lck(thread_list_mutex_); + auto db_pair = db_key_map_.find(db_key); + if (UNLIKELY(db_pair == db_key_map_.end())) { + // In some occasional cases such as DB::Open fails, we won't + // register ColumnFamilyInfo for a db. + return; + } + + size_t result __attribute__((unused)) = 0; + for (auto cf_key : db_pair->second) { + auto cf_pair = cf_info_map_.find(cf_key); + if (cf_pair == cf_info_map_.end()) { + continue; + } + cf_pair->second.reset(); + result = cf_info_map_.erase(cf_key); + assert(result); + } + db_key_map_.erase(db_key); +} + +#else + +void ThreadStatusUpdater::RegisterThread( + ThreadStatus::ThreadType ttype, uint64_t thread_id) { +} + +void ThreadStatusUpdater::UnregisterThread() { +} + +void ThreadStatusUpdater::ResetThreadStatus() { +} + +void ThreadStatusUpdater::SetColumnFamilyInfoKey( + const void* cf_key) { +} + +void ThreadStatusUpdater::SetThreadOperation( + const ThreadStatus::OperationType type) { +} + +void ThreadStatusUpdater::ClearThreadOperation() { +} + +void ThreadStatusUpdater::SetThreadState( + const ThreadStatus::StateType type) { +} + +void ThreadStatusUpdater::ClearThreadState() { +} + +Status ThreadStatusUpdater::GetThreadList( + std::vector* thread_list) { + return Status::NotSupported( + "GetThreadList is not supported in the current running environment."); +} + +void ThreadStatusUpdater::NewColumnFamilyInfo( + const void* db_key, const std::string& db_name, + const void* cf_key, const std::string& cf_name) { +} + +void ThreadStatusUpdater::EraseColumnFamilyInfo(const void* cf_key) { +} + +void ThreadStatusUpdater::EraseDatabaseInfo(const void* db_key) { +} + +void ThreadStatusUpdater::SetThreadOperationProperty( + int i, uint64_t value) { +} + +void ThreadStatusUpdater::IncreaseThreadOperationProperty( + int i, uint64_t delta) { +} + +#endif // ROCKSDB_USING_THREAD_STATUS +} // namespace rocksdb diff --git a/monitoring/thread_status_updater.h b/monitoring/thread_status_updater.h new file mode 100644 index 00000000000..69b4d4f7ecb --- /dev/null +++ b/monitoring/thread_status_updater.h @@ -0,0 +1,234 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// The implementation of ThreadStatus. +// +// Note that we make get and set access to ThreadStatusData lockless. +// As a result, ThreadStatusData as a whole is not atomic. However, +// we guarantee consistent ThreadStatusData all the time whenever +// user call GetThreadList(). This consistency guarantee is done +// by having the following constraint in the internal implementation +// of set and get order: +// +// 1. When reset any information in ThreadStatusData, always start from +// clearing up the lower-level information first. +// 2. When setting any information in ThreadStatusData, always start from +// setting the higher-level information. +// 3. When returning ThreadStatusData to the user, fields are fetched from +// higher-level to lower-level. In addition, where there's a nullptr +// in one field, then all fields that has lower-level than that field +// should be ignored. +// +// The high to low level information would be: +// thread_id > thread_type > db > cf > operation > state +// +// This means user might not always get full information, but whenever +// returned by the GetThreadList() is guaranteed to be consistent. +#pragma once +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rocksdb/status.h" +#include "rocksdb/thread_status.h" +#include "port/port.h" +#include "util/thread_operation.h" + +namespace rocksdb { + +class ColumnFamilyHandle; + +// The structure that keeps constant information about a column family. +struct ConstantColumnFamilyInfo { +#ifdef ROCKSDB_USING_THREAD_STATUS + public: + ConstantColumnFamilyInfo( + const void* _db_key, + const std::string& _db_name, + const std::string& _cf_name) : + db_key(_db_key), db_name(_db_name), cf_name(_cf_name) {} + const void* db_key; + const std::string db_name; + const std::string cf_name; +#endif // ROCKSDB_USING_THREAD_STATUS +}; + +// the internal data-structure that is used to reflect the current +// status of a thread using a set of atomic pointers. +struct ThreadStatusData { +#ifdef ROCKSDB_USING_THREAD_STATUS + explicit ThreadStatusData() : enable_tracking(false) { + thread_id.store(0); + thread_type.store(ThreadStatus::USER); + cf_key.store(nullptr); + operation_type.store(ThreadStatus::OP_UNKNOWN); + op_start_time.store(0); + state_type.store(ThreadStatus::STATE_UNKNOWN); + } + + // A flag to indicate whether the thread tracking is enabled + // in the current thread. This value will be updated based on whether + // the associated Options::enable_thread_tracking is set to true + // in ThreadStatusUtil::SetColumnFamily(). + // + // If set to false, then SetThreadOperation and SetThreadState + // will be no-op. + bool enable_tracking; + + std::atomic thread_id; + std::atomic thread_type; + std::atomic cf_key; + std::atomic operation_type; + std::atomic op_start_time; + std::atomic operation_stage; + std::atomic op_properties[ThreadStatus::kNumOperationProperties]; + std::atomic state_type; +#endif // ROCKSDB_USING_THREAD_STATUS +}; + +// The class that stores and updates the status of the current thread +// using a thread-local ThreadStatusData. +// +// In most of the case, you should use ThreadStatusUtil to update +// the status of the current thread instead of using ThreadSatusUpdater +// directly. +// +// @see ThreadStatusUtil +class ThreadStatusUpdater { + public: + ThreadStatusUpdater() {} + + // Releases all ThreadStatusData of all active threads. + virtual ~ThreadStatusUpdater() {} + + // Unregister the current thread. + void UnregisterThread(); + + // Reset the status of the current thread. This includes resetting + // ColumnFamilyInfoKey, ThreadOperation, and ThreadState. + void ResetThreadStatus(); + + // Set the id of the current thread. + void SetThreadID(uint64_t thread_id); + + // Register the current thread for tracking. + void RegisterThread(ThreadStatus::ThreadType ttype, uint64_t thread_id); + + // Update the column-family info of the current thread by setting + // its thread-local pointer of ThreadStateInfo to the correct entry. + void SetColumnFamilyInfoKey(const void* cf_key); + + // returns the column family info key. + const void* GetColumnFamilyInfoKey(); + + // Update the thread operation of the current thread. + void SetThreadOperation(const ThreadStatus::OperationType type); + + // The start time of the current thread operation. It is in the format + // of micro-seconds since some fixed point in time. + void SetOperationStartTime(const uint64_t start_time); + + // Set the "i"th property of the current operation. + // + // NOTE: Our practice here is to set all the thread operation properties + // and stage before we set thread operation, and thread operation + // will be set in std::memory_order_release. This is to ensure + // whenever a thread operation is not OP_UNKNOWN, we will always + // have a consistent information on its properties. + void SetThreadOperationProperty( + int i, uint64_t value); + + // Increase the "i"th property of the current operation with + // the specified delta. + void IncreaseThreadOperationProperty( + int i, uint64_t delta); + + // Update the thread operation stage of the current thread. + ThreadStatus::OperationStage SetThreadOperationStage( + const ThreadStatus::OperationStage stage); + + // Clear thread operation of the current thread. + void ClearThreadOperation(); + + // Reset all thread-operation-properties to 0. + void ClearThreadOperationProperties(); + + // Update the thread state of the current thread. + void SetThreadState(const ThreadStatus::StateType type); + + // Clear the thread state of the current thread. + void ClearThreadState(); + + // Obtain the status of all active registered threads. + Status GetThreadList( + std::vector* thread_list); + + // Create an entry in the global ColumnFamilyInfo table for the + // specified column family. This function should be called only + // when the current thread does not hold db_mutex. + void NewColumnFamilyInfo( + const void* db_key, const std::string& db_name, + const void* cf_key, const std::string& cf_name); + + // Erase all ConstantColumnFamilyInfo that is associated with the + // specified db instance. This function should be called only when + // the current thread does not hold db_mutex. + void EraseDatabaseInfo(const void* db_key); + + // Erase the ConstantColumnFamilyInfo that is associated with the + // specified ColumnFamilyData. This function should be called only + // when the current thread does not hold db_mutex. + void EraseColumnFamilyInfo(const void* cf_key); + + // Verifies whether the input ColumnFamilyHandles matches + // the information stored in the current cf_info_map. + void TEST_VerifyColumnFamilyInfoMap( + const std::vector& handles, + bool check_exist); + + protected: +#ifdef ROCKSDB_USING_THREAD_STATUS + // The thread-local variable for storing thread status. + static __thread ThreadStatusData* thread_status_data_; + + // Returns the pointer to the thread status data only when the + // thread status data is non-null and has enable_tracking == true. + ThreadStatusData* GetLocalThreadStatus(); + + // Directly returns the pointer to thread_status_data_ without + // checking whether enabling_tracking is true of not. + ThreadStatusData* Get() { + return thread_status_data_; + } + + // The mutex that protects cf_info_map and db_key_map. + std::mutex thread_list_mutex_; + + // The current status data of all active threads. + std::unordered_set thread_data_set_; + + // A global map that keeps the column family information. It is stored + // globally instead of inside DB is to avoid the situation where DB is + // closing while GetThreadList function already get the pointer to its + // CopnstantColumnFamilyInfo. + std::unordered_map< + const void*, std::unique_ptr> cf_info_map_; + + // A db_key to cf_key map that allows erasing elements in cf_info_map + // associated to the same db_key faster. + std::unordered_map< + const void*, std::unordered_set> db_key_map_; + +#else + static ThreadStatusData* thread_status_data_; +#endif // ROCKSDB_USING_THREAD_STATUS +}; + +} // namespace rocksdb diff --git a/monitoring/thread_status_updater_debug.cc b/monitoring/thread_status_updater_debug.cc new file mode 100644 index 00000000000..eec52e1887d --- /dev/null +++ b/monitoring/thread_status_updater_debug.cc @@ -0,0 +1,46 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include + +#include "db/column_family.h" +#include "monitoring/thread_status_updater.h" + +namespace rocksdb { + +#ifndef NDEBUG +#ifdef ROCKSDB_USING_THREAD_STATUS +void ThreadStatusUpdater::TEST_VerifyColumnFamilyInfoMap( + const std::vector& handles, + bool check_exist) { + std::unique_lock lock(thread_list_mutex_); + if (check_exist) { + assert(cf_info_map_.size() == handles.size()); + } + for (auto* handle : handles) { + auto* cfd = reinterpret_cast(handle)->cfd(); + auto iter __attribute__((unused)) = cf_info_map_.find(cfd); + if (check_exist) { + assert(iter != cf_info_map_.end()); + assert(iter->second); + assert(iter->second->cf_name == cfd->GetName()); + } else { + assert(iter == cf_info_map_.end()); + } + } +} + +#else + +void ThreadStatusUpdater::TEST_VerifyColumnFamilyInfoMap( + const std::vector& handles, + bool check_exist) { +} + +#endif // ROCKSDB_USING_THREAD_STATUS +#endif // !NDEBUG + + +} // namespace rocksdb diff --git a/monitoring/thread_status_util.cc b/monitoring/thread_status_util.cc new file mode 100644 index 00000000000..50692dfe55d --- /dev/null +++ b/monitoring/thread_status_util.cc @@ -0,0 +1,222 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "monitoring/thread_status_util.h" + +#include "monitoring/thread_status_updater.h" +#include "rocksdb/env.h" + +namespace rocksdb { + + +#ifdef ROCKSDB_USING_THREAD_STATUS +__thread ThreadStatusUpdater* + ThreadStatusUtil::thread_updater_local_cache_ = nullptr; +__thread bool ThreadStatusUtil::thread_updater_initialized_ = false; + +void ThreadStatusUtil::RegisterThread( + const Env* env, ThreadStatus::ThreadType thread_type) { + if (!MaybeInitThreadLocalUpdater(env)) { + return; + } + assert(thread_updater_local_cache_); + thread_updater_local_cache_->RegisterThread( + thread_type, env->GetThreadID()); +} + +void ThreadStatusUtil::UnregisterThread() { + thread_updater_initialized_ = false; + if (thread_updater_local_cache_ != nullptr) { + thread_updater_local_cache_->UnregisterThread(); + thread_updater_local_cache_ = nullptr; + } +} + +void ThreadStatusUtil::SetColumnFamily(const ColumnFamilyData* cfd, + const Env* env, + bool enable_thread_tracking) { + if (!MaybeInitThreadLocalUpdater(env)) { + return; + } + assert(thread_updater_local_cache_); + if (cfd != nullptr && enable_thread_tracking) { + thread_updater_local_cache_->SetColumnFamilyInfoKey(cfd); + } else { + // When cfd == nullptr or enable_thread_tracking == false, we set + // ColumnFamilyInfoKey to nullptr, which makes SetThreadOperation + // and SetThreadState become no-op. + thread_updater_local_cache_->SetColumnFamilyInfoKey(nullptr); + } +} + +void ThreadStatusUtil::SetThreadOperation(ThreadStatus::OperationType op) { + if (thread_updater_local_cache_ == nullptr) { + // thread_updater_local_cache_ must be set in SetColumnFamily + // or other ThreadStatusUtil functions. + return; + } + + if (op != ThreadStatus::OP_UNKNOWN) { + uint64_t current_time = Env::Default()->NowMicros(); + thread_updater_local_cache_->SetOperationStartTime(current_time); + } else { + // TDOO(yhchiang): we could report the time when we set operation to + // OP_UNKNOWN once the whole instrumentation has been done. + thread_updater_local_cache_->SetOperationStartTime(0); + } + thread_updater_local_cache_->SetThreadOperation(op); +} + +ThreadStatus::OperationStage ThreadStatusUtil::SetThreadOperationStage( + ThreadStatus::OperationStage stage) { + if (thread_updater_local_cache_ == nullptr) { + // thread_updater_local_cache_ must be set in SetColumnFamily + // or other ThreadStatusUtil functions. + return ThreadStatus::STAGE_UNKNOWN; + } + + return thread_updater_local_cache_->SetThreadOperationStage(stage); +} + +void ThreadStatusUtil::SetThreadOperationProperty( + int code, uint64_t value) { + if (thread_updater_local_cache_ == nullptr) { + // thread_updater_local_cache_ must be set in SetColumnFamily + // or other ThreadStatusUtil functions. + return; + } + + thread_updater_local_cache_->SetThreadOperationProperty( + code, value); +} + +void ThreadStatusUtil::IncreaseThreadOperationProperty( + int code, uint64_t delta) { + if (thread_updater_local_cache_ == nullptr) { + // thread_updater_local_cache_ must be set in SetColumnFamily + // or other ThreadStatusUtil functions. + return; + } + + thread_updater_local_cache_->IncreaseThreadOperationProperty( + code, delta); +} + +void ThreadStatusUtil::SetThreadState(ThreadStatus::StateType state) { + if (thread_updater_local_cache_ == nullptr) { + // thread_updater_local_cache_ must be set in SetColumnFamily + // or other ThreadStatusUtil functions. + return; + } + + thread_updater_local_cache_->SetThreadState(state); +} + +void ThreadStatusUtil::ResetThreadStatus() { + if (thread_updater_local_cache_ == nullptr) { + return; + } + thread_updater_local_cache_->ResetThreadStatus(); +} + +void ThreadStatusUtil::NewColumnFamilyInfo(const DB* db, + const ColumnFamilyData* cfd, + const std::string& cf_name, + const Env* env) { + if (!MaybeInitThreadLocalUpdater(env)) { + return; + } + assert(thread_updater_local_cache_); + if (thread_updater_local_cache_) { + thread_updater_local_cache_->NewColumnFamilyInfo(db, db->GetName(), cfd, + cf_name); + } +} + +void ThreadStatusUtil::EraseColumnFamilyInfo( + const ColumnFamilyData* cfd) { + if (thread_updater_local_cache_ == nullptr) { + return; + } + thread_updater_local_cache_->EraseColumnFamilyInfo(cfd); +} + +void ThreadStatusUtil::EraseDatabaseInfo(const DB* db) { + ThreadStatusUpdater* thread_updater = db->GetEnv()->GetThreadStatusUpdater(); + if (thread_updater == nullptr) { + return; + } + thread_updater->EraseDatabaseInfo(db); +} + +bool ThreadStatusUtil::MaybeInitThreadLocalUpdater(const Env* env) { + if (!thread_updater_initialized_ && env != nullptr) { + thread_updater_initialized_ = true; + thread_updater_local_cache_ = env->GetThreadStatusUpdater(); + } + return (thread_updater_local_cache_ != nullptr); +} + +AutoThreadOperationStageUpdater::AutoThreadOperationStageUpdater( + ThreadStatus::OperationStage stage) { + prev_stage_ = ThreadStatusUtil::SetThreadOperationStage(stage); +} + +AutoThreadOperationStageUpdater::~AutoThreadOperationStageUpdater() { + ThreadStatusUtil::SetThreadOperationStage(prev_stage_); +} + +#else + +ThreadStatusUpdater* ThreadStatusUtil::thread_updater_local_cache_ = nullptr; +bool ThreadStatusUtil::thread_updater_initialized_ = false; + +bool ThreadStatusUtil::MaybeInitThreadLocalUpdater(const Env* env) { + return false; +} + +void ThreadStatusUtil::SetColumnFamily(const ColumnFamilyData* cfd, + const Env* env, + bool enable_thread_tracking) {} + +void ThreadStatusUtil::SetThreadOperation(ThreadStatus::OperationType op) { +} + +void ThreadStatusUtil::SetThreadOperationProperty( + int code, uint64_t value) { +} + +void ThreadStatusUtil::IncreaseThreadOperationProperty( + int code, uint64_t delta) { +} + +void ThreadStatusUtil::SetThreadState(ThreadStatus::StateType state) { +} + +void ThreadStatusUtil::NewColumnFamilyInfo(const DB* db, + const ColumnFamilyData* cfd, + const std::string& cf_name, + const Env* env) {} + +void ThreadStatusUtil::EraseColumnFamilyInfo( + const ColumnFamilyData* cfd) { +} + +void ThreadStatusUtil::EraseDatabaseInfo(const DB* db) { +} + +void ThreadStatusUtil::ResetThreadStatus() { +} + +AutoThreadOperationStageUpdater::AutoThreadOperationStageUpdater( + ThreadStatus::OperationStage stage) { +} + +AutoThreadOperationStageUpdater::~AutoThreadOperationStageUpdater() { +} + +#endif // ROCKSDB_USING_THREAD_STATUS + +} // namespace rocksdb diff --git a/monitoring/thread_status_util.h b/monitoring/thread_status_util.h new file mode 100644 index 00000000000..a403435c3d0 --- /dev/null +++ b/monitoring/thread_status_util.h @@ -0,0 +1,134 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include + +#include "monitoring/thread_status_updater.h" +#include "rocksdb/db.h" +#include "rocksdb/env.h" +#include "rocksdb/thread_status.h" + +namespace rocksdb { + +class ColumnFamilyData; + +// The static utility class for updating thread-local status. +// +// The thread-local status is updated via the thread-local cached +// pointer thread_updater_local_cache_. During each function call, +// when ThreadStatusUtil finds thread_updater_local_cache_ is +// left uninitialized (determined by thread_updater_initialized_), +// it will tries to initialize it using the return value of +// Env::GetThreadStatusUpdater(). When thread_updater_local_cache_ +// is initialized by a non-null pointer, each function call will +// then update the status of the current thread. Otherwise, +// all function calls to ThreadStatusUtil will be no-op. +class ThreadStatusUtil { + public: + // Register the current thread for tracking. + static void RegisterThread( + const Env* env, ThreadStatus::ThreadType thread_type); + + // Unregister the current thread. + static void UnregisterThread(); + + // Create an entry in the global ColumnFamilyInfo table for the + // specified column family. This function should be called only + // when the current thread does not hold db_mutex. + static void NewColumnFamilyInfo(const DB* db, const ColumnFamilyData* cfd, + const std::string& cf_name, const Env* env); + + // Erase the ConstantColumnFamilyInfo that is associated with the + // specified ColumnFamilyData. This function should be called only + // when the current thread does not hold db_mutex. + static void EraseColumnFamilyInfo(const ColumnFamilyData* cfd); + + // Erase all ConstantColumnFamilyInfo that is associated with the + // specified db instance. This function should be called only when + // the current thread does not hold db_mutex. + static void EraseDatabaseInfo(const DB* db); + + // Update the thread status to indicate the current thread is doing + // something related to the specified column family. + static void SetColumnFamily(const ColumnFamilyData* cfd, const Env* env, + bool enable_thread_tracking); + + static void SetThreadOperation(ThreadStatus::OperationType type); + + static ThreadStatus::OperationStage SetThreadOperationStage( + ThreadStatus::OperationStage stage); + + static void SetThreadOperationProperty( + int code, uint64_t value); + + static void IncreaseThreadOperationProperty( + int code, uint64_t delta); + + static void SetThreadState(ThreadStatus::StateType type); + + static void ResetThreadStatus(); + +#ifndef NDEBUG + static void TEST_SetStateDelay( + const ThreadStatus::StateType state, int micro); + static void TEST_StateDelay(const ThreadStatus::StateType state); +#endif + + protected: + // Initialize the thread-local ThreadStatusUpdater when it finds + // the cached value is nullptr. Returns true if it has cached + // a non-null pointer. + static bool MaybeInitThreadLocalUpdater(const Env* env); + +#ifdef ROCKSDB_USING_THREAD_STATUS + // A boolean flag indicating whether thread_updater_local_cache_ + // is initialized. It is set to true when an Env uses any + // ThreadStatusUtil functions using the current thread other + // than UnregisterThread(). It will be set to false when + // UnregisterThread() is called. + // + // When this variable is set to true, thread_updater_local_cache_ + // will not be updated until this variable is again set to false + // in UnregisterThread(). + static __thread bool thread_updater_initialized_; + + // The thread-local cached ThreadStatusUpdater that caches the + // thread_status_updater_ of the first Env that uses any ThreadStatusUtil + // function other than UnregisterThread(). This variable will + // be cleared when UnregisterThread() is called. + // + // When this variable is set to a non-null pointer, then the status + // of the current thread will be updated when a function of + // ThreadStatusUtil is called. Otherwise, all functions of + // ThreadStatusUtil will be no-op. + // + // When thread_updater_initialized_ is set to true, this variable + // will not be updated until this thread_updater_initialized_ is + // again set to false in UnregisterThread(). + static __thread ThreadStatusUpdater* thread_updater_local_cache_; +#else + static bool thread_updater_initialized_; + static ThreadStatusUpdater* thread_updater_local_cache_; +#endif +}; + +// A helper class for updating thread state. It will set the +// thread state according to the input parameter in its constructor +// and set the thread state to the previous state in its destructor. +class AutoThreadOperationStageUpdater { + public: + explicit AutoThreadOperationStageUpdater( + ThreadStatus::OperationStage stage); + ~AutoThreadOperationStageUpdater(); + +#ifdef ROCKSDB_USING_THREAD_STATUS + private: + ThreadStatus::OperationStage prev_stage_; +#endif +}; + +} // namespace rocksdb diff --git a/monitoring/thread_status_util_debug.cc b/monitoring/thread_status_util_debug.cc new file mode 100644 index 00000000000..b4fa584747d --- /dev/null +++ b/monitoring/thread_status_util_debug.cc @@ -0,0 +1,32 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include + +#include "monitoring/thread_status_updater.h" +#include "monitoring/thread_status_util.h" +#include "rocksdb/env.h" + +namespace rocksdb { + +#ifndef NDEBUG +// the delay for debugging purpose. +static std::atomic states_delay[ThreadStatus::NUM_STATE_TYPES]; + +void ThreadStatusUtil::TEST_SetStateDelay( + const ThreadStatus::StateType state, int micro) { + states_delay[state].store(micro, std::memory_order_relaxed); +} + +void ThreadStatusUtil::TEST_StateDelay(const ThreadStatus::StateType state) { + auto delay = states_delay[state].load(std::memory_order_relaxed); + if (delay > 0) { + Env::Default()->SleepForMicroseconds(delay); + } +} + +#endif // !NDEBUG + +} // namespace rocksdb diff --git a/options/cf_options.cc b/options/cf_options.cc new file mode 100644 index 00000000000..67cbef68f6f --- /dev/null +++ b/options/cf_options.cc @@ -0,0 +1,179 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "options/cf_options.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include +#include +#include +#include "options/db_options.h" +#include "port/port.h" +#include "rocksdb/env.h" +#include "rocksdb/options.h" + +namespace rocksdb { + +ImmutableCFOptions::ImmutableCFOptions(const Options& options) + : ImmutableCFOptions(ImmutableDBOptions(options), options) {} + +ImmutableCFOptions::ImmutableCFOptions(const ImmutableDBOptions& db_options, + const ColumnFamilyOptions& cf_options) + : compaction_style(cf_options.compaction_style), + compaction_pri(cf_options.compaction_pri), + compaction_options_universal(cf_options.compaction_options_universal), + compaction_options_fifo(cf_options.compaction_options_fifo), + prefix_extractor(cf_options.prefix_extractor.get()), + user_comparator(cf_options.comparator), + internal_comparator(InternalKeyComparator(cf_options.comparator)), + merge_operator(cf_options.merge_operator.get()), + compaction_filter(cf_options.compaction_filter), + compaction_filter_factory(cf_options.compaction_filter_factory.get()), + min_write_buffer_number_to_merge( + cf_options.min_write_buffer_number_to_merge), + max_write_buffer_number_to_maintain( + cf_options.max_write_buffer_number_to_maintain), + inplace_update_support(cf_options.inplace_update_support), + inplace_callback(cf_options.inplace_callback), + info_log(db_options.info_log.get()), + statistics(db_options.statistics.get()), + rate_limiter(db_options.rate_limiter.get()), + env(db_options.env), + allow_mmap_reads(db_options.allow_mmap_reads), + allow_mmap_writes(db_options.allow_mmap_writes), + db_paths(db_options.db_paths), + memtable_factory(cf_options.memtable_factory.get()), + table_factory(cf_options.table_factory.get()), + table_properties_collector_factories( + cf_options.table_properties_collector_factories), + advise_random_on_open(db_options.advise_random_on_open), + bloom_locality(cf_options.bloom_locality), + purge_redundant_kvs_while_flush( + cf_options.purge_redundant_kvs_while_flush), + use_fsync(db_options.use_fsync), + compression_per_level(cf_options.compression_per_level), + bottommost_compression(cf_options.bottommost_compression), + compression_opts(cf_options.compression_opts), + level_compaction_dynamic_level_bytes( + cf_options.level_compaction_dynamic_level_bytes), + access_hint_on_compaction_start( + db_options.access_hint_on_compaction_start), + new_table_reader_for_compaction_inputs( + db_options.new_table_reader_for_compaction_inputs), + compaction_readahead_size(db_options.compaction_readahead_size), + num_levels(cf_options.num_levels), + optimize_filters_for_hits(cf_options.optimize_filters_for_hits), + force_consistency_checks(cf_options.force_consistency_checks), + allow_ingest_behind(db_options.allow_ingest_behind), + listeners(db_options.listeners), + row_cache(db_options.row_cache), + max_subcompactions(db_options.max_subcompactions), + memtable_insert_with_hint_prefix_extractor( + cf_options.memtable_insert_with_hint_prefix_extractor.get()) {} + +// Multiple two operands. If they overflow, return op1. +uint64_t MultiplyCheckOverflow(uint64_t op1, double op2) { + if (op1 == 0 || op2 <= 0) { + return 0; + } + if (port::kMaxUint64 / op1 < op2) { + return op1; + } + return static_cast(op1 * op2); +} + +void MutableCFOptions::RefreshDerivedOptions(int num_levels, + CompactionStyle compaction_style) { + max_file_size.resize(num_levels); + for (int i = 0; i < num_levels; ++i) { + if (i == 0 && compaction_style == kCompactionStyleUniversal) { + max_file_size[i] = ULLONG_MAX; + } else if (i > 1) { + max_file_size[i] = MultiplyCheckOverflow(max_file_size[i - 1], + target_file_size_multiplier); + } else { + max_file_size[i] = target_file_size_base; + } + } +} + +uint64_t MutableCFOptions::MaxFileSizeForLevel(int level) const { + assert(level >= 0); + assert(level < (int)max_file_size.size()); + return max_file_size[level]; +} + +void MutableCFOptions::Dump(Logger* log) const { + // Memtable related options + ROCKS_LOG_INFO(log, + " write_buffer_size: %" ROCKSDB_PRIszt, + write_buffer_size); + ROCKS_LOG_INFO(log, " max_write_buffer_number: %d", + max_write_buffer_number); + ROCKS_LOG_INFO(log, + " arena_block_size: %" ROCKSDB_PRIszt, + arena_block_size); + ROCKS_LOG_INFO(log, " memtable_prefix_bloom_ratio: %f", + memtable_prefix_bloom_size_ratio); + ROCKS_LOG_INFO(log, + " memtable_huge_page_size: %" ROCKSDB_PRIszt, + memtable_huge_page_size); + ROCKS_LOG_INFO(log, + " max_successive_merges: %" ROCKSDB_PRIszt, + max_successive_merges); + ROCKS_LOG_INFO(log, + " inplace_update_num_locks: %" ROCKSDB_PRIszt, + inplace_update_num_locks); + ROCKS_LOG_INFO(log, " disable_auto_compactions: %d", + disable_auto_compactions); + ROCKS_LOG_INFO(log, " soft_pending_compaction_bytes_limit: %" PRIu64, + soft_pending_compaction_bytes_limit); + ROCKS_LOG_INFO(log, " hard_pending_compaction_bytes_limit: %" PRIu64, + hard_pending_compaction_bytes_limit); + ROCKS_LOG_INFO(log, " level0_file_num_compaction_trigger: %d", + level0_file_num_compaction_trigger); + ROCKS_LOG_INFO(log, " level0_slowdown_writes_trigger: %d", + level0_slowdown_writes_trigger); + ROCKS_LOG_INFO(log, " level0_stop_writes_trigger: %d", + level0_stop_writes_trigger); + ROCKS_LOG_INFO(log, " max_compaction_bytes: %" PRIu64, + max_compaction_bytes); + ROCKS_LOG_INFO(log, " target_file_size_base: %" PRIu64, + target_file_size_base); + ROCKS_LOG_INFO(log, " target_file_size_multiplier: %d", + target_file_size_multiplier); + ROCKS_LOG_INFO(log, " max_bytes_for_level_base: %" PRIu64, + max_bytes_for_level_base); + ROCKS_LOG_INFO(log, " max_bytes_for_level_multiplier: %f", + max_bytes_for_level_multiplier); + std::string result; + char buf[10]; + for (const auto m : max_bytes_for_level_multiplier_additional) { + snprintf(buf, sizeof(buf), "%d, ", m); + result += buf; + } + if (result.size() >= 2) { + result.resize(result.size() - 2); + } else { + result = ""; + } + + ROCKS_LOG_INFO(log, "max_bytes_for_level_multiplier_additional: %s", + result.c_str()); + ROCKS_LOG_INFO(log, " max_sequential_skip_in_iterations: %" PRIu64, + max_sequential_skip_in_iterations); + ROCKS_LOG_INFO(log, " paranoid_file_checks: %d", + paranoid_file_checks); + ROCKS_LOG_INFO(log, " report_bg_io_stats: %d", + report_bg_io_stats); + ROCKS_LOG_INFO(log, " compression: %d", + static_cast(compression)); +} + +} // namespace rocksdb diff --git a/options/cf_options.h b/options/cf_options.h new file mode 100644 index 00000000000..f376729f853 --- /dev/null +++ b/options/cf_options.h @@ -0,0 +1,239 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include + +#include "db/dbformat.h" +#include "options/db_options.h" +#include "rocksdb/options.h" +#include "util/compression.h" + +namespace rocksdb { + +// ImmutableCFOptions is a data struct used by RocksDB internal. It contains a +// subset of Options that should not be changed during the entire lifetime +// of DB. Raw pointers defined in this struct do not have ownership to the data +// they point to. Options contains shared_ptr to these data. +struct ImmutableCFOptions { + ImmutableCFOptions(); + explicit ImmutableCFOptions(const Options& options); + + ImmutableCFOptions(const ImmutableDBOptions& db_options, + const ColumnFamilyOptions& cf_options); + + CompactionStyle compaction_style; + + CompactionPri compaction_pri; + + CompactionOptionsUniversal compaction_options_universal; + CompactionOptionsFIFO compaction_options_fifo; + + const SliceTransform* prefix_extractor; + + const Comparator* user_comparator; + InternalKeyComparator internal_comparator; + + MergeOperator* merge_operator; + + const CompactionFilter* compaction_filter; + + CompactionFilterFactory* compaction_filter_factory; + + int min_write_buffer_number_to_merge; + + int max_write_buffer_number_to_maintain; + + bool inplace_update_support; + + UpdateStatus (*inplace_callback)(char* existing_value, + uint32_t* existing_value_size, + Slice delta_value, + std::string* merged_value); + + Logger* info_log; + + Statistics* statistics; + + RateLimiter* rate_limiter; + + InfoLogLevel info_log_level; + + Env* env; + + // Allow the OS to mmap file for reading sst tables. Default: false + bool allow_mmap_reads; + + // Allow the OS to mmap file for writing. Default: false + bool allow_mmap_writes; + + std::vector db_paths; + + MemTableRepFactory* memtable_factory; + + TableFactory* table_factory; + + Options::TablePropertiesCollectorFactories + table_properties_collector_factories; + + bool advise_random_on_open; + + // This options is required by PlainTableReader. May need to move it + // to PlainTableOptions just like bloom_bits_per_key + uint32_t bloom_locality; + + bool purge_redundant_kvs_while_flush; + + bool use_fsync; + + std::vector compression_per_level; + + CompressionType bottommost_compression; + + CompressionOptions compression_opts; + + bool level_compaction_dynamic_level_bytes; + + Options::AccessHint access_hint_on_compaction_start; + + bool new_table_reader_for_compaction_inputs; + + size_t compaction_readahead_size; + + int num_levels; + + bool optimize_filters_for_hits; + + bool force_consistency_checks; + + bool allow_ingest_behind; + + // A vector of EventListeners which call-back functions will be called + // when specific RocksDB event happens. + std::vector> listeners; + + std::shared_ptr row_cache; + + uint32_t max_subcompactions; + + const SliceTransform* memtable_insert_with_hint_prefix_extractor; +}; + +struct MutableCFOptions { + explicit MutableCFOptions(const ColumnFamilyOptions& options) + : write_buffer_size(options.write_buffer_size), + max_write_buffer_number(options.max_write_buffer_number), + arena_block_size(options.arena_block_size), + memtable_prefix_bloom_size_ratio( + options.memtable_prefix_bloom_size_ratio), + memtable_huge_page_size(options.memtable_huge_page_size), + max_successive_merges(options.max_successive_merges), + inplace_update_num_locks(options.inplace_update_num_locks), + disable_auto_compactions(options.disable_auto_compactions), + soft_pending_compaction_bytes_limit( + options.soft_pending_compaction_bytes_limit), + hard_pending_compaction_bytes_limit( + options.hard_pending_compaction_bytes_limit), + level0_file_num_compaction_trigger( + options.level0_file_num_compaction_trigger), + level0_slowdown_writes_trigger(options.level0_slowdown_writes_trigger), + level0_stop_writes_trigger(options.level0_stop_writes_trigger), + max_compaction_bytes(options.max_compaction_bytes), + target_file_size_base(options.target_file_size_base), + target_file_size_multiplier(options.target_file_size_multiplier), + max_bytes_for_level_base(options.max_bytes_for_level_base), + max_bytes_for_level_multiplier(options.max_bytes_for_level_multiplier), + max_bytes_for_level_multiplier_additional( + options.max_bytes_for_level_multiplier_additional), + max_sequential_skip_in_iterations( + options.max_sequential_skip_in_iterations), + paranoid_file_checks(options.paranoid_file_checks), + report_bg_io_stats(options.report_bg_io_stats), + compression(options.compression) { + RefreshDerivedOptions(options.num_levels, options.compaction_style); + } + + MutableCFOptions() + : write_buffer_size(0), + max_write_buffer_number(0), + arena_block_size(0), + memtable_prefix_bloom_size_ratio(0), + memtable_huge_page_size(0), + max_successive_merges(0), + inplace_update_num_locks(0), + disable_auto_compactions(false), + soft_pending_compaction_bytes_limit(0), + hard_pending_compaction_bytes_limit(0), + level0_file_num_compaction_trigger(0), + level0_slowdown_writes_trigger(0), + level0_stop_writes_trigger(0), + max_compaction_bytes(0), + target_file_size_base(0), + target_file_size_multiplier(0), + max_bytes_for_level_base(0), + max_bytes_for_level_multiplier(0), + max_sequential_skip_in_iterations(0), + paranoid_file_checks(false), + report_bg_io_stats(false), + compression(Snappy_Supported() ? kSnappyCompression : kNoCompression) {} + + // Must be called after any change to MutableCFOptions + void RefreshDerivedOptions(int num_levels, CompactionStyle compaction_style); + + void RefreshDerivedOptions(const ImmutableCFOptions& ioptions) { + RefreshDerivedOptions(ioptions.num_levels, ioptions.compaction_style); + } + + // Get the max file size in a given level. + uint64_t MaxFileSizeForLevel(int level) const; + int MaxBytesMultiplerAdditional(int level) const { + if (level >= + static_cast(max_bytes_for_level_multiplier_additional.size())) { + return 1; + } + return max_bytes_for_level_multiplier_additional[level]; + } + + void Dump(Logger* log) const; + + // Memtable related options + size_t write_buffer_size; + int max_write_buffer_number; + size_t arena_block_size; + double memtable_prefix_bloom_size_ratio; + size_t memtable_huge_page_size; + size_t max_successive_merges; + size_t inplace_update_num_locks; + + // Compaction related options + bool disable_auto_compactions; + uint64_t soft_pending_compaction_bytes_limit; + uint64_t hard_pending_compaction_bytes_limit; + int level0_file_num_compaction_trigger; + int level0_slowdown_writes_trigger; + int level0_stop_writes_trigger; + uint64_t max_compaction_bytes; + uint64_t target_file_size_base; + int target_file_size_multiplier; + uint64_t max_bytes_for_level_base; + double max_bytes_for_level_multiplier; + std::vector max_bytes_for_level_multiplier_additional; + + // Misc options + uint64_t max_sequential_skip_in_iterations; + bool paranoid_file_checks; + bool report_bg_io_stats; + CompressionType compression; + + // Derived options + // Per-level target file size. + std::vector max_file_size; +}; + +uint64_t MultiplyCheckOverflow(uint64_t op1, double op2); + +} // namespace rocksdb diff --git a/options/db_options.cc b/options/db_options.cc new file mode 100644 index 00000000000..61775757d54 --- /dev/null +++ b/options/db_options.cc @@ -0,0 +1,275 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "options/db_options.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include + +#include "port/port.h" +#include "rocksdb/cache.h" +#include "rocksdb/env.h" +#include "rocksdb/sst_file_manager.h" +#include "rocksdb/wal_filter.h" +#include "util/logging.h" + +namespace rocksdb { + +ImmutableDBOptions::ImmutableDBOptions() : ImmutableDBOptions(Options()) {} + +ImmutableDBOptions::ImmutableDBOptions(const DBOptions& options) + : create_if_missing(options.create_if_missing), + create_missing_column_families(options.create_missing_column_families), + error_if_exists(options.error_if_exists), + paranoid_checks(options.paranoid_checks), + env(options.env), + rate_limiter(options.rate_limiter), + sst_file_manager(options.sst_file_manager), + info_log(options.info_log), + info_log_level(options.info_log_level), + max_file_opening_threads(options.max_file_opening_threads), + statistics(options.statistics), + use_fsync(options.use_fsync), + db_paths(options.db_paths), + db_log_dir(options.db_log_dir), + wal_dir(options.wal_dir), + max_subcompactions(options.max_subcompactions), + max_background_flushes(options.max_background_flushes), + max_log_file_size(options.max_log_file_size), + log_file_time_to_roll(options.log_file_time_to_roll), + keep_log_file_num(options.keep_log_file_num), + recycle_log_file_num(options.recycle_log_file_num), + max_manifest_file_size(options.max_manifest_file_size), + table_cache_numshardbits(options.table_cache_numshardbits), + wal_ttl_seconds(options.WAL_ttl_seconds), + wal_size_limit_mb(options.WAL_size_limit_MB), + manifest_preallocation_size(options.manifest_preallocation_size), + allow_mmap_reads(options.allow_mmap_reads), + allow_mmap_writes(options.allow_mmap_writes), + use_direct_reads(options.use_direct_reads), + use_direct_io_for_flush_and_compaction( + options.use_direct_io_for_flush_and_compaction), + allow_fallocate(options.allow_fallocate), + is_fd_close_on_exec(options.is_fd_close_on_exec), + advise_random_on_open(options.advise_random_on_open), + db_write_buffer_size(options.db_write_buffer_size), + write_buffer_manager(options.write_buffer_manager), + access_hint_on_compaction_start(options.access_hint_on_compaction_start), + new_table_reader_for_compaction_inputs( + options.new_table_reader_for_compaction_inputs), + compaction_readahead_size(options.compaction_readahead_size), + random_access_max_buffer_size(options.random_access_max_buffer_size), + writable_file_max_buffer_size(options.writable_file_max_buffer_size), + use_adaptive_mutex(options.use_adaptive_mutex), + bytes_per_sync(options.bytes_per_sync), + wal_bytes_per_sync(options.wal_bytes_per_sync), + listeners(options.listeners), + enable_thread_tracking(options.enable_thread_tracking), + enable_pipelined_write(options.enable_pipelined_write), + allow_concurrent_memtable_write(options.allow_concurrent_memtable_write), + enable_write_thread_adaptive_yield( + options.enable_write_thread_adaptive_yield), + write_thread_max_yield_usec(options.write_thread_max_yield_usec), + write_thread_slow_yield_usec(options.write_thread_slow_yield_usec), + skip_stats_update_on_db_open(options.skip_stats_update_on_db_open), + wal_recovery_mode(options.wal_recovery_mode), + allow_2pc(options.allow_2pc), + row_cache(options.row_cache), +#ifndef ROCKSDB_LITE + wal_filter(options.wal_filter), +#endif // ROCKSDB_LITE + fail_if_options_file_error(options.fail_if_options_file_error), + dump_malloc_stats(options.dump_malloc_stats), + avoid_flush_during_recovery(options.avoid_flush_during_recovery), + allow_ingest_behind(options.allow_ingest_behind), + concurrent_prepare(options.concurrent_prepare), + manual_wal_flush(options.manual_wal_flush) { +} + +void ImmutableDBOptions::Dump(Logger* log) const { + ROCKS_LOG_HEADER(log, " Options.error_if_exists: %d", + error_if_exists); + ROCKS_LOG_HEADER(log, " Options.create_if_missing: %d", + create_if_missing); + ROCKS_LOG_HEADER(log, " Options.paranoid_checks: %d", + paranoid_checks); + ROCKS_LOG_HEADER(log, " Options.env: %p", + env); + ROCKS_LOG_HEADER(log, " Options.info_log: %p", + info_log.get()); + ROCKS_LOG_HEADER(log, " Options.max_file_opening_threads: %d", + max_file_opening_threads); + ROCKS_LOG_HEADER(log, " Options.use_fsync: %d", + use_fsync); + ROCKS_LOG_HEADER( + log, " Options.max_log_file_size: %" ROCKSDB_PRIszt, + max_log_file_size); + ROCKS_LOG_HEADER(log, + " Options.max_manifest_file_size: %" PRIu64, + max_manifest_file_size); + ROCKS_LOG_HEADER( + log, " Options.log_file_time_to_roll: %" ROCKSDB_PRIszt, + log_file_time_to_roll); + ROCKS_LOG_HEADER( + log, " Options.keep_log_file_num: %" ROCKSDB_PRIszt, + keep_log_file_num); + ROCKS_LOG_HEADER( + log, " Options.recycle_log_file_num: %" ROCKSDB_PRIszt, + recycle_log_file_num); + ROCKS_LOG_HEADER(log, " Options.allow_fallocate: %d", + allow_fallocate); + ROCKS_LOG_HEADER(log, " Options.allow_mmap_reads: %d", + allow_mmap_reads); + ROCKS_LOG_HEADER(log, " Options.allow_mmap_writes: %d", + allow_mmap_writes); + ROCKS_LOG_HEADER(log, " Options.use_direct_reads: %d", + use_direct_reads); + ROCKS_LOG_HEADER(log, + " " + "Options.use_direct_io_for_flush_and_compaction: %d", + use_direct_io_for_flush_and_compaction); + ROCKS_LOG_HEADER(log, " Options.create_missing_column_families: %d", + create_missing_column_families); + ROCKS_LOG_HEADER(log, " Options.db_log_dir: %s", + db_log_dir.c_str()); + ROCKS_LOG_HEADER(log, " Options.wal_dir: %s", + wal_dir.c_str()); + ROCKS_LOG_HEADER(log, " Options.table_cache_numshardbits: %d", + table_cache_numshardbits); + ROCKS_LOG_HEADER(log, + " Options.max_subcompactions: %" PRIu32, + max_subcompactions); + ROCKS_LOG_HEADER(log, " Options.max_background_flushes: %d", + max_background_flushes); + ROCKS_LOG_HEADER(log, + " Options.WAL_ttl_seconds: %" PRIu64, + wal_ttl_seconds); + ROCKS_LOG_HEADER(log, + " Options.WAL_size_limit_MB: %" PRIu64, + wal_size_limit_mb); + ROCKS_LOG_HEADER( + log, " Options.manifest_preallocation_size: %" ROCKSDB_PRIszt, + manifest_preallocation_size); + ROCKS_LOG_HEADER(log, " Options.is_fd_close_on_exec: %d", + is_fd_close_on_exec); + ROCKS_LOG_HEADER(log, " Options.advise_random_on_open: %d", + advise_random_on_open); + ROCKS_LOG_HEADER( + log, " Options.db_write_buffer_size: %" ROCKSDB_PRIszt, + db_write_buffer_size); + ROCKS_LOG_HEADER(log, " Options.write_buffer_manager: %p", + write_buffer_manager.get()); + ROCKS_LOG_HEADER(log, " Options.access_hint_on_compaction_start: %d", + static_cast(access_hint_on_compaction_start)); + ROCKS_LOG_HEADER(log, " Options.new_table_reader_for_compaction_inputs: %d", + new_table_reader_for_compaction_inputs); + ROCKS_LOG_HEADER( + log, " Options.compaction_readahead_size: %" ROCKSDB_PRIszt, + compaction_readahead_size); + ROCKS_LOG_HEADER( + log, " Options.random_access_max_buffer_size: %" ROCKSDB_PRIszt, + random_access_max_buffer_size); + ROCKS_LOG_HEADER( + log, " Options.writable_file_max_buffer_size: %" ROCKSDB_PRIszt, + writable_file_max_buffer_size); + ROCKS_LOG_HEADER(log, " Options.use_adaptive_mutex: %d", + use_adaptive_mutex); + ROCKS_LOG_HEADER(log, " Options.rate_limiter: %p", + rate_limiter.get()); + Header( + log, " Options.sst_file_manager.rate_bytes_per_sec: %" PRIi64, + sst_file_manager ? sst_file_manager->GetDeleteRateBytesPerSecond() : 0); + ROCKS_LOG_HEADER(log, + " Options.bytes_per_sync: %" PRIu64, + bytes_per_sync); + ROCKS_LOG_HEADER(log, + " Options.wal_bytes_per_sync: %" PRIu64, + wal_bytes_per_sync); + ROCKS_LOG_HEADER(log, " Options.wal_recovery_mode: %d", + wal_recovery_mode); + ROCKS_LOG_HEADER(log, " Options.enable_thread_tracking: %d", + enable_thread_tracking); + ROCKS_LOG_HEADER(log, " Options.enable_pipelined_write: %d", + enable_pipelined_write); + ROCKS_LOG_HEADER(log, " Options.allow_concurrent_memtable_write: %d", + allow_concurrent_memtable_write); + ROCKS_LOG_HEADER(log, " Options.enable_write_thread_adaptive_yield: %d", + enable_write_thread_adaptive_yield); + ROCKS_LOG_HEADER(log, + " Options.write_thread_max_yield_usec: %" PRIu64, + write_thread_max_yield_usec); + ROCKS_LOG_HEADER(log, + " Options.write_thread_slow_yield_usec: %" PRIu64, + write_thread_slow_yield_usec); + if (row_cache) { + ROCKS_LOG_HEADER( + log, " Options.row_cache: %" PRIu64, + row_cache->GetCapacity()); + } else { + ROCKS_LOG_HEADER(log, + " Options.row_cache: None"); + } +#ifndef ROCKSDB_LITE + ROCKS_LOG_HEADER(log, " Options.wal_filter: %s", + wal_filter ? wal_filter->Name() : "None"); +#endif // ROCKDB_LITE + + ROCKS_LOG_HEADER(log, " Options.avoid_flush_during_recovery: %d", + avoid_flush_during_recovery); + ROCKS_LOG_HEADER(log, " Options.allow_ingest_behind: %d", + allow_ingest_behind); + ROCKS_LOG_HEADER(log, " Options.concurrent_prepare: %d", + concurrent_prepare); + ROCKS_LOG_HEADER(log, " Options.manual_wal_flush: %d", + manual_wal_flush); +} + +MutableDBOptions::MutableDBOptions() + : max_background_jobs(2), + base_background_compactions(-1), + max_background_compactions(-1), + avoid_flush_during_shutdown(false), + delayed_write_rate(2 * 1024U * 1024U), + max_total_wal_size(0), + delete_obsolete_files_period_micros(6ULL * 60 * 60 * 1000000), + stats_dump_period_sec(600), + max_open_files(-1) {} + +MutableDBOptions::MutableDBOptions(const DBOptions& options) + : max_background_jobs(options.max_background_jobs), + base_background_compactions(options.base_background_compactions), + max_background_compactions(options.max_background_compactions), + avoid_flush_during_shutdown(options.avoid_flush_during_shutdown), + delayed_write_rate(options.delayed_write_rate), + max_total_wal_size(options.max_total_wal_size), + delete_obsolete_files_period_micros( + options.delete_obsolete_files_period_micros), + stats_dump_period_sec(options.stats_dump_period_sec), + max_open_files(options.max_open_files) {} + +void MutableDBOptions::Dump(Logger* log) const { + ROCKS_LOG_HEADER(log, " Options.max_background_jobs: %d", + max_background_jobs); + ROCKS_LOG_HEADER(log, " Options.max_background_compactions: %d", + max_background_compactions); + ROCKS_LOG_HEADER(log, " Options.avoid_flush_during_shutdown: %d", + avoid_flush_during_shutdown); + ROCKS_LOG_HEADER(log, " Options.delayed_write_rate : %" PRIu64, + delayed_write_rate); + ROCKS_LOG_HEADER(log, " Options.max_total_wal_size: %" PRIu64, + max_total_wal_size); + ROCKS_LOG_HEADER( + log, " Options.delete_obsolete_files_period_micros: %" PRIu64, + delete_obsolete_files_period_micros); + ROCKS_LOG_HEADER(log, " Options.stats_dump_period_sec: %u", + stats_dump_period_sec); + ROCKS_LOG_HEADER(log, " Options.max_open_files: %d", + max_open_files); +} + +} // namespace rocksdb diff --git a/options/db_options.h b/options/db_options.h new file mode 100644 index 00000000000..18d1a5fb675 --- /dev/null +++ b/options/db_options.h @@ -0,0 +1,104 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include + +#include "rocksdb/options.h" + +namespace rocksdb { + +struct ImmutableDBOptions { + ImmutableDBOptions(); + explicit ImmutableDBOptions(const DBOptions& options); + + void Dump(Logger* log) const; + + bool create_if_missing; + bool create_missing_column_families; + bool error_if_exists; + bool paranoid_checks; + Env* env; + std::shared_ptr rate_limiter; + std::shared_ptr sst_file_manager; + std::shared_ptr info_log; + InfoLogLevel info_log_level; + int max_file_opening_threads; + std::shared_ptr statistics; + bool use_fsync; + std::vector db_paths; + std::string db_log_dir; + std::string wal_dir; + uint32_t max_subcompactions; + int max_background_flushes; + size_t max_log_file_size; + size_t log_file_time_to_roll; + size_t keep_log_file_num; + size_t recycle_log_file_num; + uint64_t max_manifest_file_size; + int table_cache_numshardbits; + uint64_t wal_ttl_seconds; + uint64_t wal_size_limit_mb; + size_t manifest_preallocation_size; + bool allow_mmap_reads; + bool allow_mmap_writes; + bool use_direct_reads; + bool use_direct_io_for_flush_and_compaction; + bool allow_fallocate; + bool is_fd_close_on_exec; + bool advise_random_on_open; + size_t db_write_buffer_size; + std::shared_ptr write_buffer_manager; + DBOptions::AccessHint access_hint_on_compaction_start; + bool new_table_reader_for_compaction_inputs; + size_t compaction_readahead_size; + size_t random_access_max_buffer_size; + size_t writable_file_max_buffer_size; + bool use_adaptive_mutex; + uint64_t bytes_per_sync; + uint64_t wal_bytes_per_sync; + std::vector> listeners; + bool enable_thread_tracking; + bool enable_pipelined_write; + bool allow_concurrent_memtable_write; + bool enable_write_thread_adaptive_yield; + uint64_t write_thread_max_yield_usec; + uint64_t write_thread_slow_yield_usec; + bool skip_stats_update_on_db_open; + WALRecoveryMode wal_recovery_mode; + bool allow_2pc; + std::shared_ptr row_cache; +#ifndef ROCKSDB_LITE + WalFilter* wal_filter; +#endif // ROCKSDB_LITE + bool fail_if_options_file_error; + bool dump_malloc_stats; + bool avoid_flush_during_recovery; + bool allow_ingest_behind; + bool concurrent_prepare; + bool manual_wal_flush; +}; + +struct MutableDBOptions { + MutableDBOptions(); + explicit MutableDBOptions(const MutableDBOptions& options) = default; + explicit MutableDBOptions(const DBOptions& options); + + void Dump(Logger* log) const; + + int max_background_jobs; + int base_background_compactions; + int max_background_compactions; + bool avoid_flush_during_shutdown; + uint64_t delayed_write_rate; + uint64_t max_total_wal_size; + uint64_t delete_obsolete_files_period_micros; + unsigned int stats_dump_period_sec; + int max_open_files; +}; + +} // namespace rocksdb diff --git a/options/options.cc b/options/options.cc new file mode 100644 index 00000000000..7bd2c9582f4 --- /dev/null +++ b/options/options.cc @@ -0,0 +1,636 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "rocksdb/options.h" + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include + +#include "monitoring/statistics.h" +#include "options/db_options.h" +#include "options/options_helper.h" +#include "rocksdb/cache.h" +#include "rocksdb/compaction_filter.h" +#include "rocksdb/comparator.h" +#include "rocksdb/env.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/merge_operator.h" +#include "rocksdb/slice.h" +#include "rocksdb/slice_transform.h" +#include "rocksdb/sst_file_manager.h" +#include "rocksdb/table.h" +#include "rocksdb/table_properties.h" +#include "rocksdb/wal_filter.h" +#include "table/block_based_table_factory.h" +#include "util/compression.h" + +namespace rocksdb { + +AdvancedColumnFamilyOptions::AdvancedColumnFamilyOptions() { + assert(memtable_factory.get() != nullptr); +} + +AdvancedColumnFamilyOptions::AdvancedColumnFamilyOptions(const Options& options) + : max_write_buffer_number(options.max_write_buffer_number), + min_write_buffer_number_to_merge( + options.min_write_buffer_number_to_merge), + max_write_buffer_number_to_maintain( + options.max_write_buffer_number_to_maintain), + inplace_update_support(options.inplace_update_support), + inplace_update_num_locks(options.inplace_update_num_locks), + inplace_callback(options.inplace_callback), + memtable_prefix_bloom_size_ratio( + options.memtable_prefix_bloom_size_ratio), + memtable_huge_page_size(options.memtable_huge_page_size), + memtable_insert_with_hint_prefix_extractor( + options.memtable_insert_with_hint_prefix_extractor), + bloom_locality(options.bloom_locality), + arena_block_size(options.arena_block_size), + compression_per_level(options.compression_per_level), + num_levels(options.num_levels), + level0_slowdown_writes_trigger(options.level0_slowdown_writes_trigger), + level0_stop_writes_trigger(options.level0_stop_writes_trigger), + target_file_size_base(options.target_file_size_base), + target_file_size_multiplier(options.target_file_size_multiplier), + level_compaction_dynamic_level_bytes( + options.level_compaction_dynamic_level_bytes), + max_bytes_for_level_multiplier(options.max_bytes_for_level_multiplier), + max_bytes_for_level_multiplier_additional( + options.max_bytes_for_level_multiplier_additional), + max_compaction_bytes(options.max_compaction_bytes), + soft_pending_compaction_bytes_limit( + options.soft_pending_compaction_bytes_limit), + hard_pending_compaction_bytes_limit( + options.hard_pending_compaction_bytes_limit), + compaction_style(options.compaction_style), + compaction_pri(options.compaction_pri), + compaction_options_universal(options.compaction_options_universal), + compaction_options_fifo(options.compaction_options_fifo), + max_sequential_skip_in_iterations( + options.max_sequential_skip_in_iterations), + memtable_factory(options.memtable_factory), + table_properties_collector_factories( + options.table_properties_collector_factories), + max_successive_merges(options.max_successive_merges), + optimize_filters_for_hits(options.optimize_filters_for_hits), + paranoid_file_checks(options.paranoid_file_checks), + force_consistency_checks(options.force_consistency_checks), + report_bg_io_stats(options.report_bg_io_stats) { + assert(memtable_factory.get() != nullptr); + if (max_bytes_for_level_multiplier_additional.size() < + static_cast(num_levels)) { + max_bytes_for_level_multiplier_additional.resize(num_levels, 1); + } +} + +ColumnFamilyOptions::ColumnFamilyOptions() + : compression(Snappy_Supported() ? kSnappyCompression : kNoCompression), + table_factory( + std::shared_ptr(new BlockBasedTableFactory())) {} + +ColumnFamilyOptions::ColumnFamilyOptions(const Options& options) + : AdvancedColumnFamilyOptions(options), + comparator(options.comparator), + merge_operator(options.merge_operator), + compaction_filter(options.compaction_filter), + compaction_filter_factory(options.compaction_filter_factory), + write_buffer_size(options.write_buffer_size), + compression(options.compression), + bottommost_compression(options.bottommost_compression), + compression_opts(options.compression_opts), + level0_file_num_compaction_trigger( + options.level0_file_num_compaction_trigger), + prefix_extractor(options.prefix_extractor), + max_bytes_for_level_base(options.max_bytes_for_level_base), + disable_auto_compactions(options.disable_auto_compactions), + table_factory(options.table_factory) {} + +DBOptions::DBOptions() {} + +DBOptions::DBOptions(const Options& options) + : create_if_missing(options.create_if_missing), + create_missing_column_families(options.create_missing_column_families), + error_if_exists(options.error_if_exists), + paranoid_checks(options.paranoid_checks), + env(options.env), + rate_limiter(options.rate_limiter), + sst_file_manager(options.sst_file_manager), + info_log(options.info_log), + info_log_level(options.info_log_level), + max_open_files(options.max_open_files), + max_file_opening_threads(options.max_file_opening_threads), + max_total_wal_size(options.max_total_wal_size), + statistics(options.statistics), + use_fsync(options.use_fsync), + db_paths(options.db_paths), + db_log_dir(options.db_log_dir), + wal_dir(options.wal_dir), + delete_obsolete_files_period_micros( + options.delete_obsolete_files_period_micros), + max_background_jobs(options.max_background_jobs), + base_background_compactions(options.base_background_compactions), + max_background_compactions(options.max_background_compactions), + max_subcompactions(options.max_subcompactions), + max_background_flushes(options.max_background_flushes), + max_log_file_size(options.max_log_file_size), + log_file_time_to_roll(options.log_file_time_to_roll), + keep_log_file_num(options.keep_log_file_num), + recycle_log_file_num(options.recycle_log_file_num), + max_manifest_file_size(options.max_manifest_file_size), + table_cache_numshardbits(options.table_cache_numshardbits), + WAL_ttl_seconds(options.WAL_ttl_seconds), + WAL_size_limit_MB(options.WAL_size_limit_MB), + manifest_preallocation_size(options.manifest_preallocation_size), + allow_mmap_reads(options.allow_mmap_reads), + allow_mmap_writes(options.allow_mmap_writes), + use_direct_reads(options.use_direct_reads), + use_direct_io_for_flush_and_compaction( + options.use_direct_io_for_flush_and_compaction), + allow_fallocate(options.allow_fallocate), + is_fd_close_on_exec(options.is_fd_close_on_exec), + skip_log_error_on_recovery(options.skip_log_error_on_recovery), + stats_dump_period_sec(options.stats_dump_period_sec), + advise_random_on_open(options.advise_random_on_open), + db_write_buffer_size(options.db_write_buffer_size), + write_buffer_manager(options.write_buffer_manager), + access_hint_on_compaction_start(options.access_hint_on_compaction_start), + new_table_reader_for_compaction_inputs( + options.new_table_reader_for_compaction_inputs), + compaction_readahead_size(options.compaction_readahead_size), + random_access_max_buffer_size(options.random_access_max_buffer_size), + writable_file_max_buffer_size(options.writable_file_max_buffer_size), + use_adaptive_mutex(options.use_adaptive_mutex), + bytes_per_sync(options.bytes_per_sync), + wal_bytes_per_sync(options.wal_bytes_per_sync), + listeners(options.listeners), + enable_thread_tracking(options.enable_thread_tracking), + delayed_write_rate(options.delayed_write_rate), + enable_pipelined_write(options.enable_pipelined_write), + allow_concurrent_memtable_write(options.allow_concurrent_memtable_write), + enable_write_thread_adaptive_yield( + options.enable_write_thread_adaptive_yield), + write_thread_max_yield_usec(options.write_thread_max_yield_usec), + write_thread_slow_yield_usec(options.write_thread_slow_yield_usec), + skip_stats_update_on_db_open(options.skip_stats_update_on_db_open), + wal_recovery_mode(options.wal_recovery_mode), + row_cache(options.row_cache), +#ifndef ROCKSDB_LITE + wal_filter(options.wal_filter), +#endif // ROCKSDB_LITE + fail_if_options_file_error(options.fail_if_options_file_error), + dump_malloc_stats(options.dump_malloc_stats), + avoid_flush_during_recovery(options.avoid_flush_during_recovery), + avoid_flush_during_shutdown(options.avoid_flush_during_shutdown), + allow_ingest_behind(options.allow_ingest_behind) { +} + +void DBOptions::Dump(Logger* log) const { + ImmutableDBOptions(*this).Dump(log); + MutableDBOptions(*this).Dump(log); +} // DBOptions::Dump + +void ColumnFamilyOptions::Dump(Logger* log) const { + ROCKS_LOG_HEADER(log, " Options.comparator: %s", + comparator->Name()); + ROCKS_LOG_HEADER(log, " Options.merge_operator: %s", + merge_operator ? merge_operator->Name() : "None"); + ROCKS_LOG_HEADER(log, " Options.compaction_filter: %s", + compaction_filter ? compaction_filter->Name() : "None"); + ROCKS_LOG_HEADER( + log, " Options.compaction_filter_factory: %s", + compaction_filter_factory ? compaction_filter_factory->Name() : "None"); + ROCKS_LOG_HEADER(log, " Options.memtable_factory: %s", + memtable_factory->Name()); + ROCKS_LOG_HEADER(log, " Options.table_factory: %s", + table_factory->Name()); + ROCKS_LOG_HEADER(log, " table_factory options: %s", + table_factory->GetPrintableTableOptions().c_str()); + ROCKS_LOG_HEADER(log, " Options.write_buffer_size: %" ROCKSDB_PRIszt, + write_buffer_size); + ROCKS_LOG_HEADER(log, " Options.max_write_buffer_number: %d", + max_write_buffer_number); + if (!compression_per_level.empty()) { + for (unsigned int i = 0; i < compression_per_level.size(); i++) { + ROCKS_LOG_HEADER( + log, " Options.compression[%d]: %s", i, + CompressionTypeToString(compression_per_level[i]).c_str()); + } + } else { + ROCKS_LOG_HEADER(log, " Options.compression: %s", + CompressionTypeToString(compression).c_str()); + } + ROCKS_LOG_HEADER( + log, " Options.bottommost_compression: %s", + bottommost_compression == kDisableCompressionOption + ? "Disabled" + : CompressionTypeToString(bottommost_compression).c_str()); + ROCKS_LOG_HEADER( + log, " Options.prefix_extractor: %s", + prefix_extractor == nullptr ? "nullptr" : prefix_extractor->Name()); + ROCKS_LOG_HEADER(log, + " Options.memtable_insert_with_hint_prefix_extractor: %s", + memtable_insert_with_hint_prefix_extractor == nullptr + ? "nullptr" + : memtable_insert_with_hint_prefix_extractor->Name()); + ROCKS_LOG_HEADER(log, " Options.num_levels: %d", num_levels); + ROCKS_LOG_HEADER(log, " Options.min_write_buffer_number_to_merge: %d", + min_write_buffer_number_to_merge); + ROCKS_LOG_HEADER(log, " Options.max_write_buffer_number_to_maintain: %d", + max_write_buffer_number_to_maintain); + ROCKS_LOG_HEADER(log, " Options.compression_opts.window_bits: %d", + compression_opts.window_bits); + ROCKS_LOG_HEADER(log, " Options.compression_opts.level: %d", + compression_opts.level); + ROCKS_LOG_HEADER(log, " Options.compression_opts.strategy: %d", + compression_opts.strategy); + ROCKS_LOG_HEADER( + log, + " Options.compression_opts.max_dict_bytes: %" ROCKSDB_PRIszt, + compression_opts.max_dict_bytes); + ROCKS_LOG_HEADER(log, " Options.level0_file_num_compaction_trigger: %d", + level0_file_num_compaction_trigger); + ROCKS_LOG_HEADER(log, " Options.level0_slowdown_writes_trigger: %d", + level0_slowdown_writes_trigger); + ROCKS_LOG_HEADER(log, " Options.level0_stop_writes_trigger: %d", + level0_stop_writes_trigger); + ROCKS_LOG_HEADER( + log, " Options.target_file_size_base: %" PRIu64, + target_file_size_base); + ROCKS_LOG_HEADER(log, " Options.target_file_size_multiplier: %d", + target_file_size_multiplier); + ROCKS_LOG_HEADER( + log, " Options.max_bytes_for_level_base: %" PRIu64, + max_bytes_for_level_base); + ROCKS_LOG_HEADER(log, "Options.level_compaction_dynamic_level_bytes: %d", + level_compaction_dynamic_level_bytes); + ROCKS_LOG_HEADER(log, " Options.max_bytes_for_level_multiplier: %f", + max_bytes_for_level_multiplier); + for (size_t i = 0; i < max_bytes_for_level_multiplier_additional.size(); + i++) { + ROCKS_LOG_HEADER( + log, "Options.max_bytes_for_level_multiplier_addtl[%" ROCKSDB_PRIszt + "]: %d", + i, max_bytes_for_level_multiplier_additional[i]); + } + ROCKS_LOG_HEADER( + log, " Options.max_sequential_skip_in_iterations: %" PRIu64, + max_sequential_skip_in_iterations); + ROCKS_LOG_HEADER( + log, " Options.max_compaction_bytes: %" PRIu64, + max_compaction_bytes); + ROCKS_LOG_HEADER( + log, + " Options.arena_block_size: %" ROCKSDB_PRIszt, + arena_block_size); + ROCKS_LOG_HEADER(log, + " Options.soft_pending_compaction_bytes_limit: %" PRIu64, + soft_pending_compaction_bytes_limit); + ROCKS_LOG_HEADER(log, + " Options.hard_pending_compaction_bytes_limit: %" PRIu64, + hard_pending_compaction_bytes_limit); + ROCKS_LOG_HEADER(log, " Options.rate_limit_delay_max_milliseconds: %u", + rate_limit_delay_max_milliseconds); + ROCKS_LOG_HEADER(log, " Options.disable_auto_compactions: %d", + disable_auto_compactions); + + const auto& it_compaction_style = + compaction_style_to_string.find(compaction_style); + std::string str_compaction_style; + if (it_compaction_style == compaction_style_to_string.end()) { + assert(false); + str_compaction_style = "unknown_" + std::to_string(compaction_style); + } else { + str_compaction_style = it_compaction_style->second; + } + ROCKS_LOG_HEADER(log, + " Options.compaction_style: %s", + str_compaction_style.c_str()); + + const auto& it_compaction_pri = + compaction_pri_to_string.find(compaction_pri); + std::string str_compaction_pri; + if (it_compaction_pri == compaction_pri_to_string.end()) { + assert(false); + str_compaction_pri = "unknown_" + std::to_string(compaction_pri); + } else { + str_compaction_pri = it_compaction_pri->second; + } + ROCKS_LOG_HEADER(log, + " Options.compaction_pri: %s", + str_compaction_pri.c_str()); + ROCKS_LOG_HEADER(log, + "Options.compaction_options_universal.size_ratio: %u", + compaction_options_universal.size_ratio); + ROCKS_LOG_HEADER(log, + "Options.compaction_options_universal.min_merge_width: %u", + compaction_options_universal.min_merge_width); + ROCKS_LOG_HEADER(log, + "Options.compaction_options_universal.max_merge_width: %u", + compaction_options_universal.max_merge_width); + ROCKS_LOG_HEADER( + log, + "Options.compaction_options_universal." + "max_size_amplification_percent: %u", + compaction_options_universal.max_size_amplification_percent); + ROCKS_LOG_HEADER( + log, + "Options.compaction_options_universal.compression_size_percent: %d", + compaction_options_universal.compression_size_percent); + const auto& it_compaction_stop_style = compaction_stop_style_to_string.find( + compaction_options_universal.stop_style); + std::string str_compaction_stop_style; + if (it_compaction_stop_style == compaction_stop_style_to_string.end()) { + assert(false); + str_compaction_stop_style = + "unknown_" + std::to_string(compaction_options_universal.stop_style); + } else { + str_compaction_stop_style = it_compaction_stop_style->second; + } + ROCKS_LOG_HEADER(log, + "Options.compaction_options_universal.stop_style: %s", + str_compaction_stop_style.c_str()); + ROCKS_LOG_HEADER( + log, "Options.compaction_options_fifo.max_table_files_size: %" PRIu64, + compaction_options_fifo.max_table_files_size); + ROCKS_LOG_HEADER(log, + "Options.compaction_options_fifo.allow_compaction: %d", + compaction_options_fifo.allow_compaction); + ROCKS_LOG_HEADER(log, "Options.compaction_options_fifo.ttl: %" PRIu64, + compaction_options_fifo.ttl); + std::string collector_names; + for (const auto& collector_factory : table_properties_collector_factories) { + collector_names.append(collector_factory->Name()); + collector_names.append("; "); + } + ROCKS_LOG_HEADER( + log, " Options.table_properties_collectors: %s", + collector_names.c_str()); + ROCKS_LOG_HEADER(log, + " Options.inplace_update_support: %d", + inplace_update_support); + ROCKS_LOG_HEADER( + log, + " Options.inplace_update_num_locks: %" ROCKSDB_PRIszt, + inplace_update_num_locks); + // TODO: easier config for bloom (maybe based on avg key/value size) + ROCKS_LOG_HEADER( + log, " Options.memtable_prefix_bloom_size_ratio: %f", + memtable_prefix_bloom_size_ratio); + + ROCKS_LOG_HEADER(log, " Options.memtable_huge_page_size: %" ROCKSDB_PRIszt, + memtable_huge_page_size); + ROCKS_LOG_HEADER(log, + " Options.bloom_locality: %d", + bloom_locality); + + ROCKS_LOG_HEADER( + log, + " Options.max_successive_merges: %" ROCKSDB_PRIszt, + max_successive_merges); + ROCKS_LOG_HEADER(log, + " Options.optimize_filters_for_hits: %d", + optimize_filters_for_hits); + ROCKS_LOG_HEADER(log, " Options.paranoid_file_checks: %d", + paranoid_file_checks); + ROCKS_LOG_HEADER(log, " Options.force_consistency_checks: %d", + force_consistency_checks); + ROCKS_LOG_HEADER(log, " Options.report_bg_io_stats: %d", + report_bg_io_stats); +} // ColumnFamilyOptions::Dump + +void Options::Dump(Logger* log) const { + DBOptions::Dump(log); + ColumnFamilyOptions::Dump(log); +} // Options::Dump + +void Options::DumpCFOptions(Logger* log) const { + ColumnFamilyOptions::Dump(log); +} // Options::DumpCFOptions + +// +// The goal of this method is to create a configuration that +// allows an application to write all files into L0 and +// then do a single compaction to output all files into L1. +Options* +Options::PrepareForBulkLoad() +{ + // never slowdown ingest. + level0_file_num_compaction_trigger = (1<<30); + level0_slowdown_writes_trigger = (1<<30); + level0_stop_writes_trigger = (1<<30); + soft_pending_compaction_bytes_limit = 0; + hard_pending_compaction_bytes_limit = 0; + + // no auto compactions please. The application should issue a + // manual compaction after all data is loaded into L0. + disable_auto_compactions = true; + // A manual compaction run should pick all files in L0 in + // a single compaction run. + max_compaction_bytes = (static_cast(1) << 60); + + // It is better to have only 2 levels, otherwise a manual + // compaction would compact at every possible level, thereby + // increasing the total time needed for compactions. + num_levels = 2; + + // Need to allow more write buffers to allow more parallism + // of flushes. + max_write_buffer_number = 6; + min_write_buffer_number_to_merge = 1; + + // When compaction is disabled, more parallel flush threads can + // help with write throughput. + max_background_flushes = 4; + + // Prevent a memtable flush to automatically promote files + // to L1. This is helpful so that all files that are + // input to the manual compaction are all at L0. + max_background_compactions = 2; + + // The compaction would create large files in L1. + target_file_size_base = 256 * 1024 * 1024; + return this; +} + +Options* Options::OptimizeForSmallDb() { + ColumnFamilyOptions::OptimizeForSmallDb(); + DBOptions::OptimizeForSmallDb(); + return this; +} + +Options* Options::OldDefaults(int rocksdb_major_version, + int rocksdb_minor_version) { + ColumnFamilyOptions::OldDefaults(rocksdb_major_version, + rocksdb_minor_version); + DBOptions::OldDefaults(rocksdb_major_version, rocksdb_minor_version); + return this; +} + +DBOptions* DBOptions::OldDefaults(int rocksdb_major_version, + int rocksdb_minor_version) { + if (rocksdb_major_version < 4 || + (rocksdb_major_version == 4 && rocksdb_minor_version < 7)) { + max_file_opening_threads = 1; + table_cache_numshardbits = 4; + } + if (rocksdb_major_version < 5 || + (rocksdb_major_version == 5 && rocksdb_minor_version < 2)) { + delayed_write_rate = 2 * 1024U * 1024U; + } else if (rocksdb_major_version < 5 || + (rocksdb_major_version == 5 && rocksdb_minor_version < 6)) { + delayed_write_rate = 16 * 1024U * 1024U; + } + max_open_files = 5000; + wal_recovery_mode = WALRecoveryMode::kTolerateCorruptedTailRecords; + return this; +} + +ColumnFamilyOptions* ColumnFamilyOptions::OldDefaults( + int rocksdb_major_version, int rocksdb_minor_version) { + if (rocksdb_major_version < 4 || + (rocksdb_major_version == 4 && rocksdb_minor_version < 7)) { + write_buffer_size = 4 << 20; + target_file_size_base = 2 * 1048576; + max_bytes_for_level_base = 10 * 1048576; + soft_pending_compaction_bytes_limit = 0; + hard_pending_compaction_bytes_limit = 0; + } + if (rocksdb_major_version < 5) { + level0_stop_writes_trigger = 24; + } else if (rocksdb_major_version == 5 && rocksdb_minor_version < 2) { + level0_stop_writes_trigger = 30; + } + compaction_pri = CompactionPri::kByCompensatedSize; + + return this; +} + +// Optimization functions +DBOptions* DBOptions::OptimizeForSmallDb() { + max_file_opening_threads = 1; + max_open_files = 5000; + return this; +} + +ColumnFamilyOptions* ColumnFamilyOptions::OptimizeForSmallDb() { + write_buffer_size = 2 << 20; + target_file_size_base = 2 * 1048576; + max_bytes_for_level_base = 10 * 1048576; + soft_pending_compaction_bytes_limit = 256 * 1048576; + hard_pending_compaction_bytes_limit = 1073741824ul; + return this; +} + +#ifndef ROCKSDB_LITE +ColumnFamilyOptions* ColumnFamilyOptions::OptimizeForPointLookup( + uint64_t block_cache_size_mb) { + prefix_extractor.reset(NewNoopTransform()); + BlockBasedTableOptions block_based_options; + block_based_options.index_type = BlockBasedTableOptions::kHashSearch; + block_based_options.filter_policy.reset(NewBloomFilterPolicy(10)); + block_based_options.block_cache = + NewLRUCache(static_cast(block_cache_size_mb * 1024 * 1024)); + table_factory.reset(new BlockBasedTableFactory(block_based_options)); + memtable_prefix_bloom_size_ratio = 0.02; + return this; +} + +ColumnFamilyOptions* ColumnFamilyOptions::OptimizeLevelStyleCompaction( + uint64_t memtable_memory_budget) { + write_buffer_size = static_cast(memtable_memory_budget / 4); + // merge two memtables when flushing to L0 + min_write_buffer_number_to_merge = 2; + // this means we'll use 50% extra memory in the worst case, but will reduce + // write stalls. + max_write_buffer_number = 6; + // start flushing L0->L1 as soon as possible. each file on level0 is + // (memtable_memory_budget / 2). This will flush level 0 when it's bigger than + // memtable_memory_budget. + level0_file_num_compaction_trigger = 2; + // doesn't really matter much, but we don't want to create too many files + target_file_size_base = memtable_memory_budget / 8; + // make Level1 size equal to Level0 size, so that L0->L1 compactions are fast + max_bytes_for_level_base = memtable_memory_budget; + + // level style compaction + compaction_style = kCompactionStyleLevel; + + // only compress levels >= 2 + compression_per_level.resize(num_levels); + for (int i = 0; i < num_levels; ++i) { + if (i < 2) { + compression_per_level[i] = kNoCompression; + } else { + compression_per_level[i] = kSnappyCompression; + } + } + return this; +} + +ColumnFamilyOptions* ColumnFamilyOptions::OptimizeUniversalStyleCompaction( + uint64_t memtable_memory_budget) { + write_buffer_size = static_cast(memtable_memory_budget / 4); + // merge two memtables when flushing to L0 + min_write_buffer_number_to_merge = 2; + // this means we'll use 50% extra memory in the worst case, but will reduce + // write stalls. + max_write_buffer_number = 6; + // universal style compaction + compaction_style = kCompactionStyleUniversal; + compaction_options_universal.compression_size_percent = 80; + return this; +} + +DBOptions* DBOptions::IncreaseParallelism(int total_threads) { + max_background_compactions = total_threads - 1; + max_background_flushes = 1; + env->SetBackgroundThreads(total_threads, Env::LOW); + env->SetBackgroundThreads(1, Env::HIGH); + return this; +} + +#endif // !ROCKSDB_LITE + +ReadOptions::ReadOptions() + : snapshot(nullptr), + iterate_upper_bound(nullptr), + readahead_size(0), + max_skippable_internal_keys(0), + read_tier(kReadAllTier), + verify_checksums(true), + fill_cache(true), + tailing(false), + managed(false), + total_order_seek(false), + prefix_same_as_start(false), + pin_data(false), + background_purge_on_iterator_cleanup(false), + ignore_range_deletions(false) {} + +ReadOptions::ReadOptions(bool cksum, bool cache) + : snapshot(nullptr), + iterate_upper_bound(nullptr), + readahead_size(0), + max_skippable_internal_keys(0), + read_tier(kReadAllTier), + verify_checksums(cksum), + fill_cache(cache), + tailing(false), + managed(false), + total_order_seek(false), + prefix_same_as_start(false), + pin_data(false), + background_purge_on_iterator_cleanup(false), + ignore_range_deletions(false) {} + +} // namespace rocksdb diff --git a/options/options_helper.cc b/options/options_helper.cc new file mode 100644 index 00000000000..5cf548fb9e2 --- /dev/null +++ b/options/options_helper.cc @@ -0,0 +1,1133 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#include "options/options_helper.h" + +#include +#include +#include +#include +#include +#include "rocksdb/cache.h" +#include "rocksdb/compaction_filter.h" +#include "rocksdb/convenience.h" +#include "rocksdb/filter_policy.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/merge_operator.h" +#include "rocksdb/options.h" +#include "rocksdb/rate_limiter.h" +#include "rocksdb/slice_transform.h" +#include "rocksdb/table.h" +#include "table/block_based_table_factory.h" +#include "table/plain_table_factory.h" +#include "util/cast_util.h" +#include "util/string_util.h" + +namespace rocksdb { + +DBOptions BuildDBOptions(const ImmutableDBOptions& immutable_db_options, + const MutableDBOptions& mutable_db_options) { + DBOptions options; + + options.create_if_missing = immutable_db_options.create_if_missing; + options.create_missing_column_families = + immutable_db_options.create_missing_column_families; + options.error_if_exists = immutable_db_options.error_if_exists; + options.paranoid_checks = immutable_db_options.paranoid_checks; + options.env = immutable_db_options.env; + options.rate_limiter = immutable_db_options.rate_limiter; + options.sst_file_manager = immutable_db_options.sst_file_manager; + options.info_log = immutable_db_options.info_log; + options.info_log_level = immutable_db_options.info_log_level; + options.max_open_files = mutable_db_options.max_open_files; + options.max_file_opening_threads = + immutable_db_options.max_file_opening_threads; + options.max_total_wal_size = mutable_db_options.max_total_wal_size; + options.statistics = immutable_db_options.statistics; + options.use_fsync = immutable_db_options.use_fsync; + options.db_paths = immutable_db_options.db_paths; + options.db_log_dir = immutable_db_options.db_log_dir; + options.wal_dir = immutable_db_options.wal_dir; + options.delete_obsolete_files_period_micros = + mutable_db_options.delete_obsolete_files_period_micros; + options.max_background_jobs = mutable_db_options.max_background_jobs; + options.base_background_compactions = + mutable_db_options.base_background_compactions; + options.max_background_compactions = + mutable_db_options.max_background_compactions; + options.max_subcompactions = immutable_db_options.max_subcompactions; + options.max_background_flushes = immutable_db_options.max_background_flushes; + options.max_log_file_size = immutable_db_options.max_log_file_size; + options.log_file_time_to_roll = immutable_db_options.log_file_time_to_roll; + options.keep_log_file_num = immutable_db_options.keep_log_file_num; + options.recycle_log_file_num = immutable_db_options.recycle_log_file_num; + options.max_manifest_file_size = immutable_db_options.max_manifest_file_size; + options.table_cache_numshardbits = + immutable_db_options.table_cache_numshardbits; + options.WAL_ttl_seconds = immutable_db_options.wal_ttl_seconds; + options.WAL_size_limit_MB = immutable_db_options.wal_size_limit_mb; + options.manifest_preallocation_size = + immutable_db_options.manifest_preallocation_size; + options.allow_mmap_reads = immutable_db_options.allow_mmap_reads; + options.allow_mmap_writes = immutable_db_options.allow_mmap_writes; + options.use_direct_reads = immutable_db_options.use_direct_reads; + options.use_direct_io_for_flush_and_compaction = + immutable_db_options.use_direct_io_for_flush_and_compaction; + options.allow_fallocate = immutable_db_options.allow_fallocate; + options.is_fd_close_on_exec = immutable_db_options.is_fd_close_on_exec; + options.stats_dump_period_sec = mutable_db_options.stats_dump_period_sec; + options.advise_random_on_open = immutable_db_options.advise_random_on_open; + options.db_write_buffer_size = immutable_db_options.db_write_buffer_size; + options.write_buffer_manager = immutable_db_options.write_buffer_manager; + options.access_hint_on_compaction_start = + immutable_db_options.access_hint_on_compaction_start; + options.new_table_reader_for_compaction_inputs = + immutable_db_options.new_table_reader_for_compaction_inputs; + options.compaction_readahead_size = + immutable_db_options.compaction_readahead_size; + options.random_access_max_buffer_size = + immutable_db_options.random_access_max_buffer_size; + options.writable_file_max_buffer_size = + immutable_db_options.writable_file_max_buffer_size; + options.use_adaptive_mutex = immutable_db_options.use_adaptive_mutex; + options.bytes_per_sync = immutable_db_options.bytes_per_sync; + options.wal_bytes_per_sync = immutable_db_options.wal_bytes_per_sync; + options.listeners = immutable_db_options.listeners; + options.enable_thread_tracking = immutable_db_options.enable_thread_tracking; + options.delayed_write_rate = mutable_db_options.delayed_write_rate; + options.allow_concurrent_memtable_write = + immutable_db_options.allow_concurrent_memtable_write; + options.enable_write_thread_adaptive_yield = + immutable_db_options.enable_write_thread_adaptive_yield; + options.write_thread_max_yield_usec = + immutable_db_options.write_thread_max_yield_usec; + options.write_thread_slow_yield_usec = + immutable_db_options.write_thread_slow_yield_usec; + options.skip_stats_update_on_db_open = + immutable_db_options.skip_stats_update_on_db_open; + options.wal_recovery_mode = immutable_db_options.wal_recovery_mode; + options.allow_2pc = immutable_db_options.allow_2pc; + options.row_cache = immutable_db_options.row_cache; +#ifndef ROCKSDB_LITE + options.wal_filter = immutable_db_options.wal_filter; +#endif // ROCKSDB_LITE + options.fail_if_options_file_error = + immutable_db_options.fail_if_options_file_error; + options.dump_malloc_stats = immutable_db_options.dump_malloc_stats; + options.avoid_flush_during_recovery = + immutable_db_options.avoid_flush_during_recovery; + options.avoid_flush_during_shutdown = + mutable_db_options.avoid_flush_during_shutdown; + options.allow_ingest_behind = + immutable_db_options.allow_ingest_behind; + + return options; +} + +ColumnFamilyOptions BuildColumnFamilyOptions( + const ColumnFamilyOptions& options, + const MutableCFOptions& mutable_cf_options) { + ColumnFamilyOptions cf_opts(options); + + // Memtable related options + cf_opts.write_buffer_size = mutable_cf_options.write_buffer_size; + cf_opts.max_write_buffer_number = mutable_cf_options.max_write_buffer_number; + cf_opts.arena_block_size = mutable_cf_options.arena_block_size; + cf_opts.memtable_prefix_bloom_size_ratio = + mutable_cf_options.memtable_prefix_bloom_size_ratio; + cf_opts.memtable_huge_page_size = mutable_cf_options.memtable_huge_page_size; + cf_opts.max_successive_merges = mutable_cf_options.max_successive_merges; + cf_opts.inplace_update_num_locks = + mutable_cf_options.inplace_update_num_locks; + + // Compaction related options + cf_opts.disable_auto_compactions = + mutable_cf_options.disable_auto_compactions; + cf_opts.level0_file_num_compaction_trigger = + mutable_cf_options.level0_file_num_compaction_trigger; + cf_opts.level0_slowdown_writes_trigger = + mutable_cf_options.level0_slowdown_writes_trigger; + cf_opts.level0_stop_writes_trigger = + mutable_cf_options.level0_stop_writes_trigger; + cf_opts.max_compaction_bytes = mutable_cf_options.max_compaction_bytes; + cf_opts.target_file_size_base = mutable_cf_options.target_file_size_base; + cf_opts.target_file_size_multiplier = + mutable_cf_options.target_file_size_multiplier; + cf_opts.max_bytes_for_level_base = + mutable_cf_options.max_bytes_for_level_base; + cf_opts.max_bytes_for_level_multiplier = + mutable_cf_options.max_bytes_for_level_multiplier; + + cf_opts.max_bytes_for_level_multiplier_additional.clear(); + for (auto value : + mutable_cf_options.max_bytes_for_level_multiplier_additional) { + cf_opts.max_bytes_for_level_multiplier_additional.emplace_back(value); + } + + // Misc options + cf_opts.max_sequential_skip_in_iterations = + mutable_cf_options.max_sequential_skip_in_iterations; + cf_opts.paranoid_file_checks = mutable_cf_options.paranoid_file_checks; + cf_opts.report_bg_io_stats = mutable_cf_options.report_bg_io_stats; + cf_opts.compression = mutable_cf_options.compression; + + cf_opts.table_factory = options.table_factory; + // TODO(yhchiang): find some way to handle the following derived options + // * max_file_size + + return cf_opts; +} + +#ifndef ROCKSDB_LITE + +namespace { +template +bool ParseEnum(const std::unordered_map& type_map, + const std::string& type, T* value) { + auto iter = type_map.find(type); + if (iter != type_map.end()) { + *value = iter->second; + return true; + } + return false; +} + +template +bool SerializeEnum(const std::unordered_map& type_map, + const T& type, std::string* value) { + for (const auto& pair : type_map) { + if (pair.second == type) { + *value = pair.first; + return true; + } + } + return false; +} + +bool SerializeVectorCompressionType(const std::vector& types, + std::string* value) { + std::stringstream ss; + bool result; + for (size_t i = 0; i < types.size(); ++i) { + if (i > 0) { + ss << ':'; + } + std::string string_type; + result = SerializeEnum(compression_type_string_map, + types[i], &string_type); + if (result == false) { + return result; + } + ss << string_type; + } + *value = ss.str(); + return true; +} + +bool ParseVectorCompressionType( + const std::string& value, + std::vector* compression_per_level) { + compression_per_level->clear(); + size_t start = 0; + while (start < value.size()) { + size_t end = value.find(':', start); + bool is_ok; + CompressionType type; + if (end == std::string::npos) { + is_ok = ParseEnum(compression_type_string_map, + value.substr(start), &type); + if (!is_ok) { + return false; + } + compression_per_level->emplace_back(type); + break; + } else { + is_ok = ParseEnum( + compression_type_string_map, value.substr(start, end - start), &type); + if (!is_ok) { + return false; + } + compression_per_level->emplace_back(type); + start = end + 1; + } + } + return true; +} + +bool ParseSliceTransformHelper( + const std::string& kFixedPrefixName, const std::string& kCappedPrefixName, + const std::string& value, + std::shared_ptr* slice_transform) { + + auto& pe_value = value; + if (pe_value.size() > kFixedPrefixName.size() && + pe_value.compare(0, kFixedPrefixName.size(), kFixedPrefixName) == 0) { + int prefix_length = ParseInt(trim(value.substr(kFixedPrefixName.size()))); + slice_transform->reset(NewFixedPrefixTransform(prefix_length)); + } else if (pe_value.size() > kCappedPrefixName.size() && + pe_value.compare(0, kCappedPrefixName.size(), kCappedPrefixName) == + 0) { + int prefix_length = + ParseInt(trim(pe_value.substr(kCappedPrefixName.size()))); + slice_transform->reset(NewCappedPrefixTransform(prefix_length)); + } else if (value == kNullptrString) { + slice_transform->reset(); + } else { + return false; + } + + return true; +} + +bool ParseSliceTransform( + const std::string& value, + std::shared_ptr* slice_transform) { + // While we normally don't convert the string representation of a + // pointer-typed option into its instance, here we do so for backward + // compatibility as we allow this action in SetOption(). + + // TODO(yhchiang): A possible better place for these serialization / + // deserialization is inside the class definition of pointer-typed + // option itself, but this requires a bigger change of public API. + bool result = + ParseSliceTransformHelper("fixed:", "capped:", value, slice_transform); + if (result) { + return result; + } + result = ParseSliceTransformHelper( + "rocksdb.FixedPrefix.", "rocksdb.CappedPrefix.", value, slice_transform); + if (result) { + return result; + } + // TODO(yhchiang): we can further support other default + // SliceTransforms here. + return false; +} +} // anonymouse namespace + +bool ParseOptionHelper(char* opt_address, const OptionType& opt_type, + const std::string& value) { + switch (opt_type) { + case OptionType::kBoolean: + *reinterpret_cast(opt_address) = ParseBoolean("", value); + break; + case OptionType::kInt: + *reinterpret_cast(opt_address) = ParseInt(value); + break; + case OptionType::kVectorInt: + *reinterpret_cast*>(opt_address) = ParseVectorInt(value); + break; + case OptionType::kUInt: + *reinterpret_cast(opt_address) = ParseUint32(value); + break; + case OptionType::kUInt32T: + *reinterpret_cast(opt_address) = ParseUint32(value); + break; + case OptionType::kUInt64T: + PutUnaligned(reinterpret_cast(opt_address), ParseUint64(value)); + break; + case OptionType::kSizeT: + PutUnaligned(reinterpret_cast(opt_address), ParseSizeT(value)); + break; + case OptionType::kString: + *reinterpret_cast(opt_address) = value; + break; + case OptionType::kDouble: + *reinterpret_cast(opt_address) = ParseDouble(value); + break; + case OptionType::kCompactionStyle: + return ParseEnum( + compaction_style_string_map, value, + reinterpret_cast(opt_address)); + case OptionType::kCompactionPri: + return ParseEnum( + compaction_pri_string_map, value, + reinterpret_cast(opt_address)); + case OptionType::kCompressionType: + return ParseEnum( + compression_type_string_map, value, + reinterpret_cast(opt_address)); + case OptionType::kVectorCompressionType: + return ParseVectorCompressionType( + value, reinterpret_cast*>(opt_address)); + case OptionType::kSliceTransform: + return ParseSliceTransform( + value, reinterpret_cast*>( + opt_address)); + case OptionType::kChecksumType: + return ParseEnum( + checksum_type_string_map, value, + reinterpret_cast(opt_address)); + case OptionType::kBlockBasedTableIndexType: + return ParseEnum( + block_base_table_index_type_string_map, value, + reinterpret_cast(opt_address)); + case OptionType::kEncodingType: + return ParseEnum( + encoding_type_string_map, value, + reinterpret_cast(opt_address)); + case OptionType::kWALRecoveryMode: + return ParseEnum( + wal_recovery_mode_string_map, value, + reinterpret_cast(opt_address)); + case OptionType::kAccessHint: + return ParseEnum( + access_hint_string_map, value, + reinterpret_cast(opt_address)); + case OptionType::kInfoLogLevel: + return ParseEnum( + info_log_level_string_map, value, + reinterpret_cast(opt_address)); + default: + return false; + } + return true; +} + +bool SerializeSingleOptionHelper(const char* opt_address, + const OptionType opt_type, + std::string* value) { + + assert(value); + switch (opt_type) { + case OptionType::kBoolean: + *value = *(reinterpret_cast(opt_address)) ? "true" : "false"; + break; + case OptionType::kInt: + *value = ToString(*(reinterpret_cast(opt_address))); + break; + case OptionType::kVectorInt: + return SerializeIntVector( + *reinterpret_cast*>(opt_address), value); + case OptionType::kUInt: + *value = ToString(*(reinterpret_cast(opt_address))); + break; + case OptionType::kUInt32T: + *value = ToString(*(reinterpret_cast(opt_address))); + break; + case OptionType::kUInt64T: + { + uint64_t v; + GetUnaligned(reinterpret_cast(opt_address), &v); + *value = ToString(v); + } + break; + case OptionType::kSizeT: + { + size_t v; + GetUnaligned(reinterpret_cast(opt_address), &v); + *value = ToString(v); + } + break; + case OptionType::kDouble: + *value = ToString(*(reinterpret_cast(opt_address))); + break; + case OptionType::kString: + *value = EscapeOptionString( + *(reinterpret_cast(opt_address))); + break; + case OptionType::kCompactionStyle: + return SerializeEnum( + compaction_style_string_map, + *(reinterpret_cast(opt_address)), value); + case OptionType::kCompactionPri: + return SerializeEnum( + compaction_pri_string_map, + *(reinterpret_cast(opt_address)), value); + case OptionType::kCompressionType: + return SerializeEnum( + compression_type_string_map, + *(reinterpret_cast(opt_address)), value); + case OptionType::kVectorCompressionType: + return SerializeVectorCompressionType( + *(reinterpret_cast*>(opt_address)), + value); + break; + case OptionType::kSliceTransform: { + const auto* slice_transform_ptr = + reinterpret_cast*>( + opt_address); + *value = slice_transform_ptr->get() ? slice_transform_ptr->get()->Name() + : kNullptrString; + break; + } + case OptionType::kTableFactory: { + const auto* table_factory_ptr = + reinterpret_cast*>( + opt_address); + *value = table_factory_ptr->get() ? table_factory_ptr->get()->Name() + : kNullptrString; + break; + } + case OptionType::kComparator: { + // it's a const pointer of const Comparator* + const auto* ptr = reinterpret_cast(opt_address); + // Since the user-specified comparator will be wrapped by + // InternalKeyComparator, we should persist the user-specified one + // instead of InternalKeyComparator. + if (*ptr == nullptr) { + *value = kNullptrString; + } else { + const Comparator* root_comp = (*ptr)->GetRootComparator(); + if (root_comp == nullptr) { + root_comp = (*ptr); + } + *value = root_comp->Name(); + } + break; + } + case OptionType::kCompactionFilter: { + // it's a const pointer of const CompactionFilter* + const auto* ptr = + reinterpret_cast(opt_address); + *value = *ptr ? (*ptr)->Name() : kNullptrString; + break; + } + case OptionType::kCompactionFilterFactory: { + const auto* ptr = + reinterpret_cast*>( + opt_address); + *value = ptr->get() ? ptr->get()->Name() : kNullptrString; + break; + } + case OptionType::kMemTableRepFactory: { + const auto* ptr = + reinterpret_cast*>( + opt_address); + *value = ptr->get() ? ptr->get()->Name() : kNullptrString; + break; + } + case OptionType::kMergeOperator: { + const auto* ptr = + reinterpret_cast*>(opt_address); + *value = ptr->get() ? ptr->get()->Name() : kNullptrString; + break; + } + case OptionType::kFilterPolicy: { + const auto* ptr = + reinterpret_cast*>(opt_address); + *value = ptr->get() ? ptr->get()->Name() : kNullptrString; + break; + } + case OptionType::kChecksumType: + return SerializeEnum( + checksum_type_string_map, + *reinterpret_cast(opt_address), value); + case OptionType::kBlockBasedTableIndexType: + return SerializeEnum( + block_base_table_index_type_string_map, + *reinterpret_cast( + opt_address), + value); + case OptionType::kFlushBlockPolicyFactory: { + const auto* ptr = + reinterpret_cast*>( + opt_address); + *value = ptr->get() ? ptr->get()->Name() : kNullptrString; + break; + } + case OptionType::kEncodingType: + return SerializeEnum( + encoding_type_string_map, + *reinterpret_cast(opt_address), value); + case OptionType::kWALRecoveryMode: + return SerializeEnum( + wal_recovery_mode_string_map, + *reinterpret_cast(opt_address), value); + case OptionType::kAccessHint: + return SerializeEnum( + access_hint_string_map, + *reinterpret_cast(opt_address), value); + case OptionType::kInfoLogLevel: + return SerializeEnum( + info_log_level_string_map, + *reinterpret_cast(opt_address), value); + default: + return false; + } + return true; +} + +Status GetMutableOptionsFromStrings( + const MutableCFOptions& base_options, + const std::unordered_map& options_map, + MutableCFOptions* new_options) { + assert(new_options); + *new_options = base_options; + for (const auto& o : options_map) { + try { + auto iter = cf_options_type_info.find(o.first); + if (iter == cf_options_type_info.end()) { + return Status::InvalidArgument("Unrecognized option: " + o.first); + } + const auto& opt_info = iter->second; + if (!opt_info.is_mutable) { + return Status::InvalidArgument("Option not changeable: " + o.first); + } + bool is_ok = ParseOptionHelper( + reinterpret_cast(new_options) + opt_info.mutable_offset, + opt_info.type, o.second); + if (!is_ok) { + return Status::InvalidArgument("Error parsing " + o.first); + } + } catch (std::exception& e) { + return Status::InvalidArgument("Error parsing " + o.first + ":" + + std::string(e.what())); + } + } + return Status::OK(); +} + +Status GetMutableDBOptionsFromStrings( + const MutableDBOptions& base_options, + const std::unordered_map& options_map, + MutableDBOptions* new_options) { + assert(new_options); + *new_options = base_options; + for (const auto& o : options_map) { + try { + auto iter = db_options_type_info.find(o.first); + if (iter == db_options_type_info.end()) { + return Status::InvalidArgument("Unrecognized option: " + o.first); + } + const auto& opt_info = iter->second; + if (!opt_info.is_mutable) { + return Status::InvalidArgument("Option not changeable: " + o.first); + } + bool is_ok = ParseOptionHelper( + reinterpret_cast(new_options) + opt_info.mutable_offset, + opt_info.type, o.second); + if (!is_ok) { + return Status::InvalidArgument("Error parsing " + o.first); + } + } catch (std::exception& e) { + return Status::InvalidArgument("Error parsing " + o.first + ":" + + std::string(e.what())); + } + } + return Status::OK(); +} + +Status StringToMap(const std::string& opts_str, + std::unordered_map* opts_map) { + assert(opts_map); + // Example: + // opts_str = "write_buffer_size=1024;max_write_buffer_number=2;" + // "nested_opt={opt1=1;opt2=2};max_bytes_for_level_base=100" + size_t pos = 0; + std::string opts = trim(opts_str); + while (pos < opts.size()) { + size_t eq_pos = opts.find('=', pos); + if (eq_pos == std::string::npos) { + return Status::InvalidArgument("Mismatched key value pair, '=' expected"); + } + std::string key = trim(opts.substr(pos, eq_pos - pos)); + if (key.empty()) { + return Status::InvalidArgument("Empty key found"); + } + + // skip space after '=' and look for '{' for possible nested options + pos = eq_pos + 1; + while (pos < opts.size() && isspace(opts[pos])) { + ++pos; + } + // Empty value at the end + if (pos >= opts.size()) { + (*opts_map)[key] = ""; + break; + } + if (opts[pos] == '{') { + int count = 1; + size_t brace_pos = pos + 1; + while (brace_pos < opts.size()) { + if (opts[brace_pos] == '{') { + ++count; + } else if (opts[brace_pos] == '}') { + --count; + if (count == 0) { + break; + } + } + ++brace_pos; + } + // found the matching closing brace + if (count == 0) { + (*opts_map)[key] = trim(opts.substr(pos + 1, brace_pos - pos - 1)); + // skip all whitespace and move to the next ';' + // brace_pos points to the next position after the matching '}' + pos = brace_pos + 1; + while (pos < opts.size() && isspace(opts[pos])) { + ++pos; + } + if (pos < opts.size() && opts[pos] != ';') { + return Status::InvalidArgument( + "Unexpected chars after nested options"); + } + ++pos; + } else { + return Status::InvalidArgument( + "Mismatched curly braces for nested options"); + } + } else { + size_t sc_pos = opts.find(';', pos); + if (sc_pos == std::string::npos) { + (*opts_map)[key] = trim(opts.substr(pos)); + // It either ends with a trailing semi-colon or the last key-value pair + break; + } else { + (*opts_map)[key] = trim(opts.substr(pos, sc_pos - pos)); + } + pos = sc_pos + 1; + } + } + + return Status::OK(); +} + +Status ParseColumnFamilyOption(const std::string& name, + const std::string& org_value, + ColumnFamilyOptions* new_options, + bool input_strings_escaped = false) { + const std::string& value = + input_strings_escaped ? UnescapeOptionString(org_value) : org_value; + try { + if (name == "block_based_table_factory") { + // Nested options + BlockBasedTableOptions table_opt, base_table_options; + BlockBasedTableFactory* block_based_table_factory = + static_cast_with_check( + new_options->table_factory.get()); + if (block_based_table_factory != nullptr) { + base_table_options = block_based_table_factory->table_options(); + } + Status table_opt_s = GetBlockBasedTableOptionsFromString( + base_table_options, value, &table_opt); + if (!table_opt_s.ok()) { + return Status::InvalidArgument( + "unable to parse the specified CF option " + name); + } + new_options->table_factory.reset(NewBlockBasedTableFactory(table_opt)); + } else if (name == "plain_table_factory") { + // Nested options + PlainTableOptions table_opt, base_table_options; + PlainTableFactory* plain_table_factory = + static_cast_with_check( + new_options->table_factory.get()); + if (plain_table_factory != nullptr) { + base_table_options = plain_table_factory->table_options(); + } + Status table_opt_s = GetPlainTableOptionsFromString( + base_table_options, value, &table_opt); + if (!table_opt_s.ok()) { + return Status::InvalidArgument( + "unable to parse the specified CF option " + name); + } + new_options->table_factory.reset(NewPlainTableFactory(table_opt)); + } else if (name == "memtable") { + std::unique_ptr new_mem_factory; + Status mem_factory_s = + GetMemTableRepFactoryFromString(value, &new_mem_factory); + if (!mem_factory_s.ok()) { + return Status::InvalidArgument( + "unable to parse the specified CF option " + name); + } + new_options->memtable_factory.reset(new_mem_factory.release()); + } else if (name == "compression_opts") { + size_t start = 0; + size_t end = value.find(':'); + if (end == std::string::npos) { + return Status::InvalidArgument( + "unable to parse the specified CF option " + name); + } + new_options->compression_opts.window_bits = + ParseInt(value.substr(start, end - start)); + start = end + 1; + end = value.find(':', start); + if (end == std::string::npos) { + return Status::InvalidArgument( + "unable to parse the specified CF option " + name); + } + new_options->compression_opts.level = + ParseInt(value.substr(start, end - start)); + start = end + 1; + if (start >= value.size()) { + return Status::InvalidArgument( + "unable to parse the specified CF option " + name); + } + end = value.find(':', start); + new_options->compression_opts.strategy = + ParseInt(value.substr(start, value.size() - start)); + // max_dict_bytes is optional for backwards compatibility + if (end != std::string::npos) { + start = end + 1; + if (start >= value.size()) { + return Status::InvalidArgument( + "unable to parse the specified CF option " + name); + } + new_options->compression_opts.max_dict_bytes = + ParseInt(value.substr(start, value.size() - start)); + } + } else if (name == "compaction_options_fifo") { + new_options->compaction_options_fifo.max_table_files_size = + ParseUint64(value); + } else { + auto iter = cf_options_type_info.find(name); + if (iter == cf_options_type_info.end()) { + return Status::InvalidArgument( + "Unable to parse the specified CF option " + name); + } + const auto& opt_info = iter->second; + if (opt_info.verification != OptionVerificationType::kDeprecated && + ParseOptionHelper( + reinterpret_cast(new_options) + opt_info.offset, + opt_info.type, value)) { + return Status::OK(); + } + switch (opt_info.verification) { + case OptionVerificationType::kByName: + case OptionVerificationType::kByNameAllowNull: + return Status::NotSupported( + "Deserializing the specified CF option " + name + + " is not supported"); + case OptionVerificationType::kDeprecated: + return Status::OK(); + default: + return Status::InvalidArgument( + "Unable to parse the specified CF option " + name); + } + } + } catch (const std::exception&) { + return Status::InvalidArgument( + "unable to parse the specified option " + name); + } + return Status::OK(); +} + +bool SerializeSingleDBOption(std::string* opt_string, + const DBOptions& db_options, + const std::string& name, + const std::string& delimiter) { + auto iter = db_options_type_info.find(name); + if (iter == db_options_type_info.end()) { + return false; + } + auto& opt_info = iter->second; + const char* opt_address = + reinterpret_cast(&db_options) + opt_info.offset; + std::string value; + bool result = SerializeSingleOptionHelper(opt_address, opt_info.type, &value); + if (result) { + *opt_string = name + "=" + value + delimiter; + } + return result; +} + +Status GetStringFromDBOptions(std::string* opt_string, + const DBOptions& db_options, + const std::string& delimiter) { + assert(opt_string); + opt_string->clear(); + for (auto iter = db_options_type_info.begin(); + iter != db_options_type_info.end(); ++iter) { + if (iter->second.verification == OptionVerificationType::kDeprecated) { + // If the option is no longer used in rocksdb and marked as deprecated, + // we skip it in the serialization. + continue; + } + std::string single_output; + bool result = SerializeSingleDBOption(&single_output, db_options, + iter->first, delimiter); + assert(result); + if (result) { + opt_string->append(single_output); + } + } + return Status::OK(); +} + +bool SerializeSingleColumnFamilyOption(std::string* opt_string, + const ColumnFamilyOptions& cf_options, + const std::string& name, + const std::string& delimiter) { + auto iter = cf_options_type_info.find(name); + if (iter == cf_options_type_info.end()) { + return false; + } + auto& opt_info = iter->second; + const char* opt_address = + reinterpret_cast(&cf_options) + opt_info.offset; + std::string value; + bool result = SerializeSingleOptionHelper(opt_address, opt_info.type, &value); + if (result) { + *opt_string = name + "=" + value + delimiter; + } + return result; +} + +Status GetStringFromColumnFamilyOptions(std::string* opt_string, + const ColumnFamilyOptions& cf_options, + const std::string& delimiter) { + assert(opt_string); + opt_string->clear(); + for (auto iter = cf_options_type_info.begin(); + iter != cf_options_type_info.end(); ++iter) { + if (iter->second.verification == OptionVerificationType::kDeprecated) { + // If the option is no longer used in rocksdb and marked as deprecated, + // we skip it in the serialization. + continue; + } + std::string single_output; + bool result = SerializeSingleColumnFamilyOption(&single_output, cf_options, + iter->first, delimiter); + if (result) { + opt_string->append(single_output); + } else { + return Status::InvalidArgument("failed to serialize %s\n", + iter->first.c_str()); + } + assert(result); + } + return Status::OK(); +} + +Status GetStringFromCompressionType(std::string* compression_str, + CompressionType compression_type) { + bool ok = SerializeEnum(compression_type_string_map, + compression_type, compression_str); + if (ok) { + return Status::OK(); + } else { + return Status::InvalidArgument("Invalid compression types"); + } +} + +std::vector GetSupportedCompressions() { + std::vector supported_compressions; + for (const auto& comp_to_name : compression_type_string_map) { + CompressionType t = comp_to_name.second; + if (t != kDisableCompressionOption && CompressionTypeSupported(t)) { + supported_compressions.push_back(t); + } + } + return supported_compressions; +} + +Status ParseDBOption(const std::string& name, + const std::string& org_value, + DBOptions* new_options, + bool input_strings_escaped = false) { + const std::string& value = + input_strings_escaped ? UnescapeOptionString(org_value) : org_value; + try { + if (name == "rate_limiter_bytes_per_sec") { + new_options->rate_limiter.reset( + NewGenericRateLimiter(static_cast(ParseUint64(value)))); + } else { + auto iter = db_options_type_info.find(name); + if (iter == db_options_type_info.end()) { + return Status::InvalidArgument("Unrecognized option DBOptions:", name); + } + const auto& opt_info = iter->second; + if (opt_info.verification != OptionVerificationType::kDeprecated && + ParseOptionHelper( + reinterpret_cast(new_options) + opt_info.offset, + opt_info.type, value)) { + return Status::OK(); + } + switch (opt_info.verification) { + case OptionVerificationType::kByName: + case OptionVerificationType::kByNameAllowNull: + return Status::NotSupported( + "Deserializing the specified DB option " + name + + " is not supported"); + case OptionVerificationType::kDeprecated: + return Status::OK(); + default: + return Status::InvalidArgument( + "Unable to parse the specified DB option " + name); + } + } + } catch (const std::exception&) { + return Status::InvalidArgument("Unable to parse DBOptions:", name); + } + return Status::OK(); +} + +Status GetColumnFamilyOptionsFromMap( + const ColumnFamilyOptions& base_options, + const std::unordered_map& opts_map, + ColumnFamilyOptions* new_options, bool input_strings_escaped, + bool ignore_unknown_options) { + return GetColumnFamilyOptionsFromMapInternal( + base_options, opts_map, new_options, input_strings_escaped, nullptr, + ignore_unknown_options); +} + +Status GetColumnFamilyOptionsFromMapInternal( + const ColumnFamilyOptions& base_options, + const std::unordered_map& opts_map, + ColumnFamilyOptions* new_options, bool input_strings_escaped, + std::vector* unsupported_options_names, + bool ignore_unknown_options) { + assert(new_options); + *new_options = base_options; + if (unsupported_options_names) { + unsupported_options_names->clear(); + } + for (const auto& o : opts_map) { + auto s = ParseColumnFamilyOption(o.first, o.second, new_options, + input_strings_escaped); + if (!s.ok()) { + if (s.IsNotSupported()) { + // If the deserialization of the specified option is not supported + // and an output vector of unsupported_options is provided, then + // we log the name of the unsupported option and proceed. + if (unsupported_options_names != nullptr) { + unsupported_options_names->push_back(o.first); + } + // Note that we still return Status::OK in such case to maintain + // the backward compatibility in the old public API defined in + // rocksdb/convenience.h + } else if (s.IsInvalidArgument() && ignore_unknown_options) { + continue; + } else { + // Restore "new_options" to the default "base_options". + *new_options = base_options; + return s; + } + } + } + return Status::OK(); +} + +Status GetColumnFamilyOptionsFromString( + const ColumnFamilyOptions& base_options, + const std::string& opts_str, + ColumnFamilyOptions* new_options) { + std::unordered_map opts_map; + Status s = StringToMap(opts_str, &opts_map); + if (!s.ok()) { + *new_options = base_options; + return s; + } + return GetColumnFamilyOptionsFromMap(base_options, opts_map, new_options); +} + +Status GetDBOptionsFromMap( + const DBOptions& base_options, + const std::unordered_map& opts_map, + DBOptions* new_options, bool input_strings_escaped, + bool ignore_unknown_options) { + return GetDBOptionsFromMapInternal(base_options, opts_map, new_options, + input_strings_escaped, nullptr, + ignore_unknown_options); +} + +Status GetDBOptionsFromMapInternal( + const DBOptions& base_options, + const std::unordered_map& opts_map, + DBOptions* new_options, bool input_strings_escaped, + std::vector* unsupported_options_names, + bool ignore_unknown_options) { + assert(new_options); + *new_options = base_options; + if (unsupported_options_names) { + unsupported_options_names->clear(); + } + for (const auto& o : opts_map) { + auto s = ParseDBOption(o.first, o.second, + new_options, input_strings_escaped); + if (!s.ok()) { + if (s.IsNotSupported()) { + // If the deserialization of the specified option is not supported + // and an output vector of unsupported_options is provided, then + // we log the name of the unsupported option and proceed. + if (unsupported_options_names != nullptr) { + unsupported_options_names->push_back(o.first); + } + // Note that we still return Status::OK in such case to maintain + // the backward compatibility in the old public API defined in + // rocksdb/convenience.h + } else if (s.IsInvalidArgument() && ignore_unknown_options) { + continue; + } else { + // Restore "new_options" to the default "base_options". + *new_options = base_options; + return s; + } + } + } + return Status::OK(); +} + +Status GetDBOptionsFromString( + const DBOptions& base_options, + const std::string& opts_str, + DBOptions* new_options) { + std::unordered_map opts_map; + Status s = StringToMap(opts_str, &opts_map); + if (!s.ok()) { + *new_options = base_options; + return s; + } + return GetDBOptionsFromMap(base_options, opts_map, new_options); +} + +Status GetOptionsFromString(const Options& base_options, + const std::string& opts_str, Options* new_options) { + std::unordered_map opts_map; + Status s = StringToMap(opts_str, &opts_map); + if (!s.ok()) { + return s; + } + DBOptions new_db_options(base_options); + ColumnFamilyOptions new_cf_options(base_options); + for (const auto& o : opts_map) { + if (ParseDBOption(o.first, o.second, &new_db_options).ok()) { + } else if (ParseColumnFamilyOption( + o.first, o.second, &new_cf_options).ok()) { + } else { + return Status::InvalidArgument("Can't parse option " + o.first); + } + } + *new_options = Options(new_db_options, new_cf_options); + return Status::OK(); +} + +Status GetTableFactoryFromMap( + const std::string& factory_name, + const std::unordered_map& opt_map, + std::shared_ptr* table_factory, bool ignore_unknown_options) { + Status s; + if (factory_name == BlockBasedTableFactory().Name()) { + BlockBasedTableOptions bbt_opt; + s = GetBlockBasedTableOptionsFromMap(BlockBasedTableOptions(), opt_map, + &bbt_opt, + true, /* input_strings_escaped */ + ignore_unknown_options); + if (!s.ok()) { + return s; + } + table_factory->reset(new BlockBasedTableFactory(bbt_opt)); + return Status::OK(); + } else if (factory_name == PlainTableFactory().Name()) { + PlainTableOptions pt_opt; + s = GetPlainTableOptionsFromMap(PlainTableOptions(), opt_map, &pt_opt, + true, /* input_strings_escaped */ + ignore_unknown_options); + if (!s.ok()) { + return s; + } + table_factory->reset(new PlainTableFactory(pt_opt)); + return Status::OK(); + } + // Return OK for not supported table factories as TableFactory + // Deserialization is optional. + table_factory->reset(); + return Status::OK(); +} + +#endif // !ROCKSDB_LITE + +} // namespace rocksdb diff --git a/options/options_helper.h b/options/options_helper.h new file mode 100644 index 00000000000..67b04271ff3 --- /dev/null +++ b/options/options_helper.h @@ -0,0 +1,650 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include +#include +#include + +#include "options/cf_options.h" +#include "options/db_options.h" +#include "rocksdb/options.h" +#include "rocksdb/status.h" +#include "rocksdb/table.h" + +namespace rocksdb { + +DBOptions BuildDBOptions(const ImmutableDBOptions& immutable_db_options, + const MutableDBOptions& mutable_db_options); + +ColumnFamilyOptions BuildColumnFamilyOptions( + const ColumnFamilyOptions& ioptions, + const MutableCFOptions& mutable_cf_options); + +static std::map compaction_style_to_string = { + {kCompactionStyleLevel, "kCompactionStyleLevel"}, + {kCompactionStyleUniversal, "kCompactionStyleUniversal"}, + {kCompactionStyleFIFO, "kCompactionStyleFIFO"}, + {kCompactionStyleNone, "kCompactionStyleNone"}}; + +static std::map compaction_pri_to_string = { + {kByCompensatedSize, "kByCompensatedSize"}, + {kOldestLargestSeqFirst, "kOldestLargestSeqFirst"}, + {kOldestSmallestSeqFirst, "kOldestSmallestSeqFirst"}, + {kMinOverlappingRatio, "kMinOverlappingRatio"}}; + +static std::map + compaction_stop_style_to_string = { + {kCompactionStopStyleSimilarSize, "kCompactionStopStyleSimilarSize"}, + {kCompactionStopStyleTotalSize, "kCompactionStopStyleTotalSize"}}; + +static std::unordered_map checksum_type_string_map = + {{"kNoChecksum", kNoChecksum}, {"kCRC32c", kCRC32c}, {"kxxHash", kxxHash}}; + +#ifndef ROCKSDB_LITE + +Status GetMutableOptionsFromStrings( + const MutableCFOptions& base_options, + const std::unordered_map& options_map, + MutableCFOptions* new_options); + +Status GetMutableDBOptionsFromStrings( + const MutableDBOptions& base_options, + const std::unordered_map& options_map, + MutableDBOptions* new_options); + +Status GetTableFactoryFromMap( + const std::string& factory_name, + const std::unordered_map& opt_map, + std::shared_ptr* table_factory, + bool ignore_unknown_options = false); + +enum class OptionType { + kBoolean, + kInt, + kVectorInt, + kUInt, + kUInt32T, + kUInt64T, + kSizeT, + kString, + kDouble, + kCompactionStyle, + kCompactionPri, + kSliceTransform, + kCompressionType, + kVectorCompressionType, + kTableFactory, + kComparator, + kCompactionFilter, + kCompactionFilterFactory, + kMergeOperator, + kMemTableRepFactory, + kBlockBasedTableIndexType, + kFilterPolicy, + kFlushBlockPolicyFactory, + kChecksumType, + kEncodingType, + kWALRecoveryMode, + kAccessHint, + kInfoLogLevel, + kUnknown +}; + +enum class OptionVerificationType { + kNormal, + kByName, // The option is pointer typed so we can only verify + // based on it's name. + kByNameAllowNull, // Same as kByName, but it also allows the case + // where one of them is a nullptr. + kDeprecated // The option is no longer used in rocksdb. The RocksDB + // OptionsParser will still accept this option if it + // happen to exists in some Options file. However, the + // parser will not include it in serialization and + // verification processes. +}; + +// A struct for storing constant option information such as option name, +// option type, and offset. +struct OptionTypeInfo { + int offset; + OptionType type; + OptionVerificationType verification; + bool is_mutable; + int mutable_offset; +}; + +// A helper function that converts "opt_address" to a std::string +// based on the specified OptionType. +bool SerializeSingleOptionHelper(const char* opt_address, + const OptionType opt_type, std::string* value); + +// In addition to its public version defined in rocksdb/convenience.h, +// this further takes an optional output vector "unsupported_options_names", +// which stores the name of all the unsupported options specified in "opts_map". +Status GetDBOptionsFromMapInternal( + const DBOptions& base_options, + const std::unordered_map& opts_map, + DBOptions* new_options, bool input_strings_escaped, + std::vector* unsupported_options_names = nullptr, + bool ignore_unknown_options = false); + +// In addition to its public version defined in rocksdb/convenience.h, +// this further takes an optional output vector "unsupported_options_names", +// which stores the name of all the unsupported options specified in "opts_map". +Status GetColumnFamilyOptionsFromMapInternal( + const ColumnFamilyOptions& base_options, + const std::unordered_map& opts_map, + ColumnFamilyOptions* new_options, bool input_strings_escaped, + std::vector* unsupported_options_names = nullptr, + bool ignore_unknown_options = false); + +static std::unordered_map db_options_type_info = { + /* + // not yet supported + Env* env; + std::shared_ptr row_cache; + std::shared_ptr delete_scheduler; + std::shared_ptr info_log; + std::shared_ptr rate_limiter; + std::shared_ptr statistics; + std::vector db_paths; + std::vector> listeners; + */ + {"advise_random_on_open", + {offsetof(struct DBOptions, advise_random_on_open), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"allow_mmap_reads", + {offsetof(struct DBOptions, allow_mmap_reads), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"allow_fallocate", + {offsetof(struct DBOptions, allow_fallocate), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"allow_mmap_writes", + {offsetof(struct DBOptions, allow_mmap_writes), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"use_direct_reads", + {offsetof(struct DBOptions, use_direct_reads), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"use_direct_writes", + {0, OptionType::kBoolean, OptionVerificationType::kDeprecated, false, 0}}, + {"use_direct_io_for_flush_and_compaction", + {offsetof(struct DBOptions, use_direct_io_for_flush_and_compaction), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"allow_2pc", + {offsetof(struct DBOptions, allow_2pc), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"allow_os_buffer", + {0, OptionType::kBoolean, OptionVerificationType::kDeprecated, true, 0}}, + {"create_if_missing", + {offsetof(struct DBOptions, create_if_missing), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"create_missing_column_families", + {offsetof(struct DBOptions, create_missing_column_families), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"disableDataSync", + {0, OptionType::kBoolean, OptionVerificationType::kDeprecated, false, 0}}, + {"disable_data_sync", // for compatibility + {0, OptionType::kBoolean, OptionVerificationType::kDeprecated, false, 0}}, + {"enable_thread_tracking", + {offsetof(struct DBOptions, enable_thread_tracking), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"error_if_exists", + {offsetof(struct DBOptions, error_if_exists), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"is_fd_close_on_exec", + {offsetof(struct DBOptions, is_fd_close_on_exec), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"paranoid_checks", + {offsetof(struct DBOptions, paranoid_checks), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"skip_log_error_on_recovery", + {offsetof(struct DBOptions, skip_log_error_on_recovery), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"skip_stats_update_on_db_open", + {offsetof(struct DBOptions, skip_stats_update_on_db_open), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"new_table_reader_for_compaction_inputs", + {offsetof(struct DBOptions, new_table_reader_for_compaction_inputs), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"compaction_readahead_size", + {offsetof(struct DBOptions, compaction_readahead_size), OptionType::kSizeT, + OptionVerificationType::kNormal, false, 0}}, + {"random_access_max_buffer_size", + {offsetof(struct DBOptions, random_access_max_buffer_size), + OptionType::kSizeT, OptionVerificationType::kNormal, false, 0}}, + {"writable_file_max_buffer_size", + {offsetof(struct DBOptions, writable_file_max_buffer_size), + OptionType::kSizeT, OptionVerificationType::kNormal, false, 0}}, + {"use_adaptive_mutex", + {offsetof(struct DBOptions, use_adaptive_mutex), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"use_fsync", + {offsetof(struct DBOptions, use_fsync), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"max_background_jobs", + {offsetof(struct DBOptions, max_background_jobs), OptionType::kInt, + OptionVerificationType::kNormal, true, + offsetof(struct MutableDBOptions, max_background_jobs)}}, + {"max_background_compactions", + {offsetof(struct DBOptions, max_background_compactions), OptionType::kInt, + OptionVerificationType::kNormal, true, + offsetof(struct MutableDBOptions, max_background_compactions)}}, + {"base_background_compactions", + {offsetof(struct DBOptions, base_background_compactions), OptionType::kInt, + OptionVerificationType::kNormal, true, + offsetof(struct MutableDBOptions, base_background_compactions)}}, + {"max_background_flushes", + {offsetof(struct DBOptions, max_background_flushes), OptionType::kInt, + OptionVerificationType::kNormal, false, 0}}, + {"max_file_opening_threads", + {offsetof(struct DBOptions, max_file_opening_threads), OptionType::kInt, + OptionVerificationType::kNormal, false, 0}}, + {"max_open_files", + {offsetof(struct DBOptions, max_open_files), OptionType::kInt, + OptionVerificationType::kNormal, true, + offsetof(struct MutableDBOptions, max_open_files)}}, + {"table_cache_numshardbits", + {offsetof(struct DBOptions, table_cache_numshardbits), OptionType::kInt, + OptionVerificationType::kNormal, false, 0}}, + {"db_write_buffer_size", + {offsetof(struct DBOptions, db_write_buffer_size), OptionType::kSizeT, + OptionVerificationType::kNormal, false, 0}}, + {"keep_log_file_num", + {offsetof(struct DBOptions, keep_log_file_num), OptionType::kSizeT, + OptionVerificationType::kNormal, false, 0}}, + {"recycle_log_file_num", + {offsetof(struct DBOptions, recycle_log_file_num), OptionType::kSizeT, + OptionVerificationType::kNormal, false, 0}}, + {"log_file_time_to_roll", + {offsetof(struct DBOptions, log_file_time_to_roll), OptionType::kSizeT, + OptionVerificationType::kNormal, false, 0}}, + {"manifest_preallocation_size", + {offsetof(struct DBOptions, manifest_preallocation_size), + OptionType::kSizeT, OptionVerificationType::kNormal, false, 0}}, + {"max_log_file_size", + {offsetof(struct DBOptions, max_log_file_size), OptionType::kSizeT, + OptionVerificationType::kNormal, false, 0}}, + {"db_log_dir", + {offsetof(struct DBOptions, db_log_dir), OptionType::kString, + OptionVerificationType::kNormal, false, 0}}, + {"wal_dir", + {offsetof(struct DBOptions, wal_dir), OptionType::kString, + OptionVerificationType::kNormal, false, 0}}, + {"max_subcompactions", + {offsetof(struct DBOptions, max_subcompactions), OptionType::kUInt32T, + OptionVerificationType::kNormal, false, 0}}, + {"WAL_size_limit_MB", + {offsetof(struct DBOptions, WAL_size_limit_MB), OptionType::kUInt64T, + OptionVerificationType::kNormal, false, 0}}, + {"WAL_ttl_seconds", + {offsetof(struct DBOptions, WAL_ttl_seconds), OptionType::kUInt64T, + OptionVerificationType::kNormal, false, 0}}, + {"bytes_per_sync", + {offsetof(struct DBOptions, bytes_per_sync), OptionType::kUInt64T, + OptionVerificationType::kNormal, false, 0}}, + {"delayed_write_rate", + {offsetof(struct DBOptions, delayed_write_rate), OptionType::kUInt64T, + OptionVerificationType::kNormal, true, + offsetof(struct MutableDBOptions, delayed_write_rate)}}, + {"delete_obsolete_files_period_micros", + {offsetof(struct DBOptions, delete_obsolete_files_period_micros), + OptionType::kUInt64T, OptionVerificationType::kNormal, true, + offsetof(struct MutableDBOptions, delete_obsolete_files_period_micros)}}, + {"max_manifest_file_size", + {offsetof(struct DBOptions, max_manifest_file_size), OptionType::kUInt64T, + OptionVerificationType::kNormal, false, 0}}, + {"max_total_wal_size", + {offsetof(struct DBOptions, max_total_wal_size), OptionType::kUInt64T, + OptionVerificationType::kNormal, true, + offsetof(struct MutableDBOptions, max_total_wal_size)}}, + {"wal_bytes_per_sync", + {offsetof(struct DBOptions, wal_bytes_per_sync), OptionType::kUInt64T, + OptionVerificationType::kNormal, false, 0}}, + {"stats_dump_period_sec", + {offsetof(struct DBOptions, stats_dump_period_sec), OptionType::kUInt, + OptionVerificationType::kNormal, true, + offsetof(struct MutableDBOptions, stats_dump_period_sec)}}, + {"fail_if_options_file_error", + {offsetof(struct DBOptions, fail_if_options_file_error), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"enable_pipelined_write", + {offsetof(struct DBOptions, enable_pipelined_write), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"allow_concurrent_memtable_write", + {offsetof(struct DBOptions, allow_concurrent_memtable_write), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"wal_recovery_mode", + {offsetof(struct DBOptions, wal_recovery_mode), + OptionType::kWALRecoveryMode, OptionVerificationType::kNormal, false, 0}}, + {"enable_write_thread_adaptive_yield", + {offsetof(struct DBOptions, enable_write_thread_adaptive_yield), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"write_thread_slow_yield_usec", + {offsetof(struct DBOptions, write_thread_slow_yield_usec), + OptionType::kUInt64T, OptionVerificationType::kNormal, false, 0}}, + {"write_thread_max_yield_usec", + {offsetof(struct DBOptions, write_thread_max_yield_usec), + OptionType::kUInt64T, OptionVerificationType::kNormal, false, 0}}, + {"access_hint_on_compaction_start", + {offsetof(struct DBOptions, access_hint_on_compaction_start), + OptionType::kAccessHint, OptionVerificationType::kNormal, false, 0}}, + {"info_log_level", + {offsetof(struct DBOptions, info_log_level), OptionType::kInfoLogLevel, + OptionVerificationType::kNormal, false, 0}}, + {"dump_malloc_stats", + {offsetof(struct DBOptions, dump_malloc_stats), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"avoid_flush_during_recovery", + {offsetof(struct DBOptions, avoid_flush_during_recovery), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"avoid_flush_during_shutdown", + {offsetof(struct DBOptions, avoid_flush_during_shutdown), + OptionType::kBoolean, OptionVerificationType::kNormal, true, + offsetof(struct MutableDBOptions, avoid_flush_during_shutdown)}}, + {"allow_ingest_behind", + {offsetof(struct DBOptions, allow_ingest_behind), OptionType::kBoolean, + OptionVerificationType::kNormal, false, + offsetof(struct ImmutableDBOptions, allow_ingest_behind)}}, + {"concurrent_prepare", + {offsetof(struct DBOptions, concurrent_prepare), OptionType::kBoolean, + OptionVerificationType::kNormal, false, + offsetof(struct ImmutableDBOptions, concurrent_prepare)}}, + {"manual_wal_flush", + {offsetof(struct DBOptions, manual_wal_flush), OptionType::kBoolean, + OptionVerificationType::kNormal, false, + offsetof(struct ImmutableDBOptions, manual_wal_flush)}}}; + +// offset_of is used to get the offset of a class data member +// ex: offset_of(&ColumnFamilyOptions::num_levels) +// This call will return the offset of num_levels in ColumnFamilyOptions class +// +// This is the same as offsetof() but allow us to work with non standard-layout +// classes and structures +// refs: +// http://en.cppreference.com/w/cpp/concept/StandardLayoutType +// https://gist.github.com/graphitemaster/494f21190bb2c63c5516 +template +inline int offset_of(T1 T2::*member) { + static T2 obj; + return int(size_t(&(obj.*member)) - size_t(&obj)); +} + +static std::unordered_map cf_options_type_info = { + /* not yet supported + CompactionOptionsFIFO compaction_options_fifo; + CompactionOptionsUniversal compaction_options_universal; + CompressionOptions compression_opts; + TablePropertiesCollectorFactories table_properties_collector_factories; + typedef std::vector> + TablePropertiesCollectorFactories; + UpdateStatus (*inplace_callback)(char* existing_value, + uint34_t* existing_value_size, + Slice delta_value, + std::string* merged_value); + */ + {"report_bg_io_stats", + {offset_of(&ColumnFamilyOptions::report_bg_io_stats), OptionType::kBoolean, + OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, report_bg_io_stats)}}, + {"compaction_measure_io_stats", + {0, OptionType::kBoolean, OptionVerificationType::kDeprecated, false, 0}}, + {"disable_auto_compactions", + {offset_of(&ColumnFamilyOptions::disable_auto_compactions), + OptionType::kBoolean, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, disable_auto_compactions)}}, + {"filter_deletes", + {0, OptionType::kBoolean, OptionVerificationType::kDeprecated, true, 0}}, + {"inplace_update_support", + {offset_of(&ColumnFamilyOptions::inplace_update_support), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"level_compaction_dynamic_level_bytes", + {offset_of(&ColumnFamilyOptions::level_compaction_dynamic_level_bytes), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"optimize_filters_for_hits", + {offset_of(&ColumnFamilyOptions::optimize_filters_for_hits), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"paranoid_file_checks", + {offset_of(&ColumnFamilyOptions::paranoid_file_checks), + OptionType::kBoolean, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, paranoid_file_checks)}}, + {"force_consistency_checks", + {offset_of(&ColumnFamilyOptions::force_consistency_checks), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"purge_redundant_kvs_while_flush", + {offset_of(&ColumnFamilyOptions::purge_redundant_kvs_while_flush), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"verify_checksums_in_compaction", + {0, OptionType::kBoolean, OptionVerificationType::kDeprecated, true, 0}}, + {"soft_pending_compaction_bytes_limit", + {offset_of(&ColumnFamilyOptions::soft_pending_compaction_bytes_limit), + OptionType::kUInt64T, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, soft_pending_compaction_bytes_limit)}}, + {"hard_pending_compaction_bytes_limit", + {offset_of(&ColumnFamilyOptions::hard_pending_compaction_bytes_limit), + OptionType::kUInt64T, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, hard_pending_compaction_bytes_limit)}}, + {"hard_rate_limit", + {0, OptionType::kDouble, OptionVerificationType::kDeprecated, true, 0}}, + {"soft_rate_limit", + {0, OptionType::kDouble, OptionVerificationType::kDeprecated, true, 0}}, + {"max_compaction_bytes", + {offset_of(&ColumnFamilyOptions::max_compaction_bytes), + OptionType::kUInt64T, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, max_compaction_bytes)}}, + {"expanded_compaction_factor", + {0, OptionType::kInt, OptionVerificationType::kDeprecated, true, 0}}, + {"level0_file_num_compaction_trigger", + {offset_of(&ColumnFamilyOptions::level0_file_num_compaction_trigger), + OptionType::kInt, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, level0_file_num_compaction_trigger)}}, + {"level0_slowdown_writes_trigger", + {offset_of(&ColumnFamilyOptions::level0_slowdown_writes_trigger), + OptionType::kInt, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, level0_slowdown_writes_trigger)}}, + {"level0_stop_writes_trigger", + {offset_of(&ColumnFamilyOptions::level0_stop_writes_trigger), + OptionType::kInt, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, level0_stop_writes_trigger)}}, + {"max_grandparent_overlap_factor", + {0, OptionType::kInt, OptionVerificationType::kDeprecated, true, 0}}, + {"max_mem_compaction_level", + {0, OptionType::kInt, OptionVerificationType::kDeprecated, false, 0}}, + {"max_write_buffer_number", + {offset_of(&ColumnFamilyOptions::max_write_buffer_number), + OptionType::kInt, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, max_write_buffer_number)}}, + {"max_write_buffer_number_to_maintain", + {offset_of(&ColumnFamilyOptions::max_write_buffer_number_to_maintain), + OptionType::kInt, OptionVerificationType::kNormal, false, 0}}, + {"min_write_buffer_number_to_merge", + {offset_of(&ColumnFamilyOptions::min_write_buffer_number_to_merge), + OptionType::kInt, OptionVerificationType::kNormal, false, 0}}, + {"num_levels", + {offset_of(&ColumnFamilyOptions::num_levels), OptionType::kInt, + OptionVerificationType::kNormal, false, 0}}, + {"source_compaction_factor", + {0, OptionType::kInt, OptionVerificationType::kDeprecated, true, 0}}, + {"target_file_size_multiplier", + {offset_of(&ColumnFamilyOptions::target_file_size_multiplier), + OptionType::kInt, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, target_file_size_multiplier)}}, + {"arena_block_size", + {offset_of(&ColumnFamilyOptions::arena_block_size), OptionType::kSizeT, + OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, arena_block_size)}}, + {"inplace_update_num_locks", + {offset_of(&ColumnFamilyOptions::inplace_update_num_locks), + OptionType::kSizeT, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, inplace_update_num_locks)}}, + {"max_successive_merges", + {offset_of(&ColumnFamilyOptions::max_successive_merges), + OptionType::kSizeT, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, max_successive_merges)}}, + {"memtable_huge_page_size", + {offset_of(&ColumnFamilyOptions::memtable_huge_page_size), + OptionType::kSizeT, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, memtable_huge_page_size)}}, + {"memtable_prefix_bloom_huge_page_tlb_size", + {0, OptionType::kSizeT, OptionVerificationType::kDeprecated, true, 0}}, + {"write_buffer_size", + {offset_of(&ColumnFamilyOptions::write_buffer_size), OptionType::kSizeT, + OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, write_buffer_size)}}, + {"bloom_locality", + {offset_of(&ColumnFamilyOptions::bloom_locality), OptionType::kUInt32T, + OptionVerificationType::kNormal, false, 0}}, + {"memtable_prefix_bloom_bits", + {0, OptionType::kUInt32T, OptionVerificationType::kDeprecated, true, 0}}, + {"memtable_prefix_bloom_size_ratio", + {offset_of(&ColumnFamilyOptions::memtable_prefix_bloom_size_ratio), + OptionType::kDouble, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, memtable_prefix_bloom_size_ratio)}}, + {"memtable_prefix_bloom_probes", + {0, OptionType::kUInt32T, OptionVerificationType::kDeprecated, true, 0}}, + {"min_partial_merge_operands", + {0, OptionType::kUInt32T, OptionVerificationType::kDeprecated, true, 0}}, + {"max_bytes_for_level_base", + {offset_of(&ColumnFamilyOptions::max_bytes_for_level_base), + OptionType::kUInt64T, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, max_bytes_for_level_base)}}, + {"max_bytes_for_level_multiplier", + {offset_of(&ColumnFamilyOptions::max_bytes_for_level_multiplier), + OptionType::kDouble, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, max_bytes_for_level_multiplier)}}, + {"max_bytes_for_level_multiplier_additional", + {offset_of( + &ColumnFamilyOptions::max_bytes_for_level_multiplier_additional), + OptionType::kVectorInt, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, + max_bytes_for_level_multiplier_additional)}}, + {"max_sequential_skip_in_iterations", + {offset_of(&ColumnFamilyOptions::max_sequential_skip_in_iterations), + OptionType::kUInt64T, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, max_sequential_skip_in_iterations)}}, + {"target_file_size_base", + {offset_of(&ColumnFamilyOptions::target_file_size_base), + OptionType::kUInt64T, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, target_file_size_base)}}, + {"rate_limit_delay_max_milliseconds", + {0, OptionType::kUInt, OptionVerificationType::kDeprecated, false, 0}}, + {"compression", + {offset_of(&ColumnFamilyOptions::compression), + OptionType::kCompressionType, OptionVerificationType::kNormal, true, + offsetof(struct MutableCFOptions, compression)}}, + {"compression_per_level", + {offset_of(&ColumnFamilyOptions::compression_per_level), + OptionType::kVectorCompressionType, OptionVerificationType::kNormal, + false, 0}}, + {"bottommost_compression", + {offset_of(&ColumnFamilyOptions::bottommost_compression), + OptionType::kCompressionType, OptionVerificationType::kNormal, false, 0}}, + {"comparator", + {offset_of(&ColumnFamilyOptions::comparator), OptionType::kComparator, + OptionVerificationType::kByName, false, 0}}, + {"prefix_extractor", + {offset_of(&ColumnFamilyOptions::prefix_extractor), + OptionType::kSliceTransform, OptionVerificationType::kByNameAllowNull, + false, 0}}, + {"memtable_insert_with_hint_prefix_extractor", + {offset_of( + &ColumnFamilyOptions::memtable_insert_with_hint_prefix_extractor), + OptionType::kSliceTransform, OptionVerificationType::kByNameAllowNull, + false, 0}}, + {"memtable_factory", + {offset_of(&ColumnFamilyOptions::memtable_factory), + OptionType::kMemTableRepFactory, OptionVerificationType::kByName, false, + 0}}, + {"table_factory", + {offset_of(&ColumnFamilyOptions::table_factory), OptionType::kTableFactory, + OptionVerificationType::kByName, false, 0}}, + {"compaction_filter", + {offset_of(&ColumnFamilyOptions::compaction_filter), + OptionType::kCompactionFilter, OptionVerificationType::kByName, false, + 0}}, + {"compaction_filter_factory", + {offset_of(&ColumnFamilyOptions::compaction_filter_factory), + OptionType::kCompactionFilterFactory, OptionVerificationType::kByName, + false, 0}}, + {"merge_operator", + {offset_of(&ColumnFamilyOptions::merge_operator), + OptionType::kMergeOperator, OptionVerificationType::kByName, false, 0}}, + {"compaction_style", + {offset_of(&ColumnFamilyOptions::compaction_style), + OptionType::kCompactionStyle, OptionVerificationType::kNormal, false, 0}}, + {"compaction_pri", + {offset_of(&ColumnFamilyOptions::compaction_pri), + OptionType::kCompactionPri, OptionVerificationType::kNormal, false, 0}}}; + +static std::unordered_map + compression_type_string_map = { + {"kNoCompression", kNoCompression}, + {"kSnappyCompression", kSnappyCompression}, + {"kZlibCompression", kZlibCompression}, + {"kBZip2Compression", kBZip2Compression}, + {"kLZ4Compression", kLZ4Compression}, + {"kLZ4HCCompression", kLZ4HCCompression}, + {"kXpressCompression", kXpressCompression}, + {"kZSTD", kZSTD}, + {"kZSTDNotFinalCompression", kZSTDNotFinalCompression}, + {"kDisableCompressionOption", kDisableCompressionOption}}; + +static std::unordered_map + block_base_table_index_type_string_map = { + {"kBinarySearch", BlockBasedTableOptions::IndexType::kBinarySearch}, + {"kHashSearch", BlockBasedTableOptions::IndexType::kHashSearch}, + {"kTwoLevelIndexSearch", + BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch}}; + +static std::unordered_map encoding_type_string_map = + {{"kPlain", kPlain}, {"kPrefix", kPrefix}}; + +static std::unordered_map + compaction_style_string_map = { + {"kCompactionStyleLevel", kCompactionStyleLevel}, + {"kCompactionStyleUniversal", kCompactionStyleUniversal}, + {"kCompactionStyleFIFO", kCompactionStyleFIFO}, + {"kCompactionStyleNone", kCompactionStyleNone}}; + +static std::unordered_map + compaction_pri_string_map = { + {"kByCompensatedSize", kByCompensatedSize}, + {"kOldestLargestSeqFirst", kOldestLargestSeqFirst}, + {"kOldestSmallestSeqFirst", kOldestSmallestSeqFirst}, + {"kMinOverlappingRatio", kMinOverlappingRatio}}; + +static std::unordered_map wal_recovery_mode_string_map = { + {"kTolerateCorruptedTailRecords", + WALRecoveryMode::kTolerateCorruptedTailRecords}, + {"kAbsoluteConsistency", WALRecoveryMode::kAbsoluteConsistency}, + {"kPointInTimeRecovery", WALRecoveryMode::kPointInTimeRecovery}, + {"kSkipAnyCorruptedRecords", WALRecoveryMode::kSkipAnyCorruptedRecords}}; + +static std::unordered_map + access_hint_string_map = {{"NONE", DBOptions::AccessHint::NONE}, + {"NORMAL", DBOptions::AccessHint::NORMAL}, + {"SEQUENTIAL", DBOptions::AccessHint::SEQUENTIAL}, + {"WILLNEED", DBOptions::AccessHint::WILLNEED}}; + +static std::unordered_map info_log_level_string_map = + {{"DEBUG_LEVEL", InfoLogLevel::DEBUG_LEVEL}, + {"INFO_LEVEL", InfoLogLevel::INFO_LEVEL}, + {"WARN_LEVEL", InfoLogLevel::WARN_LEVEL}, + {"ERROR_LEVEL", InfoLogLevel::ERROR_LEVEL}, + {"FATAL_LEVEL", InfoLogLevel::FATAL_LEVEL}, + {"HEADER_LEVEL", InfoLogLevel::HEADER_LEVEL}}; + +extern Status StringToMap( + const std::string& opts_str, + std::unordered_map* opts_map); + +extern bool ParseOptionHelper(char* opt_address, const OptionType& opt_type, + const std::string& value); +#endif // !ROCKSDB_LITE + +} // namespace rocksdb diff --git a/options/options_parser.cc b/options/options_parser.cc new file mode 100644 index 00000000000..2cb60a068ca --- /dev/null +++ b/options/options_parser.cc @@ -0,0 +1,792 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include "options/options_parser.h" + +#include +#include +#include +#include +#include + +#include "options/options_helper.h" +#include "rocksdb/convenience.h" +#include "rocksdb/db.h" +#include "util/cast_util.h" +#include "util/string_util.h" +#include "util/sync_point.h" + +#include "port/port.h" + +namespace rocksdb { + +static const std::string option_file_header = + "# This is a RocksDB option file.\n" + "#\n" + "# For detailed file format spec, please refer to the example file\n" + "# in examples/rocksdb_option_file_example.ini\n" + "#\n" + "\n"; + +Status PersistRocksDBOptions(const DBOptions& db_opt, + const std::vector& cf_names, + const std::vector& cf_opts, + const std::string& file_name, Env* env) { + TEST_SYNC_POINT("PersistRocksDBOptions:start"); + if (cf_names.size() != cf_opts.size()) { + return Status::InvalidArgument( + "cf_names.size() and cf_opts.size() must be the same"); + } + std::unique_ptr writable; + + Status s = env->NewWritableFile(file_name, &writable, EnvOptions()); + if (!s.ok()) { + return s; + } + std::string options_file_content; + + writable->Append(option_file_header + "[" + + opt_section_titles[kOptionSectionVersion] + + "]\n" + " rocksdb_version=" + + ToString(ROCKSDB_MAJOR) + "." + ToString(ROCKSDB_MINOR) + + "." + ToString(ROCKSDB_PATCH) + "\n"); + writable->Append(" options_file_version=" + + ToString(ROCKSDB_OPTION_FILE_MAJOR) + "." + + ToString(ROCKSDB_OPTION_FILE_MINOR) + "\n"); + writable->Append("\n[" + opt_section_titles[kOptionSectionDBOptions] + + "]\n "); + + s = GetStringFromDBOptions(&options_file_content, db_opt, "\n "); + if (!s.ok()) { + writable->Close(); + return s; + } + writable->Append(options_file_content + "\n"); + + for (size_t i = 0; i < cf_opts.size(); ++i) { + // CFOptions section + writable->Append("\n[" + opt_section_titles[kOptionSectionCFOptions] + + " \"" + EscapeOptionString(cf_names[i]) + "\"]\n "); + s = GetStringFromColumnFamilyOptions(&options_file_content, cf_opts[i], + "\n "); + if (!s.ok()) { + writable->Close(); + return s; + } + writable->Append(options_file_content + "\n"); + // TableOptions section + auto* tf = cf_opts[i].table_factory.get(); + if (tf != nullptr) { + writable->Append("[" + opt_section_titles[kOptionSectionTableOptions] + + tf->Name() + " \"" + EscapeOptionString(cf_names[i]) + + "\"]\n "); + options_file_content.clear(); + s = tf->GetOptionString(&options_file_content, "\n "); + if (!s.ok()) { + return s; + } + writable->Append(options_file_content + "\n"); + } + } + writable->Flush(); + writable->Fsync(); + writable->Close(); + + return RocksDBOptionsParser::VerifyRocksDBOptionsFromFile( + db_opt, cf_names, cf_opts, file_name, env); +} + +RocksDBOptionsParser::RocksDBOptionsParser() { Reset(); } + +void RocksDBOptionsParser::Reset() { + db_opt_ = DBOptions(); + db_opt_map_.clear(); + cf_names_.clear(); + cf_opts_.clear(); + cf_opt_maps_.clear(); + has_version_section_ = false; + has_db_options_ = false; + has_default_cf_options_ = false; + for (int i = 0; i < 3; ++i) { + db_version[i] = 0; + opt_file_version[i] = 0; + } +} + +bool RocksDBOptionsParser::IsSection(const std::string& line) { + if (line.size() < 2) { + return false; + } + if (line[0] != '[' || line[line.size() - 1] != ']') { + return false; + } + return true; +} + +Status RocksDBOptionsParser::ParseSection(OptionSection* section, + std::string* title, + std::string* argument, + const std::string& line, + const int line_num) { + *section = kOptionSectionUnknown; + // A section is of the form [ ""], where + // "" is optional. + size_t arg_start_pos = line.find("\""); + size_t arg_end_pos = line.rfind("\""); + // The following if-then check tries to identify whether the input + // section has the optional section argument. + if (arg_start_pos != std::string::npos && arg_start_pos != arg_end_pos) { + *title = TrimAndRemoveComment(line.substr(1, arg_start_pos - 1), true); + *argument = UnescapeOptionString( + line.substr(arg_start_pos + 1, arg_end_pos - arg_start_pos - 1)); + } else { + *title = TrimAndRemoveComment(line.substr(1, line.size() - 2), true); + *argument = ""; + } + for (int i = 0; i < kOptionSectionUnknown; ++i) { + if (title->find(opt_section_titles[i]) == 0) { + if (i == kOptionSectionVersion || i == kOptionSectionDBOptions || + i == kOptionSectionCFOptions) { + if (title->size() == opt_section_titles[i].size()) { + // if true, then it indicats equal + *section = static_cast(i); + return CheckSection(*section, *argument, line_num); + } + } else if (i == kOptionSectionTableOptions) { + // This type of sections has a sufffix at the end of the + // section title + if (title->size() > opt_section_titles[i].size()) { + *section = static_cast(i); + return CheckSection(*section, *argument, line_num); + } + } + } + } + return Status::InvalidArgument(std::string("Unknown section ") + line); +} + +Status RocksDBOptionsParser::InvalidArgument(const int line_num, + const std::string& message) { + return Status::InvalidArgument( + "[RocksDBOptionsParser Error] ", + message + " (at line " + ToString(line_num) + ")"); +} + +Status RocksDBOptionsParser::ParseStatement(std::string* name, + std::string* value, + const std::string& line, + const int line_num) { + size_t eq_pos = line.find("="); + if (eq_pos == std::string::npos) { + return InvalidArgument(line_num, "A valid statement must have a '='."); + } + + *name = TrimAndRemoveComment(line.substr(0, eq_pos), true); + *value = + TrimAndRemoveComment(line.substr(eq_pos + 1, line.size() - eq_pos - 1)); + if (name->empty()) { + return InvalidArgument(line_num, + "A valid statement must have a variable name."); + } + return Status::OK(); +} + +namespace { +bool ReadOneLine(std::istringstream* iss, SequentialFile* seq_file, + std::string* output, bool* has_data, Status* result) { + const int kBufferSize = 4096; + char buffer[kBufferSize + 1]; + Slice input_slice; + + std::string line; + bool has_complete_line = false; + while (!has_complete_line) { + if (std::getline(*iss, line)) { + has_complete_line = !iss->eof(); + } else { + has_complete_line = false; + } + if (!has_complete_line) { + // if we're not sure whether we have a complete line, + // further read from the file. + if (*has_data) { + *result = seq_file->Read(kBufferSize, &input_slice, buffer); + } + if (input_slice.size() == 0) { + // meaning we have read all the data + *has_data = false; + break; + } else { + iss->str(line + input_slice.ToString()); + // reset the internal state of iss so that we can keep reading it. + iss->clear(); + *has_data = (input_slice.size() == kBufferSize); + continue; + } + } + } + *output = line; + return *has_data || has_complete_line; +} +} // namespace + +Status RocksDBOptionsParser::Parse(const std::string& file_name, Env* env, + bool ignore_unknown_options) { + Reset(); + + std::unique_ptr seq_file; + Status s = env->NewSequentialFile(file_name, &seq_file, EnvOptions()); + if (!s.ok()) { + return s; + } + + OptionSection section = kOptionSectionUnknown; + std::string title; + std::string argument; + std::unordered_map opt_map; + std::istringstream iss; + std::string line; + bool has_data = true; + // we only support single-lined statement. + for (int line_num = 1; + ReadOneLine(&iss, seq_file.get(), &line, &has_data, &s); ++line_num) { + if (!s.ok()) { + return s; + } + line = TrimAndRemoveComment(line); + if (line.empty()) { + continue; + } + if (IsSection(line)) { + s = EndSection(section, title, argument, opt_map, ignore_unknown_options); + opt_map.clear(); + if (!s.ok()) { + return s; + } + s = ParseSection(§ion, &title, &argument, line, line_num); + if (!s.ok()) { + return s; + } + } else { + std::string name; + std::string value; + s = ParseStatement(&name, &value, line, line_num); + if (!s.ok()) { + return s; + } + opt_map.insert({name, value}); + } + } + + s = EndSection(section, title, argument, opt_map, ignore_unknown_options); + opt_map.clear(); + if (!s.ok()) { + return s; + } + return ValidityCheck(); +} + +Status RocksDBOptionsParser::CheckSection(const OptionSection section, + const std::string& section_arg, + const int line_num) { + if (section == kOptionSectionDBOptions) { + if (has_db_options_) { + return InvalidArgument( + line_num, + "More than one DBOption section found in the option config file"); + } + has_db_options_ = true; + } else if (section == kOptionSectionCFOptions) { + bool is_default_cf = (section_arg == kDefaultColumnFamilyName); + if (cf_opts_.size() == 0 && !is_default_cf) { + return InvalidArgument( + line_num, + "Default column family must be the first CFOptions section " + "in the option config file"); + } else if (cf_opts_.size() != 0 && is_default_cf) { + return InvalidArgument( + line_num, + "Default column family must be the first CFOptions section " + "in the optio/n config file"); + } else if (GetCFOptions(section_arg) != nullptr) { + return InvalidArgument( + line_num, + "Two identical column families found in option config file"); + } + has_default_cf_options_ |= is_default_cf; + } else if (section == kOptionSectionTableOptions) { + if (GetCFOptions(section_arg) == nullptr) { + return InvalidArgument( + line_num, std::string( + "Does not find a matched column family name in " + "TableOptions section. Column Family Name:") + + section_arg); + } + } else if (section == kOptionSectionVersion) { + if (has_version_section_) { + return InvalidArgument( + line_num, + "More than one Version section found in the option config file."); + } + has_version_section_ = true; + } + return Status::OK(); +} + +Status RocksDBOptionsParser::ParseVersionNumber(const std::string& ver_name, + const std::string& ver_string, + const int max_count, + int* version) { + int version_index = 0; + int current_number = 0; + int current_digit_count = 0; + bool has_dot = false; + for (int i = 0; i < max_count; ++i) { + version[i] = 0; + } + const int kBufferSize = 200; + char buffer[kBufferSize]; + for (size_t i = 0; i < ver_string.size(); ++i) { + if (ver_string[i] == '.') { + if (version_index >= max_count - 1) { + snprintf(buffer, sizeof(buffer) - 1, + "A valid %s can only contains at most %d dots.", + ver_name.c_str(), max_count - 1); + return Status::InvalidArgument(buffer); + } + if (current_digit_count == 0) { + snprintf(buffer, sizeof(buffer) - 1, + "A valid %s must have at least one digit before each dot.", + ver_name.c_str()); + return Status::InvalidArgument(buffer); + } + version[version_index++] = current_number; + current_number = 0; + current_digit_count = 0; + has_dot = true; + } else if (isdigit(ver_string[i])) { + current_number = current_number * 10 + (ver_string[i] - '0'); + current_digit_count++; + } else { + snprintf(buffer, sizeof(buffer) - 1, + "A valid %s can only contains dots and numbers.", + ver_name.c_str()); + return Status::InvalidArgument(buffer); + } + } + version[version_index] = current_number; + if (has_dot && current_digit_count == 0) { + snprintf(buffer, sizeof(buffer) - 1, + "A valid %s must have at least one digit after each dot.", + ver_name.c_str()); + return Status::InvalidArgument(buffer); + } + return Status::OK(); +} + +Status RocksDBOptionsParser::EndSection( + const OptionSection section, const std::string& section_title, + const std::string& section_arg, + const std::unordered_map& opt_map, + bool ignore_unknown_options) { + Status s; + if (section == kOptionSectionDBOptions) { + s = GetDBOptionsFromMap(DBOptions(), opt_map, &db_opt_, true, + ignore_unknown_options); + if (!s.ok()) { + return s; + } + db_opt_map_ = opt_map; + } else if (section == kOptionSectionCFOptions) { + // This condition should be ensured earlier in ParseSection + // so we make an assertion here. + assert(GetCFOptions(section_arg) == nullptr); + cf_names_.emplace_back(section_arg); + cf_opts_.emplace_back(); + s = GetColumnFamilyOptionsFromMap(ColumnFamilyOptions(), opt_map, + &cf_opts_.back(), true, + ignore_unknown_options); + if (!s.ok()) { + return s; + } + // keep the parsed string. + cf_opt_maps_.emplace_back(opt_map); + } else if (section == kOptionSectionTableOptions) { + assert(GetCFOptions(section_arg) != nullptr); + auto* cf_opt = GetCFOptionsImpl(section_arg); + if (cf_opt == nullptr) { + return Status::InvalidArgument( + "The specified column family must be defined before the " + "TableOptions section:", + section_arg); + } + // Ignore error as table factory deserialization is optional + s = GetTableFactoryFromMap( + section_title.substr( + opt_section_titles[kOptionSectionTableOptions].size()), + opt_map, &(cf_opt->table_factory), ignore_unknown_options); + if (!s.ok()) { + return s; + } + } else if (section == kOptionSectionVersion) { + for (const auto pair : opt_map) { + if (pair.first == "rocksdb_version") { + s = ParseVersionNumber(pair.first, pair.second, 3, db_version); + if (!s.ok()) { + return s; + } + } else if (pair.first == "options_file_version") { + s = ParseVersionNumber(pair.first, pair.second, 2, opt_file_version); + if (!s.ok()) { + return s; + } + if (opt_file_version[0] < 1) { + return Status::InvalidArgument( + "A valid options_file_version must be at least 1."); + } + } + } + } + return Status::OK(); +} + +Status RocksDBOptionsParser::ValidityCheck() { + if (!has_db_options_) { + return Status::Corruption( + "A RocksDB Option file must have a single DBOptions section"); + } + if (!has_default_cf_options_) { + return Status::Corruption( + "A RocksDB Option file must have a single CFOptions:default section"); + } + + return Status::OK(); +} + +std::string RocksDBOptionsParser::TrimAndRemoveComment(const std::string& line, + bool trim_only) { + size_t start = 0; + size_t end = line.size(); + + // we only support "#" style comment + if (!trim_only) { + size_t search_pos = 0; + while (search_pos < line.size()) { + size_t comment_pos = line.find('#', search_pos); + if (comment_pos == std::string::npos) { + break; + } + if (comment_pos == 0 || line[comment_pos - 1] != '\\') { + end = comment_pos; + break; + } + search_pos = comment_pos + 1; + } + } + + while (start < end && isspace(line[start]) != 0) { + ++start; + } + + // start < end implies end > 0. + while (start < end && isspace(line[end - 1]) != 0) { + --end; + } + + if (start < end) { + return line.substr(start, end - start); + } + + return ""; +} + +namespace { +bool AreEqualDoubles(const double a, const double b) { + return (fabs(a - b) < 0.00001); +} +} // namespace + +bool AreEqualOptions( + const char* opt1, const char* opt2, const OptionTypeInfo& type_info, + const std::string& opt_name, + const std::unordered_map* opt_map) { + const char* offset1 = opt1 + type_info.offset; + const char* offset2 = opt2 + type_info.offset; + + switch (type_info.type) { + case OptionType::kBoolean: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kInt: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kVectorInt: + return (*reinterpret_cast*>(offset1) == + *reinterpret_cast*>(offset2)); + case OptionType::kUInt: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kUInt32T: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kUInt64T: + { + uint64_t v1, v2; + GetUnaligned(reinterpret_cast(offset1), &v1); + GetUnaligned(reinterpret_cast(offset2), &v2); + return (v1 == v2); + } + case OptionType::kSizeT: + { + size_t v1, v2; + GetUnaligned(reinterpret_cast(offset1), &v1); + GetUnaligned(reinterpret_cast(offset2), &v2); + return (v1 == v2); + } + case OptionType::kString: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kDouble: + return AreEqualDoubles(*reinterpret_cast(offset1), + *reinterpret_cast(offset2)); + case OptionType::kCompactionStyle: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kCompactionPri: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kCompressionType: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kVectorCompressionType: { + const auto* vec1 = + reinterpret_cast*>(offset1); + const auto* vec2 = + reinterpret_cast*>(offset2); + return (*vec1 == *vec2); + } + case OptionType::kChecksumType: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kBlockBasedTableIndexType: + return ( + *reinterpret_cast( + offset1) == + *reinterpret_cast(offset2)); + case OptionType::kWALRecoveryMode: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kAccessHint: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + case OptionType::kInfoLogLevel: + return (*reinterpret_cast(offset1) == + *reinterpret_cast(offset2)); + default: + if (type_info.verification == OptionVerificationType::kByName || + type_info.verification == OptionVerificationType::kByNameAllowNull) { + std::string value1; + bool result = + SerializeSingleOptionHelper(offset1, type_info.type, &value1); + if (result == false) { + return false; + } + if (opt_map == nullptr) { + return true; + } + auto iter = opt_map->find(opt_name); + if (iter == opt_map->end()) { + return true; + } else { + if (type_info.verification == + OptionVerificationType::kByNameAllowNull) { + if (iter->second == kNullptrString || value1 == kNullptrString) { + return true; + } + } + return (value1 == iter->second); + } + } + return false; + } +} + +Status RocksDBOptionsParser::VerifyRocksDBOptionsFromFile( + const DBOptions& db_opt, const std::vector& cf_names, + const std::vector& cf_opts, + const std::string& file_name, Env* env, + OptionsSanityCheckLevel sanity_check_level, bool ignore_unknown_options) { + RocksDBOptionsParser parser; + std::unique_ptr seq_file; + Status s = parser.Parse(file_name, env, ignore_unknown_options); + if (!s.ok()) { + return s; + } + + // Verify DBOptions + s = VerifyDBOptions(db_opt, *parser.db_opt(), parser.db_opt_map(), + sanity_check_level); + if (!s.ok()) { + return s; + } + + // Verify ColumnFamily Name + if (cf_names.size() != parser.cf_names()->size()) { + if (sanity_check_level >= kSanityLevelLooselyCompatible) { + return Status::InvalidArgument( + "[RocksDBOptionParser Error] The persisted options does not have " + "the same number of column family names as the db instance."); + } else if (cf_opts.size() > parser.cf_opts()->size()) { + return Status::InvalidArgument( + "[RocksDBOptionsParser Error]", + "The persisted options file has less number of column family " + "names than that of the specified one."); + } + } + for (size_t i = 0; i < cf_names.size(); ++i) { + if (cf_names[i] != parser.cf_names()->at(i)) { + return Status::InvalidArgument( + "[RocksDBOptionParser Error] The persisted options and the db" + "instance does not have the same name for column family ", + ToString(i)); + } + } + + // Verify Column Family Options + if (cf_opts.size() != parser.cf_opts()->size()) { + if (sanity_check_level >= kSanityLevelLooselyCompatible) { + return Status::InvalidArgument( + "[RocksDBOptionsParser Error]", + "The persisted options does not have the same number of " + "column families as the db instance."); + } else if (cf_opts.size() > parser.cf_opts()->size()) { + return Status::InvalidArgument( + "[RocksDBOptionsParser Error]", + "The persisted options file has less number of column families " + "than that of the specified number."); + } + } + for (size_t i = 0; i < cf_opts.size(); ++i) { + s = VerifyCFOptions(cf_opts[i], parser.cf_opts()->at(i), + &(parser.cf_opt_maps()->at(i)), sanity_check_level); + if (!s.ok()) { + return s; + } + s = VerifyTableFactory(cf_opts[i].table_factory.get(), + parser.cf_opts()->at(i).table_factory.get(), + sanity_check_level); + if (!s.ok()) { + return s; + } + } + + return Status::OK(); +} + +Status RocksDBOptionsParser::VerifyDBOptions( + const DBOptions& base_opt, const DBOptions& persisted_opt, + const std::unordered_map* opt_map, + OptionsSanityCheckLevel sanity_check_level) { + for (auto pair : db_options_type_info) { + if (pair.second.verification == OptionVerificationType::kDeprecated) { + // We skip checking deprecated variables as they might + // contain random values since they might not be initialized + continue; + } + if (DBOptionSanityCheckLevel(pair.first) <= sanity_check_level) { + if (!AreEqualOptions(reinterpret_cast(&base_opt), + reinterpret_cast(&persisted_opt), + pair.second, pair.first, nullptr)) { + const size_t kBufferSize = 2048; + char buffer[kBufferSize]; + std::string base_value; + std::string persisted_value; + SerializeSingleOptionHelper( + reinterpret_cast(&base_opt) + pair.second.offset, + pair.second.type, &base_value); + SerializeSingleOptionHelper( + reinterpret_cast(&persisted_opt) + pair.second.offset, + pair.second.type, &persisted_value); + snprintf(buffer, sizeof(buffer), + "[RocksDBOptionsParser]: " + "failed the verification on DBOptions::%s --- " + "The specified one is %s while the persisted one is %s.\n", + pair.first.c_str(), base_value.c_str(), + persisted_value.c_str()); + return Status::InvalidArgument(Slice(buffer, strlen(buffer))); + } + } + } + return Status::OK(); +} + +Status RocksDBOptionsParser::VerifyCFOptions( + const ColumnFamilyOptions& base_opt, + const ColumnFamilyOptions& persisted_opt, + const std::unordered_map* persisted_opt_map, + OptionsSanityCheckLevel sanity_check_level) { + for (auto& pair : cf_options_type_info) { + if (pair.second.verification == OptionVerificationType::kDeprecated) { + // We skip checking deprecated variables as they might + // contain random values since they might not be initialized + continue; + } + if (CFOptionSanityCheckLevel(pair.first) <= sanity_check_level) { + if (!AreEqualOptions(reinterpret_cast(&base_opt), + reinterpret_cast(&persisted_opt), + pair.second, pair.first, persisted_opt_map)) { + const size_t kBufferSize = 2048; + char buffer[kBufferSize]; + std::string base_value; + std::string persisted_value; + SerializeSingleOptionHelper( + reinterpret_cast(&base_opt) + pair.second.offset, + pair.second.type, &base_value); + SerializeSingleOptionHelper( + reinterpret_cast(&persisted_opt) + pair.second.offset, + pair.second.type, &persisted_value); + snprintf(buffer, sizeof(buffer), + "[RocksDBOptionsParser]: " + "failed the verification on ColumnFamilyOptions::%s --- " + "The specified one is %s while the persisted one is %s.\n", + pair.first.c_str(), base_value.c_str(), + persisted_value.c_str()); + return Status::InvalidArgument(Slice(buffer, sizeof(buffer))); + } + } + } + return Status::OK(); +} + +Status RocksDBOptionsParser::VerifyTableFactory( + const TableFactory* base_tf, const TableFactory* file_tf, + OptionsSanityCheckLevel sanity_check_level) { + if (base_tf && file_tf) { + if (sanity_check_level > kSanityLevelNone && + std::string(base_tf->Name()) != std::string(file_tf->Name())) { + return Status::Corruption( + "[RocksDBOptionsParser]: " + "failed the verification on TableFactory->Name()"); + } + if (base_tf->Name() == BlockBasedTableFactory::kName) { + return VerifyBlockBasedTableFactory( + static_cast_with_check(base_tf), + static_cast_with_check(file_tf), + sanity_check_level); + } + // TODO(yhchiang): add checks for other table factory types + } else { + // TODO(yhchiang): further support sanity check here + } + return Status::OK(); +} +} // namespace rocksdb + +#endif // !ROCKSDB_LITE diff --git a/options/options_parser.h b/options/options_parser.h new file mode 100644 index 00000000000..5545c0b0fa8 --- /dev/null +++ b/options/options_parser.h @@ -0,0 +1,146 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include +#include + +#include "options/options_helper.h" +#include "options/options_sanity_check.h" +#include "rocksdb/env.h" +#include "rocksdb/options.h" +#include "table/block_based_table_factory.h" + +namespace rocksdb { + +#ifndef ROCKSDB_LITE + +#define ROCKSDB_OPTION_FILE_MAJOR 1 +#define ROCKSDB_OPTION_FILE_MINOR 1 + +enum OptionSection : char { + kOptionSectionVersion = 0, + kOptionSectionDBOptions, + kOptionSectionCFOptions, + kOptionSectionTableOptions, + kOptionSectionUnknown +}; + +static const std::string opt_section_titles[] = { + "Version", "DBOptions", "CFOptions", "TableOptions/", "Unknown"}; + +Status PersistRocksDBOptions(const DBOptions& db_opt, + const std::vector& cf_names, + const std::vector& cf_opts, + const std::string& file_name, Env* env); + +extern bool AreEqualOptions( + const char* opt1, const char* opt2, const OptionTypeInfo& type_info, + const std::string& opt_name, + const std::unordered_map* opt_map); + +class RocksDBOptionsParser { + public: + explicit RocksDBOptionsParser(); + ~RocksDBOptionsParser() {} + void Reset(); + + Status Parse(const std::string& file_name, Env* env, + bool ignore_unknown_options = false); + static std::string TrimAndRemoveComment(const std::string& line, + const bool trim_only = false); + + const DBOptions* db_opt() const { return &db_opt_; } + const std::unordered_map* db_opt_map() const { + return &db_opt_map_; + } + const std::vector* cf_opts() const { return &cf_opts_; } + const std::vector* cf_names() const { return &cf_names_; } + const std::vector>* cf_opt_maps() + const { + return &cf_opt_maps_; + } + + const ColumnFamilyOptions* GetCFOptions(const std::string& name) { + return GetCFOptionsImpl(name); + } + size_t NumColumnFamilies() { return cf_opts_.size(); } + + static Status VerifyRocksDBOptionsFromFile( + const DBOptions& db_opt, const std::vector& cf_names, + const std::vector& cf_opts, + const std::string& file_name, Env* env, + OptionsSanityCheckLevel sanity_check_level = kSanityLevelExactMatch, + bool ignore_unknown_options = false); + + static Status VerifyDBOptions( + const DBOptions& base_opt, const DBOptions& new_opt, + const std::unordered_map* new_opt_map = nullptr, + OptionsSanityCheckLevel sanity_check_level = kSanityLevelExactMatch); + + static Status VerifyCFOptions( + const ColumnFamilyOptions& base_opt, const ColumnFamilyOptions& new_opt, + const std::unordered_map* new_opt_map = nullptr, + OptionsSanityCheckLevel sanity_check_level = kSanityLevelExactMatch); + + static Status VerifyTableFactory( + const TableFactory* base_tf, const TableFactory* file_tf, + OptionsSanityCheckLevel sanity_check_level = kSanityLevelExactMatch); + + static Status ExtraParserCheck(const RocksDBOptionsParser& input_parser); + + protected: + bool IsSection(const std::string& line); + Status ParseSection(OptionSection* section, std::string* title, + std::string* argument, const std::string& line, + const int line_num); + + Status CheckSection(const OptionSection section, + const std::string& section_arg, const int line_num); + + Status ParseStatement(std::string* name, std::string* value, + const std::string& line, const int line_num); + + Status EndSection(const OptionSection section, const std::string& title, + const std::string& section_arg, + const std::unordered_map& opt_map, + bool ignore_unknown_options); + + Status ValidityCheck(); + + Status InvalidArgument(const int line_num, const std::string& message); + + Status ParseVersionNumber(const std::string& ver_name, + const std::string& ver_string, const int max_count, + int* version); + + ColumnFamilyOptions* GetCFOptionsImpl(const std::string& name) { + assert(cf_names_.size() == cf_opts_.size()); + for (size_t i = 0; i < cf_names_.size(); ++i) { + if (cf_names_[i] == name) { + return &cf_opts_[i]; + } + } + return nullptr; + } + + private: + DBOptions db_opt_; + std::unordered_map db_opt_map_; + std::vector cf_names_; + std::vector cf_opts_; + std::vector> cf_opt_maps_; + bool has_version_section_; + bool has_db_options_; + bool has_default_cf_options_; + int db_version[3]; + int opt_file_version[3]; +}; + +#endif // !ROCKSDB_LITE + +} // namespace rocksdb diff --git a/options/options_sanity_check.cc b/options/options_sanity_check.cc new file mode 100644 index 00000000000..d3afcc060ed --- /dev/null +++ b/options/options_sanity_check.cc @@ -0,0 +1,38 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#include "options/options_sanity_check.h" + +namespace rocksdb { + +namespace { +OptionsSanityCheckLevel SanityCheckLevelHelper( + const std::unordered_map& smap, + const std::string& name) { + auto iter = smap.find(name); + return iter != smap.end() ? iter->second : kSanityLevelExactMatch; +} +} + +OptionsSanityCheckLevel DBOptionSanityCheckLevel( + const std::string& option_name) { + return SanityCheckLevelHelper(sanity_level_db_options, option_name); +} + +OptionsSanityCheckLevel CFOptionSanityCheckLevel( + const std::string& option_name) { + return SanityCheckLevelHelper(sanity_level_cf_options, option_name); +} + +OptionsSanityCheckLevel BBTOptionSanityCheckLevel( + const std::string& option_name) { + return SanityCheckLevelHelper(sanity_level_bbt_options, option_name); +} + +} // namespace rocksdb + +#endif // !ROCKSDB_LITE diff --git a/options/options_sanity_check.h b/options/options_sanity_check.h new file mode 100644 index 00000000000..118fdd208ba --- /dev/null +++ b/options/options_sanity_check.h @@ -0,0 +1,48 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include + +#ifndef ROCKSDB_LITE +namespace rocksdb { +// This enum defines the RocksDB options sanity level. +enum OptionsSanityCheckLevel : unsigned char { + // Performs no sanity check at all. + kSanityLevelNone = 0x00, + // Performs minimum check to ensure the RocksDB instance can be + // opened without corrupting / mis-interpreting the data. + kSanityLevelLooselyCompatible = 0x01, + // Perform exact match sanity check. + kSanityLevelExactMatch = 0xFF, +}; + +// The sanity check level for DB options +static const std::unordered_map + sanity_level_db_options {}; + +// The sanity check level for column-family options +static const std::unordered_map + sanity_level_cf_options = { + {"comparator", kSanityLevelLooselyCompatible}, + {"table_factory", kSanityLevelLooselyCompatible}, + {"merge_operator", kSanityLevelLooselyCompatible}}; + +// The sanity check level for block-based table options +static const std::unordered_map + sanity_level_bbt_options {}; + +OptionsSanityCheckLevel DBOptionSanityCheckLevel( + const std::string& options_name); +OptionsSanityCheckLevel CFOptionSanityCheckLevel( + const std::string& options_name); +OptionsSanityCheckLevel BBTOptionSanityCheckLevel( + const std::string& options_name); + +} // namespace rocksdb + +#endif // !ROCKSDB_LITE diff --git a/options/options_settable_test.cc b/options/options_settable_test.cc new file mode 100644 index 00000000000..ab9989fb46f --- /dev/null +++ b/options/options_settable_test.cc @@ -0,0 +1,454 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include + +#include "options/options_parser.h" +#include "rocksdb/convenience.h" +#include "util/testharness.h" + +#ifndef GFLAGS +bool FLAGS_enable_print = false; +#else +#include +using GFLAGS::ParseCommandLineFlags; +DEFINE_bool(enable_print, false, "Print options generated to console."); +#endif // GFLAGS + +namespace rocksdb { + +// Verify options are settable from options strings. +// We take the approach that depends on compiler behavior that copy constructor +// won't touch implicit padding bytes, so that the test is fragile. +// As a result, we only run the tests to verify new fields in options are +// settable through string on limited platforms as it depends on behavior of +// compilers. +#ifndef ROCKSDB_LITE +#if defined OS_LINUX || defined OS_WIN +#ifndef __clang__ + +class OptionsSettableTest : public testing::Test { + public: + OptionsSettableTest() {} +}; + +const char kSpecialChar = 'z'; +typedef std::vector> OffsetGap; + +void FillWithSpecialChar(char* start_ptr, size_t total_size, + const OffsetGap& blacklist) { + size_t offset = 0; + for (auto& pair : blacklist) { + std::memset(start_ptr + offset, kSpecialChar, pair.first - offset); + offset = pair.first + pair.second; + } + std::memset(start_ptr + offset, kSpecialChar, total_size - offset); +} + +int NumUnsetBytes(char* start_ptr, size_t total_size, + const OffsetGap& blacklist) { + int total_unset_bytes_base = 0; + size_t offset = 0; + for (auto& pair : blacklist) { + for (char* ptr = start_ptr + offset; ptr < start_ptr + pair.first; ptr++) { + if (*ptr == kSpecialChar) { + total_unset_bytes_base++; + } + } + offset = pair.first + pair.second; + } + for (char* ptr = start_ptr + offset; ptr < start_ptr + total_size; ptr++) { + if (*ptr == kSpecialChar) { + total_unset_bytes_base++; + } + } + return total_unset_bytes_base; +} + +// If the test fails, likely a new option is added to BlockBasedTableOptions +// but it cannot be set through GetBlockBasedTableOptionsFromString(), or the +// test is not updated accordingly. +// After adding an option, we need to make sure it is settable by +// GetBlockBasedTableOptionsFromString() and add the option to the input string +// passed to the GetBlockBasedTableOptionsFromString() in this test. +// If it is a complicated type, you also need to add the field to +// kBbtoBlacklist, and maybe add customized verification for it. +TEST_F(OptionsSettableTest, BlockBasedTableOptionsAllFieldsSettable) { + // Items in the form of . Need to be in ascending order + // and not overlapping. Need to updated if new pointer-option is added. + const OffsetGap kBbtoBlacklist = { + {offsetof(struct BlockBasedTableOptions, flush_block_policy_factory), + sizeof(std::shared_ptr)}, + {offsetof(struct BlockBasedTableOptions, block_cache), + sizeof(std::shared_ptr)}, + {offsetof(struct BlockBasedTableOptions, persistent_cache), + sizeof(std::shared_ptr)}, + {offsetof(struct BlockBasedTableOptions, block_cache_compressed), + sizeof(std::shared_ptr)}, + {offsetof(struct BlockBasedTableOptions, filter_policy), + sizeof(std::shared_ptr)}, + }; + + // In this test, we catch a new option of BlockBasedTableOptions that is not + // settable through GetBlockBasedTableOptionsFromString(). + // We count padding bytes of the option struct, and assert it to be the same + // as unset bytes of an option struct initialized by + // GetBlockBasedTableOptionsFromString(). + + char* bbto_ptr = new char[sizeof(BlockBasedTableOptions)]; + + // Count padding bytes by setting all bytes in the memory to a special char, + // copy a well constructed struct to this memory and see how many special + // bytes left. + BlockBasedTableOptions* bbto = new (bbto_ptr) BlockBasedTableOptions(); + FillWithSpecialChar(bbto_ptr, sizeof(BlockBasedTableOptions), kBbtoBlacklist); + // It based on the behavior of compiler that padding bytes are not changed + // when copying the struct. It's prone to failure when compiler behavior + // changes. We verify there is unset bytes to detect the case. + *bbto = BlockBasedTableOptions(); + int unset_bytes_base = + NumUnsetBytes(bbto_ptr, sizeof(BlockBasedTableOptions), kBbtoBlacklist); + ASSERT_GT(unset_bytes_base, 0); + bbto->~BlockBasedTableOptions(); + + // Construct the base option passed into + // GetBlockBasedTableOptionsFromString(). + bbto = new (bbto_ptr) BlockBasedTableOptions(); + FillWithSpecialChar(bbto_ptr, sizeof(BlockBasedTableOptions), kBbtoBlacklist); + // This option is not setable: + bbto->use_delta_encoding = true; + + char* new_bbto_ptr = new char[sizeof(BlockBasedTableOptions)]; + BlockBasedTableOptions* new_bbto = + new (new_bbto_ptr) BlockBasedTableOptions(); + FillWithSpecialChar(new_bbto_ptr, sizeof(BlockBasedTableOptions), + kBbtoBlacklist); + + // Need to update the option string if a new option is added. + ASSERT_OK(GetBlockBasedTableOptionsFromString( + *bbto, + "cache_index_and_filter_blocks=1;" + "cache_index_and_filter_blocks_with_high_priority=true;" + "pin_l0_filter_and_index_blocks_in_cache=1;" + "index_type=kHashSearch;" + "checksum=kxxHash;hash_index_allow_collision=1;no_block_cache=1;" + "block_cache=1M;block_cache_compressed=1k;block_size=1024;" + "block_size_deviation=8;block_restart_interval=4; " + "metadata_block_size=1024;" + "partition_filters=false;" + "index_block_restart_interval=4;" + "filter_policy=bloomfilter:4:true;whole_key_filtering=1;" + "format_version=1;" + "hash_index_allow_collision=false;" + "verify_compression=true;read_amp_bytes_per_bit=0", + new_bbto)); + + ASSERT_EQ(unset_bytes_base, + NumUnsetBytes(new_bbto_ptr, sizeof(BlockBasedTableOptions), + kBbtoBlacklist)); + + ASSERT_TRUE(new_bbto->block_cache.get() != nullptr); + ASSERT_TRUE(new_bbto->block_cache_compressed.get() != nullptr); + ASSERT_TRUE(new_bbto->filter_policy.get() != nullptr); + + bbto->~BlockBasedTableOptions(); + new_bbto->~BlockBasedTableOptions(); + + delete[] bbto_ptr; + delete[] new_bbto_ptr; +} + +// If the test fails, likely a new option is added to DBOptions +// but it cannot be set through GetDBOptionsFromString(), or the test is not +// updated accordingly. +// After adding an option, we need to make sure it is settable by +// GetDBOptionsFromString() and add the option to the input string passed to +// DBOptionsFromString()in this test. +// If it is a complicated type, you also need to add the field to +// kDBOptionsBlacklist, and maybe add customized verification for it. +TEST_F(OptionsSettableTest, DBOptionsAllFieldsSettable) { + const OffsetGap kDBOptionsBlacklist = { + {offsetof(struct DBOptions, env), sizeof(Env*)}, + {offsetof(struct DBOptions, rate_limiter), + sizeof(std::shared_ptr)}, + {offsetof(struct DBOptions, sst_file_manager), + sizeof(std::shared_ptr)}, + {offsetof(struct DBOptions, info_log), sizeof(std::shared_ptr)}, + {offsetof(struct DBOptions, statistics), + sizeof(std::shared_ptr)}, + {offsetof(struct DBOptions, db_paths), sizeof(std::vector)}, + {offsetof(struct DBOptions, db_log_dir), sizeof(std::string)}, + {offsetof(struct DBOptions, wal_dir), sizeof(std::string)}, + {offsetof(struct DBOptions, write_buffer_manager), + sizeof(std::shared_ptr)}, + {offsetof(struct DBOptions, listeners), + sizeof(std::vector>)}, + {offsetof(struct DBOptions, row_cache), sizeof(std::shared_ptr)}, + {offsetof(struct DBOptions, wal_filter), sizeof(const WalFilter*)}, + }; + + char* options_ptr = new char[sizeof(DBOptions)]; + + // Count padding bytes by setting all bytes in the memory to a special char, + // copy a well constructed struct to this memory and see how many special + // bytes left. + DBOptions* options = new (options_ptr) DBOptions(); + FillWithSpecialChar(options_ptr, sizeof(DBOptions), kDBOptionsBlacklist); + // It based on the behavior of compiler that padding bytes are not changed + // when copying the struct. It's prone to failure when compiler behavior + // changes. We verify there is unset bytes to detect the case. + *options = DBOptions(); + int unset_bytes_base = + NumUnsetBytes(options_ptr, sizeof(DBOptions), kDBOptionsBlacklist); + ASSERT_GT(unset_bytes_base, 0); + options->~DBOptions(); + + options = new (options_ptr) DBOptions(); + FillWithSpecialChar(options_ptr, sizeof(DBOptions), kDBOptionsBlacklist); + + char* new_options_ptr = new char[sizeof(DBOptions)]; + DBOptions* new_options = new (new_options_ptr) DBOptions(); + FillWithSpecialChar(new_options_ptr, sizeof(DBOptions), kDBOptionsBlacklist); + + // Need to update the option string if a new option is added. + ASSERT_OK( + GetDBOptionsFromString(*options, + "wal_bytes_per_sync=4295048118;" + "delete_obsolete_files_period_micros=4294967758;" + "WAL_ttl_seconds=4295008036;" + "WAL_size_limit_MB=4295036161;" + "wal_dir=path/to/wal_dir;" + "db_write_buffer_size=2587;" + "max_subcompactions=64330;" + "table_cache_numshardbits=28;" + "max_open_files=72;" + "max_file_opening_threads=35;" + "max_background_jobs=8;" + "base_background_compactions=3;" + "max_background_compactions=33;" + "use_fsync=true;" + "use_adaptive_mutex=false;" + "max_total_wal_size=4295005604;" + "compaction_readahead_size=0;" + "new_table_reader_for_compaction_inputs=false;" + "keep_log_file_num=4890;" + "skip_stats_update_on_db_open=false;" + "max_manifest_file_size=4295009941;" + "db_log_dir=path/to/db_log_dir;" + "skip_log_error_on_recovery=true;" + "writable_file_max_buffer_size=1048576;" + "paranoid_checks=true;" + "is_fd_close_on_exec=false;" + "bytes_per_sync=4295013613;" + "enable_thread_tracking=false;" + "recycle_log_file_num=0;" + "create_missing_column_families=true;" + "log_file_time_to_roll=3097;" + "max_background_flushes=35;" + "create_if_missing=false;" + "error_if_exists=true;" + "delayed_write_rate=4294976214;" + "manifest_preallocation_size=1222;" + "allow_mmap_writes=false;" + "stats_dump_period_sec=70127;" + "allow_fallocate=true;" + "allow_mmap_reads=false;" + "use_direct_reads=false;" + "use_direct_io_for_flush_and_compaction=false;" + "max_log_file_size=4607;" + "random_access_max_buffer_size=1048576;" + "advise_random_on_open=true;" + "fail_if_options_file_error=false;" + "enable_pipelined_write=false;" + "allow_concurrent_memtable_write=true;" + "wal_recovery_mode=kPointInTimeRecovery;" + "enable_write_thread_adaptive_yield=true;" + "write_thread_slow_yield_usec=5;" + "write_thread_max_yield_usec=1000;" + "access_hint_on_compaction_start=NONE;" + "info_log_level=DEBUG_LEVEL;" + "dump_malloc_stats=false;" + "allow_2pc=false;" + "avoid_flush_during_recovery=false;" + "avoid_flush_during_shutdown=false;" + "allow_ingest_behind=false;" + "concurrent_prepare=false;" + "manual_wal_flush=false;", + new_options)); + + ASSERT_EQ(unset_bytes_base, NumUnsetBytes(new_options_ptr, sizeof(DBOptions), + kDBOptionsBlacklist)); + + options->~DBOptions(); + new_options->~DBOptions(); + + delete[] options_ptr; + delete[] new_options_ptr; +} + +// If the test fails, likely a new option is added to ColumnFamilyOptions +// but it cannot be set through GetColumnFamilyOptionsFromString(), or the +// test is not updated accordingly. +// After adding an option, we need to make sure it is settable by +// GetColumnFamilyOptionsFromString() and add the option to the input +// string passed to GetColumnFamilyOptionsFromString()in this test. +// If it is a complicated type, you also need to add the field to +// kColumnFamilyOptionsBlacklist, and maybe add customized verification +// for it. +TEST_F(OptionsSettableTest, ColumnFamilyOptionsAllFieldsSettable) { + // options in the blacklist need to appear in the same order as in + // ColumnFamilyOptions. + const OffsetGap kColumnFamilyOptionsBlacklist = { + {offset_of(&ColumnFamilyOptions::inplace_callback), + sizeof(UpdateStatus(*)(char*, uint32_t*, Slice, std::string*))}, + {offset_of( + &ColumnFamilyOptions::memtable_insert_with_hint_prefix_extractor), + sizeof(std::shared_ptr)}, + {offset_of(&ColumnFamilyOptions::compression_per_level), + sizeof(std::vector)}, + {offset_of( + &ColumnFamilyOptions::max_bytes_for_level_multiplier_additional), + sizeof(std::vector)}, + {offset_of(&ColumnFamilyOptions::memtable_factory), + sizeof(std::shared_ptr)}, + {offset_of(&ColumnFamilyOptions::table_properties_collector_factories), + sizeof(ColumnFamilyOptions::TablePropertiesCollectorFactories)}, + {offset_of(&ColumnFamilyOptions::comparator), sizeof(Comparator*)}, + {offset_of(&ColumnFamilyOptions::merge_operator), + sizeof(std::shared_ptr)}, + {offset_of(&ColumnFamilyOptions::compaction_filter), + sizeof(const CompactionFilter*)}, + {offset_of(&ColumnFamilyOptions::compaction_filter_factory), + sizeof(std::shared_ptr)}, + {offset_of(&ColumnFamilyOptions::prefix_extractor), + sizeof(std::shared_ptr)}, + {offset_of(&ColumnFamilyOptions::table_factory), + sizeof(std::shared_ptr)}, + }; + + char* options_ptr = new char[sizeof(ColumnFamilyOptions)]; + + // Count padding bytes by setting all bytes in the memory to a special char, + // copy a well constructed struct to this memory and see how many special + // bytes left. + ColumnFamilyOptions* options = new (options_ptr) ColumnFamilyOptions(); + FillWithSpecialChar(options_ptr, sizeof(ColumnFamilyOptions), + kColumnFamilyOptionsBlacklist); + // It based on the behavior of compiler that padding bytes are not changed + // when copying the struct. It's prone to failure when compiler behavior + // changes. We verify there is unset bytes to detect the case. + *options = ColumnFamilyOptions(); + + // Deprecatd option which is not initialized. Need to set it to avoid + // Valgrind error + options->max_mem_compaction_level = 0; + + int unset_bytes_base = NumUnsetBytes(options_ptr, sizeof(ColumnFamilyOptions), + kColumnFamilyOptionsBlacklist); + ASSERT_GT(unset_bytes_base, 0); + options->~ColumnFamilyOptions(); + + options = new (options_ptr) ColumnFamilyOptions(); + FillWithSpecialChar(options_ptr, sizeof(ColumnFamilyOptions), + kColumnFamilyOptionsBlacklist); + + // Following options are not settable through + // GetColumnFamilyOptionsFromString(): + options->rate_limit_delay_max_milliseconds = 33; + options->compaction_options_universal = CompactionOptionsUniversal(); + options->compression_opts = CompressionOptions(); + options->hard_rate_limit = 0; + options->soft_rate_limit = 0; + options->compaction_options_fifo = CompactionOptionsFIFO(); + options->max_mem_compaction_level = 0; + + char* new_options_ptr = new char[sizeof(ColumnFamilyOptions)]; + ColumnFamilyOptions* new_options = + new (new_options_ptr) ColumnFamilyOptions(); + FillWithSpecialChar(new_options_ptr, sizeof(ColumnFamilyOptions), + kColumnFamilyOptionsBlacklist); + + // Need to update the option string if a new option is added. + ASSERT_OK(GetColumnFamilyOptionsFromString( + *options, + "compaction_filter_factory=mpudlojcujCompactionFilterFactory;" + "table_factory=PlainTable;" + "prefix_extractor=rocksdb.CappedPrefix.13;" + "comparator=leveldb.BytewiseComparator;" + "compression_per_level=kBZip2Compression:kBZip2Compression:" + "kBZip2Compression:kNoCompression:kZlibCompression:kBZip2Compression:" + "kSnappyCompression;" + "max_bytes_for_level_base=986;" + "bloom_locality=8016;" + "target_file_size_base=4294976376;" + "memtable_huge_page_size=2557;" + "max_successive_merges=5497;" + "max_sequential_skip_in_iterations=4294971408;" + "arena_block_size=1893;" + "target_file_size_multiplier=35;" + "min_write_buffer_number_to_merge=9;" + "max_write_buffer_number=84;" + "write_buffer_size=1653;" + "max_compaction_bytes=64;" + "max_bytes_for_level_multiplier=60;" + "memtable_factory=SkipListFactory;" + "compression=kNoCompression;" + "bottommost_compression=kDisableCompressionOption;" + "level0_stop_writes_trigger=33;" + "num_levels=99;" + "level0_slowdown_writes_trigger=22;" + "level0_file_num_compaction_trigger=14;" + "compaction_filter=urxcqstuwnCompactionFilter;" + "soft_rate_limit=530.615385;" + "soft_pending_compaction_bytes_limit=0;" + "max_write_buffer_number_to_maintain=84;" + "merge_operator=aabcxehazrMergeOperator;" + "memtable_prefix_bloom_size_ratio=0.4642;" + "memtable_insert_with_hint_prefix_extractor=rocksdb.CappedPrefix.13;" + "paranoid_file_checks=true;" + "force_consistency_checks=true;" + "inplace_update_num_locks=7429;" + "optimize_filters_for_hits=false;" + "level_compaction_dynamic_level_bytes=false;" + "inplace_update_support=false;" + "compaction_style=kCompactionStyleFIFO;" + "compaction_pri=kMinOverlappingRatio;" + "purge_redundant_kvs_while_flush=true;" + "hard_pending_compaction_bytes_limit=0;" + "disable_auto_compactions=false;" + "report_bg_io_stats=true;", + new_options)); + + ASSERT_EQ(unset_bytes_base, + NumUnsetBytes(new_options_ptr, sizeof(ColumnFamilyOptions), + kColumnFamilyOptionsBlacklist)); + + options->~ColumnFamilyOptions(); + new_options->~ColumnFamilyOptions(); + + delete[] options_ptr; + delete[] new_options_ptr; +} +#endif // !__clang__ +#endif // OS_LINUX || OS_WIN +#endif // !ROCKSDB_LITE + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); +#ifdef GFLAGS + ParseCommandLineFlags(&argc, &argv, true); +#endif // GFLAGS + return RUN_ALL_TESTS(); +} diff --git a/options/options_test.cc b/options/options_test.cc new file mode 100644 index 00000000000..fc4939beb41 --- /dev/null +++ b/options/options_test.cc @@ -0,0 +1,1655 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include +#include +#include +#include + +#include "options/options_helper.h" +#include "options/options_parser.h" +#include "options/options_sanity_check.h" +#include "rocksdb/cache.h" +#include "rocksdb/convenience.h" +#include "rocksdb/memtablerep.h" +#include "rocksdb/utilities/leveldb_options.h" +#include "util/random.h" +#include "util/stderr_logger.h" +#include "util/string_util.h" +#include "util/testharness.h" +#include "util/testutil.h" + +#ifndef GFLAGS +bool FLAGS_enable_print = false; +#else +#include +using GFLAGS::ParseCommandLineFlags; +DEFINE_bool(enable_print, false, "Print options generated to console."); +#endif // GFLAGS + +namespace rocksdb { + +class OptionsTest : public testing::Test {}; + +#ifndef ROCKSDB_LITE // GetOptionsFromMap is not supported in ROCKSDB_LITE +TEST_F(OptionsTest, GetOptionsFromMapTest) { + std::unordered_map cf_options_map = { + {"write_buffer_size", "1"}, + {"max_write_buffer_number", "2"}, + {"min_write_buffer_number_to_merge", "3"}, + {"max_write_buffer_number_to_maintain", "99"}, + {"compression", "kSnappyCompression"}, + {"compression_per_level", + "kNoCompression:" + "kSnappyCompression:" + "kZlibCompression:" + "kBZip2Compression:" + "kLZ4Compression:" + "kLZ4HCCompression:" + "kXpressCompression:" + "kZSTD:" + "kZSTDNotFinalCompression"}, + {"bottommost_compression", "kLZ4Compression"}, + {"compression_opts", "4:5:6:7"}, + {"num_levels", "8"}, + {"level0_file_num_compaction_trigger", "8"}, + {"level0_slowdown_writes_trigger", "9"}, + {"level0_stop_writes_trigger", "10"}, + {"target_file_size_base", "12"}, + {"target_file_size_multiplier", "13"}, + {"max_bytes_for_level_base", "14"}, + {"level_compaction_dynamic_level_bytes", "true"}, + {"max_bytes_for_level_multiplier", "15.0"}, + {"max_bytes_for_level_multiplier_additional", "16:17:18"}, + {"max_compaction_bytes", "21"}, + {"soft_rate_limit", "1.1"}, + {"hard_rate_limit", "2.1"}, + {"hard_pending_compaction_bytes_limit", "211"}, + {"arena_block_size", "22"}, + {"disable_auto_compactions", "true"}, + {"compaction_style", "kCompactionStyleLevel"}, + {"compaction_pri", "kOldestSmallestSeqFirst"}, + {"verify_checksums_in_compaction", "false"}, + {"compaction_options_fifo", "23"}, + {"max_sequential_skip_in_iterations", "24"}, + {"inplace_update_support", "true"}, + {"report_bg_io_stats", "true"}, + {"compaction_measure_io_stats", "false"}, + {"inplace_update_num_locks", "25"}, + {"memtable_prefix_bloom_size_ratio", "0.26"}, + {"memtable_huge_page_size", "28"}, + {"bloom_locality", "29"}, + {"max_successive_merges", "30"}, + {"min_partial_merge_operands", "31"}, + {"prefix_extractor", "fixed:31"}, + {"optimize_filters_for_hits", "true"}, + }; + + std::unordered_map db_options_map = { + {"create_if_missing", "false"}, + {"create_missing_column_families", "true"}, + {"error_if_exists", "false"}, + {"paranoid_checks", "true"}, + {"max_open_files", "32"}, + {"max_total_wal_size", "33"}, + {"use_fsync", "true"}, + {"db_log_dir", "/db_log_dir"}, + {"wal_dir", "/wal_dir"}, + {"delete_obsolete_files_period_micros", "34"}, + {"max_background_compactions", "35"}, + {"max_background_flushes", "36"}, + {"max_log_file_size", "37"}, + {"log_file_time_to_roll", "38"}, + {"keep_log_file_num", "39"}, + {"recycle_log_file_num", "5"}, + {"max_manifest_file_size", "40"}, + {"table_cache_numshardbits", "41"}, + {"WAL_ttl_seconds", "43"}, + {"WAL_size_limit_MB", "44"}, + {"manifest_preallocation_size", "45"}, + {"allow_mmap_reads", "true"}, + {"allow_mmap_writes", "false"}, + {"use_direct_reads", "false"}, + {"use_direct_io_for_flush_and_compaction", "false"}, + {"is_fd_close_on_exec", "true"}, + {"skip_log_error_on_recovery", "false"}, + {"stats_dump_period_sec", "46"}, + {"advise_random_on_open", "true"}, + {"use_adaptive_mutex", "false"}, + {"new_table_reader_for_compaction_inputs", "true"}, + {"compaction_readahead_size", "100"}, + {"random_access_max_buffer_size", "3145728"}, + {"writable_file_max_buffer_size", "314159"}, + {"bytes_per_sync", "47"}, + {"wal_bytes_per_sync", "48"}, + }; + + ColumnFamilyOptions base_cf_opt; + ColumnFamilyOptions new_cf_opt; + ASSERT_OK(GetColumnFamilyOptionsFromMap( + base_cf_opt, cf_options_map, &new_cf_opt)); + ASSERT_EQ(new_cf_opt.write_buffer_size, 1U); + ASSERT_EQ(new_cf_opt.max_write_buffer_number, 2); + ASSERT_EQ(new_cf_opt.min_write_buffer_number_to_merge, 3); + ASSERT_EQ(new_cf_opt.max_write_buffer_number_to_maintain, 99); + ASSERT_EQ(new_cf_opt.compression, kSnappyCompression); + ASSERT_EQ(new_cf_opt.compression_per_level.size(), 9U); + ASSERT_EQ(new_cf_opt.compression_per_level[0], kNoCompression); + ASSERT_EQ(new_cf_opt.compression_per_level[1], kSnappyCompression); + ASSERT_EQ(new_cf_opt.compression_per_level[2], kZlibCompression); + ASSERT_EQ(new_cf_opt.compression_per_level[3], kBZip2Compression); + ASSERT_EQ(new_cf_opt.compression_per_level[4], kLZ4Compression); + ASSERT_EQ(new_cf_opt.compression_per_level[5], kLZ4HCCompression); + ASSERT_EQ(new_cf_opt.compression_per_level[6], kXpressCompression); + ASSERT_EQ(new_cf_opt.compression_per_level[7], kZSTD); + ASSERT_EQ(new_cf_opt.compression_per_level[8], kZSTDNotFinalCompression); + ASSERT_EQ(new_cf_opt.compression_opts.window_bits, 4); + ASSERT_EQ(new_cf_opt.compression_opts.level, 5); + ASSERT_EQ(new_cf_opt.compression_opts.strategy, 6); + ASSERT_EQ(new_cf_opt.compression_opts.max_dict_bytes, 7); + ASSERT_EQ(new_cf_opt.bottommost_compression, kLZ4Compression); + ASSERT_EQ(new_cf_opt.num_levels, 8); + ASSERT_EQ(new_cf_opt.level0_file_num_compaction_trigger, 8); + ASSERT_EQ(new_cf_opt.level0_slowdown_writes_trigger, 9); + ASSERT_EQ(new_cf_opt.level0_stop_writes_trigger, 10); + ASSERT_EQ(new_cf_opt.target_file_size_base, static_cast(12)); + ASSERT_EQ(new_cf_opt.target_file_size_multiplier, 13); + ASSERT_EQ(new_cf_opt.max_bytes_for_level_base, 14U); + ASSERT_EQ(new_cf_opt.level_compaction_dynamic_level_bytes, true); + ASSERT_EQ(new_cf_opt.max_bytes_for_level_multiplier, 15.0); + ASSERT_EQ(new_cf_opt.max_bytes_for_level_multiplier_additional.size(), 3U); + ASSERT_EQ(new_cf_opt.max_bytes_for_level_multiplier_additional[0], 16); + ASSERT_EQ(new_cf_opt.max_bytes_for_level_multiplier_additional[1], 17); + ASSERT_EQ(new_cf_opt.max_bytes_for_level_multiplier_additional[2], 18); + ASSERT_EQ(new_cf_opt.max_compaction_bytes, 21); + ASSERT_EQ(new_cf_opt.hard_pending_compaction_bytes_limit, 211); + ASSERT_EQ(new_cf_opt.arena_block_size, 22U); + ASSERT_EQ(new_cf_opt.disable_auto_compactions, true); + ASSERT_EQ(new_cf_opt.compaction_style, kCompactionStyleLevel); + ASSERT_EQ(new_cf_opt.compaction_pri, kOldestSmallestSeqFirst); + ASSERT_EQ(new_cf_opt.compaction_options_fifo.max_table_files_size, + static_cast(23)); + ASSERT_EQ(new_cf_opt.max_sequential_skip_in_iterations, + static_cast(24)); + ASSERT_EQ(new_cf_opt.inplace_update_support, true); + ASSERT_EQ(new_cf_opt.inplace_update_num_locks, 25U); + ASSERT_EQ(new_cf_opt.memtable_prefix_bloom_size_ratio, 0.26); + ASSERT_EQ(new_cf_opt.memtable_huge_page_size, 28U); + ASSERT_EQ(new_cf_opt.bloom_locality, 29U); + ASSERT_EQ(new_cf_opt.max_successive_merges, 30U); + ASSERT_TRUE(new_cf_opt.prefix_extractor != nullptr); + ASSERT_EQ(new_cf_opt.optimize_filters_for_hits, true); + ASSERT_EQ(std::string(new_cf_opt.prefix_extractor->Name()), + "rocksdb.FixedPrefix.31"); + + cf_options_map["write_buffer_size"] = "hello"; + ASSERT_NOK(GetColumnFamilyOptionsFromMap( + base_cf_opt, cf_options_map, &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + cf_options_map["write_buffer_size"] = "1"; + ASSERT_OK(GetColumnFamilyOptionsFromMap( + base_cf_opt, cf_options_map, &new_cf_opt)); + + cf_options_map["unknown_option"] = "1"; + ASSERT_NOK(GetColumnFamilyOptionsFromMap( + base_cf_opt, cf_options_map, &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + ASSERT_OK(GetColumnFamilyOptionsFromMap(base_cf_opt, cf_options_map, + &new_cf_opt, + false, /* input_strings_escaped */ + true /* ignore_unknown_options */)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions( + base_cf_opt, new_cf_opt, nullptr, /* new_opt_map */ + kSanityLevelLooselyCompatible /* from CheckOptionsCompatibility*/)); + ASSERT_NOK(RocksDBOptionsParser::VerifyCFOptions( + base_cf_opt, new_cf_opt, nullptr, /* new_opt_map */ + kSanityLevelExactMatch /* default for VerifyCFOptions */)); + + DBOptions base_db_opt; + DBOptions new_db_opt; + ASSERT_OK(GetDBOptionsFromMap(base_db_opt, db_options_map, &new_db_opt)); + ASSERT_EQ(new_db_opt.create_if_missing, false); + ASSERT_EQ(new_db_opt.create_missing_column_families, true); + ASSERT_EQ(new_db_opt.error_if_exists, false); + ASSERT_EQ(new_db_opt.paranoid_checks, true); + ASSERT_EQ(new_db_opt.max_open_files, 32); + ASSERT_EQ(new_db_opt.max_total_wal_size, static_cast(33)); + ASSERT_EQ(new_db_opt.use_fsync, true); + ASSERT_EQ(new_db_opt.db_log_dir, "/db_log_dir"); + ASSERT_EQ(new_db_opt.wal_dir, "/wal_dir"); + ASSERT_EQ(new_db_opt.delete_obsolete_files_period_micros, + static_cast(34)); + ASSERT_EQ(new_db_opt.max_background_compactions, 35); + ASSERT_EQ(new_db_opt.max_background_flushes, 36); + ASSERT_EQ(new_db_opt.max_log_file_size, 37U); + ASSERT_EQ(new_db_opt.log_file_time_to_roll, 38U); + ASSERT_EQ(new_db_opt.keep_log_file_num, 39U); + ASSERT_EQ(new_db_opt.recycle_log_file_num, 5U); + ASSERT_EQ(new_db_opt.max_manifest_file_size, static_cast(40)); + ASSERT_EQ(new_db_opt.table_cache_numshardbits, 41); + ASSERT_EQ(new_db_opt.WAL_ttl_seconds, static_cast(43)); + ASSERT_EQ(new_db_opt.WAL_size_limit_MB, static_cast(44)); + ASSERT_EQ(new_db_opt.manifest_preallocation_size, 45U); + ASSERT_EQ(new_db_opt.allow_mmap_reads, true); + ASSERT_EQ(new_db_opt.allow_mmap_writes, false); + ASSERT_EQ(new_db_opt.use_direct_reads, false); + ASSERT_EQ(new_db_opt.use_direct_io_for_flush_and_compaction, false); + ASSERT_EQ(new_db_opt.is_fd_close_on_exec, true); + ASSERT_EQ(new_db_opt.skip_log_error_on_recovery, false); + ASSERT_EQ(new_db_opt.stats_dump_period_sec, 46U); + ASSERT_EQ(new_db_opt.advise_random_on_open, true); + ASSERT_EQ(new_db_opt.use_adaptive_mutex, false); + ASSERT_EQ(new_db_opt.new_table_reader_for_compaction_inputs, true); + ASSERT_EQ(new_db_opt.compaction_readahead_size, 100); + ASSERT_EQ(new_db_opt.random_access_max_buffer_size, 3145728); + ASSERT_EQ(new_db_opt.writable_file_max_buffer_size, 314159); + ASSERT_EQ(new_db_opt.bytes_per_sync, static_cast(47)); + ASSERT_EQ(new_db_opt.wal_bytes_per_sync, static_cast(48)); + + db_options_map["max_open_files"] = "hello"; + ASSERT_NOK(GetDBOptionsFromMap(base_db_opt, db_options_map, &new_db_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyDBOptions(base_db_opt, new_db_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyDBOptions( + base_db_opt, new_db_opt, nullptr, /* new_opt_map */ + kSanityLevelLooselyCompatible /* from CheckOptionsCompatibility */)); + + // unknow options should fail parsing without ignore_unknown_options = true + db_options_map["unknown_db_option"] = "1"; + ASSERT_NOK(GetDBOptionsFromMap(base_db_opt, db_options_map, &new_db_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyDBOptions(base_db_opt, new_db_opt)); + + ASSERT_OK(GetDBOptionsFromMap(base_db_opt, db_options_map, &new_db_opt, + false, /* input_strings_escaped */ + true /* ignore_unknown_options */)); + ASSERT_OK(RocksDBOptionsParser::VerifyDBOptions( + base_db_opt, new_db_opt, nullptr, /* new_opt_map */ + kSanityLevelLooselyCompatible /* from CheckOptionsCompatibility */)); + ASSERT_NOK(RocksDBOptionsParser::VerifyDBOptions( + base_db_opt, new_db_opt, nullptr, /* new_opt_mat */ + kSanityLevelExactMatch /* default for VerifyDBOptions */)); +} +#endif // !ROCKSDB_LITE + +#ifndef ROCKSDB_LITE // GetColumnFamilyOptionsFromString is not supported in + // ROCKSDB_LITE +TEST_F(OptionsTest, GetColumnFamilyOptionsFromStringTest) { + ColumnFamilyOptions base_cf_opt; + ColumnFamilyOptions new_cf_opt; + base_cf_opt.table_factory.reset(); + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, "", &new_cf_opt)); + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=5", &new_cf_opt)); + ASSERT_EQ(new_cf_opt.write_buffer_size, 5U); + ASSERT_TRUE(new_cf_opt.table_factory == nullptr); + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=6;", &new_cf_opt)); + ASSERT_EQ(new_cf_opt.write_buffer_size, 6U); + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + " write_buffer_size = 7 ", &new_cf_opt)); + ASSERT_EQ(new_cf_opt.write_buffer_size, 7U); + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + " write_buffer_size = 8 ; ", &new_cf_opt)); + ASSERT_EQ(new_cf_opt.write_buffer_size, 8U); + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=9;max_write_buffer_number=10", &new_cf_opt)); + ASSERT_EQ(new_cf_opt.write_buffer_size, 9U); + ASSERT_EQ(new_cf_opt.max_write_buffer_number, 10); + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=11; max_write_buffer_number = 12 ;", + &new_cf_opt)); + ASSERT_EQ(new_cf_opt.write_buffer_size, 11U); + ASSERT_EQ(new_cf_opt.max_write_buffer_number, 12); + // Wrong name "max_write_buffer_number_" + ASSERT_NOK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=13;max_write_buffer_number_=14;", + &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + // Wrong key/value pair + ASSERT_NOK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=13;max_write_buffer_number;", &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + // Error Paring value + ASSERT_NOK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=13;max_write_buffer_number=;", &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + // Missing option name + ASSERT_NOK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=13; =100;", &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + const int64_t kilo = 1024UL; + const int64_t mega = 1024 * kilo; + const int64_t giga = 1024 * mega; + const int64_t tera = 1024 * giga; + + // Units (k) + ASSERT_OK(GetColumnFamilyOptionsFromString( + base_cf_opt, "max_write_buffer_number=15K", &new_cf_opt)); + ASSERT_EQ(new_cf_opt.max_write_buffer_number, 15 * kilo); + // Units (m) + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "max_write_buffer_number=16m;inplace_update_num_locks=17M", + &new_cf_opt)); + ASSERT_EQ(new_cf_opt.max_write_buffer_number, 16 * mega); + ASSERT_EQ(new_cf_opt.inplace_update_num_locks, 17 * mega); + // Units (g) + ASSERT_OK(GetColumnFamilyOptionsFromString( + base_cf_opt, + "write_buffer_size=18g;prefix_extractor=capped:8;" + "arena_block_size=19G", + &new_cf_opt)); + + ASSERT_EQ(new_cf_opt.write_buffer_size, 18 * giga); + ASSERT_EQ(new_cf_opt.arena_block_size, 19 * giga); + ASSERT_TRUE(new_cf_opt.prefix_extractor.get() != nullptr); + std::string prefix_name(new_cf_opt.prefix_extractor->Name()); + ASSERT_EQ(prefix_name, "rocksdb.CappedPrefix.8"); + + // Units (t) + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=20t;arena_block_size=21T", &new_cf_opt)); + ASSERT_EQ(new_cf_opt.write_buffer_size, 20 * tera); + ASSERT_EQ(new_cf_opt.arena_block_size, 21 * tera); + + // Nested block based table options + // Empty + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "block_based_table_factory={};arena_block_size=1024", + &new_cf_opt)); + ASSERT_TRUE(new_cf_opt.table_factory != nullptr); + // Non-empty + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "block_based_table_factory={block_cache=1M;block_size=4;};" + "arena_block_size=1024", + &new_cf_opt)); + ASSERT_TRUE(new_cf_opt.table_factory != nullptr); + // Last one + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "block_based_table_factory={block_cache=1M;block_size=4;}", + &new_cf_opt)); + ASSERT_TRUE(new_cf_opt.table_factory != nullptr); + // Mismatch curly braces + ASSERT_NOK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "block_based_table_factory={{{block_size=4;};" + "arena_block_size=1024", + &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + // Unexpected chars after closing curly brace + ASSERT_NOK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "block_based_table_factory={block_size=4;}};" + "arena_block_size=1024", + &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + ASSERT_NOK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "block_based_table_factory={block_size=4;}xdfa;" + "arena_block_size=1024", + &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + ASSERT_NOK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "block_based_table_factory={block_size=4;}xdfa", + &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + // Invalid block based table option + ASSERT_NOK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "block_based_table_factory={xx_block_size=4;}", + &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "optimize_filters_for_hits=true", + &new_cf_opt)); + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "optimize_filters_for_hits=false", + &new_cf_opt)); + + ASSERT_NOK(GetColumnFamilyOptionsFromString(base_cf_opt, + "optimize_filters_for_hits=junk", + &new_cf_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_cf_opt, new_cf_opt)); + + // Nested plain table options + // Empty + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "plain_table_factory={};arena_block_size=1024", + &new_cf_opt)); + ASSERT_TRUE(new_cf_opt.table_factory != nullptr); + ASSERT_EQ(std::string(new_cf_opt.table_factory->Name()), "PlainTable"); + // Non-empty + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "plain_table_factory={user_key_len=66;bloom_bits_per_key=20;};" + "arena_block_size=1024", + &new_cf_opt)); + ASSERT_TRUE(new_cf_opt.table_factory != nullptr); + ASSERT_EQ(std::string(new_cf_opt.table_factory->Name()), "PlainTable"); + + // memtable factory + ASSERT_OK(GetColumnFamilyOptionsFromString(base_cf_opt, + "write_buffer_size=10;max_write_buffer_number=16;" + "memtable=skip_list:10;arena_block_size=1024", + &new_cf_opt)); + ASSERT_TRUE(new_cf_opt.memtable_factory != nullptr); + ASSERT_EQ(std::string(new_cf_opt.memtable_factory->Name()), "SkipListFactory"); +} +#endif // !ROCKSDB_LITE + +#ifndef ROCKSDB_LITE // GetBlockBasedTableOptionsFromString is not supported +TEST_F(OptionsTest, GetBlockBasedTableOptionsFromString) { + BlockBasedTableOptions table_opt; + BlockBasedTableOptions new_opt; + // make sure default values are overwritten by something else + ASSERT_OK(GetBlockBasedTableOptionsFromString(table_opt, + "cache_index_and_filter_blocks=1;index_type=kHashSearch;" + "checksum=kxxHash;hash_index_allow_collision=1;no_block_cache=1;" + "block_cache=1M;block_cache_compressed=1k;block_size=1024;" + "block_size_deviation=8;block_restart_interval=4;" + "filter_policy=bloomfilter:4:true;whole_key_filtering=1;", + &new_opt)); + ASSERT_TRUE(new_opt.cache_index_and_filter_blocks); + ASSERT_EQ(new_opt.index_type, BlockBasedTableOptions::kHashSearch); + ASSERT_EQ(new_opt.checksum, ChecksumType::kxxHash); + ASSERT_TRUE(new_opt.hash_index_allow_collision); + ASSERT_TRUE(new_opt.no_block_cache); + ASSERT_TRUE(new_opt.block_cache != nullptr); + ASSERT_EQ(new_opt.block_cache->GetCapacity(), 1024UL*1024UL); + ASSERT_TRUE(new_opt.block_cache_compressed != nullptr); + ASSERT_EQ(new_opt.block_cache_compressed->GetCapacity(), 1024UL); + ASSERT_EQ(new_opt.block_size, 1024UL); + ASSERT_EQ(new_opt.block_size_deviation, 8); + ASSERT_EQ(new_opt.block_restart_interval, 4); + ASSERT_TRUE(new_opt.filter_policy != nullptr); + + // unknown option + ASSERT_NOK(GetBlockBasedTableOptionsFromString(table_opt, + "cache_index_and_filter_blocks=1;index_type=kBinarySearch;" + "bad_option=1", + &new_opt)); + ASSERT_EQ(table_opt.cache_index_and_filter_blocks, + new_opt.cache_index_and_filter_blocks); + ASSERT_EQ(table_opt.index_type, new_opt.index_type); + + // unrecognized index type + ASSERT_NOK(GetBlockBasedTableOptionsFromString(table_opt, + "cache_index_and_filter_blocks=1;index_type=kBinarySearchXX", + &new_opt)); + ASSERT_EQ(table_opt.cache_index_and_filter_blocks, + new_opt.cache_index_and_filter_blocks); + ASSERT_EQ(table_opt.index_type, new_opt.index_type); + + // unrecognized checksum type + ASSERT_NOK(GetBlockBasedTableOptionsFromString(table_opt, + "cache_index_and_filter_blocks=1;checksum=kxxHashXX", + &new_opt)); + ASSERT_EQ(table_opt.cache_index_and_filter_blocks, + new_opt.cache_index_and_filter_blocks); + ASSERT_EQ(table_opt.index_type, new_opt.index_type); + + // unrecognized filter policy name + ASSERT_NOK(GetBlockBasedTableOptionsFromString(table_opt, + "cache_index_and_filter_blocks=1;" + "filter_policy=bloomfilterxx:4:true", + &new_opt)); + ASSERT_EQ(table_opt.cache_index_and_filter_blocks, + new_opt.cache_index_and_filter_blocks); + ASSERT_EQ(table_opt.filter_policy, new_opt.filter_policy); + + // unrecognized filter policy config + ASSERT_NOK(GetBlockBasedTableOptionsFromString(table_opt, + "cache_index_and_filter_blocks=1;" + "filter_policy=bloomfilter:4", + &new_opt)); + ASSERT_EQ(table_opt.cache_index_and_filter_blocks, + new_opt.cache_index_and_filter_blocks); + ASSERT_EQ(table_opt.filter_policy, new_opt.filter_policy); +} +#endif // !ROCKSDB_LITE + + +#ifndef ROCKSDB_LITE // GetPlainTableOptionsFromString is not supported +TEST_F(OptionsTest, GetPlainTableOptionsFromString) { + PlainTableOptions table_opt; + PlainTableOptions new_opt; + // make sure default values are overwritten by something else + ASSERT_OK(GetPlainTableOptionsFromString(table_opt, + "user_key_len=66;bloom_bits_per_key=20;hash_table_ratio=0.5;" + "index_sparseness=8;huge_page_tlb_size=4;encoding_type=kPrefix;" + "full_scan_mode=true;store_index_in_file=true", + &new_opt)); + ASSERT_EQ(new_opt.user_key_len, 66); + ASSERT_EQ(new_opt.bloom_bits_per_key, 20); + ASSERT_EQ(new_opt.hash_table_ratio, 0.5); + ASSERT_EQ(new_opt.index_sparseness, 8); + ASSERT_EQ(new_opt.huge_page_tlb_size, 4); + ASSERT_EQ(new_opt.encoding_type, EncodingType::kPrefix); + ASSERT_TRUE(new_opt.full_scan_mode); + ASSERT_TRUE(new_opt.store_index_in_file); + + // unknown option + ASSERT_NOK(GetPlainTableOptionsFromString(table_opt, + "user_key_len=66;bloom_bits_per_key=20;hash_table_ratio=0.5;" + "bad_option=1", + &new_opt)); + + // unrecognized EncodingType + ASSERT_NOK(GetPlainTableOptionsFromString(table_opt, + "user_key_len=66;bloom_bits_per_key=20;hash_table_ratio=0.5;" + "encoding_type=kPrefixXX", + &new_opt)); +} +#endif // !ROCKSDB_LITE + +#ifndef ROCKSDB_LITE // GetMemTableRepFactoryFromString is not supported +TEST_F(OptionsTest, GetMemTableRepFactoryFromString) { + std::unique_ptr new_mem_factory = nullptr; + + ASSERT_OK(GetMemTableRepFactoryFromString("skip_list", &new_mem_factory)); + ASSERT_OK(GetMemTableRepFactoryFromString("skip_list:16", &new_mem_factory)); + ASSERT_EQ(std::string(new_mem_factory->Name()), "SkipListFactory"); + ASSERT_NOK(GetMemTableRepFactoryFromString("skip_list:16:invalid_opt", + &new_mem_factory)); + + ASSERT_OK(GetMemTableRepFactoryFromString("prefix_hash", &new_mem_factory)); + ASSERT_OK(GetMemTableRepFactoryFromString("prefix_hash:1000", + &new_mem_factory)); + ASSERT_EQ(std::string(new_mem_factory->Name()), "HashSkipListRepFactory"); + ASSERT_NOK(GetMemTableRepFactoryFromString("prefix_hash:1000:invalid_opt", + &new_mem_factory)); + + ASSERT_OK(GetMemTableRepFactoryFromString("hash_linkedlist", + &new_mem_factory)); + ASSERT_OK(GetMemTableRepFactoryFromString("hash_linkedlist:1000", + &new_mem_factory)); + ASSERT_EQ(std::string(new_mem_factory->Name()), "HashLinkListRepFactory"); + ASSERT_NOK(GetMemTableRepFactoryFromString("hash_linkedlist:1000:invalid_opt", + &new_mem_factory)); + + ASSERT_OK(GetMemTableRepFactoryFromString("vector", &new_mem_factory)); + ASSERT_OK(GetMemTableRepFactoryFromString("vector:1024", &new_mem_factory)); + ASSERT_EQ(std::string(new_mem_factory->Name()), "VectorRepFactory"); + ASSERT_NOK(GetMemTableRepFactoryFromString("vector:1024:invalid_opt", + &new_mem_factory)); + + ASSERT_NOK(GetMemTableRepFactoryFromString("cuckoo", &new_mem_factory)); + ASSERT_OK(GetMemTableRepFactoryFromString("cuckoo:1024", &new_mem_factory)); + ASSERT_EQ(std::string(new_mem_factory->Name()), "HashCuckooRepFactory"); + + ASSERT_NOK(GetMemTableRepFactoryFromString("bad_factory", &new_mem_factory)); +} +#endif // !ROCKSDB_LITE + +#ifndef ROCKSDB_LITE // GetOptionsFromString is not supported in RocksDB Lite +TEST_F(OptionsTest, GetOptionsFromStringTest) { + Options base_options, new_options; + base_options.write_buffer_size = 20; + base_options.min_write_buffer_number_to_merge = 15; + BlockBasedTableOptions block_based_table_options; + block_based_table_options.cache_index_and_filter_blocks = true; + base_options.table_factory.reset( + NewBlockBasedTableFactory(block_based_table_options)); + ASSERT_OK(GetOptionsFromString( + base_options, + "write_buffer_size=10;max_write_buffer_number=16;" + "block_based_table_factory={block_cache=1M;block_size=4;};" + "compression_opts=4:5:6;create_if_missing=true;max_open_files=1;" + "rate_limiter_bytes_per_sec=1024", + &new_options)); + + ASSERT_EQ(new_options.compression_opts.window_bits, 4); + ASSERT_EQ(new_options.compression_opts.level, 5); + ASSERT_EQ(new_options.compression_opts.strategy, 6); + ASSERT_EQ(new_options.compression_opts.max_dict_bytes, 0); + ASSERT_EQ(new_options.bottommost_compression, kDisableCompressionOption); + ASSERT_EQ(new_options.write_buffer_size, 10U); + ASSERT_EQ(new_options.max_write_buffer_number, 16); + BlockBasedTableOptions new_block_based_table_options = + dynamic_cast(new_options.table_factory.get()) + ->table_options(); + ASSERT_EQ(new_block_based_table_options.block_cache->GetCapacity(), 1U << 20); + ASSERT_EQ(new_block_based_table_options.block_size, 4U); + // don't overwrite block based table options + ASSERT_TRUE(new_block_based_table_options.cache_index_and_filter_blocks); + + ASSERT_EQ(new_options.create_if_missing, true); + ASSERT_EQ(new_options.max_open_files, 1); + ASSERT_TRUE(new_options.rate_limiter.get() != nullptr); +} + +TEST_F(OptionsTest, DBOptionsSerialization) { + Options base_options, new_options; + Random rnd(301); + + // Phase 1: Make big change in base_options + test::RandomInitDBOptions(&base_options, &rnd); + + // Phase 2: obtain a string from base_option + std::string base_options_file_content; + ASSERT_OK(GetStringFromDBOptions(&base_options_file_content, base_options)); + + // Phase 3: Set new_options from the derived string and expect + // new_options == base_options + ASSERT_OK(GetDBOptionsFromString(DBOptions(), base_options_file_content, + &new_options)); + ASSERT_OK(RocksDBOptionsParser::VerifyDBOptions(base_options, new_options)); +} + +TEST_F(OptionsTest, ColumnFamilyOptionsSerialization) { + ColumnFamilyOptions base_opt, new_opt; + Random rnd(302); + // Phase 1: randomly assign base_opt + // custom type options + test::RandomInitCFOptions(&base_opt, &rnd); + + // Phase 2: obtain a string from base_opt + std::string base_options_file_content; + ASSERT_OK( + GetStringFromColumnFamilyOptions(&base_options_file_content, base_opt)); + + // Phase 3: Set new_opt from the derived string and expect + // new_opt == base_opt + ASSERT_OK(GetColumnFamilyOptionsFromString( + ColumnFamilyOptions(), base_options_file_content, &new_opt)); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(base_opt, new_opt)); + if (base_opt.compaction_filter) { + delete base_opt.compaction_filter; + } +} + +#endif // !ROCKSDB_LITE + +Status StringToMap( + const std::string& opts_str, + std::unordered_map* opts_map); + +#ifndef ROCKSDB_LITE // StringToMap is not supported in ROCKSDB_LITE +TEST_F(OptionsTest, StringToMapTest) { + std::unordered_map opts_map; + // Regular options + ASSERT_OK(StringToMap("k1=v1;k2=v2;k3=v3", &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_EQ(opts_map["k2"], "v2"); + ASSERT_EQ(opts_map["k3"], "v3"); + // Value with '=' + opts_map.clear(); + ASSERT_OK(StringToMap("k1==v1;k2=v2=;", &opts_map)); + ASSERT_EQ(opts_map["k1"], "=v1"); + ASSERT_EQ(opts_map["k2"], "v2="); + // Overwrriten option + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k1=v2;k3=v3", &opts_map)); + ASSERT_EQ(opts_map["k1"], "v2"); + ASSERT_EQ(opts_map["k3"], "v3"); + // Empty value + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k2=;k3=v3;k4=", &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_TRUE(opts_map.find("k2") != opts_map.end()); + ASSERT_EQ(opts_map["k2"], ""); + ASSERT_EQ(opts_map["k3"], "v3"); + ASSERT_TRUE(opts_map.find("k4") != opts_map.end()); + ASSERT_EQ(opts_map["k4"], ""); + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k2=;k3=v3;k4= ", &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_TRUE(opts_map.find("k2") != opts_map.end()); + ASSERT_EQ(opts_map["k2"], ""); + ASSERT_EQ(opts_map["k3"], "v3"); + ASSERT_TRUE(opts_map.find("k4") != opts_map.end()); + ASSERT_EQ(opts_map["k4"], ""); + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k2=;k3=", &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_TRUE(opts_map.find("k2") != opts_map.end()); + ASSERT_EQ(opts_map["k2"], ""); + ASSERT_TRUE(opts_map.find("k3") != opts_map.end()); + ASSERT_EQ(opts_map["k3"], ""); + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k2=;k3=;", &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_TRUE(opts_map.find("k2") != opts_map.end()); + ASSERT_EQ(opts_map["k2"], ""); + ASSERT_TRUE(opts_map.find("k3") != opts_map.end()); + ASSERT_EQ(opts_map["k3"], ""); + // Regular nested options + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k2={nk1=nv1;nk2=nv2};k3=v3", &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_EQ(opts_map["k2"], "nk1=nv1;nk2=nv2"); + ASSERT_EQ(opts_map["k3"], "v3"); + // Multi-level nested options + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k2={nk1=nv1;nk2={nnk1=nnk2}};" + "k3={nk1={nnk1={nnnk1=nnnv1;nnnk2;nnnv2}}};k4=v4", + &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_EQ(opts_map["k2"], "nk1=nv1;nk2={nnk1=nnk2}"); + ASSERT_EQ(opts_map["k3"], "nk1={nnk1={nnnk1=nnnv1;nnnk2;nnnv2}}"); + ASSERT_EQ(opts_map["k4"], "v4"); + // Garbage inside curly braces + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k2={dfad=};k3={=};k4=v4", + &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_EQ(opts_map["k2"], "dfad="); + ASSERT_EQ(opts_map["k3"], "="); + ASSERT_EQ(opts_map["k4"], "v4"); + // Empty nested options + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k2={};", &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_EQ(opts_map["k2"], ""); + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k2={{{{}}}{}{}};", &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_EQ(opts_map["k2"], "{{{}}}{}{}"); + // With random spaces + opts_map.clear(); + ASSERT_OK(StringToMap(" k1 = v1 ; k2= {nk1=nv1; nk2={nnk1=nnk2}} ; " + "k3={ { } }; k4= v4 ", + &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_EQ(opts_map["k2"], "nk1=nv1; nk2={nnk1=nnk2}"); + ASSERT_EQ(opts_map["k3"], "{ }"); + ASSERT_EQ(opts_map["k4"], "v4"); + + // Empty key + ASSERT_NOK(StringToMap("k1=v1;k2=v2;=", &opts_map)); + ASSERT_NOK(StringToMap("=v1;k2=v2", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2v2;", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2=v2;fadfa", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2=v2;;", &opts_map)); + // Mismatch curly braces + ASSERT_NOK(StringToMap("k1=v1;k2={;k3=v3", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2={{};k3=v3", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2={}};k3=v3", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2={{}{}}};k3=v3", &opts_map)); + // However this is valid! + opts_map.clear(); + ASSERT_OK(StringToMap("k1=v1;k2=};k3=v3", &opts_map)); + ASSERT_EQ(opts_map["k1"], "v1"); + ASSERT_EQ(opts_map["k2"], "}"); + ASSERT_EQ(opts_map["k3"], "v3"); + + // Invalid chars after closing curly brace + ASSERT_NOK(StringToMap("k1=v1;k2={{}}{};k3=v3", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2={{}}cfda;k3=v3", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2={{}} cfda;k3=v3", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2={{}} cfda", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2={{}}{}", &opts_map)); + ASSERT_NOK(StringToMap("k1=v1;k2={{dfdl}adfa}{}", &opts_map)); +} +#endif // ROCKSDB_LITE + +#ifndef ROCKSDB_LITE // StringToMap is not supported in ROCKSDB_LITE +TEST_F(OptionsTest, StringToMapRandomTest) { + std::unordered_map opts_map; + // Make sure segfault is not hit by semi-random strings + + std::vector bases = { + "a={aa={};tt={xxx={}}};c=defff", + "a={aa={};tt={xxx={}}};c=defff;d={{}yxx{}3{xx}}", + "abc={{}{}{}{{{}}}{{}{}{}{}{}{}{}"}; + + for (std::string base : bases) { + for (int rand_seed = 301; rand_seed < 401; rand_seed++) { + Random rnd(rand_seed); + for (int attempt = 0; attempt < 10; attempt++) { + std::string str = base; + // Replace random position to space + size_t pos = static_cast( + rnd.Uniform(static_cast(base.size()))); + str[pos] = ' '; + Status s = StringToMap(str, &opts_map); + ASSERT_TRUE(s.ok() || s.IsInvalidArgument()); + opts_map.clear(); + } + } + } + + // Random Construct a string + std::vector chars = {'{', '}', ' ', '=', ';', 'c'}; + for (int rand_seed = 301; rand_seed < 1301; rand_seed++) { + Random rnd(rand_seed); + int len = rnd.Uniform(30); + std::string str = ""; + for (int attempt = 0; attempt < len; attempt++) { + // Add a random character + size_t pos = static_cast( + rnd.Uniform(static_cast(chars.size()))); + str.append(1, chars[pos]); + } + Status s = StringToMap(str, &opts_map); + ASSERT_TRUE(s.ok() || s.IsInvalidArgument()); + s = StringToMap("name=" + str, &opts_map); + ASSERT_TRUE(s.ok() || s.IsInvalidArgument()); + opts_map.clear(); + } +} + +TEST_F(OptionsTest, GetStringFromCompressionType) { + std::string res; + + ASSERT_OK(GetStringFromCompressionType(&res, kNoCompression)); + ASSERT_EQ(res, "kNoCompression"); + + ASSERT_OK(GetStringFromCompressionType(&res, kSnappyCompression)); + ASSERT_EQ(res, "kSnappyCompression"); + + ASSERT_OK(GetStringFromCompressionType(&res, kDisableCompressionOption)); + ASSERT_EQ(res, "kDisableCompressionOption"); + + ASSERT_OK(GetStringFromCompressionType(&res, kLZ4Compression)); + ASSERT_EQ(res, "kLZ4Compression"); + + ASSERT_OK(GetStringFromCompressionType(&res, kZlibCompression)); + ASSERT_EQ(res, "kZlibCompression"); + + ASSERT_NOK( + GetStringFromCompressionType(&res, static_cast(-10))); +} +#endif // !ROCKSDB_LITE + +TEST_F(OptionsTest, ConvertOptionsTest) { + LevelDBOptions leveldb_opt; + Options converted_opt = ConvertOptions(leveldb_opt); + + ASSERT_EQ(converted_opt.create_if_missing, leveldb_opt.create_if_missing); + ASSERT_EQ(converted_opt.error_if_exists, leveldb_opt.error_if_exists); + ASSERT_EQ(converted_opt.paranoid_checks, leveldb_opt.paranoid_checks); + ASSERT_EQ(converted_opt.env, leveldb_opt.env); + ASSERT_EQ(converted_opt.info_log.get(), leveldb_opt.info_log); + ASSERT_EQ(converted_opt.write_buffer_size, leveldb_opt.write_buffer_size); + ASSERT_EQ(converted_opt.max_open_files, leveldb_opt.max_open_files); + ASSERT_EQ(converted_opt.compression, leveldb_opt.compression); + + std::shared_ptr tb_guard = converted_opt.table_factory; + BlockBasedTableFactory* table_factory = + dynamic_cast(converted_opt.table_factory.get()); + + ASSERT_TRUE(table_factory != nullptr); + + const BlockBasedTableOptions table_opt = table_factory->table_options(); + + ASSERT_EQ(table_opt.block_cache->GetCapacity(), 8UL << 20); + ASSERT_EQ(table_opt.block_size, leveldb_opt.block_size); + ASSERT_EQ(table_opt.block_restart_interval, + leveldb_opt.block_restart_interval); + ASSERT_EQ(table_opt.filter_policy.get(), leveldb_opt.filter_policy); +} + +#ifndef ROCKSDB_LITE +class OptionsParserTest : public testing::Test { + public: + OptionsParserTest() { env_.reset(new test::StringEnv(Env::Default())); } + + protected: + std::unique_ptr env_; +}; + +TEST_F(OptionsParserTest, Comment) { + DBOptions db_opt; + db_opt.max_open_files = 12345; + db_opt.max_background_flushes = 301; + db_opt.max_total_wal_size = 1024; + ColumnFamilyOptions cf_opt; + + std::string options_file_content = + "# This is a testing option string.\n" + "# Currently we only support \"#\" styled comment.\n" + "\n" + "[Version]\n" + " rocksdb_version=3.14.0\n" + " options_file_version=1\n" + "[ DBOptions ]\n" + " # note that we don't support space around \"=\"\n" + " max_open_files=12345;\n" + " max_background_flushes=301 # comment after a statement is fine\n" + " # max_background_flushes=1000 # this line would be ignored\n" + " # max_background_compactions=2000 # so does this one\n" + " max_total_wal_size=1024 # keep_log_file_num=1000\n" + "[CFOptions \"default\"] # column family must be specified\n" + " # in the correct order\n" + " # if a section is blank, we will use the default\n"; + + const std::string kTestFileName = "test-rocksdb-options.ini"; + env_->WriteToNewFile(kTestFileName, options_file_content); + RocksDBOptionsParser parser; + ASSERT_OK(parser.Parse(kTestFileName, env_.get())); + + ASSERT_OK(RocksDBOptionsParser::VerifyDBOptions(*parser.db_opt(), db_opt)); + ASSERT_EQ(parser.NumColumnFamilies(), 1U); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions( + *parser.GetCFOptions("default"), cf_opt)); +} + +TEST_F(OptionsParserTest, ExtraSpace) { + std::string options_file_content = + "# This is a testing option string.\n" + "# Currently we only support \"#\" styled comment.\n" + "\n" + "[ Version ]\n" + " rocksdb_version = 3.14.0 \n" + " options_file_version=1 # some comment\n" + "[DBOptions ] # some comment\n" + "max_open_files=12345 \n" + " max_background_flushes = 301 \n" + " max_total_wal_size = 1024 # keep_log_file_num=1000\n" + " [CFOptions \"default\" ]\n" + " # if a section is blank, we will use the default\n"; + + const std::string kTestFileName = "test-rocksdb-options.ini"; + env_->WriteToNewFile(kTestFileName, options_file_content); + RocksDBOptionsParser parser; + ASSERT_OK(parser.Parse(kTestFileName, env_.get())); +} + +TEST_F(OptionsParserTest, MissingDBOptions) { + std::string options_file_content = + "# This is a testing option string.\n" + "# Currently we only support \"#\" styled comment.\n" + "\n" + "[Version]\n" + " rocksdb_version=3.14.0\n" + " options_file_version=1\n" + "[CFOptions \"default\"]\n" + " # if a section is blank, we will use the default\n"; + + const std::string kTestFileName = "test-rocksdb-options.ini"; + env_->WriteToNewFile(kTestFileName, options_file_content); + RocksDBOptionsParser parser; + ASSERT_NOK(parser.Parse(kTestFileName, env_.get())); +} + +TEST_F(OptionsParserTest, DoubleDBOptions) { + DBOptions db_opt; + db_opt.max_open_files = 12345; + db_opt.max_background_flushes = 301; + db_opt.max_total_wal_size = 1024; + ColumnFamilyOptions cf_opt; + + std::string options_file_content = + "# This is a testing option string.\n" + "# Currently we only support \"#\" styled comment.\n" + "\n" + "[Version]\n" + " rocksdb_version=3.14.0\n" + " options_file_version=1\n" + "[DBOptions]\n" + " max_open_files=12345\n" + " max_background_flushes=301\n" + " max_total_wal_size=1024 # keep_log_file_num=1000\n" + "[DBOptions]\n" + "[CFOptions \"default\"]\n" + " # if a section is blank, we will use the default\n"; + + const std::string kTestFileName = "test-rocksdb-options.ini"; + env_->WriteToNewFile(kTestFileName, options_file_content); + RocksDBOptionsParser parser; + ASSERT_NOK(parser.Parse(kTestFileName, env_.get())); +} + +TEST_F(OptionsParserTest, NoDefaultCFOptions) { + DBOptions db_opt; + db_opt.max_open_files = 12345; + db_opt.max_background_flushes = 301; + db_opt.max_total_wal_size = 1024; + ColumnFamilyOptions cf_opt; + + std::string options_file_content = + "# This is a testing option string.\n" + "# Currently we only support \"#\" styled comment.\n" + "\n" + "[Version]\n" + " rocksdb_version=3.14.0\n" + " options_file_version=1\n" + "[DBOptions]\n" + " max_open_files=12345\n" + " max_background_flushes=301\n" + " max_total_wal_size=1024 # keep_log_file_num=1000\n" + "[CFOptions \"something_else\"]\n" + " # if a section is blank, we will use the default\n"; + + const std::string kTestFileName = "test-rocksdb-options.ini"; + env_->WriteToNewFile(kTestFileName, options_file_content); + RocksDBOptionsParser parser; + ASSERT_NOK(parser.Parse(kTestFileName, env_.get())); +} + +TEST_F(OptionsParserTest, DefaultCFOptionsMustBeTheFirst) { + DBOptions db_opt; + db_opt.max_open_files = 12345; + db_opt.max_background_flushes = 301; + db_opt.max_total_wal_size = 1024; + ColumnFamilyOptions cf_opt; + + std::string options_file_content = + "# This is a testing option string.\n" + "# Currently we only support \"#\" styled comment.\n" + "\n" + "[Version]\n" + " rocksdb_version=3.14.0\n" + " options_file_version=1\n" + "[DBOptions]\n" + " max_open_files=12345\n" + " max_background_flushes=301\n" + " max_total_wal_size=1024 # keep_log_file_num=1000\n" + "[CFOptions \"something_else\"]\n" + " # if a section is blank, we will use the default\n" + "[CFOptions \"default\"]\n" + " # if a section is blank, we will use the default\n"; + + const std::string kTestFileName = "test-rocksdb-options.ini"; + env_->WriteToNewFile(kTestFileName, options_file_content); + RocksDBOptionsParser parser; + ASSERT_NOK(parser.Parse(kTestFileName, env_.get())); +} + +TEST_F(OptionsParserTest, DuplicateCFOptions) { + DBOptions db_opt; + db_opt.max_open_files = 12345; + db_opt.max_background_flushes = 301; + db_opt.max_total_wal_size = 1024; + ColumnFamilyOptions cf_opt; + + std::string options_file_content = + "# This is a testing option string.\n" + "# Currently we only support \"#\" styled comment.\n" + "\n" + "[Version]\n" + " rocksdb_version=3.14.0\n" + " options_file_version=1\n" + "[DBOptions]\n" + " max_open_files=12345\n" + " max_background_flushes=301\n" + " max_total_wal_size=1024 # keep_log_file_num=1000\n" + "[CFOptions \"default\"]\n" + "[CFOptions \"something_else\"]\n" + "[CFOptions \"something_else\"]\n"; + + const std::string kTestFileName = "test-rocksdb-options.ini"; + env_->WriteToNewFile(kTestFileName, options_file_content); + RocksDBOptionsParser parser; + ASSERT_NOK(parser.Parse(kTestFileName, env_.get())); +} + +TEST_F(OptionsParserTest, IgnoreUnknownOptions) { + DBOptions db_opt; + db_opt.max_open_files = 12345; + db_opt.max_background_flushes = 301; + db_opt.max_total_wal_size = 1024; + ColumnFamilyOptions cf_opt; + + std::string options_file_content = + "# This is a testing option string.\n" + "# Currently we only support \"#\" styled comment.\n" + "\n" + "[Version]\n" + " rocksdb_version=3.14.0\n" + " options_file_version=1\n" + "[DBOptions]\n" + " max_open_files=12345\n" + " max_background_flushes=301\n" + " max_total_wal_size=1024 # keep_log_file_num=1000\n" + " unknown_db_option1=321\n" + " unknown_db_option2=false\n" + "[CFOptions \"default\"]\n" + " unknown_cf_option1=hello\n" + "[CFOptions \"something_else\"]\n" + " unknown_cf_option2=world\n" + " # if a section is blank, we will use the default\n"; + + const std::string kTestFileName = "test-rocksdb-options.ini"; + env_->WriteToNewFile(kTestFileName, options_file_content); + RocksDBOptionsParser parser; + ASSERT_NOK(parser.Parse(kTestFileName, env_.get())); + ASSERT_OK(parser.Parse(kTestFileName, env_.get(), + true /* ignore_unknown_options */)); +} + +TEST_F(OptionsParserTest, ParseVersion) { + DBOptions db_opt; + db_opt.max_open_files = 12345; + db_opt.max_background_flushes = 301; + db_opt.max_total_wal_size = 1024; + ColumnFamilyOptions cf_opt; + + std::string file_template = + "# This is a testing option string.\n" + "# Currently we only support \"#\" styled comment.\n" + "\n" + "[Version]\n" + " rocksdb_version=3.13.1\n" + " options_file_version=%s\n" + "[DBOptions]\n" + "[CFOptions \"default\"]\n"; + const int kLength = 1000; + char buffer[kLength]; + RocksDBOptionsParser parser; + + const std::vector invalid_versions = { + "a.b.c", "3.2.2b", "3.-12", "3. 1", // only digits and dots are allowed + "1.2.3.4", + "1.2.3" // can only contains at most one dot. + "0", // options_file_version must be at least one + "3..2", + ".", ".1.2", // must have at least one digit before each dot + "1.2.", "1.", "2.34."}; // must have at least one digit after each dot + for (auto iv : invalid_versions) { + snprintf(buffer, kLength - 1, file_template.c_str(), iv.c_str()); + + parser.Reset(); + env_->WriteToNewFile(iv, buffer); + ASSERT_NOK(parser.Parse(iv, env_.get())); + } + + const std::vector valid_versions = { + "1.232", "100", "3.12", "1", "12.3 ", " 1.25 "}; + for (auto vv : valid_versions) { + snprintf(buffer, kLength - 1, file_template.c_str(), vv.c_str()); + parser.Reset(); + env_->WriteToNewFile(vv, buffer); + ASSERT_OK(parser.Parse(vv, env_.get())); + } +} + +void VerifyCFPointerTypedOptions( + ColumnFamilyOptions* base_cf_opt, const ColumnFamilyOptions* new_cf_opt, + const std::unordered_map* new_cf_opt_map) { + std::string name_buffer; + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(*base_cf_opt, *new_cf_opt, + new_cf_opt_map)); + + // change the name of merge operator back-and-forth + { + auto* merge_operator = dynamic_cast( + base_cf_opt->merge_operator.get()); + if (merge_operator != nullptr) { + name_buffer = merge_operator->Name(); + // change the name and expect non-ok status + merge_operator->SetName("some-other-name"); + ASSERT_NOK(RocksDBOptionsParser::VerifyCFOptions( + *base_cf_opt, *new_cf_opt, new_cf_opt_map)); + // change the name back and expect ok status + merge_operator->SetName(name_buffer); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(*base_cf_opt, *new_cf_opt, + new_cf_opt_map)); + } + } + + // change the name of the compaction filter factory back-and-forth + { + auto* compaction_filter_factory = + dynamic_cast( + base_cf_opt->compaction_filter_factory.get()); + if (compaction_filter_factory != nullptr) { + name_buffer = compaction_filter_factory->Name(); + // change the name and expect non-ok status + compaction_filter_factory->SetName("some-other-name"); + ASSERT_NOK(RocksDBOptionsParser::VerifyCFOptions( + *base_cf_opt, *new_cf_opt, new_cf_opt_map)); + // change the name back and expect ok status + compaction_filter_factory->SetName(name_buffer); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(*base_cf_opt, *new_cf_opt, + new_cf_opt_map)); + } + } + + // test by setting compaction_filter to nullptr + { + auto* tmp_compaction_filter = base_cf_opt->compaction_filter; + if (tmp_compaction_filter != nullptr) { + base_cf_opt->compaction_filter = nullptr; + // set compaction_filter to nullptr and expect non-ok status + ASSERT_NOK(RocksDBOptionsParser::VerifyCFOptions( + *base_cf_opt, *new_cf_opt, new_cf_opt_map)); + // set the value back and expect ok status + base_cf_opt->compaction_filter = tmp_compaction_filter; + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(*base_cf_opt, *new_cf_opt, + new_cf_opt_map)); + } + } + + // test by setting table_factory to nullptr + { + auto tmp_table_factory = base_cf_opt->table_factory; + if (tmp_table_factory != nullptr) { + base_cf_opt->table_factory.reset(); + // set table_factory to nullptr and expect non-ok status + ASSERT_NOK(RocksDBOptionsParser::VerifyCFOptions( + *base_cf_opt, *new_cf_opt, new_cf_opt_map)); + // set the value back and expect ok status + base_cf_opt->table_factory = tmp_table_factory; + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(*base_cf_opt, *new_cf_opt, + new_cf_opt_map)); + } + } + + // test by setting memtable_factory to nullptr + { + auto tmp_memtable_factory = base_cf_opt->memtable_factory; + if (tmp_memtable_factory != nullptr) { + base_cf_opt->memtable_factory.reset(); + // set memtable_factory to nullptr and expect non-ok status + ASSERT_NOK(RocksDBOptionsParser::VerifyCFOptions( + *base_cf_opt, *new_cf_opt, new_cf_opt_map)); + // set the value back and expect ok status + base_cf_opt->memtable_factory = tmp_memtable_factory; + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions(*base_cf_opt, *new_cf_opt, + new_cf_opt_map)); + } + } +} + +TEST_F(OptionsParserTest, DumpAndParse) { + DBOptions base_db_opt; + std::vector base_cf_opts; + std::vector cf_names = {"default", "cf1", "cf2", "cf3", + "c:f:4:4:4" + "p\\i\\k\\a\\chu\\\\\\", + "###rocksdb#1-testcf#2###"}; + const int num_cf = static_cast(cf_names.size()); + Random rnd(302); + test::RandomInitDBOptions(&base_db_opt, &rnd); + base_db_opt.db_log_dir += "/#odd #but #could #happen #path #/\\\\#OMG"; + + BlockBasedTableOptions special_bbto; + special_bbto.cache_index_and_filter_blocks = true; + special_bbto.block_size = 999999; + + for (int c = 0; c < num_cf; ++c) { + ColumnFamilyOptions cf_opt; + Random cf_rnd(0xFB + c); + test::RandomInitCFOptions(&cf_opt, &cf_rnd); + if (c < 4) { + cf_opt.prefix_extractor.reset(test::RandomSliceTransform(&rnd, c)); + } + if (c < 3) { + cf_opt.table_factory.reset(test::RandomTableFactory(&rnd, c)); + } else if (c == 4) { + cf_opt.table_factory.reset(NewBlockBasedTableFactory(special_bbto)); + } + base_cf_opts.emplace_back(cf_opt); + } + + const std::string kOptionsFileName = "test-persisted-options.ini"; + ASSERT_OK(PersistRocksDBOptions(base_db_opt, cf_names, base_cf_opts, + kOptionsFileName, env_.get())); + + RocksDBOptionsParser parser; + ASSERT_OK(parser.Parse(kOptionsFileName, env_.get())); + + // Make sure block-based table factory options was deserialized correctly + std::shared_ptr ttf = (*parser.cf_opts())[4].table_factory; + ASSERT_EQ(BlockBasedTableFactory::kName, std::string(ttf->Name())); + const BlockBasedTableOptions& parsed_bbto = + static_cast(ttf.get())->table_options(); + ASSERT_EQ(special_bbto.block_size, parsed_bbto.block_size); + ASSERT_EQ(special_bbto.cache_index_and_filter_blocks, + parsed_bbto.cache_index_and_filter_blocks); + + ASSERT_OK(RocksDBOptionsParser::VerifyRocksDBOptionsFromFile( + base_db_opt, cf_names, base_cf_opts, kOptionsFileName, env_.get())); + + ASSERT_OK( + RocksDBOptionsParser::VerifyDBOptions(*parser.db_opt(), base_db_opt)); + for (int c = 0; c < num_cf; ++c) { + const auto* cf_opt = parser.GetCFOptions(cf_names[c]); + ASSERT_NE(cf_opt, nullptr); + ASSERT_OK(RocksDBOptionsParser::VerifyCFOptions( + base_cf_opts[c], *cf_opt, &(parser.cf_opt_maps()->at(c)))); + } + + // Further verify pointer-typed options + for (int c = 0; c < num_cf; ++c) { + const auto* cf_opt = parser.GetCFOptions(cf_names[c]); + ASSERT_NE(cf_opt, nullptr); + VerifyCFPointerTypedOptions(&base_cf_opts[c], cf_opt, + &(parser.cf_opt_maps()->at(c))); + } + + ASSERT_EQ(parser.GetCFOptions("does not exist"), nullptr); + + base_db_opt.max_open_files++; + ASSERT_NOK(RocksDBOptionsParser::VerifyRocksDBOptionsFromFile( + base_db_opt, cf_names, base_cf_opts, kOptionsFileName, env_.get())); + + for (int c = 0; c < num_cf; ++c) { + if (base_cf_opts[c].compaction_filter) { + delete base_cf_opts[c].compaction_filter; + } + } +} + +TEST_F(OptionsParserTest, DifferentDefault) { + const std::string kOptionsFileName = "test-persisted-options.ini"; + + ColumnFamilyOptions cf_level_opts; + cf_level_opts.OptimizeLevelStyleCompaction(); + + ColumnFamilyOptions cf_univ_opts; + cf_univ_opts.OptimizeUniversalStyleCompaction(); + + ASSERT_OK(PersistRocksDBOptions(DBOptions(), {"default", "universal"}, + {cf_level_opts, cf_univ_opts}, + kOptionsFileName, env_.get())); + + RocksDBOptionsParser parser; + ASSERT_OK(parser.Parse(kOptionsFileName, env_.get())); + + { + Options old_default_opts; + old_default_opts.OldDefaults(); + ASSERT_EQ(10 * 1048576, old_default_opts.max_bytes_for_level_base); + ASSERT_EQ(5000, old_default_opts.max_open_files); + ASSERT_EQ(2 * 1024U * 1024U, old_default_opts.delayed_write_rate); + ASSERT_EQ(WALRecoveryMode::kTolerateCorruptedTailRecords, + old_default_opts.wal_recovery_mode); + } + { + Options old_default_opts; + old_default_opts.OldDefaults(4, 6); + ASSERT_EQ(10 * 1048576, old_default_opts.max_bytes_for_level_base); + ASSERT_EQ(5000, old_default_opts.max_open_files); + } + { + Options old_default_opts; + old_default_opts.OldDefaults(4, 7); + ASSERT_NE(10 * 1048576, old_default_opts.max_bytes_for_level_base); + ASSERT_NE(4, old_default_opts.table_cache_numshardbits); + ASSERT_EQ(5000, old_default_opts.max_open_files); + ASSERT_EQ(2 * 1024U * 1024U, old_default_opts.delayed_write_rate); + } + { + ColumnFamilyOptions old_default_cf_opts; + old_default_cf_opts.OldDefaults(); + ASSERT_EQ(2 * 1048576, old_default_cf_opts.target_file_size_base); + ASSERT_EQ(4 << 20, old_default_cf_opts.write_buffer_size); + ASSERT_EQ(2 * 1048576, old_default_cf_opts.target_file_size_base); + ASSERT_EQ(0, old_default_cf_opts.soft_pending_compaction_bytes_limit); + ASSERT_EQ(0, old_default_cf_opts.hard_pending_compaction_bytes_limit); + ASSERT_EQ(CompactionPri::kByCompensatedSize, + old_default_cf_opts.compaction_pri); + } + { + ColumnFamilyOptions old_default_cf_opts; + old_default_cf_opts.OldDefaults(4, 6); + ASSERT_EQ(2 * 1048576, old_default_cf_opts.target_file_size_base); + ASSERT_EQ(CompactionPri::kByCompensatedSize, + old_default_cf_opts.compaction_pri); + } + { + ColumnFamilyOptions old_default_cf_opts; + old_default_cf_opts.OldDefaults(4, 7); + ASSERT_NE(2 * 1048576, old_default_cf_opts.target_file_size_base); + ASSERT_EQ(CompactionPri::kByCompensatedSize, + old_default_cf_opts.compaction_pri); + } + { + Options old_default_opts; + old_default_opts.OldDefaults(5, 1); + ASSERT_EQ(2 * 1024U * 1024U, old_default_opts.delayed_write_rate); + } + { + Options old_default_opts; + old_default_opts.OldDefaults(5, 2); + ASSERT_EQ(16 * 1024U * 1024U, old_default_opts.delayed_write_rate); + } + + Options small_opts; + small_opts.OptimizeForSmallDb(); + ASSERT_EQ(2 << 20, small_opts.write_buffer_size); + ASSERT_EQ(5000, small_opts.max_open_files); +} + +class OptionsSanityCheckTest : public OptionsParserTest { + public: + OptionsSanityCheckTest() {} + + protected: + Status SanityCheckCFOptions(const ColumnFamilyOptions& cf_opts, + OptionsSanityCheckLevel level) { + return RocksDBOptionsParser::VerifyRocksDBOptionsFromFile( + DBOptions(), {"default"}, {cf_opts}, kOptionsFileName, env_.get(), + level); + } + + Status PersistCFOptions(const ColumnFamilyOptions& cf_opts) { + Status s = env_->DeleteFile(kOptionsFileName); + if (!s.ok()) { + return s; + } + return PersistRocksDBOptions(DBOptions(), {"default"}, {cf_opts}, + kOptionsFileName, env_.get()); + } + + const std::string kOptionsFileName = "OPTIONS"; +}; + +TEST_F(OptionsSanityCheckTest, SanityCheck) { + ColumnFamilyOptions opts; + Random rnd(301); + + // default ColumnFamilyOptions + { + ASSERT_OK(PersistCFOptions(opts)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + } + + // prefix_extractor + { + // Okay to change prefix_extractor form nullptr to non-nullptr + ASSERT_EQ(opts.prefix_extractor.get(), nullptr); + opts.prefix_extractor.reset(NewCappedPrefixTransform(10)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelLooselyCompatible)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelNone)); + + // persist the change + ASSERT_OK(PersistCFOptions(opts)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + + // use same prefix extractor but with different parameter + opts.prefix_extractor.reset(NewCappedPrefixTransform(15)); + // expect pass only in kSanityLevelLooselyCompatible + ASSERT_NOK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelLooselyCompatible)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelNone)); + + // repeat the test with FixedPrefixTransform + opts.prefix_extractor.reset(NewFixedPrefixTransform(10)); + ASSERT_NOK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelLooselyCompatible)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelNone)); + + // persist the change of prefix_extractor + ASSERT_OK(PersistCFOptions(opts)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + + // use same prefix extractor but with different parameter + opts.prefix_extractor.reset(NewFixedPrefixTransform(15)); + // expect pass only in kSanityLevelLooselyCompatible + ASSERT_NOK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelLooselyCompatible)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelNone)); + + // Change prefix extractor from non-nullptr to nullptr + opts.prefix_extractor.reset(); + // expect pass as it's safe to change prefix_extractor + // from non-null to null + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelLooselyCompatible)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelNone)); + } + // persist the change + ASSERT_OK(PersistCFOptions(opts)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + + // table_factory + { + for (int tb = 0; tb <= 2; ++tb) { + // change the table factory + opts.table_factory.reset(test::RandomTableFactory(&rnd, tb)); + ASSERT_NOK(SanityCheckCFOptions(opts, kSanityLevelLooselyCompatible)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelNone)); + + // persist the change + ASSERT_OK(PersistCFOptions(opts)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + } + } + + // merge_operator + { + for (int test = 0; test < 5; ++test) { + // change the merge operator + opts.merge_operator.reset(test::RandomMergeOperator(&rnd)); + ASSERT_NOK(SanityCheckCFOptions(opts, kSanityLevelLooselyCompatible)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelNone)); + + // persist the change + ASSERT_OK(PersistCFOptions(opts)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + } + } + + // compaction_filter + { + for (int test = 0; test < 5; ++test) { + // change the compaction filter + opts.compaction_filter = test::RandomCompactionFilter(&rnd); + ASSERT_NOK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelLooselyCompatible)); + + // persist the change + ASSERT_OK(PersistCFOptions(opts)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + delete opts.compaction_filter; + opts.compaction_filter = nullptr; + } + } + + // compaction_filter_factory + { + for (int test = 0; test < 5; ++test) { + // change the compaction filter factory + opts.compaction_filter_factory.reset( + test::RandomCompactionFilterFactory(&rnd)); + ASSERT_NOK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelLooselyCompatible)); + + // persist the change + ASSERT_OK(PersistCFOptions(opts)); + ASSERT_OK(SanityCheckCFOptions(opts, kSanityLevelExactMatch)); + } + } +} + +namespace { +bool IsEscapedString(const std::string& str) { + for (size_t i = 0; i < str.size(); ++i) { + if (str[i] == '\\') { + // since we already handle those two consecutive '\'s in + // the next if-then branch, any '\' appear at the end + // of an escaped string in such case is not valid. + if (i == str.size() - 1) { + return false; + } + if (str[i + 1] == '\\') { + // if there're two consecutive '\'s, skip the second one. + i++; + continue; + } + switch (str[i + 1]) { + case ':': + case '\\': + case '#': + continue; + default: + // if true, '\' together with str[i + 1] is not a valid escape. + if (UnescapeChar(str[i + 1]) == str[i + 1]) { + return false; + } + } + } else if (isSpecialChar(str[i]) && (i == 0 || str[i - 1] != '\\')) { + return false; + } + } + return true; +} +} // namespace + +TEST_F(OptionsParserTest, EscapeOptionString) { + ASSERT_EQ(UnescapeOptionString( + "This is a test string with \\# \\: and \\\\ escape chars."), + "This is a test string with # : and \\ escape chars."); + + ASSERT_EQ( + EscapeOptionString("This is a test string with # : and \\ escape chars."), + "This is a test string with \\# \\: and \\\\ escape chars."); + + std::string readible_chars = + "A String like this \"1234567890-=_)(*&^%$#@!ertyuiop[]{POIU" + "YTREWQasdfghjkl;':LKJHGFDSAzxcvbnm,.?>" + " -// implementation. Also, some implementations are much -// slower than a memory-barrier based implementation (~16ns for -// based acquire-load vs. ~1ns for a barrier based -// acquire-load). -// This code is based on atomicops-internals-* in Google's perftools: -// http://code.google.com/p/google-perftools/source/browse/#svn%2Ftrunk%2Fsrc%2Fbase - -#ifndef PORT_ATOMIC_POINTER_H_ -#define PORT_ATOMIC_POINTER_H_ - -#include -#ifdef ROCKSDB_ATOMIC_PRESENT -#include -#endif -#ifdef OS_WIN -#include -#endif -#ifdef OS_MACOSX -#include -#endif - -#if defined(_M_X64) || defined(__x86_64__) -#define ARCH_CPU_X86_FAMILY 1 -#elif defined(_M_IX86) || defined(__i386__) || defined(__i386) -#define ARCH_CPU_X86_FAMILY 1 -#elif defined(__ARMEL__) -#define ARCH_CPU_ARM_FAMILY 1 -#endif - -namespace rocksdb { -namespace port { - -// Define MemoryBarrier() if available -// Windows on x86 -#if defined(OS_WIN) && defined(COMPILER_MSVC) && defined(ARCH_CPU_X86_FAMILY) -// windows.h already provides a MemoryBarrier(void) macro -// http://msdn.microsoft.com/en-us/library/ms684208(v=vs.85).aspx -#define ROCKSDB_HAVE_MEMORY_BARRIER - -// Gcc on x86 -#elif defined(ARCH_CPU_X86_FAMILY) && defined(__GNUC__) -inline void MemoryBarrier() { - // See http://gcc.gnu.org/ml/gcc/2003-04/msg01180.html for a discussion on - // this idiom. Also see http://en.wikipedia.org/wiki/Memory_ordering. - __asm__ __volatile__("" : : : "memory"); -} -#define ROCKSDB_HAVE_MEMORY_BARRIER - -// Sun Studio -#elif defined(ARCH_CPU_X86_FAMILY) && defined(__SUNPRO_CC) -inline void MemoryBarrier() { - // See http://gcc.gnu.org/ml/gcc/2003-04/msg01180.html for a discussion on - // this idiom. Also see http://en.wikipedia.org/wiki/Memory_ordering. - asm volatile("" : : : "memory"); -} -#define ROCKSDB_HAVE_MEMORY_BARRIER - -// Mac OS -#elif defined(OS_MACOSX) -inline void MemoryBarrier() { - OSMemoryBarrier(); -} -#define ROCKSDB_HAVE_MEMORY_BARRIER - -// ARM Linux -#elif defined(ARCH_CPU_ARM_FAMILY) && defined(__linux__) -typedef void (*LinuxKernelMemoryBarrierFunc)(void); -// The Linux ARM kernel provides a highly optimized device-specific memory -// barrier function at a fixed memory address that is mapped in every -// user-level process. -// -// This beats using CPU-specific instructions which are, on single-core -// devices, un-necessary and very costly (e.g. ARMv7-A "dmb" takes more -// than 180ns on a Cortex-A8 like the one on a Nexus One). Benchmarking -// shows that the extra function call cost is completely negligible on -// multi-core devices. -// -inline void MemoryBarrier() { - (*(LinuxKernelMemoryBarrierFunc)0xffff0fa0)(); -} -#define ROCKSDB_HAVE_MEMORY_BARRIER - -#endif - -// AtomicPointer built using platform-specific MemoryBarrier() -#if defined(ROCKSDB_HAVE_MEMORY_BARRIER) -class AtomicPointer { - private: - void* rep_; - public: - AtomicPointer() { } - explicit AtomicPointer(void* p) : rep_(p) {} - inline void* NoBarrier_Load() const { return rep_; } - inline void NoBarrier_Store(void* v) { rep_ = v; } - inline void* Acquire_Load() const { - void* result = rep_; - MemoryBarrier(); - return result; - } - inline void Release_Store(void* v) { - MemoryBarrier(); - rep_ = v; - } -}; - -// AtomicPointer based on -#elif defined(ROCKSDB_ATOMIC_PRESENT) -class AtomicPointer { - private: - std::atomic rep_; - public: - AtomicPointer() { } - explicit AtomicPointer(void* v) : rep_(v) { } - inline void* Acquire_Load() const { - return rep_.load(std::memory_order_acquire); - } - inline void Release_Store(void* v) { - rep_.store(v, std::memory_order_release); - } - inline void* NoBarrier_Load() const { - return rep_.load(std::memory_order_relaxed); - } - inline void NoBarrier_Store(void* v) { - rep_.store(v, std::memory_order_relaxed); - } -}; - -// We have neither MemoryBarrier(), nor -#else -#error Please implement AtomicPointer for this platform. - -#endif - -#undef ROCKSDB_HAVE_MEMORY_BARRIER -#undef ARCH_CPU_X86_FAMILY -#undef ARCH_CPU_ARM_FAMILY - -} // namespace port -} // namespace rocksdb - -#endif // PORT_ATOMIC_POINTER_H_ diff --git a/port/dirent.h b/port/dirent.h new file mode 100644 index 00000000000..7bcc3569780 --- /dev/null +++ b/port/dirent.h @@ -0,0 +1,47 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// See port_example.h for documentation for the following types/functions. + +#ifndef STORAGE_LEVELDB_PORT_DIRENT_H_ +#define STORAGE_LEVELDB_PORT_DIRENT_H_ + +#ifdef ROCKSDB_PLATFORM_POSIX +#include +#include +#elif defined(OS_WIN) + +namespace rocksdb { +namespace port { + +struct dirent { + char d_name[_MAX_PATH]; /* filename */ +}; + +struct DIR; + +DIR* opendir(const char* name); + +dirent* readdir(DIR* dirp); + +int closedir(DIR* dirp); + +} // namespace port + +using port::dirent; +using port::DIR; +using port::opendir; +using port::readdir; +using port::closedir; + +} // namespace rocksdb + +#endif // OS_WIN + +#endif // STORAGE_LEVELDB_PORT_DIRENT_H_ diff --git a/port/likely.h b/port/likely.h index ede0df5a15b..e5ef786f2ec 100644 --- a/port/likely.h +++ b/port/likely.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be diff --git a/port/port.h b/port/port.h index 2dc9a0fa646..13aa56d47b1 100644 --- a/port/port.h +++ b/port/port.h @@ -1,22 +1,21 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. -#ifndef STORAGE_LEVELDB_PORT_PORT_H_ -#define STORAGE_LEVELDB_PORT_PORT_H_ +#pragma once -#include +#include // Include the appropriate platform specific file below. If you are // porting to a new platform, see "port_example.h" for documentation // of what the new port_.h file must provide. #if defined(ROCKSDB_PLATFORM_POSIX) -# include "port/port_posix.h" +#include "port/port_posix.h" +#elif defined(OS_WIN) +#include "port/win/port_win.h" #endif - -#endif // STORAGE_LEVELDB_PORT_PORT_H_ diff --git a/port/port_example.h b/port/port_example.h index f124abb068b..05b3240669d 100644 --- a/port/port_example.h +++ b/port/port_example.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -75,35 +75,6 @@ typedef intptr_t OnceType; #define LEVELDB_ONCE_INIT 0 extern void InitOnce(port::OnceType*, void (*initializer)()); -// A type that holds a pointer that can be read or written atomically -// (i.e., without word-tearing.) -class AtomicPointer { - private: - intptr_t rep_; - public: - // Initialize to arbitrary value - AtomicPointer(); - - // Initialize to hold v - explicit AtomicPointer(void* v) : rep_(v) { } - - // Read and return the stored pointer with the guarantee that no - // later memory access (read or write) by this thread can be - // reordered ahead of this read. - void* Acquire_Load() const; - - // Set v as the stored pointer with the guarantee that no earlier - // memory access (read or write) by this thread can be reordered - // after this store. - void Release_Store(void* v); - - // Read the stored pointer with no ordering guarantees. - void* NoBarrier_Load() const; - - // Set va as the stored pointer with no ordering guarantees. - void NoBarrier_Store(void* v); -}; - // ------------------ Compression ------------------- // Store the snappy compression of "input[0,input_length-1]" in *output. diff --git a/port/port_posix.cc b/port/port_posix.cc index c5ea439eb52..129933bb1f9 100644 --- a/port/port_posix.cc +++ b/port/port_posix.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,11 +9,18 @@ #include "port/port_posix.h" -#include #include +#if defined(__i386__) || defined(__x86_64__) +#include +#endif #include -#include +#include +#include +#include #include +#include +#include +#include #include #include "util/logging.h" @@ -29,7 +36,7 @@ static int PthreadCall(const char* label, int result) { } Mutex::Mutex(bool adaptive) { -#ifdef OS_LINUX +#ifdef ROCKSDB_PTHREAD_ADAPTIVE_MUTEX if (!adaptive) { PthreadCall("init mutex", pthread_mutex_init(&mu_, nullptr)); } else { @@ -42,9 +49,9 @@ Mutex::Mutex(bool adaptive) { PthreadCall("destroy mutex attr", pthread_mutexattr_destroy(&mutex_attr)); } -#else // ignore adaptive for non-linux platform +#else PthreadCall("init mutex", pthread_mutex_init(&mu_, nullptr)); -#endif // OS_LINUX +#endif // ROCKSDB_PTHREAD_ADAPTIVE_MUTEX } Mutex::~Mutex() { PthreadCall("destroy mutex", pthread_mutex_destroy(&mu_)); } @@ -88,8 +95,8 @@ void CondVar::Wait() { bool CondVar::TimedWait(uint64_t abs_time_us) { struct timespec ts; - ts.tv_sec = abs_time_us / 1000000; - ts.tv_nsec = (abs_time_us % 1000000) * 1000; + ts.tv_sec = static_cast(abs_time_us / 1000000); + ts.tv_nsec = static_cast((abs_time_us % 1000000) * 1000); #ifndef NDEBUG mu_->locked_ = false; @@ -129,9 +136,72 @@ void RWMutex::ReadUnlock() { PthreadCall("read unlock", pthread_rwlock_unlock(&m void RWMutex::WriteUnlock() { PthreadCall("write unlock", pthread_rwlock_unlock(&mu_)); } +int PhysicalCoreID() { +#if defined(ROCKSDB_SCHED_GETCPU_PRESENT) && defined(__x86_64__) && \ + (__GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 22)) + // sched_getcpu uses VDSO getcpu() syscall since 2.22. I believe Linux offers VDSO + // support only on x86_64. This is the fastest/preferred method if available. + int cpuno = sched_getcpu(); + if (cpuno < 0) { + return -1; + } + return cpuno; +#elif defined(__x86_64__) || defined(__i386__) + // clang/gcc both provide cpuid.h, which defines __get_cpuid(), for x86_64 and i386. + unsigned eax, ebx = 0, ecx, edx; + if (!__get_cpuid(1, &eax, &ebx, &ecx, &edx)) { + return -1; + } + return ebx >> 24; +#else + // give up, the caller can generate a random number or something. + return -1; +#endif +} + void InitOnce(OnceType* once, void (*initializer)()) { PthreadCall("once", pthread_once(once, initializer)); } +void Crash(const std::string& srcfile, int srcline) { + fprintf(stdout, "Crashing at %s:%d\n", srcfile.c_str(), srcline); + fflush(stdout); + kill(getpid(), SIGTERM); +} + +int GetMaxOpenFiles() { +#if defined(RLIMIT_NOFILE) + struct rlimit no_files_limit; + if (getrlimit(RLIMIT_NOFILE, &no_files_limit) != 0) { + return -1; + } + // protect against overflow + if (no_files_limit.rlim_cur >= std::numeric_limits::max()) { + return std::numeric_limits::max(); + } + return static_cast(no_files_limit.rlim_cur); +#endif + return -1; +} + +void *cacheline_aligned_alloc(size_t size) { +#if __GNUC__ < 5 && defined(__SANITIZE_ADDRESS__) + return malloc(size); +#elif defined(_ISOC11_SOURCE) + return aligned_alloc(CACHE_LINE_SIZE, size); +#elif ( _POSIX_C_SOURCE >= 200112L || _XOPEN_SOURCE >= 600 || defined(__APPLE__)) + void *m; + errno = posix_memalign(&m, CACHE_LINE_SIZE, size); + return errno ? NULL : m; +#else + return malloc(size); +#endif +} + +void cacheline_aligned_free(void *memblock) { + free(memblock); +} + + } // namespace port } // namespace rocksdb diff --git a/port/port_posix.h b/port/port_posix.h index 2e3c868b38a..fe0d42644c4 100644 --- a/port/port_posix.h +++ b/port/port_posix.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,8 +9,17 @@ // // See port_example.h for documentation for the following types/functions. -#ifndef STORAGE_LEVELDB_PORT_PORT_POSIX_H_ -#define STORAGE_LEVELDB_PORT_PORT_POSIX_H_ +#pragma once + +#include +// size_t printf formatting named in the manner of C99 standard formatting +// strings such as PRIu64 +// in fact, we could use that one +#define ROCKSDB_PRIszt "zu" + +#define __declspec(S) + +#define ROCKSDB_NOEXCEPT noexcept #undef PLATFORM_IS_LITTLE_ENDIAN #if defined(OS_MACOSX) @@ -26,36 +35,26 @@ #else #define PLATFORM_IS_LITTLE_ENDIAN false #endif -#elif defined(OS_FREEBSD) || defined(OS_OPENBSD) || defined(OS_NETBSD) ||\ - defined(OS_DRAGONFLYBSD) || defined(OS_ANDROID) + #include +#elif defined(OS_AIX) #include + #include + #define PLATFORM_IS_LITTLE_ENDIAN (BYTE_ORDER == LITTLE_ENDIAN) + #include +#elif defined(OS_FREEBSD) || defined(OS_OPENBSD) || defined(OS_NETBSD) || \ + defined(OS_DRAGONFLYBSD) || defined(OS_ANDROID) #include + #include + #define PLATFORM_IS_LITTLE_ENDIAN (_BYTE_ORDER == _LITTLE_ENDIAN) #else #include #endif #include -#ifdef SNAPPY -#include -#endif - -#ifdef ZLIB -#include -#endif - -#ifdef BZIP2 -#include -#endif - -#if defined(LZ4) -#include -#include -#endif #include -#include #include -#include "rocksdb/options.h" -#include "port/atomic_pointer.h" +#include +#include #ifndef PLATFORM_IS_LITTLE_ENDIAN #define PLATFORM_IS_LITTLE_ENDIAN (__BYTE_ORDER == __LITTLE_ENDIAN) @@ -63,7 +62,7 @@ #if defined(OS_MACOSX) || defined(OS_SOLARIS) || defined(OS_FREEBSD) ||\ defined(OS_NETBSD) || defined(OS_OPENBSD) || defined(OS_DRAGONFLYBSD) ||\ - defined(OS_ANDROID) + defined(OS_ANDROID) || defined(CYGWIN) || defined(OS_AIX) // Use fread/fwrite/fflush on platforms without _unlocked variants #define fread_unlocked fread #define fwrite_unlocked fwrite @@ -78,13 +77,19 @@ #if defined(OS_ANDROID) && __ANDROID_API__ < 9 // fdatasync() was only introduced in API level 9 on Android. Use fsync() -// when targetting older platforms. +// when targeting older platforms. #define fdatasync fsync #endif namespace rocksdb { namespace port { +// For use at db/file_indexer.h kLevelMaxIndex +const int kMaxInt32 = std::numeric_limits::max(); +const uint64_t kMaxUint64 = std::numeric_limits::max(); +const int64_t kMaxInt64 = std::numeric_limits::max(); +const size_t kMaxSizet = std::numeric_limits::max(); + static const bool kLittleEndian = PLATFORM_IS_LITTLE_ENDIAN; #undef PLATFORM_IS_LITTLE_ENDIAN @@ -92,7 +97,19 @@ class CondVar; class Mutex { public: - /* implicit */ Mutex(bool adaptive = false); +// We want to give users opportunity to default all the mutexes to adaptive if +// not specified otherwise. This enables a quick way to conduct various +// performance related experiements. +// +// NB! Support for adaptive mutexes is turned on by definining +// ROCKSDB_PTHREAD_ADAPTIVE_MUTEX during the compilation. If you use RocksDB +// build environment then this happens automatically; otherwise it's up to the +// consumer to define the identifier. +#ifdef ROCKSDB_DEFAULT_TO_ADAPTIVE_MUTEX + explicit Mutex(bool adaptive = true); +#else + explicit Mutex(bool adaptive = false); +#endif ~Mutex(); void Lock(); @@ -146,348 +163,48 @@ class CondVar { Mutex* mu_; }; -typedef pthread_once_t OnceType; -#define LEVELDB_ONCE_INIT PTHREAD_ONCE_INIT -extern void InitOnce(OnceType* once, void (*initializer)()); +using Thread = std::thread; -inline bool Snappy_Compress(const CompressionOptions& opts, const char* input, - size_t length, ::std::string* output) { -#ifdef SNAPPY - output->resize(snappy::MaxCompressedLength(length)); - size_t outlen; - snappy::RawCompress(input, length, &(*output)[0], &outlen); - output->resize(outlen); - return true; -#endif - - return false; -} - -inline bool Snappy_GetUncompressedLength(const char* input, size_t length, - size_t* result) { -#ifdef SNAPPY - return snappy::GetUncompressedLength(input, length, result); -#else - return false; +static inline void AsmVolatilePause() { +#if defined(__i386__) || defined(__x86_64__) + asm volatile("pause"); +#elif defined(__aarch64__) + asm volatile("wfe"); +#elif defined(__powerpc64__) + asm volatile("or 27,27,27"); #endif + // it's okay for other platforms to be no-ops } -inline bool Snappy_Uncompress(const char* input, size_t length, - char* output) { -#ifdef SNAPPY - return snappy::RawUncompress(input, length, output); -#else - return false; -#endif -} +// Returns -1 if not available on this platform +extern int PhysicalCoreID(); -inline bool Zlib_Compress(const CompressionOptions& opts, const char* input, - size_t length, ::std::string* output) { -#ifdef ZLIB - // The memLevel parameter specifies how much memory should be allocated for - // the internal compression state. - // memLevel=1 uses minimum memory but is slow and reduces compression ratio. - // memLevel=9 uses maximum memory for optimal speed. - // The default value is 8. See zconf.h for more details. - static const int memLevel = 8; - z_stream _stream; - memset(&_stream, 0, sizeof(z_stream)); - int st = deflateInit2(&_stream, opts.level, Z_DEFLATED, opts.window_bits, - memLevel, opts.strategy); - if (st != Z_OK) { - return false; - } - - // Resize output to be the plain data length. - // This may not be big enough if the compression actually expands data. - output->resize(length); - - // Compress the input, and put compressed data in output. - _stream.next_in = (Bytef *)input; - _stream.avail_in = length; - - // Initialize the output size. - _stream.avail_out = length; - _stream.next_out = (Bytef *)&(*output)[0]; - - int old_sz =0, new_sz =0, new_sz_delta =0; - bool done = false; - while (!done) { - int st = deflate(&_stream, Z_FINISH); - switch (st) { - case Z_STREAM_END: - done = true; - break; - case Z_OK: - // No output space. Increase the output space by 20%. - // (Should we fail the compression since it expands the size?) - old_sz = output->size(); - new_sz_delta = (int)(output->size() * 0.2); - new_sz = output->size() + (new_sz_delta < 10 ? 10 : new_sz_delta); - output->resize(new_sz); - // Set more output. - _stream.next_out = (Bytef *)&(*output)[old_sz]; - _stream.avail_out = new_sz - old_sz; - break; - case Z_BUF_ERROR: - default: - deflateEnd(&_stream); - return false; - } - } - - output->resize(output->size() - _stream.avail_out); - deflateEnd(&_stream); - return true; -#endif - return false; -} +typedef pthread_once_t OnceType; +#define LEVELDB_ONCE_INIT PTHREAD_ONCE_INIT +extern void InitOnce(OnceType* once, void (*initializer)()); -inline char* Zlib_Uncompress(const char* input_data, size_t input_length, - int* decompress_size, int windowBits = -14) { -#ifdef ZLIB - z_stream _stream; - memset(&_stream, 0, sizeof(z_stream)); - - // For raw inflate, the windowBits should be -8..-15. - // If windowBits is bigger than zero, it will use either zlib - // header or gzip header. Adding 32 to it will do automatic detection. - int st = inflateInit2(&_stream, - windowBits > 0 ? windowBits + 32 : windowBits); - if (st != Z_OK) { - return nullptr; - } - - _stream.next_in = (Bytef *)input_data; - _stream.avail_in = input_length; - - // Assume the decompressed data size will 5x of compressed size. - int output_len = input_length * 5; - char* output = new char[output_len]; - int old_sz = output_len; - - _stream.next_out = (Bytef *)output; - _stream.avail_out = output_len; - - char* tmp = nullptr; - int output_len_delta; - bool done = false; - - //while(_stream.next_in != nullptr && _stream.avail_in != 0) { - while (!done) { - int st = inflate(&_stream, Z_SYNC_FLUSH); - switch (st) { - case Z_STREAM_END: - done = true; - break; - case Z_OK: - // No output space. Increase the output space by 20%. - old_sz = output_len; - output_len_delta = (int)(output_len * 0.2); - output_len += output_len_delta < 10 ? 10 : output_len_delta; - tmp = new char[output_len]; - memcpy(tmp, output, old_sz); - delete[] output; - output = tmp; - - // Set more output. - _stream.next_out = (Bytef *)(output + old_sz); - _stream.avail_out = output_len - old_sz; - break; - case Z_BUF_ERROR: - default: - delete[] output; - inflateEnd(&_stream); - return nullptr; - } - } - - *decompress_size = output_len - _stream.avail_out; - inflateEnd(&_stream); - return output; +#ifndef CACHE_LINE_SIZE + #if defined(__s390__) + #define CACHE_LINE_SIZE 256U + #elif defined(__powerpc__) || defined(__aarch64__) + #define CACHE_LINE_SIZE 128U + #else + #define CACHE_LINE_SIZE 64U + #endif #endif - return nullptr; -} - -inline bool BZip2_Compress(const CompressionOptions& opts, const char* input, - size_t length, ::std::string* output) { -#ifdef BZIP2 - bz_stream _stream; - memset(&_stream, 0, sizeof(bz_stream)); - - // Block size 1 is 100K. - // 0 is for silent. - // 30 is the default workFactor - int st = BZ2_bzCompressInit(&_stream, 1, 0, 30); - if (st != BZ_OK) { - return false; - } - - // Resize output to be the plain data length. - // This may not be big enough if the compression actually expands data. - output->resize(length); - - // Compress the input, and put compressed data in output. - _stream.next_in = (char *)input; - _stream.avail_in = length; - - // Initialize the output size. - _stream.next_out = (char *)&(*output)[0]; - _stream.avail_out = length; - - int old_sz =0, new_sz =0; - while(_stream.next_in != nullptr && _stream.avail_in != 0) { - int st = BZ2_bzCompress(&_stream, BZ_FINISH); - switch (st) { - case BZ_STREAM_END: - break; - case BZ_FINISH_OK: - // No output space. Increase the output space by 20%. - // (Should we fail the compression since it expands the size?) - old_sz = output->size(); - new_sz = (int)(output->size() * 1.2); - output->resize(new_sz); - // Set more output. - _stream.next_out = (char *)&(*output)[old_sz]; - _stream.avail_out = new_sz - old_sz; - break; - case BZ_SEQUENCE_ERROR: - default: - BZ2_bzCompressEnd(&_stream); - return false; - } - } - - output->resize(output->size() - _stream.avail_out); - BZ2_bzCompressEnd(&_stream); - return true; -#endif - return false; -} -inline char* BZip2_Uncompress(const char* input_data, size_t input_length, - int* decompress_size) { -#ifdef BZIP2 - bz_stream _stream; - memset(&_stream, 0, sizeof(bz_stream)); - - int st = BZ2_bzDecompressInit(&_stream, 0, 0); - if (st != BZ_OK) { - return nullptr; - } - - _stream.next_in = (char *)input_data; - _stream.avail_in = input_length; - - // Assume the decompressed data size will be 5x of compressed size. - int output_len = input_length * 5; - char* output = new char[output_len]; - int old_sz = output_len; - - _stream.next_out = (char *)output; - _stream.avail_out = output_len; - - char* tmp = nullptr; - - while(_stream.next_in != nullptr && _stream.avail_in != 0) { - int st = BZ2_bzDecompress(&_stream); - switch (st) { - case BZ_STREAM_END: - break; - case BZ_OK: - // No output space. Increase the output space by 20%. - old_sz = output_len; - output_len = (int)(output_len * 1.2); - tmp = new char[output_len]; - memcpy(tmp, output, old_sz); - delete[] output; - output = tmp; - - // Set more output. - _stream.next_out = (char *)(output + old_sz); - _stream.avail_out = output_len - old_sz; - break; - default: - delete[] output; - BZ2_bzDecompressEnd(&_stream); - return nullptr; - } - } - - *decompress_size = output_len - _stream.avail_out; - BZ2_bzDecompressEnd(&_stream); - return output; -#endif - return nullptr; -} +extern void *cacheline_aligned_alloc(size_t size); -inline bool LZ4_Compress(const CompressionOptions &opts, const char *input, - size_t length, ::std::string* output) { -#ifdef LZ4 - int compressBound = LZ4_compressBound(length); - output->resize(8 + compressBound); - char *p = const_cast(output->c_str()); - memcpy(p, &length, sizeof(length)); - size_t outlen; - outlen = LZ4_compress_limitedOutput(input, p + 8, length, compressBound); - if (outlen == 0) { - return false; - } - output->resize(8 + outlen); - return true; -#endif - return false; -} +extern void cacheline_aligned_free(void *memblock); -inline char* LZ4_Uncompress(const char* input_data, size_t input_length, - int* decompress_size) { -#ifdef LZ4 - if (input_length < 8) { - return nullptr; - } - int output_len; - memcpy(&output_len, input_data, sizeof(output_len)); - char *output = new char[output_len]; - *decompress_size = LZ4_decompress_safe_partial( - input_data + 8, output, input_length - 8, output_len, output_len); - if (*decompress_size < 0) { - delete[] output; - return nullptr; - } - return output; -#endif - return nullptr; -} +#define ALIGN_AS(n) alignas(n) -inline bool LZ4HC_Compress(const CompressionOptions &opts, const char* input, - size_t length, ::std::string* output) { -#ifdef LZ4 - int compressBound = LZ4_compressBound(length); - output->resize(8 + compressBound); - char *p = const_cast(output->c_str()); - memcpy(p, &length, sizeof(length)); - size_t outlen; -#ifdef LZ4_VERSION_MAJOR // they only started defining this since r113 - outlen = LZ4_compressHC2_limitedOutput(input, p + 8, length, compressBound, - opts.level); -#else - outlen = LZ4_compressHC_limitedOutput(input, p + 8, length, compressBound); -#endif - if (outlen == 0) { - return false; - } - output->resize(8 + outlen); - return true; -#endif - return false; -} +#define PREFETCH(addr, rw, locality) __builtin_prefetch(addr, rw, locality) -#define CACHE_LINE_SIZE 64U +extern void Crash(const std::string& srcfile, int srcline); -#define PREFETCH(addr, rw, locality) __builtin_prefetch(addr, rw, locality) +extern int GetMaxOpenFiles(); } // namespace port } // namespace rocksdb - -#endif // STORAGE_LEVELDB_PORT_PORT_POSIX_H_ diff --git a/port/stack_trace.cc b/port/stack_trace.cc index 76866e63cc8..baaf140142d 100644 --- a/port/stack_trace.cc +++ b/port/stack_trace.cc @@ -1,19 +1,21 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #include "port/stack_trace.h" -namespace rocksdb { -namespace port { - -#if defined(ROCKSDB_LITE) || !(defined(OS_LINUX) || defined(OS_MACOSX)) +#if defined(ROCKSDB_LITE) || !(defined(ROCKSDB_BACKTRACE) || defined(OS_MACOSX)) || \ + defined(CYGWIN) || defined(OS_FREEBSD) || defined(OS_SOLARIS) // noop +namespace rocksdb { +namespace port { void InstallStackTraceHandler() {} void PrintStack(int first_frames_to_skip) {} +} // namespace port +} // namespace rocksdb #else @@ -25,6 +27,9 @@ void PrintStack(int first_frames_to_skip) {} #include #include +namespace rocksdb { +namespace port { + namespace { #ifdef OS_LINUX @@ -33,7 +38,7 @@ const char* GetExecutableName() { char link[1024]; snprintf(link, sizeof(link), "/proc/%d/exe", getpid()); - auto read = readlink(link, name, sizeof(name)); + auto read = readlink(link, name, sizeof(name) - 1); if (-1 == read) { return nullptr; } else { @@ -67,7 +72,7 @@ void PrintStackTraceLine(const char* symbol, void* frame) { fprintf(stderr, "\n"); } -#elif OS_MACOSX +#elif defined(OS_MACOSX) void PrintStackTraceLine(const char* symbol, void* frame) { static int pid = getpid(); @@ -105,6 +110,7 @@ void PrintStack(int first_frames_to_skip) { fprintf(stderr, "#%-2d ", i - first_frames_to_skip); PrintStackTraceLine((symbols != nullptr) ? symbols[i] : nullptr, frames[i]); } + free(symbols); } static void StackTraceHandler(int sig) { @@ -126,7 +132,7 @@ void InstallStackTraceHandler() { signal(SIGABRT, StackTraceHandler); } -#endif - } // namespace port } // namespace rocksdb + +#endif diff --git a/port/stack_trace.h b/port/stack_trace.h index 8bc6c7d2ecc..f1d4f1febfa 100644 --- a/port/stack_trace.h +++ b/port/stack_trace.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #pragma once namespace rocksdb { diff --git a/port/sys_time.h b/port/sys_time.h new file mode 100644 index 00000000000..1e2ad0f5d6d --- /dev/null +++ b/port/sys_time.h @@ -0,0 +1,48 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +// This file is a portable substitute for sys/time.h which does not exist on +// Windows + +#ifndef STORAGE_LEVELDB_PORT_SYS_TIME_H_ +#define STORAGE_LEVELDB_PORT_SYS_TIME_H_ + +#if defined(OS_WIN) && defined(_MSC_VER) + +#include + +namespace rocksdb { + +namespace port { + +// Avoid including winsock2.h for this definition +typedef struct timeval { + long tv_sec; + long tv_usec; +} timeval; + +void gettimeofday(struct timeval* tv, struct timezone* tz); + +inline struct tm* localtime_r(const time_t* timep, struct tm* result) { + errno_t ret = localtime_s(result, timep); + return (ret == 0) ? result : NULL; +} +} + +using port::timeval; +using port::gettimeofday; +using port::localtime_r; +} + +#else +#include +#include +#endif + +#endif // STORAGE_LEVELDB_PORT_SYS_TIME_H_ diff --git a/port/util_logger.h b/port/util_logger.h new file mode 100644 index 00000000000..a8255ad6d65 --- /dev/null +++ b/port/util_logger.h @@ -0,0 +1,23 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef STORAGE_LEVELDB_PORT_UTIL_LOGGER_H_ +#define STORAGE_LEVELDB_PORT_UTIL_LOGGER_H_ + +// Include the appropriate platform specific file below. If you are +// porting to a new platform, see "port_example.h" for documentation +// of what the new port_.h file must provide. + +#if defined(ROCKSDB_PLATFORM_POSIX) +#include "env/posix_logger.h" +#elif defined(OS_WIN) +#include "port/win/win_logger.h" +#endif + +#endif // STORAGE_LEVELDB_PORT_UTIL_LOGGER_H_ diff --git a/port/win/env_default.cc b/port/win/env_default.cc new file mode 100644 index 00000000000..52a984f74ce --- /dev/null +++ b/port/win/env_default.cc @@ -0,0 +1,42 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include + +#include +#include "port/win/env_win.h" + +namespace rocksdb { +namespace port { + +// We choose to create this on the heap and using std::once for the following +// reasons +// 1) Currently available MS compiler does not implement atomic C++11 +// initialization of +// function local statics +// 2) We choose not to destroy the env because joining the threads from the +// system loader +// which destroys the statics (same as from DLLMain) creates a system loader +// dead-lock. +// in this manner any remaining threads are terminated OK. +namespace { + std::once_flag winenv_once_flag; + Env* envptr; +}; + +} + +Env* Env::Default() { + using namespace port; + std::call_once(winenv_once_flag, []() { envptr = new WinEnv(); }); + return envptr; +} + +} + diff --git a/port/win/env_win.cc b/port/win/env_win.cc new file mode 100644 index 00000000000..462148893b3 --- /dev/null +++ b/port/win/env_win.cc @@ -0,0 +1,1127 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "port/win/env_win.h" +#include "port/win/win_thread.h" +#include +#include +#include + +#include +#include // _getpid +#include // _access +#include // _rmdir, _mkdir, _getcwd +#include +#include + +#include "rocksdb/env.h" +#include "rocksdb/slice.h" + +#include "port/port.h" +#include "port/dirent.h" +#include "port/win/win_logger.h" +#include "port/win/io_win.h" + +#include "monitoring/iostats_context_imp.h" + +#include "monitoring/thread_status_updater.h" +#include "monitoring/thread_status_util.h" + +#include // for uuid generation +#include + +namespace rocksdb { + +ThreadStatusUpdater* CreateThreadStatusUpdater() { + return new ThreadStatusUpdater(); +} + +namespace { + +// RAII helpers for HANDLEs +const auto CloseHandleFunc = [](HANDLE h) { ::CloseHandle(h); }; +typedef std::unique_ptr UniqueCloseHandlePtr; + +void WinthreadCall(const char* label, std::error_code result) { + if (0 != result.value()) { + fprintf(stderr, "pthread %s: %s\n", label, strerror(result.value())); + abort(); + } +} + +} + +namespace port { + +WinEnvIO::WinEnvIO(Env* hosted_env) + : hosted_env_(hosted_env), + page_size_(4 * 1012), + allocation_granularity_(page_size_), + perf_counter_frequency_(0), + GetSystemTimePreciseAsFileTime_(NULL) { + + SYSTEM_INFO sinfo; + GetSystemInfo(&sinfo); + + page_size_ = sinfo.dwPageSize; + allocation_granularity_ = sinfo.dwAllocationGranularity; + + { + LARGE_INTEGER qpf; + BOOL ret = QueryPerformanceFrequency(&qpf); + assert(ret == TRUE); + perf_counter_frequency_ = qpf.QuadPart; + } + + HMODULE module = GetModuleHandle("kernel32.dll"); + if (module != NULL) { + GetSystemTimePreciseAsFileTime_ = (FnGetSystemTimePreciseAsFileTime)GetProcAddress( + module, "GetSystemTimePreciseAsFileTime"); + } +} + +WinEnvIO::~WinEnvIO() { +} + +Status WinEnvIO::DeleteFile(const std::string& fname) { + Status result; + + if (_unlink(fname.c_str())) { + result = IOError("Failed to delete: " + fname, errno); + } + + return result; +} + +Status WinEnvIO::GetCurrentTime(int64_t* unix_time) { + time_t time = std::time(nullptr); + if (time == (time_t)(-1)) { + return Status::NotSupported("Failed to get time"); + } + + *unix_time = time; + return Status::OK(); +} + +Status WinEnvIO::NewSequentialFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) { + Status s; + + result->reset(); + + // Corruption test needs to rename and delete files of these kind + // while they are still open with another handle. For that reason we + // allow share_write and delete(allows rename). + HANDLE hFile = INVALID_HANDLE_VALUE; + + DWORD fileFlags = FILE_ATTRIBUTE_READONLY; + + if (options.use_direct_reads && !options.use_mmap_reads) { + fileFlags |= FILE_FLAG_NO_BUFFERING; + } + + { + IOSTATS_TIMER_GUARD(open_nanos); + hFile = CreateFileA( + fname.c_str(), GENERIC_READ, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, + OPEN_EXISTING, // Original fopen mode is "rb" + fileFlags, NULL); + } + + if (INVALID_HANDLE_VALUE == hFile) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError("Failed to open NewSequentialFile" + fname, + lastError); + } else { + result->reset(new WinSequentialFile(fname, hFile, options)); + } + return s; +} + +Status WinEnvIO::NewRandomAccessFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) { + result->reset(); + Status s; + + // Open the file for read-only random access + // Random access is to disable read-ahead as the system reads too much data + DWORD fileFlags = FILE_ATTRIBUTE_READONLY; + + if (options.use_direct_reads && !options.use_mmap_reads) { + fileFlags |= FILE_FLAG_NO_BUFFERING; + } else { + fileFlags |= FILE_FLAG_RANDOM_ACCESS; + } + + /// Shared access is necessary for corruption test to pass + // almost all tests would work with a possible exception of fault_injection + HANDLE hFile = 0; + { + IOSTATS_TIMER_GUARD(open_nanos); + hFile = + CreateFileA(fname.c_str(), GENERIC_READ, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, + NULL, OPEN_EXISTING, fileFlags, NULL); + } + + if (INVALID_HANDLE_VALUE == hFile) { + auto lastError = GetLastError(); + return IOErrorFromWindowsError( + "NewRandomAccessFile failed to Create/Open: " + fname, lastError); + } + + UniqueCloseHandlePtr fileGuard(hFile, CloseHandleFunc); + + // CAUTION! This will map the entire file into the process address space + if (options.use_mmap_reads && sizeof(void*) >= 8) { + // Use mmap when virtual address-space is plentiful. + uint64_t fileSize; + + s = GetFileSize(fname, &fileSize); + + if (s.ok()) { + // Will not map empty files + if (fileSize == 0) { + return IOError( + "NewRandomAccessFile failed to map empty file: " + fname, EINVAL); + } + + HANDLE hMap = CreateFileMappingA(hFile, NULL, PAGE_READONLY, + 0, // Whole file at its present length + 0, + NULL); // Mapping name + + if (!hMap) { + auto lastError = GetLastError(); + return IOErrorFromWindowsError( + "Failed to create file mapping for NewRandomAccessFile: " + fname, + lastError); + } + + UniqueCloseHandlePtr mapGuard(hMap, CloseHandleFunc); + + const void* mapped_region = + MapViewOfFileEx(hMap, FILE_MAP_READ, + 0, // High DWORD of access start + 0, // Low DWORD + fileSize, + NULL); // Let the OS choose the mapping + + if (!mapped_region) { + auto lastError = GetLastError(); + return IOErrorFromWindowsError( + "Failed to MapViewOfFile for NewRandomAccessFile: " + fname, + lastError); + } + + result->reset(new WinMmapReadableFile(fname, hFile, hMap, mapped_region, + fileSize)); + + mapGuard.release(); + fileGuard.release(); + } + } else { + result->reset(new WinRandomAccessFile(fname, hFile, page_size_, options)); + fileGuard.release(); + } + return s; +} + +Status WinEnvIO::OpenWritableFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options, + bool reopen) { + + const size_t c_BufferCapacity = 64 * 1024; + + EnvOptions local_options(options); + + result->reset(); + Status s; + + DWORD fileFlags = FILE_ATTRIBUTE_NORMAL; + + if (local_options.use_direct_writes && !local_options.use_mmap_writes) { + fileFlags = FILE_FLAG_NO_BUFFERING; + } + + // Desired access. We are want to write only here but if we want to memory + // map + // the file then there is no write only mode so we have to create it + // Read/Write + // However, MapViewOfFile specifies only Write only + DWORD desired_access = GENERIC_WRITE; + DWORD shared_mode = FILE_SHARE_READ; + + if (local_options.use_mmap_writes) { + desired_access |= GENERIC_READ; + } + else { + // Adding this solely for tests to pass (fault_injection_test, + // wal_manager_test). + shared_mode |= (FILE_SHARE_WRITE | FILE_SHARE_DELETE); + } + + // This will always truncate the file + DWORD creation_disposition = CREATE_ALWAYS; + if (reopen) { + creation_disposition = OPEN_ALWAYS; + } + + HANDLE hFile = 0; + { + IOSTATS_TIMER_GUARD(open_nanos); + hFile = CreateFileA( + fname.c_str(), + desired_access, // Access desired + shared_mode, + NULL, // Security attributes + creation_disposition, // Posix env says (reopen) ? (O_CREATE | O_APPEND) : O_CREAT | O_TRUNC + fileFlags, // Flags + NULL); // Template File + } + + if (INVALID_HANDLE_VALUE == hFile) { + auto lastError = GetLastError(); + return IOErrorFromWindowsError( + "Failed to create a NewWriteableFile: " + fname, lastError); + } + + // We will start writing at the end, appending + if (reopen) { + LARGE_INTEGER zero_move; + zero_move.QuadPart = 0; + BOOL ret = SetFilePointerEx(hFile, zero_move, NULL, FILE_END); + if (!ret) { + auto lastError = GetLastError(); + return IOErrorFromWindowsError( + "Failed to create a ReopenWritableFile move to the end: " + fname, lastError); + } + } + + if (options.use_mmap_writes) { + // We usually do not use mmmapping on SSD and thus we pass memory + // page_size + result->reset(new WinMmapFile(fname, hFile, page_size_, + allocation_granularity_, local_options)); + } else { + // Here we want the buffer allocation to be aligned by the SSD page size + // and to be a multiple of it + result->reset(new WinWritableFile(fname, hFile, page_size_, + c_BufferCapacity, local_options)); + } + return s; +} + +Status WinEnvIO::NewRandomRWFile(const std::string & fname, + std::unique_ptr* result, const EnvOptions & options) { + + Status s; + + // Open the file for read-only random access + // Random access is to disable read-ahead as the system reads too much data + DWORD desired_access = GENERIC_READ | GENERIC_WRITE; + DWORD shared_mode = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE; + DWORD creation_disposition = OPEN_ALWAYS; // Create if necessary or open existing + DWORD file_flags = FILE_FLAG_RANDOM_ACCESS; + + if (options.use_direct_reads && options.use_direct_writes) { + file_flags |= FILE_FLAG_NO_BUFFERING; + } + + /// Shared access is necessary for corruption test to pass + // almost all tests would work with a possible exception of fault_injection + HANDLE hFile = 0; + { + IOSTATS_TIMER_GUARD(open_nanos); + hFile = + CreateFileA(fname.c_str(), + desired_access, + shared_mode, + NULL, // Security attributes + creation_disposition, + file_flags, + NULL); + } + + if (INVALID_HANDLE_VALUE == hFile) { + auto lastError = GetLastError(); + return IOErrorFromWindowsError( + "NewRandomRWFile failed to Create/Open: " + fname, lastError); + } + + UniqueCloseHandlePtr fileGuard(hFile, CloseHandleFunc); + result->reset(new WinRandomRWFile(fname, hFile, page_size_, options)); + fileGuard.release(); + + return s; +} + +Status WinEnvIO::NewDirectory(const std::string& name, + std::unique_ptr* result) { + Status s; + // Must be nullptr on failure + result->reset(); + // Must fail if directory does not exist + if (!DirExists(name)) { + s = IOError("Directory does not exist: " + name, EEXIST); + } else { + IOSTATS_TIMER_GUARD(open_nanos); + result->reset(new WinDirectory); + } + return s; +} + +Status WinEnvIO::FileExists(const std::string& fname) { + // F_OK == 0 + const int F_OK_ = 0; + return _access(fname.c_str(), F_OK_) == 0 ? Status::OK() + : Status::NotFound(); +} + +Status WinEnvIO::GetChildren(const std::string& dir, + std::vector* result) { + + result->clear(); + std::vector output; + + Status status; + + auto CloseDir = [](DIR* p) { closedir(p); }; + std::unique_ptr dirp(opendir(dir.c_str()), + CloseDir); + + if (!dirp) { + switch (errno) { + case EACCES: + case ENOENT: + case ENOTDIR: + return Status::NotFound(); + default: + return IOError(dir, errno); + } + } else { + if (result->capacity() > 0) { + output.reserve(result->capacity()); + } + + struct dirent* ent = readdir(dirp.get()); + while (ent) { + output.push_back(ent->d_name); + ent = readdir(dirp.get()); + } + } + + output.swap(*result); + + return status; +} + +Status WinEnvIO::CreateDir(const std::string& name) { + Status result; + + if (_mkdir(name.c_str()) != 0) { + auto code = errno; + result = IOError("Failed to create dir: " + name, code); + } + + return result; +} + +Status WinEnvIO::CreateDirIfMissing(const std::string& name) { + Status result; + + if (DirExists(name)) { + return result; + } + + if (_mkdir(name.c_str()) != 0) { + if (errno == EEXIST) { + result = + Status::IOError("`" + name + "' exists but is not a directory"); + } else { + auto code = errno; + result = IOError("Failed to create dir: " + name, code); + } + } + + return result; +} + +Status WinEnvIO::DeleteDir(const std::string& name) { + Status result; + if (_rmdir(name.c_str()) != 0) { + auto code = errno; + result = IOError("Failed to remove dir: " + name, code); + } + return result; +} + +Status WinEnvIO::GetFileSize(const std::string& fname, + uint64_t* size) { + Status s; + + WIN32_FILE_ATTRIBUTE_DATA attrs; + if (GetFileAttributesExA(fname.c_str(), GetFileExInfoStandard, &attrs)) { + ULARGE_INTEGER file_size; + file_size.HighPart = attrs.nFileSizeHigh; + file_size.LowPart = attrs.nFileSizeLow; + *size = file_size.QuadPart; + } else { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError("Can not get size for: " + fname, lastError); + } + return s; +} + +uint64_t WinEnvIO::FileTimeToUnixTime(const FILETIME& ftTime) { + const uint64_t c_FileTimePerSecond = 10000000U; + // UNIX epoch starts on 1970-01-01T00:00:00Z + // Windows FILETIME starts on 1601-01-01T00:00:00Z + // Therefore, we need to subtract the below number of seconds from + // the seconds that we obtain from FILETIME with an obvious loss of + // precision + const uint64_t c_SecondBeforeUnixEpoch = 11644473600U; + + ULARGE_INTEGER li; + li.HighPart = ftTime.dwHighDateTime; + li.LowPart = ftTime.dwLowDateTime; + + uint64_t result = + (li.QuadPart / c_FileTimePerSecond) - c_SecondBeforeUnixEpoch; + return result; +} + +Status WinEnvIO::GetFileModificationTime(const std::string& fname, + uint64_t* file_mtime) { + Status s; + + WIN32_FILE_ATTRIBUTE_DATA attrs; + if (GetFileAttributesExA(fname.c_str(), GetFileExInfoStandard, &attrs)) { + *file_mtime = FileTimeToUnixTime(attrs.ftLastWriteTime); + } else { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError( + "Can not get file modification time for: " + fname, lastError); + *file_mtime = 0; + } + + return s; +} + +Status WinEnvIO::RenameFile(const std::string& src, + const std::string& target) { + Status result; + + // rename() is not capable of replacing the existing file as on Linux + // so use OS API directly + if (!MoveFileExA(src.c_str(), target.c_str(), MOVEFILE_REPLACE_EXISTING)) { + DWORD lastError = GetLastError(); + + std::string text("Failed to rename: "); + text.append(src).append(" to: ").append(target); + + result = IOErrorFromWindowsError(text, lastError); + } + + return result; +} + +Status WinEnvIO::LinkFile(const std::string& src, + const std::string& target) { + Status result; + + if (!CreateHardLinkA(target.c_str(), src.c_str(), NULL)) { + DWORD lastError = GetLastError(); + + std::string text("Failed to link: "); + text.append(src).append(" to: ").append(target); + + result = IOErrorFromWindowsError(text, lastError); + } + + return result; +} + +Status WinEnvIO::LockFile(const std::string& lockFname, + FileLock** lock) { + assert(lock != nullptr); + + *lock = NULL; + Status result; + + // No-sharing, this is a LOCK file + const DWORD ExclusiveAccessON = 0; + + // Obtain exclusive access to the LOCK file + // Previously, instead of NORMAL attr we set DELETE on close and that worked + // well except with fault_injection test that insists on deleting it. + HANDLE hFile = 0; + { + IOSTATS_TIMER_GUARD(open_nanos); + hFile = CreateFileA(lockFname.c_str(), (GENERIC_READ | GENERIC_WRITE), + ExclusiveAccessON, NULL, CREATE_ALWAYS, + FILE_ATTRIBUTE_NORMAL, NULL); + } + + if (INVALID_HANDLE_VALUE == hFile) { + auto lastError = GetLastError(); + result = IOErrorFromWindowsError( + "Failed to create lock file: " + lockFname, lastError); + } else { + *lock = new WinFileLock(hFile); + } + + return result; +} + +Status WinEnvIO::UnlockFile(FileLock* lock) { + Status result; + + assert(lock != nullptr); + + delete lock; + + return result; +} + +Status WinEnvIO::GetTestDirectory(std::string* result) { + std::string output; + + const char* env = getenv("TEST_TMPDIR"); + if (env && env[0] != '\0') { + output = env; + CreateDir(output); + } else { + env = getenv("TMP"); + + if (env && env[0] != '\0') { + output = env; + } else { + output = "c:\\tmp"; + } + + CreateDir(output); + } + + output.append("\\testrocksdb-"); + output.append(std::to_string(_getpid())); + + CreateDir(output); + + output.swap(*result); + + return Status::OK(); +} + +Status WinEnvIO::NewLogger(const std::string& fname, + std::shared_ptr* result) { + Status s; + + result->reset(); + + HANDLE hFile = 0; + { + IOSTATS_TIMER_GUARD(open_nanos); + hFile = CreateFileA( + fname.c_str(), GENERIC_WRITE, + FILE_SHARE_READ | FILE_SHARE_DELETE, // In RocksDb log files are + // renamed and deleted before + // they are closed. This enables + // doing so. + NULL, + CREATE_ALWAYS, // Original fopen mode is "w" + FILE_ATTRIBUTE_NORMAL, NULL); + } + + if (INVALID_HANDLE_VALUE == hFile) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError("Failed to open LogFile" + fname, lastError); + } else { + { + // With log files we want to set the true creation time as of now + // because the system + // for some reason caches the attributes of the previous file that just + // been renamed from + // this name so auto_roll_logger_test fails + FILETIME ft; + GetSystemTimeAsFileTime(&ft); + // Set creation, last access and last write time to the same value + SetFileTime(hFile, &ft, &ft, &ft); + } + result->reset(new WinLogger(&WinEnvThreads::gettid, hosted_env_, hFile)); + } + return s; +} + +uint64_t WinEnvIO::NowMicros() { + + if (GetSystemTimePreciseAsFileTime_ != NULL) { + // all std::chrono clocks on windows proved to return + // values that may repeat that is not good enough for some uses. + const int64_t c_UnixEpochStartTicks = 116444736000000000LL; + const int64_t c_FtToMicroSec = 10; + + // This interface needs to return system time and not + // just any microseconds because it is often used as an argument + // to TimedWait() on condition variable + FILETIME ftSystemTime; + GetSystemTimePreciseAsFileTime_(&ftSystemTime); + + LARGE_INTEGER li; + li.LowPart = ftSystemTime.dwLowDateTime; + li.HighPart = ftSystemTime.dwHighDateTime; + // Subtract unix epoch start + li.QuadPart -= c_UnixEpochStartTicks; + // Convert to microsecs + li.QuadPart /= c_FtToMicroSec; + return li.QuadPart; + } + using namespace std::chrono; + return duration_cast(system_clock::now().time_since_epoch()).count(); +} + +uint64_t WinEnvIO::NowNanos() { + // all std::chrono clocks on windows have the same resolution that is only + // good enough for microseconds but not nanoseconds + // On Windows 8 and Windows 2012 Server + // GetSystemTimePreciseAsFileTime(¤t_time) can be used + LARGE_INTEGER li; + QueryPerformanceCounter(&li); + // Convert to nanoseconds first to avoid loss of precision + // and divide by frequency + li.QuadPart *= std::nano::den; + li.QuadPart /= perf_counter_frequency_; + return li.QuadPart; +} + +Status WinEnvIO::GetHostName(char* name, uint64_t len) { + Status s; + DWORD nSize = static_cast( + std::min(len, std::numeric_limits::max())); + + if (!::GetComputerNameA(name, &nSize)) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError("GetHostName", lastError); + } else { + name[nSize] = 0; + } + + return s; +} + +Status WinEnvIO::GetAbsolutePath(const std::string& db_path, + std::string* output_path) { + // Check if we already have an absolute path + // that starts with non dot and has a semicolon in it + if ((!db_path.empty() && (db_path[0] == '/' || db_path[0] == '\\')) || + (db_path.size() > 2 && db_path[0] != '.' && + ((db_path[1] == ':' && db_path[2] == '\\') || + (db_path[1] == ':' && db_path[2] == '/')))) { + *output_path = db_path; + return Status::OK(); + } + + std::string result; + result.resize(_MAX_PATH); + + char* ret = _getcwd(&result[0], _MAX_PATH); + if (ret == nullptr) { + return Status::IOError("Failed to get current working directory", + strerror(errno)); + } + + result.resize(strlen(result.data())); + + result.swap(*output_path); + return Status::OK(); +} + +std::string WinEnvIO::TimeToString(uint64_t secondsSince1970) { + std::string result; + + const time_t seconds = secondsSince1970; + const int maxsize = 64; + + struct tm t; + errno_t ret = localtime_s(&t, &seconds); + + if (ret) { + result = std::to_string(seconds); + } else { + result.resize(maxsize); + char* p = &result[0]; + + int len = snprintf(p, maxsize, "%04d/%02d/%02d-%02d:%02d:%02d ", + t.tm_year + 1900, t.tm_mon + 1, t.tm_mday, t.tm_hour, + t.tm_min, t.tm_sec); + assert(len > 0); + + result.resize(len); + } + + return result; +} + +EnvOptions WinEnvIO::OptimizeForLogWrite(const EnvOptions& env_options, + const DBOptions& db_options) const { + EnvOptions optimized = env_options; + optimized.bytes_per_sync = db_options.wal_bytes_per_sync; + optimized.use_mmap_writes = false; + // This is because we flush only whole pages on unbuffered io and + // the last records are not guaranteed to be flushed. + optimized.use_direct_writes = false; + // TODO(icanadi) it's faster if fallocate_with_keep_size is false, but it + // breaks TransactionLogIteratorStallAtLastRecord unit test. Fix the unit + // test and make this false + optimized.fallocate_with_keep_size = true; + return optimized; +} + +EnvOptions WinEnvIO::OptimizeForManifestWrite( + const EnvOptions& env_options) const { + EnvOptions optimized = env_options; + optimized.use_mmap_writes = false; + optimized.use_direct_writes = false; + optimized.fallocate_with_keep_size = true; + return optimized; +} + +// Returns true iff the named directory exists and is a directory. +bool WinEnvIO::DirExists(const std::string& dname) { + WIN32_FILE_ATTRIBUTE_DATA attrs; + if (GetFileAttributesExA(dname.c_str(), GetFileExInfoStandard, &attrs)) { + return 0 != (attrs.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY); + } + return false; +} + +//////////////////////////////////////////////////////////////////////// +// WinEnvThreads + +WinEnvThreads::WinEnvThreads(Env* hosted_env) : hosted_env_(hosted_env), thread_pools_(Env::Priority::TOTAL) { + + for (int pool_id = 0; pool_id < Env::Priority::TOTAL; ++pool_id) { + thread_pools_[pool_id].SetThreadPriority( + static_cast(pool_id)); + // This allows later initializing the thread-local-env of each thread. + thread_pools_[pool_id].SetHostEnv(hosted_env); + } +} + +WinEnvThreads::~WinEnvThreads() { + + WaitForJoin(); + + for (auto& thpool : thread_pools_) { + thpool.JoinAllThreads(); + } +} + +void WinEnvThreads::Schedule(void(*function)(void*), void* arg, Env::Priority pri, + void* tag, void(*unschedFunction)(void* arg)) { + assert(pri >= Env::Priority::BOTTOM && pri <= Env::Priority::HIGH); + thread_pools_[pri].Schedule(function, arg, tag, unschedFunction); +} + +int WinEnvThreads::UnSchedule(void* arg, Env::Priority pri) { + return thread_pools_[pri].UnSchedule(arg); +} + +namespace { + + struct StartThreadState { + void(*user_function)(void*); + void* arg; + }; + + void* StartThreadWrapper(void* arg) { + std::unique_ptr state( + reinterpret_cast(arg)); + state->user_function(state->arg); + return nullptr; + } + +} + +void WinEnvThreads::StartThread(void(*function)(void* arg), void* arg) { + std::unique_ptr state(new StartThreadState); + state->user_function = function; + state->arg = arg; + try { + + rocksdb::port::WindowsThread th(&StartThreadWrapper, state.get()); + state.release(); + + std::lock_guard lg(mu_); + threads_to_join_.push_back(std::move(th)); + + } catch (const std::system_error& ex) { + WinthreadCall("start thread", ex.code()); + } +} + +void WinEnvThreads::WaitForJoin() { + for (auto& th : threads_to_join_) { + th.join(); + } + threads_to_join_.clear(); +} + +unsigned int WinEnvThreads::GetThreadPoolQueueLen(Env::Priority pri) const { + assert(pri >= Env::Priority::BOTTOM && pri <= Env::Priority::HIGH); + return thread_pools_[pri].GetQueueLen(); +} + +uint64_t WinEnvThreads::gettid() { + uint64_t thread_id = GetCurrentThreadId(); + return thread_id; +} + +uint64_t WinEnvThreads::GetThreadID() const { return gettid(); } + +void WinEnvThreads::SleepForMicroseconds(int micros) { + std::this_thread::sleep_for(std::chrono::microseconds(micros)); +} + +void WinEnvThreads::SetBackgroundThreads(int num, Env::Priority pri) { + assert(pri >= Env::Priority::BOTTOM && pri <= Env::Priority::HIGH); + thread_pools_[pri].SetBackgroundThreads(num); +} + +int WinEnvThreads::GetBackgroundThreads(Env::Priority pri) { + assert(pri >= Env::Priority::BOTTOM && pri <= Env::Priority::HIGH); + return thread_pools_[pri].GetBackgroundThreads(); +} + +void WinEnvThreads::IncBackgroundThreadsIfNeeded(int num, Env::Priority pri) { + assert(pri >= Env::Priority::BOTTOM && pri <= Env::Priority::HIGH); + thread_pools_[pri].IncBackgroundThreadsIfNeeded(num); +} + +///////////////////////////////////////////////////////////////////////// +// WinEnv + +WinEnv::WinEnv() : winenv_io_(this), winenv_threads_(this) { + // Protected member of the base class + thread_status_updater_ = CreateThreadStatusUpdater(); +} + + +WinEnv::~WinEnv() { + // All threads must be joined before the deletion of + // thread_status_updater_. + delete thread_status_updater_; +} + +Status WinEnv::GetThreadList( + std::vector* thread_list) { + assert(thread_status_updater_); + return thread_status_updater_->GetThreadList(thread_list); +} + +Status WinEnv::DeleteFile(const std::string& fname) { + return winenv_io_.DeleteFile(fname); +} + +Status WinEnv::GetCurrentTime(int64_t* unix_time) { + return winenv_io_.GetCurrentTime(unix_time); +} + +Status WinEnv::NewSequentialFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) { + return winenv_io_.NewSequentialFile(fname, result, options); +} + +Status WinEnv::NewRandomAccessFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) { + return winenv_io_.NewRandomAccessFile(fname, result, options); +} + +Status WinEnv::NewWritableFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) { + return winenv_io_.OpenWritableFile(fname, result, options, false); +} + +Status WinEnv::ReopenWritableFile(const std::string& fname, + std::unique_ptr* result, const EnvOptions& options) { + return winenv_io_.OpenWritableFile(fname, result, options, true); +} + +Status WinEnv::NewRandomRWFile(const std::string & fname, + unique_ptr* result, const EnvOptions & options) { + return winenv_io_.NewRandomRWFile(fname, result, options); +} + +Status WinEnv::NewDirectory(const std::string& name, + std::unique_ptr* result) { + return winenv_io_.NewDirectory(name, result); +} + +Status WinEnv::FileExists(const std::string& fname) { + return winenv_io_.FileExists(fname); +} + +Status WinEnv::GetChildren(const std::string& dir, + std::vector* result) { + return winenv_io_.GetChildren(dir, result); +} + +Status WinEnv::CreateDir(const std::string& name) { + return winenv_io_.CreateDir(name); +} + +Status WinEnv::CreateDirIfMissing(const std::string& name) { + return winenv_io_.CreateDirIfMissing(name); +} + +Status WinEnv::DeleteDir(const std::string& name) { + return winenv_io_.DeleteDir(name); +} + +Status WinEnv::GetFileSize(const std::string& fname, + uint64_t* size) { + return winenv_io_.GetFileSize(fname, size); +} + +Status WinEnv::GetFileModificationTime(const std::string& fname, + uint64_t* file_mtime) { + return winenv_io_.GetFileModificationTime(fname, file_mtime); +} + +Status WinEnv::RenameFile(const std::string& src, + const std::string& target) { + return winenv_io_.RenameFile(src, target); +} + +Status WinEnv::LinkFile(const std::string& src, + const std::string& target) { + return winenv_io_.LinkFile(src, target); +} + +Status WinEnv::LockFile(const std::string& lockFname, + FileLock** lock) { + return winenv_io_.LockFile(lockFname, lock); +} + +Status WinEnv::UnlockFile(FileLock* lock) { + return winenv_io_.UnlockFile(lock); +} + +Status WinEnv::GetTestDirectory(std::string* result) { + return winenv_io_.GetTestDirectory(result); +} + +Status WinEnv::NewLogger(const std::string& fname, + std::shared_ptr* result) { + return winenv_io_.NewLogger(fname, result); +} + +uint64_t WinEnv::NowMicros() { + return winenv_io_.NowMicros(); +} + +uint64_t WinEnv::NowNanos() { + return winenv_io_.NowNanos(); +} + +Status WinEnv::GetHostName(char* name, uint64_t len) { + return winenv_io_.GetHostName(name, len); +} + +Status WinEnv::GetAbsolutePath(const std::string& db_path, + std::string* output_path) { + return winenv_io_.GetAbsolutePath(db_path, output_path); +} + +std::string WinEnv::TimeToString(uint64_t secondsSince1970) { + return winenv_io_.TimeToString(secondsSince1970); +} + +void WinEnv::Schedule(void(*function)(void*), void* arg, Env::Priority pri, + void* tag, + void(*unschedFunction)(void* arg)) { + return winenv_threads_.Schedule(function, arg, pri, tag, unschedFunction); +} + +int WinEnv::UnSchedule(void* arg, Env::Priority pri) { + return winenv_threads_.UnSchedule(arg, pri); +} + +void WinEnv::StartThread(void(*function)(void* arg), void* arg) { + return winenv_threads_.StartThread(function, arg); +} + +void WinEnv::WaitForJoin() { + return winenv_threads_.WaitForJoin(); +} + +unsigned int WinEnv::GetThreadPoolQueueLen(Env::Priority pri) const { + return winenv_threads_.GetThreadPoolQueueLen(pri); +} + +uint64_t WinEnv::GetThreadID() const { + return winenv_threads_.GetThreadID(); +} + +void WinEnv::SleepForMicroseconds(int micros) { + return winenv_threads_.SleepForMicroseconds(micros); +} + +// Allow increasing the number of worker threads. +void WinEnv::SetBackgroundThreads(int num, Env::Priority pri) { + return winenv_threads_.SetBackgroundThreads(num, pri); +} + +int WinEnv::GetBackgroundThreads(Env::Priority pri) { + return winenv_threads_.GetBackgroundThreads(pri); +} + +void WinEnv::IncBackgroundThreadsIfNeeded(int num, Env::Priority pri) { + return winenv_threads_.IncBackgroundThreadsIfNeeded(num, pri); +} + +EnvOptions WinEnv::OptimizeForLogWrite(const EnvOptions& env_options, + const DBOptions& db_options) const { + return winenv_io_.OptimizeForLogWrite(env_options, db_options); +} + +EnvOptions WinEnv::OptimizeForManifestWrite( + const EnvOptions& env_options) const { + return winenv_io_.OptimizeForManifestWrite(env_options); +} + +} // namespace port + +std::string Env::GenerateUniqueId() { + std::string result; + + UUID uuid; + UuidCreateSequential(&uuid); + + RPC_CSTR rpc_str; + auto status = UuidToStringA(&uuid, &rpc_str); + (void)status; + assert(status == RPC_S_OK); + + result = reinterpret_cast(rpc_str); + + status = RpcStringFreeA(&rpc_str); + assert(status == RPC_S_OK); + + return result; +} + +} // namespace rocksdb diff --git a/port/win/env_win.h b/port/win/env_win.h new file mode 100644 index 00000000000..ce1a61d4161 --- /dev/null +++ b/port/win/env_win.h @@ -0,0 +1,310 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// An Env is an interface used by the rocksdb implementation to access +// operating system functionality like the filesystem etc. Callers +// may wish to provide a custom Env object when opening a database to +// get fine gain control; e.g., to rate limit file system operations. +// +// All Env implementations are safe for concurrent access from +// multiple threads without any external synchronization. + +#pragma once + +#include "port/win/win_thread.h" +#include +#include "util/threadpool_imp.h" + +#include +#include + +#include +#include +#include + + +#undef GetCurrentTime +#undef DeleteFile +#undef GetTickCount + +namespace rocksdb { +namespace port { + +// Currently not designed for inheritance but rather a replacement +class WinEnvThreads { +public: + + explicit WinEnvThreads(Env* hosted_env); + + ~WinEnvThreads(); + + WinEnvThreads(const WinEnvThreads&) = delete; + WinEnvThreads& operator=(const WinEnvThreads&) = delete; + + void Schedule(void(*function)(void*), void* arg, Env::Priority pri, + void* tag, + void(*unschedFunction)(void* arg)); + + int UnSchedule(void* arg, Env::Priority pri); + + void StartThread(void(*function)(void* arg), void* arg); + + void WaitForJoin(); + + unsigned int GetThreadPoolQueueLen(Env::Priority pri) const; + + static uint64_t gettid(); + + uint64_t GetThreadID() const; + + void SleepForMicroseconds(int micros); + + // Allow increasing the number of worker threads. + void SetBackgroundThreads(int num, Env::Priority pri); + int GetBackgroundThreads(Env::Priority pri); + + void IncBackgroundThreadsIfNeeded(int num, Env::Priority pri); + +private: + + Env* hosted_env_; + mutable std::mutex mu_; + std::vector thread_pools_; + std::vector threads_to_join_; + +}; + +// Designed for inheritance so can be re-used +// but certain parts replaced +class WinEnvIO { +public: + explicit WinEnvIO(Env* hosted_env); + + virtual ~WinEnvIO(); + + virtual Status DeleteFile(const std::string& fname); + + virtual Status GetCurrentTime(int64_t* unix_time); + + virtual Status NewSequentialFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options); + + // Helper for NewWritable and ReopenWritableFile + virtual Status OpenWritableFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options, + bool reopen); + + virtual Status NewRandomAccessFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options); + + // The returned file will only be accessed by one thread at a time. + virtual Status NewRandomRWFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options); + + virtual Status NewDirectory(const std::string& name, + std::unique_ptr* result); + + virtual Status FileExists(const std::string& fname); + + virtual Status GetChildren(const std::string& dir, + std::vector* result); + + virtual Status CreateDir(const std::string& name); + + virtual Status CreateDirIfMissing(const std::string& name); + + virtual Status DeleteDir(const std::string& name); + + virtual Status GetFileSize(const std::string& fname, + uint64_t* size); + + static uint64_t FileTimeToUnixTime(const FILETIME& ftTime); + + virtual Status GetFileModificationTime(const std::string& fname, + uint64_t* file_mtime); + + virtual Status RenameFile(const std::string& src, + const std::string& target); + + virtual Status LinkFile(const std::string& src, + const std::string& target); + + virtual Status LockFile(const std::string& lockFname, + FileLock** lock); + + virtual Status UnlockFile(FileLock* lock); + + virtual Status GetTestDirectory(std::string* result); + + virtual Status NewLogger(const std::string& fname, + std::shared_ptr* result); + + virtual uint64_t NowMicros(); + + virtual uint64_t NowNanos(); + + virtual Status GetHostName(char* name, uint64_t len); + + virtual Status GetAbsolutePath(const std::string& db_path, + std::string* output_path); + + virtual std::string TimeToString(uint64_t secondsSince1970); + + virtual EnvOptions OptimizeForLogWrite(const EnvOptions& env_options, + const DBOptions& db_options) const; + + virtual EnvOptions OptimizeForManifestWrite( + const EnvOptions& env_options) const; + + size_t GetPageSize() const { return page_size_; } + + size_t GetAllocationGranularity() const { return allocation_granularity_; } + + uint64_t GetPerfCounterFrequency() const { return perf_counter_frequency_; } + +private: + // Returns true iff the named directory exists and is a directory. + virtual bool DirExists(const std::string& dname); + + typedef VOID(WINAPI * FnGetSystemTimePreciseAsFileTime)(LPFILETIME); + + Env* hosted_env_; + size_t page_size_; + size_t allocation_granularity_; + uint64_t perf_counter_frequency_; + FnGetSystemTimePreciseAsFileTime GetSystemTimePreciseAsFileTime_; +}; + +class WinEnv : public Env { +public: + WinEnv(); + + ~WinEnv(); + + Status DeleteFile(const std::string& fname) override; + + Status GetCurrentTime(int64_t* unix_time) override; + + Status NewSequentialFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) override; + + Status NewRandomAccessFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) override; + + Status NewWritableFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) override; + + // Create an object that writes to a new file with the specified + // name. Deletes any existing file with the same name and creates a + // new file. On success, stores a pointer to the new file in + // *result and returns OK. On failure stores nullptr in *result and + // returns non-OK. + // + // The returned file will only be accessed by one thread at a time. + Status ReopenWritableFile(const std::string& fname, + std::unique_ptr* result, + const EnvOptions& options) override; + + // The returned file will only be accessed by one thread at a time. + Status NewRandomRWFile(const std::string& fname, + unique_ptr* result, + const EnvOptions& options) override; + + Status NewDirectory(const std::string& name, + std::unique_ptr* result) override; + + Status FileExists(const std::string& fname) override; + + Status GetChildren(const std::string& dir, + std::vector* result) override; + + Status CreateDir(const std::string& name) override; + + Status CreateDirIfMissing(const std::string& name) override; + + Status DeleteDir(const std::string& name) override; + + Status GetFileSize(const std::string& fname, + uint64_t* size) override; + + Status GetFileModificationTime(const std::string& fname, + uint64_t* file_mtime) override; + + Status RenameFile(const std::string& src, + const std::string& target) override; + + Status LinkFile(const std::string& src, + const std::string& target) override; + + Status LockFile(const std::string& lockFname, + FileLock** lock) override; + + Status UnlockFile(FileLock* lock) override; + + Status GetTestDirectory(std::string* result) override; + + Status NewLogger(const std::string& fname, + std::shared_ptr* result) override; + + uint64_t NowMicros() override; + + uint64_t NowNanos() override; + + Status GetHostName(char* name, uint64_t len) override; + + Status GetAbsolutePath(const std::string& db_path, + std::string* output_path) override; + + std::string TimeToString(uint64_t secondsSince1970) override; + + Status GetThreadList( + std::vector* thread_list) override; + + void Schedule(void(*function)(void*), void* arg, Env::Priority pri, + void* tag, + void(*unschedFunction)(void* arg)) override; + + int UnSchedule(void* arg, Env::Priority pri) override; + + void StartThread(void(*function)(void* arg), void* arg) override; + + void WaitForJoin(); + + unsigned int GetThreadPoolQueueLen(Env::Priority pri) const override; + + uint64_t GetThreadID() const override; + + void SleepForMicroseconds(int micros) override; + + // Allow increasing the number of worker threads. + void SetBackgroundThreads(int num, Env::Priority pri) override; + int GetBackgroundThreads(Env::Priority pri) override; + + void IncBackgroundThreadsIfNeeded(int num, Env::Priority pri) override; + + EnvOptions OptimizeForLogWrite(const EnvOptions& env_options, + const DBOptions& db_options) const override; + + EnvOptions OptimizeForManifestWrite( + const EnvOptions& env_options) const override; + +private: + + WinEnvIO winenv_io_; + WinEnvThreads winenv_threads_; +}; + +} // namespace port +} // namespace rocksdb diff --git a/port/win/io_win.cc b/port/win/io_win.cc new file mode 100644 index 00000000000..3d2533a2efe --- /dev/null +++ b/port/win/io_win.cc @@ -0,0 +1,1029 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "port/win/io_win.h" + +#include "monitoring/iostats_context_imp.h" +#include "util/aligned_buffer.h" +#include "util/coding.h" +#include "util/sync_point.h" + +namespace rocksdb { +namespace port { + +/* +* DirectIOHelper +*/ +namespace { + +const size_t kSectorSize = 512; + +inline +bool IsPowerOfTwo(const size_t alignment) { + return ((alignment) & (alignment - 1)) == 0; +} + +inline +bool IsSectorAligned(const size_t off) { + return (off & (kSectorSize - 1)) == 0; +} + +inline +bool IsAligned(size_t alignment, const void* ptr) { + return ((uintptr_t(ptr)) & (alignment - 1)) == 0; +} +} + + +std::string GetWindowsErrSz(DWORD err) { + LPSTR lpMsgBuf; + FormatMessageA(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | + FORMAT_MESSAGE_IGNORE_INSERTS, + NULL, err, + 0, // Default language + reinterpret_cast(&lpMsgBuf), 0, NULL); + + std::string Err = lpMsgBuf; + LocalFree(lpMsgBuf); + return Err; +} + +// We preserve the original name of this interface to denote the original idea +// behind it. +// All reads happen by a specified offset and pwrite interface does not change +// the position of the file pointer. Judging from the man page and errno it does +// execute +// lseek atomically to return the position of the file back where it was. +// WriteFile() does not +// have this capability. Therefore, for both pread and pwrite the pointer is +// advanced to the next position +// which is fine for writes because they are (should be) sequential. +// Because all the reads/writes happen by the specified offset, the caller in +// theory should not +// rely on the current file offset. +SSIZE_T pwrite(HANDLE hFile, const char* src, size_t numBytes, + uint64_t offset) { + assert(numBytes <= std::numeric_limits::max()); + OVERLAPPED overlapped = { 0 }; + ULARGE_INTEGER offsetUnion; + offsetUnion.QuadPart = offset; + + overlapped.Offset = offsetUnion.LowPart; + overlapped.OffsetHigh = offsetUnion.HighPart; + + SSIZE_T result = 0; + + unsigned long bytesWritten = 0; + + if (FALSE == WriteFile(hFile, src, static_cast(numBytes), &bytesWritten, + &overlapped)) { + result = -1; + } else { + result = bytesWritten; + } + + return result; +} + +// See comments for pwrite above +SSIZE_T pread(HANDLE hFile, char* src, size_t numBytes, uint64_t offset) { + assert(numBytes <= std::numeric_limits::max()); + OVERLAPPED overlapped = { 0 }; + ULARGE_INTEGER offsetUnion; + offsetUnion.QuadPart = offset; + + overlapped.Offset = offsetUnion.LowPart; + overlapped.OffsetHigh = offsetUnion.HighPart; + + SSIZE_T result = 0; + + unsigned long bytesRead = 0; + + if (FALSE == ReadFile(hFile, src, static_cast(numBytes), &bytesRead, + &overlapped)) { + return -1; + } else { + result = bytesRead; + } + + return result; +} + +// SetFileInformationByHandle() is capable of fast pre-allocates. +// However, this does not change the file end position unless the file is +// truncated and the pre-allocated space is not considered filled with zeros. +Status fallocate(const std::string& filename, HANDLE hFile, + uint64_t to_size) { + Status status; + + FILE_ALLOCATION_INFO alloc_info; + alloc_info.AllocationSize.QuadPart = to_size; + + if (!SetFileInformationByHandle(hFile, FileAllocationInfo, &alloc_info, + sizeof(FILE_ALLOCATION_INFO))) { + auto lastError = GetLastError(); + status = IOErrorFromWindowsError( + "Failed to pre-allocate space: " + filename, lastError); + } + + return status; +} + +Status ftruncate(const std::string& filename, HANDLE hFile, + uint64_t toSize) { + Status status; + + FILE_END_OF_FILE_INFO end_of_file; + end_of_file.EndOfFile.QuadPart = toSize; + + if (!SetFileInformationByHandle(hFile, FileEndOfFileInfo, &end_of_file, + sizeof(FILE_END_OF_FILE_INFO))) { + auto lastError = GetLastError(); + status = IOErrorFromWindowsError("Failed to Set end of file: " + filename, + lastError); + } + + return status; +} + +size_t GetUniqueIdFromFile(HANDLE hFile, char* id, size_t max_size) { + + if (max_size < kMaxVarint64Length * 3) { + return 0; + } + + // This function has to be re-worked for cases when + // ReFS file system introduced on Windows Server 2012 is used + BY_HANDLE_FILE_INFORMATION FileInfo; + + BOOL result = GetFileInformationByHandle(hFile, &FileInfo); + + TEST_SYNC_POINT_CALLBACK("GetUniqueIdFromFile:FS_IOC_GETVERSION", &result); + + if (!result) { + return 0; + } + + char* rid = id; + rid = EncodeVarint64(rid, uint64_t(FileInfo.dwVolumeSerialNumber)); + rid = EncodeVarint64(rid, uint64_t(FileInfo.nFileIndexHigh)); + rid = EncodeVarint64(rid, uint64_t(FileInfo.nFileIndexLow)); + + assert(rid >= id); + return static_cast(rid - id); +} + +//////////////////////////////////////////////////////////////////////////////////////////////////// +// WinMmapReadableFile + +WinMmapReadableFile::WinMmapReadableFile(const std::string& fileName, + HANDLE hFile, HANDLE hMap, + const void* mapped_region, + size_t length) + : WinFileData(fileName, hFile, false /* use_direct_io */), + hMap_(hMap), + mapped_region_(mapped_region), + length_(length) {} + +WinMmapReadableFile::~WinMmapReadableFile() { + BOOL ret = ::UnmapViewOfFile(mapped_region_); + (void)ret; + assert(ret); + + ret = ::CloseHandle(hMap_); + assert(ret); +} + +Status WinMmapReadableFile::Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + Status s; + + if (offset > length_) { + *result = Slice(); + return IOError(filename_, EINVAL); + } else if (offset + n > length_) { + n = length_ - offset; + } + *result = + Slice(reinterpret_cast(mapped_region_)+offset, n); + return s; +} + +Status WinMmapReadableFile::InvalidateCache(size_t offset, size_t length) { + return Status::OK(); +} + +size_t WinMmapReadableFile::GetUniqueId(char* id, size_t max_size) const { + return GetUniqueIdFromFile(hFile_, id, max_size); +} + +/////////////////////////////////////////////////////////////////////////////// +/// WinMmapFile + + +// Can only truncate or reserve to a sector size aligned if +// used on files that are opened with Unbuffered I/O +Status WinMmapFile::TruncateFile(uint64_t toSize) { + return ftruncate(filename_, hFile_, toSize); +} + +Status WinMmapFile::UnmapCurrentRegion() { + Status status; + + if (mapped_begin_ != nullptr) { + if (!::UnmapViewOfFile(mapped_begin_)) { + status = IOErrorFromWindowsError( + "Failed to unmap file view: " + filename_, GetLastError()); + } + + // Move on to the next portion of the file + file_offset_ += view_size_; + + // UnmapView automatically sends data to disk but not the metadata + // which is good and provides some equivalent of fdatasync() on Linux + // therefore, we donot need separate flag for metadata + mapped_begin_ = nullptr; + mapped_end_ = nullptr; + dst_ = nullptr; + + last_sync_ = nullptr; + pending_sync_ = false; + } + + return status; +} + +Status WinMmapFile::MapNewRegion() { + + Status status; + + assert(mapped_begin_ == nullptr); + + size_t minDiskSize = file_offset_ + view_size_; + + if (minDiskSize > reserved_size_) { + status = Allocate(file_offset_, view_size_); + if (!status.ok()) { + return status; + } + } + + // Need to remap + if (hMap_ == NULL || reserved_size_ > mapping_size_) { + + if (hMap_ != NULL) { + // Unmap the previous one + BOOL ret = ::CloseHandle(hMap_); + assert(ret); + hMap_ = NULL; + } + + ULARGE_INTEGER mappingSize; + mappingSize.QuadPart = reserved_size_; + + hMap_ = CreateFileMappingA( + hFile_, + NULL, // Security attributes + PAGE_READWRITE, // There is not a write only mode for mapping + mappingSize.HighPart, // Enable mapping the whole file but the actual + // amount mapped is determined by MapViewOfFile + mappingSize.LowPart, + NULL); // Mapping name + + if (NULL == hMap_) { + return IOErrorFromWindowsError( + "WindowsMmapFile failed to create file mapping for: " + filename_, + GetLastError()); + } + + mapping_size_ = reserved_size_; + } + + ULARGE_INTEGER offset; + offset.QuadPart = file_offset_; + + // View must begin at the granularity aligned offset + mapped_begin_ = reinterpret_cast( + MapViewOfFileEx(hMap_, FILE_MAP_WRITE, offset.HighPart, offset.LowPart, + view_size_, NULL)); + + if (!mapped_begin_) { + status = IOErrorFromWindowsError( + "WindowsMmapFile failed to map file view: " + filename_, + GetLastError()); + } else { + mapped_end_ = mapped_begin_ + view_size_; + dst_ = mapped_begin_; + last_sync_ = mapped_begin_; + pending_sync_ = false; + } + return status; +} + +Status WinMmapFile::PreallocateInternal(uint64_t spaceToReserve) { + return fallocate(filename_, hFile_, spaceToReserve); +} + +WinMmapFile::WinMmapFile(const std::string& fname, HANDLE hFile, size_t page_size, + size_t allocation_granularity, const EnvOptions& options) + : WinFileData(fname, hFile, false), + hMap_(NULL), + page_size_(page_size), + allocation_granularity_(allocation_granularity), + reserved_size_(0), + mapping_size_(0), + view_size_(0), + mapped_begin_(nullptr), + mapped_end_(nullptr), + dst_(nullptr), + last_sync_(nullptr), + file_offset_(0), + pending_sync_(false) { + // Allocation granularity must be obtained from GetSystemInfo() and must be + // a power of two. + assert(allocation_granularity > 0); + assert((allocation_granularity & (allocation_granularity - 1)) == 0); + + assert(page_size > 0); + assert((page_size & (page_size - 1)) == 0); + + // Only for memory mapped writes + assert(options.use_mmap_writes); + + // View size must be both the multiple of allocation_granularity AND the + // page size and the granularity is usually a multiple of a page size. + const size_t viewSize = 32 * 1024; // 32Kb similar to the Windows File Cache in buffered mode + view_size_ = Roundup(viewSize, allocation_granularity_); +} + +WinMmapFile::~WinMmapFile() { + if (hFile_) { + this->Close(); + } +} + +Status WinMmapFile::Append(const Slice& data) { + const char* src = data.data(); + size_t left = data.size(); + + while (left > 0) { + assert(mapped_begin_ <= dst_); + size_t avail = mapped_end_ - dst_; + + if (avail == 0) { + Status s = UnmapCurrentRegion(); + if (s.ok()) { + s = MapNewRegion(); + } + + if (!s.ok()) { + return s; + } + } else { + size_t n = std::min(left, avail); + memcpy(dst_, src, n); + dst_ += n; + src += n; + left -= n; + pending_sync_ = true; + } + } + + // Now make sure that the last partial page is padded with zeros if needed + size_t bytesToPad = Roundup(size_t(dst_), page_size_) - size_t(dst_); + if (bytesToPad > 0) { + memset(dst_, 0, bytesToPad); + } + + return Status::OK(); +} + +// Means Close() will properly take care of truncate +// and it does not need any additional information +Status WinMmapFile::Truncate(uint64_t size) { + return Status::OK(); +} + +Status WinMmapFile::Close() { + Status s; + + assert(NULL != hFile_); + + // We truncate to the precise size so no + // uninitialized data at the end. SetEndOfFile + // which we use does not write zeros and it is good. + uint64_t targetSize = GetFileSize(); + + if (mapped_begin_ != nullptr) { + // Sync before unmapping to make sure everything + // is on disk and there is not a lazy writing + // so we are deterministic with the tests + Sync(); + s = UnmapCurrentRegion(); + } + + if (NULL != hMap_) { + BOOL ret = ::CloseHandle(hMap_); + if (!ret && s.ok()) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError( + "Failed to Close mapping for file: " + filename_, lastError); + } + + hMap_ = NULL; + } + + if (hFile_ != NULL) { + + TruncateFile(targetSize); + + BOOL ret = ::CloseHandle(hFile_); + hFile_ = NULL; + + if (!ret && s.ok()) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError( + "Failed to close file map handle: " + filename_, lastError); + } + } + + return s; +} + +Status WinMmapFile::Flush() { return Status::OK(); } + +// Flush only data +Status WinMmapFile::Sync() { + Status s; + + // Some writes occurred since last sync + if (dst_ > last_sync_) { + assert(mapped_begin_); + assert(dst_); + assert(dst_ > mapped_begin_); + assert(dst_ < mapped_end_); + + size_t page_begin = + TruncateToPageBoundary(page_size_, last_sync_ - mapped_begin_); + size_t page_end = + TruncateToPageBoundary(page_size_, dst_ - mapped_begin_ - 1); + + // Flush only the amount of that is a multiple of pages + if (!::FlushViewOfFile(mapped_begin_ + page_begin, + (page_end - page_begin) + page_size_)) { + s = IOErrorFromWindowsError("Failed to FlushViewOfFile: " + filename_, + GetLastError()); + } else { + last_sync_ = dst_; + } + } + + return s; +} + +/** +* Flush data as well as metadata to stable storage. +*/ +Status WinMmapFile::Fsync() { + Status s = Sync(); + + // Flush metadata + if (s.ok() && pending_sync_) { + if (!::FlushFileBuffers(hFile_)) { + s = IOErrorFromWindowsError("Failed to FlushFileBuffers: " + filename_, + GetLastError()); + } + pending_sync_ = false; + } + + return s; +} + +/** +* Get the size of valid data in the file. This will not match the +* size that is returned from the filesystem because we use mmap +* to extend file by map_size every time. +*/ +uint64_t WinMmapFile::GetFileSize() { + size_t used = dst_ - mapped_begin_; + return file_offset_ + used; +} + +Status WinMmapFile::InvalidateCache(size_t offset, size_t length) { + return Status::OK(); +} + +Status WinMmapFile::Allocate(uint64_t offset, uint64_t len) { + Status status; + TEST_KILL_RANDOM("WinMmapFile::Allocate", rocksdb_kill_odds); + + // Make sure that we reserve an aligned amount of space + // since the reservation block size is driven outside so we want + // to check if we are ok with reservation here + size_t spaceToReserve = Roundup(offset + len, view_size_); + // Nothing to do + if (spaceToReserve <= reserved_size_) { + return status; + } + + IOSTATS_TIMER_GUARD(allocate_nanos); + status = PreallocateInternal(spaceToReserve); + if (status.ok()) { + reserved_size_ = spaceToReserve; + } + return status; +} + +size_t WinMmapFile::GetUniqueId(char* id, size_t max_size) const { + return GetUniqueIdFromFile(hFile_, id, max_size); +} + +////////////////////////////////////////////////////////////////////////////////// +// WinSequentialFile + +WinSequentialFile::WinSequentialFile(const std::string& fname, HANDLE f, + const EnvOptions& options) + : WinFileData(fname, f, options.use_direct_reads) {} + +WinSequentialFile::~WinSequentialFile() { + assert(hFile_ != INVALID_HANDLE_VALUE); +} + +Status WinSequentialFile::Read(size_t n, Slice* result, char* scratch) { + assert(result != nullptr && !WinFileData::use_direct_io()); + Status s; + size_t r = 0; + + // Windows ReadFile API accepts a DWORD. + // While it is possible to read in a loop if n is > UINT_MAX + // it is a highly unlikely case. + if (n > UINT_MAX) { + return IOErrorFromWindowsError(filename_, ERROR_INVALID_PARAMETER); + } + + DWORD bytesToRead = static_cast(n); //cast is safe due to the check above + DWORD bytesRead = 0; + BOOL ret = ReadFile(hFile_, scratch, bytesToRead, &bytesRead, NULL); + if (ret == TRUE) { + r = bytesRead; + } else { + return IOErrorFromWindowsError(filename_, GetLastError()); + } + + *result = Slice(scratch, r); + + return s; +} + +SSIZE_T WinSequentialFile::PositionedReadInternal(char* src, size_t numBytes, + uint64_t offset) const { + return pread(GetFileHandle(), src, numBytes, offset); +} + +Status WinSequentialFile::PositionedRead(uint64_t offset, size_t n, Slice* result, + char* scratch) { + + Status s; + + assert(WinFileData::use_direct_io()); + + // Windows ReadFile API accepts a DWORD. + // While it is possible to read in a loop if n is > UINT_MAX + // it is a highly unlikely case. + if (n > UINT_MAX) { + return IOErrorFromWindowsError(GetName(), ERROR_INVALID_PARAMETER); + } + + auto r = PositionedReadInternal(scratch, n, offset); + + if (r < 0) { + auto lastError = GetLastError(); + // Posix impl wants to treat reads from beyond + // of the file as OK. + if (lastError != ERROR_HANDLE_EOF) { + s = IOErrorFromWindowsError(GetName(), lastError); + } + } + + *result = Slice(scratch, (r < 0) ? 0 : size_t(r)); + return s; +} + + +Status WinSequentialFile::Skip(uint64_t n) { + // Can't handle more than signed max as SetFilePointerEx accepts a signed 64-bit + // integer. As such it is a highly unlikley case to have n so large. + if (n > _I64_MAX) { + return IOErrorFromWindowsError(filename_, ERROR_INVALID_PARAMETER); + } + + LARGE_INTEGER li; + li.QuadPart = static_cast(n); //cast is safe due to the check above + BOOL ret = SetFilePointerEx(hFile_, li, NULL, FILE_CURRENT); + if (ret == FALSE) { + return IOErrorFromWindowsError(filename_, GetLastError()); + } + return Status::OK(); +} + +Status WinSequentialFile::InvalidateCache(size_t offset, size_t length) { + return Status::OK(); +} + +////////////////////////////////////////////////////////////////////////////////////////////////// +/// WinRandomAccessBase + +inline +SSIZE_T WinRandomAccessImpl::PositionedReadInternal(char* src, + size_t numBytes, + uint64_t offset) const { + return pread(file_base_->GetFileHandle(), src, numBytes, offset); +} + +inline +WinRandomAccessImpl::WinRandomAccessImpl(WinFileData* file_base, + size_t alignment, + const EnvOptions& options) : + file_base_(file_base), + alignment_(alignment) { + + assert(!options.use_mmap_reads); +} + +inline +Status WinRandomAccessImpl::ReadImpl(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + + Status s; + + // Check buffer alignment + if (file_base_->use_direct_io()) { + if (!IsAligned(alignment_, scratch)) { + return Status::InvalidArgument("WinRandomAccessImpl::ReadImpl: scratch is not properly aligned"); + } + } + + if (n == 0) { + *result = Slice(scratch, 0); + return s; + } + + size_t left = n; + char* dest = scratch; + + SSIZE_T r = PositionedReadInternal(scratch, left, offset); + if (r > 0) { + left -= r; + } else if (r < 0) { + auto lastError = GetLastError(); + // Posix impl wants to treat reads from beyond + // of the file as OK. + if(lastError != ERROR_HANDLE_EOF) { + s = IOErrorFromWindowsError(file_base_->GetName(), lastError); + } + } + + *result = Slice(scratch, (r < 0) ? 0 : n - left); + + return s; +} + +/////////////////////////////////////////////////////////////////////////////////////////////////// +/// WinRandomAccessFile + +WinRandomAccessFile::WinRandomAccessFile(const std::string& fname, HANDLE hFile, + size_t alignment, + const EnvOptions& options) + : WinFileData(fname, hFile, options.use_direct_reads), + WinRandomAccessImpl(this, alignment, options) {} + +WinRandomAccessFile::~WinRandomAccessFile() { +} + +Status WinRandomAccessFile::Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + return ReadImpl(offset, n, result, scratch); +} + +Status WinRandomAccessFile::InvalidateCache(size_t offset, size_t length) { + return Status::OK(); +} + +size_t WinRandomAccessFile::GetUniqueId(char* id, size_t max_size) const { + return GetUniqueIdFromFile(GetFileHandle(), id, max_size); +} + +size_t WinRandomAccessFile::GetRequiredBufferAlignment() const { + return GetAlignment(); +} + +///////////////////////////////////////////////////////////////////////////// +// WinWritableImpl +// + +inline +Status WinWritableImpl::PreallocateInternal(uint64_t spaceToReserve) { + return fallocate(file_data_->GetName(), file_data_->GetFileHandle(), spaceToReserve); +} + +inline +WinWritableImpl::WinWritableImpl(WinFileData* file_data, size_t alignment) + : file_data_(file_data), + alignment_(alignment), + next_write_offset_(0), + reservedsize_(0) { + + // Query current position in case ReopenWritableFile is called + // This position is only important for buffered writes + // for unbuffered writes we explicitely specify the position. + LARGE_INTEGER zero_move; + zero_move.QuadPart = 0; // Do not move + LARGE_INTEGER pos; + pos.QuadPart = 0; + BOOL ret = SetFilePointerEx(file_data_->GetFileHandle(), zero_move, &pos, + FILE_CURRENT); + // Querying no supped to fail + if (ret) { + next_write_offset_ = pos.QuadPart; + } else { + assert(false); + } +} + +inline +Status WinWritableImpl::AppendImpl(const Slice& data) { + + Status s; + + assert(data.size() < std::numeric_limits::max()); + + uint64_t written = 0; + (void)written; + + if (file_data_->use_direct_io()) { + + // With no offset specified we are appending + // to the end of the file + + assert(IsSectorAligned(next_write_offset_)); + assert(IsSectorAligned(data.size())); + assert(IsAligned(GetAlignement(), data.data())); + + SSIZE_T ret = pwrite(file_data_->GetFileHandle(), data.data(), + data.size(), next_write_offset_); + + if (ret < 0) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError( + "Failed to pwrite for: " + file_data_->GetName(), lastError); + } + else { + written = ret; + } + + } else { + + DWORD bytesWritten = 0; + if (!WriteFile(file_data_->GetFileHandle(), data.data(), + static_cast(data.size()), &bytesWritten, NULL)) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError( + "Failed to WriteFile: " + file_data_->GetName(), + lastError); + } + else { + written = bytesWritten; + } + } + + if(s.ok()) { + assert(written == data.size()); + next_write_offset_ += data.size(); + } + + return s; +} + +inline +Status WinWritableImpl::PositionedAppendImpl(const Slice& data, uint64_t offset) { + + if(file_data_->use_direct_io()) { + assert(IsSectorAligned(offset)); + assert(IsSectorAligned(data.size())); + assert(IsAligned(GetAlignement(), data.data())); + } + + Status s; + + SSIZE_T ret = pwrite(file_data_->GetFileHandle(), data.data(), data.size(), offset); + + // Error break + if (ret < 0) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError( + "Failed to pwrite for: " + file_data_->GetName(), lastError); + } + else { + assert(size_t(ret) == data.size()); + // For sequential write this would be simple + // size extension by data.size() + uint64_t write_end = offset + data.size(); + if (write_end >= next_write_offset_) { + next_write_offset_ = write_end; + } + } + return s; +} + +// Need to implement this so the file is truncated correctly +// when buffered and unbuffered mode +inline +Status WinWritableImpl::TruncateImpl(uint64_t size) { + Status s = ftruncate(file_data_->GetName(), file_data_->GetFileHandle(), + size); + if (s.ok()) { + next_write_offset_ = size; + } + return s; +} + +inline +Status WinWritableImpl::CloseImpl() { + + Status s; + + auto hFile = file_data_->GetFileHandle(); + assert(INVALID_HANDLE_VALUE != hFile); + + if (fsync(hFile) < 0) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError("fsync failed at Close() for: " + + file_data_->GetName(), + lastError); + } + + if(!file_data_->CloseFile()) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError("CloseHandle failed for: " + file_data_->GetName(), + lastError); + } + return s; +} + +inline +Status WinWritableImpl::SyncImpl() { + Status s; + // Calls flush buffers + if (fsync(file_data_->GetFileHandle()) < 0) { + auto lastError = GetLastError(); + s = IOErrorFromWindowsError( + "fsync failed at Sync() for: " + file_data_->GetName(), lastError); + } + return s; +} + + +inline +Status WinWritableImpl::AllocateImpl(uint64_t offset, uint64_t len) { + Status status; + TEST_KILL_RANDOM("WinWritableFile::Allocate", rocksdb_kill_odds); + + // Make sure that we reserve an aligned amount of space + // since the reservation block size is driven outside so we want + // to check if we are ok with reservation here + size_t spaceToReserve = Roundup(offset + len, alignment_); + // Nothing to do + if (spaceToReserve <= reservedsize_) { + return status; + } + + IOSTATS_TIMER_GUARD(allocate_nanos); + status = PreallocateInternal(spaceToReserve); + if (status.ok()) { + reservedsize_ = spaceToReserve; + } + return status; +} + + +//////////////////////////////////////////////////////////////////////////////// +/// WinWritableFile + +WinWritableFile::WinWritableFile(const std::string& fname, HANDLE hFile, + size_t alignment, size_t /* capacity */, + const EnvOptions& options) + : WinFileData(fname, hFile, options.use_direct_writes), + WinWritableImpl(this, alignment) { + assert(!options.use_mmap_writes); +} + +WinWritableFile::~WinWritableFile() { +} + +// Indicates if the class makes use of direct I/O +bool WinWritableFile::use_direct_io() const { return WinFileData::use_direct_io(); } + +size_t WinWritableFile::GetRequiredBufferAlignment() const { + return GetAlignement(); +} + +Status WinWritableFile::Append(const Slice& data) { + return AppendImpl(data); +} + +Status WinWritableFile::PositionedAppend(const Slice& data, uint64_t offset) { + return PositionedAppendImpl(data, offset); +} + +// Need to implement this so the file is truncated correctly +// when buffered and unbuffered mode +Status WinWritableFile::Truncate(uint64_t size) { + return TruncateImpl(size); +} + +Status WinWritableFile::Close() { + return CloseImpl(); +} + + // write out the cached data to the OS cache + // This is now taken care of the WritableFileWriter +Status WinWritableFile::Flush() { + return Status::OK(); +} + +Status WinWritableFile::Sync() { + return SyncImpl(); +} + +Status WinWritableFile::Fsync() { return SyncImpl(); } + +uint64_t WinWritableFile::GetFileSize() { + return GetFileNextWriteOffset(); +} + +Status WinWritableFile::Allocate(uint64_t offset, uint64_t len) { + return AllocateImpl(offset, len); +} + +size_t WinWritableFile::GetUniqueId(char* id, size_t max_size) const { + return GetUniqueIdFromFile(GetFileHandle(), id, max_size); +} + +///////////////////////////////////////////////////////////////////////// +/// WinRandomRWFile + +WinRandomRWFile::WinRandomRWFile(const std::string& fname, HANDLE hFile, + size_t alignment, const EnvOptions& options) + : WinFileData(fname, hFile, + options.use_direct_reads && options.use_direct_writes), + WinRandomAccessImpl(this, alignment, options), + WinWritableImpl(this, alignment) {} + +bool WinRandomRWFile::use_direct_io() const { return WinFileData::use_direct_io(); } + +size_t WinRandomRWFile::GetRequiredBufferAlignment() const { + return GetAlignement(); +} + +Status WinRandomRWFile::Write(uint64_t offset, const Slice & data) { + return PositionedAppendImpl(data, offset); +} + +Status WinRandomRWFile::Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const { + return ReadImpl(offset, n, result, scratch); +} + +Status WinRandomRWFile::Flush() { + return Status::OK(); +} + +Status WinRandomRWFile::Sync() { + return SyncImpl(); +} + +Status WinRandomRWFile::Close() { + return CloseImpl(); +} + +////////////////////////////////////////////////////////////////////////// +/// WinDirectory + +Status WinDirectory::Fsync() { return Status::OK(); } + +////////////////////////////////////////////////////////////////////////// +/// WinFileLock + +WinFileLock::~WinFileLock() { + BOOL ret = ::CloseHandle(hFile_); + assert(ret); +} + +} +} diff --git a/port/win/io_win.h b/port/win/io_win.h new file mode 100644 index 00000000000..2c1d5a1ea9e --- /dev/null +++ b/port/win/io_win.h @@ -0,0 +1,441 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#pragma once + +#include +#include +#include + +#include "rocksdb/status.h" +#include "rocksdb/env.h" +#include "util/aligned_buffer.h" + +#include + + +namespace rocksdb { +namespace port { + +std::string GetWindowsErrSz(DWORD err); + +inline Status IOErrorFromWindowsError(const std::string& context, DWORD err) { + return ((err == ERROR_HANDLE_DISK_FULL) || (err == ERROR_DISK_FULL)) + ? Status::NoSpace(context, GetWindowsErrSz(err)) + : Status::IOError(context, GetWindowsErrSz(err)); +} + +inline Status IOErrorFromLastWindowsError(const std::string& context) { + return IOErrorFromWindowsError(context, GetLastError()); +} + +inline Status IOError(const std::string& context, int err_number) { + return (err_number == ENOSPC) + ? Status::NoSpace(context, strerror(err_number)) + : Status::IOError(context, strerror(err_number)); +} + +// Note the below two do not set errno because they are used only here in this +// file +// on a Windows handle and, therefore, not necessary. Translating GetLastError() +// to errno +// is a sad business +inline int fsync(HANDLE hFile) { + if (!FlushFileBuffers(hFile)) { + return -1; + } + + return 0; +} + +SSIZE_T pwrite(HANDLE hFile, const char* src, size_t numBytes, uint64_t offset); + +SSIZE_T pread(HANDLE hFile, char* src, size_t numBytes, uint64_t offset); + +Status fallocate(const std::string& filename, HANDLE hFile, uint64_t to_size); + +Status ftruncate(const std::string& filename, HANDLE hFile, uint64_t toSize); + +size_t GetUniqueIdFromFile(HANDLE hFile, char* id, size_t max_size); + +class WinFileData { + protected: + const std::string filename_; + HANDLE hFile_; + // If ture, the I/O issued would be direct I/O which the buffer + // will need to be aligned (not sure there is a guarantee that the buffer + // passed in is aligned). + const bool use_direct_io_; + + public: + // We want this class be usable both for inheritance (prive + // or protected) and for containment so __ctor and __dtor public + WinFileData(const std::string& filename, HANDLE hFile, bool direct_io) + : filename_(filename), hFile_(hFile), use_direct_io_(direct_io) {} + + virtual ~WinFileData() { this->CloseFile(); } + + bool CloseFile() { + bool result = true; + + if (hFile_ != NULL && hFile_ != INVALID_HANDLE_VALUE) { + result = ::CloseHandle(hFile_); + assert(result); + hFile_ = NULL; + } + return result; + } + + const std::string& GetName() const { return filename_; } + + HANDLE GetFileHandle() const { return hFile_; } + + bool use_direct_io() const { return use_direct_io_; } + + WinFileData(const WinFileData&) = delete; + WinFileData& operator=(const WinFileData&) = delete; +}; + +class WinSequentialFile : protected WinFileData, public SequentialFile { + + // Override for behavior change when creating a custom env + virtual SSIZE_T PositionedReadInternal(char* src, size_t numBytes, + uint64_t offset) const; + +public: + WinSequentialFile(const std::string& fname, HANDLE f, + const EnvOptions& options); + + ~WinSequentialFile(); + + WinSequentialFile(const WinSequentialFile&) = delete; + WinSequentialFile& operator=(const WinSequentialFile&) = delete; + + virtual Status Read(size_t n, Slice* result, char* scratch) override; + virtual Status PositionedRead(uint64_t offset, size_t n, Slice* result, + char* scratch) override; + + virtual Status Skip(uint64_t n) override; + + virtual Status InvalidateCache(size_t offset, size_t length) override; + + virtual bool use_direct_io() const override { return WinFileData::use_direct_io(); } +}; + +// mmap() based random-access +class WinMmapReadableFile : private WinFileData, public RandomAccessFile { + HANDLE hMap_; + + const void* mapped_region_; + const size_t length_; + + public: + // mapped_region_[0,length-1] contains the mmapped contents of the file. + WinMmapReadableFile(const std::string& fileName, HANDLE hFile, HANDLE hMap, + const void* mapped_region, size_t length); + + ~WinMmapReadableFile(); + + WinMmapReadableFile(const WinMmapReadableFile&) = delete; + WinMmapReadableFile& operator=(const WinMmapReadableFile&) = delete; + + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const override; + + virtual Status InvalidateCache(size_t offset, size_t length) override; + + virtual size_t GetUniqueId(char* id, size_t max_size) const override; +}; + +// We preallocate and use memcpy to append new +// data to the file. This is safe since we either properly close the +// file before reading from it, or for log files, the reading code +// knows enough to skip zero suffixes. +class WinMmapFile : private WinFileData, public WritableFile { + private: + HANDLE hMap_; + + const size_t page_size_; // We flush the mapping view in page_size + // increments. We may decide if this is a memory + // page size or SSD page size + const size_t + allocation_granularity_; // View must start at such a granularity + + size_t reserved_size_; // Preallocated size + + size_t mapping_size_; // The max size of the mapping object + // we want to guess the final file size to minimize the remapping + size_t view_size_; // How much memory to map into a view at a time + + char* mapped_begin_; // Must begin at the file offset that is aligned with + // allocation_granularity_ + char* mapped_end_; + char* dst_; // Where to write next (in range [mapped_begin_,mapped_end_]) + char* last_sync_; // Where have we synced up to + + uint64_t file_offset_; // Offset of mapped_begin_ in file + + // Do we have unsynced writes? + bool pending_sync_; + + // Can only truncate or reserve to a sector size aligned if + // used on files that are opened with Unbuffered I/O + Status TruncateFile(uint64_t toSize); + + Status UnmapCurrentRegion(); + + Status MapNewRegion(); + + virtual Status PreallocateInternal(uint64_t spaceToReserve); + + public: + WinMmapFile(const std::string& fname, HANDLE hFile, size_t page_size, + size_t allocation_granularity, const EnvOptions& options); + + ~WinMmapFile(); + + WinMmapFile(const WinMmapFile&) = delete; + WinMmapFile& operator=(const WinMmapFile&) = delete; + + virtual Status Append(const Slice& data) override; + + // Means Close() will properly take care of truncate + // and it does not need any additional information + virtual Status Truncate(uint64_t size) override; + + virtual Status Close() override; + + virtual Status Flush() override; + + // Flush only data + virtual Status Sync() override; + + /** + * Flush data as well as metadata to stable storage. + */ + virtual Status Fsync() override; + + /** + * Get the size of valid data in the file. This will not match the + * size that is returned from the filesystem because we use mmap + * to extend file by map_size every time. + */ + virtual uint64_t GetFileSize() override; + + virtual Status InvalidateCache(size_t offset, size_t length) override; + + virtual Status Allocate(uint64_t offset, uint64_t len) override; + + virtual size_t GetUniqueId(char* id, size_t max_size) const override; +}; + +class WinRandomAccessImpl { + protected: + WinFileData* file_base_; + size_t alignment_; + + // Override for behavior change when creating a custom env + virtual SSIZE_T PositionedReadInternal(char* src, size_t numBytes, + uint64_t offset) const; + + WinRandomAccessImpl(WinFileData* file_base, size_t alignment, + const EnvOptions& options); + + virtual ~WinRandomAccessImpl() {} + + Status ReadImpl(uint64_t offset, size_t n, Slice* result, + char* scratch) const; + + size_t GetAlignment() const { return alignment_; } + + public: + + WinRandomAccessImpl(const WinRandomAccessImpl&) = delete; + WinRandomAccessImpl& operator=(const WinRandomAccessImpl&) = delete; +}; + +// pread() based random-access +class WinRandomAccessFile + : private WinFileData, + protected WinRandomAccessImpl, // Want to be able to override + // PositionedReadInternal + public RandomAccessFile { + public: + WinRandomAccessFile(const std::string& fname, HANDLE hFile, size_t alignment, + const EnvOptions& options); + + ~WinRandomAccessFile(); + + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const override; + + virtual size_t GetUniqueId(char* id, size_t max_size) const override; + + virtual bool use_direct_io() const override { return WinFileData::use_direct_io(); } + + virtual Status InvalidateCache(size_t offset, size_t length) override; + + virtual size_t GetRequiredBufferAlignment() const override; +}; + +// This is a sequential write class. It has been mimicked (as others) after +// the original Posix class. We add support for unbuffered I/O on windows as +// well +// we utilize the original buffer as an alignment buffer to write directly to +// file with no buffering. +// No buffering requires that the provided buffer is aligned to the physical +// sector size (SSD page size) and +// that all SetFilePointer() operations to occur with such an alignment. +// We thus always write in sector/page size increments to the drive and leave +// the tail for the next write OR for Close() at which point we pad with zeros. +// No padding is required for +// buffered access. +class WinWritableImpl { + protected: + WinFileData* file_data_; + const uint64_t alignment_; + uint64_t next_write_offset_; // Needed because Windows does not support O_APPEND + uint64_t reservedsize_; // how far we have reserved space + + virtual Status PreallocateInternal(uint64_t spaceToReserve); + + WinWritableImpl(WinFileData* file_data, size_t alignment); + + ~WinWritableImpl() {} + + uint64_t GetAlignement() const { return alignment_; } + + Status AppendImpl(const Slice& data); + + // Requires that the data is aligned as specified by + // GetRequiredBufferAlignment() + Status PositionedAppendImpl(const Slice& data, uint64_t offset); + + Status TruncateImpl(uint64_t size); + + Status CloseImpl(); + + Status SyncImpl(); + + uint64_t GetFileNextWriteOffset() { + // Double accounting now here with WritableFileWriter + // and this size will be wrong when unbuffered access is used + // but tests implement their own writable files and do not use + // WritableFileWrapper + // so we need to squeeze a square peg through + // a round hole here. + return next_write_offset_; + } + + Status AllocateImpl(uint64_t offset, uint64_t len); + + public: + WinWritableImpl(const WinWritableImpl&) = delete; + WinWritableImpl& operator=(const WinWritableImpl&) = delete; +}; + +class WinWritableFile : private WinFileData, + protected WinWritableImpl, + public WritableFile { + public: + WinWritableFile(const std::string& fname, HANDLE hFile, size_t alignment, + size_t capacity, const EnvOptions& options); + + ~WinWritableFile(); + + virtual Status Append(const Slice& data) override; + + // Requires that the data is aligned as specified by + // GetRequiredBufferAlignment() + virtual Status PositionedAppend(const Slice& data, uint64_t offset) override; + + // Need to implement this so the file is truncated correctly + // when buffered and unbuffered mode + virtual Status Truncate(uint64_t size) override; + + virtual Status Close() override; + + // write out the cached data to the OS cache + // This is now taken care of the WritableFileWriter + virtual Status Flush() override; + + virtual Status Sync() override; + + virtual Status Fsync() override; + + // Indicates if the class makes use of direct I/O + // Use PositionedAppend + virtual bool use_direct_io() const override; + + virtual size_t GetRequiredBufferAlignment() const override; + + virtual uint64_t GetFileSize() override; + + virtual Status Allocate(uint64_t offset, uint64_t len) override; + + virtual size_t GetUniqueId(char* id, size_t max_size) const override; +}; + +class WinRandomRWFile : private WinFileData, + protected WinRandomAccessImpl, + protected WinWritableImpl, + public RandomRWFile { + public: + WinRandomRWFile(const std::string& fname, HANDLE hFile, size_t alignment, + const EnvOptions& options); + + ~WinRandomRWFile() {} + + // Indicates if the class makes use of direct I/O + // If false you must pass aligned buffer to Write() + virtual bool use_direct_io() const override; + + // Use the returned alignment value to allocate aligned + // buffer for Write() when use_direct_io() returns true + virtual size_t GetRequiredBufferAlignment() const override; + + // Write bytes in `data` at offset `offset`, Returns Status::OK() on success. + // Pass aligned buffer when use_direct_io() returns true. + virtual Status Write(uint64_t offset, const Slice& data) override; + + // Read up to `n` bytes starting from offset `offset` and store them in + // result, provided `scratch` size should be at least `n`. + // Returns Status::OK() on success. + virtual Status Read(uint64_t offset, size_t n, Slice* result, + char* scratch) const override; + + virtual Status Flush() override; + + virtual Status Sync() override; + + virtual Status Fsync() { return Sync(); } + + virtual Status Close() override; +}; + +class WinDirectory : public Directory { + public: + WinDirectory() {} + + virtual Status Fsync() override; +}; + +class WinFileLock : public FileLock { + public: + explicit WinFileLock(HANDLE hFile) : hFile_(hFile) { + assert(hFile != NULL); + assert(hFile != INVALID_HANDLE_VALUE); + } + + ~WinFileLock(); + + private: + HANDLE hFile_; +}; +} +} diff --git a/port/win/port_win.cc b/port/win/port_win.cc new file mode 100644 index 00000000000..b3fccbd9308 --- /dev/null +++ b/port/win/port_win.cc @@ -0,0 +1,230 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#if !defined(OS_WIN) && !defined(WIN32) && !defined(_WIN32) +#error Windows Specific Code +#endif + +#include "port/win/port_win.h" + +#include +#include "port/dirent.h" +#include "port/sys_time.h" + +#include +#include +#include +#include + +#include +#include +#include + +#include "util/logging.h" + +namespace rocksdb { +namespace port { + +void gettimeofday(struct timeval* tv, struct timezone* /* tz */) { + using namespace std::chrono; + + microseconds usNow( + duration_cast(system_clock::now().time_since_epoch())); + + seconds secNow(duration_cast(usNow)); + + tv->tv_sec = static_cast(secNow.count()); + tv->tv_usec = static_cast(usNow.count() - + duration_cast(secNow).count()); +} + +Mutex::~Mutex() {} + +CondVar::~CondVar() {} + +void CondVar::Wait() { + // Caller must ensure that mutex is held prior to calling this method + std::unique_lock lk(mu_->getLock(), std::adopt_lock); +#ifndef NDEBUG + mu_->locked_ = false; +#endif + cv_.wait(lk); +#ifndef NDEBUG + mu_->locked_ = true; +#endif + // Release ownership of the lock as we don't want it to be unlocked when + // it goes out of scope (as we adopted the lock and didn't lock it ourselves) + lk.release(); +} + +bool CondVar::TimedWait(uint64_t abs_time_us) { + + using namespace std::chrono; + + // MSVC++ library implements wait_until in terms of wait_for so + // we need to convert absolute wait into relative wait. + microseconds usAbsTime(abs_time_us); + + microseconds usNow( + duration_cast(system_clock::now().time_since_epoch())); + microseconds relTimeUs = + (usAbsTime > usNow) ? (usAbsTime - usNow) : microseconds::zero(); + + // Caller must ensure that mutex is held prior to calling this method + std::unique_lock lk(mu_->getLock(), std::adopt_lock); +#ifndef NDEBUG + mu_->locked_ = false; +#endif + std::cv_status cvStatus = cv_.wait_for(lk, relTimeUs); +#ifndef NDEBUG + mu_->locked_ = true; +#endif + // Release ownership of the lock as we don't want it to be unlocked when + // it goes out of scope (as we adopted the lock and didn't lock it ourselves) + lk.release(); + + if (cvStatus == std::cv_status::timeout) { + return true; + } + + return false; +} + +void CondVar::Signal() { cv_.notify_one(); } + +void CondVar::SignalAll() { cv_.notify_all(); } + +int PhysicalCoreID() { return GetCurrentProcessorNumber(); } + +void InitOnce(OnceType* once, void (*initializer)()) { + std::call_once(once->flag_, initializer); +} + +// Private structure, exposed only by pointer +struct DIR { + intptr_t handle_; + bool firstread_; + struct __finddata64_t data_; + dirent entry_; + + DIR() : handle_(-1), firstread_(true) {} + + DIR(const DIR&) = delete; + DIR& operator=(const DIR&) = delete; + + ~DIR() { + if (-1 != handle_) { + _findclose(handle_); + } + } +}; + +DIR* opendir(const char* name) { + if (!name || *name == 0) { + errno = ENOENT; + return nullptr; + } + + std::string pattern(name); + pattern.append("\\").append("*"); + + std::unique_ptr

    dir(new DIR); + + dir->handle_ = _findfirst64(pattern.c_str(), &dir->data_); + + if (dir->handle_ == -1) { + return nullptr; + } + + strcpy_s(dir->entry_.d_name, sizeof(dir->entry_.d_name), dir->data_.name); + + return dir.release(); +} + +struct dirent* readdir(DIR* dirp) { + if (!dirp || dirp->handle_ == -1) { + errno = EBADF; + return nullptr; + } + + if (dirp->firstread_) { + dirp->firstread_ = false; + return &dirp->entry_; + } + + auto ret = _findnext64(dirp->handle_, &dirp->data_); + + if (ret != 0) { + return nullptr; + } + + strcpy_s(dirp->entry_.d_name, sizeof(dirp->entry_.d_name), dirp->data_.name); + + return &dirp->entry_; +} + +int closedir(DIR* dirp) { + delete dirp; + return 0; +} + +int truncate(const char* path, int64_t len) { + if (path == nullptr) { + errno = EFAULT; + return -1; + } + + if (len < 0) { + errno = EINVAL; + return -1; + } + + HANDLE hFile = + CreateFile(path, GENERIC_READ | GENERIC_WRITE, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, + NULL, // Security attrs + OPEN_EXISTING, // Truncate existing file only + FILE_ATTRIBUTE_NORMAL, NULL); + + if (INVALID_HANDLE_VALUE == hFile) { + auto lastError = GetLastError(); + if (lastError == ERROR_FILE_NOT_FOUND) { + errno = ENOENT; + } else if (lastError == ERROR_ACCESS_DENIED) { + errno = EACCES; + } else { + errno = EIO; + } + return -1; + } + + int result = 0; + FILE_END_OF_FILE_INFO end_of_file; + end_of_file.EndOfFile.QuadPart = len; + + if (!SetFileInformationByHandle(hFile, FileEndOfFileInfo, &end_of_file, + sizeof(FILE_END_OF_FILE_INFO))) { + errno = EIO; + result = -1; + } + + CloseHandle(hFile); + return result; +} + +void Crash(const std::string& srcfile, int srcline) { + fprintf(stdout, "Crashing at %s:%d\n", srcfile.c_str(), srcline); + fflush(stdout); + abort(); +} + +int GetMaxOpenFiles() { return -1; } + +} // namespace port +} // namespace rocksdb diff --git a/port/win/port_win.h b/port/win/port_win.h new file mode 100644 index 00000000000..f3c86690515 --- /dev/null +++ b/port/win/port_win.h @@ -0,0 +1,347 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// See port_example.h for documentation for the following types/functions. + +#ifndef STORAGE_LEVELDB_PORT_PORT_WIN_H_ +#define STORAGE_LEVELDB_PORT_PORT_WIN_H_ + +// Always want minimum headers +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif + +// Assume that for everywhere +#undef PLATFORM_IS_LITTLE_ENDIAN +#define PLATFORM_IS_LITTLE_ENDIAN true + +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "port/win/win_thread.h" + +#include "rocksdb/options.h" + +#undef min +#undef max +#undef DeleteFile +#undef GetCurrentTime + + +#ifndef strcasecmp +#define strcasecmp _stricmp +#endif + +#undef GetCurrentTime +#undef DeleteFile + +#ifndef _SSIZE_T_DEFINED +typedef SSIZE_T ssize_t; +#endif + +// size_t printf formatting named in the manner of C99 standard formatting +// strings such as PRIu64 +// in fact, we could use that one +#ifndef ROCKSDB_PRIszt +#define ROCKSDB_PRIszt "Iu" +#endif + +#ifdef _MSC_VER +#define __attribute__(A) + +// Thread local storage on Linux +// There is thread_local in C++11 +#ifndef __thread +#define __thread __declspec(thread) +#endif + +#endif + +#ifndef PLATFORM_IS_LITTLE_ENDIAN +#define PLATFORM_IS_LITTLE_ENDIAN (__BYTE_ORDER == __LITTLE_ENDIAN) +#endif + +namespace rocksdb { + +#define PREFETCH(addr, rw, locality) + +namespace port { + +// VS 15 +#if (defined _MSC_VER) && (_MSC_VER >= 1900) + +#define ROCKSDB_NOEXCEPT noexcept + +// For use at db/file_indexer.h kLevelMaxIndex +const int kMaxInt32 = std::numeric_limits::max(); +const uint64_t kMaxUint64 = std::numeric_limits::max(); +const int64_t kMaxInt64 = std::numeric_limits::max(); + +const size_t kMaxSizet = std::numeric_limits::max(); + +#else //_MSC_VER + +// VS 15 has snprintf +#define snprintf _snprintf + +#define ROCKSDB_NOEXCEPT +// std::numeric_limits::max() is not constexpr just yet +// therefore, use the same limits + +// For use at db/file_indexer.h kLevelMaxIndex +const int kMaxInt32 = INT32_MAX; +const int64_t kMaxInt64 = INT64_MAX; +const uint64_t kMaxUint64 = UINT64_MAX; + +#ifdef _WIN64 +const size_t kMaxSizet = UINT64_MAX; +#else +const size_t kMaxSizet = UINT_MAX; +#endif + +#endif //_MSC_VER + +const bool kLittleEndian = true; + +class CondVar; + +class Mutex { + public: + + /* implicit */ Mutex(bool adaptive = false) +#ifndef NDEBUG + : locked_(false) +#endif + { } + + ~Mutex(); + + void Lock() { + mutex_.lock(); +#ifndef NDEBUG + locked_ = true; +#endif + } + + void Unlock() { +#ifndef NDEBUG + locked_ = false; +#endif + mutex_.unlock(); + } + + // this will assert if the mutex is not locked + // it does NOT verify that mutex is held by a calling thread + void AssertHeld() { +#ifndef NDEBUG + assert(locked_); +#endif + } + + // Mutex is move only with lock ownership transfer + Mutex(const Mutex&) = delete; + void operator=(const Mutex&) = delete; + + private: + + friend class CondVar; + + std::mutex& getLock() { + return mutex_; + } + + std::mutex mutex_; +#ifndef NDEBUG + bool locked_; +#endif +}; + +class RWMutex { + public: + RWMutex() { InitializeSRWLock(&srwLock_); } + + void ReadLock() { AcquireSRWLockShared(&srwLock_); } + + void WriteLock() { AcquireSRWLockExclusive(&srwLock_); } + + void ReadUnlock() { ReleaseSRWLockShared(&srwLock_); } + + void WriteUnlock() { ReleaseSRWLockExclusive(&srwLock_); } + + // Empty as in POSIX + void AssertHeld() {} + + private: + SRWLOCK srwLock_; + // No copying allowed + RWMutex(const RWMutex&); + void operator=(const RWMutex&); +}; + +class CondVar { + public: + explicit CondVar(Mutex* mu) : mu_(mu) { + } + + ~CondVar(); + void Wait(); + bool TimedWait(uint64_t expiration_time); + void Signal(); + void SignalAll(); + + // Condition var is not copy/move constructible + CondVar(const CondVar&) = delete; + CondVar& operator=(const CondVar&) = delete; + + CondVar(CondVar&&) = delete; + CondVar& operator=(CondVar&&) = delete; + + private: + std::condition_variable cv_; + Mutex* mu_; +}; + +// Wrapper around the platform efficient +// or otherwise preferrable implementation +using Thread = WindowsThread; + +// OnceInit type helps emulate +// Posix semantics with initialization +// adopted in the project +struct OnceType { + + struct Init {}; + + OnceType() {} + OnceType(const Init&) {} + OnceType(const OnceType&) = delete; + OnceType& operator=(const OnceType&) = delete; + + std::once_flag flag_; +}; + +#define LEVELDB_ONCE_INIT port::OnceType::Init() +extern void InitOnce(OnceType* once, void (*initializer)()); + +#ifndef CACHE_LINE_SIZE +#define CACHE_LINE_SIZE 64U +#endif + +#ifdef ROCKSDB_JEMALLOC +#include "jemalloc/jemalloc.h" +// Separate inlines so they can be replaced if needed +inline void* jemalloc_aligned_alloc( size_t size, size_t alignment) { + return je_aligned_alloc(alignment, size); +} +inline void jemalloc_aligned_free(void* p) { + je_free(p); +} +#endif + +inline void *cacheline_aligned_alloc(size_t size) { +#ifdef ROCKSDB_JEMALLOC + return jemalloc_aligned_alloc(size, CACHE_LINE_SIZE); +#else + return _aligned_malloc(size, CACHE_LINE_SIZE); +#endif +} + +inline void cacheline_aligned_free(void *memblock) { +#ifdef ROCKSDB_JEMALLOC + jemalloc_aligned_free(memblock); +#else + _aligned_free(memblock); +#endif +} + +// https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52991 for MINGW32 +// could not be worked around with by -mno-ms-bitfields +#ifndef __MINGW32__ +#define ALIGN_AS(n) __declspec(align(n)) +#else +#define ALIGN_AS(n) +#endif + +static inline void AsmVolatilePause() { +#if defined(_M_IX86) || defined(_M_X64) + YieldProcessor(); +#endif + // it would be nice to get "wfe" on ARM here +} + +extern int PhysicalCoreID(); + +// For Thread Local Storage abstraction +typedef DWORD pthread_key_t; + +inline int pthread_key_create(pthread_key_t* key, void (*destructor)(void*)) { + // Not used + (void)destructor; + + pthread_key_t k = TlsAlloc(); + if (TLS_OUT_OF_INDEXES == k) { + return ENOMEM; + } + + *key = k; + return 0; +} + +inline int pthread_key_delete(pthread_key_t key) { + if (!TlsFree(key)) { + return EINVAL; + } + return 0; +} + +inline int pthread_setspecific(pthread_key_t key, const void* value) { + if (!TlsSetValue(key, const_cast(value))) { + return ENOMEM; + } + return 0; +} + +inline void* pthread_getspecific(pthread_key_t key) { + void* result = TlsGetValue(key); + if (!result) { + if (GetLastError() != ERROR_SUCCESS) { + errno = EINVAL; + } else { + errno = NOERROR; + } + } + return result; +} + +// UNIX equiv although errno numbers will be off +// using C-runtime to implement. Note, this does not +// feel space with zeros in case the file is extended. +int truncate(const char* path, int64_t length); +void Crash(const std::string& srcfile, int srcline); +extern int GetMaxOpenFiles(); + +} // namespace port + +using port::pthread_key_t; +using port::pthread_key_create; +using port::pthread_key_delete; +using port::pthread_setspecific; +using port::pthread_getspecific; +using port::truncate; + +} // namespace rocksdb + +#endif // STORAGE_LEVELDB_PORT_PORT_WIN_H_ diff --git a/port/win/stdint.h b/port/win/stdint.h deleted file mode 100644 index 39edd0db13f..00000000000 --- a/port/win/stdint.h +++ /dev/null @@ -1,24 +0,0 @@ -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -// MSVC didn't ship with this file until the 2010 version. - -#ifndef STORAGE_LEVELDB_PORT_WIN_STDINT_H_ -#define STORAGE_LEVELDB_PORT_WIN_STDINT_H_ - -#if !defined(_MSC_VER) -#error This file should only be included when compiling with MSVC. -#endif - -// Define C99 equivalent types. -typedef signed char int8_t; -typedef signed short int16_t; -typedef signed int int32_t; -typedef signed long long int64_t; -typedef unsigned char uint8_t; -typedef unsigned short uint16_t; -typedef unsigned int uint32_t; -typedef unsigned long long uint64_t; - -#endif // STORAGE_LEVELDB_PORT_WIN_STDINT_H_ diff --git a/port/win/win_jemalloc.cc b/port/win/win_jemalloc.cc new file mode 100644 index 00000000000..fc46e189c4f --- /dev/null +++ b/port/win/win_jemalloc.cc @@ -0,0 +1,47 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under the BSD-style license found in the +// LICENSE file in the root directory of this source tree. An additional grant +// of patent rights can be found in the PATENTS file in the same directory. +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#ifndef ROCKSDB_JEMALLOC +# error This file can only be part of jemalloc aware build +#endif + +#include +#include "jemalloc/jemalloc.h" + +// Global operators to be replaced by a linker when this file is +// a part of the build + +void* operator new(size_t size) { + void* p = je_malloc(size); + if (!p) { + throw std::bad_alloc(); + } + return p; +} + +void* operator new[](size_t size) { + void* p = je_malloc(size); + if (!p) { + throw std::bad_alloc(); + } + return p; +} + +void operator delete(void* p) { + if (p) { + je_free(p); + } +} + +void operator delete[](void* p) { + if (p) { + je_free(p); + } +} + diff --git a/port/win/win_logger.cc b/port/win/win_logger.cc new file mode 100644 index 00000000000..0bace9f31f8 --- /dev/null +++ b/port/win/win_logger.cc @@ -0,0 +1,160 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Logger implementation that can be shared by all environments +// where enough posix functionality is available. + +#include "port/win/win_logger.h" +#include "port/win/io_win.h" + +#include +#include +#include +#include +#include + +#include "rocksdb/env.h" + +#include "monitoring/iostats_context_imp.h" +#include "port/sys_time.h" + +namespace rocksdb { + +namespace port { + +WinLogger::WinLogger(uint64_t (*gettid)(), Env* env, HANDLE file, + const InfoLogLevel log_level) + : Logger(log_level), + file_(file), + gettid_(gettid), + log_size_(0), + last_flush_micros_(0), + env_(env), + flush_pending_(false) {} + +void WinLogger::DebugWriter(const char* str, int len) { + DWORD bytesWritten = 0; + BOOL ret = WriteFile(file_, str, len, &bytesWritten, NULL); + if (ret == FALSE) { + std::string errSz = GetWindowsErrSz(GetLastError()); + fprintf(stderr, errSz.c_str()); + } +} + +WinLogger::~WinLogger() { close(); } + +void WinLogger::close() { CloseHandle(file_); } + +void WinLogger::Flush() { + if (flush_pending_) { + flush_pending_ = false; + // With Windows API writes go to OS buffers directly so no fflush needed + // unlike with C runtime API. We don't flush all the way to disk + // for perf reasons. + } + + last_flush_micros_ = env_->NowMicros(); +} + +void WinLogger::Logv(const char* format, va_list ap) { + IOSTATS_TIMER_GUARD(logger_nanos); + + const uint64_t thread_id = (*gettid_)(); + + // We try twice: the first time with a fixed-size stack allocated buffer, + // and the second time with a much larger dynamically allocated buffer. + char buffer[500]; + std::unique_ptr largeBuffer; + for (int iter = 0; iter < 2; ++iter) { + char* base; + int bufsize; + if (iter == 0) { + bufsize = sizeof(buffer); + base = buffer; + } else { + bufsize = 30000; + largeBuffer.reset(new char[bufsize]); + base = largeBuffer.get(); + } + + char* p = base; + char* limit = base + bufsize; + + struct timeval now_tv; + gettimeofday(&now_tv, nullptr); + const time_t seconds = now_tv.tv_sec; + struct tm t; + localtime_s(&t, &seconds); + p += snprintf(p, limit - p, "%04d/%02d/%02d-%02d:%02d:%02d.%06d %llx ", + t.tm_year + 1900, t.tm_mon + 1, t.tm_mday, t.tm_hour, + t.tm_min, t.tm_sec, static_cast(now_tv.tv_usec), + static_cast(thread_id)); + + // Print the message + if (p < limit) { + va_list backup_ap; + va_copy(backup_ap, ap); + int done = vsnprintf(p, limit - p, format, backup_ap); + if (done > 0) { + p += done; + } else { + continue; + } + va_end(backup_ap); + } + + // Truncate to available space if necessary + if (p >= limit) { + if (iter == 0) { + continue; // Try again with larger buffer + } else { + p = limit - 1; + } + } + + // Add newline if necessary + if (p == base || p[-1] != '\n') { + *p++ = '\n'; + } + + assert(p <= limit); + const size_t write_size = p - base; + + DWORD bytesWritten = 0; + BOOL ret = WriteFile(file_, base, static_cast(write_size), + &bytesWritten, NULL); + if (ret == FALSE) { + std::string errSz = GetWindowsErrSz(GetLastError()); + fprintf(stderr, errSz.c_str()); + } + + flush_pending_ = true; + assert((bytesWritten == write_size) || (ret == FALSE)); + if (bytesWritten > 0) { + log_size_ += write_size; + } + + uint64_t now_micros = + static_cast(now_tv.tv_sec) * 1000000 + now_tv.tv_usec; + if (now_micros - last_flush_micros_ >= flush_every_seconds_ * 1000000) { + flush_pending_ = false; + // With Windows API writes go to OS buffers directly so no fflush needed + // unlike with C runtime API. We don't flush all the way to disk + // for perf reasons. + last_flush_micros_ = now_micros; + } + break; + } +} + +size_t WinLogger::GetLogFileSize() const { return log_size_; } + +} + +} // namespace rocksdb diff --git a/port/win/win_logger.h b/port/win/win_logger.h new file mode 100644 index 00000000000..2d44f506d1a --- /dev/null +++ b/port/win/win_logger.h @@ -0,0 +1,63 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// Logger implementation that can be shared by all environments +// where enough posix functionality is available. + +#pragma once + +#include + +#include "rocksdb/env.h" + +#include +#include + +namespace rocksdb { + +class Env; + +namespace port { + +class WinLogger : public rocksdb::Logger { + public: + WinLogger(uint64_t (*gettid)(), Env* env, HANDLE file, + const InfoLogLevel log_level = InfoLogLevel::ERROR_LEVEL); + + virtual ~WinLogger(); + + WinLogger(const WinLogger&) = delete; + + WinLogger& operator=(const WinLogger&) = delete; + + void close(); + + void Flush() override; + + using rocksdb::Logger::Logv; + void Logv(const char* format, va_list ap) override; + + size_t GetLogFileSize() const override; + + void DebugWriter(const char* str, int len); + + private: + HANDLE file_; + uint64_t (*gettid_)(); // Return the thread id for the current thread + std::atomic_size_t log_size_; + std::atomic_uint_fast64_t last_flush_micros_; + Env* env_; + bool flush_pending_; + + const static uint64_t flush_every_seconds_ = 5; +}; + +} + +} // namespace rocksdb diff --git a/port/win/win_thread.cc b/port/win/win_thread.cc new file mode 100644 index 00000000000..e55ca7450b1 --- /dev/null +++ b/port/win/win_thread.cc @@ -0,0 +1,166 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "port/win/win_thread.h" + +#include +#include // __beginthreadex +#include + +#include +#include +#include + +namespace rocksdb { +namespace port { + +struct WindowsThread::Data { + + std::function func_; + uintptr_t handle_; + + Data(std::function&& func) : + func_(std::move(func)), + handle_(0) { + } + + Data(const Data&) = delete; + Data& operator=(const Data&) = delete; + + static unsigned int __stdcall ThreadProc(void* arg); +}; + + +void WindowsThread::Init(std::function&& func) { + + data_.reset(new Data(std::move(func))); + + data_->handle_ = _beginthreadex(NULL, + 0, // stack size + &Data::ThreadProc, + data_.get(), + 0, // init flag + &th_id_); + + if (data_->handle_ == 0) { + throw std::system_error(std::make_error_code( + std::errc::resource_unavailable_try_again), + "Unable to create a thread"); + } +} + +WindowsThread::WindowsThread() : + data_(nullptr), + th_id_(0) +{} + + +WindowsThread::~WindowsThread() { + // Must be joined or detached + // before destruction. + // This is the same as std::thread + if (data_) { + if (joinable()) { + assert(false); + std::terminate(); + } + data_.reset(); + } +} + +WindowsThread::WindowsThread(WindowsThread&& o) noexcept : + WindowsThread() { + *this = std::move(o); +} + +WindowsThread& WindowsThread::operator=(WindowsThread&& o) noexcept { + + if (joinable()) { + assert(false); + std::terminate(); + } + + data_ = std::move(o.data_); + + // Per spec both instances will have the same id + th_id_ = o.th_id_; + + return *this; +} + +bool WindowsThread::joinable() const { + return (data_ && data_->handle_ != 0); +} + +WindowsThread::native_handle_type WindowsThread::native_handle() const { + return reinterpret_cast(data_->handle_); +} + +unsigned WindowsThread::hardware_concurrency() { + return std::thread::hardware_concurrency(); +} + +void WindowsThread::join() { + + if (!joinable()) { + assert(false); + throw std::system_error( + std::make_error_code(std::errc::invalid_argument), + "Thread is no longer joinable"); + } + + if (GetThreadId(GetCurrentThread()) == th_id_) { + assert(false); + throw std::system_error( + std::make_error_code(std::errc::resource_deadlock_would_occur), + "Can not join itself"); + } + + auto ret = WaitForSingleObject(reinterpret_cast(data_->handle_), + INFINITE); + if (ret != WAIT_OBJECT_0) { + auto lastError = GetLastError(); + assert(false); + throw std::system_error(static_cast(lastError), + std::system_category(), + "WaitForSingleObjectFailed"); + } + + CloseHandle(reinterpret_cast(data_->handle_)); + data_->handle_ = 0; +} + +bool WindowsThread::detach() { + + if (!joinable()) { + assert(false); + throw std::system_error( + std::make_error_code(std::errc::invalid_argument), + "Thread is no longer available"); + } + + BOOL ret = CloseHandle(reinterpret_cast(data_->handle_)); + data_->handle_ = 0; + + return (ret == TRUE); +} + +void WindowsThread::swap(WindowsThread& o) { + data_.swap(o.data_); + std::swap(th_id_, o.th_id_); +} + +unsigned int __stdcall WindowsThread::Data::ThreadProc(void* arg) { + auto data = reinterpret_cast(arg); + data->func_(); + _endthreadex(0); + return 0; +} +} // namespace port +} // namespace rocksdb diff --git a/port/win/win_thread.h b/port/win/win_thread.h new file mode 100644 index 00000000000..993cc027316 --- /dev/null +++ b/port/win/win_thread.h @@ -0,0 +1,121 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include +#include +#include + +namespace rocksdb { +namespace port { + +// This class is a replacement for std::thread +// 2 reasons we do not like std::thread: +// -- is that it dynamically allocates its internals that are automatically +// freed when the thread terminates and not on the destruction of the +// object. This makes it difficult to control the source of memory +// allocation +// - This implements Pimpl so we can easily replace the guts of the +// object in our private version if necessary. +class WindowsThread { + + struct Data; + + std::unique_ptr data_; + unsigned int th_id_; + + void Init(std::function&&); + +public: + + typedef void* native_handle_type; + + // Construct with no thread + WindowsThread(); + + // Template constructor + // + // This templated constructor accomplishes several things + // + // - Allows the class as whole to be not a template + // + // - take "universal" references to support both _lvalues and _rvalues + // + // - because this constructor is a catchall case in many respects it + // may prevent us from using both the default __ctor, the move __ctor. + // Also it may circumvent copy __ctor deletion. To work around this + // we make sure this one has at least one argument and eliminate + // it from the overload selection when WindowsThread is the first + // argument. + // + // - construct with Fx(Ax...) with a variable number of types/arguments. + // + // - Gathers together the callable object with its arguments and constructs + // a single callable entity + // + // - Makes use of std::function to convert it to a specification-template + // dependent type that both checks the signature conformance to ensure + // that all of the necessary arguments are provided and allows pimpl + // implementation. + template::type, + WindowsThread>::value>::type> + explicit WindowsThread(Fn&& fx, Args&&... ax) : + WindowsThread() { + + // Use binder to create a single callable entity + auto binder = std::bind(std::forward(fx), + std::forward(ax)...); + // Use std::function to take advantage of the type erasure + // so we can still hide implementation within pimpl + // This also makes sure that the binder signature is compliant + std::function target = binder; + + Init(std::move(target)); + } + + + ~WindowsThread(); + + WindowsThread(const WindowsThread&) = delete; + + WindowsThread& operator=(const WindowsThread&) = delete; + + WindowsThread(WindowsThread&&) noexcept; + + WindowsThread& operator=(WindowsThread&&) noexcept; + + bool joinable() const; + + unsigned int get_id() const { return th_id_; } + + native_handle_type native_handle() const; + + static unsigned hardware_concurrency(); + + void join(); + + bool detach(); + + void swap(WindowsThread&); +}; +} // namespace port +} // namespace rocksdb + +namespace std { + inline + void swap(rocksdb::port::WindowsThread& th1, + rocksdb::port::WindowsThread& th2) { + th1.swap(th2); + } +} // namespace std + diff --git a/port/win/xpress_win.cc b/port/win/xpress_win.cc new file mode 100644 index 00000000000..9ab23c534d9 --- /dev/null +++ b/port/win/xpress_win.cc @@ -0,0 +1,226 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "port/win/xpress_win.h" +#include + +#include +#include +#include +#include + +#ifdef XPRESS + +// Put this under ifdef so windows systems w/o this +// can still build +#include + +namespace rocksdb { +namespace port { +namespace xpress { + +// Helpers +namespace { + +auto CloseCompressorFun = [](void* h) { + if (NULL != h) { + ::CloseCompressor(reinterpret_cast(h)); + } +}; + +auto CloseDecompressorFun = [](void* h) { + if (NULL != h) { + ::CloseDecompressor(reinterpret_cast(h)); + } +}; +} + +bool Compress(const char* input, size_t length, std::string* output) { + + assert(input != nullptr); + assert(output != nullptr); + + if (length == 0) { + output->clear(); + return true; + } + + COMPRESS_ALLOCATION_ROUTINES* allocRoutinesPtr = nullptr; + + COMPRESSOR_HANDLE compressor = NULL; + + BOOL success = CreateCompressor( + COMPRESS_ALGORITHM_XPRESS, // Compression Algorithm + allocRoutinesPtr, // Optional allocation routine + &compressor); // Handle + + if (!success) { +#ifdef _DEBUG + std::cerr << "XPRESS: Failed to create Compressor LastError: " << + GetLastError() << std::endl; +#endif + return false; + } + + std::unique_ptr + compressorGuard(compressor, CloseCompressorFun); + + SIZE_T compressedBufferSize = 0; + + // Query compressed buffer size. + success = ::Compress( + compressor, // Compressor Handle + const_cast(input), // Input buffer + length, // Uncompressed data size + NULL, // Compressed Buffer + 0, // Compressed Buffer size + &compressedBufferSize); // Compressed Data size + + if (!success) { + + auto lastError = GetLastError(); + + if (lastError != ERROR_INSUFFICIENT_BUFFER) { +#ifdef _DEBUG + std::cerr << + "XPRESS: Failed to estimate compressed buffer size LastError " << + lastError << std::endl; +#endif + return false; + } + } + + assert(compressedBufferSize > 0); + + std::string result; + result.resize(compressedBufferSize); + + SIZE_T compressedDataSize = 0; + + // Compress + success = ::Compress( + compressor, // Compressor Handle + const_cast(input), // Input buffer + length, // Uncompressed data size + &result[0], // Compressed Buffer + compressedBufferSize, // Compressed Buffer size + &compressedDataSize); // Compressed Data size + + if (!success) { +#ifdef _DEBUG + std::cerr << "XPRESS: Failed to compress LastError " << + GetLastError() << std::endl; +#endif + return false; + } + + result.resize(compressedDataSize); + output->swap(result); + + return true; +} + +char* Decompress(const char* input_data, size_t input_length, + int* decompress_size) { + + assert(input_data != nullptr); + assert(decompress_size != nullptr); + + if (input_length == 0) { + return nullptr; + } + + COMPRESS_ALLOCATION_ROUTINES* allocRoutinesPtr = nullptr; + + DECOMPRESSOR_HANDLE decompressor = NULL; + + BOOL success = CreateDecompressor( + COMPRESS_ALGORITHM_XPRESS, // Compression Algorithm + allocRoutinesPtr, // Optional allocation routine + &decompressor); // Handle + + + if (!success) { +#ifdef _DEBUG + std::cerr << "XPRESS: Failed to create Decompressor LastError " + << GetLastError() << std::endl; +#endif + return nullptr; + } + + std::unique_ptr + compressorGuard(decompressor, CloseDecompressorFun); + + SIZE_T decompressedBufferSize = 0; + + success = ::Decompress( + decompressor, // Compressor Handle + const_cast(input_data), // Compressed data + input_length, // Compressed data size + NULL, // Buffer set to NULL + 0, // Buffer size set to 0 + &decompressedBufferSize); // Decompressed Data size + + if (!success) { + + auto lastError = GetLastError(); + + if (lastError != ERROR_INSUFFICIENT_BUFFER) { +#ifdef _DEBUG + std::cerr + << "XPRESS: Failed to estimate decompressed buffer size LastError " + << lastError << std::endl; +#endif + return nullptr; + } + } + + assert(decompressedBufferSize > 0); + + // On Windows we are limited to a 32-bit int for the + // output data size argument + // so we hopefully never get here + if (decompressedBufferSize > std::numeric_limits::max()) { + assert(false); + return nullptr; + } + + // The callers are deallocating using delete[] + // thus we must allocate with new[] + std::unique_ptr outputBuffer(new char[decompressedBufferSize]); + + SIZE_T decompressedDataSize = 0; + + success = ::Decompress( + decompressor, + const_cast(input_data), + input_length, + outputBuffer.get(), + decompressedBufferSize, + &decompressedDataSize); + + if (!success) { +#ifdef _DEBUG + std::cerr << + "XPRESS: Failed to decompress LastError " << + GetLastError() << std::endl; +#endif + return nullptr; + } + + *decompress_size = static_cast(decompressedDataSize); + + // Return the raw buffer to the caller supporting the tradition + return outputBuffer.release(); +} +} +} +} + +#endif diff --git a/port/win/xpress_win.h b/port/win/xpress_win.h new file mode 100644 index 00000000000..5b11e7da9c6 --- /dev/null +++ b/port/win/xpress_win.h @@ -0,0 +1,26 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include + +namespace rocksdb { +namespace port { +namespace xpress { + +bool Compress(const char* input, size_t length, std::string* output); + +char* Decompress(const char* input_data, size_t input_length, + int* decompress_size); + +} +} +} + diff --git a/port/xpress.h b/port/xpress.h new file mode 100644 index 00000000000..457025f666e --- /dev/null +++ b/port/xpress.h @@ -0,0 +1,17 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +// Xpress on Windows is implemeted using Win API +#if defined(ROCKSDB_PLATFORM_POSIX) +#error "Xpress compression not implemented" +#elif defined(OS_WIN) +#include "port/win/xpress_win.h" +#endif diff --git a/src.mk b/src.mk new file mode 100644 index 00000000000..5bd5236fa16 --- /dev/null +++ b/src.mk @@ -0,0 +1,400 @@ +# These are the sources from which librocksdb.a is built: +LIB_SOURCES = \ + cache/clock_cache.cc \ + cache/lru_cache.cc \ + cache/sharded_cache.cc \ + db/builder.cc \ + db/c.cc \ + db/column_family.cc \ + db/compacted_db_impl.cc \ + db/compaction.cc \ + db/compaction_iterator.cc \ + db/compaction_job.cc \ + db/compaction_picker.cc \ + db/compaction_picker_universal.cc \ + db/convenience.cc \ + db/db_filesnapshot.cc \ + db/db_impl.cc \ + db/db_impl_write.cc \ + db/db_impl_compaction_flush.cc \ + db/db_impl_files.cc \ + db/db_impl_open.cc \ + db/db_impl_debug.cc \ + db/db_impl_experimental.cc \ + db/db_impl_readonly.cc \ + db/db_info_dumper.cc \ + db/db_iter.cc \ + db/dbformat.cc \ + db/event_helpers.cc \ + db/experimental.cc \ + db/external_sst_file_ingestion_job.cc \ + db/file_indexer.cc \ + db/flush_job.cc \ + db/flush_scheduler.cc \ + db/forward_iterator.cc \ + db/internal_stats.cc \ + db/log_reader.cc \ + db/log_writer.cc \ + db/malloc_stats.cc \ + db/managed_iterator.cc \ + db/memtable.cc \ + db/memtable_list.cc \ + db/merge_helper.cc \ + db/merge_operator.cc \ + db/range_del_aggregator.cc \ + db/repair.cc \ + db/snapshot_impl.cc \ + db/table_cache.cc \ + db/table_properties_collector.cc \ + db/transaction_log_impl.cc \ + db/version_builder.cc \ + db/version_edit.cc \ + db/version_set.cc \ + db/wal_manager.cc \ + db/write_batch.cc \ + db/write_batch_base.cc \ + db/write_controller.cc \ + db/write_thread.cc \ + env/env.cc \ + env/env_chroot.cc \ + env/env_encryption.cc \ + env/env_hdfs.cc \ + env/env_posix.cc \ + env/io_posix.cc \ + env/mock_env.cc \ + memtable/alloc_tracker.cc \ + memtable/hash_cuckoo_rep.cc \ + memtable/hash_linklist_rep.cc \ + memtable/hash_skiplist_rep.cc \ + memtable/skiplistrep.cc \ + memtable/vectorrep.cc \ + memtable/write_buffer_manager.cc \ + monitoring/histogram.cc \ + monitoring/histogram_windowing.cc \ + monitoring/instrumented_mutex.cc \ + monitoring/iostats_context.cc \ + monitoring/perf_context.cc \ + monitoring/perf_level.cc \ + monitoring/statistics.cc \ + monitoring/thread_status_impl.cc \ + monitoring/thread_status_updater.cc \ + monitoring/thread_status_updater_debug.cc \ + monitoring/thread_status_util.cc \ + monitoring/thread_status_util_debug.cc \ + options/cf_options.cc \ + options/db_options.cc \ + options/options.cc \ + options/options_helper.cc \ + options/options_parser.cc \ + options/options_sanity_check.cc \ + port/port_posix.cc \ + port/stack_trace.cc \ + table/adaptive_table_factory.cc \ + table/block.cc \ + table/block_based_filter_block.cc \ + table/block_based_table_builder.cc \ + table/block_based_table_factory.cc \ + table/block_based_table_reader.cc \ + table/block_builder.cc \ + table/block_prefix_index.cc \ + table/bloom_block.cc \ + table/cuckoo_table_builder.cc \ + table/cuckoo_table_factory.cc \ + table/cuckoo_table_reader.cc \ + table/flush_block_policy.cc \ + table/format.cc \ + table/full_filter_block.cc \ + table/get_context.cc \ + table/index_builder.cc \ + table/iterator.cc \ + table/merging_iterator.cc \ + table/meta_blocks.cc \ + table/partitioned_filter_block.cc \ + table/persistent_cache_helper.cc \ + table/plain_table_builder.cc \ + table/plain_table_factory.cc \ + table/plain_table_index.cc \ + table/plain_table_key_coding.cc \ + table/plain_table_reader.cc \ + table/sst_file_writer.cc \ + table/table_properties.cc \ + table/two_level_iterator.cc \ + tools/dump/db_dump_tool.cc \ + util/arena.cc \ + util/auto_roll_logger.cc \ + util/bloom.cc \ + util/build_version.cc \ + util/coding.cc \ + util/compaction_job_stats_impl.cc \ + util/comparator.cc \ + util/concurrent_arena.cc \ + util/crc32c.cc \ + util/delete_scheduler.cc \ + util/dynamic_bloom.cc \ + util/event_logger.cc \ + util/file_reader_writer.cc \ + util/file_util.cc \ + util/filename.cc \ + util/filter_policy.cc \ + util/hash.cc \ + util/log_buffer.cc \ + util/murmurhash.cc \ + util/random.cc \ + util/rate_limiter.cc \ + util/slice.cc \ + util/sst_file_manager_impl.cc \ + util/status.cc \ + util/status_message.cc \ + util/string_util.cc \ + util/sync_point.cc \ + util/thread_local.cc \ + util/threadpool_imp.cc \ + util/transaction_test_util.cc \ + util/xxhash.cc \ + utilities/backupable/backupable_db.cc \ + utilities/blob_db/blob_db.cc \ + utilities/blob_db/blob_db_impl.cc \ + utilities/blob_db/blob_file.cc \ + utilities/blob_db/blob_log_reader.cc \ + utilities/blob_db/blob_log_writer.cc \ + utilities/blob_db/blob_log_format.cc \ + utilities/blob_db/ttl_extractor.cc \ + utilities/cassandra/cassandra_compaction_filter.cc \ + utilities/cassandra/format.cc \ + utilities/cassandra/merge_operator.cc \ + utilities/checkpoint/checkpoint_impl.cc \ + utilities/compaction_filters/remove_emptyvalue_compactionfilter.cc \ + utilities/convenience/info_log_finder.cc \ + utilities/date_tiered/date_tiered_db_impl.cc \ + utilities/debug.cc \ + utilities/document/document_db.cc \ + utilities/document/json_document.cc \ + utilities/document/json_document_builder.cc \ + utilities/env_mirror.cc \ + utilities/env_timed.cc \ + utilities/geodb/geodb_impl.cc \ + utilities/leveldb_options/leveldb_options.cc \ + utilities/lua/rocks_lua_compaction_filter.cc \ + utilities/memory/memory_util.cc \ + utilities/merge_operators/max.cc \ + utilities/merge_operators/put.cc \ + utilities/merge_operators/string_append/stringappend.cc \ + utilities/merge_operators/string_append/stringappend2.cc \ + utilities/merge_operators/uint64add.cc \ + utilities/option_change_migration/option_change_migration.cc \ + utilities/options/options_util.cc \ + utilities/persistent_cache/block_cache_tier.cc \ + utilities/persistent_cache/block_cache_tier_file.cc \ + utilities/persistent_cache/block_cache_tier_metadata.cc \ + utilities/persistent_cache/persistent_cache_tier.cc \ + utilities/persistent_cache/volatile_tier_impl.cc \ + utilities/redis/redis_lists.cc \ + utilities/simulator_cache/sim_cache.cc \ + utilities/spatialdb/spatial_db.cc \ + utilities/table_properties_collectors/compact_on_deletion_collector.cc \ + utilities/transactions/optimistic_transaction_db_impl.cc \ + utilities/transactions/optimistic_transaction.cc \ + utilities/transactions/transaction_base.cc \ + utilities/transactions/pessimistic_transaction_db.cc \ + utilities/transactions/transaction_db_mutex_impl.cc \ + utilities/transactions/pessimistic_transaction.cc \ + utilities/transactions/transaction_lock_mgr.cc \ + utilities/transactions/transaction_util.cc \ + utilities/transactions/write_prepared_txn.cc \ + utilities/ttl/db_ttl_impl.cc \ + utilities/write_batch_with_index/write_batch_with_index.cc \ + utilities/write_batch_with_index/write_batch_with_index_internal.cc \ + +TOOL_LIB_SOURCES = \ + tools/ldb_cmd.cc \ + tools/ldb_tool.cc \ + tools/sst_dump_tool.cc \ + utilities/blob_db/blob_dump_tool.cc \ + +MOCK_LIB_SOURCES = \ + table/mock_table.cc \ + util/fault_injection_test_env.cc + +BENCH_LIB_SOURCES = \ + tools/db_bench_tool.cc \ + +EXP_LIB_SOURCES = \ + utilities/col_buf_encoder.cc \ + utilities/col_buf_decoder.cc \ + utilities/column_aware_encoding_util.cc + +TEST_LIB_SOURCES = \ + util/testharness.cc \ + util/testutil.cc \ + db/db_test_util.cc \ + utilities/cassandra/test_utils.cc \ + +MAIN_SOURCES = \ + cache/cache_bench.cc \ + cache/cache_test.cc \ + db/column_family_test.cc \ + db/compaction_job_stats_test.cc \ + db/compaction_job_test.cc \ + db/compaction_picker_test.cc \ + db/comparator_db_test.cc \ + db/corruption_test.cc \ + db/cuckoo_table_db_test.cc \ + db/db_basic_test.cc \ + db/db_block_cache_test.cc \ + db/db_bloom_filter_test.cc \ + db/db_compaction_filter_test.cc \ + db/db_compaction_test.cc \ + db/db_dynamic_level_test.cc \ + db/db_encryption_test.cc \ + db/db_flush_test.cc \ + db/db_inplace_update_test.cc \ + db/db_io_failure_test.cc \ + db/db_iter_test.cc \ + db/db_iterator_test.cc \ + db/db_log_iter_test.cc \ + db/db_memtable_test.cc \ + db/db_merge_operator_test.cc \ + db/db_options_test.cc \ + db/db_range_del_test.cc \ + db/db_sst_test.cc \ + db/db_statistics_test.cc \ + db/db_table_properties_test.cc \ + db/db_tailing_iter_test.cc \ + db/db_test.cc \ + db/db_universal_compaction_test.cc \ + db/db_wal_test.cc \ + db/db_write_test.cc \ + db/dbformat_test.cc \ + db/deletefile_test.cc \ + db/external_sst_file_basic_test.cc \ + db/external_sst_file_test.cc \ + db/fault_injection_test.cc \ + db/file_indexer_test.cc \ + db/filename_test.cc \ + db/flush_job_test.cc \ + db/listener_test.cc \ + db/log_test.cc \ + db/manual_compaction_test.cc \ + db/merge_test.cc \ + db/options_file_test.cc \ + db/perf_context_test.cc \ + db/plain_table_db_test.cc \ + db/prefix_test.cc \ + db/table_properties_collector_test.cc \ + db/version_builder_test.cc \ + db/version_edit_test.cc \ + db/version_set_test.cc \ + db/wal_manager_test.cc \ + db/write_batch_test.cc \ + db/write_callback_test.cc \ + db/write_controller_test.cc \ + env/env_basic_test.cc \ + env/env_test.cc \ + env/mock_env_test.cc \ + memtable/inlineskiplist_test.cc \ + memtable/memtablerep_bench.cc \ + memtable/skiplist_test.cc \ + memtable/write_buffer_manager_test.cc \ + monitoring/histogram_test.cc \ + monitoring/iostats_context_test.cc \ + monitoring/statistics_test.cc \ + options/options_test.cc \ + table/block_based_filter_block_test.cc \ + table/block_test.cc \ + table/cleanable_test.cc \ + table/cuckoo_table_builder_test.cc \ + table/cuckoo_table_reader_test.cc \ + table/full_filter_block_test.cc \ + table/merger_test.cc \ + table/table_reader_bench.cc \ + table/table_test.cc \ + third-party/gtest-1.7.0/fused-src/gtest/gtest-all.cc \ + tools/db_bench.cc \ + tools/db_bench_tool_test.cc \ + tools/db_sanity_test.cc \ + tools/ldb_cmd_test.cc \ + tools/reduce_levels_test.cc \ + tools/sst_dump_test.cc \ + util/arena_test.cc \ + util/auto_roll_logger_test.cc \ + util/autovector_test.cc \ + util/bloom_test.cc \ + util/coding_test.cc \ + util/crc32c_test.cc \ + util/dynamic_bloom_test.cc \ + util/event_logger_test.cc \ + util/filelock_test.cc \ + util/log_write_bench.cc \ + util/rate_limiter_test.cc \ + util/slice_transform_test.cc \ + util/timer_queue_test.cc \ + util/thread_list_test.cc \ + util/thread_local_test.cc \ + utilities/backupable/backupable_db_test.cc \ + utilities/blob_db/blob_db_test.cc \ + utilities/cassandra/cassandra_format_test.cc \ + utilities/cassandra/cassandra_functional_test.cc \ + utilities/cassandra/cassandra_row_merge_test.cc \ + utilities/cassandra/cassandra_serialize_test.cc \ + utilities/checkpoint/checkpoint_test.cc \ + utilities/column_aware_encoding_exp.cc \ + utilities/column_aware_encoding_test.cc \ + utilities/date_tiered/date_tiered_test.cc \ + utilities/document/document_db_test.cc \ + utilities/document/json_document_test.cc \ + utilities/geodb/geodb_test.cc \ + utilities/lua/rocks_lua_test.cc \ + utilities/memory/memory_test.cc \ + utilities/merge_operators/string_append/stringappend_test.cc \ + utilities/object_registry_test.cc \ + utilities/option_change_migration/option_change_migration_test.cc \ + utilities/options/options_util_test.cc \ + utilities/redis/redis_lists_test.cc \ + utilities/simulator_cache/sim_cache_test.cc \ + utilities/spatialdb/spatial_db_test.cc \ + utilities/table_properties_collectors/compact_on_deletion_collector_test.cc \ + utilities/transactions/optimistic_transaction_test.cc \ + utilities/transactions/transaction_test.cc \ + utilities/ttl/ttl_test.cc \ + utilities/write_batch_with_index/write_batch_with_index_test.cc \ + +JNI_NATIVE_SOURCES = \ + java/rocksjni/backupenginejni.cc \ + java/rocksjni/backupablejni.cc \ + java/rocksjni/checkpoint.cc \ + java/rocksjni/clock_cache.cc \ + java/rocksjni/columnfamilyhandle.cc \ + java/rocksjni/compaction_filter.cc \ + java/rocksjni/compaction_options_fifo.cc \ + java/rocksjni/compaction_options_universal.cc \ + java/rocksjni/comparator.cc \ + java/rocksjni/comparatorjnicallback.cc \ + java/rocksjni/compression_options.cc \ + java/rocksjni/env.cc \ + java/rocksjni/env_options.cc \ + java/rocksjni/ingest_external_file_options.cc \ + java/rocksjni/filter.cc \ + java/rocksjni/iterator.cc \ + java/rocksjni/loggerjnicallback.cc \ + java/rocksjni/lru_cache.cc \ + java/rocksjni/memtablejni.cc \ + java/rocksjni/merge_operator.cc \ + java/rocksjni/options.cc \ + java/rocksjni/ratelimiterjni.cc \ + java/rocksjni/remove_emptyvalue_compactionfilterjni.cc \ + java/rocksjni/cassandra_compactionfilterjni.cc \ + java/rocksjni/restorejni.cc \ + java/rocksjni/rocksjni.cc \ + java/rocksjni/rocksdb_exception_test.cc \ + java/rocksjni/slice.cc \ + java/rocksjni/snapshot.cc \ + java/rocksjni/sst_file_writerjni.cc \ + java/rocksjni/statistics.cc \ + java/rocksjni/statisticsjni.cc \ + java/rocksjni/table.cc \ + java/rocksjni/transaction_log.cc \ + java/rocksjni/ttl.cc \ + java/rocksjni/write_batch.cc \ + java/rocksjni/writebatchhandlerjnicallback.cc \ + java/rocksjni/write_batch_test.cc \ + java/rocksjni/write_batch_with_index.cc diff --git a/table/adaptive_table_factory.cc b/table/adaptive_table_factory.cc index a259e79d8a5..47069f86695 100644 --- a/table/adaptive_table_factory.cc +++ b/table/adaptive_table_factory.cc @@ -5,7 +5,9 @@ #ifndef ROCKSDB_LITE #include "table/adaptive_table_factory.h" +#include "table/table_builder.h" #include "table/format.h" +#include "port/port.h" namespace rocksdb { @@ -18,9 +20,6 @@ AdaptiveTableFactory::AdaptiveTableFactory( block_based_table_factory_(block_based_table_factory), plain_table_factory_(plain_table_factory), cuckoo_table_factory_(cuckoo_table_factory) { - if (!table_factory_to_write_) { - table_factory_to_write_ = block_based_table_factory_; - } if (!plain_table_factory_) { plain_table_factory_.reset(NewPlainTableFactory()); } @@ -30,6 +29,9 @@ AdaptiveTableFactory::AdaptiveTableFactory( if (!cuckoo_table_factory_) { cuckoo_table_factory_.reset(NewCuckooTableFactory()); } + if (!table_factory_to_write_) { + table_factory_to_write_ = block_based_table_factory_; + } } extern const uint64_t kPlainTableMagicNumber; @@ -39,35 +41,37 @@ extern const uint64_t kLegacyBlockBasedTableMagicNumber; extern const uint64_t kCuckooTableMagicNumber; Status AdaptiveTableFactory::NewTableReader( - const Options& options, const EnvOptions& soptions, - const InternalKeyComparator& icomp, unique_ptr&& file, - uint64_t file_size, unique_ptr* table) const { + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + unique_ptr* table, + bool prefetch_index_and_filter_in_cache) const { Footer footer; - auto s = ReadFooterFromFile(file.get(), file_size, &footer); + auto s = ReadFooterFromFile(file.get(), nullptr /* prefetch_buffer */, + file_size, &footer); if (!s.ok()) { return s; } if (footer.table_magic_number() == kPlainTableMagicNumber || footer.table_magic_number() == kLegacyPlainTableMagicNumber) { return plain_table_factory_->NewTableReader( - options, soptions, icomp, std::move(file), file_size, table); + table_reader_options, std::move(file), file_size, table); } else if (footer.table_magic_number() == kBlockBasedTableMagicNumber || footer.table_magic_number() == kLegacyBlockBasedTableMagicNumber) { return block_based_table_factory_->NewTableReader( - options, soptions, icomp, std::move(file), file_size, table); + table_reader_options, std::move(file), file_size, table); } else if (footer.table_magic_number() == kCuckooTableMagicNumber) { return cuckoo_table_factory_->NewTableReader( - options, soptions, icomp, std::move(file), file_size, table); + table_reader_options, std::move(file), file_size, table); } else { return Status::NotSupported("Unidentified table format"); } } TableBuilder* AdaptiveTableFactory::NewTableBuilder( - const Options& options, const InternalKeyComparator& internal_comparator, - WritableFile* file, CompressionType compression_type) const { - return table_factory_to_write_->NewTableBuilder(options, internal_comparator, - file, compression_type); + const TableBuilderOptions& table_builder_options, uint32_t column_family_id, + WritableFileWriter* file) const { + return table_factory_to_write_->NewTableBuilder(table_builder_options, + column_family_id, file); } std::string AdaptiveTableFactory::GetPrintableTableOptions() const { @@ -76,27 +80,30 @@ std::string AdaptiveTableFactory::GetPrintableTableOptions() const { const int kBufferSize = 200; char buffer[kBufferSize]; - if (!table_factory_to_write_) { + if (table_factory_to_write_) { snprintf(buffer, kBufferSize, " write factory (%s) options:\n%s\n", - table_factory_to_write_->Name(), + (table_factory_to_write_->Name() ? table_factory_to_write_->Name() + : ""), table_factory_to_write_->GetPrintableTableOptions().c_str()); ret.append(buffer); } - if (!plain_table_factory_) { + if (plain_table_factory_) { snprintf(buffer, kBufferSize, " %s options:\n%s\n", - plain_table_factory_->Name(), + plain_table_factory_->Name() ? plain_table_factory_->Name() : "", plain_table_factory_->GetPrintableTableOptions().c_str()); ret.append(buffer); } - if (!block_based_table_factory_) { - snprintf(buffer, kBufferSize, " %s options:\n%s\n", - block_based_table_factory_->Name(), - block_based_table_factory_->GetPrintableTableOptions().c_str()); + if (block_based_table_factory_) { + snprintf( + buffer, kBufferSize, " %s options:\n%s\n", + (block_based_table_factory_->Name() ? block_based_table_factory_->Name() + : ""), + block_based_table_factory_->GetPrintableTableOptions().c_str()); ret.append(buffer); } - if (!cuckoo_table_factory_) { + if (cuckoo_table_factory_) { snprintf(buffer, kBufferSize, " %s options:\n%s\n", - cuckoo_table_factory_->Name(), + cuckoo_table_factory_->Name() ? cuckoo_table_factory_->Name() : "", cuckoo_table_factory_->GetPrintableTableOptions().c_str()); ret.append(buffer); } diff --git a/table/adaptive_table_factory.h b/table/adaptive_table_factory.h index f119d97b1cd..b7b52ba96fc 100644 --- a/table/adaptive_table_factory.h +++ b/table/adaptive_table_factory.h @@ -12,7 +12,6 @@ namespace rocksdb { -struct Options; struct EnvOptions; using std::unique_ptr; @@ -31,23 +30,22 @@ class AdaptiveTableFactory : public TableFactory { std::shared_ptr block_based_table_factory, std::shared_ptr plain_table_factory, std::shared_ptr cuckoo_table_factory); + const char* Name() const override { return "AdaptiveTableFactory"; } - Status NewTableReader(const Options& options, const EnvOptions& soptions, - const InternalKeyComparator& internal_comparator, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table) const override; - TableBuilder* NewTableBuilder(const Options& options, - const InternalKeyComparator& icomparator, - WritableFile* file, - CompressionType compression_type) const - override; + + Status NewTableReader( + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + unique_ptr* table, + bool prefetch_index_and_filter_in_cache = true) const override; + + TableBuilder* NewTableBuilder( + const TableBuilderOptions& table_builder_options, + uint32_t column_family_id, WritableFileWriter* file) const override; // Sanitizes the specified DB Options. - Status SanitizeDBOptions(const DBOptions* db_opts) const override { - if (db_opts->allow_mmap_reads == false) { - return Status::NotSupported( - "AdaptiveTable with allow_mmap_reads == false is not supported."); - } + Status SanitizeOptions(const DBOptions& db_opts, + const ColumnFamilyOptions& cf_opts) const override { return Status::OK(); } diff --git a/table/block.cc b/table/block.cc index 0db23a1bd80..372bbd2f0b5 100644 --- a/table/block.cc +++ b/table/block.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,16 +10,17 @@ // Decodes the blocks generated by block_builder.cc. #include "table/block.h" - #include #include #include #include +#include "monitoring/perf_context_imp.h" +#include "port/port.h" +#include "port/stack_trace.h" #include "rocksdb/comparator.h" -#include "table/format.h" -#include "table/block_hash_index.h" #include "table/block_prefix_index.h" +#include "table/format.h" #include "util/coding.h" #include "util/logging.h" @@ -63,6 +64,40 @@ void BlockIter::Next() { void BlockIter::Prev() { assert(Valid()); + assert(prev_entries_idx_ == -1 || + static_cast(prev_entries_idx_) < prev_entries_.size()); + // Check if we can use cached prev_entries_ + if (prev_entries_idx_ > 0 && + prev_entries_[prev_entries_idx_].offset == current_) { + // Read cached CachedPrevEntry + prev_entries_idx_--; + const CachedPrevEntry& current_prev_entry = + prev_entries_[prev_entries_idx_]; + + const char* key_ptr = nullptr; + if (current_prev_entry.key_ptr != nullptr) { + // The key is not delta encoded and stored in the data block + key_ptr = current_prev_entry.key_ptr; + key_pinned_ = true; + } else { + // The key is delta encoded and stored in prev_entries_keys_buff_ + key_ptr = prev_entries_keys_buff_.data() + current_prev_entry.key_offset; + key_pinned_ = false; + } + const Slice current_key(key_ptr, current_prev_entry.key_size); + + current_ = current_prev_entry.offset; + key_.SetInternalKey(current_key, false /* copy */); + value_ = current_prev_entry.value; + + return; + } + + // Clear prev entries cache + prev_entries_idx_ = -1; + prev_entries_.clear(); + prev_entries_keys_buff_.clear(); + // Scan backwards to a restart point before current_ const uint32_t original = current_; while (GetRestartPoint(restart_index_) >= original) { @@ -76,12 +111,32 @@ void BlockIter::Prev() { } SeekToRestartPoint(restart_index_); + do { + if (!ParseNextKey()) { + break; + } + Slice current_key = key(); + + if (key_.IsKeyPinned()) { + // The key is not delta encoded + prev_entries_.emplace_back(current_, current_key.data(), 0, + current_key.size(), value()); + } else { + // The key is delta encoded, cache decoded key in buffer + size_t new_key_offset = prev_entries_keys_buff_.size(); + prev_entries_keys_buff_.append(current_key.data(), current_key.size()); + + prev_entries_.emplace_back(current_, nullptr, new_key_offset, + current_key.size(), value()); + } // Loop until end of current entry hits the start of original entry - } while (ParseNextKey() && NextEntryOffset() < original); + } while (NextEntryOffset() < original); + prev_entries_idx_ = static_cast(prev_entries_.size()) - 1; } void BlockIter::Seek(const Slice& target) { + PERF_TIMER_GUARD(block_seek_nanos); if (data_ == nullptr) { // Not init yet return; } @@ -90,8 +145,7 @@ void BlockIter::Seek(const Slice& target) { if (prefix_index_) { ok = PrefixSeek(target, &index); } else { - ok = hash_index_ ? HashSeek(target, &index) - : BinarySeek(target, 0, num_restarts_ - 1, &index); + ok = BinarySeek(target, 0, num_restarts_ - 1, &index); } if (!ok) { @@ -101,12 +155,38 @@ void BlockIter::Seek(const Slice& target) { // Linear search (within restart block) for first key >= target while (true) { - if (!ParseNextKey() || Compare(key_.GetKey(), target) >= 0) { + if (!ParseNextKey() || Compare(key_.GetInternalKey(), target) >= 0) { return; } } } +void BlockIter::SeekForPrev(const Slice& target) { + PERF_TIMER_GUARD(block_seek_nanos); + if (data_ == nullptr) { // Not init yet + return; + } + uint32_t index = 0; + bool ok = false; + ok = BinarySeek(target, 0, num_restarts_ - 1, &index); + + if (!ok) { + return; + } + SeekToRestartPoint(index); + // Linear search (within restart block) for first key >= target + + while (ParseNextKey() && Compare(key_.GetInternalKey(), target) < 0) { + } + if (!Valid()) { + SeekToLast(); + } else { + while (Valid() && Compare(key_.GetInternalKey(), target) > 0) { + Prev(); + } + } +} + void BlockIter::SeekToFirst() { if (data_ == nullptr) { // Not init yet return; @@ -134,46 +214,81 @@ void BlockIter::CorruptionError() { } bool BlockIter::ParseNextKey() { - current_ = NextEntryOffset(); - const char* p = data_ + current_; - const char* limit = data_ + restarts_; // Restarts come right after data - if (p >= limit) { - // No more entries to return. Mark as invalid. - current_ = restarts_; - restart_index_ = num_restarts_; - return false; - } + current_ = NextEntryOffset(); + const char* p = data_ + current_; + const char* limit = data_ + restarts_; // Restarts come right after data + if (p >= limit) { + // No more entries to return. Mark as invalid. + current_ = restarts_; + restart_index_ = num_restarts_; + return false; + } - // Decode next entry - uint32_t shared, non_shared, value_length; - p = DecodeEntry(p, limit, &shared, &non_shared, &value_length); - if (p == nullptr || key_.Size() < shared) { - CorruptionError(); - return false; + // Decode next entry + uint32_t shared, non_shared, value_length; + p = DecodeEntry(p, limit, &shared, &non_shared, &value_length); + if (p == nullptr || key_.Size() < shared) { + CorruptionError(); + return false; + } else { + if (shared == 0) { + // If this key dont share any bytes with prev key then we dont need + // to decode it and can use it's address in the block directly. + key_.SetInternalKey(Slice(p, non_shared), false /* copy */); + key_pinned_ = true; } else { + // This key share `shared` bytes with prev key, we need to decode it key_.TrimAppend(shared, p, non_shared); - value_ = Slice(p + non_shared, value_length); - while (restart_index_ + 1 < num_restarts_ && - GetRestartPoint(restart_index_ + 1) < current_) { - ++restart_index_; + key_pinned_ = false; + } + + if (global_seqno_ != kDisableGlobalSequenceNumber) { + // If we are reading a file with a global sequence number we should + // expect that all encoded sequence numbers are zeros and any value + // type is kTypeValue, kTypeMerge or kTypeDeletion + assert(GetInternalKeySeqno(key_.GetInternalKey()) == 0); + + ValueType value_type = ExtractValueType(key_.GetInternalKey()); + assert(value_type == ValueType::kTypeValue || + value_type == ValueType::kTypeMerge || + value_type == ValueType::kTypeDeletion); + + if (key_pinned_) { + // TODO(tec): Investigate updating the seqno in the loaded block + // directly instead of doing a copy and update. + + // We cannot use the key address in the block directly because + // we have a global_seqno_ that will overwrite the encoded one. + key_.OwnKey(); + key_pinned_ = false; } - return true; + + key_.UpdateInternalKey(global_seqno_, value_type); } + + value_ = Slice(p + non_shared, value_length); + while (restart_index_ + 1 < num_restarts_ && + GetRestartPoint(restart_index_ + 1) < current_) { + ++restart_index_; + } + return true; } +} -// Binary search in restart array to find the first restart point -// with a key >= target (TODO: this comment is inaccurate) +// Binary search in restart array to find the first restart point that +// is either the last restart point with a key less than target, +// which means the key of next restart point is larger than target, or +// the first restart point with a key = target bool BlockIter::BinarySeek(const Slice& target, uint32_t left, uint32_t right, - uint32_t* index) { + uint32_t* index) { assert(left <= right); while (left < right) { uint32_t mid = (left + right + 1) / 2; uint32_t region_offset = GetRestartPoint(mid); uint32_t shared, non_shared, value_length; - const char* key_ptr = - DecodeEntry(data_ + region_offset, data_ + restarts_, &shared, - &non_shared, &value_length); + const char* key_ptr = DecodeEntry(data_ + region_offset, data_ + restarts_, + &shared, &non_shared, &value_length); if (key_ptr == nullptr || (shared != 0)) { CorruptionError(); return false; @@ -215,13 +330,13 @@ int BlockIter::CompareBlockKey(uint32_t block_index, const Slice& target) { // Binary search in block_ids to find the first block // with a key >= target bool BlockIter::BinaryBlockIndexSeek(const Slice& target, uint32_t* block_ids, - uint32_t left, uint32_t right, - uint32_t* index) { + uint32_t left, uint32_t right, + uint32_t* index) { assert(left <= right); uint32_t left_bound = left; while (left <= right) { - uint32_t mid = (left + right) / 2; + uint32_t mid = (right + left) / 2; int cmp = CompareBlockKey(block_ids[mid], target); if (!status_.ok()) { @@ -264,21 +379,6 @@ bool BlockIter::BinaryBlockIndexSeek(const Slice& target, uint32_t* block_ids, } } -bool BlockIter::HashSeek(const Slice& target, uint32_t* index) { - assert(hash_index_); - auto restart_index = hash_index_->GetRestartIndex(target); - if (restart_index == nullptr) { - current_ = restarts_; - return false; - } - - // the elements in restart_array[index : index + num_blocks] - // are all with same prefix. We'll do binary search in that small range. - auto left = restart_index->first_index; - auto right = restart_index->first_index + restart_index->num_blocks - 1; - return BinarySeek(target, left, right, index); -} - bool BlockIter::PrefixSeek(const Slice& target, uint32_t* index) { assert(prefix_index_); uint32_t* block_ids = nullptr; @@ -297,38 +397,37 @@ uint32_t Block::NumRestarts() const { return DecodeFixed32(data_ + size_ - sizeof(uint32_t)); } -Block::Block(const BlockContents& contents) - : data_(contents.data.data()), - size_(contents.data.size()), - owned_(contents.heap_allocated), - cachable_(contents.cachable), - compression_type_(contents.compression_type) { +Block::Block(BlockContents&& contents, SequenceNumber _global_seqno, + size_t read_amp_bytes_per_bit, Statistics* statistics) + : contents_(std::move(contents)), + data_(contents_.data.data()), + size_(contents_.data.size()), + global_seqno_(_global_seqno) { if (size_ < sizeof(uint32_t)) { size_ = 0; // Error marker } else { - restart_offset_ = size_ - (1 + NumRestarts()) * sizeof(uint32_t); + restart_offset_ = + static_cast(size_) - (1 + NumRestarts()) * sizeof(uint32_t); if (restart_offset_ > size_ - sizeof(uint32_t)) { // The size is too small for NumRestarts() and therefore // restart_offset_ wrapped around. size_ = 0; } } -} - -Block::~Block() { - if (owned_) { - delete[] data_; + if (read_amp_bytes_per_bit != 0 && statistics && size_ != 0) { + read_amp_bitmap_.reset(new BlockReadAmpBitmap( + restart_offset_, read_amp_bytes_per_bit, statistics)); } } -Iterator* Block::NewIterator( - const Comparator* cmp, BlockIter* iter, bool total_order_seek) { +InternalIterator* Block::NewIterator(const Comparator* cmp, BlockIter* iter, + bool total_order_seek, Statistics* stats) { if (size_ < 2*sizeof(uint32_t)) { if (iter != nullptr) { iter->SetStatus(Status::Corruption("bad block contents")); return iter; } else { - return NewErrorIterator(Status::Corruption("bad block contents")); + return NewErrorInternalIterator(Status::Corruption("bad block contents")); } } const uint32_t num_restarts = NumRestarts(); @@ -337,39 +436,38 @@ Iterator* Block::NewIterator( iter->SetStatus(Status::OK()); return iter; } else { - return NewEmptyIterator(); + return NewEmptyInternalIterator(); } } else { - BlockHashIndex* hash_index_ptr = - total_order_seek ? nullptr : hash_index_.get(); BlockPrefixIndex* prefix_index_ptr = total_order_seek ? nullptr : prefix_index_.get(); if (iter != nullptr) { iter->Initialize(cmp, data_, restart_offset_, num_restarts, - hash_index_ptr, prefix_index_ptr); + prefix_index_ptr, global_seqno_, read_amp_bitmap_.get()); } else { iter = new BlockIter(cmp, data_, restart_offset_, num_restarts, - hash_index_ptr, prefix_index_ptr); + prefix_index_ptr, global_seqno_, + read_amp_bitmap_.get()); + } + + if (read_amp_bitmap_) { + if (read_amp_bitmap_->GetStatistics() != stats) { + // DB changed the Statistics pointer, we need to notify read_amp_bitmap_ + read_amp_bitmap_->SetStatistics(stats); + } } } return iter; } -void Block::SetBlockHashIndex(BlockHashIndex* hash_index) { - hash_index_.reset(hash_index); -} - void Block::SetBlockPrefixIndex(BlockPrefixIndex* prefix_index) { prefix_index_.reset(prefix_index); } size_t Block::ApproximateMemoryUsage() const { - size_t usage = size(); - if (hash_index_) { - usage += hash_index_->ApproximateMemoryUsage(); - } + size_t usage = usable_size(); if (prefix_index_) { usage += prefix_index_->ApproximateMemoryUsage(); } diff --git a/table/block.h b/table/block.h index 49bcf12cf30..59dc1674337 100644 --- a/table/block.h +++ b/table/block.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,31 +10,150 @@ #pragma once #include #include +#include +#include +#ifdef ROCKSDB_MALLOC_USABLE_SIZE +#ifdef OS_FREEBSD +#include +#else +#include +#endif +#endif +#include "db/dbformat.h" +#include "db/pinned_iterators_manager.h" #include "rocksdb/iterator.h" #include "rocksdb/options.h" -#include "db/dbformat.h" +#include "rocksdb/statistics.h" +#include "table/block_prefix_index.h" +#include "table/internal_iterator.h" +#include "util/random.h" +#include "util/sync_point.h" +#include "format.h" namespace rocksdb { struct BlockContents; class Comparator; class BlockIter; -class BlockHashIndex; class BlockPrefixIndex; +// BlockReadAmpBitmap is a bitmap that map the rocksdb::Block data bytes to +// a bitmap with ratio bytes_per_bit. Whenever we access a range of bytes in +// the Block we update the bitmap and increment READ_AMP_ESTIMATE_USEFUL_BYTES. +class BlockReadAmpBitmap { + public: + explicit BlockReadAmpBitmap(size_t block_size, size_t bytes_per_bit, + Statistics* statistics) + : bitmap_(nullptr), + bytes_per_bit_pow_(0), + statistics_(statistics), + rnd_( + Random::GetTLSInstance()->Uniform(static_cast(bytes_per_bit))) { + TEST_SYNC_POINT_CALLBACK("BlockReadAmpBitmap:rnd", &rnd_); + assert(block_size > 0 && bytes_per_bit > 0); + + // convert bytes_per_bit to be a power of 2 + while (bytes_per_bit >>= 1) { + bytes_per_bit_pow_++; + } + + // num_bits_needed = ceil(block_size / bytes_per_bit) + size_t num_bits_needed = + ((block_size - 1) >> bytes_per_bit_pow_) + 1; + assert(num_bits_needed > 0); + + // bitmap_size = ceil(num_bits_needed / kBitsPerEntry) + size_t bitmap_size = (num_bits_needed - 1) / kBitsPerEntry + 1; + + // Create bitmap and set all the bits to 0 + bitmap_ = new std::atomic[bitmap_size](); + + RecordTick(GetStatistics(), READ_AMP_TOTAL_READ_BYTES, block_size); + } + + ~BlockReadAmpBitmap() { delete[] bitmap_; } + + void Mark(uint32_t start_offset, uint32_t end_offset) { + assert(end_offset >= start_offset); + // Index of first bit in mask + uint32_t start_bit = + (start_offset + (1 << bytes_per_bit_pow_) - rnd_ - 1) >> + bytes_per_bit_pow_; + // Index of last bit in mask + 1 + uint32_t exclusive_end_bit = + (end_offset + (1 << bytes_per_bit_pow_) - rnd_) >> bytes_per_bit_pow_; + if (start_bit >= exclusive_end_bit) { + return; + } + assert(exclusive_end_bit > 0); + + if (GetAndSet(start_bit) == 0) { + uint32_t new_useful_bytes = (exclusive_end_bit - start_bit) + << bytes_per_bit_pow_; + RecordTick(GetStatistics(), READ_AMP_ESTIMATE_USEFUL_BYTES, + new_useful_bytes); + } + } + + Statistics* GetStatistics() { + return statistics_.load(std::memory_order_relaxed); + } + + void SetStatistics(Statistics* stats) { statistics_.store(stats); } + + uint32_t GetBytesPerBit() { return 1 << bytes_per_bit_pow_; } + + private: + // Get the current value of bit at `bit_idx` and set it to 1 + inline bool GetAndSet(uint32_t bit_idx) { + const uint32_t byte_idx = bit_idx / kBitsPerEntry; + const uint32_t bit_mask = 1 << (bit_idx % kBitsPerEntry); + + return bitmap_[byte_idx].fetch_or(bit_mask, std::memory_order_relaxed) & + bit_mask; + } + + const uint32_t kBytesPersEntry = sizeof(uint32_t); // 4 bytes + const uint32_t kBitsPerEntry = kBytesPersEntry * 8; // 32 bits + + // Bitmap used to record the bytes that we read, use atomic to protect + // against multiple threads updating the same bit + std::atomic* bitmap_; + // (1 << bytes_per_bit_pow_) is bytes_per_bit. Use power of 2 to optimize + // muliplication and division + uint8_t bytes_per_bit_pow_; + // Pointer to DB Statistics object, Since this bitmap may outlive the DB + // this pointer maybe invalid, but the DB will update it to a valid pointer + // by using SetStatistics() before calling Mark() + std::atomic statistics_; + uint32_t rnd_; +}; + class Block { public: // Initialize the block with the specified contents. - explicit Block(const BlockContents& contents); + explicit Block(BlockContents&& contents, SequenceNumber _global_seqno, + size_t read_amp_bytes_per_bit = 0, + Statistics* statistics = nullptr); - ~Block(); + ~Block() = default; size_t size() const { return size_; } const char* data() const { return data_; } - bool cachable() const { return cachable_; } + bool cachable() const { return contents_.cachable; } + size_t usable_size() const { +#ifdef ROCKSDB_MALLOC_USABLE_SIZE + if (contents_.allocation.get() != nullptr) { + return malloc_usable_size(contents_.allocation.get()); + } +#endif // ROCKSDB_MALLOC_USABLE_SIZE + return size_; + } uint32_t NumRestarts() const; - CompressionType compression_type() const { return compression_type_; } + CompressionType compression_type() const { + return contents_.compression_type; + } // If hash index lookup is enabled and `use_hash_index` is true. This block // will do hash lookup for the key prefix. @@ -49,30 +168,34 @@ class Block { // If total_order_seek is true, hash_index_ and prefix_index_ are ignored. // This option only applies for index block. For data block, hash_index_ // and prefix_index_ are null, so this option does not matter. - Iterator* NewIterator(const Comparator* comparator, - BlockIter* iter = nullptr, bool total_order_seek = true); - void SetBlockHashIndex(BlockHashIndex* hash_index); + InternalIterator* NewIterator(const Comparator* comparator, + BlockIter* iter = nullptr, + bool total_order_seek = true, + Statistics* stats = nullptr); void SetBlockPrefixIndex(BlockPrefixIndex* prefix_index); // Report an approximation of how much memory has been used. size_t ApproximateMemoryUsage() const; + SequenceNumber global_seqno() const { return global_seqno_; } + private: - const char* data_; - size_t size_; + BlockContents contents_; + const char* data_; // contents_.data.data() + size_t size_; // contents_.data.size() uint32_t restart_offset_; // Offset in data_ of restart array - bool owned_; // Block owns data_[] - bool cachable_; - CompressionType compression_type_; - std::unique_ptr hash_index_; std::unique_ptr prefix_index_; + std::unique_ptr read_amp_bitmap_; + // All keys in the block will have seqno = global_seqno_, regardless of + // the encoded value (kDisableGlobalSequenceNumber means disabled) + const SequenceNumber global_seqno_; // No copying allowed Block(const Block&); void operator=(const Block&); }; -class BlockIter : public Iterator { +class BlockIter : public InternalIterator { public: BlockIter() : comparator_(nullptr), @@ -82,20 +205,24 @@ class BlockIter : public Iterator { current_(0), restart_index_(0), status_(Status::OK()), - hash_index_(nullptr), - prefix_index_(nullptr) {} + prefix_index_(nullptr), + key_pinned_(false), + global_seqno_(kDisableGlobalSequenceNumber), + read_amp_bitmap_(nullptr), + last_bitmap_offset_(0) {} BlockIter(const Comparator* comparator, const char* data, uint32_t restarts, - uint32_t num_restarts, BlockHashIndex* hash_index, - BlockPrefixIndex* prefix_index) + uint32_t num_restarts, BlockPrefixIndex* prefix_index, + SequenceNumber global_seqno, BlockReadAmpBitmap* read_amp_bitmap) : BlockIter() { - Initialize(comparator, data, restarts, num_restarts, - hash_index, prefix_index); + Initialize(comparator, data, restarts, num_restarts, prefix_index, + global_seqno, read_amp_bitmap); } void Initialize(const Comparator* comparator, const char* data, - uint32_t restarts, uint32_t num_restarts, BlockHashIndex* hash_index, - BlockPrefixIndex* prefix_index) { + uint32_t restarts, uint32_t num_restarts, + BlockPrefixIndex* prefix_index, SequenceNumber global_seqno, + BlockReadAmpBitmap* read_amp_bitmap) { assert(data_ == nullptr); // Ensure it is called only once assert(num_restarts > 0); // Ensure the param is valid @@ -105,8 +232,10 @@ class BlockIter : public Iterator { num_restarts_ = num_restarts; current_ = restarts_; restart_index_ = num_restarts_; - hash_index_ = hash_index; prefix_index_ = prefix_index; + global_seqno_ = global_seqno; + read_amp_bitmap_ = read_amp_bitmap; + last_bitmap_offset_ = current_ + 1; } void SetStatus(Status s) { @@ -117,10 +246,16 @@ class BlockIter : public Iterator { virtual Status status() const override { return status_; } virtual Slice key() const override { assert(Valid()); - return key_.GetKey(); + return key_.GetInternalKey(); } virtual Slice value() const override { assert(Valid()); + if (read_amp_bitmap_ && current_ < restarts_ && + current_ != last_bitmap_offset_) { + read_amp_bitmap_->Mark(current_ /* current entry offset */, + NextEntryOffset() - 1); + last_bitmap_offset_ = current_; + } return value_; } @@ -130,10 +265,35 @@ class BlockIter : public Iterator { virtual void Seek(const Slice& target) override; + virtual void SeekForPrev(const Slice& target) override; + virtual void SeekToFirst() override; virtual void SeekToLast() override; +#ifndef NDEBUG + ~BlockIter() { + // Assert that the BlockIter is never deleted while Pinning is Enabled. + assert(!pinned_iters_mgr_ || + (pinned_iters_mgr_ && !pinned_iters_mgr_->PinningEnabled())); + } + virtual void SetPinnedItersMgr( + PinnedIteratorsManager* pinned_iters_mgr) override { + pinned_iters_mgr_ = pinned_iters_mgr; + } + PinnedIteratorsManager* pinned_iters_mgr_ = nullptr; +#endif + + virtual bool IsKeyPinned() const override { return key_pinned_; } + + virtual bool IsValuePinned() const override { return true; } + + size_t TEST_CurrentEntrySize() { return NextEntryOffset() - current_; } + + uint32_t ValueOffset() const { + return static_cast(value_.data() - data_); + } + private: const Comparator* comparator_; const char* data_; // underlying block contents @@ -146,8 +306,38 @@ class BlockIter : public Iterator { IterKey key_; Slice value_; Status status_; - BlockHashIndex* hash_index_; BlockPrefixIndex* prefix_index_; + bool key_pinned_; + SequenceNumber global_seqno_; + + // read-amp bitmap + BlockReadAmpBitmap* read_amp_bitmap_; + // last `current_` value we report to read-amp bitmp + mutable uint32_t last_bitmap_offset_; + + struct CachedPrevEntry { + explicit CachedPrevEntry(uint32_t _offset, const char* _key_ptr, + size_t _key_offset, size_t _key_size, Slice _value) + : offset(_offset), + key_ptr(_key_ptr), + key_offset(_key_offset), + key_size(_key_size), + value(_value) {} + + // offset of entry in block + uint32_t offset; + // Pointer to key data in block (nullptr if key is delta-encoded) + const char* key_ptr; + // offset of key in prev_entries_keys_buff_ (0 if key_ptr is not nullptr) + size_t key_offset; + // size of key + size_t key_size; + // value slice pointing to data in block + Slice value; + }; + std::string prev_entries_keys_buff_; + std::vector prev_entries_; + int32_t prev_entries_idx_ = -1; inline int Compare(const Slice& a, const Slice& b) const { return comparator_->Compare(a, b); @@ -155,7 +345,8 @@ class BlockIter : public Iterator { // Return the offset in data_ just past the end of the current entry. inline uint32_t NextEntryOffset() const { - return (value_.data() + value_.size()) - data_; + // NOTE: We don't support blocks bigger than 2GB + return static_cast((value_.data() + value_.size()) - data_); } uint32_t GetRestartPoint(uint32_t index) { @@ -186,8 +377,6 @@ class BlockIter : public Iterator { uint32_t left, uint32_t right, uint32_t* index); - bool HashSeek(const Slice& target, uint32_t* index); - bool PrefixSeek(const Slice& target, uint32_t* index); }; diff --git a/table/block_based_filter_block.cc b/table/block_based_filter_block.cc new file mode 100644 index 00000000000..697c11a42f0 --- /dev/null +++ b/table/block_based_filter_block.cc @@ -0,0 +1,255 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "table/block_based_filter_block.h" +#include + +#include "db/dbformat.h" +#include "monitoring/perf_context_imp.h" +#include "rocksdb/filter_policy.h" +#include "util/coding.h" +#include "util/string_util.h" + +namespace rocksdb { + +namespace { + +void AppendItem(std::string* props, const std::string& key, + const std::string& value) { + char cspace = ' '; + std::string value_str(""); + size_t i = 0; + const size_t dataLength = 64; + const size_t tabLength = 2; + const size_t offLength = 16; + + value_str.append(&value[i], std::min(size_t(dataLength), value.size())); + i += dataLength; + while (i < value.size()) { + value_str.append("\n"); + value_str.append(offLength, cspace); + value_str.append(&value[i], std::min(size_t(dataLength), value.size() - i)); + i += dataLength; + } + + std::string result(""); + if (key.size() < (offLength - tabLength)) + result.append(size_t((offLength - tabLength)) - key.size(), cspace); + result.append(key); + + props->append(result + ": " + value_str + "\n"); +} + +template +void AppendItem(std::string* props, const TKey& key, const std::string& value) { + std::string key_str = rocksdb::ToString(key); + AppendItem(props, key_str, value); +} +} // namespace + + +// See doc/table_format.txt for an explanation of the filter block format. + +// Generate new filter every 2KB of data +static const size_t kFilterBaseLg = 11; +static const size_t kFilterBase = 1 << kFilterBaseLg; + +BlockBasedFilterBlockBuilder::BlockBasedFilterBlockBuilder( + const SliceTransform* prefix_extractor, + const BlockBasedTableOptions& table_opt) + : policy_(table_opt.filter_policy.get()), + prefix_extractor_(prefix_extractor), + whole_key_filtering_(table_opt.whole_key_filtering), + prev_prefix_start_(0), + prev_prefix_size_(0) { + assert(policy_); +} + +void BlockBasedFilterBlockBuilder::StartBlock(uint64_t block_offset) { + uint64_t filter_index = (block_offset / kFilterBase); + assert(filter_index >= filter_offsets_.size()); + while (filter_index > filter_offsets_.size()) { + GenerateFilter(); + } +} + +void BlockBasedFilterBlockBuilder::Add(const Slice& key) { + if (prefix_extractor_ && prefix_extractor_->InDomain(key)) { + AddPrefix(key); + } + + if (whole_key_filtering_) { + AddKey(key); + } +} + +// Add key to filter if needed +inline void BlockBasedFilterBlockBuilder::AddKey(const Slice& key) { + start_.push_back(entries_.size()); + entries_.append(key.data(), key.size()); +} + +// Add prefix to filter if needed +inline void BlockBasedFilterBlockBuilder::AddPrefix(const Slice& key) { + // get slice for most recently added entry + Slice prev; + if (prev_prefix_size_ > 0) { + prev = Slice(entries_.data() + prev_prefix_start_, prev_prefix_size_); + } + + Slice prefix = prefix_extractor_->Transform(key); + // insert prefix only when it's different from the previous prefix. + if (prev.size() == 0 || prefix != prev) { + start_.push_back(entries_.size()); + prev_prefix_start_ = entries_.size(); + prev_prefix_size_ = prefix.size(); + entries_.append(prefix.data(), prefix.size()); + } +} + +Slice BlockBasedFilterBlockBuilder::Finish(const BlockHandle& tmp, + Status* status) { + // In this impl we ignore BlockHandle + *status = Status::OK(); + if (!start_.empty()) { + GenerateFilter(); + } + + // Append array of per-filter offsets + const uint32_t array_offset = static_cast(result_.size()); + for (size_t i = 0; i < filter_offsets_.size(); i++) { + PutFixed32(&result_, filter_offsets_[i]); + } + + PutFixed32(&result_, array_offset); + result_.push_back(kFilterBaseLg); // Save encoding parameter in result + return Slice(result_); +} + +void BlockBasedFilterBlockBuilder::GenerateFilter() { + const size_t num_entries = start_.size(); + if (num_entries == 0) { + // Fast path if there are no keys for this filter + filter_offsets_.push_back(static_cast(result_.size())); + return; + } + + // Make list of keys from flattened key structure + start_.push_back(entries_.size()); // Simplify length computation + tmp_entries_.resize(num_entries); + for (size_t i = 0; i < num_entries; i++) { + const char* base = entries_.data() + start_[i]; + size_t length = start_[i + 1] - start_[i]; + tmp_entries_[i] = Slice(base, length); + } + + // Generate filter for current set of keys and append to result_. + filter_offsets_.push_back(static_cast(result_.size())); + policy_->CreateFilter(&tmp_entries_[0], static_cast(num_entries), + &result_); + + tmp_entries_.clear(); + entries_.clear(); + start_.clear(); + prev_prefix_start_ = 0; + prev_prefix_size_ = 0; +} + +BlockBasedFilterBlockReader::BlockBasedFilterBlockReader( + const SliceTransform* prefix_extractor, + const BlockBasedTableOptions& table_opt, bool _whole_key_filtering, + BlockContents&& contents, Statistics* stats) + : FilterBlockReader(contents.data.size(), stats, _whole_key_filtering), + policy_(table_opt.filter_policy.get()), + prefix_extractor_(prefix_extractor), + data_(nullptr), + offset_(nullptr), + num_(0), + base_lg_(0), + contents_(std::move(contents)) { + assert(policy_); + size_t n = contents_.data.size(); + if (n < 5) return; // 1 byte for base_lg_ and 4 for start of offset array + base_lg_ = contents_.data[n - 1]; + uint32_t last_word = DecodeFixed32(contents_.data.data() + n - 5); + if (last_word > n - 5) return; + data_ = contents_.data.data(); + offset_ = data_ + last_word; + num_ = (n - 5 - last_word) / 4; +} + +bool BlockBasedFilterBlockReader::KeyMayMatch( + const Slice& key, uint64_t block_offset, const bool no_io, + const Slice* const const_ikey_ptr) { + assert(block_offset != kNotValid); + if (!whole_key_filtering_) { + return true; + } + return MayMatch(key, block_offset); +} + +bool BlockBasedFilterBlockReader::PrefixMayMatch( + const Slice& prefix, uint64_t block_offset, const bool no_io, + const Slice* const const_ikey_ptr) { + assert(block_offset != kNotValid); + if (!prefix_extractor_) { + return true; + } + return MayMatch(prefix, block_offset); +} + +bool BlockBasedFilterBlockReader::MayMatch(const Slice& entry, + uint64_t block_offset) { + uint64_t index = block_offset >> base_lg_; + if (index < num_) { + uint32_t start = DecodeFixed32(offset_ + index * 4); + uint32_t limit = DecodeFixed32(offset_ + index * 4 + 4); + if (start <= limit && limit <= (uint32_t)(offset_ - data_)) { + Slice filter = Slice(data_ + start, limit - start); + bool const may_match = policy_->KeyMayMatch(entry, filter); + if (may_match) { + PERF_COUNTER_ADD(bloom_sst_hit_count, 1); + return true; + } else { + PERF_COUNTER_ADD(bloom_sst_miss_count, 1); + return false; + } + } else if (start == limit) { + // Empty filters do not match any entries + return false; + } + } + return true; // Errors are treated as potential matches +} + +size_t BlockBasedFilterBlockReader::ApproximateMemoryUsage() const { + return num_ * 4 + 5 + (offset_ - data_); +} + +std::string BlockBasedFilterBlockReader::ToString() const { + std::string result, filter_meta; + result.reserve(1024); + + std::string s_bo("Block offset"), s_hd("Hex dump"), s_fb("# filter blocks"); + AppendItem(&result, s_fb, rocksdb::ToString(num_)); + AppendItem(&result, s_bo, s_hd); + + for (size_t index = 0; index < num_; index++) { + uint32_t start = DecodeFixed32(offset_ + index * 4); + uint32_t limit = DecodeFixed32(offset_ + index * 4 + 4); + + if (start != limit) { + result.append(" filter block # " + rocksdb::ToString(index + 1) + "\n"); + Slice filter = Slice(data_ + start, limit - start); + AppendItem(&result, start, filter.ToString(true)); + } + } + return result; +} +} // namespace rocksdb diff --git a/table/block_based_filter_block.h b/table/block_based_filter_block.h new file mode 100644 index 00000000000..52b79fea501 --- /dev/null +++ b/table/block_based_filter_block.h @@ -0,0 +1,112 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +// +// A filter block is stored near the end of a Table file. It contains +// filters (e.g., bloom filters) for all data blocks in the table combined +// into a single filter block. + +#pragma once + +#include +#include +#include +#include +#include +#include "rocksdb/options.h" +#include "rocksdb/slice.h" +#include "rocksdb/slice_transform.h" +#include "table/filter_block.h" +#include "util/hash.h" + +namespace rocksdb { + + +// A BlockBasedFilterBlockBuilder is used to construct all of the filters for a +// particular Table. It generates a single string which is stored as +// a special block in the Table. +// +// The sequence of calls to BlockBasedFilterBlockBuilder must match the regexp: +// (StartBlock Add*)* Finish +class BlockBasedFilterBlockBuilder : public FilterBlockBuilder { + public: + BlockBasedFilterBlockBuilder(const SliceTransform* prefix_extractor, + const BlockBasedTableOptions& table_opt); + + virtual bool IsBlockBased() override { return true; } + virtual void StartBlock(uint64_t block_offset) override; + virtual void Add(const Slice& key) override; + virtual Slice Finish(const BlockHandle& tmp, Status* status) override; + using FilterBlockBuilder::Finish; + + private: + void AddKey(const Slice& key); + void AddPrefix(const Slice& key); + void GenerateFilter(); + + // important: all of these might point to invalid addresses + // at the time of destruction of this filter block. destructor + // should NOT dereference them. + const FilterPolicy* policy_; + const SliceTransform* prefix_extractor_; + bool whole_key_filtering_; + + size_t prev_prefix_start_; // the position of the last appended prefix + // to "entries_". + size_t prev_prefix_size_; // the length of the last appended prefix to + // "entries_". + std::string entries_; // Flattened entry contents + std::vector start_; // Starting index in entries_ of each entry + std::string result_; // Filter data computed so far + std::vector tmp_entries_; // policy_->CreateFilter() argument + std::vector filter_offsets_; + + // No copying allowed + BlockBasedFilterBlockBuilder(const BlockBasedFilterBlockBuilder&); + void operator=(const BlockBasedFilterBlockBuilder&); +}; + +// A FilterBlockReader is used to parse filter from SST table. +// KeyMayMatch and PrefixMayMatch would trigger filter checking +class BlockBasedFilterBlockReader : public FilterBlockReader { + public: + // REQUIRES: "contents" and *policy must stay live while *this is live. + BlockBasedFilterBlockReader(const SliceTransform* prefix_extractor, + const BlockBasedTableOptions& table_opt, + bool whole_key_filtering, + BlockContents&& contents, Statistics* statistics); + virtual bool IsBlockBased() override { return true; } + virtual bool KeyMayMatch( + const Slice& key, uint64_t block_offset = kNotValid, + const bool no_io = false, + const Slice* const const_ikey_ptr = nullptr) override; + virtual bool PrefixMayMatch( + const Slice& prefix, uint64_t block_offset = kNotValid, + const bool no_io = false, + const Slice* const const_ikey_ptr = nullptr) override; + virtual size_t ApproximateMemoryUsage() const override; + + // convert this object to a human readable form + std::string ToString() const override; + + private: + const FilterPolicy* policy_; + const SliceTransform* prefix_extractor_; + const char* data_; // Pointer to filter data (at block-start) + const char* offset_; // Pointer to beginning of offset array (at block-end) + size_t num_; // Number of entries in offset array + size_t base_lg_; // Encoding parameter (see kFilterBaseLg in .cc file) + BlockContents contents_; + + bool MayMatch(const Slice& entry, uint64_t block_offset); + + // No copying allowed + BlockBasedFilterBlockReader(const BlockBasedFilterBlockReader&); + void operator=(const BlockBasedFilterBlockReader&); +}; +} // namespace rocksdb diff --git a/table/block_based_filter_block_test.cc b/table/block_based_filter_block_test.cc new file mode 100644 index 00000000000..f666ba25242 --- /dev/null +++ b/table/block_based_filter_block_test.cc @@ -0,0 +1,248 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "table/block_based_filter_block.h" + +#include "rocksdb/filter_policy.h" +#include "util/coding.h" +#include "util/hash.h" +#include "util/string_util.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +// For testing: emit an array with one hash value per key +class TestHashFilter : public FilterPolicy { + public: + virtual const char* Name() const override { return "TestHashFilter"; } + + virtual void CreateFilter(const Slice* keys, int n, + std::string* dst) const override { + for (int i = 0; i < n; i++) { + uint32_t h = Hash(keys[i].data(), keys[i].size(), 1); + PutFixed32(dst, h); + } + } + + virtual bool KeyMayMatch(const Slice& key, + const Slice& filter) const override { + uint32_t h = Hash(key.data(), key.size(), 1); + for (unsigned int i = 0; i + 4 <= filter.size(); i += 4) { + if (h == DecodeFixed32(filter.data() + i)) { + return true; + } + } + return false; + } +}; + +class FilterBlockTest : public testing::Test { + public: + TestHashFilter policy_; + BlockBasedTableOptions table_options_; + + FilterBlockTest() { + table_options_.filter_policy.reset(new TestHashFilter()); + } +}; + +TEST_F(FilterBlockTest, EmptyBuilder) { + BlockBasedFilterBlockBuilder builder(nullptr, table_options_); + BlockContents block(builder.Finish(), false, kNoCompression); + ASSERT_EQ("\\x00\\x00\\x00\\x00\\x0b", EscapeString(block.data)); + BlockBasedFilterBlockReader reader(nullptr, table_options_, true, + std::move(block), nullptr); + ASSERT_TRUE(reader.KeyMayMatch("foo", 0)); + ASSERT_TRUE(reader.KeyMayMatch("foo", 100000)); +} + +TEST_F(FilterBlockTest, SingleChunk) { + BlockBasedFilterBlockBuilder builder(nullptr, table_options_); + builder.StartBlock(100); + builder.Add("foo"); + builder.Add("bar"); + builder.Add("box"); + builder.StartBlock(200); + builder.Add("box"); + builder.StartBlock(300); + builder.Add("hello"); + BlockContents block(builder.Finish(), false, kNoCompression); + BlockBasedFilterBlockReader reader(nullptr, table_options_, true, + std::move(block), nullptr); + ASSERT_TRUE(reader.KeyMayMatch("foo", 100)); + ASSERT_TRUE(reader.KeyMayMatch("bar", 100)); + ASSERT_TRUE(reader.KeyMayMatch("box", 100)); + ASSERT_TRUE(reader.KeyMayMatch("hello", 100)); + ASSERT_TRUE(reader.KeyMayMatch("foo", 100)); + ASSERT_TRUE(!reader.KeyMayMatch("missing", 100)); + ASSERT_TRUE(!reader.KeyMayMatch("other", 100)); +} + +TEST_F(FilterBlockTest, MultiChunk) { + BlockBasedFilterBlockBuilder builder(nullptr, table_options_); + + // First filter + builder.StartBlock(0); + builder.Add("foo"); + builder.StartBlock(2000); + builder.Add("bar"); + + // Second filter + builder.StartBlock(3100); + builder.Add("box"); + + // Third filter is empty + + // Last filter + builder.StartBlock(9000); + builder.Add("box"); + builder.Add("hello"); + + BlockContents block(builder.Finish(), false, kNoCompression); + BlockBasedFilterBlockReader reader(nullptr, table_options_, true, + std::move(block), nullptr); + + // Check first filter + ASSERT_TRUE(reader.KeyMayMatch("foo", 0)); + ASSERT_TRUE(reader.KeyMayMatch("bar", 2000)); + ASSERT_TRUE(!reader.KeyMayMatch("box", 0)); + ASSERT_TRUE(!reader.KeyMayMatch("hello", 0)); + + // Check second filter + ASSERT_TRUE(reader.KeyMayMatch("box", 3100)); + ASSERT_TRUE(!reader.KeyMayMatch("foo", 3100)); + ASSERT_TRUE(!reader.KeyMayMatch("bar", 3100)); + ASSERT_TRUE(!reader.KeyMayMatch("hello", 3100)); + + // Check third filter (empty) + ASSERT_TRUE(!reader.KeyMayMatch("foo", 4100)); + ASSERT_TRUE(!reader.KeyMayMatch("bar", 4100)); + ASSERT_TRUE(!reader.KeyMayMatch("box", 4100)); + ASSERT_TRUE(!reader.KeyMayMatch("hello", 4100)); + + // Check last filter + ASSERT_TRUE(reader.KeyMayMatch("box", 9000)); + ASSERT_TRUE(reader.KeyMayMatch("hello", 9000)); + ASSERT_TRUE(!reader.KeyMayMatch("foo", 9000)); + ASSERT_TRUE(!reader.KeyMayMatch("bar", 9000)); +} + +// Test for block based filter block +// use new interface in FilterPolicy to create filter builder/reader +class BlockBasedFilterBlockTest : public testing::Test { + public: + BlockBasedTableOptions table_options_; + + BlockBasedFilterBlockTest() { + table_options_.filter_policy.reset(NewBloomFilterPolicy(10)); + } + + ~BlockBasedFilterBlockTest() {} +}; + +TEST_F(BlockBasedFilterBlockTest, BlockBasedEmptyBuilder) { + FilterBlockBuilder* builder = new BlockBasedFilterBlockBuilder( + nullptr, table_options_); + BlockContents block(builder->Finish(), false, kNoCompression); + ASSERT_EQ("\\x00\\x00\\x00\\x00\\x0b", EscapeString(block.data)); + FilterBlockReader* reader = new BlockBasedFilterBlockReader( + nullptr, table_options_, true, std::move(block), nullptr); + ASSERT_TRUE(reader->KeyMayMatch("foo", 0)); + ASSERT_TRUE(reader->KeyMayMatch("foo", 100000)); + + delete builder; + delete reader; +} + +TEST_F(BlockBasedFilterBlockTest, BlockBasedSingleChunk) { + FilterBlockBuilder* builder = new BlockBasedFilterBlockBuilder( + nullptr, table_options_); + builder->StartBlock(100); + builder->Add("foo"); + builder->Add("bar"); + builder->Add("box"); + builder->StartBlock(200); + builder->Add("box"); + builder->StartBlock(300); + builder->Add("hello"); + BlockContents block(builder->Finish(), false, kNoCompression); + FilterBlockReader* reader = new BlockBasedFilterBlockReader( + nullptr, table_options_, true, std::move(block), nullptr); + ASSERT_TRUE(reader->KeyMayMatch("foo", 100)); + ASSERT_TRUE(reader->KeyMayMatch("bar", 100)); + ASSERT_TRUE(reader->KeyMayMatch("box", 100)); + ASSERT_TRUE(reader->KeyMayMatch("hello", 100)); + ASSERT_TRUE(reader->KeyMayMatch("foo", 100)); + ASSERT_TRUE(!reader->KeyMayMatch("missing", 100)); + ASSERT_TRUE(!reader->KeyMayMatch("other", 100)); + + delete builder; + delete reader; +} + +TEST_F(BlockBasedFilterBlockTest, BlockBasedMultiChunk) { + FilterBlockBuilder* builder = new BlockBasedFilterBlockBuilder( + nullptr, table_options_); + + // First filter + builder->StartBlock(0); + builder->Add("foo"); + builder->StartBlock(2000); + builder->Add("bar"); + + // Second filter + builder->StartBlock(3100); + builder->Add("box"); + + // Third filter is empty + + // Last filter + builder->StartBlock(9000); + builder->Add("box"); + builder->Add("hello"); + + BlockContents block(builder->Finish(), false, kNoCompression); + FilterBlockReader* reader = new BlockBasedFilterBlockReader( + nullptr, table_options_, true, std::move(block), nullptr); + + // Check first filter + ASSERT_TRUE(reader->KeyMayMatch("foo", 0)); + ASSERT_TRUE(reader->KeyMayMatch("bar", 2000)); + ASSERT_TRUE(!reader->KeyMayMatch("box", 0)); + ASSERT_TRUE(!reader->KeyMayMatch("hello", 0)); + + // Check second filter + ASSERT_TRUE(reader->KeyMayMatch("box", 3100)); + ASSERT_TRUE(!reader->KeyMayMatch("foo", 3100)); + ASSERT_TRUE(!reader->KeyMayMatch("bar", 3100)); + ASSERT_TRUE(!reader->KeyMayMatch("hello", 3100)); + + // Check third filter (empty) + ASSERT_TRUE(!reader->KeyMayMatch("foo", 4100)); + ASSERT_TRUE(!reader->KeyMayMatch("bar", 4100)); + ASSERT_TRUE(!reader->KeyMayMatch("box", 4100)); + ASSERT_TRUE(!reader->KeyMayMatch("hello", 4100)); + + // Check last filter + ASSERT_TRUE(reader->KeyMayMatch("box", 9000)); + ASSERT_TRUE(reader->KeyMayMatch("hello", 9000)); + ASSERT_TRUE(!reader->KeyMayMatch("foo", 9000)); + ASSERT_TRUE(!reader->KeyMayMatch("bar", 9000)); + + delete builder; + delete reader; +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/table/block_based_table_builder.cc b/table/block_based_table_builder.cc index 03f1e199c67..e82f91aec7f 100644 --- a/table/block_based_table_builder.cc +++ b/table/block_based_table_builder.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,13 +10,14 @@ #include "table/block_based_table_builder.h" #include -#include #include +#include #include #include #include #include +#include #include "db/dbformat.h" @@ -25,22 +26,30 @@ #include "rocksdb/env.h" #include "rocksdb/filter_policy.h" #include "rocksdb/flush_block_policy.h" -#include "rocksdb/options.h" +#include "rocksdb/merge_operator.h" #include "rocksdb/table.h" #include "table/block.h" +#include "table/block_based_filter_block.h" +#include "table/block_based_table_factory.h" #include "table/block_based_table_reader.h" #include "table/block_builder.h" #include "table/filter_block.h" #include "table/format.h" +#include "table/full_filter_block.h" #include "table/meta_blocks.h" #include "table/table_builder.h" +#include "util/string_util.h" #include "util/coding.h" +#include "util/compression.h" #include "util/crc32c.h" #include "util/stop_watch.h" #include "util/xxhash.h" +#include "table/index_builder.h" +#include "table/partitioned_filter_block.h" + namespace rocksdb { extern const std::string kHashIndexPrefixesBlock; @@ -48,231 +57,40 @@ extern const std::string kHashIndexPrefixesMetadataBlock; typedef BlockBasedTableOptions::IndexType IndexType; -// The interface for building index. -// Instruction for adding a new concrete IndexBuilder: -// 1. Create a subclass instantiated from IndexBuilder. -// 2. Add a new entry associated with that subclass in TableOptions::IndexType. -// 3. Add a create function for the new subclass in CreateIndexBuilder. -// Note: we can devise more advanced design to simplify the process for adding -// new subclass, which will, on the other hand, increase the code complexity and -// catch unwanted attention from readers. Given that we won't add/change -// indexes frequently, it makes sense to just embrace a more straightforward -// design that just works. -class IndexBuilder { - public: - // Index builder will construct a set of blocks which contain: - // 1. One primary index block. - // 2. (Optional) a set of metablocks that contains the metadata of the - // primary index. - struct IndexBlocks { - Slice index_block_contents; - std::unordered_map meta_blocks; - }; - explicit IndexBuilder(const Comparator* comparator) - : comparator_(comparator) {} - - virtual ~IndexBuilder() {} - - // Add a new index entry to index block. - // To allow further optimization, we provide `last_key_in_current_block` and - // `first_key_in_next_block`, based on which the specific implementation can - // determine the best index key to be used for the index block. - // @last_key_in_current_block: this parameter maybe overridden with the value - // "substitute key". - // @first_key_in_next_block: it will be nullptr if the entry being added is - // the last one in the table - // - // REQUIRES: Finish() has not yet been called. - virtual void AddIndexEntry(std::string* last_key_in_current_block, - const Slice* first_key_in_next_block, - const BlockHandle& block_handle) = 0; - - // This method will be called whenever a key is added. The subclasses may - // override OnKeyAdded() if they need to collect additional information. - virtual void OnKeyAdded(const Slice& key) {} - - // Inform the index builder that all entries has been written. Block builder - // may therefore perform any operation required for block finalization. - // - // REQUIRES: Finish() has not yet been called. - virtual Status Finish(IndexBlocks* index_blocks) = 0; - - // Get the estimated size for index block. - virtual size_t EstimatedSize() const = 0; - - protected: - const Comparator* comparator_; -}; +// Without anonymous namespace here, we fail the warning -Wmissing-prototypes +namespace { -// This index builder builds space-efficient index block. -// -// Optimizations: -// 1. Made block's `block_restart_interval` to be 1, which will avoid linear -// search when doing index lookup. -// 2. Shorten the key length for index block. Other than honestly using the -// last key in the data block as the index key, we instead find a shortest -// substitute key that serves the same function. -class ShortenedIndexBuilder : public IndexBuilder { - public: - explicit ShortenedIndexBuilder(const Comparator* comparator) - : IndexBuilder(comparator), - index_block_builder_(1 /* block_restart_interval == 1 */) {} - - virtual void AddIndexEntry(std::string* last_key_in_current_block, - const Slice* first_key_in_next_block, - const BlockHandle& block_handle) override { - if (first_key_in_next_block != nullptr) { - comparator_->FindShortestSeparator(last_key_in_current_block, - *first_key_in_next_block); - } else { - comparator_->FindShortSuccessor(last_key_in_current_block); - } +// Create a filter block builder based on its type. +FilterBlockBuilder* CreateFilterBlockBuilder( + const ImmutableCFOptions& opt, const BlockBasedTableOptions& table_opt, + PartitionedIndexBuilder* const p_index_builder) { + if (table_opt.filter_policy == nullptr) return nullptr; - std::string handle_encoding; - block_handle.EncodeTo(&handle_encoding); - index_block_builder_.Add(*last_key_in_current_block, handle_encoding); - } - - virtual Status Finish(IndexBlocks* index_blocks) { - index_blocks->index_block_contents = index_block_builder_.Finish(); - return Status::OK(); - } - - virtual size_t EstimatedSize() const { - return index_block_builder_.CurrentSizeEstimate(); - } - - private: - BlockBuilder index_block_builder_; -}; - -// HashIndexBuilder contains a binary-searchable primary index and the -// metadata for secondary hash index construction. -// The metadata for hash index consists two parts: -// - a metablock that compactly contains a sequence of prefixes. All prefixes -// are stored consectively without any metadata (like, prefix sizes) being -// stored, which is kept in the other metablock. -// - a metablock contains the metadata of the prefixes, including prefix size, -// restart index and number of block it spans. The format looks like: -// -// +-----------------+---------------------------+---------------------+ <=prefix 1 -// | length: 4 bytes | restart interval: 4 bytes | num-blocks: 4 bytes | -// +-----------------+---------------------------+---------------------+ <=prefix 2 -// | length: 4 bytes | restart interval: 4 bytes | num-blocks: 4 bytes | -// +-----------------+---------------------------+---------------------+ -// | | -// | .... | -// | | -// +-----------------+---------------------------+---------------------+ <=prefix n -// | length: 4 bytes | restart interval: 4 bytes | num-blocks: 4 bytes | -// +-----------------+---------------------------+---------------------+ -// -// The reason of separating these two metablocks is to enable the efficiently -// reuse the first metablock during hash index construction without unnecessary -// data copy or small heap allocations for prefixes. -class HashIndexBuilder : public IndexBuilder { - public: - explicit HashIndexBuilder(const Comparator* comparator, - const SliceTransform* hash_key_extractor) - : IndexBuilder(comparator), - primary_index_builder(comparator), - hash_key_extractor_(hash_key_extractor) {} - - virtual void AddIndexEntry(std::string* last_key_in_current_block, - const Slice* first_key_in_next_block, - const BlockHandle& block_handle) override { - ++current_restart_index_; - primary_index_builder.AddIndexEntry(last_key_in_current_block, - first_key_in_next_block, block_handle); - } - - virtual void OnKeyAdded(const Slice& key) override { - auto key_prefix = hash_key_extractor_->Transform(key); - bool is_first_entry = pending_block_num_ == 0; - - // Keys may share the prefix - if (is_first_entry || pending_entry_prefix_ != key_prefix) { - if (!is_first_entry) { - FlushPendingPrefix(); - } - - // need a hard copy otherwise the underlying data changes all the time. - // TODO(kailiu) ToString() is expensive. We may speed up can avoid data - // copy. - pending_entry_prefix_ = key_prefix.ToString(); - pending_block_num_ = 1; - pending_entry_index_ = current_restart_index_; + FilterBitsBuilder* filter_bits_builder = + table_opt.filter_policy->GetFilterBitsBuilder(); + if (filter_bits_builder == nullptr) { + return new BlockBasedFilterBlockBuilder(opt.prefix_extractor, table_opt); + } else { + if (table_opt.partition_filters) { + assert(p_index_builder != nullptr); + // Since after partition cut request from filter builder it takes time + // until index builder actully cuts the partition, we take the lower bound + // as partition size. + assert(table_opt.block_size_deviation <= 100); + auto partition_size = static_cast( + table_opt.metadata_block_size * + (100 - table_opt.block_size_deviation)); + partition_size = std::max(partition_size, static_cast(1)); + return new PartitionedFilterBlockBuilder( + opt.prefix_extractor, table_opt.whole_key_filtering, + filter_bits_builder, table_opt.index_block_restart_interval, + p_index_builder, partition_size); } else { - // entry number increments when keys share the prefix reside in - // differnt data blocks. - auto last_restart_index = pending_entry_index_ + pending_block_num_ - 1; - assert(last_restart_index <= current_restart_index_); - if (last_restart_index != current_restart_index_) { - ++pending_block_num_; - } + return new FullFilterBlockBuilder(opt.prefix_extractor, + table_opt.whole_key_filtering, + filter_bits_builder); } } - - virtual Status Finish(IndexBlocks* index_blocks) { - FlushPendingPrefix(); - primary_index_builder.Finish(index_blocks); - index_blocks->meta_blocks.insert( - {kHashIndexPrefixesBlock.c_str(), prefix_block_}); - index_blocks->meta_blocks.insert( - {kHashIndexPrefixesMetadataBlock.c_str(), prefix_meta_block_}); - return Status::OK(); - } - - virtual size_t EstimatedSize() const { - return primary_index_builder.EstimatedSize() + prefix_block_.size() + - prefix_meta_block_.size(); - } - - private: - void FlushPendingPrefix() { - prefix_block_.append(pending_entry_prefix_.data(), - pending_entry_prefix_.size()); - PutVarint32(&prefix_meta_block_, pending_entry_prefix_.size()); - PutVarint32(&prefix_meta_block_, pending_entry_index_); - PutVarint32(&prefix_meta_block_, pending_block_num_); - } - - ShortenedIndexBuilder primary_index_builder; - const SliceTransform* hash_key_extractor_; - - // stores a sequence of prefixes - std::string prefix_block_; - // stores the metadata of prefixes - std::string prefix_meta_block_; - - // The following 3 variables keeps unflushed prefix and its metadata. - // The details of block_num and entry_index can be found in - // "block_hash_index.{h,cc}" - uint32_t pending_block_num_ = 0; - uint32_t pending_entry_index_ = 0; - std::string pending_entry_prefix_; - - uint64_t current_restart_index_ = 0; -}; - -// Create a index builder based on its type. -IndexBuilder* CreateIndexBuilder(IndexType type, const Comparator* comparator, - const SliceTransform* prefix_extractor) { - switch (type) { - case BlockBasedTableOptions::kBinarySearch: { - return new ShortenedIndexBuilder(comparator); - } - case BlockBasedTableOptions::kHashSearch: { - return new HashIndexBuilder(comparator, prefix_extractor); - } - default: { - assert(!"Do not recognize the index type "); - return nullptr; - } - } - // impossible. - assert(false); - return nullptr; } bool GoodCompressionRatio(size_t compressed_size, size_t raw_size) { @@ -280,9 +98,14 @@ bool GoodCompressionRatio(size_t compressed_size, size_t raw_size) { return compressed_size < raw_size - (raw_size / 8u); } +} // namespace + +// format_version is the block format as defined in include/rocksdb/table.h Slice CompressBlock(const Slice& raw, const CompressionOptions& compression_options, - CompressionType* type, std::string* compressed_output) { + CompressionType* type, uint32_t format_version, + const Slice& compression_dict, + std::string* compressed_output) { if (*type == kNoCompression) { return raw; } @@ -291,36 +114,59 @@ Slice CompressBlock(const Slice& raw, // supported in this platform and (2) the compression rate is "good enough". switch (*type) { case kSnappyCompression: - if (port::Snappy_Compress(compression_options, raw.data(), raw.size(), - compressed_output) && + if (Snappy_Compress(compression_options, raw.data(), raw.size(), + compressed_output) && GoodCompressionRatio(compressed_output->size(), raw.size())) { return *compressed_output; } break; // fall back to no compression. case kZlibCompression: - if (port::Zlib_Compress(compression_options, raw.data(), raw.size(), - compressed_output) && + if (Zlib_Compress( + compression_options, + GetCompressFormatForVersion(kZlibCompression, format_version), + raw.data(), raw.size(), compressed_output, compression_dict) && GoodCompressionRatio(compressed_output->size(), raw.size())) { return *compressed_output; } break; // fall back to no compression. case kBZip2Compression: - if (port::BZip2_Compress(compression_options, raw.data(), raw.size(), - compressed_output) && + if (BZip2_Compress( + compression_options, + GetCompressFormatForVersion(kBZip2Compression, format_version), + raw.data(), raw.size(), compressed_output) && GoodCompressionRatio(compressed_output->size(), raw.size())) { return *compressed_output; } break; // fall back to no compression. case kLZ4Compression: - if (port::LZ4_Compress(compression_options, raw.data(), raw.size(), - compressed_output) && + if (LZ4_Compress( + compression_options, + GetCompressFormatForVersion(kLZ4Compression, format_version), + raw.data(), raw.size(), compressed_output, compression_dict) && GoodCompressionRatio(compressed_output->size(), raw.size())) { return *compressed_output; } break; // fall back to no compression. case kLZ4HCCompression: - if (port::LZ4HC_Compress(compression_options, raw.data(), raw.size(), - compressed_output) && + if (LZ4HC_Compress( + compression_options, + GetCompressFormatForVersion(kLZ4HCCompression, format_version), + raw.data(), raw.size(), compressed_output, compression_dict) && + GoodCompressionRatio(compressed_output->size(), raw.size())) { + return *compressed_output; + } + break; // fall back to no compression. + case kXpressCompression: + if (XPRESS_Compress(raw.data(), raw.size(), + compressed_output) && + GoodCompressionRatio(compressed_output->size(), raw.size())) { + return *compressed_output; + } + break; + case kZSTD: + case kZSTDNotFinalCompression: + if (ZSTD_Compress(compression_options, raw.data(), raw.size(), + compressed_output, compression_dict) && GoodCompressionRatio(compressed_output->size(), raw.size())) { return *compressed_output; } @@ -337,71 +183,88 @@ Slice CompressBlock(const Slice& raw, // kBlockBasedTableMagicNumber was picked by running // echo rocksdb.table.block_based | sha1sum // and taking the leading 64 bits. -// Please note that kBlockBasedTableMagicNumber may also be accessed by -// other .cc files so it have to be explicitly declared with "extern". -extern const uint64_t kBlockBasedTableMagicNumber = 0x88e241b785f4cff7ull; +// Please note that kBlockBasedTableMagicNumber may also be accessed by other +// .cc files +// for that reason we declare it extern in the header but to get the space +// allocated +// it must be not extern in one place. +const uint64_t kBlockBasedTableMagicNumber = 0x88e241b785f4cff7ull; // We also support reading and writing legacy block based table format (for // backwards compatibility) -extern const uint64_t kLegacyBlockBasedTableMagicNumber = 0xdb4775248b80fb57ull; +const uint64_t kLegacyBlockBasedTableMagicNumber = 0xdb4775248b80fb57ull; // A collector that collects properties of interest to block-based table. // For now this class looks heavy-weight since we only write one additional // property. -// But in the forseeable future, we will add more and more properties that are +// But in the foreseeable future, we will add more and more properties that are // specific to block-based table. class BlockBasedTableBuilder::BlockBasedTablePropertiesCollector - : public TablePropertiesCollector { + : public IntTblPropCollector { public: explicit BlockBasedTablePropertiesCollector( - BlockBasedTableOptions::IndexType index_type) - : index_type_(index_type) {} - - virtual Status Add(const Slice& key, const Slice& value) { + BlockBasedTableOptions::IndexType index_type, bool whole_key_filtering, + bool prefix_filtering) + : index_type_(index_type), + whole_key_filtering_(whole_key_filtering), + prefix_filtering_(prefix_filtering) {} + + virtual Status InternalAdd(const Slice& key, const Slice& value, + uint64_t file_size) override { // Intentionally left blank. Have no interest in collecting stats for // individual key/value pairs. return Status::OK(); } - virtual Status Finish(UserCollectedProperties* properties) { + virtual Status Finish(UserCollectedProperties* properties) override { std::string val; PutFixed32(&val, static_cast(index_type_)); properties->insert({BlockBasedTablePropertyNames::kIndexType, val}); - + properties->insert({BlockBasedTablePropertyNames::kWholeKeyFiltering, + whole_key_filtering_ ? kPropTrue : kPropFalse}); + properties->insert({BlockBasedTablePropertyNames::kPrefixFiltering, + prefix_filtering_ ? kPropTrue : kPropFalse}); return Status::OK(); } // The name of the properties collector can be used for debugging purpose. - virtual const char* Name() const { + virtual const char* Name() const override { return "BlockBasedTablePropertiesCollector"; } - virtual UserCollectedProperties GetReadableProperties() const { + virtual UserCollectedProperties GetReadableProperties() const override { // Intentionally left blank. return UserCollectedProperties(); } private: BlockBasedTableOptions::IndexType index_type_; + bool whole_key_filtering_; + bool prefix_filtering_; }; struct BlockBasedTableBuilder::Rep { - const Options options; + const ImmutableCFOptions ioptions; const BlockBasedTableOptions table_options; const InternalKeyComparator& internal_comparator; - WritableFile* file; + WritableFileWriter* file; uint64_t offset = 0; Status status; BlockBuilder data_block; + BlockBuilder range_del_block; InternalKeySliceTransform internal_prefix_transform; std::unique_ptr index_builder; + PartitionedIndexBuilder* p_index_builder_ = nullptr; std::string last_key; - CompressionType compression_type; + const CompressionType compression_type; + const CompressionOptions compression_opts; + // Data for presetting the compression library's dictionary, or nullptr. + const std::string* compression_dict; TableProperties props; bool closed = false; // Either Finish() or Abandon() has been called. - FilterBlockBuilder* filter_block; + std::unique_ptr filter_builder; char compressed_cache_key_prefix[BlockBasedTable::kMaxCacheKeyPrefixSize]; size_t compressed_cache_key_prefix_size; @@ -409,51 +272,106 @@ struct BlockBasedTableBuilder::Rep { std::string compressed_output; std::unique_ptr flush_block_policy; + uint32_t column_family_id; + const std::string& column_family_name; + uint64_t creation_time = 0; + uint64_t oldest_key_time = 0; - std::vector> - table_properties_collectors; + std::vector> table_properties_collectors; - Rep(const Options& opt, const BlockBasedTableOptions& table_opt, + Rep(const ImmutableCFOptions& _ioptions, + const BlockBasedTableOptions& table_opt, const InternalKeyComparator& icomparator, - WritableFile* f, CompressionType compression_type) - : options(opt), + const std::vector>* + int_tbl_prop_collector_factories, + uint32_t _column_family_id, WritableFileWriter* f, + const CompressionType _compression_type, + const CompressionOptions& _compression_opts, + const std::string* _compression_dict, const bool skip_filters, + const std::string& _column_family_name, const uint64_t _creation_time, + const uint64_t _oldest_key_time) + : ioptions(_ioptions), table_options(table_opt), internal_comparator(icomparator), file(f), - data_block(table_options.block_restart_interval), - internal_prefix_transform(options.prefix_extractor.get()), - index_builder(CreateIndexBuilder( - table_options.index_type, &internal_comparator, - &this->internal_prefix_transform)), - compression_type(compression_type), - filter_block(table_options.filter_policy == nullptr ? - nullptr : - new FilterBlockBuilder(opt, table_options, &internal_comparator)), + data_block(table_options.block_restart_interval, + table_options.use_delta_encoding), + range_del_block(1), // TODO(andrewkr): restart_interval unnecessary + internal_prefix_transform(_ioptions.prefix_extractor), + compression_type(_compression_type), + compression_opts(_compression_opts), + compression_dict(_compression_dict), flush_block_policy( table_options.flush_block_policy_factory->NewFlushBlockPolicy( - table_options, data_block)) { - for (auto& collector_factories : - options.table_properties_collector_factories) { + table_options, data_block)), + column_family_id(_column_family_id), + column_family_name(_column_family_name), + creation_time(_creation_time), + oldest_key_time(_oldest_key_time) { + if (table_options.index_type == + BlockBasedTableOptions::kTwoLevelIndexSearch) { + p_index_builder_ = PartitionedIndexBuilder::CreateIndexBuilder( + &internal_comparator, table_options); + index_builder.reset(p_index_builder_); + } else { + index_builder.reset(IndexBuilder::CreateIndexBuilder( + table_options.index_type, &internal_comparator, + &this->internal_prefix_transform, table_options)); + } + if (skip_filters) { + filter_builder = nullptr; + } else { + filter_builder.reset( + CreateFilterBlockBuilder(_ioptions, table_options, p_index_builder_)); + } + + for (auto& collector_factories : *int_tbl_prop_collector_factories) { table_properties_collectors.emplace_back( - collector_factories->CreateTablePropertiesCollector()); + collector_factories->CreateIntTblPropCollector(column_family_id)); } table_properties_collectors.emplace_back( - new BlockBasedTablePropertiesCollector(table_options.index_type)); + new BlockBasedTablePropertiesCollector( + table_options.index_type, table_options.whole_key_filtering, + _ioptions.prefix_extractor != nullptr)); } }; BlockBasedTableBuilder::BlockBasedTableBuilder( - const Options& options, const BlockBasedTableOptions& table_options, - const InternalKeyComparator& internal_comparator, WritableFile* file, - CompressionType compression_type) - : rep_(new Rep(options, table_options, internal_comparator, - file, compression_type)) { - if (rep_->filter_block != nullptr) { - rep_->filter_block->StartBlock(0); + const ImmutableCFOptions& ioptions, + const BlockBasedTableOptions& table_options, + const InternalKeyComparator& internal_comparator, + const std::vector>* + int_tbl_prop_collector_factories, + uint32_t column_family_id, WritableFileWriter* file, + const CompressionType compression_type, + const CompressionOptions& compression_opts, + const std::string* compression_dict, const bool skip_filters, + const std::string& column_family_name, const uint64_t creation_time, + const uint64_t oldest_key_time) { + BlockBasedTableOptions sanitized_table_options(table_options); + if (sanitized_table_options.format_version == 0 && + sanitized_table_options.checksum != kCRC32c) { + ROCKS_LOG_WARN( + ioptions.info_log, + "Silently converting format_version to 1 because checksum is " + "non-default"); + // silently convert format_version to 1 to keep consistent with current + // behavior + sanitized_table_options.format_version = 1; + } + + rep_ = + new Rep(ioptions, sanitized_table_options, internal_comparator, + int_tbl_prop_collector_factories, column_family_id, file, + compression_type, compression_opts, compression_dict, + skip_filters, column_family_name, creation_time, oldest_key_time); + + if (rep_->filter_builder != nullptr) { + rep_->filter_builder->StartBlock(0); } if (table_options.block_cache_compressed.get() != nullptr) { BlockBasedTable::GenerateCachePrefix( - table_options.block_cache_compressed.get(), file, + table_options.block_cache_compressed.get(), file->writable_file(), &rep_->compressed_cache_key_prefix[0], &rep_->compressed_cache_key_prefix_size); } @@ -461,7 +379,6 @@ BlockBasedTableBuilder::BlockBasedTableBuilder( BlockBasedTableBuilder::~BlockBasedTableBuilder() { assert(rep_->closed); // Catch errors where caller forgot to call Finish() - delete rep_->filter_block; delete rep_; } @@ -469,41 +386,59 @@ void BlockBasedTableBuilder::Add(const Slice& key, const Slice& value) { Rep* r = rep_; assert(!r->closed); if (!ok()) return; - if (r->props.num_entries > 0) { - assert(r->internal_comparator.Compare(key, Slice(r->last_key)) > 0); - } - - auto should_flush = r->flush_block_policy->Update(key, value); - if (should_flush) { - assert(!r->data_block.empty()); - Flush(); - - // Add item to index block. - // We do not emit the index entry for a block until we have seen the - // first key for the next data block. This allows us to use shorter - // keys in the index block. For example, consider a block boundary - // between the keys "the quick brown fox" and "the who". We can use - // "the r" as the key for the index block entry since it is >= all - // entries in the first block and < all entries in subsequent - // blocks. - if (ok()) { - r->index_builder->AddIndexEntry(&r->last_key, &key, r->pending_handle); + ValueType value_type = ExtractValueType(key); + if (IsValueType(value_type)) { + if (r->props.num_entries > 0) { + assert(r->internal_comparator.Compare(key, Slice(r->last_key)) > 0); } - } - if (r->filter_block != nullptr) { - r->filter_block->AddKey(ExtractUserKey(key)); - } + auto should_flush = r->flush_block_policy->Update(key, value); + if (should_flush) { + assert(!r->data_block.empty()); + Flush(); + + // Add item to index block. + // We do not emit the index entry for a block until we have seen the + // first key for the next data block. This allows us to use shorter + // keys in the index block. For example, consider a block boundary + // between the keys "the quick brown fox" and "the who". We can use + // "the r" as the key for the index block entry since it is >= all + // entries in the first block and < all entries in subsequent + // blocks. + if (ok()) { + r->index_builder->AddIndexEntry(&r->last_key, &key, r->pending_handle); + } + } - r->last_key.assign(key.data(), key.size()); - r->data_block.Add(key, value); - r->props.num_entries++; - r->props.raw_key_size += key.size(); - r->props.raw_value_size += value.size(); + // Note: PartitionedFilterBlockBuilder requires key being added to filter + // builder after being added to index builder. + if (r->filter_builder != nullptr) { + r->filter_builder->Add(ExtractUserKey(key)); + } - r->index_builder->OnKeyAdded(key); - NotifyCollectTableCollectorsOnAdd(key, value, r->table_properties_collectors, - r->options.info_log.get()); + r->last_key.assign(key.data(), key.size()); + r->data_block.Add(key, value); + r->props.num_entries++; + r->props.raw_key_size += key.size(); + r->props.raw_value_size += value.size(); + + r->index_builder->OnKeyAdded(key); + NotifyCollectTableCollectorsOnAdd(key, value, r->offset, + r->table_properties_collectors, + r->ioptions.info_log); + + } else if (value_type == kTypeRangeDeletion) { + // TODO(wanning&andrewkr) add num_tomestone to table properties + r->range_del_block.Add(key, value); + ++r->props.num_entries; + r->props.raw_key_size += key.size(); + r->props.raw_value_size += value.size(); + NotifyCollectTableCollectorsOnAdd(key, value, r->offset, + r->table_properties_collectors, + r->ioptions.info_log); + } else { + assert(false); + } } void BlockBasedTableBuilder::Flush() { @@ -511,25 +446,24 @@ void BlockBasedTableBuilder::Flush() { assert(!r->closed); if (!ok()) return; if (r->data_block.empty()) return; - WriteBlock(&r->data_block, &r->pending_handle); - if (ok()) { - r->status = r->file->Flush(); - } - if (r->filter_block != nullptr) { - r->filter_block->StartBlock(r->offset); + WriteBlock(&r->data_block, &r->pending_handle, true /* is_data_block */); + if (r->filter_builder != nullptr) { + r->filter_builder->StartBlock(r->offset); } r->props.data_size = r->offset; ++r->props.num_data_blocks; } void BlockBasedTableBuilder::WriteBlock(BlockBuilder* block, - BlockHandle* handle) { - WriteBlock(block->Finish(), handle); + BlockHandle* handle, + bool is_data_block) { + WriteBlock(block->Finish(), handle, is_data_block); block->Reset(); } void BlockBasedTableBuilder::WriteBlock(const Slice& raw_block_contents, - BlockHandle* handle) { + BlockHandle* handle, + bool is_data_block) { // File format contains a sequence of blocks where each block has: // block_data: uint8[n] // type: uint8 @@ -539,15 +473,69 @@ void BlockBasedTableBuilder::WriteBlock(const Slice& raw_block_contents, auto type = r->compression_type; Slice block_contents; + bool abort_compression = false; + + StopWatchNano timer(r->ioptions.env, + ShouldReportDetailedTime(r->ioptions.env, r->ioptions.statistics)); + if (raw_block_contents.size() < kCompressionSizeLimit) { - block_contents = - CompressBlock(raw_block_contents, r->options.compression_opts, &type, - &r->compressed_output); + Slice compression_dict; + if (is_data_block && r->compression_dict && r->compression_dict->size()) { + compression_dict = *r->compression_dict; + } + + block_contents = CompressBlock(raw_block_contents, r->compression_opts, + &type, r->table_options.format_version, + compression_dict, &r->compressed_output); + + // Some of the compression algorithms are known to be unreliable. If + // the verify_compression flag is set then try to de-compress the + // compressed data and compare to the input. + if (type != kNoCompression && r->table_options.verify_compression) { + // Retrieve the uncompressed contents into a new buffer + BlockContents contents; + Status stat = UncompressBlockContentsForCompressionType( + block_contents.data(), block_contents.size(), &contents, + r->table_options.format_version, compression_dict, type, + r->ioptions); + + if (stat.ok()) { + bool compressed_ok = contents.data.compare(raw_block_contents) == 0; + if (!compressed_ok) { + // The result of the compression was invalid. abort. + abort_compression = true; + ROCKS_LOG_ERROR(r->ioptions.info_log, + "Decompressed block did not match raw block"); + r->status = + Status::Corruption("Decompressed block did not match raw block"); + } + } else { + // Decompression reported an error. abort. + r->status = Status::Corruption("Could not decompress"); + abort_compression = true; + } + } } else { - RecordTick(r->options.statistics.get(), NUMBER_BLOCK_NOT_COMPRESSED); + // Block is too big to be compressed. + abort_compression = true; + } + + // Abort compression if the block is too big, or did not pass + // verification. + if (abort_compression) { + RecordTick(r->ioptions.statistics, NUMBER_BLOCK_NOT_COMPRESSED); type = kNoCompression; block_contents = raw_block_contents; + } else if (type != kNoCompression && + ShouldReportDetailedTime(r->ioptions.env, + r->ioptions.statistics)) { + MeasureTime(r->ioptions.statistics, COMPRESSION_TIMES_NANOS, + timer.ElapsedNanos()); + MeasureTime(r->ioptions.statistics, BYTES_COMPRESSED, + raw_block_contents.size()); + RecordTick(r->ioptions.statistics, NUMBER_BLOCK_COMPRESSED); } + WriteRawBlock(block_contents, type, handle); r->compressed_output.clear(); } @@ -556,10 +544,10 @@ void BlockBasedTableBuilder::WriteRawBlock(const Slice& block_contents, CompressionType type, BlockHandle* handle) { Rep* r = rep_; - StopWatch sw(r->options.env, r->options.statistics.get(), - WRITE_RAW_BLOCK_MICROS); + StopWatch sw(r->ioptions.env, r->ioptions.statistics, WRITE_RAW_BLOCK_MICROS); handle->set_offset(r->offset); handle->set_size(block_contents.size()); + assert(r->status.ok()); r->status = r->file->Append(block_contents); if (r->status.ok()) { char trailer[kBlockTrailerSize]; @@ -567,9 +555,8 @@ void BlockBasedTableBuilder::WriteRawBlock(const Slice& block_contents, char* trailer_without_type = trailer + 1; switch (r->table_options.checksum) { case kNoChecksum: - // we don't support no checksum yet - assert(false); - // intentional fallthrough in release binary + EncodeFixed32(trailer_without_type, 0); + break; case kCRC32c: { auto crc = crc32c::Value(block_contents.data(), block_contents.size()); crc = crc32c::Extend(crc, trailer, 1); // Extend to cover block type @@ -578,13 +565,15 @@ void BlockBasedTableBuilder::WriteRawBlock(const Slice& block_contents, } case kxxHash: { void* xxh = XXH32_init(0); - XXH32_update(xxh, block_contents.data(), block_contents.size()); + XXH32_update(xxh, block_contents.data(), + static_cast(block_contents.size())); XXH32_update(xxh, trailer, 1); // Extend to cover block type EncodeFixed32(trailer_without_type, XXH32_digest(xxh)); break; } } + assert(r->status.ok()); r->status = r->file->Append(Slice(trailer, kBlockTrailerSize)); if (r->status.ok()) { r->status = InsertBlockInCache(block_contents, type, handle); @@ -615,21 +604,15 @@ Status BlockBasedTableBuilder::InsertBlockInCache(const Slice& block_contents, if (type != kNoCompression && block_cache_compressed != nullptr) { - Cache::Handle* cache_handle = nullptr; size_t size = block_contents.size(); - char* ubuf = new char[size + 1]; // make a new copy - memcpy(ubuf, block_contents.data(), size); + std::unique_ptr ubuf(new char[size + 1]); + memcpy(ubuf.get(), block_contents.data(), size); ubuf[size] = type; - BlockContents results; - Slice sl(ubuf, size); - results.data = sl; - results.cachable = true; // XXX - results.heap_allocated = true; - results.compression_type = type; + BlockContents results(std::move(ubuf), size, true, type); - Block* block = new Block(results); + Block* block = new Block(std::move(results), kDisableGlobalSequenceNumber); // make cache key by appending the file offset to the cache prefix id char* end = EncodeVarint64( @@ -640,12 +623,11 @@ Status BlockBasedTableBuilder::InsertBlockInCache(const Slice& block_contents, (end - r->compressed_cache_key_prefix)); // Insert into compressed block cache. - cache_handle = block_cache_compressed->Insert(key, block, block->size(), - &DeleteCachedBlock); - block_cache_compressed->Release(cache_handle); + block_cache_compressed->Insert(key, block, block->usable_size(), + &DeleteCachedBlock); // Invalidate OS cache. - r->file->InvalidateCache(r->offset, size); + r->file->InvalidateCache(static_cast(r->offset), size); } return Status::OK(); } @@ -657,17 +639,6 @@ Status BlockBasedTableBuilder::Finish() { assert(!r->closed); r->closed = true; - BlockHandle filter_block_handle, - metaindex_block_handle, - index_block_handle; - - // Write filter block - if (ok() && r->filter_block != nullptr) { - auto filter_contents = r->filter_block->Finish(); - r->props.filter_size = filter_contents.size(); - WriteRawBlock(filter_contents, kNoCompression, &filter_block_handle); - } - // To make sure properties block is able to keep the accurate size of index // block, we will finish writing all index entries here and flush them // to storage after metaindex block is written. @@ -676,49 +647,109 @@ Status BlockBasedTableBuilder::Finish() { &r->last_key, nullptr /* no next data block */, r->pending_handle); } + BlockHandle filter_block_handle, metaindex_block_handle, index_block_handle, + compression_dict_block_handle, range_del_block_handle; + // Write filter block + if (ok() && r->filter_builder != nullptr) { + Status s = Status::Incomplete(); + while (s.IsIncomplete()) { + Slice filter_content = r->filter_builder->Finish(filter_block_handle, &s); + assert(s.ok() || s.IsIncomplete()); + r->props.filter_size += filter_content.size(); + WriteRawBlock(filter_content, kNoCompression, &filter_block_handle); + } + } + IndexBuilder::IndexBlocks index_blocks; - auto s = r->index_builder->Finish(&index_blocks); - if (!s.ok()) { - return s; + auto index_builder_status = r->index_builder->Finish(&index_blocks); + if (index_builder_status.IsIncomplete()) { + // We we have more than one index partition then meta_blocks are not + // supported for the index. Currently meta_blocks are used only by + // HashIndexBuilder which is not multi-partition. + assert(index_blocks.meta_blocks.empty()); + } else if (!index_builder_status.ok()) { + return index_builder_status; } // Write meta blocks and metaindex block with the following order. // 1. [meta block: filter] - // 2. [other meta blocks] - // 3. [meta block: properties] - // 4. [metaindex block] + // 2. [meta block: properties] + // 3. [meta block: compression dictionary] + // 4. [meta block: range deletion tombstone] + // 5. [metaindex block] // write meta blocks MetaIndexBuilder meta_index_builder; for (const auto& item : index_blocks.meta_blocks) { BlockHandle block_handle; - WriteBlock(item.second, &block_handle); + WriteBlock(item.second, &block_handle, false /* is_data_block */); meta_index_builder.Add(item.first, block_handle); } if (ok()) { - if (r->filter_block != nullptr) { + if (r->filter_builder != nullptr) { // Add mapping from ".Name" to location // of filter data. - std::string key = BlockBasedTable::kFilterBlockPrefix; + std::string key; + if (r->filter_builder->IsBlockBased()) { + key = BlockBasedTable::kFilterBlockPrefix; + } else { + key = r->table_options.partition_filters + ? BlockBasedTable::kPartitionedFilterBlockPrefix + : BlockBasedTable::kFullFilterBlockPrefix; + } key.append(r->table_options.filter_policy->Name()); meta_index_builder.Add(key, filter_block_handle); } - // Write properties block. + // Write properties and compression dictionary blocks. { PropertyBlockBuilder property_block_builder; - std::vector failed_user_prop_collectors; + r->props.column_family_id = r->column_family_id; + r->props.column_family_name = r->column_family_name; r->props.filter_policy_name = r->table_options.filter_policy != nullptr ? r->table_options.filter_policy->Name() : ""; r->props.index_size = r->index_builder->EstimatedSize() + kBlockTrailerSize; + r->props.comparator_name = r->ioptions.user_comparator != nullptr + ? r->ioptions.user_comparator->Name() + : "nullptr"; + r->props.merge_operator_name = r->ioptions.merge_operator != nullptr + ? r->ioptions.merge_operator->Name() + : "nullptr"; + r->props.compression_name = CompressionTypeToString(r->compression_type); + r->props.prefix_extractor_name = + r->ioptions.prefix_extractor != nullptr + ? r->ioptions.prefix_extractor->Name() + : "nullptr"; + + std::string property_collectors_names = "["; + property_collectors_names = "["; + for (size_t i = 0; + i < r->ioptions.table_properties_collector_factories.size(); ++i) { + if (i != 0) { + property_collectors_names += ","; + } + property_collectors_names += + r->ioptions.table_properties_collector_factories[i]->Name(); + } + property_collectors_names += "]"; + r->props.property_collectors_names = property_collectors_names; + if (r->table_options.index_type == + BlockBasedTableOptions::kTwoLevelIndexSearch) { + assert(r->p_index_builder_ != nullptr); + r->props.index_partitions = r->p_index_builder_->NumPartitions(); + r->props.top_level_index_size = + r->p_index_builder_->EstimateTopLevelIndexSize(r->offset); + } + r->props.creation_time = r->creation_time; + r->props.oldest_key_time = r->oldest_key_time; // Add basic properties property_block_builder.AddTableProperty(r->props); // Add use collected properties NotifyCollectTableCollectorsOnFinish(r->table_properties_collectors, - r->options.info_log.get(), + r->ioptions.info_log, &property_block_builder); BlockHandle properties_block_handle; @@ -727,9 +758,22 @@ Status BlockBasedTableBuilder::Finish() { kNoCompression, &properties_block_handle ); - meta_index_builder.Add(kPropertiesBlock, properties_block_handle); - } // end of properties block writing + + // Write compression dictionary block + if (r->compression_dict && r->compression_dict->size()) { + WriteRawBlock(*r->compression_dict, kNoCompression, + &compression_dict_block_handle); + meta_index_builder.Add(kCompressionDictBlock, + compression_dict_block_handle); + } + } // end of properties/compression dictionary block writing + + if (ok() && !r->range_del_block.empty()) { + WriteRawBlock(r->range_del_block.Finish(), kNoCompression, + &range_del_block_handle); + meta_index_builder.Add(kRangeDelBlock, range_del_block_handle); + } // range deletion tombstone meta block } // meta blocks // Write index block @@ -737,7 +781,21 @@ Status BlockBasedTableBuilder::Finish() { // flush the meta index block WriteRawBlock(meta_index_builder.Finish(), kNoCompression, &metaindex_block_handle); - WriteBlock(index_blocks.index_block_contents, &index_block_handle); + + const bool is_data_block = true; + WriteBlock(index_blocks.index_block_contents, &index_block_handle, + !is_data_block); + // If there are more index partitions, finish them and write them out + Status& s = index_builder_status; + while (s.IsIncomplete()) { + s = r->index_builder->Finish(&index_blocks, index_block_handle); + if (!s.ok() && !s.IsIncomplete()) { + return s; + } + WriteBlock(index_blocks.index_block_contents, &index_block_handle, + !is_data_block); + // The last index_block_handle will be for the partition index block + } } // Write footer @@ -749,44 +807,25 @@ Status BlockBasedTableBuilder::Finish() { // TODO(icanadi) at some point in the future, when we're absolutely sure // nobody will roll back to RocksDB 2.x versions, retire the legacy magic // number and always write new table files with new magic number - bool legacy = (r->table_options.checksum == kCRC32c); + bool legacy = (r->table_options.format_version == 0); + // this is guaranteed by BlockBasedTableBuilder's constructor + assert(r->table_options.checksum == kCRC32c || + r->table_options.format_version != 0); Footer footer(legacy ? kLegacyBlockBasedTableMagicNumber - : kBlockBasedTableMagicNumber); + : kBlockBasedTableMagicNumber, + r->table_options.format_version); footer.set_metaindex_handle(metaindex_block_handle); footer.set_index_handle(index_block_handle); footer.set_checksum(r->table_options.checksum); std::string footer_encoding; footer.EncodeTo(&footer_encoding); + assert(r->status.ok()); r->status = r->file->Append(footer_encoding); if (r->status.ok()) { r->offset += footer_encoding.size(); } } - // Print out the table stats - if (ok()) { - // user collected properties - std::string user_collected; - user_collected.reserve(1024); - for (const auto& collector : r->table_properties_collectors) { - for (const auto& prop : collector->GetReadableProperties()) { - user_collected.append(prop.first); - user_collected.append("="); - user_collected.append(prop.second); - user_collected.append("; "); - } - } - - Log( - r->options.info_log, - "Table was constructed:\n" - " [basic properties]: %s\n" - " [user collected properties]: %s", - r->props.ToString().c_str(), - user_collected.c_str() - ); - } - return r->status; } @@ -804,6 +843,28 @@ uint64_t BlockBasedTableBuilder::FileSize() const { return rep_->offset; } -const std::string BlockBasedTable::kFilterBlockPrefix = "filter."; +bool BlockBasedTableBuilder::NeedCompact() const { + for (const auto& collector : rep_->table_properties_collectors) { + if (collector->NeedCompact()) { + return true; + } + } + return false; +} +TableProperties BlockBasedTableBuilder::GetTableProperties() const { + TableProperties ret = rep_->props; + for (const auto& collector : rep_->table_properties_collectors) { + for (const auto& prop : collector->GetReadableProperties()) { + ret.readable_properties.insert(prop); + } + collector->Finish(&ret.user_collected_properties); + } + return ret; +} + +const std::string BlockBasedTable::kFilterBlockPrefix = "filter."; +const std::string BlockBasedTable::kFullFilterBlockPrefix = "fullfilter."; +const std::string BlockBasedTable::kPartitionedFilterBlockPrefix = + "partitionedfilter."; } // namespace rocksdb diff --git a/table/block_based_table_builder.h b/table/block_based_table_builder.h index 72a2f207a66..36dfce1f0fb 100644 --- a/table/block_based_table_builder.h +++ b/table/block_based_table_builder.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,8 +10,12 @@ #pragma once #include #include +#include +#include +#include #include "rocksdb/flush_block_policy.h" +#include "rocksdb/listener.h" #include "rocksdb/options.h" #include "rocksdb/status.h" #include "table/table_builder.h" @@ -23,15 +27,28 @@ class BlockHandle; class WritableFile; struct BlockBasedTableOptions; +extern const uint64_t kBlockBasedTableMagicNumber; +extern const uint64_t kLegacyBlockBasedTableMagicNumber; + class BlockBasedTableBuilder : public TableBuilder { public: // Create a builder that will store the contents of the table it is // building in *file. Does not close the file. It is up to the // caller to close the file after calling Finish(). - BlockBasedTableBuilder(const Options& options, - const BlockBasedTableOptions& table_options, - const InternalKeyComparator& internal_comparator, - WritableFile* file, CompressionType compression_type); + // @param compression_dict Data for presetting the compression library's + // dictionary, or nullptr. + BlockBasedTableBuilder( + const ImmutableCFOptions& ioptions, + const BlockBasedTableOptions& table_options, + const InternalKeyComparator& internal_comparator, + const std::vector>* + int_tbl_prop_collector_factories, + uint32_t column_family_id, WritableFileWriter* file, + const CompressionType compression_type, + const CompressionOptions& compression_opts, + const std::string* compression_dict, const bool skip_filters, + const std::string& column_family_name, const uint64_t creation_time = 0, + const uint64_t oldest_key_time = 0); // REQUIRES: Either Finish() or Abandon() has been called. ~BlockBasedTableBuilder(); @@ -63,13 +80,22 @@ class BlockBasedTableBuilder : public TableBuilder { // Finish() call, returns the size of the final generated file. uint64_t FileSize() const override; + bool NeedCompact() const override; + + // Get table properties + TableProperties GetTableProperties() const override; + private: bool ok() const { return status().ok(); } - // Call block's Finish() method and then write the finalize block contents to - // file. - void WriteBlock(BlockBuilder* block, BlockHandle* handle); - // Directly write block content to the file. - void WriteBlock(const Slice& block_contents, BlockHandle* handle); + + // Call block's Finish() method + // and then write the compressed block contents to file. + void WriteBlock(BlockBuilder* block, BlockHandle* handle, bool is_data_block); + + // Compress and write block content to the file. + void WriteBlock(const Slice& block_contents, BlockHandle* handle, + bool is_data_block); + // Directly write data to the file. void WriteRawBlock(const Slice& data, CompressionType, BlockHandle* handle); Status InsertBlockInCache(const Slice& block_contents, const CompressionType type, @@ -94,4 +120,10 @@ class BlockBasedTableBuilder : public TableBuilder { void operator=(const BlockBasedTableBuilder&) = delete; }; +Slice CompressBlock(const Slice& raw, + const CompressionOptions& compression_options, + CompressionType* type, uint32_t format_version, + const Slice& compression_dict, + std::string* compressed_output); + } // namespace rocksdb diff --git a/table/block_based_table_factory.cc b/table/block_based_table_factory.cc index de30fb383eb..0c6bbbcb64b 100644 --- a/table/block_based_table_factory.cc +++ b/table/block_based_table_factory.cc @@ -1,30 +1,33 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. - #include "table/block_based_table_factory.h" #include #include #include -#include "rocksdb/flush_block_policy.h" +#include "options/options_helper.h" +#include "port/port.h" #include "rocksdb/cache.h" +#include "rocksdb/convenience.h" +#include "rocksdb/flush_block_policy.h" #include "table/block_based_table_builder.h" #include "table/block_based_table_reader.h" -#include "port/port.h" +#include "table/format.h" +#include "util/string_util.h" namespace rocksdb { BlockBasedTableFactory::BlockBasedTableFactory( - const BlockBasedTableOptions& table_options) - : table_options_(table_options) { + const BlockBasedTableOptions& _table_options) + : table_options_(_table_options) { if (table_options_.flush_block_policy_factory == nullptr) { table_options_.flush_block_policy_factory.reset( new FlushBlockBySizePolicyFactory()); @@ -38,28 +41,77 @@ BlockBasedTableFactory::BlockBasedTableFactory( table_options_.block_size_deviation > 100) { table_options_.block_size_deviation = 0; } + if (table_options_.block_restart_interval < 1) { + table_options_.block_restart_interval = 1; + } + if (table_options_.index_block_restart_interval < 1) { + table_options_.index_block_restart_interval = 1; + } + if (table_options_.partition_filters && + table_options_.index_type != + BlockBasedTableOptions::kTwoLevelIndexSearch) { + // We do not support partitioned filters without partitioning indexes + table_options_.partition_filters = false; + } } Status BlockBasedTableFactory::NewTableReader( - const Options& options, const EnvOptions& soptions, - const InternalKeyComparator& internal_comparator, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table_reader) const { - return BlockBasedTable::Open(options, soptions, table_options_, - internal_comparator, std::move(file), file_size, - table_reader); + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + unique_ptr* table_reader, + bool prefetch_index_and_filter_in_cache) const { + return BlockBasedTable::Open( + table_reader_options.ioptions, table_reader_options.env_options, + table_options_, table_reader_options.internal_comparator, std::move(file), + file_size, table_reader, prefetch_index_and_filter_in_cache, + table_reader_options.skip_filters, table_reader_options.level); } TableBuilder* BlockBasedTableFactory::NewTableBuilder( - const Options& options, const InternalKeyComparator& internal_comparator, - WritableFile* file, CompressionType compression_type) const { - + const TableBuilderOptions& table_builder_options, uint32_t column_family_id, + WritableFileWriter* file) const { auto table_builder = new BlockBasedTableBuilder( - options, table_options_, internal_comparator, file, compression_type); + table_builder_options.ioptions, table_options_, + table_builder_options.internal_comparator, + table_builder_options.int_tbl_prop_collector_factories, column_family_id, + file, table_builder_options.compression_type, + table_builder_options.compression_opts, + table_builder_options.compression_dict, + table_builder_options.skip_filters, + table_builder_options.column_family_name, + table_builder_options.creation_time, + table_builder_options.oldest_key_time); return table_builder; } +Status BlockBasedTableFactory::SanitizeOptions( + const DBOptions& db_opts, + const ColumnFamilyOptions& cf_opts) const { + if (table_options_.index_type == BlockBasedTableOptions::kHashSearch && + cf_opts.prefix_extractor == nullptr) { + return Status::InvalidArgument("Hash index is specified for block-based " + "table, but prefix_extractor is not given"); + } + if (table_options_.cache_index_and_filter_blocks && + table_options_.no_block_cache) { + return Status::InvalidArgument("Enable cache_index_and_filter_blocks, " + ", but block cache is disabled"); + } + if (table_options_.pin_l0_filter_and_index_blocks_in_cache && + table_options_.no_block_cache) { + return Status::InvalidArgument( + "Enable pin_l0_filter_and_index_blocks_in_cache, " + ", but block cache is disabled"); + } + if (!BlockBasedTableSupportedVersion(table_options_.format_version)) { + return Status::InvalidArgument( + "Unsupported BlockBasedTable format_version. Please check " + "include/rocksdb/table.h for more info"); + } + return Status::OK(); +} + std::string BlockBasedTableFactory::GetPrintableTableOptions() const { std::string ret; ret.reserve(20000); @@ -68,11 +120,19 @@ std::string BlockBasedTableFactory::GetPrintableTableOptions() const { snprintf(buffer, kBufferSize, " flush_block_policy_factory: %s (%p)\n", table_options_.flush_block_policy_factory->Name(), - table_options_.flush_block_policy_factory.get()); + static_cast(table_options_.flush_block_policy_factory.get())); ret.append(buffer); snprintf(buffer, kBufferSize, " cache_index_and_filter_blocks: %d\n", table_options_.cache_index_and_filter_blocks); ret.append(buffer); + snprintf(buffer, kBufferSize, + " cache_index_and_filter_blocks_with_high_priority: %d\n", + table_options_.cache_index_and_filter_blocks_with_high_priority); + ret.append(buffer); + snprintf(buffer, kBufferSize, + " pin_l0_filter_and_index_blocks_in_cache: %d\n", + table_options_.pin_l0_filter_and_index_blocks_in_cache); + ret.append(buffer); snprintf(buffer, kBufferSize, " index_type: %d\n", table_options_.index_type); ret.append(buffer); @@ -86,22 +146,41 @@ std::string BlockBasedTableFactory::GetPrintableTableOptions() const { table_options_.no_block_cache); ret.append(buffer); snprintf(buffer, kBufferSize, " block_cache: %p\n", - table_options_.block_cache.get()); + static_cast(table_options_.block_cache.get())); ret.append(buffer); if (table_options_.block_cache) { - snprintf(buffer, kBufferSize, " block_cache_size: %zd\n", - table_options_.block_cache->GetCapacity()); - ret.append(buffer); + const char* block_cache_name = table_options_.block_cache->Name(); + if (block_cache_name != nullptr) { + snprintf(buffer, kBufferSize, " block_cache_name: %s\n", + block_cache_name); + ret.append(buffer); + } + ret.append(" block_cache_options:\n"); + ret.append(table_options_.block_cache->GetPrintableOptions()); } snprintf(buffer, kBufferSize, " block_cache_compressed: %p\n", - table_options_.block_cache_compressed.get()); + static_cast(table_options_.block_cache_compressed.get())); ret.append(buffer); if (table_options_.block_cache_compressed) { - snprintf(buffer, kBufferSize, " block_cache_compressed_size: %zd\n", - table_options_.block_cache_compressed->GetCapacity()); + const char* block_cache_compressed_name = + table_options_.block_cache_compressed->Name(); + if (block_cache_compressed_name != nullptr) { + snprintf(buffer, kBufferSize, " block_cache_name: %s\n", + block_cache_compressed_name); + ret.append(buffer); + } + ret.append(" block_cache_compressed_options:\n"); + ret.append(table_options_.block_cache_compressed->GetPrintableOptions()); + } + snprintf(buffer, kBufferSize, " persistent_cache: %p\n", + static_cast(table_options_.persistent_cache.get())); + ret.append(buffer); + if (table_options_.persistent_cache) { + snprintf(buffer, kBufferSize, " persistent_cache_options:\n"); ret.append(buffer); + ret.append(table_options_.persistent_cache->GetPrintableOptions()); } - snprintf(buffer, kBufferSize, " block_size: %zd\n", + snprintf(buffer, kBufferSize, " block_size: %" ROCKSDB_PRIszt "\n", table_options_.block_size); ret.append(buffer); snprintf(buffer, kBufferSize, " block_size_deviation: %d\n", @@ -110,6 +189,9 @@ std::string BlockBasedTableFactory::GetPrintableTableOptions() const { snprintf(buffer, kBufferSize, " block_restart_interval: %d\n", table_options_.block_restart_interval); ret.append(buffer); + snprintf(buffer, kBufferSize, " index_block_restart_interval: %d\n", + table_options_.index_block_restart_interval); + ret.append(buffer); snprintf(buffer, kBufferSize, " filter_policy: %s\n", table_options_.filter_policy == nullptr ? "nullptr" : table_options_.filter_policy->Name()); @@ -117,18 +199,219 @@ std::string BlockBasedTableFactory::GetPrintableTableOptions() const { snprintf(buffer, kBufferSize, " whole_key_filtering: %d\n", table_options_.whole_key_filtering); ret.append(buffer); + snprintf(buffer, kBufferSize, " format_version: %d\n", + table_options_.format_version); + ret.append(buffer); return ret; } +#ifndef ROCKSDB_LITE +namespace { +bool SerializeSingleBlockBasedTableOption( + std::string* opt_string, const BlockBasedTableOptions& bbt_options, + const std::string& name, const std::string& delimiter) { + auto iter = block_based_table_type_info.find(name); + if (iter == block_based_table_type_info.end()) { + return false; + } + auto& opt_info = iter->second; + const char* opt_address = + reinterpret_cast(&bbt_options) + opt_info.offset; + std::string value; + bool result = SerializeSingleOptionHelper(opt_address, opt_info.type, &value); + if (result) { + *opt_string = name + "=" + value + delimiter; + } + return result; +} +} // namespace + +Status BlockBasedTableFactory::GetOptionString( + std::string* opt_string, const std::string& delimiter) const { + assert(opt_string); + opt_string->clear(); + for (auto iter = block_based_table_type_info.begin(); + iter != block_based_table_type_info.end(); ++iter) { + if (iter->second.verification == OptionVerificationType::kDeprecated) { + // If the option is no longer used in rocksdb and marked as deprecated, + // we skip it in the serialization. + continue; + } + std::string single_output; + bool result = SerializeSingleBlockBasedTableOption( + &single_output, table_options_, iter->first, delimiter); + assert(result); + if (result) { + opt_string->append(single_output); + } + } + return Status::OK(); +} +#else +Status BlockBasedTableFactory::GetOptionString( + std::string* opt_string, const std::string& delimiter) const { + return Status::OK(); +} +#endif // !ROCKSDB_LITE + +const BlockBasedTableOptions& BlockBasedTableFactory::table_options() const { + return table_options_; +} + +#ifndef ROCKSDB_LITE +namespace { +std::string ParseBlockBasedTableOption(const std::string& name, + const std::string& org_value, + BlockBasedTableOptions* new_options, + bool input_strings_escaped = false, + bool ignore_unknown_options = false) { + const std::string& value = + input_strings_escaped ? UnescapeOptionString(org_value) : org_value; + if (!input_strings_escaped) { + // if the input string is not escaped, it means this function is + // invoked from SetOptions, which takes the old format. + if (name == "block_cache") { + new_options->block_cache = NewLRUCache(ParseSizeT(value)); + return ""; + } else if (name == "block_cache_compressed") { + new_options->block_cache_compressed = NewLRUCache(ParseSizeT(value)); + return ""; + } else if (name == "filter_policy") { + // Expect the following format + // bloomfilter:int:bool + const std::string kName = "bloomfilter:"; + if (value.compare(0, kName.size(), kName) != 0) { + return "Invalid filter policy name"; + } + size_t pos = value.find(':', kName.size()); + if (pos == std::string::npos) { + return "Invalid filter policy config, missing bits_per_key"; + } + int bits_per_key = + ParseInt(trim(value.substr(kName.size(), pos - kName.size()))); + bool use_block_based_builder = + ParseBoolean("use_block_based_builder", trim(value.substr(pos + 1))); + new_options->filter_policy.reset( + NewBloomFilterPolicy(bits_per_key, use_block_based_builder)); + return ""; + } + } + const auto iter = block_based_table_type_info.find(name); + if (iter == block_based_table_type_info.end()) { + if (ignore_unknown_options) { + return ""; + } else { + return "Unrecognized option"; + } + } + const auto& opt_info = iter->second; + if (opt_info.verification != OptionVerificationType::kDeprecated && + !ParseOptionHelper(reinterpret_cast(new_options) + opt_info.offset, + opt_info.type, value)) { + return "Invalid value"; + } + return ""; +} +} // namespace + +Status GetBlockBasedTableOptionsFromString( + const BlockBasedTableOptions& table_options, const std::string& opts_str, + BlockBasedTableOptions* new_table_options) { + std::unordered_map opts_map; + Status s = StringToMap(opts_str, &opts_map); + if (!s.ok()) { + return s; + } + + return GetBlockBasedTableOptionsFromMap(table_options, opts_map, + new_table_options); +} + +Status GetBlockBasedTableOptionsFromMap( + const BlockBasedTableOptions& table_options, + const std::unordered_map& opts_map, + BlockBasedTableOptions* new_table_options, bool input_strings_escaped, + bool ignore_unknown_options) { + assert(new_table_options); + *new_table_options = table_options; + for (const auto& o : opts_map) { + auto error_message = ParseBlockBasedTableOption( + o.first, o.second, new_table_options, input_strings_escaped, + ignore_unknown_options); + if (error_message != "") { + const auto iter = block_based_table_type_info.find(o.first); + if (iter == block_based_table_type_info.end() || + !input_strings_escaped || // !input_strings_escaped indicates + // the old API, where everything is + // parsable. + (iter->second.verification != OptionVerificationType::kByName && + iter->second.verification != + OptionVerificationType::kByNameAllowNull && + iter->second.verification != OptionVerificationType::kDeprecated)) { + // Restore "new_options" to the default "base_options". + *new_table_options = table_options; + return Status::InvalidArgument("Can't parse BlockBasedTableOptions:", + o.first + " " + error_message); + } + } + } + return Status::OK(); +} + +Status VerifyBlockBasedTableFactory( + const BlockBasedTableFactory* base_tf, + const BlockBasedTableFactory* file_tf, + OptionsSanityCheckLevel sanity_check_level) { + if ((base_tf != nullptr) != (file_tf != nullptr) && + sanity_check_level > kSanityLevelNone) { + return Status::Corruption( + "[RocksDBOptionsParser]: Inconsistent TableFactory class type"); + } + if (base_tf == nullptr) { + return Status::OK(); + } + assert(file_tf != nullptr); + + const auto& base_opt = base_tf->table_options(); + const auto& file_opt = file_tf->table_options(); + + for (auto& pair : block_based_table_type_info) { + if (pair.second.verification == OptionVerificationType::kDeprecated) { + // We skip checking deprecated variables as they might + // contain random values since they might not be initialized + continue; + } + if (BBTOptionSanityCheckLevel(pair.first) <= sanity_check_level) { + if (!AreEqualOptions(reinterpret_cast(&base_opt), + reinterpret_cast(&file_opt), + pair.second, pair.first, nullptr)) { + return Status::Corruption( + "[RocksDBOptionsParser]: " + "failed the verification on BlockBasedTableOptions::", + pair.first); + } + } + } + return Status::OK(); +} +#endif // !ROCKSDB_LITE + TableFactory* NewBlockBasedTableFactory( - const BlockBasedTableOptions& table_options) { - return new BlockBasedTableFactory(table_options); + const BlockBasedTableOptions& _table_options) { + return new BlockBasedTableFactory(_table_options); } +const std::string BlockBasedTableFactory::kName = "BlockBasedTable"; const std::string BlockBasedTablePropertyNames::kIndexType = "rocksdb.block.based.table.index.type"; +const std::string BlockBasedTablePropertyNames::kWholeKeyFiltering = + "rocksdb.block.based.table.whole.key.filtering"; +const std::string BlockBasedTablePropertyNames::kPrefixFiltering = + "rocksdb.block.based.table.prefix.filtering"; const std::string kHashIndexPrefixesBlock = "rocksdb.hashindex.prefixes"; const std::string kHashIndexPrefixesMetadataBlock = "rocksdb.hashindex.metadata"; +const std::string kPropTrue = "1"; +const std::string kPropFalse = "0"; } // namespace rocksdb diff --git a/table/block_based_table_factory.h b/table/block_based_table_factory.h index d7045346af0..39e3eac0b37 100644 --- a/table/block_based_table_factory.h +++ b/table/block_based_table_factory.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -13,14 +13,14 @@ #include #include +#include "db/dbformat.h" +#include "options/options_helper.h" +#include "options/options_parser.h" #include "rocksdb/flush_block_policy.h" -#include "rocksdb/options.h" #include "rocksdb/table.h" -#include "db/dbformat.h" namespace rocksdb { -struct Options; struct EnvOptions; using std::unique_ptr; @@ -33,29 +33,125 @@ class BlockBasedTableFactory : public TableFactory { ~BlockBasedTableFactory() {} - const char* Name() const override { return "BlockBasedTable"; } + const char* Name() const override { return kName.c_str(); } - Status NewTableReader(const Options& options, const EnvOptions& soptions, - const InternalKeyComparator& internal_comparator, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table_reader) const override; + Status NewTableReader( + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + unique_ptr* table_reader, + bool prefetch_index_and_filter_in_cache = true) const override; TableBuilder* NewTableBuilder( - const Options& options, const InternalKeyComparator& internal_comparator, - WritableFile* file, CompressionType compression_type) const override; + const TableBuilderOptions& table_builder_options, + uint32_t column_family_id, WritableFileWriter* file) const override; // Sanitizes the specified DB Options. - Status SanitizeDBOptions(const DBOptions* db_opts) const override { - return Status::OK(); - } + Status SanitizeOptions(const DBOptions& db_opts, + const ColumnFamilyOptions& cf_opts) const override; std::string GetPrintableTableOptions() const override; + Status GetOptionString(std::string* opt_string, + const std::string& delimiter) const override; + + const BlockBasedTableOptions& table_options() const; + + void* GetOptions() override { return &table_options_; } + + bool IsDeleteRangeSupported() const override { return true; } + + static const std::string kName; + private: BlockBasedTableOptions table_options_; }; extern const std::string kHashIndexPrefixesBlock; extern const std::string kHashIndexPrefixesMetadataBlock; - +extern const std::string kPropTrue; +extern const std::string kPropFalse; + +#ifndef ROCKSDB_LITE +extern Status VerifyBlockBasedTableFactory( + const BlockBasedTableFactory* base_tf, + const BlockBasedTableFactory* file_tf, + OptionsSanityCheckLevel sanity_check_level); + +static std::unordered_map + block_based_table_type_info = { + /* currently not supported + std::shared_ptr block_cache = nullptr; + std::shared_ptr block_cache_compressed = nullptr; + */ + {"flush_block_policy_factory", + {offsetof(struct BlockBasedTableOptions, flush_block_policy_factory), + OptionType::kFlushBlockPolicyFactory, OptionVerificationType::kByName, + false, 0}}, + {"cache_index_and_filter_blocks", + {offsetof(struct BlockBasedTableOptions, + cache_index_and_filter_blocks), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"cache_index_and_filter_blocks_with_high_priority", + {offsetof(struct BlockBasedTableOptions, + cache_index_and_filter_blocks_with_high_priority), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"pin_l0_filter_and_index_blocks_in_cache", + {offsetof(struct BlockBasedTableOptions, + pin_l0_filter_and_index_blocks_in_cache), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"index_type", + {offsetof(struct BlockBasedTableOptions, index_type), + OptionType::kBlockBasedTableIndexType, + OptionVerificationType::kNormal, false, 0}}, + {"hash_index_allow_collision", + {offsetof(struct BlockBasedTableOptions, hash_index_allow_collision), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"checksum", + {offsetof(struct BlockBasedTableOptions, checksum), + OptionType::kChecksumType, OptionVerificationType::kNormal, false, + 0}}, + {"no_block_cache", + {offsetof(struct BlockBasedTableOptions, no_block_cache), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"block_size", + {offsetof(struct BlockBasedTableOptions, block_size), + OptionType::kSizeT, OptionVerificationType::kNormal, false, 0}}, + {"block_size_deviation", + {offsetof(struct BlockBasedTableOptions, block_size_deviation), + OptionType::kInt, OptionVerificationType::kNormal, false, 0}}, + {"block_restart_interval", + {offsetof(struct BlockBasedTableOptions, block_restart_interval), + OptionType::kInt, OptionVerificationType::kNormal, false, 0}}, + {"index_block_restart_interval", + {offsetof(struct BlockBasedTableOptions, index_block_restart_interval), + OptionType::kInt, OptionVerificationType::kNormal, false, 0}}, + {"index_per_partition", + {0, OptionType::kUInt64T, OptionVerificationType::kDeprecated, false, + 0}}, + {"metadata_block_size", + {offsetof(struct BlockBasedTableOptions, metadata_block_size), + OptionType::kUInt64T, OptionVerificationType::kNormal, false, 0}}, + {"partition_filters", + {offsetof(struct BlockBasedTableOptions, partition_filters), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"filter_policy", + {offsetof(struct BlockBasedTableOptions, filter_policy), + OptionType::kFilterPolicy, OptionVerificationType::kByName, false, + 0}}, + {"whole_key_filtering", + {offsetof(struct BlockBasedTableOptions, whole_key_filtering), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"skip_table_builder_flush", + {0, OptionType::kBoolean, OptionVerificationType::kDeprecated, false, + 0}}, + {"format_version", + {offsetof(struct BlockBasedTableOptions, format_version), + OptionType::kUInt32T, OptionVerificationType::kNormal, false, 0}}, + {"verify_compression", + {offsetof(struct BlockBasedTableOptions, verify_compression), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}, + {"read_amp_bytes_per_bit", + {offsetof(struct BlockBasedTableOptions, read_amp_bytes_per_bit), + OptionType::kSizeT, OptionVerificationType::kNormal, false, 0}}}; +#endif // !ROCKSDB_LITE } // namespace rocksdb diff --git a/table/block_based_table_reader.cc b/table/block_based_table_reader.cc index 0be38a1dc33..d8c6d807c8c 100644 --- a/table/block_based_table_reader.cc +++ b/table/block_based_table_reader.cc @@ -1,18 +1,21 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. - #include "table/block_based_table_reader.h" +#include +#include #include #include +#include #include "db/dbformat.h" +#include "db/pinned_iterators_manager.h" #include "rocksdb/cache.h" #include "rocksdb/comparator.h" @@ -25,16 +28,26 @@ #include "rocksdb/table_properties.h" #include "table/block.h" -#include "table/filter_block.h" -#include "table/block_hash_index.h" +#include "table/block_based_filter_block.h" +#include "table/block_based_table_factory.h" #include "table/block_prefix_index.h" +#include "table/filter_block.h" #include "table/format.h" +#include "table/full_filter_block.h" +#include "table/get_context.h" +#include "table/internal_iterator.h" #include "table/meta_blocks.h" +#include "table/partitioned_filter_block.h" +#include "table/persistent_cache_helper.h" +#include "table/sst_file_writer_collectors.h" #include "table/two_level_iterator.h" +#include "monitoring/perf_context_imp.h" #include "util/coding.h" -#include "util/perf_context_imp.h" +#include "util/file_reader_writer.h" #include "util/stop_watch.h" +#include "util/string_util.h" +#include "util/sync_point.h" namespace rocksdb { @@ -45,27 +58,32 @@ using std::unique_ptr; typedef BlockBasedTable::IndexReader IndexReader; -namespace { - -// The longest the prefix of the cache key used to identify blocks can be. -// We are using the fact that we know for Posix files the unique ID is three -// varints. -// For some reason, compiling for iOS complains that this variable is unused -const size_t kMaxCacheKeyPrefixSize __attribute__((unused)) = - kMaxVarint64Length * 3 + 1; +BlockBasedTable::~BlockBasedTable() { + Close(); + delete rep_; +} +namespace { // Read the block identified by "handle" from "file". // The only relevant option is options.verify_checksums for now. // On failure return non-OK. // On success fill *result and return OK - caller owns *result -Status ReadBlockFromFile(RandomAccessFile* file, const Footer& footer, - const ReadOptions& options, const BlockHandle& handle, - Block** result, Env* env, bool do_uncompress = true) { +// @param compression_dict Data for presetting the compression library's +// dictionary. +Status ReadBlockFromFile( + RandomAccessFileReader* file, FilePrefetchBuffer* prefetch_buffer, + const Footer& footer, const ReadOptions& options, const BlockHandle& handle, + std::unique_ptr* result, const ImmutableCFOptions& ioptions, + bool do_uncompress, const Slice& compression_dict, + const PersistentCacheOptions& cache_options, SequenceNumber global_seqno, + size_t read_amp_bytes_per_bit) { BlockContents contents; - Status s = ReadBlockContents(file, footer, options, handle, &contents, env, - do_uncompress); + Status s = ReadBlockContents(file, prefetch_buffer, footer, options, handle, + &contents, ioptions, do_uncompress, + compression_dict, cache_options); if (s.ok()) { - *result = new Block(contents); + result->reset(new Block(std::move(contents), global_seqno, + read_amp_bytes_per_bit, ioptions.statistics)); } return s; @@ -84,6 +102,9 @@ void DeleteCachedEntry(const Slice& key, void* value) { delete entry; } +void DeleteCachedFilterEntry(const Slice& key, void* value); +void DeleteCachedIndexEntry(const Slice& key, void* value); + // Release the cached entry and decrement its ref count. void ReleaseCachedEntry(void* arg, void* h) { Cache* cache = reinterpret_cast(arg); @@ -91,14 +112,14 @@ void ReleaseCachedEntry(void* arg, void* h) { cache->Release(handle); } -Slice GetCacheKey(const char* cache_key_prefix, size_t cache_key_prefix_size, - const BlockHandle& handle, char* cache_key) { +Slice GetCacheKeyFromOffset(const char* cache_key_prefix, + size_t cache_key_prefix_size, uint64_t offset, + char* cache_key) { assert(cache_key != nullptr); assert(cache_key_prefix_size != 0); - assert(cache_key_prefix_size <= kMaxCacheKeyPrefixSize); + assert(cache_key_prefix_size <= BlockBasedTable::kMaxCacheKeyPrefixSize); memcpy(cache_key, cache_key_prefix, cache_key_prefix_size); - char* end = - EncodeVarint64(cache_key + cache_key_prefix_size, handle.offset()); + char* end = EncodeVarint64(cache_key + cache_key_prefix_size, offset); return Slice(cache_key, static_cast(end - cache_key)); } @@ -106,11 +127,14 @@ Cache::Handle* GetEntryFromCache(Cache* block_cache, const Slice& key, Tickers block_cache_miss_ticker, Tickers block_cache_hit_ticker, Statistics* statistics) { - auto cache_handle = block_cache->Lookup(key); + auto cache_handle = block_cache->Lookup(key, statistics); if (cache_handle != nullptr) { PERF_COUNTER_ADD(block_cache_hit_count, 1); // overall cache hit RecordTick(statistics, BLOCK_CACHE_HIT); + // total bytes read from cache + RecordTick(statistics, BLOCK_CACHE_BYTES_READ, + block_cache->GetUsage(cache_handle)); // block-type specific cache hit RecordTick(statistics, block_cache_hit_ticker); } else { @@ -125,30 +149,151 @@ Cache::Handle* GetEntryFromCache(Cache* block_cache, const Slice& key, } // namespace -// -- IndexReader and its subclasses -// IndexReader is the interface that provide the functionality for index access. -class BlockBasedTable::IndexReader { +// Index that allows binary search lookup in a two-level index structure. +class PartitionIndexReader : public IndexReader, public Cleanable { public: - explicit IndexReader(const Comparator* comparator) - : comparator_(comparator) {} + // Read the partition index from the file and create an instance for + // `PartitionIndexReader`. + // On success, index_reader will be populated; otherwise it will remain + // unmodified. + static Status Create(BlockBasedTable* table, RandomAccessFileReader* file, + FilePrefetchBuffer* prefetch_buffer, + const Footer& footer, const BlockHandle& index_handle, + const ImmutableCFOptions& ioptions, + const InternalKeyComparator* icomparator, + IndexReader** index_reader, + const PersistentCacheOptions& cache_options, + const int level) { + std::unique_ptr index_block; + auto s = ReadBlockFromFile( + file, prefetch_buffer, footer, ReadOptions(), index_handle, + &index_block, ioptions, true /* decompress */, + Slice() /*compression dict*/, cache_options, + kDisableGlobalSequenceNumber, 0 /* read_amp_bytes_per_bit */); + + if (s.ok()) { + *index_reader = + new PartitionIndexReader(table, icomparator, std::move(index_block), + ioptions.statistics, level); + } + + return s; + } + + // return a two-level iterator: first level is on the partition index + virtual InternalIterator* NewIterator(BlockIter* iter = nullptr, + bool dont_care = true) override { + // Filters are already checked before seeking the index + const bool skip_filters = true; + const bool is_index = true; + return NewTwoLevelIterator( + new BlockBasedTable::BlockEntryIteratorState( + table_, ReadOptions(), icomparator_, skip_filters, is_index, + partition_map_.size() ? &partition_map_ : nullptr), + index_block_->NewIterator(icomparator_, nullptr, true)); + // TODO(myabandeh): Update TwoLevelIterator to be able to make use of + // on-stack BlockIter while the state is on heap. Currentlly it assumes + // the first level iter is always on heap and will attempt to delete it + // in its destructor. + } + + virtual void CacheDependencies(bool pin) override { + // Before read partitions, prefetch them to avoid lots of IOs + auto rep = table_->rep_; + BlockIter biter; + BlockHandle handle; + index_block_->NewIterator(icomparator_, &biter, true); + // Index partitions are assumed to be consecuitive. Prefetch them all. + // Read the first block offset + biter.SeekToFirst(); + Slice input = biter.value(); + Status s = handle.DecodeFrom(&input); + assert(s.ok()); + if (!s.ok()) { + ROCKS_LOG_WARN(rep->ioptions.info_log, + "Could not read first index partition"); + return; + } + uint64_t prefetch_off = handle.offset(); - virtual ~IndexReader() {} + // Read the last block's offset + biter.SeekToLast(); + input = biter.value(); + s = handle.DecodeFrom(&input); + assert(s.ok()); + if (!s.ok()) { + ROCKS_LOG_WARN(rep->ioptions.info_log, + "Could not read last index partition"); + return; + } + uint64_t last_off = handle.offset() + handle.size() + kBlockTrailerSize; + uint64_t prefetch_len = last_off - prefetch_off; + std::unique_ptr prefetch_buffer; + auto& file = table_->rep_->file; + prefetch_buffer.reset(new FilePrefetchBuffer()); + s = prefetch_buffer->Prefetch(file.get(), prefetch_off, prefetch_len); + + // After prefetch, read the partitions one by one + biter.SeekToFirst(); + auto ro = ReadOptions(); + Cache* block_cache = rep->table_options.block_cache.get(); + for (; biter.Valid(); biter.Next()) { + input = biter.value(); + s = handle.DecodeFrom(&input); + assert(s.ok()); + if (!s.ok()) { + ROCKS_LOG_WARN(rep->ioptions.info_log, + "Could not read index partition"); + continue; + } - // Create an iterator for index access. - // An iter is passed in, if it is not null, update this one and return it - // If it is null, create a new Iterator - virtual Iterator* NewIterator( - BlockIter* iter = nullptr, bool total_order_seek = true) = 0; + BlockBasedTable::CachableEntry block; + Slice compression_dict; + if (rep->compression_dict_block) { + compression_dict = rep->compression_dict_block->data; + } + const bool is_index = true; + s = table_->MaybeLoadDataBlockToCache(prefetch_buffer.get(), rep, ro, + handle, compression_dict, &block, + is_index); + + assert(s.ok() || block.value == nullptr); + if (s.ok() && block.value != nullptr) { + assert(block.cache_handle != nullptr); + if (pin) { + partition_map_[handle.offset()] = block; + RegisterCleanup(&ReleaseCachedEntry, block_cache, block.cache_handle); + } else { + block_cache->Release(block.cache_handle); + } + } + } + } - // The size of the index. - virtual size_t size() const = 0; + virtual size_t size() const override { return index_block_->size(); } + virtual size_t usable_size() const override { + return index_block_->usable_size(); + } - // Report an approximation of how much memory has been used other than memory - // that was allocated in block cache. - virtual size_t ApproximateMemoryUsage() const = 0; + virtual size_t ApproximateMemoryUsage() const override { + assert(index_block_); + return index_block_->ApproximateMemoryUsage(); + } - protected: - const Comparator* comparator_; + private: + PartitionIndexReader(BlockBasedTable* table, + const InternalKeyComparator* icomparator, + std::unique_ptr&& index_block, Statistics* stats, + const int level) + : IndexReader(icomparator, stats), + table_(table), + index_block_(std::move(index_block)) { + assert(index_block_ != nullptr); + } + BlockBasedTable* table_; + std::unique_ptr index_block_; + std::unordered_map> + partition_map_; }; // Index that allows binary search lookup for the first key of each block. @@ -160,27 +305,37 @@ class BinarySearchIndexReader : public IndexReader { // `BinarySearchIndexReader`. // On success, index_reader will be populated; otherwise it will remain // unmodified. - static Status Create(RandomAccessFile* file, const Footer& footer, - const BlockHandle& index_handle, Env* env, - const Comparator* comparator, - IndexReader** index_reader) { - Block* index_block = nullptr; - auto s = ReadBlockFromFile(file, footer, ReadOptions(), index_handle, - &index_block, env); + static Status Create(RandomAccessFileReader* file, + FilePrefetchBuffer* prefetch_buffer, + const Footer& footer, const BlockHandle& index_handle, + const ImmutableCFOptions& ioptions, + const InternalKeyComparator* icomparator, + IndexReader** index_reader, + const PersistentCacheOptions& cache_options) { + std::unique_ptr index_block; + auto s = ReadBlockFromFile( + file, prefetch_buffer, footer, ReadOptions(), index_handle, + &index_block, ioptions, true /* decompress */, + Slice() /*compression dict*/, cache_options, + kDisableGlobalSequenceNumber, 0 /* read_amp_bytes_per_bit */); if (s.ok()) { - *index_reader = new BinarySearchIndexReader(comparator, index_block); + *index_reader = new BinarySearchIndexReader( + icomparator, std::move(index_block), ioptions.statistics); } return s; } - virtual Iterator* NewIterator( - BlockIter* iter = nullptr, bool dont_care = true) override { - return index_block_->NewIterator(comparator_, iter, true); + virtual InternalIterator* NewIterator(BlockIter* iter = nullptr, + bool dont_care = true) override { + return index_block_->NewIterator(icomparator_, iter, true); } virtual size_t size() const override { return index_block_->size(); } + virtual size_t usable_size() const override { + return index_block_->usable_size(); + } virtual size_t ApproximateMemoryUsage() const override { assert(index_block_); @@ -188,8 +343,10 @@ class BinarySearchIndexReader : public IndexReader { } private: - BinarySearchIndexReader(const Comparator* comparator, Block* index_block) - : IndexReader(comparator), index_block_(index_block) { + BinarySearchIndexReader(const InternalKeyComparator* icomparator, + std::unique_ptr&& index_block, + Statistics* stats) + : IndexReader(icomparator, stats), index_block_(std::move(index_block)) { assert(index_block_ != nullptr); } std::unique_ptr index_block_; @@ -200,14 +357,21 @@ class BinarySearchIndexReader : public IndexReader { class HashIndexReader : public IndexReader { public: static Status Create(const SliceTransform* hash_key_extractor, - const Footer& footer, RandomAccessFile* file, Env* env, - const Comparator* comparator, + const Footer& footer, RandomAccessFileReader* file, + FilePrefetchBuffer* prefetch_buffer, + const ImmutableCFOptions& ioptions, + const InternalKeyComparator* icomparator, const BlockHandle& index_handle, - Iterator* meta_index_iter, IndexReader** index_reader, - bool hash_index_allow_collision) { - Block* index_block = nullptr; - auto s = ReadBlockFromFile(file, footer, ReadOptions(), index_handle, - &index_block, env); + InternalIterator* meta_index_iter, + IndexReader** index_reader, + bool hash_index_allow_collision, + const PersistentCacheOptions& cache_options) { + std::unique_ptr index_block; + auto s = ReadBlockFromFile( + file, prefetch_buffer, footer, ReadOptions(), index_handle, + &index_block, ioptions, true /* decompress */, + Slice() /*compression dict*/, cache_options, + kDisableGlobalSequenceNumber, 0 /* read_amp_bytes_per_bit */); if (!s.ok()) { return s; @@ -218,7 +382,8 @@ class HashIndexReader : public IndexReader { // So, Create will succeed regardless, from this point on. auto new_index_reader = - new HashIndexReader(comparator, index_block); + new HashIndexReader(icomparator, std::move(index_block), + ioptions.statistics); *index_reader = new_index_reader; // Get prefixes block @@ -241,68 +406,43 @@ class HashIndexReader : public IndexReader { // Read contents for the blocks BlockContents prefixes_contents; - s = ReadBlockContents(file, footer, ReadOptions(), prefixes_handle, - &prefixes_contents, env, true /* do decompression */); + s = ReadBlockContents(file, prefetch_buffer, footer, ReadOptions(), + prefixes_handle, &prefixes_contents, ioptions, + true /* decompress */, Slice() /*compression dict*/, + cache_options); if (!s.ok()) { return s; } BlockContents prefixes_meta_contents; - s = ReadBlockContents(file, footer, ReadOptions(), prefixes_meta_handle, - &prefixes_meta_contents, env, - true /* do decompression */); + s = ReadBlockContents(file, prefetch_buffer, footer, ReadOptions(), + prefixes_meta_handle, &prefixes_meta_contents, + ioptions, true /* decompress */, + Slice() /*compression dict*/, cache_options); if (!s.ok()) { - if (prefixes_contents.heap_allocated) { - delete[] prefixes_contents.data.data(); - } // TODO: log error return Status::OK(); } - if (!hash_index_allow_collision) { - // TODO: deprecate once hash_index_allow_collision proves to be stable. - BlockHashIndex* hash_index = nullptr; - s = CreateBlockHashIndex(hash_key_extractor, - prefixes_contents.data, - prefixes_meta_contents.data, - &hash_index); - // TODO: log error - if (s.ok()) { - new_index_reader->index_block_->SetBlockHashIndex(hash_index); - new_index_reader->OwnPrefixesContents(prefixes_contents); - } - } else { - BlockPrefixIndex* prefix_index = nullptr; - s = BlockPrefixIndex::Create(hash_key_extractor, - prefixes_contents.data, - prefixes_meta_contents.data, - &prefix_index); - // TODO: log error - if (s.ok()) { - new_index_reader->index_block_->SetBlockPrefixIndex(prefix_index); - } - } - - // Always release prefix meta block - if (prefixes_meta_contents.heap_allocated) { - delete[] prefixes_meta_contents.data.data(); - } - - // Release prefix content block if we don't own it. - if (!new_index_reader->own_prefixes_contents_) { - if (prefixes_contents.heap_allocated) { - delete[] prefixes_contents.data.data(); - } + BlockPrefixIndex* prefix_index = nullptr; + s = BlockPrefixIndex::Create(hash_key_extractor, prefixes_contents.data, + prefixes_meta_contents.data, &prefix_index); + // TODO: log error + if (s.ok()) { + new_index_reader->index_block_->SetBlockPrefixIndex(prefix_index); } return Status::OK(); } - virtual Iterator* NewIterator( - BlockIter* iter = nullptr, bool total_order_seek = true) override { - return index_block_->NewIterator(comparator_, iter, total_order_seek); + virtual InternalIterator* NewIterator(BlockIter* iter = nullptr, + bool total_order_seek = true) override { + return index_block_->NewIterator(icomparator_, iter, total_order_seek); } virtual size_t size() const override { return index_block_->size(); } + virtual size_t usable_size() const override { + return index_block_->usable_size(); + } virtual size_t ApproximateMemoryUsage() const override { assert(index_block_); @@ -311,109 +451,39 @@ class HashIndexReader : public IndexReader { } private: - HashIndexReader(const Comparator* comparator, Block* index_block) - : IndexReader(comparator), - index_block_(index_block), - own_prefixes_contents_(false) { + HashIndexReader(const InternalKeyComparator* icomparator, + std::unique_ptr&& index_block, Statistics* stats) + : IndexReader(icomparator, stats), index_block_(std::move(index_block)) { assert(index_block_ != nullptr); } ~HashIndexReader() { - if (own_prefixes_contents_ && prefixes_contents_.heap_allocated) { - delete[] prefixes_contents_.data.data(); - } - } - - void OwnPrefixesContents(const BlockContents& prefixes_contents) { - prefixes_contents_ = prefixes_contents; - own_prefixes_contents_ = true; } std::unique_ptr index_block_; - bool own_prefixes_contents_; BlockContents prefixes_contents_; }; - -struct BlockBasedTable::Rep { - Rep(const EnvOptions& storage_options, - const BlockBasedTableOptions& table_opt, - const InternalKeyComparator& internal_comparator) - : soptions(storage_options), table_options(table_opt), - filter_policy(table_opt.filter_policy.get()), - internal_comparator(internal_comparator) {} - - Options options; - const EnvOptions& soptions; - const BlockBasedTableOptions& table_options; - const FilterPolicy* const filter_policy; - const InternalKeyComparator& internal_comparator; - Status status; - unique_ptr file; - char cache_key_prefix[kMaxCacheKeyPrefixSize]; - size_t cache_key_prefix_size = 0; - char compressed_cache_key_prefix[kMaxCacheKeyPrefixSize]; - size_t compressed_cache_key_prefix_size = 0; - - // Footer contains the fixed table information - Footer footer; - // index_reader and filter will be populated and used only when - // options.block_cache is nullptr; otherwise we will get the index block via - // the block cache. - unique_ptr index_reader; - unique_ptr filter; - - std::shared_ptr table_properties; - BlockBasedTableOptions::IndexType index_type; - bool hash_index_allow_collision; - // TODO(kailiu) It is very ugly to use internal key in table, since table - // module should not be relying on db module. However to make things easier - // and compatible with existing code, we introduce a wrapper that allows - // block to extract prefix without knowing if a key is internal or not. - unique_ptr internal_prefix_transform; -}; - -BlockBasedTable::~BlockBasedTable() { - delete rep_; -} - -// CachableEntry represents the entries that *may* be fetched from block cache. -// field `value` is the item we want to get. -// field `cache_handle` is the cache handle to the block cache. If the value -// was not read from cache, `cache_handle` will be nullptr. -template -struct BlockBasedTable::CachableEntry { - CachableEntry(TValue* value, Cache::Handle* cache_handle) - : value(value) - , cache_handle(cache_handle) { - } - CachableEntry(): CachableEntry(nullptr, nullptr) { } - void Release(Cache* cache) { - if (cache_handle) { - cache->Release(cache_handle); - value = nullptr; - cache_handle = nullptr; - } - } - - TValue* value = nullptr; - // if the entry is from the cache, cache_handle will be populated. - Cache::Handle* cache_handle = nullptr; -}; - // Helper function to setup the cache key's prefix for the Table. -void BlockBasedTable::SetupCacheKeyPrefix(Rep* rep) { +void BlockBasedTable::SetupCacheKeyPrefix(Rep* rep, uint64_t file_size) { assert(kMaxCacheKeyPrefixSize >= 10); rep->cache_key_prefix_size = 0; rep->compressed_cache_key_prefix_size = 0; if (rep->table_options.block_cache != nullptr) { - GenerateCachePrefix(rep->table_options.block_cache.get(), rep->file.get(), - &rep->cache_key_prefix[0], - &rep->cache_key_prefix_size); + GenerateCachePrefix(rep->table_options.block_cache.get(), rep->file->file(), + &rep->cache_key_prefix[0], &rep->cache_key_prefix_size); + // Create dummy offset of index reader which is beyond the file size. + rep->dummy_index_reader_offset = + file_size + rep->table_options.block_cache->NewId(); + } + if (rep->table_options.persistent_cache != nullptr) { + GenerateCachePrefix(/*cache=*/nullptr, rep->file->file(), + &rep->persistent_cache_key_prefix[0], + &rep->persistent_cache_key_prefix_size); } if (rep->table_options.block_cache_compressed != nullptr) { GenerateCachePrefix(rep->table_options.block_cache_compressed.get(), - rep->file.get(), &rep->compressed_cache_key_prefix[0], + rep->file->file(), &rep->compressed_cache_key_prefix[0], &rep->compressed_cache_key_prefix_size); } } @@ -426,7 +496,7 @@ void BlockBasedTable::GenerateCachePrefix(Cache* cc, // If the prefix wasn't generated or was too long, // create one from the cache. - if (*size == 0) { + if (cc && *size == 0) { char* end = EncodeVarint64(buffer, cc->NewId()); *size = static_cast(end - buffer); } @@ -446,91 +516,354 @@ void BlockBasedTable::GenerateCachePrefix(Cache* cc, } } -Status BlockBasedTable::Open(const Options& options, const EnvOptions& soptions, +namespace { +// Return True if table_properties has `user_prop_name` has a `true` value +// or it doesn't contain this property (for backward compatible). +bool IsFeatureSupported(const TableProperties& table_properties, + const std::string& user_prop_name, Logger* info_log) { + auto& props = table_properties.user_collected_properties; + auto pos = props.find(user_prop_name); + // Older version doesn't have this value set. Skip this check. + if (pos != props.end()) { + if (pos->second == kPropFalse) { + return false; + } else if (pos->second != kPropTrue) { + ROCKS_LOG_WARN(info_log, "Property %s has invalidate value %s", + user_prop_name.c_str(), pos->second.c_str()); + } + } + return true; +} + +SequenceNumber GetGlobalSequenceNumber(const TableProperties& table_properties, + Logger* info_log) { + auto& props = table_properties.user_collected_properties; + + auto version_pos = props.find(ExternalSstFilePropertyNames::kVersion); + auto seqno_pos = props.find(ExternalSstFilePropertyNames::kGlobalSeqno); + + if (version_pos == props.end()) { + if (seqno_pos != props.end()) { + // This is not an external sst file, global_seqno is not supported. + assert(false); + ROCKS_LOG_ERROR( + info_log, + "A non-external sst file have global seqno property with value %s", + seqno_pos->second.c_str()); + } + return kDisableGlobalSequenceNumber; + } + + uint32_t version = DecodeFixed32(version_pos->second.c_str()); + if (version < 2) { + if (seqno_pos != props.end() || version != 1) { + // This is a v1 external sst file, global_seqno is not supported. + assert(false); + ROCKS_LOG_ERROR( + info_log, + "An external sst file with version %u have global seqno property " + "with value %s", + version, seqno_pos->second.c_str()); + } + return kDisableGlobalSequenceNumber; + } + + SequenceNumber global_seqno = DecodeFixed64(seqno_pos->second.c_str()); + + if (global_seqno > kMaxSequenceNumber) { + assert(false); + ROCKS_LOG_ERROR( + info_log, + "An external sst file with version %u have global seqno property " + "with value %llu, which is greater than kMaxSequenceNumber", + version, global_seqno); + } + + return global_seqno; +} +} // namespace + +Slice BlockBasedTable::GetCacheKey(const char* cache_key_prefix, + size_t cache_key_prefix_size, + const BlockHandle& handle, char* cache_key) { + assert(cache_key != nullptr); + assert(cache_key_prefix_size != 0); + assert(cache_key_prefix_size <= kMaxCacheKeyPrefixSize); + memcpy(cache_key, cache_key_prefix, cache_key_prefix_size); + char* end = + EncodeVarint64(cache_key + cache_key_prefix_size, handle.offset()); + return Slice(cache_key, static_cast(end - cache_key)); +} + +Status BlockBasedTable::Open(const ImmutableCFOptions& ioptions, + const EnvOptions& env_options, const BlockBasedTableOptions& table_options, const InternalKeyComparator& internal_comparator, - unique_ptr&& file, + unique_ptr&& file, uint64_t file_size, - unique_ptr* table_reader) { + unique_ptr* table_reader, + const bool prefetch_index_and_filter_in_cache, + const bool skip_filters, const int level) { table_reader->reset(); - Footer footer(kBlockBasedTableMagicNumber); - auto s = ReadFooterFromFile(file.get(), file_size, &footer); - if (!s.ok()) return s; + Footer footer; + + std::unique_ptr prefetch_buffer; + + // Before read footer, readahead backwards to prefetch data + const size_t kTailPrefetchSize = 512 * 1024; + size_t prefetch_off; + size_t prefetch_len; + if (file_size < kTailPrefetchSize) { + prefetch_off = 0; + prefetch_len = file_size; + } else { + prefetch_off = file_size - kTailPrefetchSize; + prefetch_len = kTailPrefetchSize; + } + Status s; + // TODO should not have this special logic in the future. + if (!file->use_direct_io()) { + s = file->Prefetch(prefetch_off, prefetch_len); + } else { + prefetch_buffer.reset(new FilePrefetchBuffer()); + s = prefetch_buffer->Prefetch(file.get(), prefetch_off, prefetch_len); + } + s = ReadFooterFromFile(file.get(), prefetch_buffer.get(), file_size, &footer, + kBlockBasedTableMagicNumber); + if (!s.ok()) { + return s; + } + if (!BlockBasedTableSupportedVersion(footer.version())) { + return Status::Corruption( + "Unknown Footer version. Maybe this file was created with newer " + "version of RocksDB?"); + } - // We've successfully read the footer and the index block: we're - // ready to serve requests. - Rep* rep = new BlockBasedTable::Rep( - soptions, table_options, internal_comparator); - rep->options = options; + // We've successfully read the footer. We are ready to serve requests. + // Better not mutate rep_ after the creation. eg. internal_prefix_transform + // raw pointer will be used to create HashIndexReader, whose reset may + // access a dangling pointer. + Rep* rep = new BlockBasedTable::Rep(ioptions, env_options, table_options, + internal_comparator, skip_filters); rep->file = std::move(file); rep->footer = footer; rep->index_type = table_options.index_type; rep->hash_index_allow_collision = table_options.hash_index_allow_collision; - SetupCacheKeyPrefix(rep); + // We need to wrap data with internal_prefix_transform to make sure it can + // handle prefix correctly. + rep->internal_prefix_transform.reset( + new InternalKeySliceTransform(rep->ioptions.prefix_extractor)); + SetupCacheKeyPrefix(rep, file_size); unique_ptr new_table(new BlockBasedTable(rep)); + // page cache options + rep->persistent_cache_options = + PersistentCacheOptions(rep->table_options.persistent_cache, + std::string(rep->persistent_cache_key_prefix, + rep->persistent_cache_key_prefix_size), + rep->ioptions.statistics); + // Read meta index std::unique_ptr meta; - std::unique_ptr meta_iter; - s = ReadMetaBlock(rep, &meta, &meta_iter); + std::unique_ptr meta_iter; + s = ReadMetaBlock(rep, prefetch_buffer.get(), &meta, &meta_iter); + if (!s.ok()) { + return s; + } + + // Find filter handle and filter type + if (rep->filter_policy) { + for (auto filter_type : + {Rep::FilterType::kFullFilter, Rep::FilterType::kPartitionedFilter, + Rep::FilterType::kBlockFilter}) { + std::string prefix; + switch (filter_type) { + case Rep::FilterType::kFullFilter: + prefix = kFullFilterBlockPrefix; + break; + case Rep::FilterType::kPartitionedFilter: + prefix = kPartitionedFilterBlockPrefix; + break; + case Rep::FilterType::kBlockFilter: + prefix = kFilterBlockPrefix; + break; + default: + assert(0); + } + std::string filter_block_key = prefix; + filter_block_key.append(rep->filter_policy->Name()); + if (FindMetaBlock(meta_iter.get(), filter_block_key, &rep->filter_handle) + .ok()) { + rep->filter_type = filter_type; + break; + } + } + } // Read the properties bool found_properties_block = true; s = SeekToPropertiesBlock(meta_iter.get(), &found_properties_block); - if (found_properties_block) { + if (!s.ok()) { + ROCKS_LOG_WARN(rep->ioptions.info_log, + "Error when seeking to properties block from file: %s", + s.ToString().c_str()); + } else if (found_properties_block) { s = meta_iter->status(); TableProperties* table_properties = nullptr; if (s.ok()) { - s = ReadProperties(meta_iter->value(), rep->file.get(), rep->footer, - rep->options.env, rep->options.info_log.get(), + s = ReadProperties(meta_iter->value(), rep->file.get(), + prefetch_buffer.get(), rep->footer, rep->ioptions, &table_properties); } if (!s.ok()) { - auto err_msg = - "[Warning] Encountered error while reading data from properties " - "block " + s.ToString(); - Log(rep->options.info_log, "%s", err_msg.c_str()); + ROCKS_LOG_WARN(rep->ioptions.info_log, + "Encountered error while reading data from properties " + "block %s", + s.ToString().c_str()); } else { rep->table_properties.reset(table_properties); } } else { - Log(WARN_LEVEL, rep->options.info_log, - "Cannot find Properties block from file."); + ROCKS_LOG_ERROR(rep->ioptions.info_log, + "Cannot find Properties block from file."); + } + + // Read the compression dictionary meta block + bool found_compression_dict; + s = SeekToCompressionDictBlock(meta_iter.get(), &found_compression_dict); + if (!s.ok()) { + ROCKS_LOG_WARN( + rep->ioptions.info_log, + "Error when seeking to compression dictionary block from file: %s", + s.ToString().c_str()); + } else if (found_compression_dict) { + // TODO(andrewkr): Add to block cache if cache_index_and_filter_blocks is + // true. + unique_ptr compression_dict_block{new BlockContents()}; + // TODO(andrewkr): ReadMetaBlock repeats SeekToCompressionDictBlock(). + // maybe decode a handle from meta_iter + // and do ReadBlockContents(handle) instead + s = rocksdb::ReadMetaBlock(rep->file.get(), prefetch_buffer.get(), + file_size, kBlockBasedTableMagicNumber, + rep->ioptions, rocksdb::kCompressionDictBlock, + compression_dict_block.get()); + if (!s.ok()) { + ROCKS_LOG_WARN( + rep->ioptions.info_log, + "Encountered error while reading data from compression dictionary " + "block %s", + s.ToString().c_str()); + } else { + rep->compression_dict_block = std::move(compression_dict_block); + } + } + + // Read the range del meta block + bool found_range_del_block; + s = SeekToRangeDelBlock(meta_iter.get(), &found_range_del_block, + &rep->range_del_handle); + if (!s.ok()) { + ROCKS_LOG_WARN( + rep->ioptions.info_log, + "Error when seeking to range delete tombstones block from file: %s", + s.ToString().c_str()); + } else { + if (found_range_del_block && !rep->range_del_handle.IsNull()) { + ReadOptions read_options; + s = MaybeLoadDataBlockToCache( + prefetch_buffer.get(), rep, read_options, rep->range_del_handle, + Slice() /* compression_dict */, &rep->range_del_entry); + if (!s.ok()) { + ROCKS_LOG_WARN( + rep->ioptions.info_log, + "Encountered error while reading data from range del block %s", + s.ToString().c_str()); + } + } } - // Will use block cache for index/filter blocks access? - if (table_options.block_cache && - table_options.cache_index_and_filter_blocks) { - // Hack: Call NewIndexIterator() to implicitly add index to the block_cache - unique_ptr iter(new_table->NewIndexIterator(ReadOptions())); - s = iter->status(); + // Determine whether whole key filtering is supported. + if (rep->table_properties) { + rep->whole_key_filtering &= + IsFeatureSupported(*(rep->table_properties), + BlockBasedTablePropertyNames::kWholeKeyFiltering, + rep->ioptions.info_log); + rep->prefix_filtering &= IsFeatureSupported( + *(rep->table_properties), + BlockBasedTablePropertyNames::kPrefixFiltering, rep->ioptions.info_log); + + rep->global_seqno = GetGlobalSequenceNumber(*(rep->table_properties), + rep->ioptions.info_log); + } - if (s.ok()) { - // Hack: Call GetFilter() to implicitly add filter to the block_cache - auto filter_entry = new_table->GetFilter(); - filter_entry.Release(table_options.block_cache.get()); + const bool pin = + rep->table_options.pin_l0_filter_and_index_blocks_in_cache && level == 0; + // pre-fetching of blocks is turned on + // Will use block cache for index/filter blocks access + // Always prefetch index and filter for level 0 + if (table_options.cache_index_and_filter_blocks) { + if (prefetch_index_and_filter_in_cache || level == 0) { + assert(table_options.block_cache != nullptr); + // Hack: Call NewIndexIterator() to implicitly add index to the + // block_cache + + CachableEntry index_entry; + unique_ptr iter( + new_table->NewIndexIterator(ReadOptions(), nullptr, &index_entry)); + index_entry.value->CacheDependencies(pin); + if (pin) { + rep->index_entry = std::move(index_entry); + } else { + index_entry.Release(table_options.block_cache.get()); + } + s = iter->status(); + + if (s.ok()) { + // Hack: Call GetFilter() to implicitly add filter to the block_cache + auto filter_entry = new_table->GetFilter(); + if (filter_entry.value != nullptr) { + filter_entry.value->CacheDependencies(pin); + } + // if pin_l0_filter_and_index_blocks_in_cache is true, and this is + // a level0 file, then save it in rep_->filter_entry; it will be + // released in the destructor only, hence it will be pinned in the + // cache while this reader is alive + if (pin) { + rep->filter_entry = filter_entry; + } else { + filter_entry.Release(table_options.block_cache.get()); + } + } } } else { // If we don't use block cache for index/filter blocks access, we'll // pre-load these blocks, which will kept in member variables in Rep // and with a same life-time as this table object. IndexReader* index_reader = nullptr; - // TODO: we never really verify check sum for index block - s = new_table->CreateIndexReader(&index_reader, meta_iter.get()); - + s = new_table->CreateIndexReader(prefetch_buffer.get(), &index_reader, + meta_iter.get(), level); if (s.ok()) { rep->index_reader.reset(index_reader); + // The partitions of partitioned index are always stored in cache. They + // are hence follow the configuration for pin and prefetch regardless of + // the value of cache_index_and_filter_blocks + if (prefetch_index_and_filter_in_cache || level == 0) { + rep->index_reader->CacheDependencies(pin); + } // Set filter block if (rep->filter_policy) { - std::string key = kFilterBlockPrefix; - key.append(rep->filter_policy->Name()); - BlockHandle handle; - if (FindMetaBlock(meta_iter.get(), key, &handle).ok()) { - rep->filter.reset(ReadFilter(handle, rep)); + const bool is_a_filter_partition = true; + auto filter = new_table->ReadFilter( + prefetch_buffer.get(), rep->filter_handle, !is_a_filter_partition); + rep->filter.reset(filter); + // Refer to the comment above about paritioned indexes always being + // cached + if (filter && (prefetch_index_and_filter_in_cache || level == 0)) { + filter->CacheDependencies(pin); } } } else { @@ -546,22 +879,21 @@ Status BlockBasedTable::Open(const Options& options, const EnvOptions& soptions, } void BlockBasedTable::SetupForCompaction() { - switch (rep_->options.access_hint_on_compaction_start) { + switch (rep_->ioptions.access_hint_on_compaction_start) { case Options::NONE: break; case Options::NORMAL: - rep_->file->Hint(RandomAccessFile::NORMAL); + rep_->file->file()->Hint(RandomAccessFile::NORMAL); break; case Options::SEQUENTIAL: - rep_->file->Hint(RandomAccessFile::SEQUENTIAL); + rep_->file->file()->Hint(RandomAccessFile::SEQUENTIAL); break; case Options::WILLNEED: - rep_->file->Hint(RandomAccessFile::WILLNEED); + rep_->file->file()->Hint(RandomAccessFile::WILLNEED); break; default: assert(false); } - compaction_optimized_ = true; } std::shared_ptr BlockBasedTable::GetTableProperties() @@ -582,53 +914,52 @@ size_t BlockBasedTable::ApproximateMemoryUsage() const { // Load the meta-block from the file. On success, return the loaded meta block // and its iterator. -Status BlockBasedTable::ReadMetaBlock( - Rep* rep, - std::unique_ptr* meta_block, - std::unique_ptr* iter) { +Status BlockBasedTable::ReadMetaBlock(Rep* rep, + FilePrefetchBuffer* prefetch_buffer, + std::unique_ptr* meta_block, + std::unique_ptr* iter) { // TODO(sanjay): Skip this if footer.metaindex_handle() size indicates // it is an empty block. - // TODO: we never really verify check sum for meta index block - Block* meta = nullptr; + std::unique_ptr meta; Status s = ReadBlockFromFile( - rep->file.get(), - rep->footer, - ReadOptions(), - rep->footer.metaindex_handle(), - &meta, - rep->options.env); + rep->file.get(), prefetch_buffer, rep->footer, ReadOptions(), + rep->footer.metaindex_handle(), &meta, rep->ioptions, + true /* decompress */, Slice() /*compression dict*/, + rep->persistent_cache_options, kDisableGlobalSequenceNumber, + 0 /* read_amp_bytes_per_bit */); - if (!s.ok()) { - auto err_msg = - "[Warning] Encountered error while reading data from properties" - "block " + s.ToString(); - Log(rep->options.info_log, "%s", err_msg.c_str()); - } if (!s.ok()) { - delete meta; + ROCKS_LOG_ERROR(rep->ioptions.info_log, + "Encountered error while reading data from properties" + " block %s", + s.ToString().c_str()); return s; } - meta_block->reset(meta); + *meta_block = std::move(meta); // meta block uses bytewise comparator. - iter->reset(meta->NewIterator(BytewiseComparator())); + iter->reset(meta_block->get()->NewIterator(BytewiseComparator())); return Status::OK(); } Status BlockBasedTable::GetDataBlockFromCache( const Slice& block_cache_key, const Slice& compressed_block_cache_key, - Cache* block_cache, Cache* block_cache_compressed, Statistics* statistics, - const ReadOptions& read_options, - BlockBasedTable::CachableEntry* block) { + Cache* block_cache, Cache* block_cache_compressed, + const ImmutableCFOptions& ioptions, const ReadOptions& read_options, + BlockBasedTable::CachableEntry* block, uint32_t format_version, + const Slice& compression_dict, size_t read_amp_bytes_per_bit, + bool is_index) { Status s; Block* compressed_block = nullptr; Cache::Handle* block_cache_compressed_handle = nullptr; + Statistics* statistics = ioptions.statistics; // Lookup uncompressed cache first if (block_cache != nullptr) { - block->cache_handle = - GetEntryFromCache(block_cache, block_cache_key, BLOCK_CACHE_DATA_MISS, - BLOCK_CACHE_DATA_HIT, statistics); + block->cache_handle = GetEntryFromCache( + block_cache, block_cache_key, + is_index ? BLOCK_CACHE_INDEX_MISS : BLOCK_CACHE_DATA_MISS, + is_index ? BLOCK_CACHE_INDEX_HIT : BLOCK_CACHE_DATA_HIT, statistics); if (block->cache_handle != nullptr) { block->value = reinterpret_cast(block_cache->Value(block->cache_handle)); @@ -662,19 +993,42 @@ Status BlockBasedTable::GetDataBlockFromCache( // Retrieve the uncompressed contents into a new buffer BlockContents contents; s = UncompressBlockContents(compressed_block->data(), - compressed_block->size(), &contents); + compressed_block->size(), &contents, + format_version, compression_dict, + ioptions); // Insert uncompressed block into block cache if (s.ok()) { - block->value = new Block(contents); // uncompressed block + block->value = + new Block(std::move(contents), compressed_block->global_seqno(), + read_amp_bytes_per_bit, + statistics); // uncompressed block assert(block->value->compression_type() == kNoCompression); if (block_cache != nullptr && block->value->cachable() && read_options.fill_cache) { - block->cache_handle = - block_cache->Insert(block_cache_key, block->value, - block->value->size(), &DeleteCachedEntry); - assert(reinterpret_cast( - block_cache->Value(block->cache_handle)) == block->value); + s = block_cache->Insert( + block_cache_key, block->value, block->value->usable_size(), + &DeleteCachedEntry, &(block->cache_handle)); + block_cache->TEST_mark_as_data_block(block_cache_key, + block->value->usable_size()); + if (s.ok()) { + RecordTick(statistics, BLOCK_CACHE_ADD); + if (is_index) { + RecordTick(statistics, BLOCK_CACHE_INDEX_ADD); + RecordTick(statistics, BLOCK_CACHE_INDEX_BYTES_INSERT, + block->value->usable_size()); + } else { + RecordTick(statistics, BLOCK_CACHE_DATA_ADD); + RecordTick(statistics, BLOCK_CACHE_DATA_BYTES_INSERT, + block->value->usable_size()); + } + RecordTick(statistics, BLOCK_CACHE_BYTES_WRITE, + block->value->usable_size()); + } else { + RecordTick(statistics, BLOCK_CACHE_ADD_FAILURES); + delete block->value; + block->value = nullptr; + } } } @@ -686,17 +1040,20 @@ Status BlockBasedTable::GetDataBlockFromCache( Status BlockBasedTable::PutDataBlockToCache( const Slice& block_cache_key, const Slice& compressed_block_cache_key, Cache* block_cache, Cache* block_cache_compressed, - const ReadOptions& read_options, Statistics* statistics, - CachableEntry* block, Block* raw_block) { + const ReadOptions& read_options, const ImmutableCFOptions& ioptions, + CachableEntry* block, Block* raw_block, uint32_t format_version, + const Slice& compression_dict, size_t read_amp_bytes_per_bit, bool is_index, + Cache::Priority priority) { assert(raw_block->compression_type() == kNoCompression || block_cache_compressed != nullptr); Status s; // Retrieve the uncompressed contents into a new buffer BlockContents contents; + Statistics* statistics = ioptions.statistics; if (raw_block->compression_type() != kNoCompression) { - s = UncompressBlockContents(raw_block->data(), raw_block->size(), - &contents); + s = UncompressBlockContents(raw_block->data(), raw_block->size(), &contents, + format_version, compression_dict, ioptions); } if (!s.ok()) { delete raw_block; @@ -704,7 +1061,9 @@ Status BlockBasedTable::PutDataBlockToCache( } if (raw_block->compression_type() != kNoCompression) { - block->value = new Block(contents); // uncompressed block + block->value = new Block(std::move(contents), raw_block->global_seqno(), + read_amp_bytes_per_bit, + statistics); // uncompressed block } else { block->value = raw_block; raw_block = nullptr; @@ -714,54 +1073,129 @@ Status BlockBasedTable::PutDataBlockToCache( // Release the hold on the compressed cache entry immediately. if (block_cache_compressed != nullptr && raw_block != nullptr && raw_block->cachable()) { - auto cache_handle = block_cache_compressed->Insert( - compressed_block_cache_key, raw_block, raw_block->size(), - &DeleteCachedEntry); - block_cache_compressed->Release(cache_handle); - RecordTick(statistics, BLOCK_CACHE_COMPRESSED_MISS); - // Avoid the following code to delete this cached block. - raw_block = nullptr; + s = block_cache_compressed->Insert(compressed_block_cache_key, raw_block, + raw_block->usable_size(), + &DeleteCachedEntry); + if (s.ok()) { + // Avoid the following code to delete this cached block. + raw_block = nullptr; + RecordTick(statistics, BLOCK_CACHE_COMPRESSED_ADD); + } else { + RecordTick(statistics, BLOCK_CACHE_COMPRESSED_ADD_FAILURES); + } } delete raw_block; // insert into uncompressed block cache assert((block->value->compression_type() == kNoCompression)); if (block_cache != nullptr && block->value->cachable()) { - block->cache_handle = - block_cache->Insert(block_cache_key, block->value, block->value->size(), - &DeleteCachedEntry); - RecordTick(statistics, BLOCK_CACHE_ADD); - assert(reinterpret_cast(block_cache->Value(block->cache_handle)) == - block->value); + s = block_cache->Insert( + block_cache_key, block->value, block->value->usable_size(), + &DeleteCachedEntry, &(block->cache_handle), priority); + block_cache->TEST_mark_as_data_block(block_cache_key, + block->value->usable_size()); + if (s.ok()) { + assert(block->cache_handle != nullptr); + RecordTick(statistics, BLOCK_CACHE_ADD); + if (is_index) { + RecordTick(statistics, BLOCK_CACHE_INDEX_ADD); + RecordTick(statistics, BLOCK_CACHE_INDEX_BYTES_INSERT, + block->value->usable_size()); + } else { + RecordTick(statistics, BLOCK_CACHE_DATA_ADD); + RecordTick(statistics, BLOCK_CACHE_DATA_BYTES_INSERT, + block->value->usable_size()); + } + RecordTick(statistics, BLOCK_CACHE_BYTES_WRITE, + block->value->usable_size()); + assert(reinterpret_cast( + block_cache->Value(block->cache_handle)) == block->value); + } else { + RecordTick(statistics, BLOCK_CACHE_ADD_FAILURES); + delete block->value; + block->value = nullptr; + } } return s; } -FilterBlockReader* BlockBasedTable::ReadFilter(const BlockHandle& filter_handle, - BlockBasedTable::Rep* rep, - size_t* filter_size) { +FilterBlockReader* BlockBasedTable::ReadFilter( + FilePrefetchBuffer* prefetch_buffer, const BlockHandle& filter_handle, + const bool is_a_filter_partition) const { + auto& rep = rep_; // TODO: We might want to unify with ReadBlockFromFile() if we start // requiring checksum verification in Table::Open. - ReadOptions opt; + if (rep->filter_type == Rep::FilterType::kNoFilter) { + return nullptr; + } BlockContents block; - if (!ReadBlockContents(rep->file.get(), rep->footer, opt, filter_handle, - &block, rep->options.env, false).ok()) { + if (!ReadBlockContents(rep->file.get(), prefetch_buffer, rep->footer, + ReadOptions(), filter_handle, &block, rep->ioptions, + false /* decompress */, Slice() /*compression dict*/, + rep->persistent_cache_options) + .ok()) { + // Error reading the block return nullptr; } - if (filter_size) { - *filter_size = block.data.size(); + assert(rep->filter_policy); + + auto filter_type = rep->filter_type; + if (rep->filter_type == Rep::FilterType::kPartitionedFilter && + is_a_filter_partition) { + filter_type = Rep::FilterType::kFullFilter; + } + + switch (filter_type) { + case Rep::FilterType::kPartitionedFilter: { + return new PartitionedFilterBlockReader( + rep->prefix_filtering ? rep->ioptions.prefix_extractor : nullptr, + rep->whole_key_filtering, std::move(block), nullptr, + rep->ioptions.statistics, rep->internal_comparator, this); + } + + case Rep::FilterType::kBlockFilter: + return new BlockBasedFilterBlockReader( + rep->prefix_filtering ? rep->ioptions.prefix_extractor : nullptr, + rep->table_options, rep->whole_key_filtering, std::move(block), + rep->ioptions.statistics); + + case Rep::FilterType::kFullFilter: { + auto filter_bits_reader = + rep->filter_policy->GetFilterBitsReader(block.data); + assert(filter_bits_reader != nullptr); + return new FullFilterBlockReader( + rep->prefix_filtering ? rep->ioptions.prefix_extractor : nullptr, + rep->whole_key_filtering, std::move(block), filter_bits_reader, + rep->ioptions.statistics); + } + + default: + // filter_type is either kNoFilter (exited the function at the first if), + // or it must be covered in this switch block + assert(false); + return nullptr; } +} - return new FilterBlockReader( - rep->options, rep->table_options, block.data, block.heap_allocated); +BlockBasedTable::CachableEntry BlockBasedTable::GetFilter( + FilePrefetchBuffer* prefetch_buffer, bool no_io) const { + const BlockHandle& filter_blk_handle = rep_->filter_handle; + const bool is_a_filter_partition = true; + return GetFilter(prefetch_buffer, filter_blk_handle, !is_a_filter_partition, + no_io); } BlockBasedTable::CachableEntry BlockBasedTable::GetFilter( - bool no_io) const { - // filter pre-populated - if (rep_->filter != nullptr) { + FilePrefetchBuffer* prefetch_buffer, const BlockHandle& filter_blk_handle, + const bool is_a_filter_partition, bool no_io) const { + // If cache_index_and_filter_blocks is false, filter should be pre-populated. + // We will return rep_->filter anyway. rep_->filter can be nullptr if filter + // read fails at Open() time. We don't want to reload again since it will + // most probably fail again. + if (!is_a_filter_partition && + !rep_->table_options.cache_index_and_filter_blocks) { return {rep_->filter.get(), nullptr /* cache handle */}; } @@ -771,45 +1205,48 @@ BlockBasedTable::CachableEntry BlockBasedTable::GetFilter( return {nullptr /* filter */, nullptr /* cache handle */}; } + if (!is_a_filter_partition && rep_->filter_entry.IsSet()) { + return rep_->filter_entry; + } + + PERF_TIMER_GUARD(read_filter_block_nanos); + // Fetching from the cache char cache_key[kMaxCacheKeyPrefixSize + kMaxVarint64Length]; - auto key = GetCacheKey( - rep_->cache_key_prefix, - rep_->cache_key_prefix_size, - rep_->footer.metaindex_handle(), - cache_key - ); - - Statistics* statistics = rep_->options.statistics.get(); + auto key = GetCacheKey(rep_->cache_key_prefix, rep_->cache_key_prefix_size, + filter_blk_handle, cache_key); + + Statistics* statistics = rep_->ioptions.statistics; auto cache_handle = GetEntryFromCache(block_cache, key, BLOCK_CACHE_FILTER_MISS, BLOCK_CACHE_FILTER_HIT, statistics); FilterBlockReader* filter = nullptr; if (cache_handle != nullptr) { - filter = reinterpret_cast( - block_cache->Value(cache_handle)); + filter = reinterpret_cast( + block_cache->Value(cache_handle)); } else if (no_io) { // Do not invoke any io. return CachableEntry(); } else { - size_t filter_size = 0; - std::unique_ptr meta; - std::unique_ptr iter; - auto s = ReadMetaBlock(rep_, &meta, &iter); - - if (s.ok()) { - std::string filter_block_key = kFilterBlockPrefix; - filter_block_key.append(rep_->filter_policy->Name()); - BlockHandle handle; - if (FindMetaBlock(iter.get(), filter_block_key, &handle).ok()) { - filter = ReadFilter(handle, rep_, &filter_size); - assert(filter); - assert(filter_size > 0); - - cache_handle = block_cache->Insert( - key, filter, filter_size, &DeleteCachedEntry); + filter = + ReadFilter(prefetch_buffer, filter_blk_handle, is_a_filter_partition); + if (filter != nullptr) { + assert(filter->size() > 0); + Status s = block_cache->Insert( + key, filter, filter->size(), &DeleteCachedFilterEntry, &cache_handle, + rep_->table_options.cache_index_and_filter_blocks_with_high_priority + ? Cache::Priority::HIGH + : Cache::Priority::LOW); + if (s.ok()) { RecordTick(statistics, BLOCK_CACHE_ADD); + RecordTick(statistics, BLOCK_CACHE_FILTER_ADD); + RecordTick(statistics, BLOCK_CACHE_FILTER_BYTES_INSERT, filter->size()); + RecordTick(statistics, BLOCK_CACHE_BYTES_WRITE, filter->size()); + } else { + RecordTick(statistics, BLOCK_CACHE_ADD_FAILURES); + delete filter; + return CachableEntry(); } } } @@ -817,20 +1254,29 @@ BlockBasedTable::CachableEntry BlockBasedTable::GetFilter( return { filter, cache_handle }; } -Iterator* BlockBasedTable::NewIndexIterator(const ReadOptions& read_options, - BlockIter* input_iter) { +InternalIterator* BlockBasedTable::NewIndexIterator( + const ReadOptions& read_options, BlockIter* input_iter, + CachableEntry* index_entry) { // index reader has already been pre-populated. if (rep_->index_reader) { return rep_->index_reader->NewIterator( input_iter, read_options.total_order_seek); } + // we have a pinned index block + if (rep_->index_entry.IsSet()) { + return rep_->index_entry.value->NewIterator(input_iter, + read_options.total_order_seek); + } - bool no_io = read_options.read_tier == kBlockCacheTier; + PERF_TIMER_GUARD(read_index_block_nanos); + + const bool no_io = read_options.read_tier == kBlockCacheTier; Cache* block_cache = rep_->table_options.block_cache.get(); char cache_key[kMaxCacheKeyPrefixSize + kMaxVarint64Length]; - auto key = GetCacheKey(rep_->cache_key_prefix, rep_->cache_key_prefix_size, - rep_->footer.index_handle(), cache_key); - Statistics* statistics = rep_->options.statistics.get(); + auto key = + GetCacheKeyFromOffset(rep_->cache_key_prefix, rep_->cache_key_prefix_size, + rep_->dummy_index_reader_offset, cache_key); + Statistics* statistics = rep_->ioptions.statistics; auto cache_handle = GetEntryFromCache(block_cache, key, BLOCK_CACHE_INDEX_MISS, BLOCK_CACHE_INDEX_HIT, statistics); @@ -840,7 +1286,7 @@ Iterator* BlockBasedTable::NewIndexIterator(const ReadOptions& read_options, input_iter->SetStatus(Status::Incomplete("no blocking io")); return input_iter; } else { - return NewErrorIterator(Status::Incomplete("no blocking io")); + return NewErrorInternalIterator(Status::Incomplete("no blocking io")); } } @@ -851,157 +1297,260 @@ Iterator* BlockBasedTable::NewIndexIterator(const ReadOptions& read_options, } else { // Create index reader and put it in the cache. Status s; - s = CreateIndexReader(&index_reader); + TEST_SYNC_POINT("BlockBasedTable::NewIndexIterator::thread2:2"); + s = CreateIndexReader(nullptr /* prefetch_buffer */, &index_reader); + TEST_SYNC_POINT("BlockBasedTable::NewIndexIterator::thread1:1"); + TEST_SYNC_POINT("BlockBasedTable::NewIndexIterator::thread2:3"); + TEST_SYNC_POINT("BlockBasedTable::NewIndexIterator::thread1:4"); + if (s.ok()) { + assert(index_reader != nullptr); + s = block_cache->Insert( + key, index_reader, index_reader->usable_size(), + &DeleteCachedIndexEntry, &cache_handle, + rep_->table_options.cache_index_and_filter_blocks_with_high_priority + ? Cache::Priority::HIGH + : Cache::Priority::LOW); + } - if (!s.ok()) { + if (s.ok()) { + size_t usable_size = index_reader->usable_size(); + RecordTick(statistics, BLOCK_CACHE_ADD); + RecordTick(statistics, BLOCK_CACHE_INDEX_ADD); + RecordTick(statistics, BLOCK_CACHE_INDEX_BYTES_INSERT, usable_size); + RecordTick(statistics, BLOCK_CACHE_BYTES_WRITE, usable_size); + } else { + if (index_reader != nullptr) { + delete index_reader; + } + RecordTick(statistics, BLOCK_CACHE_ADD_FAILURES); // make sure if something goes wrong, index_reader shall remain intact. - assert(index_reader == nullptr); if (input_iter != nullptr) { input_iter->SetStatus(s); return input_iter; } else { - return NewErrorIterator(s); + return NewErrorInternalIterator(s); } } - cache_handle = block_cache->Insert(key, index_reader, index_reader->size(), - &DeleteCachedEntry); - RecordTick(statistics, BLOCK_CACHE_ADD); } assert(cache_handle); auto* iter = index_reader->NewIterator( input_iter, read_options.total_order_seek); - iter->RegisterCleanup(&ReleaseCachedEntry, block_cache, cache_handle); + + // the caller would like to take ownership of the index block + // don't call RegisterCleanup() in this case, the caller will take care of it + if (index_entry != nullptr) { + *index_entry = {index_reader, cache_handle}; + } else { + iter->RegisterCleanup(&ReleaseCachedEntry, block_cache, cache_handle); + } + return iter; } +InternalIterator* BlockBasedTable::NewDataBlockIterator( + Rep* rep, const ReadOptions& ro, const Slice& index_value, + BlockIter* input_iter, bool is_index) { + BlockHandle handle; + Slice input = index_value; + // We intentionally allow extra stuff in index_value so that we + // can add more features in the future. + Status s = handle.DecodeFrom(&input); + return NewDataBlockIterator(rep, ro, handle, input_iter, is_index, s); +} + // Convert an index iterator value (i.e., an encoded BlockHandle) // into an iterator over the contents of the corresponding block. // If input_iter is null, new a iterator // If input_iter is not null, update this iter and return it -Iterator* BlockBasedTable::NewDataBlockIterator(Rep* rep, - const ReadOptions& ro, const Slice& index_value, - BlockIter* input_iter) { +InternalIterator* BlockBasedTable::NewDataBlockIterator( + Rep* rep, const ReadOptions& ro, const BlockHandle& handle, + BlockIter* input_iter, bool is_index, Status s) { + PERF_TIMER_GUARD(new_table_block_iter_nanos); + const bool no_io = (ro.read_tier == kBlockCacheTier); Cache* block_cache = rep->table_options.block_cache.get(); - Cache* block_cache_compressed = - rep->table_options.block_cache_compressed.get(); CachableEntry block; - - BlockHandle handle; - Slice input = index_value; - // We intentionally allow extra stuff in index_value so that we - // can add more features in the future. - Status s = handle.DecodeFrom(&input); - - if (!s.ok()) { - if (input_iter != nullptr) { - input_iter->SetStatus(s); - return input_iter; - } else { - return NewErrorIterator(s); - } - } - - // If either block cache is enabled, we'll try to read from it. - if (block_cache != nullptr || block_cache_compressed != nullptr) { - Statistics* statistics = rep->options.statistics.get(); - char cache_key[kMaxCacheKeyPrefixSize + kMaxVarint64Length]; - char compressed_cache_key[kMaxCacheKeyPrefixSize + kMaxVarint64Length]; - Slice key, /* key to the block cache */ - ckey /* key to the compressed block cache */; - - // create key for block cache - if (block_cache != nullptr) { - key = GetCacheKey(rep->cache_key_prefix, - rep->cache_key_prefix_size, handle, cache_key); - } - - if (block_cache_compressed != nullptr) { - ckey = GetCacheKey(rep->compressed_cache_key_prefix, - rep->compressed_cache_key_prefix_size, handle, - compressed_cache_key); - } - - s = GetDataBlockFromCache(key, ckey, block_cache, block_cache_compressed, - statistics, ro, &block); - - if (block.value == nullptr && !no_io && ro.fill_cache) { - Block* raw_block = nullptr; - { - StopWatch sw(rep->options.env, statistics, READ_BLOCK_GET_MICROS); - s = ReadBlockFromFile(rep->file.get(), rep->footer, ro, handle, - &raw_block, rep->options.env, - block_cache_compressed == nullptr); - } - - if (s.ok()) { - s = PutDataBlockToCache(key, ckey, block_cache, block_cache_compressed, - ro, statistics, &block, raw_block); - } + Slice compression_dict; + if (s.ok()) { + if (rep->compression_dict_block) { + compression_dict = rep->compression_dict_block->data; } + s = MaybeLoadDataBlockToCache(nullptr /*prefetch_buffer*/, rep, ro, handle, + compression_dict, &block, is_index); } // Didn't get any data from block caches. - if (block.value == nullptr) { + if (s.ok() && block.value == nullptr) { if (no_io) { // Could not read from block_cache and can't do IO if (input_iter != nullptr) { input_iter->SetStatus(Status::Incomplete("no blocking io")); return input_iter; } else { - return NewErrorIterator(Status::Incomplete("no blocking io")); + return NewErrorInternalIterator(Status::Incomplete("no blocking io")); } } - s = ReadBlockFromFile(rep->file.get(), rep->footer, ro, handle, - &block.value, rep->options.env); + std::unique_ptr block_value; + s = ReadBlockFromFile(rep->file.get(), nullptr /* prefetch_buffer */, + rep->footer, ro, handle, &block_value, rep->ioptions, + true /* compress */, compression_dict, + rep->persistent_cache_options, rep->global_seqno, + rep->table_options.read_amp_bytes_per_bit); + if (s.ok()) { + block.value = block_value.release(); + } } - Iterator* iter; - if (block.value != nullptr) { - iter = block.value->NewIterator(&rep->internal_comparator, input_iter); + InternalIterator* iter; + if (s.ok()) { + assert(block.value != nullptr); + iter = block.value->NewIterator(&rep->internal_comparator, input_iter, true, + rep->ioptions.statistics); if (block.cache_handle != nullptr) { iter->RegisterCleanup(&ReleaseCachedEntry, block_cache, - block.cache_handle); + block.cache_handle); } else { iter->RegisterCleanup(&DeleteHeldResource, block.value, nullptr); } } else { + assert(block.value == nullptr); if (input_iter != nullptr) { input_iter->SetStatus(s); iter = input_iter; } else { - iter = NewErrorIterator(s); + iter = NewErrorInternalIterator(s); } } return iter; } -class BlockBasedTable::BlockEntryIteratorState : public TwoLevelIteratorState { - public: - BlockEntryIteratorState(BlockBasedTable* table, - const ReadOptions& read_options) - : TwoLevelIteratorState(table->rep_->options.prefix_extractor != nullptr), - table_(table), - read_options_(read_options) {} +Status BlockBasedTable::MaybeLoadDataBlockToCache( + FilePrefetchBuffer* prefetch_buffer, Rep* rep, const ReadOptions& ro, + const BlockHandle& handle, Slice compression_dict, + CachableEntry* block_entry, bool is_index) { + assert(block_entry != nullptr); + const bool no_io = (ro.read_tier == kBlockCacheTier); + Cache* block_cache = rep->table_options.block_cache.get(); + Cache* block_cache_compressed = + rep->table_options.block_cache_compressed.get(); - Iterator* NewSecondaryIterator(const Slice& index_value) override { - return NewDataBlockIterator(table_->rep_, read_options_, index_value); + // If either block cache is enabled, we'll try to read from it. + Status s; + if (block_cache != nullptr || block_cache_compressed != nullptr) { + Statistics* statistics = rep->ioptions.statistics; + char cache_key[kMaxCacheKeyPrefixSize + kMaxVarint64Length]; + char compressed_cache_key[kMaxCacheKeyPrefixSize + kMaxVarint64Length]; + Slice key, /* key to the block cache */ + ckey /* key to the compressed block cache */; + + // create key for block cache + if (block_cache != nullptr) { + key = GetCacheKey(rep->cache_key_prefix, rep->cache_key_prefix_size, + handle, cache_key); + } + + if (block_cache_compressed != nullptr) { + ckey = GetCacheKey(rep->compressed_cache_key_prefix, + rep->compressed_cache_key_prefix_size, handle, + compressed_cache_key); + } + + s = GetDataBlockFromCache( + key, ckey, block_cache, block_cache_compressed, rep->ioptions, ro, + block_entry, rep->table_options.format_version, compression_dict, + rep->table_options.read_amp_bytes_per_bit, is_index); + + if (block_entry->value == nullptr && !no_io && ro.fill_cache) { + std::unique_ptr raw_block; + { + StopWatch sw(rep->ioptions.env, statistics, READ_BLOCK_GET_MICROS); + s = ReadBlockFromFile( + rep->file.get(), prefetch_buffer, rep->footer, ro, handle, + &raw_block, rep->ioptions, block_cache_compressed == nullptr, + compression_dict, rep->persistent_cache_options, rep->global_seqno, + rep->table_options.read_amp_bytes_per_bit); + } + + if (s.ok()) { + s = PutDataBlockToCache( + key, ckey, block_cache, block_cache_compressed, ro, rep->ioptions, + block_entry, raw_block.release(), rep->table_options.format_version, + compression_dict, rep->table_options.read_amp_bytes_per_bit, + is_index, + is_index && + rep->table_options + .cache_index_and_filter_blocks_with_high_priority + ? Cache::Priority::HIGH + : Cache::Priority::LOW); + } + } } + assert(s.ok() || block_entry->value == nullptr); + return s; +} - bool PrefixMayMatch(const Slice& internal_key) override { - if (read_options_.total_order_seek) { - return true; +BlockBasedTable::BlockEntryIteratorState::BlockEntryIteratorState( + BlockBasedTable* table, const ReadOptions& read_options, + const InternalKeyComparator* icomparator, bool skip_filters, bool is_index, + std::unordered_map>* block_map) + : TwoLevelIteratorState(table->rep_->ioptions.prefix_extractor != nullptr), + table_(table), + read_options_(read_options), + icomparator_(icomparator), + skip_filters_(skip_filters), + is_index_(is_index), + block_map_(block_map) {} + +InternalIterator* +BlockBasedTable::BlockEntryIteratorState::NewSecondaryIterator( + const Slice& index_value) { + // Return a block iterator on the index partition + BlockHandle handle; + Slice input = index_value; + Status s = handle.DecodeFrom(&input); + auto rep = table_->rep_; + if (block_map_) { + auto block = block_map_->find(handle.offset()); + // This is a possible scenario since block cache might not have had space + // for the partition + if (block != block_map_->end()) { + PERF_COUNTER_ADD(block_cache_hit_count, 1); + RecordTick(rep->ioptions.statistics, BLOCK_CACHE_INDEX_HIT); + RecordTick(rep->ioptions.statistics, BLOCK_CACHE_HIT); + Cache* block_cache = rep->table_options.block_cache.get(); + assert(block_cache); + RecordTick(rep->ioptions.statistics, BLOCK_CACHE_BYTES_READ, + block_cache->GetUsage(block->second.cache_handle)); + return block->second.value->NewIterator( + &rep->internal_comparator, nullptr, true, rep->ioptions.statistics); } - return table_->PrefixMayMatch(internal_key); } + return NewDataBlockIterator(rep, read_options_, handle, nullptr, is_index_, + s); +} - private: - // Don't own table_ - BlockBasedTable* table_; - const ReadOptions read_options_; -}; +bool BlockBasedTable::BlockEntryIteratorState::PrefixMayMatch( + const Slice& internal_key) { + if (read_options_.total_order_seek || skip_filters_) { + return true; + } + return table_->PrefixMayMatch(internal_key); +} + +bool BlockBasedTable::BlockEntryIteratorState::KeyReachedUpperBound( + const Slice& internal_key) { + bool reached_upper_bound = read_options_.iterate_upper_bound != nullptr && + icomparator_ != nullptr && + icomparator_->user_comparator()->Compare( + ExtractUserKey(internal_key), + *read_options_.iterate_upper_bound) >= 0; + TEST_SYNC_POINT_CALLBACK( + "BlockBasedTable::BlockEntryIteratorState::KeyReachedUpperBound", + &reached_upper_bound); + return reached_upper_bound; +} // This will be broken if the user specifies an unusual implementation // of Options.comparator, or if the user specifies an unusual @@ -1020,143 +1569,350 @@ bool BlockBasedTable::PrefixMayMatch(const Slice& internal_key) { return true; } - assert(rep_->options.prefix_extractor != nullptr); - auto prefix = rep_->options.prefix_extractor->Transform( - ExtractUserKey(internal_key)); - InternalKey internal_key_prefix(prefix, 0, kTypeValue); - auto internal_prefix = internal_key_prefix.Encode(); + assert(rep_->ioptions.prefix_extractor != nullptr); + auto user_key = ExtractUserKey(internal_key); + if (!rep_->ioptions.prefix_extractor->InDomain(user_key) || + rep_->table_properties->prefix_extractor_name.compare( + rep_->ioptions.prefix_extractor->Name()) != 0) { + return true; + } + auto prefix = rep_->ioptions.prefix_extractor->Transform(user_key); bool may_match = true; Status s; - // To prevent any io operation in this method, we set `read_tier` to make - // sure we always read index or filter only when they have already been - // loaded to memory. - ReadOptions no_io_read_options; - no_io_read_options.read_tier = kBlockCacheTier; - unique_ptr iiter(NewIndexIterator(no_io_read_options)); - iiter->Seek(internal_prefix); - - if (!iiter->Valid()) { - // we're past end of file - // if it's incomplete, it means that we avoided I/O - // and we're not really sure that we're past the end - // of the file - may_match = iiter->status().IsIncomplete(); - } else if (ExtractUserKey(iiter->key()).starts_with( - ExtractUserKey(internal_prefix))) { - // we need to check for this subtle case because our only - // guarantee is that "the key is a string >= last key in that data - // block" according to the doc/table_format.txt spec. - // - // Suppose iiter->key() starts with the desired prefix; it is not - // necessarily the case that the corresponding data block will - // contain the prefix, since iiter->key() need not be in the - // block. However, the next data block may contain the prefix, so - // we return true to play it safe. - may_match = true; - } else { - // iiter->key() does NOT start with the desired prefix. Because - // Seek() finds the first key that is >= the seek target, this - // means that iiter->key() > prefix. Thus, any data blocks coming - // after the data block corresponding to iiter->key() cannot - // possibly contain the key. Thus, the corresponding data block - // is the only one which could potentially contain the prefix. - Slice handle_value = iiter->value(); - BlockHandle handle; - s = handle.DecodeFrom(&handle_value); - assert(s.ok()); - auto filter_entry = GetFilter(true /* no io */); - may_match = filter_entry.value == nullptr || - filter_entry.value->PrefixMayMatch(handle.offset(), prefix); - filter_entry.Release(rep_->table_options.block_cache.get()); + // First, try check with full filter + auto filter_entry = GetFilter(); + FilterBlockReader* filter = filter_entry.value; + if (filter != nullptr) { + if (!filter->IsBlockBased()) { + const Slice* const const_ikey_ptr = &internal_key; + may_match = + filter->PrefixMayMatch(prefix, kNotValid, false, const_ikey_ptr); + } else { + InternalKey internal_key_prefix(prefix, kMaxSequenceNumber, kTypeValue); + auto internal_prefix = internal_key_prefix.Encode(); + + // To prevent any io operation in this method, we set `read_tier` to make + // sure we always read index or filter only when they have already been + // loaded to memory. + ReadOptions no_io_read_options; + no_io_read_options.read_tier = kBlockCacheTier; + + // Then, try find it within each block + unique_ptr iiter(NewIndexIterator(no_io_read_options)); + iiter->Seek(internal_prefix); + + if (!iiter->Valid()) { + // we're past end of file + // if it's incomplete, it means that we avoided I/O + // and we're not really sure that we're past the end + // of the file + may_match = iiter->status().IsIncomplete(); + } else if (ExtractUserKey(iiter->key()) + .starts_with(ExtractUserKey(internal_prefix))) { + // we need to check for this subtle case because our only + // guarantee is that "the key is a string >= last key in that data + // block" according to the doc/table_format.txt spec. + // + // Suppose iiter->key() starts with the desired prefix; it is not + // necessarily the case that the corresponding data block will + // contain the prefix, since iiter->key() need not be in the + // block. However, the next data block may contain the prefix, so + // we return true to play it safe. + may_match = true; + } else if (filter->IsBlockBased()) { + // iiter->key() does NOT start with the desired prefix. Because + // Seek() finds the first key that is >= the seek target, this + // means that iiter->key() > prefix. Thus, any data blocks coming + // after the data block corresponding to iiter->key() cannot + // possibly contain the key. Thus, the corresponding data block + // is the only on could potentially contain the prefix. + Slice handle_value = iiter->value(); + BlockHandle handle; + s = handle.DecodeFrom(&handle_value); + assert(s.ok()); + may_match = filter->PrefixMayMatch(prefix, handle.offset()); + } + } } - Statistics* statistics = rep_->options.statistics.get(); + Statistics* statistics = rep_->ioptions.statistics; RecordTick(statistics, BLOOM_FILTER_PREFIX_CHECKED); if (!may_match) { RecordTick(statistics, BLOOM_FILTER_PREFIX_USEFUL); } + // if rep_->filter_entry is not set, we should call Release(); otherwise + // don't call, in this case we have a local copy in rep_->filter_entry, + // it's pinned to the cache and will be released in the destructor + if (!rep_->filter_entry.IsSet()) { + filter_entry.Release(rep_->table_options.block_cache.get()); + } + return may_match; } -Iterator* BlockBasedTable::NewIterator(const ReadOptions& read_options, - Arena* arena) { - return NewTwoLevelIterator(new BlockEntryIteratorState(this, read_options), - NewIndexIterator(read_options), arena); +InternalIterator* BlockBasedTable::NewIterator(const ReadOptions& read_options, + Arena* arena, + bool skip_filters) { + return NewTwoLevelIterator( + new BlockEntryIteratorState(this, read_options, + &rep_->internal_comparator, skip_filters), + NewIndexIterator(read_options), arena); } -Status BlockBasedTable::Get( - const ReadOptions& read_options, const Slice& key, void* handle_context, - bool (*result_handler)(void* handle_context, const ParsedInternalKey& k, - const Slice& v), - void (*mark_key_may_exist_handler)(void* handle_context)) { - Status s; - BlockIter iiter; - NewIndexIterator(read_options, &iiter); +InternalIterator* BlockBasedTable::NewRangeTombstoneIterator( + const ReadOptions& read_options) { + if (rep_->range_del_handle.IsNull()) { + // The block didn't exist, nullptr indicates no range tombstones. + return nullptr; + } + if (rep_->range_del_entry.cache_handle != nullptr) { + // We have a handle to an uncompressed block cache entry that's held for + // this table's lifetime. Increment its refcount before returning an + // iterator based on it since the returned iterator may outlive this table + // reader. + assert(rep_->range_del_entry.value != nullptr); + Cache* block_cache = rep_->table_options.block_cache.get(); + assert(block_cache != nullptr); + if (block_cache->Ref(rep_->range_del_entry.cache_handle)) { + auto iter = rep_->range_del_entry.value->NewIterator( + &rep_->internal_comparator, nullptr /* iter */, + true /* total_order_seek */, rep_->ioptions.statistics); + iter->RegisterCleanup(&ReleaseCachedEntry, block_cache, + rep_->range_del_entry.cache_handle); + return iter; + } + } + std::string str; + rep_->range_del_handle.EncodeTo(&str); + // The meta-block exists but isn't in uncompressed block cache (maybe because + // it is disabled), so go through the full lookup process. + return NewDataBlockIterator(rep_, read_options, Slice(str)); +} - auto filter_entry = GetFilter(read_options.read_tier == kBlockCacheTier); +bool BlockBasedTable::FullFilterKeyMayMatch(const ReadOptions& read_options, + FilterBlockReader* filter, + const Slice& internal_key, + const bool no_io) const { + if (filter == nullptr || filter->IsBlockBased()) { + return true; + } + Slice user_key = ExtractUserKey(internal_key); + const Slice* const const_ikey_ptr = &internal_key; + if (filter->whole_key_filtering()) { + return filter->KeyMayMatch(user_key, kNotValid, no_io, const_ikey_ptr); + } + if (!read_options.total_order_seek && rep_->ioptions.prefix_extractor && + rep_->table_properties->prefix_extractor_name.compare( + rep_->ioptions.prefix_extractor->Name()) == 0 && + rep_->ioptions.prefix_extractor->InDomain(user_key) && + !filter->PrefixMayMatch( + rep_->ioptions.prefix_extractor->Transform(user_key), kNotValid, + false, const_ikey_ptr)) { + return false; + } + return true; +} + +Status BlockBasedTable::Get(const ReadOptions& read_options, const Slice& key, + GetContext* get_context, bool skip_filters) { + Status s; + const bool no_io = read_options.read_tier == kBlockCacheTier; + CachableEntry filter_entry; + if (!skip_filters) { + filter_entry = GetFilter(/*prefetch_buffer*/ nullptr, + read_options.read_tier == kBlockCacheTier); + } FilterBlockReader* filter = filter_entry.value; - bool done = false; - for (iiter.Seek(key); iiter.Valid() && !done; iiter.Next()) { - Slice handle_value = iiter.value(); - BlockHandle handle; - bool may_not_exist_in_filter = - filter != nullptr && handle.DecodeFrom(&handle_value).ok() && - !filter->KeyMayMatch(handle.offset(), ExtractUserKey(key)); - - if (may_not_exist_in_filter) { - // Not found - // TODO: think about interaction with Merge. If a user key cannot - // cross one data block, we should be fine. - RecordTick(rep_->options.statistics.get(), BLOOM_FILTER_USEFUL); - break; - } else { - BlockIter biter; - NewDataBlockIterator(rep_, read_options, iiter.value(), &biter); - - if (read_options.read_tier && biter.status().IsIncomplete()) { - // couldn't get block from block_cache - // Update Saver.state to Found because we are only looking for whether - // we can guarantee the key is not there when "no_io" is set - (*mark_key_may_exist_handler)(handle_context); + // First check the full filter + // If full filter not useful, Then go into each block + if (!FullFilterKeyMayMatch(read_options, filter, key, no_io)) { + RecordTick(rep_->ioptions.statistics, BLOOM_FILTER_USEFUL); + } else { + BlockIter iiter_on_stack; + auto iiter = NewIndexIterator(read_options, &iiter_on_stack); + std::unique_ptr iiter_unique_ptr; + if (iiter != &iiter_on_stack) { + iiter_unique_ptr.reset(iiter); + } + + bool done = false; + for (iiter->Seek(key); iiter->Valid() && !done; iiter->Next()) { + Slice handle_value = iiter->value(); + + BlockHandle handle; + bool not_exist_in_filter = + filter != nullptr && filter->IsBlockBased() == true && + handle.DecodeFrom(&handle_value).ok() && + !filter->KeyMayMatch(ExtractUserKey(key), handle.offset(), no_io); + + if (not_exist_in_filter) { + // Not found + // TODO: think about interaction with Merge. If a user key cannot + // cross one data block, we should be fine. + RecordTick(rep_->ioptions.statistics, BLOOM_FILTER_USEFUL); break; - } - if (!biter.status().ok()) { + } else { + BlockIter biter; + NewDataBlockIterator(rep_, read_options, iiter->value(), &biter); + + if (read_options.read_tier == kBlockCacheTier && + biter.status().IsIncomplete()) { + // couldn't get block from block_cache + // Update Saver.state to Found because we are only looking for whether + // we can guarantee the key is not there when "no_io" is set + get_context->MarkKeyMayExist(); + break; + } + if (!biter.status().ok()) { + s = biter.status(); + break; + } + + // Call the *saver function on each entry/block until it returns false + for (biter.Seek(key); biter.Valid(); biter.Next()) { + ParsedInternalKey parsed_key; + if (!ParseInternalKey(biter.key(), &parsed_key)) { + s = Status::Corruption(Slice()); + } + + if (!get_context->SaveValue(parsed_key, biter.value(), &biter)) { + done = true; + break; + } + } s = biter.status(); + } + if (done) { + // Avoid the extra Next which is expensive in two-level indexes break; } + } + if (s.ok()) { + s = iiter->status(); + } + } - // Call the *saver function on each entry/block until it returns false - for (biter.Seek(key); biter.Valid(); biter.Next()) { - ParsedInternalKey parsed_key; - if (!ParseInternalKey(biter.key(), &parsed_key)) { - s = Status::Corruption(Slice()); - } + // if rep_->filter_entry is not set, we should call Release(); otherwise + // don't call, in this case we have a local copy in rep_->filter_entry, + // it's pinned to the cache and will be released in the destructor + if (!rep_->filter_entry.IsSet()) { + filter_entry.Release(rep_->table_options.block_cache.get()); + } + return s; +} - if (!(*result_handler)(handle_context, parsed_key, - biter.value())) { - done = true; - break; - } +Status BlockBasedTable::Prefetch(const Slice* const begin, + const Slice* const end) { + auto& comparator = rep_->internal_comparator; + // pre-condition + if (begin && end && comparator.Compare(*begin, *end) > 0) { + return Status::InvalidArgument(*begin, *end); + } + + BlockIter iiter_on_stack; + auto iiter = NewIndexIterator(ReadOptions(), &iiter_on_stack); + std::unique_ptr iiter_unique_ptr; + if (iiter != &iiter_on_stack) { + iiter_unique_ptr = std::unique_ptr(iiter); + } + + if (!iiter->status().ok()) { + // error opening index iterator + return iiter->status(); + } + + // indicates if we are on the last page that need to be pre-fetched + bool prefetching_boundary_page = false; + + for (begin ? iiter->Seek(*begin) : iiter->SeekToFirst(); iiter->Valid(); + iiter->Next()) { + Slice block_handle = iiter->value(); + + if (end && comparator.Compare(iiter->key(), *end) >= 0) { + if (prefetching_boundary_page) { + break; } - s = biter.status(); + + // The index entry represents the last key in the data block. + // We should load this page into memory as well, but no more + prefetching_boundary_page = true; + } + + // Load the block specified by the block_handle into the block cache + BlockIter biter; + NewDataBlockIterator(rep_, ReadOptions(), block_handle, &biter); + + if (!biter.status().ok()) { + // there was an unexpected error while pre-fetching + return biter.status(); } } - filter_entry.Release(rep_->table_options.block_cache.get()); + return Status::OK(); +} + +Status BlockBasedTable::VerifyChecksum() { + Status s; + // Check Meta blocks + std::unique_ptr meta; + std::unique_ptr meta_iter; + s = ReadMetaBlock(rep_, nullptr /* prefetch buffer */, &meta, &meta_iter); if (s.ok()) { - s = iiter.status(); + s = VerifyChecksumInBlocks(meta_iter.get()); + if (!s.ok()) { + return s; + } + } else { + return s; } + // Check Data blocks + BlockIter iiter_on_stack; + InternalIterator* iiter = NewIndexIterator(ReadOptions(), &iiter_on_stack); + std::unique_ptr iiter_unique_ptr; + if (iiter != &iiter_on_stack) { + iiter_unique_ptr = std::unique_ptr(iiter); + } + if (!iiter->status().ok()) { + // error opening index iterator + return iiter->status(); + } + s = VerifyChecksumInBlocks(iiter); + return s; +} +Status BlockBasedTable::VerifyChecksumInBlocks(InternalIterator* index_iter) { + Status s; + for (index_iter->SeekToFirst(); index_iter->Valid(); index_iter->Next()) { + s = index_iter->status(); + if (!s.ok()) { + break; + } + BlockHandle handle; + Slice input = index_iter->value(); + s = handle.DecodeFrom(&input); + if (!s.ok()) { + break; + } + BlockContents contents; + s = ReadBlockContents(rep_->file.get(), nullptr /* prefetch buffer */, + rep_->footer, ReadOptions(), handle, &contents, + rep_->ioptions, false /* decompress */, + Slice() /*compression dict*/, + rep_->persistent_cache_options); + if (!s.ok()) { + break; + } + } return s; } bool BlockBasedTable::TEST_KeyInCache(const ReadOptions& options, const Slice& key) { - std::unique_ptr iiter(NewIndexIterator(options)); + std::unique_ptr iiter(NewIndexIterator(options)); iiter->Seek(key); assert(iiter->Valid()); CachableEntry block; @@ -1170,12 +1926,16 @@ bool BlockBasedTable::TEST_KeyInCache(const ReadOptions& options, char cache_key_storage[kMaxCacheKeyPrefixSize + kMaxVarint64Length]; Slice cache_key = - GetCacheKey(rep_->cache_key_prefix, rep_->cache_key_prefix_size, handle, - cache_key_storage); + GetCacheKey(rep_->cache_key_prefix, rep_->cache_key_prefix_size, + handle, cache_key_storage); Slice ckey; - s = GetDataBlockFromCache(cache_key, ckey, block_cache, nullptr, nullptr, - options, &block); + s = GetDataBlockFromCache( + cache_key, ckey, block_cache, nullptr, rep_->ioptions, options, &block, + rep_->table_options.format_version, + rep_->compression_dict_block ? rep_->compression_dict_block->data + : Slice(), + 0 /* read_amp_bytes_per_bit */); assert(s.ok()); bool in_cache = block.value != nullptr; if (in_cache) { @@ -1190,8 +1950,9 @@ bool BlockBasedTable::TEST_KeyInCache(const ReadOptions& options, // 3. options // 4. internal_comparator // 5. index_type -Status BlockBasedTable::CreateIndexReader(IndexReader** index_reader, - Iterator* preloaded_meta_index_iter) { +Status BlockBasedTable::CreateIndexReader( + FilePrefetchBuffer* prefetch_buffer, IndexReader** index_reader, + InternalIterator* preloaded_meta_index_iter, int level) { // Some old version of block-based tables don't have index type present in // table properties. If that's the case we can safely use the kBinarySearch. auto index_type_on_file = BlockBasedTableOptions::kBinarySearch; @@ -1205,61 +1966,66 @@ Status BlockBasedTable::CreateIndexReader(IndexReader** index_reader, } auto file = rep_->file.get(); - auto env = rep_->options.env; - auto comparator = &rep_->internal_comparator; + const InternalKeyComparator* icomparator = &rep_->internal_comparator; const Footer& footer = rep_->footer; - if (index_type_on_file == BlockBasedTableOptions::kHashSearch && - rep_->options.prefix_extractor == nullptr) { - Log(rep_->options.info_log, - "BlockBasedTableOptions::kHashSearch requires " - "options.prefix_extractor to be set." - " Fall back to binary seach index."); + rep_->ioptions.prefix_extractor == nullptr) { + ROCKS_LOG_WARN(rep_->ioptions.info_log, + "BlockBasedTableOptions::kHashSearch requires " + "options.prefix_extractor to be set." + " Fall back to binary search index."); index_type_on_file = BlockBasedTableOptions::kBinarySearch; } switch (index_type_on_file) { + case BlockBasedTableOptions::kTwoLevelIndexSearch: { + return PartitionIndexReader::Create( + this, file, prefetch_buffer, footer, footer.index_handle(), + rep_->ioptions, icomparator, index_reader, + rep_->persistent_cache_options, level); + } case BlockBasedTableOptions::kBinarySearch: { return BinarySearchIndexReader::Create( - file, footer, footer.index_handle(), env, comparator, index_reader); + file, prefetch_buffer, footer, footer.index_handle(), rep_->ioptions, + icomparator, index_reader, rep_->persistent_cache_options); } case BlockBasedTableOptions::kHashSearch: { std::unique_ptr meta_guard; - std::unique_ptr meta_iter_guard; + std::unique_ptr meta_iter_guard; auto meta_index_iter = preloaded_meta_index_iter; if (meta_index_iter == nullptr) { - auto s = ReadMetaBlock(rep_, &meta_guard, &meta_iter_guard); + auto s = + ReadMetaBlock(rep_, prefetch_buffer, &meta_guard, &meta_iter_guard); if (!s.ok()) { // we simply fall back to binary search in case there is any // problem with prefix hash index loading. - Log(rep_->options.info_log, - "Unable to read the metaindex block." - " Fall back to binary seach index."); + ROCKS_LOG_WARN(rep_->ioptions.info_log, + "Unable to read the metaindex block." + " Fall back to binary search index."); return BinarySearchIndexReader::Create( - file, footer, footer.index_handle(), env, comparator, index_reader); + file, prefetch_buffer, footer, footer.index_handle(), + rep_->ioptions, icomparator, index_reader, + rep_->persistent_cache_options); } meta_index_iter = meta_iter_guard.get(); } - // We need to wrap data with internal_prefix_transform to make sure it can - // handle prefix correctly. - rep_->internal_prefix_transform.reset( - new InternalKeySliceTransform(rep_->options.prefix_extractor.get())); return HashIndexReader::Create( - rep_->internal_prefix_transform.get(), footer, file, env, comparator, - footer.index_handle(), meta_index_iter, index_reader, - rep_->hash_index_allow_collision); + rep_->internal_prefix_transform.get(), footer, file, prefetch_buffer, + rep_->ioptions, icomparator, footer.index_handle(), meta_index_iter, + index_reader, rep_->hash_index_allow_collision, + rep_->persistent_cache_options); } default: { std::string error_message = - "Unrecognized index type: " + std::to_string(rep_->index_type); + "Unrecognized index type: " + ToString(index_type_on_file); return Status::InvalidArgument(error_message.c_str()); } } } uint64_t BlockBasedTable::ApproximateOffsetOf(const Slice& key) { - unique_ptr index_iter(NewIndexIterator(ReadOptions())); + unique_ptr index_iter(NewIndexIterator(ReadOptions())); index_iter->Seek(key); uint64_t result; @@ -1299,4 +2065,391 @@ bool BlockBasedTable::TEST_index_reader_preloaded() const { return rep_->index_reader != nullptr; } +Status BlockBasedTable::GetKVPairsFromDataBlocks( + std::vector* kv_pair_blocks) { + std::unique_ptr blockhandles_iter( + NewIndexIterator(ReadOptions())); + + Status s = blockhandles_iter->status(); + if (!s.ok()) { + // Cannot read Index Block + return s; + } + + for (blockhandles_iter->SeekToFirst(); blockhandles_iter->Valid(); + blockhandles_iter->Next()) { + s = blockhandles_iter->status(); + + if (!s.ok()) { + break; + } + + std::unique_ptr datablock_iter; + datablock_iter.reset( + NewDataBlockIterator(rep_, ReadOptions(), blockhandles_iter->value())); + s = datablock_iter->status(); + + if (!s.ok()) { + // Error reading the block - Skipped + continue; + } + + KVPairBlock kv_pair_block; + for (datablock_iter->SeekToFirst(); datablock_iter->Valid(); + datablock_iter->Next()) { + s = datablock_iter->status(); + if (!s.ok()) { + // Error reading the block - Skipped + break; + } + const Slice& key = datablock_iter->key(); + const Slice& value = datablock_iter->value(); + std::string key_copy = std::string(key.data(), key.size()); + std::string value_copy = std::string(value.data(), value.size()); + + kv_pair_block.push_back( + std::make_pair(std::move(key_copy), std::move(value_copy))); + } + kv_pair_blocks->push_back(std::move(kv_pair_block)); + } + return Status::OK(); +} + +Status BlockBasedTable::DumpTable(WritableFile* out_file) { + // Output Footer + out_file->Append( + "Footer Details:\n" + "--------------------------------------\n" + " "); + out_file->Append(rep_->footer.ToString().c_str()); + out_file->Append("\n"); + + // Output MetaIndex + out_file->Append( + "Metaindex Details:\n" + "--------------------------------------\n"); + std::unique_ptr meta; + std::unique_ptr meta_iter; + Status s = + ReadMetaBlock(rep_, nullptr /* prefetch_buffer */, &meta, &meta_iter); + if (s.ok()) { + for (meta_iter->SeekToFirst(); meta_iter->Valid(); meta_iter->Next()) { + s = meta_iter->status(); + if (!s.ok()) { + return s; + } + if (meta_iter->key() == rocksdb::kPropertiesBlock) { + out_file->Append(" Properties block handle: "); + out_file->Append(meta_iter->value().ToString(true).c_str()); + out_file->Append("\n"); + } else if (meta_iter->key() == rocksdb::kCompressionDictBlock) { + out_file->Append(" Compression dictionary block handle: "); + out_file->Append(meta_iter->value().ToString(true).c_str()); + out_file->Append("\n"); + } else if (strstr(meta_iter->key().ToString().c_str(), + "filter.rocksdb.") != nullptr) { + out_file->Append(" Filter block handle: "); + out_file->Append(meta_iter->value().ToString(true).c_str()); + out_file->Append("\n"); + } else if (meta_iter->key() == rocksdb::kRangeDelBlock) { + out_file->Append(" Range deletion block handle: "); + out_file->Append(meta_iter->value().ToString(true).c_str()); + out_file->Append("\n"); + } + } + out_file->Append("\n"); + } else { + return s; + } + + // Output TableProperties + const rocksdb::TableProperties* table_properties; + table_properties = rep_->table_properties.get(); + + if (table_properties != nullptr) { + out_file->Append( + "Table Properties:\n" + "--------------------------------------\n" + " "); + out_file->Append(table_properties->ToString("\n ", ": ").c_str()); + out_file->Append("\n"); + } + + // Output Filter blocks + if (!rep_->filter && !table_properties->filter_policy_name.empty()) { + // Support only BloomFilter as off now + rocksdb::BlockBasedTableOptions table_options; + table_options.filter_policy.reset(rocksdb::NewBloomFilterPolicy(1)); + if (table_properties->filter_policy_name.compare( + table_options.filter_policy->Name()) == 0) { + std::string filter_block_key = kFilterBlockPrefix; + filter_block_key.append(table_properties->filter_policy_name); + BlockHandle handle; + if (FindMetaBlock(meta_iter.get(), filter_block_key, &handle).ok()) { + BlockContents block; + if (ReadBlockContents(rep_->file.get(), nullptr /* prefetch_buffer */, + rep_->footer, ReadOptions(), handle, &block, + rep_->ioptions, false /*decompress*/, + Slice() /*compression dict*/, + rep_->persistent_cache_options) + .ok()) { + rep_->filter.reset(new BlockBasedFilterBlockReader( + rep_->ioptions.prefix_extractor, table_options, + table_options.whole_key_filtering, std::move(block), + rep_->ioptions.statistics)); + } + } + } + } + if (rep_->filter) { + out_file->Append( + "Filter Details:\n" + "--------------------------------------\n" + " "); + out_file->Append(rep_->filter->ToString().c_str()); + out_file->Append("\n"); + } + + // Output Index block + s = DumpIndexBlock(out_file); + if (!s.ok()) { + return s; + } + + // Output compression dictionary + if (rep_->compression_dict_block != nullptr) { + auto compression_dict = rep_->compression_dict_block->data; + out_file->Append( + "Compression Dictionary:\n" + "--------------------------------------\n"); + out_file->Append(" size (bytes): "); + out_file->Append(rocksdb::ToString(compression_dict.size())); + out_file->Append("\n\n"); + out_file->Append(" HEX "); + out_file->Append(compression_dict.ToString(true).c_str()); + out_file->Append("\n\n"); + } + + // Output range deletions block + auto* range_del_iter = NewRangeTombstoneIterator(ReadOptions()); + if (range_del_iter != nullptr) { + range_del_iter->SeekToFirst(); + if (range_del_iter->Valid()) { + out_file->Append( + "Range deletions:\n" + "--------------------------------------\n" + " "); + for (; range_del_iter->Valid(); range_del_iter->Next()) { + DumpKeyValue(range_del_iter->key(), range_del_iter->value(), out_file); + } + out_file->Append("\n"); + } + delete range_del_iter; + } + // Output Data blocks + s = DumpDataBlocks(out_file); + + return s; +} + +void BlockBasedTable::Close() { + if (rep_->closed) { + return; + } + rep_->filter_entry.Release(rep_->table_options.block_cache.get()); + rep_->index_entry.Release(rep_->table_options.block_cache.get()); + rep_->range_del_entry.Release(rep_->table_options.block_cache.get()); + // cleanup index and filter blocks to avoid accessing dangling pointer + if (!rep_->table_options.no_block_cache) { + char cache_key[kMaxCacheKeyPrefixSize + kMaxVarint64Length]; + // Get the filter block key + auto key = GetCacheKey(rep_->cache_key_prefix, rep_->cache_key_prefix_size, + rep_->filter_handle, cache_key); + rep_->table_options.block_cache.get()->Erase(key); + // Get the index block key + key = GetCacheKeyFromOffset(rep_->cache_key_prefix, + rep_->cache_key_prefix_size, + rep_->dummy_index_reader_offset, cache_key); + rep_->table_options.block_cache.get()->Erase(key); + } + rep_->closed = true; +} + +Status BlockBasedTable::DumpIndexBlock(WritableFile* out_file) { + out_file->Append( + "Index Details:\n" + "--------------------------------------\n"); + + std::unique_ptr blockhandles_iter( + NewIndexIterator(ReadOptions())); + Status s = blockhandles_iter->status(); + if (!s.ok()) { + out_file->Append("Can not read Index Block \n\n"); + return s; + } + + out_file->Append(" Block key hex dump: Data block handle\n"); + out_file->Append(" Block key ascii\n\n"); + for (blockhandles_iter->SeekToFirst(); blockhandles_iter->Valid(); + blockhandles_iter->Next()) { + s = blockhandles_iter->status(); + if (!s.ok()) { + break; + } + Slice key = blockhandles_iter->key(); + InternalKey ikey; + ikey.DecodeFrom(key); + + out_file->Append(" HEX "); + out_file->Append(ikey.user_key().ToString(true).c_str()); + out_file->Append(": "); + out_file->Append(blockhandles_iter->value().ToString(true).c_str()); + out_file->Append("\n"); + + std::string str_key = ikey.user_key().ToString(); + std::string res_key(""); + char cspace = ' '; + for (size_t i = 0; i < str_key.size(); i++) { + res_key.append(&str_key[i], 1); + res_key.append(1, cspace); + } + out_file->Append(" ASCII "); + out_file->Append(res_key.c_str()); + out_file->Append("\n ------\n"); + } + out_file->Append("\n"); + return Status::OK(); +} + +Status BlockBasedTable::DumpDataBlocks(WritableFile* out_file) { + std::unique_ptr blockhandles_iter( + NewIndexIterator(ReadOptions())); + Status s = blockhandles_iter->status(); + if (!s.ok()) { + out_file->Append("Can not read Index Block \n\n"); + return s; + } + + uint64_t datablock_size_min = std::numeric_limits::max(); + uint64_t datablock_size_max = 0; + uint64_t datablock_size_sum = 0; + + size_t block_id = 1; + for (blockhandles_iter->SeekToFirst(); blockhandles_iter->Valid(); + block_id++, blockhandles_iter->Next()) { + s = blockhandles_iter->status(); + if (!s.ok()) { + break; + } + + Slice bh_val = blockhandles_iter->value(); + BlockHandle bh; + bh.DecodeFrom(&bh_val); + uint64_t datablock_size = bh.size(); + datablock_size_min = std::min(datablock_size_min, datablock_size); + datablock_size_max = std::max(datablock_size_max, datablock_size); + datablock_size_sum += datablock_size; + + out_file->Append("Data Block # "); + out_file->Append(rocksdb::ToString(block_id)); + out_file->Append(" @ "); + out_file->Append(blockhandles_iter->value().ToString(true).c_str()); + out_file->Append("\n"); + out_file->Append("--------------------------------------\n"); + + std::unique_ptr datablock_iter; + datablock_iter.reset( + NewDataBlockIterator(rep_, ReadOptions(), blockhandles_iter->value())); + s = datablock_iter->status(); + + if (!s.ok()) { + out_file->Append("Error reading the block - Skipped \n\n"); + continue; + } + + for (datablock_iter->SeekToFirst(); datablock_iter->Valid(); + datablock_iter->Next()) { + s = datablock_iter->status(); + if (!s.ok()) { + out_file->Append("Error reading the block - Skipped \n"); + break; + } + DumpKeyValue(datablock_iter->key(), datablock_iter->value(), out_file); + } + out_file->Append("\n"); + } + + uint64_t num_datablocks = block_id - 1; + if (num_datablocks) { + double datablock_size_avg = + static_cast(datablock_size_sum) / num_datablocks; + out_file->Append("Data Block Summary:\n"); + out_file->Append("--------------------------------------"); + out_file->Append("\n # data blocks: "); + out_file->Append(rocksdb::ToString(num_datablocks)); + out_file->Append("\n min data block size: "); + out_file->Append(rocksdb::ToString(datablock_size_min)); + out_file->Append("\n max data block size: "); + out_file->Append(rocksdb::ToString(datablock_size_max)); + out_file->Append("\n avg data block size: "); + out_file->Append(rocksdb::ToString(datablock_size_avg)); + out_file->Append("\n"); + } + + return Status::OK(); +} + +void BlockBasedTable::DumpKeyValue(const Slice& key, const Slice& value, + WritableFile* out_file) { + InternalKey ikey; + ikey.DecodeFrom(key); + + out_file->Append(" HEX "); + out_file->Append(ikey.user_key().ToString(true).c_str()); + out_file->Append(": "); + out_file->Append(value.ToString(true).c_str()); + out_file->Append("\n"); + + std::string str_key = ikey.user_key().ToString(); + std::string str_value = value.ToString(); + std::string res_key(""), res_value(""); + char cspace = ' '; + for (size_t i = 0; i < str_key.size(); i++) { + res_key.append(&str_key[i], 1); + res_key.append(1, cspace); + } + for (size_t i = 0; i < str_value.size(); i++) { + res_value.append(&str_value[i], 1); + res_value.append(1, cspace); + } + + out_file->Append(" ASCII "); + out_file->Append(res_key.c_str()); + out_file->Append(": "); + out_file->Append(res_value.c_str()); + out_file->Append("\n ------\n"); +} + +namespace { + +void DeleteCachedFilterEntry(const Slice& key, void* value) { + FilterBlockReader* filter = reinterpret_cast(value); + if (filter->statistics() != nullptr) { + RecordTick(filter->statistics(), BLOCK_CACHE_FILTER_BYTES_EVICT, + filter->size()); + } + delete filter; +} + +void DeleteCachedIndexEntry(const Slice& key, void* value) { + IndexReader* index_reader = reinterpret_cast(value); + if (index_reader->statistics() != nullptr) { + RecordTick(index_reader->statistics(), BLOCK_CACHE_INDEX_BYTES_EVICT, + index_reader->usable_size()); + } + delete index_reader; +} + +} // anonymous namespace + } // namespace rocksdb diff --git a/table/block_based_table_reader.h b/table/block_based_table_reader.h index 3ff97dda68f..a5426cdedf7 100644 --- a/table/block_based_table_reader.h +++ b/table/block_based_table_reader.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -11,14 +11,25 @@ #include #include -#include +#include #include +#include +#include +#include "options/cf_options.h" +#include "rocksdb/options.h" +#include "rocksdb/persistent_cache.h" #include "rocksdb/statistics.h" #include "rocksdb/status.h" #include "rocksdb/table.h" +#include "table/filter_block.h" +#include "table/format.h" +#include "table/persistent_cache_helper.h" +#include "table/table_properties_internal.h" #include "table/table_reader.h" +#include "table/two_level_iterator.h" #include "util/coding.h" +#include "util/file_reader_writer.h" namespace rocksdb { @@ -27,6 +38,8 @@ class BlockIter; class BlockHandle; class Cache; class FilterBlockReader; +class BlockBasedFilterBlockReader; +class FullFilterBlockReader; class Footer; class InternalKeyComparator; class Iterator; @@ -36,17 +49,25 @@ class TableReader; class WritableFile; struct BlockBasedTableOptions; struct EnvOptions; -struct Options; struct ReadOptions; +class GetContext; +class InternalIterator; using std::unique_ptr; +typedef std::vector> KVPairBlock; + // A Table is a sorted map from strings to strings. Tables are // immutable and persistent. A Table may be safely accessed from // multiple threads without external synchronization. class BlockBasedTable : public TableReader { public: static const std::string kFilterBlockPrefix; + static const std::string kFullFilterBlockPrefix; + static const std::string kPartitionedFilterBlockPrefix; + // The longest prefix of the cache key used to identify blocks. + // For Posix files the unique ID is three varints. + static const size_t kMaxCacheKeyPrefixSize = kMaxVarint64Length * 3 + 1; // Attempt to open the table that is stored in bytes [0..file_size) // of "file", and read the metadata entries necessary to allow @@ -57,26 +78,43 @@ class BlockBasedTable : public TableReader { // If there was an error while initializing the table, sets "*table_reader" // to nullptr and returns a non-ok status. // - // *file must remain live while this Table is in use. - static Status Open(const Options& db_options, const EnvOptions& env_options, + // @param file must remain live while this Table is in use. + // @param prefetch_index_and_filter_in_cache can be used to disable + // prefetching of + // index and filter blocks into block cache at startup + // @param skip_filters Disables loading/accessing the filter block. Overrides + // prefetch_index_and_filter_in_cache, so filter will be skipped if both + // are set. + static Status Open(const ImmutableCFOptions& ioptions, + const EnvOptions& env_options, const BlockBasedTableOptions& table_options, const InternalKeyComparator& internal_key_comparator, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table_reader); + unique_ptr&& file, + uint64_t file_size, unique_ptr* table_reader, + bool prefetch_index_and_filter_in_cache = true, + bool skip_filters = false, int level = -1); bool PrefixMayMatch(const Slice& internal_key); // Returns a new iterator over the table contents. // The result of NewIterator() is initially invalid (caller must // call one of the Seek methods on the iterator before using it). - Iterator* NewIterator(const ReadOptions&, Arena* arena = nullptr) override; + // @param skip_filters Disables loading/accessing the filter block + InternalIterator* NewIterator( + const ReadOptions&, Arena* arena = nullptr, + bool skip_filters = false) override; + InternalIterator* NewRangeTombstoneIterator( + const ReadOptions& read_options) override; + + // @param skip_filters Disables loading/accessing the filter block Status Get(const ReadOptions& readOptions, const Slice& key, - void* handle_context, - bool (*result_handler)(void* handle_context, - const ParsedInternalKey& k, const Slice& v), - void (*mark_key_may_exist_handler)(void* handle_context) = - nullptr) override; + GetContext* get_context, bool skip_filters = false) override; + + // Pre-fetch the disk blocks that correspond to the key range specified by + // (kbegin, kend). The call will return error status in the event of + // IO or iteration error. + Status Prefetch(const Slice* begin, const Slice* end) override; // Given a key, return an approximate byte offset in the file where // the data for that key begins (or would begin if the key were @@ -98,31 +136,118 @@ class BlockBasedTable : public TableReader { size_t ApproximateMemoryUsage() const override; + // convert SST file to a human readable form + Status DumpTable(WritableFile* out_file) override; + + Status VerifyChecksum() override; + + void Close() override; + ~BlockBasedTable(); bool TEST_filter_block_preloaded() const; bool TEST_index_reader_preloaded() const; - // Implementation of IndexReader will be exposed to internal cc file only. - class IndexReader; - private: + // IndexReader is the interface that provide the functionality for index + // access. + class IndexReader { + public: + explicit IndexReader(const InternalKeyComparator* icomparator, + Statistics* stats) + : icomparator_(icomparator), statistics_(stats) {} + + virtual ~IndexReader() {} + + // Create an iterator for index access. + // If iter is null then a new object is created on heap and the callee will + // have the ownership. If a non-null iter is passed in it will be used, and + // the returned value is either the same as iter or a new on-heap object + // that + // wrapps the passed iter. In the latter case the return value would point + // to + // a different object then iter and the callee has the ownership of the + // returned object. + virtual InternalIterator* NewIterator(BlockIter* iter = nullptr, + bool total_order_seek = true) = 0; + + // The size of the index. + virtual size_t size() const = 0; + // Memory usage of the index block + virtual size_t usable_size() const = 0; + // return the statistics pointer + virtual Statistics* statistics() const { return statistics_; } + // Report an approximation of how much memory has been used other than + // memory + // that was allocated in block cache. + virtual size_t ApproximateMemoryUsage() const = 0; + + virtual void CacheDependencies(bool /* unused */) {} + + // Prefetch all the blocks referenced by this index to the buffer + void PrefetchBlocks(FilePrefetchBuffer* buf); + + protected: + const InternalKeyComparator* icomparator_; + + private: + Statistics* statistics_; + }; + + static Slice GetCacheKey(const char* cache_key_prefix, + size_t cache_key_prefix_size, + const BlockHandle& handle, char* cache_key); + + // Retrieve all key value pairs from data blocks in the table. + // The key retrieved are internal keys. + Status GetKVPairsFromDataBlocks(std::vector* kv_pair_blocks); + + class BlockEntryIteratorState; + + friend class PartitionIndexReader; + + protected: template struct CachableEntry; - struct Rep; Rep* rep_; - bool compaction_optimized_; + explicit BlockBasedTable(Rep* rep) : rep_(rep) {} - class BlockEntryIteratorState; + private: + friend class MockedBlockBasedTable; // input_iter: if it is not null, update this one and return it as Iterator - static Iterator* NewDataBlockIterator(Rep* rep, const ReadOptions& ro, - const Slice& index_value, - BlockIter* input_iter = nullptr); + static InternalIterator* NewDataBlockIterator(Rep* rep, const ReadOptions& ro, + const Slice& index_value, + BlockIter* input_iter = nullptr, + bool is_index = false); + static InternalIterator* NewDataBlockIterator(Rep* rep, const ReadOptions& ro, + const BlockHandle& block_hanlde, + BlockIter* input_iter = nullptr, + bool is_index = false, + Status s = Status()); + // If block cache enabled (compressed or uncompressed), looks for the block + // identified by handle in (1) uncompressed cache, (2) compressed cache, and + // then (3) file. If found, inserts into the cache(s) that were searched + // unsuccessfully (e.g., if found in file, will add to both uncompressed and + // compressed caches if they're enabled). + // + // @param block_entry value is set to the uncompressed block if found. If + // in uncompressed block cache, also sets cache_handle to reference that + // block. + static Status MaybeLoadDataBlockToCache(FilePrefetchBuffer* prefetch_buffer, + Rep* rep, const ReadOptions& ro, + const BlockHandle& handle, + Slice compression_dict, + CachableEntry* block_entry, + bool is_index = false); // For the following two functions: // if `no_io == true`, we will not try to read filter/index from sst file // were they not present in cache yet. - CachableEntry GetFilter(bool no_io = false) const; + CachableEntry GetFilter( + FilePrefetchBuffer* prefetch_buffer = nullptr, bool no_io = false) const; + virtual CachableEntry GetFilter( + FilePrefetchBuffer* prefetch_buffer, const BlockHandle& filter_blk_handle, + const bool is_a_filter_partition, bool no_io) const; // Get the iterator from the index reader. // If input_iter is not set, return new Iterator @@ -134,18 +259,24 @@ class BlockBasedTable : public TableReader { // 2. index is not present in block cache. // 3. We disallowed any io to be performed, that is, read_options == // kBlockCacheTier - Iterator* NewIndexIterator(const ReadOptions& read_options, - BlockIter* input_iter = nullptr); + InternalIterator* NewIndexIterator( + const ReadOptions& read_options, BlockIter* input_iter = nullptr, + CachableEntry* index_entry = nullptr); // Read block cache from block caches (if set): block_cache and // block_cache_compressed. // On success, Status::OK with be returned and @block will be populated with // pointer to the block as well as its block handle. + // @param compression_dict Data for presetting the compression library's + // dictionary. static Status GetDataBlockFromCache( const Slice& block_cache_key, const Slice& compressed_block_cache_key, - Cache* block_cache, Cache* block_cache_compressed, Statistics* statistics, - const ReadOptions& read_options, - BlockBasedTable::CachableEntry* block); + Cache* block_cache, Cache* block_cache_compressed, + const ImmutableCFOptions& ioptions, const ReadOptions& read_options, + BlockBasedTable::CachableEntry* block, uint32_t format_version, + const Slice& compression_dict, size_t read_amp_bytes_per_bit, + bool is_index = false); + // Put a raw block (maybe compressed) to the corresponding block caches. // This method will perform decompression against raw_block if needed and then // populate the block caches. @@ -154,11 +285,15 @@ class BlockBasedTable : public TableReader { // // REQUIRES: raw_block is heap-allocated. PutDataBlockToCache() will be // responsible for releasing its memory if error occurs. + // @param compression_dict Data for presetting the compression library's + // dictionary. static Status PutDataBlockToCache( const Slice& block_cache_key, const Slice& compressed_block_cache_key, Cache* block_cache, Cache* block_cache_compressed, - const ReadOptions& read_options, Statistics* statistics, - CachableEntry* block, Block* raw_block); + const ReadOptions& read_options, const ImmutableCFOptions& ioptions, + CachableEntry* block, Block* raw_block, uint32_t format_version, + const Slice& compression_dict, size_t read_amp_bytes_per_bit, + bool is_index = false, Cache::Priority pri = Cache::Priority::LOW); // Calls (*handle_result)(arg, ...) repeatedly, starting with the entry found // after a call to Seek(key), until handle_result returns false. @@ -172,23 +307,28 @@ class BlockBasedTable : public TableReader { // Optionally, user can pass a preloaded meta_index_iter for the index that // need to access extra meta blocks for index construction. This parameter // helps avoid re-reading meta index block if caller already created one. - Status CreateIndexReader(IndexReader** index_reader, - Iterator* preloaded_meta_index_iter = nullptr); + Status CreateIndexReader( + FilePrefetchBuffer* prefetch_buffer, IndexReader** index_reader, + InternalIterator* preloaded_meta_index_iter = nullptr, + const int level = -1); + + bool FullFilterKeyMayMatch(const ReadOptions& read_options, + FilterBlockReader* filter, const Slice& user_key, + const bool no_io) const; // Read the meta block from sst. - static Status ReadMetaBlock( - Rep* rep, - std::unique_ptr* meta_block, - std::unique_ptr* iter); + static Status ReadMetaBlock(Rep* rep, FilePrefetchBuffer* prefetch_buffer, + std::unique_ptr* meta_block, + std::unique_ptr* iter); - // Create the filter from the filter block. - static FilterBlockReader* ReadFilter(const BlockHandle& filter_handle, - Rep* rep, size_t* filter_size = nullptr); + Status VerifyChecksumInBlocks(InternalIterator* index_iter); - static void SetupCacheKeyPrefix(Rep* rep); + // Create the filter from the filter block. + FilterBlockReader* ReadFilter(FilePrefetchBuffer* prefetch_buffer, + const BlockHandle& filter_handle, + const bool is_a_filter_partition) const; - explicit BlockBasedTable(Rep* rep) - : rep_(rep), compaction_optimized_(false) {} + static void SetupCacheKeyPrefix(Rep* rep, uint64_t file_size); // Generate a cache key prefix from the file static void GenerateCachePrefix(Cache* cc, @@ -196,13 +336,151 @@ class BlockBasedTable : public TableReader { static void GenerateCachePrefix(Cache* cc, WritableFile* file, char* buffer, size_t* size); - // The longest prefix of the cache key used to identify blocks. - // For Posix files the unique ID is three varints. - static const size_t kMaxCacheKeyPrefixSize = kMaxVarint64Length*3+1; + // Helper functions for DumpTable() + Status DumpIndexBlock(WritableFile* out_file); + Status DumpDataBlocks(WritableFile* out_file); + void DumpKeyValue(const Slice& key, const Slice& value, + WritableFile* out_file); // No copying allowed explicit BlockBasedTable(const TableReader&) = delete; void operator=(const TableReader&) = delete; + + friend class PartitionedFilterBlockReader; + friend class PartitionedFilterBlockTest; +}; + +// Maitaning state of a two-level iteration on a partitioned index structure +class BlockBasedTable::BlockEntryIteratorState : public TwoLevelIteratorState { + public: + BlockEntryIteratorState( + BlockBasedTable* table, const ReadOptions& read_options, + const InternalKeyComparator* icomparator, bool skip_filters, + bool is_index = false, + std::unordered_map>* block_map = nullptr); + InternalIterator* NewSecondaryIterator(const Slice& index_value) override; + bool PrefixMayMatch(const Slice& internal_key) override; + bool KeyReachedUpperBound(const Slice& internal_key) override; + + private: + // Don't own table_ + BlockBasedTable* table_; + const ReadOptions read_options_; + const InternalKeyComparator* icomparator_; + bool skip_filters_; + // true if the 2nd level iterator is on indexes instead of on user data. + bool is_index_; + std::unordered_map>* block_map_; + port::RWMutex cleaner_mu; +}; + +// CachableEntry represents the entries that *may* be fetched from block cache. +// field `value` is the item we want to get. +// field `cache_handle` is the cache handle to the block cache. If the value +// was not read from cache, `cache_handle` will be nullptr. +template +struct BlockBasedTable::CachableEntry { + CachableEntry(TValue* _value, Cache::Handle* _cache_handle) + : value(_value), cache_handle(_cache_handle) {} + CachableEntry() : CachableEntry(nullptr, nullptr) {} + void Release(Cache* cache, bool force_erase = false) { + if (cache_handle) { + cache->Release(cache_handle, force_erase); + value = nullptr; + cache_handle = nullptr; + } + } + bool IsSet() const { return cache_handle != nullptr; } + + TValue* value = nullptr; + // if the entry is from the cache, cache_handle will be populated. + Cache::Handle* cache_handle = nullptr; +}; + +struct BlockBasedTable::Rep { + Rep(const ImmutableCFOptions& _ioptions, const EnvOptions& _env_options, + const BlockBasedTableOptions& _table_opt, + const InternalKeyComparator& _internal_comparator, bool skip_filters) + : ioptions(_ioptions), + env_options(_env_options), + table_options(_table_opt), + filter_policy(skip_filters ? nullptr : _table_opt.filter_policy.get()), + internal_comparator(_internal_comparator), + filter_type(FilterType::kNoFilter), + whole_key_filtering(_table_opt.whole_key_filtering), + prefix_filtering(true), + range_del_handle(BlockHandle::NullBlockHandle()), + global_seqno(kDisableGlobalSequenceNumber) {} + + const ImmutableCFOptions& ioptions; + const EnvOptions& env_options; + const BlockBasedTableOptions& table_options; + const FilterPolicy* const filter_policy; + const InternalKeyComparator& internal_comparator; + Status status; + unique_ptr file; + char cache_key_prefix[kMaxCacheKeyPrefixSize]; + size_t cache_key_prefix_size = 0; + char persistent_cache_key_prefix[kMaxCacheKeyPrefixSize]; + size_t persistent_cache_key_prefix_size = 0; + char compressed_cache_key_prefix[kMaxCacheKeyPrefixSize]; + size_t compressed_cache_key_prefix_size = 0; + uint64_t dummy_index_reader_offset = + 0; // ID that is unique for the block cache. + PersistentCacheOptions persistent_cache_options; + + // Footer contains the fixed table information + Footer footer; + // index_reader and filter will be populated and used only when + // options.block_cache is nullptr; otherwise we will get the index block via + // the block cache. + unique_ptr index_reader; + unique_ptr filter; + + enum class FilterType { + kNoFilter, + kFullFilter, + kBlockFilter, + kPartitionedFilter, + }; + FilterType filter_type; + BlockHandle filter_handle; + + std::shared_ptr table_properties; + // Block containing the data for the compression dictionary. We take ownership + // for the entire block struct, even though we only use its Slice member. This + // is easier because the Slice member depends on the continued existence of + // another member ("allocation"). + std::unique_ptr compression_dict_block; + BlockBasedTableOptions::IndexType index_type; + bool hash_index_allow_collision; + bool whole_key_filtering; + bool prefix_filtering; + // TODO(kailiu) It is very ugly to use internal key in table, since table + // module should not be relying on db module. However to make things easier + // and compatible with existing code, we introduce a wrapper that allows + // block to extract prefix without knowing if a key is internal or not. + unique_ptr internal_prefix_transform; + + // only used in level 0 files: + // when pin_l0_filter_and_index_blocks_in_cache is true, we do use the + // LRU cache, but we always keep the filter & idndex block's handle checked + // out here (=we don't call Release()), plus the parsed out objects + // the LRU cache will never push flush them out, hence they're pinned + CachableEntry filter_entry; + CachableEntry index_entry; + // range deletion meta-block is pinned through reader's lifetime when LRU + // cache is enabled. + CachableEntry range_del_entry; + BlockHandle range_del_handle; + + // If global_seqno is used, all Keys in this file will have the same + // seqno with value `global_seqno`. + // + // A value of kDisableGlobalSequenceNumber means that this feature is disabled + // and every key have it's own seqno. + SequenceNumber global_seqno; + bool closed = false; }; } // namespace rocksdb diff --git a/table/block_builder.cc b/table/block_builder.cc index f8627743a02..39bfffe5118 100644 --- a/table/block_builder.cc +++ b/table/block_builder.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -41,30 +41,27 @@ namespace rocksdb { -BlockBuilder::BlockBuilder(int block_restart_interval) +BlockBuilder::BlockBuilder(int block_restart_interval, bool use_delta_encoding) : block_restart_interval_(block_restart_interval), + use_delta_encoding_(use_delta_encoding), restarts_(), counter_(0), finished_(false) { assert(block_restart_interval_ >= 1); restarts_.push_back(0); // First restart point is at offset 0 + estimate_ = sizeof(uint32_t) + sizeof(uint32_t); } void BlockBuilder::Reset() { buffer_.clear(); restarts_.clear(); restarts_.push_back(0); // First restart point is at offset 0 + estimate_ = sizeof(uint32_t) + sizeof(uint32_t); counter_ = 0; finished_ = false; last_key_.clear(); } -size_t BlockBuilder::CurrentSizeEstimate() const { - return (buffer_.size() + // Raw data buffer - restarts_.size() * sizeof(uint32_t) + // Restart array - sizeof(uint32_t)); // Restart array length -} - size_t BlockBuilder::EstimateSizeAfterKV(const Slice& key, const Slice& value) const { size_t estimate = CurrentSizeEstimate(); @@ -85,43 +82,50 @@ Slice BlockBuilder::Finish() { for (size_t i = 0; i < restarts_.size(); i++) { PutFixed32(&buffer_, restarts_[i]); } - PutFixed32(&buffer_, restarts_.size()); + PutFixed32(&buffer_, static_cast(restarts_.size())); finished_ = true; return Slice(buffer_); } void BlockBuilder::Add(const Slice& key, const Slice& value) { - Slice last_key_piece(last_key_); assert(!finished_); assert(counter_ <= block_restart_interval_); - size_t shared = 0; - if (counter_ < block_restart_interval_) { - // See how much sharing to do with previous string - const size_t min_length = std::min(last_key_piece.size(), key.size()); - while ((shared < min_length) && (last_key_piece[shared] == key[shared])) { - shared++; - } - } else { + size_t shared = 0; // number of bytes shared with prev key + if (counter_ >= block_restart_interval_) { // Restart compression - restarts_.push_back(buffer_.size()); + restarts_.push_back(static_cast(buffer_.size())); + estimate_ += sizeof(uint32_t); counter_ = 0; + + if (use_delta_encoding_) { + // Update state + last_key_.assign(key.data(), key.size()); + } + } else if (use_delta_encoding_) { + Slice last_key_piece(last_key_); + // See how much sharing to do with previous string + shared = key.difference_offset(last_key_piece); + + // Update state + // We used to just copy the changed data here, but it appears to be + // faster to just copy the whole thing. + last_key_.assign(key.data(), key.size()); } + const size_t non_shared = key.size() - shared; + const size_t curr_size = buffer_.size(); // Add "" to buffer_ - PutVarint32(&buffer_, shared); - PutVarint32(&buffer_, non_shared); - PutVarint32(&buffer_, value.size()); + PutVarint32Varint32Varint32(&buffer_, static_cast(shared), + static_cast(non_shared), + static_cast(value.size())); // Add string delta to buffer_ followed by value buffer_.append(key.data() + shared, non_shared); buffer_.append(value.data(), value.size()); - // Update state - last_key_.resize(shared); - last_key_.append(key.data() + shared, non_shared); - assert(Slice(last_key_) == key); counter_++; + estimate_ += buffer_.size() - curr_size; } } // namespace rocksdb diff --git a/table/block_builder.h b/table/block_builder.h index 3b5b2b44449..6b5297d0410 100644 --- a/table/block_builder.h +++ b/table/block_builder.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -15,19 +15,18 @@ namespace rocksdb { -class Comparator; - class BlockBuilder { public: BlockBuilder(const BlockBuilder&) = delete; void operator=(const BlockBuilder&) = delete; - - explicit BlockBuilder(int block_restart_interval); - + + explicit BlockBuilder(int block_restart_interval, + bool use_delta_encoding = true); + // Reset the contents as if the BlockBuilder was just constructed. void Reset(); - // REQUIRES: Finish() has not been callled since the last call to Reset(). + // REQUIRES: Finish() has not been called since the last call to Reset(). // REQUIRES: key is larger than any previously added key void Add(const Slice& key, const Slice& value); @@ -38,7 +37,7 @@ class BlockBuilder { // Returns an estimate of the current (uncompressed) size of the block // we are building. - size_t CurrentSizeEstimate() const; + inline size_t CurrentSizeEstimate() const { return estimate_; } // Returns an estimated block size after appending key and value. size_t EstimateSizeAfterKV(const Slice& key, const Slice& value) const; @@ -50,9 +49,11 @@ class BlockBuilder { private: const int block_restart_interval_; + const bool use_delta_encoding_; std::string buffer_; // Destination buffer std::vector restarts_; // Restart points + size_t estimate_; int counter_; // Number of entries emitted since restart bool finished_; // Has Finish() been called? std::string last_key_; diff --git a/table/block_hash_index.cc b/table/block_hash_index.cc deleted file mode 100644 index 7a6e219a0a6..00000000000 --- a/table/block_hash_index.cc +++ /dev/null @@ -1,157 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -#include "table/block_hash_index.h" - -#include - -#include "rocksdb/comparator.h" -#include "rocksdb/iterator.h" -#include "rocksdb/slice_transform.h" -#include "util/coding.h" - -namespace rocksdb { - -Status CreateBlockHashIndex(const SliceTransform* hash_key_extractor, - const Slice& prefixes, const Slice& prefix_meta, - BlockHashIndex** hash_index) { - uint64_t pos = 0; - auto meta_pos = prefix_meta; - Status s; - *hash_index = new BlockHashIndex( - hash_key_extractor, - false /* external module manages memory space for prefixes */); - - while (!meta_pos.empty()) { - uint32_t prefix_size = 0; - uint32_t entry_index = 0; - uint32_t num_blocks = 0; - if (!GetVarint32(&meta_pos, &prefix_size) || - !GetVarint32(&meta_pos, &entry_index) || - !GetVarint32(&meta_pos, &num_blocks)) { - s = Status::Corruption( - "Corrupted prefix meta block: unable to read from it."); - break; - } - Slice prefix(prefixes.data() + pos, prefix_size); - (*hash_index)->Add(prefix, entry_index, num_blocks); - - pos += prefix_size; - } - - if (s.ok() && pos != prefixes.size()) { - s = Status::Corruption("Corrupted prefix meta block"); - } - - if (!s.ok()) { - delete *hash_index; - } - - return s; -} - -BlockHashIndex* CreateBlockHashIndexOnTheFly( - Iterator* index_iter, Iterator* data_iter, const uint32_t num_restarts, - const Comparator* comparator, const SliceTransform* hash_key_extractor) { - assert(hash_key_extractor); - auto hash_index = new BlockHashIndex( - hash_key_extractor, - true /* hash_index will copy prefix when Add() is called */); - uint64_t current_restart_index = 0; - - std::string pending_entry_prefix; - // pending_block_num == 0 also implies there is no entry inserted at all. - uint32_t pending_block_num = 0; - uint32_t pending_entry_index = 0; - - // scan all the entries and create a hash index based on their prefixes. - data_iter->SeekToFirst(); - for (index_iter->SeekToFirst(); - index_iter->Valid() && current_restart_index < num_restarts; - index_iter->Next()) { - Slice last_key_in_block = index_iter->key(); - assert(data_iter->Valid() && data_iter->status().ok()); - - // scan through all entries within a data block. - while (data_iter->Valid() && - comparator->Compare(data_iter->key(), last_key_in_block) <= 0) { - auto key_prefix = hash_key_extractor->Transform(data_iter->key()); - bool is_first_entry = pending_block_num == 0; - - // Keys may share the prefix - if (is_first_entry || pending_entry_prefix != key_prefix) { - if (!is_first_entry) { - bool succeeded = hash_index->Add( - pending_entry_prefix, pending_entry_index, pending_block_num); - if (!succeeded) { - delete hash_index; - return nullptr; - } - } - - // update the status. - // needs a hard copy otherwise the underlying data changes all the time. - pending_entry_prefix = key_prefix.ToString(); - pending_block_num = 1; - pending_entry_index = current_restart_index; - } else { - // entry number increments when keys share the prefix reside in - // differnt data blocks. - auto last_restart_index = pending_entry_index + pending_block_num - 1; - assert(last_restart_index <= current_restart_index); - if (last_restart_index != current_restart_index) { - ++pending_block_num; - } - } - data_iter->Next(); - } - - ++current_restart_index; - } - - // make sure all entries has been scaned. - assert(!index_iter->Valid()); - assert(!data_iter->Valid()); - - if (pending_block_num > 0) { - auto succeeded = hash_index->Add(pending_entry_prefix, pending_entry_index, - pending_block_num); - if (!succeeded) { - delete hash_index; - return nullptr; - } - } - - return hash_index; -} - -bool BlockHashIndex::Add(const Slice& prefix, uint32_t restart_index, - uint32_t num_blocks) { - auto prefix_to_insert = prefix; - if (kOwnPrefixes) { - auto prefix_ptr = arena_.Allocate(prefix.size()); - std::copy(prefix.data() /* begin */, - prefix.data() + prefix.size() /* end */, - prefix_ptr /* destination */); - prefix_to_insert = Slice(prefix_ptr, prefix.size()); - } - auto result = restart_indices_.insert( - {prefix_to_insert, RestartIndex(restart_index, num_blocks)}); - return result.second; -} - -const BlockHashIndex::RestartIndex* BlockHashIndex::GetRestartIndex( - const Slice& key) { - auto key_prefix = hash_key_extractor_->Transform(key); - - auto pos = restart_indices_.find(key_prefix); - if (pos == restart_indices_.end()) { - return nullptr; - } - - return &pos->second; -} - -} // namespace rocksdb diff --git a/table/block_hash_index.h b/table/block_hash_index.h deleted file mode 100644 index d5603d3660c..00000000000 --- a/table/block_hash_index.h +++ /dev/null @@ -1,85 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -#pragma once - -#include -#include - -#include "rocksdb/status.h" -#include "util/arena.h" -#include "util/murmurhash.h" - -namespace rocksdb { - -class Comparator; -class Iterator; -class Slice; -class SliceTransform; - -// Build a hash-based index to speed up the lookup for "index block". -// BlockHashIndex accepts a key and, if found, returns its restart index within -// that index block. -class BlockHashIndex { - public: - // Represents a restart index in the index block's restart array. - struct RestartIndex { - explicit RestartIndex(uint32_t first_index, uint32_t num_blocks = 1) - : first_index(first_index), num_blocks(num_blocks) {} - - // For a given prefix, what is the restart index for the first data block - // that contains it. - uint32_t first_index = 0; - - // How many data blocks contains this prefix? - uint32_t num_blocks = 1; - }; - - // @params own_prefixes indicate if we should take care the memory space for - // the `key_prefix` - // passed by Add() - explicit BlockHashIndex(const SliceTransform* hash_key_extractor, - bool own_prefixes) - : hash_key_extractor_(hash_key_extractor), kOwnPrefixes(own_prefixes) {} - - // Maps a key to its restart first_index. - // Returns nullptr if the restart first_index is found - const RestartIndex* GetRestartIndex(const Slice& key); - - bool Add(const Slice& key_prefix, uint32_t restart_index, - uint32_t num_blocks); - - size_t ApproximateMemoryUsage() const { - return arena_.ApproximateMemoryUsage(); - } - - private: - const SliceTransform* hash_key_extractor_; - std::unordered_map restart_indices_; - - Arena arena_; - bool kOwnPrefixes; -}; - -// Create hash index by reading from the metadata blocks. -// @params prefixes: a sequence of prefixes. -// @params prefix_meta: contains the "metadata" to of the prefixes. -Status CreateBlockHashIndex(const SliceTransform* hash_key_extractor, - const Slice& prefixes, const Slice& prefix_meta, - BlockHashIndex** hash_index); - -// Create hash index by scanning the entries in index as well as the whole -// dataset. -// @params index_iter: an iterator with the pointer to the first entry in a -// block. -// @params data_iter: an iterator that can scan all the entries reside in a -// table. -// @params num_restarts: used for correctness verification. -// @params hash_key_extractor: extract the hashable part of a given key. -// On error, nullptr will be returned. -BlockHashIndex* CreateBlockHashIndexOnTheFly( - Iterator* index_iter, Iterator* data_iter, const uint32_t num_restarts, - const Comparator* comparator, const SliceTransform* hash_key_extractor); - -} // namespace rocksdb diff --git a/table/block_hash_index_test.cc b/table/block_hash_index_test.cc deleted file mode 100644 index 6f7bcb2b763..00000000000 --- a/table/block_hash_index_test.cc +++ /dev/null @@ -1,117 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -#include -#include -#include - -#include "rocksdb/comparator.h" -#include "rocksdb/iterator.h" -#include "rocksdb/slice_transform.h" -#include "table/block_hash_index.h" -#include "util/testharness.h" -#include "util/testutil.h" - -namespace rocksdb { - -typedef std::map Data; - -class MapIterator : public Iterator { - public: - explicit MapIterator(const Data& data) : data_(data), pos_(data_.end()) {} - - virtual bool Valid() const { return pos_ != data_.end(); } - - virtual void SeekToFirst() { pos_ = data_.begin(); } - - virtual void SeekToLast() { - pos_ = data_.end(); - --pos_; - } - - virtual void Seek(const Slice& target) { - pos_ = data_.find(target.ToString()); - } - - virtual void Next() { ++pos_; } - - virtual void Prev() { --pos_; } - - virtual Slice key() const { return pos_->first; } - - virtual Slice value() const { return pos_->second; } - - virtual Status status() const { return Status::OK(); } - - private: - const Data& data_; - Data::const_iterator pos_; -}; - -class BlockTest {}; - -TEST(BlockTest, BasicTest) { - const size_t keys_per_block = 4; - const size_t prefix_size = 2; - std::vector keys = {/* block 1 */ - "0101", "0102", "0103", "0201", - /* block 2 */ - "0202", "0203", "0301", "0401", - /* block 3 */ - "0501", "0601", "0701", "0801", - /* block 4 */ - "0802", "0803", "0804", "0805", - /* block 5 */ - "0806", "0807", "0808", "0809", }; - - Data data_entries; - for (const auto key : keys) { - data_entries.insert({key, key}); - } - - Data index_entries; - for (size_t i = 3; i < keys.size(); i += keys_per_block) { - // simply ignore the value part - index_entries.insert({keys[i], ""}); - } - - MapIterator data_iter(data_entries); - MapIterator index_iter(index_entries); - - auto prefix_extractor = NewFixedPrefixTransform(prefix_size); - std::unique_ptr block_hash_index(CreateBlockHashIndexOnTheFly( - &index_iter, &data_iter, index_entries.size(), BytewiseComparator(), - prefix_extractor)); - - std::map expected = { - {"01xx", BlockHashIndex::RestartIndex(0, 1)}, - {"02yy", BlockHashIndex::RestartIndex(0, 2)}, - {"03zz", BlockHashIndex::RestartIndex(1, 1)}, - {"04pp", BlockHashIndex::RestartIndex(1, 1)}, - {"05ww", BlockHashIndex::RestartIndex(2, 1)}, - {"06xx", BlockHashIndex::RestartIndex(2, 1)}, - {"07pp", BlockHashIndex::RestartIndex(2, 1)}, - {"08xz", BlockHashIndex::RestartIndex(2, 3)}, }; - - const BlockHashIndex::RestartIndex* index = nullptr; - // search existed prefixes - for (const auto& item : expected) { - index = block_hash_index->GetRestartIndex(item.first); - ASSERT_TRUE(index != nullptr); - ASSERT_EQ(item.second.first_index, index->first_index); - ASSERT_EQ(item.second.num_blocks, index->num_blocks); - } - - // search non exist prefixes - ASSERT_TRUE(!block_hash_index->GetRestartIndex("00xx")); - ASSERT_TRUE(!block_hash_index->GetRestartIndex("10yy")); - ASSERT_TRUE(!block_hash_index->GetRestartIndex("20zz")); - - delete prefix_extractor; -} - -} // namespace rocksdb - -int main(int argc, char** argv) { return rocksdb::test::RunAllTests(); } diff --git a/table/block_prefix_index.cc b/table/block_prefix_index.cc index f06dcd9fe76..df37b5fc2b3 100644 --- a/table/block_prefix_index.cc +++ b/table/block_prefix_index.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #include "table/block_prefix_index.h" @@ -87,7 +87,7 @@ class BlockPrefixIndex::Builder { BlockPrefixIndex* Finish() { // For now, use roughly 1:1 prefix to bucket ratio. - uint32_t num_buckets = prefixes_.size() + 1; + uint32_t num_buckets = static_cast(prefixes_.size()) + 1; // Collect prefix records that hash to the same bucket, into a single // linklist. @@ -136,6 +136,7 @@ class BlockPrefixIndex::Builder { assert(prefixes_per_bucket[i]->next == nullptr); buckets[i] = prefixes_per_bucket[i]->start_block; } else { + assert(total_block_array_entries > 0); assert(prefixes_per_bucket[i] != nullptr); buckets[i] = EncodeIndex(offset); block_array_buffer[offset] = num_blocks; @@ -143,8 +144,8 @@ class BlockPrefixIndex::Builder { auto current = prefixes_per_bucket[i]; // populate block ids from largest to smallest while (current != nullptr) { - for (uint32_t i = 0; i < current->num_blocks; i++) { - *last_block = current->end_block - i; + for (uint32_t iter = 0; iter < current->num_blocks; iter++) { + *last_block = current->end_block - iter; last_block--; } current = current->next; @@ -210,8 +211,8 @@ Status BlockPrefixIndex::Create(const SliceTransform* internal_prefix_extractor, return s; } -const uint32_t BlockPrefixIndex::GetBlocks(const Slice& key, - uint32_t** blocks) { +uint32_t BlockPrefixIndex::GetBlocks(const Slice& key, + uint32_t** blocks) { Slice prefix = internal_prefix_extractor_->Transform(key); uint32_t bucket = PrefixToBucket(prefix, num_buckets_); diff --git a/table/block_prefix_index.h b/table/block_prefix_index.h index 2afecadd268..dd4282d17b8 100644 --- a/table/block_prefix_index.h +++ b/table/block_prefix_index.h @@ -1,9 +1,10 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once +#include #include "rocksdb/status.h" namespace rocksdb { @@ -23,7 +24,7 @@ class BlockPrefixIndex { // the key, based on the prefix. // Returns the total number of relevant blocks, 0 means the key does // not exist. - const uint32_t GetBlocks(const Slice& key, uint32_t** blocks); + uint32_t GetBlocks(const Slice& key, uint32_t** blocks); size_t ApproximateMemoryUsage() const { return sizeof(BlockPrefixIndex) + diff --git a/table/block_test.cc b/table/block_test.cc index b36787f8f53..f5c543975f4 100644 --- a/table/block_test.cc +++ b/table/block_test.cc @@ -1,15 +1,19 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #include +#include +#include #include +#include +#include #include #include "db/dbformat.h" -#include "db/memtable.h" #include "db/write_batch_internal.h" +#include "db/memtable.h" #include "rocksdb/db.h" #include "rocksdb/env.h" #include "rocksdb/iterator.h" @@ -18,7 +22,6 @@ #include "table/block.h" #include "table/block_builder.h" #include "table/format.h" -#include "table/block_hash_index.h" #include "util/random.h" #include "util/testharness.h" #include "util/testutil.h" @@ -65,10 +68,10 @@ void GenerateRandomKVs(std::vector *keys, } } -class BlockTest {}; +class BlockTest : public testing::Test {}; // block test -TEST(BlockTest, SimpleTest) { +TEST_F(BlockTest, SimpleTest) { Random rnd(301); Options options = Options(); std::unique_ptr ic; @@ -92,12 +95,11 @@ TEST(BlockTest, SimpleTest) { BlockContents contents; contents.data = rawblock; contents.cachable = false; - contents.heap_allocated = false; - Block reader(contents); + Block reader(std::move(contents), kDisableGlobalSequenceNumber); // read contents of block sequentially int count = 0; - Iterator* iter = reader.NewIterator(options.comparator); + InternalIterator *iter = reader.NewIterator(options.comparator); for (iter->SeekToFirst();iter->Valid(); count++, iter->Next()) { // read kv from block @@ -143,7 +145,6 @@ BlockContents GetBlockContents(std::unique_ptr *builder, BlockContents contents; contents.data = rawblock; contents.cachable = false; - contents.heap_allocated = false; return contents; } @@ -153,36 +154,24 @@ void CheckBlockContents(BlockContents contents, const int max_key, const std::vector &values) { const size_t prefix_size = 6; // create block reader - Block reader1(contents); - Block reader2(contents); + BlockContents contents_ref(contents.data, contents.cachable, + contents.compression_type); + Block reader1(std::move(contents), kDisableGlobalSequenceNumber); + Block reader2(std::move(contents_ref), kDisableGlobalSequenceNumber); std::unique_ptr prefix_extractor( NewFixedPrefixTransform(prefix_size)); - { - auto iter1 = reader1.NewIterator(nullptr); - auto iter2 = reader1.NewIterator(nullptr); - reader1.SetBlockHashIndex(CreateBlockHashIndexOnTheFly( - iter1, iter2, keys.size(), BytewiseComparator(), - prefix_extractor.get())); - - delete iter1; - delete iter2; - } - - std::unique_ptr hash_iter( - reader1.NewIterator(BytewiseComparator(), nullptr, false)); - - std::unique_ptr regular_iter( + std::unique_ptr regular_iter( reader2.NewIterator(BytewiseComparator())); // Seek existent keys for (size_t i = 0; i < keys.size(); i++) { - hash_iter->Seek(keys[i]); - ASSERT_OK(hash_iter->status()); - ASSERT_TRUE(hash_iter->Valid()); + regular_iter->Seek(keys[i]); + ASSERT_OK(regular_iter->status()); + ASSERT_TRUE(regular_iter->Valid()); - Slice v = hash_iter->value(); + Slice v = regular_iter->value(); ASSERT_EQ(v.ToString().compare(values[i]), 0); } @@ -192,16 +181,13 @@ void CheckBlockContents(BlockContents contents, const int max_key, // return the one that is closest. for (int i = 1; i < max_key - 1; i += 2) { auto key = GenerateKey(i, 0, 0, nullptr); - hash_iter->Seek(key); - ASSERT_TRUE(!hash_iter->Valid()); - regular_iter->Seek(key); ASSERT_TRUE(regular_iter->Valid()); } } // In this test case, no two key share same prefix. -TEST(BlockTest, SimpleIndexHash) { +TEST_F(BlockTest, SimpleIndexHash) { const int kMaxKey = 100000; std::vector keys; std::vector values; @@ -212,10 +198,10 @@ TEST(BlockTest, SimpleIndexHash) { std::unique_ptr builder; auto contents = GetBlockContents(&builder, keys, values); - CheckBlockContents(contents, kMaxKey, keys, values); + CheckBlockContents(std::move(contents), kMaxKey, keys, values); } -TEST(BlockTest, IndexHashWithSharedPrefix) { +TEST_F(BlockTest, IndexHashWithSharedPrefix) { const int kMaxKey = 100000; // for each prefix, there will be 5 keys starts with it. const int kPrefixGroup = 5; @@ -231,11 +217,271 @@ TEST(BlockTest, IndexHashWithSharedPrefix) { std::unique_ptr builder; auto contents = GetBlockContents(&builder, keys, values, kPrefixGroup); - CheckBlockContents(contents, kMaxKey, keys, values); + CheckBlockContents(std::move(contents), kMaxKey, keys, values); +} + +// A slow and accurate version of BlockReadAmpBitmap that simply store +// all the marked ranges in a set. +class BlockReadAmpBitmapSlowAndAccurate { + public: + void Mark(size_t start_offset, size_t end_offset) { + assert(end_offset >= start_offset); + marked_ranges_.emplace(end_offset, start_offset); + } + + // Return true if any byte in this range was Marked + bool IsPinMarked(size_t offset) { + auto it = marked_ranges_.lower_bound( + std::make_pair(offset, static_cast(0))); + if (it == marked_ranges_.end()) { + return false; + } + return offset <= it->first && offset >= it->second; + } + + private: + std::set> marked_ranges_; +}; + +TEST_F(BlockTest, BlockReadAmpBitmap) { + uint32_t pin_offset = 0; + SyncPoint::GetInstance()->SetCallBack( + "BlockReadAmpBitmap:rnd", [&pin_offset](void* arg) { + pin_offset = *(static_cast(arg)); + }); + SyncPoint::GetInstance()->EnableProcessing(); + std::vector block_sizes = { + 1, // 1 byte + 32, // 32 bytes + 61, // 61 bytes + 64, // 64 bytes + 512, // 0.5 KB + 1024, // 1 KB + 1024 * 4, // 4 KB + 1024 * 10, // 10 KB + 1024 * 50, // 50 KB + 1024 * 1024, // 1 MB + 1024 * 1024 * 4, // 4 MB + 1024 * 1024 * 50, // 10 MB + 777, + 124653, + }; + const size_t kBytesPerBit = 64; + + Random rnd(301); + for (size_t block_size : block_sizes) { + std::shared_ptr stats = rocksdb::CreateDBStatistics(); + BlockReadAmpBitmap read_amp_bitmap(block_size, kBytesPerBit, stats.get()); + BlockReadAmpBitmapSlowAndAccurate read_amp_slow_and_accurate; + + size_t needed_bits = (block_size / kBytesPerBit); + if (block_size % kBytesPerBit != 0) { + needed_bits++; + } + size_t bitmap_size = needed_bits / 32; + if (needed_bits % 32 != 0) { + bitmap_size++; + } + + ASSERT_EQ(stats->getTickerCount(READ_AMP_TOTAL_READ_BYTES), block_size); + + // Generate some random entries + std::vector random_entry_offsets; + for (int i = 0; i < 1000; i++) { + random_entry_offsets.push_back(rnd.Next() % block_size); + } + std::sort(random_entry_offsets.begin(), random_entry_offsets.end()); + auto it = + std::unique(random_entry_offsets.begin(), random_entry_offsets.end()); + random_entry_offsets.resize( + std::distance(random_entry_offsets.begin(), it)); + + std::vector> random_entries; + for (size_t i = 0; i < random_entry_offsets.size(); i++) { + size_t entry_start = random_entry_offsets[i]; + size_t entry_end; + if (i + 1 < random_entry_offsets.size()) { + entry_end = random_entry_offsets[i + 1] - 1; + } else { + entry_end = block_size - 1; + } + random_entries.emplace_back(entry_start, entry_end); + } + + for (size_t i = 0; i < random_entries.size(); i++) { + auto ¤t_entry = random_entries[rnd.Next() % random_entries.size()]; + + read_amp_bitmap.Mark(static_cast(current_entry.first), + static_cast(current_entry.second)); + read_amp_slow_and_accurate.Mark(current_entry.first, + current_entry.second); + + size_t total_bits = 0; + for (size_t bit_idx = 0; bit_idx < needed_bits; bit_idx++) { + total_bits += read_amp_slow_and_accurate.IsPinMarked( + bit_idx * kBytesPerBit + pin_offset); + } + size_t expected_estimate_useful = total_bits * kBytesPerBit; + size_t got_estimate_useful = + stats->getTickerCount(READ_AMP_ESTIMATE_USEFUL_BYTES); + ASSERT_EQ(expected_estimate_useful, got_estimate_useful); + } + } + SyncPoint::GetInstance()->DisableProcessing(); + SyncPoint::GetInstance()->ClearAllCallBacks(); +} + +TEST_F(BlockTest, BlockWithReadAmpBitmap) { + Random rnd(301); + Options options = Options(); + std::unique_ptr ic; + ic.reset(new test::PlainInternalKeyComparator(options.comparator)); + + std::vector keys; + std::vector values; + BlockBuilder builder(16); + int num_records = 10000; + + GenerateRandomKVs(&keys, &values, 0, num_records, 1); + // add a bunch of records to a block + for (int i = 0; i < num_records; i++) { + builder.Add(keys[i], values[i]); + } + + Slice rawblock = builder.Finish(); + const size_t kBytesPerBit = 8; + + // Read the block sequentially using Next() + { + std::shared_ptr stats = rocksdb::CreateDBStatistics(); + + // create block reader + BlockContents contents; + contents.data = rawblock; + contents.cachable = true; + Block reader(std::move(contents), kDisableGlobalSequenceNumber, + kBytesPerBit, stats.get()); + + // read contents of block sequentially + size_t read_bytes = 0; + BlockIter *iter = static_cast( + reader.NewIterator(options.comparator, nullptr, true, stats.get())); + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + iter->value(); + read_bytes += iter->TEST_CurrentEntrySize(); + + double semi_acc_read_amp = + static_cast(read_bytes) / rawblock.size(); + double read_amp = static_cast(stats->getTickerCount( + READ_AMP_ESTIMATE_USEFUL_BYTES)) / + stats->getTickerCount(READ_AMP_TOTAL_READ_BYTES); + + // Error in read amplification will be less than 1% if we are reading + // sequentially + double error_pct = fabs(semi_acc_read_amp - read_amp) * 100; + EXPECT_LT(error_pct, 1); + } + + delete iter; + } + + // Read the block sequentially using Seek() + { + std::shared_ptr stats = rocksdb::CreateDBStatistics(); + + // create block reader + BlockContents contents; + contents.data = rawblock; + contents.cachable = true; + Block reader(std::move(contents), kDisableGlobalSequenceNumber, + kBytesPerBit, stats.get()); + + size_t read_bytes = 0; + BlockIter *iter = static_cast( + reader.NewIterator(options.comparator, nullptr, true, stats.get())); + for (int i = 0; i < num_records; i++) { + Slice k(keys[i]); + + // search in block for this key + iter->Seek(k); + iter->value(); + read_bytes += iter->TEST_CurrentEntrySize(); + + double semi_acc_read_amp = + static_cast(read_bytes) / rawblock.size(); + double read_amp = static_cast(stats->getTickerCount( + READ_AMP_ESTIMATE_USEFUL_BYTES)) / + stats->getTickerCount(READ_AMP_TOTAL_READ_BYTES); + + // Error in read amplification will be less than 1% if we are reading + // sequentially + double error_pct = fabs(semi_acc_read_amp - read_amp) * 100; + EXPECT_LT(error_pct, 1); + } + delete iter; + } + + // Read the block randomly + { + std::shared_ptr stats = rocksdb::CreateDBStatistics(); + + // create block reader + BlockContents contents; + contents.data = rawblock; + contents.cachable = true; + Block reader(std::move(contents), kDisableGlobalSequenceNumber, + kBytesPerBit, stats.get()); + + size_t read_bytes = 0; + BlockIter *iter = static_cast( + reader.NewIterator(options.comparator, nullptr, true, stats.get())); + std::unordered_set read_keys; + for (int i = 0; i < num_records; i++) { + int index = rnd.Uniform(num_records); + Slice k(keys[index]); + + iter->Seek(k); + iter->value(); + if (read_keys.find(index) == read_keys.end()) { + read_keys.insert(index); + read_bytes += iter->TEST_CurrentEntrySize(); + } + + double semi_acc_read_amp = + static_cast(read_bytes) / rawblock.size(); + double read_amp = static_cast(stats->getTickerCount( + READ_AMP_ESTIMATE_USEFUL_BYTES)) / + stats->getTickerCount(READ_AMP_TOTAL_READ_BYTES); + + double error_pct = fabs(semi_acc_read_amp - read_amp) * 100; + // Error in read amplification will be less than 2% if we are reading + // randomly + EXPECT_LT(error_pct, 2); + } + delete iter; + } +} + +TEST_F(BlockTest, ReadAmpBitmapPow2) { + std::shared_ptr stats = rocksdb::CreateDBStatistics(); + ASSERT_EQ(BlockReadAmpBitmap(100, 1, stats.get()).GetBytesPerBit(), 1); + ASSERT_EQ(BlockReadAmpBitmap(100, 2, stats.get()).GetBytesPerBit(), 2); + ASSERT_EQ(BlockReadAmpBitmap(100, 4, stats.get()).GetBytesPerBit(), 4); + ASSERT_EQ(BlockReadAmpBitmap(100, 8, stats.get()).GetBytesPerBit(), 8); + ASSERT_EQ(BlockReadAmpBitmap(100, 16, stats.get()).GetBytesPerBit(), 16); + ASSERT_EQ(BlockReadAmpBitmap(100, 32, stats.get()).GetBytesPerBit(), 32); + + ASSERT_EQ(BlockReadAmpBitmap(100, 3, stats.get()).GetBytesPerBit(), 2); + ASSERT_EQ(BlockReadAmpBitmap(100, 7, stats.get()).GetBytesPerBit(), 4); + ASSERT_EQ(BlockReadAmpBitmap(100, 11, stats.get()).GetBytesPerBit(), 8); + ASSERT_EQ(BlockReadAmpBitmap(100, 17, stats.get()).GetBytesPerBit(), 16); + ASSERT_EQ(BlockReadAmpBitmap(100, 33, stats.get()).GetBytesPerBit(), 32); + ASSERT_EQ(BlockReadAmpBitmap(100, 35, stats.get()).GetBytesPerBit(), 32); } } // namespace rocksdb -int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); +int main(int argc, char **argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/table/bloom_block.cc b/table/bloom_block.cc index c44ab66ca2c..61959030a22 100644 --- a/table/bloom_block.cc +++ b/table/bloom_block.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #include "table/bloom_block.h" @@ -11,7 +11,7 @@ namespace rocksdb { -void BloomBlockBuilder::AddKeysHashes(const std::vector keys_hashes) { +void BloomBlockBuilder::AddKeysHashes(const std::vector& keys_hashes) { for (auto hash : keys_hashes) { bloom_.AddHash(hash); } diff --git a/table/bloom_block.h b/table/bloom_block.h index d55453edaae..9ff610badd4 100644 --- a/table/bloom_block.h +++ b/table/bloom_block.h @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #include @@ -18,15 +18,16 @@ class BloomBlockBuilder { explicit BloomBlockBuilder(uint32_t num_probes = 6) : bloom_(num_probes, nullptr) {} - void SetTotalBits(Arena* arena, uint32_t total_bits, uint32_t locality, - size_t huge_page_tlb_size, Logger* logger) { - bloom_.SetTotalBits(arena, total_bits, locality, huge_page_tlb_size, + void SetTotalBits(Allocator* allocator, uint32_t total_bits, + uint32_t locality, size_t huge_page_tlb_size, + Logger* logger) { + bloom_.SetTotalBits(allocator, total_bits, locality, huge_page_tlb_size, logger); } uint32_t GetNumBlocks() const { return bloom_.GetNumBlocks(); } - void AddKeysHashes(const std::vector keys_hashes); + void AddKeysHashes(const std::vector& keys_hashes); Slice Finish(); diff --git a/table/cleanable_test.cc b/table/cleanable_test.cc new file mode 100644 index 00000000000..f18c33b8399 --- /dev/null +++ b/table/cleanable_test.cc @@ -0,0 +1,277 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include + +#include "port/port.h" +#include "port/stack_trace.h" +#include "rocksdb/iostats_context.h" +#include "rocksdb/perf_context.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +class CleanableTest : public testing::Test {}; + +// Use this to keep track of the cleanups that were actually performed +void Multiplier(void* arg1, void* arg2) { + int* res = reinterpret_cast(arg1); + int* num = reinterpret_cast(arg2); + *res *= *num; +} + +// the first Cleanup is on stack and the rest on heap, so test with both cases +TEST_F(CleanableTest, Register) { + int n2 = 2, n3 = 3; + int res = 1; + { Cleanable c1; } + // ~Cleanable + ASSERT_EQ(1, res); + + res = 1; + { + Cleanable c1; + c1.RegisterCleanup(Multiplier, &res, &n2); // res = 2; + } + // ~Cleanable + ASSERT_EQ(2, res); + + res = 1; + { + Cleanable c1; + c1.RegisterCleanup(Multiplier, &res, &n2); // res = 2; + c1.RegisterCleanup(Multiplier, &res, &n3); // res = 2 * 3; + } + // ~Cleanable + ASSERT_EQ(6, res); + + // Test the Reset does cleanup + res = 1; + { + Cleanable c1; + c1.RegisterCleanup(Multiplier, &res, &n2); // res = 2; + c1.RegisterCleanup(Multiplier, &res, &n3); // res = 2 * 3; + c1.Reset(); + ASSERT_EQ(6, res); + } + // ~Cleanable + ASSERT_EQ(6, res); + + // Test Clenable is usable after Reset + res = 1; + { + Cleanable c1; + c1.RegisterCleanup(Multiplier, &res, &n2); // res = 2; + c1.Reset(); + ASSERT_EQ(2, res); + c1.RegisterCleanup(Multiplier, &res, &n3); // res = 2 * 3; + } + // ~Cleanable + ASSERT_EQ(6, res); +} + +// the first Cleanup is on stack and the rest on heap, +// so test all the combinations of them +TEST_F(CleanableTest, Delegation) { + int n2 = 2, n3 = 3, n5 = 5, n7 = 7; + int res = 1; + { + Cleanable c2; + { + Cleanable c1; + c1.RegisterCleanup(Multiplier, &res, &n2); // res = 2; + c1.DelegateCleanupsTo(&c2); + } + // ~Cleanable + ASSERT_EQ(1, res); + } + // ~Cleanable + ASSERT_EQ(2, res); + + res = 1; + { + Cleanable c2; + { + Cleanable c1; + c1.DelegateCleanupsTo(&c2); + } + // ~Cleanable + ASSERT_EQ(1, res); + } + // ~Cleanable + ASSERT_EQ(1, res); + + res = 1; + { + Cleanable c2; + { + Cleanable c1; + c1.RegisterCleanup(Multiplier, &res, &n2); // res = 2; + c1.RegisterCleanup(Multiplier, &res, &n3); // res = 2 * 3; + c1.DelegateCleanupsTo(&c2); + } + // ~Cleanable + ASSERT_EQ(1, res); + } + // ~Cleanable + ASSERT_EQ(6, res); + + res = 1; + { + Cleanable c2; + c2.RegisterCleanup(Multiplier, &res, &n5); // res = 5; + { + Cleanable c1; + c1.RegisterCleanup(Multiplier, &res, &n2); // res = 2; + c1.RegisterCleanup(Multiplier, &res, &n3); // res = 2 * 3; + c1.DelegateCleanupsTo(&c2); // res = 2 * 3 * 5; + } + // ~Cleanable + ASSERT_EQ(1, res); + } + // ~Cleanable + ASSERT_EQ(30, res); + + res = 1; + { + Cleanable c2; + c2.RegisterCleanup(Multiplier, &res, &n5); // res = 5; + c2.RegisterCleanup(Multiplier, &res, &n7); // res = 5 * 7; + { + Cleanable c1; + c1.RegisterCleanup(Multiplier, &res, &n2); // res = 2; + c1.RegisterCleanup(Multiplier, &res, &n3); // res = 2 * 3; + c1.DelegateCleanupsTo(&c2); // res = 2 * 3 * 5 * 7; + } + // ~Cleanable + ASSERT_EQ(1, res); + } + // ~Cleanable + ASSERT_EQ(210, res); + + res = 1; + { + Cleanable c2; + c2.RegisterCleanup(Multiplier, &res, &n5); // res = 5; + c2.RegisterCleanup(Multiplier, &res, &n7); // res = 5 * 7; + { + Cleanable c1; + c1.RegisterCleanup(Multiplier, &res, &n2); // res = 2; + c1.DelegateCleanupsTo(&c2); // res = 2 * 5 * 7; + } + // ~Cleanable + ASSERT_EQ(1, res); + } + // ~Cleanable + ASSERT_EQ(70, res); + + res = 1; + { + Cleanable c2; + c2.RegisterCleanup(Multiplier, &res, &n5); // res = 5; + c2.RegisterCleanup(Multiplier, &res, &n7); // res = 5 * 7; + { + Cleanable c1; + c1.DelegateCleanupsTo(&c2); // res = 5 * 7; + } + // ~Cleanable + ASSERT_EQ(1, res); + } + // ~Cleanable + ASSERT_EQ(35, res); + + res = 1; + { + Cleanable c2; + c2.RegisterCleanup(Multiplier, &res, &n5); // res = 5; + { + Cleanable c1; + c1.DelegateCleanupsTo(&c2); // res = 5; + } + // ~Cleanable + ASSERT_EQ(1, res); + } + // ~Cleanable + ASSERT_EQ(5, res); +} + +static void ReleaseStringHeap(void* s, void*) { + delete reinterpret_cast(s); +} + +class PinnableSlice4Test : public PinnableSlice { + public: + void TestStringIsRegistered(std::string* s) { + ASSERT_TRUE(cleanup_.function == ReleaseStringHeap); + ASSERT_EQ(cleanup_.arg1, s); + ASSERT_EQ(cleanup_.arg2, nullptr); + ASSERT_EQ(cleanup_.next, nullptr); + } +}; + +// Putting the PinnableSlice tests here due to similarity to Cleanable tests +TEST_F(CleanableTest, PinnableSlice) { + int n2 = 2; + int res = 1; + const std::string const_str = "123"; + + { + res = 1; + PinnableSlice4Test value; + Slice slice(const_str); + value.PinSlice(slice, Multiplier, &res, &n2); + std::string str; + str.assign(value.data(), value.size()); + ASSERT_EQ(const_str, str); + } + // ~Cleanable + ASSERT_EQ(2, res); + + { + res = 1; + PinnableSlice4Test value; + Slice slice(const_str); + { + Cleanable c1; + c1.RegisterCleanup(Multiplier, &res, &n2); // res = 2; + value.PinSlice(slice, &c1); + } + // ~Cleanable + ASSERT_EQ(1, res); // cleanups must have be delegated to value + std::string str; + str.assign(value.data(), value.size()); + ASSERT_EQ(const_str, str); + } + // ~Cleanable + ASSERT_EQ(2, res); + + { + PinnableSlice4Test value; + Slice slice(const_str); + value.PinSelf(slice); + std::string str; + str.assign(value.data(), value.size()); + ASSERT_EQ(const_str, str); + } + + { + PinnableSlice4Test value; + std::string* self_str_ptr = value.GetSelf(); + self_str_ptr->assign(const_str); + value.PinSelf(); + std::string str; + str.assign(value.data(), value.size()); + ASSERT_EQ(const_str, str); + } +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + rocksdb::port::InstallStackTraceHandler(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/table/cuckoo_table_builder.cc b/table/cuckoo_table_builder.cc index 6326d37876b..e3ed314b36f 100644 --- a/table/cuckoo_table_builder.cc +++ b/table/cuckoo_table_builder.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef ROCKSDB_LITE #include "table/cuckoo_table_builder.h" @@ -20,7 +20,9 @@ #include "table/format.h" #include "table/meta_blocks.h" #include "util/autovector.h" +#include "util/file_reader_writer.h" #include "util/random.h" +#include "util/string_util.h" namespace rocksdb { const std::string CuckooTablePropertyNames::kEmptyKey = @@ -35,36 +37,52 @@ const std::string CuckooTablePropertyNames::kIsLastLevel = "rocksdb.cuckoo.file.islastlevel"; const std::string CuckooTablePropertyNames::kCuckooBlockSize = "rocksdb.cuckoo.hash.cuckooblocksize"; +const std::string CuckooTablePropertyNames::kIdentityAsFirstHash = + "rocksdb.cuckoo.hash.identityfirst"; +const std::string CuckooTablePropertyNames::kUseModuleHash = + "rocksdb.cuckoo.hash.usemodule"; +const std::string CuckooTablePropertyNames::kUserKeyLength = + "rocksdb.cuckoo.hash.userkeylength"; // Obtained by running echo rocksdb.table.cuckoo | sha1sum extern const uint64_t kCuckooTableMagicNumber = 0x926789d0c5f17873ull; CuckooTableBuilder::CuckooTableBuilder( - WritableFile* file, double max_hash_table_ratio, + WritableFileWriter* file, double max_hash_table_ratio, uint32_t max_num_hash_table, uint32_t max_search_depth, const Comparator* user_comparator, uint32_t cuckoo_block_size, - uint64_t (*get_slice_hash)(const Slice&, uint32_t, uint64_t)) + bool use_module_hash, bool identity_as_first_hash, + uint64_t (*get_slice_hash)(const Slice&, uint32_t, uint64_t), + uint32_t column_family_id, const std::string& column_family_name) : num_hash_func_(2), file_(file), max_hash_table_ratio_(max_hash_table_ratio), max_num_hash_func_(max_num_hash_table), max_search_depth_(max_search_depth), cuckoo_block_size_(std::max(1U, cuckoo_block_size)), - hash_table_size_(2), + hash_table_size_(use_module_hash ? 0 : 2), is_last_level_file_(false), has_seen_first_key_(false), + has_seen_first_value_(false), + key_size_(0), + value_size_(0), + num_entries_(0), + num_values_(0), ucomp_(user_comparator), + use_module_hash_(use_module_hash), + identity_as_first_hash_(identity_as_first_hash), get_slice_hash_(get_slice_hash), closed_(false) { - properties_.num_entries = 0; // Data is in a huge block. properties_.num_data_blocks = 1; properties_.index_size = 0; properties_.filter_size = 0; + properties_.column_family_id = column_family_id; + properties_.column_family_name = column_family_name; } void CuckooTableBuilder::Add(const Slice& key, const Slice& value) { - if (properties_.num_entries >= kMaxVectorIdx - 1) { + if (num_entries_ >= kMaxVectorIdx - 1) { status_ = Status::NotSupported("Number of keys in a file must be < 2^32-1"); return; } @@ -73,6 +91,12 @@ void CuckooTableBuilder::Add(const Slice& key, const Slice& value) { status_ = Status::Corruption("Unable to parse key into inernal key."); return; } + if (ikey.type != kTypeDeletion && ikey.type != kTypeValue) { + status_ = Status::NotSupported("Unsupported key type " + + ToString(ikey.type)); + return; + } + // Determine if we can ignore the sequence number and value type from // internal keys by looking at sequence number from first key. We assume // that if first key has a zero sequence number, then all the remaining @@ -82,15 +106,38 @@ void CuckooTableBuilder::Add(const Slice& key, const Slice& value) { has_seen_first_key_ = true; smallest_user_key_.assign(ikey.user_key.data(), ikey.user_key.size()); largest_user_key_.assign(ikey.user_key.data(), ikey.user_key.size()); + key_size_ = is_last_level_file_ ? ikey.user_key.size() : key.size(); + } + if (key_size_ != (is_last_level_file_ ? ikey.user_key.size() : key.size())) { + status_ = Status::NotSupported("all keys have to be the same size"); + return; } - // Even if one sequence number is non-zero, then it is not last level. - assert(!is_last_level_file_ || ikey.sequence == 0); - if (is_last_level_file_) { - kvs_.emplace_back(std::make_pair( - ikey.user_key.ToString(), value.ToString())); + + if (ikey.type == kTypeValue) { + if (!has_seen_first_value_) { + has_seen_first_value_ = true; + value_size_ = value.size(); + } + if (value_size_ != value.size()) { + status_ = Status::NotSupported("all values have to be the same size"); + return; + } + + if (is_last_level_file_) { + kvs_.append(ikey.user_key.data(), ikey.user_key.size()); + } else { + kvs_.append(key.data(), key.size()); + } + kvs_.append(value.data(), value.size()); + ++num_values_; } else { - kvs_.emplace_back(std::make_pair(key.ToString(), value.ToString())); + if (is_last_level_file_) { + deleted_keys_.append(ikey.user_key.data(), ikey.user_key.size()); + } else { + deleted_keys_.append(key.data(), key.size()); + } } + ++num_entries_; // In order to fill the empty buckets in the hash table, we identify a // key which is not used so far (unused_user_key). We determine this by @@ -102,25 +149,52 @@ void CuckooTableBuilder::Add(const Slice& key, const Slice& value) { } else if (ikey.user_key.compare(largest_user_key_) > 0) { largest_user_key_.assign(ikey.user_key.data(), ikey.user_key.size()); } - if (hash_table_size_ < kvs_.size() / max_hash_table_ratio_) { - hash_table_size_ *= 2; + if (!use_module_hash_) { + if (hash_table_size_ < num_entries_ / max_hash_table_ratio_) { + hash_table_size_ *= 2; + } } } +bool CuckooTableBuilder::IsDeletedKey(uint64_t idx) const { + assert(closed_); + return idx >= num_values_; +} + +Slice CuckooTableBuilder::GetKey(uint64_t idx) const { + assert(closed_); + if (IsDeletedKey(idx)) { + return Slice(&deleted_keys_[(idx - num_values_) * key_size_], key_size_); + } + return Slice(&kvs_[idx * (key_size_ + value_size_)], key_size_); +} + +Slice CuckooTableBuilder::GetUserKey(uint64_t idx) const { + assert(closed_); + return is_last_level_file_ ? GetKey(idx) : ExtractUserKey(GetKey(idx)); +} + +Slice CuckooTableBuilder::GetValue(uint64_t idx) const { + assert(closed_); + if (IsDeletedKey(idx)) { + static std::string empty_value(value_size_, 'a'); + return Slice(empty_value); + } + return Slice(&kvs_[idx * (key_size_ + value_size_) + key_size_], value_size_); +} + Status CuckooTableBuilder::MakeHashTable(std::vector* buckets) { - uint64_t hash_table_size_minus_one = hash_table_size_ - 1; - buckets->resize(hash_table_size_minus_one + cuckoo_block_size_); - uint64_t make_space_for_key_call_id = 0; - for (uint32_t vector_idx = 0; vector_idx < kvs_.size(); vector_idx++) { + buckets->resize(hash_table_size_ + cuckoo_block_size_ - 1); + uint32_t make_space_for_key_call_id = 0; + for (uint32_t vector_idx = 0; vector_idx < num_entries_; vector_idx++) { uint64_t bucket_id; bool bucket_found = false; autovector hash_vals; - Slice user_key = is_last_level_file_ ? kvs_[vector_idx].first : - ExtractUserKey(kvs_[vector_idx].first); + Slice user_key = GetUserKey(vector_idx); for (uint32_t hash_cnt = 0; hash_cnt < num_hash_func_ && !bucket_found; ++hash_cnt) { - uint64_t hash_val = CuckooHash(user_key, hash_cnt, - hash_table_size_minus_one, get_slice_hash_); + uint64_t hash_val = CuckooHash(user_key, hash_cnt, use_module_hash_, + hash_table_size_, identity_as_first_hash_, get_slice_hash_); // If there is a collision, check next cuckoo_block_size_ locations for // empty locations. While checking, if we reach end of the hash table, // stop searching and proceed for next hash function. @@ -131,10 +205,8 @@ Status CuckooTableBuilder::MakeHashTable(std::vector* buckets) { bucket_found = true; break; } else { - if (ucomp_->Compare(user_key, is_last_level_file_ - ? Slice(kvs_[(*buckets)[hash_val].vector_idx].first) - : ExtractUserKey( - kvs_[(*buckets)[hash_val].vector_idx].first)) == 0) { + if (ucomp_->Compare(user_key, + GetUserKey((*buckets)[hash_val].vector_idx)) == 0) { return Status::NotSupported("Same key is being inserted again."); } hash_vals.push_back(hash_val); @@ -149,8 +221,8 @@ Status CuckooTableBuilder::MakeHashTable(std::vector* buckets) { } // We don't really need to rehash the entire table because old hashes are // still valid and we only increased the number of hash functions. - uint64_t hash_val = CuckooHash(user_key, num_hash_func_, - hash_table_size_minus_one, get_slice_hash_); + uint64_t hash_val = CuckooHash(user_key, num_hash_func_, use_module_hash_, + hash_table_size_, identity_as_first_hash_, get_slice_hash_); ++num_hash_func_; for (uint32_t block_idx = 0; block_idx < cuckoo_block_size_; ++block_idx, ++hash_val) { @@ -174,14 +246,19 @@ Status CuckooTableBuilder::Finish() { std::vector buckets; Status s; std::string unused_bucket; - if (!kvs_.empty()) { + if (num_entries_ > 0) { + // Calculate the real hash size if module hash is enabled. + if (use_module_hash_) { + hash_table_size_ = + static_cast(num_entries_ / max_hash_table_ratio_); + } s = MakeHashTable(&buckets); if (!s.ok()) { return s; } // Determine unused_user_key to fill empty buckets. std::string unused_user_key = smallest_user_key_; - int curr_pos = unused_user_key.size() - 1; + int curr_pos = static_cast(unused_user_key.size()) - 1; while (curr_pos >= 0) { --unused_user_key[curr_pos]; if (Slice(unused_user_key).compare(smallest_user_key_) < 0) { @@ -192,7 +269,7 @@ Status CuckooTableBuilder::Finish() { if (curr_pos < 0) { // Try using the largest key to identify an unused key. unused_user_key = largest_user_key_; - curr_pos = unused_user_key.size() - 1; + curr_pos = static_cast(unused_user_key.size()) - 1; while (curr_pos >= 0) { ++unused_user_key[curr_pos]; if (Slice(unused_user_key).compare(largest_user_key_) > 0) { @@ -211,14 +288,13 @@ Status CuckooTableBuilder::Finish() { AppendInternalKey(&unused_bucket, ikey); } } - properties_.num_entries = kvs_.size(); - properties_.fixed_key_len = unused_bucket.size(); - uint32_t value_length = kvs_.empty() ? 0 : kvs_[0].second.size(); - uint32_t bucket_size = value_length + properties_.fixed_key_len; + properties_.num_entries = num_entries_; + properties_.fixed_key_len = key_size_; properties_.user_collected_properties[ CuckooTablePropertyNames::kValueLength].assign( - reinterpret_cast(&value_length), sizeof(value_length)); + reinterpret_cast(&value_size_), sizeof(value_size_)); + uint64_t bucket_size = key_size_ + value_size_; unused_bucket.resize(bucket_size, 'a'); // Write the table. uint32_t num_added = 0; @@ -227,9 +303,11 @@ Status CuckooTableBuilder::Finish() { s = file_->Append(Slice(unused_bucket)); } else { ++num_added; - s = file_->Append(kvs_[bucket.vector_idx].first); + s = file_->Append(GetKey(bucket.vector_idx)); if (s.ok()) { - s = file_->Append(kvs_[bucket.vector_idx].second); + if (value_size_ > 0) { + s = file_->Append(GetValue(bucket.vector_idx)); + } } } if (!s.ok()) { @@ -238,7 +316,7 @@ Status CuckooTableBuilder::Finish() { } assert(num_added == NumEntries()); properties_.raw_key_size = num_added * properties_.fixed_key_len; - properties_.raw_value_size = num_added * value_length; + properties_.raw_value_size = num_added * value_size_; uint64_t offset = buckets.size() * bucket_size; properties_.data_size = offset; @@ -249,11 +327,10 @@ Status CuckooTableBuilder::Finish() { CuckooTablePropertyNames::kNumHashFunc].assign( reinterpret_cast(&num_hash_func_), sizeof(num_hash_func_)); - uint64_t hash_table_size = buckets.size() - cuckoo_block_size_ + 1; properties_.user_collected_properties[ CuckooTablePropertyNames::kHashTableSize].assign( - reinterpret_cast(&hash_table_size), - sizeof(hash_table_size)); + reinterpret_cast(&hash_table_size_), + sizeof(hash_table_size_)); properties_.user_collected_properties[ CuckooTablePropertyNames::kIsLastLevel].assign( reinterpret_cast(&is_last_level_file_), @@ -262,6 +339,19 @@ Status CuckooTableBuilder::Finish() { CuckooTablePropertyNames::kCuckooBlockSize].assign( reinterpret_cast(&cuckoo_block_size_), sizeof(cuckoo_block_size_)); + properties_.user_collected_properties[ + CuckooTablePropertyNames::kIdentityAsFirstHash].assign( + reinterpret_cast(&identity_as_first_hash_), + sizeof(identity_as_first_hash_)); + properties_.user_collected_properties[ + CuckooTablePropertyNames::kUseModuleHash].assign( + reinterpret_cast(&use_module_hash_), + sizeof(use_module_hash_)); + uint32_t user_key_len = static_cast(smallest_user_key_.size()); + properties_.user_collected_properties[ + CuckooTablePropertyNames::kUserKeyLength].assign( + reinterpret_cast(&user_key_len), + sizeof(user_key_len)); // Write meta blocks. MetaIndexBuilder meta_index_builder; @@ -290,7 +380,7 @@ Status CuckooTableBuilder::Finish() { return s; } - Footer footer(kCuckooTableMagicNumber); + Footer footer(kCuckooTableMagicNumber, 1); footer.set_metaindex_handle(meta_index_block_handle); footer.set_index_handle(BlockHandle::NullBlockHandle()); std::string footer_encoding; @@ -305,26 +395,31 @@ void CuckooTableBuilder::Abandon() { } uint64_t CuckooTableBuilder::NumEntries() const { - return kvs_.size(); + return num_entries_; } uint64_t CuckooTableBuilder::FileSize() const { if (closed_) { return file_->GetFileSize(); - } else if (properties_.num_entries == 0) { + } else if (num_entries_ == 0) { return 0; } - // Account for buckets being a power of two. - // As elements are added, file size remains constant for a while and doubles - // its size. Since compaction algorithm stops adding elements only after it - // exceeds the file limit, we account for the extra element being added here. - uint64_t expected_hash_table_size = hash_table_size_; - if (expected_hash_table_size < (kvs_.size() + 1) / max_hash_table_ratio_) { - expected_hash_table_size *= 2; + if (use_module_hash_) { + return static_cast((key_size_ + value_size_) * + num_entries_ / max_hash_table_ratio_); + } else { + // Account for buckets being a power of two. + // As elements are added, file size remains constant for a while and + // doubles its size. Since compaction algorithm stops adding elements + // only after it exceeds the file limit, we account for the extra element + // being added here. + uint64_t expected_hash_table_size = hash_table_size_; + if (expected_hash_table_size < (num_entries_ + 1) / max_hash_table_ratio_) { + expected_hash_table_size *= 2; + } + return (key_size_ + value_size_) * expected_hash_table_size - 1; } - return (kvs_[0].first.size() + kvs_[0].second.size()) * - expected_hash_table_size; } // This method is invoked when there is no place to insert the target key. @@ -339,15 +434,14 @@ uint64_t CuckooTableBuilder::FileSize() const { // If tree depth exceedes max depth, we return false indicating failure. bool CuckooTableBuilder::MakeSpaceForKey( const autovector& hash_vals, - const uint64_t make_space_for_key_call_id, - std::vector* buckets, - uint64_t* bucket_id) { + const uint32_t make_space_for_key_call_id, + std::vector* buckets, uint64_t* bucket_id) { struct CuckooNode { uint64_t bucket_id; uint32_t depth; uint32_t parent_pos; - CuckooNode(uint64_t bucket_id, uint32_t depth, int parent_pos) - : bucket_id(bucket_id), depth(depth), parent_pos(parent_pos) {} + CuckooNode(uint64_t _bucket_id, uint32_t _depth, int _parent_pos) + : bucket_id(_bucket_id), depth(_depth), parent_pos(_parent_pos) {} }; // This is BFS search tree that is stored simply as a vector. // Each node stores the index of parent node in the vector. @@ -359,14 +453,12 @@ bool CuckooTableBuilder::MakeSpaceForKey( // of the method. We store this number into the nodes that we explore in // current method call. // It is unlikely for the increment operation to overflow because the maximum - // no. of times this will be called is <= max_num_hash_func_ + kvs_.size(). + // no. of times this will be called is <= max_num_hash_func_ + num_entries_. for (uint32_t hash_cnt = 0; hash_cnt < num_hash_func_; ++hash_cnt) { - uint64_t bucket_id = hash_vals[hash_cnt]; - (*buckets)[bucket_id].make_space_for_key_call_id = - make_space_for_key_call_id; - tree.push_back(CuckooNode(bucket_id, 0, 0)); + uint64_t bid = hash_vals[hash_cnt]; + (*buckets)[bid].make_space_for_key_call_id = make_space_for_key_call_id; + tree.push_back(CuckooNode(bid, 0, 0)); } - uint64_t hash_table_size_minus_one = hash_table_size_ - 1; bool null_found = false; uint32_t curr_pos = 0; while (!null_found && curr_pos < tree.size()) { @@ -378,10 +470,9 @@ bool CuckooTableBuilder::MakeSpaceForKey( CuckooBucket& curr_bucket = (*buckets)[curr_node.bucket_id]; for (uint32_t hash_cnt = 0; hash_cnt < num_hash_func_ && !null_found; ++hash_cnt) { - uint64_t child_bucket_id = CuckooHash( - (is_last_level_file_ ? kvs_[curr_bucket.vector_idx].first : - ExtractUserKey(Slice(kvs_[curr_bucket.vector_idx].first))), - hash_cnt, hash_table_size_minus_one, get_slice_hash_); + uint64_t child_bucket_id = CuckooHash(GetUserKey(curr_bucket.vector_idx), + hash_cnt, use_module_hash_, hash_table_size_, identity_as_first_hash_, + get_slice_hash_); // Iterate inside Cuckoo Block. for (uint32_t block_idx = 0; block_idx < cuckoo_block_size_; ++block_idx, ++child_bucket_id) { @@ -408,7 +499,7 @@ bool CuckooTableBuilder::MakeSpaceForKey( // child with the parent. Stop when first level is reached in the tree // (happens when 0 <= bucket_to_replace_pos < num_hash_func_) and return // this location in first level for target key to be inserted. - uint32_t bucket_to_replace_pos = tree.size()-1; + uint32_t bucket_to_replace_pos = static_cast(tree.size()) - 1; while (bucket_to_replace_pos >= num_hash_func_) { CuckooNode& curr_node = tree[bucket_to_replace_pos]; (*buckets)[curr_node.bucket_id] = diff --git a/table/cuckoo_table_builder.h b/table/cuckoo_table_builder.h index 2bf206102e4..3829541b39a 100644 --- a/table/cuckoo_table_builder.h +++ b/table/cuckoo_table_builder.h @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #ifndef ROCKSDB_LITE @@ -10,6 +10,7 @@ #include #include #include +#include "port/port.h" #include "rocksdb/status.h" #include "table/table_builder.h" #include "rocksdb/table.h" @@ -20,11 +21,15 @@ namespace rocksdb { class CuckooTableBuilder: public TableBuilder { public: - CuckooTableBuilder( - WritableFile* file, double max_hash_table_ratio, - uint32_t max_num_hash_func, uint32_t max_search_depth, - const Comparator* user_comparator, uint32_t cuckoo_block_size, - uint64_t (*get_slice_hash)(const Slice&, uint32_t, uint64_t)); + CuckooTableBuilder(WritableFileWriter* file, double max_hash_table_ratio, + uint32_t max_num_hash_func, uint32_t max_search_depth, + const Comparator* user_comparator, + uint32_t cuckoo_block_size, bool use_module_hash, + bool identity_as_first_hash, + uint64_t (*get_slice_hash)(const Slice&, uint32_t, + uint64_t), + uint32_t column_family_id, + const std::string& column_family_name); // REQUIRES: Either Finish() or Abandon() has been called. ~CuckooTableBuilder() {} @@ -56,6 +61,8 @@ class CuckooTableBuilder: public TableBuilder { // Finish() call, returns the size of the final generated file. uint64_t FileSize() const override; + TableProperties GetTableProperties() const override { return properties_; } + private: struct CuckooBucket { CuckooBucket() @@ -65,28 +72,44 @@ class CuckooTableBuilder: public TableBuilder { // We assume number of items is <= 2^32. uint32_t make_space_for_key_call_id; }; - static const uint32_t kMaxVectorIdx = std::numeric_limits::max(); + static const uint32_t kMaxVectorIdx = port::kMaxInt32; - bool MakeSpaceForKey( - const autovector& hash_vals, - const uint64_t call_id, - std::vector* buckets, - uint64_t* bucket_id); + bool MakeSpaceForKey(const autovector& hash_vals, + const uint32_t call_id, + std::vector* buckets, uint64_t* bucket_id); Status MakeHashTable(std::vector* buckets); + inline bool IsDeletedKey(uint64_t idx) const; + inline Slice GetKey(uint64_t idx) const; + inline Slice GetUserKey(uint64_t idx) const; + inline Slice GetValue(uint64_t idx) const; + uint32_t num_hash_func_; - WritableFile* file_; + WritableFileWriter* file_; const double max_hash_table_ratio_; const uint32_t max_num_hash_func_; const uint32_t max_search_depth_; const uint32_t cuckoo_block_size_; uint64_t hash_table_size_; bool is_last_level_file_; + bool has_seen_first_key_; + bool has_seen_first_value_; + uint64_t key_size_; + uint64_t value_size_; + // A list of fixed-size key-value pairs concatenating into a string. + // Use GetKey(), GetUserKey(), and GetValue() to retrieve a specific + // key / value given an index + std::string kvs_; + std::string deleted_keys_; + // Number of key-value pairs stored in kvs_ + number of deleted keys + uint64_t num_entries_; + // Number of keys that contain value (non-deletion op) + uint64_t num_values_; Status status_; - std::vector> kvs_; TableProperties properties_; - bool has_seen_first_key_; const Comparator* ucomp_; + bool use_module_hash_; + bool identity_as_first_hash_; uint64_t (*get_slice_hash_)(const Slice& s, uint32_t index, uint64_t max_num_buckets); std::string largest_user_key_ = ""; diff --git a/table/cuckoo_table_builder_test.cc b/table/cuckoo_table_builder_test.cc index 69647d410ec..93daaca472b 100644 --- a/table/cuckoo_table_builder_test.cc +++ b/table/cuckoo_table_builder_test.cc @@ -1,7 +1,9 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE #include #include @@ -10,6 +12,7 @@ #include "table/meta_blocks.h" #include "table/cuckoo_table_builder.h" +#include "util/file_reader_writer.h" #include "util/testharness.h" #include "util/testutil.h" @@ -25,7 +28,7 @@ uint64_t GetSliceHash(const Slice& s, uint32_t index, } } // namespace -class CuckooBuilderTest { +class CuckooBuilderTest : public testing::Test { public: CuckooBuilderTest() { env_ = Env::Default(); @@ -46,24 +49,25 @@ class CuckooBuilderTest { uint64_t read_file_size; ASSERT_OK(env_->GetFileSize(fname, &read_file_size)); + Options options; + options.allow_mmap_reads = true; + ImmutableCFOptions ioptions(options); + // Assert Table Properties. TableProperties* props = nullptr; - ASSERT_OK(ReadTableProperties(read_file.get(), read_file_size, - kCuckooTableMagicNumber, env_, nullptr, &props)); - ASSERT_EQ(props->num_entries, keys.size()); - ASSERT_EQ(props->fixed_key_len, keys.empty() ? 0 : keys[0].size()); - ASSERT_EQ(props->data_size, expected_unused_bucket.size() * - (expected_table_size + expected_cuckoo_block_size - 1)); - ASSERT_EQ(props->raw_key_size, keys.size()*props->fixed_key_len); - + unique_ptr file_reader( + new RandomAccessFileReader(std::move(read_file), fname)); + ASSERT_OK(ReadTableProperties(file_reader.get(), read_file_size, + kCuckooTableMagicNumber, ioptions, + &props)); // Check unused bucket. std::string unused_key = props->user_collected_properties[ CuckooTablePropertyNames::kEmptyKey]; ASSERT_EQ(expected_unused_bucket.substr(0, props->fixed_key_len), unused_key); - uint32_t value_len_found = - *reinterpret_cast(props->user_collected_properties[ + uint64_t value_len_found = + *reinterpret_cast(props->user_collected_properties[ CuckooTablePropertyNames::kValueLength].data()); ASSERT_EQ(values.empty() ? 0 : values[0].size(), value_len_found); ASSERT_EQ(props->raw_value_size, values.size()*value_len_found); @@ -83,27 +87,40 @@ class CuckooBuilderTest { *reinterpret_cast(props->user_collected_properties[ CuckooTablePropertyNames::kIsLastLevel].data()); ASSERT_EQ(expected_is_last_level, is_last_level_found); + + ASSERT_EQ(props->num_entries, keys.size()); + ASSERT_EQ(props->fixed_key_len, keys.empty() ? 0 : keys[0].size()); + ASSERT_EQ(props->data_size, expected_unused_bucket.size() * + (expected_table_size + expected_cuckoo_block_size - 1)); + ASSERT_EQ(props->raw_key_size, keys.size()*props->fixed_key_len); + ASSERT_EQ(props->column_family_id, 0); + ASSERT_EQ(props->column_family_name, kDefaultColumnFamilyName); delete props; // Check contents of the bucket. std::vector keys_found(keys.size(), false); - uint32_t bucket_size = expected_unused_bucket.size(); + size_t bucket_size = expected_unused_bucket.size(); for (uint32_t i = 0; i < table_size + cuckoo_block_size - 1; ++i) { Slice read_slice; - ASSERT_OK(read_file->Read(i*bucket_size, bucket_size, - &read_slice, nullptr)); - uint32_t key_idx = std::find(expected_locations.begin(), - expected_locations.end(), i) - expected_locations.begin(); + ASSERT_OK(file_reader->Read(i * bucket_size, bucket_size, &read_slice, + nullptr)); + size_t key_idx = + std::find(expected_locations.begin(), expected_locations.end(), i) - + expected_locations.begin(); if (key_idx == keys.size()) { - // i is not one of the expected locaitons. Empty bucket. - ASSERT_EQ(read_slice.compare(expected_unused_bucket), 0); + // i is not one of the expected locations. Empty bucket. + if (read_slice.data() == nullptr) { + ASSERT_EQ(0, expected_unused_bucket.size()); + } else { + ASSERT_EQ(read_slice.compare(expected_unused_bucket), 0); + } } else { keys_found[key_idx] = true; ASSERT_EQ(read_slice.compare(keys[key_idx] + values[key_idx]), 0); } } for (auto key_found : keys_found) { - // Check that all keys were found. + // Check that all keys wereReader found. ASSERT_TRUE(key_found); } } @@ -111,7 +128,7 @@ class CuckooBuilderTest { std::string GetInternalKey(Slice user_key, bool zero_seqno) { IterKey ikey; ikey.SetInternalKey(user_key, zero_seqno ? 0 : 1000, kTypeValue); - return ikey.GetKey().ToString(); + return ikey.GetInternalKey().ToString(); } uint64_t NextPowOf2(uint64_t num) { @@ -122,137 +139,179 @@ class CuckooBuilderTest { return n; } + uint64_t GetExpectedTableSize(uint64_t num) { + return NextPowOf2(static_cast(num / kHashTableRatio)); + } + + Env* env_; EnvOptions env_options_; std::string fname; const double kHashTableRatio = 0.9; }; -TEST(CuckooBuilderTest, SuccessWithEmptyFile) { +TEST_F(CuckooBuilderTest, SuccessWithEmptyFile) { unique_ptr writable_file; fname = test::TmpDir() + "/EmptyFile"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - 4, 100, BytewiseComparator(), 1, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder(file_writer.get(), kHashTableRatio, 4, 100, + BytewiseComparator(), 1, false, false, + GetSliceHash, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); + ASSERT_EQ(0UL, builder.FileSize()); ASSERT_OK(builder.Finish()); - ASSERT_OK(writable_file->Close()); - CheckFileContents({}, {}, {}, "", 0, 2, false); + ASSERT_OK(file_writer->Close()); + CheckFileContents({}, {}, {}, "", 2, 2, false); } -TEST(CuckooBuilderTest, WriteSuccessNoCollisionFullKey) { +TEST_F(CuckooBuilderTest, WriteSuccessNoCollisionFullKey) { uint32_t num_hash_fun = 4; std::vector user_keys = {"key01", "key02", "key03", "key04"}; std::vector values = {"v01", "v02", "v03", "v04"}; - hash_map = { - {user_keys[0], {0, 1, 2, 3}}, - {user_keys[1], {1, 2, 3, 4}}, - {user_keys[2], {2, 3, 4, 5}}, - {user_keys[3], {3, 4, 5, 6}} - }; + // Need to have a temporary variable here as VS compiler does not currently + // support operator= with initializer_list as a parameter + std::unordered_map> hm = { + {user_keys[0], {0, 1, 2, 3}}, + {user_keys[1], {1, 2, 3, 4}}, + {user_keys[2], {2, 3, 4, 5}}, + {user_keys[3], {3, 4, 5, 6}}}; + hash_map = std::move(hm); + std::vector expected_locations = {0, 1, 2, 3}; std::vector keys; for (auto& user_key : user_keys) { keys.push_back(GetInternalKey(user_key, false)); } + uint64_t expected_table_size = GetExpectedTableSize(keys.size()); unique_ptr writable_file; fname = test::TmpDir() + "/NoCollisionFullKey"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - num_hash_fun, 100, BytewiseComparator(), 1, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder(file_writer.get(), kHashTableRatio, num_hash_fun, + 100, BytewiseComparator(), 1, false, false, + GetSliceHash, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint32_t i = 0; i < user_keys.size(); i++) { builder.Add(Slice(keys[i]), Slice(values[i])); ASSERT_EQ(builder.NumEntries(), i + 1); ASSERT_OK(builder.status()); } + size_t bucket_size = keys[0].size() + values[0].size(); + ASSERT_EQ(expected_table_size * bucket_size - 1, builder.FileSize()); ASSERT_OK(builder.Finish()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); + ASSERT_LE(expected_table_size * bucket_size, builder.FileSize()); - uint32_t expected_table_size = NextPowOf2(keys.size() / kHashTableRatio); std::string expected_unused_bucket = GetInternalKey("key00", true); expected_unused_bucket += std::string(values[0].size(), 'a'); CheckFileContents(keys, values, expected_locations, expected_unused_bucket, expected_table_size, 2, false); } -TEST(CuckooBuilderTest, WriteSuccessWithCollisionFullKey) { +TEST_F(CuckooBuilderTest, WriteSuccessWithCollisionFullKey) { uint32_t num_hash_fun = 4; std::vector user_keys = {"key01", "key02", "key03", "key04"}; std::vector values = {"v01", "v02", "v03", "v04"}; - hash_map = { - {user_keys[0], {0, 1, 2, 3}}, - {user_keys[1], {0, 1, 2, 3}}, - {user_keys[2], {0, 1, 2, 3}}, - {user_keys[3], {0, 1, 2, 3}}, + // Need to have a temporary variable here as VS compiler does not currently + // support operator= with initializer_list as a parameter + std::unordered_map> hm = { + {user_keys[0], {0, 1, 2, 3}}, + {user_keys[1], {0, 1, 2, 3}}, + {user_keys[2], {0, 1, 2, 3}}, + {user_keys[3], {0, 1, 2, 3}}, }; + hash_map = std::move(hm); + std::vector expected_locations = {0, 1, 2, 3}; std::vector keys; for (auto& user_key : user_keys) { keys.push_back(GetInternalKey(user_key, false)); } + uint64_t expected_table_size = GetExpectedTableSize(keys.size()); unique_ptr writable_file; fname = test::TmpDir() + "/WithCollisionFullKey"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - num_hash_fun, 100, BytewiseComparator(), 1, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder(file_writer.get(), kHashTableRatio, num_hash_fun, + 100, BytewiseComparator(), 1, false, false, + GetSliceHash, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint32_t i = 0; i < user_keys.size(); i++) { builder.Add(Slice(keys[i]), Slice(values[i])); ASSERT_EQ(builder.NumEntries(), i + 1); ASSERT_OK(builder.status()); } + size_t bucket_size = keys[0].size() + values[0].size(); + ASSERT_EQ(expected_table_size * bucket_size - 1, builder.FileSize()); ASSERT_OK(builder.Finish()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); + ASSERT_LE(expected_table_size * bucket_size, builder.FileSize()); - uint32_t expected_table_size = NextPowOf2(keys.size() / kHashTableRatio); std::string expected_unused_bucket = GetInternalKey("key00", true); expected_unused_bucket += std::string(values[0].size(), 'a'); CheckFileContents(keys, values, expected_locations, expected_unused_bucket, expected_table_size, 4, false); } -TEST(CuckooBuilderTest, WriteSuccessWithCollisionAndCuckooBlock) { +TEST_F(CuckooBuilderTest, WriteSuccessWithCollisionAndCuckooBlock) { uint32_t num_hash_fun = 4; std::vector user_keys = {"key01", "key02", "key03", "key04"}; std::vector values = {"v01", "v02", "v03", "v04"}; - hash_map = { - {user_keys[0], {0, 1, 2, 3}}, - {user_keys[1], {0, 1, 2, 3}}, - {user_keys[2], {0, 1, 2, 3}}, - {user_keys[3], {0, 1, 2, 3}}, + // Need to have a temporary variable here as VS compiler does not currently + // support operator= with initializer_list as a parameter + std::unordered_map> hm = { + {user_keys[0], {0, 1, 2, 3}}, + {user_keys[1], {0, 1, 2, 3}}, + {user_keys[2], {0, 1, 2, 3}}, + {user_keys[3], {0, 1, 2, 3}}, }; + hash_map = std::move(hm); + std::vector expected_locations = {0, 1, 2, 3}; std::vector keys; for (auto& user_key : user_keys) { keys.push_back(GetInternalKey(user_key, false)); } + uint64_t expected_table_size = GetExpectedTableSize(keys.size()); unique_ptr writable_file; uint32_t cuckoo_block_size = 2; fname = test::TmpDir() + "/WithCollisionFullKey2"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - num_hash_fun, 100, BytewiseComparator(), cuckoo_block_size, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder( + file_writer.get(), kHashTableRatio, num_hash_fun, 100, + BytewiseComparator(), cuckoo_block_size, false, false, GetSliceHash, + 0 /* column_family_id */, kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint32_t i = 0; i < user_keys.size(); i++) { builder.Add(Slice(keys[i]), Slice(values[i])); ASSERT_EQ(builder.NumEntries(), i + 1); ASSERT_OK(builder.status()); } + size_t bucket_size = keys[0].size() + values[0].size(); + ASSERT_EQ(expected_table_size * bucket_size - 1, builder.FileSize()); ASSERT_OK(builder.Finish()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); + ASSERT_LE(expected_table_size * bucket_size, builder.FileSize()); - uint32_t expected_table_size = NextPowOf2(keys.size() / kHashTableRatio); std::string expected_unused_bucket = GetInternalKey("key00", true); expected_unused_bucket += std::string(values[0].size(), 'a'); CheckFileContents(keys, values, expected_locations, expected_unused_bucket, expected_table_size, 3, false, cuckoo_block_size); } -TEST(CuckooBuilderTest, WithCollisionPathFullKey) { +TEST_F(CuckooBuilderTest, WithCollisionPathFullKey) { // Have two hash functions. Insert elements with overlapping hashes. // Finally insert an element with hash value somewhere in the middle // so that it displaces all the elements after that. @@ -260,200 +319,261 @@ TEST(CuckooBuilderTest, WithCollisionPathFullKey) { std::vector user_keys = {"key01", "key02", "key03", "key04", "key05"}; std::vector values = {"v01", "v02", "v03", "v04", "v05"}; - hash_map = { - {user_keys[0], {0, 1}}, - {user_keys[1], {1, 2}}, - {user_keys[2], {2, 3}}, - {user_keys[3], {3, 4}}, - {user_keys[4], {0, 2}}, + // Need to have a temporary variable here as VS compiler does not currently + // support operator= with initializer_list as a parameter + std::unordered_map> hm = { + {user_keys[0], {0, 1}}, + {user_keys[1], {1, 2}}, + {user_keys[2], {2, 3}}, + {user_keys[3], {3, 4}}, + {user_keys[4], {0, 2}}, }; + hash_map = std::move(hm); + std::vector expected_locations = {0, 1, 3, 4, 2}; std::vector keys; for (auto& user_key : user_keys) { keys.push_back(GetInternalKey(user_key, false)); } + uint64_t expected_table_size = GetExpectedTableSize(keys.size()); unique_ptr writable_file; fname = test::TmpDir() + "/WithCollisionPathFullKey"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - num_hash_fun, 100, BytewiseComparator(), 1, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder(file_writer.get(), kHashTableRatio, num_hash_fun, + 100, BytewiseComparator(), 1, false, false, + GetSliceHash, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint32_t i = 0; i < user_keys.size(); i++) { builder.Add(Slice(keys[i]), Slice(values[i])); ASSERT_EQ(builder.NumEntries(), i + 1); ASSERT_OK(builder.status()); } + size_t bucket_size = keys[0].size() + values[0].size(); + ASSERT_EQ(expected_table_size * bucket_size - 1, builder.FileSize()); ASSERT_OK(builder.Finish()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); + ASSERT_LE(expected_table_size * bucket_size, builder.FileSize()); - uint32_t expected_table_size = NextPowOf2(keys.size() / kHashTableRatio); std::string expected_unused_bucket = GetInternalKey("key00", true); expected_unused_bucket += std::string(values[0].size(), 'a'); CheckFileContents(keys, values, expected_locations, expected_unused_bucket, expected_table_size, 2, false); } -TEST(CuckooBuilderTest, WithCollisionPathFullKeyAndCuckooBlock) { +TEST_F(CuckooBuilderTest, WithCollisionPathFullKeyAndCuckooBlock) { uint32_t num_hash_fun = 2; std::vector user_keys = {"key01", "key02", "key03", "key04", "key05"}; std::vector values = {"v01", "v02", "v03", "v04", "v05"}; - hash_map = { - {user_keys[0], {0, 1}}, - {user_keys[1], {1, 2}}, - {user_keys[2], {3, 4}}, - {user_keys[3], {4, 5}}, - {user_keys[4], {0, 3}}, + // Need to have a temporary variable here as VS compiler does not currently + // support operator= with initializer_list as a parameter + std::unordered_map> hm = { + {user_keys[0], {0, 1}}, + {user_keys[1], {1, 2}}, + {user_keys[2], {3, 4}}, + {user_keys[3], {4, 5}}, + {user_keys[4], {0, 3}}, }; + hash_map = std::move(hm); + std::vector expected_locations = {2, 1, 3, 4, 0}; std::vector keys; for (auto& user_key : user_keys) { keys.push_back(GetInternalKey(user_key, false)); } + uint64_t expected_table_size = GetExpectedTableSize(keys.size()); unique_ptr writable_file; fname = test::TmpDir() + "/WithCollisionPathFullKeyAndCuckooBlock"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - num_hash_fun, 100, BytewiseComparator(), 2, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder(file_writer.get(), kHashTableRatio, num_hash_fun, + 100, BytewiseComparator(), 2, false, false, + GetSliceHash, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint32_t i = 0; i < user_keys.size(); i++) { builder.Add(Slice(keys[i]), Slice(values[i])); ASSERT_EQ(builder.NumEntries(), i + 1); ASSERT_OK(builder.status()); } + size_t bucket_size = keys[0].size() + values[0].size(); + ASSERT_EQ(expected_table_size * bucket_size - 1, builder.FileSize()); ASSERT_OK(builder.Finish()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); + ASSERT_LE(expected_table_size * bucket_size, builder.FileSize()); - uint32_t expected_table_size = NextPowOf2(keys.size() / kHashTableRatio); std::string expected_unused_bucket = GetInternalKey("key00", true); expected_unused_bucket += std::string(values[0].size(), 'a'); CheckFileContents(keys, values, expected_locations, expected_unused_bucket, expected_table_size, 2, false, 2); } -TEST(CuckooBuilderTest, WriteSuccessNoCollisionUserKey) { +TEST_F(CuckooBuilderTest, WriteSuccessNoCollisionUserKey) { uint32_t num_hash_fun = 4; std::vector user_keys = {"key01", "key02", "key03", "key04"}; std::vector values = {"v01", "v02", "v03", "v04"}; - hash_map = { - {user_keys[0], {0, 1, 2, 3}}, - {user_keys[1], {1, 2, 3, 4}}, - {user_keys[2], {2, 3, 4, 5}}, - {user_keys[3], {3, 4, 5, 6}} - }; + // Need to have a temporary variable here as VS compiler does not currently + // support operator= with initializer_list as a parameter + std::unordered_map> hm = { + {user_keys[0], {0, 1, 2, 3}}, + {user_keys[1], {1, 2, 3, 4}}, + {user_keys[2], {2, 3, 4, 5}}, + {user_keys[3], {3, 4, 5, 6}}}; + hash_map = std::move(hm); + std::vector expected_locations = {0, 1, 2, 3}; + uint64_t expected_table_size = GetExpectedTableSize(user_keys.size()); unique_ptr writable_file; fname = test::TmpDir() + "/NoCollisionUserKey"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - num_hash_fun, 100, BytewiseComparator(), 1, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder(file_writer.get(), kHashTableRatio, num_hash_fun, + 100, BytewiseComparator(), 1, false, false, + GetSliceHash, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint32_t i = 0; i < user_keys.size(); i++) { builder.Add(Slice(GetInternalKey(user_keys[i], true)), Slice(values[i])); ASSERT_EQ(builder.NumEntries(), i + 1); ASSERT_OK(builder.status()); } + size_t bucket_size = user_keys[0].size() + values[0].size(); + ASSERT_EQ(expected_table_size * bucket_size - 1, builder.FileSize()); ASSERT_OK(builder.Finish()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); + ASSERT_LE(expected_table_size * bucket_size, builder.FileSize()); - uint32_t expected_table_size = NextPowOf2(user_keys.size() / kHashTableRatio); std::string expected_unused_bucket = "key00"; expected_unused_bucket += std::string(values[0].size(), 'a'); CheckFileContents(user_keys, values, expected_locations, expected_unused_bucket, expected_table_size, 2, true); } -TEST(CuckooBuilderTest, WriteSuccessWithCollisionUserKey) { +TEST_F(CuckooBuilderTest, WriteSuccessWithCollisionUserKey) { uint32_t num_hash_fun = 4; std::vector user_keys = {"key01", "key02", "key03", "key04"}; std::vector values = {"v01", "v02", "v03", "v04"}; - hash_map = { - {user_keys[0], {0, 1, 2, 3}}, - {user_keys[1], {0, 1, 2, 3}}, - {user_keys[2], {0, 1, 2, 3}}, - {user_keys[3], {0, 1, 2, 3}}, + // Need to have a temporary variable here as VS compiler does not currently + // support operator= with initializer_list as a parameter + std::unordered_map> hm = { + {user_keys[0], {0, 1, 2, 3}}, + {user_keys[1], {0, 1, 2, 3}}, + {user_keys[2], {0, 1, 2, 3}}, + {user_keys[3], {0, 1, 2, 3}}, }; + hash_map = std::move(hm); + std::vector expected_locations = {0, 1, 2, 3}; + uint64_t expected_table_size = GetExpectedTableSize(user_keys.size()); unique_ptr writable_file; fname = test::TmpDir() + "/WithCollisionUserKey"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - num_hash_fun, 100, BytewiseComparator(), 1, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder(file_writer.get(), kHashTableRatio, num_hash_fun, + 100, BytewiseComparator(), 1, false, false, + GetSliceHash, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint32_t i = 0; i < user_keys.size(); i++) { builder.Add(Slice(GetInternalKey(user_keys[i], true)), Slice(values[i])); ASSERT_EQ(builder.NumEntries(), i + 1); ASSERT_OK(builder.status()); } + size_t bucket_size = user_keys[0].size() + values[0].size(); + ASSERT_EQ(expected_table_size * bucket_size - 1, builder.FileSize()); ASSERT_OK(builder.Finish()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); + ASSERT_LE(expected_table_size * bucket_size, builder.FileSize()); - uint32_t expected_table_size = NextPowOf2(user_keys.size() / kHashTableRatio); std::string expected_unused_bucket = "key00"; expected_unused_bucket += std::string(values[0].size(), 'a'); CheckFileContents(user_keys, values, expected_locations, expected_unused_bucket, expected_table_size, 4, true); } -TEST(CuckooBuilderTest, WithCollisionPathUserKey) { +TEST_F(CuckooBuilderTest, WithCollisionPathUserKey) { uint32_t num_hash_fun = 2; std::vector user_keys = {"key01", "key02", "key03", "key04", "key05"}; std::vector values = {"v01", "v02", "v03", "v04", "v05"}; - hash_map = { - {user_keys[0], {0, 1}}, - {user_keys[1], {1, 2}}, - {user_keys[2], {2, 3}}, - {user_keys[3], {3, 4}}, - {user_keys[4], {0, 2}}, + // Need to have a temporary variable here as VS compiler does not currently + // support operator= with initializer_list as a parameter + std::unordered_map> hm = { + {user_keys[0], {0, 1}}, + {user_keys[1], {1, 2}}, + {user_keys[2], {2, 3}}, + {user_keys[3], {3, 4}}, + {user_keys[4], {0, 2}}, }; + hash_map = std::move(hm); + std::vector expected_locations = {0, 1, 3, 4, 2}; + uint64_t expected_table_size = GetExpectedTableSize(user_keys.size()); unique_ptr writable_file; fname = test::TmpDir() + "/WithCollisionPathUserKey"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - num_hash_fun, 2, BytewiseComparator(), 1, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder(file_writer.get(), kHashTableRatio, num_hash_fun, + 2, BytewiseComparator(), 1, false, false, + GetSliceHash, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint32_t i = 0; i < user_keys.size(); i++) { builder.Add(Slice(GetInternalKey(user_keys[i], true)), Slice(values[i])); ASSERT_EQ(builder.NumEntries(), i + 1); ASSERT_OK(builder.status()); } + size_t bucket_size = user_keys[0].size() + values[0].size(); + ASSERT_EQ(expected_table_size * bucket_size - 1, builder.FileSize()); ASSERT_OK(builder.Finish()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); + ASSERT_LE(expected_table_size * bucket_size, builder.FileSize()); - uint32_t expected_table_size = NextPowOf2(user_keys.size() / kHashTableRatio); std::string expected_unused_bucket = "key00"; expected_unused_bucket += std::string(values[0].size(), 'a'); CheckFileContents(user_keys, values, expected_locations, expected_unused_bucket, expected_table_size, 2, true); } -TEST(CuckooBuilderTest, FailWhenCollisionPathTooLong) { +TEST_F(CuckooBuilderTest, FailWhenCollisionPathTooLong) { // Have two hash functions. Insert elements with overlapping hashes. // Finally try inserting an element with hash value somewhere in the middle // and it should fail because the no. of elements to displace is too high. uint32_t num_hash_fun = 2; std::vector user_keys = {"key01", "key02", "key03", "key04", "key05"}; - hash_map = { - {user_keys[0], {0, 1}}, - {user_keys[1], {1, 2}}, - {user_keys[2], {2, 3}}, - {user_keys[3], {3, 4}}, - {user_keys[4], {0, 1}}, + // Need to have a temporary variable here as VS compiler does not currently + // support operator= with initializer_list as a parameter + std::unordered_map> hm = { + {user_keys[0], {0, 1}}, + {user_keys[1], {1, 2}}, + {user_keys[2], {2, 3}}, + {user_keys[3], {3, 4}}, + {user_keys[4], {0, 1}}, }; + hash_map = std::move(hm); unique_ptr writable_file; fname = test::TmpDir() + "/WithCollisionPathUserKey"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - num_hash_fun, 2, BytewiseComparator(), 1, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder(file_writer.get(), kHashTableRatio, num_hash_fun, + 2, BytewiseComparator(), 1, false, false, + GetSliceHash, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint32_t i = 0; i < user_keys.size(); i++) { builder.Add(Slice(GetInternalKey(user_keys[i], false)), Slice("value")); @@ -461,19 +581,27 @@ TEST(CuckooBuilderTest, FailWhenCollisionPathTooLong) { ASSERT_OK(builder.status()); } ASSERT_TRUE(builder.Finish().IsNotSupported()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); } -TEST(CuckooBuilderTest, FailWhenSameKeyInserted) { - hash_map = {{"repeatedkey", {0, 1, 2, 3}}}; +TEST_F(CuckooBuilderTest, FailWhenSameKeyInserted) { + // Need to have a temporary variable here as VS compiler does not currently + // support operator= with initializer_list as a parameter + std::unordered_map> hm = { + {"repeatedkey", {0, 1, 2, 3}}}; + hash_map = std::move(hm); uint32_t num_hash_fun = 4; std::string user_key = "repeatedkey"; unique_ptr writable_file; fname = test::TmpDir() + "/FailWhenSameKeyInserted"; ASSERT_OK(env_->NewWritableFile(fname, &writable_file, env_options_)); - CuckooTableBuilder builder(writable_file.get(), kHashTableRatio, - num_hash_fun, 100, BytewiseComparator(), 1, GetSliceHash); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), EnvOptions())); + CuckooTableBuilder builder(file_writer.get(), kHashTableRatio, num_hash_fun, + 100, BytewiseComparator(), 1, false, false, + GetSliceHash, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); builder.Add(Slice(GetInternalKey(user_key, false)), Slice("value1")); @@ -484,8 +612,21 @@ TEST(CuckooBuilderTest, FailWhenSameKeyInserted) { ASSERT_OK(builder.status()); ASSERT_TRUE(builder.Finish().IsNotSupported()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); } } // namespace rocksdb -int main(int argc, char** argv) { return rocksdb::test::RunAllTests(); } +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, "SKIPPED as Cuckoo table is not supported in ROCKSDB_LITE\n"); + return 0; +} + +#endif // ROCKSDB_LITE diff --git a/table/cuckoo_table_factory.cc b/table/cuckoo_table_factory.cc index e2cc6fd891c..2325bcf77c4 100644 --- a/table/cuckoo_table_factory.cc +++ b/table/cuckoo_table_factory.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef ROCKSDB_LITE #include "table/cuckoo_table_factory.h" @@ -11,12 +11,15 @@ #include "table/cuckoo_table_reader.h" namespace rocksdb { -Status CuckooTableFactory::NewTableReader(const Options& options, - const EnvOptions& soptions, const InternalKeyComparator& icomp, - std::unique_ptr&& file, uint64_t file_size, - std::unique_ptr* table) const { - std::unique_ptr new_reader(new CuckooTableReader(options, - std::move(file), file_size, icomp.user_comparator(), nullptr)); + +Status CuckooTableFactory::NewTableReader( + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + std::unique_ptr* table, + bool prefetch_index_and_filter_in_cache) const { + std::unique_ptr new_reader(new CuckooTableReader( + table_reader_options.ioptions, std::move(file), file_size, + table_reader_options.internal_comparator.user_comparator(), nullptr)); Status s = new_reader->status(); if (s.ok()) { *table = std::move(new_reader); @@ -25,10 +28,19 @@ Status CuckooTableFactory::NewTableReader(const Options& options, } TableBuilder* CuckooTableFactory::NewTableBuilder( - const Options& options, const InternalKeyComparator& internal_comparator, - WritableFile* file, CompressionType compression_type) const { - return new CuckooTableBuilder(file, hash_table_ratio_, 64, max_search_depth_, - internal_comparator.user_comparator(), cuckoo_block_size_, nullptr); + const TableBuilderOptions& table_builder_options, uint32_t column_family_id, + WritableFileWriter* file) const { + // Ignore the skipFIlters flag. Does not apply to this file format + // + + // TODO: change builder to take the option struct + return new CuckooTableBuilder( + file, table_options_.hash_table_ratio, 64, + table_options_.max_search_depth, + table_builder_options.internal_comparator.user_comparator(), + table_options_.cuckoo_block_size, table_options_.use_module_hash, + table_options_.identity_as_first_hash, nullptr /* get_slice_hash */, + column_family_id, table_builder_options.column_family_name); } std::string CuckooTableFactory::GetPrintableTableOptions() const { @@ -38,21 +50,22 @@ std::string CuckooTableFactory::GetPrintableTableOptions() const { char buffer[kBufferSize]; snprintf(buffer, kBufferSize, " hash_table_ratio: %lf\n", - hash_table_ratio_); + table_options_.hash_table_ratio); ret.append(buffer); snprintf(buffer, kBufferSize, " max_search_depth: %u\n", - max_search_depth_); + table_options_.max_search_depth); ret.append(buffer); snprintf(buffer, kBufferSize, " cuckoo_block_size: %u\n", - cuckoo_block_size_); + table_options_.cuckoo_block_size); + ret.append(buffer); + snprintf(buffer, kBufferSize, " identity_as_first_hash: %d\n", + table_options_.identity_as_first_hash); ret.append(buffer); return ret; } -TableFactory* NewCuckooTableFactory(double hash_table_ratio, - uint32_t max_search_depth, uint32_t cuckoo_block_size) { - return new CuckooTableFactory( - hash_table_ratio, max_search_depth, cuckoo_block_size); +TableFactory* NewCuckooTableFactory(const CuckooTableOptions& table_options) { + return new CuckooTableFactory(table_options); } } // namespace rocksdb diff --git a/table/cuckoo_table_factory.h b/table/cuckoo_table_factory.h index 5799a7f23fe..db860c3d002 100644 --- a/table/cuckoo_table_factory.h +++ b/table/cuckoo_table_factory.h @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #ifndef ROCKSDB_LITE @@ -9,21 +9,35 @@ #include #include "rocksdb/table.h" #include "util/murmurhash.h" +#include "rocksdb/options.h" namespace rocksdb { const uint32_t kCuckooMurmurSeedMultiplier = 816922183; static inline uint64_t CuckooHash( - const Slice& user_key, uint32_t hash_cnt, uint64_t table_size_minus_one, + const Slice& user_key, uint32_t hash_cnt, bool use_module_hash, + uint64_t table_size_, bool identity_as_first_hash, uint64_t (*get_slice_hash)(const Slice&, uint32_t, uint64_t)) { -#ifndef NDEBUG - // This part is used only in unit tests. +#if !defined NDEBUG || defined OS_WIN + // This part is used only in unit tests but we have to keep it for Windows + // build as we run test in both debug and release modes under Windows. if (get_slice_hash != nullptr) { - return get_slice_hash(user_key, hash_cnt, table_size_minus_one + 1); + return get_slice_hash(user_key, hash_cnt, table_size_); } #endif - return MurmurHash(user_key.data(), user_key.size(), - kCuckooMurmurSeedMultiplier * hash_cnt) & table_size_minus_one; + + uint64_t value = 0; + if (hash_cnt == 0 && identity_as_first_hash) { + value = (*reinterpret_cast(user_key.data())); + } else { + value = MurmurHash(user_key.data(), static_cast(user_key.size()), + kCuckooMurmurSeedMultiplier * hash_cnt); + } + if (use_module_hash) { + return value % table_size_; + } else { + return value & (table_size_ - 1); + } } // Cuckoo Table is designed for applications that require fast point lookups @@ -33,38 +47,42 @@ static inline uint64_t CuckooHash( // - Key length and Value length are fixed. // - Does not support Snapshot. // - Does not support Merge operations. +// - Does not support prefix bloom filters. class CuckooTableFactory : public TableFactory { public: - CuckooTableFactory(double hash_table_ratio, uint32_t max_search_depth, - uint32_t cuckoo_block_size) - : hash_table_ratio_(hash_table_ratio), - max_search_depth_(max_search_depth), - cuckoo_block_size_(cuckoo_block_size) {} + explicit CuckooTableFactory(const CuckooTableOptions& table_options) + : table_options_(table_options) {} ~CuckooTableFactory() {} const char* Name() const override { return "CuckooTable"; } Status NewTableReader( - const Options& options, const EnvOptions& soptions, - const InternalKeyComparator& internal_comparator, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table) const override; + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + unique_ptr* table, + bool prefetch_index_and_filter_in_cache = true) const override; - TableBuilder* NewTableBuilder(const Options& options, - const InternalKeyComparator& icomparator, WritableFile* file, - CompressionType compression_type) const override; + TableBuilder* NewTableBuilder( + const TableBuilderOptions& table_builder_options, + uint32_t column_family_id, WritableFileWriter* file) const override; // Sanitizes the specified DB Options. - Status SanitizeDBOptions(const DBOptions* db_opts) const override { + Status SanitizeOptions(const DBOptions& db_opts, + const ColumnFamilyOptions& cf_opts) const override { return Status::OK(); } std::string GetPrintableTableOptions() const override; + void* GetOptions() override { return &table_options_; } + + Status GetOptionString(std::string* opt_string, + const std::string& delimiter) const override { + return Status::OK(); + } + private: - const double hash_table_ratio_; - const uint32_t max_search_depth_; - const uint32_t cuckoo_block_size_; + CuckooTableOptions table_options_; }; } // namespace rocksdb diff --git a/table/cuckoo_table_reader.cc b/table/cuckoo_table_reader.cc index f1dcbc3bb63..9cecebaebb7 100644 --- a/table/cuckoo_table_reader.cc +++ b/table/cuckoo_table_reader.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -16,33 +16,36 @@ #include #include #include "rocksdb/iterator.h" +#include "rocksdb/table.h" +#include "table/internal_iterator.h" #include "table/meta_blocks.h" #include "table/cuckoo_table_factory.h" +#include "table/get_context.h" #include "util/arena.h" #include "util/coding.h" namespace rocksdb { namespace { - static const uint64_t CACHE_LINE_MASK = ~((uint64_t)CACHE_LINE_SIZE - 1); +const uint64_t CACHE_LINE_MASK = ~((uint64_t)CACHE_LINE_SIZE - 1); +const uint32_t kInvalidIndex = std::numeric_limits::max(); } extern const uint64_t kCuckooTableMagicNumber; CuckooTableReader::CuckooTableReader( - const Options& options, - std::unique_ptr&& file, - uint64_t file_size, + const ImmutableCFOptions& ioptions, + std::unique_ptr&& file, uint64_t file_size, const Comparator* comparator, uint64_t (*get_slice_hash)(const Slice&, uint32_t, uint64_t)) : file_(std::move(file)), ucomp_(comparator), get_slice_hash_(get_slice_hash) { - if (!options.allow_mmap_reads) { + if (!ioptions.allow_mmap_reads) { status_ = Status::InvalidArgument("File is not mmaped"); } TableProperties* props = nullptr; status_ = ReadTableProperties(file_.get(), file_size, kCuckooTableMagicNumber, - options.env, options.info_log.get(), &props); + ioptions, &props); if (!status_.ok()) { return; } @@ -50,21 +53,29 @@ CuckooTableReader::CuckooTableReader( auto& user_props = props->user_collected_properties; auto hash_funs = user_props.find(CuckooTablePropertyNames::kNumHashFunc); if (hash_funs == user_props.end()) { - status_ = Status::InvalidArgument("Number of hash functions not found"); + status_ = Status::Corruption("Number of hash functions not found"); return; } num_hash_func_ = *reinterpret_cast(hash_funs->second.data()); auto unused_key = user_props.find(CuckooTablePropertyNames::kEmptyKey); if (unused_key == user_props.end()) { - status_ = Status::InvalidArgument("Empty bucket value not found"); + status_ = Status::Corruption("Empty bucket value not found"); return; } unused_key_ = unused_key->second; - key_length_ = props->fixed_key_len; + key_length_ = static_cast(props->fixed_key_len); + auto user_key_len = user_props.find(CuckooTablePropertyNames::kUserKeyLength); + if (user_key_len == user_props.end()) { + status_ = Status::Corruption("User key length not found"); + return; + } + user_key_length_ = *reinterpret_cast( + user_key_len->second.data()); + auto value_length = user_props.find(CuckooTablePropertyNames::kValueLength); if (value_length == user_props.end()) { - status_ = Status::InvalidArgument("Value length not found"); + status_ = Status::Corruption("Value length not found"); return; } value_length_ = *reinterpret_cast( @@ -74,21 +85,40 @@ CuckooTableReader::CuckooTableReader( auto hash_table_size = user_props.find( CuckooTablePropertyNames::kHashTableSize); if (hash_table_size == user_props.end()) { - status_ = Status::InvalidArgument("Hash table size not found"); + status_ = Status::Corruption("Hash table size not found"); return; } - table_size_minus_one_ = *reinterpret_cast( - hash_table_size->second.data()) - 1; + table_size_ = *reinterpret_cast( + hash_table_size->second.data()); + auto is_last_level = user_props.find(CuckooTablePropertyNames::kIsLastLevel); if (is_last_level == user_props.end()) { - status_ = Status::InvalidArgument("Is last level not found"); + status_ = Status::Corruption("Is last level not found"); return; } is_last_level_ = *reinterpret_cast(is_last_level->second.data()); + + auto identity_as_first_hash = user_props.find( + CuckooTablePropertyNames::kIdentityAsFirstHash); + if (identity_as_first_hash == user_props.end()) { + status_ = Status::Corruption("identity as first hash not found"); + return; + } + identity_as_first_hash_ = *reinterpret_cast( + identity_as_first_hash->second.data()); + + auto use_module_hash = user_props.find( + CuckooTablePropertyNames::kUseModuleHash); + if (use_module_hash == user_props.end()) { + status_ = Status::Corruption("hash type is not found"); + return; + } + use_module_hash_ = *reinterpret_cast( + use_module_hash->second.data()); auto cuckoo_block_size = user_props.find( CuckooTablePropertyNames::kCuckooBlockSize); if (cuckoo_block_size == user_props.end()) { - status_ = Status::InvalidArgument("Cuckoo block size not found"); + status_ = Status::Corruption("Cuckoo block size not found"); return; } cuckoo_block_size_ = *reinterpret_cast( @@ -97,36 +127,37 @@ CuckooTableReader::CuckooTableReader( status_ = file_->Read(0, file_size, &file_data_, nullptr); } -Status CuckooTableReader::Get( - const ReadOptions& readOptions, const Slice& key, void* handle_context, - bool (*result_handler)(void* arg, const ParsedInternalKey& k, - const Slice& v), - void (*mark_key_may_exist_handler)(void* handle_context)) { +Status CuckooTableReader::Get(const ReadOptions& readOptions, const Slice& key, + GetContext* get_context, bool skip_filters) { assert(key.size() == key_length_ + (is_last_level_ ? 8 : 0)); Slice user_key = ExtractUserKey(key); for (uint32_t hash_cnt = 0; hash_cnt < num_hash_func_; ++hash_cnt) { uint64_t offset = bucket_length_ * CuckooHash( - user_key, hash_cnt, table_size_minus_one_, get_slice_hash_); + user_key, hash_cnt, use_module_hash_, table_size_, + identity_as_first_hash_, get_slice_hash_); const char* bucket = &file_data_.data()[offset]; for (uint32_t block_idx = 0; block_idx < cuckoo_block_size_; - ++block_idx, bucket += bucket_length_) { - if (ucomp_->Compare(Slice(unused_key_.data(), user_key.size()), - Slice(bucket, user_key.size())) == 0) { + ++block_idx, bucket += bucket_length_) { + if (ucomp_->Equal(Slice(unused_key_.data(), user_key.size()), + Slice(bucket, user_key.size()))) { return Status::OK(); } // Here, we compare only the user key part as we support only one entry - // per user key and we don't support sanpshot. - if (ucomp_->Compare(user_key, Slice(bucket, user_key.size())) == 0) { - Slice value = Slice(&bucket[key_length_], value_length_); + // per user key and we don't support snapshot. + if (ucomp_->Equal(user_key, Slice(bucket, user_key.size()))) { + Slice value(bucket + key_length_, value_length_); if (is_last_level_) { - ParsedInternalKey found_ikey( - Slice(bucket, key_length_), 0, kTypeValue); - result_handler(handle_context, found_ikey, value); + // Sequence number is not stored at the last level, so we will use + // kMaxSequenceNumber since it is unknown. This could cause some + // transactions to fail to lock a key due to known sequence number. + // However, it is expected for anyone to use a CuckooTable in a + // TransactionDB. + get_context->SaveValue(value, kMaxSequenceNumber); } else { Slice full_key(bucket, key_length_); ParsedInternalKey found_ikey; ParseInternalKey(full_key, &found_ikey); - result_handler(handle_context, found_ikey, value); + get_context->SaveValue(found_ikey, value); } // We don't support merge operations. So, we return here. return Status::OK(); @@ -140,14 +171,15 @@ void CuckooTableReader::Prepare(const Slice& key) { // Prefetch the first Cuckoo Block. Slice user_key = ExtractUserKey(key); uint64_t addr = reinterpret_cast(file_data_.data()) + - bucket_length_ * CuckooHash(user_key, 0, table_size_minus_one_, nullptr); + bucket_length_ * CuckooHash(user_key, 0, use_module_hash_, table_size_, + identity_as_first_hash_, nullptr); uint64_t end_addr = addr + cuckoo_block_bytes_minus_one_; for (addr &= CACHE_LINE_MASK; addr < end_addr; addr += CACHE_LINE_SIZE) { PREFETCH(reinterpret_cast(addr), 0, 3); } } -class CuckooTableIterator : public Iterator { +class CuckooTableIterator : public InternalIterator { public: explicit CuckooTableIterator(CuckooTableReader* reader); ~CuckooTableIterator() {} @@ -155,38 +187,49 @@ class CuckooTableIterator : public Iterator { void SeekToFirst() override; void SeekToLast() override; void Seek(const Slice& target) override; + void SeekForPrev(const Slice& target) override; void Next() override; void Prev() override; Slice key() const override; Slice value() const override; Status status() const override { return status_; } - void LoadKeysFromReader(); + void InitIfNeeded(); private: - struct CompareKeys { - CompareKeys(const Comparator* ucomp, const bool last_level) - : ucomp_(ucomp), - is_last_level_(last_level) {} - bool operator()(const std::pair& first, - const std::pair& second) const { - if (is_last_level_) { - return ucomp_->Compare(first.first, second.first) < 0; - } else { - return ucomp_->Compare(ExtractUserKey(first.first), - ExtractUserKey(second.first)) < 0; - } + struct BucketComparator { + BucketComparator(const Slice& file_data, const Comparator* ucomp, + uint32_t bucket_len, uint32_t user_key_len, + const Slice& target = Slice()) + : file_data_(file_data), + ucomp_(ucomp), + bucket_len_(bucket_len), + user_key_len_(user_key_len), + target_(target) {} + bool operator()(const uint32_t first, const uint32_t second) const { + const char* first_bucket = + (first == kInvalidIndex) ? target_.data() : + &file_data_.data()[first * bucket_len_]; + const char* second_bucket = + (second == kInvalidIndex) ? target_.data() : + &file_data_.data()[second * bucket_len_]; + return ucomp_->Compare(Slice(first_bucket, user_key_len_), + Slice(second_bucket, user_key_len_)) < 0; } - private: + const Slice file_data_; const Comparator* ucomp_; - const bool is_last_level_; + const uint32_t bucket_len_; + const uint32_t user_key_len_; + const Slice target_; }; - const CompareKeys comparator_; + + const BucketComparator bucket_comparator_; void PrepareKVAtCurrIdx(); CuckooTableReader* reader_; + bool initialized_; Status status_; // Contains a map of keys to bucket_id sorted in key order. - std::vector> key_to_bucket_id_; + std::vector sorted_bucket_ids_; // We assume that the number of items can be stored in uint32 (4 Billion). uint32_t curr_key_idx_; Slice curr_value_; @@ -197,57 +240,72 @@ class CuckooTableIterator : public Iterator { }; CuckooTableIterator::CuckooTableIterator(CuckooTableReader* reader) - : comparator_(reader->ucomp_, reader->is_last_level_), + : bucket_comparator_(reader->file_data_, reader->ucomp_, + reader->bucket_length_, reader->user_key_length_), reader_(reader), - curr_key_idx_(std::numeric_limits::max()) { - key_to_bucket_id_.clear(); + initialized_(false), + curr_key_idx_(kInvalidIndex) { + sorted_bucket_ids_.clear(); curr_value_.clear(); curr_key_.Clear(); } -void CuckooTableIterator::LoadKeysFromReader() { - key_to_bucket_id_.reserve(reader_->GetTableProperties()->num_entries); - uint64_t num_buckets = reader_->table_size_minus_one_ + - reader_->cuckoo_block_size_; - for (uint32_t bucket_id = 0; bucket_id < num_buckets; bucket_id++) { - Slice read_key; - status_ = reader_->file_->Read(bucket_id * reader_->bucket_length_, - reader_->key_length_, &read_key, nullptr); - if (read_key != Slice(reader_->unused_key_)) { - key_to_bucket_id_.push_back(std::make_pair(read_key, bucket_id)); +void CuckooTableIterator::InitIfNeeded() { + if (initialized_) { + return; + } + sorted_bucket_ids_.reserve(reader_->GetTableProperties()->num_entries); + uint64_t num_buckets = reader_->table_size_ + reader_->cuckoo_block_size_ - 1; + assert(num_buckets < kInvalidIndex); + const char* bucket = reader_->file_data_.data(); + for (uint32_t bucket_id = 0; bucket_id < num_buckets; ++bucket_id) { + if (Slice(bucket, reader_->key_length_) != Slice(reader_->unused_key_)) { + sorted_bucket_ids_.push_back(bucket_id); } + bucket += reader_->bucket_length_; } - assert(key_to_bucket_id_.size() == + assert(sorted_bucket_ids_.size() == reader_->GetTableProperties()->num_entries); - std::sort(key_to_bucket_id_.begin(), key_to_bucket_id_.end(), comparator_); - curr_key_idx_ = key_to_bucket_id_.size(); + std::sort(sorted_bucket_ids_.begin(), sorted_bucket_ids_.end(), + bucket_comparator_); + curr_key_idx_ = kInvalidIndex; + initialized_ = true; } void CuckooTableIterator::SeekToFirst() { + InitIfNeeded(); curr_key_idx_ = 0; PrepareKVAtCurrIdx(); } void CuckooTableIterator::SeekToLast() { - curr_key_idx_ = key_to_bucket_id_.size() - 1; + InitIfNeeded(); + curr_key_idx_ = static_cast(sorted_bucket_ids_.size()) - 1; PrepareKVAtCurrIdx(); } void CuckooTableIterator::Seek(const Slice& target) { - // We assume that the target is an internal key. If this is last level file, - // we need to take only the user key part to seek. - Slice target_to_search = reader_->is_last_level_ ? - ExtractUserKey(target) : target; - auto seek_it = std::lower_bound(key_to_bucket_id_.begin(), - key_to_bucket_id_.end(), - std::make_pair(target_to_search, 0), - comparator_); - curr_key_idx_ = std::distance(key_to_bucket_id_.begin(), seek_it); + InitIfNeeded(); + const BucketComparator seek_comparator( + reader_->file_data_, reader_->ucomp_, + reader_->bucket_length_, reader_->user_key_length_, + ExtractUserKey(target)); + auto seek_it = std::lower_bound(sorted_bucket_ids_.begin(), + sorted_bucket_ids_.end(), + kInvalidIndex, + seek_comparator); + curr_key_idx_ = + static_cast(std::distance(sorted_bucket_ids_.begin(), seek_it)); PrepareKVAtCurrIdx(); } +void CuckooTableIterator::SeekForPrev(const Slice& target) { + // Not supported + assert(false); +} + bool CuckooTableIterator::Valid() const { - return curr_key_idx_ < key_to_bucket_id_.size(); + return curr_key_idx_ < sorted_bucket_ids_.size(); } void CuckooTableIterator::PrepareKVAtCurrIdx() { @@ -256,15 +314,17 @@ void CuckooTableIterator::PrepareKVAtCurrIdx() { curr_key_.Clear(); return; } - uint64_t offset = ((uint64_t) key_to_bucket_id_[curr_key_idx_].second - * reader_->bucket_length_) + reader_->key_length_; - status_ = reader_->file_->Read(offset, reader_->value_length_, - &curr_value_, nullptr); + uint32_t id = sorted_bucket_ids_[curr_key_idx_]; + const char* offset = reader_->file_data_.data() + + id * reader_->bucket_length_; if (reader_->is_last_level_) { // Always return internal key. - curr_key_.SetInternalKey( - key_to_bucket_id_[curr_key_idx_].first, 0, kTypeValue); + curr_key_.SetInternalKey(Slice(offset, reader_->user_key_length_), + 0, kTypeValue); + } else { + curr_key_.SetInternalKey(Slice(offset, reader_->key_length_)); } + curr_value_ = Slice(offset + reader_->key_length_, reader_->value_length_); } void CuckooTableIterator::Next() { @@ -279,7 +339,7 @@ void CuckooTableIterator::Next() { void CuckooTableIterator::Prev() { if (curr_key_idx_ == 0) { - curr_key_idx_ = key_to_bucket_id_.size(); + curr_key_idx_ = static_cast(sorted_bucket_ids_.size()); } if (!Valid()) { curr_value_.clear(); @@ -292,11 +352,7 @@ void CuckooTableIterator::Prev() { Slice CuckooTableIterator::key() const { assert(Valid()); - if (reader_->is_last_level_) { - return curr_key_.GetKey(); - } else { - return key_to_bucket_id_[curr_key_idx_].first; - } + return curr_key_.GetInternalKey(); } Slice CuckooTableIterator::value() const { @@ -304,18 +360,15 @@ Slice CuckooTableIterator::value() const { return curr_value_; } -extern Iterator* NewErrorIterator(const Status& status, Arena* arena); +extern InternalIterator* NewErrorInternalIterator(const Status& status, + Arena* arena); -Iterator* CuckooTableReader::NewIterator( - const ReadOptions& read_options, Arena* arena) { +InternalIterator* CuckooTableReader::NewIterator( + const ReadOptions& read_options, Arena* arena, bool skip_filters) { if (!status().ok()) { - return NewErrorIterator( + return NewErrorInternalIterator( Status::Corruption("CuckooTableReader status is not okay."), arena); } - if (read_options.total_order_seek) { - return NewErrorIterator( - Status::InvalidArgument("total_order_seek is not supported."), arena); - } CuckooTableIterator* iter; if (arena == nullptr) { iter = new CuckooTableIterator(this); @@ -323,9 +376,6 @@ Iterator* CuckooTableReader::NewIterator( auto iter_mem = arena->AllocateAligned(sizeof(CuckooTableIterator)); iter = new (iter_mem) CuckooTableIterator(this); } - if (iter->status().ok()) { - iter->LoadKeysFromReader(); - } return iter; } diff --git a/table/cuckoo_table_reader.h b/table/cuckoo_table_reader.h index 05d5c339787..4beac8f9d07 100644 --- a/table/cuckoo_table_reader.h +++ b/table/cuckoo_table_reader.h @@ -1,7 +1,7 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -15,22 +15,25 @@ #include #include "db/dbformat.h" +#include "options/cf_options.h" #include "rocksdb/env.h" +#include "rocksdb/options.h" #include "table/table_reader.h" +#include "util/file_reader_writer.h" namespace rocksdb { class Arena; class TableReader; +class InternalIterator; class CuckooTableReader: public TableReader { public: - CuckooTableReader( - const Options& options, - std::unique_ptr&& file, - uint64_t file_size, - const Comparator* user_comparator, - uint64_t (*get_slice_hash)(const Slice&, uint32_t, uint64_t)); + CuckooTableReader(const ImmutableCFOptions& ioptions, + std::unique_ptr&& file, + uint64_t file_size, const Comparator* user_comparator, + uint64_t (*get_slice_hash)(const Slice&, uint32_t, + uint64_t)); ~CuckooTableReader() {} std::shared_ptr GetTableProperties() const override { @@ -39,14 +42,12 @@ class CuckooTableReader: public TableReader { Status status() const { return status_; } - Status Get( - const ReadOptions& readOptions, const Slice& key, void* handle_context, - bool (*result_handler)(void* arg, const ParsedInternalKey& k, - const Slice& v), - void (*mark_key_may_exist_handler)(void* handle_context) = nullptr) - override; + Status Get(const ReadOptions& read_options, const Slice& key, + GetContext* get_context, bool skip_filters = false) override; - Iterator* NewIterator(const ReadOptions&, Arena* arena = nullptr) override; + InternalIterator* NewIterator( + const ReadOptions&, Arena* arena = nullptr, + bool skip_filters = false) override; void Prepare(const Slice& target) override; // Report an approximation of how much memory has been used. @@ -60,19 +61,22 @@ class CuckooTableReader: public TableReader { private: friend class CuckooTableIterator; void LoadAllKeys(std::vector>* key_to_bucket_id); - std::unique_ptr file_; + std::unique_ptr file_; Slice file_data_; bool is_last_level_; + bool identity_as_first_hash_; + bool use_module_hash_; std::shared_ptr table_props_; Status status_; uint32_t num_hash_func_; std::string unused_key_; uint32_t key_length_; + uint32_t user_key_length_; uint32_t value_length_; uint32_t bucket_length_; uint32_t cuckoo_block_size_; uint32_t cuckoo_block_bytes_minus_one_; - uint64_t table_size_minus_one_; + uint64_t table_size_; const Comparator* ucomp_; uint64_t (*get_slice_hash_)(const Slice& s, uint32_t index, uint64_t max_num_buckets); diff --git a/table/cuckoo_table_reader_test.cc b/table/cuckoo_table_reader_test.cc index 63fe0ae5ba1..7e131e56e31 100644 --- a/table/cuckoo_table_reader_test.cc +++ b/table/cuckoo_table_reader_test.cc @@ -1,17 +1,22 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE #ifndef GFLAGS #include int main() { - fprintf(stderr, "Please install gflags to run this test\n"); - return 1; + fprintf(stderr, "Please install gflags to run this test... Skipping...\n"); + return 0; } #else +#ifndef __STDC_FORMAT_MACROS #define __STDC_FORMAT_MACROS +#endif + #include #include #include @@ -22,8 +27,10 @@ int main() { #include "table/cuckoo_table_builder.h" #include "table/cuckoo_table_reader.h" #include "table/cuckoo_table_factory.h" +#include "table/get_context.h" #include "util/arena.h" #include "util/random.h" +#include "util/string_util.h" #include "util/testharness.h" #include "util/testutil.h" @@ -35,6 +42,7 @@ DEFINE_string(file_dir, "", "Directory where the files will be created" DEFINE_bool(enable_perf, false, "Run Benchmark Tests too."); DEFINE_bool(write, false, "Should write new values to file in performance tests?"); +DEFINE_bool(identity_as_first_hash, true, "use identity as first hash"); namespace rocksdb { @@ -56,38 +64,20 @@ uint64_t GetSliceHash(const Slice& s, uint32_t index, uint64_t max_num_buckets) { return hash_map[s.ToString()][index]; } - -// Methods, variables for checking key and values read. -struct ValuesToAssert { - ValuesToAssert(const std::string& key, const Slice& value) - : expected_user_key(key), - expected_value(value), - call_count(0) {} - std::string expected_user_key; - Slice expected_value; - int call_count; -}; - -bool AssertValues(void* assert_obj, - const ParsedInternalKey& k, const Slice& v) { - ValuesToAssert *ptr = reinterpret_cast(assert_obj); - ASSERT_EQ(ptr->expected_value.ToString(), v.ToString()); - ASSERT_EQ(ptr->expected_user_key, k.user_key.ToString()); - ++ptr->call_count; - return false; -} } // namespace -class CuckooReaderTest { +class CuckooReaderTest : public testing::Test { public: + using testing::Test::SetUp; + CuckooReaderTest() { options.allow_mmap_reads = true; env = options.env; env_options = EnvOptions(options); } - void SetUp(int num_items) { - this->num_items = num_items; + void SetUp(int num) { + num_items = num; hash_map.clear(); keys.clear(); keys.resize(num_items); @@ -105,8 +95,12 @@ class CuckooReaderTest { const Comparator* ucomp = BytewiseComparator()) { std::unique_ptr writable_file; ASSERT_OK(env->NewWritableFile(fname, &writable_file, env_options)); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), env_options)); + CuckooTableBuilder builder( - writable_file.get(), 0.9, kNumHashFunc, 100, ucomp, 2, GetSliceHash); + file_writer.get(), 0.9, kNumHashFunc, 100, ucomp, 2, false, false, + GetSliceHash, 0 /* column_family_id */, kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint32_t key_idx = 0; key_idx < num_items; ++key_idx) { builder.Add(Slice(keys[key_idx]), Slice(values[key_idx])); @@ -116,23 +110,25 @@ class CuckooReaderTest { ASSERT_OK(builder.Finish()); ASSERT_EQ(num_items, builder.NumEntries()); file_size = builder.FileSize(); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); // Check reader now. std::unique_ptr read_file; ASSERT_OK(env->NewRandomAccessFile(fname, &read_file, env_options)); - CuckooTableReader reader( - options, - std::move(read_file), - file_size, - ucomp, - GetSliceHash); + unique_ptr file_reader( + new RandomAccessFileReader(std::move(read_file), fname)); + const ImmutableCFOptions ioptions(options); + CuckooTableReader reader(ioptions, std::move(file_reader), file_size, ucomp, + GetSliceHash); ASSERT_OK(reader.status()); + // Assume no merge/deletion for (uint32_t i = 0; i < num_items; ++i) { - ValuesToAssert v(user_keys[i], values[i]); - ASSERT_OK(reader.Get( - ReadOptions(), Slice(keys[i]), &v, AssertValues, nullptr)); - ASSERT_EQ(1, v.call_count); + PinnableSlice value; + GetContext get_context(ucomp, nullptr, nullptr, nullptr, + GetContext::kNotFound, Slice(user_keys[i]), &value, + nullptr, nullptr, nullptr, nullptr); + ASSERT_OK(reader.Get(ReadOptions(), Slice(keys[i]), &get_context)); + ASSERT_STREQ(values[i].c_str(), value.data()); } } void UpdateKeys(bool with_zero_seqno) { @@ -147,14 +143,13 @@ class CuckooReaderTest { void CheckIterator(const Comparator* ucomp = BytewiseComparator()) { std::unique_ptr read_file; ASSERT_OK(env->NewRandomAccessFile(fname, &read_file, env_options)); - CuckooTableReader reader( - options, - std::move(read_file), - file_size, - ucomp, - GetSliceHash); + unique_ptr file_reader( + new RandomAccessFileReader(std::move(read_file), fname)); + const ImmutableCFOptions ioptions(options); + CuckooTableReader reader(ioptions, std::move(file_reader), file_size, ucomp, + GetSliceHash); ASSERT_OK(reader.status()); - Iterator* it = reader.NewIterator(ReadOptions(), nullptr); + InternalIterator* it = reader.NewIterator(ReadOptions(), nullptr); ASSERT_OK(it->status()); ASSERT_TRUE(!it->Valid()); it->SeekToFirst(); @@ -169,7 +164,7 @@ class CuckooReaderTest { ASSERT_EQ(static_cast(cnt), num_items); it->SeekToLast(); - cnt = num_items - 1; + cnt = static_cast(num_items) - 1; ASSERT_TRUE(it->Valid()); while (it->Valid()) { ASSERT_OK(it->status()); @@ -180,7 +175,7 @@ class CuckooReaderTest { } ASSERT_EQ(cnt, -1); - cnt = num_items / 2; + cnt = static_cast(num_items) / 2; it->Seek(keys[cnt]); while (it->Valid()) { ASSERT_OK(it->status()); @@ -202,7 +197,7 @@ class CuckooReaderTest { ASSERT_TRUE(keys[num_items/2] == it->key()); ASSERT_TRUE(values[num_items/2] == it->value()); ASSERT_OK(it->status()); - it->~Iterator(); + it->~InternalIterator(); } std::vector keys; @@ -216,7 +211,7 @@ class CuckooReaderTest { EnvOptions env_options; }; -TEST(CuckooReaderTest, WhenKeyExists) { +TEST_F(CuckooReaderTest, WhenKeyExists) { SetUp(kNumHashFunc); fname = test::TmpDir() + "/CuckooReader_WhenKeyExists"; for (uint64_t i = 0; i < num_items; i++) { @@ -243,7 +238,7 @@ TEST(CuckooReaderTest, WhenKeyExists) { CreateCuckooFileAndCheckReader(); } -TEST(CuckooReaderTest, WhenKeyExistsWithUint64Comparator) { +TEST_F(CuckooReaderTest, WhenKeyExistsWithUint64Comparator) { SetUp(kNumHashFunc); fname = test::TmpDir() + "/CuckooReaderUint64_WhenKeyExists"; for (uint64_t i = 0; i < num_items; i++) { @@ -271,7 +266,7 @@ TEST(CuckooReaderTest, WhenKeyExistsWithUint64Comparator) { CreateCuckooFileAndCheckReader(test::Uint64Comparator()); } -TEST(CuckooReaderTest, CheckIterator) { +TEST_F(CuckooReaderTest, CheckIterator) { SetUp(2*kNumHashFunc); fname = test::TmpDir() + "/CuckooReader_CheckIterator"; for (uint64_t i = 0; i < num_items; i++) { @@ -290,7 +285,7 @@ TEST(CuckooReaderTest, CheckIterator) { CheckIterator(); } -TEST(CuckooReaderTest, CheckIteratorUint64) { +TEST_F(CuckooReaderTest, CheckIteratorUint64) { SetUp(2*kNumHashFunc); fname = test::TmpDir() + "/CuckooReader_CheckIterator"; for (uint64_t i = 0; i < num_items; i++) { @@ -310,7 +305,7 @@ TEST(CuckooReaderTest, CheckIteratorUint64) { CheckIterator(test::Uint64Comparator()); } -TEST(CuckooReaderTest, WhenKeyNotFound) { +TEST_F(CuckooReaderTest, WhenKeyNotFound) { // Add keys with colliding hash values. SetUp(kNumHashFunc); fname = test::TmpDir() + "/CuckooReader_WhenKeyNotFound"; @@ -322,15 +317,15 @@ TEST(CuckooReaderTest, WhenKeyNotFound) { // Make all hash values collide. AddHashLookups(user_keys[i], 0, kNumHashFunc); } + auto* ucmp = BytewiseComparator(); CreateCuckooFileAndCheckReader(); std::unique_ptr read_file; ASSERT_OK(env->NewRandomAccessFile(fname, &read_file, env_options)); - CuckooTableReader reader( - options, - std::move(read_file), - file_size, - BytewiseComparator(), - GetSliceHash); + unique_ptr file_reader( + new RandomAccessFileReader(std::move(read_file), fname)); + const ImmutableCFOptions ioptions(options); + CuckooTableReader reader(ioptions, std::move(file_reader), file_size, ucmp, + GetSliceHash); ASSERT_OK(reader.status()); // Search for a key with colliding hash values. std::string not_found_user_key = "key" + NumToStr(num_items); @@ -338,10 +333,12 @@ TEST(CuckooReaderTest, WhenKeyNotFound) { AddHashLookups(not_found_user_key, 0, kNumHashFunc); ParsedInternalKey ikey(not_found_user_key, 1000, kTypeValue); AppendInternalKey(¬_found_key, ikey); - ValuesToAssert v("", ""); - ASSERT_OK(reader.Get( - ReadOptions(), Slice(not_found_key), &v, AssertValues, nullptr)); - ASSERT_EQ(0, v.call_count); + PinnableSlice value; + GetContext get_context(ucmp, nullptr, nullptr, nullptr, GetContext::kNotFound, + Slice(not_found_key), &value, nullptr, nullptr, + nullptr, nullptr); + ASSERT_OK(reader.Get(ReadOptions(), Slice(not_found_key), &get_context)); + ASSERT_TRUE(value.empty()); ASSERT_OK(reader.status()); // Search for a key with an independent hash value. std::string not_found_user_key2 = "key" + NumToStr(num_items + 1); @@ -349,9 +346,12 @@ TEST(CuckooReaderTest, WhenKeyNotFound) { ParsedInternalKey ikey2(not_found_user_key2, 1000, kTypeValue); std::string not_found_key2; AppendInternalKey(¬_found_key2, ikey2); - ASSERT_OK(reader.Get( - ReadOptions(), Slice(not_found_key2), &v, AssertValues, nullptr)); - ASSERT_EQ(0, v.call_count); + value.Reset(); + GetContext get_context2(ucmp, nullptr, nullptr, nullptr, + GetContext::kNotFound, Slice(not_found_key2), &value, + nullptr, nullptr, nullptr, nullptr); + ASSERT_OK(reader.Get(ReadOptions(), Slice(not_found_key2), &get_context2)); + ASSERT_TRUE(value.empty()); ASSERT_OK(reader.status()); // Test read when key is unused key. @@ -361,34 +361,26 @@ TEST(CuckooReaderTest, WhenKeyNotFound) { // Add hash values that map to empty buckets. AddHashLookups(ExtractUserKey(unused_key).ToString(), kNumHashFunc, kNumHashFunc); - ASSERT_OK(reader.Get( - ReadOptions(), Slice(unused_key), &v, AssertValues, nullptr)); - ASSERT_EQ(0, v.call_count); + value.Reset(); + GetContext get_context3(ucmp, nullptr, nullptr, nullptr, + GetContext::kNotFound, Slice(unused_key), &value, + nullptr, nullptr, nullptr, nullptr); + ASSERT_OK(reader.Get(ReadOptions(), Slice(unused_key), &get_context3)); + ASSERT_TRUE(value.empty()); ASSERT_OK(reader.status()); } // Performance tests namespace { -bool DoNothing(void* arg, const ParsedInternalKey& k, const Slice& v) { - // Deliberately empty. - return false; -} - -bool CheckValue(void* cnt_ptr, const ParsedInternalKey& k, const Slice& v) { - ++*reinterpret_cast(cnt_ptr); - std::string expected_value; - AppendInternalKey(&expected_value, k); - ASSERT_EQ(0, v.compare(Slice(&expected_value[0], v.size()))); - return false; -} - void GetKeys(uint64_t num, std::vector* keys) { + keys->clear(); IterKey k; k.SetInternalKey("", 0, kTypeValue); - std::string internal_key_suffix = k.GetKey().ToString(); + std::string internal_key_suffix = k.GetInternalKey().ToString(); ASSERT_EQ(static_cast(8), internal_key_suffix.size()); for (uint64_t key_idx = 0; key_idx < num; ++key_idx) { - std::string new_key(reinterpret_cast(&key_idx), sizeof(key_idx)); + uint64_t value = 2 * key_idx; + std::string new_key(reinterpret_cast(&value), sizeof(value)); new_key += internal_key_suffix; keys->push_back(new_key); } @@ -399,7 +391,7 @@ std::string GetFileName(uint64_t num) { FLAGS_file_dir = test::TmpDir(); } return FLAGS_file_dir + "/cuckoo_read_benchmark" + - std::to_string(num/1000000) + "Mkeys"; + ToString(num/1000000) + "Mkeys"; } // Create last level file as we are interested in measuring performance of @@ -414,9 +406,12 @@ void WriteFile(const std::vector& keys, std::unique_ptr writable_file; ASSERT_OK(env->NewWritableFile(fname, &writable_file, env_options)); + unique_ptr file_writer( + new WritableFileWriter(std::move(writable_file), env_options)); CuckooTableBuilder builder( - writable_file.get(), hash_ratio, - 64, 1000, test::Uint64Comparator(), 5, nullptr); + file_writer.get(), hash_ratio, 64, 1000, test::Uint64Comparator(), 5, + false, FLAGS_identity_as_first_hash, nullptr, 0 /* column_family_id */, + kDefaultColumnFamilyName); ASSERT_OK(builder.status()); for (uint64_t key_idx = 0; key_idx < num; ++key_idx) { // Value is just a part of key. @@ -426,25 +421,30 @@ void WriteFile(const std::vector& keys, } ASSERT_OK(builder.Finish()); ASSERT_EQ(num, builder.NumEntries()); - ASSERT_OK(writable_file->Close()); + ASSERT_OK(file_writer->Close()); uint64_t file_size; env->GetFileSize(fname, &file_size); std::unique_ptr read_file; ASSERT_OK(env->NewRandomAccessFile(fname, &read_file, env_options)); + unique_ptr file_reader( + new RandomAccessFileReader(std::move(read_file), fname)); - CuckooTableReader reader( - options, std::move(read_file), file_size, - test::Uint64Comparator(), nullptr); + const ImmutableCFOptions ioptions(options); + CuckooTableReader reader(ioptions, std::move(file_reader), file_size, + test::Uint64Comparator(), nullptr); ASSERT_OK(reader.status()); ReadOptions r_options; + PinnableSlice value; + // Assume only the fast path is triggered + GetContext get_context(nullptr, nullptr, nullptr, nullptr, + GetContext::kNotFound, Slice(), &value, nullptr, + nullptr, nullptr, nullptr); for (uint64_t i = 0; i < num; ++i) { - int cnt = 0; - ASSERT_OK(reader.Get(r_options, Slice(keys[i]), &cnt, CheckValue, nullptr)); - if (cnt != 1) { - fprintf(stderr, "%" PRIu64 " not found.\n", i); - ASSERT_EQ(1, cnt); - } + value.Reset(); + value.clear(); + ASSERT_OK(reader.Get(r_options, Slice(keys[i]), &get_context)); + ASSERT_TRUE(Slice(keys[i]) == Slice(&keys[i][0], 4)); } } @@ -459,10 +459,12 @@ void ReadKeys(uint64_t num, uint32_t batch_size) { env->GetFileSize(fname, &file_size); std::unique_ptr read_file; ASSERT_OK(env->NewRandomAccessFile(fname, &read_file, env_options)); + unique_ptr file_reader( + new RandomAccessFileReader(std::move(read_file), fname)); - CuckooTableReader reader( - options, std::move(read_file), file_size, test::Uint64Comparator(), - nullptr); + const ImmutableCFOptions ioptions(options); + CuckooTableReader reader(ioptions, std::move(file_reader), file_size, + test::Uint64Comparator(), nullptr); ASSERT_OK(reader.status()); const UserCollectedProperties user_props = reader.GetTableProperties()->user_collected_properties; @@ -474,49 +476,62 @@ void ReadKeys(uint64_t num, uint32_t batch_size) { " hash functions: %u.\n", num, num * 100.0 / (table_size), num_hash_fun); ReadOptions r_options; + std::vector keys; + keys.reserve(num); + for (uint64_t i = 0; i < num; ++i) { + keys.push_back(2 * i); + } + std::random_shuffle(keys.begin(), keys.end()); + + PinnableSlice value; + // Assume only the fast path is triggered + GetContext get_context(nullptr, nullptr, nullptr, nullptr, + GetContext::kNotFound, Slice(), &value, nullptr, + nullptr, nullptr, nullptr); uint64_t start_time = env->NowMicros(); if (batch_size > 0) { for (uint64_t i = 0; i < num; i += batch_size) { for (uint64_t j = i; j < i+batch_size && j < num; ++j) { - reader.Prepare(Slice(reinterpret_cast(&j), 16)); + reader.Prepare(Slice(reinterpret_cast(&keys[j]), 16)); } for (uint64_t j = i; j < i+batch_size && j < num; ++j) { - reader.Get(r_options, Slice(reinterpret_cast(&j), 16), - nullptr, DoNothing, nullptr); + reader.Get(r_options, Slice(reinterpret_cast(&keys[j]), 16), + &get_context); } } } else { for (uint64_t i = 0; i < num; i++) { - reader.Get(r_options, Slice(reinterpret_cast(&i), 16), nullptr, - DoNothing, nullptr); + reader.Get(r_options, Slice(reinterpret_cast(&keys[i]), 16), + &get_context); } } - float time_per_op = (env->NowMicros() - start_time) * 1.0 / num; + float time_per_op = (env->NowMicros() - start_time) * 1.0f / num; fprintf(stderr, "Time taken per op is %.3fus (%.1f Mqps) with batch size of %u\n", time_per_op, 1.0 / time_per_op, batch_size); } } // namespace. -TEST(CuckooReaderTest, TestReadPerformance) { +TEST_F(CuckooReaderTest, TestReadPerformance) { if (!FLAGS_enable_perf) { return; } double hash_ratio = 0.95; - // These numbers are chosen to have a hash utilizaiton % close to + // These numbers are chosen to have a hash utilization % close to // 0.9, 0.75, 0.6 and 0.5 respectively. // They all create 128 M buckets. - std::vector nums = {120*1000*1000, 100*1000*1000, 80*1000*1000, - 70*1000*1000}; + std::vector nums = {120*1024*1024, 100*1024*1024, 80*1024*1024, + 70*1024*1024}; #ifndef NDEBUG fprintf(stdout, "WARNING: Not compiled with DNDEBUG. Performance tests may be slow.\n"); #endif - std::vector keys; - GetKeys(*std::max_element(nums.begin(), nums.end()), &keys); for (uint64_t num : nums) { - if (FLAGS_write || !Env::Default()->FileExists(GetFileName(num))) { - WriteFile(keys, num, hash_ratio); + if (FLAGS_write || + Env::Default()->FileExists(GetFileName(num)).IsNotFound()) { + std::vector all_keys; + GetKeys(num, &all_keys); + WriteFile(all_keys, num, hash_ratio); } ReadKeys(num, 0); ReadKeys(num, 10); @@ -529,9 +544,25 @@ TEST(CuckooReaderTest, TestReadPerformance) { } // namespace rocksdb int main(int argc, char** argv) { - ParseCommandLineFlags(&argc, &argv, true); - rocksdb::test::RunAllTests(); - return 0; + if (rocksdb::port::kLittleEndian) { + ::testing::InitGoogleTest(&argc, argv); + ParseCommandLineFlags(&argc, &argv, true); + return RUN_ALL_TESTS(); + } + else { + fprintf(stderr, "SKIPPED as Cuckoo table doesn't support Big Endian\n"); + return 0; + } } #endif // GFLAGS. + +#else +#include + +int main(int argc, char** argv) { + fprintf(stderr, "SKIPPED as Cuckoo table is not supported in ROCKSDB_LITE\n"); + return 0; +} + +#endif // ROCKSDB_LITE diff --git a/table/filter_block.cc b/table/filter_block.cc deleted file mode 100644 index 6b4ff1c10da..00000000000 --- a/table/filter_block.cc +++ /dev/null @@ -1,186 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Copyright (c) 2012 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#include "table/filter_block.h" - -#include "db/dbformat.h" -#include "rocksdb/filter_policy.h" -#include "util/coding.h" - -namespace rocksdb { - -// See doc/table_format.txt for an explanation of the filter block format. - -// Generate new filter every 2KB of data -static const size_t kFilterBaseLg = 11; -static const size_t kFilterBase = 1 << kFilterBaseLg; - -FilterBlockBuilder::FilterBlockBuilder(const Options& opt, - const BlockBasedTableOptions& table_opt, - const Comparator* internal_comparator) - : policy_(table_opt.filter_policy.get()), - prefix_extractor_(opt.prefix_extractor.get()), - whole_key_filtering_(table_opt.whole_key_filtering), - comparator_(internal_comparator) {} - -void FilterBlockBuilder::StartBlock(uint64_t block_offset) { - uint64_t filter_index = (block_offset / kFilterBase); - assert(filter_index >= filter_offsets_.size()); - while (filter_index > filter_offsets_.size()) { - GenerateFilter(); - } -} - -bool FilterBlockBuilder::SamePrefix(const Slice &key1, - const Slice &key2) const { - if (!prefix_extractor_->InDomain(key1) && - !prefix_extractor_->InDomain(key2)) { - return true; - } else if (!prefix_extractor_->InDomain(key1) || - !prefix_extractor_->InDomain(key2)) { - return false; - } else { - return (prefix_extractor_->Transform(key1) == - prefix_extractor_->Transform(key2)); - } -} - -void FilterBlockBuilder::AddKey(const Slice& key) { - // get slice for most recently added entry - Slice prev; - size_t added_to_start = 0; - - // add key to filter if needed - if (whole_key_filtering_) { - start_.push_back(entries_.size()); - ++added_to_start; - entries_.append(key.data(), key.size()); - } - - if (start_.size() > added_to_start) { - size_t prev_start = start_[start_.size() - 1 - added_to_start]; - const char* base = entries_.data() + prev_start; - size_t length = entries_.size() - prev_start; - prev = Slice(base, length); - } - - // add prefix to filter if needed - if (prefix_extractor_ && prefix_extractor_->InDomain(key)) { - // this assumes prefix(prefix(key)) == prefix(key), as the last - // entry in entries_ may be either a key or prefix, and we use - // prefix(last entry) to get the prefix of the last key. - if (prev.size() == 0 || !SamePrefix(key, prev)) { - Slice prefix = prefix_extractor_->Transform(key); - start_.push_back(entries_.size()); - entries_.append(prefix.data(), prefix.size()); - } - } -} - -Slice FilterBlockBuilder::Finish() { - if (!start_.empty()) { - GenerateFilter(); - } - - // Append array of per-filter offsets - const uint32_t array_offset = result_.size(); - for (size_t i = 0; i < filter_offsets_.size(); i++) { - PutFixed32(&result_, filter_offsets_[i]); - } - - PutFixed32(&result_, array_offset); - result_.push_back(kFilterBaseLg); // Save encoding parameter in result - return Slice(result_); -} - -void FilterBlockBuilder::GenerateFilter() { - const size_t num_entries = start_.size(); - if (num_entries == 0) { - // Fast path if there are no keys for this filter - filter_offsets_.push_back(result_.size()); - return; - } - - // Make list of keys from flattened key structure - start_.push_back(entries_.size()); // Simplify length computation - tmp_entries_.resize(num_entries); - for (size_t i = 0; i < num_entries; i++) { - const char* base = entries_.data() + start_[i]; - size_t length = start_[i+1] - start_[i]; - tmp_entries_[i] = Slice(base, length); - } - - // Generate filter for current set of keys and append to result_. - filter_offsets_.push_back(result_.size()); - policy_->CreateFilter(&tmp_entries_[0], num_entries, &result_); - - tmp_entries_.clear(); - entries_.clear(); - start_.clear(); -} - -FilterBlockReader::FilterBlockReader( - const Options& opt, const BlockBasedTableOptions& table_opt, - const Slice& contents, bool delete_contents_after_use) - : policy_(table_opt.filter_policy.get()), - prefix_extractor_(opt.prefix_extractor.get()), - whole_key_filtering_(table_opt.whole_key_filtering), - data_(nullptr), - offset_(nullptr), - num_(0), - base_lg_(0) { - size_t n = contents.size(); - if (n < 5) return; // 1 byte for base_lg_ and 4 for start of offset array - base_lg_ = contents[n-1]; - uint32_t last_word = DecodeFixed32(contents.data() + n - 5); - if (last_word > n - 5) return; - data_ = contents.data(); - offset_ = data_ + last_word; - num_ = (n - 5 - last_word) / 4; - if (delete_contents_after_use) { - filter_data.reset(contents.data()); - } -} - -bool FilterBlockReader::KeyMayMatch(uint64_t block_offset, - const Slice& key) { - if (!whole_key_filtering_) { - return true; - } - return MayMatch(block_offset, key); -} - -bool FilterBlockReader::PrefixMayMatch(uint64_t block_offset, - const Slice& prefix) { - if (!prefix_extractor_) { - return true; - } - return MayMatch(block_offset, prefix); -} - -bool FilterBlockReader::MayMatch(uint64_t block_offset, const Slice& entry) { - uint64_t index = block_offset >> base_lg_; - if (index < num_) { - uint32_t start = DecodeFixed32(offset_ + index*4); - uint32_t limit = DecodeFixed32(offset_ + index*4 + 4); - if (start <= limit && limit <= (uint32_t)(offset_ - data_)) { - Slice filter = Slice(data_ + start, limit - start); - return policy_->KeyMayMatch(entry, filter); - } else if (start == limit) { - // Empty filters do not match any entries - return false; - } - } - return true; // Errors are treated as potential matches -} - -size_t FilterBlockReader::ApproximateMemoryUsage() const { - return num_ * 4 + 5 + (offset_ - data_); -} -} diff --git a/table/filter_block.h b/table/filter_block.h index 5041393f6b8..7bf3b31324d 100644 --- a/table/filter_block.h +++ b/table/filter_block.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2012 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -10,6 +10,11 @@ // A filter block is stored near the end of a Table file. It contains // filters (e.g., bloom filters) for all data blocks in the table combined // into a single filter block. +// +// It is a base class for BlockBasedFilter and FullFilter. +// These two are both used in BlockBasedTable. The first one contain filter +// For a part of keys in sst file, the second contain filter for all keys +// in sst file. #pragma once @@ -23,9 +28,11 @@ #include "rocksdb/slice_transform.h" #include "rocksdb/table.h" #include "util/hash.h" +#include "format.h" namespace rocksdb { +const uint64_t kNotValid = ULLONG_MAX; class FilterPolicy; // A FilterBlockBuilder is used to construct all of the filters for a @@ -33,64 +40,92 @@ class FilterPolicy; // a special block in the Table. // // The sequence of calls to FilterBlockBuilder must match the regexp: -// (StartBlock AddKey*)* Finish +// (StartBlock Add*)* Finish +// +// BlockBased/Full FilterBlock would be called in the same way. class FilterBlockBuilder { public: - explicit FilterBlockBuilder(const Options& opt, - const BlockBasedTableOptions& table_opt, - const Comparator* internal_comparator); + explicit FilterBlockBuilder() {} + virtual ~FilterBlockBuilder() {} - void StartBlock(uint64_t block_offset); - void AddKey(const Slice& key); - Slice Finish(); + virtual bool IsBlockBased() = 0; // If is blockbased filter + virtual void StartBlock(uint64_t block_offset) = 0; // Start new block filter + virtual void Add(const Slice& key) = 0; // Add a key to current filter + Slice Finish() { // Generate Filter + const BlockHandle empty_handle; + Status dont_care_status; + auto ret = Finish(empty_handle, &dont_care_status); + assert(dont_care_status.ok()); + return ret; + } + virtual Slice Finish(const BlockHandle& tmp, Status* status) = 0; private: - bool SamePrefix(const Slice &key1, const Slice &key2) const; - void GenerateFilter(); - - // important: all of these might point to invalid addresses - // at the time of destruction of this filter block. destructor - // should NOT dereference them. - const FilterPolicy* policy_; - const SliceTransform* prefix_extractor_; - bool whole_key_filtering_; - const Comparator* comparator_; - - std::string entries_; // Flattened entry contents - std::vector start_; // Starting index in entries_ of each entry - std::string result_; // Filter data computed so far - std::vector tmp_entries_; // policy_->CreateFilter() argument - std::vector filter_offsets_; - // No copying allowed FilterBlockBuilder(const FilterBlockBuilder&); void operator=(const FilterBlockBuilder&); }; +// A FilterBlockReader is used to parse filter from SST table. +// KeyMayMatch and PrefixMayMatch would trigger filter checking +// +// BlockBased/Full FilterBlock would be called in the same way. class FilterBlockReader { public: - // REQUIRES: "contents" and *policy must stay live while *this is live. - FilterBlockReader( - const Options& opt, - const BlockBasedTableOptions& table_opt, - const Slice& contents, - bool delete_contents_after_use = false); - bool KeyMayMatch(uint64_t block_offset, const Slice& key); - bool PrefixMayMatch(uint64_t block_offset, const Slice& prefix); - size_t ApproximateMemoryUsage() const; + explicit FilterBlockReader() + : whole_key_filtering_(true), size_(0), statistics_(nullptr) {} + explicit FilterBlockReader(size_t s, Statistics* stats, + bool _whole_key_filtering) + : whole_key_filtering_(_whole_key_filtering), + size_(s), + statistics_(stats) {} + virtual ~FilterBlockReader() {} - private: - const FilterPolicy* policy_; - const SliceTransform* prefix_extractor_; - bool whole_key_filtering_; - const char* data_; // Pointer to filter data (at block-start) - const char* offset_; // Pointer to beginning of offset array (at block-end) - size_t num_; // Number of entries in offset array - size_t base_lg_; // Encoding parameter (see kFilterBaseLg in .cc file) - std::unique_ptr filter_data; + virtual bool IsBlockBased() = 0; // If is blockbased filter + /** + * If no_io is set, then it returns true if it cannot answer the query without + * reading data from disk. This is used in PartitionedFilterBlockReader to + * avoid reading partitions that are not in block cache already + * + * Normally filters are built on only the user keys and the InternalKey is not + * needed for a query. The index in PartitionedFilterBlockReader however is + * built upon InternalKey and must be provided via const_ikey_ptr when running + * queries. + */ + virtual bool KeyMayMatch(const Slice& key, uint64_t block_offset = kNotValid, + const bool no_io = false, + const Slice* const const_ikey_ptr = nullptr) = 0; + /** + * no_io and const_ikey_ptr here means the same as in KeyMayMatch + */ + virtual bool PrefixMayMatch(const Slice& prefix, + uint64_t block_offset = kNotValid, + const bool no_io = false, + const Slice* const const_ikey_ptr = nullptr) = 0; + virtual size_t ApproximateMemoryUsage() const = 0; + virtual size_t size() const { return size_; } + virtual Statistics* statistics() const { return statistics_; } + + bool whole_key_filtering() const { return whole_key_filtering_; } + // convert this object to a human readable form + virtual std::string ToString() const { + std::string error_msg("Unsupported filter \n"); + return error_msg; + } - bool MayMatch(uint64_t block_offset, const Slice& entry); + virtual void CacheDependencies(bool pin) {} + + protected: + bool whole_key_filtering_; + + private: + // No copying allowed + FilterBlockReader(const FilterBlockReader&); + void operator=(const FilterBlockReader&); + size_t size_; + Statistics* statistics_; + int level_ = -1; }; -} +} // namespace rocksdb diff --git a/table/filter_block_test.cc b/table/filter_block_test.cc deleted file mode 100644 index 95496a82c20..00000000000 --- a/table/filter_block_test.cc +++ /dev/null @@ -1,139 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Copyright (c) 2012 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#include "table/filter_block.h" - -#include "rocksdb/filter_policy.h" -#include "util/coding.h" -#include "util/hash.h" -#include "util/logging.h" -#include "util/testharness.h" -#include "util/testutil.h" - -namespace rocksdb { - -// For testing: emit an array with one hash value per key -class TestHashFilter : public FilterPolicy { - public: - virtual const char* Name() const { - return "TestHashFilter"; - } - - virtual void CreateFilter(const Slice* keys, int n, std::string* dst) const { - for (int i = 0; i < n; i++) { - uint32_t h = Hash(keys[i].data(), keys[i].size(), 1); - PutFixed32(dst, h); - } - } - - virtual bool KeyMayMatch(const Slice& key, const Slice& filter) const { - uint32_t h = Hash(key.data(), key.size(), 1); - for (unsigned int i = 0; i + 4 <= filter.size(); i += 4) { - if (h == DecodeFixed32(filter.data() + i)) { - return true; - } - } - return false; - } -}; - -class FilterBlockTest { - public: - Options options_; - BlockBasedTableOptions table_options_; - - FilterBlockTest() { - options_ = Options(); - table_options_.filter_policy.reset(new TestHashFilter()); - } -}; - -TEST(FilterBlockTest, EmptyBuilder) { - FilterBlockBuilder builder(options_, table_options_, options_.comparator); - Slice block = builder.Finish(); - ASSERT_EQ("\\x00\\x00\\x00\\x00\\x0b", EscapeString(block)); - FilterBlockReader reader(options_, table_options_, block); - ASSERT_TRUE(reader.KeyMayMatch(0, "foo")); - ASSERT_TRUE(reader.KeyMayMatch(100000, "foo")); -} - -TEST(FilterBlockTest, SingleChunk) { - FilterBlockBuilder builder(options_, table_options_, options_.comparator); - builder.StartBlock(100); - builder.AddKey("foo"); - builder.AddKey("bar"); - builder.AddKey("box"); - builder.StartBlock(200); - builder.AddKey("box"); - builder.StartBlock(300); - builder.AddKey("hello"); - Slice block = builder.Finish(); - FilterBlockReader reader(options_, table_options_, block); - ASSERT_TRUE(reader.KeyMayMatch(100, "foo")); - ASSERT_TRUE(reader.KeyMayMatch(100, "bar")); - ASSERT_TRUE(reader.KeyMayMatch(100, "box")); - ASSERT_TRUE(reader.KeyMayMatch(100, "hello")); - ASSERT_TRUE(reader.KeyMayMatch(100, "foo")); - ASSERT_TRUE(! reader.KeyMayMatch(100, "missing")); - ASSERT_TRUE(! reader.KeyMayMatch(100, "other")); -} - -TEST(FilterBlockTest, MultiChunk) { - FilterBlockBuilder builder(options_, table_options_, options_.comparator); - - // First filter - builder.StartBlock(0); - builder.AddKey("foo"); - builder.StartBlock(2000); - builder.AddKey("bar"); - - // Second filter - builder.StartBlock(3100); - builder.AddKey("box"); - - // Third filter is empty - - // Last filter - builder.StartBlock(9000); - builder.AddKey("box"); - builder.AddKey("hello"); - - Slice block = builder.Finish(); - FilterBlockReader reader(options_, table_options_, block); - - // Check first filter - ASSERT_TRUE(reader.KeyMayMatch(0, "foo")); - ASSERT_TRUE(reader.KeyMayMatch(2000, "bar")); - ASSERT_TRUE(! reader.KeyMayMatch(0, "box")); - ASSERT_TRUE(! reader.KeyMayMatch(0, "hello")); - - // Check second filter - ASSERT_TRUE(reader.KeyMayMatch(3100, "box")); - ASSERT_TRUE(! reader.KeyMayMatch(3100, "foo")); - ASSERT_TRUE(! reader.KeyMayMatch(3100, "bar")); - ASSERT_TRUE(! reader.KeyMayMatch(3100, "hello")); - - // Check third filter (empty) - ASSERT_TRUE(! reader.KeyMayMatch(4100, "foo")); - ASSERT_TRUE(! reader.KeyMayMatch(4100, "bar")); - ASSERT_TRUE(! reader.KeyMayMatch(4100, "box")); - ASSERT_TRUE(! reader.KeyMayMatch(4100, "hello")); - - // Check last filter - ASSERT_TRUE(reader.KeyMayMatch(9000, "box")); - ASSERT_TRUE(reader.KeyMayMatch(9000, "hello")); - ASSERT_TRUE(! reader.KeyMayMatch(9000, "foo")); - ASSERT_TRUE(! reader.KeyMayMatch(9000, "bar")); -} - -} // namespace rocksdb - -int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); -} diff --git a/table/flush_block_policy.cc b/table/flush_block_policy.cc index 4c12b30bb2b..9a8dea4cb0c 100644 --- a/table/flush_block_policy.cc +++ b/table/flush_block_policy.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #include "rocksdb/options.h" #include "rocksdb/flush_block_policy.h" @@ -21,11 +21,11 @@ class FlushBlockBySizePolicy : public FlushBlockPolicy { // reaches the configured FlushBlockBySizePolicy(const uint64_t block_size, const uint64_t block_size_deviation, - const BlockBuilder& data_block_builder) : - block_size_(block_size), - block_size_deviation_(block_size_deviation), - data_block_builder_(data_block_builder) { - } + const BlockBuilder& data_block_builder) + : block_size_(block_size), + block_size_deviation_limit_( + ((block_size * (100 - block_size_deviation)) + 99) / 100), + data_block_builder_(data_block_builder) {} virtual bool Update(const Slice& key, const Slice& value) override { @@ -46,18 +46,20 @@ class FlushBlockBySizePolicy : public FlushBlockPolicy { private: bool BlockAlmostFull(const Slice& key, const Slice& value) const { + if (block_size_deviation_limit_ == 0) { + return false; + } + const auto curr_size = data_block_builder_.CurrentSizeEstimate(); const auto estimated_size_after = data_block_builder_.EstimateSizeAfterKV(key, value); - return - estimated_size_after > block_size_ && - block_size_deviation_ > 0 && - curr_size * 100 > block_size_ * (100 - block_size_deviation_); + return estimated_size_after > block_size_ && + curr_size > block_size_deviation_limit_; } const uint64_t block_size_; - const uint64_t block_size_deviation_; + const uint64_t block_size_deviation_limit_; const BlockBuilder& data_block_builder_; }; @@ -69,4 +71,10 @@ FlushBlockPolicy* FlushBlockBySizePolicyFactory::NewFlushBlockPolicy( data_block_builder); } +FlushBlockPolicy* FlushBlockBySizePolicyFactory::NewFlushBlockPolicy( + const uint64_t size, const int deviation, + const BlockBuilder& data_block_builder) { + return new FlushBlockBySizePolicy(size, deviation, data_block_builder); +} + } // namespace rocksdb diff --git a/table/format.cc b/table/format.cc index 46105247f37..364766e9a80 100644 --- a/table/format.cc +++ b/table/format.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -12,12 +12,19 @@ #include #include -#include "port/port.h" +#include "monitoring/perf_context_imp.h" +#include "monitoring/statistics.h" #include "rocksdb/env.h" #include "table/block.h" +#include "table/block_based_table_reader.h" +#include "table/persistent_cache_helper.h" #include "util/coding.h" +#include "util/compression.h" #include "util/crc32c.h" -#include "util/perf_context_imp.h" +#include "util/file_reader_writer.h" +#include "util/logging.h" +#include "util/stop_watch.h" +#include "util/string_util.h" #include "util/xxhash.h" namespace rocksdb { @@ -35,12 +42,16 @@ const uint64_t kPlainTableMagicNumber = 0; #endif const uint32_t DefaultStackBufferSize = 5000; +bool ShouldReportDetailedTime(Env* env, Statistics* stats) { + return env != nullptr && stats != nullptr && + stats->stats_level_ > kExceptDetailedTimers; +} + void BlockHandle::EncodeTo(std::string* dst) const { // Sanity check that all fields have been set assert(offset_ != ~static_cast(0)); assert(size_ != ~static_cast(0)); - PutVarint64(dst, offset_); - PutVarint64(dst, size_); + PutVarint64Varint64(dst, offset_, size_); } Status BlockHandle::DecodeFrom(Slice* input) { @@ -48,25 +59,58 @@ Status BlockHandle::DecodeFrom(Slice* input) { GetVarint64(input, &size_)) { return Status::OK(); } else { + // reset in case failure after partially decoding + offset_ = 0; + size_ = 0; return Status::Corruption("bad block handle"); } } + +// Return a string that contains the copy of handle. +std::string BlockHandle::ToString(bool hex) const { + std::string handle_str; + EncodeTo(&handle_str); + if (hex) { + return Slice(handle_str).ToString(true); + } else { + return handle_str; + } +} + const BlockHandle BlockHandle::kNullBlockHandle(0, 0); +namespace { +inline bool IsLegacyFooterFormat(uint64_t magic_number) { + return magic_number == kLegacyBlockBasedTableMagicNumber || + magic_number == kLegacyPlainTableMagicNumber; +} +inline uint64_t UpconvertLegacyFooterFormat(uint64_t magic_number) { + if (magic_number == kLegacyBlockBasedTableMagicNumber) { + return kBlockBasedTableMagicNumber; + } + if (magic_number == kLegacyPlainTableMagicNumber) { + return kPlainTableMagicNumber; + } + assert(false); + return 0; +} +} // namespace + // legacy footer format: // metaindex handle (varint64 offset, varint64 size) // index handle (varint64 offset, varint64 size) // to make the total size 2 * BlockHandle::kMaxEncodedLength // table_magic_number (8 bytes) // new footer format: -// checksum (char, 1 byte) +// checksum type (char, 1 byte) // metaindex handle (varint64 offset, varint64 size) // index handle (varint64 offset, varint64 size) // to make the total size 2 * BlockHandle::kMaxEncodedLength + 1 // footer version (4 bytes) // table_magic_number (8 bytes) void Footer::EncodeTo(std::string* dst) const { - if (version() == kLegacyFooter) { + assert(HasInitializedTableMagicNumber()); + if (IsLegacyFooterFormat(table_magic_number())) { // has to be default checksum with legacy footer assert(checksum_ == kCRC32c); const size_t original_size = dst->size(); @@ -81,39 +125,24 @@ void Footer::EncodeTo(std::string* dst) const { dst->push_back(static_cast(checksum_)); metaindex_handle_.EncodeTo(dst); index_handle_.EncodeTo(dst); - dst->resize(original_size + kVersion1EncodedLength - 12); // Padding - PutFixed32(dst, kFooterVersion); + dst->resize(original_size + kNewVersionsEncodedLength - 12); // Padding + PutFixed32(dst, version()); PutFixed32(dst, static_cast(table_magic_number() & 0xffffffffu)); PutFixed32(dst, static_cast(table_magic_number() >> 32)); - assert(dst->size() == original_size + kVersion1EncodedLength); + assert(dst->size() == original_size + kNewVersionsEncodedLength); } } -namespace { -inline bool IsLegacyFooterFormat(uint64_t magic_number) { - return magic_number == kLegacyBlockBasedTableMagicNumber || - magic_number == kLegacyPlainTableMagicNumber; -} - -inline uint64_t UpconvertLegacyFooterFormat(uint64_t magic_number) { - if (magic_number == kLegacyBlockBasedTableMagicNumber) { - return kBlockBasedTableMagicNumber; - } - if (magic_number == kLegacyPlainTableMagicNumber) { - return kPlainTableMagicNumber; - } - assert(false); - return 0; -} -} // namespace - -Footer::Footer(uint64_t table_magic_number) - : version_(IsLegacyFooterFormat(table_magic_number) ? kLegacyFooter - : kFooterVersion), +Footer::Footer(uint64_t _table_magic_number, uint32_t _version) + : version_(_version), checksum_(kCRC32c), - table_magic_number_(table_magic_number) {} + table_magic_number_(_table_magic_number) { + // This should be guaranteed by constructor callers + assert(!IsLegacyFooterFormat(_table_magic_number) || version_ == 0); +} Status Footer::DecodeFrom(Slice* input) { + assert(!HasInitializedTableMagicNumber()); assert(input != nullptr); assert(input->size() >= kMinEncodedLength); @@ -129,42 +158,29 @@ Status Footer::DecodeFrom(Slice* input) { if (legacy) { magic = UpconvertLegacyFooterFormat(magic); } - if (HasInitializedTableMagicNumber()) { - if (magic != table_magic_number()) { - char buffer[80]; - snprintf(buffer, sizeof(buffer) - 1, - "not an sstable (bad magic number --- %lx)", - (long)magic); - return Status::InvalidArgument(buffer); - } - } else { - set_table_magic_number(magic); - } + set_table_magic_number(magic); if (legacy) { // The size is already asserted to be at least kMinEncodedLength // at the beginning of the function input->remove_prefix(input->size() - kVersion0EncodedLength); - version_ = kLegacyFooter; + version_ = 0 /* legacy */; checksum_ = kCRC32c; } else { version_ = DecodeFixed32(magic_ptr - 4); - if (version_ != kFooterVersion) { - return Status::Corruption("bad footer version"); - } - // Footer version 1 will always occupy exactly this many bytes. + // Footer version 1 and higher will always occupy exactly this many bytes. // It consists of the checksum type, two block handles, padding, // a version number, and a magic number - if (input->size() < kVersion1EncodedLength) { - return Status::InvalidArgument("input is too short to be an sstable"); + if (input->size() < kNewVersionsEncodedLength) { + return Status::Corruption("input is too short to be an sstable"); } else { - input->remove_prefix(input->size() - kVersion1EncodedLength); + input->remove_prefix(input->size() - kNewVersionsEncodedLength); } - uint32_t checksum; - if (!GetVarint32(input, &checksum)) { + uint32_t chksum; + if (!GetVarint32(input, &chksum)) { return Status::Corruption("bad checksum type"); } - checksum_ = static_cast(checksum); + checksum_ = static_cast(chksum); } Status result = metaindex_handle_.DecodeFrom(input); @@ -179,74 +195,109 @@ Status Footer::DecodeFrom(Slice* input) { return result; } -Status ReadFooterFromFile(RandomAccessFile* file, - uint64_t file_size, - Footer* footer) { +std::string Footer::ToString() const { + std::string result, handle_; + result.reserve(1024); + + bool legacy = IsLegacyFooterFormat(table_magic_number_); + if (legacy) { + result.append("metaindex handle: " + metaindex_handle_.ToString() + "\n "); + result.append("index handle: " + index_handle_.ToString() + "\n "); + result.append("table_magic_number: " + + rocksdb::ToString(table_magic_number_) + "\n "); + } else { + result.append("checksum: " + rocksdb::ToString(checksum_) + "\n "); + result.append("metaindex handle: " + metaindex_handle_.ToString() + "\n "); + result.append("index handle: " + index_handle_.ToString() + "\n "); + result.append("footer version: " + rocksdb::ToString(version_) + "\n "); + result.append("table_magic_number: " + + rocksdb::ToString(table_magic_number_) + "\n "); + } + return result; +} + +Status ReadFooterFromFile(RandomAccessFileReader* file, + FilePrefetchBuffer* prefetch_buffer, + uint64_t file_size, Footer* footer, + uint64_t enforce_table_magic_number) { if (file_size < Footer::kMinEncodedLength) { - return Status::InvalidArgument("file is too short to be an sstable"); + return Status::Corruption( + "file is too short (" + ToString(file_size) + " bytes) to be an " + "sstable: " + file->file_name()); } char footer_space[Footer::kMaxEncodedLength]; Slice footer_input; - size_t read_offset = (file_size > Footer::kMaxEncodedLength) - ? (file_size - Footer::kMaxEncodedLength) - : 0; - Status s = file->Read(read_offset, Footer::kMaxEncodedLength, &footer_input, - footer_space); - if (!s.ok()) return s; + size_t read_offset = + (file_size > Footer::kMaxEncodedLength) + ? static_cast(file_size - Footer::kMaxEncodedLength) + : 0; + Status s; + if (prefetch_buffer == nullptr || + !prefetch_buffer->TryReadFromCache(read_offset, Footer::kMaxEncodedLength, + &footer_input)) { + s = file->Read(read_offset, Footer::kMaxEncodedLength, &footer_input, + footer_space); + if (!s.ok()) return s; + } // Check that we actually read the whole footer from the file. It may be // that size isn't correct. if (footer_input.size() < Footer::kMinEncodedLength) { - return Status::InvalidArgument("file is too short to be an sstable"); - } - - return footer->DecodeFrom(&footer_input); -} - -// Read a block and check its CRC -// contents is the result of reading. -// According to the implementation of file->Read, contents may not point to buf -Status ReadBlock(RandomAccessFile* file, const Footer& footer, - const ReadOptions& options, const BlockHandle& handle, - Slice* contents, /* result of reading */ char* buf) { - size_t n = static_cast(handle.size()); - Status s; - - { - PERF_TIMER_GUARD(block_read_time); - s = file->Read(handle.offset(), n + kBlockTrailerSize, contents, buf); + return Status::Corruption( + "file is too short (" + ToString(file_size) + " bytes) to be an " + "sstable" + file->file_name()); } - PERF_COUNTER_ADD(block_read_count, 1); - PERF_COUNTER_ADD(block_read_byte, n + kBlockTrailerSize); - + s = footer->DecodeFrom(&footer_input); if (!s.ok()) { return s; } - if (contents->size() != n + kBlockTrailerSize) { - return Status::Corruption("truncated block read"); + if (enforce_table_magic_number != 0 && + enforce_table_magic_number != footer->table_magic_number()) { + return Status::Corruption( + "Bad table magic number: expected " + + ToString(enforce_table_magic_number) + ", found " + + ToString(footer->table_magic_number()) + + " in " + file->file_name()); } + return Status::OK(); +} +// Without anonymous namespace here, we fail the warning -Wmissing-prototypes +namespace { +Status CheckBlockChecksum(const ReadOptions& options, const Footer& footer, + const Slice& contents, size_t block_size, + RandomAccessFileReader* file, + const BlockHandle& handle) { + Status s; // Check the crc of the type and the block contents - const char* data = contents->data(); // Pointer to where Read put the data if (options.verify_checksums) { + const char* data = contents.data(); // Pointer to where Read put the data PERF_TIMER_GUARD(block_checksum_time); - uint32_t value = DecodeFixed32(data + n + 1); + uint32_t value = DecodeFixed32(data + block_size + 1); uint32_t actual = 0; switch (footer.checksum()) { + case kNoChecksum: + break; case kCRC32c: value = crc32c::Unmask(value); - actual = crc32c::Value(data, n + 1); + actual = crc32c::Value(data, block_size + 1); break; case kxxHash: - actual = XXH32(data, n + 1, 0); + actual = XXH32(data, static_cast(block_size) + 1, 0); break; default: - s = Status::Corruption("unknown checksum type"); + s = Status::Corruption( + "unknown checksum type " + ToString(footer.checksum()) + " in " + + file->file_name() + " offset " + ToString(handle.offset()) + + " size " + ToString(block_size)); } if (s.ok() && actual != value) { - s = Status::Corruption("block checksum mismatch"); + s = Status::Corruption( + "block checksum mismatch: expected " + ToString(actual) + ", got " + + ToString(value) + " in " + file->file_name() + " offset " + + ToString(handle.offset()) + " size " + ToString(block_size)); } if (!s.ok()) { return s; @@ -255,192 +306,290 @@ Status ReadBlock(RandomAccessFile* file, const Footer& footer, return s; } -// Decompress a block according to params -// May need to malloc a space for cache usage -Status DecompressBlock(BlockContents* result, size_t block_size, - bool do_uncompress, const char* buf, - const Slice& contents, bool use_stack_buf) { +// Read a block and check its CRC +// contents is the result of reading. +// According to the implementation of file->Read, contents may not point to buf +Status ReadBlock(RandomAccessFileReader* file, const Footer& footer, + const ReadOptions& options, const BlockHandle& handle, + Slice* contents, /* result of reading */ char* buf) { + size_t n = static_cast(handle.size()); Status s; - size_t n = block_size; - const char* data = contents.data(); - - result->data = Slice(); - result->cachable = false; - result->heap_allocated = false; - PERF_TIMER_GUARD(block_decompress_time); - rocksdb::CompressionType compression_type = - static_cast(data[n]); - // If the caller has requested that the block not be uncompressed - if (!do_uncompress || compression_type == kNoCompression) { - if (data != buf) { - // File implementation gave us pointer to some other data. - // Use it directly under the assumption that it will be live - // while the file is open. - result->data = Slice(data, n); - result->heap_allocated = false; - result->cachable = false; // Do not double-cache - } else { - if (use_stack_buf) { - // Need to allocate space in heap for cache usage - char* new_buf = new char[n]; - memcpy(new_buf, buf, n); - result->data = Slice(new_buf, n); - } else { - result->data = Slice(buf, n); - } - - result->heap_allocated = true; - result->cachable = true; - } - result->compression_type = compression_type; - s = Status::OK(); - } else { - s = UncompressBlockContents(data, n, result); + { + PERF_TIMER_GUARD(block_read_time); + s = file->Read(handle.offset(), n + kBlockTrailerSize, contents, buf); } - return s; -} -// Read and Decompress block -// Use buf in stack as temp reading buffer -Status ReadAndDecompressFast(RandomAccessFile* file, const Footer& footer, - const ReadOptions& options, - const BlockHandle& handle, BlockContents* result, - Env* env, bool do_uncompress) { - Status s; - Slice contents; - size_t n = static_cast(handle.size()); - char buf[DefaultStackBufferSize]; + PERF_COUNTER_ADD(block_read_count, 1); + PERF_COUNTER_ADD(block_read_byte, n + kBlockTrailerSize); - s = ReadBlock(file, footer, options, handle, &contents, buf); if (!s.ok()) { return s; } - s = DecompressBlock(result, n, do_uncompress, buf, contents, true); - if (!s.ok()) { - return s; + if (contents->size() != n + kBlockTrailerSize) { + return Status::Corruption("truncated block read from " + file->file_name() + + " offset " + ToString(handle.offset()) + + ", expected " + ToString(n + kBlockTrailerSize) + + " bytes, got " + ToString(contents->size())); } - return s; + return CheckBlockChecksum(options, footer, *contents, n, file, handle); } -// Read and Decompress block -// Use buf in heap as temp reading buffer -Status ReadAndDecompress(RandomAccessFile* file, const Footer& footer, - const ReadOptions& options, const BlockHandle& handle, - BlockContents* result, Env* env, bool do_uncompress) { - Status s; - Slice contents; - size_t n = static_cast(handle.size()); - char* buf = new char[n + kBlockTrailerSize]; +} // namespace - s = ReadBlock(file, footer, options, handle, &contents, buf); - if (!s.ok()) { - delete[] buf; - return s; +Status ReadBlockContents(RandomAccessFileReader* file, + FilePrefetchBuffer* prefetch_buffer, + const Footer& footer, const ReadOptions& read_options, + const BlockHandle& handle, BlockContents* contents, + const ImmutableCFOptions& ioptions, + bool decompression_requested, + const Slice& compression_dict, + const PersistentCacheOptions& cache_options) { + Status status; + Slice slice; + size_t n = static_cast(handle.size()); + std::unique_ptr heap_buf; + char stack_buf[DefaultStackBufferSize]; + char* used_buf = nullptr; + rocksdb::CompressionType compression_type; + + if (cache_options.persistent_cache && + !cache_options.persistent_cache->IsCompressed()) { + status = PersistentCacheHelper::LookupUncompressedPage(cache_options, + handle, contents); + if (status.ok()) { + // uncompressed page is found for the block handle + return status; + } else { + // uncompressed page is not found + if (ioptions.info_log && !status.IsNotFound()) { + assert(!status.ok()); + ROCKS_LOG_INFO(ioptions.info_log, + "Error reading from persistent cache. %s", + status.ToString().c_str()); + } + } } - s = DecompressBlock(result, n, do_uncompress, buf, contents, false); - if (!s.ok()) { - delete[] buf; - return s; + + bool got_from_prefetch_buffer = false; + if (prefetch_buffer != nullptr && + prefetch_buffer->TryReadFromCache( + handle.offset(), + static_cast(handle.size()) + kBlockTrailerSize, &slice)) { + status = + CheckBlockChecksum(read_options, footer, slice, + static_cast(handle.size()), file, handle); + if (!status.ok()) { + return status; + } + got_from_prefetch_buffer = true; + used_buf = const_cast(slice.data()); + } else if (cache_options.persistent_cache && + cache_options.persistent_cache->IsCompressed()) { + // lookup uncompressed cache mode p-cache + status = PersistentCacheHelper::LookupRawPage( + cache_options, handle, &heap_buf, n + kBlockTrailerSize); + } else { + status = Status::NotFound(); } - if (result->data.data() != buf) { - delete[] buf; + if (!got_from_prefetch_buffer) { + if (status.ok()) { + // cache hit + used_buf = heap_buf.get(); + slice = Slice(heap_buf.get(), n); + } else { + if (ioptions.info_log && !status.IsNotFound()) { + assert(!status.ok()); + ROCKS_LOG_INFO(ioptions.info_log, + "Error reading from persistent cache. %s", + status.ToString().c_str()); + } + // cache miss read from device + if (decompression_requested && + n + kBlockTrailerSize < DefaultStackBufferSize) { + // If we've got a small enough hunk of data, read it in to the + // trivially allocated stack buffer instead of needing a full malloc() + used_buf = &stack_buf[0]; + } else { + heap_buf = std::unique_ptr(new char[n + kBlockTrailerSize]); + used_buf = heap_buf.get(); + } + + status = ReadBlock(file, footer, read_options, handle, &slice, used_buf); + if (status.ok() && read_options.fill_cache && + cache_options.persistent_cache && + cache_options.persistent_cache->IsCompressed()) { + // insert to raw cache + PersistentCacheHelper::InsertRawPage(cache_options, handle, used_buf, + n + kBlockTrailerSize); + } + } + + if (!status.ok()) { + return status; + } } - return s; -} -Status ReadBlockContents(RandomAccessFile* file, const Footer& footer, - const ReadOptions& options, const BlockHandle& handle, - BlockContents* result, Env* env, bool do_uncompress) { - size_t n = static_cast(handle.size()); - if (do_uncompress && n + kBlockTrailerSize < DefaultStackBufferSize) { - return ReadAndDecompressFast(file, footer, options, handle, result, env, - do_uncompress); + PERF_TIMER_GUARD(block_decompress_time); + + compression_type = static_cast(slice.data()[n]); + + if (decompression_requested && compression_type != kNoCompression) { + // compressed page, uncompress, update cache + status = UncompressBlockContents(slice.data(), n, contents, + footer.version(), compression_dict, + ioptions); + } else if (slice.data() != used_buf) { + // the slice content is not the buffer provided + *contents = BlockContents(Slice(slice.data(), n), false, compression_type); } else { - return ReadAndDecompress(file, footer, options, handle, result, env, - do_uncompress); + // page is uncompressed, the buffer either stack or heap provided + if (got_from_prefetch_buffer || used_buf == &stack_buf[0]) { + heap_buf = std::unique_ptr(new char[n]); + memcpy(heap_buf.get(), used_buf, n); + } + *contents = BlockContents(std::move(heap_buf), n, true, compression_type); } + + if (status.ok() && !got_from_prefetch_buffer && read_options.fill_cache && + cache_options.persistent_cache && + !cache_options.persistent_cache->IsCompressed()) { + // insert to uncompressed cache + PersistentCacheHelper::InsertUncompressedPage(cache_options, handle, + *contents); + } + + return status; } -// -// The 'data' points to the raw block contents that was read in from file. -// This method allocates a new heap buffer and the raw block -// contents are uncompresed into this buffer. This -// buffer is returned via 'result' and it is upto the caller to -// free this buffer. -Status UncompressBlockContents(const char* data, size_t n, - BlockContents* result) { - char* ubuf = nullptr; +Status UncompressBlockContentsForCompressionType( + const char* data, size_t n, BlockContents* contents, + uint32_t format_version, const Slice& compression_dict, + CompressionType compression_type, const ImmutableCFOptions &ioptions) { + std::unique_ptr ubuf; + + assert(compression_type != kNoCompression && "Invalid compression type"); + + StopWatchNano timer(ioptions.env, + ShouldReportDetailedTime(ioptions.env, ioptions.statistics)); int decompress_size = 0; - assert(data[n] != kNoCompression); - switch (data[n]) { + switch (compression_type) { case kSnappyCompression: { size_t ulength = 0; static char snappy_corrupt_msg[] = "Snappy not supported or corrupted Snappy compressed block contents"; - if (!port::Snappy_GetUncompressedLength(data, n, &ulength)) { + if (!Snappy_GetUncompressedLength(data, n, &ulength)) { return Status::Corruption(snappy_corrupt_msg); } - ubuf = new char[ulength]; - if (!port::Snappy_Uncompress(data, n, ubuf)) { - delete[] ubuf; + ubuf.reset(new char[ulength]); + if (!Snappy_Uncompress(data, n, ubuf.get())) { return Status::Corruption(snappy_corrupt_msg); } - result->data = Slice(ubuf, ulength); - result->heap_allocated = true; - result->cachable = true; + *contents = BlockContents(std::move(ubuf), ulength, true, kNoCompression); break; } case kZlibCompression: - ubuf = port::Zlib_Uncompress(data, n, &decompress_size); - static char zlib_corrupt_msg[] = - "Zlib not supported or corrupted Zlib compressed block contents"; + ubuf.reset(Zlib_Uncompress( + data, n, &decompress_size, + GetCompressFormatForVersion(kZlibCompression, format_version), + compression_dict)); if (!ubuf) { + static char zlib_corrupt_msg[] = + "Zlib not supported or corrupted Zlib compressed block contents"; return Status::Corruption(zlib_corrupt_msg); } - result->data = Slice(ubuf, decompress_size); - result->heap_allocated = true; - result->cachable = true; + *contents = + BlockContents(std::move(ubuf), decompress_size, true, kNoCompression); break; case kBZip2Compression: - ubuf = port::BZip2_Uncompress(data, n, &decompress_size); - static char bzip2_corrupt_msg[] = - "Bzip2 not supported or corrupted Bzip2 compressed block contents"; + ubuf.reset(BZip2_Uncompress( + data, n, &decompress_size, + GetCompressFormatForVersion(kBZip2Compression, format_version))); if (!ubuf) { + static char bzip2_corrupt_msg[] = + "Bzip2 not supported or corrupted Bzip2 compressed block contents"; return Status::Corruption(bzip2_corrupt_msg); } - result->data = Slice(ubuf, decompress_size); - result->heap_allocated = true; - result->cachable = true; + *contents = + BlockContents(std::move(ubuf), decompress_size, true, kNoCompression); break; case kLZ4Compression: - ubuf = port::LZ4_Uncompress(data, n, &decompress_size); - static char lz4_corrupt_msg[] = - "LZ4 not supported or corrupted LZ4 compressed block contents"; + ubuf.reset(LZ4_Uncompress( + data, n, &decompress_size, + GetCompressFormatForVersion(kLZ4Compression, format_version), + compression_dict)); if (!ubuf) { + static char lz4_corrupt_msg[] = + "LZ4 not supported or corrupted LZ4 compressed block contents"; return Status::Corruption(lz4_corrupt_msg); } - result->data = Slice(ubuf, decompress_size); - result->heap_allocated = true; - result->cachable = true; + *contents = + BlockContents(std::move(ubuf), decompress_size, true, kNoCompression); break; case kLZ4HCCompression: - ubuf = port::LZ4_Uncompress(data, n, &decompress_size); - static char lz4hc_corrupt_msg[] = - "LZ4HC not supported or corrupted LZ4HC compressed block contents"; + ubuf.reset(LZ4_Uncompress( + data, n, &decompress_size, + GetCompressFormatForVersion(kLZ4HCCompression, format_version), + compression_dict)); if (!ubuf) { + static char lz4hc_corrupt_msg[] = + "LZ4HC not supported or corrupted LZ4HC compressed block contents"; return Status::Corruption(lz4hc_corrupt_msg); } - result->data = Slice(ubuf, decompress_size); - result->heap_allocated = true; - result->cachable = true; + *contents = + BlockContents(std::move(ubuf), decompress_size, true, kNoCompression); + break; + case kXpressCompression: + ubuf.reset(XPRESS_Uncompress(data, n, &decompress_size)); + if (!ubuf) { + static char xpress_corrupt_msg[] = + "XPRESS not supported or corrupted XPRESS compressed block contents"; + return Status::Corruption(xpress_corrupt_msg); + } + *contents = + BlockContents(std::move(ubuf), decompress_size, true, kNoCompression); + break; + case kZSTD: + case kZSTDNotFinalCompression: + ubuf.reset(ZSTD_Uncompress(data, n, &decompress_size, compression_dict)); + if (!ubuf) { + static char zstd_corrupt_msg[] = + "ZSTD not supported or corrupted ZSTD compressed block contents"; + return Status::Corruption(zstd_corrupt_msg); + } + *contents = + BlockContents(std::move(ubuf), decompress_size, true, kNoCompression); break; default: return Status::Corruption("bad block type"); } - result->compression_type = kNoCompression; // not compressed any more + + if(ShouldReportDetailedTime(ioptions.env, ioptions.statistics)){ + MeasureTime(ioptions.statistics, DECOMPRESSION_TIMES_NANOS, + timer.ElapsedNanos()); + MeasureTime(ioptions.statistics, BYTES_DECOMPRESSED, contents->data.size()); + RecordTick(ioptions.statistics, NUMBER_BLOCK_DECOMPRESSED); + } + return Status::OK(); } +// +// The 'data' points to the raw block contents that was read in from file. +// This method allocates a new heap buffer and the raw block +// contents are uncompresed into this buffer. This +// buffer is returned via 'result' and it is upto the caller to +// free this buffer. +// format_version is the block format as defined in include/rocksdb/table.h +Status UncompressBlockContents(const char* data, size_t n, + BlockContents* contents, uint32_t format_version, + const Slice& compression_dict, + const ImmutableCFOptions &ioptions) { + assert(data[n] != kNoCompression); + return UncompressBlockContentsForCompressionType( + data, n, contents, format_version, compression_dict, + (CompressionType)data[n], ioptions); +} + } // namespace rocksdb diff --git a/table/format.h b/table/format.h index a971c1a67c9..512b4a32bfa 100644 --- a/table/format.h +++ b/table/format.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -15,12 +15,19 @@ #include "rocksdb/options.h" #include "rocksdb/table.h" +#include "options/cf_options.h" +#include "port/port.h" // noexcept +#include "table/persistent_cache_options.h" +#include "util/file_reader_writer.h" + namespace rocksdb { class Block; class RandomAccessFile; struct ReadOptions; +extern bool ShouldReportDetailedTime(Env* env, Statistics* stats); + // the length of the magic number in bytes. const int kMagicNumberLengthByte = 8; @@ -33,15 +40,18 @@ class BlockHandle { // The offset of the block in the file. uint64_t offset() const { return offset_; } - void set_offset(uint64_t offset) { offset_ = offset; } + void set_offset(uint64_t _offset) { offset_ = _offset; } // The size of the stored block uint64_t size() const { return size_; } - void set_size(uint64_t size) { size_ = size; } + void set_size(uint64_t _size) { size_ = _size; } void EncodeTo(std::string* dst) const; Status DecodeFrom(Slice* input); + // Return a string that contains the copy of handle. + std::string ToString(bool hex = true) const; + // if the block handle's offset and size are both "0", we will view it // as a null block handle that points to no where. bool IsNull() const { @@ -56,12 +66,28 @@ class BlockHandle { enum { kMaxEncodedLength = 10 + 10 }; private: - uint64_t offset_ = 0; - uint64_t size_ = 0; + uint64_t offset_; + uint64_t size_; static const BlockHandle kNullBlockHandle; }; +inline uint32_t GetCompressFormatForVersion(CompressionType compression_type, + uint32_t version) { + // snappy is not versioned + assert(compression_type != kSnappyCompression && + compression_type != kXpressCompression && + compression_type != kNoCompression); + // As of version 2, we encode compressed block with + // compress_format_version == 2. Before that, the version is 1. + // DO NOT CHANGE THIS FUNCTION, it affects disk format + return version >= 2 ? 2 : 1; +} + +inline bool BlockBasedTableSupportedVersion(uint32_t version) { + return version <= 2; +} + // Footer encapsulates the fixed information stored at the tail // end of every table file. class Footer { @@ -69,12 +95,13 @@ class Footer { // Constructs a footer without specifying its table magic number. // In such case, the table magic number of such footer should be // initialized via @ReadFooterFromFile(). - Footer() : Footer(kInvalidTableMagicNumber) {} + // Use this when you plan to load Footer with DecodeFrom(). Never use this + // when you plan to EncodeTo. + Footer() : Footer(kInvalidTableMagicNumber, 0) {} - // @table_magic_number serves two purposes: - // 1. Identify different types of the tables. - // 2. Help us to identify if a given file is a valid sst. - explicit Footer(uint64_t table_magic_number); + // Use this constructor when you plan to write out the footer using + // EncodeTo(). Never use this constructor with DecodeFrom(). + Footer(uint64_t table_magic_number, uint32_t version); // The version of the footer in this file uint32_t version() const { return version_; } @@ -94,20 +121,13 @@ class Footer { uint64_t table_magic_number() const { return table_magic_number_; } - // The version of Footer we encode - enum { - kLegacyFooter = 0, - kFooterVersion = 1, - }; - void EncodeTo(std::string* dst) const; - // Set the current footer based on the input slice. If table_magic_number_ - // is not set (i.e., HasInitializedTableMagicNumber() is true), then this - // function will also initialize table_magic_number_. Otherwise, this - // function will verify whether the magic number specified in the input - // slice matches table_magic_number_ and update the current footer only - // when the test passes. + // Set the current footer based on the input slice. + // + // REQUIRES: table_magic_number_ is not set (i.e., + // HasInitializedTableMagicNumber() is true). The function will initialize the + // magic number Status DecodeFrom(Slice* input); // Encoded length of a Footer. Note that the serialization of a Footer will @@ -118,17 +138,19 @@ class Footer { // Footer version 0 (legacy) will always occupy exactly this many bytes. // It consists of two block handles, padding, and a magic number. kVersion0EncodedLength = 2 * BlockHandle::kMaxEncodedLength + 8, - // Footer version 1 will always occupy exactly this many bytes. - // It consists of the checksum type, two block handles, padding, - // a version number, and a magic number - kVersion1EncodedLength = 1 + 2 * BlockHandle::kMaxEncodedLength + 4 + 8, - + // Footer of versions 1 and higher will always occupy exactly this many + // bytes. It consists of the checksum type, two block handles, padding, + // a version number (bigger than 1), and a magic number + kNewVersionsEncodedLength = 1 + 2 * BlockHandle::kMaxEncodedLength + 4 + 8, kMinEncodedLength = kVersion0EncodedLength, - kMaxEncodedLength = kVersion1EncodedLength + kMaxEncodedLength = kNewVersionsEncodedLength, }; static const uint64_t kInvalidTableMagicNumber = 0; + // convert this object to a human readable form + std::string ToString() const; + private: // REQUIRES: magic number wasn't initialized. void set_table_magic_number(uint64_t magic_number) { @@ -150,9 +172,12 @@ class Footer { }; // Read the footer from file -Status ReadFooterFromFile(RandomAccessFile* file, - uint64_t file_size, - Footer* footer); +// If enforce_table_magic_number != 0, ReadFooterFromFile() will return +// corruption if table_magic number is not equal to enforce_table_magic_number +Status ReadFooterFromFile(RandomAccessFileReader* file, + FilePrefetchBuffer* prefetch_buffer, + uint64_t file_size, Footer* footer, + uint64_t enforce_table_magic_number = 0); // 1-byte type + 32-bit crc static const size_t kBlockTrailerSize = 5; @@ -160,39 +185,74 @@ static const size_t kBlockTrailerSize = 5; struct BlockContents { Slice data; // Actual contents of data bool cachable; // True iff data can be cached - bool heap_allocated; // True iff caller should delete[] data.data() CompressionType compression_type; + std::unique_ptr allocation; + + BlockContents() : cachable(false), compression_type(kNoCompression) {} + + BlockContents(const Slice& _data, bool _cachable, + CompressionType _compression_type) + : data(_data), cachable(_cachable), compression_type(_compression_type) {} + + BlockContents(std::unique_ptr&& _data, size_t _size, bool _cachable, + CompressionType _compression_type) + : data(_data.get(), _size), + cachable(_cachable), + compression_type(_compression_type), + allocation(std::move(_data)) {} + + BlockContents(BlockContents&& other) ROCKSDB_NOEXCEPT { *this = std::move(other); } + + BlockContents& operator=(BlockContents&& other) { + data = std::move(other.data); + cachable = other.cachable; + compression_type = other.compression_type; + allocation = std::move(other.allocation); + return *this; + } }; // Read the block identified by "handle" from "file". On failure // return non-OK. On success fill *result and return OK. -extern Status ReadBlockContents(RandomAccessFile* file, - const Footer& footer, - const ReadOptions& options, - const BlockHandle& handle, - BlockContents* result, - Env* env, - bool do_uncompress); +extern Status ReadBlockContents( + RandomAccessFileReader* file, FilePrefetchBuffer* prefetch_buffer, + const Footer& footer, const ReadOptions& options, const BlockHandle& handle, + BlockContents* contents, const ImmutableCFOptions& ioptions, + bool do_uncompress = true, const Slice& compression_dict = Slice(), + const PersistentCacheOptions& cache_options = PersistentCacheOptions()); // The 'data' points to the raw block contents read in from file. // This method allocates a new heap buffer and the raw block // contents are uncompresed into this buffer. This buffer is // returned via 'result' and it is upto the caller to // free this buffer. -extern Status UncompressBlockContents(const char* data, - size_t n, - BlockContents* result); +// For description of compress_format_version and possible values, see +// util/compression.h +extern Status UncompressBlockContents(const char* data, size_t n, + BlockContents* contents, + uint32_t compress_format_version, + const Slice& compression_dict, + const ImmutableCFOptions &ioptions); + +// This is an extension to UncompressBlockContents that accepts +// a specific compression type. This is used by un-wrapped blocks +// with no compression header. +extern Status UncompressBlockContentsForCompressionType( + const char* data, size_t n, BlockContents* contents, + uint32_t compress_format_version, const Slice& compression_dict, + CompressionType compression_type, const ImmutableCFOptions &ioptions); // Implementation details follow. Clients should ignore, +// TODO(andrewkr): we should prefer one way of representing a null/uninitialized +// BlockHandle. Currently we use zeros for null and use negation-of-zeros for +// uninitialized. inline BlockHandle::BlockHandle() : BlockHandle(~static_cast(0), ~static_cast(0)) { } -inline BlockHandle::BlockHandle(uint64_t offset, uint64_t size) - : offset_(offset), - size_(size) { -} +inline BlockHandle::BlockHandle(uint64_t _offset, uint64_t _size) + : offset_(_offset), size_(_size) {} } // namespace rocksdb diff --git a/table/full_filter_bits_builder.h b/table/full_filter_bits_builder.h new file mode 100644 index 00000000000..b3be7e897f0 --- /dev/null +++ b/table/full_filter_bits_builder.h @@ -0,0 +1,73 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// Copyright (c) 2012 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include +#include +#include + +#include "rocksdb/filter_policy.h" + +namespace rocksdb { + +class Slice; + +class FullFilterBitsBuilder : public FilterBitsBuilder { + public: + explicit FullFilterBitsBuilder(const size_t bits_per_key, + const size_t num_probes); + + ~FullFilterBitsBuilder(); + + virtual void AddKey(const Slice& key) override; + + // Create a filter that for hashes [0, n-1], the filter is allocated here + // When creating filter, it is ensured that + // total_bits = num_lines * CACHE_LINE_SIZE * 8 + // dst len is >= 5, 1 for num_probes, 4 for num_lines + // Then total_bits = (len - 5) * 8, and cache_line_size could be calculated + // +----------------------------------------------------------------+ + // | filter data with length total_bits/8 | + // +----------------------------------------------------------------+ + // | | + // | ... | + // | | + // +----------------------------------------------------------------+ + // | ... | num_probes : 1 byte | num_lines : 4 bytes | + // +----------------------------------------------------------------+ + virtual Slice Finish(std::unique_ptr* buf) override; + + // Calculate num of entries fit into a space. + virtual int CalculateNumEntry(const uint32_t space) override; + + // Calculate space for new filter. This is reverse of CalculateNumEntry. + uint32_t CalculateSpace(const int num_entry, uint32_t* total_bits, + uint32_t* num_lines); + + private: + size_t bits_per_key_; + size_t num_probes_; + std::vector hash_entries_; + + // Get totalbits that optimized for cpu cache line + uint32_t GetTotalBitsForLocality(uint32_t total_bits); + + // Reserve space for new filter + char* ReserveSpace(const int num_entry, uint32_t* total_bits, + uint32_t* num_lines); + + // Assuming single threaded access to this function. + void AddHash(uint32_t h, char* data, uint32_t num_lines, uint32_t total_bits); + + // No Copy allowed + FullFilterBitsBuilder(const FullFilterBitsBuilder&); + void operator=(const FullFilterBitsBuilder&); +}; + +} // namespace rocksdb diff --git a/table/full_filter_block.cc b/table/full_filter_block.cc new file mode 100644 index 00000000000..5739494e8dd --- /dev/null +++ b/table/full_filter_block.cc @@ -0,0 +1,113 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "table/full_filter_block.h" + +#include "monitoring/perf_context_imp.h" +#include "port/port.h" +#include "rocksdb/filter_policy.h" +#include "util/coding.h" + +namespace rocksdb { + +FullFilterBlockBuilder::FullFilterBlockBuilder( + const SliceTransform* prefix_extractor, bool whole_key_filtering, + FilterBitsBuilder* filter_bits_builder) + : prefix_extractor_(prefix_extractor), + whole_key_filtering_(whole_key_filtering), + num_added_(0) { + assert(filter_bits_builder != nullptr); + filter_bits_builder_.reset(filter_bits_builder); +} + +void FullFilterBlockBuilder::Add(const Slice& key) { + if (whole_key_filtering_) { + AddKey(key); + } + if (prefix_extractor_ && prefix_extractor_->InDomain(key)) { + AddPrefix(key); + } +} + +// Add key to filter if needed +inline void FullFilterBlockBuilder::AddKey(const Slice& key) { + filter_bits_builder_->AddKey(key); + num_added_++; +} + +// Add prefix to filter if needed +inline void FullFilterBlockBuilder::AddPrefix(const Slice& key) { + Slice prefix = prefix_extractor_->Transform(key); + AddKey(prefix); +} + +Slice FullFilterBlockBuilder::Finish(const BlockHandle& tmp, Status* status) { + // In this impl we ignore BlockHandle + *status = Status::OK(); + if (num_added_ != 0) { + num_added_ = 0; + return filter_bits_builder_->Finish(&filter_data_); + } + return Slice(); +} + +FullFilterBlockReader::FullFilterBlockReader( + const SliceTransform* prefix_extractor, bool _whole_key_filtering, + const Slice& contents, FilterBitsReader* filter_bits_reader, + Statistics* stats) + : FilterBlockReader(contents.size(), stats, _whole_key_filtering), + prefix_extractor_(prefix_extractor), + contents_(contents) { + assert(filter_bits_reader != nullptr); + filter_bits_reader_.reset(filter_bits_reader); +} + +FullFilterBlockReader::FullFilterBlockReader( + const SliceTransform* prefix_extractor, bool _whole_key_filtering, + BlockContents&& contents, FilterBitsReader* filter_bits_reader, + Statistics* stats) + : FullFilterBlockReader(prefix_extractor, _whole_key_filtering, + contents.data, filter_bits_reader, stats) { + block_contents_ = std::move(contents); +} + +bool FullFilterBlockReader::KeyMayMatch(const Slice& key, uint64_t block_offset, + const bool no_io, + const Slice* const const_ikey_ptr) { + assert(block_offset == kNotValid); + if (!whole_key_filtering_) { + return true; + } + return MayMatch(key); +} + +bool FullFilterBlockReader::PrefixMayMatch(const Slice& prefix, + uint64_t block_offset, + const bool no_io, + const Slice* const const_ikey_ptr) { + assert(block_offset == kNotValid); + if (!prefix_extractor_) { + return true; + } + return MayMatch(prefix); +} + +bool FullFilterBlockReader::MayMatch(const Slice& entry) { + if (contents_.size() != 0) { + if (filter_bits_reader_->MayMatch(entry)) { + PERF_COUNTER_ADD(bloom_sst_hit_count, 1); + return true; + } else { + PERF_COUNTER_ADD(bloom_sst_miss_count, 1); + return false; + } + } + return true; // remain the same with block_based filter +} + +size_t FullFilterBlockReader::ApproximateMemoryUsage() const { + return contents_.size(); +} +} // namespace rocksdb diff --git a/table/full_filter_block.h b/table/full_filter_block.h new file mode 100644 index 00000000000..be27c58b61d --- /dev/null +++ b/table/full_filter_block.h @@ -0,0 +1,117 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include +#include +#include +#include +#include "rocksdb/options.h" +#include "rocksdb/slice.h" +#include "rocksdb/slice_transform.h" +#include "db/dbformat.h" +#include "util/hash.h" +#include "table/filter_block.h" + +namespace rocksdb { + +class FilterPolicy; +class FilterBitsBuilder; +class FilterBitsReader; + +// A FullFilterBlockBuilder is used to construct a full filter for a +// particular Table. It generates a single string which is stored as +// a special block in the Table. +// The format of full filter block is: +// +----------------------------------------------------------------+ +// | full filter for all keys in sst file | +// +----------------------------------------------------------------+ +// The full filter can be very large. At the end of it, we put +// num_probes: how many hash functions are used in bloom filter +// +class FullFilterBlockBuilder : public FilterBlockBuilder { + public: + explicit FullFilterBlockBuilder(const SliceTransform* prefix_extractor, + bool whole_key_filtering, + FilterBitsBuilder* filter_bits_builder); + // bits_builder is created in filter_policy, it should be passed in here + // directly. and be deleted here + ~FullFilterBlockBuilder() {} + + virtual bool IsBlockBased() override { return false; } + virtual void StartBlock(uint64_t block_offset) override {} + virtual void Add(const Slice& key) override; + virtual Slice Finish(const BlockHandle& tmp, Status* status) override; + using FilterBlockBuilder::Finish; + + protected: + virtual void AddKey(const Slice& key); + std::unique_ptr filter_bits_builder_; + + private: + // important: all of these might point to invalid addresses + // at the time of destruction of this filter block. destructor + // should NOT dereference them. + const SliceTransform* prefix_extractor_; + bool whole_key_filtering_; + + uint32_t num_added_; + std::unique_ptr filter_data_; + + void AddPrefix(const Slice& key); + + // No copying allowed + FullFilterBlockBuilder(const FullFilterBlockBuilder&); + void operator=(const FullFilterBlockBuilder&); +}; + +// A FilterBlockReader is used to parse filter from SST table. +// KeyMayMatch and PrefixMayMatch would trigger filter checking +class FullFilterBlockReader : public FilterBlockReader { + public: + // REQUIRES: "contents" and filter_bits_reader must stay live + // while *this is live. + explicit FullFilterBlockReader(const SliceTransform* prefix_extractor, + bool whole_key_filtering, + const Slice& contents, + FilterBitsReader* filter_bits_reader, + Statistics* statistics); + explicit FullFilterBlockReader(const SliceTransform* prefix_extractor, + bool whole_key_filtering, + BlockContents&& contents, + FilterBitsReader* filter_bits_reader, + Statistics* statistics); + + // bits_reader is created in filter_policy, it should be passed in here + // directly. and be deleted here + ~FullFilterBlockReader() {} + + virtual bool IsBlockBased() override { return false; } + virtual bool KeyMayMatch( + const Slice& key, uint64_t block_offset = kNotValid, + const bool no_io = false, + const Slice* const const_ikey_ptr = nullptr) override; + virtual bool PrefixMayMatch( + const Slice& prefix, uint64_t block_offset = kNotValid, + const bool no_io = false, + const Slice* const const_ikey_ptr = nullptr) override; + virtual size_t ApproximateMemoryUsage() const override; + + private: + const SliceTransform* prefix_extractor_; + Slice contents_; + std::unique_ptr filter_bits_reader_; + BlockContents block_contents_; + std::unique_ptr filter_data_; + + // No copying allowed + FullFilterBlockReader(const FullFilterBlockReader&); + bool MayMatch(const Slice& entry); + void operator=(const FullFilterBlockReader&); +}; + +} // namespace rocksdb diff --git a/table/full_filter_block_test.cc b/table/full_filter_block_test.cc new file mode 100644 index 00000000000..5fbda4c6f03 --- /dev/null +++ b/table/full_filter_block_test.cc @@ -0,0 +1,189 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "table/full_filter_block.h" + +#include "rocksdb/filter_policy.h" +#include "util/coding.h" +#include "util/hash.h" +#include "util/string_util.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +class TestFilterBitsBuilder : public FilterBitsBuilder { + public: + explicit TestFilterBitsBuilder() {} + + // Add Key to filter + virtual void AddKey(const Slice& key) override { + hash_entries_.push_back(Hash(key.data(), key.size(), 1)); + } + + // Generate the filter using the keys that are added + virtual Slice Finish(std::unique_ptr* buf) override { + uint32_t len = static_cast(hash_entries_.size()) * 4; + char* data = new char[len]; + for (size_t i = 0; i < hash_entries_.size(); i++) { + EncodeFixed32(data + i * 4, hash_entries_[i]); + } + const char* const_data = data; + buf->reset(const_data); + return Slice(data, len); + } + + private: + std::vector hash_entries_; +}; + +class TestFilterBitsReader : public FilterBitsReader { + public: + explicit TestFilterBitsReader(const Slice& contents) + : data_(contents.data()), len_(static_cast(contents.size())) {} + + virtual bool MayMatch(const Slice& entry) override { + uint32_t h = Hash(entry.data(), entry.size(), 1); + for (size_t i = 0; i + 4 <= len_; i += 4) { + if (h == DecodeFixed32(data_ + i)) { + return true; + } + } + return false; + } + + private: + const char* data_; + uint32_t len_; +}; + + +class TestHashFilter : public FilterPolicy { + public: + virtual const char* Name() const override { return "TestHashFilter"; } + + virtual void CreateFilter(const Slice* keys, int n, + std::string* dst) const override { + for (int i = 0; i < n; i++) { + uint32_t h = Hash(keys[i].data(), keys[i].size(), 1); + PutFixed32(dst, h); + } + } + + virtual bool KeyMayMatch(const Slice& key, + const Slice& filter) const override { + uint32_t h = Hash(key.data(), key.size(), 1); + for (unsigned int i = 0; i + 4 <= filter.size(); i += 4) { + if (h == DecodeFixed32(filter.data() + i)) { + return true; + } + } + return false; + } + + virtual FilterBitsBuilder* GetFilterBitsBuilder() const override { + return new TestFilterBitsBuilder(); + } + + virtual FilterBitsReader* GetFilterBitsReader(const Slice& contents) + const override { + return new TestFilterBitsReader(contents); + } +}; + +class PluginFullFilterBlockTest : public testing::Test { + public: + BlockBasedTableOptions table_options_; + + PluginFullFilterBlockTest() { + table_options_.filter_policy.reset(new TestHashFilter()); + } +}; + +TEST_F(PluginFullFilterBlockTest, PluginEmptyBuilder) { + FullFilterBlockBuilder builder( + nullptr, true, table_options_.filter_policy->GetFilterBitsBuilder()); + Slice block = builder.Finish(); + ASSERT_EQ("", EscapeString(block)); + + FullFilterBlockReader reader( + nullptr, true, block, + table_options_.filter_policy->GetFilterBitsReader(block), nullptr); + // Remain same symantic with blockbased filter + ASSERT_TRUE(reader.KeyMayMatch("foo")); +} + +TEST_F(PluginFullFilterBlockTest, PluginSingleChunk) { + FullFilterBlockBuilder builder( + nullptr, true, table_options_.filter_policy->GetFilterBitsBuilder()); + builder.Add("foo"); + builder.Add("bar"); + builder.Add("box"); + builder.Add("box"); + builder.Add("hello"); + Slice block = builder.Finish(); + FullFilterBlockReader reader( + nullptr, true, block, + table_options_.filter_policy->GetFilterBitsReader(block), nullptr); + ASSERT_TRUE(reader.KeyMayMatch("foo")); + ASSERT_TRUE(reader.KeyMayMatch("bar")); + ASSERT_TRUE(reader.KeyMayMatch("box")); + ASSERT_TRUE(reader.KeyMayMatch("hello")); + ASSERT_TRUE(reader.KeyMayMatch("foo")); + ASSERT_TRUE(!reader.KeyMayMatch("missing")); + ASSERT_TRUE(!reader.KeyMayMatch("other")); +} + +class FullFilterBlockTest : public testing::Test { + public: + BlockBasedTableOptions table_options_; + + FullFilterBlockTest() { + table_options_.filter_policy.reset(NewBloomFilterPolicy(10, false)); + } + + ~FullFilterBlockTest() {} +}; + +TEST_F(FullFilterBlockTest, EmptyBuilder) { + FullFilterBlockBuilder builder( + nullptr, true, table_options_.filter_policy->GetFilterBitsBuilder()); + Slice block = builder.Finish(); + ASSERT_EQ("", EscapeString(block)); + + FullFilterBlockReader reader( + nullptr, true, block, + table_options_.filter_policy->GetFilterBitsReader(block), nullptr); + // Remain same symantic with blockbased filter + ASSERT_TRUE(reader.KeyMayMatch("foo")); +} + +TEST_F(FullFilterBlockTest, SingleChunk) { + FullFilterBlockBuilder builder( + nullptr, true, table_options_.filter_policy->GetFilterBitsBuilder()); + builder.Add("foo"); + builder.Add("bar"); + builder.Add("box"); + builder.Add("box"); + builder.Add("hello"); + Slice block = builder.Finish(); + FullFilterBlockReader reader( + nullptr, true, block, + table_options_.filter_policy->GetFilterBitsReader(block), nullptr); + ASSERT_TRUE(reader.KeyMayMatch("foo")); + ASSERT_TRUE(reader.KeyMayMatch("bar")); + ASSERT_TRUE(reader.KeyMayMatch("box")); + ASSERT_TRUE(reader.KeyMayMatch("hello")); + ASSERT_TRUE(reader.KeyMayMatch("foo")); + ASSERT_TRUE(!reader.KeyMayMatch("missing")); + ASSERT_TRUE(!reader.KeyMayMatch("other")); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/table/get_context.cc b/table/get_context.cc new file mode 100644 index 00000000000..258891ec4c3 --- /dev/null +++ b/table/get_context.cc @@ -0,0 +1,213 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "table/get_context.h" +#include "db/merge_helper.h" +#include "db/pinned_iterators_manager.h" +#include "monitoring/file_read_sample.h" +#include "monitoring/perf_context_imp.h" +#include "monitoring/statistics.h" +#include "rocksdb/env.h" +#include "rocksdb/merge_operator.h" +#include "rocksdb/statistics.h" + +namespace rocksdb { + +namespace { + +void appendToReplayLog(std::string* replay_log, ValueType type, Slice value) { +#ifndef ROCKSDB_LITE + if (replay_log) { + if (replay_log->empty()) { + // Optimization: in the common case of only one operation in the + // log, we allocate the exact amount of space needed. + replay_log->reserve(1 + VarintLength(value.size()) + value.size()); + } + replay_log->push_back(type); + PutLengthPrefixedSlice(replay_log, value); + } +#endif // ROCKSDB_LITE +} + +} // namespace + +GetContext::GetContext( + const Comparator* ucmp, const MergeOperator* merge_operator, Logger* logger, + Statistics* statistics, GetState init_state, const Slice& user_key, + PinnableSlice* pinnable_val, bool* value_found, MergeContext* merge_context, + RangeDelAggregator* _range_del_agg, Env* env, SequenceNumber* seq, + PinnedIteratorsManager* _pinned_iters_mgr, bool* is_blob_index) + : ucmp_(ucmp), + merge_operator_(merge_operator), + logger_(logger), + statistics_(statistics), + state_(init_state), + user_key_(user_key), + pinnable_val_(pinnable_val), + value_found_(value_found), + merge_context_(merge_context), + range_del_agg_(_range_del_agg), + env_(env), + seq_(seq), + replay_log_(nullptr), + pinned_iters_mgr_(_pinned_iters_mgr), + is_blob_index_(is_blob_index) { + if (seq_) { + *seq_ = kMaxSequenceNumber; + } + sample_ = should_sample_file_read(); +} + +// Called from TableCache::Get and Table::Get when file/block in which +// key may exist are not there in TableCache/BlockCache respectively. In this +// case we can't guarantee that key does not exist and are not permitted to do +// IO to be certain.Set the status=kFound and value_found=false to let the +// caller know that key may exist but is not there in memory +void GetContext::MarkKeyMayExist() { + state_ = kFound; + if (value_found_ != nullptr) { + *value_found_ = false; + } +} + +void GetContext::SaveValue(const Slice& value, SequenceNumber seq) { + assert(state_ == kNotFound); + appendToReplayLog(replay_log_, kTypeValue, value); + + state_ = kFound; + if (LIKELY(pinnable_val_ != nullptr)) { + pinnable_val_->PinSelf(value); + } +} + +bool GetContext::SaveValue(const ParsedInternalKey& parsed_key, + const Slice& value, Cleanable* value_pinner) { + assert((state_ != kMerge && parsed_key.type != kTypeMerge) || + merge_context_ != nullptr); + if (ucmp_->Equal(parsed_key.user_key, user_key_)) { + appendToReplayLog(replay_log_, parsed_key.type, value); + + if (seq_ != nullptr) { + // Set the sequence number if it is uninitialized + if (*seq_ == kMaxSequenceNumber) { + *seq_ = parsed_key.sequence; + } + } + + auto type = parsed_key.type; + // Key matches. Process it + if ((type == kTypeValue || type == kTypeMerge || type == kTypeBlobIndex) && + range_del_agg_ != nullptr && range_del_agg_->ShouldDelete(parsed_key)) { + type = kTypeRangeDeletion; + } + switch (type) { + case kTypeValue: + case kTypeBlobIndex: + assert(state_ == kNotFound || state_ == kMerge); + if (type == kTypeBlobIndex && is_blob_index_ == nullptr) { + // Blob value not supported. Stop. + state_ = kBlobIndex; + return false; + } + if (kNotFound == state_) { + state_ = kFound; + if (LIKELY(pinnable_val_ != nullptr)) { + if (LIKELY(value_pinner != nullptr)) { + // If the backing resources for the value are provided, pin them + pinnable_val_->PinSlice(value, value_pinner); + } else { + // Otherwise copy the value + pinnable_val_->PinSelf(value); + } + } + } else if (kMerge == state_) { + assert(merge_operator_ != nullptr); + state_ = kFound; + if (LIKELY(pinnable_val_ != nullptr)) { + Status merge_status = MergeHelper::TimedFullMerge( + merge_operator_, user_key_, &value, + merge_context_->GetOperands(), pinnable_val_->GetSelf(), + logger_, statistics_, env_); + pinnable_val_->PinSelf(); + if (!merge_status.ok()) { + state_ = kCorrupt; + } + } + } + if (is_blob_index_ != nullptr) { + *is_blob_index_ = (type == kTypeBlobIndex); + } + return false; + + case kTypeDeletion: + case kTypeSingleDeletion: + case kTypeRangeDeletion: + // TODO(noetzli): Verify correctness once merge of single-deletes + // is supported + assert(state_ == kNotFound || state_ == kMerge); + if (kNotFound == state_) { + state_ = kDeleted; + } else if (kMerge == state_) { + state_ = kFound; + if (LIKELY(pinnable_val_ != nullptr)) { + Status merge_status = MergeHelper::TimedFullMerge( + merge_operator_, user_key_, nullptr, + merge_context_->GetOperands(), pinnable_val_->GetSelf(), + logger_, statistics_, env_); + pinnable_val_->PinSelf(); + if (!merge_status.ok()) { + state_ = kCorrupt; + } + } + } + return false; + + case kTypeMerge: + assert(state_ == kNotFound || state_ == kMerge); + state_ = kMerge; + // value_pinner is not set from plain_table_reader.cc for example. + if (pinned_iters_mgr() && pinned_iters_mgr()->PinningEnabled() && + value_pinner != nullptr) { + value_pinner->DelegateCleanupsTo(pinned_iters_mgr()); + merge_context_->PushOperand(value, true /*value_pinned*/); + } else { + merge_context_->PushOperand(value, false); + } + return true; + + default: + assert(false); + break; + } + } + + // state_ could be Corrupt, merge or notfound + return false; +} + +void replayGetContextLog(const Slice& replay_log, const Slice& user_key, + GetContext* get_context, Cleanable* value_pinner) { +#ifndef ROCKSDB_LITE + Slice s = replay_log; + while (s.size()) { + auto type = static_cast(*s.data()); + s.remove_prefix(1); + Slice value; + bool ret = GetLengthPrefixedSlice(&s, &value); + assert(ret); + (void)ret; + + // Since SequenceNumber is not stored and unknown, we will use + // kMaxSequenceNumber. + get_context->SaveValue( + ParsedInternalKey(user_key, kMaxSequenceNumber, type), value, + value_pinner); + } +#else // ROCKSDB_LITE + assert(false); +#endif // ROCKSDB_LITE +} + +} // namespace rocksdb diff --git a/table/get_context.h b/table/get_context.h new file mode 100644 index 00000000000..a708f6be745 --- /dev/null +++ b/table/get_context.h @@ -0,0 +1,95 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#include +#include "db/merge_context.h" +#include "db/range_del_aggregator.h" +#include "rocksdb/env.h" +#include "rocksdb/types.h" +#include "table/block.h" + +namespace rocksdb { +class MergeContext; +class PinnedIteratorsManager; + +class GetContext { + public: + enum GetState { + kNotFound, + kFound, + kDeleted, + kCorrupt, + kMerge, // saver contains the current merge result (the operands) + kBlobIndex, + }; + + GetContext(const Comparator* ucmp, const MergeOperator* merge_operator, + Logger* logger, Statistics* statistics, GetState init_state, + const Slice& user_key, PinnableSlice* value, bool* value_found, + MergeContext* merge_context, RangeDelAggregator* range_del_agg, + Env* env, SequenceNumber* seq = nullptr, + PinnedIteratorsManager* _pinned_iters_mgr = nullptr, + bool* is_blob_index = nullptr); + + void MarkKeyMayExist(); + + // Records this key, value, and any meta-data (such as sequence number and + // state) into this GetContext. + // + // Returns True if more keys need to be read (due to merges) or + // False if the complete value has been found. + bool SaveValue(const ParsedInternalKey& parsed_key, const Slice& value, + Cleanable* value_pinner = nullptr); + + // Simplified version of the previous function. Should only be used when we + // know that the operation is a Put. + void SaveValue(const Slice& value, SequenceNumber seq); + + GetState State() const { return state_; } + + RangeDelAggregator* range_del_agg() { return range_del_agg_; } + + PinnedIteratorsManager* pinned_iters_mgr() { return pinned_iters_mgr_; } + + // If a non-null string is passed, all the SaveValue calls will be + // logged into the string. The operations can then be replayed on + // another GetContext with replayGetContextLog. + void SetReplayLog(std::string* replay_log) { replay_log_ = replay_log; } + + // Do we need to fetch the SequenceNumber for this key? + bool NeedToReadSequence() const { return (seq_ != nullptr); } + + bool sample() const { return sample_; } + + private: + const Comparator* ucmp_; + const MergeOperator* merge_operator_; + // the merge operations encountered; + Logger* logger_; + Statistics* statistics_; + + GetState state_; + Slice user_key_; + PinnableSlice* pinnable_val_; + bool* value_found_; // Is value set correctly? Used by KeyMayExist + MergeContext* merge_context_; + RangeDelAggregator* range_del_agg_; + Env* env_; + // If a key is found, seq_ will be set to the SequenceNumber of most recent + // write to the key or kMaxSequenceNumber if unknown + SequenceNumber* seq_; + std::string* replay_log_; + // Used to temporarily pin blocks when state_ == GetContext::kMerge + PinnedIteratorsManager* pinned_iters_mgr_; + bool sample_; + bool* is_blob_index_; +}; + +void replayGetContextLog(const Slice& replay_log, const Slice& user_key, + GetContext* get_context, + Cleanable* value_pinner = nullptr); + +} // namespace rocksdb diff --git a/table/index_builder.cc b/table/index_builder.cc new file mode 100644 index 00000000000..cdf20aee920 --- /dev/null +++ b/table/index_builder.cc @@ -0,0 +1,187 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "table/index_builder.h" +#include +#include + +#include +#include + +#include "rocksdb/comparator.h" +#include "rocksdb/flush_block_policy.h" +#include "table/format.h" +#include "table/partitioned_filter_block.h" + +// Without anonymous namespace here, we fail the warning -Wmissing-prototypes +namespace rocksdb { +// using namespace rocksdb; +// Create a index builder based on its type. +IndexBuilder* IndexBuilder::CreateIndexBuilder( + BlockBasedTableOptions::IndexType index_type, + const InternalKeyComparator* comparator, + const InternalKeySliceTransform* int_key_slice_transform, + const BlockBasedTableOptions& table_opt) { + switch (index_type) { + case BlockBasedTableOptions::kBinarySearch: { + return new ShortenedIndexBuilder(comparator, + table_opt.index_block_restart_interval); + } + case BlockBasedTableOptions::kHashSearch: { + return new HashIndexBuilder(comparator, int_key_slice_transform, + table_opt.index_block_restart_interval); + } + case BlockBasedTableOptions::kTwoLevelIndexSearch: { + return PartitionedIndexBuilder::CreateIndexBuilder(comparator, table_opt); + } + default: { + assert(!"Do not recognize the index type "); + return nullptr; + } + } + // impossible. + assert(false); + return nullptr; +} + +PartitionedIndexBuilder* PartitionedIndexBuilder::CreateIndexBuilder( + const InternalKeyComparator* comparator, + const BlockBasedTableOptions& table_opt) { + return new PartitionedIndexBuilder(comparator, table_opt); +} + +PartitionedIndexBuilder::PartitionedIndexBuilder( + const InternalKeyComparator* comparator, + const BlockBasedTableOptions& table_opt) + : IndexBuilder(comparator), + index_block_builder_(table_opt.index_block_restart_interval), + sub_index_builder_(nullptr), + table_opt_(table_opt) {} + +PartitionedIndexBuilder::~PartitionedIndexBuilder() { + delete sub_index_builder_; +} + +void PartitionedIndexBuilder::MakeNewSubIndexBuilder() { + assert(sub_index_builder_ == nullptr); + sub_index_builder_ = new ShortenedIndexBuilder( + comparator_, table_opt_.index_block_restart_interval); + flush_policy_.reset(FlushBlockBySizePolicyFactory::NewFlushBlockPolicy( + table_opt_.metadata_block_size, table_opt_.block_size_deviation, + sub_index_builder_->index_block_builder_)); + partition_cut_requested_ = false; +} + +void PartitionedIndexBuilder::RequestPartitionCut() { + partition_cut_requested_ = true; +} + +void PartitionedIndexBuilder::AddIndexEntry( + std::string* last_key_in_current_block, + const Slice* first_key_in_next_block, const BlockHandle& block_handle) { + // Note: to avoid two consecuitive flush in the same method call, we do not + // check flush policy when adding the last key + if (UNLIKELY(first_key_in_next_block == nullptr)) { // no more keys + if (sub_index_builder_ == nullptr) { + MakeNewSubIndexBuilder(); + } + sub_index_builder_->AddIndexEntry(last_key_in_current_block, + first_key_in_next_block, block_handle); + sub_index_last_key_ = std::string(*last_key_in_current_block); + entries_.push_back( + {sub_index_last_key_, + std::unique_ptr(sub_index_builder_)}); + sub_index_builder_ = nullptr; + cut_filter_block = true; + } else { + // apply flush policy only to non-empty sub_index_builder_ + if (sub_index_builder_ != nullptr) { + std::string handle_encoding; + block_handle.EncodeTo(&handle_encoding); + bool do_flush = + partition_cut_requested_ || + flush_policy_->Update(*last_key_in_current_block, handle_encoding); + if (do_flush) { + entries_.push_back( + {sub_index_last_key_, + std::unique_ptr(sub_index_builder_)}); + cut_filter_block = true; + sub_index_builder_ = nullptr; + } + } + if (sub_index_builder_ == nullptr) { + MakeNewSubIndexBuilder(); + } + sub_index_builder_->AddIndexEntry(last_key_in_current_block, + first_key_in_next_block, block_handle); + sub_index_last_key_ = std::string(*last_key_in_current_block); + } +} + +Status PartitionedIndexBuilder::Finish( + IndexBlocks* index_blocks, const BlockHandle& last_partition_block_handle) { + assert(!entries_.empty()); + // It must be set to null after last key is added + assert(sub_index_builder_ == nullptr); + if (finishing_indexes == true) { + Entry& last_entry = entries_.front(); + std::string handle_encoding; + last_partition_block_handle.EncodeTo(&handle_encoding); + index_block_builder_.Add(last_entry.key, handle_encoding); + entries_.pop_front(); + } + // If there is no sub_index left, then return the 2nd level index. + if (UNLIKELY(entries_.empty())) { + index_blocks->index_block_contents = index_block_builder_.Finish(); + return Status::OK(); + } else { + // Finish the next partition index in line and Incomplete() to indicate we + // expect more calls to Finish + Entry& entry = entries_.front(); + auto s = entry.value->Finish(index_blocks); + finishing_indexes = true; + return s.ok() ? Status::Incomplete() : s; + } +} + +// Estimate size excluding the top-level index +// It is assumed that this method is called before writing index partition +// starts +size_t PartitionedIndexBuilder::EstimatedSize() const { + size_t total = 0; + for (auto it = entries_.begin(); it != entries_.end(); ++it) { + total += it->value->EstimatedSize(); + } + total += + sub_index_builder_ == nullptr ? 0 : sub_index_builder_->EstimatedSize(); + return total; +} + +// Since when this method is called we do not know the index block offsets yet, +// the top-level index does not exist. Hence we estimate the block offsets and +// create a temporary top-level index. +size_t PartitionedIndexBuilder::EstimateTopLevelIndexSize( + uint64_t offset) const { + BlockBuilder tmp_builder( + table_opt_.index_block_restart_interval); // tmp top-level index builder + for (auto it = entries_.begin(); it != entries_.end(); ++it) { + std::string tmp_handle_encoding; + uint64_t size = it->value->EstimatedSize(); + BlockHandle tmp_block_handle(offset, size); + tmp_block_handle.EncodeTo(&tmp_handle_encoding); + tmp_builder.Add(it->key, tmp_handle_encoding); + offset += size; + } + return tmp_builder.CurrentSizeEstimate(); +} + +size_t PartitionedIndexBuilder::NumPartitions() const { + return entries_.size(); +} +} // namespace rocksdb diff --git a/table/index_builder.h b/table/index_builder.h new file mode 100644 index 00000000000..d591e0e533c --- /dev/null +++ b/table/index_builder.h @@ -0,0 +1,342 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#pragma once + +#include +#include + +#include +#include +#include + +#include "rocksdb/comparator.h" +#include "table/block_based_table_factory.h" +#include "table/block_builder.h" +#include "table/format.h" + +namespace rocksdb { +// The interface for building index. +// Instruction for adding a new concrete IndexBuilder: +// 1. Create a subclass instantiated from IndexBuilder. +// 2. Add a new entry associated with that subclass in TableOptions::IndexType. +// 3. Add a create function for the new subclass in CreateIndexBuilder. +// Note: we can devise more advanced design to simplify the process for adding +// new subclass, which will, on the other hand, increase the code complexity and +// catch unwanted attention from readers. Given that we won't add/change +// indexes frequently, it makes sense to just embrace a more straightforward +// design that just works. +class IndexBuilder { + public: + static IndexBuilder* CreateIndexBuilder( + BlockBasedTableOptions::IndexType index_type, + const rocksdb::InternalKeyComparator* comparator, + const InternalKeySliceTransform* int_key_slice_transform, + const BlockBasedTableOptions& table_opt); + + // Index builder will construct a set of blocks which contain: + // 1. One primary index block. + // 2. (Optional) a set of metablocks that contains the metadata of the + // primary index. + struct IndexBlocks { + Slice index_block_contents; + std::unordered_map meta_blocks; + }; + explicit IndexBuilder(const InternalKeyComparator* comparator) + : comparator_(comparator) {} + + virtual ~IndexBuilder() {} + + // Add a new index entry to index block. + // To allow further optimization, we provide `last_key_in_current_block` and + // `first_key_in_next_block`, based on which the specific implementation can + // determine the best index key to be used for the index block. + // @last_key_in_current_block: this parameter maybe overridden with the value + // "substitute key". + // @first_key_in_next_block: it will be nullptr if the entry being added is + // the last one in the table + // + // REQUIRES: Finish() has not yet been called. + virtual void AddIndexEntry(std::string* last_key_in_current_block, + const Slice* first_key_in_next_block, + const BlockHandle& block_handle) = 0; + + // This method will be called whenever a key is added. The subclasses may + // override OnKeyAdded() if they need to collect additional information. + virtual void OnKeyAdded(const Slice& key) {} + + // Inform the index builder that all entries has been written. Block builder + // may therefore perform any operation required for block finalization. + // + // REQUIRES: Finish() has not yet been called. + inline Status Finish(IndexBlocks* index_blocks) { + // Throw away the changes to last_partition_block_handle. It has no effect + // on the first call to Finish anyway. + BlockHandle last_partition_block_handle; + return Finish(index_blocks, last_partition_block_handle); + } + + // This override of Finish can be utilized to build the 2nd level index in + // PartitionIndexBuilder. + // + // index_blocks will be filled with the resulting index data. If the return + // value is Status::InComplete() then it means that the index is partitioned + // and the callee should keep calling Finish until Status::OK() is returned. + // In that case, last_partition_block_handle is pointer to the block written + // with the result of the last call to Finish. This can be utilized to build + // the second level index pointing to each block of partitioned indexes. The + // last call to Finish() that returns Status::OK() populates index_blocks with + // the 2nd level index content. + virtual Status Finish(IndexBlocks* index_blocks, + const BlockHandle& last_partition_block_handle) = 0; + + // Get the estimated size for index block. + virtual size_t EstimatedSize() const = 0; + + protected: + const InternalKeyComparator* comparator_; +}; + +// This index builder builds space-efficient index block. +// +// Optimizations: +// 1. Made block's `block_restart_interval` to be 1, which will avoid linear +// search when doing index lookup (can be disabled by setting +// index_block_restart_interval). +// 2. Shorten the key length for index block. Other than honestly using the +// last key in the data block as the index key, we instead find a shortest +// substitute key that serves the same function. +class ShortenedIndexBuilder : public IndexBuilder { + public: + explicit ShortenedIndexBuilder(const InternalKeyComparator* comparator, + int index_block_restart_interval) + : IndexBuilder(comparator), + index_block_builder_(index_block_restart_interval) {} + + virtual void AddIndexEntry(std::string* last_key_in_current_block, + const Slice* first_key_in_next_block, + const BlockHandle& block_handle) override { + if (first_key_in_next_block != nullptr) { + comparator_->FindShortestSeparator(last_key_in_current_block, + *first_key_in_next_block); + } else { + comparator_->FindShortSuccessor(last_key_in_current_block); + } + + std::string handle_encoding; + block_handle.EncodeTo(&handle_encoding); + index_block_builder_.Add(*last_key_in_current_block, handle_encoding); + } + + using IndexBuilder::Finish; + virtual Status Finish( + IndexBlocks* index_blocks, + const BlockHandle& last_partition_block_handle) override { + index_blocks->index_block_contents = index_block_builder_.Finish(); + return Status::OK(); + } + + virtual size_t EstimatedSize() const override { + return index_block_builder_.CurrentSizeEstimate(); + } + + friend class PartitionedIndexBuilder; + + private: + BlockBuilder index_block_builder_; +}; + +// HashIndexBuilder contains a binary-searchable primary index and the +// metadata for secondary hash index construction. +// The metadata for hash index consists two parts: +// - a metablock that compactly contains a sequence of prefixes. All prefixes +// are stored consectively without any metadata (like, prefix sizes) being +// stored, which is kept in the other metablock. +// - a metablock contains the metadata of the prefixes, including prefix size, +// restart index and number of block it spans. The format looks like: +// +// +-----------------+---------------------------+---------------------+ +// <=prefix 1 +// | length: 4 bytes | restart interval: 4 bytes | num-blocks: 4 bytes | +// +-----------------+---------------------------+---------------------+ +// <=prefix 2 +// | length: 4 bytes | restart interval: 4 bytes | num-blocks: 4 bytes | +// +-----------------+---------------------------+---------------------+ +// | | +// | .... | +// | | +// +-----------------+---------------------------+---------------------+ +// <=prefix n +// | length: 4 bytes | restart interval: 4 bytes | num-blocks: 4 bytes | +// +-----------------+---------------------------+---------------------+ +// +// The reason of separating these two metablocks is to enable the efficiently +// reuse the first metablock during hash index construction without unnecessary +// data copy or small heap allocations for prefixes. +class HashIndexBuilder : public IndexBuilder { + public: + explicit HashIndexBuilder(const InternalKeyComparator* comparator, + const SliceTransform* hash_key_extractor, + int index_block_restart_interval) + : IndexBuilder(comparator), + primary_index_builder_(comparator, index_block_restart_interval), + hash_key_extractor_(hash_key_extractor) {} + + virtual void AddIndexEntry(std::string* last_key_in_current_block, + const Slice* first_key_in_next_block, + const BlockHandle& block_handle) override { + ++current_restart_index_; + primary_index_builder_.AddIndexEntry(last_key_in_current_block, + first_key_in_next_block, block_handle); + } + + virtual void OnKeyAdded(const Slice& key) override { + auto key_prefix = hash_key_extractor_->Transform(key); + bool is_first_entry = pending_block_num_ == 0; + + // Keys may share the prefix + if (is_first_entry || pending_entry_prefix_ != key_prefix) { + if (!is_first_entry) { + FlushPendingPrefix(); + } + + // need a hard copy otherwise the underlying data changes all the time. + // TODO(kailiu) ToString() is expensive. We may speed up can avoid data + // copy. + pending_entry_prefix_ = key_prefix.ToString(); + pending_block_num_ = 1; + pending_entry_index_ = static_cast(current_restart_index_); + } else { + // entry number increments when keys share the prefix reside in + // different data blocks. + auto last_restart_index = pending_entry_index_ + pending_block_num_ - 1; + assert(last_restart_index <= current_restart_index_); + if (last_restart_index != current_restart_index_) { + ++pending_block_num_; + } + } + } + + virtual Status Finish( + IndexBlocks* index_blocks, + const BlockHandle& last_partition_block_handle) override { + FlushPendingPrefix(); + primary_index_builder_.Finish(index_blocks, last_partition_block_handle); + index_blocks->meta_blocks.insert( + {kHashIndexPrefixesBlock.c_str(), prefix_block_}); + index_blocks->meta_blocks.insert( + {kHashIndexPrefixesMetadataBlock.c_str(), prefix_meta_block_}); + return Status::OK(); + } + + virtual size_t EstimatedSize() const override { + return primary_index_builder_.EstimatedSize() + prefix_block_.size() + + prefix_meta_block_.size(); + } + + private: + void FlushPendingPrefix() { + prefix_block_.append(pending_entry_prefix_.data(), + pending_entry_prefix_.size()); + PutVarint32Varint32Varint32( + &prefix_meta_block_, + static_cast(pending_entry_prefix_.size()), + pending_entry_index_, pending_block_num_); + } + + ShortenedIndexBuilder primary_index_builder_; + const SliceTransform* hash_key_extractor_; + + // stores a sequence of prefixes + std::string prefix_block_; + // stores the metadata of prefixes + std::string prefix_meta_block_; + + // The following 3 variables keeps unflushed prefix and its metadata. + // The details of block_num and entry_index can be found in + // "block_hash_index.{h,cc}" + uint32_t pending_block_num_ = 0; + uint32_t pending_entry_index_ = 0; + std::string pending_entry_prefix_; + + uint64_t current_restart_index_ = 0; +}; + +/** + * IndexBuilder for two-level indexing. Internally it creates a new index for + * each partition and Finish then in order when Finish is called on it + * continiously until Status::OK() is returned. + * + * The format on the disk would be I I I I I I IP where I is block containing a + * partition of indexes built using ShortenedIndexBuilder and IP is a block + * containing a secondary index on the partitions, built using + * ShortenedIndexBuilder. + */ +class PartitionedIndexBuilder : public IndexBuilder { + public: + static PartitionedIndexBuilder* CreateIndexBuilder( + const rocksdb::InternalKeyComparator* comparator, + const BlockBasedTableOptions& table_opt); + + explicit PartitionedIndexBuilder(const InternalKeyComparator* comparator, + const BlockBasedTableOptions& table_opt); + + virtual ~PartitionedIndexBuilder(); + + virtual void AddIndexEntry(std::string* last_key_in_current_block, + const Slice* first_key_in_next_block, + const BlockHandle& block_handle) override; + + virtual Status Finish( + IndexBlocks* index_blocks, + const BlockHandle& last_partition_block_handle) override; + + virtual size_t EstimatedSize() const override; + size_t EstimateTopLevelIndexSize(uint64_t) const; + size_t NumPartitions() const; + + inline bool ShouldCutFilterBlock() { + // Current policy is to align the partitions of index and filters + if (cut_filter_block) { + cut_filter_block = false; + return true; + } + return false; + } + + std::string& GetPartitionKey() { return sub_index_last_key_; } + + // Called when an external entity (such as filter partition builder) request + // cutting the next partition + void RequestPartitionCut(); + + private: + void MakeNewSubIndexBuilder(); + + struct Entry { + std::string key; + std::unique_ptr value; + }; + std::list entries_; // list of partitioned indexes and their keys + BlockBuilder index_block_builder_; // top-level index builder + // the active partition index builder + ShortenedIndexBuilder* sub_index_builder_; + // the last key in the active partition index builder + std::string sub_index_last_key_; + std::unique_ptr flush_policy_; + // true if Finish is called once but not complete yet. + bool finishing_indexes = false; + const BlockBasedTableOptions& table_opt_; + // true if an external entity (such as filter partition builder) request + // cutting the next partition + bool partition_cut_requested_ = true; + // true if it should cut the next filter partition block + bool cut_filter_block = false; +}; +} // namespace rocksdb diff --git a/table/internal_iterator.h b/table/internal_iterator.h new file mode 100644 index 00000000000..2bfdb7d952a --- /dev/null +++ b/table/internal_iterator.h @@ -0,0 +1,121 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// + +#pragma once + +#include +#include "rocksdb/comparator.h" +#include "rocksdb/iterator.h" +#include "rocksdb/status.h" + +namespace rocksdb { + +class PinnedIteratorsManager; + +class InternalIterator : public Cleanable { + public: + InternalIterator() {} + virtual ~InternalIterator() {} + + // An iterator is either positioned at a key/value pair, or + // not valid. This method returns true iff the iterator is valid. + virtual bool Valid() const = 0; + + // Position at the first key in the source. The iterator is Valid() + // after this call iff the source is not empty. + virtual void SeekToFirst() = 0; + + // Position at the last key in the source. The iterator is + // Valid() after this call iff the source is not empty. + virtual void SeekToLast() = 0; + + // Position at the first key in the source that at or past target + // The iterator is Valid() after this call iff the source contains + // an entry that comes at or past target. + virtual void Seek(const Slice& target) = 0; + + // Position at the first key in the source that at or before target + // The iterator is Valid() after this call iff the source contains + // an entry that comes at or before target. + virtual void SeekForPrev(const Slice& target) = 0; + + // Moves to the next entry in the source. After this call, Valid() is + // true iff the iterator was not positioned at the last entry in the source. + // REQUIRES: Valid() + virtual void Next() = 0; + + // Moves to the previous entry in the source. After this call, Valid() is + // true iff the iterator was not positioned at the first entry in source. + // REQUIRES: Valid() + virtual void Prev() = 0; + + // Return the key for the current entry. The underlying storage for + // the returned slice is valid only until the next modification of + // the iterator. + // REQUIRES: Valid() + virtual Slice key() const = 0; + + // Return the value for the current entry. The underlying storage for + // the returned slice is valid only until the next modification of + // the iterator. + // REQUIRES: !AtEnd() && !AtStart() + virtual Slice value() const = 0; + + // If an error has occurred, return it. Else return an ok status. + // If non-blocking IO is requested and this operation cannot be + // satisfied without doing some IO, then this returns Status::Incomplete(). + virtual Status status() const = 0; + + // Pass the PinnedIteratorsManager to the Iterator, most Iterators dont + // communicate with PinnedIteratorsManager so default implementation is no-op + // but for Iterators that need to communicate with PinnedIteratorsManager + // they will implement this function and use the passed pointer to communicate + // with PinnedIteratorsManager. + virtual void SetPinnedItersMgr(PinnedIteratorsManager* pinned_iters_mgr) {} + + // If true, this means that the Slice returned by key() is valid as long as + // PinnedIteratorsManager::ReleasePinnedData is not called and the + // Iterator is not deleted. + // + // IsKeyPinned() is guaranteed to always return true if + // - Iterator is created with ReadOptions::pin_data = true + // - DB tables were created with BlockBasedTableOptions::use_delta_encoding + // set to false. + virtual bool IsKeyPinned() const { return false; } + + // If true, this means that the Slice returned by value() is valid as long as + // PinnedIteratorsManager::ReleasePinnedData is not called and the + // Iterator is not deleted. + virtual bool IsValuePinned() const { return false; } + + virtual Status GetProperty(std::string prop_name, std::string* prop) { + return Status::NotSupported(""); + } + + protected: + void SeekForPrevImpl(const Slice& target, const Comparator* cmp) { + Seek(target); + if (!Valid()) { + SeekToLast(); + } + while (Valid() && cmp->Compare(target, key()) < 0) { + Prev(); + } + } + + private: + // No copying allowed + InternalIterator(const InternalIterator&) = delete; + InternalIterator& operator=(const InternalIterator&) = delete; +}; + +// Return an empty iterator (yields nothing). +extern InternalIterator* NewEmptyInternalIterator(); + +// Return an empty iterator with the specified status. +extern InternalIterator* NewErrorInternalIterator(const Status& status); + +} // namespace rocksdb diff --git a/table/iter_heap.h b/table/iter_heap.h index 9569d363896..74c06caeaf8 100644 --- a/table/iter_heap.h +++ b/table/iter_heap.h @@ -1,40 +1,38 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // #pragma once -#include #include "rocksdb/comparator.h" #include "table/iterator_wrapper.h" namespace rocksdb { -// Return the max of two keys. +// When used with std::priority_queue, this comparison functor puts the +// iterator with the max/largest key on top. class MaxIteratorComparator { public: MaxIteratorComparator(const Comparator* comparator) : comparator_(comparator) {} - bool operator()(IteratorWrapper* a, IteratorWrapper* b) { - return comparator_->Compare(a->key(), b->key()) <= 0; + bool operator()(IteratorWrapper* a, IteratorWrapper* b) const { + return comparator_->Compare(a->key(), b->key()) < 0; } private: const Comparator* comparator_; }; -// Return the max of two keys. +// When used with std::priority_queue, this comparison functor puts the +// iterator with the min/smallest key on top. class MinIteratorComparator { public: - // if maxHeap is set comparator returns the max value. - // else returns the min Value. - // Can use to create a minHeap or a maxHeap. MinIteratorComparator(const Comparator* comparator) : comparator_(comparator) {} - bool operator()(IteratorWrapper* a, IteratorWrapper* b) { + bool operator()(IteratorWrapper* a, IteratorWrapper* b) const { return comparator_->Compare(a->key(), b->key()) > 0; } private: diff --git a/table/iterator.cc b/table/iterator.cc index 4c360205a21..ed6a2cdea44 100644 --- a/table/iterator.cc +++ b/table/iterator.cc @@ -1,36 +1,86 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #include "rocksdb/iterator.h" +#include "table/internal_iterator.h" #include "table/iterator_wrapper.h" #include "util/arena.h" namespace rocksdb { -Iterator::Iterator() { +Cleanable::Cleanable() { cleanup_.function = nullptr; cleanup_.next = nullptr; } -Iterator::~Iterator() { - if (cleanup_.function != nullptr) { - (*cleanup_.function)(cleanup_.arg1, cleanup_.arg2); - for (Cleanup* c = cleanup_.next; c != nullptr; ) { - (*c->function)(c->arg1, c->arg2); - Cleanup* next = c->next; - delete c; - c = next; - } +Cleanable::~Cleanable() { DoCleanup(); } + +Cleanable::Cleanable(Cleanable&& other) { + *this = std::move(other); +} + +Cleanable& Cleanable::operator=(Cleanable&& other) { + if (this != &other) { + cleanup_ = other.cleanup_; + other.cleanup_.function = nullptr; + other.cleanup_.next = nullptr; + } + return *this; +} + +// If the entire linked list was on heap we could have simply add attach one +// link list to another. However the head is an embeded object to avoid the cost +// of creating objects for most of the use cases when the Cleanable has only one +// Cleanup to do. We could put evernything on heap if benchmarks show no +// negative impact on performance. +// Also we need to iterate on the linked list since there is no pointer to the +// tail. We can add the tail pointer but maintainin it might negatively impact +// the perforamnce for the common case of one cleanup where tail pointer is not +// needed. Again benchmarks could clarify that. +// Even without a tail pointer we could iterate on the list, find the tail, and +// have only that node updated without the need to insert the Cleanups one by +// one. This however would be redundant when the source Cleanable has one or a +// few Cleanups which is the case most of the time. +// TODO(myabandeh): if the list is too long we should maintain a tail pointer +// and have the entire list (minus the head that has to be inserted separately) +// merged with the target linked list at once. +void Cleanable::DelegateCleanupsTo(Cleanable* other) { + assert(other != nullptr); + if (cleanup_.function == nullptr) { + return; + } + Cleanup* c = &cleanup_; + other->RegisterCleanup(c->function, c->arg1, c->arg2); + c = c->next; + while (c != nullptr) { + Cleanup* next = c->next; + other->RegisterCleanup(c); + c = next; + } + cleanup_.function = nullptr; + cleanup_.next = nullptr; +} + +void Cleanable::RegisterCleanup(Cleanable::Cleanup* c) { + assert(c != nullptr); + if (cleanup_.function == nullptr) { + cleanup_.function = c->function; + cleanup_.arg1 = c->arg1; + cleanup_.arg2 = c->arg2; + delete c; + } else { + c->next = cleanup_.next; + cleanup_.next = c; } } -void Iterator::RegisterCleanup(CleanupFunction func, void* arg1, void* arg2) { +void Cleanable::RegisterCleanup(CleanupFunction func, void* arg1, void* arg2) { assert(func != nullptr); Cleanup* c; if (cleanup_.function == nullptr) { @@ -45,19 +95,62 @@ void Iterator::RegisterCleanup(CleanupFunction func, void* arg1, void* arg2) { c->arg2 = arg2; } +Status Iterator::GetProperty(std::string prop_name, std::string* prop) { + if (prop == nullptr) { + return Status::InvalidArgument("prop is nullptr"); + } + if (prop_name == "rocksdb.iterator.is-key-pinned") { + *prop = "0"; + return Status::OK(); + } + return Status::InvalidArgument("Undentified property."); +} + namespace { class EmptyIterator : public Iterator { public: explicit EmptyIterator(const Status& s) : status_(s) { } - virtual bool Valid() const { return false; } - virtual void Seek(const Slice& target) { } - virtual void SeekToFirst() { } - virtual void SeekToLast() { } - virtual void Next() { assert(false); } - virtual void Prev() { assert(false); } - Slice key() const { assert(false); return Slice(); } - Slice value() const { assert(false); return Slice(); } - virtual Status status() const { return status_; } + virtual bool Valid() const override { return false; } + virtual void Seek(const Slice& target) override {} + virtual void SeekForPrev(const Slice& target) override {} + virtual void SeekToFirst() override {} + virtual void SeekToLast() override {} + virtual void Next() override { assert(false); } + virtual void Prev() override { assert(false); } + Slice key() const override { + assert(false); + return Slice(); + } + Slice value() const override { + assert(false); + return Slice(); + } + virtual Status status() const override { return status_; } + + private: + Status status_; +}; + +class EmptyInternalIterator : public InternalIterator { + public: + explicit EmptyInternalIterator(const Status& s) : status_(s) {} + virtual bool Valid() const override { return false; } + virtual void Seek(const Slice& target) override {} + virtual void SeekForPrev(const Slice& target) override {} + virtual void SeekToFirst() override {} + virtual void SeekToLast() override {} + virtual void Next() override { assert(false); } + virtual void Prev() override { assert(false); } + Slice key() const override { + assert(false); + return Slice(); + } + Slice value() const override { + assert(false); + return Slice(); + } + virtual Status status() const override { return status_; } + private: Status status_; }; @@ -67,25 +160,33 @@ Iterator* NewEmptyIterator() { return new EmptyIterator(Status::OK()); } -Iterator* NewEmptyIterator(Arena* arena) { +Iterator* NewErrorIterator(const Status& status) { + return new EmptyIterator(status); +} + +InternalIterator* NewEmptyInternalIterator() { + return new EmptyInternalIterator(Status::OK()); +} + +InternalIterator* NewEmptyInternalIterator(Arena* arena) { if (arena == nullptr) { - return NewEmptyIterator(); + return NewEmptyInternalIterator(); } else { auto mem = arena->AllocateAligned(sizeof(EmptyIterator)); - return new (mem) EmptyIterator(Status::OK()); + return new (mem) EmptyInternalIterator(Status::OK()); } } -Iterator* NewErrorIterator(const Status& status) { - return new EmptyIterator(status); +InternalIterator* NewErrorInternalIterator(const Status& status) { + return new EmptyInternalIterator(status); } -Iterator* NewErrorIterator(const Status& status, Arena* arena) { +InternalIterator* NewErrorInternalIterator(const Status& status, Arena* arena) { if (arena == nullptr) { - return NewErrorIterator(status); + return NewErrorInternalIterator(status); } else { auto mem = arena->AllocateAligned(sizeof(EmptyIterator)); - return new (mem) EmptyIterator(status); + return new (mem) EmptyInternalIterator(status); } } diff --git a/table/iterator_wrapper.h b/table/iterator_wrapper.h index 502cacb3e85..f14acdb9bf6 100644 --- a/table/iterator_wrapper.h +++ b/table/iterator_wrapper.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,40 +9,46 @@ #pragma once -#include "rocksdb/iterator.h" +#include + +#include "table/internal_iterator.h" namespace rocksdb { -// A internal wrapper class with an interface similar to Iterator that -// caches the valid() and key() results for an underlying iterator. +// A internal wrapper class with an interface similar to Iterator that caches +// the valid() and key() results for an underlying iterator. // This can help avoid virtual function calls and also gives better // cache locality. class IteratorWrapper { public: - IteratorWrapper(): iter_(nullptr), valid_(false) { } - explicit IteratorWrapper(Iterator* iter): iter_(nullptr) { - Set(iter); + IteratorWrapper() : iter_(nullptr), valid_(false) {} + explicit IteratorWrapper(InternalIterator* _iter) : iter_(nullptr) { + Set(_iter); } ~IteratorWrapper() {} - Iterator* iter() const { return iter_; } + InternalIterator* iter() const { return iter_; } + + // Set the underlying Iterator to _iter and return + // previous underlying Iterator. + InternalIterator* Set(InternalIterator* _iter) { + InternalIterator* old_iter = iter_; - // Takes ownership of "iter" and will delete it when destroyed, or - // when Set() is invoked again. - void Set(Iterator* iter) { - delete iter_; - iter_ = iter; + iter_ = _iter; if (iter_ == nullptr) { valid_ = false; } else { Update(); } + return old_iter; } void DeleteIter(bool is_arena_mode) { - if (!is_arena_mode) { - delete iter_; - } else { - iter_->~Iterator(); + if (iter_) { + if (!is_arena_mode) { + delete iter_; + } else { + iter_->~InternalIterator(); + } } } @@ -55,9 +61,27 @@ class IteratorWrapper { void Next() { assert(iter_); iter_->Next(); Update(); } void Prev() { assert(iter_); iter_->Prev(); Update(); } void Seek(const Slice& k) { assert(iter_); iter_->Seek(k); Update(); } + void SeekForPrev(const Slice& k) { + assert(iter_); + iter_->SeekForPrev(k); + Update(); + } void SeekToFirst() { assert(iter_); iter_->SeekToFirst(); Update(); } void SeekToLast() { assert(iter_); iter_->SeekToLast(); Update(); } + void SetPinnedItersMgr(PinnedIteratorsManager* pinned_iters_mgr) { + assert(iter_); + iter_->SetPinnedItersMgr(pinned_iters_mgr); + } + bool IsKeyPinned() const { + assert(Valid()); + return iter_->IsKeyPinned(); + } + bool IsValuePinned() const { + assert(Valid()); + return iter_->IsValuePinned(); + } + private: void Update() { valid_ = iter_->Valid(); @@ -66,16 +90,17 @@ class IteratorWrapper { } } - Iterator* iter_; + InternalIterator* iter_; bool valid_; Slice key_; }; class Arena; // Return an empty iterator (yields nothing) allocated from arena. -extern Iterator* NewEmptyIterator(Arena* arena); +extern InternalIterator* NewEmptyInternalIterator(Arena* arena); // Return an empty iterator with the specified status, allocated arena. -extern Iterator* NewErrorIterator(const Status& status, Arena* arena); +extern InternalIterator* NewErrorInternalIterator(const Status& status, + Arena* arena); } // namespace rocksdb diff --git a/table/merger.cc b/table/merger.cc deleted file mode 100644 index a53376cebce..00000000000 --- a/table/merger.cc +++ /dev/null @@ -1,353 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. -// -// Copyright (c) 2011 The LevelDB Authors. All rights reserved. -// Use of this source code is governed by a BSD-style license that can be -// found in the LICENSE file. See the AUTHORS file for names of contributors. - -#include "table/merger.h" - -#include -#include - -#include "rocksdb/comparator.h" -#include "rocksdb/iterator.h" -#include "rocksdb/options.h" -#include "table/iter_heap.h" -#include "table/iterator_wrapper.h" -#include "util/arena.h" -#include "util/stop_watch.h" -#include "util/perf_context_imp.h" -#include "util/autovector.h" - -namespace rocksdb { -namespace merger { -typedef std::priority_queue< - IteratorWrapper*, - std::vector, - MaxIteratorComparator> MaxIterHeap; - -typedef std::priority_queue< - IteratorWrapper*, - std::vector, - MinIteratorComparator> MinIterHeap; - -// Return's a new MaxHeap of IteratorWrapper's using the provided Comparator. -MaxIterHeap NewMaxIterHeap(const Comparator* comparator) { - return MaxIterHeap(MaxIteratorComparator(comparator)); -} - -// Return's a new MinHeap of IteratorWrapper's using the provided Comparator. -MinIterHeap NewMinIterHeap(const Comparator* comparator) { - return MinIterHeap(MinIteratorComparator(comparator)); -} -} // namespace merger - -const size_t kNumIterReserve = 4; - -class MergingIterator : public Iterator { - public: - MergingIterator(const Comparator* comparator, Iterator** children, int n, - bool is_arena_mode) - : is_arena_mode_(is_arena_mode), - comparator_(comparator), - current_(nullptr), - use_heap_(true), - direction_(kForward), - maxHeap_(merger::NewMaxIterHeap(comparator_)), - minHeap_(merger::NewMinIterHeap(comparator_)) { - children_.resize(n); - for (int i = 0; i < n; i++) { - children_[i].Set(children[i]); - } - for (auto& child : children_) { - if (child.Valid()) { - minHeap_.push(&child); - } - } - } - - virtual void AddIterator(Iterator* iter) { - assert(direction_ == kForward); - children_.emplace_back(iter); - auto new_wrapper = children_.back(); - if (new_wrapper.Valid()) { - minHeap_.push(&new_wrapper); - } - } - - virtual ~MergingIterator() { - for (auto& child : children_) { - child.DeleteIter(is_arena_mode_); - } - } - - virtual bool Valid() const { - return (current_ != nullptr); - } - - virtual void SeekToFirst() { - ClearHeaps(); - for (auto& child : children_) { - child.SeekToFirst(); - if (child.Valid()) { - minHeap_.push(&child); - } - } - FindSmallest(); - direction_ = kForward; - } - - virtual void SeekToLast() { - ClearHeaps(); - for (auto& child : children_) { - child.SeekToLast(); - if (child.Valid()) { - maxHeap_.push(&child); - } - } - FindLargest(); - direction_ = kReverse; - } - - virtual void Seek(const Slice& target) { - // Invalidate the heap. - use_heap_ = false; - IteratorWrapper* first_child = nullptr; - - for (auto& child : children_) { - { - PERF_TIMER_GUARD(seek_child_seek_time); - child.Seek(target); - } - PERF_COUNTER_ADD(seek_child_seek_count, 1); - - if (child.Valid()) { - // This child has valid key - if (!use_heap_) { - if (first_child == nullptr) { - // It's the first child has valid key. Only put it int - // current_. Now the values in the heap should be invalid. - first_child = &child; - } else { - // We have more than one children with valid keys. Initialize - // the heap and put the first child into the heap. - PERF_TIMER_GUARD(seek_min_heap_time); - ClearHeaps(); - minHeap_.push(first_child); - } - } - if (use_heap_) { - PERF_TIMER_GUARD(seek_min_heap_time); - minHeap_.push(&child); - } - } - } - if (use_heap_) { - // If heap is valid, need to put the smallest key to curent_. - PERF_TIMER_GUARD(seek_min_heap_time); - FindSmallest(); - } else { - // The heap is not valid, then the current_ iterator is the first - // one, or null if there is no first child. - current_ = first_child; - } - direction_ = kForward; - } - - virtual void Next() { - assert(Valid()); - - // Ensure that all children are positioned after key(). - // If we are moving in the forward direction, it is already - // true for all of the non-current_ children since current_ is - // the smallest child and key() == current_->key(). Otherwise, - // we explicitly position the non-current_ children. - if (direction_ != kForward) { - ClearHeaps(); - for (auto& child : children_) { - if (&child != current_) { - child.Seek(key()); - if (child.Valid() && - comparator_->Compare(key(), child.key()) == 0) { - child.Next(); - } - if (child.Valid()) { - minHeap_.push(&child); - } - } - } - direction_ = kForward; - } - - // as the current points to the current record. move the iterator forward. - // and if it is valid add it to the heap. - current_->Next(); - if (use_heap_) { - if (current_->Valid()) { - minHeap_.push(current_); - } - FindSmallest(); - } else if (!current_->Valid()) { - current_ = nullptr; - } - } - - virtual void Prev() { - assert(Valid()); - // Ensure that all children are positioned before key(). - // If we are moving in the reverse direction, it is already - // true for all of the non-current_ children since current_ is - // the largest child and key() == current_->key(). Otherwise, - // we explicitly position the non-current_ children. - if (direction_ != kReverse) { - ClearHeaps(); - for (auto& child : children_) { - if (&child != current_) { - child.Seek(key()); - if (child.Valid()) { - // Child is at first entry >= key(). Step back one to be < key() - child.Prev(); - } else { - // Child has no entries >= key(). Position at last entry. - child.SeekToLast(); - } - if (child.Valid()) { - maxHeap_.push(&child); - } - } - } - direction_ = kReverse; - } - - current_->Prev(); - if (current_->Valid()) { - maxHeap_.push(current_); - } - FindLargest(); - } - - virtual Slice key() const { - assert(Valid()); - return current_->key(); - } - - virtual Slice value() const { - assert(Valid()); - return current_->value(); - } - - virtual Status status() const { - Status status; - for (auto& child : children_) { - status = child.status(); - if (!status.ok()) { - break; - } - } - return status; - } - - private: - void FindSmallest(); - void FindLargest(); - void ClearHeaps(); - - bool is_arena_mode_; - const Comparator* comparator_; - autovector children_; - IteratorWrapper* current_; - // If the value is true, both of iterators in the heap and current_ - // contain valid rows. If it is false, only current_ can possibly contain - // valid rows. - // This flag is always true for reverse direction, as we always use heap for - // the reverse iterating case. - bool use_heap_; - // Which direction is the iterator moving? - enum Direction { - kForward, - kReverse - }; - Direction direction_; - merger::MaxIterHeap maxHeap_; - merger::MinIterHeap minHeap_; -}; - -void MergingIterator::FindSmallest() { - assert(use_heap_); - if (minHeap_.empty()) { - current_ = nullptr; - } else { - current_ = minHeap_.top(); - assert(current_->Valid()); - minHeap_.pop(); - } -} - -void MergingIterator::FindLargest() { - assert(use_heap_); - if (maxHeap_.empty()) { - current_ = nullptr; - } else { - current_ = maxHeap_.top(); - assert(current_->Valid()); - maxHeap_.pop(); - } -} - -void MergingIterator::ClearHeaps() { - use_heap_ = true; - maxHeap_ = merger::NewMaxIterHeap(comparator_); - minHeap_ = merger::NewMinIterHeap(comparator_); -} - -Iterator* NewMergingIterator(const Comparator* cmp, Iterator** list, int n, - Arena* arena) { - assert(n >= 0); - if (n == 0) { - return NewEmptyIterator(arena); - } else if (n == 1) { - return list[0]; - } else { - if (arena == nullptr) { - return new MergingIterator(cmp, list, n, false); - } else { - auto mem = arena->AllocateAligned(sizeof(MergingIterator)); - return new (mem) MergingIterator(cmp, list, n, true); - } - } -} - -MergeIteratorBuilder::MergeIteratorBuilder(const Comparator* comparator, - Arena* a) - : first_iter(nullptr), use_merging_iter(false), arena(a) { - - auto mem = arena->AllocateAligned(sizeof(MergingIterator)); - merge_iter = new (mem) MergingIterator(comparator, nullptr, 0, true); -} - -void MergeIteratorBuilder::AddIterator(Iterator* iter) { - if (!use_merging_iter && first_iter != nullptr) { - merge_iter->AddIterator(first_iter); - use_merging_iter = true; - } - if (use_merging_iter) { - merge_iter->AddIterator(iter); - } else { - first_iter = iter; - } -} - -Iterator* MergeIteratorBuilder::Finish() { - if (!use_merging_iter) { - return first_iter; - } else { - auto ret = merge_iter; - merge_iter = nullptr; - return ret; - } -} - -} // namespace rocksdb diff --git a/table/merger_test.cc b/table/merger_test.cc new file mode 100644 index 00000000000..379a6f412d5 --- /dev/null +++ b/table/merger_test.cc @@ -0,0 +1,169 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include +#include + +#include "table/merging_iterator.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +class MergerTest : public testing::Test { + public: + MergerTest() + : rnd_(3), merging_iterator_(nullptr), single_iterator_(nullptr) {} + ~MergerTest() = default; + std::vector GenerateStrings(size_t len, int string_len) { + std::vector ret; + for (size_t i = 0; i < len; ++i) { + ret.push_back(test::RandomHumanReadableString(&rnd_, string_len)); + } + return ret; + } + + void AssertEquivalence() { + auto a = merging_iterator_.get(); + auto b = single_iterator_.get(); + if (!a->Valid()) { + ASSERT_TRUE(!b->Valid()); + } else { + ASSERT_TRUE(b->Valid()); + ASSERT_EQ(b->key().ToString(), a->key().ToString()); + ASSERT_EQ(b->value().ToString(), a->value().ToString()); + } + } + + void SeekToRandom() { Seek(test::RandomHumanReadableString(&rnd_, 5)); } + + void Seek(std::string target) { + merging_iterator_->Seek(target); + single_iterator_->Seek(target); + } + + void SeekToFirst() { + merging_iterator_->SeekToFirst(); + single_iterator_->SeekToFirst(); + } + + void SeekToLast() { + merging_iterator_->SeekToLast(); + single_iterator_->SeekToLast(); + } + + void Next(int times) { + for (int i = 0; i < times && merging_iterator_->Valid(); ++i) { + AssertEquivalence(); + merging_iterator_->Next(); + single_iterator_->Next(); + } + AssertEquivalence(); + } + + void Prev(int times) { + for (int i = 0; i < times && merging_iterator_->Valid(); ++i) { + AssertEquivalence(); + merging_iterator_->Prev(); + single_iterator_->Prev(); + } + AssertEquivalence(); + } + + void NextAndPrev(int times) { + for (int i = 0; i < times && merging_iterator_->Valid(); ++i) { + AssertEquivalence(); + if (rnd_.OneIn(2)) { + merging_iterator_->Prev(); + single_iterator_->Prev(); + } else { + merging_iterator_->Next(); + single_iterator_->Next(); + } + } + AssertEquivalence(); + } + + void Generate(size_t num_iterators, size_t strings_per_iterator, + int letters_per_string) { + std::vector small_iterators; + for (size_t i = 0; i < num_iterators; ++i) { + auto strings = GenerateStrings(strings_per_iterator, letters_per_string); + small_iterators.push_back(new test::VectorIterator(strings)); + all_keys_.insert(all_keys_.end(), strings.begin(), strings.end()); + } + + merging_iterator_.reset( + NewMergingIterator(BytewiseComparator(), &small_iterators[0], + static_cast(small_iterators.size()))); + single_iterator_.reset(new test::VectorIterator(all_keys_)); + } + + Random rnd_; + std::unique_ptr merging_iterator_; + std::unique_ptr single_iterator_; + std::vector all_keys_; +}; + +TEST_F(MergerTest, SeekToRandomNextTest) { + Generate(1000, 50, 50); + for (int i = 0; i < 10; ++i) { + SeekToRandom(); + AssertEquivalence(); + Next(50000); + } +} + +TEST_F(MergerTest, SeekToRandomNextSmallStringsTest) { + Generate(1000, 50, 2); + for (int i = 0; i < 10; ++i) { + SeekToRandom(); + AssertEquivalence(); + Next(50000); + } +} + +TEST_F(MergerTest, SeekToRandomPrevTest) { + Generate(1000, 50, 50); + for (int i = 0; i < 10; ++i) { + SeekToRandom(); + AssertEquivalence(); + Prev(50000); + } +} + +TEST_F(MergerTest, SeekToRandomRandomTest) { + Generate(200, 50, 50); + for (int i = 0; i < 3; ++i) { + SeekToRandom(); + AssertEquivalence(); + NextAndPrev(5000); + } +} + +TEST_F(MergerTest, SeekToFirstTest) { + Generate(1000, 50, 50); + for (int i = 0; i < 10; ++i) { + SeekToFirst(); + AssertEquivalence(); + Next(50000); + } +} + +TEST_F(MergerTest, SeekToLastTest) { + Generate(1000, 50, 50); + for (int i = 0; i < 10; ++i) { + SeekToLast(); + AssertEquivalence(); + Prev(50000); + } +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/table/merging_iterator.cc b/table/merging_iterator.cc new file mode 100644 index 00000000000..da30e1e6352 --- /dev/null +++ b/table/merging_iterator.cc @@ -0,0 +1,404 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. + +#include "table/merging_iterator.h" +#include +#include +#include "db/pinned_iterators_manager.h" +#include "monitoring/perf_context_imp.h" +#include "rocksdb/comparator.h" +#include "rocksdb/iterator.h" +#include "rocksdb/options.h" +#include "table/internal_iterator.h" +#include "table/iter_heap.h" +#include "table/iterator_wrapper.h" +#include "util/arena.h" +#include "util/autovector.h" +#include "util/heap.h" +#include "util/stop_watch.h" +#include "util/sync_point.h" + +namespace rocksdb { +// Without anonymous namespace here, we fail the warning -Wmissing-prototypes +namespace { +typedef BinaryHeap MergerMaxIterHeap; +typedef BinaryHeap MergerMinIterHeap; +} // namespace + +const size_t kNumIterReserve = 4; + +class MergingIterator : public InternalIterator { + public: + MergingIterator(const Comparator* comparator, InternalIterator** children, + int n, bool is_arena_mode, bool prefix_seek_mode) + : is_arena_mode_(is_arena_mode), + comparator_(comparator), + current_(nullptr), + direction_(kForward), + minHeap_(comparator_), + prefix_seek_mode_(prefix_seek_mode), + pinned_iters_mgr_(nullptr) { + children_.resize(n); + for (int i = 0; i < n; i++) { + children_[i].Set(children[i]); + } + for (auto& child : children_) { + if (child.Valid()) { + minHeap_.push(&child); + } + } + current_ = CurrentForward(); + } + + virtual void AddIterator(InternalIterator* iter) { + assert(direction_ == kForward); + children_.emplace_back(iter); + if (pinned_iters_mgr_) { + iter->SetPinnedItersMgr(pinned_iters_mgr_); + } + auto new_wrapper = children_.back(); + if (new_wrapper.Valid()) { + minHeap_.push(&new_wrapper); + current_ = CurrentForward(); + } + } + + virtual ~MergingIterator() { + for (auto& child : children_) { + child.DeleteIter(is_arena_mode_); + } + } + + virtual bool Valid() const override { return (current_ != nullptr); } + + virtual void SeekToFirst() override { + ClearHeaps(); + for (auto& child : children_) { + child.SeekToFirst(); + if (child.Valid()) { + minHeap_.push(&child); + } + } + direction_ = kForward; + current_ = CurrentForward(); + } + + virtual void SeekToLast() override { + ClearHeaps(); + InitMaxHeap(); + for (auto& child : children_) { + child.SeekToLast(); + if (child.Valid()) { + maxHeap_->push(&child); + } + } + direction_ = kReverse; + current_ = CurrentReverse(); + } + + virtual void Seek(const Slice& target) override { + ClearHeaps(); + for (auto& child : children_) { + { + PERF_TIMER_GUARD(seek_child_seek_time); + child.Seek(target); + } + PERF_COUNTER_ADD(seek_child_seek_count, 1); + + if (child.Valid()) { + PERF_TIMER_GUARD(seek_min_heap_time); + minHeap_.push(&child); + } + } + direction_ = kForward; + { + PERF_TIMER_GUARD(seek_min_heap_time); + current_ = CurrentForward(); + } + } + + virtual void SeekForPrev(const Slice& target) override { + ClearHeaps(); + InitMaxHeap(); + + for (auto& child : children_) { + { + PERF_TIMER_GUARD(seek_child_seek_time); + child.SeekForPrev(target); + } + PERF_COUNTER_ADD(seek_child_seek_count, 1); + + if (child.Valid()) { + PERF_TIMER_GUARD(seek_max_heap_time); + maxHeap_->push(&child); + } + } + direction_ = kReverse; + { + PERF_TIMER_GUARD(seek_max_heap_time); + current_ = CurrentReverse(); + } + } + + virtual void Next() override { + assert(Valid()); + + // Ensure that all children are positioned after key(). + // If we are moving in the forward direction, it is already + // true for all of the non-current children since current_ is + // the smallest child and key() == current_->key(). + if (direction_ != kForward) { + // Otherwise, advance the non-current children. We advance current_ + // just after the if-block. + ClearHeaps(); + for (auto& child : children_) { + if (&child != current_) { + child.Seek(key()); + if (child.Valid() && comparator_->Equal(key(), child.key())) { + child.Next(); + } + } + if (child.Valid()) { + minHeap_.push(&child); + } + } + direction_ = kForward; + // The loop advanced all non-current children to be > key() so current_ + // should still be strictly the smallest key. + assert(current_ == CurrentForward()); + } + + // For the heap modifications below to be correct, current_ must be the + // current top of the heap. + assert(current_ == CurrentForward()); + + // as the current points to the current record. move the iterator forward. + current_->Next(); + if (current_->Valid()) { + // current is still valid after the Next() call above. Call + // replace_top() to restore the heap property. When the same child + // iterator yields a sequence of keys, this is cheap. + minHeap_.replace_top(current_); + } else { + // current stopped being valid, remove it from the heap. + minHeap_.pop(); + } + current_ = CurrentForward(); + } + + virtual void Prev() override { + assert(Valid()); + // Ensure that all children are positioned before key(). + // If we are moving in the reverse direction, it is already + // true for all of the non-current children since current_ is + // the largest child and key() == current_->key(). + if (direction_ != kReverse) { + // Otherwise, retreat the non-current children. We retreat current_ + // just after the if-block. + ClearHeaps(); + InitMaxHeap(); + for (auto& child : children_) { + if (&child != current_) { + if (!prefix_seek_mode_) { + child.Seek(key()); + if (child.Valid()) { + // Child is at first entry >= key(). Step back one to be < key() + TEST_SYNC_POINT_CALLBACK("MergeIterator::Prev:BeforePrev", + &child); + child.Prev(); + } else { + // Child has no entries >= key(). Position at last entry. + TEST_SYNC_POINT("MergeIterator::Prev:BeforeSeekToLast"); + child.SeekToLast(); + } + } else { + child.SeekForPrev(key()); + if (child.Valid() && comparator_->Equal(key(), child.key())) { + child.Prev(); + } + } + } + if (child.Valid()) { + maxHeap_->push(&child); + } + } + direction_ = kReverse; + if (!prefix_seek_mode_) { + // Note that we don't do assert(current_ == CurrentReverse()) here + // because it is possible to have some keys larger than the seek-key + // inserted between Seek() and SeekToLast(), which makes current_ not + // equal to CurrentReverse(). + current_ = CurrentReverse(); + } + // The loop advanced all non-current children to be < key() so current_ + // should still be strictly the smallest key. + assert(current_ == CurrentReverse()); + } + + // For the heap modifications below to be correct, current_ must be the + // current top of the heap. + assert(current_ == CurrentReverse()); + + current_->Prev(); + if (current_->Valid()) { + // current is still valid after the Prev() call above. Call + // replace_top() to restore the heap property. When the same child + // iterator yields a sequence of keys, this is cheap. + maxHeap_->replace_top(current_); + } else { + // current stopped being valid, remove it from the heap. + maxHeap_->pop(); + } + current_ = CurrentReverse(); + } + + virtual Slice key() const override { + assert(Valid()); + return current_->key(); + } + + virtual Slice value() const override { + assert(Valid()); + return current_->value(); + } + + virtual Status status() const override { + Status s; + for (auto& child : children_) { + s = child.status(); + if (!s.ok()) { + break; + } + } + return s; + } + + virtual void SetPinnedItersMgr( + PinnedIteratorsManager* pinned_iters_mgr) override { + pinned_iters_mgr_ = pinned_iters_mgr; + for (auto& child : children_) { + child.SetPinnedItersMgr(pinned_iters_mgr); + } + } + + virtual bool IsKeyPinned() const override { + assert(Valid()); + return pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled() && + current_->IsKeyPinned(); + } + + virtual bool IsValuePinned() const override { + assert(Valid()); + return pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled() && + current_->IsValuePinned(); + } + + private: + // Clears heaps for both directions, used when changing direction or seeking + void ClearHeaps(); + // Ensures that maxHeap_ is initialized when starting to go in the reverse + // direction + void InitMaxHeap(); + + bool is_arena_mode_; + const Comparator* comparator_; + autovector children_; + + // Cached pointer to child iterator with the current key, or nullptr if no + // child iterators are valid. This is the top of minHeap_ or maxHeap_ + // depending on the direction. + IteratorWrapper* current_; + // Which direction is the iterator moving? + enum Direction { + kForward, + kReverse + }; + Direction direction_; + MergerMinIterHeap minHeap_; + bool prefix_seek_mode_; + + // Max heap is used for reverse iteration, which is way less common than + // forward. Lazily initialize it to save memory. + std::unique_ptr maxHeap_; + PinnedIteratorsManager* pinned_iters_mgr_; + + IteratorWrapper* CurrentForward() const { + assert(direction_ == kForward); + return !minHeap_.empty() ? minHeap_.top() : nullptr; + } + + IteratorWrapper* CurrentReverse() const { + assert(direction_ == kReverse); + assert(maxHeap_); + return !maxHeap_->empty() ? maxHeap_->top() : nullptr; + } +}; + +void MergingIterator::ClearHeaps() { + minHeap_.clear(); + if (maxHeap_) { + maxHeap_->clear(); + } +} + +void MergingIterator::InitMaxHeap() { + if (!maxHeap_) { + maxHeap_.reset(new MergerMaxIterHeap(comparator_)); + } +} + +InternalIterator* NewMergingIterator(const Comparator* cmp, + InternalIterator** list, int n, + Arena* arena, bool prefix_seek_mode) { + assert(n >= 0); + if (n == 0) { + return NewEmptyInternalIterator(arena); + } else if (n == 1) { + return list[0]; + } else { + if (arena == nullptr) { + return new MergingIterator(cmp, list, n, false, prefix_seek_mode); + } else { + auto mem = arena->AllocateAligned(sizeof(MergingIterator)); + return new (mem) MergingIterator(cmp, list, n, true, prefix_seek_mode); + } + } +} + +MergeIteratorBuilder::MergeIteratorBuilder(const Comparator* comparator, + Arena* a, bool prefix_seek_mode) + : first_iter(nullptr), use_merging_iter(false), arena(a) { + auto mem = arena->AllocateAligned(sizeof(MergingIterator)); + merge_iter = + new (mem) MergingIterator(comparator, nullptr, 0, true, prefix_seek_mode); +} + +void MergeIteratorBuilder::AddIterator(InternalIterator* iter) { + if (!use_merging_iter && first_iter != nullptr) { + merge_iter->AddIterator(first_iter); + use_merging_iter = true; + } + if (use_merging_iter) { + merge_iter->AddIterator(iter); + } else { + first_iter = iter; + } +} + +InternalIterator* MergeIteratorBuilder::Finish() { + if (!use_merging_iter) { + return first_iter; + } else { + auto ret = merge_iter; + merge_iter = nullptr; + return ret; + } +} + +} // namespace rocksdb diff --git a/table/merger.h b/table/merging_iterator.h similarity index 64% rename from table/merger.h rename to table/merging_iterator.h index 7dcf2afe789..48a28d86fd9 100644 --- a/table/merger.h +++ b/table/merging_iterator.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -14,7 +14,7 @@ namespace rocksdb { class Comparator; -class Iterator; +class InternalIterator; class Env; class Arena; @@ -26,9 +26,10 @@ class Arena; // key is present in K child iterators, it will be yielded K times. // // REQUIRES: n >= 0 -extern Iterator* NewMergingIterator(const Comparator* comparator, - Iterator** children, int n, - Arena* arena = nullptr); +extern InternalIterator* NewMergingIterator(const Comparator* comparator, + InternalIterator** children, int n, + Arena* arena = nullptr, + bool prefix_seek_mode = false); class MergingIterator; @@ -37,22 +38,23 @@ class MergeIteratorBuilder { public: // comparator: the comparator used in merging comparator // arena: where the merging iterator needs to be allocated from. - explicit MergeIteratorBuilder(const Comparator* comparator, Arena* arena); + explicit MergeIteratorBuilder(const Comparator* comparator, Arena* arena, + bool prefix_seek_mode = false); ~MergeIteratorBuilder() {} // Add iter to the merging iterator. - void AddIterator(Iterator* iter); + void AddIterator(InternalIterator* iter); // Get arena used to build the merging iterator. It is called one a child // iterator needs to be allocated. Arena* GetArena() { return arena; } // Return the result merging iterator. - Iterator* Finish(); + InternalIterator* Finish(); private: MergingIterator* merge_iter; - Iterator* first_iter; + InternalIterator* first_iter; bool use_merging_iter; Arena* arena; }; diff --git a/table/meta_blocks.cc b/table/meta_blocks.cc index d9d0ed6c991..19925d78897 100644 --- a/table/meta_blocks.cc +++ b/table/meta_blocks.cc @@ -1,17 +1,22 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #include "table/meta_blocks.h" #include #include +#include "db/table_properties_collector.h" #include "rocksdb/table.h" #include "rocksdb/table_properties.h" #include "table/block.h" #include "table/format.h" +#include "table/internal_iterator.h" +#include "table/persistent_cache_helper.h" +#include "table/table_properties_internal.h" #include "util/coding.h" +#include "util/file_reader_writer.h" namespace rocksdb { @@ -61,15 +66,43 @@ void PropertyBlockBuilder::AddTableProperty(const TableProperties& props) { Add(TablePropertiesNames::kRawValueSize, props.raw_value_size); Add(TablePropertiesNames::kDataSize, props.data_size); Add(TablePropertiesNames::kIndexSize, props.index_size); + if (props.index_partitions != 0) { + Add(TablePropertiesNames::kIndexPartitions, props.index_partitions); + Add(TablePropertiesNames::kTopLevelIndexSize, props.top_level_index_size); + } Add(TablePropertiesNames::kNumEntries, props.num_entries); Add(TablePropertiesNames::kNumDataBlocks, props.num_data_blocks); Add(TablePropertiesNames::kFilterSize, props.filter_size); Add(TablePropertiesNames::kFormatVersion, props.format_version); Add(TablePropertiesNames::kFixedKeyLen, props.fixed_key_len); + Add(TablePropertiesNames::kColumnFamilyId, props.column_family_id); + Add(TablePropertiesNames::kCreationTime, props.creation_time); + Add(TablePropertiesNames::kOldestKeyTime, props.oldest_key_time); if (!props.filter_policy_name.empty()) { - Add(TablePropertiesNames::kFilterPolicy, - props.filter_policy_name); + Add(TablePropertiesNames::kFilterPolicy, props.filter_policy_name); + } + if (!props.comparator_name.empty()) { + Add(TablePropertiesNames::kComparator, props.comparator_name); + } + + if (!props.merge_operator_name.empty()) { + Add(TablePropertiesNames::kMergeOperator, props.merge_operator_name); + } + if (!props.prefix_extractor_name.empty()) { + Add(TablePropertiesNames::kPrefixExtractorName, + props.prefix_extractor_name); + } + if (!props.property_collectors_names.empty()) { + Add(TablePropertiesNames::kPropertyCollectors, + props.property_collectors_names); + } + if (!props.column_family_name.empty()) { + Add(TablePropertiesNames::kColumnFamilyName, props.column_family_name); + } + + if (!props.compression_name.empty()) { + Add(TablePropertiesNames::kCompression, props.compression_name); } } @@ -86,18 +119,18 @@ void LogPropertiesCollectionError( assert(method == "Add" || method == "Finish"); std::string msg = - "[Warning] encountered error when calling TablePropertiesCollector::" + + "Encountered error when calling TablePropertiesCollector::" + method + "() with collector name: " + name; - Log(info_log, "%s", msg.c_str()); + ROCKS_LOG_ERROR(info_log, "%s", msg.c_str()); } bool NotifyCollectTableCollectorsOnAdd( - const Slice& key, const Slice& value, - const std::vector>& collectors, + const Slice& key, const Slice& value, uint64_t file_size, + const std::vector>& collectors, Logger* info_log) { bool all_succeeded = true; for (auto& collector : collectors) { - Status s = collector->Add(key, value); + Status s = collector->InternalAdd(key, value, file_size); all_succeeded = all_succeeded && s.ok(); if (!s.ok()) { LogPropertiesCollectionError(info_log, "Add" /* method */, @@ -108,7 +141,7 @@ bool NotifyCollectTableCollectorsOnAdd( } bool NotifyCollectTableCollectorsOnFinish( - const std::vector>& collectors, + const std::vector>& collectors, Logger* info_log, PropertyBlockBuilder* builder) { bool all_succeeded = true; for (auto& collector : collectors) { @@ -127,9 +160,10 @@ bool NotifyCollectTableCollectorsOnFinish( return all_succeeded; } -Status ReadProperties(const Slice &handle_value, RandomAccessFile *file, - const Footer &footer, Env *env, Logger *logger, - TableProperties **table_properties) { +Status ReadProperties(const Slice& handle_value, RandomAccessFileReader* file, + FilePrefetchBuffer* prefetch_buffer, const Footer& footer, + const ImmutableCFOptions& ioptions, + TableProperties** table_properties) { assert(table_properties); Slice v = handle_value; @@ -141,22 +175,28 @@ Status ReadProperties(const Slice &handle_value, RandomAccessFile *file, BlockContents block_contents; ReadOptions read_options; read_options.verify_checksums = false; - Status s = ReadBlockContents(file, footer, read_options, handle, - &block_contents, env, false); + Status s; + s = ReadBlockContents(file, prefetch_buffer, footer, read_options, handle, + &block_contents, ioptions, false /* decompress */); if (!s.ok()) { return s; } - Block properties_block(block_contents); - std::unique_ptr iter( - properties_block.NewIterator(BytewiseComparator())); + Block properties_block(std::move(block_contents), + kDisableGlobalSequenceNumber); + BlockIter iter; + properties_block.NewIterator(BytewiseComparator(), &iter); auto new_table_properties = new TableProperties(); // All pre-defined properties of type uint64_t std::unordered_map predefined_uint64_properties = { {TablePropertiesNames::kDataSize, &new_table_properties->data_size}, {TablePropertiesNames::kIndexSize, &new_table_properties->index_size}, + {TablePropertiesNames::kIndexPartitions, + &new_table_properties->index_partitions}, + {TablePropertiesNames::kTopLevelIndexSize, + &new_table_properties->top_level_index_size}, {TablePropertiesNames::kFilterSize, &new_table_properties->filter_size}, {TablePropertiesNames::kRawKeySize, &new_table_properties->raw_key_size}, {TablePropertiesNames::kRawValueSize, @@ -167,38 +207,60 @@ Status ReadProperties(const Slice &handle_value, RandomAccessFile *file, {TablePropertiesNames::kFormatVersion, &new_table_properties->format_version}, {TablePropertiesNames::kFixedKeyLen, - &new_table_properties->fixed_key_len}, }; + &new_table_properties->fixed_key_len}, + {TablePropertiesNames::kColumnFamilyId, + &new_table_properties->column_family_id}, + {TablePropertiesNames::kCreationTime, + &new_table_properties->creation_time}, + {TablePropertiesNames::kOldestKeyTime, + &new_table_properties->oldest_key_time}, + }; std::string last_key; - for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { - s = iter->status(); + for (iter.SeekToFirst(); iter.Valid(); iter.Next()) { + s = iter.status(); if (!s.ok()) { break; } - auto key = iter->key().ToString(); + auto key = iter.key().ToString(); // properties block is strictly sorted with no duplicate key. assert(last_key.empty() || BytewiseComparator()->Compare(key, last_key) > 0); last_key = key; - auto raw_val = iter->value(); + auto raw_val = iter.value(); auto pos = predefined_uint64_properties.find(key); + new_table_properties->properties_offsets.insert( + {key, handle.offset() + iter.ValueOffset()}); + if (pos != predefined_uint64_properties.end()) { // handle predefined rocksdb properties uint64_t val; if (!GetVarint64(&raw_val, &val)) { // skip malformed value auto error_msg = - "[Warning] detect malformed value in properties meta-block:" + "Detect malformed value in properties meta-block:" "\tkey: " + key + "\tval: " + raw_val.ToString(); - Log(logger, "%s", error_msg.c_str()); + ROCKS_LOG_ERROR(ioptions.info_log, "%s", error_msg.c_str()); continue; } *(pos->second) = val; } else if (key == TablePropertiesNames::kFilterPolicy) { new_table_properties->filter_policy_name = raw_val.ToString(); + } else if (key == TablePropertiesNames::kColumnFamilyName) { + new_table_properties->column_family_name = raw_val.ToString(); + } else if (key == TablePropertiesNames::kComparator) { + new_table_properties->comparator_name = raw_val.ToString(); + } else if (key == TablePropertiesNames::kMergeOperator) { + new_table_properties->merge_operator_name = raw_val.ToString(); + } else if (key == TablePropertiesNames::kPrefixExtractorName) { + new_table_properties->prefix_extractor_name = raw_val.ToString(); + } else if (key == TablePropertiesNames::kPropertyCollectors) { + new_table_properties->property_collectors_names = raw_val.ToString(); + } else if (key == TablePropertiesNames::kCompression) { + new_table_properties->compression_name = raw_val.ToString(); } else { // handle user-collected properties new_table_properties->user_collected_properties.insert( @@ -214,12 +276,14 @@ Status ReadProperties(const Slice &handle_value, RandomAccessFile *file, return s; } -Status ReadTableProperties(RandomAccessFile* file, uint64_t file_size, - uint64_t table_magic_number, Env* env, - Logger* info_log, TableProperties** properties) { +Status ReadTableProperties(RandomAccessFileReader* file, uint64_t file_size, + uint64_t table_magic_number, + const ImmutableCFOptions &ioptions, + TableProperties** properties) { // -- Read metaindex block - Footer footer(table_magic_number); - auto s = ReadFooterFromFile(file, file_size, &footer); + Footer footer; + auto s = ReadFooterFromFile(file, nullptr /* prefetch_buffer */, file_size, + &footer, table_magic_number); if (!s.ok()) { return s; } @@ -228,13 +292,15 @@ Status ReadTableProperties(RandomAccessFile* file, uint64_t file_size, BlockContents metaindex_contents; ReadOptions read_options; read_options.verify_checksums = false; - s = ReadBlockContents(file, footer, read_options, metaindex_handle, - &metaindex_contents, env, false); + s = ReadBlockContents(file, nullptr /* prefetch_buffer */, footer, + read_options, metaindex_handle, &metaindex_contents, + ioptions, false /* decompress */); if (!s.ok()) { return s; } - Block metaindex_block(metaindex_contents); - std::unique_ptr meta_iter( + Block metaindex_block(std::move(metaindex_contents), + kDisableGlobalSequenceNumber); + std::unique_ptr meta_iter( metaindex_block.NewIterator(BytewiseComparator())); // -- Read property block @@ -246,8 +312,8 @@ Status ReadTableProperties(RandomAccessFile* file, uint64_t file_size, TableProperties table_properties; if (found_properties_block == true) { - s = ReadProperties(meta_iter->value(), file, footer, env, info_log, - properties); + s = ReadProperties(meta_iter->value(), file, nullptr /* prefetch_buffer */, + footer, ioptions, properties); } else { s = Status::NotFound(); } @@ -255,7 +321,7 @@ Status ReadTableProperties(RandomAccessFile* file, uint64_t file_size, return s; } -Status FindMetaBlock(Iterator* meta_index_iter, +Status FindMetaBlock(InternalIterator* meta_index_iter, const std::string& meta_block_name, BlockHandle* block_handle) { meta_index_iter->Seek(meta_block_name); @@ -268,12 +334,14 @@ Status FindMetaBlock(Iterator* meta_index_iter, } } -Status FindMetaBlock(RandomAccessFile* file, uint64_t file_size, - uint64_t table_magic_number, Env* env, +Status FindMetaBlock(RandomAccessFileReader* file, uint64_t file_size, + uint64_t table_magic_number, + const ImmutableCFOptions &ioptions, const std::string& meta_block_name, BlockHandle* block_handle) { - Footer footer(table_magic_number); - auto s = ReadFooterFromFile(file, file_size, &footer); + Footer footer; + auto s = ReadFooterFromFile(file, nullptr /* prefetch_buffer */, file_size, + &footer, table_magic_number); if (!s.ok()) { return s; } @@ -282,27 +350,33 @@ Status FindMetaBlock(RandomAccessFile* file, uint64_t file_size, BlockContents metaindex_contents; ReadOptions read_options; read_options.verify_checksums = false; - s = ReadBlockContents(file, footer, read_options, metaindex_handle, - &metaindex_contents, env, false); + s = ReadBlockContents(file, nullptr /* prefetch_buffer */, footer, + read_options, metaindex_handle, &metaindex_contents, + ioptions, false /* do decompression */); if (!s.ok()) { return s; } - Block metaindex_block(metaindex_contents); + Block metaindex_block(std::move(metaindex_contents), + kDisableGlobalSequenceNumber); - std::unique_ptr meta_iter; + std::unique_ptr meta_iter; meta_iter.reset(metaindex_block.NewIterator(BytewiseComparator())); return FindMetaBlock(meta_iter.get(), meta_block_name, block_handle); } -Status ReadMetaBlock(RandomAccessFile* file, uint64_t file_size, - uint64_t table_magic_number, Env* env, +Status ReadMetaBlock(RandomAccessFileReader* file, + FilePrefetchBuffer* prefetch_buffer, uint64_t file_size, + uint64_t table_magic_number, + const ImmutableCFOptions& ioptions, const std::string& meta_block_name, BlockContents* contents) { - Footer footer(table_magic_number); - auto s = ReadFooterFromFile(file, file_size, &footer); - if (!s.ok()) { - return s; + Status status; + Footer footer; + status = ReadFooterFromFile(file, prefetch_buffer, file_size, &footer, + table_magic_number); + if (!status.ok()) { + return status; } // Reading metaindex block @@ -310,30 +384,31 @@ Status ReadMetaBlock(RandomAccessFile* file, uint64_t file_size, BlockContents metaindex_contents; ReadOptions read_options; read_options.verify_checksums = false; - s = ReadBlockContents(file, footer, read_options, metaindex_handle, - &metaindex_contents, env, false); - if (!s.ok()) { - return s; + status = ReadBlockContents(file, prefetch_buffer, footer, read_options, + metaindex_handle, &metaindex_contents, ioptions, + false /* decompress */); + if (!status.ok()) { + return status; } // Finding metablock - Block metaindex_block(metaindex_contents); + Block metaindex_block(std::move(metaindex_contents), + kDisableGlobalSequenceNumber); - std::unique_ptr meta_iter; + std::unique_ptr meta_iter; meta_iter.reset(metaindex_block.NewIterator(BytewiseComparator())); BlockHandle block_handle; - s = FindMetaBlock(meta_iter.get(), meta_block_name, &block_handle); + status = FindMetaBlock(meta_iter.get(), meta_block_name, &block_handle); - if (!s.ok()) { - return s; + if (!status.ok()) { + return status; } // Reading metablock - s = ReadBlockContents(file, footer, read_options, block_handle, contents, env, - false); - - return s; + return ReadBlockContents(file, prefetch_buffer, footer, read_options, + block_handle, contents, ioptions, + false /* decompress */); } } // namespace rocksdb diff --git a/table/meta_blocks.h b/table/meta_blocks.h index 798a18af013..220985d9e10 100644 --- a/table/meta_blocks.h +++ b/table/meta_blocks.h @@ -1,19 +1,20 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #include #include -#include #include +#include #include "db/builder.h" +#include "db/table_properties_collector.h" +#include "util/kv_map.h" #include "rocksdb/comparator.h" #include "rocksdb/options.h" #include "rocksdb/slice.h" -#include "rocksdb/table_properties.h" #include "table/block_builder.h" #include "table/format.h" @@ -26,22 +27,7 @@ class Footer; class Logger; class RandomAccessFile; struct TableProperties; - -// An STL style comparator that does the bytewise comparator comparasion -// internally. -struct BytewiseLessThan { - bool operator()(const std::string& key1, const std::string& key2) const { - // smaller entries will be placed in front. - return comparator->Compare(key1, key2) <= 0; - } - - const Comparator* comparator = BytewiseComparator(); -}; - -// When writing to a block that requires entries to be sorted by -// `BytewiseComparator`, we can buffer the content to `BytewiseSortedMap` -// before writng to store. -typedef std::map BytewiseSortedMap; +class InternalIterator; class MetaIndexBuilder { public: @@ -57,7 +43,7 @@ class MetaIndexBuilder { private: // store the sorted key/handle of the metablocks. - BytewiseSortedMap meta_block_handles_; + stl_wrappers::KVMap meta_block_handles_; std::unique_ptr meta_index_block_; }; @@ -78,7 +64,7 @@ class PropertyBlockBuilder { private: std::unique_ptr properties_block_; - BytewiseSortedMap props_; + stl_wrappers::KVMap props_; }; // Were we encounter any error occurs during user-defined statistics collection, @@ -93,53 +79,53 @@ void LogPropertiesCollectionError( // NotifyCollectTableCollectorsOnAdd() triggers the `Add` event for all // property collectors. bool NotifyCollectTableCollectorsOnAdd( - const Slice& key, const Slice& value, - const std::vector>& collectors, + const Slice& key, const Slice& value, uint64_t file_size, + const std::vector>& collectors, Logger* info_log); // NotifyCollectTableCollectorsOnAdd() triggers the `Finish` event for all // property collectors. The collected properties will be added to `builder`. bool NotifyCollectTableCollectorsOnFinish( - const std::vector>& collectors, + const std::vector>& collectors, Logger* info_log, PropertyBlockBuilder* builder); // Read the properties from the table. // @returns a status to indicate if the operation succeeded. On success, // *table_properties will point to a heap-allocated TableProperties // object, otherwise value of `table_properties` will not be modified. -Status ReadProperties(const Slice &handle_value, RandomAccessFile *file, - const Footer &footer, Env *env, Logger *logger, - TableProperties **table_properties); +Status ReadProperties(const Slice& handle_value, RandomAccessFileReader* file, + FilePrefetchBuffer* prefetch_buffer, const Footer& footer, + const ImmutableCFOptions& ioptions, + TableProperties** table_properties); // Directly read the properties from the properties block of a plain table. // @returns a status to indicate if the operation succeeded. On success, // *table_properties will point to a heap-allocated TableProperties // object, otherwise value of `table_properties` will not be modified. -Status ReadTableProperties(RandomAccessFile* file, uint64_t file_size, - uint64_t table_magic_number, Env* env, - Logger* info_log, TableProperties** properties); - -// Seek to the properties block. -// If it successfully seeks to the properties block, "is_found" will be -// set to true. -extern Status SeekToPropertiesBlock(Iterator* meta_iter, bool* is_found); +Status ReadTableProperties(RandomAccessFileReader* file, uint64_t file_size, + uint64_t table_magic_number, + const ImmutableCFOptions &ioptions, + TableProperties** properties); // Find the meta block from the meta index block. -Status FindMetaBlock(Iterator* meta_index_iter, +Status FindMetaBlock(InternalIterator* meta_index_iter, const std::string& meta_block_name, BlockHandle* block_handle); // Find the meta block -Status FindMetaBlock(RandomAccessFile* file, uint64_t file_size, - uint64_t table_magic_number, Env* env, +Status FindMetaBlock(RandomAccessFileReader* file, uint64_t file_size, + uint64_t table_magic_number, + const ImmutableCFOptions &ioptions, const std::string& meta_block_name, BlockHandle* block_handle); // Read the specified meta block with name meta_block_name // from `file` and initialize `contents` with contents of this block. // Return Status::OK in case of success. -Status ReadMetaBlock(RandomAccessFile* file, uint64_t file_size, - uint64_t table_magic_number, Env* env, +Status ReadMetaBlock(RandomAccessFileReader* file, + FilePrefetchBuffer* prefetch_buffer, uint64_t file_size, + uint64_t table_magic_number, + const ImmutableCFOptions& ioptions, const std::string& meta_block_name, BlockContents* contents); diff --git a/table/mock_table.cc b/table/mock_table.cc new file mode 100644 index 00000000000..86c380865c6 --- /dev/null +++ b/table/mock_table.cc @@ -0,0 +1,142 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "table/mock_table.h" + +#include "db/dbformat.h" +#include "port/port.h" +#include "rocksdb/table_properties.h" +#include "table/get_context.h" +#include "util/coding.h" +#include "util/file_reader_writer.h" + +namespace rocksdb { +namespace mock { + +namespace { + +const InternalKeyComparator icmp_(BytewiseComparator()); + +} // namespace + +stl_wrappers::KVMap MakeMockFile( + std::initializer_list> l) { + return stl_wrappers::KVMap(l, stl_wrappers::LessOfComparator(&icmp_)); +} + +InternalIterator* MockTableReader::NewIterator(const ReadOptions&, + Arena* arena, + bool skip_filters) { + return new MockTableIterator(table_); +} + +Status MockTableReader::Get(const ReadOptions&, const Slice& key, + GetContext* get_context, bool skip_filters) { + std::unique_ptr iter(new MockTableIterator(table_)); + for (iter->Seek(key); iter->Valid(); iter->Next()) { + ParsedInternalKey parsed_key; + if (!ParseInternalKey(iter->key(), &parsed_key)) { + return Status::Corruption(Slice()); + } + + if (!get_context->SaveValue(parsed_key, iter->value())) { + break; + } + } + return Status::OK(); +} + +std::shared_ptr MockTableReader::GetTableProperties() + const { + return std::shared_ptr(new TableProperties()); +} + +MockTableFactory::MockTableFactory() : next_id_(1) {} + +Status MockTableFactory::NewTableReader( + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + unique_ptr* table_reader, + bool prefetch_index_and_filter_in_cache) const { + uint32_t id = GetIDFromFile(file.get()); + + MutexLock lock_guard(&file_system_.mutex); + + auto it = file_system_.files.find(id); + if (it == file_system_.files.end()) { + return Status::IOError("Mock file not found"); + } + + table_reader->reset(new MockTableReader(it->second)); + + return Status::OK(); +} + +TableBuilder* MockTableFactory::NewTableBuilder( + const TableBuilderOptions& table_builder_options, uint32_t column_family_id, + WritableFileWriter* file) const { + uint32_t id = GetAndWriteNextID(file); + + return new MockTableBuilder(id, &file_system_); +} + +Status MockTableFactory::CreateMockTable(Env* env, const std::string& fname, + stl_wrappers::KVMap file_contents) { + std::unique_ptr file; + auto s = env->NewWritableFile(fname, &file, EnvOptions()); + if (!s.ok()) { + return s; + } + + WritableFileWriter file_writer(std::move(file), EnvOptions()); + + uint32_t id = GetAndWriteNextID(&file_writer); + file_system_.files.insert({id, std::move(file_contents)}); + return Status::OK(); +} + +uint32_t MockTableFactory::GetAndWriteNextID(WritableFileWriter* file) const { + uint32_t next_id = next_id_.fetch_add(1); + char buf[4]; + EncodeFixed32(buf, next_id); + file->Append(Slice(buf, 4)); + return next_id; +} + +uint32_t MockTableFactory::GetIDFromFile(RandomAccessFileReader* file) const { + char buf[4]; + Slice result; + file->Read(0, 4, &result, buf); + assert(result.size() == 4); + return DecodeFixed32(buf); +} + +void MockTableFactory::AssertSingleFile( + const stl_wrappers::KVMap& file_contents) { + ASSERT_EQ(file_system_.files.size(), 1U); + ASSERT_EQ(file_contents, file_system_.files.begin()->second); +} + +void MockTableFactory::AssertLatestFile( + const stl_wrappers::KVMap& file_contents) { + ASSERT_GE(file_system_.files.size(), 1U); + auto latest = file_system_.files.end(); + --latest; + + if (file_contents != latest->second) { + std::cout << "Wrong content! Content of latest file:" << std::endl; + for (const auto& kv : latest->second) { + ParsedInternalKey ikey; + std::string key, value; + std::tie(key, value) = kv; + ParseInternalKey(Slice(key), &ikey); + std::cout << ikey.DebugString(false) << " -> " << value << std::endl; + } + FAIL(); + } +} + +} // namespace mock +} // namespace rocksdb diff --git a/table/mock_table.h b/table/mock_table.h new file mode 100644 index 00000000000..71609a173fb --- /dev/null +++ b/table/mock_table.h @@ -0,0 +1,194 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once + +#include +#include +#include +#include +#include +#include +#include + +#include "util/kv_map.h" +#include "port/port.h" +#include "rocksdb/comparator.h" +#include "rocksdb/table.h" +#include "table/internal_iterator.h" +#include "table/table_builder.h" +#include "table/table_reader.h" +#include "util/mutexlock.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { +namespace mock { + +stl_wrappers::KVMap MakeMockFile( + std::initializer_list> l = {}); + +struct MockTableFileSystem { + port::Mutex mutex; + std::map files; +}; + +class MockTableReader : public TableReader { + public: + explicit MockTableReader(const stl_wrappers::KVMap& table) : table_(table) {} + + InternalIterator* NewIterator(const ReadOptions&, + Arena* arena, + bool skip_filters = false) override; + + Status Get(const ReadOptions&, const Slice& key, GetContext* get_context, + bool skip_filters = false) override; + + uint64_t ApproximateOffsetOf(const Slice& key) override { return 0; } + + virtual size_t ApproximateMemoryUsage() const override { return 0; } + + void SetupForCompaction() override {} + + std::shared_ptr GetTableProperties() const override; + + ~MockTableReader() {} + + private: + const stl_wrappers::KVMap& table_; +}; + +class MockTableIterator : public InternalIterator { + public: + explicit MockTableIterator(const stl_wrappers::KVMap& table) : table_(table) { + itr_ = table_.end(); + } + + bool Valid() const override { return itr_ != table_.end(); } + + void SeekToFirst() override { itr_ = table_.begin(); } + + void SeekToLast() override { + itr_ = table_.end(); + --itr_; + } + + void Seek(const Slice& target) override { + std::string str_target(target.data(), target.size()); + itr_ = table_.lower_bound(str_target); + } + + void SeekForPrev(const Slice& target) override { + std::string str_target(target.data(), target.size()); + itr_ = table_.upper_bound(str_target); + Prev(); + } + + void Next() override { ++itr_; } + + void Prev() override { + if (itr_ == table_.begin()) { + itr_ = table_.end(); + } else { + --itr_; + } + } + + Slice key() const override { return Slice(itr_->first); } + + Slice value() const override { return Slice(itr_->second); } + + Status status() const override { return Status::OK(); } + + private: + const stl_wrappers::KVMap& table_; + stl_wrappers::KVMap::const_iterator itr_; +}; + +class MockTableBuilder : public TableBuilder { + public: + MockTableBuilder(uint32_t id, MockTableFileSystem* file_system) + : id_(id), file_system_(file_system) { + table_ = MakeMockFile({}); + } + + // REQUIRES: Either Finish() or Abandon() has been called. + ~MockTableBuilder() {} + + // Add key,value to the table being constructed. + // REQUIRES: key is after any previously added key according to comparator. + // REQUIRES: Finish(), Abandon() have not been called + void Add(const Slice& key, const Slice& value) override { + table_.insert({key.ToString(), value.ToString()}); + } + + // Return non-ok iff some error has been detected. + Status status() const override { return Status::OK(); } + + Status Finish() override { + MutexLock lock_guard(&file_system_->mutex); + file_system_->files.insert({id_, table_}); + return Status::OK(); + } + + void Abandon() override {} + + uint64_t NumEntries() const override { return table_.size(); } + + uint64_t FileSize() const override { return table_.size(); } + + TableProperties GetTableProperties() const override { + return TableProperties(); + } + + private: + uint32_t id_; + MockTableFileSystem* file_system_; + stl_wrappers::KVMap table_; +}; + +class MockTableFactory : public TableFactory { + public: + MockTableFactory(); + const char* Name() const override { return "MockTable"; } + Status NewTableReader( + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + unique_ptr* table_reader, + bool prefetch_index_and_filter_in_cache = true) const override; + TableBuilder* NewTableBuilder( + const TableBuilderOptions& table_builder_options, + uint32_t column_familly_id, WritableFileWriter* file) const override; + + // This function will directly create mock table instead of going through + // MockTableBuilder. file_contents has to have a format of . Those key-value pairs will then be inserted into the mock table. + Status CreateMockTable(Env* env, const std::string& fname, + stl_wrappers::KVMap file_contents); + + virtual Status SanitizeOptions( + const DBOptions& db_opts, + const ColumnFamilyOptions& cf_opts) const override { + return Status::OK(); + } + + virtual std::string GetPrintableTableOptions() const override { + return std::string(); + } + + // This function will assert that only a single file exists and that the + // contents are equal to file_contents + void AssertSingleFile(const stl_wrappers::KVMap& file_contents); + void AssertLatestFile(const stl_wrappers::KVMap& file_contents); + + private: + uint32_t GetAndWriteNextID(WritableFileWriter* file) const; + uint32_t GetIDFromFile(RandomAccessFileReader* file) const; + + mutable MockTableFileSystem file_system_; + mutable std::atomic next_id_; +}; + +} // namespace mock +} // namespace rocksdb diff --git a/table/partitioned_filter_block.cc b/table/partitioned_filter_block.cc new file mode 100644 index 00000000000..202245939fe --- /dev/null +++ b/table/partitioned_filter_block.cc @@ -0,0 +1,311 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "table/partitioned_filter_block.h" + +#include + +#include "monitoring/perf_context_imp.h" +#include "port/port.h" +#include "rocksdb/filter_policy.h" +#include "table/block.h" +#include "table/block_based_table_reader.h" +#include "util/coding.h" + +namespace rocksdb { + +PartitionedFilterBlockBuilder::PartitionedFilterBlockBuilder( + const SliceTransform* prefix_extractor, bool whole_key_filtering, + FilterBitsBuilder* filter_bits_builder, int index_block_restart_interval, + PartitionedIndexBuilder* const p_index_builder, + const uint32_t partition_size) + : FullFilterBlockBuilder(prefix_extractor, whole_key_filtering, + filter_bits_builder), + index_on_filter_block_builder_(index_block_restart_interval), + p_index_builder_(p_index_builder), + filters_in_partition_(0) { + filters_per_partition_ = + filter_bits_builder_->CalculateNumEntry(partition_size); +} + +PartitionedFilterBlockBuilder::~PartitionedFilterBlockBuilder() {} + +void PartitionedFilterBlockBuilder::MaybeCutAFilterBlock() { + // Use == to send the request only once + if (filters_in_partition_ == filters_per_partition_) { + // Currently only index builder is in charge of cutting a partition. We keep + // requesting until it is granted. + p_index_builder_->RequestPartitionCut(); + } + if (!p_index_builder_->ShouldCutFilterBlock()) { + return; + } + filter_gc.push_back(std::unique_ptr(nullptr)); + Slice filter = filter_bits_builder_->Finish(&filter_gc.back()); + std::string& index_key = p_index_builder_->GetPartitionKey(); + filters.push_back({index_key, filter}); + filters_in_partition_ = 0; +} + +void PartitionedFilterBlockBuilder::AddKey(const Slice& key) { + MaybeCutAFilterBlock(); + filter_bits_builder_->AddKey(key); + filters_in_partition_++; +} + +Slice PartitionedFilterBlockBuilder::Finish( + const BlockHandle& last_partition_block_handle, Status* status) { + if (finishing_filters == true) { + // Record the handle of the last written filter block in the index + FilterEntry& last_entry = filters.front(); + std::string handle_encoding; + last_partition_block_handle.EncodeTo(&handle_encoding); + index_on_filter_block_builder_.Add(last_entry.key, handle_encoding); + filters.pop_front(); + } else { + MaybeCutAFilterBlock(); + } + // If there is no filter partition left, then return the index on filter + // partitions + if (UNLIKELY(filters.empty())) { + *status = Status::OK(); + if (finishing_filters) { + return index_on_filter_block_builder_.Finish(); + } else { + // This is the rare case where no key was added to the filter + return Slice(); + } + } else { + // Return the next filter partition in line and set Incomplete() status to + // indicate we expect more calls to Finish + *status = Status::Incomplete(); + finishing_filters = true; + return filters.front().filter; + } +} + +PartitionedFilterBlockReader::PartitionedFilterBlockReader( + const SliceTransform* prefix_extractor, bool _whole_key_filtering, + BlockContents&& contents, FilterBitsReader* filter_bits_reader, + Statistics* stats, const Comparator& comparator, + const BlockBasedTable* table) + : FilterBlockReader(contents.data.size(), stats, _whole_key_filtering), + prefix_extractor_(prefix_extractor), + comparator_(comparator), + table_(table) { + idx_on_fltr_blk_.reset(new Block(std::move(contents), + kDisableGlobalSequenceNumber, + 0 /* read_amp_bytes_per_bit */, stats)); +} + +PartitionedFilterBlockReader::~PartitionedFilterBlockReader() { + // TODO(myabandeh): if instead of filter object we store only the blocks in + // block cache, then we don't have to manually earse them from block cache + // here. + auto block_cache = table_->rep_->table_options.block_cache.get(); + if (UNLIKELY(block_cache == nullptr)) { + return; + } + char cache_key[BlockBasedTable::kMaxCacheKeyPrefixSize + kMaxVarint64Length]; + BlockIter biter; + BlockHandle handle; + idx_on_fltr_blk_->NewIterator(&comparator_, &biter, true); + biter.SeekToFirst(); + for (; biter.Valid(); biter.Next()) { + auto input = biter.value(); + auto s = handle.DecodeFrom(&input); + assert(s.ok()); + if (!s.ok()) { + continue; + } + auto key = BlockBasedTable::GetCacheKey(table_->rep_->cache_key_prefix, + table_->rep_->cache_key_prefix_size, + handle, cache_key); + block_cache->Erase(key); + } +} + +bool PartitionedFilterBlockReader::KeyMayMatch( + const Slice& key, uint64_t block_offset, const bool no_io, + const Slice* const const_ikey_ptr) { + assert(const_ikey_ptr != nullptr); + assert(block_offset == kNotValid); + if (!whole_key_filtering_) { + return true; + } + if (UNLIKELY(idx_on_fltr_blk_->size() == 0)) { + return true; + } + auto filter_handle = GetFilterPartitionHandle(*const_ikey_ptr); + if (UNLIKELY(filter_handle.size() == 0)) { // key is out of range + return false; + } + bool cached = false; + auto filter_partition = GetFilterPartition(nullptr /* prefetch_buffer */, + &filter_handle, no_io, &cached); + if (UNLIKELY(!filter_partition.value)) { + return true; + } + auto res = filter_partition.value->KeyMayMatch(key, block_offset, no_io); + if (cached) { + return res; + } + if (LIKELY(filter_partition.IsSet())) { + filter_partition.Release(table_->rep_->table_options.block_cache.get()); + } else { + delete filter_partition.value; + } + return res; +} + +bool PartitionedFilterBlockReader::PrefixMayMatch( + const Slice& prefix, uint64_t block_offset, const bool no_io, + const Slice* const const_ikey_ptr) { + assert(const_ikey_ptr != nullptr); + assert(block_offset == kNotValid); + if (!prefix_extractor_) { + return true; + } + if (UNLIKELY(idx_on_fltr_blk_->size() == 0)) { + return true; + } + auto filter_handle = GetFilterPartitionHandle(*const_ikey_ptr); + if (UNLIKELY(filter_handle.size() == 0)) { // prefix is out of range + return false; + } + bool cached = false; + auto filter_partition = GetFilterPartition(nullptr /* prefetch_buffer */, + &filter_handle, no_io, &cached); + if (UNLIKELY(!filter_partition.value)) { + return true; + } + auto res = filter_partition.value->PrefixMayMatch(prefix, kNotValid, no_io); + if (cached) { + return res; + } + if (LIKELY(filter_partition.IsSet())) { + filter_partition.Release(table_->rep_->table_options.block_cache.get()); + } else { + delete filter_partition.value; + } + return res; +} + +Slice PartitionedFilterBlockReader::GetFilterPartitionHandle( + const Slice& entry) { + BlockIter iter; + idx_on_fltr_blk_->NewIterator(&comparator_, &iter, true); + iter.Seek(entry); + if (UNLIKELY(!iter.Valid())) { + return Slice(); + } + assert(iter.Valid()); + Slice handle_value = iter.value(); + return handle_value; +} + +BlockBasedTable::CachableEntry +PartitionedFilterBlockReader::GetFilterPartition( + FilePrefetchBuffer* prefetch_buffer, Slice* handle_value, const bool no_io, + bool* cached) { + BlockHandle fltr_blk_handle; + auto s = fltr_blk_handle.DecodeFrom(handle_value); + assert(s.ok()); + const bool is_a_filter_partition = true; + auto block_cache = table_->rep_->table_options.block_cache.get(); + if (LIKELY(block_cache != nullptr)) { + if (filter_map_.size() != 0) { + auto iter = filter_map_.find(fltr_blk_handle.offset()); + // This is a possible scenario since block cache might not have had space + // for the partition + if (iter != filter_map_.end()) { + PERF_COUNTER_ADD(block_cache_hit_count, 1); + RecordTick(statistics(), BLOCK_CACHE_FILTER_HIT); + RecordTick(statistics(), BLOCK_CACHE_HIT); + RecordTick(statistics(), BLOCK_CACHE_BYTES_READ, + block_cache->GetUsage(iter->second.cache_handle)); + *cached = true; + return iter->second; + } + } + return table_->GetFilter(/*prefetch_buffer*/ nullptr, fltr_blk_handle, + is_a_filter_partition, no_io); + } else { + auto filter = table_->ReadFilter(prefetch_buffer, fltr_blk_handle, + is_a_filter_partition); + return {filter, nullptr}; + } +} + +size_t PartitionedFilterBlockReader::ApproximateMemoryUsage() const { + return idx_on_fltr_blk_->size(); +} + +// TODO(myabandeh): merge this with the same function in IndexReader +void PartitionedFilterBlockReader::CacheDependencies(bool pin) { + // Before read partitions, prefetch them to avoid lots of IOs + auto rep = table_->rep_; + BlockIter biter; + BlockHandle handle; + idx_on_fltr_blk_->NewIterator(&comparator_, &biter, true); + // Index partitions are assumed to be consecuitive. Prefetch them all. + // Read the first block offset + biter.SeekToFirst(); + Slice input = biter.value(); + Status s = handle.DecodeFrom(&input); + assert(s.ok()); + if (!s.ok()) { + ROCKS_LOG_WARN(rep->ioptions.info_log, + "Could not read first index partition"); + return; + } + uint64_t prefetch_off = handle.offset(); + + // Read the last block's offset + biter.SeekToLast(); + input = biter.value(); + s = handle.DecodeFrom(&input); + assert(s.ok()); + if (!s.ok()) { + ROCKS_LOG_WARN(rep->ioptions.info_log, + "Could not read last index partition"); + return; + } + uint64_t last_off = handle.offset() + handle.size() + kBlockTrailerSize; + uint64_t prefetch_len = last_off - prefetch_off; + std::unique_ptr prefetch_buffer; + auto& file = table_->rep_->file; + prefetch_buffer.reset(new FilePrefetchBuffer()); + s = prefetch_buffer->Prefetch(file.get(), prefetch_off, prefetch_len); + + // After prefetch, read the partitions one by one + biter.SeekToFirst(); + Cache* block_cache = rep->table_options.block_cache.get(); + for (; biter.Valid(); biter.Next()) { + input = biter.value(); + s = handle.DecodeFrom(&input); + assert(s.ok()); + if (!s.ok()) { + ROCKS_LOG_WARN(rep->ioptions.info_log, "Could not read index partition"); + continue; + } + + const bool no_io = true; + const bool is_a_filter_partition = true; + auto filter = table_->GetFilter(prefetch_buffer.get(), handle, + is_a_filter_partition, !no_io); + if (LIKELY(filter.IsSet())) { + if (pin) { + filter_map_[handle.offset()] = std::move(filter); + } else { + block_cache->Release(filter.cache_handle); + } + } else { + delete filter.value; + } + } +} + +} // namespace rocksdb diff --git a/table/partitioned_filter_block.h b/table/partitioned_filter_block.h new file mode 100644 index 00000000000..1a00a86e6ce --- /dev/null +++ b/table/partitioned_filter_block.h @@ -0,0 +1,102 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include +#include +#include +#include "db/dbformat.h" +#include "rocksdb/options.h" +#include "rocksdb/slice.h" +#include "rocksdb/slice_transform.h" + +#include "table/block.h" +#include "table/block_based_table_reader.h" +#include "table/full_filter_block.h" +#include "table/index_builder.h" +#include "util/autovector.h" + +namespace rocksdb { + +class PartitionedFilterBlockBuilder : public FullFilterBlockBuilder { + public: + explicit PartitionedFilterBlockBuilder( + const SliceTransform* prefix_extractor, bool whole_key_filtering, + FilterBitsBuilder* filter_bits_builder, int index_block_restart_interval, + PartitionedIndexBuilder* const p_index_builder, + const uint32_t partition_size); + + virtual ~PartitionedFilterBlockBuilder(); + + void AddKey(const Slice& key) override; + + virtual Slice Finish(const BlockHandle& last_partition_block_handle, + Status* status) override; + + private: + // Filter data + BlockBuilder index_on_filter_block_builder_; // top-level index builder + struct FilterEntry { + std::string key; + Slice filter; + }; + std::list filters; // list of partitioned indexes and their keys + std::unique_ptr value; + std::vector> filter_gc; + bool finishing_filters = + false; // true if Finish is called once but not complete yet. + // The policy of when cut a filter block and Finish it + void MaybeCutAFilterBlock(); + // Currently we keep the same number of partitions for filters and indexes. + // This would allow for some potentioal optimizations in future. If such + // optimizations did not realize we can use different number of partitions and + // eliminate p_index_builder_ + PartitionedIndexBuilder* const p_index_builder_; + // The desired number of filters per partition + uint32_t filters_per_partition_; + // The current number of filters in the last partition + uint32_t filters_in_partition_; +}; + +class PartitionedFilterBlockReader : public FilterBlockReader { + public: + explicit PartitionedFilterBlockReader(const SliceTransform* prefix_extractor, + bool whole_key_filtering, + BlockContents&& contents, + FilterBitsReader* filter_bits_reader, + Statistics* stats, + const Comparator& comparator, + const BlockBasedTable* table); + virtual ~PartitionedFilterBlockReader(); + + virtual bool IsBlockBased() override { return false; } + virtual bool KeyMayMatch( + const Slice& key, uint64_t block_offset = kNotValid, + const bool no_io = false, + const Slice* const const_ikey_ptr = nullptr) override; + virtual bool PrefixMayMatch( + const Slice& prefix, uint64_t block_offset = kNotValid, + const bool no_io = false, + const Slice* const const_ikey_ptr = nullptr) override; + virtual size_t ApproximateMemoryUsage() const override; + + private: + Slice GetFilterPartitionHandle(const Slice& entry); + BlockBasedTable::CachableEntry GetFilterPartition( + FilePrefetchBuffer* prefetch_buffer, Slice* handle, const bool no_io, + bool* cached); + virtual void CacheDependencies(bool pin) override; + + const SliceTransform* prefix_extractor_; + std::unique_ptr idx_on_fltr_blk_; + const Comparator& comparator_; + const BlockBasedTable* table_; + std::unordered_map> + filter_map_; +}; + +} // namespace rocksdb diff --git a/table/partitioned_filter_block_test.cc b/table/partitioned_filter_block_test.cc new file mode 100644 index 00000000000..1bc529ed974 --- /dev/null +++ b/table/partitioned_filter_block_test.cc @@ -0,0 +1,299 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include + +#include "rocksdb/filter_policy.h" + +#include "table/full_filter_bits_builder.h" +#include "table/index_builder.h" +#include "table/partitioned_filter_block.h" +#include "util/coding.h" +#include "util/hash.h" +#include "util/logging.h" +#include "util/testharness.h" +#include "util/testutil.h" + +namespace rocksdb { + +std::map slices; + +class MockedBlockBasedTable : public BlockBasedTable { + public: + explicit MockedBlockBasedTable(Rep* rep) : BlockBasedTable(rep) { + // Initialize what Open normally does as much as necessary for the test + rep->cache_key_prefix_size = 10; + } + + virtual CachableEntry GetFilter( + FilePrefetchBuffer*, const BlockHandle& filter_blk_handle, + const bool /* unused */, bool /* unused */) const override { + Slice slice = slices[filter_blk_handle.offset()]; + auto obj = new FullFilterBlockReader( + nullptr, true, BlockContents(slice, false, kNoCompression), + rep_->table_options.filter_policy->GetFilterBitsReader(slice), nullptr); + return {obj, nullptr}; + } +}; + +class PartitionedFilterBlockTest : public testing::Test { + public: + BlockBasedTableOptions table_options_; + InternalKeyComparator icomp = InternalKeyComparator(BytewiseComparator()); + + PartitionedFilterBlockTest() { + table_options_.filter_policy.reset(NewBloomFilterPolicy(10, false)); + table_options_.no_block_cache = true; // Otherwise BlockBasedTable::Close + // will access variable that are not + // initialized in our mocked version + } + + std::shared_ptr cache_; + ~PartitionedFilterBlockTest() {} + + const std::string keys[4] = {"afoo", "bar", "box", "hello"}; + const std::string missing_keys[2] = {"missing", "other"}; + + uint64_t MaxIndexSize() { + int num_keys = sizeof(keys) / sizeof(*keys); + uint64_t max_key_size = 0; + for (int i = 1; i < num_keys; i++) { + max_key_size = std::max(max_key_size, static_cast(keys[i].size())); + } + uint64_t max_index_size = num_keys * (max_key_size + 8 /*handle*/); + return max_index_size; + } + + uint64_t MaxFilterSize() { + uint32_t dont_care1, dont_care2; + int num_keys = sizeof(keys) / sizeof(*keys); + auto filter_bits_reader = dynamic_cast( + table_options_.filter_policy->GetFilterBitsBuilder()); + assert(filter_bits_reader); + auto partition_size = + filter_bits_reader->CalculateSpace(num_keys, &dont_care1, &dont_care2); + delete filter_bits_reader; + return partition_size + table_options_.block_size_deviation; + } + + int last_offset = 10; + BlockHandle Write(const Slice& slice) { + BlockHandle bh(last_offset + 1, slice.size()); + slices[bh.offset()] = slice; + last_offset += bh.size(); + return bh; + } + + PartitionedIndexBuilder* NewIndexBuilder() { + return PartitionedIndexBuilder::CreateIndexBuilder(&icomp, table_options_); + } + + PartitionedFilterBlockBuilder* NewBuilder( + PartitionedIndexBuilder* const p_index_builder) { + assert(table_options_.block_size_deviation <= 100); + auto partition_size = static_cast( + table_options_.metadata_block_size * + ( 100 - table_options_.block_size_deviation)); + partition_size = std::max(partition_size, static_cast(1)); + return new PartitionedFilterBlockBuilder( + nullptr, table_options_.whole_key_filtering, + table_options_.filter_policy->GetFilterBitsBuilder(), + table_options_.index_block_restart_interval, p_index_builder, + partition_size); + } + + std::unique_ptr table; + + PartitionedFilterBlockReader* NewReader( + PartitionedFilterBlockBuilder* builder) { + BlockHandle bh; + Status status; + Slice slice; + do { + slice = builder->Finish(bh, &status); + bh = Write(slice); + } while (status.IsIncomplete()); + const Options options; + const ImmutableCFOptions ioptions(options); + const EnvOptions env_options; + table.reset(new MockedBlockBasedTable(new BlockBasedTable::Rep( + ioptions, env_options, table_options_, icomp, false))); + auto reader = new PartitionedFilterBlockReader( + nullptr, true, BlockContents(slice, false, kNoCompression), nullptr, + nullptr, *icomp.user_comparator(), table.get()); + return reader; + } + + void VerifyReader(PartitionedFilterBlockBuilder* builder, + bool empty = false) { + std::unique_ptr reader(NewReader(builder)); + // Querying added keys + const bool no_io = true; + for (auto key : keys) { + auto ikey = InternalKey(key, 0, ValueType::kTypeValue); + const Slice ikey_slice = Slice(*ikey.rep()); + ASSERT_TRUE(reader->KeyMayMatch(key, kNotValid, !no_io, &ikey_slice)); + } + { + // querying a key twice + auto ikey = InternalKey(keys[0], 0, ValueType::kTypeValue); + const Slice ikey_slice = Slice(*ikey.rep()); + ASSERT_TRUE(reader->KeyMayMatch(keys[0], kNotValid, !no_io, &ikey_slice)); + } + // querying missing keys + for (auto key : missing_keys) { + auto ikey = InternalKey(key, 0, ValueType::kTypeValue); + const Slice ikey_slice = Slice(*ikey.rep()); + if (empty) { + ASSERT_TRUE(reader->KeyMayMatch(key, kNotValid, !no_io, &ikey_slice)); + } else { + // assuming a good hash function + ASSERT_FALSE(reader->KeyMayMatch(key, kNotValid, !no_io, &ikey_slice)); + } + } + } + + int TestBlockPerKey() { + std::unique_ptr pib(NewIndexBuilder()); + std::unique_ptr builder( + NewBuilder(pib.get())); + int i = 0; + builder->Add(keys[i]); + CutABlock(pib.get(), keys[i], keys[i + 1]); + i++; + builder->Add(keys[i]); + CutABlock(pib.get(), keys[i], keys[i + 1]); + i++; + builder->Add(keys[i]); + builder->Add(keys[i]); + CutABlock(pib.get(), keys[i], keys[i + 1]); + i++; + builder->Add(keys[i]); + CutABlock(pib.get(), keys[i]); + + VerifyReader(builder.get()); + return CountNumOfIndexPartitions(pib.get()); + } + + void TestBlockPerTwoKeys() { + std::unique_ptr pib(NewIndexBuilder()); + std::unique_ptr builder( + NewBuilder(pib.get())); + int i = 0; + builder->Add(keys[i]); + i++; + builder->Add(keys[i]); + CutABlock(pib.get(), keys[i], keys[i + 1]); + i++; + builder->Add(keys[i]); + builder->Add(keys[i]); + i++; + builder->Add(keys[i]); + CutABlock(pib.get(), keys[i]); + + VerifyReader(builder.get()); + } + + void TestBlockPerAllKeys() { + std::unique_ptr pib(NewIndexBuilder()); + std::unique_ptr builder( + NewBuilder(pib.get())); + int i = 0; + builder->Add(keys[i]); + i++; + builder->Add(keys[i]); + i++; + builder->Add(keys[i]); + builder->Add(keys[i]); + i++; + builder->Add(keys[i]); + CutABlock(pib.get(), keys[i]); + + VerifyReader(builder.get()); + } + + void CutABlock(PartitionedIndexBuilder* builder, + const std::string& user_key) { + // Assuming a block is cut, add an entry to the index + std::string key = + std::string(*InternalKey(user_key, 0, ValueType::kTypeValue).rep()); + BlockHandle dont_care_block_handle(1, 1); + builder->AddIndexEntry(&key, nullptr, dont_care_block_handle); + } + + void CutABlock(PartitionedIndexBuilder* builder, const std::string& user_key, + const std::string& next_user_key) { + // Assuming a block is cut, add an entry to the index + std::string key = + std::string(*InternalKey(user_key, 0, ValueType::kTypeValue).rep()); + std::string next_key = std::string( + *InternalKey(next_user_key, 0, ValueType::kTypeValue).rep()); + BlockHandle dont_care_block_handle(1, 1); + Slice slice = Slice(next_key.data(), next_key.size()); + builder->AddIndexEntry(&key, &slice, dont_care_block_handle); + } + + int CountNumOfIndexPartitions(PartitionedIndexBuilder* builder) { + IndexBuilder::IndexBlocks dont_care_ib; + BlockHandle dont_care_bh(10, 10); + Status s; + int cnt = 0; + do { + s = builder->Finish(&dont_care_ib, dont_care_bh); + cnt++; + } while (s.IsIncomplete()); + return cnt - 1; // 1 is 2nd level index + } +}; + +TEST_F(PartitionedFilterBlockTest, EmptyBuilder) { + std::unique_ptr pib(NewIndexBuilder()); + std::unique_ptr builder(NewBuilder(pib.get())); + const bool empty = true; + VerifyReader(builder.get(), empty); +} + +TEST_F(PartitionedFilterBlockTest, OneBlock) { + uint64_t max_index_size = MaxIndexSize(); + for (uint64_t i = 1; i < max_index_size + 1; i++) { + table_options_.metadata_block_size = i; + TestBlockPerAllKeys(); + } +} + +TEST_F(PartitionedFilterBlockTest, TwoBlocksPerKey) { + uint64_t max_index_size = MaxIndexSize(); + for (uint64_t i = 1; i < max_index_size + 1; i++) { + table_options_.metadata_block_size = i; + TestBlockPerTwoKeys(); + } +} + +TEST_F(PartitionedFilterBlockTest, OneBlockPerKey) { + uint64_t max_index_size = MaxIndexSize(); + for (uint64_t i = 1; i < max_index_size + 1; i++) { + table_options_.metadata_block_size = i; + TestBlockPerKey(); + } +} + +TEST_F(PartitionedFilterBlockTest, PartitionCount) { + int num_keys = sizeof(keys) / sizeof(*keys); + table_options_.metadata_block_size = + std::max(MaxIndexSize(), MaxFilterSize()); + int partitions = TestBlockPerKey(); + ASSERT_EQ(partitions, 1); + // A low number ensures cutting a block after each key + table_options_.metadata_block_size = 1; + partitions = TestBlockPerKey(); + ASSERT_EQ(partitions, num_keys - 1 /* last two keys make one flush */); +} + +} // namespace rocksdb + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/table/persistent_cache_helper.cc b/table/persistent_cache_helper.cc new file mode 100644 index 00000000000..ec1cac0b9db --- /dev/null +++ b/table/persistent_cache_helper.cc @@ -0,0 +1,114 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "table/persistent_cache_helper.h" +#include "table/block_based_table_reader.h" +#include "table/format.h" + +namespace rocksdb { + +void PersistentCacheHelper::InsertRawPage( + const PersistentCacheOptions& cache_options, const BlockHandle& handle, + const char* data, const size_t size) { + assert(cache_options.persistent_cache); + assert(cache_options.persistent_cache->IsCompressed()); + + // construct the page key + char cache_key[BlockBasedTable::kMaxCacheKeyPrefixSize + kMaxVarint64Length]; + auto key = BlockBasedTable::GetCacheKey(cache_options.key_prefix.c_str(), + cache_options.key_prefix.size(), + handle, cache_key); + // insert content to cache + cache_options.persistent_cache->Insert(key, data, size); +} + +void PersistentCacheHelper::InsertUncompressedPage( + const PersistentCacheOptions& cache_options, const BlockHandle& handle, + const BlockContents& contents) { + assert(cache_options.persistent_cache); + assert(!cache_options.persistent_cache->IsCompressed()); + if (!contents.cachable || contents.compression_type != kNoCompression) { + // We shouldn't cache this. Either + // (1) content is not cacheable + // (2) content is compressed + return; + } + + // construct the page key + char cache_key[BlockBasedTable::kMaxCacheKeyPrefixSize + kMaxVarint64Length]; + auto key = BlockBasedTable::GetCacheKey(cache_options.key_prefix.c_str(), + cache_options.key_prefix.size(), + handle, cache_key); + // insert block contents to page cache + cache_options.persistent_cache->Insert(key, contents.data.data(), + contents.data.size()); +} + +Status PersistentCacheHelper::LookupRawPage( + const PersistentCacheOptions& cache_options, const BlockHandle& handle, + std::unique_ptr* raw_data, const size_t raw_data_size) { + assert(cache_options.persistent_cache); + assert(cache_options.persistent_cache->IsCompressed()); + + // construct the page key + char cache_key[BlockBasedTable::kMaxCacheKeyPrefixSize + kMaxVarint64Length]; + auto key = BlockBasedTable::GetCacheKey(cache_options.key_prefix.c_str(), + cache_options.key_prefix.size(), + handle, cache_key); + // Lookup page + size_t size; + Status s = cache_options.persistent_cache->Lookup(key, raw_data, &size); + if (!s.ok()) { + // cache miss + RecordTick(cache_options.statistics, PERSISTENT_CACHE_MISS); + return s; + } + + // cache hit + assert(raw_data_size == handle.size() + kBlockTrailerSize); + assert(size == raw_data_size); + RecordTick(cache_options.statistics, PERSISTENT_CACHE_HIT); + return Status::OK(); +} + +Status PersistentCacheHelper::LookupUncompressedPage( + const PersistentCacheOptions& cache_options, const BlockHandle& handle, + BlockContents* contents) { + assert(cache_options.persistent_cache); + assert(!cache_options.persistent_cache->IsCompressed()); + if (!contents) { + // We shouldn't lookup in the cache. Either + // (1) Nowhere to store + return Status::NotFound(); + } + + // construct the page key + char cache_key[BlockBasedTable::kMaxCacheKeyPrefixSize + kMaxVarint64Length]; + auto key = BlockBasedTable::GetCacheKey(cache_options.key_prefix.c_str(), + cache_options.key_prefix.size(), + handle, cache_key); + // Lookup page + std::unique_ptr data; + size_t size; + Status s = cache_options.persistent_cache->Lookup(key, &data, &size); + if (!s.ok()) { + // cache miss + RecordTick(cache_options.statistics, PERSISTENT_CACHE_MISS); + return s; + } + + // please note we are potentially comparing compressed data size with + // uncompressed data size + assert(handle.size() <= size); + + // update stats + RecordTick(cache_options.statistics, PERSISTENT_CACHE_HIT); + // construct result and return + *contents = + BlockContents(std::move(data), size, false /*cacheable*/, kNoCompression); + return Status::OK(); +} + +} // namespace rocksdb diff --git a/table/persistent_cache_helper.h b/table/persistent_cache_helper.h new file mode 100644 index 00000000000..ac8ee0389bb --- /dev/null +++ b/table/persistent_cache_helper.h @@ -0,0 +1,44 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once + +#include + +#include "monitoring/statistics.h" +#include "table/format.h" +#include "table/persistent_cache_options.h" + +namespace rocksdb { + +struct BlockContents; + +// PersistentCacheHelper +// +// Encapsulates some of the helper logic for read and writing from the cache +class PersistentCacheHelper { + public: + // insert block into raw page cache + static void InsertRawPage(const PersistentCacheOptions& cache_options, + const BlockHandle& handle, const char* data, + const size_t size); + + // insert block into uncompressed cache + static void InsertUncompressedPage( + const PersistentCacheOptions& cache_options, const BlockHandle& handle, + const BlockContents& contents); + + // lookup block from raw page cacge + static Status LookupRawPage(const PersistentCacheOptions& cache_options, + const BlockHandle& handle, + std::unique_ptr* raw_data, + const size_t raw_data_size); + + // lookup block from uncompressed cache + static Status LookupUncompressedPage( + const PersistentCacheOptions& cache_options, const BlockHandle& handle, + BlockContents* contents); +}; + +} // namespace rocksdb diff --git a/table/persistent_cache_options.h b/table/persistent_cache_options.h new file mode 100644 index 00000000000..acd640369ad --- /dev/null +++ b/table/persistent_cache_options.h @@ -0,0 +1,34 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +#pragma once + +#include + +#include "monitoring/statistics.h" +#include "rocksdb/persistent_cache.h" + +namespace rocksdb { + +// PersistentCacheOptions +// +// This describe the caching behavior for page cache +// This is used to pass the context for caching and the cache handle +struct PersistentCacheOptions { + PersistentCacheOptions() {} + explicit PersistentCacheOptions( + const std::shared_ptr& _persistent_cache, + const std::string _key_prefix, Statistics* const _statistics) + : persistent_cache(_persistent_cache), + key_prefix(_key_prefix), + statistics(_statistics) {} + + virtual ~PersistentCacheOptions() {} + + std::shared_ptr persistent_cache; + std::string key_prefix; + Statistics* statistics = nullptr; +}; + +} // namespace rocksdb diff --git a/table/plain_table_builder.cc b/table/plain_table_builder.cc index 4f3b62ad4eb..964804358a6 100644 --- a/table/plain_table_builder.cc +++ b/table/plain_table_builder.cc @@ -1,13 +1,15 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef ROCKSDB_LITE #include "table/plain_table_builder.h" -#include #include + +#include +#include #include #include "rocksdb/comparator.h" @@ -20,11 +22,11 @@ #include "table/block_builder.h" #include "table/bloom_block.h" #include "table/plain_table_index.h" -#include "table/filter_block.h" #include "table/format.h" #include "table/meta_blocks.h" #include "util/coding.h" #include "util/crc32c.h" +#include "util/file_reader_writer.h" #include "util/stop_watch.h" namespace rocksdb { @@ -34,11 +36,8 @@ namespace { // a utility that helps writing block content to the file // @offset will advance if @block_contents was successfully written. // @block_handle the block handle this particular block. -Status WriteBlock( - const Slice& block_contents, - WritableFile* file, - uint64_t* offset, - BlockHandle* block_handle) { +Status WriteBlock(const Slice& block_contents, WritableFileWriter* file, + uint64_t* offset, BlockHandle* block_handle) { block_handle->set_offset(*offset); block_handle->set_size(block_contents.size()); Status s = file->Append(block_contents); @@ -58,26 +57,29 @@ extern const uint64_t kPlainTableMagicNumber = 0x8242229663bf9564ull; extern const uint64_t kLegacyPlainTableMagicNumber = 0x4f3418eb7a8f13b8ull; PlainTableBuilder::PlainTableBuilder( - const Options& options, WritableFile* file, uint32_t user_key_len, + const ImmutableCFOptions& ioptions, + const std::vector>* + int_tbl_prop_collector_factories, + uint32_t column_family_id, WritableFileWriter* file, uint32_t user_key_len, EncodingType encoding_type, size_t index_sparseness, - uint32_t bloom_bits_per_key, uint32_t num_probes, size_t huge_page_tlb_size, - double hash_table_ratio, bool store_index_in_file) - : options_(options), + uint32_t bloom_bits_per_key, const std::string& column_family_name, + uint32_t num_probes, size_t huge_page_tlb_size, double hash_table_ratio, + bool store_index_in_file) + : ioptions_(ioptions), bloom_block_(num_probes), file_(file), bloom_bits_per_key_(bloom_bits_per_key), huge_page_tlb_size_(huge_page_tlb_size), - encoder_(encoding_type, user_key_len, options.prefix_extractor.get(), + encoder_(encoding_type, user_key_len, ioptions.prefix_extractor, index_sparseness), store_index_in_file_(store_index_in_file), - prefix_extractor_(options.prefix_extractor.get()) { + prefix_extractor_(ioptions.prefix_extractor) { // Build index block and save it in the file if hash_table_ratio > 0 if (store_index_in_file_) { assert(hash_table_ratio > 0 || IsTotalOrderMode()); index_builder_.reset( - new PlainTableIndexBuilder(&arena_, options, index_sparseness, + new PlainTableIndexBuilder(&arena_, ioptions, index_sparseness, hash_table_ratio, huge_page_tlb_size_)); - assert(bloom_bits_per_key_ > 0); properties_.user_collected_properties [PlainTablePropertyNames::kBloomVersion] = "1"; // For future use } @@ -92,22 +94,20 @@ PlainTableBuilder::PlainTableBuilder( // To support roll-back to previous version, now still use version 0 for // plain encoding. properties_.format_version = (encoding_type == kPlain) ? 0 : 1; - - if (options_.prefix_extractor) { - properties_.user_collected_properties - [PlainTablePropertyNames::kPrefixExtractorName] = - options_.prefix_extractor->Name(); - } + properties_.column_family_id = column_family_id; + properties_.column_family_name = column_family_name; + properties_.prefix_extractor_name = ioptions_.prefix_extractor != nullptr + ? ioptions_.prefix_extractor->Name() + : "nullptr"; std::string val; PutFixed32(&val, static_cast(encoder_.GetEncodingType())); properties_.user_collected_properties [PlainTablePropertyNames::kEncodingType] = val; - for (auto& collector_factories : - options.table_properties_collector_factories) { + for (auto& collector_factories : *int_tbl_prop_collector_factories) { table_properties_collectors_.emplace_back( - collector_factories->CreateTablePropertiesCollector()); + collector_factories->CreateIntTblPropCollector(column_family_id)); } } @@ -120,21 +120,29 @@ void PlainTableBuilder::Add(const Slice& key, const Slice& value) { size_t meta_bytes_buf_size = 0; ParsedInternalKey internal_key; - ParseInternalKey(key, &internal_key); + if (!ParseInternalKey(key, &internal_key)) { + assert(false); + return; + } + if (internal_key.type == kTypeRangeDeletion) { + status_ = Status::NotSupported("Range deletion unsupported"); + return; + } // Store key hash if (store_index_in_file_) { - if (options_.prefix_extractor.get() == nullptr) { + if (ioptions_.prefix_extractor == nullptr) { keys_or_prefixes_hashes_.push_back(GetSliceHash(internal_key.user_key)); } else { Slice prefix = - options_.prefix_extractor->Transform(internal_key.user_key); + ioptions_.prefix_extractor->Transform(internal_key.user_key); keys_or_prefixes_hashes_.push_back(GetSliceHash(prefix)); } } // Write value - auto prev_offset = offset_; + assert(offset_ <= std::numeric_limits::max()); + auto prev_offset = static_cast(offset_); // Write out the key encoder_.AppendKey(key, file_, &offset_, meta_bytes_buf, &meta_bytes_buf_size); @@ -143,7 +151,7 @@ void PlainTableBuilder::Add(const Slice& key, const Slice& value) { } // Write value length - int value_size = value.size(); + uint32_t value_size = static_cast(value.size()); char* end_ptr = EncodeVarint32(meta_bytes_buf + meta_bytes_buf_size, value_size); assert(end_ptr <= meta_bytes_buf + sizeof(meta_bytes_buf)); @@ -159,8 +167,8 @@ void PlainTableBuilder::Add(const Slice& key, const Slice& value) { properties_.raw_value_size += value.size(); // notify property collectors - NotifyCollectTableCollectorsOnAdd(key, value, table_properties_collectors_, - options_.info_log.get()); + NotifyCollectTableCollectorsOnAdd( + key, value, offset_, table_properties_collectors_, ioptions_.info_log); } Status PlainTableBuilder::status() const { return status_; } @@ -181,36 +189,41 @@ Status PlainTableBuilder::Finish() { MetaIndexBuilder meta_index_builer; if (store_index_in_file_ && (properties_.num_entries > 0)) { - bloom_block_.SetTotalBits( - &arena_, properties_.num_entries * bloom_bits_per_key_, - options_.bloom_locality, huge_page_tlb_size_, options_.info_log.get()); + assert(properties_.num_entries <= std::numeric_limits::max()); + Status s; + BlockHandle bloom_block_handle; + if (bloom_bits_per_key_ > 0) { + bloom_block_.SetTotalBits( + &arena_, + static_cast(properties_.num_entries) * bloom_bits_per_key_, + ioptions_.bloom_locality, huge_page_tlb_size_, ioptions_.info_log); - PutVarint32(&properties_.user_collected_properties - [PlainTablePropertyNames::kNumBloomBlocks], - bloom_block_.GetNumBlocks()); + PutVarint32(&properties_.user_collected_properties + [PlainTablePropertyNames::kNumBloomBlocks], + bloom_block_.GetNumBlocks()); - bloom_block_.AddKeysHashes(keys_or_prefixes_hashes_); - BlockHandle bloom_block_handle; - auto finish_result = bloom_block_.Finish(); + bloom_block_.AddKeysHashes(keys_or_prefixes_hashes_); - properties_.filter_size = finish_result.size(); - auto s = WriteBlock(finish_result, file_, &offset_, &bloom_block_handle); + Slice bloom_finish_result = bloom_block_.Finish(); - if (!s.ok()) { - return s; - } + properties_.filter_size = bloom_finish_result.size(); + s = WriteBlock(bloom_finish_result, file_, &offset_, &bloom_block_handle); + if (!s.ok()) { + return s; + } + meta_index_builer.Add(BloomBlockBuilder::kBloomBlock, bloom_block_handle); + } BlockHandle index_block_handle; - finish_result = index_builder_->Finish(); + Slice index_finish_result = index_builder_->Finish(); - properties_.index_size = finish_result.size(); - s = WriteBlock(finish_result, file_, &offset_, &index_block_handle); + properties_.index_size = index_finish_result.size(); + s = WriteBlock(index_finish_result, file_, &offset_, &index_block_handle); if (!s.ok()) { return s; } - meta_index_builer.Add(BloomBlockBuilder::kBloomBlock, bloom_block_handle); meta_index_builer.Add(PlainTableIndexBuilder::kPlainTableIndexBlock, index_block_handle); } @@ -224,7 +237,7 @@ Status PlainTableBuilder::Finish() { // -- Add user collected properties NotifyCollectTableCollectorsOnFinish(table_properties_collectors_, - options_.info_log.get(), + ioptions_.info_log, &property_block_builder); // -- Write property block @@ -254,7 +267,7 @@ Status PlainTableBuilder::Finish() { // Write Footer // no need to write out new footer if we're using default checksum - Footer footer(kLegacyPlainTableMagicNumber); + Footer footer(kLegacyPlainTableMagicNumber, 0); footer.set_metaindex_handle(metaindex_block_handle); footer.set_index_handle(BlockHandle::NullBlockHandle()); std::string footer_encoding; diff --git a/table/plain_table_builder.h b/table/plain_table_builder.h index 2871d887e82..1d1f6c7586e 100644 --- a/table/plain_table_builder.h +++ b/table/plain_table_builder.h @@ -1,20 +1,21 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #ifndef ROCKSDB_LITE #include +#include #include #include "rocksdb/options.h" #include "rocksdb/status.h" -#include "table/table_builder.h" -#include "table/plain_table_key_coding.h" #include "rocksdb/table.h" #include "rocksdb/table_properties.h" #include "table/bloom_block.h" #include "table/plain_table_index.h" +#include "table/plain_table_key_coding.h" +#include "table/table_builder.h" namespace rocksdb { @@ -30,12 +31,16 @@ class PlainTableBuilder: public TableBuilder { // caller to close the file after calling Finish(). The output file // will be part of level specified by 'level'. A value of -1 means // that the caller does not know which level the output file will reside. - PlainTableBuilder(const Options& options, WritableFile* file, - uint32_t user_key_size, EncodingType encoding_type, - size_t index_sparseness, uint32_t bloom_bits_per_key, - uint32_t num_probes = 6, size_t huge_page_tlb_size = 0, - double hash_table_ratio = 0, - bool store_index_in_file = false); + PlainTableBuilder( + const ImmutableCFOptions& ioptions, + const std::vector>* + int_tbl_prop_collector_factories, + uint32_t column_family_id, WritableFileWriter* file, + uint32_t user_key_size, EncodingType encoding_type, + size_t index_sparseness, uint32_t bloom_bits_per_key, + const std::string& column_family_name, uint32_t num_probes = 6, + size_t huge_page_tlb_size = 0, double hash_table_ratio = 0, + bool store_index_in_file = false); // REQUIRES: Either Finish() or Abandon() has been called. ~PlainTableBuilder(); @@ -67,21 +72,23 @@ class PlainTableBuilder: public TableBuilder { // Finish() call, returns the size of the final generated file. uint64_t FileSize() const override; + TableProperties GetTableProperties() const override { return properties_; } + bool SaveIndexInFile() const { return store_index_in_file_; } private: Arena arena_; - Options options_; - std::vector> + const ImmutableCFOptions& ioptions_; + std::vector> table_properties_collectors_; BloomBlockBuilder bloom_block_; std::unique_ptr index_builder_; - WritableFile* file_; + WritableFileWriter* file_; uint64_t offset_ = 0; uint32_t bloom_bits_per_key_; - uint32_t huge_page_tlb_size_; + size_t huge_page_tlb_size_; Status status_; TableProperties properties_; PlainTableKeyEncoder encoder_; diff --git a/table/plain_table_factory.cc b/table/plain_table_factory.cc index 145179bae1c..5f7809b967d 100644 --- a/table/plain_table_factory.cc +++ b/table/plain_table_factory.cc @@ -5,34 +5,46 @@ #ifndef ROCKSDB_LITE #include "table/plain_table_factory.h" -#include #include +#include #include "db/dbformat.h" +#include "options/options_helper.h" +#include "port/port.h" +#include "rocksdb/convenience.h" #include "table/plain_table_builder.h" #include "table/plain_table_reader.h" -#include "port/port.h" +#include "util/string_util.h" namespace rocksdb { -Status PlainTableFactory::NewTableReader(const Options& options, - const EnvOptions& soptions, - const InternalKeyComparator& icomp, - unique_ptr&& file, - uint64_t file_size, - unique_ptr* table) const { - return PlainTableReader::Open(options, soptions, icomp, std::move(file), - file_size, table, bloom_bits_per_key_, - hash_table_ratio_, index_sparseness_, - huge_page_tlb_size_, full_scan_mode_); +Status PlainTableFactory::NewTableReader( + const TableReaderOptions& table_reader_options, + unique_ptr&& file, uint64_t file_size, + unique_ptr* table, + bool prefetch_index_and_filter_in_cache) const { + return PlainTableReader::Open( + table_reader_options.ioptions, table_reader_options.env_options, + table_reader_options.internal_comparator, std::move(file), file_size, + table, table_options_.bloom_bits_per_key, table_options_.hash_table_ratio, + table_options_.index_sparseness, table_options_.huge_page_tlb_size, + table_options_.full_scan_mode); } TableBuilder* PlainTableFactory::NewTableBuilder( - const Options& options, const InternalKeyComparator& internal_comparator, - WritableFile* file, CompressionType compression_type) const { - return new PlainTableBuilder(options, file, user_key_len_, encoding_type_, - index_sparseness_, bloom_bits_per_key_, 6, - huge_page_tlb_size_, hash_table_ratio_, - store_index_in_file_); + const TableBuilderOptions& table_builder_options, uint32_t column_family_id, + WritableFileWriter* file) const { + // Ignore the skip_filters flag. PlainTable format is optimized for small + // in-memory dbs. The skip_filters optimization is not useful for plain + // tables + // + return new PlainTableBuilder( + table_builder_options.ioptions, + table_builder_options.int_tbl_prop_collector_factories, column_family_id, + file, table_options_.user_key_len, table_options_.encoding_type, + table_options_.index_sparseness, table_options_.bloom_bits_per_key, + table_builder_options.column_family_name, 6, + table_options_.huge_page_tlb_size, table_options_.hash_table_ratio, + table_options_.store_index_in_file); } std::string PlainTableFactory::GetPrintableTableOptions() const { @@ -42,39 +54,177 @@ std::string PlainTableFactory::GetPrintableTableOptions() const { char buffer[kBufferSize]; snprintf(buffer, kBufferSize, " user_key_len: %u\n", - user_key_len_); + table_options_.user_key_len); ret.append(buffer); snprintf(buffer, kBufferSize, " bloom_bits_per_key: %d\n", - bloom_bits_per_key_); + table_options_.bloom_bits_per_key); ret.append(buffer); snprintf(buffer, kBufferSize, " hash_table_ratio: %lf\n", - hash_table_ratio_); + table_options_.hash_table_ratio); ret.append(buffer); - snprintf(buffer, kBufferSize, " index_sparseness: %zd\n", - index_sparseness_); + snprintf(buffer, kBufferSize, " index_sparseness: %" ROCKSDB_PRIszt "\n", + table_options_.index_sparseness); ret.append(buffer); - snprintf(buffer, kBufferSize, " huge_page_tlb_size: %zd\n", - huge_page_tlb_size_); + snprintf(buffer, kBufferSize, " huge_page_tlb_size: %" ROCKSDB_PRIszt "\n", + table_options_.huge_page_tlb_size); ret.append(buffer); snprintf(buffer, kBufferSize, " encoding_type: %d\n", - encoding_type_); + table_options_.encoding_type); ret.append(buffer); snprintf(buffer, kBufferSize, " full_scan_mode: %d\n", - full_scan_mode_); + table_options_.full_scan_mode); ret.append(buffer); snprintf(buffer, kBufferSize, " store_index_in_file: %d\n", - store_index_in_file_); + table_options_.store_index_in_file); ret.append(buffer); return ret; } +const PlainTableOptions& PlainTableFactory::table_options() const { + return table_options_; +} + +Status GetPlainTableOptionsFromString(const PlainTableOptions& table_options, + const std::string& opts_str, + PlainTableOptions* new_table_options) { + std::unordered_map opts_map; + Status s = StringToMap(opts_str, &opts_map); + if (!s.ok()) { + return s; + } + return GetPlainTableOptionsFromMap(table_options, opts_map, + new_table_options); +} + +Status GetMemTableRepFactoryFromString( + const std::string& opts_str, + std::unique_ptr* new_mem_factory) { + std::vector opts_list = StringSplit(opts_str, ':'); + size_t len = opts_list.size(); + + if (opts_list.size() <= 0 || opts_list.size() > 2) { + return Status::InvalidArgument("Can't parse memtable_factory option ", + opts_str); + } + + MemTableRepFactory* mem_factory = nullptr; + + if (opts_list[0] == "skip_list") { + // Expecting format + // skip_list: + if (2 == len) { + size_t lookahead = ParseSizeT(opts_list[1]); + mem_factory = new SkipListFactory(lookahead); + } else if (1 == len) { + mem_factory = new SkipListFactory(); + } + } else if (opts_list[0] == "prefix_hash") { + // Expecting format + // prfix_hash: + if (2 == len) { + size_t hash_bucket_count = ParseSizeT(opts_list[1]); + mem_factory = NewHashSkipListRepFactory(hash_bucket_count); + } else if (1 == len) { + mem_factory = NewHashSkipListRepFactory(); + } + } else if (opts_list[0] == "hash_linkedlist") { + // Expecting format + // hash_linkedlist: + if (2 == len) { + size_t hash_bucket_count = ParseSizeT(opts_list[1]); + mem_factory = NewHashLinkListRepFactory(hash_bucket_count); + } else if (1 == len) { + mem_factory = NewHashLinkListRepFactory(); + } + } else if (opts_list[0] == "vector") { + // Expecting format + // vector: + if (2 == len) { + size_t count = ParseSizeT(opts_list[1]); + mem_factory = new VectorRepFactory(count); + } else if (1 == len) { + mem_factory = new VectorRepFactory(); + } + } else if (opts_list[0] == "cuckoo") { + // Expecting format + // cuckoo: + if (2 == len) { + size_t write_buffer_size = ParseSizeT(opts_list[1]); + mem_factory = NewHashCuckooRepFactory(write_buffer_size); + } else if (1 == len) { + return Status::InvalidArgument("Can't parse memtable_factory option ", + opts_str); + } + } else { + return Status::InvalidArgument("Unrecognized memtable_factory option ", + opts_str); + } + + if (mem_factory != nullptr) { + new_mem_factory->reset(mem_factory); + } + + return Status::OK(); +} + +std::string ParsePlainTableOptions(const std::string& name, + const std::string& org_value, + PlainTableOptions* new_options, + bool input_strings_escaped = false, + bool ignore_unknown_options = false) { + const std::string& value = + input_strings_escaped ? UnescapeOptionString(org_value) : org_value; + const auto iter = plain_table_type_info.find(name); + if (iter == plain_table_type_info.end()) { + if (ignore_unknown_options) { + return ""; + } else { + return "Unrecognized option"; + } + } + const auto& opt_info = iter->second; + if (opt_info.verification != OptionVerificationType::kDeprecated && + !ParseOptionHelper(reinterpret_cast(new_options) + opt_info.offset, + opt_info.type, value)) { + return "Invalid value"; + } + return ""; +} + +Status GetPlainTableOptionsFromMap( + const PlainTableOptions& table_options, + const std::unordered_map& opts_map, + PlainTableOptions* new_table_options, bool input_strings_escaped, + bool ignore_unknown_options) { + assert(new_table_options); + *new_table_options = table_options; + for (const auto& o : opts_map) { + auto error_message = ParsePlainTableOptions( + o.first, o.second, new_table_options, input_strings_escaped); + if (error_message != "") { + const auto iter = plain_table_type_info.find(o.first); + if (iter == plain_table_type_info.end() || + !input_strings_escaped || // !input_strings_escaped indicates + // the old API, where everything is + // parsable. + (iter->second.verification != OptionVerificationType::kByName && + iter->second.verification != + OptionVerificationType::kByNameAllowNull && + iter->second.verification != OptionVerificationType::kDeprecated)) { + // Restore "new_options" to the default "base_options". + *new_table_options = table_options; + return Status::InvalidArgument("Can't parse PlainTableOptions:", + o.first + " " + error_message); + } + } + } + return Status::OK(); +} + extern TableFactory* NewPlainTableFactory(const PlainTableOptions& options) { return new PlainTableFactory(options); } -const std::string PlainTablePropertyNames::kPrefixExtractorName = - "rocksdb.prefix.extractor.name"; - const std::string PlainTablePropertyNames::kEncodingType = "rocksdb.plain.table.encoding.type"; diff --git a/table/plain_table_factory.h b/table/plain_table_factory.h index d1cf0cae61a..6c9ca44f30e 100644 --- a/table/plain_table_factory.h +++ b/table/plain_table_factory.h @@ -9,12 +9,12 @@ #include #include +#include "options/options_helper.h" #include "rocksdb/options.h" #include "rocksdb/table.h" namespace rocksdb { -struct Options; struct EnvOptions; using std::unique_ptr; @@ -128,7 +128,7 @@ class TableBuilder; class PlainTableFactory : public TableFactory { public: ~PlainTableFactory() {} - // user_key_size is the length of the user key. If it is set to be + // user_key_len is the length of the user key. If it is set to be // kPlainTableVariableLength, then it means variable length. Otherwise, all // the keys need to have the fix length of this value. bloom_bits_per_key is // number of bits used for bloom filer per key. hash_table_ratio is @@ -143,50 +143,68 @@ class PlainTableFactory : public TableFactory { // huge_page_tlb_size determines whether to allocate hash indexes from huge // page TLB and the page size if allocating from there. See comments of // Arena::AllocateAligned() for details. - explicit PlainTableFactory(const PlainTableOptions& options = - PlainTableOptions()) - : user_key_len_(options.user_key_len), - bloom_bits_per_key_(options.bloom_bits_per_key), - hash_table_ratio_(options.hash_table_ratio), - index_sparseness_(options.index_sparseness), - huge_page_tlb_size_(options.huge_page_tlb_size), - encoding_type_(options.encoding_type), - full_scan_mode_(options.full_scan_mode), - store_index_in_file_(options.store_index_in_file) {} + explicit PlainTableFactory( + const PlainTableOptions& _table_options = PlainTableOptions()) + : table_options_(_table_options) {} + const char* Name() const override { return "PlainTable"; } - Status NewTableReader(const Options& options, const EnvOptions& soptions, - const InternalKeyComparator& internal_comparator, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table) const override; - TableBuilder* NewTableBuilder(const Options& options, - const InternalKeyComparator& icomparator, - WritableFile* file, - CompressionType compression_type) const - override; + Status NewTableReader(const TableReaderOptions& table_reader_options, + unique_ptr&& file, + uint64_t file_size, unique_ptr* table, + bool prefetch_index_and_filter_in_cache) const override; + + TableBuilder* NewTableBuilder( + const TableBuilderOptions& table_builder_options, + uint32_t column_family_id, WritableFileWriter* file) const override; std::string GetPrintableTableOptions() const override; - static const char kValueTypeSeqId0 = 0xFF; + const PlainTableOptions& table_options() const; + + static const char kValueTypeSeqId0 = char(0xFF); // Sanitizes the specified DB Options. - Status SanitizeDBOptions(const DBOptions* db_opts) const override { - if (db_opts->allow_mmap_reads == false) { - return Status::NotSupported( - "PlainTable with allow_mmap_reads == false is not supported."); - } + Status SanitizeOptions(const DBOptions& db_opts, + const ColumnFamilyOptions& cf_opts) const override { + return Status::OK(); + } + + void* GetOptions() override { return &table_options_; } + + Status GetOptionString(std::string* opt_string, + const std::string& delimiter) const override { return Status::OK(); } private: - uint32_t user_key_len_; - int bloom_bits_per_key_; - double hash_table_ratio_; - size_t index_sparseness_; - size_t huge_page_tlb_size_; - EncodingType encoding_type_; - bool full_scan_mode_; - bool store_index_in_file_; + PlainTableOptions table_options_; }; +static std::unordered_map plain_table_type_info = { + {"user_key_len", + {offsetof(struct PlainTableOptions, user_key_len), OptionType::kUInt32T, + OptionVerificationType::kNormal, false, 0}}, + {"bloom_bits_per_key", + {offsetof(struct PlainTableOptions, bloom_bits_per_key), OptionType::kInt, + OptionVerificationType::kNormal, false, 0}}, + {"hash_table_ratio", + {offsetof(struct PlainTableOptions, hash_table_ratio), OptionType::kDouble, + OptionVerificationType::kNormal, false, 0}}, + {"index_sparseness", + {offsetof(struct PlainTableOptions, index_sparseness), OptionType::kSizeT, + OptionVerificationType::kNormal, false, 0}}, + {"huge_page_tlb_size", + {offsetof(struct PlainTableOptions, huge_page_tlb_size), + OptionType::kSizeT, OptionVerificationType::kNormal, false, 0}}, + {"encoding_type", + {offsetof(struct PlainTableOptions, encoding_type), + OptionType::kEncodingType, OptionVerificationType::kByName, false, 0}}, + {"full_scan_mode", + {offsetof(struct PlainTableOptions, full_scan_mode), OptionType::kBoolean, + OptionVerificationType::kNormal, false, 0}}, + {"store_index_in_file", + {offsetof(struct PlainTableOptions, store_index_in_file), + OptionType::kBoolean, OptionVerificationType::kNormal, false, 0}}}; + } // namespace rocksdb #endif // ROCKSDB_LITE diff --git a/table/plain_table_index.cc b/table/plain_table_index.cc index efba9b71dac..39a6b53d602 100644 --- a/table/plain_table_index.cc +++ b/table/plain_table_index.cc @@ -1,7 +1,15 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#include #include "table/plain_table_index.h" #include "util/coding.h" @@ -24,7 +32,8 @@ Status PlainTableIndex::InitFromRawData(Slice data) { if (!GetVarint32(&data, &num_prefixes_)) { return Status::Corruption("Couldn't read the index size!"); } - sub_index_size_ = data.size() - index_size_ * kOffsetLen; + sub_index_size_ = + static_cast(data.size()) - index_size_ * kOffsetLen; char* index_data_begin = const_cast(data.data()); index_ = reinterpret_cast(index_data_begin); @@ -35,7 +44,7 @@ Status PlainTableIndex::InitFromRawData(Slice data) { PlainTableIndex::IndexSearchResult PlainTableIndex::GetOffset( uint32_t prefix_hash, uint32_t* bucket_value) const { int bucket = GetBucketIdFromHash(prefix_hash, index_size_); - *bucket_value = index_[bucket]; + GetUnaligned(index_ + bucket, bucket_value); if ((*bucket_value & kSubIndexMask) == kSubIndexMask) { *bucket_value ^= kSubIndexMask; return kSubindex; @@ -48,7 +57,7 @@ PlainTableIndex::IndexSearchResult PlainTableIndex::GetOffset( } } -void PlainTableIndexBuilder::IndexRecordList::AddRecord(murmur_t hash, +void PlainTableIndexBuilder::IndexRecordList::AddRecord(uint32_t hash, uint32_t offset) { if (num_records_in_current_group_ == kNumRecordsPerGroup) { current_group_ = AllocateNewGroup(); @@ -61,7 +70,7 @@ void PlainTableIndexBuilder::IndexRecordList::AddRecord(murmur_t hash, } void PlainTableIndexBuilder::AddKeyPrefix(Slice key_prefix_slice, - uint64_t key_offset) { + uint32_t key_offset) { if (is_first_record_ || prev_key_prefix_ != key_prefix_slice.ToString()) { ++num_prefixes_; if (!is_first_record_) { @@ -93,8 +102,8 @@ Slice PlainTableIndexBuilder::Finish() { BucketizeIndexes(&hash_to_offsets, &entries_per_bucket); keys_per_prefix_hist_.Add(num_keys_per_prefix_); - Log(options_.info_log, "Number of Keys per prefix Histogram: %s", - keys_per_prefix_hist_.ToString().c_str()); + ROCKS_LOG_INFO(ioptions_.info_log, "Number of Keys per prefix Histogram: %s", + keys_per_prefix_hist_.ToString().c_str()); // From the temp data structure, populate indexes. return FillIndexes(hash_to_offsets, entries_per_bucket); @@ -107,7 +116,8 @@ void PlainTableIndexBuilder::AllocateIndex() { index_size_ = 1; } else { double hash_table_size_multipier = 1.0 / hash_table_ratio_; - index_size_ = num_prefixes_ * hash_table_size_multipier + 1; + index_size_ = + static_cast(num_prefixes_ * hash_table_size_multipier) + 1; assert(index_size_ > 0); } } @@ -147,35 +157,36 @@ void PlainTableIndexBuilder::BucketizeIndexes( Slice PlainTableIndexBuilder::FillIndexes( const std::vector& hash_to_offsets, const std::vector& entries_per_bucket) { - Log(options_.info_log, "Reserving %zu bytes for plain table's sub_index", - sub_index_size_); + ROCKS_LOG_DEBUG(ioptions_.info_log, + "Reserving %" PRIu32 " bytes for plain table's sub_index", + sub_index_size_); auto total_allocate_size = GetTotalSize(); char* allocated = arena_->AllocateAligned( - total_allocate_size, huge_page_tlb_size_, options_.info_log.get()); + total_allocate_size, huge_page_tlb_size_, ioptions_.info_log); auto temp_ptr = EncodeVarint32(allocated, index_size_); uint32_t* index = reinterpret_cast(EncodeVarint32(temp_ptr, num_prefixes_)); char* sub_index = reinterpret_cast(index + index_size_); - size_t sub_index_offset = 0; + uint32_t sub_index_offset = 0; for (uint32_t i = 0; i < index_size_; i++) { uint32_t num_keys_for_bucket = entries_per_bucket[i]; switch (num_keys_for_bucket) { case 0: // No key for bucket - index[i] = PlainTableIndex::kMaxFileSize; + PutUnaligned(index + i, (uint32_t)PlainTableIndex::kMaxFileSize); break; case 1: // point directly to the file offset - index[i] = hash_to_offsets[i]->offset; + PutUnaligned(index + i, hash_to_offsets[i]->offset); break; default: // point to second level indexes. - index[i] = sub_index_offset | PlainTableIndex::kSubIndexMask; + PutUnaligned(index + i, sub_index_offset | PlainTableIndex::kSubIndexMask); char* prev_ptr = &sub_index[sub_index_offset]; char* cur_ptr = EncodeVarint32(prev_ptr, num_keys_for_bucket); - sub_index_offset += (cur_ptr - prev_ptr); + sub_index_offset += static_cast(cur_ptr - prev_ptr); char* sub_index_pos = &sub_index[sub_index_offset]; IndexRecord* record = hash_to_offsets[i]; int j; @@ -191,11 +202,14 @@ Slice PlainTableIndexBuilder::FillIndexes( } assert(sub_index_offset == sub_index_size_); - Log(options_.info_log, "hash table size: %d, suffix_map length %zu", - index_size_, sub_index_size_); + ROCKS_LOG_DEBUG(ioptions_.info_log, + "hash table size: %d, suffix_map length %" ROCKSDB_PRIszt, + index_size_, sub_index_size_); return Slice(allocated, GetTotalSize()); } const std::string PlainTableIndexBuilder::kPlainTableIndexBlock = "PlainTableIndexBlock"; }; // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/table/plain_table_index.h b/table/plain_table_index.h index f63bbd0d522..2916be4192b 100644 --- a/table/plain_table_index.h +++ b/table/plain_table_index.h @@ -1,19 +1,22 @@ -// Copyright (c) 2014, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once +#ifndef ROCKSDB_LITE + #include #include #include "db/dbformat.h" +#include "monitoring/histogram.h" +#include "options/cf_options.h" #include "rocksdb/options.h" -#include "util/murmurhash.h" -#include "util/hash.h" #include "util/arena.h" -#include "util/histogram.h" +#include "util/hash.h" +#include "util/murmurhash.h" namespace rocksdb { @@ -92,7 +95,7 @@ class PlainTableIndex { private: uint32_t index_size_; - size_t sub_index_size_; + uint32_t sub_index_size_; uint32_t num_prefixes_; uint32_t* index_; @@ -108,11 +111,11 @@ class PlainTableIndex { // #wiki-in-memory-index-format class PlainTableIndexBuilder { public: - PlainTableIndexBuilder(Arena* arena, const Options& options, - uint32_t index_sparseness, double hash_table_ratio, - double huge_page_tlb_size) + PlainTableIndexBuilder(Arena* arena, const ImmutableCFOptions& ioptions, + size_t index_sparseness, double hash_table_ratio, + size_t huge_page_tlb_size) : arena_(arena), - options_(options), + ioptions_(ioptions), record_list_(kRecordsPerGroup), is_first_record_(true), due_index_(false), @@ -120,11 +123,11 @@ class PlainTableIndexBuilder { num_keys_per_prefix_(0), prev_key_prefix_hash_(0), index_sparseness_(index_sparseness), - prefix_extractor_(options.prefix_extractor.get()), + prefix_extractor_(ioptions.prefix_extractor), hash_table_ratio_(hash_table_ratio), huge_page_tlb_size_(huge_page_tlb_size) {} - void AddKeyPrefix(Slice key_prefix_slice, uint64_t key_offset); + void AddKeyPrefix(Slice key_prefix_slice, uint32_t key_offset); Slice Finish(); @@ -156,7 +159,7 @@ class PlainTableIndexBuilder { } } - void AddRecord(murmur_t hash, uint32_t offset); + void AddRecord(uint32_t hash, uint32_t offset); size_t GetNumRecords() const { return (groups_.size() - 1) * kNumRecordsPerGroup + @@ -196,7 +199,7 @@ class PlainTableIndexBuilder { const std::vector& entries_per_bucket); Arena* arena_; - Options options_; + const ImmutableCFOptions ioptions_; HistogramImpl keys_per_prefix_hist_; IndexRecordList record_list_; bool is_first_record_; @@ -205,13 +208,13 @@ class PlainTableIndexBuilder { uint32_t num_keys_per_prefix_; uint32_t prev_key_prefix_hash_; - uint32_t index_sparseness_; + size_t index_sparseness_; uint32_t index_size_; - size_t sub_index_size_; + uint32_t sub_index_size_; const SliceTransform* prefix_extractor_; double hash_table_ratio_; - double huge_page_tlb_size_; + size_t huge_page_tlb_size_; std::string prev_key_prefix_; @@ -219,3 +222,5 @@ class PlainTableIndexBuilder { }; }; // namespace rocksdb + +#endif // ROCKSDB_LITE diff --git a/table/plain_table_key_coding.cc b/table/plain_table_key_coding.cc index eedf58aeaa1..3e87c03d13f 100644 --- a/table/plain_table_key_coding.cc +++ b/table/plain_table_key_coding.cc @@ -1,24 +1,28 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef ROCKSDB_LITE #include "table/plain_table_key_coding.h" -#include "table/plain_table_factory.h" +#include +#include #include "db/dbformat.h" +#include "table/plain_table_reader.h" +#include "table/plain_table_factory.h" +#include "util/file_reader_writer.h" namespace rocksdb { -namespace { - -enum EntryType : unsigned char { +enum PlainTableEntryType : unsigned char { kFullKey = 0, kPrefixFromPreviousKey = 1, kKeySuffix = 2, }; +namespace { + // Control byte: // First two bits indicate type of entry // Other bytes are inlined sizes. If all bits are 1 (0x03F), overflow bytes @@ -27,10 +31,11 @@ enum EntryType : unsigned char { const unsigned char kSizeInlineLimit = 0x3F; // Return 0 for error -size_t EncodeSize(EntryType type, uint32_t key_size, char* out_buffer) { +size_t EncodeSize(PlainTableEntryType type, uint32_t key_size, + char* out_buffer) { out_buffer[0] = type << 6; - if (key_size < 0x3F) { + if (key_size < static_cast(kSizeInlineLimit)) { // size inlined out_buffer[0] |= static_cast(key_size); return 1; @@ -40,30 +45,43 @@ size_t EncodeSize(EntryType type, uint32_t key_size, char* out_buffer) { return ptr - out_buffer; } } +} // namespace -// Return position after the size byte(s). nullptr means error -const char* DecodeSize(const char* offset, const char* limit, - EntryType* entry_type, size_t* key_size) { - assert(offset < limit); - *entry_type = static_cast( - (static_cast(offset[0]) & ~kSizeInlineLimit) >> 6); - char inline_key_size = offset[0] & kSizeInlineLimit; +// Fill bytes_read with number of bytes read. +inline Status PlainTableKeyDecoder::DecodeSize(uint32_t start_offset, + PlainTableEntryType* entry_type, + uint32_t* key_size, + uint32_t* bytes_read) { + Slice next_byte_slice; + bool success = file_reader_.Read(start_offset, 1, &next_byte_slice); + if (!success) { + return file_reader_.status(); + } + *entry_type = static_cast( + (static_cast(next_byte_slice[0]) & ~kSizeInlineLimit) >> + 6); + char inline_key_size = next_byte_slice[0] & kSizeInlineLimit; if (inline_key_size < kSizeInlineLimit) { *key_size = inline_key_size; - return offset + 1; + *bytes_read = 1; + return Status::OK(); } else { uint32_t extra_size; - const char* ptr = GetVarint32Ptr(offset + 1, limit, &extra_size); - if (ptr == nullptr) { - return nullptr; + uint32_t tmp_bytes_read; + success = file_reader_.ReadVarint32(start_offset + 1, &extra_size, + &tmp_bytes_read); + if (!success) { + return file_reader_.status(); } + assert(tmp_bytes_read > 0); *key_size = kSizeInlineLimit + extra_size; - return ptr; + *bytes_read = tmp_bytes_read + 1; + return Status::OK(); } } -} // namespace -Status PlainTableKeyEncoder::AppendKey(const Slice& key, WritableFile* file, +Status PlainTableKeyEncoder::AppendKey(const Slice& key, + WritableFileWriter* file, uint64_t* offset, char* meta_bytes_buf, size_t* meta_bytes_buf_size) { ParsedInternalKey parsed_key; @@ -73,10 +91,9 @@ Status PlainTableKeyEncoder::AppendKey(const Slice& key, WritableFile* file, Slice key_to_write = key; // Portion of internal key to write out. - size_t user_key_size = fixed_user_key_len_; + uint32_t user_key_size = static_cast(key.size() - 8); if (encoding_type_ == kPlain) { if (fixed_user_key_len_ == kPlainTableVariableLength) { - user_key_size = key.size() - 8; // Write key length char key_size_buf[5]; // tmp buffer for key size as varint32 char* ptr = EncodeVarint32(key_size_buf, user_key_size); @@ -93,14 +110,12 @@ Status PlainTableKeyEncoder::AppendKey(const Slice& key, WritableFile* file, char size_bytes[12]; size_t size_bytes_pos = 0; - user_key_size = key.size() - 8; - Slice prefix = prefix_extractor_->Transform(Slice(key.data(), user_key_size)); - if (key_count_for_prefix == 0 || prefix != pre_prefix_.GetKey() || - key_count_for_prefix % index_sparseness_ == 0) { - key_count_for_prefix = 1; - pre_prefix_.SetKey(prefix); + if (key_count_for_prefix_ == 0 || prefix != pre_prefix_.GetUserKey() || + key_count_for_prefix_ % index_sparseness_ == 0) { + key_count_for_prefix_ = 1; + pre_prefix_.SetUserKey(prefix); size_bytes_pos += EncodeSize(kFullKey, user_key_size, size_bytes); Status s = file->Append(Slice(size_bytes, size_bytes_pos)); if (!s.ok()) { @@ -108,14 +123,16 @@ Status PlainTableKeyEncoder::AppendKey(const Slice& key, WritableFile* file, } *offset += size_bytes_pos; } else { - key_count_for_prefix++; - if (key_count_for_prefix == 2) { + key_count_for_prefix_++; + if (key_count_for_prefix_ == 2) { // For second key within a prefix, need to encode prefix length size_bytes_pos += - EncodeSize(kPrefixFromPreviousKey, pre_prefix_.GetKey().size(), + EncodeSize(kPrefixFromPreviousKey, + static_cast(pre_prefix_.GetUserKey().size()), size_bytes + size_bytes_pos); } - size_t prefix_len = pre_prefix_.GetKey().size(); + uint32_t prefix_len = + static_cast(pre_prefix_.GetUserKey().size()); size_bytes_pos += EncodeSize(kKeySuffix, user_key_size - prefix_len, size_bytes + size_bytes_pos); Status s = file->Append(Slice(size_bytes, size_bytes_pos)); @@ -148,28 +165,116 @@ Status PlainTableKeyEncoder::AppendKey(const Slice& key, WritableFile* file, return Status::OK(); } -namespace { -Status ReadInternalKey(const char* key_ptr, const char* limit, - uint32_t user_key_size, ParsedInternalKey* parsed_key, - size_t* bytes_read, bool* internal_key_valid, - Slice* internal_key) { - if (key_ptr + user_key_size + 1 >= limit) { - return Status::Corruption("Unexpected EOF when reading the next key"); +Slice PlainTableFileReader::GetFromBuffer(Buffer* buffer, uint32_t file_offset, + uint32_t len) { + assert(file_offset + len <= file_info_->data_end_offset); + return Slice(buffer->buf.get() + (file_offset - buffer->buf_start_offset), + len); +} + +bool PlainTableFileReader::ReadNonMmap(uint32_t file_offset, uint32_t len, + Slice* out) { + const uint32_t kPrefetchSize = 256u; + + // Try to read from buffers. + for (uint32_t i = 0; i < num_buf_; i++) { + Buffer* buffer = buffers_[num_buf_ - 1 - i].get(); + if (file_offset >= buffer->buf_start_offset && + file_offset + len <= buffer->buf_start_offset + buffer->buf_len) { + *out = GetFromBuffer(buffer, file_offset, len); + return true; + } + } + + Buffer* new_buffer; + // Data needed is not in any of the buffer. Allocate a new buffer. + if (num_buf_ < buffers_.size()) { + // Add a new buffer + new_buffer = new Buffer(); + buffers_[num_buf_++].reset(new_buffer); + } else { + // Now simply replace the last buffer. Can improve the placement policy + // if needed. + new_buffer = buffers_[num_buf_ - 1].get(); + } + + assert(file_offset + len <= file_info_->data_end_offset); + uint32_t size_to_read = std::min(file_info_->data_end_offset - file_offset, + std::max(kPrefetchSize, len)); + if (size_to_read > new_buffer->buf_capacity) { + new_buffer->buf.reset(new char[size_to_read]); + new_buffer->buf_capacity = size_to_read; + new_buffer->buf_len = 0; + } + Slice read_result; + Status s = file_info_->file->Read(file_offset, size_to_read, &read_result, + new_buffer->buf.get()); + if (!s.ok()) { + status_ = s; + return false; + } + new_buffer->buf_start_offset = file_offset; + new_buffer->buf_len = size_to_read; + *out = GetFromBuffer(new_buffer, file_offset, len); + return true; +} + +inline bool PlainTableFileReader::ReadVarint32(uint32_t offset, uint32_t* out, + uint32_t* bytes_read) { + if (file_info_->is_mmap_mode) { + const char* start = file_info_->file_data.data() + offset; + const char* limit = + file_info_->file_data.data() + file_info_->data_end_offset; + const char* key_ptr = GetVarint32Ptr(start, limit, out); + assert(key_ptr != nullptr); + *bytes_read = static_cast(key_ptr - start); + return true; + } else { + return ReadVarint32NonMmap(offset, out, bytes_read); } - if (*(key_ptr + user_key_size) == PlainTableFactory::kValueTypeSeqId0) { +} + +bool PlainTableFileReader::ReadVarint32NonMmap(uint32_t offset, uint32_t* out, + uint32_t* bytes_read) { + const char* start; + const char* limit; + const uint32_t kMaxVarInt32Size = 6u; + uint32_t bytes_to_read = + std::min(file_info_->data_end_offset - offset, kMaxVarInt32Size); + Slice bytes; + if (!Read(offset, bytes_to_read, &bytes)) { + return false; + } + start = bytes.data(); + limit = bytes.data() + bytes.size(); + + const char* key_ptr = GetVarint32Ptr(start, limit, out); + *bytes_read = + (key_ptr != nullptr) ? static_cast(key_ptr - start) : 0; + return true; +} + +Status PlainTableKeyDecoder::ReadInternalKey( + uint32_t file_offset, uint32_t user_key_size, ParsedInternalKey* parsed_key, + uint32_t* bytes_read, bool* internal_key_valid, Slice* internal_key) { + Slice tmp_slice; + bool success = file_reader_.Read(file_offset, user_key_size + 1, &tmp_slice); + if (!success) { + return file_reader_.status(); + } + if (tmp_slice[user_key_size] == PlainTableFactory::kValueTypeSeqId0) { // Special encoding for the row with seqID=0 - parsed_key->user_key = Slice(key_ptr, user_key_size); + parsed_key->user_key = Slice(tmp_slice.data(), user_key_size); parsed_key->sequence = 0; parsed_key->type = kTypeValue; *bytes_read += user_key_size + 1; *internal_key_valid = false; } else { - if (key_ptr + user_key_size + 8 >= limit) { - return Status::Corruption( - "Unexpected EOF when reading internal bytes of the next key"); + success = file_reader_.Read(file_offset, user_key_size + 8, internal_key); + if (!success) { + return file_reader_.status(); } *internal_key_valid = true; - *internal_key = Slice(key_ptr, user_key_size + 8); if (!ParseInternalKey(*internal_key, parsed_key)) { return Status::Corruption( Slice("Incorrect value type found when reading the next key")); @@ -178,83 +283,107 @@ Status ReadInternalKey(const char* key_ptr, const char* limit, } return Status::OK(); } -} // namespace -Status PlainTableKeyDecoder::NextPlainEncodingKey( - const char* start, const char* limit, ParsedInternalKey* parsed_key, - Slice* internal_key, size_t* bytes_read, bool* seekable) { - const char* key_ptr = start; - size_t user_key_size = 0; +Status PlainTableKeyDecoder::NextPlainEncodingKey(uint32_t start_offset, + ParsedInternalKey* parsed_key, + Slice* internal_key, + uint32_t* bytes_read, + bool* seekable) { + uint32_t user_key_size = 0; + Status s; if (fixed_user_key_len_ != kPlainTableVariableLength) { user_key_size = fixed_user_key_len_; - key_ptr = start; } else { uint32_t tmp_size = 0; - key_ptr = GetVarint32Ptr(start, limit, &tmp_size); - if (key_ptr == nullptr) { - return Status::Corruption( - "Unexpected EOF when reading the next key's size"); + uint32_t tmp_read; + bool success = + file_reader_.ReadVarint32(start_offset, &tmp_size, &tmp_read); + if (!success) { + return file_reader_.status(); } - user_key_size = static_cast(tmp_size); - *bytes_read = key_ptr - start; + assert(tmp_read > 0); + user_key_size = tmp_size; + *bytes_read = tmp_read; } // dummy initial value to avoid compiler complain bool decoded_internal_key_valid = true; Slice decoded_internal_key; - Status s = - ReadInternalKey(key_ptr, limit, user_key_size, parsed_key, bytes_read, - &decoded_internal_key_valid, &decoded_internal_key); + s = ReadInternalKey(start_offset + *bytes_read, user_key_size, parsed_key, + bytes_read, &decoded_internal_key_valid, + &decoded_internal_key); if (!s.ok()) { return s; } - if (internal_key != nullptr) { + if (!file_reader_.file_info()->is_mmap_mode) { + cur_key_.SetInternalKey(*parsed_key); + parsed_key->user_key = + Slice(cur_key_.GetInternalKey().data(), user_key_size); + if (internal_key != nullptr) { + *internal_key = cur_key_.GetInternalKey(); + } + } else if (internal_key != nullptr) { if (decoded_internal_key_valid) { *internal_key = decoded_internal_key; } else { // Need to copy out the internal key cur_key_.SetInternalKey(*parsed_key); - *internal_key = cur_key_.GetKey(); + *internal_key = cur_key_.GetInternalKey(); } } return Status::OK(); } Status PlainTableKeyDecoder::NextPrefixEncodingKey( - const char* start, const char* limit, ParsedInternalKey* parsed_key, - Slice* internal_key, size_t* bytes_read, bool* seekable) { - const char* key_ptr = start; - EntryType entry_type; + uint32_t start_offset, ParsedInternalKey* parsed_key, Slice* internal_key, + uint32_t* bytes_read, bool* seekable) { + PlainTableEntryType entry_type; bool expect_suffix = false; + Status s; do { - size_t size = 0; + uint32_t size = 0; // dummy initial value to avoid compiler complain bool decoded_internal_key_valid = true; - const char* pos = DecodeSize(key_ptr, limit, &entry_type, &size); - if (pos == nullptr) { + uint32_t my_bytes_read = 0; + s = DecodeSize(start_offset + *bytes_read, &entry_type, &size, + &my_bytes_read); + if (!s.ok()) { + return s; + } + if (my_bytes_read == 0) { return Status::Corruption("Unexpected EOF when reading size of the key"); } - *bytes_read += pos - key_ptr; - key_ptr = pos; + *bytes_read += my_bytes_read; switch (entry_type) { case kFullKey: { expect_suffix = false; Slice decoded_internal_key; - Status s = - ReadInternalKey(key_ptr, limit, size, parsed_key, bytes_read, - &decoded_internal_key_valid, &decoded_internal_key); + s = ReadInternalKey(start_offset + *bytes_read, size, parsed_key, + bytes_read, &decoded_internal_key_valid, + &decoded_internal_key); if (!s.ok()) { return s; } - saved_user_key_ = parsed_key->user_key; - if (internal_key != nullptr) { - if (decoded_internal_key_valid) { + if (!file_reader_.file_info()->is_mmap_mode || + (internal_key != nullptr && !decoded_internal_key_valid)) { + // In non-mmap mode, always need to make a copy of keys returned to + // users, because after reading value for the key, the key might + // be invalid. + cur_key_.SetInternalKey(*parsed_key); + saved_user_key_ = cur_key_.GetUserKey(); + if (!file_reader_.file_info()->is_mmap_mode) { + parsed_key->user_key = + Slice(cur_key_.GetInternalKey().data(), size); + } + if (internal_key != nullptr) { + *internal_key = cur_key_.GetInternalKey(); + } + } else { + if (internal_key != nullptr) { *internal_key = decoded_internal_key; - } else { - cur_key_.SetInternalKey(*parsed_key); - *internal_key = cur_key_.GetKey(); } + saved_user_key_ = parsed_key->user_key; } break; } @@ -275,51 +404,95 @@ Status PlainTableKeyDecoder::NextPrefixEncodingKey( if (seekable != nullptr) { *seekable = false; } - assert(prefix_len_ >= 0); - cur_key_.Reserve(prefix_len_ + size); Slice tmp_slice; - Status s = ReadInternalKey(key_ptr, limit, size, parsed_key, bytes_read, - &decoded_internal_key_valid, &tmp_slice); + s = ReadInternalKey(start_offset + *bytes_read, size, parsed_key, + bytes_read, &decoded_internal_key_valid, + &tmp_slice); if (!s.ok()) { return s; } - cur_key_.SetInternalKey(Slice(saved_user_key_.data(), prefix_len_), - *parsed_key); - assert( - prefix_extractor_ == nullptr || - prefix_extractor_->Transform(ExtractUserKey(cur_key_.GetKey())) == - Slice(saved_user_key_.data(), prefix_len_)); - parsed_key->user_key = ExtractUserKey(cur_key_.GetKey()); + if (!file_reader_.file_info()->is_mmap_mode) { + // In non-mmap mode, we need to make a copy of keys returned to + // users, because after reading value for the key, the key might + // be invalid. + // saved_user_key_ points to cur_key_. We are making a copy of + // the prefix part to another string, and construct the current + // key from the prefix part and the suffix part back to cur_key_. + std::string tmp = + Slice(saved_user_key_.data(), prefix_len_).ToString(); + cur_key_.Reserve(prefix_len_ + size); + cur_key_.SetInternalKey(tmp, *parsed_key); + parsed_key->user_key = + Slice(cur_key_.GetInternalKey().data(), prefix_len_ + size); + saved_user_key_ = cur_key_.GetUserKey(); + } else { + cur_key_.Reserve(prefix_len_ + size); + cur_key_.SetInternalKey(Slice(saved_user_key_.data(), prefix_len_), + *parsed_key); + } + parsed_key->user_key = cur_key_.GetUserKey(); if (internal_key != nullptr) { - *internal_key = cur_key_.GetKey(); + *internal_key = cur_key_.GetInternalKey(); } break; } default: - return Status::Corruption("Identified size flag."); + return Status::Corruption("Un-identified size flag."); } } while (expect_suffix); // Another round if suffix is expected. return Status::OK(); } -Status PlainTableKeyDecoder::NextKey(const char* start, const char* limit, +Status PlainTableKeyDecoder::NextKey(uint32_t start_offset, ParsedInternalKey* parsed_key, - Slice* internal_key, size_t* bytes_read, - bool* seekable) { + Slice* internal_key, Slice* value, + uint32_t* bytes_read, bool* seekable) { + assert(value != nullptr); + Status s = NextKeyNoValue(start_offset, parsed_key, internal_key, bytes_read, + seekable); + if (s.ok()) { + assert(bytes_read != nullptr); + uint32_t value_size; + uint32_t value_size_bytes; + bool success = file_reader_.ReadVarint32(start_offset + *bytes_read, + &value_size, &value_size_bytes); + if (!success) { + return file_reader_.status(); + } + if (value_size_bytes == 0) { + return Status::Corruption( + "Unexpected EOF when reading the next value's size."); + } + *bytes_read += value_size_bytes; + success = file_reader_.Read(start_offset + *bytes_read, value_size, value); + if (!success) { + return file_reader_.status(); + } + *bytes_read += value_size; + } + return s; +} + +Status PlainTableKeyDecoder::NextKeyNoValue(uint32_t start_offset, + ParsedInternalKey* parsed_key, + Slice* internal_key, + uint32_t* bytes_read, + bool* seekable) { *bytes_read = 0; if (seekable != nullptr) { *seekable = true; } + Status s; if (encoding_type_ == kPlain) { - return NextPlainEncodingKey(start, limit, parsed_key, internal_key, + return NextPlainEncodingKey(start_offset, parsed_key, internal_key, bytes_read, seekable); } else { assert(encoding_type_ == kPrefix); - return NextPrefixEncodingKey(start, limit, parsed_key, internal_key, + return NextPrefixEncodingKey(start_offset, parsed_key, internal_key, bytes_read, seekable); } } } // namespace rocksdb -#endif // ROCKSDB_LITE +#endif // ROCKSDB_LIT diff --git a/table/plain_table_key_coding.h b/table/plain_table_key_coding.h index ba66c26452a..321e0aed594 100644 --- a/table/plain_table_key_coding.h +++ b/table/plain_table_key_coding.h @@ -1,18 +1,22 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #pragma once #ifndef ROCKSDB_LITE +#include #include "rocksdb/slice.h" #include "db/dbformat.h" +#include "table/plain_table_reader.h" namespace rocksdb { class WritableFile; struct ParsedInternalKey; +struct PlainTableReaderFileInfo; +enum PlainTableEntryType : unsigned char; // Helper class to write out a key to an output file // Actual data format of the key is documented in plain_table_factory.h @@ -26,7 +30,7 @@ class PlainTableKeyEncoder { fixed_user_key_len_(user_key_len), prefix_extractor_(prefix_extractor), index_sparseness_((index_sparseness > 1) ? index_sparseness : 1), - key_count_for_prefix(0) {} + key_count_for_prefix_(0) {} // key: the key to write out, in the format of internal key. // file: the output file to write out // offset: offset in the file. Needs to be updated after appending bytes @@ -34,7 +38,7 @@ class PlainTableKeyEncoder { // meta_bytes_buf: buffer for extra meta bytes // meta_bytes_buf_size: offset to append extra meta bytes. Will be updated // if meta_bytes_buf is updated. - Status AppendKey(const Slice& key, WritableFile* file, uint64_t* offset, + Status AppendKey(const Slice& key, WritableFileWriter* file, uint64_t* offset, char* meta_bytes_buf, size_t* meta_bytes_buf_size); // Return actual encoding type to be picked @@ -45,18 +49,88 @@ class PlainTableKeyEncoder { uint32_t fixed_user_key_len_; const SliceTransform* prefix_extractor_; const size_t index_sparseness_; - size_t key_count_for_prefix; + size_t key_count_for_prefix_; IterKey pre_prefix_; }; +class PlainTableFileReader { + public: + explicit PlainTableFileReader(const PlainTableReaderFileInfo* _file_info) + : file_info_(_file_info), num_buf_(0) {} + // In mmaped mode, the results point to mmaped area of the file, which + // means it is always valid before closing the file. + // In non-mmap mode, the results point to an internal buffer. If the caller + // makes another read call, the results may not be valid. So callers should + // make a copy when needed. + // In order to save read calls to files, we keep two internal buffers: + // the first read and the most recent read. This is efficient because it + // columns these two common use cases: + // (1) hash index only identify one location, we read the key to verify + // the location, and read key and value if it is the right location. + // (2) after hash index checking, we identify two locations (because of + // hash bucket conflicts), we binary search the two location to see + // which one is what we need and start to read from the location. + // These two most common use cases will be covered by the two buffers + // so that we don't need to re-read the same location. + // Currently we keep a fixed size buffer. If a read doesn't exactly fit + // the buffer, we replace the second buffer with the location user reads. + // + // If return false, status code is stored in status_. + bool Read(uint32_t file_offset, uint32_t len, Slice* out) { + if (file_info_->is_mmap_mode) { + assert(file_offset + len <= file_info_->data_end_offset); + *out = Slice(file_info_->file_data.data() + file_offset, len); + return true; + } else { + return ReadNonMmap(file_offset, len, out); + } + } + + // If return false, status code is stored in status_. + bool ReadNonMmap(uint32_t file_offset, uint32_t len, Slice* output); + + // *bytes_read = 0 means eof. false means failure and status is saved + // in status_. Not directly returning Status to save copying status + // object to map previous performance of mmap mode. + inline bool ReadVarint32(uint32_t offset, uint32_t* output, + uint32_t* bytes_read); + + bool ReadVarint32NonMmap(uint32_t offset, uint32_t* output, + uint32_t* bytes_read); + + Status status() const { return status_; } + + const PlainTableReaderFileInfo* file_info() { return file_info_; } + + private: + const PlainTableReaderFileInfo* file_info_; + + struct Buffer { + Buffer() : buf_start_offset(0), buf_len(0), buf_capacity(0) {} + std::unique_ptr buf; + uint32_t buf_start_offset; + uint32_t buf_len; + uint32_t buf_capacity; + }; + + // Keep buffers for two recent reads. + std::array, 2> buffers_; + uint32_t num_buf_; + Status status_; + + Slice GetFromBuffer(Buffer* buf, uint32_t file_offset, uint32_t len); +}; + // A helper class to decode keys from input buffer // Actual data format of the key is documented in plain_table_factory.h class PlainTableKeyDecoder { public: - explicit PlainTableKeyDecoder(EncodingType encoding_type, + explicit PlainTableKeyDecoder(const PlainTableReaderFileInfo* file_info, + EncodingType encoding_type, uint32_t user_key_len, const SliceTransform* prefix_extractor) - : encoding_type_(encoding_type), + : file_reader_(file_info), + encoding_type_(encoding_type), prefix_len_(0), fixed_user_key_len_(user_key_len), prefix_extractor_(prefix_extractor), @@ -70,9 +144,15 @@ class PlainTableKeyDecoder { // bytes_read: how many bytes read from start. Output // seekable: whether key can be read from this place. Used when building // indexes. Output. - Status NextKey(const char* start, const char* limit, - ParsedInternalKey* parsed_key, Slice* internal_key, - size_t* bytes_read, bool* seekable = nullptr); + Status NextKey(uint32_t start_offset, ParsedInternalKey* parsed_key, + Slice* internal_key, Slice* value, uint32_t* bytes_read, + bool* seekable = nullptr); + + Status NextKeyNoValue(uint32_t start_offset, ParsedInternalKey* parsed_key, + Slice* internal_key, uint32_t* bytes_read, + bool* seekable = nullptr); + + PlainTableFileReader file_reader_; EncodingType encoding_type_; uint32_t prefix_len_; uint32_t fixed_user_key_len_; @@ -82,14 +162,20 @@ class PlainTableKeyDecoder { bool in_prefix_; private: - Status NextPlainEncodingKey(const char* start, const char* limit, + Status NextPlainEncodingKey(uint32_t start_offset, ParsedInternalKey* parsed_key, - Slice* internal_key, size_t* bytes_read, + Slice* internal_key, uint32_t* bytes_read, bool* seekable = nullptr); - Status NextPrefixEncodingKey(const char* start, const char* limit, + Status NextPrefixEncodingKey(uint32_t start_offset, ParsedInternalKey* parsed_key, - Slice* internal_key, size_t* bytes_read, + Slice* internal_key, uint32_t* bytes_read, bool* seekable = nullptr); + Status ReadInternalKey(uint32_t file_offset, uint32_t user_key_size, + ParsedInternalKey* parsed_key, uint32_t* bytes_read, + bool* internal_key_valid, Slice* internal_key); + inline Status DecodeSize(uint32_t start_offset, + PlainTableEntryType* entry_type, uint32_t* key_size, + uint32_t* bytes_read); }; } // namespace rocksdb diff --git a/table/plain_table_reader.cc b/table/plain_table_reader.cc index b5eccd310c5..d4d9edb7412 100644 --- a/table/plain_table_reader.cc +++ b/table/plain_table_reader.cc @@ -22,20 +22,22 @@ #include "table/bloom_block.h" #include "table/filter_block.h" #include "table/format.h" +#include "table/internal_iterator.h" #include "table/meta_blocks.h" #include "table/two_level_iterator.h" #include "table/plain_table_factory.h" #include "table/plain_table_key_coding.h" +#include "table/get_context.h" +#include "monitoring/histogram.h" +#include "monitoring/perf_context_imp.h" #include "util/arena.h" #include "util/coding.h" #include "util/dynamic_bloom.h" #include "util/hash.h" -#include "util/histogram.h" #include "util/murmurhash.h" -#include "util/perf_context_imp.h" #include "util/stop_watch.h" - +#include "util/string_util.h" namespace rocksdb { @@ -49,28 +51,30 @@ inline uint32_t GetFixed32Element(const char* base, size_t offset) { } // namespace // Iterator to iterate IndexedTable -class PlainTableIterator : public Iterator { +class PlainTableIterator : public InternalIterator { public: explicit PlainTableIterator(PlainTableReader* table, bool use_prefix_seek); ~PlainTableIterator(); - bool Valid() const; + bool Valid() const override; + + void SeekToFirst() override; - void SeekToFirst(); + void SeekToLast() override; - void SeekToLast(); + void Seek(const Slice& target) override; - void Seek(const Slice& target); + void SeekForPrev(const Slice& target) override; - void Next(); + void Next() override; - void Prev(); + void Prev() override; - Slice key() const; + Slice key() const override; - Slice value() const; + Slice value() const override; - Status status() const; + Status status() const override; private: PlainTableReader* table_; @@ -87,8 +91,8 @@ class PlainTableIterator : public Iterator { }; extern const uint64_t kPlainTableMagicNumber; -PlainTableReader::PlainTableReader(const Options& options, - unique_ptr&& file, +PlainTableReader::PlainTableReader(const ImmutableCFOptions& ioptions, + unique_ptr&& file, const EnvOptions& storage_options, const InternalKeyComparator& icomparator, EncodingType encoding_type, @@ -97,52 +101,52 @@ PlainTableReader::PlainTableReader(const Options& options, : internal_comparator_(icomparator), encoding_type_(encoding_type), full_scan_mode_(false), - data_end_offset_(table_properties->data_size), - user_key_len_(table_properties->fixed_key_len), - prefix_extractor_(options.prefix_extractor.get()), + user_key_len_(static_cast(table_properties->fixed_key_len)), + prefix_extractor_(ioptions.prefix_extractor), enable_bloom_(false), bloom_(6, nullptr), - options_(options), - file_(std::move(file)), + file_info_(std::move(file), storage_options, + static_cast(table_properties->data_size)), + ioptions_(ioptions), file_size_(file_size), table_properties_(nullptr) {} PlainTableReader::~PlainTableReader() { } -Status PlainTableReader::Open(const Options& options, - const EnvOptions& soptions, +Status PlainTableReader::Open(const ImmutableCFOptions& ioptions, + const EnvOptions& env_options, const InternalKeyComparator& internal_comparator, - unique_ptr&& file, + unique_ptr&& file, uint64_t file_size, unique_ptr* table_reader, const int bloom_bits_per_key, double hash_table_ratio, size_t index_sparseness, size_t huge_page_tlb_size, bool full_scan_mode) { - assert(options.allow_mmap_reads); if (file_size > PlainTableIndex::kMaxFileSize) { return Status::NotSupported("File is too large for PlainTableReader!"); } TableProperties* props = nullptr; auto s = ReadTableProperties(file.get(), file_size, kPlainTableMagicNumber, - options.env, options.info_log.get(), &props); + ioptions, &props); if (!s.ok()) { return s; } assert(hash_table_ratio >= 0.0); auto& user_props = props->user_collected_properties; - auto prefix_extractor_in_file = - user_props.find(PlainTablePropertyNames::kPrefixExtractorName); + auto prefix_extractor_in_file = props->prefix_extractor_name; - if (!full_scan_mode && prefix_extractor_in_file != user_props.end()) { - if (!options.prefix_extractor) { + if (!full_scan_mode && + !prefix_extractor_in_file.empty() /* old version sst file*/ + && prefix_extractor_in_file != "nullptr") { + if (!ioptions.prefix_extractor) { return Status::InvalidArgument( "Prefix extractor is missing when opening a PlainTable built " "using a prefix extractor"); - } else if (prefix_extractor_in_file->second.compare( - options.prefix_extractor->Name()) != 0) { + } else if (prefix_extractor_in_file.compare( + ioptions.prefix_extractor->Name()) != 0) { return Status::InvalidArgument( "Prefix extractor given doesn't match the one used to build " "PlainTable"); @@ -158,10 +162,10 @@ Status PlainTableReader::Open(const Options& options, } std::unique_ptr new_reader(new PlainTableReader( - options, std::move(file), soptions, internal_comparator, encoding_type, - file_size, props)); + ioptions, std::move(file), env_options, internal_comparator, + encoding_type, file_size, props)); - s = new_reader->MmapDataFile(); + s = new_reader->MmapDataIfNeeded(); if (!s.ok()) { return s; } @@ -185,30 +189,29 @@ Status PlainTableReader::Open(const Options& options, void PlainTableReader::SetupForCompaction() { } -Iterator* PlainTableReader::NewIterator(const ReadOptions& options, - Arena* arena) { - if (options.total_order_seek && !IsTotalOrderMode()) { - return NewErrorIterator( - Status::InvalidArgument("total_order_seek not supported"), arena); - } +InternalIterator* PlainTableReader::NewIterator(const ReadOptions& options, + Arena* arena, + bool skip_filters) { + bool use_prefix_seek = !IsTotalOrderMode() && !options.total_order_seek; if (arena == nullptr) { - return new PlainTableIterator(this, prefix_extractor_ != nullptr); + return new PlainTableIterator(this, use_prefix_seek); } else { auto mem = arena->AllocateAligned(sizeof(PlainTableIterator)); - return new (mem) PlainTableIterator(this, prefix_extractor_ != nullptr); + return new (mem) PlainTableIterator(this, use_prefix_seek); } } Status PlainTableReader::PopulateIndexRecordList( PlainTableIndexBuilder* index_builder, vector* prefix_hashes) { Slice prev_key_prefix_slice; + std::string prev_key_prefix_buf; uint32_t pos = data_start_offset_; bool is_first_record = true; Slice key_prefix_slice; - PlainTableKeyDecoder decoder(encoding_type_, user_key_len_, - options_.prefix_extractor.get()); - while (pos < data_end_offset_) { + PlainTableKeyDecoder decoder(&file_info_, encoding_type_, user_key_len_, + ioptions_.prefix_extractor); + while (pos < file_info_.data_end_offset) { uint32_t key_offset = pos; ParsedInternalKey key; Slice value_slice; @@ -226,7 +229,12 @@ Status PlainTableReader::PopulateIndexRecordList( if (!is_first_record) { prefix_hashes->push_back(GetSliceHash(prev_key_prefix_slice)); } - prev_key_prefix_slice = key_prefix_slice; + if (file_info_.is_mmap_mode) { + prev_key_prefix_slice = key_prefix_slice; + } else { + prev_key_prefix_buf = key_prefix_slice.ToString(); + prev_key_prefix_slice = prev_key_prefix_buf; + } } } @@ -252,8 +260,8 @@ void PlainTableReader::AllocateAndFillBloom(int bloom_bits_per_key, uint32_t bloom_total_bits = num_prefixes * bloom_bits_per_key; if (bloom_total_bits > 0) { enable_bloom_ = true; - bloom_.SetTotalBits(&arena_, bloom_total_bits, options_.bloom_locality, - huge_page_tlb_size, options_.info_log.get()); + bloom_.SetTotalBits(&arena_, bloom_total_bits, ioptions_.bloom_locality, + huge_page_tlb_size, ioptions_.info_log); FillBloom(prefix_hashes); } } @@ -266,9 +274,12 @@ void PlainTableReader::FillBloom(vector* prefix_hashes) { } } -Status PlainTableReader::MmapDataFile() { - // Get mmapped memory to file_data_. - return file_->Read(0, file_size_, &file_data_, nullptr); +Status PlainTableReader::MmapDataIfNeeded() { + if (file_info_.is_mmap_mode) { + // Get mmapped memory. + return file_info_.file->Read(0, file_size_, &file_info_.file_data, nullptr); + } + return Status::OK(); } Status PlainTableReader::PopulateIndex(TableProperties* props, @@ -279,39 +290,49 @@ Status PlainTableReader::PopulateIndex(TableProperties* props, assert(props != nullptr); table_properties_.reset(props); - BlockContents bloom_block_contents; - auto s = ReadMetaBlock(file_.get(), file_size_, kPlainTableMagicNumber, - options_.env, BloomBlockBuilder::kBloomBlock, - &bloom_block_contents); - bool index_in_file = s.ok(); - BlockContents index_block_contents; - s = ReadMetaBlock(file_.get(), file_size_, kPlainTableMagicNumber, - options_.env, PlainTableIndexBuilder::kPlainTableIndexBlock, - &index_block_contents); + Status s = ReadMetaBlock(file_info_.file.get(), nullptr /* prefetch_buffer */, + file_size_, kPlainTableMagicNumber, ioptions_, + PlainTableIndexBuilder::kPlainTableIndexBlock, + &index_block_contents); - index_in_file &= s.ok(); + bool index_in_file = s.ok(); - Slice* bloom_block; + BlockContents bloom_block_contents; + bool bloom_in_file = false; + // We only need to read the bloom block if index block is in file. if (index_in_file) { + s = ReadMetaBlock(file_info_.file.get(), nullptr /* prefetch_buffer */, + file_size_, kPlainTableMagicNumber, ioptions_, + BloomBlockBuilder::kBloomBlock, &bloom_block_contents); + bloom_in_file = s.ok() && bloom_block_contents.data.size() > 0; + } + + Slice* bloom_block; + if (bloom_in_file) { + // If bloom_block_contents.allocation is not empty (which will be the case + // for non-mmap mode), it holds the alloated memory for the bloom block. + // It needs to be kept alive to keep `bloom_block` valid. + bloom_block_alloc_ = std::move(bloom_block_contents.allocation); bloom_block = &bloom_block_contents.data; } else { bloom_block = nullptr; } - // index_in_file == true only if there are kBloomBlock and - // kPlainTableIndexBlock - // in file - Slice* index_block; if (index_in_file) { + // If index_block_contents.allocation is not empty (which will be the case + // for non-mmap mode), it holds the alloated memory for the index block. + // It needs to be kept alive to keep `index_block` valid. + index_block_alloc_ = std::move(index_block_contents.allocation); index_block = &index_block_contents.data; } else { index_block = nullptr; } - if ((options_.prefix_extractor.get() == nullptr) && (hash_table_ratio != 0)) { - // options.prefix_extractor is requried for a hash-based look-up. + if ((ioptions_.prefix_extractor == nullptr) && + (hash_table_ratio != 0)) { + // ioptions.prefix_extractor is requried for a hash-based look-up. return Status::NotSupported( "PlainTable requires a prefix extractor enable prefix hash mode."); } @@ -325,14 +346,15 @@ Status PlainTableReader::PopulateIndex(TableProperties* props, // Allocate bloom filter here for total order mode. if (IsTotalOrderMode()) { uint32_t num_bloom_bits = - table_properties_->num_entries * bloom_bits_per_key; + static_cast(table_properties_->num_entries) * + bloom_bits_per_key; if (num_bloom_bits > 0) { enable_bloom_ = true; - bloom_.SetTotalBits(&arena_, num_bloom_bits, options_.bloom_locality, - huge_page_tlb_size, options_.info_log.get()); + bloom_.SetTotalBits(&arena_, num_bloom_bits, ioptions_.bloom_locality, + huge_page_tlb_size, ioptions_.info_log); } } - } else { + } else if (bloom_in_file) { enable_bloom_ = true; auto num_blocks_property = props->user_collected_properties.find( PlainTablePropertyNames::kNumBloomBlocks); @@ -348,20 +370,24 @@ Status PlainTableReader::PopulateIndex(TableProperties* props, bloom_.SetRawData( const_cast( reinterpret_cast(bloom_block->data())), - bloom_block->size() * 8, num_blocks); + static_cast(bloom_block->size()) * 8, num_blocks); + } else { + // Index in file but no bloom in file. Disable bloom filter in this case. + enable_bloom_ = false; + bloom_bits_per_key = 0; } - PlainTableIndexBuilder index_builder(&arena_, options_, index_sparseness, + PlainTableIndexBuilder index_builder(&arena_, ioptions_, index_sparseness, hash_table_ratio, huge_page_tlb_size); std::vector prefix_hashes; if (!index_in_file) { - Status s = PopulateIndexRecordList(&index_builder, &prefix_hashes); + s = PopulateIndexRecordList(&index_builder, &prefix_hashes); if (!s.ok()) { return s; } } else { - Status s = index_.InitFromRawData(*index_block); + s = index_.InitFromRawData(*index_block); if (!s.ok()) { return s; } @@ -377,27 +403,28 @@ Status PlainTableReader::PopulateIndex(TableProperties* props, // Fill two table properties. if (!index_in_file) { props->user_collected_properties["plain_table_hash_table_size"] = - std::to_string(index_.GetIndexSize() * PlainTableIndex::kOffsetLen); + ToString(index_.GetIndexSize() * PlainTableIndex::kOffsetLen); props->user_collected_properties["plain_table_sub_index_size"] = - std::to_string(index_.GetSubIndexSize()); + ToString(index_.GetSubIndexSize()); } else { props->user_collected_properties["plain_table_hash_table_size"] = - std::to_string(0); + ToString(0); props->user_collected_properties["plain_table_sub_index_size"] = - std::to_string(0); + ToString(0); } return Status::OK(); } -Status PlainTableReader::GetOffset(const Slice& target, const Slice& prefix, +Status PlainTableReader::GetOffset(PlainTableKeyDecoder* decoder, + const Slice& target, const Slice& prefix, uint32_t prefix_hash, bool& prefix_matched, uint32_t* offset) const { prefix_matched = false; uint32_t prefix_index_offset; auto res = index_.GetOffset(prefix_hash, &prefix_index_offset); if (res == PlainTableIndex::kNoPrefixForBucket) { - *offset = data_end_offset_; + *offset = file_info_.data_end_offset; return Status::OK(); } else if (res == PlainTableIndex::kDirectToFile) { *offset = prefix_index_offset; @@ -420,12 +447,8 @@ Status PlainTableReader::GetOffset(const Slice& target, const Slice& prefix, while (high - low > 1) { uint32_t mid = (high + low) / 2; uint32_t file_offset = GetFixed32Element(base_ptr, mid); - size_t tmp; - Status s = PlainTableKeyDecoder(encoding_type_, user_key_len_, - options_.prefix_extractor.get()) - .NextKey(file_data_.data() + file_offset, - file_data_.data() + data_end_offset_, &mid_key, - nullptr, &tmp); + uint32_t tmp; + Status s = decoder->NextKeyNoValue(file_offset, &mid_key, nullptr, &tmp); if (!s.ok()) { return s; } @@ -448,13 +471,9 @@ Status PlainTableReader::GetOffset(const Slice& target, const Slice& prefix, // prefix as target. We need to rule out one of them to avoid to go // to the wrong prefix. ParsedInternalKey low_key; - size_t tmp; + uint32_t tmp; uint32_t low_key_offset = GetFixed32Element(base_ptr, low); - Status s = PlainTableKeyDecoder(encoding_type_, user_key_len_, - options_.prefix_extractor.get()) - .NextKey(file_data_.data() + low_key_offset, - file_data_.data() + data_end_offset_, &low_key, - nullptr, &tmp); + Status s = decoder->NextKeyNoValue(low_key_offset, &low_key, nullptr, &tmp); if (!s.ok()) { return s; } @@ -469,50 +488,45 @@ Status PlainTableReader::GetOffset(const Slice& target, const Slice& prefix, } else { // target is larger than a key of the last prefix in this bucket // but with a different prefix. Key does not exist. - *offset = data_end_offset_; + *offset = file_info_.data_end_offset; } return Status::OK(); } bool PlainTableReader::MatchBloom(uint32_t hash) const { - return !enable_bloom_ || bloom_.MayContainHash(hash); -} + if (!enable_bloom_) { + return true; + } + if (bloom_.MayContainHash(hash)) { + PERF_COUNTER_ADD(bloom_sst_hit_count, 1); + return true; + } else { + PERF_COUNTER_ADD(bloom_sst_miss_count, 1); + return false; + } +} Status PlainTableReader::Next(PlainTableKeyDecoder* decoder, uint32_t* offset, ParsedInternalKey* parsed_key, Slice* internal_key, Slice* value, bool* seekable) const { - if (*offset == data_end_offset_) { - *offset = data_end_offset_; + if (*offset == file_info_.data_end_offset) { + *offset = file_info_.data_end_offset; return Status::OK(); } - if (*offset > data_end_offset_) { + if (*offset > file_info_.data_end_offset) { return Status::Corruption("Offset is out of file size"); } - const char* start = file_data_.data() + *offset; - size_t bytes_for_key; - Status s = - decoder->NextKey(start, file_data_.data() + data_end_offset_, parsed_key, - internal_key, &bytes_for_key, seekable); + uint32_t bytes_read; + Status s = decoder->NextKey(*offset, parsed_key, internal_key, value, + &bytes_read, seekable); if (!s.ok()) { return s; } - uint32_t value_size; - const char* value_ptr = GetVarint32Ptr( - start + bytes_for_key, file_data_.data() + data_end_offset_, &value_size); - if (value_ptr == nullptr) { - return Status::Corruption( - "Unexpected EOF when reading the next value's size."); - } - *offset = *offset + (value_ptr - start) + value_size; - if (*offset > data_end_offset_) { - return Status::Corruption("Unexpected EOF when reading the next value. "); - } - *value = Slice(value_ptr, value_size); - + *offset = *offset + bytes_read; return Status::OK(); } @@ -524,10 +538,7 @@ void PlainTableReader::Prepare(const Slice& target) { } Status PlainTableReader::Get(const ReadOptions& ro, const Slice& target, - void* arg, - bool (*saver)(void*, const ParsedInternalKey&, - const Slice&), - void (*mark_key_may_exist)(void*)) { + GetContext* get_context, bool skip_filters) { // Check bloom filter first. Slice prefix_slice; uint32_t prefix_hash; @@ -553,8 +564,11 @@ Status PlainTableReader::Get(const ReadOptions& ro, const Slice& target, } uint32_t offset; bool prefix_match; - Status s = - GetOffset(target, prefix_slice, prefix_hash, prefix_match, &offset); + PlainTableKeyDecoder decoder(&file_info_, encoding_type_, user_key_len_, + ioptions_.prefix_extractor); + Status s = GetOffset(&decoder, target, prefix_slice, prefix_hash, + prefix_match, &offset); + if (!s.ok()) { return s; } @@ -564,10 +578,8 @@ Status PlainTableReader::Get(const ReadOptions& ro, const Slice& target, return Status::Corruption(Slice()); } Slice found_value; - PlainTableKeyDecoder decoder(encoding_type_, user_key_len_, - options_.prefix_extractor.get()); - while (offset < data_end_offset_) { - Status s = Next(&decoder, &offset, &found_key, nullptr, &found_value); + while (offset < file_info_.data_end_offset) { + s = Next(&decoder, &offset, &found_key, nullptr, &found_value); if (!s.ok()) { return s; } @@ -579,8 +591,10 @@ Status PlainTableReader::Get(const ReadOptions& ro, const Slice& target, } prefix_match = true; } + // TODO(ljin): since we know the key comparison result here, + // can we enable the fast path? if (internal_comparator_.Compare(found_key, parsed_target) >= 0) { - if (!(*saver)(arg, found_key, found_value)) { + if (!get_context->SaveValue(found_key, found_value)) { break; } } @@ -595,24 +609,24 @@ uint64_t PlainTableReader::ApproximateOffsetOf(const Slice& key) { PlainTableIterator::PlainTableIterator(PlainTableReader* table, bool use_prefix_seek) : table_(table), - decoder_(table_->encoding_type_, table_->user_key_len_, - table_->prefix_extractor_), + decoder_(&table_->file_info_, table_->encoding_type_, + table_->user_key_len_, table_->prefix_extractor_), use_prefix_seek_(use_prefix_seek) { - next_offset_ = offset_ = table_->data_end_offset_; + next_offset_ = offset_ = table_->file_info_.data_end_offset; } PlainTableIterator::~PlainTableIterator() { } bool PlainTableIterator::Valid() const { - return offset_ < table_->data_end_offset_ - && offset_ >= table_->data_start_offset_; + return offset_ < table_->file_info_.data_end_offset && + offset_ >= table_->data_start_offset_; } void PlainTableIterator::SeekToFirst() { next_offset_ = table_->data_start_offset_; - if (next_offset_ >= table_->data_end_offset_) { - next_offset_ = offset_ = table_->data_end_offset_; + if (next_offset_ >= table_->file_info_.data_end_offset) { + next_offset_ = offset_ = table_->file_info_.data_end_offset; } else { Next(); } @@ -624,20 +638,33 @@ void PlainTableIterator::SeekToLast() { } void PlainTableIterator::Seek(const Slice& target) { + if (use_prefix_seek_ != !table_->IsTotalOrderMode()) { + // This check is done here instead of NewIterator() to permit creating an + // iterator with total_order_seek = true even if we won't be able to Seek() + // it. This is needed for compaction: it creates iterator with + // total_order_seek = true but usually never does Seek() on it, + // only SeekToFirst(). + status_ = + Status::InvalidArgument( + "total_order_seek not implemented for PlainTable."); + offset_ = next_offset_ = table_->file_info_.data_end_offset; + return; + } + // If the user doesn't set prefix seek option and we are not able to do a // total Seek(). assert failure. - if (!use_prefix_seek_) { + if (table_->IsTotalOrderMode()) { if (table_->full_scan_mode_) { status_ = Status::InvalidArgument("Seek() is not allowed in full scan mode."); - offset_ = next_offset_ = table_->data_end_offset_; + offset_ = next_offset_ = table_->file_info_.data_end_offset; return; } else if (table_->GetIndexSize() > 1) { assert(false); status_ = Status::NotSupported( "PlainTable cannot issue non-prefix seek unless in total order " "mode."); - offset_ = next_offset_ = table_->data_end_offset_; + offset_ = next_offset_ = table_->file_info_.data_end_offset; return; } } @@ -648,24 +675,24 @@ void PlainTableIterator::Seek(const Slice& target) { if (!table_->IsTotalOrderMode()) { prefix_hash = GetSliceHash(prefix_slice); if (!table_->MatchBloom(prefix_hash)) { - offset_ = next_offset_ = table_->data_end_offset_; + offset_ = next_offset_ = table_->file_info_.data_end_offset; return; } } bool prefix_match; - status_ = table_->GetOffset(target, prefix_slice, prefix_hash, prefix_match, - &next_offset_); + status_ = table_->GetOffset(&decoder_, target, prefix_slice, prefix_hash, + prefix_match, &next_offset_); if (!status_.ok()) { - offset_ = next_offset_ = table_->data_end_offset_; + offset_ = next_offset_ = table_->file_info_.data_end_offset; return; } - if (next_offset_ < table_-> data_end_offset_) { + if (next_offset_ < table_->file_info_.data_end_offset) { for (Next(); status_.ok() && Valid(); Next()) { if (!prefix_match) { // Need to verify the first key's prefix if (table_->GetPrefix(key()) != prefix_slice) { - offset_ = next_offset_ = table_->data_end_offset_; + offset_ = next_offset_ = table_->file_info_.data_end_offset; break; } prefix_match = true; @@ -675,19 +702,25 @@ void PlainTableIterator::Seek(const Slice& target) { } } } else { - offset_ = table_->data_end_offset_; + offset_ = table_->file_info_.data_end_offset; } } +void PlainTableIterator::SeekForPrev(const Slice& target) { + assert(false); + status_ = + Status::NotSupported("SeekForPrev() is not supported in PlainTable"); +} + void PlainTableIterator::Next() { offset_ = next_offset_; - if (offset_ < table_->data_end_offset_) { + if (offset_ < table_->file_info_.data_end_offset) { Slice tmp_slice; ParsedInternalKey parsed_key; status_ = table_->Next(&decoder_, &next_offset_, &parsed_key, &key_, &value_); if (!status_.ok()) { - offset_ = next_offset_ = table_->data_end_offset_; + offset_ = next_offset_ = table_->file_info_.data_end_offset; } } } diff --git a/table/plain_table_reader.h b/table/plain_table_reader.h index 4a626979a83..6bf8da2f988 100644 --- a/table/plain_table_reader.h +++ b/table/plain_table_reader.h @@ -22,6 +22,7 @@ #include "table/plain_table_index.h" #include "util/arena.h" #include "util/dynamic_bloom.h" +#include "util/file_reader_writer.h" namespace rocksdb { @@ -36,12 +37,28 @@ class TableCache; class TableReader; class InternalKeyComparator; class PlainTableKeyDecoder; +class GetContext; +class InternalIterator; using std::unique_ptr; using std::unordered_map; using std::vector; extern const uint32_t kPlainTableVariableLength; +struct PlainTableReaderFileInfo { + bool is_mmap_mode; + Slice file_data; + uint32_t data_end_offset; + unique_ptr file; + + PlainTableReaderFileInfo(unique_ptr&& _file, + const EnvOptions& storage_options, + uint32_t _data_size_offset) + : is_mmap_mode(storage_options.use_mmap_reads), + data_end_offset(_data_size_offset), + file(std::move(_file)) {} +}; + // Based on following output file format shown in plain_table_factory.h // When opening the output file, IndexedTableReader creates a hash table // from key prefixes to offset of the output file. IndexedTable will decide @@ -52,29 +69,30 @@ extern const uint32_t kPlainTableVariableLength; // The implementation of IndexedTableReader requires output file is mmaped class PlainTableReader: public TableReader { public: - static Status Open(const Options& options, const EnvOptions& soptions, + static Status Open(const ImmutableCFOptions& ioptions, + const EnvOptions& env_options, const InternalKeyComparator& internal_comparator, - unique_ptr&& file, uint64_t file_size, - unique_ptr* table, + unique_ptr&& file, + uint64_t file_size, unique_ptr* table, const int bloom_bits_per_key, double hash_table_ratio, size_t index_sparseness, size_t huge_page_tlb_size, bool full_scan_mode); - Iterator* NewIterator(const ReadOptions&, Arena* arena = nullptr) override; + InternalIterator* NewIterator(const ReadOptions&, + Arena* arena = nullptr, + bool skip_filters = false) override; - void Prepare(const Slice& target); + void Prepare(const Slice& target) override; - Status Get(const ReadOptions&, const Slice& key, void* arg, - bool (*result_handler)(void* arg, const ParsedInternalKey& k, - const Slice& v), - void (*mark_key_may_exist)(void*) = nullptr); + Status Get(const ReadOptions&, const Slice& key, GetContext* get_context, + bool skip_filters = false) override; - uint64_t ApproximateOffsetOf(const Slice& key); + uint64_t ApproximateOffsetOf(const Slice& key) override; uint32_t GetIndexSize() const { return index_.GetIndexSize(); } - void SetupForCompaction(); + void SetupForCompaction() override; - std::shared_ptr GetTableProperties() const { + std::shared_ptr GetTableProperties() const override { return table_properties_; } @@ -82,8 +100,9 @@ class PlainTableReader: public TableReader { return arena_.MemoryAllocatedBytes(); } - PlainTableReader(const Options& options, unique_ptr&& file, - const EnvOptions& storage_options, + PlainTableReader(const ImmutableCFOptions& ioptions, + unique_ptr&& file, + const EnvOptions& env_options, const InternalKeyComparator& internal_comparator, EncodingType encoding_type, uint64_t file_size, const TableProperties* table_properties); @@ -106,14 +125,13 @@ class PlainTableReader: public TableReader { double hash_table_ratio, size_t index_sparseness, size_t huge_page_tlb_size); - Status MmapDataFile(); + Status MmapDataIfNeeded(); private: const InternalKeyComparator internal_comparator_; EncodingType encoding_type_; // represents plain table's current status. Status status_; - Slice file_data_; PlainTableIndex index_; bool full_scan_mode_; @@ -121,8 +139,7 @@ class PlainTableReader: public TableReader { // data_start_offset_ and data_end_offset_ defines the range of the // sst file that stores data. const uint32_t data_start_offset_ = 0; - const uint32_t data_end_offset_; - const size_t user_key_len_; + const uint32_t user_key_len_; const SliceTransform* prefix_extractor_; static const size_t kNumInternalBytes = 8; @@ -130,11 +147,13 @@ class PlainTableReader: public TableReader { // Bloom filter is used to rule out non-existent key bool enable_bloom_; DynamicBloom bloom_; + PlainTableReaderFileInfo file_info_; Arena arena_; + std::unique_ptr index_block_alloc_; + std::unique_ptr bloom_block_alloc_; - const Options& options_; - unique_ptr file_; - uint32_t file_size_; + const ImmutableCFOptions& ioptions_; + uint64_t file_size_; std::shared_ptr table_properties_; bool IsFixedLength() const { @@ -201,9 +220,9 @@ class PlainTableReader: public TableReader { // Get file offset for key target. // return value prefix_matched is set to true if the offset is confirmed // for a key with the same prefix as target. - Status GetOffset(const Slice& target, const Slice& prefix, - uint32_t prefix_hash, bool& prefix_matched, - uint32_t* offset) const; + Status GetOffset(PlainTableKeyDecoder* decoder, const Slice& target, + const Slice& prefix, uint32_t prefix_hash, + bool& prefix_matched, uint32_t* offset) const; bool IsTotalOrderMode() const { return (prefix_extractor_ == nullptr); } diff --git a/table/scoped_arena_iterator.h b/table/scoped_arena_iterator.h new file mode 100644 index 00000000000..1de570dc7f2 --- /dev/null +++ b/table/scoped_arena_iterator.h @@ -0,0 +1,61 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). +// Copyright (c) 2011 The LevelDB Authors. All rights reserved. +// Use of this source code is governed by a BSD-style license that can be +// found in the LICENSE file. See the AUTHORS file for names of contributors. +#pragma once + +#include "table/internal_iterator.h" +#include "port/port.h" + +namespace rocksdb { +class ScopedArenaIterator { + + void reset(InternalIterator* iter) ROCKSDB_NOEXCEPT { + if (iter_ != nullptr) { + iter_->~InternalIterator(); + } + iter_ = iter; + } + + public: + + explicit ScopedArenaIterator(InternalIterator* iter = nullptr) + : iter_(iter) {} + + ScopedArenaIterator(const ScopedArenaIterator&) = delete; + ScopedArenaIterator& operator=(const ScopedArenaIterator&) = delete; + + ScopedArenaIterator(ScopedArenaIterator&& o) ROCKSDB_NOEXCEPT { + iter_ = o.iter_; + o.iter_ = nullptr; + } + + ScopedArenaIterator& operator=(ScopedArenaIterator&& o) ROCKSDB_NOEXCEPT { + reset(o.iter_); + o.iter_ = nullptr; + return *this; + } + + InternalIterator* operator->() { return iter_; } + InternalIterator* get() { return iter_; } + + void set(InternalIterator* iter) { reset(iter); } + + InternalIterator* release() { + assert(iter_ != nullptr); + auto* res = iter_; + iter_ = nullptr; + return res; + } + + ~ScopedArenaIterator() { + reset(nullptr); + } + + private: + InternalIterator* iter_; +}; +} // namespace rocksdb diff --git a/table/sst_file_writer.cc b/table/sst_file_writer.cc new file mode 100644 index 00000000000..adcd91f92ea --- /dev/null +++ b/table/sst_file_writer.cc @@ -0,0 +1,262 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#include "rocksdb/sst_file_writer.h" + +#include +#include "db/dbformat.h" +#include "rocksdb/table.h" +#include "table/block_based_table_builder.h" +#include "table/sst_file_writer_collectors.h" +#include "util/file_reader_writer.h" +#include "util/sync_point.h" + +namespace rocksdb { + +const std::string ExternalSstFilePropertyNames::kVersion = + "rocksdb.external_sst_file.version"; +const std::string ExternalSstFilePropertyNames::kGlobalSeqno = + "rocksdb.external_sst_file.global_seqno"; + +#ifndef ROCKSDB_LITE + +const size_t kFadviseTrigger = 1024 * 1024; // 1MB + +struct SstFileWriter::Rep { + Rep(const EnvOptions& _env_options, const Options& options, + Env::IOPriority _io_priority, const Comparator* _user_comparator, + ColumnFamilyHandle* _cfh, bool _invalidate_page_cache) + : env_options(_env_options), + ioptions(options), + mutable_cf_options(options), + io_priority(_io_priority), + internal_comparator(_user_comparator), + cfh(_cfh), + invalidate_page_cache(_invalidate_page_cache), + last_fadvise_size(0) {} + + std::unique_ptr file_writer; + std::unique_ptr builder; + EnvOptions env_options; + ImmutableCFOptions ioptions; + MutableCFOptions mutable_cf_options; + Env::IOPriority io_priority; + InternalKeyComparator internal_comparator; + ExternalSstFileInfo file_info; + InternalKey ikey; + std::string column_family_name; + ColumnFamilyHandle* cfh; + // If true, We will give the OS a hint that this file pages is not needed + // everytime we write 1MB to the file. + bool invalidate_page_cache; + // The size of the file during the last time we called Fadvise to remove + // cached pages from page cache. + uint64_t last_fadvise_size; + Status Add(const Slice& user_key, const Slice& value, + const ValueType value_type) { + if (!builder) { + return Status::InvalidArgument("File is not opened"); + } + + if (file_info.num_entries == 0) { + file_info.smallest_key.assign(user_key.data(), user_key.size()); + } else { + if (internal_comparator.user_comparator()->Compare( + user_key, file_info.largest_key) <= 0) { + // Make sure that keys are added in order + return Status::InvalidArgument("Keys must be added in order"); + } + } + + // TODO(tec) : For external SST files we could omit the seqno and type. + switch (value_type) { + case ValueType::kTypeValue: + ikey.Set(user_key, 0 /* Sequence Number */, + ValueType::kTypeValue /* Put */); + break; + case ValueType::kTypeMerge: + ikey.Set(user_key, 0 /* Sequence Number */, + ValueType::kTypeMerge /* Merge */); + break; + case ValueType::kTypeDeletion: + ikey.Set(user_key, 0 /* Sequence Number */, + ValueType::kTypeDeletion /* Delete */); + break; + default: + return Status::InvalidArgument("Value type is not supported"); + } + builder->Add(ikey.Encode(), value); + + // update file info + file_info.num_entries++; + file_info.largest_key.assign(user_key.data(), user_key.size()); + file_info.file_size = builder->FileSize(); + + InvalidatePageCache(false /* closing */); + + return Status::OK(); + } + + void InvalidatePageCache(bool closing) { + if (invalidate_page_cache == false) { + // Fadvise disabled + return; + } + uint64_t bytes_since_last_fadvise = + builder->FileSize() - last_fadvise_size; + if (bytes_since_last_fadvise > kFadviseTrigger || closing) { + TEST_SYNC_POINT_CALLBACK("SstFileWriter::Rep::InvalidatePageCache", + &(bytes_since_last_fadvise)); + // Tell the OS that we dont need this file in page cache + file_writer->InvalidateCache(0, 0); + last_fadvise_size = builder->FileSize(); + } + } + +}; + +SstFileWriter::SstFileWriter(const EnvOptions& env_options, + const Options& options, + const Comparator* user_comparator, + ColumnFamilyHandle* column_family, + bool invalidate_page_cache, + Env::IOPriority io_priority) + : rep_(new Rep(env_options, options, io_priority, user_comparator, + column_family, invalidate_page_cache)) { + rep_->file_info.file_size = 0; +} + +SstFileWriter::~SstFileWriter() { + if (rep_->builder) { + // User did not call Finish() or Finish() failed, we need to + // abandon the builder. + rep_->builder->Abandon(); + } +} + +Status SstFileWriter::Open(const std::string& file_path) { + Rep* r = rep_.get(); + Status s; + std::unique_ptr sst_file; + s = r->ioptions.env->NewWritableFile(file_path, &sst_file, r->env_options); + if (!s.ok()) { + return s; + } + + sst_file->SetIOPriority(r->io_priority); + + CompressionType compression_type; + if (r->ioptions.bottommost_compression != kDisableCompressionOption) { + compression_type = r->ioptions.bottommost_compression; + } else if (!r->ioptions.compression_per_level.empty()) { + // Use the compression of the last level if we have per level compression + compression_type = *(r->ioptions.compression_per_level.rbegin()); + } else { + compression_type = r->mutable_cf_options.compression; + } + + std::vector> + int_tbl_prop_collector_factories; + + // SstFileWriter properties collector to add SstFileWriter version. + int_tbl_prop_collector_factories.emplace_back( + new SstFileWriterPropertiesCollectorFactory(2 /* version */, + 0 /* global_seqno*/)); + + // User collector factories + auto user_collector_factories = + r->ioptions.table_properties_collector_factories; + for (size_t i = 0; i < user_collector_factories.size(); i++) { + int_tbl_prop_collector_factories.emplace_back( + new UserKeyTablePropertiesCollectorFactory( + user_collector_factories[i])); + } + int unknown_level = -1; + uint32_t cf_id; + + if (r->cfh != nullptr) { + // user explicitly specified that this file will be ingested into cfh, + // we can persist this information in the file. + cf_id = r->cfh->GetID(); + r->column_family_name = r->cfh->GetName(); + } else { + r->column_family_name = ""; + cf_id = TablePropertiesCollectorFactory::Context::kUnknownColumnFamily; + } + + TableBuilderOptions table_builder_options( + r->ioptions, r->internal_comparator, &int_tbl_prop_collector_factories, + compression_type, r->ioptions.compression_opts, + nullptr /* compression_dict */, false /* skip_filters */, + r->column_family_name, unknown_level); + r->file_writer.reset( + new WritableFileWriter(std::move(sst_file), r->env_options)); + + // TODO(tec) : If table_factory is using compressed block cache, we will + // be adding the external sst file blocks into it, which is wasteful. + r->builder.reset(r->ioptions.table_factory->NewTableBuilder( + table_builder_options, cf_id, r->file_writer.get())); + + r->file_info.file_path = file_path; + r->file_info.file_size = 0; + r->file_info.num_entries = 0; + r->file_info.sequence_number = 0; + r->file_info.version = 2; + return s; +} + +Status SstFileWriter::Add(const Slice& user_key, const Slice& value) { + return rep_->Add(user_key, value, ValueType::kTypeValue); +} + +Status SstFileWriter::Put(const Slice& user_key, const Slice& value) { + return rep_->Add(user_key, value, ValueType::kTypeValue); +} + +Status SstFileWriter::Merge(const Slice& user_key, const Slice& value) { + return rep_->Add(user_key, value, ValueType::kTypeMerge); +} + +Status SstFileWriter::Delete(const Slice& user_key) { + return rep_->Add(user_key, Slice(), ValueType::kTypeDeletion); +} + +Status SstFileWriter::Finish(ExternalSstFileInfo* file_info) { + Rep* r = rep_.get(); + if (!r->builder) { + return Status::InvalidArgument("File is not opened"); + } + if (r->file_info.num_entries == 0) { + return Status::InvalidArgument("Cannot create sst file with no entries"); + } + + Status s = r->builder->Finish(); + r->file_info.file_size = r->builder->FileSize(); + + if (s.ok()) { + s = r->file_writer->Sync(r->ioptions.use_fsync); + r->InvalidatePageCache(true /* closing */); + if (s.ok()) { + s = r->file_writer->Close(); + } + } + if (!s.ok()) { + r->ioptions.env->DeleteFile(r->file_info.file_path); + } + + if (file_info != nullptr) { + *file_info = r->file_info; + } + + r->builder.reset(); + return s; +} + +uint64_t SstFileWriter::FileSize() { + return rep_->file_info.file_size; +} +#endif // !ROCKSDB_LITE + +} // namespace rocksdb diff --git a/table/sst_file_writer_collectors.h b/table/sst_file_writer_collectors.h new file mode 100644 index 00000000000..ce3a45f5a74 --- /dev/null +++ b/table/sst_file_writer_collectors.h @@ -0,0 +1,84 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once +#include +#include "rocksdb/types.h" +#include "util/string_util.h" + +namespace rocksdb { + +// Table Properties that are specific to tables created by SstFileWriter. +struct ExternalSstFilePropertyNames { + // value of this property is a fixed uint32 number. + static const std::string kVersion; + // value of this property is a fixed uint64 number. + static const std::string kGlobalSeqno; +}; + +// PropertiesCollector used to add properties specific to tables +// generated by SstFileWriter +class SstFileWriterPropertiesCollector : public IntTblPropCollector { + public: + explicit SstFileWriterPropertiesCollector(int32_t version, + SequenceNumber global_seqno) + : version_(version), global_seqno_(global_seqno) {} + + virtual Status InternalAdd(const Slice& key, const Slice& value, + uint64_t file_size) override { + // Intentionally left blank. Have no interest in collecting stats for + // individual key/value pairs. + return Status::OK(); + } + + virtual Status Finish(UserCollectedProperties* properties) override { + // File version + std::string version_val; + PutFixed32(&version_val, static_cast(version_)); + properties->insert({ExternalSstFilePropertyNames::kVersion, version_val}); + + // Global Sequence number + std::string seqno_val; + PutFixed64(&seqno_val, static_cast(global_seqno_)); + properties->insert({ExternalSstFilePropertyNames::kGlobalSeqno, seqno_val}); + + return Status::OK(); + } + + virtual const char* Name() const override { + return "SstFileWriterPropertiesCollector"; + } + + virtual UserCollectedProperties GetReadableProperties() const override { + return {{ExternalSstFilePropertyNames::kVersion, ToString(version_)}}; + } + + private: + int32_t version_; + SequenceNumber global_seqno_; +}; + +class SstFileWriterPropertiesCollectorFactory + : public IntTblPropCollectorFactory { + public: + explicit SstFileWriterPropertiesCollectorFactory(int32_t version, + SequenceNumber global_seqno) + : version_(version), global_seqno_(global_seqno) {} + + virtual IntTblPropCollector* CreateIntTblPropCollector( + uint32_t column_family_id) override { + return new SstFileWriterPropertiesCollector(version_, global_seqno_); + } + + virtual const char* Name() const override { + return "SstFileWriterPropertiesCollector"; + } + + private: + int32_t version_; + SequenceNumber global_seqno_; +}; + +} // namespace rocksdb diff --git a/table/table_builder.h b/table/table_builder.h index ee32cff8636..e5e7d6e22f7 100644 --- a/table/table_builder.h +++ b/table/table_builder.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,11 +9,79 @@ #pragma once +#include +#include +#include +#include +#include "db/table_properties_collector.h" +#include "options/cf_options.h" +#include "rocksdb/options.h" +#include "rocksdb/table_properties.h" +#include "util/file_reader_writer.h" + namespace rocksdb { class Slice; class Status; +struct TableReaderOptions { + // @param skip_filters Disables loading/accessing the filter block + TableReaderOptions(const ImmutableCFOptions& _ioptions, + const EnvOptions& _env_options, + const InternalKeyComparator& _internal_comparator, + bool _skip_filters = false, int _level = -1) + : ioptions(_ioptions), + env_options(_env_options), + internal_comparator(_internal_comparator), + skip_filters(_skip_filters), + level(_level) {} + + const ImmutableCFOptions& ioptions; + const EnvOptions& env_options; + const InternalKeyComparator& internal_comparator; + // This is only used for BlockBasedTable (reader) + bool skip_filters; + // what level this table/file is on, -1 for "not set, don't know" + int level; +}; + +struct TableBuilderOptions { + TableBuilderOptions( + const ImmutableCFOptions& _ioptions, + const InternalKeyComparator& _internal_comparator, + const std::vector>* + _int_tbl_prop_collector_factories, + CompressionType _compression_type, + const CompressionOptions& _compression_opts, + const std::string* _compression_dict, bool _skip_filters, + const std::string& _column_family_name, int _level, + const uint64_t _creation_time = 0, const int64_t _oldest_key_time = 0) + : ioptions(_ioptions), + internal_comparator(_internal_comparator), + int_tbl_prop_collector_factories(_int_tbl_prop_collector_factories), + compression_type(_compression_type), + compression_opts(_compression_opts), + compression_dict(_compression_dict), + skip_filters(_skip_filters), + column_family_name(_column_family_name), + level(_level), + creation_time(_creation_time), + oldest_key_time(_oldest_key_time) {} + const ImmutableCFOptions& ioptions; + const InternalKeyComparator& internal_comparator; + const std::vector>* + int_tbl_prop_collector_factories; + CompressionType compression_type; + const CompressionOptions& compression_opts; + // Data for presetting the compression library's dictionary, or nullptr. + const std::string* compression_dict; + bool skip_filters; // only used by BlockBasedTableBuilder + const std::string& column_family_name; + int level; // what level this table/file is on, -1 for "not set, don't know" + const uint64_t creation_time; + const int64_t oldest_key_time; +}; + // TableBuilder provides the interface used to build a Table // (an immutable and sorted map from keys to values). // @@ -50,6 +118,13 @@ class TableBuilder { // Size of the file generated so far. If invoked after a successful // Finish() call, returns the size of the final generated file. virtual uint64_t FileSize() const = 0; + + // If the user defined table properties collector suggest the file to + // be further compacted. + virtual bool NeedCompact() const { return false; } + + // Returns table properties + virtual TableProperties GetTableProperties() const = 0; }; } // namespace rocksdb diff --git a/table/table_properties.cc b/table/table_properties.cc index c7e14194379..24453f6f9cd 100644 --- a/table/table_properties.cc +++ b/table/table_properties.cc @@ -1,14 +1,22 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #include "rocksdb/table_properties.h" -#include "rocksdb/iterator.h" +#include "port/port.h" #include "rocksdb/env.h" +#include "rocksdb/iterator.h" +#include "table/block.h" +#include "table/internal_iterator.h" +#include "table/table_properties_internal.h" +#include "util/string_util.h" namespace rocksdb { +const uint32_t TablePropertiesCollectorFactory::Context::kUnknownColumnFamily = + port::kMaxInt32; + namespace { void AppendProperty( std::string& props, @@ -30,9 +38,34 @@ namespace { const std::string& prop_delim, const std::string& kv_delim) { AppendProperty( - props, key, std::to_string(value), prop_delim, kv_delim + props, key, ToString(value), prop_delim, kv_delim ); } + + // Seek to the specified meta block. + // Return true if it successfully seeks to that block. + Status SeekToMetaBlock(InternalIterator* meta_iter, + const std::string& block_name, bool* is_found, + BlockHandle* block_handle = nullptr) { + if (block_handle != nullptr) { + *block_handle = BlockHandle::NullBlockHandle(); + } + *is_found = true; + meta_iter->Seek(block_name); + if (meta_iter->status().ok()) { + if (meta_iter->Valid() && meta_iter->key() == block_name) { + *is_found = true; + if (block_handle) { + Slice v = meta_iter->value(); + return block_handle->DecodeFrom(&v); + } + } else { + *is_found = false; + return Status::OK(); + } + } + return meta_iter->status(); + } } std::string TableProperties::ToString( @@ -58,6 +91,12 @@ std::string TableProperties::ToString( AppendProperty(result, "data block size", data_size, prop_delim, kv_delim); AppendProperty(result, "index block size", index_size, prop_delim, kv_delim); + if (index_partitions != 0) { + AppendProperty(result, "# index partitions", index_partitions, prop_delim, + kv_delim); + AppendProperty(result, "top-level index size", top_level_index_size, prop_delim, + kv_delim); + } AppendProperty(result, "filter block size", filter_size, prop_delim, kv_delim); AppendProperty(result, "(estimated) table size", @@ -68,13 +107,64 @@ std::string TableProperties::ToString( filter_policy_name.empty() ? std::string("N/A") : filter_policy_name, prop_delim, kv_delim); + AppendProperty(result, "column family ID", + column_family_id == rocksdb::TablePropertiesCollectorFactory:: + Context::kUnknownColumnFamily + ? std::string("N/A") + : rocksdb::ToString(column_family_id), + prop_delim, kv_delim); + AppendProperty( + result, "column family name", + column_family_name.empty() ? std::string("N/A") : column_family_name, + prop_delim, kv_delim); + + AppendProperty(result, "comparator name", + comparator_name.empty() ? std::string("N/A") : comparator_name, + prop_delim, kv_delim); + + AppendProperty( + result, "merge operator name", + merge_operator_name.empty() ? std::string("N/A") : merge_operator_name, + prop_delim, kv_delim); + + AppendProperty(result, "property collectors names", + property_collectors_names.empty() ? std::string("N/A") + : property_collectors_names, + prop_delim, kv_delim); + + AppendProperty( + result, "SST file compression algo", + compression_name.empty() ? std::string("N/A") : compression_name, + prop_delim, kv_delim); + + AppendProperty(result, "creation time", creation_time, prop_delim, kv_delim); + + AppendProperty(result, "time stamp of earliest key", oldest_key_time, + prop_delim, kv_delim); + return result; } +void TableProperties::Add(const TableProperties& tp) { + data_size += tp.data_size; + index_size += tp.index_size; + index_partitions += tp.index_partitions; + top_level_index_size += tp.top_level_index_size; + filter_size += tp.filter_size; + raw_key_size += tp.raw_key_size; + raw_value_size += tp.raw_value_size; + num_data_blocks += tp.num_data_blocks; + num_entries += tp.num_entries; +} + const std::string TablePropertiesNames::kDataSize = "rocksdb.data.size"; const std::string TablePropertiesNames::kIndexSize = "rocksdb.index.size"; +const std::string TablePropertiesNames::kIndexPartitions = + "rocksdb.index.partitions"; +const std::string TablePropertiesNames::kTopLevelIndexSize = + "rocksdb.top-level.index.size"; const std::string TablePropertiesNames::kFilterSize = "rocksdb.filter.size"; const std::string TablePropertiesNames::kRawKeySize = @@ -91,25 +181,47 @@ const std::string TablePropertiesNames::kFormatVersion = "rocksdb.format.version"; const std::string TablePropertiesNames::kFixedKeyLen = "rocksdb.fixed.key.length"; +const std::string TablePropertiesNames::kColumnFamilyId = + "rocksdb.column.family.id"; +const std::string TablePropertiesNames::kColumnFamilyName = + "rocksdb.column.family.name"; +const std::string TablePropertiesNames::kComparator = "rocksdb.comparator"; +const std::string TablePropertiesNames::kMergeOperator = + "rocksdb.merge.operator"; +const std::string TablePropertiesNames::kPrefixExtractorName = + "rocksdb.prefix.extractor.name"; +const std::string TablePropertiesNames::kPropertyCollectors = + "rocksdb.property.collectors"; +const std::string TablePropertiesNames::kCompression = "rocksdb.compression"; +const std::string TablePropertiesNames::kCreationTime = "rocksdb.creation.time"; +const std::string TablePropertiesNames::kOldestKeyTime = + "rocksdb.oldest.key.time"; extern const std::string kPropertiesBlock = "rocksdb.properties"; // Old property block name for backward compatibility extern const std::string kPropertiesBlockOldName = "rocksdb.stats"; +extern const std::string kCompressionDictBlock = "rocksdb.compression_dict"; +extern const std::string kRangeDelBlock = "rocksdb.range_del"; // Seek to the properties block. // Return true if it successfully seeks to the properties block. -Status SeekToPropertiesBlock(Iterator* meta_iter, bool* is_found) { - *is_found = true; - meta_iter->Seek(kPropertiesBlock); - if (meta_iter->status().ok() && - (!meta_iter->Valid() || meta_iter->key() != kPropertiesBlock)) { - meta_iter->Seek(kPropertiesBlockOldName); - if (meta_iter->status().ok() && - (!meta_iter->Valid() || meta_iter->key() != kPropertiesBlockOldName)) { - *is_found = false; - } +Status SeekToPropertiesBlock(InternalIterator* meta_iter, bool* is_found) { + Status status = SeekToMetaBlock(meta_iter, kPropertiesBlock, is_found); + if (!*is_found && status.ok()) { + status = SeekToMetaBlock(meta_iter, kPropertiesBlockOldName, is_found); } - return meta_iter->status(); + return status; +} + +// Seek to the compression dictionary block. +// Return true if it successfully seeks to that block. +Status SeekToCompressionDictBlock(InternalIterator* meta_iter, bool* is_found) { + return SeekToMetaBlock(meta_iter, kCompressionDictBlock, is_found); +} + +Status SeekToRangeDelBlock(InternalIterator* meta_iter, bool* is_found, + BlockHandle* block_handle = nullptr) { + return SeekToMetaBlock(meta_iter, kRangeDelBlock, is_found, block_handle); } } // namespace rocksdb diff --git a/table/table_properties_internal.h b/table/table_properties_internal.h new file mode 100644 index 00000000000..2a89427341b --- /dev/null +++ b/table/table_properties_internal.h @@ -0,0 +1,30 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#pragma once + +#include "rocksdb/status.h" +#include "rocksdb/iterator.h" + +namespace rocksdb { + +class InternalIterator; +class BlockHandle; + +// Seek to the properties block. +// If it successfully seeks to the properties block, "is_found" will be +// set to true. +Status SeekToPropertiesBlock(InternalIterator* meta_iter, bool* is_found); + +// Seek to the compression dictionary block. +// If it successfully seeks to the properties block, "is_found" will be +// set to true. +Status SeekToCompressionDictBlock(InternalIterator* meta_iter, bool* is_found); + +// TODO(andrewkr) should not put all meta block in table_properties.h/cc +Status SeekToRangeDelBlock(InternalIterator* meta_iter, bool* is_found, + BlockHandle* block_handle); + +} // namespace rocksdb diff --git a/table/table_reader.h b/table/table_reader.h index 22f5a859e4f..18fcda27370 100644 --- a/table/table_reader.h +++ b/table/table_reader.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -9,6 +9,7 @@ #pragma once #include +#include "table/internal_iterator.h" namespace rocksdb { @@ -18,6 +19,8 @@ class Slice; class Arena; struct ReadOptions; struct TableProperties; +class GetContext; +class InternalIterator; // A Table is a sorted map from strings to strings. Tables are // immutable and persistent. A Table may be safely accessed from @@ -33,7 +36,16 @@ class TableReader { // When destroying the iterator, the caller will not call "delete" // but Iterator::~Iterator() directly. The destructor needs to destroy // all the states but those allocated in arena. - virtual Iterator* NewIterator(const ReadOptions&, Arena* arena = nullptr) = 0; + // skip_filters: disables checking the bloom filters even if they exist. This + // option is effective only for block-based table format. + virtual InternalIterator* NewIterator(const ReadOptions&, + Arena* arena = nullptr, + bool skip_filters = false) = 0; + + virtual InternalIterator* NewRangeTombstoneIterator( + const ReadOptions& read_options) { + return nullptr; + } // Given a key, return an approximate byte offset in the file where // the data for that key begins (or would begin if the key were @@ -55,23 +67,43 @@ class TableReader { // Report an approximation of how much memory has been used. virtual size_t ApproximateMemoryUsage() const = 0; - // Calls (*result_handler)(handle_context, ...) repeatedly, starting with - // the entry found after a call to Seek(key), until result_handler returns - // false, where k is the actual internal key for a row found and v as the - // value of the key. May not make such a call if filter policy says that key - // is not present. + // Calls get_context->SaveValue() repeatedly, starting with + // the entry found after a call to Seek(key), until it returns false. + // May not make such a call if filter policy says that key is not present. // - // mark_key_may_exist_handler needs to be called when it is configured to be - // memory only and the key is not found in the block cache, with - // the parameter to be handle_context. + // get_context->MarkKeyMayExist needs to be called when it is configured to be + // memory only and the key is not found in the block cache. // // readOptions is the options for the read // key is the key to search for - virtual Status Get( - const ReadOptions& readOptions, const Slice& key, void* handle_context, - bool (*result_handler)(void* arg, const ParsedInternalKey& k, - const Slice& v), - void (*mark_key_may_exist_handler)(void* handle_context) = nullptr) = 0; + // skip_filters: disables checking the bloom filters even if they exist. This + // option is effective only for block-based table format. + virtual Status Get(const ReadOptions& readOptions, const Slice& key, + GetContext* get_context, bool skip_filters = false) = 0; + + // Prefetch data corresponding to a give range of keys + // Typically this functionality is required for table implementations that + // persists the data on a non volatile storage medium like disk/SSD + virtual Status Prefetch(const Slice* begin = nullptr, + const Slice* end = nullptr) { + (void) begin; + (void) end; + // Default implementation is NOOP. + // The child class should implement functionality when applicable + return Status::OK(); + } + + // convert db file to a human readable form + virtual Status DumpTable(WritableFile* out_file) { + return Status::NotSupported("DumpTable() not supported"); + } + + // check whether there is corruption in this db file + virtual Status VerifyChecksum() { + return Status::NotSupported("VerifyChecksum() not supported"); + } + + virtual void Close() {} }; } // namespace rocksdb diff --git a/table/table_reader_bench.cc b/table/table_reader_bench.cc index ed2c7c52dc8..85e48c1fea7 100644 --- a/table/table_reader_bench.cc +++ b/table/table_reader_bench.cc @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). #ifndef GFLAGS #include @@ -13,16 +13,18 @@ int main() { #include +#include "db/db_impl.h" +#include "db/dbformat.h" +#include "monitoring/histogram.h" #include "rocksdb/db.h" #include "rocksdb/slice_transform.h" #include "rocksdb/table.h" -#include "db/db_impl.h" -#include "db/dbformat.h" -#include "port/atomic_pointer.h" #include "table/block_based_table_factory.h" +#include "table/get_context.h" +#include "table/internal_iterator.h" #include "table/plain_table_factory.h" #include "table/table_builder.h" -#include "util/histogram.h" +#include "util/file_reader_writer.h" #include "util/testharness.h" #include "util/testutil.h" @@ -48,11 +50,6 @@ static std::string MakeKey(int i, int j, bool through_db) { return key.Encode().ToString(); } -static bool DummySaveValue(void* arg, const ParsedInternalKey& ikey, - const Slice& v) { - return false; -} - uint64_t Now(Env* env, bool measured_by_nanosecond) { return measured_by_nanosecond ? env->NowNanos() : env->NowMicros(); } @@ -83,15 +80,29 @@ void TableReaderBenchmark(Options& opts, EnvOptions& env_options, + "/rocksdb_table_reader_benchmark"; std::string dbname = test::TmpDir() + "/rocksdb_table_reader_bench_db"; WriteOptions wo; - unique_ptr file; Env* env = Env::Default(); TableBuilder* tb = nullptr; DB* db = nullptr; Status s; + const ImmutableCFOptions ioptions(opts); + unique_ptr file_writer; if (!through_db) { + unique_ptr file; env->NewWritableFile(file_name, &file, env_options); - tb = opts.table_factory->NewTableBuilder(opts, ikc, file.get(), - CompressionType::kNoCompression); + + std::vector > + int_tbl_prop_collector_factories; + + file_writer.reset(new WritableFileWriter(std::move(file), env_options)); + int unknown_level = -1; + tb = opts.table_factory->NewTableBuilder( + TableBuilderOptions(ioptions, ikc, &int_tbl_prop_collector_factories, + CompressionType::kNoCompression, + CompressionOptions(), + nullptr /* compression_dict */, + false /* skip_filters */, kDefaultColumnFamilyName, + unknown_level), + 0 /* column_family_id */, file_writer.get()); } else { s = DB::Open(opts, dbname, &db); ASSERT_OK(s); @@ -110,26 +121,36 @@ void TableReaderBenchmark(Options& opts, EnvOptions& env_options, } if (!through_db) { tb->Finish(); - file->Close(); + file_writer->Close(); } else { db->Flush(FlushOptions()); } unique_ptr table_reader; - unique_ptr raf; if (!through_db) { - Status s = env->NewRandomAccessFile(file_name, &raf, env_options); + unique_ptr raf; + s = env->NewRandomAccessFile(file_name, &raf, env_options); + if (!s.ok()) { + fprintf(stderr, "Create File Error: %s\n", s.ToString().c_str()); + exit(1); + } uint64_t file_size; env->GetFileSize(file_name, &file_size); + unique_ptr file_reader( + new RandomAccessFileReader(std::move(raf), file_name)); s = opts.table_factory->NewTableReader( - opts, env_options, ikc, std::move(raf), file_size, &table_reader); + TableReaderOptions(ioptions, env_options, ikc), std::move(file_reader), + file_size, &table_reader); + if (!s.ok()) { + fprintf(stderr, "Open Table Error: %s\n", s.ToString().c_str()); + exit(1); + } } Random rnd(301); std::string result; HistogramImpl hist; - void* arg = nullptr; for (int it = 0; it < num_iter; it++) { for (int i = 0; i < num_keys1; i++) { for (int j = 0; j < num_keys2; j++) { @@ -145,8 +166,15 @@ void TableReaderBenchmark(Options& opts, EnvOptions& env_options, std::string key = MakeKey(r1, r2, through_db); uint64_t start_time = Now(env, measured_by_nanosecond); if (!through_db) { - s = table_reader->Get(read_options, key, arg, DummySaveValue, - nullptr); + PinnableSlice value; + MergeContext merge_context; + RangeDelAggregator range_del_agg(ikc, {} /* snapshots */); + GetContext get_context(ioptions.user_comparator, + ioptions.merge_operator, ioptions.info_log, + ioptions.statistics, GetContext::kNotFound, + Slice(key), &value, nullptr, &merge_context, + &range_del_agg, env); + s = table_reader->Get(read_options, key, &get_context); } else { s = db->Get(read_options, key, &result); } @@ -165,20 +193,24 @@ void TableReaderBenchmark(Options& opts, EnvOptions& env_options, std::string end_key = MakeKey(r1, r2 + r2_len, through_db); uint64_t total_time = 0; uint64_t start_time = Now(env, measured_by_nanosecond); - Iterator* iter; + Iterator* iter = nullptr; + InternalIterator* iiter = nullptr; if (!through_db) { - iter = table_reader->NewIterator(read_options); + iiter = table_reader->NewIterator(read_options); } else { iter = db->NewIterator(read_options); } int count = 0; - for(iter->Seek(start_key); iter->Valid(); iter->Next()) { + for (through_db ? iter->Seek(start_key) : iiter->Seek(start_key); + through_db ? iter->Valid() : iiter->Valid(); + through_db ? iter->Next() : iiter->Next()) { if (if_query_empty_keys) { break; } // verify key; total_time += Now(env, measured_by_nanosecond) - start_time; - assert(Slice(MakeKey(r1, r2 + count, through_db)) == iter->key()); + assert(Slice(MakeKey(r1, r2 + count, through_db)) == + (through_db ? iter->key() : iiter->key())); start_time = Now(env, measured_by_nanosecond); if (++count >= r2_len) { break; @@ -232,6 +264,7 @@ DEFINE_bool(iterator, false, "For test iterator"); DEFINE_bool(through_db, false, "If enable, a DB instance will be created and " "the query will be against DB. Otherwise, will be directly against " "a table reader."); +DEFINE_bool(mmap_read, true, "Whether use mmap read"); DEFINE_string(table_factory, "block_based", "Table factory to use: `block_based` (default), `plain_table` or " "`cuckoo_hash`."); @@ -256,13 +289,20 @@ int main(int argc, char** argv) { options.compression = rocksdb::CompressionType::kNoCompression; if (FLAGS_table_factory == "cuckoo_hash") { - options.allow_mmap_reads = true; - env_options.use_mmap_reads = true; - - tf.reset(rocksdb::NewCuckooTableFactory(0.75)); +#ifndef ROCKSDB_LITE + options.allow_mmap_reads = FLAGS_mmap_read; + env_options.use_mmap_reads = FLAGS_mmap_read; + rocksdb::CuckooTableOptions table_options; + table_options.hash_table_ratio = 0.75; + tf.reset(rocksdb::NewCuckooTableFactory(table_options)); +#else + fprintf(stderr, "Plain table is not supported in lite mode\n"); + exit(1); +#endif // ROCKSDB_LITE } else if (FLAGS_table_factory == "plain_table") { - options.allow_mmap_reads = true; - env_options.use_mmap_reads = true; +#ifndef ROCKSDB_LITE + options.allow_mmap_reads = FLAGS_mmap_read; + env_options.use_mmap_reads = FLAGS_mmap_read; rocksdb::PlainTableOptions plain_table_options; plain_table_options.user_key_len = 16; @@ -272,6 +312,10 @@ int main(int argc, char** argv) { tf.reset(new rocksdb::PlainTableFactory(plain_table_options)); options.prefix_extractor.reset(rocksdb::NewFixedPrefixTransform( FLAGS_prefix_len)); +#else + fprintf(stderr, "Cuckoo table is not supported in lite mode\n"); + exit(1); +#endif // ROCKSDB_LITE } else if (FLAGS_table_factory == "block_based") { tf.reset(new rocksdb::BlockBasedTableFactory()); } else { diff --git a/table/table_test.cc b/table/table_test.cc index 500abf48f24..178cf4243d7 100644 --- a/table/table_test.cc +++ b/table/table_test.cc @@ -1,47 +1,56 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. -#include #include #include #include #include -#include #include +#include #include +#include "cache/lru_cache.h" #include "db/dbformat.h" #include "db/memtable.h" #include "db/write_batch_internal.h" - +#include "memtable/stl_wrappers.h" +#include "monitoring/statistics.h" +#include "port/port.h" #include "rocksdb/cache.h" #include "rocksdb/db.h" #include "rocksdb/env.h" #include "rocksdb/iterator.h" #include "rocksdb/memtablerep.h" +#include "rocksdb/perf_context.h" #include "rocksdb/slice_transform.h" #include "rocksdb/statistics.h" - +#include "rocksdb/write_buffer_manager.h" #include "table/block.h" #include "table/block_based_table_builder.h" #include "table/block_based_table_factory.h" #include "table/block_based_table_reader.h" #include "table/block_builder.h" #include "table/format.h" +#include "table/get_context.h" +#include "table/internal_iterator.h" #include "table/meta_blocks.h" #include "table/plain_table_factory.h" - +#include "table/scoped_arena_iterator.h" +#include "table/sst_file_writer_collectors.h" +#include "util/compression.h" #include "util/random.h" -#include "util/statistics.h" +#include "util/string_util.h" +#include "util/sync_point.h" #include "util/testharness.h" #include "util/testutil.h" +#include "utilities/merge_operators.h" namespace rocksdb { @@ -52,6 +61,40 @@ extern const uint64_t kPlainTableMagicNumber; namespace { +// DummyPropertiesCollector used to test BlockBasedTableProperties +class DummyPropertiesCollector : public TablePropertiesCollector { + public: + const char* Name() const { return ""; } + + Status Finish(UserCollectedProperties* properties) { return Status::OK(); } + + Status Add(const Slice& user_key, const Slice& value) { return Status::OK(); } + + virtual UserCollectedProperties GetReadableProperties() const { + return UserCollectedProperties{}; + } +}; + +class DummyPropertiesCollectorFactory1 + : public TablePropertiesCollectorFactory { + public: + virtual TablePropertiesCollector* CreateTablePropertiesCollector( + TablePropertiesCollectorFactory::Context context) { + return new DummyPropertiesCollector(); + } + const char* Name() const { return "DummyPropertiesCollector1"; } +}; + +class DummyPropertiesCollectorFactory2 + : public TablePropertiesCollectorFactory { + public: + virtual TablePropertiesCollector* CreateTablePropertiesCollector( + TablePropertiesCollectorFactory::Context context) { + return new DummyPropertiesCollector(); + } + const char* Name() const { return "DummyPropertiesCollector2"; } +}; + // Return reverse of "key". // Used to test non-lexicographic comparators. std::string Reverse(const Slice& key) { @@ -62,24 +105,23 @@ std::string Reverse(const Slice& key) { class ReverseKeyComparator : public Comparator { public: - virtual const char* Name() const { + virtual const char* Name() const override { return "rocksdb.ReverseBytewiseComparator"; } - virtual int Compare(const Slice& a, const Slice& b) const { + virtual int Compare(const Slice& a, const Slice& b) const override { return BytewiseComparator()->Compare(Reverse(a), Reverse(b)); } - virtual void FindShortestSeparator( - std::string* start, - const Slice& limit) const { + virtual void FindShortestSeparator(std::string* start, + const Slice& limit) const override { std::string s = Reverse(*start); std::string l = Reverse(limit); BytewiseComparator()->FindShortestSeparator(&s, l); *start = Reverse(s); } - virtual void FindShortSuccessor(std::string* key) const { + virtual void FindShortSuccessor(std::string* key) const override { std::string s = Reverse(*key); BytewiseComparator()->FindShortSuccessor(&s); *key = Reverse(s); @@ -99,91 +141,14 @@ void Increment(const Comparator* cmp, std::string* key) { } } -// An STL comparator that uses a Comparator -struct STLLessThan { - const Comparator* cmp; - - STLLessThan() : cmp(BytewiseComparator()) { } - explicit STLLessThan(const Comparator* c) : cmp(c) { } - bool operator()(const std::string& a, const std::string& b) const { - return cmp->Compare(Slice(a), Slice(b)) < 0; - } -}; - } // namespace -class StringSink: public WritableFile { - public: - ~StringSink() { } - - const std::string& contents() const { return contents_; } - - virtual Status Close() { return Status::OK(); } - virtual Status Flush() { return Status::OK(); } - virtual Status Sync() { return Status::OK(); } - - virtual Status Append(const Slice& data) { - contents_.append(data.data(), data.size()); - return Status::OK(); - } - - private: - std::string contents_; -}; - - -class StringSource: public RandomAccessFile { - public: - StringSource(const Slice& contents, uint64_t uniq_id, bool mmap) - : contents_(contents.data(), contents.size()), uniq_id_(uniq_id), - mmap_(mmap) { - } - - virtual ~StringSource() { } - - uint64_t Size() const { return contents_.size(); } - - virtual Status Read(uint64_t offset, size_t n, Slice* result, - char* scratch) const { - if (offset > contents_.size()) { - return Status::InvalidArgument("invalid Read offset"); - } - if (offset + n > contents_.size()) { - n = contents_.size() - offset; - } - if (!mmap_) { - memcpy(scratch, &contents_[offset], n); - *result = Slice(scratch, n); - } else { - *result = Slice(&contents_[offset], n); - } - return Status::OK(); - } - - virtual size_t GetUniqueId(char* id, size_t max_size) const { - if (max_size < 20) { - return 0; - } - - char* rid = id; - rid = EncodeVarint64(rid, uniq_id_); - rid = EncodeVarint64(rid, 0); - return static_cast(rid-id); - } - - private: - std::string contents_; - uint64_t uniq_id_; - bool mmap_; -}; - -typedef std::map KVMap; - // Helper class for tests to unify the interface between // BlockBuilder/TableBuilder and Block/Table. class Constructor { public: - explicit Constructor(const Comparator* cmp) : data_(STLLessThan(cmp)) {} + explicit Constructor(const Comparator* cmp) + : data_(stl_wrappers::LessOfComparator(cmp)) {} virtual ~Constructor() { } void Add(const std::string& key, const Slice& value) { @@ -193,40 +158,44 @@ class Constructor { // Finish constructing the data structure with all the keys that have // been added so far. Returns the keys in sorted order in "*keys" // and stores the key/value pairs in "*kvmap" - void Finish(const Options& options, + void Finish(const Options& options, const ImmutableCFOptions& ioptions, const BlockBasedTableOptions& table_options, const InternalKeyComparator& internal_comparator, - std::vector* keys, KVMap* kvmap) { + std::vector* keys, stl_wrappers::KVMap* kvmap) { last_internal_key_ = &internal_comparator; *kvmap = data_; keys->clear(); - for (KVMap::const_iterator it = data_.begin(); - it != data_.end(); - ++it) { - keys->push_back(it->first); + for (const auto& kv : data_) { + keys->push_back(kv.first); } data_.clear(); - Status s = FinishImpl(options, table_options, internal_comparator, *kvmap); + Status s = FinishImpl(options, ioptions, table_options, + internal_comparator, *kvmap); ASSERT_TRUE(s.ok()) << s.ToString(); } // Construct the data structure from the data in "data" virtual Status FinishImpl(const Options& options, + const ImmutableCFOptions& ioptions, const BlockBasedTableOptions& table_options, const InternalKeyComparator& internal_comparator, - const KVMap& data) = 0; + const stl_wrappers::KVMap& data) = 0; - virtual Iterator* NewIterator() const = 0; + virtual InternalIterator* NewIterator() const = 0; - virtual const KVMap& data() { return data_; } + virtual const stl_wrappers::KVMap& data() { return data_; } + + virtual bool IsArenaMode() const { return false; } virtual DB* db() const { return nullptr; } // Overridden in DBConstructor + virtual bool AnywayDeleteIterator() const { return false; } + protected: const InternalKeyComparator* last_internal_key_; private: - KVMap data_; + stl_wrappers::KVMap data_; }; class BlockConstructor: public Constructor { @@ -239,28 +208,26 @@ class BlockConstructor: public Constructor { delete block_; } virtual Status FinishImpl(const Options& options, + const ImmutableCFOptions& ioptions, const BlockBasedTableOptions& table_options, const InternalKeyComparator& internal_comparator, - const KVMap& data) { + const stl_wrappers::KVMap& kv_map) override { delete block_; block_ = nullptr; BlockBuilder builder(table_options.block_restart_interval); - for (KVMap::const_iterator it = data.begin(); - it != data.end(); - ++it) { - builder.Add(it->first, it->second); + for (const auto kv : kv_map) { + builder.Add(kv.first, kv.second); } // Open the block data_ = builder.Finish().ToString(); BlockContents contents; contents.data = data_; contents.cachable = false; - contents.heap_allocated = false; - block_ = new Block(contents); + block_ = new Block(std::move(contents), kDisableGlobalSequenceNumber); return Status::OK(); } - virtual Iterator* NewIterator() const { + virtual InternalIterator* NewIterator() const override { return block_->NewIterator(comparator_); } @@ -273,40 +240,55 @@ class BlockConstructor: public Constructor { }; // A helper class that converts internal format keys into user keys -class KeyConvertingIterator: public Iterator { +class KeyConvertingIterator : public InternalIterator { public: - explicit KeyConvertingIterator(Iterator* iter) : iter_(iter) { } - virtual ~KeyConvertingIterator() { delete iter_; } - virtual bool Valid() const { return iter_->Valid(); } - virtual void Seek(const Slice& target) { + explicit KeyConvertingIterator(InternalIterator* iter, + bool arena_mode = false) + : iter_(iter), arena_mode_(arena_mode) {} + virtual ~KeyConvertingIterator() { + if (arena_mode_) { + iter_->~InternalIterator(); + } else { + delete iter_; + } + } + virtual bool Valid() const override { return iter_->Valid(); } + virtual void Seek(const Slice& target) override { ParsedInternalKey ikey(target, kMaxSequenceNumber, kTypeValue); std::string encoded; AppendInternalKey(&encoded, ikey); iter_->Seek(encoded); } - virtual void SeekToFirst() { iter_->SeekToFirst(); } - virtual void SeekToLast() { iter_->SeekToLast(); } - virtual void Next() { iter_->Next(); } - virtual void Prev() { iter_->Prev(); } + virtual void SeekForPrev(const Slice& target) override { + ParsedInternalKey ikey(target, kMaxSequenceNumber, kTypeValue); + std::string encoded; + AppendInternalKey(&encoded, ikey); + iter_->SeekForPrev(encoded); + } + virtual void SeekToFirst() override { iter_->SeekToFirst(); } + virtual void SeekToLast() override { iter_->SeekToLast(); } + virtual void Next() override { iter_->Next(); } + virtual void Prev() override { iter_->Prev(); } - virtual Slice key() const { + virtual Slice key() const override { assert(Valid()); - ParsedInternalKey key; - if (!ParseInternalKey(iter_->key(), &key)) { + ParsedInternalKey parsed_key; + if (!ParseInternalKey(iter_->key(), &parsed_key)) { status_ = Status::Corruption("malformed internal key"); return Slice("corrupted key"); } - return key.user_key; + return parsed_key.user_key; } - virtual Slice value() const { return iter_->value(); } - virtual Status status() const { + virtual Slice value() const override { return iter_->value(); } + virtual Status status() const override { return status_.ok() ? iter_->status() : status_; } private: mutable Status status_; - Iterator* iter_; + InternalIterator* iter_; + bool arena_mode_; // No copying allowed KeyConvertingIterator(const KeyConvertingIterator&); @@ -322,45 +304,57 @@ class TableConstructor: public Constructor { ~TableConstructor() { Reset(); } virtual Status FinishImpl(const Options& options, + const ImmutableCFOptions& ioptions, const BlockBasedTableOptions& table_options, const InternalKeyComparator& internal_comparator, - const KVMap& data) { + const stl_wrappers::KVMap& kv_map) override { Reset(); - sink_.reset(new StringSink()); + soptions.use_mmap_reads = ioptions.allow_mmap_reads; + file_writer_.reset(test::GetWritableFileWriter(new test::StringSink())); unique_ptr builder; - builder.reset(options.table_factory->NewTableBuilder( - options, internal_comparator, sink_.get(), options.compression)); - - for (KVMap::const_iterator it = data.begin(); - it != data.end(); - ++it) { + std::vector> + int_tbl_prop_collector_factories; + std::string column_family_name; + int unknown_level = -1; + builder.reset(ioptions.table_factory->NewTableBuilder( + TableBuilderOptions(ioptions, internal_comparator, + &int_tbl_prop_collector_factories, + options.compression, CompressionOptions(), + nullptr /* compression_dict */, + false /* skip_filters */, column_family_name, + unknown_level), + TablePropertiesCollectorFactory::Context::kUnknownColumnFamily, + file_writer_.get())); + + for (const auto kv : kv_map) { if (convert_to_internal_key_) { - ParsedInternalKey ikey(it->first, kMaxSequenceNumber, kTypeValue); + ParsedInternalKey ikey(kv.first, kMaxSequenceNumber, kTypeValue); std::string encoded; AppendInternalKey(&encoded, ikey); - builder->Add(encoded, it->second); + builder->Add(encoded, kv.second); } else { - builder->Add(it->first, it->second); + builder->Add(kv.first, kv.second); } - ASSERT_TRUE(builder->status().ok()); + EXPECT_TRUE(builder->status().ok()); } Status s = builder->Finish(); - ASSERT_TRUE(s.ok()) << s.ToString(); + file_writer_->Flush(); + EXPECT_TRUE(s.ok()) << s.ToString(); - ASSERT_EQ(sink_->contents().size(), builder->FileSize()); + EXPECT_EQ(GetSink()->contents().size(), builder->FileSize()); // Open the table uniq_id_ = cur_uniq_id_++; - source_.reset(new StringSource(sink_->contents(), uniq_id_, - options.allow_mmap_reads)); - return options.table_factory->NewTableReader( - options, soptions, internal_comparator, std::move(source_), - sink_->contents().size(), &table_reader_); + file_reader_.reset(test::GetRandomAccessFileReader(new test::StringSource( + GetSink()->contents(), uniq_id_, ioptions.allow_mmap_reads))); + return ioptions.table_factory->NewTableReader( + TableReaderOptions(ioptions, soptions, internal_comparator), + std::move(file_reader_), GetSink()->contents().size(), &table_reader_); } - virtual Iterator* NewIterator() const { + virtual InternalIterator* NewIterator() const override { ReadOptions ro; - Iterator* iter = table_reader_->NewIterator(ro); + InternalIterator* iter = table_reader_->NewIterator(ro); if (convert_to_internal_key_) { return new KeyConvertingIterator(iter); } else { @@ -369,85 +363,131 @@ class TableConstructor: public Constructor { } uint64_t ApproximateOffsetOf(const Slice& key) const { + if (convert_to_internal_key_) { + InternalKey ikey(key, kMaxSequenceNumber, kTypeValue); + const Slice skey = ikey.Encode(); + return table_reader_->ApproximateOffsetOf(skey); + } return table_reader_->ApproximateOffsetOf(key); } - virtual Status Reopen(const Options& options) { - source_.reset( - new StringSource(sink_->contents(), uniq_id_, - options.allow_mmap_reads)); - return options.table_factory->NewTableReader( - options, soptions, *last_internal_key_, std::move(source_), - sink_->contents().size(), &table_reader_); + virtual Status Reopen(const ImmutableCFOptions& ioptions) { + file_reader_.reset(test::GetRandomAccessFileReader(new test::StringSource( + GetSink()->contents(), uniq_id_, ioptions.allow_mmap_reads))); + return ioptions.table_factory->NewTableReader( + TableReaderOptions(ioptions, soptions, *last_internal_key_), + std::move(file_reader_), GetSink()->contents().size(), &table_reader_); } virtual TableReader* GetTableReader() { return table_reader_.get(); } + virtual bool AnywayDeleteIterator() const override { + return convert_to_internal_key_; + } + + void ResetTableReader() { table_reader_.reset(); } + + bool ConvertToInternalKey() { return convert_to_internal_key_; } + private: void Reset() { uniq_id_ = 0; table_reader_.reset(); - sink_.reset(); - source_.reset(); + file_writer_.reset(); + file_reader_.reset(); + } + + test::StringSink* GetSink() { + return static_cast(file_writer_->writable_file()); } - bool convert_to_internal_key_; uint64_t uniq_id_; - unique_ptr sink_; - unique_ptr source_; + unique_ptr file_writer_; + unique_ptr file_reader_; unique_ptr table_reader_; + bool convert_to_internal_key_; TableConstructor(); static uint64_t cur_uniq_id_; - const EnvOptions soptions; + EnvOptions soptions; }; uint64_t TableConstructor::cur_uniq_id_ = 1; class MemTableConstructor: public Constructor { public: - explicit MemTableConstructor(const Comparator* cmp) + explicit MemTableConstructor(const Comparator* cmp, WriteBufferManager* wb) : Constructor(cmp), internal_comparator_(cmp), + write_buffer_manager_(wb), table_factory_(new SkipListFactory) { - Options options; - options.memtable_factory = table_factory_; - memtable_ = new MemTable(internal_comparator_, options); + options_.memtable_factory = table_factory_; + ImmutableCFOptions ioptions(options_); + memtable_ = + new MemTable(internal_comparator_, ioptions, MutableCFOptions(options_), + wb, kMaxSequenceNumber, 0 /* column_family_id */); memtable_->Ref(); } ~MemTableConstructor() { delete memtable_->Unref(); } - virtual Status FinishImpl(const Options& options, + virtual Status FinishImpl(const Options&, const ImmutableCFOptions& ioptions, const BlockBasedTableOptions& table_options, const InternalKeyComparator& internal_comparator, - const KVMap& data) { + const stl_wrappers::KVMap& kv_map) override { delete memtable_->Unref(); - Options memtable_options; - memtable_options.memtable_factory = table_factory_; - memtable_ = new MemTable(internal_comparator_, memtable_options); + ImmutableCFOptions mem_ioptions(ioptions); + memtable_ = new MemTable(internal_comparator_, mem_ioptions, + MutableCFOptions(options_), write_buffer_manager_, + kMaxSequenceNumber, 0 /* column_family_id */); memtable_->Ref(); int seq = 1; - for (KVMap::const_iterator it = data.begin(); - it != data.end(); - ++it) { - memtable_->Add(seq, kTypeValue, it->first, it->second); + for (const auto kv : kv_map) { + memtable_->Add(seq, kTypeValue, kv.first, kv.second); seq++; } return Status::OK(); } - virtual Iterator* NewIterator() const { - return new KeyConvertingIterator(memtable_->NewIterator(ReadOptions())); + virtual InternalIterator* NewIterator() const override { + return new KeyConvertingIterator( + memtable_->NewIterator(ReadOptions(), &arena_), true); } + virtual bool AnywayDeleteIterator() const override { return true; } + + virtual bool IsArenaMode() const override { return true; } + private: + mutable Arena arena_; InternalKeyComparator internal_comparator_; + Options options_; + WriteBufferManager* write_buffer_manager_; MemTable* memtable_; std::shared_ptr table_factory_; }; +class InternalIteratorFromIterator : public InternalIterator { + public: + explicit InternalIteratorFromIterator(Iterator* it) : it_(it) {} + virtual bool Valid() const override { return it_->Valid(); } + virtual void Seek(const Slice& target) override { it_->Seek(target); } + virtual void SeekForPrev(const Slice& target) override { + it_->SeekForPrev(target); + } + virtual void SeekToFirst() override { it_->SeekToFirst(); } + virtual void SeekToLast() override { it_->SeekToLast(); } + virtual void Next() override { it_->Next(); } + virtual void Prev() override { it_->Prev(); } + Slice key() const override { return it_->key(); } + Slice value() const override { return it_->value(); } + virtual Status status() const override { return it_->status(); } + + private: + unique_ptr it_; +}; + class DBConstructor: public Constructor { public: explicit DBConstructor(const Comparator* cmp) @@ -460,26 +500,26 @@ class DBConstructor: public Constructor { delete db_; } virtual Status FinishImpl(const Options& options, + const ImmutableCFOptions& ioptions, const BlockBasedTableOptions& table_options, const InternalKeyComparator& internal_comparator, - const KVMap& data) { + const stl_wrappers::KVMap& kv_map) override { delete db_; db_ = nullptr; NewDB(); - for (KVMap::const_iterator it = data.begin(); - it != data.end(); - ++it) { + for (const auto kv : kv_map) { WriteBatch batch; - batch.Put(it->first, it->second); - ASSERT_TRUE(db_->Write(WriteOptions(), &batch).ok()); + batch.Put(kv.first, kv.second); + EXPECT_TRUE(db_->Write(WriteOptions(), &batch).ok()); } return Status::OK(); } - virtual Iterator* NewIterator() const { - return db_->NewIterator(ReadOptions()); + + virtual InternalIterator* NewIterator() const override { + return new InternalIteratorFromIterator(db_->NewIterator(ReadOptions())); } - virtual DB* db() const { return db_; } + virtual DB* db() const override { return db_; } private: void NewDB() { @@ -501,69 +541,13 @@ class DBConstructor: public Constructor { DB* db_; }; -static bool SnappyCompressionSupported() { -#ifdef SNAPPY - std::string out; - Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; - return port::Snappy_Compress(Options().compression_opts, - in.data(), in.size(), - &out); -#else - return false; -#endif -} - -static bool ZlibCompressionSupported() { -#ifdef ZLIB - std::string out; - Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; - return port::Zlib_Compress(Options().compression_opts, - in.data(), in.size(), - &out); -#else - return false; -#endif -} - -static bool BZip2CompressionSupported() { -#ifdef BZIP2 - std::string out; - Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; - return port::BZip2_Compress(Options().compression_opts, - in.data(), in.size(), - &out); -#else - return false; -#endif -} - -static bool LZ4CompressionSupported() { -#ifdef LZ4 - std::string out; - Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; - return port::LZ4_Compress(Options().compression_opts, in.data(), in.size(), - &out); -#else - return false; -#endif -} - -static bool LZ4HCCompressionSupported() { -#ifdef LZ4 - std::string out; - Slice in = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; - return port::LZ4HC_Compress(Options().compression_opts, in.data(), in.size(), - &out); -#else - return false; -#endif -} - enum TestType { BLOCK_BASED_TABLE_TEST, +#ifndef ROCKSDB_LITE PLAIN_TABLE_SEMI_FIXED_PREFIX, PLAIN_TABLE_FULL_STR_PREFIX, PLAIN_TABLE_TOTAL_ORDER, +#endif // !ROCKSDB_LITE BLOCK_TEST, MEMTABLE_TEST, DB_TEST @@ -574,50 +558,72 @@ struct TestArgs { bool reverse_compare; int restart_interval; CompressionType compression; + uint32_t format_version; + bool use_mmap; }; static std::vector GenerateArgList() { std::vector test_args; std::vector test_types = { - BLOCK_BASED_TABLE_TEST, PLAIN_TABLE_SEMI_FIXED_PREFIX, - PLAIN_TABLE_FULL_STR_PREFIX, PLAIN_TABLE_TOTAL_ORDER, - BLOCK_TEST, MEMTABLE_TEST, - DB_TEST}; + BLOCK_BASED_TABLE_TEST, +#ifndef ROCKSDB_LITE + PLAIN_TABLE_SEMI_FIXED_PREFIX, + PLAIN_TABLE_FULL_STR_PREFIX, + PLAIN_TABLE_TOTAL_ORDER, +#endif // !ROCKSDB_LITE + BLOCK_TEST, + MEMTABLE_TEST, DB_TEST}; std::vector reverse_compare_types = {false, true}; std::vector restart_intervals = {16, 1, 1024}; // Only add compression if it is supported - std::vector compression_types; - compression_types.push_back(kNoCompression); - if (SnappyCompressionSupported()) { - compression_types.push_back(kSnappyCompression); + std::vector> compression_types; + compression_types.emplace_back(kNoCompression, false); + if (Snappy_Supported()) { + compression_types.emplace_back(kSnappyCompression, false); + } + if (Zlib_Supported()) { + compression_types.emplace_back(kZlibCompression, false); + compression_types.emplace_back(kZlibCompression, true); } - if (ZlibCompressionSupported()) { - compression_types.push_back(kZlibCompression); + if (BZip2_Supported()) { + compression_types.emplace_back(kBZip2Compression, false); + compression_types.emplace_back(kBZip2Compression, true); } - if (BZip2CompressionSupported()) { - compression_types.push_back(kBZip2Compression); + if (LZ4_Supported()) { + compression_types.emplace_back(kLZ4Compression, false); + compression_types.emplace_back(kLZ4Compression, true); + compression_types.emplace_back(kLZ4HCCompression, false); + compression_types.emplace_back(kLZ4HCCompression, true); } - if (LZ4CompressionSupported()) { - compression_types.push_back(kLZ4Compression); + if (XPRESS_Supported()) { + compression_types.emplace_back(kXpressCompression, false); + compression_types.emplace_back(kXpressCompression, true); } - if (LZ4HCCompressionSupported()) { - compression_types.push_back(kLZ4HCCompression); + if (ZSTD_Supported()) { + compression_types.emplace_back(kZSTD, false); + compression_types.emplace_back(kZSTD, true); } for (auto test_type : test_types) { for (auto reverse_compare : reverse_compare_types) { +#ifndef ROCKSDB_LITE if (test_type == PLAIN_TABLE_SEMI_FIXED_PREFIX || - test_type == PLAIN_TABLE_FULL_STR_PREFIX) { + test_type == PLAIN_TABLE_FULL_STR_PREFIX || + test_type == PLAIN_TABLE_TOTAL_ORDER) { // Plain table doesn't use restart index or compression. TestArgs one_arg; one_arg.type = test_type; one_arg.reverse_compare = reverse_compare; one_arg.restart_interval = restart_intervals[0]; - one_arg.compression = compression_types[0]; + one_arg.compression = compression_types[0].first; + one_arg.use_mmap = true; + test_args.push_back(one_arg); + one_arg.use_mmap = false; test_args.push_back(one_arg); continue; } +#endif // !ROCKSDB_LITE for (auto restart_interval : restart_intervals) { for (auto compression_type : compression_types) { @@ -625,7 +631,9 @@ static std::vector GenerateArgList() { one_arg.type = test_type; one_arg.reverse_compare = reverse_compare; one_arg.restart_interval = restart_interval; - one_arg.compression = compression_type; + one_arg.compression = compression_type.first; + one_arg.format_version = compression_type.second ? 2 : 1; + one_arg.use_mmap = false; test_args.push_back(one_arg); } } @@ -647,11 +655,9 @@ class FixedOrLessPrefixTransform : public SliceTransform { prefix_len_(prefix_len) { } - virtual const char* Name() const { - return "rocksdb.FixedPrefix"; - } + virtual const char* Name() const override { return "rocksdb.FixedPrefix"; } - virtual Slice Transform(const Slice& src) const { + virtual Slice Transform(const Slice& src) const override { assert(InDomain(src)); if (src.size() < prefix_len_) { return src; @@ -659,18 +665,19 @@ class FixedOrLessPrefixTransform : public SliceTransform { return Slice(src.data(), prefix_len_); } - virtual bool InDomain(const Slice& src) const { - return true; - } + virtual bool InDomain(const Slice& src) const override { return true; } - virtual bool InRange(const Slice& dst) const { + virtual bool InRange(const Slice& dst) const override { return (dst.size() <= prefix_len_); } }; -class Harness { +class HarnessTest : public testing::Test { public: - Harness() : constructor_(nullptr) { } + HarnessTest() + : ioptions_(options_), + constructor_(nullptr), + write_buffer_(options_.db_write_buffer_size) {} void Init(const TestArgs& args) { delete constructor_; @@ -688,23 +695,31 @@ class Harness { support_prev_ = true; only_support_prefix_seek_ = false; + options_.allow_mmap_reads = args.use_mmap; switch (args.type) { case BLOCK_BASED_TABLE_TEST: table_options_.flush_block_policy_factory.reset( new FlushBlockBySizePolicyFactory()); table_options_.block_size = 256; table_options_.block_restart_interval = args.restart_interval; + table_options_.index_block_restart_interval = args.restart_interval; + table_options_.format_version = args.format_version; options_.table_factory.reset( new BlockBasedTableFactory(table_options_)); - constructor_ = new TableConstructor(options_.comparator); + constructor_ = new TableConstructor( + options_.comparator, true /* convert_to_internal_key_ */); + internal_comparator_.reset( + new InternalKeyComparator(options_.comparator)); break; +// Plain table is not supported in ROCKSDB_LITE +#ifndef ROCKSDB_LITE case PLAIN_TABLE_SEMI_FIXED_PREFIX: support_prev_ = false; only_support_prefix_seek_ = true; options_.prefix_extractor.reset(new FixedOrLessPrefixTransform(2)); - options_.allow_mmap_reads = true; options_.table_factory.reset(NewPlainTableFactory()); - constructor_ = new TableConstructor(options_.comparator, true); + constructor_ = new TableConstructor( + options_.comparator, true /* convert_to_internal_key_ */); internal_comparator_.reset( new InternalKeyComparator(options_.comparator)); break; @@ -712,9 +727,9 @@ class Harness { support_prev_ = false; only_support_prefix_seek_ = true; options_.prefix_extractor.reset(NewNoopTransform()); - options_.allow_mmap_reads = true; options_.table_factory.reset(NewPlainTableFactory()); - constructor_ = new TableConstructor(options_.comparator, true); + constructor_ = new TableConstructor( + options_.comparator, true /* convert_to_internal_key_ */); internal_comparator_.reset( new InternalKeyComparator(options_.comparator)); break; @@ -722,7 +737,6 @@ class Harness { support_prev_ = false; only_support_prefix_seek_ = false; options_.prefix_extractor = nullptr; - options_.allow_mmap_reads = true; { PlainTableOptions plain_table_options; @@ -733,10 +747,12 @@ class Harness { options_.table_factory.reset( NewPlainTableFactory(plain_table_options)); } - constructor_ = new TableConstructor(options_.comparator, true); + constructor_ = new TableConstructor( + options_.comparator, true /* convert_to_internal_key_ */); internal_comparator_.reset( new InternalKeyComparator(options_.comparator)); break; +#endif // !ROCKSDB_LITE case BLOCK_TEST: table_options_.block_size = 256; options_.table_factory.reset( @@ -747,7 +763,8 @@ class Harness { table_options_.block_size = 256; options_.table_factory.reset( new BlockBasedTableFactory(table_options_)); - constructor_ = new MemTableConstructor(options_.comparator); + constructor_ = new MemTableConstructor(options_.comparator, + &write_buffer_); break; case DB_TEST: table_options_.block_size = 256; @@ -756,11 +773,10 @@ class Harness { constructor_ = new DBConstructor(options_.comparator); break; } + ioptions_ = ImmutableCFOptions(options_); } - ~Harness() { - delete constructor_; - } + ~HarnessTest() { delete constructor_; } void Add(const std::string& key, const std::string& value) { constructor_->Add(key, value); @@ -768,9 +784,9 @@ class Harness { void Test(Random* rnd) { std::vector keys; - KVMap data; - constructor_->Finish(options_, table_options_, *internal_comparator_, - &keys, &data); + stl_wrappers::KVMap data; + constructor_->Finish(options_, ioptions_, table_options_, + *internal_comparator_, &keys, &data); TestForwardScan(keys, data); if (support_prev_) { @@ -780,42 +796,47 @@ class Harness { } void TestForwardScan(const std::vector& keys, - const KVMap& data) { - Iterator* iter = constructor_->NewIterator(); + const stl_wrappers::KVMap& data) { + InternalIterator* iter = constructor_->NewIterator(); ASSERT_TRUE(!iter->Valid()); iter->SeekToFirst(); - for (KVMap::const_iterator model_iter = data.begin(); - model_iter != data.end(); - ++model_iter) { + for (stl_wrappers::KVMap::const_iterator model_iter = data.begin(); + model_iter != data.end(); ++model_iter) { ASSERT_EQ(ToString(data, model_iter), ToString(iter)); iter->Next(); } ASSERT_TRUE(!iter->Valid()); - delete iter; + if (constructor_->IsArenaMode() && !constructor_->AnywayDeleteIterator()) { + iter->~InternalIterator(); + } else { + delete iter; + } } void TestBackwardScan(const std::vector& keys, - const KVMap& data) { - Iterator* iter = constructor_->NewIterator(); + const stl_wrappers::KVMap& data) { + InternalIterator* iter = constructor_->NewIterator(); ASSERT_TRUE(!iter->Valid()); iter->SeekToLast(); - for (KVMap::const_reverse_iterator model_iter = data.rbegin(); - model_iter != data.rend(); - ++model_iter) { + for (stl_wrappers::KVMap::const_reverse_iterator model_iter = data.rbegin(); + model_iter != data.rend(); ++model_iter) { ASSERT_EQ(ToString(data, model_iter), ToString(iter)); iter->Prev(); } ASSERT_TRUE(!iter->Valid()); - delete iter; + if (constructor_->IsArenaMode() && !constructor_->AnywayDeleteIterator()) { + iter->~InternalIterator(); + } else { + delete iter; + } } - void TestRandomAccess(Random* rnd, - const std::vector& keys, - const KVMap& data) { + void TestRandomAccess(Random* rnd, const std::vector& keys, + const stl_wrappers::KVMap& data) { static const bool kVerbose = false; - Iterator* iter = constructor_->NewIterator(); + InternalIterator* iter = constructor_->NewIterator(); ASSERT_TRUE(!iter->Valid()); - KVMap::const_iterator model_iter = data.begin(); + stl_wrappers::KVMap::const_iterator model_iter = data.begin(); if (kVerbose) fprintf(stderr, "---\n"); for (int i = 0; i < 200; i++) { const int toss = rnd->Uniform(support_prev_ ? 5 : 3); @@ -876,10 +897,15 @@ class Harness { } } } - delete iter; + if (constructor_->IsArenaMode() && !constructor_->AnywayDeleteIterator()) { + iter->~InternalIterator(); + } else { + delete iter; + } } - std::string ToString(const KVMap& data, const KVMap::const_iterator& it) { + std::string ToString(const stl_wrappers::KVMap& data, + const stl_wrappers::KVMap::const_iterator& it) { if (it == data.end()) { return "END"; } else { @@ -887,8 +913,8 @@ class Harness { } } - std::string ToString(const KVMap& data, - const KVMap::const_reverse_iterator& it) { + std::string ToString(const stl_wrappers::KVMap& data, + const stl_wrappers::KVMap::const_reverse_iterator& it) { if (it == data.rend()) { return "END"; } else { @@ -896,7 +922,7 @@ class Harness { } } - std::string ToString(const Iterator* it) { + std::string ToString(const InternalIterator* it) { if (!it->Valid()) { return "END"; } else { @@ -908,7 +934,7 @@ class Harness { if (keys.empty()) { return "foo"; } else { - const int index = rnd->Uniform(keys.size()); + const int index = rnd->Uniform(static_cast(keys.size())); std::string result = keys[index]; switch (rnd->Uniform(support_prev_ ? 3 : 1)) { case 0: @@ -939,8 +965,10 @@ class Harness { private: Options options_ = Options(); + ImmutableCFOptions ioptions_; BlockBasedTableOptions table_options_ = BlockBasedTableOptions(); Constructor* constructor_; + WriteBufferManager write_buffer_; bool support_prev_; bool only_support_prefix_seek_; shared_ptr internal_comparator_; @@ -958,7 +986,7 @@ static bool Between(uint64_t val, uint64_t low, uint64_t high) { } // Tests against all kinds of tables -class TableTest { +class TableTest : public testing::Test { public: const InternalKeyComparator& GetPlainInternalComparator( const Comparator* comp) { @@ -968,6 +996,7 @@ class TableTest { } return *plain_internal_comparator; } + void IndexTest(BlockBasedTableOptions table_options); private: std::unique_ptr plain_internal_comparator; @@ -976,11 +1005,11 @@ class TableTest { class GeneralTableTest : public TableTest {}; class BlockBasedTableTest : public TableTest {}; class PlainTableTest : public TableTest {}; -class TablePropertyTest {}; +class TablePropertyTest : public testing::Test {}; // This test serves as the living tutorial for the prefix scan of user collected // properties. -TEST(TablePropertyTest, PrefixScanTest) { +TEST_F(TablePropertyTest, PrefixScanTest) { UserCollectedProperties props{{"num.111.1", "1"}, {"num.111.2", "2"}, {"num.111.3", "3"}, @@ -999,9 +1028,9 @@ TEST(TablePropertyTest, PrefixScanTest) { pos->first.compare(0, prefix.size(), prefix) == 0; ++pos) { ++num; - auto key = prefix + "." + std::to_string(num); + auto key = prefix + "." + ToString(num); ASSERT_EQ(key, pos->first); - ASSERT_EQ(std::to_string(num), pos->second); + ASSERT_EQ(ToString(num), pos->second); } ASSERT_EQ(3, num); } @@ -1017,8 +1046,8 @@ TEST(TablePropertyTest, PrefixScanTest) { // This test include all the basic checks except those for index size and block // size, which will be conducted in separated unit tests. -TEST(BlockBasedTableTest, BasicBlockBasedTableProperties) { - TableConstructor c(BytewiseComparator()); +TEST_F(BlockBasedTableTest, BasicBlockBasedTableProperties) { + TableConstructor c(BytewiseComparator(), true /* convert_to_internal_key_ */); c.Add("a1", "val1"); c.Add("b2", "val2"); @@ -1029,16 +1058,18 @@ TEST(BlockBasedTableTest, BasicBlockBasedTableProperties) { c.Add("g7", "val7"); c.Add("h8", "val8"); c.Add("j9", "val9"); + uint64_t diff_internal_user_bytes = 9 * 8; // 8 is seq size, 9 k-v totally std::vector keys; - KVMap kvmap; + stl_wrappers::KVMap kvmap; Options options; options.compression = kNoCompression; BlockBasedTableOptions table_options; table_options.block_restart_interval = 1; options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - c.Finish(options, table_options, + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, GetPlainInternalComparator(options.comparator), &keys, &kvmap); auto& props = *c.GetTableReader()->GetTableProperties(); @@ -1047,7 +1078,7 @@ TEST(BlockBasedTableTest, BasicBlockBasedTableProperties) { auto raw_key_size = kvmap.size() * 2ul; auto raw_value_size = kvmap.size() * 4ul; - ASSERT_EQ(raw_key_size, props.raw_key_size); + ASSERT_EQ(raw_key_size + diff_internal_user_bytes, props.raw_key_size); ASSERT_EQ(raw_value_size, props.raw_value_size); ASSERT_EQ(1ul, props.num_data_blocks); ASSERT_EQ("", props.filter_policy_name); // no filter policy is used @@ -1058,26 +1089,276 @@ TEST(BlockBasedTableTest, BasicBlockBasedTableProperties) { block_builder.Add(item.first, item.second); } Slice content = block_builder.Finish(); - ASSERT_EQ(content.size() + kBlockTrailerSize, props.data_size); + ASSERT_EQ(content.size() + kBlockTrailerSize + diff_internal_user_bytes, + props.data_size); + c.ResetTableReader(); +} + +TEST_F(BlockBasedTableTest, BlockBasedTableProperties2) { + TableConstructor c(&reverse_key_comparator); + std::vector keys; + stl_wrappers::KVMap kvmap; + + { + Options options; + options.compression = CompressionType::kNoCompression; + BlockBasedTableOptions table_options; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, + GetPlainInternalComparator(options.comparator), &keys, &kvmap); + + auto& props = *c.GetTableReader()->GetTableProperties(); + + // Default comparator + ASSERT_EQ("leveldb.BytewiseComparator", props.comparator_name); + // No merge operator + ASSERT_EQ("nullptr", props.merge_operator_name); + // No prefix extractor + ASSERT_EQ("nullptr", props.prefix_extractor_name); + // No property collectors + ASSERT_EQ("[]", props.property_collectors_names); + // No filter policy is used + ASSERT_EQ("", props.filter_policy_name); + // Compression type == that set: + ASSERT_EQ("NoCompression", props.compression_name); + c.ResetTableReader(); + } + + { + Options options; + BlockBasedTableOptions table_options; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + options.comparator = &reverse_key_comparator; + options.merge_operator = MergeOperators::CreateUInt64AddOperator(); + options.prefix_extractor.reset(NewNoopTransform()); + options.table_properties_collector_factories.emplace_back( + new DummyPropertiesCollectorFactory1()); + options.table_properties_collector_factories.emplace_back( + new DummyPropertiesCollectorFactory2()); + + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, + GetPlainInternalComparator(options.comparator), &keys, &kvmap); + + auto& props = *c.GetTableReader()->GetTableProperties(); + + ASSERT_EQ("rocksdb.ReverseBytewiseComparator", props.comparator_name); + ASSERT_EQ("UInt64AddOperator", props.merge_operator_name); + ASSERT_EQ("rocksdb.Noop", props.prefix_extractor_name); + ASSERT_EQ("[DummyPropertiesCollector1,DummyPropertiesCollector2]", + props.property_collectors_names); + ASSERT_EQ("", props.filter_policy_name); // no filter policy is used + c.ResetTableReader(); + } +} + +TEST_F(BlockBasedTableTest, RangeDelBlock) { + TableConstructor c(BytewiseComparator()); + std::vector keys = {"1pika", "2chu"}; + std::vector vals = {"p", "c"}; + + for (int i = 0; i < 2; i++) { + RangeTombstone t(keys[i], vals[i], i); + std::pair p = t.Serialize(); + c.Add(p.first.Encode().ToString(), p.second); + } + + std::vector sorted_keys; + stl_wrappers::KVMap kvmap; + Options options; + options.compression = kNoCompression; + BlockBasedTableOptions table_options; + table_options.block_restart_interval = 1; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + const ImmutableCFOptions ioptions(options); + std::unique_ptr internal_cmp( + new InternalKeyComparator(options.comparator)); + c.Finish(options, ioptions, table_options, *internal_cmp, &sorted_keys, + &kvmap); + + for (int j = 0; j < 2; ++j) { + std::unique_ptr iter( + c.GetTableReader()->NewRangeTombstoneIterator(ReadOptions())); + if (j > 0) { + // For second iteration, delete the table reader object and verify the + // iterator can still access its metablock's range tombstones. + c.ResetTableReader(); + } + ASSERT_FALSE(iter->Valid()); + iter->SeekToFirst(); + ASSERT_TRUE(iter->Valid()); + for (int i = 0; i < 2; i++) { + ASSERT_TRUE(iter->Valid()); + ParsedInternalKey parsed_key; + ASSERT_TRUE(ParseInternalKey(iter->key(), &parsed_key)); + RangeTombstone t(parsed_key, iter->value()); + ASSERT_EQ(t.start_key_, keys[i]); + ASSERT_EQ(t.end_key_, vals[i]); + ASSERT_EQ(t.seq_, i); + iter->Next(); + } + ASSERT_TRUE(!iter->Valid()); + } } -TEST(BlockBasedTableTest, FilterPolicyNameProperties) { - TableConstructor c(BytewiseComparator(), true); +TEST_F(BlockBasedTableTest, FilterPolicyNameProperties) { + TableConstructor c(BytewiseComparator(), true /* convert_to_internal_key_ */); c.Add("a1", "val1"); std::vector keys; - KVMap kvmap; + stl_wrappers::KVMap kvmap; BlockBasedTableOptions table_options; table_options.filter_policy.reset(NewBloomFilterPolicy(10)); Options options; options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - c.Finish(options, table_options, + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, GetPlainInternalComparator(options.comparator), &keys, &kvmap); auto& props = *c.GetTableReader()->GetTableProperties(); ASSERT_EQ("rocksdb.BuiltinBloomFilter", props.filter_policy_name); + c.ResetTableReader(); } -TEST(BlockBasedTableTest, TotalOrderSeekOnHashIndex) { +// +// BlockBasedTableTest::PrefetchTest +// +void AssertKeysInCache(BlockBasedTable* table_reader, + const std::vector& keys_in_cache, + const std::vector& keys_not_in_cache, + bool convert = false) { + if (convert) { + for (auto key : keys_in_cache) { + InternalKey ikey(key, kMaxSequenceNumber, kTypeValue); + ASSERT_TRUE(table_reader->TEST_KeyInCache(ReadOptions(), ikey.Encode())); + } + for (auto key : keys_not_in_cache) { + InternalKey ikey(key, kMaxSequenceNumber, kTypeValue); + ASSERT_TRUE(!table_reader->TEST_KeyInCache(ReadOptions(), ikey.Encode())); + } + } else { + for (auto key : keys_in_cache) { + ASSERT_TRUE(table_reader->TEST_KeyInCache(ReadOptions(), key)); + } + for (auto key : keys_not_in_cache) { + ASSERT_TRUE(!table_reader->TEST_KeyInCache(ReadOptions(), key)); + } + } +} + +void PrefetchRange(TableConstructor* c, Options* opt, + BlockBasedTableOptions* table_options, const char* key_begin, + const char* key_end, + const std::vector& keys_in_cache, + const std::vector& keys_not_in_cache, + const Status expected_status = Status::OK()) { + // reset the cache and reopen the table + table_options->block_cache = NewLRUCache(16 * 1024 * 1024, 4); + opt->table_factory.reset(NewBlockBasedTableFactory(*table_options)); + const ImmutableCFOptions ioptions2(*opt); + ASSERT_OK(c->Reopen(ioptions2)); + + // prefetch + auto* table_reader = dynamic_cast(c->GetTableReader()); + Status s; + unique_ptr begin, end; + unique_ptr i_begin, i_end; + if (key_begin != nullptr) { + if (c->ConvertToInternalKey()) { + i_begin.reset(new InternalKey(key_begin, kMaxSequenceNumber, kTypeValue)); + begin.reset(new Slice(i_begin->Encode())); + } else { + begin.reset(new Slice(key_begin)); + } + } + if (key_end != nullptr) { + if (c->ConvertToInternalKey()) { + i_end.reset(new InternalKey(key_end, kMaxSequenceNumber, kTypeValue)); + end.reset(new Slice(i_end->Encode())); + } else { + end.reset(new Slice(key_end)); + } + } + s = table_reader->Prefetch(begin.get(), end.get()); + + ASSERT_TRUE(s.code() == expected_status.code()); + + // assert our expectation in cache warmup + AssertKeysInCache(table_reader, keys_in_cache, keys_not_in_cache, + c->ConvertToInternalKey()); + c->ResetTableReader(); +} + +TEST_F(BlockBasedTableTest, PrefetchTest) { + // The purpose of this test is to test the prefetching operation built into + // BlockBasedTable. + Options opt; + unique_ptr ikc; + ikc.reset(new test::PlainInternalKeyComparator(opt.comparator)); + opt.compression = kNoCompression; + BlockBasedTableOptions table_options; + table_options.block_size = 1024; + // big enough so we don't ever lose cached values. + table_options.block_cache = NewLRUCache(16 * 1024 * 1024, 4); + opt.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + TableConstructor c(BytewiseComparator(), true /* convert_to_internal_key_ */); + c.Add("k01", "hello"); + c.Add("k02", "hello2"); + c.Add("k03", std::string(10000, 'x')); + c.Add("k04", std::string(200000, 'x')); + c.Add("k05", std::string(300000, 'x')); + c.Add("k06", "hello3"); + c.Add("k07", std::string(100000, 'x')); + std::vector keys; + stl_wrappers::KVMap kvmap; + const ImmutableCFOptions ioptions(opt); + c.Finish(opt, ioptions, table_options, *ikc, &keys, &kvmap); + c.ResetTableReader(); + + // We get the following data spread : + // + // Data block Index + // ======================== + // [ k01 k02 k03 ] k03 + // [ k04 ] k04 + // [ k05 ] k05 + // [ k06 k07 ] k07 + + + // Simple + PrefetchRange(&c, &opt, &table_options, + /*key_range=*/"k01", "k05", + /*keys_in_cache=*/{"k01", "k02", "k03", "k04", "k05"}, + /*keys_not_in_cache=*/{"k06", "k07"}); + PrefetchRange(&c, &opt, &table_options, "k01", "k01", {"k01", "k02", "k03"}, + {"k04", "k05", "k06", "k07"}); + // odd + PrefetchRange(&c, &opt, &table_options, "a", "z", + {"k01", "k02", "k03", "k04", "k05", "k06", "k07"}, {}); + PrefetchRange(&c, &opt, &table_options, "k00", "k00", {"k01", "k02", "k03"}, + {"k04", "k05", "k06", "k07"}); + // Edge cases + PrefetchRange(&c, &opt, &table_options, "k00", "k06", + {"k01", "k02", "k03", "k04", "k05", "k06", "k07"}, {}); + PrefetchRange(&c, &opt, &table_options, "k00", "zzz", + {"k01", "k02", "k03", "k04", "k05", "k06", "k07"}, {}); + // null keys + PrefetchRange(&c, &opt, &table_options, nullptr, nullptr, + {"k01", "k02", "k03", "k04", "k05", "k06", "k07"}, {}); + PrefetchRange(&c, &opt, &table_options, "k04", nullptr, + {"k04", "k05", "k06", "k07"}, {"k01", "k02", "k03"}); + PrefetchRange(&c, &opt, &table_options, nullptr, "k05", + {"k01", "k02", "k03", "k04", "k05"}, {"k06", "k07"}); + // invalid + PrefetchRange(&c, &opt, &table_options, "k06", "k00", {}, {}, + Status::InvalidArgument(Slice("k06 "), Slice("k07"))); + c.ResetTableReader(); +} + +TEST_F(BlockBasedTableTest, TotalOrderSeekOnHashIndex) { BlockBasedTableOptions table_options; for (int i = 0; i < 4; ++i) { Options options; @@ -1103,16 +1384,22 @@ TEST(BlockBasedTableTest, TotalOrderSeekOnHashIndex) { options.prefix_extractor.reset(NewFixedPrefixTransform(4)); break; case 3: - default: // Hash search index with filter policy table_options.index_type = BlockBasedTableOptions::kHashSearch; table_options.filter_policy.reset(NewBloomFilterPolicy(10)); options.table_factory.reset(new BlockBasedTableFactory(table_options)); options.prefix_extractor.reset(NewFixedPrefixTransform(4)); break; + case 4: + default: + // Binary search index + table_options.index_type = BlockBasedTableOptions::kTwoLevelIndexSearch; + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + break; } - TableConstructor c(BytewiseComparator(), true); + TableConstructor c(BytewiseComparator(), + true /* convert_to_internal_key_ */); c.Add("aaaa1", std::string('a', 56)); c.Add("bbaa1", std::string('a', 56)); c.Add("cccc1", std::string('a', 56)); @@ -1121,15 +1408,16 @@ TEST(BlockBasedTableTest, TotalOrderSeekOnHashIndex) { c.Add("abbb1", std::string('a', 56)); c.Add("cccc2", std::string('a', 56)); std::vector keys; - KVMap kvmap; - c.Finish(options, table_options, + stl_wrappers::KVMap kvmap; + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, GetPlainInternalComparator(options.comparator), &keys, &kvmap); auto props = c.GetTableReader()->GetTableProperties(); ASSERT_EQ(7u, props->num_data_blocks); auto* reader = c.GetTableReader(); ReadOptions ro; ro.total_order_seek = true; - std::unique_ptr iter(reader->NewIterator(ro)); + std::unique_ptr iter(reader->NewIterator(ro)); iter->Seek(InternalKey("b", 0, kTypeValue).Encode()); ASSERT_OK(iter->status()); @@ -1160,20 +1448,93 @@ TEST(BlockBasedTableTest, TotalOrderSeekOnHashIndex) { } } +TEST_F(BlockBasedTableTest, NoopTransformSeek) { + BlockBasedTableOptions table_options; + table_options.filter_policy.reset(NewBloomFilterPolicy(10)); + + Options options; + options.comparator = BytewiseComparator(); + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + options.prefix_extractor.reset(NewNoopTransform()); + + TableConstructor c(options.comparator); + // To tickle the PrefixMayMatch bug it is important that the + // user-key is a single byte so that the index key exactly matches + // the user-key. + InternalKey key("a", 1, kTypeValue); + c.Add(key.Encode().ToString(), "b"); + std::vector keys; + stl_wrappers::KVMap kvmap; + const ImmutableCFOptions ioptions(options); + const InternalKeyComparator internal_comparator(options.comparator); + c.Finish(options, ioptions, table_options, internal_comparator, &keys, + &kvmap); + + auto* reader = c.GetTableReader(); + for (int i = 0; i < 2; ++i) { + ReadOptions ro; + ro.total_order_seek = (i == 0); + std::unique_ptr iter(reader->NewIterator(ro)); + + iter->Seek(key.Encode()); + ASSERT_OK(iter->status()); + ASSERT_TRUE(iter->Valid()); + ASSERT_EQ("a", ExtractUserKey(iter->key()).ToString()); + } +} + +TEST_F(BlockBasedTableTest, SkipPrefixBloomFilter) { + // if DB is opened with a prefix extractor of a different name, + // prefix bloom is skipped when read the file + BlockBasedTableOptions table_options; + table_options.filter_policy.reset(NewBloomFilterPolicy(2)); + table_options.whole_key_filtering = false; + + Options options; + options.comparator = BytewiseComparator(); + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + options.prefix_extractor.reset(NewFixedPrefixTransform(1)); + + TableConstructor c(options.comparator); + InternalKey key("abcdefghijk", 1, kTypeValue); + c.Add(key.Encode().ToString(), "test"); + std::vector keys; + stl_wrappers::KVMap kvmap; + const ImmutableCFOptions ioptions(options); + const InternalKeyComparator internal_comparator(options.comparator); + c.Finish(options, ioptions, table_options, internal_comparator, &keys, + &kvmap); + options.prefix_extractor.reset(NewFixedPrefixTransform(9)); + const ImmutableCFOptions new_ioptions(options); + c.Reopen(new_ioptions); + auto reader = c.GetTableReader(); + std::unique_ptr db_iter(reader->NewIterator(ReadOptions())); + + // Test point lookup + // only one kv + for (auto& kv : kvmap) { + db_iter->Seek(kv.first); + ASSERT_TRUE(db_iter->Valid()); + ASSERT_OK(db_iter->status()); + ASSERT_EQ(db_iter->key(), kv.first); + ASSERT_EQ(db_iter->value(), kv.second); + } +} + static std::string RandomString(Random* rnd, int len) { std::string r; test::RandomString(rnd, len, &r); return r; } -void AddInternalKey(TableConstructor* c, const std::string prefix, +void AddInternalKey(TableConstructor* c, const std::string& prefix, int suffix_len = 800) { static Random rnd(1023); InternalKey k(prefix + RandomString(&rnd, 800), 0, kTypeValue); c->Add(k.Encode().ToString(), "v"); } -TEST(TableTest, HashIndexTest) { +void TableTest::IndexTest(BlockBasedTableOptions table_options) { TableConstructor c(BytewiseComparator()); // keys with prefix length 3, make sure the key/value is big enough to fill @@ -1194,25 +1555,24 @@ TEST(TableTest, HashIndexTest) { AddInternalKey(&c, "0095"); std::vector keys; - KVMap kvmap; + stl_wrappers::KVMap kvmap; Options options; options.prefix_extractor.reset(NewFixedPrefixTransform(3)); - BlockBasedTableOptions table_options; - table_options.index_type = BlockBasedTableOptions::kHashSearch; - table_options.hash_index_allow_collision = true; table_options.block_size = 1700; - table_options.block_cache = NewLRUCache(1024); + table_options.block_cache = NewLRUCache(1024, 4); options.table_factory.reset(NewBlockBasedTableFactory(table_options)); std::unique_ptr comparator( new InternalKeyComparator(BytewiseComparator())); - c.Finish(options, table_options, *comparator, &keys, &kvmap); + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, *comparator, &keys, &kvmap); auto reader = c.GetTableReader(); auto props = reader->GetTableProperties(); ASSERT_EQ(5u, props->num_data_blocks); - std::unique_ptr hash_iter(reader->NewIterator(ReadOptions())); + std::unique_ptr index_iter( + reader->NewIterator(ReadOptions())); // -- Find keys do not exist, but have common prefix. std::vector prefixes = {"001", "003", "005", "007", "009"}; @@ -1221,13 +1581,13 @@ TEST(TableTest, HashIndexTest) { // find the lower bound of the prefix for (size_t i = 0; i < prefixes.size(); ++i) { - hash_iter->Seek(InternalKey(prefixes[i], 0, kTypeValue).Encode()); - ASSERT_OK(hash_iter->status()); - ASSERT_TRUE(hash_iter->Valid()); + index_iter->Seek(InternalKey(prefixes[i], 0, kTypeValue).Encode()); + ASSERT_OK(index_iter->status()); + ASSERT_TRUE(index_iter->Valid()); // seek the first element in the block - ASSERT_EQ(lower_bound[i], hash_iter->key().ToString()); - ASSERT_EQ("v", hash_iter->value().ToString()); + ASSERT_EQ(lower_bound[i], index_iter->key().ToString()); + ASSERT_EQ("v", index_iter->value().ToString()); } // find the upper bound of prefixes @@ -1236,56 +1596,81 @@ TEST(TableTest, HashIndexTest) { // find existing keys for (const auto& item : kvmap) { auto ukey = ExtractUserKey(item.first).ToString(); - hash_iter->Seek(ukey); + index_iter->Seek(ukey); // ASSERT_OK(regular_iter->status()); - ASSERT_OK(hash_iter->status()); + ASSERT_OK(index_iter->status()); // ASSERT_TRUE(regular_iter->Valid()); - ASSERT_TRUE(hash_iter->Valid()); + ASSERT_TRUE(index_iter->Valid()); - ASSERT_EQ(item.first, hash_iter->key().ToString()); - ASSERT_EQ(item.second, hash_iter->value().ToString()); + ASSERT_EQ(item.first, index_iter->key().ToString()); + ASSERT_EQ(item.second, index_iter->value().ToString()); } for (size_t i = 0; i < prefixes.size(); ++i) { // the key is greater than any existing keys. auto key = prefixes[i] + "9"; - hash_iter->Seek(InternalKey(key, 0, kTypeValue).Encode()); + index_iter->Seek(InternalKey(key, 0, kTypeValue).Encode()); - ASSERT_OK(hash_iter->status()); + ASSERT_OK(index_iter->status()); if (i == prefixes.size() - 1) { // last key - ASSERT_TRUE(!hash_iter->Valid()); + ASSERT_TRUE(!index_iter->Valid()); } else { - ASSERT_TRUE(hash_iter->Valid()); + ASSERT_TRUE(index_iter->Valid()); // seek the first element in the block - ASSERT_EQ(upper_bound[i], hash_iter->key().ToString()); - ASSERT_EQ("v", hash_iter->value().ToString()); + ASSERT_EQ(upper_bound[i], index_iter->key().ToString()); + ASSERT_EQ("v", index_iter->value().ToString()); } } // find keys with prefix that don't match any of the existing prefixes. std::vector non_exist_prefixes = {"002", "004", "006", "008"}; for (const auto& prefix : non_exist_prefixes) { - hash_iter->Seek(InternalKey(prefix, 0, kTypeValue).Encode()); + index_iter->Seek(InternalKey(prefix, 0, kTypeValue).Encode()); // regular_iter->Seek(prefix); - ASSERT_OK(hash_iter->status()); + ASSERT_OK(index_iter->status()); // Seek to non-existing prefixes should yield either invalid, or a // key with prefix greater than the target. - if (hash_iter->Valid()) { - Slice ukey = ExtractUserKey(hash_iter->key()); + if (index_iter->Valid()) { + Slice ukey = ExtractUserKey(index_iter->key()); Slice ukey_prefix = options.prefix_extractor->Transform(ukey); ASSERT_TRUE(BytewiseComparator()->Compare(prefix, ukey_prefix) < 0); } } + c.ResetTableReader(); +} + +TEST_F(TableTest, BinaryIndexTest) { + BlockBasedTableOptions table_options; + table_options.index_type = BlockBasedTableOptions::kBinarySearch; + IndexTest(table_options); +} + +TEST_F(TableTest, HashIndexTest) { + BlockBasedTableOptions table_options; + table_options.index_type = BlockBasedTableOptions::kHashSearch; + IndexTest(table_options); +} + +TEST_F(TableTest, PartitionIndexTest) { + const int max_index_keys = 5; + const int est_max_index_key_value_size = 32; + const int est_max_index_size = max_index_keys * est_max_index_key_value_size; + for (int i = 1; i <= est_max_index_size + 1; i++) { + BlockBasedTableOptions table_options; + table_options.index_type = BlockBasedTableOptions::kTwoLevelIndexSearch; + table_options.metadata_block_size = i; + IndexTest(table_options); + } } // It's very hard to figure out the index block size of a block accurately. // To make sure we get the index size, we just make sure as key number // grows, the filter block size also grows. -TEST(BlockBasedTableTest, IndexSizeStat) { +TEST_F(BlockBasedTableTest, IndexSizeStat) { uint64_t last_index_size = 0; // we need to use random keys since the pure human readable texts @@ -1301,30 +1686,33 @@ TEST(BlockBasedTableTest, IndexSizeStat) { // Each time we load one more key to the table. the table index block // size is expected to be larger than last time's. for (size_t i = 1; i < keys.size(); ++i) { - TableConstructor c(BytewiseComparator()); + TableConstructor c(BytewiseComparator(), + true /* convert_to_internal_key_ */); for (size_t j = 0; j < i; ++j) { c.Add(keys[j], "val"); } std::vector ks; - KVMap kvmap; + stl_wrappers::KVMap kvmap; Options options; options.compression = kNoCompression; BlockBasedTableOptions table_options; table_options.block_restart_interval = 1; options.table_factory.reset(NewBlockBasedTableFactory(table_options)); - c.Finish(options, table_options, + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, GetPlainInternalComparator(options.comparator), &ks, &kvmap); auto index_size = c.GetTableReader()->GetTableProperties()->index_size; ASSERT_GT(index_size, last_index_size); last_index_size = index_size; + c.ResetTableReader(); } } -TEST(BlockBasedTableTest, NumBlockStat) { +TEST_F(BlockBasedTableTest, NumBlockStat) { Random rnd(test::RandomSeed()); - TableConstructor c(BytewiseComparator()); + TableConstructor c(BytewiseComparator(), true /* convert_to_internal_key_ */); Options options; options.compression = kNoCompression; BlockBasedTableOptions table_options; @@ -1339,11 +1727,13 @@ TEST(BlockBasedTableTest, NumBlockStat) { } std::vector ks; - KVMap kvmap; - c.Finish(options, table_options, + stl_wrappers::KVMap kvmap; + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, GetPlainInternalComparator(options.comparator), &ks, &kvmap); ASSERT_EQ(kvmap.size(), c.GetTableReader()->GetTableProperties()->num_data_blocks); + c.ResetTableReader(); } // A simple tool that takes the snapshot of block cache statistics. @@ -1359,35 +1749,43 @@ class BlockCachePropertiesSnapshot { filter_block_cache_miss = statistics->getTickerCount(BLOCK_CACHE_FILTER_MISS); filter_block_cache_hit = statistics->getTickerCount(BLOCK_CACHE_FILTER_HIT); + block_cache_bytes_read = statistics->getTickerCount(BLOCK_CACHE_BYTES_READ); + block_cache_bytes_write = + statistics->getTickerCount(BLOCK_CACHE_BYTES_WRITE); } - void AssertIndexBlockStat(int64_t index_block_cache_miss, - int64_t index_block_cache_hit) { - ASSERT_EQ(index_block_cache_miss, this->index_block_cache_miss); - ASSERT_EQ(index_block_cache_hit, this->index_block_cache_hit); + void AssertIndexBlockStat(int64_t expected_index_block_cache_miss, + int64_t expected_index_block_cache_hit) { + ASSERT_EQ(expected_index_block_cache_miss, index_block_cache_miss); + ASSERT_EQ(expected_index_block_cache_hit, index_block_cache_hit); } - void AssertFilterBlockStat(int64_t filter_block_cache_miss, - int64_t filter_block_cache_hit) { - ASSERT_EQ(filter_block_cache_miss, this->filter_block_cache_miss); - ASSERT_EQ(filter_block_cache_hit, this->filter_block_cache_hit); + void AssertFilterBlockStat(int64_t expected_filter_block_cache_miss, + int64_t expected_filter_block_cache_hit) { + ASSERT_EQ(expected_filter_block_cache_miss, filter_block_cache_miss); + ASSERT_EQ(expected_filter_block_cache_hit, filter_block_cache_hit); } // Check if the fetched props matches the expected ones. // TODO(kailiu) Use this only when you disabled filter policy! - void AssertEqual(int64_t index_block_cache_miss, - int64_t index_block_cache_hit, int64_t data_block_cache_miss, - int64_t data_block_cache_hit) const { - ASSERT_EQ(index_block_cache_miss, this->index_block_cache_miss); - ASSERT_EQ(index_block_cache_hit, this->index_block_cache_hit); - ASSERT_EQ(data_block_cache_miss, this->data_block_cache_miss); - ASSERT_EQ(data_block_cache_hit, this->data_block_cache_hit); - ASSERT_EQ(index_block_cache_miss + data_block_cache_miss, - this->block_cache_miss); - ASSERT_EQ(index_block_cache_hit + data_block_cache_hit, - this->block_cache_hit); + void AssertEqual(int64_t expected_index_block_cache_miss, + int64_t expected_index_block_cache_hit, + int64_t expected_data_block_cache_miss, + int64_t expected_data_block_cache_hit) const { + ASSERT_EQ(expected_index_block_cache_miss, index_block_cache_miss); + ASSERT_EQ(expected_index_block_cache_hit, index_block_cache_hit); + ASSERT_EQ(expected_data_block_cache_miss, data_block_cache_miss); + ASSERT_EQ(expected_data_block_cache_hit, data_block_cache_hit); + ASSERT_EQ(expected_index_block_cache_miss + expected_data_block_cache_miss, + block_cache_miss); + ASSERT_EQ(expected_index_block_cache_hit + expected_data_block_cache_hit, + block_cache_hit); } + int64_t GetCacheBytesRead() { return block_cache_bytes_read; } + + int64_t GetCacheBytesWrite() { return block_cache_bytes_write; } + private: int64_t block_cache_miss = 0; int64_t block_cache_hit = 0; @@ -1397,26 +1795,27 @@ class BlockCachePropertiesSnapshot { int64_t data_block_cache_hit = 0; int64_t filter_block_cache_miss = 0; int64_t filter_block_cache_hit = 0; + int64_t block_cache_bytes_read = 0; + int64_t block_cache_bytes_write = 0; }; // Make sure, by default, index/filter blocks were pre-loaded (meaning we won't // use block cache to store them). -TEST(BlockBasedTableTest, BlockCacheDisabledTest) { +TEST_F(BlockBasedTableTest, BlockCacheDisabledTest) { Options options; options.create_if_missing = true; options.statistics = CreateDBStatistics(); BlockBasedTableOptions table_options; - // Intentionally commented out: table_options.cache_index_and_filter_blocks = - // true; - table_options.block_cache = NewLRUCache(1024); + table_options.block_cache = NewLRUCache(1024, 4); table_options.filter_policy.reset(NewBloomFilterPolicy(10)); options.table_factory.reset(new BlockBasedTableFactory(table_options)); std::vector keys; - KVMap kvmap; + stl_wrappers::KVMap kvmap; - TableConstructor c(BytewiseComparator(), true); + TableConstructor c(BytewiseComparator(), true /* convert_to_internal_key_ */); c.Add("key", "value"); - c.Finish(options, table_options, + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, GetPlainInternalComparator(options.comparator), &keys, &kvmap); // preloading filter/index blocks is enabled. @@ -1432,8 +1831,11 @@ TEST(BlockBasedTableTest, BlockCacheDisabledTest) { } { + GetContext get_context(options.comparator, nullptr, nullptr, nullptr, + GetContext::kNotFound, Slice(), nullptr, nullptr, + nullptr, nullptr, nullptr); // a hack that just to trigger BlockBasedTable::GetFilter. - reader->Get(ReadOptions(), "non-exist-key", nullptr, nullptr, nullptr); + reader->Get(ReadOptions(), "non-exist-key", &get_context); BlockCachePropertiesSnapshot props(options.statistics.get()); props.AssertIndexBlockStat(0, 0); props.AssertFilterBlockStat(0, 0); @@ -1442,7 +1844,7 @@ TEST(BlockBasedTableTest, BlockCacheDisabledTest) { // Due to the difficulities of the intersaction between statistics, this test // only tests the case when "index block is put to block cache" -TEST(BlockBasedTableTest, FilterBlockInBlockCache) { +TEST_F(BlockBasedTableTest, FilterBlockInBlockCache) { // -- Table construction Options options; options.create_if_missing = true; @@ -1450,31 +1852,37 @@ TEST(BlockBasedTableTest, FilterBlockInBlockCache) { // Enable the cache for index/filter blocks BlockBasedTableOptions table_options; - table_options.block_cache = NewLRUCache(1024); + table_options.block_cache = NewLRUCache(1024, 4); table_options.cache_index_and_filter_blocks = true; options.table_factory.reset(new BlockBasedTableFactory(table_options)); std::vector keys; - KVMap kvmap; + stl_wrappers::KVMap kvmap; - TableConstructor c(BytewiseComparator()); + TableConstructor c(BytewiseComparator(), true /* convert_to_internal_key_ */); c.Add("key", "value"); - c.Finish(options, table_options, + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, GetPlainInternalComparator(options.comparator), &keys, &kvmap); // preloading filter/index blocks is prohibited. - auto reader = dynamic_cast(c.GetTableReader()); + auto* reader = dynamic_cast(c.GetTableReader()); ASSERT_TRUE(!reader->TEST_filter_block_preloaded()); ASSERT_TRUE(!reader->TEST_index_reader_preloaded()); // -- PART 1: Open with regular block cache. // Since block_cache is disabled, no cache activities will be involved. - unique_ptr iter; + unique_ptr iter; + int64_t last_cache_bytes_read = 0; // At first, no block will be accessed. { BlockCachePropertiesSnapshot props(options.statistics.get()); // index will be added to block cache. props.AssertEqual(1, // index block miss 0, 0, 0); + ASSERT_EQ(props.GetCacheBytesRead(), 0); + ASSERT_EQ(props.GetCacheBytesWrite(), + table_options.block_cache->GetUsage()); + last_cache_bytes_read = props.GetCacheBytesRead(); } // Only index block will be accessed @@ -1486,6 +1894,11 @@ TEST(BlockBasedTableTest, FilterBlockInBlockCache) { // value; other numbers remain the same. props.AssertEqual(1, 0 + 1, // index block hit 0, 0); + // Cache hit, bytes read from cache should increase + ASSERT_GT(props.GetCacheBytesRead(), last_cache_bytes_read); + ASSERT_EQ(props.GetCacheBytesWrite(), + table_options.block_cache->GetUsage()); + last_cache_bytes_read = props.GetCacheBytesRead(); } // Only data block will be accessed @@ -1494,6 +1907,11 @@ TEST(BlockBasedTableTest, FilterBlockInBlockCache) { BlockCachePropertiesSnapshot props(options.statistics.get()); props.AssertEqual(1, 1, 0 + 1, // data block miss 0); + // Cache miss, Bytes read from cache should not change + ASSERT_EQ(props.GetCacheBytesRead(), last_cache_bytes_read); + ASSERT_EQ(props.GetCacheBytesWrite(), + table_options.block_cache->GetUsage()); + last_cache_bytes_read = props.GetCacheBytesRead(); } // Data block will be in cache @@ -1503,40 +1921,32 @@ TEST(BlockBasedTableTest, FilterBlockInBlockCache) { BlockCachePropertiesSnapshot props(options.statistics.get()); props.AssertEqual(1, 1 + 1, /* index block hit */ 1, 0 + 1 /* data block hit */); + // Cache hit, bytes read from cache should increase + ASSERT_GT(props.GetCacheBytesRead(), last_cache_bytes_read); + ASSERT_EQ(props.GetCacheBytesWrite(), + table_options.block_cache->GetUsage()); } // release the iterator so that the block cache can reset correctly. iter.reset(); - // -- PART 2: Open without block cache - table_options.no_block_cache = true; - table_options.block_cache.reset(); - options.table_factory.reset(new BlockBasedTableFactory(table_options)); - options.statistics = CreateDBStatistics(); // reset the stats - c.Reopen(options); - table_options.no_block_cache = false; - - { - iter.reset(c.NewIterator()); - iter->SeekToFirst(); - ASSERT_EQ("key", iter->key().ToString()); - BlockCachePropertiesSnapshot props(options.statistics.get()); - // Nothing is affected at all - props.AssertEqual(0, 0, 0, 0); - } + c.ResetTableReader(); - // -- PART 3: Open with very small block cache + // -- PART 2: Open with very small block cache // In this test, no block will ever get hit since the block cache is // too small to fit even one entry. - table_options.block_cache = NewLRUCache(1); + table_options.block_cache = NewLRUCache(1, 4); + options.statistics = CreateDBStatistics(); options.table_factory.reset(new BlockBasedTableFactory(table_options)); - c.Reopen(options); + const ImmutableCFOptions ioptions2(options); + c.Reopen(ioptions2); { BlockCachePropertiesSnapshot props(options.statistics.get()); props.AssertEqual(1, // index block miss 0, 0, 0); + // Cache miss, Bytes read from cache should not change + ASSERT_EQ(props.GetCacheBytesRead(), 0); } - { // Both index and data block get accessed. // It first cache index block then data block. But since the cache size @@ -1546,6 +1956,8 @@ TEST(BlockBasedTableTest, FilterBlockInBlockCache) { props.AssertEqual(1 + 1, // index block miss 0, 0, // data block miss 0); + // Cache hit, bytes read from cache should increase + ASSERT_EQ(props.GetCacheBytesRead(), 0); } { @@ -1555,10 +1967,296 @@ TEST(BlockBasedTableTest, FilterBlockInBlockCache) { BlockCachePropertiesSnapshot props(options.statistics.get()); props.AssertEqual(2, 0, 0 + 1, // data block miss 0); + // Cache miss, Bytes read from cache should not change + ASSERT_EQ(props.GetCacheBytesRead(), 0); + } + iter.reset(); + c.ResetTableReader(); + + // -- PART 3: Open table with bloom filter enabled but not in SST file + table_options.block_cache = NewLRUCache(4096, 4); + table_options.cache_index_and_filter_blocks = false; + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + TableConstructor c3(BytewiseComparator()); + std::string user_key = "k01"; + InternalKey internal_key(user_key, 0, kTypeValue); + c3.Add(internal_key.Encode().ToString(), "hello"); + ImmutableCFOptions ioptions3(options); + // Generate table without filter policy + c3.Finish(options, ioptions3, table_options, + GetPlainInternalComparator(options.comparator), &keys, &kvmap); + c3.ResetTableReader(); + + // Open table with filter policy + table_options.filter_policy.reset(NewBloomFilterPolicy(1)); + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + options.statistics = CreateDBStatistics(); + ImmutableCFOptions ioptions4(options); + ASSERT_OK(c3.Reopen(ioptions4)); + reader = dynamic_cast(c3.GetTableReader()); + ASSERT_TRUE(!reader->TEST_filter_block_preloaded()); + PinnableSlice value; + GetContext get_context(options.comparator, nullptr, nullptr, nullptr, + GetContext::kNotFound, user_key, &value, nullptr, + nullptr, nullptr, nullptr); + ASSERT_OK(reader->Get(ReadOptions(), user_key, &get_context)); + ASSERT_STREQ(value.data(), "hello"); + BlockCachePropertiesSnapshot props(options.statistics.get()); + props.AssertFilterBlockStat(0, 0); + c3.ResetTableReader(); +} + +void ValidateBlockSizeDeviation(int value, int expected) { + BlockBasedTableOptions table_options; + table_options.block_size_deviation = value; + BlockBasedTableFactory* factory = new BlockBasedTableFactory(table_options); + + const BlockBasedTableOptions* normalized_table_options = + (const BlockBasedTableOptions*)factory->GetOptions(); + ASSERT_EQ(normalized_table_options->block_size_deviation, expected); + + delete factory; +} + +void ValidateBlockRestartInterval(int value, int expected) { + BlockBasedTableOptions table_options; + table_options.block_restart_interval = value; + BlockBasedTableFactory* factory = new BlockBasedTableFactory(table_options); + + const BlockBasedTableOptions* normalized_table_options = + (const BlockBasedTableOptions*)factory->GetOptions(); + ASSERT_EQ(normalized_table_options->block_restart_interval, expected); + + delete factory; +} + +TEST_F(BlockBasedTableTest, InvalidOptions) { + // invalid values for block_size_deviation (<0 or >100) are silently set to 0 + ValidateBlockSizeDeviation(-10, 0); + ValidateBlockSizeDeviation(-1, 0); + ValidateBlockSizeDeviation(0, 0); + ValidateBlockSizeDeviation(1, 1); + ValidateBlockSizeDeviation(99, 99); + ValidateBlockSizeDeviation(100, 100); + ValidateBlockSizeDeviation(101, 0); + ValidateBlockSizeDeviation(1000, 0); + + // invalid values for block_restart_interval (<1) are silently set to 1 + ValidateBlockRestartInterval(-10, 1); + ValidateBlockRestartInterval(-1, 1); + ValidateBlockRestartInterval(0, 1); + ValidateBlockRestartInterval(1, 1); + ValidateBlockRestartInterval(2, 2); + ValidateBlockRestartInterval(1000, 1000); +} + +TEST_F(BlockBasedTableTest, BlockReadCountTest) { + // bloom_filter_type = 0 -- block-based filter + // bloom_filter_type = 0 -- full filter + for (int bloom_filter_type = 0; bloom_filter_type < 2; ++bloom_filter_type) { + for (int index_and_filter_in_cache = 0; index_and_filter_in_cache < 2; + ++index_and_filter_in_cache) { + Options options; + options.create_if_missing = true; + + BlockBasedTableOptions table_options; + table_options.block_cache = NewLRUCache(1, 0); + table_options.cache_index_and_filter_blocks = index_and_filter_in_cache; + table_options.filter_policy.reset( + NewBloomFilterPolicy(10, bloom_filter_type == 0)); + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + std::vector keys; + stl_wrappers::KVMap kvmap; + + TableConstructor c(BytewiseComparator()); + std::string user_key = "k04"; + InternalKey internal_key(user_key, 0, kTypeValue); + std::string encoded_key = internal_key.Encode().ToString(); + c.Add(encoded_key, "hello"); + ImmutableCFOptions ioptions(options); + // Generate table with filter policy + c.Finish(options, ioptions, table_options, + GetPlainInternalComparator(options.comparator), &keys, &kvmap); + auto reader = c.GetTableReader(); + PinnableSlice value; + GetContext get_context(options.comparator, nullptr, nullptr, nullptr, + GetContext::kNotFound, user_key, &value, nullptr, + nullptr, nullptr, nullptr); + get_perf_context()->Reset(); + ASSERT_OK(reader->Get(ReadOptions(), encoded_key, &get_context)); + if (index_and_filter_in_cache) { + // data, index and filter block + ASSERT_EQ(get_perf_context()->block_read_count, 3); + } else { + // just the data block + ASSERT_EQ(get_perf_context()->block_read_count, 1); + } + ASSERT_EQ(get_context.State(), GetContext::kFound); + ASSERT_STREQ(value.data(), "hello"); + + // Get non-existing key + user_key = "does-not-exist"; + internal_key = InternalKey(user_key, 0, kTypeValue); + encoded_key = internal_key.Encode().ToString(); + + value.Reset(); + get_context = GetContext(options.comparator, nullptr, nullptr, nullptr, + GetContext::kNotFound, user_key, &value, nullptr, + nullptr, nullptr, nullptr); + get_perf_context()->Reset(); + ASSERT_OK(reader->Get(ReadOptions(), encoded_key, &get_context)); + ASSERT_EQ(get_context.State(), GetContext::kNotFound); + + if (index_and_filter_in_cache) { + if (bloom_filter_type == 0) { + // with block-based, we read index and then the filter + ASSERT_EQ(get_perf_context()->block_read_count, 2); + } else { + // with full-filter, we read filter first and then we stop + ASSERT_EQ(get_perf_context()->block_read_count, 1); + } + } else { + // filter is already in memory and it figures out that the key doesn't + // exist + ASSERT_EQ(get_perf_context()->block_read_count, 0); + } + } + } +} + +// A wrapper around LRICache that also keeps track of data blocks (in contrast +// with the objects) in the cache. The class is very simple and can be used only +// for trivial tests. +class MockCache : public LRUCache { + public: + MockCache(size_t capacity, int num_shard_bits, bool strict_capacity_limit, + double high_pri_pool_ratio) + : LRUCache(capacity, num_shard_bits, strict_capacity_limit, + high_pri_pool_ratio) {} + virtual Status Insert(const Slice& key, void* value, size_t charge, + void (*deleter)(const Slice& key, void* value), + Handle** handle = nullptr, + Priority priority = Priority::LOW) override { + // Replace the deleter with our own so that we keep track of data blocks + // erased from the cache + deleters_[key.ToString()] = deleter; + return ShardedCache::Insert(key, value, charge, &MockDeleter, handle, + priority); + } + // This is called by the application right after inserting a data block + virtual void TEST_mark_as_data_block(const Slice& key, + size_t charge) override { + marked_data_in_cache_[key.ToString()] = charge; + marked_size_ += charge; + } + using DeleterFunc = void (*)(const Slice& key, void* value); + static std::map deleters_; + static std::map marked_data_in_cache_; + static size_t marked_size_; + static void MockDeleter(const Slice& key, void* value) { + // If the item was marked for being data block, decrease its usage from the + // total data block usage of the cache + if (marked_data_in_cache_.find(key.ToString()) != + marked_data_in_cache_.end()) { + marked_size_ -= marked_data_in_cache_[key.ToString()]; + } + // Then call the origianl deleter + assert(deleters_.find(key.ToString()) != deleters_.end()); + auto deleter = deleters_[key.ToString()]; + deleter(key, value); + } +}; + +size_t MockCache::marked_size_ = 0; +std::map MockCache::deleters_; +std::map MockCache::marked_data_in_cache_; + +// Block cache can contain raw data blocks as well as general objects. If an +// object depends on the table to be live, it then must be destructed before the +// table is closed. This test makes sure that the only items remains in the +// cache after the table is closed are raw data blocks. +TEST_F(BlockBasedTableTest, NoObjectInCacheAfterTableClose) { + for (auto index_type : + {BlockBasedTableOptions::IndexType::kBinarySearch, + BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch}) { + for (bool block_based_filter : {true, false}) { + for (bool partition_filter : {true, false}) { + if (partition_filter && + (block_based_filter || + index_type != + BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch)) { + continue; + } + for (bool index_and_filter_in_cache : {true, false}) { + for (bool pin_l0 : {true, false}) { + if (pin_l0 && !index_and_filter_in_cache) { + continue; + } + // Create a table + Options opt; + unique_ptr ikc; + ikc.reset(new test::PlainInternalKeyComparator(opt.comparator)); + opt.compression = kNoCompression; + BlockBasedTableOptions table_options; + table_options.block_size = 1024; + table_options.index_type = + BlockBasedTableOptions::IndexType::kTwoLevelIndexSearch; + table_options.pin_l0_filter_and_index_blocks_in_cache = pin_l0; + table_options.partition_filters = partition_filter; + table_options.cache_index_and_filter_blocks = + index_and_filter_in_cache; + // big enough so we don't ever lose cached values. + table_options.block_cache = std::shared_ptr( + new MockCache(16 * 1024 * 1024, 4, false, 0.0)); + table_options.filter_policy.reset( + rocksdb::NewBloomFilterPolicy(10, block_based_filter)); + opt.table_factory.reset(NewBlockBasedTableFactory(table_options)); + + TableConstructor c(BytewiseComparator()); + std::string user_key = "k01"; + std::string key = + InternalKey(user_key, 0, kTypeValue).Encode().ToString(); + c.Add(key, "hello"); + std::vector keys; + stl_wrappers::KVMap kvmap; + const ImmutableCFOptions ioptions(opt); + c.Finish(opt, ioptions, table_options, *ikc, &keys, &kvmap); + + // Doing a read to make index/filter loaded into the cache + auto table_reader = + dynamic_cast(c.GetTableReader()); + PinnableSlice value; + GetContext get_context(opt.comparator, nullptr, nullptr, nullptr, + GetContext::kNotFound, user_key, &value, + nullptr, nullptr, nullptr, nullptr); + InternalKey ikey(user_key, 0, kTypeValue); + auto s = table_reader->Get(ReadOptions(), key, &get_context); + ASSERT_EQ(get_context.State(), GetContext::kFound); + ASSERT_STREQ(value.data(), "hello"); + + // Close the table + c.ResetTableReader(); + + auto usage = table_options.block_cache->GetUsage(); + auto pinned_usage = table_options.block_cache->GetPinnedUsage(); + // The only usage must be for marked data blocks + ASSERT_EQ(usage, MockCache::marked_size_); + // There must be some pinned data since PinnableSlice has not + // released them yet + ASSERT_GT(pinned_usage, 0); + // Release pinnable slice reousrces + value.Reset(); + pinned_usage = table_options.block_cache->GetPinnedUsage(); + ASSERT_EQ(pinned_usage, 0); + } + } + } + } } } -TEST(BlockBasedTableTest, BlockCacheLeak) { +TEST_F(BlockBasedTableTest, BlockCacheLeak) { // Check that when we reopen a table we don't lose access to blocks already // in the cache. This test checks whether the Table actually makes use of the // unique ID from the file. @@ -1570,10 +2268,10 @@ TEST(BlockBasedTableTest, BlockCacheLeak) { BlockBasedTableOptions table_options; table_options.block_size = 1024; // big enough so we don't ever lose cached values. - table_options.block_cache = NewLRUCache(16 * 1024 * 1024); + table_options.block_cache = NewLRUCache(16 * 1024 * 1024, 4); opt.table_factory.reset(NewBlockBasedTableFactory(table_options)); - TableConstructor c(BytewiseComparator()); + TableConstructor c(BytewiseComparator(), true /* convert_to_internal_key_ */); c.Add("k01", "hello"); c.Add("k02", "hello2"); c.Add("k03", std::string(10000, 'x')); @@ -1582,10 +2280,11 @@ TEST(BlockBasedTableTest, BlockCacheLeak) { c.Add("k06", "hello3"); c.Add("k07", std::string(100000, 'x')); std::vector keys; - KVMap kvmap; - c.Finish(opt, table_options, *ikc, &keys, &kvmap); + stl_wrappers::KVMap kvmap; + const ImmutableCFOptions ioptions(opt); + c.Finish(opt, ioptions, table_options, *ikc, &keys, &kvmap); - unique_ptr iter(c.NewIterator()); + unique_ptr iter(c.NewIterator()); iter->SeekToFirst(); while (iter->Valid()) { iter->key(); @@ -1594,34 +2293,112 @@ TEST(BlockBasedTableTest, BlockCacheLeak) { } ASSERT_OK(iter->status()); - ASSERT_OK(c.Reopen(opt)); + const ImmutableCFOptions ioptions1(opt); + ASSERT_OK(c.Reopen(ioptions1)); auto table_reader = dynamic_cast(c.GetTableReader()); for (const std::string& key : keys) { ASSERT_TRUE(table_reader->TEST_KeyInCache(ReadOptions(), key)); } + c.ResetTableReader(); // rerun with different block cache - table_options.block_cache = NewLRUCache(16 * 1024 * 1024); + table_options.block_cache = NewLRUCache(16 * 1024 * 1024, 4); opt.table_factory.reset(NewBlockBasedTableFactory(table_options)); - ASSERT_OK(c.Reopen(opt)); + const ImmutableCFOptions ioptions2(opt); + ASSERT_OK(c.Reopen(ioptions2)); table_reader = dynamic_cast(c.GetTableReader()); for (const std::string& key : keys) { ASSERT_TRUE(!table_reader->TEST_KeyInCache(ReadOptions(), key)); } + c.ResetTableReader(); +} + +TEST_F(BlockBasedTableTest, NewIndexIteratorLeak) { + // A regression test to avoid data race described in + // https://github.com/facebook/rocksdb/issues/1267 + TableConstructor c(BytewiseComparator(), true /* convert_to_internal_key_ */); + std::vector keys; + stl_wrappers::KVMap kvmap; + c.Add("a1", "val1"); + Options options; + options.prefix_extractor.reset(NewFixedPrefixTransform(1)); + BlockBasedTableOptions table_options; + table_options.index_type = BlockBasedTableOptions::kHashSearch; + table_options.cache_index_and_filter_blocks = true; + table_options.block_cache = NewLRUCache(0); + options.table_factory.reset(NewBlockBasedTableFactory(table_options)); + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, + GetPlainInternalComparator(options.comparator), &keys, &kvmap); + + rocksdb::SyncPoint::GetInstance()->LoadDependencyAndMarkers( + { + {"BlockBasedTable::NewIndexIterator::thread1:1", + "BlockBasedTable::NewIndexIterator::thread2:2"}, + {"BlockBasedTable::NewIndexIterator::thread2:3", + "BlockBasedTable::NewIndexIterator::thread1:4"}, + }, + { + {"BlockBasedTableTest::NewIndexIteratorLeak:Thread1Marker", + "BlockBasedTable::NewIndexIterator::thread1:1"}, + {"BlockBasedTableTest::NewIndexIteratorLeak:Thread1Marker", + "BlockBasedTable::NewIndexIterator::thread1:4"}, + {"BlockBasedTableTest::NewIndexIteratorLeak:Thread2Marker", + "BlockBasedTable::NewIndexIterator::thread2:2"}, + {"BlockBasedTableTest::NewIndexIteratorLeak:Thread2Marker", + "BlockBasedTable::NewIndexIterator::thread2:3"}, + }); + + rocksdb::SyncPoint::GetInstance()->EnableProcessing(); + ReadOptions ro; + auto* reader = c.GetTableReader(); + + std::function func1 = [&]() { + TEST_SYNC_POINT("BlockBasedTableTest::NewIndexIteratorLeak:Thread1Marker"); + std::unique_ptr iter(reader->NewIterator(ro)); + iter->Seek(InternalKey("a1", 0, kTypeValue).Encode()); + }; + + std::function func2 = [&]() { + TEST_SYNC_POINT("BlockBasedTableTest::NewIndexIteratorLeak:Thread2Marker"); + std::unique_ptr iter(reader->NewIterator(ro)); + }; + + auto thread1 = port::Thread(func1); + auto thread2 = port::Thread(func2); + thread1.join(); + thread2.join(); + rocksdb::SyncPoint::GetInstance()->DisableProcessing(); + c.ResetTableReader(); } -TEST(PlainTableTest, BasicPlainTableProperties) { +// Plain table is not supported in ROCKSDB_LITE +#ifndef ROCKSDB_LITE +TEST_F(PlainTableTest, BasicPlainTableProperties) { PlainTableOptions plain_table_options; plain_table_options.user_key_len = 8; plain_table_options.bloom_bits_per_key = 8; plain_table_options.hash_table_ratio = 0; PlainTableFactory factory(plain_table_options); - StringSink sink; + test::StringSink sink; + unique_ptr file_writer( + test::GetWritableFileWriter(new test::StringSink())); Options options; + const ImmutableCFOptions ioptions(options); InternalKeyComparator ikc(options.comparator); - std::unique_ptr builder( - factory.NewTableBuilder(options, ikc, &sink, kNoCompression)); + std::vector> + int_tbl_prop_collector_factories; + std::string column_family_name; + int unknown_level = -1; + std::unique_ptr builder(factory.NewTableBuilder( + TableBuilderOptions(ioptions, ikc, &int_tbl_prop_collector_factories, + kNoCompression, CompressionOptions(), + nullptr /* compression_dict */, + false /* skip_filters */, column_family_name, + unknown_level), + TablePropertiesCollectorFactory::Context::kUnknownColumnFamily, + file_writer.get())); for (char c = 'a'; c <= 'z'; ++c) { std::string key(8, c); @@ -1630,12 +2407,17 @@ TEST(PlainTableTest, BasicPlainTableProperties) { builder->Add(key, value); } ASSERT_OK(builder->Finish()); + file_writer->Flush(); - StringSource source(sink.contents(), 72242, true); + test::StringSink* ss = + static_cast(file_writer->writable_file()); + unique_ptr file_reader( + test::GetRandomAccessFileReader( + new test::StringSource(ss->contents(), 72242, true))); TableProperties* props = nullptr; - auto s = ReadTableProperties(&source, sink.contents().size(), - kPlainTableMagicNumber, Env::Default(), nullptr, + auto s = ReadTableProperties(file_reader.get(), ss->contents().size(), + kPlainTableMagicNumber, ioptions, &props); std::unique_ptr props_guard(props); ASSERT_OK(s); @@ -1647,9 +2429,10 @@ TEST(PlainTableTest, BasicPlainTableProperties) { ASSERT_EQ(26ul, props->num_entries); ASSERT_EQ(1ul, props->num_data_blocks); } +#endif // !ROCKSDB_LITE -TEST(GeneralTableTest, ApproximateOffsetOfPlain) { - TableConstructor c(BytewiseComparator()); +TEST_F(GeneralTableTest, ApproximateOffsetOfPlain) { + TableConstructor c(BytewiseComparator(), true /* convert_to_internal_key_ */); c.Add("k01", "hello"); c.Add("k02", "hello2"); c.Add("k03", std::string(10000, 'x')); @@ -1658,13 +2441,15 @@ TEST(GeneralTableTest, ApproximateOffsetOfPlain) { c.Add("k06", "hello3"); c.Add("k07", std::string(100000, 'x')); std::vector keys; - KVMap kvmap; + stl_wrappers::KVMap kvmap; Options options; test::PlainInternalKeyComparator internal_comparator(options.comparator); options.compression = kNoCompression; BlockBasedTableOptions table_options; table_options.block_size = 1024; - c.Finish(options, table_options, internal_comparator, &keys, &kvmap); + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, internal_comparator, + &keys, &kvmap); ASSERT_TRUE(Between(c.ApproximateOffsetOf("abc"), 0, 0)); ASSERT_TRUE(Between(c.ApproximateOffsetOf("k01"), 0, 0)); @@ -1672,29 +2457,33 @@ TEST(GeneralTableTest, ApproximateOffsetOfPlain) { ASSERT_TRUE(Between(c.ApproximateOffsetOf("k02"), 0, 0)); ASSERT_TRUE(Between(c.ApproximateOffsetOf("k03"), 0, 0)); ASSERT_TRUE(Between(c.ApproximateOffsetOf("k04"), 10000, 11000)); - ASSERT_TRUE(Between(c.ApproximateOffsetOf("k04a"), 210000, 211000)); + // k04 and k05 will be in two consecutive blocks, the index is + // an arbitrary slice between k04 and k05, either before or after k04a + ASSERT_TRUE(Between(c.ApproximateOffsetOf("k04a"), 10000, 211000)); ASSERT_TRUE(Between(c.ApproximateOffsetOf("k05"), 210000, 211000)); ASSERT_TRUE(Between(c.ApproximateOffsetOf("k06"), 510000, 511000)); ASSERT_TRUE(Between(c.ApproximateOffsetOf("k07"), 510000, 511000)); ASSERT_TRUE(Between(c.ApproximateOffsetOf("xyz"), 610000, 612000)); + c.ResetTableReader(); } static void DoCompressionTest(CompressionType comp) { Random rnd(301); - TableConstructor c(BytewiseComparator()); + TableConstructor c(BytewiseComparator(), true /* convert_to_internal_key_ */); std::string tmp; c.Add("k01", "hello"); c.Add("k02", test::CompressibleString(&rnd, 0.25, 10000, &tmp)); c.Add("k03", "hello3"); c.Add("k04", test::CompressibleString(&rnd, 0.25, 10000, &tmp)); std::vector keys; - KVMap kvmap; + stl_wrappers::KVMap kvmap; Options options; test::PlainInternalKeyComparator ikc(options.comparator); options.compression = comp; BlockBasedTableOptions table_options; table_options.block_size = 1024; - c.Finish(options, table_options, ikc, &keys, &kvmap); + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, ikc, &keys, &kvmap); ASSERT_TRUE(Between(c.ApproximateOffsetOf("abc"), 0, 0)); ASSERT_TRUE(Between(c.ApproximateOffsetOf("k01"), 0, 0)); @@ -1702,17 +2491,18 @@ static void DoCompressionTest(CompressionType comp) { ASSERT_TRUE(Between(c.ApproximateOffsetOf("k03"), 2000, 3000)); ASSERT_TRUE(Between(c.ApproximateOffsetOf("k04"), 2000, 3000)); ASSERT_TRUE(Between(c.ApproximateOffsetOf("xyz"), 4000, 6100)); + c.ResetTableReader(); } -TEST(GeneralTableTest, ApproximateOffsetOfCompressed) { +TEST_F(GeneralTableTest, ApproximateOffsetOfCompressed) { std::vector compression_state; - if (!SnappyCompressionSupported()) { + if (!Snappy_Supported()) { fprintf(stderr, "skipping snappy compression tests\n"); } else { compression_state.push_back(kSnappyCompression); } - if (!ZlibCompressionSupported()) { + if (!Zlib_Supported()) { fprintf(stderr, "skipping zlib compression tests\n"); } else { compression_state.push_back(kZlibCompression); @@ -1720,23 +2510,25 @@ TEST(GeneralTableTest, ApproximateOffsetOfCompressed) { // TODO(kailiu) DoCompressionTest() doesn't work with BZip2. /* - if (!BZip2CompressionSupported()) { + if (!BZip2_Supported()) { fprintf(stderr, "skipping bzip2 compression tests\n"); } else { compression_state.push_back(kBZip2Compression); } */ - if (!LZ4CompressionSupported()) { - fprintf(stderr, "skipping lz4 compression tests\n"); + if (!LZ4_Supported()) { + fprintf(stderr, "skipping lz4 and lz4hc compression tests\n"); } else { compression_state.push_back(kLZ4Compression); + compression_state.push_back(kLZ4HCCompression); } - if (!LZ4HCCompressionSupported()) { - fprintf(stderr, "skipping lz4hc compression tests\n"); - } else { - compression_state.push_back(kLZ4HCCompression); + if (!XPRESS_Supported()) { + fprintf(stderr, "skipping xpress and xpress compression tests\n"); + } + else { + compression_state.push_back(kXpressCompression); } for (auto state : compression_state) { @@ -1744,7 +2536,7 @@ TEST(GeneralTableTest, ApproximateOffsetOfCompressed) { } } -TEST(Harness, Randomized) { +TEST_F(HarnessTest, Randomized) { std::vector args = GenerateArgList(); for (unsigned int i = 0; i < args.size(); i++) { Init(args[i]); @@ -1765,9 +2557,10 @@ TEST(Harness, Randomized) { } } -TEST(Harness, RandomizedLongDB) { +#ifndef ROCKSDB_LITE +TEST_F(HarnessTest, RandomizedLongDB) { Random rnd(test::RandomSeed()); - TestArgs args = { DB_TEST, false, 16, kNoCompression }; + TestArgs args = {DB_TEST, false, 16, kNoCompression, 0, false}; Init(args); int num_entries = 100000; for (int e = 0; e < num_entries; e++) { @@ -1788,15 +2581,20 @@ TEST(Harness, RandomizedLongDB) { } ASSERT_GT(files, 0); } +#endif // ROCKSDB_LITE -class MemTableTest { }; +class MemTableTest : public testing::Test {}; -TEST(MemTableTest, Simple) { +TEST_F(MemTableTest, Simple) { InternalKeyComparator cmp(BytewiseComparator()); auto table_factory = std::make_shared(); Options options; options.memtable_factory = table_factory; - MemTable* memtable = new MemTable(cmp, options); + ImmutableCFOptions ioptions(options); + WriteBufferManager wb(options.db_write_buffer_size); + MemTable* memtable = + new MemTable(cmp, ioptions, MutableCFOptions(options), &wb, + kMaxSequenceNumber, 0 /* column_family_id */); memtable->Ref(); WriteBatch batch; WriteBatchInternal::SetSequence(&batch, 100); @@ -1804,24 +2602,40 @@ TEST(MemTableTest, Simple) { batch.Put(std::string("k2"), std::string("v2")); batch.Put(std::string("k3"), std::string("v3")); batch.Put(std::string("largekey"), std::string("vlarge")); - ColumnFamilyMemTablesDefault cf_mems_default(memtable, &options); - ASSERT_TRUE(WriteBatchInternal::InsertInto(&batch, &cf_mems_default).ok()); - - Iterator* iter = memtable->NewIterator(ReadOptions()); - iter->SeekToFirst(); - while (iter->Valid()) { - fprintf(stderr, "key: '%s' -> '%s'\n", - iter->key().ToString().c_str(), - iter->value().ToString().c_str()); - iter->Next(); + batch.DeleteRange(std::string("chi"), std::string("xigua")); + batch.DeleteRange(std::string("begin"), std::string("end")); + ColumnFamilyMemTablesDefault cf_mems_default(memtable); + ASSERT_TRUE( + WriteBatchInternal::InsertInto(&batch, &cf_mems_default, nullptr).ok()); + + for (int i = 0; i < 2; ++i) { + Arena arena; + ScopedArenaIterator arena_iter_guard; + std::unique_ptr iter_guard; + InternalIterator* iter; + if (i == 0) { + iter = memtable->NewIterator(ReadOptions(), &arena); + arena_iter_guard.set(iter); + } else { + iter = memtable->NewRangeTombstoneIterator(ReadOptions()); + iter_guard.reset(iter); + } + if (iter == nullptr) { + continue; + } + iter->SeekToFirst(); + while (iter->Valid()) { + fprintf(stderr, "key: '%s' -> '%s'\n", iter->key().ToString().c_str(), + iter->value().ToString().c_str()); + iter->Next(); + } } - delete iter; delete memtable->Unref(); } // Test the empty key -TEST(Harness, SimpleEmptyKey) { +TEST_F(HarnessTest, SimpleEmptyKey) { auto args = GenerateArgList(); for (const auto& arg : args) { Init(arg); @@ -1831,7 +2645,7 @@ TEST(Harness, SimpleEmptyKey) { } } -TEST(Harness, SimpleSingle) { +TEST_F(HarnessTest, SimpleSingle) { auto args = GenerateArgList(); for (const auto& arg : args) { Init(arg); @@ -1841,7 +2655,7 @@ TEST(Harness, SimpleSingle) { } } -TEST(Harness, SimpleMulti) { +TEST_F(HarnessTest, SimpleMulti) { auto args = GenerateArgList(); for (const auto& arg : args) { Init(arg); @@ -1853,7 +2667,7 @@ TEST(Harness, SimpleMulti) { } } -TEST(Harness, SimpleSpecialKey) { +TEST_F(HarnessTest, SimpleSpecialKey) { auto args = GenerateArgList(); for (const auto& arg : args) { Init(arg); @@ -1863,11 +2677,11 @@ TEST(Harness, SimpleSpecialKey) { } } -TEST(Harness, FooterTests) { +TEST_F(HarnessTest, FooterTests) { { // upconvert legacy block based std::string encoded; - Footer footer(kLegacyBlockBasedTableMagicNumber); + Footer footer(kLegacyBlockBasedTableMagicNumber, 0); BlockHandle meta_index(10, 5), index(20, 15); footer.set_metaindex_handle(meta_index); footer.set_index_handle(index); @@ -1881,11 +2695,12 @@ TEST(Harness, FooterTests) { ASSERT_EQ(decoded_footer.metaindex_handle().size(), meta_index.size()); ASSERT_EQ(decoded_footer.index_handle().offset(), index.offset()); ASSERT_EQ(decoded_footer.index_handle().size(), index.size()); + ASSERT_EQ(decoded_footer.version(), 0U); } { // xxhash block based std::string encoded; - Footer footer(kBlockBasedTableMagicNumber); + Footer footer(kBlockBasedTableMagicNumber, 1); BlockHandle meta_index(10, 5), index(20, 15); footer.set_metaindex_handle(meta_index); footer.set_index_handle(index); @@ -1900,11 +2715,14 @@ TEST(Harness, FooterTests) { ASSERT_EQ(decoded_footer.metaindex_handle().size(), meta_index.size()); ASSERT_EQ(decoded_footer.index_handle().offset(), index.offset()); ASSERT_EQ(decoded_footer.index_handle().size(), index.size()); + ASSERT_EQ(decoded_footer.version(), 1U); } +// Plain table is not supported in ROCKSDB_LITE +#ifndef ROCKSDB_LITE { // upconvert legacy plain table std::string encoded; - Footer footer(kLegacyPlainTableMagicNumber); + Footer footer(kLegacyPlainTableMagicNumber, 0); BlockHandle meta_index(10, 5), index(20, 15); footer.set_metaindex_handle(meta_index); footer.set_index_handle(index); @@ -1918,11 +2736,12 @@ TEST(Harness, FooterTests) { ASSERT_EQ(decoded_footer.metaindex_handle().size(), meta_index.size()); ASSERT_EQ(decoded_footer.index_handle().offset(), index.offset()); ASSERT_EQ(decoded_footer.index_handle().size(), index.size()); + ASSERT_EQ(decoded_footer.version(), 0U); } { // xxhash block based std::string encoded; - Footer footer(kPlainTableMagicNumber); + Footer footer(kPlainTableMagicNumber, 1); BlockHandle meta_index(10, 5), index(20, 15); footer.set_metaindex_handle(meta_index); footer.set_index_handle(index); @@ -1937,11 +2756,353 @@ TEST(Harness, FooterTests) { ASSERT_EQ(decoded_footer.metaindex_handle().size(), meta_index.size()); ASSERT_EQ(decoded_footer.index_handle().offset(), index.offset()); ASSERT_EQ(decoded_footer.index_handle().size(), index.size()); + ASSERT_EQ(decoded_footer.version(), 1U); + } +#endif // !ROCKSDB_LITE + { + // version == 2 + std::string encoded; + Footer footer(kBlockBasedTableMagicNumber, 2); + BlockHandle meta_index(10, 5), index(20, 15); + footer.set_metaindex_handle(meta_index); + footer.set_index_handle(index); + footer.EncodeTo(&encoded); + Footer decoded_footer; + Slice encoded_slice(encoded); + decoded_footer.DecodeFrom(&encoded_slice); + ASSERT_EQ(decoded_footer.table_magic_number(), kBlockBasedTableMagicNumber); + ASSERT_EQ(decoded_footer.checksum(), kCRC32c); + ASSERT_EQ(decoded_footer.metaindex_handle().offset(), meta_index.offset()); + ASSERT_EQ(decoded_footer.metaindex_handle().size(), meta_index.size()); + ASSERT_EQ(decoded_footer.index_handle().offset(), index.offset()); + ASSERT_EQ(decoded_footer.index_handle().size(), index.size()); + ASSERT_EQ(decoded_footer.version(), 2U); } } +class IndexBlockRestartIntervalTest + : public BlockBasedTableTest, + public ::testing::WithParamInterface { + public: + static std::vector GetRestartValues() { return {-1, 0, 1, 8, 16, 32}; } +}; + +INSTANTIATE_TEST_CASE_P( + IndexBlockRestartIntervalTest, IndexBlockRestartIntervalTest, + ::testing::ValuesIn(IndexBlockRestartIntervalTest::GetRestartValues())); + +TEST_P(IndexBlockRestartIntervalTest, IndexBlockRestartInterval) { + const int kKeysInTable = 10000; + const int kKeySize = 100; + const int kValSize = 500; + + int index_block_restart_interval = GetParam(); + + Options options; + BlockBasedTableOptions table_options; + table_options.block_size = 64; // small block size to get big index block + table_options.index_block_restart_interval = index_block_restart_interval; + options.table_factory.reset(new BlockBasedTableFactory(table_options)); + + TableConstructor c(BytewiseComparator()); + static Random rnd(301); + for (int i = 0; i < kKeysInTable; i++) { + InternalKey k(RandomString(&rnd, kKeySize), 0, kTypeValue); + c.Add(k.Encode().ToString(), RandomString(&rnd, kValSize)); + } + + std::vector keys; + stl_wrappers::KVMap kvmap; + std::unique_ptr comparator( + new InternalKeyComparator(BytewiseComparator())); + const ImmutableCFOptions ioptions(options); + c.Finish(options, ioptions, table_options, *comparator, &keys, &kvmap); + auto reader = c.GetTableReader(); + + std::unique_ptr db_iter(reader->NewIterator(ReadOptions())); + + // Test point lookup + for (auto& kv : kvmap) { + db_iter->Seek(kv.first); + + ASSERT_TRUE(db_iter->Valid()); + ASSERT_OK(db_iter->status()); + ASSERT_EQ(db_iter->key(), kv.first); + ASSERT_EQ(db_iter->value(), kv.second); + } + + // Test iterating + auto kv_iter = kvmap.begin(); + for (db_iter->SeekToFirst(); db_iter->Valid(); db_iter->Next()) { + ASSERT_EQ(db_iter->key(), kv_iter->first); + ASSERT_EQ(db_iter->value(), kv_iter->second); + kv_iter++; + } + ASSERT_EQ(kv_iter, kvmap.end()); + c.ResetTableReader(); +} + +class PrefixTest : public testing::Test { + public: + PrefixTest() : testing::Test() {} + ~PrefixTest() {} +}; + +namespace { +// A simple PrefixExtractor that only works for test PrefixAndWholeKeyTest +class TestPrefixExtractor : public rocksdb::SliceTransform { + public: + ~TestPrefixExtractor() override{}; + const char* Name() const override { return "TestPrefixExtractor"; } + + rocksdb::Slice Transform(const rocksdb::Slice& src) const override { + assert(IsValid(src)); + return rocksdb::Slice(src.data(), 3); + } + + bool InDomain(const rocksdb::Slice& src) const override { + assert(IsValid(src)); + return true; + } + + bool InRange(const rocksdb::Slice& dst) const override { return true; } + + bool IsValid(const rocksdb::Slice& src) const { + if (src.size() != 4) { + return false; + } + if (src[0] != '[') { + return false; + } + if (src[1] < '0' || src[1] > '9') { + return false; + } + if (src[2] != ']') { + return false; + } + if (src[3] < '0' || src[3] > '9') { + return false; + } + return true; + } +}; +} // namespace + +TEST_F(PrefixTest, PrefixAndWholeKeyTest) { + rocksdb::Options options; + options.compaction_style = rocksdb::kCompactionStyleUniversal; + options.num_levels = 20; + options.create_if_missing = true; + options.optimize_filters_for_hits = false; + options.target_file_size_base = 268435456; + options.prefix_extractor = std::make_shared(); + rocksdb::BlockBasedTableOptions bbto; + bbto.filter_policy.reset(rocksdb::NewBloomFilterPolicy(10)); + bbto.block_size = 262144; + bbto.whole_key_filtering = true; + + const std::string kDBPath = test::TmpDir() + "/table_prefix_test"; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + DestroyDB(kDBPath, options); + rocksdb::DB* db; + ASSERT_OK(rocksdb::DB::Open(options, kDBPath, &db)); + + // Create a bunch of keys with 10 filters. + for (int i = 0; i < 10; i++) { + std::string prefix = "[" + std::to_string(i) + "]"; + for (int j = 0; j < 10; j++) { + std::string key = prefix + std::to_string(j); + db->Put(rocksdb::WriteOptions(), key, "1"); + } + } + + // Trigger compaction. + db->CompactRange(CompactRangeOptions(), nullptr, nullptr); + delete db; + // In the second round, turn whole_key_filtering off and expect + // rocksdb still works. +} + +TEST_F(BlockBasedTableTest, TableWithGlobalSeqno) { + BlockBasedTableOptions bbto; + test::StringSink* sink = new test::StringSink(); + unique_ptr file_writer(test::GetWritableFileWriter(sink)); + Options options; + options.table_factory.reset(NewBlockBasedTableFactory(bbto)); + const ImmutableCFOptions ioptions(options); + InternalKeyComparator ikc(options.comparator); + std::vector> + int_tbl_prop_collector_factories; + int_tbl_prop_collector_factories.emplace_back( + new SstFileWriterPropertiesCollectorFactory(2 /* version */, + 0 /* global_seqno*/)); + std::string column_family_name; + std::unique_ptr builder(options.table_factory->NewTableBuilder( + TableBuilderOptions(ioptions, ikc, &int_tbl_prop_collector_factories, + kNoCompression, CompressionOptions(), + nullptr /* compression_dict */, + false /* skip_filters */, column_family_name, -1), + TablePropertiesCollectorFactory::Context::kUnknownColumnFamily, + file_writer.get())); + + for (char c = 'a'; c <= 'z'; ++c) { + std::string key(8, c); + std::string value = key; + InternalKey ik(key, 0, kTypeValue); + + builder->Add(ik.Encode(), value); + } + ASSERT_OK(builder->Finish()); + file_writer->Flush(); + + test::RandomRWStringSink ss_rw(sink); + uint32_t version; + uint64_t global_seqno; + uint64_t global_seqno_offset; + + // Helper function to get version, global_seqno, global_seqno_offset + std::function GetVersionAndGlobalSeqno = [&]() { + unique_ptr file_reader( + test::GetRandomAccessFileReader( + new test::StringSource(ss_rw.contents(), 73342, true))); + + TableProperties* props = nullptr; + ASSERT_OK(ReadTableProperties(file_reader.get(), ss_rw.contents().size(), + kBlockBasedTableMagicNumber, ioptions, + &props)); + + UserCollectedProperties user_props = props->user_collected_properties; + version = DecodeFixed32( + user_props[ExternalSstFilePropertyNames::kVersion].c_str()); + global_seqno = DecodeFixed64( + user_props[ExternalSstFilePropertyNames::kGlobalSeqno].c_str()); + global_seqno_offset = + props->properties_offsets[ExternalSstFilePropertyNames::kGlobalSeqno]; + + delete props; + }; + + // Helper function to update the value of the global seqno in the file + std::function SetGlobalSeqno = [&](uint64_t val) { + std::string new_global_seqno; + PutFixed64(&new_global_seqno, val); + + ASSERT_OK(ss_rw.Write(global_seqno_offset, new_global_seqno)); + }; + + // Helper function to get the contents of the table InternalIterator + unique_ptr table_reader; + std::function GetTableInternalIter = [&]() { + unique_ptr file_reader( + test::GetRandomAccessFileReader( + new test::StringSource(ss_rw.contents(), 73342, true))); + + options.table_factory->NewTableReader( + TableReaderOptions(ioptions, EnvOptions(), ikc), std::move(file_reader), + ss_rw.contents().size(), &table_reader); + + return table_reader->NewIterator(ReadOptions()); + }; + + GetVersionAndGlobalSeqno(); + ASSERT_EQ(2, version); + ASSERT_EQ(0, global_seqno); + + InternalIterator* iter = GetTableInternalIter(); + char current_c = 'a'; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ParsedInternalKey pik; + ASSERT_TRUE(ParseInternalKey(iter->key(), &pik)); + + ASSERT_EQ(pik.type, ValueType::kTypeValue); + ASSERT_EQ(pik.sequence, 0); + ASSERT_EQ(pik.user_key, iter->value()); + ASSERT_EQ(pik.user_key.ToString(), std::string(8, current_c)); + current_c++; + } + ASSERT_EQ(current_c, 'z' + 1); + delete iter; + + // Update global sequence number to 10 + SetGlobalSeqno(10); + GetVersionAndGlobalSeqno(); + ASSERT_EQ(2, version); + ASSERT_EQ(10, global_seqno); + + iter = GetTableInternalIter(); + current_c = 'a'; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ParsedInternalKey pik; + ASSERT_TRUE(ParseInternalKey(iter->key(), &pik)); + + ASSERT_EQ(pik.type, ValueType::kTypeValue); + ASSERT_EQ(pik.sequence, 10); + ASSERT_EQ(pik.user_key, iter->value()); + ASSERT_EQ(pik.user_key.ToString(), std::string(8, current_c)); + current_c++; + } + ASSERT_EQ(current_c, 'z' + 1); + + // Verify Seek + for (char c = 'a'; c <= 'z'; c++) { + std::string k = std::string(8, c); + InternalKey ik(k, 10, kValueTypeForSeek); + iter->Seek(ik.Encode()); + ASSERT_TRUE(iter->Valid()); + + ParsedInternalKey pik; + ASSERT_TRUE(ParseInternalKey(iter->key(), &pik)); + + ASSERT_EQ(pik.type, ValueType::kTypeValue); + ASSERT_EQ(pik.sequence, 10); + ASSERT_EQ(pik.user_key.ToString(), k); + ASSERT_EQ(iter->value().ToString(), k); + } + delete iter; + + // Update global sequence number to 3 + SetGlobalSeqno(3); + GetVersionAndGlobalSeqno(); + ASSERT_EQ(2, version); + ASSERT_EQ(3, global_seqno); + + iter = GetTableInternalIter(); + current_c = 'a'; + for (iter->SeekToFirst(); iter->Valid(); iter->Next()) { + ParsedInternalKey pik; + ASSERT_TRUE(ParseInternalKey(iter->key(), &pik)); + + ASSERT_EQ(pik.type, ValueType::kTypeValue); + ASSERT_EQ(pik.sequence, 3); + ASSERT_EQ(pik.user_key, iter->value()); + ASSERT_EQ(pik.user_key.ToString(), std::string(8, current_c)); + current_c++; + } + ASSERT_EQ(current_c, 'z' + 1); + + // Verify Seek + for (char c = 'a'; c <= 'z'; c++) { + std::string k = std::string(8, c); + // seqno=4 is less than 3 so we still should get our key + InternalKey ik(k, 4, kValueTypeForSeek); + iter->Seek(ik.Encode()); + ASSERT_TRUE(iter->Valid()); + + ParsedInternalKey pik; + ASSERT_TRUE(ParseInternalKey(iter->key(), &pik)); + + ASSERT_EQ(pik.type, ValueType::kTypeValue); + ASSERT_EQ(pik.sequence, 3); + ASSERT_EQ(pik.user_key.ToString(), k); + ASSERT_EQ(iter->value().ToString(), k); + } + + delete iter; +} + } // namespace rocksdb int main(int argc, char** argv) { - return rocksdb::test::RunAllTests(); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); } diff --git a/table/two_level_iterator.cc b/table/two_level_iterator.cc index ae4e46239bf..2236a2a726a 100644 --- a/table/two_level_iterator.cc +++ b/table/two_level_iterator.cc @@ -1,14 +1,14 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. See the AUTHORS file for names of contributors. #include "table/two_level_iterator.h" - +#include "db/pinned_iterators_manager.h" #include "rocksdb/options.h" #include "rocksdb/table.h" #include "table/block.h" @@ -19,34 +19,43 @@ namespace rocksdb { namespace { -class TwoLevelIterator: public Iterator { +class TwoLevelIterator : public InternalIterator { public: explicit TwoLevelIterator(TwoLevelIteratorState* state, - Iterator* first_level_iter); + InternalIterator* first_level_iter, + bool need_free_iter_and_state); virtual ~TwoLevelIterator() { - first_level_iter_.DeleteIter(false); + // Assert that the TwoLevelIterator is never deleted while Pinning is + // Enabled. + assert(!pinned_iters_mgr_ || + (pinned_iters_mgr_ && !pinned_iters_mgr_->PinningEnabled())); + first_level_iter_.DeleteIter(!need_free_iter_and_state_); second_level_iter_.DeleteIter(false); + if (need_free_iter_and_state_) { + delete state_; + } else { + state_->~TwoLevelIteratorState(); + } } - virtual void Seek(const Slice& target); - virtual void SeekToFirst(); - virtual void SeekToLast(); - virtual void Next(); - virtual void Prev(); + virtual void Seek(const Slice& target) override; + virtual void SeekForPrev(const Slice& target) override; + virtual void SeekToFirst() override; + virtual void SeekToLast() override; + virtual void Next() override; + virtual void Prev() override; - virtual bool Valid() const { - return second_level_iter_.Valid(); - } - virtual Slice key() const { + virtual bool Valid() const override { return second_level_iter_.Valid(); } + virtual Slice key() const override { assert(Valid()); return second_level_iter_.key(); } - virtual Slice value() const { + virtual Slice value() const override { assert(Valid()); return second_level_iter_.value(); } - virtual Status status() const { + virtual Status status() const override { // It'd be nice if status() returned a const Status& instead of a Status if (!first_level_iter_.status().ok()) { return first_level_iter_.status(); @@ -57,6 +66,22 @@ class TwoLevelIterator: public Iterator { return status_; } } + virtual void SetPinnedItersMgr( + PinnedIteratorsManager* pinned_iters_mgr) override { + pinned_iters_mgr_ = pinned_iters_mgr; + first_level_iter_.SetPinnedItersMgr(pinned_iters_mgr); + if (second_level_iter_.iter()) { + second_level_iter_.SetPinnedItersMgr(pinned_iters_mgr); + } + } + virtual bool IsKeyPinned() const override { + return pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled() && + second_level_iter_.iter() && second_level_iter_.IsKeyPinned(); + } + virtual bool IsValuePinned() const override { + return pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled() && + second_level_iter_.iter() && second_level_iter_.IsValuePinned(); + } private: void SaveError(const Status& s) { @@ -64,12 +89,14 @@ class TwoLevelIterator: public Iterator { } void SkipEmptyDataBlocksForward(); void SkipEmptyDataBlocksBackward(); - void SetSecondLevelIterator(Iterator* iter); + void SetSecondLevelIterator(InternalIterator* iter); void InitDataBlock(); - std::unique_ptr state_; + TwoLevelIteratorState* state_; IteratorWrapper first_level_iter_; IteratorWrapper second_level_iter_; // May be nullptr + bool need_free_iter_and_state_; + PinnedIteratorsManager* pinned_iters_mgr_; Status status_; // If second_level_iter is non-nullptr, then "data_block_handle_" holds the // "index_value" passed to block_function_ to create the second_level_iter. @@ -77,8 +104,12 @@ class TwoLevelIterator: public Iterator { }; TwoLevelIterator::TwoLevelIterator(TwoLevelIteratorState* state, - Iterator* first_level_iter) - : state_(state), first_level_iter_(first_level_iter) {} + InternalIterator* first_level_iter, + bool need_free_iter_and_state) + : state_(state), + first_level_iter_(first_level_iter), + need_free_iter_and_state_(need_free_iter_and_state), + pinned_iters_mgr_(nullptr) {} void TwoLevelIterator::Seek(const Slice& target) { if (state_->check_prefix_may_match && @@ -95,6 +126,28 @@ void TwoLevelIterator::Seek(const Slice& target) { SkipEmptyDataBlocksForward(); } +void TwoLevelIterator::SeekForPrev(const Slice& target) { + if (state_->check_prefix_may_match && !state_->PrefixMayMatch(target)) { + SetSecondLevelIterator(nullptr); + return; + } + first_level_iter_.Seek(target); + InitDataBlock(); + if (second_level_iter_.iter() != nullptr) { + second_level_iter_.SeekForPrev(target); + } + if (!Valid()) { + if (!first_level_iter_.Valid()) { + first_level_iter_.SeekToLast(); + InitDataBlock(); + if (second_level_iter_.iter() != nullptr) { + second_level_iter_.SeekForPrev(target); + } + } + SkipEmptyDataBlocksBackward(); + } +} + void TwoLevelIterator::SeekToFirst() { first_level_iter_.SeekToFirst(); InitDataBlock(); @@ -125,13 +178,13 @@ void TwoLevelIterator::Prev() { SkipEmptyDataBlocksBackward(); } - void TwoLevelIterator::SkipEmptyDataBlocksForward() { while (second_level_iter_.iter() == nullptr || (!second_level_iter_.Valid() && - !second_level_iter_.status().IsIncomplete())) { + !second_level_iter_.status().IsIncomplete())) { // Move to next block - if (!first_level_iter_.Valid()) { + if (!first_level_iter_.Valid() || + state_->KeyReachedUpperBound(first_level_iter_.key())) { SetSecondLevelIterator(nullptr); return; } @@ -146,7 +199,7 @@ void TwoLevelIterator::SkipEmptyDataBlocksForward() { void TwoLevelIterator::SkipEmptyDataBlocksBackward() { while (second_level_iter_.iter() == nullptr || (!second_level_iter_.Valid() && - !second_level_iter_.status().IsIncomplete())) { + !second_level_iter_.status().IsIncomplete())) { // Move to next block if (!first_level_iter_.Valid()) { SetSecondLevelIterator(nullptr); @@ -160,11 +213,21 @@ void TwoLevelIterator::SkipEmptyDataBlocksBackward() { } } -void TwoLevelIterator::SetSecondLevelIterator(Iterator* iter) { +void TwoLevelIterator::SetSecondLevelIterator(InternalIterator* iter) { if (second_level_iter_.iter() != nullptr) { SaveError(second_level_iter_.status()); } - second_level_iter_.Set(iter); + + if (pinned_iters_mgr_ && iter) { + iter->SetPinnedItersMgr(pinned_iters_mgr_); + } + + InternalIterator* old_iter = second_level_iter_.Set(iter); + if (pinned_iters_mgr_ && pinned_iters_mgr_->PinningEnabled()) { + pinned_iters_mgr_->PinIterator(old_iter); + } else { + delete old_iter; + } } void TwoLevelIterator::InitDataBlock() { @@ -178,7 +241,7 @@ void TwoLevelIterator::InitDataBlock() { // second_level_iter is already constructed with this iterator, so // no need to change anything } else { - Iterator* iter = state_->NewSecondaryIterator(handle); + InternalIterator* iter = state_->NewSecondaryIterator(handle); data_block_handle_.assign(handle.data(), handle.size()); SetSecondLevelIterator(iter); } @@ -187,13 +250,17 @@ void TwoLevelIterator::InitDataBlock() { } // namespace -Iterator* NewTwoLevelIterator(TwoLevelIteratorState* state, - Iterator* first_level_iter, Arena* arena) { +InternalIterator* NewTwoLevelIterator(TwoLevelIteratorState* state, + InternalIterator* first_level_iter, + Arena* arena, + bool need_free_iter_and_state) { if (arena == nullptr) { - return new TwoLevelIterator(state, first_level_iter); + return new TwoLevelIterator(state, first_level_iter, + need_free_iter_and_state); } else { auto mem = arena->AllocateAligned(sizeof(TwoLevelIterator)); - return new (mem) TwoLevelIterator(state, first_level_iter); + return new (mem) + TwoLevelIterator(state, first_level_iter, need_free_iter_and_state); } } diff --git a/table/two_level_iterator.h b/table/two_level_iterator.h index d955dd76318..34b33c83f65 100644 --- a/table/two_level_iterator.h +++ b/table/two_level_iterator.h @@ -1,7 +1,7 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). // // Copyright (c) 2011 The LevelDB Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be @@ -19,12 +19,13 @@ class InternalKeyComparator; class Arena; struct TwoLevelIteratorState { - explicit TwoLevelIteratorState(bool check_prefix_may_match) - : check_prefix_may_match(check_prefix_may_match) {} + explicit TwoLevelIteratorState(bool _check_prefix_may_match) + : check_prefix_may_match(_check_prefix_may_match) {} virtual ~TwoLevelIteratorState() {} - virtual Iterator* NewSecondaryIterator(const Slice& handle) = 0; + virtual InternalIterator* NewSecondaryIterator(const Slice& handle) = 0; virtual bool PrefixMayMatch(const Slice& internal_key) = 0; + virtual bool KeyReachedUpperBound(const Slice& internal_key) = 0; // If call PrefixMayMatch() bool check_prefix_may_match; @@ -43,8 +44,10 @@ struct TwoLevelIteratorState { // arena: If not null, the arena is used to allocate the Iterator. // When destroying the iterator, the destructor will destroy // all the states but those allocated in arena. -extern Iterator* NewTwoLevelIterator(TwoLevelIteratorState* state, - Iterator* first_level_iter, - Arena* arena = nullptr); +// need_free_iter_and_state: free `state` and `first_level_iter` if +// true. Otherwise, just call destructor. +extern InternalIterator* NewTwoLevelIterator( + TwoLevelIteratorState* state, InternalIterator* first_level_iter, + Arena* arena = nullptr, bool need_free_iter_and_state = true); } // namespace rocksdb diff --git a/third-party/fbson/COMMIT.md b/third-party/fbson/COMMIT.md new file mode 100644 index 00000000000..b38b5424d3b --- /dev/null +++ b/third-party/fbson/COMMIT.md @@ -0,0 +1,5 @@ +fbson commit: +https://github.com/facebook/mysql-5.6/commit/55ef9ff25c934659a70b4094e9b406c48e9dd43d + +# TODO. +* Had to convert zero sized array to [1] sized arrays due to the fact that MS Compiler complains about it not being standard. At some point need to contribute this change back to MySql where this code was taken from. diff --git a/third-party/fbson/FbsonDocument.h b/third-party/fbson/FbsonDocument.h new file mode 100644 index 00000000000..6fb8a93f171 --- /dev/null +++ b/third-party/fbson/FbsonDocument.h @@ -0,0 +1,893 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +/* + * This header defines FbsonDocument, FbsonKeyValue, and various value classes + * which are derived from FbsonValue, and a forward iterator for container + * values - essentially everything that is related to FBSON binary data + * structures. + * + * Implementation notes: + * + * None of the classes in this header file can be instantiated directly (i.e. + * you cannot create a FbsonKeyValue or FbsonValue object - all constructors + * are declared non-public). We use the classes as wrappers on the packed FBSON + * bytes (serialized), and cast the classes (types) to the underlying packed + * byte array. + * + * For the same reason, we cannot define any FBSON value class to be virtual, + * since we never call constructors, and will not instantiate vtbl and vptrs. + * + * Therefore, the classes are defined as packed structures (i.e. no data + * alignment and padding), and the private member variables of the classes are + * defined precisely in the same order as the FBSON spec. This ensures we + * access the packed FBSON bytes correctly. + * + * The packed structures are highly optimized for in-place operations with low + * overhead. The reads (and in-place writes) are performed directly on packed + * bytes. There is no memory allocation at all at runtime. + * + * For updates/writes of values that will expand the original FBSON size, the + * write will fail, and the caller needs to handle buffer increase. + * + * ** Iterator ** + * Both ObjectVal class and ArrayVal class have iterator type that you can use + * to declare an iterator on a container object to go through the key-value + * pairs or value list. The iterator has both non-const and const types. + * + * Note: iterators are forward direction only. + * + * ** Query ** + * Querying into containers is through the member functions find (for key/value + * pairs) and get (for array elements), and is in streaming style. We don't + * need to read/scan the whole FBSON packed bytes in order to return results. + * Once the key/index is found, we will stop search. You can use text to query + * both objects and array (for array, text will be converted to integer index), + * and use index to retrieve from array. Array index is 0-based. + * + * ** External dictionary ** + * During query processing, you can also pass a call-back function, so the + * search will first try to check if the key string exists in the dictionary. + * If so, search will be based on the id instead of the key string. + * + * @author Tian Xia + */ + +#ifndef FBSON_FBSONDOCUMENT_H +#define FBSON_FBSONDOCUMENT_H + +#include +#include +#include + +namespace fbson { + +#pragma pack(push, 1) + +#define FBSON_VER 1 + +// forward declaration +class FbsonValue; +class ObjectVal; + +/* + * FbsonDocument is the main object that accesses and queries FBSON packed + * bytes. NOTE: FbsonDocument only allows object container as the top level + * FBSON value. However, you can use the static method "createValue" to get any + * FbsonValue object from the packed bytes. + * + * FbsonDocument object also dereferences to an object container value + * (ObjectVal) once FBSON is loaded. + * + * ** Load ** + * FbsonDocument is usable after loading packed bytes (memory location) into + * the object. We only need the header and first few bytes of the payload after + * header to verify the FBSON. + * + * Note: creating an FbsonDocument (through createDocument) does not allocate + * any memory. The document object is an efficient wrapper on the packed bytes + * which is accessed directly. + * + * ** Query ** + * Query is through dereferencing into ObjectVal. + */ +class FbsonDocument { + public: + // create an FbsonDocument object from FBSON packed bytes + static FbsonDocument* createDocument(const char* pb, uint32_t size); + + // create an FbsonValue from FBSON packed bytes + static FbsonValue* createValue(const char* pb, uint32_t size); + + uint8_t version() { return header_.ver_; } + + FbsonValue* getValue() { return ((FbsonValue*)payload_); } + + ObjectVal* operator->() { return ((ObjectVal*)payload_); } + + const ObjectVal* operator->() const { return ((const ObjectVal*)payload_); } + + private: + /* + * FbsonHeader class defines FBSON header (internal to FbsonDocument). + * + * Currently it only contains version information (1-byte). We may expand the + * header to include checksum of the FBSON binary for more security. + */ + struct FbsonHeader { + uint8_t ver_; + } header_; + + char payload_[1]; + + FbsonDocument(); + + FbsonDocument(const FbsonDocument&) = delete; + FbsonDocument& operator=(const FbsonDocument&) = delete; +}; + +/* + * FbsonFwdIteratorT implements FBSON's iterator template. + * + * Note: it is an FORWARD iterator only due to the design of FBSON format. + */ +template +class FbsonFwdIteratorT { + typedef Iter_Type iterator; + typedef typename std::iterator_traits::pointer pointer; + typedef typename std::iterator_traits::reference reference; + + public: + explicit FbsonFwdIteratorT(const iterator& i) : current_(i) {} + + // allow non-const to const iterator conversion (same container type) + template + FbsonFwdIteratorT(const FbsonFwdIteratorT& rhs) + : current_(rhs.base()) {} + + bool operator==(const FbsonFwdIteratorT& rhs) const { + return (current_ == rhs.current_); + } + + bool operator!=(const FbsonFwdIteratorT& rhs) const { + return !operator==(rhs); + } + + bool operator<(const FbsonFwdIteratorT& rhs) const { + return (current_ < rhs.current_); + } + + bool operator>(const FbsonFwdIteratorT& rhs) const { return !operator<(rhs); } + + FbsonFwdIteratorT& operator++() { + current_ = (iterator)(((char*)current_) + current_->numPackedBytes()); + return *this; + } + + FbsonFwdIteratorT operator++(int) { + auto tmp = *this; + current_ = (iterator)(((char*)current_) + current_->numPackedBytes()); + return tmp; + } + + explicit operator pointer() { return current_; } + + reference operator*() const { return *current_; } + + pointer operator->() const { return current_; } + + iterator base() const { return current_; } + + private: + iterator current_; +}; + +typedef int (*hDictInsert)(const char* key, unsigned len); +typedef int (*hDictFind)(const char* key, unsigned len); + +/* + * FbsonType defines 10 primitive types and 2 container types, as described + * below. + * + * primitive_value ::= + * 0x00 //null value (0 byte) + * | 0x01 //boolean true (0 byte) + * | 0x02 //boolean false (0 byte) + * | 0x03 int8 //char/int8 (1 byte) + * | 0x04 int16 //int16 (2 bytes) + * | 0x05 int32 //int32 (4 bytes) + * | 0x06 int64 //int64 (8 bytes) + * | 0x07 double //floating point (8 bytes) + * | 0x08 string //variable length string + * | 0x09 binary //variable length binary + * + * container ::= + * 0x0A int32 key_value_list //object, int32 is the total bytes of the object + * | 0x0B int32 value_list //array, int32 is the total bytes of the array + */ +enum class FbsonType : char { + T_Null = 0x00, + T_True = 0x01, + T_False = 0x02, + T_Int8 = 0x03, + T_Int16 = 0x04, + T_Int32 = 0x05, + T_Int64 = 0x06, + T_Double = 0x07, + T_String = 0x08, + T_Binary = 0x09, + T_Object = 0x0A, + T_Array = 0x0B, + NUM_TYPES, +}; + +typedef std::underlying_type::type FbsonTypeUnder; + +/* + * FbsonKeyValue class defines FBSON key type, as described below. + * + * key ::= + * 0x00 int8 //1-byte dictionary id + * | int8 (byte*) //int8 (>0) is the size of the key string + * + * value ::= primitive_value | container + * + * FbsonKeyValue can be either an id mapping to the key string in an external + * dictionary, or it is the original key string. Whether to read an id or a + * string is decided by the first byte (size_). + * + * Note: a key object must be followed by a value object. Therefore, a key + * object implicitly refers to a key-value pair, and you can get the value + * object right after the key object. The function numPackedBytes hence + * indicates the total size of the key-value pair, so that we will be able go + * to next pair from the key. + * + * ** Dictionary size ** + * By default, the dictionary size is 255 (1-byte). Users can define + * "USE_LARGE_DICT" to increase the dictionary size to 655535 (2-byte). + */ +class FbsonKeyValue { + public: +#ifdef USE_LARGE_DICT + static const int sMaxKeyId = 65535; + typedef uint16_t keyid_type; +#else + static const int sMaxKeyId = 255; + typedef uint8_t keyid_type; +#endif // #ifdef USE_LARGE_DICT + + static const uint8_t sMaxKeyLen = 64; + + // size of the key. 0 indicates it is stored as id + uint8_t klen() const { return size_; } + + // get the key string. Note the string may not be null terminated. + const char* getKeyStr() const { return key_.str_; } + + keyid_type getKeyId() const { return key_.id_; } + + unsigned int keyPackedBytes() const { + return size_ ? (sizeof(size_) + size_) + : (sizeof(size_) + sizeof(keyid_type)); + } + + FbsonValue* value() const { + return (FbsonValue*)(((char*)this) + keyPackedBytes()); + } + + // size of the total packed bytes (key+value) + unsigned int numPackedBytes() const; + + private: + uint8_t size_; + + union key_ { + keyid_type id_; + char str_[1]; + } key_; + + FbsonKeyValue(); +}; + +/* + * FbsonValue is the base class of all FBSON types. It contains only one member + * variable - type info, which can be retrieved by member functions is[Type]() + * or type(). + */ +class FbsonValue { + public: + static const uint32_t sMaxValueLen = 1 << 24; // 16M + + bool isNull() const { return (type_ == FbsonType::T_Null); } + bool isTrue() const { return (type_ == FbsonType::T_True); } + bool isFalse() const { return (type_ == FbsonType::T_False); } + bool isInt8() const { return (type_ == FbsonType::T_Int8); } + bool isInt16() const { return (type_ == FbsonType::T_Int16); } + bool isInt32() const { return (type_ == FbsonType::T_Int32); } + bool isInt64() const { return (type_ == FbsonType::T_Int64); } + bool isDouble() const { return (type_ == FbsonType::T_Double); } + bool isString() const { return (type_ == FbsonType::T_String); } + bool isBinary() const { return (type_ == FbsonType::T_Binary); } + bool isObject() const { return (type_ == FbsonType::T_Object); } + bool isArray() const { return (type_ == FbsonType::T_Array); } + + FbsonType type() const { return type_; } + + // size of the total packed bytes + unsigned int numPackedBytes() const; + + // size of the value in bytes + unsigned int size() const; + + // get the raw byte array of the value + const char* getValuePtr() const; + + // find the FBSON value by a key path string (null terminated) + FbsonValue* findPath(const char* key_path, + const char* delim = ".", + hDictFind handler = nullptr) { + return findPath(key_path, (unsigned int)strlen(key_path), delim, handler); + } + + // find the FBSON value by a key path string (with length) + FbsonValue* findPath(const char* key_path, + unsigned int len, + const char* delim, + hDictFind handler); + + protected: + FbsonType type_; // type info + + FbsonValue(); +}; + +/* + * NumerValT is the template class (derived from FbsonValue) of all number + * types (integers and double). + */ +template +class NumberValT : public FbsonValue { + public: + T val() const { return num_; } + + unsigned int numPackedBytes() const { return sizeof(FbsonValue) + sizeof(T); } + + // catch all unknow specialization of the template class + bool setVal(T value) { return false; } + + private: + T num_; + + NumberValT(); +}; + +typedef NumberValT Int8Val; + +// override setVal for Int8Val +template <> +inline bool Int8Val::setVal(int8_t value) { + if (!isInt8()) { + return false; + } + + num_ = value; + return true; +} + +typedef NumberValT Int16Val; + +// override setVal for Int16Val +template <> +inline bool Int16Val::setVal(int16_t value) { + if (!isInt16()) { + return false; + } + + num_ = value; + return true; +} + +typedef NumberValT Int32Val; + +// override setVal for Int32Val +template <> +inline bool Int32Val::setVal(int32_t value) { + if (!isInt32()) { + return false; + } + + num_ = value; + return true; +} + +typedef NumberValT Int64Val; + +// override setVal for Int64Val +template <> +inline bool Int64Val::setVal(int64_t value) { + if (!isInt64()) { + return false; + } + + num_ = value; + return true; +} + +typedef NumberValT DoubleVal; + +// override setVal for DoubleVal +template <> +inline bool DoubleVal::setVal(double value) { + if (!isDouble()) { + return false; + } + + num_ = value; + return true; +} + +/* + * BlobVal is the base class (derived from FbsonValue) for string and binary + * types. The size_ indicates the total bytes of the payload_. + */ +class BlobVal : public FbsonValue { + public: + // size of the blob payload only + unsigned int getBlobLen() const { return size_; } + + // return the blob as byte array + const char* getBlob() const { return payload_; } + + // size of the total packed bytes + unsigned int numPackedBytes() const { + return sizeof(FbsonValue) + sizeof(size_) + size_; + } + + protected: + uint32_t size_; + char payload_[1]; + + // set new blob bytes + bool internalSetVal(const char* blob, uint32_t blobSize) { + // if we cannot fit the new blob, fail the operation + if (blobSize > size_) { + return false; + } + + memcpy(payload_, blob, blobSize); + + // Set the reset of the bytes to 0. Note we cannot change the size_ of the + // current payload, as all values are packed. + memset(payload_ + blobSize, 0, size_ - blobSize); + + return true; + } + + BlobVal(); + + private: + // Disable as this class can only be allocated dynamically + BlobVal(const BlobVal&) = delete; + BlobVal& operator=(const BlobVal&) = delete; +}; + +/* + * Binary type + */ +class BinaryVal : public BlobVal { + public: + bool setVal(const char* blob, uint32_t blobSize) { + if (!isBinary()) { + return false; + } + + return internalSetVal(blob, blobSize); + } + + private: + BinaryVal(); +}; + +/* + * String type + * Note: FBSON string may not be a c-string (NULL-terminated) + */ +class StringVal : public BlobVal { + public: + bool setVal(const char* str, uint32_t blobSize) { + if (!isString()) { + return false; + } + + return internalSetVal(str, blobSize); + } + + private: + StringVal(); +}; + +/* + * ContainerVal is the base class (derived from FbsonValue) for object and + * array types. The size_ indicates the total bytes of the payload_. + */ +class ContainerVal : public FbsonValue { + public: + // size of the container payload only + unsigned int getContainerSize() const { return size_; } + + // return the container payload as byte array + const char* getPayload() const { return payload_; } + + // size of the total packed bytes + unsigned int numPackedBytes() const { + return sizeof(FbsonValue) + sizeof(size_) + size_; + } + + protected: + uint32_t size_; + char payload_[1]; + + ContainerVal(); + + ContainerVal(const ContainerVal&) = delete; + ContainerVal& operator=(const ContainerVal&) = delete; +}; + +/* + * Object type + */ +class ObjectVal : public ContainerVal { + public: + // find the FBSON value by a key string (null terminated) + FbsonValue* find(const char* key, hDictFind handler = nullptr) const { + if (!key) + return nullptr; + + return find(key, (unsigned int)strlen(key), handler); + } + + // find the FBSON value by a key string (with length) + FbsonValue* find(const char* key, + unsigned int klen, + hDictFind handler = nullptr) const { + if (!key || !klen) + return nullptr; + + int key_id = -1; + if (handler && (key_id = handler(key, klen)) >= 0) { + return find(key_id); + } + + return internalFind(key, klen); + } + + // find the FBSON value by a key dictionary ID + FbsonValue* find(int key_id) const { + if (key_id < 0 || key_id > FbsonKeyValue::sMaxKeyId) + return nullptr; + + const char* pch = payload_; + const char* fence = payload_ + size_; + + while (pch < fence) { + FbsonKeyValue* pkey = (FbsonKeyValue*)(pch); + if (!pkey->klen() && key_id == pkey->getKeyId()) { + return pkey->value(); + } + pch += pkey->numPackedBytes(); + } + + assert(pch == fence); + + return nullptr; + } + + typedef FbsonKeyValue value_type; + typedef value_type* pointer; + typedef const value_type* const_pointer; + typedef FbsonFwdIteratorT iterator; + typedef FbsonFwdIteratorT const_iterator; + + iterator begin() { return iterator((pointer)payload_); } + + const_iterator begin() const { return const_iterator((pointer)payload_); } + + iterator end() { return iterator((pointer)(payload_ + size_)); } + + const_iterator end() const { + return const_iterator((pointer)(payload_ + size_)); + } + + private: + FbsonValue* internalFind(const char* key, unsigned int klen) const { + const char* pch = payload_; + const char* fence = payload_ + size_; + + while (pch < fence) { + FbsonKeyValue* pkey = (FbsonKeyValue*)(pch); + if (klen == pkey->klen() && strncmp(key, pkey->getKeyStr(), klen) == 0) { + return pkey->value(); + } + pch += pkey->numPackedBytes(); + } + + assert(pch == fence); + + return nullptr; + } + + private: + ObjectVal(); +}; + +/* + * Array type + */ +class ArrayVal : public ContainerVal { + public: + // get the FBSON value at index + FbsonValue* get(int idx) const { + if (idx < 0) + return nullptr; + + const char* pch = payload_; + const char* fence = payload_ + size_; + + while (pch < fence && idx-- > 0) + pch += ((FbsonValue*)pch)->numPackedBytes(); + + if (idx == -1) + return (FbsonValue*)pch; + else { + assert(pch == fence); + return nullptr; + } + } + + // Get number of elements in array + unsigned int numElem() const { + const char* pch = payload_; + const char* fence = payload_ + size_; + + unsigned int num = 0; + while (pch < fence) { + ++num; + pch += ((FbsonValue*)pch)->numPackedBytes(); + } + + assert(pch == fence); + + return num; + } + + typedef FbsonValue value_type; + typedef value_type* pointer; + typedef const value_type* const_pointer; + typedef FbsonFwdIteratorT iterator; + typedef FbsonFwdIteratorT const_iterator; + + iterator begin() { return iterator((pointer)payload_); } + + const_iterator begin() const { return const_iterator((pointer)payload_); } + + iterator end() { return iterator((pointer)(payload_ + size_)); } + + const_iterator end() const { + return const_iterator((pointer)(payload_ + size_)); + } + + private: + ArrayVal(); +}; + +inline FbsonDocument* FbsonDocument::createDocument(const char* pb, + uint32_t size) { + if (!pb || size < sizeof(FbsonHeader) + sizeof(FbsonValue)) { + return nullptr; + } + + FbsonDocument* doc = (FbsonDocument*)pb; + if (doc->header_.ver_ != FBSON_VER) { + return nullptr; + } + + FbsonValue* val = (FbsonValue*)doc->payload_; + if (!val->isObject() || size != sizeof(FbsonHeader) + val->numPackedBytes()) { + return nullptr; + } + + return doc; +} + +inline FbsonValue* FbsonDocument::createValue(const char* pb, uint32_t size) { + if (!pb || size < sizeof(FbsonHeader) + sizeof(FbsonValue)) { + return nullptr; + } + + FbsonDocument* doc = (FbsonDocument*)pb; + if (doc->header_.ver_ != FBSON_VER) { + return nullptr; + } + + FbsonValue* val = (FbsonValue*)doc->payload_; + if (size != sizeof(FbsonHeader) + val->numPackedBytes()) { + return nullptr; + } + + return val; +} + +inline unsigned int FbsonKeyValue::numPackedBytes() const { + unsigned int ks = keyPackedBytes(); + FbsonValue* val = (FbsonValue*)(((char*)this) + ks); + return ks + val->numPackedBytes(); +} + +// Poor man's "virtual" function FbsonValue::numPackedBytes +inline unsigned int FbsonValue::numPackedBytes() const { + switch (type_) { + case FbsonType::T_Null: + case FbsonType::T_True: + case FbsonType::T_False: { + return sizeof(type_); + } + + case FbsonType::T_Int8: { + return sizeof(type_) + sizeof(int8_t); + } + case FbsonType::T_Int16: { + return sizeof(type_) + sizeof(int16_t); + } + case FbsonType::T_Int32: { + return sizeof(type_) + sizeof(int32_t); + } + case FbsonType::T_Int64: { + return sizeof(type_) + sizeof(int64_t); + } + case FbsonType::T_Double: { + return sizeof(type_) + sizeof(double); + } + case FbsonType::T_String: + case FbsonType::T_Binary: { + return ((BlobVal*)(this))->numPackedBytes(); + } + + case FbsonType::T_Object: + case FbsonType::T_Array: { + return ((ContainerVal*)(this))->numPackedBytes(); + } + default: + return 0; + } +} + +inline unsigned int FbsonValue::size() const { + switch (type_) { + case FbsonType::T_Int8: { + return sizeof(int8_t); + } + case FbsonType::T_Int16: { + return sizeof(int16_t); + } + case FbsonType::T_Int32: { + return sizeof(int32_t); + } + case FbsonType::T_Int64: { + return sizeof(int64_t); + } + case FbsonType::T_Double: { + return sizeof(double); + } + case FbsonType::T_String: + case FbsonType::T_Binary: { + return ((BlobVal*)(this))->getBlobLen(); + } + + case FbsonType::T_Object: + case FbsonType::T_Array: { + return ((ContainerVal*)(this))->getContainerSize(); + } + case FbsonType::T_Null: + case FbsonType::T_True: + case FbsonType::T_False: + default: + return 0; + } +} + +inline const char* FbsonValue::getValuePtr() const { + switch (type_) { + case FbsonType::T_Int8: + case FbsonType::T_Int16: + case FbsonType::T_Int32: + case FbsonType::T_Int64: + case FbsonType::T_Double: + return ((char*)this) + sizeof(FbsonType); + + case FbsonType::T_String: + case FbsonType::T_Binary: + return ((BlobVal*)(this))->getBlob(); + + case FbsonType::T_Object: + case FbsonType::T_Array: + return ((ContainerVal*)(this))->getPayload(); + + case FbsonType::T_Null: + case FbsonType::T_True: + case FbsonType::T_False: + default: + return nullptr; + } +} + +inline FbsonValue* FbsonValue::findPath(const char* key_path, + unsigned int kp_len, + const char* delim = ".", + hDictFind handler = nullptr) { + if (!key_path || !kp_len) + return nullptr; + + if (!delim) + delim = "."; // default delimiter + + FbsonValue* pval = this; + const char* fence = key_path + kp_len; + char idx_buf[21]; // buffer to parse array index (integer value) + + while (pval && key_path < fence) { + const char* key = key_path; + unsigned int klen = 0; + // find the current key + for (; key_path != fence && *key_path != *delim; ++key_path, ++klen) + ; + + if (!klen) + return nullptr; + + switch (pval->type_) { + case FbsonType::T_Object: { + pval = ((ObjectVal*)pval)->find(key, klen, handler); + break; + } + + case FbsonType::T_Array: { + // parse string into an integer (array index) + if (klen >= sizeof(idx_buf)) + return nullptr; + + memcpy(idx_buf, key, klen); + idx_buf[klen] = 0; + + char* end = nullptr; + int index = (int)strtol(idx_buf, &end, 10); + if (end && !*end) + pval = ((fbson::ArrayVal*)pval)->get(index); + else + // incorrect index string + return nullptr; + break; + } + + default: + return nullptr; + } + + // skip the delimiter + if (key_path < fence) { + ++key_path; + if (key_path == fence) + // we have a trailing delimiter at the end + return nullptr; + } + } + + return pval; +} + +#pragma pack(pop) + +} // namespace fbson + +#endif // FBSON_FBSONDOCUMENT_H diff --git a/third-party/fbson/FbsonJsonParser.h b/third-party/fbson/FbsonJsonParser.h new file mode 100644 index 00000000000..63b03e2b908 --- /dev/null +++ b/third-party/fbson/FbsonJsonParser.h @@ -0,0 +1,741 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +/* + * This file defines FbsonJsonParserT (template) and FbsonJsonParser. + * + * FbsonJsonParserT is a template class which implements a JSON parser. + * FbsonJsonParserT parses JSON text, and serialize it to FBSON binary format + * by using FbsonWriterT object. By default, FbsonJsonParserT creates a new + * FbsonWriterT object with an output stream object. However, you can also + * pass in your FbsonWriterT or any stream object that implements some basic + * interface of std::ostream (see FbsonStream.h). + * + * FbsonJsonParser specializes FbsonJsonParserT with FbsonOutStream type (see + * FbsonStream.h). So unless you want to provide own a different output stream + * type, use FbsonJsonParser object. + * + * ** Parsing JSON ** + * FbsonJsonParserT parses JSON string, and directly serializes into FBSON + * packed bytes. There are three ways to parse a JSON string: (1) using + * c-string, (2) using string with len, (3) using std::istream object. You can + * use custome streambuf to redirect output. FbsonOutBuffer is a streambuf used + * internally if the input is raw character buffer. + * + * You can reuse an FbsonJsonParserT object to parse/serialize multiple JSON + * strings, and the previous FBSON will be overwritten. + * + * If parsing fails (returned false), the error code will be set to one of + * FbsonErrType, and can be retrieved by calling getErrorCode(). + * + * ** External dictionary ** + * During parsing a JSON string, you can pass a call-back function to map a key + * string to an id, and store the dictionary id in FBSON to save space. The + * purpose of using an external dictionary is more towards a collection of + * documents (which has common keys) rather than a single document, so that + * space saving will be significant. + * + * ** Endianness ** + * Note: FBSON serialization doesn't assume endianness of the server. However + * you will need to ensure that the endianness at the reader side is the same + * as that at the writer side (if they are on different machines). Otherwise, + * proper conversion is needed when a number value is returned to the + * caller/writer. + * + * @author Tian Xia + */ + +#ifndef FBSON_FBSONPARSER_H +#define FBSON_FBSONPARSER_H + +#include +#include +#include "FbsonDocument.h" +#include "FbsonWriter.h" + +namespace fbson { + +const char* const kJsonDelim = " ,]}\t\r\n"; +const char* const kWhiteSpace = " \t\n\r"; + +/* + * Error codes + */ +enum class FbsonErrType { + E_NONE = 0, + E_INVALID_VER, + E_EMPTY_STR, + E_OUTPUT_FAIL, + E_INVALID_DOCU, + E_INVALID_VALUE, + E_INVALID_KEY, + E_INVALID_STR, + E_INVALID_OBJ, + E_INVALID_ARR, + E_INVALID_HEX, + E_INVALID_OCTAL, + E_INVALID_DECIMAL, + E_INVALID_EXPONENT, + E_HEX_OVERFLOW, + E_OCTAL_OVERFLOW, + E_DECIMAL_OVERFLOW, + E_DOUBLE_OVERFLOW, + E_EXPONENT_OVERFLOW, +}; + +/* + * Template FbsonJsonParserT + */ +template +class FbsonJsonParserT { + public: + FbsonJsonParserT() : err_(FbsonErrType::E_NONE) {} + + explicit FbsonJsonParserT(OS_TYPE& os) + : writer_(os), err_(FbsonErrType::E_NONE) {} + + // parse a UTF-8 JSON string + bool parse(const std::string& str, hDictInsert handler = nullptr) { + return parse(str.c_str(), (unsigned int)str.size(), handler); + } + + // parse a UTF-8 JSON c-style string (NULL terminated) + bool parse(const char* c_str, hDictInsert handler = nullptr) { + return parse(c_str, (unsigned int)strlen(c_str), handler); + } + + // parse a UTF-8 JSON string with length + bool parse(const char* pch, unsigned int len, hDictInsert handler = nullptr) { + if (!pch || len == 0) { + err_ = FbsonErrType::E_EMPTY_STR; + return false; + } + + FbsonInBuffer sb(pch, len); + std::istream in(&sb); + return parse(in, handler); + } + + // parse UTF-8 JSON text from an input stream + bool parse(std::istream& in, hDictInsert handler = nullptr) { + bool res = false; + + // reset output stream + writer_.reset(); + + trim(in); + + if (in.peek() == '{') { + in.ignore(); + res = parseObject(in, handler); + } else if (in.peek() == '[') { + in.ignore(); + res = parseArray(in, handler); + } else { + err_ = FbsonErrType::E_INVALID_DOCU; + } + + trim(in); + if (res && !in.eof()) { + err_ = FbsonErrType::E_INVALID_DOCU; + return false; + } + + return res; + } + + FbsonWriterT& getWriter() { return writer_; } + + FbsonErrType getErrorCode() { return err_; } + + // clear error code + void clearErr() { err_ = FbsonErrType::E_NONE; } + + private: + // parse a JSON object (comma-separated list of key-value pairs) + bool parseObject(std::istream& in, hDictInsert handler) { + if (!writer_.writeStartObject()) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + + trim(in); + + if (in.peek() == '}') { + in.ignore(); + // empty object + if (!writer_.writeEndObject()) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + return true; + } + + while (in.good()) { + if (in.get() != '"') { + err_ = FbsonErrType::E_INVALID_KEY; + return false; + } + + if (!parseKVPair(in, handler)) { + return false; + } + + trim(in); + + char ch = in.get(); + if (ch == '}') { + // end of the object + if (!writer_.writeEndObject()) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + return true; + } else if (ch != ',') { + err_ = FbsonErrType::E_INVALID_OBJ; + return false; + } + + trim(in); + } + + err_ = FbsonErrType::E_INVALID_OBJ; + return false; + } + + // parse a JSON array (comma-separated list of values) + bool parseArray(std::istream& in, hDictInsert handler) { + if (!writer_.writeStartArray()) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + + trim(in); + + if (in.peek() == ']') { + in.ignore(); + // empty array + if (!writer_.writeEndArray()) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + return true; + } + + while (in.good()) { + if (!parseValue(in, handler)) { + return false; + } + + trim(in); + + char ch = in.get(); + if (ch == ']') { + // end of the array + if (!writer_.writeEndArray()) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + return true; + } else if (ch != ',') { + err_ = FbsonErrType::E_INVALID_ARR; + return false; + } + + trim(in); + } + + err_ = FbsonErrType::E_INVALID_ARR; + return false; + } + + // parse a key-value pair, separated by ":" + bool parseKVPair(std::istream& in, hDictInsert handler) { + if (parseKey(in, handler) && parseValue(in, handler)) { + return true; + } + + return false; + } + + // parse a key (must be string) + bool parseKey(std::istream& in, hDictInsert handler) { + char key[FbsonKeyValue::sMaxKeyLen]; + int i = 0; + while (in.good() && in.peek() != '"' && i < FbsonKeyValue::sMaxKeyLen) { + key[i++] = in.get(); + } + + if (!in.good() || in.peek() != '"' || i == 0) { + err_ = FbsonErrType::E_INVALID_KEY; + return false; + } + + in.ignore(); // discard '"' + + int key_id = -1; + if (handler) { + key_id = handler(key, i); + } + + if (key_id < 0) { + writer_.writeKey(key, i); + } else { + writer_.writeKey(key_id); + } + + trim(in); + + if (in.get() != ':') { + err_ = FbsonErrType::E_INVALID_OBJ; + return false; + } + + return true; + } + + // parse a value + bool parseValue(std::istream& in, hDictInsert handler) { + bool res = false; + + trim(in); + + switch (in.peek()) { + case 'N': + case 'n': { + in.ignore(); + res = parseNull(in); + break; + } + case 'T': + case 't': { + in.ignore(); + res = parseTrue(in); + break; + } + case 'F': + case 'f': { + in.ignore(); + res = parseFalse(in); + break; + } + case '"': { + in.ignore(); + res = parseString(in); + break; + } + case '{': { + in.ignore(); + res = parseObject(in, handler); + break; + } + case '[': { + in.ignore(); + res = parseArray(in, handler); + break; + } + default: { + res = parseNumber(in); + break; + } + } + + return res; + } + + // parse NULL value + bool parseNull(std::istream& in) { + if (tolower(in.get()) == 'u' && tolower(in.get()) == 'l' && + tolower(in.get()) == 'l') { + writer_.writeNull(); + return true; + } + + err_ = FbsonErrType::E_INVALID_VALUE; + return false; + } + + // parse TRUE value + bool parseTrue(std::istream& in) { + if (tolower(in.get()) == 'r' && tolower(in.get()) == 'u' && + tolower(in.get()) == 'e') { + writer_.writeBool(true); + return true; + } + + err_ = FbsonErrType::E_INVALID_VALUE; + return false; + } + + // parse FALSE value + bool parseFalse(std::istream& in) { + if (tolower(in.get()) == 'a' && tolower(in.get()) == 'l' && + tolower(in.get()) == 's' && tolower(in.get()) == 'e') { + writer_.writeBool(false); + return true; + } + + err_ = FbsonErrType::E_INVALID_VALUE; + return false; + } + + // parse a string + bool parseString(std::istream& in) { + if (!writer_.writeStartString()) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + + bool escaped = false; + char buffer[4096]; // write 4KB at a time + int nread = 0; + while (in.good()) { + char ch = in.get(); + if (ch != '"' || escaped) { + buffer[nread++] = ch; + if (nread == 4096) { + // flush buffer + if (!writer_.writeString(buffer, nread)) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + nread = 0; + } + // set/reset escape + if (ch == '\\' || escaped) { + escaped = !escaped; + } + } else { + // write all remaining bytes in the buffer + if (nread > 0) { + if (!writer_.writeString(buffer, nread)) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + } + // end writing string + if (!writer_.writeEndString()) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + return true; + } + } + + err_ = FbsonErrType::E_INVALID_STR; + return false; + } + + // parse a number + // Number format can be hex, octal, or decimal (including float). + // Only decimal can have (+/-) sign prefix. + bool parseNumber(std::istream& in) { + bool ret = false; + switch (in.peek()) { + case '0': { + in.ignore(); + + if (in.peek() == 'x' || in.peek() == 'X') { + in.ignore(); + ret = parseHex(in); + } else if (in.peek() == '.') { + in.ignore(); + ret = parseDouble(in, 0, 0, 1); + } else { + ret = parseOctal(in); + } + + break; + } + case '-': { + in.ignore(); + ret = parseDecimal(in, -1); + break; + } + case '+': + in.ignore(); + // fall through + default: + ret = parseDecimal(in, 1); + break; + } + + return ret; + } + + // parse a number in hex format + bool parseHex(std::istream& in) { + uint64_t val = 0; + int num_digits = 0; + char ch = tolower(in.peek()); + while (in.good() && !strchr(kJsonDelim, ch) && (++num_digits) <= 16) { + if (ch >= '0' && ch <= '9') { + val = (val << 4) + (ch - '0'); + } else if (ch >= 'a' && ch <= 'f') { + val = (val << 4) + (ch - 'a' + 10); + } else { // unrecognized hex digit + err_ = FbsonErrType::E_INVALID_HEX; + return false; + } + + in.ignore(); + ch = tolower(in.peek()); + } + + int size = 0; + if (num_digits <= 2) { + size = writer_.writeInt8((int8_t)val); + } else if (num_digits <= 4) { + size = writer_.writeInt16((int16_t)val); + } else if (num_digits <= 8) { + size = writer_.writeInt32((int32_t)val); + } else if (num_digits <= 16) { + size = writer_.writeInt64(val); + } else { + err_ = FbsonErrType::E_HEX_OVERFLOW; + return false; + } + + if (size == 0) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + + return true; + } + + // parse a number in octal format + bool parseOctal(std::istream& in) { + int64_t val = 0; + char ch = in.peek(); + while (in.good() && !strchr(kJsonDelim, ch)) { + if (ch >= '0' && ch <= '7') { + val = val * 8 + (ch - '0'); + } else { + err_ = FbsonErrType::E_INVALID_OCTAL; + return false; + } + + // check if the number overflows + if (val < 0) { + err_ = FbsonErrType::E_OCTAL_OVERFLOW; + return false; + } + + in.ignore(); + ch = in.peek(); + } + + int size = 0; + if (val <= std::numeric_limits::max()) { + size = writer_.writeInt8((int8_t)val); + } else if (val <= std::numeric_limits::max()) { + size = writer_.writeInt16((int16_t)val); + } else if (val <= std::numeric_limits::max()) { + size = writer_.writeInt32((int32_t)val); + } else { // val <= INT64_MAX + size = writer_.writeInt64(val); + } + + if (size == 0) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + + return true; + } + + // parse a number in decimal (including float) + bool parseDecimal(std::istream& in, int sign) { + int64_t val = 0; + int precision = 0; + + char ch = 0; + while (in.good() && (ch = in.peek()) == '0') + in.ignore(); + + while (in.good() && !strchr(kJsonDelim, ch)) { + if (ch >= '0' && ch <= '9') { + val = val * 10 + (ch - '0'); + ++precision; + } else if (ch == '.') { + // note we don't pop out '.' + return parseDouble(in, static_cast(val), precision, sign); + } else { + err_ = FbsonErrType::E_INVALID_DECIMAL; + return false; + } + + in.ignore(); + + // if the number overflows int64_t, first parse it as double iff we see a + // decimal point later. Otherwise, will treat it as overflow + if (val < 0 && val > std::numeric_limits::min()) { + return parseDouble(in, static_cast(val), precision, sign); + } + + ch = in.peek(); + } + + if (sign < 0) { + val = -val; + } + + int size = 0; + if (val >= std::numeric_limits::min() && + val <= std::numeric_limits::max()) { + size = writer_.writeInt8((int8_t)val); + } else if (val >= std::numeric_limits::min() && + val <= std::numeric_limits::max()) { + size = writer_.writeInt16((int16_t)val); + } else if (val >= std::numeric_limits::min() && + val <= std::numeric_limits::max()) { + size = writer_.writeInt32((int32_t)val); + } else { // val <= INT64_MAX + size = writer_.writeInt64(val); + } + + if (size == 0) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + + return true; + } + + // parse IEEE745 double precision: + // Significand precision length - 15 + // Maximum exponent value - 308 + // + // "If a decimal string with at most 15 significant digits is converted to + // IEEE 754 double precision representation and then converted back to a + // string with the same number of significant digits, then the final string + // should match the original" + bool parseDouble(std::istream& in, double val, int precision, int sign) { + int integ = precision; + int frac = 0; + bool is_frac = false; + + char ch = in.peek(); + if (ch == '.') { + is_frac = true; + in.ignore(); + ch = in.peek(); + } + + int exp = 0; + while (in.good() && !strchr(kJsonDelim, ch)) { + if (ch >= '0' && ch <= '9') { + if (precision < 15) { + val = val * 10 + (ch - '0'); + if (is_frac) { + ++frac; + } else { + ++integ; + } + ++precision; + } else if (!is_frac) { + ++exp; + } + } else if (ch == 'e' || ch == 'E') { + in.ignore(); + int exp2; + if (!parseExponent(in, exp2)) { + return false; + } + + exp += exp2; + // check if exponent overflows + if (exp > 308 || exp < -308) { + err_ = FbsonErrType::E_EXPONENT_OVERFLOW; + return false; + } + + is_frac = true; + break; + } + + in.ignore(); + ch = in.peek(); + } + + if (!is_frac) { + err_ = FbsonErrType::E_DECIMAL_OVERFLOW; + return false; + } + + val *= std::pow(10, exp - frac); + if (std::isnan(val) || std::isinf(val)) { + err_ = FbsonErrType::E_DOUBLE_OVERFLOW; + return false; + } + + if (sign < 0) { + val = -val; + } + + if (writer_.writeDouble(val) == 0) { + err_ = FbsonErrType::E_OUTPUT_FAIL; + return false; + } + + return true; + } + + // parse the exponent part of a double number + bool parseExponent(std::istream& in, int& exp) { + bool neg = false; + + char ch = in.peek(); + if (ch == '+') { + in.ignore(); + ch = in.peek(); + } else if (ch == '-') { + neg = true; + in.ignore(); + ch = in.peek(); + } + + exp = 0; + while (in.good() && !strchr(kJsonDelim, ch)) { + if (ch >= '0' && ch <= '9') { + exp = exp * 10 + (ch - '0'); + } else { + err_ = FbsonErrType::E_INVALID_EXPONENT; + return false; + } + + if (exp > 308) { + err_ = FbsonErrType::E_EXPONENT_OVERFLOW; + return false; + } + + in.ignore(); + ch = in.peek(); + } + + if (neg) { + exp = -exp; + } + + return true; + } + + void trim(std::istream& in) { + while (in.good() && strchr(kWhiteSpace, in.peek())) { + in.ignore(); + } + } + + private: + FbsonWriterT writer_; + FbsonErrType err_; +}; + +typedef FbsonJsonParserT FbsonJsonParser; + +} // namespace fbson + +#endif // FBSON_FBSONPARSER_H diff --git a/third-party/fbson/FbsonStream.h b/third-party/fbson/FbsonStream.h new file mode 100644 index 00000000000..12723ea30e2 --- /dev/null +++ b/third-party/fbson/FbsonStream.h @@ -0,0 +1,182 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +/* + * This header file defines FbsonInBuffer and FbsonOutStream classes. + * + * ** Input Buffer ** + * FbsonInBuffer is a customer input buffer to wrap raw character buffer. Its + * object instances are used to create std::istream objects interally. + * + * ** Output Stream ** + * FbsonOutStream is a custom output stream classes, to contain the FBSON + * serialized binary. The class is conveniently used to specialize templates of + * FbsonParser and FbsonWriter. + * + * @author Tian Xia + */ + +#ifndef FBSON_FBSONSTREAM_H +#define FBSON_FBSONSTREAM_H + +#ifndef __STDC_FORMAT_MACROS +#define __STDC_FORMAT_MACROS +#endif + +#if defined OS_WIN && !defined snprintf +#define snprintf _snprintf +#endif + +#include +#include + +namespace fbson { + +// lengths includes sign +#define MAX_INT_DIGITS 11 +#define MAX_INT64_DIGITS 20 +#define MAX_DOUBLE_DIGITS 23 // 1(sign)+16(significant)+1(decimal)+5(exponent) + +/* + * FBSON's implementation of input buffer + */ +class FbsonInBuffer : public std::streambuf { + public: + FbsonInBuffer(const char* str, uint32_t len) { + // this is read buffer and the str will not be changed + // so we use const_cast (ugly!) to remove constness + char* pch(const_cast(str)); + setg(pch, pch, pch + len); + } +}; + +/* + * FBSON's implementation of output stream. + * + * This is a wrapper of a char buffer. By default, the buffer capacity is 1024 + * bytes. We will double the buffer if realloc is needed for writes. + */ +class FbsonOutStream : public std::ostream { + public: + explicit FbsonOutStream(uint32_t capacity = 1024) + : std::ostream(nullptr), + head_(nullptr), + size_(0), + capacity_(capacity), + alloc_(true) { + if (capacity_ == 0) { + capacity_ = 1024; + } + + head_ = (char*)malloc(capacity_); + } + + FbsonOutStream(char* buffer, uint32_t capacity) + : std::ostream(nullptr), + head_(buffer), + size_(0), + capacity_(capacity), + alloc_(false) { + assert(buffer && capacity_ > 0); + } + + ~FbsonOutStream() { + if (alloc_) { + free(head_); + } + } + + void put(char c) { write(&c, 1); } + + void write(const char* c_str) { write(c_str, (uint32_t)strlen(c_str)); } + + void write(const char* bytes, uint32_t len) { + if (len == 0) + return; + + if (size_ + len > capacity_) { + realloc(len); + } + + memcpy(head_ + size_, bytes, len); + size_ += len; + } + + // write the integer to string + void write(int i) { + // snprintf automatically adds a NULL, so we need one more char + if (size_ + MAX_INT_DIGITS + 1 > capacity_) { + realloc(MAX_INT_DIGITS + 1); + } + + int len = snprintf(head_ + size_, MAX_INT_DIGITS + 1, "%d", i); + assert(len > 0); + size_ += len; + } + + // write the 64bit integer to string + void write(int64_t l) { + // snprintf automatically adds a NULL, so we need one more char + if (size_ + MAX_INT64_DIGITS + 1 > capacity_) { + realloc(MAX_INT64_DIGITS + 1); + } + + int len = snprintf(head_ + size_, MAX_INT64_DIGITS + 1, "%" PRIi64, l); + assert(len > 0); + size_ += len; + } + + // write the double to string + void write(double d) { + // snprintf automatically adds a NULL, so we need one more char + if (size_ + MAX_DOUBLE_DIGITS + 1 > capacity_) { + realloc(MAX_DOUBLE_DIGITS + 1); + } + + int len = snprintf(head_ + size_, MAX_DOUBLE_DIGITS + 1, "%.15g", d); + assert(len > 0); + size_ += len; + } + + pos_type tellp() const { return size_; } + + void seekp(pos_type pos) { size_ = (uint32_t)pos; } + + const char* getBuffer() const { return head_; } + + pos_type getSize() const { return tellp(); } + + private: + void realloc(uint32_t len) { + assert(capacity_ > 0); + + capacity_ *= 2; + while (capacity_ < size_ + len) { + capacity_ *= 2; + } + + if (alloc_) { + char* new_buf = (char*)::realloc(head_, capacity_); + assert(new_buf); + head_ = new_buf; + } else { + char* new_buf = (char*)::malloc(capacity_); + assert(new_buf); + memcpy(new_buf, head_, size_); + head_ = new_buf; + alloc_ = true; + } + } + + private: + char* head_; + uint32_t size_; + uint32_t capacity_; + bool alloc_; +}; + +} // namespace fbson + +#endif // FBSON_FBSONSTREAM_H diff --git a/third-party/fbson/FbsonUtil.h b/third-party/fbson/FbsonUtil.h new file mode 100644 index 00000000000..2b6d6f5c973 --- /dev/null +++ b/third-party/fbson/FbsonUtil.h @@ -0,0 +1,163 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +/* + * This header file defines miscellaneous utility classes. + * + * @author Tian Xia + */ + +#ifndef FBSON_FBSONUTIL_H +#define FBSON_FBSONUTIL_H + +#include +#include "FbsonDocument.h" + +namespace fbson { + +#define OUT_BUF_SIZE 1024 + +/* + * FbsonToJson converts an FbsonValue object to a JSON string. + */ +class FbsonToJson { + public: + FbsonToJson() : os_(buffer_, OUT_BUF_SIZE) {} + + // get json string + const char* json(const FbsonValue* pval) { + os_.clear(); + os_.seekp(0); + + if (pval) { + intern_json(pval); + } + + os_.put(0); + return os_.getBuffer(); + } + + private: + // recursively convert FbsonValue + void intern_json(const FbsonValue* val) { + switch (val->type()) { + case FbsonType::T_Null: { + os_.write("null", 4); + break; + } + case FbsonType::T_True: { + os_.write("true", 4); + break; + } + case FbsonType::T_False: { + os_.write("false", 5); + break; + } + case FbsonType::T_Int8: { + os_.write(((Int8Val*)val)->val()); + break; + } + case FbsonType::T_Int16: { + os_.write(((Int16Val*)val)->val()); + break; + } + case FbsonType::T_Int32: { + os_.write(((Int32Val*)val)->val()); + break; + } + case FbsonType::T_Int64: { + os_.write(((Int64Val*)val)->val()); + break; + } + case FbsonType::T_Double: { + os_.write(((DoubleVal*)val)->val()); + break; + } + case FbsonType::T_String: { + os_.put('"'); + os_.write(((StringVal*)val)->getBlob(), ((StringVal*)val)->getBlobLen()); + os_.put('"'); + break; + } + case FbsonType::T_Binary: { + os_.write("\"", 9); + os_.write(((BinaryVal*)val)->getBlob(), ((BinaryVal*)val)->getBlobLen()); + os_.write("\"", 9); + break; + } + case FbsonType::T_Object: { + object_to_json((ObjectVal*)val); + break; + } + case FbsonType::T_Array: { + array_to_json((ArrayVal*)val); + break; + } + default: + break; + } + } + + // convert object + void object_to_json(const ObjectVal* val) { + os_.put('{'); + + auto iter = val->begin(); + auto iter_fence = val->end(); + + while (iter < iter_fence) { + // write key + if (iter->klen()) { + os_.put('"'); + os_.write(iter->getKeyStr(), iter->klen()); + os_.put('"'); + } else { + os_.write(iter->getKeyId()); + } + os_.put(':'); + + // convert value + intern_json(iter->value()); + + ++iter; + if (iter != iter_fence) { + os_.put(','); + } + } + + assert(iter == iter_fence); + + os_.put('}'); + } + + // convert array to json + void array_to_json(const ArrayVal* val) { + os_.put('['); + + auto iter = val->begin(); + auto iter_fence = val->end(); + + while (iter != iter_fence) { + // convert value + intern_json((const FbsonValue*)iter); + ++iter; + if (iter != iter_fence) { + os_.put(','); + } + } + + assert(iter == iter_fence); + + os_.put(']'); + } + + private: + FbsonOutStream os_; + char buffer_[OUT_BUF_SIZE]; +}; + +} // namespace fbson + +#endif // FBSON_FBSONUTIL_H diff --git a/third-party/fbson/FbsonWriter.h b/third-party/fbson/FbsonWriter.h new file mode 100644 index 00000000000..a254e9bbf83 --- /dev/null +++ b/third-party/fbson/FbsonWriter.h @@ -0,0 +1,430 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +/* + * This file defines FbsonWriterT (template) and FbsonWriter. + * + * FbsonWriterT is a template class which implements an FBSON serializer. + * Users call various write functions of FbsonWriterT object to write values + * directly to FBSON packed bytes. All write functions of value or key return + * the number of bytes written to FBSON, or 0 if there is an error. To write an + * object, an array, or a string, you must call writeStart[..] before writing + * values or key, and call writeEnd[..] after finishing at the end. + * + * By default, an FbsonWriterT object creates an output stream buffer. + * Alternatively, you can also pass any output stream object to a writer, as + * long as the stream object implements some basic functions of std::ostream + * (such as FbsonOutStream, see FbsonStream.h). + * + * FbsonWriter specializes FbsonWriterT with FbsonOutStream type (see + * FbsonStream.h). So unless you want to provide own a different output stream + * type, use FbsonParser object. + * + * @author Tian Xia + */ + +#ifndef FBSON_FBSONWRITER_H +#define FBSON_FBSONWRITER_H + +#include +#include "FbsonDocument.h" +#include "FbsonStream.h" + +namespace fbson { + +template +class FbsonWriterT { + public: + FbsonWriterT() + : alloc_(true), hasHdr_(false), kvState_(WS_Value), str_pos_(0) { + os_ = new OS_TYPE(); + } + + explicit FbsonWriterT(OS_TYPE& os) + : os_(&os), + alloc_(false), + hasHdr_(false), + kvState_(WS_Value), + str_pos_(0) {} + + ~FbsonWriterT() { + if (alloc_) { + delete os_; + } + } + + void reset() { + os_->clear(); + os_->seekp(0); + hasHdr_ = false; + kvState_ = WS_Value; + for (; !stack_.empty(); stack_.pop()) + ; + } + + // write a key string (or key id if an external dict is provided) + uint32_t writeKey(const char* key, + uint8_t len, + hDictInsert handler = nullptr) { + if (len && !stack_.empty() && verifyKeyState()) { + int key_id = -1; + if (handler) { + key_id = handler(key, len); + } + + uint32_t size = sizeof(uint8_t); + if (key_id < 0) { + os_->put(len); + os_->write(key, len); + size += len; + } else if (key_id <= FbsonKeyValue::sMaxKeyId) { + FbsonKeyValue::keyid_type idx = key_id; + os_->put(0); + os_->write((char*)&idx, sizeof(FbsonKeyValue::keyid_type)); + size += sizeof(FbsonKeyValue::keyid_type); + } else { // key id overflow + assert(0); + return 0; + } + + kvState_ = WS_Key; + return size; + } + + return 0; + } + + // write a key id + uint32_t writeKey(FbsonKeyValue::keyid_type idx) { + if (!stack_.empty() && verifyKeyState()) { + os_->put(0); + os_->write((char*)&idx, sizeof(FbsonKeyValue::keyid_type)); + kvState_ = WS_Key; + return sizeof(uint8_t) + sizeof(FbsonKeyValue::keyid_type); + } + + return 0; + } + + uint32_t writeNull() { + if (!stack_.empty() && verifyValueState()) { + os_->put((FbsonTypeUnder)FbsonType::T_Null); + kvState_ = WS_Value; + return sizeof(FbsonValue); + } + + return 0; + } + + uint32_t writeBool(bool b) { + if (!stack_.empty() && verifyValueState()) { + if (b) { + os_->put((FbsonTypeUnder)FbsonType::T_True); + } else { + os_->put((FbsonTypeUnder)FbsonType::T_False); + } + + kvState_ = WS_Value; + return sizeof(FbsonValue); + } + + return 0; + } + + uint32_t writeInt8(int8_t v) { + if (!stack_.empty() && verifyValueState()) { + os_->put((FbsonTypeUnder)FbsonType::T_Int8); + os_->put(v); + kvState_ = WS_Value; + return sizeof(Int8Val); + } + + return 0; + } + + uint32_t writeInt16(int16_t v) { + if (!stack_.empty() && verifyValueState()) { + os_->put((FbsonTypeUnder)FbsonType::T_Int16); + os_->write((char*)&v, sizeof(int16_t)); + kvState_ = WS_Value; + return sizeof(Int16Val); + } + + return 0; + } + + uint32_t writeInt32(int32_t v) { + if (!stack_.empty() && verifyValueState()) { + os_->put((FbsonTypeUnder)FbsonType::T_Int32); + os_->write((char*)&v, sizeof(int32_t)); + kvState_ = WS_Value; + return sizeof(Int32Val); + } + + return 0; + } + + uint32_t writeInt64(int64_t v) { + if (!stack_.empty() && verifyValueState()) { + os_->put((FbsonTypeUnder)FbsonType::T_Int64); + os_->write((char*)&v, sizeof(int64_t)); + kvState_ = WS_Value; + return sizeof(Int64Val); + } + + return 0; + } + + uint32_t writeDouble(double v) { + if (!stack_.empty() && verifyValueState()) { + os_->put((FbsonTypeUnder)FbsonType::T_Double); + os_->write((char*)&v, sizeof(double)); + kvState_ = WS_Value; + return sizeof(DoubleVal); + } + + return 0; + } + + // must call writeStartString before writing a string val + bool writeStartString() { + if (!stack_.empty() && verifyValueState()) { + os_->put((FbsonTypeUnder)FbsonType::T_String); + str_pos_ = os_->tellp(); + + // fill the size bytes with 0 for now + uint32_t size = 0; + os_->write((char*)&size, sizeof(uint32_t)); + + kvState_ = WS_String; + return true; + } + + return false; + } + + // finish writing a string val + bool writeEndString() { + if (kvState_ == WS_String) { + std::streampos cur_pos = os_->tellp(); + int32_t size = (int32_t)(cur_pos - str_pos_ - sizeof(uint32_t)); + assert(size >= 0); + + os_->seekp(str_pos_); + os_->write((char*)&size, sizeof(uint32_t)); + os_->seekp(cur_pos); + + kvState_ = WS_Value; + return true; + } + + return false; + } + + uint32_t writeString(const char* str, uint32_t len) { + if (kvState_ == WS_String) { + os_->write(str, len); + return len; + } + + return 0; + } + + uint32_t writeString(char ch) { + if (kvState_ == WS_String) { + os_->put(ch); + return 1; + } + + return 0; + } + + // must call writeStartBinary before writing a binary val + bool writeStartBinary() { + if (!stack_.empty() && verifyValueState()) { + os_->put((FbsonTypeUnder)FbsonType::T_Binary); + str_pos_ = os_->tellp(); + + // fill the size bytes with 0 for now + uint32_t size = 0; + os_->write((char*)&size, sizeof(uint32_t)); + + kvState_ = WS_Binary; + return true; + } + + return false; + } + + // finish writing a binary val + bool writeEndBinary() { + if (kvState_ == WS_Binary) { + std::streampos cur_pos = os_->tellp(); + int32_t size = (int32_t)(cur_pos - str_pos_ - sizeof(uint32_t)); + assert(size >= 0); + + os_->seekp(str_pos_); + os_->write((char*)&size, sizeof(uint32_t)); + os_->seekp(cur_pos); + + kvState_ = WS_Value; + return true; + } + + return false; + } + + uint32_t writeBinary(const char* bin, uint32_t len) { + if (kvState_ == WS_Binary) { + os_->write(bin, len); + return len; + } + + return 0; + } + + // must call writeStartObject before writing an object val + bool writeStartObject() { + if (stack_.empty() || verifyValueState()) { + if (stack_.empty()) { + // if this is a new FBSON, write the header + if (!hasHdr_) { + writeHeader(); + } else + return false; + } + + os_->put((FbsonTypeUnder)FbsonType::T_Object); + // save the size position + stack_.push(WriteInfo({WS_Object, os_->tellp()})); + + // fill the size bytes with 0 for now + uint32_t size = 0; + os_->write((char*)&size, sizeof(uint32_t)); + + kvState_ = WS_Value; + return true; + } + + return false; + } + + // finish writing an object val + bool writeEndObject() { + if (!stack_.empty() && stack_.top().state == WS_Object && + kvState_ == WS_Value) { + WriteInfo& ci = stack_.top(); + std::streampos cur_pos = os_->tellp(); + int32_t size = (int32_t)(cur_pos - ci.sz_pos - sizeof(uint32_t)); + assert(size >= 0); + + os_->seekp(ci.sz_pos); + os_->write((char*)&size, sizeof(uint32_t)); + os_->seekp(cur_pos); + stack_.pop(); + + return true; + } + + return false; + } + + // must call writeStartArray before writing an array val + bool writeStartArray() { + if (stack_.empty() || verifyValueState()) { + if (stack_.empty()) { + // if this is a new FBSON, write the header + if (!hasHdr_) { + writeHeader(); + } else + return false; + } + + os_->put((FbsonTypeUnder)FbsonType::T_Array); + // save the size position + stack_.push(WriteInfo({WS_Array, os_->tellp()})); + + // fill the size bytes with 0 for now + uint32_t size = 0; + os_->write((char*)&size, sizeof(uint32_t)); + + kvState_ = WS_Value; + return true; + } + + return false; + } + + // finish writing an array val + bool writeEndArray() { + if (!stack_.empty() && stack_.top().state == WS_Array && + kvState_ == WS_Value) { + WriteInfo& ci = stack_.top(); + std::streampos cur_pos = os_->tellp(); + int32_t size = (int32_t)(cur_pos - ci.sz_pos - sizeof(uint32_t)); + assert(size >= 0); + + os_->seekp(ci.sz_pos); + os_->write((char*)&size, sizeof(uint32_t)); + os_->seekp(cur_pos); + stack_.pop(); + + return true; + } + + return false; + } + + OS_TYPE* getOutput() { return os_; } + + private: + // verify we are in the right state before writing a value + bool verifyValueState() { + assert(!stack_.empty()); + return (stack_.top().state == WS_Object && kvState_ == WS_Key) || + (stack_.top().state == WS_Array && kvState_ == WS_Value); + } + + // verify we are in the right state before writing a key + bool verifyKeyState() { + assert(!stack_.empty()); + return stack_.top().state == WS_Object && kvState_ == WS_Value; + } + + void writeHeader() { + os_->put(FBSON_VER); + hasHdr_ = true; + } + + private: + enum WriteState { + WS_NONE, + WS_Array, + WS_Object, + WS_Key, + WS_Value, + WS_String, + WS_Binary, + }; + + struct WriteInfo { + WriteState state; + std::streampos sz_pos; + }; + + private: + OS_TYPE* os_; + bool alloc_; + bool hasHdr_; + WriteState kvState_; // key or value state + std::streampos str_pos_; + std::stack stack_; +}; + +typedef FbsonWriterT FbsonWriter; + +} // namespace fbson + +#endif // FBSON_FBSONWRITER_H diff --git a/third-party/gtest-1.7.0/fused-src/gtest/CMakeLists.txt b/third-party/gtest-1.7.0/fused-src/gtest/CMakeLists.txt new file mode 100644 index 00000000000..90cff088070 --- /dev/null +++ b/third-party/gtest-1.7.0/fused-src/gtest/CMakeLists.txt @@ -0,0 +1 @@ +add_library(gtest gtest-all.cc) diff --git a/third-party/gtest-1.7.0/fused-src/gtest/gtest-all.cc b/third-party/gtest-1.7.0/fused-src/gtest/gtest-all.cc new file mode 100644 index 00000000000..7c5217202f8 --- /dev/null +++ b/third-party/gtest-1.7.0/fused-src/gtest/gtest-all.cc @@ -0,0 +1,10261 @@ +// Copyright 2008, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: mheule@google.com (Markus Heule) +// +// Google C++ Testing Framework (Google Test) +// +// Sometimes it's desirable to build Google Test by compiling a single file. +// This file serves this purpose. + +// Suppress clang analyzer warnings. +#ifndef __clang_analyzer__ + +// This line ensures that gtest.h can be compiled on its own, even +// when it's fused. +#include "gtest/gtest.h" + +// The following lines pull in the real gtest *.cc files. +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) +// +// The Google C++ Testing Framework (Google Test) + +// Copyright 2007, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) +// +// Utilities for testing Google Test itself and code that uses Google Test +// (e.g. frameworks built on top of Google Test). + +#ifndef GTEST_INCLUDE_GTEST_GTEST_SPI_H_ +#define GTEST_INCLUDE_GTEST_GTEST_SPI_H_ + +namespace testing { + +// This helper class can be used to mock out Google Test failure reporting +// so that we can test Google Test or code that builds on Google Test. +// +// An object of this class appends a TestPartResult object to the +// TestPartResultArray object given in the constructor whenever a Google Test +// failure is reported. It can either intercept only failures that are +// generated in the same thread that created this object or it can intercept +// all generated failures. The scope of this mock object can be controlled with +// the second argument to the two arguments constructor. +class GTEST_API_ ScopedFakeTestPartResultReporter + : public TestPartResultReporterInterface { + public: + // The two possible mocking modes of this object. + enum InterceptMode { + INTERCEPT_ONLY_CURRENT_THREAD, // Intercepts only thread local failures. + INTERCEPT_ALL_THREADS // Intercepts all failures. + }; + + // The c'tor sets this object as the test part result reporter used + // by Google Test. The 'result' parameter specifies where to report the + // results. This reporter will only catch failures generated in the current + // thread. DEPRECATED + explicit ScopedFakeTestPartResultReporter(TestPartResultArray* result); + + // Same as above, but you can choose the interception scope of this object. + ScopedFakeTestPartResultReporter(InterceptMode intercept_mode, + TestPartResultArray* result); + + // The d'tor restores the previous test part result reporter. + virtual ~ScopedFakeTestPartResultReporter(); + + // Appends the TestPartResult object to the TestPartResultArray + // received in the constructor. + // + // This method is from the TestPartResultReporterInterface + // interface. + virtual void ReportTestPartResult(const TestPartResult& result); + private: + void Init(); + + const InterceptMode intercept_mode_; + TestPartResultReporterInterface* old_reporter_; + TestPartResultArray* const result_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(ScopedFakeTestPartResultReporter); +}; + +namespace internal { + +// A helper class for implementing EXPECT_FATAL_FAILURE() and +// EXPECT_NONFATAL_FAILURE(). Its destructor verifies that the given +// TestPartResultArray contains exactly one failure that has the given +// type and contains the given substring. If that's not the case, a +// non-fatal failure will be generated. +class GTEST_API_ SingleFailureChecker { + public: + // The constructor remembers the arguments. + SingleFailureChecker(const TestPartResultArray* results, + TestPartResult::Type type, + const string& substr); + ~SingleFailureChecker(); + private: + const TestPartResultArray* const results_; + const TestPartResult::Type type_; + const string substr_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(SingleFailureChecker); +}; + +} // namespace internal + +} // namespace testing + +// A set of macros for testing Google Test assertions or code that's expected +// to generate Google Test fatal failures. It verifies that the given +// statement will cause exactly one fatal Google Test failure with 'substr' +// being part of the failure message. +// +// There are two different versions of this macro. EXPECT_FATAL_FAILURE only +// affects and considers failures generated in the current thread and +// EXPECT_FATAL_FAILURE_ON_ALL_THREADS does the same but for all threads. +// +// The verification of the assertion is done correctly even when the statement +// throws an exception or aborts the current function. +// +// Known restrictions: +// - 'statement' cannot reference local non-static variables or +// non-static members of the current object. +// - 'statement' cannot return a value. +// - You cannot stream a failure message to this macro. +// +// Note that even though the implementations of the following two +// macros are much alike, we cannot refactor them to use a common +// helper macro, due to some peculiarity in how the preprocessor +// works. The AcceptsMacroThatExpandsToUnprotectedComma test in +// gtest_unittest.cc will fail to compile if we do that. +#define EXPECT_FATAL_FAILURE(statement, substr) \ + do { \ + class GTestExpectFatalFailureHelper {\ + public:\ + static void Execute() { statement; }\ + };\ + ::testing::TestPartResultArray gtest_failures;\ + ::testing::internal::SingleFailureChecker gtest_checker(\ + >est_failures, ::testing::TestPartResult::kFatalFailure, (substr));\ + {\ + ::testing::ScopedFakeTestPartResultReporter gtest_reporter(\ + ::testing::ScopedFakeTestPartResultReporter:: \ + INTERCEPT_ONLY_CURRENT_THREAD, >est_failures);\ + GTestExpectFatalFailureHelper::Execute();\ + }\ + } while (::testing::internal::AlwaysFalse()) + +#define EXPECT_FATAL_FAILURE_ON_ALL_THREADS(statement, substr) \ + do { \ + class GTestExpectFatalFailureHelper {\ + public:\ + static void Execute() { statement; }\ + };\ + ::testing::TestPartResultArray gtest_failures;\ + ::testing::internal::SingleFailureChecker gtest_checker(\ + >est_failures, ::testing::TestPartResult::kFatalFailure, (substr));\ + {\ + ::testing::ScopedFakeTestPartResultReporter gtest_reporter(\ + ::testing::ScopedFakeTestPartResultReporter:: \ + INTERCEPT_ALL_THREADS, >est_failures);\ + GTestExpectFatalFailureHelper::Execute();\ + }\ + } while (::testing::internal::AlwaysFalse()) + +// A macro for testing Google Test assertions or code that's expected to +// generate Google Test non-fatal failures. It asserts that the given +// statement will cause exactly one non-fatal Google Test failure with 'substr' +// being part of the failure message. +// +// There are two different versions of this macro. EXPECT_NONFATAL_FAILURE only +// affects and considers failures generated in the current thread and +// EXPECT_NONFATAL_FAILURE_ON_ALL_THREADS does the same but for all threads. +// +// 'statement' is allowed to reference local variables and members of +// the current object. +// +// The verification of the assertion is done correctly even when the statement +// throws an exception or aborts the current function. +// +// Known restrictions: +// - You cannot stream a failure message to this macro. +// +// Note that even though the implementations of the following two +// macros are much alike, we cannot refactor them to use a common +// helper macro, due to some peculiarity in how the preprocessor +// works. If we do that, the code won't compile when the user gives +// EXPECT_NONFATAL_FAILURE() a statement that contains a macro that +// expands to code containing an unprotected comma. The +// AcceptsMacroThatExpandsToUnprotectedComma test in gtest_unittest.cc +// catches that. +// +// For the same reason, we have to write +// if (::testing::internal::AlwaysTrue()) { statement; } +// instead of +// GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement) +// to avoid an MSVC warning on unreachable code. +#define EXPECT_NONFATAL_FAILURE(statement, substr) \ + do {\ + ::testing::TestPartResultArray gtest_failures;\ + ::testing::internal::SingleFailureChecker gtest_checker(\ + >est_failures, ::testing::TestPartResult::kNonFatalFailure, \ + (substr));\ + {\ + ::testing::ScopedFakeTestPartResultReporter gtest_reporter(\ + ::testing::ScopedFakeTestPartResultReporter:: \ + INTERCEPT_ONLY_CURRENT_THREAD, >est_failures);\ + if (::testing::internal::AlwaysTrue()) { statement; }\ + }\ + } while (::testing::internal::AlwaysFalse()) + +#define EXPECT_NONFATAL_FAILURE_ON_ALL_THREADS(statement, substr) \ + do {\ + ::testing::TestPartResultArray gtest_failures;\ + ::testing::internal::SingleFailureChecker gtest_checker(\ + >est_failures, ::testing::TestPartResult::kNonFatalFailure, \ + (substr));\ + {\ + ::testing::ScopedFakeTestPartResultReporter gtest_reporter(\ + ::testing::ScopedFakeTestPartResultReporter::INTERCEPT_ALL_THREADS, \ + >est_failures);\ + if (::testing::internal::AlwaysTrue()) { statement; }\ + }\ + } while (::testing::internal::AlwaysFalse()) + +#endif // GTEST_INCLUDE_GTEST_GTEST_SPI_H_ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include // NOLINT +#include +#include + +#if GTEST_OS_LINUX + +// TODO(kenton@google.com): Use autoconf to detect availability of +// gettimeofday(). +# define GTEST_HAS_GETTIMEOFDAY_ 1 + +# include // NOLINT +# include // NOLINT +# include // NOLINT +// Declares vsnprintf(). This header is not available on Windows. +# include // NOLINT +# include // NOLINT +# include // NOLINT +# include // NOLINT +# include + +#elif GTEST_OS_SYMBIAN +# define GTEST_HAS_GETTIMEOFDAY_ 1 +# include // NOLINT + +#elif GTEST_OS_ZOS +# define GTEST_HAS_GETTIMEOFDAY_ 1 +# include // NOLINT + +// On z/OS we additionally need strings.h for strcasecmp. +# include // NOLINT + +#elif GTEST_OS_WINDOWS_MOBILE // We are on Windows CE. + +# include // NOLINT +# undef min + +#elif GTEST_OS_WINDOWS // We are on Windows proper. + +# include // NOLINT +# include // NOLINT +# include // NOLINT +# include // NOLINT + +# if GTEST_OS_WINDOWS_MINGW +// MinGW has gettimeofday() but not _ftime64(). +// TODO(kenton@google.com): Use autoconf to detect availability of +// gettimeofday(). +// TODO(kenton@google.com): There are other ways to get the time on +// Windows, like GetTickCount() or GetSystemTimeAsFileTime(). MinGW +// supports these. consider using them instead. +# define GTEST_HAS_GETTIMEOFDAY_ 1 +# include // NOLINT +# endif // GTEST_OS_WINDOWS_MINGW + +// cpplint thinks that the header is already included, so we want to +// silence it. +# include // NOLINT +# undef min + +#else + +// Assume other platforms have gettimeofday(). +// TODO(kenton@google.com): Use autoconf to detect availability of +// gettimeofday(). +# define GTEST_HAS_GETTIMEOFDAY_ 1 + +// cpplint thinks that the header is already included, so we want to +// silence it. +# include // NOLINT +# include // NOLINT + +#endif // GTEST_OS_LINUX + +#if GTEST_HAS_EXCEPTIONS +# include +#endif + +#if GTEST_CAN_STREAM_RESULTS_ +# include // NOLINT +# include // NOLINT +# include // NOLINT +# include // NOLINT +#endif + +// Indicates that this translation unit is part of Google Test's +// implementation. It must come before gtest-internal-inl.h is +// included, or there will be a compiler error. This trick is to +// prevent a user from accidentally including gtest-internal-inl.h in +// his code. +#define GTEST_IMPLEMENTATION_ 1 +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Utility functions and classes used by the Google C++ testing framework. +// +// Author: wan@google.com (Zhanyong Wan) +// +// This file contains purely Google Test's internal implementation. Please +// DO NOT #INCLUDE IT IN A USER PROGRAM. + +#ifndef GTEST_SRC_GTEST_INTERNAL_INL_H_ +#define GTEST_SRC_GTEST_INTERNAL_INL_H_ + +// GTEST_IMPLEMENTATION_ is defined to 1 iff the current translation unit is +// part of Google Test's implementation; otherwise it's undefined. +#if !GTEST_IMPLEMENTATION_ +// If this file is included from the user's code, just say no. +# error "gtest-internal-inl.h is part of Google Test's internal implementation." +# error "It must not be included except by Google Test itself." +#endif // GTEST_IMPLEMENTATION_ + +#ifndef _WIN32_WCE +# include +#endif // !_WIN32_WCE +#include +#include // For strtoll/_strtoul64/malloc/free. +#include // For memmove. + +#include +#include +#include + + +#if GTEST_CAN_STREAM_RESULTS_ +# include // NOLINT +# include // NOLINT +#endif + +#if GTEST_OS_WINDOWS +# include // NOLINT +#endif // GTEST_OS_WINDOWS + + +namespace testing { + +// Declares the flags. +// +// We don't want the users to modify this flag in the code, but want +// Google Test's own unit tests to be able to access it. Therefore we +// declare it here as opposed to in gtest.h. +GTEST_DECLARE_bool_(death_test_use_fork); + +namespace internal { + +// The value of GetTestTypeId() as seen from within the Google Test +// library. This is solely for testing GetTestTypeId(). +GTEST_API_ extern const TypeId kTestTypeIdInGoogleTest; + +// Names of the flags (needed for parsing Google Test flags). +const char kAlsoRunDisabledTestsFlag[] = "also_run_disabled_tests"; +const char kBreakOnFailureFlag[] = "break_on_failure"; +const char kCatchExceptionsFlag[] = "catch_exceptions"; +const char kColorFlag[] = "color"; +const char kFilterFlag[] = "filter"; +const char kListTestsFlag[] = "list_tests"; +const char kOutputFlag[] = "output"; +const char kPrintTimeFlag[] = "print_time"; +const char kRandomSeedFlag[] = "random_seed"; +const char kRepeatFlag[] = "repeat"; +const char kShuffleFlag[] = "shuffle"; +const char kStackTraceDepthFlag[] = "stack_trace_depth"; +const char kStreamResultToFlag[] = "stream_result_to"; +const char kThrowOnFailureFlag[] = "throw_on_failure"; + +// A valid random seed must be in [1, kMaxRandomSeed]. +const int kMaxRandomSeed = 99999; + +// g_help_flag is true iff the --help flag or an equivalent form is +// specified on the command line. +GTEST_API_ extern bool g_help_flag; + +// Returns the current time in milliseconds. +GTEST_API_ TimeInMillis GetTimeInMillis(); + +// Returns true iff Google Test should use colors in the output. +GTEST_API_ bool ShouldUseColor(bool stdout_is_tty); + +// Formats the given time in milliseconds as seconds. +GTEST_API_ std::string FormatTimeInMillisAsSeconds(TimeInMillis ms); + +// Converts the given time in milliseconds to a date string in the ISO 8601 +// format, without the timezone information. N.B.: due to the use the +// non-reentrant localtime() function, this function is not thread safe. Do +// not use it in any code that can be called from multiple threads. +GTEST_API_ std::string FormatEpochTimeInMillisAsIso8601(TimeInMillis ms); + +// Parses a string for an Int32 flag, in the form of "--flag=value". +// +// On success, stores the value of the flag in *value, and returns +// true. On failure, returns false without changing *value. +GTEST_API_ bool ParseInt32Flag( + const char* str, const char* flag, Int32* value); + +// Returns a random seed in range [1, kMaxRandomSeed] based on the +// given --gtest_random_seed flag value. +inline int GetRandomSeedFromFlag(Int32 random_seed_flag) { + const unsigned int raw_seed = (random_seed_flag == 0) ? + static_cast(GetTimeInMillis()) : + static_cast(random_seed_flag); + + // Normalizes the actual seed to range [1, kMaxRandomSeed] such that + // it's easy to type. + const int normalized_seed = + static_cast((raw_seed - 1U) % + static_cast(kMaxRandomSeed)) + 1; + return normalized_seed; +} + +// Returns the first valid random seed after 'seed'. The behavior is +// undefined if 'seed' is invalid. The seed after kMaxRandomSeed is +// considered to be 1. +inline int GetNextRandomSeed(int seed) { + GTEST_CHECK_(1 <= seed && seed <= kMaxRandomSeed) + << "Invalid random seed " << seed << " - must be in [1, " + << kMaxRandomSeed << "]."; + const int next_seed = seed + 1; + return (next_seed > kMaxRandomSeed) ? 1 : next_seed; +} + +// This class saves the values of all Google Test flags in its c'tor, and +// restores them in its d'tor. +class GTestFlagSaver { + public: + // The c'tor. + GTestFlagSaver() { + also_run_disabled_tests_ = GTEST_FLAG(also_run_disabled_tests); + break_on_failure_ = GTEST_FLAG(break_on_failure); + catch_exceptions_ = GTEST_FLAG(catch_exceptions); + color_ = GTEST_FLAG(color); + death_test_style_ = GTEST_FLAG(death_test_style); + death_test_use_fork_ = GTEST_FLAG(death_test_use_fork); + filter_ = GTEST_FLAG(filter); + internal_run_death_test_ = GTEST_FLAG(internal_run_death_test); + list_tests_ = GTEST_FLAG(list_tests); + output_ = GTEST_FLAG(output); + print_time_ = GTEST_FLAG(print_time); + random_seed_ = GTEST_FLAG(random_seed); + repeat_ = GTEST_FLAG(repeat); + shuffle_ = GTEST_FLAG(shuffle); + stack_trace_depth_ = GTEST_FLAG(stack_trace_depth); + stream_result_to_ = GTEST_FLAG(stream_result_to); + throw_on_failure_ = GTEST_FLAG(throw_on_failure); + } + + // The d'tor is not virtual. DO NOT INHERIT FROM THIS CLASS. + ~GTestFlagSaver() { + GTEST_FLAG(also_run_disabled_tests) = also_run_disabled_tests_; + GTEST_FLAG(break_on_failure) = break_on_failure_; + GTEST_FLAG(catch_exceptions) = catch_exceptions_; + GTEST_FLAG(color) = color_; + GTEST_FLAG(death_test_style) = death_test_style_; + GTEST_FLAG(death_test_use_fork) = death_test_use_fork_; + GTEST_FLAG(filter) = filter_; + GTEST_FLAG(internal_run_death_test) = internal_run_death_test_; + GTEST_FLAG(list_tests) = list_tests_; + GTEST_FLAG(output) = output_; + GTEST_FLAG(print_time) = print_time_; + GTEST_FLAG(random_seed) = random_seed_; + GTEST_FLAG(repeat) = repeat_; + GTEST_FLAG(shuffle) = shuffle_; + GTEST_FLAG(stack_trace_depth) = stack_trace_depth_; + GTEST_FLAG(stream_result_to) = stream_result_to_; + GTEST_FLAG(throw_on_failure) = throw_on_failure_; + } + + private: + // Fields for saving the original values of flags. + bool also_run_disabled_tests_; + bool break_on_failure_; + bool catch_exceptions_; + std::string color_; + std::string death_test_style_; + bool death_test_use_fork_; + std::string filter_; + std::string internal_run_death_test_; + bool list_tests_; + std::string output_; + bool print_time_; + internal::Int32 random_seed_; + internal::Int32 repeat_; + bool shuffle_; + internal::Int32 stack_trace_depth_; + std::string stream_result_to_; + bool throw_on_failure_; +} GTEST_ATTRIBUTE_UNUSED_; + +// Converts a Unicode code point to a narrow string in UTF-8 encoding. +// code_point parameter is of type UInt32 because wchar_t may not be +// wide enough to contain a code point. +// If the code_point is not a valid Unicode code point +// (i.e. outside of Unicode range U+0 to U+10FFFF) it will be converted +// to "(Invalid Unicode 0xXXXXXXXX)". +GTEST_API_ std::string CodePointToUtf8(UInt32 code_point); + +// Converts a wide string to a narrow string in UTF-8 encoding. +// The wide string is assumed to have the following encoding: +// UTF-16 if sizeof(wchar_t) == 2 (on Windows, Cygwin, Symbian OS) +// UTF-32 if sizeof(wchar_t) == 4 (on Linux) +// Parameter str points to a null-terminated wide string. +// Parameter num_chars may additionally limit the number +// of wchar_t characters processed. -1 is used when the entire string +// should be processed. +// If the string contains code points that are not valid Unicode code points +// (i.e. outside of Unicode range U+0 to U+10FFFF) they will be output +// as '(Invalid Unicode 0xXXXXXXXX)'. If the string is in UTF16 encoding +// and contains invalid UTF-16 surrogate pairs, values in those pairs +// will be encoded as individual Unicode characters from Basic Normal Plane. +GTEST_API_ std::string WideStringToUtf8(const wchar_t* str, int num_chars); + +// Reads the GTEST_SHARD_STATUS_FILE environment variable, and creates the file +// if the variable is present. If a file already exists at this location, this +// function will write over it. If the variable is present, but the file cannot +// be created, prints an error and exits. +void WriteToShardStatusFileIfNeeded(); + +// Checks whether sharding is enabled by examining the relevant +// environment variable values. If the variables are present, +// but inconsistent (e.g., shard_index >= total_shards), prints +// an error and exits. If in_subprocess_for_death_test, sharding is +// disabled because it must only be applied to the original test +// process. Otherwise, we could filter out death tests we intended to execute. +GTEST_API_ bool ShouldShard(const char* total_shards_str, + const char* shard_index_str, + bool in_subprocess_for_death_test); + +// Parses the environment variable var as an Int32. If it is unset, +// returns default_val. If it is not an Int32, prints an error and +// aborts. +GTEST_API_ Int32 Int32FromEnvOrDie(const char* env_var, Int32 default_val); + +// Given the total number of shards, the shard index, and the test id, +// returns true iff the test should be run on this shard. The test id is +// some arbitrary but unique non-negative integer assigned to each test +// method. Assumes that 0 <= shard_index < total_shards. +GTEST_API_ bool ShouldRunTestOnShard( + int total_shards, int shard_index, int test_id); + +// STL container utilities. + +// Returns the number of elements in the given container that satisfy +// the given predicate. +template +inline int CountIf(const Container& c, Predicate predicate) { + // Implemented as an explicit loop since std::count_if() in libCstd on + // Solaris has a non-standard signature. + int count = 0; + for (typename Container::const_iterator it = c.begin(); it != c.end(); ++it) { + if (predicate(*it)) + ++count; + } + return count; +} + +// Applies a function/functor to each element in the container. +template +void ForEach(const Container& c, Functor functor) { + std::for_each(c.begin(), c.end(), functor); +} + +// Returns the i-th element of the vector, or default_value if i is not +// in range [0, v.size()). +template +inline E GetElementOr(const std::vector& v, int i, E default_value) { + return (i < 0 || i >= static_cast(v.size())) ? default_value : v[i]; +} + +// Performs an in-place shuffle of a range of the vector's elements. +// 'begin' and 'end' are element indices as an STL-style range; +// i.e. [begin, end) are shuffled, where 'end' == size() means to +// shuffle to the end of the vector. +template +void ShuffleRange(internal::Random* random, int begin, int end, + std::vector* v) { + const int size = static_cast(v->size()); + GTEST_CHECK_(0 <= begin && begin <= size) + << "Invalid shuffle range start " << begin << ": must be in range [0, " + << size << "]."; + GTEST_CHECK_(begin <= end && end <= size) + << "Invalid shuffle range finish " << end << ": must be in range [" + << begin << ", " << size << "]."; + + // Fisher-Yates shuffle, from + // http://en.wikipedia.org/wiki/Fisher-Yates_shuffle + for (int range_width = end - begin; range_width >= 2; range_width--) { + const int last_in_range = begin + range_width - 1; + const int selected = begin + random->Generate(range_width); + std::swap((*v)[selected], (*v)[last_in_range]); + } +} + +// Performs an in-place shuffle of the vector's elements. +template +inline void Shuffle(internal::Random* random, std::vector* v) { + ShuffleRange(random, 0, static_cast(v->size()), v); +} + +// A function for deleting an object. Handy for being used as a +// functor. +template +static void Delete(T* x) { + delete x; +} + +// A predicate that checks the key of a TestProperty against a known key. +// +// TestPropertyKeyIs is copyable. +class TestPropertyKeyIs { + public: + // Constructor. + // + // TestPropertyKeyIs has NO default constructor. + explicit TestPropertyKeyIs(const std::string& key) : key_(key) {} + + // Returns true iff the test name of test property matches on key_. + bool operator()(const TestProperty& test_property) const { + return test_property.key() == key_; + } + + private: + std::string key_; +}; + +// Class UnitTestOptions. +// +// This class contains functions for processing options the user +// specifies when running the tests. It has only static members. +// +// In most cases, the user can specify an option using either an +// environment variable or a command line flag. E.g. you can set the +// test filter using either GTEST_FILTER or --gtest_filter. If both +// the variable and the flag are present, the latter overrides the +// former. +class GTEST_API_ UnitTestOptions { + public: + // Functions for processing the gtest_output flag. + + // Returns the output format, or "" for normal printed output. + static std::string GetOutputFormat(); + + // Returns the absolute path of the requested output file, or the + // default (test_detail.xml in the original working directory) if + // none was explicitly specified. + static std::string GetAbsolutePathToOutputFile(); + + // Functions for processing the gtest_filter flag. + + // Returns true iff the wildcard pattern matches the string. The + // first ':' or '\0' character in pattern marks the end of it. + // + // This recursive algorithm isn't very efficient, but is clear and + // works well enough for matching test names, which are short. + static bool PatternMatchesString(const char *pattern, const char *str); + + // Returns true iff the user-specified filter matches the test case + // name and the test name. + static bool FilterMatchesTest(const std::string &test_case_name, + const std::string &test_name); + +#if GTEST_OS_WINDOWS + // Function for supporting the gtest_catch_exception flag. + + // Returns EXCEPTION_EXECUTE_HANDLER if Google Test should handle the + // given SEH exception, or EXCEPTION_CONTINUE_SEARCH otherwise. + // This function is useful as an __except condition. + static int GTestShouldProcessSEH(DWORD exception_code); +#endif // GTEST_OS_WINDOWS + + // Returns true if "name" matches the ':' separated list of glob-style + // filters in "filter". + static bool MatchesFilter(const std::string& name, const char* filter); +}; + +// Returns the current application's name, removing directory path if that +// is present. Used by UnitTestOptions::GetOutputFile. +GTEST_API_ FilePath GetCurrentExecutableName(); + +// The role interface for getting the OS stack trace as a string. +class OsStackTraceGetterInterface { + public: + OsStackTraceGetterInterface() {} + virtual ~OsStackTraceGetterInterface() {} + + // Returns the current OS stack trace as an std::string. Parameters: + // + // max_depth - the maximum number of stack frames to be included + // in the trace. + // skip_count - the number of top frames to be skipped; doesn't count + // against max_depth. + virtual string CurrentStackTrace(int max_depth, int skip_count) = 0; + + // UponLeavingGTest() should be called immediately before Google Test calls + // user code. It saves some information about the current stack that + // CurrentStackTrace() will use to find and hide Google Test stack frames. + virtual void UponLeavingGTest() = 0; + + private: + GTEST_DISALLOW_COPY_AND_ASSIGN_(OsStackTraceGetterInterface); +}; + +// A working implementation of the OsStackTraceGetterInterface interface. +class OsStackTraceGetter : public OsStackTraceGetterInterface { + public: + OsStackTraceGetter() : caller_frame_(NULL) {} + + virtual string CurrentStackTrace(int max_depth, int skip_count) + GTEST_LOCK_EXCLUDED_(mutex_); + + virtual void UponLeavingGTest() GTEST_LOCK_EXCLUDED_(mutex_); + + // This string is inserted in place of stack frames that are part of + // Google Test's implementation. + static const char* const kElidedFramesMarker; + + private: + Mutex mutex_; // protects all internal state + + // We save the stack frame below the frame that calls user code. + // We do this because the address of the frame immediately below + // the user code changes between the call to UponLeavingGTest() + // and any calls to CurrentStackTrace() from within the user code. + void* caller_frame_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(OsStackTraceGetter); +}; + +// Information about a Google Test trace point. +struct TraceInfo { + const char* file; + int line; + std::string message; +}; + +// This is the default global test part result reporter used in UnitTestImpl. +// This class should only be used by UnitTestImpl. +class DefaultGlobalTestPartResultReporter + : public TestPartResultReporterInterface { + public: + explicit DefaultGlobalTestPartResultReporter(UnitTestImpl* unit_test); + // Implements the TestPartResultReporterInterface. Reports the test part + // result in the current test. + virtual void ReportTestPartResult(const TestPartResult& result); + + private: + UnitTestImpl* const unit_test_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(DefaultGlobalTestPartResultReporter); +}; + +// This is the default per thread test part result reporter used in +// UnitTestImpl. This class should only be used by UnitTestImpl. +class DefaultPerThreadTestPartResultReporter + : public TestPartResultReporterInterface { + public: + explicit DefaultPerThreadTestPartResultReporter(UnitTestImpl* unit_test); + // Implements the TestPartResultReporterInterface. The implementation just + // delegates to the current global test part result reporter of *unit_test_. + virtual void ReportTestPartResult(const TestPartResult& result); + + private: + UnitTestImpl* const unit_test_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(DefaultPerThreadTestPartResultReporter); +}; + +// The private implementation of the UnitTest class. We don't protect +// the methods under a mutex, as this class is not accessible by a +// user and the UnitTest class that delegates work to this class does +// proper locking. +class GTEST_API_ UnitTestImpl { + public: + explicit UnitTestImpl(UnitTest* parent); + virtual ~UnitTestImpl(); + + // There are two different ways to register your own TestPartResultReporter. + // You can register your own repoter to listen either only for test results + // from the current thread or for results from all threads. + // By default, each per-thread test result repoter just passes a new + // TestPartResult to the global test result reporter, which registers the + // test part result for the currently running test. + + // Returns the global test part result reporter. + TestPartResultReporterInterface* GetGlobalTestPartResultReporter(); + + // Sets the global test part result reporter. + void SetGlobalTestPartResultReporter( + TestPartResultReporterInterface* reporter); + + // Returns the test part result reporter for the current thread. + TestPartResultReporterInterface* GetTestPartResultReporterForCurrentThread(); + + // Sets the test part result reporter for the current thread. + void SetTestPartResultReporterForCurrentThread( + TestPartResultReporterInterface* reporter); + + // Gets the number of successful test cases. + int successful_test_case_count() const; + + // Gets the number of failed test cases. + int failed_test_case_count() const; + + // Gets the number of all test cases. + int total_test_case_count() const; + + // Gets the number of all test cases that contain at least one test + // that should run. + int test_case_to_run_count() const; + + // Gets the number of successful tests. + int successful_test_count() const; + + // Gets the number of failed tests. + int failed_test_count() const; + + // Gets the number of disabled tests that will be reported in the XML report. + int reportable_disabled_test_count() const; + + // Gets the number of disabled tests. + int disabled_test_count() const; + + // Gets the number of tests to be printed in the XML report. + int reportable_test_count() const; + + // Gets the number of all tests. + int total_test_count() const; + + // Gets the number of tests that should run. + int test_to_run_count() const; + + // Gets the time of the test program start, in ms from the start of the + // UNIX epoch. + TimeInMillis start_timestamp() const { return start_timestamp_; } + + // Gets the elapsed time, in milliseconds. + TimeInMillis elapsed_time() const { return elapsed_time_; } + + // Returns true iff the unit test passed (i.e. all test cases passed). + bool Passed() const { return !Failed(); } + + // Returns true iff the unit test failed (i.e. some test case failed + // or something outside of all tests failed). + bool Failed() const { + return failed_test_case_count() > 0 || ad_hoc_test_result()->Failed(); + } + + // Gets the i-th test case among all the test cases. i can range from 0 to + // total_test_case_count() - 1. If i is not in that range, returns NULL. + const TestCase* GetTestCase(int i) const { + const int index = GetElementOr(test_case_indices_, i, -1); + return index < 0 ? NULL : test_cases_[i]; + } + + // Gets the i-th test case among all the test cases. i can range from 0 to + // total_test_case_count() - 1. If i is not in that range, returns NULL. + TestCase* GetMutableTestCase(int i) { + const int index = GetElementOr(test_case_indices_, i, -1); + return index < 0 ? NULL : test_cases_[index]; + } + + // Provides access to the event listener list. + TestEventListeners* listeners() { return &listeners_; } + + // Returns the TestResult for the test that's currently running, or + // the TestResult for the ad hoc test if no test is running. + TestResult* current_test_result(); + + // Returns the TestResult for the ad hoc test. + const TestResult* ad_hoc_test_result() const { return &ad_hoc_test_result_; } + + // Sets the OS stack trace getter. + // + // Does nothing if the input and the current OS stack trace getter + // are the same; otherwise, deletes the old getter and makes the + // input the current getter. + void set_os_stack_trace_getter(OsStackTraceGetterInterface* getter); + + // Returns the current OS stack trace getter if it is not NULL; + // otherwise, creates an OsStackTraceGetter, makes it the current + // getter, and returns it. + OsStackTraceGetterInterface* os_stack_trace_getter(); + + // Returns the current OS stack trace as an std::string. + // + // The maximum number of stack frames to be included is specified by + // the gtest_stack_trace_depth flag. The skip_count parameter + // specifies the number of top frames to be skipped, which doesn't + // count against the number of frames to be included. + // + // For example, if Foo() calls Bar(), which in turn calls + // CurrentOsStackTraceExceptTop(1), Foo() will be included in the + // trace but Bar() and CurrentOsStackTraceExceptTop() won't. + std::string CurrentOsStackTraceExceptTop(int skip_count) GTEST_NO_INLINE_; + + // Finds and returns a TestCase with the given name. If one doesn't + // exist, creates one and returns it. + // + // Arguments: + // + // test_case_name: name of the test case + // type_param: the name of the test's type parameter, or NULL if + // this is not a typed or a type-parameterized test. + // set_up_tc: pointer to the function that sets up the test case + // tear_down_tc: pointer to the function that tears down the test case + TestCase* GetTestCase(const char* test_case_name, + const char* type_param, + Test::SetUpTestCaseFunc set_up_tc, + Test::TearDownTestCaseFunc tear_down_tc); + + // Adds a TestInfo to the unit test. + // + // Arguments: + // + // set_up_tc: pointer to the function that sets up the test case + // tear_down_tc: pointer to the function that tears down the test case + // test_info: the TestInfo object + void AddTestInfo(Test::SetUpTestCaseFunc set_up_tc, + Test::TearDownTestCaseFunc tear_down_tc, + TestInfo* test_info) { + // In order to support thread-safe death tests, we need to + // remember the original working directory when the test program + // was first invoked. We cannot do this in RUN_ALL_TESTS(), as + // the user may have changed the current directory before calling + // RUN_ALL_TESTS(). Therefore we capture the current directory in + // AddTestInfo(), which is called to register a TEST or TEST_F + // before main() is reached. + if (original_working_dir_.IsEmpty()) { + original_working_dir_.Set(FilePath::GetCurrentDir()); + GTEST_CHECK_(!original_working_dir_.IsEmpty()) + << "Failed to get the current working directory."; + } + + GetTestCase(test_info->test_case_name(), + test_info->type_param(), + set_up_tc, + tear_down_tc)->AddTestInfo(test_info); + } + +#if GTEST_HAS_PARAM_TEST + // Returns ParameterizedTestCaseRegistry object used to keep track of + // value-parameterized tests and instantiate and register them. + internal::ParameterizedTestCaseRegistry& parameterized_test_registry() { + return parameterized_test_registry_; + } +#endif // GTEST_HAS_PARAM_TEST + + // Sets the TestCase object for the test that's currently running. + void set_current_test_case(TestCase* a_current_test_case) { + current_test_case_ = a_current_test_case; + } + + // Sets the TestInfo object for the test that's currently running. If + // current_test_info is NULL, the assertion results will be stored in + // ad_hoc_test_result_. + void set_current_test_info(TestInfo* a_current_test_info) { + current_test_info_ = a_current_test_info; + } + + // Registers all parameterized tests defined using TEST_P and + // INSTANTIATE_TEST_CASE_P, creating regular tests for each test/parameter + // combination. This method can be called more then once; it has guards + // protecting from registering the tests more then once. If + // value-parameterized tests are disabled, RegisterParameterizedTests is + // present but does nothing. + void RegisterParameterizedTests(); + + // Runs all tests in this UnitTest object, prints the result, and + // returns true if all tests are successful. If any exception is + // thrown during a test, this test is considered to be failed, but + // the rest of the tests will still be run. + bool RunAllTests(); + + // Clears the results of all tests, except the ad hoc tests. + void ClearNonAdHocTestResult() { + ForEach(test_cases_, TestCase::ClearTestCaseResult); + } + + // Clears the results of ad-hoc test assertions. + void ClearAdHocTestResult() { + ad_hoc_test_result_.Clear(); + } + + // Adds a TestProperty to the current TestResult object when invoked in a + // context of a test or a test case, or to the global property set. If the + // result already contains a property with the same key, the value will be + // updated. + void RecordProperty(const TestProperty& test_property); + + enum ReactionToSharding { + HONOR_SHARDING_PROTOCOL, + IGNORE_SHARDING_PROTOCOL + }; + + // Matches the full name of each test against the user-specified + // filter to decide whether the test should run, then records the + // result in each TestCase and TestInfo object. + // If shard_tests == HONOR_SHARDING_PROTOCOL, further filters tests + // based on sharding variables in the environment. + // Returns the number of tests that should run. + int FilterTests(ReactionToSharding shard_tests); + + // Prints the names of the tests matching the user-specified filter flag. + void ListTestsMatchingFilter(); + + const TestCase* current_test_case() const { return current_test_case_; } + TestInfo* current_test_info() { return current_test_info_; } + const TestInfo* current_test_info() const { return current_test_info_; } + + // Returns the vector of environments that need to be set-up/torn-down + // before/after the tests are run. + std::vector& environments() { return environments_; } + + // Getters for the per-thread Google Test trace stack. + std::vector& gtest_trace_stack() { + return *(gtest_trace_stack_.pointer()); + } + const std::vector& gtest_trace_stack() const { + return gtest_trace_stack_.get(); + } + +#if GTEST_HAS_DEATH_TEST + void InitDeathTestSubprocessControlInfo() { + internal_run_death_test_flag_.reset(ParseInternalRunDeathTestFlag()); + } + // Returns a pointer to the parsed --gtest_internal_run_death_test + // flag, or NULL if that flag was not specified. + // This information is useful only in a death test child process. + // Must not be called before a call to InitGoogleTest. + const InternalRunDeathTestFlag* internal_run_death_test_flag() const { + return internal_run_death_test_flag_.get(); + } + + // Returns a pointer to the current death test factory. + internal::DeathTestFactory* death_test_factory() { + return death_test_factory_.get(); + } + + void SuppressTestEventsIfInSubprocess(); + + friend class ReplaceDeathTestFactory; +#endif // GTEST_HAS_DEATH_TEST + + // Initializes the event listener performing XML output as specified by + // UnitTestOptions. Must not be called before InitGoogleTest. + void ConfigureXmlOutput(); + +#if GTEST_CAN_STREAM_RESULTS_ + // Initializes the event listener for streaming test results to a socket. + // Must not be called before InitGoogleTest. + void ConfigureStreamingOutput(); +#endif + + // Performs initialization dependent upon flag values obtained in + // ParseGoogleTestFlagsOnly. Is called from InitGoogleTest after the call to + // ParseGoogleTestFlagsOnly. In case a user neglects to call InitGoogleTest + // this function is also called from RunAllTests. Since this function can be + // called more than once, it has to be idempotent. + void PostFlagParsingInit(); + + // Gets the random seed used at the start of the current test iteration. + int random_seed() const { return random_seed_; } + + // Gets the random number generator. + internal::Random* random() { return &random_; } + + // Shuffles all test cases, and the tests within each test case, + // making sure that death tests are still run first. + void ShuffleTests(); + + // Restores the test cases and tests to their order before the first shuffle. + void UnshuffleTests(); + + // Returns the value of GTEST_FLAG(catch_exceptions) at the moment + // UnitTest::Run() starts. + bool catch_exceptions() const { return catch_exceptions_; } + + private: + friend class ::testing::UnitTest; + + // Used by UnitTest::Run() to capture the state of + // GTEST_FLAG(catch_exceptions) at the moment it starts. + void set_catch_exceptions(bool value) { catch_exceptions_ = value; } + + // The UnitTest object that owns this implementation object. + UnitTest* const parent_; + + // The working directory when the first TEST() or TEST_F() was + // executed. + internal::FilePath original_working_dir_; + + // The default test part result reporters. + DefaultGlobalTestPartResultReporter default_global_test_part_result_reporter_; + DefaultPerThreadTestPartResultReporter + default_per_thread_test_part_result_reporter_; + + // Points to (but doesn't own) the global test part result reporter. + TestPartResultReporterInterface* global_test_part_result_repoter_; + + // Protects read and write access to global_test_part_result_reporter_. + internal::Mutex global_test_part_result_reporter_mutex_; + + // Points to (but doesn't own) the per-thread test part result reporter. + internal::ThreadLocal + per_thread_test_part_result_reporter_; + + // The vector of environments that need to be set-up/torn-down + // before/after the tests are run. + std::vector environments_; + + // The vector of TestCases in their original order. It owns the + // elements in the vector. + std::vector test_cases_; + + // Provides a level of indirection for the test case list to allow + // easy shuffling and restoring the test case order. The i-th + // element of this vector is the index of the i-th test case in the + // shuffled order. + std::vector test_case_indices_; + +#if GTEST_HAS_PARAM_TEST + // ParameterizedTestRegistry object used to register value-parameterized + // tests. + internal::ParameterizedTestCaseRegistry parameterized_test_registry_; + + // Indicates whether RegisterParameterizedTests() has been called already. + bool parameterized_tests_registered_; +#endif // GTEST_HAS_PARAM_TEST + + // Index of the last death test case registered. Initially -1. + int last_death_test_case_; + + // This points to the TestCase for the currently running test. It + // changes as Google Test goes through one test case after another. + // When no test is running, this is set to NULL and Google Test + // stores assertion results in ad_hoc_test_result_. Initially NULL. + TestCase* current_test_case_; + + // This points to the TestInfo for the currently running test. It + // changes as Google Test goes through one test after another. When + // no test is running, this is set to NULL and Google Test stores + // assertion results in ad_hoc_test_result_. Initially NULL. + TestInfo* current_test_info_; + + // Normally, a user only writes assertions inside a TEST or TEST_F, + // or inside a function called by a TEST or TEST_F. Since Google + // Test keeps track of which test is current running, it can + // associate such an assertion with the test it belongs to. + // + // If an assertion is encountered when no TEST or TEST_F is running, + // Google Test attributes the assertion result to an imaginary "ad hoc" + // test, and records the result in ad_hoc_test_result_. + TestResult ad_hoc_test_result_; + + // The list of event listeners that can be used to track events inside + // Google Test. + TestEventListeners listeners_; + + // The OS stack trace getter. Will be deleted when the UnitTest + // object is destructed. By default, an OsStackTraceGetter is used, + // but the user can set this field to use a custom getter if that is + // desired. + OsStackTraceGetterInterface* os_stack_trace_getter_; + + // True iff PostFlagParsingInit() has been called. + bool post_flag_parse_init_performed_; + + // The random number seed used at the beginning of the test run. + int random_seed_; + + // Our random number generator. + internal::Random random_; + + // The time of the test program start, in ms from the start of the + // UNIX epoch. + TimeInMillis start_timestamp_; + + // How long the test took to run, in milliseconds. + TimeInMillis elapsed_time_; + +#if GTEST_HAS_DEATH_TEST + // The decomposed components of the gtest_internal_run_death_test flag, + // parsed when RUN_ALL_TESTS is called. + internal::scoped_ptr internal_run_death_test_flag_; + internal::scoped_ptr death_test_factory_; +#endif // GTEST_HAS_DEATH_TEST + + // A per-thread stack of traces created by the SCOPED_TRACE() macro. + internal::ThreadLocal > gtest_trace_stack_; + + // The value of GTEST_FLAG(catch_exceptions) at the moment RunAllTests() + // starts. + bool catch_exceptions_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(UnitTestImpl); +}; // class UnitTestImpl + +// Convenience function for accessing the global UnitTest +// implementation object. +inline UnitTestImpl* GetUnitTestImpl() { + return UnitTest::GetInstance()->impl(); +} + +#if GTEST_USES_SIMPLE_RE + +// Internal helper functions for implementing the simple regular +// expression matcher. +GTEST_API_ bool IsInSet(char ch, const char* str); +GTEST_API_ bool IsAsciiDigit(char ch); +GTEST_API_ bool IsAsciiPunct(char ch); +GTEST_API_ bool IsRepeat(char ch); +GTEST_API_ bool IsAsciiWhiteSpace(char ch); +GTEST_API_ bool IsAsciiWordChar(char ch); +GTEST_API_ bool IsValidEscape(char ch); +GTEST_API_ bool AtomMatchesChar(bool escaped, char pattern, char ch); +GTEST_API_ bool ValidateRegex(const char* regex); +GTEST_API_ bool MatchRegexAtHead(const char* regex, const char* str); +GTEST_API_ bool MatchRepetitionAndRegexAtHead( + bool escaped, char ch, char repeat, const char* regex, const char* str); +GTEST_API_ bool MatchRegexAnywhere(const char* regex, const char* str); + +#endif // GTEST_USES_SIMPLE_RE + +// Parses the command line for Google Test flags, without initializing +// other parts of Google Test. +GTEST_API_ void ParseGoogleTestFlagsOnly(int* argc, char** argv); +GTEST_API_ void ParseGoogleTestFlagsOnly(int* argc, wchar_t** argv); + +#if GTEST_HAS_DEATH_TEST + +// Returns the message describing the last system error, regardless of the +// platform. +GTEST_API_ std::string GetLastErrnoDescription(); + +// Attempts to parse a string into a positive integer pointed to by the +// number parameter. Returns true if that is possible. +// GTEST_HAS_DEATH_TEST implies that we have ::std::string, so we can use +// it here. +template +bool ParseNaturalNumber(const ::std::string& str, Integer* number) { + // Fail fast if the given string does not begin with a digit; + // this bypasses strtoXXX's "optional leading whitespace and plus + // or minus sign" semantics, which are undesirable here. + if (str.empty() || !IsDigit(str[0])) { + return false; + } + errno = 0; + + char* end; + // BiggestConvertible is the largest integer type that system-provided + // string-to-number conversion routines can return. + +# if GTEST_OS_WINDOWS && !defined(__GNUC__) + + // MSVC and C++ Builder define __int64 instead of the standard long long. + typedef unsigned __int64 BiggestConvertible; + const BiggestConvertible parsed = _strtoui64(str.c_str(), &end, 10); + +# else + + typedef unsigned long long BiggestConvertible; // NOLINT + const BiggestConvertible parsed = strtoull(str.c_str(), &end, 10); + +# endif // GTEST_OS_WINDOWS && !defined(__GNUC__) + + const bool parse_success = *end == '\0' && errno == 0; + + // TODO(vladl@google.com): Convert this to compile time assertion when it is + // available. + GTEST_CHECK_(sizeof(Integer) <= sizeof(parsed)); + + const Integer result = static_cast(parsed); + if (parse_success && static_cast(result) == parsed) { + *number = result; + return true; + } + return false; +} +#endif // GTEST_HAS_DEATH_TEST + +// TestResult contains some private methods that should be hidden from +// Google Test user but are required for testing. This class allow our tests +// to access them. +// +// This class is supplied only for the purpose of testing Google Test's own +// constructs. Do not use it in user tests, either directly or indirectly. +class TestResultAccessor { + public: + static void RecordProperty(TestResult* test_result, + const std::string& xml_element, + const TestProperty& property) { + test_result->RecordProperty(xml_element, property); + } + + static void ClearTestPartResults(TestResult* test_result) { + test_result->ClearTestPartResults(); + } + + static const std::vector& test_part_results( + const TestResult& test_result) { + return test_result.test_part_results(); + } +}; + +#if GTEST_CAN_STREAM_RESULTS_ + +// Streams test results to the given port on the given host machine. +class StreamingListener : public EmptyTestEventListener { + public: + // Abstract base class for writing strings to a socket. + class AbstractSocketWriter { + public: + virtual ~AbstractSocketWriter() {} + + // Sends a string to the socket. + virtual void Send(const string& message) = 0; + + // Closes the socket. + virtual void CloseConnection() {} + + // Sends a string and a newline to the socket. + void SendLn(const string& message) { + Send(message + "\n"); + } + }; + + // Concrete class for actually writing strings to a socket. + class SocketWriter : public AbstractSocketWriter { + public: + SocketWriter(const string& host, const string& port) + : sockfd_(-1), host_name_(host), port_num_(port) { + MakeConnection(); + } + + virtual ~SocketWriter() { + if (sockfd_ != -1) + CloseConnection(); + } + + // Sends a string to the socket. + virtual void Send(const string& message) { + GTEST_CHECK_(sockfd_ != -1) + << "Send() can be called only when there is a connection."; + + const int len = static_cast(message.length()); + if (write(sockfd_, message.c_str(), len) != len) { + GTEST_LOG_(WARNING) + << "stream_result_to: failed to stream to " + << host_name_ << ":" << port_num_; + } + } + + private: + // Creates a client socket and connects to the server. + void MakeConnection(); + + // Closes the socket. + void CloseConnection() { + GTEST_CHECK_(sockfd_ != -1) + << "CloseConnection() can be called only when there is a connection."; + + close(sockfd_); + sockfd_ = -1; + } + + int sockfd_; // socket file descriptor + const string host_name_; + const string port_num_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(SocketWriter); + }; // class SocketWriter + + // Escapes '=', '&', '%', and '\n' characters in str as "%xx". + static string UrlEncode(const char* str); + + StreamingListener(const string& host, const string& port) + : socket_writer_(new SocketWriter(host, port)) { Start(); } + + explicit StreamingListener(AbstractSocketWriter* socket_writer) + : socket_writer_(socket_writer) { Start(); } + + void OnTestProgramStart(const UnitTest& /* unit_test */) { + SendLn("event=TestProgramStart"); + } + + void OnTestProgramEnd(const UnitTest& unit_test) { + // Note that Google Test current only report elapsed time for each + // test iteration, not for the entire test program. + SendLn("event=TestProgramEnd&passed=" + FormatBool(unit_test.Passed())); + + // Notify the streaming server to stop. + socket_writer_->CloseConnection(); + } + + void OnTestIterationStart(const UnitTest& /* unit_test */, int iteration) { + SendLn("event=TestIterationStart&iteration=" + + StreamableToString(iteration)); + } + + void OnTestIterationEnd(const UnitTest& unit_test, int /* iteration */) { + SendLn("event=TestIterationEnd&passed=" + + FormatBool(unit_test.Passed()) + "&elapsed_time=" + + StreamableToString(unit_test.elapsed_time()) + "ms"); + } + + void OnTestCaseStart(const TestCase& test_case) { + SendLn(std::string("event=TestCaseStart&name=") + test_case.name()); + } + + void OnTestCaseEnd(const TestCase& test_case) { + SendLn("event=TestCaseEnd&passed=" + FormatBool(test_case.Passed()) + + "&elapsed_time=" + StreamableToString(test_case.elapsed_time()) + + "ms"); + } + + void OnTestStart(const TestInfo& test_info) { + SendLn(std::string("event=TestStart&name=") + test_info.name()); + } + + void OnTestEnd(const TestInfo& test_info) { + SendLn("event=TestEnd&passed=" + + FormatBool((test_info.result())->Passed()) + + "&elapsed_time=" + + StreamableToString((test_info.result())->elapsed_time()) + "ms"); + } + + void OnTestPartResult(const TestPartResult& test_part_result) { + const char* file_name = test_part_result.file_name(); + if (file_name == NULL) + file_name = ""; + SendLn("event=TestPartResult&file=" + UrlEncode(file_name) + + "&line=" + StreamableToString(test_part_result.line_number()) + + "&message=" + UrlEncode(test_part_result.message())); + } + + private: + // Sends the given message and a newline to the socket. + void SendLn(const string& message) { socket_writer_->SendLn(message); } + + // Called at the start of streaming to notify the receiver what + // protocol we are using. + void Start() { SendLn("gtest_streaming_protocol_version=1.0"); } + + string FormatBool(bool value) { return value ? "1" : "0"; } + + const scoped_ptr socket_writer_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(StreamingListener); +}; // class StreamingListener + +#endif // GTEST_CAN_STREAM_RESULTS_ + +} // namespace internal +} // namespace testing + +#endif // GTEST_SRC_GTEST_INTERNAL_INL_H_ +#undef GTEST_IMPLEMENTATION_ + +#if GTEST_OS_WINDOWS +# define vsnprintf _vsnprintf +#endif // GTEST_OS_WINDOWS + +namespace testing { + +using internal::CountIf; +using internal::ForEach; +using internal::GetElementOr; +using internal::Shuffle; + +// Constants. + +// A test whose test case name or test name matches this filter is +// disabled and not run. +static const char kDisableTestFilter[] = "DISABLED_*:*/DISABLED_*"; + +// A test case whose name matches this filter is considered a death +// test case and will be run before test cases whose name doesn't +// match this filter. +static const char kDeathTestCaseFilter[] = "*DeathTest:*DeathTest/*"; + +// A test filter that matches everything. +static const char kUniversalFilter[] = "*"; + +// The default output file for XML output. +static const char kDefaultOutputFile[] = "test_detail.xml"; + +// The environment variable name for the test shard index. +static const char kTestShardIndex[] = "GTEST_SHARD_INDEX"; +// The environment variable name for the total number of test shards. +static const char kTestTotalShards[] = "GTEST_TOTAL_SHARDS"; +// The environment variable name for the test shard status file. +static const char kTestShardStatusFile[] = "GTEST_SHARD_STATUS_FILE"; + +namespace internal { + +// The text used in failure messages to indicate the start of the +// stack trace. +const char kStackTraceMarker[] = "\nStack trace:\n"; + +// g_help_flag is true iff the --help flag or an equivalent form is +// specified on the command line. +bool g_help_flag = false; + +} // namespace internal + +static const char* GetDefaultFilter() { + return kUniversalFilter; +} + +GTEST_DEFINE_bool_( + also_run_disabled_tests, + internal::BoolFromGTestEnv("also_run_disabled_tests", false), + "Run disabled tests too, in addition to the tests normally being run."); + +GTEST_DEFINE_bool_( + break_on_failure, + internal::BoolFromGTestEnv("break_on_failure", false), + "True iff a failed assertion should be a debugger break-point."); + +GTEST_DEFINE_bool_( + catch_exceptions, + internal::BoolFromGTestEnv("catch_exceptions", true), + "True iff " GTEST_NAME_ + " should catch exceptions and treat them as test failures."); + +GTEST_DEFINE_string_( + color, + internal::StringFromGTestEnv("color", "auto"), + "Whether to use colors in the output. Valid values: yes, no, " + "and auto. 'auto' means to use colors if the output is " + "being sent to a terminal and the TERM environment variable " + "is set to a terminal type that supports colors."); + +GTEST_DEFINE_string_( + filter, + internal::StringFromGTestEnv("filter", GetDefaultFilter()), + "A colon-separated list of glob (not regex) patterns " + "for filtering the tests to run, optionally followed by a " + "'-' and a : separated list of negative patterns (tests to " + "exclude). A test is run if it matches one of the positive " + "patterns and does not match any of the negative patterns."); + +GTEST_DEFINE_bool_(list_tests, false, + "List all tests without running them."); + +GTEST_DEFINE_string_( + output, + internal::StringFromGTestEnv("output", ""), + "A format (currently must be \"xml\"), optionally followed " + "by a colon and an output file name or directory. A directory " + "is indicated by a trailing pathname separator. " + "Examples: \"xml:filename.xml\", \"xml::directoryname/\". " + "If a directory is specified, output files will be created " + "within that directory, with file-names based on the test " + "executable's name and, if necessary, made unique by adding " + "digits."); + +GTEST_DEFINE_bool_( + print_time, + internal::BoolFromGTestEnv("print_time", true), + "True iff " GTEST_NAME_ + " should display elapsed time in text output."); + +GTEST_DEFINE_int32_( + random_seed, + internal::Int32FromGTestEnv("random_seed", 0), + "Random number seed to use when shuffling test orders. Must be in range " + "[1, 99999], or 0 to use a seed based on the current time."); + +GTEST_DEFINE_int32_( + repeat, + internal::Int32FromGTestEnv("repeat", 1), + "How many times to repeat each test. Specify a negative number " + "for repeating forever. Useful for shaking out flaky tests."); + +GTEST_DEFINE_bool_( + show_internal_stack_frames, false, + "True iff " GTEST_NAME_ " should include internal stack frames when " + "printing test failure stack traces."); + +GTEST_DEFINE_bool_( + shuffle, + internal::BoolFromGTestEnv("shuffle", false), + "True iff " GTEST_NAME_ + " should randomize tests' order on every run."); + +GTEST_DEFINE_int32_( + stack_trace_depth, + internal::Int32FromGTestEnv("stack_trace_depth", kMaxStackTraceDepth), + "The maximum number of stack frames to print when an " + "assertion fails. The valid range is 0 through 100, inclusive."); + +GTEST_DEFINE_string_( + stream_result_to, + internal::StringFromGTestEnv("stream_result_to", ""), + "This flag specifies the host name and the port number on which to stream " + "test results. Example: \"localhost:555\". The flag is effective only on " + "Linux."); + +GTEST_DEFINE_bool_( + throw_on_failure, + internal::BoolFromGTestEnv("throw_on_failure", false), + "When this flag is specified, a failed assertion will throw an exception " + "if exceptions are enabled or exit the program with a non-zero code " + "otherwise."); + +namespace internal { + +// Generates a random number from [0, range), using a Linear +// Congruential Generator (LCG). Crashes if 'range' is 0 or greater +// than kMaxRange. +UInt32 Random::Generate(UInt32 range) { + // These constants are the same as are used in glibc's rand(3). + state_ = (1103515245U*state_ + 12345U) % kMaxRange; + + GTEST_CHECK_(range > 0) + << "Cannot generate a number in the range [0, 0)."; + GTEST_CHECK_(range <= kMaxRange) + << "Generation of a number in [0, " << range << ") was requested, " + << "but this can only generate numbers in [0, " << kMaxRange << ")."; + + // Converting via modulus introduces a bit of downward bias, but + // it's simple, and a linear congruential generator isn't too good + // to begin with. + return state_ % range; +} + +// GTestIsInitialized() returns true iff the user has initialized +// Google Test. Useful for catching the user mistake of not initializing +// Google Test before calling RUN_ALL_TESTS(). +// +// A user must call testing::InitGoogleTest() to initialize Google +// Test. g_init_gtest_count is set to the number of times +// InitGoogleTest() has been called. We don't protect this variable +// under a mutex as it is only accessed in the main thread. +GTEST_API_ int g_init_gtest_count = 0; +static bool GTestIsInitialized() { return g_init_gtest_count != 0; } + +// Iterates over a vector of TestCases, keeping a running sum of the +// results of calling a given int-returning method on each. +// Returns the sum. +static int SumOverTestCaseList(const std::vector& case_list, + int (TestCase::*method)() const) { + int sum = 0; + for (size_t i = 0; i < case_list.size(); i++) { + sum += (case_list[i]->*method)(); + } + return sum; +} + +// Returns true iff the test case passed. +static bool TestCasePassed(const TestCase* test_case) { + return test_case->should_run() && test_case->Passed(); +} + +// Returns true iff the test case failed. +static bool TestCaseFailed(const TestCase* test_case) { + return test_case->should_run() && test_case->Failed(); +} + +// Returns true iff test_case contains at least one test that should +// run. +static bool ShouldRunTestCase(const TestCase* test_case) { + return test_case->should_run(); +} + +// AssertHelper constructor. +AssertHelper::AssertHelper(TestPartResult::Type type, + const char* file, + int line, + const char* message) + : data_(new AssertHelperData(type, file, line, message)) { +} + +AssertHelper::~AssertHelper() { + delete data_; +} + +// Message assignment, for assertion streaming support. +void AssertHelper::operator=(const Message& message) const { + UnitTest::GetInstance()-> + AddTestPartResult(data_->type, data_->file, data_->line, + AppendUserMessage(data_->message, message), + UnitTest::GetInstance()->impl() + ->CurrentOsStackTraceExceptTop(1) + // Skips the stack frame for this function itself. + ); // NOLINT +} + +// Mutex for linked pointers. +GTEST_API_ GTEST_DEFINE_STATIC_MUTEX_(g_linked_ptr_mutex); + +// Application pathname gotten in InitGoogleTest. +std::string g_executable_path; + +// Returns the current application's name, removing directory path if that +// is present. +FilePath GetCurrentExecutableName() { + FilePath result; + +#if GTEST_OS_WINDOWS + result.Set(FilePath(g_executable_path).RemoveExtension("exe")); +#else + result.Set(FilePath(g_executable_path)); +#endif // GTEST_OS_WINDOWS + + return result.RemoveDirectoryName(); +} + +// Functions for processing the gtest_output flag. + +// Returns the output format, or "" for normal printed output. +std::string UnitTestOptions::GetOutputFormat() { + const char* const gtest_output_flag = GTEST_FLAG(output).c_str(); + if (gtest_output_flag == NULL) return std::string(""); + + const char* const colon = strchr(gtest_output_flag, ':'); + return (colon == NULL) ? + std::string(gtest_output_flag) : + std::string(gtest_output_flag, colon - gtest_output_flag); +} + +// Returns the name of the requested output file, or the default if none +// was explicitly specified. +std::string UnitTestOptions::GetAbsolutePathToOutputFile() { + const char* const gtest_output_flag = GTEST_FLAG(output).c_str(); + if (gtest_output_flag == NULL) + return ""; + + const char* const colon = strchr(gtest_output_flag, ':'); + if (colon == NULL) + return internal::FilePath::ConcatPaths( + internal::FilePath( + UnitTest::GetInstance()->original_working_dir()), + internal::FilePath(kDefaultOutputFile)).string(); + + internal::FilePath output_name(colon + 1); + if (!output_name.IsAbsolutePath()) + // TODO(wan@google.com): on Windows \some\path is not an absolute + // path (as its meaning depends on the current drive), yet the + // following logic for turning it into an absolute path is wrong. + // Fix it. + output_name = internal::FilePath::ConcatPaths( + internal::FilePath(UnitTest::GetInstance()->original_working_dir()), + internal::FilePath(colon + 1)); + + if (!output_name.IsDirectory()) + return output_name.string(); + + internal::FilePath result(internal::FilePath::GenerateUniqueFileName( + output_name, internal::GetCurrentExecutableName(), + GetOutputFormat().c_str())); + return result.string(); +} + +// Returns true iff the wildcard pattern matches the string. The +// first ':' or '\0' character in pattern marks the end of it. +// +// This recursive algorithm isn't very efficient, but is clear and +// works well enough for matching test names, which are short. +bool UnitTestOptions::PatternMatchesString(const char *pattern, + const char *str) { + switch (*pattern) { + case '\0': + case ':': // Either ':' or '\0' marks the end of the pattern. + return *str == '\0'; + case '?': // Matches any single character. + return *str != '\0' && PatternMatchesString(pattern + 1, str + 1); + case '*': // Matches any string (possibly empty) of characters. + return (*str != '\0' && PatternMatchesString(pattern, str + 1)) || + PatternMatchesString(pattern + 1, str); + default: // Non-special character. Matches itself. + return *pattern == *str && + PatternMatchesString(pattern + 1, str + 1); + } +} + +bool UnitTestOptions::MatchesFilter( + const std::string& name, const char* filter) { + const char *cur_pattern = filter; + for (;;) { + if (PatternMatchesString(cur_pattern, name.c_str())) { + return true; + } + + // Finds the next pattern in the filter. + cur_pattern = strchr(cur_pattern, ':'); + + // Returns if no more pattern can be found. + if (cur_pattern == NULL) { + return false; + } + + // Skips the pattern separater (the ':' character). + cur_pattern++; + } +} + +// Returns true iff the user-specified filter matches the test case +// name and the test name. +bool UnitTestOptions::FilterMatchesTest(const std::string &test_case_name, + const std::string &test_name) { + const std::string& full_name = test_case_name + "." + test_name.c_str(); + + // Split --gtest_filter at '-', if there is one, to separate into + // positive filter and negative filter portions + const char* const p = GTEST_FLAG(filter).c_str(); + const char* const dash = strchr(p, '-'); + std::string positive; + std::string negative; + if (dash == NULL) { + positive = GTEST_FLAG(filter).c_str(); // Whole string is a positive filter + negative = ""; + } else { + positive = std::string(p, dash); // Everything up to the dash + negative = std::string(dash + 1); // Everything after the dash + if (positive.empty()) { + // Treat '-test1' as the same as '*-test1' + positive = kUniversalFilter; + } + } + + // A filter is a colon-separated list of patterns. It matches a + // test if any pattern in it matches the test. + return (MatchesFilter(full_name, positive.c_str()) && + !MatchesFilter(full_name, negative.c_str())); +} + +#if GTEST_HAS_SEH +// Returns EXCEPTION_EXECUTE_HANDLER if Google Test should handle the +// given SEH exception, or EXCEPTION_CONTINUE_SEARCH otherwise. +// This function is useful as an __except condition. +int UnitTestOptions::GTestShouldProcessSEH(DWORD exception_code) { + // Google Test should handle a SEH exception if: + // 1. the user wants it to, AND + // 2. this is not a breakpoint exception, AND + // 3. this is not a C++ exception (VC++ implements them via SEH, + // apparently). + // + // SEH exception code for C++ exceptions. + // (see http://support.microsoft.com/kb/185294 for more information). + const DWORD kCxxExceptionCode = 0xe06d7363; + + bool should_handle = true; + + if (!GTEST_FLAG(catch_exceptions)) + should_handle = false; + else if (exception_code == EXCEPTION_BREAKPOINT) + should_handle = false; + else if (exception_code == kCxxExceptionCode) + should_handle = false; + + return should_handle ? EXCEPTION_EXECUTE_HANDLER : EXCEPTION_CONTINUE_SEARCH; +} +#endif // GTEST_HAS_SEH + +} // namespace internal + +// The c'tor sets this object as the test part result reporter used by +// Google Test. The 'result' parameter specifies where to report the +// results. Intercepts only failures from the current thread. +ScopedFakeTestPartResultReporter::ScopedFakeTestPartResultReporter( + TestPartResultArray* result) + : intercept_mode_(INTERCEPT_ONLY_CURRENT_THREAD), + result_(result) { + Init(); +} + +// The c'tor sets this object as the test part result reporter used by +// Google Test. The 'result' parameter specifies where to report the +// results. +ScopedFakeTestPartResultReporter::ScopedFakeTestPartResultReporter( + InterceptMode intercept_mode, TestPartResultArray* result) + : intercept_mode_(intercept_mode), + result_(result) { + Init(); +} + +void ScopedFakeTestPartResultReporter::Init() { + internal::UnitTestImpl* const impl = internal::GetUnitTestImpl(); + if (intercept_mode_ == INTERCEPT_ALL_THREADS) { + old_reporter_ = impl->GetGlobalTestPartResultReporter(); + impl->SetGlobalTestPartResultReporter(this); + } else { + old_reporter_ = impl->GetTestPartResultReporterForCurrentThread(); + impl->SetTestPartResultReporterForCurrentThread(this); + } +} + +// The d'tor restores the test part result reporter used by Google Test +// before. +ScopedFakeTestPartResultReporter::~ScopedFakeTestPartResultReporter() { + internal::UnitTestImpl* const impl = internal::GetUnitTestImpl(); + if (intercept_mode_ == INTERCEPT_ALL_THREADS) { + impl->SetGlobalTestPartResultReporter(old_reporter_); + } else { + impl->SetTestPartResultReporterForCurrentThread(old_reporter_); + } +} + +// Increments the test part result count and remembers the result. +// This method is from the TestPartResultReporterInterface interface. +void ScopedFakeTestPartResultReporter::ReportTestPartResult( + const TestPartResult& result) { + result_->Append(result); +} + +namespace internal { + +// Returns the type ID of ::testing::Test. We should always call this +// instead of GetTypeId< ::testing::Test>() to get the type ID of +// testing::Test. This is to work around a suspected linker bug when +// using Google Test as a framework on Mac OS X. The bug causes +// GetTypeId< ::testing::Test>() to return different values depending +// on whether the call is from the Google Test framework itself or +// from user test code. GetTestTypeId() is guaranteed to always +// return the same value, as it always calls GetTypeId<>() from the +// gtest.cc, which is within the Google Test framework. +TypeId GetTestTypeId() { + return GetTypeId(); +} + +// The value of GetTestTypeId() as seen from within the Google Test +// library. This is solely for testing GetTestTypeId(). +extern const TypeId kTestTypeIdInGoogleTest = GetTestTypeId(); + +// This predicate-formatter checks that 'results' contains a test part +// failure of the given type and that the failure message contains the +// given substring. +AssertionResult HasOneFailure(const char* /* results_expr */, + const char* /* type_expr */, + const char* /* substr_expr */, + const TestPartResultArray& results, + TestPartResult::Type type, + const string& substr) { + const std::string expected(type == TestPartResult::kFatalFailure ? + "1 fatal failure" : + "1 non-fatal failure"); + Message msg; + if (results.size() != 1) { + msg << "Expected: " << expected << "\n" + << " Actual: " << results.size() << " failures"; + for (int i = 0; i < results.size(); i++) { + msg << "\n" << results.GetTestPartResult(i); + } + return AssertionFailure() << msg; + } + + const TestPartResult& r = results.GetTestPartResult(0); + if (r.type() != type) { + return AssertionFailure() << "Expected: " << expected << "\n" + << " Actual:\n" + << r; + } + + if (strstr(r.message(), substr.c_str()) == NULL) { + return AssertionFailure() << "Expected: " << expected << " containing \"" + << substr << "\"\n" + << " Actual:\n" + << r; + } + + return AssertionSuccess(); +} + +// The constructor of SingleFailureChecker remembers where to look up +// test part results, what type of failure we expect, and what +// substring the failure message should contain. +SingleFailureChecker:: SingleFailureChecker( + const TestPartResultArray* results, + TestPartResult::Type type, + const string& substr) + : results_(results), + type_(type), + substr_(substr) {} + +// The destructor of SingleFailureChecker verifies that the given +// TestPartResultArray contains exactly one failure that has the given +// type and contains the given substring. If that's not the case, a +// non-fatal failure will be generated. +SingleFailureChecker::~SingleFailureChecker() { + EXPECT_PRED_FORMAT3(HasOneFailure, *results_, type_, substr_); +} + +DefaultGlobalTestPartResultReporter::DefaultGlobalTestPartResultReporter( + UnitTestImpl* unit_test) : unit_test_(unit_test) {} + +void DefaultGlobalTestPartResultReporter::ReportTestPartResult( + const TestPartResult& result) { + unit_test_->current_test_result()->AddTestPartResult(result); + unit_test_->listeners()->repeater()->OnTestPartResult(result); +} + +DefaultPerThreadTestPartResultReporter::DefaultPerThreadTestPartResultReporter( + UnitTestImpl* unit_test) : unit_test_(unit_test) {} + +void DefaultPerThreadTestPartResultReporter::ReportTestPartResult( + const TestPartResult& result) { + unit_test_->GetGlobalTestPartResultReporter()->ReportTestPartResult(result); +} + +// Returns the global test part result reporter. +TestPartResultReporterInterface* +UnitTestImpl::GetGlobalTestPartResultReporter() { + internal::MutexLock lock(&global_test_part_result_reporter_mutex_); + return global_test_part_result_repoter_; +} + +// Sets the global test part result reporter. +void UnitTestImpl::SetGlobalTestPartResultReporter( + TestPartResultReporterInterface* reporter) { + internal::MutexLock lock(&global_test_part_result_reporter_mutex_); + global_test_part_result_repoter_ = reporter; +} + +// Returns the test part result reporter for the current thread. +TestPartResultReporterInterface* +UnitTestImpl::GetTestPartResultReporterForCurrentThread() { + return per_thread_test_part_result_reporter_.get(); +} + +// Sets the test part result reporter for the current thread. +void UnitTestImpl::SetTestPartResultReporterForCurrentThread( + TestPartResultReporterInterface* reporter) { + per_thread_test_part_result_reporter_.set(reporter); +} + +// Gets the number of successful test cases. +int UnitTestImpl::successful_test_case_count() const { + return CountIf(test_cases_, TestCasePassed); +} + +// Gets the number of failed test cases. +int UnitTestImpl::failed_test_case_count() const { + return CountIf(test_cases_, TestCaseFailed); +} + +// Gets the number of all test cases. +int UnitTestImpl::total_test_case_count() const { + return static_cast(test_cases_.size()); +} + +// Gets the number of all test cases that contain at least one test +// that should run. +int UnitTestImpl::test_case_to_run_count() const { + return CountIf(test_cases_, ShouldRunTestCase); +} + +// Gets the number of successful tests. +int UnitTestImpl::successful_test_count() const { + return SumOverTestCaseList(test_cases_, &TestCase::successful_test_count); +} + +// Gets the number of failed tests. +int UnitTestImpl::failed_test_count() const { + return SumOverTestCaseList(test_cases_, &TestCase::failed_test_count); +} + +// Gets the number of disabled tests that will be reported in the XML report. +int UnitTestImpl::reportable_disabled_test_count() const { + return SumOverTestCaseList(test_cases_, + &TestCase::reportable_disabled_test_count); +} + +// Gets the number of disabled tests. +int UnitTestImpl::disabled_test_count() const { + return SumOverTestCaseList(test_cases_, &TestCase::disabled_test_count); +} + +// Gets the number of tests to be printed in the XML report. +int UnitTestImpl::reportable_test_count() const { + return SumOverTestCaseList(test_cases_, &TestCase::reportable_test_count); +} + +// Gets the number of all tests. +int UnitTestImpl::total_test_count() const { + return SumOverTestCaseList(test_cases_, &TestCase::total_test_count); +} + +// Gets the number of tests that should run. +int UnitTestImpl::test_to_run_count() const { + return SumOverTestCaseList(test_cases_, &TestCase::test_to_run_count); +} + +// Returns the current OS stack trace as an std::string. +// +// The maximum number of stack frames to be included is specified by +// the gtest_stack_trace_depth flag. The skip_count parameter +// specifies the number of top frames to be skipped, which doesn't +// count against the number of frames to be included. +// +// For example, if Foo() calls Bar(), which in turn calls +// CurrentOsStackTraceExceptTop(1), Foo() will be included in the +// trace but Bar() and CurrentOsStackTraceExceptTop() won't. +std::string UnitTestImpl::CurrentOsStackTraceExceptTop(int skip_count) { + (void)skip_count; + return ""; +} + +// Returns the current time in milliseconds. +TimeInMillis GetTimeInMillis() { +#if GTEST_OS_WINDOWS_MOBILE || defined(__BORLANDC__) + // Difference between 1970-01-01 and 1601-01-01 in milliseconds. + // http://analogous.blogspot.com/2005/04/epoch.html + const TimeInMillis kJavaEpochToWinFileTimeDelta = + static_cast(116444736UL) * 100000UL; + const DWORD kTenthMicrosInMilliSecond = 10000; + + SYSTEMTIME now_systime; + FILETIME now_filetime; + ULARGE_INTEGER now_int64; + // TODO(kenton@google.com): Shouldn't this just use + // GetSystemTimeAsFileTime()? + GetSystemTime(&now_systime); + if (SystemTimeToFileTime(&now_systime, &now_filetime)) { + now_int64.LowPart = now_filetime.dwLowDateTime; + now_int64.HighPart = now_filetime.dwHighDateTime; + now_int64.QuadPart = (now_int64.QuadPart / kTenthMicrosInMilliSecond) - + kJavaEpochToWinFileTimeDelta; + return now_int64.QuadPart; + } + return 0; +#elif GTEST_OS_WINDOWS && !GTEST_HAS_GETTIMEOFDAY_ + __timeb64 now; + + // MSVC 8 deprecates _ftime64(), so we want to suppress warning 4996 + // (deprecated function) there. + // TODO(kenton@google.com): Use GetTickCount()? Or use + // SystemTimeToFileTime() + GTEST_DISABLE_MSC_WARNINGS_PUSH_(4996) + _ftime64(&now); + GTEST_DISABLE_MSC_WARNINGS_POP_() + + return static_cast(now.time) * 1000 + now.millitm; +#elif GTEST_HAS_GETTIMEOFDAY_ + struct timeval now; + gettimeofday(&now, NULL); + return static_cast(now.tv_sec) * 1000 + now.tv_usec / 1000; +#else +# error "Don't know how to get the current time on your system." +#endif +} + +// Utilities + +// class String. + +#if GTEST_OS_WINDOWS_MOBILE +// Creates a UTF-16 wide string from the given ANSI string, allocating +// memory using new. The caller is responsible for deleting the return +// value using delete[]. Returns the wide string, or NULL if the +// input is NULL. +LPCWSTR String::AnsiToUtf16(const char* ansi) { + if (!ansi) return NULL; + const int length = strlen(ansi); + const int unicode_length = + MultiByteToWideChar(CP_ACP, 0, ansi, length, + NULL, 0); + WCHAR* unicode = new WCHAR[unicode_length + 1]; + MultiByteToWideChar(CP_ACP, 0, ansi, length, + unicode, unicode_length); + unicode[unicode_length] = 0; + return unicode; +} + +// Creates an ANSI string from the given wide string, allocating +// memory using new. The caller is responsible for deleting the return +// value using delete[]. Returns the ANSI string, or NULL if the +// input is NULL. +const char* String::Utf16ToAnsi(LPCWSTR utf16_str) { + if (!utf16_str) return NULL; + const int ansi_length = + WideCharToMultiByte(CP_ACP, 0, utf16_str, -1, + NULL, 0, NULL, NULL); + char* ansi = new char[ansi_length + 1]; + WideCharToMultiByte(CP_ACP, 0, utf16_str, -1, + ansi, ansi_length, NULL, NULL); + ansi[ansi_length] = 0; + return ansi; +} + +#endif // GTEST_OS_WINDOWS_MOBILE + +// Compares two C strings. Returns true iff they have the same content. +// +// Unlike strcmp(), this function can handle NULL argument(s). A NULL +// C string is considered different to any non-NULL C string, +// including the empty string. +bool String::CStringEquals(const char * lhs, const char * rhs) { + if ( lhs == NULL ) return rhs == NULL; + + if ( rhs == NULL ) return false; + + return strcmp(lhs, rhs) == 0; +} + +#if GTEST_HAS_STD_WSTRING || GTEST_HAS_GLOBAL_WSTRING + +// Converts an array of wide chars to a narrow string using the UTF-8 +// encoding, and streams the result to the given Message object. +static void StreamWideCharsToMessage(const wchar_t* wstr, size_t length, + Message* msg) { + for (size_t i = 0; i != length; ) { // NOLINT + if (wstr[i] != L'\0') { + *msg << WideStringToUtf8(wstr + i, static_cast(length - i)); + while (i != length && wstr[i] != L'\0') + i++; + } else { + *msg << '\0'; + i++; + } + } +} + +#endif // GTEST_HAS_STD_WSTRING || GTEST_HAS_GLOBAL_WSTRING + +} // namespace internal + +// Constructs an empty Message. +// We allocate the stringstream separately because otherwise each use of +// ASSERT/EXPECT in a procedure adds over 200 bytes to the procedure's +// stack frame leading to huge stack frames in some cases; gcc does not reuse +// the stack space. +Message::Message() : ss_(new ::std::stringstream) { + // By default, we want there to be enough precision when printing + // a double to a Message. + *ss_ << std::setprecision(std::numeric_limits::digits10 + 2); +} + +// These two overloads allow streaming a wide C string to a Message +// using the UTF-8 encoding. +Message& Message::operator <<(const wchar_t* wide_c_str) { + return *this << internal::String::ShowWideCString(wide_c_str); +} +Message& Message::operator <<(wchar_t* wide_c_str) { + return *this << internal::String::ShowWideCString(wide_c_str); +} + +#if GTEST_HAS_STD_WSTRING +// Converts the given wide string to a narrow string using the UTF-8 +// encoding, and streams the result to this Message object. +Message& Message::operator <<(const ::std::wstring& wstr) { + internal::StreamWideCharsToMessage(wstr.c_str(), wstr.length(), this); + return *this; +} +#endif // GTEST_HAS_STD_WSTRING + +#if GTEST_HAS_GLOBAL_WSTRING +// Converts the given wide string to a narrow string using the UTF-8 +// encoding, and streams the result to this Message object. +Message& Message::operator <<(const ::wstring& wstr) { + internal::StreamWideCharsToMessage(wstr.c_str(), wstr.length(), this); + return *this; +} +#endif // GTEST_HAS_GLOBAL_WSTRING + +// Gets the text streamed to this object so far as an std::string. +// Each '\0' character in the buffer is replaced with "\\0". +std::string Message::GetString() const { + return internal::StringStreamToString(ss_.get()); +} + +// AssertionResult constructors. +// Used in EXPECT_TRUE/FALSE(assertion_result). +AssertionResult::AssertionResult(const AssertionResult& other) + : success_(other.success_), + message_(other.message_.get() != NULL ? + new ::std::string(*other.message_) : + static_cast< ::std::string*>(NULL)) { +} + +// Swaps two AssertionResults. +void AssertionResult::swap(AssertionResult& other) { + using std::swap; + swap(success_, other.success_); + swap(message_, other.message_); +} + +// Returns the assertion's negation. Used with EXPECT/ASSERT_FALSE. +AssertionResult AssertionResult::operator!() const { + AssertionResult negation(!success_); + if (message_.get() != NULL) + negation << *message_; + return negation; +} + +// Makes a successful assertion result. +AssertionResult AssertionSuccess() { + return AssertionResult(true); +} + +// Makes a failed assertion result. +AssertionResult AssertionFailure() { + return AssertionResult(false); +} + +// Makes a failed assertion result with the given failure message. +// Deprecated; use AssertionFailure() << message. +AssertionResult AssertionFailure(const Message& message) { + return AssertionFailure() << message; +} + +namespace internal { + +namespace edit_distance { +std::vector CalculateOptimalEdits(const std::vector& left, + const std::vector& right) { + std::vector > costs( + left.size() + 1, std::vector(right.size() + 1)); + std::vector > best_move( + left.size() + 1, std::vector(right.size() + 1)); + + // Populate for empty right. + for (size_t l_i = 0; l_i < costs.size(); ++l_i) { + costs[l_i][0] = static_cast(l_i); + best_move[l_i][0] = kRemove; + } + // Populate for empty left. + for (size_t r_i = 1; r_i < costs[0].size(); ++r_i) { + costs[0][r_i] = static_cast(r_i); + best_move[0][r_i] = kAdd; + } + + for (size_t l_i = 0; l_i < left.size(); ++l_i) { + for (size_t r_i = 0; r_i < right.size(); ++r_i) { + if (left[l_i] == right[r_i]) { + // Found a match. Consume it. + costs[l_i + 1][r_i + 1] = costs[l_i][r_i]; + best_move[l_i + 1][r_i + 1] = kMatch; + continue; + } + + const double add = costs[l_i + 1][r_i]; + const double remove = costs[l_i][r_i + 1]; + const double replace = costs[l_i][r_i]; + if (add < remove && add < replace) { + costs[l_i + 1][r_i + 1] = add + 1; + best_move[l_i + 1][r_i + 1] = kAdd; + } else if (remove < add && remove < replace) { + costs[l_i + 1][r_i + 1] = remove + 1; + best_move[l_i + 1][r_i + 1] = kRemove; + } else { + // We make replace a little more expensive than add/remove to lower + // their priority. + costs[l_i + 1][r_i + 1] = replace + 1.00001; + best_move[l_i + 1][r_i + 1] = kReplace; + } + } + } + + // Reconstruct the best path. We do it in reverse order. + std::vector best_path; + for (size_t l_i = left.size(), r_i = right.size(); l_i > 0 || r_i > 0;) { + EditType move = best_move[l_i][r_i]; + best_path.push_back(move); + l_i -= move != kAdd; + r_i -= move != kRemove; + } + std::reverse(best_path.begin(), best_path.end()); + return best_path; +} + +namespace { + +// Helper class to convert string into ids with deduplication. +class InternalStrings { + public: + size_t GetId(const std::string& str) { + IdMap::iterator it = ids_.find(str); + if (it != ids_.end()) return it->second; + size_t id = ids_.size(); + return ids_[str] = id; + } + + private: + typedef std::map IdMap; + IdMap ids_; +}; + +} // namespace + +std::vector CalculateOptimalEdits( + const std::vector& left, + const std::vector& right) { + std::vector left_ids, right_ids; + { + InternalStrings intern_table; + for (size_t i = 0; i < left.size(); ++i) { + left_ids.push_back(intern_table.GetId(left[i])); + } + for (size_t i = 0; i < right.size(); ++i) { + right_ids.push_back(intern_table.GetId(right[i])); + } + } + return CalculateOptimalEdits(left_ids, right_ids); +} + +namespace { + +// Helper class that holds the state for one hunk and prints it out to the +// stream. +// It reorders adds/removes when possible to group all removes before all +// adds. It also adds the hunk header before printint into the stream. +class Hunk { + public: + Hunk(size_t left_start, size_t right_start) + : left_start_(left_start), + right_start_(right_start), + adds_(), + removes_(), + common_() {} + + void PushLine(char edit, const char* line) { + switch (edit) { + case ' ': + ++common_; + FlushEdits(); + hunk_.push_back(std::make_pair(' ', line)); + break; + case '-': + ++removes_; + hunk_removes_.push_back(std::make_pair('-', line)); + break; + case '+': + ++adds_; + hunk_adds_.push_back(std::make_pair('+', line)); + break; + } + } + + void PrintTo(std::ostream* os) { + PrintHeader(os); + FlushEdits(); + for (std::list >::const_iterator it = + hunk_.begin(); + it != hunk_.end(); ++it) { + *os << it->first << it->second << "\n"; + } + } + + bool has_edits() const { return adds_ || removes_; } + + private: + void FlushEdits() { + hunk_.splice(hunk_.end(), hunk_removes_); + hunk_.splice(hunk_.end(), hunk_adds_); + } + + // Print a unified diff header for one hunk. + // The format is + // "@@ -, +, @@" + // where the left/right parts are omitted if unnecessary. + void PrintHeader(std::ostream* ss) const { + *ss << "@@ "; + if (removes_) { + *ss << "-" << left_start_ << "," << (removes_ + common_); + } + if (removes_ && adds_) { + *ss << " "; + } + if (adds_) { + *ss << "+" << right_start_ << "," << (adds_ + common_); + } + *ss << " @@\n"; + } + + size_t left_start_, right_start_; + size_t adds_, removes_, common_; + std::list > hunk_, hunk_adds_, hunk_removes_; +}; + +} // namespace + +// Create a list of diff hunks in Unified diff format. +// Each hunk has a header generated by PrintHeader above plus a body with +// lines prefixed with ' ' for no change, '-' for deletion and '+' for +// addition. +// 'context' represents the desired unchanged prefix/suffix around the diff. +// If two hunks are close enough that their contexts overlap, then they are +// joined into one hunk. +std::string CreateUnifiedDiff(const std::vector& left, + const std::vector& right, + size_t context) { + const std::vector edits = CalculateOptimalEdits(left, right); + + size_t l_i = 0, r_i = 0, edit_i = 0; + std::stringstream ss; + while (edit_i < edits.size()) { + // Find first edit. + while (edit_i < edits.size() && edits[edit_i] == kMatch) { + ++l_i; + ++r_i; + ++edit_i; + } + + // Find the first line to include in the hunk. + const size_t prefix_context = std::min(l_i, context); + Hunk hunk(l_i - prefix_context + 1, r_i - prefix_context + 1); + for (size_t i = prefix_context; i > 0; --i) { + hunk.PushLine(' ', left[l_i - i].c_str()); + } + + // Iterate the edits until we found enough suffix for the hunk or the input + // is over. + size_t n_suffix = 0; + for (; edit_i < edits.size(); ++edit_i) { + if (n_suffix >= context) { + // Continue only if the next hunk is very close. + std::vector::const_iterator it = edits.begin() + edit_i; + while (it != edits.end() && *it == kMatch) ++it; + if (it == edits.end() || (it - edits.begin()) - edit_i >= context) { + // There is no next edit or it is too far away. + break; + } + } + + EditType edit = edits[edit_i]; + // Reset count when a non match is found. + n_suffix = edit == kMatch ? n_suffix + 1 : 0; + + if (edit == kMatch || edit == kRemove || edit == kReplace) { + hunk.PushLine(edit == kMatch ? ' ' : '-', left[l_i].c_str()); + } + if (edit == kAdd || edit == kReplace) { + hunk.PushLine('+', right[r_i].c_str()); + } + + // Advance indices, depending on edit type. + l_i += edit != kAdd; + r_i += edit != kRemove; + } + + if (!hunk.has_edits()) { + // We are done. We don't want this hunk. + break; + } + + hunk.PrintTo(&ss); + } + return ss.str(); +} + +} // namespace edit_distance + +namespace { + +// The string representation of the values received in EqFailure() are already +// escaped. Split them on escaped '\n' boundaries. Leave all other escaped +// characters the same. +std::vector SplitEscapedString(const std::string& str) { + std::vector lines; + size_t start = 0, end = str.size(); + if (end > 2 && str[0] == '"' && str[end - 1] == '"') { + ++start; + --end; + } + bool escaped = false; + for (size_t i = start; i + 1 < end; ++i) { + if (escaped) { + escaped = false; + if (str[i] == 'n') { + lines.push_back(str.substr(start, i - start - 1)); + start = i + 1; + } + } else { + escaped = str[i] == '\\'; + } + } + lines.push_back(str.substr(start, end - start)); + return lines; +} + +} // namespace + +// Constructs and returns the message for an equality assertion +// (e.g. ASSERT_EQ, EXPECT_STREQ, etc) failure. +// +// The first four parameters are the expressions used in the assertion +// and their values, as strings. For example, for ASSERT_EQ(foo, bar) +// where foo is 5 and bar is 6, we have: +// +// expected_expression: "foo" +// actual_expression: "bar" +// expected_value: "5" +// actual_value: "6" +// +// The ignoring_case parameter is true iff the assertion is a +// *_STRCASEEQ*. When it's true, the string " (ignoring case)" will +// be inserted into the message. +AssertionResult EqFailure(const char* expected_expression, + const char* actual_expression, + const std::string& expected_value, + const std::string& actual_value, + bool ignoring_case) { + Message msg; + msg << "Value of: " << actual_expression; + if (actual_value != actual_expression) { + msg << "\n Actual: " << actual_value; + } + + msg << "\nExpected: " << expected_expression; + if (ignoring_case) { + msg << " (ignoring case)"; + } + if (expected_value != expected_expression) { + msg << "\nWhich is: " << expected_value; + } + + if (!expected_value.empty() && !actual_value.empty()) { + const std::vector expected_lines = + SplitEscapedString(expected_value); + const std::vector actual_lines = + SplitEscapedString(actual_value); + if (expected_lines.size() > 1 || actual_lines.size() > 1) { + msg << "\nWith diff:\n" + << edit_distance::CreateUnifiedDiff(expected_lines, actual_lines); + } + } + + return AssertionFailure() << msg; +} + +// Constructs a failure message for Boolean assertions such as EXPECT_TRUE. +std::string GetBoolAssertionFailureMessage( + const AssertionResult& assertion_result, + const char* expression_text, + const char* actual_predicate_value, + const char* expected_predicate_value) { + const char* actual_message = assertion_result.message(); + Message msg; + msg << "Value of: " << expression_text + << "\n Actual: " << actual_predicate_value; + if (actual_message[0] != '\0') + msg << " (" << actual_message << ")"; + msg << "\nExpected: " << expected_predicate_value; + return msg.GetString(); +} + +// Helper function for implementing ASSERT_NEAR. +AssertionResult DoubleNearPredFormat(const char* expr1, + const char* expr2, + const char* abs_error_expr, + double val1, + double val2, + double abs_error) { + const double diff = fabs(val1 - val2); + if (diff <= abs_error) return AssertionSuccess(); + + // TODO(wan): do not print the value of an expression if it's + // already a literal. + return AssertionFailure() + << "The difference between " << expr1 << " and " << expr2 + << " is " << diff << ", which exceeds " << abs_error_expr << ", where\n" + << expr1 << " evaluates to " << val1 << ",\n" + << expr2 << " evaluates to " << val2 << ", and\n" + << abs_error_expr << " evaluates to " << abs_error << "."; +} + + +// Helper template for implementing FloatLE() and DoubleLE(). +template +AssertionResult FloatingPointLE(const char* expr1, + const char* expr2, + RawType val1, + RawType val2) { + // Returns success if val1 is less than val2, + if (val1 < val2) { + return AssertionSuccess(); + } + + // or if val1 is almost equal to val2. + const FloatingPoint lhs(val1), rhs(val2); + if (lhs.AlmostEquals(rhs)) { + return AssertionSuccess(); + } + + // Note that the above two checks will both fail if either val1 or + // val2 is NaN, as the IEEE floating-point standard requires that + // any predicate involving a NaN must return false. + + ::std::stringstream val1_ss; + val1_ss << std::setprecision(std::numeric_limits::digits10 + 2) + << val1; + + ::std::stringstream val2_ss; + val2_ss << std::setprecision(std::numeric_limits::digits10 + 2) + << val2; + + return AssertionFailure() + << "Expected: (" << expr1 << ") <= (" << expr2 << ")\n" + << " Actual: " << StringStreamToString(&val1_ss) << " vs " + << StringStreamToString(&val2_ss); +} + +} // namespace internal + +// Asserts that val1 is less than, or almost equal to, val2. Fails +// otherwise. In particular, it fails if either val1 or val2 is NaN. +AssertionResult FloatLE(const char* expr1, const char* expr2, + float val1, float val2) { + return internal::FloatingPointLE(expr1, expr2, val1, val2); +} + +// Asserts that val1 is less than, or almost equal to, val2. Fails +// otherwise. In particular, it fails if either val1 or val2 is NaN. +AssertionResult DoubleLE(const char* expr1, const char* expr2, + double val1, double val2) { + return internal::FloatingPointLE(expr1, expr2, val1, val2); +} + +namespace internal { + +// The helper function for {ASSERT|EXPECT}_EQ with int or enum +// arguments. +AssertionResult CmpHelperEQ(const char* expected_expression, + const char* actual_expression, + BiggestInt expected, + BiggestInt actual) { + if (expected == actual) { + return AssertionSuccess(); + } + + return EqFailure(expected_expression, + actual_expression, + FormatForComparisonFailureMessage(expected, actual), + FormatForComparisonFailureMessage(actual, expected), + false); +} + +// A macro for implementing the helper functions needed to implement +// ASSERT_?? and EXPECT_?? with integer or enum arguments. It is here +// just to avoid copy-and-paste of similar code. +#define GTEST_IMPL_CMP_HELPER_(op_name, op)\ +AssertionResult CmpHelper##op_name(const char* expr1, const char* expr2, \ + BiggestInt val1, BiggestInt val2) {\ + if (val1 op val2) {\ + return AssertionSuccess();\ + } else {\ + return AssertionFailure() \ + << "Expected: (" << expr1 << ") " #op " (" << expr2\ + << "), actual: " << FormatForComparisonFailureMessage(val1, val2)\ + << " vs " << FormatForComparisonFailureMessage(val2, val1);\ + }\ +} + +// Implements the helper function for {ASSERT|EXPECT}_NE with int or +// enum arguments. +GTEST_IMPL_CMP_HELPER_(NE, !=) +// Implements the helper function for {ASSERT|EXPECT}_LE with int or +// enum arguments. +GTEST_IMPL_CMP_HELPER_(LE, <=) +// Implements the helper function for {ASSERT|EXPECT}_LT with int or +// enum arguments. +GTEST_IMPL_CMP_HELPER_(LT, < ) +// Implements the helper function for {ASSERT|EXPECT}_GE with int or +// enum arguments. +GTEST_IMPL_CMP_HELPER_(GE, >=) +// Implements the helper function for {ASSERT|EXPECT}_GT with int or +// enum arguments. +GTEST_IMPL_CMP_HELPER_(GT, > ) + +#undef GTEST_IMPL_CMP_HELPER_ + +// The helper function for {ASSERT|EXPECT}_STREQ. +AssertionResult CmpHelperSTREQ(const char* expected_expression, + const char* actual_expression, + const char* expected, + const char* actual) { + if (String::CStringEquals(expected, actual)) { + return AssertionSuccess(); + } + + return EqFailure(expected_expression, + actual_expression, + PrintToString(expected), + PrintToString(actual), + false); +} + +// The helper function for {ASSERT|EXPECT}_STRCASEEQ. +AssertionResult CmpHelperSTRCASEEQ(const char* expected_expression, + const char* actual_expression, + const char* expected, + const char* actual) { + if (String::CaseInsensitiveCStringEquals(expected, actual)) { + return AssertionSuccess(); + } + + return EqFailure(expected_expression, + actual_expression, + PrintToString(expected), + PrintToString(actual), + true); +} + +// The helper function for {ASSERT|EXPECT}_STRNE. +AssertionResult CmpHelperSTRNE(const char* s1_expression, + const char* s2_expression, + const char* s1, + const char* s2) { + if (!String::CStringEquals(s1, s2)) { + return AssertionSuccess(); + } else { + return AssertionFailure() << "Expected: (" << s1_expression << ") != (" + << s2_expression << "), actual: \"" + << s1 << "\" vs \"" << s2 << "\""; + } +} + +// The helper function for {ASSERT|EXPECT}_STRCASENE. +AssertionResult CmpHelperSTRCASENE(const char* s1_expression, + const char* s2_expression, + const char* s1, + const char* s2) { + if (!String::CaseInsensitiveCStringEquals(s1, s2)) { + return AssertionSuccess(); + } else { + return AssertionFailure() + << "Expected: (" << s1_expression << ") != (" + << s2_expression << ") (ignoring case), actual: \"" + << s1 << "\" vs \"" << s2 << "\""; + } +} + +} // namespace internal + +namespace { + +// Helper functions for implementing IsSubString() and IsNotSubstring(). + +// This group of overloaded functions return true iff needle is a +// substring of haystack. NULL is considered a substring of itself +// only. + +bool IsSubstringPred(const char* needle, const char* haystack) { + if (needle == NULL || haystack == NULL) + return needle == haystack; + + return strstr(haystack, needle) != NULL; +} + +bool IsSubstringPred(const wchar_t* needle, const wchar_t* haystack) { + if (needle == NULL || haystack == NULL) + return needle == haystack; + + return wcsstr(haystack, needle) != NULL; +} + +// StringType here can be either ::std::string or ::std::wstring. +template +bool IsSubstringPred(const StringType& needle, + const StringType& haystack) { + return haystack.find(needle) != StringType::npos; +} + +// This function implements either IsSubstring() or IsNotSubstring(), +// depending on the value of the expected_to_be_substring parameter. +// StringType here can be const char*, const wchar_t*, ::std::string, +// or ::std::wstring. +template +AssertionResult IsSubstringImpl( + bool expected_to_be_substring, + const char* needle_expr, const char* haystack_expr, + const StringType& needle, const StringType& haystack) { + if (IsSubstringPred(needle, haystack) == expected_to_be_substring) + return AssertionSuccess(); + + const bool is_wide_string = sizeof(needle[0]) > 1; + const char* const begin_string_quote = is_wide_string ? "L\"" : "\""; + return AssertionFailure() + << "Value of: " << needle_expr << "\n" + << " Actual: " << begin_string_quote << needle << "\"\n" + << "Expected: " << (expected_to_be_substring ? "" : "not ") + << "a substring of " << haystack_expr << "\n" + << "Which is: " << begin_string_quote << haystack << "\""; +} + +} // namespace + +// IsSubstring() and IsNotSubstring() check whether needle is a +// substring of haystack (NULL is considered a substring of itself +// only), and return an appropriate error message when they fail. + +AssertionResult IsSubstring( + const char* needle_expr, const char* haystack_expr, + const char* needle, const char* haystack) { + return IsSubstringImpl(true, needle_expr, haystack_expr, needle, haystack); +} + +AssertionResult IsSubstring( + const char* needle_expr, const char* haystack_expr, + const wchar_t* needle, const wchar_t* haystack) { + return IsSubstringImpl(true, needle_expr, haystack_expr, needle, haystack); +} + +AssertionResult IsNotSubstring( + const char* needle_expr, const char* haystack_expr, + const char* needle, const char* haystack) { + return IsSubstringImpl(false, needle_expr, haystack_expr, needle, haystack); +} + +AssertionResult IsNotSubstring( + const char* needle_expr, const char* haystack_expr, + const wchar_t* needle, const wchar_t* haystack) { + return IsSubstringImpl(false, needle_expr, haystack_expr, needle, haystack); +} + +AssertionResult IsSubstring( + const char* needle_expr, const char* haystack_expr, + const ::std::string& needle, const ::std::string& haystack) { + return IsSubstringImpl(true, needle_expr, haystack_expr, needle, haystack); +} + +AssertionResult IsNotSubstring( + const char* needle_expr, const char* haystack_expr, + const ::std::string& needle, const ::std::string& haystack) { + return IsSubstringImpl(false, needle_expr, haystack_expr, needle, haystack); +} + +#if GTEST_HAS_STD_WSTRING +AssertionResult IsSubstring( + const char* needle_expr, const char* haystack_expr, + const ::std::wstring& needle, const ::std::wstring& haystack) { + return IsSubstringImpl(true, needle_expr, haystack_expr, needle, haystack); +} + +AssertionResult IsNotSubstring( + const char* needle_expr, const char* haystack_expr, + const ::std::wstring& needle, const ::std::wstring& haystack) { + return IsSubstringImpl(false, needle_expr, haystack_expr, needle, haystack); +} +#endif // GTEST_HAS_STD_WSTRING + +namespace internal { + +#if GTEST_OS_WINDOWS + +namespace { + +// Helper function for IsHRESULT{SuccessFailure} predicates +AssertionResult HRESULTFailureHelper(const char* expr, + const char* expected, + long hr) { // NOLINT +# if GTEST_OS_WINDOWS_MOBILE + + // Windows CE doesn't support FormatMessage. + const char error_text[] = ""; + +# else + + // Looks up the human-readable system message for the HRESULT code + // and since we're not passing any params to FormatMessage, we don't + // want inserts expanded. + const DWORD kFlags = FORMAT_MESSAGE_FROM_SYSTEM | + FORMAT_MESSAGE_IGNORE_INSERTS; + const DWORD kBufSize = 4096; + // Gets the system's human readable message string for this HRESULT. + char error_text[kBufSize] = { '\0' }; + DWORD message_length = ::FormatMessageA(kFlags, + 0, // no source, we're asking system + hr, // the error + 0, // no line width restrictions + error_text, // output buffer + kBufSize, // buf size + NULL); // no arguments for inserts + // Trims tailing white space (FormatMessage leaves a trailing CR-LF) + for (; message_length && IsSpace(error_text[message_length - 1]); + --message_length) { + error_text[message_length - 1] = '\0'; + } + +# endif // GTEST_OS_WINDOWS_MOBILE + + const std::string error_hex("0x" + String::FormatHexInt(hr)); + return ::testing::AssertionFailure() + << "Expected: " << expr << " " << expected << ".\n" + << " Actual: " << error_hex << " " << error_text << "\n"; +} + +} // namespace + +AssertionResult IsHRESULTSuccess(const char* expr, long hr) { // NOLINT + if (SUCCEEDED(hr)) { + return AssertionSuccess(); + } + return HRESULTFailureHelper(expr, "succeeds", hr); +} + +AssertionResult IsHRESULTFailure(const char* expr, long hr) { // NOLINT + if (FAILED(hr)) { + return AssertionSuccess(); + } + return HRESULTFailureHelper(expr, "fails", hr); +} + +#endif // GTEST_OS_WINDOWS + +// Utility functions for encoding Unicode text (wide strings) in +// UTF-8. + +// A Unicode code-point can have upto 21 bits, and is encoded in UTF-8 +// like this: +// +// Code-point length Encoding +// 0 - 7 bits 0xxxxxxx +// 8 - 11 bits 110xxxxx 10xxxxxx +// 12 - 16 bits 1110xxxx 10xxxxxx 10xxxxxx +// 17 - 21 bits 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx + +// The maximum code-point a one-byte UTF-8 sequence can represent. +const UInt32 kMaxCodePoint1 = (static_cast(1) << 7) - 1; + +// The maximum code-point a two-byte UTF-8 sequence can represent. +const UInt32 kMaxCodePoint2 = (static_cast(1) << (5 + 6)) - 1; + +// The maximum code-point a three-byte UTF-8 sequence can represent. +const UInt32 kMaxCodePoint3 = (static_cast(1) << (4 + 2*6)) - 1; + +// The maximum code-point a four-byte UTF-8 sequence can represent. +const UInt32 kMaxCodePoint4 = (static_cast(1) << (3 + 3*6)) - 1; + +// Chops off the n lowest bits from a bit pattern. Returns the n +// lowest bits. As a side effect, the original bit pattern will be +// shifted to the right by n bits. +inline UInt32 ChopLowBits(UInt32* bits, int n) { + const UInt32 low_bits = *bits & ((static_cast(1) << n) - 1); + *bits >>= n; + return low_bits; +} + +// Converts a Unicode code point to a narrow string in UTF-8 encoding. +// code_point parameter is of type UInt32 because wchar_t may not be +// wide enough to contain a code point. +// If the code_point is not a valid Unicode code point +// (i.e. outside of Unicode range U+0 to U+10FFFF) it will be converted +// to "(Invalid Unicode 0xXXXXXXXX)". +std::string CodePointToUtf8(UInt32 code_point) { + if (code_point > kMaxCodePoint4) { + return "(Invalid Unicode 0x" + String::FormatHexInt(code_point) + ")"; + } + + char str[5]; // Big enough for the largest valid code point. + if (code_point <= kMaxCodePoint1) { + str[1] = '\0'; + str[0] = static_cast(code_point); // 0xxxxxxx + } else if (code_point <= kMaxCodePoint2) { + str[2] = '\0'; + str[1] = static_cast(0x80 | ChopLowBits(&code_point, 6)); // 10xxxxxx + str[0] = static_cast(0xC0 | code_point); // 110xxxxx + } else if (code_point <= kMaxCodePoint3) { + str[3] = '\0'; + str[2] = static_cast(0x80 | ChopLowBits(&code_point, 6)); // 10xxxxxx + str[1] = static_cast(0x80 | ChopLowBits(&code_point, 6)); // 10xxxxxx + str[0] = static_cast(0xE0 | code_point); // 1110xxxx + } else { // code_point <= kMaxCodePoint4 + str[4] = '\0'; + str[3] = static_cast(0x80 | ChopLowBits(&code_point, 6)); // 10xxxxxx + str[2] = static_cast(0x80 | ChopLowBits(&code_point, 6)); // 10xxxxxx + str[1] = static_cast(0x80 | ChopLowBits(&code_point, 6)); // 10xxxxxx + str[0] = static_cast(0xF0 | code_point); // 11110xxx + } + return str; +} + +// The following two functions only make sense if the system +// uses UTF-16 for wide string encoding. All supported systems +// with 16 bit wchar_t (Windows, Cygwin, Symbian OS) do use UTF-16. + +// Determines if the arguments constitute UTF-16 surrogate pair +// and thus should be combined into a single Unicode code point +// using CreateCodePointFromUtf16SurrogatePair. +inline bool IsUtf16SurrogatePair(wchar_t first, wchar_t second) { + return sizeof(wchar_t) == 2 && + (first & 0xFC00) == 0xD800 && (second & 0xFC00) == 0xDC00; +} + +// Creates a Unicode code point from UTF16 surrogate pair. +inline UInt32 CreateCodePointFromUtf16SurrogatePair(wchar_t first, + wchar_t second) { + const UInt32 mask = (1 << 10) - 1; + return (sizeof(wchar_t) == 2) ? + (((first & mask) << 10) | (second & mask)) + 0x10000 : + // This function should not be called when the condition is + // false, but we provide a sensible default in case it is. + static_cast(first); +} + +// Converts a wide string to a narrow string in UTF-8 encoding. +// The wide string is assumed to have the following encoding: +// UTF-16 if sizeof(wchar_t) == 2 (on Windows, Cygwin, Symbian OS) +// UTF-32 if sizeof(wchar_t) == 4 (on Linux) +// Parameter str points to a null-terminated wide string. +// Parameter num_chars may additionally limit the number +// of wchar_t characters processed. -1 is used when the entire string +// should be processed. +// If the string contains code points that are not valid Unicode code points +// (i.e. outside of Unicode range U+0 to U+10FFFF) they will be output +// as '(Invalid Unicode 0xXXXXXXXX)'. If the string is in UTF16 encoding +// and contains invalid UTF-16 surrogate pairs, values in those pairs +// will be encoded as individual Unicode characters from Basic Normal Plane. +std::string WideStringToUtf8(const wchar_t* str, int num_chars) { + if (num_chars == -1) + num_chars = static_cast(wcslen(str)); + + ::std::stringstream stream; + for (int i = 0; i < num_chars; ++i) { + UInt32 unicode_code_point; + + if (str[i] == L'\0') { + break; + } else if (i + 1 < num_chars && IsUtf16SurrogatePair(str[i], str[i + 1])) { + unicode_code_point = CreateCodePointFromUtf16SurrogatePair(str[i], + str[i + 1]); + i++; + } else { + unicode_code_point = static_cast(str[i]); + } + + stream << CodePointToUtf8(unicode_code_point); + } + return StringStreamToString(&stream); +} + +// Converts a wide C string to an std::string using the UTF-8 encoding. +// NULL will be converted to "(null)". +std::string String::ShowWideCString(const wchar_t * wide_c_str) { + if (wide_c_str == NULL) return "(null)"; + + return internal::WideStringToUtf8(wide_c_str, -1); +} + +// Compares two wide C strings. Returns true iff they have the same +// content. +// +// Unlike wcscmp(), this function can handle NULL argument(s). A NULL +// C string is considered different to any non-NULL C string, +// including the empty string. +bool String::WideCStringEquals(const wchar_t * lhs, const wchar_t * rhs) { + if (lhs == NULL) return rhs == NULL; + + if (rhs == NULL) return false; + + return wcscmp(lhs, rhs) == 0; +} + +// Helper function for *_STREQ on wide strings. +AssertionResult CmpHelperSTREQ(const char* expected_expression, + const char* actual_expression, + const wchar_t* expected, + const wchar_t* actual) { + if (String::WideCStringEquals(expected, actual)) { + return AssertionSuccess(); + } + + return EqFailure(expected_expression, + actual_expression, + PrintToString(expected), + PrintToString(actual), + false); +} + +// Helper function for *_STRNE on wide strings. +AssertionResult CmpHelperSTRNE(const char* s1_expression, + const char* s2_expression, + const wchar_t* s1, + const wchar_t* s2) { + if (!String::WideCStringEquals(s1, s2)) { + return AssertionSuccess(); + } + + return AssertionFailure() << "Expected: (" << s1_expression << ") != (" + << s2_expression << "), actual: " + << PrintToString(s1) + << " vs " << PrintToString(s2); +} + +// Compares two C strings, ignoring case. Returns true iff they have +// the same content. +// +// Unlike strcasecmp(), this function can handle NULL argument(s). A +// NULL C string is considered different to any non-NULL C string, +// including the empty string. +bool String::CaseInsensitiveCStringEquals(const char * lhs, const char * rhs) { + if (lhs == NULL) + return rhs == NULL; + if (rhs == NULL) + return false; + return posix::StrCaseCmp(lhs, rhs) == 0; +} + + // Compares two wide C strings, ignoring case. Returns true iff they + // have the same content. + // + // Unlike wcscasecmp(), this function can handle NULL argument(s). + // A NULL C string is considered different to any non-NULL wide C string, + // including the empty string. + // NB: The implementations on different platforms slightly differ. + // On windows, this method uses _wcsicmp which compares according to LC_CTYPE + // environment variable. On GNU platform this method uses wcscasecmp + // which compares according to LC_CTYPE category of the current locale. + // On MacOS X, it uses towlower, which also uses LC_CTYPE category of the + // current locale. +bool String::CaseInsensitiveWideCStringEquals(const wchar_t* lhs, + const wchar_t* rhs) { + if (lhs == NULL) return rhs == NULL; + + if (rhs == NULL) return false; + +#if GTEST_OS_WINDOWS + return _wcsicmp(lhs, rhs) == 0; +#elif GTEST_OS_LINUX && !GTEST_OS_LINUX_ANDROID + return wcscasecmp(lhs, rhs) == 0; +#else + // Android, Mac OS X and Cygwin don't define wcscasecmp. + // Other unknown OSes may not define it either. + wint_t left, right; + do { + left = towlower(*lhs++); + right = towlower(*rhs++); + } while (left && left == right); + return left == right; +#endif // OS selector +} + +// Returns true iff str ends with the given suffix, ignoring case. +// Any string is considered to end with an empty suffix. +bool String::EndsWithCaseInsensitive( + const std::string& str, const std::string& suffix) { + const size_t str_len = str.length(); + const size_t suffix_len = suffix.length(); + return (str_len >= suffix_len) && + CaseInsensitiveCStringEquals(str.c_str() + str_len - suffix_len, + suffix.c_str()); +} + +// Formats an int value as "%02d". +std::string String::FormatIntWidth2(int value) { + std::stringstream ss; + ss << std::setfill('0') << std::setw(2) << value; + return ss.str(); +} + +// Formats an int value as "%X". +std::string String::FormatHexInt(int value) { + std::stringstream ss; + ss << std::hex << std::uppercase << value; + return ss.str(); +} + +// Formats a byte as "%02X". +std::string String::FormatByte(unsigned char value) { + std::stringstream ss; + ss << std::setfill('0') << std::setw(2) << std::hex << std::uppercase + << static_cast(value); + return ss.str(); +} + +// Converts the buffer in a stringstream to an std::string, converting NUL +// bytes to "\\0" along the way. +std::string StringStreamToString(::std::stringstream* ss) { + const ::std::string& str = ss->str(); + const char* const start = str.c_str(); + const char* const end = start + str.length(); + + std::string result; + result.reserve(2 * (end - start)); + for (const char* ch = start; ch != end; ++ch) { + if (*ch == '\0') { + result += "\\0"; // Replaces NUL with "\\0"; + } else { + result += *ch; + } + } + + return result; +} + +// Appends the user-supplied message to the Google-Test-generated message. +std::string AppendUserMessage(const std::string& gtest_msg, + const Message& user_msg) { + // Appends the user message if it's non-empty. + const std::string user_msg_string = user_msg.GetString(); + if (user_msg_string.empty()) { + return gtest_msg; + } + + return gtest_msg + "\n" + user_msg_string; +} + +} // namespace internal + +// class TestResult + +// Creates an empty TestResult. +TestResult::TestResult() + : death_test_count_(0), + elapsed_time_(0) { +} + +// D'tor. +TestResult::~TestResult() { +} + +// Returns the i-th test part result among all the results. i can +// range from 0 to total_part_count() - 1. If i is not in that range, +// aborts the program. +const TestPartResult& TestResult::GetTestPartResult(int i) const { + if (i < 0 || i >= total_part_count()) + internal::posix::Abort(); + return test_part_results_.at(i); +} + +// Returns the i-th test property. i can range from 0 to +// test_property_count() - 1. If i is not in that range, aborts the +// program. +const TestProperty& TestResult::GetTestProperty(int i) const { + if (i < 0 || i >= test_property_count()) + internal::posix::Abort(); + return test_properties_.at(i); +} + +// Clears the test part results. +void TestResult::ClearTestPartResults() { + test_part_results_.clear(); +} + +// Adds a test part result to the list. +void TestResult::AddTestPartResult(const TestPartResult& test_part_result) { + test_part_results_.push_back(test_part_result); +} + +// Adds a test property to the list. If a property with the same key as the +// supplied property is already represented, the value of this test_property +// replaces the old value for that key. +void TestResult::RecordProperty(const std::string& xml_element, + const TestProperty& test_property) { + if (!ValidateTestProperty(xml_element, test_property)) { + return; + } + internal::MutexLock lock(&test_properites_mutex_); + const std::vector::iterator property_with_matching_key = + std::find_if(test_properties_.begin(), test_properties_.end(), + internal::TestPropertyKeyIs(test_property.key())); + if (property_with_matching_key == test_properties_.end()) { + test_properties_.push_back(test_property); + return; + } + property_with_matching_key->SetValue(test_property.value()); +} + +// The list of reserved attributes used in the element of XML +// output. +static const char* const kReservedTestSuitesAttributes[] = { + "disabled", + "errors", + "failures", + "name", + "random_seed", + "tests", + "time", + "timestamp" +}; + +// The list of reserved attributes used in the element of XML +// output. +static const char* const kReservedTestSuiteAttributes[] = { + "disabled", + "errors", + "failures", + "name", + "tests", + "time" +}; + +// The list of reserved attributes used in the element of XML output. +static const char* const kReservedTestCaseAttributes[] = { + "classname", + "name", + "status", + "time", + "type_param", + "value_param" +}; + +template +std::vector ArrayAsVector(const char* const (&array)[kSize]) { + return std::vector(array, array + kSize); +} + +static std::vector GetReservedAttributesForElement( + const std::string& xml_element) { + if (xml_element == "testsuites") { + return ArrayAsVector(kReservedTestSuitesAttributes); + } else if (xml_element == "testsuite") { + return ArrayAsVector(kReservedTestSuiteAttributes); + } else if (xml_element == "testcase") { + return ArrayAsVector(kReservedTestCaseAttributes); + } else { + GTEST_CHECK_(false) << "Unrecognized xml_element provided: " << xml_element; + } + // This code is unreachable but some compilers may not realizes that. + return std::vector(); +} + +static std::string FormatWordList(const std::vector& words) { + Message word_list; + for (size_t i = 0; i < words.size(); ++i) { + if (i > 0 && words.size() > 2) { + word_list << ", "; + } + if (i == words.size() - 1) { + word_list << "and "; + } + word_list << "'" << words[i] << "'"; + } + return word_list.GetString(); +} + +bool ValidateTestPropertyName(const std::string& property_name, + const std::vector& reserved_names) { + if (std::find(reserved_names.begin(), reserved_names.end(), property_name) != + reserved_names.end()) { + ADD_FAILURE() << "Reserved key used in RecordProperty(): " << property_name + << " (" << FormatWordList(reserved_names) + << " are reserved by " << GTEST_NAME_ << ")"; + return false; + } + return true; +} + +// Adds a failure if the key is a reserved attribute of the element named +// xml_element. Returns true if the property is valid. +bool TestResult::ValidateTestProperty(const std::string& xml_element, + const TestProperty& test_property) { + return ValidateTestPropertyName(test_property.key(), + GetReservedAttributesForElement(xml_element)); +} + +// Clears the object. +void TestResult::Clear() { + test_part_results_.clear(); + test_properties_.clear(); + death_test_count_ = 0; + elapsed_time_ = 0; +} + +// Returns true iff the test failed. +bool TestResult::Failed() const { + for (int i = 0; i < total_part_count(); ++i) { + if (GetTestPartResult(i).failed()) + return true; + } + return false; +} + +// Returns true iff the test part fatally failed. +static bool TestPartFatallyFailed(const TestPartResult& result) { + return result.fatally_failed(); +} + +// Returns true iff the test fatally failed. +bool TestResult::HasFatalFailure() const { + return CountIf(test_part_results_, TestPartFatallyFailed) > 0; +} + +// Returns true iff the test part non-fatally failed. +static bool TestPartNonfatallyFailed(const TestPartResult& result) { + return result.nonfatally_failed(); +} + +// Returns true iff the test has a non-fatal failure. +bool TestResult::HasNonfatalFailure() const { + return CountIf(test_part_results_, TestPartNonfatallyFailed) > 0; +} + +// Gets the number of all test parts. This is the sum of the number +// of successful test parts and the number of failed test parts. +int TestResult::total_part_count() const { + return static_cast(test_part_results_.size()); +} + +// Returns the number of the test properties. +int TestResult::test_property_count() const { + return static_cast(test_properties_.size()); +} + +// class Test + +// Creates a Test object. + +// The c'tor saves the values of all Google Test flags. +Test::Test() + : gtest_flag_saver_(new internal::GTestFlagSaver) { +} + +// The d'tor restores the values of all Google Test flags. +Test::~Test() { + delete gtest_flag_saver_; +} + +// Sets up the test fixture. +// +// A sub-class may override this. +void Test::SetUp() { +} + +// Tears down the test fixture. +// +// A sub-class may override this. +void Test::TearDown() { +} + +// Allows user supplied key value pairs to be recorded for later output. +void Test::RecordProperty(const std::string& key, const std::string& value) { + UnitTest::GetInstance()->RecordProperty(key, value); +} + +// Allows user supplied key value pairs to be recorded for later output. +void Test::RecordProperty(const std::string& key, int value) { + Message value_message; + value_message << value; + RecordProperty(key, value_message.GetString().c_str()); +} + +namespace internal { + +void ReportFailureInUnknownLocation(TestPartResult::Type result_type, + const std::string& message) { + // This function is a friend of UnitTest and as such has access to + // AddTestPartResult. + UnitTest::GetInstance()->AddTestPartResult( + result_type, + NULL, // No info about the source file where the exception occurred. + -1, // We have no info on which line caused the exception. + message, + ""); // No stack trace, either. +} + +} // namespace internal + +// Google Test requires all tests in the same test case to use the same test +// fixture class. This function checks if the current test has the +// same fixture class as the first test in the current test case. If +// yes, it returns true; otherwise it generates a Google Test failure and +// returns false. +bool Test::HasSameFixtureClass() { + internal::UnitTestImpl* const impl = internal::GetUnitTestImpl(); + const TestCase* const test_case = impl->current_test_case(); + + // Info about the first test in the current test case. + const TestInfo* const first_test_info = test_case->test_info_list()[0]; + const internal::TypeId first_fixture_id = first_test_info->fixture_class_id_; + const char* const first_test_name = first_test_info->name(); + + // Info about the current test. + const TestInfo* const this_test_info = impl->current_test_info(); + const internal::TypeId this_fixture_id = this_test_info->fixture_class_id_; + const char* const this_test_name = this_test_info->name(); + + if (this_fixture_id != first_fixture_id) { + // Is the first test defined using TEST? + const bool first_is_TEST = first_fixture_id == internal::GetTestTypeId(); + // Is this test defined using TEST? + const bool this_is_TEST = this_fixture_id == internal::GetTestTypeId(); + + if (first_is_TEST || this_is_TEST) { + // Both TEST and TEST_F appear in same test case, which is incorrect. + // Tell the user how to fix this. + + // Gets the name of the TEST and the name of the TEST_F. Note + // that first_is_TEST and this_is_TEST cannot both be true, as + // the fixture IDs are different for the two tests. + const char* const TEST_name = + first_is_TEST ? first_test_name : this_test_name; + const char* const TEST_F_name = + first_is_TEST ? this_test_name : first_test_name; + + ADD_FAILURE() + << "All tests in the same test case must use the same test fixture\n" + << "class, so mixing TEST_F and TEST in the same test case is\n" + << "illegal. In test case " << this_test_info->test_case_name() + << ",\n" + << "test " << TEST_F_name << " is defined using TEST_F but\n" + << "test " << TEST_name << " is defined using TEST. You probably\n" + << "want to change the TEST to TEST_F or move it to another test\n" + << "case."; + } else { + // Two fixture classes with the same name appear in two different + // namespaces, which is not allowed. Tell the user how to fix this. + ADD_FAILURE() + << "All tests in the same test case must use the same test fixture\n" + << "class. However, in test case " + << this_test_info->test_case_name() << ",\n" + << "you defined test " << first_test_name + << " and test " << this_test_name << "\n" + << "using two different test fixture classes. This can happen if\n" + << "the two classes are from different namespaces or translation\n" + << "units and have the same name. You should probably rename one\n" + << "of the classes to put the tests into different test cases."; + } + return false; + } + + return true; +} + +#if GTEST_HAS_SEH + +// Adds an "exception thrown" fatal failure to the current test. This +// function returns its result via an output parameter pointer because VC++ +// prohibits creation of objects with destructors on stack in functions +// using __try (see error C2712). +static std::string* FormatSehExceptionMessage(DWORD exception_code, + const char* location) { + Message message; + message << "SEH exception with code 0x" << std::setbase(16) << + exception_code << std::setbase(10) << " thrown in " << location << "."; + + return new std::string(message.GetString()); +} + +#endif // GTEST_HAS_SEH + +namespace internal { + +#if GTEST_HAS_EXCEPTIONS + +// Adds an "exception thrown" fatal failure to the current test. +static std::string FormatCxxExceptionMessage(const char* description, + const char* location) { + Message message; + if (description != NULL) { + message << "C++ exception with description \"" << description << "\""; + } else { + message << "Unknown C++ exception"; + } + message << " thrown in " << location << "."; + + return message.GetString(); +} + +static std::string PrintTestPartResultToString( + const TestPartResult& test_part_result); + +GoogleTestFailureException::GoogleTestFailureException( + const TestPartResult& failure) + : ::std::runtime_error(PrintTestPartResultToString(failure).c_str()) {} + +#endif // GTEST_HAS_EXCEPTIONS + +// We put these helper functions in the internal namespace as IBM's xlC +// compiler rejects the code if they were declared static. + +// Runs the given method and handles SEH exceptions it throws, when +// SEH is supported; returns the 0-value for type Result in case of an +// SEH exception. (Microsoft compilers cannot handle SEH and C++ +// exceptions in the same function. Therefore, we provide a separate +// wrapper function for handling SEH exceptions.) +template +Result HandleSehExceptionsInMethodIfSupported( + T* object, Result (T::*method)(), const char* location) { +#if GTEST_HAS_SEH + __try { + return (object->*method)(); + } __except (internal::UnitTestOptions::GTestShouldProcessSEH( // NOLINT + GetExceptionCode())) { + // We create the exception message on the heap because VC++ prohibits + // creation of objects with destructors on stack in functions using __try + // (see error C2712). + std::string* exception_message = FormatSehExceptionMessage( + GetExceptionCode(), location); + internal::ReportFailureInUnknownLocation(TestPartResult::kFatalFailure, + *exception_message); + delete exception_message; + return static_cast(0); + } +#else + (void)location; + return (object->*method)(); +#endif // GTEST_HAS_SEH +} + +// Runs the given method and catches and reports C++ and/or SEH-style +// exceptions, if they are supported; returns the 0-value for type +// Result in case of an SEH exception. +template +Result HandleExceptionsInMethodIfSupported( + T* object, Result (T::*method)(), const char* location) { + // NOTE: The user code can affect the way in which Google Test handles + // exceptions by setting GTEST_FLAG(catch_exceptions), but only before + // RUN_ALL_TESTS() starts. It is technically possible to check the flag + // after the exception is caught and either report or re-throw the + // exception based on the flag's value: + // + // try { + // // Perform the test method. + // } catch (...) { + // if (GTEST_FLAG(catch_exceptions)) + // // Report the exception as failure. + // else + // throw; // Re-throws the original exception. + // } + // + // However, the purpose of this flag is to allow the program to drop into + // the debugger when the exception is thrown. On most platforms, once the + // control enters the catch block, the exception origin information is + // lost and the debugger will stop the program at the point of the + // re-throw in this function -- instead of at the point of the original + // throw statement in the code under test. For this reason, we perform + // the check early, sacrificing the ability to affect Google Test's + // exception handling in the method where the exception is thrown. + if (internal::GetUnitTestImpl()->catch_exceptions()) { +#if GTEST_HAS_EXCEPTIONS + try { + return HandleSehExceptionsInMethodIfSupported(object, method, location); + } catch (const internal::GoogleTestFailureException&) { // NOLINT + // This exception type can only be thrown by a failed Google + // Test assertion with the intention of letting another testing + // framework catch it. Therefore we just re-throw it. + throw; + } catch (const std::exception& e) { // NOLINT + internal::ReportFailureInUnknownLocation( + TestPartResult::kFatalFailure, + FormatCxxExceptionMessage(e.what(), location)); + } catch (...) { // NOLINT + internal::ReportFailureInUnknownLocation( + TestPartResult::kFatalFailure, + FormatCxxExceptionMessage(NULL, location)); + } + return static_cast(0); +#else + return HandleSehExceptionsInMethodIfSupported(object, method, location); +#endif // GTEST_HAS_EXCEPTIONS + } else { + return (object->*method)(); + } +} + +} // namespace internal + +// Runs the test and updates the test result. +void Test::Run() { + if (!HasSameFixtureClass()) return; + + internal::UnitTestImpl* const impl = internal::GetUnitTestImpl(); + impl->os_stack_trace_getter()->UponLeavingGTest(); + internal::HandleExceptionsInMethodIfSupported(this, &Test::SetUp, "SetUp()"); + // We will run the test only if SetUp() was successful. + if (!HasFatalFailure()) { + impl->os_stack_trace_getter()->UponLeavingGTest(); + internal::HandleExceptionsInMethodIfSupported( + this, &Test::TestBody, "the test body"); + } + + // However, we want to clean up as much as possible. Hence we will + // always call TearDown(), even if SetUp() or the test body has + // failed. + impl->os_stack_trace_getter()->UponLeavingGTest(); + internal::HandleExceptionsInMethodIfSupported( + this, &Test::TearDown, "TearDown()"); +} + +// Returns true iff the current test has a fatal failure. +bool Test::HasFatalFailure() { + return internal::GetUnitTestImpl()->current_test_result()->HasFatalFailure(); +} + +// Returns true iff the current test has a non-fatal failure. +bool Test::HasNonfatalFailure() { + return internal::GetUnitTestImpl()->current_test_result()-> + HasNonfatalFailure(); +} + +// class TestInfo + +// Constructs a TestInfo object. It assumes ownership of the test factory +// object. +TestInfo::TestInfo(const std::string& a_test_case_name, + const std::string& a_name, + const char* a_type_param, + const char* a_value_param, + internal::TypeId fixture_class_id, + internal::TestFactoryBase* factory) + : test_case_name_(a_test_case_name), + name_(a_name), + type_param_(a_type_param ? new std::string(a_type_param) : NULL), + value_param_(a_value_param ? new std::string(a_value_param) : NULL), + fixture_class_id_(fixture_class_id), + should_run_(false), + is_disabled_(false), + matches_filter_(false), + factory_(factory), + result_() {} + +// Destructs a TestInfo object. +TestInfo::~TestInfo() { delete factory_; } + +namespace internal { + +// Creates a new TestInfo object and registers it with Google Test; +// returns the created object. +// +// Arguments: +// +// test_case_name: name of the test case +// name: name of the test +// type_param: the name of the test's type parameter, or NULL if +// this is not a typed or a type-parameterized test. +// value_param: text representation of the test's value parameter, +// or NULL if this is not a value-parameterized test. +// fixture_class_id: ID of the test fixture class +// set_up_tc: pointer to the function that sets up the test case +// tear_down_tc: pointer to the function that tears down the test case +// factory: pointer to the factory that creates a test object. +// The newly created TestInfo instance will assume +// ownership of the factory object. +TestInfo* MakeAndRegisterTestInfo( + const char* test_case_name, + const char* name, + const char* type_param, + const char* value_param, + TypeId fixture_class_id, + SetUpTestCaseFunc set_up_tc, + TearDownTestCaseFunc tear_down_tc, + TestFactoryBase* factory) { + TestInfo* const test_info = + new TestInfo(test_case_name, name, type_param, value_param, + fixture_class_id, factory); + GetUnitTestImpl()->AddTestInfo(set_up_tc, tear_down_tc, test_info); + return test_info; +} + +#if GTEST_HAS_PARAM_TEST +void ReportInvalidTestCaseType(const char* test_case_name, + const char* file, int line) { + Message errors; + errors + << "Attempted redefinition of test case " << test_case_name << ".\n" + << "All tests in the same test case must use the same test fixture\n" + << "class. However, in test case " << test_case_name << ", you tried\n" + << "to define a test using a fixture class different from the one\n" + << "used earlier. This can happen if the two fixture classes are\n" + << "from different namespaces and have the same name. You should\n" + << "probably rename one of the classes to put the tests into different\n" + << "test cases."; + + fprintf(stderr, "%s %s", FormatFileLocation(file, line).c_str(), + errors.GetString().c_str()); +} +#endif // GTEST_HAS_PARAM_TEST + +} // namespace internal + +namespace { + +// A predicate that checks the test name of a TestInfo against a known +// value. +// +// This is used for implementation of the TestCase class only. We put +// it in the anonymous namespace to prevent polluting the outer +// namespace. +// +// TestNameIs is copyable. +class TestNameIs { + public: + // Constructor. + // + // TestNameIs has NO default constructor. + explicit TestNameIs(const char* name) + : name_(name) {} + + // Returns true iff the test name of test_info matches name_. + bool operator()(const TestInfo * test_info) const { + return test_info && test_info->name() == name_; + } + + private: + std::string name_; +}; + +} // namespace + +namespace internal { + +// This method expands all parameterized tests registered with macros TEST_P +// and INSTANTIATE_TEST_CASE_P into regular tests and registers those. +// This will be done just once during the program runtime. +void UnitTestImpl::RegisterParameterizedTests() { +#if GTEST_HAS_PARAM_TEST + if (!parameterized_tests_registered_) { + parameterized_test_registry_.RegisterTests(); + parameterized_tests_registered_ = true; + } +#endif +} + +} // namespace internal + +// Creates the test object, runs it, records its result, and then +// deletes it. +void TestInfo::Run() { + if (!should_run_) return; + + // Tells UnitTest where to store test result. + internal::UnitTestImpl* const impl = internal::GetUnitTestImpl(); + impl->set_current_test_info(this); + + TestEventListener* repeater = UnitTest::GetInstance()->listeners().repeater(); + + // Notifies the unit test event listeners that a test is about to start. + repeater->OnTestStart(*this); + + const TimeInMillis start = internal::GetTimeInMillis(); + + impl->os_stack_trace_getter()->UponLeavingGTest(); + + // Creates the test object. + Test* const test = internal::HandleExceptionsInMethodIfSupported( + factory_, &internal::TestFactoryBase::CreateTest, + "the test fixture's constructor"); + + // Runs the test only if the test object was created and its + // constructor didn't generate a fatal failure. + if ((test != NULL) && !Test::HasFatalFailure()) { + // This doesn't throw as all user code that can throw are wrapped into + // exception handling code. + test->Run(); + } + + // Deletes the test object. + impl->os_stack_trace_getter()->UponLeavingGTest(); + internal::HandleExceptionsInMethodIfSupported( + test, &Test::DeleteSelf_, "the test fixture's destructor"); + + result_.set_elapsed_time(internal::GetTimeInMillis() - start); + + // Notifies the unit test event listener that a test has just finished. + repeater->OnTestEnd(*this); + + // Tells UnitTest to stop associating assertion results to this + // test. + impl->set_current_test_info(NULL); +} + +// class TestCase + +// Gets the number of successful tests in this test case. +int TestCase::successful_test_count() const { + return CountIf(test_info_list_, TestPassed); +} + +// Gets the number of failed tests in this test case. +int TestCase::failed_test_count() const { + return CountIf(test_info_list_, TestFailed); +} + +// Gets the number of disabled tests that will be reported in the XML report. +int TestCase::reportable_disabled_test_count() const { + return CountIf(test_info_list_, TestReportableDisabled); +} + +// Gets the number of disabled tests in this test case. +int TestCase::disabled_test_count() const { + return CountIf(test_info_list_, TestDisabled); +} + +// Gets the number of tests to be printed in the XML report. +int TestCase::reportable_test_count() const { + return CountIf(test_info_list_, TestReportable); +} + +// Get the number of tests in this test case that should run. +int TestCase::test_to_run_count() const { + return CountIf(test_info_list_, ShouldRunTest); +} + +// Gets the number of all tests. +int TestCase::total_test_count() const { + return static_cast(test_info_list_.size()); +} + +// Creates a TestCase with the given name. +// +// Arguments: +// +// name: name of the test case +// a_type_param: the name of the test case's type parameter, or NULL if +// this is not a typed or a type-parameterized test case. +// set_up_tc: pointer to the function that sets up the test case +// tear_down_tc: pointer to the function that tears down the test case +TestCase::TestCase(const char* a_name, const char* a_type_param, + Test::SetUpTestCaseFunc set_up_tc, + Test::TearDownTestCaseFunc tear_down_tc) + : name_(a_name), + type_param_(a_type_param ? new std::string(a_type_param) : NULL), + set_up_tc_(set_up_tc), + tear_down_tc_(tear_down_tc), + should_run_(false), + elapsed_time_(0) { +} + +// Destructor of TestCase. +TestCase::~TestCase() { + // Deletes every Test in the collection. + ForEach(test_info_list_, internal::Delete); +} + +// Returns the i-th test among all the tests. i can range from 0 to +// total_test_count() - 1. If i is not in that range, returns NULL. +const TestInfo* TestCase::GetTestInfo(int i) const { + const int index = GetElementOr(test_indices_, i, -1); + return index < 0 ? NULL : test_info_list_[index]; +} + +// Returns the i-th test among all the tests. i can range from 0 to +// total_test_count() - 1. If i is not in that range, returns NULL. +TestInfo* TestCase::GetMutableTestInfo(int i) { + const int index = GetElementOr(test_indices_, i, -1); + return index < 0 ? NULL : test_info_list_[index]; +} + +// Adds a test to this test case. Will delete the test upon +// destruction of the TestCase object. +void TestCase::AddTestInfo(TestInfo * test_info) { + test_info_list_.push_back(test_info); + test_indices_.push_back(static_cast(test_indices_.size())); +} + +// Runs every test in this TestCase. +void TestCase::Run() { + if (!should_run_) return; + + internal::UnitTestImpl* const impl = internal::GetUnitTestImpl(); + impl->set_current_test_case(this); + + TestEventListener* repeater = UnitTest::GetInstance()->listeners().repeater(); + + repeater->OnTestCaseStart(*this); + impl->os_stack_trace_getter()->UponLeavingGTest(); + internal::HandleExceptionsInMethodIfSupported( + this, &TestCase::RunSetUpTestCase, "SetUpTestCase()"); + + const internal::TimeInMillis start = internal::GetTimeInMillis(); + for (int i = 0; i < total_test_count(); i++) { + GetMutableTestInfo(i)->Run(); + } + elapsed_time_ = internal::GetTimeInMillis() - start; + + impl->os_stack_trace_getter()->UponLeavingGTest(); + internal::HandleExceptionsInMethodIfSupported( + this, &TestCase::RunTearDownTestCase, "TearDownTestCase()"); + + repeater->OnTestCaseEnd(*this); + impl->set_current_test_case(NULL); +} + +// Clears the results of all tests in this test case. +void TestCase::ClearResult() { + ad_hoc_test_result_.Clear(); + ForEach(test_info_list_, TestInfo::ClearTestResult); +} + +// Shuffles the tests in this test case. +void TestCase::ShuffleTests(internal::Random* random) { + Shuffle(random, &test_indices_); +} + +// Restores the test order to before the first shuffle. +void TestCase::UnshuffleTests() { + for (size_t i = 0; i < test_indices_.size(); i++) { + test_indices_[i] = static_cast(i); + } +} + +// Formats a countable noun. Depending on its quantity, either the +// singular form or the plural form is used. e.g. +// +// FormatCountableNoun(1, "formula", "formuli") returns "1 formula". +// FormatCountableNoun(5, "book", "books") returns "5 books". +static std::string FormatCountableNoun(int count, + const char * singular_form, + const char * plural_form) { + return internal::StreamableToString(count) + " " + + (count == 1 ? singular_form : plural_form); +} + +// Formats the count of tests. +static std::string FormatTestCount(int test_count) { + return FormatCountableNoun(test_count, "test", "tests"); +} + +// Formats the count of test cases. +static std::string FormatTestCaseCount(int test_case_count) { + return FormatCountableNoun(test_case_count, "test case", "test cases"); +} + +// Converts a TestPartResult::Type enum to human-friendly string +// representation. Both kNonFatalFailure and kFatalFailure are translated +// to "Failure", as the user usually doesn't care about the difference +// between the two when viewing the test result. +static const char * TestPartResultTypeToString(TestPartResult::Type type) { + switch (type) { + case TestPartResult::kSuccess: + return "Success"; + + case TestPartResult::kNonFatalFailure: + case TestPartResult::kFatalFailure: +#ifdef _MSC_VER + return "error: "; +#else + return "Failure\n"; +#endif + default: + return "Unknown result type"; + } +} + +namespace internal { + +// Prints a TestPartResult to an std::string. +static std::string PrintTestPartResultToString( + const TestPartResult& test_part_result) { + return (Message() + << internal::FormatFileLocation(test_part_result.file_name(), + test_part_result.line_number()) + << " " << TestPartResultTypeToString(test_part_result.type()) + << test_part_result.message()).GetString(); +} + +// Prints a TestPartResult. +static void PrintTestPartResult(const TestPartResult& test_part_result) { + const std::string& result = + PrintTestPartResultToString(test_part_result); + printf("%s\n", result.c_str()); + fflush(stdout); + // If the test program runs in Visual Studio or a debugger, the + // following statements add the test part result message to the Output + // window such that the user can double-click on it to jump to the + // corresponding source code location; otherwise they do nothing. +#if GTEST_OS_WINDOWS && !GTEST_OS_WINDOWS_MOBILE + // We don't call OutputDebugString*() on Windows Mobile, as printing + // to stdout is done by OutputDebugString() there already - we don't + // want the same message printed twice. + ::OutputDebugStringA(result.c_str()); + ::OutputDebugStringA("\n"); +#endif +} + +// class PrettyUnitTestResultPrinter + +enum GTestColor { + COLOR_DEFAULT, + COLOR_RED, + COLOR_GREEN, + COLOR_YELLOW +}; + +#if GTEST_OS_WINDOWS && !GTEST_OS_WINDOWS_MOBILE && \ + !GTEST_OS_WINDOWS_PHONE && !GTEST_OS_WINDOWS_RT + +// Returns the character attribute for the given color. +WORD GetColorAttribute(GTestColor color) { + switch (color) { + case COLOR_RED: return FOREGROUND_RED; + case COLOR_GREEN: return FOREGROUND_GREEN; + case COLOR_YELLOW: return FOREGROUND_RED | FOREGROUND_GREEN; + default: return 0; + } +} + +#else + +// Returns the ANSI color code for the given color. COLOR_DEFAULT is +// an invalid input. +const char* GetAnsiColorCode(GTestColor color) { + switch (color) { + case COLOR_RED: return "1"; + case COLOR_GREEN: return "2"; + case COLOR_YELLOW: return "3"; + default: return NULL; + }; +} + +#endif // GTEST_OS_WINDOWS && !GTEST_OS_WINDOWS_MOBILE + +// Returns true iff Google Test should use colors in the output. +bool ShouldUseColor(bool stdout_is_tty) { + const char* const gtest_color = GTEST_FLAG(color).c_str(); + + if (String::CaseInsensitiveCStringEquals(gtest_color, "auto")) { +#if GTEST_OS_WINDOWS + // On Windows the TERM variable is usually not set, but the + // console there does support colors. + return stdout_is_tty; +#else + // On non-Windows platforms, we rely on the TERM variable. + const char* const term = posix::GetEnv("TERM"); + const bool term_supports_color = + String::CStringEquals(term, "xterm") || + String::CStringEquals(term, "xterm-color") || + String::CStringEquals(term, "xterm-256color") || + String::CStringEquals(term, "screen") || + String::CStringEquals(term, "screen-256color") || + String::CStringEquals(term, "linux") || + String::CStringEquals(term, "cygwin"); + return stdout_is_tty && term_supports_color; +#endif // GTEST_OS_WINDOWS + } + + return String::CaseInsensitiveCStringEquals(gtest_color, "yes") || + String::CaseInsensitiveCStringEquals(gtest_color, "true") || + String::CaseInsensitiveCStringEquals(gtest_color, "t") || + String::CStringEquals(gtest_color, "1"); + // We take "yes", "true", "t", and "1" as meaning "yes". If the + // value is neither one of these nor "auto", we treat it as "no" to + // be conservative. +} + +// Helpers for printing colored strings to stdout. Note that on Windows, we +// cannot simply emit special characters and have the terminal change colors. +// This routine must actually emit the characters rather than return a string +// that would be colored when printed, as can be done on Linux. +void ColoredPrintf(GTestColor color, const char* fmt, ...) { + va_list args; + va_start(args, fmt); + +#if GTEST_OS_WINDOWS_MOBILE || GTEST_OS_SYMBIAN || GTEST_OS_ZOS || \ + GTEST_OS_IOS || GTEST_OS_WINDOWS_PHONE || GTEST_OS_WINDOWS_RT + const bool use_color = AlwaysFalse(); +#else + static const bool in_color_mode = + ShouldUseColor(posix::IsATTY(posix::FileNo(stdout)) != 0); + const bool use_color = in_color_mode && (color != COLOR_DEFAULT); +#endif // GTEST_OS_WINDOWS_MOBILE || GTEST_OS_SYMBIAN || GTEST_OS_ZOS + // The '!= 0' comparison is necessary to satisfy MSVC 7.1. + + if (!use_color) { + vprintf(fmt, args); + va_end(args); + return; + } + +#if GTEST_OS_WINDOWS && !GTEST_OS_WINDOWS_MOBILE && \ + !GTEST_OS_WINDOWS_PHONE && !GTEST_OS_WINDOWS_RT + const HANDLE stdout_handle = GetStdHandle(STD_OUTPUT_HANDLE); + + // Gets the current text color. + CONSOLE_SCREEN_BUFFER_INFO buffer_info; + GetConsoleScreenBufferInfo(stdout_handle, &buffer_info); + const WORD old_color_attrs = buffer_info.wAttributes; + + // We need to flush the stream buffers into the console before each + // SetConsoleTextAttribute call lest it affect the text that is already + // printed but has not yet reached the console. + fflush(stdout); + SetConsoleTextAttribute(stdout_handle, + GetColorAttribute(color) | FOREGROUND_INTENSITY); + vprintf(fmt, args); + + fflush(stdout); + // Restores the text color. + SetConsoleTextAttribute(stdout_handle, old_color_attrs); +#else + printf("\033[0;3%sm", GetAnsiColorCode(color)); + vprintf(fmt, args); + printf("\033[m"); // Resets the terminal to default. +#endif // GTEST_OS_WINDOWS && !GTEST_OS_WINDOWS_MOBILE + va_end(args); +} + +// Text printed in Google Test's text output and --gunit_list_tests +// output to label the type parameter and value parameter for a test. +static const char kTypeParamLabel[] = "TypeParam"; +static const char kValueParamLabel[] = "GetParam()"; + +void PrintFullTestCommentIfPresent(const TestInfo& test_info) { + const char* const type_param = test_info.type_param(); + const char* const value_param = test_info.value_param(); + + if (type_param != NULL || value_param != NULL) { + printf(", where "); + if (type_param != NULL) { + printf("%s = %s", kTypeParamLabel, type_param); + if (value_param != NULL) + printf(" and "); + } + if (value_param != NULL) { + printf("%s = %s", kValueParamLabel, value_param); + } + } +} + +// This class implements the TestEventListener interface. +// +// Class PrettyUnitTestResultPrinter is copyable. +class PrettyUnitTestResultPrinter : public TestEventListener { + public: + PrettyUnitTestResultPrinter() {} + static void PrintTestName(const char * test_case, const char * test) { + printf("%s.%s", test_case, test); + } + + // The following methods override what's in the TestEventListener class. + virtual void OnTestProgramStart(const UnitTest& /*unit_test*/) {} + virtual void OnTestIterationStart(const UnitTest& unit_test, int iteration); + virtual void OnEnvironmentsSetUpStart(const UnitTest& unit_test); + virtual void OnEnvironmentsSetUpEnd(const UnitTest& /*unit_test*/) {} + virtual void OnTestCaseStart(const TestCase& test_case); + virtual void OnTestStart(const TestInfo& test_info); + virtual void OnTestPartResult(const TestPartResult& result); + virtual void OnTestEnd(const TestInfo& test_info); + virtual void OnTestCaseEnd(const TestCase& test_case); + virtual void OnEnvironmentsTearDownStart(const UnitTest& unit_test); + virtual void OnEnvironmentsTearDownEnd(const UnitTest& /*unit_test*/) {} + virtual void OnTestIterationEnd(const UnitTest& unit_test, int iteration); + virtual void OnTestProgramEnd(const UnitTest& /*unit_test*/) {} + + private: + static void PrintFailedTests(const UnitTest& unit_test); +}; + + // Fired before each iteration of tests starts. +void PrettyUnitTestResultPrinter::OnTestIterationStart( + const UnitTest& unit_test, int iteration) { + if (GTEST_FLAG(repeat) != 1) + printf("\nRepeating all tests (iteration %d) . . .\n\n", iteration + 1); + + const char* const filter = GTEST_FLAG(filter).c_str(); + + // Prints the filter if it's not *. This reminds the user that some + // tests may be skipped. + if (!String::CStringEquals(filter, kUniversalFilter)) { + ColoredPrintf(COLOR_YELLOW, + "Note: %s filter = %s\n", GTEST_NAME_, filter); + } + + if (internal::ShouldShard(kTestTotalShards, kTestShardIndex, false)) { + const Int32 shard_index = Int32FromEnvOrDie(kTestShardIndex, -1); + ColoredPrintf(COLOR_YELLOW, + "Note: This is test shard %d of %s.\n", + static_cast(shard_index) + 1, + internal::posix::GetEnv(kTestTotalShards)); + } + + if (GTEST_FLAG(shuffle)) { + ColoredPrintf(COLOR_YELLOW, + "Note: Randomizing tests' orders with a seed of %d .\n", + unit_test.random_seed()); + } + + ColoredPrintf(COLOR_GREEN, "[==========] "); + printf("Running %s from %s.\n", + FormatTestCount(unit_test.test_to_run_count()).c_str(), + FormatTestCaseCount(unit_test.test_case_to_run_count()).c_str()); + fflush(stdout); +} + +void PrettyUnitTestResultPrinter::OnEnvironmentsSetUpStart( + const UnitTest& /*unit_test*/) { + ColoredPrintf(COLOR_GREEN, "[----------] "); + printf("Global test environment set-up.\n"); + fflush(stdout); +} + +void PrettyUnitTestResultPrinter::OnTestCaseStart(const TestCase& test_case) { + const std::string counts = + FormatCountableNoun(test_case.test_to_run_count(), "test", "tests"); + ColoredPrintf(COLOR_GREEN, "[----------] "); + printf("%s from %s", counts.c_str(), test_case.name()); + if (test_case.type_param() == NULL) { + printf("\n"); + } else { + printf(", where %s = %s\n", kTypeParamLabel, test_case.type_param()); + } + fflush(stdout); +} + +void PrettyUnitTestResultPrinter::OnTestStart(const TestInfo& test_info) { + ColoredPrintf(COLOR_GREEN, "[ RUN ] "); + PrintTestName(test_info.test_case_name(), test_info.name()); + printf("\n"); + fflush(stdout); +} + +// Called after an assertion failure. +void PrettyUnitTestResultPrinter::OnTestPartResult( + const TestPartResult& result) { + // If the test part succeeded, we don't need to do anything. + if (result.type() == TestPartResult::kSuccess) + return; + + // Print failure message from the assertion (e.g. expected this and got that). + PrintTestPartResult(result); + fflush(stdout); +} + +void PrettyUnitTestResultPrinter::OnTestEnd(const TestInfo& test_info) { + if (test_info.result()->Passed()) { + ColoredPrintf(COLOR_GREEN, "[ OK ] "); + } else { + ColoredPrintf(COLOR_RED, "[ FAILED ] "); + } + PrintTestName(test_info.test_case_name(), test_info.name()); + if (test_info.result()->Failed()) + PrintFullTestCommentIfPresent(test_info); + + if (GTEST_FLAG(print_time)) { + printf(" (%s ms)\n", internal::StreamableToString( + test_info.result()->elapsed_time()).c_str()); + } else { + printf("\n"); + } + fflush(stdout); +} + +void PrettyUnitTestResultPrinter::OnTestCaseEnd(const TestCase& test_case) { + if (!GTEST_FLAG(print_time)) return; + + const std::string counts = + FormatCountableNoun(test_case.test_to_run_count(), "test", "tests"); + ColoredPrintf(COLOR_GREEN, "[----------] "); + printf("%s from %s (%s ms total)\n\n", + counts.c_str(), test_case.name(), + internal::StreamableToString(test_case.elapsed_time()).c_str()); + fflush(stdout); +} + +void PrettyUnitTestResultPrinter::OnEnvironmentsTearDownStart( + const UnitTest& /*unit_test*/) { + ColoredPrintf(COLOR_GREEN, "[----------] "); + printf("Global test environment tear-down\n"); + fflush(stdout); +} + +// Internal helper for printing the list of failed tests. +void PrettyUnitTestResultPrinter::PrintFailedTests(const UnitTest& unit_test) { + const int failed_test_count = unit_test.failed_test_count(); + if (failed_test_count == 0) { + return; + } + + for (int i = 0; i < unit_test.total_test_case_count(); ++i) { + const TestCase& test_case = *unit_test.GetTestCase(i); + if (!test_case.should_run() || (test_case.failed_test_count() == 0)) { + continue; + } + for (int j = 0; j < test_case.total_test_count(); ++j) { + const TestInfo& test_info = *test_case.GetTestInfo(j); + if (!test_info.should_run() || test_info.result()->Passed()) { + continue; + } + ColoredPrintf(COLOR_RED, "[ FAILED ] "); + printf("%s.%s", test_case.name(), test_info.name()); + PrintFullTestCommentIfPresent(test_info); + printf("\n"); + } + } +} + +void PrettyUnitTestResultPrinter::OnTestIterationEnd(const UnitTest& unit_test, + int /*iteration*/) { + ColoredPrintf(COLOR_GREEN, "[==========] "); + printf("%s from %s ran.", + FormatTestCount(unit_test.test_to_run_count()).c_str(), + FormatTestCaseCount(unit_test.test_case_to_run_count()).c_str()); + if (GTEST_FLAG(print_time)) { + printf(" (%s ms total)", + internal::StreamableToString(unit_test.elapsed_time()).c_str()); + } + printf("\n"); + ColoredPrintf(COLOR_GREEN, "[ PASSED ] "); + printf("%s.\n", FormatTestCount(unit_test.successful_test_count()).c_str()); + + int num_failures = unit_test.failed_test_count(); + if (!unit_test.Passed()) { + const int failed_test_count = unit_test.failed_test_count(); + ColoredPrintf(COLOR_RED, "[ FAILED ] "); + printf("%s, listed below:\n", FormatTestCount(failed_test_count).c_str()); + PrintFailedTests(unit_test); + printf("\n%2d FAILED %s\n", num_failures, + num_failures == 1 ? "TEST" : "TESTS"); + } + + int num_disabled = unit_test.reportable_disabled_test_count(); + if (num_disabled && !GTEST_FLAG(also_run_disabled_tests)) { + if (!num_failures) { + printf("\n"); // Add a spacer if no FAILURE banner is displayed. + } + ColoredPrintf(COLOR_YELLOW, + " YOU HAVE %d DISABLED %s\n\n", + num_disabled, + num_disabled == 1 ? "TEST" : "TESTS"); + } + // Ensure that Google Test output is printed before, e.g., heapchecker output. + fflush(stdout); +} + +// End PrettyUnitTestResultPrinter + +// class TestEventRepeater +// +// This class forwards events to other event listeners. +class TestEventRepeater : public TestEventListener { + public: + TestEventRepeater() : forwarding_enabled_(true) {} + virtual ~TestEventRepeater(); + void Append(TestEventListener *listener); + TestEventListener* Release(TestEventListener* listener); + + // Controls whether events will be forwarded to listeners_. Set to false + // in death test child processes. + bool forwarding_enabled() const { return forwarding_enabled_; } + void set_forwarding_enabled(bool enable) { forwarding_enabled_ = enable; } + + virtual void OnTestProgramStart(const UnitTest& unit_test); + virtual void OnTestIterationStart(const UnitTest& unit_test, int iteration); + virtual void OnEnvironmentsSetUpStart(const UnitTest& unit_test); + virtual void OnEnvironmentsSetUpEnd(const UnitTest& unit_test); + virtual void OnTestCaseStart(const TestCase& test_case); + virtual void OnTestStart(const TestInfo& test_info); + virtual void OnTestPartResult(const TestPartResult& result); + virtual void OnTestEnd(const TestInfo& test_info); + virtual void OnTestCaseEnd(const TestCase& test_case); + virtual void OnEnvironmentsTearDownStart(const UnitTest& unit_test); + virtual void OnEnvironmentsTearDownEnd(const UnitTest& unit_test); + virtual void OnTestIterationEnd(const UnitTest& unit_test, int iteration); + virtual void OnTestProgramEnd(const UnitTest& unit_test); + + private: + // Controls whether events will be forwarded to listeners_. Set to false + // in death test child processes. + bool forwarding_enabled_; + // The list of listeners that receive events. + std::vector listeners_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(TestEventRepeater); +}; + +TestEventRepeater::~TestEventRepeater() { + ForEach(listeners_, Delete); +} + +void TestEventRepeater::Append(TestEventListener *listener) { + listeners_.push_back(listener); +} + +// TODO(vladl@google.com): Factor the search functionality into Vector::Find. +TestEventListener* TestEventRepeater::Release(TestEventListener *listener) { + for (size_t i = 0; i < listeners_.size(); ++i) { + if (listeners_[i] == listener) { + listeners_.erase(listeners_.begin() + i); + return listener; + } + } + + return NULL; +} + +// Since most methods are very similar, use macros to reduce boilerplate. +// This defines a member that forwards the call to all listeners. +#define GTEST_REPEATER_METHOD_(Name, Type) \ +void TestEventRepeater::Name(const Type& parameter) { \ + if (forwarding_enabled_) { \ + for (size_t i = 0; i < listeners_.size(); i++) { \ + listeners_[i]->Name(parameter); \ + } \ + } \ +} +// This defines a member that forwards the call to all listeners in reverse +// order. +#define GTEST_REVERSE_REPEATER_METHOD_(Name, Type) \ +void TestEventRepeater::Name(const Type& parameter) { \ + if (forwarding_enabled_) { \ + for (int i = static_cast(listeners_.size()) - 1; i >= 0; i--) { \ + listeners_[i]->Name(parameter); \ + } \ + } \ +} + +GTEST_REPEATER_METHOD_(OnTestProgramStart, UnitTest) +GTEST_REPEATER_METHOD_(OnEnvironmentsSetUpStart, UnitTest) +GTEST_REPEATER_METHOD_(OnTestCaseStart, TestCase) +GTEST_REPEATER_METHOD_(OnTestStart, TestInfo) +GTEST_REPEATER_METHOD_(OnTestPartResult, TestPartResult) +GTEST_REPEATER_METHOD_(OnEnvironmentsTearDownStart, UnitTest) +GTEST_REVERSE_REPEATER_METHOD_(OnEnvironmentsSetUpEnd, UnitTest) +GTEST_REVERSE_REPEATER_METHOD_(OnEnvironmentsTearDownEnd, UnitTest) +GTEST_REVERSE_REPEATER_METHOD_(OnTestEnd, TestInfo) +GTEST_REVERSE_REPEATER_METHOD_(OnTestCaseEnd, TestCase) +GTEST_REVERSE_REPEATER_METHOD_(OnTestProgramEnd, UnitTest) + +#undef GTEST_REPEATER_METHOD_ +#undef GTEST_REVERSE_REPEATER_METHOD_ + +void TestEventRepeater::OnTestIterationStart(const UnitTest& unit_test, + int iteration) { + if (forwarding_enabled_) { + for (size_t i = 0; i < listeners_.size(); i++) { + listeners_[i]->OnTestIterationStart(unit_test, iteration); + } + } +} + +void TestEventRepeater::OnTestIterationEnd(const UnitTest& unit_test, + int iteration) { + if (forwarding_enabled_) { + for (int i = static_cast(listeners_.size()) - 1; i >= 0; i--) { + listeners_[i]->OnTestIterationEnd(unit_test, iteration); + } + } +} + +// End TestEventRepeater + +// This class generates an XML output file. +class XmlUnitTestResultPrinter : public EmptyTestEventListener { + public: + explicit XmlUnitTestResultPrinter(const char* output_file); + + virtual void OnTestIterationEnd(const UnitTest& unit_test, int iteration); + + private: + // Is c a whitespace character that is normalized to a space character + // when it appears in an XML attribute value? + static bool IsNormalizableWhitespace(char c) { + return c == 0x9 || c == 0xA || c == 0xD; + } + + // May c appear in a well-formed XML document? + static bool IsValidXmlCharacter(char c) { + return IsNormalizableWhitespace(c) || c >= 0x20; + } + + // Returns an XML-escaped copy of the input string str. If + // is_attribute is true, the text is meant to appear as an attribute + // value, and normalizable whitespace is preserved by replacing it + // with character references. + static std::string EscapeXml(const std::string& str, bool is_attribute); + + // Returns the given string with all characters invalid in XML removed. + static std::string RemoveInvalidXmlCharacters(const std::string& str); + + // Convenience wrapper around EscapeXml when str is an attribute value. + static std::string EscapeXmlAttribute(const std::string& str) { + return EscapeXml(str, true); + } + + // Convenience wrapper around EscapeXml when str is not an attribute value. + static std::string EscapeXmlText(const char* str) { + return EscapeXml(str, false); + } + + // Verifies that the given attribute belongs to the given element and + // streams the attribute as XML. + static void OutputXmlAttribute(std::ostream* stream, + const std::string& element_name, + const std::string& name, + const std::string& value); + + // Streams an XML CDATA section, escaping invalid CDATA sequences as needed. + static void OutputXmlCDataSection(::std::ostream* stream, const char* data); + + // Streams an XML representation of a TestInfo object. + static void OutputXmlTestInfo(::std::ostream* stream, + const char* test_case_name, + const TestInfo& test_info); + + // Prints an XML representation of a TestCase object + static void PrintXmlTestCase(::std::ostream* stream, + const TestCase& test_case); + + // Prints an XML summary of unit_test to output stream out. + static void PrintXmlUnitTest(::std::ostream* stream, + const UnitTest& unit_test); + + // Produces a string representing the test properties in a result as space + // delimited XML attributes based on the property key="value" pairs. + // When the std::string is not empty, it includes a space at the beginning, + // to delimit this attribute from prior attributes. + static std::string TestPropertiesAsXmlAttributes(const TestResult& result); + + // The output file. + const std::string output_file_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(XmlUnitTestResultPrinter); +}; + +// Creates a new XmlUnitTestResultPrinter. +XmlUnitTestResultPrinter::XmlUnitTestResultPrinter(const char* output_file) + : output_file_(output_file) { + if (output_file_.c_str() == NULL || output_file_.empty()) { + fprintf(stderr, "XML output file may not be null\n"); + fflush(stderr); + exit(EXIT_FAILURE); + } +} + +// Called after the unit test ends. +void XmlUnitTestResultPrinter::OnTestIterationEnd(const UnitTest& unit_test, + int /*iteration*/) { + FILE* xmlout = NULL; + FilePath output_file(output_file_); + FilePath output_dir(output_file.RemoveFileName()); + + if (output_dir.CreateDirectoriesRecursively()) { + xmlout = posix::FOpen(output_file_.c_str(), "w"); + } + if (xmlout == NULL) { + // TODO(wan): report the reason of the failure. + // + // We don't do it for now as: + // + // 1. There is no urgent need for it. + // 2. It's a bit involved to make the errno variable thread-safe on + // all three operating systems (Linux, Windows, and Mac OS). + // 3. To interpret the meaning of errno in a thread-safe way, + // we need the strerror_r() function, which is not available on + // Windows. + fprintf(stderr, + "Unable to open file \"%s\"\n", + output_file_.c_str()); + fflush(stderr); + exit(EXIT_FAILURE); + } + std::stringstream stream; + PrintXmlUnitTest(&stream, unit_test); + fprintf(xmlout, "%s", StringStreamToString(&stream).c_str()); + fclose(xmlout); +} + +// Returns an XML-escaped copy of the input string str. If is_attribute +// is true, the text is meant to appear as an attribute value, and +// normalizable whitespace is preserved by replacing it with character +// references. +// +// Invalid XML characters in str, if any, are stripped from the output. +// It is expected that most, if not all, of the text processed by this +// module will consist of ordinary English text. +// If this module is ever modified to produce version 1.1 XML output, +// most invalid characters can be retained using character references. +// TODO(wan): It might be nice to have a minimally invasive, human-readable +// escaping scheme for invalid characters, rather than dropping them. +std::string XmlUnitTestResultPrinter::EscapeXml( + const std::string& str, bool is_attribute) { + Message m; + + for (size_t i = 0; i < str.size(); ++i) { + const char ch = str[i]; + switch (ch) { + case '<': + m << "<"; + break; + case '>': + m << ">"; + break; + case '&': + m << "&"; + break; + case '\'': + if (is_attribute) + m << "'"; + else + m << '\''; + break; + case '"': + if (is_attribute) + m << """; + else + m << '"'; + break; + default: + if (IsValidXmlCharacter(ch)) { + if (is_attribute && IsNormalizableWhitespace(ch)) + m << "&#x" << String::FormatByte(static_cast(ch)) + << ";"; + else + m << ch; + } + break; + } + } + + return m.GetString(); +} + +// Returns the given string with all characters invalid in XML removed. +// Currently invalid characters are dropped from the string. An +// alternative is to replace them with certain characters such as . or ?. +std::string XmlUnitTestResultPrinter::RemoveInvalidXmlCharacters( + const std::string& str) { + std::string output; + output.reserve(str.size()); + for (std::string::const_iterator it = str.begin(); it != str.end(); ++it) + if (IsValidXmlCharacter(*it)) + output.push_back(*it); + + return output; +} + +// The following routines generate an XML representation of a UnitTest +// object. +// +// This is how Google Test concepts map to the DTD: +// +// <-- corresponds to a UnitTest object +// <-- corresponds to a TestCase object +// <-- corresponds to a TestInfo object +// ... +// ... +// ... +// <-- individual assertion failures +// +// +// + +// Formats the given time in milliseconds as seconds. +std::string FormatTimeInMillisAsSeconds(TimeInMillis ms) { + ::std::stringstream ss; + ss << ms/1000.0; + return ss.str(); +} + +static bool PortableLocaltime(time_t seconds, struct tm* out) { +#if defined(_MSC_VER) + return localtime_s(out, &seconds) == 0; +#elif defined(__MINGW32__) || defined(__MINGW64__) + // MINGW provides neither localtime_r nor localtime_s, but uses + // Windows' localtime(), which has a thread-local tm buffer. + struct tm* tm_ptr = localtime(&seconds); // NOLINT + if (tm_ptr == NULL) + return false; + *out = *tm_ptr; + return true; +#else + return localtime_r(&seconds, out) != NULL; +#endif +} + +// Converts the given epoch time in milliseconds to a date string in the ISO +// 8601 format, without the timezone information. +std::string FormatEpochTimeInMillisAsIso8601(TimeInMillis ms) { + struct tm time_struct; + if (!PortableLocaltime(static_cast(ms / 1000), &time_struct)) + return ""; + // YYYY-MM-DDThh:mm:ss + return StreamableToString(time_struct.tm_year + 1900) + "-" + + String::FormatIntWidth2(time_struct.tm_mon + 1) + "-" + + String::FormatIntWidth2(time_struct.tm_mday) + "T" + + String::FormatIntWidth2(time_struct.tm_hour) + ":" + + String::FormatIntWidth2(time_struct.tm_min) + ":" + + String::FormatIntWidth2(time_struct.tm_sec); +} + +// Streams an XML CDATA section, escaping invalid CDATA sequences as needed. +void XmlUnitTestResultPrinter::OutputXmlCDataSection(::std::ostream* stream, + const char* data) { + const char* segment = data; + *stream << ""); + if (next_segment != NULL) { + stream->write( + segment, static_cast(next_segment - segment)); + *stream << "]]>]]>"); + } else { + *stream << segment; + break; + } + } + *stream << "]]>"; +} + +void XmlUnitTestResultPrinter::OutputXmlAttribute( + std::ostream* stream, + const std::string& element_name, + const std::string& name, + const std::string& value) { + const std::vector& allowed_names = + GetReservedAttributesForElement(element_name); + + GTEST_CHECK_(std::find(allowed_names.begin(), allowed_names.end(), name) != + allowed_names.end()) + << "Attribute " << name << " is not allowed for element <" << element_name + << ">."; + + *stream << " " << name << "=\"" << EscapeXmlAttribute(value) << "\""; +} + +// Prints an XML representation of a TestInfo object. +// TODO(wan): There is also value in printing properties with the plain printer. +void XmlUnitTestResultPrinter::OutputXmlTestInfo(::std::ostream* stream, + const char* test_case_name, + const TestInfo& test_info) { + const TestResult& result = *test_info.result(); + const std::string kTestcase = "testcase"; + + *stream << " \n"; + } + const string location = internal::FormatCompilerIndependentFileLocation( + part.file_name(), part.line_number()); + const string summary = location + "\n" + part.summary(); + *stream << " "; + const string detail = location + "\n" + part.message(); + OutputXmlCDataSection(stream, RemoveInvalidXmlCharacters(detail).c_str()); + *stream << "\n"; + } + } + + if (failures == 0) + *stream << " />\n"; + else + *stream << " \n"; +} + +// Prints an XML representation of a TestCase object +void XmlUnitTestResultPrinter::PrintXmlTestCase(std::ostream* stream, + const TestCase& test_case) { + const std::string kTestsuite = "testsuite"; + *stream << " <" << kTestsuite; + OutputXmlAttribute(stream, kTestsuite, "name", test_case.name()); + OutputXmlAttribute(stream, kTestsuite, "tests", + StreamableToString(test_case.reportable_test_count())); + OutputXmlAttribute(stream, kTestsuite, "failures", + StreamableToString(test_case.failed_test_count())); + OutputXmlAttribute( + stream, kTestsuite, "disabled", + StreamableToString(test_case.reportable_disabled_test_count())); + OutputXmlAttribute(stream, kTestsuite, "errors", "0"); + OutputXmlAttribute(stream, kTestsuite, "time", + FormatTimeInMillisAsSeconds(test_case.elapsed_time())); + *stream << TestPropertiesAsXmlAttributes(test_case.ad_hoc_test_result()) + << ">\n"; + + for (int i = 0; i < test_case.total_test_count(); ++i) { + if (test_case.GetTestInfo(i)->is_reportable()) + OutputXmlTestInfo(stream, test_case.name(), *test_case.GetTestInfo(i)); + } + *stream << " \n"; +} + +// Prints an XML summary of unit_test to output stream out. +void XmlUnitTestResultPrinter::PrintXmlUnitTest(std::ostream* stream, + const UnitTest& unit_test) { + const std::string kTestsuites = "testsuites"; + + *stream << "\n"; + *stream << "<" << kTestsuites; + + OutputXmlAttribute(stream, kTestsuites, "tests", + StreamableToString(unit_test.reportable_test_count())); + OutputXmlAttribute(stream, kTestsuites, "failures", + StreamableToString(unit_test.failed_test_count())); + OutputXmlAttribute( + stream, kTestsuites, "disabled", + StreamableToString(unit_test.reportable_disabled_test_count())); + OutputXmlAttribute(stream, kTestsuites, "errors", "0"); + OutputXmlAttribute( + stream, kTestsuites, "timestamp", + FormatEpochTimeInMillisAsIso8601(unit_test.start_timestamp())); + OutputXmlAttribute(stream, kTestsuites, "time", + FormatTimeInMillisAsSeconds(unit_test.elapsed_time())); + + if (GTEST_FLAG(shuffle)) { + OutputXmlAttribute(stream, kTestsuites, "random_seed", + StreamableToString(unit_test.random_seed())); + } + + *stream << TestPropertiesAsXmlAttributes(unit_test.ad_hoc_test_result()); + + OutputXmlAttribute(stream, kTestsuites, "name", "AllTests"); + *stream << ">\n"; + + for (int i = 0; i < unit_test.total_test_case_count(); ++i) { + if (unit_test.GetTestCase(i)->reportable_test_count() > 0) + PrintXmlTestCase(stream, *unit_test.GetTestCase(i)); + } + *stream << "\n"; +} + +// Produces a string representing the test properties in a result as space +// delimited XML attributes based on the property key="value" pairs. +std::string XmlUnitTestResultPrinter::TestPropertiesAsXmlAttributes( + const TestResult& result) { + Message attributes; + for (int i = 0; i < result.test_property_count(); ++i) { + const TestProperty& property = result.GetTestProperty(i); + attributes << " " << property.key() << "=" + << "\"" << EscapeXmlAttribute(property.value()) << "\""; + } + return attributes.GetString(); +} + +// End XmlUnitTestResultPrinter + +#if GTEST_CAN_STREAM_RESULTS_ + +// Checks if str contains '=', '&', '%' or '\n' characters. If yes, +// replaces them by "%xx" where xx is their hexadecimal value. For +// example, replaces "=" with "%3D". This algorithm is O(strlen(str)) +// in both time and space -- important as the input str may contain an +// arbitrarily long test failure message and stack trace. +string StreamingListener::UrlEncode(const char* str) { + string result; + result.reserve(strlen(str) + 1); + for (char ch = *str; ch != '\0'; ch = *++str) { + switch (ch) { + case '%': + case '=': + case '&': + case '\n': + result.append("%" + String::FormatByte(static_cast(ch))); + break; + default: + result.push_back(ch); + break; + } + } + return result; +} + +void StreamingListener::SocketWriter::MakeConnection() { + GTEST_CHECK_(sockfd_ == -1) + << "MakeConnection() can't be called when there is already a connection."; + + addrinfo hints; + memset(&hints, 0, sizeof(hints)); + hints.ai_family = AF_UNSPEC; // To allow both IPv4 and IPv6 addresses. + hints.ai_socktype = SOCK_STREAM; + addrinfo* servinfo = NULL; + + // Use the getaddrinfo() to get a linked list of IP addresses for + // the given host name. + const int error_num = getaddrinfo( + host_name_.c_str(), port_num_.c_str(), &hints, &servinfo); + if (error_num != 0) { + GTEST_LOG_(WARNING) << "stream_result_to: getaddrinfo() failed: " + << gai_strerror(error_num); + } + + // Loop through all the results and connect to the first we can. + for (addrinfo* cur_addr = servinfo; sockfd_ == -1 && cur_addr != NULL; + cur_addr = cur_addr->ai_next) { + sockfd_ = socket( + cur_addr->ai_family, cur_addr->ai_socktype, cur_addr->ai_protocol); + if (sockfd_ != -1) { + // Connect the client socket to the server socket. + if (connect(sockfd_, cur_addr->ai_addr, cur_addr->ai_addrlen) == -1) { + close(sockfd_); + sockfd_ = -1; + } + } + } + + freeaddrinfo(servinfo); // all done with this structure + + if (sockfd_ == -1) { + GTEST_LOG_(WARNING) << "stream_result_to: failed to connect to " + << host_name_ << ":" << port_num_; + } +} + +// End of class Streaming Listener +#endif // GTEST_CAN_STREAM_RESULTS__ + +// Class ScopedTrace + +// Pushes the given source file location and message onto a per-thread +// trace stack maintained by Google Test. +ScopedTrace::ScopedTrace(const char* file, int line, const Message& message) + GTEST_LOCK_EXCLUDED_(&UnitTest::mutex_) { + TraceInfo trace; + trace.file = file; + trace.line = line; + trace.message = message.GetString(); + + UnitTest::GetInstance()->PushGTestTrace(trace); +} + +// Pops the info pushed by the c'tor. +ScopedTrace::~ScopedTrace() + GTEST_LOCK_EXCLUDED_(&UnitTest::mutex_) { + UnitTest::GetInstance()->PopGTestTrace(); +} + + +// class OsStackTraceGetter + +// Returns the current OS stack trace as an std::string. Parameters: +// +// max_depth - the maximum number of stack frames to be included +// in the trace. +// skip_count - the number of top frames to be skipped; doesn't count +// against max_depth. +// +string OsStackTraceGetter::CurrentStackTrace(int /* max_depth */, + int /* skip_count */) + GTEST_LOCK_EXCLUDED_(mutex_) { + return ""; +} + +void OsStackTraceGetter::UponLeavingGTest() + GTEST_LOCK_EXCLUDED_(mutex_) { +} + +const char* const +OsStackTraceGetter::kElidedFramesMarker = + "... " GTEST_NAME_ " internal frames ..."; + +// A helper class that creates the premature-exit file in its +// constructor and deletes the file in its destructor. +class ScopedPrematureExitFile { + public: + explicit ScopedPrematureExitFile(const char* premature_exit_filepath) + : premature_exit_filepath_(premature_exit_filepath) { + // If a path to the premature-exit file is specified... + if (premature_exit_filepath != NULL && *premature_exit_filepath != '\0') { + // create the file with a single "0" character in it. I/O + // errors are ignored as there's nothing better we can do and we + // don't want to fail the test because of this. + FILE* pfile = posix::FOpen(premature_exit_filepath, "w"); + fwrite("0", 1, 1, pfile); + fclose(pfile); + } + } + + ~ScopedPrematureExitFile() { + if (premature_exit_filepath_ != NULL && *premature_exit_filepath_ != '\0') { + remove(premature_exit_filepath_); + } + } + + private: + const char* const premature_exit_filepath_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(ScopedPrematureExitFile); +}; + +} // namespace internal + +// class TestEventListeners + +TestEventListeners::TestEventListeners() + : repeater_(new internal::TestEventRepeater()), + default_result_printer_(NULL), + default_xml_generator_(NULL) { +} + +TestEventListeners::~TestEventListeners() { delete repeater_; } + +// Returns the standard listener responsible for the default console +// output. Can be removed from the listeners list to shut down default +// console output. Note that removing this object from the listener list +// with Release transfers its ownership to the user. +void TestEventListeners::Append(TestEventListener* listener) { + repeater_->Append(listener); +} + +// Removes the given event listener from the list and returns it. It then +// becomes the caller's responsibility to delete the listener. Returns +// NULL if the listener is not found in the list. +TestEventListener* TestEventListeners::Release(TestEventListener* listener) { + if (listener == default_result_printer_) + default_result_printer_ = NULL; + else if (listener == default_xml_generator_) + default_xml_generator_ = NULL; + return repeater_->Release(listener); +} + +// Returns repeater that broadcasts the TestEventListener events to all +// subscribers. +TestEventListener* TestEventListeners::repeater() { return repeater_; } + +// Sets the default_result_printer attribute to the provided listener. +// The listener is also added to the listener list and previous +// default_result_printer is removed from it and deleted. The listener can +// also be NULL in which case it will not be added to the list. Does +// nothing if the previous and the current listener objects are the same. +void TestEventListeners::SetDefaultResultPrinter(TestEventListener* listener) { + if (default_result_printer_ != listener) { + // It is an error to pass this method a listener that is already in the + // list. + delete Release(default_result_printer_); + default_result_printer_ = listener; + if (listener != NULL) + Append(listener); + } +} + +// Sets the default_xml_generator attribute to the provided listener. The +// listener is also added to the listener list and previous +// default_xml_generator is removed from it and deleted. The listener can +// also be NULL in which case it will not be added to the list. Does +// nothing if the previous and the current listener objects are the same. +void TestEventListeners::SetDefaultXmlGenerator(TestEventListener* listener) { + if (default_xml_generator_ != listener) { + // It is an error to pass this method a listener that is already in the + // list. + delete Release(default_xml_generator_); + default_xml_generator_ = listener; + if (listener != NULL) + Append(listener); + } +} + +// Controls whether events will be forwarded by the repeater to the +// listeners in the list. +bool TestEventListeners::EventForwardingEnabled() const { + return repeater_->forwarding_enabled(); +} + +void TestEventListeners::SuppressEventForwarding() { + repeater_->set_forwarding_enabled(false); +} + +// class UnitTest + +// Gets the singleton UnitTest object. The first time this method is +// called, a UnitTest object is constructed and returned. Consecutive +// calls will return the same object. +// +// We don't protect this under mutex_ as a user is not supposed to +// call this before main() starts, from which point on the return +// value will never change. +UnitTest* UnitTest::GetInstance() { + // When compiled with MSVC 7.1 in optimized mode, destroying the + // UnitTest object upon exiting the program messes up the exit code, + // causing successful tests to appear failed. We have to use a + // different implementation in this case to bypass the compiler bug. + // This implementation makes the compiler happy, at the cost of + // leaking the UnitTest object. + + // CodeGear C++Builder insists on a public destructor for the + // default implementation. Use this implementation to keep good OO + // design with private destructor. + +#if (_MSC_VER == 1310 && !defined(_DEBUG)) || defined(__BORLANDC__) + static UnitTest* const instance = new UnitTest; + return instance; +#else + static UnitTest instance; + return &instance; +#endif // (_MSC_VER == 1310 && !defined(_DEBUG)) || defined(__BORLANDC__) +} + +// Gets the number of successful test cases. +int UnitTest::successful_test_case_count() const { + return impl()->successful_test_case_count(); +} + +// Gets the number of failed test cases. +int UnitTest::failed_test_case_count() const { + return impl()->failed_test_case_count(); +} + +// Gets the number of all test cases. +int UnitTest::total_test_case_count() const { + return impl()->total_test_case_count(); +} + +// Gets the number of all test cases that contain at least one test +// that should run. +int UnitTest::test_case_to_run_count() const { + return impl()->test_case_to_run_count(); +} + +// Gets the number of successful tests. +int UnitTest::successful_test_count() const { + return impl()->successful_test_count(); +} + +// Gets the number of failed tests. +int UnitTest::failed_test_count() const { return impl()->failed_test_count(); } + +// Gets the number of disabled tests that will be reported in the XML report. +int UnitTest::reportable_disabled_test_count() const { + return impl()->reportable_disabled_test_count(); +} + +// Gets the number of disabled tests. +int UnitTest::disabled_test_count() const { + return impl()->disabled_test_count(); +} + +// Gets the number of tests to be printed in the XML report. +int UnitTest::reportable_test_count() const { + return impl()->reportable_test_count(); +} + +// Gets the number of all tests. +int UnitTest::total_test_count() const { return impl()->total_test_count(); } + +// Gets the number of tests that should run. +int UnitTest::test_to_run_count() const { return impl()->test_to_run_count(); } + +// Gets the time of the test program start, in ms from the start of the +// UNIX epoch. +internal::TimeInMillis UnitTest::start_timestamp() const { + return impl()->start_timestamp(); +} + +// Gets the elapsed time, in milliseconds. +internal::TimeInMillis UnitTest::elapsed_time() const { + return impl()->elapsed_time(); +} + +// Returns true iff the unit test passed (i.e. all test cases passed). +bool UnitTest::Passed() const { return impl()->Passed(); } + +// Returns true iff the unit test failed (i.e. some test case failed +// or something outside of all tests failed). +bool UnitTest::Failed() const { return impl()->Failed(); } + +// Gets the i-th test case among all the test cases. i can range from 0 to +// total_test_case_count() - 1. If i is not in that range, returns NULL. +const TestCase* UnitTest::GetTestCase(int i) const { + return impl()->GetTestCase(i); +} + +// Returns the TestResult containing information on test failures and +// properties logged outside of individual test cases. +const TestResult& UnitTest::ad_hoc_test_result() const { + return *impl()->ad_hoc_test_result(); +} + +// Gets the i-th test case among all the test cases. i can range from 0 to +// total_test_case_count() - 1. If i is not in that range, returns NULL. +TestCase* UnitTest::GetMutableTestCase(int i) { + return impl()->GetMutableTestCase(i); +} + +// Returns the list of event listeners that can be used to track events +// inside Google Test. +TestEventListeners& UnitTest::listeners() { + return *impl()->listeners(); +} + +// Registers and returns a global test environment. When a test +// program is run, all global test environments will be set-up in the +// order they were registered. After all tests in the program have +// finished, all global test environments will be torn-down in the +// *reverse* order they were registered. +// +// The UnitTest object takes ownership of the given environment. +// +// We don't protect this under mutex_, as we only support calling it +// from the main thread. +Environment* UnitTest::AddEnvironment(Environment* env) { + if (env == NULL) { + return NULL; + } + + impl_->environments().push_back(env); + return env; +} + +// Adds a TestPartResult to the current TestResult object. All Google Test +// assertion macros (e.g. ASSERT_TRUE, EXPECT_EQ, etc) eventually call +// this to report their results. The user code should use the +// assertion macros instead of calling this directly. +void UnitTest::AddTestPartResult( + TestPartResult::Type result_type, + const char* file_name, + int line_number, + const std::string& message, + const std::string& os_stack_trace) GTEST_LOCK_EXCLUDED_(mutex_) { + Message msg; + msg << message; + + internal::MutexLock lock(&mutex_); + if (impl_->gtest_trace_stack().size() > 0) { + msg << "\n" << GTEST_NAME_ << " trace:"; + + for (int i = static_cast(impl_->gtest_trace_stack().size()); + i > 0; --i) { + const internal::TraceInfo& trace = impl_->gtest_trace_stack()[i - 1]; + msg << "\n" << internal::FormatFileLocation(trace.file, trace.line) + << " " << trace.message; + } + } + + if (os_stack_trace.c_str() != NULL && !os_stack_trace.empty()) { + msg << internal::kStackTraceMarker << os_stack_trace; + } + + const TestPartResult result = + TestPartResult(result_type, file_name, line_number, + msg.GetString().c_str()); + impl_->GetTestPartResultReporterForCurrentThread()-> + ReportTestPartResult(result); + + if (result_type != TestPartResult::kSuccess) { + // gtest_break_on_failure takes precedence over + // gtest_throw_on_failure. This allows a user to set the latter + // in the code (perhaps in order to use Google Test assertions + // with another testing framework) and specify the former on the + // command line for debugging. + if (GTEST_FLAG(break_on_failure)) { +#if GTEST_OS_WINDOWS && !GTEST_OS_WINDOWS_PHONE && !GTEST_OS_WINDOWS_RT + // Using DebugBreak on Windows allows gtest to still break into a debugger + // when a failure happens and both the --gtest_break_on_failure and + // the --gtest_catch_exceptions flags are specified. + DebugBreak(); +#else + // Dereference NULL through a volatile pointer to prevent the compiler + // from removing. We use this rather than abort() or __builtin_trap() for + // portability: Symbian doesn't implement abort() well, and some debuggers + // don't correctly trap abort(). + *static_cast(NULL) = 1; +#endif // GTEST_OS_WINDOWS + } else if (GTEST_FLAG(throw_on_failure)) { +#if GTEST_HAS_EXCEPTIONS + throw internal::GoogleTestFailureException(result); +#else + // We cannot call abort() as it generates a pop-up in debug mode + // that cannot be suppressed in VC 7.1 or below. + exit(1); +#endif + } + } +} + +// Adds a TestProperty to the current TestResult object when invoked from +// inside a test, to current TestCase's ad_hoc_test_result_ when invoked +// from SetUpTestCase or TearDownTestCase, or to the global property set +// when invoked elsewhere. If the result already contains a property with +// the same key, the value will be updated. +void UnitTest::RecordProperty(const std::string& key, + const std::string& value) { + impl_->RecordProperty(TestProperty(key, value)); +} + +// Runs all tests in this UnitTest object and prints the result. +// Returns 0 if successful, or 1 otherwise. +// +// We don't protect this under mutex_, as we only support calling it +// from the main thread. +int UnitTest::Run() { + const bool in_death_test_child_process = + internal::GTEST_FLAG(internal_run_death_test).length() > 0; + + // Google Test implements this protocol for catching that a test + // program exits before returning control to Google Test: + // + // 1. Upon start, Google Test creates a file whose absolute path + // is specified by the environment variable + // TEST_PREMATURE_EXIT_FILE. + // 2. When Google Test has finished its work, it deletes the file. + // + // This allows a test runner to set TEST_PREMATURE_EXIT_FILE before + // running a Google-Test-based test program and check the existence + // of the file at the end of the test execution to see if it has + // exited prematurely. + + // If we are in the child process of a death test, don't + // create/delete the premature exit file, as doing so is unnecessary + // and will confuse the parent process. Otherwise, create/delete + // the file upon entering/leaving this function. If the program + // somehow exits before this function has a chance to return, the + // premature-exit file will be left undeleted, causing a test runner + // that understands the premature-exit-file protocol to report the + // test as having failed. + const internal::ScopedPrematureExitFile premature_exit_file( + in_death_test_child_process ? + NULL : internal::posix::GetEnv("TEST_PREMATURE_EXIT_FILE")); + + // Captures the value of GTEST_FLAG(catch_exceptions). This value will be + // used for the duration of the program. + impl()->set_catch_exceptions(GTEST_FLAG(catch_exceptions)); + +#if GTEST_HAS_SEH + // Either the user wants Google Test to catch exceptions thrown by the + // tests or this is executing in the context of death test child + // process. In either case the user does not want to see pop-up dialogs + // about crashes - they are expected. + if (impl()->catch_exceptions() || in_death_test_child_process) { +# if !GTEST_OS_WINDOWS_MOBILE && !GTEST_OS_WINDOWS_PHONE && !GTEST_OS_WINDOWS_RT + // SetErrorMode doesn't exist on CE. + SetErrorMode(SEM_FAILCRITICALERRORS | SEM_NOALIGNMENTFAULTEXCEPT | + SEM_NOGPFAULTERRORBOX | SEM_NOOPENFILEERRORBOX); +# endif // !GTEST_OS_WINDOWS_MOBILE + +# if (defined(_MSC_VER) || GTEST_OS_WINDOWS_MINGW) && !GTEST_OS_WINDOWS_MOBILE + // Death test children can be terminated with _abort(). On Windows, + // _abort() can show a dialog with a warning message. This forces the + // abort message to go to stderr instead. + _set_error_mode(_OUT_TO_STDERR); +# endif + +# if _MSC_VER >= 1400 && !GTEST_OS_WINDOWS_MOBILE + // In the debug version, Visual Studio pops up a separate dialog + // offering a choice to debug the aborted program. We need to suppress + // this dialog or it will pop up for every EXPECT/ASSERT_DEATH statement + // executed. Google Test will notify the user of any unexpected + // failure via stderr. + // + // VC++ doesn't define _set_abort_behavior() prior to the version 8.0. + // Users of prior VC versions shall suffer the agony and pain of + // clicking through the countless debug dialogs. + // TODO(vladl@google.com): find a way to suppress the abort dialog() in the + // debug mode when compiled with VC 7.1 or lower. + if (!GTEST_FLAG(break_on_failure)) + _set_abort_behavior( + 0x0, // Clear the following flags: + _WRITE_ABORT_MSG | _CALL_REPORTFAULT); // pop-up window, core dump. +# endif + } +#endif // GTEST_HAS_SEH + + return internal::HandleExceptionsInMethodIfSupported( + impl(), + &internal::UnitTestImpl::RunAllTests, + "auxiliary test code (environments or event listeners)") ? 0 : 1; +} + +// Returns the working directory when the first TEST() or TEST_F() was +// executed. +const char* UnitTest::original_working_dir() const { + return impl_->original_working_dir_.c_str(); +} + +// Returns the TestCase object for the test that's currently running, +// or NULL if no test is running. +const TestCase* UnitTest::current_test_case() const + GTEST_LOCK_EXCLUDED_(mutex_) { + internal::MutexLock lock(&mutex_); + return impl_->current_test_case(); +} + +// Returns the TestInfo object for the test that's currently running, +// or NULL if no test is running. +const TestInfo* UnitTest::current_test_info() const + GTEST_LOCK_EXCLUDED_(mutex_) { + internal::MutexLock lock(&mutex_); + return impl_->current_test_info(); +} + +// Returns the random seed used at the start of the current test run. +int UnitTest::random_seed() const { return impl_->random_seed(); } + +#if GTEST_HAS_PARAM_TEST +// Returns ParameterizedTestCaseRegistry object used to keep track of +// value-parameterized tests and instantiate and register them. +internal::ParameterizedTestCaseRegistry& + UnitTest::parameterized_test_registry() + GTEST_LOCK_EXCLUDED_(mutex_) { + return impl_->parameterized_test_registry(); +} +#endif // GTEST_HAS_PARAM_TEST + +// Creates an empty UnitTest. +UnitTest::UnitTest() { + impl_ = new internal::UnitTestImpl(this); +} + +// Destructor of UnitTest. +UnitTest::~UnitTest() { + delete impl_; +} + +// Pushes a trace defined by SCOPED_TRACE() on to the per-thread +// Google Test trace stack. +void UnitTest::PushGTestTrace(const internal::TraceInfo& trace) + GTEST_LOCK_EXCLUDED_(mutex_) { + internal::MutexLock lock(&mutex_); + impl_->gtest_trace_stack().push_back(trace); +} + +// Pops a trace from the per-thread Google Test trace stack. +void UnitTest::PopGTestTrace() + GTEST_LOCK_EXCLUDED_(mutex_) { + internal::MutexLock lock(&mutex_); + impl_->gtest_trace_stack().pop_back(); +} + +namespace internal { + +UnitTestImpl::UnitTestImpl(UnitTest* parent) + : parent_(parent), + GTEST_DISABLE_MSC_WARNINGS_PUSH_(4355 /* using this in initializer */) + default_global_test_part_result_reporter_(this), + default_per_thread_test_part_result_reporter_(this), + GTEST_DISABLE_MSC_WARNINGS_POP_() + global_test_part_result_repoter_( + &default_global_test_part_result_reporter_), + per_thread_test_part_result_reporter_( + &default_per_thread_test_part_result_reporter_), +#if GTEST_HAS_PARAM_TEST + parameterized_test_registry_(), + parameterized_tests_registered_(false), +#endif // GTEST_HAS_PARAM_TEST + last_death_test_case_(-1), + current_test_case_(NULL), + current_test_info_(NULL), + ad_hoc_test_result_(), + os_stack_trace_getter_(NULL), + post_flag_parse_init_performed_(false), + random_seed_(0), // Will be overridden by the flag before first use. + random_(0), // Will be reseeded before first use. + start_timestamp_(0), + elapsed_time_(0), +#if GTEST_HAS_DEATH_TEST + death_test_factory_(new DefaultDeathTestFactory), +#endif + // Will be overridden by the flag before first use. + catch_exceptions_(false) { + listeners()->SetDefaultResultPrinter(new PrettyUnitTestResultPrinter); +} + +UnitTestImpl::~UnitTestImpl() { + // Deletes every TestCase. + ForEach(test_cases_, internal::Delete); + + // Deletes every Environment. + ForEach(environments_, internal::Delete); + + delete os_stack_trace_getter_; +} + +// Adds a TestProperty to the current TestResult object when invoked in a +// context of a test, to current test case's ad_hoc_test_result when invoke +// from SetUpTestCase/TearDownTestCase, or to the global property set +// otherwise. If the result already contains a property with the same key, +// the value will be updated. +void UnitTestImpl::RecordProperty(const TestProperty& test_property) { + std::string xml_element; + TestResult* test_result; // TestResult appropriate for property recording. + + if (current_test_info_ != NULL) { + xml_element = "testcase"; + test_result = &(current_test_info_->result_); + } else if (current_test_case_ != NULL) { + xml_element = "testsuite"; + test_result = &(current_test_case_->ad_hoc_test_result_); + } else { + xml_element = "testsuites"; + test_result = &ad_hoc_test_result_; + } + test_result->RecordProperty(xml_element, test_property); +} + +#if GTEST_HAS_DEATH_TEST +// Disables event forwarding if the control is currently in a death test +// subprocess. Must not be called before InitGoogleTest. +void UnitTestImpl::SuppressTestEventsIfInSubprocess() { + if (internal_run_death_test_flag_.get() != NULL) + listeners()->SuppressEventForwarding(); +} +#endif // GTEST_HAS_DEATH_TEST + +// Initializes event listeners performing XML output as specified by +// UnitTestOptions. Must not be called before InitGoogleTest. +void UnitTestImpl::ConfigureXmlOutput() { + const std::string& output_format = UnitTestOptions::GetOutputFormat(); + if (output_format == "xml") { + listeners()->SetDefaultXmlGenerator(new XmlUnitTestResultPrinter( + UnitTestOptions::GetAbsolutePathToOutputFile().c_str())); + } else if (output_format != "") { + printf("WARNING: unrecognized output format \"%s\" ignored.\n", + output_format.c_str()); + fflush(stdout); + } +} + +#if GTEST_CAN_STREAM_RESULTS_ +// Initializes event listeners for streaming test results in string form. +// Must not be called before InitGoogleTest. +void UnitTestImpl::ConfigureStreamingOutput() { + const std::string& target = GTEST_FLAG(stream_result_to); + if (!target.empty()) { + const size_t pos = target.find(':'); + if (pos != std::string::npos) { + listeners()->Append(new StreamingListener(target.substr(0, pos), + target.substr(pos+1))); + } else { + printf("WARNING: unrecognized streaming target \"%s\" ignored.\n", + target.c_str()); + fflush(stdout); + } + } +} +#endif // GTEST_CAN_STREAM_RESULTS_ + +// Performs initialization dependent upon flag values obtained in +// ParseGoogleTestFlagsOnly. Is called from InitGoogleTest after the call to +// ParseGoogleTestFlagsOnly. In case a user neglects to call InitGoogleTest +// this function is also called from RunAllTests. Since this function can be +// called more than once, it has to be idempotent. +void UnitTestImpl::PostFlagParsingInit() { + // Ensures that this function does not execute more than once. + if (!post_flag_parse_init_performed_) { + post_flag_parse_init_performed_ = true; + +#if GTEST_HAS_DEATH_TEST + InitDeathTestSubprocessControlInfo(); + SuppressTestEventsIfInSubprocess(); +#endif // GTEST_HAS_DEATH_TEST + + // Registers parameterized tests. This makes parameterized tests + // available to the UnitTest reflection API without running + // RUN_ALL_TESTS. + RegisterParameterizedTests(); + + // Configures listeners for XML output. This makes it possible for users + // to shut down the default XML output before invoking RUN_ALL_TESTS. + ConfigureXmlOutput(); + +#if GTEST_CAN_STREAM_RESULTS_ + // Configures listeners for streaming test results to the specified server. + ConfigureStreamingOutput(); +#endif // GTEST_CAN_STREAM_RESULTS_ + } +} + +// A predicate that checks the name of a TestCase against a known +// value. +// +// This is used for implementation of the UnitTest class only. We put +// it in the anonymous namespace to prevent polluting the outer +// namespace. +// +// TestCaseNameIs is copyable. +class TestCaseNameIs { + public: + // Constructor. + explicit TestCaseNameIs(const std::string& name) + : name_(name) {} + + // Returns true iff the name of test_case matches name_. + bool operator()(const TestCase* test_case) const { + return test_case != NULL && strcmp(test_case->name(), name_.c_str()) == 0; + } + + private: + std::string name_; +}; + +// Finds and returns a TestCase with the given name. If one doesn't +// exist, creates one and returns it. It's the CALLER'S +// RESPONSIBILITY to ensure that this function is only called WHEN THE +// TESTS ARE NOT SHUFFLED. +// +// Arguments: +// +// test_case_name: name of the test case +// type_param: the name of the test case's type parameter, or NULL if +// this is not a typed or a type-parameterized test case. +// set_up_tc: pointer to the function that sets up the test case +// tear_down_tc: pointer to the function that tears down the test case +TestCase* UnitTestImpl::GetTestCase(const char* test_case_name, + const char* type_param, + Test::SetUpTestCaseFunc set_up_tc, + Test::TearDownTestCaseFunc tear_down_tc) { + // Can we find a TestCase with the given name? + const std::vector::const_iterator test_case = + std::find_if(test_cases_.begin(), test_cases_.end(), + TestCaseNameIs(test_case_name)); + + if (test_case != test_cases_.end()) + return *test_case; + + // No. Let's create one. + TestCase* const new_test_case = + new TestCase(test_case_name, type_param, set_up_tc, tear_down_tc); + + // Is this a death test case? + if (internal::UnitTestOptions::MatchesFilter(test_case_name, + kDeathTestCaseFilter)) { + // Yes. Inserts the test case after the last death test case + // defined so far. This only works when the test cases haven't + // been shuffled. Otherwise we may end up running a death test + // after a non-death test. + ++last_death_test_case_; + test_cases_.insert(test_cases_.begin() + last_death_test_case_, + new_test_case); + } else { + // No. Appends to the end of the list. + test_cases_.push_back(new_test_case); + } + + test_case_indices_.push_back(static_cast(test_case_indices_.size())); + return new_test_case; +} + +// Helpers for setting up / tearing down the given environment. They +// are for use in the ForEach() function. +static void SetUpEnvironment(Environment* env) { env->SetUp(); } +static void TearDownEnvironment(Environment* env) { env->TearDown(); } + +// Runs all tests in this UnitTest object, prints the result, and +// returns true if all tests are successful. If any exception is +// thrown during a test, the test is considered to be failed, but the +// rest of the tests will still be run. +// +// When parameterized tests are enabled, it expands and registers +// parameterized tests first in RegisterParameterizedTests(). +// All other functions called from RunAllTests() may safely assume that +// parameterized tests are ready to be counted and run. +bool UnitTestImpl::RunAllTests() { + // Makes sure InitGoogleTest() was called. + if (!GTestIsInitialized()) { + printf("%s", + "\nThis test program did NOT call ::testing::InitGoogleTest " + "before calling RUN_ALL_TESTS(). Please fix it.\n"); + return false; + } + + // Do not run any test if the --help flag was specified. + if (g_help_flag) + return true; + + // Repeats the call to the post-flag parsing initialization in case the + // user didn't call InitGoogleTest. + PostFlagParsingInit(); + + // Even if sharding is not on, test runners may want to use the + // GTEST_SHARD_STATUS_FILE to query whether the test supports the sharding + // protocol. + internal::WriteToShardStatusFileIfNeeded(); + + // True iff we are in a subprocess for running a thread-safe-style + // death test. + bool in_subprocess_for_death_test = false; + +#if GTEST_HAS_DEATH_TEST + in_subprocess_for_death_test = (internal_run_death_test_flag_.get() != NULL); +#endif // GTEST_HAS_DEATH_TEST + + const bool should_shard = ShouldShard(kTestTotalShards, kTestShardIndex, + in_subprocess_for_death_test); + + // Compares the full test names with the filter to decide which + // tests to run. + const bool has_tests_to_run = FilterTests(should_shard + ? HONOR_SHARDING_PROTOCOL + : IGNORE_SHARDING_PROTOCOL) > 0; + + // Lists the tests and exits if the --gtest_list_tests flag was specified. + if (GTEST_FLAG(list_tests)) { + // This must be called *after* FilterTests() has been called. + ListTestsMatchingFilter(); + return true; + } + + random_seed_ = GTEST_FLAG(shuffle) ? + GetRandomSeedFromFlag(GTEST_FLAG(random_seed)) : 0; + + // True iff at least one test has failed. + bool failed = false; + + TestEventListener* repeater = listeners()->repeater(); + + start_timestamp_ = GetTimeInMillis(); + repeater->OnTestProgramStart(*parent_); + + // How many times to repeat the tests? We don't want to repeat them + // when we are inside the subprocess of a death test. + const int repeat = in_subprocess_for_death_test ? 1 : GTEST_FLAG(repeat); + // Repeats forever if the repeat count is negative. + const bool forever = repeat < 0; + for (int i = 0; forever || i != repeat; i++) { + // We want to preserve failures generated by ad-hoc test + // assertions executed before RUN_ALL_TESTS(). + ClearNonAdHocTestResult(); + + const TimeInMillis start = GetTimeInMillis(); + + // Shuffles test cases and tests if requested. + if (has_tests_to_run && GTEST_FLAG(shuffle)) { + random()->Reseed(random_seed_); + // This should be done before calling OnTestIterationStart(), + // such that a test event listener can see the actual test order + // in the event. + ShuffleTests(); + } + + // Tells the unit test event listeners that the tests are about to start. + repeater->OnTestIterationStart(*parent_, i); + + // Runs each test case if there is at least one test to run. + if (has_tests_to_run) { + // Sets up all environments beforehand. + repeater->OnEnvironmentsSetUpStart(*parent_); + ForEach(environments_, SetUpEnvironment); + repeater->OnEnvironmentsSetUpEnd(*parent_); + + // Runs the tests only if there was no fatal failure during global + // set-up. + if (!Test::HasFatalFailure()) { + for (int test_index = 0; test_index < total_test_case_count(); + test_index++) { + GetMutableTestCase(test_index)->Run(); + } + } + + // Tears down all environments in reverse order afterwards. + repeater->OnEnvironmentsTearDownStart(*parent_); + std::for_each(environments_.rbegin(), environments_.rend(), + TearDownEnvironment); + repeater->OnEnvironmentsTearDownEnd(*parent_); + } + + elapsed_time_ = GetTimeInMillis() - start; + + // Tells the unit test event listener that the tests have just finished. + repeater->OnTestIterationEnd(*parent_, i); + + // Gets the result and clears it. + if (!Passed()) { + failed = true; + } + + // Restores the original test order after the iteration. This + // allows the user to quickly repro a failure that happens in the + // N-th iteration without repeating the first (N - 1) iterations. + // This is not enclosed in "if (GTEST_FLAG(shuffle)) { ... }", in + // case the user somehow changes the value of the flag somewhere + // (it's always safe to unshuffle the tests). + UnshuffleTests(); + + if (GTEST_FLAG(shuffle)) { + // Picks a new random seed for each iteration. + random_seed_ = GetNextRandomSeed(random_seed_); + } + } + + repeater->OnTestProgramEnd(*parent_); + + return !failed; +} + +// Reads the GTEST_SHARD_STATUS_FILE environment variable, and creates the file +// if the variable is present. If a file already exists at this location, this +// function will write over it. If the variable is present, but the file cannot +// be created, prints an error and exits. +void WriteToShardStatusFileIfNeeded() { + const char* const test_shard_file = posix::GetEnv(kTestShardStatusFile); + if (test_shard_file != NULL) { + FILE* const file = posix::FOpen(test_shard_file, "w"); + if (file == NULL) { + ColoredPrintf(COLOR_RED, + "Could not write to the test shard status file \"%s\" " + "specified by the %s environment variable.\n", + test_shard_file, kTestShardStatusFile); + fflush(stdout); + exit(EXIT_FAILURE); + } + fclose(file); + } +} + +// Checks whether sharding is enabled by examining the relevant +// environment variable values. If the variables are present, +// but inconsistent (i.e., shard_index >= total_shards), prints +// an error and exits. If in_subprocess_for_death_test, sharding is +// disabled because it must only be applied to the original test +// process. Otherwise, we could filter out death tests we intended to execute. +bool ShouldShard(const char* total_shards_env, + const char* shard_index_env, + bool in_subprocess_for_death_test) { + if (in_subprocess_for_death_test) { + return false; + } + + const Int32 total_shards = Int32FromEnvOrDie(total_shards_env, -1); + const Int32 shard_index = Int32FromEnvOrDie(shard_index_env, -1); + + if (total_shards == -1 && shard_index == -1) { + return false; + } else if (total_shards == -1 && shard_index != -1) { + const Message msg = Message() + << "Invalid environment variables: you have " + << kTestShardIndex << " = " << shard_index + << ", but have left " << kTestTotalShards << " unset.\n"; + ColoredPrintf(COLOR_RED, msg.GetString().c_str()); + fflush(stdout); + exit(EXIT_FAILURE); + } else if (total_shards != -1 && shard_index == -1) { + const Message msg = Message() + << "Invalid environment variables: you have " + << kTestTotalShards << " = " << total_shards + << ", but have left " << kTestShardIndex << " unset.\n"; + ColoredPrintf(COLOR_RED, msg.GetString().c_str()); + fflush(stdout); + exit(EXIT_FAILURE); + } else if (shard_index < 0 || shard_index >= total_shards) { + const Message msg = Message() + << "Invalid environment variables: we require 0 <= " + << kTestShardIndex << " < " << kTestTotalShards + << ", but you have " << kTestShardIndex << "=" << shard_index + << ", " << kTestTotalShards << "=" << total_shards << ".\n"; + ColoredPrintf(COLOR_RED, msg.GetString().c_str()); + fflush(stdout); + exit(EXIT_FAILURE); + } + + return total_shards > 1; +} + +// Parses the environment variable var as an Int32. If it is unset, +// returns default_val. If it is not an Int32, prints an error +// and aborts. +Int32 Int32FromEnvOrDie(const char* var, Int32 default_val) { + const char* str_val = posix::GetEnv(var); + if (str_val == NULL) { + return default_val; + } + + Int32 result; + if (!ParseInt32(Message() << "The value of environment variable " << var, + str_val, &result)) { + exit(EXIT_FAILURE); + } + return result; +} + +// Given the total number of shards, the shard index, and the test id, +// returns true iff the test should be run on this shard. The test id is +// some arbitrary but unique non-negative integer assigned to each test +// method. Assumes that 0 <= shard_index < total_shards. +bool ShouldRunTestOnShard(int total_shards, int shard_index, int test_id) { + return (test_id % total_shards) == shard_index; +} + +// Compares the name of each test with the user-specified filter to +// decide whether the test should be run, then records the result in +// each TestCase and TestInfo object. +// If shard_tests == true, further filters tests based on sharding +// variables in the environment - see +// http://code.google.com/p/googletest/wiki/GoogleTestAdvancedGuide. +// Returns the number of tests that should run. +int UnitTestImpl::FilterTests(ReactionToSharding shard_tests) { + const Int32 total_shards = shard_tests == HONOR_SHARDING_PROTOCOL ? + Int32FromEnvOrDie(kTestTotalShards, -1) : -1; + const Int32 shard_index = shard_tests == HONOR_SHARDING_PROTOCOL ? + Int32FromEnvOrDie(kTestShardIndex, -1) : -1; + + // num_runnable_tests are the number of tests that will + // run across all shards (i.e., match filter and are not disabled). + // num_selected_tests are the number of tests to be run on + // this shard. + int num_runnable_tests = 0; + int num_selected_tests = 0; + for (size_t i = 0; i < test_cases_.size(); i++) { + TestCase* const test_case = test_cases_[i]; + const std::string &test_case_name = test_case->name(); + test_case->set_should_run(false); + + for (size_t j = 0; j < test_case->test_info_list().size(); j++) { + TestInfo* const test_info = test_case->test_info_list()[j]; + const std::string test_name(test_info->name()); + // A test is disabled if test case name or test name matches + // kDisableTestFilter. + const bool is_disabled = + internal::UnitTestOptions::MatchesFilter(test_case_name, + kDisableTestFilter) || + internal::UnitTestOptions::MatchesFilter(test_name, + kDisableTestFilter); + test_info->is_disabled_ = is_disabled; + + const bool matches_filter = + internal::UnitTestOptions::FilterMatchesTest(test_case_name, + test_name); + test_info->matches_filter_ = matches_filter; + + const bool is_runnable = + (GTEST_FLAG(also_run_disabled_tests) || !is_disabled) && + matches_filter; + + const bool is_selected = is_runnable && + (shard_tests == IGNORE_SHARDING_PROTOCOL || + ShouldRunTestOnShard(total_shards, shard_index, + num_runnable_tests)); + + num_runnable_tests += is_runnable; + num_selected_tests += is_selected; + + test_info->should_run_ = is_selected; + test_case->set_should_run(test_case->should_run() || is_selected); + } + } + return num_selected_tests; +} + +// Prints the given C-string on a single line by replacing all '\n' +// characters with string "\\n". If the output takes more than +// max_length characters, only prints the first max_length characters +// and "...". +static void PrintOnOneLine(const char* str, int max_length) { + if (str != NULL) { + for (int i = 0; *str != '\0'; ++str) { + if (i >= max_length) { + printf("..."); + break; + } + if (*str == '\n') { + printf("\\n"); + i += 2; + } else { + printf("%c", *str); + ++i; + } + } + } +} + +// Prints the names of the tests matching the user-specified filter flag. +void UnitTestImpl::ListTestsMatchingFilter() { + // Print at most this many characters for each type/value parameter. + const int kMaxParamLength = 250; + + for (size_t i = 0; i < test_cases_.size(); i++) { + const TestCase* const test_case = test_cases_[i]; + bool printed_test_case_name = false; + + for (size_t j = 0; j < test_case->test_info_list().size(); j++) { + const TestInfo* const test_info = + test_case->test_info_list()[j]; + if (test_info->matches_filter_) { + if (!printed_test_case_name) { + printed_test_case_name = true; + printf("%s.", test_case->name()); + if (test_case->type_param() != NULL) { + printf(" # %s = ", kTypeParamLabel); + // We print the type parameter on a single line to make + // the output easy to parse by a program. + PrintOnOneLine(test_case->type_param(), kMaxParamLength); + } + printf("\n"); + } + printf(" %s", test_info->name()); + if (test_info->value_param() != NULL) { + printf(" # %s = ", kValueParamLabel); + // We print the value parameter on a single line to make the + // output easy to parse by a program. + PrintOnOneLine(test_info->value_param(), kMaxParamLength); + } + printf("\n"); + } + } + } + fflush(stdout); +} + +// Sets the OS stack trace getter. +// +// Does nothing if the input and the current OS stack trace getter are +// the same; otherwise, deletes the old getter and makes the input the +// current getter. +void UnitTestImpl::set_os_stack_trace_getter( + OsStackTraceGetterInterface* getter) { + if (os_stack_trace_getter_ != getter) { + delete os_stack_trace_getter_; + os_stack_trace_getter_ = getter; + } +} + +// Returns the current OS stack trace getter if it is not NULL; +// otherwise, creates an OsStackTraceGetter, makes it the current +// getter, and returns it. +OsStackTraceGetterInterface* UnitTestImpl::os_stack_trace_getter() { + if (os_stack_trace_getter_ == NULL) { + os_stack_trace_getter_ = new OsStackTraceGetter; + } + + return os_stack_trace_getter_; +} + +// Returns the TestResult for the test that's currently running, or +// the TestResult for the ad hoc test if no test is running. +TestResult* UnitTestImpl::current_test_result() { + return current_test_info_ ? + &(current_test_info_->result_) : &ad_hoc_test_result_; +} + +// Shuffles all test cases, and the tests within each test case, +// making sure that death tests are still run first. +void UnitTestImpl::ShuffleTests() { + // Shuffles the death test cases. + ShuffleRange(random(), 0, last_death_test_case_ + 1, &test_case_indices_); + + // Shuffles the non-death test cases. + ShuffleRange(random(), last_death_test_case_ + 1, + static_cast(test_cases_.size()), &test_case_indices_); + + // Shuffles the tests inside each test case. + for (size_t i = 0; i < test_cases_.size(); i++) { + test_cases_[i]->ShuffleTests(random()); + } +} + +// Restores the test cases and tests to their order before the first shuffle. +void UnitTestImpl::UnshuffleTests() { + for (size_t i = 0; i < test_cases_.size(); i++) { + // Unshuffles the tests in each test case. + test_cases_[i]->UnshuffleTests(); + // Resets the index of each test case. + test_case_indices_[i] = static_cast(i); + } +} + +// Returns the current OS stack trace as an std::string. +// +// The maximum number of stack frames to be included is specified by +// the gtest_stack_trace_depth flag. The skip_count parameter +// specifies the number of top frames to be skipped, which doesn't +// count against the number of frames to be included. +// +// For example, if Foo() calls Bar(), which in turn calls +// GetCurrentOsStackTraceExceptTop(..., 1), Foo() will be included in +// the trace but Bar() and GetCurrentOsStackTraceExceptTop() won't. +std::string GetCurrentOsStackTraceExceptTop(UnitTest* /*unit_test*/, + int skip_count) { + // We pass skip_count + 1 to skip this wrapper function in addition + // to what the user really wants to skip. + return GetUnitTestImpl()->CurrentOsStackTraceExceptTop(skip_count + 1); +} + +// Used by the GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_ macro to +// suppress unreachable code warnings. +namespace { +class ClassUniqueToAlwaysTrue {}; +} + +bool IsTrue(bool condition) { return condition; } + +bool AlwaysTrue() { +#if GTEST_HAS_EXCEPTIONS + // This condition is always false so AlwaysTrue() never actually throws, + // but it makes the compiler think that it may throw. + if (IsTrue(false)) + throw ClassUniqueToAlwaysTrue(); +#endif // GTEST_HAS_EXCEPTIONS + return true; +} + +// If *pstr starts with the given prefix, modifies *pstr to be right +// past the prefix and returns true; otherwise leaves *pstr unchanged +// and returns false. None of pstr, *pstr, and prefix can be NULL. +bool SkipPrefix(const char* prefix, const char** pstr) { + const size_t prefix_len = strlen(prefix); + if (strncmp(*pstr, prefix, prefix_len) == 0) { + *pstr += prefix_len; + return true; + } + return false; +} + +// Parses a string as a command line flag. The string should have +// the format "--flag=value". When def_optional is true, the "=value" +// part can be omitted. +// +// Returns the value of the flag, or NULL if the parsing failed. +const char* ParseFlagValue(const char* str, + const char* flag, + bool def_optional) { + // str and flag must not be NULL. + if (str == NULL || flag == NULL) return NULL; + + // The flag must start with "--" followed by GTEST_FLAG_PREFIX_. + const std::string flag_str = std::string("--") + GTEST_FLAG_PREFIX_ + flag; + const size_t flag_len = flag_str.length(); + if (strncmp(str, flag_str.c_str(), flag_len) != 0) return NULL; + + // Skips the flag name. + const char* flag_end = str + flag_len; + + // When def_optional is true, it's OK to not have a "=value" part. + if (def_optional && (flag_end[0] == '\0')) { + return flag_end; + } + + // If def_optional is true and there are more characters after the + // flag name, or if def_optional is false, there must be a '=' after + // the flag name. + if (flag_end[0] != '=') return NULL; + + // Returns the string after "=". + return flag_end + 1; +} + +// Parses a string for a bool flag, in the form of either +// "--flag=value" or "--flag". +// +// In the former case, the value is taken as true as long as it does +// not start with '0', 'f', or 'F'. +// +// In the latter case, the value is taken as true. +// +// On success, stores the value of the flag in *value, and returns +// true. On failure, returns false without changing *value. +bool ParseBoolFlag(const char* str, const char* flag, bool* value) { + // Gets the value of the flag as a string. + const char* const value_str = ParseFlagValue(str, flag, true); + + // Aborts if the parsing failed. + if (value_str == NULL) return false; + + // Converts the string value to a bool. + *value = !(*value_str == '0' || *value_str == 'f' || *value_str == 'F'); + return true; +} + +// Parses a string for an Int32 flag, in the form of +// "--flag=value". +// +// On success, stores the value of the flag in *value, and returns +// true. On failure, returns false without changing *value. +bool ParseInt32Flag(const char* str, const char* flag, Int32* value) { + // Gets the value of the flag as a string. + const char* const value_str = ParseFlagValue(str, flag, false); + + // Aborts if the parsing failed. + if (value_str == NULL) return false; + + // Sets *value to the value of the flag. + return ParseInt32(Message() << "The value of flag --" << flag, + value_str, value); +} + +// Parses a string for a string flag, in the form of +// "--flag=value". +// +// On success, stores the value of the flag in *value, and returns +// true. On failure, returns false without changing *value. +bool ParseStringFlag(const char* str, const char* flag, std::string* value) { + // Gets the value of the flag as a string. + const char* const value_str = ParseFlagValue(str, flag, false); + + // Aborts if the parsing failed. + if (value_str == NULL) return false; + + // Sets *value to the value of the flag. + *value = value_str; + return true; +} + +// Determines whether a string has a prefix that Google Test uses for its +// flags, i.e., starts with GTEST_FLAG_PREFIX_ or GTEST_FLAG_PREFIX_DASH_. +// If Google Test detects that a command line flag has its prefix but is not +// recognized, it will print its help message. Flags starting with +// GTEST_INTERNAL_PREFIX_ followed by "internal_" are considered Google Test +// internal flags and do not trigger the help message. +static bool HasGoogleTestFlagPrefix(const char* str) { + return (SkipPrefix("--", &str) || + SkipPrefix("-", &str) || + SkipPrefix("/", &str)) && + !SkipPrefix(GTEST_FLAG_PREFIX_ "internal_", &str) && + (SkipPrefix(GTEST_FLAG_PREFIX_, &str) || + SkipPrefix(GTEST_FLAG_PREFIX_DASH_, &str)); +} + +// Prints a string containing code-encoded text. The following escape +// sequences can be used in the string to control the text color: +// +// @@ prints a single '@' character. +// @R changes the color to red. +// @G changes the color to green. +// @Y changes the color to yellow. +// @D changes to the default terminal text color. +// +// TODO(wan@google.com): Write tests for this once we add stdout +// capturing to Google Test. +static void PrintColorEncoded(const char* str) { + GTestColor color = COLOR_DEFAULT; // The current color. + + // Conceptually, we split the string into segments divided by escape + // sequences. Then we print one segment at a time. At the end of + // each iteration, the str pointer advances to the beginning of the + // next segment. + for (;;) { + const char* p = strchr(str, '@'); + if (p == NULL) { + ColoredPrintf(color, "%s", str); + return; + } + + ColoredPrintf(color, "%s", std::string(str, p).c_str()); + + const char ch = p[1]; + str = p + 2; + if (ch == '@') { + ColoredPrintf(color, "@"); + } else if (ch == 'D') { + color = COLOR_DEFAULT; + } else if (ch == 'R') { + color = COLOR_RED; + } else if (ch == 'G') { + color = COLOR_GREEN; + } else if (ch == 'Y') { + color = COLOR_YELLOW; + } else { + --str; + } + } +} + +static const char kColorEncodedHelpMessage[] = +"This program contains tests written using " GTEST_NAME_ ". You can use the\n" +"following command line flags to control its behavior:\n" +"\n" +"Test Selection:\n" +" @G--" GTEST_FLAG_PREFIX_ "list_tests@D\n" +" List the names of all tests instead of running them. The name of\n" +" TEST(Foo, Bar) is \"Foo.Bar\".\n" +" @G--" GTEST_FLAG_PREFIX_ "filter=@YPOSTIVE_PATTERNS" + "[@G-@YNEGATIVE_PATTERNS]@D\n" +" Run only the tests whose name matches one of the positive patterns but\n" +" none of the negative patterns. '?' matches any single character; '*'\n" +" matches any substring; ':' separates two patterns.\n" +" @G--" GTEST_FLAG_PREFIX_ "also_run_disabled_tests@D\n" +" Run all disabled tests too.\n" +"\n" +"Test Execution:\n" +" @G--" GTEST_FLAG_PREFIX_ "repeat=@Y[COUNT]@D\n" +" Run the tests repeatedly; use a negative count to repeat forever.\n" +" @G--" GTEST_FLAG_PREFIX_ "shuffle@D\n" +" Randomize tests' orders on every iteration.\n" +" @G--" GTEST_FLAG_PREFIX_ "random_seed=@Y[NUMBER]@D\n" +" Random number seed to use for shuffling test orders (between 1 and\n" +" 99999, or 0 to use a seed based on the current time).\n" +"\n" +"Test Output:\n" +" @G--" GTEST_FLAG_PREFIX_ "color=@Y(@Gyes@Y|@Gno@Y|@Gauto@Y)@D\n" +" Enable/disable colored output. The default is @Gauto@D.\n" +" -@G-" GTEST_FLAG_PREFIX_ "print_time=0@D\n" +" Don't print the elapsed time of each test.\n" +" @G--" GTEST_FLAG_PREFIX_ "output=xml@Y[@G:@YDIRECTORY_PATH@G" + GTEST_PATH_SEP_ "@Y|@G:@YFILE_PATH]@D\n" +" Generate an XML report in the given directory or with the given file\n" +" name. @YFILE_PATH@D defaults to @Gtest_details.xml@D.\n" +#if GTEST_CAN_STREAM_RESULTS_ +" @G--" GTEST_FLAG_PREFIX_ "stream_result_to=@YHOST@G:@YPORT@D\n" +" Stream test results to the given server.\n" +#endif // GTEST_CAN_STREAM_RESULTS_ +"\n" +"Assertion Behavior:\n" +#if GTEST_HAS_DEATH_TEST && !GTEST_OS_WINDOWS +" @G--" GTEST_FLAG_PREFIX_ "death_test_style=@Y(@Gfast@Y|@Gthreadsafe@Y)@D\n" +" Set the default death test style.\n" +#endif // GTEST_HAS_DEATH_TEST && !GTEST_OS_WINDOWS +" @G--" GTEST_FLAG_PREFIX_ "break_on_failure@D\n" +" Turn assertion failures into debugger break-points.\n" +" @G--" GTEST_FLAG_PREFIX_ "throw_on_failure@D\n" +" Turn assertion failures into C++ exceptions.\n" +" @G--" GTEST_FLAG_PREFIX_ "catch_exceptions=0@D\n" +" Do not report exceptions as test failures. Instead, allow them\n" +" to crash the program or throw a pop-up (on Windows).\n" +"\n" +"Except for @G--" GTEST_FLAG_PREFIX_ "list_tests@D, you can alternatively set " + "the corresponding\n" +"environment variable of a flag (all letters in upper-case). For example, to\n" +"disable colored text output, you can either specify @G--" GTEST_FLAG_PREFIX_ + "color=no@D or set\n" +"the @G" GTEST_FLAG_PREFIX_UPPER_ "COLOR@D environment variable to @Gno@D.\n" +"\n" +"For more information, please read the " GTEST_NAME_ " documentation at\n" +"@G" GTEST_PROJECT_URL_ "@D. If you find a bug in " GTEST_NAME_ "\n" +"(not one in your own code or tests), please report it to\n" +"@G<" GTEST_DEV_EMAIL_ ">@D.\n"; + +// Parses the command line for Google Test flags, without initializing +// other parts of Google Test. The type parameter CharType can be +// instantiated to either char or wchar_t. +template +void ParseGoogleTestFlagsOnlyImpl(int* argc, CharType** argv) { + for (int i = 1; i < *argc; i++) { + const std::string arg_string = StreamableToString(argv[i]); + const char* const arg = arg_string.c_str(); + + using internal::ParseBoolFlag; + using internal::ParseInt32Flag; + using internal::ParseStringFlag; + + // Do we see a Google Test flag? + if (ParseBoolFlag(arg, kAlsoRunDisabledTestsFlag, + >EST_FLAG(also_run_disabled_tests)) || + ParseBoolFlag(arg, kBreakOnFailureFlag, + >EST_FLAG(break_on_failure)) || + ParseBoolFlag(arg, kCatchExceptionsFlag, + >EST_FLAG(catch_exceptions)) || + ParseStringFlag(arg, kColorFlag, >EST_FLAG(color)) || + ParseStringFlag(arg, kDeathTestStyleFlag, + >EST_FLAG(death_test_style)) || + ParseBoolFlag(arg, kDeathTestUseFork, + >EST_FLAG(death_test_use_fork)) || + ParseStringFlag(arg, kFilterFlag, >EST_FLAG(filter)) || + ParseStringFlag(arg, kInternalRunDeathTestFlag, + >EST_FLAG(internal_run_death_test)) || + ParseBoolFlag(arg, kListTestsFlag, >EST_FLAG(list_tests)) || + ParseStringFlag(arg, kOutputFlag, >EST_FLAG(output)) || + ParseBoolFlag(arg, kPrintTimeFlag, >EST_FLAG(print_time)) || + ParseInt32Flag(arg, kRandomSeedFlag, >EST_FLAG(random_seed)) || + ParseInt32Flag(arg, kRepeatFlag, >EST_FLAG(repeat)) || + ParseBoolFlag(arg, kShuffleFlag, >EST_FLAG(shuffle)) || + ParseInt32Flag(arg, kStackTraceDepthFlag, + >EST_FLAG(stack_trace_depth)) || + ParseStringFlag(arg, kStreamResultToFlag, + >EST_FLAG(stream_result_to)) || + ParseBoolFlag(arg, kThrowOnFailureFlag, + >EST_FLAG(throw_on_failure)) + ) { + // Yes. Shift the remainder of the argv list left by one. Note + // that argv has (*argc + 1) elements, the last one always being + // NULL. The following loop moves the trailing NULL element as + // well. + for (int j = i; j != *argc; j++) { + argv[j] = argv[j + 1]; + } + + // Decrements the argument count. + (*argc)--; + + // We also need to decrement the iterator as we just removed + // an element. + i--; + } else if (arg_string == "--help" || arg_string == "-h" || + arg_string == "-?" || arg_string == "/?" || + HasGoogleTestFlagPrefix(arg)) { + // Both help flag and unrecognized Google Test flags (excluding + // internal ones) trigger help display. + g_help_flag = true; + } + } + + if (g_help_flag) { + // We print the help here instead of in RUN_ALL_TESTS(), as the + // latter may not be called at all if the user is using Google + // Test with another testing framework. + PrintColorEncoded(kColorEncodedHelpMessage); + } +} + +// Parses the command line for Google Test flags, without initializing +// other parts of Google Test. +void ParseGoogleTestFlagsOnly(int* argc, char** argv) { + ParseGoogleTestFlagsOnlyImpl(argc, argv); +} +void ParseGoogleTestFlagsOnly(int* argc, wchar_t** argv) { + ParseGoogleTestFlagsOnlyImpl(argc, argv); +} + +// The internal implementation of InitGoogleTest(). +// +// The type parameter CharType can be instantiated to either char or +// wchar_t. +template +void InitGoogleTestImpl(int* argc, CharType** argv) { + g_init_gtest_count++; + + // We don't want to run the initialization code twice. + if (g_init_gtest_count != 1) return; + + if (*argc <= 0) return; + + internal::g_executable_path = internal::StreamableToString(argv[0]); + +#if GTEST_HAS_DEATH_TEST + + g_argvs.clear(); + for (int i = 0; i != *argc; i++) { + g_argvs.push_back(StreamableToString(argv[i])); + } + +#endif // GTEST_HAS_DEATH_TEST + + ParseGoogleTestFlagsOnly(argc, argv); + GetUnitTestImpl()->PostFlagParsingInit(); +} + +} // namespace internal + +// Initializes Google Test. This must be called before calling +// RUN_ALL_TESTS(). In particular, it parses a command line for the +// flags that Google Test recognizes. Whenever a Google Test flag is +// seen, it is removed from argv, and *argc is decremented. +// +// No value is returned. Instead, the Google Test flag variables are +// updated. +// +// Calling the function for the second time has no user-visible effect. +void InitGoogleTest(int* argc, char** argv) { + internal::InitGoogleTestImpl(argc, argv); +} + +// This overloaded version can be used in Windows programs compiled in +// UNICODE mode. +void InitGoogleTest(int* argc, wchar_t** argv) { + internal::InitGoogleTestImpl(argc, argv); +} + +} // namespace testing +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan), vladl@google.com (Vlad Losev) +// +// This file implements death tests. + + +#if GTEST_HAS_DEATH_TEST + +# if GTEST_OS_MAC +# include +# endif // GTEST_OS_MAC + +# include +# include +# include + +# if GTEST_OS_LINUX +# include +# endif // GTEST_OS_LINUX + +# include + +# if GTEST_OS_WINDOWS +# include +# else +# include +# include +# endif // GTEST_OS_WINDOWS + +# if GTEST_OS_QNX +# include +# endif // GTEST_OS_QNX + +#endif // GTEST_HAS_DEATH_TEST + + +// Indicates that this translation unit is part of Google Test's +// implementation. It must come before gtest-internal-inl.h is +// included, or there will be a compiler error. This trick exists to +// prevent the accidental inclusion of gtest-internal-inl.h in the +// user's code. +#define GTEST_IMPLEMENTATION_ 1 +#undef GTEST_IMPLEMENTATION_ + +namespace testing { + +// Constants. + +// The default death test style. +static const char kDefaultDeathTestStyle[] = "fast"; + +GTEST_DEFINE_string_( + death_test_style, + internal::StringFromGTestEnv("death_test_style", kDefaultDeathTestStyle), + "Indicates how to run a death test in a forked child process: " + "\"threadsafe\" (child process re-executes the test binary " + "from the beginning, running only the specific death test) or " + "\"fast\" (child process runs the death test immediately " + "after forking)."); + +GTEST_DEFINE_bool_( + death_test_use_fork, + internal::BoolFromGTestEnv("death_test_use_fork", false), + "Instructs to use fork()/_exit() instead of clone() in death tests. " + "Ignored and always uses fork() on POSIX systems where clone() is not " + "implemented. Useful when running under valgrind or similar tools if " + "those do not support clone(). Valgrind 3.3.1 will just fail if " + "it sees an unsupported combination of clone() flags. " + "It is not recommended to use this flag w/o valgrind though it will " + "work in 99% of the cases. Once valgrind is fixed, this flag will " + "most likely be removed."); + +namespace internal { +GTEST_DEFINE_string_( + internal_run_death_test, "", + "Indicates the file, line number, temporal index of " + "the single death test to run, and a file descriptor to " + "which a success code may be sent, all separated by " + "the '|' characters. This flag is specified if and only if the current " + "process is a sub-process launched for running a thread-safe " + "death test. FOR INTERNAL USE ONLY."); +} // namespace internal + +#if GTEST_HAS_DEATH_TEST + +namespace internal { + +// Valid only for fast death tests. Indicates the code is running in the +// child process of a fast style death test. +static bool g_in_fast_death_test_child = false; + +// Returns a Boolean value indicating whether the caller is currently +// executing in the context of the death test child process. Tools such as +// Valgrind heap checkers may need this to modify their behavior in death +// tests. IMPORTANT: This is an internal utility. Using it may break the +// implementation of death tests. User code MUST NOT use it. +bool InDeathTestChild() { +# if GTEST_OS_WINDOWS + + // On Windows, death tests are thread-safe regardless of the value of the + // death_test_style flag. + return !GTEST_FLAG(internal_run_death_test).empty(); + +# else + + if (GTEST_FLAG(death_test_style) == "threadsafe") + return !GTEST_FLAG(internal_run_death_test).empty(); + else + return g_in_fast_death_test_child; +#endif +} + +} // namespace internal + +// ExitedWithCode constructor. +ExitedWithCode::ExitedWithCode(int exit_code) : exit_code_(exit_code) { +} + +// ExitedWithCode function-call operator. +bool ExitedWithCode::operator()(int exit_status) const { +# if GTEST_OS_WINDOWS + + return exit_status == exit_code_; + +# else + + return WIFEXITED(exit_status) && WEXITSTATUS(exit_status) == exit_code_; + +# endif // GTEST_OS_WINDOWS +} + +# if !GTEST_OS_WINDOWS +// KilledBySignal constructor. +KilledBySignal::KilledBySignal(int signum) : signum_(signum) { +} + +// KilledBySignal function-call operator. +bool KilledBySignal::operator()(int exit_status) const { + return WIFSIGNALED(exit_status) && WTERMSIG(exit_status) == signum_; +} +# endif // !GTEST_OS_WINDOWS + +namespace internal { + +// Utilities needed for death tests. + +// Generates a textual description of a given exit code, in the format +// specified by wait(2). +static std::string ExitSummary(int exit_code) { + Message m; + +# if GTEST_OS_WINDOWS + + m << "Exited with exit status " << exit_code; + +# else + + if (WIFEXITED(exit_code)) { + m << "Exited with exit status " << WEXITSTATUS(exit_code); + } else if (WIFSIGNALED(exit_code)) { + m << "Terminated by signal " << WTERMSIG(exit_code); + } +# ifdef WCOREDUMP + if (WCOREDUMP(exit_code)) { + m << " (core dumped)"; + } +# endif +# endif // GTEST_OS_WINDOWS + + return m.GetString(); +} + +// Returns true if exit_status describes a process that was terminated +// by a signal, or exited normally with a nonzero exit code. +bool ExitedUnsuccessfully(int exit_status) { + return !ExitedWithCode(0)(exit_status); +} + +# if !GTEST_OS_WINDOWS +// Generates a textual failure message when a death test finds more than +// one thread running, or cannot determine the number of threads, prior +// to executing the given statement. It is the responsibility of the +// caller not to pass a thread_count of 1. +static std::string DeathTestThreadWarning(size_t thread_count) { + Message msg; + msg << "Death tests use fork(), which is unsafe particularly" + << " in a threaded context. For this test, " << GTEST_NAME_ << " "; + if (thread_count == 0) + msg << "couldn't detect the number of threads."; + else + msg << "detected " << thread_count << " threads."; + return msg.GetString(); +} +# endif // !GTEST_OS_WINDOWS + +// Flag characters for reporting a death test that did not die. +static const char kDeathTestLived = 'L'; +static const char kDeathTestReturned = 'R'; +static const char kDeathTestThrew = 'T'; +static const char kDeathTestInternalError = 'I'; + +// An enumeration describing all of the possible ways that a death test can +// conclude. DIED means that the process died while executing the test +// code; LIVED means that process lived beyond the end of the test code; +// RETURNED means that the test statement attempted to execute a return +// statement, which is not allowed; THREW means that the test statement +// returned control by throwing an exception. IN_PROGRESS means the test +// has not yet concluded. +// TODO(vladl@google.com): Unify names and possibly values for +// AbortReason, DeathTestOutcome, and flag characters above. +enum DeathTestOutcome { IN_PROGRESS, DIED, LIVED, RETURNED, THREW }; + +// Routine for aborting the program which is safe to call from an +// exec-style death test child process, in which case the error +// message is propagated back to the parent process. Otherwise, the +// message is simply printed to stderr. In either case, the program +// then exits with status 1. +void DeathTestAbort(const std::string& message) { + // On a POSIX system, this function may be called from a threadsafe-style + // death test child process, which operates on a very small stack. Use + // the heap for any additional non-minuscule memory requirements. + const InternalRunDeathTestFlag* const flag = + GetUnitTestImpl()->internal_run_death_test_flag(); + if (flag != NULL) { + FILE* parent = posix::FDOpen(flag->write_fd(), "w"); + fputc(kDeathTestInternalError, parent); + fprintf(parent, "%s", message.c_str()); + fflush(parent); + _exit(1); + } else { + fprintf(stderr, "%s", message.c_str()); + fflush(stderr); + posix::Abort(); + } +} + +// A replacement for CHECK that calls DeathTestAbort if the assertion +// fails. +# define GTEST_DEATH_TEST_CHECK_(expression) \ + do { \ + if (!::testing::internal::IsTrue(expression)) { \ + DeathTestAbort( \ + ::std::string("CHECK failed: File ") + __FILE__ + ", line " \ + + ::testing::internal::StreamableToString(__LINE__) + ": " \ + + #expression); \ + } \ + } while (::testing::internal::AlwaysFalse()) + +// This macro is similar to GTEST_DEATH_TEST_CHECK_, but it is meant for +// evaluating any system call that fulfills two conditions: it must return +// -1 on failure, and set errno to EINTR when it is interrupted and +// should be tried again. The macro expands to a loop that repeatedly +// evaluates the expression as long as it evaluates to -1 and sets +// errno to EINTR. If the expression evaluates to -1 but errno is +// something other than EINTR, DeathTestAbort is called. +# define GTEST_DEATH_TEST_CHECK_SYSCALL_(expression) \ + do { \ + int gtest_retval; \ + do { \ + gtest_retval = (expression); \ + } while (gtest_retval == -1 && errno == EINTR); \ + if (gtest_retval == -1) { \ + DeathTestAbort( \ + ::std::string("CHECK failed: File ") + __FILE__ + ", line " \ + + ::testing::internal::StreamableToString(__LINE__) + ": " \ + + #expression + " != -1"); \ + } \ + } while (::testing::internal::AlwaysFalse()) + +// Returns the message describing the last system error in errno. +std::string GetLastErrnoDescription() { + return errno == 0 ? "" : posix::StrError(errno); +} + +// This is called from a death test parent process to read a failure +// message from the death test child process and log it with the FATAL +// severity. On Windows, the message is read from a pipe handle. On other +// platforms, it is read from a file descriptor. +static void FailFromInternalError(int fd) { + Message error; + char buffer[256]; + int num_read; + + do { + while ((num_read = posix::Read(fd, buffer, 255)) > 0) { + buffer[num_read] = '\0'; + error << buffer; + } + } while (num_read == -1 && errno == EINTR); + + if (num_read == 0) { + GTEST_LOG_(FATAL) << error.GetString(); + } else { + const int last_error = errno; + GTEST_LOG_(FATAL) << "Error while reading death test internal: " + << GetLastErrnoDescription() << " [" << last_error << "]"; + } +} + +// Death test constructor. Increments the running death test count +// for the current test. +DeathTest::DeathTest() { + TestInfo* const info = GetUnitTestImpl()->current_test_info(); + if (info == NULL) { + DeathTestAbort("Cannot run a death test outside of a TEST or " + "TEST_F construct"); + } +} + +// Creates and returns a death test by dispatching to the current +// death test factory. +bool DeathTest::Create(const char* statement, const RE* regex, + const char* file, int line, DeathTest** test) { + return GetUnitTestImpl()->death_test_factory()->Create( + statement, regex, file, line, test); +} + +const char* DeathTest::LastMessage() { + return last_death_test_message_.c_str(); +} + +void DeathTest::set_last_death_test_message(const std::string& message) { + last_death_test_message_ = message; +} + +std::string DeathTest::last_death_test_message_; + +// Provides cross platform implementation for some death functionality. +class DeathTestImpl : public DeathTest { + protected: + DeathTestImpl(const char* a_statement, const RE* a_regex) + : statement_(a_statement), + regex_(a_regex), + spawned_(false), + status_(-1), + outcome_(IN_PROGRESS), + read_fd_(-1), + write_fd_(-1) {} + + // read_fd_ is expected to be closed and cleared by a derived class. + ~DeathTestImpl() { GTEST_DEATH_TEST_CHECK_(read_fd_ == -1); } + + void Abort(AbortReason reason); + virtual bool Passed(bool status_ok); + + const char* statement() const { return statement_; } + const RE* regex() const { return regex_; } + bool spawned() const { return spawned_; } + void set_spawned(bool is_spawned) { spawned_ = is_spawned; } + int status() const { return status_; } + void set_status(int a_status) { status_ = a_status; } + DeathTestOutcome outcome() const { return outcome_; } + void set_outcome(DeathTestOutcome an_outcome) { outcome_ = an_outcome; } + int read_fd() const { return read_fd_; } + void set_read_fd(int fd) { read_fd_ = fd; } + int write_fd() const { return write_fd_; } + void set_write_fd(int fd) { write_fd_ = fd; } + + // Called in the parent process only. Reads the result code of the death + // test child process via a pipe, interprets it to set the outcome_ + // member, and closes read_fd_. Outputs diagnostics and terminates in + // case of unexpected codes. + void ReadAndInterpretStatusByte(); + + private: + // The textual content of the code this object is testing. This class + // doesn't own this string and should not attempt to delete it. + const char* const statement_; + // The regular expression which test output must match. DeathTestImpl + // doesn't own this object and should not attempt to delete it. + const RE* const regex_; + // True if the death test child process has been successfully spawned. + bool spawned_; + // The exit status of the child process. + int status_; + // How the death test concluded. + DeathTestOutcome outcome_; + // Descriptor to the read end of the pipe to the child process. It is + // always -1 in the child process. The child keeps its write end of the + // pipe in write_fd_. + int read_fd_; + // Descriptor to the child's write end of the pipe to the parent process. + // It is always -1 in the parent process. The parent keeps its end of the + // pipe in read_fd_. + int write_fd_; +}; + +// Called in the parent process only. Reads the result code of the death +// test child process via a pipe, interprets it to set the outcome_ +// member, and closes read_fd_. Outputs diagnostics and terminates in +// case of unexpected codes. +void DeathTestImpl::ReadAndInterpretStatusByte() { + char flag; + int bytes_read; + + // The read() here blocks until data is available (signifying the + // failure of the death test) or until the pipe is closed (signifying + // its success), so it's okay to call this in the parent before + // the child process has exited. + do { + bytes_read = posix::Read(read_fd(), &flag, 1); + } while (bytes_read == -1 && errno == EINTR); + + if (bytes_read == 0) { + set_outcome(DIED); + } else if (bytes_read == 1) { + switch (flag) { + case kDeathTestReturned: + set_outcome(RETURNED); + break; + case kDeathTestThrew: + set_outcome(THREW); + break; + case kDeathTestLived: + set_outcome(LIVED); + break; + case kDeathTestInternalError: + FailFromInternalError(read_fd()); // Does not return. + break; + default: + GTEST_LOG_(FATAL) << "Death test child process reported " + << "unexpected status byte (" + << static_cast(flag) << ")"; + } + } else { + GTEST_LOG_(FATAL) << "Read from death test child process failed: " + << GetLastErrnoDescription(); + } + GTEST_DEATH_TEST_CHECK_SYSCALL_(posix::Close(read_fd())); + set_read_fd(-1); +} + +// Signals that the death test code which should have exited, didn't. +// Should be called only in a death test child process. +// Writes a status byte to the child's status file descriptor, then +// calls _exit(1). +void DeathTestImpl::Abort(AbortReason reason) { + // The parent process considers the death test to be a failure if + // it finds any data in our pipe. So, here we write a single flag byte + // to the pipe, then exit. + const char status_ch = + reason == TEST_DID_NOT_DIE ? kDeathTestLived : + reason == TEST_THREW_EXCEPTION ? kDeathTestThrew : kDeathTestReturned; + + GTEST_DEATH_TEST_CHECK_SYSCALL_(posix::Write(write_fd(), &status_ch, 1)); + // We are leaking the descriptor here because on some platforms (i.e., + // when built as Windows DLL), destructors of global objects will still + // run after calling _exit(). On such systems, write_fd_ will be + // indirectly closed from the destructor of UnitTestImpl, causing double + // close if it is also closed here. On debug configurations, double close + // may assert. As there are no in-process buffers to flush here, we are + // relying on the OS to close the descriptor after the process terminates + // when the destructors are not run. + _exit(1); // Exits w/o any normal exit hooks (we were supposed to crash) +} + +// Returns an indented copy of stderr output for a death test. +// This makes distinguishing death test output lines from regular log lines +// much easier. +static ::std::string FormatDeathTestOutput(const ::std::string& output) { + ::std::string ret; + for (size_t at = 0; ; ) { + const size_t line_end = output.find('\n', at); + ret += "[ DEATH ] "; + if (line_end == ::std::string::npos) { + ret += output.substr(at); + break; + } + ret += output.substr(at, line_end + 1 - at); + at = line_end + 1; + } + return ret; +} + +// Assesses the success or failure of a death test, using both private +// members which have previously been set, and one argument: +// +// Private data members: +// outcome: An enumeration describing how the death test +// concluded: DIED, LIVED, THREW, or RETURNED. The death test +// fails in the latter three cases. +// status: The exit status of the child process. On *nix, it is in the +// in the format specified by wait(2). On Windows, this is the +// value supplied to the ExitProcess() API or a numeric code +// of the exception that terminated the program. +// regex: A regular expression object to be applied to +// the test's captured standard error output; the death test +// fails if it does not match. +// +// Argument: +// status_ok: true if exit_status is acceptable in the context of +// this particular death test, which fails if it is false +// +// Returns true iff all of the above conditions are met. Otherwise, the +// first failing condition, in the order given above, is the one that is +// reported. Also sets the last death test message string. +bool DeathTestImpl::Passed(bool status_ok) { + if (!spawned()) + return false; + + const std::string error_message = GetCapturedStderr(); + + bool success = false; + Message buffer; + + buffer << "Death test: " << statement() << "\n"; + switch (outcome()) { + case LIVED: + buffer << " Result: failed to die.\n" + << " Error msg:\n" << FormatDeathTestOutput(error_message); + break; + case THREW: + buffer << " Result: threw an exception.\n" + << " Error msg:\n" << FormatDeathTestOutput(error_message); + break; + case RETURNED: + buffer << " Result: illegal return in test statement.\n" + << " Error msg:\n" << FormatDeathTestOutput(error_message); + break; + case DIED: + if (status_ok) { + const bool matched = RE::PartialMatch(error_message.c_str(), *regex()); + if (matched) { + success = true; + } else { + buffer << " Result: died but not with expected error.\n" + << " Expected: " << regex()->pattern() << "\n" + << "Actual msg:\n" << FormatDeathTestOutput(error_message); + } + } else { + buffer << " Result: died but not with expected exit code:\n" + << " " << ExitSummary(status()) << "\n" + << "Actual msg:\n" << FormatDeathTestOutput(error_message); + } + break; + case IN_PROGRESS: + default: + GTEST_LOG_(FATAL) + << "DeathTest::Passed somehow called before conclusion of test"; + } + + DeathTest::set_last_death_test_message(buffer.GetString()); + return success; +} + +# if GTEST_OS_WINDOWS +// WindowsDeathTest implements death tests on Windows. Due to the +// specifics of starting new processes on Windows, death tests there are +// always threadsafe, and Google Test considers the +// --gtest_death_test_style=fast setting to be equivalent to +// --gtest_death_test_style=threadsafe there. +// +// A few implementation notes: Like the Linux version, the Windows +// implementation uses pipes for child-to-parent communication. But due to +// the specifics of pipes on Windows, some extra steps are required: +// +// 1. The parent creates a communication pipe and stores handles to both +// ends of it. +// 2. The parent starts the child and provides it with the information +// necessary to acquire the handle to the write end of the pipe. +// 3. The child acquires the write end of the pipe and signals the parent +// using a Windows event. +// 4. Now the parent can release the write end of the pipe on its side. If +// this is done before step 3, the object's reference count goes down to +// 0 and it is destroyed, preventing the child from acquiring it. The +// parent now has to release it, or read operations on the read end of +// the pipe will not return when the child terminates. +// 5. The parent reads child's output through the pipe (outcome code and +// any possible error messages) from the pipe, and its stderr and then +// determines whether to fail the test. +// +// Note: to distinguish Win32 API calls from the local method and function +// calls, the former are explicitly resolved in the global namespace. +// +class WindowsDeathTest : public DeathTestImpl { + public: + WindowsDeathTest(const char* a_statement, + const RE* a_regex, + const char* file, + int line) + : DeathTestImpl(a_statement, a_regex), file_(file), line_(line) {} + + // All of these virtual functions are inherited from DeathTest. + virtual int Wait(); + virtual TestRole AssumeRole(); + + private: + // The name of the file in which the death test is located. + const char* const file_; + // The line number on which the death test is located. + const int line_; + // Handle to the write end of the pipe to the child process. + AutoHandle write_handle_; + // Child process handle. + AutoHandle child_handle_; + // Event the child process uses to signal the parent that it has + // acquired the handle to the write end of the pipe. After seeing this + // event the parent can release its own handles to make sure its + // ReadFile() calls return when the child terminates. + AutoHandle event_handle_; +}; + +// Waits for the child in a death test to exit, returning its exit +// status, or 0 if no child process exists. As a side effect, sets the +// outcome data member. +int WindowsDeathTest::Wait() { + if (!spawned()) + return 0; + + // Wait until the child either signals that it has acquired the write end + // of the pipe or it dies. + const HANDLE wait_handles[2] = { child_handle_.Get(), event_handle_.Get() }; + switch (::WaitForMultipleObjects(2, + wait_handles, + FALSE, // Waits for any of the handles. + INFINITE)) { + case WAIT_OBJECT_0: + case WAIT_OBJECT_0 + 1: + break; + default: + GTEST_DEATH_TEST_CHECK_(false); // Should not get here. + } + + // The child has acquired the write end of the pipe or exited. + // We release the handle on our side and continue. + write_handle_.Reset(); + event_handle_.Reset(); + + ReadAndInterpretStatusByte(); + + // Waits for the child process to exit if it haven't already. This + // returns immediately if the child has already exited, regardless of + // whether previous calls to WaitForMultipleObjects synchronized on this + // handle or not. + GTEST_DEATH_TEST_CHECK_( + WAIT_OBJECT_0 == ::WaitForSingleObject(child_handle_.Get(), + INFINITE)); + DWORD status_code; + GTEST_DEATH_TEST_CHECK_( + ::GetExitCodeProcess(child_handle_.Get(), &status_code) != FALSE); + child_handle_.Reset(); + set_status(static_cast(status_code)); + return status(); +} + +// The AssumeRole process for a Windows death test. It creates a child +// process with the same executable as the current process to run the +// death test. The child process is given the --gtest_filter and +// --gtest_internal_run_death_test flags such that it knows to run the +// current death test only. +DeathTest::TestRole WindowsDeathTest::AssumeRole() { + const UnitTestImpl* const impl = GetUnitTestImpl(); + const InternalRunDeathTestFlag* const flag = + impl->internal_run_death_test_flag(); + const TestInfo* const info = impl->current_test_info(); + const int death_test_index = info->result()->death_test_count(); + + if (flag != NULL) { + // ParseInternalRunDeathTestFlag() has performed all the necessary + // processing. + set_write_fd(flag->write_fd()); + return EXECUTE_TEST; + } + + // WindowsDeathTest uses an anonymous pipe to communicate results of + // a death test. + SECURITY_ATTRIBUTES handles_are_inheritable = { + sizeof(SECURITY_ATTRIBUTES), NULL, TRUE }; + HANDLE read_handle, write_handle; + GTEST_DEATH_TEST_CHECK_( + ::CreatePipe(&read_handle, &write_handle, &handles_are_inheritable, + 0) // Default buffer size. + != FALSE); + set_read_fd(::_open_osfhandle(reinterpret_cast(read_handle), + O_RDONLY)); + write_handle_.Reset(write_handle); + event_handle_.Reset(::CreateEvent( + &handles_are_inheritable, + TRUE, // The event will automatically reset to non-signaled state. + FALSE, // The initial state is non-signalled. + NULL)); // The even is unnamed. + GTEST_DEATH_TEST_CHECK_(event_handle_.Get() != NULL); + const std::string filter_flag = + std::string("--") + GTEST_FLAG_PREFIX_ + kFilterFlag + "=" + + info->test_case_name() + "." + info->name(); + const std::string internal_flag = + std::string("--") + GTEST_FLAG_PREFIX_ + kInternalRunDeathTestFlag + + "=" + file_ + "|" + StreamableToString(line_) + "|" + + StreamableToString(death_test_index) + "|" + + StreamableToString(static_cast(::GetCurrentProcessId())) + + // size_t has the same width as pointers on both 32-bit and 64-bit + // Windows platforms. + // See http://msdn.microsoft.com/en-us/library/tcxf1dw6.aspx. + "|" + StreamableToString(reinterpret_cast(write_handle)) + + "|" + StreamableToString(reinterpret_cast(event_handle_.Get())); + + char executable_path[_MAX_PATH + 1]; // NOLINT + GTEST_DEATH_TEST_CHECK_( + _MAX_PATH + 1 != ::GetModuleFileNameA(NULL, + executable_path, + _MAX_PATH)); + + std::string command_line = + std::string(::GetCommandLineA()) + " " + filter_flag + " \"" + + internal_flag + "\""; + + DeathTest::set_last_death_test_message(""); + + CaptureStderr(); + // Flush the log buffers since the log streams are shared with the child. + FlushInfoLog(); + + // The child process will share the standard handles with the parent. + STARTUPINFOA startup_info; + memset(&startup_info, 0, sizeof(STARTUPINFO)); + startup_info.dwFlags = STARTF_USESTDHANDLES; + startup_info.hStdInput = ::GetStdHandle(STD_INPUT_HANDLE); + startup_info.hStdOutput = ::GetStdHandle(STD_OUTPUT_HANDLE); + startup_info.hStdError = ::GetStdHandle(STD_ERROR_HANDLE); + + PROCESS_INFORMATION process_info; + GTEST_DEATH_TEST_CHECK_(::CreateProcessA( + executable_path, + const_cast(command_line.c_str()), + NULL, // Retuned process handle is not inheritable. + NULL, // Retuned thread handle is not inheritable. + TRUE, // Child inherits all inheritable handles (for write_handle_). + 0x0, // Default creation flags. + NULL, // Inherit the parent's environment. + UnitTest::GetInstance()->original_working_dir(), + &startup_info, + &process_info) != FALSE); + child_handle_.Reset(process_info.hProcess); + ::CloseHandle(process_info.hThread); + set_spawned(true); + return OVERSEE_TEST; +} +# else // We are not on Windows. + +// ForkingDeathTest provides implementations for most of the abstract +// methods of the DeathTest interface. Only the AssumeRole method is +// left undefined. +class ForkingDeathTest : public DeathTestImpl { + public: + ForkingDeathTest(const char* statement, const RE* regex); + + // All of these virtual functions are inherited from DeathTest. + virtual int Wait(); + + protected: + void set_child_pid(pid_t child_pid) { child_pid_ = child_pid; } + + private: + // PID of child process during death test; 0 in the child process itself. + pid_t child_pid_; +}; + +// Constructs a ForkingDeathTest. +ForkingDeathTest::ForkingDeathTest(const char* a_statement, const RE* a_regex) + : DeathTestImpl(a_statement, a_regex), + child_pid_(-1) {} + +// Waits for the child in a death test to exit, returning its exit +// status, or 0 if no child process exists. As a side effect, sets the +// outcome data member. +int ForkingDeathTest::Wait() { + if (!spawned()) + return 0; + + ReadAndInterpretStatusByte(); + + int status_value; + GTEST_DEATH_TEST_CHECK_SYSCALL_(waitpid(child_pid_, &status_value, 0)); + set_status(status_value); + return status_value; +} + +// A concrete death test class that forks, then immediately runs the test +// in the child process. +class NoExecDeathTest : public ForkingDeathTest { + public: + NoExecDeathTest(const char* a_statement, const RE* a_regex) : + ForkingDeathTest(a_statement, a_regex) { } + virtual TestRole AssumeRole(); +}; + +// The AssumeRole process for a fork-and-run death test. It implements a +// straightforward fork, with a simple pipe to transmit the status byte. +DeathTest::TestRole NoExecDeathTest::AssumeRole() { + const size_t thread_count = GetThreadCount(); + if (thread_count != 1) { + GTEST_LOG_(WARNING) << DeathTestThreadWarning(thread_count); + } + + int pipe_fd[2]; + GTEST_DEATH_TEST_CHECK_(pipe(pipe_fd) != -1); + + DeathTest::set_last_death_test_message(""); + CaptureStderr(); + // When we fork the process below, the log file buffers are copied, but the + // file descriptors are shared. We flush all log files here so that closing + // the file descriptors in the child process doesn't throw off the + // synchronization between descriptors and buffers in the parent process. + // This is as close to the fork as possible to avoid a race condition in case + // there are multiple threads running before the death test, and another + // thread writes to the log file. + FlushInfoLog(); + + const pid_t child_pid = fork(); + GTEST_DEATH_TEST_CHECK_(child_pid != -1); + set_child_pid(child_pid); + if (child_pid == 0) { + GTEST_DEATH_TEST_CHECK_SYSCALL_(close(pipe_fd[0])); + set_write_fd(pipe_fd[1]); + // Redirects all logging to stderr in the child process to prevent + // concurrent writes to the log files. We capture stderr in the parent + // process and append the child process' output to a log. + LogToStderr(); + // Event forwarding to the listeners of event listener API mush be shut + // down in death test subprocesses. + GetUnitTestImpl()->listeners()->SuppressEventForwarding(); + g_in_fast_death_test_child = true; + return EXECUTE_TEST; + } else { + GTEST_DEATH_TEST_CHECK_SYSCALL_(close(pipe_fd[1])); + set_read_fd(pipe_fd[0]); + set_spawned(true); + return OVERSEE_TEST; + } +} + +// A concrete death test class that forks and re-executes the main +// program from the beginning, with command-line flags set that cause +// only this specific death test to be run. +class ExecDeathTest : public ForkingDeathTest { + public: + ExecDeathTest(const char* a_statement, const RE* a_regex, + const char* file, int line) : + ForkingDeathTest(a_statement, a_regex), file_(file), line_(line) { } + virtual TestRole AssumeRole(); + private: + static ::std::vector + GetArgvsForDeathTestChildProcess() { + ::std::vector args = GetInjectableArgvs(); + return args; + } + // The name of the file in which the death test is located. + const char* const file_; + // The line number on which the death test is located. + const int line_; +}; + +// Utility class for accumulating command-line arguments. +class Arguments { + public: + Arguments() { + args_.push_back(NULL); + } + + ~Arguments() { + for (std::vector::iterator i = args_.begin(); i != args_.end(); + ++i) { + free(*i); + } + } + void AddArgument(const char* argument) { + args_.insert(args_.end() - 1, posix::StrDup(argument)); + } + + template + void AddArguments(const ::std::vector& arguments) { + for (typename ::std::vector::const_iterator i = arguments.begin(); + i != arguments.end(); + ++i) { + args_.insert(args_.end() - 1, posix::StrDup(i->c_str())); + } + } + char* const* Argv() { + return &args_[0]; + } + + private: + std::vector args_; +}; + +// A struct that encompasses the arguments to the child process of a +// threadsafe-style death test process. +struct ExecDeathTestArgs { + char* const* argv; // Command-line arguments for the child's call to exec + int close_fd; // File descriptor to close; the read end of a pipe +}; + +# if GTEST_OS_MAC +inline char** GetEnviron() { + // When Google Test is built as a framework on MacOS X, the environ variable + // is unavailable. Apple's documentation (man environ) recommends using + // _NSGetEnviron() instead. + return *_NSGetEnviron(); +} +# else +// Some POSIX platforms expect you to declare environ. extern "C" makes +// it reside in the global namespace. +extern "C" char** environ; +inline char** GetEnviron() { return environ; } +# endif // GTEST_OS_MAC + +# if !GTEST_OS_QNX +// The main function for a threadsafe-style death test child process. +// This function is called in a clone()-ed process and thus must avoid +// any potentially unsafe operations like malloc or libc functions. +static int ExecDeathTestChildMain(void* child_arg) { + ExecDeathTestArgs* const args = static_cast(child_arg); + GTEST_DEATH_TEST_CHECK_SYSCALL_(close(args->close_fd)); + + // We need to execute the test program in the same environment where + // it was originally invoked. Therefore we change to the original + // working directory first. + const char* const original_dir = + UnitTest::GetInstance()->original_working_dir(); + // We can safely call chdir() as it's a direct system call. + if (chdir(original_dir) != 0) { + DeathTestAbort(std::string("chdir(\"") + original_dir + "\") failed: " + + GetLastErrnoDescription()); + return EXIT_FAILURE; + } + + // We can safely call execve() as it's a direct system call. We + // cannot use execvp() as it's a libc function and thus potentially + // unsafe. Since execve() doesn't search the PATH, the user must + // invoke the test program via a valid path that contains at least + // one path separator. + execve(args->argv[0], args->argv, GetEnviron()); + DeathTestAbort(std::string("execve(") + args->argv[0] + ", ...) in " + + original_dir + " failed: " + + GetLastErrnoDescription()); + return EXIT_FAILURE; +} +# endif // !GTEST_OS_QNX + +// Two utility routines that together determine the direction the stack +// grows. +// This could be accomplished more elegantly by a single recursive +// function, but we want to guard against the unlikely possibility of +// a smart compiler optimizing the recursion away. +// +// GTEST_NO_INLINE_ is required to prevent GCC 4.6 from inlining +// StackLowerThanAddress into StackGrowsDown, which then doesn't give +// correct answer. +void StackLowerThanAddress(const void* ptr, bool* result) GTEST_NO_INLINE_; +void StackLowerThanAddress(const void* ptr, bool* result) { + int dummy; + *result = (&dummy < ptr); +} + +// Make sure AddressSanitizer does not tamper with the stack here. +GTEST_ATTRIBUTE_NO_SANITIZE_ADDRESS_ +bool StackGrowsDown() { + int dummy; + bool result; + StackLowerThanAddress(&dummy, &result); + return result; +} + +// Spawns a child process with the same executable as the current process in +// a thread-safe manner and instructs it to run the death test. The +// implementation uses fork(2) + exec. On systems where clone(2) is +// available, it is used instead, being slightly more thread-safe. On QNX, +// fork supports only single-threaded environments, so this function uses +// spawn(2) there instead. The function dies with an error message if +// anything goes wrong. +static pid_t ExecDeathTestSpawnChild(char* const* argv, int close_fd) { + ExecDeathTestArgs args = { argv, close_fd }; + pid_t child_pid = -1; + +# if GTEST_OS_QNX + // Obtains the current directory and sets it to be closed in the child + // process. + const int cwd_fd = open(".", O_RDONLY); + GTEST_DEATH_TEST_CHECK_(cwd_fd != -1); + GTEST_DEATH_TEST_CHECK_SYSCALL_(fcntl(cwd_fd, F_SETFD, FD_CLOEXEC)); + // We need to execute the test program in the same environment where + // it was originally invoked. Therefore we change to the original + // working directory first. + const char* const original_dir = + UnitTest::GetInstance()->original_working_dir(); + // We can safely call chdir() as it's a direct system call. + if (chdir(original_dir) != 0) { + DeathTestAbort(std::string("chdir(\"") + original_dir + "\") failed: " + + GetLastErrnoDescription()); + return EXIT_FAILURE; + } + + int fd_flags; + // Set close_fd to be closed after spawn. + GTEST_DEATH_TEST_CHECK_SYSCALL_(fd_flags = fcntl(close_fd, F_GETFD)); + GTEST_DEATH_TEST_CHECK_SYSCALL_(fcntl(close_fd, F_SETFD, + fd_flags | FD_CLOEXEC)); + struct inheritance inherit = {0}; + // spawn is a system call. + child_pid = spawn(args.argv[0], 0, NULL, &inherit, args.argv, GetEnviron()); + // Restores the current working directory. + GTEST_DEATH_TEST_CHECK_(fchdir(cwd_fd) != -1); + GTEST_DEATH_TEST_CHECK_SYSCALL_(close(cwd_fd)); + +# else // GTEST_OS_QNX +# if GTEST_OS_LINUX + // When a SIGPROF signal is received while fork() or clone() are executing, + // the process may hang. To avoid this, we ignore SIGPROF here and re-enable + // it after the call to fork()/clone() is complete. + struct sigaction saved_sigprof_action; + struct sigaction ignore_sigprof_action; + memset(&ignore_sigprof_action, 0, sizeof(ignore_sigprof_action)); + sigemptyset(&ignore_sigprof_action.sa_mask); + ignore_sigprof_action.sa_handler = SIG_IGN; + GTEST_DEATH_TEST_CHECK_SYSCALL_(sigaction( + SIGPROF, &ignore_sigprof_action, &saved_sigprof_action)); +# endif // GTEST_OS_LINUX + +# if GTEST_HAS_CLONE + const bool use_fork = GTEST_FLAG(death_test_use_fork); + + if (!use_fork) { + static const bool stack_grows_down = StackGrowsDown(); + const size_t stack_size = getpagesize(); + // MMAP_ANONYMOUS is not defined on Mac, so we use MAP_ANON instead. + void* const stack = mmap(NULL, stack_size, PROT_READ | PROT_WRITE, + MAP_ANON | MAP_PRIVATE, -1, 0); + GTEST_DEATH_TEST_CHECK_(stack != MAP_FAILED); + + // Maximum stack alignment in bytes: For a downward-growing stack, this + // amount is subtracted from size of the stack space to get an address + // that is within the stack space and is aligned on all systems we care + // about. As far as I know there is no ABI with stack alignment greater + // than 64. We assume stack and stack_size already have alignment of + // kMaxStackAlignment. + const size_t kMaxStackAlignment = 64; + void* const stack_top = + static_cast(stack) + + (stack_grows_down ? stack_size - kMaxStackAlignment : 0); + GTEST_DEATH_TEST_CHECK_(stack_size > kMaxStackAlignment && + reinterpret_cast(stack_top) % kMaxStackAlignment == 0); + + child_pid = clone(&ExecDeathTestChildMain, stack_top, SIGCHLD, &args); + + GTEST_DEATH_TEST_CHECK_(munmap(stack, stack_size) != -1); + } +# else + const bool use_fork = true; +# endif // GTEST_HAS_CLONE + + if (use_fork && (child_pid = fork()) == 0) { + ExecDeathTestChildMain(&args); + _exit(0); + } +# endif // GTEST_OS_QNX +# if GTEST_OS_LINUX + GTEST_DEATH_TEST_CHECK_SYSCALL_( + sigaction(SIGPROF, &saved_sigprof_action, NULL)); +# endif // GTEST_OS_LINUX + + GTEST_DEATH_TEST_CHECK_(child_pid != -1); + return child_pid; +} + +// The AssumeRole process for a fork-and-exec death test. It re-executes the +// main program from the beginning, setting the --gtest_filter +// and --gtest_internal_run_death_test flags to cause only the current +// death test to be re-run. +DeathTest::TestRole ExecDeathTest::AssumeRole() { + const UnitTestImpl* const impl = GetUnitTestImpl(); + const InternalRunDeathTestFlag* const flag = + impl->internal_run_death_test_flag(); + const TestInfo* const info = impl->current_test_info(); + const int death_test_index = info->result()->death_test_count(); + + if (flag != NULL) { + set_write_fd(flag->write_fd()); + return EXECUTE_TEST; + } + + int pipe_fd[2]; + GTEST_DEATH_TEST_CHECK_(pipe(pipe_fd) != -1); + // Clear the close-on-exec flag on the write end of the pipe, lest + // it be closed when the child process does an exec: + GTEST_DEATH_TEST_CHECK_(fcntl(pipe_fd[1], F_SETFD, 0) != -1); + + const std::string filter_flag = + std::string("--") + GTEST_FLAG_PREFIX_ + kFilterFlag + "=" + + info->test_case_name() + "." + info->name(); + const std::string internal_flag = + std::string("--") + GTEST_FLAG_PREFIX_ + kInternalRunDeathTestFlag + "=" + + file_ + "|" + StreamableToString(line_) + "|" + + StreamableToString(death_test_index) + "|" + + StreamableToString(pipe_fd[1]); + Arguments args; + args.AddArguments(GetArgvsForDeathTestChildProcess()); + args.AddArgument(filter_flag.c_str()); + args.AddArgument(internal_flag.c_str()); + + DeathTest::set_last_death_test_message(""); + + CaptureStderr(); + // See the comment in NoExecDeathTest::AssumeRole for why the next line + // is necessary. + FlushInfoLog(); + + const pid_t child_pid = ExecDeathTestSpawnChild(args.Argv(), pipe_fd[0]); + GTEST_DEATH_TEST_CHECK_SYSCALL_(close(pipe_fd[1])); + set_child_pid(child_pid); + set_read_fd(pipe_fd[0]); + set_spawned(true); + return OVERSEE_TEST; +} + +# endif // !GTEST_OS_WINDOWS + +// Creates a concrete DeathTest-derived class that depends on the +// --gtest_death_test_style flag, and sets the pointer pointed to +// by the "test" argument to its address. If the test should be +// skipped, sets that pointer to NULL. Returns true, unless the +// flag is set to an invalid value. +bool DefaultDeathTestFactory::Create(const char* statement, const RE* regex, + const char* file, int line, + DeathTest** test) { + UnitTestImpl* const impl = GetUnitTestImpl(); + const InternalRunDeathTestFlag* const flag = + impl->internal_run_death_test_flag(); + const int death_test_index = impl->current_test_info() + ->increment_death_test_count(); + + if (flag != NULL) { + if (death_test_index > flag->index()) { + DeathTest::set_last_death_test_message( + "Death test count (" + StreamableToString(death_test_index) + + ") somehow exceeded expected maximum (" + + StreamableToString(flag->index()) + ")"); + return false; + } + + if (!(flag->file() == file && flag->line() == line && + flag->index() == death_test_index)) { + *test = NULL; + return true; + } + } + +# if GTEST_OS_WINDOWS + + if (GTEST_FLAG(death_test_style) == "threadsafe" || + GTEST_FLAG(death_test_style) == "fast") { + *test = new WindowsDeathTest(statement, regex, file, line); + } + +# else + + if (GTEST_FLAG(death_test_style) == "threadsafe") { + *test = new ExecDeathTest(statement, regex, file, line); + } else if (GTEST_FLAG(death_test_style) == "fast") { + *test = new NoExecDeathTest(statement, regex); + } + +# endif // GTEST_OS_WINDOWS + + else { // NOLINT - this is more readable than unbalanced brackets inside #if. + DeathTest::set_last_death_test_message( + "Unknown death test style \"" + GTEST_FLAG(death_test_style) + + "\" encountered"); + return false; + } + + return true; +} + +// Splits a given string on a given delimiter, populating a given +// vector with the fields. GTEST_HAS_DEATH_TEST implies that we have +// ::std::string, so we can use it here. +static void SplitString(const ::std::string& str, char delimiter, + ::std::vector< ::std::string>* dest) { + ::std::vector< ::std::string> parsed; + ::std::string::size_type pos = 0; + while (::testing::internal::AlwaysTrue()) { + const ::std::string::size_type colon = str.find(delimiter, pos); + if (colon == ::std::string::npos) { + parsed.push_back(str.substr(pos)); + break; + } else { + parsed.push_back(str.substr(pos, colon - pos)); + pos = colon + 1; + } + } + dest->swap(parsed); +} + +# if GTEST_OS_WINDOWS +// Recreates the pipe and event handles from the provided parameters, +// signals the event, and returns a file descriptor wrapped around the pipe +// handle. This function is called in the child process only. +int GetStatusFileDescriptor(unsigned int parent_process_id, + size_t write_handle_as_size_t, + size_t event_handle_as_size_t) { + AutoHandle parent_process_handle(::OpenProcess(PROCESS_DUP_HANDLE, + FALSE, // Non-inheritable. + parent_process_id)); + if (parent_process_handle.Get() == INVALID_HANDLE_VALUE) { + DeathTestAbort("Unable to open parent process " + + StreamableToString(parent_process_id)); + } + + // TODO(vladl@google.com): Replace the following check with a + // compile-time assertion when available. + GTEST_CHECK_(sizeof(HANDLE) <= sizeof(size_t)); + + const HANDLE write_handle = + reinterpret_cast(write_handle_as_size_t); + HANDLE dup_write_handle; + + // The newly initialized handle is accessible only in in the parent + // process. To obtain one accessible within the child, we need to use + // DuplicateHandle. + if (!::DuplicateHandle(parent_process_handle.Get(), write_handle, + ::GetCurrentProcess(), &dup_write_handle, + 0x0, // Requested privileges ignored since + // DUPLICATE_SAME_ACCESS is used. + FALSE, // Request non-inheritable handler. + DUPLICATE_SAME_ACCESS)) { + DeathTestAbort("Unable to duplicate the pipe handle " + + StreamableToString(write_handle_as_size_t) + + " from the parent process " + + StreamableToString(parent_process_id)); + } + + const HANDLE event_handle = reinterpret_cast(event_handle_as_size_t); + HANDLE dup_event_handle; + + if (!::DuplicateHandle(parent_process_handle.Get(), event_handle, + ::GetCurrentProcess(), &dup_event_handle, + 0x0, + FALSE, + DUPLICATE_SAME_ACCESS)) { + DeathTestAbort("Unable to duplicate the event handle " + + StreamableToString(event_handle_as_size_t) + + " from the parent process " + + StreamableToString(parent_process_id)); + } + + const int write_fd = + ::_open_osfhandle(reinterpret_cast(dup_write_handle), O_APPEND); + if (write_fd == -1) { + DeathTestAbort("Unable to convert pipe handle " + + StreamableToString(write_handle_as_size_t) + + " to a file descriptor"); + } + + // Signals the parent that the write end of the pipe has been acquired + // so the parent can release its own write end. + ::SetEvent(dup_event_handle); + + return write_fd; +} +# endif // GTEST_OS_WINDOWS + +// Returns a newly created InternalRunDeathTestFlag object with fields +// initialized from the GTEST_FLAG(internal_run_death_test) flag if +// the flag is specified; otherwise returns NULL. +InternalRunDeathTestFlag* ParseInternalRunDeathTestFlag() { + if (GTEST_FLAG(internal_run_death_test) == "") return NULL; + + // GTEST_HAS_DEATH_TEST implies that we have ::std::string, so we + // can use it here. + int line = -1; + int index = -1; + ::std::vector< ::std::string> fields; + SplitString(GTEST_FLAG(internal_run_death_test).c_str(), '|', &fields); + int write_fd = -1; + +# if GTEST_OS_WINDOWS + + unsigned int parent_process_id = 0; + size_t write_handle_as_size_t = 0; + size_t event_handle_as_size_t = 0; + + if (fields.size() != 6 + || !ParseNaturalNumber(fields[1], &line) + || !ParseNaturalNumber(fields[2], &index) + || !ParseNaturalNumber(fields[3], &parent_process_id) + || !ParseNaturalNumber(fields[4], &write_handle_as_size_t) + || !ParseNaturalNumber(fields[5], &event_handle_as_size_t)) { + DeathTestAbort("Bad --gtest_internal_run_death_test flag: " + + GTEST_FLAG(internal_run_death_test)); + } + write_fd = GetStatusFileDescriptor(parent_process_id, + write_handle_as_size_t, + event_handle_as_size_t); +# else + + if (fields.size() != 4 + || !ParseNaturalNumber(fields[1], &line) + || !ParseNaturalNumber(fields[2], &index) + || !ParseNaturalNumber(fields[3], &write_fd)) { + DeathTestAbort("Bad --gtest_internal_run_death_test flag: " + + GTEST_FLAG(internal_run_death_test)); + } + +# endif // GTEST_OS_WINDOWS + + return new InternalRunDeathTestFlag(fields[0], line, index, write_fd); +} + +} // namespace internal + +#endif // GTEST_HAS_DEATH_TEST + +} // namespace testing +// Copyright 2008, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Authors: keith.ray@gmail.com (Keith Ray) + + +#include + +#if GTEST_OS_WINDOWS_MOBILE +# include +#elif GTEST_OS_WINDOWS +# include +# include +#elif GTEST_OS_SYMBIAN +// Symbian OpenC has PATH_MAX in sys/syslimits.h +# include +#else +# include +# include // Some Linux distributions define PATH_MAX here. +#endif // GTEST_OS_WINDOWS_MOBILE + +#if GTEST_OS_WINDOWS +# define GTEST_PATH_MAX_ _MAX_PATH +#elif defined(PATH_MAX) +# define GTEST_PATH_MAX_ PATH_MAX +#elif defined(_XOPEN_PATH_MAX) +# define GTEST_PATH_MAX_ _XOPEN_PATH_MAX +#else +# define GTEST_PATH_MAX_ _POSIX_PATH_MAX +#endif // GTEST_OS_WINDOWS + + +namespace testing { +namespace internal { + +#if GTEST_OS_WINDOWS +// On Windows, '\\' is the standard path separator, but many tools and the +// Windows API also accept '/' as an alternate path separator. Unless otherwise +// noted, a file path can contain either kind of path separators, or a mixture +// of them. +const char kPathSeparator = '\\'; +const char kAlternatePathSeparator = '/'; +const char kAlternatePathSeparatorString[] = "/"; +# if GTEST_OS_WINDOWS_MOBILE +// Windows CE doesn't have a current directory. You should not use +// the current directory in tests on Windows CE, but this at least +// provides a reasonable fallback. +const char kCurrentDirectoryString[] = "\\"; +// Windows CE doesn't define INVALID_FILE_ATTRIBUTES +const DWORD kInvalidFileAttributes = 0xffffffff; +# else +const char kCurrentDirectoryString[] = ".\\"; +# endif // GTEST_OS_WINDOWS_MOBILE +#else +const char kPathSeparator = '/'; +const char kCurrentDirectoryString[] = "./"; +#endif // GTEST_OS_WINDOWS + +// Returns whether the given character is a valid path separator. +static bool IsPathSeparator(char c) { +#if GTEST_HAS_ALT_PATH_SEP_ + return (c == kPathSeparator) || (c == kAlternatePathSeparator); +#else + return c == kPathSeparator; +#endif +} + +// Returns the current working directory, or "" if unsuccessful. +FilePath FilePath::GetCurrentDir() { +#if GTEST_OS_WINDOWS_MOBILE || GTEST_OS_WINDOWS_PHONE || GTEST_OS_WINDOWS_RT + // Windows CE doesn't have a current directory, so we just return + // something reasonable. + return FilePath(kCurrentDirectoryString); +#elif GTEST_OS_WINDOWS + char cwd[GTEST_PATH_MAX_ + 1] = { '\0' }; + return FilePath(_getcwd(cwd, sizeof(cwd)) == NULL ? "" : cwd); +#else + char cwd[GTEST_PATH_MAX_ + 1] = { '\0' }; + char* result = getcwd(cwd, sizeof(cwd)); +# if GTEST_OS_NACL + // getcwd will likely fail in NaCl due to the sandbox, so return something + // reasonable. The user may have provided a shim implementation for getcwd, + // however, so fallback only when failure is detected. + return FilePath(result == NULL ? kCurrentDirectoryString : cwd); +# endif // GTEST_OS_NACL + return FilePath(result == NULL ? "" : cwd); +#endif // GTEST_OS_WINDOWS_MOBILE +} + +// Returns a copy of the FilePath with the case-insensitive extension removed. +// Example: FilePath("dir/file.exe").RemoveExtension("EXE") returns +// FilePath("dir/file"). If a case-insensitive extension is not +// found, returns a copy of the original FilePath. +FilePath FilePath::RemoveExtension(const char* extension) const { + const std::string dot_extension = std::string(".") + extension; + if (String::EndsWithCaseInsensitive(pathname_, dot_extension)) { + return FilePath(pathname_.substr( + 0, pathname_.length() - dot_extension.length())); + } + return *this; +} + +// Returns a pointer to the last occurrence of a valid path separator in +// the FilePath. On Windows, for example, both '/' and '\' are valid path +// separators. Returns NULL if no path separator was found. +const char* FilePath::FindLastPathSeparator() const { + const char* const last_sep = strrchr(c_str(), kPathSeparator); +#if GTEST_HAS_ALT_PATH_SEP_ + const char* const last_alt_sep = strrchr(c_str(), kAlternatePathSeparator); + // Comparing two pointers of which only one is NULL is undefined. + if (last_alt_sep != NULL && + (last_sep == NULL || last_alt_sep > last_sep)) { + return last_alt_sep; + } +#endif + return last_sep; +} + +// Returns a copy of the FilePath with the directory part removed. +// Example: FilePath("path/to/file").RemoveDirectoryName() returns +// FilePath("file"). If there is no directory part ("just_a_file"), it returns +// the FilePath unmodified. If there is no file part ("just_a_dir/") it +// returns an empty FilePath (""). +// On Windows platform, '\' is the path separator, otherwise it is '/'. +FilePath FilePath::RemoveDirectoryName() const { + const char* const last_sep = FindLastPathSeparator(); + return last_sep ? FilePath(last_sep + 1) : *this; +} + +// RemoveFileName returns the directory path with the filename removed. +// Example: FilePath("path/to/file").RemoveFileName() returns "path/to/". +// If the FilePath is "a_file" or "/a_file", RemoveFileName returns +// FilePath("./") or, on Windows, FilePath(".\\"). If the filepath does +// not have a file, like "just/a/dir/", it returns the FilePath unmodified. +// On Windows platform, '\' is the path separator, otherwise it is '/'. +FilePath FilePath::RemoveFileName() const { + const char* const last_sep = FindLastPathSeparator(); + std::string dir; + if (last_sep) { + dir = std::string(c_str(), last_sep + 1 - c_str()); + } else { + dir = kCurrentDirectoryString; + } + return FilePath(dir); +} + +// Helper functions for naming files in a directory for xml output. + +// Given directory = "dir", base_name = "test", number = 0, +// extension = "xml", returns "dir/test.xml". If number is greater +// than zero (e.g., 12), returns "dir/test_12.xml". +// On Windows platform, uses \ as the separator rather than /. +FilePath FilePath::MakeFileName(const FilePath& directory, + const FilePath& base_name, + int number, + const char* extension) { + std::string file; + if (number == 0) { + file = base_name.string() + "." + extension; + } else { + file = base_name.string() + "_" + StreamableToString(number) + + "." + extension; + } + return ConcatPaths(directory, FilePath(file)); +} + +// Given directory = "dir", relative_path = "test.xml", returns "dir/test.xml". +// On Windows, uses \ as the separator rather than /. +FilePath FilePath::ConcatPaths(const FilePath& directory, + const FilePath& relative_path) { + if (directory.IsEmpty()) + return relative_path; + const FilePath dir(directory.RemoveTrailingPathSeparator()); + return FilePath(dir.string() + kPathSeparator + relative_path.string()); +} + +// Returns true if pathname describes something findable in the file-system, +// either a file, directory, or whatever. +bool FilePath::FileOrDirectoryExists() const { +#if GTEST_OS_WINDOWS_MOBILE + LPCWSTR unicode = String::AnsiToUtf16(pathname_.c_str()); + const DWORD attributes = GetFileAttributes(unicode); + delete [] unicode; + return attributes != kInvalidFileAttributes; +#else + posix::StatStruct file_stat; + return posix::Stat(pathname_.c_str(), &file_stat) == 0; +#endif // GTEST_OS_WINDOWS_MOBILE +} + +// Returns true if pathname describes a directory in the file-system +// that exists. +bool FilePath::DirectoryExists() const { + bool result = false; +#if GTEST_OS_WINDOWS + // Don't strip off trailing separator if path is a root directory on + // Windows (like "C:\\"). + const FilePath& path(IsRootDirectory() ? *this : + RemoveTrailingPathSeparator()); +#else + const FilePath& path(*this); +#endif + +#if GTEST_OS_WINDOWS_MOBILE + LPCWSTR unicode = String::AnsiToUtf16(path.c_str()); + const DWORD attributes = GetFileAttributes(unicode); + delete [] unicode; + if ((attributes != kInvalidFileAttributes) && + (attributes & FILE_ATTRIBUTE_DIRECTORY)) { + result = true; + } +#else + posix::StatStruct file_stat; + result = posix::Stat(path.c_str(), &file_stat) == 0 && + posix::IsDir(file_stat); +#endif // GTEST_OS_WINDOWS_MOBILE + + return result; +} + +// Returns true if pathname describes a root directory. (Windows has one +// root directory per disk drive.) +bool FilePath::IsRootDirectory() const { +#if GTEST_OS_WINDOWS + // TODO(wan@google.com): on Windows a network share like + // \\server\share can be a root directory, although it cannot be the + // current directory. Handle this properly. + return pathname_.length() == 3 && IsAbsolutePath(); +#else + return pathname_.length() == 1 && IsPathSeparator(pathname_.c_str()[0]); +#endif +} + +// Returns true if pathname describes an absolute path. +bool FilePath::IsAbsolutePath() const { + const char* const name = pathname_.c_str(); +#if GTEST_OS_WINDOWS + return pathname_.length() >= 3 && + ((name[0] >= 'a' && name[0] <= 'z') || + (name[0] >= 'A' && name[0] <= 'Z')) && + name[1] == ':' && + IsPathSeparator(name[2]); +#else + return IsPathSeparator(name[0]); +#endif +} + +// Returns a pathname for a file that does not currently exist. The pathname +// will be directory/base_name.extension or +// directory/base_name_.extension if directory/base_name.extension +// already exists. The number will be incremented until a pathname is found +// that does not already exist. +// Examples: 'dir/foo_test.xml' or 'dir/foo_test_1.xml'. +// There could be a race condition if two or more processes are calling this +// function at the same time -- they could both pick the same filename. +FilePath FilePath::GenerateUniqueFileName(const FilePath& directory, + const FilePath& base_name, + const char* extension) { + FilePath full_pathname; + int number = 0; + do { + full_pathname.Set(MakeFileName(directory, base_name, number++, extension)); + } while (full_pathname.FileOrDirectoryExists()); + return full_pathname; +} + +// Returns true if FilePath ends with a path separator, which indicates that +// it is intended to represent a directory. Returns false otherwise. +// This does NOT check that a directory (or file) actually exists. +bool FilePath::IsDirectory() const { + return !pathname_.empty() && + IsPathSeparator(pathname_.c_str()[pathname_.length() - 1]); +} + +// Create directories so that path exists. Returns true if successful or if +// the directories already exist; returns false if unable to create directories +// for any reason. +bool FilePath::CreateDirectoriesRecursively() const { + if (!this->IsDirectory()) { + return false; + } + + if (pathname_.length() == 0 || this->DirectoryExists()) { + return true; + } + + const FilePath parent(this->RemoveTrailingPathSeparator().RemoveFileName()); + return parent.CreateDirectoriesRecursively() && this->CreateFolder(); +} + +// Create the directory so that path exists. Returns true if successful or +// if the directory already exists; returns false if unable to create the +// directory for any reason, including if the parent directory does not +// exist. Not named "CreateDirectory" because that's a macro on Windows. +bool FilePath::CreateFolder() const { +#if GTEST_OS_WINDOWS_MOBILE + FilePath removed_sep(this->RemoveTrailingPathSeparator()); + LPCWSTR unicode = String::AnsiToUtf16(removed_sep.c_str()); + int result = CreateDirectory(unicode, NULL) ? 0 : -1; + delete [] unicode; +#elif GTEST_OS_WINDOWS + int result = _mkdir(pathname_.c_str()); +#else + int result = mkdir(pathname_.c_str(), 0777); +#endif // GTEST_OS_WINDOWS_MOBILE + + if (result == -1) { + return this->DirectoryExists(); // An error is OK if the directory exists. + } + return true; // No error. +} + +// If input name has a trailing separator character, remove it and return the +// name, otherwise return the name string unmodified. +// On Windows platform, uses \ as the separator, other platforms use /. +FilePath FilePath::RemoveTrailingPathSeparator() const { + return IsDirectory() + ? FilePath(pathname_.substr(0, pathname_.length() - 1)) + : *this; +} + +// Removes any redundant separators that might be in the pathname. +// For example, "bar///foo" becomes "bar/foo". Does not eliminate other +// redundancies that might be in a pathname involving "." or "..". +// TODO(wan@google.com): handle Windows network shares (e.g. \\server\share). +void FilePath::Normalize() { + if (pathname_.c_str() == NULL) { + pathname_ = ""; + return; + } + const char* src = pathname_.c_str(); + char* const dest = new char[pathname_.length() + 1]; + char* dest_ptr = dest; + memset(dest_ptr, 0, pathname_.length() + 1); + + while (*src != '\0') { + *dest_ptr = *src; + if (!IsPathSeparator(*src)) { + src++; + } else { +#if GTEST_HAS_ALT_PATH_SEP_ + if (*dest_ptr == kAlternatePathSeparator) { + *dest_ptr = kPathSeparator; + } +#endif + while (IsPathSeparator(*src)) + src++; + } + dest_ptr++; + } + *dest_ptr = '\0'; + pathname_ = dest; + delete[] dest; +} + +} // namespace internal +} // namespace testing +// Copyright 2008, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) + + +#include +#include +#include +#include + +#if GTEST_OS_WINDOWS +# include +# include +# include +# include // Used in ThreadLocal. +#else +# include +#endif // GTEST_OS_WINDOWS + +#if GTEST_OS_MAC +# include +# include +# include +#endif // GTEST_OS_MAC + +#if GTEST_OS_QNX +# include +# include +# include +#endif // GTEST_OS_QNX + + +// Indicates that this translation unit is part of Google Test's +// implementation. It must come before gtest-internal-inl.h is +// included, or there will be a compiler error. This trick exists to +// prevent the accidental inclusion of gtest-internal-inl.h in the +// user's code. +#define GTEST_IMPLEMENTATION_ 1 +#undef GTEST_IMPLEMENTATION_ + +namespace testing { +namespace internal { + +#if defined(_MSC_VER) || defined(__BORLANDC__) +// MSVC and C++Builder do not provide a definition of STDERR_FILENO. +const int kStdOutFileno = 1; +const int kStdErrFileno = 2; +#else +const int kStdOutFileno = STDOUT_FILENO; +const int kStdErrFileno = STDERR_FILENO; +#endif // _MSC_VER + +#if GTEST_OS_MAC + +// Returns the number of threads running in the process, or 0 to indicate that +// we cannot detect it. +size_t GetThreadCount() { + const task_t task = mach_task_self(); + mach_msg_type_number_t thread_count; + thread_act_array_t thread_list; + const kern_return_t status = task_threads(task, &thread_list, &thread_count); + if (status == KERN_SUCCESS) { + // task_threads allocates resources in thread_list and we need to free them + // to avoid leaks. + vm_deallocate(task, + reinterpret_cast(thread_list), + sizeof(thread_t) * thread_count); + return static_cast(thread_count); + } else { + return 0; + } +} + +#elif GTEST_OS_QNX + +// Returns the number of threads running in the process, or 0 to indicate that +// we cannot detect it. +size_t GetThreadCount() { + const int fd = open("/proc/self/as", O_RDONLY); + if (fd < 0) { + return 0; + } + procfs_info process_info; + const int status = + devctl(fd, DCMD_PROC_INFO, &process_info, sizeof(process_info), NULL); + close(fd); + if (status == EOK) { + return static_cast(process_info.num_threads); + } else { + return 0; + } +} + +#else + +size_t GetThreadCount() { + // There's no portable way to detect the number of threads, so we just + // return 0 to indicate that we cannot detect it. + return 0; +} + +#endif // GTEST_OS_MAC + +#if GTEST_IS_THREADSAFE && GTEST_OS_WINDOWS + +void SleepMilliseconds(int n) { + ::Sleep(n); +} + +AutoHandle::AutoHandle() + : handle_(INVALID_HANDLE_VALUE) {} + +AutoHandle::AutoHandle(Handle handle) + : handle_(handle) {} + +AutoHandle::~AutoHandle() { + Reset(); +} + +AutoHandle::Handle AutoHandle::Get() const { + return handle_; +} + +void AutoHandle::Reset() { + Reset(INVALID_HANDLE_VALUE); +} + +void AutoHandle::Reset(HANDLE handle) { + // Resetting with the same handle we already own is invalid. + if (handle_ != handle) { + if (IsCloseable()) { + ::CloseHandle(handle_); + } + handle_ = handle; + } else { + GTEST_CHECK_(!IsCloseable()) + << "Resetting a valid handle to itself is likely a programmer error " + "and thus not allowed."; + } +} + +bool AutoHandle::IsCloseable() const { + // Different Windows APIs may use either of these values to represent an + // invalid handle. + return handle_ != NULL && handle_ != INVALID_HANDLE_VALUE; +} + +Notification::Notification() + : event_(::CreateEvent(NULL, // Default security attributes. + TRUE, // Do not reset automatically. + FALSE, // Initially unset. + NULL)) { // Anonymous event. + GTEST_CHECK_(event_.Get() != NULL); +} + +void Notification::Notify() { + GTEST_CHECK_(::SetEvent(event_.Get()) != FALSE); +} + +void Notification::WaitForNotification() { + GTEST_CHECK_( + ::WaitForSingleObject(event_.Get(), INFINITE) == WAIT_OBJECT_0); +} + +Mutex::Mutex() + : owner_thread_id_(0), + type_(kDynamic), + critical_section_init_phase_(0), + critical_section_(new CRITICAL_SECTION) { + ::InitializeCriticalSection(critical_section_); +} + +Mutex::~Mutex() { + // Static mutexes are leaked intentionally. It is not thread-safe to try + // to clean them up. + // TODO(yukawa): Switch to Slim Reader/Writer (SRW) Locks, which requires + // nothing to clean it up but is available only on Vista and later. + // http://msdn.microsoft.com/en-us/library/windows/desktop/aa904937.aspx + if (type_ == kDynamic) { + ::DeleteCriticalSection(critical_section_); + delete critical_section_; + critical_section_ = NULL; + } +} + +void Mutex::Lock() { + ThreadSafeLazyInit(); + ::EnterCriticalSection(critical_section_); + owner_thread_id_ = ::GetCurrentThreadId(); +} + +void Mutex::Unlock() { + ThreadSafeLazyInit(); + // We don't protect writing to owner_thread_id_ here, as it's the + // caller's responsibility to ensure that the current thread holds the + // mutex when this is called. + owner_thread_id_ = 0; + ::LeaveCriticalSection(critical_section_); +} + +// Does nothing if the current thread holds the mutex. Otherwise, crashes +// with high probability. +void Mutex::AssertHeld() { + ThreadSafeLazyInit(); + GTEST_CHECK_(owner_thread_id_ == ::GetCurrentThreadId()) + << "The current thread is not holding the mutex @" << this; +} + +// Initializes owner_thread_id_ and critical_section_ in static mutexes. +void Mutex::ThreadSafeLazyInit() { + // Dynamic mutexes are initialized in the constructor. + if (type_ == kStatic) { + switch ( + ::InterlockedCompareExchange(&critical_section_init_phase_, 1L, 0L)) { + case 0: + // If critical_section_init_phase_ was 0 before the exchange, we + // are the first to test it and need to perform the initialization. + owner_thread_id_ = 0; + critical_section_ = new CRITICAL_SECTION; + ::InitializeCriticalSection(critical_section_); + // Updates the critical_section_init_phase_ to 2 to signal + // initialization complete. + GTEST_CHECK_(::InterlockedCompareExchange( + &critical_section_init_phase_, 2L, 1L) == + 1L); + break; + case 1: + // Somebody else is already initializing the mutex; spin until they + // are done. + while (::InterlockedCompareExchange(&critical_section_init_phase_, + 2L, + 2L) != 2L) { + // Possibly yields the rest of the thread's time slice to other + // threads. + ::Sleep(0); + } + break; + + case 2: + break; // The mutex is already initialized and ready for use. + + default: + GTEST_CHECK_(false) + << "Unexpected value of critical_section_init_phase_ " + << "while initializing a static mutex."; + } + } +} + +namespace { + +class ThreadWithParamSupport : public ThreadWithParamBase { + public: + static HANDLE CreateThread(Runnable* runnable, + Notification* thread_can_start) { + ThreadMainParam* param = new ThreadMainParam(runnable, thread_can_start); + DWORD thread_id; + // TODO(yukawa): Consider to use _beginthreadex instead. + HANDLE thread_handle = ::CreateThread( + NULL, // Default security. + 0, // Default stack size. + &ThreadWithParamSupport::ThreadMain, + param, // Parameter to ThreadMainStatic + 0x0, // Default creation flags. + &thread_id); // Need a valid pointer for the call to work under Win98. + GTEST_CHECK_(thread_handle != NULL) << "CreateThread failed with error " + << ::GetLastError() << "."; + if (thread_handle == NULL) { + delete param; + } + return thread_handle; + } + + private: + struct ThreadMainParam { + ThreadMainParam(Runnable* runnable, Notification* thread_can_start) + : runnable_(runnable), + thread_can_start_(thread_can_start) { + } + scoped_ptr runnable_; + // Does not own. + Notification* thread_can_start_; + }; + + static DWORD WINAPI ThreadMain(void* ptr) { + // Transfers ownership. + scoped_ptr param(static_cast(ptr)); + if (param->thread_can_start_ != NULL) + param->thread_can_start_->WaitForNotification(); + param->runnable_->Run(); + return 0; + } + + // Prohibit instantiation. + ThreadWithParamSupport(); + + GTEST_DISALLOW_COPY_AND_ASSIGN_(ThreadWithParamSupport); +}; + +} // namespace + +ThreadWithParamBase::ThreadWithParamBase(Runnable *runnable, + Notification* thread_can_start) + : thread_(ThreadWithParamSupport::CreateThread(runnable, + thread_can_start)) { +} + +ThreadWithParamBase::~ThreadWithParamBase() { + Join(); +} + +void ThreadWithParamBase::Join() { + GTEST_CHECK_(::WaitForSingleObject(thread_.Get(), INFINITE) == WAIT_OBJECT_0) + << "Failed to join the thread with error " << ::GetLastError() << "."; +} + +// Maps a thread to a set of ThreadIdToThreadLocals that have values +// instantiated on that thread and notifies them when the thread exits. A +// ThreadLocal instance is expected to persist until all threads it has +// values on have terminated. +class ThreadLocalRegistryImpl { + public: + // Registers thread_local_instance as having value on the current thread. + // Returns a value that can be used to identify the thread from other threads. + static ThreadLocalValueHolderBase* GetValueOnCurrentThread( + const ThreadLocalBase* thread_local_instance) { + DWORD current_thread = ::GetCurrentThreadId(); + MutexLock lock(&mutex_); + ThreadIdToThreadLocals* const thread_to_thread_locals = + GetThreadLocalsMapLocked(); + ThreadIdToThreadLocals::iterator thread_local_pos = + thread_to_thread_locals->find(current_thread); + if (thread_local_pos == thread_to_thread_locals->end()) { + thread_local_pos = thread_to_thread_locals->insert( + std::make_pair(current_thread, ThreadLocalValues())).first; + StartWatcherThreadFor(current_thread); + } + ThreadLocalValues& thread_local_values = thread_local_pos->second; + ThreadLocalValues::iterator value_pos = + thread_local_values.find(thread_local_instance); + if (value_pos == thread_local_values.end()) { + value_pos = + thread_local_values + .insert(std::make_pair( + thread_local_instance, + linked_ptr( + thread_local_instance->NewValueForCurrentThread()))) + .first; + } + return value_pos->second.get(); + } + + static void OnThreadLocalDestroyed( + const ThreadLocalBase* thread_local_instance) { + std::vector > value_holders; + // Clean up the ThreadLocalValues data structure while holding the lock, but + // defer the destruction of the ThreadLocalValueHolderBases. + { + MutexLock lock(&mutex_); + ThreadIdToThreadLocals* const thread_to_thread_locals = + GetThreadLocalsMapLocked(); + for (ThreadIdToThreadLocals::iterator it = + thread_to_thread_locals->begin(); + it != thread_to_thread_locals->end(); + ++it) { + ThreadLocalValues& thread_local_values = it->second; + ThreadLocalValues::iterator value_pos = + thread_local_values.find(thread_local_instance); + if (value_pos != thread_local_values.end()) { + value_holders.push_back(value_pos->second); + thread_local_values.erase(value_pos); + // This 'if' can only be successful at most once, so theoretically we + // could break out of the loop here, but we don't bother doing so. + } + } + } + // Outside the lock, let the destructor for 'value_holders' deallocate the + // ThreadLocalValueHolderBases. + } + + static void OnThreadExit(DWORD thread_id) { + GTEST_CHECK_(thread_id != 0) << ::GetLastError(); + std::vector > value_holders; + // Clean up the ThreadIdToThreadLocals data structure while holding the + // lock, but defer the destruction of the ThreadLocalValueHolderBases. + { + MutexLock lock(&mutex_); + ThreadIdToThreadLocals* const thread_to_thread_locals = + GetThreadLocalsMapLocked(); + ThreadIdToThreadLocals::iterator thread_local_pos = + thread_to_thread_locals->find(thread_id); + if (thread_local_pos != thread_to_thread_locals->end()) { + ThreadLocalValues& thread_local_values = thread_local_pos->second; + for (ThreadLocalValues::iterator value_pos = + thread_local_values.begin(); + value_pos != thread_local_values.end(); + ++value_pos) { + value_holders.push_back(value_pos->second); + } + thread_to_thread_locals->erase(thread_local_pos); + } + } + // Outside the lock, let the destructor for 'value_holders' deallocate the + // ThreadLocalValueHolderBases. + } + + private: + // In a particular thread, maps a ThreadLocal object to its value. + typedef std::map > ThreadLocalValues; + // Stores all ThreadIdToThreadLocals having values in a thread, indexed by + // thread's ID. + typedef std::map ThreadIdToThreadLocals; + + // Holds the thread id and thread handle that we pass from + // StartWatcherThreadFor to WatcherThreadFunc. + typedef std::pair ThreadIdAndHandle; + + static void StartWatcherThreadFor(DWORD thread_id) { + // The returned handle will be kept in thread_map and closed by + // watcher_thread in WatcherThreadFunc. + HANDLE thread = ::OpenThread(SYNCHRONIZE | THREAD_QUERY_INFORMATION, + FALSE, + thread_id); + GTEST_CHECK_(thread != NULL); + // We need to to pass a valid thread ID pointer into CreateThread for it + // to work correctly under Win98. + DWORD watcher_thread_id; + HANDLE watcher_thread = ::CreateThread( + NULL, // Default security. + 0, // Default stack size + &ThreadLocalRegistryImpl::WatcherThreadFunc, + reinterpret_cast(new ThreadIdAndHandle(thread_id, thread)), + CREATE_SUSPENDED, + &watcher_thread_id); + GTEST_CHECK_(watcher_thread != NULL); + // Give the watcher thread the same priority as ours to avoid being + // blocked by it. + ::SetThreadPriority(watcher_thread, + ::GetThreadPriority(::GetCurrentThread())); + ::ResumeThread(watcher_thread); + ::CloseHandle(watcher_thread); + } + + // Monitors exit from a given thread and notifies those + // ThreadIdToThreadLocals about thread termination. + static DWORD WINAPI WatcherThreadFunc(LPVOID param) { + const ThreadIdAndHandle* tah = + reinterpret_cast(param); + GTEST_CHECK_( + ::WaitForSingleObject(tah->second, INFINITE) == WAIT_OBJECT_0); + OnThreadExit(tah->first); + ::CloseHandle(tah->second); + delete tah; + return 0; + } + + // Returns map of thread local instances. + static ThreadIdToThreadLocals* GetThreadLocalsMapLocked() { + mutex_.AssertHeld(); + static ThreadIdToThreadLocals* map = new ThreadIdToThreadLocals; + return map; + } + + // Protects access to GetThreadLocalsMapLocked() and its return value. + static Mutex mutex_; + // Protects access to GetThreadMapLocked() and its return value. + static Mutex thread_map_mutex_; +}; + +Mutex ThreadLocalRegistryImpl::mutex_(Mutex::kStaticMutex); +Mutex ThreadLocalRegistryImpl::thread_map_mutex_(Mutex::kStaticMutex); + +ThreadLocalValueHolderBase* ThreadLocalRegistry::GetValueOnCurrentThread( + const ThreadLocalBase* thread_local_instance) { + return ThreadLocalRegistryImpl::GetValueOnCurrentThread( + thread_local_instance); +} + +void ThreadLocalRegistry::OnThreadLocalDestroyed( + const ThreadLocalBase* thread_local_instance) { + ThreadLocalRegistryImpl::OnThreadLocalDestroyed(thread_local_instance); +} + +#endif // GTEST_IS_THREADSAFE && GTEST_OS_WINDOWS + +#if GTEST_USES_POSIX_RE + +// Implements RE. Currently only needed for death tests. + +RE::~RE() { + if (is_valid_) { + // regfree'ing an invalid regex might crash because the content + // of the regex is undefined. Since the regex's are essentially + // the same, one cannot be valid (or invalid) without the other + // being so too. + regfree(&partial_regex_); + regfree(&full_regex_); + } + free(const_cast(pattern_)); +} + +// Returns true iff regular expression re matches the entire str. +bool RE::FullMatch(const char* str, const RE& re) { + if (!re.is_valid_) return false; + + regmatch_t match; + return regexec(&re.full_regex_, str, 1, &match, 0) == 0; +} + +// Returns true iff regular expression re matches a substring of str +// (including str itself). +bool RE::PartialMatch(const char* str, const RE& re) { + if (!re.is_valid_) return false; + + regmatch_t match; + return regexec(&re.partial_regex_, str, 1, &match, 0) == 0; +} + +// Initializes an RE from its string representation. +void RE::Init(const char* regex) { + pattern_ = posix::StrDup(regex); + + // Reserves enough bytes to hold the regular expression used for a + // full match. + const size_t full_regex_len = strlen(regex) + 10; + char* const full_pattern = new char[full_regex_len]; + + snprintf(full_pattern, full_regex_len, "^(%s)$", regex); + is_valid_ = regcomp(&full_regex_, full_pattern, REG_EXTENDED) == 0; + // We want to call regcomp(&partial_regex_, ...) even if the + // previous expression returns false. Otherwise partial_regex_ may + // not be properly initialized can may cause trouble when it's + // freed. + // + // Some implementation of POSIX regex (e.g. on at least some + // versions of Cygwin) doesn't accept the empty string as a valid + // regex. We change it to an equivalent form "()" to be safe. + if (is_valid_) { + const char* const partial_regex = (*regex == '\0') ? "()" : regex; + is_valid_ = regcomp(&partial_regex_, partial_regex, REG_EXTENDED) == 0; + } + EXPECT_TRUE(is_valid_) + << "Regular expression \"" << regex + << "\" is not a valid POSIX Extended regular expression."; + + delete[] full_pattern; +} + +#elif GTEST_USES_SIMPLE_RE + +// Returns true iff ch appears anywhere in str (excluding the +// terminating '\0' character). +bool IsInSet(char ch, const char* str) { + return ch != '\0' && strchr(str, ch) != NULL; +} + +// Returns true iff ch belongs to the given classification. Unlike +// similar functions in , these aren't affected by the +// current locale. +bool IsAsciiDigit(char ch) { return '0' <= ch && ch <= '9'; } +bool IsAsciiPunct(char ch) { + return IsInSet(ch, "^-!\"#$%&'()*+,./:;<=>?@[\\]_`{|}~"); +} +bool IsRepeat(char ch) { return IsInSet(ch, "?*+"); } +bool IsAsciiWhiteSpace(char ch) { return IsInSet(ch, " \f\n\r\t\v"); } +bool IsAsciiWordChar(char ch) { + return ('a' <= ch && ch <= 'z') || ('A' <= ch && ch <= 'Z') || + ('0' <= ch && ch <= '9') || ch == '_'; +} + +// Returns true iff "\\c" is a supported escape sequence. +bool IsValidEscape(char c) { + return (IsAsciiPunct(c) || IsInSet(c, "dDfnrsStvwW")); +} + +// Returns true iff the given atom (specified by escaped and pattern) +// matches ch. The result is undefined if the atom is invalid. +bool AtomMatchesChar(bool escaped, char pattern_char, char ch) { + if (escaped) { // "\\p" where p is pattern_char. + switch (pattern_char) { + case 'd': return IsAsciiDigit(ch); + case 'D': return !IsAsciiDigit(ch); + case 'f': return ch == '\f'; + case 'n': return ch == '\n'; + case 'r': return ch == '\r'; + case 's': return IsAsciiWhiteSpace(ch); + case 'S': return !IsAsciiWhiteSpace(ch); + case 't': return ch == '\t'; + case 'v': return ch == '\v'; + case 'w': return IsAsciiWordChar(ch); + case 'W': return !IsAsciiWordChar(ch); + } + return IsAsciiPunct(pattern_char) && pattern_char == ch; + } + + return (pattern_char == '.' && ch != '\n') || pattern_char == ch; +} + +// Helper function used by ValidateRegex() to format error messages. +std::string FormatRegexSyntaxError(const char* regex, int index) { + return (Message() << "Syntax error at index " << index + << " in simple regular expression \"" << regex << "\": ").GetString(); +} + +// Generates non-fatal failures and returns false if regex is invalid; +// otherwise returns true. +bool ValidateRegex(const char* regex) { + if (regex == NULL) { + // TODO(wan@google.com): fix the source file location in the + // assertion failures to match where the regex is used in user + // code. + ADD_FAILURE() << "NULL is not a valid simple regular expression."; + return false; + } + + bool is_valid = true; + + // True iff ?, *, or + can follow the previous atom. + bool prev_repeatable = false; + for (int i = 0; regex[i]; i++) { + if (regex[i] == '\\') { // An escape sequence + i++; + if (regex[i] == '\0') { + ADD_FAILURE() << FormatRegexSyntaxError(regex, i - 1) + << "'\\' cannot appear at the end."; + return false; + } + + if (!IsValidEscape(regex[i])) { + ADD_FAILURE() << FormatRegexSyntaxError(regex, i - 1) + << "invalid escape sequence \"\\" << regex[i] << "\"."; + is_valid = false; + } + prev_repeatable = true; + } else { // Not an escape sequence. + const char ch = regex[i]; + + if (ch == '^' && i > 0) { + ADD_FAILURE() << FormatRegexSyntaxError(regex, i) + << "'^' can only appear at the beginning."; + is_valid = false; + } else if (ch == '$' && regex[i + 1] != '\0') { + ADD_FAILURE() << FormatRegexSyntaxError(regex, i) + << "'$' can only appear at the end."; + is_valid = false; + } else if (IsInSet(ch, "()[]{}|")) { + ADD_FAILURE() << FormatRegexSyntaxError(regex, i) + << "'" << ch << "' is unsupported."; + is_valid = false; + } else if (IsRepeat(ch) && !prev_repeatable) { + ADD_FAILURE() << FormatRegexSyntaxError(regex, i) + << "'" << ch << "' can only follow a repeatable token."; + is_valid = false; + } + + prev_repeatable = !IsInSet(ch, "^$?*+"); + } + } + + return is_valid; +} + +// Matches a repeated regex atom followed by a valid simple regular +// expression. The regex atom is defined as c if escaped is false, +// or \c otherwise. repeat is the repetition meta character (?, *, +// or +). The behavior is undefined if str contains too many +// characters to be indexable by size_t, in which case the test will +// probably time out anyway. We are fine with this limitation as +// std::string has it too. +bool MatchRepetitionAndRegexAtHead( + bool escaped, char c, char repeat, const char* regex, + const char* str) { + const size_t min_count = (repeat == '+') ? 1 : 0; + const size_t max_count = (repeat == '?') ? 1 : + static_cast(-1) - 1; + // We cannot call numeric_limits::max() as it conflicts with the + // max() macro on Windows. + + for (size_t i = 0; i <= max_count; ++i) { + // We know that the atom matches each of the first i characters in str. + if (i >= min_count && MatchRegexAtHead(regex, str + i)) { + // We have enough matches at the head, and the tail matches too. + // Since we only care about *whether* the pattern matches str + // (as opposed to *how* it matches), there is no need to find a + // greedy match. + return true; + } + if (str[i] == '\0' || !AtomMatchesChar(escaped, c, str[i])) + return false; + } + return false; +} + +// Returns true iff regex matches a prefix of str. regex must be a +// valid simple regular expression and not start with "^", or the +// result is undefined. +bool MatchRegexAtHead(const char* regex, const char* str) { + if (*regex == '\0') // An empty regex matches a prefix of anything. + return true; + + // "$" only matches the end of a string. Note that regex being + // valid guarantees that there's nothing after "$" in it. + if (*regex == '$') + return *str == '\0'; + + // Is the first thing in regex an escape sequence? + const bool escaped = *regex == '\\'; + if (escaped) + ++regex; + if (IsRepeat(regex[1])) { + // MatchRepetitionAndRegexAtHead() calls MatchRegexAtHead(), so + // here's an indirect recursion. It terminates as the regex gets + // shorter in each recursion. + return MatchRepetitionAndRegexAtHead( + escaped, regex[0], regex[1], regex + 2, str); + } else { + // regex isn't empty, isn't "$", and doesn't start with a + // repetition. We match the first atom of regex with the first + // character of str and recurse. + return (*str != '\0') && AtomMatchesChar(escaped, *regex, *str) && + MatchRegexAtHead(regex + 1, str + 1); + } +} + +// Returns true iff regex matches any substring of str. regex must be +// a valid simple regular expression, or the result is undefined. +// +// The algorithm is recursive, but the recursion depth doesn't exceed +// the regex length, so we won't need to worry about running out of +// stack space normally. In rare cases the time complexity can be +// exponential with respect to the regex length + the string length, +// but usually it's must faster (often close to linear). +bool MatchRegexAnywhere(const char* regex, const char* str) { + if (regex == NULL || str == NULL) + return false; + + if (*regex == '^') + return MatchRegexAtHead(regex + 1, str); + + // A successful match can be anywhere in str. + do { + if (MatchRegexAtHead(regex, str)) + return true; + } while (*str++ != '\0'); + return false; +} + +// Implements the RE class. + +RE::~RE() { + free(const_cast(pattern_)); + free(const_cast(full_pattern_)); +} + +// Returns true iff regular expression re matches the entire str. +bool RE::FullMatch(const char* str, const RE& re) { + return re.is_valid_ && MatchRegexAnywhere(re.full_pattern_, str); +} + +// Returns true iff regular expression re matches a substring of str +// (including str itself). +bool RE::PartialMatch(const char* str, const RE& re) { + return re.is_valid_ && MatchRegexAnywhere(re.pattern_, str); +} + +// Initializes an RE from its string representation. +void RE::Init(const char* regex) { + pattern_ = full_pattern_ = NULL; + if (regex != NULL) { + pattern_ = posix::StrDup(regex); + } + + is_valid_ = ValidateRegex(regex); + if (!is_valid_) { + // No need to calculate the full pattern when the regex is invalid. + return; + } + + const size_t len = strlen(regex); + // Reserves enough bytes to hold the regular expression used for a + // full match: we need space to prepend a '^', append a '$', and + // terminate the string with '\0'. + char* buffer = static_cast(malloc(len + 3)); + full_pattern_ = buffer; + + if (*regex != '^') + *buffer++ = '^'; // Makes sure full_pattern_ starts with '^'. + + // We don't use snprintf or strncpy, as they trigger a warning when + // compiled with VC++ 8.0. + memcpy(buffer, regex, len); + buffer += len; + + if (len == 0 || regex[len - 1] != '$') + *buffer++ = '$'; // Makes sure full_pattern_ ends with '$'. + + *buffer = '\0'; +} + +#endif // GTEST_USES_POSIX_RE + +const char kUnknownFile[] = "unknown file"; + +// Formats a source file path and a line number as they would appear +// in an error message from the compiler used to compile this code. +GTEST_API_ ::std::string FormatFileLocation(const char* file, int line) { + const std::string file_name(file == NULL ? kUnknownFile : file); + + if (line < 0) { + return file_name + ":"; + } +#ifdef _MSC_VER + return file_name + "(" + StreamableToString(line) + "):"; +#else + return file_name + ":" + StreamableToString(line) + ":"; +#endif // _MSC_VER +} + +// Formats a file location for compiler-independent XML output. +// Although this function is not platform dependent, we put it next to +// FormatFileLocation in order to contrast the two functions. +// Note that FormatCompilerIndependentFileLocation() does NOT append colon +// to the file location it produces, unlike FormatFileLocation(). +GTEST_API_ ::std::string FormatCompilerIndependentFileLocation( + const char* file, int line) { + const std::string file_name(file == NULL ? kUnknownFile : file); + + if (line < 0) + return file_name; + else + return file_name + ":" + StreamableToString(line); +} + + +GTestLog::GTestLog(GTestLogSeverity severity, const char* file, int line) + : severity_(severity) { + const char* const marker = + severity == GTEST_INFO ? "[ INFO ]" : + severity == GTEST_WARNING ? "[WARNING]" : + severity == GTEST_ERROR ? "[ ERROR ]" : "[ FATAL ]"; + GetStream() << ::std::endl << marker << " " + << FormatFileLocation(file, line).c_str() << ": "; +} + +// Flushes the buffers and, if severity is GTEST_FATAL, aborts the program. +GTestLog::~GTestLog() { + GetStream() << ::std::endl; + if (severity_ == GTEST_FATAL) { + fflush(stderr); + posix::Abort(); + } +} +// Disable Microsoft deprecation warnings for POSIX functions called from +// this class (creat, dup, dup2, and close) +GTEST_DISABLE_MSC_WARNINGS_PUSH_(4996) + +#if GTEST_HAS_STREAM_REDIRECTION + +// Object that captures an output stream (stdout/stderr). +class CapturedStream { + public: + // The ctor redirects the stream to a temporary file. + explicit CapturedStream(int fd) : fd_(fd), uncaptured_fd_(dup(fd)) { +# if GTEST_OS_WINDOWS + char temp_dir_path[MAX_PATH + 1] = { '\0' }; // NOLINT + char temp_file_path[MAX_PATH + 1] = { '\0' }; // NOLINT + + ::GetTempPathA(sizeof(temp_dir_path), temp_dir_path); + const UINT success = ::GetTempFileNameA(temp_dir_path, + "gtest_redir", + 0, // Generate unique file name. + temp_file_path); + GTEST_CHECK_(success != 0) + << "Unable to create a temporary file in " << temp_dir_path; + const int captured_fd = creat(temp_file_path, _S_IREAD | _S_IWRITE); + GTEST_CHECK_(captured_fd != -1) << "Unable to open temporary file " + << temp_file_path; + filename_ = temp_file_path; +# else + // There's no guarantee that a test has write access to the current + // directory, so we create the temporary file in the /tmp directory + // instead. We use /tmp on most systems, and /sdcard on Android. + // That's because Android doesn't have /tmp. +# if GTEST_OS_LINUX_ANDROID + // Note: Android applications are expected to call the framework's + // Context.getExternalStorageDirectory() method through JNI to get + // the location of the world-writable SD Card directory. However, + // this requires a Context handle, which cannot be retrieved + // globally from native code. Doing so also precludes running the + // code as part of a regular standalone executable, which doesn't + // run in a Dalvik process (e.g. when running it through 'adb shell'). + // + // The location /sdcard is directly accessible from native code + // and is the only location (unofficially) supported by the Android + // team. It's generally a symlink to the real SD Card mount point + // which can be /mnt/sdcard, /mnt/sdcard0, /system/media/sdcard, or + // other OEM-customized locations. Never rely on these, and always + // use /sdcard. + char name_template[] = "/sdcard/gtest_captured_stream.XXXXXX"; +# else + char name_template[] = "/tmp/captured_stream.XXXXXX"; +# endif // GTEST_OS_LINUX_ANDROID + const int captured_fd = mkstemp(name_template); + filename_ = name_template; +# endif // GTEST_OS_WINDOWS + fflush(NULL); + dup2(captured_fd, fd_); + close(captured_fd); + } + + ~CapturedStream() { + remove(filename_.c_str()); + } + + std::string GetCapturedString() { + if (uncaptured_fd_ != -1) { + // Restores the original stream. + fflush(NULL); + dup2(uncaptured_fd_, fd_); + close(uncaptured_fd_); + uncaptured_fd_ = -1; + } + + FILE* const file = posix::FOpen(filename_.c_str(), "r"); + const std::string content = ReadEntireFile(file); + posix::FClose(file); + return content; + } + + private: + // Reads the entire content of a file as an std::string. + static std::string ReadEntireFile(FILE* file); + + // Returns the size (in bytes) of a file. + static size_t GetFileSize(FILE* file); + + const int fd_; // A stream to capture. + int uncaptured_fd_; + // Name of the temporary file holding the stderr output. + ::std::string filename_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(CapturedStream); +}; + +// Returns the size (in bytes) of a file. +size_t CapturedStream::GetFileSize(FILE* file) { + fseek(file, 0, SEEK_END); + return static_cast(ftell(file)); +} + +// Reads the entire content of a file as a string. +std::string CapturedStream::ReadEntireFile(FILE* file) { + const size_t file_size = GetFileSize(file); + char* const buffer = new char[file_size]; + + size_t bytes_last_read = 0; // # of bytes read in the last fread() + size_t bytes_read = 0; // # of bytes read so far + + fseek(file, 0, SEEK_SET); + + // Keeps reading the file until we cannot read further or the + // pre-determined file size is reached. + do { + bytes_last_read = fread(buffer+bytes_read, 1, file_size-bytes_read, file); + bytes_read += bytes_last_read; + } while (bytes_last_read > 0 && bytes_read < file_size); + + const std::string content(buffer, bytes_read); + delete[] buffer; + + return content; +} + +GTEST_DISABLE_MSC_WARNINGS_POP_() + +static CapturedStream* g_captured_stderr = NULL; +static CapturedStream* g_captured_stdout = NULL; + +// Starts capturing an output stream (stdout/stderr). +void CaptureStream(int fd, const char* stream_name, CapturedStream** stream) { + if (*stream != NULL) { + GTEST_LOG_(FATAL) << "Only one " << stream_name + << " capturer can exist at a time."; + } + *stream = new CapturedStream(fd); +} + +// Stops capturing the output stream and returns the captured string. +std::string GetCapturedStream(CapturedStream** captured_stream) { + const std::string content = (*captured_stream)->GetCapturedString(); + + delete *captured_stream; + *captured_stream = NULL; + + return content; +} + +// Starts capturing stdout. +void CaptureStdout() { + CaptureStream(kStdOutFileno, "stdout", &g_captured_stdout); +} + +// Starts capturing stderr. +void CaptureStderr() { + CaptureStream(kStdErrFileno, "stderr", &g_captured_stderr); +} + +// Stops capturing stdout and returns the captured string. +std::string GetCapturedStdout() { + return GetCapturedStream(&g_captured_stdout); +} + +// Stops capturing stderr and returns the captured string. +std::string GetCapturedStderr() { + return GetCapturedStream(&g_captured_stderr); +} + +#endif // GTEST_HAS_STREAM_REDIRECTION + +#if GTEST_HAS_DEATH_TEST + +// A copy of all command line arguments. Set by InitGoogleTest(). +::std::vector g_argvs; + +static const ::std::vector* g_injected_test_argvs = + NULL; // Owned. + +void SetInjectableArgvs(const ::std::vector* argvs) { + if (g_injected_test_argvs != argvs) + delete g_injected_test_argvs; + g_injected_test_argvs = argvs; +} + +const ::std::vector& GetInjectableArgvs() { + if (g_injected_test_argvs != NULL) { + return *g_injected_test_argvs; + } + return g_argvs; +} +#endif // GTEST_HAS_DEATH_TEST + +#if GTEST_OS_WINDOWS_MOBILE +namespace posix { +void Abort() { + DebugBreak(); + TerminateProcess(GetCurrentProcess(), 1); +} +} // namespace posix +#endif // GTEST_OS_WINDOWS_MOBILE + +// Returns the name of the environment variable corresponding to the +// given flag. For example, FlagToEnvVar("foo") will return +// "GTEST_FOO" in the open-source version. +static std::string FlagToEnvVar(const char* flag) { + const std::string full_flag = + (Message() << GTEST_FLAG_PREFIX_ << flag).GetString(); + + Message env_var; + for (size_t i = 0; i != full_flag.length(); i++) { + env_var << ToUpper(full_flag.c_str()[i]); + } + + return env_var.GetString(); +} + +// Parses 'str' for a 32-bit signed integer. If successful, writes +// the result to *value and returns true; otherwise leaves *value +// unchanged and returns false. +bool ParseInt32(const Message& src_text, const char* str, Int32* value) { + // Parses the environment variable as a decimal integer. + char* end = NULL; + const long long_value = strtol(str, &end, 10); // NOLINT + + // Has strtol() consumed all characters in the string? + if (*end != '\0') { + // No - an invalid character was encountered. + Message msg; + msg << "WARNING: " << src_text + << " is expected to be a 32-bit integer, but actually" + << " has value \"" << str << "\".\n"; + printf("%s", msg.GetString().c_str()); + fflush(stdout); + return false; + } + + // Is the parsed value in the range of an Int32? + const Int32 result = static_cast(long_value); + if (long_value == LONG_MAX || long_value == LONG_MIN || + // The parsed value overflows as a long. (strtol() returns + // LONG_MAX or LONG_MIN when the input overflows.) + result != long_value + // The parsed value overflows as an Int32. + ) { + Message msg; + msg << "WARNING: " << src_text + << " is expected to be a 32-bit integer, but actually" + << " has value " << str << ", which overflows.\n"; + printf("%s", msg.GetString().c_str()); + fflush(stdout); + return false; + } + + *value = result; + return true; +} + +// Reads and returns the Boolean environment variable corresponding to +// the given flag; if it's not set, returns default_value. +// +// The value is considered true iff it's not "0". +bool BoolFromGTestEnv(const char* flag, bool default_value) { + const std::string env_var = FlagToEnvVar(flag); + const char* const string_value = posix::GetEnv(env_var.c_str()); + return string_value == NULL ? + default_value : strcmp(string_value, "0") != 0; +} + +// Reads and returns a 32-bit integer stored in the environment +// variable corresponding to the given flag; if it isn't set or +// doesn't represent a valid 32-bit integer, returns default_value. +Int32 Int32FromGTestEnv(const char* flag, Int32 default_value) { + const std::string env_var = FlagToEnvVar(flag); + const char* const string_value = posix::GetEnv(env_var.c_str()); + if (string_value == NULL) { + // The environment variable is not set. + return default_value; + } + + Int32 result = default_value; + if (!ParseInt32(Message() << "Environment variable " << env_var, + string_value, &result)) { + printf("The default value %s is used.\n", + (Message() << default_value).GetString().c_str()); + fflush(stdout); + return default_value; + } + + return result; +} + +// Reads and returns the string environment variable corresponding to +// the given flag; if it's not set, returns default_value. +const char* StringFromGTestEnv(const char* flag, const char* default_value) { + const std::string env_var = FlagToEnvVar(flag); + const char* const value = posix::GetEnv(env_var.c_str()); + return value == NULL ? default_value : value; +} + +} // namespace internal +} // namespace testing +// Copyright 2007, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) + +// Google Test - The Google C++ Testing Framework +// +// This file implements a universal value printer that can print a +// value of any type T: +// +// void ::testing::internal::UniversalPrinter::Print(value, ostream_ptr); +// +// It uses the << operator when possible, and prints the bytes in the +// object otherwise. A user can override its behavior for a class +// type Foo by defining either operator<<(::std::ostream&, const Foo&) +// or void PrintTo(const Foo&, ::std::ostream*) in the namespace that +// defines Foo. + +#include +#include +#include +#include // NOLINT +#include + +namespace testing { + +namespace { + +using ::std::ostream; + +// Prints a segment of bytes in the given object. +GTEST_ATTRIBUTE_NO_SANITIZE_MEMORY_ +GTEST_ATTRIBUTE_NO_SANITIZE_ADDRESS_ +GTEST_ATTRIBUTE_NO_SANITIZE_THREAD_ +void PrintByteSegmentInObjectTo(const unsigned char* obj_bytes, size_t start, + size_t count, ostream* os) { + char text[5] = ""; + for (size_t i = 0; i != count; i++) { + const size_t j = start + i; + if (i != 0) { + // Organizes the bytes into groups of 2 for easy parsing by + // human. + if ((j % 2) == 0) + *os << ' '; + else + *os << '-'; + } + GTEST_SNPRINTF_(text, sizeof(text), "%02X", obj_bytes[j]); + *os << text; + } +} + +// Prints the bytes in the given value to the given ostream. +void PrintBytesInObjectToImpl(const unsigned char* obj_bytes, size_t count, + ostream* os) { + // Tells the user how big the object is. + *os << count << "-byte object <"; + + const size_t kThreshold = 132; + const size_t kChunkSize = 64; + // If the object size is bigger than kThreshold, we'll have to omit + // some details by printing only the first and the last kChunkSize + // bytes. + // TODO(wan): let the user control the threshold using a flag. + if (count < kThreshold) { + PrintByteSegmentInObjectTo(obj_bytes, 0, count, os); + } else { + PrintByteSegmentInObjectTo(obj_bytes, 0, kChunkSize, os); + *os << " ... "; + // Rounds up to 2-byte boundary. + const size_t resume_pos = (count - kChunkSize + 1)/2*2; + PrintByteSegmentInObjectTo(obj_bytes, resume_pos, count - resume_pos, os); + } + *os << ">"; +} + +} // namespace + +namespace internal2 { + +// Delegates to PrintBytesInObjectToImpl() to print the bytes in the +// given object. The delegation simplifies the implementation, which +// uses the << operator and thus is easier done outside of the +// ::testing::internal namespace, which contains a << operator that +// sometimes conflicts with the one in STL. +void PrintBytesInObjectTo(const unsigned char* obj_bytes, size_t count, + ostream* os) { + PrintBytesInObjectToImpl(obj_bytes, count, os); +} + +} // namespace internal2 + +namespace internal { + +// Depending on the value of a char (or wchar_t), we print it in one +// of three formats: +// - as is if it's a printable ASCII (e.g. 'a', '2', ' '), +// - as a hexidecimal escape sequence (e.g. '\x7F'), or +// - as a special escape sequence (e.g. '\r', '\n'). +enum CharFormat { + kAsIs, + kHexEscape, + kSpecialEscape +}; + +// Returns true if c is a printable ASCII character. We test the +// value of c directly instead of calling isprint(), which is buggy on +// Windows Mobile. +inline bool IsPrintableAscii(wchar_t c) { + return 0x20 <= c && c <= 0x7E; +} + +// Prints a wide or narrow char c as a character literal without the +// quotes, escaping it when necessary; returns how c was formatted. +// The template argument UnsignedChar is the unsigned version of Char, +// which is the type of c. +template +static CharFormat PrintAsCharLiteralTo(Char c, ostream* os) { + switch (static_cast(c)) { + case L'\0': + *os << "\\0"; + break; + case L'\'': + *os << "\\'"; + break; + case L'\\': + *os << "\\\\"; + break; + case L'\a': + *os << "\\a"; + break; + case L'\b': + *os << "\\b"; + break; + case L'\f': + *os << "\\f"; + break; + case L'\n': + *os << "\\n"; + break; + case L'\r': + *os << "\\r"; + break; + case L'\t': + *os << "\\t"; + break; + case L'\v': + *os << "\\v"; + break; + default: + if (IsPrintableAscii(c)) { + *os << static_cast(c); + return kAsIs; + } else { + *os << "\\x" + String::FormatHexInt(static_cast(c)); + return kHexEscape; + } + } + return kSpecialEscape; +} + +// Prints a wchar_t c as if it's part of a string literal, escaping it when +// necessary; returns how c was formatted. +static CharFormat PrintAsStringLiteralTo(wchar_t c, ostream* os) { + switch (c) { + case L'\'': + *os << "'"; + return kAsIs; + case L'"': + *os << "\\\""; + return kSpecialEscape; + default: + return PrintAsCharLiteralTo(c, os); + } +} + +// Prints a char c as if it's part of a string literal, escaping it when +// necessary; returns how c was formatted. +static CharFormat PrintAsStringLiteralTo(char c, ostream* os) { + return PrintAsStringLiteralTo( + static_cast(static_cast(c)), os); +} + +// Prints a wide or narrow character c and its code. '\0' is printed +// as "'\\0'", other unprintable characters are also properly escaped +// using the standard C++ escape sequence. The template argument +// UnsignedChar is the unsigned version of Char, which is the type of c. +template +void PrintCharAndCodeTo(Char c, ostream* os) { + // First, print c as a literal in the most readable form we can find. + *os << ((sizeof(c) > 1) ? "L'" : "'"); + const CharFormat format = PrintAsCharLiteralTo(c, os); + *os << "'"; + + // To aid user debugging, we also print c's code in decimal, unless + // it's 0 (in which case c was printed as '\\0', making the code + // obvious). + if (c == 0) + return; + *os << " (" << static_cast(c); + + // For more convenience, we print c's code again in hexidecimal, + // unless c was already printed in the form '\x##' or the code is in + // [1, 9]. + if (format == kHexEscape || (1 <= c && c <= 9)) { + // Do nothing. + } else { + *os << ", 0x" << String::FormatHexInt(static_cast(c)); + } + *os << ")"; +} + +void PrintTo(unsigned char c, ::std::ostream* os) { + PrintCharAndCodeTo(c, os); +} +void PrintTo(signed char c, ::std::ostream* os) { + PrintCharAndCodeTo(c, os); +} + +// Prints a wchar_t as a symbol if it is printable or as its internal +// code otherwise and also as its code. L'\0' is printed as "L'\\0'". +void PrintTo(wchar_t wc, ostream* os) { + PrintCharAndCodeTo(wc, os); +} + +// Prints the given array of characters to the ostream. CharType must be either +// char or wchar_t. +// The array starts at begin, the length is len, it may include '\0' characters +// and may not be NUL-terminated. +template +GTEST_ATTRIBUTE_NO_SANITIZE_MEMORY_ +GTEST_ATTRIBUTE_NO_SANITIZE_ADDRESS_ +GTEST_ATTRIBUTE_NO_SANITIZE_THREAD_ +static void PrintCharsAsStringTo( + const CharType* begin, size_t len, ostream* os) { + const char* const kQuoteBegin = sizeof(CharType) == 1 ? "\"" : "L\""; + *os << kQuoteBegin; + bool is_previous_hex = false; + for (size_t index = 0; index < len; ++index) { + const CharType cur = begin[index]; + if (is_previous_hex && IsXDigit(cur)) { + // Previous character is of '\x..' form and this character can be + // interpreted as another hexadecimal digit in its number. Break string to + // disambiguate. + *os << "\" " << kQuoteBegin; + } + is_previous_hex = PrintAsStringLiteralTo(cur, os) == kHexEscape; + } + *os << "\""; +} + +// Prints a (const) char/wchar_t array of 'len' elements, starting at address +// 'begin'. CharType must be either char or wchar_t. +template +GTEST_ATTRIBUTE_NO_SANITIZE_MEMORY_ +GTEST_ATTRIBUTE_NO_SANITIZE_ADDRESS_ +GTEST_ATTRIBUTE_NO_SANITIZE_THREAD_ +static void UniversalPrintCharArray( + const CharType* begin, size_t len, ostream* os) { + // The code + // const char kFoo[] = "foo"; + // generates an array of 4, not 3, elements, with the last one being '\0'. + // + // Therefore when printing a char array, we don't print the last element if + // it's '\0', such that the output matches the string literal as it's + // written in the source code. + if (len > 0 && begin[len - 1] == '\0') { + PrintCharsAsStringTo(begin, len - 1, os); + return; + } + + // If, however, the last element in the array is not '\0', e.g. + // const char kFoo[] = { 'f', 'o', 'o' }; + // we must print the entire array. We also print a message to indicate + // that the array is not NUL-terminated. + PrintCharsAsStringTo(begin, len, os); + *os << " (no terminating NUL)"; +} + +// Prints a (const) char array of 'len' elements, starting at address 'begin'. +void UniversalPrintArray(const char* begin, size_t len, ostream* os) { + UniversalPrintCharArray(begin, len, os); +} + +// Prints a (const) wchar_t array of 'len' elements, starting at address +// 'begin'. +void UniversalPrintArray(const wchar_t* begin, size_t len, ostream* os) { + UniversalPrintCharArray(begin, len, os); +} + +// Prints the given C string to the ostream. +void PrintTo(const char* s, ostream* os) { + if (s == NULL) { + *os << "NULL"; + } else { + *os << ImplicitCast_(s) << " pointing to "; + PrintCharsAsStringTo(s, strlen(s), os); + } +} + +// MSVC compiler can be configured to define whar_t as a typedef +// of unsigned short. Defining an overload for const wchar_t* in that case +// would cause pointers to unsigned shorts be printed as wide strings, +// possibly accessing more memory than intended and causing invalid +// memory accesses. MSVC defines _NATIVE_WCHAR_T_DEFINED symbol when +// wchar_t is implemented as a native type. +#if !defined(_MSC_VER) || defined(_NATIVE_WCHAR_T_DEFINED) +// Prints the given wide C string to the ostream. +void PrintTo(const wchar_t* s, ostream* os) { + if (s == NULL) { + *os << "NULL"; + } else { + *os << ImplicitCast_(s) << " pointing to "; + PrintCharsAsStringTo(s, std::wcslen(s), os); + } +} +#endif // wchar_t is native + +// Prints a ::string object. +#if GTEST_HAS_GLOBAL_STRING +void PrintStringTo(const ::string& s, ostream* os) { + PrintCharsAsStringTo(s.data(), s.size(), os); +} +#endif // GTEST_HAS_GLOBAL_STRING + +void PrintStringTo(const ::std::string& s, ostream* os) { + PrintCharsAsStringTo(s.data(), s.size(), os); +} + +// Prints a ::wstring object. +#if GTEST_HAS_GLOBAL_WSTRING +void PrintWideStringTo(const ::wstring& s, ostream* os) { + PrintCharsAsStringTo(s.data(), s.size(), os); +} +#endif // GTEST_HAS_GLOBAL_WSTRING + +#if GTEST_HAS_STD_WSTRING +void PrintWideStringTo(const ::std::wstring& s, ostream* os) { + PrintCharsAsStringTo(s.data(), s.size(), os); +} +#endif // GTEST_HAS_STD_WSTRING + +} // namespace internal + +} // namespace testing +// Copyright 2008, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: mheule@google.com (Markus Heule) +// +// The Google C++ Testing Framework (Google Test) + + +// Indicates that this translation unit is part of Google Test's +// implementation. It must come before gtest-internal-inl.h is +// included, or there will be a compiler error. This trick exists to +// prevent the accidental inclusion of gtest-internal-inl.h in the +// user's code. +#define GTEST_IMPLEMENTATION_ 1 +#undef GTEST_IMPLEMENTATION_ + +namespace testing { + +using internal::GetUnitTestImpl; + +// Gets the summary of the failure message by omitting the stack trace +// in it. +std::string TestPartResult::ExtractSummary(const char* message) { + const char* const stack_trace = strstr(message, internal::kStackTraceMarker); + return stack_trace == NULL ? message : + std::string(message, stack_trace); +} + +// Prints a TestPartResult object. +std::ostream& operator<<(std::ostream& os, const TestPartResult& result) { + return os + << result.file_name() << ":" << result.line_number() << ": " + << (result.type() == TestPartResult::kSuccess ? "Success" : + result.type() == TestPartResult::kFatalFailure ? "Fatal failure" : + "Non-fatal failure") << ":\n" + << result.message() << std::endl; +} + +// Appends a TestPartResult to the array. +void TestPartResultArray::Append(const TestPartResult& result) { + array_.push_back(result); +} + +// Returns the TestPartResult at the given index (0-based). +const TestPartResult& TestPartResultArray::GetTestPartResult(int index) const { + if (index < 0 || index >= size()) { + printf("\nInvalid index (%d) into TestPartResultArray.\n", index); + internal::posix::Abort(); + } + + return array_[index]; +} + +// Returns the number of TestPartResult objects in the array. +int TestPartResultArray::size() const { + return static_cast(array_.size()); +} + +namespace internal { + +HasNewFatalFailureHelper::HasNewFatalFailureHelper() + : has_new_fatal_failure_(false), + original_reporter_(GetUnitTestImpl()-> + GetTestPartResultReporterForCurrentThread()) { + GetUnitTestImpl()->SetTestPartResultReporterForCurrentThread(this); +} + +HasNewFatalFailureHelper::~HasNewFatalFailureHelper() { + GetUnitTestImpl()->SetTestPartResultReporterForCurrentThread( + original_reporter_); +} + +void HasNewFatalFailureHelper::ReportTestPartResult( + const TestPartResult& result) { + if (result.fatally_failed()) + has_new_fatal_failure_ = true; + original_reporter_->ReportTestPartResult(result); +} + +} // namespace internal + +} // namespace testing +// Copyright 2008 Google Inc. +// All Rights Reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) + + +namespace testing { +namespace internal { + +#if GTEST_HAS_TYPED_TEST_P + +// Skips to the first non-space char in str. Returns an empty string if str +// contains only whitespace characters. +static const char* SkipSpaces(const char* str) { + while (IsSpace(*str)) + str++; + return str; +} + +static std::vector SplitIntoTestNames(const char* src) { + std::vector name_vec; + src = SkipSpaces(src); + for (; src != NULL; src = SkipComma(src)) { + name_vec.push_back(StripTrailingSpaces(GetPrefixUntilComma(src))); + } + return name_vec; +} + +// Verifies that registered_tests match the test names in +// defined_test_names_; returns registered_tests if successful, or +// aborts the program otherwise. +const char* TypedTestCasePState::VerifyRegisteredTestNames( + const char* file, int line, const char* registered_tests) { + typedef ::std::set::const_iterator DefinedTestIter; + registered_ = true; + + std::vector name_vec = SplitIntoTestNames(registered_tests); + + Message errors; + + std::set tests; + for (std::vector::const_iterator name_it = name_vec.begin(); + name_it != name_vec.end(); ++name_it) { + const std::string& name = *name_it; + if (tests.count(name) != 0) { + errors << "Test " << name << " is listed more than once.\n"; + continue; + } + + bool found = false; + for (DefinedTestIter it = defined_test_names_.begin(); + it != defined_test_names_.end(); + ++it) { + if (name == *it) { + found = true; + break; + } + } + + if (found) { + tests.insert(name); + } else { + errors << "No test named " << name + << " can be found in this test case.\n"; + } + } + + for (DefinedTestIter it = defined_test_names_.begin(); + it != defined_test_names_.end(); + ++it) { + if (tests.count(*it) == 0) { + errors << "You forgot to list test " << *it << ".\n"; + } + } + + const std::string& errors_str = errors.GetString(); + if (errors_str != "") { + fprintf(stderr, "%s %s", FormatFileLocation(file, line).c_str(), + errors_str.c_str()); + fflush(stderr); + posix::Abort(); + } + + return registered_tests; +} + +#endif // GTEST_HAS_TYPED_TEST_P + +} // namespace internal +} // namespace testing + +#endif // __clang_analyzer__ diff --git a/third-party/gtest-1.7.0/fused-src/gtest/gtest.h b/third-party/gtest-1.7.0/fused-src/gtest/gtest.h new file mode 100644 index 00000000000..e3f0cfb95cb --- /dev/null +++ b/third-party/gtest-1.7.0/fused-src/gtest/gtest.h @@ -0,0 +1,20725 @@ +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) +// +// The Google C++ Testing Framework (Google Test) +// +// This header file defines the public API for Google Test. It should be +// included by any test program that uses Google Test. +// +// IMPORTANT NOTE: Due to limitation of the C++ language, we have to +// leave some internal implementation details in this header file. +// They are clearly marked by comments like this: +// +// // INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +// +// Such code is NOT meant to be used by a user directly, and is subject +// to CHANGE WITHOUT NOTICE. Therefore DO NOT DEPEND ON IT in a user +// program! +// +// Acknowledgment: Google Test borrowed the idea of automatic test +// registration from Barthelemy Dagenais' (barthelemy@prologique.com) +// easyUnit framework. + +#ifndef GTEST_INCLUDE_GTEST_GTEST_H_ +#define GTEST_INCLUDE_GTEST_GTEST_H_ + +#include +#include +#include + +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Authors: wan@google.com (Zhanyong Wan), eefacm@gmail.com (Sean Mcafee) +// +// The Google C++ Testing Framework (Google Test) +// +// This header file declares functions and macros used internally by +// Google Test. They are subject to change without notice. + +#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_INTERNAL_H_ +#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_INTERNAL_H_ + +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Authors: wan@google.com (Zhanyong Wan) +// +// Low-level types and utilities for porting Google Test to various +// platforms. All macros ending with _ and symbols defined in an +// internal namespace are subject to change without notice. Code +// outside Google Test MUST NOT USE THEM DIRECTLY. Macros that don't +// end with _ are part of Google Test's public API and can be used by +// code outside Google Test. +// +// This file is fundamental to Google Test. All other Google Test source +// files are expected to #include this. Therefore, it cannot #include +// any other Google Test header. + +#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PORT_H_ +#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PORT_H_ + +// Environment-describing macros +// ----------------------------- +// +// Google Test can be used in many different environments. Macros in +// this section tell Google Test what kind of environment it is being +// used in, such that Google Test can provide environment-specific +// features and implementations. +// +// Google Test tries to automatically detect the properties of its +// environment, so users usually don't need to worry about these +// macros. However, the automatic detection is not perfect. +// Sometimes it's necessary for a user to define some of the following +// macros in the build script to override Google Test's decisions. +// +// If the user doesn't define a macro in the list, Google Test will +// provide a default definition. After this header is #included, all +// macros in this list will be defined to either 1 or 0. +// +// Notes to maintainers: +// - Each macro here is a user-tweakable knob; do not grow the list +// lightly. +// - Use #if to key off these macros. Don't use #ifdef or "#if +// defined(...)", which will not work as these macros are ALWAYS +// defined. +// +// GTEST_HAS_CLONE - Define it to 1/0 to indicate that clone(2) +// is/isn't available. +// GTEST_HAS_EXCEPTIONS - Define it to 1/0 to indicate that exceptions +// are enabled. +// GTEST_HAS_GLOBAL_STRING - Define it to 1/0 to indicate that ::string +// is/isn't available (some systems define +// ::string, which is different to std::string). +// GTEST_HAS_GLOBAL_WSTRING - Define it to 1/0 to indicate that ::string +// is/isn't available (some systems define +// ::wstring, which is different to std::wstring). +// GTEST_HAS_POSIX_RE - Define it to 1/0 to indicate that POSIX regular +// expressions are/aren't available. +// GTEST_HAS_PTHREAD - Define it to 1/0 to indicate that +// is/isn't available. +// GTEST_HAS_RTTI - Define it to 1/0 to indicate that RTTI is/isn't +// enabled. +// GTEST_HAS_STD_WSTRING - Define it to 1/0 to indicate that +// std::wstring does/doesn't work (Google Test can +// be used where std::wstring is unavailable). +// GTEST_HAS_TR1_TUPLE - Define it to 1/0 to indicate tr1::tuple +// is/isn't available. +// GTEST_HAS_SEH - Define it to 1/0 to indicate whether the +// compiler supports Microsoft's "Structured +// Exception Handling". +// GTEST_HAS_STREAM_REDIRECTION +// - Define it to 1/0 to indicate whether the +// platform supports I/O stream redirection using +// dup() and dup2(). +// GTEST_USE_OWN_TR1_TUPLE - Define it to 1/0 to indicate whether Google +// Test's own tr1 tuple implementation should be +// used. Unused when the user sets +// GTEST_HAS_TR1_TUPLE to 0. +// GTEST_LANG_CXX11 - Define it to 1/0 to indicate that Google Test +// is building in C++11/C++98 mode. +// GTEST_LINKED_AS_SHARED_LIBRARY +// - Define to 1 when compiling tests that use +// Google Test as a shared library (known as +// DLL on Windows). +// GTEST_CREATE_SHARED_LIBRARY +// - Define to 1 when compiling Google Test itself +// as a shared library. + +// Platform-indicating macros +// -------------------------- +// +// Macros indicating the platform on which Google Test is being used +// (a macro is defined to 1 if compiled on the given platform; +// otherwise UNDEFINED -- it's never defined to 0.). Google Test +// defines these macros automatically. Code outside Google Test MUST +// NOT define them. +// +// GTEST_OS_AIX - IBM AIX +// GTEST_OS_CYGWIN - Cygwin +// GTEST_OS_FREEBSD - FreeBSD +// GTEST_OS_HPUX - HP-UX +// GTEST_OS_LINUX - Linux +// GTEST_OS_LINUX_ANDROID - Google Android +// GTEST_OS_MAC - Mac OS X +// GTEST_OS_IOS - iOS +// GTEST_OS_NACL - Google Native Client (NaCl) +// GTEST_OS_OPENBSD - OpenBSD +// GTEST_OS_QNX - QNX +// GTEST_OS_SOLARIS - Sun Solaris +// GTEST_OS_SYMBIAN - Symbian +// GTEST_OS_WINDOWS - Windows (Desktop, MinGW, or Mobile) +// GTEST_OS_WINDOWS_DESKTOP - Windows Desktop +// GTEST_OS_WINDOWS_MINGW - MinGW +// GTEST_OS_WINDOWS_MOBILE - Windows Mobile +// GTEST_OS_WINDOWS_PHONE - Windows Phone +// GTEST_OS_WINDOWS_RT - Windows Store App/WinRT +// GTEST_OS_ZOS - z/OS +// +// Among the platforms, Cygwin, Linux, Max OS X, and Windows have the +// most stable support. Since core members of the Google Test project +// don't have access to other platforms, support for them may be less +// stable. If you notice any problems on your platform, please notify +// googletestframework@googlegroups.com (patches for fixing them are +// even more welcome!). +// +// It is possible that none of the GTEST_OS_* macros are defined. + +// Feature-indicating macros +// ------------------------- +// +// Macros indicating which Google Test features are available (a macro +// is defined to 1 if the corresponding feature is supported; +// otherwise UNDEFINED -- it's never defined to 0.). Google Test +// defines these macros automatically. Code outside Google Test MUST +// NOT define them. +// +// These macros are public so that portable tests can be written. +// Such tests typically surround code using a feature with an #if +// which controls that code. For example: +// +// #if GTEST_HAS_DEATH_TEST +// EXPECT_DEATH(DoSomethingDeadly()); +// #endif +// +// GTEST_HAS_COMBINE - the Combine() function (for value-parameterized +// tests) +// GTEST_HAS_DEATH_TEST - death tests +// GTEST_HAS_PARAM_TEST - value-parameterized tests +// GTEST_HAS_TYPED_TEST - typed tests +// GTEST_HAS_TYPED_TEST_P - type-parameterized tests +// GTEST_IS_THREADSAFE - Google Test is thread-safe. +// GTEST_USES_POSIX_RE - enhanced POSIX regex is used. Do not confuse with +// GTEST_HAS_POSIX_RE (see above) which users can +// define themselves. +// GTEST_USES_SIMPLE_RE - our own simple regex is used; +// the above two are mutually exclusive. +// GTEST_CAN_COMPARE_NULL - accepts untyped NULL in EXPECT_EQ(). + +// Misc public macros +// ------------------ +// +// GTEST_FLAG(flag_name) - references the variable corresponding to +// the given Google Test flag. + +// Internal utilities +// ------------------ +// +// The following macros and utilities are for Google Test's INTERNAL +// use only. Code outside Google Test MUST NOT USE THEM DIRECTLY. +// +// Macros for basic C++ coding: +// GTEST_AMBIGUOUS_ELSE_BLOCKER_ - for disabling a gcc warning. +// GTEST_ATTRIBUTE_UNUSED_ - declares that a class' instances or a +// variable don't have to be used. +// GTEST_DISALLOW_ASSIGN_ - disables operator=. +// GTEST_DISALLOW_COPY_AND_ASSIGN_ - disables copy ctor and operator=. +// GTEST_MUST_USE_RESULT_ - declares that a function's result must be used. +// GTEST_INTENTIONAL_CONST_COND_PUSH_ - start code section where MSVC C4127 is +// suppressed (constant conditional). +// GTEST_INTENTIONAL_CONST_COND_POP_ - finish code section where MSVC C4127 +// is suppressed. +// +// C++11 feature wrappers: +// +// testing::internal::move - portability wrapper for std::move. +// +// Synchronization: +// Mutex, MutexLock, ThreadLocal, GetThreadCount() +// - synchronization primitives. +// +// Template meta programming: +// is_pointer - as in TR1; needed on Symbian and IBM XL C/C++ only. +// IteratorTraits - partial implementation of std::iterator_traits, which +// is not available in libCstd when compiled with Sun C++. +// +// Smart pointers: +// scoped_ptr - as in TR2. +// +// Regular expressions: +// RE - a simple regular expression class using the POSIX +// Extended Regular Expression syntax on UNIX-like +// platforms, or a reduced regular exception syntax on +// other platforms, including Windows. +// +// Logging: +// GTEST_LOG_() - logs messages at the specified severity level. +// LogToStderr() - directs all log messages to stderr. +// FlushInfoLog() - flushes informational log messages. +// +// Stdout and stderr capturing: +// CaptureStdout() - starts capturing stdout. +// GetCapturedStdout() - stops capturing stdout and returns the captured +// string. +// CaptureStderr() - starts capturing stderr. +// GetCapturedStderr() - stops capturing stderr and returns the captured +// string. +// +// Integer types: +// TypeWithSize - maps an integer to a int type. +// Int32, UInt32, Int64, UInt64, TimeInMillis +// - integers of known sizes. +// BiggestInt - the biggest signed integer type. +// +// Command-line utilities: +// GTEST_DECLARE_*() - declares a flag. +// GTEST_DEFINE_*() - defines a flag. +// GetInjectableArgvs() - returns the command line as a vector of strings. +// +// Environment variable utilities: +// GetEnv() - gets the value of an environment variable. +// BoolFromGTestEnv() - parses a bool environment variable. +// Int32FromGTestEnv() - parses an Int32 environment variable. +// StringFromGTestEnv() - parses a string environment variable. + +#include // for isspace, etc +#include // for ptrdiff_t +#include +#include +#include +#ifndef _WIN32_WCE +# include +# include +#endif // !_WIN32_WCE + +#if defined __APPLE__ +# include +# include +#endif + +#include // NOLINT +#include // NOLINT +#include // NOLINT +#include // NOLINT +#include + +#define GTEST_DEV_EMAIL_ "googletestframework@@googlegroups.com" +#define GTEST_FLAG_PREFIX_ "gtest_" +#define GTEST_FLAG_PREFIX_DASH_ "gtest-" +#define GTEST_FLAG_PREFIX_UPPER_ "GTEST_" +#define GTEST_NAME_ "Google Test" +#define GTEST_PROJECT_URL_ "http://code.google.com/p/googletest/" + +// Determines the version of gcc that is used to compile this. +#ifdef __GNUC__ +// 40302 means version 4.3.2. +# define GTEST_GCC_VER_ \ + (__GNUC__*10000 + __GNUC_MINOR__*100 + __GNUC_PATCHLEVEL__) +#endif // __GNUC__ + +// Determines the platform on which Google Test is compiled. +#ifdef __CYGWIN__ +# define GTEST_OS_CYGWIN 1 +#elif defined __SYMBIAN32__ +# define GTEST_OS_SYMBIAN 1 +#elif defined _WIN32 +# define GTEST_OS_WINDOWS 1 +# ifdef _WIN32_WCE +# define GTEST_OS_WINDOWS_MOBILE 1 +# elif defined(__MINGW__) || defined(__MINGW32__) +# define GTEST_OS_WINDOWS_MINGW 1 +# elif defined(WINAPI_FAMILY) +# include +# if WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP) +# define GTEST_OS_WINDOWS_DESKTOP 1 +# elif WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_PHONE_APP) +# define GTEST_OS_WINDOWS_PHONE 1 +# elif WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_APP) +# define GTEST_OS_WINDOWS_RT 1 +# else + // WINAPI_FAMILY defined but no known partition matched. + // Default to desktop. +# define GTEST_OS_WINDOWS_DESKTOP 1 +# endif +# else +# define GTEST_OS_WINDOWS_DESKTOP 1 +# endif // _WIN32_WCE +#elif defined __APPLE__ +# define GTEST_OS_MAC 1 +# if TARGET_OS_IPHONE +# define GTEST_OS_IOS 1 +# endif +#elif defined __FreeBSD__ +# define GTEST_OS_FREEBSD 1 +#elif defined __linux__ +# define GTEST_OS_LINUX 1 +# if defined __ANDROID__ +# define GTEST_OS_LINUX_ANDROID 1 +# endif +#elif defined __MVS__ +# define GTEST_OS_ZOS 1 +#elif defined(__sun) && defined(__SVR4) +# define GTEST_OS_SOLARIS 1 +#elif defined(_AIX) +# define GTEST_OS_AIX 1 +#elif defined(__hpux) +# define GTEST_OS_HPUX 1 +#elif defined __native_client__ +# define GTEST_OS_NACL 1 +#elif defined __OpenBSD__ +# define GTEST_OS_OPENBSD 1 +#elif defined __QNX__ +# define GTEST_OS_QNX 1 +#endif // __CYGWIN__ + +// Macros for disabling Microsoft Visual C++ warnings. +// +// GTEST_DISABLE_MSC_WARNINGS_PUSH_(4800 4385) +// /* code that triggers warnings C4800 and C4385 */ +// GTEST_DISABLE_MSC_WARNINGS_POP_() +#if _MSC_VER >= 1500 +# define GTEST_DISABLE_MSC_WARNINGS_PUSH_(warnings) \ + __pragma(warning(push)) \ + __pragma(warning(disable: warnings)) +# define GTEST_DISABLE_MSC_WARNINGS_POP_() \ + __pragma(warning(pop)) +#else +// Older versions of MSVC don't have __pragma. +# define GTEST_DISABLE_MSC_WARNINGS_PUSH_(warnings) +# define GTEST_DISABLE_MSC_WARNINGS_POP_() +#endif + +#ifndef GTEST_LANG_CXX11 +// gcc and clang define __GXX_EXPERIMENTAL_CXX0X__ when +// -std={c,gnu}++{0x,11} is passed. The C++11 standard specifies a +// value for __cplusplus, and recent versions of clang, gcc, and +// probably other compilers set that too in C++11 mode. +# if __GXX_EXPERIMENTAL_CXX0X__ || __cplusplus >= 201103L +// Compiling in at least C++11 mode. +# define GTEST_LANG_CXX11 1 +# else +# define GTEST_LANG_CXX11 0 +# endif +#endif + +// Distinct from C++11 language support, some environments don't provide +// proper C++11 library support. Notably, it's possible to build in +// C++11 mode when targeting Mac OS X 10.6, which has an old libstdc++ +// with no C++11 support. +// +// libstdc++ has sufficient C++11 support as of GCC 4.6.0, __GLIBCXX__ +// 20110325, but maintenance releases in the 4.4 and 4.5 series followed +// this date, so check for those versions by their date stamps. +// https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html#abi.versioning +#if GTEST_LANG_CXX11 && \ + (!defined(__GLIBCXX__) || ( \ + __GLIBCXX__ >= 20110325ul && /* GCC >= 4.6.0 */ \ + /* Blacklist of patch releases of older branches: */ \ + __GLIBCXX__ != 20110416ul && /* GCC 4.4.6 */ \ + __GLIBCXX__ != 20120313ul && /* GCC 4.4.7 */ \ + __GLIBCXX__ != 20110428ul && /* GCC 4.5.3 */ \ + __GLIBCXX__ != 20120702ul)) /* GCC 4.5.4 */ +# define GTEST_STDLIB_CXX11 1 +#endif + +// Only use C++11 library features if the library provides them. +#if GTEST_STDLIB_CXX11 +# define GTEST_HAS_STD_BEGIN_AND_END_ 1 +# define GTEST_HAS_STD_FORWARD_LIST_ 1 +# define GTEST_HAS_STD_FUNCTION_ 1 +# define GTEST_HAS_STD_INITIALIZER_LIST_ 1 +# define GTEST_HAS_STD_MOVE_ 1 +# define GTEST_HAS_STD_UNIQUE_PTR_ 1 +#endif + +// C++11 specifies that provides std::tuple. +// Some platforms still might not have it, however. +#if GTEST_LANG_CXX11 +# define GTEST_HAS_STD_TUPLE_ 1 +# if defined(__clang__) +// Inspired by http://clang.llvm.org/docs/LanguageExtensions.html#__has_include +# if defined(__has_include) && !__has_include() +# undef GTEST_HAS_STD_TUPLE_ +# endif +# elif defined(_MSC_VER) +// Inspired by boost/config/stdlib/dinkumware.hpp +# if defined(_CPPLIB_VER) && _CPPLIB_VER < 520 +# undef GTEST_HAS_STD_TUPLE_ +# endif +# elif defined(__GLIBCXX__) +// Inspired by boost/config/stdlib/libstdcpp3.hpp, +// http://gcc.gnu.org/gcc-4.2/changes.html and +// http://gcc.gnu.org/onlinedocs/libstdc++/manual/bk01pt01ch01.html#manual.intro.status.standard.200x +# if __GNUC__ < 4 || (__GNUC__ == 4 && __GNUC_MINOR__ < 2) +# undef GTEST_HAS_STD_TUPLE_ +# endif +# endif +#endif + +// Brings in definitions for functions used in the testing::internal::posix +// namespace (read, write, close, chdir, isatty, stat). We do not currently +// use them on Windows Mobile. +#if GTEST_OS_WINDOWS +# if !GTEST_OS_WINDOWS_MOBILE +# include +# include +# endif +// In order to avoid having to include , use forward declaration +// assuming CRITICAL_SECTION is a typedef of _RTL_CRITICAL_SECTION. +// This assumption is verified by +// WindowsTypesTest.CRITICAL_SECTIONIs_RTL_CRITICAL_SECTION. +struct _RTL_CRITICAL_SECTION; +#else +// This assumes that non-Windows OSes provide unistd.h. For OSes where this +// is not the case, we need to include headers that provide the functions +// mentioned above. +# include +# include +#endif // GTEST_OS_WINDOWS + +#if GTEST_OS_LINUX_ANDROID +// Used to define __ANDROID_API__ matching the target NDK API level. +# include // NOLINT +#endif + +// Defines this to true iff Google Test can use POSIX regular expressions. +#ifndef GTEST_HAS_POSIX_RE +# if GTEST_OS_LINUX_ANDROID +// On Android, is only available starting with Gingerbread. +# define GTEST_HAS_POSIX_RE (__ANDROID_API__ >= 9) +# else +# define GTEST_HAS_POSIX_RE (!GTEST_OS_WINDOWS) +# endif +#endif + +#if GTEST_HAS_POSIX_RE + +// On some platforms, needs someone to define size_t, and +// won't compile otherwise. We can #include it here as we already +// included , which is guaranteed to define size_t through +// . +# include // NOLINT + +# define GTEST_USES_POSIX_RE 1 + +#elif GTEST_OS_WINDOWS + +// is not available on Windows. Use our own simple regex +// implementation instead. +# define GTEST_USES_SIMPLE_RE 1 + +#else + +// may not be available on this platform. Use our own +// simple regex implementation instead. +# define GTEST_USES_SIMPLE_RE 1 + +#endif // GTEST_HAS_POSIX_RE + +#ifndef GTEST_HAS_EXCEPTIONS +// The user didn't tell us whether exceptions are enabled, so we need +// to figure it out. +# if defined(_MSC_VER) || defined(__BORLANDC__) +// MSVC's and C++Builder's implementations of the STL use the _HAS_EXCEPTIONS +// macro to enable exceptions, so we'll do the same. +// Assumes that exceptions are enabled by default. +# ifndef _HAS_EXCEPTIONS +# define _HAS_EXCEPTIONS 1 +# endif // _HAS_EXCEPTIONS +# define GTEST_HAS_EXCEPTIONS _HAS_EXCEPTIONS +# elif defined(__clang__) +// clang defines __EXCEPTIONS iff exceptions are enabled before clang 220714, +// but iff cleanups are enabled after that. In Obj-C++ files, there can be +// cleanups for ObjC exceptions which also need cleanups, even if C++ exceptions +// are disabled. clang has __has_feature(cxx_exceptions) which checks for C++ +// exceptions starting at clang r206352, but which checked for cleanups prior to +// that. To reliably check for C++ exception availability with clang, check for +// __EXCEPTIONS && __has_feature(cxx_exceptions). +# define GTEST_HAS_EXCEPTIONS (__EXCEPTIONS && __has_feature(cxx_exceptions)) +# elif defined(__GNUC__) && __EXCEPTIONS +// gcc defines __EXCEPTIONS to 1 iff exceptions are enabled. +# define GTEST_HAS_EXCEPTIONS 1 +# elif defined(__SUNPRO_CC) +// Sun Pro CC supports exceptions. However, there is no compile-time way of +// detecting whether they are enabled or not. Therefore, we assume that +// they are enabled unless the user tells us otherwise. +# define GTEST_HAS_EXCEPTIONS 1 +# elif defined(__IBMCPP__) && __EXCEPTIONS +// xlC defines __EXCEPTIONS to 1 iff exceptions are enabled. +# define GTEST_HAS_EXCEPTIONS 1 +# elif defined(__HP_aCC) +// Exception handling is in effect by default in HP aCC compiler. It has to +// be turned of by +noeh compiler option if desired. +# define GTEST_HAS_EXCEPTIONS 1 +# else +// For other compilers, we assume exceptions are disabled to be +// conservative. +# define GTEST_HAS_EXCEPTIONS 0 +# endif // defined(_MSC_VER) || defined(__BORLANDC__) +#endif // GTEST_HAS_EXCEPTIONS + +#if !defined(GTEST_HAS_STD_STRING) +// Even though we don't use this macro any longer, we keep it in case +// some clients still depend on it. +# define GTEST_HAS_STD_STRING 1 +#elif !GTEST_HAS_STD_STRING +// The user told us that ::std::string isn't available. +# error "Google Test cannot be used where ::std::string isn't available." +#endif // !defined(GTEST_HAS_STD_STRING) + +#ifndef GTEST_HAS_GLOBAL_STRING +// The user didn't tell us whether ::string is available, so we need +// to figure it out. + +# define GTEST_HAS_GLOBAL_STRING 0 + +#endif // GTEST_HAS_GLOBAL_STRING + +#ifndef GTEST_HAS_STD_WSTRING +// The user didn't tell us whether ::std::wstring is available, so we need +// to figure it out. +// TODO(wan@google.com): uses autoconf to detect whether ::std::wstring +// is available. + +// Cygwin 1.7 and below doesn't support ::std::wstring. +// Solaris' libc++ doesn't support it either. Android has +// no support for it at least as recent as Froyo (2.2). +# define GTEST_HAS_STD_WSTRING \ + (!(GTEST_OS_LINUX_ANDROID || GTEST_OS_CYGWIN || GTEST_OS_SOLARIS)) + +#endif // GTEST_HAS_STD_WSTRING + +#ifndef GTEST_HAS_GLOBAL_WSTRING +// The user didn't tell us whether ::wstring is available, so we need +// to figure it out. +# define GTEST_HAS_GLOBAL_WSTRING \ + (GTEST_HAS_STD_WSTRING && GTEST_HAS_GLOBAL_STRING) +#endif // GTEST_HAS_GLOBAL_WSTRING + +// Determines whether RTTI is available. +#ifndef GTEST_HAS_RTTI +// The user didn't tell us whether RTTI is enabled, so we need to +// figure it out. + +# ifdef _MSC_VER + +# ifdef _CPPRTTI // MSVC defines this macro iff RTTI is enabled. +# define GTEST_HAS_RTTI 1 +# else +# define GTEST_HAS_RTTI 0 +# endif + +// Starting with version 4.3.2, gcc defines __GXX_RTTI iff RTTI is enabled. +# elif defined(__GNUC__) && (GTEST_GCC_VER_ >= 40302) + +# ifdef __GXX_RTTI +// When building against STLport with the Android NDK and with +// -frtti -fno-exceptions, the build fails at link time with undefined +// references to __cxa_bad_typeid. Note sure if STL or toolchain bug, +// so disable RTTI when detected. +# if GTEST_OS_LINUX_ANDROID && defined(_STLPORT_MAJOR) && \ + !defined(__EXCEPTIONS) +# define GTEST_HAS_RTTI 0 +# else +# define GTEST_HAS_RTTI 1 +# endif // GTEST_OS_LINUX_ANDROID && __STLPORT_MAJOR && !__EXCEPTIONS +# else +# define GTEST_HAS_RTTI 0 +# endif // __GXX_RTTI + +// Clang defines __GXX_RTTI starting with version 3.0, but its manual recommends +// using has_feature instead. has_feature(cxx_rtti) is supported since 2.7, the +// first version with C++ support. +# elif defined(__clang__) + +# define GTEST_HAS_RTTI __has_feature(cxx_rtti) + +// Starting with version 9.0 IBM Visual Age defines __RTTI_ALL__ to 1 if +// both the typeid and dynamic_cast features are present. +# elif defined(__IBMCPP__) && (__IBMCPP__ >= 900) + +# ifdef __RTTI_ALL__ +# define GTEST_HAS_RTTI 1 +# else +# define GTEST_HAS_RTTI 0 +# endif + +# else + +// For all other compilers, we assume RTTI is enabled. +# define GTEST_HAS_RTTI 1 + +# endif // _MSC_VER + +#endif // GTEST_HAS_RTTI + +// It's this header's responsibility to #include when RTTI +// is enabled. +#if GTEST_HAS_RTTI +# include +#endif + +// Determines whether Google Test can use the pthreads library. +#ifndef GTEST_HAS_PTHREAD +// The user didn't tell us explicitly, so we make reasonable assumptions about +// which platforms have pthreads support. +// +// To disable threading support in Google Test, add -DGTEST_HAS_PTHREAD=0 +// to your compiler flags. +# define GTEST_HAS_PTHREAD (GTEST_OS_LINUX || GTEST_OS_MAC || GTEST_OS_HPUX \ + || GTEST_OS_QNX || GTEST_OS_FREEBSD || GTEST_OS_NACL) +#endif // GTEST_HAS_PTHREAD + +#if GTEST_HAS_PTHREAD +// gtest-port.h guarantees to #include when GTEST_HAS_PTHREAD is +// true. +# include // NOLINT + +// For timespec and nanosleep, used below. +# include // NOLINT +#endif + +// Determines whether Google Test can use tr1/tuple. You can define +// this macro to 0 to prevent Google Test from using tuple (any +// feature depending on tuple with be disabled in this mode). +#ifndef GTEST_HAS_TR1_TUPLE +# if GTEST_OS_LINUX_ANDROID && defined(_STLPORT_MAJOR) +// STLport, provided with the Android NDK, has neither or . +# define GTEST_HAS_TR1_TUPLE 0 +# else +// The user didn't tell us not to do it, so we assume it's OK. +# define GTEST_HAS_TR1_TUPLE 1 +# endif +#endif // GTEST_HAS_TR1_TUPLE + +// Determines whether Google Test's own tr1 tuple implementation +// should be used. +#ifndef GTEST_USE_OWN_TR1_TUPLE +// The user didn't tell us, so we need to figure it out. + +// We use our own TR1 tuple if we aren't sure the user has an +// implementation of it already. At this time, libstdc++ 4.0.0+ and +// MSVC 2010 are the only mainstream standard libraries that come +// with a TR1 tuple implementation. NVIDIA's CUDA NVCC compiler +// pretends to be GCC by defining __GNUC__ and friends, but cannot +// compile GCC's tuple implementation. MSVC 2008 (9.0) provides TR1 +// tuple in a 323 MB Feature Pack download, which we cannot assume the +// user has. QNX's QCC compiler is a modified GCC but it doesn't +// support TR1 tuple. libc++ only provides std::tuple, in C++11 mode, +// and it can be used with some compilers that define __GNUC__. +# if (defined(__GNUC__) && !defined(__CUDACC__) && (GTEST_GCC_VER_ >= 40000) \ + && !GTEST_OS_QNX && !defined(_LIBCPP_VERSION)) || _MSC_VER >= 1600 +# define GTEST_ENV_HAS_TR1_TUPLE_ 1 +# endif + +// C++11 specifies that provides std::tuple. Use that if gtest is used +// in C++11 mode and libstdc++ isn't very old (binaries targeting OS X 10.6 +// can build with clang but need to use gcc4.2's libstdc++). +# if GTEST_LANG_CXX11 && (!defined(__GLIBCXX__) || __GLIBCXX__ > 20110325) +# define GTEST_ENV_HAS_STD_TUPLE_ 1 +# endif + +# if GTEST_ENV_HAS_TR1_TUPLE_ || GTEST_ENV_HAS_STD_TUPLE_ +# define GTEST_USE_OWN_TR1_TUPLE 0 +# else +# define GTEST_USE_OWN_TR1_TUPLE 1 +# endif + +#endif // GTEST_USE_OWN_TR1_TUPLE + +// To avoid conditional compilation everywhere, we make it +// gtest-port.h's responsibility to #include the header implementing +// tuple. +#if GTEST_HAS_STD_TUPLE_ +# include // IWYU pragma: export +# define GTEST_TUPLE_NAMESPACE_ ::std +#endif // GTEST_HAS_STD_TUPLE_ + +// We include tr1::tuple even if std::tuple is available to define printers for +// them. +#if GTEST_HAS_TR1_TUPLE +# ifndef GTEST_TUPLE_NAMESPACE_ +# define GTEST_TUPLE_NAMESPACE_ ::std::tr1 +# endif // GTEST_TUPLE_NAMESPACE_ + +# if GTEST_USE_OWN_TR1_TUPLE +// This file was GENERATED by command: +// pump.py gtest-tuple.h.pump +// DO NOT EDIT BY HAND!!! + +// Copyright 2009 Google Inc. +// All Rights Reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) + +// Implements a subset of TR1 tuple needed by Google Test and Google Mock. + +#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_TUPLE_H_ +#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_TUPLE_H_ + +#include // For ::std::pair. + +// The compiler used in Symbian has a bug that prevents us from declaring the +// tuple template as a friend (it complains that tuple is redefined). This +// hack bypasses the bug by declaring the members that should otherwise be +// private as public. +// Sun Studio versions < 12 also have the above bug. +#if defined(__SYMBIAN32__) || (defined(__SUNPRO_CC) && __SUNPRO_CC < 0x590) +# define GTEST_DECLARE_TUPLE_AS_FRIEND_ public: +#else +# define GTEST_DECLARE_TUPLE_AS_FRIEND_ \ + template friend class tuple; \ + private: +#endif + +// Visual Studio 2010, 2012, and 2013 define symbols in std::tr1 that conflict +// with our own definitions. Therefore using our own tuple does not work on +// those compilers. +#if defined(_MSC_VER) && _MSC_VER >= 1600 /* 1600 is Visual Studio 2010 */ +# error "gtest's tuple doesn't compile on Visual Studio 2010 or later. \ +GTEST_USE_OWN_TR1_TUPLE must be set to 0 on those compilers." +#endif + +// GTEST_n_TUPLE_(T) is the type of an n-tuple. +#define GTEST_0_TUPLE_(T) tuple<> +#define GTEST_1_TUPLE_(T) tuple +#define GTEST_2_TUPLE_(T) tuple +#define GTEST_3_TUPLE_(T) tuple +#define GTEST_4_TUPLE_(T) tuple +#define GTEST_5_TUPLE_(T) tuple +#define GTEST_6_TUPLE_(T) tuple +#define GTEST_7_TUPLE_(T) tuple +#define GTEST_8_TUPLE_(T) tuple +#define GTEST_9_TUPLE_(T) tuple +#define GTEST_10_TUPLE_(T) tuple + +// GTEST_n_TYPENAMES_(T) declares a list of n typenames. +#define GTEST_0_TYPENAMES_(T) +#define GTEST_1_TYPENAMES_(T) typename T##0 +#define GTEST_2_TYPENAMES_(T) typename T##0, typename T##1 +#define GTEST_3_TYPENAMES_(T) typename T##0, typename T##1, typename T##2 +#define GTEST_4_TYPENAMES_(T) typename T##0, typename T##1, typename T##2, \ + typename T##3 +#define GTEST_5_TYPENAMES_(T) typename T##0, typename T##1, typename T##2, \ + typename T##3, typename T##4 +#define GTEST_6_TYPENAMES_(T) typename T##0, typename T##1, typename T##2, \ + typename T##3, typename T##4, typename T##5 +#define GTEST_7_TYPENAMES_(T) typename T##0, typename T##1, typename T##2, \ + typename T##3, typename T##4, typename T##5, typename T##6 +#define GTEST_8_TYPENAMES_(T) typename T##0, typename T##1, typename T##2, \ + typename T##3, typename T##4, typename T##5, typename T##6, typename T##7 +#define GTEST_9_TYPENAMES_(T) typename T##0, typename T##1, typename T##2, \ + typename T##3, typename T##4, typename T##5, typename T##6, \ + typename T##7, typename T##8 +#define GTEST_10_TYPENAMES_(T) typename T##0, typename T##1, typename T##2, \ + typename T##3, typename T##4, typename T##5, typename T##6, \ + typename T##7, typename T##8, typename T##9 + +// In theory, defining stuff in the ::std namespace is undefined +// behavior. We can do this as we are playing the role of a standard +// library vendor. +namespace std { +namespace tr1 { + +template +class tuple; + +// Anything in namespace gtest_internal is Google Test's INTERNAL +// IMPLEMENTATION DETAIL and MUST NOT BE USED DIRECTLY in user code. +namespace gtest_internal { + +// ByRef::type is T if T is a reference; otherwise it's const T&. +template +struct ByRef { typedef const T& type; }; // NOLINT +template +struct ByRef { typedef T& type; }; // NOLINT + +// A handy wrapper for ByRef. +#define GTEST_BY_REF_(T) typename ::std::tr1::gtest_internal::ByRef::type + +// AddRef::type is T if T is a reference; otherwise it's T&. This +// is the same as tr1::add_reference::type. +template +struct AddRef { typedef T& type; }; // NOLINT +template +struct AddRef { typedef T& type; }; // NOLINT + +// A handy wrapper for AddRef. +#define GTEST_ADD_REF_(T) typename ::std::tr1::gtest_internal::AddRef::type + +// A helper for implementing get(). +template class Get; + +// A helper for implementing tuple_element. kIndexValid is true +// iff k < the number of fields in tuple type T. +template +struct TupleElement; + +template +struct TupleElement { + typedef T0 type; +}; + +template +struct TupleElement { + typedef T1 type; +}; + +template +struct TupleElement { + typedef T2 type; +}; + +template +struct TupleElement { + typedef T3 type; +}; + +template +struct TupleElement { + typedef T4 type; +}; + +template +struct TupleElement { + typedef T5 type; +}; + +template +struct TupleElement { + typedef T6 type; +}; + +template +struct TupleElement { + typedef T7 type; +}; + +template +struct TupleElement { + typedef T8 type; +}; + +template +struct TupleElement { + typedef T9 type; +}; + +} // namespace gtest_internal + +template <> +class tuple<> { + public: + tuple() {} + tuple(const tuple& /* t */) {} + tuple& operator=(const tuple& /* t */) { return *this; } +}; + +template +class GTEST_1_TUPLE_(T) { + public: + template friend class gtest_internal::Get; + + tuple() : f0_() {} + + explicit tuple(GTEST_BY_REF_(T0) f0) : f0_(f0) {} + + tuple(const tuple& t) : f0_(t.f0_) {} + + template + tuple(const GTEST_1_TUPLE_(U)& t) : f0_(t.f0_) {} + + tuple& operator=(const tuple& t) { return CopyFrom(t); } + + template + tuple& operator=(const GTEST_1_TUPLE_(U)& t) { + return CopyFrom(t); + } + + GTEST_DECLARE_TUPLE_AS_FRIEND_ + + template + tuple& CopyFrom(const GTEST_1_TUPLE_(U)& t) { + f0_ = t.f0_; + return *this; + } + + T0 f0_; +}; + +template +class GTEST_2_TUPLE_(T) { + public: + template friend class gtest_internal::Get; + + tuple() : f0_(), f1_() {} + + explicit tuple(GTEST_BY_REF_(T0) f0, GTEST_BY_REF_(T1) f1) : f0_(f0), + f1_(f1) {} + + tuple(const tuple& t) : f0_(t.f0_), f1_(t.f1_) {} + + template + tuple(const GTEST_2_TUPLE_(U)& t) : f0_(t.f0_), f1_(t.f1_) {} + template + tuple(const ::std::pair& p) : f0_(p.first), f1_(p.second) {} + + tuple& operator=(const tuple& t) { return CopyFrom(t); } + + template + tuple& operator=(const GTEST_2_TUPLE_(U)& t) { + return CopyFrom(t); + } + template + tuple& operator=(const ::std::pair& p) { + f0_ = p.first; + f1_ = p.second; + return *this; + } + + GTEST_DECLARE_TUPLE_AS_FRIEND_ + + template + tuple& CopyFrom(const GTEST_2_TUPLE_(U)& t) { + f0_ = t.f0_; + f1_ = t.f1_; + return *this; + } + + T0 f0_; + T1 f1_; +}; + +template +class GTEST_3_TUPLE_(T) { + public: + template friend class gtest_internal::Get; + + tuple() : f0_(), f1_(), f2_() {} + + explicit tuple(GTEST_BY_REF_(T0) f0, GTEST_BY_REF_(T1) f1, + GTEST_BY_REF_(T2) f2) : f0_(f0), f1_(f1), f2_(f2) {} + + tuple(const tuple& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_) {} + + template + tuple(const GTEST_3_TUPLE_(U)& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_) {} + + tuple& operator=(const tuple& t) { return CopyFrom(t); } + + template + tuple& operator=(const GTEST_3_TUPLE_(U)& t) { + return CopyFrom(t); + } + + GTEST_DECLARE_TUPLE_AS_FRIEND_ + + template + tuple& CopyFrom(const GTEST_3_TUPLE_(U)& t) { + f0_ = t.f0_; + f1_ = t.f1_; + f2_ = t.f2_; + return *this; + } + + T0 f0_; + T1 f1_; + T2 f2_; +}; + +template +class GTEST_4_TUPLE_(T) { + public: + template friend class gtest_internal::Get; + + tuple() : f0_(), f1_(), f2_(), f3_() {} + + explicit tuple(GTEST_BY_REF_(T0) f0, GTEST_BY_REF_(T1) f1, + GTEST_BY_REF_(T2) f2, GTEST_BY_REF_(T3) f3) : f0_(f0), f1_(f1), f2_(f2), + f3_(f3) {} + + tuple(const tuple& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), f3_(t.f3_) {} + + template + tuple(const GTEST_4_TUPLE_(U)& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), + f3_(t.f3_) {} + + tuple& operator=(const tuple& t) { return CopyFrom(t); } + + template + tuple& operator=(const GTEST_4_TUPLE_(U)& t) { + return CopyFrom(t); + } + + GTEST_DECLARE_TUPLE_AS_FRIEND_ + + template + tuple& CopyFrom(const GTEST_4_TUPLE_(U)& t) { + f0_ = t.f0_; + f1_ = t.f1_; + f2_ = t.f2_; + f3_ = t.f3_; + return *this; + } + + T0 f0_; + T1 f1_; + T2 f2_; + T3 f3_; +}; + +template +class GTEST_5_TUPLE_(T) { + public: + template friend class gtest_internal::Get; + + tuple() : f0_(), f1_(), f2_(), f3_(), f4_() {} + + explicit tuple(GTEST_BY_REF_(T0) f0, GTEST_BY_REF_(T1) f1, + GTEST_BY_REF_(T2) f2, GTEST_BY_REF_(T3) f3, + GTEST_BY_REF_(T4) f4) : f0_(f0), f1_(f1), f2_(f2), f3_(f3), f4_(f4) {} + + tuple(const tuple& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), f3_(t.f3_), + f4_(t.f4_) {} + + template + tuple(const GTEST_5_TUPLE_(U)& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), + f3_(t.f3_), f4_(t.f4_) {} + + tuple& operator=(const tuple& t) { return CopyFrom(t); } + + template + tuple& operator=(const GTEST_5_TUPLE_(U)& t) { + return CopyFrom(t); + } + + GTEST_DECLARE_TUPLE_AS_FRIEND_ + + template + tuple& CopyFrom(const GTEST_5_TUPLE_(U)& t) { + f0_ = t.f0_; + f1_ = t.f1_; + f2_ = t.f2_; + f3_ = t.f3_; + f4_ = t.f4_; + return *this; + } + + T0 f0_; + T1 f1_; + T2 f2_; + T3 f3_; + T4 f4_; +}; + +template +class GTEST_6_TUPLE_(T) { + public: + template friend class gtest_internal::Get; + + tuple() : f0_(), f1_(), f2_(), f3_(), f4_(), f5_() {} + + explicit tuple(GTEST_BY_REF_(T0) f0, GTEST_BY_REF_(T1) f1, + GTEST_BY_REF_(T2) f2, GTEST_BY_REF_(T3) f3, GTEST_BY_REF_(T4) f4, + GTEST_BY_REF_(T5) f5) : f0_(f0), f1_(f1), f2_(f2), f3_(f3), f4_(f4), + f5_(f5) {} + + tuple(const tuple& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), f3_(t.f3_), + f4_(t.f4_), f5_(t.f5_) {} + + template + tuple(const GTEST_6_TUPLE_(U)& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), + f3_(t.f3_), f4_(t.f4_), f5_(t.f5_) {} + + tuple& operator=(const tuple& t) { return CopyFrom(t); } + + template + tuple& operator=(const GTEST_6_TUPLE_(U)& t) { + return CopyFrom(t); + } + + GTEST_DECLARE_TUPLE_AS_FRIEND_ + + template + tuple& CopyFrom(const GTEST_6_TUPLE_(U)& t) { + f0_ = t.f0_; + f1_ = t.f1_; + f2_ = t.f2_; + f3_ = t.f3_; + f4_ = t.f4_; + f5_ = t.f5_; + return *this; + } + + T0 f0_; + T1 f1_; + T2 f2_; + T3 f3_; + T4 f4_; + T5 f5_; +}; + +template +class GTEST_7_TUPLE_(T) { + public: + template friend class gtest_internal::Get; + + tuple() : f0_(), f1_(), f2_(), f3_(), f4_(), f5_(), f6_() {} + + explicit tuple(GTEST_BY_REF_(T0) f0, GTEST_BY_REF_(T1) f1, + GTEST_BY_REF_(T2) f2, GTEST_BY_REF_(T3) f3, GTEST_BY_REF_(T4) f4, + GTEST_BY_REF_(T5) f5, GTEST_BY_REF_(T6) f6) : f0_(f0), f1_(f1), f2_(f2), + f3_(f3), f4_(f4), f5_(f5), f6_(f6) {} + + tuple(const tuple& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), f3_(t.f3_), + f4_(t.f4_), f5_(t.f5_), f6_(t.f6_) {} + + template + tuple(const GTEST_7_TUPLE_(U)& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), + f3_(t.f3_), f4_(t.f4_), f5_(t.f5_), f6_(t.f6_) {} + + tuple& operator=(const tuple& t) { return CopyFrom(t); } + + template + tuple& operator=(const GTEST_7_TUPLE_(U)& t) { + return CopyFrom(t); + } + + GTEST_DECLARE_TUPLE_AS_FRIEND_ + + template + tuple& CopyFrom(const GTEST_7_TUPLE_(U)& t) { + f0_ = t.f0_; + f1_ = t.f1_; + f2_ = t.f2_; + f3_ = t.f3_; + f4_ = t.f4_; + f5_ = t.f5_; + f6_ = t.f6_; + return *this; + } + + T0 f0_; + T1 f1_; + T2 f2_; + T3 f3_; + T4 f4_; + T5 f5_; + T6 f6_; +}; + +template +class GTEST_8_TUPLE_(T) { + public: + template friend class gtest_internal::Get; + + tuple() : f0_(), f1_(), f2_(), f3_(), f4_(), f5_(), f6_(), f7_() {} + + explicit tuple(GTEST_BY_REF_(T0) f0, GTEST_BY_REF_(T1) f1, + GTEST_BY_REF_(T2) f2, GTEST_BY_REF_(T3) f3, GTEST_BY_REF_(T4) f4, + GTEST_BY_REF_(T5) f5, GTEST_BY_REF_(T6) f6, + GTEST_BY_REF_(T7) f7) : f0_(f0), f1_(f1), f2_(f2), f3_(f3), f4_(f4), + f5_(f5), f6_(f6), f7_(f7) {} + + tuple(const tuple& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), f3_(t.f3_), + f4_(t.f4_), f5_(t.f5_), f6_(t.f6_), f7_(t.f7_) {} + + template + tuple(const GTEST_8_TUPLE_(U)& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), + f3_(t.f3_), f4_(t.f4_), f5_(t.f5_), f6_(t.f6_), f7_(t.f7_) {} + + tuple& operator=(const tuple& t) { return CopyFrom(t); } + + template + tuple& operator=(const GTEST_8_TUPLE_(U)& t) { + return CopyFrom(t); + } + + GTEST_DECLARE_TUPLE_AS_FRIEND_ + + template + tuple& CopyFrom(const GTEST_8_TUPLE_(U)& t) { + f0_ = t.f0_; + f1_ = t.f1_; + f2_ = t.f2_; + f3_ = t.f3_; + f4_ = t.f4_; + f5_ = t.f5_; + f6_ = t.f6_; + f7_ = t.f7_; + return *this; + } + + T0 f0_; + T1 f1_; + T2 f2_; + T3 f3_; + T4 f4_; + T5 f5_; + T6 f6_; + T7 f7_; +}; + +template +class GTEST_9_TUPLE_(T) { + public: + template friend class gtest_internal::Get; + + tuple() : f0_(), f1_(), f2_(), f3_(), f4_(), f5_(), f6_(), f7_(), f8_() {} + + explicit tuple(GTEST_BY_REF_(T0) f0, GTEST_BY_REF_(T1) f1, + GTEST_BY_REF_(T2) f2, GTEST_BY_REF_(T3) f3, GTEST_BY_REF_(T4) f4, + GTEST_BY_REF_(T5) f5, GTEST_BY_REF_(T6) f6, GTEST_BY_REF_(T7) f7, + GTEST_BY_REF_(T8) f8) : f0_(f0), f1_(f1), f2_(f2), f3_(f3), f4_(f4), + f5_(f5), f6_(f6), f7_(f7), f8_(f8) {} + + tuple(const tuple& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), f3_(t.f3_), + f4_(t.f4_), f5_(t.f5_), f6_(t.f6_), f7_(t.f7_), f8_(t.f8_) {} + + template + tuple(const GTEST_9_TUPLE_(U)& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), + f3_(t.f3_), f4_(t.f4_), f5_(t.f5_), f6_(t.f6_), f7_(t.f7_), f8_(t.f8_) {} + + tuple& operator=(const tuple& t) { return CopyFrom(t); } + + template + tuple& operator=(const GTEST_9_TUPLE_(U)& t) { + return CopyFrom(t); + } + + GTEST_DECLARE_TUPLE_AS_FRIEND_ + + template + tuple& CopyFrom(const GTEST_9_TUPLE_(U)& t) { + f0_ = t.f0_; + f1_ = t.f1_; + f2_ = t.f2_; + f3_ = t.f3_; + f4_ = t.f4_; + f5_ = t.f5_; + f6_ = t.f6_; + f7_ = t.f7_; + f8_ = t.f8_; + return *this; + } + + T0 f0_; + T1 f1_; + T2 f2_; + T3 f3_; + T4 f4_; + T5 f5_; + T6 f6_; + T7 f7_; + T8 f8_; +}; + +template +class tuple { + public: + template friend class gtest_internal::Get; + + tuple() : f0_(), f1_(), f2_(), f3_(), f4_(), f5_(), f6_(), f7_(), f8_(), + f9_() {} + + explicit tuple(GTEST_BY_REF_(T0) f0, GTEST_BY_REF_(T1) f1, + GTEST_BY_REF_(T2) f2, GTEST_BY_REF_(T3) f3, GTEST_BY_REF_(T4) f4, + GTEST_BY_REF_(T5) f5, GTEST_BY_REF_(T6) f6, GTEST_BY_REF_(T7) f7, + GTEST_BY_REF_(T8) f8, GTEST_BY_REF_(T9) f9) : f0_(f0), f1_(f1), f2_(f2), + f3_(f3), f4_(f4), f5_(f5), f6_(f6), f7_(f7), f8_(f8), f9_(f9) {} + + tuple(const tuple& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), f3_(t.f3_), + f4_(t.f4_), f5_(t.f5_), f6_(t.f6_), f7_(t.f7_), f8_(t.f8_), f9_(t.f9_) {} + + template + tuple(const GTEST_10_TUPLE_(U)& t) : f0_(t.f0_), f1_(t.f1_), f2_(t.f2_), + f3_(t.f3_), f4_(t.f4_), f5_(t.f5_), f6_(t.f6_), f7_(t.f7_), f8_(t.f8_), + f9_(t.f9_) {} + + tuple& operator=(const tuple& t) { return CopyFrom(t); } + + template + tuple& operator=(const GTEST_10_TUPLE_(U)& t) { + return CopyFrom(t); + } + + GTEST_DECLARE_TUPLE_AS_FRIEND_ + + template + tuple& CopyFrom(const GTEST_10_TUPLE_(U)& t) { + f0_ = t.f0_; + f1_ = t.f1_; + f2_ = t.f2_; + f3_ = t.f3_; + f4_ = t.f4_; + f5_ = t.f5_; + f6_ = t.f6_; + f7_ = t.f7_; + f8_ = t.f8_; + f9_ = t.f9_; + return *this; + } + + T0 f0_; + T1 f1_; + T2 f2_; + T3 f3_; + T4 f4_; + T5 f5_; + T6 f6_; + T7 f7_; + T8 f8_; + T9 f9_; +}; + +// 6.1.3.2 Tuple creation functions. + +// Known limitations: we don't support passing an +// std::tr1::reference_wrapper to make_tuple(). And we don't +// implement tie(). + +inline tuple<> make_tuple() { return tuple<>(); } + +template +inline GTEST_1_TUPLE_(T) make_tuple(const T0& f0) { + return GTEST_1_TUPLE_(T)(f0); +} + +template +inline GTEST_2_TUPLE_(T) make_tuple(const T0& f0, const T1& f1) { + return GTEST_2_TUPLE_(T)(f0, f1); +} + +template +inline GTEST_3_TUPLE_(T) make_tuple(const T0& f0, const T1& f1, const T2& f2) { + return GTEST_3_TUPLE_(T)(f0, f1, f2); +} + +template +inline GTEST_4_TUPLE_(T) make_tuple(const T0& f0, const T1& f1, const T2& f2, + const T3& f3) { + return GTEST_4_TUPLE_(T)(f0, f1, f2, f3); +} + +template +inline GTEST_5_TUPLE_(T) make_tuple(const T0& f0, const T1& f1, const T2& f2, + const T3& f3, const T4& f4) { + return GTEST_5_TUPLE_(T)(f0, f1, f2, f3, f4); +} + +template +inline GTEST_6_TUPLE_(T) make_tuple(const T0& f0, const T1& f1, const T2& f2, + const T3& f3, const T4& f4, const T5& f5) { + return GTEST_6_TUPLE_(T)(f0, f1, f2, f3, f4, f5); +} + +template +inline GTEST_7_TUPLE_(T) make_tuple(const T0& f0, const T1& f1, const T2& f2, + const T3& f3, const T4& f4, const T5& f5, const T6& f6) { + return GTEST_7_TUPLE_(T)(f0, f1, f2, f3, f4, f5, f6); +} + +template +inline GTEST_8_TUPLE_(T) make_tuple(const T0& f0, const T1& f1, const T2& f2, + const T3& f3, const T4& f4, const T5& f5, const T6& f6, const T7& f7) { + return GTEST_8_TUPLE_(T)(f0, f1, f2, f3, f4, f5, f6, f7); +} + +template +inline GTEST_9_TUPLE_(T) make_tuple(const T0& f0, const T1& f1, const T2& f2, + const T3& f3, const T4& f4, const T5& f5, const T6& f6, const T7& f7, + const T8& f8) { + return GTEST_9_TUPLE_(T)(f0, f1, f2, f3, f4, f5, f6, f7, f8); +} + +template +inline GTEST_10_TUPLE_(T) make_tuple(const T0& f0, const T1& f1, const T2& f2, + const T3& f3, const T4& f4, const T5& f5, const T6& f6, const T7& f7, + const T8& f8, const T9& f9) { + return GTEST_10_TUPLE_(T)(f0, f1, f2, f3, f4, f5, f6, f7, f8, f9); +} + +// 6.1.3.3 Tuple helper classes. + +template struct tuple_size; + +template +struct tuple_size { + static const int value = 0; +}; + +template +struct tuple_size { + static const int value = 1; +}; + +template +struct tuple_size { + static const int value = 2; +}; + +template +struct tuple_size { + static const int value = 3; +}; + +template +struct tuple_size { + static const int value = 4; +}; + +template +struct tuple_size { + static const int value = 5; +}; + +template +struct tuple_size { + static const int value = 6; +}; + +template +struct tuple_size { + static const int value = 7; +}; + +template +struct tuple_size { + static const int value = 8; +}; + +template +struct tuple_size { + static const int value = 9; +}; + +template +struct tuple_size { + static const int value = 10; +}; + +template +struct tuple_element { + typedef typename gtest_internal::TupleElement< + k < (tuple_size::value), k, Tuple>::type type; +}; + +#define GTEST_TUPLE_ELEMENT_(k, Tuple) typename tuple_element::type + +// 6.1.3.4 Element access. + +namespace gtest_internal { + +template <> +class Get<0> { + public: + template + static GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(0, Tuple)) + Field(Tuple& t) { return t.f0_; } // NOLINT + + template + static GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(0, Tuple)) + ConstField(const Tuple& t) { return t.f0_; } +}; + +template <> +class Get<1> { + public: + template + static GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(1, Tuple)) + Field(Tuple& t) { return t.f1_; } // NOLINT + + template + static GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(1, Tuple)) + ConstField(const Tuple& t) { return t.f1_; } +}; + +template <> +class Get<2> { + public: + template + static GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(2, Tuple)) + Field(Tuple& t) { return t.f2_; } // NOLINT + + template + static GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(2, Tuple)) + ConstField(const Tuple& t) { return t.f2_; } +}; + +template <> +class Get<3> { + public: + template + static GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(3, Tuple)) + Field(Tuple& t) { return t.f3_; } // NOLINT + + template + static GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(3, Tuple)) + ConstField(const Tuple& t) { return t.f3_; } +}; + +template <> +class Get<4> { + public: + template + static GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(4, Tuple)) + Field(Tuple& t) { return t.f4_; } // NOLINT + + template + static GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(4, Tuple)) + ConstField(const Tuple& t) { return t.f4_; } +}; + +template <> +class Get<5> { + public: + template + static GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(5, Tuple)) + Field(Tuple& t) { return t.f5_; } // NOLINT + + template + static GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(5, Tuple)) + ConstField(const Tuple& t) { return t.f5_; } +}; + +template <> +class Get<6> { + public: + template + static GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(6, Tuple)) + Field(Tuple& t) { return t.f6_; } // NOLINT + + template + static GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(6, Tuple)) + ConstField(const Tuple& t) { return t.f6_; } +}; + +template <> +class Get<7> { + public: + template + static GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(7, Tuple)) + Field(Tuple& t) { return t.f7_; } // NOLINT + + template + static GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(7, Tuple)) + ConstField(const Tuple& t) { return t.f7_; } +}; + +template <> +class Get<8> { + public: + template + static GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(8, Tuple)) + Field(Tuple& t) { return t.f8_; } // NOLINT + + template + static GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(8, Tuple)) + ConstField(const Tuple& t) { return t.f8_; } +}; + +template <> +class Get<9> { + public: + template + static GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(9, Tuple)) + Field(Tuple& t) { return t.f9_; } // NOLINT + + template + static GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(9, Tuple)) + ConstField(const Tuple& t) { return t.f9_; } +}; + +} // namespace gtest_internal + +template +GTEST_ADD_REF_(GTEST_TUPLE_ELEMENT_(k, GTEST_10_TUPLE_(T))) +get(GTEST_10_TUPLE_(T)& t) { + return gtest_internal::Get::Field(t); +} + +template +GTEST_BY_REF_(GTEST_TUPLE_ELEMENT_(k, GTEST_10_TUPLE_(T))) +get(const GTEST_10_TUPLE_(T)& t) { + return gtest_internal::Get::ConstField(t); +} + +// 6.1.3.5 Relational operators + +// We only implement == and !=, as we don't have a need for the rest yet. + +namespace gtest_internal { + +// SameSizeTuplePrefixComparator::Eq(t1, t2) returns true if the +// first k fields of t1 equals the first k fields of t2. +// SameSizeTuplePrefixComparator(k1, k2) would be a compiler error if +// k1 != k2. +template +struct SameSizeTuplePrefixComparator; + +template <> +struct SameSizeTuplePrefixComparator<0, 0> { + template + static bool Eq(const Tuple1& /* t1 */, const Tuple2& /* t2 */) { + return true; + } +}; + +template +struct SameSizeTuplePrefixComparator { + template + static bool Eq(const Tuple1& t1, const Tuple2& t2) { + return SameSizeTuplePrefixComparator::Eq(t1, t2) && + ::std::tr1::get(t1) == ::std::tr1::get(t2); + } +}; + +} // namespace gtest_internal + +template +inline bool operator==(const GTEST_10_TUPLE_(T)& t, + const GTEST_10_TUPLE_(U)& u) { + return gtest_internal::SameSizeTuplePrefixComparator< + tuple_size::value, + tuple_size::value>::Eq(t, u); +} + +template +inline bool operator!=(const GTEST_10_TUPLE_(T)& t, + const GTEST_10_TUPLE_(U)& u) { return !(t == u); } + +// 6.1.4 Pairs. +// Unimplemented. + +} // namespace tr1 +} // namespace std + +#undef GTEST_0_TUPLE_ +#undef GTEST_1_TUPLE_ +#undef GTEST_2_TUPLE_ +#undef GTEST_3_TUPLE_ +#undef GTEST_4_TUPLE_ +#undef GTEST_5_TUPLE_ +#undef GTEST_6_TUPLE_ +#undef GTEST_7_TUPLE_ +#undef GTEST_8_TUPLE_ +#undef GTEST_9_TUPLE_ +#undef GTEST_10_TUPLE_ + +#undef GTEST_0_TYPENAMES_ +#undef GTEST_1_TYPENAMES_ +#undef GTEST_2_TYPENAMES_ +#undef GTEST_3_TYPENAMES_ +#undef GTEST_4_TYPENAMES_ +#undef GTEST_5_TYPENAMES_ +#undef GTEST_6_TYPENAMES_ +#undef GTEST_7_TYPENAMES_ +#undef GTEST_8_TYPENAMES_ +#undef GTEST_9_TYPENAMES_ +#undef GTEST_10_TYPENAMES_ + +#undef GTEST_DECLARE_TUPLE_AS_FRIEND_ +#undef GTEST_BY_REF_ +#undef GTEST_ADD_REF_ +#undef GTEST_TUPLE_ELEMENT_ + +#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_TUPLE_H_ +# elif GTEST_ENV_HAS_STD_TUPLE_ +# include +// C++11 puts its tuple into the ::std namespace rather than +// ::std::tr1. gtest expects tuple to live in ::std::tr1, so put it there. +// This causes undefined behavior, but supported compilers react in +// the way we intend. +namespace std { +namespace tr1 { +using ::std::get; +using ::std::make_tuple; +using ::std::tuple; +using ::std::tuple_element; +using ::std::tuple_size; +} +} + +# elif GTEST_OS_SYMBIAN + +// On Symbian, BOOST_HAS_TR1_TUPLE causes Boost's TR1 tuple library to +// use STLport's tuple implementation, which unfortunately doesn't +// work as the copy of STLport distributed with Symbian is incomplete. +// By making sure BOOST_HAS_TR1_TUPLE is undefined, we force Boost to +// use its own tuple implementation. +# ifdef BOOST_HAS_TR1_TUPLE +# undef BOOST_HAS_TR1_TUPLE +# endif // BOOST_HAS_TR1_TUPLE + +// This prevents , which defines +// BOOST_HAS_TR1_TUPLE, from being #included by Boost's . +# define BOOST_TR1_DETAIL_CONFIG_HPP_INCLUDED +# include // IWYU pragma: export // NOLINT + +# elif defined(__GNUC__) && (GTEST_GCC_VER_ >= 40000) +// GCC 4.0+ implements tr1/tuple in the header. This does +// not conform to the TR1 spec, which requires the header to be . + +# if !GTEST_HAS_RTTI && GTEST_GCC_VER_ < 40302 +// Until version 4.3.2, gcc has a bug that causes , +// which is #included by , to not compile when RTTI is +// disabled. _TR1_FUNCTIONAL is the header guard for +// . Hence the following #define is a hack to prevent +// from being included. +# define _TR1_FUNCTIONAL 1 +# include +# undef _TR1_FUNCTIONAL // Allows the user to #include + // if he chooses to. +# else +# include // NOLINT +# endif // !GTEST_HAS_RTTI && GTEST_GCC_VER_ < 40302 + +# else +// If the compiler is not GCC 4.0+, we assume the user is using a +// spec-conforming TR1 implementation. +# include // IWYU pragma: export // NOLINT +# endif // GTEST_USE_OWN_TR1_TUPLE + +#endif // GTEST_HAS_TR1_TUPLE + +// Determines whether clone(2) is supported. +// Usually it will only be available on Linux, excluding +// Linux on the Itanium architecture. +// Also see http://linux.die.net/man/2/clone. +#ifndef GTEST_HAS_CLONE +// The user didn't tell us, so we need to figure it out. + +# if GTEST_OS_LINUX && !defined(__ia64__) +# if GTEST_OS_LINUX_ANDROID +// On Android, clone() is only available on ARM starting with Gingerbread. +# if defined(__arm__) && __ANDROID_API__ >= 9 +# define GTEST_HAS_CLONE 1 +# else +# define GTEST_HAS_CLONE 0 +# endif +# else +# define GTEST_HAS_CLONE 1 +# endif +# else +# define GTEST_HAS_CLONE 0 +# endif // GTEST_OS_LINUX && !defined(__ia64__) + +#endif // GTEST_HAS_CLONE + +// Determines whether to support stream redirection. This is used to test +// output correctness and to implement death tests. +#ifndef GTEST_HAS_STREAM_REDIRECTION +// By default, we assume that stream redirection is supported on all +// platforms except known mobile ones. +# if GTEST_OS_WINDOWS_MOBILE || GTEST_OS_SYMBIAN || \ + GTEST_OS_WINDOWS_PHONE || GTEST_OS_WINDOWS_RT +# define GTEST_HAS_STREAM_REDIRECTION 0 +# else +# define GTEST_HAS_STREAM_REDIRECTION 1 +# endif // !GTEST_OS_WINDOWS_MOBILE && !GTEST_OS_SYMBIAN +#endif // GTEST_HAS_STREAM_REDIRECTION + +// Determines whether to support death tests. +// Google Test does not support death tests for VC 7.1 and earlier as +// abort() in a VC 7.1 application compiled as GUI in debug config +// pops up a dialog window that cannot be suppressed programmatically. +#if (GTEST_OS_LINUX || GTEST_OS_CYGWIN || GTEST_OS_SOLARIS || \ + (GTEST_OS_MAC && !GTEST_OS_IOS) || \ + (GTEST_OS_WINDOWS_DESKTOP && _MSC_VER >= 1400) || \ + GTEST_OS_WINDOWS_MINGW || GTEST_OS_AIX || GTEST_OS_HPUX || \ + GTEST_OS_OPENBSD || GTEST_OS_QNX || GTEST_OS_FREEBSD) +# define GTEST_HAS_DEATH_TEST 1 +# include // NOLINT +#endif + +// We don't support MSVC 7.1 with exceptions disabled now. Therefore +// all the compilers we care about are adequate for supporting +// value-parameterized tests. +#define GTEST_HAS_PARAM_TEST 1 + +// Determines whether to support type-driven tests. + +// Typed tests need and variadic macros, which GCC, VC++ 8.0, +// Sun Pro CC, IBM Visual Age, and HP aCC support. +#if defined(__GNUC__) || (_MSC_VER >= 1400) || defined(__SUNPRO_CC) || \ + defined(__IBMCPP__) || defined(__HP_aCC) +# define GTEST_HAS_TYPED_TEST 1 +# define GTEST_HAS_TYPED_TEST_P 1 +#endif + +// Determines whether to support Combine(). This only makes sense when +// value-parameterized tests are enabled. The implementation doesn't +// work on Sun Studio since it doesn't understand templated conversion +// operators. +#if GTEST_HAS_PARAM_TEST && GTEST_HAS_TR1_TUPLE && !defined(__SUNPRO_CC) +# define GTEST_HAS_COMBINE 1 +#endif + +// Determines whether the system compiler uses UTF-16 for encoding wide strings. +#define GTEST_WIDE_STRING_USES_UTF16_ \ + (GTEST_OS_WINDOWS || GTEST_OS_CYGWIN || GTEST_OS_SYMBIAN || GTEST_OS_AIX) + +// Determines whether test results can be streamed to a socket. +#if GTEST_OS_LINUX +# define GTEST_CAN_STREAM_RESULTS_ 1 +#endif + +// Defines some utility macros. + +// The GNU compiler emits a warning if nested "if" statements are followed by +// an "else" statement and braces are not used to explicitly disambiguate the +// "else" binding. This leads to problems with code like: +// +// if (gate) +// ASSERT_*(condition) << "Some message"; +// +// The "switch (0) case 0:" idiom is used to suppress this. +#ifdef __INTEL_COMPILER +# define GTEST_AMBIGUOUS_ELSE_BLOCKER_ +#else +# define GTEST_AMBIGUOUS_ELSE_BLOCKER_ switch (0) case 0: default: // NOLINT +#endif + +// Use this annotation at the end of a struct/class definition to +// prevent the compiler from optimizing away instances that are never +// used. This is useful when all interesting logic happens inside the +// c'tor and / or d'tor. Example: +// +// struct Foo { +// Foo() { ... } +// } GTEST_ATTRIBUTE_UNUSED_; +// +// Also use it after a variable or parameter declaration to tell the +// compiler the variable/parameter does not have to be used. +#if defined(__GNUC__) && !defined(COMPILER_ICC) +# define GTEST_ATTRIBUTE_UNUSED_ __attribute__ ((unused)) +#elif defined(__clang__) +# if __has_attribute(unused) +# define GTEST_ATTRIBUTE_UNUSED_ __attribute__ ((unused)) +# endif +#endif +#ifndef GTEST_ATTRIBUTE_UNUSED_ +# define GTEST_ATTRIBUTE_UNUSED_ +#endif + +// A macro to disallow operator= +// This should be used in the private: declarations for a class. +#define GTEST_DISALLOW_ASSIGN_(type)\ + void operator=(type const &) + +// A macro to disallow copy constructor and operator= +// This should be used in the private: declarations for a class. +#define GTEST_DISALLOW_COPY_AND_ASSIGN_(type)\ + type(type const &);\ + GTEST_DISALLOW_ASSIGN_(type) + +// Tell the compiler to warn about unused return values for functions declared +// with this macro. The macro should be used on function declarations +// following the argument list: +// +// Sprocket* AllocateSprocket() GTEST_MUST_USE_RESULT_; +#if defined(__GNUC__) && (GTEST_GCC_VER_ >= 30400) && !defined(COMPILER_ICC) +# define GTEST_MUST_USE_RESULT_ __attribute__ ((warn_unused_result)) +#else +# define GTEST_MUST_USE_RESULT_ +#endif // __GNUC__ && (GTEST_GCC_VER_ >= 30400) && !COMPILER_ICC + +// MS C++ compiler emits warning when a conditional expression is compile time +// constant. In some contexts this warning is false positive and needs to be +// suppressed. Use the following two macros in such cases: +// +// GTEST_INTENTIONAL_CONST_COND_PUSH_() +// while (true) { +// GTEST_INTENTIONAL_CONST_COND_POP_() +// } +# define GTEST_INTENTIONAL_CONST_COND_PUSH_() \ + GTEST_DISABLE_MSC_WARNINGS_PUSH_(4127) +# define GTEST_INTENTIONAL_CONST_COND_POP_() \ + GTEST_DISABLE_MSC_WARNINGS_POP_() + +// Determine whether the compiler supports Microsoft's Structured Exception +// Handling. This is supported by several Windows compilers but generally +// does not exist on any other system. +#ifndef GTEST_HAS_SEH +// The user didn't tell us, so we need to figure it out. + +# if defined(_MSC_VER) || defined(__BORLANDC__) +// These two compilers are known to support SEH. +# define GTEST_HAS_SEH 1 +# else +// Assume no SEH. +# define GTEST_HAS_SEH 0 +# endif + +#define GTEST_IS_THREADSAFE \ + (0 \ + || (GTEST_OS_WINDOWS && !GTEST_OS_WINDOWS_PHONE && !GTEST_OS_WINDOWS_RT) \ + || GTEST_HAS_PTHREAD) + +#endif // GTEST_HAS_SEH + +#ifdef _MSC_VER + +# if GTEST_LINKED_AS_SHARED_LIBRARY +# define GTEST_API_ __declspec(dllimport) +# elif GTEST_CREATE_SHARED_LIBRARY +# define GTEST_API_ __declspec(dllexport) +# endif + +#endif // _MSC_VER + +#ifndef GTEST_API_ +# define GTEST_API_ +#endif + +#ifdef __GNUC__ +// Ask the compiler to never inline a given function. +# define GTEST_NO_INLINE_ __attribute__((noinline)) +#else +# define GTEST_NO_INLINE_ +#endif + +// _LIBCPP_VERSION is defined by the libc++ library from the LLVM project. +#if defined(__GLIBCXX__) || defined(_LIBCPP_VERSION) +# define GTEST_HAS_CXXABI_H_ 1 +#else +# define GTEST_HAS_CXXABI_H_ 0 +#endif + +// A function level attribute to disable checking for use of uninitialized +// memory when built with MemorySanitizer. +#if defined(__clang__) +# if __has_feature(memory_sanitizer) +# define GTEST_ATTRIBUTE_NO_SANITIZE_MEMORY_ \ + __attribute__((no_sanitize_memory)) +# else +# define GTEST_ATTRIBUTE_NO_SANITIZE_MEMORY_ +# endif // __has_feature(memory_sanitizer) +#else +# define GTEST_ATTRIBUTE_NO_SANITIZE_MEMORY_ +#endif // __clang__ + +// A function level attribute to disable AddressSanitizer instrumentation. +#if defined(__clang__) +# if __has_feature(address_sanitizer) +# define GTEST_ATTRIBUTE_NO_SANITIZE_ADDRESS_ \ + __attribute__((no_sanitize_address)) +# else +# define GTEST_ATTRIBUTE_NO_SANITIZE_ADDRESS_ +# endif // __has_feature(address_sanitizer) +#else +# define GTEST_ATTRIBUTE_NO_SANITIZE_ADDRESS_ +#endif // __clang__ + +// A function level attribute to disable ThreadSanitizer instrumentation. +#if defined(__clang__) +# if __has_feature(thread_sanitizer) +# define GTEST_ATTRIBUTE_NO_SANITIZE_THREAD_ \ + __attribute__((no_sanitize_thread)) +# else +# define GTEST_ATTRIBUTE_NO_SANITIZE_THREAD_ +# endif // __has_feature(thread_sanitizer) +#else +# define GTEST_ATTRIBUTE_NO_SANITIZE_THREAD_ +#endif // __clang__ + +namespace testing { + +class Message; + +#if defined(GTEST_TUPLE_NAMESPACE_) +// Import tuple and friends into the ::testing namespace. +// It is part of our interface, having them in ::testing allows us to change +// their types as needed. +using GTEST_TUPLE_NAMESPACE_::get; +using GTEST_TUPLE_NAMESPACE_::make_tuple; +using GTEST_TUPLE_NAMESPACE_::tuple; +using GTEST_TUPLE_NAMESPACE_::tuple_size; +using GTEST_TUPLE_NAMESPACE_::tuple_element; +#endif // defined(GTEST_TUPLE_NAMESPACE_) + +namespace internal { + +// A secret type that Google Test users don't know about. It has no +// definition on purpose. Therefore it's impossible to create a +// Secret object, which is what we want. +class Secret; + +// The GTEST_COMPILE_ASSERT_ macro can be used to verify that a compile time +// expression is true. For example, you could use it to verify the +// size of a static array: +// +// GTEST_COMPILE_ASSERT_(GTEST_ARRAY_SIZE_(names) == NUM_NAMES, +// names_incorrect_size); +// +// or to make sure a struct is smaller than a certain size: +// +// GTEST_COMPILE_ASSERT_(sizeof(foo) < 128, foo_too_large); +// +// The second argument to the macro is the name of the variable. If +// the expression is false, most compilers will issue a warning/error +// containing the name of the variable. + +#if GTEST_LANG_CXX11 +# define GTEST_COMPILE_ASSERT_(expr, msg) static_assert(expr, #msg) +#else // !GTEST_LANG_CXX11 +template + struct CompileAssert { +}; + +# define GTEST_COMPILE_ASSERT_(expr, msg) \ + typedef ::testing::internal::CompileAssert<(static_cast(expr))> \ + msg[static_cast(expr) ? 1 : -1] GTEST_ATTRIBUTE_UNUSED_ +#endif // !GTEST_LANG_CXX11 + +// Implementation details of GTEST_COMPILE_ASSERT_: +// +// (In C++11, we simply use static_assert instead of the following) +// +// - GTEST_COMPILE_ASSERT_ works by defining an array type that has -1 +// elements (and thus is invalid) when the expression is false. +// +// - The simpler definition +// +// #define GTEST_COMPILE_ASSERT_(expr, msg) typedef char msg[(expr) ? 1 : -1] +// +// does not work, as gcc supports variable-length arrays whose sizes +// are determined at run-time (this is gcc's extension and not part +// of the C++ standard). As a result, gcc fails to reject the +// following code with the simple definition: +// +// int foo; +// GTEST_COMPILE_ASSERT_(foo, msg); // not supposed to compile as foo is +// // not a compile-time constant. +// +// - By using the type CompileAssert<(bool(expr))>, we ensures that +// expr is a compile-time constant. (Template arguments must be +// determined at compile-time.) +// +// - The outter parentheses in CompileAssert<(bool(expr))> are necessary +// to work around a bug in gcc 3.4.4 and 4.0.1. If we had written +// +// CompileAssert +// +// instead, these compilers will refuse to compile +// +// GTEST_COMPILE_ASSERT_(5 > 0, some_message); +// +// (They seem to think the ">" in "5 > 0" marks the end of the +// template argument list.) +// +// - The array size is (bool(expr) ? 1 : -1), instead of simply +// +// ((expr) ? 1 : -1). +// +// This is to avoid running into a bug in MS VC 7.1, which +// causes ((0.0) ? 1 : -1) to incorrectly evaluate to 1. + +// StaticAssertTypeEqHelper is used by StaticAssertTypeEq defined in gtest.h. +// +// This template is declared, but intentionally undefined. +template +struct StaticAssertTypeEqHelper; + +template +struct StaticAssertTypeEqHelper { + enum { value = true }; +}; + +// Evaluates to the number of elements in 'array'. +#define GTEST_ARRAY_SIZE_(array) (sizeof(array) / sizeof(array[0])) + +#if GTEST_HAS_GLOBAL_STRING +typedef ::string string; +#else +typedef ::std::string string; +#endif // GTEST_HAS_GLOBAL_STRING + +#if GTEST_HAS_GLOBAL_WSTRING +typedef ::wstring wstring; +#elif GTEST_HAS_STD_WSTRING +typedef ::std::wstring wstring; +#endif // GTEST_HAS_GLOBAL_WSTRING + +// A helper for suppressing warnings on constant condition. It just +// returns 'condition'. +GTEST_API_ bool IsTrue(bool condition); + +// Defines scoped_ptr. + +// This implementation of scoped_ptr is PARTIAL - it only contains +// enough stuff to satisfy Google Test's need. +template +class scoped_ptr { + public: + typedef T element_type; + + explicit scoped_ptr(T* p = NULL) : ptr_(p) {} + ~scoped_ptr() { reset(); } + + T& operator*() const { return *ptr_; } + T* operator->() const { return ptr_; } + T* get() const { return ptr_; } + + T* release() { + T* const ptr = ptr_; + ptr_ = NULL; + return ptr; + } + + void reset(T* p = NULL) { + if (p != ptr_) { + if (IsTrue(sizeof(T) > 0)) { // Makes sure T is a complete type. + delete ptr_; + } + ptr_ = p; + } + } + + friend void swap(scoped_ptr& a, scoped_ptr& b) { + using std::swap; + swap(a.ptr_, b.ptr_); + } + + private: + T* ptr_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(scoped_ptr); +}; + +// Defines RE. + +// A simple C++ wrapper for . It uses the POSIX Extended +// Regular Expression syntax. +class GTEST_API_ RE { + public: + // A copy constructor is required by the Standard to initialize object + // references from r-values. + RE(const RE& other) { Init(other.pattern()); } + + // Constructs an RE from a string. + RE(const ::std::string& regex) { Init(regex.c_str()); } // NOLINT + +#if GTEST_HAS_GLOBAL_STRING + + RE(const ::string& regex) { Init(regex.c_str()); } // NOLINT + +#endif // GTEST_HAS_GLOBAL_STRING + + RE(const char* regex) { Init(regex); } // NOLINT + ~RE(); + + // Returns the string representation of the regex. + const char* pattern() const { return pattern_; } + + // FullMatch(str, re) returns true iff regular expression re matches + // the entire str. + // PartialMatch(str, re) returns true iff regular expression re + // matches a substring of str (including str itself). + // + // TODO(wan@google.com): make FullMatch() and PartialMatch() work + // when str contains NUL characters. + static bool FullMatch(const ::std::string& str, const RE& re) { + return FullMatch(str.c_str(), re); + } + static bool PartialMatch(const ::std::string& str, const RE& re) { + return PartialMatch(str.c_str(), re); + } + +#if GTEST_HAS_GLOBAL_STRING + + static bool FullMatch(const ::string& str, const RE& re) { + return FullMatch(str.c_str(), re); + } + static bool PartialMatch(const ::string& str, const RE& re) { + return PartialMatch(str.c_str(), re); + } + +#endif // GTEST_HAS_GLOBAL_STRING + + static bool FullMatch(const char* str, const RE& re); + static bool PartialMatch(const char* str, const RE& re); + + private: + void Init(const char* regex); + + // We use a const char* instead of an std::string, as Google Test used to be + // used where std::string is not available. TODO(wan@google.com): change to + // std::string. + const char* pattern_; + bool is_valid_; + +#if GTEST_USES_POSIX_RE + + regex_t full_regex_; // For FullMatch(). + regex_t partial_regex_; // For PartialMatch(). + +#else // GTEST_USES_SIMPLE_RE + + const char* full_pattern_; // For FullMatch(); + +#endif + + GTEST_DISALLOW_ASSIGN_(RE); +}; + +// Formats a source file path and a line number as they would appear +// in an error message from the compiler used to compile this code. +GTEST_API_ ::std::string FormatFileLocation(const char* file, int line); + +// Formats a file location for compiler-independent XML output. +// Although this function is not platform dependent, we put it next to +// FormatFileLocation in order to contrast the two functions. +GTEST_API_ ::std::string FormatCompilerIndependentFileLocation(const char* file, + int line); + +// Defines logging utilities: +// GTEST_LOG_(severity) - logs messages at the specified severity level. The +// message itself is streamed into the macro. +// LogToStderr() - directs all log messages to stderr. +// FlushInfoLog() - flushes informational log messages. + +enum GTestLogSeverity { + GTEST_INFO, + GTEST_WARNING, + GTEST_ERROR, + GTEST_FATAL +}; + +// Formats log entry severity, provides a stream object for streaming the +// log message, and terminates the message with a newline when going out of +// scope. +class GTEST_API_ GTestLog { + public: + GTestLog(GTestLogSeverity severity, const char* file, int line); + + // Flushes the buffers and, if severity is GTEST_FATAL, aborts the program. + ~GTestLog(); + + ::std::ostream& GetStream() { return ::std::cerr; } + + private: + const GTestLogSeverity severity_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(GTestLog); +}; + +#define GTEST_LOG_(severity) \ + ::testing::internal::GTestLog(::testing::internal::GTEST_##severity, \ + __FILE__, __LINE__).GetStream() + +inline void LogToStderr() {} +inline void FlushInfoLog() { fflush(NULL); } + +// INTERNAL IMPLEMENTATION - DO NOT USE. +// +// GTEST_CHECK_ is an all-mode assert. It aborts the program if the condition +// is not satisfied. +// Synopsys: +// GTEST_CHECK_(boolean_condition); +// or +// GTEST_CHECK_(boolean_condition) << "Additional message"; +// +// This checks the condition and if the condition is not satisfied +// it prints message about the condition violation, including the +// condition itself, plus additional message streamed into it, if any, +// and then it aborts the program. It aborts the program irrespective of +// whether it is built in the debug mode or not. +#define GTEST_CHECK_(condition) \ + GTEST_AMBIGUOUS_ELSE_BLOCKER_ \ + if (::testing::internal::IsTrue(condition)) \ + ; \ + else \ + GTEST_LOG_(FATAL) << "Condition " #condition " failed. " + +// An all-mode assert to verify that the given POSIX-style function +// call returns 0 (indicating success). Known limitation: this +// doesn't expand to a balanced 'if' statement, so enclose the macro +// in {} if you need to use it as the only statement in an 'if' +// branch. +#define GTEST_CHECK_POSIX_SUCCESS_(posix_call) \ + if (const int gtest_error = (posix_call)) \ + GTEST_LOG_(FATAL) << #posix_call << "failed with error " \ + << gtest_error + +#if GTEST_HAS_STD_MOVE_ +using std::move; +#else // GTEST_HAS_STD_MOVE_ +template +const T& move(const T& t) { + return t; +} +#endif // GTEST_HAS_STD_MOVE_ + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// Use ImplicitCast_ as a safe version of static_cast for upcasting in +// the type hierarchy (e.g. casting a Foo* to a SuperclassOfFoo* or a +// const Foo*). When you use ImplicitCast_, the compiler checks that +// the cast is safe. Such explicit ImplicitCast_s are necessary in +// surprisingly many situations where C++ demands an exact type match +// instead of an argument type convertable to a target type. +// +// The syntax for using ImplicitCast_ is the same as for static_cast: +// +// ImplicitCast_(expr) +// +// ImplicitCast_ would have been part of the C++ standard library, +// but the proposal was submitted too late. It will probably make +// its way into the language in the future. +// +// This relatively ugly name is intentional. It prevents clashes with +// similar functions users may have (e.g., implicit_cast). The internal +// namespace alone is not enough because the function can be found by ADL. +template +inline To ImplicitCast_(To x) { return ::testing::internal::move(x); } + +// When you upcast (that is, cast a pointer from type Foo to type +// SuperclassOfFoo), it's fine to use ImplicitCast_<>, since upcasts +// always succeed. When you downcast (that is, cast a pointer from +// type Foo to type SubclassOfFoo), static_cast<> isn't safe, because +// how do you know the pointer is really of type SubclassOfFoo? It +// could be a bare Foo, or of type DifferentSubclassOfFoo. Thus, +// when you downcast, you should use this macro. In debug mode, we +// use dynamic_cast<> to double-check the downcast is legal (we die +// if it's not). In normal mode, we do the efficient static_cast<> +// instead. Thus, it's important to test in debug mode to make sure +// the cast is legal! +// This is the only place in the code we should use dynamic_cast<>. +// In particular, you SHOULDN'T be using dynamic_cast<> in order to +// do RTTI (eg code like this: +// if (dynamic_cast(foo)) HandleASubclass1Object(foo); +// if (dynamic_cast(foo)) HandleASubclass2Object(foo); +// You should design the code some other way not to need this. +// +// This relatively ugly name is intentional. It prevents clashes with +// similar functions users may have (e.g., down_cast). The internal +// namespace alone is not enough because the function can be found by ADL. +template // use like this: DownCast_(foo); +inline To DownCast_(From* f) { // so we only accept pointers + // Ensures that To is a sub-type of From *. This test is here only + // for compile-time type checking, and has no overhead in an + // optimized build at run-time, as it will be optimized away + // completely. + GTEST_INTENTIONAL_CONST_COND_PUSH_() + if (false) { + GTEST_INTENTIONAL_CONST_COND_POP_() + const To to = NULL; + ::testing::internal::ImplicitCast_(to); + } + +#if GTEST_HAS_RTTI + // RTTI: debug mode only! + GTEST_CHECK_(f == NULL || dynamic_cast(f) != NULL); +#endif + return static_cast(f); +} + +// Downcasts the pointer of type Base to Derived. +// Derived must be a subclass of Base. The parameter MUST +// point to a class of type Derived, not any subclass of it. +// When RTTI is available, the function performs a runtime +// check to enforce this. +template +Derived* CheckedDowncastToActualType(Base* base) { +#if GTEST_HAS_RTTI + GTEST_CHECK_(typeid(*base) == typeid(Derived)); + return dynamic_cast(base); // NOLINT +#else + return static_cast(base); // Poor man's downcast. +#endif +} + +#if GTEST_HAS_STREAM_REDIRECTION + +// Defines the stderr capturer: +// CaptureStdout - starts capturing stdout. +// GetCapturedStdout - stops capturing stdout and returns the captured string. +// CaptureStderr - starts capturing stderr. +// GetCapturedStderr - stops capturing stderr and returns the captured string. +// +GTEST_API_ void CaptureStdout(); +GTEST_API_ std::string GetCapturedStdout(); +GTEST_API_ void CaptureStderr(); +GTEST_API_ std::string GetCapturedStderr(); + +#endif // GTEST_HAS_STREAM_REDIRECTION + + +#if GTEST_HAS_DEATH_TEST + +const ::std::vector& GetInjectableArgvs(); +void SetInjectableArgvs(const ::std::vector* + new_argvs); + +// A copy of all command line arguments. Set by InitGoogleTest(). +extern ::std::vector g_argvs; + +#endif // GTEST_HAS_DEATH_TEST + +// Defines synchronization primitives. +#if GTEST_IS_THREADSAFE +# if GTEST_HAS_PTHREAD +// Sleeps for (roughly) n milliseconds. This function is only for testing +// Google Test's own constructs. Don't use it in user tests, either +// directly or indirectly. +inline void SleepMilliseconds(int n) { + const timespec time = { + 0, // 0 seconds. + n * 1000L * 1000L, // And n ms. + }; + nanosleep(&time, NULL); +} +# endif // GTEST_HAS_PTHREAD + +# if 0 // OS detection +# elif GTEST_HAS_PTHREAD +// Allows a controller thread to pause execution of newly created +// threads until notified. Instances of this class must be created +// and destroyed in the controller thread. +// +// This class is only for testing Google Test's own constructs. Do not +// use it in user tests, either directly or indirectly. +class Notification { + public: + Notification() : notified_(false) { + GTEST_CHECK_POSIX_SUCCESS_(pthread_mutex_init(&mutex_, NULL)); + } + ~Notification() { + pthread_mutex_destroy(&mutex_); + } + + // Notifies all threads created with this notification to start. Must + // be called from the controller thread. + void Notify() { + pthread_mutex_lock(&mutex_); + notified_ = true; + pthread_mutex_unlock(&mutex_); + } + + // Blocks until the controller thread notifies. Must be called from a test + // thread. + void WaitForNotification() { + for (;;) { + pthread_mutex_lock(&mutex_); + const bool notified = notified_; + pthread_mutex_unlock(&mutex_); + if (notified) + break; + SleepMilliseconds(10); + } + } + + private: + pthread_mutex_t mutex_; + bool notified_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(Notification); +}; + +# elif GTEST_OS_WINDOWS && !GTEST_OS_WINDOWS_PHONE && !GTEST_OS_WINDOWS_RT + +GTEST_API_ void SleepMilliseconds(int n); + +// Provides leak-safe Windows kernel handle ownership. +// Used in death tests and in threading support. +class GTEST_API_ AutoHandle { + public: + // Assume that Win32 HANDLE type is equivalent to void*. Doing so allows us to + // avoid including in this header file. Including is + // undesirable because it defines a lot of symbols and macros that tend to + // conflict with client code. This assumption is verified by + // WindowsTypesTest.HANDLEIsVoidStar. + typedef void* Handle; + AutoHandle(); + explicit AutoHandle(Handle handle); + + ~AutoHandle(); + + Handle Get() const; + void Reset(); + void Reset(Handle handle); + + private: + // Returns true iff the handle is a valid handle object that can be closed. + bool IsCloseable() const; + + Handle handle_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(AutoHandle); +}; + +// Allows a controller thread to pause execution of newly created +// threads until notified. Instances of this class must be created +// and destroyed in the controller thread. +// +// This class is only for testing Google Test's own constructs. Do not +// use it in user tests, either directly or indirectly. +class GTEST_API_ Notification { + public: + Notification(); + void Notify(); + void WaitForNotification(); + + private: + AutoHandle event_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(Notification); +}; +# endif // OS detection + +// On MinGW, we can have both GTEST_OS_WINDOWS and GTEST_HAS_PTHREAD +// defined, but we don't want to use MinGW's pthreads implementation, which +// has conformance problems with some versions of the POSIX standard. +# if GTEST_HAS_PTHREAD && !GTEST_OS_WINDOWS_MINGW + +// As a C-function, ThreadFuncWithCLinkage cannot be templated itself. +// Consequently, it cannot select a correct instantiation of ThreadWithParam +// in order to call its Run(). Introducing ThreadWithParamBase as a +// non-templated base class for ThreadWithParam allows us to bypass this +// problem. +class ThreadWithParamBase { + public: + virtual ~ThreadWithParamBase() {} + virtual void Run() = 0; +}; + +// pthread_create() accepts a pointer to a function type with the C linkage. +// According to the Standard (7.5/1), function types with different linkages +// are different even if they are otherwise identical. Some compilers (for +// example, SunStudio) treat them as different types. Since class methods +// cannot be defined with C-linkage we need to define a free C-function to +// pass into pthread_create(). +extern "C" inline void* ThreadFuncWithCLinkage(void* thread) { + static_cast(thread)->Run(); + return NULL; +} + +// Helper class for testing Google Test's multi-threading constructs. +// To use it, write: +// +// void ThreadFunc(int param) { /* Do things with param */ } +// Notification thread_can_start; +// ... +// // The thread_can_start parameter is optional; you can supply NULL. +// ThreadWithParam thread(&ThreadFunc, 5, &thread_can_start); +// thread_can_start.Notify(); +// +// These classes are only for testing Google Test's own constructs. Do +// not use them in user tests, either directly or indirectly. +template +class ThreadWithParam : public ThreadWithParamBase { + public: + typedef void UserThreadFunc(T); + + ThreadWithParam(UserThreadFunc* func, T param, Notification* thread_can_start) + : func_(func), + param_(param), + thread_can_start_(thread_can_start), + finished_(false) { + ThreadWithParamBase* const base = this; + // The thread can be created only after all fields except thread_ + // have been initialized. + GTEST_CHECK_POSIX_SUCCESS_( + pthread_create(&thread_, 0, &ThreadFuncWithCLinkage, base)); + } + ~ThreadWithParam() { Join(); } + + void Join() { + if (!finished_) { + GTEST_CHECK_POSIX_SUCCESS_(pthread_join(thread_, 0)); + finished_ = true; + } + } + + virtual void Run() { + if (thread_can_start_ != NULL) + thread_can_start_->WaitForNotification(); + func_(param_); + } + + private: + UserThreadFunc* const func_; // User-supplied thread function. + const T param_; // User-supplied parameter to the thread function. + // When non-NULL, used to block execution until the controller thread + // notifies. + Notification* const thread_can_start_; + bool finished_; // true iff we know that the thread function has finished. + pthread_t thread_; // The native thread object. + + GTEST_DISALLOW_COPY_AND_ASSIGN_(ThreadWithParam); +}; +# endif // GTEST_HAS_PTHREAD && !GTEST_OS_WINDOWS_MINGW + +# if 0 // OS detection +# elif GTEST_OS_WINDOWS && !GTEST_OS_WINDOWS_PHONE && !GTEST_OS_WINDOWS_RT + +// Mutex implements mutex on Windows platforms. It is used in conjunction +// with class MutexLock: +// +// Mutex mutex; +// ... +// MutexLock lock(&mutex); // Acquires the mutex and releases it at the +// // end of the current scope. +// +// A static Mutex *must* be defined or declared using one of the following +// macros: +// GTEST_DEFINE_STATIC_MUTEX_(g_some_mutex); +// GTEST_DECLARE_STATIC_MUTEX_(g_some_mutex); +// +// (A non-static Mutex is defined/declared in the usual way). +class GTEST_API_ Mutex { + public: + enum MutexType { kStatic = 0, kDynamic = 1 }; + // We rely on kStaticMutex being 0 as it is to what the linker initializes + // type_ in static mutexes. critical_section_ will be initialized lazily + // in ThreadSafeLazyInit(). + enum StaticConstructorSelector { kStaticMutex = 0 }; + + // This constructor intentionally does nothing. It relies on type_ being + // statically initialized to 0 (effectively setting it to kStatic) and on + // ThreadSafeLazyInit() to lazily initialize the rest of the members. + explicit Mutex(StaticConstructorSelector /*dummy*/) {} + + Mutex(); + ~Mutex(); + + void Lock(); + + void Unlock(); + + // Does nothing if the current thread holds the mutex. Otherwise, crashes + // with high probability. + void AssertHeld(); + + private: + // Initializes owner_thread_id_ and critical_section_ in static mutexes. + void ThreadSafeLazyInit(); + + // Per http://blogs.msdn.com/b/oldnewthing/archive/2004/02/23/78395.aspx, + // we assume that 0 is an invalid value for thread IDs. + unsigned int owner_thread_id_; + + // For static mutexes, we rely on these members being initialized to zeros + // by the linker. + MutexType type_; + long critical_section_init_phase_; // NOLINT + _RTL_CRITICAL_SECTION* critical_section_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(Mutex); +}; + +# define GTEST_DECLARE_STATIC_MUTEX_(mutex) \ + extern ::testing::internal::Mutex mutex + +# define GTEST_DEFINE_STATIC_MUTEX_(mutex) \ + ::testing::internal::Mutex mutex(::testing::internal::Mutex::kStaticMutex) + +// We cannot name this class MutexLock because the ctor declaration would +// conflict with a macro named MutexLock, which is defined on some +// platforms. That macro is used as a defensive measure to prevent against +// inadvertent misuses of MutexLock like "MutexLock(&mu)" rather than +// "MutexLock l(&mu)". Hence the typedef trick below. +class GTestMutexLock { + public: + explicit GTestMutexLock(Mutex* mutex) + : mutex_(mutex) { mutex_->Lock(); } + + ~GTestMutexLock() { mutex_->Unlock(); } + + private: + Mutex* const mutex_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(GTestMutexLock); +}; + +typedef GTestMutexLock MutexLock; + +// Base class for ValueHolder. Allows a caller to hold and delete a value +// without knowing its type. +class ThreadLocalValueHolderBase { + public: + virtual ~ThreadLocalValueHolderBase() {} +}; + +// Provides a way for a thread to send notifications to a ThreadLocal +// regardless of its parameter type. +class ThreadLocalBase { + public: + // Creates a new ValueHolder object holding a default value passed to + // this ThreadLocal's constructor and returns it. It is the caller's + // responsibility not to call this when the ThreadLocal instance already + // has a value on the current thread. + virtual ThreadLocalValueHolderBase* NewValueForCurrentThread() const = 0; + + protected: + ThreadLocalBase() {} + virtual ~ThreadLocalBase() {} + + private: + GTEST_DISALLOW_COPY_AND_ASSIGN_(ThreadLocalBase); +}; + +// Maps a thread to a set of ThreadLocals that have values instantiated on that +// thread and notifies them when the thread exits. A ThreadLocal instance is +// expected to persist until all threads it has values on have terminated. +class GTEST_API_ ThreadLocalRegistry { + public: + // Registers thread_local_instance as having value on the current thread. + // Returns a value that can be used to identify the thread from other threads. + static ThreadLocalValueHolderBase* GetValueOnCurrentThread( + const ThreadLocalBase* thread_local_instance); + + // Invoked when a ThreadLocal instance is destroyed. + static void OnThreadLocalDestroyed( + const ThreadLocalBase* thread_local_instance); +}; + +class GTEST_API_ ThreadWithParamBase { + public: + void Join(); + + protected: + class Runnable { + public: + virtual ~Runnable() {} + virtual void Run() = 0; + }; + + ThreadWithParamBase(Runnable *runnable, Notification* thread_can_start); + virtual ~ThreadWithParamBase(); + + private: + AutoHandle thread_; +}; + +// Helper class for testing Google Test's multi-threading constructs. +template +class ThreadWithParam : public ThreadWithParamBase { + public: + typedef void UserThreadFunc(T); + + ThreadWithParam(UserThreadFunc* func, T param, Notification* thread_can_start) + : ThreadWithParamBase(new RunnableImpl(func, param), thread_can_start) { + } + virtual ~ThreadWithParam() {} + + private: + class RunnableImpl : public Runnable { + public: + RunnableImpl(UserThreadFunc* func, T param) + : func_(func), + param_(param) { + } + virtual ~RunnableImpl() {} + virtual void Run() { + func_(param_); + } + + private: + UserThreadFunc* const func_; + const T param_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(RunnableImpl); + }; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(ThreadWithParam); +}; + +// Implements thread-local storage on Windows systems. +// +// // Thread 1 +// ThreadLocal tl(100); // 100 is the default value for each thread. +// +// // Thread 2 +// tl.set(150); // Changes the value for thread 2 only. +// EXPECT_EQ(150, tl.get()); +// +// // Thread 1 +// EXPECT_EQ(100, tl.get()); // In thread 1, tl has the original value. +// tl.set(200); +// EXPECT_EQ(200, tl.get()); +// +// The template type argument T must have a public copy constructor. +// In addition, the default ThreadLocal constructor requires T to have +// a public default constructor. +// +// The users of a TheadLocal instance have to make sure that all but one +// threads (including the main one) using that instance have exited before +// destroying it. Otherwise, the per-thread objects managed for them by the +// ThreadLocal instance are not guaranteed to be destroyed on all platforms. +// +// Google Test only uses global ThreadLocal objects. That means they +// will die after main() has returned. Therefore, no per-thread +// object managed by Google Test will be leaked as long as all threads +// using Google Test have exited when main() returns. +template +class ThreadLocal : public ThreadLocalBase { + public: + ThreadLocal() : default_() {} + explicit ThreadLocal(const T& value) : default_(value) {} + + ~ThreadLocal() { ThreadLocalRegistry::OnThreadLocalDestroyed(this); } + + T* pointer() { return GetOrCreateValue(); } + const T* pointer() const { return GetOrCreateValue(); } + const T& get() const { return *pointer(); } + void set(const T& value) { *pointer() = value; } + + private: + // Holds a value of T. Can be deleted via its base class without the caller + // knowing the type of T. + class ValueHolder : public ThreadLocalValueHolderBase { + public: + explicit ValueHolder(const T& value) : value_(value) {} + + T* pointer() { return &value_; } + + private: + T value_; + GTEST_DISALLOW_COPY_AND_ASSIGN_(ValueHolder); + }; + + + T* GetOrCreateValue() const { + return static_cast( + ThreadLocalRegistry::GetValueOnCurrentThread(this))->pointer(); + } + + virtual ThreadLocalValueHolderBase* NewValueForCurrentThread() const { + return new ValueHolder(default_); + } + + const T default_; // The default value for each thread. + + GTEST_DISALLOW_COPY_AND_ASSIGN_(ThreadLocal); +}; + +# elif GTEST_HAS_PTHREAD + +// MutexBase and Mutex implement mutex on pthreads-based platforms. +class MutexBase { + public: + // Acquires this mutex. + void Lock() { + GTEST_CHECK_POSIX_SUCCESS_(pthread_mutex_lock(&mutex_)); + owner_ = pthread_self(); + has_owner_ = true; + } + + // Releases this mutex. + void Unlock() { + // Since the lock is being released the owner_ field should no longer be + // considered valid. We don't protect writing to has_owner_ here, as it's + // the caller's responsibility to ensure that the current thread holds the + // mutex when this is called. + has_owner_ = false; + GTEST_CHECK_POSIX_SUCCESS_(pthread_mutex_unlock(&mutex_)); + } + + // Does nothing if the current thread holds the mutex. Otherwise, crashes + // with high probability. + void AssertHeld() const { + GTEST_CHECK_(has_owner_ && pthread_equal(owner_, pthread_self())) + << "The current thread is not holding the mutex @" << this; + } + + // A static mutex may be used before main() is entered. It may even + // be used before the dynamic initialization stage. Therefore we + // must be able to initialize a static mutex object at link time. + // This means MutexBase has to be a POD and its member variables + // have to be public. + public: + pthread_mutex_t mutex_; // The underlying pthread mutex. + // has_owner_ indicates whether the owner_ field below contains a valid thread + // ID and is therefore safe to inspect (e.g., to use in pthread_equal()). All + // accesses to the owner_ field should be protected by a check of this field. + // An alternative might be to memset() owner_ to all zeros, but there's no + // guarantee that a zero'd pthread_t is necessarily invalid or even different + // from pthread_self(). + bool has_owner_; + pthread_t owner_; // The thread holding the mutex. +}; + +// Forward-declares a static mutex. +# define GTEST_DECLARE_STATIC_MUTEX_(mutex) \ + extern ::testing::internal::MutexBase mutex + +// Defines and statically (i.e. at link time) initializes a static mutex. +// The initialization list here does not explicitly initialize each field, +// instead relying on default initialization for the unspecified fields. In +// particular, the owner_ field (a pthread_t) is not explicitly initialized. +// This allows initialization to work whether pthread_t is a scalar or struct. +// The flag -Wmissing-field-initializers must not be specified for this to work. +# define GTEST_DEFINE_STATIC_MUTEX_(mutex) \ + ::testing::internal::MutexBase mutex = { PTHREAD_MUTEX_INITIALIZER, false } + +// The Mutex class can only be used for mutexes created at runtime. It +// shares its API with MutexBase otherwise. +class Mutex : public MutexBase { + public: + Mutex() { + GTEST_CHECK_POSIX_SUCCESS_(pthread_mutex_init(&mutex_, NULL)); + has_owner_ = false; + } + ~Mutex() { + GTEST_CHECK_POSIX_SUCCESS_(pthread_mutex_destroy(&mutex_)); + } + + private: + GTEST_DISALLOW_COPY_AND_ASSIGN_(Mutex); +}; + +// We cannot name this class MutexLock because the ctor declaration would +// conflict with a macro named MutexLock, which is defined on some +// platforms. That macro is used as a defensive measure to prevent against +// inadvertent misuses of MutexLock like "MutexLock(&mu)" rather than +// "MutexLock l(&mu)". Hence the typedef trick below. +class GTestMutexLock { + public: + explicit GTestMutexLock(MutexBase* mutex) + : mutex_(mutex) { mutex_->Lock(); } + + ~GTestMutexLock() { mutex_->Unlock(); } + + private: + MutexBase* const mutex_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(GTestMutexLock); +}; + +typedef GTestMutexLock MutexLock; + +// Helpers for ThreadLocal. + +// pthread_key_create() requires DeleteThreadLocalValue() to have +// C-linkage. Therefore it cannot be templatized to access +// ThreadLocal. Hence the need for class +// ThreadLocalValueHolderBase. +class ThreadLocalValueHolderBase { + public: + virtual ~ThreadLocalValueHolderBase() {} +}; + +// Called by pthread to delete thread-local data stored by +// pthread_setspecific(). +extern "C" inline void DeleteThreadLocalValue(void* value_holder) { + delete static_cast(value_holder); +} + +// Implements thread-local storage on pthreads-based systems. +template +class ThreadLocal { + public: + ThreadLocal() : key_(CreateKey()), + default_() {} + explicit ThreadLocal(const T& value) : key_(CreateKey()), + default_(value) {} + + ~ThreadLocal() { + // Destroys the managed object for the current thread, if any. + DeleteThreadLocalValue(pthread_getspecific(key_)); + + // Releases resources associated with the key. This will *not* + // delete managed objects for other threads. + GTEST_CHECK_POSIX_SUCCESS_(pthread_key_delete(key_)); + } + + T* pointer() { return GetOrCreateValue(); } + const T* pointer() const { return GetOrCreateValue(); } + const T& get() const { return *pointer(); } + void set(const T& value) { *pointer() = value; } + + private: + // Holds a value of type T. + class ValueHolder : public ThreadLocalValueHolderBase { + public: + explicit ValueHolder(const T& value) : value_(value) {} + + T* pointer() { return &value_; } + + private: + T value_; + GTEST_DISALLOW_COPY_AND_ASSIGN_(ValueHolder); + }; + + static pthread_key_t CreateKey() { + pthread_key_t key; + // When a thread exits, DeleteThreadLocalValue() will be called on + // the object managed for that thread. + GTEST_CHECK_POSIX_SUCCESS_( + pthread_key_create(&key, &DeleteThreadLocalValue)); + return key; + } + + T* GetOrCreateValue() const { + ThreadLocalValueHolderBase* const holder = + static_cast(pthread_getspecific(key_)); + if (holder != NULL) { + return CheckedDowncastToActualType(holder)->pointer(); + } + + ValueHolder* const new_holder = new ValueHolder(default_); + ThreadLocalValueHolderBase* const holder_base = new_holder; + GTEST_CHECK_POSIX_SUCCESS_(pthread_setspecific(key_, holder_base)); + return new_holder->pointer(); + } + + // A key pthreads uses for looking up per-thread values. + const pthread_key_t key_; + const T default_; // The default value for each thread. + + GTEST_DISALLOW_COPY_AND_ASSIGN_(ThreadLocal); +}; + +# endif // OS detection + +#else // GTEST_IS_THREADSAFE + +// A dummy implementation of synchronization primitives (mutex, lock, +// and thread-local variable). Necessary for compiling Google Test where +// mutex is not supported - using Google Test in multiple threads is not +// supported on such platforms. + +class Mutex { + public: + Mutex() {} + void Lock() {} + void Unlock() {} + void AssertHeld() const {} +}; + +# define GTEST_DECLARE_STATIC_MUTEX_(mutex) \ + extern ::testing::internal::Mutex mutex + +# define GTEST_DEFINE_STATIC_MUTEX_(mutex) ::testing::internal::Mutex mutex + +// We cannot name this class MutexLock because the ctor declaration would +// conflict with a macro named MutexLock, which is defined on some +// platforms. That macro is used as a defensive measure to prevent against +// inadvertent misuses of MutexLock like "MutexLock(&mu)" rather than +// "MutexLock l(&mu)". Hence the typedef trick below. +class GTestMutexLock { + public: + explicit GTestMutexLock(Mutex*) {} // NOLINT +}; + +typedef GTestMutexLock MutexLock; + +template +class ThreadLocal { + public: + ThreadLocal() : value_() {} + explicit ThreadLocal(const T& value) : value_(value) {} + T* pointer() { return &value_; } + const T* pointer() const { return &value_; } + const T& get() const { return value_; } + void set(const T& value) { value_ = value; } + private: + T value_; +}; + +#endif // GTEST_IS_THREADSAFE + +// Returns the number of threads running in the process, or 0 to indicate that +// we cannot detect it. +GTEST_API_ size_t GetThreadCount(); + +// Passing non-POD classes through ellipsis (...) crashes the ARM +// compiler and generates a warning in Sun Studio. The Nokia Symbian +// and the IBM XL C/C++ compiler try to instantiate a copy constructor +// for objects passed through ellipsis (...), failing for uncopyable +// objects. We define this to ensure that only POD is passed through +// ellipsis on these systems. +#if defined(__SYMBIAN32__) || defined(__IBMCPP__) || defined(__SUNPRO_CC) +// We lose support for NULL detection where the compiler doesn't like +// passing non-POD classes through ellipsis (...). +# define GTEST_ELLIPSIS_NEEDS_POD_ 1 +#else +# define GTEST_CAN_COMPARE_NULL 1 +#endif + +// The Nokia Symbian and IBM XL C/C++ compilers cannot decide between +// const T& and const T* in a function template. These compilers +// _can_ decide between class template specializations for T and T*, +// so a tr1::type_traits-like is_pointer works. +#if defined(__SYMBIAN32__) || defined(__IBMCPP__) +# define GTEST_NEEDS_IS_POINTER_ 1 +#endif + +template +struct bool_constant { + typedef bool_constant type; + static const bool value = bool_value; +}; +template const bool bool_constant::value; + +typedef bool_constant false_type; +typedef bool_constant true_type; + +template +struct is_pointer : public false_type {}; + +template +struct is_pointer : public true_type {}; + +template +struct IteratorTraits { + typedef typename Iterator::value_type value_type; +}; + +template +struct IteratorTraits { + typedef T value_type; +}; + +template +struct IteratorTraits { + typedef T value_type; +}; + +#if GTEST_OS_WINDOWS +# define GTEST_PATH_SEP_ "\\" +# define GTEST_HAS_ALT_PATH_SEP_ 1 +// The biggest signed integer type the compiler supports. +typedef __int64 BiggestInt; +#else +# define GTEST_PATH_SEP_ "/" +# define GTEST_HAS_ALT_PATH_SEP_ 0 +typedef long long BiggestInt; // NOLINT +#endif // GTEST_OS_WINDOWS + +// Utilities for char. + +// isspace(int ch) and friends accept an unsigned char or EOF. char +// may be signed, depending on the compiler (or compiler flags). +// Therefore we need to cast a char to unsigned char before calling +// isspace(), etc. + +inline bool IsAlpha(char ch) { + return isalpha(static_cast(ch)) != 0; +} +inline bool IsAlNum(char ch) { + return isalnum(static_cast(ch)) != 0; +} +inline bool IsDigit(char ch) { + return isdigit(static_cast(ch)) != 0; +} +inline bool IsLower(char ch) { + return islower(static_cast(ch)) != 0; +} +inline bool IsSpace(char ch) { + return isspace(static_cast(ch)) != 0; +} +inline bool IsUpper(char ch) { + return isupper(static_cast(ch)) != 0; +} +inline bool IsXDigit(char ch) { + return isxdigit(static_cast(ch)) != 0; +} +inline bool IsXDigit(wchar_t ch) { + const unsigned char low_byte = static_cast(ch); + return ch == low_byte && isxdigit(low_byte) != 0; +} + +inline char ToLower(char ch) { + return static_cast(tolower(static_cast(ch))); +} +inline char ToUpper(char ch) { + return static_cast(toupper(static_cast(ch))); +} + +inline std::string StripTrailingSpaces(std::string str) { + std::string::iterator it = str.end(); + while (it != str.begin() && IsSpace(*--it)) + it = str.erase(it); + return str; +} + +// The testing::internal::posix namespace holds wrappers for common +// POSIX functions. These wrappers hide the differences between +// Windows/MSVC and POSIX systems. Since some compilers define these +// standard functions as macros, the wrapper cannot have the same name +// as the wrapped function. + +namespace posix { + +// Functions with a different name on Windows. + +#if GTEST_OS_WINDOWS + +typedef struct _stat StatStruct; + +# ifdef __BORLANDC__ +inline int IsATTY(int fd) { return isatty(fd); } +inline int StrCaseCmp(const char* s1, const char* s2) { + return stricmp(s1, s2); +} +inline char* StrDup(const char* src) { return strdup(src); } +# else // !__BORLANDC__ +# if GTEST_OS_WINDOWS_MOBILE +inline int IsATTY(int /* fd */) { return 0; } +# else +inline int IsATTY(int fd) { return _isatty(fd); } +# endif // GTEST_OS_WINDOWS_MOBILE +inline int StrCaseCmp(const char* s1, const char* s2) { + return _stricmp(s1, s2); +} +inline char* StrDup(const char* src) { return _strdup(src); } +# endif // __BORLANDC__ + +# if GTEST_OS_WINDOWS_MOBILE +inline int FileNo(FILE* file) { return reinterpret_cast(_fileno(file)); } +// Stat(), RmDir(), and IsDir() are not needed on Windows CE at this +// time and thus not defined there. +# else +inline int FileNo(FILE* file) { return _fileno(file); } +inline int Stat(const char* path, StatStruct* buf) { return _stat(path, buf); } +inline int RmDir(const char* dir) { return _rmdir(dir); } +inline bool IsDir(const StatStruct& st) { + return (_S_IFDIR & st.st_mode) != 0; +} +# endif // GTEST_OS_WINDOWS_MOBILE + +#else + +typedef struct stat StatStruct; + +inline int FileNo(FILE* file) { return fileno(file); } +inline int IsATTY(int fd) { return isatty(fd); } +inline int Stat(const char* path, StatStruct* buf) { return stat(path, buf); } +inline int StrCaseCmp(const char* s1, const char* s2) { + return strcasecmp(s1, s2); +} +inline char* StrDup(const char* src) { return strdup(src); } +inline int RmDir(const char* dir) { return rmdir(dir); } +inline bool IsDir(const StatStruct& st) { return S_ISDIR(st.st_mode); } + +#endif // GTEST_OS_WINDOWS + +// Functions deprecated by MSVC 8.0. + +GTEST_DISABLE_MSC_WARNINGS_PUSH_(4996 /* deprecated function */) + +inline const char* StrNCpy(char* dest, const char* src, size_t n) { + return strncpy(dest, src, n); +} + +// ChDir(), FReopen(), FDOpen(), Read(), Write(), Close(), and +// StrError() aren't needed on Windows CE at this time and thus not +// defined there. + +#if !GTEST_OS_WINDOWS_MOBILE && !GTEST_OS_WINDOWS_PHONE && !GTEST_OS_WINDOWS_RT +inline int ChDir(const char* dir) { return chdir(dir); } +#endif +inline FILE* FOpen(const char* path, const char* mode) { + return fopen(path, mode); +} +#if !GTEST_OS_WINDOWS_MOBILE +inline FILE *FReopen(const char* path, const char* mode, FILE* stream) { + return freopen(path, mode, stream); +} +inline FILE* FDOpen(int fd, const char* mode) { return fdopen(fd, mode); } +#endif +inline int FClose(FILE* fp) { return fclose(fp); } +#if !GTEST_OS_WINDOWS_MOBILE +inline int Read(int fd, void* buf, unsigned int count) { + return static_cast(read(fd, buf, count)); +} +inline int Write(int fd, const void* buf, unsigned int count) { + return static_cast(write(fd, buf, count)); +} +inline int Close(int fd) { return close(fd); } +inline const char* StrError(int errnum) { return strerror(errnum); } +#endif +inline const char* GetEnv(const char* name) { +#if GTEST_OS_WINDOWS_MOBILE || GTEST_OS_WINDOWS_PHONE | GTEST_OS_WINDOWS_RT + // We are on Windows CE, which has no environment variables. + static_cast(name); // To prevent 'unused argument' warning. + return NULL; +#elif defined(__BORLANDC__) || defined(__SunOS_5_8) || defined(__SunOS_5_9) + // Environment variables which we programmatically clear will be set to the + // empty string rather than unset (NULL). Handle that case. + const char* const env = getenv(name); + return (env != NULL && env[0] != '\0') ? env : NULL; +#else + return getenv(name); +#endif +} + +GTEST_DISABLE_MSC_WARNINGS_POP_() + +#if GTEST_OS_WINDOWS_MOBILE +// Windows CE has no C library. The abort() function is used in +// several places in Google Test. This implementation provides a reasonable +// imitation of standard behaviour. +void Abort(); +#else +inline void Abort() { abort(); } +#endif // GTEST_OS_WINDOWS_MOBILE + +} // namespace posix + +// MSVC "deprecates" snprintf and issues warnings wherever it is used. In +// order to avoid these warnings, we need to use _snprintf or _snprintf_s on +// MSVC-based platforms. We map the GTEST_SNPRINTF_ macro to the appropriate +// function in order to achieve that. We use macro definition here because +// snprintf is a variadic function. +#if _MSC_VER >= 1400 && !GTEST_OS_WINDOWS_MOBILE +// MSVC 2005 and above support variadic macros. +# define GTEST_SNPRINTF_(buffer, size, format, ...) \ + _snprintf_s(buffer, size, size, format, __VA_ARGS__) +#elif defined(_MSC_VER) +// Windows CE does not define _snprintf_s and MSVC prior to 2005 doesn't +// complain about _snprintf. +# define GTEST_SNPRINTF_ _snprintf +#else +# define GTEST_SNPRINTF_ snprintf +#endif + +// The maximum number a BiggestInt can represent. This definition +// works no matter BiggestInt is represented in one's complement or +// two's complement. +// +// We cannot rely on numeric_limits in STL, as __int64 and long long +// are not part of standard C++ and numeric_limits doesn't need to be +// defined for them. +const BiggestInt kMaxBiggestInt = + ~(static_cast(1) << (8*sizeof(BiggestInt) - 1)); + +// This template class serves as a compile-time function from size to +// type. It maps a size in bytes to a primitive type with that +// size. e.g. +// +// TypeWithSize<4>::UInt +// +// is typedef-ed to be unsigned int (unsigned integer made up of 4 +// bytes). +// +// Such functionality should belong to STL, but I cannot find it +// there. +// +// Google Test uses this class in the implementation of floating-point +// comparison. +// +// For now it only handles UInt (unsigned int) as that's all Google Test +// needs. Other types can be easily added in the future if need +// arises. +template +class TypeWithSize { + public: + // This prevents the user from using TypeWithSize with incorrect + // values of N. + typedef void UInt; +}; + +// The specialization for size 4. +template <> +class TypeWithSize<4> { + public: + // unsigned int has size 4 in both gcc and MSVC. + // + // As base/basictypes.h doesn't compile on Windows, we cannot use + // uint32, uint64, and etc here. + typedef int Int; + typedef unsigned int UInt; +}; + +// The specialization for size 8. +template <> +class TypeWithSize<8> { + public: +#if GTEST_OS_WINDOWS + typedef __int64 Int; + typedef unsigned __int64 UInt; +#else + typedef long long Int; // NOLINT + typedef unsigned long long UInt; // NOLINT +#endif // GTEST_OS_WINDOWS +}; + +// Integer types of known sizes. +typedef TypeWithSize<4>::Int Int32; +typedef TypeWithSize<4>::UInt UInt32; +typedef TypeWithSize<8>::Int Int64; +typedef TypeWithSize<8>::UInt UInt64; +typedef TypeWithSize<8>::Int TimeInMillis; // Represents time in milliseconds. + +// Utilities for command line flags and environment variables. + +// Macro for referencing flags. +#define GTEST_FLAG(name) FLAGS_gtest_##name + +// Macros for declaring flags. +#define GTEST_DECLARE_bool_(name) GTEST_API_ extern bool GTEST_FLAG(name) +#define GTEST_DECLARE_int32_(name) \ + GTEST_API_ extern ::testing::internal::Int32 GTEST_FLAG(name) +#define GTEST_DECLARE_string_(name) \ + GTEST_API_ extern ::std::string GTEST_FLAG(name) + +// Macros for defining flags. +#define GTEST_DEFINE_bool_(name, default_val, doc) \ + GTEST_API_ bool GTEST_FLAG(name) = (default_val) +#define GTEST_DEFINE_int32_(name, default_val, doc) \ + GTEST_API_ ::testing::internal::Int32 GTEST_FLAG(name) = (default_val) +#define GTEST_DEFINE_string_(name, default_val, doc) \ + GTEST_API_ ::std::string GTEST_FLAG(name) = (default_val) + +// Thread annotations +#define GTEST_EXCLUSIVE_LOCK_REQUIRED_(locks) +#define GTEST_LOCK_EXCLUDED_(locks) + +// Parses 'str' for a 32-bit signed integer. If successful, writes the result +// to *value and returns true; otherwise leaves *value unchanged and returns +// false. +// TODO(chandlerc): Find a better way to refactor flag and environment parsing +// out of both gtest-port.cc and gtest.cc to avoid exporting this utility +// function. +bool ParseInt32(const Message& src_text, const char* str, Int32* value); + +// Parses a bool/Int32/string from the environment variable +// corresponding to the given Google Test flag. +bool BoolFromGTestEnv(const char* flag, bool default_val); +GTEST_API_ Int32 Int32FromGTestEnv(const char* flag, Int32 default_val); +const char* StringFromGTestEnv(const char* flag, const char* default_val); + +} // namespace internal +} // namespace testing + +#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PORT_H_ + + +#if GTEST_OS_LINUX +# include +# include +# include +# include +#endif // GTEST_OS_LINUX + +#if GTEST_HAS_EXCEPTIONS +# include +#endif + +#include +#include +#include +#include +#include +#include +#include +#include + +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) +// +// The Google C++ Testing Framework (Google Test) +// +// This header file defines the Message class. +// +// IMPORTANT NOTE: Due to limitation of the C++ language, we have to +// leave some internal implementation details in this header file. +// They are clearly marked by comments like this: +// +// // INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +// +// Such code is NOT meant to be used by a user directly, and is subject +// to CHANGE WITHOUT NOTICE. Therefore DO NOT DEPEND ON IT in a user +// program! + +#ifndef GTEST_INCLUDE_GTEST_GTEST_MESSAGE_H_ +#define GTEST_INCLUDE_GTEST_GTEST_MESSAGE_H_ + +#include + + +// Ensures that there is at least one operator<< in the global namespace. +// See Message& operator<<(...) below for why. +void operator<<(const testing::internal::Secret&, int); + +namespace testing { + +// The Message class works like an ostream repeater. +// +// Typical usage: +// +// 1. You stream a bunch of values to a Message object. +// It will remember the text in a stringstream. +// 2. Then you stream the Message object to an ostream. +// This causes the text in the Message to be streamed +// to the ostream. +// +// For example; +// +// testing::Message foo; +// foo << 1 << " != " << 2; +// std::cout << foo; +// +// will print "1 != 2". +// +// Message is not intended to be inherited from. In particular, its +// destructor is not virtual. +// +// Note that stringstream behaves differently in gcc and in MSVC. You +// can stream a NULL char pointer to it in the former, but not in the +// latter (it causes an access violation if you do). The Message +// class hides this difference by treating a NULL char pointer as +// "(null)". +class GTEST_API_ Message { + private: + // The type of basic IO manipulators (endl, ends, and flush) for + // narrow streams. + typedef std::ostream& (*BasicNarrowIoManip)(std::ostream&); + + public: + // Constructs an empty Message. + Message(); + + // Copy constructor. + Message(const Message& msg) : ss_(new ::std::stringstream) { // NOLINT + *ss_ << msg.GetString(); + } + + // Constructs a Message from a C-string. + explicit Message(const char* str) : ss_(new ::std::stringstream) { + *ss_ << str; + } + +#if GTEST_OS_SYMBIAN + // Streams a value (either a pointer or not) to this object. + template + inline Message& operator <<(const T& value) { + StreamHelper(typename internal::is_pointer::type(), value); + return *this; + } +#else + // Streams a non-pointer value to this object. + template + inline Message& operator <<(const T& val) { + // Some libraries overload << for STL containers. These + // overloads are defined in the global namespace instead of ::std. + // + // C++'s symbol lookup rule (i.e. Koenig lookup) says that these + // overloads are visible in either the std namespace or the global + // namespace, but not other namespaces, including the testing + // namespace which Google Test's Message class is in. + // + // To allow STL containers (and other types that has a << operator + // defined in the global namespace) to be used in Google Test + // assertions, testing::Message must access the custom << operator + // from the global namespace. With this using declaration, + // overloads of << defined in the global namespace and those + // visible via Koenig lookup are both exposed in this function. + using ::operator <<; + *ss_ << val; + return *this; + } + + // Streams a pointer value to this object. + // + // This function is an overload of the previous one. When you + // stream a pointer to a Message, this definition will be used as it + // is more specialized. (The C++ Standard, section + // [temp.func.order].) If you stream a non-pointer, then the + // previous definition will be used. + // + // The reason for this overload is that streaming a NULL pointer to + // ostream is undefined behavior. Depending on the compiler, you + // may get "0", "(nil)", "(null)", or an access violation. To + // ensure consistent result across compilers, we always treat NULL + // as "(null)". + template + inline Message& operator <<(T* const& pointer) { // NOLINT + if (pointer == NULL) { + *ss_ << "(null)"; + } else { + *ss_ << pointer; + } + return *this; + } +#endif // GTEST_OS_SYMBIAN + + // Since the basic IO manipulators are overloaded for both narrow + // and wide streams, we have to provide this specialized definition + // of operator <<, even though its body is the same as the + // templatized version above. Without this definition, streaming + // endl or other basic IO manipulators to Message will confuse the + // compiler. + Message& operator <<(BasicNarrowIoManip val) { + *ss_ << val; + return *this; + } + + // Instead of 1/0, we want to see true/false for bool values. + Message& operator <<(bool b) { + return *this << (b ? "true" : "false"); + } + + // These two overloads allow streaming a wide C string to a Message + // using the UTF-8 encoding. + Message& operator <<(const wchar_t* wide_c_str); + Message& operator <<(wchar_t* wide_c_str); + +#if GTEST_HAS_STD_WSTRING + // Converts the given wide string to a narrow string using the UTF-8 + // encoding, and streams the result to this Message object. + Message& operator <<(const ::std::wstring& wstr); +#endif // GTEST_HAS_STD_WSTRING + +#if GTEST_HAS_GLOBAL_WSTRING + // Converts the given wide string to a narrow string using the UTF-8 + // encoding, and streams the result to this Message object. + Message& operator <<(const ::wstring& wstr); +#endif // GTEST_HAS_GLOBAL_WSTRING + + // Gets the text streamed to this object so far as an std::string. + // Each '\0' character in the buffer is replaced with "\\0". + // + // INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. + std::string GetString() const; + + private: + +#if GTEST_OS_SYMBIAN + // These are needed as the Nokia Symbian Compiler cannot decide between + // const T& and const T* in a function template. The Nokia compiler _can_ + // decide between class template specializations for T and T*, so a + // tr1::type_traits-like is_pointer works, and we can overload on that. + template + inline void StreamHelper(internal::true_type /*is_pointer*/, T* pointer) { + if (pointer == NULL) { + *ss_ << "(null)"; + } else { + *ss_ << pointer; + } + } + template + inline void StreamHelper(internal::false_type /*is_pointer*/, + const T& value) { + // See the comments in Message& operator <<(const T&) above for why + // we need this using statement. + using ::operator <<; + *ss_ << value; + } +#endif // GTEST_OS_SYMBIAN + + // We'll hold the text streamed to this object here. + const internal::scoped_ptr< ::std::stringstream> ss_; + + // We declare (but don't implement) this to prevent the compiler + // from implementing the assignment operator. + void operator=(const Message&); +}; + +// Streams a Message to an ostream. +inline std::ostream& operator <<(std::ostream& os, const Message& sb) { + return os << sb.GetString(); +} + +namespace internal { + +// Converts a streamable value to an std::string. A NULL pointer is +// converted to "(null)". When the input value is a ::string, +// ::std::string, ::wstring, or ::std::wstring object, each NUL +// character in it is replaced with "\\0". +template +std::string StreamableToString(const T& streamable) { + return (Message() << streamable).GetString(); +} + +} // namespace internal +} // namespace testing + +#endif // GTEST_INCLUDE_GTEST_GTEST_MESSAGE_H_ +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Authors: wan@google.com (Zhanyong Wan), eefacm@gmail.com (Sean Mcafee) +// +// The Google C++ Testing Framework (Google Test) +// +// This header file declares the String class and functions used internally by +// Google Test. They are subject to change without notice. They should not used +// by code external to Google Test. +// +// This header file is #included by . +// It should not be #included by other files. + +#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_STRING_H_ +#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_STRING_H_ + +#ifdef __BORLANDC__ +// string.h is not guaranteed to provide strcpy on C++ Builder. +# include +#endif + +#include +#include + + +namespace testing { +namespace internal { + +// String - an abstract class holding static string utilities. +class GTEST_API_ String { + public: + // Static utility methods + + // Clones a 0-terminated C string, allocating memory using new. The + // caller is responsible for deleting the return value using + // delete[]. Returns the cloned string, or NULL if the input is + // NULL. + // + // This is different from strdup() in string.h, which allocates + // memory using malloc(). + static const char* CloneCString(const char* c_str); + +#if GTEST_OS_WINDOWS_MOBILE + // Windows CE does not have the 'ANSI' versions of Win32 APIs. To be + // able to pass strings to Win32 APIs on CE we need to convert them + // to 'Unicode', UTF-16. + + // Creates a UTF-16 wide string from the given ANSI string, allocating + // memory using new. The caller is responsible for deleting the return + // value using delete[]. Returns the wide string, or NULL if the + // input is NULL. + // + // The wide string is created using the ANSI codepage (CP_ACP) to + // match the behaviour of the ANSI versions of Win32 calls and the + // C runtime. + static LPCWSTR AnsiToUtf16(const char* c_str); + + // Creates an ANSI string from the given wide string, allocating + // memory using new. The caller is responsible for deleting the return + // value using delete[]. Returns the ANSI string, or NULL if the + // input is NULL. + // + // The returned string is created using the ANSI codepage (CP_ACP) to + // match the behaviour of the ANSI versions of Win32 calls and the + // C runtime. + static const char* Utf16ToAnsi(LPCWSTR utf16_str); +#endif + + // Compares two C strings. Returns true iff they have the same content. + // + // Unlike strcmp(), this function can handle NULL argument(s). A + // NULL C string is considered different to any non-NULL C string, + // including the empty string. + static bool CStringEquals(const char* lhs, const char* rhs); + + // Converts a wide C string to a String using the UTF-8 encoding. + // NULL will be converted to "(null)". If an error occurred during + // the conversion, "(failed to convert from wide string)" is + // returned. + static std::string ShowWideCString(const wchar_t* wide_c_str); + + // Compares two wide C strings. Returns true iff they have the same + // content. + // + // Unlike wcscmp(), this function can handle NULL argument(s). A + // NULL C string is considered different to any non-NULL C string, + // including the empty string. + static bool WideCStringEquals(const wchar_t* lhs, const wchar_t* rhs); + + // Compares two C strings, ignoring case. Returns true iff they + // have the same content. + // + // Unlike strcasecmp(), this function can handle NULL argument(s). + // A NULL C string is considered different to any non-NULL C string, + // including the empty string. + static bool CaseInsensitiveCStringEquals(const char* lhs, + const char* rhs); + + // Compares two wide C strings, ignoring case. Returns true iff they + // have the same content. + // + // Unlike wcscasecmp(), this function can handle NULL argument(s). + // A NULL C string is considered different to any non-NULL wide C string, + // including the empty string. + // NB: The implementations on different platforms slightly differ. + // On windows, this method uses _wcsicmp which compares according to LC_CTYPE + // environment variable. On GNU platform this method uses wcscasecmp + // which compares according to LC_CTYPE category of the current locale. + // On MacOS X, it uses towlower, which also uses LC_CTYPE category of the + // current locale. + static bool CaseInsensitiveWideCStringEquals(const wchar_t* lhs, + const wchar_t* rhs); + + // Returns true iff the given string ends with the given suffix, ignoring + // case. Any string is considered to end with an empty suffix. + static bool EndsWithCaseInsensitive( + const std::string& str, const std::string& suffix); + + // Formats an int value as "%02d". + static std::string FormatIntWidth2(int value); // "%02d" for width == 2 + + // Formats an int value as "%X". + static std::string FormatHexInt(int value); + + // Formats a byte as "%02X". + static std::string FormatByte(unsigned char value); + + private: + String(); // Not meant to be instantiated. +}; // class String + +// Gets the content of the stringstream's buffer as an std::string. Each '\0' +// character in the buffer is replaced with "\\0". +GTEST_API_ std::string StringStreamToString(::std::stringstream* stream); + +} // namespace internal +} // namespace testing + +#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_STRING_H_ +// Copyright 2008, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: keith.ray@gmail.com (Keith Ray) +// +// Google Test filepath utilities +// +// This header file declares classes and functions used internally by +// Google Test. They are subject to change without notice. +// +// This file is #included in . +// Do not include this header file separately! + +#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_FILEPATH_H_ +#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_FILEPATH_H_ + + +namespace testing { +namespace internal { + +// FilePath - a class for file and directory pathname manipulation which +// handles platform-specific conventions (like the pathname separator). +// Used for helper functions for naming files in a directory for xml output. +// Except for Set methods, all methods are const or static, which provides an +// "immutable value object" -- useful for peace of mind. +// A FilePath with a value ending in a path separator ("like/this/") represents +// a directory, otherwise it is assumed to represent a file. In either case, +// it may or may not represent an actual file or directory in the file system. +// Names are NOT checked for syntax correctness -- no checking for illegal +// characters, malformed paths, etc. + +class GTEST_API_ FilePath { + public: + FilePath() : pathname_("") { } + FilePath(const FilePath& rhs) : pathname_(rhs.pathname_) { } + + explicit FilePath(const std::string& pathname) : pathname_(pathname) { + Normalize(); + } + + FilePath& operator=(const FilePath& rhs) { + Set(rhs); + return *this; + } + + void Set(const FilePath& rhs) { + pathname_ = rhs.pathname_; + } + + const std::string& string() const { return pathname_; } + const char* c_str() const { return pathname_.c_str(); } + + // Returns the current working directory, or "" if unsuccessful. + static FilePath GetCurrentDir(); + + // Given directory = "dir", base_name = "test", number = 0, + // extension = "xml", returns "dir/test.xml". If number is greater + // than zero (e.g., 12), returns "dir/test_12.xml". + // On Windows platform, uses \ as the separator rather than /. + static FilePath MakeFileName(const FilePath& directory, + const FilePath& base_name, + int number, + const char* extension); + + // Given directory = "dir", relative_path = "test.xml", + // returns "dir/test.xml". + // On Windows, uses \ as the separator rather than /. + static FilePath ConcatPaths(const FilePath& directory, + const FilePath& relative_path); + + // Returns a pathname for a file that does not currently exist. The pathname + // will be directory/base_name.extension or + // directory/base_name_.extension if directory/base_name.extension + // already exists. The number will be incremented until a pathname is found + // that does not already exist. + // Examples: 'dir/foo_test.xml' or 'dir/foo_test_1.xml'. + // There could be a race condition if two or more processes are calling this + // function at the same time -- they could both pick the same filename. + static FilePath GenerateUniqueFileName(const FilePath& directory, + const FilePath& base_name, + const char* extension); + + // Returns true iff the path is "". + bool IsEmpty() const { return pathname_.empty(); } + + // If input name has a trailing separator character, removes it and returns + // the name, otherwise return the name string unmodified. + // On Windows platform, uses \ as the separator, other platforms use /. + FilePath RemoveTrailingPathSeparator() const; + + // Returns a copy of the FilePath with the directory part removed. + // Example: FilePath("path/to/file").RemoveDirectoryName() returns + // FilePath("file"). If there is no directory part ("just_a_file"), it returns + // the FilePath unmodified. If there is no file part ("just_a_dir/") it + // returns an empty FilePath (""). + // On Windows platform, '\' is the path separator, otherwise it is '/'. + FilePath RemoveDirectoryName() const; + + // RemoveFileName returns the directory path with the filename removed. + // Example: FilePath("path/to/file").RemoveFileName() returns "path/to/". + // If the FilePath is "a_file" or "/a_file", RemoveFileName returns + // FilePath("./") or, on Windows, FilePath(".\\"). If the filepath does + // not have a file, like "just/a/dir/", it returns the FilePath unmodified. + // On Windows platform, '\' is the path separator, otherwise it is '/'. + FilePath RemoveFileName() const; + + // Returns a copy of the FilePath with the case-insensitive extension removed. + // Example: FilePath("dir/file.exe").RemoveExtension("EXE") returns + // FilePath("dir/file"). If a case-insensitive extension is not + // found, returns a copy of the original FilePath. + FilePath RemoveExtension(const char* extension) const; + + // Creates directories so that path exists. Returns true if successful or if + // the directories already exist; returns false if unable to create + // directories for any reason. Will also return false if the FilePath does + // not represent a directory (that is, it doesn't end with a path separator). + bool CreateDirectoriesRecursively() const; + + // Create the directory so that path exists. Returns true if successful or + // if the directory already exists; returns false if unable to create the + // directory for any reason, including if the parent directory does not + // exist. Not named "CreateDirectory" because that's a macro on Windows. + bool CreateFolder() const; + + // Returns true if FilePath describes something in the file-system, + // either a file, directory, or whatever, and that something exists. + bool FileOrDirectoryExists() const; + + // Returns true if pathname describes a directory in the file-system + // that exists. + bool DirectoryExists() const; + + // Returns true if FilePath ends with a path separator, which indicates that + // it is intended to represent a directory. Returns false otherwise. + // This does NOT check that a directory (or file) actually exists. + bool IsDirectory() const; + + // Returns true if pathname describes a root directory. (Windows has one + // root directory per disk drive.) + bool IsRootDirectory() const; + + // Returns true if pathname describes an absolute path. + bool IsAbsolutePath() const; + + private: + // Replaces multiple consecutive separators with a single separator. + // For example, "bar///foo" becomes "bar/foo". Does not eliminate other + // redundancies that might be in a pathname involving "." or "..". + // + // A pathname with multiple consecutive separators may occur either through + // user error or as a result of some scripts or APIs that generate a pathname + // with a trailing separator. On other platforms the same API or script + // may NOT generate a pathname with a trailing "/". Then elsewhere that + // pathname may have another "/" and pathname components added to it, + // without checking for the separator already being there. + // The script language and operating system may allow paths like "foo//bar" + // but some of the functions in FilePath will not handle that correctly. In + // particular, RemoveTrailingPathSeparator() only removes one separator, and + // it is called in CreateDirectoriesRecursively() assuming that it will change + // a pathname from directory syntax (trailing separator) to filename syntax. + // + // On Windows this method also replaces the alternate path separator '/' with + // the primary path separator '\\', so that for example "bar\\/\\foo" becomes + // "bar\\foo". + + void Normalize(); + + // Returns a pointer to the last occurence of a valid path separator in + // the FilePath. On Windows, for example, both '/' and '\' are valid path + // separators. Returns NULL if no path separator was found. + const char* FindLastPathSeparator() const; + + std::string pathname_; +}; // class FilePath + +} // namespace internal +} // namespace testing + +#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_FILEPATH_H_ +// This file was GENERATED by command: +// pump.py gtest-type-util.h.pump +// DO NOT EDIT BY HAND!!! + +// Copyright 2008 Google Inc. +// All Rights Reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) + +// Type utilities needed for implementing typed and type-parameterized +// tests. This file is generated by a SCRIPT. DO NOT EDIT BY HAND! +// +// Currently we support at most 50 types in a list, and at most 50 +// type-parameterized tests in one type-parameterized test case. +// Please contact googletestframework@googlegroups.com if you need +// more. + +#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_TYPE_UTIL_H_ +#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_TYPE_UTIL_H_ + + +// #ifdef __GNUC__ is too general here. It is possible to use gcc without using +// libstdc++ (which is where cxxabi.h comes from). +# if GTEST_HAS_CXXABI_H_ +# include +# elif defined(__HP_aCC) +# include +# endif // GTEST_HASH_CXXABI_H_ + +namespace testing { +namespace internal { + +// GetTypeName() returns a human-readable name of type T. +// NB: This function is also used in Google Mock, so don't move it inside of +// the typed-test-only section below. +template +std::string GetTypeName() { +# if GTEST_HAS_RTTI + + const char* const name = typeid(T).name(); +# if GTEST_HAS_CXXABI_H_ || defined(__HP_aCC) + int status = 0; + // gcc's implementation of typeid(T).name() mangles the type name, + // so we have to demangle it. +# if GTEST_HAS_CXXABI_H_ + using abi::__cxa_demangle; +# endif // GTEST_HAS_CXXABI_H_ + char* const readable_name = __cxa_demangle(name, 0, 0, &status); + const std::string name_str(status == 0 ? readable_name : name); + free(readable_name); + return name_str; +# else + return name; +# endif // GTEST_HAS_CXXABI_H_ || __HP_aCC + +# else + + return ""; + +# endif // GTEST_HAS_RTTI +} + +#if GTEST_HAS_TYPED_TEST || GTEST_HAS_TYPED_TEST_P + +// AssertyTypeEq::type is defined iff T1 and T2 are the same +// type. This can be used as a compile-time assertion to ensure that +// two types are equal. + +template +struct AssertTypeEq; + +template +struct AssertTypeEq { + typedef bool type; +}; + +// A unique type used as the default value for the arguments of class +// template Types. This allows us to simulate variadic templates +// (e.g. Types, Type, and etc), which C++ doesn't +// support directly. +struct None {}; + +// The following family of struct and struct templates are used to +// represent type lists. In particular, TypesN +// represents a type list with N types (T1, T2, ..., and TN) in it. +// Except for Types0, every struct in the family has two member types: +// Head for the first type in the list, and Tail for the rest of the +// list. + +// The empty type list. +struct Types0 {}; + +// Type lists of length 1, 2, 3, and so on. + +template +struct Types1 { + typedef T1 Head; + typedef Types0 Tail; +}; +template +struct Types2 { + typedef T1 Head; + typedef Types1 Tail; +}; + +template +struct Types3 { + typedef T1 Head; + typedef Types2 Tail; +}; + +template +struct Types4 { + typedef T1 Head; + typedef Types3 Tail; +}; + +template +struct Types5 { + typedef T1 Head; + typedef Types4 Tail; +}; + +template +struct Types6 { + typedef T1 Head; + typedef Types5 Tail; +}; + +template +struct Types7 { + typedef T1 Head; + typedef Types6 Tail; +}; + +template +struct Types8 { + typedef T1 Head; + typedef Types7 Tail; +}; + +template +struct Types9 { + typedef T1 Head; + typedef Types8 Tail; +}; + +template +struct Types10 { + typedef T1 Head; + typedef Types9 Tail; +}; + +template +struct Types11 { + typedef T1 Head; + typedef Types10 Tail; +}; + +template +struct Types12 { + typedef T1 Head; + typedef Types11 Tail; +}; + +template +struct Types13 { + typedef T1 Head; + typedef Types12 Tail; +}; + +template +struct Types14 { + typedef T1 Head; + typedef Types13 Tail; +}; + +template +struct Types15 { + typedef T1 Head; + typedef Types14 Tail; +}; + +template +struct Types16 { + typedef T1 Head; + typedef Types15 Tail; +}; + +template +struct Types17 { + typedef T1 Head; + typedef Types16 Tail; +}; + +template +struct Types18 { + typedef T1 Head; + typedef Types17 Tail; +}; + +template +struct Types19 { + typedef T1 Head; + typedef Types18 Tail; +}; + +template +struct Types20 { + typedef T1 Head; + typedef Types19 Tail; +}; + +template +struct Types21 { + typedef T1 Head; + typedef Types20 Tail; +}; + +template +struct Types22 { + typedef T1 Head; + typedef Types21 Tail; +}; + +template +struct Types23 { + typedef T1 Head; + typedef Types22 Tail; +}; + +template +struct Types24 { + typedef T1 Head; + typedef Types23 Tail; +}; + +template +struct Types25 { + typedef T1 Head; + typedef Types24 Tail; +}; + +template +struct Types26 { + typedef T1 Head; + typedef Types25 Tail; +}; + +template +struct Types27 { + typedef T1 Head; + typedef Types26 Tail; +}; + +template +struct Types28 { + typedef T1 Head; + typedef Types27 Tail; +}; + +template +struct Types29 { + typedef T1 Head; + typedef Types28 Tail; +}; + +template +struct Types30 { + typedef T1 Head; + typedef Types29 Tail; +}; + +template +struct Types31 { + typedef T1 Head; + typedef Types30 Tail; +}; + +template +struct Types32 { + typedef T1 Head; + typedef Types31 Tail; +}; + +template +struct Types33 { + typedef T1 Head; + typedef Types32 Tail; +}; + +template +struct Types34 { + typedef T1 Head; + typedef Types33 Tail; +}; + +template +struct Types35 { + typedef T1 Head; + typedef Types34 Tail; +}; + +template +struct Types36 { + typedef T1 Head; + typedef Types35 Tail; +}; + +template +struct Types37 { + typedef T1 Head; + typedef Types36 Tail; +}; + +template +struct Types38 { + typedef T1 Head; + typedef Types37 Tail; +}; + +template +struct Types39 { + typedef T1 Head; + typedef Types38 Tail; +}; + +template +struct Types40 { + typedef T1 Head; + typedef Types39 Tail; +}; + +template +struct Types41 { + typedef T1 Head; + typedef Types40 Tail; +}; + +template +struct Types42 { + typedef T1 Head; + typedef Types41 Tail; +}; + +template +struct Types43 { + typedef T1 Head; + typedef Types42 Tail; +}; + +template +struct Types44 { + typedef T1 Head; + typedef Types43 Tail; +}; + +template +struct Types45 { + typedef T1 Head; + typedef Types44 Tail; +}; + +template +struct Types46 { + typedef T1 Head; + typedef Types45 Tail; +}; + +template +struct Types47 { + typedef T1 Head; + typedef Types46 Tail; +}; + +template +struct Types48 { + typedef T1 Head; + typedef Types47 Tail; +}; + +template +struct Types49 { + typedef T1 Head; + typedef Types48 Tail; +}; + +template +struct Types50 { + typedef T1 Head; + typedef Types49 Tail; +}; + + +} // namespace internal + +// We don't want to require the users to write TypesN<...> directly, +// as that would require them to count the length. Types<...> is much +// easier to write, but generates horrible messages when there is a +// compiler error, as gcc insists on printing out each template +// argument, even if it has the default value (this means Types +// will appear as Types in the compiler +// errors). +// +// Our solution is to combine the best part of the two approaches: a +// user would write Types, and Google Test will translate +// that to TypesN internally to make error messages +// readable. The translation is done by the 'type' member of the +// Types template. +template +struct Types { + typedef internal::Types50 type; +}; + +template <> +struct Types { + typedef internal::Types0 type; +}; +template +struct Types { + typedef internal::Types1 type; +}; +template +struct Types { + typedef internal::Types2 type; +}; +template +struct Types { + typedef internal::Types3 type; +}; +template +struct Types { + typedef internal::Types4 type; +}; +template +struct Types { + typedef internal::Types5 type; +}; +template +struct Types { + typedef internal::Types6 type; +}; +template +struct Types { + typedef internal::Types7 type; +}; +template +struct Types { + typedef internal::Types8 type; +}; +template +struct Types { + typedef internal::Types9 type; +}; +template +struct Types { + typedef internal::Types10 type; +}; +template +struct Types { + typedef internal::Types11 type; +}; +template +struct Types { + typedef internal::Types12 type; +}; +template +struct Types { + typedef internal::Types13 type; +}; +template +struct Types { + typedef internal::Types14 type; +}; +template +struct Types { + typedef internal::Types15 type; +}; +template +struct Types { + typedef internal::Types16 type; +}; +template +struct Types { + typedef internal::Types17 type; +}; +template +struct Types { + typedef internal::Types18 type; +}; +template +struct Types { + typedef internal::Types19 type; +}; +template +struct Types { + typedef internal::Types20 type; +}; +template +struct Types { + typedef internal::Types21 type; +}; +template +struct Types { + typedef internal::Types22 type; +}; +template +struct Types { + typedef internal::Types23 type; +}; +template +struct Types { + typedef internal::Types24 type; +}; +template +struct Types { + typedef internal::Types25 type; +}; +template +struct Types { + typedef internal::Types26 type; +}; +template +struct Types { + typedef internal::Types27 type; +}; +template +struct Types { + typedef internal::Types28 type; +}; +template +struct Types { + typedef internal::Types29 type; +}; +template +struct Types { + typedef internal::Types30 type; +}; +template +struct Types { + typedef internal::Types31 type; +}; +template +struct Types { + typedef internal::Types32 type; +}; +template +struct Types { + typedef internal::Types33 type; +}; +template +struct Types { + typedef internal::Types34 type; +}; +template +struct Types { + typedef internal::Types35 type; +}; +template +struct Types { + typedef internal::Types36 type; +}; +template +struct Types { + typedef internal::Types37 type; +}; +template +struct Types { + typedef internal::Types38 type; +}; +template +struct Types { + typedef internal::Types39 type; +}; +template +struct Types { + typedef internal::Types40 type; +}; +template +struct Types { + typedef internal::Types41 type; +}; +template +struct Types { + typedef internal::Types42 type; +}; +template +struct Types { + typedef internal::Types43 type; +}; +template +struct Types { + typedef internal::Types44 type; +}; +template +struct Types { + typedef internal::Types45 type; +}; +template +struct Types { + typedef internal::Types46 type; +}; +template +struct Types { + typedef internal::Types47 type; +}; +template +struct Types { + typedef internal::Types48 type; +}; +template +struct Types { + typedef internal::Types49 type; +}; + +namespace internal { + +# define GTEST_TEMPLATE_ template class + +// The template "selector" struct TemplateSel is used to +// represent Tmpl, which must be a class template with one type +// parameter, as a type. TemplateSel::Bind::type is defined +// as the type Tmpl. This allows us to actually instantiate the +// template "selected" by TemplateSel. +// +// This trick is necessary for simulating typedef for class templates, +// which C++ doesn't support directly. +template +struct TemplateSel { + template + struct Bind { + typedef Tmpl type; + }; +}; + +# define GTEST_BIND_(TmplSel, T) \ + TmplSel::template Bind::type + +// A unique struct template used as the default value for the +// arguments of class template Templates. This allows us to simulate +// variadic templates (e.g. Templates, Templates, +// and etc), which C++ doesn't support directly. +template +struct NoneT {}; + +// The following family of struct and struct templates are used to +// represent template lists. In particular, TemplatesN represents a list of N templates (T1, T2, ..., and TN). Except +// for Templates0, every struct in the family has two member types: +// Head for the selector of the first template in the list, and Tail +// for the rest of the list. + +// The empty template list. +struct Templates0 {}; + +// Template lists of length 1, 2, 3, and so on. + +template +struct Templates1 { + typedef TemplateSel Head; + typedef Templates0 Tail; +}; +template +struct Templates2 { + typedef TemplateSel Head; + typedef Templates1 Tail; +}; + +template +struct Templates3 { + typedef TemplateSel Head; + typedef Templates2 Tail; +}; + +template +struct Templates4 { + typedef TemplateSel Head; + typedef Templates3 Tail; +}; + +template +struct Templates5 { + typedef TemplateSel Head; + typedef Templates4 Tail; +}; + +template +struct Templates6 { + typedef TemplateSel Head; + typedef Templates5 Tail; +}; + +template +struct Templates7 { + typedef TemplateSel Head; + typedef Templates6 Tail; +}; + +template +struct Templates8 { + typedef TemplateSel Head; + typedef Templates7 Tail; +}; + +template +struct Templates9 { + typedef TemplateSel Head; + typedef Templates8 Tail; +}; + +template +struct Templates10 { + typedef TemplateSel Head; + typedef Templates9 Tail; +}; + +template +struct Templates11 { + typedef TemplateSel Head; + typedef Templates10 Tail; +}; + +template +struct Templates12 { + typedef TemplateSel Head; + typedef Templates11 Tail; +}; + +template +struct Templates13 { + typedef TemplateSel Head; + typedef Templates12 Tail; +}; + +template +struct Templates14 { + typedef TemplateSel Head; + typedef Templates13 Tail; +}; + +template +struct Templates15 { + typedef TemplateSel Head; + typedef Templates14 Tail; +}; + +template +struct Templates16 { + typedef TemplateSel Head; + typedef Templates15 Tail; +}; + +template +struct Templates17 { + typedef TemplateSel Head; + typedef Templates16 Tail; +}; + +template +struct Templates18 { + typedef TemplateSel Head; + typedef Templates17 Tail; +}; + +template +struct Templates19 { + typedef TemplateSel Head; + typedef Templates18 Tail; +}; + +template +struct Templates20 { + typedef TemplateSel Head; + typedef Templates19 Tail; +}; + +template +struct Templates21 { + typedef TemplateSel Head; + typedef Templates20 Tail; +}; + +template +struct Templates22 { + typedef TemplateSel Head; + typedef Templates21 Tail; +}; + +template +struct Templates23 { + typedef TemplateSel Head; + typedef Templates22 Tail; +}; + +template +struct Templates24 { + typedef TemplateSel Head; + typedef Templates23 Tail; +}; + +template +struct Templates25 { + typedef TemplateSel Head; + typedef Templates24 Tail; +}; + +template +struct Templates26 { + typedef TemplateSel Head; + typedef Templates25 Tail; +}; + +template +struct Templates27 { + typedef TemplateSel Head; + typedef Templates26 Tail; +}; + +template +struct Templates28 { + typedef TemplateSel Head; + typedef Templates27 Tail; +}; + +template +struct Templates29 { + typedef TemplateSel Head; + typedef Templates28 Tail; +}; + +template +struct Templates30 { + typedef TemplateSel Head; + typedef Templates29 Tail; +}; + +template +struct Templates31 { + typedef TemplateSel Head; + typedef Templates30 Tail; +}; + +template +struct Templates32 { + typedef TemplateSel Head; + typedef Templates31 Tail; +}; + +template +struct Templates33 { + typedef TemplateSel Head; + typedef Templates32 Tail; +}; + +template +struct Templates34 { + typedef TemplateSel Head; + typedef Templates33 Tail; +}; + +template +struct Templates35 { + typedef TemplateSel Head; + typedef Templates34 Tail; +}; + +template +struct Templates36 { + typedef TemplateSel Head; + typedef Templates35 Tail; +}; + +template +struct Templates37 { + typedef TemplateSel Head; + typedef Templates36 Tail; +}; + +template +struct Templates38 { + typedef TemplateSel Head; + typedef Templates37 Tail; +}; + +template +struct Templates39 { + typedef TemplateSel Head; + typedef Templates38 Tail; +}; + +template +struct Templates40 { + typedef TemplateSel Head; + typedef Templates39 Tail; +}; + +template +struct Templates41 { + typedef TemplateSel Head; + typedef Templates40 Tail; +}; + +template +struct Templates42 { + typedef TemplateSel Head; + typedef Templates41 Tail; +}; + +template +struct Templates43 { + typedef TemplateSel Head; + typedef Templates42 Tail; +}; + +template +struct Templates44 { + typedef TemplateSel Head; + typedef Templates43 Tail; +}; + +template +struct Templates45 { + typedef TemplateSel Head; + typedef Templates44 Tail; +}; + +template +struct Templates46 { + typedef TemplateSel Head; + typedef Templates45 Tail; +}; + +template +struct Templates47 { + typedef TemplateSel Head; + typedef Templates46 Tail; +}; + +template +struct Templates48 { + typedef TemplateSel Head; + typedef Templates47 Tail; +}; + +template +struct Templates49 { + typedef TemplateSel Head; + typedef Templates48 Tail; +}; + +template +struct Templates50 { + typedef TemplateSel Head; + typedef Templates49 Tail; +}; + + +// We don't want to require the users to write TemplatesN<...> directly, +// as that would require them to count the length. Templates<...> is much +// easier to write, but generates horrible messages when there is a +// compiler error, as gcc insists on printing out each template +// argument, even if it has the default value (this means Templates +// will appear as Templates in the compiler +// errors). +// +// Our solution is to combine the best part of the two approaches: a +// user would write Templates, and Google Test will translate +// that to TemplatesN internally to make error messages +// readable. The translation is done by the 'type' member of the +// Templates template. +template +struct Templates { + typedef Templates50 type; +}; + +template <> +struct Templates { + typedef Templates0 type; +}; +template +struct Templates { + typedef Templates1 type; +}; +template +struct Templates { + typedef Templates2 type; +}; +template +struct Templates { + typedef Templates3 type; +}; +template +struct Templates { + typedef Templates4 type; +}; +template +struct Templates { + typedef Templates5 type; +}; +template +struct Templates { + typedef Templates6 type; +}; +template +struct Templates { + typedef Templates7 type; +}; +template +struct Templates { + typedef Templates8 type; +}; +template +struct Templates { + typedef Templates9 type; +}; +template +struct Templates { + typedef Templates10 type; +}; +template +struct Templates { + typedef Templates11 type; +}; +template +struct Templates { + typedef Templates12 type; +}; +template +struct Templates { + typedef Templates13 type; +}; +template +struct Templates { + typedef Templates14 type; +}; +template +struct Templates { + typedef Templates15 type; +}; +template +struct Templates { + typedef Templates16 type; +}; +template +struct Templates { + typedef Templates17 type; +}; +template +struct Templates { + typedef Templates18 type; +}; +template +struct Templates { + typedef Templates19 type; +}; +template +struct Templates { + typedef Templates20 type; +}; +template +struct Templates { + typedef Templates21 type; +}; +template +struct Templates { + typedef Templates22 type; +}; +template +struct Templates { + typedef Templates23 type; +}; +template +struct Templates { + typedef Templates24 type; +}; +template +struct Templates { + typedef Templates25 type; +}; +template +struct Templates { + typedef Templates26 type; +}; +template +struct Templates { + typedef Templates27 type; +}; +template +struct Templates { + typedef Templates28 type; +}; +template +struct Templates { + typedef Templates29 type; +}; +template +struct Templates { + typedef Templates30 type; +}; +template +struct Templates { + typedef Templates31 type; +}; +template +struct Templates { + typedef Templates32 type; +}; +template +struct Templates { + typedef Templates33 type; +}; +template +struct Templates { + typedef Templates34 type; +}; +template +struct Templates { + typedef Templates35 type; +}; +template +struct Templates { + typedef Templates36 type; +}; +template +struct Templates { + typedef Templates37 type; +}; +template +struct Templates { + typedef Templates38 type; +}; +template +struct Templates { + typedef Templates39 type; +}; +template +struct Templates { + typedef Templates40 type; +}; +template +struct Templates { + typedef Templates41 type; +}; +template +struct Templates { + typedef Templates42 type; +}; +template +struct Templates { + typedef Templates43 type; +}; +template +struct Templates { + typedef Templates44 type; +}; +template +struct Templates { + typedef Templates45 type; +}; +template +struct Templates { + typedef Templates46 type; +}; +template +struct Templates { + typedef Templates47 type; +}; +template +struct Templates { + typedef Templates48 type; +}; +template +struct Templates { + typedef Templates49 type; +}; + +// The TypeList template makes it possible to use either a single type +// or a Types<...> list in TYPED_TEST_CASE() and +// INSTANTIATE_TYPED_TEST_CASE_P(). + +template +struct TypeList { + typedef Types1 type; +}; + +template +struct TypeList > { + typedef typename Types::type type; +}; + +#endif // GTEST_HAS_TYPED_TEST || GTEST_HAS_TYPED_TEST_P + +} // namespace internal +} // namespace testing + +#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_TYPE_UTIL_H_ + +// Due to C++ preprocessor weirdness, we need double indirection to +// concatenate two tokens when one of them is __LINE__. Writing +// +// foo ## __LINE__ +// +// will result in the token foo__LINE__, instead of foo followed by +// the current line number. For more details, see +// http://www.parashift.com/c++-faq-lite/misc-technical-issues.html#faq-39.6 +#define GTEST_CONCAT_TOKEN_(foo, bar) GTEST_CONCAT_TOKEN_IMPL_(foo, bar) +#define GTEST_CONCAT_TOKEN_IMPL_(foo, bar) foo ## bar + +class ProtocolMessage; +namespace proto2 { class Message; } + +namespace testing { + +// Forward declarations. + +class AssertionResult; // Result of an assertion. +class Message; // Represents a failure message. +class Test; // Represents a test. +class TestInfo; // Information about a test. +class TestPartResult; // Result of a test part. +class UnitTest; // A collection of test cases. + +template +::std::string PrintToString(const T& value); + +namespace internal { + +struct TraceInfo; // Information about a trace point. +class ScopedTrace; // Implements scoped trace. +class TestInfoImpl; // Opaque implementation of TestInfo +class UnitTestImpl; // Opaque implementation of UnitTest + +// How many times InitGoogleTest() has been called. +GTEST_API_ extern int g_init_gtest_count; + +// The text used in failure messages to indicate the start of the +// stack trace. +GTEST_API_ extern const char kStackTraceMarker[]; + +// Two overloaded helpers for checking at compile time whether an +// expression is a null pointer literal (i.e. NULL or any 0-valued +// compile-time integral constant). Their return values have +// different sizes, so we can use sizeof() to test which version is +// picked by the compiler. These helpers have no implementations, as +// we only need their signatures. +// +// Given IsNullLiteralHelper(x), the compiler will pick the first +// version if x can be implicitly converted to Secret*, and pick the +// second version otherwise. Since Secret is a secret and incomplete +// type, the only expression a user can write that has type Secret* is +// a null pointer literal. Therefore, we know that x is a null +// pointer literal if and only if the first version is picked by the +// compiler. +char IsNullLiteralHelper(Secret* p); +char (&IsNullLiteralHelper(...))[2]; // NOLINT + +// A compile-time bool constant that is true if and only if x is a +// null pointer literal (i.e. NULL or any 0-valued compile-time +// integral constant). +#ifdef GTEST_ELLIPSIS_NEEDS_POD_ +// We lose support for NULL detection where the compiler doesn't like +// passing non-POD classes through ellipsis (...). +# define GTEST_IS_NULL_LITERAL_(x) false +#else +# define GTEST_IS_NULL_LITERAL_(x) \ + (sizeof(::testing::internal::IsNullLiteralHelper(x)) == 1) +#endif // GTEST_ELLIPSIS_NEEDS_POD_ + +// Appends the user-supplied message to the Google-Test-generated message. +GTEST_API_ std::string AppendUserMessage( + const std::string& gtest_msg, const Message& user_msg); + +#if GTEST_HAS_EXCEPTIONS + +// This exception is thrown by (and only by) a failed Google Test +// assertion when GTEST_FLAG(throw_on_failure) is true (if exceptions +// are enabled). We derive it from std::runtime_error, which is for +// errors presumably detectable only at run time. Since +// std::runtime_error inherits from std::exception, many testing +// frameworks know how to extract and print the message inside it. +class GTEST_API_ GoogleTestFailureException : public ::std::runtime_error { + public: + explicit GoogleTestFailureException(const TestPartResult& failure); +}; + +#endif // GTEST_HAS_EXCEPTIONS + +// A helper class for creating scoped traces in user programs. +class GTEST_API_ ScopedTrace { + public: + // The c'tor pushes the given source file location and message onto + // a trace stack maintained by Google Test. + ScopedTrace(const char* file, int line, const Message& message); + + // The d'tor pops the info pushed by the c'tor. + // + // Note that the d'tor is not virtual in order to be efficient. + // Don't inherit from ScopedTrace! + ~ScopedTrace(); + + private: + GTEST_DISALLOW_COPY_AND_ASSIGN_(ScopedTrace); +} GTEST_ATTRIBUTE_UNUSED_; // A ScopedTrace object does its job in its + // c'tor and d'tor. Therefore it doesn't + // need to be used otherwise. + +namespace edit_distance { +// Returns the optimal edits to go from 'left' to 'right'. +// All edits cost the same, with replace having lower priority than +// add/remove. +// Simple implementation of the Wagner-Fischer algorithm. +// See http://en.wikipedia.org/wiki/Wagner-Fischer_algorithm +enum EditType { kMatch, kAdd, kRemove, kReplace }; +GTEST_API_ std::vector CalculateOptimalEdits( + const std::vector& left, const std::vector& right); + +// Same as above, but the input is represented as strings. +GTEST_API_ std::vector CalculateOptimalEdits( + const std::vector& left, + const std::vector& right); + +// Create a diff of the input strings in Unified diff format. +GTEST_API_ std::string CreateUnifiedDiff(const std::vector& left, + const std::vector& right, + size_t context = 2); + +} // namespace edit_distance + +// Calculate the diff between 'left' and 'right' and return it in unified diff +// format. +// If not null, stores in 'total_line_count' the total number of lines found +// in left + right. +GTEST_API_ std::string DiffStrings(const std::string& left, + const std::string& right, + size_t* total_line_count); + +// Constructs and returns the message for an equality assertion +// (e.g. ASSERT_EQ, EXPECT_STREQ, etc) failure. +// +// The first four parameters are the expressions used in the assertion +// and their values, as strings. For example, for ASSERT_EQ(foo, bar) +// where foo is 5 and bar is 6, we have: +// +// expected_expression: "foo" +// actual_expression: "bar" +// expected_value: "5" +// actual_value: "6" +// +// The ignoring_case parameter is true iff the assertion is a +// *_STRCASEEQ*. When it's true, the string " (ignoring case)" will +// be inserted into the message. +GTEST_API_ AssertionResult EqFailure(const char* expected_expression, + const char* actual_expression, + const std::string& expected_value, + const std::string& actual_value, + bool ignoring_case); + +// Constructs a failure message for Boolean assertions such as EXPECT_TRUE. +GTEST_API_ std::string GetBoolAssertionFailureMessage( + const AssertionResult& assertion_result, + const char* expression_text, + const char* actual_predicate_value, + const char* expected_predicate_value); + +// This template class represents an IEEE floating-point number +// (either single-precision or double-precision, depending on the +// template parameters). +// +// The purpose of this class is to do more sophisticated number +// comparison. (Due to round-off error, etc, it's very unlikely that +// two floating-points will be equal exactly. Hence a naive +// comparison by the == operation often doesn't work.) +// +// Format of IEEE floating-point: +// +// The most-significant bit being the leftmost, an IEEE +// floating-point looks like +// +// sign_bit exponent_bits fraction_bits +// +// Here, sign_bit is a single bit that designates the sign of the +// number. +// +// For float, there are 8 exponent bits and 23 fraction bits. +// +// For double, there are 11 exponent bits and 52 fraction bits. +// +// More details can be found at +// http://en.wikipedia.org/wiki/IEEE_floating-point_standard. +// +// Template parameter: +// +// RawType: the raw floating-point type (either float or double) +template +class FloatingPoint { + public: + // Defines the unsigned integer type that has the same size as the + // floating point number. + typedef typename TypeWithSize::UInt Bits; + + // Constants. + + // # of bits in a number. + static const size_t kBitCount = 8*sizeof(RawType); + + // # of fraction bits in a number. + static const size_t kFractionBitCount = + std::numeric_limits::digits - 1; + + // # of exponent bits in a number. + static const size_t kExponentBitCount = kBitCount - 1 - kFractionBitCount; + + // The mask for the sign bit. + static const Bits kSignBitMask = static_cast(1) << (kBitCount - 1); + + // The mask for the fraction bits. + static const Bits kFractionBitMask = + ~static_cast(0) >> (kExponentBitCount + 1); + + // The mask for the exponent bits. + static const Bits kExponentBitMask = ~(kSignBitMask | kFractionBitMask); + + // How many ULP's (Units in the Last Place) we want to tolerate when + // comparing two numbers. The larger the value, the more error we + // allow. A 0 value means that two numbers must be exactly the same + // to be considered equal. + // + // The maximum error of a single floating-point operation is 0.5 + // units in the last place. On Intel CPU's, all floating-point + // calculations are done with 80-bit precision, while double has 64 + // bits. Therefore, 4 should be enough for ordinary use. + // + // See the following article for more details on ULP: + // http://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/ + static const size_t kMaxUlps = 4; + + // Constructs a FloatingPoint from a raw floating-point number. + // + // On an Intel CPU, passing a non-normalized NAN (Not a Number) + // around may change its bits, although the new value is guaranteed + // to be also a NAN. Therefore, don't expect this constructor to + // preserve the bits in x when x is a NAN. + explicit FloatingPoint(const RawType& x) { u_.value_ = x; } + + // Static methods + + // Reinterprets a bit pattern as a floating-point number. + // + // This function is needed to test the AlmostEquals() method. + static RawType ReinterpretBits(const Bits bits) { + FloatingPoint fp(0); + fp.u_.bits_ = bits; + return fp.u_.value_; + } + + // Returns the floating-point number that represent positive infinity. + static RawType Infinity() { + return ReinterpretBits(kExponentBitMask); + } + + // Returns the maximum representable finite floating-point number. + static RawType Max(); + + // Non-static methods + + // Returns the bits that represents this number. + const Bits &bits() const { return u_.bits_; } + + // Returns the exponent bits of this number. + Bits exponent_bits() const { return kExponentBitMask & u_.bits_; } + + // Returns the fraction bits of this number. + Bits fraction_bits() const { return kFractionBitMask & u_.bits_; } + + // Returns the sign bit of this number. + Bits sign_bit() const { return kSignBitMask & u_.bits_; } + + // Returns true iff this is NAN (not a number). + bool is_nan() const { + // It's a NAN if the exponent bits are all ones and the fraction + // bits are not entirely zeros. + return (exponent_bits() == kExponentBitMask) && (fraction_bits() != 0); + } + + // Returns true iff this number is at most kMaxUlps ULP's away from + // rhs. In particular, this function: + // + // - returns false if either number is (or both are) NAN. + // - treats really large numbers as almost equal to infinity. + // - thinks +0.0 and -0.0 are 0 DLP's apart. + bool AlmostEquals(const FloatingPoint& rhs) const { + // The IEEE standard says that any comparison operation involving + // a NAN must return false. + if (is_nan() || rhs.is_nan()) return false; + + return DistanceBetweenSignAndMagnitudeNumbers(u_.bits_, rhs.u_.bits_) + <= kMaxUlps; + } + + private: + // The data type used to store the actual floating-point number. + union FloatingPointUnion { + RawType value_; // The raw floating-point number. + Bits bits_; // The bits that represent the number. + }; + + // Converts an integer from the sign-and-magnitude representation to + // the biased representation. More precisely, let N be 2 to the + // power of (kBitCount - 1), an integer x is represented by the + // unsigned number x + N. + // + // For instance, + // + // -N + 1 (the most negative number representable using + // sign-and-magnitude) is represented by 1; + // 0 is represented by N; and + // N - 1 (the biggest number representable using + // sign-and-magnitude) is represented by 2N - 1. + // + // Read http://en.wikipedia.org/wiki/Signed_number_representations + // for more details on signed number representations. + static Bits SignAndMagnitudeToBiased(const Bits &sam) { + if (kSignBitMask & sam) { + // sam represents a negative number. + return ~sam + 1; + } else { + // sam represents a positive number. + return kSignBitMask | sam; + } + } + + // Given two numbers in the sign-and-magnitude representation, + // returns the distance between them as an unsigned number. + static Bits DistanceBetweenSignAndMagnitudeNumbers(const Bits &sam1, + const Bits &sam2) { + const Bits biased1 = SignAndMagnitudeToBiased(sam1); + const Bits biased2 = SignAndMagnitudeToBiased(sam2); + return (biased1 >= biased2) ? (biased1 - biased2) : (biased2 - biased1); + } + + FloatingPointUnion u_; +}; + +// We cannot use std::numeric_limits::max() as it clashes with the max() +// macro defined by . +template <> +inline float FloatingPoint::Max() { return FLT_MAX; } +template <> +inline double FloatingPoint::Max() { return DBL_MAX; } + +// Typedefs the instances of the FloatingPoint template class that we +// care to use. +typedef FloatingPoint Float; +typedef FloatingPoint Double; + +// In order to catch the mistake of putting tests that use different +// test fixture classes in the same test case, we need to assign +// unique IDs to fixture classes and compare them. The TypeId type is +// used to hold such IDs. The user should treat TypeId as an opaque +// type: the only operation allowed on TypeId values is to compare +// them for equality using the == operator. +typedef const void* TypeId; + +template +class TypeIdHelper { + public: + // dummy_ must not have a const type. Otherwise an overly eager + // compiler (e.g. MSVC 7.1 & 8.0) may try to merge + // TypeIdHelper::dummy_ for different Ts as an "optimization". + static bool dummy_; +}; + +template +bool TypeIdHelper::dummy_ = false; + +// GetTypeId() returns the ID of type T. Different values will be +// returned for different types. Calling the function twice with the +// same type argument is guaranteed to return the same ID. +template +TypeId GetTypeId() { + // The compiler is required to allocate a different + // TypeIdHelper::dummy_ variable for each T used to instantiate + // the template. Therefore, the address of dummy_ is guaranteed to + // be unique. + return &(TypeIdHelper::dummy_); +} + +// Returns the type ID of ::testing::Test. Always call this instead +// of GetTypeId< ::testing::Test>() to get the type ID of +// ::testing::Test, as the latter may give the wrong result due to a +// suspected linker bug when compiling Google Test as a Mac OS X +// framework. +GTEST_API_ TypeId GetTestTypeId(); + +// Defines the abstract factory interface that creates instances +// of a Test object. +class TestFactoryBase { + public: + virtual ~TestFactoryBase() {} + + // Creates a test instance to run. The instance is both created and destroyed + // within TestInfoImpl::Run() + virtual Test* CreateTest() = 0; + + protected: + TestFactoryBase() {} + + private: + GTEST_DISALLOW_COPY_AND_ASSIGN_(TestFactoryBase); +}; + +// This class provides implementation of TeastFactoryBase interface. +// It is used in TEST and TEST_F macros. +template +class TestFactoryImpl : public TestFactoryBase { + public: + virtual Test* CreateTest() { return new TestClass; } +}; + +#if GTEST_OS_WINDOWS + +// Predicate-formatters for implementing the HRESULT checking macros +// {ASSERT|EXPECT}_HRESULT_{SUCCEEDED|FAILED} +// We pass a long instead of HRESULT to avoid causing an +// include dependency for the HRESULT type. +GTEST_API_ AssertionResult IsHRESULTSuccess(const char* expr, + long hr); // NOLINT +GTEST_API_ AssertionResult IsHRESULTFailure(const char* expr, + long hr); // NOLINT + +#endif // GTEST_OS_WINDOWS + +// Types of SetUpTestCase() and TearDownTestCase() functions. +typedef void (*SetUpTestCaseFunc)(); +typedef void (*TearDownTestCaseFunc)(); + +// Creates a new TestInfo object and registers it with Google Test; +// returns the created object. +// +// Arguments: +// +// test_case_name: name of the test case +// name: name of the test +// type_param the name of the test's type parameter, or NULL if +// this is not a typed or a type-parameterized test. +// value_param text representation of the test's value parameter, +// or NULL if this is not a type-parameterized test. +// fixture_class_id: ID of the test fixture class +// set_up_tc: pointer to the function that sets up the test case +// tear_down_tc: pointer to the function that tears down the test case +// factory: pointer to the factory that creates a test object. +// The newly created TestInfo instance will assume +// ownership of the factory object. +GTEST_API_ TestInfo* MakeAndRegisterTestInfo( + const char* test_case_name, + const char* name, + const char* type_param, + const char* value_param, + TypeId fixture_class_id, + SetUpTestCaseFunc set_up_tc, + TearDownTestCaseFunc tear_down_tc, + TestFactoryBase* factory); + +// If *pstr starts with the given prefix, modifies *pstr to be right +// past the prefix and returns true; otherwise leaves *pstr unchanged +// and returns false. None of pstr, *pstr, and prefix can be NULL. +GTEST_API_ bool SkipPrefix(const char* prefix, const char** pstr); + +#if GTEST_HAS_TYPED_TEST || GTEST_HAS_TYPED_TEST_P + +// State of the definition of a type-parameterized test case. +class GTEST_API_ TypedTestCasePState { + public: + TypedTestCasePState() : registered_(false) {} + + // Adds the given test name to defined_test_names_ and return true + // if the test case hasn't been registered; otherwise aborts the + // program. + bool AddTestName(const char* file, int line, const char* case_name, + const char* test_name) { + if (registered_) { + fprintf(stderr, "%s Test %s must be defined before " + "REGISTER_TYPED_TEST_CASE_P(%s, ...).\n", + FormatFileLocation(file, line).c_str(), test_name, case_name); + fflush(stderr); + posix::Abort(); + } + defined_test_names_.insert(test_name); + return true; + } + + // Verifies that registered_tests match the test names in + // defined_test_names_; returns registered_tests if successful, or + // aborts the program otherwise. + const char* VerifyRegisteredTestNames( + const char* file, int line, const char* registered_tests); + + private: + bool registered_; + ::std::set defined_test_names_; +}; + +// Skips to the first non-space char after the first comma in 'str'; +// returns NULL if no comma is found in 'str'. +inline const char* SkipComma(const char* str) { + const char* comma = strchr(str, ','); + if (comma == NULL) { + return NULL; + } + while (IsSpace(*(++comma))) {} + return comma; +} + +// Returns the prefix of 'str' before the first comma in it; returns +// the entire string if it contains no comma. +inline std::string GetPrefixUntilComma(const char* str) { + const char* comma = strchr(str, ','); + return comma == NULL ? str : std::string(str, comma); +} + +// TypeParameterizedTest::Register() +// registers a list of type-parameterized tests with Google Test. The +// return value is insignificant - we just need to return something +// such that we can call this function in a namespace scope. +// +// Implementation note: The GTEST_TEMPLATE_ macro declares a template +// template parameter. It's defined in gtest-type-util.h. +template +class TypeParameterizedTest { + public: + // 'index' is the index of the test in the type list 'Types' + // specified in INSTANTIATE_TYPED_TEST_CASE_P(Prefix, TestCase, + // Types). Valid values for 'index' are [0, N - 1] where N is the + // length of Types. + static bool Register(const char* prefix, const char* case_name, + const char* test_names, int index) { + typedef typename Types::Head Type; + typedef Fixture FixtureClass; + typedef typename GTEST_BIND_(TestSel, Type) TestClass; + + // First, registers the first type-parameterized test in the type + // list. + MakeAndRegisterTestInfo( + (std::string(prefix) + (prefix[0] == '\0' ? "" : "/") + case_name + "/" + + StreamableToString(index)).c_str(), + StripTrailingSpaces(GetPrefixUntilComma(test_names)).c_str(), + GetTypeName().c_str(), + NULL, // No value parameter. + GetTypeId(), + TestClass::SetUpTestCase, + TestClass::TearDownTestCase, + new TestFactoryImpl); + + // Next, recurses (at compile time) with the tail of the type list. + return TypeParameterizedTest + ::Register(prefix, case_name, test_names, index + 1); + } +}; + +// The base case for the compile time recursion. +template +class TypeParameterizedTest { + public: + static bool Register(const char* /*prefix*/, const char* /*case_name*/, + const char* /*test_names*/, int /*index*/) { + return true; + } +}; + +// TypeParameterizedTestCase::Register() +// registers *all combinations* of 'Tests' and 'Types' with Google +// Test. The return value is insignificant - we just need to return +// something such that we can call this function in a namespace scope. +template +class TypeParameterizedTestCase { + public: + static bool Register(const char* prefix, const char* case_name, + const char* test_names) { + typedef typename Tests::Head Head; + + // First, register the first test in 'Test' for each type in 'Types'. + TypeParameterizedTest::Register( + prefix, case_name, test_names, 0); + + // Next, recurses (at compile time) with the tail of the test list. + return TypeParameterizedTestCase + ::Register(prefix, case_name, SkipComma(test_names)); + } +}; + +// The base case for the compile time recursion. +template +class TypeParameterizedTestCase { + public: + static bool Register(const char* /*prefix*/, const char* /*case_name*/, + const char* /*test_names*/) { + return true; + } +}; + +#endif // GTEST_HAS_TYPED_TEST || GTEST_HAS_TYPED_TEST_P + +// Returns the current OS stack trace as an std::string. +// +// The maximum number of stack frames to be included is specified by +// the gtest_stack_trace_depth flag. The skip_count parameter +// specifies the number of top frames to be skipped, which doesn't +// count against the number of frames to be included. +// +// For example, if Foo() calls Bar(), which in turn calls +// GetCurrentOsStackTraceExceptTop(..., 1), Foo() will be included in +// the trace but Bar() and GetCurrentOsStackTraceExceptTop() won't. +GTEST_API_ std::string GetCurrentOsStackTraceExceptTop( + UnitTest* unit_test, int skip_count); + +// Helpers for suppressing warnings on unreachable code or constant +// condition. + +// Always returns true. +GTEST_API_ bool AlwaysTrue(); + +// Always returns false. +inline bool AlwaysFalse() { return !AlwaysTrue(); } + +// Helper for suppressing false warning from Clang on a const char* +// variable declared in a conditional expression always being NULL in +// the else branch. +struct GTEST_API_ ConstCharPtr { + ConstCharPtr(const char* str) : value(str) {} + operator bool() const { return true; } + const char* value; +}; + +// A simple Linear Congruential Generator for generating random +// numbers with a uniform distribution. Unlike rand() and srand(), it +// doesn't use global state (and therefore can't interfere with user +// code). Unlike rand_r(), it's portable. An LCG isn't very random, +// but it's good enough for our purposes. +class GTEST_API_ Random { + public: + static const UInt32 kMaxRange = 1u << 31; + + explicit Random(UInt32 seed) : state_(seed) {} + + void Reseed(UInt32 seed) { state_ = seed; } + + // Generates a random number from [0, range). Crashes if 'range' is + // 0 or greater than kMaxRange. + UInt32 Generate(UInt32 range); + + private: + UInt32 state_; + GTEST_DISALLOW_COPY_AND_ASSIGN_(Random); +}; + +// Defining a variable of type CompileAssertTypesEqual will cause a +// compiler error iff T1 and T2 are different types. +template +struct CompileAssertTypesEqual; + +template +struct CompileAssertTypesEqual { +}; + +// Removes the reference from a type if it is a reference type, +// otherwise leaves it unchanged. This is the same as +// tr1::remove_reference, which is not widely available yet. +template +struct RemoveReference { typedef T type; }; // NOLINT +template +struct RemoveReference { typedef T type; }; // NOLINT + +// A handy wrapper around RemoveReference that works when the argument +// T depends on template parameters. +#define GTEST_REMOVE_REFERENCE_(T) \ + typename ::testing::internal::RemoveReference::type + +// Removes const from a type if it is a const type, otherwise leaves +// it unchanged. This is the same as tr1::remove_const, which is not +// widely available yet. +template +struct RemoveConst { typedef T type; }; // NOLINT +template +struct RemoveConst { typedef T type; }; // NOLINT + +// MSVC 8.0, Sun C++, and IBM XL C++ have a bug which causes the above +// definition to fail to remove the const in 'const int[3]' and 'const +// char[3][4]'. The following specialization works around the bug. +template +struct RemoveConst { + typedef typename RemoveConst::type type[N]; +}; + +#if defined(_MSC_VER) && _MSC_VER < 1400 +// This is the only specialization that allows VC++ 7.1 to remove const in +// 'const int[3] and 'const int[3][4]'. However, it causes trouble with GCC +// and thus needs to be conditionally compiled. +template +struct RemoveConst { + typedef typename RemoveConst::type type[N]; +}; +#endif + +// A handy wrapper around RemoveConst that works when the argument +// T depends on template parameters. +#define GTEST_REMOVE_CONST_(T) \ + typename ::testing::internal::RemoveConst::type + +// Turns const U&, U&, const U, and U all into U. +#define GTEST_REMOVE_REFERENCE_AND_CONST_(T) \ + GTEST_REMOVE_CONST_(GTEST_REMOVE_REFERENCE_(T)) + +// Adds reference to a type if it is not a reference type, +// otherwise leaves it unchanged. This is the same as +// tr1::add_reference, which is not widely available yet. +template +struct AddReference { typedef T& type; }; // NOLINT +template +struct AddReference { typedef T& type; }; // NOLINT + +// A handy wrapper around AddReference that works when the argument T +// depends on template parameters. +#define GTEST_ADD_REFERENCE_(T) \ + typename ::testing::internal::AddReference::type + +// Adds a reference to const on top of T as necessary. For example, +// it transforms +// +// char ==> const char& +// const char ==> const char& +// char& ==> const char& +// const char& ==> const char& +// +// The argument T must depend on some template parameters. +#define GTEST_REFERENCE_TO_CONST_(T) \ + GTEST_ADD_REFERENCE_(const GTEST_REMOVE_REFERENCE_(T)) + +// ImplicitlyConvertible::value is a compile-time bool +// constant that's true iff type From can be implicitly converted to +// type To. +template +class ImplicitlyConvertible { + private: + // We need the following helper functions only for their types. + // They have no implementations. + + // MakeFrom() is an expression whose type is From. We cannot simply + // use From(), as the type From may not have a public default + // constructor. + static typename AddReference::type MakeFrom(); + + // These two functions are overloaded. Given an expression + // Helper(x), the compiler will pick the first version if x can be + // implicitly converted to type To; otherwise it will pick the + // second version. + // + // The first version returns a value of size 1, and the second + // version returns a value of size 2. Therefore, by checking the + // size of Helper(x), which can be done at compile time, we can tell + // which version of Helper() is used, and hence whether x can be + // implicitly converted to type To. + static char Helper(To); + static char (&Helper(...))[2]; // NOLINT + + // We have to put the 'public' section after the 'private' section, + // or MSVC refuses to compile the code. + public: +#if defined(__BORLANDC__) + // C++Builder cannot use member overload resolution during template + // instantiation. The simplest workaround is to use its C++0x type traits + // functions (C++Builder 2009 and above only). + static const bool value = __is_convertible(From, To); +#else + // MSVC warns about implicitly converting from double to int for + // possible loss of data, so we need to temporarily disable the + // warning. + GTEST_DISABLE_MSC_WARNINGS_PUSH_(4244) + static const bool value = + sizeof(Helper(ImplicitlyConvertible::MakeFrom())) == 1; + GTEST_DISABLE_MSC_WARNINGS_POP_() +#endif // __BORLANDC__ +}; +template +const bool ImplicitlyConvertible::value; + +// IsAProtocolMessage::value is a compile-time bool constant that's +// true iff T is type ProtocolMessage, proto2::Message, or a subclass +// of those. +template +struct IsAProtocolMessage + : public bool_constant< + ImplicitlyConvertible::value || + ImplicitlyConvertible::value> { +}; + +// When the compiler sees expression IsContainerTest(0), if C is an +// STL-style container class, the first overload of IsContainerTest +// will be viable (since both C::iterator* and C::const_iterator* are +// valid types and NULL can be implicitly converted to them). It will +// be picked over the second overload as 'int' is a perfect match for +// the type of argument 0. If C::iterator or C::const_iterator is not +// a valid type, the first overload is not viable, and the second +// overload will be picked. Therefore, we can determine whether C is +// a container class by checking the type of IsContainerTest(0). +// The value of the expression is insignificant. +// +// Note that we look for both C::iterator and C::const_iterator. The +// reason is that C++ injects the name of a class as a member of the +// class itself (e.g. you can refer to class iterator as either +// 'iterator' or 'iterator::iterator'). If we look for C::iterator +// only, for example, we would mistakenly think that a class named +// iterator is an STL container. +// +// Also note that the simpler approach of overloading +// IsContainerTest(typename C::const_iterator*) and +// IsContainerTest(...) doesn't work with Visual Age C++ and Sun C++. +typedef int IsContainer; +template +IsContainer IsContainerTest(int /* dummy */, + typename C::iterator* /* it */ = NULL, + typename C::const_iterator* /* const_it */ = NULL) { + return 0; +} + +typedef char IsNotContainer; +template +IsNotContainer IsContainerTest(long /* dummy */) { return '\0'; } + +// EnableIf::type is void when 'Cond' is true, and +// undefined when 'Cond' is false. To use SFINAE to make a function +// overload only apply when a particular expression is true, add +// "typename EnableIf::type* = 0" as the last parameter. +template struct EnableIf; +template<> struct EnableIf { typedef void type; }; // NOLINT + +// Utilities for native arrays. + +// ArrayEq() compares two k-dimensional native arrays using the +// elements' operator==, where k can be any integer >= 0. When k is +// 0, ArrayEq() degenerates into comparing a single pair of values. + +template +bool ArrayEq(const T* lhs, size_t size, const U* rhs); + +// This generic version is used when k is 0. +template +inline bool ArrayEq(const T& lhs, const U& rhs) { return lhs == rhs; } + +// This overload is used when k >= 1. +template +inline bool ArrayEq(const T(&lhs)[N], const U(&rhs)[N]) { + return internal::ArrayEq(lhs, N, rhs); +} + +// This helper reduces code bloat. If we instead put its logic inside +// the previous ArrayEq() function, arrays with different sizes would +// lead to different copies of the template code. +template +bool ArrayEq(const T* lhs, size_t size, const U* rhs) { + for (size_t i = 0; i != size; i++) { + if (!internal::ArrayEq(lhs[i], rhs[i])) + return false; + } + return true; +} + +// Finds the first element in the iterator range [begin, end) that +// equals elem. Element may be a native array type itself. +template +Iter ArrayAwareFind(Iter begin, Iter end, const Element& elem) { + for (Iter it = begin; it != end; ++it) { + if (internal::ArrayEq(*it, elem)) + return it; + } + return end; +} + +// CopyArray() copies a k-dimensional native array using the elements' +// operator=, where k can be any integer >= 0. When k is 0, +// CopyArray() degenerates into copying a single value. + +template +void CopyArray(const T* from, size_t size, U* to); + +// This generic version is used when k is 0. +template +inline void CopyArray(const T& from, U* to) { *to = from; } + +// This overload is used when k >= 1. +template +inline void CopyArray(const T(&from)[N], U(*to)[N]) { + internal::CopyArray(from, N, *to); +} + +// This helper reduces code bloat. If we instead put its logic inside +// the previous CopyArray() function, arrays with different sizes +// would lead to different copies of the template code. +template +void CopyArray(const T* from, size_t size, U* to) { + for (size_t i = 0; i != size; i++) { + internal::CopyArray(from[i], to + i); + } +} + +// The relation between an NativeArray object (see below) and the +// native array it represents. +// We use 2 different structs to allow non-copyable types to be used, as long +// as RelationToSourceReference() is passed. +struct RelationToSourceReference {}; +struct RelationToSourceCopy {}; + +// Adapts a native array to a read-only STL-style container. Instead +// of the complete STL container concept, this adaptor only implements +// members useful for Google Mock's container matchers. New members +// should be added as needed. To simplify the implementation, we only +// support Element being a raw type (i.e. having no top-level const or +// reference modifier). It's the client's responsibility to satisfy +// this requirement. Element can be an array type itself (hence +// multi-dimensional arrays are supported). +template +class NativeArray { + public: + // STL-style container typedefs. + typedef Element value_type; + typedef Element* iterator; + typedef const Element* const_iterator; + + // Constructs from a native array. References the source. + NativeArray(const Element* array, size_t count, RelationToSourceReference) { + InitRef(array, count); + } + + // Constructs from a native array. Copies the source. + NativeArray(const Element* array, size_t count, RelationToSourceCopy) { + InitCopy(array, count); + } + + // Copy constructor. + NativeArray(const NativeArray& rhs) { + (this->*rhs.clone_)(rhs.array_, rhs.size_); + } + + ~NativeArray() { + if (clone_ != &NativeArray::InitRef) + delete[] array_; + } + + // STL-style container methods. + size_t size() const { return size_; } + const_iterator begin() const { return array_; } + const_iterator end() const { return array_ + size_; } + bool operator==(const NativeArray& rhs) const { + return size() == rhs.size() && + ArrayEq(begin(), size(), rhs.begin()); + } + + private: + enum { + kCheckTypeIsNotConstOrAReference = StaticAssertTypeEqHelper< + Element, GTEST_REMOVE_REFERENCE_AND_CONST_(Element)>::value, + }; + + // Initializes this object with a copy of the input. + void InitCopy(const Element* array, size_t a_size) { + Element* const copy = new Element[a_size]; + CopyArray(array, a_size, copy); + array_ = copy; + size_ = a_size; + clone_ = &NativeArray::InitCopy; + } + + // Initializes this object with a reference of the input. + void InitRef(const Element* array, size_t a_size) { + array_ = array; + size_ = a_size; + clone_ = &NativeArray::InitRef; + } + + const Element* array_; + size_t size_; + void (NativeArray::*clone_)(const Element*, size_t); + + GTEST_DISALLOW_ASSIGN_(NativeArray); +}; + +} // namespace internal +} // namespace testing + +#define GTEST_MESSAGE_AT_(file, line, message, result_type) \ + ::testing::internal::AssertHelper(result_type, file, line, message) \ + = ::testing::Message() + +#define GTEST_MESSAGE_(message, result_type) \ + GTEST_MESSAGE_AT_(__FILE__, __LINE__, message, result_type) + +#define GTEST_FATAL_FAILURE_(message) \ + return GTEST_MESSAGE_(message, ::testing::TestPartResult::kFatalFailure) + +#define GTEST_NONFATAL_FAILURE_(message) \ + GTEST_MESSAGE_(message, ::testing::TestPartResult::kNonFatalFailure) + +#define GTEST_SUCCESS_(message) \ + GTEST_MESSAGE_(message, ::testing::TestPartResult::kSuccess) + +// Suppresses MSVC warnings 4072 (unreachable code) for the code following +// statement if it returns or throws (or doesn't return or throw in some +// situations). +#define GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement) \ + if (::testing::internal::AlwaysTrue()) { statement; } + +#define GTEST_TEST_THROW_(statement, expected_exception, fail) \ + GTEST_AMBIGUOUS_ELSE_BLOCKER_ \ + if (::testing::internal::ConstCharPtr gtest_msg = "") { \ + bool gtest_caught_expected = false; \ + try { \ + GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement); \ + } \ + catch (expected_exception const&) { \ + gtest_caught_expected = true; \ + } \ + catch (...) { \ + gtest_msg.value = \ + "Expected: " #statement " throws an exception of type " \ + #expected_exception ".\n Actual: it throws a different type."; \ + goto GTEST_CONCAT_TOKEN_(gtest_label_testthrow_, __LINE__); \ + } \ + if (!gtest_caught_expected) { \ + gtest_msg.value = \ + "Expected: " #statement " throws an exception of type " \ + #expected_exception ".\n Actual: it throws nothing."; \ + goto GTEST_CONCAT_TOKEN_(gtest_label_testthrow_, __LINE__); \ + } \ + } else \ + GTEST_CONCAT_TOKEN_(gtest_label_testthrow_, __LINE__): \ + fail(gtest_msg.value) + +#define GTEST_TEST_NO_THROW_(statement, fail) \ + GTEST_AMBIGUOUS_ELSE_BLOCKER_ \ + if (::testing::internal::AlwaysTrue()) { \ + try { \ + GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement); \ + } \ + catch (...) { \ + goto GTEST_CONCAT_TOKEN_(gtest_label_testnothrow_, __LINE__); \ + } \ + } else \ + GTEST_CONCAT_TOKEN_(gtest_label_testnothrow_, __LINE__): \ + fail("Expected: " #statement " doesn't throw an exception.\n" \ + " Actual: it throws.") + +#define GTEST_TEST_ANY_THROW_(statement, fail) \ + GTEST_AMBIGUOUS_ELSE_BLOCKER_ \ + if (::testing::internal::AlwaysTrue()) { \ + bool gtest_caught_any = false; \ + try { \ + GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement); \ + } \ + catch (...) { \ + gtest_caught_any = true; \ + } \ + if (!gtest_caught_any) { \ + goto GTEST_CONCAT_TOKEN_(gtest_label_testanythrow_, __LINE__); \ + } \ + } else \ + GTEST_CONCAT_TOKEN_(gtest_label_testanythrow_, __LINE__): \ + fail("Expected: " #statement " throws an exception.\n" \ + " Actual: it doesn't.") + + +// Implements Boolean test assertions such as EXPECT_TRUE. expression can be +// either a boolean expression or an AssertionResult. text is a textual +// represenation of expression as it was passed into the EXPECT_TRUE. +#define GTEST_TEST_BOOLEAN_(expression, text, actual, expected, fail) \ + GTEST_AMBIGUOUS_ELSE_BLOCKER_ \ + if (const ::testing::AssertionResult gtest_ar_ = \ + ::testing::AssertionResult(expression)) \ + ; \ + else \ + fail(::testing::internal::GetBoolAssertionFailureMessage(\ + gtest_ar_, text, #actual, #expected).c_str()) + +#define GTEST_TEST_NO_FATAL_FAILURE_(statement, fail) \ + GTEST_AMBIGUOUS_ELSE_BLOCKER_ \ + if (::testing::internal::AlwaysTrue()) { \ + ::testing::internal::HasNewFatalFailureHelper gtest_fatal_failure_checker; \ + GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement); \ + if (gtest_fatal_failure_checker.has_new_fatal_failure()) { \ + goto GTEST_CONCAT_TOKEN_(gtest_label_testnofatal_, __LINE__); \ + } \ + } else \ + GTEST_CONCAT_TOKEN_(gtest_label_testnofatal_, __LINE__): \ + fail("Expected: " #statement " doesn't generate new fatal " \ + "failures in the current thread.\n" \ + " Actual: it does.") + +// Expands to the name of the class that implements the given test. +#define GTEST_TEST_CLASS_NAME_(test_case_name, test_name) \ + test_case_name##_##test_name##_Test + +// Helper macro for defining tests. +#define GTEST_TEST_(test_case_name, test_name, parent_class, parent_id)\ +class GTEST_TEST_CLASS_NAME_(test_case_name, test_name) : public parent_class {\ + public:\ + GTEST_TEST_CLASS_NAME_(test_case_name, test_name)() {}\ + private:\ + virtual void TestBody();\ + static ::testing::TestInfo* const test_info_ GTEST_ATTRIBUTE_UNUSED_;\ + GTEST_DISALLOW_COPY_AND_ASSIGN_(\ + GTEST_TEST_CLASS_NAME_(test_case_name, test_name));\ +};\ +\ +::testing::TestInfo* const GTEST_TEST_CLASS_NAME_(test_case_name, test_name)\ + ::test_info_ =\ + ::testing::internal::MakeAndRegisterTestInfo(\ + #test_case_name, #test_name, NULL, NULL, \ + (parent_id), \ + parent_class::SetUpTestCase, \ + parent_class::TearDownTestCase, \ + new ::testing::internal::TestFactoryImpl<\ + GTEST_TEST_CLASS_NAME_(test_case_name, test_name)>);\ +void GTEST_TEST_CLASS_NAME_(test_case_name, test_name)::TestBody() + +#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_INTERNAL_H_ + +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) +// +// The Google C++ Testing Framework (Google Test) +// +// This header file defines the public API for death tests. It is +// #included by gtest.h so a user doesn't need to include this +// directly. + +#ifndef GTEST_INCLUDE_GTEST_GTEST_DEATH_TEST_H_ +#define GTEST_INCLUDE_GTEST_GTEST_DEATH_TEST_H_ + +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Authors: wan@google.com (Zhanyong Wan), eefacm@gmail.com (Sean Mcafee) +// +// The Google C++ Testing Framework (Google Test) +// +// This header file defines internal utilities needed for implementing +// death tests. They are subject to change without notice. + +#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_DEATH_TEST_INTERNAL_H_ +#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_DEATH_TEST_INTERNAL_H_ + + +#include + +namespace testing { +namespace internal { + +GTEST_DECLARE_string_(internal_run_death_test); + +// Names of the flags (needed for parsing Google Test flags). +const char kDeathTestStyleFlag[] = "death_test_style"; +const char kDeathTestUseFork[] = "death_test_use_fork"; +const char kInternalRunDeathTestFlag[] = "internal_run_death_test"; + +#if GTEST_HAS_DEATH_TEST + +// DeathTest is a class that hides much of the complexity of the +// GTEST_DEATH_TEST_ macro. It is abstract; its static Create method +// returns a concrete class that depends on the prevailing death test +// style, as defined by the --gtest_death_test_style and/or +// --gtest_internal_run_death_test flags. + +// In describing the results of death tests, these terms are used with +// the corresponding definitions: +// +// exit status: The integer exit information in the format specified +// by wait(2) +// exit code: The integer code passed to exit(3), _exit(2), or +// returned from main() +class GTEST_API_ DeathTest { + public: + // Create returns false if there was an error determining the + // appropriate action to take for the current death test; for example, + // if the gtest_death_test_style flag is set to an invalid value. + // The LastMessage method will return a more detailed message in that + // case. Otherwise, the DeathTest pointer pointed to by the "test" + // argument is set. If the death test should be skipped, the pointer + // is set to NULL; otherwise, it is set to the address of a new concrete + // DeathTest object that controls the execution of the current test. + static bool Create(const char* statement, const RE* regex, + const char* file, int line, DeathTest** test); + DeathTest(); + virtual ~DeathTest() { } + + // A helper class that aborts a death test when it's deleted. + class ReturnSentinel { + public: + explicit ReturnSentinel(DeathTest* test) : test_(test) { } + ~ReturnSentinel() { test_->Abort(TEST_ENCOUNTERED_RETURN_STATEMENT); } + private: + DeathTest* const test_; + GTEST_DISALLOW_COPY_AND_ASSIGN_(ReturnSentinel); + } GTEST_ATTRIBUTE_UNUSED_; + + // An enumeration of possible roles that may be taken when a death + // test is encountered. EXECUTE means that the death test logic should + // be executed immediately. OVERSEE means that the program should prepare + // the appropriate environment for a child process to execute the death + // test, then wait for it to complete. + enum TestRole { OVERSEE_TEST, EXECUTE_TEST }; + + // An enumeration of the three reasons that a test might be aborted. + enum AbortReason { + TEST_ENCOUNTERED_RETURN_STATEMENT, + TEST_THREW_EXCEPTION, + TEST_DID_NOT_DIE + }; + + // Assumes one of the above roles. + virtual TestRole AssumeRole() = 0; + + // Waits for the death test to finish and returns its status. + virtual int Wait() = 0; + + // Returns true if the death test passed; that is, the test process + // exited during the test, its exit status matches a user-supplied + // predicate, and its stderr output matches a user-supplied regular + // expression. + // The user-supplied predicate may be a macro expression rather + // than a function pointer or functor, or else Wait and Passed could + // be combined. + virtual bool Passed(bool exit_status_ok) = 0; + + // Signals that the death test did not die as expected. + virtual void Abort(AbortReason reason) = 0; + + // Returns a human-readable outcome message regarding the outcome of + // the last death test. + static const char* LastMessage(); + + static void set_last_death_test_message(const std::string& message); + + private: + // A string containing a description of the outcome of the last death test. + static std::string last_death_test_message_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(DeathTest); +}; + +// Factory interface for death tests. May be mocked out for testing. +class DeathTestFactory { + public: + virtual ~DeathTestFactory() { } + virtual bool Create(const char* statement, const RE* regex, + const char* file, int line, DeathTest** test) = 0; +}; + +// A concrete DeathTestFactory implementation for normal use. +class DefaultDeathTestFactory : public DeathTestFactory { + public: + virtual bool Create(const char* statement, const RE* regex, + const char* file, int line, DeathTest** test); +}; + +// Returns true if exit_status describes a process that was terminated +// by a signal, or exited normally with a nonzero exit code. +GTEST_API_ bool ExitedUnsuccessfully(int exit_status); + +// Traps C++ exceptions escaping statement and reports them as test +// failures. Note that trapping SEH exceptions is not implemented here. +# if GTEST_HAS_EXCEPTIONS +# define GTEST_EXECUTE_DEATH_TEST_STATEMENT_(statement, death_test) \ + try { \ + GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement); \ + } catch (const ::std::exception& gtest_exception) { \ + fprintf(\ + stderr, \ + "\n%s: Caught std::exception-derived exception escaping the " \ + "death test statement. Exception message: %s\n", \ + ::testing::internal::FormatFileLocation(__FILE__, __LINE__).c_str(), \ + gtest_exception.what()); \ + fflush(stderr); \ + death_test->Abort(::testing::internal::DeathTest::TEST_THREW_EXCEPTION); \ + } catch (...) { \ + death_test->Abort(::testing::internal::DeathTest::TEST_THREW_EXCEPTION); \ + } + +# else +# define GTEST_EXECUTE_DEATH_TEST_STATEMENT_(statement, death_test) \ + GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement) + +# endif + +// This macro is for implementing ASSERT_DEATH*, EXPECT_DEATH*, +// ASSERT_EXIT*, and EXPECT_EXIT*. +# define GTEST_DEATH_TEST_(statement, predicate, regex, fail) \ + GTEST_AMBIGUOUS_ELSE_BLOCKER_ \ + if (::testing::internal::AlwaysTrue()) { \ + const ::testing::internal::RE& gtest_regex = (regex); \ + ::testing::internal::DeathTest* gtest_dt; \ + if (!::testing::internal::DeathTest::Create(#statement, >est_regex, \ + __FILE__, __LINE__, >est_dt)) { \ + goto GTEST_CONCAT_TOKEN_(gtest_label_, __LINE__); \ + } \ + if (gtest_dt != NULL) { \ + ::testing::internal::scoped_ptr< ::testing::internal::DeathTest> \ + gtest_dt_ptr(gtest_dt); \ + switch (gtest_dt->AssumeRole()) { \ + case ::testing::internal::DeathTest::OVERSEE_TEST: \ + if (!gtest_dt->Passed(predicate(gtest_dt->Wait()))) { \ + goto GTEST_CONCAT_TOKEN_(gtest_label_, __LINE__); \ + } \ + break; \ + case ::testing::internal::DeathTest::EXECUTE_TEST: { \ + ::testing::internal::DeathTest::ReturnSentinel \ + gtest_sentinel(gtest_dt); \ + GTEST_EXECUTE_DEATH_TEST_STATEMENT_(statement, gtest_dt); \ + gtest_dt->Abort(::testing::internal::DeathTest::TEST_DID_NOT_DIE); \ + break; \ + } \ + default: \ + break; \ + } \ + } \ + } else \ + GTEST_CONCAT_TOKEN_(gtest_label_, __LINE__): \ + fail(::testing::internal::DeathTest::LastMessage()) +// The symbol "fail" here expands to something into which a message +// can be streamed. + +// This macro is for implementing ASSERT/EXPECT_DEBUG_DEATH when compiled in +// NDEBUG mode. In this case we need the statements to be executed, the regex is +// ignored, and the macro must accept a streamed message even though the message +// is never printed. +# define GTEST_EXECUTE_STATEMENT_(statement, regex) \ + GTEST_AMBIGUOUS_ELSE_BLOCKER_ \ + if (::testing::internal::AlwaysTrue()) { \ + GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement); \ + } else \ + ::testing::Message() + +// A class representing the parsed contents of the +// --gtest_internal_run_death_test flag, as it existed when +// RUN_ALL_TESTS was called. +class InternalRunDeathTestFlag { + public: + InternalRunDeathTestFlag(const std::string& a_file, + int a_line, + int an_index, + int a_write_fd) + : file_(a_file), line_(a_line), index_(an_index), + write_fd_(a_write_fd) {} + + ~InternalRunDeathTestFlag() { + if (write_fd_ >= 0) + posix::Close(write_fd_); + } + + const std::string& file() const { return file_; } + int line() const { return line_; } + int index() const { return index_; } + int write_fd() const { return write_fd_; } + + private: + std::string file_; + int line_; + int index_; + int write_fd_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(InternalRunDeathTestFlag); +}; + +// Returns a newly created InternalRunDeathTestFlag object with fields +// initialized from the GTEST_FLAG(internal_run_death_test) flag if +// the flag is specified; otherwise returns NULL. +InternalRunDeathTestFlag* ParseInternalRunDeathTestFlag(); + +#else // GTEST_HAS_DEATH_TEST + +// This macro is used for implementing macros such as +// EXPECT_DEATH_IF_SUPPORTED and ASSERT_DEATH_IF_SUPPORTED on systems where +// death tests are not supported. Those macros must compile on such systems +// iff EXPECT_DEATH and ASSERT_DEATH compile with the same parameters on +// systems that support death tests. This allows one to write such a macro +// on a system that does not support death tests and be sure that it will +// compile on a death-test supporting system. +// +// Parameters: +// statement - A statement that a macro such as EXPECT_DEATH would test +// for program termination. This macro has to make sure this +// statement is compiled but not executed, to ensure that +// EXPECT_DEATH_IF_SUPPORTED compiles with a certain +// parameter iff EXPECT_DEATH compiles with it. +// regex - A regex that a macro such as EXPECT_DEATH would use to test +// the output of statement. This parameter has to be +// compiled but not evaluated by this macro, to ensure that +// this macro only accepts expressions that a macro such as +// EXPECT_DEATH would accept. +// terminator - Must be an empty statement for EXPECT_DEATH_IF_SUPPORTED +// and a return statement for ASSERT_DEATH_IF_SUPPORTED. +// This ensures that ASSERT_DEATH_IF_SUPPORTED will not +// compile inside functions where ASSERT_DEATH doesn't +// compile. +// +// The branch that has an always false condition is used to ensure that +// statement and regex are compiled (and thus syntactically correct) but +// never executed. The unreachable code macro protects the terminator +// statement from generating an 'unreachable code' warning in case +// statement unconditionally returns or throws. The Message constructor at +// the end allows the syntax of streaming additional messages into the +// macro, for compilational compatibility with EXPECT_DEATH/ASSERT_DEATH. +# define GTEST_UNSUPPORTED_DEATH_TEST_(statement, regex, terminator) \ + GTEST_AMBIGUOUS_ELSE_BLOCKER_ \ + if (::testing::internal::AlwaysTrue()) { \ + GTEST_LOG_(WARNING) \ + << "Death tests are not supported on this platform.\n" \ + << "Statement '" #statement "' cannot be verified."; \ + } else if (::testing::internal::AlwaysFalse()) { \ + ::testing::internal::RE::PartialMatch(".*", (regex)); \ + GTEST_SUPPRESS_UNREACHABLE_CODE_WARNING_BELOW_(statement); \ + terminator; \ + } else \ + ::testing::Message() + +#endif // GTEST_HAS_DEATH_TEST + +} // namespace internal +} // namespace testing + +#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_DEATH_TEST_INTERNAL_H_ + +namespace testing { + +// This flag controls the style of death tests. Valid values are "threadsafe", +// meaning that the death test child process will re-execute the test binary +// from the start, running only a single death test, or "fast", +// meaning that the child process will execute the test logic immediately +// after forking. +GTEST_DECLARE_string_(death_test_style); + +#if GTEST_HAS_DEATH_TEST + +namespace internal { + +// Returns a Boolean value indicating whether the caller is currently +// executing in the context of the death test child process. Tools such as +// Valgrind heap checkers may need this to modify their behavior in death +// tests. IMPORTANT: This is an internal utility. Using it may break the +// implementation of death tests. User code MUST NOT use it. +GTEST_API_ bool InDeathTestChild(); + +} // namespace internal + +// The following macros are useful for writing death tests. + +// Here's what happens when an ASSERT_DEATH* or EXPECT_DEATH* is +// executed: +// +// 1. It generates a warning if there is more than one active +// thread. This is because it's safe to fork() or clone() only +// when there is a single thread. +// +// 2. The parent process clone()s a sub-process and runs the death +// test in it; the sub-process exits with code 0 at the end of the +// death test, if it hasn't exited already. +// +// 3. The parent process waits for the sub-process to terminate. +// +// 4. The parent process checks the exit code and error message of +// the sub-process. +// +// Examples: +// +// ASSERT_DEATH(server.SendMessage(56, "Hello"), "Invalid port number"); +// for (int i = 0; i < 5; i++) { +// EXPECT_DEATH(server.ProcessRequest(i), +// "Invalid request .* in ProcessRequest()") +// << "Failed to die on request " << i; +// } +// +// ASSERT_EXIT(server.ExitNow(), ::testing::ExitedWithCode(0), "Exiting"); +// +// bool KilledBySIGHUP(int exit_code) { +// return WIFSIGNALED(exit_code) && WTERMSIG(exit_code) == SIGHUP; +// } +// +// ASSERT_EXIT(client.HangUpServer(), KilledBySIGHUP, "Hanging up!"); +// +// On the regular expressions used in death tests: +// +// On POSIX-compliant systems (*nix), we use the library, +// which uses the POSIX extended regex syntax. +// +// On other platforms (e.g. Windows), we only support a simple regex +// syntax implemented as part of Google Test. This limited +// implementation should be enough most of the time when writing +// death tests; though it lacks many features you can find in PCRE +// or POSIX extended regex syntax. For example, we don't support +// union ("x|y"), grouping ("(xy)"), brackets ("[xy]"), and +// repetition count ("x{5,7}"), among others. +// +// Below is the syntax that we do support. We chose it to be a +// subset of both PCRE and POSIX extended regex, so it's easy to +// learn wherever you come from. In the following: 'A' denotes a +// literal character, period (.), or a single \\ escape sequence; +// 'x' and 'y' denote regular expressions; 'm' and 'n' are for +// natural numbers. +// +// c matches any literal character c +// \\d matches any decimal digit +// \\D matches any character that's not a decimal digit +// \\f matches \f +// \\n matches \n +// \\r matches \r +// \\s matches any ASCII whitespace, including \n +// \\S matches any character that's not a whitespace +// \\t matches \t +// \\v matches \v +// \\w matches any letter, _, or decimal digit +// \\W matches any character that \\w doesn't match +// \\c matches any literal character c, which must be a punctuation +// . matches any single character except \n +// A? matches 0 or 1 occurrences of A +// A* matches 0 or many occurrences of A +// A+ matches 1 or many occurrences of A +// ^ matches the beginning of a string (not that of each line) +// $ matches the end of a string (not that of each line) +// xy matches x followed by y +// +// If you accidentally use PCRE or POSIX extended regex features +// not implemented by us, you will get a run-time failure. In that +// case, please try to rewrite your regular expression within the +// above syntax. +// +// This implementation is *not* meant to be as highly tuned or robust +// as a compiled regex library, but should perform well enough for a +// death test, which already incurs significant overhead by launching +// a child process. +// +// Known caveats: +// +// A "threadsafe" style death test obtains the path to the test +// program from argv[0] and re-executes it in the sub-process. For +// simplicity, the current implementation doesn't search the PATH +// when launching the sub-process. This means that the user must +// invoke the test program via a path that contains at least one +// path separator (e.g. path/to/foo_test and +// /absolute/path/to/bar_test are fine, but foo_test is not). This +// is rarely a problem as people usually don't put the test binary +// directory in PATH. +// +// TODO(wan@google.com): make thread-safe death tests search the PATH. + +// Asserts that a given statement causes the program to exit, with an +// integer exit status that satisfies predicate, and emitting error output +// that matches regex. +# define ASSERT_EXIT(statement, predicate, regex) \ + GTEST_DEATH_TEST_(statement, predicate, regex, GTEST_FATAL_FAILURE_) + +// Like ASSERT_EXIT, but continues on to successive tests in the +// test case, if any: +# define EXPECT_EXIT(statement, predicate, regex) \ + GTEST_DEATH_TEST_(statement, predicate, regex, GTEST_NONFATAL_FAILURE_) + +// Asserts that a given statement causes the program to exit, either by +// explicitly exiting with a nonzero exit code or being killed by a +// signal, and emitting error output that matches regex. +# define ASSERT_DEATH(statement, regex) \ + ASSERT_EXIT(statement, ::testing::internal::ExitedUnsuccessfully, regex) + +// Like ASSERT_DEATH, but continues on to successive tests in the +// test case, if any: +# define EXPECT_DEATH(statement, regex) \ + EXPECT_EXIT(statement, ::testing::internal::ExitedUnsuccessfully, regex) + +// Two predicate classes that can be used in {ASSERT,EXPECT}_EXIT*: + +// Tests that an exit code describes a normal exit with a given exit code. +class GTEST_API_ ExitedWithCode { + public: + explicit ExitedWithCode(int exit_code); + bool operator()(int exit_status) const; + private: + // No implementation - assignment is unsupported. + void operator=(const ExitedWithCode& other); + + const int exit_code_; +}; + +# if !GTEST_OS_WINDOWS +// Tests that an exit code describes an exit due to termination by a +// given signal. +class GTEST_API_ KilledBySignal { + public: + explicit KilledBySignal(int signum); + bool operator()(int exit_status) const; + private: + const int signum_; +}; +# endif // !GTEST_OS_WINDOWS + +// EXPECT_DEBUG_DEATH asserts that the given statements die in debug mode. +// The death testing framework causes this to have interesting semantics, +// since the sideeffects of the call are only visible in opt mode, and not +// in debug mode. +// +// In practice, this can be used to test functions that utilize the +// LOG(DFATAL) macro using the following style: +// +// int DieInDebugOr12(int* sideeffect) { +// if (sideeffect) { +// *sideeffect = 12; +// } +// LOG(DFATAL) << "death"; +// return 12; +// } +// +// TEST(TestCase, TestDieOr12WorksInDgbAndOpt) { +// int sideeffect = 0; +// // Only asserts in dbg. +// EXPECT_DEBUG_DEATH(DieInDebugOr12(&sideeffect), "death"); +// +// #ifdef NDEBUG +// // opt-mode has sideeffect visible. +// EXPECT_EQ(12, sideeffect); +// #else +// // dbg-mode no visible sideeffect. +// EXPECT_EQ(0, sideeffect); +// #endif +// } +// +// This will assert that DieInDebugReturn12InOpt() crashes in debug +// mode, usually due to a DCHECK or LOG(DFATAL), but returns the +// appropriate fallback value (12 in this case) in opt mode. If you +// need to test that a function has appropriate side-effects in opt +// mode, include assertions against the side-effects. A general +// pattern for this is: +// +// EXPECT_DEBUG_DEATH({ +// // Side-effects here will have an effect after this statement in +// // opt mode, but none in debug mode. +// EXPECT_EQ(12, DieInDebugOr12(&sideeffect)); +// }, "death"); +// +# ifdef NDEBUG + +# define EXPECT_DEBUG_DEATH(statement, regex) \ + GTEST_EXECUTE_STATEMENT_(statement, regex) + +# define ASSERT_DEBUG_DEATH(statement, regex) \ + GTEST_EXECUTE_STATEMENT_(statement, regex) + +# else + +# define EXPECT_DEBUG_DEATH(statement, regex) \ + EXPECT_DEATH(statement, regex) + +# define ASSERT_DEBUG_DEATH(statement, regex) \ + ASSERT_DEATH(statement, regex) + +# endif // NDEBUG for EXPECT_DEBUG_DEATH +#endif // GTEST_HAS_DEATH_TEST + +// EXPECT_DEATH_IF_SUPPORTED(statement, regex) and +// ASSERT_DEATH_IF_SUPPORTED(statement, regex) expand to real death tests if +// death tests are supported; otherwise they just issue a warning. This is +// useful when you are combining death test assertions with normal test +// assertions in one test. +#if GTEST_HAS_DEATH_TEST +# define EXPECT_DEATH_IF_SUPPORTED(statement, regex) \ + EXPECT_DEATH(statement, regex) +# define ASSERT_DEATH_IF_SUPPORTED(statement, regex) \ + ASSERT_DEATH(statement, regex) +#else +# define EXPECT_DEATH_IF_SUPPORTED(statement, regex) \ + GTEST_UNSUPPORTED_DEATH_TEST_(statement, regex, ) +# define ASSERT_DEATH_IF_SUPPORTED(statement, regex) \ + GTEST_UNSUPPORTED_DEATH_TEST_(statement, regex, return) +#endif + +} // namespace testing + +#endif // GTEST_INCLUDE_GTEST_GTEST_DEATH_TEST_H_ +// This file was GENERATED by command: +// pump.py gtest-param-test.h.pump +// DO NOT EDIT BY HAND!!! + +// Copyright 2008, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Authors: vladl@google.com (Vlad Losev) +// +// Macros and functions for implementing parameterized tests +// in Google C++ Testing Framework (Google Test) +// +// This file is generated by a SCRIPT. DO NOT EDIT BY HAND! +// +#ifndef GTEST_INCLUDE_GTEST_GTEST_PARAM_TEST_H_ +#define GTEST_INCLUDE_GTEST_GTEST_PARAM_TEST_H_ + + +// Value-parameterized tests allow you to test your code with different +// parameters without writing multiple copies of the same test. +// +// Here is how you use value-parameterized tests: + +#if 0 + +// To write value-parameterized tests, first you should define a fixture +// class. It is usually derived from testing::TestWithParam (see below for +// another inheritance scheme that's sometimes useful in more complicated +// class hierarchies), where the type of your parameter values. +// TestWithParam is itself derived from testing::Test. T can be any +// copyable type. If it's a raw pointer, you are responsible for managing the +// lifespan of the pointed values. + +class FooTest : public ::testing::TestWithParam { + // You can implement all the usual class fixture members here. +}; + +// Then, use the TEST_P macro to define as many parameterized tests +// for this fixture as you want. The _P suffix is for "parameterized" +// or "pattern", whichever you prefer to think. + +TEST_P(FooTest, DoesBlah) { + // Inside a test, access the test parameter with the GetParam() method + // of the TestWithParam class: + EXPECT_TRUE(foo.Blah(GetParam())); + ... +} + +TEST_P(FooTest, HasBlahBlah) { + ... +} + +// Finally, you can use INSTANTIATE_TEST_CASE_P to instantiate the test +// case with any set of parameters you want. Google Test defines a number +// of functions for generating test parameters. They return what we call +// (surprise!) parameter generators. Here is a summary of them, which +// are all in the testing namespace: +// +// +// Range(begin, end [, step]) - Yields values {begin, begin+step, +// begin+step+step, ...}. The values do not +// include end. step defaults to 1. +// Values(v1, v2, ..., vN) - Yields values {v1, v2, ..., vN}. +// ValuesIn(container) - Yields values from a C-style array, an STL +// ValuesIn(begin,end) container, or an iterator range [begin, end). +// Bool() - Yields sequence {false, true}. +// Combine(g1, g2, ..., gN) - Yields all combinations (the Cartesian product +// for the math savvy) of the values generated +// by the N generators. +// +// For more details, see comments at the definitions of these functions below +// in this file. +// +// The following statement will instantiate tests from the FooTest test case +// each with parameter values "meeny", "miny", and "moe". + +INSTANTIATE_TEST_CASE_P(InstantiationName, + FooTest, + Values("meeny", "miny", "moe")); + +// To distinguish different instances of the pattern, (yes, you +// can instantiate it more then once) the first argument to the +// INSTANTIATE_TEST_CASE_P macro is a prefix that will be added to the +// actual test case name. Remember to pick unique prefixes for different +// instantiations. The tests from the instantiation above will have +// these names: +// +// * InstantiationName/FooTest.DoesBlah/0 for "meeny" +// * InstantiationName/FooTest.DoesBlah/1 for "miny" +// * InstantiationName/FooTest.DoesBlah/2 for "moe" +// * InstantiationName/FooTest.HasBlahBlah/0 for "meeny" +// * InstantiationName/FooTest.HasBlahBlah/1 for "miny" +// * InstantiationName/FooTest.HasBlahBlah/2 for "moe" +// +// You can use these names in --gtest_filter. +// +// This statement will instantiate all tests from FooTest again, each +// with parameter values "cat" and "dog": + +const char* pets[] = {"cat", "dog"}; +INSTANTIATE_TEST_CASE_P(AnotherInstantiationName, FooTest, ValuesIn(pets)); + +// The tests from the instantiation above will have these names: +// +// * AnotherInstantiationName/FooTest.DoesBlah/0 for "cat" +// * AnotherInstantiationName/FooTest.DoesBlah/1 for "dog" +// * AnotherInstantiationName/FooTest.HasBlahBlah/0 for "cat" +// * AnotherInstantiationName/FooTest.HasBlahBlah/1 for "dog" +// +// Please note that INSTANTIATE_TEST_CASE_P will instantiate all tests +// in the given test case, whether their definitions come before or +// AFTER the INSTANTIATE_TEST_CASE_P statement. +// +// Please also note that generator expressions (including parameters to the +// generators) are evaluated in InitGoogleTest(), after main() has started. +// This allows the user on one hand, to adjust generator parameters in order +// to dynamically determine a set of tests to run and on the other hand, +// give the user a chance to inspect the generated tests with Google Test +// reflection API before RUN_ALL_TESTS() is executed. +// +// You can see samples/sample7_unittest.cc and samples/sample8_unittest.cc +// for more examples. +// +// In the future, we plan to publish the API for defining new parameter +// generators. But for now this interface remains part of the internal +// implementation and is subject to change. +// +// +// A parameterized test fixture must be derived from testing::Test and from +// testing::WithParamInterface, where T is the type of the parameter +// values. Inheriting from TestWithParam satisfies that requirement because +// TestWithParam inherits from both Test and WithParamInterface. In more +// complicated hierarchies, however, it is occasionally useful to inherit +// separately from Test and WithParamInterface. For example: + +class BaseTest : public ::testing::Test { + // You can inherit all the usual members for a non-parameterized test + // fixture here. +}; + +class DerivedTest : public BaseTest, public ::testing::WithParamInterface { + // The usual test fixture members go here too. +}; + +TEST_F(BaseTest, HasFoo) { + // This is an ordinary non-parameterized test. +} + +TEST_P(DerivedTest, DoesBlah) { + // GetParam works just the same here as if you inherit from TestWithParam. + EXPECT_TRUE(foo.Blah(GetParam())); +} + +#endif // 0 + + +#if !GTEST_OS_SYMBIAN +# include +#endif + +// scripts/fuse_gtest.py depends on gtest's own header being #included +// *unconditionally*. Therefore these #includes cannot be moved +// inside #if GTEST_HAS_PARAM_TEST. +// Copyright 2008 Google Inc. +// All Rights Reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: vladl@google.com (Vlad Losev) + +// Type and function utilities for implementing parameterized tests. + +#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_H_ +#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_H_ + +#include +#include +#include + +// scripts/fuse_gtest.py depends on gtest's own header being #included +// *unconditionally*. Therefore these #includes cannot be moved +// inside #if GTEST_HAS_PARAM_TEST. +// Copyright 2003 Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Authors: Dan Egnor (egnor@google.com) +// +// A "smart" pointer type with reference tracking. Every pointer to a +// particular object is kept on a circular linked list. When the last pointer +// to an object is destroyed or reassigned, the object is deleted. +// +// Used properly, this deletes the object when the last reference goes away. +// There are several caveats: +// - Like all reference counting schemes, cycles lead to leaks. +// - Each smart pointer is actually two pointers (8 bytes instead of 4). +// - Every time a pointer is assigned, the entire list of pointers to that +// object is traversed. This class is therefore NOT SUITABLE when there +// will often be more than two or three pointers to a particular object. +// - References are only tracked as long as linked_ptr<> objects are copied. +// If a linked_ptr<> is converted to a raw pointer and back, BAD THINGS +// will happen (double deletion). +// +// A good use of this class is storing object references in STL containers. +// You can safely put linked_ptr<> in a vector<>. +// Other uses may not be as good. +// +// Note: If you use an incomplete type with linked_ptr<>, the class +// *containing* linked_ptr<> must have a constructor and destructor (even +// if they do nothing!). +// +// Bill Gibbons suggested we use something like this. +// +// Thread Safety: +// Unlike other linked_ptr implementations, in this implementation +// a linked_ptr object is thread-safe in the sense that: +// - it's safe to copy linked_ptr objects concurrently, +// - it's safe to copy *from* a linked_ptr and read its underlying +// raw pointer (e.g. via get()) concurrently, and +// - it's safe to write to two linked_ptrs that point to the same +// shared object concurrently. +// TODO(wan@google.com): rename this to safe_linked_ptr to avoid +// confusion with normal linked_ptr. + +#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_LINKED_PTR_H_ +#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_LINKED_PTR_H_ + +#include +#include + + +namespace testing { +namespace internal { + +// Protects copying of all linked_ptr objects. +GTEST_API_ GTEST_DECLARE_STATIC_MUTEX_(g_linked_ptr_mutex); + +// This is used internally by all instances of linked_ptr<>. It needs to be +// a non-template class because different types of linked_ptr<> can refer to +// the same object (linked_ptr(obj) vs linked_ptr(obj)). +// So, it needs to be possible for different types of linked_ptr to participate +// in the same circular linked list, so we need a single class type here. +// +// DO NOT USE THIS CLASS DIRECTLY YOURSELF. Use linked_ptr. +class linked_ptr_internal { + public: + // Create a new circle that includes only this instance. + void join_new() { + next_ = this; + } + + // Many linked_ptr operations may change p.link_ for some linked_ptr + // variable p in the same circle as this object. Therefore we need + // to prevent two such operations from occurring concurrently. + // + // Note that different types of linked_ptr objects can coexist in a + // circle (e.g. linked_ptr, linked_ptr, and + // linked_ptr). Therefore we must use a single mutex to + // protect all linked_ptr objects. This can create serious + // contention in production code, but is acceptable in a testing + // framework. + + // Join an existing circle. + void join(linked_ptr_internal const* ptr) + GTEST_LOCK_EXCLUDED_(g_linked_ptr_mutex) { + MutexLock lock(&g_linked_ptr_mutex); + + linked_ptr_internal const* p = ptr; + while (p->next_ != ptr) { + assert(p->next_ != this && + "Trying to join() a linked ring we are already in. " + "Is GMock thread safety enabled?"); + p = p->next_; + } + p->next_ = this; + next_ = ptr; + } + + // Leave whatever circle we're part of. Returns true if we were the + // last member of the circle. Once this is done, you can join() another. + bool depart() + GTEST_LOCK_EXCLUDED_(g_linked_ptr_mutex) { + MutexLock lock(&g_linked_ptr_mutex); + + if (next_ == this) return true; + linked_ptr_internal const* p = next_; + while (p->next_ != this) { + assert(p->next_ != next_ && + "Trying to depart() a linked ring we are not in. " + "Is GMock thread safety enabled?"); + p = p->next_; + } + p->next_ = next_; + return false; + } + + private: + mutable linked_ptr_internal const* next_; +}; + +template +class linked_ptr { + public: + typedef T element_type; + + // Take over ownership of a raw pointer. This should happen as soon as + // possible after the object is created. + explicit linked_ptr(T* ptr = NULL) { capture(ptr); } + ~linked_ptr() { depart(); } + + // Copy an existing linked_ptr<>, adding ourselves to the list of references. + template linked_ptr(linked_ptr const& ptr) { copy(&ptr); } + linked_ptr(linked_ptr const& ptr) { // NOLINT + assert(&ptr != this); + copy(&ptr); + } + + // Assignment releases the old value and acquires the new. + template linked_ptr& operator=(linked_ptr const& ptr) { + depart(); + copy(&ptr); + return *this; + } + + linked_ptr& operator=(linked_ptr const& ptr) { + if (&ptr != this) { + depart(); + copy(&ptr); + } + return *this; + } + + // Smart pointer members. + void reset(T* ptr = NULL) { + depart(); + capture(ptr); + } + T* get() const { return value_; } + T* operator->() const { return value_; } + T& operator*() const { return *value_; } + + bool operator==(T* p) const { return value_ == p; } + bool operator!=(T* p) const { return value_ != p; } + template + bool operator==(linked_ptr const& ptr) const { + return value_ == ptr.get(); + } + template + bool operator!=(linked_ptr const& ptr) const { + return value_ != ptr.get(); + } + + private: + template + friend class linked_ptr; + + T* value_; + linked_ptr_internal link_; + + void depart() { + if (link_.depart()) delete value_; + } + + void capture(T* ptr) { + value_ = ptr; + link_.join_new(); + } + + template void copy(linked_ptr const* ptr) { + value_ = ptr->get(); + if (value_) + link_.join(&ptr->link_); + else + link_.join_new(); + } +}; + +template inline +bool operator==(T* ptr, const linked_ptr& x) { + return ptr == x.get(); +} + +template inline +bool operator!=(T* ptr, const linked_ptr& x) { + return ptr != x.get(); +} + +// A function to convert T* into linked_ptr +// Doing e.g. make_linked_ptr(new FooBarBaz(arg)) is a shorter notation +// for linked_ptr >(new FooBarBaz(arg)) +template +linked_ptr make_linked_ptr(T* ptr) { + return linked_ptr(ptr); +} + +} // namespace internal +} // namespace testing + +#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_LINKED_PTR_H_ +// Copyright 2007, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) + +// Google Test - The Google C++ Testing Framework +// +// This file implements a universal value printer that can print a +// value of any type T: +// +// void ::testing::internal::UniversalPrinter::Print(value, ostream_ptr); +// +// A user can teach this function how to print a class type T by +// defining either operator<<() or PrintTo() in the namespace that +// defines T. More specifically, the FIRST defined function in the +// following list will be used (assuming T is defined in namespace +// foo): +// +// 1. foo::PrintTo(const T&, ostream*) +// 2. operator<<(ostream&, const T&) defined in either foo or the +// global namespace. +// +// If none of the above is defined, it will print the debug string of +// the value if it is a protocol buffer, or print the raw bytes in the +// value otherwise. +// +// To aid debugging: when T is a reference type, the address of the +// value is also printed; when T is a (const) char pointer, both the +// pointer value and the NUL-terminated string it points to are +// printed. +// +// We also provide some convenient wrappers: +// +// // Prints a value to a string. For a (const or not) char +// // pointer, the NUL-terminated string (but not the pointer) is +// // printed. +// std::string ::testing::PrintToString(const T& value); +// +// // Prints a value tersely: for a reference type, the referenced +// // value (but not the address) is printed; for a (const or not) char +// // pointer, the NUL-terminated string (but not the pointer) is +// // printed. +// void ::testing::internal::UniversalTersePrint(const T& value, ostream*); +// +// // Prints value using the type inferred by the compiler. The difference +// // from UniversalTersePrint() is that this function prints both the +// // pointer and the NUL-terminated string for a (const or not) char pointer. +// void ::testing::internal::UniversalPrint(const T& value, ostream*); +// +// // Prints the fields of a tuple tersely to a string vector, one +// // element for each field. Tuple support must be enabled in +// // gtest-port.h. +// std::vector UniversalTersePrintTupleFieldsToStrings( +// const Tuple& value); +// +// Known limitation: +// +// The print primitives print the elements of an STL-style container +// using the compiler-inferred type of *iter where iter is a +// const_iterator of the container. When const_iterator is an input +// iterator but not a forward iterator, this inferred type may not +// match value_type, and the print output may be incorrect. In +// practice, this is rarely a problem as for most containers +// const_iterator is a forward iterator. We'll fix this if there's an +// actual need for it. Note that this fix cannot rely on value_type +// being defined as many user-defined container types don't have +// value_type. + +#ifndef GTEST_INCLUDE_GTEST_GTEST_PRINTERS_H_ +#define GTEST_INCLUDE_GTEST_GTEST_PRINTERS_H_ + +#include // NOLINT +#include +#include +#include +#include + +#if GTEST_HAS_STD_TUPLE_ +# include +#endif + +namespace testing { + +// Definitions in the 'internal' and 'internal2' name spaces are +// subject to change without notice. DO NOT USE THEM IN USER CODE! +namespace internal2 { + +// Prints the given number of bytes in the given object to the given +// ostream. +GTEST_API_ void PrintBytesInObjectTo(const unsigned char* obj_bytes, + size_t count, + ::std::ostream* os); + +// For selecting which printer to use when a given type has neither << +// nor PrintTo(). +enum TypeKind { + kProtobuf, // a protobuf type + kConvertibleToInteger, // a type implicitly convertible to BiggestInt + // (e.g. a named or unnamed enum type) + kOtherType // anything else +}; + +// TypeWithoutFormatter::PrintValue(value, os) is called +// by the universal printer to print a value of type T when neither +// operator<< nor PrintTo() is defined for T, where kTypeKind is the +// "kind" of T as defined by enum TypeKind. +template +class TypeWithoutFormatter { + public: + // This default version is called when kTypeKind is kOtherType. + static void PrintValue(const T& value, ::std::ostream* os) { + PrintBytesInObjectTo(reinterpret_cast(&value), + sizeof(value), os); + } +}; + +// We print a protobuf using its ShortDebugString() when the string +// doesn't exceed this many characters; otherwise we print it using +// DebugString() for better readability. +const size_t kProtobufOneLinerMaxLength = 50; + +template +class TypeWithoutFormatter { + public: + static void PrintValue(const T& value, ::std::ostream* os) { + const ::testing::internal::string short_str = value.ShortDebugString(); + const ::testing::internal::string pretty_str = + short_str.length() <= kProtobufOneLinerMaxLength ? + short_str : ("\n" + value.DebugString()); + *os << ("<" + pretty_str + ">"); + } +}; + +template +class TypeWithoutFormatter { + public: + // Since T has no << operator or PrintTo() but can be implicitly + // converted to BiggestInt, we print it as a BiggestInt. + // + // Most likely T is an enum type (either named or unnamed), in which + // case printing it as an integer is the desired behavior. In case + // T is not an enum, printing it as an integer is the best we can do + // given that it has no user-defined printer. + static void PrintValue(const T& value, ::std::ostream* os) { + const internal::BiggestInt kBigInt = value; + *os << kBigInt; + } +}; + +// Prints the given value to the given ostream. If the value is a +// protocol message, its debug string is printed; if it's an enum or +// of a type implicitly convertible to BiggestInt, it's printed as an +// integer; otherwise the bytes in the value are printed. This is +// what UniversalPrinter::Print() does when it knows nothing about +// type T and T has neither << operator nor PrintTo(). +// +// A user can override this behavior for a class type Foo by defining +// a << operator in the namespace where Foo is defined. +// +// We put this operator in namespace 'internal2' instead of 'internal' +// to simplify the implementation, as much code in 'internal' needs to +// use << in STL, which would conflict with our own << were it defined +// in 'internal'. +// +// Note that this operator<< takes a generic std::basic_ostream type instead of the more restricted std::ostream. If +// we define it to take an std::ostream instead, we'll get an +// "ambiguous overloads" compiler error when trying to print a type +// Foo that supports streaming to std::basic_ostream, as the compiler cannot tell whether +// operator<<(std::ostream&, const T&) or +// operator<<(std::basic_stream, const Foo&) is more +// specific. +template +::std::basic_ostream& operator<<( + ::std::basic_ostream& os, const T& x) { + TypeWithoutFormatter::value ? kProtobuf : + internal::ImplicitlyConvertible::value ? + kConvertibleToInteger : kOtherType)>::PrintValue(x, &os); + return os; +} + +} // namespace internal2 +} // namespace testing + +// This namespace MUST NOT BE NESTED IN ::testing, or the name look-up +// magic needed for implementing UniversalPrinter won't work. +namespace testing_internal { + +// Used to print a value that is not an STL-style container when the +// user doesn't define PrintTo() for it. +template +void DefaultPrintNonContainerTo(const T& value, ::std::ostream* os) { + // With the following statement, during unqualified name lookup, + // testing::internal2::operator<< appears as if it was declared in + // the nearest enclosing namespace that contains both + // ::testing_internal and ::testing::internal2, i.e. the global + // namespace. For more details, refer to the C++ Standard section + // 7.3.4-1 [namespace.udir]. This allows us to fall back onto + // testing::internal2::operator<< in case T doesn't come with a << + // operator. + // + // We cannot write 'using ::testing::internal2::operator<<;', which + // gcc 3.3 fails to compile due to a compiler bug. + using namespace ::testing::internal2; // NOLINT + + // Assuming T is defined in namespace foo, in the next statement, + // the compiler will consider all of: + // + // 1. foo::operator<< (thanks to Koenig look-up), + // 2. ::operator<< (as the current namespace is enclosed in ::), + // 3. testing::internal2::operator<< (thanks to the using statement above). + // + // The operator<< whose type matches T best will be picked. + // + // We deliberately allow #2 to be a candidate, as sometimes it's + // impossible to define #1 (e.g. when foo is ::std, defining + // anything in it is undefined behavior unless you are a compiler + // vendor.). + *os << value; +} + +} // namespace testing_internal + +namespace testing { +namespace internal { + +// UniversalPrinter::Print(value, ostream_ptr) prints the given +// value to the given ostream. The caller must ensure that +// 'ostream_ptr' is not NULL, or the behavior is undefined. +// +// We define UniversalPrinter as a class template (as opposed to a +// function template), as we need to partially specialize it for +// reference types, which cannot be done with function templates. +template +class UniversalPrinter; + +template +void UniversalPrint(const T& value, ::std::ostream* os); + +// Used to print an STL-style container when the user doesn't define +// a PrintTo() for it. +template +void DefaultPrintTo(IsContainer /* dummy */, + false_type /* is not a pointer */, + const C& container, ::std::ostream* os) { + const size_t kMaxCount = 32; // The maximum number of elements to print. + *os << '{'; + size_t count = 0; + for (typename C::const_iterator it = container.begin(); + it != container.end(); ++it, ++count) { + if (count > 0) { + *os << ','; + if (count == kMaxCount) { // Enough has been printed. + *os << " ..."; + break; + } + } + *os << ' '; + // We cannot call PrintTo(*it, os) here as PrintTo() doesn't + // handle *it being a native array. + internal::UniversalPrint(*it, os); + } + + if (count > 0) { + *os << ' '; + } + *os << '}'; +} + +// Used to print a pointer that is neither a char pointer nor a member +// pointer, when the user doesn't define PrintTo() for it. (A member +// variable pointer or member function pointer doesn't really point to +// a location in the address space. Their representation is +// implementation-defined. Therefore they will be printed as raw +// bytes.) +template +void DefaultPrintTo(IsNotContainer /* dummy */, + true_type /* is a pointer */, + T* p, ::std::ostream* os) { + if (p == NULL) { + *os << "NULL"; + } else { + // C++ doesn't allow casting from a function pointer to any object + // pointer. + // + // IsTrue() silences warnings: "Condition is always true", + // "unreachable code". + if (IsTrue(ImplicitlyConvertible::value)) { + // T is not a function type. We just call << to print p, + // relying on ADL to pick up user-defined << for their pointer + // types, if any. + *os << p; + } else { + // T is a function type, so '*os << p' doesn't do what we want + // (it just prints p as bool). We want to print p as a const + // void*. However, we cannot cast it to const void* directly, + // even using reinterpret_cast, as earlier versions of gcc + // (e.g. 3.4.5) cannot compile the cast when p is a function + // pointer. Casting to UInt64 first solves the problem. + *os << reinterpret_cast( + reinterpret_cast(p)); + } + } +} + +// Used to print a non-container, non-pointer value when the user +// doesn't define PrintTo() for it. +template +void DefaultPrintTo(IsNotContainer /* dummy */, + false_type /* is not a pointer */, + const T& value, ::std::ostream* os) { + ::testing_internal::DefaultPrintNonContainerTo(value, os); +} + +// Prints the given value using the << operator if it has one; +// otherwise prints the bytes in it. This is what +// UniversalPrinter::Print() does when PrintTo() is not specialized +// or overloaded for type T. +// +// A user can override this behavior for a class type Foo by defining +// an overload of PrintTo() in the namespace where Foo is defined. We +// give the user this option as sometimes defining a << operator for +// Foo is not desirable (e.g. the coding style may prevent doing it, +// or there is already a << operator but it doesn't do what the user +// wants). +template +void PrintTo(const T& value, ::std::ostream* os) { + // DefaultPrintTo() is overloaded. The type of its first two + // arguments determine which version will be picked. If T is an + // STL-style container, the version for container will be called; if + // T is a pointer, the pointer version will be called; otherwise the + // generic version will be called. + // + // Note that we check for container types here, prior to we check + // for protocol message types in our operator<<. The rationale is: + // + // For protocol messages, we want to give people a chance to + // override Google Mock's format by defining a PrintTo() or + // operator<<. For STL containers, other formats can be + // incompatible with Google Mock's format for the container + // elements; therefore we check for container types here to ensure + // that our format is used. + // + // The second argument of DefaultPrintTo() is needed to bypass a bug + // in Symbian's C++ compiler that prevents it from picking the right + // overload between: + // + // PrintTo(const T& x, ...); + // PrintTo(T* x, ...); + DefaultPrintTo(IsContainerTest(0), is_pointer(), value, os); +} + +// The following list of PrintTo() overloads tells +// UniversalPrinter::Print() how to print standard types (built-in +// types, strings, plain arrays, and pointers). + +// Overloads for various char types. +GTEST_API_ void PrintTo(unsigned char c, ::std::ostream* os); +GTEST_API_ void PrintTo(signed char c, ::std::ostream* os); +inline void PrintTo(char c, ::std::ostream* os) { + // When printing a plain char, we always treat it as unsigned. This + // way, the output won't be affected by whether the compiler thinks + // char is signed or not. + PrintTo(static_cast(c), os); +} + +// Overloads for other simple built-in types. +inline void PrintTo(bool x, ::std::ostream* os) { + *os << (x ? "true" : "false"); +} + +// Overload for wchar_t type. +// Prints a wchar_t as a symbol if it is printable or as its internal +// code otherwise and also as its decimal code (except for L'\0'). +// The L'\0' char is printed as "L'\\0'". The decimal code is printed +// as signed integer when wchar_t is implemented by the compiler +// as a signed type and is printed as an unsigned integer when wchar_t +// is implemented as an unsigned type. +GTEST_API_ void PrintTo(wchar_t wc, ::std::ostream* os); + +// Overloads for C strings. +GTEST_API_ void PrintTo(const char* s, ::std::ostream* os); +inline void PrintTo(char* s, ::std::ostream* os) { + PrintTo(ImplicitCast_(s), os); +} + +// signed/unsigned char is often used for representing binary data, so +// we print pointers to it as void* to be safe. +inline void PrintTo(const signed char* s, ::std::ostream* os) { + PrintTo(ImplicitCast_(s), os); +} +inline void PrintTo(signed char* s, ::std::ostream* os) { + PrintTo(ImplicitCast_(s), os); +} +inline void PrintTo(const unsigned char* s, ::std::ostream* os) { + PrintTo(ImplicitCast_(s), os); +} +inline void PrintTo(unsigned char* s, ::std::ostream* os) { + PrintTo(ImplicitCast_(s), os); +} + +// MSVC can be configured to define wchar_t as a typedef of unsigned +// short. It defines _NATIVE_WCHAR_T_DEFINED when wchar_t is a native +// type. When wchar_t is a typedef, defining an overload for const +// wchar_t* would cause unsigned short* be printed as a wide string, +// possibly causing invalid memory accesses. +#if !defined(_MSC_VER) || defined(_NATIVE_WCHAR_T_DEFINED) +// Overloads for wide C strings +GTEST_API_ void PrintTo(const wchar_t* s, ::std::ostream* os); +inline void PrintTo(wchar_t* s, ::std::ostream* os) { + PrintTo(ImplicitCast_(s), os); +} +#endif + +// Overload for C arrays. Multi-dimensional arrays are printed +// properly. + +// Prints the given number of elements in an array, without printing +// the curly braces. +template +void PrintRawArrayTo(const T a[], size_t count, ::std::ostream* os) { + UniversalPrint(a[0], os); + for (size_t i = 1; i != count; i++) { + *os << ", "; + UniversalPrint(a[i], os); + } +} + +// Overloads for ::string and ::std::string. +#if GTEST_HAS_GLOBAL_STRING +GTEST_API_ void PrintStringTo(const ::string&s, ::std::ostream* os); +inline void PrintTo(const ::string& s, ::std::ostream* os) { + PrintStringTo(s, os); +} +#endif // GTEST_HAS_GLOBAL_STRING + +GTEST_API_ void PrintStringTo(const ::std::string&s, ::std::ostream* os); +inline void PrintTo(const ::std::string& s, ::std::ostream* os) { + PrintStringTo(s, os); +} + +// Overloads for ::wstring and ::std::wstring. +#if GTEST_HAS_GLOBAL_WSTRING +GTEST_API_ void PrintWideStringTo(const ::wstring&s, ::std::ostream* os); +inline void PrintTo(const ::wstring& s, ::std::ostream* os) { + PrintWideStringTo(s, os); +} +#endif // GTEST_HAS_GLOBAL_WSTRING + +#if GTEST_HAS_STD_WSTRING +GTEST_API_ void PrintWideStringTo(const ::std::wstring&s, ::std::ostream* os); +inline void PrintTo(const ::std::wstring& s, ::std::ostream* os) { + PrintWideStringTo(s, os); +} +#endif // GTEST_HAS_STD_WSTRING + +#if GTEST_HAS_TR1_TUPLE || GTEST_HAS_STD_TUPLE_ +// Helper function for printing a tuple. T must be instantiated with +// a tuple type. +template +void PrintTupleTo(const T& t, ::std::ostream* os); +#endif // GTEST_HAS_TR1_TUPLE || GTEST_HAS_STD_TUPLE_ + +#if GTEST_HAS_TR1_TUPLE +// Overload for ::std::tr1::tuple. Needed for printing function arguments, +// which are packed as tuples. + +// Overloaded PrintTo() for tuples of various arities. We support +// tuples of up-to 10 fields. The following implementation works +// regardless of whether tr1::tuple is implemented using the +// non-standard variadic template feature or not. + +inline void PrintTo(const ::std::tr1::tuple<>& t, ::std::ostream* os) { + PrintTupleTo(t, os); +} + +template +void PrintTo(const ::std::tr1::tuple& t, ::std::ostream* os) { + PrintTupleTo(t, os); +} + +template +void PrintTo(const ::std::tr1::tuple& t, ::std::ostream* os) { + PrintTupleTo(t, os); +} + +template +void PrintTo(const ::std::tr1::tuple& t, ::std::ostream* os) { + PrintTupleTo(t, os); +} + +template +void PrintTo(const ::std::tr1::tuple& t, ::std::ostream* os) { + PrintTupleTo(t, os); +} + +template +void PrintTo(const ::std::tr1::tuple& t, + ::std::ostream* os) { + PrintTupleTo(t, os); +} + +template +void PrintTo(const ::std::tr1::tuple& t, + ::std::ostream* os) { + PrintTupleTo(t, os); +} + +template +void PrintTo(const ::std::tr1::tuple& t, + ::std::ostream* os) { + PrintTupleTo(t, os); +} + +template +void PrintTo(const ::std::tr1::tuple& t, + ::std::ostream* os) { + PrintTupleTo(t, os); +} + +template +void PrintTo(const ::std::tr1::tuple& t, + ::std::ostream* os) { + PrintTupleTo(t, os); +} + +template +void PrintTo( + const ::std::tr1::tuple& t, + ::std::ostream* os) { + PrintTupleTo(t, os); +} +#endif // GTEST_HAS_TR1_TUPLE + +#if GTEST_HAS_STD_TUPLE_ +template +void PrintTo(const ::std::tuple& t, ::std::ostream* os) { + PrintTupleTo(t, os); +} +#endif // GTEST_HAS_STD_TUPLE_ + +// Overload for std::pair. +template +void PrintTo(const ::std::pair& value, ::std::ostream* os) { + *os << '('; + // We cannot use UniversalPrint(value.first, os) here, as T1 may be + // a reference type. The same for printing value.second. + UniversalPrinter::Print(value.first, os); + *os << ", "; + UniversalPrinter::Print(value.second, os); + *os << ')'; +} + +// Implements printing a non-reference type T by letting the compiler +// pick the right overload of PrintTo() for T. +template +class UniversalPrinter { + public: + // MSVC warns about adding const to a function type, so we want to + // disable the warning. + GTEST_DISABLE_MSC_WARNINGS_PUSH_(4180) + + // Note: we deliberately don't call this PrintTo(), as that name + // conflicts with ::testing::internal::PrintTo in the body of the + // function. + static void Print(const T& value, ::std::ostream* os) { + // By default, ::testing::internal::PrintTo() is used for printing + // the value. + // + // Thanks to Koenig look-up, if T is a class and has its own + // PrintTo() function defined in its namespace, that function will + // be visible here. Since it is more specific than the generic ones + // in ::testing::internal, it will be picked by the compiler in the + // following statement - exactly what we want. + PrintTo(value, os); + } + + GTEST_DISABLE_MSC_WARNINGS_POP_() +}; + +// UniversalPrintArray(begin, len, os) prints an array of 'len' +// elements, starting at address 'begin'. +template +void UniversalPrintArray(const T* begin, size_t len, ::std::ostream* os) { + if (len == 0) { + *os << "{}"; + } else { + *os << "{ "; + const size_t kThreshold = 18; + const size_t kChunkSize = 8; + // If the array has more than kThreshold elements, we'll have to + // omit some details by printing only the first and the last + // kChunkSize elements. + // TODO(wan@google.com): let the user control the threshold using a flag. + if (len <= kThreshold) { + PrintRawArrayTo(begin, len, os); + } else { + PrintRawArrayTo(begin, kChunkSize, os); + *os << ", ..., "; + PrintRawArrayTo(begin + len - kChunkSize, kChunkSize, os); + } + *os << " }"; + } +} +// This overload prints a (const) char array compactly. +GTEST_API_ void UniversalPrintArray( + const char* begin, size_t len, ::std::ostream* os); + +// This overload prints a (const) wchar_t array compactly. +GTEST_API_ void UniversalPrintArray( + const wchar_t* begin, size_t len, ::std::ostream* os); + +// Implements printing an array type T[N]. +template +class UniversalPrinter { + public: + // Prints the given array, omitting some elements when there are too + // many. + static void Print(const T (&a)[N], ::std::ostream* os) { + UniversalPrintArray(a, N, os); + } +}; + +// Implements printing a reference type T&. +template +class UniversalPrinter { + public: + // MSVC warns about adding const to a function type, so we want to + // disable the warning. + GTEST_DISABLE_MSC_WARNINGS_PUSH_(4180) + + static void Print(const T& value, ::std::ostream* os) { + // Prints the address of the value. We use reinterpret_cast here + // as static_cast doesn't compile when T is a function type. + *os << "@" << reinterpret_cast(&value) << " "; + + // Then prints the value itself. + UniversalPrint(value, os); + } + + GTEST_DISABLE_MSC_WARNINGS_POP_() +}; + +// Prints a value tersely: for a reference type, the referenced value +// (but not the address) is printed; for a (const) char pointer, the +// NUL-terminated string (but not the pointer) is printed. + +template +class UniversalTersePrinter { + public: + static void Print(const T& value, ::std::ostream* os) { + UniversalPrint(value, os); + } +}; +template +class UniversalTersePrinter { + public: + static void Print(const T& value, ::std::ostream* os) { + UniversalPrint(value, os); + } +}; +template +class UniversalTersePrinter { + public: + static void Print(const T (&value)[N], ::std::ostream* os) { + UniversalPrinter::Print(value, os); + } +}; +template <> +class UniversalTersePrinter { + public: + static void Print(const char* str, ::std::ostream* os) { + if (str == NULL) { + *os << "NULL"; + } else { + UniversalPrint(string(str), os); + } + } +}; +template <> +class UniversalTersePrinter { + public: + static void Print(char* str, ::std::ostream* os) { + UniversalTersePrinter::Print(str, os); + } +}; + +#if GTEST_HAS_STD_WSTRING +template <> +class UniversalTersePrinter { + public: + static void Print(const wchar_t* str, ::std::ostream* os) { + if (str == NULL) { + *os << "NULL"; + } else { + UniversalPrint(::std::wstring(str), os); + } + } +}; +#endif + +template <> +class UniversalTersePrinter { + public: + static void Print(wchar_t* str, ::std::ostream* os) { + UniversalTersePrinter::Print(str, os); + } +}; + +template +void UniversalTersePrint(const T& value, ::std::ostream* os) { + UniversalTersePrinter::Print(value, os); +} + +// Prints a value using the type inferred by the compiler. The +// difference between this and UniversalTersePrint() is that for a +// (const) char pointer, this prints both the pointer and the +// NUL-terminated string. +template +void UniversalPrint(const T& value, ::std::ostream* os) { + // A workarond for the bug in VC++ 7.1 that prevents us from instantiating + // UniversalPrinter with T directly. + typedef T T1; + UniversalPrinter::Print(value, os); +} + +typedef ::std::vector Strings; + +// TuplePolicy must provide: +// - tuple_size +// size of tuple TupleT. +// - get(const TupleT& t) +// static function extracting element I of tuple TupleT. +// - tuple_element::type +// type of element I of tuple TupleT. +template +struct TuplePolicy; + +#if GTEST_HAS_TR1_TUPLE +template +struct TuplePolicy { + typedef TupleT Tuple; + static const size_t tuple_size = ::std::tr1::tuple_size::value; + + template + struct tuple_element : ::std::tr1::tuple_element {}; + + template + static typename AddReference< + const typename ::std::tr1::tuple_element::type>::type get( + const Tuple& tuple) { + return ::std::tr1::get(tuple); + } +}; +template +const size_t TuplePolicy::tuple_size; +#endif // GTEST_HAS_TR1_TUPLE + +#if GTEST_HAS_STD_TUPLE_ +template +struct TuplePolicy< ::std::tuple > { + typedef ::std::tuple Tuple; + static const size_t tuple_size = ::std::tuple_size::value; + + template + struct tuple_element : ::std::tuple_element {}; + + template + static const typename ::std::tuple_element::type& get( + const Tuple& tuple) { + return ::std::get(tuple); + } +}; +template +const size_t TuplePolicy< ::std::tuple >::tuple_size; +#endif // GTEST_HAS_STD_TUPLE_ + +#if GTEST_HAS_TR1_TUPLE || GTEST_HAS_STD_TUPLE_ +// This helper template allows PrintTo() for tuples and +// UniversalTersePrintTupleFieldsToStrings() to be defined by +// induction on the number of tuple fields. The idea is that +// TuplePrefixPrinter::PrintPrefixTo(t, os) prints the first N +// fields in tuple t, and can be defined in terms of +// TuplePrefixPrinter. +// +// The inductive case. +template +struct TuplePrefixPrinter { + // Prints the first N fields of a tuple. + template + static void PrintPrefixTo(const Tuple& t, ::std::ostream* os) { + TuplePrefixPrinter::PrintPrefixTo(t, os); + GTEST_INTENTIONAL_CONST_COND_PUSH_() + if (N > 1) { + GTEST_INTENTIONAL_CONST_COND_POP_() + *os << ", "; + } + UniversalPrinter< + typename TuplePolicy::template tuple_element::type> + ::Print(TuplePolicy::template get(t), os); + } + + // Tersely prints the first N fields of a tuple to a string vector, + // one element for each field. + template + static void TersePrintPrefixToStrings(const Tuple& t, Strings* strings) { + TuplePrefixPrinter::TersePrintPrefixToStrings(t, strings); + ::std::stringstream ss; + UniversalTersePrint(TuplePolicy::template get(t), &ss); + strings->push_back(ss.str()); + } +}; + +// Base case. +template <> +struct TuplePrefixPrinter<0> { + template + static void PrintPrefixTo(const Tuple&, ::std::ostream*) {} + + template + static void TersePrintPrefixToStrings(const Tuple&, Strings*) {} +}; + +// Helper function for printing a tuple. +// Tuple must be either std::tr1::tuple or std::tuple type. +template +void PrintTupleTo(const Tuple& t, ::std::ostream* os) { + *os << "("; + TuplePrefixPrinter::tuple_size>::PrintPrefixTo(t, os); + *os << ")"; +} + +// Prints the fields of a tuple tersely to a string vector, one +// element for each field. See the comment before +// UniversalTersePrint() for how we define "tersely". +template +Strings UniversalTersePrintTupleFieldsToStrings(const Tuple& value) { + Strings result; + TuplePrefixPrinter::tuple_size>:: + TersePrintPrefixToStrings(value, &result); + return result; +} +#endif // GTEST_HAS_TR1_TUPLE || GTEST_HAS_STD_TUPLE_ + +} // namespace internal + +template +::std::string PrintToString(const T& value) { + ::std::stringstream ss; + internal::UniversalTersePrinter::Print(value, &ss); + return ss.str(); +} + +} // namespace testing + +#endif // GTEST_INCLUDE_GTEST_GTEST_PRINTERS_H_ + +#if GTEST_HAS_PARAM_TEST + +namespace testing { +namespace internal { + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// Outputs a message explaining invalid registration of different +// fixture class for the same test case. This may happen when +// TEST_P macro is used to define two tests with the same name +// but in different namespaces. +GTEST_API_ void ReportInvalidTestCaseType(const char* test_case_name, + const char* file, int line); + +template class ParamGeneratorInterface; +template class ParamGenerator; + +// Interface for iterating over elements provided by an implementation +// of ParamGeneratorInterface. +template +class ParamIteratorInterface { + public: + virtual ~ParamIteratorInterface() {} + // A pointer to the base generator instance. + // Used only for the purposes of iterator comparison + // to make sure that two iterators belong to the same generator. + virtual const ParamGeneratorInterface* BaseGenerator() const = 0; + // Advances iterator to point to the next element + // provided by the generator. The caller is responsible + // for not calling Advance() on an iterator equal to + // BaseGenerator()->End(). + virtual void Advance() = 0; + // Clones the iterator object. Used for implementing copy semantics + // of ParamIterator. + virtual ParamIteratorInterface* Clone() const = 0; + // Dereferences the current iterator and provides (read-only) access + // to the pointed value. It is the caller's responsibility not to call + // Current() on an iterator equal to BaseGenerator()->End(). + // Used for implementing ParamGenerator::operator*(). + virtual const T* Current() const = 0; + // Determines whether the given iterator and other point to the same + // element in the sequence generated by the generator. + // Used for implementing ParamGenerator::operator==(). + virtual bool Equals(const ParamIteratorInterface& other) const = 0; +}; + +// Class iterating over elements provided by an implementation of +// ParamGeneratorInterface. It wraps ParamIteratorInterface +// and implements the const forward iterator concept. +template +class ParamIterator { + public: + typedef T value_type; + typedef const T& reference; + typedef ptrdiff_t difference_type; + + // ParamIterator assumes ownership of the impl_ pointer. + ParamIterator(const ParamIterator& other) : impl_(other.impl_->Clone()) {} + ParamIterator& operator=(const ParamIterator& other) { + if (this != &other) + impl_.reset(other.impl_->Clone()); + return *this; + } + + const T& operator*() const { return *impl_->Current(); } + const T* operator->() const { return impl_->Current(); } + // Prefix version of operator++. + ParamIterator& operator++() { + impl_->Advance(); + return *this; + } + // Postfix version of operator++. + ParamIterator operator++(int /*unused*/) { + ParamIteratorInterface* clone = impl_->Clone(); + impl_->Advance(); + return ParamIterator(clone); + } + bool operator==(const ParamIterator& other) const { + return impl_.get() == other.impl_.get() || impl_->Equals(*other.impl_); + } + bool operator!=(const ParamIterator& other) const { + return !(*this == other); + } + + private: + friend class ParamGenerator; + explicit ParamIterator(ParamIteratorInterface* impl) : impl_(impl) {} + scoped_ptr > impl_; +}; + +// ParamGeneratorInterface is the binary interface to access generators +// defined in other translation units. +template +class ParamGeneratorInterface { + public: + typedef T ParamType; + + virtual ~ParamGeneratorInterface() {} + + // Generator interface definition + virtual ParamIteratorInterface* Begin() const = 0; + virtual ParamIteratorInterface* End() const = 0; +}; + +// Wraps ParamGeneratorInterface and provides general generator syntax +// compatible with the STL Container concept. +// This class implements copy initialization semantics and the contained +// ParamGeneratorInterface instance is shared among all copies +// of the original object. This is possible because that instance is immutable. +template +class ParamGenerator { + public: + typedef ParamIterator iterator; + + explicit ParamGenerator(ParamGeneratorInterface* impl) : impl_(impl) {} + ParamGenerator(const ParamGenerator& other) : impl_(other.impl_) {} + + ParamGenerator& operator=(const ParamGenerator& other) { + impl_ = other.impl_; + return *this; + } + + iterator begin() const { return iterator(impl_->Begin()); } + iterator end() const { return iterator(impl_->End()); } + + private: + linked_ptr > impl_; +}; + +// Generates values from a range of two comparable values. Can be used to +// generate sequences of user-defined types that implement operator+() and +// operator<(). +// This class is used in the Range() function. +template +class RangeGenerator : public ParamGeneratorInterface { + public: + RangeGenerator(T begin, T end, IncrementT step) + : begin_(begin), end_(end), + step_(step), end_index_(CalculateEndIndex(begin, end, step)) {} + virtual ~RangeGenerator() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, begin_, 0, step_); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, end_, end_index_, step_); + } + + private: + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, T value, int index, + IncrementT step) + : base_(base), value_(value), index_(index), step_(step) {} + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + virtual void Advance() { + value_ = value_ + step_; + index_++; + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + virtual const T* Current() const { return &value_; } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + const int other_index = + CheckedDowncastToActualType(&other)->index_; + return index_ == other_index; + } + + private: + Iterator(const Iterator& other) + : ParamIteratorInterface(), + base_(other.base_), value_(other.value_), index_(other.index_), + step_(other.step_) {} + + // No implementation - assignment is unsupported. + void operator=(const Iterator& other); + + const ParamGeneratorInterface* const base_; + T value_; + int index_; + const IncrementT step_; + }; // class RangeGenerator::Iterator + + static int CalculateEndIndex(const T& begin, + const T& end, + const IncrementT& step) { + int end_index = 0; + for (T i = begin; i < end; i = i + step) + end_index++; + return end_index; + } + + // No implementation - assignment is unsupported. + void operator=(const RangeGenerator& other); + + const T begin_; + const T end_; + const IncrementT step_; + // The index for the end() iterator. All the elements in the generated + // sequence are indexed (0-based) to aid iterator comparison. + const int end_index_; +}; // class RangeGenerator + + +// Generates values from a pair of STL-style iterators. Used in the +// ValuesIn() function. The elements are copied from the source range +// since the source can be located on the stack, and the generator +// is likely to persist beyond that stack frame. +template +class ValuesInIteratorRangeGenerator : public ParamGeneratorInterface { + public: + template + ValuesInIteratorRangeGenerator(ForwardIterator begin, ForwardIterator end) + : container_(begin, end) {} + virtual ~ValuesInIteratorRangeGenerator() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, container_.begin()); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, container_.end()); + } + + private: + typedef typename ::std::vector ContainerType; + + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, + typename ContainerType::const_iterator iterator) + : base_(base), iterator_(iterator) {} + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + virtual void Advance() { + ++iterator_; + value_.reset(); + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + // We need to use cached value referenced by iterator_ because *iterator_ + // can return a temporary object (and of type other then T), so just + // having "return &*iterator_;" doesn't work. + // value_ is updated here and not in Advance() because Advance() + // can advance iterator_ beyond the end of the range, and we cannot + // detect that fact. The client code, on the other hand, is + // responsible for not calling Current() on an out-of-range iterator. + virtual const T* Current() const { + if (value_.get() == NULL) + value_.reset(new T(*iterator_)); + return value_.get(); + } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + return iterator_ == + CheckedDowncastToActualType(&other)->iterator_; + } + + private: + Iterator(const Iterator& other) + // The explicit constructor call suppresses a false warning + // emitted by gcc when supplied with the -Wextra option. + : ParamIteratorInterface(), + base_(other.base_), + iterator_(other.iterator_) {} + + const ParamGeneratorInterface* const base_; + typename ContainerType::const_iterator iterator_; + // A cached value of *iterator_. We keep it here to allow access by + // pointer in the wrapping iterator's operator->(). + // value_ needs to be mutable to be accessed in Current(). + // Use of scoped_ptr helps manage cached value's lifetime, + // which is bound by the lifespan of the iterator itself. + mutable scoped_ptr value_; + }; // class ValuesInIteratorRangeGenerator::Iterator + + // No implementation - assignment is unsupported. + void operator=(const ValuesInIteratorRangeGenerator& other); + + const ContainerType container_; +}; // class ValuesInIteratorRangeGenerator + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// Stores a parameter value and later creates tests parameterized with that +// value. +template +class ParameterizedTestFactory : public TestFactoryBase { + public: + typedef typename TestClass::ParamType ParamType; + explicit ParameterizedTestFactory(ParamType parameter) : + parameter_(parameter) {} + virtual Test* CreateTest() { + TestClass::SetParam(¶meter_); + return new TestClass(); + } + + private: + const ParamType parameter_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(ParameterizedTestFactory); +}; + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// TestMetaFactoryBase is a base class for meta-factories that create +// test factories for passing into MakeAndRegisterTestInfo function. +template +class TestMetaFactoryBase { + public: + virtual ~TestMetaFactoryBase() {} + + virtual TestFactoryBase* CreateTestFactory(ParamType parameter) = 0; +}; + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// TestMetaFactory creates test factories for passing into +// MakeAndRegisterTestInfo function. Since MakeAndRegisterTestInfo receives +// ownership of test factory pointer, same factory object cannot be passed +// into that method twice. But ParameterizedTestCaseInfo is going to call +// it for each Test/Parameter value combination. Thus it needs meta factory +// creator class. +template +class TestMetaFactory + : public TestMetaFactoryBase { + public: + typedef typename TestCase::ParamType ParamType; + + TestMetaFactory() {} + + virtual TestFactoryBase* CreateTestFactory(ParamType parameter) { + return new ParameterizedTestFactory(parameter); + } + + private: + GTEST_DISALLOW_COPY_AND_ASSIGN_(TestMetaFactory); +}; + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// ParameterizedTestCaseInfoBase is a generic interface +// to ParameterizedTestCaseInfo classes. ParameterizedTestCaseInfoBase +// accumulates test information provided by TEST_P macro invocations +// and generators provided by INSTANTIATE_TEST_CASE_P macro invocations +// and uses that information to register all resulting test instances +// in RegisterTests method. The ParameterizeTestCaseRegistry class holds +// a collection of pointers to the ParameterizedTestCaseInfo objects +// and calls RegisterTests() on each of them when asked. +class ParameterizedTestCaseInfoBase { + public: + virtual ~ParameterizedTestCaseInfoBase() {} + + // Base part of test case name for display purposes. + virtual const string& GetTestCaseName() const = 0; + // Test case id to verify identity. + virtual TypeId GetTestCaseTypeId() const = 0; + // UnitTest class invokes this method to register tests in this + // test case right before running them in RUN_ALL_TESTS macro. + // This method should not be called more then once on any single + // instance of a ParameterizedTestCaseInfoBase derived class. + virtual void RegisterTests() = 0; + + protected: + ParameterizedTestCaseInfoBase() {} + + private: + GTEST_DISALLOW_COPY_AND_ASSIGN_(ParameterizedTestCaseInfoBase); +}; + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// ParameterizedTestCaseInfo accumulates tests obtained from TEST_P +// macro invocations for a particular test case and generators +// obtained from INSTANTIATE_TEST_CASE_P macro invocations for that +// test case. It registers tests with all values generated by all +// generators when asked. +template +class ParameterizedTestCaseInfo : public ParameterizedTestCaseInfoBase { + public: + // ParamType and GeneratorCreationFunc are private types but are required + // for declarations of public methods AddTestPattern() and + // AddTestCaseInstantiation(). + typedef typename TestCase::ParamType ParamType; + // A function that returns an instance of appropriate generator type. + typedef ParamGenerator(GeneratorCreationFunc)(); + + explicit ParameterizedTestCaseInfo(const char* name) + : test_case_name_(name) {} + + // Test case base name for display purposes. + virtual const string& GetTestCaseName() const { return test_case_name_; } + // Test case id to verify identity. + virtual TypeId GetTestCaseTypeId() const { return GetTypeId(); } + // TEST_P macro uses AddTestPattern() to record information + // about a single test in a LocalTestInfo structure. + // test_case_name is the base name of the test case (without invocation + // prefix). test_base_name is the name of an individual test without + // parameter index. For the test SequenceA/FooTest.DoBar/1 FooTest is + // test case base name and DoBar is test base name. + void AddTestPattern(const char* test_case_name, + const char* test_base_name, + TestMetaFactoryBase* meta_factory) { + tests_.push_back(linked_ptr(new TestInfo(test_case_name, + test_base_name, + meta_factory))); + } + // INSTANTIATE_TEST_CASE_P macro uses AddGenerator() to record information + // about a generator. + int AddTestCaseInstantiation(const string& instantiation_name, + GeneratorCreationFunc* func, + const char* /* file */, + int /* line */) { + instantiations_.push_back(::std::make_pair(instantiation_name, func)); + return 0; // Return value used only to run this method in namespace scope. + } + // UnitTest class invokes this method to register tests in this test case + // test cases right before running tests in RUN_ALL_TESTS macro. + // This method should not be called more then once on any single + // instance of a ParameterizedTestCaseInfoBase derived class. + // UnitTest has a guard to prevent from calling this method more then once. + virtual void RegisterTests() { + for (typename TestInfoContainer::iterator test_it = tests_.begin(); + test_it != tests_.end(); ++test_it) { + linked_ptr test_info = *test_it; + for (typename InstantiationContainer::iterator gen_it = + instantiations_.begin(); gen_it != instantiations_.end(); + ++gen_it) { + const string& instantiation_name = gen_it->first; + ParamGenerator generator((*gen_it->second)()); + + string test_case_name; + if ( !instantiation_name.empty() ) + test_case_name = instantiation_name + "/"; + test_case_name += test_info->test_case_base_name; + + int i = 0; + for (typename ParamGenerator::iterator param_it = + generator.begin(); + param_it != generator.end(); ++param_it, ++i) { + Message test_name_stream; + test_name_stream << test_info->test_base_name << "/" << i; + MakeAndRegisterTestInfo( + test_case_name.c_str(), + test_name_stream.GetString().c_str(), + NULL, // No type parameter. + PrintToString(*param_it).c_str(), + GetTestCaseTypeId(), + TestCase::SetUpTestCase, + TestCase::TearDownTestCase, + test_info->test_meta_factory->CreateTestFactory(*param_it)); + } // for param_it + } // for gen_it + } // for test_it + } // RegisterTests + + private: + // LocalTestInfo structure keeps information about a single test registered + // with TEST_P macro. + struct TestInfo { + TestInfo(const char* a_test_case_base_name, + const char* a_test_base_name, + TestMetaFactoryBase* a_test_meta_factory) : + test_case_base_name(a_test_case_base_name), + test_base_name(a_test_base_name), + test_meta_factory(a_test_meta_factory) {} + + const string test_case_base_name; + const string test_base_name; + const scoped_ptr > test_meta_factory; + }; + typedef ::std::vector > TestInfoContainer; + // Keeps pairs of + // received from INSTANTIATE_TEST_CASE_P macros. + typedef ::std::vector > + InstantiationContainer; + + const string test_case_name_; + TestInfoContainer tests_; + InstantiationContainer instantiations_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(ParameterizedTestCaseInfo); +}; // class ParameterizedTestCaseInfo + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// ParameterizedTestCaseRegistry contains a map of ParameterizedTestCaseInfoBase +// classes accessed by test case names. TEST_P and INSTANTIATE_TEST_CASE_P +// macros use it to locate their corresponding ParameterizedTestCaseInfo +// descriptors. +class ParameterizedTestCaseRegistry { + public: + ParameterizedTestCaseRegistry() {} + ~ParameterizedTestCaseRegistry() { + for (TestCaseInfoContainer::iterator it = test_case_infos_.begin(); + it != test_case_infos_.end(); ++it) { + delete *it; + } + } + + // Looks up or creates and returns a structure containing information about + // tests and instantiations of a particular test case. + template + ParameterizedTestCaseInfo* GetTestCasePatternHolder( + const char* test_case_name, + const char* file, + int line) { + ParameterizedTestCaseInfo* typed_test_info = NULL; + for (TestCaseInfoContainer::iterator it = test_case_infos_.begin(); + it != test_case_infos_.end(); ++it) { + if ((*it)->GetTestCaseName() == test_case_name) { + if ((*it)->GetTestCaseTypeId() != GetTypeId()) { + // Complain about incorrect usage of Google Test facilities + // and terminate the program since we cannot guaranty correct + // test case setup and tear-down in this case. + ReportInvalidTestCaseType(test_case_name, file, line); + posix::Abort(); + } else { + // At this point we are sure that the object we found is of the same + // type we are looking for, so we downcast it to that type + // without further checks. + typed_test_info = CheckedDowncastToActualType< + ParameterizedTestCaseInfo >(*it); + } + break; + } + } + if (typed_test_info == NULL) { + typed_test_info = new ParameterizedTestCaseInfo(test_case_name); + test_case_infos_.push_back(typed_test_info); + } + return typed_test_info; + } + void RegisterTests() { + for (TestCaseInfoContainer::iterator it = test_case_infos_.begin(); + it != test_case_infos_.end(); ++it) { + (*it)->RegisterTests(); + } + } + + private: + typedef ::std::vector TestCaseInfoContainer; + + TestCaseInfoContainer test_case_infos_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(ParameterizedTestCaseRegistry); +}; + +} // namespace internal +} // namespace testing + +#endif // GTEST_HAS_PARAM_TEST + +#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_H_ +// This file was GENERATED by command: +// pump.py gtest-param-util-generated.h.pump +// DO NOT EDIT BY HAND!!! + +// Copyright 2008 Google Inc. +// All Rights Reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: vladl@google.com (Vlad Losev) + +// Type and function utilities for implementing parameterized tests. +// This file is generated by a SCRIPT. DO NOT EDIT BY HAND! +// +// Currently Google Test supports at most 50 arguments in Values, +// and at most 10 arguments in Combine. Please contact +// googletestframework@googlegroups.com if you need more. +// Please note that the number of arguments to Combine is limited +// by the maximum arity of the implementation of tuple which is +// currently set at 10. + +#ifndef GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_GENERATED_H_ +#define GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_GENERATED_H_ + +// scripts/fuse_gtest.py depends on gtest's own header being #included +// *unconditionally*. Therefore these #includes cannot be moved +// inside #if GTEST_HAS_PARAM_TEST. + +#if GTEST_HAS_PARAM_TEST + +namespace testing { + +// Forward declarations of ValuesIn(), which is implemented in +// include/gtest/gtest-param-test.h. +template +internal::ParamGenerator< + typename ::testing::internal::IteratorTraits::value_type> +ValuesIn(ForwardIterator begin, ForwardIterator end); + +template +internal::ParamGenerator ValuesIn(const T (&array)[N]); + +template +internal::ParamGenerator ValuesIn( + const Container& container); + +namespace internal { + +// Used in the Values() function to provide polymorphic capabilities. +template +class ValueArray1 { + public: + explicit ValueArray1(T1 v1) : v1_(v1) {} + + template + operator ParamGenerator() const { return ValuesIn(&v1_, &v1_ + 1); } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray1& other); + + const T1 v1_; +}; + +template +class ValueArray2 { + public: + ValueArray2(T1 v1, T2 v2) : v1_(v1), v2_(v2) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray2& other); + + const T1 v1_; + const T2 v2_; +}; + +template +class ValueArray3 { + public: + ValueArray3(T1 v1, T2 v2, T3 v3) : v1_(v1), v2_(v2), v3_(v3) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray3& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; +}; + +template +class ValueArray4 { + public: + ValueArray4(T1 v1, T2 v2, T3 v3, T4 v4) : v1_(v1), v2_(v2), v3_(v3), + v4_(v4) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray4& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; +}; + +template +class ValueArray5 { + public: + ValueArray5(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5) : v1_(v1), v2_(v2), v3_(v3), + v4_(v4), v5_(v5) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray5& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; +}; + +template +class ValueArray6 { + public: + ValueArray6(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6) : v1_(v1), v2_(v2), + v3_(v3), v4_(v4), v5_(v5), v6_(v6) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray6& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; +}; + +template +class ValueArray7 { + public: + ValueArray7(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7) : v1_(v1), + v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray7& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; +}; + +template +class ValueArray8 { + public: + ValueArray8(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, + T8 v8) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray8& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; +}; + +template +class ValueArray9 { + public: + ValueArray9(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, + T9 v9) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray9& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; +}; + +template +class ValueArray10 { + public: + ValueArray10(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray10& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; +}; + +template +class ValueArray11 { + public: + ValueArray11(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), + v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray11& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; +}; + +template +class ValueArray12 { + public: + ValueArray12(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), + v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray12& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; +}; + +template +class ValueArray13 { + public: + ValueArray13(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), + v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), + v12_(v12), v13_(v13) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray13& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; +}; + +template +class ValueArray14 { + public: + ValueArray14(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14) : v1_(v1), v2_(v2), v3_(v3), + v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), + v11_(v11), v12_(v12), v13_(v13), v14_(v14) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray14& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; +}; + +template +class ValueArray15 { + public: + ValueArray15(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15) : v1_(v1), v2_(v2), + v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), + v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray15& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; +}; + +template +class ValueArray16 { + public: + ValueArray16(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16) : v1_(v1), + v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), + v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), + v16_(v16) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray16& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; +}; + +template +class ValueArray17 { + public: + ValueArray17(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, + T17 v17) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), + v15_(v15), v16_(v16), v17_(v17) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray17& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; +}; + +template +class ValueArray18 { + public: + ValueArray18(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), + v15_(v15), v16_(v16), v17_(v17), v18_(v18) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray18& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; +}; + +template +class ValueArray19 { + public: + ValueArray19(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), + v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), + v14_(v14), v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray19& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; +}; + +template +class ValueArray20 { + public: + ValueArray20(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), + v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), + v13_(v13), v14_(v14), v15_(v15), v16_(v16), v17_(v17), v18_(v18), + v19_(v19), v20_(v20) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray20& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; +}; + +template +class ValueArray21 { + public: + ValueArray21(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), + v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), + v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), v17_(v17), + v18_(v18), v19_(v19), v20_(v20), v21_(v21) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray21& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; +}; + +template +class ValueArray22 { + public: + ValueArray22(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22) : v1_(v1), v2_(v2), v3_(v3), + v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), + v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), + v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray22& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; +}; + +template +class ValueArray23 { + public: + ValueArray23(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23) : v1_(v1), v2_(v2), + v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), + v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), + v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22), + v23_(v23) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray23& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; +}; + +template +class ValueArray24 { + public: + ValueArray24(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24) : v1_(v1), + v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), + v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), + v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), + v22_(v22), v23_(v23), v24_(v24) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray24& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; +}; + +template +class ValueArray25 { + public: + ValueArray25(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, + T25 v25) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), + v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), + v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray25& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; +}; + +template +class ValueArray26 { + public: + ValueArray26(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), + v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), + v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25), v26_(v26) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray26& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; +}; + +template +class ValueArray27 { + public: + ValueArray27(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), + v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), + v14_(v14), v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), + v20_(v20), v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25), + v26_(v26), v27_(v27) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray27& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; +}; + +template +class ValueArray28 { + public: + ValueArray28(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), + v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), + v13_(v13), v14_(v14), v15_(v15), v16_(v16), v17_(v17), v18_(v18), + v19_(v19), v20_(v20), v21_(v21), v22_(v22), v23_(v23), v24_(v24), + v25_(v25), v26_(v26), v27_(v27), v28_(v28) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray28& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; +}; + +template +class ValueArray29 { + public: + ValueArray29(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), + v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), + v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), v17_(v17), + v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22), v23_(v23), + v24_(v24), v25_(v25), v26_(v26), v27_(v27), v28_(v28), v29_(v29) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray29& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; +}; + +template +class ValueArray30 { + public: + ValueArray30(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30) : v1_(v1), v2_(v2), v3_(v3), + v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), + v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), + v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22), + v23_(v23), v24_(v24), v25_(v25), v26_(v26), v27_(v27), v28_(v28), + v29_(v29), v30_(v30) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray30& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; +}; + +template +class ValueArray31 { + public: + ValueArray31(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31) : v1_(v1), v2_(v2), + v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), + v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), + v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22), + v23_(v23), v24_(v24), v25_(v25), v26_(v26), v27_(v27), v28_(v28), + v29_(v29), v30_(v30), v31_(v31) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray31& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; +}; + +template +class ValueArray32 { + public: + ValueArray32(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32) : v1_(v1), + v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), + v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), + v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), + v22_(v22), v23_(v23), v24_(v24), v25_(v25), v26_(v26), v27_(v27), + v28_(v28), v29_(v29), v30_(v30), v31_(v31), v32_(v32) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray32& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; +}; + +template +class ValueArray33 { + public: + ValueArray33(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, + T33 v33) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), + v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), + v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25), v26_(v26), + v27_(v27), v28_(v28), v29_(v29), v30_(v30), v31_(v31), v32_(v32), + v33_(v33) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray33& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; +}; + +template +class ValueArray34 { + public: + ValueArray34(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), + v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), + v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25), v26_(v26), + v27_(v27), v28_(v28), v29_(v29), v30_(v30), v31_(v31), v32_(v32), + v33_(v33), v34_(v34) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray34& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; +}; + +template +class ValueArray35 { + public: + ValueArray35(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), + v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), + v14_(v14), v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), + v20_(v20), v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25), + v26_(v26), v27_(v27), v28_(v28), v29_(v29), v30_(v30), v31_(v31), + v32_(v32), v33_(v33), v34_(v34), v35_(v35) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray35& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; +}; + +template +class ValueArray36 { + public: + ValueArray36(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), + v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), + v13_(v13), v14_(v14), v15_(v15), v16_(v16), v17_(v17), v18_(v18), + v19_(v19), v20_(v20), v21_(v21), v22_(v22), v23_(v23), v24_(v24), + v25_(v25), v26_(v26), v27_(v27), v28_(v28), v29_(v29), v30_(v30), + v31_(v31), v32_(v32), v33_(v33), v34_(v34), v35_(v35), v36_(v36) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray36& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; +}; + +template +class ValueArray37 { + public: + ValueArray37(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), + v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), + v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), v17_(v17), + v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22), v23_(v23), + v24_(v24), v25_(v25), v26_(v26), v27_(v27), v28_(v28), v29_(v29), + v30_(v30), v31_(v31), v32_(v32), v33_(v33), v34_(v34), v35_(v35), + v36_(v36), v37_(v37) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray37& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; +}; + +template +class ValueArray38 { + public: + ValueArray38(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38) : v1_(v1), v2_(v2), v3_(v3), + v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), + v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), + v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22), + v23_(v23), v24_(v24), v25_(v25), v26_(v26), v27_(v27), v28_(v28), + v29_(v29), v30_(v30), v31_(v31), v32_(v32), v33_(v33), v34_(v34), + v35_(v35), v36_(v36), v37_(v37), v38_(v38) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray38& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; +}; + +template +class ValueArray39 { + public: + ValueArray39(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39) : v1_(v1), v2_(v2), + v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), + v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), + v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22), + v23_(v23), v24_(v24), v25_(v25), v26_(v26), v27_(v27), v28_(v28), + v29_(v29), v30_(v30), v31_(v31), v32_(v32), v33_(v33), v34_(v34), + v35_(v35), v36_(v36), v37_(v37), v38_(v38), v39_(v39) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray39& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; +}; + +template +class ValueArray40 { + public: + ValueArray40(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40) : v1_(v1), + v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), + v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), + v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), + v22_(v22), v23_(v23), v24_(v24), v25_(v25), v26_(v26), v27_(v27), + v28_(v28), v29_(v29), v30_(v30), v31_(v31), v32_(v32), v33_(v33), + v34_(v34), v35_(v35), v36_(v36), v37_(v37), v38_(v38), v39_(v39), + v40_(v40) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray40& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; +}; + +template +class ValueArray41 { + public: + ValueArray41(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, + T41 v41) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), + v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), + v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25), v26_(v26), + v27_(v27), v28_(v28), v29_(v29), v30_(v30), v31_(v31), v32_(v32), + v33_(v33), v34_(v34), v35_(v35), v36_(v36), v37_(v37), v38_(v38), + v39_(v39), v40_(v40), v41_(v41) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_), static_cast(v41_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray41& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; + const T41 v41_; +}; + +template +class ValueArray42 { + public: + ValueArray42(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), + v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), + v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25), v26_(v26), + v27_(v27), v28_(v28), v29_(v29), v30_(v30), v31_(v31), v32_(v32), + v33_(v33), v34_(v34), v35_(v35), v36_(v36), v37_(v37), v38_(v38), + v39_(v39), v40_(v40), v41_(v41), v42_(v42) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_), static_cast(v41_), + static_cast(v42_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray42& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; + const T41 v41_; + const T42 v42_; +}; + +template +class ValueArray43 { + public: + ValueArray43(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42, T43 v43) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), + v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), + v14_(v14), v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), + v20_(v20), v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25), + v26_(v26), v27_(v27), v28_(v28), v29_(v29), v30_(v30), v31_(v31), + v32_(v32), v33_(v33), v34_(v34), v35_(v35), v36_(v36), v37_(v37), + v38_(v38), v39_(v39), v40_(v40), v41_(v41), v42_(v42), v43_(v43) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_), static_cast(v41_), + static_cast(v42_), static_cast(v43_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray43& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; + const T41 v41_; + const T42 v42_; + const T43 v43_; +}; + +template +class ValueArray44 { + public: + ValueArray44(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42, T43 v43, T44 v44) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), + v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), + v13_(v13), v14_(v14), v15_(v15), v16_(v16), v17_(v17), v18_(v18), + v19_(v19), v20_(v20), v21_(v21), v22_(v22), v23_(v23), v24_(v24), + v25_(v25), v26_(v26), v27_(v27), v28_(v28), v29_(v29), v30_(v30), + v31_(v31), v32_(v32), v33_(v33), v34_(v34), v35_(v35), v36_(v36), + v37_(v37), v38_(v38), v39_(v39), v40_(v40), v41_(v41), v42_(v42), + v43_(v43), v44_(v44) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_), static_cast(v41_), + static_cast(v42_), static_cast(v43_), static_cast(v44_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray44& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; + const T41 v41_; + const T42 v42_; + const T43 v43_; + const T44 v44_; +}; + +template +class ValueArray45 { + public: + ValueArray45(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42, T43 v43, T44 v44, T45 v45) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), + v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), v11_(v11), + v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), v17_(v17), + v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22), v23_(v23), + v24_(v24), v25_(v25), v26_(v26), v27_(v27), v28_(v28), v29_(v29), + v30_(v30), v31_(v31), v32_(v32), v33_(v33), v34_(v34), v35_(v35), + v36_(v36), v37_(v37), v38_(v38), v39_(v39), v40_(v40), v41_(v41), + v42_(v42), v43_(v43), v44_(v44), v45_(v45) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_), static_cast(v41_), + static_cast(v42_), static_cast(v43_), static_cast(v44_), + static_cast(v45_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray45& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; + const T41 v41_; + const T42 v42_; + const T43 v43_; + const T44 v44_; + const T45 v45_; +}; + +template +class ValueArray46 { + public: + ValueArray46(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42, T43 v43, T44 v44, T45 v45, T46 v46) : v1_(v1), v2_(v2), v3_(v3), + v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), + v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), + v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22), + v23_(v23), v24_(v24), v25_(v25), v26_(v26), v27_(v27), v28_(v28), + v29_(v29), v30_(v30), v31_(v31), v32_(v32), v33_(v33), v34_(v34), + v35_(v35), v36_(v36), v37_(v37), v38_(v38), v39_(v39), v40_(v40), + v41_(v41), v42_(v42), v43_(v43), v44_(v44), v45_(v45), v46_(v46) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_), static_cast(v41_), + static_cast(v42_), static_cast(v43_), static_cast(v44_), + static_cast(v45_), static_cast(v46_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray46& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; + const T41 v41_; + const T42 v42_; + const T43 v43_; + const T44 v44_; + const T45 v45_; + const T46 v46_; +}; + +template +class ValueArray47 { + public: + ValueArray47(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42, T43 v43, T44 v44, T45 v45, T46 v46, T47 v47) : v1_(v1), v2_(v2), + v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), v10_(v10), + v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), v16_(v16), + v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), v22_(v22), + v23_(v23), v24_(v24), v25_(v25), v26_(v26), v27_(v27), v28_(v28), + v29_(v29), v30_(v30), v31_(v31), v32_(v32), v33_(v33), v34_(v34), + v35_(v35), v36_(v36), v37_(v37), v38_(v38), v39_(v39), v40_(v40), + v41_(v41), v42_(v42), v43_(v43), v44_(v44), v45_(v45), v46_(v46), + v47_(v47) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_), static_cast(v41_), + static_cast(v42_), static_cast(v43_), static_cast(v44_), + static_cast(v45_), static_cast(v46_), static_cast(v47_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray47& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; + const T41 v41_; + const T42 v42_; + const T43 v43_; + const T44 v44_; + const T45 v45_; + const T46 v46_; + const T47 v47_; +}; + +template +class ValueArray48 { + public: + ValueArray48(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42, T43 v43, T44 v44, T45 v45, T46 v46, T47 v47, T48 v48) : v1_(v1), + v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), v8_(v8), v9_(v9), + v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), v15_(v15), + v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), v21_(v21), + v22_(v22), v23_(v23), v24_(v24), v25_(v25), v26_(v26), v27_(v27), + v28_(v28), v29_(v29), v30_(v30), v31_(v31), v32_(v32), v33_(v33), + v34_(v34), v35_(v35), v36_(v36), v37_(v37), v38_(v38), v39_(v39), + v40_(v40), v41_(v41), v42_(v42), v43_(v43), v44_(v44), v45_(v45), + v46_(v46), v47_(v47), v48_(v48) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_), static_cast(v41_), + static_cast(v42_), static_cast(v43_), static_cast(v44_), + static_cast(v45_), static_cast(v46_), static_cast(v47_), + static_cast(v48_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray48& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; + const T41 v41_; + const T42 v42_; + const T43 v43_; + const T44 v44_; + const T45 v45_; + const T46 v46_; + const T47 v47_; + const T48 v48_; +}; + +template +class ValueArray49 { + public: + ValueArray49(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42, T43 v43, T44 v44, T45 v45, T46 v46, T47 v47, T48 v48, + T49 v49) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), + v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), + v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25), v26_(v26), + v27_(v27), v28_(v28), v29_(v29), v30_(v30), v31_(v31), v32_(v32), + v33_(v33), v34_(v34), v35_(v35), v36_(v36), v37_(v37), v38_(v38), + v39_(v39), v40_(v40), v41_(v41), v42_(v42), v43_(v43), v44_(v44), + v45_(v45), v46_(v46), v47_(v47), v48_(v48), v49_(v49) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_), static_cast(v41_), + static_cast(v42_), static_cast(v43_), static_cast(v44_), + static_cast(v45_), static_cast(v46_), static_cast(v47_), + static_cast(v48_), static_cast(v49_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray49& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; + const T41 v41_; + const T42 v42_; + const T43 v43_; + const T44 v44_; + const T45 v45_; + const T46 v46_; + const T47 v47_; + const T48 v48_; + const T49 v49_; +}; + +template +class ValueArray50 { + public: + ValueArray50(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42, T43 v43, T44 v44, T45 v45, T46 v46, T47 v47, T48 v48, T49 v49, + T50 v50) : v1_(v1), v2_(v2), v3_(v3), v4_(v4), v5_(v5), v6_(v6), v7_(v7), + v8_(v8), v9_(v9), v10_(v10), v11_(v11), v12_(v12), v13_(v13), v14_(v14), + v15_(v15), v16_(v16), v17_(v17), v18_(v18), v19_(v19), v20_(v20), + v21_(v21), v22_(v22), v23_(v23), v24_(v24), v25_(v25), v26_(v26), + v27_(v27), v28_(v28), v29_(v29), v30_(v30), v31_(v31), v32_(v32), + v33_(v33), v34_(v34), v35_(v35), v36_(v36), v37_(v37), v38_(v38), + v39_(v39), v40_(v40), v41_(v41), v42_(v42), v43_(v43), v44_(v44), + v45_(v45), v46_(v46), v47_(v47), v48_(v48), v49_(v49), v50_(v50) {} + + template + operator ParamGenerator() const { + const T array[] = {static_cast(v1_), static_cast(v2_), + static_cast(v3_), static_cast(v4_), static_cast(v5_), + static_cast(v6_), static_cast(v7_), static_cast(v8_), + static_cast(v9_), static_cast(v10_), static_cast(v11_), + static_cast(v12_), static_cast(v13_), static_cast(v14_), + static_cast(v15_), static_cast(v16_), static_cast(v17_), + static_cast(v18_), static_cast(v19_), static_cast(v20_), + static_cast(v21_), static_cast(v22_), static_cast(v23_), + static_cast(v24_), static_cast(v25_), static_cast(v26_), + static_cast(v27_), static_cast(v28_), static_cast(v29_), + static_cast(v30_), static_cast(v31_), static_cast(v32_), + static_cast(v33_), static_cast(v34_), static_cast(v35_), + static_cast(v36_), static_cast(v37_), static_cast(v38_), + static_cast(v39_), static_cast(v40_), static_cast(v41_), + static_cast(v42_), static_cast(v43_), static_cast(v44_), + static_cast(v45_), static_cast(v46_), static_cast(v47_), + static_cast(v48_), static_cast(v49_), static_cast(v50_)}; + return ValuesIn(array); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const ValueArray50& other); + + const T1 v1_; + const T2 v2_; + const T3 v3_; + const T4 v4_; + const T5 v5_; + const T6 v6_; + const T7 v7_; + const T8 v8_; + const T9 v9_; + const T10 v10_; + const T11 v11_; + const T12 v12_; + const T13 v13_; + const T14 v14_; + const T15 v15_; + const T16 v16_; + const T17 v17_; + const T18 v18_; + const T19 v19_; + const T20 v20_; + const T21 v21_; + const T22 v22_; + const T23 v23_; + const T24 v24_; + const T25 v25_; + const T26 v26_; + const T27 v27_; + const T28 v28_; + const T29 v29_; + const T30 v30_; + const T31 v31_; + const T32 v32_; + const T33 v33_; + const T34 v34_; + const T35 v35_; + const T36 v36_; + const T37 v37_; + const T38 v38_; + const T39 v39_; + const T40 v40_; + const T41 v41_; + const T42 v42_; + const T43 v43_; + const T44 v44_; + const T45 v45_; + const T46 v46_; + const T47 v47_; + const T48 v48_; + const T49 v49_; + const T50 v50_; +}; + +# if GTEST_HAS_COMBINE +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// Generates values from the Cartesian product of values produced +// by the argument generators. +// +template +class CartesianProductGenerator2 + : public ParamGeneratorInterface< ::testing::tuple > { + public: + typedef ::testing::tuple ParamType; + + CartesianProductGenerator2(const ParamGenerator& g1, + const ParamGenerator& g2) + : g1_(g1), g2_(g2) {} + virtual ~CartesianProductGenerator2() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, g1_, g1_.begin(), g2_, g2_.begin()); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, g1_, g1_.end(), g2_, g2_.end()); + } + + private: + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, + const ParamGenerator& g1, + const typename ParamGenerator::iterator& current1, + const ParamGenerator& g2, + const typename ParamGenerator::iterator& current2) + : base_(base), + begin1_(g1.begin()), end1_(g1.end()), current1_(current1), + begin2_(g2.begin()), end2_(g2.end()), current2_(current2) { + ComputeCurrentValue(); + } + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + // Advance should not be called on beyond-of-range iterators + // so no component iterators must be beyond end of range, either. + virtual void Advance() { + assert(!AtEnd()); + ++current2_; + if (current2_ == end2_) { + current2_ = begin2_; + ++current1_; + } + ComputeCurrentValue(); + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + virtual const ParamType* Current() const { return ¤t_value_; } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + const Iterator* typed_other = + CheckedDowncastToActualType(&other); + // We must report iterators equal if they both point beyond their + // respective ranges. That can happen in a variety of fashions, + // so we have to consult AtEnd(). + return (AtEnd() && typed_other->AtEnd()) || + ( + current1_ == typed_other->current1_ && + current2_ == typed_other->current2_); + } + + private: + Iterator(const Iterator& other) + : base_(other.base_), + begin1_(other.begin1_), + end1_(other.end1_), + current1_(other.current1_), + begin2_(other.begin2_), + end2_(other.end2_), + current2_(other.current2_) { + ComputeCurrentValue(); + } + + void ComputeCurrentValue() { + if (!AtEnd()) + current_value_ = ParamType(*current1_, *current2_); + } + bool AtEnd() const { + // We must report iterator past the end of the range when either of the + // component iterators has reached the end of its range. + return + current1_ == end1_ || + current2_ == end2_; + } + + // No implementation - assignment is unsupported. + void operator=(const Iterator& other); + + const ParamGeneratorInterface* const base_; + // begin[i]_ and end[i]_ define the i-th range that Iterator traverses. + // current[i]_ is the actual traversing iterator. + const typename ParamGenerator::iterator begin1_; + const typename ParamGenerator::iterator end1_; + typename ParamGenerator::iterator current1_; + const typename ParamGenerator::iterator begin2_; + const typename ParamGenerator::iterator end2_; + typename ParamGenerator::iterator current2_; + ParamType current_value_; + }; // class CartesianProductGenerator2::Iterator + + // No implementation - assignment is unsupported. + void operator=(const CartesianProductGenerator2& other); + + const ParamGenerator g1_; + const ParamGenerator g2_; +}; // class CartesianProductGenerator2 + + +template +class CartesianProductGenerator3 + : public ParamGeneratorInterface< ::testing::tuple > { + public: + typedef ::testing::tuple ParamType; + + CartesianProductGenerator3(const ParamGenerator& g1, + const ParamGenerator& g2, const ParamGenerator& g3) + : g1_(g1), g2_(g2), g3_(g3) {} + virtual ~CartesianProductGenerator3() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, g1_, g1_.begin(), g2_, g2_.begin(), g3_, + g3_.begin()); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, g1_, g1_.end(), g2_, g2_.end(), g3_, g3_.end()); + } + + private: + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, + const ParamGenerator& g1, + const typename ParamGenerator::iterator& current1, + const ParamGenerator& g2, + const typename ParamGenerator::iterator& current2, + const ParamGenerator& g3, + const typename ParamGenerator::iterator& current3) + : base_(base), + begin1_(g1.begin()), end1_(g1.end()), current1_(current1), + begin2_(g2.begin()), end2_(g2.end()), current2_(current2), + begin3_(g3.begin()), end3_(g3.end()), current3_(current3) { + ComputeCurrentValue(); + } + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + // Advance should not be called on beyond-of-range iterators + // so no component iterators must be beyond end of range, either. + virtual void Advance() { + assert(!AtEnd()); + ++current3_; + if (current3_ == end3_) { + current3_ = begin3_; + ++current2_; + } + if (current2_ == end2_) { + current2_ = begin2_; + ++current1_; + } + ComputeCurrentValue(); + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + virtual const ParamType* Current() const { return ¤t_value_; } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + const Iterator* typed_other = + CheckedDowncastToActualType(&other); + // We must report iterators equal if they both point beyond their + // respective ranges. That can happen in a variety of fashions, + // so we have to consult AtEnd(). + return (AtEnd() && typed_other->AtEnd()) || + ( + current1_ == typed_other->current1_ && + current2_ == typed_other->current2_ && + current3_ == typed_other->current3_); + } + + private: + Iterator(const Iterator& other) + : base_(other.base_), + begin1_(other.begin1_), + end1_(other.end1_), + current1_(other.current1_), + begin2_(other.begin2_), + end2_(other.end2_), + current2_(other.current2_), + begin3_(other.begin3_), + end3_(other.end3_), + current3_(other.current3_) { + ComputeCurrentValue(); + } + + void ComputeCurrentValue() { + if (!AtEnd()) + current_value_ = ParamType(*current1_, *current2_, *current3_); + } + bool AtEnd() const { + // We must report iterator past the end of the range when either of the + // component iterators has reached the end of its range. + return + current1_ == end1_ || + current2_ == end2_ || + current3_ == end3_; + } + + // No implementation - assignment is unsupported. + void operator=(const Iterator& other); + + const ParamGeneratorInterface* const base_; + // begin[i]_ and end[i]_ define the i-th range that Iterator traverses. + // current[i]_ is the actual traversing iterator. + const typename ParamGenerator::iterator begin1_; + const typename ParamGenerator::iterator end1_; + typename ParamGenerator::iterator current1_; + const typename ParamGenerator::iterator begin2_; + const typename ParamGenerator::iterator end2_; + typename ParamGenerator::iterator current2_; + const typename ParamGenerator::iterator begin3_; + const typename ParamGenerator::iterator end3_; + typename ParamGenerator::iterator current3_; + ParamType current_value_; + }; // class CartesianProductGenerator3::Iterator + + // No implementation - assignment is unsupported. + void operator=(const CartesianProductGenerator3& other); + + const ParamGenerator g1_; + const ParamGenerator g2_; + const ParamGenerator g3_; +}; // class CartesianProductGenerator3 + + +template +class CartesianProductGenerator4 + : public ParamGeneratorInterface< ::testing::tuple > { + public: + typedef ::testing::tuple ParamType; + + CartesianProductGenerator4(const ParamGenerator& g1, + const ParamGenerator& g2, const ParamGenerator& g3, + const ParamGenerator& g4) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4) {} + virtual ~CartesianProductGenerator4() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, g1_, g1_.begin(), g2_, g2_.begin(), g3_, + g3_.begin(), g4_, g4_.begin()); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, g1_, g1_.end(), g2_, g2_.end(), g3_, g3_.end(), + g4_, g4_.end()); + } + + private: + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, + const ParamGenerator& g1, + const typename ParamGenerator::iterator& current1, + const ParamGenerator& g2, + const typename ParamGenerator::iterator& current2, + const ParamGenerator& g3, + const typename ParamGenerator::iterator& current3, + const ParamGenerator& g4, + const typename ParamGenerator::iterator& current4) + : base_(base), + begin1_(g1.begin()), end1_(g1.end()), current1_(current1), + begin2_(g2.begin()), end2_(g2.end()), current2_(current2), + begin3_(g3.begin()), end3_(g3.end()), current3_(current3), + begin4_(g4.begin()), end4_(g4.end()), current4_(current4) { + ComputeCurrentValue(); + } + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + // Advance should not be called on beyond-of-range iterators + // so no component iterators must be beyond end of range, either. + virtual void Advance() { + assert(!AtEnd()); + ++current4_; + if (current4_ == end4_) { + current4_ = begin4_; + ++current3_; + } + if (current3_ == end3_) { + current3_ = begin3_; + ++current2_; + } + if (current2_ == end2_) { + current2_ = begin2_; + ++current1_; + } + ComputeCurrentValue(); + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + virtual const ParamType* Current() const { return ¤t_value_; } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + const Iterator* typed_other = + CheckedDowncastToActualType(&other); + // We must report iterators equal if they both point beyond their + // respective ranges. That can happen in a variety of fashions, + // so we have to consult AtEnd(). + return (AtEnd() && typed_other->AtEnd()) || + ( + current1_ == typed_other->current1_ && + current2_ == typed_other->current2_ && + current3_ == typed_other->current3_ && + current4_ == typed_other->current4_); + } + + private: + Iterator(const Iterator& other) + : base_(other.base_), + begin1_(other.begin1_), + end1_(other.end1_), + current1_(other.current1_), + begin2_(other.begin2_), + end2_(other.end2_), + current2_(other.current2_), + begin3_(other.begin3_), + end3_(other.end3_), + current3_(other.current3_), + begin4_(other.begin4_), + end4_(other.end4_), + current4_(other.current4_) { + ComputeCurrentValue(); + } + + void ComputeCurrentValue() { + if (!AtEnd()) + current_value_ = ParamType(*current1_, *current2_, *current3_, + *current4_); + } + bool AtEnd() const { + // We must report iterator past the end of the range when either of the + // component iterators has reached the end of its range. + return + current1_ == end1_ || + current2_ == end2_ || + current3_ == end3_ || + current4_ == end4_; + } + + // No implementation - assignment is unsupported. + void operator=(const Iterator& other); + + const ParamGeneratorInterface* const base_; + // begin[i]_ and end[i]_ define the i-th range that Iterator traverses. + // current[i]_ is the actual traversing iterator. + const typename ParamGenerator::iterator begin1_; + const typename ParamGenerator::iterator end1_; + typename ParamGenerator::iterator current1_; + const typename ParamGenerator::iterator begin2_; + const typename ParamGenerator::iterator end2_; + typename ParamGenerator::iterator current2_; + const typename ParamGenerator::iterator begin3_; + const typename ParamGenerator::iterator end3_; + typename ParamGenerator::iterator current3_; + const typename ParamGenerator::iterator begin4_; + const typename ParamGenerator::iterator end4_; + typename ParamGenerator::iterator current4_; + ParamType current_value_; + }; // class CartesianProductGenerator4::Iterator + + // No implementation - assignment is unsupported. + void operator=(const CartesianProductGenerator4& other); + + const ParamGenerator g1_; + const ParamGenerator g2_; + const ParamGenerator g3_; + const ParamGenerator g4_; +}; // class CartesianProductGenerator4 + + +template +class CartesianProductGenerator5 + : public ParamGeneratorInterface< ::testing::tuple > { + public: + typedef ::testing::tuple ParamType; + + CartesianProductGenerator5(const ParamGenerator& g1, + const ParamGenerator& g2, const ParamGenerator& g3, + const ParamGenerator& g4, const ParamGenerator& g5) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5) {} + virtual ~CartesianProductGenerator5() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, g1_, g1_.begin(), g2_, g2_.begin(), g3_, + g3_.begin(), g4_, g4_.begin(), g5_, g5_.begin()); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, g1_, g1_.end(), g2_, g2_.end(), g3_, g3_.end(), + g4_, g4_.end(), g5_, g5_.end()); + } + + private: + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, + const ParamGenerator& g1, + const typename ParamGenerator::iterator& current1, + const ParamGenerator& g2, + const typename ParamGenerator::iterator& current2, + const ParamGenerator& g3, + const typename ParamGenerator::iterator& current3, + const ParamGenerator& g4, + const typename ParamGenerator::iterator& current4, + const ParamGenerator& g5, + const typename ParamGenerator::iterator& current5) + : base_(base), + begin1_(g1.begin()), end1_(g1.end()), current1_(current1), + begin2_(g2.begin()), end2_(g2.end()), current2_(current2), + begin3_(g3.begin()), end3_(g3.end()), current3_(current3), + begin4_(g4.begin()), end4_(g4.end()), current4_(current4), + begin5_(g5.begin()), end5_(g5.end()), current5_(current5) { + ComputeCurrentValue(); + } + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + // Advance should not be called on beyond-of-range iterators + // so no component iterators must be beyond end of range, either. + virtual void Advance() { + assert(!AtEnd()); + ++current5_; + if (current5_ == end5_) { + current5_ = begin5_; + ++current4_; + } + if (current4_ == end4_) { + current4_ = begin4_; + ++current3_; + } + if (current3_ == end3_) { + current3_ = begin3_; + ++current2_; + } + if (current2_ == end2_) { + current2_ = begin2_; + ++current1_; + } + ComputeCurrentValue(); + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + virtual const ParamType* Current() const { return ¤t_value_; } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + const Iterator* typed_other = + CheckedDowncastToActualType(&other); + // We must report iterators equal if they both point beyond their + // respective ranges. That can happen in a variety of fashions, + // so we have to consult AtEnd(). + return (AtEnd() && typed_other->AtEnd()) || + ( + current1_ == typed_other->current1_ && + current2_ == typed_other->current2_ && + current3_ == typed_other->current3_ && + current4_ == typed_other->current4_ && + current5_ == typed_other->current5_); + } + + private: + Iterator(const Iterator& other) + : base_(other.base_), + begin1_(other.begin1_), + end1_(other.end1_), + current1_(other.current1_), + begin2_(other.begin2_), + end2_(other.end2_), + current2_(other.current2_), + begin3_(other.begin3_), + end3_(other.end3_), + current3_(other.current3_), + begin4_(other.begin4_), + end4_(other.end4_), + current4_(other.current4_), + begin5_(other.begin5_), + end5_(other.end5_), + current5_(other.current5_) { + ComputeCurrentValue(); + } + + void ComputeCurrentValue() { + if (!AtEnd()) + current_value_ = ParamType(*current1_, *current2_, *current3_, + *current4_, *current5_); + } + bool AtEnd() const { + // We must report iterator past the end of the range when either of the + // component iterators has reached the end of its range. + return + current1_ == end1_ || + current2_ == end2_ || + current3_ == end3_ || + current4_ == end4_ || + current5_ == end5_; + } + + // No implementation - assignment is unsupported. + void operator=(const Iterator& other); + + const ParamGeneratorInterface* const base_; + // begin[i]_ and end[i]_ define the i-th range that Iterator traverses. + // current[i]_ is the actual traversing iterator. + const typename ParamGenerator::iterator begin1_; + const typename ParamGenerator::iterator end1_; + typename ParamGenerator::iterator current1_; + const typename ParamGenerator::iterator begin2_; + const typename ParamGenerator::iterator end2_; + typename ParamGenerator::iterator current2_; + const typename ParamGenerator::iterator begin3_; + const typename ParamGenerator::iterator end3_; + typename ParamGenerator::iterator current3_; + const typename ParamGenerator::iterator begin4_; + const typename ParamGenerator::iterator end4_; + typename ParamGenerator::iterator current4_; + const typename ParamGenerator::iterator begin5_; + const typename ParamGenerator::iterator end5_; + typename ParamGenerator::iterator current5_; + ParamType current_value_; + }; // class CartesianProductGenerator5::Iterator + + // No implementation - assignment is unsupported. + void operator=(const CartesianProductGenerator5& other); + + const ParamGenerator g1_; + const ParamGenerator g2_; + const ParamGenerator g3_; + const ParamGenerator g4_; + const ParamGenerator g5_; +}; // class CartesianProductGenerator5 + + +template +class CartesianProductGenerator6 + : public ParamGeneratorInterface< ::testing::tuple > { + public: + typedef ::testing::tuple ParamType; + + CartesianProductGenerator6(const ParamGenerator& g1, + const ParamGenerator& g2, const ParamGenerator& g3, + const ParamGenerator& g4, const ParamGenerator& g5, + const ParamGenerator& g6) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5), g6_(g6) {} + virtual ~CartesianProductGenerator6() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, g1_, g1_.begin(), g2_, g2_.begin(), g3_, + g3_.begin(), g4_, g4_.begin(), g5_, g5_.begin(), g6_, g6_.begin()); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, g1_, g1_.end(), g2_, g2_.end(), g3_, g3_.end(), + g4_, g4_.end(), g5_, g5_.end(), g6_, g6_.end()); + } + + private: + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, + const ParamGenerator& g1, + const typename ParamGenerator::iterator& current1, + const ParamGenerator& g2, + const typename ParamGenerator::iterator& current2, + const ParamGenerator& g3, + const typename ParamGenerator::iterator& current3, + const ParamGenerator& g4, + const typename ParamGenerator::iterator& current4, + const ParamGenerator& g5, + const typename ParamGenerator::iterator& current5, + const ParamGenerator& g6, + const typename ParamGenerator::iterator& current6) + : base_(base), + begin1_(g1.begin()), end1_(g1.end()), current1_(current1), + begin2_(g2.begin()), end2_(g2.end()), current2_(current2), + begin3_(g3.begin()), end3_(g3.end()), current3_(current3), + begin4_(g4.begin()), end4_(g4.end()), current4_(current4), + begin5_(g5.begin()), end5_(g5.end()), current5_(current5), + begin6_(g6.begin()), end6_(g6.end()), current6_(current6) { + ComputeCurrentValue(); + } + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + // Advance should not be called on beyond-of-range iterators + // so no component iterators must be beyond end of range, either. + virtual void Advance() { + assert(!AtEnd()); + ++current6_; + if (current6_ == end6_) { + current6_ = begin6_; + ++current5_; + } + if (current5_ == end5_) { + current5_ = begin5_; + ++current4_; + } + if (current4_ == end4_) { + current4_ = begin4_; + ++current3_; + } + if (current3_ == end3_) { + current3_ = begin3_; + ++current2_; + } + if (current2_ == end2_) { + current2_ = begin2_; + ++current1_; + } + ComputeCurrentValue(); + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + virtual const ParamType* Current() const { return ¤t_value_; } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + const Iterator* typed_other = + CheckedDowncastToActualType(&other); + // We must report iterators equal if they both point beyond their + // respective ranges. That can happen in a variety of fashions, + // so we have to consult AtEnd(). + return (AtEnd() && typed_other->AtEnd()) || + ( + current1_ == typed_other->current1_ && + current2_ == typed_other->current2_ && + current3_ == typed_other->current3_ && + current4_ == typed_other->current4_ && + current5_ == typed_other->current5_ && + current6_ == typed_other->current6_); + } + + private: + Iterator(const Iterator& other) + : base_(other.base_), + begin1_(other.begin1_), + end1_(other.end1_), + current1_(other.current1_), + begin2_(other.begin2_), + end2_(other.end2_), + current2_(other.current2_), + begin3_(other.begin3_), + end3_(other.end3_), + current3_(other.current3_), + begin4_(other.begin4_), + end4_(other.end4_), + current4_(other.current4_), + begin5_(other.begin5_), + end5_(other.end5_), + current5_(other.current5_), + begin6_(other.begin6_), + end6_(other.end6_), + current6_(other.current6_) { + ComputeCurrentValue(); + } + + void ComputeCurrentValue() { + if (!AtEnd()) + current_value_ = ParamType(*current1_, *current2_, *current3_, + *current4_, *current5_, *current6_); + } + bool AtEnd() const { + // We must report iterator past the end of the range when either of the + // component iterators has reached the end of its range. + return + current1_ == end1_ || + current2_ == end2_ || + current3_ == end3_ || + current4_ == end4_ || + current5_ == end5_ || + current6_ == end6_; + } + + // No implementation - assignment is unsupported. + void operator=(const Iterator& other); + + const ParamGeneratorInterface* const base_; + // begin[i]_ and end[i]_ define the i-th range that Iterator traverses. + // current[i]_ is the actual traversing iterator. + const typename ParamGenerator::iterator begin1_; + const typename ParamGenerator::iterator end1_; + typename ParamGenerator::iterator current1_; + const typename ParamGenerator::iterator begin2_; + const typename ParamGenerator::iterator end2_; + typename ParamGenerator::iterator current2_; + const typename ParamGenerator::iterator begin3_; + const typename ParamGenerator::iterator end3_; + typename ParamGenerator::iterator current3_; + const typename ParamGenerator::iterator begin4_; + const typename ParamGenerator::iterator end4_; + typename ParamGenerator::iterator current4_; + const typename ParamGenerator::iterator begin5_; + const typename ParamGenerator::iterator end5_; + typename ParamGenerator::iterator current5_; + const typename ParamGenerator::iterator begin6_; + const typename ParamGenerator::iterator end6_; + typename ParamGenerator::iterator current6_; + ParamType current_value_; + }; // class CartesianProductGenerator6::Iterator + + // No implementation - assignment is unsupported. + void operator=(const CartesianProductGenerator6& other); + + const ParamGenerator g1_; + const ParamGenerator g2_; + const ParamGenerator g3_; + const ParamGenerator g4_; + const ParamGenerator g5_; + const ParamGenerator g6_; +}; // class CartesianProductGenerator6 + + +template +class CartesianProductGenerator7 + : public ParamGeneratorInterface< ::testing::tuple > { + public: + typedef ::testing::tuple ParamType; + + CartesianProductGenerator7(const ParamGenerator& g1, + const ParamGenerator& g2, const ParamGenerator& g3, + const ParamGenerator& g4, const ParamGenerator& g5, + const ParamGenerator& g6, const ParamGenerator& g7) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5), g6_(g6), g7_(g7) {} + virtual ~CartesianProductGenerator7() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, g1_, g1_.begin(), g2_, g2_.begin(), g3_, + g3_.begin(), g4_, g4_.begin(), g5_, g5_.begin(), g6_, g6_.begin(), g7_, + g7_.begin()); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, g1_, g1_.end(), g2_, g2_.end(), g3_, g3_.end(), + g4_, g4_.end(), g5_, g5_.end(), g6_, g6_.end(), g7_, g7_.end()); + } + + private: + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, + const ParamGenerator& g1, + const typename ParamGenerator::iterator& current1, + const ParamGenerator& g2, + const typename ParamGenerator::iterator& current2, + const ParamGenerator& g3, + const typename ParamGenerator::iterator& current3, + const ParamGenerator& g4, + const typename ParamGenerator::iterator& current4, + const ParamGenerator& g5, + const typename ParamGenerator::iterator& current5, + const ParamGenerator& g6, + const typename ParamGenerator::iterator& current6, + const ParamGenerator& g7, + const typename ParamGenerator::iterator& current7) + : base_(base), + begin1_(g1.begin()), end1_(g1.end()), current1_(current1), + begin2_(g2.begin()), end2_(g2.end()), current2_(current2), + begin3_(g3.begin()), end3_(g3.end()), current3_(current3), + begin4_(g4.begin()), end4_(g4.end()), current4_(current4), + begin5_(g5.begin()), end5_(g5.end()), current5_(current5), + begin6_(g6.begin()), end6_(g6.end()), current6_(current6), + begin7_(g7.begin()), end7_(g7.end()), current7_(current7) { + ComputeCurrentValue(); + } + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + // Advance should not be called on beyond-of-range iterators + // so no component iterators must be beyond end of range, either. + virtual void Advance() { + assert(!AtEnd()); + ++current7_; + if (current7_ == end7_) { + current7_ = begin7_; + ++current6_; + } + if (current6_ == end6_) { + current6_ = begin6_; + ++current5_; + } + if (current5_ == end5_) { + current5_ = begin5_; + ++current4_; + } + if (current4_ == end4_) { + current4_ = begin4_; + ++current3_; + } + if (current3_ == end3_) { + current3_ = begin3_; + ++current2_; + } + if (current2_ == end2_) { + current2_ = begin2_; + ++current1_; + } + ComputeCurrentValue(); + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + virtual const ParamType* Current() const { return ¤t_value_; } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + const Iterator* typed_other = + CheckedDowncastToActualType(&other); + // We must report iterators equal if they both point beyond their + // respective ranges. That can happen in a variety of fashions, + // so we have to consult AtEnd(). + return (AtEnd() && typed_other->AtEnd()) || + ( + current1_ == typed_other->current1_ && + current2_ == typed_other->current2_ && + current3_ == typed_other->current3_ && + current4_ == typed_other->current4_ && + current5_ == typed_other->current5_ && + current6_ == typed_other->current6_ && + current7_ == typed_other->current7_); + } + + private: + Iterator(const Iterator& other) + : base_(other.base_), + begin1_(other.begin1_), + end1_(other.end1_), + current1_(other.current1_), + begin2_(other.begin2_), + end2_(other.end2_), + current2_(other.current2_), + begin3_(other.begin3_), + end3_(other.end3_), + current3_(other.current3_), + begin4_(other.begin4_), + end4_(other.end4_), + current4_(other.current4_), + begin5_(other.begin5_), + end5_(other.end5_), + current5_(other.current5_), + begin6_(other.begin6_), + end6_(other.end6_), + current6_(other.current6_), + begin7_(other.begin7_), + end7_(other.end7_), + current7_(other.current7_) { + ComputeCurrentValue(); + } + + void ComputeCurrentValue() { + if (!AtEnd()) + current_value_ = ParamType(*current1_, *current2_, *current3_, + *current4_, *current5_, *current6_, *current7_); + } + bool AtEnd() const { + // We must report iterator past the end of the range when either of the + // component iterators has reached the end of its range. + return + current1_ == end1_ || + current2_ == end2_ || + current3_ == end3_ || + current4_ == end4_ || + current5_ == end5_ || + current6_ == end6_ || + current7_ == end7_; + } + + // No implementation - assignment is unsupported. + void operator=(const Iterator& other); + + const ParamGeneratorInterface* const base_; + // begin[i]_ and end[i]_ define the i-th range that Iterator traverses. + // current[i]_ is the actual traversing iterator. + const typename ParamGenerator::iterator begin1_; + const typename ParamGenerator::iterator end1_; + typename ParamGenerator::iterator current1_; + const typename ParamGenerator::iterator begin2_; + const typename ParamGenerator::iterator end2_; + typename ParamGenerator::iterator current2_; + const typename ParamGenerator::iterator begin3_; + const typename ParamGenerator::iterator end3_; + typename ParamGenerator::iterator current3_; + const typename ParamGenerator::iterator begin4_; + const typename ParamGenerator::iterator end4_; + typename ParamGenerator::iterator current4_; + const typename ParamGenerator::iterator begin5_; + const typename ParamGenerator::iterator end5_; + typename ParamGenerator::iterator current5_; + const typename ParamGenerator::iterator begin6_; + const typename ParamGenerator::iterator end6_; + typename ParamGenerator::iterator current6_; + const typename ParamGenerator::iterator begin7_; + const typename ParamGenerator::iterator end7_; + typename ParamGenerator::iterator current7_; + ParamType current_value_; + }; // class CartesianProductGenerator7::Iterator + + // No implementation - assignment is unsupported. + void operator=(const CartesianProductGenerator7& other); + + const ParamGenerator g1_; + const ParamGenerator g2_; + const ParamGenerator g3_; + const ParamGenerator g4_; + const ParamGenerator g5_; + const ParamGenerator g6_; + const ParamGenerator g7_; +}; // class CartesianProductGenerator7 + + +template +class CartesianProductGenerator8 + : public ParamGeneratorInterface< ::testing::tuple > { + public: + typedef ::testing::tuple ParamType; + + CartesianProductGenerator8(const ParamGenerator& g1, + const ParamGenerator& g2, const ParamGenerator& g3, + const ParamGenerator& g4, const ParamGenerator& g5, + const ParamGenerator& g6, const ParamGenerator& g7, + const ParamGenerator& g8) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5), g6_(g6), g7_(g7), + g8_(g8) {} + virtual ~CartesianProductGenerator8() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, g1_, g1_.begin(), g2_, g2_.begin(), g3_, + g3_.begin(), g4_, g4_.begin(), g5_, g5_.begin(), g6_, g6_.begin(), g7_, + g7_.begin(), g8_, g8_.begin()); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, g1_, g1_.end(), g2_, g2_.end(), g3_, g3_.end(), + g4_, g4_.end(), g5_, g5_.end(), g6_, g6_.end(), g7_, g7_.end(), g8_, + g8_.end()); + } + + private: + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, + const ParamGenerator& g1, + const typename ParamGenerator::iterator& current1, + const ParamGenerator& g2, + const typename ParamGenerator::iterator& current2, + const ParamGenerator& g3, + const typename ParamGenerator::iterator& current3, + const ParamGenerator& g4, + const typename ParamGenerator::iterator& current4, + const ParamGenerator& g5, + const typename ParamGenerator::iterator& current5, + const ParamGenerator& g6, + const typename ParamGenerator::iterator& current6, + const ParamGenerator& g7, + const typename ParamGenerator::iterator& current7, + const ParamGenerator& g8, + const typename ParamGenerator::iterator& current8) + : base_(base), + begin1_(g1.begin()), end1_(g1.end()), current1_(current1), + begin2_(g2.begin()), end2_(g2.end()), current2_(current2), + begin3_(g3.begin()), end3_(g3.end()), current3_(current3), + begin4_(g4.begin()), end4_(g4.end()), current4_(current4), + begin5_(g5.begin()), end5_(g5.end()), current5_(current5), + begin6_(g6.begin()), end6_(g6.end()), current6_(current6), + begin7_(g7.begin()), end7_(g7.end()), current7_(current7), + begin8_(g8.begin()), end8_(g8.end()), current8_(current8) { + ComputeCurrentValue(); + } + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + // Advance should not be called on beyond-of-range iterators + // so no component iterators must be beyond end of range, either. + virtual void Advance() { + assert(!AtEnd()); + ++current8_; + if (current8_ == end8_) { + current8_ = begin8_; + ++current7_; + } + if (current7_ == end7_) { + current7_ = begin7_; + ++current6_; + } + if (current6_ == end6_) { + current6_ = begin6_; + ++current5_; + } + if (current5_ == end5_) { + current5_ = begin5_; + ++current4_; + } + if (current4_ == end4_) { + current4_ = begin4_; + ++current3_; + } + if (current3_ == end3_) { + current3_ = begin3_; + ++current2_; + } + if (current2_ == end2_) { + current2_ = begin2_; + ++current1_; + } + ComputeCurrentValue(); + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + virtual const ParamType* Current() const { return ¤t_value_; } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + const Iterator* typed_other = + CheckedDowncastToActualType(&other); + // We must report iterators equal if they both point beyond their + // respective ranges. That can happen in a variety of fashions, + // so we have to consult AtEnd(). + return (AtEnd() && typed_other->AtEnd()) || + ( + current1_ == typed_other->current1_ && + current2_ == typed_other->current2_ && + current3_ == typed_other->current3_ && + current4_ == typed_other->current4_ && + current5_ == typed_other->current5_ && + current6_ == typed_other->current6_ && + current7_ == typed_other->current7_ && + current8_ == typed_other->current8_); + } + + private: + Iterator(const Iterator& other) + : base_(other.base_), + begin1_(other.begin1_), + end1_(other.end1_), + current1_(other.current1_), + begin2_(other.begin2_), + end2_(other.end2_), + current2_(other.current2_), + begin3_(other.begin3_), + end3_(other.end3_), + current3_(other.current3_), + begin4_(other.begin4_), + end4_(other.end4_), + current4_(other.current4_), + begin5_(other.begin5_), + end5_(other.end5_), + current5_(other.current5_), + begin6_(other.begin6_), + end6_(other.end6_), + current6_(other.current6_), + begin7_(other.begin7_), + end7_(other.end7_), + current7_(other.current7_), + begin8_(other.begin8_), + end8_(other.end8_), + current8_(other.current8_) { + ComputeCurrentValue(); + } + + void ComputeCurrentValue() { + if (!AtEnd()) + current_value_ = ParamType(*current1_, *current2_, *current3_, + *current4_, *current5_, *current6_, *current7_, *current8_); + } + bool AtEnd() const { + // We must report iterator past the end of the range when either of the + // component iterators has reached the end of its range. + return + current1_ == end1_ || + current2_ == end2_ || + current3_ == end3_ || + current4_ == end4_ || + current5_ == end5_ || + current6_ == end6_ || + current7_ == end7_ || + current8_ == end8_; + } + + // No implementation - assignment is unsupported. + void operator=(const Iterator& other); + + const ParamGeneratorInterface* const base_; + // begin[i]_ and end[i]_ define the i-th range that Iterator traverses. + // current[i]_ is the actual traversing iterator. + const typename ParamGenerator::iterator begin1_; + const typename ParamGenerator::iterator end1_; + typename ParamGenerator::iterator current1_; + const typename ParamGenerator::iterator begin2_; + const typename ParamGenerator::iterator end2_; + typename ParamGenerator::iterator current2_; + const typename ParamGenerator::iterator begin3_; + const typename ParamGenerator::iterator end3_; + typename ParamGenerator::iterator current3_; + const typename ParamGenerator::iterator begin4_; + const typename ParamGenerator::iterator end4_; + typename ParamGenerator::iterator current4_; + const typename ParamGenerator::iterator begin5_; + const typename ParamGenerator::iterator end5_; + typename ParamGenerator::iterator current5_; + const typename ParamGenerator::iterator begin6_; + const typename ParamGenerator::iterator end6_; + typename ParamGenerator::iterator current6_; + const typename ParamGenerator::iterator begin7_; + const typename ParamGenerator::iterator end7_; + typename ParamGenerator::iterator current7_; + const typename ParamGenerator::iterator begin8_; + const typename ParamGenerator::iterator end8_; + typename ParamGenerator::iterator current8_; + ParamType current_value_; + }; // class CartesianProductGenerator8::Iterator + + // No implementation - assignment is unsupported. + void operator=(const CartesianProductGenerator8& other); + + const ParamGenerator g1_; + const ParamGenerator g2_; + const ParamGenerator g3_; + const ParamGenerator g4_; + const ParamGenerator g5_; + const ParamGenerator g6_; + const ParamGenerator g7_; + const ParamGenerator g8_; +}; // class CartesianProductGenerator8 + + +template +class CartesianProductGenerator9 + : public ParamGeneratorInterface< ::testing::tuple > { + public: + typedef ::testing::tuple ParamType; + + CartesianProductGenerator9(const ParamGenerator& g1, + const ParamGenerator& g2, const ParamGenerator& g3, + const ParamGenerator& g4, const ParamGenerator& g5, + const ParamGenerator& g6, const ParamGenerator& g7, + const ParamGenerator& g8, const ParamGenerator& g9) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5), g6_(g6), g7_(g7), g8_(g8), + g9_(g9) {} + virtual ~CartesianProductGenerator9() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, g1_, g1_.begin(), g2_, g2_.begin(), g3_, + g3_.begin(), g4_, g4_.begin(), g5_, g5_.begin(), g6_, g6_.begin(), g7_, + g7_.begin(), g8_, g8_.begin(), g9_, g9_.begin()); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, g1_, g1_.end(), g2_, g2_.end(), g3_, g3_.end(), + g4_, g4_.end(), g5_, g5_.end(), g6_, g6_.end(), g7_, g7_.end(), g8_, + g8_.end(), g9_, g9_.end()); + } + + private: + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, + const ParamGenerator& g1, + const typename ParamGenerator::iterator& current1, + const ParamGenerator& g2, + const typename ParamGenerator::iterator& current2, + const ParamGenerator& g3, + const typename ParamGenerator::iterator& current3, + const ParamGenerator& g4, + const typename ParamGenerator::iterator& current4, + const ParamGenerator& g5, + const typename ParamGenerator::iterator& current5, + const ParamGenerator& g6, + const typename ParamGenerator::iterator& current6, + const ParamGenerator& g7, + const typename ParamGenerator::iterator& current7, + const ParamGenerator& g8, + const typename ParamGenerator::iterator& current8, + const ParamGenerator& g9, + const typename ParamGenerator::iterator& current9) + : base_(base), + begin1_(g1.begin()), end1_(g1.end()), current1_(current1), + begin2_(g2.begin()), end2_(g2.end()), current2_(current2), + begin3_(g3.begin()), end3_(g3.end()), current3_(current3), + begin4_(g4.begin()), end4_(g4.end()), current4_(current4), + begin5_(g5.begin()), end5_(g5.end()), current5_(current5), + begin6_(g6.begin()), end6_(g6.end()), current6_(current6), + begin7_(g7.begin()), end7_(g7.end()), current7_(current7), + begin8_(g8.begin()), end8_(g8.end()), current8_(current8), + begin9_(g9.begin()), end9_(g9.end()), current9_(current9) { + ComputeCurrentValue(); + } + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + // Advance should not be called on beyond-of-range iterators + // so no component iterators must be beyond end of range, either. + virtual void Advance() { + assert(!AtEnd()); + ++current9_; + if (current9_ == end9_) { + current9_ = begin9_; + ++current8_; + } + if (current8_ == end8_) { + current8_ = begin8_; + ++current7_; + } + if (current7_ == end7_) { + current7_ = begin7_; + ++current6_; + } + if (current6_ == end6_) { + current6_ = begin6_; + ++current5_; + } + if (current5_ == end5_) { + current5_ = begin5_; + ++current4_; + } + if (current4_ == end4_) { + current4_ = begin4_; + ++current3_; + } + if (current3_ == end3_) { + current3_ = begin3_; + ++current2_; + } + if (current2_ == end2_) { + current2_ = begin2_; + ++current1_; + } + ComputeCurrentValue(); + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + virtual const ParamType* Current() const { return ¤t_value_; } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + const Iterator* typed_other = + CheckedDowncastToActualType(&other); + // We must report iterators equal if they both point beyond their + // respective ranges. That can happen in a variety of fashions, + // so we have to consult AtEnd(). + return (AtEnd() && typed_other->AtEnd()) || + ( + current1_ == typed_other->current1_ && + current2_ == typed_other->current2_ && + current3_ == typed_other->current3_ && + current4_ == typed_other->current4_ && + current5_ == typed_other->current5_ && + current6_ == typed_other->current6_ && + current7_ == typed_other->current7_ && + current8_ == typed_other->current8_ && + current9_ == typed_other->current9_); + } + + private: + Iterator(const Iterator& other) + : base_(other.base_), + begin1_(other.begin1_), + end1_(other.end1_), + current1_(other.current1_), + begin2_(other.begin2_), + end2_(other.end2_), + current2_(other.current2_), + begin3_(other.begin3_), + end3_(other.end3_), + current3_(other.current3_), + begin4_(other.begin4_), + end4_(other.end4_), + current4_(other.current4_), + begin5_(other.begin5_), + end5_(other.end5_), + current5_(other.current5_), + begin6_(other.begin6_), + end6_(other.end6_), + current6_(other.current6_), + begin7_(other.begin7_), + end7_(other.end7_), + current7_(other.current7_), + begin8_(other.begin8_), + end8_(other.end8_), + current8_(other.current8_), + begin9_(other.begin9_), + end9_(other.end9_), + current9_(other.current9_) { + ComputeCurrentValue(); + } + + void ComputeCurrentValue() { + if (!AtEnd()) + current_value_ = ParamType(*current1_, *current2_, *current3_, + *current4_, *current5_, *current6_, *current7_, *current8_, + *current9_); + } + bool AtEnd() const { + // We must report iterator past the end of the range when either of the + // component iterators has reached the end of its range. + return + current1_ == end1_ || + current2_ == end2_ || + current3_ == end3_ || + current4_ == end4_ || + current5_ == end5_ || + current6_ == end6_ || + current7_ == end7_ || + current8_ == end8_ || + current9_ == end9_; + } + + // No implementation - assignment is unsupported. + void operator=(const Iterator& other); + + const ParamGeneratorInterface* const base_; + // begin[i]_ and end[i]_ define the i-th range that Iterator traverses. + // current[i]_ is the actual traversing iterator. + const typename ParamGenerator::iterator begin1_; + const typename ParamGenerator::iterator end1_; + typename ParamGenerator::iterator current1_; + const typename ParamGenerator::iterator begin2_; + const typename ParamGenerator::iterator end2_; + typename ParamGenerator::iterator current2_; + const typename ParamGenerator::iterator begin3_; + const typename ParamGenerator::iterator end3_; + typename ParamGenerator::iterator current3_; + const typename ParamGenerator::iterator begin4_; + const typename ParamGenerator::iterator end4_; + typename ParamGenerator::iterator current4_; + const typename ParamGenerator::iterator begin5_; + const typename ParamGenerator::iterator end5_; + typename ParamGenerator::iterator current5_; + const typename ParamGenerator::iterator begin6_; + const typename ParamGenerator::iterator end6_; + typename ParamGenerator::iterator current6_; + const typename ParamGenerator::iterator begin7_; + const typename ParamGenerator::iterator end7_; + typename ParamGenerator::iterator current7_; + const typename ParamGenerator::iterator begin8_; + const typename ParamGenerator::iterator end8_; + typename ParamGenerator::iterator current8_; + const typename ParamGenerator::iterator begin9_; + const typename ParamGenerator::iterator end9_; + typename ParamGenerator::iterator current9_; + ParamType current_value_; + }; // class CartesianProductGenerator9::Iterator + + // No implementation - assignment is unsupported. + void operator=(const CartesianProductGenerator9& other); + + const ParamGenerator g1_; + const ParamGenerator g2_; + const ParamGenerator g3_; + const ParamGenerator g4_; + const ParamGenerator g5_; + const ParamGenerator g6_; + const ParamGenerator g7_; + const ParamGenerator g8_; + const ParamGenerator g9_; +}; // class CartesianProductGenerator9 + + +template +class CartesianProductGenerator10 + : public ParamGeneratorInterface< ::testing::tuple > { + public: + typedef ::testing::tuple ParamType; + + CartesianProductGenerator10(const ParamGenerator& g1, + const ParamGenerator& g2, const ParamGenerator& g3, + const ParamGenerator& g4, const ParamGenerator& g5, + const ParamGenerator& g6, const ParamGenerator& g7, + const ParamGenerator& g8, const ParamGenerator& g9, + const ParamGenerator& g10) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5), g6_(g6), g7_(g7), g8_(g8), + g9_(g9), g10_(g10) {} + virtual ~CartesianProductGenerator10() {} + + virtual ParamIteratorInterface* Begin() const { + return new Iterator(this, g1_, g1_.begin(), g2_, g2_.begin(), g3_, + g3_.begin(), g4_, g4_.begin(), g5_, g5_.begin(), g6_, g6_.begin(), g7_, + g7_.begin(), g8_, g8_.begin(), g9_, g9_.begin(), g10_, g10_.begin()); + } + virtual ParamIteratorInterface* End() const { + return new Iterator(this, g1_, g1_.end(), g2_, g2_.end(), g3_, g3_.end(), + g4_, g4_.end(), g5_, g5_.end(), g6_, g6_.end(), g7_, g7_.end(), g8_, + g8_.end(), g9_, g9_.end(), g10_, g10_.end()); + } + + private: + class Iterator : public ParamIteratorInterface { + public: + Iterator(const ParamGeneratorInterface* base, + const ParamGenerator& g1, + const typename ParamGenerator::iterator& current1, + const ParamGenerator& g2, + const typename ParamGenerator::iterator& current2, + const ParamGenerator& g3, + const typename ParamGenerator::iterator& current3, + const ParamGenerator& g4, + const typename ParamGenerator::iterator& current4, + const ParamGenerator& g5, + const typename ParamGenerator::iterator& current5, + const ParamGenerator& g6, + const typename ParamGenerator::iterator& current6, + const ParamGenerator& g7, + const typename ParamGenerator::iterator& current7, + const ParamGenerator& g8, + const typename ParamGenerator::iterator& current8, + const ParamGenerator& g9, + const typename ParamGenerator::iterator& current9, + const ParamGenerator& g10, + const typename ParamGenerator::iterator& current10) + : base_(base), + begin1_(g1.begin()), end1_(g1.end()), current1_(current1), + begin2_(g2.begin()), end2_(g2.end()), current2_(current2), + begin3_(g3.begin()), end3_(g3.end()), current3_(current3), + begin4_(g4.begin()), end4_(g4.end()), current4_(current4), + begin5_(g5.begin()), end5_(g5.end()), current5_(current5), + begin6_(g6.begin()), end6_(g6.end()), current6_(current6), + begin7_(g7.begin()), end7_(g7.end()), current7_(current7), + begin8_(g8.begin()), end8_(g8.end()), current8_(current8), + begin9_(g9.begin()), end9_(g9.end()), current9_(current9), + begin10_(g10.begin()), end10_(g10.end()), current10_(current10) { + ComputeCurrentValue(); + } + virtual ~Iterator() {} + + virtual const ParamGeneratorInterface* BaseGenerator() const { + return base_; + } + // Advance should not be called on beyond-of-range iterators + // so no component iterators must be beyond end of range, either. + virtual void Advance() { + assert(!AtEnd()); + ++current10_; + if (current10_ == end10_) { + current10_ = begin10_; + ++current9_; + } + if (current9_ == end9_) { + current9_ = begin9_; + ++current8_; + } + if (current8_ == end8_) { + current8_ = begin8_; + ++current7_; + } + if (current7_ == end7_) { + current7_ = begin7_; + ++current6_; + } + if (current6_ == end6_) { + current6_ = begin6_; + ++current5_; + } + if (current5_ == end5_) { + current5_ = begin5_; + ++current4_; + } + if (current4_ == end4_) { + current4_ = begin4_; + ++current3_; + } + if (current3_ == end3_) { + current3_ = begin3_; + ++current2_; + } + if (current2_ == end2_) { + current2_ = begin2_; + ++current1_; + } + ComputeCurrentValue(); + } + virtual ParamIteratorInterface* Clone() const { + return new Iterator(*this); + } + virtual const ParamType* Current() const { return ¤t_value_; } + virtual bool Equals(const ParamIteratorInterface& other) const { + // Having the same base generator guarantees that the other + // iterator is of the same type and we can downcast. + GTEST_CHECK_(BaseGenerator() == other.BaseGenerator()) + << "The program attempted to compare iterators " + << "from different generators." << std::endl; + const Iterator* typed_other = + CheckedDowncastToActualType(&other); + // We must report iterators equal if they both point beyond their + // respective ranges. That can happen in a variety of fashions, + // so we have to consult AtEnd(). + return (AtEnd() && typed_other->AtEnd()) || + ( + current1_ == typed_other->current1_ && + current2_ == typed_other->current2_ && + current3_ == typed_other->current3_ && + current4_ == typed_other->current4_ && + current5_ == typed_other->current5_ && + current6_ == typed_other->current6_ && + current7_ == typed_other->current7_ && + current8_ == typed_other->current8_ && + current9_ == typed_other->current9_ && + current10_ == typed_other->current10_); + } + + private: + Iterator(const Iterator& other) + : base_(other.base_), + begin1_(other.begin1_), + end1_(other.end1_), + current1_(other.current1_), + begin2_(other.begin2_), + end2_(other.end2_), + current2_(other.current2_), + begin3_(other.begin3_), + end3_(other.end3_), + current3_(other.current3_), + begin4_(other.begin4_), + end4_(other.end4_), + current4_(other.current4_), + begin5_(other.begin5_), + end5_(other.end5_), + current5_(other.current5_), + begin6_(other.begin6_), + end6_(other.end6_), + current6_(other.current6_), + begin7_(other.begin7_), + end7_(other.end7_), + current7_(other.current7_), + begin8_(other.begin8_), + end8_(other.end8_), + current8_(other.current8_), + begin9_(other.begin9_), + end9_(other.end9_), + current9_(other.current9_), + begin10_(other.begin10_), + end10_(other.end10_), + current10_(other.current10_) { + ComputeCurrentValue(); + } + + void ComputeCurrentValue() { + if (!AtEnd()) + current_value_ = ParamType(*current1_, *current2_, *current3_, + *current4_, *current5_, *current6_, *current7_, *current8_, + *current9_, *current10_); + } + bool AtEnd() const { + // We must report iterator past the end of the range when either of the + // component iterators has reached the end of its range. + return + current1_ == end1_ || + current2_ == end2_ || + current3_ == end3_ || + current4_ == end4_ || + current5_ == end5_ || + current6_ == end6_ || + current7_ == end7_ || + current8_ == end8_ || + current9_ == end9_ || + current10_ == end10_; + } + + // No implementation - assignment is unsupported. + void operator=(const Iterator& other); + + const ParamGeneratorInterface* const base_; + // begin[i]_ and end[i]_ define the i-th range that Iterator traverses. + // current[i]_ is the actual traversing iterator. + const typename ParamGenerator::iterator begin1_; + const typename ParamGenerator::iterator end1_; + typename ParamGenerator::iterator current1_; + const typename ParamGenerator::iterator begin2_; + const typename ParamGenerator::iterator end2_; + typename ParamGenerator::iterator current2_; + const typename ParamGenerator::iterator begin3_; + const typename ParamGenerator::iterator end3_; + typename ParamGenerator::iterator current3_; + const typename ParamGenerator::iterator begin4_; + const typename ParamGenerator::iterator end4_; + typename ParamGenerator::iterator current4_; + const typename ParamGenerator::iterator begin5_; + const typename ParamGenerator::iterator end5_; + typename ParamGenerator::iterator current5_; + const typename ParamGenerator::iterator begin6_; + const typename ParamGenerator::iterator end6_; + typename ParamGenerator::iterator current6_; + const typename ParamGenerator::iterator begin7_; + const typename ParamGenerator::iterator end7_; + typename ParamGenerator::iterator current7_; + const typename ParamGenerator::iterator begin8_; + const typename ParamGenerator::iterator end8_; + typename ParamGenerator::iterator current8_; + const typename ParamGenerator::iterator begin9_; + const typename ParamGenerator::iterator end9_; + typename ParamGenerator::iterator current9_; + const typename ParamGenerator::iterator begin10_; + const typename ParamGenerator::iterator end10_; + typename ParamGenerator::iterator current10_; + ParamType current_value_; + }; // class CartesianProductGenerator10::Iterator + + // No implementation - assignment is unsupported. + void operator=(const CartesianProductGenerator10& other); + + const ParamGenerator g1_; + const ParamGenerator g2_; + const ParamGenerator g3_; + const ParamGenerator g4_; + const ParamGenerator g5_; + const ParamGenerator g6_; + const ParamGenerator g7_; + const ParamGenerator g8_; + const ParamGenerator g9_; + const ParamGenerator g10_; +}; // class CartesianProductGenerator10 + + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// Helper classes providing Combine() with polymorphic features. They allow +// casting CartesianProductGeneratorN to ParamGenerator if T is +// convertible to U. +// +template +class CartesianProductHolder2 { + public: +CartesianProductHolder2(const Generator1& g1, const Generator2& g2) + : g1_(g1), g2_(g2) {} + template + operator ParamGenerator< ::testing::tuple >() const { + return ParamGenerator< ::testing::tuple >( + new CartesianProductGenerator2( + static_cast >(g1_), + static_cast >(g2_))); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const CartesianProductHolder2& other); + + const Generator1 g1_; + const Generator2 g2_; +}; // class CartesianProductHolder2 + +template +class CartesianProductHolder3 { + public: +CartesianProductHolder3(const Generator1& g1, const Generator2& g2, + const Generator3& g3) + : g1_(g1), g2_(g2), g3_(g3) {} + template + operator ParamGenerator< ::testing::tuple >() const { + return ParamGenerator< ::testing::tuple >( + new CartesianProductGenerator3( + static_cast >(g1_), + static_cast >(g2_), + static_cast >(g3_))); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const CartesianProductHolder3& other); + + const Generator1 g1_; + const Generator2 g2_; + const Generator3 g3_; +}; // class CartesianProductHolder3 + +template +class CartesianProductHolder4 { + public: +CartesianProductHolder4(const Generator1& g1, const Generator2& g2, + const Generator3& g3, const Generator4& g4) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4) {} + template + operator ParamGenerator< ::testing::tuple >() const { + return ParamGenerator< ::testing::tuple >( + new CartesianProductGenerator4( + static_cast >(g1_), + static_cast >(g2_), + static_cast >(g3_), + static_cast >(g4_))); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const CartesianProductHolder4& other); + + const Generator1 g1_; + const Generator2 g2_; + const Generator3 g3_; + const Generator4 g4_; +}; // class CartesianProductHolder4 + +template +class CartesianProductHolder5 { + public: +CartesianProductHolder5(const Generator1& g1, const Generator2& g2, + const Generator3& g3, const Generator4& g4, const Generator5& g5) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5) {} + template + operator ParamGenerator< ::testing::tuple >() const { + return ParamGenerator< ::testing::tuple >( + new CartesianProductGenerator5( + static_cast >(g1_), + static_cast >(g2_), + static_cast >(g3_), + static_cast >(g4_), + static_cast >(g5_))); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const CartesianProductHolder5& other); + + const Generator1 g1_; + const Generator2 g2_; + const Generator3 g3_; + const Generator4 g4_; + const Generator5 g5_; +}; // class CartesianProductHolder5 + +template +class CartesianProductHolder6 { + public: +CartesianProductHolder6(const Generator1& g1, const Generator2& g2, + const Generator3& g3, const Generator4& g4, const Generator5& g5, + const Generator6& g6) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5), g6_(g6) {} + template + operator ParamGenerator< ::testing::tuple >() const { + return ParamGenerator< ::testing::tuple >( + new CartesianProductGenerator6( + static_cast >(g1_), + static_cast >(g2_), + static_cast >(g3_), + static_cast >(g4_), + static_cast >(g5_), + static_cast >(g6_))); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const CartesianProductHolder6& other); + + const Generator1 g1_; + const Generator2 g2_; + const Generator3 g3_; + const Generator4 g4_; + const Generator5 g5_; + const Generator6 g6_; +}; // class CartesianProductHolder6 + +template +class CartesianProductHolder7 { + public: +CartesianProductHolder7(const Generator1& g1, const Generator2& g2, + const Generator3& g3, const Generator4& g4, const Generator5& g5, + const Generator6& g6, const Generator7& g7) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5), g6_(g6), g7_(g7) {} + template + operator ParamGenerator< ::testing::tuple >() const { + return ParamGenerator< ::testing::tuple >( + new CartesianProductGenerator7( + static_cast >(g1_), + static_cast >(g2_), + static_cast >(g3_), + static_cast >(g4_), + static_cast >(g5_), + static_cast >(g6_), + static_cast >(g7_))); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const CartesianProductHolder7& other); + + const Generator1 g1_; + const Generator2 g2_; + const Generator3 g3_; + const Generator4 g4_; + const Generator5 g5_; + const Generator6 g6_; + const Generator7 g7_; +}; // class CartesianProductHolder7 + +template +class CartesianProductHolder8 { + public: +CartesianProductHolder8(const Generator1& g1, const Generator2& g2, + const Generator3& g3, const Generator4& g4, const Generator5& g5, + const Generator6& g6, const Generator7& g7, const Generator8& g8) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5), g6_(g6), g7_(g7), + g8_(g8) {} + template + operator ParamGenerator< ::testing::tuple >() const { + return ParamGenerator< ::testing::tuple >( + new CartesianProductGenerator8( + static_cast >(g1_), + static_cast >(g2_), + static_cast >(g3_), + static_cast >(g4_), + static_cast >(g5_), + static_cast >(g6_), + static_cast >(g7_), + static_cast >(g8_))); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const CartesianProductHolder8& other); + + const Generator1 g1_; + const Generator2 g2_; + const Generator3 g3_; + const Generator4 g4_; + const Generator5 g5_; + const Generator6 g6_; + const Generator7 g7_; + const Generator8 g8_; +}; // class CartesianProductHolder8 + +template +class CartesianProductHolder9 { + public: +CartesianProductHolder9(const Generator1& g1, const Generator2& g2, + const Generator3& g3, const Generator4& g4, const Generator5& g5, + const Generator6& g6, const Generator7& g7, const Generator8& g8, + const Generator9& g9) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5), g6_(g6), g7_(g7), g8_(g8), + g9_(g9) {} + template + operator ParamGenerator< ::testing::tuple >() const { + return ParamGenerator< ::testing::tuple >( + new CartesianProductGenerator9( + static_cast >(g1_), + static_cast >(g2_), + static_cast >(g3_), + static_cast >(g4_), + static_cast >(g5_), + static_cast >(g6_), + static_cast >(g7_), + static_cast >(g8_), + static_cast >(g9_))); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const CartesianProductHolder9& other); + + const Generator1 g1_; + const Generator2 g2_; + const Generator3 g3_; + const Generator4 g4_; + const Generator5 g5_; + const Generator6 g6_; + const Generator7 g7_; + const Generator8 g8_; + const Generator9 g9_; +}; // class CartesianProductHolder9 + +template +class CartesianProductHolder10 { + public: +CartesianProductHolder10(const Generator1& g1, const Generator2& g2, + const Generator3& g3, const Generator4& g4, const Generator5& g5, + const Generator6& g6, const Generator7& g7, const Generator8& g8, + const Generator9& g9, const Generator10& g10) + : g1_(g1), g2_(g2), g3_(g3), g4_(g4), g5_(g5), g6_(g6), g7_(g7), g8_(g8), + g9_(g9), g10_(g10) {} + template + operator ParamGenerator< ::testing::tuple >() const { + return ParamGenerator< ::testing::tuple >( + new CartesianProductGenerator10( + static_cast >(g1_), + static_cast >(g2_), + static_cast >(g3_), + static_cast >(g4_), + static_cast >(g5_), + static_cast >(g6_), + static_cast >(g7_), + static_cast >(g8_), + static_cast >(g9_), + static_cast >(g10_))); + } + + private: + // No implementation - assignment is unsupported. + void operator=(const CartesianProductHolder10& other); + + const Generator1 g1_; + const Generator2 g2_; + const Generator3 g3_; + const Generator4 g4_; + const Generator5 g5_; + const Generator6 g6_; + const Generator7 g7_; + const Generator8 g8_; + const Generator9 g9_; + const Generator10 g10_; +}; // class CartesianProductHolder10 + +# endif // GTEST_HAS_COMBINE + +} // namespace internal +} // namespace testing + +#endif // GTEST_HAS_PARAM_TEST + +#endif // GTEST_INCLUDE_GTEST_INTERNAL_GTEST_PARAM_UTIL_GENERATED_H_ + +#if GTEST_HAS_PARAM_TEST + +namespace testing { + +// Functions producing parameter generators. +// +// Google Test uses these generators to produce parameters for value- +// parameterized tests. When a parameterized test case is instantiated +// with a particular generator, Google Test creates and runs tests +// for each element in the sequence produced by the generator. +// +// In the following sample, tests from test case FooTest are instantiated +// each three times with parameter values 3, 5, and 8: +// +// class FooTest : public TestWithParam { ... }; +// +// TEST_P(FooTest, TestThis) { +// } +// TEST_P(FooTest, TestThat) { +// } +// INSTANTIATE_TEST_CASE_P(TestSequence, FooTest, Values(3, 5, 8)); +// + +// Range() returns generators providing sequences of values in a range. +// +// Synopsis: +// Range(start, end) +// - returns a generator producing a sequence of values {start, start+1, +// start+2, ..., }. +// Range(start, end, step) +// - returns a generator producing a sequence of values {start, start+step, +// start+step+step, ..., }. +// Notes: +// * The generated sequences never include end. For example, Range(1, 5) +// returns a generator producing a sequence {1, 2, 3, 4}. Range(1, 9, 2) +// returns a generator producing {1, 3, 5, 7}. +// * start and end must have the same type. That type may be any integral or +// floating-point type or a user defined type satisfying these conditions: +// * It must be assignable (have operator=() defined). +// * It must have operator+() (operator+(int-compatible type) for +// two-operand version). +// * It must have operator<() defined. +// Elements in the resulting sequences will also have that type. +// * Condition start < end must be satisfied in order for resulting sequences +// to contain any elements. +// +template +internal::ParamGenerator Range(T start, T end, IncrementT step) { + return internal::ParamGenerator( + new internal::RangeGenerator(start, end, step)); +} + +template +internal::ParamGenerator Range(T start, T end) { + return Range(start, end, 1); +} + +// ValuesIn() function allows generation of tests with parameters coming from +// a container. +// +// Synopsis: +// ValuesIn(const T (&array)[N]) +// - returns a generator producing sequences with elements from +// a C-style array. +// ValuesIn(const Container& container) +// - returns a generator producing sequences with elements from +// an STL-style container. +// ValuesIn(Iterator begin, Iterator end) +// - returns a generator producing sequences with elements from +// a range [begin, end) defined by a pair of STL-style iterators. These +// iterators can also be plain C pointers. +// +// Please note that ValuesIn copies the values from the containers +// passed in and keeps them to generate tests in RUN_ALL_TESTS(). +// +// Examples: +// +// This instantiates tests from test case StringTest +// each with C-string values of "foo", "bar", and "baz": +// +// const char* strings[] = {"foo", "bar", "baz"}; +// INSTANTIATE_TEST_CASE_P(StringSequence, SrtingTest, ValuesIn(strings)); +// +// This instantiates tests from test case StlStringTest +// each with STL strings with values "a" and "b": +// +// ::std::vector< ::std::string> GetParameterStrings() { +// ::std::vector< ::std::string> v; +// v.push_back("a"); +// v.push_back("b"); +// return v; +// } +// +// INSTANTIATE_TEST_CASE_P(CharSequence, +// StlStringTest, +// ValuesIn(GetParameterStrings())); +// +// +// This will also instantiate tests from CharTest +// each with parameter values 'a' and 'b': +// +// ::std::list GetParameterChars() { +// ::std::list list; +// list.push_back('a'); +// list.push_back('b'); +// return list; +// } +// ::std::list l = GetParameterChars(); +// INSTANTIATE_TEST_CASE_P(CharSequence2, +// CharTest, +// ValuesIn(l.begin(), l.end())); +// +template +internal::ParamGenerator< + typename ::testing::internal::IteratorTraits::value_type> +ValuesIn(ForwardIterator begin, ForwardIterator end) { + typedef typename ::testing::internal::IteratorTraits + ::value_type ParamType; + return internal::ParamGenerator( + new internal::ValuesInIteratorRangeGenerator(begin, end)); +} + +template +internal::ParamGenerator ValuesIn(const T (&array)[N]) { + return ValuesIn(array, array + N); +} + +template +internal::ParamGenerator ValuesIn( + const Container& container) { + return ValuesIn(container.begin(), container.end()); +} + +// Values() allows generating tests from explicitly specified list of +// parameters. +// +// Synopsis: +// Values(T v1, T v2, ..., T vN) +// - returns a generator producing sequences with elements v1, v2, ..., vN. +// +// For example, this instantiates tests from test case BarTest each +// with values "one", "two", and "three": +// +// INSTANTIATE_TEST_CASE_P(NumSequence, BarTest, Values("one", "two", "three")); +// +// This instantiates tests from test case BazTest each with values 1, 2, 3.5. +// The exact type of values will depend on the type of parameter in BazTest. +// +// INSTANTIATE_TEST_CASE_P(FloatingNumbers, BazTest, Values(1, 2, 3.5)); +// +// Currently, Values() supports from 1 to 50 parameters. +// +template +internal::ValueArray1 Values(T1 v1) { + return internal::ValueArray1(v1); +} + +template +internal::ValueArray2 Values(T1 v1, T2 v2) { + return internal::ValueArray2(v1, v2); +} + +template +internal::ValueArray3 Values(T1 v1, T2 v2, T3 v3) { + return internal::ValueArray3(v1, v2, v3); +} + +template +internal::ValueArray4 Values(T1 v1, T2 v2, T3 v3, T4 v4) { + return internal::ValueArray4(v1, v2, v3, v4); +} + +template +internal::ValueArray5 Values(T1 v1, T2 v2, T3 v3, T4 v4, + T5 v5) { + return internal::ValueArray5(v1, v2, v3, v4, v5); +} + +template +internal::ValueArray6 Values(T1 v1, T2 v2, T3 v3, + T4 v4, T5 v5, T6 v6) { + return internal::ValueArray6(v1, v2, v3, v4, v5, v6); +} + +template +internal::ValueArray7 Values(T1 v1, T2 v2, T3 v3, + T4 v4, T5 v5, T6 v6, T7 v7) { + return internal::ValueArray7(v1, v2, v3, v4, v5, + v6, v7); +} + +template +internal::ValueArray8 Values(T1 v1, T2 v2, + T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8) { + return internal::ValueArray8(v1, v2, v3, v4, + v5, v6, v7, v8); +} + +template +internal::ValueArray9 Values(T1 v1, T2 v2, + T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9) { + return internal::ValueArray9(v1, v2, v3, + v4, v5, v6, v7, v8, v9); +} + +template +internal::ValueArray10 Values(T1 v1, + T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10) { + return internal::ValueArray10(v1, + v2, v3, v4, v5, v6, v7, v8, v9, v10); +} + +template +internal::ValueArray11 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11) { + return internal::ValueArray11(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11); +} + +template +internal::ValueArray12 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12) { + return internal::ValueArray12(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12); +} + +template +internal::ValueArray13 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13) { + return internal::ValueArray13(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13); +} + +template +internal::ValueArray14 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14) { + return internal::ValueArray14(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, + v14); +} + +template +internal::ValueArray15 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, + T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15) { + return internal::ValueArray15(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, + v13, v14, v15); +} + +template +internal::ValueArray16 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, + T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, + T16 v16) { + return internal::ValueArray16(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, + v12, v13, v14, v15, v16); +} + +template +internal::ValueArray17 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, + T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, + T16 v16, T17 v17) { + return internal::ValueArray17(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, + v11, v12, v13, v14, v15, v16, v17); +} + +template +internal::ValueArray18 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, + T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, + T16 v16, T17 v17, T18 v18) { + return internal::ValueArray18(v1, v2, v3, v4, v5, v6, v7, v8, v9, + v10, v11, v12, v13, v14, v15, v16, v17, v18); +} + +template +internal::ValueArray19 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, + T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, + T15 v15, T16 v16, T17 v17, T18 v18, T19 v19) { + return internal::ValueArray19(v1, v2, v3, v4, v5, v6, v7, v8, + v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19); +} + +template +internal::ValueArray20 Values(T1 v1, T2 v2, T3 v3, T4 v4, + T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, + T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20) { + return internal::ValueArray20(v1, v2, v3, v4, v5, v6, v7, + v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20); +} + +template +internal::ValueArray21 Values(T1 v1, T2 v2, T3 v3, T4 v4, + T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, + T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21) { + return internal::ValueArray21(v1, v2, v3, v4, v5, v6, + v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21); +} + +template +internal::ValueArray22 Values(T1 v1, T2 v2, T3 v3, + T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, + T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, + T21 v21, T22 v22) { + return internal::ValueArray22(v1, v2, v3, v4, + v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, + v20, v21, v22); +} + +template +internal::ValueArray23 Values(T1 v1, T2 v2, + T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, + T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, + T21 v21, T22 v22, T23 v23) { + return internal::ValueArray23(v1, v2, v3, + v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, + v20, v21, v22, v23); +} + +template +internal::ValueArray24 Values(T1 v1, T2 v2, + T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, + T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, + T21 v21, T22 v22, T23 v23, T24 v24) { + return internal::ValueArray24(v1, v2, + v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, + v19, v20, v21, v22, v23, v24); +} + +template +internal::ValueArray25 Values(T1 v1, + T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, + T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, + T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25) { + return internal::ValueArray25(v1, + v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, + v18, v19, v20, v21, v22, v23, v24, v25); +} + +template +internal::ValueArray26 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26) { + return internal::ValueArray26(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, + v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26); +} + +template +internal::ValueArray27 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27) { + return internal::ValueArray27(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, + v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27); +} + +template +internal::ValueArray28 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28) { + return internal::ValueArray28(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, + v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, + v28); +} + +template +internal::ValueArray29 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29) { + return internal::ValueArray29(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, + v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, + v27, v28, v29); +} + +template +internal::ValueArray30 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, + T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, + T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, + T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, T30 v30) { + return internal::ValueArray30(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, + v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, + v26, v27, v28, v29, v30); +} + +template +internal::ValueArray31 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, + T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, + T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, + T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31) { + return internal::ValueArray31(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, + v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, + v25, v26, v27, v28, v29, v30, v31); +} + +template +internal::ValueArray32 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, + T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, + T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, + T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, + T32 v32) { + return internal::ValueArray32(v1, v2, v3, v4, v5, v6, v7, v8, v9, + v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, + v24, v25, v26, v27, v28, v29, v30, v31, v32); +} + +template +internal::ValueArray33 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, + T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, + T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, + T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, + T32 v32, T33 v33) { + return internal::ValueArray33(v1, v2, v3, v4, v5, v6, v7, v8, + v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, + v24, v25, v26, v27, v28, v29, v30, v31, v32, v33); +} + +template +internal::ValueArray34 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, + T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, + T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, + T23 v23, T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, + T31 v31, T32 v32, T33 v33, T34 v34) { + return internal::ValueArray34(v1, v2, v3, v4, v5, v6, v7, + v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, + v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34); +} + +template +internal::ValueArray35 Values(T1 v1, T2 v2, T3 v3, T4 v4, + T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, + T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, + T22 v22, T23 v23, T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, + T30 v30, T31 v31, T32 v32, T33 v33, T34 v34, T35 v35) { + return internal::ValueArray35(v1, v2, v3, v4, v5, v6, + v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, + v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35); +} + +template +internal::ValueArray36 Values(T1 v1, T2 v2, T3 v3, T4 v4, + T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, + T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, + T22 v22, T23 v23, T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, + T30 v30, T31 v31, T32 v32, T33 v33, T34 v34, T35 v35, T36 v36) { + return internal::ValueArray36(v1, v2, v3, v4, + v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, + v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, + v34, v35, v36); +} + +template +internal::ValueArray37 Values(T1 v1, T2 v2, T3 v3, + T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, + T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, + T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, + T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, T34 v34, T35 v35, T36 v36, + T37 v37) { + return internal::ValueArray37(v1, v2, v3, + v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, + v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, + v34, v35, v36, v37); +} + +template +internal::ValueArray38 Values(T1 v1, T2 v2, + T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, + T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, + T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, + T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, T34 v34, T35 v35, T36 v36, + T37 v37, T38 v38) { + return internal::ValueArray38(v1, v2, + v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, + v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, + v33, v34, v35, v36, v37, v38); +} + +template +internal::ValueArray39 Values(T1 v1, T2 v2, + T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, + T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, + T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, + T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, T34 v34, T35 v35, T36 v36, + T37 v37, T38 v38, T39 v39) { + return internal::ValueArray39(v1, + v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, + v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, + v32, v33, v34, v35, v36, v37, v38, v39); +} + +template +internal::ValueArray40 Values(T1 v1, + T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, + T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, + T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, T26 v26, T27 v27, + T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, T34 v34, T35 v35, + T36 v36, T37 v37, T38 v38, T39 v39, T40 v40) { + return internal::ValueArray40(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, + v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, + v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40); +} + +template +internal::ValueArray41 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41) { + return internal::ValueArray41(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, + v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, + v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41); +} + +template +internal::ValueArray42 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42) { + return internal::ValueArray42(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, + v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, + v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, + v42); +} + +template +internal::ValueArray43 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42, T43 v43) { + return internal::ValueArray43(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, + v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, + v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v40, + v41, v42, v43); +} + +template +internal::ValueArray44 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, + T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, T17 v17, + T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, T25 v25, + T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, T33 v33, + T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, T41 v41, + T42 v42, T43 v43, T44 v44) { + return internal::ValueArray44(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, + v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, + v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, + v40, v41, v42, v43, v44); +} + +template +internal::ValueArray45 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, T8 v8, + T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, T16 v16, + T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, T24 v24, + T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, T32 v32, + T33 v33, T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, T40 v40, + T41 v41, T42 v42, T43 v43, T44 v44, T45 v45) { + return internal::ValueArray45(v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, + v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, v24, + v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, + v39, v40, v41, v42, v43, v44, v45); +} + +template +internal::ValueArray46 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, + T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, + T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, + T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, + T32 v32, T33 v33, T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, + T40 v40, T41 v41, T42 v42, T43 v43, T44 v44, T45 v45, T46 v46) { + return internal::ValueArray46(v1, v2, v3, v4, v5, v6, v7, v8, v9, + v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, + v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, + v38, v39, v40, v41, v42, v43, v44, v45, v46); +} + +template +internal::ValueArray47 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, T7 v7, + T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, + T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, + T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, + T32 v32, T33 v33, T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, + T40 v40, T41 v41, T42 v42, T43 v43, T44 v44, T45 v45, T46 v46, T47 v47) { + return internal::ValueArray47(v1, v2, v3, v4, v5, v6, v7, v8, + v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, v23, + v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, v37, + v38, v39, v40, v41, v42, v43, v44, v45, v46, v47); +} + +template +internal::ValueArray48 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6, + T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, T15 v15, + T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, T23 v23, + T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, T31 v31, + T32 v32, T33 v33, T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, T39 v39, + T40 v40, T41 v41, T42 v42, T43 v43, T44 v44, T45 v45, T46 v46, T47 v47, + T48 v48) { + return internal::ValueArray48(v1, v2, v3, v4, v5, v6, v7, + v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, v22, + v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, v36, + v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48); +} + +template +internal::ValueArray49 Values(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, + T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, T14 v14, + T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, T22 v22, + T23 v23, T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, T30 v30, + T31 v31, T32 v32, T33 v33, T34 v34, T35 v35, T36 v36, T37 v37, T38 v38, + T39 v39, T40 v40, T41 v41, T42 v42, T43 v43, T44 v44, T45 v45, T46 v46, + T47 v47, T48 v48, T49 v49) { + return internal::ValueArray49(v1, v2, v3, v4, v5, v6, + v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v20, v21, + v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, v34, v35, + v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49); +} + +template +internal::ValueArray50 Values(T1 v1, T2 v2, T3 v3, T4 v4, + T5 v5, T6 v6, T7 v7, T8 v8, T9 v9, T10 v10, T11 v11, T12 v12, T13 v13, + T14 v14, T15 v15, T16 v16, T17 v17, T18 v18, T19 v19, T20 v20, T21 v21, + T22 v22, T23 v23, T24 v24, T25 v25, T26 v26, T27 v27, T28 v28, T29 v29, + T30 v30, T31 v31, T32 v32, T33 v33, T34 v34, T35 v35, T36 v36, T37 v37, + T38 v38, T39 v39, T40 v40, T41 v41, T42 v42, T43 v43, T44 v44, T45 v45, + T46 v46, T47 v47, T48 v48, T49 v49, T50 v50) { + return internal::ValueArray50(v1, v2, v3, v4, + v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, + v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31, v32, v33, + v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47, + v48, v49, v50); +} + +// Bool() allows generating tests with parameters in a set of (false, true). +// +// Synopsis: +// Bool() +// - returns a generator producing sequences with elements {false, true}. +// +// It is useful when testing code that depends on Boolean flags. Combinations +// of multiple flags can be tested when several Bool()'s are combined using +// Combine() function. +// +// In the following example all tests in the test case FlagDependentTest +// will be instantiated twice with parameters false and true. +// +// class FlagDependentTest : public testing::TestWithParam { +// virtual void SetUp() { +// external_flag = GetParam(); +// } +// } +// INSTANTIATE_TEST_CASE_P(BoolSequence, FlagDependentTest, Bool()); +// +inline internal::ParamGenerator Bool() { + return Values(false, true); +} + +# if GTEST_HAS_COMBINE +// Combine() allows the user to combine two or more sequences to produce +// values of a Cartesian product of those sequences' elements. +// +// Synopsis: +// Combine(gen1, gen2, ..., genN) +// - returns a generator producing sequences with elements coming from +// the Cartesian product of elements from the sequences generated by +// gen1, gen2, ..., genN. The sequence elements will have a type of +// tuple where T1, T2, ..., TN are the types +// of elements from sequences produces by gen1, gen2, ..., genN. +// +// Combine can have up to 10 arguments. This number is currently limited +// by the maximum number of elements in the tuple implementation used by Google +// Test. +// +// Example: +// +// This will instantiate tests in test case AnimalTest each one with +// the parameter values tuple("cat", BLACK), tuple("cat", WHITE), +// tuple("dog", BLACK), and tuple("dog", WHITE): +// +// enum Color { BLACK, GRAY, WHITE }; +// class AnimalTest +// : public testing::TestWithParam > {...}; +// +// TEST_P(AnimalTest, AnimalLooksNice) {...} +// +// INSTANTIATE_TEST_CASE_P(AnimalVariations, AnimalTest, +// Combine(Values("cat", "dog"), +// Values(BLACK, WHITE))); +// +// This will instantiate tests in FlagDependentTest with all variations of two +// Boolean flags: +// +// class FlagDependentTest +// : public testing::TestWithParam > { +// virtual void SetUp() { +// // Assigns external_flag_1 and external_flag_2 values from the tuple. +// tie(external_flag_1, external_flag_2) = GetParam(); +// } +// }; +// +// TEST_P(FlagDependentTest, TestFeature1) { +// // Test your code using external_flag_1 and external_flag_2 here. +// } +// INSTANTIATE_TEST_CASE_P(TwoBoolSequence, FlagDependentTest, +// Combine(Bool(), Bool())); +// +template +internal::CartesianProductHolder2 Combine( + const Generator1& g1, const Generator2& g2) { + return internal::CartesianProductHolder2( + g1, g2); +} + +template +internal::CartesianProductHolder3 Combine( + const Generator1& g1, const Generator2& g2, const Generator3& g3) { + return internal::CartesianProductHolder3( + g1, g2, g3); +} + +template +internal::CartesianProductHolder4 Combine( + const Generator1& g1, const Generator2& g2, const Generator3& g3, + const Generator4& g4) { + return internal::CartesianProductHolder4( + g1, g2, g3, g4); +} + +template +internal::CartesianProductHolder5 Combine( + const Generator1& g1, const Generator2& g2, const Generator3& g3, + const Generator4& g4, const Generator5& g5) { + return internal::CartesianProductHolder5( + g1, g2, g3, g4, g5); +} + +template +internal::CartesianProductHolder6 Combine( + const Generator1& g1, const Generator2& g2, const Generator3& g3, + const Generator4& g4, const Generator5& g5, const Generator6& g6) { + return internal::CartesianProductHolder6( + g1, g2, g3, g4, g5, g6); +} + +template +internal::CartesianProductHolder7 Combine( + const Generator1& g1, const Generator2& g2, const Generator3& g3, + const Generator4& g4, const Generator5& g5, const Generator6& g6, + const Generator7& g7) { + return internal::CartesianProductHolder7( + g1, g2, g3, g4, g5, g6, g7); +} + +template +internal::CartesianProductHolder8 Combine( + const Generator1& g1, const Generator2& g2, const Generator3& g3, + const Generator4& g4, const Generator5& g5, const Generator6& g6, + const Generator7& g7, const Generator8& g8) { + return internal::CartesianProductHolder8( + g1, g2, g3, g4, g5, g6, g7, g8); +} + +template +internal::CartesianProductHolder9 Combine( + const Generator1& g1, const Generator2& g2, const Generator3& g3, + const Generator4& g4, const Generator5& g5, const Generator6& g6, + const Generator7& g7, const Generator8& g8, const Generator9& g9) { + return internal::CartesianProductHolder9( + g1, g2, g3, g4, g5, g6, g7, g8, g9); +} + +template +internal::CartesianProductHolder10 Combine( + const Generator1& g1, const Generator2& g2, const Generator3& g3, + const Generator4& g4, const Generator5& g5, const Generator6& g6, + const Generator7& g7, const Generator8& g8, const Generator9& g9, + const Generator10& g10) { + return internal::CartesianProductHolder10( + g1, g2, g3, g4, g5, g6, g7, g8, g9, g10); +} +# endif // GTEST_HAS_COMBINE + + + +# define TEST_P(test_case_name, test_name) \ + class GTEST_TEST_CLASS_NAME_(test_case_name, test_name) \ + : public test_case_name { \ + public: \ + GTEST_TEST_CLASS_NAME_(test_case_name, test_name)() {} \ + virtual void TestBody(); \ + private: \ + static int AddToRegistry() { \ + ::testing::UnitTest::GetInstance()->parameterized_test_registry(). \ + GetTestCasePatternHolder(\ + #test_case_name, __FILE__, __LINE__)->AddTestPattern(\ + #test_case_name, \ + #test_name, \ + new ::testing::internal::TestMetaFactory< \ + GTEST_TEST_CLASS_NAME_(test_case_name, test_name)>()); \ + return 0; \ + } \ + static int gtest_registering_dummy_ GTEST_ATTRIBUTE_UNUSED_; \ + GTEST_DISALLOW_COPY_AND_ASSIGN_(\ + GTEST_TEST_CLASS_NAME_(test_case_name, test_name)); \ + }; \ + int GTEST_TEST_CLASS_NAME_(test_case_name, \ + test_name)::gtest_registering_dummy_ = \ + GTEST_TEST_CLASS_NAME_(test_case_name, test_name)::AddToRegistry(); \ + void GTEST_TEST_CLASS_NAME_(test_case_name, test_name)::TestBody() + +# define INSTANTIATE_TEST_CASE_P(prefix, test_case_name, generator) \ + ::testing::internal::ParamGenerator \ + gtest_##prefix##test_case_name##_EvalGenerator_() { return generator; } \ + int gtest_##prefix##test_case_name##_dummy_ = \ + ::testing::UnitTest::GetInstance()->parameterized_test_registry(). \ + GetTestCasePatternHolder(\ + #test_case_name, __FILE__, __LINE__)->AddTestCaseInstantiation(\ + #prefix, \ + >est_##prefix##test_case_name##_EvalGenerator_, \ + __FILE__, __LINE__) + +} // namespace testing + +#endif // GTEST_HAS_PARAM_TEST + +#endif // GTEST_INCLUDE_GTEST_GTEST_PARAM_TEST_H_ +// Copyright 2006, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) +// +// Google C++ Testing Framework definitions useful in production code. + +#ifndef GTEST_INCLUDE_GTEST_GTEST_PROD_H_ +#define GTEST_INCLUDE_GTEST_GTEST_PROD_H_ + +// When you need to test the private or protected members of a class, +// use the FRIEND_TEST macro to declare your tests as friends of the +// class. For example: +// +// class MyClass { +// private: +// void MyMethod(); +// FRIEND_TEST(MyClassTest, MyMethod); +// }; +// +// class MyClassTest : public testing::Test { +// // ... +// }; +// +// TEST_F(MyClassTest, MyMethod) { +// // Can call MyClass::MyMethod() here. +// } + +#define FRIEND_TEST(test_case_name, test_name)\ +friend class test_case_name##_##test_name##_Test + +#endif // GTEST_INCLUDE_GTEST_GTEST_PROD_H_ +// Copyright 2008, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: mheule@google.com (Markus Heule) +// + +#ifndef GTEST_INCLUDE_GTEST_GTEST_TEST_PART_H_ +#define GTEST_INCLUDE_GTEST_GTEST_TEST_PART_H_ + +#include +#include + +namespace testing { + +// A copyable object representing the result of a test part (i.e. an +// assertion or an explicit FAIL(), ADD_FAILURE(), or SUCCESS()). +// +// Don't inherit from TestPartResult as its destructor is not virtual. +class GTEST_API_ TestPartResult { + public: + // The possible outcomes of a test part (i.e. an assertion or an + // explicit SUCCEED(), FAIL(), or ADD_FAILURE()). + enum Type { + kSuccess, // Succeeded. + kNonFatalFailure, // Failed but the test can continue. + kFatalFailure // Failed and the test should be terminated. + }; + + // C'tor. TestPartResult does NOT have a default constructor. + // Always use this constructor (with parameters) to create a + // TestPartResult object. + TestPartResult(Type a_type, + const char* a_file_name, + int a_line_number, + const char* a_message) + : type_(a_type), + file_name_(a_file_name == NULL ? "" : a_file_name), + line_number_(a_line_number), + summary_(ExtractSummary(a_message)), + message_(a_message) { + } + + // Gets the outcome of the test part. + Type type() const { return type_; } + + // Gets the name of the source file where the test part took place, or + // NULL if it's unknown. + const char* file_name() const { + return file_name_.empty() ? NULL : file_name_.c_str(); + } + + // Gets the line in the source file where the test part took place, + // or -1 if it's unknown. + int line_number() const { return line_number_; } + + // Gets the summary of the failure message. + const char* summary() const { return summary_.c_str(); } + + // Gets the message associated with the test part. + const char* message() const { return message_.c_str(); } + + // Returns true iff the test part passed. + bool passed() const { return type_ == kSuccess; } + + // Returns true iff the test part failed. + bool failed() const { return type_ != kSuccess; } + + // Returns true iff the test part non-fatally failed. + bool nonfatally_failed() const { return type_ == kNonFatalFailure; } + + // Returns true iff the test part fatally failed. + bool fatally_failed() const { return type_ == kFatalFailure; } + + private: + Type type_; + + // Gets the summary of the failure message by omitting the stack + // trace in it. + static std::string ExtractSummary(const char* message); + + // The name of the source file where the test part took place, or + // "" if the source file is unknown. + std::string file_name_; + // The line in the source file where the test part took place, or -1 + // if the line number is unknown. + int line_number_; + std::string summary_; // The test failure summary. + std::string message_; // The test failure message. +}; + +// Prints a TestPartResult object. +std::ostream& operator<<(std::ostream& os, const TestPartResult& result); + +// An array of TestPartResult objects. +// +// Don't inherit from TestPartResultArray as its destructor is not +// virtual. +class GTEST_API_ TestPartResultArray { + public: + TestPartResultArray() {} + + // Appends the given TestPartResult to the array. + void Append(const TestPartResult& result); + + // Returns the TestPartResult at the given index (0-based). + const TestPartResult& GetTestPartResult(int index) const; + + // Returns the number of TestPartResult objects in the array. + int size() const; + + private: + std::vector array_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(TestPartResultArray); +}; + +// This interface knows how to report a test part result. +class TestPartResultReporterInterface { + public: + virtual ~TestPartResultReporterInterface() {} + + virtual void ReportTestPartResult(const TestPartResult& result) = 0; +}; + +namespace internal { + +// This helper class is used by {ASSERT|EXPECT}_NO_FATAL_FAILURE to check if a +// statement generates new fatal failures. To do so it registers itself as the +// current test part result reporter. Besides checking if fatal failures were +// reported, it only delegates the reporting to the former result reporter. +// The original result reporter is restored in the destructor. +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +class GTEST_API_ HasNewFatalFailureHelper + : public TestPartResultReporterInterface { + public: + HasNewFatalFailureHelper(); + virtual ~HasNewFatalFailureHelper(); + virtual void ReportTestPartResult(const TestPartResult& result); + bool has_new_fatal_failure() const { return has_new_fatal_failure_; } + private: + bool has_new_fatal_failure_; + TestPartResultReporterInterface* original_reporter_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(HasNewFatalFailureHelper); +}; + +} // namespace internal + +} // namespace testing + +#endif // GTEST_INCLUDE_GTEST_GTEST_TEST_PART_H_ +// Copyright 2008 Google Inc. +// All Rights Reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) + +#ifndef GTEST_INCLUDE_GTEST_GTEST_TYPED_TEST_H_ +#define GTEST_INCLUDE_GTEST_GTEST_TYPED_TEST_H_ + +// This header implements typed tests and type-parameterized tests. + +// Typed (aka type-driven) tests repeat the same test for types in a +// list. You must know which types you want to test with when writing +// typed tests. Here's how you do it: + +#if 0 + +// First, define a fixture class template. It should be parameterized +// by a type. Remember to derive it from testing::Test. +template +class FooTest : public testing::Test { + public: + ... + typedef std::list List; + static T shared_; + T value_; +}; + +// Next, associate a list of types with the test case, which will be +// repeated for each type in the list. The typedef is necessary for +// the macro to parse correctly. +typedef testing::Types MyTypes; +TYPED_TEST_CASE(FooTest, MyTypes); + +// If the type list contains only one type, you can write that type +// directly without Types<...>: +// TYPED_TEST_CASE(FooTest, int); + +// Then, use TYPED_TEST() instead of TEST_F() to define as many typed +// tests for this test case as you want. +TYPED_TEST(FooTest, DoesBlah) { + // Inside a test, refer to TypeParam to get the type parameter. + // Since we are inside a derived class template, C++ requires use to + // visit the members of FooTest via 'this'. + TypeParam n = this->value_; + + // To visit static members of the fixture, add the TestFixture:: + // prefix. + n += TestFixture::shared_; + + // To refer to typedefs in the fixture, add the "typename + // TestFixture::" prefix. + typename TestFixture::List values; + values.push_back(n); + ... +} + +TYPED_TEST(FooTest, HasPropertyA) { ... } + +#endif // 0 + +// Type-parameterized tests are abstract test patterns parameterized +// by a type. Compared with typed tests, type-parameterized tests +// allow you to define the test pattern without knowing what the type +// parameters are. The defined pattern can be instantiated with +// different types any number of times, in any number of translation +// units. +// +// If you are designing an interface or concept, you can define a +// suite of type-parameterized tests to verify properties that any +// valid implementation of the interface/concept should have. Then, +// each implementation can easily instantiate the test suite to verify +// that it conforms to the requirements, without having to write +// similar tests repeatedly. Here's an example: + +#if 0 + +// First, define a fixture class template. It should be parameterized +// by a type. Remember to derive it from testing::Test. +template +class FooTest : public testing::Test { + ... +}; + +// Next, declare that you will define a type-parameterized test case +// (the _P suffix is for "parameterized" or "pattern", whichever you +// prefer): +TYPED_TEST_CASE_P(FooTest); + +// Then, use TYPED_TEST_P() to define as many type-parameterized tests +// for this type-parameterized test case as you want. +TYPED_TEST_P(FooTest, DoesBlah) { + // Inside a test, refer to TypeParam to get the type parameter. + TypeParam n = 0; + ... +} + +TYPED_TEST_P(FooTest, HasPropertyA) { ... } + +// Now the tricky part: you need to register all test patterns before +// you can instantiate them. The first argument of the macro is the +// test case name; the rest are the names of the tests in this test +// case. +REGISTER_TYPED_TEST_CASE_P(FooTest, + DoesBlah, HasPropertyA); + +// Finally, you are free to instantiate the pattern with the types you +// want. If you put the above code in a header file, you can #include +// it in multiple C++ source files and instantiate it multiple times. +// +// To distinguish different instances of the pattern, the first +// argument to the INSTANTIATE_* macro is a prefix that will be added +// to the actual test case name. Remember to pick unique prefixes for +// different instances. +typedef testing::Types MyTypes; +INSTANTIATE_TYPED_TEST_CASE_P(My, FooTest, MyTypes); + +// If the type list contains only one type, you can write that type +// directly without Types<...>: +// INSTANTIATE_TYPED_TEST_CASE_P(My, FooTest, int); + +#endif // 0 + + +// Implements typed tests. + +#if GTEST_HAS_TYPED_TEST + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// Expands to the name of the typedef for the type parameters of the +// given test case. +# define GTEST_TYPE_PARAMS_(TestCaseName) gtest_type_params_##TestCaseName##_ + +// The 'Types' template argument below must have spaces around it +// since some compilers may choke on '>>' when passing a template +// instance (e.g. Types) +# define TYPED_TEST_CASE(CaseName, Types) \ + typedef ::testing::internal::TypeList< Types >::type \ + GTEST_TYPE_PARAMS_(CaseName) + +# define TYPED_TEST(CaseName, TestName) \ + template \ + class GTEST_TEST_CLASS_NAME_(CaseName, TestName) \ + : public CaseName { \ + private: \ + typedef CaseName TestFixture; \ + typedef gtest_TypeParam_ TypeParam; \ + virtual void TestBody(); \ + }; \ + bool gtest_##CaseName##_##TestName##_registered_ GTEST_ATTRIBUTE_UNUSED_ = \ + ::testing::internal::TypeParameterizedTest< \ + CaseName, \ + ::testing::internal::TemplateSel< \ + GTEST_TEST_CLASS_NAME_(CaseName, TestName)>, \ + GTEST_TYPE_PARAMS_(CaseName)>::Register(\ + "", #CaseName, #TestName, 0); \ + template \ + void GTEST_TEST_CLASS_NAME_(CaseName, TestName)::TestBody() + +#endif // GTEST_HAS_TYPED_TEST + +// Implements type-parameterized tests. + +#if GTEST_HAS_TYPED_TEST_P + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// Expands to the namespace name that the type-parameterized tests for +// the given type-parameterized test case are defined in. The exact +// name of the namespace is subject to change without notice. +# define GTEST_CASE_NAMESPACE_(TestCaseName) \ + gtest_case_##TestCaseName##_ + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// +// Expands to the name of the variable used to remember the names of +// the defined tests in the given test case. +# define GTEST_TYPED_TEST_CASE_P_STATE_(TestCaseName) \ + gtest_typed_test_case_p_state_##TestCaseName##_ + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE DIRECTLY. +// +// Expands to the name of the variable used to remember the names of +// the registered tests in the given test case. +# define GTEST_REGISTERED_TEST_NAMES_(TestCaseName) \ + gtest_registered_test_names_##TestCaseName##_ + +// The variables defined in the type-parameterized test macros are +// static as typically these macros are used in a .h file that can be +// #included in multiple translation units linked together. +# define TYPED_TEST_CASE_P(CaseName) \ + static ::testing::internal::TypedTestCasePState \ + GTEST_TYPED_TEST_CASE_P_STATE_(CaseName) + +# define TYPED_TEST_P(CaseName, TestName) \ + namespace GTEST_CASE_NAMESPACE_(CaseName) { \ + template \ + class TestName : public CaseName { \ + private: \ + typedef CaseName TestFixture; \ + typedef gtest_TypeParam_ TypeParam; \ + virtual void TestBody(); \ + }; \ + static bool gtest_##TestName##_defined_ GTEST_ATTRIBUTE_UNUSED_ = \ + GTEST_TYPED_TEST_CASE_P_STATE_(CaseName).AddTestName(\ + __FILE__, __LINE__, #CaseName, #TestName); \ + } \ + template \ + void GTEST_CASE_NAMESPACE_(CaseName)::TestName::TestBody() + +# define REGISTER_TYPED_TEST_CASE_P(CaseName, ...) \ + namespace GTEST_CASE_NAMESPACE_(CaseName) { \ + typedef ::testing::internal::Templates<__VA_ARGS__>::type gtest_AllTests_; \ + } \ + static const char* const GTEST_REGISTERED_TEST_NAMES_(CaseName) = \ + GTEST_TYPED_TEST_CASE_P_STATE_(CaseName).VerifyRegisteredTestNames(\ + __FILE__, __LINE__, #__VA_ARGS__) + +// The 'Types' template argument below must have spaces around it +// since some compilers may choke on '>>' when passing a template +// instance (e.g. Types) +# define INSTANTIATE_TYPED_TEST_CASE_P(Prefix, CaseName, Types) \ + bool gtest_##Prefix##_##CaseName GTEST_ATTRIBUTE_UNUSED_ = \ + ::testing::internal::TypeParameterizedTestCase::type>::Register(\ + #Prefix, #CaseName, GTEST_REGISTERED_TEST_NAMES_(CaseName)) + +#endif // GTEST_HAS_TYPED_TEST_P + +#endif // GTEST_INCLUDE_GTEST_GTEST_TYPED_TEST_H_ + +// Depending on the platform, different string classes are available. +// On Linux, in addition to ::std::string, Google also makes use of +// class ::string, which has the same interface as ::std::string, but +// has a different implementation. +// +// You can define GTEST_HAS_GLOBAL_STRING to 1 to indicate that +// ::string is available AND is a distinct type to ::std::string, or +// define it to 0 to indicate otherwise. +// +// If ::std::string and ::string are the same class on your platform +// due to aliasing, you should define GTEST_HAS_GLOBAL_STRING to 0. +// +// If you do not define GTEST_HAS_GLOBAL_STRING, it is defined +// heuristically. + +namespace testing { + +// Declares the flags. + +// This flag temporary enables the disabled tests. +GTEST_DECLARE_bool_(also_run_disabled_tests); + +// This flag brings the debugger on an assertion failure. +GTEST_DECLARE_bool_(break_on_failure); + +// This flag controls whether Google Test catches all test-thrown exceptions +// and logs them as failures. +GTEST_DECLARE_bool_(catch_exceptions); + +// This flag enables using colors in terminal output. Available values are +// "yes" to enable colors, "no" (disable colors), or "auto" (the default) +// to let Google Test decide. +GTEST_DECLARE_string_(color); + +// This flag sets up the filter to select by name using a glob pattern +// the tests to run. If the filter is not given all tests are executed. +GTEST_DECLARE_string_(filter); + +// This flag causes the Google Test to list tests. None of the tests listed +// are actually run if the flag is provided. +GTEST_DECLARE_bool_(list_tests); + +// This flag controls whether Google Test emits a detailed XML report to a file +// in addition to its normal textual output. +GTEST_DECLARE_string_(output); + +// This flags control whether Google Test prints the elapsed time for each +// test. +GTEST_DECLARE_bool_(print_time); + +// This flag specifies the random number seed. +GTEST_DECLARE_int32_(random_seed); + +// This flag sets how many times the tests are repeated. The default value +// is 1. If the value is -1 the tests are repeating forever. +GTEST_DECLARE_int32_(repeat); + +// This flag controls whether Google Test includes Google Test internal +// stack frames in failure stack traces. +GTEST_DECLARE_bool_(show_internal_stack_frames); + +// When this flag is specified, tests' order is randomized on every iteration. +GTEST_DECLARE_bool_(shuffle); + +// This flag specifies the maximum number of stack frames to be +// printed in a failure message. +GTEST_DECLARE_int32_(stack_trace_depth); + +// When this flag is specified, a failed assertion will throw an +// exception if exceptions are enabled, or exit the program with a +// non-zero code otherwise. +GTEST_DECLARE_bool_(throw_on_failure); + +// When this flag is set with a "host:port" string, on supported +// platforms test results are streamed to the specified port on +// the specified host machine. +GTEST_DECLARE_string_(stream_result_to); + +// The upper limit for valid stack trace depths. +const int kMaxStackTraceDepth = 100; + +namespace internal { + +class AssertHelper; +class DefaultGlobalTestPartResultReporter; +class ExecDeathTest; +class NoExecDeathTest; +class FinalSuccessChecker; +class GTestFlagSaver; +class StreamingListenerTest; +class TestResultAccessor; +class TestEventListenersAccessor; +class TestEventRepeater; +class UnitTestRecordPropertyTestHelper; +class WindowsDeathTest; +class UnitTestImpl* GetUnitTestImpl(); +void ReportFailureInUnknownLocation(TestPartResult::Type result_type, + const std::string& message); + +} // namespace internal + +// The friend relationship of some of these classes is cyclic. +// If we don't forward declare them the compiler might confuse the classes +// in friendship clauses with same named classes on the scope. +class Test; +class TestCase; +class TestInfo; +class UnitTest; + +// A class for indicating whether an assertion was successful. When +// the assertion wasn't successful, the AssertionResult object +// remembers a non-empty message that describes how it failed. +// +// To create an instance of this class, use one of the factory functions +// (AssertionSuccess() and AssertionFailure()). +// +// This class is useful for two purposes: +// 1. Defining predicate functions to be used with Boolean test assertions +// EXPECT_TRUE/EXPECT_FALSE and their ASSERT_ counterparts +// 2. Defining predicate-format functions to be +// used with predicate assertions (ASSERT_PRED_FORMAT*, etc). +// +// For example, if you define IsEven predicate: +// +// testing::AssertionResult IsEven(int n) { +// if ((n % 2) == 0) +// return testing::AssertionSuccess(); +// else +// return testing::AssertionFailure() << n << " is odd"; +// } +// +// Then the failed expectation EXPECT_TRUE(IsEven(Fib(5))) +// will print the message +// +// Value of: IsEven(Fib(5)) +// Actual: false (5 is odd) +// Expected: true +// +// instead of a more opaque +// +// Value of: IsEven(Fib(5)) +// Actual: false +// Expected: true +// +// in case IsEven is a simple Boolean predicate. +// +// If you expect your predicate to be reused and want to support informative +// messages in EXPECT_FALSE and ASSERT_FALSE (negative assertions show up +// about half as often as positive ones in our tests), supply messages for +// both success and failure cases: +// +// testing::AssertionResult IsEven(int n) { +// if ((n % 2) == 0) +// return testing::AssertionSuccess() << n << " is even"; +// else +// return testing::AssertionFailure() << n << " is odd"; +// } +// +// Then a statement EXPECT_FALSE(IsEven(Fib(6))) will print +// +// Value of: IsEven(Fib(6)) +// Actual: true (8 is even) +// Expected: false +// +// NB: Predicates that support negative Boolean assertions have reduced +// performance in positive ones so be careful not to use them in tests +// that have lots (tens of thousands) of positive Boolean assertions. +// +// To use this class with EXPECT_PRED_FORMAT assertions such as: +// +// // Verifies that Foo() returns an even number. +// EXPECT_PRED_FORMAT1(IsEven, Foo()); +// +// you need to define: +// +// testing::AssertionResult IsEven(const char* expr, int n) { +// if ((n % 2) == 0) +// return testing::AssertionSuccess(); +// else +// return testing::AssertionFailure() +// << "Expected: " << expr << " is even\n Actual: it's " << n; +// } +// +// If Foo() returns 5, you will see the following message: +// +// Expected: Foo() is even +// Actual: it's 5 +// +class GTEST_API_ AssertionResult { + public: + // Copy constructor. + // Used in EXPECT_TRUE/FALSE(assertion_result). + AssertionResult(const AssertionResult& other); + + GTEST_DISABLE_MSC_WARNINGS_PUSH_(4800 /* forcing value to bool */) + + // Used in the EXPECT_TRUE/FALSE(bool_expression). + // + // T must be contextually convertible to bool. + // + // The second parameter prevents this overload from being considered if + // the argument is implicitly convertible to AssertionResult. In that case + // we want AssertionResult's copy constructor to be used. + template + explicit AssertionResult( + const T& success, + typename internal::EnableIf< + !internal::ImplicitlyConvertible::value>::type* + /*enabler*/ = NULL) + : success_(success) {} + + GTEST_DISABLE_MSC_WARNINGS_POP_() + + // Assignment operator. + AssertionResult& operator=(AssertionResult other) { + swap(other); + return *this; + } + + // Returns true iff the assertion succeeded. + operator bool() const { return success_; } // NOLINT + + // Returns the assertion's negation. Used with EXPECT/ASSERT_FALSE. + AssertionResult operator!() const; + + // Returns the text streamed into this AssertionResult. Test assertions + // use it when they fail (i.e., the predicate's outcome doesn't match the + // assertion's expectation). When nothing has been streamed into the + // object, returns an empty string. + const char* message() const { + return message_.get() != NULL ? message_->c_str() : ""; + } + // TODO(vladl@google.com): Remove this after making sure no clients use it. + // Deprecated; please use message() instead. + const char* failure_message() const { return message(); } + + // Streams a custom failure message into this object. + template AssertionResult& operator<<(const T& value) { + AppendMessage(Message() << value); + return *this; + } + + // Allows streaming basic output manipulators such as endl or flush into + // this object. + AssertionResult& operator<<( + ::std::ostream& (*basic_manipulator)(::std::ostream& stream)) { + AppendMessage(Message() << basic_manipulator); + return *this; + } + + private: + // Appends the contents of message to message_. + void AppendMessage(const Message& a_message) { + if (message_.get() == NULL) + message_.reset(new ::std::string); + message_->append(a_message.GetString().c_str()); + } + + // Swap the contents of this AssertionResult with other. + void swap(AssertionResult& other); + + // Stores result of the assertion predicate. + bool success_; + // Stores the message describing the condition in case the expectation + // construct is not satisfied with the predicate's outcome. + // Referenced via a pointer to avoid taking too much stack frame space + // with test assertions. + internal::scoped_ptr< ::std::string> message_; +}; + +// Makes a successful assertion result. +GTEST_API_ AssertionResult AssertionSuccess(); + +// Makes a failed assertion result. +GTEST_API_ AssertionResult AssertionFailure(); + +// Makes a failed assertion result with the given failure message. +// Deprecated; use AssertionFailure() << msg. +GTEST_API_ AssertionResult AssertionFailure(const Message& msg); + +// The abstract class that all tests inherit from. +// +// In Google Test, a unit test program contains one or many TestCases, and +// each TestCase contains one or many Tests. +// +// When you define a test using the TEST macro, you don't need to +// explicitly derive from Test - the TEST macro automatically does +// this for you. +// +// The only time you derive from Test is when defining a test fixture +// to be used a TEST_F. For example: +// +// class FooTest : public testing::Test { +// protected: +// void SetUp() override { ... } +// void TearDown() override { ... } +// ... +// }; +// +// TEST_F(FooTest, Bar) { ... } +// TEST_F(FooTest, Baz) { ... } +// +// Test is not copyable. +class GTEST_API_ Test { + public: + friend class TestInfo; + + // Defines types for pointers to functions that set up and tear down + // a test case. + typedef internal::SetUpTestCaseFunc SetUpTestCaseFunc; + typedef internal::TearDownTestCaseFunc TearDownTestCaseFunc; + + // The d'tor is virtual as we intend to inherit from Test. + virtual ~Test(); + + // Sets up the stuff shared by all tests in this test case. + // + // Google Test will call Foo::SetUpTestCase() before running the first + // test in test case Foo. Hence a sub-class can define its own + // SetUpTestCase() method to shadow the one defined in the super + // class. + static void SetUpTestCase() {} + + // Tears down the stuff shared by all tests in this test case. + // + // Google Test will call Foo::TearDownTestCase() after running the last + // test in test case Foo. Hence a sub-class can define its own + // TearDownTestCase() method to shadow the one defined in the super + // class. + static void TearDownTestCase() {} + + // Returns true iff the current test has a fatal failure. + static bool HasFatalFailure(); + + // Returns true iff the current test has a non-fatal failure. + static bool HasNonfatalFailure(); + + // Returns true iff the current test has a (either fatal or + // non-fatal) failure. + static bool HasFailure() { return HasFatalFailure() || HasNonfatalFailure(); } + + // Logs a property for the current test, test case, or for the entire + // invocation of the test program when used outside of the context of a + // test case. Only the last value for a given key is remembered. These + // are public static so they can be called from utility functions that are + // not members of the test fixture. Calls to RecordProperty made during + // lifespan of the test (from the moment its constructor starts to the + // moment its destructor finishes) will be output in XML as attributes of + // the element. Properties recorded from fixture's + // SetUpTestCase or TearDownTestCase are logged as attributes of the + // corresponding element. Calls to RecordProperty made in the + // global context (before or after invocation of RUN_ALL_TESTS and from + // SetUp/TearDown method of Environment objects registered with Google + // Test) will be output as attributes of the element. + static void RecordProperty(const std::string& key, const std::string& value); + static void RecordProperty(const std::string& key, int value); + + protected: + // Creates a Test object. + Test(); + + // Sets up the test fixture. + virtual void SetUp(); + + // Tears down the test fixture. + virtual void TearDown(); + + private: + // Returns true iff the current test has the same fixture class as + // the first test in the current test case. + static bool HasSameFixtureClass(); + + // Runs the test after the test fixture has been set up. + // + // A sub-class must implement this to define the test logic. + // + // DO NOT OVERRIDE THIS FUNCTION DIRECTLY IN A USER PROGRAM. + // Instead, use the TEST or TEST_F macro. + virtual void TestBody() = 0; + + // Sets up, executes, and tears down the test. + void Run(); + + // Deletes self. We deliberately pick an unusual name for this + // internal method to avoid clashing with names used in user TESTs. + void DeleteSelf_() { delete this; } + + // Uses a GTestFlagSaver to save and restore all Google Test flags. + const internal::GTestFlagSaver* const gtest_flag_saver_; + + // Often a user misspells SetUp() as Setup() and spends a long time + // wondering why it is never called by Google Test. The declaration of + // the following method is solely for catching such an error at + // compile time: + // + // - The return type is deliberately chosen to be not void, so it + // will be a conflict if void Setup() is declared in the user's + // test fixture. + // + // - This method is private, so it will be another compiler error + // if the method is called from the user's test fixture. + // + // DO NOT OVERRIDE THIS FUNCTION. + // + // If you see an error about overriding the following function or + // about it being private, you have mis-spelled SetUp() as Setup(). + struct Setup_should_be_spelled_SetUp {}; + virtual Setup_should_be_spelled_SetUp* Setup() { return NULL; } + + // We disallow copying Tests. + GTEST_DISALLOW_COPY_AND_ASSIGN_(Test); +}; + +typedef internal::TimeInMillis TimeInMillis; + +// A copyable object representing a user specified test property which can be +// output as a key/value string pair. +// +// Don't inherit from TestProperty as its destructor is not virtual. +class TestProperty { + public: + // C'tor. TestProperty does NOT have a default constructor. + // Always use this constructor (with parameters) to create a + // TestProperty object. + TestProperty(const std::string& a_key, const std::string& a_value) : + key_(a_key), value_(a_value) { + } + + // Gets the user supplied key. + const char* key() const { + return key_.c_str(); + } + + // Gets the user supplied value. + const char* value() const { + return value_.c_str(); + } + + // Sets a new value, overriding the one supplied in the constructor. + void SetValue(const std::string& new_value) { + value_ = new_value; + } + + private: + // The key supplied by the user. + std::string key_; + // The value supplied by the user. + std::string value_; +}; + +// The result of a single Test. This includes a list of +// TestPartResults, a list of TestProperties, a count of how many +// death tests there are in the Test, and how much time it took to run +// the Test. +// +// TestResult is not copyable. +class GTEST_API_ TestResult { + public: + // Creates an empty TestResult. + TestResult(); + + // D'tor. Do not inherit from TestResult. + ~TestResult(); + + // Gets the number of all test parts. This is the sum of the number + // of successful test parts and the number of failed test parts. + int total_part_count() const; + + // Returns the number of the test properties. + int test_property_count() const; + + // Returns true iff the test passed (i.e. no test part failed). + bool Passed() const { return !Failed(); } + + // Returns true iff the test failed. + bool Failed() const; + + // Returns true iff the test fatally failed. + bool HasFatalFailure() const; + + // Returns true iff the test has a non-fatal failure. + bool HasNonfatalFailure() const; + + // Returns the elapsed time, in milliseconds. + TimeInMillis elapsed_time() const { return elapsed_time_; } + + // Returns the i-th test part result among all the results. i can range + // from 0 to test_property_count() - 1. If i is not in that range, aborts + // the program. + const TestPartResult& GetTestPartResult(int i) const; + + // Returns the i-th test property. i can range from 0 to + // test_property_count() - 1. If i is not in that range, aborts the + // program. + const TestProperty& GetTestProperty(int i) const; + + private: + friend class TestInfo; + friend class TestCase; + friend class UnitTest; + friend class internal::DefaultGlobalTestPartResultReporter; + friend class internal::ExecDeathTest; + friend class internal::TestResultAccessor; + friend class internal::UnitTestImpl; + friend class internal::WindowsDeathTest; + + // Gets the vector of TestPartResults. + const std::vector& test_part_results() const { + return test_part_results_; + } + + // Gets the vector of TestProperties. + const std::vector& test_properties() const { + return test_properties_; + } + + // Sets the elapsed time. + void set_elapsed_time(TimeInMillis elapsed) { elapsed_time_ = elapsed; } + + // Adds a test property to the list. The property is validated and may add + // a non-fatal failure if invalid (e.g., if it conflicts with reserved + // key names). If a property is already recorded for the same key, the + // value will be updated, rather than storing multiple values for the same + // key. xml_element specifies the element for which the property is being + // recorded and is used for validation. + void RecordProperty(const std::string& xml_element, + const TestProperty& test_property); + + // Adds a failure if the key is a reserved attribute of Google Test + // testcase tags. Returns true if the property is valid. + // TODO(russr): Validate attribute names are legal and human readable. + static bool ValidateTestProperty(const std::string& xml_element, + const TestProperty& test_property); + + // Adds a test part result to the list. + void AddTestPartResult(const TestPartResult& test_part_result); + + // Returns the death test count. + int death_test_count() const { return death_test_count_; } + + // Increments the death test count, returning the new count. + int increment_death_test_count() { return ++death_test_count_; } + + // Clears the test part results. + void ClearTestPartResults(); + + // Clears the object. + void Clear(); + + // Protects mutable state of the property vector and of owned + // properties, whose values may be updated. + internal::Mutex test_properites_mutex_; + + // The vector of TestPartResults + std::vector test_part_results_; + // The vector of TestProperties + std::vector test_properties_; + // Running count of death tests. + int death_test_count_; + // The elapsed time, in milliseconds. + TimeInMillis elapsed_time_; + + // We disallow copying TestResult. + GTEST_DISALLOW_COPY_AND_ASSIGN_(TestResult); +}; // class TestResult + +// A TestInfo object stores the following information about a test: +// +// Test case name +// Test name +// Whether the test should be run +// A function pointer that creates the test object when invoked +// Test result +// +// The constructor of TestInfo registers itself with the UnitTest +// singleton such that the RUN_ALL_TESTS() macro knows which tests to +// run. +class GTEST_API_ TestInfo { + public: + // Destructs a TestInfo object. This function is not virtual, so + // don't inherit from TestInfo. + ~TestInfo(); + + // Returns the test case name. + const char* test_case_name() const { return test_case_name_.c_str(); } + + // Returns the test name. + const char* name() const { return name_.c_str(); } + + // Returns the name of the parameter type, or NULL if this is not a typed + // or a type-parameterized test. + const char* type_param() const { + if (type_param_.get() != NULL) + return type_param_->c_str(); + return NULL; + } + + // Returns the text representation of the value parameter, or NULL if this + // is not a value-parameterized test. + const char* value_param() const { + if (value_param_.get() != NULL) + return value_param_->c_str(); + return NULL; + } + + // Returns true if this test should run, that is if the test is not + // disabled (or it is disabled but the also_run_disabled_tests flag has + // been specified) and its full name matches the user-specified filter. + // + // Google Test allows the user to filter the tests by their full names. + // The full name of a test Bar in test case Foo is defined as + // "Foo.Bar". Only the tests that match the filter will run. + // + // A filter is a colon-separated list of glob (not regex) patterns, + // optionally followed by a '-' and a colon-separated list of + // negative patterns (tests to exclude). A test is run if it + // matches one of the positive patterns and does not match any of + // the negative patterns. + // + // For example, *A*:Foo.* is a filter that matches any string that + // contains the character 'A' or starts with "Foo.". + bool should_run() const { return should_run_; } + + // Returns true iff this test will appear in the XML report. + bool is_reportable() const { + // For now, the XML report includes all tests matching the filter. + // In the future, we may trim tests that are excluded because of + // sharding. + return matches_filter_; + } + + // Returns the result of the test. + const TestResult* result() const { return &result_; } + + private: +#if GTEST_HAS_DEATH_TEST + friend class internal::DefaultDeathTestFactory; +#endif // GTEST_HAS_DEATH_TEST + friend class Test; + friend class TestCase; + friend class internal::UnitTestImpl; + friend class internal::StreamingListenerTest; + friend TestInfo* internal::MakeAndRegisterTestInfo( + const char* test_case_name, + const char* name, + const char* type_param, + const char* value_param, + internal::TypeId fixture_class_id, + Test::SetUpTestCaseFunc set_up_tc, + Test::TearDownTestCaseFunc tear_down_tc, + internal::TestFactoryBase* factory); + + // Constructs a TestInfo object. The newly constructed instance assumes + // ownership of the factory object. + TestInfo(const std::string& test_case_name, + const std::string& name, + const char* a_type_param, // NULL if not a type-parameterized test + const char* a_value_param, // NULL if not a value-parameterized test + internal::TypeId fixture_class_id, + internal::TestFactoryBase* factory); + + // Increments the number of death tests encountered in this test so + // far. + int increment_death_test_count() { + return result_.increment_death_test_count(); + } + + // Creates the test object, runs it, records its result, and then + // deletes it. + void Run(); + + static void ClearTestResult(TestInfo* test_info) { + test_info->result_.Clear(); + } + + // These fields are immutable properties of the test. + const std::string test_case_name_; // Test case name + const std::string name_; // Test name + // Name of the parameter type, or NULL if this is not a typed or a + // type-parameterized test. + const internal::scoped_ptr type_param_; + // Text representation of the value parameter, or NULL if this is not a + // value-parameterized test. + const internal::scoped_ptr value_param_; + const internal::TypeId fixture_class_id_; // ID of the test fixture class + bool should_run_; // True iff this test should run + bool is_disabled_; // True iff this test is disabled + bool matches_filter_; // True if this test matches the + // user-specified filter. + internal::TestFactoryBase* const factory_; // The factory that creates + // the test object + + // This field is mutable and needs to be reset before running the + // test for the second time. + TestResult result_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(TestInfo); +}; + +// A test case, which consists of a vector of TestInfos. +// +// TestCase is not copyable. +class GTEST_API_ TestCase { + public: + // Creates a TestCase with the given name. + // + // TestCase does NOT have a default constructor. Always use this + // constructor to create a TestCase object. + // + // Arguments: + // + // name: name of the test case + // a_type_param: the name of the test's type parameter, or NULL if + // this is not a type-parameterized test. + // set_up_tc: pointer to the function that sets up the test case + // tear_down_tc: pointer to the function that tears down the test case + TestCase(const char* name, const char* a_type_param, + Test::SetUpTestCaseFunc set_up_tc, + Test::TearDownTestCaseFunc tear_down_tc); + + // Destructor of TestCase. + virtual ~TestCase(); + + // Gets the name of the TestCase. + const char* name() const { return name_.c_str(); } + + // Returns the name of the parameter type, or NULL if this is not a + // type-parameterized test case. + const char* type_param() const { + if (type_param_.get() != NULL) + return type_param_->c_str(); + return NULL; + } + + // Returns true if any test in this test case should run. + bool should_run() const { return should_run_; } + + // Gets the number of successful tests in this test case. + int successful_test_count() const; + + // Gets the number of failed tests in this test case. + int failed_test_count() const; + + // Gets the number of disabled tests that will be reported in the XML report. + int reportable_disabled_test_count() const; + + // Gets the number of disabled tests in this test case. + int disabled_test_count() const; + + // Gets the number of tests to be printed in the XML report. + int reportable_test_count() const; + + // Get the number of tests in this test case that should run. + int test_to_run_count() const; + + // Gets the number of all tests in this test case. + int total_test_count() const; + + // Returns true iff the test case passed. + bool Passed() const { return !Failed(); } + + // Returns true iff the test case failed. + bool Failed() const { return failed_test_count() > 0; } + + // Returns the elapsed time, in milliseconds. + TimeInMillis elapsed_time() const { return elapsed_time_; } + + // Returns the i-th test among all the tests. i can range from 0 to + // total_test_count() - 1. If i is not in that range, returns NULL. + const TestInfo* GetTestInfo(int i) const; + + // Returns the TestResult that holds test properties recorded during + // execution of SetUpTestCase and TearDownTestCase. + const TestResult& ad_hoc_test_result() const { return ad_hoc_test_result_; } + + private: + friend class Test; + friend class internal::UnitTestImpl; + + // Gets the (mutable) vector of TestInfos in this TestCase. + std::vector& test_info_list() { return test_info_list_; } + + // Gets the (immutable) vector of TestInfos in this TestCase. + const std::vector& test_info_list() const { + return test_info_list_; + } + + // Returns the i-th test among all the tests. i can range from 0 to + // total_test_count() - 1. If i is not in that range, returns NULL. + TestInfo* GetMutableTestInfo(int i); + + // Sets the should_run member. + void set_should_run(bool should) { should_run_ = should; } + + // Adds a TestInfo to this test case. Will delete the TestInfo upon + // destruction of the TestCase object. + void AddTestInfo(TestInfo * test_info); + + // Clears the results of all tests in this test case. + void ClearResult(); + + // Clears the results of all tests in the given test case. + static void ClearTestCaseResult(TestCase* test_case) { + test_case->ClearResult(); + } + + // Runs every test in this TestCase. + void Run(); + + // Runs SetUpTestCase() for this TestCase. This wrapper is needed + // for catching exceptions thrown from SetUpTestCase(). + void RunSetUpTestCase() { (*set_up_tc_)(); } + + // Runs TearDownTestCase() for this TestCase. This wrapper is + // needed for catching exceptions thrown from TearDownTestCase(). + void RunTearDownTestCase() { (*tear_down_tc_)(); } + + // Returns true iff test passed. + static bool TestPassed(const TestInfo* test_info) { + return test_info->should_run() && test_info->result()->Passed(); + } + + // Returns true iff test failed. + static bool TestFailed(const TestInfo* test_info) { + return test_info->should_run() && test_info->result()->Failed(); + } + + // Returns true iff the test is disabled and will be reported in the XML + // report. + static bool TestReportableDisabled(const TestInfo* test_info) { + return test_info->is_reportable() && test_info->is_disabled_; + } + + // Returns true iff test is disabled. + static bool TestDisabled(const TestInfo* test_info) { + return test_info->is_disabled_; + } + + // Returns true iff this test will appear in the XML report. + static bool TestReportable(const TestInfo* test_info) { + return test_info->is_reportable(); + } + + // Returns true if the given test should run. + static bool ShouldRunTest(const TestInfo* test_info) { + return test_info->should_run(); + } + + // Shuffles the tests in this test case. + void ShuffleTests(internal::Random* random); + + // Restores the test order to before the first shuffle. + void UnshuffleTests(); + + // Name of the test case. + std::string name_; + // Name of the parameter type, or NULL if this is not a typed or a + // type-parameterized test. + const internal::scoped_ptr type_param_; + // The vector of TestInfos in their original order. It owns the + // elements in the vector. + std::vector test_info_list_; + // Provides a level of indirection for the test list to allow easy + // shuffling and restoring the test order. The i-th element in this + // vector is the index of the i-th test in the shuffled test list. + std::vector test_indices_; + // Pointer to the function that sets up the test case. + Test::SetUpTestCaseFunc set_up_tc_; + // Pointer to the function that tears down the test case. + Test::TearDownTestCaseFunc tear_down_tc_; + // True iff any test in this test case should run. + bool should_run_; + // Elapsed time, in milliseconds. + TimeInMillis elapsed_time_; + // Holds test properties recorded during execution of SetUpTestCase and + // TearDownTestCase. + TestResult ad_hoc_test_result_; + + // We disallow copying TestCases. + GTEST_DISALLOW_COPY_AND_ASSIGN_(TestCase); +}; + +// An Environment object is capable of setting up and tearing down an +// environment. You should subclass this to define your own +// environment(s). +// +// An Environment object does the set-up and tear-down in virtual +// methods SetUp() and TearDown() instead of the constructor and the +// destructor, as: +// +// 1. You cannot safely throw from a destructor. This is a problem +// as in some cases Google Test is used where exceptions are enabled, and +// we may want to implement ASSERT_* using exceptions where they are +// available. +// 2. You cannot use ASSERT_* directly in a constructor or +// destructor. +class Environment { + public: + // The d'tor is virtual as we need to subclass Environment. + virtual ~Environment() {} + + // Override this to define how to set up the environment. + virtual void SetUp() {} + + // Override this to define how to tear down the environment. + virtual void TearDown() {} + private: + // If you see an error about overriding the following function or + // about it being private, you have mis-spelled SetUp() as Setup(). + struct Setup_should_be_spelled_SetUp {}; + virtual Setup_should_be_spelled_SetUp* Setup() { return NULL; } +}; + +// The interface for tracing execution of tests. The methods are organized in +// the order the corresponding events are fired. +class TestEventListener { + public: + virtual ~TestEventListener() {} + + // Fired before any test activity starts. + virtual void OnTestProgramStart(const UnitTest& unit_test) = 0; + + // Fired before each iteration of tests starts. There may be more than + // one iteration if GTEST_FLAG(repeat) is set. iteration is the iteration + // index, starting from 0. + virtual void OnTestIterationStart(const UnitTest& unit_test, + int iteration) = 0; + + // Fired before environment set-up for each iteration of tests starts. + virtual void OnEnvironmentsSetUpStart(const UnitTest& unit_test) = 0; + + // Fired after environment set-up for each iteration of tests ends. + virtual void OnEnvironmentsSetUpEnd(const UnitTest& unit_test) = 0; + + // Fired before the test case starts. + virtual void OnTestCaseStart(const TestCase& test_case) = 0; + + // Fired before the test starts. + virtual void OnTestStart(const TestInfo& test_info) = 0; + + // Fired after a failed assertion or a SUCCEED() invocation. + virtual void OnTestPartResult(const TestPartResult& test_part_result) = 0; + + // Fired after the test ends. + virtual void OnTestEnd(const TestInfo& test_info) = 0; + + // Fired after the test case ends. + virtual void OnTestCaseEnd(const TestCase& test_case) = 0; + + // Fired before environment tear-down for each iteration of tests starts. + virtual void OnEnvironmentsTearDownStart(const UnitTest& unit_test) = 0; + + // Fired after environment tear-down for each iteration of tests ends. + virtual void OnEnvironmentsTearDownEnd(const UnitTest& unit_test) = 0; + + // Fired after each iteration of tests finishes. + virtual void OnTestIterationEnd(const UnitTest& unit_test, + int iteration) = 0; + + // Fired after all test activities have ended. + virtual void OnTestProgramEnd(const UnitTest& unit_test) = 0; +}; + +// The convenience class for users who need to override just one or two +// methods and are not concerned that a possible change to a signature of +// the methods they override will not be caught during the build. For +// comments about each method please see the definition of TestEventListener +// above. +class EmptyTestEventListener : public TestEventListener { + public: + virtual void OnTestProgramStart(const UnitTest& /*unit_test*/) {} + virtual void OnTestIterationStart(const UnitTest& /*unit_test*/, + int /*iteration*/) {} + virtual void OnEnvironmentsSetUpStart(const UnitTest& /*unit_test*/) {} + virtual void OnEnvironmentsSetUpEnd(const UnitTest& /*unit_test*/) {} + virtual void OnTestCaseStart(const TestCase& /*test_case*/) {} + virtual void OnTestStart(const TestInfo& /*test_info*/) {} + virtual void OnTestPartResult(const TestPartResult& /*test_part_result*/) {} + virtual void OnTestEnd(const TestInfo& /*test_info*/) {} + virtual void OnTestCaseEnd(const TestCase& /*test_case*/) {} + virtual void OnEnvironmentsTearDownStart(const UnitTest& /*unit_test*/) {} + virtual void OnEnvironmentsTearDownEnd(const UnitTest& /*unit_test*/) {} + virtual void OnTestIterationEnd(const UnitTest& /*unit_test*/, + int /*iteration*/) {} + virtual void OnTestProgramEnd(const UnitTest& /*unit_test*/) {} +}; + +// TestEventListeners lets users add listeners to track events in Google Test. +class GTEST_API_ TestEventListeners { + public: + TestEventListeners(); + ~TestEventListeners(); + + // Appends an event listener to the end of the list. Google Test assumes + // the ownership of the listener (i.e. it will delete the listener when + // the test program finishes). + void Append(TestEventListener* listener); + + // Removes the given event listener from the list and returns it. It then + // becomes the caller's responsibility to delete the listener. Returns + // NULL if the listener is not found in the list. + TestEventListener* Release(TestEventListener* listener); + + // Returns the standard listener responsible for the default console + // output. Can be removed from the listeners list to shut down default + // console output. Note that removing this object from the listener list + // with Release transfers its ownership to the caller and makes this + // function return NULL the next time. + TestEventListener* default_result_printer() const { + return default_result_printer_; + } + + // Returns the standard listener responsible for the default XML output + // controlled by the --gtest_output=xml flag. Can be removed from the + // listeners list by users who want to shut down the default XML output + // controlled by this flag and substitute it with custom one. Note that + // removing this object from the listener list with Release transfers its + // ownership to the caller and makes this function return NULL the next + // time. + TestEventListener* default_xml_generator() const { + return default_xml_generator_; + } + + private: + friend class TestCase; + friend class TestInfo; + friend class internal::DefaultGlobalTestPartResultReporter; + friend class internal::NoExecDeathTest; + friend class internal::TestEventListenersAccessor; + friend class internal::UnitTestImpl; + + // Returns repeater that broadcasts the TestEventListener events to all + // subscribers. + TestEventListener* repeater(); + + // Sets the default_result_printer attribute to the provided listener. + // The listener is also added to the listener list and previous + // default_result_printer is removed from it and deleted. The listener can + // also be NULL in which case it will not be added to the list. Does + // nothing if the previous and the current listener objects are the same. + void SetDefaultResultPrinter(TestEventListener* listener); + + // Sets the default_xml_generator attribute to the provided listener. The + // listener is also added to the listener list and previous + // default_xml_generator is removed from it and deleted. The listener can + // also be NULL in which case it will not be added to the list. Does + // nothing if the previous and the current listener objects are the same. + void SetDefaultXmlGenerator(TestEventListener* listener); + + // Controls whether events will be forwarded by the repeater to the + // listeners in the list. + bool EventForwardingEnabled() const; + void SuppressEventForwarding(); + + // The actual list of listeners. + internal::TestEventRepeater* repeater_; + // Listener responsible for the standard result output. + TestEventListener* default_result_printer_; + // Listener responsible for the creation of the XML output file. + TestEventListener* default_xml_generator_; + + // We disallow copying TestEventListeners. + GTEST_DISALLOW_COPY_AND_ASSIGN_(TestEventListeners); +}; + +// A UnitTest consists of a vector of TestCases. +// +// This is a singleton class. The only instance of UnitTest is +// created when UnitTest::GetInstance() is first called. This +// instance is never deleted. +// +// UnitTest is not copyable. +// +// This class is thread-safe as long as the methods are called +// according to their specification. +class GTEST_API_ UnitTest { + public: + // Gets the singleton UnitTest object. The first time this method + // is called, a UnitTest object is constructed and returned. + // Consecutive calls will return the same object. + static UnitTest* GetInstance(); + + // Runs all tests in this UnitTest object and prints the result. + // Returns 0 if successful, or 1 otherwise. + // + // This method can only be called from the main thread. + // + // INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. + int Run() GTEST_MUST_USE_RESULT_; + + // Returns the working directory when the first TEST() or TEST_F() + // was executed. The UnitTest object owns the string. + const char* original_working_dir() const; + + // Returns the TestCase object for the test that's currently running, + // or NULL if no test is running. + const TestCase* current_test_case() const + GTEST_LOCK_EXCLUDED_(mutex_); + + // Returns the TestInfo object for the test that's currently running, + // or NULL if no test is running. + const TestInfo* current_test_info() const + GTEST_LOCK_EXCLUDED_(mutex_); + + // Returns the random seed used at the start of the current test run. + int random_seed() const; + +#if GTEST_HAS_PARAM_TEST + // Returns the ParameterizedTestCaseRegistry object used to keep track of + // value-parameterized tests and instantiate and register them. + // + // INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. + internal::ParameterizedTestCaseRegistry& parameterized_test_registry() + GTEST_LOCK_EXCLUDED_(mutex_); +#endif // GTEST_HAS_PARAM_TEST + + // Gets the number of successful test cases. + int successful_test_case_count() const; + + // Gets the number of failed test cases. + int failed_test_case_count() const; + + // Gets the number of all test cases. + int total_test_case_count() const; + + // Gets the number of all test cases that contain at least one test + // that should run. + int test_case_to_run_count() const; + + // Gets the number of successful tests. + int successful_test_count() const; + + // Gets the number of failed tests. + int failed_test_count() const; + + // Gets the number of disabled tests that will be reported in the XML report. + int reportable_disabled_test_count() const; + + // Gets the number of disabled tests. + int disabled_test_count() const; + + // Gets the number of tests to be printed in the XML report. + int reportable_test_count() const; + + // Gets the number of all tests. + int total_test_count() const; + + // Gets the number of tests that should run. + int test_to_run_count() const; + + // Gets the time of the test program start, in ms from the start of the + // UNIX epoch. + TimeInMillis start_timestamp() const; + + // Gets the elapsed time, in milliseconds. + TimeInMillis elapsed_time() const; + + // Returns true iff the unit test passed (i.e. all test cases passed). + bool Passed() const; + + // Returns true iff the unit test failed (i.e. some test case failed + // or something outside of all tests failed). + bool Failed() const; + + // Gets the i-th test case among all the test cases. i can range from 0 to + // total_test_case_count() - 1. If i is not in that range, returns NULL. + const TestCase* GetTestCase(int i) const; + + // Returns the TestResult containing information on test failures and + // properties logged outside of individual test cases. + const TestResult& ad_hoc_test_result() const; + + // Returns the list of event listeners that can be used to track events + // inside Google Test. + TestEventListeners& listeners(); + + private: + // Registers and returns a global test environment. When a test + // program is run, all global test environments will be set-up in + // the order they were registered. After all tests in the program + // have finished, all global test environments will be torn-down in + // the *reverse* order they were registered. + // + // The UnitTest object takes ownership of the given environment. + // + // This method can only be called from the main thread. + Environment* AddEnvironment(Environment* env); + + // Adds a TestPartResult to the current TestResult object. All + // Google Test assertion macros (e.g. ASSERT_TRUE, EXPECT_EQ, etc) + // eventually call this to report their results. The user code + // should use the assertion macros instead of calling this directly. + void AddTestPartResult(TestPartResult::Type result_type, + const char* file_name, + int line_number, + const std::string& message, + const std::string& os_stack_trace) + GTEST_LOCK_EXCLUDED_(mutex_); + + // Adds a TestProperty to the current TestResult object when invoked from + // inside a test, to current TestCase's ad_hoc_test_result_ when invoked + // from SetUpTestCase or TearDownTestCase, or to the global property set + // when invoked elsewhere. If the result already contains a property with + // the same key, the value will be updated. + void RecordProperty(const std::string& key, const std::string& value); + + // Gets the i-th test case among all the test cases. i can range from 0 to + // total_test_case_count() - 1. If i is not in that range, returns NULL. + TestCase* GetMutableTestCase(int i); + + // Accessors for the implementation object. + internal::UnitTestImpl* impl() { return impl_; } + const internal::UnitTestImpl* impl() const { return impl_; } + + // These classes and funcions are friends as they need to access private + // members of UnitTest. + friend class Test; + friend class internal::AssertHelper; + friend class internal::ScopedTrace; + friend class internal::StreamingListenerTest; + friend class internal::UnitTestRecordPropertyTestHelper; + friend Environment* AddGlobalTestEnvironment(Environment* env); + friend internal::UnitTestImpl* internal::GetUnitTestImpl(); + friend void internal::ReportFailureInUnknownLocation( + TestPartResult::Type result_type, + const std::string& message); + + // Creates an empty UnitTest. + UnitTest(); + + // D'tor + virtual ~UnitTest(); + + // Pushes a trace defined by SCOPED_TRACE() on to the per-thread + // Google Test trace stack. + void PushGTestTrace(const internal::TraceInfo& trace) + GTEST_LOCK_EXCLUDED_(mutex_); + + // Pops a trace from the per-thread Google Test trace stack. + void PopGTestTrace() + GTEST_LOCK_EXCLUDED_(mutex_); + + // Protects mutable state in *impl_. This is mutable as some const + // methods need to lock it too. + mutable internal::Mutex mutex_; + + // Opaque implementation object. This field is never changed once + // the object is constructed. We don't mark it as const here, as + // doing so will cause a warning in the constructor of UnitTest. + // Mutable state in *impl_ is protected by mutex_. + internal::UnitTestImpl* impl_; + + // We disallow copying UnitTest. + GTEST_DISALLOW_COPY_AND_ASSIGN_(UnitTest); +}; + +// A convenient wrapper for adding an environment for the test +// program. +// +// You should call this before RUN_ALL_TESTS() is called, probably in +// main(). If you use gtest_main, you need to call this before main() +// starts for it to take effect. For example, you can define a global +// variable like this: +// +// testing::Environment* const foo_env = +// testing::AddGlobalTestEnvironment(new FooEnvironment); +// +// However, we strongly recommend you to write your own main() and +// call AddGlobalTestEnvironment() there, as relying on initialization +// of global variables makes the code harder to read and may cause +// problems when you register multiple environments from different +// translation units and the environments have dependencies among them +// (remember that the compiler doesn't guarantee the order in which +// global variables from different translation units are initialized). +inline Environment* AddGlobalTestEnvironment(Environment* env) { + return UnitTest::GetInstance()->AddEnvironment(env); +} + +// Initializes Google Test. This must be called before calling +// RUN_ALL_TESTS(). In particular, it parses a command line for the +// flags that Google Test recognizes. Whenever a Google Test flag is +// seen, it is removed from argv, and *argc is decremented. +// +// No value is returned. Instead, the Google Test flag variables are +// updated. +// +// Calling the function for the second time has no user-visible effect. +GTEST_API_ void InitGoogleTest(int* argc, char** argv); + +// This overloaded version can be used in Windows programs compiled in +// UNICODE mode. +GTEST_API_ void InitGoogleTest(int* argc, wchar_t** argv); + +namespace internal { + +// FormatForComparison::Format(value) formats a +// value of type ToPrint that is an operand of a comparison assertion +// (e.g. ASSERT_EQ). OtherOperand is the type of the other operand in +// the comparison, and is used to help determine the best way to +// format the value. In particular, when the value is a C string +// (char pointer) and the other operand is an STL string object, we +// want to format the C string as a string, since we know it is +// compared by value with the string object. If the value is a char +// pointer but the other operand is not an STL string object, we don't +// know whether the pointer is supposed to point to a NUL-terminated +// string, and thus want to print it as a pointer to be safe. +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. + +// The default case. +template +class FormatForComparison { + public: + static ::std::string Format(const ToPrint& value) { + return ::testing::PrintToString(value); + } +}; + +// Array. +template +class FormatForComparison { + public: + static ::std::string Format(const ToPrint* value) { + return FormatForComparison::Format(value); + } +}; + +// By default, print C string as pointers to be safe, as we don't know +// whether they actually point to a NUL-terminated string. + +#define GTEST_IMPL_FORMAT_C_STRING_AS_POINTER_(CharType) \ + template \ + class FormatForComparison { \ + public: \ + static ::std::string Format(CharType* value) { \ + return ::testing::PrintToString(static_cast(value)); \ + } \ + } + +GTEST_IMPL_FORMAT_C_STRING_AS_POINTER_(char); +GTEST_IMPL_FORMAT_C_STRING_AS_POINTER_(const char); +GTEST_IMPL_FORMAT_C_STRING_AS_POINTER_(wchar_t); +GTEST_IMPL_FORMAT_C_STRING_AS_POINTER_(const wchar_t); + +#undef GTEST_IMPL_FORMAT_C_STRING_AS_POINTER_ + +// If a C string is compared with an STL string object, we know it's meant +// to point to a NUL-terminated string, and thus can print it as a string. + +#define GTEST_IMPL_FORMAT_C_STRING_AS_STRING_(CharType, OtherStringType) \ + template <> \ + class FormatForComparison { \ + public: \ + static ::std::string Format(CharType* value) { \ + return ::testing::PrintToString(value); \ + } \ + } + +GTEST_IMPL_FORMAT_C_STRING_AS_STRING_(char, ::std::string); +GTEST_IMPL_FORMAT_C_STRING_AS_STRING_(const char, ::std::string); + +#if GTEST_HAS_GLOBAL_STRING +GTEST_IMPL_FORMAT_C_STRING_AS_STRING_(char, ::string); +GTEST_IMPL_FORMAT_C_STRING_AS_STRING_(const char, ::string); +#endif + +#if GTEST_HAS_GLOBAL_WSTRING +GTEST_IMPL_FORMAT_C_STRING_AS_STRING_(wchar_t, ::wstring); +GTEST_IMPL_FORMAT_C_STRING_AS_STRING_(const wchar_t, ::wstring); +#endif + +#if GTEST_HAS_STD_WSTRING +GTEST_IMPL_FORMAT_C_STRING_AS_STRING_(wchar_t, ::std::wstring); +GTEST_IMPL_FORMAT_C_STRING_AS_STRING_(const wchar_t, ::std::wstring); +#endif + +#undef GTEST_IMPL_FORMAT_C_STRING_AS_STRING_ + +// Formats a comparison assertion (e.g. ASSERT_EQ, EXPECT_LT, and etc) +// operand to be used in a failure message. The type (but not value) +// of the other operand may affect the format. This allows us to +// print a char* as a raw pointer when it is compared against another +// char* or void*, and print it as a C string when it is compared +// against an std::string object, for example. +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +template +std::string FormatForComparisonFailureMessage( + const T1& value, const T2& /* other_operand */) { + return FormatForComparison::Format(value); +} + +// Separate the error generating code from the code path to reduce the stack +// frame size of CmpHelperEQ. This helps reduce the overhead of some sanitizers +// when calling EXPECT_* in a tight loop. +template +AssertionResult CmpHelperEQFailure(const char* expected_expression, + const char* actual_expression, + const T1& expected, const T2& actual) { + return EqFailure(expected_expression, + actual_expression, + FormatForComparisonFailureMessage(expected, actual), + FormatForComparisonFailureMessage(actual, expected), + false); +} + +// The helper function for {ASSERT|EXPECT}_EQ. +template +AssertionResult CmpHelperEQ(const char* expected_expression, + const char* actual_expression, + const T1& expected, + const T2& actual) { +GTEST_DISABLE_MSC_WARNINGS_PUSH_(4389 /* signed/unsigned mismatch */) + if (expected == actual) { + return AssertionSuccess(); + } +GTEST_DISABLE_MSC_WARNINGS_POP_() + + return CmpHelperEQFailure(expected_expression, actual_expression, expected, + actual); +} + +// With this overloaded version, we allow anonymous enums to be used +// in {ASSERT|EXPECT}_EQ when compiled with gcc 4, as anonymous enums +// can be implicitly cast to BiggestInt. +GTEST_API_ AssertionResult CmpHelperEQ(const char* expected_expression, + const char* actual_expression, + BiggestInt expected, + BiggestInt actual); + +// The helper class for {ASSERT|EXPECT}_EQ. The template argument +// lhs_is_null_literal is true iff the first argument to ASSERT_EQ() +// is a null pointer literal. The following default implementation is +// for lhs_is_null_literal being false. +template +class EqHelper { + public: + // This templatized version is for the general case. + template + static AssertionResult Compare(const char* expected_expression, + const char* actual_expression, + const T1& expected, + const T2& actual) { + return CmpHelperEQ(expected_expression, actual_expression, expected, + actual); + } + + // With this overloaded version, we allow anonymous enums to be used + // in {ASSERT|EXPECT}_EQ when compiled with gcc 4, as anonymous + // enums can be implicitly cast to BiggestInt. + // + // Even though its body looks the same as the above version, we + // cannot merge the two, as it will make anonymous enums unhappy. + static AssertionResult Compare(const char* expected_expression, + const char* actual_expression, + BiggestInt expected, + BiggestInt actual) { + return CmpHelperEQ(expected_expression, actual_expression, expected, + actual); + } +}; + +// This specialization is used when the first argument to ASSERT_EQ() +// is a null pointer literal, like NULL, false, or 0. +template <> +class EqHelper { + public: + // We define two overloaded versions of Compare(). The first + // version will be picked when the second argument to ASSERT_EQ() is + // NOT a pointer, e.g. ASSERT_EQ(0, AnIntFunction()) or + // EXPECT_EQ(false, a_bool). + template + static AssertionResult Compare( + const char* expected_expression, + const char* actual_expression, + const T1& expected, + const T2& actual, + // The following line prevents this overload from being considered if T2 + // is not a pointer type. We need this because ASSERT_EQ(NULL, my_ptr) + // expands to Compare("", "", NULL, my_ptr), which requires a conversion + // to match the Secret* in the other overload, which would otherwise make + // this template match better. + typename EnableIf::value>::type* = 0) { + return CmpHelperEQ(expected_expression, actual_expression, expected, + actual); + } + + // This version will be picked when the second argument to ASSERT_EQ() is a + // pointer, e.g. ASSERT_EQ(NULL, a_pointer). + template + static AssertionResult Compare( + const char* expected_expression, + const char* actual_expression, + // We used to have a second template parameter instead of Secret*. That + // template parameter would deduce to 'long', making this a better match + // than the first overload even without the first overload's EnableIf. + // Unfortunately, gcc with -Wconversion-null warns when "passing NULL to + // non-pointer argument" (even a deduced integral argument), so the old + // implementation caused warnings in user code. + Secret* /* expected (NULL) */, + T* actual) { + // We already know that 'expected' is a null pointer. + return CmpHelperEQ(expected_expression, actual_expression, + static_cast(NULL), actual); + } +}; + +// Separate the error generating code from the code path to reduce the stack +// frame size of CmpHelperOP. This helps reduce the overhead of some sanitizers +// when calling EXPECT_OP in a tight loop. +template +AssertionResult CmpHelperOpFailure(const char* expr1, const char* expr2, + const T1& val1, const T2& val2, + const char* op) { + return AssertionFailure() + << "Expected: (" << expr1 << ") " << op << " (" << expr2 + << "), actual: " << FormatForComparisonFailureMessage(val1, val2) + << " vs " << FormatForComparisonFailureMessage(val2, val1); +} + +// A macro for implementing the helper functions needed to implement +// ASSERT_?? and EXPECT_??. It is here just to avoid copy-and-paste +// of similar code. +// +// For each templatized helper function, we also define an overloaded +// version for BiggestInt in order to reduce code bloat and allow +// anonymous enums to be used with {ASSERT|EXPECT}_?? when compiled +// with gcc 4. +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. + +#define GTEST_IMPL_CMP_HELPER_(op_name, op)\ +template \ +AssertionResult CmpHelper##op_name(const char* expr1, const char* expr2, \ + const T1& val1, const T2& val2) {\ + if (val1 op val2) {\ + return AssertionSuccess();\ + } else {\ + return CmpHelperOpFailure(expr1, expr2, val1, val2, #op);\ + }\ +}\ +GTEST_API_ AssertionResult CmpHelper##op_name(\ + const char* expr1, const char* expr2, BiggestInt val1, BiggestInt val2) + +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. + +// Implements the helper function for {ASSERT|EXPECT}_NE +GTEST_IMPL_CMP_HELPER_(NE, !=); +// Implements the helper function for {ASSERT|EXPECT}_LE +GTEST_IMPL_CMP_HELPER_(LE, <=); +// Implements the helper function for {ASSERT|EXPECT}_LT +GTEST_IMPL_CMP_HELPER_(LT, <); +// Implements the helper function for {ASSERT|EXPECT}_GE +GTEST_IMPL_CMP_HELPER_(GE, >=); +// Implements the helper function for {ASSERT|EXPECT}_GT +GTEST_IMPL_CMP_HELPER_(GT, >); + +#undef GTEST_IMPL_CMP_HELPER_ + +// The helper function for {ASSERT|EXPECT}_STREQ. +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +GTEST_API_ AssertionResult CmpHelperSTREQ(const char* expected_expression, + const char* actual_expression, + const char* expected, + const char* actual); + +// The helper function for {ASSERT|EXPECT}_STRCASEEQ. +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +GTEST_API_ AssertionResult CmpHelperSTRCASEEQ(const char* expected_expression, + const char* actual_expression, + const char* expected, + const char* actual); + +// The helper function for {ASSERT|EXPECT}_STRNE. +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +GTEST_API_ AssertionResult CmpHelperSTRNE(const char* s1_expression, + const char* s2_expression, + const char* s1, + const char* s2); + +// The helper function for {ASSERT|EXPECT}_STRCASENE. +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +GTEST_API_ AssertionResult CmpHelperSTRCASENE(const char* s1_expression, + const char* s2_expression, + const char* s1, + const char* s2); + + +// Helper function for *_STREQ on wide strings. +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +GTEST_API_ AssertionResult CmpHelperSTREQ(const char* expected_expression, + const char* actual_expression, + const wchar_t* expected, + const wchar_t* actual); + +// Helper function for *_STRNE on wide strings. +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +GTEST_API_ AssertionResult CmpHelperSTRNE(const char* s1_expression, + const char* s2_expression, + const wchar_t* s1, + const wchar_t* s2); + +} // namespace internal + +// IsSubstring() and IsNotSubstring() are intended to be used as the +// first argument to {EXPECT,ASSERT}_PRED_FORMAT2(), not by +// themselves. They check whether needle is a substring of haystack +// (NULL is considered a substring of itself only), and return an +// appropriate error message when they fail. +// +// The {needle,haystack}_expr arguments are the stringified +// expressions that generated the two real arguments. +GTEST_API_ AssertionResult IsSubstring( + const char* needle_expr, const char* haystack_expr, + const char* needle, const char* haystack); +GTEST_API_ AssertionResult IsSubstring( + const char* needle_expr, const char* haystack_expr, + const wchar_t* needle, const wchar_t* haystack); +GTEST_API_ AssertionResult IsNotSubstring( + const char* needle_expr, const char* haystack_expr, + const char* needle, const char* haystack); +GTEST_API_ AssertionResult IsNotSubstring( + const char* needle_expr, const char* haystack_expr, + const wchar_t* needle, const wchar_t* haystack); +GTEST_API_ AssertionResult IsSubstring( + const char* needle_expr, const char* haystack_expr, + const ::std::string& needle, const ::std::string& haystack); +GTEST_API_ AssertionResult IsNotSubstring( + const char* needle_expr, const char* haystack_expr, + const ::std::string& needle, const ::std::string& haystack); + +#if GTEST_HAS_STD_WSTRING +GTEST_API_ AssertionResult IsSubstring( + const char* needle_expr, const char* haystack_expr, + const ::std::wstring& needle, const ::std::wstring& haystack); +GTEST_API_ AssertionResult IsNotSubstring( + const char* needle_expr, const char* haystack_expr, + const ::std::wstring& needle, const ::std::wstring& haystack); +#endif // GTEST_HAS_STD_WSTRING + +namespace internal { + +// Helper template function for comparing floating-points. +// +// Template parameter: +// +// RawType: the raw floating-point type (either float or double) +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +template +AssertionResult CmpHelperFloatingPointEQ(const char* expected_expression, + const char* actual_expression, + RawType expected, + RawType actual) { + const FloatingPoint lhs(expected), rhs(actual); + + if (lhs.AlmostEquals(rhs)) { + return AssertionSuccess(); + } + + ::std::stringstream expected_ss; + expected_ss << std::setprecision(std::numeric_limits::digits10 + 2) + << expected; + + ::std::stringstream actual_ss; + actual_ss << std::setprecision(std::numeric_limits::digits10 + 2) + << actual; + + return EqFailure(expected_expression, + actual_expression, + StringStreamToString(&expected_ss), + StringStreamToString(&actual_ss), + false); +} + +// Helper function for implementing ASSERT_NEAR. +// +// INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM. +GTEST_API_ AssertionResult DoubleNearPredFormat(const char* expr1, + const char* expr2, + const char* abs_error_expr, + double val1, + double val2, + double abs_error); + +// INTERNAL IMPLEMENTATION - DO NOT USE IN USER CODE. +// A class that enables one to stream messages to assertion macros +class GTEST_API_ AssertHelper { + public: + // Constructor. + AssertHelper(TestPartResult::Type type, + const char* file, + int line, + const char* message); + ~AssertHelper(); + + // Message assignment is a semantic trick to enable assertion + // streaming; see the GTEST_MESSAGE_ macro below. + void operator=(const Message& message) const; + + private: + // We put our data in a struct so that the size of the AssertHelper class can + // be as small as possible. This is important because gcc is incapable of + // re-using stack space even for temporary variables, so every EXPECT_EQ + // reserves stack space for another AssertHelper. + struct AssertHelperData { + AssertHelperData(TestPartResult::Type t, + const char* srcfile, + int line_num, + const char* msg) + : type(t), file(srcfile), line(line_num), message(msg) { } + + TestPartResult::Type const type; + const char* const file; + int const line; + std::string const message; + + private: + GTEST_DISALLOW_COPY_AND_ASSIGN_(AssertHelperData); + }; + + AssertHelperData* const data_; + + GTEST_DISALLOW_COPY_AND_ASSIGN_(AssertHelper); +}; + +} // namespace internal + +#if GTEST_HAS_PARAM_TEST +// The pure interface class that all value-parameterized tests inherit from. +// A value-parameterized class must inherit from both ::testing::Test and +// ::testing::WithParamInterface. In most cases that just means inheriting +// from ::testing::TestWithParam, but more complicated test hierarchies +// may need to inherit from Test and WithParamInterface at different levels. +// +// This interface has support for accessing the test parameter value via +// the GetParam() method. +// +// Use it with one of the parameter generator defining functions, like Range(), +// Values(), ValuesIn(), Bool(), and Combine(). +// +// class FooTest : public ::testing::TestWithParam { +// protected: +// FooTest() { +// // Can use GetParam() here. +// } +// virtual ~FooTest() { +// // Can use GetParam() here. +// } +// virtual void SetUp() { +// // Can use GetParam() here. +// } +// virtual void TearDown { +// // Can use GetParam() here. +// } +// }; +// TEST_P(FooTest, DoesBar) { +// // Can use GetParam() method here. +// Foo foo; +// ASSERT_TRUE(foo.DoesBar(GetParam())); +// } +// INSTANTIATE_TEST_CASE_P(OneToTenRange, FooTest, ::testing::Range(1, 10)); + +template +class WithParamInterface { + public: + typedef T ParamType; + virtual ~WithParamInterface() {} + + // The current parameter value. Is also available in the test fixture's + // constructor. This member function is non-static, even though it only + // references static data, to reduce the opportunity for incorrect uses + // like writing 'WithParamInterface::GetParam()' for a test that + // uses a fixture whose parameter type is int. + const ParamType& GetParam() const { + GTEST_CHECK_(parameter_ != NULL) + << "GetParam() can only be called inside a value-parameterized test " + << "-- did you intend to write TEST_P instead of TEST_F?"; + return *parameter_; + } + + private: + // Sets parameter value. The caller is responsible for making sure the value + // remains alive and unchanged throughout the current test. + static void SetParam(const ParamType* parameter) { + parameter_ = parameter; + } + + // Static value used for accessing parameter during a test lifetime. + static const ParamType* parameter_; + + // TestClass must be a subclass of WithParamInterface and Test. + template friend class internal::ParameterizedTestFactory; +}; + +template +const T* WithParamInterface::parameter_ = NULL; + +// Most value-parameterized classes can ignore the existence of +// WithParamInterface, and can just inherit from ::testing::TestWithParam. + +template +class TestWithParam : public Test, public WithParamInterface { +}; + +#endif // GTEST_HAS_PARAM_TEST + +// Macros for indicating success/failure in test code. + +// ADD_FAILURE unconditionally adds a failure to the current test. +// SUCCEED generates a success - it doesn't automatically make the +// current test successful, as a test is only successful when it has +// no failure. +// +// EXPECT_* verifies that a certain condition is satisfied. If not, +// it behaves like ADD_FAILURE. In particular: +// +// EXPECT_TRUE verifies that a Boolean condition is true. +// EXPECT_FALSE verifies that a Boolean condition is false. +// +// FAIL and ASSERT_* are similar to ADD_FAILURE and EXPECT_*, except +// that they will also abort the current function on failure. People +// usually want the fail-fast behavior of FAIL and ASSERT_*, but those +// writing data-driven tests often find themselves using ADD_FAILURE +// and EXPECT_* more. + +// Generates a nonfatal failure with a generic message. +#define ADD_FAILURE() GTEST_NONFATAL_FAILURE_("Failed") + +// Generates a nonfatal failure at the given source file location with +// a generic message. +#define ADD_FAILURE_AT(file, line) \ + GTEST_MESSAGE_AT_(file, line, "Failed", \ + ::testing::TestPartResult::kNonFatalFailure) + +// Generates a fatal failure with a generic message. +#define GTEST_FAIL() GTEST_FATAL_FAILURE_("Failed") + +// Define this macro to 1 to omit the definition of FAIL(), which is a +// generic name and clashes with some other libraries. +#if !GTEST_DONT_DEFINE_FAIL +# define FAIL() GTEST_FAIL() +#endif + +// Generates a success with a generic message. +#define GTEST_SUCCEED() GTEST_SUCCESS_("Succeeded") + +// Define this macro to 1 to omit the definition of SUCCEED(), which +// is a generic name and clashes with some other libraries. +#if !GTEST_DONT_DEFINE_SUCCEED +# define SUCCEED() GTEST_SUCCEED() +#endif + +// Macros for testing exceptions. +// +// * {ASSERT|EXPECT}_THROW(statement, expected_exception): +// Tests that the statement throws the expected exception. +// * {ASSERT|EXPECT}_NO_THROW(statement): +// Tests that the statement doesn't throw any exception. +// * {ASSERT|EXPECT}_ANY_THROW(statement): +// Tests that the statement throws an exception. + +#define EXPECT_THROW(statement, expected_exception) \ + GTEST_TEST_THROW_(statement, expected_exception, GTEST_NONFATAL_FAILURE_) +#define EXPECT_NO_THROW(statement) \ + GTEST_TEST_NO_THROW_(statement, GTEST_NONFATAL_FAILURE_) +#define EXPECT_ANY_THROW(statement) \ + GTEST_TEST_ANY_THROW_(statement, GTEST_NONFATAL_FAILURE_) +#define ASSERT_THROW(statement, expected_exception) \ + GTEST_TEST_THROW_(statement, expected_exception, GTEST_FATAL_FAILURE_) +#define ASSERT_NO_THROW(statement) \ + GTEST_TEST_NO_THROW_(statement, GTEST_FATAL_FAILURE_) +#define ASSERT_ANY_THROW(statement) \ + GTEST_TEST_ANY_THROW_(statement, GTEST_FATAL_FAILURE_) + +// Boolean assertions. Condition can be either a Boolean expression or an +// AssertionResult. For more information on how to use AssertionResult with +// these macros see comments on that class. +#define EXPECT_TRUE(condition) \ + GTEST_TEST_BOOLEAN_(condition, #condition, false, true, \ + GTEST_NONFATAL_FAILURE_) +#define EXPECT_FALSE(condition) \ + GTEST_TEST_BOOLEAN_(!(condition), #condition, true, false, \ + GTEST_NONFATAL_FAILURE_) +#define ASSERT_TRUE(condition) \ + GTEST_TEST_BOOLEAN_(condition, #condition, false, true, \ + GTEST_FATAL_FAILURE_) +#define ASSERT_FALSE(condition) \ + GTEST_TEST_BOOLEAN_(!(condition), #condition, true, false, \ + GTEST_FATAL_FAILURE_) + +// Includes the auto-generated header that implements a family of +// generic predicate assertion macros. +// Copyright 2006, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// This file is AUTOMATICALLY GENERATED on 10/31/2011 by command +// 'gen_gtest_pred_impl.py 5'. DO NOT EDIT BY HAND! +// +// Implements a family of generic predicate assertion macros. + +#ifndef GTEST_INCLUDE_GTEST_GTEST_PRED_IMPL_H_ +#define GTEST_INCLUDE_GTEST_GTEST_PRED_IMPL_H_ + +// Makes sure this header is not included before gtest.h. +#ifndef GTEST_INCLUDE_GTEST_GTEST_H_ +# error Do not include gtest_pred_impl.h directly. Include gtest.h instead. +#endif // GTEST_INCLUDE_GTEST_GTEST_H_ + +// This header implements a family of generic predicate assertion +// macros: +// +// ASSERT_PRED_FORMAT1(pred_format, v1) +// ASSERT_PRED_FORMAT2(pred_format, v1, v2) +// ... +// +// where pred_format is a function or functor that takes n (in the +// case of ASSERT_PRED_FORMATn) values and their source expression +// text, and returns a testing::AssertionResult. See the definition +// of ASSERT_EQ in gtest.h for an example. +// +// If you don't care about formatting, you can use the more +// restrictive version: +// +// ASSERT_PRED1(pred, v1) +// ASSERT_PRED2(pred, v1, v2) +// ... +// +// where pred is an n-ary function or functor that returns bool, +// and the values v1, v2, ..., must support the << operator for +// streaming to std::ostream. +// +// We also define the EXPECT_* variations. +// +// For now we only support predicates whose arity is at most 5. +// Please email googletestframework@googlegroups.com if you need +// support for higher arities. + +// GTEST_ASSERT_ is the basic statement to which all of the assertions +// in this file reduce. Don't use this in your code. + +#define GTEST_ASSERT_(expression, on_failure) \ + GTEST_AMBIGUOUS_ELSE_BLOCKER_ \ + if (const ::testing::AssertionResult gtest_ar = (expression)) \ + ; \ + else \ + on_failure(gtest_ar.failure_message()) + + +// Helper function for implementing {EXPECT|ASSERT}_PRED1. Don't use +// this in your code. +template +AssertionResult AssertPred1Helper(const char* pred_text, + const char* e1, + Pred pred, + const T1& v1) { + if (pred(v1)) return AssertionSuccess(); + + return AssertionFailure() << pred_text << "(" + << e1 << ") evaluates to false, where" + << "\n" << e1 << " evaluates to " << v1; +} + +// Internal macro for implementing {EXPECT|ASSERT}_PRED_FORMAT1. +// Don't use this in your code. +#define GTEST_PRED_FORMAT1_(pred_format, v1, on_failure)\ + GTEST_ASSERT_(pred_format(#v1, v1), \ + on_failure) + +// Internal macro for implementing {EXPECT|ASSERT}_PRED1. Don't use +// this in your code. +#define GTEST_PRED1_(pred, v1, on_failure)\ + GTEST_ASSERT_(::testing::AssertPred1Helper(#pred, \ + #v1, \ + pred, \ + v1), on_failure) + +// Unary predicate assertion macros. +#define EXPECT_PRED_FORMAT1(pred_format, v1) \ + GTEST_PRED_FORMAT1_(pred_format, v1, GTEST_NONFATAL_FAILURE_) +#define EXPECT_PRED1(pred, v1) \ + GTEST_PRED1_(pred, v1, GTEST_NONFATAL_FAILURE_) +#define ASSERT_PRED_FORMAT1(pred_format, v1) \ + GTEST_PRED_FORMAT1_(pred_format, v1, GTEST_FATAL_FAILURE_) +#define ASSERT_PRED1(pred, v1) \ + GTEST_PRED1_(pred, v1, GTEST_FATAL_FAILURE_) + + + +// Helper function for implementing {EXPECT|ASSERT}_PRED2. Don't use +// this in your code. +template +AssertionResult AssertPred2Helper(const char* pred_text, + const char* e1, + const char* e2, + Pred pred, + const T1& v1, + const T2& v2) { + if (pred(v1, v2)) return AssertionSuccess(); + + return AssertionFailure() << pred_text << "(" + << e1 << ", " + << e2 << ") evaluates to false, where" + << "\n" << e1 << " evaluates to " << v1 + << "\n" << e2 << " evaluates to " << v2; +} + +// Internal macro for implementing {EXPECT|ASSERT}_PRED_FORMAT2. +// Don't use this in your code. +#define GTEST_PRED_FORMAT2_(pred_format, v1, v2, on_failure)\ + GTEST_ASSERT_(pred_format(#v1, #v2, v1, v2), \ + on_failure) + +// Internal macro for implementing {EXPECT|ASSERT}_PRED2. Don't use +// this in your code. +#define GTEST_PRED2_(pred, v1, v2, on_failure)\ + GTEST_ASSERT_(::testing::AssertPred2Helper(#pred, \ + #v1, \ + #v2, \ + pred, \ + v1, \ + v2), on_failure) + +// Binary predicate assertion macros. +#define EXPECT_PRED_FORMAT2(pred_format, v1, v2) \ + GTEST_PRED_FORMAT2_(pred_format, v1, v2, GTEST_NONFATAL_FAILURE_) +#define EXPECT_PRED2(pred, v1, v2) \ + GTEST_PRED2_(pred, v1, v2, GTEST_NONFATAL_FAILURE_) +#define ASSERT_PRED_FORMAT2(pred_format, v1, v2) \ + GTEST_PRED_FORMAT2_(pred_format, v1, v2, GTEST_FATAL_FAILURE_) +#define ASSERT_PRED2(pred, v1, v2) \ + GTEST_PRED2_(pred, v1, v2, GTEST_FATAL_FAILURE_) + + + +// Helper function for implementing {EXPECT|ASSERT}_PRED3. Don't use +// this in your code. +template +AssertionResult AssertPred3Helper(const char* pred_text, + const char* e1, + const char* e2, + const char* e3, + Pred pred, + const T1& v1, + const T2& v2, + const T3& v3) { + if (pred(v1, v2, v3)) return AssertionSuccess(); + + return AssertionFailure() << pred_text << "(" + << e1 << ", " + << e2 << ", " + << e3 << ") evaluates to false, where" + << "\n" << e1 << " evaluates to " << v1 + << "\n" << e2 << " evaluates to " << v2 + << "\n" << e3 << " evaluates to " << v3; +} + +// Internal macro for implementing {EXPECT|ASSERT}_PRED_FORMAT3. +// Don't use this in your code. +#define GTEST_PRED_FORMAT3_(pred_format, v1, v2, v3, on_failure)\ + GTEST_ASSERT_(pred_format(#v1, #v2, #v3, v1, v2, v3), \ + on_failure) + +// Internal macro for implementing {EXPECT|ASSERT}_PRED3. Don't use +// this in your code. +#define GTEST_PRED3_(pred, v1, v2, v3, on_failure)\ + GTEST_ASSERT_(::testing::AssertPred3Helper(#pred, \ + #v1, \ + #v2, \ + #v3, \ + pred, \ + v1, \ + v2, \ + v3), on_failure) + +// Ternary predicate assertion macros. +#define EXPECT_PRED_FORMAT3(pred_format, v1, v2, v3) \ + GTEST_PRED_FORMAT3_(pred_format, v1, v2, v3, GTEST_NONFATAL_FAILURE_) +#define EXPECT_PRED3(pred, v1, v2, v3) \ + GTEST_PRED3_(pred, v1, v2, v3, GTEST_NONFATAL_FAILURE_) +#define ASSERT_PRED_FORMAT3(pred_format, v1, v2, v3) \ + GTEST_PRED_FORMAT3_(pred_format, v1, v2, v3, GTEST_FATAL_FAILURE_) +#define ASSERT_PRED3(pred, v1, v2, v3) \ + GTEST_PRED3_(pred, v1, v2, v3, GTEST_FATAL_FAILURE_) + + + +// Helper function for implementing {EXPECT|ASSERT}_PRED4. Don't use +// this in your code. +template +AssertionResult AssertPred4Helper(const char* pred_text, + const char* e1, + const char* e2, + const char* e3, + const char* e4, + Pred pred, + const T1& v1, + const T2& v2, + const T3& v3, + const T4& v4) { + if (pred(v1, v2, v3, v4)) return AssertionSuccess(); + + return AssertionFailure() << pred_text << "(" + << e1 << ", " + << e2 << ", " + << e3 << ", " + << e4 << ") evaluates to false, where" + << "\n" << e1 << " evaluates to " << v1 + << "\n" << e2 << " evaluates to " << v2 + << "\n" << e3 << " evaluates to " << v3 + << "\n" << e4 << " evaluates to " << v4; +} + +// Internal macro for implementing {EXPECT|ASSERT}_PRED_FORMAT4. +// Don't use this in your code. +#define GTEST_PRED_FORMAT4_(pred_format, v1, v2, v3, v4, on_failure)\ + GTEST_ASSERT_(pred_format(#v1, #v2, #v3, #v4, v1, v2, v3, v4), \ + on_failure) + +// Internal macro for implementing {EXPECT|ASSERT}_PRED4. Don't use +// this in your code. +#define GTEST_PRED4_(pred, v1, v2, v3, v4, on_failure)\ + GTEST_ASSERT_(::testing::AssertPred4Helper(#pred, \ + #v1, \ + #v2, \ + #v3, \ + #v4, \ + pred, \ + v1, \ + v2, \ + v3, \ + v4), on_failure) + +// 4-ary predicate assertion macros. +#define EXPECT_PRED_FORMAT4(pred_format, v1, v2, v3, v4) \ + GTEST_PRED_FORMAT4_(pred_format, v1, v2, v3, v4, GTEST_NONFATAL_FAILURE_) +#define EXPECT_PRED4(pred, v1, v2, v3, v4) \ + GTEST_PRED4_(pred, v1, v2, v3, v4, GTEST_NONFATAL_FAILURE_) +#define ASSERT_PRED_FORMAT4(pred_format, v1, v2, v3, v4) \ + GTEST_PRED_FORMAT4_(pred_format, v1, v2, v3, v4, GTEST_FATAL_FAILURE_) +#define ASSERT_PRED4(pred, v1, v2, v3, v4) \ + GTEST_PRED4_(pred, v1, v2, v3, v4, GTEST_FATAL_FAILURE_) + + + +// Helper function for implementing {EXPECT|ASSERT}_PRED5. Don't use +// this in your code. +template +AssertionResult AssertPred5Helper(const char* pred_text, + const char* e1, + const char* e2, + const char* e3, + const char* e4, + const char* e5, + Pred pred, + const T1& v1, + const T2& v2, + const T3& v3, + const T4& v4, + const T5& v5) { + if (pred(v1, v2, v3, v4, v5)) return AssertionSuccess(); + + return AssertionFailure() << pred_text << "(" + << e1 << ", " + << e2 << ", " + << e3 << ", " + << e4 << ", " + << e5 << ") evaluates to false, where" + << "\n" << e1 << " evaluates to " << v1 + << "\n" << e2 << " evaluates to " << v2 + << "\n" << e3 << " evaluates to " << v3 + << "\n" << e4 << " evaluates to " << v4 + << "\n" << e5 << " evaluates to " << v5; +} + +// Internal macro for implementing {EXPECT|ASSERT}_PRED_FORMAT5. +// Don't use this in your code. +#define GTEST_PRED_FORMAT5_(pred_format, v1, v2, v3, v4, v5, on_failure)\ + GTEST_ASSERT_(pred_format(#v1, #v2, #v3, #v4, #v5, v1, v2, v3, v4, v5), \ + on_failure) + +// Internal macro for implementing {EXPECT|ASSERT}_PRED5. Don't use +// this in your code. +#define GTEST_PRED5_(pred, v1, v2, v3, v4, v5, on_failure)\ + GTEST_ASSERT_(::testing::AssertPred5Helper(#pred, \ + #v1, \ + #v2, \ + #v3, \ + #v4, \ + #v5, \ + pred, \ + v1, \ + v2, \ + v3, \ + v4, \ + v5), on_failure) + +// 5-ary predicate assertion macros. +#define EXPECT_PRED_FORMAT5(pred_format, v1, v2, v3, v4, v5) \ + GTEST_PRED_FORMAT5_(pred_format, v1, v2, v3, v4, v5, GTEST_NONFATAL_FAILURE_) +#define EXPECT_PRED5(pred, v1, v2, v3, v4, v5) \ + GTEST_PRED5_(pred, v1, v2, v3, v4, v5, GTEST_NONFATAL_FAILURE_) +#define ASSERT_PRED_FORMAT5(pred_format, v1, v2, v3, v4, v5) \ + GTEST_PRED_FORMAT5_(pred_format, v1, v2, v3, v4, v5, GTEST_FATAL_FAILURE_) +#define ASSERT_PRED5(pred, v1, v2, v3, v4, v5) \ + GTEST_PRED5_(pred, v1, v2, v3, v4, v5, GTEST_FATAL_FAILURE_) + + + +#endif // GTEST_INCLUDE_GTEST_GTEST_PRED_IMPL_H_ + +// Macros for testing equalities and inequalities. +// +// * {ASSERT|EXPECT}_EQ(expected, actual): Tests that expected == actual +// * {ASSERT|EXPECT}_NE(v1, v2): Tests that v1 != v2 +// * {ASSERT|EXPECT}_LT(v1, v2): Tests that v1 < v2 +// * {ASSERT|EXPECT}_LE(v1, v2): Tests that v1 <= v2 +// * {ASSERT|EXPECT}_GT(v1, v2): Tests that v1 > v2 +// * {ASSERT|EXPECT}_GE(v1, v2): Tests that v1 >= v2 +// +// When they are not, Google Test prints both the tested expressions and +// their actual values. The values must be compatible built-in types, +// or you will get a compiler error. By "compatible" we mean that the +// values can be compared by the respective operator. +// +// Note: +// +// 1. It is possible to make a user-defined type work with +// {ASSERT|EXPECT}_??(), but that requires overloading the +// comparison operators and is thus discouraged by the Google C++ +// Usage Guide. Therefore, you are advised to use the +// {ASSERT|EXPECT}_TRUE() macro to assert that two objects are +// equal. +// +// 2. The {ASSERT|EXPECT}_??() macros do pointer comparisons on +// pointers (in particular, C strings). Therefore, if you use it +// with two C strings, you are testing how their locations in memory +// are related, not how their content is related. To compare two C +// strings by content, use {ASSERT|EXPECT}_STR*(). +// +// 3. {ASSERT|EXPECT}_EQ(expected, actual) is preferred to +// {ASSERT|EXPECT}_TRUE(expected == actual), as the former tells you +// what the actual value is when it fails, and similarly for the +// other comparisons. +// +// 4. Do not depend on the order in which {ASSERT|EXPECT}_??() +// evaluate their arguments, which is undefined. +// +// 5. These macros evaluate their arguments exactly once. +// +// Examples: +// +// EXPECT_NE(5, Foo()); +// EXPECT_EQ(NULL, a_pointer); +// ASSERT_LT(i, array_size); +// ASSERT_GT(records.size(), 0) << "There is no record left."; + +#define EXPECT_EQ(expected, actual) \ + EXPECT_PRED_FORMAT2(::testing::internal:: \ + EqHelper::Compare, \ + expected, actual) +#define EXPECT_NE(expected, actual) \ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperNE, expected, actual) +#define EXPECT_LE(val1, val2) \ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperLE, val1, val2) +#define EXPECT_LT(val1, val2) \ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperLT, val1, val2) +#define EXPECT_GE(val1, val2) \ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperGE, val1, val2) +#define EXPECT_GT(val1, val2) \ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperGT, val1, val2) + +#define GTEST_ASSERT_EQ(expected, actual) \ + ASSERT_PRED_FORMAT2(::testing::internal:: \ + EqHelper::Compare, \ + expected, actual) +#define GTEST_ASSERT_NE(val1, val2) \ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperNE, val1, val2) +#define GTEST_ASSERT_LE(val1, val2) \ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperLE, val1, val2) +#define GTEST_ASSERT_LT(val1, val2) \ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperLT, val1, val2) +#define GTEST_ASSERT_GE(val1, val2) \ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperGE, val1, val2) +#define GTEST_ASSERT_GT(val1, val2) \ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperGT, val1, val2) + +// Define macro GTEST_DONT_DEFINE_ASSERT_XY to 1 to omit the definition of +// ASSERT_XY(), which clashes with some users' own code. + +#if !GTEST_DONT_DEFINE_ASSERT_EQ +# define ASSERT_EQ(val1, val2) GTEST_ASSERT_EQ(val1, val2) +#endif + +#if !GTEST_DONT_DEFINE_ASSERT_NE +# define ASSERT_NE(val1, val2) GTEST_ASSERT_NE(val1, val2) +#endif + +#if !GTEST_DONT_DEFINE_ASSERT_LE +# define ASSERT_LE(val1, val2) GTEST_ASSERT_LE(val1, val2) +#endif + +#if !GTEST_DONT_DEFINE_ASSERT_LT +# define ASSERT_LT(val1, val2) GTEST_ASSERT_LT(val1, val2) +#endif + +#if !GTEST_DONT_DEFINE_ASSERT_GE +# define ASSERT_GE(val1, val2) GTEST_ASSERT_GE(val1, val2) +#endif + +#if !GTEST_DONT_DEFINE_ASSERT_GT +# define ASSERT_GT(val1, val2) GTEST_ASSERT_GT(val1, val2) +#endif + +// C-string Comparisons. All tests treat NULL and any non-NULL string +// as different. Two NULLs are equal. +// +// * {ASSERT|EXPECT}_STREQ(s1, s2): Tests that s1 == s2 +// * {ASSERT|EXPECT}_STRNE(s1, s2): Tests that s1 != s2 +// * {ASSERT|EXPECT}_STRCASEEQ(s1, s2): Tests that s1 == s2, ignoring case +// * {ASSERT|EXPECT}_STRCASENE(s1, s2): Tests that s1 != s2, ignoring case +// +// For wide or narrow string objects, you can use the +// {ASSERT|EXPECT}_??() macros. +// +// Don't depend on the order in which the arguments are evaluated, +// which is undefined. +// +// These macros evaluate their arguments exactly once. + +#define EXPECT_STREQ(expected, actual) \ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperSTREQ, expected, actual) +#define EXPECT_STRNE(s1, s2) \ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperSTRNE, s1, s2) +#define EXPECT_STRCASEEQ(expected, actual) \ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperSTRCASEEQ, expected, actual) +#define EXPECT_STRCASENE(s1, s2)\ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperSTRCASENE, s1, s2) + +#define ASSERT_STREQ(expected, actual) \ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperSTREQ, expected, actual) +#define ASSERT_STRNE(s1, s2) \ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperSTRNE, s1, s2) +#define ASSERT_STRCASEEQ(expected, actual) \ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperSTRCASEEQ, expected, actual) +#define ASSERT_STRCASENE(s1, s2)\ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperSTRCASENE, s1, s2) + +// Macros for comparing floating-point numbers. +// +// * {ASSERT|EXPECT}_FLOAT_EQ(expected, actual): +// Tests that two float values are almost equal. +// * {ASSERT|EXPECT}_DOUBLE_EQ(expected, actual): +// Tests that two double values are almost equal. +// * {ASSERT|EXPECT}_NEAR(v1, v2, abs_error): +// Tests that v1 and v2 are within the given distance to each other. +// +// Google Test uses ULP-based comparison to automatically pick a default +// error bound that is appropriate for the operands. See the +// FloatingPoint template class in gtest-internal.h if you are +// interested in the implementation details. + +#define EXPECT_FLOAT_EQ(expected, actual)\ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperFloatingPointEQ, \ + expected, actual) + +#define EXPECT_DOUBLE_EQ(expected, actual)\ + EXPECT_PRED_FORMAT2(::testing::internal::CmpHelperFloatingPointEQ, \ + expected, actual) + +#define ASSERT_FLOAT_EQ(expected, actual)\ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperFloatingPointEQ, \ + expected, actual) + +#define ASSERT_DOUBLE_EQ(expected, actual)\ + ASSERT_PRED_FORMAT2(::testing::internal::CmpHelperFloatingPointEQ, \ + expected, actual) + +#define EXPECT_NEAR(val1, val2, abs_error)\ + EXPECT_PRED_FORMAT3(::testing::internal::DoubleNearPredFormat, \ + val1, val2, abs_error) + +#define ASSERT_NEAR(val1, val2, abs_error)\ + ASSERT_PRED_FORMAT3(::testing::internal::DoubleNearPredFormat, \ + val1, val2, abs_error) + +// These predicate format functions work on floating-point values, and +// can be used in {ASSERT|EXPECT}_PRED_FORMAT2*(), e.g. +// +// EXPECT_PRED_FORMAT2(testing::DoubleLE, Foo(), 5.0); + +// Asserts that val1 is less than, or almost equal to, val2. Fails +// otherwise. In particular, it fails if either val1 or val2 is NaN. +GTEST_API_ AssertionResult FloatLE(const char* expr1, const char* expr2, + float val1, float val2); +GTEST_API_ AssertionResult DoubleLE(const char* expr1, const char* expr2, + double val1, double val2); + + +#if GTEST_OS_WINDOWS + +// Macros that test for HRESULT failure and success, these are only useful +// on Windows, and rely on Windows SDK macros and APIs to compile. +// +// * {ASSERT|EXPECT}_HRESULT_{SUCCEEDED|FAILED}(expr) +// +// When expr unexpectedly fails or succeeds, Google Test prints the +// expected result and the actual result with both a human-readable +// string representation of the error, if available, as well as the +// hex result code. +# define EXPECT_HRESULT_SUCCEEDED(expr) \ + EXPECT_PRED_FORMAT1(::testing::internal::IsHRESULTSuccess, (expr)) + +# define ASSERT_HRESULT_SUCCEEDED(expr) \ + ASSERT_PRED_FORMAT1(::testing::internal::IsHRESULTSuccess, (expr)) + +# define EXPECT_HRESULT_FAILED(expr) \ + EXPECT_PRED_FORMAT1(::testing::internal::IsHRESULTFailure, (expr)) + +# define ASSERT_HRESULT_FAILED(expr) \ + ASSERT_PRED_FORMAT1(::testing::internal::IsHRESULTFailure, (expr)) + +#endif // GTEST_OS_WINDOWS + +// Macros that execute statement and check that it doesn't generate new fatal +// failures in the current thread. +// +// * {ASSERT|EXPECT}_NO_FATAL_FAILURE(statement); +// +// Examples: +// +// EXPECT_NO_FATAL_FAILURE(Process()); +// ASSERT_NO_FATAL_FAILURE(Process()) << "Process() failed"; +// +#define ASSERT_NO_FATAL_FAILURE(statement) \ + GTEST_TEST_NO_FATAL_FAILURE_(statement, GTEST_FATAL_FAILURE_) +#define EXPECT_NO_FATAL_FAILURE(statement) \ + GTEST_TEST_NO_FATAL_FAILURE_(statement, GTEST_NONFATAL_FAILURE_) + +// Causes a trace (including the source file path, the current line +// number, and the given message) to be included in every test failure +// message generated by code in the current scope. The effect is +// undone when the control leaves the current scope. +// +// The message argument can be anything streamable to std::ostream. +// +// In the implementation, we include the current line number as part +// of the dummy variable name, thus allowing multiple SCOPED_TRACE()s +// to appear in the same block - as long as they are on different +// lines. +#define SCOPED_TRACE(message) \ + ::testing::internal::ScopedTrace GTEST_CONCAT_TOKEN_(gtest_trace_, __LINE__)(\ + __FILE__, __LINE__, ::testing::Message() << (message)) + +// Compile-time assertion for type equality. +// StaticAssertTypeEq() compiles iff type1 and type2 are +// the same type. The value it returns is not interesting. +// +// Instead of making StaticAssertTypeEq a class template, we make it a +// function template that invokes a helper class template. This +// prevents a user from misusing StaticAssertTypeEq by +// defining objects of that type. +// +// CAVEAT: +// +// When used inside a method of a class template, +// StaticAssertTypeEq() is effective ONLY IF the method is +// instantiated. For example, given: +// +// template class Foo { +// public: +// void Bar() { testing::StaticAssertTypeEq(); } +// }; +// +// the code: +// +// void Test1() { Foo foo; } +// +// will NOT generate a compiler error, as Foo::Bar() is never +// actually instantiated. Instead, you need: +// +// void Test2() { Foo foo; foo.Bar(); } +// +// to cause a compiler error. +template +bool StaticAssertTypeEq() { + (void)internal::StaticAssertTypeEqHelper(); + return true; +} + +// Defines a test. +// +// The first parameter is the name of the test case, and the second +// parameter is the name of the test within the test case. +// +// The convention is to end the test case name with "Test". For +// example, a test case for the Foo class can be named FooTest. +// +// Test code should appear between braces after an invocation of +// this macro. Example: +// +// TEST(FooTest, InitializesCorrectly) { +// Foo foo; +// EXPECT_TRUE(foo.StatusIsOK()); +// } + +// Note that we call GetTestTypeId() instead of GetTypeId< +// ::testing::Test>() here to get the type ID of testing::Test. This +// is to work around a suspected linker bug when using Google Test as +// a framework on Mac OS X. The bug causes GetTypeId< +// ::testing::Test>() to return different values depending on whether +// the call is from the Google Test framework itself or from user test +// code. GetTestTypeId() is guaranteed to always return the same +// value, as it always calls GetTypeId<>() from the Google Test +// framework. +#define GTEST_TEST(test_case_name, test_name)\ + GTEST_TEST_(test_case_name, test_name, \ + ::testing::Test, ::testing::internal::GetTestTypeId()) + +// Define this macro to 1 to omit the definition of TEST(), which +// is a generic name and clashes with some other libraries. +#if !GTEST_DONT_DEFINE_TEST +# define TEST(test_case_name, test_name) GTEST_TEST(test_case_name, test_name) +#endif + +// Defines a test that uses a test fixture. +// +// The first parameter is the name of the test fixture class, which +// also doubles as the test case name. The second parameter is the +// name of the test within the test case. +// +// A test fixture class must be declared earlier. The user should put +// his test code between braces after using this macro. Example: +// +// class FooTest : public testing::Test { +// protected: +// virtual void SetUp() { b_.AddElement(3); } +// +// Foo a_; +// Foo b_; +// }; +// +// TEST_F(FooTest, InitializesCorrectly) { +// EXPECT_TRUE(a_.StatusIsOK()); +// } +// +// TEST_F(FooTest, ReturnsElementCountCorrectly) { +// EXPECT_EQ(0, a_.size()); +// EXPECT_EQ(1, b_.size()); +// } + +#define TEST_F(test_fixture, test_name)\ + GTEST_TEST_(test_fixture, test_name, test_fixture, \ + ::testing::internal::GetTypeId()) + +} // namespace testing + +// Use this function in main() to run all tests. It returns 0 if all +// tests are successful, or 1 otherwise. +// +// RUN_ALL_TESTS() should be invoked after the command line has been +// parsed by InitGoogleTest(). +// +// This function was formerly a macro; thus, it is in the global +// namespace and has an all-caps name. +int RUN_ALL_TESTS() GTEST_MUST_USE_RESULT_; + +inline int RUN_ALL_TESTS() { + return ::testing::UnitTest::GetInstance()->Run(); +} + +#endif // GTEST_INCLUDE_GTEST_GTEST_H_ diff --git a/third-party/rapidjson/document.h b/third-party/rapidjson/document.h deleted file mode 100644 index 83d95a33d08..00000000000 --- a/third-party/rapidjson/document.h +++ /dev/null @@ -1,821 +0,0 @@ -#ifndef RAPIDJSON_DOCUMENT_H_ -#define RAPIDJSON_DOCUMENT_H_ - -#include "reader.h" -#include "internal/strfunc.h" -#include // placement new - -#ifdef _MSC_VER -#pragma warning(push) -#pragma warning(disable : 4127) // conditional expression is constant -#endif - -namespace rapidjson { - -/////////////////////////////////////////////////////////////////////////////// -// GenericValue - -//! Represents a JSON value. Use Value for UTF8 encoding and default allocator. -/*! - A JSON value can be one of 7 types. This class is a variant type supporting - these types. - - Use the Value if UTF8 and default allocator - - \tparam Encoding Encoding of the value. (Even non-string values need to have the same encoding in a document) - \tparam Allocator Allocator type for allocating memory of object, array and string. -*/ -#pragma pack (push, 4) -template > -class GenericValue { -public: - //! Name-value pair in an object. - struct Member { - GenericValue name; //!< name of member (must be a string) - GenericValue value; //!< value of member. - }; - - typedef Encoding EncodingType; //!< Encoding type from template parameter. - typedef Allocator AllocatorType; //!< Allocator type from template parameter. - typedef typename Encoding::Ch Ch; //!< Character type derived from Encoding. - typedef Member* MemberIterator; //!< Member iterator for iterating in object. - typedef const Member* ConstMemberIterator; //!< Constant member iterator for iterating in object. - typedef GenericValue* ValueIterator; //!< Value iterator for iterating in array. - typedef const GenericValue* ConstValueIterator; //!< Constant value iterator for iterating in array. - - //!@name Constructors and destructor. - //@{ - - //! Default constructor creates a null value. - GenericValue() : flags_(kNullFlag) {} - - //! Copy constructor is not permitted. -private: - GenericValue(const GenericValue& rhs); - -public: - - //! Constructor with JSON value type. - /*! This creates a Value of specified type with default content. - \param type Type of the value. - \note Default content for number is zero. - */ - GenericValue(Type type) { - static const unsigned defaultFlags[7] = { - kNullFlag, kFalseFlag, kTrueFlag, kObjectFlag, kArrayFlag, kConstStringFlag, - kNumberFlag | kIntFlag | kUintFlag | kInt64Flag | kUint64Flag | kDoubleFlag - }; - RAPIDJSON_ASSERT(type <= kNumberType); - flags_ = defaultFlags[type]; - memset(&data_, 0, sizeof(data_)); - } - - //! Constructor for boolean value. - GenericValue(bool b) : flags_(b ? kTrueFlag : kFalseFlag) {} - - //! Constructor for int value. - GenericValue(int i) : flags_(kNumberIntFlag) { - data_.n.i64 = i; - if (i >= 0) - flags_ |= kUintFlag | kUint64Flag; - } - - //! Constructor for unsigned value. - GenericValue(unsigned u) : flags_(kNumberUintFlag) { - data_.n.u64 = u; - if (!(u & 0x80000000)) - flags_ |= kIntFlag | kInt64Flag; - } - - //! Constructor for int64_t value. - GenericValue(int64_t i64) : flags_(kNumberInt64Flag) { - data_.n.i64 = i64; - if (i64 >= 0) { - flags_ |= kNumberUint64Flag; - if (!(i64 & 0xFFFFFFFF00000000LL)) - flags_ |= kUintFlag; - if (!(i64 & 0xFFFFFFFF80000000LL)) - flags_ |= kIntFlag; - } - else if (i64 >= -2147483648LL) - flags_ |= kIntFlag; - } - - //! Constructor for uint64_t value. - GenericValue(uint64_t u64) : flags_(kNumberUint64Flag) { - data_.n.u64 = u64; - if (!(u64 & 0x8000000000000000ULL)) - flags_ |= kInt64Flag; - if (!(u64 & 0xFFFFFFFF00000000ULL)) - flags_ |= kUintFlag; - if (!(u64 & 0xFFFFFFFF80000000ULL)) - flags_ |= kIntFlag; - } - - //! Constructor for double value. - GenericValue(double d) : flags_(kNumberDoubleFlag) { data_.n.d = d; } - - //! Constructor for constant string (i.e. do not make a copy of string) - GenericValue(const Ch* s, SizeType length) { - RAPIDJSON_ASSERT(s != NULL); - flags_ = kConstStringFlag; - data_.s.str = s; - data_.s.length = length; - } - - //! Constructor for constant string (i.e. do not make a copy of string) - GenericValue(const Ch* s) { SetStringRaw(s, internal::StrLen(s)); } - - //! Constructor for copy-string (i.e. do make a copy of string) - GenericValue(const Ch* s, SizeType length, Allocator& allocator) { SetStringRaw(s, length, allocator); } - - //! Constructor for copy-string (i.e. do make a copy of string) - GenericValue(const Ch*s, Allocator& allocator) { SetStringRaw(s, internal::StrLen(s), allocator); } - - //! Destructor. - /*! Need to destruct elements of array, members of object, or copy-string. - */ - ~GenericValue() { - if (Allocator::kNeedFree) { // Shortcut by Allocator's trait - switch(flags_) { - case kArrayFlag: - for (GenericValue* v = data_.a.elements; v != data_.a.elements + data_.a.size; ++v) - v->~GenericValue(); - Allocator::Free(data_.a.elements); - break; - - case kObjectFlag: - for (Member* m = data_.o.members; m != data_.o.members + data_.o.size; ++m) { - m->name.~GenericValue(); - m->value.~GenericValue(); - } - Allocator::Free(data_.o.members); - break; - - case kCopyStringFlag: - Allocator::Free(const_cast(data_.s.str)); - break; - } - } - } - - //@} - - //!@name Assignment operators - //@{ - - //! Assignment with move semantics. - /*! \param rhs Source of the assignment. It will become a null value after assignment. - */ - GenericValue& operator=(GenericValue& rhs) { - RAPIDJSON_ASSERT(this != &rhs); - this->~GenericValue(); - memcpy(this, &rhs, sizeof(GenericValue)); - rhs.flags_ = kNullFlag; - return *this; - } - - //! Assignment with primitive types. - /*! \tparam T Either Type, int, unsigned, int64_t, uint64_t, const Ch* - \param value The value to be assigned. - */ - template - GenericValue& operator=(T value) { - this->~GenericValue(); - new (this) GenericValue(value); - return *this; - } - //@} - - //!@name Type - //@{ - - Type GetType() const { return static_cast(flags_ & kTypeMask); } - bool IsNull() const { return flags_ == kNullFlag; } - bool IsFalse() const { return flags_ == kFalseFlag; } - bool IsTrue() const { return flags_ == kTrueFlag; } - bool IsBool() const { return (flags_ & kBoolFlag) != 0; } - bool IsObject() const { return flags_ == kObjectFlag; } - bool IsArray() const { return flags_ == kArrayFlag; } - bool IsNumber() const { return (flags_ & kNumberFlag) != 0; } - bool IsInt() const { return (flags_ & kIntFlag) != 0; } - bool IsUint() const { return (flags_ & kUintFlag) != 0; } - bool IsInt64() const { return (flags_ & kInt64Flag) != 0; } - bool IsUint64() const { return (flags_ & kUint64Flag) != 0; } - bool IsDouble() const { return (flags_ & kDoubleFlag) != 0; } - bool IsString() const { return (flags_ & kStringFlag) != 0; } - - //@} - - //!@name Null - //@{ - - GenericValue& SetNull() { this->~GenericValue(); new (this) GenericValue(); return *this; } - - //@} - - //!@name Bool - //@{ - - bool GetBool() const { RAPIDJSON_ASSERT(IsBool()); return flags_ == kTrueFlag; } - GenericValue& SetBool(bool b) { this->~GenericValue(); new (this) GenericValue(b); return *this; } - - //@} - - //!@name Object - //@{ - - //! Set this value as an empty object. - GenericValue& SetObject() { this->~GenericValue(); new (this) GenericValue(kObjectType); return *this; } - - //! Get the value associated with the object's name. - GenericValue& operator[](const Ch* name) { - if (Member* member = FindMember(name)) - return member->value; - else { - static GenericValue NullValue; - return NullValue; - } - } - const GenericValue& operator[](const Ch* name) const { return const_cast(*this)[name]; } - - //! Member iterators. - ConstMemberIterator MemberBegin() const { RAPIDJSON_ASSERT(IsObject()); return data_.o.members; } - ConstMemberIterator MemberEnd() const { RAPIDJSON_ASSERT(IsObject()); return data_.o.members + data_.o.size; } - MemberIterator MemberBegin() { RAPIDJSON_ASSERT(IsObject()); return data_.o.members; } - MemberIterator MemberEnd() { RAPIDJSON_ASSERT(IsObject()); return data_.o.members + data_.o.size; } - - //! Check whether a member exists in the object. - bool HasMember(const Ch* name) const { return FindMember(name) != 0; } - - //! Add a member (name-value pair) to the object. - /*! \param name A string value as name of member. - \param value Value of any type. - \param allocator Allocator for reallocating memory. - \return The value itself for fluent API. - \note The ownership of name and value will be transfered to this object if success. - */ - GenericValue& AddMember(GenericValue& name, GenericValue& value, Allocator& allocator) { - RAPIDJSON_ASSERT(IsObject()); - RAPIDJSON_ASSERT(name.IsString()); - Object& o = data_.o; - if (o.size >= o.capacity) { - if (o.capacity == 0) { - o.capacity = kDefaultObjectCapacity; - o.members = (Member*)allocator.Malloc(o.capacity * sizeof(Member)); - } - else { - SizeType oldCapacity = o.capacity; - o.capacity *= 2; - o.members = (Member*)allocator.Realloc(o.members, oldCapacity * sizeof(Member), o.capacity * sizeof(Member)); - } - } - o.members[o.size].name.RawAssign(name); - o.members[o.size].value.RawAssign(value); - o.size++; - return *this; - } - - GenericValue& AddMember(const Ch* name, Allocator& nameAllocator, GenericValue& value, Allocator& allocator) { - GenericValue n(name, internal::StrLen(name), nameAllocator); - return AddMember(n, value, allocator); - } - - GenericValue& AddMember(const Ch* name, GenericValue& value, Allocator& allocator) { - GenericValue n(name, internal::StrLen(name)); - return AddMember(n, value, allocator); - } - - template - GenericValue& AddMember(const Ch* name, T value, Allocator& allocator) { - GenericValue n(name, internal::StrLen(name)); - GenericValue v(value); - return AddMember(n, v, allocator); - } - - //! Remove a member in object by its name. - /*! \param name Name of member to be removed. - \return Whether the member existed. - \note Removing member is implemented by moving the last member. So the ordering of members is changed. - */ - bool RemoveMember(const Ch* name) { - RAPIDJSON_ASSERT(IsObject()); - if (Member* m = FindMember(name)) { - RAPIDJSON_ASSERT(data_.o.size > 0); - RAPIDJSON_ASSERT(data_.o.members != 0); - - Member* last = data_.o.members + (data_.o.size - 1); - if (data_.o.size > 1 && m != last) { - // Move the last one to this place - m->name = last->name; - m->value = last->value; - } - else { - // Only one left, just destroy - m->name.~GenericValue(); - m->value.~GenericValue(); - } - --data_.o.size; - return true; - } - return false; - } - - //@} - - //!@name Array - //@{ - - //! Set this value as an empty array. - GenericValue& SetArray() { this->~GenericValue(); new (this) GenericValue(kArrayType); return *this; } - - //! Get the number of elements in array. - SizeType Size() const { RAPIDJSON_ASSERT(IsArray()); return data_.a.size; } - - //! Get the capacity of array. - SizeType Capacity() const { RAPIDJSON_ASSERT(IsArray()); return data_.a.capacity; } - - //! Check whether the array is empty. - bool Empty() const { RAPIDJSON_ASSERT(IsArray()); return data_.a.size == 0; } - - //! Remove all elements in the array. - /*! This function do not deallocate memory in the array, i.e. the capacity is unchanged. - */ - void Clear() { - RAPIDJSON_ASSERT(IsArray()); - for (SizeType i = 0; i < data_.a.size; ++i) - data_.a.elements[i].~GenericValue(); - data_.a.size = 0; - } - - //! Get an element from array by index. - /*! \param index Zero-based index of element. - \note -\code -Value a(kArrayType); -a.PushBack(123); -int x = a[0].GetInt(); // Error: operator[ is ambiguous, as 0 also mean a null pointer of const char* type. -int y = a[SizeType(0)].GetInt(); // Cast to SizeType will work. -int z = a[0u].GetInt(); // This works too. -\endcode - */ - GenericValue& operator[](SizeType index) { - RAPIDJSON_ASSERT(IsArray()); - RAPIDJSON_ASSERT(index < data_.a.size); - return data_.a.elements[index]; - } - const GenericValue& operator[](SizeType index) const { return const_cast(*this)[index]; } - - //! Element iterator - ValueIterator Begin() { RAPIDJSON_ASSERT(IsArray()); return data_.a.elements; } - ValueIterator End() { RAPIDJSON_ASSERT(IsArray()); return data_.a.elements + data_.a.size; } - ConstValueIterator Begin() const { return const_cast(*this).Begin(); } - ConstValueIterator End() const { return const_cast(*this).End(); } - - //! Request the array to have enough capacity to store elements. - /*! \param newCapacity The capacity that the array at least need to have. - \param allocator The allocator for allocating memory. It must be the same one use previously. - \return The value itself for fluent API. - */ - GenericValue& Reserve(SizeType newCapacity, Allocator &allocator) { - RAPIDJSON_ASSERT(IsArray()); - if (newCapacity > data_.a.capacity) { - data_.a.elements = (GenericValue*)allocator.Realloc(data_.a.elements, data_.a.capacity * sizeof(GenericValue), newCapacity * sizeof(GenericValue)); - data_.a.capacity = newCapacity; - } - return *this; - } - - //! Append a value at the end of the array. - /*! \param value The value to be appended. - \param allocator The allocator for allocating memory. It must be the same one use previously. - \return The value itself for fluent API. - \note The ownership of the value will be transfered to this object if success. - \note If the number of elements to be appended is known, calls Reserve() once first may be more efficient. - */ - GenericValue& PushBack(GenericValue& value, Allocator& allocator) { - RAPIDJSON_ASSERT(IsArray()); - if (data_.a.size >= data_.a.capacity) - Reserve(data_.a.capacity == 0 ? kDefaultArrayCapacity : data_.a.capacity * 2, allocator); - data_.a.elements[data_.a.size++].RawAssign(value); - return *this; - } - - template - GenericValue& PushBack(T value, Allocator& allocator) { - GenericValue v(value); - return PushBack(v, allocator); - } - - //! Remove the last element in the array. - GenericValue& PopBack() { - RAPIDJSON_ASSERT(IsArray()); - RAPIDJSON_ASSERT(!Empty()); - data_.a.elements[--data_.a.size].~GenericValue(); - return *this; - } - //@} - - //!@name Number - //@{ - - int GetInt() const { RAPIDJSON_ASSERT(flags_ & kIntFlag); return data_.n.i.i; } - unsigned GetUint() const { RAPIDJSON_ASSERT(flags_ & kUintFlag); return data_.n.u.u; } - int64_t GetInt64() const { RAPIDJSON_ASSERT(flags_ & kInt64Flag); return data_.n.i64; } - uint64_t GetUint64() const { RAPIDJSON_ASSERT(flags_ & kUint64Flag); return data_.n.u64; } - - double GetDouble() const { - RAPIDJSON_ASSERT(IsNumber()); - if ((flags_ & kDoubleFlag) != 0) return data_.n.d; // exact type, no conversion. - if ((flags_ & kIntFlag) != 0) return data_.n.i.i; // int -> double - if ((flags_ & kUintFlag) != 0) return data_.n.u.u; // unsigned -> double - if ((flags_ & kInt64Flag) != 0) return (double)data_.n.i64; // int64_t -> double (may lose precision) - RAPIDJSON_ASSERT((flags_ & kUint64Flag) != 0); return (double)data_.n.u64; // uint64_t -> double (may lose precision) - } - - GenericValue& SetInt(int i) { this->~GenericValue(); new (this) GenericValue(i); return *this; } - GenericValue& SetUint(unsigned u) { this->~GenericValue(); new (this) GenericValue(u); return *this; } - GenericValue& SetInt64(int64_t i64) { this->~GenericValue(); new (this) GenericValue(i64); return *this; } - GenericValue& SetUint64(uint64_t u64) { this->~GenericValue(); new (this) GenericValue(u64); return *this; } - GenericValue& SetDouble(double d) { this->~GenericValue(); new (this) GenericValue(d); return *this; } - - //@} - - //!@name String - //@{ - - const Ch* GetString() const { RAPIDJSON_ASSERT(IsString()); return data_.s.str; } - - //! Get the length of string. - /*! Since rapidjson permits "\u0000" in the json string, strlen(v.GetString()) may not equal to v.GetStringLength(). - */ - SizeType GetStringLength() const { RAPIDJSON_ASSERT(IsString()); return data_.s.length; } - - //! Set this value as a string without copying source string. - /*! This version has better performance with supplied length, and also support string containing null character. - \param s source string pointer. - \param length The length of source string, excluding the trailing null terminator. - \return The value itself for fluent API. - */ - GenericValue& SetString(const Ch* s, SizeType length) { this->~GenericValue(); SetStringRaw(s, length); return *this; } - - //! Set this value as a string without copying source string. - /*! \param s source string pointer. - \return The value itself for fluent API. - */ - GenericValue& SetString(const Ch* s) { return SetString(s, internal::StrLen(s)); } - - //! Set this value as a string by copying from source string. - /*! This version has better performance with supplied length, and also support string containing null character. - \param s source string. - \param length The length of source string, excluding the trailing null terminator. - \param allocator Allocator for allocating copied buffer. Commonly use document.GetAllocator(). - \return The value itself for fluent API. - */ - GenericValue& SetString(const Ch* s, SizeType length, Allocator& allocator) { this->~GenericValue(); SetStringRaw(s, length, allocator); return *this; } - - //! Set this value as a string by copying from source string. - /*! \param s source string. - \param allocator Allocator for allocating copied buffer. Commonly use document.GetAllocator(). - \return The value itself for fluent API. - */ - GenericValue& SetString(const Ch* s, Allocator& allocator) { SetString(s, internal::StrLen(s), allocator); return *this; } - - //@} - - //! Generate events of this value to a Handler. - /*! This function adopts the GoF visitor pattern. - Typical usage is to output this JSON value as JSON text via Writer, which is a Handler. - It can also be used to deep clone this value via GenericDocument, which is also a Handler. - \tparam Handler type of handler. - \param handler An object implementing concept Handler. - */ - template - const GenericValue& Accept(Handler& handler) const { - switch(GetType()) { - case kNullType: handler.Null(); break; - case kFalseType: handler.Bool(false); break; - case kTrueType: handler.Bool(true); break; - - case kObjectType: - handler.StartObject(); - for (Member* m = data_.o.members; m != data_.o.members + data_.o.size; ++m) { - handler.String(m->name.data_.s.str, m->name.data_.s.length, false); - m->value.Accept(handler); - } - handler.EndObject(data_.o.size); - break; - - case kArrayType: - handler.StartArray(); - for (GenericValue* v = data_.a.elements; v != data_.a.elements + data_.a.size; ++v) - v->Accept(handler); - handler.EndArray(data_.a.size); - break; - - case kStringType: - handler.String(data_.s.str, data_.s.length, false); - break; - - case kNumberType: - if (IsInt()) handler.Int(data_.n.i.i); - else if (IsUint()) handler.Uint(data_.n.u.u); - else if (IsInt64()) handler.Int64(data_.n.i64); - else if (IsUint64()) handler.Uint64(data_.n.u64); - else handler.Double(data_.n.d); - break; - } - return *this; - } - -private: - template - friend class GenericDocument; - - enum { - kBoolFlag = 0x100, - kNumberFlag = 0x200, - kIntFlag = 0x400, - kUintFlag = 0x800, - kInt64Flag = 0x1000, - kUint64Flag = 0x2000, - kDoubleFlag = 0x4000, - kStringFlag = 0x100000, - kCopyFlag = 0x200000, - - // Initial flags of different types. - kNullFlag = kNullType, - kTrueFlag = kTrueType | kBoolFlag, - kFalseFlag = kFalseType | kBoolFlag, - kNumberIntFlag = kNumberType | kNumberFlag | kIntFlag | kInt64Flag, - kNumberUintFlag = kNumberType | kNumberFlag | kUintFlag | kUint64Flag | kInt64Flag, - kNumberInt64Flag = kNumberType | kNumberFlag | kInt64Flag, - kNumberUint64Flag = kNumberType | kNumberFlag | kUint64Flag, - kNumberDoubleFlag = kNumberType | kNumberFlag | kDoubleFlag, - kConstStringFlag = kStringType | kStringFlag, - kCopyStringFlag = kStringType | kStringFlag | kCopyFlag, - kObjectFlag = kObjectType, - kArrayFlag = kArrayType, - - kTypeMask = 0xFF // bitwise-and with mask of 0xFF can be optimized by compiler - }; - - static const SizeType kDefaultArrayCapacity = 16; - static const SizeType kDefaultObjectCapacity = 16; - - struct String { - const Ch* str; - SizeType length; - unsigned hashcode; //!< reserved - }; // 12 bytes in 32-bit mode, 16 bytes in 64-bit mode - - // By using proper binary layout, retrieval of different integer types do not need conversions. - union Number { -#if RAPIDJSON_ENDIAN == RAPIDJSON_LITTLEENDIAN - struct I { - int i; - char padding[4]; - }i; - struct U { - unsigned u; - char padding2[4]; - }u; -#else - struct I { - char padding[4]; - int i; - }i; - struct U { - char padding2[4]; - unsigned u; - }u; -#endif - int64_t i64; - uint64_t u64; - double d; - }; // 8 bytes - - struct Object { - Member* members; - SizeType size; - SizeType capacity; - }; // 12 bytes in 32-bit mode, 16 bytes in 64-bit mode - - struct Array { - GenericValue* elements; - SizeType size; - SizeType capacity; - }; // 12 bytes in 32-bit mode, 16 bytes in 64-bit mode - - union Data { - String s; - Number n; - Object o; - Array a; - }; // 12 bytes in 32-bit mode, 16 bytes in 64-bit mode - - //! Find member by name. - Member* FindMember(const Ch* name) { - RAPIDJSON_ASSERT(name); - RAPIDJSON_ASSERT(IsObject()); - - SizeType length = internal::StrLen(name); - - Object& o = data_.o; - for (Member* member = o.members; member != data_.o.members + data_.o.size; ++member) - if (length == member->name.data_.s.length && memcmp(member->name.data_.s.str, name, length * sizeof(Ch)) == 0) - return member; - - return 0; - } - const Member* FindMember(const Ch* name) const { return const_cast(*this).FindMember(name); } - - // Initialize this value as array with initial data, without calling destructor. - void SetArrayRaw(GenericValue* values, SizeType count, Allocator& alloctaor) { - flags_ = kArrayFlag; - data_.a.elements = (GenericValue*)alloctaor.Malloc(count * sizeof(GenericValue)); - memcpy(data_.a.elements, values, count * sizeof(GenericValue)); - data_.a.size = data_.a.capacity = count; - } - - //! Initialize this value as object with initial data, without calling destructor. - void SetObjectRaw(Member* members, SizeType count, Allocator& alloctaor) { - flags_ = kObjectFlag; - data_.o.members = (Member*)alloctaor.Malloc(count * sizeof(Member)); - memcpy(data_.o.members, members, count * sizeof(Member)); - data_.o.size = data_.o.capacity = count; - } - - //! Initialize this value as constant string, without calling destructor. - void SetStringRaw(const Ch* s, SizeType length) { - RAPIDJSON_ASSERT(s != NULL); - flags_ = kConstStringFlag; - data_.s.str = s; - data_.s.length = length; - } - - //! Initialize this value as copy string with initial data, without calling destructor. - void SetStringRaw(const Ch* s, SizeType length, Allocator& allocator) { - RAPIDJSON_ASSERT(s != NULL); - flags_ = kCopyStringFlag; - data_.s.str = (Ch *)allocator.Malloc((length + 1) * sizeof(Ch)); - data_.s.length = length; - memcpy(const_cast(data_.s.str), s, length * sizeof(Ch)); - const_cast(data_.s.str)[length] = '\0'; - } - - //! Assignment without calling destructor - void RawAssign(GenericValue& rhs) { - memcpy(this, &rhs, sizeof(GenericValue)); - rhs.flags_ = kNullFlag; - } - - Data data_; - unsigned flags_; -}; -#pragma pack (pop) - -//! Value with UTF8 encoding. -typedef GenericValue > Value; - -/////////////////////////////////////////////////////////////////////////////// -// GenericDocument - -//! A document for parsing JSON text as DOM. -/*! - \implements Handler - \tparam Encoding encoding for both parsing and string storage. - \tparam Alloactor allocator for allocating memory for the DOM, and the stack during parsing. -*/ -template > -class GenericDocument : public GenericValue { -public: - typedef typename Encoding::Ch Ch; //!< Character type derived from Encoding. - typedef GenericValue ValueType; //!< Value type of the document. - typedef Allocator AllocatorType; //!< Allocator type from template parameter. - - //! Constructor - /*! \param allocator Optional allocator for allocating stack memory. - \param stackCapacity Initial capacity of stack in bytes. - */ - GenericDocument(Allocator* allocator = 0, size_t stackCapacity = kDefaultStackCapacity) : stack_(allocator, stackCapacity), parseError_(0), errorOffset_(0) {} - - //! Parse JSON text from an input stream. - /*! \tparam parseFlags Combination of ParseFlag. - \param stream Input stream to be parsed. - \return The document itself for fluent API. - */ - template - GenericDocument& ParseStream(Stream& stream) { - ValueType::SetNull(); // Remove existing root if exist - GenericReader reader; - if (reader.template Parse(stream, *this)) { - RAPIDJSON_ASSERT(stack_.GetSize() == sizeof(ValueType)); // Got one and only one root object - this->RawAssign(*stack_.template Pop(1)); // Add this-> to prevent issue 13. - parseError_ = 0; - errorOffset_ = 0; - } - else { - parseError_ = reader.GetParseError(); - errorOffset_ = reader.GetErrorOffset(); - ClearStack(); - } - return *this; - } - - //! Parse JSON text from a mutable string. - /*! \tparam parseFlags Combination of ParseFlag. - \param str Mutable zero-terminated string to be parsed. - \return The document itself for fluent API. - */ - template - GenericDocument& ParseInsitu(Ch* str) { - GenericInsituStringStream s(str); - return ParseStream(s); - } - - //! Parse JSON text from a read-only string. - /*! \tparam parseFlags Combination of ParseFlag (must not contain kParseInsituFlag). - \param str Read-only zero-terminated string to be parsed. - */ - template - GenericDocument& Parse(const Ch* str) { - RAPIDJSON_ASSERT(!(parseFlags & kParseInsituFlag)); - GenericStringStream s(str); - return ParseStream(s); - } - - //! Whether a parse error was occured in the last parsing. - bool HasParseError() const { return parseError_ != 0; } - - //! Get the message of parsing error. - const char* GetParseError() const { return parseError_; } - - //! Get the offset in character of the parsing error. - size_t GetErrorOffset() const { return errorOffset_; } - - //! Get the allocator of this document. - Allocator& GetAllocator() { return stack_.GetAllocator(); } - - //! Get the capacity of stack in bytes. - size_t GetStackCapacity() const { return stack_.GetCapacity(); } - -private: - // Prohibit assignment - GenericDocument& operator=(const GenericDocument&); - - friend class GenericReader; // for Reader to call the following private handler functions - - // Implementation of Handler - void Null() { new (stack_.template Push()) ValueType(); } - void Bool(bool b) { new (stack_.template Push()) ValueType(b); } - void Int(int i) { new (stack_.template Push()) ValueType(i); } - void Uint(unsigned i) { new (stack_.template Push()) ValueType(i); } - void Int64(int64_t i) { new (stack_.template Push()) ValueType(i); } - void Uint64(uint64_t i) { new (stack_.template Push()) ValueType(i); } - void Double(double d) { new (stack_.template Push()) ValueType(d); } - - void String(const Ch* str, SizeType length, bool copy) { - if (copy) - new (stack_.template Push()) ValueType(str, length, GetAllocator()); - else - new (stack_.template Push()) ValueType(str, length); - } - - void StartObject() { new (stack_.template Push()) ValueType(kObjectType); } - - void EndObject(SizeType memberCount) { - typename ValueType::Member* members = stack_.template Pop(memberCount); - stack_.template Top()->SetObjectRaw(members, (SizeType)memberCount, GetAllocator()); - } - - void StartArray() { new (stack_.template Push()) ValueType(kArrayType); } - - void EndArray(SizeType elementCount) { - ValueType* elements = stack_.template Pop(elementCount); - stack_.template Top()->SetArrayRaw(elements, elementCount, GetAllocator()); - } - - void ClearStack() { - if (Allocator::kNeedFree) - while (stack_.GetSize() > 0) // Here assumes all elements in stack array are GenericValue (Member is actually 2 GenericValue objects) - (stack_.template Pop(1))->~ValueType(); - else - stack_.Clear(); - } - - static const size_t kDefaultStackCapacity = 1024; - internal::Stack stack_; - const char* parseError_; - size_t errorOffset_; -}; - -typedef GenericDocument > Document; - -} // namespace rapidjson - -#ifdef _MSC_VER -#pragma warning(pop) -#endif - -#endif // RAPIDJSON_DOCUMENT_H_ diff --git a/third-party/rapidjson/filestream.h b/third-party/rapidjson/filestream.h deleted file mode 100644 index 885894963f6..00000000000 --- a/third-party/rapidjson/filestream.h +++ /dev/null @@ -1,46 +0,0 @@ -#ifndef RAPIDJSON_FILESTREAM_H_ -#define RAPIDJSON_FILESTREAM_H_ - -#include - -namespace rapidjson { - -//! Wrapper of C file stream for input or output. -/*! - This simple wrapper does not check the validity of the stream. - \implements Stream -*/ -class FileStream { -public: - typedef char Ch; //!< Character type. Only support char. - - FileStream(FILE* fp) : fp_(fp), count_(0) { Read(); } - char Peek() const { return current_; } - char Take() { char c = current_; Read(); return c; } - size_t Tell() const { return count_; } - void Put(char c) { fputc(c, fp_); } - - // Not implemented - char* PutBegin() { return 0; } - size_t PutEnd(char*) { return 0; } - -private: - void Read() { - RAPIDJSON_ASSERT(fp_ != 0); - int c = fgetc(fp_); - if (c != EOF) { - current_ = (char)c; - count_++; - } - else - current_ = '\0'; - } - - FILE* fp_; - char current_; - size_t count_; -}; - -} // namespace rapidjson - -#endif // RAPIDJSON_FILESTREAM_H_ diff --git a/third-party/rapidjson/internal/pow10.h b/third-party/rapidjson/internal/pow10.h deleted file mode 100644 index bf3a9afb041..00000000000 --- a/third-party/rapidjson/internal/pow10.h +++ /dev/null @@ -1,54 +0,0 @@ -#ifndef RAPIDJSON_POW10_ -#define RAPIDJSON_POW10_ - -namespace rapidjson { -namespace internal { - -//! Computes integer powers of 10 in double (10.0^n). -/*! This function uses lookup table for fast and accurate results. - \param n positive/negative exponent. Must <= 308. - \return 10.0^n -*/ -inline double Pow10(int n) { - static const double e[] = { // 1e-308...1e308: 617 * 8 bytes = 4936 bytes - 1e-308,1e-307,1e-306,1e-305,1e-304,1e-303,1e-302,1e-301,1e-300, - 1e-299,1e-298,1e-297,1e-296,1e-295,1e-294,1e-293,1e-292,1e-291,1e-290,1e-289,1e-288,1e-287,1e-286,1e-285,1e-284,1e-283,1e-282,1e-281,1e-280, - 1e-279,1e-278,1e-277,1e-276,1e-275,1e-274,1e-273,1e-272,1e-271,1e-270,1e-269,1e-268,1e-267,1e-266,1e-265,1e-264,1e-263,1e-262,1e-261,1e-260, - 1e-259,1e-258,1e-257,1e-256,1e-255,1e-254,1e-253,1e-252,1e-251,1e-250,1e-249,1e-248,1e-247,1e-246,1e-245,1e-244,1e-243,1e-242,1e-241,1e-240, - 1e-239,1e-238,1e-237,1e-236,1e-235,1e-234,1e-233,1e-232,1e-231,1e-230,1e-229,1e-228,1e-227,1e-226,1e-225,1e-224,1e-223,1e-222,1e-221,1e-220, - 1e-219,1e-218,1e-217,1e-216,1e-215,1e-214,1e-213,1e-212,1e-211,1e-210,1e-209,1e-208,1e-207,1e-206,1e-205,1e-204,1e-203,1e-202,1e-201,1e-200, - 1e-199,1e-198,1e-197,1e-196,1e-195,1e-194,1e-193,1e-192,1e-191,1e-190,1e-189,1e-188,1e-187,1e-186,1e-185,1e-184,1e-183,1e-182,1e-181,1e-180, - 1e-179,1e-178,1e-177,1e-176,1e-175,1e-174,1e-173,1e-172,1e-171,1e-170,1e-169,1e-168,1e-167,1e-166,1e-165,1e-164,1e-163,1e-162,1e-161,1e-160, - 1e-159,1e-158,1e-157,1e-156,1e-155,1e-154,1e-153,1e-152,1e-151,1e-150,1e-149,1e-148,1e-147,1e-146,1e-145,1e-144,1e-143,1e-142,1e-141,1e-140, - 1e-139,1e-138,1e-137,1e-136,1e-135,1e-134,1e-133,1e-132,1e-131,1e-130,1e-129,1e-128,1e-127,1e-126,1e-125,1e-124,1e-123,1e-122,1e-121,1e-120, - 1e-119,1e-118,1e-117,1e-116,1e-115,1e-114,1e-113,1e-112,1e-111,1e-110,1e-109,1e-108,1e-107,1e-106,1e-105,1e-104,1e-103,1e-102,1e-101,1e-100, - 1e-99, 1e-98, 1e-97, 1e-96, 1e-95, 1e-94, 1e-93, 1e-92, 1e-91, 1e-90, 1e-89, 1e-88, 1e-87, 1e-86, 1e-85, 1e-84, 1e-83, 1e-82, 1e-81, 1e-80, - 1e-79, 1e-78, 1e-77, 1e-76, 1e-75, 1e-74, 1e-73, 1e-72, 1e-71, 1e-70, 1e-69, 1e-68, 1e-67, 1e-66, 1e-65, 1e-64, 1e-63, 1e-62, 1e-61, 1e-60, - 1e-59, 1e-58, 1e-57, 1e-56, 1e-55, 1e-54, 1e-53, 1e-52, 1e-51, 1e-50, 1e-49, 1e-48, 1e-47, 1e-46, 1e-45, 1e-44, 1e-43, 1e-42, 1e-41, 1e-40, - 1e-39, 1e-38, 1e-37, 1e-36, 1e-35, 1e-34, 1e-33, 1e-32, 1e-31, 1e-30, 1e-29, 1e-28, 1e-27, 1e-26, 1e-25, 1e-24, 1e-23, 1e-22, 1e-21, 1e-20, - 1e-19, 1e-18, 1e-17, 1e-16, 1e-15, 1e-14, 1e-13, 1e-12, 1e-11, 1e-10, 1e-9, 1e-8, 1e-7, 1e-6, 1e-5, 1e-4, 1e-3, 1e-2, 1e-1, 1e+0, - 1e+1, 1e+2, 1e+3, 1e+4, 1e+5, 1e+6, 1e+7, 1e+8, 1e+9, 1e+10, 1e+11, 1e+12, 1e+13, 1e+14, 1e+15, 1e+16, 1e+17, 1e+18, 1e+19, 1e+20, - 1e+21, 1e+22, 1e+23, 1e+24, 1e+25, 1e+26, 1e+27, 1e+28, 1e+29, 1e+30, 1e+31, 1e+32, 1e+33, 1e+34, 1e+35, 1e+36, 1e+37, 1e+38, 1e+39, 1e+40, - 1e+41, 1e+42, 1e+43, 1e+44, 1e+45, 1e+46, 1e+47, 1e+48, 1e+49, 1e+50, 1e+51, 1e+52, 1e+53, 1e+54, 1e+55, 1e+56, 1e+57, 1e+58, 1e+59, 1e+60, - 1e+61, 1e+62, 1e+63, 1e+64, 1e+65, 1e+66, 1e+67, 1e+68, 1e+69, 1e+70, 1e+71, 1e+72, 1e+73, 1e+74, 1e+75, 1e+76, 1e+77, 1e+78, 1e+79, 1e+80, - 1e+81, 1e+82, 1e+83, 1e+84, 1e+85, 1e+86, 1e+87, 1e+88, 1e+89, 1e+90, 1e+91, 1e+92, 1e+93, 1e+94, 1e+95, 1e+96, 1e+97, 1e+98, 1e+99, 1e+100, - 1e+101,1e+102,1e+103,1e+104,1e+105,1e+106,1e+107,1e+108,1e+109,1e+110,1e+111,1e+112,1e+113,1e+114,1e+115,1e+116,1e+117,1e+118,1e+119,1e+120, - 1e+121,1e+122,1e+123,1e+124,1e+125,1e+126,1e+127,1e+128,1e+129,1e+130,1e+131,1e+132,1e+133,1e+134,1e+135,1e+136,1e+137,1e+138,1e+139,1e+140, - 1e+141,1e+142,1e+143,1e+144,1e+145,1e+146,1e+147,1e+148,1e+149,1e+150,1e+151,1e+152,1e+153,1e+154,1e+155,1e+156,1e+157,1e+158,1e+159,1e+160, - 1e+161,1e+162,1e+163,1e+164,1e+165,1e+166,1e+167,1e+168,1e+169,1e+170,1e+171,1e+172,1e+173,1e+174,1e+175,1e+176,1e+177,1e+178,1e+179,1e+180, - 1e+181,1e+182,1e+183,1e+184,1e+185,1e+186,1e+187,1e+188,1e+189,1e+190,1e+191,1e+192,1e+193,1e+194,1e+195,1e+196,1e+197,1e+198,1e+199,1e+200, - 1e+201,1e+202,1e+203,1e+204,1e+205,1e+206,1e+207,1e+208,1e+209,1e+210,1e+211,1e+212,1e+213,1e+214,1e+215,1e+216,1e+217,1e+218,1e+219,1e+220, - 1e+221,1e+222,1e+223,1e+224,1e+225,1e+226,1e+227,1e+228,1e+229,1e+230,1e+231,1e+232,1e+233,1e+234,1e+235,1e+236,1e+237,1e+238,1e+239,1e+240, - 1e+241,1e+242,1e+243,1e+244,1e+245,1e+246,1e+247,1e+248,1e+249,1e+250,1e+251,1e+252,1e+253,1e+254,1e+255,1e+256,1e+257,1e+258,1e+259,1e+260, - 1e+261,1e+262,1e+263,1e+264,1e+265,1e+266,1e+267,1e+268,1e+269,1e+270,1e+271,1e+272,1e+273,1e+274,1e+275,1e+276,1e+277,1e+278,1e+279,1e+280, - 1e+281,1e+282,1e+283,1e+284,1e+285,1e+286,1e+287,1e+288,1e+289,1e+290,1e+291,1e+292,1e+293,1e+294,1e+295,1e+296,1e+297,1e+298,1e+299,1e+300, - 1e+301,1e+302,1e+303,1e+304,1e+305,1e+306,1e+307,1e+308 - }; - RAPIDJSON_ASSERT(n <= 308); - return n < -308 ? 0.0 : e[n + 308]; -} - -} // namespace internal -} // namespace rapidjson - -#endif // RAPIDJSON_POW10_ diff --git a/third-party/rapidjson/internal/stack.h b/third-party/rapidjson/internal/stack.h deleted file mode 100644 index 966893b3fc0..00000000000 --- a/third-party/rapidjson/internal/stack.h +++ /dev/null @@ -1,82 +0,0 @@ -#ifndef RAPIDJSON_INTERNAL_STACK_H_ -#define RAPIDJSON_INTERNAL_STACK_H_ - -namespace rapidjson { -namespace internal { - -/////////////////////////////////////////////////////////////////////////////// -// Stack - -//! A type-unsafe stack for storing different types of data. -/*! \tparam Allocator Allocator for allocating stack memory. -*/ -template -class Stack { -public: - Stack(Allocator* allocator, size_t stack_capacity) : allocator_(allocator), own_allocator_(0), stack_(0), stack_top_(0), stack_end_(0), stack_capacity_(stack_capacity) { - RAPIDJSON_ASSERT(stack_capacity_ > 0); - if (!allocator_) - own_allocator_ = allocator_ = new Allocator(); - stack_top_ = stack_ = (char*)allocator_->Malloc(stack_capacity_); - stack_end_ = stack_ + stack_capacity_; - } - - ~Stack() { - Allocator::Free(stack_); - delete own_allocator_; // Only delete if it is owned by the stack - } - - void Clear() { /*stack_top_ = 0;*/ stack_top_ = stack_; } - - template - T* Push(size_t count = 1) { - // Expand the stack if needed - if (stack_top_ + sizeof(T) * count >= stack_end_) { - size_t new_capacity = stack_capacity_ * 2; - size_t size = GetSize(); - size_t new_size = GetSize() + sizeof(T) * count; - if (new_capacity < new_size) - new_capacity = new_size; - stack_ = (char*)allocator_->Realloc(stack_, stack_capacity_, new_capacity); - stack_capacity_ = new_capacity; - stack_top_ = stack_ + size; - stack_end_ = stack_ + stack_capacity_; - } - T* ret = (T*)stack_top_; - stack_top_ += sizeof(T) * count; - return ret; - } - - template - T* Pop(size_t count) { - RAPIDJSON_ASSERT(GetSize() >= count * sizeof(T)); - stack_top_ -= count * sizeof(T); - return (T*)stack_top_; - } - - template - T* Top() { - RAPIDJSON_ASSERT(GetSize() >= sizeof(T)); - return (T*)(stack_top_ - sizeof(T)); - } - - template - T* Bottom() { return (T*)stack_; } - - Allocator& GetAllocator() { return *allocator_; } - size_t GetSize() const { return stack_top_ - stack_; } - size_t GetCapacity() const { return stack_capacity_; } - -private: - Allocator* allocator_; - Allocator* own_allocator_; - char *stack_; - char *stack_top_; - char *stack_end_; - size_t stack_capacity_; -}; - -} // namespace internal -} // namespace rapidjson - -#endif // RAPIDJSON_STACK_H_ diff --git a/third-party/rapidjson/internal/strfunc.h b/third-party/rapidjson/internal/strfunc.h deleted file mode 100644 index bbf444fe6df..00000000000 --- a/third-party/rapidjson/internal/strfunc.h +++ /dev/null @@ -1,24 +0,0 @@ -#ifndef RAPIDJSON_INTERNAL_STRFUNC_H_ -#define RAPIDJSON_INTERNAL_STRFUNC_H_ - -namespace rapidjson { -namespace internal { - -//! Custom strlen() which works on different character types. -/*! \tparam Ch Character type (e.g. char, wchar_t, short) - \param s Null-terminated input string. - \return Number of characters in the string. - \note This has the same semantics as strlen(), the return value is not number of Unicode codepoints. -*/ -template -inline SizeType StrLen(const Ch* s) { - const Ch* p = s; - while (*p != '\0') - ++p; - return SizeType(p - s); -} - -} // namespace internal -} // namespace rapidjson - -#endif // RAPIDJSON_INTERNAL_STRFUNC_H_ diff --git a/third-party/rapidjson/license.txt b/third-party/rapidjson/license.txt deleted file mode 100644 index 03d97d163ed..00000000000 --- a/third-party/rapidjson/license.txt +++ /dev/null @@ -1,19 +0,0 @@ -Copyright (C) 2011 Milo Yip - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in -all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -THE SOFTWARE. \ No newline at end of file diff --git a/third-party/rapidjson/prettywriter.h b/third-party/rapidjson/prettywriter.h deleted file mode 100644 index 238ff5ff629..00000000000 --- a/third-party/rapidjson/prettywriter.h +++ /dev/null @@ -1,156 +0,0 @@ -#ifndef RAPIDJSON_PRETTYWRITER_H_ -#define RAPIDJSON_PRETTYWRITER_H_ - -#include "writer.h" - -namespace rapidjson { - -//! Writer with indentation and spacing. -/*! - \tparam Stream Type of ouptut stream. - \tparam Encoding Encoding of both source strings and output. - \tparam Allocator Type of allocator for allocating memory of stack. -*/ -template, typename Allocator = MemoryPoolAllocator<> > -class PrettyWriter : public Writer { -public: - typedef Writer Base; - typedef typename Base::Ch Ch; - - //! Constructor - /*! \param stream Output stream. - \param allocator User supplied allocator. If it is null, it will create a private one. - \param levelDepth Initial capacity of - */ - PrettyWriter(Stream& stream, Allocator* allocator = 0, size_t levelDepth = Base::kDefaultLevelDepth) : - Base(stream, allocator, levelDepth), indentChar_(' '), indentCharCount_(4) {} - - //! Set custom indentation. - /*! \param indentChar Character for indentation. Must be whitespace character (' ', '\t', '\n', '\r'). - \param indentCharCount Number of indent characters for each indentation level. - \note The default indentation is 4 spaces. - */ - PrettyWriter& SetIndent(Ch indentChar, unsigned indentCharCount) { - RAPIDJSON_ASSERT(indentChar == ' ' || indentChar == '\t' || indentChar == '\n' || indentChar == '\r'); - indentChar_ = indentChar; - indentCharCount_ = indentCharCount; - return *this; - } - - //@name Implementation of Handler. - //@{ - - PrettyWriter& Null() { PrettyPrefix(kNullType); Base::WriteNull(); return *this; } - PrettyWriter& Bool(bool b) { PrettyPrefix(b ? kTrueType : kFalseType); Base::WriteBool(b); return *this; } - PrettyWriter& Int(int i) { PrettyPrefix(kNumberType); Base::WriteInt(i); return *this; } - PrettyWriter& Uint(unsigned u) { PrettyPrefix(kNumberType); Base::WriteUint(u); return *this; } - PrettyWriter& Int64(int64_t i64) { PrettyPrefix(kNumberType); Base::WriteInt64(i64); return *this; } - PrettyWriter& Uint64(uint64_t u64) { PrettyPrefix(kNumberType); Base::WriteUint64(u64); return *this; } - PrettyWriter& Double(double d) { PrettyPrefix(kNumberType); Base::WriteDouble(d); return *this; } - - PrettyWriter& String(const Ch* str, SizeType length, bool copy = false) { - (void)copy; - PrettyPrefix(kStringType); - Base::WriteString(str, length); - return *this; - } - - PrettyWriter& StartObject() { - PrettyPrefix(kObjectType); - new (Base::level_stack_.template Push()) typename Base::Level(false); - Base::WriteStartObject(); - return *this; - } - - PrettyWriter& EndObject(SizeType memberCount = 0) { - (void)memberCount; - RAPIDJSON_ASSERT(Base::level_stack_.GetSize() >= sizeof(typename Base::Level)); - RAPIDJSON_ASSERT(!Base::level_stack_.template Top()->inArray); - bool empty = Base::level_stack_.template Pop(1)->valueCount == 0; - - if (!empty) { - Base::stream_.Put('\n'); - WriteIndent(); - } - Base::WriteEndObject(); - return *this; - } - - PrettyWriter& StartArray() { - PrettyPrefix(kArrayType); - new (Base::level_stack_.template Push()) typename Base::Level(true); - Base::WriteStartArray(); - return *this; - } - - PrettyWriter& EndArray(SizeType memberCount = 0) { - (void)memberCount; - RAPIDJSON_ASSERT(Base::level_stack_.GetSize() >= sizeof(typename Base::Level)); - RAPIDJSON_ASSERT(Base::level_stack_.template Top()->inArray); - bool empty = Base::level_stack_.template Pop(1)->valueCount == 0; - - if (!empty) { - Base::stream_.Put('\n'); - WriteIndent(); - } - Base::WriteEndArray(); - return *this; - } - - //@} - - //! Simpler but slower overload. - PrettyWriter& String(const Ch* str) { return String(str, internal::StrLen(str)); } - -protected: - void PrettyPrefix(Type type) { - (void)type; - if (Base::level_stack_.GetSize() != 0) { // this value is not at root - typename Base::Level* level = Base::level_stack_.template Top(); - - if (level->inArray) { - if (level->valueCount > 0) { - Base::stream_.Put(','); // add comma if it is not the first element in array - Base::stream_.Put('\n'); - } - else - Base::stream_.Put('\n'); - WriteIndent(); - } - else { // in object - if (level->valueCount > 0) { - if (level->valueCount % 2 == 0) { - Base::stream_.Put(','); - Base::stream_.Put('\n'); - } - else { - Base::stream_.Put(':'); - Base::stream_.Put(' '); - } - } - else - Base::stream_.Put('\n'); - - if (level->valueCount % 2 == 0) - WriteIndent(); - } - if (!level->inArray && level->valueCount % 2 == 0) - RAPIDJSON_ASSERT(type == kStringType); // if it's in object, then even number should be a name - level->valueCount++; - } - else - RAPIDJSON_ASSERT(type == kObjectType || type == kArrayType); - } - - void WriteIndent() { - size_t count = (Base::level_stack_.GetSize() / sizeof(typename Base::Level)) * indentCharCount_; - PutN(Base::stream_, indentChar_, count); - } - - Ch indentChar_; - unsigned indentCharCount_; -}; - -} // namespace rapidjson - -#endif // RAPIDJSON_RAPIDJSON_H_ diff --git a/third-party/rapidjson/rapidjson.h b/third-party/rapidjson/rapidjson.h deleted file mode 100644 index 7acb2aa4fd0..00000000000 --- a/third-party/rapidjson/rapidjson.h +++ /dev/null @@ -1,525 +0,0 @@ -#ifndef RAPIDJSON_RAPIDJSON_H_ -#define RAPIDJSON_RAPIDJSON_H_ - -// Copyright (c) 2011-2012 Milo Yip (miloyip@gmail.com) -// Version 0.11 - -#include // malloc(), realloc(), free() -#include // memcpy() - -/////////////////////////////////////////////////////////////////////////////// -// RAPIDJSON_NO_INT64DEFINE - -// Here defines int64_t and uint64_t types in global namespace. -// If user have their own definition, can define RAPIDJSON_NO_INT64DEFINE to disable this. -#ifndef RAPIDJSON_NO_INT64DEFINE -#ifdef _MSC_VER -typedef __int64 int64_t; -typedef unsigned __int64 uint64_t; -#else -#include -#endif -#endif // RAPIDJSON_NO_INT64TYPEDEF - -/////////////////////////////////////////////////////////////////////////////// -// RAPIDJSON_ENDIAN -#define RAPIDJSON_LITTLEENDIAN 0 //!< Little endian machine -#define RAPIDJSON_BIGENDIAN 1 //!< Big endian machine - -//! Endianness of the machine. -/*! GCC provided macro for detecting endianness of the target machine. But other - compilers may not have this. User can define RAPIDJSON_ENDIAN to either - RAPIDJSON_LITTLEENDIAN or RAPIDJSON_BIGENDIAN. -*/ -#ifndef RAPIDJSON_ENDIAN -#ifdef __BYTE_ORDER__ -#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ -#define RAPIDJSON_ENDIAN RAPIDJSON_LITTLEENDIAN -#else -#define RAPIDJSON_ENDIAN RAPIDJSON_BIGENDIAN -#endif // __BYTE_ORDER__ -#else -#define RAPIDJSON_ENDIAN RAPIDJSON_LITTLEENDIAN // Assumes little endian otherwise. -#endif -#endif // RAPIDJSON_ENDIAN - -/////////////////////////////////////////////////////////////////////////////// -// RAPIDJSON_SSE2/RAPIDJSON_SSE42/RAPIDJSON_SIMD - -// Enable SSE2 optimization. -//#define RAPIDJSON_SSE2 - -// Enable SSE4.2 optimization. -//#define RAPIDJSON_SSE42 - -#if defined(RAPIDJSON_SSE2) || defined(RAPIDJSON_SSE42) -#define RAPIDJSON_SIMD -#endif - -/////////////////////////////////////////////////////////////////////////////// -// RAPIDJSON_NO_SIZETYPEDEFINE - -#ifndef RAPIDJSON_NO_SIZETYPEDEFINE -namespace rapidjson { -//! Use 32-bit array/string indices even for 64-bit platform, instead of using size_t. -/*! User may override the SizeType by defining RAPIDJSON_NO_SIZETYPEDEFINE. -*/ -typedef unsigned SizeType; -} // namespace rapidjson -#endif - -/////////////////////////////////////////////////////////////////////////////// -// RAPIDJSON_ASSERT - -//! Assertion. -/*! By default, rapidjson uses C assert() for assertion. - User can override it by defining RAPIDJSON_ASSERT(x) macro. -*/ -#ifndef RAPIDJSON_ASSERT -#include -#define RAPIDJSON_ASSERT(x) assert(x) -#endif // RAPIDJSON_ASSERT - -/////////////////////////////////////////////////////////////////////////////// -// Helpers - -#define RAPIDJSON_MULTILINEMACRO_BEGIN do { -#define RAPIDJSON_MULTILINEMACRO_END \ -} while((void)0, 0) - -namespace rapidjson { - -/////////////////////////////////////////////////////////////////////////////// -// Allocator - -/*! \class rapidjson::Allocator - \brief Concept for allocating, resizing and freeing memory block. - - Note that Malloc() and Realloc() are non-static but Free() is static. - - So if an allocator need to support Free(), it needs to put its pointer in - the header of memory block. - -\code -concept Allocator { - static const bool kNeedFree; //!< Whether this allocator needs to call Free(). - - // Allocate a memory block. - // \param size of the memory block in bytes. - // \returns pointer to the memory block. - void* Malloc(size_t size); - - // Resize a memory block. - // \param originalPtr The pointer to current memory block. Null pointer is permitted. - // \param originalSize The current size in bytes. (Design issue: since some allocator may not book-keep this, explicitly pass to it can save memory.) - // \param newSize the new size in bytes. - void* Realloc(void* originalPtr, size_t originalSize, size_t newSize); - - // Free a memory block. - // \param pointer to the memory block. Null pointer is permitted. - static void Free(void *ptr); -}; -\endcode -*/ - -/////////////////////////////////////////////////////////////////////////////// -// CrtAllocator - -//! C-runtime library allocator. -/*! This class is just wrapper for standard C library memory routines. - \implements Allocator -*/ -class CrtAllocator { -public: - static const bool kNeedFree = true; - void* Malloc(size_t size) { return malloc(size); } - void* Realloc(void* originalPtr, size_t originalSize, size_t newSize) { (void)originalSize; return realloc(originalPtr, newSize); } - static void Free(void *ptr) { free(ptr); } -}; - -/////////////////////////////////////////////////////////////////////////////// -// MemoryPoolAllocator - -//! Default memory allocator used by the parser and DOM. -/*! This allocator allocate memory blocks from pre-allocated memory chunks. - - It does not free memory blocks. And Realloc() only allocate new memory. - - The memory chunks are allocated by BaseAllocator, which is CrtAllocator by default. - - User may also supply a buffer as the first chunk. - - If the user-buffer is full then additional chunks are allocated by BaseAllocator. - - The user-buffer is not deallocated by this allocator. - - \tparam BaseAllocator the allocator type for allocating memory chunks. Default is CrtAllocator. - \implements Allocator -*/ -template -class MemoryPoolAllocator { -public: - static const bool kNeedFree = false; //!< Tell users that no need to call Free() with this allocator. (concept Allocator) - - //! Constructor with chunkSize. - /*! \param chunkSize The size of memory chunk. The default is kDefaultChunkSize. - \param baseAllocator The allocator for allocating memory chunks. - */ - MemoryPoolAllocator(size_t chunkSize = kDefaultChunkCapacity, BaseAllocator* baseAllocator = 0) : - chunkHead_(0), chunk_capacity_(chunkSize), userBuffer_(0), baseAllocator_(baseAllocator), ownBaseAllocator_(0) - { - if (!baseAllocator_) - ownBaseAllocator_ = baseAllocator_ = new BaseAllocator(); - AddChunk(chunk_capacity_); - } - - //! Constructor with user-supplied buffer. - /*! The user buffer will be used firstly. When it is full, memory pool allocates new chunk with chunk size. - - The user buffer will not be deallocated when this allocator is destructed. - - \param buffer User supplied buffer. - \param size Size of the buffer in bytes. It must at least larger than sizeof(ChunkHeader). - \param chunkSize The size of memory chunk. The default is kDefaultChunkSize. - \param baseAllocator The allocator for allocating memory chunks. - */ - MemoryPoolAllocator(char *buffer, size_t size, size_t chunkSize = kDefaultChunkCapacity, BaseAllocator* baseAllocator = 0) : - chunkHead_(0), chunk_capacity_(chunkSize), userBuffer_(buffer), baseAllocator_(baseAllocator), ownBaseAllocator_(0) - { - RAPIDJSON_ASSERT(buffer != 0); - RAPIDJSON_ASSERT(size > sizeof(ChunkHeader)); - chunkHead_ = (ChunkHeader*)buffer; - chunkHead_->capacity = size - sizeof(ChunkHeader); - chunkHead_->size = 0; - chunkHead_->next = 0; - } - - //! Destructor. - /*! This deallocates all memory chunks, excluding the user-supplied buffer. - */ - ~MemoryPoolAllocator() { - Clear(); - delete ownBaseAllocator_; - } - - //! Deallocates all memory chunks, excluding the user-supplied buffer. - void Clear() { - while(chunkHead_ != 0 && chunkHead_ != (ChunkHeader *)userBuffer_) { - ChunkHeader* next = chunkHead_->next; - baseAllocator_->Free(chunkHead_); - chunkHead_ = next; - } - } - - //! Computes the total capacity of allocated memory chunks. - /*! \return total capacity in bytes. - */ - size_t Capacity() { - size_t capacity = 0; - for (ChunkHeader* c = chunkHead_; c != 0; c = c->next) - capacity += c->capacity; - return capacity; - } - - //! Computes the memory blocks allocated. - /*! \return total used bytes. - */ - size_t Size() { - size_t size = 0; - for (ChunkHeader* c = chunkHead_; c != 0; c = c->next) - size += c->size; - return size; - } - - //! Allocates a memory block. (concept Allocator) - void* Malloc(size_t size) { - size = (size + 3) & ~3; // Force aligning size to 4 - - if (chunkHead_->size + size > chunkHead_->capacity) - AddChunk(chunk_capacity_ > size ? chunk_capacity_ : size); - - char *buffer = (char *)(chunkHead_ + 1) + chunkHead_->size; - RAPIDJSON_ASSERT(((uintptr_t)buffer & 3) == 0); // returned buffer is aligned to 4 - chunkHead_->size += size; - - return buffer; - } - - //! Resizes a memory block (concept Allocator) - void* Realloc(void* originalPtr, size_t originalSize, size_t newSize) { - if (originalPtr == 0) - return Malloc(newSize); - - // Do not shrink if new size is smaller than original - if (originalSize >= newSize) - return originalPtr; - - // Simply expand it if it is the last allocation and there is sufficient space - if (originalPtr == (char *)(chunkHead_ + 1) + chunkHead_->size - originalSize) { - size_t increment = newSize - originalSize; - increment = (increment + 3) & ~3; // Force aligning size to 4 - if (chunkHead_->size + increment <= chunkHead_->capacity) { - chunkHead_->size += increment; - RAPIDJSON_ASSERT(((uintptr_t)originalPtr & 3) == 0); // returned buffer is aligned to 4 - return originalPtr; - } - } - - // Realloc process: allocate and copy memory, do not free original buffer. - void* newBuffer = Malloc(newSize); - RAPIDJSON_ASSERT(newBuffer != 0); // Do not handle out-of-memory explicitly. - return memcpy(newBuffer, originalPtr, originalSize); - } - - //! Frees a memory block (concept Allocator) - static void Free(void *) {} // Do nothing - -private: - //! Creates a new chunk. - /*! \param capacity Capacity of the chunk in bytes. - */ - void AddChunk(size_t capacity) { - ChunkHeader* chunk = (ChunkHeader*)baseAllocator_->Malloc(sizeof(ChunkHeader) + capacity); - chunk->capacity = capacity; - chunk->size = 0; - chunk->next = chunkHead_; - chunkHead_ = chunk; - } - - static const int kDefaultChunkCapacity = 64 * 1024; //!< Default chunk capacity. - - //! Chunk header for perpending to each chunk. - /*! Chunks are stored as a singly linked list. - */ - struct ChunkHeader { - size_t capacity; //!< Capacity of the chunk in bytes (excluding the header itself). - size_t size; //!< Current size of allocated memory in bytes. - ChunkHeader *next; //!< Next chunk in the linked list. - }; - - ChunkHeader *chunkHead_; //!< Head of the chunk linked-list. Only the head chunk serves allocation. - size_t chunk_capacity_; //!< The minimum capacity of chunk when they are allocated. - char *userBuffer_; //!< User supplied buffer. - BaseAllocator* baseAllocator_; //!< base allocator for allocating memory chunks. - BaseAllocator* ownBaseAllocator_; //!< base allocator created by this object. -}; - -/////////////////////////////////////////////////////////////////////////////// -// Encoding - -/*! \class rapidjson::Encoding - \brief Concept for encoding of Unicode characters. - -\code -concept Encoding { - typename Ch; //! Type of character. - - //! \brief Encode a Unicode codepoint to a buffer. - //! \param buffer pointer to destination buffer to store the result. It should have sufficient size of encoding one character. - //! \param codepoint An unicode codepoint, ranging from 0x0 to 0x10FFFF inclusively. - //! \returns the pointer to the next character after the encoded data. - static Ch* Encode(Ch *buffer, unsigned codepoint); -}; -\endcode -*/ - -/////////////////////////////////////////////////////////////////////////////// -// UTF8 - -//! UTF-8 encoding. -/*! http://en.wikipedia.org/wiki/UTF-8 - \tparam CharType Type for storing 8-bit UTF-8 data. Default is char. - \implements Encoding -*/ -template -struct UTF8 { - typedef CharType Ch; - - static Ch* Encode(Ch *buffer, unsigned codepoint) { - if (codepoint <= 0x7F) - *buffer++ = codepoint & 0xFF; - else if (codepoint <= 0x7FF) { - *buffer++ = 0xC0 | ((codepoint >> 6) & 0xFF); - *buffer++ = 0x80 | ((codepoint & 0x3F)); - } - else if (codepoint <= 0xFFFF) { - *buffer++ = 0xE0 | ((codepoint >> 12) & 0xFF); - *buffer++ = 0x80 | ((codepoint >> 6) & 0x3F); - *buffer++ = 0x80 | (codepoint & 0x3F); - } - else { - RAPIDJSON_ASSERT(codepoint <= 0x10FFFF); - *buffer++ = 0xF0 | ((codepoint >> 18) & 0xFF); - *buffer++ = 0x80 | ((codepoint >> 12) & 0x3F); - *buffer++ = 0x80 | ((codepoint >> 6) & 0x3F); - *buffer++ = 0x80 | (codepoint & 0x3F); - } - return buffer; - } -}; - -/////////////////////////////////////////////////////////////////////////////// -// UTF16 - -//! UTF-16 encoding. -/*! http://en.wikipedia.org/wiki/UTF-16 - \tparam CharType Type for storing 16-bit UTF-16 data. Default is wchar_t. C++11 may use char16_t instead. - \implements Encoding -*/ -template -struct UTF16 { - typedef CharType Ch; - - static Ch* Encode(Ch* buffer, unsigned codepoint) { - if (codepoint <= 0xFFFF) { - RAPIDJSON_ASSERT(codepoint < 0xD800 || codepoint > 0xDFFF); // Code point itself cannot be surrogate pair - *buffer++ = static_cast(codepoint); - } - else { - RAPIDJSON_ASSERT(codepoint <= 0x10FFFF); - unsigned v = codepoint - 0x10000; - *buffer++ = static_cast((v >> 10) + 0xD800); - *buffer++ = (v & 0x3FF) + 0xDC00; - } - return buffer; - } -}; - -/////////////////////////////////////////////////////////////////////////////// -// UTF32 - -//! UTF-32 encoding. -/*! http://en.wikipedia.org/wiki/UTF-32 - \tparam Ch Type for storing 32-bit UTF-32 data. Default is unsigned. C++11 may use char32_t instead. - \implements Encoding -*/ -template -struct UTF32 { - typedef CharType Ch; - - static Ch *Encode(Ch* buffer, unsigned codepoint) { - RAPIDJSON_ASSERT(codepoint <= 0x10FFFF); - *buffer++ = codepoint; - return buffer; - } -}; - -/////////////////////////////////////////////////////////////////////////////// -// Stream - -/*! \class rapidjson::Stream - \brief Concept for reading and writing characters. - - For read-only stream, no need to implement PutBegin(), Put() and PutEnd(). - - For write-only stream, only need to implement Put(). - -\code -concept Stream { - typename Ch; //!< Character type of the stream. - - //! Read the current character from stream without moving the read cursor. - Ch Peek() const; - - //! Read the current character from stream and moving the read cursor to next character. - Ch Take(); - - //! Get the current read cursor. - //! \return Number of characters read from start. - size_t Tell(); - - //! Begin writing operation at the current read pointer. - //! \return The begin writer pointer. - Ch* PutBegin(); - - //! Write a character. - void Put(Ch c); - - //! End the writing operation. - //! \param begin The begin write pointer returned by PutBegin(). - //! \return Number of characters written. - size_t PutEnd(Ch* begin); -} -\endcode -*/ - -//! Put N copies of a character to a stream. -template -inline void PutN(Stream& stream, Ch c, size_t n) { - for (size_t i = 0; i < n; i++) - stream.Put(c); -} - -/////////////////////////////////////////////////////////////////////////////// -// StringStream - -//! Read-only string stream. -/*! \implements Stream -*/ -template -struct GenericStringStream { - typedef typename Encoding::Ch Ch; - - GenericStringStream(const Ch *src) : src_(src), head_(src) {} - - Ch Peek() const { return *src_; } - Ch Take() { return *src_++; } - size_t Tell() const { return src_ - head_; } - - Ch* PutBegin() { RAPIDJSON_ASSERT(false); return 0; } - void Put(Ch) { RAPIDJSON_ASSERT(false); } - size_t PutEnd(Ch*) { RAPIDJSON_ASSERT(false); return 0; } - - const Ch* src_; //!< Current read position. - const Ch* head_; //!< Original head of the string. -}; - -typedef GenericStringStream > StringStream; - -/////////////////////////////////////////////////////////////////////////////// -// InsituStringStream - -//! A read-write string stream. -/*! This string stream is particularly designed for in-situ parsing. - \implements Stream -*/ -template -struct GenericInsituStringStream { - typedef typename Encoding::Ch Ch; - - GenericInsituStringStream(Ch *src) : src_(src), dst_(0), head_(src) {} - - // Read - Ch Peek() { return *src_; } - Ch Take() { return *src_++; } - size_t Tell() { return src_ - head_; } - - // Write - Ch* PutBegin() { return dst_ = src_; } - void Put(Ch c) { RAPIDJSON_ASSERT(dst_ != 0); *dst_++ = c; } - size_t PutEnd(Ch* begin) { return dst_ - begin; } - - Ch* src_; - Ch* dst_; - Ch* head_; -}; - -typedef GenericInsituStringStream > InsituStringStream; - -/////////////////////////////////////////////////////////////////////////////// -// Type - -//! Type of JSON value -enum Type { - kNullType = 0, //!< null - kFalseType = 1, //!< false - kTrueType = 2, //!< true - kObjectType = 3, //!< object - kArrayType = 4, //!< array - kStringType = 5, //!< string - kNumberType = 6, //!< number -}; - -} // namespace rapidjson - -#endif // RAPIDJSON_RAPIDJSON_H_ diff --git a/third-party/rapidjson/reader.h b/third-party/rapidjson/reader.h deleted file mode 100644 index 78391add3c7..00000000000 --- a/third-party/rapidjson/reader.h +++ /dev/null @@ -1,683 +0,0 @@ -#ifndef RAPIDJSON_READER_H_ -#define RAPIDJSON_READER_H_ - -// Copyright (c) 2011 Milo Yip (miloyip@gmail.com) -// Version 0.1 - -#include "rapidjson.h" -#include "internal/pow10.h" -#include "internal/stack.h" -#include - -#ifdef RAPIDJSON_SSE42 -#include -#elif defined(RAPIDJSON_SSE2) -#include -#endif - -#ifdef _MSC_VER -#pragma warning(push) -#pragma warning(disable : 4127) // conditional expression is constant -#endif - -#ifndef RAPIDJSON_PARSE_ERROR -#define RAPIDJSON_PARSE_ERROR(msg, offset) \ - RAPIDJSON_MULTILINEMACRO_BEGIN \ - parseError_ = msg; \ - errorOffset_ = offset; \ - longjmp(jmpbuf_, 1); \ - RAPIDJSON_MULTILINEMACRO_END -#endif - -namespace rapidjson { - -/////////////////////////////////////////////////////////////////////////////// -// ParseFlag - -//! Combination of parseFlags -enum ParseFlag { - kParseDefaultFlags = 0, //!< Default parse flags. Non-destructive parsing. Text strings are decoded into allocated buffer. - kParseInsituFlag = 1 //!< In-situ(destructive) parsing. -}; - -/////////////////////////////////////////////////////////////////////////////// -// Handler - -/*! \class rapidjson::Handler - \brief Concept for receiving events from GenericReader upon parsing. -\code -concept Handler { - typename Ch; - - void Null(); - void Bool(bool b); - void Int(int i); - void Uint(unsigned i); - void Int64(int64_t i); - void Uint64(uint64_t i); - void Double(double d); - void String(const Ch* str, SizeType length, bool copy); - void StartObject(); - void EndObject(SizeType memberCount); - void StartArray(); - void EndArray(SizeType elementCount); -}; -\endcode -*/ -/////////////////////////////////////////////////////////////////////////////// -// BaseReaderHandler - -//! Default implementation of Handler. -/*! This can be used as base class of any reader handler. - \implements Handler -*/ -template > -struct BaseReaderHandler { - typedef typename Encoding::Ch Ch; - - void Default() {} - void Null() { Default(); } - void Bool(bool) { Default(); } - void Int(int) { Default(); } - void Uint(unsigned) { Default(); } - void Int64(int64_t) { Default(); } - void Uint64(uint64_t) { Default(); } - void Double(double) { Default(); } - void String(const Ch*, SizeType, bool) { Default(); } - void StartObject() { Default(); } - void EndObject(SizeType) { Default(); } - void StartArray() { Default(); } - void EndArray(SizeType) { Default(); } -}; - -/////////////////////////////////////////////////////////////////////////////// -// SkipWhitespace - -//! Skip the JSON white spaces in a stream. -/*! \param stream A input stream for skipping white spaces. - \note This function has SSE2/SSE4.2 specialization. -*/ -template -void SkipWhitespace(Stream& stream) { - Stream s = stream; // Use a local copy for optimization - while (s.Peek() == ' ' || s.Peek() == '\n' || s.Peek() == '\r' || s.Peek() == '\t') - s.Take(); - stream = s; -} - -#ifdef RAPIDJSON_SSE42 -//! Skip whitespace with SSE 4.2 pcmpistrm instruction, testing 16 8-byte characters at once. -inline const char *SkipWhitespace_SIMD(const char* p) { - static const char whitespace[16] = " \n\r\t"; - __m128i w = _mm_loadu_si128((const __m128i *)&whitespace[0]); - - for (;;) { - __m128i s = _mm_loadu_si128((const __m128i *)p); - unsigned r = _mm_cvtsi128_si32(_mm_cmpistrm(w, s, _SIDD_UBYTE_OPS | _SIDD_CMP_EQUAL_ANY | _SIDD_BIT_MASK | _SIDD_NEGATIVE_POLARITY)); - if (r == 0) // all 16 characters are whitespace - p += 16; - else { // some of characters may be non-whitespace -#ifdef _MSC_VER // Find the index of first non-whitespace - unsigned long offset; - if (_BitScanForward(&offset, r)) - return p + offset; -#else - if (r != 0) - return p + __builtin_ffs(r) - 1; -#endif - } - } -} - -#elif defined(RAPIDJSON_SSE2) - -//! Skip whitespace with SSE2 instructions, testing 16 8-byte characters at once. -inline const char *SkipWhitespace_SIMD(const char* p) { - static const char whitespaces[4][17] = { - " ", - "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n", - "\r\r\r\r\r\r\r\r\r\r\r\r\r\r\r\r", - "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t"}; - - __m128i w0 = _mm_loadu_si128((const __m128i *)&whitespaces[0][0]); - __m128i w1 = _mm_loadu_si128((const __m128i *)&whitespaces[1][0]); - __m128i w2 = _mm_loadu_si128((const __m128i *)&whitespaces[2][0]); - __m128i w3 = _mm_loadu_si128((const __m128i *)&whitespaces[3][0]); - - for (;;) { - __m128i s = _mm_loadu_si128((const __m128i *)p); - __m128i x = _mm_cmpeq_epi8(s, w0); - x = _mm_or_si128(x, _mm_cmpeq_epi8(s, w1)); - x = _mm_or_si128(x, _mm_cmpeq_epi8(s, w2)); - x = _mm_or_si128(x, _mm_cmpeq_epi8(s, w3)); - unsigned short r = ~_mm_movemask_epi8(x); - if (r == 0) // all 16 characters are whitespace - p += 16; - else { // some of characters may be non-whitespace -#ifdef _MSC_VER // Find the index of first non-whitespace - unsigned long offset; - if (_BitScanForward(&offset, r)) - return p + offset; -#else - if (r != 0) - return p + __builtin_ffs(r) - 1; -#endif - } - } -} - -#endif // RAPIDJSON_SSE2 - -#ifdef RAPIDJSON_SIMD -//! Template function specialization for InsituStringStream -template<> inline void SkipWhitespace(InsituStringStream& stream) { - stream.src_ = const_cast(SkipWhitespace_SIMD(stream.src_)); -} - -//! Template function specialization for StringStream -template<> inline void SkipWhitespace(StringStream& stream) { - stream.src_ = SkipWhitespace_SIMD(stream.src_); -} -#endif // RAPIDJSON_SIMD - -/////////////////////////////////////////////////////////////////////////////// -// GenericReader - -//! SAX-style JSON parser. Use Reader for UTF8 encoding and default allocator. -/*! GenericReader parses JSON text from a stream, and send events synchronously to an - object implementing Handler concept. - - It needs to allocate a stack for storing a single decoded string during - non-destructive parsing. - - For in-situ parsing, the decoded string is directly written to the source - text string, no temporary buffer is required. - - A GenericReader object can be reused for parsing multiple JSON text. - - \tparam Encoding Encoding of both the stream and the parse output. - \tparam Allocator Allocator type for stack. -*/ -template > -class GenericReader { -public: - typedef typename Encoding::Ch Ch; - - //! Constructor. - /*! \param allocator Optional allocator for allocating stack memory. (Only use for non-destructive parsing) - \param stackCapacity stack capacity in bytes for storing a single decoded string. (Only use for non-destructive parsing) - */ - GenericReader(Allocator* allocator = 0, size_t stackCapacity = kDefaultStackCapacity) : stack_(allocator, stackCapacity), parseError_(0), errorOffset_(0) {} - - //! Parse JSON text. - /*! \tparam parseFlags Combination of ParseFlag. - \tparam Stream Type of input stream. - \tparam Handler Type of handler which must implement Handler concept. - \param stream Input stream to be parsed. - \param handler The handler to receive events. - \return Whether the parsing is successful. - */ - template - bool Parse(Stream& stream, Handler& handler) { - parseError_ = 0; - errorOffset_ = 0; - -#ifdef _MSC_VER -#pragma warning(push) -#pragma warning(disable : 4611) // interaction between '_setjmp' and C++ object destruction is non-portable -#endif - if (setjmp(jmpbuf_)) { -#ifdef _MSC_VER -#pragma warning(pop) -#endif - stack_.Clear(); - return false; - } - - SkipWhitespace(stream); - - if (stream.Peek() == '\0') - RAPIDJSON_PARSE_ERROR("Text only contains white space(s)", stream.Tell()); - else { - switch (stream.Peek()) { - case '{': ParseObject(stream, handler); break; - case '[': ParseArray(stream, handler); break; - default: RAPIDJSON_PARSE_ERROR("Expect either an object or array at root", stream.Tell()); - } - SkipWhitespace(stream); - - if (stream.Peek() != '\0') - RAPIDJSON_PARSE_ERROR("Nothing should follow the root object or array.", stream.Tell()); - } - - return true; - } - - bool HasParseError() const { return parseError_ != 0; } - const char* GetParseError() const { return parseError_; } - size_t GetErrorOffset() const { return errorOffset_; } - -private: - // Parse object: { string : value, ... } - template - void ParseObject(Stream& stream, Handler& handler) { - RAPIDJSON_ASSERT(stream.Peek() == '{'); - stream.Take(); // Skip '{' - handler.StartObject(); - SkipWhitespace(stream); - - if (stream.Peek() == '}') { - stream.Take(); - handler.EndObject(0); // empty object - return; - } - - for (SizeType memberCount = 0;;) { - if (stream.Peek() != '"') { - RAPIDJSON_PARSE_ERROR("Name of an object member must be a string", stream.Tell()); - break; - } - - ParseString(stream, handler); - SkipWhitespace(stream); - - if (stream.Take() != ':') { - RAPIDJSON_PARSE_ERROR("There must be a colon after the name of object member", stream.Tell()); - break; - } - SkipWhitespace(stream); - - ParseValue(stream, handler); - SkipWhitespace(stream); - - ++memberCount; - - switch(stream.Take()) { - case ',': SkipWhitespace(stream); break; - case '}': handler.EndObject(memberCount); return; - default: RAPIDJSON_PARSE_ERROR("Must be a comma or '}' after an object member", stream.Tell()); - } - } - } - - // Parse array: [ value, ... ] - template - void ParseArray(Stream& stream, Handler& handler) { - RAPIDJSON_ASSERT(stream.Peek() == '['); - stream.Take(); // Skip '[' - handler.StartArray(); - SkipWhitespace(stream); - - if (stream.Peek() == ']') { - stream.Take(); - handler.EndArray(0); // empty array - return; - } - - for (SizeType elementCount = 0;;) { - ParseValue(stream, handler); - ++elementCount; - SkipWhitespace(stream); - - switch (stream.Take()) { - case ',': SkipWhitespace(stream); break; - case ']': handler.EndArray(elementCount); return; - default: RAPIDJSON_PARSE_ERROR("Must be a comma or ']' after an array element.", stream.Tell()); - } - } - } - - template - void ParseNull(Stream& stream, Handler& handler) { - RAPIDJSON_ASSERT(stream.Peek() == 'n'); - stream.Take(); - - if (stream.Take() == 'u' && stream.Take() == 'l' && stream.Take() == 'l') - handler.Null(); - else - RAPIDJSON_PARSE_ERROR("Invalid value", stream.Tell() - 1); - } - - template - void ParseTrue(Stream& stream, Handler& handler) { - RAPIDJSON_ASSERT(stream.Peek() == 't'); - stream.Take(); - - if (stream.Take() == 'r' && stream.Take() == 'u' && stream.Take() == 'e') - handler.Bool(true); - else - RAPIDJSON_PARSE_ERROR("Invalid value", stream.Tell()); - } - - template - void ParseFalse(Stream& stream, Handler& handler) { - RAPIDJSON_ASSERT(stream.Peek() == 'f'); - stream.Take(); - - if (stream.Take() == 'a' && stream.Take() == 'l' && stream.Take() == 's' && stream.Take() == 'e') - handler.Bool(false); - else - RAPIDJSON_PARSE_ERROR("Invalid value", stream.Tell() - 1); - } - - // Helper function to parse four hexidecimal digits in \uXXXX in ParseString(). - template - unsigned ParseHex4(Stream& stream) { - Stream s = stream; // Use a local copy for optimization - unsigned codepoint = 0; - for (int i = 0; i < 4; i++) { - Ch c = s.Take(); - codepoint <<= 4; - codepoint += c; - if (c >= '0' && c <= '9') - codepoint -= '0'; - else if (c >= 'A' && c <= 'F') - codepoint -= 'A' - 10; - else if (c >= 'a' && c <= 'f') - codepoint -= 'a' - 10; - else - RAPIDJSON_PARSE_ERROR("Incorrect hex digit after \\u escape", s.Tell() - 1); - } - stream = s; // Restore stream - return codepoint; - } - - // Parse string, handling the prefix and suffix double quotes and escaping. - template - void ParseString(Stream& stream, Handler& handler) { -#define Z16 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 - static const Ch escape[256] = { - Z16, Z16, 0, 0,'\"', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,'/', - Z16, Z16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,'\\', 0, 0, 0, - 0, 0,'\b', 0, 0, 0,'\f', 0, 0, 0, 0, 0, 0, 0,'\n', 0, - 0, 0,'\r', 0,'\t', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - Z16, Z16, Z16, Z16, Z16, Z16, Z16, Z16 - }; -#undef Z16 - - Stream s = stream; // Use a local copy for optimization - RAPIDJSON_ASSERT(s.Peek() == '\"'); - s.Take(); // Skip '\"' - Ch *head; - SizeType len; - if (parseFlags & kParseInsituFlag) - head = s.PutBegin(); - else - len = 0; - -#define RAPIDJSON_PUT(x) \ - do { \ - if (parseFlags & kParseInsituFlag) \ - s.Put(x); \ - else { \ - *stack_.template Push() = x; \ - ++len; \ - } \ - } while(false) - - for (;;) { - Ch c = s.Take(); - if (c == '\\') { // Escape - Ch e = s.Take(); - if ((sizeof(Ch) == 1 || (int)e < 256) && escape[(unsigned char)e]) - RAPIDJSON_PUT(escape[(unsigned char)e]); - else if (e == 'u') { // Unicode - unsigned codepoint = ParseHex4(s); - if (codepoint >= 0xD800 && codepoint <= 0xDBFF) { // Handle UTF-16 surrogate pair - if (s.Take() != '\\' || s.Take() != 'u') { - RAPIDJSON_PARSE_ERROR("Missing the second \\u in surrogate pair", s.Tell() - 2); - return; - } - unsigned codepoint2 = ParseHex4(s); - if (codepoint2 < 0xDC00 || codepoint2 > 0xDFFF) { - RAPIDJSON_PARSE_ERROR("The second \\u in surrogate pair is invalid", s.Tell() - 2); - return; - } - codepoint = (((codepoint - 0xD800) << 10) | (codepoint2 - 0xDC00)) + 0x10000; - } - - Ch buffer[4]; - SizeType count = SizeType(Encoding::Encode(buffer, codepoint) - &buffer[0]); - - if (parseFlags & kParseInsituFlag) - for (SizeType i = 0; i < count; i++) - s.Put(buffer[i]); - else { - memcpy(stack_.template Push(count), buffer, count * sizeof(Ch)); - len += count; - } - } - else { - RAPIDJSON_PARSE_ERROR("Unknown escape character", stream.Tell() - 1); - return; - } - } - else if (c == '"') { // Closing double quote - if (parseFlags & kParseInsituFlag) { - size_t length = s.PutEnd(head); - RAPIDJSON_ASSERT(length <= 0xFFFFFFFF); - RAPIDJSON_PUT('\0'); // null-terminate the string - handler.String(head, SizeType(length), false); - } - else { - RAPIDJSON_PUT('\0'); - handler.String(stack_.template Pop(len), len - 1, true); - } - stream = s; // restore stream - return; - } - else if (c == '\0') { - RAPIDJSON_PARSE_ERROR("lacks ending quotation before the end of string", stream.Tell() - 1); - return; - } - else if ((unsigned)c < 0x20) { // RFC 4627: unescaped = %x20-21 / %x23-5B / %x5D-10FFFF - RAPIDJSON_PARSE_ERROR("Incorrect unescaped character in string", stream.Tell() - 1); - return; - } - else - RAPIDJSON_PUT(c); // Normal character, just copy - } -#undef RAPIDJSON_PUT - } - - template - void ParseNumber(Stream& stream, Handler& handler) { - Stream s = stream; // Local copy for optimization - // Parse minus - bool minus = false; - if (s.Peek() == '-') { - minus = true; - s.Take(); - } - - // Parse int: zero / ( digit1-9 *DIGIT ) - unsigned i; - bool try64bit = false; - if (s.Peek() == '0') { - i = 0; - s.Take(); - } - else if (s.Peek() >= '1' && s.Peek() <= '9') { - i = s.Take() - '0'; - - if (minus) - while (s.Peek() >= '0' && s.Peek() <= '9') { - if (i >= 214748364) { // 2^31 = 2147483648 - if (i != 214748364 || s.Peek() > '8') { - try64bit = true; - break; - } - } - i = i * 10 + (s.Take() - '0'); - } - else - while (s.Peek() >= '0' && s.Peek() <= '9') { - if (i >= 429496729) { // 2^32 - 1 = 4294967295 - if (i != 429496729 || s.Peek() > '5') { - try64bit = true; - break; - } - } - i = i * 10 + (s.Take() - '0'); - } - } - else { - RAPIDJSON_PARSE_ERROR("Expect a value here.", stream.Tell()); - return; - } - - // Parse 64bit int - uint64_t i64 = 0; - bool useDouble = false; - if (try64bit) { - i64 = i; - if (minus) - while (s.Peek() >= '0' && s.Peek() <= '9') { - if (i64 >= 922337203685477580uLL) // 2^63 = 9223372036854775808 - if (i64 != 922337203685477580uLL || s.Peek() > '8') { - useDouble = true; - break; - } - i64 = i64 * 10 + (s.Take() - '0'); - } - else - while (s.Peek() >= '0' && s.Peek() <= '9') { - if (i64 >= 1844674407370955161uLL) // 2^64 - 1 = 18446744073709551615 - if (i64 != 1844674407370955161uLL || s.Peek() > '5') { - useDouble = true; - break; - } - i64 = i64 * 10 + (s.Take() - '0'); - } - } - - // Force double for big integer - double d = 0.0; - if (useDouble) { - d = (double)i64; - while (s.Peek() >= '0' && s.Peek() <= '9') { - if (d >= 1E307) { - RAPIDJSON_PARSE_ERROR("Number too big to store in double", stream.Tell()); - return; - } - d = d * 10 + (s.Take() - '0'); - } - } - - // Parse frac = decimal-point 1*DIGIT - int expFrac = 0; - if (s.Peek() == '.') { - if (!useDouble) { - d = try64bit ? (double)i64 : (double)i; - useDouble = true; - } - s.Take(); - - if (s.Peek() >= '0' && s.Peek() <= '9') { - d = d * 10 + (s.Take() - '0'); - --expFrac; - } - else { - RAPIDJSON_PARSE_ERROR("At least one digit in fraction part", stream.Tell()); - return; - } - - while (s.Peek() >= '0' && s.Peek() <= '9') { - if (expFrac > -16) { - d = d * 10 + (s.Peek() - '0'); - --expFrac; - } - s.Take(); - } - } - - // Parse exp = e [ minus / plus ] 1*DIGIT - int exp = 0; - if (s.Peek() == 'e' || s.Peek() == 'E') { - if (!useDouble) { - d = try64bit ? (double)i64 : (double)i; - useDouble = true; - } - s.Take(); - - bool expMinus = false; - if (s.Peek() == '+') - s.Take(); - else if (s.Peek() == '-') { - s.Take(); - expMinus = true; - } - - if (s.Peek() >= '0' && s.Peek() <= '9') { - exp = s.Take() - '0'; - while (s.Peek() >= '0' && s.Peek() <= '9') { - exp = exp * 10 + (s.Take() - '0'); - if (exp > 308) { - RAPIDJSON_PARSE_ERROR("Number too big to store in double", stream.Tell()); - return; - } - } - } - else { - RAPIDJSON_PARSE_ERROR("At least one digit in exponent", s.Tell()); - return; - } - - if (expMinus) - exp = -exp; - } - - // Finish parsing, call event according to the type of number. - if (useDouble) { - d *= internal::Pow10(exp + expFrac); - handler.Double(minus ? -d : d); - } - else { - if (try64bit) { - if (minus) - handler.Int64(-(int64_t)i64); - else - handler.Uint64(i64); - } - else { - if (minus) - handler.Int(-(int)i); - else - handler.Uint(i); - } - } - - stream = s; // restore stream - } - - // Parse any JSON value - template - void ParseValue(Stream& stream, Handler& handler) { - switch (stream.Peek()) { - case 'n': ParseNull (stream, handler); break; - case 't': ParseTrue (stream, handler); break; - case 'f': ParseFalse (stream, handler); break; - case '"': ParseString(stream, handler); break; - case '{': ParseObject(stream, handler); break; - case '[': ParseArray (stream, handler); break; - default : ParseNumber(stream, handler); - } - } - - static const size_t kDefaultStackCapacity = 256; //!< Default stack capacity in bytes for storing a single decoded string. - internal::Stack stack_; //!< A stack for storing decoded string temporarily during non-destructive parsing. - jmp_buf jmpbuf_; //!< setjmp buffer for fast exit from nested parsing function calls. - const char* parseError_; - size_t errorOffset_; -}; // class GenericReader - -//! Reader with UTF8 encoding and default allocator. -typedef GenericReader > Reader; - -} // namespace rapidjson - -#ifdef _MSC_VER -#pragma warning(pop) -#endif - -#endif // RAPIDJSON_READER_H_ diff --git a/third-party/rapidjson/stringbuffer.h b/third-party/rapidjson/stringbuffer.h deleted file mode 100644 index 269ae10761d..00000000000 --- a/third-party/rapidjson/stringbuffer.h +++ /dev/null @@ -1,49 +0,0 @@ -#ifndef RAPIDJSON_STRINGBUFFER_H_ -#define RAPIDJSON_STRINGBUFFER_H_ - -#include "rapidjson.h" -#include "internal/stack.h" - -namespace rapidjson { - -//! Represents an in-memory output stream. -/*! - \tparam Encoding Encoding of the stream. - \tparam Allocator type for allocating memory buffer. - \implements Stream -*/ -template -struct GenericStringBuffer { - typedef typename Encoding::Ch Ch; - - GenericStringBuffer(Allocator* allocator = 0, size_t capacity = kDefaultCapacity) : stack_(allocator, capacity) {} - - void Put(Ch c) { *stack_.template Push() = c; } - - void Clear() { stack_.Clear(); } - - const char* GetString() const { - // Push and pop a null terminator. This is safe. - *stack_.template Push() = '\0'; - stack_.template Pop(1); - - return stack_.template Bottom(); - } - - size_t Size() const { return stack_.GetSize(); } - - static const size_t kDefaultCapacity = 256; - mutable internal::Stack stack_; -}; - -typedef GenericStringBuffer > StringBuffer; - -//! Implement specialized version of PutN() with memset() for better performance. -template<> -inline void PutN(GenericStringBuffer >& stream, char c, size_t n) { - memset(stream.stack_.Push(n), c, n * sizeof(c)); -} - -} // namespace rapidjson - -#endif // RAPIDJSON_STRINGBUFFER_H_ diff --git a/third-party/rapidjson/writer.h b/third-party/rapidjson/writer.h deleted file mode 100644 index d96f2081a91..00000000000 --- a/third-party/rapidjson/writer.h +++ /dev/null @@ -1,241 +0,0 @@ -#ifndef RAPIDJSON_WRITER_H_ -#define RAPIDJSON_WRITER_H_ - -#include "rapidjson.h" -#include "internal/stack.h" -#include "internal/strfunc.h" -#include // snprintf() or _sprintf_s() -#include // placement new - -#ifdef _MSC_VER -#pragma warning(push) -#pragma warning(disable : 4127) // conditional expression is constant -#endif - -namespace rapidjson { - -//! JSON writer -/*! Writer implements the concept Handler. - It generates JSON text by events to an output stream. - - User may programmatically calls the functions of a writer to generate JSON text. - - On the other side, a writer can also be passed to objects that generates events, - - for example Reader::Parse() and Document::Accept(). - - \tparam Stream Type of ouptut stream. - \tparam Encoding Encoding of both source strings and output. - \implements Handler -*/ -template, typename Allocator = MemoryPoolAllocator<> > -class Writer { -public: - typedef typename Encoding::Ch Ch; - - Writer(Stream& stream, Allocator* allocator = 0, size_t levelDepth = kDefaultLevelDepth) : - stream_(stream), level_stack_(allocator, levelDepth * sizeof(Level)) {} - - //@name Implementation of Handler - //@{ - Writer& Null() { Prefix(kNullType); WriteNull(); return *this; } - Writer& Bool(bool b) { Prefix(b ? kTrueType : kFalseType); WriteBool(b); return *this; } - Writer& Int(int i) { Prefix(kNumberType); WriteInt(i); return *this; } - Writer& Uint(unsigned u) { Prefix(kNumberType); WriteUint(u); return *this; } - Writer& Int64(int64_t i64) { Prefix(kNumberType); WriteInt64(i64); return *this; } - Writer& Uint64(uint64_t u64) { Prefix(kNumberType); WriteUint64(u64); return *this; } - Writer& Double(double d) { Prefix(kNumberType); WriteDouble(d); return *this; } - - Writer& String(const Ch* str, SizeType length, bool copy = false) { - (void)copy; - Prefix(kStringType); - WriteString(str, length); - return *this; - } - - Writer& StartObject() { - Prefix(kObjectType); - new (level_stack_.template Push()) Level(false); - WriteStartObject(); - return *this; - } - - Writer& EndObject(SizeType memberCount = 0) { - (void)memberCount; - RAPIDJSON_ASSERT(level_stack_.GetSize() >= sizeof(Level)); - RAPIDJSON_ASSERT(!level_stack_.template Top()->inArray); - level_stack_.template Pop(1); - WriteEndObject(); - return *this; - } - - Writer& StartArray() { - Prefix(kArrayType); - new (level_stack_.template Push()) Level(true); - WriteStartArray(); - return *this; - } - - Writer& EndArray(SizeType elementCount = 0) { - (void)elementCount; - RAPIDJSON_ASSERT(level_stack_.GetSize() >= sizeof(Level)); - RAPIDJSON_ASSERT(level_stack_.template Top()->inArray); - level_stack_.template Pop(1); - WriteEndArray(); - return *this; - } - //@} - - //! Simpler but slower overload. - Writer& String(const Ch* str) { return String(str, internal::StrLen(str)); } - -protected: - //! Information for each nested level - struct Level { - Level(bool inArray_) : inArray(inArray_), valueCount(0) {} - bool inArray; //!< true if in array, otherwise in object - size_t valueCount; //!< number of values in this level - }; - - static const size_t kDefaultLevelDepth = 32; - - void WriteNull() { - stream_.Put('n'); stream_.Put('u'); stream_.Put('l'); stream_.Put('l'); - } - - void WriteBool(bool b) { - if (b) { - stream_.Put('t'); stream_.Put('r'); stream_.Put('u'); stream_.Put('e'); - } - else { - stream_.Put('f'); stream_.Put('a'); stream_.Put('l'); stream_.Put('s'); stream_.Put('e'); - } - } - - void WriteInt(int i) { - if (i < 0) { - stream_.Put('-'); - i = -i; - } - WriteUint((unsigned)i); - } - - void WriteUint(unsigned u) { - char buffer[10]; - char *p = buffer; - do { - *p++ = (u % 10) + '0'; - u /= 10; - } while (u > 0); - - do { - --p; - stream_.Put(*p); - } while (p != buffer); - } - - void WriteInt64(int64_t i64) { - if (i64 < 0) { - stream_.Put('-'); - i64 = -i64; - } - WriteUint64((uint64_t)i64); - } - - void WriteUint64(uint64_t u64) { - char buffer[20]; - char *p = buffer; - do { - *p++ = char(u64 % 10) + '0'; - u64 /= 10; - } while (u64 > 0); - - do { - --p; - stream_.Put(*p); - } while (p != buffer); - } - - //! \todo Optimization with custom double-to-string converter. - void WriteDouble(double d) { - char buffer[100]; -#if _MSC_VER - int ret = sprintf_s(buffer, sizeof(buffer), "%g", d); -#else - int ret = snprintf(buffer, sizeof(buffer), "%g", d); -#endif - RAPIDJSON_ASSERT(ret >= 1); - for (int i = 0; i < ret; i++) - stream_.Put(buffer[i]); - } - - void WriteString(const Ch* str, SizeType length) { - static const char hexDigits[] = "0123456789ABCDEF"; - static const char escape[256] = { -#define Z16 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 - //0 1 2 3 4 5 6 7 8 9 A B C D E F - 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'b', 't', 'n', 'u', 'f', 'r', 'u', 'u', // 00 - 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', 'u', // 10 - 0, 0, '"', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // 20 - Z16, Z16, // 30~4F - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,'\\', 0, 0, 0, // 50 - Z16, Z16, Z16, Z16, Z16, Z16, Z16, Z16, Z16, Z16 // 60~FF -#undef Z16 - }; - - stream_.Put('\"'); - for (const Ch* p = str; p != str + length; ++p) { - if ((sizeof(Ch) == 1 || *p < 256) && escape[(unsigned char)*p]) { - stream_.Put('\\'); - stream_.Put(escape[(unsigned char)*p]); - if (escape[(unsigned char)*p] == 'u') { - stream_.Put('0'); - stream_.Put('0'); - stream_.Put(hexDigits[(*p) >> 4]); - stream_.Put(hexDigits[(*p) & 0xF]); - } - } - else - stream_.Put(*p); - } - stream_.Put('\"'); - } - - void WriteStartObject() { stream_.Put('{'); } - void WriteEndObject() { stream_.Put('}'); } - void WriteStartArray() { stream_.Put('['); } - void WriteEndArray() { stream_.Put(']'); } - - void Prefix(Type type) { - (void)type; - if (level_stack_.GetSize() != 0) { // this value is not at root - Level* level = level_stack_.template Top(); - if (level->valueCount > 0) { - if (level->inArray) - stream_.Put(','); // add comma if it is not the first element in array - else // in object - stream_.Put((level->valueCount % 2 == 0) ? ',' : ':'); - } - if (!level->inArray && level->valueCount % 2 == 0) - RAPIDJSON_ASSERT(type == kStringType); // if it's in object, then even number should be a name - level->valueCount++; - } - else - RAPIDJSON_ASSERT(type == kObjectType || type == kArrayType); - } - - Stream& stream_; - internal::Stack level_stack_; - -private: - // Prohibit assignment for VC C4512 warning - Writer& operator=(const Writer& w); -}; - -} // namespace rapidjson - -#ifdef _MSC_VER -#pragma warning(pop) -#endif - -#endif // RAPIDJSON_RAPIDJSON_H_ diff --git a/thirdparty.inc b/thirdparty.inc new file mode 100644 index 00000000000..a364d1d4480 --- /dev/null +++ b/thirdparty.inc @@ -0,0 +1,254 @@ +# Edit definitions below to specify paths to include files and libraries of all 3rd party libraries + +# +# Edit these lines to set defaults for use of external libraries +# +set(USE_GFLAGS_DEFAULT 0) # GFLAGS is disabled by default, enable with -DGFLAGS=1 cmake command line agrument +set(USE_SNAPPY_DEFAULT 0) # SNAPPY is disabled by default, enable with -DSNAPPY=1 cmake command line agrument +set(USE_LZ4_DEFAULT 0) # LZ4 is disabled by default, enable with -DLZ4=1 cmake command line agrument +set(USE_ZLIB_DEFAULT 0) # ZLIB is disabled by default, enable with -DZLIB=1 cmake command line agrument +set(USE_XPRESS_DEFAULT 0) # XPRESS is disabled by default, enable with -DXPRESS=1 cmake command line agrument + +# +# This example assumes all the libraries locate in directories under THIRDPARTY_HOME environment variable +# Set environment variable THIRDPARTY_HOME to point to your third party libraries home (Unix style dir separators) +# or change the paths below to reflect where the libraries actually reside +# +set (THIRDPARTY_LIBS "") # Initialization, don't touch + +# +# Edit these 4 lines to define paths to GFLAGS +# +set(GFLAGS_HOME $ENV{THIRDPARTY_HOME}/Gflags.Library) +set(GFLAGS_INCLUDE ${GFLAGS_HOME}/inc/include) +set(GFLAGS_LIB_DEBUG ${GFLAGS_HOME}/bin/debug/amd64/gflags.lib) +set(GFLAGS_LIB_RELEASE ${GFLAGS_HOME}/bin/retail/amd64/gflags.lib) + +# ================================================== GFLAGS ================================================== +# +# Don't touch these lines +# +if (DEFINED GFLAGS) + set(USE_GFLAGS ${GFLAGS}) +else () + set(USE_GFLAGS ${USE_GFLAGS_DEFAULT}) +endif () + +if (${USE_GFLAGS} EQUAL 1) + message(STATUS "GFLAGS library is enabled") + + if(DEFINED ENV{GFLAGS_INCLUDE}) + set(GFLAGS_INCLUDE $ENV{GFLAGS_INCLUDE}) + endif() + + if(DEFINED ENV{GFLAGS_LIB_DEBUG}) + set(GFLAGS_LIB_DEBUG $ENV{GFLAGS_LIB_DEBUG}) + endif() + + if(DEFINED ENV{GFLAGS_LIB_RELEASE}) + set(GFLAGS_LIB_RELEASE $ENV{GFLAGS_LIB_RELEASE}) + endif() + + set(GFLAGS_CXX_FLAGS -DGFLAGS=gflags) + set(GFLAGS_LIBS debug ${GFLAGS_LIB_DEBUG} optimized ${GFLAGS_LIB_RELEASE}) + + add_definitions(${GFLAGS_CXX_FLAGS}) + include_directories(${GFLAGS_INCLUDE}) + set (THIRDPARTY_LIBS ${THIRDPARTY_LIBS} ${GFLAGS_LIBS}) +else () + message(STATUS "GFLAGS library is disabled") +endif () + +# ================================================== SNAPPY ================================================== +# +# Edit these 4 lines to define paths to Snappy +# +set(SNAPPY_HOME $ENV{THIRDPARTY_HOME}/Snappy.Library) +set(SNAPPY_INCLUDE ${SNAPPY_HOME}/inc/inc) +set(SNAPPY_LIB_DEBUG ${SNAPPY_HOME}/bin/debug/amd64/snappy.lib) +set(SNAPPY_LIB_RELEASE ${SNAPPY_HOME}/bin/retail/amd64/snappy.lib) + +# +# Don't touch these lines +# +if (DEFINED SNAPPY) + set(USE_SNAPPY ${SNAPPY}) +else () + set(USE_SNAPPY ${USE_SNAPPY_DEFAULT}) +endif () + +if (${USE_SNAPPY} EQUAL 1) + message(STATUS "SNAPPY library is enabled") + + if(DEFINED ENV{SNAPPY_INCLUDE}) + set(SNAPPY_INCLUDE $ENV{SNAPPY_INCLUDE}) + endif() + + if(DEFINED ENV{SNAPPY_LIB_DEBUG}) + set(SNAPPY_LIB_DEBUG $ENV{SNAPPY_LIB_DEBUG}) + endif() + + if(DEFINED ENV{SNAPPY_LIB_RELEASE}) + set(SNAPPY_LIB_RELEASE $ENV{SNAPPY_LIB_RELEASE}) + endif() + + set(SNAPPY_CXX_FLAGS -DSNAPPY) + set(SNAPPY_LIBS debug ${SNAPPY_LIB_DEBUG} optimized ${SNAPPY_LIB_RELEASE}) + + add_definitions(${SNAPPY_CXX_FLAGS}) + include_directories(${SNAPPY_INCLUDE}) + set (THIRDPARTY_LIBS ${THIRDPARTY_LIBS} ${SNAPPY_LIBS}) +else () + message(STATUS "SNAPPY library is disabled") +endif () + +# ================================================== LZ4 ================================================== +# +# Edit these 4 lines to define paths to LZ4 +# +set(LZ4_HOME $ENV{THIRDPARTY_HOME}/LZ4.Library) +set(LZ4_INCLUDE ${LZ4_HOME}/inc/include) +set(LZ4_LIB_DEBUG ${LZ4_HOME}/bin/debug/amd64/lz4.lib) +set(LZ4_LIB_RELEASE ${LZ4_HOME}/bin/retail/amd64/lz4.lib) + +# +# Don't touch these lines +# +if (DEFINED LZ4) + set(USE_LZ4 ${LZ4}) +else () + set(USE_LZ4 ${USE_LZ4_DEFAULT}) +endif () + +if (${USE_LZ4} EQUAL 1) + message(STATUS "LZ4 library is enabled") + + if(DEFINED ENV{LZ4_INCLUDE}) + set(LZ4_INCLUDE $ENV{LZ4_INCLUDE}) + endif() + + if(DEFINED ENV{LZ4_LIB_DEBUG}) + set(LZ4_LIB_DEBUG $ENV{LZ4_LIB_DEBUG}) + endif() + + if(DEFINED ENV{LZ4_LIB_RELEASE}) + set(LZ4_LIB_RELEASE $ENV{LZ4_LIB_RELEASE}) + endif() + + set(LZ4_CXX_FLAGS -DLZ4) + set(LZ4_LIBS debug ${LZ4_LIB_DEBUG} optimized ${LZ4_LIB_RELEASE}) + + add_definitions(${LZ4_CXX_FLAGS}) + include_directories(${LZ4_INCLUDE}) + set (THIRDPARTY_LIBS ${THIRDPARTY_LIBS} ${LZ4_LIBS}) +else () + message(STATUS "LZ4 library is disabled") +endif () + +# ================================================== ZLIB ================================================== +# +# Edit these 4 lines to define paths to ZLIB +# +set(ZLIB_HOME $ENV{THIRDPARTY_HOME}/ZLIB.Library) +set(ZLIB_INCLUDE ${ZLIB_HOME}/inc/include) +set(ZLIB_LIB_DEBUG ${ZLIB_HOME}/bin/debug/amd64/zlib.lib) +set(ZLIB_LIB_RELEASE ${ZLIB_HOME}/bin/retail/amd64/zlib.lib) + +# +# Don't touch these lines +# +if (DEFINED ZLIB) + set(USE_ZLIB ${ZLIB}) +else () + set(USE_ZLIB ${USE_ZLIB_DEFAULT}) +endif () + +if (${USE_ZLIB} EQUAL 1) + message(STATUS "ZLIB library is enabled") + + if(DEFINED ENV{ZLIB_INCLUDE}) + set(ZLIB_INCLUDE $ENV{ZLIB_INCLUDE}) + endif() + + if(DEFINED ENV{ZLIB_LIB_DEBUG}) + set(ZLIB_LIB_DEBUG $ENV{ZLIB_LIB_DEBUG}) + endif() + + if(DEFINED ENV{ZLIB_LIB_RELEASE}) + set(ZLIB_LIB_RELEASE $ENV{ZLIB_LIB_RELEASE}) + endif() + + set(ZLIB_CXX_FLAGS -DZLIB) + set(ZLIB_LIBS debug ${ZLIB_LIB_DEBUG} optimized ${ZLIB_LIB_RELEASE}) + + add_definitions(${ZLIB_CXX_FLAGS}) + include_directories(${ZLIB_INCLUDE}) + set (THIRDPARTY_LIBS ${THIRDPARTY_LIBS} ${ZLIB_LIBS}) +else () + message(STATUS "ZLIB library is disabled") +endif () + +if (DEFINED XPRESS) + set(USE_XPRESS ${XPRESS}) +else () + set(USE_XPRESS ${USE_XPRESS_DEFAULT}) +endif () + +if (${USE_XPRESS} EQUAL 1) + message(STATUS "XPRESS is enabled") + + add_definitions(-DXPRESS) + + # We are using the implementation provided by the system + set (SYSTEM_LIBS ${SYSTEM_LIBS} Cabinet.lib) +else () + message(STATUS "XPRESS is disabled") +endif () + +# +# Edit these 4 lines to define paths to Jemalloc +# +set(JEMALLOC_HOME $ENV{THIRDPARTY_HOME}/Jemalloc.Library) +set(JEMALLOC_INCLUDE ${JEMALLOC_HOME}/inc/include) +set(JEMALLOC_LIB_DEBUG ${JEMALLOC_HOME}/bin/debug/amd64/jemalloc.lib) +set(JEMALLOC_LIB_RELEASE ${JEMALLOC_HOME}/bin/retail/amd64/jemalloc.lib) + +# ================================================== JEMALLOC ================================================== +# +# Don't touch these lines +# + +# For compatibilty with previous +if(JEMALLOC) + set(WITH_JEMALLOC ON) +endif() + +if (WITH_JEMALLOC) + message(STATUS "JEMALLOC library is enabled") + set(JEMALLOC_CXX_FLAGS "-DROCKSDB_JEMALLOC -DJEMALLOC_EXPORT= ") + + if(DEFINED ENV{JEMALLOC_INCLUDE}) + set(JEMALLOC_INCLUDE $ENV{JEMALLOC_INCLUDE}) + endif() + + if(DEFINED ENV{JEMALLOC_LIB_DEBUG}) + set(JEMALLOC_LIB_DEBUG $ENV{JEMALLOC_LIB_DEBUG}) + endif() + + if(DEFINED ENV{JEMALLOC_LIB_RELEASE}) + set(JEMALLOC_LIB_RELEASE $ENV{JEMALLOC_LIB_RELEASE}) + endif() + + set(JEMALLOC_LIBS debug ${JEMALLOC_LIB_DEBUG} optimized ${JEMALLOC_LIB_RELEASE}) + + add_definitions(${JEMALLOC_CXX_FLAGS}) + include_directories(${JEMALLOC_INCLUDE}) + set (THIRDPARTY_LIBS ${THIRDPARTY_LIBS} ${JEMALLOC_LIBS}) + set (ARTIFACT_SUFFIX "_je") + + set(WITH_JEMALLOC ON) + +else () + set (ARTIFACT_SUFFIX "") + message(STATUS "JEMALLOC library is disabled") +endif () diff --git a/tools/CMakeLists.txt b/tools/CMakeLists.txt new file mode 100644 index 00000000000..6c4733a717b --- /dev/null +++ b/tools/CMakeLists.txt @@ -0,0 +1,21 @@ +set(TOOLS + sst_dump.cc + db_sanity_test.cc + db_stress.cc + write_stress.cc + ldb.cc + db_repl_stress.cc + dump/rocksdb_dump.cc + dump/rocksdb_undump.cc) +foreach(src ${TOOLS}) + get_filename_component(exename ${src} NAME_WE) + add_executable(${exename}${ARTIFACT_SUFFIX} + ${src}) + target_link_libraries(${exename}${ARTIFACT_SUFFIX} ${LIBS}) + list(APPEND tool_deps ${exename}) +endforeach() +add_custom_target(tools + DEPENDS ${tool_deps}) +add_custom_target(ldb_tests + COMMAND python ${CMAKE_CURRENT_SOURCE_DIR}/ldb_tests.py + DEPENDS ldb) diff --git a/tools/Dockerfile b/tools/Dockerfile new file mode 100644 index 00000000000..1d5ead7fdb1 --- /dev/null +++ b/tools/Dockerfile @@ -0,0 +1,5 @@ +FROM buildpack-deps:wheezy + +ADD ./ldb /rocksdb/tools/ldb + +CMD /rocksdb/tools/ldb diff --git a/tools/auto_sanity_test.sh b/tools/auto_sanity_test.sh index 2d63c0a85f5..54577ffb77c 100755 --- a/tools/auto_sanity_test.sh +++ b/tools/auto_sanity_test.sh @@ -1,4 +1,4 @@ -TMP_DIR="/tmp/rocksdb-sanity-test" +TMP_DIR="${TMPDIR:-/tmp}/rocksdb-sanity-test" if [ "$#" -lt 2 ]; then echo "usage: ./auto_sanity_test.sh [new_commit] [old_commit]" @@ -37,35 +37,55 @@ echo "Running db sanity check with commits $commit_new and $commit_old." echo "=============================================================" echo "Making build $commit_new" +git checkout $commit_new +if [ $? -ne 0 ]; then + echo "[ERROR] Can't checkout $commit_new" + exit 1 +fi makestuff mv db_sanity_test new_db_sanity_test echo "Creating db based on the new commit --- $commit_new" ./new_db_sanity_test $dir_new create +cp ./tools/db_sanity_test.cc $dir_new +cp ./tools/auto_sanity_test.sh $dir_new echo "=============================================================" echo "Making build $commit_old" +git checkout $commit_old +if [ $? -ne 0 ]; then + echo "[ERROR] Can't checkout $commit_old" + exit 1 +fi +cp -f $dir_new/db_sanity_test.cc ./tools/. +cp -f $dir_new/auto_sanity_test.sh ./tools/. makestuff mv db_sanity_test old_db_sanity_test echo "Creating db based on the old commit --- $commit_old" ./old_db_sanity_test $dir_old create echo "=============================================================" -echo "Verifying new db $dir_new using the old commit --- $commit_old" -./old_db_sanity_test $dir_new verify +echo "[Backward Compatibility Check]" +echo "Verifying old db $dir_old using the new commit --- $commit_new" +./new_db_sanity_test $dir_old verify if [ $? -ne 0 ]; then - echo "[ERROR] Verification of $dir_new using commit $commit_old failed." + echo "[ERROR] Backward Compatibility Check fails:" + echo " Verification of $dir_old using commit $commit_new failed." exit 2 fi echo "=============================================================" -echo "Verifying old db $dir_old using the new commit --- $commit_new" -./new_db_sanity_test $dir_old verify +echo "[Forward Compatibility Check]" +echo "Verifying new db $dir_new using the old commit --- $commit_old" +./old_db_sanity_test $dir_new verify if [ $? -ne 0 ]; then - echo "[ERROR] Verification of $dir_old using commit $commit_new failed." + echo "[ERROR] Forward Compatibility Check fails:" + echo " $dir_new using commit $commit_old failed." exit 2 fi rm old_db_sanity_test rm new_db_sanity_test +rm -rf $dir_new +rm -rf $dir_old echo "Auto sanity test passed!" diff --git a/tools/benchmark.sh b/tools/benchmark.sh new file mode 100755 index 00000000000..1a2c38439cf --- /dev/null +++ b/tools/benchmark.sh @@ -0,0 +1,511 @@ +#!/usr/bin/env bash +# REQUIRE: db_bench binary exists in the current directory + +if [ $# -ne 1 ]; then + echo -n "./benchmark.sh [bulkload/fillseq/overwrite/filluniquerandom/" + echo "readrandom/readwhilewriting/readwhilemerging/updaterandom/" + echo "mergerandom/randomtransaction/compact]" + exit 0 +fi + +# Make it easier to run only the compaction test. Getting valid data requires +# a number of iterations and having an ability to run the test separately from +# rest of the benchmarks helps. +if [ "$COMPACTION_TEST" == "1" -a "$1" != "universal_compaction" ]; then + echo "Skipping $1 because it's not a compaction test." + exit 0 +fi + +# size constants +K=1024 +M=$((1024 * K)) +G=$((1024 * M)) + +if [ -z $DB_DIR ]; then + echo "DB_DIR is not defined" + exit 0 +fi + +if [ -z $WAL_DIR ]; then + echo "WAL_DIR is not defined" + exit 0 +fi + +output_dir=${OUTPUT_DIR:-/tmp/} +if [ ! -d $output_dir ]; then + mkdir -p $output_dir +fi + +# all multithreaded tests run with sync=1 unless +# $DB_BENCH_NO_SYNC is defined +syncval="1" +if [ ! -z $DB_BENCH_NO_SYNC ]; then + echo "Turning sync off for all multithreaded tests" + syncval="0"; +fi + +num_threads=${NUM_THREADS:-16} +mb_written_per_sec=${MB_WRITE_PER_SEC:-0} +# Only for tests that do range scans +num_nexts_per_seek=${NUM_NEXTS_PER_SEEK:-10} +cache_size=${CACHE_SIZE:-$((1 * G))} +compression_max_dict_bytes=${COMPRESSION_MAX_DICT_BYTES:-0} +compression_type=${COMPRESSION_TYPE:-snappy} +duration=${DURATION:-0} + +num_keys=${NUM_KEYS:-$((1 * G))} +key_size=${KEY_SIZE:-20} +value_size=${VALUE_SIZE:-400} +block_size=${BLOCK_SIZE:-8192} + +const_params=" + --db=$DB_DIR \ + --wal_dir=$WAL_DIR \ + \ + --num=$num_keys \ + --num_levels=6 \ + --key_size=$key_size \ + --value_size=$value_size \ + --block_size=$block_size \ + --cache_size=$cache_size \ + --cache_numshardbits=6 \ + --compression_max_dict_bytes=$compression_max_dict_bytes \ + --compression_ratio=0.5 \ + --compression_type=$compression_type \ + --level_compaction_dynamic_level_bytes=true \ + --bytes_per_sync=$((8 * M)) \ + --cache_index_and_filter_blocks=0 \ + --pin_l0_filter_and_index_blocks_in_cache=1 \ + --benchmark_write_rate_limit=$(( 1024 * 1024 * $mb_written_per_sec )) \ + \ + --hard_rate_limit=3 \ + --rate_limit_delay_max_milliseconds=1000000 \ + --write_buffer_size=$((128 * M)) \ + --target_file_size_base=$((128 * M)) \ + --max_bytes_for_level_base=$((1 * G)) \ + \ + --verify_checksum=1 \ + --delete_obsolete_files_period_micros=$((60 * M)) \ + --max_bytes_for_level_multiplier=8 \ + \ + --statistics=0 \ + --stats_per_interval=1 \ + --stats_interval_seconds=60 \ + --histogram=1 \ + \ + --memtablerep=skip_list \ + --bloom_bits=10 \ + --open_files=-1" + +l0_config=" + --level0_file_num_compaction_trigger=4 \ + --level0_slowdown_writes_trigger=12 \ + --level0_stop_writes_trigger=20" + +if [ $duration -gt 0 ]; then + const_params="$const_params --duration=$duration" +fi + +params_w="$const_params \ + $l0_config \ + --max_background_compactions=16 \ + --max_write_buffer_number=8 \ + --max_background_flushes=7" + +params_bulkload="$const_params \ + --max_background_compactions=16 \ + --max_write_buffer_number=8 \ + --max_background_flushes=7 \ + --level0_file_num_compaction_trigger=$((10 * M)) \ + --level0_slowdown_writes_trigger=$((10 * M)) \ + --level0_stop_writes_trigger=$((10 * M))" + +# +# Tune values for level and universal compaction. +# For universal compaction, these level0_* options mean total sorted of runs in +# LSM. In level-based compaction, it means number of L0 files. +# +params_level_compact="$const_params \ + --max_background_flushes=4 \ + --max_write_buffer_number=4 \ + --level0_file_num_compaction_trigger=4 \ + --level0_slowdown_writes_trigger=16 \ + --level0_stop_writes_trigger=20" + +params_univ_compact="$const_params \ + --max_background_flushes=4 \ + --max_write_buffer_number=4 \ + --level0_file_num_compaction_trigger=8 \ + --level0_slowdown_writes_trigger=16 \ + --level0_stop_writes_trigger=20" + +function summarize_result { + test_out=$1 + test_name=$2 + bench_name=$3 + + # Note that this function assumes that the benchmark executes long enough so + # that "Compaction Stats" is written to stdout at least once. If it won't + # happen then empty output from grep when searching for "Sum" will cause + # syntax errors. + uptime=$( grep ^Uptime\(secs $test_out | tail -1 | awk '{ printf "%.0f", $2 }' ) + stall_time=$( grep "^Cumulative stall" $test_out | tail -1 | awk '{ print $3 }' ) + stall_pct=$( grep "^Cumulative stall" $test_out| tail -1 | awk '{ print $5 }' ) + ops_sec=$( grep ^${bench_name} $test_out | awk '{ print $5 }' ) + mb_sec=$( grep ^${bench_name} $test_out | awk '{ print $7 }' ) + lo_wgb=$( grep "^ L0" $test_out | tail -1 | awk '{ print $8 }' ) + sum_wgb=$( grep "^ Sum" $test_out | tail -1 | awk '{ print $8 }' ) + sum_size=$( grep "^ Sum" $test_out | tail -1 | awk '{ printf "%.1f", $3 / 1024.0 }' ) + wamp=$( echo "scale=1; $sum_wgb / $lo_wgb" | bc ) + wmb_ps=$( echo "scale=1; ( $sum_wgb * 1024.0 ) / $uptime" | bc ) + usecs_op=$( grep ^${bench_name} $test_out | awk '{ printf "%.1f", $3 }' ) + p50=$( grep "^Percentiles:" $test_out | tail -1 | awk '{ printf "%.1f", $3 }' ) + p75=$( grep "^Percentiles:" $test_out | tail -1 | awk '{ printf "%.1f", $5 }' ) + p99=$( grep "^Percentiles:" $test_out | tail -1 | awk '{ printf "%.0f", $7 }' ) + p999=$( grep "^Percentiles:" $test_out | tail -1 | awk '{ printf "%.0f", $9 }' ) + p9999=$( grep "^Percentiles:" $test_out | tail -1 | awk '{ printf "%.0f", $11 }' ) + echo -e "$ops_sec\t$mb_sec\t$sum_size\t$lo_wgb\t$sum_wgb\t$wamp\t$wmb_ps\t$usecs_op\t$p50\t$p75\t$p99\t$p999\t$p9999\t$uptime\t$stall_time\t$stall_pct\t$test_name" \ + >> $output_dir/report.txt +} + +function run_bulkload { + # This runs with a vector memtable and the WAL disabled to load faster. It is still crash safe and the + # client can discover where to restart a load after a crash. I think this is a good way to load. + echo "Bulk loading $num_keys random keys" + cmd="./db_bench --benchmarks=fillrandom \ + --use_existing_db=0 \ + --disable_auto_compactions=1 \ + --sync=0 \ + $params_bulkload \ + --threads=1 \ + --memtablerep=vector \ + --disable_wal=1 \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/benchmark_bulkload_fillrandom.log" + echo $cmd | tee $output_dir/benchmark_bulkload_fillrandom.log + eval $cmd + summarize_result $output_dir/benchmark_bulkload_fillrandom.log bulkload fillrandom + echo "Compacting..." + cmd="./db_bench --benchmarks=compact \ + --use_existing_db=1 \ + --disable_auto_compactions=1 \ + --sync=0 \ + $params_w \ + --threads=1 \ + 2>&1 | tee -a $output_dir/benchmark_bulkload_compact.log" + echo $cmd | tee $output_dir/benchmark_bulkload_compact.log + eval $cmd +} + +# +# Parameter description: +# +# $1 - 1 if I/O statistics should be collected. +# $2 - compaction type to use (level=0, universal=1). +# $3 - number of subcompactions. +# $4 - number of maximum background compactions. +# +function run_manual_compaction_worker { + # This runs with a vector memtable and the WAL disabled to load faster. + # It is still crash safe and the client can discover where to restart a + # load after a crash. I think this is a good way to load. + echo "Bulk loading $num_keys random keys for manual compaction." + + fillrandom_output_file=$output_dir/benchmark_man_compact_fillrandom_$3.log + man_compact_output_log=$output_dir/benchmark_man_compact_$3.log + + if [ "$2" == "1" ]; then + extra_params=$params_univ_compact + else + extra_params=$params_level_compact + fi + + # Make sure that fillrandom uses the same compaction options as compact. + cmd="./db_bench --benchmarks=fillrandom \ + --use_existing_db=0 \ + --disable_auto_compactions=0 \ + --sync=0 \ + $extra_params \ + --threads=$num_threads \ + --compaction_measure_io_stats=$1 \ + --compaction_style=$2 \ + --subcompactions=$3 \ + --memtablerep=vector \ + --disable_wal=1 \ + --max_background_compactions=$4 \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $fillrandom_output_file" + + echo $cmd | tee $fillrandom_output_file + eval $cmd + + summarize_result $fillrandom_output_file man_compact_fillrandom_$3 fillrandom + + echo "Compacting with $3 subcompactions specified ..." + + # This is the part we're really interested in. Given that compact benchmark + # doesn't output regular statistics then we'll just use the time command to + # measure how long this step takes. + cmd="{ \ + time ./db_bench --benchmarks=compact \ + --use_existing_db=1 \ + --disable_auto_compactions=0 \ + --sync=0 \ + $extra_params \ + --threads=$num_threads \ + --compaction_measure_io_stats=$1 \ + --compaction_style=$2 \ + --subcompactions=$3 \ + --max_background_compactions=$4 \ + ;} + 2>&1 | tee -a $man_compact_output_log" + + echo $cmd | tee $man_compact_output_log + eval $cmd + + # Can't use summarize_result here. One way to analyze the results is to run + # "grep real" on the resulting log files. +} + +function run_univ_compaction { + # Always ask for I/O statistics to be measured. + io_stats=1 + + # Values: kCompactionStyleLevel = 0x0, kCompactionStyleUniversal = 0x1. + compaction_style=1 + + # Define a set of benchmarks. + subcompactions=(1 2 4 8 16) + max_background_compactions=(16 16 8 4 2) + + i=0 + total=${#subcompactions[@]} + + # Execute a set of benchmarks to cover variety of scenarios. + while [ "$i" -lt "$total" ] + do + run_manual_compaction_worker $io_stats $compaction_style ${subcompactions[$i]} \ + ${max_background_compactions[$i]} + ((i++)) + done +} + +function run_fillseq { + # This runs with a vector memtable. WAL can be either disabled or enabled + # depending on the input parameter (1 for disabled, 0 for enabled). The main + # benefit behind disabling WAL is to make loading faster. It is still crash + # safe and the client can discover where to restart a load after a crash. I + # think this is a good way to load. + + # Make sure that we'll have unique names for all the files so that data won't + # be overwritten. + if [ $1 == 1 ]; then + log_file_name=$output_dir/benchmark_fillseq.wal_disabled.v${value_size}.log + test_name=fillseq.wal_disabled.v${value_size} + else + log_file_name=$output_dir/benchmark_fillseq.wal_enabled.v${value_size}.log + test_name=fillseq.wal_enabled.v${value_size} + fi + + echo "Loading $num_keys keys sequentially" + cmd="./db_bench --benchmarks=fillseq \ + --use_existing_db=0 \ + --sync=0 \ + $params_w \ + --min_level_to_compress=0 \ + --threads=1 \ + --memtablerep=vector \ + --disable_wal=$1 \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $log_file_name" + echo $cmd | tee $log_file_name + eval $cmd + + # The constant "fillseq" which we pass to db_bench is the benchmark name. + summarize_result $log_file_name $test_name fillseq +} + +function run_change { + operation=$1 + echo "Do $num_keys random $operation" + out_name="benchmark_${operation}.t${num_threads}.s${syncval}.log" + cmd="./db_bench --benchmarks=$operation \ + --use_existing_db=1 \ + --sync=$syncval \ + $params_w \ + --threads=$num_threads \ + --merge_operator=\"put\" \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/${out_name}" + echo $cmd | tee $output_dir/${out_name} + eval $cmd + summarize_result $output_dir/${out_name} ${operation}.t${num_threads}.s${syncval} $operation +} + +function run_filluniquerandom { + echo "Loading $num_keys unique keys randomly" + cmd="./db_bench --benchmarks=filluniquerandom \ + --use_existing_db=0 \ + --sync=0 \ + $params_w \ + --threads=1 \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/benchmark_filluniquerandom.log" + echo $cmd | tee $output_dir/benchmark_filluniquerandom.log + eval $cmd + summarize_result $output_dir/benchmark_filluniquerandom.log filluniquerandom filluniquerandom +} + +function run_readrandom { + echo "Reading $num_keys random keys" + out_name="benchmark_readrandom.t${num_threads}.log" + cmd="./db_bench --benchmarks=readrandom \ + --use_existing_db=1 \ + $params_w \ + --threads=$num_threads \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/${out_name}" + echo $cmd | tee $output_dir/${out_name} + eval $cmd + summarize_result $output_dir/${out_name} readrandom.t${num_threads} readrandom +} + +function run_readwhile { + operation=$1 + echo "Reading $num_keys random keys while $operation" + out_name="benchmark_readwhile${operation}.t${num_threads}.log" + cmd="./db_bench --benchmarks=readwhile${operation} \ + --use_existing_db=1 \ + --sync=$syncval \ + $params_w \ + --threads=$num_threads \ + --merge_operator=\"put\" \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/${out_name}" + echo $cmd | tee $output_dir/${out_name} + eval $cmd + summarize_result $output_dir/${out_name} readwhile${operation}.t${num_threads} readwhile${operation} +} + +function run_rangewhile { + operation=$1 + full_name=$2 + reverse_arg=$3 + out_name="benchmark_${full_name}.t${num_threads}.log" + echo "Range scan $num_keys random keys while ${operation} for reverse_iter=${reverse_arg}" + cmd="./db_bench --benchmarks=seekrandomwhile${operation} \ + --use_existing_db=1 \ + --sync=$syncval \ + $params_w \ + --threads=$num_threads \ + --merge_operator=\"put\" \ + --seek_nexts=$num_nexts_per_seek \ + --reverse_iterator=$reverse_arg \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/${out_name}" + echo $cmd | tee $output_dir/${out_name} + eval $cmd + summarize_result $output_dir/${out_name} ${full_name}.t${num_threads} seekrandomwhile${operation} +} + +function run_range { + full_name=$1 + reverse_arg=$2 + out_name="benchmark_${full_name}.t${num_threads}.log" + echo "Range scan $num_keys random keys for reverse_iter=${reverse_arg}" + cmd="./db_bench --benchmarks=seekrandom \ + --use_existing_db=1 \ + $params_w \ + --threads=$num_threads \ + --seek_nexts=$num_nexts_per_seek \ + --reverse_iterator=$reverse_arg \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/${out_name}" + echo $cmd | tee $output_dir/${out_name} + eval $cmd + summarize_result $output_dir/${out_name} ${full_name}.t${num_threads} seekrandom +} + +function run_randomtransaction { + echo "..." + cmd="./db_bench $params_r --benchmarks=randomtransaction \ + --num=$num_keys \ + --transaction_db \ + --threads=5 \ + --transaction_sets=5 \ + 2>&1 | tee $output_dir/benchmark_randomtransaction.log" + echo $cmd | tee $output_dir/benchmark_rangescanwhilewriting.log + eval $cmd +} + +function now() { + echo `date +"%s"` +} + +report="$output_dir/report.txt" +schedule="$output_dir/schedule.txt" + +echo "===== Benchmark =====" + +# Run!!! +IFS=',' read -a jobs <<< $1 +for job in ${jobs[@]}; do + + if [ $job != debug ]; then + echo "Start $job at `date`" | tee -a $schedule + fi + + start=$(now) + if [ $job = bulkload ]; then + run_bulkload + elif [ $job = fillseq_disable_wal ]; then + run_fillseq 1 + elif [ $job = fillseq_enable_wal ]; then + run_fillseq 0 + elif [ $job = overwrite ]; then + run_change overwrite + elif [ $job = updaterandom ]; then + run_change updaterandom + elif [ $job = mergerandom ]; then + run_change mergerandom + elif [ $job = filluniquerandom ]; then + run_filluniquerandom + elif [ $job = readrandom ]; then + run_readrandom + elif [ $job = fwdrange ]; then + run_range $job false + elif [ $job = revrange ]; then + run_range $job true + elif [ $job = readwhilewriting ]; then + run_readwhile writing + elif [ $job = readwhilemerging ]; then + run_readwhile merging + elif [ $job = fwdrangewhilewriting ]; then + run_rangewhile writing $job false + elif [ $job = revrangewhilewriting ]; then + run_rangewhile writing $job true + elif [ $job = fwdrangewhilemerging ]; then + run_rangewhile merging $job false + elif [ $job = revrangewhilemerging ]; then + run_rangewhile merging $job true + elif [ $job = randomtransaction ]; then + run_randomtransaction + elif [ $job = universal_compaction ]; then + run_univ_compaction + elif [ $job = debug ]; then + num_keys=1000; # debug + echo "Setting num_keys to $num_keys" + else + echo "unknown job $job" + exit + fi + end=$(now) + + if [ $job != debug ]; then + echo "Complete $job in $((end-start)) seconds" | tee -a $schedule + fi + + echo -e "ops/sec\tmb/sec\tSize-GB\tL0_GB\tSum_GB\tW-Amp\tW-MB/s\tusec/op\tp50\tp75\tp99\tp99.9\tp99.99\tUptime\tStall-time\tStall%\tTest" + tail -1 $output_dir/report.txt + +done diff --git a/tools/benchmark_leveldb.sh b/tools/benchmark_leveldb.sh new file mode 100755 index 00000000000..7769969809f --- /dev/null +++ b/tools/benchmark_leveldb.sh @@ -0,0 +1,185 @@ +#!/usr/bin/env bash +# REQUIRE: db_bench binary exists in the current directory +# +# This should be used with the LevelDB fork listed here to use additional test options. +# For more details on the changes see the blog post listed below. +# https://github.com/mdcallag/leveldb-1 +# http://smalldatum.blogspot.com/2015/04/comparing-leveldb-and-rocksdb-take-2.html + +if [ $# -ne 1 ]; then + echo -n "./benchmark.sh [fillseq/overwrite/readrandom/readwhilewriting]" + exit 0 +fi + +# size constants +K=1024 +M=$((1024 * K)) +G=$((1024 * M)) + +if [ -z $DB_DIR ]; then + echo "DB_DIR is not defined" + exit 0 +fi + +output_dir=${OUTPUT_DIR:-/tmp/} +if [ ! -d $output_dir ]; then + mkdir -p $output_dir +fi + +# all multithreaded tests run with sync=1 unless +# $DB_BENCH_NO_SYNC is defined +syncval="1" +if [ ! -z $DB_BENCH_NO_SYNC ]; then + echo "Turning sync off for all multithreaded tests" + syncval="0"; +fi + +num_threads=${NUM_THREADS:-16} +# Only for *whilewriting, *whilemerging +writes_per_second=${WRITES_PER_SECOND:-$((10 * K))} +cache_size=${CACHE_SIZE:-$((1 * G))} + +num_keys=${NUM_KEYS:-$((1 * G))} +key_size=20 +value_size=${VALUE_SIZE:-400} +block_size=${BLOCK_SIZE:-4096} + +const_params=" + --db=$DB_DIR \ + \ + --num=$num_keys \ + --value_size=$value_size \ + --cache_size=$cache_size \ + --compression_ratio=0.5 \ + \ + --write_buffer_size=$((2 * M)) \ + \ + --histogram=1 \ + \ + --bloom_bits=10 \ + --open_files=$((20 * K))" + +params_w="$const_params " + +function summarize_result { + test_out=$1 + test_name=$2 + bench_name=$3 + nthr=$4 + + usecs_op=$( grep ^${bench_name} $test_out | awk '{ printf "%.1f", $3 }' ) + mb_sec=$( grep ^${bench_name} $test_out | awk '{ printf "%.1f", $5 }' ) + ops=$( grep "^Count:" $test_out | awk '{ print $2 }' ) + ops_sec=$( echo "scale=0; (1000000.0 * $nthr) / $usecs_op" | bc ) + avg=$( grep "^Count:" $test_out | awk '{ printf "%.1f", $4 }' ) + p50=$( grep "^Min:" $test_out | awk '{ printf "%.1f", $4 }' ) + echo -e "$ops_sec\t$mb_sec\t$usecs_op\t$avg\t$p50\t$test_name" \ + >> $output_dir/report.txt +} + +function run_fillseq { + # This runs with a vector memtable and the WAL disabled to load faster. It is still crash safe and the + # client can discover where to restart a load after a crash. I think this is a good way to load. + echo "Loading $num_keys keys sequentially" + cmd="./db_bench --benchmarks=fillseq \ + --use_existing_db=0 \ + --sync=0 \ + $params_w \ + --threads=1 \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/benchmark_fillseq.v${value_size}.log" + echo $cmd | tee $output_dir/benchmark_fillseq.v${value_size}.log + eval $cmd + summarize_result $output_dir/benchmark_fillseq.v${value_size}.log fillseq.v${value_size} fillseq 1 +} + +function run_change { + operation=$1 + echo "Do $num_keys random $operation" + out_name="benchmark_${operation}.t${num_threads}.s${syncval}.log" + cmd="./db_bench --benchmarks=$operation \ + --use_existing_db=1 \ + --sync=$syncval \ + $params_w \ + --threads=$num_threads \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/${out_name}" + echo $cmd | tee $output_dir/${out_name} + eval $cmd + summarize_result $output_dir/${out_name} ${operation}.t${num_threads}.s${syncval} $operation $num_threads +} + +function run_readrandom { + echo "Reading $num_keys random keys" + out_name="benchmark_readrandom.t${num_threads}.log" + cmd="./db_bench --benchmarks=readrandom \ + --use_existing_db=1 \ + $params_w \ + --threads=$num_threads \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/${out_name}" + echo $cmd | tee $output_dir/${out_name} + eval $cmd + summarize_result $output_dir/${out_name} readrandom.t${num_threads} readrandom $num_threads +} + +function run_readwhile { + operation=$1 + echo "Reading $num_keys random keys while $operation" + out_name="benchmark_readwhile${operation}.t${num_threads}.log" + cmd="./db_bench --benchmarks=readwhile${operation} \ + --use_existing_db=1 \ + --sync=$syncval \ + $params_w \ + --threads=$num_threads \ + --writes_per_second=$writes_per_second \ + --seed=$( date +%s ) \ + 2>&1 | tee -a $output_dir/${out_name}" + echo $cmd | tee $output_dir/${out_name} + eval $cmd + summarize_result $output_dir/${out_name} readwhile${operation}.t${num_threads} readwhile${operation} $num_threads +} + +function now() { + echo `date +"%s"` +} + +report="$output_dir/report.txt" +schedule="$output_dir/schedule.txt" + +echo "===== Benchmark =====" + +# Run!!! +IFS=',' read -a jobs <<< $1 +for job in ${jobs[@]}; do + + if [ $job != debug ]; then + echo "Start $job at `date`" | tee -a $schedule + fi + + start=$(now) + if [ $job = fillseq ]; then + run_fillseq + elif [ $job = overwrite ]; then + run_change overwrite + elif [ $job = readrandom ]; then + run_readrandom + elif [ $job = readwhilewriting ]; then + run_readwhile writing + elif [ $job = debug ]; then + num_keys=1000; # debug + echo "Setting num_keys to $num_keys" + else + echo "unknown job $job" + exit + fi + end=$(now) + + if [ $job != debug ]; then + echo "Complete $job in $((end-start)) seconds" | tee -a $schedule + fi + + echo -e "ops/sec\tmb/sec\tusec/op\tavg\tp50\tTest" + tail -1 $output_dir/report.txt + +done diff --git a/tools/blob_dump.cc b/tools/blob_dump.cc new file mode 100644 index 00000000000..73601f2d80f --- /dev/null +++ b/tools/blob_dump.cc @@ -0,0 +1,89 @@ +// Copyright (c) 2011-present, Facebook, Inc. All rights reserved. +// This source code is licensed under both the GPLv2 (found in the +// COPYING file in the root directory) and Apache 2.0 License +// (found in the LICENSE.Apache file in the root directory). + +#ifndef ROCKSDB_LITE +#include +#include +#include +#include + +#include "utilities/blob_db/blob_dump_tool.h" + +using namespace rocksdb; +using namespace rocksdb::blob_db; + +int main(int argc, char** argv) { + using DisplayType = BlobDumpTool::DisplayType; + const std::unordered_map display_types = { + {"none", DisplayType::kNone}, + {"raw", DisplayType::kRaw}, + {"hex", DisplayType::kHex}, + {"detail", DisplayType::kDetail}, + }; + const struct option options[] = { + {"help", no_argument, nullptr, 'h'}, + {"file", required_argument, nullptr, 'f'}, + {"show_key", optional_argument, nullptr, 'k'}, + {"show_blob", optional_argument, nullptr, 'b'}, + }; + DisplayType show_key = DisplayType::kRaw; + DisplayType show_blob = DisplayType::kNone; + std::string file; + while (true) { + int c = getopt_long(argc, argv, "hk::b::f:", options, nullptr); + if (c < 0) { + break; + } + std::string arg_str(optarg ? optarg : ""); + switch (c) { + case 'h': + fprintf(stdout, + "Usage: blob_dump --file=filename " + "[--show_key[=none|raw|hex|detail]] " + "[--show_blob[=none|raw|hex|detail]]\n"); + return 0; + case 'f': + file = optarg; + break; + case 'k': + if (optarg) { + if (display_types.count(arg_str) == 0) { + fprintf(stderr, "Unrecognized key display type.\n"); + return -1; + } + show_key = display_types.at(arg_str); + } + break; + case 'b': + if (optarg) { + if (display_types.count(arg_str) == 0) { + fprintf(stderr, "Unrecognized blob display type.\n"); + return -1; + } + show_blob = display_types.at(arg_str); + } else { + show_blob = DisplayType::kDetail; + } + break; + default: + fprintf(stderr, "Unrecognized option.\n"); + return -1; + } + } + BlobDumpTool tool; + Status s = tool.Run(file, show_key, show_blob); + if (!s.ok()) { + fprintf(stderr, "Failed: %s\n", s.ToString().c_str()); + return -1; + } + return 0; +} +#else +#include +int main(int argc, char** argv) { + fprintf(stderr, "Not supported in lite mode.\n"); + return -1; +} +#endif // ROCKSDB_LITE diff --git a/tools/blob_store_bench.cc b/tools/blob_store_bench.cc deleted file mode 100644 index 60a0b84a63e..00000000000 --- a/tools/blob_store_bench.cc +++ /dev/null @@ -1,280 +0,0 @@ -// Copyright (c) 2013, Facebook, Inc. All rights reserved. -// This source code is licensed under the BSD-style license found in the -// LICENSE file in the root directory of this source tree. An additional grant -// of patent rights can be found in the PATENTS file in the same directory. - -#include -#include -#include - -#include "rocksdb/env.h" -#include "util/blob_store.h" -#include "util/testutil.h" - -#define KB 1024LL -#define MB 1024*1024LL -// BlobStore does costly asserts to make sure it's running correctly, which -// significantly impacts benchmark runtime. -// NDEBUG will compile out those asserts. -#ifndef NDEBUG -#define NDEBUG -#endif - -using namespace rocksdb; -using namespace std; - -// used by all threads -uint64_t timeout_sec; -Env *env; -BlobStore* bs; - -namespace { -std::string RandomString(Random* rnd, uint64_t len) { - std::string r; - test::RandomString(rnd, len, &r); - return r; -} -} // namespace - -struct Result { - uint32_t writes; - uint32_t reads; - uint32_t deletes; - uint64_t data_written; - uint64_t data_read; - - void print() { - printf("Total writes = %u\n", writes); - printf("Total reads = %u\n", reads); - printf("Total deletes = %u\n", deletes); - printf("Write throughput = %lf MB/s\n", - (double)data_written / (1024*1024.0) / timeout_sec); - printf("Read throughput = %lf MB/s\n", - (double)data_read / (1024*1024.0) / timeout_sec); - printf("Total throughput = %lf MB/s\n", - (double)(data_read + data_written) / (1024*1024.0) / timeout_sec); - } - - Result() { - writes = reads = deletes = data_read = data_written = 0; - } - - Result (uint32_t writes, uint32_t reads, uint32_t deletes, - uint64_t data_written, uint64_t data_read) : - writes(writes), reads(reads), deletes(deletes), - data_written(data_written), data_read(data_read) {} - -}; - -namespace { -Result operator + (const Result &a, const Result &b) { - return Result(a.writes + b.writes, a.reads + b.reads, - a.deletes + b.deletes, a.data_written + b.data_written, - a.data_read + b.data_read); -} -} // namespace - -struct WorkerThread { - uint64_t data_size_from, data_size_to; - double read_ratio; - uint64_t working_set_size; // start deleting once you reach this - Result result; - atomic stopped; - - WorkerThread(uint64_t data_size_from, uint64_t data_size_to, - double read_ratio, uint64_t working_set_size) : - data_size_from(data_size_from), data_size_to(data_size_to), - read_ratio(read_ratio), working_set_size(working_set_size), - stopped(false) {} - - WorkerThread(const WorkerThread& wt) : - data_size_from(wt.data_size_from), data_size_to(wt.data_size_to), - read_ratio(wt.read_ratio), working_set_size(wt.working_set_size), - stopped(false) {} -}; - -static void WorkerThreadBody(void* arg) { - WorkerThread* t = reinterpret_cast(arg); - Random rnd(5); - string buf; - vector> blobs; - vector random_strings; - - for (int i = 0; i < 10; ++i) { - random_strings.push_back(RandomString(&rnd, t->data_size_to)); - } - - uint64_t total_size = 0; - - uint64_t start_micros = env->NowMicros(); - while (env->NowMicros() - start_micros < timeout_sec * 1000 * 1000) { - if (blobs.size() && rand() < RAND_MAX * t->read_ratio) { - // read - int bi = rand() % blobs.size(); - Status s = bs->Get(blobs[bi].first, &buf); - assert(s.ok()); - t->result.data_read += buf.size(); - t->result.reads++; - } else { - // write - uint64_t size = rand() % (t->data_size_to - t->data_size_from) + - t->data_size_from; - total_size += size; - string put_str = random_strings[rand() % random_strings.size()]; - blobs.push_back(make_pair(Blob(), size)); - Status s = bs->Put(Slice(put_str.data(), size), &blobs.back().first); - assert(s.ok()); - t->result.data_written += size; - t->result.writes++; - } - - while (total_size >= t->working_set_size) { - // delete random - int bi = rand() % blobs.size(); - total_size -= blobs[bi].second; - bs->Delete(blobs[bi].first); - blobs.erase(blobs.begin() + bi); - t->result.deletes++; - } - } - t->stopped.store(true); -} - -namespace { -Result StartBenchmark(vector& config) { - for (auto w : config) { - env->StartThread(WorkerThreadBody, w); - } - - Result result; - - for (auto w : config) { - while (!w->stopped.load()); - result = result + w->result; - } - - for (auto w : config) { - delete w; - } - - delete bs; - - return result; -} - -vector SetupBenchmarkBalanced() { - string test_path; - env->GetTestDirectory(&test_path); - test_path.append("/blob_store"); - - // config start - uint32_t block_size = 16*KB; - uint32_t file_size = 1*MB; - double read_write_ratio = 0.5; - uint64_t data_read_from = 16*KB; - uint64_t data_read_to = 32*KB; - int number_of_threads = 10; - uint64_t working_set_size = 5*MB; - timeout_sec = 5; - // config end - - bs = new BlobStore(test_path, block_size, file_size / block_size, 10000, env); - - vector config; - - for (int i = 0; i < number_of_threads; ++i) { - config.push_back(new WorkerThread(data_read_from, - data_read_to, - read_write_ratio, - working_set_size)); - }; - - return config; -} - -vector SetupBenchmarkWriteHeavy() { - string test_path; - env->GetTestDirectory(&test_path); - test_path.append("/blob_store"); - - // config start - uint32_t block_size = 16*KB; - uint32_t file_size = 1*MB; - double read_write_ratio = 0.1; - uint64_t data_read_from = 16*KB; - uint64_t data_read_to = 32*KB; - int number_of_threads = 10; - uint64_t working_set_size = 5*MB; - timeout_sec = 5; - // config end - - bs = new BlobStore(test_path, block_size, file_size / block_size, 10000, env); - - vector config; - - for (int i = 0; i < number_of_threads; ++i) { - config.push_back(new WorkerThread(data_read_from, - data_read_to, - read_write_ratio, - working_set_size)); - }; - - return config; -} - -vector SetupBenchmarkReadHeavy() { - string test_path; - env->GetTestDirectory(&test_path); - test_path.append("/blob_store"); - - // config start - uint32_t block_size = 16*KB; - uint32_t file_size = 1*MB; - double read_write_ratio = 0.9; - uint64_t data_read_from = 16*KB; - uint64_t data_read_to = 32*KB; - int number_of_threads = 10; - uint64_t working_set_size = 5*MB; - timeout_sec = 5; - // config end - - bs = new BlobStore(test_path, block_size, file_size / block_size, 10000, env); - - vector config; - - for (int i = 0; i < number_of_threads; ++i) { - config.push_back(new WorkerThread(data_read_from, - data_read_to, - read_write_ratio, - working_set_size)); - }; - - return config; -} -} // namespace - -int main(int argc, const char** argv) { - srand(33); - env = Env::Default(); - - { - printf("--- Balanced read/write benchmark ---\n"); - vector config = SetupBenchmarkBalanced(); - Result r = StartBenchmark(config); - r.print(); - } - { - printf("--- Write heavy benchmark ---\n"); - vector config = SetupBenchmarkWriteHeavy(); - Result r = StartBenchmark(config); - r.print(); - } - { - printf("--- Read heavy benchmark ---\n"); - vector config = SetupBenchmarkReadHeavy(); - Result r = StartBenchmark(config); - r.print(); - } - - return 0; -} diff --git a/tools/check_format_compatible.sh b/tools/check_format_compatible.sh new file mode 100755 index 00000000000..801648963ec --- /dev/null +++ b/tools/check_format_compatible.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# +# A shell script to load some pre generated data file to a DB using ldb tool +# ./ldb needs to be avaible to be executed. +# +# Usage:

    `sL$+BpHsL z7>z2ZHT_;#*p5-E>A`|_wR=$^$-pB%MEDkOgU}AjW)GuTwpi?|uv%|#vMvkFoUrdv zagUBm&Ne`7n@>Yu-6NL1*@~dzg=h6;&kiQ)%oo4)P}bP*8Cn#`L7ok)v&g_!Q;M}h zP^_)O*M=7Sr+({UDaJ--I4uq^W6O<*%5~G>!#KGNGvX%+%nHX2#?hb?=;XB2LN$fW z$d-D<^8jwIMX3_!P-%?im1H~O9U2?2g1o#Z*+|44 zCGtVZwv&H+RLql7zC=I2Sp48bf8LFEmIjvBbqjV zm0$*L^#w8gM8!;~j2Mj(2gH;>u+r{(S-kS~gfc|XIOf#V!Uakao#e`LF( zk|+?$OILPSNCJ$C8V#D=f$l`-u+B*V_q$6U6l-A&V(fW^p!WJy!~y2sIS=jHTd41X z$4x)&Ljy{fXZOVrBLsmK!R4%Rxm(8dsxAsgBuz7?Wma9eGI&ElDb@asK~Y2}Nkef6e}M{J>>7TGA>5_`hhD1QPp7NK0qw1gDYLY}lrk0X zx~eU=Clk!A0c`8?mN4vP66hEaaNRkxsKoSI*q>*x_2MofOmvjQ1MKH!f6T@&r)hq! z9MHf@t?TI9JENwlIbn(N&Ss8itPDejfv*<5#5b$)NRiy;K_{cl!SWkKrD zD(M0N35l8!#ym{8*G(EmWnW-&za!kJ5J5yxwA=L+{0*y6WSH4mE}*&Usd1k$z#OvS zx)E^;8n$09DcH;jfE~2%Frj#1opl*Y< z;HLR8-4a11smFdz)PEDnxo*{+n&i=%b6Ph@3^q}Gn=@@CTP$y;SP(Hc#rK_Y^T^ESKXKZ@hB zYYauM&K;cYy}P*puvf-7i5FlSuLDX}sf@V*>6wzce%1`0#nBTzRqw&HNi&_ke<;cD zhjL6z9x{MUDdEKq8d88aUX`P5q@Ok>x~B}f3(_!k%K#?eFEZE$zh_kcQQXYd_q9jc z$e^Pv%d`Rot>d6BLZH8S>^oi~&{=@j^ORSmt%D&gTdwM6R%jILFu3lamdJq9UeF~- zom>F><1U63o-4@yfTI}8K63|1x;?uZ6sb%V5tlvI67#&;KH!3UZ?%~Z63oPCP|kqy zcXNG=Us5SqF#%4A40LeK?`f(XMDw_jl-y_;_&<#|w2bRHa92nhW1F;go|1xTC(4~Q zNrD*s3qLc;x>7(O*(GOe0FS>l3e9Wwix0qCx+@4(A%QuQVvWVv<9a*!}RBIz{)2M_vI9>nZ?du4hdrz2Scjp2Lc@y z%2)oB8i7sVvZ|Ie-SktMbz5H1SU*UXmf?QQ3EPm{tA;7v4*&&2U=}V*)?75Gu#ChS z_G!4xxUCKom5!QrSWQ~1ZO4FQKJ~eW*t4k5u2l`~zM8Ex{n>bAItGf`R*WF!%Ec(5 z&i+#7`RwQ`K6h~np;76|F9q5pd^8#J0*4^#GUu59`%|g9_bxBI{>Wu7y*92>u1&7& z^#%h@Pu_5eg%8I-j1$YrI(fimOJ2$?(ytpYgX8X8_c&F;D1r|j=`cc(W33L&An{fwr$(CZF4W%wr$(CZF{}_oqNx#_qtM*s^mu}opdMF zJ?9vdZ!hkI8P+#e#4a)XH3U<=7(UY$3xJ}G$^MFt%_X9RoU^{vqb?c`;pechPFAO#vL;=^3f(BlFp6It^GX{NL)Fs;%NA6thI^P=X0pHvj|!sOijo z^GGO)QFh808GtcDpl#H;gZA5nk0+Q+ z#vzKNGvg_aGCP_M{SyNF0Rt8U%l)K~r{#n(gQ3DvVW6_!G7mP~ahh zn%1pi`DstIlanRXxUMGs>kh(65c8yJPANwJElh$AmJH4jF~HOu)zXvHuv+nyVTZBw za^sNKxFw?)PsaGbz5k2J;m_e)y_0ceif<=H#mUbH+VK;eo$Of^qgO6I13c-N2!`S1 z-)sE)nitdCH*c57*gh28n3TXASb*)0Raj){Pjs(5E%(_)f5@~(ZoUD9xEg2@?boJhACZ7p7g{;qgkdY5|l=dt4f0 z9AT!8P|zNimM#E7_+8h?5ebv|PA)Tpcrjc+Fp2nFbPyQvkHS*uGG;3O)m(9fz(OMm zFxdDP(Yn!_==G8l;vLrV2ed3H&Cncj-EUB(I5e~Ts2@20m@rQ!rX?fH1Cei77^R{I zdj=FuSTfs(ryMPU40ni$og0KPmm^~9g1q8ud^$^lPHs`4^9h8-YF)5peMF$!&9ITV zLqJqVz&u)#+fXZ7y`MyWYWKCfn)$Wc&Q?0QQ_SO~EFo#}y@#nvEoaWFLy4KZ2K8);;K8`8>uewOPabxVmULO-|n$YW?9~6xrZqO{%dbFCIyTJm?zQ+d%?}|=!no06Jw;nT9`8?ex4?( zokITI;@iQ0F}0P+VK}!Z1*d-e)z_Cm)pQE4m1+LyAzSyyz<_j^zD!q4kip^AO}AoD z6Fg|2nE29X4zxAo(nD~NcPPgcPzl1i^it+uRc$HHdrHpTxHZ%jt-mt@yJg0P zhb|KDL@&9O1_9}JI<}Vmt32k%FpwE<4Ff*=w?Rqe4|K6jGIcr%KkD@<8&nmX?*pE; zd?r-ECo!{zPI-{{?$DY_{ymdM5Lowj9K4eR$E0^tVr}58d%`rW z);TtY5M4BjsC|THW5`@Aloe3&U=dQ1BCFy?IrXM&X$EVY3ClD_Qv_wa+ODlpF%T4T zCY;dRuA%bd{+o;0R6COULN}t*+a3&W#1V)tB%Q1!*15yIN@70gK1l7B71of^lZj2^ zg$FTye5n0Pk0{B88ATxJd)~Kj5&bW#P|yjyk6xV_X8U2`r8AU2ukhFx=FF9Uc{k0> zeo>B(gt+{&Sx|GtGWF6trA@RUMYuV`ftHv;9vNMhAen-(*>^!#cSUE@Bst#Q2o`5? z=&P1aW?{22q2t^*XeQe6$tq{-kfY7y2Wm-0cC$2VNv}?OOi}#E4#{aJn%phR2T=^K&)OedFa}YjJfm#Z9(# zXU*{)wow^Lx47mx651ph7tJ=_Q_0a>iz|R9LRiZLE*^XMoIPnEW{4PirG+6{4O)6B z5?yQoF;>e>UXEn;9NBE~k~ort+|sP%mXGIPD$>j{=p?eX2dLwzfqo+#_qNePMvl7u z*-$MN*Js#u?16PF+g!TrZ1EWHf)iIXaBgzYyo9&|4D@go0v4eMZ|91d7AmXaArKT zeIzJ5Ys2i~z|GiK_(L;jAK#tir&nSZl9%0H1*;Lio|~H@X2G%8TqcB&C9|Y(H$QcA z5b}PKwAM&zHfWfg5}^zX@Ea|lDGzDR$D7P?&M46AFQl<;&*xV(0bgXIqyvAAe zK5Pr2S!Xz@yuQ=9dR51Mn+bM`(8P;Y#YGXM<0uPF< z+f8y%(exf{w^)y-z6*}wIfA{NhOkCVw($MJBcW4PxJes3N|Q=yh|w~i@3KQnCx3TU z)gaB&fN5p*iiUACX+z(Q%)Q6TShIthBUezuR+fyG&MRmdu15aH`H1Wg0w3fo+Fh96PNkMRJ4h$ zs8Z$@lxlew+#kgzJb08GZQuoR4+OJN-%U&u*K^prHX6}u#&ni#hC8L&FOYRQn85T< z&~^j@O29ctg_chT(Eqt!V5+l2{iv|c_oT&qh^SG70$>Tx!@C#z^gJA@YA-fTz* zlDokHzEv|#5Zz}Jfhf=C7}0ZoY_>tRJikieKx_3>IONqzyMZNS%CEK<9^}f;6Y!1D zkWz@Qf~YQ$6OMBoA7-b_TGxMR#vA?BU_*)ds^uAM@Pr5B3z^r}r!Hd3WV0I@Rv{zi ze^EKLSV7mS7nO9N3-6mJi>fG@MEV!RIytwKooPgZ4rew&A!(2)&RPdwdH3mLw=XaY-L5-|krpjN6T%3w z<9?SgmL{FR%d}dTjP3`k*<;bETj58DYRil(vyOLo#YL{uL-zaoE$QU*d^^`X!x0mx ziri%d#SCbyBBcc#>hexV!7&vf0x$zzI)W37y>5tD;%<#Ks1+-}og`rn6R+hQb#YX* zQseZa!TDD)XR|5?a2o=VpUc%kE`M|2K(DiOi&K7I^^Lx8SFkm~2>GFw%xzoM^~Srs z@sZwdstPa;MTah}FYyok-)5^Y|9;~?12@Pezefa4mT6plO;(pgCxzAiF-oJ{?Kt(y-SB_g(P-b~*_;3o?DfV$ zSL;Uiyq!mA3RHMd6>=?$y&S*x=Pp(T7b9;6f6X?oohoie$yDr2UwT$-jgvmoDMF8K zH@@-hy2bfxaFf(7*H-Ih9r%WX2SE^s3}FAl<|C5n+mNmS83i2SvuIb60Yn&>)X@z% zAT13DYvN_x!GvGp`~Ua?I6Tb1j!MsN z9Ln-qI|aG+!e`O(2C29~H=6Rv2ys8Ho2OiSq_T&7Mw=s!vN|LWn+qVJTT%{b-#zLmv?YXcwDR#)Umzx)o=EqBgI+E;^Nia)6U6P!1JIoi$k`zU zj-xEQ~fb zRs2HHaX4Z{nt)|@;)4IR)ES{l(K1_AZd+HYGgqhQr9-5O*AMYzS#L23#|$dJ+;b6j zV~I2;&9a4&OYtL-^G+3`d7K>oh0{8-JSDP_JAnUZvw+YivDi|mWu8j-2kswrq4 z{xhe5-Cs}S_;T>S-$5f*)W7&m(SowezuE=8+jyw#&lqg9c{INE)8GAF_D9vELHQI} z_LFTV&v~2;Be*x)M+A9t}$YDSkctq6+AO!zT=R-C5ha#{_5Qi1XY8ZLi z0$TEOBq6ArY>p=~G*P@DVD6w$6y#59hvr#FJ(Vs2R~CZ6B0mdaMW%L~m3k@VIqrh{ zjoxpeFf9<}oEzm7-8+!pd^XE$|Ix~3j?Xq5oeySl_u#GBwb^FPfUXUU@S*OA6^|U( zi4QBPw^CxRXi_b|(~ULGZK^U7QnNQfR4Xl28EP2nP8!IV3QF!D%^OxJ2X9`k*5ClL zf*}()ftBGqn;NO<@xU^xl|RtZfh zBDT>D8@O^=;{vfpb$4`*r3jn)+xTWNVgbjsLttbHw|*IOrJlO_O{sZWLydCfX2Ky! zb}U@+6-|^Jqiuf5A3?qk=Z;wYW+8Zy#YfqQ#dSqDFmqYW+4LA{x?ZpJ{j69ji@S@p zhdkbRW2X8dU(dWIKdTmJi;R*kt7thOx~ArvYe+S&4&$w4?h>|=4Ag`t=@m1$bG6#X z*x!DEjh|;=@zN8cgY(IDvT-}trg>_^LbP>P)8`K&0rzBORxzKdqQlFH->*|b%D+U= z1*A`R*HC&jYa+Fj$OY7S7nU;&pPLi3%bGRe&zM)6bFxqzg;DUyP5#Cv#1zi#A}7>A zCZwHT>`?N)ui>xDPM%#Gz)l4k z1lxw*L60uHL|7g~yKI&>T!Pbs5^%zsNq~E)7?^E>RaJxbgcIfLoV6PY4Gj4ZKJyBU zDaD9vhJvBPouRXe7ws+#4MbtejafTvW8Dkt!kmip^;DgTjd&GBfovK`U%ZMFnhg4> znB>swLhA5~f}M0()`8Zx3BAcx`&lh9p3Xo8aLy$P9Yq%tMz)Vm1L?D3N)`1C!h`QL zLI*0bK5Ou!`{)Rcv3hh@h>f|wIab0=5jNZj8V_C}GI80|l)Ez3hc11=05AF%o!euz zv~RA&b{uC2YHgGbYr9~G)M}(DK>`CAIt4war2sD;<4ZjiKF9Z?tVK_v7Xg_JIfXh~ zcg!9hsn&n0XDi9Ij~%Q+3SjBzq%l`o(l=LJZ?0&#^$jfi=2oIf+Cem^r;=;W#uuXC(k zGult_qfyaq#^qhBTgVYXxGCnUa`$xcUZ zE&aK!B|o-)IADAyeXW+gwVk1Pts;BG!)&T-4FUyRw&WZRlPr4hcir|c-1ffPjU!D{Lm|^6NgnHX$KOf*o?e%pP4yYFag?!3W}*?xv&Af&rR-T@ zr3=D3fIq|2t24vA1f4*0UkPg23AnTk1O;ZziimdJa8w-I_nfjyU8Ag#Wnc>yrA#dm zl>d!`T^M3OH_6j3Nbb#KhCq;yBv)9Kl6mi~>z4fV<|r3++MESMga=dgSX=*ck`7XKr)&Dg3375SVJ?}9Zn)8mbGrirlu~2Uy2Ty_HGjWPt zglgwONC#*`l==A6+F%H(a4m?WKw;q`J2%y3t0`VRIob}oPm_+rmNre&!VPAQH6^~<4? z7cK_Pf^nk|m4m0=R5Ns=8y*?U+j66mjge0&O*ggltY3V&tIh(OiS*b=NqQJoZSDU_Ob!MftqYK$(={tk(_Rz}N65?{6Evo(JUE8tS|hz{q^i zT+w$L>hd0KAM&YN_b9aW`uh&|Y~tsH7N94dG6w=o++hV;)y!3ll{Aaiv&M_or zPh92 z2*}kcYLrKa<0;w>8g=#cK%U(lsKn`K+Y5>oK(w26xBdp4PD`>-2$(M53z=Q1(1E8c z<-F;u2@h}A6Sby1YMO-pEEGfZ5yiNo^agYm&vkz?qI7qHwT2MF^5IcQl7diXxlNUi z6Hh+AF1ch43gGX&zS4AxZ74BvYtjH8u5LX?0ZvF zY4}Hk)_e1ne_cy%hq9g=&n^o%h>1x>*=6sYQ@^5)tl(>6>9dcrj>|;!bOvK(<$2*b z?4otZ>1fq*XwJOKt3p7N)*#jr{M^%WnnosMbFszXb0|u^ufJ_s|5Xc`(do~IbIw4C zu6fySZL2nmPGcwy|BbHnFylEO)~0DkmGNSpz<+I z%TBJ~l^8s0SzH)>2T7=*mYKE;L+HfP6x#bJ2-TU}<-?PDq z_Sq|<-wW%`v0nGd`~oVJg1u*JMEg2Y3DQY*(kJTEgK9T3lE@Ka;rmSgQRi5eSA;K= z%V5WT=n6(Ie|yBR#e~koJR?Rjp(_nU1h^k`S~B#bI;exO`O_@iDi(P>!l0_C!$$}a zfh*oeDBK@Vm^`bf&2RjZ%jEne7@Z|aYBqxB-YgU=XN4vf0x~KU2@?ZKmxmi=m2u|J zCT)AvgU*u*f{a1>;cqOW!J^Uz;y+SNL>owSzi4CAO zxdhSV(>d zhNjC5WN_}#o3>~i=b6M{7Ezp>TCTRh$=4PYlyEV$B^Epbt{k1tl9am=7=JSo_2kcV zmS-%>{i$F&!sg*{(a=;lSvgNBZbz6>x&Nijyg=Tz-rMZ9dvFDqow%B^b=ipGTZNJ5;UrFl-DOh>H-Y7{Gd0j^1*uG6v5|EPFWh9UX8y++T%VwHB$E zVd*;&ycQ_a=c;0BI^&@=XdAnw&BERX=8WFoP)uvIV46=nEQ#%A(?$M#*++F;d($}tb3Px~mv}Qtf#46XE&=mFy|4CH z=8GP%ClD$=Z3zE^?a+5skk&#b(9A;Iy7QEJRPpQA-2N(-Q%VV@Nc?Dczecw}bcA+` zD@u@*;c;PvC`xVLiM@TD!YlJd@J<8-p(N)Bh0NrXRQb-`u*1y(DNN}jy?r$UM@Ml> zMM-B%M^SS@M@MM^bg#BlXwysxa5m)f)<69ihtvMv27l6xg^mR(FR%o!+sgITz{Zbh z6BmkJZ4pEv2=E_Lpbt7IA3x$!J4Y59K45Q{BaKC(tA8}iLk4^8tL59_!XKtYkNMO9 zpU}Kt?y9i!R3*(aW@vMk76;hy0MJtp1vDc046z=Ig<*eu)!SU$vxk?~mWU5*=!*b4 z`PLmlSYVDy>kwJpVReL1P+bhSZQ;O?Yn4zzFSZ2?WC$F~b=Vjv6u!eCx01Kjhhll8 zE{^4nY|tO?Jmp#yh|YF+E!WVXHIf&s?GH~1DsdMF4InRM<Xgywfm52v*rq;I}^xUTWa}rj~o+NBL@& zAlI3pp9~ZXa95Pk(u7XU&4--px`_JbGSj`~qw;N3qI(CA7{xdcK;yYfO0|vwa`osnhI# zK4)o5XdM#lx;E-5G{OB=?(ndGO7=0fel-rU*ts9&0X=$txOHBii4i`duzUyFiYKe_ zgtdV8v=M`(Y};_tJ~M|dLjbTKSEQT%kmNJhN!&{v<)mcbNlN;biRe(WxeOTI2#fk5 zw6d+kMdzMF&~tz3G*aldRM_HO>MoXim2QsMMo?0L3raJ+s%|U|b*Q!4wM?MQec@rl z6R8h=%BBy%GdJU*a^J42yv(K#^4Z~GJC9)e3bj^>-F1DR1&}JemG{`=e?><;Obd`i z3CZR;dQdOkKPYO+$!7UqAxR=109^<4T@y7X;^~aXry;MCd{U9GRF>>A7*vr}0if(v zUqy*xzl0LzdFM%Byy}UQzY%gU637OkkM5Zm1i|re8!zuspvPyBbyYIy@0k%0jO<(E zm~ey~J?vp6+41jD=|6eKM`|>Ksp?0|QZE_entFd9$m-Zg!nN^$ut0`mdSW2F8Im&s z41?OT=6c9xQ1_!qA2gF3OE6Zvaj2K_=NFlo(wO>uNOF`7poPt%rMMTuiX}h3c!^M64zu$MJ1^_ANfdwlz6Qn8yyI9rk{> zDn;APUC8x8asuUcqCr`&&TR4XIiQ+gJK;a({p z5PMViiUJFoY4K8VqKfN*FPSEM8!gMn^Gn>|f#p?y-N)+s!=907>`#Bq4vZ3GWu|OUnhr z+j$+H)Ka38bV_Kcs*3Aqe3H0v_2Dsj+;)9?jCm%J@L5=jve4@P`s6hRaC1Gkms7C=4dC6YF%`;|(84cI-`7Twzu~&|RK%i%Oi%dc%l$_;=!cmNAZ5NMs6U~kru55g#$FiDb8_#16h$Y7EE z0S3H;d-*THY4PzqY9yLSR))=feO<2}_CyR6XjCiJX9Yus6x(nEd3}R2w{8__YN={* z%HgthS1c(*6MKB4oc7#3#-B#XC#iwRI&g7WWIZ5h<3|?tkAlYXb%s<8CGqRTv9fKJ zk#^=$9{SHA8QLK;jP=JeC(_47OFuckvY&&cM-=~0My8*U8htcMHYU#8fyDyxw}s$? zrGen2;P;k@8ze~~M1LYC|AHX#S;I9P_9x`>-+xrl-EQNMRLKSzaxxLYnwi=12r|fx zf=Tfa?eUnn@Luk%i(7Z6%Z+hH87?DWLCT#ZRil$y7KnSEj|bIed~N(g8OyhG*!PVb zW{y(C!bF5fL72=gHJZ5ntW{FQvJi1xuKu)gd0ES4^w~Vnw`yD|3Ur?@wd*Xu5L19G zR0$!$ECXqEWjRb_GBEo}crrpK!8d13i#)+AG(&9_o>L^kNrAtr`flo^Q@OXBPa%0e zZ>?vI=o8y+)XMMP?h__q-UqFqswadBPZf-tVw#<74R1Hi|LFHWA7h!V-$f~pL z!wpUw(dIqCh$gu<@5`u_kh0=)J!+-@O*V1o#AOE2q>g9(0V@O|N;H@}ceYNKRt+$r z&swuAVFZK?llQF>-m_om;U{+>t1(+5!vHiOySNmzDo`XwQcbiehN4hx7|yYcQ#3E7 z-t8*XVLUrVpA;+arA9w5bF`fdu#Yw_I4LYZo%cIOIgikCjrcM$F0qKyYx#M(D`852 zGS#rxjf|$~Nc{OR3f(oo?&#A*7iX28tITYa6HIIs5y0(JUwguL1F<>;{u1@pOwFpb z%r*XV21)>A%;+{36^|yU6loni0wM5RV63nBKIUf3W-@A7WubW?i*F>Uqm^!{BZ(81 zZK}*T@sIgugH~JffDm;cBkUQeD&3lArTW%icrHs(x|2sB!)4|Aa8}uvnWlAOL-&rx3p{HEa<&qep2b1cdm*Bx6fo z6hfZ(ncIA@T*#=K0vY;a5Ael3?^3zJ66iNCkqEq$C*VY5A)ey$n+`G&uti_YVz z1d|k&GipnBz&EzE{Tf>`xRAZ#dAx$*fYkk=$ph?%G3`Tn=e1X+tqA<$z76-csOvSM z)-e!#e;EtlG*ybaeD`iZLq=k}fb>bb*Xh);QXetvTJORPhkNRy+3+%$KmQN9vTZj@ zSX`a&z3HKMh+&{!J?bt-pJBTAlkkvLRMaetw`a%s#fc$<{UD%cG~mfCdL0E^0p{03 zCW%*b8EbiZYE-R)nHrIKR)ug(yfK%tTZzY+jaExdhdrYX72cpKYTZhvNgIKQm&oAg zY0B@BC%%6wKFlv+2cS+(>FcGt1S(;1_CGlN&$@EqQmC zqlLb%HtorN@-tx1P%<3JU+-bgz(Gme$yot6OcpPRfj+UBjF+OmcX0rAo-`mk zB+{2ooEk_|vK}zx0QNB$pgO0EA8H|AOB|Jim2$SGA~UJZ#fPUiRtMS^nL>+xckA8| zbccpCuXoT521N}>6T;cC^$qpzFbj&cnX-z)+ZGcrt$9_^dvlp5H3vpx zIo_Bf)GZC-TL8qWxw#{<{G6+#TVM9udS?-s0nPBJJgK%g;oYyQ7Z#I^$N7Z1nm+`z z3`w1h$5C>&L^c6^=ZBLsby!M6`8e}eD6D#o_`wk>*4T_;>G_dE3MPM3)L*tvOFfuDMr2>*;K@DeD|LNaBsR-Zj%0?v4)jo9RBKpX~ zrL3|H6z9o-M;F#x@_kJ_GA1Y^n3KAYO-lF%Dl?H}_I6DMN)u_uKpr{ES4yrCt*4*?Dj8P5fk_j)fmAp{9AYN**(Rr%Cwtb$5waSmPz5GrgGuQZunsNiyLY2j8C!&nCtb+^j&>{$_f0;m$P7)} zSkG9HLb$-X#dAoC3lv{g60X6&_mK~jZ+a~F-cKiw-??H!;yq-kE`cSbS@(J3hUe;u znbc3Yx|iPGF^)FB&F0~N_2*e_9TzvtKPtB(S21j{TjSqr?X&%_Q6qnq6C>yLDDO^P zjXP9w(lvCYJGU7RI#`Um%HQl&hY4%$2D6W{t%P=cUDgjD=>50(T-Pk0(E&1RFi10? z?|6_rJmqroky9(N@-wC^s$nQ<^z3d(JKl$UgV5{XXC0?UXp$lwKU5pjxi%I3!jJ1_ z7%ng|!m=fg%}Q=;;i*+X*-IOw)TJ z2Lg6j=hH6M3@g%>v=rHsEhxuY76fb2m5ib^D3NiBmiC*=eH?8tG9>B5q?*%stKk;u zmWOT5pBS5@!>-Y-NE~D-_0WR)#f`|n$p4BTOi*+o!HA*>9cPvJL)1)e^!p`btOev5+iU=M)h zz3tI0^6|72pBx)2LpUl*y>mz^x@z)gy)tN>_TS;XfuIWX4IewPVwa2#Aba|EiG47U z!iCv!#s@!$Ah^7TQ&yffol`unzO~ksTDr&!1>(+nc=+o1(s+S5lvz^6(PRIN&k9|< z+Tkt}7a?@>_Kmy(R|K7%$VLm}ZC46tG00&OtjyB(ZgN|>Rf;E6<F`0^O+=?3r||Dwc{0VGZd9?>-AL`9p=#*_YL%|~mpJp~`;2&2 z&ur$;g2@kwxWW87+@tAOw4zz=GnkNPQ@X)`5_YgxJc@BQ4f!O?U81gB?A$f;kvwnj z!K_N~c%x4ENuJTayR@)KR8XDfuneSXiqMh>>5QF28-#1@3q_TbIf}rW%pIukeo2Uv zru-~<+XS+;#2?cY7*Mu*9e7M!#D3%|mzI*>)~iu+zg0H~TYVh*V-Hg433CU|`;(*J zEs4$<%4kDry2~2kJ!LKsI)!TcJQXhHdpNl?g=H!=-~V#9uDWmtuxjJc+tImT!YgTW zq)kI#sqnQOf2UcOgnT|!bI3v?8gbEcS?uQt6aBFuh=dkQy0~Zu{PFGN0lHxKRBE`Y zUvy++Jg`;4t=auG*@2BV85;WFy4&*u@RA119s}*e&7H8@2IPD%Lr)%?&Z=e&oGf$3za7#W7J-gg0y`S!z#@@Vebf=e$c z%h=Bl`j<;`?Tn6LLxQvy=@06)dG5MWVFs$W^P(dzb zux6p?t2YRaqn=PmeCl;E&!1!-D-Q-$JueanA{C{mt&-Y=Ww?W`#7*&)huDO-5C~d3 zJ#JrI54C6#pH}L$r9zq*W8{ky9EF%9IgU|TKY*&TOAUkl5DlI&f{JHCe8ubCA$g+@ z?OZYuDD59A;;DGzEna0x4o*J)_!Q8jtPesC@CKZIzdEZf@Jdu`*Bk@4?h`o`vg}FnbRg|ioCPG7e-AqGoW=F5cZaG zui*6DP^V(jg#@)^hkQb5U_$Xh4EH1-<+uzRUQFtsEBf9(i!Ccn1oC0ocDvebHcVGi z<$NaU;HX?btoaA_k)NBIba1?Q;U+)DO-q#;X$k6((h8QHk-@iKpuR$Rb{_J%GQfje zlwv}GBFcuF#uTpe$W1)mb+^RLvr*9_?d;fC_D7a8Z?tJW)AFA2mu|v}HERy!7{4aJ z2|Ng#Xw^7o5FcAZ);!P=!7QdKyc-0d^rJW1_JJ8Rikk@lNZpAG$sZ6jzO@_R%Si0s6u%yh+B@2B<8PP6 z_vaJyaeydLGDj|&|o1kWvt+cOhadQ%&UP3WwfTtoJSuc&`7S*)Bo*l%SYmFa< zu1#VfKi5#`Wpze?aF6#6gtjuc!QcjHA?~72@-&n+@n1PDYmv;=v^>-x%uDrT)&7`| zZYwhYns@)1bB)>z*b2^Q_(MlFyX1S~zA)g&P{7mo3K)rc^5iizVRcVa+VTfM_DB71 z{-2GF=hVd~RP@|!Dj};z?v``v%7-+Oyux$&4B1CqZ*gT2&n0R{RSBLJ z*t7o*WvChe9(orO_5fI6?`nEzHPZ=>klqTbC9Ce;*zuFU+QA$>G=|52C>v-*`tcS1 zJ&2p0OF;kiSfDEMGkRF+iJM+-)Ipk)bz1B&?`0V>S=7wy^+MA?x-M{io%KrOz3q-E z{;%?zV~uV!bGs&m8Q;oUxl?s>Ed<+4MsGC8Q*NJpfj(uLWz$9o5MqeTNHcQYMsS10 z9E@&>=OYzb2i{AX^h=uDRW07?)`Hqr#@r@BDK?#X!LWrV#?-HT*2ECXV zdXD-ibl@NDH-5ZxvmBMwSeLUWP6&C=m*_t(w|^$YFw4Q(1}|$wNKa?|79qFT|S&v56Bp%$$3CozU8QdY8X zE?t-u&+o6KPO84JME>(|_1G@&(r%4xvHTrfdI@VG0|~M-cOblK2QnmR@eo{+M@ZFJ zSm{iCwrReu-C62>&kVZ#WBMIfWMNlq`K&MDE~$0cnmry%bFQm)`a=1-@3SM)>MOS;N))#K*CNf9ixTXH)YrQ6xKjA z94Adg?9l#`US7gfNCTK$%rq}74xI`l2$W@)OH;BYIm_6N#9A);R*)a>>!xNI!8@1w~#<47+hNeBXJPEc93B` z812%GsA6+!FDgWCWjjn9?F};%Y$c7U0kabfeGI4DbezUd8!VV{!aS)lM3J7k(?)%Jy#6NXJEB9y%yaOL*}iZ17epH$)V(`G$Uu-KND4tf0>_Y}V2B>)(nanjKoTUu z>vzGAU!LF}iQe5%+60A#E!0N`D{L$!jTnO{g3VcV$~6G}}W&*({P|q=Z9-#Q|oL64DO~dJyI#M-V3v9+7=< zSy9QdFC+RtaPnMyzMgn0fdA1=DV10aLy-I*T@G>+a<_q~w3V-zH62g`#j~lh7k^_I ziYRi<_KM@t3S~iwN`=vpi(7ZU*6jmFSpnQI3D7w3~z2{a7Shqjlos((;pu>m}+6y|G|y_hy7N(`STxr{r^+^U;g~( z&ibFizy7Nf3_**6s0bol7Fq6+k;^!ES+rGmOAZ{$xM6VI)JH^KAzMLQC>RhK8XIsf z-q_0HctxOOI@K5`sNXtwHY5UvMcLvc5QqY7g zU0o|-wPDrw#JUlS;@PJO4gF8(m81TBEU7chvZ^G>kNBUmh#U6*a`;tWOg-(pnUUlX zAaEBLQ2sdwB5?;>utqre1cWI(>{P_XRGz>J$F0{(SB`VYY<0bRTf9uMZt-Cr;>NOzpZKcfj!;n>&4wcfZqUTwgA4WMTr> zeI$Vgr{#?t+y~I|iQdPi&iVpJz;@t57BIHGx8I|?@*w^*knxrQ{$I-UgzQGpWlWKO z;D5~HoNMM;F-7vF&qfv2*#A*Rt*kI`8ac4vLkj=<^kU~|Nr}Vqh(nRZ;lNvz)-DTzxK`EsTi zT(>4B0A^U=^zv5jXk#(Pj`0Xv6YcBmZlmmjGoPGzw%vaq(wB19X<~fDwH9iz+5B(I zW}E-t_y2$QK-5WkKunC-o8zpTRbaj{G+i4m&Me!NTcZF^A$}5SpVe&~!METkVTh5e z)9&QMRrzYJf!eSF|FG4_>$O1dVE;=w_bS2FwiE8Cn8L);;_?c6_U+M)pZ9Klx4f+- z!iDSLVYW^U$h`dVgz*Waso%E=K?Y#~`G?Y%i|(4J&MTSc6g55E%xX@fNecr7=sW#B z_81YC*K^D2~`$puw}8-`Rqw?V$Y7XHoM@K z18XgHM`J7VxC0gZt4^3a{twpipS+7`A?CjZ@6L$3R3a~D<_sF&T2_JZwE zN>I|UK((;I@f?hnVXB8?hM~v;{jYiaZLR3mKXMnZlcth}9+FjE<%>y=&0T5%Wi(&9 z2mSM-Q}vI#VB+%3?G#BcP+Jw2QA#R98`EOBFKr&NrL3TsPBa7v>a?;##9Ewj>`p7F zS|5aWvw`qKcSsmpR)#QF8D*+njZ*wL4e#nLk%xvFfRZBnmbPTG#JQAUyUMLh4uCq8 zTMLS+IEtzLg4+KRi1f&r9s&I0$Wzm~Y?BH9vx^jN6ixUU-Q!I1X?|1s--Q+IhFvDm zN%WF^`hMl5r6g6iIA5+vkLMisZT|G#T)yO*T=ej;P(VNd0ZCnm{$=qe!%jzXE)n7) zAbZyiMUb?P;;sGW?W{SGRWOap`UDn>>ZJfE^eHHzBfB(&hDe0$83cPgY ztPAq$&_^pO3kiG^CNq~g6)wle{|nJBY|U;sghf8A1@&wguc+i{L|- z2a!vnsP?yeUJ6QI0R@DAa=`of|941tefifbumPa{XAEn3e3rf1Hc4-TuR6qTQtwU$ z{=dD@)91FKJV)wLZwrBBGEQ>5`IZ|vl)VCF!Vws*1ax%pXNogLnjm^r!|>8nzQrx^zo61lV7Zw`^gni0E3H z#Wp@A0@|KBUC!bN=YPi3Hiz?5sG7(bp}GQg6`0gp@^A$S30K{9owADQVoumJ>Q*P* z5FO98q&#d0ATiE@zC8$HsBD2wuG9Dbr?0PoilbTDUL?3f2<}dBcMA|CcyM=jmjnqK zEV#P`mk`|DA-E^FyD#$ZlH|SbckelW?VO(8nc1G+n(nIVr+fFw>!DhbHPf~#6szXX z41Ty?Rw}@Lrh%!pt>-$3Oj|rcbjuiddVqX;ajQnYN7vu7`*j&Fo z=s->8htr3Z;U4>X>DJ$_OO<7(id)#b({NnLM1!V%>f1&5FoR#^cb{S1z$h;TGSOx- zmO-iDDI^xp!dQ4sUH^Y*2OUP={o}QwTp|9rt$gRdZmZxh1Eqy@ozBZ} zWLb@g^r;*2RbrO$Edo76@l+l``4LB!)KDuiE_2)$6hzpltQ~EPfjncW7sxtLw914U z2R2@Wi<4k2fx{1Os5)oDk6U5%WgZiE<}ZYi4yjnNsX9H4y^Xx)$V>?6qZcj1&HQsK z;T3oxi_@HQI1gpn&yvq2%MuqaXQGsNI0dZDntbE@WBvrC7BAk2- zk<~txlrCw5XGG5+4?|R=n9S`r?R)Ql%NppZTEOQut`lumSJ&g2{ZQj+r@2&@Of|{_ z@A%0`43FvuRzwKA_5}5A@#9Xo`H;h14N^cvmo)78Ram)Q+SuO8o`Jrxp{WC9Rh5%_ zf`bDP>%w&R{G*Lm`w{OXEdBN`bS##yF|r6n7FkC6rW}FoC;^3b2);Ot)*a^4&M@6a zSs!@K3h^g0{>#eKPW~rCkQ*oTggWg_PQh`ygt%rB@-bPREuowwn5nTnFTqfh9QM%I zoX6-kvJ=D9>Tp)@?+8Pr%h|F@-N!yoI7)|ou(^iOm>l-N9sk-od0OeJl*WsQWD6$^ zZ$tQppFt{BScBgSaR={~*QOJ#8Lqsc!%r>SgTwJkrDi-`ByoVMWqRdunGf|YUssP< z>zy)3m>%M)WJe+AP~A@yKi6L^kZ8~%Xwa}3QG#XuhKtnPTB1ki7DtRYiA+2vFY0If z2+qio)VM~dGR<@^R}>^Wsdrk%7zqyv2wT4v3)WTPA&>Mvl#Vt}VT0&z0ItIrKYy|? z4=$>=enVh2DYuvL!Jl_hbrOQHzMVgJxUoBSV1!2LKtuXJn`kB39I&1m)B#;#g>gix zM6J+&)Ser2d;Nl~C;EK8ue76RA_gGAO#Wl0d}Wgj;tF9&z-Fs|!?>cpUb?7{dAh3J z+#AwiHgC|lT6Dha-6I7ZB)e}n0~_FWm>xYeGLx$)y)qi$tEN60VSh@*u}unJ*a=4dxtzcdc!e!tdWefZWzEMRY}a zYZX0#N-qfw3L~gNVFWfD-_n$F{|vBed&O0llIF4b%%?&pEFqEPnZE_p4|L>KI*i2? zyjx)$pPYvmDE{G&1b~S)tWV@jtl|Ht=0CfXyT0v(vl&z&AJnd%k##S}DK%eswm^-& z`S!?IJfK5+*-*QdU{e+TpUvm$&2@l58wtOw5tpk`-CWthmEe(7#?Z}jK@${7c$Rv0 zF5JLVn=75?i3U^*nN$q4lu};NS&mRJA)#qC}jvjNJ9c1RY>7nv`^II@j|_A)wF z%bsoy*DFv$d!qL;xylEO{8}xdxj9&kVs^|a!`N2b7Km{@%GVl13GyfRrsfgi008I* zVadQMWpHx>2rFdzJ!jA^v`nABoA9{I^8rB0SlA6CgpfPodn>iNZ$7pIp3}}PwBBL? z9ly9H+VxM`w>Ky0$h987@)&vaFq+s?IN6AoMxIR>&rsTi@Y}3JM~E|Zep@xPfq+Qo@v0M^${alS$Vj@RtMgBh*vCm8S>96L7=<2l z(uedDCP9OPhU{<~-=F@*wO8g=^H13MxSm6H!U;{nP6aJ(pmBc$$B4dNe);K0LWK6+ z7!X-1#`#y`;#`dBY@gNUP;j_XQ{%5j=?KUcf2KY>^7SqUFu%D5%y+mHm0&+3usFXIJ<#LkioFZ+E-2T!rSUbBcyUQ(^ENpKFvm(id6KYW8tgc~9+jbOqWOz|m`q473`2 zF5oH2E@MMXR-xMK%r7}l)%24;uf9JI@>g&-kjSLR>Kmu>m+SLa%Y!c&ay2-aWnPRC@V;8ubF8pqH(c&Y zpppCWniv)GyNcaVPJFMss@dvsaA?M9NIujhS|54RaC~j12i9Prb{oCIbwt6n)>t_1 z*z(<|YV!G|m3&C^O>r;Rd$zW%JE!;2lhim&baWTZd*+weysO$OR%VMy*G@%CQEa;W zuMk2FaDohyrb~kiRJ%qj>kaBb?*rpopnxd}ciWB`Y;t~lr zUj7a-dh#!_?~BYg_N9TWu;}VCuO#6}gG54pv38ksn|wmU&@ed`?#M3wE7IBV$%1)H zS4cv*dP*S|3RcNe7C;ObBop^aM-_Ky@z=W)Jd#7DxCngr#gXxgc6BtoWWpAlMWP~B zORO#}qSamoydqaYq+qjnTb^)enG7EJR_~qz-AzHVb;@Lybc?pBLe(=dFyAw6!+F}< z`jq^cc&_)8{c~rmcl*o{xz7EZ|C3Y#^WKA*1*6;D*^}LdjF!{Z%Z=Qofnqo}VzFlO zLPwAopHn1N$5woZb}KboZKx2;O5bSkyO1ZR9<+?41Nmjo?<*RKyJ9^+CT&E}b!xIU zT>)YO<(Eha5ZcTQCr*w1?9w`pYVit=adE=>)tDHo1+`9WQ})GM;=Xj%52M_*B5LEz zSOrpXJz`oE=LE~Tnia7=Y*8i}3E<5mWBY%$x|M+iQND>#Ic)5^OzM3j?s;reVl~QeAwp;p7oYg3EP`^$fw$t{aGtpL=&Xxq{{G2M+Kf6W>sKUm zb~trGvotzZ5GeOqUX%0>WU+uh;?Uy$;91}rt0e@hje`TG92Js3m*2m%-da&LtDA4a zKeZ>%1px~cTV10|#~KNBMN8y6-Jy49(fq#z=vKBi3?hv}J7hnV9aOi~Q4Wi!My zgxc0MIAsPK99CUkRN-%K*#be_z(;5YVkZSJkI6YI)|D4MUm%N}7GRg9t(74Jie@X& zU_I}eGmixYD+b0^c$`}vM;_UTudZsI+JjqKe4-Z%(P5AxqWV6ylh2{GuDwM3)Y5T7 ziE?P`-RK{JrbGdw)|IZ1p3-&hf#~%tK>6E+XZ9%n&bUB6a4X6{2>s3Kh|L7~jKbCE zES*=m?=}ry)>^(9HwN*dT8?~^=hwO8C?%U^z;1vCkU*+$P=T8f4E?gPMQW)5W~mP5 zbF-RSNw9Fc>HwO-rvmIRhaS7Dw$xH?0|l9;kZ7u^uon2zXBkQ>C) z)&2xh|DtXMh`qS^3kCd|E%pfkT!+{RH!dES$SSG7J1KASbT5i!ndf>N%it zHra1lG~7P<)KKq(rrY{tqj{iQtUTGSY8*ASlXSg0Dv;$^(~z_UE9~9E2>;V# zz~&f-Dc%;PMkE2ABo~6r7&NF{3Sad@8=4BeKV~Lh*g!*ZJ$b1h9EV>jV4i2aF^uxaXcQqOXO6$9d}g1Fxd2{Gb6HI7rT1{e?#>MoQUZ6JTv zKcEsxgR>A#l#pjm~l<8&$va`pLCRPHIeL3;+oYjnD zdL#Zqwci7Anl~HabYl^U)HuRr`?cL^R*-E6^Bj0&1gUuZL;c8(>0Za@7`FJs!PRx| z`uqd+;^d*mNZkq-9kA!7P_0kzi4|q#=+Sar3^k14!+LHcA5S&5%e&L-q%&DbXI>(y zcWqQON1wo^Z_Fw6gRCBR)0pH|XWHk64M)SSu?6;73@&Pb@3aNvZv}u;{giQZ6MBk!Sn!HjZF~IaVQtoPnMpgFJq~~K_6W0Rd*HK@PZG?K+V_@&`<=&b zHIG(Wo%O=P+Vumkuz$`swRs z{~^)4$mJZMYkHyI=t5?rkR02}DCg*PK;AKjtZIpBnPx3{b@XHk;{i@0gWux0yo={h8U9a_u{A`So`pWi+z*mACT z?LD#`W`ST&uv*D9I7(n+W2*sESAskZ9SDF*28q_x@bGY{SLVmq>&F5;aOe#o%)^Wb z)<)bo#&3R0P?~=RTK{WX$&HyQY)$>1O3dM_#V&4gO-fKd5lRa6y}sWYY$XcU@-dBA zDefus-!$m*(ZqV*AF`jLStR-7@|WasdCvf=+_>CD5H3rZhwTuT{bM=!ULa|2AOD>J z0C13*|7P3&W`0)w4iG^X`hctXEW>{@fA=J-^Gsf_Pv^&s0YY@5aHRQhO0@YZM{Qb< zjM&0$(Z7RD!0^8w|DR6^`)JVoE#%+(c3{&;9m{X@x6H`7cg=ncGh|05-h^iSc@SSl z^>+1pRKV3Id3(?(Ug3*+3$$4AsB#V~pKuRxLEh0L4iASn`F&x-$xKff1;ru>UP z^Mdi`vQUog3FVu`_+vp(h1%0zL$niA@d#|$|JB2vY7TnC(E%$<-%2Z z87gle@joyooV{!#j*!Y@(dqp9*l1wi73nTq8#d7<-qoR?eTEgv>yd z237r_X{3`8%ih73a+ChOt4djrqqB=%KuYaJ%VZX5uS^v;Qy(?jVY*5(QqQ4EO6TsN zJ?CV9jcuygYa8Vhj?9ruh_NPPa_(FkIh3o-cuu|xRRS9y z%#s`JtxfCFR4h8y-rqgE))*9# z-sQycs&4G{>agIGr3KSybKyfR_seHJ4}2(7W<<`{%PbWULt^J-6!e2jp@?OH95W0# zI7wr1s#5giofbd686)D4zj@q?T|>Y_07VeCUPU11onzT(pgBLk*m2nlp@0hq4w}sP z^bRZC5>3{4D;zrD8qMiAoZRq}hKR)4ZRE_U+}(w?=H zHB73EcVzSZZHw8x4eu;wyj3UB=Czm!>L>{;$~7Ybq0TO7`T^xkx_=426!KIbq!=Xrz z5A_kY+eor{Z+(9v74>gzpZA55|MWQU`}Y59VGKt7iDKikjOj1@R3TNq zO7UnfTM5S|_z1D;(~uK_{^?$1qJbht@f7DJ7^K6L#P(X$?Ym+R#62&~LYre4ZD-Kw z%WPPaw$lA3s0o@QYzhrWm#+TStCEbL&G7$6=oXC1fZ~l>>X2aIxSkq<6^;b-47S$$ z^L;@kdh;GV8M=p zBV51ip24|Kq3q)1hA+x@VpZNheKS+hik!?e!LcS##W=r`){u|BKQBBg)5XC_PSVhp zvOXZlFaZJoFIqJ2?n@4Dx5oJv^}>yQD=hDbOQlVqk zS%MPGLMP7ov>HZJ)bAiS2{3(09AEUKM7|9ofQ$GW%kwh=wQk>{rfs7Vu2LKDf#?Db z2ba&a1dn9gJj{bC`6}1YK4fHDW5ch{#NVj26&KHQu{k{*9ia=6k^QRkWBLS#jR{?x zI=|s@6#okS%~G=b_%d1?#su&4l11 zC@MxMSr{t4JiJHG1DndbC!yEK1>VaFU{m^0jX&q933;UYr7Y|$wEoXVecd`0_rWDgJ!?Jd**HHZWWQ&qz~`RC%4=0`k9Fut2$yoq=npu>sna<8JL9ow@a@v| zH@!_TEjh@3%b(9%f=8JkAr_jv+WWY-4J6)oRm>;*?L$W_hLD_*vN-fF`qn}~uGBMu zZpb*Wy6>G({&2z!iaKr21D)f6l8P?m|H5RfLgzDzsGy%aVaVjMHb8<|gsWy?j?e7e z_rqpgHrxMTc$-7-fR-nhERW84xogzUtLWxV-8#ZmhuPQq#;dRKkCC4zD?#&>&-Dcv zm6&@kvbt^hcE!XW)jc89lb|{tt%}UhFRpLb5c911v z=M9mmZTza;_fU*SsdtVq)JMoheg1Uz699l&$Fmx&-?sn&H|VJE@9>i1&jsLSRrDWa zBY+OK8^K`PK+}F$v?L<}e7k>W%0ORev378_iC0ISR;CuK3bzVKDqz4^a6YgsQTrMN zwZPacQ$J`IpG!9m71JfELh?-_nAUYuciyjA{Mn)ztJ#XY%OxbQq2=o{Osu8G{%0l? z{ZNu7+Om-~z=W^7?(twG@?X^g08sM4)dK(?JX*R&$~XI_ss5gpyIalIZu@LIB|?mG z7xLqEiQd!b_V=Wo2me-LtP6(aF(3pWt5hzox&rl4)_CiCa~7wT4PT<#!WrX9tw!b) zRz@8A=K=x^l5DhTkjd)Av`fAf&N3=AQmtuRjJ37vfJH!`;!lK_dQF4r^ zJHLpSIjiy+T7I3V{epu*+$DS%Lm+ehX7WLf>U>%X?RR}KgdMvOz+VG=vu8mteZ}g2 zKuf^a?BTX6PVPR6p@Rzm$s=ay@x3)%yPYTOwI;{zan30x=eP{u=EwK7 zLPX(kpwRJn`D*Wwe&(Bn2_X=hT4{?AX^edl5ho7czJquZJT0qIn}j9B76gNfwfv{6$%-(y zyz}EVdksrB+>nksHIgwf4BmXKv@tbjHM3)Gjh#6JQdGX_h)lQH4*&AKJ&ZG`Hfx*- z`Za~a`Or0H-7t*uRo&Ay?o4_rm0rUUecGJ%HxY#S6b5N30dKOelZy$*vmXVT+C`e| z>1Dk9TA15@nW$7}z6sl}i_H)yb6)Y5cgjC5=t|YAe&I((9oAFLo@lQ;12Gx{P__{531QmLl(@ZB5jQJX+^xuCcKE39MhvS z$*|2>3)c(!_1u%>WEs=0uVLj2wkn4MnGQNy>DGFJT?-(LI#F`RjrpU(4)$n^r!WGX zjl9|G1IK$l)P8S}MzBB>q|!u^=zat!`i4cOhz#SmiuoJhI)(D-cVX)RUP+i7Jh0G= zP|Kp`W=Z%);5Dta{Fo<$F4M_VBy4&Awm$m-cD@rPh7SlG+&Lbc{sfXWO#F9_!GG*p#~zx5xmV+z&J9SlpM;k;^^!lX`+cKD5J0d ztrZG%r5#Ef$3q=+Gxsl_)$#X6gZQgm@cccm)@a_e=CLFtLQI%hzrj+W%w~nr_4Mu4 zRV_i!!>zNn_cOlD`i`VISE0~cdMzL8sk4-bZBH;T^jiD2^{FV@{#| zVV-9C6@d{qpA;p$;VNL_VwW|3v6i{3K<4Y>TH=&i&<5U9>C;`TaSu2Dtq#zlI?fl` z2f{JhJHDE)ge*&rdA?uQB0s@^MfHpiNiZJrS=)^Cdm4GCIy`j{tc=#H{4hpkz}%1sWeTaw1juZE|{3RWH|on`c= zHTM{4BXzWI$GJ^HHYxQ@uQFeC`;kEhhvyh+tNW6h07qs<<;XUt&{;7wxReGiqQZk3 z1y?i6n6!D^gE6$KsYyzp17oy6GbCvhzoUxdyBZ?yxn%@g--?%m`bazbHaWc6!sqlz z{7t_FzuiM@Bh6o^^=wpv{(+<){l@vdNOVOV%=T20GOPp}oHR z$_=N8mjaf~a5gDpuzhCS+Ki)3JM)e94bNd5psuV$^p-5^5yX5DIge!19RKvdY>U$v z9QNa$X1zznc!Q&mhj$T*{rUI?%05XRR;rIzH7S!Z~qj zcd;o%(lI$2gh9Eqo84JrOSm>l1N5F+SSt*4iHbLK3sra5(A5$|yF$gGm_k}tJox=I zl2|$2*_p3S=gu0((ergW`O4Qq6R{-(U)G=my%xR-3Z3naBMR2QRseij%2SBii9Lfm zlQ^tu>kAOOb_n1`VVqMV%77ucveewAvkA?@dn7+e3QcErH;uNxZIs`@Z0;kVXOMpO^I&ZKm52$+>n$+}Z`qc;V8v7X% z)QY@osh7<0vUg6p9)9G5rEo!v)fVUuHirl0ua@5g+R0*|kS)?{h2>Jp^j@1KipWr* zlnYOY`3R&Rq@)#|VOB#Fa?En9AE+mL;SF6(S4`o?r}xUvb|fJqa9zZaz*!4U7T6o%GRaaJ5(eH)8lc{()(v0r45$rw<#<@+3%swtg zIILTqU(Y%(@7=`>k8cPj=P1USGC)~3c@px{7njg)V40KcySluM3Or$`xcw-<)xfg# z-D-|Q+FVhwOf4vjK(WUiqNYuE?sV$H+JS{kh9}d;Mo};AX%ko=!er4#$8ktgm7cOR zeUJepx**>cIIA~y!Q)lf>nHwoi#uNq^>QTkhbE0!jq2e7?=M=I0>)*}i>r(AqGk5- z533&;uLSDxB6zl&pOVX5I>d#>VEI(*Lvx0rWf&)ua$I_*o1dayzK@(;E|OAwe8RPh z{Ccn@#!1sagX)JC&jkBOg1A=XtGT3d5nv8e@p{)SxRG(>cO>;Fh>R z_hO{C9ntbqqlRVw7(Nyu+VeTY!74Uz}L) zE=df>-i4p64a)VTfGCC5BxN$|o@_AjZBYog?SHoWiE&dL?vgw*8X2lOMtr}V_r|g~#Iw6aR-4re-uWqDx*Yy&L#^{9`&s9LI8bkID0wNCSV0Vq% zX;X|_@4d@VzM}PLUl(#FCKsm?N=(}kXy#|O)mSO40+dB8riC);gsz!sprFB{PQFfsgO>frXnVE}2?w{tP zK4F;hr|pv}`-iui1IZI}b!HwZYV~Q{yT*2)6`;^k69p0LaFXRfN$DrUI$zX^HMC&v zi`;{8c?YJZk9)B~cX$^TB~6dF2V*TKnD1K~xlM{jtQF6X5Q%#^> zXq-8m+3N6rYWvFCBiO66Ck6+|u3_v@3-HL4>|~sj3b>h_av26qri^my;&c*2L_B@= kdcXg*EVXT)YK>GN5mlJ&#w{zytyBLOAB#aEco_8m0Fc7L@&Et; literal 0 HcmV?d00001 diff --git a/docs/static/fonts/LatoLatin-Regular.woff b/docs/static/fonts/LatoLatin-Regular.woff new file mode 100644 index 0000000000000000000000000000000000000000..bf73a6d9f97ef166c3f3e8ddbb900fc8e42d9b02 GIT binary patch literal 72456 zcmb?@1yo$kx@F_;F2UVBxLa^*++BmayF+mI5ZoOCH0~Ch1Pku&Hst^Bym#-DSu?X< zuU)%#?NeW!KC8P=*#UP&Nl5@00000ZfCZqv=iuV}f0f_me|$+PDXP9#3;+Ol2LS+^ zFCqMiFH)*NQ2>Bn`rV&=5B^nZnv|@l%FG4;2;#oS>%J$$jEy306FVb^ch3a?fcyXe zKnLu;MI)NHxsU(=qRH=dSl<&b1U9w0xr2q>yEhB~K$QRh2x&{0KhG_UoZn-m-xC1# z9|_gM*3&k_K@@gy7{?{~0wb^(B42E1P@ zA^-q4C?S}F=-_1b9*ZCI9_Rg@qHbeem^`viay)(12Aa=oEOLh-LtI2fNsLfYMZmzY zB$Sm=QBl=;`KOA8<%ET&c!&nT#36!)DUu>cqyY&WV7%x*za}XJk*s$Um6j( z>FTJD1iKmby<59`Z9AS~?r7+6n=jWsy3?uP?F3qiMv%-XGT;d65g*Mc_Q#~N`xaWV znLn1o{XUKCGgrXQ2}Q9BNfxz#`e{=&Xj6k<(-}GyqC86-jeEl`@?u-)*ZehdZRpJZ z(AU!u8T%#+>2ua%@w!R;P2}k;dE4YLRuO*e5nZFX>l|#^irC7OK4a%M@2U&lVdYWQ zRld=ViaVRNqfU3vnmffy`?K{af2JYHY6r-E$bJublznWLkCwL6Pa;x0^}&nboSXY- zwK}DZS+OK%lYC7spb4&M2fyxic zZ@)jrc`+WjmbXyxQt|S5eVR{cKa)D!t##OT=AO&BDq{95J^FKcb%?M9K^mu6O=e*ci8vC89MOZ9 zUoELnBKWpZ3PXR9OO7fIAY4YJQ!+wTwa=!bqNP9;IHK@uVV0-GpoG7ZTA zrDqn#rIguZ$^sjcAY^IM!_7h@_r<$0(hz-#sm5}sSZ@qK4f^CDV{V201+IQaQd!o+ z$T9ia&@9T5S$2T!p-Yo#9TR5fsF$G|3$I=-d&8A$kI*oQIWl5>l{5Li%W;poFm4Ir znkWwK!5oc@jeC7X7S-L4JZu=9RqwU*MRjgoKS{`{CR)=j6n1tEU=JsBVmBjvPhgVb zG=x>@WIPY0ZTzb@jWQ{Q^@_*%Checf@0 zH;?QLyMKwJI9@x15uqOx5DConi5hsp6{me$p)&3d*RM7ip|lH4ZafdYI@!@Wan-8s zWTcl!I3H>Upu!uA_@cs~gFaklwkBmXfx8DTj911opt=WpZ{y8iH$q&~obG7DOe^c7 z=74-R{LTUUh(wSmflM1l=dk?3lJGz)wTvc zQV%@~a?zhMtjzfC2R07Q9JFkRoQ|*?_+SX8N#lMoH~j=A;sGy ztrRX@iBK2_*B_v%MBIdTLfkr?HZsMn46EYkCY4TD{R0bsUN(# zXn4nCqQwBo`7!GaR&X*n)tLWF-cy*mtU$CM7U8w_PgstKtAGOmUE4w)Kcw$w)W z9Z_F|^cDIQ`4vn*h3O=3gEL=h4Sv!-;A4+Yfb^VtJ#+<#&ydpt=R1r_5B{S8!WzsG z$m<+z2_OhbVATL;3$0V@OMee*3-PlDxfbIXoH%%O1EUtk2mUt5cZ0o_m=KT`987{l z2p)dXgLV$Z4c`ShB+Sbaymd|wkqw~kp-s00%h;fQ_Q5v|v^5@*Vl}6!`g}f}NB_`o zNz35gdn#KAq&<6uk-J3?{N9mQ+L-U{xvx(&s)q3D9yCMhAc#BJA`a6zjDE61irL5r z?}akTBS!eLct>MSaQd9e*Db;NGlF}b?NgjY%V46aI_SD-I~nWaY9~VHG7|DAk0kS| zMCvhl7j*XxS1vzW>ffNzkxj0m99=*+vl$_$AN~uURmNpwbo(vS5C>ce(xLX%Ql}eY zs?$lAJY)`Bz1y&Wx8|V zmkXRg%1ez_ecI;Ihi}m*m(Fz%K4kIH0s-kJuT17<7O!cV^Yr^cFnZ^p7x?2W(4}n-{e!5S1?(yoBoBpZ^xjW6|%r3VUv|#fuQvkSLrX7z8=N7u6XTE@|b0822O_>3f~w2 zcUP3=Iz7tm!j^mEiy}Z6N5o|MA{lj-G4Uxq5nNSe_*KI6jjY}uZMi$**g*W2nd}Lt zCvM1SXsdS1vEC2mpV35i=X46-9lbGq`4TR%l#87)h`?;S1s@UuVXjvSaGHoPxz?tsGPg@Vf)Vk?LhhD@4;LXm}{ zmYXF)%x*@v%%5VLE5oHonN^ZFp*pf#W_qPo@A+w+`uVhZ`KF=scPF#z7!$k8lJb8;=Axzu4 z>@sQRPqK!znuGkQCQoDITaw$adbc7xkI!b1HK}4sQKU2jsL93=QTt8TWl}f1lU}C& zAWw*URNJqyLls1R|4BXTj2%0SP77 z7R51Iy+Ff(2oYyW+T9XI5RF7q*(syHUbDj1u{Y9|?~7o__cr1m@5zz<8~cgff5$)7eI>QAv5?1U_!gcn z#rs6e>{cLhjM6KDY03N+7%HOKihvMryqTrx0f#VF>i~hB(?_w! zYWx)qj5WrCntYJe7_8_+;OxncpqFdkCi6jC3{Q@kg2e%CwmwmQTJ*0EBdch?R%IG{ zniXryviYR)F)LcV0S}M7?lpbYJxfi*9k)h{6zWXpsVP5WbBNhwc#G2KM>Dpqms&Hi zvSHa*YNK8Z2m30wNz|Qa=$E5Y2w@>kDIYsfFDkU=3kJ)HO#RR_(<4HW>duwPywTBC zV`)CiL%JJRA5@--jo*DJwB?|GPJpe5`(o;(G=JKBJ=m3&OE7cAsm?sTFOa$xyx)mj zfis$&I#QwLh*Qc_U$)_v2p2tUIW!Vtpgf{1QcsLz{L$+zukkrIHSRDKtnoe$TdVOt z9ecU)J`tPR(x_Mh+n14?EVV&(rFZmXXfjw$YbZjDozo1OJTE`DA;B9wJtavU$I9s2 zfR?uD^luKv>Y3k_=f?U?8BzkQQHL$LmW57N>VC-zuBWL-89*X$NHtNihHwVrYHefrhsNtk}; z{%tQ~CrZ`_Y}Pi>Y*coMR85jDXs$YgD>T(cX(gC}$}t6!%!Wv3#0aRH9=2-6y=OKL zU<|eK`@{jv*G$8~58^xA8hB-6bKL;(^E~5Gxm@e;)j_~6n()Y3{5nIN4AE$Ja zr~D5bIo$A;`{{)g@jg5A%cQ?Je1xjMu<^)xK;X@sCRQ7 z0X4?3Z|pn51BCWT#c%Q(8jn$M{o6B#@b9d8bq)pz4xMYXdm+1jM+|cR=8L<}m0=h{ zi^Z=alzPv{0{e9dS#e4xgu4xTQWw)lj((HdYDp_jnT)a6wgnTpqsuz>s}R;n25+Op4IJlv*6Cj4M{7{U@M5C%9&;raC#c&|L2> zi`P;wH^Tl7&mi6?_r$&(3_aSIOn*Ik-F-gomaCE=tYHX15|+gWwKpmRmmb~to&%PNH)e`d>@;RdS0prA7yC>Hss~%nC#x&Ymg7`W{jdhF(l!M0 zt3&;0XJ=g?`u0IBZ*jL53x}U)iqEK5hcLxJjXo3S#IHav!)xdFyyw6-R6k;w&x-~R zy&GbONr9Q}Ohpo83sb>RR8HxUgyy|Z?>QWT+%^kF7fV7P`iN2FcCEORCK(Sy!V+hl zTjcuzo^f$x_JqDab$G`6qcVq${*EOJ&-jlLZfdIU>Pj=u^WF&CFFlml+~|$rnFZl5 z_xzckUkGJtiu^rH5bZY<3&M9;^I#sszo!Hwp;}DX6C)Ge57DMkj@#`srqK_k z(Pi81JEk#zc)LAeyM5x(y1=!}a+SC7Gt#mt+QX^+c1L%Cw^T%zr}#Smr5@SCqsgq-OVjCU4zM^w4MrMY*v=&^h~qu?^^ek*p4jG-0APyQ~Z#h%r*D6 zdS6D4VYaylmk*>+i{OLk7OV&&I;UMixDA{F`8fc5kdcEp9AF8A-UOu{u2;!5j=4D!P|qRdQ3JPr;+%(X3jWdxOpY391M_Q=ebJZ$M^;feqv5;0D{)|+j)2Cy=C^3-_#cpXdo*$~bweRn2W8_0O4c^rpR#UZUW7E0< zrvL>i18`#r|3bz(v5$OW_h^wG=oE6aH=E@8{R(jGUR>%^JK?)_Kl$#R>)ySoCp)27 zG+%OKFmoh9F4hWss)=2@*4q|b*}FWYU27`8rEGjlpy$yYv9NDc_I>o0I5z&+PC#f% zZ2xTt(LuC2q?6w0Oy+bIRBc~0%7Ft!FgR_v_{j$y&)E2ek05B~_5>1f3$>cd3!h(q z=vsLDElqS`p$wF zqu^}&IWX3)uC8i9E_dM+Fd>34P4o+aHy|z_#smHuvQyapoMRv2F(}X}EEN$tu%~nq zl?4+KqOS**1w975s0a6@@fsjPidxW1f4HRjjz*SeBZW`EN&o`ApPR9;z!hU48UzWdqMDezZkRENe( z&+1dFH?jMfD$(#D#Igqo0g6&iF!vXL-<7Z+(Xd6xr&39E&spo7b72xL-=VWgINt3aKrW9KS!2?b!w9<%t_UY5e}B%Cv%m) zE`WI;PLC}M%sA8)$fp*-AkK;%NcpP)lgjpseH2O=Dp#R+mTBG6TMg6pRP=v_Hh+xb zmzsZBcbqUpK_t%|i`ZlP-4^4)^tY_4@QF=lBL6lPalmHW9`kP!|5kZQ5i$wH(WaAy zY#xxY@xS{Nrz9?%afquP@LE&`a9TrFO>lj1*&eptx<3m|34EnsZb}fx`Or%RZXx!y z?7m4KN{A*Zc0QfxofAnz*|hpIdJtDa9QD8${xu_;pi=wHzQ=eT79%Q97w=_Ed=nDw z)hi*eS(xjftk_ygtviRobO?v@f3k}|48ll(a5iY}(Fq|AK!{z5$R`}b;ObiCW~v09 zef{l+vtcdE%M9Us#SSHyr^&rpDjurv_A;CDv|g#;Tlx}H+4~t70u^{LI1eFP~GUpDB~diwjJjC zi^i=>kOaLF{LEoOwkEw_g|r{KE)w3UVu?xrn>9blt4K~d)f!qj&l;8#&l-mIu=EX5 zF92<8b==-rX>4PUM<3~c;zwjJe<4de%{V#yRm4A}Sf@?FT9eY5=v)Wt4U>51L}?4& zjbNZxo^YpP{+iaG5*J3!fVc(gjFy}W_W!8ttAn1aiY;v>DEOVGd@)vfZ>ILlM)z5A zV4}1Y534&SDo=QTx#+ir(gnM?8*74|v?&hya;Uf)>3@`kzq!Z%NYu4r-_su`1b27>ZGNptp5Tgi+Gng%>GuA*`C<1GlFtku@t+~dL&?T z!6=v7GV~F=p&nQzeSj-e?J0O=2jGY)fBFr5V2%swFB}_19KP=c6rO)(*!!#JmDH7m zoDhd(hzy`K*E9}$r$lJkABH2Onoh8(gh7-CwG>FHUwhE$8R~lf2Psj%b{kpK35KB# zgpTkxeF~wYbL_UvKf7%Vi*ab%&4WnNHy(El>3JX%`g%`iC0Sz zohen*=UnRTRO)?M<_&KsPi5_nzpH5!hxIR{0L`fg_FrOrG)PN}g&x7&tf!J8Jq45QL(0i_CQyL;PW@)EO52RwUdZPUlM7dmDvz!8sy31ZL;)QX6k2DMdhsEZvl7Uamo!UQw8qeT7>fV%Sf?`o%zo_^I% z!9O_}8897>nOQhW_*(F#F5ZX`N&{kBQ(-eBm2h$4A=r%!sSw9Q0uNYNK0p(N2M!r+ zIO19c{#{-%%KrfUKS;L$f%1li7`W*W5YC2%$cW?rIM25}5FxGyA@nW?q{?!RiP+dD zOn}(f^Is8&0XH2ALb2kkf!WB{AjnnLvomm!IPs0^M~gn|VuBm4Wq9B}(|hGrmT9E_ zNX|xvOo-#5fo&`-IMBPjAYI&a1c9HUJMO%F8eu-;xJO(rH3s|*U%=BE_uuTaLh*+pWhzVVmWs;F;Zw%G(plVY4_z>0#n%#EUBFQ%TZso`N-8Vaft32Y4h2 z-21#oSamwZ6Uj6$<$dD9D!!!|&|W+A$s+!W%R9SedFy+Xd|0e1NKL7T4qZg^4r;+S z&=sflkVU{xB;H_p3+LJ)od|ih�iT<391F-_cRjG_ocX=+oC#)Ux6}hh;a=M>gQz zYCtQM;cq{~MZ%Wv2f9W@LwCM%99xIInL^xSf5ji1GIgCa=~K5&k4L(yGZJjzovNH< zuecgN9PTs&y zCggKzeM8=BCy81W6_}g>K^hA)+|9u_lH{%v1VHM4-2N5e>r=Qgc4r2Su{@S%%3LGl zc+JbcOp)2}(~`6#D)l(&Ogu*?utV6!d^rrOo8Z@YgV^~og2(N^+nt4I?$F?0Tg*aH zJ%+Hjqq(lQ2?FoC%=f+&d&-6!HA4aKTXwhcfA2H*KebEBZQiJTk13Bn(5>(yqR3-N zQ%oJB9@|OUxg2qNFm3ax*A>)adbbT;GPI+f!~Ptn2WD&0`5T`3q%zSIGoYiVPU|4a%w&RrzYxuIsV9JevH(A)U-NmQ8p`LvPUL``{@Ua!9 z5$Xhx0+TTd)t0f!72s_Q0P*dtiKrT9dTW_? z3Y;)R!0J~&M_Yn!PI7k($T7sZ$32G53lP{)t;IQp$P0Yg0NfK2LRxL0-d9W^$Invz zSpXNA&^y%Vzq0xQRlhmLvPs!3ibSUzEHjsn&RHt{KPf}XQdpY6zo~=f;M29iROt<^ zq2#Ip%CP>JYaUKpnVQvI0&a=l+mvn7vIdZARddP5c>0I3-JM^>yXbELFE!CX|6qQm zanoT$%-@g=Q;7;}1*%Ki?zJ80ZW_I(GD~`J9rQ--l*iQ+L3EM%GGwLb%<3Bh@ZU89 zap=#Dq4X<3(O4dJ;QyT|@p2Gp8_X zt_1rWEWt1gc0s=g{^8`O znO7ide0ec!^osFJnr3pW1dpZ*QZ!xde(Pg;!^*>=OXtqmsQX z`cYHg^&8X+aq$`RLDqP&R%r#B1hu(C>!7>eVYx>4ORO&Yt7P@T8C%-E&THID=7sw) z>ENA$KByVE)QtR7lEuybF^^%gC!Da}hexX`J4Rn2jEPz-EvaB^WPHbTYwa&om~OoT zWq43(xZimWL$>r|#-hY-r}W=ga7}R4J0Ay|BK#kuw_@tDL8CVh7O-C)<<@M@9T%qa zh>v-+vKtO?giL{bln$+s$Z4QWbZ%Yn@IU7Af<&`xf?a%)UG#9{fiWJQciXeRFM?+Y zKYTn&?^e|wgMF+HAHF>*w{$Gjw#5A&5#GLHXBh>0h)n&?1t(e(4!pt@9=?i?B6g! zI`$Ww+lFcjDfy0+mb(8p zQuLcsLYb~99Ztrm#6L?qGF7aEvZ!G-3Da^HHXv^Xot}X5?cRFkW4Cv^9)>F?F{3cT zELjt;wjQH4yA`W;rMf}%?C&PXs)7Y+*}25VL2V2-;Y`J{6<>bR1>Eq^qUYUUzVf){ zLl0aEo7wpzjmC*g`6~yt&cy}#gVdsUUUU2*rr{ZHXm-h3A>ZgcA&{qGoN-vl!-;psW5GPrEFAu+{BoQy*;N6wJ46uvO8T2GwMD_w&w#d zI%E}HFY!5%OBoUGv<%00Q&Oxsq?20{$^M=yE_~z()k7k8ek(JSF*XO}{3fl8?9U9Z zr9!tMiz(qhDle(37w4qILluJV(2{dhu7MSn%Bt9=Pnth*Z3s;4Y%1mZSHvsrqW^A1`A-wdl4clY zniYGQG%q?)f7*U;n5GwxGeUVQR@nr%3A<>J#hOvFYM>Memoq_`fYqA6D2&~jyNF9| zrtA(?0Y&6mDUYc4php~K^V+a5oWjfU6H3)V0j0oPQ8=zwgtFU_K>vk42R-&glthE^ zOr-)x1;)gocmwadY&5dKA)}8;=*2w{l!Q0?`h-iHJp98B$s$Br(y|ZM4wp14CDPoi zFt#+xe0OG?|8bq4^=fe{Z*@%-rKOqIw2{}!=dGrvScCZVxpF@| zX4SEoSh{{}U(BTOCPGwG%n=B(+g!r_{1z1Jf< z0anM2XOG!0)>#jI2bG(vpcX-_<(=-$vSG&`9Rhp`Qem%hMEX@06R`$kaN)Mc|sa&?>j~@43?j5sl@{3<* zdebw#qx#)_ljp+T^+iY;rUB=Yu)BlXVB^l_`Fr6?@@U>RmS@J@q{9Rg0*^uJfd5Jf zggnlr)QX3-Zs&M8tQJGgG351n&f#8@(7{&Fuf8dMt%5t!v6~qlgJh}N-(*j4o${2e zba!JInwlicBODJOirP)P=H8BQx^I9+*>?9zXY3kUm(lIF2;KfuJUZB$@-L%2tLit0 zUalx<yB#NJ?F>v-+^o|QJdZL z+jDe6`ah}`I`dn=x_|t*>FA1CyO=XGGl)FA8YGNHwD;O>8J%joCy!tJyzs)n^*YHl z{~Dk>F;x?JnyC?AHt(I*w5;QMqH)u}>$H@0AjCd5CAFzeV5)? z2BDW(nKM%|+@Jen9$#?W@&^$;rc`_dRH+&=-P*$PM%ssZf-l#snM^A@1VcG3G)Edn za*cxi=4W^qBMIhkBn-*HTPG&boJ znBCJ<;6zr$u*f;IKUQq+oSSfSQ=t-yz>71->!G=)`n{gbfF9B5@=8a4Z{0e}+FzwA zSE7qrV@~s_%**DqVf13O8t&4LM;5uL<&%SJb9R4|8pn>e=dt8AZ5VH|!g0z^p0~>C z7Y2=T*dFzVb{}9>4_BK^S42t;M)c7(UA6CIhtMh}f%i`MGshV7{EEiFZA~L}?v7-e z>}eNTiRAbTwgk}8zG`NX_W@dHJ#A_fjB|HMP2!SZsqn1@KJ<8arG=zgeMHhJim-~z zj)Q9jQ}*+I+Uq*a4-G0Et4*Yi%Vah}^2zeX+D6_kzpCcK!#f*OPW!_*55G@_cUVP& z4c4RcySg`&CJ&C5rc;0$9(&ZP;1MsuV=n6pKbmmjrx=tEgsC^o5wWSX86;}Fs4ZOvVWL7UqZ)sfnO;)zl+EyD+b~iWRwC*pl{~ z-jp)q%yOF&XJ2TkfOJ}HmXc6oG(VZmQbug&IW$@B!{{{b=YEwqk2<|$Lq9$qvSoqO zUJGJp@%rdr6Gw{uMAoByhWGjp})M7ux)5GSivxNtd5PE-gbAunZ(=rzB|dC~5QNRRpip1rdOPS1N)w(so=~ zFLQ);w955z-Wjbmv# zD14gwGV#3QY^Wo{RJEzfX9;QK{8ST(;zYN{n~Fo88pA z;%0fr(^wUP$cI(@E4uo9*Q2Ij;$gcnQRt+P(f{|}><0Bioj%5hKmM5yeb%C-M%bj@ zi+qk>N41}qTFOO3b;{vQjhA3mH97xBwY^ktE5$LgI1jV8bzvfz6VV;LCE}{fpPaVe zA8}+8Xs+p)RjiE3QT=AQop;RV$$>b{&kQ&1Z-d^zaz5Frek-lX2h~pIm7KH>6*``3 zlxlidbNf%mHx=~bJ0*wfE)@dv?r)pV^WJVZ&cN73saW^Pv0ToT9Fo4A*NM-rCv0x* z!_qjWHrfUhdAj{J2)GYyi<83v{@tH{c|+yx*6)5&%Ue&HA?%-mEVM+~1ILS4Wd4HG zU5O|3vHT#*OS^)szxcU$vHtAK1)VAJ*6Pfq{9*dp!AzW)LR+BM+2D$<8kV!gsUWq` zMsD8rdfg!(mi>if))A*W9OoV8{Gxhk9uRmMdDvlHy-uvD$JzDxP2&C03(?;+~s*o!8I(s^N@;4huHv<+kTpxDsK08O``gHz&7JJg59rIJV z-3yX0hxc8uWEVN@e~ljXvzGquUK@2cKM&G%eWT@kT_Z>xk+P|7h#IBu)*~&MC(CNu zn(r(m_H+U~l(A-5;d|OqX?<<@@u+X<%i;a1iP;^m^6W=*<8*DKKoY2tEQCzEY zPzNEf7CL%}r=p7YL)J{%K84hG_{)b8!<>HK@7_Y;H6Fq)R9ONvkA@-!$FLyemu~YC z3uU1c2?!we4Rv_jhMs^kbRy4C(LM&Z&v0(i``9wOfX*4ZVjQoR(0pbpny|~bIb96J zna4WP^IZ$J%^B?cP)d>QQ)5EXXyXvw`5~PmYf}L{*QnU2Q7=f<4pmY&62!9w(;5q} zY|3`OsoW{4PwgJn2BR~8D)+(qdI_u@)gaxj_;B5jRJl`AuU=$mRVC5 zN)BKbmgfiqPeRJAhZKQ)M&&D!K>f;+@0Kd>d`KwoS=v=@bXYe3vCP#z-x^DPo-V=K z%%&{mMulESs;bg|ezcs9QXfd$Q!aC%43w!YSMgCxK`$v!wgl#hSBz3f0;yQb*)D)x zGiscLR*u?)^wk>V%H{YM-xh3#usW$OzK!~XEP9!#q|r6-)!aPx>i|>1%4OhGfHLB- zqijk*TEF*_Dp01fT;;HgPD1syy5_r%;zLtlxfodrb7wO1ONGzK@z3(99Nx%qAm^); zOlrK}vu4wi%G>oD{ir0wW8H`D15UdDFD~Jl;~R_A7tGJkS6t_3Zx8!#;g}OeI@?$~ z^Lc*gZiG}ms{=iYeL4^(-=9?KcMH=yP)e{N_0 z{ND1IgiWqm#p0Kh??=?u`Lq*vxGtoj<=!~$eqqidKGk__&a6s{`RkcyOuo~n#IFnZ z(y>x?=q=WHJAJoj4dap1e%V{~5R~l^ne#R0;_;%rs;fq$sO5$Jr_J-M(9yzgP2QEVizjwsNhWNtzz!|DmQq+sK_R>We_~o4CMRVl^+GV`f#nxph@XGX9mFjfjRcf``Wh6~1BG$#_^)e8}x=3n*k*cFR zRmt>nR_f_06~e`;MlxB4baqkYb-__eE2b(d+ck|NQdX)E#qyb>LhonyQG5NQ#Jf}M zLZ@$*%U!IVAX9Zn+Y|-Npbm5`f2<)Kx46~6KM;C}hd%@dCUlKW3$b?=`Y&kQv{}Y# zBBD=4ygpW^>wSAY=}VheP+a7AY5#URmbnsW{rL3kqh(dICiCVmv0D6_QG7tBbu^)e z?obpMhp1kMQ|@G18wDasArA%;hLscIRTi?U?F<{HkVB}MLYEI;S6xH~bVtbvVP!PM z5S9%|hD61#&~beS+Io~NovYVv78@7EMa4>EVI|N(m$RwYUV2AddY6uB_KZFi(3wD0 z;KHil(Q8VG{;}T6rKPZH%9|C1y#^8zRpl`^{(z^q6fFS2Bik&I$5KrhS90njoJbHP z9GVFt85wMuN(4843+|Ss@fM@9xisis!s2KE* zI601#;zpf(O74dW>yuzaNgEE6@(u4di=$E}_wK`JN2{)7)-@y1HDTc&c|FGz+qB~1 z)A%T0o>{dE`(mCeU>+!Du54!>*|%G&o=MGPo;q)py9Y~~*WH&r?b^_*&>&NzWs=78 zDUWC1v$OWeZp`P*mU*MK8M-SpdY^dtHR>yaQ2KF)DNa9-$*%5Byw5IuT2B5kVofhk zb;y-263#Vm-Ptvj?u<`s`Q2vbS{U^W6Mr$vrWF%K({z%@%SGwHK*jbqk4K;~D`5OZOD9RKtK|~l_T)&1|&zB10Y*1IMV+uG#pXnFE zD~*K=VVS@{0)S%*inK@ed~&$68W4BDe~&UmA3_F+83mRSRg^j)C@Lx{2&9BGN<#&d~&U2{ow-% zqJaXVxFxVT>Df6%mEt;d*xxCsxcu#AJ8Y*mTg<2L6Bb2Mu&oK7_kBW&PY=G}CMH-t z+$H0B8DO10XtvTEtJ^tFNVCVNLlqK=R!%ueH7GHeDzQGfh##v|9Fu9rLybC2d+r-@ z-JBUrUPIE0?XKq`nj!@LOblfEk65DXZ}Gi&AYo|s7_=Bu%q?kSHr4b|j&BXoipvvV z-eZU5pGfByUk*zJ@r!2gD=zR$B01C&b7v{!XU1e^Y~ykrqlz71%9omlzGxH76A zcZFrp5YK4J5>`aGiOc?JM+1II==I~?LsIwf4*C^hdJ75kg7I1TB2y1R8>9=_HENPJy_?CazEnJZ23*0qG0cwD8LBcs4> z5ubFL4WC8OOlNDdv`~uonYN`;XC~H3;NxJdyH2a!YVj$&yPb<^BhCkm5ARQBg=qYs z-%7cn^HSEYT%0^Qxv8l;Gd|rp|D`L$JYbg_J<<8S?TqU#X>pZ0%loU&TGerTmv>Mb zb6cdhm2-{5YG>YAT$`+!sL&Ta@aly?dc1GwJZVmoyV-#onKpq)R4~EK%z>0d{eu_;+2K_f-`E+3aE@=K81065YI^B<<;MKO%sAyGKA8tc zm9@%fKI)BCT3DetYJ#L=*s-@52N)5M9gjBQ(3aGk z8DD!=ii}oDHt`Ey@QZ`-%OGNaAFI?07c@V9-Tv4|W)lg8U#f@ybrrv`(+?)}(IRJE z{HW^cN7YGXAO+8YmC&-T^K+`rj=Kt9rSdBnIfVz!`#(nbbMFBGeg@WN6=b2|AqiC%ArO76;q}D;5;jdnd(EKAR3n#jD4e7 z=a{)dpQHi{EeD%uhV!ehO&d;zen7#Cc$%$meWQDKq>=sl<7=azy;+uUqFb01Rngta zX^HP$(uh(}OeY8>)KY-myjh#0*_kKjP;#CH^OH6D&S}?)9n6y;W5t z);bq%xWlL=S$tq|3~tu>%LU&H9czAlWV5mytWB;Kk0>TkHGlguj(l3;Gg9D4xim~$ zbH!kBjL;RxRrKY3I7XnQ^m0MlceR=*0Dl|e#dDu@z-W#~<#Eo9?x#4I4f}z%@3&=Q zt>N~x%qX^*t|Cw_=aQFki?PpZb7dxNlIBFo-23rDPxt zWX{Fu)Px?GBN|qlG@ROu;XT zgkEAz%|Qza?N-Z6d5cdLK;v;QZxJ@W=;r~ZiPSb}<@V1`VZ^f@U)nY zD_uerRcS^)(ZuPZiM$v%JAY#}Y5mMBGZ~k1;JDl#z`_V>d8ikQreoD&BAsNIDx(;? zluy~#>?%&=AG@N%GqEZ&&EEM*M_cP7MTx;6q*N%n3lT1Z&+k%N*+U+_285PRGw2(< zGu-NE)d5MSa@m@Cf^pR+*Cy<+*_vvF#4>0Yl8@aPk&gkFhE-^lXNz}fN^_z^hNwJv zKUfkE+^8#WCy$6w>xrX%9H7nk;t0INebRuI87XK|_8O;b*Oa;3EhIKXtP3V4iXFe$ zl9kb&x%Gq0Js{QyLgrKEIU|tu$A8vX|1!Uw*ajQV)aQ@uQ{PE*+ zrRhc9*GND&DWjn)9GQq#T06MOtdIPR!m_}MJu`Fkt*sUC0+h?33W+UrtP zYk~lsHTk)1iRx)6)7NL-Az?%6ZnlWPN>YxG*IZcB{GuGDNhyxa*1V?2AdV zJ_O6Zd%hX!gpm@{8@ul9FMgDt`-H_m9J9^V#ZwC+_(t`6>Z*bXS+%X;YpKH=KJIu^ zZKB#Z6vsx@jN8-x?e5qy22TqXaEx02kgi?*RD=}=p;BN{M4L_xY@PtP#L}*-i(afR zlJs?Po|S{8&!7l1>rLaP?t$OH8l8Mwy6eQjuxP3W_i^3u&5j2a1d2}>lW+Z=&PCmI zy5IKzuP4du=QH~HVO-bQD7PB04s^jP98kVO)0hF3*y5HbVy2Nd0fd0Q<9Gp606FNq zimo#E6~O-jS3s!0dS{C{KEO6}80;{I!7g(c>@nvyz!%J6@FjB?1k7O&GRFhB#~cRt znZw{K<}i4`oCd(x%wg~ia~MR-VGuLN1=wc}gAQ{TbeY58K+*M4m=AeD8$jP`bPMP^ zT~AnSV)4DA8zc4)ywL0)HDdNp8Zq@}MYR$27hY)UuNpD+H;tJ3P|?j1^${;L^>>Y! z`iDkLeXO2NbkO z$H7VGWOF!*9MYZ=PNa>|C*-8>N5y-J>?^ zWv09-X|CGLcyqL-_a37?J|;0E#-1ft%hf|Slk#NYu^LTGxGa}CePfoBPT&w6E{8)G zL7sUW6^8@JL}20aNB}vqkOoo$;sLOePyHoG9i&qKg?#LgE|S05y?Qg(s69kvzfLHL zQeCm7&gs7F{-;g}+L$x)Fo+D|rI~Z|E{r$H53dAacBDF8{`i4OLaLN%drA&rPf=^g zC%xpp3!iFl>8{BdLo_=|8D~@{$7#ZqVwxu){J0o;OQWwTNbfO^Fmq#;Fv60gGsp7; zd_K*Ayba&raJc`1ywRKx#G3^01b}Do@?%gL6zGpUGhG}eVP2zkU}z}W{o6}HX59he z-z4K!W)g|+zh*(Y|Bfq(bRT234}z`C!%8hZFvvtbz-Txm=Q$3CItyt8LA}OtIMGlz z)b<7~Veou)cv_zq4zLjzWW@Q&%d4UW2bs43r+!6^`GFG~U+K><6m%C2K3KT!%5sK# z#9TZ*ec?JrO!Y9DsJto8`|cS??by-mn9)$E3V(R>ltc3}k^TW^`taAZgg(njfEOtd zSn-Jf(v>@-kdk>AKpe*dzX`XI9OEGa`VEGp$jv6UI4_gfiGI~xQbwTYQviL zh0kv)uiErXZ{Dotif9>#5l`+}JohS)JoFHdT$?+yp*2Wsv01A2|7PRn-|VZDMkX7j zm=ESC=A#tqKmw$|BP?ux7q)+UiW4fs;z}fL!<;=uLrdyE8I(<}k4&$%#g<3NxH4|A zCPEb#-%u61=kteKZ^N@JSoFs9K7N*>#2J|yrI1Nna-M)^)R`lK`UQ($1=P`rMG+&L zhHXmkgX$G=v`{o9!KGlM8B!>#rY&I5#5fOz>icq=qmxOqn>cnAnBd z1=(jjq$h;bcq|$8N5zAoUE~sX4Ycsz3=aAXnuDKV&QaO$88f`__2iLf4ZP7EnacL` zUw4}qBXI#t8QpoA4?I2AOKNaTclGcZ2k8pn1*T+z|AG}nmHXl~atU>l5e~W^fX}0Q zr$M_fg!YEKjtnswW%e3YIDst7l1PpimeVDJVq^olaNc+OpJ2ZD$(&g)gWyLS_ph3h z=n}_jLvmVIOkVN&_VTJdzqqgKX!(Z<4e_-5z3M7#`KN*8>KtuSVO)z+D3!!S>uL_X zvwq9(?yt7SfJ}Ap0uS$W9;;`7_=bOW_%EGzZRqB;Yb1>v6tUC2lpb)eBkn`;5}Z)@ z%s=2WBnK)U`J-r)=TX%7=yCWjwQ1-kNrRr#2+^ri5Y@HQl;q$4zIh#?qa;f}2=h7m z^XU+*z&JSu$O2e#?HarSoSDoy@JskD+`ml`kE7cX{L<5wj%%S4HsCl{3-N~xe@#3L z@gqra$G7AFR}O$U5o(9bPCT5s68V6R~TiO`E9HG}1o%!vx{XJta?F91;;EFon5WX`wWTNY|>|eONz`9{DLAD>%_A zqTd7 z>6^oZO~fzUJ73>m5ME|XP^95u4~^G2Ml=S)7TNYpjILI zP_sa(p+2D!?A-0=NP{WwlLN|>(Hl_x<4^*uKn(hougmmaNf;iVDmP;gj zGMgX~@?Zs(rnRYt;04JO2i<&Am#ikg$A04o#)ifXl3GHb4bvIiiH`_i=GZ|F~;KTvQiUZ3sf61=Y4Z=|6= z!Tx)Y8A1*fP=h00e94tfWo4VLEb)5sZ?Dardku(Bo&@68=FNMJ`S#?--|VZd-uIi0 z8{gSiRkiOO*p}0V4?z0w!8kz8NrN`RntiG{2};V6g{|oI?ocm06oV+tBZHv}88{KV z^cbR=pVg9euo>}DQB1bX>cgzgy;9qy%G6iC($|j}Uh?MTI%Q;J*M}vLg^g?8EUSx) zM@(&bcM|#2namfDVa`*MwuWl~H4{W$#QbMC7lvwBoplUf=OI^DpW-2KI)?d^|#vHK*`K(@PU(O7V> z`oOyzH@wVGylUiWBs8~$YWPt9kQ`CcZ?O{KoCTGt;=g|G&m8Jrn$Lz z(-?D%t?nx!e&MmQjUKDJxmBqJa`PCI?0)w6+9Zxohwou|P5$yij(kqj;0;RE8ltFS zU30>tkN%dN`M5*N?37tz4_pQJZ~f`I9NUx)lcPh4Phu-(=dNAv{=}`1B{(4}_c?mN zHT%Gn%8k9{3RMjDVFklqlc@jYdd3%@A#qjyo*)v&U4+QTbLx|`aCQdOimQe`gg$jqOdW{Gt+t(#uCdqF{E8Otkk6!Y2^ zkk?zpuHdeOaC9A*CJz|J#%zAC`mSYbJbZ7pI6pv8yMpSPd~1=R%y8VN*5W)3MjFTf z90qDFW4p>EQ8TIk+zg`ra|+)Fb6aW;yie*EgEJ)nR;o+}nIQ*cV8_f>->?n}Pehnvxk6L%muwSh?@Y{_We|xazs(r?;BU$qf zwUM)i23Gy{XiLk{->!n+kF*_{myPf-4`Y0G&v?S~4^<*Y!k9$TgP#pGXkC8)(|rm{ zgkmIm3BzGck#o&kJ1Y>`R>TNBFf_2}wFhQOg%Re25KlN$fWjQ-G&9wH>WQW+I|^F_=}SZI#Pl&d2@mFdxyF3VuZf>-h|mNzA|g(1jW zD2gRfh`{9mG;9f2?z}-6HT0@l%V*Z|gP83>8s#PMFUE3JrR0Nq;UG7Iua>^RL~A5O zTZqj4D~X2qh?S5t;6v5SipOtVMu&aSc+9Q^W$o|y%$ z`AHhNP5|DZB2tRtVltv-il~%GqOYztt!sCqd$~;85uI*P849|KvlceFsG56M7-9_y z3G3tYnRDdZn9qhWe0sVC{F(?RU^K}m%&6dh4EuoXG-`_@LKI9-#DAnm#?jfr7^&Q! zqh6&?vs)FkOcq$Kbg$rJk_(bk1arL`P}U$$2&WxN>ltCCz{kAq<|TL4rFWLc!Ipbw(enKU=lSLJyJu$wYu($a#H!x> z;`VGyutdg9GBkJ2Nxk=_w9uxx%QA}>Pqr2u@K*}an7^>ribn5+w8=FouDLdY^IBbVp!3d{<_gJ}cIhw_r|DcSSDmetMcTzMyZTlaK1vKBqm z-L=gw;G36R{C9!#tv4<|SvzS{c}vl{=a=-qwWp?{xZMm1z`oNke4X4z(Re;-{Pc`H zCBo05@W@xq50*%y993x*i>oYgRr8DT8(h(HwJ62VFk_Bu`l;nbxhtMtGWS$2nJ*M+ z!?ljC-7W2VX4oUl5wiT)42!&E)3sHLUflsL4%3HYGooL7yLlwi%RzuyBzua6+?IZZ z9erzTu@&LL?C7hGi)*NiZ9XtJ%fnRC1l;c~E$8P27;9Jat2Vu`lw~(>oIJ3tnxOvP zg{qWrqMj8{J>sQMmrvn!c;M>vGeGWSCb%ns69Ed8M7rD@h@? zn`US?voSB$vAh6N0A?H}huvXU*-4{6PoU%=;wtYEf3E)ZuL}o>FAJTozSz0F$x=Is zY`?iJdR0wF-kO)!^u4^jDmTr&=#PJdJk$?=3vG8ZjFAi2sK7NcF9X6!l{Y}}umHzD zp2h6U1!hYk$VG#IOQYuU2utab`b09=Q86>8VsWJgWp# z_I!Gxsi6Nt-?T^8ls7hSeJOuyn%pQXiwWh$Mde7dZ7b7d1|jQjC!=2+oKE1z00<>;b3xz;^}OlnwKRyd<1Nv0MN zy*aZgV(;I`L?d5Ny6vqEbDrN)mA&NbytuHA-P4mIqM&iHd_cd?@=@&1hhHcZh#3?t z@kY<#_3$^j?a0Dh9*^gKXpEaJ5=pH!3-TvJ zmVK_+$Ge~i@?Obly@h7(SQs>wI9{MJ3Vrj=1iZ*(Q{Ru55w}5G2&Hkh=2zW`#AL@P zsj)54;q(if)Z0ot+BlGK0p&s0g&F<^19=a~?|gp?n(blE!xQ%yY=?0?`6(UE(_y8; zFNtG~)~D0_!C8~j< zxV>d)Zp-C_+C!YjR#r~7sU~E7B6VU}xO^_SwJ=!8fRhXds*mYc(ERRbdC-7PAfEMX zo+^8%PpvzVvGj?>=2|@uh}8zwg%d>AMW(^SMcUM(eK|ovQ-k;fIB?KD6u~y*Xc_>; zfo$XsyogPHbcq+ziev}7RXWL*2M=QG7ZBOBWzW;EMC-4Mr3dK)dpx-P>EeD1=}@oG z&%rz?&R-g|l!d(Th$t)vW{VWf?<(wAvcuaF4SHQ^5i3&GGNDG;FW|{msY3ccJmVp? z>C9t6GlTgZ^ER^_Q2Y48jeMbEqXO{Z_>bs60qL7Kd%^dZqf#6PAoy_zM{}Zu-g)_v z;ffw}4Q|0XIawZw*OcY*@EDCIHc}4C{rj-yVyRUj7> z_Y@lui;WkVoWTD6{7Whg!-%DafG%TN;cP!1aUbTT!Zu0aHlqa)Bo>KYWj85qt!8}C&#E! zj~?$7ethkYbaBUi;`Zv{_;h0M)5$?gbM8||4+E6^d(%=Z?5dx`J zQC%>RW4`&@?A~f?P*8_JK!8bgkEWNw7AJ5#KxA-`Q@35f*NHC&P4C$fXp5Wn*d?0F2qi4z0JMCG2sl^2}zW7hAyRUQz4D{mYy z{54$+^ZsSbIZ_xs zPxY-1du9efY9eKk0Pi%aLZy@fH9Nn+g1IB}bxv0z@T@X;Z3%qotWOJDG1rIt%y_NNRzM{b7zEOS+VT}m zi87^C6BQjy3DhYqCHX6-rze!pa8%5WP!^`Elak_s<94){uV_rvqqU)3Pziseui{;> z0g_}p$QzybUL+Tc%Vw`~h~Y;N9-fvX;Y2}9DEvd~5ok|`T@eT`5w%`qkliB^-6J!I zR)2LOXbc%{kU+UhF5jgT0O?R;0FoRq{4J#UKk%zQFz@W_Ewn(8n|GoIA-~W&4?8WPW9&6OG=%X!QPr%9J>wmqetOo3D-VU0N&>@|Azj3i zA`n$j?u!xlm`VVo9((?m$0lw$GJ?EY4|ep`iHESOtY5^mAiw`-TI6?3?*z0^iy&I@ zuwkkpVYCkcxgm(vyXTIm4n29{8SSwPVd>E4#2;#UL&YV;O!r4P#y?{rP&z?m?(!VQ7{5#dIxn_YlZD)JWePQb2rWq-mO1yz+ot#Jk{yv6d-YOW^`7(X0$VaG(GJBefb|*d3-p>Zo6|#L?fntO_Qw(Z@m-6 zn;V(~l4LjERcXZ1*>j135%L6DC!}m=jsO=-@qeQhUZ5UYw`#w;2m%fmxnFMh2b z@o6?q2~#8fpdPsEpdNxaI!@Kd+B6R!6?j)A)xd8dhFy#vweaRWKEgH8pG0IeqvhyW z9+RvR3uxXSe2A-0T&dCMN@+3>PcTQ+rZ zLH*8InZeqT1!yMb%}Z^2W~Q<_1-)q5886JBm8PZ3`ue@9Uok(mG0VWNO|$C}yO^_- zkm9jxnckHIcE!mudhNkuU3tk%%XU9tD(J{=-ry4CJvy%itr#vsD~5JtT4?6~#m?Pm zDRX5_N8tvvin*&UH;u`oE2I5r;x$XtSelOUG)Gg+d#Kk-aN0)u#fyIAHkG#&Y`BWa z73VT`UI?Pq#GEDH^~yJHtyVU=ja*JUX|Ubvw^n;0_jIp!NwjmjLlk5t={%Qrt(JK2 z#wGA&`4sP3ZAor*H0uY5HzCas&lnCfN|jCMhoV z=+zV%H~Sumbdy#T=Bi6|uO+`zhq8X4aQJId32iV7UT`PgdPj`iwMM)z(n`&)%2aEa zJ=9u0t#ewrHD%`RsdFCBisP$g@|g7c?9}R1t+jl5=k#)`t$lM-*Mk{kwOSpj(Zo0+ zEU9L_AttLiyJT5&O0ly@ED1A)g(hYh<6IVfWNglq{EWGEwp2vatLDStV1)C;`1u<;vKb9YaQvGTdQY0h#EyMk8xHw<4aO>X$_qXY1YD;LaR2d zHLqwvZTyAewU_AgSyLC-vziK0Z1zY?sx_&kr@DG&qjjWp@}kp9>~R^B3mi4M&a6^f zWPYk4u3=5XkeU2w_PLb>Y<@~S2Kk5PrSPUXgl}kM6QXyL+t$I%g$E=dk&47}dl+6X zp$c&}(jEi@6m42t{>^LTG!e>K*fD+hYvKhu2aOms%8T%k3|HX93B101-O6C~nXw6x zLNcg7|J7@ zeT!OS^s1*X_4Ou!BRq~@Iwv@B%sJ`-90S)wc5e}{-;Vb&;XL_gEgSc0J@Nd>h<1Bx zS9Mmh`#-rhzZ`ejdY?GN6&&iP#Ntq&f9p^`fS78KhQu5_Day71l@8nAgwMv@P7o^n zg8M%cX`Hxn3&P&^E`+qQ0@?a4=L#n71%LLL;TX*^vA5_Qfu{ugyNril-fy6NPWJ5k z@}ESz4U^+lH^z+L9<>-Rz9sX;0R;Y*o!z1HM=W-n_9tu%R(#gvSv%)Ru$?csQ#)7R zt=wa}0C%!~ZU}S5lb)2;&IogCVR3jmi}osGqE5joV+!%8$FVSGDg*KZEPJY95o?trnsl#XhL~=e~E(R1gOu^1~sHN_u z(IqBRNi?xDSjCP_a6IM`7UItX|a`C#%)(-9HzEhHZR-nRY}IN&{)IP!bGyT)mw9?#4&T5U`)TtdY3a z$0a7k5vvjN0IheEWe_Kt!%%WZZA2%_#6tJ`M4X!ziHI2Yhs0OpSw)oCZQfBXHU$$O z?nL=X7nw`6)8{xsl=*r$J3H@aCp!3ZNk>+wBgS5CVji)toaAiC2+xws`J#}dU*$c5jyj*sHAjpT|z?L;?jo6lN(y<>*=%cwM$CN zme$3`*DWn8T~ZtGcK3F5&7R%W)yvxD25JWp3~2~P_U~9okYI?E+~1?pNxq=Zsx;Td zQb53a6SzTyk(Y(;Re1IhSilzsqwLeQYOR!cgO?SgX1=5)AOhnCMu_{O2RF7S6qOSf z|1Jqt%{eGBMABy&Mk>CpQNeqIxk03g#qF^TWDqLmvNTM&_HQziftSU6Ea3vVItcSU zjoL?C!)LrX-i-lhD>L32KwR^z`qO-hUHA7a=hHzHyPyv^4N#VM@Z9p4nPuXg(UZvj zlg55@Wtrt?~iD&6;936Vj>9TvQ$z8L{N4SID-&UZL zpWMM-{M#C8C-D%pOO+?$vU`tep#}kat7hkit&tehK3?A<-MU1rX)wv}!_&gHM)5}h8Ssw=;5JJ#15 z)Lf#K{($vW3cq*{>B)tiD(kM^h;)_3#7;_!j7*yp8&l?r>~bbqT`p^qlQzW_rG$p2 z6vf3B*+N5YMX?EKX$jV}G=%>*^mU?|SATo_zlpFHnJp!$5fP~+7ITq3qSa|b_-#(^ zNlQT@j1LPemOS*CXYs@TBDI4o!oEu3@2w0Yekpzb=VamYKR+QfC{C^40z#kCYBnnbdL^8qJ?oZS@x=64>#ct2sz5!KL(!Qg=Q z+t~w0S&c8mBO-m9b+qxB#`ckz?p@C&oQO~s$;HfaIP3^Rgv1%4_{J9+LizeJ@)dnWg0(_F)S%A@Pj@-9sjpa_!d^@N zjeZpEqm0JK2pl=2|Iu)b%77w#13Irk`sYtPbOiEmQQ__%>UPY^sM~g_tLxIXx{O&n z>fAqsM}hFDDs%g>MS1gI9NM#I=*9VYi;lIMtD;VIg_%B*&Ny&o&YCN0a&zH7bFLhi zA^pe{*2SkImYsTQ@uBaZ>gagt`$LQ0I<+i<=Cf-bXQ_u_Y-RTFh$=$hHnQL<11Jsg z3(Mgb>fzkiTK}C6EYKRGkIEuT=;&N)fb!-NWC2tBQE_|<62brBV$yCli(b;DWS`~7-a!41~$R=<;n_bU`iU^8`4CRtMGBrQn8J-vs ztW7SlUgwDfJPB7Am7Hk+#m=MxwZ$H8oiV-5JUq zhHbD6!yl8da0&4xif-P3vY9xKQS8~s#|Q$; zI+J`Bl+!<=`Xq`(7^~r9QxZarKF-C3hg9Y|r6gY$9i)s93l-$ErN@u#N;3xO`H}6t zeW@w~+DAT{d8D-7d`{B&@+&i zcIFx=0$j{XI68S`F+cIhjK zR#9?frOITa{}^e+_ddg%0(rC>$5eKi>6slnX!j3i*ckA6<}4`0zwy{#(MF2rckDp- z41LD@W$3S*vGf!;p+zpTure}PM6_1u(?s-DW8@8s5fzHytR_UnT;h~BhFcB3WaPxD z!x+ycLQMXa*yH`}5*a!45wT}I{%tKGrZ0^Aww6x0@wWH3eC89fad--@dj->9vT^92 z4=-QN^7anrBl0=&4b&&bI*NBPyS7wTZRyU;?A}sUxuq+Um|V3LU2UzZf>-cY)Th8p zV1W2M(DMs5v5xSFw3v{Pn6!v+N2~_A9`BVC1=~*ed>ivVc%Lgpa|{ZlT!YMs9nW0I zq`1U)^LvAXr8O13`QhRDy%jam;Naf*#67=f<~2(m5t_|w0wT=DrQA1_S~GK4?N z%tJEeLY!A1PE=_mm9U_$nCm&`$g?C8PM1>-4D);W6j#h2!ORzPDL%#x4n8Z@$i#H= z(vLs+e8tMYefsed8!eWFNS_4<5&mbH4+%4DcQLyMLyb0~8qfd)D$lAznZ2MdR84RF z@t0xB5?*+T60yO_VLk-sJo#yi$uU*&z}9sk%iXm#AZ127@bB4ZpTjD@9?> zy8g+r!Sdjef`StG5j(lR&KV}6w{|zB&F-9;5}ltKEf7Sd=0~T@?3|s}+$C4mCN(TA zPVIz0=GM)v%}bd%vn#cDX+vU-QjU2ofV`^5@fr`Xs3Tl?4oBi*C`f#-y`X50P>(R9 z7b*iVF4T`OPNdi6Lhd>d7fH2Bc~`TS^(cWL+RJ)Vw_I71h*kTn$QdZj9c-cePn+c#35884;c_z3PQVVh(9DeArBJ!bFw%g zhHPUxhQ8NmNC7yJ6=4H99{60NVM2I1V2jA|qk&G+V%vpkU`atKv0(-#X&|_tvL0O-;$QwQp(ad@zIfy}dNu5uvL{ib_i~n@q~^N#Gx7$yyti7GX|L zicfTvwrAJfXO5m(JhB*X3e#G1%+>;XT!Ou1YF^E%DD&)!jJdV89Cn?$WY|Tgz<3em zU*5`d-W<$8r@Vih`N#3uXYR?(z30qqGLGilaM5qxaFJ89dw170?dncLI5rZ`Qxsbl zjh&w}Tf8&0z)n%?Wm?{6ItfoM{E(|tZUq~c5YH>5;6+D*#-?M6B}(Ei>{zfI3{q-1 zNQ91^|Ka*3fQXy_vAzx*)n6OA7mH6Cy2!hEn*&>Y1YfocqWD zpu@G0pgrRSRdW)M0bMX18Xl^rgXsb0gQJlqQ{+)#MB_HRpO^*jH*(&_94}nkS%(Yc_rS@Gc1m8WO zk5cWM_T7QnftBS+4U3DiGaOxeT5Hx%b5RQo!Mup%;>47?^zdMl(+rN86M2MEJ|93I zbS>?~wEWH@Q++Y>xkSBx%3SBHgHuaZ&uIv+-d8>`1@psaquuRy}f^WS=M->meQ>LvF- zH+SB1n=31-YD;1#b!9X@v7<$$gzeise4VROQ-ourMv3D}qkI3R9!174`AKnlPsO#Z&Nki1*Zs-CZv}#D0e~T2Jp; zI@xNSymZg$*3JKTvO96|y?f4ZD4+a6oJdXvrwR8yBnPgKre+-;noUQGLhRm7KS5FY ztMF3{4iL3`G4oESHY@}riTP~I-bz}CztZQ}dE4>xgrj?0Yf8(# z#l_27?Dm#r#l`owq^t>dPKt}0PF+&c2UV3?3Q}FK1@b@j`hBZImQ>n9+Boqn6=0x)wue#)VWsE8~ zT$(1+zx}os%AEWViNaPoqbRCm%iR3*1t(?&flzgUc=kYMyizBLO|MGc#LIrPY=#E5I9BFU%7W zI(0#C`>{p21#4bd2Cr0w!Sx#*iF~O<6lu+kOU-S_GzSaiG`W(S-np&u7Old+BP0G0H1x3t{W%BTEE9P5iUZo^dBUut; zv==7=gZB&^)nlpOYfoo`Ih)STkeUMK`JUW%{+F2{eYLV4(beippq!~A{|ij z4chyOMLoT%b1~e6%bBY80(KvowNpAD;Ujann*jl9)Oxi3NNu& zQkZZoht^Ybfn8z3Aq-w}C+%oLfkzVWeV_k5cu`D8o44m*)N8qde>K#J)lx25`h6)k zQD57*KTONz|04f$bjx5FH$J?!;roUfy@8%Y_A!^in-%&{=CUC+ooN**!(^b1IUXGW z9)SQ*QmzjJJ>f>C4V*FSnd2@xC(n*;3Ruvn*7&}rbYeOtF~CErP~zAvOH z5w8j?y75oEJ`DA)gZ$&Ft7`V(Eo@ld96Lmq1bow~1P|{&cwcWEDhif?pQleNh>9tm z=5)1}+JZ!}(NiUL{e$gO2A3At>gQ%Ln`7$DXN{rrvK28h@XL(3^A_gjPS1&mOs`Hd zY3SgYhkG0_P@Gn~ZPmS5%o?E>^3yc@H5CH+De#}o_0_9K{T7CO8g>gi=Fu^VSIc@y zZU2MqQ;samG3B?X@7z_f{)x`T5A~I4tHRU8VoOnPUB#jbb9nupzV>x>(UzjAPUlp} zyVw#N}X_kYM`3gv}k?u|-4B`01uL(oKx zE+^8R-V>~5wty8H*%b2l5T3v3gSsFf&O{Uo`j5}PH6`!x$>muTtm6Q4`lj_gz9h3| z$CdfHOS_v4XqZSTn^9P?yeZLAb8n6Fp6tFQ#3M=ROb)p!q+sdb)Wy$lDAC3_!z*|75Sr<(WYU324vK6?m^g=vB+*j-Y#(#*SU%nbezKwy-h=hO4;+ z^Qz;(3AI_~+Fn(+Z$W;^n)9<~J-R$!6Q5&YPKj+1r^$a8%`G=4Rrk0G=U2wYSI%Mh z`cz#)qO*6PbK2>ZB}FUG&B|Fgx7Wq+6`nftvmk?Z;(a`q3;U8?=Iu%@*lu>F_W(Ul zAPW|gmJjANHa3Fi6IyvB$*0m@cCW~Qko=4rz@3ipM?%Spw>7QlB8OuR#wG}!$j|=%yHUXwm7+j63v+t?yO3R zDbC3Vf0)b1w2Oy-APOPvG`~K>Q%~GsMwqg+@e=YIlI``Gk@s$nE1qIYZ^(|9y3`gv z9jvz+Ba$M6WtRM;=F$wCEnO6())8VyjU!ZVFN#@fjb7<$NY`tV%aTF_0$ET*xJnrj zqY94?lQe^@)Rfeu7^Rp(?GDEgqJ#J&jPTK_yCc;_vRy-Ea(KH2*uuRIej*V~p2xhc zP?ABQxgQ9>iLo)yL2uOw6CiIG)~2sPcV zkHmWE=A5UZsNJ{@zPEU+U1p%7nA!W7IpEMf@Bq=y)PwU|z++4!YX^sjcJey$iFf|^ z5ZOL-n)qZ3{%t+yeTql5;P)2@fJFdY0-zB9@=<1xIXWG5F^^0Kt)RJsISXd~8FVs7 zJ3tp`?O-0kFlG?t@LimjC=<07pKXk1X@D`&w`|OH7x~IiKA1Unb!_Z4H=4l z!RR#z!Aj!QpkSIS6@C(=3tkKMmxF!x$+ST%uD?KBpFvz`E=q;-nP}+X%$8_$VuAc@ z=w>iNWYyX&Os+DR_-Q$_U~RB2=o6uoD^z|Oq?LKz%h1iKh4yj*wv`mEr$cK|jhJ#b z`SMT!`7+bHpP38p-~Wd##1~u9wEI@DoY~4g{{p9$e1+NzpI3XHA1~PpBzNcrFlcO{ zp~-6Sf=)hkgXGF}5Z$W+DHJg19bs7TO3+^pR;&yTW9vD-3Gx1F6y6ajVH$zOd=ueC ze{vmoL7f~LYSS>KIyuDqvvOt=LiJ9NluJtmze4y~y&VLZMqV4rnY8_9 ztB+rwGQ-w?(1Gw(GZd1&!#53ZbSC2rP}vpUbNDw(wI$ywsBr(fLOu&lqN zcz$KfqSt|N;3yFPbW!T`O%Fah1L+_Sj?WU9LyJ(A3xI4E#Uf@rxkmd&Fs{;94_D(| zjz`myL0QXYRmB4Xob$%h2j*tRm(6f?KK;Pj6p2pKcc{&5%xz7J znzVjKnq$T5yCVcrK5VH49ENTnPvBU!#dSGp zvydrZZ%9&^%?3x2a7VGlQtSx#etO|VaB|D4%F0!(He+UGTwGI=eZuVSTIBkkvTHQeNF;Z#y_U zz5KwR4>s+aol&{%O5f}wHUjMEk1tJ&6AR_S`&Ccr~rAUtIcQX%>Uvj76%z;ftc@oSpk7Q3~ zx^hUij`C-sg3h&-rt@z+Dxx(XJih}2~?XGim?_j~qFvx;ODA~BZ; z&6rwnv$$vfbgMYH5&}SI=F|ez#}Z&1_B-xVC}tJl+{&|oMlrH+oBEym4~K3%78VO? zG-34pLNQGX`QT|fOu>EX$6DerOo-G{rrS_p40Y-0Qg)AV33HY@T)cZ8&$FtmK^oQRURnDBHe)PnZ$bWeDBntp?9tP^ip>87E{Tg}nW*uKh-0R*h zR z$tBDX;OHfMl@y$3x>hk=;MgkSB|@vuj0n%vyFYV(omUW8?EIwI1WbO?lw(xulxf)Z*H zY62nQ3p(eF^Ub;Uo;&w`=bZolInT3M@7{~I?Ck8lSNp9uSE$BdZUXS2>sBYsFij=t zmvZ3eEZ0{pL6*a4O0Ga9N#*#&veKo6SA2^q+jNqhH#OoRGs8I?bIUe?m6=ECU&1! z%TmzF`AjlwM71cj2I?myVaR*Fcvs*7A;vY3+j{95#GSde{g;+HXV+^1T2St+T)(@%p@u5+aUS7V*2nq>$$N2CyG#cRf;ym`!=XQ0pYDf9 ztv~kY^J~ANGD4WOrSElueF49ke;f>WOhg@yf`U`94b7?hIX+S5%k*KE= z*vNM&ya^y&COF&T^Z9jV5A|@S+f%pU*J3GS!)3eR(3dYm!MkO{TI3#M#-p!aFPO4$ zn$2Zrj%k^Do_6|M^wo9&#>Qc6+4U=-Qt(#=V)K5tZu!%Wl_(xk+a*gCpL2{;BacZDpRJGNX1uQg%cJ2JmTvw@ zDt3mRzQn+54{$f_IpOx521dcV_;(lc0SiUmw90XA1Bdz=o-|Qryq|G=A})5@$+;o# z3Gm|f8)*}3zq`03_j2(7GJV0YXqA=WAyq^hUk_-~nQ{ z0I{(l$L$!5tr^?BuLLd1ae4tH8gZM|cJ$12nxAjUlv>xgge{RFh(Lq(4&rP=O42kn zwp$SmOK&LyNe@uE+QY@}bFV2?I3DzC)LBj1aMRbYo)j(bNHI{)3Y^Aiip%8Gz(IBm zpHD}&(=#u+d=fMqdsZXXVWb~>#MSWSRN|4ssi%phw>Mha{W$`^z(8Tj36y!qd^z`0 zx5_Jbnbvs3+ehXgTE}w7u$#RPlpQ4R+*=8E;V`5eJ#Xxuz4)qhCp)u7@;T(CEf2sJ zcn^AypX+Y$0GF`!-9bs8ocQSh+2~c=T8b+~2>0W)(JQRw;30&mbSUN_D+tTKaBY z=c}txbUM6j#lJ8WN#-?(LjWtKZRqfwmzT;UNCd~uPxw;hfk{YXmp_MNELlF-fun5q)|gWauMWpnF1 z!NWM2E^_X7AW)He&*ae%O{6cU|GA=@Y^{3Qu}5DhYy9B{VQ1GY+ulLLHC>yi1KHR@ z!0u@Q4FYXu?G}h>%M47=v8jFvH6O>ojc4;wdm=oS{4uK$fk~1>!>yGd2fdvL)WRtPkhf#^1F|a{lgCIzTn!XBK_6P2 z9I;J=u_VCqvL(8S{Ehg1T?Bk&M_G;5L-O7SBOJL$Wz0W4daO$sNN9TpSn(NVK-fli znEyG_m=Ji?^VL$d-h^%yWnxu=mhapzlN~f)T3j=VZU)Ua)VO+|^5%Gk0(vWlM6Vcc z?r4$+1c)avLc8jp-;d>mdULiid?a9sAX8x|Oxb!VK;6)O6S9M%8GTjdHod=Zj7= zr9DJRc7m5)y|j|>bIeVnBWVkuKYPrDIxbg7LhA%dbXO&2nOM1lH~@*kLS~2tIaraA z)0kz#++pg9y^aK5cppNf`=sx~bDDtjka?AkI(fAE+n@{aM>Y^*S=W>JJFz@qOVJ?Y zw`rKYL)Imd?LC6->fC8LFOq$MI@{}H*^tv;0;UmoHaj!-{ zl<{bpSfzv5AE}tX%!yv6uB~=G$(AmIJu6#s%Ctm#U6wy*#Ge6(bg5!3os$VFFzMD0 z0OQ!be)4v&i=;6i^8c`P?*n={H2`VJAsNWXoU#}WF&eeyJl96q0pU68jL-mN0Wv?( zYxh;i)F$P&uCUi`py=)#?-2S6GxAf7LsBMI9*xsPUgbQ&|N1aVHoha@@0N~|k6fbZ zXj8Fa^&Nux<^k&_p?5UAx??z8~YEZ@GEdFou}G(B_qSU6<~ z&J5$l1r_&UkI`+*lW%B-?r|k#~uin#_oUbPtz7k;;cB=0R-13oe+Lx@;XeK#~16*jF8LM{N zr98b1CGWw#E&!z#B%rU7IF2(>O>m$QSB#^a(pyh+iE*Sa_Nub=(O0NJ!Z(RYxlFmD zpcjhx*n=@5R_{B8cRs}O%gQO=ECQL_ysYR^6qotNQK0^{Q>41L2Du}xAyU$u~R??Q@1(L`)5$MO_N8SNBroc^c0 z-mjb>I^YXiAi@wN0BWb;j+@m@+RXPsDv z=k;gy9a(?5kssTx~5sDY2!o6Mmsr7X}7R_e;jPn+DfMF~<( z%VXNza)P46$8dvP2gxRJ5B&ZeEOI410Lqf15f>HqN?-{p-`8Kn@#@ukw;W3=*)<-^ zOYoMmSie`X9I^`LtQH7`+o6#d;a3Z@0NO09T3o5dxmFCw>ZSj+IGfbE8 z>u^?Xz_=F0VROSvFY39mF<`pj#+wELJ5+p(4hQQBwLac!ehZHJ7{B@O>8yme<1>kr ztZ?dfUQyprvZNHZ%lKd0RbBMfB$zB>8xS2(1SK?JUrLaSykJbcIl_9LqmW-86u)Wo zRK{Z7MX%}GA}h;epWZXio_H~0+7J0?)&2pUT1-HoU8tH*V}HM!E6}RsmQmT){8H`n zx3$L1u$2zl0Sx6{L1qm#JT=bqq9$1*xhy&|n|Q!4i_YgF;NH5}WtI5X-d|IG9 z$-OFYFwjz>c`&d*QLj9Cu0*p(NvuTZaM)=n#k&Kftn*rI3QuiVcYZR;kIquaXLB^k zhD;Z^d?9XdOKHrY(cSLbn zI1keM#9H_^={bTk?)Ud_J~KC8Iuwo#2?0!jnogOdOqhbNehl8Zb?(&_qUYbv37817 zP4ZwPMxyqx9*HEqd?U1T0$1wio3j*|CQ|?kEPAgLrqW@JO`8!}AaN|0xV>O~=L4qD z;%eHrvADD=9b01tA))kE>DnX)&VrwUw8!q15VduUQO1mJ;z3sx&1Ur@}VN`Wb5YZW1>n{0s6_0(M(#qb^@b1 zC&pUL@H(-H_9ltlhlH5;fO` z^w7t2`g}o(VOCIEt<9<@>Te%$D6(Qa^(3;NWK=~v=sQJ?&`=60yMvkDHb(^9z{tHU zIp1xJZ74Q%sTi}mKg3W;ty?KPnDGT61VOWR-|GQRvw$}#`g)4=vSx~<0i2&bY{-m? zhu`$d+wggHzZqsvmT01;Wxk&Ym~j%HKgt$dn75X-8~)%4aCFOovVjus+eQRP zO>P!ejh5X}jsv(9(lDYRkEC8C4?l3E_cEV2A`kNyYCV?q%?kTjYUSh0oL6sS0VG+* zfI|Jy+Mf^CjKa;2u-g{)$dZK^qQj+_tNYPMKaZ|ljngPL zy*7G(;42@nTT0k)S8spqd~H=w+;LWe$Tgv-KO^o%&W11!Uh);dl1o2hpYq=8OnNI( zl>W3fi6fsd>7A#GDW}uhSZ~J7;^cI@S2fEZED)^dXO}i+3QC~*jJdr%Bz4bh+{l_$ zyog0$ysuh`a>c-hU3n;1t5Su0DN+*Vc!~JAXHdDe!?ewkoD2Kndm8rPM-(@9GK*X94UAM6i8_iqryQL;B)85e z{KW-O)x`K^H77x8ZEw$2SDcgZ#5hwYNPd3nba1HRTsEP#9-Ma1L1vTm_=>_X_P(h2 zh-3VYoNsnL<1K+SD?rs7<$2qg5%HW1hr-8pdxHn;{p11WZ1#t@-vUR8d&9R{vMwk) zauCfL?|)+Vp0IvbfHHsc;H}0|IBBxd`YB&miIjKdwN{Lt@pJ$#=zWTV%B6-ibr2b$#rlor%6y5uIu&5k&`cUbHe|r1yCt za3eX!ndFKAm1O+1p;-6pn&s8Zl@z z&4Y8o`?qBC3L(n+RQ%@j&Y>bR_EzD?H66QrPRIZT#(M7E5yC#ah z6u2&rnOK9`a8Q72H}P%Ly6jFr6V*k-S9?aFxKIIs@SRZfxr9 z4^TN!dVF`I0rd*|N!z5`4WcjFD5=}Ske4!FJaJ_iZa0u5rtacG^2!n=bkQNN$zIgx z^0{ps9K+jm-lfwBDGnq|T{b}ny95w%Mh8`Z(JIx3br}3o#3VMV2;yL1rp!9I4W46d zY<06%JeW*{L3-Zna0ap~b8qpdJsTCaM_yuI0LJBkR+GIj95J zy5Xwf<+jx<59%Htl*|qR@C)uIp>`Q$1t!j0!{a1rT zS0hK8&FTI(AW+0nkKv#?x}YM+*d-<_lSLbX6T$G~cFbPSm%lni<6JPb5$5 zXTiC!9v?#p4^4A}_1u-b5D6i%8R%qVvpRL|4@lmYgDbjGw7vhlvT1K1b1H8yO~teP zuH~wFhk?mB2+r=A+Fb)T!%rCv=tb{Oby4KV56anC>wd3iO~o_wVN^2Z)#nK@bbv$hc?AlEg$DL5i| zmLGVNWna$Gwm$vIImDxi6h1+Me0OWt44S6ds*ai5jP--aOX5dz6G@3q$+}Ax=hEf1 z%sl+9$2~6I37?Meq>E>nNS{E>f;o0m9mi^a(K4)sOT6=%=XpF%k(6=*aH)Sl0bwf&lQP@HD zC_>BSM4PKu0;q)M7Q<#~as{`smejI@mn>WBQ)ZZpKJ2*MNLkn#VM+x>OLvCoLra$V52|LbEZrpn_2$$0ch2l4M88^y6E58Xnor>cU|0iZ*&JmMUq~cxljv@2r ztd5cFRdk*m&NGCVazS?$CyM30sjGSC(FFc9Z5Kq5R%kM|7m8D!c@~lcH8*X_i!lI8 z4!8s?nd%QS+xr4Xc;B*{H$nD`!eu*27pIc@OpH5+L_kuPj(H@Nai|yMLtPfFg>eW9 zLbyq)`eG-K={k)CEzQ5l|8xv!Xlbs{{NIH8sebG|wfisb-%on!)x+ndU1mN1x6b>M zJZ`92Pr|HFl~dHqUtYeXtv0K@fB)xiPa~0eKIswTfaFg-xO(mCy{ms8d3(P|Fp0$S zoZufm$hQu1-*YOw&!l30d9_R|nZ%OA@bA3s-~0F<&l9&5I#qFF_t&O!F?Vtb^Ti2P zYkBRm0*Bxs`sIF2lx5{kqnW4%yoiWG;FXH>Cyq#Xv(9JT8VTEoQfxYeMWXY${`q`Y zy4br$M)SG4y6g5-_f{>B=`8SPIJQlK$)s{^N%lw!?s{C6HO=>>VK zmYX|{0ueXrLPbb3l3EHU$PVqu=3;R-jvVI=(!yG#aAg=3H?UX?<1#lYw((qLQ$n}K z*S#k!YDxz}>zasL&M&@J5#V=Srm*t&2HS6GCxvLUm;6BX`ylA?Ck+`5n6aCd}GR{kbB+Qv4FP$@W|u z>_}S4P%o`wdH$=yWEK1O77ev7m!6_be?wK=HC?q&;~QNPJr^8DJ+|!*EG+T5{N_{9 zocGHlgS&R|+l!7G{o93NEKI_s?=?0#?o8O`%4rpi6^t^qXC@1U_sTA~*gP4z%@Irt zxgtqgxO_V^q{fwsBSK~Q2Sp$vh8Or(6+ZW%D9(QNl-?=Jhih|?)xZY)QF>2#>W##Q zVc@ID$>o@@~Xrsa&(Xy;jc(HOE#t4}syUOqlKF@L5fGQ^P1EN5Rj(-ZAtOCAT+ zXgsM>@_NirQn$lAB*~}kFc0@ge9Ad5WAaLJ8TVkYk<9H9RfmOS=j?}@#459(jOS)n zehK3yF%m>(<7oz06t+(=3truBctyk{OY&l&2eDBe=8KTqR9y_9LPd59cUQI)113Xc zV#2+_SnbT|0PXNyXW$pAGey$x?n!rCwBAfx|Dl)!Pv8cao)N;>Y^v=I^5FDk=uf?E zyYQIRZsF!{(>Hlay17sMH*sA))!#f`zb!sPC1cLB6}nhO9KYVLL*qL%Cs(Ui6BRuw zKYca6C5VP+;rfop^tHAQ5lp$sZ368mV&gjRU*saolx`Eu946HzDF>%Xb}?JeD&Q@& z2HvkN>iO9fk2YAF#hF1cCmK+tNX^xKLSgQ<%~>DtURfEZpHWW^0B6>2)`^19peT2E zVG66NJu8kmt{PE1v+{V$?o3mKr$SX6bMh|REUH1JHe2)dw%#;w6kKCrv>ReY;scE? zT{5nu;P8Q|P4`si?^-~JYVFRIlLcSPH3_(tTk^JW0bj|xB}PuSaj3NIgM`a9xcFw7ZY-KhbHwfSOMsxygdNdXidiWj9tT5@A5Oncs1ilxxM{%#0igx)a`ztf` z0^?XMq{@Od8%cX}zlx}~z3oq(3-?$0q?hz7Nfwr|1|2E4z5T*p;y=qVpr?KG!D-<8 zzwuQp0D03tMoYj<^Q)Rup+B>rndUA7_g{%l+k{qs>P{*$!1gGb&&)IMFW2KQ-F||+ zeG=?qaP~~6oN7;=v#WI497lk4jk{LH-P^w(ef}ij6*3?-r#`i2TP!sMHf8kjtS|Vn zw@(~%*YMZ!lFUeYFWfCyMb+!U*zmyamUz2z+KWfc(k@!*jxra$cFSq%Yi<&Ib;=Ylzx(P23_dzcBv$vf`&A*wni6vgX>rJBGc|(%YW?3LJ zED~arx&7Uo?=1Xb&t=Jm7}-FiyiH>v0E!$-G+P&pNxly$6)_^>AOBBeshS7}3d+BI zRjxHnzLF1D+7X#-)$)qy<;gX<%jaQZJO547H%w*xHc<^k&a>Vw(JP6IeK7M*q+ZME zS&GR<3Ac+A@YObY>K0L`LEO#c4Jlm|Ir6oiqH%Dl zn5oW&{c}ZN=b=u0rQJ#8#N?OwMiS8SAZXI~gO(m$a%pMQoRS{h%3mEywxZ*zi3G32U-}e-kW_4!H&XID&x^&&O ziVxTVZJisRVc&d2&FFU+iE3RhVV(ZsYN}ofyu&N!C4N)MSdcjrlDXw_y48pJJ#&@} z4hcGC-x>v(=@I9>GzS}f{+4L4*2mvzM{MYBiGFL@Q)T0x*v4pPw`7|UL2aKWqspK8 zMs%r5lKMHL_A()OA5?q{UJk$sOOsSq#Ju`TwV>`1zsTum6?FfoH0hY4IR2gGDOKz?_RUYaTHgC$MX}ltC6#*L z=zOX5#WKTus3_EuufZ_j6Z(Sx7-bCWj8I5H6~$aSpiYmmVS8_dQh_%1sS}61LCO zh(#(lGIbe@Q3&(?V5ig}+tgsQVgBYGRBGr_saQta)nOSk;5p%@yKd)ABXVr_i*#KY z&b(0Lt#|8uaY^{K{kXDd3Y$BT#?P}X-*&5*X3OKgQKeZXCsJMYzL;yM@?U1_j1%Fxj` z($nMCV3O&2daVskutO=o=;Bl;%R=4A*d6A``J5ekZT^X?Apm0kwb~%lY9te*vU77@ zJil^^p<@^+K99K5qEa%&0BJ^Nqsq{LP1feCMGWprRoiE}M5cFA4SQH0gj7#B`xzz) z?|!*Ge>PXkThGoAof>ynLW7`EDA65Rzt!@D8;D+|P163Bx~(7AJ~1Dnm%*D;zDm9r z$a=Q)gEr>e)Qzn$YQ-gx)G3xgyYe3n*70H|V}@PA;!Ds^*=l0PrbTV7w<;H{rfHKvOW7Z z!Th*#WLdF#N+y5Bp>-3k2Q_IBm}{_aWjNB^4EP#Y%KwSH*=4Eom67Cg@*HQ#xO8PF zmQ#dGqDdi2M8CLYU%IkCK(EYGBg$r>1kE4NFpOa`_2P_q4;rr7{5eb2jAmQCMIAQ1 zlNTrh_!xQKA3V&-5-i-#2vNI9Xcke4pSsUf?{&Feu(lYk4Oh0n|CFT%EJYuZeJOu* zLr0@^L#I;iY4WPqL3W&_YMxrOn3hrHt0QV?2AhvE)pTi_15|0U$h5A0n_zz017uj) zyhMkvuh*(<7LMVEhcxm9ZM2G^ldXjgSf~qBoqtYPxJfj+P}CRa`v;QBIwmX;%XO0+ zpv%}~G1#@JTRB?ln!uDQ+MJUp+N+9^t5)4E`aIzgmFh9VIi1|&Z89CxXz!%~EX-z0 z4Gd5_jyuox0ihb@cJ^6@uXFbT({dLse7(*qakq)I|IuUT(C2hX8UGn8&OSu1$E1q$ z_$KlLbkFr%+c~or!ft~DNQSh?$_^=`B~;tW*KP@q_sR&ahjnY}W8ROrv|`|d2ZD>lzxh)-bY(!k<~8VwMW0dxcMs>R zi{XafLGxT%NqNy+^q(cBh!pG80M|m_dofJm$?y;Q1=*k{zV)F;!LSnywwVS*>6q##0I0!{Ea>4L6DPXlay2KsSJD zs_q;&M2JRxCxrf^vaZ)O=5ls#QSLF9!5HisLM2Gy81s240T;NB$_H^`|1?YfIA9;M zhWd=iUs3@j1ae~k1Vh-41M9crhQ6?HPR;y=Ko~=XN$fp8X9)M?@Sz^wknD%o9-`E7emN#4 zaa*_R`V7UBs*BEHPF3ul?dmfZTP&!z8l+1YV$ZWZJ-vB0Fg48s8hEF3L-QJL=?^Y@ zj*uTL2*bhAc;-0~MHy?`K8o$MJa*Aq%Lvc^4>D6d1ToZ~D_APm_@3WcxQNN4CUm*y z&q3ABxEzDzj`v}1BiKWVNtV~NL5I9EdJIgOOh30;8K1~y0(WJfIxmO>tGLE_@AT5~pJ#nmh{aj#Q|AXUVY}8il}~RT zNB&usihGWaQJJe5UH&Mp<>O1nuw)?~-6pS|A?Lego(xap;6JO}MxAD%ud8H#rf`4J zboz1?cX}gm&@*Iv6Rx;J)wkf06OCGU-3XPZbAl^$E(G0|w@rYn|3yL~efZU!naqNP zJ5$Yx+9=FI!bSu0IY}wIXwt%^O4V%XS0T>hWlbQrgtVR<0o=14o)`s#lTT2M%RwK| ziGN{<8pwWvDqoiP(DSMc!FDld{ zpSu8q=l>gJ|A;L>UL|MbGYs(iDxjV14@GmTMcDwsV%6%9+2Ki0 z@HzSP|CTi?X!GbdPpm1?KXlWU?o*}gbuyQx=Wjugwkuvd=7oheG5 zM9I#BF_jz68=!8Lg@kSob4G7(9!Xaaqjdv*~1OH9Lp|(cS_F>}2=0 z>nU2jm)*a;Kr&!)F}Cy3O(LW6_}xc!i7vh8R?9qh`_dv?%HQKvT8_pg)#-rBopZ7z z)Ey7-UQ-A~W;ov4B}?#wypsS^nvsR{b7k9AP^Foq;{sf0HN@#5P3bp7>eqFb9cIW7l%*`{CG!e*~vAVivn9cQo!KOnI*j!(x17}cVO9H5Cv2O_yzqjbtCNu2ScLZ-7 zHWh}4Vn-uGu*2JO)Wpl=NF_9F9}vIBeb6i?@zYn^)+eJmeKNX5!3r-yyDZ%?V2N*m zH>`u-J(In3mng4yPmM*>&+lZ{#yTil+88CXuB3lv0*x)6Hd^22Qxj#$m%15GxFY$T z;6PNiZCqcUgVBV4-PxINFGSDu&Fnp#t|$}WIROuk>lqon84^Ky`(dsH0VSPn+B3H7 zKT+P6-RMvM3#qeCwB=SW?pE-5k|d^N@VCe~%O2 ze50@c68SW);@vpQB#3sT*(_$ySrcy^udB+H)69EQ&Y8^K1~ETMgxbZ8@u}t4{s_;| zi5(mNUP@l=7b)Wd9Zm(dN#bxC8|}WOn1r=xg?wq29QVi`vT1JRyh+VH|EAkrbHfxt zN(>HC6z1&1<8vlGj=bm#h)g6Uj$DUa_;7+jrb-Mb7miK;KM?cRTok^Z|Or0qcs~Jcqo)&C@l+E zX`HOxbcZQkj>DIFn5{O-BnmxcOJ}>Scixs>R-RQhxC%3 zZ_-8A%g14`%L?$5J9R}n=6eqcdxClnIs-LhV+!5-aQtzR8LehJ67Gy*6&&=_9V&~b z3_0$e)B2^^{-4i6a>jd`G0l9;tP7ztB#EQh8ZB#Ldv;`S?L*;>xo6+QyMfew2^}@| zl`$+iarh9g9*5G&bO<>+8S}AUFtg?EpktU^!j43?BoTA>4 zUC|QnBXDs2ApnsPXmUR8Z#YjMu6>I@oP@x~gg2<9uH$CZqEsYtKccThS8dh=WQj*& zILC?Q-&6ZPI^hSOFaN@oGxZ^`x}Tk~bnzF- zZJ`*^X28^9qKvSqHmk{4kwdK}iZ&l!@JqRj37JmEP!)@Khjm)0pew6m*mf16XQ$8% zrJp(I#zc_=UR7D2K-+y2(qCB=;(Qt%kHrF6zQL&M-|dW}9~-wfT=X~kXOb2HiRVwX zQEyVe_ELVfhNtQ@AC|gXo&#)ghKmDE@2`wptiFd5&sn1oeOlmWRyKwO^E8iugmqi) zy6G05yf_UhYq5?a%=tLk^7MR#>O9YL9c{3OdU6Gz1b>&vt;~{^@BS!ZonG16)9Mq_ zQ_CYdT;DEF-bi`(sLOJRcetTh-jrU^0fT+OAF9O%I`#p{hGqp*djEUXafUC%P-%3L zz1Jg;qc;7`+oqdjPw_tQq0bfTWKSJkIcN6L@KKU4Gh!xpx5nugZeXW@u^WLytS=N@ znc`g^;#Edoj2EC@Zx)>2lIzgi&$O*@#cT!tH|~K{ffA?p81GeJM-+Ug(ZYSx?(CX^ z?(8DJ#ktFMpO~m<%b2MAfFo>Nn%IzjDtnH)Q_+=Hy~9qD@Own*RuKNuHp-=tL;Bsi z(UGB$QQ>7_2nI>ABuAFev4X_E#8;zZ97))AkXOMqeeW#baYFG8GTitw@;85LEstfT z-(e6>h2P=QN@Z!oyl3+GmNr&ac<$7vtwLl^)J+|`*L-~BR((!yQPXXxh@%2g63I#^ z`3$B>EfL=?FgW{Wvg>UA!l)O+_>uDd=G-MPR?g_056|;TmKl%V<|^n&V`P-tyJ}^#V`%AipPMF{CrgMBK}ATR~=*Bwzf;0iJewo2mb2YtJ7*M z^Jm5jX;MSowpdDM1IHKm7`CA>zu6fA>q+qd9)ZA&X4?u6s9Ku8-j;jW0Dl_u1Xbr1 zNeUH7oA0~8ljBmyd0+I0U8{wGqwUiUp}kY{)T5xPf&S?dyz!TuY6T8z*qW75NonnJ z3~Ha6gz;P<1D!5O`9EBn78Or<~m*7Y0^jL#a=o{srQDpVobW!KCAVu zTtIFAxJs#gj7~Mp^}KTKirxLsd>WkPza2^%GPpF>)x9TWHn)VatB{M{q zWj&gg&cK5Q4--0pMik{WAS)8N3IqpzKK!NrI!E**+;Pn<>n_HF=(~2}vm-pQiMuAh zDo@85xw)tZrE@&xp4VtTrSEc9T4=|8!^VYg(&Rg9FMNT=h~V>%e$s8C?y+xBDf0l) zoS|3r1aY0XjM}1C#%D|a3r_KDcRkbA@UYA02^%;@maaG!kDjy0^$iCy^_Hb*(fTFb z=x){0a5!&lhJ{#}7_qyV#r)9Ci$><9Qu#7u^Ugo&fIbDDB1A}^J-*xLespHKel*rW z+&Yu0Z$|J?S&n$kzUk#3@DCDt+aUXxKWOnZ-akxl2(yEXiC65|Ca9r?moPtI6qijJ z2OHpL66S;-F|q)IGD{ulv0PEO1Y1-&b8unr83#`<@Zi&=?f_R(dRm+x0C}7Wq(>0l zWYag4Eed3A=r*xFt()ikAi*vWx%G%x4O+P%xbwC}=vNQ!6Stnl%X$S>+WEii@XI6| z#}1dtTT(1v!r(Uin0gphw4C#w7r{5Y4nSPpU>e_u?s{#T*sN#{sgv4H+; z3K~R{u%gB+o%362yq?iFPv;ChyE>~Q+m&wb>6=zk!`NoK9T|8U`aR~Aw+R>i{p*B= zC;dZqnRQ$_s~-!c@CZ-;^zYT@Tq@3AKE|;%z2cI1{zW-{;p3~XYzD)>CmpXC*bIIo zn(?vEpCxhSvoU}D$g83(^_)({8T#=NTT>L5gxiQpUJ@)EO6 zamX3i{eNSMkt9FL7*`HT#^tN3&OfA#$S)h^bVNR7m!F0J{^f|A{PYUoUyiWltf1NY zV3P8EJ5uo@ZvOErfPW&32PqXYLh7%+3?Rs%LuaUMDXye7Hj_Zw2=5VuMN#S^ACrUGa${&AcvhNUK++P%(TXrm<>>V;@Wm%|Dg+M zIj}V5x&0qp0(ksmIXYX9i^Jk~h{wIvv|A3w@1}mU`SCE(p~+vfrB`b{So`w^j)QU6 zuGWZZRw&yjN3tx%bZMn4?bg0V!Qa72&q#5 zI^!_jQUsr~_~#nlW-0kVDn9S9J%9_Bv?0dgI*?JBznm$4w_MM_as>@ ziV@ORbxyct9LnIGg$Ud*rDKGYpB@E4qhn%9;fQWLLTbC(i!CO3HKTP!|2Kp3cRt%g zBcmB3P|rDe7s00p$;-c2sOE;j$AV`Xm4U+?{v>>2$M8J<6Z{uK3p_Ugb`ubS{tgam zq3KXHr8x@XxWla)E?j*?AFn ze2DvH9)m-6sgnLAOFhZ!Es*YO7BAwy4^O|0COB&MPqM;yg_6ATfaG6IcoFw}82eiu zga0H;6CAheoaB`cWc|9xi%|BF=x_OREb4y0$KbGCjwDnrkoc<{FXFDxpJOdKBZwLE z(>0tLP+>8XV@+8+()D}y`#k*=8u780s!t}2-@Sq<8u-oH+E44aVII@bGq7vX(#k1Y zy9lNhIy72840JuqUOQ{8=E;4k%HmiYDrKu%Ix@0vQgnf->WH*Fljs-L+j4f97EwkUU@nG2Yd z*5XINbDUK$QS{@5yHPT{Zp7U1+-~-y#xPp1$51NO|D5@*J1eO%k=9EKs;;{5AWG(j z8xJ@9KV@n{g;W=mqFQ*}7`fqZyG@cBV`#npGbVOW5wl(X>@2u|(c2M;+rXy3VaW;B zXeM*;O?pL}_&W_Ue!^LzNoymA}4cV8*_@tf|VW;fI$E49an} zS#LWdC>c>rm)#byF_IARsLGb>JCyOWF56G>eCkH~ouB-u3}S4PYPN4U-0*X0{9S?` z+-}EdlQqw483h!CqC6_cT<{ky9c;-Wd~~N0TFap$Qm5bJATKcy-bs6$fC8vny+&hU zccNoXXd#5ye^U{qw!>9F@;Igj&b^a)03P)@95hPMqK%8GUHvin!)ntbU4`{qM9w#t z*2$klp#f}*f12JbPOHTKH9XNh=N;zSBG0<~1Jr|Njj@1#v;H|F;d^I{@HNLT`@!#s zugU$ie*OXV&;MrqvqmDww5g3pzWB|`p6Vte^s+wj3OoVtYx+N-1^^PccAR4C3WR;X~bzWX* zZwdhr!v4Vw=t6f>6o3wPefUvkhNOg>^FNj`JBHa^%taKQ3I*L*3pfg*KHfQzxqF(1 zBgJ0B(qr9m_M;=CD(jX<1-S{BZMs_Y=CMJLiNa>(rVZ8uTL!AgIha1Eu@G9KJa)$= z=GL94j$Y3ypoz;8%mds$*z@r);%M08BnpU-XO-s~ImVnA=vi2sgun4%u&&oW*@ipX zMP077QR~rZn!Sla5Vt}p^wy1hgx5yaM)pqG_Zj!M{Ut?Z_Bx;xqCIB1Bf}>&*qd0A z)|~7-&I0PuLbCcRY8^@+wes))^>y-nrDNKFH-TZ<`LK0@fN-HP?&gPjYkVg8oBJ6T zw|^*%9qB{M)a)%F<2_V{`BF`9tIn%*N2 zQI_|Ka)!Y zzux}2o@+M#nasZ{zYc=sY`m3B%C>&}zbk(kgjd`5>*;3IpUM2Y^6MbLR6&U#hT28~ zNf_C(@y}>jD9+h?%6Oe{AGH?XL%)xb!x0`w?knzh9r@i=xTlaEB)P^pu8@~AICA0J zK6U5&KIJ30wi1ZB5O51YgcZQDVF|GF?G{2nq3FKTUlf@4*G~|kQ?`l2V=sG$Q%0rI zkB@-Of(i3STz7NwMk20_gikBE;b3xsA#Fc1?t!SA@2Bn|E}DEE;WODwJODDd%bfb+ z_k!)*mLuU%(@&=&YdXieV2WeWW5BV3BJFCYL!tKxNH$PrBouecjLbI<2f&Tzf`(P@ zYV6J54N63e42J6&to0>#Na+#5jF~e5R<6T8Hk?;Gp4y92e~w(|+7~&N3<#Bfki++0 zVNWUH^mSlV;2RtRmJG{>bvX7?;;_bL;|Q@X`)_0^j~!pdAvc8fk4_TrnteRIz7}&7 zM0TtdD84ojh>3gZ37IrmI|8z~cdY2`3KN^`dq)Yu%*dNxD{#0^YE12dNSfhpZauC- zz~&fH<9+o1Y41FMn(EfQ4jcnY-gqIcfjwLlg~hNz*s-yToVnt8ts85L%Xa67l`R1*)n~;AQVKldE`OkaH<~imQE4eL9Rj11)h#(n8~kO1GX5}Ul|rahj0uv zf!PNaDNBS6pQowx(U}&)#UqmiYIJgKNdH|>y2=#YWFcHS@{2%?Zmtulli~A!57a49 zAk(=Adpr9*n9WnEjkd;jRH7ifeJgD!;H1Jf%d5XS(Qi}?Cb#+sqOg&QU_)hCXLF+cV1QLWh{8@Pf*sXto#oSyO7!ax6O~;hgf!2mGP0q<5ZyT32-{*UDErkrh|-0k2i{!zeGK4s2c(aV9{ z*vD|0{Wm#El*7RVXx>1+48*(pd#C<9XjdVmq|z0CYRlY;h=_T+-w!t(8OW} zuKszLd0NOK_)TP95W_+B$87bBo9ng)N^z%;e!%`YvdyAInPvzKla>v!lnoO(4dr3g zFGSaa4X|;ipdZ*gObWW~1GZpDqst>jiwfS^cU4ro`Q~0#_pihNxJ6b;Lz_+5t)_eM z>aRrqptwdKo3KaCZxLm#HJyW`8bWNpN0hLYx!3d#3Tpfw!J^^7R_0dIGx&Q1bWl!X zgRRV?=C_DI30Z$}B=hNG3@V2zGB~M~*}Pd6H2~K?_QRb>)t+CDon$?7aSl9V$?!d4 z%X~j$#z~k2$aR$r3hqb~l|>QSw*DEa+qNS688c7vBtV|4aZvDJny4J=pFjYL$F_C3 zKPBVjwFJn0RT~Qao+kPyc)m5Hua$k})bLbqO^J9$=prTOte}D@Gr5%w{Q;ep)0VgM z4yc6MN^@J&w{0~%E!sVnFW^kUC?L~ucBIZ8gdlwWg5!S7Y(B0c`nKfnPa==0=tRFn*5=;%z2*yWm_P52isw zelGoa6$JvtkriRBrl&KwD1g%K;3N$2;%|!!%(=EQ48Dk!3Xu9N9E7=x>B`9_uZT41 zLKLbfu`{)@0A=V;`2+j+e#Sy13rRBbQHke%K}=}I8eg9y`?Zt>dR)mOyiIzx-XE?$ z!#rZkQJ4gWAeYc2r`#7eF0@a1PHj$2&_d`Ebj8{jY0`DR$gG0uAdbOQUeEGf4)^=d z@3a~Cl4SRhD(W*Lg*;!04B9lRn5?l_6jcoVJw9Oo=RCD6;vDRIa-S3Dei+QQtAu?3 zI$Q5@s8%Y7ab8FbzF6I3lPx83R_ZpliFh6{> z|1#3Oa9IyF#HOFl{lFGtWDeRsVgF?$e1V&vXk*Pz=Z3J!m^yS@0rp=;V_HX%!I>a3 z5lkRlqUN|fc?{<;iBS!>?!*PQTrt%-Ny|sX7Y{KdIVC49VVE~ub62wWEy;H9Uu0vs z9ka$dT7VmmJ^u=T-d$T5?kbAbr^@C4DuVrw{~EBjwloYWO48@d z1_A@Yr^o2O0+8=A&r7u1hQRQ$@qq+jqT{~?>@hD&blb+jsI#vFRl!lmpgC`LdRWQzRv7rso!?|z9x)Aj$oCRM4J3*qY8nVd$ z@VwK|IdXrVV4J`?#l01K@XIY&NIuJdXtn{t|Q-tac7 zn`)dH-SS2r`omk!TXN1qcfY^}rxrlTuFcRouhue@Q>M!X*bFfxH#d6n<(G&(a|a35 zK*-C@4W1-IeLP!f<&-E*e^yv8cpmCc=KTK}jhvK2eY{#rP!^dx>tGW^Q!1`v{Ci7o z?oT`X0?}gw4_;tf{dqhCsL`)>MKSbPsr?sN7Qe5-3C<%+3`G9>Ghp0!BP4{I`u^ zE$a;?ibur=%Y*5^!INhQht`Rc_b%S8tLd2)|4OsbbZn%teAKJ9yu{0*Y{TBdWaC#T zca);m`Ja6i*sq@*}lB;oc>PVSomyqFwY-Tc9=3JrEUF^e{bgQ z_M#-xd;FtMXHyuK6)hMj)ZEoZJ4;V@>Yd{TnK|3355Thf?BEbdgy2Hv?Oe`6&fKG6 z33NUh|I}zuf0s#)go8{`JFbM)$9$%KX}u?NSMQo9{VWOHB&FPnbMMUCCIKJA?ZRo@ zhIDtC{}G_9m!IDbX(HJEjbeHGWs~YQWrO_h6V9$jB{@4}I~hAfK9RF2Fv1q4X2G8t z$RyXsI-L;Cu)R&>;}Z^Kn5Cbl;J~d6v@m+`5_H)v^xS#(A#y6H2|<3AB}GxXO_Y@T z07s(NG>{i)@(cdJ5TLOqy+Tw_7zDR??_9Fr*xC1$@g+CLwWpMmd=(b=Vjx@`zg46{ ztT~eW6jt|srNHUcrfvi$lv|H)$KmavZ{0Gm;>;Q3~-cLv^N3B<#DY$#z{ zu$u9H!Yjzrwocrzy5{V7#&0(B1`Q3({vxO99kqz~7^c0e;@sR;-7!pJKU1UJiHVPi ziDCMg!G(+a4*4q@aZ44eZuuTmj9MNNBrs@`P#RUDIcC)T($^@1dM74{n>1EeoIh4; z=(UCBqkVoU&8;DmA@w!8vOH&r}WlIs94+B5PGArJPpOH`NXJMi&}MW~#Q+%Rkhu0WLCvOhE8C zrg);!my*lrHUq;06RYxEPwC=0&3*{SCf6pb^d)?M{{GfXjlC(KX&ju~D5Tga$EjI( z?m<-TXJWlhz%d<$(qzECdYDa&uU~uB300tOkDm?_HEd>EQ^Eb=YnDQmgb z>eqW3mu8}8^!3I8+X~*ZNA>=90>l*|B9#u~QX8i3P7V9z4t0ZDu2fAjGq&3=G`2bv zm$DwglqdlhuL#MK(Rvq*d_5EK~<+1b-jx%2!=OnZz;H{D+H zU7EgbKCAR^v>M%jy@X@s^BY-r2|sZOe+WmvVFT#&d{f^S^3$Ast@K@xr{Kri*eXK~ z(t9-a@h=QX>=R9MfcG|jP##3e6_Wm-Fup|H9>RF*F|}HU!!FS|+0bJ`$B>)j1bp+C z=#-fi;8v!ZtZQ^FI>C8$YYKsC*%ID8U3k2m@nV40mjLVXH2K2LbINZ7leeWmGs=>f zUw#qlJ|7zSF#e#5T>e|(1ig3oKvlpdNk9j~g!mct({R;};IO5RwRy0sSUpSsWhD<@ zDAZ53z093?TcQpo3(mcT{#%9dh$OODixXVyLvk35!3B7lucS=ZSY2m+_SwPZG}um(cY! z?m>Wu@0+n@qVF=@L!X_%e&J!26HE3%v4$X%TTOeb_deUhXcf14BV?yHJD0NFz)gTQ zW%BYZi7uN5$buBS3v$JS>g7o@s;#oRON#Utm$w33qRh*m8XHWh8W@oYbK0(6oVCma zEQ|L9N%BvGoV2+t%=Hm`q42Q`3U{iSBRrvGt(+r15jmTk*Lykl6cSFXJR5S5P2~3` zfe3WjFRn%A*z0B8M1F@$c(cbCC5CA^V6!D5oA5;VFg!u_n@hEeOc3+(*=BUQQs`YG zpStUC7wNVu`ynmW2SwM~I<{RdgunTWrCc87w#;*a_|wckrta84jgzRT(hFm!tMXJ0O5?f$ zXZdm8s3EQDmn6Z(+K}v6mHnAmSAUsBTKc}B&IcB}L#`>Bn6;A#rHZF(`q2c$=Mr|M zLp+=sOVHeegyM{Z(q1m7p)Q0z-`QhYovt=PP;U)167b?$s9{Wy`=k>^*Q@}q&*}Qg z-mWZ*y2jx5@A+;%G101J4lY-e0nCX&wnU)pN=v`Ty>#Vo$@kk9Nf_&^%w28OZje1$ zHHVcGN;Nt!i62MoKDt4QDW>5abn7wL;M#h-3j1M>5P>dRt;~|P5A%Ckd{2wv!F~D_ z?X0^cLs#wb(+sb#=Y_n#-N+>_(B^%$82$bt`fd^|vNDfZk1$zcT->B#Q3ysx@?_V- z#hOu9rLp(fy*3YGY5_Zd8XFmxtrz8>E@2LShbaa6lVrr=+xCQqDb?m*@w|G4@@otz z-*S4a)DL)O@=a?!+O4o(3Gu-TvJ?7j+GsKR^mg^+>Dxj%{6y8)U{m?om(TJX-^CU9 zkR}@%zhEQ#m|-y%MOyT-+?a9b&N+QMEuzzuzU!nHsm1zH!;VV!*jw<r=>{N+B()jG&o=5 z=GSy0XO7zG&VOQ)8+B()Ek{i**IoO3=Y9>))?M^ir=T2cjXwR2un%X9@!;#zc0DTA#GOv&QC+oE94E;kgLoGTSZRe9QUV>V(Pb zVt04ua=*UAE3Mnu%U`nEBcRF$`y$b5ENKYei|)rGV!bFv+vZ+R0d_!_6?KnfiLB=( zMzW7lw>hA<$vhGyKA?0}nG-h_PicH-3>Qozv7xrGgY-?3;ZiH@XQ@^KeW}(NTa~F> zT?bbzANA}q6RlkyEp$!p`ot4}{~kbF`jn(p=i$JrtL+5gbRhZM3(fHk2Oa6bF9e{Z ziR)mNQ?;j`)*ro?uX#q8(SC&m?w#{}Li_Wxg}Kj!6Dykn8A1ZHwmD|)g0@Z6(IHY4 z_tgw?eMk_F@Z#@elM7FoYk@Wh*VokuYLcw2Da$%nJfZAYUn0G9Z6Wh^=kLjdk=Kjq zOCQOGvjEJcN((<@*_w5(X>%4OO@iphzk&^87~I4{g6RZZSw3fncp5wSYVOpOObY}M z20&aFQytZxqAmlI9%GAng1!%}l%CP-LHnr7IfssjYcQwp5gk)Ea~uPV+Wl3V4z24kEP9N&`d}}O7Ji;H z7HBVAdrlYoL`D%s{)cIw5J3f zz#Mj((M9JwzSE3!ZDG^yRZY-A7D=HOT$JQ$4ynUohf^LOH?9b%()QsId6(tWopjsr zJ!iBwEBDgYqb*(0QLAo00FQ5zMETKOPiD~7K7Cy{IbsRx$SzdO2t>7mG$;5%sf6@{ zZqx|fP*|Ew8=}4{1B>>)aZGwwrF2FE^FeZ0A<3=hBvosjN{@e)&WHBizQui2y+X!H z652u-g|*6*=qt{Us!^NArBZpg?sW&#p{G@iK8Vl)2 zRyW2fpnEN|sN&jpG&hW+dx94MU*^&)-wTGaqp8I^U}tm=dis?&jj9z5?>lkZ@HEvu zQ1Y7mM!&F*DBIFK7I%e4Z66I^_JU;2VbV3Q6U(;1*61tmL0M;f${U)#(a`X*bWgvJ zcU8w%XD;}RuFq`gR}Sbg7`yX10aws6RS!8Re<7JOiBIVV)56{>E&8BRvTo>NA2EiztDS$yzp zc61-{yk4R4j;g+*smY?!b)~pW)}CGdoA+?CZfI3*fej3IrxAG7(d^rnquKPXA8*ta z8Ni7G-vl!`f-IrHX&fqcsugw;Q&i>k7))}j`v}b%y>xMc#O|@ zgv>;I-JZzNGqR&6LQ}5zm&PwySCUBgu+($ZJz-8>e4r2!?MAizwYHwpA&)rQO20Jm z>%%aX{spy&TciPxMcLw2Z?QlY@iRhb4JZic`5|{5V+~!!^}aez4+PTSlDcu&s&>eh zM%GqRj(+tY$78L`WY6f$4lCxWI158veA6|+*$Edex&*i zfbhVFtpbk25Q}dDNvgMyN(DMB9+w$R=5~Rw*`h$kdv1CRG$gMRmj=bU!-W)Qu%Xyc?@M9Am0H{mo?)9-hNa9I) zf}h(Rag|m}hhOZcVAL|-hwucOGr7$SynLk*n+Q=2)a=t^I17+ijlZ4E*1b=4L-6^+ z_(watDamSPuJo$cT0Qc$X}T%4?h~Gt0`+8}VO`|tkHv-$M+6Ct@O7`<#Jz9W*I=)E zT?Y6?X2vPL7AjsOBV3on3g+%=JwrTteTN&QcF0x5>DFkHuM%JET(lrK!&*|(Tv_Pd zA7W?LU^LXvpfjaIcvW4V5jiNym&s60caN~<)w9;e2dsn=JUY6@U7_(dti2&=GGtLr zg`ZyC=6xsh?E}yfw6m+`uFF04pNd8QoUcy=zlOcjOvB6+lRXH47#0_euX;)t$#1Ht zm(e8YNOx~oF z*u@x){$R7`NIY_n$k&Q{b(29S*fhVU(Xe1-qu*kLt3z7JAwAv8(+nh*F1=|L+{4fC z);np)QYBXH9i@-dk4h~~-2|aGQCw2Blh$fd1;g^nMWB-5GUHi0vk5d3`cBZhz{$Gt zlcyg~W0ZFMrq(0q$TzbS`0`b!#+mjP+fN87+F}+O-rXDo05+4?r0h>S3X47g=%|q- zkW7cG<;8UORM%cht{HEUraUsF?~~78v8`#|H^eS!>wJHEdcWkU;;ZU{R<&?|l0Fp0 zQ}_|rawWsVwAzOiq=_P6bc?rrJL#@lMw1gU9b$gZdY@fteKa|13Hcse30SS;tN&y}&ZqC1~me9_~Tmi^C+}+%llQ6uc^2qvH zis*PlC=hqm!u?tT-wf;B*$&qm{8n6C)}Upgo=i4!HuCR;tjyyX(weu#pNBG~HvpIb z*KQg}--?^DW0W8u;3D9AIj~R@*MZ;pOwVoNW9K*xQ)Ndk*QI_#f<13H$y>LoWIef~ nLkPJ-A4`Z5b0v*Z_F&6cANtFPi`6?m)pf(id(Y1E2Jim?!Lk1a literal 0 HcmV?d00001 diff --git a/docs/static/fonts/LatoLatin-Regular.woff2 b/docs/static/fonts/LatoLatin-Regular.woff2 new file mode 100644 index 0000000000000000000000000000000000000000..a4d084bfb7a5f329b56cf141d14aa5ed99fcabfd GIT binary patch literal 43760 zcmZ^}1CV50w=G((>auOywr$(CZQHhOblJAiW!rY${(i(e_r*OkGGb?DWbDkGbL@e& zmz$g@BLE=4->SR?K=^wA66XH3eSxI>Km_=V3r&~ zM|*b3*pcfa9{&i#mNV@sSO^f*kX_%Pm?if2bpW%{0uV@WCK%kmJOD zS=g$P<;!sFKP_5de+b0lDrA7C7$U&jm|-;d^3Tw1?DZg-aA#EEP!rt}PZA@f3>T2g z(ncvNNbEVTu@E=mUY@`c1##)K;4HD&#I%yH!j5gUV{IW=+#x&K@&Ru(i^ z`=s?70cKG*8w_K#iMR$dHH;uU#N-4ee5u~dMc1dXu4E@qdkXP)Zp#wkD7egUzT}{t zUr7u42q-Bz#aNmUi`@;gk3Rigqi`)#V-bS2HBd*|H-40QW#nYV&EaEY>^}DTU}W$e{ibM_vTroM zso>q8f!OMC3f2L&WNz`;=^;K~!|mg=cQP}uu@tkn)x$j(i?ewCi$EP3${zWkKxVITfOO@XRex`+VA`;zN;UTAG?pA^e)`bx;^0A-Z3r0tH57hd?ho~ zI;lcjKricQlSfPId3zzBW><#Ic|ojL2B)Nx%2|B^r=Jwb^SA^P2At@=i390N>y|BB zhxrv!KsQ96hXw0(qkM}@JWZW+eU#MzD$2wSAjkI+(a}&`ga$?8zf^m_uM~97_#^Jc zm430|U$p@%;IikhVSmIbi9e~fxRFd1DR` zE{&|dbl#YKmH+7M^gi`^edl$Uy>xBCDIl4LM-20Q*71D!9K%#r@rkdK2UuzjOBM@b zj$@Iuu}Q?gIRfvBmr4HG32~>K6CYeKrtI(4RrMd6pm|=`Ly;&2p!5eN9-P-SQF{QO zoxfxlWEzKn5oe_p6sf@DCTpaAGJ-)oDJJGXD zX}=c`fnl#$WTNDsj$gUqrYJ6GE|DaMF&u}=?W0YA>HJfzE+{y}`deZy*b9CQwG=7H z919NcS-EWg$6UsUq^!JBStlOW{ViPa0Jw527W=yjI&1`)#(2ev1+S5+$`%guF_O}J zq^^?x2SFMddLUqv`IdQC=Z;@fkR}97s^DrU8U!apY5cgFm;E8_7rlV& z8T;%}BA)U+50$?^2*fL$|BqgluS9^wvXbSkq^se<&cY5Kj+>m4W+KH`I_HO6ScN#e z;n=A7XU&+SZ9T~w9)LZED(YB`J>M_B^}x8>5Epo#u?G28C;SS18L<#%7+=kS#i`1I`HP&u;ZD1x1GcBJBXL|9~TGtXb3 zu&9QlH6?5;*NLFeS4mVcRjB}qT%*MI@=V(U?k79y8k$-*_d&k?|`YZ)&HtuPYZpu-Zx$)rasKak>Jlv_UxW-5tF8M9E$w1FR50V_nnj%nm zDh}!9j)CA70$vQ_gbTFyL1hL$0M)2ytQKE@y6>ZD%~=_GP}S~p$5#Qe8h-L|{l5rc zWZ4}s^cP?~AxiCq!4uCPe9C?_T}zG4K9L7o7aLT!Nf=Vx<@P|j!ao!0>l$ud4A#GYj|IGb-!oX6g^}juo{uwsVbycBiWSJj~spp{H|{kmYxEUp(LYYVqMcIjvb)vfuyFMAf(ym zRB)JAu&^yXCN;xy>*WE_5D_1QS&EO*4|F4d#9E0M(ZIf zN!eP_$xAe_a+wJJ;R857Kq3^7OciKLxJQws1)@NcSL2UgQYdz&y+3L%eO1KPTh);x zB4=Bd6%=1JF8hUPij=Pm})}hGs$T>#YxQl^VP_>E{24GSq>J3}@j| zX`D-`h}xo=diK|!yd~FZ8~{4_qet1));+cK7%7C!6~+)aGiWLAIkx~C!u~8b{@IC3 zjuC=r-w-;-S-9jdOeElfte|+@8xvH50R8mq&CU&@4};PnKZJ~M$y2%H)|9aLt!67)tB}=% z%0mOb-9dBwHwEjFK&gELfEq|RRlg0DKSxU*zR*@AFF20Ah01reM`wIiAf43Z7fZ|K zQ9K$42$y~Xy_eys)G05IYii+<_5)5GSC=*SPM$aXBy_u>Qv_oV78DTl@`=w-kNNZg zH#kdWn2}KL?YDW!YcxRw{rJ5+QM7n^Q9rUW(EvMZ*th!Hc>}Ww!Sc^*pphGF`1F(~xcm(hHy)1jFwW zc(?r6fZ!h^a=3c~D0k?uSU+QF<|(M(yaa1oyp@E++Yk~C8o}2oe_QmvI+)(8K+~b4 zAq<)^K&d9-4;)l~?n~TWFpK3mlSea%LKsLR2zo-D+58d#o2WqdSYu3GAV^OnFee`d zF$hH1Z|`SqG7KaME(hrDagE#gEx_7jbhRGA#1JXc0P+hrtb3#YZcYn7U;`0E&3f1x z%Mdfp{xe_!fW6Pb=k_nHcmdz{A5R(M^Jrovfi%dWn3M@pa(lz3FbZX_q^}fL2N#Di zDFMDsN4O*^GnUH%AO6up@+UgwzVY4!PUGP8j5Lu544E?8a4hj&D|VE0x`4wh@qvmN zi{94vEzm=ryd>P}VSDc9HR>p-k4}A|1c93dVBu;sF}}`r3kiD_`Z18%Op67JQ)@i% z{cJ)qE+43y)H~cc={^1F`;i0eGJgYX>?4CuqqqmSCyOWw))1&!VG@86kdkM2Mto;@ zTn1SqjfOVrf9_AX<#u0Wk$UD zmHVq_`wL+L;0K+>3}GzKkdJNuOBLc;fUSUR-faG+C{Lf&2tQpp8b~SntRL2Jr>}jt z`&N2bMsKO+6}P!)h%Fp$z@A$y?~!y_WTZ-0wje820v+hEfUSHP9!|OGlI9xQ$*rGo z32_|~WMbAcu~Ts#XYf~Ip_UL<01etKQ@CQ|v51Wy_JyoolPyseQLDAy@ekF}vw7mF zVd9|!%1qsELLnjzAvKjO2%goTRgxWbTfh9B_k^;_PbrO$mC;l7uV@*ZlT{mg1^R3a zM>bEpa~ei?$z8D>{{N{?0YxKq5yM^PX2vY z@gQ_mdGn}9yQ7fAV}!)s>(-;_s=>k`?CAdR(fR1I@yOHl*mU8E)#aJC>8V$3x|2(u znnwIeR99WGwYk1vz@efeq{NQN)G4Sw{`1qQD@#|Zupx%G@QTRLWNq9%{ORRt-rftA z1H~(fHl^)YMxc-qghC>K@PP@ef3h5c#|@qSehOgNKCt+7N;aLzj$o}fGy!s`^$HgY zK;-&Zh66~WQDa%s4XdRmLH57Rt{g(5EwoXAsL%QzC@&z zHJzF@Z1Mmqn6N>ZGKnggvW2w9t&9y_##Cx0NvJ(=D`E5N2}b}j1e`~YA#dv!$d6n_3@95HelsQ=%ls`%u0;>%m}X4w&{%coHg}XD zauOsuGM69Qab*!0Xa#Q60s94=P5nmbm-Bgal@@(j`@+!ZWlMHFm90x0F zHlc$d!rdB_YD~uzLcEakLia971?8wR)tV(K*&_PPlwy+RGYEL{5ZYi84F#4hdYUYV z(JiX_2_@9+9_vjbq>0rIDEHE7r~4S*9o!w`;oS!FY;u8Wp1e!(4Vo4-gg`@f0ma9` z(5hKJ1u-Ee8XX$6I6PsB3~vRdfSgPo?+h|Q=%9aq&{$|$YI=;Cs=C7ZJKXpE9>yPj zs$RQNxn}SwhM?O2WF4`4;&OOB8pnUuK{R^t|;O0}pdfhUL%}(R2)h4amyP?!$Vx7aC5K5U(@urfZ3|nf;jcz4idT~{HsuxxuwbIn$f~!I zIMH%pE56y$`NorAhNAlfEJWZ+!iuJe@=08nLmOxDN*1BYgZaEnJqlGwT!uYn+`yrY ztLOi(NkJT_&2~B9Q}MlcKiC9mh&Ti{NqXDyR}t3OEafX=`hakeki_t@Z4B#T_SwF{dX{WJgTp?!e+4QKr%sqSge;M)k??Z< z_drWmSv!eNCf!CCJydUV-QUFQJ~GEkHx0gdwH?7d%OItb_|CnLdsVgl5Nl@nM5dY) zA3q(bV&B6zlxSttT7V1}hD^hyHZW^kM6m`~uS8KDuT%X_$iMqiYA>R4I6KU54Nt|% zah;<5MPs7xVzaX6I+=ZrTMoGLfa^*2043+37Q=!*pd@HKcsyew5#@w=Q05<}Fw>rz z)E8`vgv|}h_pBH$AP$b3?r;NBO)s{V_L8w7JAC2Z!(mvno=pk9@@s@X#S}*vLLGWO zC72m7fuz$V6;rfwJ^(kgX zvfH}i0#RZrhnidTjWJe%i-mFDQ2LI zwhP2W#f9-a&7FAGjCR z@88eaur+ho>DNfb zxjpaC&aODYRZC3%lp&mUhcX}+saLX*xya}Mg7p958|FdBE_}(qjM8DC`TM~0vZ(WE zl^U~U%49fvG&Fe_)>L;x&`3UvNeZNYPIO$RQYFh3hf^lw_$Vqt&j)z_hJ`&S^D1A? z5rQ#Q7ll*?Q<~qpvQ3DX>L|a#AQa)vFcdQGE_#skk$P6a$J$>B23V84aieNa_>SCR z!3*rwmO~(kg}OX_UO3v>bZ|=pKEx?oK*^M7wttcCM)ufyf*{mDZpE6H5;Z9nD^p$v z++49-hzTqWZ5k!DFdbi0G0Q|VA{V5@-3BwcUrnb02pqY_5Meoga>%=pT73FmsXu?(nqaGbdVrS4%0>A$IKRlu!p)%LMGT-SOP+n6$4IF7wamb z;*p1vNF|PN=u;d!rVI6N_wdd{`YvQGq1`8=(4)oN%Dem6h*LCa`k?Nf1V0MzBVJM!$?e zjbMZV^of#%f)kLpWXy7?;{M0XE7T&!iQ*$VsR$@Ya-zGaC1G`11$(D=VXr$*dcbif zyo|4J38xYbCgrIt`>C?%)NV@Tth_;byROgxPhNB9~K~dmN19Pp41Skj>H1WC`9O2b>HYRB_Y!oD4%I$f1?g9 znFROBwfN;DCN5U6TESB<&W}(qZGIH0|Y3E2$Fxex2Ss=cnL+oqJRJh&>eQ?6k{P@Ea-Hfp_2*jLzU_Ieojba-(t${?)PaC=z0@ zl5ck1(gZEe_21Y|O~vEZGaZww?4fY5T-$Q(+&#xGF~`)lV2RRmhSR59?FFW1)pM~^ z!)i@p`$id!-A6KpaEya-f$IAM^-E4rR&W-ja%vy!%ZRmz<2RD5LmMbchM96E9NV>K&b-EwOa~_A>oHk!<;5vjyV&(xKrG1f&-zG1P06=tbV8JUXYD9VHF?rJX0x5z6G_@P(8RCTl*wBCNj65`% zc%-R%EZRRdMuts#(p9_8#M-CEp!-(@c1g4(70=bNWU6atVi66l=skHT(qFHEFO7_< z$Bm?FpGm=hRu7=j*y+b^+*S^N4?<+asNF$6c^61NPC=CMFSGrU@v}k>8Hz+KVulcA zOfkL&fkgE8G{(pp|IQAwc4OwRYs4N`g_Z|oL3P>*E4Rf#n*qFaAcn*ke-SP)?j)i& zY_;uD0+@u7b@R`)AWxhsZsHiCSh7+a;^rANZ~XhPL2xGHPafYW9~xl{JiEC%;u4C| zJ#fCs2|z#^f||!?jb5MslKwiZ2Utp74>t5a=l##Xb>aVGjW-;4((T2`BKJVjM$sVP z$dDD|wX>MwbjE7*PS-K?D=j2~);VBvUk#Se%=<9Bn&guRm^@a3#Fwk1OdcGw&q%Pb5~) z9BK9qllsC!RWO+Sd(otPBz8zx>uG(xN+?r_sF7vyqrV@fi6!NLBct@rfE~z0A zx`rg;iIGsa>v^%FibS|NKo_EX&(P_uV;~K*@srY1b0z`)h9BgA(kn*l0qFlH`c^;m zz#U?>A0F!EMOA1#;w34$u2r!|Tef@VQ5y-^|LFk|$iaU9VGxnzj&6k zaWSKYHEi1G%_6P6Trv4S@R=dR9al_r=o>RQeB6fbg9QO#SI^m&CmJEOk}(xLOt+HN zNip9{&%gix@Xz2l!InrQ{`;UHBqS`XJpa1ay4w8t%0iYhjT>7{Ln8n%@01ANngoQ< z_IYqj>W4141A2=dsT0B%$PpaC0Y@()B!)a<1o(e5=lahS>Oh4nADqxXxf4fO^UT?a zY_b#Q&^)S0oan#*YdMl+1DVh(FmDesHp^hD0W~^xmvFui=emOoOO8@m}zQ4iB!k+AhPwoQko6d%k|0c*iefNrLinQR|6D2LZ}mVbU%=;|K4JXrQ{?gg-lsmt0SXsBxPyQC1dyW2xwScZZ_8%i zJZwQ6V94HobO$;hjBw`xr1~d~Nq|E&fHlf41E~Y<#6sgb!C3+BU#${9>;G@7693aG zzLIgtpq5T@@yk(=ar3Y}5Q+H;7`4^8DMH=97?~eC8G`sb1q{&ge*tpg3$^#`aB*n$5RPyjzZJOs#p4}kxAc^E}7%hYEW^<>WczFsaa z-StQ_ra=zIIw0g9U+`GSKr`tNyyur2dard;7KE$O$xw69pxd6gqjMkfDs_{xy6t|E z^FqUpWnlP^07nBYL;u5CXOE#1@=KPlLQv}qb5u%oAt!j3?`{QN?ChKzozTJB7@3qb zrw+_>LlbG3Ee)A$dU4w5E0|-nvR2~hjQy0T4?=Z9IIyh!aSTA#MC_+1m4SxK@&Xpi zK;;Uhx`>!BU&PAlpcoAxio{BJk58c8jS34hOJ@H>H)kusqR7-JaOlEh>BjO5=d+4G z;34M6&cT?&j7WseBImykDCMeli@%p$#rFNbPiMa)>vYOmR*-hWtYS;#3S2#$MS8wr zexiPE232rQoO3tWIvcqB|gvlH4dyx>IP;@b?4mf(k)K#EGrB(laqJJs8a_&ZKh~5z+gsdxf@#l%%xCbb=ZPD})n~)(=NBv((EmHc z6h{Qh8vGqI^C0i>+=$13I=0rr17&OFZ2vtQcb92?yG|BSJ{iZS{ozP`A%AI(bn75^5^;i(Ai05_3 zTAuBR!j%jKo$(c&_ezY|6>+x@wxHjwxouKgXO1L3wKY2}ZiV;SnXzTt)|nCAfR`|( zTh{7M7B1!esZQ6LR$H2)HKZF#7hBi1K;0P{H?7ql@Ox& z)MR310eobYLJD5J%5ep%z-7#?md%&g`1bIf#$;lfeY04POYyelQtA~ScRSBqmJk}*!h@t$qQ>hM*e6vFIQ3Q7MPn=B_;?K=n zU|+o6)S>1(yZv(zbyqX{a}%o0v!H?eS5S&&j+tT^{)I2Je>FR2MJIwZ8~Nt^3=16- zBTM+4HhGvXB+0GkuR)Oe*d1TCBuS!DyykX!)DY1@F%gC$hvJR zgF=Xtj(@dc1zL}Tu$)J0dlss(=@F4Wf#0an@2n6&z)wUMNt01Wsb3s z>j(Voci<@Y2Y~ulG<~VwtNel)8l*%B&0NRyZJ!6(%Uag9s55n58BO)W*@_6d)JY1{ z^~b~FQ)eBYzBiP$lYzl+AKk{Yp5%-}iHr~@gY8Vd)Nljfo_p0vkGI;EoP}Dq)`r#{ zv+Jn!nlHZ(wy-hauke^jeJ<}4g|4Pj+tX8VJ$$O!jRRRQ&~~uZG%PR$udM(B3~Iev zaPzAbHM0~ez>0i>bq<>#SkQrSkB*aA=rSFX%zGtk`ucQ& zc*YB;K454t;4-z)kXc-=BUhGTd&P>pG|`1?#_o}KiN{YKil0>7H6%m8faHX~D;|7P zKrx6-lu?+{IU>NODI${uQwEn5n|JN28w5Dp6qh!@Zn3XF!VPK4mRYf71xsp-jtiN9 z9X!quiup>-Y7(uDGgYqT43(WDO;yl+P8HyG3$Pcb4}md9)i~Tkwx;sv^3?e85sMPy zk`iRV$$!7G6;xFO4W87t)DgVL$le+`?7+ZASWL@+5zC(=!nvPCJ|py%VRAIXY)M{d zLWUDX5TYbmQ4oTli>C)2UHA9FtHqTZL`kXVlC+5=1%a=|*vTRl015}4agjk$!QTGL zsDxba8P3!lGf#X_Zb~kSA&6;khKp>*a%+oS*Z!4A?leR5faW&N@`P+SRt0c6*N}JWfJesu`r3_^R zsc0~OA=OZNYa4oJka=Wnsq}jn_w$h&dM`Ir<`r#qFTmS8nEmrp^&`aN!?t##X+!ZkK5{> z&{ld4bcxduUV3NZ0Ook%ukKJ*AsNsZmKB6;p7dUvgXCXBRp;+bMO?7Y$4qyuxD+fb zNCB@96olN8U=Q1{^}zFkGujZz1(tq~+kE|lD#VnN5ye_e_roAyUOim_qd&ACK39ub zw!L87Tig6xE3t8sZ84ZM;r-)1Y{HU) zyO>*-Q^4fRVS+h=+U|>A*iY|<5!*aMjdBzX?hE{ZOL-jm_*lR@&bcy+Pl++2XB6$) zQJbC|cY8q2<$e&}*|9^JZrm8cq?wKv~B+z37)_$+_ttWFY4 zP~$*VCvk$wzhaAY#W%V*kdKxM-)^HRc!{!6lQXUSgFi~#P2Ns2`_th1jYc zJy&2Kdl_7*J}4fE=iwGU6J4M9A75>IH{txwWqBP=n6XA8i3zBlvPT|%sC-W zf9x=`kH|;gLjJb+fDA}bSx-(tDw~9!pLoj6e0GAtXv}CbZaSVYnaZ|1Hmc$;N8l?W z!N1O3xI3m0UucxiAT?w(OtRlXh7>O;u@;e8u61@o+-+X*!MFfpoXQof(y#7Hg4CWO zw^wJ`0bq?QAOd}x?&_A@QFP}nFCADkfb*7W5{gh{syTM+s@bprhbZJds#q(?Ut;-N z(*@{!I^j6%b#ZHpB=rbxBGB+p^xvIfJEa>9Ztgn;{zbv(=##&GFWM|;Jiu81tJC5o z)ozY*-^TCH`_CGE=EA=qUu!U*CCTyR?Y(19qqX%!TlTzj9$_(hOO@V?-STAk-d*as z++E^vYQfvgGtNPoACnj3GrAMdFt3suKp{D-;IIX?ys{!XPvi7bPeK!!Vt#veTvUrH zC9+&dwCT1W6jjMp8(a~%oHFfKDXDE<%_Et0KPRA~MRU)XEeRjWsz=T>9ymA$@)lgv zkF02jD|XjWI0&lUhJo|ZyV=c>2#*};vE36xgvE*}o%pS&ENnd<@{1H7ZA8pV%eZ}c z1%@T`--Z^wmXRCTTx2BE5VqgV7?c0HK|hYb`Spzuhxb_kN1&OAGkOSAk#GQBJ7%_U zhsPFEF?xSRZT7t@M-291Sb8A#O=fbT_qbRV;^~2+mDp`{KL+;}!no=%?t$bvI6Akl zv!}f?9Naz*M57|sc?&q2=;saw3P)J+wN)!fwIC+JRD0DK)EH9B>wVb0T6T{kWQLG( zJApOaQBHxHnx+3%X_7zVYt5a(`4kkpKU3Pil~wi<{H1A3-UE)a2Yi++B@T)m{9@^a zZPH(nF{(b3$joN?&^XX%mSsI3N>*Q}=-?wMhy8uBCa_dIyAqH?Rp*c+INo9wFpeF6!y*P&hBKtn#0xu9w1Db zzZs)=-C<7s$$(6o`Fn-6cRQRv?Zj05qheoX^eKeNkuCJc&K8y_xiVC~hGL*v=8tQ3 zmHm+h)JfPpc-x&Ah<0ZFSFf~6{fP4;r}ghLn-k})kJ~WL5}_q*z-Vaaet}W}qPa&q z=_b@jRit1ErQ$zojUViBZ2^j4cNR&^(*hZA<*-{kE+BoyUzMGXL3^aIjIEobn;0-| zcuX&wWY)?5++TkqAC4Hql;F9Xqmbs7a*pI6Kp>FU)`Jx35ucK>fETd!Ok%mB)vo1(ZRgB+c9=J8RBdLZAkq?4Jn7w%EHnHWY>Y_XazP&yL`ykdb zlL1mgo1urjlAs4YcYbBt{qBL^=CJscaf_t4$n3k9nWe|#z21(Obc<%o?sK*?e9*4N z%YEPVSa>zr%RKv_W~fND9~Td`#^dI6<(ywUZJ&Z&PsDI*U9Yw4(fNAPpV_Rg$UXY= zT!K}`;|dDT^z=LBJPuRyil8dKbH6;ibakLH_eCbHUz93O4U8Azl=r8xS@ivQ> zuMZ7_>h%((>giUCh@Kliuu2Oilsl7GG29isBcFlSx)}j>@I3=4f4-cc+X~drK~(*c zcGF}<4liPiM1u6w@L2QaP7@~n8ctdDqmX&xTc8_NG>mzXcQ#WLqKF*lfdz@h)Tff! zQt-}EV8E3|L^vNh+7q9j8qxc!miVGxX=?b_f*{}a*gl=FG-SX6rX+SEaOrJ>sxWC+ z1#9;krL$YTP1T@yfBbKPX^hwk_?5XPzCiwTLDC@kY>~3)3B5M=v-^|+tKXyf4B*u2 zO=p2t*Y1ZQZauFk_*4n5Lsolp5QOveBnOBI3T8bEjj&Zvpz9^_${}~k?gEj!HR9jx z);%r1{i8&e%$e18YK;K#h;;b5)(h@jze^*g@}1TBGa7>-4^B;kDIPfga{&2C_B(rJGF>TSuF+r)>=w0S`VmjkXj|12x7c4 z-sjbC1^~}6v2D)VmoJES-;TF_<=9DYCk6q?$P>(u&_TYE1}p#x zjPrkgzGFvbgXn2Psb3AcMtNGePXu z;lxH?Y-%jX1~9*!bpHNiv-nAcza8*!+YHA!w$^IhOI919&Sbu+^}6=jEf7Q4HM72* z40gcl=gR9VkFp2znnF|uqdH|#6%3B(4^M)Oq;9`8lZb#+v0SoX4?SyZQe9do7E0qa zDg`hx>;ju_T-heX9L9zNAt##_X3!wnBH5~^3{yjwfA@Hot~JZD~)Cnbbq}@UxM?;cr7C$1$yA;jM2nDHwK|w%)p8I5h-U)2hH2Xw2J@mq*q3y7G9xcx zlUejHX5g+FV2Rp=FHtE*J%lbaM%O_D-hzG_nl47iiH4^C9TOo?8k&w0tY=>_A|LIg zeU%FuJVHaAYw+yqXwH&?oOx*MR%qxswbBz6E^7?O=GC6=NYVJ3Z0eDDV3G^?TTyHG zf*vpIm1AH>;pcaX!fzu%YDmhk!p+G1jOzE{Oud>(2O^V_ln$C%owv$Or-cn!_o`|~ zY_v<^C(QEt6TsX7{t>-ENW5wyVk+gU5QZ}?dC+f$&NxAVKf zfXc9AW1GP7OTGTm)r9F6kJjO0V^|=no`^Fk`%5gkApAKcP2;mOKISTKNWa(Q@^!+S z8th7$-oTpgwEperiI|$4mwYyO8(soHY6gJ5NmyX?^+-N5^LLAhq{OJ{{ppj#=}y+x z*)^N{=57v!)sB>diU?eF<-AzTN>cg8 z4F~@d>--Jpcqe~;YHRr_t^QZae&zv|iP~9y)O`As{g$iha3II*Jry|PC>UbTIqAxf zXQ@YB*7T+Ea~H}wZJ(s1Bg~R30vh2m1UhFeKroOxY4QfB<3B$0=0FzJyZi9u0nn(x zK#BAc2gi6YA>l>MCK$Ms5A!p`qzou+;c49^YDInbyea+HeK5l1C^VX;*Pw^R`0I|v zfvqG3^jClkRZj1~^Y$i<5UPS&o+Z_$znxc)(@1R^-4eQ(^5{PN17ee)e&2iafy7Ik zdIxIp%`lrrXK!UWFz5@cJwssNm5lKX8Y^gRj4qi2V#)24$X0pqMk#E>`3IHNEiSgA z48NJg*>yz49=qS5Re!O@i&4+w2hzBH8IDlYTQVA4shp>Bg=kv$xa6)Dvf=yDb#RK^ z=x+$K5~@~?Lf(ew!+# zqW~P1b%0J5)TDq=sA8*uo9sdzn&0Hlu&dTjSy;=crdGrQX}M6lPG+u;g*?ryRLe)? z>e#I}=;IKQr_%faJw7l3{kJ&vYiARD)wrE64&UGurYNxafQrF{LXIEkZMz#c+Mb2tnZ&A`bd0tfER5r7{aqp&P!&uPoydss0fS{~=fm+pNs2q<2K1m8WQn{xSdl}^eEQ0Otp{L)rM2;hVm+HK6ZP z_M&4S#Pl_(GG)OKX()P-A6wX5v8QsKU~tS^hAL$OXqyf@ zUaDTbaG}jra8{8BCFwZKgE)NQ%v`}T2#%7Jl(Ir?zax?#Qsl(p2u;0F|3I?vd@K*D4~c( z(rs-BM4~8fje5k_$b;UC9(w9jBv5w~DzXJmkcdF-W3}Hf(kyIH29|TltDtZw+@m^l z=8Wln1TiSqTr{%#ykEXh!55PrgLt8!wK8%t>Qxt0`zhc{L#YHwQnJ1X=>qSl*0`QL zZsAU~eu?@TX*e`f+1uy4J=-GA=TX6nz19Tt3CACh4#J(I)C& zf8(r(CtZSrR9t>pLO8rafexoZo(Bzo72S*7a)I|VB~#Vv^hRCs)3H3-u0X5n<6mo*T4IRo`MtmH{Evq$a z5e94%i!=-J0xjm4sy589MNGDhbK8Eo_qGw0x4g^T4ZkRo>`SSS{Dz53B^wW5Wi5M_n_B`@T^j@_`rceSWHR+n~_7i5G zatkSPCOiUsCIMGZ1dHvt6Ae!)^s5S<=F}P%!aEMi#Dj&)JV&8%P&bz#b=^x`3IhZX z)VHClz#pShLSCQ92kPYIk-6+*L(5;NZlpG=y7XvcOTVUv42shrnTHcSvD`~eIt4}#fcwNK%oBffT)vnN>QL(6(QCmW{eG`gtQln~u&n*WDLe!lH z(L+r@H9Iqt<3BCgx6`K~2TxFMO5?D0!CVu_2~0!85>hjkV=aacr-RWEGf#L_NC_O8 z`YGnLuXjo@!`NjTeU#1z0z4A9b);hS#L+1%g|;(n)p=rjf`~sEUm~ph0p7+T>4&u; zZ1(&_$5E@@tQ=B7j#{^wTEQa)rzL4sIXF1?MBrHQ7_vuiyDuCl9Po#+4mnlxB^eS2 zQJI)@Y3T$o>K>osHiUy|dBO`l-A#r7FtDKnehH)qpydbfzI)ijF2J&Mt+ z7cq_ja}hd!w`g(?A2)06jECfm7WopEl~cDTUurs2aHC9HN0rjdT@?0V_1xsjR1wCYvq_kKAU-{6&y)R3llfc7!b0-Py&Uy(woO9-N_I^oZwJH(a> z>BXgfzITopW3i6F@Y5`UCQ<-6P;Vy6f-~7eEM_Wd#%c?}WllD#)14EvtdZj}R&~o| z*ug!`LS0PvS*>%zvyrgGWEumh@?Xnn^48Rt|13C~rSTppN7Xqq!pMa_7H0dnvE>C; zV!b3)oy9{g$lD<{=2UQ-Sifq^;`1ST*-0kNmVs6^DTEuTs10{+ZJK7uK+tIpf7VAm zMx}3zho2PxE|FDYcYH}KC~9b!pM^hp#Jqb}MUrNn$M`=~onx3NySA>od$w)cw%xOB zo3m}(wr$(CZQHi(zH_a;&$;*J$)EaDsZ>TXMvYXycb8+3x)LFPQ+i=SZ`U*mkWXdA zR?s{J*+uNNHM*s==-1=|@14~IoydN`5D*3uXuYn?vdb?DIGMRm@3FStUIQMqB0^*Q zHY1jLu!T6O77f^DWFnZ2w3#x}yH1JcNkmV6x#U?w^(KM6*1AUlz_WFeA^E74v$!I9 zr$mLy1zNI(W)M81DqYGl;F3C7#^>KgSitf)7LYq9nwa_cvSw0JI%mend-!`6c8ek% z&=Y2}P0pez=MmJZjQ6RZ*8a?jHaJ;5!azC$fqTc$o#Vo6Gi@4U6ut8aUyfpP%p)q=zgl2PP-K>G7{yT0j+P5)=c~wfyOe)lzS^yK zDhDf^T27d^W?1ISpW5n!Is8&vwM!-AeJQ1}@8lE|s!b@0*qg$496J*~KWNQo(LTXC zp#$IvmSW4Qoj7=_sjmiIjpW?_C{Zz_d$_;zkBK4K?e+u279rS|QCp666KvCo`m_{* zRp^aka_ePtP>QwJ35qsHYh*?qW16`W%qF_UqbsX;NAgQ)P;U$Wc32Io}t;%yAPj;9r_74Yi5p1DkM3F2Ta;C6zjDzD*if&&n^#Haz@N zti)+mp9sxX_GKq#;VE^8Xu~gV`2a$+pdCzEZ*fU zpAWy|I$IsF;F=52D2B2Rf(x1j?OSS#d*_6BQKZ6qt6W|!BuyuH5r*86`Gpl`TSvx{ zD+9-vENdo4)5jPp7Dpm@6nc#!yj&bH0&2x13lz>w(L{klHK$i2r|XZ$q~_w9dj^!X zEXE3nU>04?J9WE7>d|}BKoU!j+>PsG^M;{eO6Pb1TW{zlRNps`Kgvg~Lhe>^TCU65 z)En(KmGJ+-e%4Two=y*{b6X}#l-z;78T-gcZw0g)Ol8A3e9n{lzHT<~ z4)?}YT0LkP|MKPP>QvdyxR7Eh;r$k60GV5&GI^@S24W$`VSn1r>#Z{m++Y|#{UgB% z-`*e?PcN^IBqTg1Aq~VB$G$S~%~Xv9^xfwybZP!^-oK2S2_eLPy;+@2g7s$9JS78o7y(JOr#! z{{aTJKW%gYJH#ZHH$%DL0?_Bm>%-EPY5Pr&s@WLxZKfsZQi{XY)G<8hP)z+}Fqj0E z?VJsHkaees1U%`VH_Crj!iW$?4pREj_B?2g9-q)hy6|^lWVcL>m^OZ_tD>+EuGo27 zh=8S8nF_3TZSSC1a3h{Lkb9j+7Bz&MRO)jOGk{lT>ScX>A8-BcgR78G)OOv_1O=2K zXX5kkarDD_M|W#iwE3yW7g`gs2)p6Y(pItT`K9FI8g;}?`?^yPg0cq(Y8uIUTfIyp zIzaZ*R}MLPe8nQT24!jv12^%tw*}`f`!K1o1wijgQtl>Fe?IcH<^H^6M1y@7 z1LMq~T^Dm`_Eda?~AeM}cnJQ%Utj$c#6 zo6E!I7Ysn%YBSk4p1_}_Fv?Xrxr^@2KSaJ74nl?IZoFxq*!lsvv}Y}$Z; zC=u`6@m+-;K)U#9jP;qeOb8}EC}DCCN+mx=MwYE}xVKEM?Wc!shFU34Ymvp6P1j^u z_YdV@zI4ITjL$>`w+tnD1*JROR%zJ}xd7;de0IL&xXwZp|84PDzai}e6yZVz^(Yo6f$0wg}Qlgr&rsMi0M1cgq}i zDYVisqDAs zr~A*6=Vlf*$qyh0WKcb1f;le{vm_IK0K%&Af%q$VOT}u2-~kP0{nKgd&+q!ASHhI@ zn;T}4Wb#}UDed69X9g>A4umSf3srb5VWVaT7~=FW3-&L;C}MTN!g(syT(rFE&AvJB zaS_C-H4-~5o!_As>fN9C>4Hi&Tt&&H>jl$Wcb?K%HIHv@+V@?#jbmRob5*~|s6)RoAM=$$!N4HXCT=rh9L(-|34fV_UZ1+Qk0+?VI@P)c?X zi|(>EW8I312w|em%yAm%ao>4Ol$KxOUEc59M7#H0cqh1h2MxmY%WOq>lI>q6&)ChFwJC-VY2hCra@t~_nk5?6QvpFRV_8wu!pL#0vZu28H@nyyHIom7H zvX(HJK0sgU3D)|BaY4hgKdk?mk*O^;bKV8!li2+vf%ct-=v(cI#Mn2Z)&1UrTKv#V zwed&G=w3oK3|0h{E~h?m!=m~X*pLz|lyA>E&#w#qvw@d9wo_Nwq}Mg~2D|j~ydE_A znZFQzm*04s8R(k$X+13oQk=82r;f(t-E!EIL+L25pf4o-g&}J>B)pUM4gzEDF#!YW z;6;Ta;eq_9Z;Az9yba3;XkxFN*@$spWJ?*!2O)3Tja-|IpD@}}@7~byz#|VzL@LOa z*$z1AT|oO;kXEeEI|`L|F~w+|KUqHyAGzZMQHfy+7=MTW;Sp=QvDSnNq$VQPH=NwM z8)2AW-wb#;a`Bju;z0U13Q?|iuKa_wY%=HJpU@4q09_-JABd+BtJc+1uwzQ}47Qtw zM=42kOuyl>f|FUUG3SCz3VV*O_Dj_0_#Xd5o(YOYU{f1MJVjmF=i*V~S2gaQw<@tZ z=fn<%UU{iiVdwcnd^Qg4&C_Gs15vL* z$fm^F5U)mL>hK+4IGBwqvZr>}P7)P6KZTw?8>I`#EFt4(c02i(ws*`AE4e+YxI!Mu z^e%dS6MKX2^@BIeqH?utO$?j~YK^Jv!xWI+?wTC%s< znxRkK?prO7rN&^LB<+bG5Q;6l>>8hO*|ULiEHx9&Q5~HdOE=y=jwcvq9d7BF2+5Da zxkELOr94B%C+oEj*G(U|>scFqqZ=b{Pz1&Xp)C?8Og1?oHGNq#=RoA9wXp0|R2+X2 z)hF;OZ!<>wJ4lz8)~ktNlZy9u;k|0{@G$AICTJj#O2M@`N|t?1*%$&IdCwkp2aPU) zrZMRz*xL`2U0+LgfiBd~76Jo5l)hFew_cU?hO%>3JVm_0SbrQ75mF%Ee`7nOdsM99 zTx_*U9!J0leAbdSM@xdr@0~tHLz;s(0U4ajE6R;jSWi0N4{4r8O+FN#ncZxdDo(X= zi491i^m)H&{CERlBR%5HH&1N?!tv7O?!<`d<@xmX@J5;-xElpUv(reQw;xsQOz>3 z;wPu?T)9PDKXXDk#;hUw-pD8G?Kp<;Xs+217hNEuku5q(yKLUP^Xd+sb}MTs;W+-D zl_u}OCW>Kwl8nhWtl=s`;h>)&q-yGkOC6F^s`E}^2sp926D@){FuQqqR0;?VE8was zk~je7n$d5t`#g=tZU$S$`-~{R>nZ=i(oM!`%ckO)(21`6ie={fXc?}!OX*BoF!PlA zZLF+W&{$t8aot=HS!Pmi(_PdPw{B%JqdDm>=;hYZs=d?C(`trqh&{H~(xlAB@4A5a z=1%?OUEs@%L%dq$lVBd&wajDMkbH}B)_v|Qz`V<&;1XytBu`?7BRW1(otyeP5)ldb z@uQE-kazTA0Nrs)mWo3{anOcn?h@TZMMUN61DtwK7k&NsLqH+###Db|^rNq!uHLTaa`9&|@j z_ zBgp1rhxVGuzZ7lfnPVBeN5|_>izuB=nH%n!sXing`lznMUP47u!w4@@R(JX7QAtnV zy?e5Xq+*Dpt^@jLV2q!G8%1?M$KjF}NB#;){Ye`6(}A8Sb-||7LrSN+cX zOS<*j5|%U9IhF8HW-8NeCT0Wh_bu!4(4of**v@B;9we(0Wo@AVF z5^6oqz&&WYRIHpRInIK^i-sAB3~s(chMSfjW>Z#XY$@e>EzxLM3g{C_qH=9u(F$x$ z)Vm|8baDlgcq*i*JPZtkr2WMg9Yhw z%`t2N>rwZhoS2NAnQD#XI5kZ8726?H8@{r+q#?H61Ynj|YQQ$7#L2pt6J8@4&NCLZ zbtN9mndsN|8xS>{Q`>X&{;Q6bZN29yVF7KkXY!dv1efjYc*hY|&W;6r zT=EL?knz95e<>2BpD!4$4MhQ~(~F0cS-L zOtGTxyC4VuVcczV3+eNw;=Tp0%Oi(RlU_s1MIl#mq8*fl`_rrzlntw~4G?!fP5stq_PNpPstM9fda@FFLg5{w?0s?e$^kzy;;Tm* zLrFLkm2;8SVwvG{70vml^EeK3J8xNYPw3DgzJVf)$8($(JLaH5nNSKFlY>K3jr7Y} zuBpZ0R`uVPlK&n6DD! zDs96O&nFWySek(ggHXyFinv^`nGa@+xJQi~^E0cE3a}d+sEn1#3+K#MdOcqffFGWG zAC@rn?AmzFUPxW7*$Sr)tm*Wm1DmvNiVG}Mw7Qi^b4Q#lZP&B-M$>Z3rSFpoiS>W zh_cWwBHW0)i%?_&B0rkdoQU0ocD>VKEyLJvKwZ-$B~{##UM}bxrIkf>lPX(&dmDwB zQz=?SW6nL(s+y>`Bu{`bBCYY!#=K9W&Qv3$O{Z0>PtOE#^kzYCA1G98zj2C>%4RmJ z(iJbpA6~nx-f!A?_~sVT{G|n;t?>7vQS&E5NC{pD~wybzvG!6)*?w0E6(2G}--`^C-v@1DbDFb|yYem7KaB z5m4}OvN~;5p(F1Y+3;ZgHhvHEm&ue}I+a_;{UVuaH`*CK+_RPam-V}g-E9XHHMln+ zZH|eNzwE31tVOwDE=b~^QY|rl^8lRd!!ua)#^W=CU3D1b=7Tk)@kauc+cwh?f!B~b z?tHsZjW+0_*Z71?|EBvQ6hjEwgH%a$&)te>ehkqbC}f8}fp7G%Ux2A)(OJ}G z+Y@yXqC8dtOze0ZzXHXKf*gca^gN>CEKZJP_o<81r+W$VaH@+a_Ou1Jb#AbbI;QmW z_zLqxb|jq~@Rk;qa-8P_qj0)NQ1CF5dqUT1`qj#`elqC{nytrKR5-r^=mCQYdZ0~& zsRm{85m^`rHsagfSVziyE(;;jNk=(5T;Hd~!ph-L74NN*j!;}Wo@i5KArMxm`WQA4(>DFwvJ6R~HTGorPWEQ5{6ha#m*r4vY~yr*Cx()a() zs~IR8sG={W4=zHoU!9r)DM2c)fD=|WriY1~{6r*i=ZVe$_Zs?0APFw-<6eC6iEvNN zKVA;}1P;tBUq;7ILQOz&cXUIrTN^MjzWOy9HmdzOd4j*n7G_V{2h+p64XAAA9Pp>g zYD`5^dDGsG@Z?*;#%1_1L(&G(A{2n}?*3wPD7fVt63o!G1nwj-AiDL)I)n=6 zi#RwhKnY{Ydo~SDyRNUPO$#Iw^pFJ2v`O5yNw(VtO-3eekZ-SYhR~^5Vn^;jNLVG(bo2HQc4njaW|u?L6~m#+Lsl&QM3 zask!;HfKX|d22Lv0hIYOL!kOLNv0Ii@kq^Lz+0kp_RdGY!0I=t*q#6kBptUHpT`OH zY?!1PgL|D2Q4$saCrb63E4gPlpP8CpJ-G_slf@`)mR*9A*%A<`Ia^>7jzRxnfH8h_ z9r=%xV4O2CY0o9F4~hRu`VHRfd>OP%SOTG9@NHZRqZkN*h?Ib<2qZ_yKz{NOrF)si zgVgz91suJMp>pzYUt*i<+B998wOEPx4evWNZcD#$pbwm)Q}Xg^kWJfDO2fm91ms5XtD*LF9@59L8P;ELrF7moob*U$EVTes2Sw<3* z0a1N0X*gU7)^O3Lu0Z91$zap%MJR}i`K;&;=#p`U>Ddt`H}qvmBTIExT|=T^K>xu; zwI>UmtdPj8b>ARKC$z;W4CAgsPe_I3`uu~>V(^89*SJ}3>txX-!osoyIj@Ug`YLoN zgW8%1{>&+yf=W64)~$j_8aiWwsFG=N=b?Yq$uZv7xJg5YND2>t!(I*0CtGGP*;r6r#A%M<}e`H^PAY>=l5?_w*x7|m= z^7&oZ6ZnRUz!$pOhVHmU?UpF9J8fgEAf0psZt{EM7mL>oU2}dv79f80N9yi7;a+-inW%VoQDCA#FbErP{ zhxbB3S#EupZp~gDzDNo+KaaMbqvyrV-{h;IKPG$1j#Lj^5k%5CGR(dbXU5Cseo~e7 z%}L7r8m<2HGcYpxe%Z_)og;0Xz+Y1dCdLmgRh%Ax=NV3MAav8V#H3i$Yfq?zECS&4 zJ96Mf*(ge_mt3rz+>hD!y7F_R+AJ^x9}A#3s660&ztU_r{;Vz^`HqvPUYh8Trx6Zr zD+Km_QYHT?FW9}G(S4R0$yNjap9|JoR1vVPtV#;-1=E&0U4S)>-6KGT4B(@N^mpbB zn)%)JA&7-+>~UI;+!BIenCt@%RxR+k_`dis%)(Kij0pp;8C*mXXDM$h|EaP$Qrlfx z%DXz!T6NbaY#GObh_AJY`I55oU2yBj0Qxs5YQsEnn_J;4beiv^=ixSNAo#HRC{*-1plN9f|_!VIwaNc6^_2 zM^{(`13@F2q;&uv^UxH1;`Iw}H<%u-F>3KUoIu^si+iD^(i0by>MycDWZElh6GoF7 z0LNZMANTYDA;!S=UBkBGw%pLoyF-5-fU_~yuWX2!f%vyhGa!T)eMfkjZm%sA3HOmA z^7n_t7WVg1Ipxg&?j|9ze^RjE(|K)R!-PA$#Gf#jkz>eoXgA7Rx#c=D%~{blE`ghq zyVi7DJn6Oe9!L;yu=GWch^x=F9=Aa^wAQoM$HI5gK72x~c62NokWME(e-Ij{uBN7h zD@dw=@r#?o^>#=_B=mF@iMbN$OCp0ODb01xP+0(zrO(L#Olpx8HP;ohx#WcAW@!16 z#}?Z7AKN4^InDcF!a}Vad9D+FN&2V<;)ls;G4qJyC6_{;jg_=&Y;A0kv4JgH^8z}M z=)_z}B^MhEbdpxWZDkOgB13$U&$&ztTd~b__Y(L3gfk>b+0CB)b#fwm!g!jTghUXA-IX@FKo_7vH31;W;>L zX=5=j#N^1z)m6)YTJk20AJfm_d4Gt91F_9Q09~OnM`id{FoQQc@%*h$lFoG_j_q3OGZCff zQa#Wt)QDwO)ooo>L?9Nln>DL}K$*~bkzkQwNZhOVpvzNxqU2lgLLrFT4-Enb$ZeqP zzvW1?VNF_Y41s**)%=0MzDm9j1acKcNqy?B0`aPfZSK$P=fYD|>7CFPLlP&^YnctI zKv6Mc2`NSbEJYgbt3+8`$cGbU_Q*tk`d%FD1QqlP^_K8MU?|Y(XelIxt88E6LiKW_ z)COm&^k!A}qIt*}<0|CD=RlQ`)E7F^n@Z8HMn(i_|FEtK&Oe(e2Yi; zbGbu1FXNYQlsB|H5Md*rYNIqb2P_DgQFj?x?uY`;2$jedc51r4%`&O$NL=K0p=hN{ zJ6n(h;h-80f)4}_7 z-ZOPlK)&HPfr+Q{G;nz1<3LomF6DbWa;)-Vnj?rpv1Z9qTFku`Sg?P=X~U+5rC%o0 zVb6H`EcsnNM4SC>z&Op9Gwmctt3JC{20;ERWAE(v*|k`YrhIUMw!Y zC*jF%vBZlOXu;lvHE4^v-ujrQEqp>-Z?~l0%SX@T5xzg%=vAsAz3*XGjMe)^AHZqI zJJ28D-!7X4^QH_*xANh>$nr8UAV7-Td^*Lh1%d{;Ax}S_|=Ln28$k zWnqh>ALmj`6r6Ozw;1r(k3spjEiLJu7kP)A7U!V~8Jt=syvR0Dz?K_O+J)8SHndlr z+L0X3rwZS|hPfYpIG;FE2xWe`X`-da6EyOd>`=wNg?)(ZgedUEi)mxnV9o5ee9R;# zTr@d?U^QcJEugrbUD@!u&bhvVED~;{o|#tSV@EykJYfs>(A9qQ*3vR$B#*R)?Xpq8o`yXX{t~_*X7wg3F&?;?S?_x9r!$IB*%WT<1l z9D44n@V4THGQ?T+>U`3>ONTHKP%_~jt+0~e(Qm$uW@W=U#-t)^`$BHs#iQLWF@#7= z(veQCAjD)Uijz*0urM^w>{WFoT)R<*{!-Y}l3n`wgLZ~GLDEjWvVp`Y+GEI&p+LWp zA)=d4r?S0(#46ci%4#AyMI~Z!6`fjnk&30`8!#+`* zn4x`M%=vPx%)ByRn?h`Ro#n3L{@l@gqhe@K6h1Do66sJskqR)GabjeRk+c-f_;Z6A zW4_hYyt$2X`7w>he$#lbf9|yYj_#LBP>eOW{LotG1cRv$;{U#y&){TrDV8Wk^jlP9 zSqetN(KwH>G0ym^L_rTbbeM`)HBg~UJ@4f=%G{#dk*2z<4kF~IV5h?E!u8ff9lJ_C zMhAP-2%m}li z*RahF0;?wEcEB)XT5%y?3SqJ#uyYkjo}xN3^WW>k1N)3>5)h3^HVTDbA2FTzo;n#tv?b_LC;w#DWvpr*bt z2Uc5M_?Z_ExR`6^XTgyeUL8Cco}0p-r&TDPl!FrpG@x_9!(hW42XceFc?Pn|c$S2F zw@*tCPNN-8CVS#!EHQDaP_YDk%C~AXAwZixch*fr%HPv661efN*l0R1t$##4pE7{? zFk4`g7Y@Q%iqa9K=N0DBo*zWCutQS8cfuSS1krDV8?yy%fiPHQ2WfLTY(4@AUjy3C zNRmUv11$R(%LVyvm7LVhvOd_+s3yM$=1RQ-6-wb^BppvPuIoT{JdDH`#hK2MKh+z5 zJDtW|9FUAyaRC4VDaha@Ji=AC z#v{IhZSrWQ=nhlBRY^!fQSFu-7Np&^q5lC==4?0hOH>mx%7aYupQG0_p*@vTvITF0C@a*`QNRr! zo2(e?*;@{O+ePZcPK!VGrm9(u0arKQUfG0f6MmQpOJC)ihb4TdTB-%wfRL%mYU1lr zarUm&pUY#Mf@KpmMc{fm6=Zo zv*@ih2oL`$3m0Q|JLy&cv-@NGuy11{w(vdG>+=@GQzn)8hs0h;c`j|9UXq)xo{(t) zGkcgOCec_V5%-ly?MSCPicmS4fwJn7rx>WuQ}aRGujvrHrY_B^k{Cq;MLk)cX8Ob6 zAmb5eR?))Qf|Ry3>EeUchm%D2{?=R=58lRC4S)Uz>M3l(g_q<`GVRg9(ENjp>$!!) z{hip7iY834K&2MZoISIL?_+QC_q2-ue#QniiNQ!&EQ6!bD0VQ~w!G{$3Hv^g(q~{M zyhc8xHxJ{Ndf2x4gsanaEB5hS#ZA9yjD0hkatX-qht^dHOshvv&0FBoz0XpG*NO+} z=o2FArL*vbIzKack4C)4PMB2CIVy69tW>mn5EaPQW;YL+rb)+TXPIWHYrkkww=Xx} zrMucuQ&u$ObSd{QV~cVXO|@5EN=l$&(Uo^qB3Lu%Aemv5+KG5%Z(9n;QO6Yjjg5~M ztZ#~$H14MOijA;uRhF3vj#WGh0XAl^Bp*IYc$I)=d4>J(y5m|PY7p%^U|wVeznVNd z3=%a(S3p*Yu<>7MmILhlB{jfmNxn-&DtY>yLEbu<7<%TBuy>Uq8$sWy6~L zr=76hJyJiN^^@DzQOw#tmhWU`IH@&V+dcZ!1I$?tPdIm}^hqo-B z@xCf2NJy{el9N|}la5S;aByTwBsy29Yi9<8Bsb1w%GaNN(lZX|!Am_whm?+w=r_bm zt-N^p49sofSJokLH?*!Wy^3?&$Si_8>nT2wXE_2Ttw-sylf2J&RB-KzTMKknNp?vf zf7DO&K@;7Q8LcN|GDb4S-_`dGy$to;YyHf|xx6Ybygpnb+2%)3Ya?Q-2;c+Roz4Bb z4MoYN3a=|!p$8?|N$k#~3!mgCbCB`ScMT&Ex;;pgN{QQ&Gc&T=JGTO3NTPTBT6A9;R{TjO*>~?#h-9k|sO`F|}D} zSa@IPL||bp6KyZjf|UCW{8&D~)o)D2#m}day+GVbq63~6*QKw-Iyt&j7)A!zJL)gtz@2HXsg|{K59FAHMXmv|a9b=hF+`anZIFep7C(FAe zS5@(!c%pK+x852Y6!M7(VC)* zbjMkcgxkvk*CDs1>L|X{MDNxf$HP}%qpLO>n*A$3IHZqudPE1O0Xp>mpokMfvyn!a zEmO;Ia8igrs$-`sXJT6?7H+KRXB%OKTEeQJVIgH^5HgEE1`ty8?{225RYp(!Nf~(5 z*?(qw%IhDiDl$PFpEz6Ap-iPlcUP6GGL0zZ(IS&wQ!z*rXGX;lvvjsj>#TmJpCv9V zJL6f;)S%2TDJgd-iM9)OvC+QknY?C}mevFGUH@cE8qaM-o( z6T{Qr?baR37Gs& z0P4A@??cc<9ul{@vpbw`6JA74YjTM$M zYuGdsQwf6CH2q4ihr+)$V`VW+o9a(a9UF4an^*B1FiULxS!WF`9&DWhn4s)xF^Jtp ztj&+aOWX4?1r0^`-ZwVNCw(m``O>URoUuWBlOfX4f{5$_=(|ZUJM&2v|LFNbBg|1r zch0MMKzcU0_g9=8mq}^6gqFM5wK!VV3IrYyt9|fMZ)^j+f(&j(?YkA_ znOk&wn&eo*w&2eh(Yn1COjZ&YG{27PMZt>sUT6!)C)2tHnj?FS4Gd@W0LCM7^u8@H zcPc!O-U=T=xn`iTR#>C%8JuD(xywv!ALDfXboQd-GQ;7aUZ_-*1H&y<DqzjBB`o!Gt3sD^Eu1X6o&-3U@{(M67y_oDX+|$G6lp zvo&XyKW9?V9kpq)@z#8w5S~;JY?)x=Rr5ymOpXAW@*cN~yF1y&)P#KgqvaiD7LTh;?V>m8e#-HRChwP+E~|!8^auJ4?voIifQq*#bI3=gv=mj< z*$-OjAi|eh|B&~A67*Cl1cC71Fhtw3AM3L6*qh1d6BoW{X<0h~3pX?ZFR%Dn*C4?XuL@VsY{t(4oUy#(R{_F4WB3?D#UvdT%xyd&KouWdg zDCB9vur)8k4+Ll`sHKwU_=qcHK4J)j@?pe|8+c&9h)3mem5QjXd@sus^O|=l+R7c* z6N=DWQ$S!_;upNdOXXT$6M*Kr^*v7c{q4eA(|EYU1LqC! zC60Tbd3X58wE|+md6I^JW^k`yXlRzFh6U~}E!7lQ&5!W@JXd1dBP4BV5NBd=oJ&WNPfw`xuqLJD5Urj-ycQh`t&Wol3niN4&Zl#@`OdbJ}mzS0mp+#_k6!NaRekssY79s6B>g$jm|N$ET> zkkCN|CroDIrP}qRE{$^hpCzV@`pgeBC;RBBa1!g9UJkj{yl3t0wP?T@%KjpKvYm=V z{18or+k)o1wHTa%1P7LpeKIcceM5Co&4rI+Y1aVWr?BCm0~V8lZD9SImj$|24D`;? zF%51^fL(rt(quw(qgGO}t~Y~nVN>Tp9kCaoQ&TH~0WLdlcxuR(7z`38q}6f2N?O$q zIF6navYeRc&*WtWwhMX_7dYwEq%qq_X@DfCOco}wnV~e*40_0P+nJa~cFFB`it??u zQ>T(^q~}V%+)#i;a6vexkWBHr$bh!MCNQH1zaatf?^gdX>d_+KtABzGu7LE`@Qt(+ zf8}erDsS2AcvX?rf+6(|fPU)|?SgG~@e>sPIL>4>P6(zQr~dMFJo;^WJb5iQcIfvP zE;cFhZt6{_w?{Pvl~g!3@cY4|Iva-bcZt+#_$61b;=*SyLb9LqoY4U{NK%j$vL$ne z`JREw*rOH&wrqj<7;EIBs5!maarqyNDk`*m@-t+7W?cTe5cFqJ%V2FO)lUTLLc05F zNg#X;3E|l&Lv_L6m0A2`GrFL3ua?aB*?A?=5psCJJkJy5>>H;&&GiQy3cnFee>6?j z3wb7Y=7=0;(8>V1QWU-l1WQBU6(|R?^jCBut>(I)v#}CF{-w@>8&Y}q zr;25w;gfC@yP?R@yO>9<^0?81SVsai9Y&;%2iQ;Sf-KeZ7hDh6sXJ2^qffgiXAABM24xR!P_XWXfl$i?-b@gDj(%9u!7g6cnw!HoC|ySo8O= z$yO4E8Hw;M!V=VZe-AN6=*T*rQl0x*?Bml3o2+zmZhC9Faua1!;hw@tdl_{lz`REw zAy@~7@;hjXHac7{f)DnnTE8qZp4vNoWv5c(1FO!__gu+i%1RnD{iu_9C)Y!pjCFi= z6!{`jLy8o-b2?(w(-S-13%4I;!*}iPPUcOp)>Vb0XHgcPmQHwkl!Qz~ueZB?Oe(QH z%G(Q`m+mg`84U0h$Jqs*Nm8E|gfS?s*EWQc&f`0Oa%?&bHWt|MAum*~j#VPshxy1r zPByHdOO~uhwAahub<>(>n3sO|VrA$drM5(0e*qK#C_0*^j#LL7xFy`V{*eBb=17Eq z&rq1X?Rc9lTgRMAn2=Go*tMW^PXl!r;_k79(6LTykHp?j1@JSW&^pLUu}TACqyq`{ z0s{C20!}>%9UCPAE;R|=?OEODNMMcORl6gY`#Eav2lyIlW+mS!X=$)3e-%vbL(`+c zo_q%G2Fh>JtYj+-WeND4h_C#6>T=*Q?$$i; zI`S6w{hj)Z#lBsM-d|DRM+SzQgRTCo$*$aN@@w> zmOXT9tZO%lZv-&DVVGCiSD(QFadS=gy}!kWxOe$Cf}Wpfuh2c zS*sbcA-0hS;wToGV=OAJP}`!bw;f9`=vQi{!Z&;C$Bs>Dx47MK%Uim>Tt93iBAO7a z45XzkX&Qzr6*sby1~Lj-=qDQDR~%$&#@??R{~a6%4z)Wj$3O6YW%L=3Nd8ATn%(V8 zRbVN-rIE#J6sQXyo|}>TU&z5stkvjkL`!<^RLLIv5jm&1(HV)21bn zI@_fCdb+v$+2}w}Gem~MWU6M(W`16PW);@IPnyzYHY#r^L-L$}XmbP-O zW?4@;EW2}=5648hIC@?`mT>;1Z;`?I^lT|AS;5J83@#9%eZM^m2-^d|R~A+x3iw6N zM+^@tL|H_V`GtuQ{xu1fU(5@}3^j2zP67z1enTYVSmu;YHJQ`-ELBrFQf@l;oai1f zhp-_3mmDDbrxzOy_`laZKTX*;*|JLy91R&k|3C#~)t8kyqALDlKWk|2IPBUdI(JXE zu4D<%*)=B?HTkpv_-f+nMdeQBOHUSyjTOmM1j{0F1O=zFscKmVm>ij)JHK`v0*X+7K6; znyUYOcFm5hJx3=f?id#bqMG3INvY-xbq4>{RO_Ay3NZj*{iHx}bu#|dC8Kt#9KL6& zIA{R9Yw$oQfZkTf!1~NoTgd;|t8V#N{~s`~=rlq~h$!fSiQL>-gGCckG3I1+hrggU zj7OfbCuDSNKcTz{V7_6UQAAq~oX7-46O#O>P!Xa36lTKpJTEzr|J=~mO2nmRntn@g zb<6t#ITtfBy@`LF(@D8z1^u=&!$&En*$@5N$R>N~4V5-`RKl*TJX+7Dr-xMi`Y&Zq zES#70=0o*b)o#j(ZPRhW%4O4O=IbQjm$LGbGTzRo)|ZzD-lAxZ=kw6YNQ;dOsy~hb z(p=DY)KzBNj9;2TG>0~w!z9B9wu4$Z9|uwo7tKE^Tk$yzfTn&5%}H`95Kj=$xO>uu zi7j)fY9Avk!wLLSQuapm#pK1sZUxOOTZ~RPj!(s0Q@(B9Rw{>}$yWy)=@)z4V=fke z{e)-xh-olJ%=HCIDh5>P>|081V1-80sZQmi>TQBkM^<^sRhy<)&#aHok9V1uPVhf; zEjdL66(u%$rpTp$|5#t=#kA}x@jvjyoz-zBj(Z9WD%R#hOe(N7b zL^qL$AC6i`InwKi^Gv_iJ{%bN1olew@8+xbm&fbR@l^}{CGssmHw>+_PF%CU{g$%* zI*iJBr`y<`_yq+BAD!UnCc6@u@DHM6I?(y*dU&p#LaCiP9;F-vqKz|)(zIkH+l*@> zJIf>5^-AQ%x~2f+IbUHv1-biieL|-5^PZ|cu~NE(kl$IGOg^eZrm(`cMwnp13Bq<4 zA_xEh@sCb~ixo&RFAxBH?Upz=ye%1^GyuNdRXY?9B)l4FFz{bWNN+vE5VHqKZVNCo z#OM^tvCSX;sP!wNL2a_pUlsaB>P@Fhrj49keXEb@Yx-6mYuDs6?mC(=eG9xlyj-tK zRFzx!!E)r#xPF59f0vk7c8r-uK+b@%^sQ7P>EUtXZQ~jO?+F~Je{f^CNA4^1|FHB5 z7qF7$7g?GIw)^JB)`pzmet96v#++1@R<#!X*Av+iddPI0jF8pLD**G z7hFEJ04oxC6^JCb08#efNPEP;zKHp+F9L>-6DpQlI9yg)r_smJ?mClU(yOkSd7+Fs zUf|7fn51wh$vMfA?com;zK%$twh+XeOYXZlkaU6=d*vP*FT%v|iMg;X6& zF;YC%$}2fd;qAAdPqLiHmUxm~KkJdiVs@P8)|nKvjJm<^4i6sy&%Mv~9}fm?c*c>Y ztF@<5=U}Fqr>m}lvuDn%k+rR78J&|Eg698;BLN&h z6f1w7&rtXMH~f$V3jf#F>|0daB&R#gE@i+5{GCgvSZb_inemcLGhBGx0DlmIS`n%l zoRCN+g4qfbW3rMg8eY}ho;i1&Jd9qVPydzO&x#Fd>W;Za{|D3YUrv^zyI`A|e3S9> z|0(P$z@q5d_KK8rOLv2GNhly4(%rQ*(p`ddNq0zhcP=g6Ei6h%hm`ccpuVr)fBkWG zuh}zaW_D&yJm-GyXJ_x!f+ck%jA>uUwX^=7eNCFy=G@`voS7HsP&QGcD7`Jp8h3xs zz(FpVjdz!A)E7;<(KrBT2D%avGZ8U`9M-D1he-&40WmC2?rMP<$y`I`8S6@nv|XZ% z`64E7&hj!6r!;B<+pTrs+A2DsU1G&ko*Z5aWs?o-kQ0qpB zl}r(Yn`(X&FsjP1F1r|AOR|dKNB;3KNbe)uq?81LL?ePe7(H0EPBD5e2%aS13IR5X z1sTr-79Vr>yIsUFrK<>Kw30+Y4A9Q{^&Spyv-?y!5TvbVDr%#@Fu7?~-hX1;&WL6Z zz5ETsg=@GPUD_og-RzsbQZWkc&>lGa?FLUQH_f8oi?aBe*>WoifvLvtaI=V@e(BX+ z(jgM!P@zY8m6yY$d?k;QzJqvff(1f)SAK2}KM9%ZFlGjC!uK9Ene^oSFAYqkcH4rg z$Zw`F4EE$P*y~~$U*a6rcp4O^pOs8~ZciooOg!4Yp0yVh1#7?PCKR6NM}j~sD2~Gz zBm3#C0SLQ>T%``0h&>DD+js$A3Ah@Lu(5)&$X^b_E>}$Y2Rt6m~z7;Z`>ylLA*oa&=m#B#d1@PVM zewk;X8S03eoTQ+i@BUPfrDh~c%979+b%X1DQ&3^N97ASAhx79 zJsSp#VQi+8XH3HaenO%Zd6~V6`+QXLL}E*kcGv~+8&a%4lMg_f1y8t!BNwmfWtnDy zSf(W`qOw#I*XafIgwfuMMoodwmgtm{_S!Cb&O%!fDZ37gT1A}6MVV~CF|i(*!r|BU z$Ow4lwv<3Y&V3Wm1<$ya?MD99)G`lR)dn~R4-e)nn5;Rk9 zacJ$w%a7D;yR#dqSciaRgPJESPB~wT!|FY$Pu}l&*uNVB*9#<~B%(Q#TuZCp3*C;+ z+=KM)P6i1MFZV{Cd(I^^9NMNifJFjo=^4W??fMe6C`l4AeGzHTJdO+FtOC#~JSBoo-*w09!v6kenbXJSj=D^*Jo?JI zx-hX@-^V?o2QiiSD(kXP%v1z7-Os_j^2s99i;htSe_ms$6 zGESR>uG3?4;=wvJ2G9u)}QL-mokfT*+>Sx=`W&h{Pxu*=@O52m)}khIYz0)HNO!=qeaSaCB=yA z71$B>N-FaCK*Urt%cXZ{>)Gyotxc5;yyc4UT2n3dC;fniCJayZ0ZnkVrlj~+0zI@W z!*}V8+m9I`uE=G9Dp3OwnBOhlCN1J9vHwJAzIyT{TQ&I0|)yoF`mub&!)pH{U<_ zc%(Z5J5Z06@@vtIukvqlnl)gi020X_7wE=muruXn$u5|PJ`<2vL z>TaHheFJvnN=6X1BX(xy%j9lmu5HzEzNhui$)7l3Z<2pO{NSHkx!+n-en86IrNWmk zM|OO8i$gPN;I3xj60gl?1Zcj9ap7_k0p?#20 zRoF}Uu(-o4jTMc-OiqrD*Pzed4FIsPLom57=nazKSLK}_2@SLWQutn_(o|-L9xu5w zXE~YLW#ny5M_!E*nkCUK^O?E$EL;V&*p%*UxO40q6Sk1;qOQjgOg*s+JMv*5skDW$ zMgF@xsM=|CY>GZV;gwZ%DhA%T05~bb{X}BKf5K>190QjXJ=Jfx%0ePJsXzC(37{T& zFnxn`x~LjSz^j~8nPi}|5IaFlMIF%Y%L1khfo)1XOYQdjOr|ac;IY|;uz8*(inW*H z0*8T@pI-@5?my!R?PgqE{0N0c9cM=t&oz6A78?_NGN3GD5}1dgMX6@UtsFQ_wX~aR z!@7s)HJZaA$LHskf<|LaNfJrih;*9Kb6WeW(SKRuHj&L?$lr)x(HWiJU{w**=fGFc zX{&ZK`#1GFzk^Y0FF{3U@>hq^o(Ch`E+R zC%P6F19?(P=Bj$xq3SX(uS2aB3IVNk^3}AeR%}YGYK zHLqJRL@sOmsWP1J*c4TH-Fz!DZc3^Hj9khJcI^sR3a@N`Z<<+a}7>di? z%zG2;L1R8oy~(EgT-c~AAu)3?Bg3x`Fv%#z zhFyPj$7JkN2uHF;!6XE=Rho%TFG{CInaMX^|oY zv&S8lZ5E^ofb8dl!!`0@MIO%*N>bZA#!clrrcO!LUYC_ps;GptR@mT2OBXBWUt{!K zsPDdLG*-ktbHi0+h<@QDyZK%_m8nfQlD*O)EweIEFc)TB$?e?@vC+HE_=1N{(RK}0 z$MsaVLpS*iD)^GeQ>{-oCS+OeR|Cf>O3^`-d;VV3scN~mM@l#3EltKc-LP6i9*9pj zY=BJ$zKDmo^-d7Cetc~1+_sG-!@b_y<0;ZbTASk{gL|9(yV{9LyUh4g?9Qp!K;GYSi+Fp0xevYw*c_e%Uf5@gQY^oQIqA8L; zR&DRAY6;+5S%ZWwV|5N;&Pv_EbCou0R)Nb6u2w#L2y_vRL@MZ0sfe;$(zGV~FHWw? zE^c?$;T~^93085yoJ-&g$)vhEBCRf{7NxDzj7_Va$%epL-1N)Fxhn~{Aswk4P!m(p>+IWPTsMYWIz>)6pfc{i^Tx;*a;-sT^ldw(}X`s_i2^QfG>B{0c1*9?dg z`EJ_HP>)j72?leGdD04~`Flq;X$*EJMJdV%v~^SYfFe+&mky?YGfz z{**}|Mh(Q7;~?M*GUbv?Y$=!>I&y-+&G5m6pt*;nfrHN4bVO~n@t5hO)Of3|*Ad+y z{wS|!NrimqEjzMMLak-b(tYo`=Cg(l>(w$m!XJVRKx!H4Oy^@8$5>KGB);ybZ|}xL zM~@~Ox-RdqhTy@Z+nN%R1RW7)1(r0@A|df)l|ZCl#)qZiG_j0ba?@zLMs+(EE$=AzIuUY|Nj7@CO1L=KurhNUz3Dp^M&q9CcY5M)_`43%9NGYuX(myhr zzeP7QS8b;;Bp$_b-GB5uU*e+Wg}y1yrPtxaFMIorg$tR(D4Kird(_|p$*DX2nF=~C z9gQyJk=kM0;X@_Vn@OviDN~g`szW~rSnbd{&SPJ2+ssGAf3vTFKPeNmON>ZORd?)q zJPx0fN<*&T3u$EY<1no4GG#PYf0FU{gA(gI*v^M^1^AA5^T;hJkcrsE$6*Pd++2fp zNN(LRx_PI9*e$CkJh;of#cu6I=N_Bv6o>3kHRDiq?X0HdFTB6T6S>AyzS~^B3k4uY z_r7zyt0rZvRvtpNNiaz+@DG;C)(V`=Q!-moo1YHCO#4hp4OSa#MfxG8O#lNcPO7lQ z)`>dL*f`h<5oWY;u#Rq42PpDK9aTmrgt3Nwr7cI7gX*FDjAh}j20P5?niGY97Kx4% zH6V~zr`8RF+x`Kn^Fj1fa^g{AbRhmGeI1rlICV|0#V+xdfoltUe<%?|8}~8Y-9ILh z7n-$ZMZZz8FW0@xiMkrWaI&o@x`GNP`}N%P*di$Fy8AgQKwV{?!62qeJK`p$Ara^#sh^euKpZ7xMco0Km~1tBWcn$^RN8Xkim%w@#SiADC;- zf)4|I!`MLPjBi$ZYdVoK3uMmcXHFJ#UpM96DQYq#g)o|FkK);P9~{0=^k5Xpl*<=S zr+~-SxsU$CO?1cn=luUe=KtRM`+%&5|KZ4givI7de|Ed-Qxh$&v8}v!wL({rG=oSF zK;#~Xb6&9vWmoki8O?kg$~u%R`vX>gvYz8;n_RWHhez{*A>`y^`t2*xPyb#Q{Oz#> z;QuZt{Jlq^?7u)S_xkzh>1*V>OtU|=;!E^<_un5F0DunD3-tde%!2)ARe$BZX&Of4 zgYh2RjlN@l@@b%sglF`!-;<$Eb)api+x0|3LVf#T|Hlg|d$@?%;vuyvXVj~@Srya% zI)r-F+$RA4G!iR@Po)?;n7I61sh?@zK{&AUsD}_yQquaMv8;lO?1`AQWwXf+@79j2 zOUe_6+r`q)$c#(<4jgmcZZU_mkjZYcmx_&G(9Br8sI*U&!>+mL2a3c{8tbldbd7yb z%FvSV3$3P!#eM57ysfpD3x}xrbKa`rYI0!EIIaBC2bomYx%5nZj0g4ccUC5tnH+UR z$-N`iHU{p}i1ilF-D^H`_f974NqM!(49S+c_6>ejX>K;FCm3JRtG_4U%c4;4FX8R% zc5RKYJe|8RyKTnzS4hp5wYnmA4o3_UKre)(830A`23OK8l zk9;%QSlIOiFEe$xj6aghykv-$UL?z`e;F}|P_lhhw#CREw_HONET|*s2^H?#?#*i{V(}{sfGV9HxqijJ2wjotTqmSC&yqzC7gnUk4{-oBJoDDy)v|QY^ z%W??D*=g%YbH`|`*JFmT7J?+?9Xx5T9#HSHbIc> z>M|sUXB?1B7Jd0|!{-w5H%R*v)(80hL&7~|D5dG-0da}|K|0V+J^~II6V3HiX&#?$ z&6?FgZcSeasRLPqX-|DS0R_l+0{{{awlw$Qek89f-yn;1Lu;W75GDvvx$y7%kg^gn z8656nk7Ie~v0Iid3JL5dhT8MKlGx=02D-T!8Qebs+2VX9Igmak5qo zL;Zt~ydC$ZDbN_zgR^_VhVb}njYC;Vmd(;@-ZAx;Fr;BZ56-rFoqvjKO0CETWj`iL zMSZiR%C@hRtyxC;eHac`jPoQ5vU4n4b1*+pMDkZrp5a0TbfQaO;rsV~WgyZ!;e;A3 z7Zt4f(WC_`%3(>$A%h{6p#4xV{ptpwN9*(Y?6Eljzf95i(g2is!;Z6``GnRj zxFpMruJbn$cM_Le(8W(mV3tx}`M~ES|KW4~lZHFv=jfeZ6f|Ru#cF{vEJ?x`DY>q*g}klZ-D2t_%*c#$)nv!Vn0`!F`b4kT{D|M zEqEa(T6#oy08>xga5xLg3lj~JtTU!n#!v4woy9S5s$(6-It*N)i#f}p7mxelL(G}N=Nv{9c^HU&%vT8JTZbb zIh+;pYw|#9?+m=G6N%3=40~Kg^8QRuTMX17J^G>^!rQwwVvCJ1ldy7dE*=-J`f&ZJ*KYf8;Zk29!TNXVv*052dDJ1K{HEOG@s!ot z{H9+d5!bwm0R@N5V;M)Pycs+a#Q%+6HHa%UY~z($+bcVFJZ5?hZz_+IhOJ6 zon6X$RUQ$?AIMT{@${0nnSRIKsl`rw$CxrxXv(5IY&>Ur?yY`q2#-0%rR^!mB#Rqk z%Iy6^CvTR?hOnkzGNN8%PizsIj)ElrveD*C`Lc`kvXhU@d+hzDs3wT?0u?xHHNtaF zG@RyRoN#ttji_5t;)7V>B&xkBU*zGrM~lNqpM z4}X^D6GWUn>_a>_Fwm_G(@EG9%rC65Kuq_AD=NEQBZ_wkd}P+ez%J1ISt_>wXbjsa zCA(x2>-Eqc3fdQSY?dDYm$T=@!`incLF!r~KjiT&5d?-;QtXGthf#?QtY0LG(`#vs z_U*;i3A~AFzFKl3B$i#MzaZA>bvBPu06W*qs_>Pc9J&Y+Bf501_M~@NRB4YgF&8<^ zN9eB+ge37>?Gm5CbBj~^7{k$YGQLh7gF7^{8it3fx0@W3Gtfsoii{h4&;Lb|%{4%v z(>RPe7Pfwm*8EIhdcDXufV~ZD_0=hV`eV?6CqiZ+t+by>%{ow8=d0BsZNHclw~tqY zfC=1|Uk!GF%k)gJ$&5r)Ci7y$SV`W3W7o;d$>ouafEITca3(GDwJtde>Cr}EuxbhJN#Uw^fZ?{f9x%Fiq3lS7=&tthj(|#! zCZjj7*OW1|+bXE}`vMlNc^AyB%s3`>z_TWvPjHSZ>TkC8tdL4s=#qT9g{>n(BW!}u zV=3gfpH<^h8I^v?i=7s8?kn=wR*x~!&K>hb=8%XWjw=lEiI31X6r~(RYN1I5p#{$t zUzs_76K6rj)K^1>&CbZ+6eoNMm7h*T6TYH_nu@U)rOr#A7=3OjaC)yl;2`@yA9W3jyW$&$qSUUgFaZK{Y@&%}4q zCMQ67Wun$Pp>|_{!NvfwY%fq^7`exmot#|OQx7TJuQk{pRBX@O#7`A>+*EVLc2P#5s7oP2pZKXaz1#3Z>0mlQ>M~lT_3oU8A?h_}Tld-R7UVzh)$l!b% z4m5wLHzLB)9AK%2nK*UC054R(R}yso__-p{vu>buQ7GPs*PL`CLYS$-GBmcSHu|1} zacXlVBG!Aw)9R^~uS>2nN?M35jbw4U+B~LAy_vlnzEH>>?TnLKVXDWjnCKJX-t+4b zGSIG!^f6F5h|aT^W8;j8F*TLYDSvd_hHKfgoRENS`EGT$Wxq%SoiWyK(n6;$S6S8Y zRpyc|c;<8Ykb+@8$7$_mJVzV#jaxbLac@5;3y|m!C{89V;=u{y9HEQ+U`Uv^P7K zp`~qUq+F4>S_hn}V5MJ1_UlFBMpLBi8LsuBEV)(7L~qR1ZcXowQM}U+GAbl za9cj2xHk+`xEa_LCKmn_jjz8r5Oz|U%_ozSdDC=N>RdB~1oIFuLnmy#rW0c?JTpyy zn(0)bMhN6X1_tI1C1%8g@@1vt+7VG>+=yiHESVxtSXv{I+B*2P*^)|x%hV!c8EOh;kcN{>b%O-$?CayC;+`%DmP;|wWJ|~O zq@|)Qeq%s$hdd(MAd>#=6H%+cXQ{9pspzoC5`R`tX49#93_a&j*o_<@$9pu}Rrqmi z-h%)n)%}UV(W!0u z{8>IFsu23Kg-?I=%SY|$J3LEI!~;8o0#1*vBx z@6+JQBvo^paig6G0k<{vEG}*wI$de)83znMiN+0gl0K6tz;T74zR2II@`VMt1`2Dp zM5PXoN2q)FMEZ-K@q8krMO=^XQ^MtG+~G>gmW8GNN+J_E{TXQnsEi$2;Vw*7YYQho z;j7?7b}U4$R7Rna7F=u?c=k3=F-^ZMzcw73SNT!}rF#vtWYXk|2F<3#@N(BTrFP+% z$_CJaAt+~Ago7neWG6Fv5{j2pu1ixn*6F#(Y@d`%zGn_j`^O$EHjP;sN%6si4;LOS zp*rMON<5jJmR`>7qp3xk=rQlf%3r;;%Os?8h%lOOZ5mDw7?%=hMlz(f=lP@z&1d_Z zKQ2HVr&>o`NlKjA*(xoSa;WPgl3S@dV2YPa_ePLY?v>?oOW01(UFWS2DF|`8B7vI+l76+B}?E@_>hckRDm#p_}Qk%h>2eo+~#I^AzGYY6TOAWzTh*g%8=qL0IRYWdb zp(<bw)!*6srocFifTm@(Mv7G(oiXtbabN{%#DmR zWQOUeMmywmm)Rz!+GWuZLd{z8Dt8xJ=K6A`>*BSGuo5k7=?YbA`}BMFNj>M5s=j$n zHEmgCPb1A*i>dPNR<0kr2X(kPrpwHBsfu(J_2)Y;W;(6__BBY`Y<%*naJTO&yaJI3 zwoP;T!V@X9Z2E_$cHbX|rd5HT)NBsEYZ8Vr3&n;6(#40hY_X;c<#yvtMcq&7r5Jb@ zgLk$}K&D(faBt|(Y@D2l>8l91Az?FjOrjbBfv8^2tdgaBl1ix`TVp6o}pX~3C0cvjxmj16{6Z)n4EDA8 z_qaahVhK|&mczS0?HF@sGO965BZ}bW(_3Ea^NAi&lKIkcM-U@QRrOQCPgyEnL_P*8 zPdyv4WLU@MU7G!Un~hpvsfOkL^I%9rc^DD)9}88j9&~Vb8zi^|XJ&AM3;{Cu z^1Sb!-E;QS?%uBMFJ0ZIey8eI)xGt9{QbBEAkt7#R{@}*0RU)E7vS*$Af=?Ete~f@ zBgtrQ!{g{?YtQH-z{}70+S}8?#hy`JLzPkEy|OB!k2pUEH>11vCnpClKSmiw34WZ% zRe%xz{n@kslqbV@D$id$e~y9i90v;v^93Fb9v&_ZE-pR+2@yU4F##?v5jhd@OHwj2 zGCV>GN^(+45>hhK|NI0E{plNw=h)AmW0T_J;*A1*Wi z`hR0RJ^vf*|KK8i;(CUGfsTRoA1<_KzE3wgF~;*(0+=KUAFw{Tzhn{&dqJw0T2R-A z%`BvIMrP$PgG0_D3}Z$92kpPe{_nuT|G&uo7uf&iS^?mpqdj#VIx#>FaK5cj>+$>@ zyPo)(Dw(-iF0JRyf|kbVpG6(pUi)hW*~`BIiR&|mo0^3`(vN)wHPmU6>1w6Csx__L zWSv4fr50sk5_qQv=+1y9yo7i?$(FM%4lasa6EBLTG11S;#@*@?cs0GyrS+BDj8t|x z<>?Yxt5$O++(&PVZVT&84t3|0R5pogd|h0sJQs^!*{rsqeVVQ0j%P~nJk_>n9mN1& znAWOyjXSpO585|->J%_uV#x&PyGmE=#PnWXlu=~=nO;+;QFRu5>oP$9@!{+=dGF%L zJtBZ)J8L^(g4vmY(~DQ)u42&i13;c%M#iz-jCZ0JIVU5R_cjf4SbiIsYGSg&@07>3 z($*(Vho!4~;2& zT({%c1WPmuQvNC-bu%KscV9_ND))MKMlUeK zx9`8e^=;aJ`vP}gk!cd9+q}6BnTK7Oo$wty@G=3o#>BfPi;E$XSd%0OKz1ZceE;pv9{w?{DQ1>u>aKa- zXgCC;FJv0$BL7PW{oV6ozU4;%j51%8ohY5V(0EN4rnYqBfaRX=2up zoF%|W60*sjKigg~zBW2~B^xyV2(ZyoS$!9J(0<&szEa8-w@V&t+_UW7nlykB%v|CJ zjeEdF<@Z-}w9o7Xcv0QH#oZlBHiKwG$*y)D%I!<~Z9CheX8BGh7!_`oa#2NBF=x@y zMHxz7D<%C&%q41rF0333WM(GyygtvQs0ZhTbGWXncU!}3Kn(sQSXdZ_ zRd=SSQ6S>huBCZe_?{ynXX4ju`~mAa_ar%-bF8$_qnvQ%3-^0vZ9!Agg-cp2VV@hl$8&9S#S6Te{0M?ySNwzmP4KjJpw^pT#$c@LIXU)+hQm~RIzM<6eN zb)yJIdl9@Abzx`Px~kOFC>20BSAmtO!Ot4cu{amxOnz}U5h@;bN@v8n8(nGY$X|=~ zPv@EHyf@j*b!bx4wE%gZCv0`T`>eFw6P9xs^k7@m+%Gg;Dt}kG-1+?01+>>!O;?uC zXo*tirbs@<7q@yZqw(*qhr0Dv;*NKtOt`B_Hd{97zA{kl>)gu7s9t&wWteKzi2%xxF z*Bm%pTyf3&t{R z|9|D><~rG&^Go=Oy2-)|IkKEWu85WS*9N#8K6ij%HhC6r^$ z*T`99{?*jdb_t5&NpKR>P2Z%Q?_bDflCZIK+Pj{g)_&PGzyYPD{)(3nHuY&XLab2Y z7&tIslx&k!k2YQZ2(TP3nW?Vv()9jf+F6%`x!jT>L+~d=a)T@3mLMaFM1oE-SopXk z{M4iUunrJ7pY`%#&7R>aysFPXEn!GftX86w$S7r?nIm;YWhzUV2aHe7F|L>&d-pz& zhb;1PJ*6k?q$vJR^;m~M{|OaMsPyq9VPFzhQxj5Z(-G)dwL0qO^v>jUE(d$ddwx0U z^Qkx@gx2(o(avRn&Gv2jJy-K8Qz%&Up}lQ2lTFB8;Z8(JnDhTvoeQ!P1zDI}z(OoRs-% zfXGGcFsVQlj9r`qC`de_S+%R*rIot5T^}$0tl4`67^zL26+n*+J}K){QeIFF>6~!I zvZVq&eEl@3XdnKh=8r+-$tbkeV zyOVzv*PVZ0!6;p~ZnRvcpHCBPLUX`ZWmR|XMEEwvFYbgqv4`zh`CNwYo6>}TDY*bF zIHyd)#M&dEQ;<`kx9>>%`6$dS=w`lP?K9qb*o|)IP<#VA_#)C$PE4l0Q-?du^yFB! zb@!)oD{nES4TgOiy!ns!c%@ZU%X1tA%{#6PXEh~H&apzu83hs{!W#O!|(xFQ&y1)>=-@4NpShDxE3rDFLGD&Zg)*4V2PP9q58L#K4|Ke znW>R+k*7xfyu^SVrTMga^E})Avc#A^%3xA6$4~oAw#%X0eX2&)F>&}x`lF8wT1D%S zkWl$`l4^;f#}_gw&wYmn3&1F5Zfz;SmTFsh>j<^)u+a&9QG}XV3uUitr$KXO;rh|b z_#nwi;As1`5Aqw~_H%#wuv;%+ey^^2-RI=j0_2k(Y8M4R ztlYsb_P*Z$74Ku4RuduRM?NN?lN7GE181*Pk2K*SzZ=Zz4T5M48y8XtFFng!()T57 z??x)NGrz3)3ypN+JE)uv>Q1y^lSeETJ zrVlsWOKtD98Qxj(qesAD*+7AbX#tZ0{mGAyahiTPKDHQW|5WBs;7eOu9o1iLe@x20 zbxmLR3nifPAtor?0bqqD*P5aWIi5V)M>TeK_OkJtC9uOIfbK#n!KLG~`U4fr{U))$ zongg*Ogk!o>3vMgjKB(}P!uEsBYrz$H9gJO(B|{p&$hI>$-UP?M_N+wq{N>;jV`&2 zjV_*f<6nys9+>0TbU8B-fTT|>RCfir;7%W(sdz}~uLfxZaOfZNvVY-~{Sz7T-a(pk zf3V^aAShdg%B(bAOUCO@AGUp1ZuZED+A8vSCnFi`y`QSWB0r`=SOTJm?9_yHxrp1& z+Y|J9q%)?;V_6ZX(<2tUdit(X*AIy9Gg$(dA8I_=dJSsSPvgZsOHje=B zS}q6YBBwv@;LiC`m(+y%?T~s05*t-lX-glS%~L*<-q;-1$5Pb_u$J?XYzCe6cf^S2 z9dFrP_ts-h1N;Cm<52DYve*Nm$ zYyXQz?@bGD$Z5s-@N8Vr?7_p)M_mkx4-w5;TS*C67el(`VkLb>wK1uo6fXe1#|BI! zYKK3_*~88kJw`o=*vF^Jb+yvpw<+Ue(AZ!-n7GMlgr9udIPAf_GcIyC4hewWtE@~I zx_>t{)cbI#PDwV_$hb|ST~`f38CfvSH)hs%E}pWsa5YsY&NYU-eQ7LZ%GwgHrQ0H} zjGMd2mmk6e*K-Ss6ZQF*n(2M>O=a(yh_$Hb#_??vDq$jFr4y*c;E;A zczD(QZEV1)4pEEoy#rY3$ig{ccM{S`usJ0>pX)6T|8_ZC_cTcUW>fnlezMC$SF2H8CVN)#?GK?4ntC3Rzf!8SO72)TZGm9-86xzCP%V+MFvo21B)GIqz0s za~%Y({qhjZnW0(fDCVD0RGh6VToN1^TP4;+u)NN6i{o=f;ESjuJG7uf{u=N2n(Lg87`b#T_2q)loOygM- z>J+2)j4cwkP13WH!&{0Al)NwUCPXtEUX;x6Fd_f-J!r!OTL$#-@}yPv@^pjlzF3wD z)uD51P*UPZt}Cu9ZD1uaL#PyP{Ek2>E?R5I#g=2$Kz4GzzEZQyQl+et#!7Wp$Xt&e zH}Quto4Rvm)YcODxylO}0%LiIH%b?|dF5W4NZcd@446(-9R83zgig5FoiN=ka3{9J zli{!-Fv}W~@EmxSA4MCtLDXHHvYoH=R~H`L8xzceGQNn{vKpJT%VZ@fH@=ty>0&lg$tgFzFZhA$$bapEa#e^|0^!&8q^=GS&Ke>B?nN|yZzK~iY2)O>UHe%ig3w<|mX3ar}l zRJXK~g=hOM@iyA%-o9pi1WbEcmIY&?BoMBYZ&Qo2t(g3LTb+x}7??Hepiaw{ua{#y4isFWh_-4=m$#k&nBWq3vR>r8PI-ZGc?l_y zB{Vm83?UTGDR)ZzYU^cWP4ynVc4Qm8esru&8~n>35+OVasrKdyvJj8*(JUML@*vQF zI8+8TO~qCTXYpI^tKpoP;C8h>0z&Hv_l)!$9GKtSk_VCm)8ycTIW0+ej=ZzA&JBd` zXl)k35SZncf8-w8C8#C+mT#)#K;II-q>a_g@CUnEYouHRL;;_?!0BN|mFW?HfaK2> zb~K!=pXWCmI&@yBfAo^8KwOsV#!qWCq_-L|x2AsLw#NA5YhIi*n2^gbIIqd5JHAKP z@IY4S3quVDmZ?}?QP*3_59U!BdG!gy+e>*r`WPy*#!0*jKJf_`>OEa)SP?alL-M`O z>--j4{br^{8^`mK>Ot>s;pR~8U@`q&GP0WFBxYB$JX7X=II(!+kVxc0afKpx_{H<) za`91#J+eImhNc>@rR~DRw)oLi&c{bUMV4Gi5Ox9Zfnm-oK8FX`yLhwK@Mb2ccvBm< zMuDal+d^&~GD%3j(wqkmvKx727qx3tOYy^i`5I?Mcx%bZtiv%Vw)A0ogLWZ~@R){z z{q~oY^&=qYxCWwyyh^X&-T!8Th5F&ncsCU6Hz1bHL`E|#=_N9YBHy(^8l=r=)U~e< z_JQT&WIyHBi67|f^Cmc&&-}ZSoXnKy%O7J;5U@X=Z!;04eDIElIMHp8QMg0ruEQic z)tydKwp~Z@ewICyOr4{mtdY`K(tz%oI8qQ1xp#9AvM*IT4IuJ~)D}wfOOsUS)qD=$}mx|G+e}5HD8EU)MSuvWM1d@SjY^7$nyE*Jk+%u7;eK5 zJfzMYG+2SP6L&^P+8jYp(xTAO)n3nBeaG=K0=#M-hUg*d8^;6}Yuay#BpQo9lT%=zn52BiKpLnyxQjVh{FoKAK&=m>njLLSjmE1nd?-_Xn08@?E zW)JX_cEy>EW{MIiLVtvE<`*$^;;RY2 zuMBqkrEMMeah@ePv}!XOHD{_{tNZKv3MZt$Kyoo3{4R)~-@$xUF})5+>qtYR>VzQc zg5?u$fOOz~Ras1bU*oYUt`P(f8v||p7*8Jbt)%Rsl zN-oiGPA-)Jf(O_3J7tjR&|q@ghY6%bC5H8baeNua)8-&OdA5s2`MyDx^`N3s6oP+~ zv?N-WejSN*2_XZy2_?yJHg3PcW~VH&NZ<`(>uG@`F`7e4_gg+D5VPHPh!eMORR`)qE-4si6;AvN}e=u z?G^rAFT@EYgRqQfrf^`7!7N8lid!7`bd%u`qH(?TV)M!8#3Pt{qzwXdYp!tfOX%rQ znP-aUbGDIJv7{~%rt#)q?gNK*TL}}u1gH>rmq<;g5S$Ntz$;&%$(X0WS9S=e?uHuW zU_9+Ms4ulXMKw+ml&mo`{>`Mn?Se488y^)tOue}*yc#ndbq!wGX%0IKw&L0kVz^Rl zPg=u$1d#taQGK0N+d5UMfo_=`EQ~B|C1mM3Z zw?*LVkkOI3G!pFTQ6ond#y^8M7$|Faz#FcV7H0AYeA*LMU6d^Fm7FtS zoVrsp^2i-Zw8d%M-}6(WoZ~)=Pib02dzowCJ^4!`;G)(e;N3r>Uqxq5)H^3~b}}!u z+x|KSDwQvz&6_#=TC-mO)x0~s`i6?Hpq;7`O5YC+;nt@~PLLJ)zM*crz}g?a=~s7m zs9U~PO;~sW@j_)||74_-^?4dF1y<#;OkW=kcNpZ>KLU5{Q44TG9oDWaWtq-BhQ-2q zRHnCQ0Ie(5uf6o1M?keISMyJ`8_!?pr)w;ZGHPUet5pRIGIwVFYH}qY7FqVzOw({0 zW^`!=i%pIC?@g=r7<|(Is!Z!DE`&lHNLz9S{cVq{3T*>E(PUm4xbF9N#s`mDQex1n zewwNduxWW|9?F_T6@4!(|NHJRb2!%Ce`42Y75y-U^AYgMoVeyGtLoOla4Q8WU46UM z^ui@|*SNY*Ss4RQED$W2Jly3vY#}q}#lycoUfNI@SjW{peVh=MQbvdGN1g*a(xTs+ zYKR~Cnplzoj;4Ch=WQtNhB&2mPDDB-4ls*DHHJMCN_^`IAB+5ICRvh#EMt+|Wl2;w6KiM1bA@>R6% z_iHYtGcOaLCg4vqr?y+E=yQSdRywo^f3ZYaM#l&7s2AQ6Rv?}ozmHK|p9|EKIC-a; zl}Db59Hr;pER1>0TfSByaTfJ@iJIJ9D%s`(`*T!y+{tISErG67(eJpX-Y`2Jmwv5F z?W-{@(qygsZGNXL53}oWYugjNx;t#z9OS`K%OC4u?HjgDm4EPtgJo=~t2pj|H$ZQP z+jhK(>(g)$(ob12;dJ${ZCy@!iCN+rDa8ZTusLH@YAUc1%11va1dAvlv!l? zh(?{mt9|dD6v?$rIc2RLO#NmIEB!{71!eg%WSlmb?BIb9xfB`9u(4_a%&xW<84Py% z^wCeSZ8-RSz!FDk@bo4D1pnk1WFmC}NP> zeL79*Z{cc(=45Uzmxj*K$cl@-Zrxfwc_bykY z*x=%z^FrxIxlXd4uT|xpBIRWpb*+{m+zS%IIj!CIjg^;5=Hz_3jSZ-AILPurj^tfP z66?cPNW}v&4|H$}OQ~uI-IU~*SVby~Yd#`uC{UXqIwam+wEo(od0xst_u>jNpZt3O zW_yxz#-Q{tU;BOd+_yFJM_Wu$sGVDJxX@Ur>QAPI?*^>v%u{x#fZ!f)UqUkj{eR)! zEC#a1B3Ez0Xj(mo&>dsLO~N8h)hs#Ay*dh~yKkt1;wi!;5u-K05$|07N3N8#VLduI zG)sj*r?cT;K2-Pu1)l55D`;StdGxW|nbJT$fwT#?Hb@fX{V4GehLZqkY^Lukpv_{+(U&_A^RXGnQ>%FwM zhO=P7v!%$Fz2r+KZ0SIIU-}Zd#)iQkLJud3K3nH)fV~5YcS1rpO^$kgG^py0wdS5i%V9bh8DjykA z`}E%imctADD1>1Z3!VuP<@Vqt{cRSvykK{Ru1-n}frS(3JJ6n<|Im%Qo8YS@S{5_rfZV=+|i%@1p&;I8|aH!-}? zcAOlzRJ;=WE1y{SzCm^C-se~HS5cPO9wr5{&K|7lK#B5|b$kOjl4r-*M5GQKq=QZM zSLy=Na@yk%f)30Qbz5`fDsj|RN#NlfUb=4goPNNpUNGfRclVK#lx0TQ!!2S9Y5ca9 zj!BI5X^zu1cZhQ@>{g(m?o$xQ%SS-i26g9br%WZtUq*9K$P6eMG)V05&$GO)6|6>O zopL-pmG{A8S%=_-jH>gV*$V45;&>?U7X=5-UEPo46YPZ3&;*ga-x3hN>&WeOccfD1FHzAI`SYAq}tU4XtwxAbl!pf0F_{tzol)O>0i(MUIn$N zCT^4e6C#bQUe9E)@s=Pc2)hm)89{5qD!Z?g`z-hJ^8;g3{n^6y;tJPqGgib9!*~Eq zbAJT0oVU@|C7!qh*R7xvbBv-Z%HWrR%%Ss2NtK^igM;q6?JqhFjrw? z*=ry1&*C3kJWTTK?s)G@vWG2jx^8iKv)RA=fkm5|HqsPi?^$l}gJ zZ08kCH?PSXH0j1-$v-f<;j|G_Zu?jtnWx|JppuJFLh{`+`Kpat2J7AoY<}#5%Z)|u zkr|NZ9%vQ-<`NGp112p=9@f3J{eDT(wi11*Zs=SvSSwAgbz355;1k z=bZX;T7{~FMJ6UD)mZ~~&DDGK@@Cbd*KhZ6bHM~|m1%~d6gieHxFB|T*`L!ICBG_@ zp`g-nx`(`%^|4^@krx%e2&jxRQs*T4hJY$nYdY0ErnKG&?UmqDJHM$ve_;&a3`E^5 zsOX_~y;^hQU-9?(#hOmThij4J-!h(c9!sf-A1^&?e*`3NHk(KHeeM(~Ls|A4n(XY> zB(i7)&^Et+LCU+RxQ}}>icUE^mx~1BK432+2nj(#ece3s?9D*KG!T!~2RsLs9J1d( zVZJ1@Xitaegm3?DKLQ#Fp=~{mgh@B_kv~7Dfv~jCZncTpI`*%tZyh*;Y7c0EZd)qhnxndF7_2(aj#(tW_t2a)aq`g?~ z659#x>4%n-kATmcFYaE&52)%!e3-XI0{-&6n&TO)-r1r%P6J@M^-jJcEPy26Ih^ec z+7GwCE8Fsowmh3(G*~oPBL6krKxuuy?a@KltS84Wnj>FGu6*SMGY~E_iZ99;QDb?C5F*=_?TKs1cwZ*aM_QmTor1$(Z@qwXv zsc_Nzh>q=H)o8vcwzRn`?Xv~Tm%4sw$DyJ?=Rp2#QH5Tn&dp&4V!Uqr*}GTeA`g`T z?h@@yU=Nm7g?~m2`Dnog%e)!z|UWpFdMD(t(-Vy zKk8Jj;nB;Fh#)uG&TjS3=z14iT#sEYXBb~+?Hv;j$7ZrpslLUuLr)N37e^RPmrI&m zb*w>&B|+Ru0}Fv&BAYjVnP!5i9a2y^?`Jzzt4t3(p`Wheb=Xx%)Osn0Rg^d|>>Z|D z&9s0p(=L4CrW|K&b)8UX^RH76H=;Yl$=~1p9+FF+Y+;*Jpw&Sz!(xqBU(G=l$4d*S zvs*Kp-rsFIn%~F{JoLYbP_7|@(BU9Wl33g%g7H2JJ%WR zuI_{ryky{&1WQ0NL!4Wk0%Z_fJh&=zRev%Y7urb%fV3VO!>Uxqq?l`>c-94U4QAj; z!MySYjd#bM%y7%39>NQ4%s}nV={qivH@=XQ3c25+0p6HkNhmr*PEkU}kq`rg^9Xo@ zOyEBh81A;={*b2cZJjwX!VU%&ARhs-@UxKUH~y1}_He0jsBN@Kdz0QbrQxbUGI961 z6t=sq!0|>f*B#ke?o%Y~ElRwp{6mReRNLomYDb6{$ESxWjg63w=ksL(v_hL(uFS8mi#(k z!M-w$e(Trq{=$wWtwS$tN?iCuHdDjz{tvVF+Nn=V-ty_t%JUpcReSpKGHxJ8d)+q{ z+9Y|i6Y)zxa&XhdX0$BT<~}{?G>^B8=$>}FWFUc6dt_nod(UvRqwhB>|6oy>XJVQG z=DI`v-b2k5;cANx$_>?Hr03zPtsKfLztmSSu{^j!kp(_ayi$-jyfCNVeRc3$ zvx!+D%NM5S%hpw7gi?lg@k*)doN9IpMadebvb+JvA9ck}21yxG!5Jc`=Rlb@$jO=w z@>xLipR24B6Rw3GZ(Mlg=WH7X=l@eu z__5b>F4;;>GW-Op)-*r-jpq;XW+s92cxa#J{7taP0X+*cuLdmQz}~WO0-Ij=mPwPVeRxbMB}0@&;jMm4Ns0WnZ;6(H-AxCl)+UY<$sDXHK+wFkR_IN)2UTkpoov zl9}$F>$mq-KBpanenqyF30*tLtnf8AMdfTEO z=KznuTeD|LNJtUdO!&TzoQy47Mp5&G^z*N6nwh0unewQp6{Xqow15B+VHex}cAb*} znwfMT1{TtG)y^@`#<9%Gr5XGU&Sp-&{j2HdbvwKnvrM!arg32O)=MMMAHYkUKQD`z z@YaI$61=!8A8oX*{xfr6Nq_o48HhK-*s;XpV%yQd>RiopLNk$(RF>6?E(3&}#%wk+ zf#Q&c1K&OOMcrDn*fP84n0geKKLRL^BQ9(HJ>ck91U0}P!poVZ9g1Gz^mx4#Zi8+; z4@@q*FTA;u@^-^*eQyRcWG}y}sS=@V~TQMb+tnto-bsJW}hV-hQTWu7U-~lD0a;`@Aji4vF6zg`P4{fG#o}D1~A#-k1oQ0dA*u=da0mQ#YY6rY+9R#87xG8{?wpT(? zA5F{O@#C|{MP6Gq+6VGymaDFaRWP@lX~#EZ)d#+D{!SVPH%?=J2|)KG*PfVS#z0RI z6v%f)u@*UiU!Fyk6c6*j+Wqcszw(T%q?6(FqV;YOU`pC|1qY+7;r6lG@jR9Wi>Eeo z77F-F8MiOqp1T1s?0@!+T-G4qstHZWIO`~FW0dtzODM2XWUwV`N3+>irFfhMy=Bjs zUt1yJqd{7t%1XL{cnK#jwt}on*}il)ePjN+IL~KoHqT>O_L8eTmQwo4l!f_+Xx&FV zj*`k2+w#P&BIo`%?bRLAz5LO3YNMH8?8wf@OXvSz-jaX6G^hTlrdz@4I9FhXk> z56ae(WkNRRR~sKKF13gazCCv8r{Meo`HJprMv?%)k~Y+TF)=p#W}%D>w={?qZb^Nx zK|Dv#l_!c&O5(6FYGJxoXT({JNI?5nmvl(+9e(Eorbf0ze_TX#w#+x7s>w$zSW4G- zTjcuhQrw_XTU(jOW)ok)OikWF4iviKZ;R8+8ZmyM9YU=0YP>tQz?1rJ$Cxcwc1N-Kck&Q4dR`+>#v*7x?iDA`@7kk#rl0( zpzkPC*ZPE?d1?UaHa z3(9bcr_uOb-0o{<`;taS3Dh1$QLO>!bE)s2=l5EWHg*KW1j-bztkVagoSBo9&W~@$ z&?aY9Jg1j>WeHHR6_B_RR1QK*`&}m(4qsVS?XGQ{=<4UTt^YwcRuHz{u?veX@i!a{ zNsyy>np}a`(mZU@w!YLZ^!kGy+AX)EG_pq9$uJS4l?umv<{qVdo_gv+j%x?Nr0$EZ%x63wel zm@%mceWbp)kM{y7*rlnF`BBuii(PQB@YmyCxh+EY7^KKj#VN5EgumoFEs;@Rw!vg= z+8ta6EC+9JYCg;iWZirBAPO#AlY_AiR$BT44!mfN83rlDR=!9^#oIfnrR`)y$nG#Z zicVb1t4XJV!=wAxSp%8NCO(nHnl`Cqy?85;#=a8^;9ztIe}(+$u}#KC>Go~l5UbOL zqv9@?cj9%FSE)(#bz}w{cjVbWkFx9~7?wG1KKIHe)xP?sX(G)pEar^2&8@apHHN+O zj=)S*FUc40`_7>1YLEB6_a_0J{`FCd{ku6-&#jSTKI!F8K0fYa#e+q7{}@l$8=tK- zvR0ZT^{IUTq`JpidnID%LGIQkJ!ngd?*sqJTj4GT;gKJ!ZYff#FEQRznSP{+dclI8 zi@aX&hSB8Xq0WyqyHgeBPFb72W{w!Dz6M>499tR%ip`z6R9#Hri^bW2c^t*UM;i@A zXN)HGVpddEaPbGpv2Hu<3)(aFrBs+oqP-CuqhB8Z*cXW*aE3wg%T4;bgEkCw>AcexGxJpPa1Y06ZKly@N@HZ zAM`9N`~_J-&@H!I-xRvTxkGzZwn@g@!V@YXL354GBi5Lt&^neVs704hChV9Gx`}&d z?F^UE)=px;9qnFrDa_1M`qjbY=Z2kd;;qBNNt*i%#P<4~&Y*^lzf+<+aS=DSwN}7V z53Ky6G}E`}4ht3FTC7yQc356uGVZZ-lQp_~p3#^%BR?|IOIK$pNSGvfdQ1j6uz1xl z?R3I(vAU39+BJ)9lfZG?cUJ7nh_~&2U~;n*&(qBf)G9f6kDF7D)(Cv@b%;A|+4$AR zG$If!*pQKaVPpzccLr4Ls9SLQMB?Z0d~hVu#s0Ox2rN7IVub2wwoAI-`?aD#0nNWd zciYDO#hj~#hRZqxLt|NgkaeG|gt_D4E~T@G#BSC?kt+`@NkViuE)F1JX6=e6bejAF z60()kY&~$(+)@lGGt9Semi7H`rAK5T5nKB#fqy;qPBoX_Q9H&Gz>bP9YpyGBT>KuH z;q#$!k+VW)L#xPhpslx0woF3&AnsNUHragB6WS@`>ju^uUsm3QHkK~pV2Y-x-H-sY zMCK1vfKS3%{OMeMF`W8{1qK4p9DFs9ALRhE#K)&rla~KZ_uVpQ|5;N9k+Q0^lw6oonZxCDb8$1Q3&$vvOeb-vNK$Y9B00&Q=7ksf)3^o4 zHagtcfn&;rot8r~&wfE^%SSNMB$w1b-4Y-AM-_?1?=`g6&oX^iXFBUT({F2&c?Vq-{~@6=)g& z)5MT8sOYBcn(LG(To<)&3^ytwIXQUc_N9pA=%nQJ(hy+`Sg`HvS2z6}AB!ZZ zFXzDfxDT|W1ZP7U-XF{C+(7858{L~ou>mesq{vk=T-*SmiIi#kGEU31J~-y9%w`ys z@!Y9fVntBR5lI-E|K!TJ11px1pQ83YAc^Ls3z(M2N!H(()gWHcxV`^LX(b`O)Dqgi zgR1!OrEo#5Ch6c2;I|2sxl0zKN`RJ6en{KNY+PCAPJl#hVsvwT-grKjZPY%8=#KR< z8}fX%CK$NQdqCpeilE&>oYkc9MOzZ(c~bAcd@a6aVl)}D{v?5a5W0srcT<9{<_1if zQ$KrkDK5^B3MWeQInp|3|FWWS$C^>$R~X$2$jU?H6FQD^U+FFnkw1J`e~10PH)@hf z?PFgUV-#k$-(152v$q5f;izR2@TWy}QbB-hWx$TW7tQjqSt7z$3ZI}#8;y-Ja+-^U zb&JlWH-LPW=71WQTGq47pYv;F>5MQnr3@bAOyM>u_WCouyEdfi>0UelG9lPUudp3fk~S z=hC2o$wq;52+hh)>6$3P^|8h3**F~kcp`GEr@JZK2osR#26I!8YDn64pKUXzf^Nih zB`YHwT3g4fr*Z?e-I=vr+O?_@NfUgln~fWw@39!FZJ_X@pZsx=&(io2G-_8OPjVAk zCXn2Rv#Yu?V0Jloc9us;?uLS&b&kXNx+zWeF78yRXWtOWE)SL{3a8Tmmd#zh=0!_e zloOai_<&hC)zBg^3?;U$o)W7Z}>NUAj*h z$1Tz`+vLReYV2unxXK^ ztgelE^wYF5Wr?wX89>iXe|aVi28X;q+Q~97-^}y$;Dsw*)O=pblM|TaEb8IuBFy%( zB)l^^OUhy0s4N@swM!s2I5grVI8j4e)auQ3AH6X&B^=~&HE-6&GXJe!cOq>H8 z%o0+$lryEM~%kqz2E2 zBQFjFYAdH9;N^eyBxu`Jt>)*6@9_k{)YuyE~^>;oh0)F-^V(UzG)&4#0 z+7@NOMNrp9Z5rhK5z{?vRtVM1X$}0wa2U4A@NcG~|9u{Z zgnBYvT$u%@fa+RE-ClnHL&6PPLm7~|-M;@l z1;wAbvnlm2{lZ+a8EUgpK^i+a4IXOe>+%noXL3&h%w$89Au>cYfa~qSO(W9b-&Kk| z1#w3t*7Le9^|90lup|lLVRP5c^+YYw7;XmRVlSG9CFEA~mVpl$usYeR9GpLiL>`%5 z1Br$qer@_HnAjOxj{r#|uxD9nz>=WT+Pz|*I;rwApYw>(_E&LbE3u!9=%Xe#Dm)f6 zhD=IHYfI?JfL235!8P7ArR(lgA9O8SdSSlaTYaxYGku=?n^ z<*c#}9|0=OUsvKQBhT041ZW~S#HB+(aW`fUOR~5O1R|!Dg)8q0Sd04T7ScnCJ5oc* zeDWs1IbG-Sqmao>0_{p;OOrJ%JhAVh1+KJWDcURr`s!0(--L+j$4aAb#=)i4E8CAK znimac$E{t3hJcJ7p7bCI#+R#UXv5+VeH|=0U)?ICLA5gy3w~7+1gEl8nQaQF+c_jM z7{O_Z)4zHa{`I@5HL?^ig+>umR%c=jD8 zf@%$&Ff$`HMe5Go8xCn!u7?Zeek!)0B=|)bVrJMGJigh~-F3z_Eb4C=SF?wt2o-rQ z+Q3qrlOw*aLB;5%gsE?RzLz;-UiN)fPp@h;(oaD=zp9%Gsg<2n)N#u#y0ql)6@@#l zw*8lS84QFCg}=p55ZCJJ0$&!zs`5P z@65=Zf3u_zJtc~{4%M7HC6ECs0rv2{C8Z&Ubw(xJ`>SLfNLz9 zcbcx?aEQlXLMD4cxS*T5NX4L{EW-cGfs}>DXol-kC(Xg8Vt8_tDI}|iYQNd2;c5z= zl~O5iGvJIVFsxh8iMJu9UN#qOby*$tONO2;#d05m?do91dtR)?$rx$L|DtK10mLD! zD6w0UBNOP*(*Y7lQ@`%}q-IV&o0lC;hBK=YxHpAh&7KY#QhWL0M_W5sy}Eisy)`v~ zD;vQeMvH{b<{8fww)y!A#RXVe|IK7ydf-*?EJ2mhSvkuYHiIlQV@A|iC^=X@CTDD0 z%ScS39sT8n+zJGKm=Tftb1yeQ9UX&_*e*6#ZIdfNg`U_pN#Jjt?;_`b71^k&V)-YY z%oxo;V6{+`um~+Bs1K4aXD4ND*|$j@7zE?dHJXP+cG}3dM<{C%Y*)yAeWysG?z@2Lw6`QGN>=8Y-d`azmv0lzjAS{!ZJ$K0n*3Hv zWLENf8Red!R<&a+kEh^7e)}5%Ne8vd5h?k4R)SRF%2AQh6T5QJck%|i^I~r4j0PX} ztD+$rkSJ$a!8_IY>f)>r3RJprq&UdM8+%1y^Xo$0b=*$VX_DZY9oOgyUkyb#8smcm z#F|j?OGwPgOr}QD*6DL#{ts%GCTBR;!HFyfU;nzlr%Rnu`HO@lPT!zSFgau3GJd`V zR}88N4#9TVV2`75#w zXfWU{Bx&#GgZDvJ$V=Wkd?0xa+1%m#V-i)5L?#zqC?_FSQtpyf@B5r~=fDZ(Z%c($ zvmwkQ4;*(Y3lIFL(*F_sCIi{$U)SzD`>8-kqD~SB#|nr$0s5Si$u;M)n5qV;IIUQARDh++7#P+Uz=wv4%)n z%3#4Hfmu#FoQ{7w)sIt_&;xI9ZP0waSSRIPxIOtMv)@jF+S*y5nmwa~AN;iX3eB48QM``U9d^p*Q1?6LkpO69X5~VTfPH_JS(5WbCK+BP zc7`7?vaU9A25>s^Ym$z_C5fg1HYxT0zEY?YaG;TKMaud$% zd)G9%RBrY+ijYR$fY;v_o-)zA zXW{sDeIo5H?59#y)L?_$5P1Z8SLWx#?}Z*G@mGlEvao@!bloL9%d3_jDysEFe&pi= zA6oXm22(COwpj|E9tZ#ZIhTd>n2x3eyKkaq*;Px5i1}?I4e+#%s{L zLGa{jvo*lH+i!TPBQkuV5?BM1$K5#}hHJ(?9&4UYi0l^O%DBu({AA=125Z=L9}rJt z6I)1QhT2h;DA^(L?m5l?s&w^JOKm0l<&+})D>Q~WReU)vX`!jr^L7pA) z*Vtrq_pe9&pMPfW5dP6VBfhZE<%Y`I#RT^g+++X`MO+R^IqTBAy8J}j%z0E<+h@=g z$LWgwCjS6}abL!C`)Jo{nRLku0087%k^O55wH-KJZf6y9U-oVNiFB`oJ|vICn%r%F zty!JczI*OSpZnkr2_23&;~eu}Qe0~{NN~3j0qCx4V_JkGwmMfMe=t&VYsi%4En6BlNE_@eRlt-OxzHFv*FcMI?ns0hDu-z#wy9T6|9Zkaf=pYt|<7`#({L z+^nS(&n+0#0}w0qt6LQ58G5 z1ovvAb##jiRsuNG#XHE;FvOoQQM8`wHiA9L^{(wcbDfm?}J>m zs&XB;dOq6v?0Zka?~bkD`)xV3i}>TUw_rS&ZUj#v{{VNB#(jEsuhQRx-Zi=LmwCa(x}^uhN4wSLy;9b3>F8wh5TiVKKf*{|rzZjU z&3YB;&bKS&*;w(F7*l#nbK2WphTkge06bTU#d^9&?Ng|04|bNI? z*R^TbcuvP((C*Np>blfm-ddSt&zl}X$gDf<+^#X$KVB>Jx5R(5=fe*a>L&YB@J5|= zcvTS2USQ-P0dm3eZ~*7?tb1?SGvRDnrM2ashvm|)p~#9`O+w@4+z2CUyD0=5V->%( zrrqpi%`RSByC0t43cqL15&T*4Hle3@OGA6q(!l$5^}$jL`Qn}Z`n@P{{X@sH~6*TJE`J$(@~0b2!RVSD=y*( zJg(oZd#gwT5ZTUaM~s{oBuw2T-Os{U3rGT?24w`GW5W+m=T>jm?G_SC0|$tYnk@Xt z-%kGkg;$OSxU}6L5=3*yr9K3MRAP?bXS+Ncld@m%HRz>uOVHTR^yz#>Z3H&^P2IpK ztDn5;2?OeI0QWV|wwrY&uiE3am+Ya6dAknH-kIs2YSHlavwh)BS{JoNH?~(vJWnEq zK*+$1gVYiS74`oB!5A0fUw|yM-6rP3!L6l}`(DZ2wrnkss}Ut|z~>|$bIw7pKO0%l zf~jksu3LqME~BR@TT8Dq{?&9=>57k&=ElTLGSa3VQV!$-$f1aJZL z&QEe{qMfYK*-5Tek0i5|FOu>(NJ-Vqi^l{WqQ3tC{{Vu1Yu1*wnlv!Ma|O(LRnOWc znUu#VH@Q<72O|Jr?#CmJEAyf&mX7buV0rt))E>3=H|(=L?uFp3ZsSInF1vZ6-AiFK zu1Z5Cyy+|9bBuW+cEQgo4S9H&b44Q>Zr4ZY#;2`EbZ{7S=DB@Z_BBRhAQ-Qmw9k%O zjrWIijZQ_5VSjFA4Gu!?S!9WZ3FGFBkTQ59n#sBNyQ%9kT)?|+jAedhZP^SyQ|*!* zf52)yqp{R{FVOX07G3GOq$1Nz(b-+ErLurUI?5S`k|jHxo9;0rjvv<* z!hC+wt|er*hSjE?2ariB{pmIa`PM#VN~+Z* zbbNhbb10ewhyvxX6m?_oUYgohhV1QPT~5?o-4VIuMk=b_fR0EWzomKgj*E4sYLKnW zMmEQo4ng@xf8|;i8iUM8KQUdv1KjYye=%O&J8pTizR1ta2Ye?72c5{ke@(xUs#n@6jz(yKDmXaodwvF@ zr8v!4x^)~K&fmiREYxGXA7Qe4xRr{mgoIXM&H+4U9joYX3TjtcyqZ#8pwr-lnbtVf zT3A@9%LW-Lc;^QhBd-f!J|iUX{h<1VUUCJ^+pG;A6~WSHzQZ@-lJovTTO8_ zk&NDHxA!Va&lXCuoQyCG2+vQHgU)L=gQq%e$Kp0vx{;bv_)o1q6T3xbgGQ~kUOVD9bM*qfN8s+a9--hhP}xv7 zE`10G`q9Y<`_Wc!bEVXr;-e>Ywl@{Od5XHq1X&$>Rz%a7-J3N+e-uY-tgOS*nUhB- zx=Q$a_JX?A;Ts{=^tj}hgDTUNp2lXU~w2ZB>Rf`NOm|S(5+;#%4FRt%u}7<5=r#;HSNE$ zpTocSSUw=vZ6LlnefYOYBcE|ojo>h_kFy0jMB z?t`gBvf5loq2r!50E5p%gI-1ZO8(4W2fRP_T~kZcZ+tneU%2u#Jw6|`M{r5TLKagJ z=c^XSpv`>h%3n32@A;oiPdQObnOpKa`tw&BoXd52t3{^i2sb;kub9X=J4O#D*Nl4C z&VC}jxYRssu}uZt#j9RjPcPb=7c1t=xgQuA#C~2t?~%oQb^9&;$QE8A*7coBUDP6z zQn8Xd8*j3@&l;u`+Tm4(H!)(JK2WFo!oHUHkNYNgTgIA&tXJY#)atY9CTIXjJn3VK zAt4}O{l+-P2|YNkTML3%NHpKP`W{X{F?h)+rS7l9{J{8$rHwu)p=gTSBKgr{ch0yQ zfG4Op>DYnOHQ~M`(JU_YNVgnEa(0R1`_Z?lAH+Vq``6w-6TTHZI^M6~-B`3bu9-8f zo&LhBIuE-d`w&J)^}(-+d~c*r;k|LKt|AD~&Q+1Whk`#Y2>dI0QoZ8OCBDa)>Ka^{ zRKZ{7jln4(95DX?A8KqdxPTE8;g3*<2E7rjY_)ALLvqnWhY!7^#~J4Y@zTBj0O3!K zyfdL2X0Wu=q>=!jONDj>ch4rV#ZAy$(T&cU5LI2I?0lmQmG!^wng)!o@VrDH)K-n7 z%?pyRAl#)GMZm!o`cb6*(O=p2=6OPhly(E>K7fA_{ReYV-TY_L{8@Elcdcl;T#WL6 zqT5HNM{{w<@n-;&{{R9|FnKlPEX}Ry{y*>!N9s$YkIc64`n;qgGrA$fMM-DJp&r%r z*X&WDY7_X^OrKIqm~>k!ml8(tl8qeE`Ev#(NZYd@jB*JT`X{65-VX5`Hg|Gqc2~ES zn_k-L?@ozc2JB;HT(973(EK%^>GpaoS1k~~x{*WqmlSdgF$cvTGyxg_H<4w(Y9 zygB7j-;zyY+$qj^s8!`1g2$R4+55xLCYU1ehNK$C z8LfezM^+CMoBm&x>2ODw*f>AK>0I^4?Ad8+teLb=8bfhuJ*?K#TI+Vw#Q2gnLIcFI z2I?3R1_&ORuV0n|-3MCBx`CmScOsY8W{Od{@}KR0;Su2dAK{geveT_DE%b>jrhP2# zg5lBBw3)yq*_^W$0Q|#@V!mCv1a5RcyjIBikMe&y`^Wax@X*yh4C#8}ATsG!>z4lP za;ZlA*sD>hPjBR2k9Hr9pyUsglNABJnGHo|z4_QL+w zMhFa$IOLA}9=}Xi7k-D&DxjQzKp%}*(sjKuDHr zpYH;Z)e%cWjpiEYkdaC(A=vF$t?k-qU*vTPT?2d-<6@sGy4D7Hek5l1A?vM}xx zyLJO6zHHY20BIY2bHv9-(-z|L z-HUizUy|+6+a;W)P{)D-l1IKrYN_y3;;pazBJC-N+D2XDD$X{>%d~7h{PF99>0c<` zc;@2USkv`4yOLXYA7#aZu@OxR{IV+n!N@)PX1f0XhqHfV_@h?vYffae)FW5Z<(H#K z7jzDN*$7kaThhAYM)B8k7SvCrb^id1amX1h=2QOTX1vqHzY_HeJN+q`D_C7zi)rQE zgAs5P=SL98g>rbup&sD# znxAHt;m_FV^&4BQOIOu2iS8zp>Ofs-(#~>WxsU!?gV+(<-xal?U!M|bvECPk8=RQ# zW0>tLD)8T#=kc#TweeB$K8tUBh2hjK(QZ~rFu8T=-FV&42N>Pbuj_gR*NCpuSkZ2t z3td7rxRN;Sq>wD9h8aa5ayona)~l5(NT@ z{{R!VK==C8wr~mNSc`4>Nn^Kxis;g4(%qwt{;Gvuy!gW$nNgY+Ow2wpV zEAoru_rvJ?OReeCPQTdKv)!(rsR>t`S$8Tk5IW;3cT}YJ9c(Wd6v<_HmoScXrp;dajSF+q`m4 z=mC!AG7Ah26f(My2|snZ;=D`37K=AFQ z-IRS_NVkghnVc>zt%-_ppZ1i?f!`}%o4Qr5n*nQS?n=y~5hyJh1OD*Fe;P3Ns|Oi+ z7ddh?(cD7j6cWuaiV?sJef$3apIXz=z9Bx5sHoC_ySkJJ#mq!DbDxzsJ@KCW_OBSeU;pj&0{Qx3^KMtu{iujarxKSUI_R{;olT!ZDf}s*HPLEOUBebn|s&JVxt^1 zxn-&C(Uhx4n`1}eJo@Fm^W9xv+|6}9)sW30*cTv`9f=3H`d8>4jSyWXIJ+vSAC-Q4 zctS_hJacE^e-P={52@MR#S}V*opf3$20>Esw*g4tkWbK8=w68VgsyUME1HtCW;mUZ zk*Zi^=M~86mbeFucCSd(-~|b;CtHjtITgD{9gjcMA^Dq(Ry@#P=M~dm#tSwp9&`hz zPfE$V8$}t5ZV1LPp7mD#!*qQ!-=%A&GBehhakY^9)Hx5ATb_UMo5SP9AG7a`H1eC> zH2H2-MlxC--wb{+iu}ieD2*70&uo3%bg$Ov;EwA|zOk4C1;lL&k5I}%{&o2aZziRr z>hdfy6Fg2)gAA_+2R_yB<+8k7PCD_kD_&b%G`r$SBV`z691&HddBIaUHss(q7+^oI z<6GV$)~#Lb=7uDW`#72;C6R_ZXQfSRdt-kBG-VQeg=l7Nsyzwqk9zibV&1IgyX;{^ zlU}CCZdJCPA}fX;fa~~XgQ8x~9`ujT0O0vVF&|JlAB}9squOe>lH2(+Sl!3ZmPQ-? z`S19CReH|PRMMmq#<6QK#xOw7nuF6nczrs1P@&8m);P%$PdNmuwn2p=P6r(`{(D!i zd^5S!Y-N(x+fkQG6Gk3ct#1L2RU9wMIb3Am`{RnMG-?NRZ|j&Wy9yHlyHsz009}uJk3g1gSMz}o~KEF z;q>t)lW(cXG?!Ywk3ZQS=W#MR^6HC-0rZ{XDCD+61LU{#8-W(?cDR#KxF2OyEfdXIp<9P8dP zuxS4Pvad9|MwrO=dWcx!VgLc6OmE2q=ZqWwCHXjQ~XMx0w`nOey1 zraQQdkrl>kxzZ*6*YKPgtT!edYr}4+xd3^v7s};{0d})DUcO)+mFu=&0JK|0SG~Ww zx?$C03lx9U)ONoNG>aWdac3pOFkC|R^2X9`R#Gv^BRL&G^#Iq;{{U`bRP3AexAf0? zIVL5m$uCRm{zpNfcn829EXPPBNzTS59CdJQzDg4RQzeXz^Zv z&1a_RFnyIySL|~vasCR$Nc^jS_)Gr)1sXSEGIUD%Ld=JX6MyiB>fRcKHjQ;?hD^pM=D@%J%IW9yAv1CuU%h-;Y ztks!tSAcyn`B&D{ZO54TiKL}s+)>*~Hp>c(9uHdBi%Yk?_ zO5s!H_a~tBt&3P4UT0;lu$ugd#)Rvpv*rO{&}bLWr9^{=tN;GAAFf&6=| z=>k`Y?yRv4atsLy$fT9^z$E^4l;FN7jXR&FnoP=V^6D5qK~gv(qSK{KLKzvDE5$Ka zUrZh`{OY>hTdMtvJBZE7pe+z#xj{WhrbB9U@b?S2tAn`!$v&9%>@+f$?xZU?q8 zUn+mWMSc-4j(!!}_^v5gFFY#Ty zUhwv-rE9MM((i3v-r+7JRaQn)8k4zuOE$kXQf&AYf8S77$>m1N#ws= zhE`IWglmJzZb8Tcf=Asje$M{@3H(*??QPe>x@%a=8S-xJlnL>U9cN#ho`*TEN#i4f zc5!;1l?=XEm~(R_`Ja`SS{g35VW_N+C)gESTCZ6)l2<3|$4d6E25Q%`U)sI9y|bU* zvbR+wbF}>j>sbE)89oZ!dN1F zntbKAHNqV8+^PQn8uBsojc?vX9aw5}q}@3^QQmw$znfP0&j7e0MiN6IkPI9inf$Tu zfO_XYd}vpLaUBK)etP^3l1KQek`^Ro17!68@%=0A4J%p^G5NT~aKlCwla_~fsNOG@ zd9EK>xem?FX>a^d8z~#d=UmP1zXQ3;gIhTqoR6T}mED|*$Gij+fCegijeYrZbRx^{O)JMM(!3#bjz;CWbujk$t)AUVq{*jD8-`9Ie); z2z|j!ah>myamZZb@x^46r2W-%qme~>+8&oKspZ8X&N$6}SNPI4wefDgt1ifr{^54} zi5kA>)&3w~w}`Yb>LKv`)x>d5pvaPiXCo}w1CR*;Ngn67#9tWvIWL6%D0rVolY2F` zp)d^Ovoe+}q?5@c5Pur_Obt0wPn&jmIEJL}6Oy_qGNW)L9Gv&f5ozyogs26FJ3;3a zBoW&HOtRs^oF?4!jyb6;Vr~1r?`(t3dcLO^>7&%zIAoS*nm9((F58S_)6@#tu+}c( z5y^9JEXchsEzj>1e}~qV*7AEYt#KSP#!f@-+D&Zf?`+P2Bx`k-=0!0#%g;MbFh+g5 zR?${^686-!VX5ibyQ|!&yT>B()k54y{``N$KhA5v(vV+RM+4qRsaVO6D(N_ATz%Y* zppWtFYZpZDLU^B6wy|9~bqo3A2?UyPDRAs@j40%uYwvH_ul@;_crGH={vzrV+SoV_ znlxA!QVzmt2LO9ux4PETpDMF5QM=GzgI^D%_@|~=>b@G%tZy}E+M0#T(Z(i*QNufu zP6J~(T=nUi`wPGyv!;#V`*AkBS6U)xVlagy*zHT{Y8CG@VuHSuZdv7tsIx37^lI^AaPYUt(?5Uz_;pfn9 zq|t7Ah+i_mPvi41=6;p)uZ?~Rc&ort7QWM&?Lq$lkIQe~MUVG~PxufO_x&TFkxH-x z5rPkT9`)&(oiWpYjvef9vu8 z0Q3s$d_n%YFQNWaQCWSWi&Fmp$GwmH-_Y0QpY2co03}mY{{VTvofKBIelhfC&i??# zeg5(ORr)jg9Dm8R=pXj={{W*JiYv^{e}a!+DE=gmoPXerf7mkP{{TX-5cm=I z^xyao@-$IjJwNwT)1Ujyk4pFn{{STkzurpv+TZx7qNX1S&n?W2OZe-am+v3*+KMX~ z?j`WDt6%$-(%q{<^{c0HpCzMSAPv`5t6;`JOEY-XD!>Y5xH6^1ttY zjTBbqYjb|z{y`A^f315TkALJy_;3FJVCjFF(M5E};)$I@*8c$Dnji9a*1z@$y8i&# zYxZPOMPXOsG<^b!D6E7OQAGd;r8l{vicCl0kH=s6IUgQ>*jDfV07kWQ&G%#f0C9hn S6jzx?;5|%#@jgrPNB`Ly5G8{E literal 0 HcmV?d00001 diff --git a/docs/static/images/compaction/full-range.png b/docs/static/images/compaction/full-range.png new file mode 100644 index 0000000000000000000000000000000000000000..5b2c9fc61e4df0c44ae77813e1f95df8b5195c7d GIT binary patch literal 193353 zcmeFZXHZjn`vqzTMMaL%n~K6ws?vl2K@kyY0!l9;NDoCIp@oQuCIdbHj z_QU%|M~<8iJ96ZdCF?2XPwY(o=3#z1>SLsJ??`d40G|2HX|IRoK1Ysl|8?+j)Cn=b z{1f#f+V}51@jtpc#g_KO397j6)4O_1hw%Q^t(2%+@7r^qZg>>&6g|57X3Fo`#b>3% zfo_!~5Ol3)9Ft$PQ&cpb4+TX_d^{q;c2u5E-pI56rfa2Z!_$kN?~^7k9$W1WU}zjO zR{;1pLYzVckz0FzdR5R26A*IRgit{qdwl5<>Ei@o1y3bTC!B$jWg?= zOOd=`5Wk{W#2@HZ&Ie^~s7seZky%~vFW0iT~)z+ zy#l3LzMx<0S=lkd&yG%|n8P>bjKLFS`j#I_|6A0Qwbgkl;)}B@|EJpsid{ZC?wvIB z`hVH-L-qfDp1dKK;6sk$eC5wf~&+2mga_nz|{f==XnrLe5d@HU&^w z=`7fU`;UgikXrL5wf`fGU+P}TEDfED{%>K;5ILz}Dn+%BD5RsJ{*O?c|EdnYfLY7CXhlvK*5ffWv)1m|6FGE6@25PeEqO1WHdwfws2^r zozNWVM}zwvpH2jL#fh@(Z_WP!nTzpvwYwc!YO4j!+af)EmmLU(kOCf0vjTa@eY?2Vr{-PjTzw%7?* znF#3NTp6MHTZqbc+~t4`-7OX)c5piwzH~c9_QjZ!J>(s2IdXxq+>q zk^%5Z&dR2k6;gpeW96DQj1x-~M%AhLVS4s*O6pum=|L!E0T{TiTxP|xZdE$~u6mCwoAE`kCqp6e}?-GU_vdebdaKf;*%xG_6#(#>vTpqG5OOvTj zOw>6vO~1sc?<=y}9FkYB)U7i-@ty1sY6_dw?Wyg41@Om@2Y&=@#3gVi4eUoppSk*n z)H_4cAiDM;NRqt4jwjxjOv<9SmrnW3DPZ|;*5@bU_!V&gDTA8{%QfLFKW^QNGFT$$ z+w#q1N0Cp!MZF0s=qACY^c19MqpB*zh7c@Nu4dK@m%FK7%iJr zfQcr+2tK9)-oL!Z5HDPWP@FcYHD^1ruki+~tWzp&`BZxyrnN2?uhhirTYnbCv$brz zUcck=b?%M*JZP}we&4#r1{hjMoVX}xue>b%cai}-eaGqod4Jz4T^MMPz8o+c#AS!y z2f7A;x_T7p4zw!J#PZRx<2!ZhEjlr&J`QB{ZhM)Lt*h!D9Dvb9cmYA{aSd%QnZ@4Htf zAz9?2Q2oQ-FTM$oDyZy%o!z|7%(%LdUu3tPnv>u72j`1Fx%%yKCRAd1rc@Hh5foXd zA>+^7tgyi7a)5#jnE)uHmD(t^Y`ZaUWU740uF;2p8GEkrrg!lH ze%2^OieL~(moqP$jaBq8qm^>8ds>dYKG_um5Awy7jrMGiuO1=Qs2(?%L6n@0_p$!~ z>vR;}NJV>Pk&yGw0{sLVIk-Bh&)*#R1-<6sQB_bWh4w` zO@kf}Gc}DLUnw!GUflYIawkEF1DsHftJI0SNq&+~+Xm@9I;wQBZ33fphcfU<5WsNl zO7sJ|`AMx6uB=$@eEeR&UH4^-*w!6+-2CF5Jyp9gU+oj!i2|@qyyq&v*+$|Dl3bLz zHc?H4H_Kdy_G>!mRuvkJ+>1L*_%5>vN&W#g^|Juo$;V`w@oiq}Q(wxG$H z<1J!W@PyX}%XXbFM&5wNC0Dx_q{4YM!!CI#Gym4rNtV6BWPCQC37$R;hDpAn&CI zNScrKUS87m&(>PsciU)q-_tM{5tCC^T~y$Pra_xqGPo0xRBua-h2JV^G_GNwRrVlv z`>5s4R}M=A$G6SjLrU++CORe_(mkFnvJF94qNdcb7w-0%mKwI5HVl0J%mq+PfD8aUF zlhdcX78@nsDDnDH`M5k`+WYxrO~L^3J76#HL;yxYi06*Ir>)rj>USqQnoTaOdN#80 z?_xd65Lp2G7yYENH(&pZGZy=x`f8@(LjSPDn7kLqHextFj zL}j<76l%Pm_TbKzF!@BrAPw?4eYPh0d2|4rSdS-i@#6rzxfz2nLJX%s z3Z^iZk%}9;IyTcjx){@y%w=#a7C*N(_Vl*i%H2MeOGWbViRvc>9iHr-X4KlR<0cfR zSy@%AwRHljV2skT87ii-K7>;!aIv>dbm}N`oNM!iU8whhWgje^ha~c==1d8hDaO%X znGIkckwXe1qEOwF_~-PLIb~xMwAnGJ+VPt`J-+*IHQC9twb-*lu&jIj zR5fGl5wza5WplEkp2`N*Ndh9R?PCOUJfF2is_2z`iTOA=>Lb6WA~xH~ADC3UpDrq2 z1ZivJCjWWCLYXcx_#xD?+*wy8$!x}NM#O@bo*p2J0^)eUWn}$^u2Q);!jHt(8OE-* zUA&a3^o>rB5;nZ)XKl@xzSJXQuxT6nVIJpMe3s^92r^{ohGes|+k&nAt*tBCujwRl zk5ptn+h{IxY6}(`@XeJoAFi+!I}AIj|GB_@;gsqx>5q7he_WrMXT-d0YE{Hj6Fef1 zOt+pNS{6~+ey5ZLuUu2J#SXkM&~?h05Xso+2FFfy`g62cj@T~OO1Rp;UxSj!wlBeF zDI<9?DVTi8aAUZ!l&}wF3Z1Zqv`yZk2g)mhVIVEJhIQ2erfZ(b?-N)}d z&QW1oSy{5JduiR*xc_P+3vXUtHoq*lk}_hAU>^<4-nieW!-DRpuNPrWNqgd(!up#2 z)uKqvt*bq=glm|K1Zk0NOnf3OY2gIV;qH-6%JI`$rwBZ^?8}BJ62{tmF~1-82a^tA zR+s@k{~|Pu&PLMUMNJ;U1T_D=;P0=1gK}60KVZWZYu3zIay;8JhktL-8DuTK#XJUR;uOqO~tr~(8uge+Nq)0bVS4lu<4!@bc@ zPDNHu`439NHcP+WTDB|uR$eR29xui<97BlRfF+9VX>F;D!H42(pXgNb+kY@J5?B3O z$2I2{ZS%`Bt4pD6_FBT2b!dtZuf0vbrO&aVt#Dz@saET+lovdT3yO$!zeJc^u+%PDcaJ_IvN9wCT-1hCIef`EGaR1(^X6aF8sx~48BqFt|2Y74&OZj_y~jR_+;yRa zykpt#a7k3T1CNb;AGCc^6FuUumVOZ%#@bpPcv0WsZMtE7Zo__|7v%{n=Uc})WhQiJ zq>h=oH|&q+6(fz|c*P;`d4$n&tt6q;pAf@{4Dus*k4++%(yJ~n`XK6O-_^v>=Frb< zSE?u;*;KUvjl;2jD05|(UH_VV`=2Y-_lG*1?s0-oA5_tz3b4%TSgbIy4AMew84DLY z@!%@~|9t&I3$1iB$dC!tKdg_oq6&CRB#+(l)ahIhkAIYOm7#pT5Q;;z(0(vEjd2e&jHTBuD`8B#P0ld ztlU{9+6LH6Cp1YAl75=a-6|5`tdzDI0`7hpJ-WV9G!&bst&pUCC-b9HHM~KD~oPPv~0wB+ok4|>P zfduJDoF;QxsWe?h7r<&cd*Sc>liRvxZ}(>CaNbJ2=3RdUhR zdwh+XsrFif9k{%oPW9YsQ^gk7H(ayGm_D}PF8(>=RMzi5%|!S7y;;mc!} zDP-rEBfaysxX*z$7s(xC;Ue>U?Dnv3ZvA;7Ue(3+=mI1!!j?JC_Y;+}Y8hO(1}|UO zH?nose0V>Z^neyU9@Hfc=luD7i;skvSeY4N6>Gc$o^7l!pYn&22_EFK@P(g>=+0Ro z*XzG5AJG-sQ-IU{5zj5liM}VCoqmPW?ZTL_0P>&5z$}g9Putl3%&wx2tudKmaQ5}JJF zLKB*N^$_55FoRfGr#MSXvHUiQLzqVwe7h|+XZ#9)kM#<0TBA$c_LO!SPiaToO&5Hi+Ck=C;NVuIn_S$` zX0-_sxz;^%P6NhK|M5Jg1QT&n132}>oF<&AP#A(`@)9=QZXlG#bN(Srdqef5HRXT< zyzi1&!Nt|RHDik*1diW-one1np~zPLGqITdpfJmeWti2Fv?FD{LA6IYleWHU9?x+I zFk=594{ftT)(4@Av1zNSw*%K>pWIYDTVnZ?eG>`s4QeIh;WEyxga?8yiO3p5EOJMM zF}UurHckLIGZZbwH87@->2XWN@g^J{F&n5V3%)0DetMeGr;+_4Ta_)gGmvgj1N^}w z!&sae7Bp~YZXtmUo+dZz6^deD1Bb(}Jh;xOrKKv+e>yv6!E4lF6Pt49m1|jX5$5annXNWL0K$xLNY~ z&T3YgZFNTTzoq*!(XHNbb42bFkEsfeVMerj+$N;Y#P66Xj+CC2dKU%g3v){1Nw!46sfygYP7njK&JN+@c*ggT_|^R32Z+6B<~yZWA51%Iq%ErVR2^Qt7*#ONt5Aqog%!VhB&XTJPE|dgc&RN099*eaL}9gr#asSPD61lzAFuY^o11$-qVmk5E8@A$E!D%l z$<5u|&J^Jb-TMpXM-S;(Uuutu3@-yKf_6yF4?YBd`fJRM0F}9pY4pIpmos4q*<$tT&V(MD=%{(YQ|#@Z8fUr?rc%Pf$&K8coQsv?wy=Y3 z+vP}B1f}M=u{@f*&Gc}eoOAP^+YSP=5r>fPQ2D3A5xiM8X z{dn}Y>R*-f1J6Hg$K}4)>4%%H%1i~D_W7@`^+qvZcY_@5FCCJX>aUu&*;vE$n@VL5 z5$HnSv;FdjH+?b^Vn54L<@*a4nZJ1rs&3dbO4p=`FKV{TeJ)+2!g@XF%%NR*foF`7 zYXV%TFL2v&z@`XXmZEgasBKOz7(C5GsdAQPbKAy z_0rU0$yW^EayDB}CT2tSQM+5k*U*gIL>!x2z}hFpDSz4|s~}vNIir#{aF86pGr+L? zBypE_S3-Ad^((c=URT4W3#*Ww2NEu6*z8QNR7tPem{90HF7w!v7|GJx5m+4Kwb#mK zkR*l}gJL9`8kswVz5EL^Rv>ArWR~v|}PuyX~&CaXf@Qk9eUKhl@h>#@;9% zYx-IMxWNR^9d*n#XsVI%X^kqU5w@jYHIh9j>2+1he!jc7H-C3R^j6!1KUmM`)$ojaW7BF+nIV?U+@>UpelkdN4ey9!2au(d%xVlfqp63IIeU>IM5Uu;sg4b zC~4^-ou94?%VD+{QEcH8a9cUG>?%U8D~Wr!VgNR&oudjwx``>2q$3&9!}|?^hilP; zB7Hlyw++y8(T@&UkI&17$}!M*3-1%3LM-lpQ}iW{a361vv5h)|(#ucc!swQMHK9hR zbkPB?5!jCM@yfB{c|dEc+lpIERyFyl&X(Ia;CY-1xmG;p-35%jy;EOy$tRVm$%+hU zdgYf+G5ebKqi{J%LCX}`vbmW2N;9(J51L^(YWCuy)RAU)bfnj}ZN8D~E2-1#= zN6e~}zzIHrk#ppo^YD)^Txer2ZTk0TrQgQV8|uChQwnYZB3;D>VP-0W#Bg3-ZK*Ya zek3?DWajlrHk+r|pWxf@pxF1;cUj0tVTMx(PH(t=PMCN7FbKjlJ;c09zYD-wSe*Xn z{_Pv7zWSUOK!G~dCB|PDspgCp1gQ4j$mgAE@_+W{!@DpwN8btBG|D5kbp;cG4_O0w z@DvyBc|jq+^!QB}U5kEhK;FB3h+j9xg-#S10H&tsGD{q&YX&htp7jHlYN2i~mM98? z@&Z!eo_gD7KJMN(&~UXXP#NMsl4`sv_$q3_rBY^}^X}+rn%Y{V-^x{gf%;UghFLyo zRi+A1Y-1%9Y;+etGogAM5x$XeWAaMCcN5xDXiaXVD`;0e4fhy;+Ui`GD>uiB&D#F$ z@7q6l)xXIPRn%l=#g670)~{lC1!f;eg2THJJoJLvr-4@JPhvKG4cP(2*|wd;(a1M9p(tQJuC6|h?cycMtyC@fte~)fqAJ$E93AMatE)%VTE?BeN~R~ltZXqW>91e|9oVG@9W-|cRB*R z2?GPEIxY^Q37aQ_y>z)`cwZm@H=0#*VG1U92i{9#{EQd8MF?m4RmzP*N7dtc^(jn==fU^X;zwKi z-!O{+HCoT37Z?W={4|Q0hIW$L22%S;{)69La_gor*GsMgb=ky*3FSg>HU{RU=Ynnr z-E-292i7vZG1=lwH^A(hm{hY3yFK05B2Ve)iQ*F8r~9cFyS_1?J+AF5T{_m{DXLv7 z^=3{YA_eUL6aIg0?o8bCESrfkd$+}%>Hi#Mra{jIuIt9*tK59`i(!rI(nC<7M)6>2 zv7`N>HrTkF-mxud)3E#HPI35>NuX7^Ot(X?wZHD&3l$;3l0C8F0#0!!x4ITYVs!$A zQona7xg;VI7RSt)-k}5>-uzbOHv+#xR9s^Tb$Sh?^cYy}ZcZ-Oq=m87*KeWk%aGnx zr77wO35o~$p3r6N&g@N22$%Y<&jqA-P8l{*TTM-FHo12xozJ}MI9@KpED6rd;cQ6t zAQjORf(#?8)-ZyzbFRaWUSxYclEVSmNUV#~!J!e1-2-H%DkuxELf{p#{xmIvJNRw> z*%7xYOM)zgBBccN-z{4abwG%UrwnWTy<5(&XtM+CaozrB{p%qA9P^n-QMd8sqTW9P z6hZL|07g%Zj5B;*O!LS$ygZ%{Unt|Od=3z1QXi{aJoIFvk09|hzAF4Kt|t`(r6k8% ze012GWerrl5od{OUnZXGY;UZ>furh+_dt-xKdamYD6X~v3)eYpthp~vRcp8aL^)St zj<}C=AA|ao!(CVT>Lig73BjhDKlM`IhD^oy2&~>Zh!a>tVv<*`=lr^ljGoBZ<>7~y z!8s~qrU@s=ew#sjAaCpfWX6S~SP*ZmT%-JTuLLyDz2h=H4AtIdfWD=au0eGYRaMUi zjgId0BisV$q5X))!~G||aq4W0GG8%CjKv{susKuuqnHb+53w-Hn;i}avw_hRoveQQ zt==S4tw4<4jI3*bkS3q*3id4D&Ml&K-RvaOQ^I)3lnH4nlGjb|uj3$BxBYD%cmKw^ zoijB7<%#0IZK*JZDy(})XJlGR(dP@uj^%L(68qU=BGa37V-SoyF1M~5#PvX`c1-FW zIrpg>i(DG-(V63c$m70l6(qThXM47TB&bpv0$*uCI&I1Rj8dvWPCa0BQNqUN_&ns)fmMa)N*@HJpJW^Rcusr{ce*IeM|8;DrfHd_ zbtR%VR(*-6d535^sEm_K=bRb4u5CmK#tv{6Nd27@bv$xVIkVMiy~fGlrUgh+T(A%K zEKv-?)sq)bouh*FW+*#v^6I71fiQYn%lK(7fnM~s_)WJaPC8>OO+wZ4z?GDOnH;oa zF!(m4;GiBG=1yf}uO1jxj z?~I>@K`fD?70Jb2U#B*^8?O&DHk`UoWyd&CY;qbBZR^O;Z-}g7hk$R${xzSNkmj#n zVg6qeO-!TJ+Jg>%@Hqz->v7J?!@TZZuTFyLn@QE2@Y{Mp&oPzwb9rI#jbBT3uiQUWjo?M?_XYJmK<_-WIX2L)yq4pfikjPZi)t7 zN~vcL*3EiF5RV0?kG}=djm-P$2p!A8LpAF5OgT0fWW}h2;X6&`k`s%;tn+(%nyo_v z1KX3hbNT||dJVrgmD@qb3EXll4UQYxmXF|bKN_>g*Mwo5_LKft=B zA8f8lhh>F2+x;5Qccz(fLS%cu7gaA3b0=#W*4zqV8MnWQ>lz%_a~V*%c=7#D;`QNu zKVv1duVCtMK12RfsXeL_a-UWp!m9kKm2ZS_Iyh^u+P=b5t7_9t+lD$ISnXh2znq^5 z;1}|}0C*cAT`#52Ad`ShD#x=xmF@q17l0UAKS=8Lzd>cM;6HzVap>|(r7pLcGz<=s2o&u)*4&y(1vK!uVpk;7yU#Rile6) zLofHd4|G4iA>G_Pub}MMwy}nQElJw(u+-TA@Wv>z5iuvEnx&Ww4Y>pj_M8|+uyrn6 zoFgJ}TA)@__@YL?l3pg|ZO-uBe;zN)Fy6^va-A%nCpr!ZySmev@7h9F&=oE`a;X#7 z&suoxK`;ZUPV(s2Bj2Qa-im8{N=+Usna&dLEPc)TT7RE26P@mo0uztf5A+XaDtSnCd9Tjb~Jt= zNtylllVxfXu+Y2F9{FRbY-5~U)i%tSOzcb{5^1Z4dxA2IPnPLhfAKpM5wYlB)*Be* zt=KCVNdQ&^gijkhxmkTZU|MDVOLmsMf!f{`S#xnP5Nf)55@My}_B$T^;fa=zY(vO! zL-dMilea%DrKCcHwGKBK5E9%ka8*5^4~D4nTN$ap`k2%&vm4K}F~{~mqSN^5gORrf zJW1)>&t-p&*mC@~7{OG#KfnTuN_Lt}94p5mk-a>Cas8XWE#*Ao=_q;p%9r9%RWfK} z%K6fXsq$W?_q<8r*aB>@ahjg)5M?!l!?<_Sn5Uec34>h8FQcqztI9Hm#Y}RySx}~D$DUo7ku@-pWuqSie7RxF3+A_)_+15tKrpcR@kFEP!w!>dEWKbb`uRGkX zZ`7OckqR9@V6)$J1--=-1riAL;0H1lJ8!?>BqfWR07NNTNK|E~=w&ah5HdGmDzj|E zFyQ4hvjx@hb61q$8eRsNyjkjX!@i3kJPb(LuSgi9Em5(7mR8>wH}j(}>?*N$E+VsX zi^J(Luli?iOu<_a2>rt)TNP%Jf@@l&p0wP0lQ|ayj$wen4S*(i6Kc@8`=yt&7N*- z3=^xs%MHru+BhdWX5n-PJ940-qnz82_|d6~rTZMC6dcoZJN@K|Vgj9Ynv0;%2T)Qt ze?@O$9;6IDFb)?Gzoe{A(?(g}&}$OxL`>yN5|rgHUfeaydTA`+4I2D5;5!nAzC)bM zmIo~dn-$?2c8hyl2fYMub?>ZAK$t!|=^K-wN+$dpt){keKlXWzZT+2qgO(d(%v@%l zO{U`=D+A(27@uBk(5d0s41aY2#bBlt!_!Tn>Tr{XdJgLca@S{l^Hje>u;bBVEDtj* zbc$~%KRuyv-i$}?xr#y53GCRX#Q={JSerzqo&AMNRc1YaYNP<5Jw2i6`LJ`LMg4hy zO7=HknwqhkN8wtnrhJ4eZ1oHU5bo0z&kvs5=>py;OnrVgW!~x1IgSZR&pT|+xdZ2* zav623^y2(EO}9vTi(j?O+c@u&1L)WZmHg77bO|1k_xN;sfa9vq^YO>e|NO`2lX%V3 zb4I}26y6A%X$kkS%mtYZFvFdLtdg2bV_+ha7keObabB z1BCE%s2fYU#C={sU3!0mM3VxOY4346ukYyxS(7z~+@;It2{j&+Im!A8`!9+AoL_P2 z?I(7|@1vM*#J}@~izm`?z`C_0V6DeQ6$nzT9Kj?+1UbpSHKLa7Bf22WmYM*FTFxF| zj{aJ)#Z601dtI$e|5l=@dP#zS0 zS;Nfd!W>2TQ73t8>ut31>~2$Z3eepG^SzuYrs^d^n~GEOA+Yr~g*a=wka?Oi5%hc0 z-Me9mJ6CPCW?gWy?G&^=@T*K*dJW?L(X?Nz&Y$N5E4~pQ*3%7X>uz zfD$?dL#=sBJUuNZ%SkdzNYljZs2wN|Q4d}c6ZHJUsd-Y540Q$gVk* z`!{*@ovH7%ucl$!6Kc7H^x3qLfootQK+d*waPeWcJGR$3+L1T3%#=9hQf_!Rb(~wm z4D6c{-6U>v0`h#Y>e{_?(yfx5kIqv(Y^}}DI4FNt= zq)kJ;Ii#fD3z(dTRx011MEc0agj`RUa`*b%OoO)LtJp}rqJcENYh)O{v6+dAa$a+~$I)ZGB0T_z@6JCYyN-lzoTz#- z5-c`eB2eP#Ynhlugfn?}@=_JJ(ZDGS#tXQuz)sBSSon`q1f1@p+|FBk_MBGn4vVHe ziU5Q883v4rL@kR>z=-<(B{!5q3|RfYS0Qtyx(01Ihli-;g#FV+LwaSR3bE;xH(Oq6 zRf2woY)i$sjRc`S-_QW{H<4(&uMV=|9j>8lr zYN|zg%3x6a+F)Jg22T+=8~i*jaa#71{dwH@0rbqSTqagKuT!H~649Qe;HENtXg0C> z_lVlBIWdO%;DFrJ#Zx^4^IskPI4*(I^vlCYht}f>$6O}acec?h`w+74#CgroQNc~b z8C-&BtQyTOHqhE`>I55wDy;-K780HbpQ_(VzVJD}Z4+yw`%%cS^O10xt(3626<{WE zJ9lmn=29g)0Tsy>X1f2V@l>@~Msebz;s z(8vWK}9x`XW0{TD4tyJ{5qAeeLeh@@@7EM3hWxpV&V#9I>v||#^Z_nsN`*~s$)W4@gj{Q3 zzjGSu)b@w*)A&(U1wt)0#tWCalb#c*vLoJtLTD7`qkVAOX+VZ@UQE&Y zR{EftoXCRUo0&?A_24H$N$66DfTFz4I^PGRxaKrtA707c@+|p`QxmD(j zubSKlIb3crh~pjXQMod7j%{_ibmb`rYop8lG}U78n$lM8TzU00W`I(!ADmr{q)=}% z555sl8&*cWzy)D-poUaDv3)#L?XY0m-kwl$UDXLW`Kt%KYH4 zAGuO@%^E7!D4`P(g-L=1Y*C(?>aDrQcxXPGFSdqVnAx3(be&64y`Bo)P7>q)NVF|$ zXmay_yqRRK4Cy+I-D8$jRw4G1Xa;7tky_m@OH6?3oI+H>+%|HUQRXfQ+>f-7to7Z8 z2U&=)QaT-J8<4&HY?|`nkW=s`a|UN}kCYwKC7XZ##Z!D1Ds${HQCbC-(i>?#qQ-Nj z5eO`?ffdF=adICVLB#3~`2M^zWuLOyb#RLE)2--;Z4^^0Z<0mF_ekp9)ucZX|?F6L5$@(67! z!F}#R38e4r8wxN|ST9uxg%mC{0S*j4ZB8{M&%dWwsTEW>kw=xtl?whz*ZHreQRoMU zR1H+kFd43si=%#9lcP5_u5*lA@REO@p5vMDYocl0+ed&JD+4s%nBe?8}WXw$}s7?({z0fBHM# zQ`4-z(rd$CY)HW_^x&lT`)2Ca?52b({*QmQ-}NjcJxkp49d0CI_A5OjQ4APcKwYUp z80Ze}-_?f0)4%Vv?5g0YfkdlKxvK#-)D!vm;-rK)bB17*)~cHTQBNf{79BXvW!(e$ zA%GjzXEK@Pzgn^n^t379kC)b$kPSbW@}H!(iI(yHfAO6#pDWXtiIuN@eh4y=SvzJ~ z?a}DQ6deW#gzYHrvX(yEHy6+=13q>`1LyN+ zwi|xHY~J)0M_m|?*mOk}S>)3sdUO=C>s_mJ%3X6Xo5kg25Ms*`8U(1WOv4>SShNs0?nH0>d{GSgjvlarLx4HRmS*Dl6l zOxCkQN=VY?^vG+HZBn&uB{dJkex)(!pWO+dk3vIh%XKjo^hn!2lzu;m~OY=kv7;Fet>z+X zv8i|~N~*-$w&A-?${GYZ*9u2d1RFH?6-zA1-=?pN(^a|# zAQeA;B}ogJRe5jKRU+Io!M03e*C1Ka{k3QQ@cw|&jdU*_IsUOm*Vj2l`iG1p=9`6h zm{yUP+Rt2vy~E|aWT+&61ajI`sNtGqR#IE{nd6zHF)c6r(TzC>lk1%Ne(6dsh0z@i zB$>#lhs_k-8WfXa=c=*!RCu8dw_MGwxQh$#78 z+9+rpeqn4?+2J?g+~-Q?T9)l;#`}+BiK(6qbiZ#CqMP*W#x+*gRa|6DGz!8CdNSF)&=vTA15alr!QEQ8wq@SomVFlR`l+bWX7pW zdm_)IuGQkka7Pvwn`?1sRTv>?n(UL_txBY-5w}#D(8S4jUQc90YE_Y0BUmGa8t3SP zvDP`-?Ax$eo9Iv(6aK^5(2S2O9CzMP3&PZOWUpA3Lig)DyLXQ7+-)+d zkvVDMG@T%{QsZ=`KznAqDLpOO2I+<=KE5@DDt`Yy=U$qUV$ZPQtK~m|Hv}x$E@VlJ z)FaOQ<;Qm-dZUZ#*wPVyF_ms`yK(MMgHOXl14k8~7#%$W;AY?Pq-?%wIl&=ZROOv* zU3sfAXmG0@)TVP=?}V#?j+zW@R|#LfuA}#sG;b7$wyp=|iRyRr#ru}0 z@xyp>F!=c@OQj|}uy8XLZ&|tb+S|si&*B-ltd2Z;t-5q?A6myZXN+%riQg&WS0KBb z90Xr}927V|pI8dXA-8f~fj;o_a!b{iGqA6uMBtsa!|-pH3v5XFBTa0*uVtK%Pcbtk zSX|Y6w@IO9Xldw$cu3G%5WlsvU-3f(zQ`5iOG{2}OdUXL)HCctbLNm@+;Udf+*7#2 z;}v$vna0+9m(Xr;XkuMC3D$M5tFDtyOL^_WCV#zOFw3QvUums znARG+c&|X%;OIooV))R8-bQLE-?UP#->SVJc6|OV$P(qDZxBNp9xA7%oL%Y2+P*)` zNWHCAv)L!JqNfPjcR8~-HSYdt`^12)iYKX!=&az_>vrP~KuHa({U|f!@1m`x zw7J0GE6E;Xz(%4TzM}>~*ci%Q(Z8HcZj~P#ZuHc*Yb~16lMjLf&Mw7lS4_-_ReS75 z`6@I{4a#~t#R#Me_U}IZkSybDpW=mWgoIqBK}iNaR%-srP&J3rztk$-$I6lfRmOfs zU4>>t9`BTE0RJEM-a0I*?)w8(LIF_`Bwmy-5Tpg^8WTiPN*Y82Nu?V{36YkP1`&}) zQW`}`2?0S`>5%Ted*0U>MrCl&@4e6OK9B$K9L|}u_u6ZHR_wLc-iPbVloU>Dt;P7& z!dIFg-RGUd-b{vCQ>C$;&bhZKqFObAhT)2ddkQ>`xNf-w)tgH`{p#$;nXz|^kSwLO z!r7B$>YYW})%F6BLc&B{R=u9Kv^fT=euW>0xn#9gt249WR?-W}C?&k#uU4EmC~!w~ z3^#S{dJN~pEYV&G(WR_5qMnHTFXzm{$&5vh)i<8XG2qrW0VeHi2;axW;LC#t1-J-> z1x<$Tx~6I|`R5u>6miAkrHX53ubeD65On3y83#Ur*K-2HI%97r__DtaTVZhFUyLHT z-=3{)@Ns=@>RB6+rM~^B)fb(}Rr1t>A4iOz(|Sc%_P+gu>00~pW+gP%m9lQi#>Q(V zR>;2HFlQ{1hk!+#VEDw-B?XdnEm8^#E2f8hsi`pL;!All2|4PUK0jqJWM#yKatQzn z?^OMC@gC(gn*fv>hZ)l#Hljn84J3gq>gFHfk9pY9?fa7Cqh0>qQu*{nbK!~}b^hhj z3B#@|eTDAx7BijdLKE#gXQu9iY#gjr%$;W;l}oRtS6ua+6)SP{;}_1m&eealw>z${ zCT@IV(701--$nB?3d%X-9oC&hw?D50CKoLB#spM3icGjI#F4MdDRXg=``>?kTF$0X zOn&YC{@&1{HrLIC%ij3|Y zBw_T+?7mc2ndF7o&mA?xl^I+DRNM}lxw-rjyFaOuQ^bcowz+GP&}%qcrz{hmwXl9w zE%OUmT%+xVt0IZx6Fqyn+zz$vfYZ<*MAEv+eg zm7RuhX;I9W=GBdlZMF@AT`V;m%=UuoiZ*Ri!h?C^{3tU_QH)?}p*@_5YBFCki7VSc zbl4l;J)>0T`NBq@8!e^XTcU%5-EUz&!D#KZgB_+g{-a>w;Dp?7`w| z8{(7<*KAsz`6mW=vnwY6<#bahmb1@0#R0DuE5G zk3GbRPm6HEa+Z56JzacRxI+2mOC!aMIwz-YX{l7lK@JHv>ZfPLIq9?xjX$qiSt!(Q z&#tlLpy)l0dU#l1cr?1{gs5{Bz**4+l|1S$vbyxbV^uXHLF9cfvfU1?&}>Y`p@F-kJ~R- zjx4IQgrhE4^cM%G5gJe^uxv|-{ox8`kwfoGOGR+p8zN(R8(!Icr0JF%9QQibrMka$ zeysP1(Ma!}H#`h2lztJftRylxuGNnh)k%u+w;6coV=dGB$O$&xSWo7&R9m>{euf!k zHWW+r1U0@=h&+QoN*e;U8$ccQ-8?237*S!pSl3+HpkT8W(iVBRo2qm_Xv+qq%;L0B-M>*yw1B`%l;v_YczWra~ zX2q$imG}+Wiws-SKW^lyk49#j^fyT12IYhZ(a09p>v@b~Ez=N?dfIW{RJ5@$>P;o< zrQkAXU(6~~2uHPR0t?3<95b02X0O13k~|nV><>NcHsmxMyApkZMbXS6<4mUtpUur- zs^_#mlpk(@=*s@CK8(>r?j`Bkr0aGz-VU!?`%dF8w`>p0n0u_c|Kz9K+IHy6$EA{8 z>as#!J=S(q+4nY!yk&T-C}i)h$OunG)=HoW45UGA)0i zS_nL0UMpD(YPY0)nC!IH2fghN%7h4EjTg?;7M4fenYO(^;$_I`Xv5aS!e~FtAYtD) zMcZxN5^l-=$-p2(Ung0`y2*_;VmdNiYRULCe%tgyvEswgxNM_>qSmmKLqiuo7X*+} zY`7409o3syySxx?-mG|4!mgb!(D{RJ^02`2@@1`Cb?+j_yYn6AIxRP3N^@_OTVFzL za3Ue{MUUmgQ2qe6tr)tF%lIv5;=t~1TedgA|1xob!{LU>=qJ+l&X2HQ#XTm*(I@cL z${j23yLoR@;VgH@D&En5rjHY<2=_ zE31+D%$y+<5hvTRDm&?x1d!C*RPLCK?bk8~>>|kNJ zl3N$qd3Vy*gZ-ma;fV-WR+@lzv@$oN(=jD2bhi|)DnsZ18amCFo$Rh_XKtnTYt09ol6rs5>F`&%-h z$YvpHy&JtjYIiD+K_hVAGFr^tte>5kI%s<H=#TcZime&;`j1?YP#s<=6lpVyWPbf!@i`#9N5_9lX$ z^f_5V3zz_}T9lrXARH=$WCIV|7+xIL`+Q}5sd=w){#f=}VfW*IP}|IRM8IaMNy=T^ zPRAi$*tLSTH4LQ1W+$=FGXy=`)fwMuz)_aQ)hQ&mU(;Y^(IvDIRCbU;UxUcWzOr+e zit1=fbHbUgcrTX++p~>2g3IFYGf?l6-vzY*XJxlbv!T}N{}h3Md!b{W_fZ~$sdk1l zVTANkW5cAcil6k{b#?VAJnjM_%j(NAYU6HX=>o$m5oY$19*WD93vczTK9{h2cNb`t zd(p0>amWzrDt}&`sT1N?o=#toO|MFgy*JI`%6QbTD^WPMMKW$;A$DP0D9(S{Qs=;1 zalUNBRYyZf18P)ygHpXWC#4nnCAbU<%(W|&W8RPp&;SNu@(9bv+p9fF)cye-s1JgG z;k*Q6qF`ub4pXhcWV3UV6H=8m5(cPG@<2KK_cT)bsfcjG_mx}2K@NQiHan0MmX4sR z_1x_Dd&dfq#=<)9-f(5J*3XO%tT!He@&(l!D4Cmf4+A55-$Ucu2%?<9sqw0G)}z}y zj4f28WED^~I5>c)vx7#1AcUf=xs`jTLK0=9CQ#}bU_x;6%(x>iGasU5X)YD4B~23?%0v)Mm$; zk`YVptPs%D?iPgT&$WXzj4umE3F$M#)TPPp@9>{16H%ewVv|i$!2f}q!XRzdwcC3JiNeB@ZZc4o4{(&S)m-mKN<^81VVVIF4Cwm z6a$ls`Hv^8%#Q~chd{U>_p8yp-blIH<<@O=P%YCvTcn9|SfSY;{_ zL1?GOr=4#LRfRa*v?UUA%KteMM(iHntnHM3eFfPCu~{>Vf&4_N9G9-BziqQgPOD*1(zU0vwImi#F{H z6kQ`m^V0?Edkn?*38e*&wuspLI;B#DkN&(?sN6blO2fD#3qyOvA*EK_{EsuEJhO2 zkl#IO-AD~_c=nGj7))j=$x~F|`h<3qWFGgZhcv>H;gJ&E1K7GlUee{AxAX$E1j%KB zGm5SNwDd(e39|E+o`IIAvy$OibUIhi63=py=+0ZZ2U^%i4 zs1R^`Lc9i1#55TOWM}z1vbzQ$$h->4K}0`iL3+cm>fgmzj0!4a_*no*;zkFxy3Eu# zZ5`!u@R32sLDfq>*V&ziM*gyZz-OKcoZmnMQdn=IF!$sfMH^IbdJkU!Es@#F!n5NV zWWe{hMqiyra1rfzCYO)^t45B`03L-%SpgG)6e^)lTm4m_g=%^so3mRA(3FM02f~5C z$=^cpet2|+x;oulz16N&j&)O9zaP~^6TiSdXPmn}J$FI*o+knz_{kt%uFtM5+quD) z!nhyx;Boc=IC~%`z*sNS*Y0@IwhI<|pF;}0Nx&qY^eUsFIPg zmex003%v%LlqPgOLQD{T;PTHl8UCgPps7884e>e7sBQq&H4;uCw26Utg&z(qMSa$L z7dVfdNkA6g%$9iP^BoWs-A9g!KpCjhxfLb@haGc@j2u8U`gVXP+>eRUuqq|!5&viQ zMX7z(C11?Z(8;)=+%9!}JfwMee;ERJzg7Ua zAkOV8n0b?^CDbx|vEE@M1d-2*Qih7=5WR{-g#|eKf9?rC0|6QL`C7iqPn4{EgTTh-fMq$WIwJ7xfkaR0zGmK#jwjnVfMKE-sKEb2^8dA^R!PMKI@wi$^D@ zag@T;65vxZDGPwl^kU2=#N^?J7q%K2j7}&9Fg+w`@uF}6`d_C|D%o^$LII$r0zn1X!vf_WB4n#pn;>qw|%@ zs87j7HAr4Uk#qHPIVz4z%U(uoD7X8e$1Pyt;!<-5s@79)V8e~Vjsb<>^xuoWJp85k z`3sW2Ao&ZDzgqBD3;t@sUoH5n1^@rl0%!=t<=eb~F0h18-qA!gP+Nf{F=0u8NK0ko zBU5pP(K>tIphRKvgmYK3?cre&WA~<-Wu@8dt{4$v8|7z4%`H!J<|r@mHnbl%&nO}( zOkSXC@a-ar51q>zGAtDK^_tBV=}bz!WIQVH$n06prIB0qOg73B($7v5GF{^u)&J0L z<9(cB*w8l*9A&z77cO^32N+guH*RFfa(tVSK~u2kRXfLao`?8uLIEuT%c%1KEqyzl z=OIsPpyObQmp^05^+!{VC|D_*eLn}!pS1!@u#K;FY`bd&J`qHT*`Y*`d{Va--+68Z znUO;hV8+TPWQmw9{+68t8>i;XE&RSUf=Oz%AuPBQY+Dpr==gcxpxJA736Ffi_4PD{ zxb{(9Wk)+hZmkxs40hhU_>u>6*{^CoDi>1h*&SIqr#fxAGuZtmvd?@K&M_)_+O*NN zG0V}S@{g7Z*H`QT8JkR+!m&d)55#?FKgt;^tlaSb z)uVTTh|~hktVmQtjI|ibt;py}ynZhLtzc9LqQVU&Fu&8@9~7mY0kooaB?OVh`Mp;D z(y}RDB=Obng(e*|g^TDq1iJW{HWgp-dd@xNH1v~qbXg_C#o)`2AB)Xrh&v_O-l}qg z5>ACOvX=dKarfp#Xt(Amrxe?*y(UtF_dRgzb7j;F#B|>e;zqW>zkZ}41*|Lba@;R% z;8sx6pu~uoc~#DYY~{B4-4@6zIf_BG=*e9kFetH6{XnyYbOJ|RkGJB|X5{1wlB$&V zzwErymtaW#^M&Hp8!luP+QJ*P%Oo)>o2k7!Qc)(80i?e{F*1j8!N!^z)vE>%k#Fv} zO;!eKd|v%8#J?8T77T>MB}% z#roK~OKPcWKRqI{BijQ^issN`0eBV5z-a<4{7A>;Bbtpe>NrA4f}C}V9q^LS#f`6> zHSWX;%X=qfa+KT85S~3BrgrfZ)<0$ezIP#9cYV%nV38IE<6sgFP~qSXi&48nS>0J* zRaHMFRlFG_u2RA6?zZps6Gk)5lNvY_m_^h@S%#Kt{Ix7ywG@T(;w{O%a)UAr&y2c9 zOd|$oW{+1ascl-~tpbjE4(Q^!>71(kFQF^lLaWPPYva?Bb75x+O z@@8FNy&f2?ytm{Oj9or4B}Jv>w_NAe%F4e4kjA+R9D3t>p|pF!hzrYzu^Z1ZUy!s& zso;~Riy6H9bW0f`3}nf5?4G@~#g@H|cY11rzSPvPe`)Pv*Xpe6L6S}q*yvz3Cm-?O z=8G#h{EL}cIWEK|)1w4EEoq|_(I%hTyGHMoSS~o?r>bwTuBMq~VRq^wmfgSec}VU4 z%N-~$*p$AwYq=$*h^nh|RnJOudWknDn-s~{ND>SQd|4?%WOnMWK^Ix>9 z8p~94%!enI=!ve=2bFfz5CtUbpzG}~LamrjM@W9h2vAq7YdUWMXQnLuoq;arJJw4| zvHD6L2(WB_F{N^iKe7+Ww?^>*pcLW?Bf>jJF?7o9Hy@I>nr(ePaC2a>5`?B#d{I|{v@X=tbw2u+3dhg>xRVr zEg+V5tz_w00QvZ)$~VDE>WXtI;CHkRy;5cN&{lPaTdVCGgun>I+>XF1Df#~;eVbkV zg0SzjIFaOu%A6nNZ*0fHilCo-Jy=S$WFh#q+j8;xnXmCIf;_x@U+pIldeKJ%rY9BW{c3P7>tu=po!WK~(N__&VcDYawQ6Ko-kkDmBJuiTDv!@t* zmm)G^(Pkk;0tDAQ;>xM$_PUj!fQ*8`O__^ra~J1U{${H;k+p>fHhn&8ScoP*x8_+g z2xkpDD-OeTeG66j6&+YOeJ6p~8#Ct+dsTk@%FhGNgyr7NPsp(TmR!Cu#>WrX8HvW` z9haOT1K=-Ewn!TM1hFsuII1o*}{JO9D)D80TeT( zD4a`iD~)JwXsyBJj%*4m42RDmv2ZT#)|5ibT%le#28_>n8{9aN1J0kt3%?^G4yLJ1 zr=C*6#}Dm*t;mGFa`>+UXf2+Yq`{bhF*$nYxkV}N9`p;D%i-J9ML->V+6t)qs*+Q9 zB3u`}wF77qR+yxy;qehf)~f=XOnf%SZS>pjilL;Eu6Wh(F>(0z&CPD%1N!dHS%%Mc z*D}^EHg_3P5s>z=AE{&K{ zv&4oa3MD$0*uqo`K6Rclhvz!C?+s#z!IElw`t3HEDvg^^Fd1K-Por9{6Y?q~egu}S z@#9M;h%X)L>fk3*%U^a_)Ew4x<@<>G{A4(r|HWP}$>Q8w(}P>&$A&oa3p)|;P9U+$ zUJ$dgoJl;keklpDP5QgGewIt|Md}ElM}s~+)+dqkg*9MRgi{EU7(O|qCxs;yC+h~+ z$<@>rK)Og7y>t=OXy02?;i*+=eplqY7WIB;<=HAJ_y+TPkdGKsL`&;M{VZP=I~z5} z2i#zpvyaNxQGdD>j2hyyhaPZxaAe-;DIIT9u>7W4XR3qw>>7I$JW2W4 z4P%Ps=E1d*gI^?%I8fDgdXfg7FKDJG*tl;|*JMoP zkZjH*wA1~AFvV?R_^hQoGM9biJ1vHbZ<80>-_bY;5;MN9`2Q#eLGVHW@LSC|ACZjO zzEczky$nvGq;RV)-HPjIHwd)>r5dCESJTv0x8SS>(+3N3=jpK)SHf^Wr(Qxw925&?1o4>V8AZ~`I!sIvJjEbV9R}U z7*T>Gm0NNx_vruDI3o=5s_liPHO@_&9JL>y!|=&ta=1G!mWpDmzOy<$%xXDv@l}D{ z7->pRIeI7_3IR)HOUnJ7lQnA-R1WqQgR-BR+X-)RQsC_6`igr=+ALXnZBvDGvw#Q^ z%j2xRp`pYN8)+SKvCp*t%?erQ2I)g8?u{-_$x=5ybr&wKIwu|5x->*YNDZMr1XVk3 zPKO97qs~Jo#y7}7Oj1Ox>-;lc{B)~%` z*;51?m+D(m9myQ7=P9*@pjk9R%>uPrr;E7oXwVng2cLyuY!zP??)P7a0Rc_c&heb9 zue!*DKn?scoQO?6gW;<~$#6UeeS31hk99yz6z8b1e}|sP zD-@_BgdYt){*GW!u-KMVj0n&Ia2W%tVWCY`a|XKV_7uV&bhwW8yY)+0a-PSQ53znp z%xyYx9w0RN&thf(!ta+a^}eXYjw+(NbSVU8TeE&n8}N_@hd}o-AoL&0eE1!DQto1_ z_-PI%G*1zN(tWmj+>Nhs7}WGlIU!}9ghpY0k%_b+Vzd<2~L zn%v#t_3aMyB~Ax3X<2&8CAbyX{OOiBfDwpfVt*!3n~vdsNFP#HzaR`*3!ri!mQM+( zda#3ce(njn3=5>Va!&mGXa{#RJ_0kF_*j7!o59A97isk;&syQm5u2q^8E zr~JK^&tnui=}E(nj{I4f(7ID{59rFShe5~;*~Zdt{Rrw0CId1dE=s?|;I<>h1gyfO zR{JwmxdtYe;~8*LDXfU{Z7#@>E`IG6r+8lh$d zM=JB~>iLZgAV|WZrU8c4|9LdY)+&JX%k60VzXcXQ%+$i)sj+FMhSk_OF19bFId*^1 zdY2YgwAeTFHEp2~G0pJ_0zs47Of&ohukD!fJ#ah}KgaZ5Fpapouy*Otyilx1Yh1yE zA)M<_4=kI1<`NKRLT;-Ls8sTAWy=<3=5J-o_c(;jj{XAWFHq33Tz`Rr22egvVO*l4 z+;EvdGd|Jq#HgiBK2+}wpU>kETnFEf93tPn2LEmnH8Irloh*Iq3q;;#TX8|XU<;ow zp?ZkD%u(hbWJ*2~d>=lo^(K81@o9OH%MRV6*YpqH5xnhr&mjz9K1#^Je6R&)s2tvA zJZ!;IAmCaPdR6m~aW2eYRRTXFEr{T zc{u3RvlL@>pb}~4BZM5o@3w#R=EuPjuzP5y*X*m+-%RDl!L|f^nV!x{$Vo$U=0!pH zLz?#md-5i-bc;c-xT5m{p3o{G!tPP8pj`e)^(K;YRWK~q@>rM>H)$=48Xib%y%{q( z=6`Vg)fN_Kvq?B#2r48&WyGiJV$Q>E$(nxWV+5DF+f7ybT zUi?MNmiqp`VvFAifiX?EAfOVkm6$K@3-^5io2g*CpPs1a5gG;Ui?*9ANAeF_QuLoO zfJA^O?@ieadQkGUpRdL6plzImGpr)Y?M?9||3PD?6om1(23r-=#MArFN}a&**XH5- zI)z+}LjQf610B(Ae2BrOr9S zos>POUho$*eADkl;)hhcKQ28c%?k@4k|kiGiH%W+*PMn0ZxM18t0;d%%W7<7<$e@X za1a}UXWMzbXQv2+oCG7zJ<|DZd9Vo_)u0ra6%K#Z6Ys6#q1m}nDi^B26iy}A8nhZG z(9MDHKR8@>{2lvub!sDrz(P@ZEAuy30KJS|m5x855=n$*XoknRStEzqO;XoWz!o2M z9{031kw+ zm=#LFz6wN9HIE*)k>n3G!16k3Bbunqttduizj9jA8nG$yUdah)R}3E)dH~$y*$*^S zf^B@KxCpWTtZr=?sEGYO%Riz8`nPThj$@Ev^TC zXiWLlpUs?3!p=+T5ZSEZ+Cqp(8vVW5KnQvWSlW4}jz81F$Dp!Y+~_18S|v>=$(#jA zt`(9^o4pnP5p<)v_-j_Skon(o%&pwe|EY4!zh>oUb^mKtw(ubTt7rvUxt{#3$Q#t- zt{k8WrH~&yLglKU7DhRL5l8sIe#h>7WY>qv;{N=bVUYNI&b}|;U!{Y};U2y;)tt6h z(t+aqD!By(kL&^4cH5udLgx-H?kr)#rPJvAberG&Eo#}y9R4k8*+ScPWBdS?yi-z6 zPxYCG5$kCMV8!r!t)b7iY~-f^OjnIMLfT|a<@19&$90>@k2y1% z8{pWkN3SPR?bhDW3hO8oSs=Da`Q(5Bbj`E~{1;IAKuc=P+nvv`Kgl3LE9-NXhMQ=ApYb(xJBy?Mu?-p%cet$4HPV=BGh)EFv;H7_w%^-RVPy`|oI zCy)UUf6?v-mjyK!$We5hN>TrhbJTqBJ(aC<$w=JX#%f@du@KaqddDt`T!vo;J1AF8 zJrTa3Oil{+pK0D|gqDp;!MolTA-gd)%Dci#%H^6@s2n5M*>C#=rkvmRDUd>xIN%kosHFulHlmIZl}8o4nr7(vhfe_cihR2F_uhE1?+LTs8qc&c~~?r9aba z-*$&YL)y=>4=!y;=w5w%tJd^X@ReJS>!#%Rz=3c4)hl6&Rik!;|JY)56<+BQ5H-M| zzPAhG+aDkG-9_euEkVtPAiH*B5kPi9?-OwA_f(FI`JUK!gApI&-Ht0Kp2NH0ZIGGoPR`0RP)ZJ=2styNrMU~4oK3>Ex{_K2LlEf_xB>?9 z{U{uFzlmDndcGGn0$4{;)L75C{h9(A1x2YdscZK<_%=E?5vc!-vcfK)eqqV-*i!QQ zfp~cA0-N{lJ|{)$W5DU)HKCBZZZ@`k75B*wqTkCdXb<7!xWM%Ix#{ssH!l3g`hSUo z`V!%It0>GmAw(pFgVB{br7E!2JJPJWkPxUaA*snu1V<%*(LaZL=IWB1FU05g`@r@AI zGhN2cDm7y`v7v=bIsC<~c?+F79Tgz2ZF8TO`>afj_j|cGF>bZ+>n53aw{`69v+BdS z3Rr;$0g%}phUJ17i$~zS1u7lMh&u)oksru0*N!8eAVC?Od-sZW3DxvWTsVz1boBA} zLl3M87*6Me10rkTy0bk8e(@9r;dMd(-yRwQFM!GzdXVsyT^c_6i$0N<{!MX^kO+Tw zk`(Ms|I4y1II}g<0pI?zY$x3MmyLf}_LpT_8s!*z0?LnFNTy6f7Q3NLZehY9^Rmbk z{44-5!y$+30u32g3?+)AES?x?_c?wSA^c#vfB2VG5IeIWK}=F=uj{RZCEwf;f86p! zqN`r=$7+-=_Rc)gXx?-Z&eUK_5c~_Rd!b|VSVz*E^ot9) zatuC7feb>;7^ z%qMl*kNohm1PDSZ6dQ@)%mW^_>rs@t^)PQ1&05b3uLnDDwNbCI3v7J~Y<&xCAAf=m z7S@}_O)Rk1=)qoyh#g|+H9Vxdt>T@jhh9Wr^9rM@i5g#Vte{d85ho|QPI4mq(u!gb zE3O`C#6%2a1S4lWPeGu5=Ismb2fGnHBf((ndnKO^O^#og9vA;UNL<|qKKfC7MgZ|r z480F$3cz2jGzDh(rj!uRQFK)D>G0&Z`1H8+56uIF)ugf{BN~cfzOyl2-Xw3?zEkpv z&}=^~dgY5W6(;7d(6fmb=u3tY0a2}m4mNfK@1buFfkFlD!TY|5ky{_KdZr_z))p&` z_|;|`sIE$gK2y@9U?ROS%>oj(8)dTy1@K5H8JOJ{(J1Q_&KQdwgiF5U1s<99qup|n|cCWT3E!?~yK=~Uoks1u)9z6hFD4%jWHVci08;iq+*cTxY3Y z+&eRXSDY7De8KT^n&Q9%2Xi{~=f-ee(J@1H&DSom^;j-?w&`ZY$7BuD7xVhV+`QH$ zYNf)}o3ZDq=0m*`JKu&h2QtGRNihQHx%Z2m`3BQ@oQ}@r_|zrM)%D_(LY3Q>$eIqi zvF%i{QA1d>txAbcy3?|*9f=KNeL+R2jYG|P3(KRSoLfo#E7!4dZf{ZRAa#l=0y>i2 zao6`$aqq-dp`r&mMF%FX{1N5@9@{Y=O_X0c0U|YB31ZAk2t*+n$erE$OBo!b=(F&D z%`*b#5MMSaRM5ZPu2Lp0?hFvtb<1PfqyXwUp;1H#Hnq}g_rmVd|qTT1+EJ@!|IH#B~>b2kPZSDDI z@ej*sB~3-UNv|!lYQN`?nd6VG;r}O*Um-)AIsaxiXYzZ_n5LQOg|*u)x#irQN1C&< z1?5CcmC|okp4I8`ch~Ni@V!fQqE|qy*4vVqGT^R_Bjtzqrm?Xx2CC}DSLWJDDg1=; zgJk_jPuNqPRLk)Hu=lj~0e}6IG;4yiEw%F21#@%yZ#$zbLPPy318c0wWokn_-vv@= z9*;g)WhddUza~DtAeJkCtAhWQC@p)RO(bJWy+q8%t8KayHm#@juv7{LXB5TJhOdk3 zJnd)UIa1GhL@9IVK39OhOZTwepsE)eZT0no9G4}7OgTG4-OuQ`noZpgOC9W=+9x_Z z*gv=C|9-TEFFUxq%R+fZzx3#NX(~aNu6^OUbJLdOy#QXO$%Wik)!c5hI^FitpJ%JjaqfX%$CqGODF# ze!1_%sa_|04nwWwWP6B%PjrVHbMnT$j&h}iW$rnD{JA}yg|xC3q55Jgv)%UA3|Xxy<`rk1G_gk4Lw^K5Nb2$**#Z`7T&9LSIeJDw@(~ zH4M7-^7NcxXt#JPJ)OPmfJ@#*cd-4YgZIt$%Yq}3MBa4~QCI7H^^4A{F!W9El#U6j z=@15T7T@)3&ovxvG*nepRfAZ(%{Hx5<+ShV6Y=zjQ5%>U*EH)mks5C6Z!$4-Wtan; z>*`2(S@{7axhI#bUb&r4{Tl&3)!5<>ADTb)Dd=eYGvS=-E?cJWz2DXBfl9mi^s@a7 zgtmBanpeDXbnO7o27`LXwCwqmURBId+cDvzgJsZ_1;HT5V)jktUcLpvL1Xzn{m7HI z9a|%U-N_yCqI|;IWvpEzE-QxaTILgY&gOnjb6%~fte+3yh6h~vNOLSX=&kH)FoqkEFsl4HrH~lhI&z*v*^}MKvd@W}=8He9R8{J@2_gaf!*BlA>FtV(k+u6Qh z{wZ2&tU}L^zSsKk$K~hq6vo=xHOG9}#;{Xv3PMkvP?TY@I_h;ve`G?GVIe&$naOg& zkbS5(VI=Q`Q$>+Ln11Q#Rj-9TYRrZK)uq)ZkJ{Vo=`0kRTVx9!F=Nb4BK2f;%y{Hf zM0aB4iHE^?=5_}G#>60A-b}Nyuh#0`%}uIHeOd>(3_D)EqzzibH_+GQchHEJS>8Xq zal$0nA1m+L9?pgh>x*sHk=JTh-e{U$O3Ev`5Ib9(mBS_2RNPh{`46rAz2<0-RP0o3 zL$TtP3qD$=Cz*1yN8eTB-r!90Dv&5mpD^HAHOTd1ac4GsMVFg5m1948 zWhnQeT8H3ZU4>qTfjzd18QU^viPj$kBhV?@&&i8)iBL7>bM@UV)S$8 zh**8S#p0joXzegBGI>|A)xVydP-Erg#R|29_Xl1u;K~KKeRx;7 zf;;-JGB2Lbx)AE^#3E^`Vt+p?W;ZcOOH%~>;KYKLMVqOu;jBfUvmA?mTAP;1ntkDM z%1HjGl>z}(6&ZHn^Myj!d?$O^QtSG7ad_{1CamB;&oD?jty@|K$BJ5n(MurCh#kyRSw6wFYToX~#fn2sV417|J>zQ0wG{?Si+ zcxKR_O|qJ8WcuLVwamL8+4IWBMuv0)*Wi|QXMPx>iOcw?g+h(6A@StR#gqb1u~BEj zo2%w$m6UzOvJGk#gw%ZW%aW`&jxAU_GJkp#e1Xbs_U^;Jo@No3{8so``6elXI;kOk zaxiO-GncPLI9$9Y02M3;gls34@SV(Gh!s@i<*QCdq4En?hCd2&(4F4}#m6{SZYNbV*zx7~# zGIq`SgoNeb^v^4o+Fp29?QNkhHanqSSrDwh6o9!@Y)|!Mx*hlV#kjs8DWkgqx7)JT zmTKolTPzojPdWZpoKixEXSHxeU$~Qb0YVi#p=q_C8TzPdah&43Fzdxm|2a9ha&;d1v__+^ft-N2a>&_B2>GiZkS3 z-;OCL;Afk7(d&53G!ZP}E0{kkUZN+=rBzb3HA+wJUsuU$&k5HRR}DY2%v(v}d5c8< zX;R~>yo9_jZ@jY0MpwLyMZ9~bq?%6a`|^CmA70cvdUDw9!5fKJiEQ$g@;bH|K_BU> z4Dky|dDHa)xV37Z8+FWGAU9+g?|BEjfl4^WUr-o|Fd6 zswF*N*O73usghpqE_A$ilTVE26r*}UooQx9p1}w-G%;bcnCo$Miw>XK2qHnd1OAqL zBE3%F9mGwOcVXPJ)|S95^|_O)S11|1>dOV|lyxt879qv^QAcQ=a6Ua(U)Z}KFXG`W zs!(Mbe8VW`x*4tYz4Th88B&a}9E7AMcxK&hs=YRNOlX0XVf z$3{x7anf4bOlv}5ok!2~MNI`+U%GU6YElx<%@>AW1(?a{*Q#`}&MnNdD6Dyie9DbU z$TM0mvcEK|kyJoxGxJc6D-+MY>p|3tL~65e0QU&3=hJxI}yn>J&qN9v!i>-5z1 z3l1+}KXKFdu)cm@zm7b1BijFN|EpZWgiu=V?l8kN35?LVN({b1^GuFTvM6Ky02&Ss z*W1L)S$(XuiVO?!8vfST{`q)%W;yY_uDi6Jdr~vE_z}~BhFZJWnklwszdoOP`(6#l zrr#f&cx)&xG}Ks=Mip{r(e{Q4Z^aF~1%Hu87xNokXhQ7OP?_IrkeYCP+pe?~_ZH13e{0Lif z^;2r{Q~`yuSyjcM)05IBo(6%Vs|Q6>9RH~!YI`wuIXAWHy!2^>(9>*j@m_2h9`gMS zLzO0l@Aq}QlJ$>1O2EvpRK)M0W2(}fceQ7KKUpRbXHL&XE zSn)6LuD&~fo68(Am9p3RK_nByWh(j!Z`ukDn~1SOxvoer-xjFTmj`N2tsZy2Hq+qq zTGTPIU)TlpB^gpZUN}2A4W0G1GiNfS`Ufwu`(46limtK#diHGZKq5=q?6UCBK15iI*C(5#?`w+L<&B|S#_5>?+cO)wz@Z?DLm>Z4dUa}T zS9+Jedho4E{td=ln|Gb*>TD*jv#%# zkK;~8JWkv-p<&-=*G*>4l9XVym{y>Ub?`oUM)Q+d$4@C!m)uDLE}arrKiTq$xMBTn z@M=GAL~#83_Eb+53WGJQB$hUZEbIHQG$8AXdkfg65i;4Lg12rwkgxbs+r<(m>c}py zx)DK|ZIpmN-OuaV*?9POG9w-`yK~ifO$lU!y)r}BN7~*#ay8^1ZAtyaY@XsPB=oSk ziG!|p^)*_nhgdfSUZIe|@9*^rHXkwrZQm=5%`X=QRd?#Nw$OZ3bql3m-D^ z3Xe(;(BU4V?B%X?OINy?jJ0^mRya;3;fxG}?SgRc{;sO<$j+6H1>+dXv-IZCG}k|+ zx_KVqny-kQOE)dVY8lbSuG;W_N%<+Y;T|m^Ybe*3dy_2MmSzxXc}Z{F00*@cznJ8` z|IBbH4i2vxr1%yYW`nr+WF3nDEoHIEjTc`@8wPXjtJwt(xHc_a3p6Loq*q%w=hn(p z(Ug+Yy)Q>JedWBO+k#E8&4U0MKZZ#EwaDv-Q#a(@PP9n*5EzSCyKBqgH>({RpRpCc zdM}%o+ZL=CEJ>dyD z>fO;4ghy7cr)QleKZ(Pup?j*eZ8b4dBJz}G%-|mX{mZg9uDb7K!g-aEkl^q<=6>=# zo$o6W#gakH3;UN~p;tvgtq;d>_3ZtU zC*s$&ly~OCteEiAB`%7@P{*#Ww#E3M&T!*EZK;h*Wu3Q`%}wr#*Bl*tp`m$B>oLnK zK8c!BIJ{C%W0GH1&OYltW%@EDiXIB0tyuF&%1@W;2Ll2)T)8h>@8;|lFPmK}J@VqIbNC-DZ|4q8PgyI$9h6+KuMx*nw4 z=0EO`Tg5)2p1#4(qfYhkU0~~joE0i1R@+Bog_FDDX7zgmkbHTKi4$OQsB1^y=Jk{S(3TwIFQ($lIwDx*&|AVhV z)7VS;)B5z00p$wSq+uW%6thBpU=rfcI?w~wH{R_|xQ6{Z{+OU5t0cv&N`R@gdDi{Q z*@En>exv^9F(#5(9oNQ+uUJ*?t8NWSuderCKGU6*%s5Oze_=_}R$sb;Rv6HIHvsNCCgEKu^AqQ?xTM!M>qv#}id&5L`|$3;%6jq4h&O5!dkUC24c{)POu zrHdiAiM(TTSF@Re!ttquBTioMCthx2*7icDa0XgdaNIuHNL%x(GS z^<3z?E~U4d%X|zEPp-R_#O`#Tf+enqjs;6Uv`aLXFpB?pC^HF@lI*GSrIaW1gZio_ z<})8#GehH3EDMKaNBAqH3ZBH*cCDQ{Drh*-dwezjMb|zM_H`XMe!BZj!+>|b`e^** z{i;$PS_-%4`{Vtw&IedKaF5+dy5@8)7Y{@*P)(e-fz3>h50*F{XmGOA70^+cBhKLC zNG(jBTC2=1bMM-GdB&pD-nHtgayC$K1Jkib+9?oZwSh$F?%J~(q7vu zWYn-s`JJ>pdva#h=>=e9dzxtr>lGy!3)gx16b#Mt3pIk_12LF1HjgGhvlw5$=0kk;g63fI zg<%E0gl3Ddj?y(z$u|7ckfD}rr!&Q;O;-}t!@F!7v$FRvuC;wiegA@4v1Dj)WsGXz zfa1F;(*`GF<}2edj@c^#O-rS!s^)3Y7G`IKXrYyo2jrVABR`B>BldP(9?uX^JDeKH za2iTvKl;y%-hzJo#&j~f$D8?&Ok>%ju{ldV5#i!oUecz9bK3v3s<)5aR8EmGYM5{1 zY_pSU%Nk%$ALHAYa~~~a1q+>>(`vnnD@G=Q%v16`bIJbNUq%xlp&J#`tO*N;ChZ%? zWV~Jf#T+Kqn^@n}$_0;MOpA$yCc1^t^qxWMuArJVJbMePVxL}%;?)C7Z%i{Jv$~Ei z$+DY2u|JY2?NCQQpNuD0c43Bj1< z6ALPVH#1$%t$9)DDNa09QtITJv|^#APb?Q?+SpL$2^y^xux!nQn&Yime<&o?D<|g; z@=IrKNyMEH@a9unD(_F1n!I2eIb&%77LYS$p2F6Z>KW2N4#8^ahW1((>C$-<`}C}( z^yu_t#f*Vi(j22lr?tkjvyK(>PuESqD<_x8%DTYwJ24maX9ns-@f7hStNNVU`hAYZ zd`)y~r%5lM5%8je*tB~nK zk*1!CedG&Pl~?_*4Be4PPaejc9*kK!@acf%ffWjMrwQjaZuf=I@_Q6z90~_N1!Tm5 z&~d`TvsQV9=ZG$p^CDR$L=wLg19B%na9!6x-WOB)heyr$0T}$jBNc^Xts&$(!fQ%1r_xp1~+z z`!q=T%j4RqwRJVmu6boYKhnQ+V#%&ho3ALki6wjCO_kM{^fZQGmvKtrx$%pQ6RRWm zIR@_eCyu2i2D!A?TJZ}Gvb>*qSP^yzqG5;o_C~dr4~xn4RjO&?W*t-2xp%NQc~KVk z|KaSd1DgKVzHtMCvXGK8K}rOrOHc%9MLGr|B{8~TA__`(j*@1K(G61+kZwk95~Bu8 zHb#uGJ@ef6IluGVzjJ?&&-rig+5Y&vAbYW_r>MlzlAuB=*0CA^l3 zDPq06D`qwfWB;E;5`P2fEq|PvAq~aF)^XldDzFpMUp8>bGC!exEhzvh2;C~%CG5$3 zD_>|viN@gJS4gBtw>j0U`3;etWjf%X7-7z(LW0A6_VbJ1bjLGgGhk}6Qcy$Eh-0AO za>a0R-W>nHgssyHmUkH}0)DAU0ff!W_eM6qK&cVPoa2}shSzYRx%5>nM@LsZUY;J? zt1cz06i)H4`Vk_LD0XV^C$kLB5Dpz724>G&mpvRZxJ91o-dAV_iTIf#5H zR#yG%dqQmQRQr~!!auXf|DKilJGv5*cm9V!O7&+HAk5u9&YClMrXmS6r()m_W-EZU zd8LexbMm?kGwmtx>;jFqM5pT)kxG5;jStf8o1SIh$;&}w@=X^*wKK|)cN)fya;_Ij z{IPAM*CXZU`K?g>R=TCEVuJz8u;X`GkJ*=;8^qxfNlVr8U2D7A7)K?3zD#cf z6$oX5l%^umvKM{8;H3>eRS_xA!OQey2d&Tizy%;&^lG5-<#tx}#lR{M+tbpd%yygG zoAb4qp2`=jhJ~tX{J+{wOH5E%^AJ(K>u{&b@gm|pRa*UC1I+a@kNG8gkDIrow$d7* z7tsIFjsBubb6e9B&Y9Yve|fATAveQ|-q>-s^k>PB^`LB9Uwd*|UhZ9=zKtx94zH=i zs{b#Uf;QbkEH5M*xoUc&`-2$b#PM`@){+$?T$)LMc38~-cE&+x@q)lGUE`I?AmH5s z$|Yz8+@WWbI8(E@!+x^sflJxd*?g|Q60>??X4kGu9phyLhw`hsH%eIg&%F*za*)S z;yUHh>3@`g{lQWBk$GbOE;>3T?DAc;lhBE!Lr$Z+Sx5Iv+zFY^))F&vmJq#{Xc@&2 zP2*%jo5Z<%FAG?F$T85(N8XA)nY~aaeu+*&6)kM3&kU0lHqhHQSHsX)=0)# zAoKWRv4Lk5X{QAQMWiZgJoGKh#0ZYl*xv-^M|*-qPiebp1xD{R*y4x#XCLsu^r&~& zqPTOk&i675;@KsjTbFt$vs#>P*KI)!87%FGamjpf!cZT6)=1+(cC+&q8wxDRG zQXC2e4Vi^sp{6t=ILsiGKDV~}sX$@1t-{8L?i|Nhf zmxSLy1orpAw0hpQt{EyMRo~G_(M(=C^`v2UH=aRaZ?N+U4Pm2kMiUFo45vVTt4<6H zfvIx?qBEkZ4ojScwbH$8K}if-ZzY>Oi9&-?i|R*V3(!Slb_n3=r!-26+`%~|W_l2ED$Llj&@zcckJK*Y0FW@=+?#iwrU_-LQV z(T4a+Ug9UoQ?;=a&gg^lf&u~^V|D|jNw3L%xU+#Vtp)y|lS|R5DRLV@H^r0adQPnV zh>;HP?$KLSW)kJ4Qk`%iLSVjV5i}oq7;mO^ySa7&az(-Um!r6nXNZ-1IKgo#m{sh)8U|?4mGzvUQ5u6h6!S93{|g znkgxH`D=EZ*4wC!5;yO9$z#!LKqKCTiOF`sq?K2XVCO8oURwB!h!sB+hJ<(ce;94(J=Veb&ChF~qq*n`Kt zcI+PHLhuuXfKx&r+9~B|Bdj&V3|T9ZT2GcVng12p+ne!t50-w+jYykae*84}#@E%@ z7mT<_rm<>Z(3u$O$%MuDN|8!v6W8s368i(BF$`N!C+RsOw(P--mU7o zTabfnC&=&YChB6y?>e9wF!*@ZD==47AM^58wh z?F_OiJE0Q5F^ZzG>qx>MZ!A}rmz8Z_f3};mg(zxOl5bew33u6pSPKdk;a_KJxTRz^ zOm%HFuPB*j%_U(Jr)Nj_rZKvwPttUSK2nn{^Tw?&S(Wu){&UoiT2s{`+0&x>w*sNN zFGD`ivuejhg*aE0nsZ3HZ1RswGuW6tE%dpFmV#R=&s-&J-0tEc_U6(r?VP?tL$o%2 zxHWX;O{)<(d)-c7)5dY;T8>)1couKSaO|@^^kHj^IpzxO&^>(tPoLW6iXE`;9Jg*i z=53eGlKX%OD}!g&@W=LxGEdtif(IYD*!DJT&q=X(30gdQP_7J5)fd(oSbehGr|xc z*MSSJbYBuhCoM2nH$Q(hmmi~EbaEz_j(XlMht%+9mm|5Q-J|-$9R3)Zc>CzadEfop ze*3&tzX9|QLO->$<$7cko_#gO=qi00J*hJA^G9>iIiM%y8_6s4Ta=1UdouPCh